2023-10-03T09:54:35.779 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-03T09:54:35.780 DEBUG:teuthology.run:Teuthology command: teuthology --name dparmar-2023-10-03_09:48:43-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi --description fs:upgrade/upgraded_client/{bluestore-bitmap branch/pacific centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-install 1-mount/mount/fuse 2-upgrade 3-workload/stress_tests/{0-client-upgrade 1-tests/dbench}}} --owner scheduled_dparmar@teuthology --archive /home/teuthworker/archive/dparmar-2023-10-03_09:48:43-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7409565 --verbose -- /home/teuthworker/archive/dparmar-2023-10-03_09:48:43-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7409565/orig.config.yaml 2023-10-03T09:54:35.826 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-03T09:54:35.916 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/dparmar-2023-10-03_09:48:43-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7409565 branch: fix-nfs-apply-err-reporting description: fs:upgrade/upgraded_client/{bluestore-bitmap branch/pacific centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-install 1-mount/mount/fuse 2-upgrade 3-workload/stress_tests/{0-client-upgrade 1-tests/dbench}}} email: null first_in_suite: false job_id: '7409565' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: dparmar-2023-10-03_09:48:43-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 8.stream overrides: admin_socket: branch: fix-nfs-apply-err-reporting ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: bluestore warn on legacy statfs: false bluestore warn on no per pool omap: false mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon warn on osd down out interval zero: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - scrub mismatch - ScrubResult - wrongly marked - \(POOL_APP_NOT_ENABLED\) - \(SLOW_OPS\) - overall HEALTH_ - \(MON_MSGR2_NOT_ENABLED\) - slow request sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62953 sha1: a6297898cc696a7db13e71aaa0b24c78a4c800c5 owner: scheduled_dparmar@teuthology priority: 55 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mgr.x - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - - client.0 seed: 6051 sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 sleep_before_teardown: 0 suite: fs:upgrade suite_branch: wip-62953 suite_path: /home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa suite_relpath: qa suite_repo: https://github.com/dparmar18/ceph suite_sha1: a6297898cc696a7db13e71aaa0b24c78a4c800c5 targets: smithi003.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCtr4kcGEi5jcuobzEHMX+y83QO1weeTgYpAhL0wlGpu0Q/PbpE4hZOfSYBQz0oZE+5f2EomyGa26hSmhuzfin8= smithi022.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKCqvRZ4EVfy757SyycWQMn6oUIteEVFUHylwlmm7Wv6zfVG6TRFcNBVK79xTzP0dUavmY0OoCHdZL15rMPBmSk= tasks: - install: branch: pacific exclude_packages: - cephadm - ceph-mgr-cephadm - ceph-immutable-object-cache - python3-rados - python3-rgw - python3-rbd - python3-cephfs - ceph-volume extra_packages: - python-rados - python-rgw - python-rbd - python-cephfs extra_system_packages: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - print: '**** done installing pacific' - ceph: conf: global: mon warn on pool no app: false ms bind msgr2: false log-ignorelist: - overall HEALTH_ - \(FS_ - \(MDS_ - \(OSD_ - \(MON_DOWN\) - \(CACHE_POOL_ - \(POOL_ - \(MGR_DOWN\) - \(PG_ - \(SMALLER_PGP_NUM\) - Monitor daemon marked osd - Behind on trimming - Manager daemon - exec: osd.0: - ceph osd set-require-min-compat-client pacific - print: '**** done ceph' - ceph-fuse: null - mds_pre_upgrade: null - print: '**** done mds pre-upgrade sequence' - install.upgrade: mon.a: branch: reef - print: '**** done install.upgrade the host' - ceph.restart: daemons: - mon.* - mgr.* mon-health-to-clog: false wait-for-healthy: false - ceph.healthy: null - ceph.restart: daemons: - osd.* wait-for-healthy: false wait-for-osds-up: true - ceph.stop: - mds.* - ceph.restart: daemons: - mds.* wait-for-healthy: false wait-for-osds-up: true - exec: mon.a: - ceph osd dump -f json-pretty - ceph versions - for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done - ceph osd require-osd-release reef - ceph.healthy: null - print: '**** done ceph.restart' - ceph-fuse: client.0: mounted: false - print: '**** done unmount client.0' - install.upgrade: client.0: null - print: '**** done install.upgrade on client.0' - ceph-fuse: client.0: null - print: '**** done remount client.0' - workunit: clients: all: - suites/dbench.sh teuthology: branch: pacific fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end - if not is_fuse() then reject() end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-03_09:48:43 tube: smithi user: dparmar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-03T09:54:35.916 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa; will attempt to use it 2023-10-03T09:54:35.917 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks 2023-10-03T09:54:35.917 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-03T09:54:35.918 INFO:teuthology.task.internal:Checking packages... 2023-10-03T09:54:35.939 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3d6eb91756d99e822eeed7c6b783104a6ad86054' 2023-10-03T09:54:35.940 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-03T09:54:35.940 INFO:teuthology.packaging:ref: None 2023-10-03T09:54:35.940 INFO:teuthology.packaging:tag: None 2023-10-03T09:54:35.940 INFO:teuthology.packaging:branch: fix-nfs-apply-err-reporting 2023-10-03T09:54:35.940 INFO:teuthology.packaging:sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 2023-10-03T09:54:35.940 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=fix-nfs-apply-err-reporting 2023-10-03T09:54:36.108 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-5795.g3d6eb917 2023-10-03T09:54:36.108 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-03T09:54:36.115 INFO:teuthology.task.internal:no buildpackages task found 2023-10-03T09:54:36.115 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-03T09:54:36.135 INFO:teuthology.task.internal:Saving configuration 2023-10-03T09:54:36.151 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-03T09:54:36.156 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-03T09:54:36.181 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi003.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2023-10-03_09:48:43-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7409565', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-03 09:49:45.253093', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCtr4kcGEi5jcuobzEHMX+y83QO1weeTgYpAhL0wlGpu0Q/PbpE4hZOfSYBQz0oZE+5f2EomyGa26hSmhuzfin8='} 2023-10-03T09:54:36.203 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi022.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2023-10-03_09:48:43-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7409565', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-03 09:49:45.251149', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKCqvRZ4EVfy757SyycWQMn6oUIteEVFUHylwlmm7Wv6zfVG6TRFcNBVK79xTzP0dUavmY0OoCHdZL15rMPBmSk='} 2023-10-03T09:54:36.203 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-03T09:54:36.209 INFO:teuthology.task.internal:roles: ubuntu@smithi003.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2023-10-03T09:54:36.209 INFO:teuthology.task.internal:roles: ubuntu@smithi022.front.sepia.ceph.com - ['client.0'] 2023-10-03T09:54:36.210 INFO:teuthology.run_tasks:Running task console_log... 2023-10-03T09:54:36.283 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa1914daf70>, signals=[15]) 2023-10-03T09:54:36.283 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-03T09:54:36.290 INFO:teuthology.task.internal:Opening connections... 2023-10-03T09:54:36.290 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi003.front.sepia.ceph.com 2023-10-03T09:54:36.291 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-03T09:54:36.358 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi022.front.sepia.ceph.com 2023-10-03T09:54:36.359 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-03T09:54:36.427 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-03T09:54:36.434 DEBUG:teuthology.orchestra.run.smithi003:> uname -m 2023-10-03T09:54:36.457 INFO:teuthology.orchestra.run.smithi003.stdout:x86_64 2023-10-03T09:54:36.457 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release 2023-10-03T09:54:36.518 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="CentOS Stream" 2023-10-03T09:54:36.518 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="8" 2023-10-03T09:54:36.518 INFO:teuthology.orchestra.run.smithi003.stdout:ID="centos" 2023-10-03T09:54:36.518 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE="rhel fedora" 2023-10-03T09:54:36.518 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="8" 2023-10-03T09:54:36.518 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM_ID="platform:el8" 2023-10-03T09:54:36.518 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-03T09:54:36.518 INFO:teuthology.orchestra.run.smithi003.stdout:ANSI_COLOR="0;31" 2023-10-03T09:54:36.518 INFO:teuthology.orchestra.run.smithi003.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-03T09:54:36.519 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://centos.org/" 2023-10-03T09:54:36.519 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-03T09:54:36.519 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-03T09:54:36.519 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-03T09:54:36.519 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server 2023-10-03T09:54:36.548 DEBUG:teuthology.orchestra.run.smithi022:> uname -m 2023-10-03T09:54:36.571 INFO:teuthology.orchestra.run.smithi022.stdout:x86_64 2023-10-03T09:54:36.571 DEBUG:teuthology.orchestra.run.smithi022:> cat /etc/os-release 2023-10-03T09:54:36.633 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="CentOS Stream" 2023-10-03T09:54:36.633 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="8" 2023-10-03T09:54:36.633 INFO:teuthology.orchestra.run.smithi022.stdout:ID="centos" 2023-10-03T09:54:36.633 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE="rhel fedora" 2023-10-03T09:54:36.633 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="8" 2023-10-03T09:54:36.634 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM_ID="platform:el8" 2023-10-03T09:54:36.634 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-03T09:54:36.634 INFO:teuthology.orchestra.run.smithi022.stdout:ANSI_COLOR="0;31" 2023-10-03T09:54:36.634 INFO:teuthology.orchestra.run.smithi022.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-03T09:54:36.634 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://centos.org/" 2023-10-03T09:54:36.634 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-03T09:54:36.634 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-03T09:54:36.634 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-03T09:54:36.634 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server 2023-10-03T09:54:36.656 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-03T09:54:36.664 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-03T09:54:36.670 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-03T09:54:36.670 DEBUG:teuthology.orchestra.run.smithi003:> test '!' -e /home/ubuntu/cephtest 2023-10-03T09:54:36.672 DEBUG:teuthology.orchestra.run.smithi022:> test '!' -e /home/ubuntu/cephtest 2023-10-03T09:54:36.693 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-03T09:54:36.700 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-03T09:54:36.700 DEBUG:teuthology.orchestra.run.smithi003:> test -z $(ls -A /var/lib/ceph) 2023-10-03T09:54:36.734 DEBUG:teuthology.orchestra.run.smithi022:> test -z $(ls -A /var/lib/ceph) 2023-10-03T09:54:36.764 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-03T09:54:36.820 INFO:teuthology.run_tasks:Running task kernel... 2023-10-03T09:54:36.834 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-10-03T09:54:36.834 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-10-03T09:54:36.834 DEBUG:teuthology.orchestra.run.smithi003:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-03T09:54:36.835 DEBUG:teuthology.orchestra.run.smithi022:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-03T09:54:36.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-03T09:54:36.856 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2023-10-03T09:54:36.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-03T09:54:36.857 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2023-10-03T09:54:36.917 INFO:teuthology.orchestra.run.smithi022.stdout:4.18.0-489.el8.x86_64 2023-10-03T09:54:36.918 INFO:teuthology.task.kernel:Running kernel on smithi022: 4.18.0-489.el8.x86_64 2023-10-03T09:54:36.918 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum install -y kernel 2023-10-03T09:54:36.919 INFO:teuthology.orchestra.run.smithi003.stdout:4.18.0-489.el8.x86_64 2023-10-03T09:54:36.919 INFO:teuthology.task.kernel:Running kernel on smithi003: 4.18.0-489.el8.x86_64 2023-10-03T09:54:36.920 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum install -y kernel 2023-10-03T09:54:41.777 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 8 - AppStream 14 MB/s | 33 MB 00:02 2023-10-03T09:54:41.971 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 8 - AppStream 14 MB/s | 33 MB 00:02 2023-10-03T09:54:50.535 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 8 - BaseOS 21 MB/s | 48 MB 00:02 2023-10-03T09:54:50.721 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 8 - BaseOS 21 MB/s | 48 MB 00:02 2023-10-03T09:54:58.034 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 8 - Extras 91 kB/s | 18 kB 00:00 2023-10-03T09:54:58.229 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 8 - Extras 91 kB/s | 18 kB 00:00 2023-10-03T09:54:58.393 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 8 - Extras common packages 41 kB/s | 6.9 kB 00:00 2023-10-03T09:54:58.571 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 8 - Extras common packages 45 kB/s | 6.9 kB 00:00 2023-10-03T09:54:58.695 INFO:teuthology.orchestra.run.smithi003.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-10-03T09:54:58.864 INFO:teuthology.orchestra.run.smithi022.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2023-10-03T09:54:59.773 INFO:teuthology.orchestra.run.smithi022.stdout:Extra Packages for Enterprise Linux 21 MB/s | 16 MB 00:00 2023-10-03T09:55:03.832 INFO:teuthology.orchestra.run.smithi022.stdout:lab-extras 199 kB/s | 24 kB 00:00 2023-10-03T09:55:05.447 INFO:teuthology.orchestra.run.smithi003.stdout:Extra Packages for Enterprise Linux 2.4 MB/s | 16 MB 00:06 2023-10-03T09:55:07.531 INFO:teuthology.orchestra.run.smithi022.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2023-10-03T09:55:07.531 INFO:teuthology.orchestra.run.smithi022.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2023-10-03T09:55:07.802 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2023-10-03T09:55:07.802 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2023-10-03T09:55:07.803 INFO:teuthology.orchestra.run.smithi022.stdout: Package Architecture Version Repository Size 2023-10-03T09:55:07.803 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2023-10-03T09:55:07.803 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2023-10-03T09:55:07.803 INFO:teuthology.orchestra.run.smithi022.stdout: kernel x86_64 4.18.0-514.el8 baseos 10 M 2023-10-03T09:55:07.803 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2023-10-03T09:55:07.803 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-core x86_64 4.18.0-514.el8 baseos 43 M 2023-10-03T09:55:07.803 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules x86_64 4.18.0-514.el8 baseos 35 M 2023-10-03T09:55:07.803 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T09:55:07.803 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2023-10-03T09:55:07.803 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2023-10-03T09:55:07.803 INFO:teuthology.orchestra.run.smithi022.stdout:Install 3 Packages 2023-10-03T09:55:07.803 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T09:55:07.804 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 88 M 2023-10-03T09:55:07.804 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 96 M 2023-10-03T09:55:07.804 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2023-10-03T09:55:09.520 INFO:teuthology.orchestra.run.smithi003.stdout:lab-extras 611 kB/s | 24 kB 00:00 2023-10-03T09:55:10.283 INFO:teuthology.orchestra.run.smithi022.stdout:(1/3): kernel-4.18.0-514.el8.x86_64.rpm 4.8 MB/s | 10 MB 00:02 2023-10-03T09:55:11.333 INFO:teuthology.orchestra.run.smithi022.stdout:(2/3): kernel-core-4.18.0-514.el8.x86_64.rpm 14 MB/s | 43 MB 00:03 2023-10-03T09:55:11.984 INFO:teuthology.orchestra.run.smithi022.stdout:(3/3): kernel-modules-4.18.0-514.el8.x86_64.rpm 9.3 MB/s | 35 MB 00:03 2023-10-03T09:55:11.984 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2023-10-03T09:55:11.984 INFO:teuthology.orchestra.run.smithi022.stdout:Total 21 MB/s | 88 MB 00:04 2023-10-03T09:55:12.686 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2023-10-03T09:55:13.121 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2023-10-03T09:55:13.121 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2023-10-03T09:55:13.152 INFO:teuthology.orchestra.run.smithi003.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2023-10-03T09:55:13.153 INFO:teuthology.orchestra.run.smithi003.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2023-10-03T09:55:13.420 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2023-10-03T09:55:13.420 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2023-10-03T09:55:13.420 INFO:teuthology.orchestra.run.smithi003.stdout: Package Architecture Version Repository Size 2023-10-03T09:55:13.420 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2023-10-03T09:55:13.420 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2023-10-03T09:55:13.420 INFO:teuthology.orchestra.run.smithi003.stdout: kernel x86_64 4.18.0-514.el8 baseos 10 M 2023-10-03T09:55:13.420 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2023-10-03T09:55:13.420 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-core x86_64 4.18.0-514.el8 baseos 43 M 2023-10-03T09:55:13.420 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules x86_64 4.18.0-514.el8 baseos 35 M 2023-10-03T09:55:13.421 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T09:55:13.421 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2023-10-03T09:55:13.421 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2023-10-03T09:55:13.421 INFO:teuthology.orchestra.run.smithi003.stdout:Install 3 Packages 2023-10-03T09:55:13.421 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T09:55:13.421 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 88 M 2023-10-03T09:55:13.422 INFO:teuthology.orchestra.run.smithi003.stdout:Installed size: 96 M 2023-10-03T09:55:13.422 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2023-10-03T09:55:14.397 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2023-10-03T09:55:14.421 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2023-10-03T09:55:15.288 INFO:teuthology.orchestra.run.smithi003.stdout:(1/3): kernel-4.18.0-514.el8.x86_64.rpm 6.0 MB/s | 10 MB 00:01 2023-10-03T09:55:16.330 INFO:teuthology.orchestra.run.smithi003.stdout:(2/3): kernel-modules-4.18.0-514.el8.x86_64.rpm 13 MB/s | 35 MB 00:02 2023-10-03T09:55:16.921 INFO:teuthology.orchestra.run.smithi003.stdout:(3/3): kernel-core-4.18.0-514.el8.x86_64.rpm 13 MB/s | 43 MB 00:03 2023-10-03T09:55:16.922 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2023-10-03T09:55:16.922 INFO:teuthology.orchestra.run.smithi003.stdout:Total 25 MB/s | 88 MB 00:03 2023-10-03T09:55:17.342 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2023-10-03T09:55:17.629 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2023-10-03T09:55:17.850 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-core-4.18.0-514.el8.x86_64 1/3 2023-10-03T09:55:18.067 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2023-10-03T09:55:18.067 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2023-10-03T09:55:19.360 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2023-10-03T09:55:19.383 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2023-10-03T09:55:21.663 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 1/3 2023-10-03T09:55:22.033 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-10-03T09:55:22.353 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2023-10-03T09:55:22.879 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-core-4.18.0-514.el8.x86_64 1/3 2023-10-03T09:55:25.654 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-10-03T09:55:26.026 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-4.18.0-514.el8.x86_64 3/3 2023-10-03T09:55:26.616 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 1/3 2023-10-03T09:55:26.987 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-10-03T09:55:30.612 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-10-03T09:55:30.930 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-4.18.0-514.el8.x86_64 3/3 2023-10-03T09:55:49.140 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 3/3 2023-10-03T09:55:49.164 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-10-03T09:55:49.554 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-4.18.0-514.el8.x86_64 3/3 2023-10-03T09:55:49.554 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-4.18.0-514.el8.x86_64 1/3 2023-10-03T09:55:49.555 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-core-4.18.0-514.el8.x86_64 2/3 2023-10-03T09:55:50.326 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-10-03T09:55:50.326 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T09:55:50.326 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2023-10-03T09:55:50.326 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-4.18.0-514.el8.x86_64 kernel-core-4.18.0-514.el8.x86_64 2023-10-03T09:55:50.326 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules-4.18.0-514.el8.x86_64 2023-10-03T09:55:50.326 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T09:55:50.326 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2023-10-03T09:55:50.468 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q kernel | sort -rV | head -n 1 2023-10-03T09:55:50.692 INFO:teuthology.orchestra.run.smithi022.stdout:kernel-4.18.0-514.el8.x86_64 2023-10-03T09:55:50.692 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-10-03T09:55:50.692 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-514.el8.x86_64 2023-10-03T09:55:50.692 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-03T09:55:50.692 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-03T09:55:50.693 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2023-10-03T09:55:50.693 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi022.front.sepia.ceph.com, path=None, version=distro) 2023-10-03T09:55:50.693 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q kernel | sort -rV | head -n 1 2023-10-03T09:55:50.909 INFO:teuthology.orchestra.run.smithi022.stdout:kernel-4.18.0-514.el8.x86_64 2023-10-03T09:55:50.909 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-10-03T09:55:50.910 DEBUG:teuthology.orchestra.run.smithi022:> sudo rpm -qi grub2-tools 2023-10-03T09:55:50.944 INFO:teuthology.orchestra.run.smithi022.stdout:Name : grub2-tools 2023-10-03T09:55:50.944 INFO:teuthology.orchestra.run.smithi022.stdout:Epoch : 1 2023-10-03T09:55:50.944 INFO:teuthology.orchestra.run.smithi022.stdout:Version : 2.02 2023-10-03T09:55:50.944 INFO:teuthology.orchestra.run.smithi022.stdout:Release : 129.el8 2023-10-03T09:55:50.944 INFO:teuthology.orchestra.run.smithi022.stdout:Architecture: x86_64 2023-10-03T09:55:50.944 INFO:teuthology.orchestra.run.smithi022.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-10-03T09:55:50.944 INFO:teuthology.orchestra.run.smithi022.stdout:Group : System Environment/Base 2023-10-03T09:55:50.944 INFO:teuthology.orchestra.run.smithi022.stdout:Size : 9265586 2023-10-03T09:55:50.944 INFO:teuthology.orchestra.run.smithi022.stdout:License : GPLv3+ 2023-10-03T09:55:50.944 INFO:teuthology.orchestra.run.smithi022.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-10-03T09:55:50.944 INFO:teuthology.orchestra.run.smithi022.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-10-03T09:55:50.945 INFO:teuthology.orchestra.run.smithi022.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-10-03T09:55:50.945 INFO:teuthology.orchestra.run.smithi022.stdout:Build Host : kbuilder.bsys.centos.org 2023-10-03T09:55:50.945 INFO:teuthology.orchestra.run.smithi022.stdout:Relocations : (not relocatable) 2023-10-03T09:55:50.945 INFO:teuthology.orchestra.run.smithi022.stdout:Packager : CentOS BuildSystem 2023-10-03T09:55:50.945 INFO:teuthology.orchestra.run.smithi022.stdout:Vendor : CentOS 2023-10-03T09:55:50.945 INFO:teuthology.orchestra.run.smithi022.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-03T09:55:50.945 INFO:teuthology.orchestra.run.smithi022.stdout:Summary : Support tools for GRUB. 2023-10-03T09:55:50.945 INFO:teuthology.orchestra.run.smithi022.stdout:Description : 2023-10-03T09:55:50.945 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T09:55:50.945 INFO:teuthology.orchestra.run.smithi022.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-03T09:55:50.945 INFO:teuthology.orchestra.run.smithi022.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-03T09:55:50.945 INFO:teuthology.orchestra.run.smithi022.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-03T09:55:50.945 INFO:teuthology.orchestra.run.smithi022.stdout:hardware devices. 2023-10-03T09:55:50.945 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T09:55:50.946 INFO:teuthology.orchestra.run.smithi022.stdout:This subpackage provides tools for support of all platforms. 2023-10-03T09:55:50.946 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-03T09:55:50.946 INFO:teuthology.task.kernel:Updating grub on smithi022 to boot 4.18.0-514.el8.x86_64 2023-10-03T09:55:50.946 DEBUG:teuthology.orchestra.run.smithi022:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-03T09:55:52.312 INFO:teuthology.orchestra.run.smithi022.stderr:Generating grub configuration file ... 2023-10-03T09:55:54.284 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 3/3 2023-10-03T09:55:54.307 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-10-03T09:55:54.472 INFO:teuthology.orchestra.run.smithi022.stderr:done 2023-10-03T09:55:54.474 DEBUG:teuthology.orchestra.run.smithi022:> mktemp 2023-10-03T09:55:54.504 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.Aee4ry5OHd 2023-10-03T09:55:54.504 DEBUG:teuthology.orchestra.run.smithi022:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Aee4ry5OHd 2023-10-03T09:55:54.576 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.Aee4ry5OHd 2023-10-03T09:55:54.661 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-4.18.0-514.el8.x86_64 3/3 2023-10-03T09:55:54.661 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-4.18.0-514.el8.x86_64 1/3 2023-10-03T09:55:54.661 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-core-4.18.0-514.el8.x86_64 2/3 2023-10-03T09:55:54.722 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.Aee4ry5OHd is 6KB 2023-10-03T09:55:54.770 DEBUG:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.Aee4ry5OHd 2023-10-03T09:55:54.788 DEBUG:teuthology.orchestra.run.smithi022:> sudo /bin/ls /boot/loader/entries || true 2023-10-03T09:55:54.854 INFO:teuthology.orchestra.run.smithi022.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-10-03T09:55:54.854 INFO:teuthology.orchestra.run.smithi022.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-10-03T09:55:54.854 INFO:teuthology.orchestra.run.smithi022.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-10-03T09:55:54.854 INFO:teuthology.orchestra.run.smithi022.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64.conf 2023-10-03T09:55:54.855 DEBUG:teuthology.orchestra.run.smithi022:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64 2023-10-03T09:55:55.039 DEBUG:teuthology.orchestra.run.smithi022:> sudo shutdown -r now 2023-10-03T09:55:55.365 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-10-03T09:55:55.366 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T09:55:55.366 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2023-10-03T09:55:55.366 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-4.18.0-514.el8.x86_64 kernel-core-4.18.0-514.el8.x86_64 2023-10-03T09:55:55.366 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules-4.18.0-514.el8.x86_64 2023-10-03T09:55:55.366 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T09:55:55.366 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2023-10-03T09:55:55.503 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q kernel | sort -rV | head -n 1 2023-10-03T09:55:55.723 INFO:teuthology.orchestra.run.smithi003.stdout:kernel-4.18.0-514.el8.x86_64 2023-10-03T09:55:55.724 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-10-03T09:55:55.724 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-514.el8.x86_64 2023-10-03T09:55:55.724 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-03T09:55:55.724 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-03T09:55:55.724 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-10-03T09:55:55.725 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi003.front.sepia.ceph.com, path=None, version=distro) 2023-10-03T09:55:55.725 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q kernel | sort -rV | head -n 1 2023-10-03T09:55:55.942 INFO:teuthology.orchestra.run.smithi003.stdout:kernel-4.18.0-514.el8.x86_64 2023-10-03T09:55:55.943 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-10-03T09:55:55.943 DEBUG:teuthology.orchestra.run.smithi003:> sudo rpm -qi grub2-tools 2023-10-03T09:55:55.977 INFO:teuthology.orchestra.run.smithi003.stdout:Name : grub2-tools 2023-10-03T09:55:55.977 INFO:teuthology.orchestra.run.smithi003.stdout:Epoch : 1 2023-10-03T09:55:55.977 INFO:teuthology.orchestra.run.smithi003.stdout:Version : 2.02 2023-10-03T09:55:55.978 INFO:teuthology.orchestra.run.smithi003.stdout:Release : 129.el8 2023-10-03T09:55:55.978 INFO:teuthology.orchestra.run.smithi003.stdout:Architecture: x86_64 2023-10-03T09:55:55.978 INFO:teuthology.orchestra.run.smithi003.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-10-03T09:55:55.978 INFO:teuthology.orchestra.run.smithi003.stdout:Group : System Environment/Base 2023-10-03T09:55:55.978 INFO:teuthology.orchestra.run.smithi003.stdout:Size : 9265586 2023-10-03T09:55:55.978 INFO:teuthology.orchestra.run.smithi003.stdout:License : GPLv3+ 2023-10-03T09:55:55.978 INFO:teuthology.orchestra.run.smithi003.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-10-03T09:55:55.978 INFO:teuthology.orchestra.run.smithi003.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-10-03T09:55:55.978 INFO:teuthology.orchestra.run.smithi003.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-10-03T09:55:55.978 INFO:teuthology.orchestra.run.smithi003.stdout:Build Host : kbuilder.bsys.centos.org 2023-10-03T09:55:55.978 INFO:teuthology.orchestra.run.smithi003.stdout:Relocations : (not relocatable) 2023-10-03T09:55:55.978 INFO:teuthology.orchestra.run.smithi003.stdout:Packager : CentOS BuildSystem 2023-10-03T09:55:55.979 INFO:teuthology.orchestra.run.smithi003.stdout:Vendor : CentOS 2023-10-03T09:55:55.979 INFO:teuthology.orchestra.run.smithi003.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-03T09:55:55.979 INFO:teuthology.orchestra.run.smithi003.stdout:Summary : Support tools for GRUB. 2023-10-03T09:55:55.979 INFO:teuthology.orchestra.run.smithi003.stdout:Description : 2023-10-03T09:55:55.979 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T09:55:55.979 INFO:teuthology.orchestra.run.smithi003.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-03T09:55:55.979 INFO:teuthology.orchestra.run.smithi003.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-03T09:55:55.979 INFO:teuthology.orchestra.run.smithi003.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-03T09:55:55.979 INFO:teuthology.orchestra.run.smithi003.stdout:hardware devices. 2023-10-03T09:55:55.979 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T09:55:55.979 INFO:teuthology.orchestra.run.smithi003.stdout:This subpackage provides tools for support of all platforms. 2023-10-03T09:55:55.980 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-03T09:55:55.980 INFO:teuthology.task.kernel:Updating grub on smithi003 to boot 4.18.0-514.el8.x86_64 2023-10-03T09:55:55.980 DEBUG:teuthology.orchestra.run.smithi003:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-03T09:55:57.400 INFO:teuthology.orchestra.run.smithi003.stderr:Generating grub configuration file ... 2023-10-03T09:55:59.740 INFO:teuthology.orchestra.run.smithi003.stderr:done 2023-10-03T09:55:59.745 DEBUG:teuthology.orchestra.run.smithi003:> mktemp 2023-10-03T09:55:59.772 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.kZqDkRdeEI 2023-10-03T09:55:59.773 DEBUG:teuthology.orchestra.run.smithi003:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.kZqDkRdeEI 2023-10-03T09:55:59.845 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0666 /tmp/tmp.kZqDkRdeEI 2023-10-03T09:55:59.990 DEBUG:teuthology.orchestra.remote:smithi003:/tmp/tmp.kZqDkRdeEI is 6KB 2023-10-03T09:56:00.038 DEBUG:teuthology.orchestra.run.smithi003:> rm -fr /tmp/tmp.kZqDkRdeEI 2023-10-03T09:56:00.056 DEBUG:teuthology.orchestra.run.smithi003:> sudo /bin/ls /boot/loader/entries || true 2023-10-03T09:56:00.122 INFO:teuthology.orchestra.run.smithi003.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-10-03T09:56:00.122 INFO:teuthology.orchestra.run.smithi003.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-10-03T09:56:00.123 INFO:teuthology.orchestra.run.smithi003.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-10-03T09:56:00.123 INFO:teuthology.orchestra.run.smithi003.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64.conf 2023-10-03T09:56:00.124 DEBUG:teuthology.orchestra.run.smithi003:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64 2023-10-03T09:56:00.273 DEBUG:teuthology.orchestra.run.smithi003:> sudo shutdown -r now 2023-10-03T09:56:25.045 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-03T09:56:25.046 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2023-10-03T09:56:25.047 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-03T09:56:30.285 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-10-03T09:56:30.286 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-10-03T09:56:30.286 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-03T09:56:43.515 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.22 2023-10-03T09:56:48.634 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.3 2023-10-03T09:56:52.520 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2023-10-03T09:56:52.520 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-03T09:56:57.641 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-10-03T09:56:57.641 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-03T09:57:52.579 DEBUG:teuthology.orchestra.remote:timed out 2023-10-03T09:57:57.649 DEBUG:teuthology.orchestra.remote:timed out 2023-10-03T09:58:04.587 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2023-10-03T09:58:04.589 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-03T09:58:04.886 DEBUG:teuthology.orchestra.run.smithi022:> true 2023-10-03T09:58:05.191 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2023-10-03T09:58:05.191 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "4.18.0-514.el8.x86_64"... 2023-10-03T09:58:05.192 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2023-10-03T09:58:05.213 INFO:teuthology.orchestra.run.smithi022.stdout:4.18.0-514.el8.x86_64 2023-10-03T09:58:05.214 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-514.el8.x86_64 vs 4.18.0-514.el8.x86_64 2023-10-03T09:58:05.214 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-03T09:58:05.214 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-03T09:58:06.214 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2023-10-03T09:58:06.215 DEBUG:teuthology.orchestra.run.smithi022:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-03T09:58:06.302 INFO:teuthology.orchestra.run.smithi022.stdout:ttyS1 2023-10-03T09:58:06.316 DEBUG:teuthology.parallel:result is None 2023-10-03T09:58:09.651 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-10-03T09:58:09.651 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-03T09:58:09.988 DEBUG:teuthology.orchestra.run.smithi003:> true 2023-10-03T09:58:10.302 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-10-03T09:58:10.302 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-514.el8.x86_64"... 2023-10-03T09:58:10.302 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2023-10-03T09:58:10.324 INFO:teuthology.orchestra.run.smithi003.stdout:4.18.0-514.el8.x86_64 2023-10-03T09:58:10.325 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-514.el8.x86_64 vs 4.18.0-514.el8.x86_64 2023-10-03T09:58:10.325 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-03T09:58:10.325 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-03T09:58:11.325 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-10-03T09:58:11.326 DEBUG:teuthology.orchestra.run.smithi003:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-03T09:58:11.406 INFO:teuthology.orchestra.run.smithi003.stdout:ttyS1 2023-10-03T09:58:11.419 DEBUG:teuthology.parallel:result is None 2023-10-03T09:58:11.419 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-03T09:58:11.427 INFO:teuthology.task.internal:Creating test directory... 2023-10-03T09:58:11.427 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-03T09:58:11.429 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-03T09:58:11.452 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-03T09:58:11.459 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-03T09:58:11.469 INFO:teuthology.task.internal:Creating archive directory... 2023-10-03T09:58:11.469 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-03T09:58:11.492 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-03T09:58:11.540 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-03T09:58:11.547 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-03T09:58:11.548 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-03T09:58:11.572 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-03T09:58:11.613 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-03T09:58:11.625 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-03T09:58:11.626 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-03T09:58:11.638 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-03T09:58:11.640 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-03T09:58:11.646 INFO:teuthology.task.internal:Configuring sudo... 2023-10-03T09:58:11.646 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-03T09:58:11.648 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-03T09:58:11.683 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-03T09:58:11.691 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-03T09:58:11.691 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-03T09:58:11.724 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-03T09:58:11.747 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-03T09:58:11.834 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-03T09:58:11.895 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-03T09:58:11.944 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-03T09:58:11.996 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-03T09:58:11.996 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-03T09:58:12.071 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-03T09:58:12.120 DEBUG:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-03T09:58:12.177 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-03T09:58:12.266 DEBUG:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-03T09:58:12.319 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-10-03T09:58:12.319 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-03T09:58:12.393 DEBUG:teuthology.orchestra.run.smithi003:> sudo service rsyslog restart 2023-10-03T09:58:12.396 DEBUG:teuthology.orchestra.run.smithi022:> sudo service rsyslog restart 2023-10-03T09:58:12.463 INFO:teuthology.orchestra.run.smithi003.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-03T09:58:12.504 INFO:teuthology.orchestra.run.smithi022.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-03T09:58:12.938 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-03T09:58:12.945 INFO:teuthology.task.internal:Starting timer... 2023-10-03T09:58:12.945 INFO:teuthology.run_tasks:Running task pcp... 2023-10-03T09:58:12.954 INFO:teuthology.run_tasks:Running task selinux... 2023-10-03T09:58:12.961 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-10-03T09:58:12.962 DEBUG:teuthology.orchestra.run.smithi003:> sudo service auditd rotate 2023-10-03T09:58:13.082 INFO:teuthology.orchestra.run.smithi003.stdout:Rotating logs: 2023-10-03T09:58:13.084 DEBUG:teuthology.orchestra.run.smithi022:> sudo service auditd rotate 2023-10-03T09:58:13.172 INFO:teuthology.orchestra.run.smithi022.stdout:Rotating logs: 2023-10-03T09:58:13.174 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-03T09:58:13.174 DEBUG:teuthology.orchestra.run.smithi003:> /usr/sbin/getenforce 2023-10-03T09:58:13.230 INFO:teuthology.orchestra.run.smithi003.stdout:Permissive 2023-10-03T09:58:13.230 DEBUG:teuthology.orchestra.run.smithi022:> /usr/sbin/getenforce 2023-10-03T09:58:13.277 INFO:teuthology.orchestra.run.smithi022.stdout:Permissive 2023-10-03T09:58:13.277 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi003.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi022.front.sepia.ceph.com': 'permissive'} 2023-10-03T09:58:13.277 DEBUG:teuthology.orchestra.run.smithi003:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-03T09:58:13.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-03T09:58:13.312 DEBUG:teuthology.orchestra.run.smithi022:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-03T09:58:13.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-03T09:58:13.347 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-03T09:58:13.347 DEBUG:teuthology.orchestra.run.smithi003:> sudo /usr/sbin/setenforce permissive 2023-10-03T09:58:13.387 DEBUG:teuthology.orchestra.run.smithi022:> sudo /usr/sbin/setenforce permissive 2023-10-03T09:58:13.428 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-03T09:58:13.436 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-03T09:58:13.437 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-03T09:58:13.453 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-03T09:58:13.455 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi003.front.sepia.ceph.com,smithi022.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-03T10:06:26.058 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi003.front.sepia.ceph.com'), Remote(name='ubuntu@smithi022.front.sepia.ceph.com')] 2023-10-03T10:06:26.059 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-10-03T10:06:26.059 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-03T10:06:26.130 DEBUG:teuthology.orchestra.run.smithi003:> true 2023-10-03T10:06:26.209 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-10-03T10:06:26.209 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2023-10-03T10:06:26.209 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-03T10:06:26.282 DEBUG:teuthology.orchestra.run.smithi022:> true 2023-10-03T10:06:26.357 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2023-10-03T10:06:26.357 INFO:teuthology.run_tasks:Running task clock... 2023-10-03T10:06:26.371 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-03T10:06:26.371 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-03T10:06:26.371 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-03T10:06:26.374 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-03T10:06:26.374 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-03T10:06:26.411 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-03T10:06:26.429 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-03T10:06:26.438 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-03T10:06:26.455 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-03T10:06:26.507 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: ntpd: command not found 2023-10-03T10:06:26.522 INFO:teuthology.orchestra.run.smithi003.stdout:506 Cannot talk to daemon 2023-10-03T10:06:26.524 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: ntpd: command not found 2023-10-03T10:06:26.538 INFO:teuthology.orchestra.run.smithi022.stdout:506 Cannot talk to daemon 2023-10-03T10:06:26.539 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-03T10:06:26.555 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-03T10:06:26.556 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-03T10:06:26.571 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-03T10:06:26.638 INFO:teuthology.orchestra.run.smithi003.stderr:bash: ntpq: command not found 2023-10-03T10:06:26.639 INFO:teuthology.orchestra.run.smithi022.stderr:bash: ntpq: command not found 2023-10-03T10:06:26.643 INFO:teuthology.orchestra.run.smithi003.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-03T10:06:26.643 INFO:teuthology.orchestra.run.smithi003.stdout:=============================================================================== 2023-10-03T10:06:26.643 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-03T10:06:26.643 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-03T10:06:26.643 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-03T10:06:26.643 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-03T10:06:26.645 INFO:teuthology.orchestra.run.smithi022.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-03T10:06:26.645 INFO:teuthology.orchestra.run.smithi022.stdout:=============================================================================== 2023-10-03T10:06:26.645 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-03T10:06:26.645 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-03T10:06:26.645 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-03T10:06:26.646 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-03T10:06:26.646 INFO:teuthology.run_tasks:Running task install... 2023-10-03T10:06:26.656 DEBUG:teuthology.task.install:project ceph 2023-10-03T10:06:26.656 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054'}} 2023-10-03T10:06:26.656 DEBUG:teuthology.task.install:config {'branch': 'pacific', 'exclude_packages': ['cephadm', 'ceph-mgr-cephadm', 'ceph-immutable-object-cache', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'ceph-volume'], 'extra_packages': ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'extra_system_packages': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux'], 'flavor': 'default', 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054'} 2023-10-03T10:06:26.657 INFO:teuthology.task.install:Using flavor: default 2023-10-03T10:06:26.662 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-03T10:06:26.662 INFO:teuthology.task.install:extra packages: ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'] 2023-10-03T10:06:26.663 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'pacific', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['cephadm', 'ceph-mgr-cephadm', 'ceph-immutable-object-cache', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'ceph-volume'], 'extra_packages': ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'extra_system_packages': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux'], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054', 'tag': None, 'wait_for_package': False} 2023-10-03T10:06:26.663 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-03T10:06:26.663 INFO:teuthology.packaging:ref: None 2023-10-03T10:06:26.663 INFO:teuthology.packaging:tag: None 2023-10-03T10:06:26.663 INFO:teuthology.packaging:branch: pacific 2023-10-03T10:06:26.663 INFO:teuthology.packaging:sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 2023-10-03T10:06:26.664 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=pacific 2023-10-03T10:06:26.665 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'pacific', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['cephadm', 'ceph-mgr-cephadm', 'ceph-immutable-object-cache', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'ceph-volume'], 'extra_packages': ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'extra_system_packages': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux'], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054', 'tag': None, 'wait_for_package': False} 2023-10-03T10:06:26.665 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-03T10:06:26.666 INFO:teuthology.packaging:ref: None 2023-10-03T10:06:26.666 INFO:teuthology.packaging:tag: None 2023-10-03T10:06:26.666 INFO:teuthology.packaging:branch: pacific 2023-10-03T10:06:26.666 INFO:teuthology.packaging:sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 2023-10-03T10:06:26.666 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=pacific 2023-10-03T10:06:26.975 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/ 2023-10-03T10:06:26.975 INFO:teuthology.task.install.rpm:Package version is 16.2.14-44.geeb13ac9 2023-10-03T10:06:26.980 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/ 2023-10-03T10:06:26.980 INFO:teuthology.task.install.rpm:Package version is 16.2.14-44.geeb13ac9 2023-10-03T10:06:27.094 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-03T10:06:27.095 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-03T10:06:27.095 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-03T10:06:27.131 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, rbd-fuse, rbd-mirror, rbd-nbd, python-rados, python-rgw, python-rbd, python-cephfs, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux on remote rpm x86_64 2023-10-03T10:06:27.131 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-03T10:06:27.131 INFO:teuthology.packaging:ref: None 2023-10-03T10:06:27.131 INFO:teuthology.packaging:tag: None 2023-10-03T10:06:27.131 INFO:teuthology.packaging:branch: pacific 2023-10-03T10:06:27.132 INFO:teuthology.packaging:sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 2023-10-03T10:06:27.132 DEBUG:teuthology.orchestra.run.smithi003:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/pacific/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-03T10:06:27.144 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-03T10:06:27.144 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-10-03T10:06:27.144 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-03T10:06:27.180 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, rbd-fuse, rbd-mirror, rbd-nbd, python-rados, python-rgw, python-rbd, python-cephfs, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux on remote rpm x86_64 2023-10-03T10:06:27.180 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-03T10:06:27.180 INFO:teuthology.packaging:ref: None 2023-10-03T10:06:27.180 INFO:teuthology.packaging:tag: None 2023-10-03T10:06:27.180 INFO:teuthology.packaging:branch: pacific 2023-10-03T10:06:27.181 INFO:teuthology.packaging:sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 2023-10-03T10:06:27.181 DEBUG:teuthology.orchestra.run.smithi022:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/pacific/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-03T10:06:27.208 DEBUG:teuthology.orchestra.run.smithi003:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-03T10:06:27.257 DEBUG:teuthology.orchestra.run.smithi022:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-03T10:06:27.296 DEBUG:teuthology.orchestra.run.smithi003:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-03T10:06:27.329 INFO:teuthology.orchestra.run.smithi003.stdout:check_obsoletes = 1 2023-10-03T10:06:27.331 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum clean all 2023-10-03T10:06:27.345 DEBUG:teuthology.orchestra.run.smithi022:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-03T10:06:27.378 INFO:teuthology.orchestra.run.smithi022.stdout:check_obsoletes = 1 2023-10-03T10:06:27.380 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum clean all 2023-10-03T10:06:27.643 INFO:teuthology.orchestra.run.smithi003.stdout:76 files removed 2023-10-03T10:06:27.675 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 rbd-fuse rbd-mirror rbd-nbd python-rados python-rgw python-rbd python-cephfs bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux 2023-10-03T10:06:27.692 INFO:teuthology.orchestra.run.smithi022.stdout:76 files removed 2023-10-03T10:06:27.728 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 rbd-fuse rbd-mirror rbd-nbd python-rados python-rgw python-rbd python-cephfs bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux 2023-10-03T10:06:28.542 INFO:teuthology.orchestra.run.smithi003.stdout:ceph packages for x86_64 190 kB/s | 76 kB 00:00 2023-10-03T10:06:28.678 INFO:teuthology.orchestra.run.smithi022.stdout:ceph packages for x86_64 159 kB/s | 76 kB 00:00 2023-10-03T10:06:28.868 INFO:teuthology.orchestra.run.smithi003.stdout:ceph noarch packages 54 kB/s | 15 kB 00:00 2023-10-03T10:06:29.013 INFO:teuthology.orchestra.run.smithi022.stdout:ceph noarch packages 53 kB/s | 15 kB 00:00 2023-10-03T10:06:29.177 INFO:teuthology.orchestra.run.smithi003.stdout:ceph source packages 5.2 kB/s | 1.4 kB 00:00 2023-10-03T10:06:29.280 INFO:teuthology.orchestra.run.smithi022.stdout:ceph source packages 6.1 kB/s | 1.4 kB 00:00 2023-10-03T10:06:31.772 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 8 - AppStream 13 MB/s | 33 MB 00:02 2023-10-03T10:06:32.063 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 8 - AppStream 12 MB/s | 33 MB 00:02 2023-10-03T10:06:40.625 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 8 - BaseOS 22 MB/s | 48 MB 00:02 2023-10-03T10:06:41.321 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 8 - BaseOS 19 MB/s | 48 MB 00:02 2023-10-03T10:06:48.561 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 8 - Extras 51 kB/s | 18 kB 00:00 2023-10-03T10:06:49.061 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 8 - Extras common packages 23 kB/s | 6.9 kB 00:00 2023-10-03T10:06:49.205 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 8 - Extras 55 kB/s | 18 kB 00:00 2023-10-03T10:06:49.564 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 8 - Extras common packages 43 kB/s | 6.9 kB 00:00 2023-10-03T10:06:49.721 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 8 - PowerTools 13 MB/s | 6.5 MB 00:00 2023-10-03T10:06:50.674 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 8 - PowerTools 6.9 MB/s | 6.5 MB 00:00 2023-10-03T10:06:51.256 INFO:teuthology.orchestra.run.smithi003.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2023-10-03T10:06:52.225 INFO:teuthology.orchestra.run.smithi022.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2023-10-03T10:06:52.573 INFO:teuthology.orchestra.run.smithi003.stdout:Extra Packages for Enterprise Linux 14 MB/s | 16 MB 00:01 2023-10-03T10:06:56.775 INFO:teuthology.orchestra.run.smithi022.stdout:Extra Packages for Enterprise Linux 3.6 MB/s | 16 MB 00:04 2023-10-03T10:06:56.815 INFO:teuthology.orchestra.run.smithi003.stdout:lab-extras 384 kB/s | 24 kB 00:00 2023-10-03T10:07:01.175 INFO:teuthology.orchestra.run.smithi022.stdout:lab-extras 404 kB/s | 24 kB 00:00 2023-10-03T10:07:01.465 INFO:teuthology.orchestra.run.smithi003.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-03T10:07:01.467 INFO:teuthology.orchestra.run.smithi003.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-03T10:07:01.494 INFO:teuthology.orchestra.run.smithi003.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2023-10-03T10:07:01.494 INFO:teuthology.orchestra.run.smithi003.stdout:Package NetworkManager-initscripts-updown-1:1.40.16-4.el8.noarch is already installed. 2023-10-03T10:07:01.495 INFO:teuthology.orchestra.run.smithi003.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2023-10-03T10:07:01.496 INFO:teuthology.orchestra.run.smithi003.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2023-10-03T10:07:01.728 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2023-10-03T10:07:01.740 INFO:teuthology.orchestra.run.smithi003.stdout:==================================================================================================== 2023-10-03T10:07:01.741 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repository Size 2023-10-03T10:07:01.741 INFO:teuthology.orchestra.run.smithi003.stdout:==================================================================================================== 2023-10-03T10:07:01.741 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2023-10-03T10:07:01.741 INFO:teuthology.orchestra.run.smithi003.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2023-10-03T10:07:01.741 INFO:teuthology.orchestra.run.smithi003.stdout: ceph x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 6.4 k 2023-10-03T10:07:01.741 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 6.1 M 2023-10-03T10:07:01.741 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-fuse x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 910 k 2023-10-03T10:07:01.741 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 1.6 M 2023-10-03T10:07:01.741 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 3.1 M 2023-10-03T10:07:01.741 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 8.6 M 2023-10-03T10:07:01.741 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-rook noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 27 k 2023-10-03T10:07:01.741 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-radosgw x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 12 M 2023-10-03T10:07:01.742 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-test x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 52 M 2023-10-03T10:07:01.742 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 baseos 62 k 2023-10-03T10:07:01.742 INFO:teuthology.orchestra.run.smithi003.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2023-10-03T10:07:01.742 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-devel x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 25 k 2023-10-03T10:07:01.742 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 824 k 2023-10-03T10:07:01.742 INFO:teuthology.orchestra.run.smithi003.stdout: librados-devel x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 131 k 2023-10-03T10:07:01.742 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 baseos 3.2 M 2023-10-03T10:07:01.742 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cephfs x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 215 k 2023-10-03T10:07:01.742 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 390 k 2023-10-03T10:07:01.742 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rbd x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 369 k 2023-10-03T10:07:01.742 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rgw x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 114 k 2023-10-03T10:07:01.743 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-fuse x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 88 k 2023-10-03T10:07:01.743 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-mirror x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 3.7 M 2023-10-03T10:07:01.743 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-nbd x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 173 k 2023-10-03T10:07:01.743 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrading: 2023-10-03T10:07:01.743 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 baseos 143 k 2023-10-03T10:07:01.743 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-debuginfod-client x86_64 0.189-3.el8 baseos 76 k 2023-10-03T10:07:01.743 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libelf x86_64 0.189-3.el8 baseos 232 k 2023-10-03T10:07:01.743 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libs x86_64 0.189-3.el8 baseos 303 k 2023-10-03T10:07:01.743 INFO:teuthology.orchestra.run.smithi003.stdout: iproute x86_64 6.2.0-5.el8 baseos 854 k 2023-10-03T10:07:01.743 INFO:teuthology.orchestra.run.smithi003.stdout: libblkid x86_64 2.32.1-43.el8 baseos 220 k 2023-10-03T10:07:01.743 INFO:teuthology.orchestra.run.smithi003.stdout: libfdisk x86_64 2.32.1-43.el8 baseos 254 k 2023-10-03T10:07:01.743 INFO:teuthology.orchestra.run.smithi003.stdout: libmount x86_64 2.32.1-43.el8 baseos 236 k 2023-10-03T10:07:01.744 INFO:teuthology.orchestra.run.smithi003.stdout: librados2 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 3.7 M 2023-10-03T10:07:01.744 INFO:teuthology.orchestra.run.smithi003.stdout: librbd1 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 4.2 M 2023-10-03T10:07:01.744 INFO:teuthology.orchestra.run.smithi003.stdout: libsmartcols x86_64 2.32.1-43.el8 baseos 179 k 2023-10-03T10:07:01.744 INFO:teuthology.orchestra.run.smithi003.stdout: libuuid x86_64 2.32.1-43.el8 baseos 99 k 2023-10-03T10:07:01.744 INFO:teuthology.orchestra.run.smithi003.stdout: util-linux x86_64 2.32.1-43.el8 baseos 2.5 M 2023-10-03T10:07:01.744 INFO:teuthology.orchestra.run.smithi003.stdout: util-linux-user x86_64 2.32.1-43.el8 baseos 103 k 2023-10-03T10:07:01.744 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2023-10-03T10:07:01.744 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-common x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 24 M 2023-10-03T10:07:01.744 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-grafana-dashboards noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 24 k 2023-10-03T10:07:01.744 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mds x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 2.4 M 2023-10-03T10:07:01.744 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 244 k 2023-10-03T10:07:01.744 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mon x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 5.0 M 2023-10-03T10:07:01.745 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-osd x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 18 M 2023-10-03T10:07:01.745 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-prometheus-alerts noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 15 k 2023-10-03T10:07:01.745 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-selinux x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 26 k 2023-10-03T10:07:01.745 INFO:teuthology.orchestra.run.smithi003.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-10-03T10:07:01.745 INFO:teuthology.orchestra.run.smithi003.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-03T10:07:01.745 INFO:teuthology.orchestra.run.smithi003.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-10-03T10:07:01.745 INFO:teuthology.orchestra.run.smithi003.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2023-10-03T10:07:01.745 INFO:teuthology.orchestra.run.smithi003.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2023-10-03T10:07:01.745 INFO:teuthology.orchestra.run.smithi003.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2023-10-03T10:07:01.745 INFO:teuthology.orchestra.run.smithi003.stdout: libcephsqlite x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 174 k 2023-10-03T10:07:01.745 INFO:teuthology.orchestra.run.smithi003.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2023-10-03T10:07:01.745 INFO:teuthology.orchestra.run.smithi003.stdout: libgfortran x86_64 8.5.0-20.el8 baseos 645 k 2023-10-03T10:07:01.746 INFO:teuthology.orchestra.run.smithi003.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-03T10:07:01.746 INFO:teuthology.orchestra.run.smithi003.stdout: libquadmath x86_64 8.5.0-20.el8 baseos 172 k 2023-10-03T10:07:01.746 INFO:teuthology.orchestra.run.smithi003.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-10-03T10:07:01.746 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 493 k 2023-10-03T10:07:01.746 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-10-03T10:07:01.746 INFO:teuthology.orchestra.run.smithi003.stdout: librgw2 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 3.8 M 2023-10-03T10:07:01.746 INFO:teuthology.orchestra.run.smithi003.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2023-10-03T10:07:01.746 INFO:teuthology.orchestra.run.smithi003.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2023-10-03T10:07:01.746 INFO:teuthology.orchestra.run.smithi003.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-03T10:07:01.746 INFO:teuthology.orchestra.run.smithi003.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2023-10-03T10:07:01.746 INFO:teuthology.orchestra.run.smithi003.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2023-10-03T10:07:01.746 INFO:teuthology.orchestra.run.smithi003.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-10-03T10:07:01.747 INFO:teuthology.orchestra.run.smithi003.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-10-03T10:07:01.747 INFO:teuthology.orchestra.run.smithi003.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-10-03T10:07:01.747 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-10-03T10:07:01.747 INFO:teuthology.orchestra.run.smithi003.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2023-10-03T10:07:01.747 INFO:teuthology.orchestra.run.smithi003.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2023-10-03T10:07:01.747 INFO:teuthology.orchestra.run.smithi003.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2023-10-03T10:07:01.747 INFO:teuthology.orchestra.run.smithi003.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-03T10:07:01.747 INFO:teuthology.orchestra.run.smithi003.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-03T10:07:01.747 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-03T10:07:01.747 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 45 k 2023-10-03T10:07:01.747 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 99 k 2023-10-03T10:07:01.747 INFO:teuthology.orchestra.run.smithi003.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-03T10:07:01.747 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-10-03T10:07:01.748 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-03T10:07:01.748 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-03T10:07:01.748 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-10-03T10:07:01.748 INFO:teuthology.orchestra.run.smithi003.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-03T10:07:01.748 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-03T10:07:01.748 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-03T10:07:01.748 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-10-03T10:07:01.748 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-10-03T10:07:01.748 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-10-03T10:07:01.748 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-03T10:07:01.748 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-03T10:07:01.748 INFO:teuthology.orchestra.run.smithi003.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-10-03T10:07:01.749 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-10-03T10:07:01.749 INFO:teuthology.orchestra.run.smithi003.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-03T10:07:01.749 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-10-03T10:07:01.749 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-10-03T10:07:01.749 INFO:teuthology.orchestra.run.smithi003.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-10-03T10:07:01.749 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-03T10:07:01.749 INFO:teuthology.orchestra.run.smithi003.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-03T10:07:01.749 INFO:teuthology.orchestra.run.smithi003.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-10-03T10:07:01.749 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-10-03T10:07:01.749 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-10-03T10:07:01.749 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-10-03T10:07:01.749 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-10-03T10:07:01.749 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2023-10-03T10:07:01.750 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-03T10:07:01.750 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-10-03T10:07:01.750 INFO:teuthology.orchestra.run.smithi003.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-03T10:07:01.750 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-03T10:07:01.750 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-10-03T10:07:01.750 INFO:teuthology.orchestra.run.smithi003.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-03T10:07:01.750 INFO:teuthology.orchestra.run.smithi003.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-03T10:07:01.750 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-03T10:07:01.750 INFO:teuthology.orchestra.run.smithi003.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-03T10:07:01.750 INFO:teuthology.orchestra.run.smithi003.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-03T10:07:01.750 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-03T10:07:01.750 INFO:teuthology.orchestra.run.smithi003.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-03T10:07:01.751 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-03T10:07:01.751 INFO:teuthology.orchestra.run.smithi003.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-10-03T10:07:01.751 INFO:teuthology.orchestra.run.smithi003.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-03T10:07:01.751 INFO:teuthology.orchestra.run.smithi003.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-10-03T10:07:01.751 INFO:teuthology.orchestra.run.smithi003.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-10-03T10:07:01.751 INFO:teuthology.orchestra.run.smithi003.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2023-10-03T10:07:01.751 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:07:01.751 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2023-10-03T10:07:01.751 INFO:teuthology.orchestra.run.smithi003.stdout:==================================================================================================== 2023-10-03T10:07:01.751 INFO:teuthology.orchestra.run.smithi003.stdout:Install 108 Packages 2023-10-03T10:07:01.751 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrade 14 Packages 2023-10-03T10:07:01.751 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:07:01.752 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 200 M 2023-10-03T10:07:01.752 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2023-10-03T10:07:02.757 INFO:teuthology.orchestra.run.smithi003.stdout:(1/122): ceph-16.2.14-44.geeb13ac9.el8.x86_64.r 22 kB/s | 6.4 kB 00:00 2023-10-03T10:07:03.508 INFO:teuthology.orchestra.run.smithi003.stdout:(2/122): ceph-fuse-16.2.14-44.geeb13ac9.el8.x86 1.2 MB/s | 910 kB 00:00 2023-10-03T10:07:04.308 INFO:teuthology.orchestra.run.smithi003.stdout:(3/122): ceph-mds-16.2.14-44.geeb13ac9.el8.x86_ 3.1 MB/s | 2.4 MB 00:00 2023-10-03T10:07:04.392 INFO:teuthology.orchestra.run.smithi003.stdout:(4/122): ceph-base-16.2.14-44.geeb13ac9.el8.x86 3.2 MB/s | 6.1 MB 00:01 2023-10-03T10:07:04.684 INFO:teuthology.orchestra.run.smithi003.stdout:(5/122): ceph-mgr-16.2.14-44.geeb13ac9.el8.x86_ 4.3 MB/s | 1.6 MB 00:00 2023-10-03T10:07:05.050 INFO:teuthology.orchestra.run.smithi003.stdout:(6/122): ceph-mon-16.2.14-44.geeb13ac9.el8.x86_ 7.6 MB/s | 5.0 MB 00:00 2023-10-03T10:07:05.836 INFO:teuthology.orchestra.run.smithi022.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-03T10:07:05.837 INFO:teuthology.orchestra.run.smithi022.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-03T10:07:05.864 INFO:teuthology.orchestra.run.smithi022.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2023-10-03T10:07:05.864 INFO:teuthology.orchestra.run.smithi022.stdout:Package NetworkManager-initscripts-updown-1:1.40.16-4.el8.noarch is already installed. 2023-10-03T10:07:05.865 INFO:teuthology.orchestra.run.smithi022.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2023-10-03T10:07:05.867 INFO:teuthology.orchestra.run.smithi022.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2023-10-03T10:07:06.101 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2023-10-03T10:07:06.114 INFO:teuthology.orchestra.run.smithi022.stdout:==================================================================================================== 2023-10-03T10:07:06.114 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2023-10-03T10:07:06.114 INFO:teuthology.orchestra.run.smithi022.stdout:==================================================================================================== 2023-10-03T10:07:06.114 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2023-10-03T10:07:06.114 INFO:teuthology.orchestra.run.smithi022.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2023-10-03T10:07:06.114 INFO:teuthology.orchestra.run.smithi022.stdout: ceph x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 6.4 k 2023-10-03T10:07:06.115 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 6.1 M 2023-10-03T10:07:06.115 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-fuse x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 910 k 2023-10-03T10:07:06.115 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 1.6 M 2023-10-03T10:07:06.115 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 3.1 M 2023-10-03T10:07:06.115 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 8.6 M 2023-10-03T10:07:06.115 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-rook noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 27 k 2023-10-03T10:07:06.115 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-radosgw x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 12 M 2023-10-03T10:07:06.115 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-test x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 52 M 2023-10-03T10:07:06.115 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 baseos 62 k 2023-10-03T10:07:06.115 INFO:teuthology.orchestra.run.smithi022.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2023-10-03T10:07:06.115 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs-devel x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 25 k 2023-10-03T10:07:06.116 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs2 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 824 k 2023-10-03T10:07:06.116 INFO:teuthology.orchestra.run.smithi022.stdout: librados-devel x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 131 k 2023-10-03T10:07:06.116 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 baseos 3.2 M 2023-10-03T10:07:06.116 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cephfs x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 215 k 2023-10-03T10:07:06.116 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rados x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 390 k 2023-10-03T10:07:06.116 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rbd x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 369 k 2023-10-03T10:07:06.116 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rgw x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 114 k 2023-10-03T10:07:06.116 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-fuse x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 88 k 2023-10-03T10:07:06.116 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-mirror x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 3.7 M 2023-10-03T10:07:06.116 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-nbd x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 173 k 2023-10-03T10:07:06.117 INFO:teuthology.orchestra.run.smithi022.stdout:Upgrading: 2023-10-03T10:07:06.117 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 baseos 143 k 2023-10-03T10:07:06.117 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-debuginfod-client x86_64 0.189-3.el8 baseos 76 k 2023-10-03T10:07:06.117 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libelf x86_64 0.189-3.el8 baseos 232 k 2023-10-03T10:07:06.117 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libs x86_64 0.189-3.el8 baseos 303 k 2023-10-03T10:07:06.117 INFO:teuthology.orchestra.run.smithi022.stdout: iproute x86_64 6.2.0-5.el8 baseos 854 k 2023-10-03T10:07:06.117 INFO:teuthology.orchestra.run.smithi022.stdout: libblkid x86_64 2.32.1-43.el8 baseos 220 k 2023-10-03T10:07:06.117 INFO:teuthology.orchestra.run.smithi022.stdout: libfdisk x86_64 2.32.1-43.el8 baseos 254 k 2023-10-03T10:07:06.117 INFO:teuthology.orchestra.run.smithi022.stdout: libmount x86_64 2.32.1-43.el8 baseos 236 k 2023-10-03T10:07:06.117 INFO:teuthology.orchestra.run.smithi022.stdout: librados2 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 3.7 M 2023-10-03T10:07:06.117 INFO:teuthology.orchestra.run.smithi022.stdout: librbd1 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 4.2 M 2023-10-03T10:07:06.118 INFO:teuthology.orchestra.run.smithi022.stdout: libsmartcols x86_64 2.32.1-43.el8 baseos 179 k 2023-10-03T10:07:06.118 INFO:teuthology.orchestra.run.smithi022.stdout: libuuid x86_64 2.32.1-43.el8 baseos 99 k 2023-10-03T10:07:06.118 INFO:teuthology.orchestra.run.smithi022.stdout: util-linux x86_64 2.32.1-43.el8 baseos 2.5 M 2023-10-03T10:07:06.118 INFO:teuthology.orchestra.run.smithi022.stdout: util-linux-user x86_64 2.32.1-43.el8 baseos 103 k 2023-10-03T10:07:06.118 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2023-10-03T10:07:06.118 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-common x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 24 M 2023-10-03T10:07:06.118 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-grafana-dashboards noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 24 k 2023-10-03T10:07:06.118 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mds x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 2.4 M 2023-10-03T10:07:06.118 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-modules-core noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 244 k 2023-10-03T10:07:06.118 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mon x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 5.0 M 2023-10-03T10:07:06.119 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-osd x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 18 M 2023-10-03T10:07:06.119 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-prometheus-alerts noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 15 k 2023-10-03T10:07:06.119 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-selinux x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 26 k 2023-10-03T10:07:06.119 INFO:teuthology.orchestra.run.smithi022.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-10-03T10:07:06.119 INFO:teuthology.orchestra.run.smithi022.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-03T10:07:06.119 INFO:teuthology.orchestra.run.smithi022.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-10-03T10:07:06.119 INFO:teuthology.orchestra.run.smithi022.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2023-10-03T10:07:06.119 INFO:teuthology.orchestra.run.smithi022.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2023-10-03T10:07:06.119 INFO:teuthology.orchestra.run.smithi022.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2023-10-03T10:07:06.119 INFO:teuthology.orchestra.run.smithi022.stdout: libcephsqlite x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 174 k 2023-10-03T10:07:06.120 INFO:teuthology.orchestra.run.smithi022.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2023-10-03T10:07:06.120 INFO:teuthology.orchestra.run.smithi022.stdout: libgfortran x86_64 8.5.0-20.el8 baseos 645 k 2023-10-03T10:07:06.120 INFO:teuthology.orchestra.run.smithi022.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-03T10:07:06.120 INFO:teuthology.orchestra.run.smithi022.stdout: libquadmath x86_64 8.5.0-20.el8 baseos 172 k 2023-10-03T10:07:06.120 INFO:teuthology.orchestra.run.smithi022.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-10-03T10:07:06.120 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 493 k 2023-10-03T10:07:06.120 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-10-03T10:07:06.120 INFO:teuthology.orchestra.run.smithi022.stdout: librgw2 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 3.8 M 2023-10-03T10:07:06.120 INFO:teuthology.orchestra.run.smithi022.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2023-10-03T10:07:06.120 INFO:teuthology.orchestra.run.smithi022.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2023-10-03T10:07:06.120 INFO:teuthology.orchestra.run.smithi022.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-03T10:07:06.120 INFO:teuthology.orchestra.run.smithi022.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2023-10-03T10:07:06.121 INFO:teuthology.orchestra.run.smithi022.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2023-10-03T10:07:06.121 INFO:teuthology.orchestra.run.smithi022.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-10-03T10:07:06.121 INFO:teuthology.orchestra.run.smithi022.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-10-03T10:07:06.121 INFO:teuthology.orchestra.run.smithi022.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-10-03T10:07:06.121 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-10-03T10:07:06.121 INFO:teuthology.orchestra.run.smithi022.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2023-10-03T10:07:06.121 INFO:teuthology.orchestra.run.smithi022.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2023-10-03T10:07:06.121 INFO:teuthology.orchestra.run.smithi022.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2023-10-03T10:07:06.121 INFO:teuthology.orchestra.run.smithi022.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-03T10:07:06.121 INFO:teuthology.orchestra.run.smithi022.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-03T10:07:06.121 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-03T10:07:06.121 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-argparse x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 45 k 2023-10-03T10:07:06.122 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-common x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 99 k 2023-10-03T10:07:06.122 INFO:teuthology.orchestra.run.smithi022.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-03T10:07:06.122 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-10-03T10:07:06.122 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-03T10:07:06.122 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-03T10:07:06.122 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-10-03T10:07:06.122 INFO:teuthology.orchestra.run.smithi022.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-03T10:07:06.122 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-03T10:07:06.122 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-03T10:07:06.122 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-10-03T10:07:06.122 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-10-03T10:07:06.122 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-10-03T10:07:06.123 INFO:teuthology.orchestra.run.smithi022.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-03T10:07:06.123 INFO:teuthology.orchestra.run.smithi022.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-03T10:07:06.123 INFO:teuthology.orchestra.run.smithi022.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-10-03T10:07:06.123 INFO:teuthology.orchestra.run.smithi022.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-10-03T10:07:06.123 INFO:teuthology.orchestra.run.smithi022.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-03T10:07:06.123 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-10-03T10:07:06.123 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-10-03T10:07:06.123 INFO:teuthology.orchestra.run.smithi022.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-10-03T10:07:06.123 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-03T10:07:06.123 INFO:teuthology.orchestra.run.smithi022.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-03T10:07:06.123 INFO:teuthology.orchestra.run.smithi022.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-10-03T10:07:06.123 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-10-03T10:07:06.123 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-10-03T10:07:06.124 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-10-03T10:07:06.124 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-10-03T10:07:06.124 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2023-10-03T10:07:06.124 INFO:teuthology.orchestra.run.smithi022.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-03T10:07:06.124 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-10-03T10:07:06.124 INFO:teuthology.orchestra.run.smithi022.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-03T10:07:06.124 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-03T10:07:06.124 INFO:teuthology.orchestra.run.smithi022.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-10-03T10:07:06.124 INFO:teuthology.orchestra.run.smithi022.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-03T10:07:06.124 INFO:teuthology.orchestra.run.smithi022.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-03T10:07:06.124 INFO:teuthology.orchestra.run.smithi022.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-03T10:07:06.124 INFO:teuthology.orchestra.run.smithi022.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-03T10:07:06.125 INFO:teuthology.orchestra.run.smithi022.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-03T10:07:06.125 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-03T10:07:06.125 INFO:teuthology.orchestra.run.smithi022.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-03T10:07:06.125 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-03T10:07:06.125 INFO:teuthology.orchestra.run.smithi022.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-10-03T10:07:06.125 INFO:teuthology.orchestra.run.smithi022.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-03T10:07:06.125 INFO:teuthology.orchestra.run.smithi022.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-10-03T10:07:06.125 INFO:teuthology.orchestra.run.smithi022.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-10-03T10:07:06.125 INFO:teuthology.orchestra.run.smithi022.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2023-10-03T10:07:06.125 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:07:06.125 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2023-10-03T10:07:06.125 INFO:teuthology.orchestra.run.smithi022.stdout:==================================================================================================== 2023-10-03T10:07:06.126 INFO:teuthology.orchestra.run.smithi022.stdout:Install 108 Packages 2023-10-03T10:07:06.126 INFO:teuthology.orchestra.run.smithi022.stdout:Upgrade 14 Packages 2023-10-03T10:07:06.126 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:07:06.126 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 200 M 2023-10-03T10:07:06.126 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2023-10-03T10:07:06.251 INFO:teuthology.orchestra.run.smithi003.stdout:(7/122): ceph-radosgw-16.2.14-44.geeb13ac9.el8. 9.6 MB/s | 12 MB 00:01 2023-10-03T10:07:06.542 INFO:teuthology.orchestra.run.smithi003.stdout:(8/122): ceph-common-16.2.14-44.geeb13ac9.el8.x 5.9 MB/s | 24 MB 00:04 2023-10-03T10:07:06.559 INFO:teuthology.orchestra.run.smithi003.stdout:(9/122): ceph-selinux-16.2.14-44.geeb13ac9.el8. 83 kB/s | 26 kB 00:00 2023-10-03T10:07:06.593 INFO:teuthology.orchestra.run.smithi022.stdout:(1/122): ceph-16.2.14-44.geeb13ac9.el8.x86_64.r 35 kB/s | 6.4 kB 00:00 2023-10-03T10:07:06.618 INFO:teuthology.orchestra.run.smithi003.stdout:(10/122): libcephfs-devel-16.2.14-44.geeb13ac9. 438 kB/s | 25 kB 00:00 2023-10-03T10:07:06.927 INFO:teuthology.orchestra.run.smithi003.stdout:(11/122): libcephfs2-16.2.14-44.geeb13ac9.el8.x 2.6 MB/s | 824 kB 00:00 2023-10-03T10:07:07.035 INFO:teuthology.orchestra.run.smithi003.stdout:(12/122): libcephsqlite-16.2.14-44.geeb13ac9.el 1.6 MB/s | 174 kB 00:00 2023-10-03T10:07:07.119 INFO:teuthology.orchestra.run.smithi003.stdout:(13/122): librados-devel-16.2.14-44.geeb13ac9.e 1.5 MB/s | 131 kB 00:00 2023-10-03T10:07:07.328 INFO:teuthology.orchestra.run.smithi003.stdout:(14/122): ceph-osd-16.2.14-44.geeb13ac9.el8.x86 6.9 MB/s | 18 MB 00:02 2023-10-03T10:07:07.361 INFO:teuthology.orchestra.run.smithi003.stdout:(15/122): libradosstriper1-16.2.14-44.geeb13ac9 2.0 MB/s | 493 kB 00:00 2023-10-03T10:07:07.461 INFO:teuthology.orchestra.run.smithi003.stdout:(16/122): python3-ceph-argparse-16.2.14-44.geeb 451 kB/s | 45 kB 00:00 2023-10-03T10:07:07.562 INFO:teuthology.orchestra.run.smithi003.stdout:(17/122): python3-ceph-common-16.2.14-44.geeb13 986 kB/s | 99 kB 00:00 2023-10-03T10:07:07.568 INFO:teuthology.orchestra.run.smithi022.stdout:(2/122): ceph-base-16.2.14-44.geeb13ac9.el8.x86 5.2 MB/s | 6.1 MB 00:01 2023-10-03T10:07:07.635 INFO:teuthology.orchestra.run.smithi022.stdout:(3/122): ceph-fuse-16.2.14-44.geeb13ac9.el8.x86 874 kB/s | 910 kB 00:01 2023-10-03T10:07:07.670 INFO:teuthology.orchestra.run.smithi003.stdout:(18/122): python3-cephfs-16.2.14-44.geeb13ac9.e 1.9 MB/s | 215 kB 00:00 2023-10-03T10:07:08.004 INFO:teuthology.orchestra.run.smithi003.stdout:(19/122): librgw2-16.2.14-44.geeb13ac9.el8.x86_ 5.6 MB/s | 3.8 MB 00:00 2023-10-03T10:07:08.035 INFO:teuthology.orchestra.run.smithi022.stdout:(4/122): ceph-mds-16.2.14-44.geeb13ac9.el8.x86_ 5.2 MB/s | 2.4 MB 00:00 2023-10-03T10:07:08.054 INFO:teuthology.orchestra.run.smithi003.stdout:(20/122): python3-rados-16.2.14-44.geeb13ac9.el 1.0 MB/s | 390 kB 00:00 2023-10-03T10:07:08.121 INFO:teuthology.orchestra.run.smithi003.stdout:(21/122): python3-rbd-16.2.14-44.geeb13ac9.el8. 3.1 MB/s | 369 kB 00:00 2023-10-03T10:07:08.146 INFO:teuthology.orchestra.run.smithi003.stdout:(22/122): python3-rgw-16.2.14-44.geeb13ac9.el8. 1.2 MB/s | 114 kB 00:00 2023-10-03T10:07:08.205 INFO:teuthology.orchestra.run.smithi003.stdout:(23/122): rbd-fuse-16.2.14-44.geeb13ac9.el8.x86 1.0 MB/s | 88 kB 00:00 2023-10-03T10:07:08.473 INFO:teuthology.orchestra.run.smithi003.stdout:(24/122): rbd-nbd-16.2.14-44.geeb13ac9.el8.x86_ 651 kB/s | 173 kB 00:00 2023-10-03T10:07:08.581 INFO:teuthology.orchestra.run.smithi003.stdout:(25/122): ceph-grafana-dashboards-16.2.14-44.ge 225 kB/s | 24 kB 00:00 2023-10-03T10:07:08.652 INFO:teuthology.orchestra.run.smithi022.stdout:(5/122): ceph-mgr-16.2.14-44.geeb13ac9.el8.x86_ 1.6 MB/s | 1.6 MB 00:01 2023-10-03T10:07:09.053 INFO:teuthology.orchestra.run.smithi022.stdout:(6/122): ceph-mon-16.2.14-44.geeb13ac9.el8.x86_ 4.9 MB/s | 5.0 MB 00:01 2023-10-03T10:07:09.314 INFO:teuthology.orchestra.run.smithi003.stdout:(26/122): rbd-mirror-16.2.14-44.geeb13ac9.el8.x 3.1 MB/s | 3.7 MB 00:01 2023-10-03T10:07:10.298 INFO:teuthology.orchestra.run.smithi003.stdout:(27/122): ceph-mgr-dashboard-16.2.14-44.geeb13a 1.8 MB/s | 3.1 MB 00:01 2023-10-03T10:07:10.523 INFO:teuthology.orchestra.run.smithi003.stdout:(28/122): ceph-mgr-modules-core-16.2.14-44.geeb 1.1 MB/s | 244 kB 00:00 2023-10-03T10:07:10.624 INFO:teuthology.orchestra.run.smithi003.stdout:(29/122): ceph-mgr-rook-16.2.14-44.geeb13ac9.el 274 kB/s | 27 kB 00:00 2023-10-03T10:07:10.724 INFO:teuthology.orchestra.run.smithi003.stdout:(30/122): ceph-prometheus-alerts-16.2.14-44.gee 149 kB/s | 15 kB 00:00 2023-10-03T10:07:11.641 INFO:teuthology.orchestra.run.smithi003.stdout:(31/122): bison-3.0.4-10.el8.x86_64.rpm 751 kB/s | 688 kB 00:00 2023-10-03T10:07:11.791 INFO:teuthology.orchestra.run.smithi003.stdout:(32/122): flex-2.6.1-9.el8.x86_64.rpm 2.1 MB/s | 320 kB 00:00 2023-10-03T10:07:11.936 INFO:teuthology.orchestra.run.smithi022.stdout:(7/122): ceph-radosgw-16.2.14-44.geeb13ac9.el8. 4.0 MB/s | 12 MB 00:02 2023-10-03T10:07:11.958 INFO:teuthology.orchestra.run.smithi003.stdout:(33/122): jq-1.6-7.el8.x86_64.rpm 1.2 MB/s | 202 kB 00:00 2023-10-03T10:07:12.028 INFO:teuthology.orchestra.run.smithi022.stdout:(8/122): ceph-selinux-16.2.14-44.geeb13ac9.el8. 280 kB/s | 26 kB 00:00 2023-10-03T10:07:12.117 INFO:teuthology.orchestra.run.smithi003.stdout:(34/122): librdkafka-0.11.4-3.el8.x86_64.rpm 2.2 MB/s | 354 kB 00:00 2023-10-03T10:07:12.267 INFO:teuthology.orchestra.run.smithi003.stdout:(35/122): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.7 MB/s | 259 kB 00:00 2023-10-03T10:07:12.409 INFO:teuthology.orchestra.run.smithi003.stdout:(36/122): oniguruma-6.8.2-2.el8.x86_64.rpm 1.3 MB/s | 187 kB 00:00 2023-10-03T10:07:12.893 INFO:teuthology.orchestra.run.smithi003.stdout:(37/122): openblas-0.3.15-4.el8.x86_64.rpm 9.8 MB/s | 4.7 MB 00:00 2023-10-03T10:07:13.251 INFO:teuthology.orchestra.run.smithi003.stdout:(38/122): openblas-threads-0.3.15-4.el8.x86_64. 14 MB/s | 4.9 MB 00:00 2023-10-03T10:07:13.368 INFO:teuthology.orchestra.run.smithi003.stdout:(39/122): python3-jsonpatch-1.21-2.el8.noarch.r 228 kB/s | 27 kB 00:00 2023-10-03T10:07:13.485 INFO:teuthology.orchestra.run.smithi003.stdout:(40/122): python3-jsonpointer-1.10-11.el8.noarc 171 kB/s | 20 kB 00:00 2023-10-03T10:07:13.619 INFO:teuthology.orchestra.run.smithi003.stdout:(41/122): python3-mako-1.0.6-14.el8.noarch.rpm 1.2 MB/s | 157 kB 00:00 2023-10-03T10:07:13.761 INFO:teuthology.orchestra.run.smithi003.stdout:(42/122): python3-markupsafe-0.23-19.el8.x86_64 274 kB/s | 39 kB 00:00 2023-10-03T10:07:14.120 INFO:teuthology.orchestra.run.smithi003.stdout:(43/122): python3-numpy-1.14.3-10.el8.x86_64.rp 10 MB/s | 3.7 MB 00:00 2023-10-03T10:07:14.270 INFO:teuthology.orchestra.run.smithi003.stdout:(44/122): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.5 MB/s | 225 kB 00:00 2023-10-03T10:07:14.387 INFO:teuthology.orchestra.run.smithi003.stdout:(45/122): python3-prettytable-0.7.2-14.el8.noar 376 kB/s | 44 kB 00:00 2023-10-03T10:07:14.520 INFO:teuthology.orchestra.run.smithi003.stdout:(46/122): python3-pyOpenSSL-19.0.0-1.el8.noarch 777 kB/s | 103 kB 00:00 2023-10-03T10:07:14.636 INFO:teuthology.orchestra.run.smithi022.stdout:(9/122): ceph-common-16.2.14-44.geeb13ac9.el8.x 2.9 MB/s | 24 MB 00:08 2023-10-03T10:07:14.671 INFO:teuthology.orchestra.run.smithi003.stdout:(47/122): python3-pyasn1-0.3.7-6.el8.noarch.rpm 843 kB/s | 126 kB 00:00 2023-10-03T10:07:14.728 INFO:teuthology.orchestra.run.smithi022.stdout:(10/122): libcephfs-devel-16.2.14-44.geeb13ac9. 280 kB/s | 25 kB 00:00 2023-10-03T10:07:14.821 INFO:teuthology.orchestra.run.smithi003.stdout:(48/122): python3-pyasn1-modules-0.3.7-6.el8.no 734 kB/s | 110 kB 00:00 2023-10-03T10:07:14.963 INFO:teuthology.orchestra.run.smithi003.stdout:(49/122): ceph-mgr-diskprediction-local-16.2.14 1.5 MB/s | 8.6 MB 00:05 2023-10-03T10:07:14.988 INFO:teuthology.orchestra.run.smithi003.stdout:(50/122): python3-pytz-2017.2-11.el8.noarch.rpm 342 kB/s | 57 kB 00:00 2023-10-03T10:07:15.130 INFO:teuthology.orchestra.run.smithi003.stdout:(51/122): python3-werkzeug-0.12.2-4.el8.noarch. 3.2 MB/s | 457 kB 00:00 2023-10-03T10:07:15.270 INFO:teuthology.orchestra.run.smithi022.stdout:(11/122): libcephfs2-16.2.14-44.geeb13ac9.el8.x 1.5 MB/s | 824 kB 00:00 2023-10-03T10:07:15.356 INFO:teuthology.orchestra.run.smithi003.stdout:(52/122): socat-1.7.4.1-1.el8.x86_64.rpm 1.4 MB/s | 323 kB 00:00 2023-10-03T10:07:15.437 INFO:teuthology.orchestra.run.smithi022.stdout:(12/122): libcephsqlite-16.2.14-44.geeb13ac9.el 1.0 MB/s | 174 kB 00:00 2023-10-03T10:07:15.473 INFO:teuthology.orchestra.run.smithi003.stdout:(53/122): xmlstarlet-1.6.1-20.el8.x86_64.rpm 605 kB/s | 71 kB 00:00 2023-10-03T10:07:15.612 INFO:teuthology.orchestra.run.smithi022.stdout:(13/122): librados-devel-16.2.14-44.geeb13ac9.e 749 kB/s | 131 kB 00:00 2023-10-03T10:07:15.896 INFO:teuthology.orchestra.run.smithi022.stdout:(14/122): libradosstriper1-16.2.14-44.geeb13ac9 1.7 MB/s | 493 kB 00:00 2023-10-03T10:07:15.948 INFO:teuthology.orchestra.run.smithi003.stdout:(54/122): elfutils-libelf-devel-0.189-3.el8.x86 130 kB/s | 62 kB 00:00 2023-10-03T10:07:16.222 INFO:teuthology.orchestra.run.smithi003.stdout:(55/122): fuse-2.9.7-17.el8.x86_64.rpm 303 kB/s | 83 kB 00:00 2023-10-03T10:07:16.365 INFO:teuthology.orchestra.run.smithi003.stdout:(56/122): keyutils-libs-devel-1.5.10-9.el8.x86_ 338 kB/s | 48 kB 00:00 2023-10-03T10:07:16.532 INFO:teuthology.orchestra.run.smithi003.stdout:(57/122): python3-scipy-1.0.0-21.module_el8.5.0 8.7 MB/s | 14 MB 00:01 2023-10-03T10:07:16.691 INFO:teuthology.orchestra.run.smithi003.stdout:(58/122): krb5-devel-1.18.2-22.el8.x86_64.rpm 1.7 MB/s | 561 kB 00:00 2023-10-03T10:07:16.841 INFO:teuthology.orchestra.run.smithi003.stdout:(59/122): libcom_err-devel-1.45.6-5.el8.x86_64. 126 kB/s | 39 kB 00:00 2023-10-03T10:07:16.874 INFO:teuthology.orchestra.run.smithi003.stdout:(60/122): libgfortran-8.5.0-20.el8.x86_64.rpm 3.4 MB/s | 645 kB 00:00 2023-10-03T10:07:16.983 INFO:teuthology.orchestra.run.smithi003.stdout:(61/122): librabbitmq-0.9.0-4.el8.x86_64.rpm 433 kB/s | 47 kB 00:00 2023-10-03T10:07:17.092 INFO:teuthology.orchestra.run.smithi003.stdout:(62/122): libselinux-devel-2.9-8.el8.x86_64.rpm 1.8 MB/s | 200 kB 00:00 2023-10-03T10:07:17.142 INFO:teuthology.orchestra.run.smithi003.stdout:(63/122): libquadmath-8.5.0-20.el8.x86_64.rpm 573 kB/s | 172 kB 00:00 2023-10-03T10:07:17.201 INFO:teuthology.orchestra.run.smithi003.stdout:(64/122): libsepol-devel-2.9-3.el8.x86_64.rpm 800 kB/s | 87 kB 00:00 2023-10-03T10:07:17.243 INFO:teuthology.orchestra.run.smithi003.stdout:(65/122): libverto-devel-0.3.2-2.el8.x86_64.rpm 180 kB/s | 18 kB 00:00 2023-10-03T10:07:17.301 INFO:teuthology.orchestra.run.smithi003.stdout:(66/122): libzstd-devel-1.4.4-1.el8.x86_64.rpm 437 kB/s | 44 kB 00:00 2023-10-03T10:07:17.435 INFO:teuthology.orchestra.run.smithi003.stdout:(67/122): pcre2-devel-10.32-3.el8.x86_64.rpm 4.4 MB/s | 605 kB 00:00 2023-10-03T10:07:17.543 INFO:teuthology.orchestra.run.smithi003.stdout:(68/122): pcre2-utf16-10.32-3.el8.x86_64.rpm 2.1 MB/s | 229 kB 00:00 2023-10-03T10:07:17.660 INFO:teuthology.orchestra.run.smithi003.stdout:(69/122): pcre2-utf32-10.32-3.el8.x86_64.rpm 1.8 MB/s | 220 kB 00:00 2023-10-03T10:07:17.744 INFO:teuthology.orchestra.run.smithi003.stdout:(70/122): openssl-devel-1.1.1k-9.el8.x86_64.rpm 6.3 MB/s | 3.2 MB 00:00 2023-10-03T10:07:17.769 INFO:teuthology.orchestra.run.smithi003.stdout:(71/122): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.2 MB/s | 249 kB 00:00 2023-10-03T10:07:17.804 INFO:teuthology.orchestra.run.smithi022.stdout:(15/122): librgw2-16.2.14-44.geeb13ac9.el8.x86_ 2.0 MB/s | 3.8 MB 00:01 2023-10-03T10:07:17.886 INFO:teuthology.orchestra.run.smithi003.stdout:(72/122): python3-cryptography-3.2.1-6.el8.x86_ 4.4 MB/s | 641 kB 00:00 2023-10-03T10:07:17.896 INFO:teuthology.orchestra.run.smithi022.stdout:(16/122): python3-ceph-argparse-16.2.14-44.geeb 493 kB/s | 45 kB 00:00 2023-10-03T10:07:17.911 INFO:teuthology.orchestra.run.smithi003.stdout:(73/122): python3-jwt-1.6.1-2.el8.noarch.rpm 300 kB/s | 43 kB 00:00 2023-10-03T10:07:17.987 INFO:teuthology.orchestra.run.smithi003.stdout:(74/122): python3-oauthlib-2.1.0-1.el8.noarch.r 1.5 MB/s | 155 kB 00:00 2023-10-03T10:07:17.988 INFO:teuthology.orchestra.run.smithi022.stdout:(17/122): python3-ceph-common-16.2.14-44.geeb13 1.1 MB/s | 99 kB 00:00 2023-10-03T10:07:18.012 INFO:teuthology.orchestra.run.smithi003.stdout:(75/122): python3-pycparser-2.14-14.el8.noarch. 1.1 MB/s | 109 kB 00:00 2023-10-03T10:07:18.104 INFO:teuthology.orchestra.run.smithi003.stdout:(76/122): python3-requests-oauthlib-1.0.0-1.el8 365 kB/s | 43 kB 00:00 2023-10-03T10:07:18.129 INFO:teuthology.orchestra.run.smithi003.stdout:(77/122): zlib-devel-1.2.11-21.el8.x86_64.rpm 498 kB/s | 58 kB 00:00 2023-10-03T10:07:18.163 INFO:teuthology.orchestra.run.smithi022.stdout:(18/122): python3-cephfs-16.2.14-44.geeb13ac9.e 1.2 MB/s | 215 kB 00:00 2023-10-03T10:07:18.414 INFO:teuthology.orchestra.run.smithi022.stdout:(19/122): python3-rados-16.2.14-44.geeb13ac9.el 1.5 MB/s | 390 kB 00:00 2023-10-03T10:07:18.517 INFO:teuthology.orchestra.run.smithi003.stdout:(78/122): leveldb-1.22-1.el8.x86_64.rpm 467 kB/s | 181 kB 00:00 2023-10-03T10:07:18.546 INFO:teuthology.orchestra.run.smithi003.stdout:(79/122): gperftools-libs-2.7-9.el8.x86_64.rpm 693 kB/s | 306 kB 00:00 2023-10-03T10:07:18.655 INFO:teuthology.orchestra.run.smithi022.stdout:(20/122): python3-rbd-16.2.14-44.geeb13ac9.el8. 1.5 MB/s | 369 kB 00:00 2023-10-03T10:07:18.745 INFO:teuthology.orchestra.run.smithi003.stdout:(80/122): liboath-2.6.2-3.el8.x86_64.rpm 258 kB/s | 59 kB 00:00 2023-10-03T10:07:18.747 INFO:teuthology.orchestra.run.smithi022.stdout:(21/122): python3-rgw-16.2.14-44.geeb13ac9.el8. 1.2 MB/s | 114 kB 00:00 2023-10-03T10:07:18.780 INFO:teuthology.orchestra.run.smithi003.stdout:(81/122): libunwind-1.3.1-3.el8.x86_64.rpm 321 kB/s | 75 kB 00:00 2023-10-03T10:07:18.822 INFO:teuthology.orchestra.run.smithi003.stdout:(82/122): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 568 kB/s | 44 kB 00:00 2023-10-03T10:07:18.848 INFO:teuthology.orchestra.run.smithi022.stdout:(22/122): rbd-fuse-16.2.14-44.geeb13ac9.el8.x86 882 kB/s | 88 kB 00:00 2023-10-03T10:07:18.897 INFO:teuthology.orchestra.run.smithi003.stdout:(83/122): python3-beautifulsoup4-4.6.3-2.el8.1. 1.6 MB/s | 185 kB 00:00 2023-10-03T10:07:18.939 INFO:teuthology.orchestra.run.smithi003.stdout:(84/122): python3-cachetools-3.1.1-4.el8.noarch 282 kB/s | 33 kB 00:00 2023-10-03T10:07:19.189 INFO:teuthology.orchestra.run.smithi003.stdout:(85/122): python3-certifi-2018.10.15-7.el8.noar 56 kB/s | 16 kB 00:00 2023-10-03T10:07:19.265 INFO:teuthology.orchestra.run.smithi003.stdout:(86/122): python3-cheroot-8.5.2-1.el8.noarch.rp 530 kB/s | 173 kB 00:00 2023-10-03T10:07:19.374 INFO:teuthology.orchestra.run.smithi003.stdout:(87/122): python3-cherrypy-18.4.0-1.el8.noarch. 2.0 MB/s | 384 kB 00:00 2023-10-03T10:07:19.399 INFO:teuthology.orchestra.run.smithi003.stdout:(88/122): python3-google-auth-1.1.1-10.el8.noar 608 kB/s | 81 kB 00:00 2023-10-03T10:07:19.466 INFO:teuthology.orchestra.run.smithi003.stdout:(89/122): python3-jaraco-6.2-6.el8.noarch.rpm 121 kB/s | 11 kB 00:00 2023-10-03T10:07:19.483 INFO:teuthology.orchestra.run.smithi003.stdout:(90/122): python3-jaraco-functools-2.0-4.el8.no 210 kB/s | 18 kB 00:00 2023-10-03T10:07:19.567 INFO:teuthology.orchestra.run.smithi003.stdout:(91/122): python3-logutils-0.3.5-11.el8.noarch. 585 kB/s | 49 kB 00:00 2023-10-03T10:07:19.709 INFO:teuthology.orchestra.run.smithi003.stdout:(92/122): python3-kubernetes-11.0.0-6.el8.noarc 4.8 MB/s | 1.2 MB 00:00 2023-10-03T10:07:19.726 INFO:teuthology.orchestra.run.smithi003.stdout:(93/122): python3-more-itertools-7.2.0-3.el8.no 370 kB/s | 59 kB 00:00 2023-10-03T10:07:19.793 INFO:teuthology.orchestra.run.smithi003.stdout:(94/122): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2023-10-03T10:07:19.810 INFO:teuthology.orchestra.run.smithi003.stdout:(95/122): python3-portend-2.6-1.el8.noarch.rpm 186 kB/s | 16 kB 00:00 2023-10-03T10:07:19.860 INFO:teuthology.orchestra.run.smithi003.stdout:(96/122): python3-repoze-lru-0.7-6.el8.noarch.r 503 kB/s | 34 kB 00:00 2023-10-03T10:07:19.935 INFO:teuthology.orchestra.run.smithi003.stdout:(97/122): python3-routes-2.4.1-12.el8.noarch.rp 1.5 MB/s | 196 kB 00:00 2023-10-03T10:07:19.952 INFO:teuthology.orchestra.run.smithi003.stdout:(98/122): python3-rsa-4.9-2.el8.noarch.rpm 699 kB/s | 64 kB 00:00 2023-10-03T10:07:20.011 INFO:teuthology.orchestra.run.smithi003.stdout:(99/122): python3-simplegeneric-0.8.1-17.el8.no 249 kB/s | 19 kB 00:00 2023-10-03T10:07:20.036 INFO:teuthology.orchestra.run.smithi003.stdout:(100/122): python3-singledispatch-3.4.0.3-18.el 286 kB/s | 24 kB 00:00 2023-10-03T10:07:20.086 INFO:teuthology.orchestra.run.smithi003.stdout:(101/122): python3-tempora-1.14.1-5.el8.noarch. 389 kB/s | 29 kB 00:00 2023-10-03T10:07:20.145 INFO:teuthology.orchestra.run.smithi003.stdout:(102/122): python3-trustme-0.6.0-4.el8.noarch.r 254 kB/s | 27 kB 00:00 2023-10-03T10:07:20.195 INFO:teuthology.orchestra.run.smithi003.stdout:(103/122): python3-waitress-1.4.3-1.el8.noarch. 2.3 MB/s | 255 kB 00:00 2023-10-03T10:07:20.221 INFO:teuthology.orchestra.run.smithi003.stdout:(104/122): python3-webob-1.8.5-1.el8.1.noarch.r 3.3 MB/s | 251 kB 00:00 2023-10-03T10:07:20.271 INFO:teuthology.orchestra.run.smithi003.stdout:(105/122): python3-websocket-client-0.56.0-5.el 813 kB/s | 61 kB 00:00 2023-10-03T10:07:20.296 INFO:teuthology.orchestra.run.smithi003.stdout:(106/122): python3-webtest-2.0.33-1.el8.noarch. 1.1 MB/s | 85 kB 00:00 2023-10-03T10:07:20.371 INFO:teuthology.orchestra.run.smithi003.stdout:(107/122): python3-zc-lockfile-2.0-2.el8.noarch 229 kB/s | 23 kB 00:00 2023-10-03T10:07:20.698 INFO:teuthology.orchestra.run.smithi022.stdout:(23/122): rbd-mirror-16.2.14-44.geeb13ac9.el8.x 2.0 MB/s | 3.7 MB 00:01 2023-10-03T10:07:20.865 INFO:teuthology.orchestra.run.smithi022.stdout:(24/122): rbd-nbd-16.2.14-44.geeb13ac9.el8.x86_ 1.0 MB/s | 173 kB 00:00 2023-10-03T10:07:20.965 INFO:teuthology.orchestra.run.smithi022.stdout:(25/122): ceph-grafana-dashboards-16.2.14-44.ge 244 kB/s | 24 kB 00:00 2023-10-03T10:07:22.749 INFO:teuthology.orchestra.run.smithi022.stdout:(26/122): ceph-mgr-dashboard-16.2.14-44.geeb13a 1.8 MB/s | 3.1 MB 00:01 2023-10-03T10:07:23.171 INFO:teuthology.orchestra.run.smithi003.stdout:(108/122): ceph-test-16.2.14-44.geeb13ac9.el8.x 3.1 MB/s | 52 MB 00:16 2023-10-03T10:07:23.288 INFO:teuthology.orchestra.run.smithi003.stdout:(109/122): NetworkManager-initscripts-updown-1. 1.2 MB/s | 143 kB 00:00 2023-10-03T10:07:23.405 INFO:teuthology.orchestra.run.smithi003.stdout:(110/122): elfutils-debuginfod-client-0.189-3.e 653 kB/s | 76 kB 00:00 2023-10-03T10:07:23.514 INFO:teuthology.orchestra.run.smithi003.stdout:(111/122): elfutils-libelf-0.189-3.el8.x86_64.r 2.1 MB/s | 232 kB 00:00 2023-10-03T10:07:23.639 INFO:teuthology.orchestra.run.smithi003.stdout:(112/122): elfutils-libs-0.189-3.el8.x86_64.rpm 2.4 MB/s | 303 kB 00:00 2023-10-03T10:07:23.789 INFO:teuthology.orchestra.run.smithi003.stdout:(113/122): iproute-6.2.0-5.el8.x86_64.rpm 5.6 MB/s | 854 kB 00:00 2023-10-03T10:07:23.915 INFO:teuthology.orchestra.run.smithi003.stdout:(114/122): libblkid-2.32.1-43.el8.x86_64.rpm 1.7 MB/s | 220 kB 00:00 2023-10-03T10:07:24.058 INFO:teuthology.orchestra.run.smithi003.stdout:(115/122): libfdisk-2.32.1-43.el8.x86_64.rpm 1.7 MB/s | 254 kB 00:00 2023-10-03T10:07:24.182 INFO:teuthology.orchestra.run.smithi003.stdout:(116/122): libmount-2.32.1-43.el8.x86_64.rpm 1.9 MB/s | 236 kB 00:00 2023-10-03T10:07:24.324 INFO:teuthology.orchestra.run.smithi003.stdout:(117/122): libsmartcols-2.32.1-43.el8.x86_64.rp 1.2 MB/s | 179 kB 00:00 2023-10-03T10:07:24.432 INFO:teuthology.orchestra.run.smithi003.stdout:(118/122): libuuid-2.32.1-43.el8.x86_64.rpm 913 kB/s | 99 kB 00:00 2023-10-03T10:07:24.752 INFO:teuthology.orchestra.run.smithi003.stdout:(119/122): util-linux-2.32.1-43.el8.x86_64.rpm 7.8 MB/s | 2.5 MB 00:00 2023-10-03T10:07:24.858 INFO:teuthology.orchestra.run.smithi003.stdout:(120/122): util-linux-user-2.32.1-43.el8.x86_64 967 kB/s | 103 kB 00:00 2023-10-03T10:07:25.475 INFO:teuthology.orchestra.run.smithi003.stdout:(121/122): librados2-16.2.14-44.geeb13ac9.el8.x 741 kB/s | 3.7 MB 00:05 2023-10-03T10:07:26.891 INFO:teuthology.orchestra.run.smithi022.stdout:(27/122): ceph-mgr-diskprediction-local-16.2.14 2.1 MB/s | 8.6 MB 00:04 2023-10-03T10:07:27.149 INFO:teuthology.orchestra.run.smithi022.stdout:(28/122): ceph-mgr-modules-core-16.2.14-44.geeb 947 kB/s | 244 kB 00:00 2023-10-03T10:07:27.241 INFO:teuthology.orchestra.run.smithi022.stdout:(29/122): ceph-mgr-rook-16.2.14-44.geeb13ac9.el 299 kB/s | 27 kB 00:00 2023-10-03T10:07:27.333 INFO:teuthology.orchestra.run.smithi022.stdout:(30/122): ceph-prometheus-alerts-16.2.14-44.gee 162 kB/s | 15 kB 00:00 2023-10-03T10:07:28.150 INFO:teuthology.orchestra.run.smithi022.stdout:(31/122): bison-3.0.4-10.el8.x86_64.rpm 844 kB/s | 688 kB 00:00 2023-10-03T10:07:28.300 INFO:teuthology.orchestra.run.smithi022.stdout:(32/122): flex-2.6.1-9.el8.x86_64.rpm 2.1 MB/s | 320 kB 00:00 2023-10-03T10:07:28.442 INFO:teuthology.orchestra.run.smithi022.stdout:(33/122): jq-1.6-7.el8.x86_64.rpm 1.4 MB/s | 202 kB 00:00 2023-10-03T10:07:28.601 INFO:teuthology.orchestra.run.smithi022.stdout:(34/122): librdkafka-0.11.4-3.el8.x86_64.rpm 2.2 MB/s | 354 kB 00:00 2023-10-03T10:07:28.850 INFO:teuthology.orchestra.run.smithi003.stdout:(122/122): librbd1-16.2.14-44.geeb13ac9.el8.x86 505 kB/s | 4.2 MB 00:08 2023-10-03T10:07:28.853 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2023-10-03T10:07:28.853 INFO:teuthology.orchestra.run.smithi003.stdout:Total 7.4 MB/s | 200 MB 00:27 2023-10-03T10:07:28.893 INFO:teuthology.orchestra.run.smithi022.stdout:(35/122): ceph-osd-16.2.14-44.geeb13ac9.el8.x86 927 kB/s | 18 MB 00:20 2023-10-03T10:07:28.918 INFO:teuthology.orchestra.run.smithi022.stdout:(36/122): lttng-ust-2.8.1-11.el8.x86_64.rpm 819 kB/s | 259 kB 00:00 2023-10-03T10:07:29.443 INFO:teuthology.orchestra.run.smithi022.stdout:(37/122): openblas-0.3.15-4.el8.x86_64.rpm 9.0 MB/s | 4.7 MB 00:00 2023-10-03T10:07:29.543 INFO:teuthology.orchestra.run.smithi022.stdout:(38/122): oniguruma-6.8.2-2.el8.x86_64.rpm 288 kB/s | 187 kB 00:00 2023-10-03T10:07:29.618 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2023-10-03T10:07:29.694 INFO:teuthology.orchestra.run.smithi022.stdout:(39/122): python3-jsonpatch-1.21-2.el8.noarch.r 177 kB/s | 27 kB 00:00 2023-10-03T10:07:29.828 INFO:teuthology.orchestra.run.smithi022.stdout:(40/122): openblas-threads-0.3.15-4.el8.x86_64. 13 MB/s | 4.9 MB 00:00 2023-10-03T10:07:29.853 INFO:teuthology.orchestra.run.smithi022.stdout:(41/122): python3-jsonpointer-1.10-11.el8.noarc 126 kB/s | 20 kB 00:00 2023-10-03T10:07:29.873 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2023-10-03T10:07:29.873 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2023-10-03T10:07:29.953 INFO:teuthology.orchestra.run.smithi022.stdout:(42/122): python3-mako-1.0.6-14.el8.noarch.rpm 1.2 MB/s | 157 kB 00:00 2023-10-03T10:07:29.987 INFO:teuthology.orchestra.run.smithi022.stdout:(43/122): python3-markupsafe-0.23-19.el8.x86_64 290 kB/s | 39 kB 00:00 2023-10-03T10:07:30.295 INFO:teuthology.orchestra.run.smithi022.stdout:(44/122): python3-numpy-f2py-1.14.3-10.el8.x86_ 730 kB/s | 225 kB 00:00 2023-10-03T10:07:30.420 INFO:teuthology.orchestra.run.smithi022.stdout:(45/122): python3-numpy-1.14.3-10.el8.x86_64.rp 8.0 MB/s | 3.7 MB 00:00 2023-10-03T10:07:30.437 INFO:teuthology.orchestra.run.smithi022.stdout:(46/122): python3-prettytable-0.7.2-14.el8.noar 309 kB/s | 44 kB 00:00 2023-10-03T10:07:30.563 INFO:teuthology.orchestra.run.smithi022.stdout:(47/122): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.0 MB/s | 126 kB 00:00 2023-10-03T10:07:30.647 INFO:teuthology.orchestra.run.smithi022.stdout:(48/122): python3-pyOpenSSL-19.0.0-1.el8.noarch 458 kB/s | 103 kB 00:00 2023-10-03T10:07:30.681 INFO:teuthology.orchestra.run.smithi022.stdout:(49/122): python3-pyasn1-modules-0.3.7-6.el8.no 941 kB/s | 110 kB 00:00 2023-10-03T10:07:30.772 INFO:teuthology.orchestra.run.smithi022.stdout:(50/122): python3-pytz-2017.2-11.el8.noarch.rpm 455 kB/s | 57 kB 00:00 2023-10-03T10:07:30.973 INFO:teuthology.orchestra.run.smithi022.stdout:(51/122): python3-werkzeug-0.12.2-4.el8.noarch. 2.2 MB/s | 457 kB 00:00 2023-10-03T10:07:31.140 INFO:teuthology.orchestra.run.smithi022.stdout:(52/122): socat-1.7.4.1-1.el8.x86_64.rpm 1.9 MB/s | 323 kB 00:00 2023-10-03T10:07:31.273 INFO:teuthology.orchestra.run.smithi022.stdout:(53/122): xmlstarlet-1.6.1-20.el8.x86_64.rpm 530 kB/s | 71 kB 00:00 2023-10-03T10:07:31.366 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2023-10-03T10:07:31.370 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2023-10-03T10:07:31.415 INFO:teuthology.orchestra.run.smithi022.stdout:(54/122): python3-scipy-1.0.0-21.module_el8.5.0 19 MB/s | 14 MB 00:00 2023-10-03T10:07:31.774 INFO:teuthology.orchestra.run.smithi022.stdout:(55/122): elfutils-libelf-devel-0.189-3.el8.x86 123 kB/s | 62 kB 00:00 2023-10-03T10:07:31.816 INFO:teuthology.orchestra.run.smithi022.stdout:(56/122): fuse-2.9.7-17.el8.x86_64.rpm 207 kB/s | 83 kB 00:00 2023-10-03T10:07:31.916 INFO:teuthology.orchestra.run.smithi022.stdout:(57/122): keyutils-libs-devel-1.5.10-9.el8.x86_ 339 kB/s | 48 kB 00:00 2023-10-03T10:07:32.049 INFO:teuthology.orchestra.run.smithi022.stdout:(58/122): libcom_err-devel-1.45.6-5.el8.x86_64. 290 kB/s | 39 kB 00:00 2023-10-03T10:07:32.200 INFO:teuthology.orchestra.run.smithi022.stdout:(59/122): krb5-devel-1.18.2-22.el8.x86_64.rpm 1.4 MB/s | 561 kB 00:00 2023-10-03T10:07:32.325 INFO:teuthology.orchestra.run.smithi022.stdout:(60/122): libquadmath-8.5.0-20.el8.x86_64.rpm 1.4 MB/s | 172 kB 00:00 2023-10-03T10:07:32.458 INFO:teuthology.orchestra.run.smithi022.stdout:(61/122): librabbitmq-0.9.0-4.el8.x86_64.rpm 354 kB/s | 47 kB 00:00 2023-10-03T10:07:32.500 INFO:teuthology.orchestra.run.smithi022.stdout:(62/122): libgfortran-8.5.0-20.el8.x86_64.rpm 1.4 MB/s | 645 kB 00:00 2023-10-03T10:07:32.584 INFO:teuthology.orchestra.run.smithi022.stdout:(63/122): libselinux-devel-2.9-8.el8.x86_64.rpm 1.6 MB/s | 200 kB 00:00 2023-10-03T10:07:32.609 INFO:teuthology.orchestra.run.smithi022.stdout:(64/122): libsepol-devel-2.9-3.el8.x86_64.rpm 801 kB/s | 87 kB 00:00 2023-10-03T10:07:32.685 INFO:teuthology.orchestra.run.smithi022.stdout:(65/122): libverto-devel-0.3.2-2.el8.x86_64.rpm 180 kB/s | 18 kB 00:00 2023-10-03T10:07:32.710 INFO:teuthology.orchestra.run.smithi022.stdout:(66/122): libzstd-devel-1.4.4-1.el8.x86_64.rpm 437 kB/s | 44 kB 00:00 2023-10-03T10:07:32.902 INFO:teuthology.orchestra.run.smithi022.stdout:(67/122): pcre2-devel-10.32-3.el8.x86_64.rpm 3.1 MB/s | 605 kB 00:00 2023-10-03T10:07:33.027 INFO:teuthology.orchestra.run.smithi022.stdout:(68/122): pcre2-utf16-10.32-3.el8.x86_64.rpm 1.8 MB/s | 229 kB 00:00 2023-10-03T10:07:33.086 INFO:teuthology.orchestra.run.smithi022.stdout:(69/122): openssl-devel-1.1.1k-9.el8.x86_64.rpm 7.9 MB/s | 3.2 MB 00:00 2023-10-03T10:07:33.144 INFO:teuthology.orchestra.run.smithi022.stdout:(70/122): pcre2-utf32-10.32-3.el8.x86_64.rpm 1.8 MB/s | 220 kB 00:00 2023-10-03T10:07:33.195 INFO:teuthology.orchestra.run.smithi022.stdout:(71/122): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.2 MB/s | 249 kB 00:00 2023-10-03T10:07:33.328 INFO:teuthology.orchestra.run.smithi022.stdout:(72/122): python3-cryptography-3.2.1-6.el8.x86_ 3.4 MB/s | 641 kB 00:00 2023-10-03T10:07:33.354 INFO:teuthology.orchestra.run.smithi022.stdout:(73/122): python3-jwt-1.6.1-2.el8.noarch.rpm 269 kB/s | 43 kB 00:00 2023-10-03T10:07:33.437 INFO:teuthology.orchestra.run.smithi022.stdout:(74/122): python3-oauthlib-2.1.0-1.el8.noarch.r 1.4 MB/s | 155 kB 00:00 2023-10-03T10:07:33.463 INFO:teuthology.orchestra.run.smithi022.stdout:(75/122): python3-pycparser-2.14-14.el8.noarch. 1.0 MB/s | 109 kB 00:00 2023-10-03T10:07:33.538 INFO:teuthology.orchestra.run.smithi022.stdout:(76/122): python3-requests-oauthlib-1.0.0-1.el8 424 kB/s | 43 kB 00:00 2023-10-03T10:07:33.555 INFO:teuthology.orchestra.run.smithi022.stdout:(77/122): zlib-devel-1.2.11-21.el8.x86_64.rpm 632 kB/s | 58 kB 00:00 2023-10-03T10:07:33.830 INFO:teuthology.orchestra.run.smithi022.stdout:(78/122): leveldb-1.22-1.el8.x86_64.rpm 658 kB/s | 181 kB 00:00 2023-10-03T10:07:33.855 INFO:teuthology.orchestra.run.smithi022.stdout:(79/122): gperftools-libs-2.7-9.el8.x86_64.rpm 966 kB/s | 306 kB 00:00 2023-10-03T10:07:33.889 INFO:teuthology.orchestra.run.smithi022.stdout:(80/122): liboath-2.6.2-3.el8.x86_64.rpm 1.0 MB/s | 59 kB 00:00 2023-10-03T10:07:33.923 INFO:teuthology.orchestra.run.smithi022.stdout:(81/122): libunwind-1.3.1-3.el8.x86_64.rpm 1.1 MB/s | 75 kB 00:00 2023-10-03T10:07:33.965 INFO:teuthology.orchestra.run.smithi022.stdout:(82/122): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 579 kB/s | 44 kB 00:00 2023-10-03T10:07:33.998 INFO:teuthology.orchestra.run.smithi022.stdout:(83/122): python3-beautifulsoup4-4.6.3-2.el8.1. 2.4 MB/s | 185 kB 00:00 2023-10-03T10:07:34.065 INFO:teuthology.orchestra.run.smithi022.stdout:(84/122): python3-cachetools-3.1.1-4.el8.noarch 330 kB/s | 33 kB 00:00 2023-10-03T10:07:34.091 INFO:teuthology.orchestra.run.smithi022.stdout:(85/122): python3-certifi-2018.10.15-7.el8.noar 177 kB/s | 16 kB 00:00 2023-10-03T10:07:34.166 INFO:teuthology.orchestra.run.smithi022.stdout:(86/122): python3-cheroot-8.5.2-1.el8.noarch.rp 1.7 MB/s | 173 kB 00:00 2023-10-03T10:07:34.208 INFO:teuthology.orchestra.run.smithi022.stdout:(87/122): python3-cherrypy-18.4.0-1.el8.noarch. 3.2 MB/s | 384 kB 00:00 2023-10-03T10:07:34.250 INFO:teuthology.orchestra.run.smithi022.stdout:(88/122): python3-google-auth-1.1.1-10.el8.noar 974 kB/s | 81 kB 00:00 2023-10-03T10:07:34.275 INFO:teuthology.orchestra.run.smithi022.stdout:(89/122): python3-jaraco-6.2-6.el8.noarch.rpm 166 kB/s | 11 kB 00:00 2023-10-03T10:07:34.317 INFO:teuthology.orchestra.run.smithi022.stdout:(90/122): python3-jaraco-functools-2.0-4.el8.no 262 kB/s | 18 kB 00:00 2023-10-03T10:07:34.593 INFO:teuthology.orchestra.run.smithi022.stdout:(91/122): python3-kubernetes-11.0.0-6.el8.noarc 3.7 MB/s | 1.2 MB 00:00 2023-10-03T10:07:34.676 INFO:teuthology.orchestra.run.smithi022.stdout:(92/122): python3-more-itertools-7.2.0-3.el8.no 706 kB/s | 59 kB 00:00 2023-10-03T10:07:34.868 INFO:teuthology.orchestra.run.smithi022.stdout:(93/122): python3-logutils-0.3.5-11.el8.noarch. 88 kB/s | 49 kB 00:00 2023-10-03T10:07:34.926 INFO:teuthology.orchestra.run.smithi022.stdout:(94/122): python3-portend-2.6-1.el8.noarch.rpm 267 kB/s | 16 kB 00:00 2023-10-03T10:07:35.035 INFO:teuthology.orchestra.run.smithi022.stdout:(95/122): python3-repoze-lru-0.7-6.el8.noarch.r 311 kB/s | 34 kB 00:00 2023-10-03T10:07:35.177 INFO:teuthology.orchestra.run.smithi022.stdout:(96/122): python3-routes-2.4.1-12.el8.noarch.rp 1.4 MB/s | 196 kB 00:00 2023-10-03T10:07:35.277 INFO:teuthology.orchestra.run.smithi022.stdout:(97/122): python3-pecan-1.3.2-9.el8.noarch.rpm 470 kB/s | 283 kB 00:00 2023-10-03T10:07:35.302 INFO:teuthology.orchestra.run.smithi022.stdout:(98/122): python3-rsa-4.9-2.el8.noarch.rpm 513 kB/s | 64 kB 00:00 2023-10-03T10:07:35.595 INFO:teuthology.orchestra.run.smithi022.stdout:(99/122): python3-singledispatch-3.4.0.3-18.el8 82 kB/s | 24 kB 00:00 2023-10-03T10:07:35.619 INFO:teuthology.orchestra.run.smithi022.stdout:(100/122): python3-simplegeneric-0.8.1-17.el8.n 55 kB/s | 19 kB 00:00 2023-10-03T10:07:35.670 INFO:teuthology.orchestra.run.smithi022.stdout:(101/122): python3-tempora-1.14.1-5.el8.noarch. 388 kB/s | 29 kB 00:00 2023-10-03T10:07:35.695 INFO:teuthology.orchestra.run.smithi022.stdout:(102/122): python3-trustme-0.6.0-4.el8.noarch.r 365 kB/s | 27 kB 00:00 2023-10-03T10:07:35.779 INFO:teuthology.orchestra.run.smithi022.stdout:(103/122): python3-webob-1.8.5-1.el8.1.noarch.r 2.9 MB/s | 251 kB 00:00 2023-10-03T10:07:35.804 INFO:teuthology.orchestra.run.smithi022.stdout:(104/122): python3-waitress-1.4.3-1.el8.noarch. 1.9 MB/s | 255 kB 00:00 2023-10-03T10:07:35.880 INFO:teuthology.orchestra.run.smithi022.stdout:(105/122): python3-websocket-client-0.56.0-5.el 610 kB/s | 61 kB 00:00 2023-10-03T10:07:35.905 INFO:teuthology.orchestra.run.smithi022.stdout:(106/122): python3-webtest-2.0.33-1.el8.noarch. 851 kB/s | 85 kB 00:00 2023-10-03T10:07:35.963 INFO:teuthology.orchestra.run.smithi022.stdout:(107/122): python3-zc-lockfile-2.0-2.el8.noarch 275 kB/s | 23 kB 00:00 2023-10-03T10:07:36.599 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2023-10-03T10:07:36.643 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: liboath-2.6.2-3.el8.x86_64 1/1 2023-10-03T10:07:36.860 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/136 2023-10-03T10:07:37.097 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : leveldb-1.22-1.el8.x86_64 2/136 2023-10-03T10:07:37.267 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/136 2023-10-03T10:07:37.272 INFO:teuthology.orchestra.run.smithi022.stdout:(108/122): ceph-test-16.2.14-44.geeb13ac9.el8.x 2.0 MB/s | 52 MB 00:25 2023-10-03T10:07:37.346 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/136 2023-10-03T10:07:37.389 INFO:teuthology.orchestra.run.smithi022.stdout:(109/122): NetworkManager-initscripts-updown-1. 1.2 MB/s | 143 kB 00:00 2023-10-03T10:07:37.498 INFO:teuthology.orchestra.run.smithi022.stdout:(110/122): elfutils-debuginfod-client-0.189-3.e 704 kB/s | 76 kB 00:00 2023-10-03T10:07:37.623 INFO:teuthology.orchestra.run.smithi022.stdout:(111/122): elfutils-libelf-0.189-3.el8.x86_64.r 1.8 MB/s | 232 kB 00:00 2023-10-03T10:07:37.659 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/136 2023-10-03T10:07:37.756 INFO:teuthology.orchestra.run.smithi022.stdout:(112/122): elfutils-libs-0.189-3.el8.x86_64.rpm 2.2 MB/s | 303 kB 00:00 2023-10-03T10:07:37.759 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/136 2023-10-03T10:07:37.924 INFO:teuthology.orchestra.run.smithi022.stdout:(113/122): iproute-6.2.0-5.el8.x86_64.rpm 5.0 MB/s | 854 kB 00:00 2023-10-03T10:07:37.948 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 5/136 2023-10-03T10:07:38.041 INFO:teuthology.orchestra.run.smithi022.stdout:(114/122): libblkid-2.32.1-43.el8.x86_64.rpm 1.8 MB/s | 220 kB 00:00 2023-10-03T10:07:38.166 INFO:teuthology.orchestra.run.smithi022.stdout:(115/122): libfdisk-2.32.1-43.el8.x86_64.rpm 2.0 MB/s | 254 kB 00:00 2023-10-03T10:07:38.173 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/136 2023-10-03T10:07:38.283 INFO:teuthology.orchestra.run.smithi022.stdout:(116/122): libmount-2.32.1-43.el8.x86_64.rpm 2.0 MB/s | 236 kB 00:00 2023-10-03T10:07:38.376 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/136 2023-10-03T10:07:38.391 INFO:teuthology.orchestra.run.smithi022.stdout:(117/122): libsmartcols-2.32.1-43.el8.x86_64.rp 1.6 MB/s | 179 kB 00:00 2023-10-03T10:07:38.531 INFO:teuthology.orchestra.run.smithi022.stdout:(118/122): libuuid-2.32.1-43.el8.x86_64.rpm 711 kB/s | 99 kB 00:00 2023-10-03T10:07:38.800 INFO:teuthology.orchestra.run.smithi022.stdout:(119/122): util-linux-2.32.1-43.el8.x86_64.rpm 9.2 MB/s | 2.5 MB 00:00 2023-10-03T10:07:38.820 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/136 2023-10-03T10:07:38.900 INFO:teuthology.orchestra.run.smithi022.stdout:(120/122): util-linux-user-2.32.1-43.el8.x86_64 1.0 MB/s | 103 kB 00:00 2023-10-03T10:07:38.984 INFO:teuthology.orchestra.run.smithi022.stdout:(121/122): librbd1-16.2.14-44.geeb13ac9.el8.x86 1.4 MB/s | 4.2 MB 00:03 2023-10-03T10:07:39.009 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 8/136 2023-10-03T10:07:39.449 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 8/136 2023-10-03T10:07:39.642 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 9/136 2023-10-03T10:07:39.775 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 9/136 2023-10-03T10:07:39.918 INFO:teuthology.orchestra.run.smithi022.stdout:(122/122): librados2-16.2.14-44.geeb13ac9.el8.x 956 kB/s | 3.7 MB 00:04 2023-10-03T10:07:39.920 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2023-10-03T10:07:39.920 INFO:teuthology.orchestra.run.smithi022.stdout:Total 5.9 MB/s | 200 MB 00:33 2023-10-03T10:07:39.967 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 10/136 2023-10-03T10:07:40.107 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 10/136 2023-10-03T10:07:40.336 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 11/136 2023-10-03T10:07:40.525 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 12/136 2023-10-03T10:07:40.695 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2023-10-03T10:07:40.943 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 12/136 2023-10-03T10:07:40.955 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2023-10-03T10:07:40.955 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2023-10-03T10:07:41.084 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 13/136 2023-10-03T10:07:41.187 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 13/136 2023-10-03T10:07:41.399 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 14/136 2023-10-03T10:07:41.567 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 15/136 2023-10-03T10:07:41.717 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 16/136 2023-10-03T10:07:41.814 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 16/136 2023-10-03T10:07:41.975 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_ 17/136 2023-10-03T10:07:42.095 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_ 17/136 2023-10-03T10:07:42.266 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 18/136 2023-10-03T10:07:42.404 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2023-10-03T10:07:42.408 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2023-10-03T10:07:42.469 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 19/136 2023-10-03T10:07:42.608 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 20/136 2023-10-03T10:07:42.749 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 20/136 2023-10-03T10:07:42.892 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 21/136 2023-10-03T10:07:43.582 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 21/136 2023-10-03T10:07:43.800 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openblas-0.3.15-4.el8.x86_64 22/136 2023-10-03T10:07:44.516 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 22/136 2023-10-03T10:07:44.683 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 23/136 2023-10-03T10:07:45.462 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 23/136 2023-10-03T10:07:46.090 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 24/136 2023-10-03T10:07:46.371 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 25/136 2023-10-03T10:07:46.607 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x 26/136 2023-10-03T10:07:46.798 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8 27/136 2023-10-03T10:07:47.058 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 28/136 2023-10-03T10:07:47.229 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2023-10-03T10:07:47.273 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: liboath-2.6.2-3.el8.x86_64 1/1 2023-10-03T10:07:47.283 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/136 2023-10-03T10:07:47.508 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/136 2023-10-03T10:07:47.706 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : leveldb-1.22-1.el8.x86_64 2/136 2023-10-03T10:07:47.896 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/136 2023-10-03T10:07:47.980 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/136 2023-10-03T10:07:48.147 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/136 2023-10-03T10:07:48.254 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/136 2023-10-03T10:07:48.469 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 5/136 2023-10-03T10:07:48.670 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/136 2023-10-03T10:07:48.854 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/136 2023-10-03T10:07:49.259 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/136 2023-10-03T10:07:49.282 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/136 2023-10-03T10:07:49.471 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 8/136 2023-10-03T10:07:49.570 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 30/136 2023-10-03T10:07:49.774 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 31/136 2023-10-03T10:07:49.944 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 8/136 2023-10-03T10:07:50.004 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rsa-4.9-2.el8.noarch 32/136 2023-10-03T10:07:50.137 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 9/136 2023-10-03T10:07:50.260 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 9/136 2023-10-03T10:07:50.410 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 33/136 2023-10-03T10:07:50.579 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 10/136 2023-10-03T10:07:50.580 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 34/136 2023-10-03T10:07:50.675 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 10/136 2023-10-03T10:07:50.881 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 35/136 2023-10-03T10:07:50.948 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 11/136 2023-10-03T10:07:51.069 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 36/136 2023-10-03T10:07:51.112 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 12/136 2023-10-03T10:07:51.234 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : elfutils-debuginfod-client-0.189-3.el8.x86_64 37/136 2023-10-03T10:07:51.457 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 38/136 2023-10-03T10:07:51.528 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 12/136 2023-10-03T10:07:51.582 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 38/136 2023-10-03T10:07:51.704 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 13/136 2023-10-03T10:07:51.764 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 13/136 2023-10-03T10:07:51.790 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 39/136 2023-10-03T10:07:51.870 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 39/136 2023-10-03T10:07:51.986 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 14/136 2023-10-03T10:07:52.049 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 40/136 2023-10-03T10:07:52.329 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 15/136 2023-10-03T10:07:52.462 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 16/136 2023-10-03T10:07:52.557 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 16/136 2023-10-03T10:07:52.666 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 40/136 2023-10-03T10:07:52.729 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_ 17/136 2023-10-03T10:07:52.847 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_ 17/136 2023-10-03T10:07:52.941 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 41/136 2023-10-03T10:07:52.959 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 41/136 2023-10-03T10:07:53.061 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 18/136 2023-10-03T10:07:53.174 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 42/136 2023-10-03T10:07:53.222 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 19/136 2023-10-03T10:07:53.336 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 20/136 2023-10-03T10:07:53.446 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 43/136 2023-10-03T10:07:53.468 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 20/136 2023-10-03T10:07:53.612 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 21/136 2023-10-03T10:07:53.690 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 44/136 2023-10-03T10:07:53.887 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 45/136 2023-10-03T10:07:54.070 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 46/136 2023-10-03T10:07:54.246 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 47/136 2023-10-03T10:07:54.254 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 21/136 2023-10-03T10:07:54.445 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openblas-0.3.15-4.el8.x86_64 22/136 2023-10-03T10:07:54.462 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 48/136 2023-10-03T10:07:54.645 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 49/136 2023-10-03T10:07:54.830 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 50/136 2023-10-03T10:07:55.040 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 51/136 2023-10-03T10:07:55.108 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 22/136 2023-10-03T10:07:55.241 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 52/136 2023-10-03T10:07:55.261 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 23/136 2023-10-03T10:07:55.495 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 53/136 2023-10-03T10:07:55.760 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 54/136 2023-10-03T10:07:55.970 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 55/136 2023-10-03T10:07:56.030 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 23/136 2023-10-03T10:07:56.175 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 56/136 2023-10-03T10:07:56.364 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 57/136 2023-10-03T10:07:56.393 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 24/136 2023-10-03T10:07:56.576 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 58/136 2023-10-03T10:07:56.623 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 25/136 2023-10-03T10:07:56.801 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 59/136 2023-10-03T10:07:56.835 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x 26/136 2023-10-03T10:07:56.996 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8 27/136 2023-10-03T10:07:57.198 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 60/136 2023-10-03T10:07:57.230 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 28/136 2023-10-03T10:07:57.428 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/136 2023-10-03T10:07:57.534 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 61/136 2023-10-03T10:07:57.755 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/136 2023-10-03T10:07:58.033 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 63/136 2023-10-03T10:07:58.283 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 64/136 2023-10-03T10:07:59.433 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/136 2023-10-03T10:07:59.486 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 65/136 2023-10-03T10:07:59.716 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 30/136 2023-10-03T10:07:59.908 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 66/136 2023-10-03T10:07:59.919 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 31/136 2023-10-03T10:08:00.165 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 67/136 2023-10-03T10:08:00.166 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rsa-4.9-2.el8.noarch 32/136 2023-10-03T10:08:00.401 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 68/136 2023-10-03T10:08:00.407 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 33/136 2023-10-03T10:08:00.634 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 34/136 2023-10-03T10:08:00.669 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 69/136 2023-10-03T10:08:00.871 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 35/136 2023-10-03T10:08:00.895 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 70/136 2023-10-03T10:08:01.101 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 36/136 2023-10-03T10:08:01.238 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 71/136 2023-10-03T10:08:01.279 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : elfutils-debuginfod-client-0.189-3.el8.x86_64 37/136 2023-10-03T10:08:01.428 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 72/136 2023-10-03T10:08:01.527 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 38/136 2023-10-03T10:08:01.609 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 38/136 2023-10-03T10:08:01.624 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 73/136 2023-10-03T10:08:01.794 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 39/136 2023-10-03T10:08:01.816 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 74/136 2023-10-03T10:08:01.897 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 39/136 2023-10-03T10:08:02.052 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 40/136 2023-10-03T10:08:02.136 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 75/136 2023-10-03T10:08:02.419 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 76/136 2023-10-03T10:08:02.570 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 77/136 2023-10-03T10:08:02.583 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 40/136 2023-10-03T10:08:02.821 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 78/136 2023-10-03T10:08:02.853 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 41/136 2023-10-03T10:08:02.871 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 41/136 2023-10-03T10:08:03.041 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 79/136 2023-10-03T10:08:03.219 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 42/136 2023-10-03T10:08:03.242 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : fuse-2.9.7-17.el8.x86_64 80/136 2023-10-03T10:08:03.484 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 43/136 2023-10-03T10:08:03.639 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 81/136 2023-10-03T10:08:03.752 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 44/136 2023-10-03T10:08:03.831 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 82/136 2023-10-03T10:08:03.949 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 45/136 2023-10-03T10:08:03.986 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 83/136 2023-10-03T10:08:04.165 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 46/136 2023-10-03T10:08:04.178 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 84/136 2023-10-03T10:08:04.382 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 47/136 2023-10-03T10:08:04.526 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-portend-2.6-1.el8.noarch 85/136 2023-10-03T10:08:04.616 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 48/136 2023-10-03T10:08:04.790 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 86/136 2023-10-03T10:08:04.797 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 49/136 2023-10-03T10:08:04.972 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 87/136 2023-10-03T10:08:05.001 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 50/136 2023-10-03T10:08:05.169 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 51/136 2023-10-03T10:08:05.403 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 52/136 2023-10-03T10:08:05.632 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 53/136 2023-10-03T10:08:05.914 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 54/136 2023-10-03T10:08:06.358 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 55/136 2023-10-03T10:08:06.662 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 56/136 2023-10-03T10:08:06.890 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 57/136 2023-10-03T10:08:07.155 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 58/136 2023-10-03T10:08:07.388 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 59/136 2023-10-03T10:08:07.772 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 88/136 2023-10-03T10:08:07.831 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 60/136 2023-10-03T10:08:08.013 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 88/136 2023-10-03T10:08:08.120 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 61/136 2023-10-03T10:08:08.325 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/136 2023-10-03T10:08:08.612 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 63/136 2023-10-03T10:08:08.793 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 88/136 2023-10-03T10:08:08.862 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 64/136 2023-10-03T10:08:09.109 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 89/136 2023-10-03T10:08:09.238 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 89/136 2023-10-03T10:08:09.412 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 90/136 2023-10-03T10:08:10.043 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 65/136 2023-10-03T10:08:10.294 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 66/136 2023-10-03T10:08:10.544 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 67/136 2023-10-03T10:08:10.754 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 68/136 2023-10-03T10:08:11.014 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 69/136 2023-10-03T10:08:11.215 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 70/136 2023-10-03T10:08:11.553 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 71/136 2023-10-03T10:08:11.781 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 72/136 2023-10-03T10:08:11.953 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 73/136 2023-10-03T10:08:12.111 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 74/136 2023-10-03T10:08:12.442 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 75/136 2023-10-03T10:08:12.697 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 76/136 2023-10-03T10:08:12.841 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 77/136 2023-10-03T10:08:13.067 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 78/136 2023-10-03T10:08:13.270 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 79/136 2023-10-03T10:08:13.454 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : fuse-2.9.7-17.el8.x86_64 80/136 2023-10-03T10:08:13.727 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 81/136 2023-10-03T10:08:13.918 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 82/136 2023-10-03T10:08:14.123 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 83/136 2023-10-03T10:08:14.323 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 84/136 2023-10-03T10:08:14.672 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-portend-2.6-1.el8.noarch 85/136 2023-10-03T10:08:15.052 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 86/136 2023-10-03T10:08:15.200 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 87/136 2023-10-03T10:08:17.960 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 88/136 2023-10-03T10:08:18.326 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 88/136 2023-10-03T10:08:19.103 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 88/136 2023-10-03T10:08:19.516 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 89/136 2023-10-03T10:08:19.640 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 89/136 2023-10-03T10:08:19.791 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 90/136 2023-10-03T10:08:30.529 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 90/136 2023-10-03T10:08:30.529 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /sys 2023-10-03T10:08:30.529 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /proc 2023-10-03T10:08:30.529 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /dev 2023-10-03T10:08:30.529 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /run 2023-10-03T10:08:30.529 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /mnt 2023-10-03T10:08:30.529 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /var/tmp 2023-10-03T10:08:30.529 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /home 2023-10-03T10:08:30.529 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /tmp 2023-10-03T10:08:30.530 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /dev 2023-10-03T10:08:30.530 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:08:30.959 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 91/136 2023-10-03T10:08:31.452 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 91/136 2023-10-03T10:08:31.609 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 92/136 2023-10-03T10:08:33.247 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 92/136 2023-10-03T10:08:33.483 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 93/136 2023-10-03T10:08:33.526 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 93/136 2023-10-03T10:08:33.797 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 94/136 2023-10-03T10:08:34.158 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 95/136 2023-10-03T10:08:34.450 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 96/136 2023-10-03T10:08:34.676 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 97/136 2023-10-03T10:08:34.914 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 98/136 2023-10-03T10:08:35.059 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 99/136 2023-10-03T10:08:35.174 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 99/136 2023-10-03T10:08:35.379 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : jq-1.6-7.el8.x86_64 100/136 2023-10-03T10:08:35.499 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el 101/136 2023-10-03T10:08:36.668 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.e 102/136 2023-10-03T10:08:36.962 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.no 103/136 2023-10-03T10:08:38.116 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.no 103/136 2023-10-03T10:08:38.408 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.14-44.geeb1 104/136 2023-10-03T10:08:38.515 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.14-44.geeb1 104/136 2023-10-03T10:08:38.952 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8 105/136 2023-10-03T10:08:39.145 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 106/136 2023-10-03T10:08:39.177 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 106/136 2023-10-03T10:08:39.333 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 107/136 2023-10-03T10:08:39.347 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 107/136 2023-10-03T10:08:41.525 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 90/136 2023-10-03T10:08:41.526 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /sys 2023-10-03T10:08:41.526 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /proc 2023-10-03T10:08:41.526 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /dev 2023-10-03T10:08:41.526 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /run 2023-10-03T10:08:41.526 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /mnt 2023-10-03T10:08:41.526 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /var/tmp 2023-10-03T10:08:41.526 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /home 2023-10-03T10:08:41.526 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /tmp 2023-10-03T10:08:41.526 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /dev 2023-10-03T10:08:41.527 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:08:42.000 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 91/136 2023-10-03T10:08:42.473 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 91/136 2023-10-03T10:08:42.671 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 92/136 2023-10-03T10:08:44.313 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 92/136 2023-10-03T10:08:44.486 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 108/136 2023-10-03T10:08:44.562 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 93/136 2023-10-03T10:08:44.605 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 93/136 2023-10-03T10:08:44.835 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 94/136 2023-10-03T10:08:45.184 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 95/136 2023-10-03T10:08:45.471 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 96/136 2023-10-03T10:08:45.779 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 97/136 2023-10-03T10:08:45.812 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 109/136 2023-10-03T10:08:46.042 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 98/136 2023-10-03T10:08:46.069 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 110/136 2023-10-03T10:08:46.236 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 99/136 2023-10-03T10:08:46.309 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 99/136 2023-10-03T10:08:46.490 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : jq-1.6-7.el8.x86_64 100/136 2023-10-03T10:08:46.506 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 110/136 2023-10-03T10:08:46.740 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 111/136 2023-10-03T10:08:46.753 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el 101/136 2023-10-03T10:08:46.845 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 111/136 2023-10-03T10:08:47.812 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 112/136 2023-10-03T10:08:47.893 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.e 102/136 2023-10-03T10:08:48.079 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 113/136 2023-10-03T10:08:48.185 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.no 103/136 2023-10-03T10:08:48.307 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 114/136 2023-10-03T10:08:48.494 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 115/136 2023-10-03T10:08:48.632 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_6 116/136 2023-10-03T10:08:48.857 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 117/136 2023-10-03T10:08:49.190 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 118/136 2023-10-03T10:08:49.342 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.no 103/136 2023-10-03T10:08:49.476 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : iproute-6.2.0-5.el8.x86_64 119/136 2023-10-03T10:08:49.511 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.14-44.geeb1 104/136 2023-10-03T10:08:49.614 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.14-44.geeb1 104/136 2023-10-03T10:08:49.648 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-9.el 120/136 2023-10-03T10:08:49.720 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 120/136 2023-10-03T10:08:49.964 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flex-2.6.1-9.el8.x86_64 121/136 2023-10-03T10:08:50.017 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8 105/136 2023-10-03T10:08:50.061 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 121/136 2023-10-03T10:08:50.210 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 106/136 2023-10-03T10:08:50.241 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 106/136 2023-10-03T10:08:50.306 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : bison-3.0.4-10.el8.x86_64 122/136 2023-10-03T10:08:50.321 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 122/136 2023-10-03T10:08:50.377 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 107/136 2023-10-03T10:08:50.392 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 107/136 2023-10-03T10:08:50.466 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 123/136 2023-10-03T10:08:50.637 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 124/136 2023-10-03T10:08:50.637 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 125/136 2023-10-03T10:08:50.858 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 125/136 2023-10-03T10:08:50.864 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-4.el 126/136 2023-10-03T10:08:51.018 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.16-4.el 126/136 2023-10-03T10:08:51.018 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 127/136 2023-10-03T10:08:51.272 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 127/136 2023-10-03T10:08:51.477 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 128/136 2023-10-03T10:08:51.618 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 129/136 2023-10-03T10:08:51.841 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 130/136 2023-10-03T10:08:51.841 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 131/136 2023-10-03T10:08:52.092 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 131/136 2023-10-03T10:08:52.092 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 132/136 2023-10-03T10:08:52.300 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 132/136 2023-10-03T10:08:52.300 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 133/136 2023-10-03T10:08:52.525 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 133/136 2023-10-03T10:08:52.525 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 134/136 2023-10-03T10:08:52.758 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 134/136 2023-10-03T10:08:52.759 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 135/136 2023-10-03T10:08:52.959 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 135/136 2023-10-03T10:08:53.150 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 136/136 2023-10-03T10:08:53.784 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: elfutils-libelf-0.189-1.el8.x86_64 136/136 2023-10-03T10:08:53.785 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 1/136 2023-10-03T10:08:53.785 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 2/136 2023-10-03T10:08:53.785 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 3/136 2023-10-03T10:08:53.785 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 4/136 2023-10-03T10:08:53.785 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 5/136 2023-10-03T10:08:53.785 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 6/136 2023-10-03T10:08:53.785 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 7/136 2023-10-03T10:08:53.785 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 8/136 2023-10-03T10:08:53.785 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 9/136 2023-10-03T10:08:53.785 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 10/136 2023-10-03T10:08:53.786 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 11/136 2023-10-03T10:08:53.786 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_6 12/136 2023-10-03T10:08:53.786 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 13/136 2023-10-03T10:08:53.786 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 14/136 2023-10-03T10:08:53.786 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 15/136 2023-10-03T10:08:53.786 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_ 16/136 2023-10-03T10:08:53.786 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 17/136 2023-10-03T10:08:53.786 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8 18/136 2023-10-03T10:08:53.786 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x 19/136 2023-10-03T10:08:53.786 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 20/136 2023-10-03T10:08:53.786 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 21/136 2023-10-03T10:08:53.786 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 22/136 2023-10-03T10:08:53.787 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 23/136 2023-10-03T10:08:53.787 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 24/136 2023-10-03T10:08:53.787 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 25/136 2023-10-03T10:08:53.787 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 26/136 2023-10-03T10:08:53.787 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.e 27/136 2023-10-03T10:08:53.796 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.no 28/136 2023-10-03T10:08:53.796 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.14-44.geeb1 29/136 2023-10-03T10:08:53.796 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8 30/136 2023-10-03T10:08:53.796 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 31/136 2023-10-03T10:08:53.796 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el 32/136 2023-10-03T10:08:53.796 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : bison-3.0.4-10.el8.x86_64 33/136 2023-10-03T10:08:53.796 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flex-2.6.1-9.el8.x86_64 34/136 2023-10-03T10:08:53.796 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : jq-1.6-7.el8.x86_64 35/136 2023-10-03T10:08:53.797 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 36/136 2023-10-03T10:08:53.797 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 37/136 2023-10-03T10:08:53.797 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 38/136 2023-10-03T10:08:53.797 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 39/136 2023-10-03T10:08:53.797 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 40/136 2023-10-03T10:08:53.797 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 41/136 2023-10-03T10:08:53.797 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 42/136 2023-10-03T10:08:53.797 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 43/136 2023-10-03T10:08:53.797 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 44/136 2023-10-03T10:08:53.797 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 45/136 2023-10-03T10:08:53.797 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 46/136 2023-10-03T10:08:53.797 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 47/136 2023-10-03T10:08:53.798 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 48/136 2023-10-03T10:08:53.798 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 49/136 2023-10-03T10:08:53.798 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 50/136 2023-10-03T10:08:53.798 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 51/136 2023-10-03T10:08:53.798 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 52/136 2023-10-03T10:08:53.798 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 53/136 2023-10-03T10:08:53.798 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 54/136 2023-10-03T10:08:53.798 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 55/136 2023-10-03T10:08:53.798 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 56/136 2023-10-03T10:08:53.798 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 57/136 2023-10-03T10:08:53.798 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 58/136 2023-10-03T10:08:53.798 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 59/136 2023-10-03T10:08:53.798 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 60/136 2023-10-03T10:08:53.799 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 61/136 2023-10-03T10:08:53.799 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 62/136 2023-10-03T10:08:53.799 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 63/136 2023-10-03T10:08:53.799 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 64/136 2023-10-03T10:08:53.799 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 65/136 2023-10-03T10:08:53.799 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 66/136 2023-10-03T10:08:53.799 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 67/136 2023-10-03T10:08:53.799 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 68/136 2023-10-03T10:08:53.799 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 69/136 2023-10-03T10:08:53.799 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 70/136 2023-10-03T10:08:53.799 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 71/136 2023-10-03T10:08:53.799 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 72/136 2023-10-03T10:08:53.800 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 73/136 2023-10-03T10:08:53.800 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/136 2023-10-03T10:08:53.800 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 75/136 2023-10-03T10:08:53.800 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 76/136 2023-10-03T10:08:53.800 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 77/136 2023-10-03T10:08:53.800 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 78/136 2023-10-03T10:08:53.800 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 79/136 2023-10-03T10:08:53.800 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : leveldb-1.22-1.el8.x86_64 80/136 2023-10-03T10:08:53.800 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 81/136 2023-10-03T10:08:53.800 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 82/136 2023-10-03T10:08:53.800 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/136 2023-10-03T10:08:53.800 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 84/136 2023-10-03T10:08:53.801 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 85/136 2023-10-03T10:08:53.801 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 86/136 2023-10-03T10:08:53.801 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 87/136 2023-10-03T10:08:53.801 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 88/136 2023-10-03T10:08:53.801 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 89/136 2023-10-03T10:08:53.801 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 90/136 2023-10-03T10:08:53.801 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 91/136 2023-10-03T10:08:53.801 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 92/136 2023-10-03T10:08:53.801 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 93/136 2023-10-03T10:08:53.801 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 94/136 2023-10-03T10:08:53.801 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 95/136 2023-10-03T10:08:53.802 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-portend-2.6-1.el8.noarch 96/136 2023-10-03T10:08:53.802 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 97/136 2023-10-03T10:08:53.802 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 98/136 2023-10-03T10:08:53.802 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 99/136 2023-10-03T10:08:53.802 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 100/136 2023-10-03T10:08:53.802 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 101/136 2023-10-03T10:08:53.802 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 102/136 2023-10-03T10:08:53.802 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 103/136 2023-10-03T10:08:53.802 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 104/136 2023-10-03T10:08:53.802 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 105/136 2023-10-03T10:08:53.802 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 106/136 2023-10-03T10:08:53.802 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 107/136 2023-10-03T10:08:53.803 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 108/136 2023-10-03T10:08:53.803 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 109/136 2023-10-03T10:08:53.803 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 110/136 2023-10-03T10:08:53.803 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 111/136 2023-10-03T10:08:53.803 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 112/136 2023-10-03T10:08:53.803 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 113/136 2023-10-03T10:08:53.803 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-4.el 114/136 2023-10-03T10:08:53.803 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el8.x86_64 115/136 2023-10-03T10:08:53.803 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 116/136 2023-10-03T10:08:53.803 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 117/136 2023-10-03T10:08:53.803 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 118/136 2023-10-03T10:08:53.803 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 119/136 2023-10-03T10:08:53.804 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 120/136 2023-10-03T10:08:53.804 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : iproute-6.2.0-5.el8.x86_64 121/136 2023-10-03T10:08:53.804 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 122/136 2023-10-03T10:08:53.804 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 123/136 2023-10-03T10:08:53.804 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 124/136 2023-10-03T10:08:53.804 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 125/136 2023-10-03T10:08:53.804 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 126/136 2023-10-03T10:08:53.804 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 127/136 2023-10-03T10:08:53.804 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 128/136 2023-10-03T10:08:53.804 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 129/136 2023-10-03T10:08:53.804 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 130/136 2023-10-03T10:08:53.805 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 131/136 2023-10-03T10:08:53.805 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 132/136 2023-10-03T10:08:53.805 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 133/136 2023-10-03T10:08:53.805 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 134/136 2023-10-03T10:08:53.805 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 135/136 2023-10-03T10:08:55.520 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 108/136 2023-10-03T10:08:55.993 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 136/136 2023-10-03T10:08:55.993 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:08:56.108 INFO:teuthology.orchestra.run.smithi003.stdout:Upgraded: 2023-10-03T10:08:56.108 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2023-10-03T10:08:56.108 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-debuginfod-client-0.189-3.el8.x86_64 2023-10-03T10:08:56.108 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-10-03T10:08:56.108 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-10-03T10:08:56.109 INFO:teuthology.orchestra.run.smithi003.stdout: iproute-6.2.0-5.el8.x86_64 2023-10-03T10:08:56.109 INFO:teuthology.orchestra.run.smithi003.stdout: libblkid-2.32.1-43.el8.x86_64 2023-10-03T10:08:56.109 INFO:teuthology.orchestra.run.smithi003.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-10-03T10:08:56.109 INFO:teuthology.orchestra.run.smithi003.stdout: libmount-2.32.1-43.el8.x86_64 2023-10-03T10:08:56.109 INFO:teuthology.orchestra.run.smithi003.stdout: librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:08:56.109 INFO:teuthology.orchestra.run.smithi003.stdout: librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:08:56.109 INFO:teuthology.orchestra.run.smithi003.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-10-03T10:08:56.109 INFO:teuthology.orchestra.run.smithi003.stdout: libuuid-2.32.1-43.el8.x86_64 2023-10-03T10:08:56.109 INFO:teuthology.orchestra.run.smithi003.stdout: util-linux-2.32.1-43.el8.x86_64 2023-10-03T10:08:56.109 INFO:teuthology.orchestra.run.smithi003.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-10-03T10:08:56.109 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2023-10-03T10:08:56.110 INFO:teuthology.orchestra.run.smithi003.stdout: bison-3.0.4-10.el8.x86_64 2023-10-03T10:08:56.110 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:08:56.110 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:08:56.110 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:08:56.110 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:08:56.110 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.el8.noarch 2023-10-03T10:08:56.110 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:08:56.110 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:08:56.110 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.noarch 2023-10-03T10:08:56.110 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-diskprediction-local-2:16.2.14-44.geeb13ac9.el8.noarch 2023-10-03T10:08:56.110 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8.noarch 2023-10-03T10:08:56.110 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 2023-10-03T10:08:56.111 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:08:56.111 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:08:56.111 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el8.noarch 2023-10-03T10:08:56.111 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:08:56.111 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:08:56.111 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:08:56.111 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-10-03T10:08:56.111 INFO:teuthology.orchestra.run.smithi003.stdout: flex-2.6.1-9.el8.x86_64 2023-10-03T10:08:56.111 INFO:teuthology.orchestra.run.smithi003.stdout: fuse-2.9.7-17.el8.x86_64 2023-10-03T10:08:56.111 INFO:teuthology.orchestra.run.smithi003.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-03T10:08:56.111 INFO:teuthology.orchestra.run.smithi003.stdout: jq-1.6-7.el8.x86_64 2023-10-03T10:08:56.111 INFO:teuthology.orchestra.run.smithi003.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-03T10:08:56.112 INFO:teuthology.orchestra.run.smithi003.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-10-03T10:08:56.112 INFO:teuthology.orchestra.run.smithi003.stdout: leveldb-1.22-1.el8.x86_64 2023-10-03T10:08:56.112 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:08:56.112 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:08:56.112 INFO:teuthology.orchestra.run.smithi003.stdout: libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:08:56.112 INFO:teuthology.orchestra.run.smithi003.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-10-03T10:08:56.112 INFO:teuthology.orchestra.run.smithi003.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-10-03T10:08:56.112 INFO:teuthology.orchestra.run.smithi003.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-03T10:08:56.112 INFO:teuthology.orchestra.run.smithi003.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-10-03T10:08:56.112 INFO:teuthology.orchestra.run.smithi003.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-10-03T10:08:56.112 INFO:teuthology.orchestra.run.smithi003.stdout: librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:08:56.113 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:08:56.113 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-03T10:08:56.113 INFO:teuthology.orchestra.run.smithi003.stdout: librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:08:56.113 INFO:teuthology.orchestra.run.smithi003.stdout: libselinux-devel-2.9-8.el8.x86_64 2023-10-03T10:08:56.113 INFO:teuthology.orchestra.run.smithi003.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-03T10:08:56.113 INFO:teuthology.orchestra.run.smithi003.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-03T10:08:56.113 INFO:teuthology.orchestra.run.smithi003.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-10-03T10:08:56.113 INFO:teuthology.orchestra.run.smithi003.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-10-03T10:08:56.113 INFO:teuthology.orchestra.run.smithi003.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-03T10:08:56.113 INFO:teuthology.orchestra.run.smithi003.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-03T10:08:56.113 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-0.3.15-4.el8.x86_64 2023-10-03T10:08:56.114 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-10-03T10:08:56.114 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-10-03T10:08:56.114 INFO:teuthology.orchestra.run.smithi003.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-10-03T10:08:56.114 INFO:teuthology.orchestra.run.smithi003.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-10-03T10:08:56.114 INFO:teuthology.orchestra.run.smithi003.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-10-03T10:08:56.114 INFO:teuthology.orchestra.run.smithi003.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-03T10:08:56.114 INFO:teuthology.orchestra.run.smithi003.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-03T10:08:56.114 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-03T10:08:56.114 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:08:56.114 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:08:56.114 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:08:56.114 INFO:teuthology.orchestra.run.smithi003.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-03T10:08:56.115 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-10-03T10:08:56.115 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-03T10:08:56.115 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-03T10:08:56.115 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-10-03T10:08:56.115 INFO:teuthology.orchestra.run.smithi003.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-03T10:08:56.115 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-03T10:08:56.115 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-03T10:08:56.115 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-03T10:08:56.115 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-03T10:08:56.115 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-03T10:08:56.115 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-03T10:08:56.115 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-03T10:08:56.116 INFO:teuthology.orchestra.run.smithi003.stdout: python3-mako-1.0.6-14.el8.noarch 2023-10-03T10:08:56.116 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-03T10:08:56.116 INFO:teuthology.orchestra.run.smithi003.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-03T10:08:56.116 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-03T10:08:56.116 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-03T10:08:56.116 INFO:teuthology.orchestra.run.smithi003.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-03T10:08:56.116 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-03T10:08:56.116 INFO:teuthology.orchestra.run.smithi003.stdout: python3-portend-2.6-1.el8.noarch 2023-10-03T10:08:56.116 INFO:teuthology.orchestra.run.smithi003.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-03T10:08:56.116 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-03T10:08:56.116 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-03T10:08:56.116 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-03T10:08:56.117 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-03T10:08:56.117 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytz-2017.2-11.el8.noarch 2023-10-03T10:08:56.117 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:08:56.117 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:08:56.117 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-03T10:08:56.117 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-03T10:08:56.117 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:08:56.117 INFO:teuthology.orchestra.run.smithi003.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-03T10:08:56.117 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-03T10:08:56.117 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-10-03T10:08:56.117 INFO:teuthology.orchestra.run.smithi003.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-03T10:08:56.117 INFO:teuthology.orchestra.run.smithi003.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-03T10:08:56.118 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-03T10:08:56.118 INFO:teuthology.orchestra.run.smithi003.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-03T10:08:56.118 INFO:teuthology.orchestra.run.smithi003.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-03T10:08:56.118 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-03T10:08:56.118 INFO:teuthology.orchestra.run.smithi003.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-03T10:08:56.118 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-03T10:08:56.118 INFO:teuthology.orchestra.run.smithi003.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-03T10:08:56.118 INFO:teuthology.orchestra.run.smithi003.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-03T10:08:56.118 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:08:56.118 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:08:56.118 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:08:56.118 INFO:teuthology.orchestra.run.smithi003.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-03T10:08:56.119 INFO:teuthology.orchestra.run.smithi003.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-03T10:08:56.119 INFO:teuthology.orchestra.run.smithi003.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-10-03T10:08:56.119 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:08:56.119 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2023-10-03T10:08:56.202 DEBUG:teuthology.parallel:result is None 2023-10-03T10:08:56.985 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 109/136 2023-10-03T10:08:57.193 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 110/136 2023-10-03T10:08:57.659 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 110/136 2023-10-03T10:08:57.851 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 111/136 2023-10-03T10:08:57.955 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 111/136 2023-10-03T10:08:58.953 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 112/136 2023-10-03T10:08:59.215 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 113/136 2023-10-03T10:08:59.433 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 114/136 2023-10-03T10:08:59.614 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 115/136 2023-10-03T10:08:59.810 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_6 116/136 2023-10-03T10:08:59.993 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 117/136 2023-10-03T10:09:00.453 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 118/136 2023-10-03T10:09:00.704 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : iproute-6.2.0-5.el8.x86_64 119/136 2023-10-03T10:09:00.842 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-9.el 120/136 2023-10-03T10:09:00.917 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 120/136 2023-10-03T10:09:01.151 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flex-2.6.1-9.el8.x86_64 121/136 2023-10-03T10:09:01.249 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 121/136 2023-10-03T10:09:01.451 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : bison-3.0.4-10.el8.x86_64 122/136 2023-10-03T10:09:01.466 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 122/136 2023-10-03T10:09:01.661 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 123/136 2023-10-03T10:09:01.821 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 124/136 2023-10-03T10:09:01.821 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 125/136 2023-10-03T10:09:02.070 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 125/136 2023-10-03T10:09:02.076 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-4.el 126/136 2023-10-03T10:09:02.204 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.16-4.el 126/136 2023-10-03T10:09:02.205 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 127/136 2023-10-03T10:09:02.450 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 127/136 2023-10-03T10:09:02.624 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 128/136 2023-10-03T10:09:02.802 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 129/136 2023-10-03T10:09:03.052 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 130/136 2023-10-03T10:09:03.052 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 131/136 2023-10-03T10:09:03.278 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 131/136 2023-10-03T10:09:03.278 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 132/136 2023-10-03T10:09:03.486 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 132/136 2023-10-03T10:09:03.487 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 133/136 2023-10-03T10:09:03.720 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 133/136 2023-10-03T10:09:03.720 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 134/136 2023-10-03T10:09:03.928 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 134/136 2023-10-03T10:09:03.928 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 135/136 2023-10-03T10:09:04.170 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 135/136 2023-10-03T10:09:04.352 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 136/136 2023-10-03T10:09:04.991 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: elfutils-libelf-0.189-1.el8.x86_64 136/136 2023-10-03T10:09:04.991 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 1/136 2023-10-03T10:09:04.991 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 2/136 2023-10-03T10:09:04.992 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 3/136 2023-10-03T10:09:04.992 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 4/136 2023-10-03T10:09:04.992 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 5/136 2023-10-03T10:09:04.992 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 6/136 2023-10-03T10:09:04.992 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 7/136 2023-10-03T10:09:04.992 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 8/136 2023-10-03T10:09:04.992 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 9/136 2023-10-03T10:09:04.992 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 10/136 2023-10-03T10:09:04.992 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 11/136 2023-10-03T10:09:04.992 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_6 12/136 2023-10-03T10:09:04.992 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 13/136 2023-10-03T10:09:04.993 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 14/136 2023-10-03T10:09:04.993 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 15/136 2023-10-03T10:09:04.993 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_ 16/136 2023-10-03T10:09:04.993 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 17/136 2023-10-03T10:09:04.993 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8 18/136 2023-10-03T10:09:04.993 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x 19/136 2023-10-03T10:09:04.993 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 20/136 2023-10-03T10:09:04.993 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 21/136 2023-10-03T10:09:04.993 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 22/136 2023-10-03T10:09:04.993 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 23/136 2023-10-03T10:09:04.993 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 24/136 2023-10-03T10:09:04.993 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 25/136 2023-10-03T10:09:04.993 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 26/136 2023-10-03T10:09:04.994 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.e 27/136 2023-10-03T10:09:04.994 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.no 28/136 2023-10-03T10:09:04.994 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.14-44.geeb1 29/136 2023-10-03T10:09:04.994 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8 30/136 2023-10-03T10:09:04.994 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 31/136 2023-10-03T10:09:04.994 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el 32/136 2023-10-03T10:09:05.002 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : bison-3.0.4-10.el8.x86_64 33/136 2023-10-03T10:09:05.002 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flex-2.6.1-9.el8.x86_64 34/136 2023-10-03T10:09:05.002 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : jq-1.6-7.el8.x86_64 35/136 2023-10-03T10:09:05.003 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 36/136 2023-10-03T10:09:05.003 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 37/136 2023-10-03T10:09:05.003 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 38/136 2023-10-03T10:09:05.003 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 39/136 2023-10-03T10:09:05.003 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 40/136 2023-10-03T10:09:05.003 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 41/136 2023-10-03T10:09:05.003 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 42/136 2023-10-03T10:09:05.003 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 43/136 2023-10-03T10:09:05.003 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 44/136 2023-10-03T10:09:05.003 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 45/136 2023-10-03T10:09:05.003 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 46/136 2023-10-03T10:09:05.003 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 47/136 2023-10-03T10:09:05.003 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 48/136 2023-10-03T10:09:05.004 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 49/136 2023-10-03T10:09:05.004 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 50/136 2023-10-03T10:09:05.004 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 51/136 2023-10-03T10:09:05.004 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 52/136 2023-10-03T10:09:05.004 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 53/136 2023-10-03T10:09:05.004 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 54/136 2023-10-03T10:09:05.004 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 55/136 2023-10-03T10:09:05.004 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 56/136 2023-10-03T10:09:05.004 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 57/136 2023-10-03T10:09:05.004 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 58/136 2023-10-03T10:09:05.004 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 59/136 2023-10-03T10:09:05.004 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 60/136 2023-10-03T10:09:05.004 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 61/136 2023-10-03T10:09:05.005 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 62/136 2023-10-03T10:09:05.005 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 63/136 2023-10-03T10:09:05.005 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 64/136 2023-10-03T10:09:05.005 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 65/136 2023-10-03T10:09:05.005 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 66/136 2023-10-03T10:09:05.005 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 67/136 2023-10-03T10:09:05.005 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 68/136 2023-10-03T10:09:05.005 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 69/136 2023-10-03T10:09:05.005 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 70/136 2023-10-03T10:09:05.005 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 71/136 2023-10-03T10:09:05.005 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 72/136 2023-10-03T10:09:05.005 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 73/136 2023-10-03T10:09:05.005 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/136 2023-10-03T10:09:05.005 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 75/136 2023-10-03T10:09:05.006 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 76/136 2023-10-03T10:09:05.006 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 77/136 2023-10-03T10:09:05.006 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 78/136 2023-10-03T10:09:05.006 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 79/136 2023-10-03T10:09:05.006 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : leveldb-1.22-1.el8.x86_64 80/136 2023-10-03T10:09:05.006 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 81/136 2023-10-03T10:09:05.006 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 82/136 2023-10-03T10:09:05.006 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/136 2023-10-03T10:09:05.006 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 84/136 2023-10-03T10:09:05.006 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 85/136 2023-10-03T10:09:05.006 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 86/136 2023-10-03T10:09:05.006 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 87/136 2023-10-03T10:09:05.006 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 88/136 2023-10-03T10:09:05.007 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 89/136 2023-10-03T10:09:05.007 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 90/136 2023-10-03T10:09:05.007 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 91/136 2023-10-03T10:09:05.007 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 92/136 2023-10-03T10:09:05.007 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 93/136 2023-10-03T10:09:05.007 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 94/136 2023-10-03T10:09:05.007 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 95/136 2023-10-03T10:09:05.007 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-portend-2.6-1.el8.noarch 96/136 2023-10-03T10:09:05.007 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 97/136 2023-10-03T10:09:05.007 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 98/136 2023-10-03T10:09:05.007 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 99/136 2023-10-03T10:09:05.007 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 100/136 2023-10-03T10:09:05.007 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 101/136 2023-10-03T10:09:05.008 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 102/136 2023-10-03T10:09:05.008 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 103/136 2023-10-03T10:09:05.008 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 104/136 2023-10-03T10:09:05.008 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 105/136 2023-10-03T10:09:05.008 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 106/136 2023-10-03T10:09:05.008 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 107/136 2023-10-03T10:09:05.008 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 108/136 2023-10-03T10:09:05.009 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 109/136 2023-10-03T10:09:05.009 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 110/136 2023-10-03T10:09:05.009 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 111/136 2023-10-03T10:09:05.009 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 112/136 2023-10-03T10:09:05.009 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 113/136 2023-10-03T10:09:05.009 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-4.el 114/136 2023-10-03T10:09:05.009 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el8.x86_64 115/136 2023-10-03T10:09:05.009 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 116/136 2023-10-03T10:09:05.009 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 117/136 2023-10-03T10:09:05.009 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 118/136 2023-10-03T10:09:05.009 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 119/136 2023-10-03T10:09:05.009 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 120/136 2023-10-03T10:09:05.009 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : iproute-6.2.0-5.el8.x86_64 121/136 2023-10-03T10:09:05.009 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 122/136 2023-10-03T10:09:05.009 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 123/136 2023-10-03T10:09:05.010 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 124/136 2023-10-03T10:09:05.010 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 125/136 2023-10-03T10:09:05.010 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 126/136 2023-10-03T10:09:05.010 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 127/136 2023-10-03T10:09:05.010 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 128/136 2023-10-03T10:09:05.010 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 129/136 2023-10-03T10:09:05.010 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 130/136 2023-10-03T10:09:05.010 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 131/136 2023-10-03T10:09:05.010 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 132/136 2023-10-03T10:09:05.010 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 133/136 2023-10-03T10:09:05.010 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 134/136 2023-10-03T10:09:05.010 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 135/136 2023-10-03T10:09:07.030 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 136/136 2023-10-03T10:09:07.030 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:07.031 INFO:teuthology.orchestra.run.smithi022.stdout:Upgraded: 2023-10-03T10:09:07.031 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2023-10-03T10:09:07.031 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-debuginfod-client-0.189-3.el8.x86_64 2023-10-03T10:09:07.031 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-10-03T10:09:07.031 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-10-03T10:09:07.031 INFO:teuthology.orchestra.run.smithi022.stdout: iproute-6.2.0-5.el8.x86_64 2023-10-03T10:09:07.031 INFO:teuthology.orchestra.run.smithi022.stdout: libblkid-2.32.1-43.el8.x86_64 2023-10-03T10:09:07.031 INFO:teuthology.orchestra.run.smithi022.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-10-03T10:09:07.031 INFO:teuthology.orchestra.run.smithi022.stdout: libmount-2.32.1-43.el8.x86_64 2023-10-03T10:09:07.031 INFO:teuthology.orchestra.run.smithi022.stdout: librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:09:07.031 INFO:teuthology.orchestra.run.smithi022.stdout: librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:09:07.031 INFO:teuthology.orchestra.run.smithi022.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-10-03T10:09:07.032 INFO:teuthology.orchestra.run.smithi022.stdout: libuuid-2.32.1-43.el8.x86_64 2023-10-03T10:09:07.032 INFO:teuthology.orchestra.run.smithi022.stdout: util-linux-2.32.1-43.el8.x86_64 2023-10-03T10:09:07.032 INFO:teuthology.orchestra.run.smithi022.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-10-03T10:09:07.032 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2023-10-03T10:09:07.032 INFO:teuthology.orchestra.run.smithi022.stdout: bison-3.0.4-10.el8.x86_64 2023-10-03T10:09:07.032 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:09:07.032 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:09:07.032 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:09:07.032 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:09:07.032 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.el8.noarch 2023-10-03T10:09:07.032 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:09:07.032 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:09:07.033 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.noarch 2023-10-03T10:09:07.033 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-diskprediction-local-2:16.2.14-44.geeb13ac9.el8.noarch 2023-10-03T10:09:07.033 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8.noarch 2023-10-03T10:09:07.033 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 2023-10-03T10:09:07.033 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:09:07.033 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:09:07.033 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el8.noarch 2023-10-03T10:09:07.033 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:09:07.033 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:09:07.033 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:09:07.033 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-10-03T10:09:07.034 INFO:teuthology.orchestra.run.smithi022.stdout: flex-2.6.1-9.el8.x86_64 2023-10-03T10:09:07.034 INFO:teuthology.orchestra.run.smithi022.stdout: fuse-2.9.7-17.el8.x86_64 2023-10-03T10:09:07.034 INFO:teuthology.orchestra.run.smithi022.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-03T10:09:07.034 INFO:teuthology.orchestra.run.smithi022.stdout: jq-1.6-7.el8.x86_64 2023-10-03T10:09:07.034 INFO:teuthology.orchestra.run.smithi022.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-03T10:09:07.034 INFO:teuthology.orchestra.run.smithi022.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-10-03T10:09:07.034 INFO:teuthology.orchestra.run.smithi022.stdout: leveldb-1.22-1.el8.x86_64 2023-10-03T10:09:07.034 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:09:07.034 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:09:07.034 INFO:teuthology.orchestra.run.smithi022.stdout: libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:09:07.034 INFO:teuthology.orchestra.run.smithi022.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-10-03T10:09:07.034 INFO:teuthology.orchestra.run.smithi022.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-10-03T10:09:07.035 INFO:teuthology.orchestra.run.smithi022.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-03T10:09:07.035 INFO:teuthology.orchestra.run.smithi022.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-10-03T10:09:07.035 INFO:teuthology.orchestra.run.smithi022.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-10-03T10:09:07.035 INFO:teuthology.orchestra.run.smithi022.stdout: librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:09:07.035 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:09:07.035 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-03T10:09:07.035 INFO:teuthology.orchestra.run.smithi022.stdout: librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:09:07.035 INFO:teuthology.orchestra.run.smithi022.stdout: libselinux-devel-2.9-8.el8.x86_64 2023-10-03T10:09:07.035 INFO:teuthology.orchestra.run.smithi022.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-03T10:09:07.035 INFO:teuthology.orchestra.run.smithi022.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-03T10:09:07.035 INFO:teuthology.orchestra.run.smithi022.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-10-03T10:09:07.035 INFO:teuthology.orchestra.run.smithi022.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-10-03T10:09:07.036 INFO:teuthology.orchestra.run.smithi022.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-03T10:09:07.036 INFO:teuthology.orchestra.run.smithi022.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-03T10:09:07.036 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-0.3.15-4.el8.x86_64 2023-10-03T10:09:07.036 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-10-03T10:09:07.036 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-10-03T10:09:07.036 INFO:teuthology.orchestra.run.smithi022.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-10-03T10:09:07.036 INFO:teuthology.orchestra.run.smithi022.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-10-03T10:09:07.036 INFO:teuthology.orchestra.run.smithi022.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-10-03T10:09:07.036 INFO:teuthology.orchestra.run.smithi022.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-03T10:09:07.036 INFO:teuthology.orchestra.run.smithi022.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-03T10:09:07.036 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-03T10:09:07.036 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:09:07.037 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:09:07.037 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:09:07.037 INFO:teuthology.orchestra.run.smithi022.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-03T10:09:07.037 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-10-03T10:09:07.037 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-03T10:09:07.037 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-03T10:09:07.037 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-10-03T10:09:07.037 INFO:teuthology.orchestra.run.smithi022.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-03T10:09:07.037 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-03T10:09:07.037 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-03T10:09:07.037 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-03T10:09:07.037 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-03T10:09:07.038 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-03T10:09:07.038 INFO:teuthology.orchestra.run.smithi022.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-03T10:09:07.038 INFO:teuthology.orchestra.run.smithi022.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-03T10:09:07.038 INFO:teuthology.orchestra.run.smithi022.stdout: python3-mako-1.0.6-14.el8.noarch 2023-10-03T10:09:07.038 INFO:teuthology.orchestra.run.smithi022.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-03T10:09:07.038 INFO:teuthology.orchestra.run.smithi022.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-03T10:09:07.038 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-03T10:09:07.038 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-03T10:09:07.038 INFO:teuthology.orchestra.run.smithi022.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-03T10:09:07.038 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-03T10:09:07.038 INFO:teuthology.orchestra.run.smithi022.stdout: python3-portend-2.6-1.el8.noarch 2023-10-03T10:09:07.039 INFO:teuthology.orchestra.run.smithi022.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-03T10:09:07.039 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-03T10:09:07.039 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-03T10:09:07.039 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-03T10:09:07.039 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-03T10:09:07.039 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pytz-2017.2-11.el8.noarch 2023-10-03T10:09:07.039 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:09:07.039 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:09:07.039 INFO:teuthology.orchestra.run.smithi022.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-03T10:09:07.039 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-03T10:09:07.039 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:09:07.039 INFO:teuthology.orchestra.run.smithi022.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-03T10:09:07.039 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-03T10:09:07.040 INFO:teuthology.orchestra.run.smithi022.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-10-03T10:09:07.040 INFO:teuthology.orchestra.run.smithi022.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-03T10:09:07.040 INFO:teuthology.orchestra.run.smithi022.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-03T10:09:07.040 INFO:teuthology.orchestra.run.smithi022.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-03T10:09:07.040 INFO:teuthology.orchestra.run.smithi022.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-03T10:09:07.040 INFO:teuthology.orchestra.run.smithi022.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-03T10:09:07.040 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-03T10:09:07.040 INFO:teuthology.orchestra.run.smithi022.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-03T10:09:07.040 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-03T10:09:07.040 INFO:teuthology.orchestra.run.smithi022.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-03T10:09:07.040 INFO:teuthology.orchestra.run.smithi022.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-03T10:09:07.040 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:09:07.041 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:09:07.041 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-10-03T10:09:07.041 INFO:teuthology.orchestra.run.smithi022.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-03T10:09:07.041 INFO:teuthology.orchestra.run.smithi022.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-03T10:09:07.041 INFO:teuthology.orchestra.run.smithi022.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-10-03T10:09:07.041 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:07.041 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2023-10-03T10:09:07.237 DEBUG:teuthology.parallel:result is None 2023-10-03T10:09:07.238 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-03T10:09:07.238 INFO:teuthology.packaging:ref: None 2023-10-03T10:09:07.238 INFO:teuthology.packaging:tag: None 2023-10-03T10:09:07.238 INFO:teuthology.packaging:branch: pacific 2023-10-03T10:09:07.238 INFO:teuthology.packaging:sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 2023-10-03T10:09:07.238 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=pacific 2023-10-03T10:09:07.702 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-03T10:09:07.733 INFO:teuthology.orchestra.run.smithi003.stdout:16.2.14-44.geeb13ac9.el8 2023-10-03T10:09:07.734 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-44.geeb13ac9.el8 2023-10-03T10:09:07.734 INFO:teuthology.task.install:The correct ceph version 16.2.14-44.geeb13ac9 is installed. 2023-10-03T10:09:07.734 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-03T10:09:07.734 INFO:teuthology.packaging:ref: None 2023-10-03T10:09:07.734 INFO:teuthology.packaging:tag: None 2023-10-03T10:09:07.734 INFO:teuthology.packaging:branch: pacific 2023-10-03T10:09:07.734 INFO:teuthology.packaging:sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 2023-10-03T10:09:07.735 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=pacific 2023-10-03T10:09:08.148 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-03T10:09:08.179 INFO:teuthology.orchestra.run.smithi022.stdout:16.2.14-44.geeb13ac9.el8 2023-10-03T10:09:08.179 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-44.geeb13ac9.el8 2023-10-03T10:09:08.179 INFO:teuthology.task.install:The correct ceph version 16.2.14-44.geeb13ac9 is installed. 2023-10-03T10:09:08.179 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-03T10:09:08.180 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-03T10:09:08.180 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-03T10:09:08.220 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-10-03T10:09:08.220 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-03T10:09:08.261 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-03T10:09:08.261 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-03T10:09:08.261 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/daemon-helper 2023-10-03T10:09:08.296 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-03T10:09:08.370 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-10-03T10:09:08.370 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/daemon-helper 2023-10-03T10:09:08.406 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-03T10:09:08.479 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-03T10:09:08.479 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-03T10:09:08.479 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-03T10:09:08.514 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-03T10:09:08.589 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-10-03T10:09:08.589 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-03T10:09:08.622 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-03T10:09:08.696 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-03T10:09:08.697 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-03T10:09:08.697 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/stdin-killer 2023-10-03T10:09:08.731 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-03T10:09:08.806 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-10-03T10:09:08.806 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/stdin-killer 2023-10-03T10:09:08.840 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-03T10:09:08.915 INFO:teuthology.run_tasks:Running task print... 2023-10-03T10:09:08.925 INFO:teuthology.task.print:**** done installing pacific 2023-10-03T10:09:08.925 INFO:teuthology.run_tasks:Running task ceph... 2023-10-03T10:09:08.994 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-03T10:09:08.995 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 777 /var/log/ceph 2023-10-03T10:09:08.997 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 777 /var/log/ceph 2023-10-03T10:09:09.032 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-03T10:09:09.032 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-03T10:09:09.072 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-03T10:09:09.106 INFO:tasks.ceph:Creating extra log directories... 2023-10-03T10:09:09.106 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-03T10:09:09.146 DEBUG:teuthology.orchestra.run.smithi022:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-03T10:09:09.189 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-03T10:09:09.189 INFO:tasks.ceph:config {'conf': {'global': {'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['overall HEALTH_', '\\(FS_', '\\(MDS_', '\\(OSD_', '\\(MON_DOWN\\)', '\\(CACHE_POOL_', '\\(POOL_', '\\(MGR_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'Monitor daemon marked osd', 'Behind on trimming', 'Manager daemon', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-03T10:09:09.189 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/dparmar-2023-10-03_09:48:43-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7409565', 'branch': 'fix-nfs-apply-err-reporting', 'description': 'fs:upgrade/upgraded_client/{bluestore-bitmap branch/pacific centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-install 1-mount/mount/fuse 2-upgrade 3-workload/stress_tests/{0-client-upgrade 1-tests/dbench}}}', 'email': None, 'first_in_suite': False, 'job_id': '7409565', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'dparmar-2023-10-03_09:48:43-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'fix-nfs-apply-err-reporting'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-62953', 'sha1': 'a6297898cc696a7db13e71aaa0b24c78a4c800c5'}}, 'owner': 'scheduled_dparmar@teuthology', 'priority': 55, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3'], ['client.0']], 'seed': 6051, 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054', 'sleep_before_teardown': 0, 'suite': 'fs:upgrade', 'suite_branch': 'wip-62953', 'suite_path': '/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/dparmar18/ceph', 'suite_sha1': 'a6297898cc696a7db13e71aaa0b24c78a4c800c5', 'targets': {'smithi003.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCtr4kcGEi5jcuobzEHMX+y83QO1weeTgYpAhL0wlGpu0Q/PbpE4hZOfSYBQz0oZE+5f2EomyGa26hSmhuzfin8=', 'smithi022.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKCqvRZ4EVfy757SyycWQMn6oUIteEVFUHylwlmm7Wv6zfVG6TRFcNBVK79xTzP0dUavmY0OoCHdZL15rMPBmSk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'branch': 'pacific', 'exclude_packages': ['cephadm', 'ceph-mgr-cephadm', 'ceph-immutable-object-cache', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'ceph-volume'], 'extra_packages': ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'extra_system_packages': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux'], 'flavor': 'default', 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054'}}, {'print': '**** done installing pacific'}, {'ceph': {'conf': {'global': {'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'log-ignorelist': ['overall HEALTH_', '\\(FS_', '\\(MDS_', '\\(OSD_', '\\(MON_DOWN\\)', '\\(CACHE_POOL_', '\\(POOL_', '\\(MGR_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'Monitor daemon marked osd', 'Behind on trimming', 'Manager daemon', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'flavor': 'default', 'fs': 'xfs', 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054', 'cluster': 'ceph'}}, {'exec': {'osd.0': ['ceph osd set-require-min-compat-client pacific']}}, {'print': '**** done ceph'}, {'ceph-fuse': None}, {'mds_pre_upgrade': None}, {'print': '**** done mds pre-upgrade sequence'}, {'install.upgrade': {'mon.a': {'branch': 'reef'}}}, {'print': '**** done install.upgrade the host'}, {'ceph.restart': {'daemons': ['mon.*', 'mgr.*'], 'mon-health-to-clog': False, 'wait-for-healthy': False}}, {'ceph.healthy': None}, {'ceph.restart': {'daemons': ['osd.*'], 'wait-for-healthy': False, 'wait-for-osds-up': True}}, {'ceph.stop': ['mds.*']}, {'ceph.restart': {'daemons': ['mds.*'], 'wait-for-healthy': False, 'wait-for-osds-up': True}}, {'exec': {'mon.a': ['ceph osd dump -f json-pretty', 'ceph versions', 'for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done', 'ceph osd require-osd-release reef']}}, {'ceph.healthy': None}, {'print': '**** done ceph.restart'}, {'ceph-fuse': {'client.0': {'mounted': False}}}, {'print': '**** done unmount client.0'}, {'install.upgrade': {'client.0': None}}, {'print': '**** done install.upgrade on client.0'}, {'ceph-fuse': {'client.0': None}}, {'print': '**** done remount client.0'}, {'workunit': {'clients': {'all': ['suites/dbench.sh']}}}], 'teuthology': {'branch': 'pacific', 'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end", 'if not is_fuse() then reject() end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-03_09:48:43', 'tube': 'smithi', 'user': 'dparmar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-03T10:09:09.190 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-03T10:09:09.221 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-03T10:09:09.243 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m0777 -- /var/run/ceph 2023-10-03T10:09:09.284 DEBUG:teuthology.orchestra.run.smithi022:> sudo install -d -m0777 -- /var/run/ceph 2023-10-03T10:09:09.318 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-03T10:09:09.318 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/scratch_devs of=/dev/stdout 2023-10-03T10:09:09.380 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-03T10:09:09.380 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_1 2023-10-03T10:09:09.443 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-03T10:09:09.443 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-03T10:09:09.443 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 6h/6d Inode: 391580 Links: 1 2023-10-03T10:09:09.443 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-03T10:09:09.443 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2023-10-03T10:09:09.443 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2023-10-03 10:08:53.389237456 +0000 2023-10-03T10:09:09.443 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2023-10-03 10:05:44.626380275 +0000 2023-10-03T10:09:09.443 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2023-10-03 10:05:44.626380275 +0000 2023-10-03T10:09:09.444 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2023-10-03T10:09:09.444 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-03T10:09:09.516 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2023-10-03T10:09:09.517 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2023-10-03T10:09:09.517 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000241246 s, 2.1 MB/s 2023-10-03T10:09:09.518 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-03T10:09:09.582 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_2 2023-10-03T10:09:09.646 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-03T10:09:09.646 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-03T10:09:09.646 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 6h/6d Inode: 388742 Links: 1 2023-10-03T10:09:09.646 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-03T10:09:09.646 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2023-10-03T10:09:09.646 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2023-10-03 10:08:53.389237456 +0000 2023-10-03T10:09:09.646 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2023-10-03 10:05:45.205373006 +0000 2023-10-03T10:09:09.646 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2023-10-03 10:05:45.205373006 +0000 2023-10-03T10:09:09.646 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2023-10-03T10:09:09.647 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-03T10:09:09.718 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2023-10-03T10:09:09.719 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2023-10-03T10:09:09.719 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000235694 s, 2.2 MB/s 2023-10-03T10:09:09.720 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-03T10:09:09.785 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_3 2023-10-03T10:09:09.847 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-03T10:09:09.847 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-03T10:09:09.847 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 6h/6d Inode: 388820 Links: 1 2023-10-03T10:09:09.847 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-03T10:09:09.847 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2023-10-03T10:09:09.847 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2023-10-03 10:08:53.389237456 +0000 2023-10-03T10:09:09.847 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2023-10-03 10:05:45.765365974 +0000 2023-10-03T10:09:09.848 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2023-10-03 10:05:45.765365974 +0000 2023-10-03T10:09:09.848 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2023-10-03T10:09:09.848 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-03T10:09:09.920 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2023-10-03T10:09:09.920 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2023-10-03T10:09:09.920 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000236392 s, 2.2 MB/s 2023-10-03T10:09:09.922 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-03T10:09:09.986 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_4 2023-10-03T10:09:10.048 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-03T10:09:10.049 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-03T10:09:10.049 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 6h/6d Inode: 388900 Links: 1 2023-10-03T10:09:10.049 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-03T10:09:10.049 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2023-10-03T10:09:10.049 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2023-10-03 10:08:53.389237456 +0000 2023-10-03T10:09:10.049 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2023-10-03 10:05:46.345358692 +0000 2023-10-03T10:09:10.049 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2023-10-03 10:05:46.345358692 +0000 2023-10-03T10:09:10.049 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2023-10-03T10:09:10.049 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-03T10:09:10.122 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2023-10-03T10:09:10.122 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2023-10-03T10:09:10.122 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000249427 s, 2.1 MB/s 2023-10-03T10:09:10.123 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-03T10:09:10.187 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-03T10:09:10.187 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi003.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2023-10-03T10:09:10.187 INFO:tasks.ceph:Generating config... 2023-10-03T10:09:10.188 INFO:tasks.ceph:[global] mon warn on pool no app = False 2023-10-03T10:09:10.188 INFO:tasks.ceph:[global] ms bind msgr2 = False 2023-10-03T10:09:10.189 INFO:tasks.ceph:[global] bluestore warn on legacy statfs = False 2023-10-03T10:09:10.189 INFO:tasks.ceph:[global] bluestore warn on no per pool omap = False 2023-10-03T10:09:10.189 INFO:tasks.ceph:[global] mon pg warn min per osd = 0 2023-10-03T10:09:10.189 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-03T10:09:10.189 INFO:tasks.ceph:[client] debug client = 20 2023-10-03T10:09:10.189 INFO:tasks.ceph:[client] debug ms = 1 2023-10-03T10:09:10.189 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-03T10:09:10.189 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-03T10:09:10.189 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-03T10:09:10.189 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-10-03T10:09:10.189 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-03T10:09:10.190 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-03T10:09:10.190 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-03T10:09:10.190 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-03T10:09:10.190 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-03T10:09:10.190 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-03T10:09:10.190 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-03T10:09:10.190 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-03T10:09:10.190 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-03T10:09:10.190 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-03T10:09:10.190 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-03T10:09:10.190 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-03T10:09:10.190 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-03T10:09:10.190 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-03T10:09:10.191 INFO:tasks.ceph:[mon] mon warn on osd down out interval zero = False 2023-10-03T10:09:10.191 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-03T10:09:10.191 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-03T10:09:10.191 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-03T10:09:10.191 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-03T10:09:10.191 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-03T10:09:10.191 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-03T10:09:10.191 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-03T10:09:10.191 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-03T10:09:10.191 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-03T10:09:10.191 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-03T10:09:10.191 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-03T10:09:10.191 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-03T10:09:10.192 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-03T10:09:10.192 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-03T10:09:10.192 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-03T10:09:10.192 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-03T10:09:10.192 INFO:tasks.ceph:Setting up mon.a... 2023-10-03T10:09:10.192 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-03T10:09:10.360 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/ceph.keyring 2023-10-03T10:09:10.362 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-03T10:09:10.414 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-03T10:09:10.447 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.3'), ('mon.b', '[v2:172.21.15.3:3301,v1:172.21.15.3:6790]'), ('mon.c', '[v2:172.21.15.3:3302,v1:172.21.15.3:6791]')] 2023-10-03T10:09:10.447 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.3,[v2:172.21.15.3:3301,v1:172.21.15.3:6790],[v2:172.21.15.3:3302,v1:172.21.15.3:6791]', 'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-03T10:09:10.448 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-03T10:09:10.448 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-03T10:09:10.510 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.3 --addv b '[v2:172.21.15.3:3301,v1:172.21.15.3:6790]' --addv c '[v2:172.21.15.3:3302,v1:172.21.15.3:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-03T10:09:10.604 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-03T10:09:10.604 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: generated fsid 68e08e7d-a212-4fd3-bd42-88e2411aa299 2023-10-03T10:09:10.604 INFO:teuthology.orchestra.run.smithi003.stdout:epoch 0 2023-10-03T10:09:10.604 INFO:teuthology.orchestra.run.smithi003.stdout:fsid 68e08e7d-a212-4fd3-bd42-88e2411aa299 2023-10-03T10:09:10.604 INFO:teuthology.orchestra.run.smithi003.stdout:last_changed 2023-10-03T10:09:10.604679+0000 2023-10-03T10:09:10.604 INFO:teuthology.orchestra.run.smithi003.stdout:created 2023-10-03T10:09:10.604679+0000 2023-10-03T10:09:10.604 INFO:teuthology.orchestra.run.smithi003.stdout:min_mon_release 0 (unknown) 2023-10-03T10:09:10.604 INFO:teuthology.orchestra.run.smithi003.stdout:election_strategy: 1 2023-10-03T10:09:10.604 INFO:teuthology.orchestra.run.smithi003.stdout:0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2023-10-03T10:09:10.604 INFO:teuthology.orchestra.run.smithi003.stdout:1: [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] mon.b 2023-10-03T10:09:10.605 INFO:teuthology.orchestra.run.smithi003.stdout:2: [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] mon.c 2023-10-03T10:09:10.605 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-03T10:09:10.606 DEBUG:teuthology.orchestra.run.smithi003:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-03T10:09:10.626 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 68e08e7d-a212-4fd3-bd42-88e2411aa299... 2023-10-03T10:09:10.627 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-03T10:09:10.668 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-03T10:09:10.722 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2023-10-03T10:09:10.722 INFO:teuthology.orchestra.run.smithi003.stdout: chdir = "" 2023-10-03T10:09:10.723 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-03T10:09:10.723 INFO:teuthology.orchestra.run.smithi003.stdout: auth supported = cephx 2023-10-03T10:09:10.723 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.723 INFO:teuthology.orchestra.run.smithi003.stdout: filestore xattr use omap = true 2023-10-03T10:09:10.723 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.723 INFO:teuthology.orchestra.run.smithi003.stdout: mon clock drift allowed = 1.000 2023-10-03T10:09:10.723 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.723 INFO:teuthology.orchestra.run.smithi003.stdout: osd crush chooseleaf type = 0 2023-10-03T10:09:10.723 INFO:teuthology.orchestra.run.smithi003.stdout: auth debug = true 2023-10-03T10:09:10.724 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.724 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on old message = true 2023-10-03T10:09:10.724 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on bug = true 2023-10-03T10:09:10.724 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.724 INFO:teuthology.orchestra.run.smithi003.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-03T10:09:10.724 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn max object skew = 0 2023-10-03T10:09:10.724 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.724 INFO:teuthology.orchestra.run.smithi003.stdout: # disable pg_autoscaler by default for new pools 2023-10-03T10:09:10.724 INFO:teuthology.orchestra.run.smithi003.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-03T10:09:10.724 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.725 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default size = 2 2023-10-03T10:09:10.725 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.725 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow primary affinity = true 2023-10-03T10:09:10.725 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow pg remap = true 2023-10-03T10:09:10.725 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on legacy crush tunables = false 2023-10-03T10:09:10.725 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on crush straw calc version zero = false 2023-10-03T10:09:10.725 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on no sortbitwise = false 2023-10-03T10:09:10.725 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = false 2023-10-03T10:09:10.725 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on too few osds = false 2023-10-03T10:09:10.725 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-03T10:09:10.725 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-03T10:09:10.726 INFO:teuthology.orchestra.run.smithi003.stdout: mon_allow_pool_size_one = true 2023-10-03T10:09:10.726 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.726 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-03T10:09:10.726 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.726 INFO:teuthology.orchestra.run.smithi003.stdout: osd default data pool replay window = 5 2023-10-03T10:09:10.726 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.726 INFO:teuthology.orchestra.run.smithi003.stdout: mon allow pool delete = true 2023-10-03T10:09:10.726 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.726 INFO:teuthology.orchestra.run.smithi003.stdout: mon cluster log file level = debug 2023-10-03T10:09:10.726 INFO:teuthology.orchestra.run.smithi003.stdout: debug asserts on shutdown = true 2023-10-03T10:09:10.727 INFO:teuthology.orchestra.run.smithi003.stdout: mon health detail to clog = false 2023-10-03T10:09:10.727 INFO:teuthology.orchestra.run.smithi003.stdout: mon host = "172.21.15.3,[v2:172.21.15.3:3301,v1:172.21.15.3:6790],[v2:172.21.15.3:3302,v1:172.21.15.3:6791]" 2023-10-03T10:09:10.727 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on pool no app = False 2023-10-03T10:09:10.727 INFO:teuthology.orchestra.run.smithi003.stdout: ms bind msgr2 = False 2023-10-03T10:09:10.727 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore warn on legacy statfs = False 2023-10-03T10:09:10.727 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore warn on no per pool omap = False 2023-10-03T10:09:10.727 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn min per osd = 0 2023-10-03T10:09:10.727 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = 68e08e7d-a212-4fd3-bd42-88e2411aa299 2023-10-03T10:09:10.727 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.727 INFO:teuthology.orchestra.run.smithi003.stdout:[osd] 2023-10-03T10:09:10.727 INFO:teuthology.orchestra.run.smithi003.stdout: osd journal size = 100 2023-10-03T10:09:10.727 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.728 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub load threshold = 5.0 2023-10-03T10:09:10.728 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub max interval = 600 2023-10-03T10:09:10.728 INFO:teuthology.orchestra.run.smithi003.stdout: osd mclock profile = high_recovery_ops 2023-10-03T10:09:10.728 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.728 INFO:teuthology.orchestra.run.smithi003.stdout: osd recover clone overlap = true 2023-10-03T10:09:10.728 INFO:teuthology.orchestra.run.smithi003.stdout: osd recovery max chunk = 1048576 2023-10-03T10:09:10.728 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.728 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug shutdown = true 2023-10-03T10:09:10.728 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug op order = true 2023-10-03T10:09:10.728 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug verify stray on activate = true 2023-10-03T10:09:10.728 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.728 INFO:teuthology.orchestra.run.smithi003.stdout: osd open classes on start = true 2023-10-03T10:09:10.728 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug pg log writeout = true 2023-10-03T10:09:10.728 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.729 INFO:teuthology.orchestra.run.smithi003.stdout: osd deep scrub update digest min age = 30 2023-10-03T10:09:10.729 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.729 INFO:teuthology.orchestra.run.smithi003.stdout: osd map max advance = 10 2023-10-03T10:09:10.729 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.729 INFO:teuthology.orchestra.run.smithi003.stdout: journal zero on create = true 2023-10-03T10:09:10.729 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.729 INFO:teuthology.orchestra.run.smithi003.stdout: filestore ondisk finisher threads = 3 2023-10-03T10:09:10.729 INFO:teuthology.orchestra.run.smithi003.stdout: filestore apply finisher threads = 3 2023-10-03T10:09:10.729 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.729 INFO:teuthology.orchestra.run.smithi003.stdout: bdev debug aio = true 2023-10-03T10:09:10.730 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug misdirected ops = true 2023-10-03T10:09:10.730 INFO:teuthology.orchestra.run.smithi003.stdout: bdev async discard = True 2023-10-03T10:09:10.730 INFO:teuthology.orchestra.run.smithi003.stdout: bdev enable discard = True 2023-10-03T10:09:10.730 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore allocator = bitmap 2023-10-03T10:09:10.730 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore block size = 96636764160 2023-10-03T10:09:10.730 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore fsck on mount = True 2023-10-03T10:09:10.730 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluefs = 1/20 2023-10-03T10:09:10.730 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluestore = 1/20 2023-10-03T10:09:10.730 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-03T10:09:10.730 INFO:teuthology.orchestra.run.smithi003.stdout: debug osd = 20 2023-10-03T10:09:10.730 INFO:teuthology.orchestra.run.smithi003.stdout: debug rocksdb = 4/10 2023-10-03T10:09:10.730 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd backfillfull_ratio = 0.85 2023-10-03T10:09:10.730 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd full ratio = 0.9 2023-10-03T10:09:10.731 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd nearfull ratio = 0.8 2023-10-03T10:09:10.731 INFO:teuthology.orchestra.run.smithi003.stdout: osd failsafe full ratio = 0.95 2023-10-03T10:09:10.731 INFO:teuthology.orchestra.run.smithi003.stdout: osd objectstore = bluestore 2023-10-03T10:09:10.731 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2023-10-03T10:09:10.731 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.731 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr] 2023-10-03T10:09:10.731 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-03T10:09:10.731 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgr = 20 2023-10-03T10:09:10.731 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2023-10-03T10:09:10.731 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2023-10-03T10:09:10.731 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2023-10-03T10:09:10.731 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2023-10-03T10:09:10.731 INFO:teuthology.orchestra.run.smithi003.stdout: mgr/telemetry/nag = false 2023-10-03T10:09:10.731 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.732 INFO:teuthology.orchestra.run.smithi003.stdout:[mon] 2023-10-03T10:09:10.732 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-03T10:09:10.732 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2023-10-03T10:09:10.732 INFO:teuthology.orchestra.run.smithi003.stdout: debug paxos = 20 2023-10-03T10:09:10.732 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2023-10-03T10:09:10.732 INFO:teuthology.orchestra.run.smithi003.stdout: mon data avail warn = 5 2023-10-03T10:09:10.732 INFO:teuthology.orchestra.run.smithi003.stdout: mon mgr mkfs grace = 240 2023-10-03T10:09:10.732 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2023-10-03T10:09:10.732 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd reporter subtree level = osd 2023-10-03T10:09:10.732 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd prime pg temp = true 2023-10-03T10:09:10.732 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2023-10-03T10:09:10.732 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.732 INFO:teuthology.orchestra.run.smithi003.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-03T10:09:10.733 INFO:teuthology.orchestra.run.smithi003.stdout: auth mon ticket ttl = 660 # 11m 2023-10-03T10:09:10.733 INFO:teuthology.orchestra.run.smithi003.stdout: auth service ticket ttl = 240 # 4m 2023-10-03T10:09:10.733 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.733 INFO:teuthology.orchestra.run.smithi003.stdout: # don't complain about insecure global_id in the test suite 2023-10-03T10:09:10.733 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-03T10:09:10.733 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-03T10:09:10.733 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.733 INFO:teuthology.orchestra.run.smithi003.stdout: # 1m isn't quite enough 2023-10-03T10:09:10.733 INFO:teuthology.orchestra.run.smithi003.stdout: mon_down_mkfs_grace = 2m 2023-10-03T10:09:10.733 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.733 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_filestore_osds = false 2023-10-03T10:09:10.733 INFO:teuthology.orchestra.run.smithi003.stdout: mon op complaint time = 120 2023-10-03T10:09:10.733 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = False 2023-10-03T10:09:10.734 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:10.734 INFO:teuthology.orchestra.run.smithi003.stdout:[client] 2023-10-03T10:09:10.734 INFO:teuthology.orchestra.run.smithi003.stdout: rgw cache enabled = true 2023-10-03T10:09:10.734 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable ops log = true 2023-10-03T10:09:10.734 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable usage log = true 2023-10-03T10:09:10.734 INFO:teuthology.orchestra.run.smithi003.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-03T10:09:10.734 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-03T10:09:10.734 INFO:teuthology.orchestra.run.smithi003.stdout: client mount timeout = 600 2023-10-03T10:09:10.734 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2023-10-03T10:09:10.734 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-03T10:09:10.734 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2023-10-03T10:09:10.734 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2023-10-03T10:09:10.735 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.a] 2023-10-03T10:09:10.735 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.b] 2023-10-03T10:09:10.735 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.c] 2023-10-03T10:09:10.735 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2023-10-03T10:09:10.735 INFO:teuthology.orchestra.run.smithi003.stdout:[mds] 2023-10-03T10:09:10.735 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds = 20 2023-10-03T10:09:10.735 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds balancer = 20 2023-10-03T10:09:10.735 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-10-03T10:09:10.735 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug frag = True 2023-10-03T10:09:10.735 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug scatterstat = True 2023-10-03T10:09:10.735 INFO:teuthology.orchestra.run.smithi003.stdout: mds op complaint time = 180 2023-10-03T10:09:10.736 INFO:teuthology.orchestra.run.smithi003.stdout: mds verify scatter = True 2023-10-03T10:09:10.736 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2023-10-03T10:09:10.736 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2023-10-03T10:09:10.736 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2023-10-03T10:09:10.736 INFO:teuthology.orchestra.run.smithi022.stdout:[global] 2023-10-03T10:09:10.736 INFO:teuthology.orchestra.run.smithi022.stdout: chdir = "" 2023-10-03T10:09:10.737 INFO:teuthology.orchestra.run.smithi022.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-03T10:09:10.737 INFO:teuthology.orchestra.run.smithi022.stdout: auth supported = cephx 2023-10-03T10:09:10.737 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.737 INFO:teuthology.orchestra.run.smithi022.stdout: filestore xattr use omap = true 2023-10-03T10:09:10.737 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.737 INFO:teuthology.orchestra.run.smithi022.stdout: mon clock drift allowed = 1.000 2023-10-03T10:09:10.737 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.737 INFO:teuthology.orchestra.run.smithi022.stdout: osd crush chooseleaf type = 0 2023-10-03T10:09:10.737 INFO:teuthology.orchestra.run.smithi022.stdout: auth debug = true 2023-10-03T10:09:10.737 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.737 INFO:teuthology.orchestra.run.smithi022.stdout: ms die on old message = true 2023-10-03T10:09:10.738 INFO:teuthology.orchestra.run.smithi022.stdout: ms die on bug = true 2023-10-03T10:09:10.738 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.738 INFO:teuthology.orchestra.run.smithi022.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-03T10:09:10.738 INFO:teuthology.orchestra.run.smithi022.stdout: mon pg warn max object skew = 0 2023-10-03T10:09:10.738 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.738 INFO:teuthology.orchestra.run.smithi022.stdout: # disable pg_autoscaler by default for new pools 2023-10-03T10:09:10.738 INFO:teuthology.orchestra.run.smithi022.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-03T10:09:10.738 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.738 INFO:teuthology.orchestra.run.smithi022.stdout: osd pool default size = 2 2023-10-03T10:09:10.738 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.738 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd allow primary affinity = true 2023-10-03T10:09:10.738 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd allow pg remap = true 2023-10-03T10:09:10.739 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on legacy crush tunables = false 2023-10-03T10:09:10.739 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on crush straw calc version zero = false 2023-10-03T10:09:10.739 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on no sortbitwise = false 2023-10-03T10:09:10.739 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on osd down out interval zero = false 2023-10-03T10:09:10.739 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on too few osds = false 2023-10-03T10:09:10.739 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-03T10:09:10.739 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-03T10:09:10.739 INFO:teuthology.orchestra.run.smithi022.stdout: mon_allow_pool_size_one = true 2023-10-03T10:09:10.739 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.739 INFO:teuthology.orchestra.run.smithi022.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-03T10:09:10.739 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.739 INFO:teuthology.orchestra.run.smithi022.stdout: osd default data pool replay window = 5 2023-10-03T10:09:10.739 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.740 INFO:teuthology.orchestra.run.smithi022.stdout: mon allow pool delete = true 2023-10-03T10:09:10.740 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.740 INFO:teuthology.orchestra.run.smithi022.stdout: mon cluster log file level = debug 2023-10-03T10:09:10.740 INFO:teuthology.orchestra.run.smithi022.stdout: debug asserts on shutdown = true 2023-10-03T10:09:10.740 INFO:teuthology.orchestra.run.smithi022.stdout: mon health detail to clog = false 2023-10-03T10:09:10.740 INFO:teuthology.orchestra.run.smithi022.stdout: mon host = "172.21.15.3,[v2:172.21.15.3:3301,v1:172.21.15.3:6790],[v2:172.21.15.3:3302,v1:172.21.15.3:6791]" 2023-10-03T10:09:10.740 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on pool no app = False 2023-10-03T10:09:10.740 INFO:teuthology.orchestra.run.smithi022.stdout: ms bind msgr2 = False 2023-10-03T10:09:10.740 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore warn on legacy statfs = False 2023-10-03T10:09:10.740 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore warn on no per pool omap = False 2023-10-03T10:09:10.740 INFO:teuthology.orchestra.run.smithi022.stdout: mon pg warn min per osd = 0 2023-10-03T10:09:10.740 INFO:teuthology.orchestra.run.smithi022.stdout: fsid = 68e08e7d-a212-4fd3-bd42-88e2411aa299 2023-10-03T10:09:10.740 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.741 INFO:teuthology.orchestra.run.smithi022.stdout:[osd] 2023-10-03T10:09:10.741 INFO:teuthology.orchestra.run.smithi022.stdout: osd journal size = 100 2023-10-03T10:09:10.741 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.741 INFO:teuthology.orchestra.run.smithi022.stdout: osd scrub load threshold = 5.0 2023-10-03T10:09:10.741 INFO:teuthology.orchestra.run.smithi022.stdout: osd scrub max interval = 600 2023-10-03T10:09:10.741 INFO:teuthology.orchestra.run.smithi022.stdout: osd mclock profile = high_recovery_ops 2023-10-03T10:09:10.741 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.741 INFO:teuthology.orchestra.run.smithi022.stdout: osd recover clone overlap = true 2023-10-03T10:09:10.741 INFO:teuthology.orchestra.run.smithi022.stdout: osd recovery max chunk = 1048576 2023-10-03T10:09:10.741 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.741 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug shutdown = true 2023-10-03T10:09:10.741 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug op order = true 2023-10-03T10:09:10.741 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug verify stray on activate = true 2023-10-03T10:09:10.742 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.742 INFO:teuthology.orchestra.run.smithi022.stdout: osd open classes on start = true 2023-10-03T10:09:10.742 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug pg log writeout = true 2023-10-03T10:09:10.742 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.742 INFO:teuthology.orchestra.run.smithi022.stdout: osd deep scrub update digest min age = 30 2023-10-03T10:09:10.742 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.742 INFO:teuthology.orchestra.run.smithi022.stdout: osd map max advance = 10 2023-10-03T10:09:10.742 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.742 INFO:teuthology.orchestra.run.smithi022.stdout: journal zero on create = true 2023-10-03T10:09:10.742 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.742 INFO:teuthology.orchestra.run.smithi022.stdout: filestore ondisk finisher threads = 3 2023-10-03T10:09:10.742 INFO:teuthology.orchestra.run.smithi022.stdout: filestore apply finisher threads = 3 2023-10-03T10:09:10.742 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.743 INFO:teuthology.orchestra.run.smithi022.stdout: bdev debug aio = true 2023-10-03T10:09:10.743 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug misdirected ops = true 2023-10-03T10:09:10.743 INFO:teuthology.orchestra.run.smithi022.stdout: bdev async discard = True 2023-10-03T10:09:10.743 INFO:teuthology.orchestra.run.smithi022.stdout: bdev enable discard = True 2023-10-03T10:09:10.743 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore allocator = bitmap 2023-10-03T10:09:10.743 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore block size = 96636764160 2023-10-03T10:09:10.743 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore fsck on mount = True 2023-10-03T10:09:10.743 INFO:teuthology.orchestra.run.smithi022.stdout: debug bluefs = 1/20 2023-10-03T10:09:10.743 INFO:teuthology.orchestra.run.smithi022.stdout: debug bluestore = 1/20 2023-10-03T10:09:10.743 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2023-10-03T10:09:10.743 INFO:teuthology.orchestra.run.smithi022.stdout: debug osd = 20 2023-10-03T10:09:10.743 INFO:teuthology.orchestra.run.smithi022.stdout: debug rocksdb = 4/10 2023-10-03T10:09:10.744 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd backfillfull_ratio = 0.85 2023-10-03T10:09:10.744 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd full ratio = 0.9 2023-10-03T10:09:10.744 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd nearfull ratio = 0.8 2023-10-03T10:09:10.744 INFO:teuthology.orchestra.run.smithi022.stdout: osd failsafe full ratio = 0.95 2023-10-03T10:09:10.744 INFO:teuthology.orchestra.run.smithi022.stdout: osd objectstore = bluestore 2023-10-03T10:09:10.744 INFO:teuthology.orchestra.run.smithi022.stdout: osd op complaint time = 180 2023-10-03T10:09:10.744 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.744 INFO:teuthology.orchestra.run.smithi022.stdout:[mgr] 2023-10-03T10:09:10.744 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2023-10-03T10:09:10.744 INFO:teuthology.orchestra.run.smithi022.stdout: debug mgr = 20 2023-10-03T10:09:10.744 INFO:teuthology.orchestra.run.smithi022.stdout: debug mon = 20 2023-10-03T10:09:10.744 INFO:teuthology.orchestra.run.smithi022.stdout: debug auth = 20 2023-10-03T10:09:10.744 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min pgs per osd = 4 2023-10-03T10:09:10.745 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min bytes per osd = 10 2023-10-03T10:09:10.745 INFO:teuthology.orchestra.run.smithi022.stdout: mgr/telemetry/nag = false 2023-10-03T10:09:10.745 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.745 INFO:teuthology.orchestra.run.smithi022.stdout:[mon] 2023-10-03T10:09:10.745 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2023-10-03T10:09:10.745 INFO:teuthology.orchestra.run.smithi022.stdout: debug mon = 20 2023-10-03T10:09:10.745 INFO:teuthology.orchestra.run.smithi022.stdout: debug paxos = 20 2023-10-03T10:09:10.745 INFO:teuthology.orchestra.run.smithi022.stdout: debug auth = 20 2023-10-03T10:09:10.745 INFO:teuthology.orchestra.run.smithi022.stdout: mon data avail warn = 5 2023-10-03T10:09:10.745 INFO:teuthology.orchestra.run.smithi022.stdout: mon mgr mkfs grace = 240 2023-10-03T10:09:10.745 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min pgs per osd = 4 2023-10-03T10:09:10.745 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd reporter subtree level = osd 2023-10-03T10:09:10.745 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd prime pg temp = true 2023-10-03T10:09:10.746 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min bytes per osd = 10 2023-10-03T10:09:10.746 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.746 INFO:teuthology.orchestra.run.smithi022.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-03T10:09:10.746 INFO:teuthology.orchestra.run.smithi022.stdout: auth mon ticket ttl = 660 # 11m 2023-10-03T10:09:10.746 INFO:teuthology.orchestra.run.smithi022.stdout: auth service ticket ttl = 240 # 4m 2023-10-03T10:09:10.746 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.746 INFO:teuthology.orchestra.run.smithi022.stdout: # don't complain about insecure global_id in the test suite 2023-10-03T10:09:10.746 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-03T10:09:10.746 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-03T10:09:10.746 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.746 INFO:teuthology.orchestra.run.smithi022.stdout: # 1m isn't quite enough 2023-10-03T10:09:10.746 INFO:teuthology.orchestra.run.smithi022.stdout: mon_down_mkfs_grace = 2m 2023-10-03T10:09:10.746 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.746 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_filestore_osds = false 2023-10-03T10:09:10.746 INFO:teuthology.orchestra.run.smithi022.stdout: mon op complaint time = 120 2023-10-03T10:09:10.747 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on osd down out interval zero = False 2023-10-03T10:09:10.747 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-10-03T10:09:10.747 INFO:teuthology.orchestra.run.smithi022.stdout:[client] 2023-10-03T10:09:10.747 INFO:teuthology.orchestra.run.smithi022.stdout: rgw cache enabled = true 2023-10-03T10:09:10.747 INFO:teuthology.orchestra.run.smithi022.stdout: rgw enable ops log = true 2023-10-03T10:09:10.747 INFO:teuthology.orchestra.run.smithi022.stdout: rgw enable usage log = true 2023-10-03T10:09:10.747 INFO:teuthology.orchestra.run.smithi022.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-03T10:09:10.747 INFO:teuthology.orchestra.run.smithi022.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-03T10:09:10.747 INFO:teuthology.orchestra.run.smithi022.stdout: client mount timeout = 600 2023-10-03T10:09:10.747 INFO:teuthology.orchestra.run.smithi022.stdout: debug client = 20 2023-10-03T10:09:10.747 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2023-10-03T10:09:10.747 INFO:teuthology.orchestra.run.smithi022.stdout: rados mon op timeout = 900 2023-10-03T10:09:10.748 INFO:teuthology.orchestra.run.smithi022.stdout: rados osd op timeout = 900 2023-10-03T10:09:10.748 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.a] 2023-10-03T10:09:10.748 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.b] 2023-10-03T10:09:10.748 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.c] 2023-10-03T10:09:10.748 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.a] 2023-10-03T10:09:10.748 INFO:teuthology.orchestra.run.smithi022.stdout:[mds] 2023-10-03T10:09:10.748 INFO:teuthology.orchestra.run.smithi022.stdout: debug mds = 20 2023-10-03T10:09:10.748 INFO:teuthology.orchestra.run.smithi022.stdout: debug mds balancer = 20 2023-10-03T10:09:10.748 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2023-10-03T10:09:10.748 INFO:teuthology.orchestra.run.smithi022.stdout: mds debug frag = True 2023-10-03T10:09:10.748 INFO:teuthology.orchestra.run.smithi022.stdout: mds debug scatterstat = True 2023-10-03T10:09:10.748 INFO:teuthology.orchestra.run.smithi022.stdout: mds op complaint time = 180 2023-10-03T10:09:10.748 INFO:teuthology.orchestra.run.smithi022.stdout: mds verify scatter = True 2023-10-03T10:09:10.748 INFO:teuthology.orchestra.run.smithi022.stdout: osd op complaint time = 180 2023-10-03T10:09:10.749 INFO:teuthology.orchestra.run.smithi022.stdout: rados mon op timeout = 900 2023-10-03T10:09:10.749 INFO:teuthology.orchestra.run.smithi022.stdout: rados osd op timeout = 900 2023-10-03T10:09:10.750 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-03T10:09:10.750 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-03T10:09:10.929 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-03T10:09:10.930 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-03T10:09:10.930 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-03T10:09:10.952 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-03T10:09:10.953 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-03T10:09:11.015 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi003.front.sepia.ceph.com 2023-10-03T10:09:11.015 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-03T10:09:11.016 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-03T10:09:11.016 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-03T10:09:11.130 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-03T10:09:11.130 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-03T10:09:11.258 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi022.front.sepia.ceph.com 2023-10-03T10:09:11.258 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-10-03T10:09:11.258 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-03T10:09:11.258 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-03T10:09:11.306 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-10-03T10:09:11.306 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-03T10:09:11.369 INFO:tasks.ceph:Setting up mon nodes... 2023-10-03T10:09:11.369 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-03T10:09:11.370 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-03T10:09:11.431 INFO:teuthology.orchestra.run.smithi003.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-03T10:09:11.434 INFO:tasks.ceph:Setting up mds nodes... 2023-10-03T10:09:11.434 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-03T10:09:11.493 INFO:teuthology.orchestra.run.smithi003.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-03T10:09:11.496 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-03T10:09:11.527 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-03T10:09:11.527 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-03T10:09:11.651 INFO:teuthology.orchestra.run.smithi022.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-03T10:09:11.667 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-03T10:09:11.667 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi003.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2023-10-03T10:09:11.667 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-03T10:09:11.702 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-03T10:09:11.702 INFO:tasks.ceph:role: osd.0 2023-10-03T10:09:11.702 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi003.front.sepia.ceph.com 2023-10-03T10:09:11.702 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-03T10:09:11.785 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-03T10:09:11.785 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-03T10:09:11.785 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-03T10:09:11.785 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-03T10:09:11.785 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-03T10:09:11.785 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2023-10-03T10:09:11.785 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-03T10:09:11.785 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-03T10:09:11.785 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-03T10:09:11.785 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-03T10:09:12.220 INFO:teuthology.orchestra.run.smithi003.stdout:Discarding blocks...Done. 2023-10-03T10:09:12.222 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2023-10-03T10:09:12.222 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-03T10:09:12.395 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-03T10:09:12.438 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-03T10:09:12.513 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-03T10:09:12.513 INFO:tasks.ceph:role: osd.1 2023-10-03T10:09:12.513 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi003.front.sepia.ceph.com 2023-10-03T10:09:12.513 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-03T10:09:12.595 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-03T10:09:12.595 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-03T10:09:12.595 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-03T10:09:12.595 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-03T10:09:12.595 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-03T10:09:12.596 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2023-10-03T10:09:12.596 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-03T10:09:12.596 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-03T10:09:12.596 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-03T10:09:12.596 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-03T10:09:13.015 INFO:teuthology.orchestra.run.smithi003.stdout:Discarding blocks...Done. 2023-10-03T10:09:13.016 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2023-10-03T10:09:13.017 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-03T10:09:13.129 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-03T10:09:13.167 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-03T10:09:13.243 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-03T10:09:13.243 INFO:tasks.ceph:role: osd.2 2023-10-03T10:09:13.243 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi003.front.sepia.ceph.com 2023-10-03T10:09:13.243 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-03T10:09:13.326 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-03T10:09:13.326 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-03T10:09:13.326 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-03T10:09:13.327 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-03T10:09:13.327 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-03T10:09:13.327 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2023-10-03T10:09:13.327 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-03T10:09:13.327 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-03T10:09:13.327 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-03T10:09:13.327 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-03T10:09:13.764 INFO:teuthology.orchestra.run.smithi003.stdout:Discarding blocks...Done. 2023-10-03T10:09:13.766 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2023-10-03T10:09:13.767 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-03T10:09:13.930 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-03T10:09:13.970 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-03T10:09:14.045 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-03T10:09:14.045 INFO:tasks.ceph:role: osd.3 2023-10-03T10:09:14.045 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi003.front.sepia.ceph.com 2023-10-03T10:09:14.045 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-03T10:09:14.128 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-03T10:09:14.128 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-03T10:09:14.128 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-03T10:09:14.128 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-03T10:09:14.128 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-03T10:09:14.128 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2023-10-03T10:09:14.128 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-03T10:09:14.128 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-03T10:09:14.128 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-03T10:09:14.129 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-03T10:09:14.555 INFO:teuthology.orchestra.run.smithi003.stdout:Discarding blocks...Done. 2023-10-03T10:09:14.557 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2023-10-03T10:09:14.557 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-03T10:09:14.736 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-03T10:09:14.775 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-03T10:09:14.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:14.885+0000 7f43eb1d1200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-03T10:09:14.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:14.885+0000 7f43eb1d1200 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-03T10:09:14.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:14.886+0000 7f43eb1d1200 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-03T10:09:14.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:14.886+0000 7f43eb1d1200 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-03T10:09:14.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:14.886+0000 7f43eb1d1200 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-03T10:09:14.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:14.886+0000 7f43eb1d1200 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-03T10:09:17.730 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-03T10:09:17.766 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-03T10:09:17.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:17.864+0000 7f287acbc200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-03T10:09:17.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:17.864+0000 7f287acbc200 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-03T10:09:17.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:17.864+0000 7f287acbc200 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-03T10:09:17.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:17.864+0000 7f287acbc200 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-03T10:09:17.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:17.864+0000 7f287acbc200 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-03T10:09:17.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:17.865+0000 7f287acbc200 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-03T10:09:20.721 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-03T10:09:20.757 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-03T10:09:20.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:20.851+0000 7fec60953200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-03T10:09:20.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:20.851+0000 7fec60953200 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-03T10:09:20.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:20.851+0000 7fec60953200 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-03T10:09:20.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:20.851+0000 7fec60953200 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-03T10:09:20.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:20.851+0000 7fec60953200 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-03T10:09:20.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:20.851+0000 7fec60953200 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-03T10:09:23.703 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-03T10:09:23.738 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-03T10:09:23.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:23.832+0000 7ff8d094e200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-03T10:09:23.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:23.833+0000 7ff8d094e200 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-03T10:09:23.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:23.833+0000 7ff8d094e200 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-03T10:09:23.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:23.833+0000 7ff8d094e200 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-03T10:09:23.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:23.833+0000 7ff8d094e200 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-03T10:09:23.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:23.833+0000 7ff8d094e200 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-03T10:09:26.676 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-03T10:09:26.712 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-03T10:09:26.712 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-03T10:09:26.712 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-03T10:09:26.787 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-03T10:09:26.787 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-03T10:09:26.862 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-03T10:09:26.862 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-03T10:09:26.937 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-03T10:09:26.937 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-03T10:09:27.012 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-03T10:09:27.012 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-03T10:09:27.087 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-03T10:09:27.087 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-03T10:09:27.162 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-10-03T10:09:27.162 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-03T10:09:27.185 INFO:tasks.ceph:Adding keys to all mons... 2023-10-03T10:09:27.186 DEBUG:teuthology.orchestra.run.smithi003:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-03T10:09:27.236 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr.x] 2023-10-03T10:09:27.236 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBH6Btlp8bEGRAAt0YlCLucaziyjZAqKTvr2g== 2023-10-03T10:09:27.236 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2023-10-03T10:09:27.236 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBH6BtlvVJ7HRAACDKLl3VxHXJK3s1D6PTzMg== 2023-10-03T10:09:27.236 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.0] 2023-10-03T10:09:27.236 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBK6BtlqaDaNBAAwlos32gsjg/uiblJRCqJqQ== 2023-10-03T10:09:27.236 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.1] 2023-10-03T10:09:27.236 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBN6BtlMA+aMxAAgOprsUUtmbcYM812IwEYUw== 2023-10-03T10:09:27.237 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.2] 2023-10-03T10:09:27.237 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBQ6Btlfi/KMhAA+xCfuDregQ37bltleqK5EQ== 2023-10-03T10:09:27.237 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.3] 2023-10-03T10:09:27.237 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBT6Btly0yzMRAAEAKKpkeF/4iXQXK5082bZA== 2023-10-03T10:09:27.237 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2023-10-03T10:09:27.237 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBH6BtlvETjJhAAbs6K21YA6EDAiQbN44hhMg== 2023-10-03T10:09:27.238 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-03T10:09:27.338 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-03T10:09:27.398 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-03T10:09:27.454 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-03T10:09:27.509 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-03T10:09:27.564 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-03T10:09:27.618 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-03T10:09:27.676 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-03T10:09:27.676 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-03T10:09:27.706 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-03T10:09:27.809 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-03T10:09:27.840 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-03T10:09:27.914 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-03T10:09:28.022 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-03T10:09:28.054 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-03T10:09:28.129 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-03T10:09:28.237 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-03T10:09:28.270 DEBUG:teuthology.orchestra.run.smithi003:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-03T10:09:28.333 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-03T10:09:28.333 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-03T10:09:28.333 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-03T10:09:28.376 INFO:tasks.ceph.mon.a:Started 2023-10-03T10:09:28.376 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-03T10:09:28.376 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-03T10:09:28.378 INFO:tasks.ceph.mon.b:Started 2023-10-03T10:09:28.378 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-03T10:09:28.378 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-03T10:09:28.380 INFO:tasks.ceph.mon.c:Started 2023-10-03T10:09:28.381 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-03T10:09:28.381 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-03T10:09:28.381 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-03T10:09:28.383 INFO:tasks.ceph.mgr.x:Started 2023-10-03T10:09:28.383 INFO:tasks.ceph:Setting crush tunables to default 2023-10-03T10:09:28.383 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph osd crush tunables default 2023-10-03T10:09:28.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.538+0000 7fb9cdfbe700 1 Processor -- start 2023-10-03T10:09:28.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.538+0000 7fb9cdfbe700 1 -- start start 2023-10-03T10:09:28.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.539+0000 7fb9cdfbe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9c80997d0 0x7fb9c8093620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:28.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.539+0000 7fb9cdfbe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb9c8127980 0x7fb9c81247a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:28.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.539+0000 7fb9cdfbe700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c8092b80 con 0x7fb9c81275a0 2023-10-03T10:09:28.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.539+0000 7fb9cdfbe700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c8092910 con 0x7fb9c80997d0 2023-10-03T10:09:28.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.539+0000 7fb9cdfbe700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c8127ff0 con 0x7fb9c8127980 2023-10-03T10:09:28.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.539+0000 7fb9c77fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb9c81275a0 0x7fb9c8124090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33402/0 (socket says 172.21.15.3:33402) 2023-10-03T10:09:28.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.539+0000 7fb9c77fe700 1 -- 172.21.15.3:0/2435091255 learned_addr learned my addr 172.21.15.3:0/2435091255 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:09:28.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.539+0000 7fb9c7fff700 1 --2- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb9c8127980 0x7fb9c81247a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:28.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.539+0000 7fb9c6ffd700 1 --2- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9c80997d0 0x7fb9c8093620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:28.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.539+0000 7fb9c7fff700 1 -- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb9c8127980 msgr2=0x7fb9c81247a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-03T10:09:28.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.539+0000 7fb9c7fff700 1 -- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb9c8127980 msgr2=0x7fb9c81247a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-03T10:09:28.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.539+0000 7fb9c7fff700 1 --2- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb9c8127980 0x7fb9c81247a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-03T10:09:28.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.539+0000 7fb9c6ffd700 1 -- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9c80997d0 msgr2=0x7fb9c8093620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-03T10:09:28.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.539+0000 7fb9c6ffd700 1 -- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9c80997d0 msgr2=0x7fb9c8093620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-03T10:09:28.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.539+0000 7fb9c7fff700 1 --2- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb9c8127980 0x7fb9c81247a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-10-03T10:09:28.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.539+0000 7fb9c6ffd700 1 --2- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9c80997d0 0x7fb9c8093620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-03T10:09:28.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.539+0000 7fb9c6ffd700 1 --2- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9c80997d0 0x7fb9c8093620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-10-03T10:09:28.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.719+0000 7fb9c67fc700 1 -- 172.21.15.3:0/2435091255 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3421576669 0 0) 0x7fb9c8092b80 con 0x7fb9c81275a0 2023-10-03T10:09:28.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.719+0000 7fb9c67fc700 1 -- 172.21.15.3:0/2435091255 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9b40033f0 con 0x7fb9c81275a0 2023-10-03T10:09:28.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.720+0000 7fb9c67fc700 1 -- 172.21.15.3:0/2435091255 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2122587840 0 0) 0x7fb9b40033f0 con 0x7fb9c81275a0 2023-10-03T10:09:28.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.720+0000 7fb9c67fc700 1 -- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb9c8127980 msgr2=0x7fb9c81247a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-03T10:09:28.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.720+0000 7fb9c67fc700 1 --2- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb9c8127980 0x7fb9c81247a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:28.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.720+0000 7fb9c67fc700 1 -- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9c80997d0 msgr2=0x7fb9c8093620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-03T10:09:28.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.720+0000 7fb9c67fc700 1 --2- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9c80997d0 0x7fb9c8093620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:28.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.720+0000 7fb9c67fc700 1 -- 172.21.15.3:0/2435091255 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c809a190 con 0x7fb9c81275a0 2023-10-03T10:09:28.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.720+0000 7fb9c67fc700 1 -- 172.21.15.3:0/2435091255 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1614054521 0 0) 0x7fb9b8001470 con 0x7fb9c81275a0 2023-10-03T10:09:28.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.720+0000 7fb9c67fc700 1 -- 172.21.15.3:0/2435091255 >> v1:172.21.15.3:6789/0 conn(0x7fb9c81275a0 legacy=0x7fb9c8124090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:28.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.720+0000 7fb9c67fc700 1 --2- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb9c8127980 0x7fb9b40038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:28.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.720+0000 7fb9c67fc700 1 --2- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9b4003ec0 0x7fb9b4004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:28.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.720+0000 7fb9c67fc700 1 --2- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9c80997d0 0x7fb9b4004b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:28.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.720+0000 7fb9c67fc700 1 -- 172.21.15.3:0/2435091255 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c8127ff0 con 0x7fb9b4003ec0 2023-10-03T10:09:28.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.720+0000 7fb9c67fc700 1 -- 172.21.15.3:0/2435091255 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b4005170 con 0x7fb9c80997d0 2023-10-03T10:09:28.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.720+0000 7fb9c67fc700 1 -- 172.21.15.3:0/2435091255 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b4005330 con 0x7fb9c8127980 2023-10-03T10:09:28.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.720+0000 7fb9c77fe700 1 --2- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb9c8127980 0x7fb9b40038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:28.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.720+0000 7fb9c6ffd700 1 --2- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9b4003ec0 0x7fb9b4004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:28.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.721+0000 7fb9c7fff700 1 --2- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9c80997d0 0x7fb9b4004b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:28.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.721+0000 7fb9c77fe700 1 -- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9c80997d0 msgr2=0x7fb9b4004b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:28.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.721+0000 7fb9c77fe700 1 --2- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9c80997d0 0x7fb9b4004b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:28.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.721+0000 7fb9c77fe700 1 -- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9b4003ec0 msgr2=0x7fb9b4004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:28.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.721+0000 7fb9c77fe700 1 --2- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9b4003ec0 0x7fb9b4004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:28.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.721+0000 7fb9c77fe700 1 -- 172.21.15.3:0/2435091255 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b40033f0 con 0x7fb9c8127980 2023-10-03T10:09:28.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.721+0000 7fb9c7fff700 1 --2- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9c80997d0 0x7fb9b4004b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:09:28.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.721+0000 7fb9c6ffd700 1 --2- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9b4003ec0 0x7fb9b4004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:28.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.721+0000 7fb9c77fe700 1 --2- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb9c8127980 0x7fb9b40038e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7fb9b800c210 tx=0x7fb9b8001e90).ready entity=mon.2 client_cookie=bf101ce5f974dda8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:28.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.723+0000 7fb9c67fc700 1 -- 172.21.15.3:0/2435091255 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8007a60 con 0x7fb9c8127980 2023-10-03T10:09:28.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.723+0000 7fb9c67fc700 1 -- 172.21.15.3:0/2435091255 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b800cae0 con 0x7fb9c8127980 2023-10-03T10:09:28.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.723+0000 7fb9c67fc700 1 -- 172.21.15.3:0/2435091255 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b800ce60 con 0x7fb9c8127980 2023-10-03T10:09:28.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.724+0000 7fb9cdfbe700 1 -- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb9c8127980 msgr2=0x7fb9b40038e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:28.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.724+0000 7fb9cdfbe700 1 --2- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb9c8127980 0x7fb9b40038e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7fb9b800c210 tx=0x7fb9b8001e90).stop 2023-10-03T10:09:28.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.724+0000 7fb9cdfbe700 1 -- 172.21.15.3:0/2435091255 shutdown_connections 2023-10-03T10:09:28.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.724+0000 7fb9cdfbe700 1 --2- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9c80997d0 0x7fb9b4004b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:28.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.724+0000 7fb9cdfbe700 1 --2- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9b4003ec0 0x7fb9b4004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:28.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.724+0000 7fb9cdfbe700 1 --2- 172.21.15.3:0/2435091255 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb9c8127980 0x7fb9b40038e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:28.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.724+0000 7fb9cdfbe700 1 -- 172.21.15.3:0/2435091255 >> 172.21.15.3:0/2435091255 conn(0x7fb9c80598d0 msgr2=0x7fb9c8097460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:28.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.724+0000 7fb9cdfbe700 1 -- 172.21.15.3:0/2435091255 shutdown_connections 2023-10-03T10:09:28.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.725+0000 7fb9cdfbe700 1 -- 172.21.15.3:0/2435091255 wait complete. 2023-10-03T10:09:28.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.726+0000 7fb9cdfbe700 1 Processor -- start 2023-10-03T10:09:28.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.726+0000 7fb9cdfbe700 1 -- start start 2023-10-03T10:09:28.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.727+0000 7fb9cdfbe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9b4003ec0 0x7fb9c8135a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:28.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.727+0000 7fb9cdfbe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb9c80997d0 0x7fb9c8135f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:28.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.727+0000 7fb9cdfbe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9c81275a0 0x7fb9c81122c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:28.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.727+0000 7fb9cdfbe700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c8094640 con 0x7fb9c81275a0 2023-10-03T10:09:28.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.727+0000 7fb9cdfbe700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c80922c0 con 0x7fb9b4003ec0 2023-10-03T10:09:28.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.727+0000 7fb9cdfbe700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c81287a0 con 0x7fb9c80997d0 2023-10-03T10:09:28.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.727+0000 7fb9c77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9b4003ec0 0x7fb9c8135a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:28.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.727+0000 7fb9c6ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb9c80997d0 0x7fb9c8135f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:28.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.727+0000 7fb9c7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9c81275a0 0x7fb9c81122c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:28.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.727+0000 7fb9c77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9b4003ec0 0x7fb9c8135a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52480/0 (socket says 172.21.15.3:52480) 2023-10-03T10:09:28.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.727+0000 7fb9c6ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb9c80997d0 0x7fb9c8135f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50172/0 (socket says 172.21.15.3:50172) 2023-10-03T10:09:28.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.727+0000 7fb9c7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9c81275a0 0x7fb9c81122c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33426/0 (socket says 172.21.15.3:33426) 2023-10-03T10:09:28.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.727+0000 7fb9c77fe700 1 -- 172.21.15.3:0/2092729223 learned_addr learned my addr 172.21.15.3:0/2092729223 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:28.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.728+0000 7fb9c77fe700 1 -- 172.21.15.3:0/2092729223 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb9c80997d0 msgr2=0x7fb9c8135f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:28.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.728+0000 7fb9c77fe700 1 --2- 172.21.15.3:0/2092729223 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb9c80997d0 0x7fb9c8135f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:28.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.728+0000 7fb9c77fe700 1 -- 172.21.15.3:0/2092729223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9c81275a0 msgr2=0x7fb9c81122c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:28.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.728+0000 7fb9c77fe700 1 --2- 172.21.15.3:0/2092729223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9c81275a0 0x7fb9c81122c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:28.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.728+0000 7fb9c77fe700 1 -- 172.21.15.3:0/2092729223 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c8115d10 con 0x7fb9b4003ec0 2023-10-03T10:09:28.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.728+0000 7fb9c77fe700 1 --2- 172.21.15.3:0/2092729223 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9b4003ec0 0x7fb9c8135a20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7fb9b8000fd0 tx=0x7fb9b8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:28.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.728+0000 7fb9bf7fe700 1 -- 172.21.15.3:0/2092729223 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b80074f0 con 0x7fb9b4003ec0 2023-10-03T10:09:28.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.728+0000 7fb9bf7fe700 1 -- 172.21.15.3:0/2092729223 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b8007670 con 0x7fb9b4003ec0 2023-10-03T10:09:28.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.728+0000 7fb9cdfbe700 1 -- 172.21.15.3:0/2092729223 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c8115770 con 0x7fb9b4003ec0 2023-10-03T10:09:28.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.728+0000 7fb9bf7fe700 1 -- 172.21.15.3:0/2092729223 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8004900 con 0x7fb9b4003ec0 2023-10-03T10:09:28.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.728+0000 7fb9cdfbe700 1 -- 172.21.15.3:0/2092729223 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c8113e00 con 0x7fb9b4003ec0 2023-10-03T10:09:28.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.728+0000 7fb9bf7fe700 1 -- 172.21.15.3:0/2092729223 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fb9b8020450 con 0x7fb9b4003ec0 2023-10-03T10:09:28.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.728+0000 7fb9bf7fe700 1 -- 172.21.15.3:0/2092729223 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fb9b80137a0 con 0x7fb9b4003ec0 2023-10-03T10:09:28.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.730+0000 7fb9cdfbe700 1 -- 172.21.15.3:0/2092729223 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c805a260 con 0x7fb9b4003ec0 2023-10-03T10:09:28.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.732+0000 7fb9bf7fe700 1 -- 172.21.15.3:0/2092729223 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fb9b8041e50 con 0x7fb9b4003ec0 2023-10-03T10:09:28.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.830+0000 7fb9cdfbe700 1 -- 172.21.15.3:0/2092729223 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fb9c8046c20 con 0x7fb9b4003ec0 2023-10-03T10:09:28.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.840+0000 7fb9bf7fe700 1 -- 172.21.15.3:0/2092729223 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fb9b8030020 con 0x7fb9b4003ec0 2023-10-03T10:09:28.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.841+0000 7fb9cdfbe700 1 -- 172.21.15.3:0/2092729223 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9b4003ec0 msgr2=0x7fb9c8135a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:28.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.841+0000 7fb9cdfbe700 1 --2- 172.21.15.3:0/2092729223 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9b4003ec0 0x7fb9c8135a20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7fb9b8000fd0 tx=0x7fb9b8015000).stop 2023-10-03T10:09:28.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.841+0000 7fb9cdfbe700 1 -- 172.21.15.3:0/2092729223 shutdown_connections 2023-10-03T10:09:28.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.842+0000 7fb9cdfbe700 1 --2- 172.21.15.3:0/2092729223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9c81275a0 0x7fb9c81122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:28.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.842+0000 7fb9cdfbe700 1 --2- 172.21.15.3:0/2092729223 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9b4003ec0 0x7fb9c8135a20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:28.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.842+0000 7fb9cdfbe700 1 --2- 172.21.15.3:0/2092729223 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb9c80997d0 0x7fb9c8135f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:28.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.842+0000 7fb9cdfbe700 1 -- 172.21.15.3:0/2092729223 >> 172.21.15.3:0/2092729223 conn(0x7fb9c80598d0 msgr2=0x7fb9c8062a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:28.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.842+0000 7fb9cdfbe700 1 -- 172.21.15.3:0/2092729223 shutdown_connections 2023-10-03T10:09:28.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:28.842+0000 7fb9cdfbe700 1 -- 172.21.15.3:0/2092729223 wait complete. 2023-10-03T10:09:28.843 INFO:teuthology.orchestra.run.smithi003.stderr:adjusted tunables profile to default 2023-10-03T10:09:28.855 INFO:tasks.ceph:check_enable_crimson: False 2023-10-03T10:09:28.855 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-03T10:09:28.855 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-03T10:09:28.855 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-03T10:09:28.885 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-03T10:09:28.885 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-03T10:09:28.901 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:09:28.900+0000 7f57e47cb680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-03T10:09:28.929 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-03T10:09:28.930 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-03T10:09:28.981 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:09:28.981+0000 7f57e47cb680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-03T10:09:29.000 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-03T10:09:29.001 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-03T10:09:29.072 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph osd new cdcdd930-3c5b-4486-a021-7132f90fb6e6 0 2023-10-03T10:09:29.078 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:09:29.078+0000 7f57e47cb680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-03T10:09:29.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.205+0000 7f29e4bf0700 1 Processor -- start 2023-10-03T10:09:29.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.205+0000 7f29e4bf0700 1 -- start start 2023-10-03T10:09:29.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.206+0000 7f29e4bf0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29e01275a0 0x7f29e00997d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:29.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.206+0000 7f29e4bf0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f29e0127980 0x7f29e01247a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:29.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.206+0000 7f29e4bf0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29e0092b80 con 0x7f29e0093620 2023-10-03T10:09:29.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.206+0000 7f29e4bf0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e0092910 con 0x7f29e0127980 2023-10-03T10:09:29.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.206+0000 7f29e4bf0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e0127ff0 con 0x7f29e01275a0 2023-10-03T10:09:29.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.206+0000 7f29de59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29e01275a0 0x7f29e00997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:29.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.206+0000 7f29de59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29e01275a0 0x7f29e00997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52488/0 (socket says 172.21.15.3:52488) 2023-10-03T10:09:29.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.206+0000 7f29de59c700 1 -- 172.21.15.3:0/1044593278 learned_addr learned my addr 172.21.15.3:0/1044593278 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:29.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.206+0000 7f29ded9d700 1 --2- 172.21.15.3:0/1044593278 >> v2:172.21.15.3:3300/0 conn(0x7f29e0127980 0x7f29e01247a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:29.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.207+0000 7f29de59c700 1 -- 172.21.15.3:0/1044593278 >> v2:172.21.15.3:3300/0 conn(0x7f29e0127980 msgr2=0x7f29e01247a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:29.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.207+0000 7f29de59c700 1 --2- 172.21.15.3:0/1044593278 >> v2:172.21.15.3:3300/0 conn(0x7f29e0127980 0x7f29e01247a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:29.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.207+0000 7f29de59c700 1 -- 172.21.15.3:0/1044593278 >> v1:172.21.15.3:6789/0 conn(0x7f29e0093620 legacy=0x7f29e0124090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:29.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.207+0000 7f29de59c700 1 -- 172.21.15.3:0/1044593278 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e009a190 con 0x7f29e01275a0 2023-10-03T10:09:29.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.207+0000 7f29ded9d700 1 --2- 172.21.15.3:0/1044593278 >> v2:172.21.15.3:3300/0 conn(0x7f29e0127980 0x7f29e01247a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:29.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.207+0000 7f29de59c700 1 --2- 172.21.15.3:0/1044593278 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29e01275a0 0x7f29e00997d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f29d0009f20 tx=0x7f29d0009840).ready entity=mon.1 client_cookie=3fe0f05c12059510 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:29.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.207+0000 7f29dd59a700 1 -- 172.21.15.3:0/1044593278 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d000e070 con 0x7f29e01275a0 2023-10-03T10:09:29.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.207+0000 7f29dd59a700 1 -- 172.21.15.3:0/1044593278 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d000b9d0 con 0x7f29e01275a0 2023-10-03T10:09:29.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.207+0000 7f29dd59a700 1 -- 172.21.15.3:0/1044593278 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d000bd50 con 0x7f29e01275a0 2023-10-03T10:09:29.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.208+0000 7f29e4bf0700 1 -- 172.21.15.3:0/1044593278 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29e01275a0 msgr2=0x7f29e00997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:29.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.208+0000 7f29e4bf0700 1 --2- 172.21.15.3:0/1044593278 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29e01275a0 0x7f29e00997d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f29d0009f20 tx=0x7f29d0009840).stop 2023-10-03T10:09:29.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.208+0000 7f29e4bf0700 1 -- 172.21.15.3:0/1044593278 shutdown_connections 2023-10-03T10:09:29.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.208+0000 7f29e4bf0700 1 --2- 172.21.15.3:0/1044593278 >> v2:172.21.15.3:3300/0 conn(0x7f29e0127980 0x7f29e01247a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:29.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.208+0000 7f29e4bf0700 1 --2- 172.21.15.3:0/1044593278 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29e01275a0 0x7f29e00997d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:29.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.208+0000 7f29e4bf0700 1 -- 172.21.15.3:0/1044593278 >> 172.21.15.3:0/1044593278 conn(0x7f29e00598d0 msgr2=0x7f29e0097460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:29.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.208+0000 7f29e4bf0700 1 -- 172.21.15.3:0/1044593278 shutdown_connections 2023-10-03T10:09:29.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.208+0000 7f29e4bf0700 1 -- 172.21.15.3:0/1044593278 wait complete. 2023-10-03T10:09:29.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.209+0000 7f29e4bf0700 1 Processor -- start 2023-10-03T10:09:29.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.209+0000 7f29e4bf0700 1 -- start start 2023-10-03T10:09:29.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.209+0000 7f29e4bf0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29e0093620 0x7f29e0112b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:29.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.209+0000 7f29e4bf0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29e01275a0 0x7f29e0112ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:29.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.209+0000 7f29e4bf0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29e0127980 0x7f29e01134e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:29.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.209+0000 7f29e4bf0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e01139d0 con 0x7f29e0093620 2023-10-03T10:09:29.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.209+0000 7f29e4bf0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e0063520 con 0x7f29e0127980 2023-10-03T10:09:29.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.209+0000 7f29e4bf0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29e0063710 con 0x7f29e01275a0 2023-10-03T10:09:29.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.209+0000 7f29de59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29e0093620 0x7f29e0112b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:29.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.209+0000 7f29de59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29e0093620 0x7f29e0112b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33448/0 (socket says 172.21.15.3:33448) 2023-10-03T10:09:29.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.209+0000 7f29de59c700 1 -- 172.21.15.3:0/1734053115 learned_addr learned my addr 172.21.15.3:0/1734053115 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:29.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.209+0000 7f29ddd9b700 1 --2- 172.21.15.3:0/1734053115 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29e01275a0 0x7f29e0112ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:29.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.210+0000 7f29ded9d700 1 --2- 172.21.15.3:0/1734053115 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29e0127980 0x7f29e01134e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:29.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.210+0000 7f29de59c700 1 -- 172.21.15.3:0/1734053115 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29e01275a0 msgr2=0x7f29e0112ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:29.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.210+0000 7f29de59c700 1 --2- 172.21.15.3:0/1734053115 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29e01275a0 0x7f29e0112ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:29.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.210+0000 7f29de59c700 1 -- 172.21.15.3:0/1734053115 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29e0127980 msgr2=0x7f29e01134e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:29.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.210+0000 7f29de59c700 1 --2- 172.21.15.3:0/1734053115 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29e0127980 0x7f29e01134e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:29.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.210+0000 7f29de59c700 1 -- 172.21.15.3:0/1734053115 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d80092a0 con 0x7f29e0093620 2023-10-03T10:09:29.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.210+0000 7f29de59c700 1 --2- 172.21.15.3:0/1734053115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29e0093620 0x7f29e0112b00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f29d0003370 tx=0x7f29d0009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:29.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.210+0000 7f29d67fc700 1 -- 172.21.15.3:0/1734053115 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d00092e0 con 0x7f29e0093620 2023-10-03T10:09:29.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.210+0000 7f29d67fc700 1 -- 172.21.15.3:0/1734053115 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d0007070 con 0x7f29e0093620 2023-10-03T10:09:29.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.210+0000 7f29e4bf0700 1 -- 172.21.15.3:0/1734053115 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d0009ae0 con 0x7f29e0093620 2023-10-03T10:09:29.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.210+0000 7f29d67fc700 1 -- 172.21.15.3:0/1734053115 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d000bd50 con 0x7f29e0093620 2023-10-03T10:09:29.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.210+0000 7f29e4bf0700 1 -- 172.21.15.3:0/1734053115 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e0113b80 con 0x7f29e0093620 2023-10-03T10:09:29.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.211+0000 7f29d67fc700 1 -- 172.21.15.3:0/1734053115 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f29d0003040 con 0x7f29e0093620 2023-10-03T10:09:29.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.211+0000 7f29d67fc700 1 -- 172.21.15.3:0/1734053115 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f29d0028790 con 0x7f29e0093620 2023-10-03T10:09:29.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.211+0000 7f29e4bf0700 1 -- 172.21.15.3:0/1734053115 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e0099630 con 0x7f29e0093620 2023-10-03T10:09:29.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.214+0000 7f29d67fc700 1 -- 172.21.15.3:0/1734053115 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f29d0023070 con 0x7f29e0093620 2023-10-03T10:09:29.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.298+0000 7f29e4bf0700 1 -- 172.21.15.3:0/1734053115 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cdcdd930-3c5b-4486-a021-7132f90fb6e6", "id": 0} v 0) v1 -- 0x7f29e005a260 con 0x7f29e0093620 2023-10-03T10:09:29.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.308+0000 7f29d67fc700 1 -- 172.21.15.3:0/1734053115 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cdcdd930-3c5b-4486-a021-7132f90fb6e6", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f29d0018040 con 0x7f29e0093620 2023-10-03T10:09:29.309 INFO:teuthology.orchestra.run.smithi003.stdout:0 2023-10-03T10:09:29.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.309+0000 7f29e4bf0700 1 -- 172.21.15.3:0/1734053115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29e0093620 msgr2=0x7f29e0112b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:29.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.309+0000 7f29e4bf0700 1 --2- 172.21.15.3:0/1734053115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29e0093620 0x7f29e0112b00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f29d0003370 tx=0x7f29d0009840).stop 2023-10-03T10:09:29.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.309+0000 7f29e4bf0700 1 -- 172.21.15.3:0/1734053115 shutdown_connections 2023-10-03T10:09:29.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.309+0000 7f29e4bf0700 1 --2- 172.21.15.3:0/1734053115 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29e0127980 0x7f29e01134e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:29.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.309+0000 7f29e4bf0700 1 --2- 172.21.15.3:0/1734053115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29e0093620 0x7f29e0112b00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:29.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.309+0000 7f29e4bf0700 1 --2- 172.21.15.3:0/1734053115 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29e01275a0 0x7f29e0112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:29.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.309+0000 7f29e4bf0700 1 -- 172.21.15.3:0/1734053115 >> 172.21.15.3:0/1734053115 conn(0x7f29e00598d0 msgr2=0x7f29e0097420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:29.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.309+0000 7f29e4bf0700 1 -- 172.21.15.3:0/1734053115 shutdown_connections 2023-10-03T10:09:29.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.309+0000 7f29e4bf0700 1 -- 172.21.15.3:0/1734053115 wait complete. 2023-10-03T10:09:29.322 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph osd new c8e58877-6762-4741-a8c1-567ebc97b708 1 2023-10-03T10:09:29.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.449+0000 7fbcbf7e5700 1 Processor -- start 2023-10-03T10:09:29.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.449+0000 7fbcbf7e5700 1 -- start start 2023-10-03T10:09:29.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.450+0000 7fbcbf7e5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbcb8062a90 0x7fbcb8129d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:29.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.450+0000 7fbcbf7e5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbcb812a270 0x7fbcb8128870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:29.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.450+0000 7fbcbf7e5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbcb8128d60 0x7fbcb8129210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:29.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.450+0000 7fbcbf7e5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcb8062e70 con 0x7fbcb812a270 2023-10-03T10:09:29.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.450+0000 7fbcbf7e5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb8099c80 con 0x7fbcb8128d60 2023-10-03T10:09:29.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.450+0000 7fbcbf7e5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb80634b0 con 0x7fbcb8062a90 2023-10-03T10:09:29.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.450+0000 7fbcbdd82700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbcb8128d60 0x7fbcb8129210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:29.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.450+0000 7fbcbcd80700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbcb812a270 0x7fbcb8128870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:29.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.450+0000 7fbcbd581700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbcb8062a90 0x7fbcb8129d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:29.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.450+0000 7fbcbdd82700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbcb8128d60 0x7fbcb8129210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52512/0 (socket says 172.21.15.3:52512) 2023-10-03T10:09:29.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.450+0000 7fbcbcd80700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbcb812a270 0x7fbcb8128870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33456/0 (socket says 172.21.15.3:33456) 2023-10-03T10:09:29.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.450+0000 7fbcbdd82700 1 -- 172.21.15.3:0/767181646 learned_addr learned my addr 172.21.15.3:0/767181646 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:29.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.451+0000 7fbcbdd82700 1 -- 172.21.15.3:0/767181646 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbcb8062a90 msgr2=0x7fbcb8129d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:29.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.451+0000 7fbcbdd82700 1 --2- 172.21.15.3:0/767181646 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbcb8062a90 0x7fbcb8129d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:29.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.451+0000 7fbcbdd82700 1 -- 172.21.15.3:0/767181646 >> v2:172.21.15.3:3300/0 conn(0x7fbcb812a270 msgr2=0x7fbcb8128870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:29.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.451+0000 7fbcbdd82700 1 --2- 172.21.15.3:0/767181646 >> v2:172.21.15.3:3300/0 conn(0x7fbcb812a270 0x7fbcb8128870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:29.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.451+0000 7fbcbdd82700 1 -- 172.21.15.3:0/767181646 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb8094f50 con 0x7fbcb8128d60 2023-10-03T10:09:29.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.451+0000 7fbcbcd80700 1 --2- 172.21.15.3:0/767181646 >> v2:172.21.15.3:3300/0 conn(0x7fbcb812a270 0x7fbcb8128870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:29.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.451+0000 7fbcbdd82700 1 --2- 172.21.15.3:0/767181646 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbcb8128d60 0x7fbcb8129210 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fbcb0006fd0 tx=0x7fbcb00090f0).ready entity=mon.1 client_cookie=1142569aef7ead29 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:29.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.451+0000 7fbcaffff700 1 -- 172.21.15.3:0/767181646 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb000f510 con 0x7fbcb8128d60 2023-10-03T10:09:29.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.451+0000 7fbcaffff700 1 -- 172.21.15.3:0/767181646 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb000f670 con 0x7fbcb8128d60 2023-10-03T10:09:29.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.451+0000 7fbcaffff700 1 -- 172.21.15.3:0/767181646 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb000fa10 con 0x7fbcb8128d60 2023-10-03T10:09:29.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.452+0000 7fbcbf7e5700 1 -- 172.21.15.3:0/767181646 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbcb8128d60 msgr2=0x7fbcb8129210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:29.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.452+0000 7fbcbf7e5700 1 --2- 172.21.15.3:0/767181646 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbcb8128d60 0x7fbcb8129210 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fbcb0006fd0 tx=0x7fbcb00090f0).stop 2023-10-03T10:09:29.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.452+0000 7fbcbf7e5700 1 -- 172.21.15.3:0/767181646 shutdown_connections 2023-10-03T10:09:29.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.452+0000 7fbcbf7e5700 1 --2- 172.21.15.3:0/767181646 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbcb8128d60 0x7fbcb8129210 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:29.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.452+0000 7fbcbf7e5700 1 --2- 172.21.15.3:0/767181646 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbcb8062a90 0x7fbcb8129d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:29.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.452+0000 7fbcbf7e5700 1 --2- 172.21.15.3:0/767181646 >> v2:172.21.15.3:3300/0 conn(0x7fbcb812a270 0x7fbcb8128870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:29.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.452+0000 7fbcbf7e5700 1 -- 172.21.15.3:0/767181646 >> 172.21.15.3:0/767181646 conn(0x7fbcb80598d0 msgr2=0x7fbcb8097460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:29.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.452+0000 7fbcbf7e5700 1 -- 172.21.15.3:0/767181646 shutdown_connections 2023-10-03T10:09:29.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.452+0000 7fbcbf7e5700 1 -- 172.21.15.3:0/767181646 wait complete. 2023-10-03T10:09:29.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.453+0000 7fbcbf7e5700 1 Processor -- start 2023-10-03T10:09:29.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.454+0000 7fbcbf7e5700 1 -- start start 2023-10-03T10:09:29.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.455+0000 7fbcbf7e5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbcb8062a90 0x7fbcb8116f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:29.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.455+0000 7fbcbf7e5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbcb8128d60 0x7fbcb81173f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:29.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.455+0000 7fbcbf7e5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcb812a270 0x7fbcb81178e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:29.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.455+0000 7fbcbf7e5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb8118ec0 con 0x7fbcb812a270 2023-10-03T10:09:29.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.455+0000 7fbcbf7e5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb8099e20 con 0x7fbcb8062a90 2023-10-03T10:09:29.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.455+0000 7fbcbf7e5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb8119d90 con 0x7fbcb8128d60 2023-10-03T10:09:29.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.455+0000 7fbcbd581700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbcb8062a90 0x7fbcb8116f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:29.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.455+0000 7fbcbdd82700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcb812a270 0x7fbcb81178e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:29.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.455+0000 7fbcbcd80700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbcb8128d60 0x7fbcb81173f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:29.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.455+0000 7fbcbd581700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbcb8062a90 0x7fbcb8116f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52526/0 (socket says 172.21.15.3:52526) 2023-10-03T10:09:29.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.455+0000 7fbcbd581700 1 -- 172.21.15.3:0/589084816 learned_addr learned my addr 172.21.15.3:0/589084816 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:29.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.455+0000 7fbcbdd82700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcb812a270 0x7fbcb81178e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33472/0 (socket says 172.21.15.3:33472) 2023-10-03T10:09:29.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.455+0000 7fbcbcd80700 1 -- 172.21.15.3:0/589084816 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbcb8062a90 msgr2=0x7fbcb8116f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:29.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.455+0000 7fbcbcd80700 1 --2- 172.21.15.3:0/589084816 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbcb8062a90 0x7fbcb8116f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:29.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.455+0000 7fbcbcd80700 1 -- 172.21.15.3:0/589084816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcb812a270 msgr2=0x7fbcb81178e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:29.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.455+0000 7fbcbcd80700 1 --2- 172.21.15.3:0/589084816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcb812a270 0x7fbcb81178e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:29.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.455+0000 7fbcbcd80700 1 -- 172.21.15.3:0/589084816 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca80092a0 con 0x7fbcb8128d60 2023-10-03T10:09:29.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.455+0000 7fbcbd581700 1 --2- 172.21.15.3:0/589084816 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbcb8062a90 0x7fbcb8116f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:29.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.455+0000 7fbcbdd82700 1 --2- 172.21.15.3:0/589084816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcb812a270 0x7fbcb81178e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:29.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.455+0000 7fbcbcd80700 1 --2- 172.21.15.3:0/589084816 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbcb8128d60 0x7fbcb81173f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fbca800e950 tx=0x7fbca8009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:29.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.455+0000 7fbcad7fa700 1 -- 172.21.15.3:0/589084816 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca8009580 con 0x7fbcb8128d60 2023-10-03T10:09:29.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.455+0000 7fbcad7fa700 1 -- 172.21.15.3:0/589084816 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca8012600 con 0x7fbcb8128d60 2023-10-03T10:09:29.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.456+0000 7fbcbf7e5700 1 -- 172.21.15.3:0/589084816 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb0009370 con 0x7fbcb8128d60 2023-10-03T10:09:29.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.456+0000 7fbcad7fa700 1 -- 172.21.15.3:0/589084816 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca80129a0 con 0x7fbcb8128d60 2023-10-03T10:09:29.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.456+0000 7fbcbf7e5700 1 -- 172.21.15.3:0/589084816 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb811aa10 con 0x7fbcb8128d60 2023-10-03T10:09:29.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.457+0000 7fbcad7fa700 1 -- 172.21.15.3:0/589084816 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fbca801b400 con 0x7fbcb8128d60 2023-10-03T10:09:29.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.457+0000 7fbcbf7e5700 1 -- 172.21.15.3:0/589084816 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcb805a260 con 0x7fbcb8128d60 2023-10-03T10:09:29.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.457+0000 7fbcad7fa700 1 -- 172.21.15.3:0/589084816 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7fbca801a770 con 0x7fbcb8128d60 2023-10-03T10:09:29.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.459+0000 7fbcad7fa700 1 -- 172.21.15.3:0/589084816 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fbca803ae50 con 0x7fbcb8128d60 2023-10-03T10:09:29.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.543+0000 7fbcbf7e5700 1 -- 172.21.15.3:0/589084816 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd new", "uuid": "c8e58877-6762-4741-a8c1-567ebc97b708", "id": 1} v 0) v1 -- 0x7fbcb8046c20 con 0x7fbcb8128d60 2023-10-03T10:09:29.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.551+0000 7fbcad7fa700 1 -- 172.21.15.3:0/589084816 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c8e58877-6762-4741-a8c1-567ebc97b708", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbca8029030 con 0x7fbcb8128d60 2023-10-03T10:09:29.552 INFO:teuthology.orchestra.run.smithi003.stdout:1 2023-10-03T10:09:29.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.552+0000 7fbcbf7e5700 1 -- 172.21.15.3:0/589084816 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbcb8128d60 msgr2=0x7fbcb81173f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:29.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.552+0000 7fbcbf7e5700 1 --2- 172.21.15.3:0/589084816 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbcb8128d60 0x7fbcb81173f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fbca800e950 tx=0x7fbca8009060).stop 2023-10-03T10:09:29.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.553+0000 7fbcbf7e5700 1 -- 172.21.15.3:0/589084816 shutdown_connections 2023-10-03T10:09:29.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.553+0000 7fbcbf7e5700 1 --2- 172.21.15.3:0/589084816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcb812a270 0x7fbcb81178e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:29.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.553+0000 7fbcbf7e5700 1 --2- 172.21.15.3:0/589084816 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbcb8062a90 0x7fbcb8116f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:29.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.553+0000 7fbcbf7e5700 1 --2- 172.21.15.3:0/589084816 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbcb8128d60 0x7fbcb81173f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:29.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.553+0000 7fbcbf7e5700 1 -- 172.21.15.3:0/589084816 >> 172.21.15.3:0/589084816 conn(0x7fbcb80598d0 msgr2=0x7fbcb812b040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:29.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.553+0000 7fbcbf7e5700 1 -- 172.21.15.3:0/589084816 shutdown_connections 2023-10-03T10:09:29.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.553+0000 7fbcbf7e5700 1 -- 172.21.15.3:0/589084816 wait complete. 2023-10-03T10:09:29.566 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph osd new 68a2a21a-7a8e-4f82-8433-5201faa50777 2 2023-10-03T10:09:29.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.694+0000 7efcdd24b700 1 Processor -- start 2023-10-03T10:09:29.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.694+0000 7efcdd24b700 1 -- start start 2023-10-03T10:09:29.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.694+0000 7efcdd24b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efcd8099480 0x7efcd80987c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:29.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.694+0000 7efcdd24b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efcd812fc70 0x7efcd8132060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:29.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.694+0000 7efcdd24b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd8091ae0 con 0x7efcd8062a90 2023-10-03T10:09:29.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.694+0000 7efcdd24b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd8062e70 con 0x7efcd8099480 2023-10-03T10:09:29.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.694+0000 7efcdd24b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efcd809a0c0 con 0x7efcd812fc70 2023-10-03T10:09:29.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.694+0000 7efcd659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efcd8099480 0x7efcd80987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:29.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.694+0000 7efcd659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efcd8099480 0x7efcd80987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52540/0 (socket says 172.21.15.3:52540) 2023-10-03T10:09:29.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.694+0000 7efcd759e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efcd812fc70 0x7efcd8132060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:29.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.694+0000 7efcd659c700 1 -- 172.21.15.3:0/3715713878 learned_addr learned my addr 172.21.15.3:0/3715713878 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:29.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.695+0000 7efcd5d9b700 1 -- 172.21.15.3:0/3715713878 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3568076356 0 0) 0x7efcd8091ae0 con 0x7efcd8062a90 2023-10-03T10:09:29.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.695+0000 7efcd759e700 1 -- 172.21.15.3:0/3715713878 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efcd8099480 msgr2=0x7efcd80987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:29.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.695+0000 7efcd759e700 1 --2- 172.21.15.3:0/3715713878 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efcd8099480 0x7efcd80987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:29.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.695+0000 7efcd759e700 1 -- 172.21.15.3:0/3715713878 >> v1:172.21.15.3:6789/0 conn(0x7efcd8062a90 legacy=0x7efcd8098d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:29.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.695+0000 7efcd759e700 1 -- 172.21.15.3:0/3715713878 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd80929c0 con 0x7efcd812fc70 2023-10-03T10:09:29.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.695+0000 7efcd759e700 1 --2- 172.21.15.3:0/3715713878 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efcd812fc70 0x7efcd8132060 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7efcd0009e90 tx=0x7efcd0009780).ready entity=mon.2 client_cookie=410ede546a12899c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:29.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.695+0000 7efcd5d9b700 1 -- 172.21.15.3:0/3715713878 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd000e070 con 0x7efcd812fc70 2023-10-03T10:09:29.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.695+0000 7efcd5d9b700 1 -- 172.21.15.3:0/3715713878 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd000bb60 con 0x7efcd812fc70 2023-10-03T10:09:29.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.695+0000 7efcd5d9b700 1 -- 172.21.15.3:0/3715713878 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0012410 con 0x7efcd812fc70 2023-10-03T10:09:29.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.696+0000 7efcdd24b700 1 -- 172.21.15.3:0/3715713878 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efcd812fc70 msgr2=0x7efcd8132060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:29.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.696+0000 7efcdd24b700 1 --2- 172.21.15.3:0/3715713878 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efcd812fc70 0x7efcd8132060 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7efcd0009e90 tx=0x7efcd0009780).stop 2023-10-03T10:09:29.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.696+0000 7efcdd24b700 1 -- 172.21.15.3:0/3715713878 shutdown_connections 2023-10-03T10:09:29.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.696+0000 7efcdd24b700 1 --2- 172.21.15.3:0/3715713878 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efcd812fc70 0x7efcd8132060 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:29.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.696+0000 7efcdd24b700 1 --2- 172.21.15.3:0/3715713878 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efcd8099480 0x7efcd80987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:29.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.696+0000 7efcdd24b700 1 -- 172.21.15.3:0/3715713878 >> 172.21.15.3:0/3715713878 conn(0x7efcd80598d0 msgr2=0x7efcd8097460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:29.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.696+0000 7efcdd24b700 1 -- 172.21.15.3:0/3715713878 shutdown_connections 2023-10-03T10:09:29.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.696+0000 7efcdd24b700 1 -- 172.21.15.3:0/3715713878 wait complete. 2023-10-03T10:09:29.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.697+0000 7efcdd24b700 1 Processor -- start 2023-10-03T10:09:29.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.697+0000 7efcdd24b700 1 -- start start 2023-10-03T10:09:29.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.698+0000 7efcdd24b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efcd8062a90 0x7efcd812d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:29.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.698+0000 7efcdd24b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcd8099480 0x7efcd812ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:29.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.698+0000 7efcdd24b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efcd812fc70 0x7efcd812b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:29.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.698+0000 7efcdd24b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd812bab0 con 0x7efcd8099480 2023-10-03T10:09:29.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.698+0000 7efcdd24b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd812cec0 con 0x7efcd8062a90 2023-10-03T10:09:29.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.698+0000 7efcdd24b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efcd812d460 con 0x7efcd812fc70 2023-10-03T10:09:29.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.698+0000 7efcd659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcd8099480 0x7efcd812ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:29.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.698+0000 7efcd759e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efcd812fc70 0x7efcd812b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:29.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.698+0000 7efcd659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcd8099480 0x7efcd812ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33474/0 (socket says 172.21.15.3:33474) 2023-10-03T10:09:29.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.698+0000 7efcd659c700 1 -- 172.21.15.3:0/1415380362 learned_addr learned my addr 172.21.15.3:0/1415380362 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:29.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.698+0000 7efcd6d9d700 1 --2- 172.21.15.3:0/1415380362 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efcd8062a90 0x7efcd812d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:29.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.698+0000 7efcd659c700 1 -- 172.21.15.3:0/1415380362 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efcd812fc70 msgr2=0x7efcd812b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:29.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.698+0000 7efcd659c700 1 --2- 172.21.15.3:0/1415380362 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efcd812fc70 0x7efcd812b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:29.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.698+0000 7efcd659c700 1 -- 172.21.15.3:0/1415380362 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efcd8062a90 msgr2=0x7efcd812d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:29.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.698+0000 7efcd659c700 1 --2- 172.21.15.3:0/1415380362 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efcd8062a90 0x7efcd812d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:29.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.698+0000 7efcd659c700 1 -- 172.21.15.3:0/1415380362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd0009a20 con 0x7efcd8099480 2023-10-03T10:09:29.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.698+0000 7efcd759e700 1 --2- 172.21.15.3:0/1415380362 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efcd812fc70 0x7efcd812b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:29.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.699+0000 7efcd659c700 1 --2- 172.21.15.3:0/1415380362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcd8099480 0x7efcd812ac50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7efccc00e900 tx=0x7efccc0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:29.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.699+0000 7efcbeffd700 1 -- 172.21.15.3:0/1415380362 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc012600 con 0x7efcd8099480 2023-10-03T10:09:29.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.699+0000 7efcbeffd700 1 -- 172.21.15.3:0/1415380362 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efccc012760 con 0x7efcd8099480 2023-10-03T10:09:29.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.699+0000 7efcbeffd700 1 -- 172.21.15.3:0/1415380362 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc012ae0 con 0x7efcd8099480 2023-10-03T10:09:29.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.699+0000 7efcdd24b700 1 -- 172.21.15.3:0/1415380362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd812ca90 con 0x7efcd8099480 2023-10-03T10:09:29.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.699+0000 7efcdd24b700 1 -- 172.21.15.3:0/1415380362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd812b7f0 con 0x7efcd8099480 2023-10-03T10:09:29.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.699+0000 7efcbeffd700 1 -- 172.21.15.3:0/1415380362 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7efccc01b660 con 0x7efcd8099480 2023-10-03T10:09:29.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.699+0000 7efcbeffd700 1 -- 172.21.15.3:0/1415380362 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7efccc01bd40 con 0x7efcd8099480 2023-10-03T10:09:29.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.700+0000 7efcdd24b700 1 -- 172.21.15.3:0/1415380362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd8131590 con 0x7efcd8099480 2023-10-03T10:09:29.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.702+0000 7efcbeffd700 1 -- 172.21.15.3:0/1415380362 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7efccc03ae50 con 0x7efcd8099480 2023-10-03T10:09:29.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.789+0000 7efcdd24b700 1 -- 172.21.15.3:0/1415380362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd new", "uuid": "68a2a21a-7a8e-4f82-8433-5201faa50777", "id": 2} v 0) v1 -- 0x7efcd805a260 con 0x7efcd8099480 2023-10-03T10:09:29.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.800+0000 7efcbeffd700 1 -- 172.21.15.3:0/1415380362 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "68a2a21a-7a8e-4f82-8433-5201faa50777", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7efccc029030 con 0x7efcd8099480 2023-10-03T10:09:29.888 INFO:teuthology.orchestra.run.smithi003.stdout:2 2023-10-03T10:09:29.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.802+0000 7efcdd24b700 1 -- 172.21.15.3:0/1415380362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcd8099480 msgr2=0x7efcd812ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:29.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.802+0000 7efcdd24b700 1 --2- 172.21.15.3:0/1415380362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcd8099480 0x7efcd812ac50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7efccc00e900 tx=0x7efccc0090f0).stop 2023-10-03T10:09:29.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.802+0000 7efcdd24b700 1 -- 172.21.15.3:0/1415380362 shutdown_connections 2023-10-03T10:09:29.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.802+0000 7efcdd24b700 1 --2- 172.21.15.3:0/1415380362 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efcd812fc70 0x7efcd812b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:29.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.802+0000 7efcdd24b700 1 --2- 172.21.15.3:0/1415380362 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efcd8062a90 0x7efcd812d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:29.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.802+0000 7efcdd24b700 1 --2- 172.21.15.3:0/1415380362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcd8099480 0x7efcd812ac50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:29.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.802+0000 7efcdd24b700 1 -- 172.21.15.3:0/1415380362 >> 172.21.15.3:0/1415380362 conn(0x7efcd80598d0 msgr2=0x7efcd8096de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:29.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.802+0000 7efcdd24b700 1 -- 172.21.15.3:0/1415380362 shutdown_connections 2023-10-03T10:09:29.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:29.802+0000 7efcdd24b700 1 -- 172.21.15.3:0/1415380362 wait complete. 2023-10-03T10:09:29.930 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph osd new c4df2903-4e1e-4208-bf77-95b9fa749241 3 2023-10-03T10:09:29.930 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:09:29.888+0000 7f57e47cb680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-03T10:09:30.015 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:09:30.014+0000 7f57e47cb680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-03T10:09:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.060+0000 7fbeb51cc700 1 Processor -- start 2023-10-03T10:09:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.060+0000 7fbeb51cc700 1 -- start start 2023-10-03T10:09:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.060+0000 7fbeb51cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbeb0062a90 0x7fbeb0129d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:30.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.060+0000 7fbeb51cc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbeb012a270 0x7fbeb0128870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:30.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.060+0000 7fbeb51cc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbeb0128d60 0x7fbeb0129210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:30.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.060+0000 7fbeb51cc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb0062e70 con 0x7fbeb0128d60 2023-10-03T10:09:30.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.060+0000 7fbeb51cc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb0099c80 con 0x7fbeb0062a90 2023-10-03T10:09:30.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.060+0000 7fbeb51cc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb00634b0 con 0x7fbeb012a270 2023-10-03T10:09:30.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.060+0000 7fbeae59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbeb012a270 0x7fbeb0128870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:30.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.060+0000 7fbeaed9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbeb0062a90 0x7fbeb0129d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:30.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.060+0000 7fbeaf59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbeb0128d60 0x7fbeb0129210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:30.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.060+0000 7fbeaed9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbeb0062a90 0x7fbeb0129d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52548/0 (socket says 172.21.15.3:52548) 2023-10-03T10:09:30.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.060+0000 7fbeae59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbeb012a270 0x7fbeb0128870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50232/0 (socket says 172.21.15.3:50232) 2023-10-03T10:09:30.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.060+0000 7fbeaed9d700 1 -- 172.21.15.3:0/2433078427 learned_addr learned my addr 172.21.15.3:0/2433078427 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:30.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.061+0000 7fbeaed9d700 1 -- 172.21.15.3:0/2433078427 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbeb012a270 msgr2=0x7fbeb0128870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:30.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.061+0000 7fbeaed9d700 1 --2- 172.21.15.3:0/2433078427 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbeb012a270 0x7fbeb0128870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:30.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.061+0000 7fbeaed9d700 1 -- 172.21.15.3:0/2433078427 >> v2:172.21.15.3:3300/0 conn(0x7fbeb0128d60 msgr2=0x7fbeb0129210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:30.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.061+0000 7fbeaed9d700 1 --2- 172.21.15.3:0/2433078427 >> v2:172.21.15.3:3300/0 conn(0x7fbeb0128d60 0x7fbeb0129210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:30.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.061+0000 7fbeaed9d700 1 -- 172.21.15.3:0/2433078427 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb0094f50 con 0x7fbeb0062a90 2023-10-03T10:09:30.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.061+0000 7fbeaf59e700 1 --2- 172.21.15.3:0/2433078427 >> v2:172.21.15.3:3300/0 conn(0x7fbeb0128d60 0x7fbeb0129210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:30.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.061+0000 7fbeae59c700 1 --2- 172.21.15.3:0/2433078427 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbeb012a270 0x7fbeb0128870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:30.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.061+0000 7fbeaed9d700 1 --2- 172.21.15.3:0/2433078427 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbeb0062a90 0x7fbeb0129d80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fbea0006fd0 tx=0x7fbea00090f0).ready entity=mon.1 client_cookie=7a7818d8db6fff98 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:30.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.061+0000 7fbeadd9b700 1 -- 172.21.15.3:0/2433078427 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea000f510 con 0x7fbeb0062a90 2023-10-03T10:09:30.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.061+0000 7fbeadd9b700 1 -- 172.21.15.3:0/2433078427 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea000f670 con 0x7fbeb0062a90 2023-10-03T10:09:30.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.061+0000 7fbeadd9b700 1 -- 172.21.15.3:0/2433078427 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea000fa10 con 0x7fbeb0062a90 2023-10-03T10:09:30.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.062+0000 7fbeb51cc700 1 -- 172.21.15.3:0/2433078427 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbeb0062a90 msgr2=0x7fbeb0129d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:30.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.062+0000 7fbeb51cc700 1 --2- 172.21.15.3:0/2433078427 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbeb0062a90 0x7fbeb0129d80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fbea0006fd0 tx=0x7fbea00090f0).stop 2023-10-03T10:09:30.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.062+0000 7fbeb51cc700 1 -- 172.21.15.3:0/2433078427 shutdown_connections 2023-10-03T10:09:30.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.062+0000 7fbeb51cc700 1 --2- 172.21.15.3:0/2433078427 >> v2:172.21.15.3:3300/0 conn(0x7fbeb0128d60 0x7fbeb0129210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:30.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.062+0000 7fbeb51cc700 1 --2- 172.21.15.3:0/2433078427 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbeb0062a90 0x7fbeb0129d80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:30.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.062+0000 7fbeb51cc700 1 --2- 172.21.15.3:0/2433078427 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbeb012a270 0x7fbeb0128870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:30.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.062+0000 7fbeb51cc700 1 -- 172.21.15.3:0/2433078427 >> 172.21.15.3:0/2433078427 conn(0x7fbeb00598d0 msgr2=0x7fbeb0097460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:30.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.062+0000 7fbeb51cc700 1 -- 172.21.15.3:0/2433078427 shutdown_connections 2023-10-03T10:09:30.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.062+0000 7fbeb51cc700 1 -- 172.21.15.3:0/2433078427 wait complete. 2023-10-03T10:09:30.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.063+0000 7fbeb51cc700 1 Processor -- start 2023-10-03T10:09:30.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.063+0000 7fbeb51cc700 1 -- start start 2023-10-03T10:09:30.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.064+0000 7fbeb51cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbeb0062a90 0x7fbeb00683e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:30.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.064+0000 7fbeb51cc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbeb0128d60 0x7fbeb00688d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:30.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.064+0000 7fbeb51cc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbeb012a270 0x7fbeb0135c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:30.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.064+0000 7fbeb51cc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb0111cf0 con 0x7fbeb012a270 2023-10-03T10:09:30.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.064+0000 7fbeb51cc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb0111a30 con 0x7fbeb0062a90 2023-10-03T10:09:30.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.064+0000 7fbeb51cc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb0099e20 con 0x7fbeb0128d60 2023-10-03T10:09:30.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.064+0000 7fbeaf59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbeb012a270 0x7fbeb0135c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:30.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.064+0000 7fbeae59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbeb0128d60 0x7fbeb00688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:30.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.064+0000 7fbeaed9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbeb0062a90 0x7fbeb00683e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:30.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.064+0000 7fbeaf59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbeb012a270 0x7fbeb0135c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33486/0 (socket says 172.21.15.3:33486) 2023-10-03T10:09:30.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.064+0000 7fbeaf59e700 1 -- 172.21.15.3:0/2583565449 learned_addr learned my addr 172.21.15.3:0/2583565449 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:30.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.064+0000 7fbeaed9d700 1 -- 172.21.15.3:0/2583565449 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbeb0128d60 msgr2=0x7fbeb00688d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:30.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.064+0000 7fbeaed9d700 1 --2- 172.21.15.3:0/2583565449 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbeb0128d60 0x7fbeb00688d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:30.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.064+0000 7fbeaed9d700 1 -- 172.21.15.3:0/2583565449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbeb012a270 msgr2=0x7fbeb0135c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:30.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.064+0000 7fbeaed9d700 1 --2- 172.21.15.3:0/2583565449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbeb012a270 0x7fbeb0135c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:30.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.064+0000 7fbeaed9d700 1 -- 172.21.15.3:0/2583565449 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea8009a20 con 0x7fbeb0062a90 2023-10-03T10:09:30.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.064+0000 7fbeaf59e700 1 --2- 172.21.15.3:0/2583565449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbeb012a270 0x7fbeb0135c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:30.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.064+0000 7fbeae59c700 1 --2- 172.21.15.3:0/2583565449 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbeb0128d60 0x7fbeb00688d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:30.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.064+0000 7fbeaed9d700 1 --2- 172.21.15.3:0/2583565449 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbeb0062a90 0x7fbeb00683e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fbea0014000 tx=0x7fbea0017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:30.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.065+0000 7fbe96ffd700 1 -- 172.21.15.3:0/2583565449 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea0006d90 con 0x7fbeb0062a90 2023-10-03T10:09:30.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.065+0000 7fbe96ffd700 1 -- 172.21.15.3:0/2583565449 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea0006930 con 0x7fbeb0062a90 2023-10-03T10:09:30.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.065+0000 7fbeb51cc700 1 -- 172.21.15.3:0/2583565449 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbea4011b40 con 0x7fbeb0062a90 2023-10-03T10:09:30.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.065+0000 7fbe96ffd700 1 -- 172.21.15.3:0/2583565449 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea000f7e0 con 0x7fbeb0062a90 2023-10-03T10:09:30.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.065+0000 7fbeb51cc700 1 -- 172.21.15.3:0/2583565449 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea0009370 con 0x7fbeb0062a90 2023-10-03T10:09:30.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.065+0000 7fbe96ffd700 1 -- 172.21.15.3:0/2583565449 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fbea0021070 con 0x7fbeb0062a90 2023-10-03T10:09:30.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.065+0000 7fbe96ffd700 1 -- 172.21.15.3:0/2583565449 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7fbea000f940 con 0x7fbeb0062a90 2023-10-03T10:09:30.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.066+0000 7fbeb51cc700 1 -- 172.21.15.3:0/2583565449 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb0099240 con 0x7fbeb0062a90 2023-10-03T10:09:30.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.068+0000 7fbe96ffd700 1 -- 172.21.15.3:0/2583565449 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fbea0043e50 con 0x7fbeb0062a90 2023-10-03T10:09:30.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.154+0000 7fbeb51cc700 1 -- 172.21.15.3:0/2583565449 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c4df2903-4e1e-4208-bf77-95b9fa749241", "id": 3} v 0) v1 -- 0x7fbeb005a260 con 0x7fbeb0062a90 2023-10-03T10:09:30.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.165+0000 7fbe96ffd700 1 -- 172.21.15.3:0/2583565449 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c4df2903-4e1e-4208-bf77-95b9fa749241", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbea0032020 con 0x7fbeb0062a90 2023-10-03T10:09:30.204 INFO:teuthology.orchestra.run.smithi003.stdout:3 2023-10-03T10:09:30.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.166+0000 7fbeb51cc700 1 -- 172.21.15.3:0/2583565449 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbeb0062a90 msgr2=0x7fbeb00683e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:30.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.166+0000 7fbeb51cc700 1 --2- 172.21.15.3:0/2583565449 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbeb0062a90 0x7fbeb00683e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fbea0014000 tx=0x7fbea0017000).stop 2023-10-03T10:09:30.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.166+0000 7fbeb51cc700 1 -- 172.21.15.3:0/2583565449 shutdown_connections 2023-10-03T10:09:30.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.166+0000 7fbeb51cc700 1 --2- 172.21.15.3:0/2583565449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbeb012a270 0x7fbeb0135c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:30.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.166+0000 7fbeb51cc700 1 --2- 172.21.15.3:0/2583565449 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbeb0062a90 0x7fbeb00683e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:30.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.166+0000 7fbeb51cc700 1 --2- 172.21.15.3:0/2583565449 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbeb0128d60 0x7fbeb00688d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:30.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.166+0000 7fbeb51cc700 1 -- 172.21.15.3:0/2583565449 >> 172.21.15.3:0/2583565449 conn(0x7fbeb00598d0 msgr2=0x7fbeb0127600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:30.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.166+0000 7fbeb51cc700 1 -- 172.21.15.3:0/2583565449 shutdown_connections 2023-10-03T10:09:30.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.166+0000 7fbeb51cc700 1 -- 172.21.15.3:0/2583565449 wait complete. 2023-10-03T10:09:30.206 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:09:30.170+0000 7f57e47cb680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-03T10:09:30.207 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-03T10:09:30.207 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-03T10:09:30.209 INFO:tasks.ceph.osd.0:Started 2023-10-03T10:09:30.209 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-03T10:09:30.209 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-03T10:09:30.211 INFO:tasks.ceph.osd.1:Started 2023-10-03T10:09:30.211 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-03T10:09:30.212 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-03T10:09:30.213 INFO:tasks.ceph.osd.2:Started 2023-10-03T10:09:30.214 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-03T10:09:30.214 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-03T10:09:30.216 INFO:tasks.ceph.osd.3:Started 2023-10-03T10:09:30.216 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-03T10:09:30.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.398+0000 7f3786907700 1 Processor -- start 2023-10-03T10:09:30.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.398+0000 7f3786907700 1 -- start start 2023-10-03T10:09:30.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.398+0000 7f3786907700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3780065340 0x7f37800985b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:30.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.398+0000 7f3786907700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f37800991b0 0x7f37801324b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:30.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.398+0000 7f3786907700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37800962f0 con 0x7f3780095e40 2023-10-03T10:09:30.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.398+0000 7f3786907700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3780065720 con 0x7f3780065340 2023-10-03T10:09:30.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.398+0000 7f3786907700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f378009a600 con 0x7f37800991b0 2023-10-03T10:09:30.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.398+0000 7f3784ea4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f37800991b0 0x7f37801324b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:30.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.398+0000 7f377ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3780065340 0x7f37800985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:30.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.399+0000 7f3784ea4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f37800991b0 0x7f37801324b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50258/0 (socket says 172.21.15.3:50258) 2023-10-03T10:09:30.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.399+0000 7f377f7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3780095e40 0x7f3780098aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33460/0 (socket says 172.21.15.3:33460) 2023-10-03T10:09:30.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.399+0000 7f377ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3780065340 0x7f37800985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33524/0 (socket says 172.21.15.3:33524) 2023-10-03T10:09:30.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.399+0000 7f3784ea4700 1 -- 172.21.15.3:0/1442234253 learned_addr learned my addr 172.21.15.3:0/1442234253 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:30.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.399+0000 7f377ffff700 1 -- 172.21.15.3:0/1442234253 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f37800991b0 msgr2=0x7f37801324b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:30.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.399+0000 7f377ffff700 1 --2- 172.21.15.3:0/1442234253 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f37800991b0 0x7f37801324b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:30.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.399+0000 7f377ffff700 1 -- 172.21.15.3:0/1442234253 >> v1:172.21.15.3:6789/0 conn(0x7f3780095e40 legacy=0x7f3780098aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:30.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.399+0000 7f377effd700 1 -- 172.21.15.3:0/1442234253 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 130966030 0 0) 0x7f37800962f0 con 0x7f3780095e40 2023-10-03T10:09:30.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.399+0000 7f377ffff700 1 -- 172.21.15.3:0/1442234253 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37800995e0 con 0x7f3780065340 2023-10-03T10:09:30.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.399+0000 7f377ffff700 1 --2- 172.21.15.3:0/1442234253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3780065340 0x7f37800985b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f3770009760 tx=0x7f3770009060).ready entity=mon.0 client_cookie=747c2e53379bde5e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:30.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.399+0000 7f377effd700 1 -- 172.21.15.3:0/1442234253 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377000f510 con 0x7f3780065340 2023-10-03T10:09:30.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.399+0000 7f377effd700 1 -- 172.21.15.3:0/1442234253 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f377000f670 con 0x7f3780065340 2023-10-03T10:09:30.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.399+0000 7f377effd700 1 -- 172.21.15.3:0/1442234253 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377000f9f0 con 0x7f3780065340 2023-10-03T10:09:30.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.400+0000 7f3786907700 1 -- 172.21.15.3:0/1442234253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3780065340 msgr2=0x7f37800985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:30.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.400+0000 7f3786907700 1 --2- 172.21.15.3:0/1442234253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3780065340 0x7f37800985b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f3770009760 tx=0x7f3770009060).stop 2023-10-03T10:09:30.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.400+0000 7f3786907700 1 -- 172.21.15.3:0/1442234253 shutdown_connections 2023-10-03T10:09:30.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.400+0000 7f3786907700 1 --2- 172.21.15.3:0/1442234253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3780065340 0x7f37800985b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:30.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.400+0000 7f3786907700 1 --2- 172.21.15.3:0/1442234253 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f37800991b0 0x7f37801324b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:30.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.400+0000 7f3786907700 1 -- 172.21.15.3:0/1442234253 >> 172.21.15.3:0/1442234253 conn(0x7f378009dc40 msgr2=0x7f378009e050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:30.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.400+0000 7f3786907700 1 -- 172.21.15.3:0/1442234253 shutdown_connections 2023-10-03T10:09:30.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.400+0000 7f3786907700 1 -- 172.21.15.3:0/1442234253 wait complete. 2023-10-03T10:09:30.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.401+0000 7f3786907700 1 Processor -- start 2023-10-03T10:09:30.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.402+0000 7f3786907700 1 -- start start 2023-10-03T10:09:30.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.402+0000 7f3786907700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3780065340 0x7f378011e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:30.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.402+0000 7f3786907700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3780095e40 0x7f378011a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:30.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.402+0000 7f3786907700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37800991b0 0x7f378011aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:30.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.402+0000 7f3786907700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3780099b60 con 0x7f37800991b0 2023-10-03T10:09:30.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.402+0000 7f3786907700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f378011dd50 con 0x7f3780095e40 2023-10-03T10:09:30.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.402+0000 7f3786907700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f378011da90 con 0x7f3780065340 2023-10-03T10:09:30.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.402+0000 7f3784ea4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37800991b0 0x7f378011aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:30.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.402+0000 7f3784ea4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37800991b0 0x7f378011aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33538/0 (socket says 172.21.15.3:33538) 2023-10-03T10:09:30.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.402+0000 7f3784ea4700 1 -- 172.21.15.3:0/4048320506 learned_addr learned my addr 172.21.15.3:0/4048320506 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:30.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.402+0000 7f377ffff700 1 --2- 172.21.15.3:0/4048320506 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3780065340 0x7f378011e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:30.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.403+0000 7f377f7fe700 1 --2- 172.21.15.3:0/4048320506 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3780095e40 0x7f378011a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:30.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.403+0000 7f3784ea4700 1 -- 172.21.15.3:0/4048320506 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3780065340 msgr2=0x7f378011e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:30.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.403+0000 7f3784ea4700 1 --2- 172.21.15.3:0/4048320506 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3780065340 0x7f378011e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:30.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.403+0000 7f3784ea4700 1 -- 172.21.15.3:0/4048320506 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3780095e40 msgr2=0x7f378011a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:30.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.403+0000 7f3784ea4700 1 --2- 172.21.15.3:0/4048320506 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3780095e40 0x7f378011a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:30.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.403+0000 7f3784ea4700 1 -- 172.21.15.3:0/4048320506 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37700092a0 con 0x7f37800991b0 2023-10-03T10:09:30.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.403+0000 7f377ffff700 1 --2- 172.21.15.3:0/4048320506 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3780065340 0x7f378011e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:30.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.403+0000 7f3784ea4700 1 --2- 172.21.15.3:0/4048320506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37800991b0 0x7f378011aa40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f3778009e50 tx=0x7f377800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:30.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.403+0000 7f3767fff700 1 -- 172.21.15.3:0/4048320506 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377800c890 con 0x7f37800991b0 2023-10-03T10:09:30.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.403+0000 7f3767fff700 1 -- 172.21.15.3:0/4048320506 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f377800f040 con 0x7f37800991b0 2023-10-03T10:09:30.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.403+0000 7f3786907700 1 -- 172.21.15.3:0/4048320506 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f378011d6a0 con 0x7f37800991b0 2023-10-03T10:09:30.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.403+0000 7f3767fff700 1 -- 172.21.15.3:0/4048320506 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377800cc00 con 0x7f37800991b0 2023-10-03T10:09:30.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.403+0000 7f3786907700 1 -- 172.21.15.3:0/4048320506 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f378011c2e0 con 0x7f37800991b0 2023-10-03T10:09:30.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.403+0000 7f3767fff700 1 -- 172.21.15.3:0/4048320506 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f3778014670 con 0x7f37800991b0 2023-10-03T10:09:30.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.404+0000 7f3767fff700 1 -- 172.21.15.3:0/4048320506 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f3778007400 con 0x7f37800991b0 2023-10-03T10:09:30.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.405+0000 7f3786907700 1 -- 172.21.15.3:0/4048320506 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f378009b3a0 con 0x7f37800991b0 2023-10-03T10:09:30.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.407+0000 7f3767fff700 1 -- 172.21.15.3:0/4048320506 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f3778022060 con 0x7f37800991b0 2023-10-03T10:09:30.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.477+0000 7f3786907700 1 -- 172.21.15.3:0/4048320506 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f378011b3b0 con 0x7f37800991b0 2023-10-03T10:09:30.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.477+0000 7f3767fff700 1 -- 172.21.15.3:0/4048320506 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4109 (secure 0 0 0) 0x7f3778014b30 con 0x7f37800991b0 2023-10-03T10:09:30.478 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:30.478 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":6,"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","created":"2023-10-03T10:09:28.703441+0000","modified":"2023-10-03T10:09:30.156546+0000","last_up_change":"0.000000","last_in_change":"2023-10-03T10:09:30.156546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"cdcdd930-3c5b-4486-a021-7132f90fb6e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c8e58877-6762-4741-a8c1-567ebc97b708","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"68a2a21a-7a8e-4f82-8433-5201faa50777","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c4df2903-4e1e-4208-bf77-95b9fa749241","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-03T10:09:30.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.478+0000 7f3786907700 1 -- 172.21.15.3:0/4048320506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37800991b0 msgr2=0x7f378011aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:30.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.478+0000 7f3786907700 1 --2- 172.21.15.3:0/4048320506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37800991b0 0x7f378011aa40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f3778009e50 tx=0x7f377800b040).stop 2023-10-03T10:09:30.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.478+0000 7f3786907700 1 -- 172.21.15.3:0/4048320506 shutdown_connections 2023-10-03T10:09:30.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.478+0000 7f3786907700 1 --2- 172.21.15.3:0/4048320506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37800991b0 0x7f378011aa40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:30.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.478+0000 7f3786907700 1 --2- 172.21.15.3:0/4048320506 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3780065340 0x7f378011e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:30.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.478+0000 7f3786907700 1 --2- 172.21.15.3:0/4048320506 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3780095e40 0x7f378011a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:30.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.478+0000 7f3786907700 1 -- 172.21.15.3:0/4048320506 >> 172.21.15.3:0/4048320506 conn(0x7f378009dc40 msgr2=0x7f378009ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:30.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.479+0000 7f3786907700 1 -- 172.21.15.3:0/4048320506 shutdown_connections 2023-10-03T10:09:30.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:30.479+0000 7f3786907700 1 -- 172.21.15.3:0/4048320506 wait complete. 2023-10-03T10:09:30.492 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-03T10:09:30.492 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-03T10:09:30.584 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-10-03T10:09:30.584+0000 7fc8e6257200 -1 Falling back to public interface 2023-10-03T10:09:30.711 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:09:30.711+0000 7f57e47cb680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-03T10:09:30.826 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-10-03T10:09:30.826+0000 7f135d21b200 -1 Falling back to public interface 2023-10-03T10:09:30.847 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-10-03T10:09:30.846+0000 7f4e52a67200 -1 Falling back to public interface 2023-10-03T10:09:30.852 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-10-03T10:09:30.851+0000 7f3134721200 -1 Falling back to public interface 2023-10-03T10:09:30.895 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-03T10:09:30.915 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:09:30.915+0000 7f57e47cb680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-03T10:09:30.991 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:09:30.991+0000 7f57e47cb680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-03T10:09:31.014 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.014+0000 7fdf4383b700 1 Processor -- start 2023-10-03T10:09:31.014 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.014+0000 7fdf4383b700 1 -- start start 2023-10-03T10:09:31.015 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.015+0000 7fdf4383b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf3c062a10 0x7fdf3c128e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:31.015 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.015+0000 7fdf4383b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdf3c129330 0x7fdf3c094360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:31.015 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.015+0000 7fdf4383b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf3c096480 con 0x7fdf3c128900 2023-10-03T10:09:31.016 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.015+0000 7fdf4383b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf3c096990 con 0x7fdf3c129330 2023-10-03T10:09:31.016 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.015+0000 7fdf4383b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c095bc0 con 0x7fdf3c062a10 2023-10-03T10:09:31.016 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.015+0000 7fdf41dd8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdf3c129330 0x7fdf3c094360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:31.016 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.015+0000 7fdf415d7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf3c062a10 0x7fdf3c128e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:31.016 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.015+0000 7fdf41dd8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdf3c129330 0x7fdf3c094360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33550/0 (socket says 172.21.15.3:33550) 2023-10-03T10:09:31.016 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.015+0000 7fdf40dd6700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdf3c128900 0x7fdf3c093c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33466/0 (socket says 172.21.15.3:33466) 2023-10-03T10:09:31.016 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.015+0000 7fdf41dd8700 1 -- 172.21.15.3:0/1750901916 learned_addr learned my addr 172.21.15.3:0/1750901916 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:31.017 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.015+0000 7fdf415d7700 1 -- 172.21.15.3:0/1750901916 >> v2:172.21.15.3:3300/0 conn(0x7fdf3c129330 msgr2=0x7fdf3c094360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:31.017 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.015+0000 7fdf415d7700 1 --2- 172.21.15.3:0/1750901916 >> v2:172.21.15.3:3300/0 conn(0x7fdf3c129330 0x7fdf3c094360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:31.018 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.015+0000 7fdf415d7700 1 -- 172.21.15.3:0/1750901916 >> v1:172.21.15.3:6789/0 conn(0x7fdf3c128900 legacy=0x7fdf3c093c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:31.018 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.015+0000 7fdf415d7700 1 -- 172.21.15.3:0/1750901916 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3c129790 con 0x7fdf3c062a10 2023-10-03T10:09:31.018 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.015+0000 7fdf41dd8700 1 --2- 172.21.15.3:0/1750901916 >> v2:172.21.15.3:3300/0 conn(0x7fdf3c129330 0x7fdf3c094360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:31.018 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.016+0000 7fdf415d7700 1 --2- 172.21.15.3:0/1750901916 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf3c062a10 0x7fdf3c128e40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fdf2c009760 tx=0x7fdf2c009060).ready entity=mon.2 client_cookie=a3644c330dfed488 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:31.018 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.016+0000 7fdf3bfff700 1 -- 172.21.15.3:0/1750901916 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c00f510 con 0x7fdf3c062a10 2023-10-03T10:09:31.018 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.016+0000 7fdf3bfff700 1 -- 172.21.15.3:0/1750901916 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2c00f670 con 0x7fdf3c062a10 2023-10-03T10:09:31.018 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.016+0000 7fdf3bfff700 1 -- 172.21.15.3:0/1750901916 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c00f9f0 con 0x7fdf3c062a10 2023-10-03T10:09:31.018 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.016+0000 7fdf4383b700 1 -- 172.21.15.3:0/1750901916 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf3c062a10 msgr2=0x7fdf3c128e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:31.018 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.016+0000 7fdf4383b700 1 --2- 172.21.15.3:0/1750901916 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf3c062a10 0x7fdf3c128e40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fdf2c009760 tx=0x7fdf2c009060).stop 2023-10-03T10:09:31.018 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.016+0000 7fdf4383b700 1 -- 172.21.15.3:0/1750901916 shutdown_connections 2023-10-03T10:09:31.018 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.016+0000 7fdf4383b700 1 --2- 172.21.15.3:0/1750901916 >> v2:172.21.15.3:3300/0 conn(0x7fdf3c129330 0x7fdf3c094360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:31.019 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.016+0000 7fdf4383b700 1 --2- 172.21.15.3:0/1750901916 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf3c062a10 0x7fdf3c128e40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:31.019 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.016+0000 7fdf4383b700 1 -- 172.21.15.3:0/1750901916 >> 172.21.15.3:0/1750901916 conn(0x7fdf3c09a9b0 msgr2=0x7fdf3c09adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:31.019 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.017+0000 7fdf4383b700 1 -- 172.21.15.3:0/1750901916 shutdown_connections 2023-10-03T10:09:31.019 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.017+0000 7fdf4383b700 1 -- 172.21.15.3:0/1750901916 wait complete. 2023-10-03T10:09:31.021 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.018+0000 7fdf4383b700 1 Processor -- start 2023-10-03T10:09:31.021 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.018+0000 7fdf4383b700 1 -- start start 2023-10-03T10:09:31.021 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.019+0000 7fdf4383b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf3c062a10 0x7fdf3c091a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:31.021 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.019+0000 7fdf4383b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf3c128900 0x7fdf3c091f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:31.021 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.019+0000 7fdf4383b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf3c129330 0x7fdf3c092450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:31.021 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.019+0000 7fdf4383b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c092940 con 0x7fdf3c129330 2023-10-03T10:09:31.021 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.019+0000 7fdf4383b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c11a4f0 con 0x7fdf3c062a10 2023-10-03T10:09:31.021 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.019+0000 7fdf4383b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c093810 con 0x7fdf3c128900 2023-10-03T10:09:31.021 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.019+0000 7fdf40dd6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf3c128900 0x7fdf3c091f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:31.021 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.019+0000 7fdf41dd8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf3c129330 0x7fdf3c092450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:31.022 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.019+0000 7fdf40dd6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf3c128900 0x7fdf3c091f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50286/0 (socket says 172.21.15.3:50286) 2023-10-03T10:09:31.022 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.019+0000 7fdf41dd8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf3c129330 0x7fdf3c092450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33552/0 (socket says 172.21.15.3:33552) 2023-10-03T10:09:31.022 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.019+0000 7fdf41dd8700 1 -- 172.21.15.3:0/1806076458 learned_addr learned my addr 172.21.15.3:0/1806076458 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:31.022 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.019+0000 7fdf415d7700 1 --2- 172.21.15.3:0/1806076458 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf3c062a10 0x7fdf3c091a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:31.022 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.019+0000 7fdf41dd8700 1 -- 172.21.15.3:0/1806076458 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf3c128900 msgr2=0x7fdf3c091f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:31.022 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.019+0000 7fdf41dd8700 1 --2- 172.21.15.3:0/1806076458 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf3c128900 0x7fdf3c091f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:31.022 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.019+0000 7fdf41dd8700 1 -- 172.21.15.3:0/1806076458 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf3c062a10 msgr2=0x7fdf3c091a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:31.022 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.019+0000 7fdf41dd8700 1 --2- 172.21.15.3:0/1806076458 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf3c062a10 0x7fdf3c091a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:31.022 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.019+0000 7fdf41dd8700 1 -- 172.21.15.3:0/1806076458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf34009ae0 con 0x7fdf3c129330 2023-10-03T10:09:31.022 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.020+0000 7fdf41dd8700 1 --2- 172.21.15.3:0/1806076458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf3c129330 0x7fdf3c092450 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fdf34009fd0 tx=0x7fdf3400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:31.022 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.020+0000 7fdf397fa700 1 -- 172.21.15.3:0/1806076458 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf340092e0 con 0x7fdf3c129330 2023-10-03T10:09:31.022 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.020+0000 7fdf397fa700 1 -- 172.21.15.3:0/1806076458 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3400f040 con 0x7fdf3c129330 2023-10-03T10:09:31.023 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.020+0000 7fdf4383b700 1 -- 172.21.15.3:0/1806076458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2c0092a0 con 0x7fdf3c129330 2023-10-03T10:09:31.023 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.020+0000 7fdf397fa700 1 -- 172.21.15.3:0/1806076458 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3400cbd0 con 0x7fdf3c129330 2023-10-03T10:09:31.023 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.020+0000 7fdf4383b700 1 -- 172.21.15.3:0/1806076458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3c095df0 con 0x7fdf3c129330 2023-10-03T10:09:31.023 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.020+0000 7fdf397fa700 1 -- 172.21.15.3:0/1806076458 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fdf340146c0 con 0x7fdf3c129330 2023-10-03T10:09:31.023 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.020+0000 7fdf397fa700 1 -- 172.21.15.3:0/1806076458 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7fdf3401e440 con 0x7fdf3c129330 2023-10-03T10:09:31.023 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.021+0000 7fdf4383b700 1 -- 172.21.15.3:0/1806076458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf3c128150 con 0x7fdf3c129330 2023-10-03T10:09:31.024 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.023+0000 7fdf397fa700 1 -- 172.21.15.3:0/1806076458 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fdf34023060 con 0x7fdf3c129330 2023-10-03T10:09:31.081 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:09:31.081+0000 7f57e47cb680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-03T10:09:31.095 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.094+0000 7fdf4383b700 1 -- 172.21.15.3:0/1806076458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf3c05a260 con 0x7fdf3c129330 2023-10-03T10:09:31.095 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.095+0000 7fdf397fa700 1 -- 172.21.15.3:0/1806076458 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4109 (secure 0 0 0) 0x7fdf34019070 con 0x7fdf3c129330 2023-10-03T10:09:31.095 INFO:teuthology.misc.health.smithi003.stdout: 2023-10-03T10:09:31.096 INFO:teuthology.misc.health.smithi003.stdout:{"epoch":6,"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","created":"2023-10-03T10:09:28.703441+0000","modified":"2023-10-03T10:09:30.156546+0000","last_up_change":"0.000000","last_in_change":"2023-10-03T10:09:30.156546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"cdcdd930-3c5b-4486-a021-7132f90fb6e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c8e58877-6762-4741-a8c1-567ebc97b708","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"68a2a21a-7a8e-4f82-8433-5201faa50777","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c4df2903-4e1e-4208-bf77-95b9fa749241","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-03T10:09:31.096 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.096+0000 7fdf4383b700 1 -- 172.21.15.3:0/1806076458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf3c129330 msgr2=0x7fdf3c092450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:31.097 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.096+0000 7fdf4383b700 1 --2- 172.21.15.3:0/1806076458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf3c129330 0x7fdf3c092450 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fdf34009fd0 tx=0x7fdf3400b040).stop 2023-10-03T10:09:31.097 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.096+0000 7fdf4383b700 1 -- 172.21.15.3:0/1806076458 shutdown_connections 2023-10-03T10:09:31.097 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.096+0000 7fdf4383b700 1 --2- 172.21.15.3:0/1806076458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf3c129330 0x7fdf3c092450 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:31.097 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.096+0000 7fdf4383b700 1 --2- 172.21.15.3:0/1806076458 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf3c062a10 0x7fdf3c091a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:31.097 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.096+0000 7fdf4383b700 1 --2- 172.21.15.3:0/1806076458 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf3c128900 0x7fdf3c091f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:31.097 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.096+0000 7fdf4383b700 1 -- 172.21.15.3:0/1806076458 >> 172.21.15.3:0/1806076458 conn(0x7fdf3c09a9b0 msgr2=0x7fdf3c06a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:31.097 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.096+0000 7fdf4383b700 1 -- 172.21.15.3:0/1806076458 shutdown_connections 2023-10-03T10:09:31.097 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:31.096+0000 7fdf4383b700 1 -- 172.21.15.3:0/1806076458 wait complete. 2023-10-03T10:09:31.108 DEBUG:teuthology.misc:0 of 4 OSDs are up 2023-10-03T10:09:31.162 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:09:31.161+0000 7f57e47cb680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-03T10:09:31.574 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:09:31.574+0000 7f57e47cb680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-03T10:09:31.671 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:09:31.670+0000 7f57e47cb680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-03T10:09:32.396 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:09:32.395+0000 7f57e47cb680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-03T10:09:32.475 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:09:32.475+0000 7f57e47cb680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-03T10:09:32.564 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:09:32.563+0000 7f57e47cb680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-03T10:09:32.902 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:09:32.901+0000 7f57e47cb680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-03T10:09:32.982 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:09:32.982+0000 7f57e47cb680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-03T10:09:33.117 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:09:33.116+0000 7f57e47cb680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-03T10:09:33.241 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:09:33.240+0000 7f57e47cb680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-03T10:09:33.425 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:09:33.424+0000 7f57e47cb680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-03T10:09:33.505 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:09:33.505+0000 7f57e47cb680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-03T10:09:34.082 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-10-03T10:09:34.082+0000 7fc8e6257200 -1 osd.3 0 log_to_monitors {default=true} 2023-10-03T10:09:34.299 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-10-03T10:09:34.298+0000 7f135d21b200 -1 osd.1 0 log_to_monitors {default=true} 2023-10-03T10:09:34.355 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-10-03T10:09:34.355+0000 7f3134721200 -1 osd.2 0 log_to_monitors {default=true} 2023-10-03T10:09:34.355 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-10-03T10:09:34.355+0000 7f4e52a67200 -1 osd.0 0 log_to_monitors {default=true} 2023-10-03T10:09:34.530 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:09:34.530+0000 7f579b20b700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-03T10:09:34.530 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:09:34.530+0000 7f579b20b700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-03T10:09:35.574 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-10-03T10:09:35.574+0000 7f312b0c2700 -1 osd.2 0 waiting for initial osdmap 2023-10-03T10:09:35.575 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-10-03T10:09:35.574+0000 7f1353bbc700 -1 osd.1 0 waiting for initial osdmap 2023-10-03T10:09:35.575 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-10-03T10:09:35.574+0000 7f4e49408700 -1 osd.0 0 waiting for initial osdmap 2023-10-03T10:09:35.575 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-10-03T10:09:35.574+0000 7fc8dcbf8700 -1 osd.3 0 waiting for initial osdmap 2023-10-03T10:09:35.580 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-10-03T10:09:35.579+0000 7f4e45b84700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-03T10:09:35.580 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-10-03T10:09:35.579+0000 7fc8d9374700 -1 osd.3 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-03T10:09:35.580 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-10-03T10:09:35.580+0000 7f312783e700 -1 osd.2 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-03T10:09:35.582 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-10-03T10:09:35.582+0000 7f1350338700 -1 osd.1 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-03T10:09:37.512 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-03T10:09:37.647 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.645+0000 7f183d2a7700 1 Processor -- start 2023-10-03T10:09:37.647 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.645+0000 7f183d2a7700 1 -- start start 2023-10-03T10:09:37.647 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.645+0000 7f183d2a7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1838095d60 0x7f1838096160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:37.648 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.645+0000 7f183d2a7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1838130200 0x7f18381325f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:37.648 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.645+0000 7f183d2a7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1838096aa0 con 0x7f1838062a10 2023-10-03T10:09:37.648 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.645+0000 7f183d2a7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1838062df0 con 0x7f1838130200 2023-10-03T10:09:37.648 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.645+0000 7f183d2a7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f183806a970 con 0x7f1838095d60 2023-10-03T10:09:37.648 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.645+0000 7f18377fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1838130200 0x7f18381325f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:37.648 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.645+0000 7f18367fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1838095d60 0x7f1838096160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:37.648 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.645+0000 7f18367fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1838095d60 0x7f1838096160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38166/0 (socket says 172.21.15.3:38166) 2023-10-03T10:09:37.648 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.645+0000 7f18377fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1838130200 0x7f18381325f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42018/0 (socket says 172.21.15.3:42018) 2023-10-03T10:09:37.648 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.645+0000 7f18367fc700 1 -- 172.21.15.3:0/3468802578 learned_addr learned my addr 172.21.15.3:0/3468802578 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:37.648 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.645+0000 7f1836ffd700 1 --1- 172.21.15.3:0/3468802578 >> v1:172.21.15.3:6789/0 conn(0x7f1838062a10 0x7f1838095650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58088/0 (socket says 172.21.15.3:58088) 2023-10-03T10:09:37.648 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.646+0000 7f18377fe700 1 -- 172.21.15.3:0/3468802578 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1838095d60 msgr2=0x7f1838096160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:37.648 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.646+0000 7f18377fe700 1 --2- 172.21.15.3:0/3468802578 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1838095d60 0x7f1838096160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:37.649 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.646+0000 7f18377fe700 1 -- 172.21.15.3:0/3468802578 >> v1:172.21.15.3:6789/0 conn(0x7f1838062a10 legacy=0x7f1838095650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:37.649 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.646+0000 7f1835ffb700 1 -- 172.21.15.3:0/3468802578 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3865664020 0 0) 0x7f1838096aa0 con 0x7f1838062a10 2023-10-03T10:09:37.649 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.646+0000 7f18377fe700 1 -- 172.21.15.3:0/3468802578 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1838097290 con 0x7f1838130200 2023-10-03T10:09:37.649 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.646+0000 7f18367fc700 1 --2- 172.21.15.3:0/3468802578 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1838095d60 0x7f1838096160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:37.649 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.646+0000 7f18377fe700 1 --2- 172.21.15.3:0/3468802578 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1838130200 0x7f18381325f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f1830009ec0 tx=0x7f1830009780).ready entity=mon.1 client_cookie=f8bbe2ca0b60bdaa server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:37.649 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.646+0000 7f1835ffb700 1 -- 172.21.15.3:0/3468802578 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183000e070 con 0x7f1838130200 2023-10-03T10:09:37.649 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.646+0000 7f1835ffb700 1 -- 172.21.15.3:0/3468802578 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183000bbf0 con 0x7f1838130200 2023-10-03T10:09:37.649 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.646+0000 7f1835ffb700 1 -- 172.21.15.3:0/3468802578 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1830012410 con 0x7f1838130200 2023-10-03T10:09:37.649 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.647+0000 7f183d2a7700 1 -- 172.21.15.3:0/3468802578 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1838130200 msgr2=0x7f18381325f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:37.649 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.647+0000 7f183d2a7700 1 --2- 172.21.15.3:0/3468802578 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1838130200 0x7f18381325f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f1830009ec0 tx=0x7f1830009780).stop 2023-10-03T10:09:37.649 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.647+0000 7f183d2a7700 1 -- 172.21.15.3:0/3468802578 shutdown_connections 2023-10-03T10:09:37.649 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.647+0000 7f183d2a7700 1 --2- 172.21.15.3:0/3468802578 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1838130200 0x7f18381325f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:37.650 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.647+0000 7f183d2a7700 1 --2- 172.21.15.3:0/3468802578 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1838095d60 0x7f1838096160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:37.650 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.647+0000 7f183d2a7700 1 -- 172.21.15.3:0/3468802578 >> 172.21.15.3:0/3468802578 conn(0x7f183809a9b0 msgr2=0x7f183809adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:37.650 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.647+0000 7f183d2a7700 1 -- 172.21.15.3:0/3468802578 shutdown_connections 2023-10-03T10:09:37.650 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.647+0000 7f183d2a7700 1 -- 172.21.15.3:0/3468802578 wait complete. 2023-10-03T10:09:37.652 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.648+0000 7f183d2a7700 1 Processor -- start 2023-10-03T10:09:37.652 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.648+0000 7f183d2a7700 1 -- start start 2023-10-03T10:09:37.652 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.648+0000 7f183d2a7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1838062a10 0x7f18381175f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:37.652 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.648+0000 7f183d2a7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1838095d60 0x7f1838117ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:37.652 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.648+0000 7f183d2a7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1838130200 0x7f1838117fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:37.652 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.648+0000 7f183d2a7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18381184c0 con 0x7f1838130200 2023-10-03T10:09:37.652 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.648+0000 7f183d2a7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1838118aa0 con 0x7f1838095d60 2023-10-03T10:09:37.652 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.648+0000 7f183d2a7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1838119340 con 0x7f1838062a10 2023-10-03T10:09:37.652 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.649+0000 7f1836ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1838062a10 0x7f18381175f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:37.652 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.649+0000 7f18367fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1838095d60 0x7f1838117ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:37.653 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.649+0000 7f18377fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1838130200 0x7f1838117fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:37.653 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.649+0000 7f18367fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1838095d60 0x7f1838117ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42024/0 (socket says 172.21.15.3:42024) 2023-10-03T10:09:37.653 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.649+0000 7f18377fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1838130200 0x7f1838117fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56724/0 (socket says 172.21.15.3:56724) 2023-10-03T10:09:37.653 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.649+0000 7f18367fc700 1 -- 172.21.15.3:0/673957496 learned_addr learned my addr 172.21.15.3:0/673957496 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:37.653 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.649+0000 7f1836ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1838062a10 0x7f18381175f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38176/0 (socket says 172.21.15.3:38176) 2023-10-03T10:09:37.653 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.649+0000 7f18367fc700 1 -- 172.21.15.3:0/673957496 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1838062a10 msgr2=0x7f18381175f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:37.653 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.649+0000 7f18367fc700 1 --2- 172.21.15.3:0/673957496 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1838062a10 0x7f18381175f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:37.653 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.649+0000 7f18367fc700 1 -- 172.21.15.3:0/673957496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1838130200 msgr2=0x7f1838117fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:37.653 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.649+0000 7f18367fc700 1 --2- 172.21.15.3:0/673957496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1838130200 0x7f1838117fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:37.653 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.649+0000 7f18367fc700 1 -- 172.21.15.3:0/673957496 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c009370 con 0x7f1838095d60 2023-10-03T10:09:37.653 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.649+0000 7f18377fe700 1 --2- 172.21.15.3:0/673957496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1838130200 0x7f1838117fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:37.653 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.649+0000 7f18367fc700 1 --2- 172.21.15.3:0/673957496 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1838095d60 0x7f1838117ae0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f182c00ea20 tx=0x7f182c0090f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:37.653 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.649+0000 7f181effd700 1 -- 172.21.15.3:0/673957496 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c00c5e0 con 0x7f1838095d60 2023-10-03T10:09:37.654 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.649+0000 7f181effd700 1 -- 172.21.15.3:0/673957496 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182c00c740 con 0x7f1838095d60 2023-10-03T10:09:37.654 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.649+0000 7f183d2a7700 1 -- 172.21.15.3:0/673957496 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1830009a20 con 0x7f1838095d60 2023-10-03T10:09:37.654 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.649+0000 7f181effd700 1 -- 172.21.15.3:0/673957496 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c00cac0 con 0x7f1838095d60 2023-10-03T10:09:37.654 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.649+0000 7f183d2a7700 1 -- 172.21.15.3:0/673957496 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1838096d20 con 0x7f1838095d60 2023-10-03T10:09:37.654 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.650+0000 7f181effd700 1 -- 172.21.15.3:0/673957496 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f182c0123f0 con 0x7f1838095d60 2023-10-03T10:09:37.654 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.650+0000 7f181effd700 1 -- 172.21.15.3:0/673957496 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 2461+0+0 (secure 0 0 0) 0x7f182c03aa10 con 0x7f1838095d60 2023-10-03T10:09:37.654 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.650+0000 7f183d2a7700 1 -- 172.21.15.3:0/673957496 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f183811b030 con 0x7f1838095d60 2023-10-03T10:09:37.655 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.655+0000 7f181effd700 1 -- 172.21.15.3:0/673957496 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f183811b030 con 0x7f1838095d60 2023-10-03T10:09:37.832 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.831+0000 7f183d2a7700 1 -- 172.21.15.3:0/673957496 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f183811b030 con 0x7f1838095d60 2023-10-03T10:09:37.832 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.832+0000 7f181effd700 1 -- 172.21.15.3:0/673957496 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+6565 (secure 0 0 0) 0x7f183811b030 con 0x7f1838095d60 2023-10-03T10:09:37.832 INFO:teuthology.misc.health.smithi003.stdout: 2023-10-03T10:09:37.833 INFO:teuthology.misc.health.smithi003.stdout:{"epoch":10,"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","created":"2023-10-03T10:09:28.703441+0000","modified":"2023-10-03T10:09:37.570571+0000","last_up_change":"2023-10-03T10:09:36.569401+0000","last_in_change":"2023-10-03T10:09:30.156546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-10-03T10:09:36.623971+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"cdcdd930-3c5b-4486-a021-7132f90fb6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6808","nonce":118845}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6809","nonce":118845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6811","nonce":118845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6810","nonce":118845}]},"public_addr":"172.21.15.3:6808/118845","cluster_addr":"172.21.15.3:6809/118845","heartbeat_back_addr":"172.21.15.3:6811/118845","heartbeat_front_addr":"172.21.15.3:6810/118845","state":["exists","up"]},{"osd":1,"uuid":"c8e58877-6762-4741-a8c1-567ebc97b708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6804","nonce":118835}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6805","nonce":118835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6807","nonce":118835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6806","nonce":118835}]},"public_addr":"172.21.15.3:6804/118835","cluster_addr":"172.21.15.3:6805/118835","heartbeat_back_addr":"172.21.15.3:6807/118835","heartbeat_front_addr":"172.21.15.3:6806/118835","state":["exists","up"]},{"osd":2,"uuid":"68a2a21a-7a8e-4f82-8433-5201faa50777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6812","nonce":118864}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6813","nonce":118864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6815","nonce":118864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6814","nonce":118864}]},"public_addr":"172.21.15.3:6812/118864","cluster_addr":"172.21.15.3:6813/118864","heartbeat_back_addr":"172.21.15.3:6815/118864","heartbeat_front_addr":"172.21.15.3:6814/118864","state":["exists","up"]},{"osd":3,"uuid":"c4df2903-4e1e-4208-bf77-95b9fa749241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6800","nonce":118836}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6801","nonce":118836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6803","nonce":118836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6802","nonce":118836}]},"public_addr":"172.21.15.3:6800/118836","cluster_addr":"172.21.15.3:6801/118836","heartbeat_back_addr":"172.21.15.3:6803/118836","heartbeat_front_addr":"172.21.15.3:6802/118836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.369708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.257825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.337931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.085222+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-03T10:09:37.834 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.834+0000 7f183d2a7700 1 -- 172.21.15.3:0/673957496 >> v1:172.21.15.3:6816/118296 conn(0x7f182402bb60 legacy=0x7f182402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:37.835 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.834+0000 7f183d2a7700 1 -- 172.21.15.3:0/673957496 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1838095d60 msgr2=0x7f1838117ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:37.835 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.834+0000 7f183d2a7700 1 --2- 172.21.15.3:0/673957496 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1838095d60 0x7f1838117ae0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f182c00ea20 tx=0x7f182c0090f0).stop 2023-10-03T10:09:37.835 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.834+0000 7f183d2a7700 1 -- 172.21.15.3:0/673957496 shutdown_connections 2023-10-03T10:09:37.835 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.834+0000 7f183d2a7700 1 --2- 172.21.15.3:0/673957496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1838130200 0x7f1838117fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:37.835 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.834+0000 7f183d2a7700 1 --2- 172.21.15.3:0/673957496 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1838062a10 0x7f18381175f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:37.835 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.834+0000 7f183d2a7700 1 --2- 172.21.15.3:0/673957496 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1838095d60 0x7f1838117ae0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:37.835 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.834+0000 7f183d2a7700 1 -- 172.21.15.3:0/673957496 >> 172.21.15.3:0/673957496 conn(0x7f183809a9b0 msgr2=0x7f18381114c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:37.835 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.834+0000 7f183d2a7700 1 -- 172.21.15.3:0/673957496 shutdown_connections 2023-10-03T10:09:37.835 INFO:teuthology.misc.health.smithi003.stderr:2023-10-03T10:09:37.834+0000 7f183d2a7700 1 -- 172.21.15.3:0/673957496 wait complete. 2023-10-03T10:09:37.846 DEBUG:teuthology.misc:4 of 4 OSDs are up 2023-10-03T10:09:37.846 INFO:tasks.ceph:Creating RBD pool 2023-10-03T10:09:37.846 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph osd pool create rbd 8 2023-10-03T10:09:37.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.975+0000 7fc235965700 1 Processor -- start 2023-10-03T10:09:37.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.975+0000 7fc235965700 1 -- start start 2023-10-03T10:09:37.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.975+0000 7fc235965700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc230065340 0x7fc2300986e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:37.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.975+0000 7fc235965700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc2300992e0 0x7fc2301324b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:37.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.975+0000 7fc235965700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc230098190 con 0x7fc230097d00 2023-10-03T10:09:37.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.975+0000 7fc235965700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc230065720 con 0x7fc230065340 2023-10-03T10:09:37.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.975+0000 7fc235965700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc230096190 con 0x7fc2300992e0 2023-10-03T10:09:37.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.975+0000 7fc22effd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc230065340 0x7fc2300986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:37.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.975+0000 7fc22f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc2300992e0 0x7fc2301324b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:37.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.975+0000 7fc22e7fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc230097d00 0x7fc230098bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58094/0 (socket says 172.21.15.3:58094) 2023-10-03T10:09:37.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.975+0000 7fc22e7fc700 1 -- 172.21.15.3:0/1948810317 learned_addr learned my addr 172.21.15.3:0/1948810317 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:09:37.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.976+0000 7fc22effd700 1 -- 172.21.15.3:0/1948810317 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc2300992e0 msgr2=0x7fc2301324b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:37.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.976+0000 7fc22effd700 1 --2- 172.21.15.3:0/1948810317 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc2300992e0 0x7fc2301324b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:37.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.976+0000 7fc22effd700 1 -- 172.21.15.3:0/1948810317 >> v1:172.21.15.3:6789/0 conn(0x7fc230097d00 legacy=0x7fc230098bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:37.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.976+0000 7fc22effd700 1 -- 172.21.15.3:0/1948810317 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc230066010 con 0x7fc230065340 2023-10-03T10:09:37.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.976+0000 7fc22dffb700 1 -- 172.21.15.3:0/1948810317 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1347766349 0 0) 0x7fc230098190 con 0x7fc230097d00 2023-10-03T10:09:37.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.976+0000 7fc22f7fe700 1 --2- 172.21.15.3:0/1948810317 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc2300992e0 0x7fc2301324b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:37.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.976+0000 7fc22effd700 1 --2- 172.21.15.3:0/1948810317 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc230065340 0x7fc2300986e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fc220009790 tx=0x7fc220009060).ready entity=mon.1 client_cookie=9e5e861825f3ba93 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:37.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.976+0000 7fc22dffb700 1 -- 172.21.15.3:0/1948810317 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2200090d0 con 0x7fc230065340 2023-10-03T10:09:37.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.976+0000 7fc22dffb700 1 -- 172.21.15.3:0/1948810317 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc22000f510 con 0x7fc230065340 2023-10-03T10:09:37.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.976+0000 7fc22dffb700 1 -- 172.21.15.3:0/1948810317 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22000f890 con 0x7fc230065340 2023-10-03T10:09:37.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.977+0000 7fc235965700 1 -- 172.21.15.3:0/1948810317 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc230065340 msgr2=0x7fc2300986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:37.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.977+0000 7fc235965700 1 --2- 172.21.15.3:0/1948810317 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc230065340 0x7fc2300986e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fc220009790 tx=0x7fc220009060).stop 2023-10-03T10:09:37.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.977+0000 7fc235965700 1 -- 172.21.15.3:0/1948810317 shutdown_connections 2023-10-03T10:09:37.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.977+0000 7fc235965700 1 --2- 172.21.15.3:0/1948810317 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc230065340 0x7fc2300986e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:37.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.977+0000 7fc235965700 1 --2- 172.21.15.3:0/1948810317 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc2300992e0 0x7fc2301324b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:37.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.977+0000 7fc235965700 1 -- 172.21.15.3:0/1948810317 >> 172.21.15.3:0/1948810317 conn(0x7fc23009dcc0 msgr2=0x7fc23009e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:37.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.977+0000 7fc235965700 1 -- 172.21.15.3:0/1948810317 shutdown_connections 2023-10-03T10:09:37.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.977+0000 7fc235965700 1 -- 172.21.15.3:0/1948810317 wait complete. 2023-10-03T10:09:37.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.978+0000 7fc235965700 1 Processor -- start 2023-10-03T10:09:37.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.978+0000 7fc235965700 1 -- start start 2023-10-03T10:09:37.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.978+0000 7fc235965700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc230065340 0x7fc23011d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:37.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.978+0000 7fc235965700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2300992e0 0x7fc23011a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:37.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.978+0000 7fc235965700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc23011a960 0x7fc23011ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:37.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.978+0000 7fc235965700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23011e4a0 con 0x7fc2300992e0 2023-10-03T10:09:37.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.978+0000 7fc235965700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc23011da00 con 0x7fc230065340 2023-10-03T10:09:37.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.978+0000 7fc235965700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc23011dfa0 con 0x7fc23011a960 2023-10-03T10:09:37.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.978+0000 7fc22e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2300992e0 0x7fc23011a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:37.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.978+0000 7fc22f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc23011a960 0x7fc23011ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:37.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.978+0000 7fc22f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc23011a960 0x7fc23011ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38196/0 (socket says 172.21.15.3:38196) 2023-10-03T10:09:37.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.978+0000 7fc22f7fe700 1 -- 172.21.15.3:0/793886985 learned_addr learned my addr 172.21.15.3:0/793886985 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:37.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.979+0000 7fc22effd700 1 --2- 172.21.15.3:0/793886985 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc230065340 0x7fc23011d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:37.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.979+0000 7fc22e7fc700 1 -- 172.21.15.3:0/793886985 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc23011a960 msgr2=0x7fc23011ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:37.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.979+0000 7fc22e7fc700 1 --2- 172.21.15.3:0/793886985 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc23011a960 0x7fc23011ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:37.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.979+0000 7fc22e7fc700 1 -- 172.21.15.3:0/793886985 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc230065340 msgr2=0x7fc23011d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:37.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.979+0000 7fc22e7fc700 1 --2- 172.21.15.3:0/793886985 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc230065340 0x7fc23011d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:37.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.979+0000 7fc22e7fc700 1 -- 172.21.15.3:0/793886985 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc228009a20 con 0x7fc2300992e0 2023-10-03T10:09:37.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.979+0000 7fc22f7fe700 1 --2- 172.21.15.3:0/793886985 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc23011a960 0x7fc23011ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:37.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.979+0000 7fc22effd700 1 --2- 172.21.15.3:0/793886985 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc230065340 0x7fc23011d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:37.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.979+0000 7fc22e7fc700 1 --2- 172.21.15.3:0/793886985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2300992e0 0x7fc23011a470 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fc224000c00 tx=0x7fc224007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:37.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.979+0000 7fc216ffd700 1 -- 172.21.15.3:0/793886985 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2240070d0 con 0x7fc2300992e0 2023-10-03T10:09:37.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.979+0000 7fc216ffd700 1 -- 172.21.15.3:0/793886985 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc224007250 con 0x7fc2300992e0 2023-10-03T10:09:37.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.979+0000 7fc235965700 1 -- 172.21.15.3:0/793886985 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2200092a0 con 0x7fc2300992e0 2023-10-03T10:09:37.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.979+0000 7fc216ffd700 1 -- 172.21.15.3:0/793886985 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc224007610 con 0x7fc2300992e0 2023-10-03T10:09:37.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.979+0000 7fc235965700 1 -- 172.21.15.3:0/793886985 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2300665e0 con 0x7fc2300992e0 2023-10-03T10:09:37.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.980+0000 7fc216ffd700 1 -- 172.21.15.3:0/793886985 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fc2240059b0 con 0x7fc2300992e0 2023-10-03T10:09:37.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.980+0000 7fc216ffd700 1 -- 172.21.15.3:0/793886985 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 2461+0+0 (secure 0 0 0) 0x7fc22403a1d0 con 0x7fc2300992e0 2023-10-03T10:09:37.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.980+0000 7fc235965700 1 -- 172.21.15.3:0/793886985 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc23005a260 con 0x7fc2300992e0 2023-10-03T10:09:37.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:37.986+0000 7fc216ffd700 1 -- 172.21.15.3:0/793886985 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fc224018030 con 0x7fc2300992e0 2023-10-03T10:09:38.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.162+0000 7fc235965700 1 -- 172.21.15.3:0/793886985 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fc230097470 con 0x7fc2300992e0 2023-10-03T10:09:38.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.575+0000 7fc216ffd700 1 -- 172.21.15.3:0/793886985 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v11) v1 ==== 109+0+0 (secure 0 0 0) 0x7fc224018020 con 0x7fc2300992e0 2023-10-03T10:09:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.580+0000 7fc235965700 1 -- 172.21.15.3:0/793886985 >> v1:172.21.15.3:6816/118296 conn(0x7fc21c02bb60 legacy=0x7fc21c02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.580+0000 7fc235965700 1 -- 172.21.15.3:0/793886985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2300992e0 msgr2=0x7fc23011a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:38.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.580+0000 7fc235965700 1 --2- 172.21.15.3:0/793886985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2300992e0 0x7fc23011a470 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fc224000c00 tx=0x7fc224007b90).stop 2023-10-03T10:09:38.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.580+0000 7fc235965700 1 -- 172.21.15.3:0/793886985 shutdown_connections 2023-10-03T10:09:38.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.580+0000 7fc235965700 1 --2- 172.21.15.3:0/793886985 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc23011a960 0x7fc23011ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:38.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.580+0000 7fc235965700 1 --2- 172.21.15.3:0/793886985 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc230065340 0x7fc23011d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:38.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.580+0000 7fc235965700 1 --2- 172.21.15.3:0/793886985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2300992e0 0x7fc23011a470 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:38.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.580+0000 7fc235965700 1 -- 172.21.15.3:0/793886985 >> 172.21.15.3:0/793886985 conn(0x7fc23009dcc0 msgr2=0x7fc23009bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:38.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.580+0000 7fc235965700 1 -- 172.21.15.3:0/793886985 shutdown_connections 2023-10-03T10:09:38.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.580+0000 7fc235965700 1 -- 172.21.15.3:0/793886985 wait complete. 2023-10-03T10:09:38.582 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'rbd' created 2023-10-03T10:09:38.596 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-10-03T10:09:38.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.732+0000 7fca517c5700 1 Processor -- start 2023-10-03T10:09:38.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.732+0000 7fca517c5700 1 -- start start 2023-10-03T10:09:38.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.733+0000 7fca517c5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca4c093970 0x7fca4c098840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:38.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.733+0000 7fca517c5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca4c098d30 0x7fca4c099110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:38.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.733+0000 7fca517c5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fca4c12c190 0x7fca4c12e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:38.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.733+0000 7fca517c5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca4c093d50 con 0x7fca4c12c190 2023-10-03T10:09:38.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.733+0000 7fca517c5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c091af0 con 0x7fca4c098d30 2023-10-03T10:09:38.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.733+0000 7fca517c5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c099ca0 con 0x7fca4c093970 2023-10-03T10:09:38.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.733+0000 7fca4a7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca4c098d30 0x7fca4c099110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:38.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.733+0000 7fca4b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fca4c12c190 0x7fca4c12e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:38.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.733+0000 7fca4affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca4c093970 0x7fca4c098840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:38.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.733+0000 7fca4a7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca4c098d30 0x7fca4c099110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42050/0 (socket says 172.21.15.3:42050) 2023-10-03T10:09:38.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.733+0000 7fca4a7fc700 1 -- 172.21.15.3:0/3803564856 learned_addr learned my addr 172.21.15.3:0/3803564856 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:38.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.733+0000 7fca4a7fc700 1 -- 172.21.15.3:0/3803564856 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca4c093970 msgr2=0x7fca4c098840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:38.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.733+0000 7fca4a7fc700 1 --2- 172.21.15.3:0/3803564856 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca4c093970 0x7fca4c098840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:38.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.733+0000 7fca4a7fc700 1 -- 172.21.15.3:0/3803564856 >> v2:172.21.15.3:3300/0 conn(0x7fca4c12c190 msgr2=0x7fca4c12e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:38.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.733+0000 7fca4a7fc700 1 --2- 172.21.15.3:0/3803564856 >> v2:172.21.15.3:3300/0 conn(0x7fca4c12c190 0x7fca4c12e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:38.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.733+0000 7fca4a7fc700 1 -- 172.21.15.3:0/3803564856 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4c093170 con 0x7fca4c098d30 2023-10-03T10:09:38.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.733+0000 7fca4b7fe700 1 --2- 172.21.15.3:0/3803564856 >> v2:172.21.15.3:3300/0 conn(0x7fca4c12c190 0x7fca4c12e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:38.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.734+0000 7fca4a7fc700 1 --2- 172.21.15.3:0/3803564856 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca4c098d30 0x7fca4c099110 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fca40006fd0 tx=0x7fca4000b3b0).ready entity=mon.1 client_cookie=4764e34ac37a22dd server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:38.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.734+0000 7fca49ffb700 1 -- 172.21.15.3:0/3803564856 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4000da70 con 0x7fca4c098d30 2023-10-03T10:09:38.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.734+0000 7fca49ffb700 1 -- 172.21.15.3:0/3803564856 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca4000dbd0 con 0x7fca4c098d30 2023-10-03T10:09:38.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.734+0000 7fca49ffb700 1 -- 172.21.15.3:0/3803564856 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca40010470 con 0x7fca4c098d30 2023-10-03T10:09:38.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.734+0000 7fca517c5700 1 -- 172.21.15.3:0/3803564856 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca4c098d30 msgr2=0x7fca4c099110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:38.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.734+0000 7fca517c5700 1 --2- 172.21.15.3:0/3803564856 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca4c098d30 0x7fca4c099110 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fca40006fd0 tx=0x7fca4000b3b0).stop 2023-10-03T10:09:38.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.734+0000 7fca517c5700 1 -- 172.21.15.3:0/3803564856 shutdown_connections 2023-10-03T10:09:38.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.734+0000 7fca517c5700 1 --2- 172.21.15.3:0/3803564856 >> v2:172.21.15.3:3300/0 conn(0x7fca4c12c190 0x7fca4c12e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:38.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.734+0000 7fca517c5700 1 --2- 172.21.15.3:0/3803564856 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca4c093970 0x7fca4c098840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:38.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.734+0000 7fca517c5700 1 --2- 172.21.15.3:0/3803564856 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca4c098d30 0x7fca4c099110 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:38.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.734+0000 7fca517c5700 1 -- 172.21.15.3:0/3803564856 >> 172.21.15.3:0/3803564856 conn(0x7fca4c09a940 msgr2=0x7fca4c09ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:38.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.734+0000 7fca517c5700 1 -- 172.21.15.3:0/3803564856 shutdown_connections 2023-10-03T10:09:38.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.734+0000 7fca517c5700 1 -- 172.21.15.3:0/3803564856 wait complete. 2023-10-03T10:09:38.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.736+0000 7fca517c5700 1 Processor -- start 2023-10-03T10:09:38.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.736+0000 7fca517c5700 1 -- start start 2023-10-03T10:09:38.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.736+0000 7fca517c5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca4c093970 0x7fca4c116060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:38.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.736+0000 7fca517c5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca4c098d30 0x7fca4c116550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:38.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.736+0000 7fca517c5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca4c12c190 0x7fca4c116a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.736+0000 7fca517c5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c116f30 con 0x7fca4c093970 2023-10-03T10:09:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.736+0000 7fca517c5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c119830 con 0x7fca4c098d30 2023-10-03T10:09:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.736+0000 7fca517c5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c11a6f0 con 0x7fca4c12c190 2023-10-03T10:09:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.736+0000 7fca4affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca4c093970 0x7fca4c116060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.736+0000 7fca4affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca4c093970 0x7fca4c116060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56754/0 (socket says 172.21.15.3:56754) 2023-10-03T10:09:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.736+0000 7fca4b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca4c12c190 0x7fca4c116a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.736+0000 7fca4affd700 1 -- 172.21.15.3:0/3093066430 learned_addr learned my addr 172.21.15.3:0/3093066430 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.736+0000 7fca4a7fc700 1 --2- 172.21.15.3:0/3093066430 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca4c098d30 0x7fca4c116550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.737+0000 7fca4a7fc700 1 -- 172.21.15.3:0/3093066430 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca4c12c190 msgr2=0x7fca4c116a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.737+0000 7fca4a7fc700 1 --2- 172.21.15.3:0/3093066430 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca4c12c190 0x7fca4c116a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.737+0000 7fca4a7fc700 1 -- 172.21.15.3:0/3093066430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca4c093970 msgr2=0x7fca4c116060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.737+0000 7fca4a7fc700 1 --2- 172.21.15.3:0/3093066430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca4c093970 0x7fca4c116060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.737+0000 7fca4a7fc700 1 -- 172.21.15.3:0/3093066430 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca44009a20 con 0x7fca4c098d30 2023-10-03T10:09:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.737+0000 7fca4b7fe700 1 --2- 172.21.15.3:0/3093066430 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca4c12c190 0x7fca4c116a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.737+0000 7fca4affd700 1 --2- 172.21.15.3:0/3093066430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca4c093970 0x7fca4c116060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.737+0000 7fca4a7fc700 1 --2- 172.21.15.3:0/3093066430 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca4c098d30 0x7fca4c116550 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7fca40016000 tx=0x7fca4c09a6c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.737+0000 7fca32ffd700 1 -- 172.21.15.3:0/3093066430 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4001e070 con 0x7fca4c098d30 2023-10-03T10:09:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.737+0000 7fca32ffd700 1 -- 172.21.15.3:0/3093066430 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca40019450 con 0x7fca4c098d30 2023-10-03T10:09:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.737+0000 7fca517c5700 1 -- 172.21.15.3:0/3093066430 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca4000b610 con 0x7fca4c098d30 2023-10-03T10:09:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.737+0000 7fca32ffd700 1 -- 172.21.15.3:0/3093066430 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca40019810 con 0x7fca4c098d30 2023-10-03T10:09:38.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.737+0000 7fca517c5700 1 -- 172.21.15.3:0/3093066430 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4c118380 con 0x7fca4c098d30 2023-10-03T10:09:38.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.738+0000 7fca32ffd700 1 -- 172.21.15.3:0/3093066430 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fca40019b80 con 0x7fca4c098d30 2023-10-03T10:09:38.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.738+0000 7fca32ffd700 1 -- 172.21.15.3:0/3093066430 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2827+0+0 (secure 0 0 0) 0x7fca4003c140 con 0x7fca4c098d30 2023-10-03T10:09:38.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.738+0000 7fca517c5700 1 -- 172.21.15.3:0/3093066430 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca4c119970 con 0x7fca4c098d30 2023-10-03T10:09:38.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.743+0000 7fca32ffd700 1 -- 172.21.15.3:0/3093066430 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fca4c119970 con 0x7fca4c098d30 2023-10-03T10:09:38.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:38.921+0000 7fca517c5700 1 -- 172.21.15.3:0/3093066430 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fca4c119970 con 0x7fca4c098d30 2023-10-03T10:09:39.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.579+0000 7fca32ffd700 1 -- 172.21.15.3:0/3093066430 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v12) v1 ==== 173+0+0 (secure 0 0 0) 0x7fca4c119970 con 0x7fca4c098d30 2023-10-03T10:09:39.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.581+0000 7fca517c5700 1 -- 172.21.15.3:0/3093066430 >> v1:172.21.15.3:6816/118296 conn(0x7fca3802bb60 legacy=0x7fca3802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:39.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.581+0000 7fca517c5700 1 -- 172.21.15.3:0/3093066430 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca4c098d30 msgr2=0x7fca4c116550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:39.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.581+0000 7fca517c5700 1 --2- 172.21.15.3:0/3093066430 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca4c098d30 0x7fca4c116550 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7fca40016000 tx=0x7fca4c09a6c0).stop 2023-10-03T10:09:39.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.582+0000 7fca517c5700 1 -- 172.21.15.3:0/3093066430 shutdown_connections 2023-10-03T10:09:39.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.582+0000 7fca517c5700 1 --2- 172.21.15.3:0/3093066430 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca4c12c190 0x7fca4c116a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:39.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.582+0000 7fca517c5700 1 --2- 172.21.15.3:0/3093066430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca4c093970 0x7fca4c116060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:39.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.582+0000 7fca517c5700 1 --2- 172.21.15.3:0/3093066430 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca4c098d30 0x7fca4c116550 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:39.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.582+0000 7fca517c5700 1 -- 172.21.15.3:0/3093066430 >> 172.21.15.3:0/3093066430 conn(0x7fca4c09a940 msgr2=0x7fca4c096240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:39.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.582+0000 7fca517c5700 1 -- 172.21.15.3:0/3093066430 shutdown_connections 2023-10-03T10:09:39.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.583+0000 7fca517c5700 1 -- 172.21.15.3:0/3093066430 wait complete. 2023-10-03T10:09:39.586 INFO:teuthology.orchestra.run.smithi003.stderr:enabled application 'rbd' on pool 'rbd' 2023-10-03T10:09:39.601 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-03T10:09:39.602 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-03T10:09:39.602 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-03T10:09:39.604 INFO:tasks.ceph.mds.a:Started 2023-10-03T10:09:39.604 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-03T10:09:39.605 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-03T10:09:39.691 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2023-10-03T10:09:39.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.741+0000 7f88ac726700 1 Processor -- start 2023-10-03T10:09:39.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.741+0000 7f88ac726700 1 -- start start 2023-10-03T10:09:39.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.742+0000 7f88ac726700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88a4062a90 0x7f88a41255e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:39.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.742+0000 7f88ac726700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f88a4094360 0x7f88a4130a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:39.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.742+0000 7f88ac726700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a4096fd0 con 0x7f88a4125ad0 2023-10-03T10:09:39.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.742+0000 7f88ac726700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88a4062e70 con 0x7f88a4094360 2023-10-03T10:09:39.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.742+0000 7f88ac726700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a4063580 con 0x7f88a4062a90 2023-10-03T10:09:39.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.742+0000 7f88aa4c2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88a4062a90 0x7f88a41255e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:39.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.742+0000 7f88aa4c2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88a4062a90 0x7f88a41255e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42080/0 (socket says 172.21.15.3:42080) 2023-10-03T10:09:39.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.742+0000 7f88aacc3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f88a4094360 0x7f88a4130a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:39.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.742+0000 7f88aa4c2700 1 -- 172.21.15.3:0/3596112795 learned_addr learned my addr 172.21.15.3:0/3596112795 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:39.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.742+0000 7f88a9cc1700 1 --1- 172.21.15.3:0/3596112795 >> v1:172.21.15.3:6789/0 conn(0x7f88a4125ad0 0x7f88a4093c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58128/0 (socket says 172.21.15.3:58128) 2023-10-03T10:09:39.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.743+0000 7f88aa4c2700 1 -- 172.21.15.3:0/3596112795 >> v2:172.21.15.3:3300/0 conn(0x7f88a4094360 msgr2=0x7f88a4130a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:39.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.743+0000 7f88aa4c2700 1 --2- 172.21.15.3:0/3596112795 >> v2:172.21.15.3:3300/0 conn(0x7f88a4094360 0x7f88a4130a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:39.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.743+0000 7f88aa4c2700 1 -- 172.21.15.3:0/3596112795 >> v1:172.21.15.3:6789/0 conn(0x7f88a4125ad0 legacy=0x7f88a4093c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:39.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.743+0000 7f88aa4c2700 1 -- 172.21.15.3:0/3596112795 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a40963e0 con 0x7f88a4062a90 2023-10-03T10:09:39.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.743+0000 7f88aacc3700 1 --2- 172.21.15.3:0/3596112795 >> v2:172.21.15.3:3300/0 conn(0x7f88a4094360 0x7f88a4130a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:39.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.743+0000 7f88aa4c2700 1 --2- 172.21.15.3:0/3596112795 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88a4062a90 0x7f88a41255e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f88a0006fd0 tx=0x7f88a00090f0).ready entity=mon.1 client_cookie=e488894b6dbccd3b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:39.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.743+0000 7f88a94c0700 1 -- 172.21.15.3:0/3596112795 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a000b580 con 0x7f88a4062a90 2023-10-03T10:09:39.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.743+0000 7f88a94c0700 1 -- 172.21.15.3:0/3596112795 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a000b6e0 con 0x7f88a4062a90 2023-10-03T10:09:39.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.743+0000 7f88a94c0700 1 -- 172.21.15.3:0/3596112795 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a000ba60 con 0x7f88a4062a90 2023-10-03T10:09:39.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.743+0000 7f88ac726700 1 -- 172.21.15.3:0/3596112795 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88a4062a90 msgr2=0x7f88a41255e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:39.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.743+0000 7f88ac726700 1 --2- 172.21.15.3:0/3596112795 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88a4062a90 0x7f88a41255e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f88a0006fd0 tx=0x7f88a00090f0).stop 2023-10-03T10:09:39.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.744+0000 7f88ac726700 1 -- 172.21.15.3:0/3596112795 shutdown_connections 2023-10-03T10:09:39.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.744+0000 7f88ac726700 1 --2- 172.21.15.3:0/3596112795 >> v2:172.21.15.3:3300/0 conn(0x7f88a4094360 0x7f88a4130a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:39.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.744+0000 7f88ac726700 1 --2- 172.21.15.3:0/3596112795 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88a4062a90 0x7f88a41255e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:39.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.744+0000 7f88ac726700 1 -- 172.21.15.3:0/3596112795 >> 172.21.15.3:0/3596112795 conn(0x7f88a4059bb0 msgr2=0x7f88a4097400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:39.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.744+0000 7f88ac726700 1 -- 172.21.15.3:0/3596112795 shutdown_connections 2023-10-03T10:09:39.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.744+0000 7f88ac726700 1 -- 172.21.15.3:0/3596112795 wait complete. 2023-10-03T10:09:39.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.744+0000 7f88ac726700 1 Processor -- start 2023-10-03T10:09:39.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.745+0000 7f88ac726700 1 -- start start 2023-10-03T10:09:39.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.745+0000 7f88ac726700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88a4062a90 0x7f88a4116590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:39.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.745+0000 7f88ac726700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88a4094360 0x7f88a4112f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:39.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.745+0000 7f88ac726700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88a4125ad0 0x7f88a4113410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:39.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.745+0000 7f88ac726700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a4116090 con 0x7f88a4062a90 2023-10-03T10:09:39.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.745+0000 7f88ac726700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a41142f0 con 0x7f88a4125ad0 2023-10-03T10:09:39.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.745+0000 7f88ac726700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f88a41150e0 con 0x7f88a4094360 2023-10-03T10:09:39.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.745+0000 7f88aa4c2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88a4062a90 0x7f88a4116590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:39.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.745+0000 7f88a9cc1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88a4094360 0x7f88a4112f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:39.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.745+0000 7f88aa4c2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88a4062a90 0x7f88a4116590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56780/0 (socket says 172.21.15.3:56780) 2023-10-03T10:09:39.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.745+0000 7f88aa4c2700 1 -- 172.21.15.3:0/2251405991 learned_addr learned my addr 172.21.15.3:0/2251405991 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:39.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.745+0000 7f88aacc3700 1 --2- 172.21.15.3:0/2251405991 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88a4125ad0 0x7f88a4113410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:39.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.745+0000 7f88aa4c2700 1 -- 172.21.15.3:0/2251405991 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88a4094360 msgr2=0x7f88a4112f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:39.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.745+0000 7f88aa4c2700 1 --2- 172.21.15.3:0/2251405991 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88a4094360 0x7f88a4112f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:39.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.746+0000 7f88aa4c2700 1 -- 172.21.15.3:0/2251405991 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88a4125ad0 msgr2=0x7f88a4113410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:39.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.746+0000 7f88aa4c2700 1 --2- 172.21.15.3:0/2251405991 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88a4125ad0 0x7f88a4113410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:39.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.746+0000 7f88aa4c2700 1 -- 172.21.15.3:0/2251405991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889c009a20 con 0x7f88a4062a90 2023-10-03T10:09:39.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.746+0000 7f88aacc3700 1 --2- 172.21.15.3:0/2251405991 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88a4125ad0 0x7f88a4113410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:39.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.746+0000 7f88aa4c2700 1 --2- 172.21.15.3:0/2251405991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88a4062a90 0x7f88a4116590 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f88a40636c0 tx=0x7f88a0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:39.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.746+0000 7f889a7fc700 1 -- 172.21.15.3:0/2251405991 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0000bc0 con 0x7f88a4062a90 2023-10-03T10:09:39.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.746+0000 7f889a7fc700 1 -- 172.21.15.3:0/2251405991 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a000b850 con 0x7f88a4062a90 2023-10-03T10:09:39.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.746+0000 7f88ac726700 1 -- 172.21.15.3:0/2251405991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88a0009370 con 0x7f88a4062a90 2023-10-03T10:09:39.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.746+0000 7f889a7fc700 1 -- 172.21.15.3:0/2251405991 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0000d20 con 0x7f88a4062a90 2023-10-03T10:09:39.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.746+0000 7f88ac726700 1 -- 172.21.15.3:0/2251405991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a4113900 con 0x7f88a4062a90 2023-10-03T10:09:39.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.747+0000 7f889a7fc700 1 -- 172.21.15.3:0/2251405991 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f88a001cad0 con 0x7f88a4062a90 2023-10-03T10:09:39.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.747+0000 7f889a7fc700 1 -- 172.21.15.3:0/2251405991 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f88a003cbe0 con 0x7f88a4062a90 2023-10-03T10:09:39.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.747+0000 7f88ac726700 1 -- 172.21.15.3:0/2251405991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88a405a260 con 0x7f88a4062a90 2023-10-03T10:09:39.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.752+0000 7f889a7fc700 1 -- 172.21.15.3:0/2251405991 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f88a002a020 con 0x7f88a4062a90 2023-10-03T10:09:39.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.929+0000 7f88ac726700 1 -- 172.21.15.3:0/2251405991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88a41140e0 con 0x7f88a4062a90 2023-10-03T10:09:39.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.929+0000 7f889a7fc700 1 -- 172.21.15.3:0/2251405991 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+8105 (secure 0 0 0) 0x7f88a0034090 con 0x7f88a4062a90 2023-10-03T10:09:39.930 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:39.930 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":12,"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","created":"2023-10-03T10:09:28.703441+0000","modified":"2023-10-03T10:09:39.573719+0000","last_up_change":"2023-10-03T10:09:36.569401+0000","last_in_change":"2023-10-03T10:09:30.156546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-10-03T10:09:36.623971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-03T10:09:38.210614+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"cdcdd930-3c5b-4486-a021-7132f90fb6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6808","nonce":118845}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6809","nonce":118845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6811","nonce":118845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6810","nonce":118845}]},"public_addr":"172.21.15.3:6808/118845","cluster_addr":"172.21.15.3:6809/118845","heartbeat_back_addr":"172.21.15.3:6811/118845","heartbeat_front_addr":"172.21.15.3:6810/118845","state":["exists","up"]},{"osd":1,"uuid":"c8e58877-6762-4741-a8c1-567ebc97b708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6804","nonce":118835}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6805","nonce":118835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6807","nonce":118835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6806","nonce":118835}]},"public_addr":"172.21.15.3:6804/118835","cluster_addr":"172.21.15.3:6805/118835","heartbeat_back_addr":"172.21.15.3:6807/118835","heartbeat_front_addr":"172.21.15.3:6806/118835","state":["exists","up"]},{"osd":2,"uuid":"68a2a21a-7a8e-4f82-8433-5201faa50777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6812","nonce":118864}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6813","nonce":118864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6815","nonce":118864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6814","nonce":118864}]},"public_addr":"172.21.15.3:6812/118864","cluster_addr":"172.21.15.3:6813/118864","heartbeat_back_addr":"172.21.15.3:6815/118864","heartbeat_front_addr":"172.21.15.3:6814/118864","state":["exists","up"]},{"osd":3,"uuid":"c4df2903-4e1e-4208-bf77-95b9fa749241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6800","nonce":118836}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6801","nonce":118836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6803","nonce":118836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6802","nonce":118836}]},"public_addr":"172.21.15.3:6800/118836","cluster_addr":"172.21.15.3:6801/118836","heartbeat_back_addr":"172.21.15.3:6803/118836","heartbeat_front_addr":"172.21.15.3:6802/118836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.369708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.257825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.337931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.085222+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-03T10:09:39.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.931+0000 7f88ac726700 1 -- 172.21.15.3:0/2251405991 >> v1:172.21.15.3:6816/118296 conn(0x7f888802bb60 legacy=0x7f888802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:39.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.931+0000 7f88ac726700 1 -- 172.21.15.3:0/2251405991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88a4062a90 msgr2=0x7f88a4116590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:39.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.931+0000 7f88ac726700 1 --2- 172.21.15.3:0/2251405991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88a4062a90 0x7f88a4116590 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f88a40636c0 tx=0x7f88a0017000).stop 2023-10-03T10:09:39.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.931+0000 7f88ac726700 1 -- 172.21.15.3:0/2251405991 shutdown_connections 2023-10-03T10:09:39.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.931+0000 7f88ac726700 1 --2- 172.21.15.3:0/2251405991 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88a4125ad0 0x7f88a4113410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:39.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.931+0000 7f88ac726700 1 --2- 172.21.15.3:0/2251405991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88a4062a90 0x7f88a4116590 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:39.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.931+0000 7f88ac726700 1 --2- 172.21.15.3:0/2251405991 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88a4094360 0x7f88a4112f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:39.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.931+0000 7f88ac726700 1 -- 172.21.15.3:0/2251405991 >> 172.21.15.3:0/2251405991 conn(0x7f88a4059bb0 msgr2=0x7f88a4098ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:39.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.932+0000 7f88ac726700 1 -- 172.21.15.3:0/2251405991 shutdown_connections 2023-10-03T10:09:39.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:39.932+0000 7f88ac726700 1 -- 172.21.15.3:0/2251405991 wait complete. 2023-10-03T10:09:39.945 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-10-03T10:09:36.623971+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-03T10:09:38.210614+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-03T10:09:39.945 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-10-03T10:09:40.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.078+0000 7f38b3aa0700 1 Processor -- start 2023-10-03T10:09:40.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.078+0000 7f38b3aa0700 1 -- start start 2023-10-03T10:09:40.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.078+0000 7f38b3aa0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38ac127630 0x7f38ac099810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:40.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.078+0000 7f38b3aa0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f38ac127bb0 0x7f38ac093640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:40.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.078+0000 7f38b3aa0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38ac124160 0x7f38ac1245e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:40.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.078+0000 7f38b3aa0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38ac127f90 con 0x7f38ac127bb0 2023-10-03T10:09:40.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.078+0000 7f38b3aa0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac091a40 con 0x7f38ac127630 2023-10-03T10:09:40.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.078+0000 7f38b3aa0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac094ef0 con 0x7f38ac124160 2023-10-03T10:09:40.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.079+0000 7f38b103b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f38ac127bb0 0x7f38ac093640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:40.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.079+0000 7f38b183c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38ac127630 0x7f38ac099810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:40.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.079+0000 7f38b103b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f38ac127bb0 0x7f38ac093640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56788/0 (socket says 172.21.15.3:56788) 2023-10-03T10:09:40.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.079+0000 7f38b183c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38ac127630 0x7f38ac099810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42096/0 (socket says 172.21.15.3:42096) 2023-10-03T10:09:40.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.079+0000 7f38b203d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38ac124160 0x7f38ac1245e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:40.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.079+0000 7f38b103b700 1 -- 172.21.15.3:0/3337979524 learned_addr learned my addr 172.21.15.3:0/3337979524 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:40.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.079+0000 7f38b103b700 1 -- 172.21.15.3:0/3337979524 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38ac124160 msgr2=0x7f38ac1245e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:40.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.079+0000 7f38b103b700 1 --2- 172.21.15.3:0/3337979524 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38ac124160 0x7f38ac1245e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:40.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.079+0000 7f38b103b700 1 -- 172.21.15.3:0/3337979524 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38ac127630 msgr2=0x7f38ac099810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:40.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.079+0000 7f38b103b700 1 --2- 172.21.15.3:0/3337979524 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38ac127630 0x7f38ac099810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:40.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.079+0000 7f38b103b700 1 -- 172.21.15.3:0/3337979524 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ac128190 con 0x7f38ac127bb0 2023-10-03T10:09:40.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.079+0000 7f38b183c700 1 --2- 172.21.15.3:0/3337979524 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38ac127630 0x7f38ac099810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:40.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.079+0000 7f38b203d700 1 --2- 172.21.15.3:0/3337979524 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38ac124160 0x7f38ac1245e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:40.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.079+0000 7f38b103b700 1 --2- 172.21.15.3:0/3337979524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38ac127bb0 0x7f38ac093640 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f389c009830 tx=0x7f389c0090f0).ready entity=mon.0 client_cookie=df0dc6a286a44968 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:40.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.079+0000 7f38b083a700 1 -- 172.21.15.3:0/3337979524 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c00f510 con 0x7f38ac127bb0 2023-10-03T10:09:40.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.079+0000 7f38b083a700 1 -- 172.21.15.3:0/3337979524 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c00f670 con 0x7f38ac127bb0 2023-10-03T10:09:40.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.079+0000 7f38b083a700 1 -- 172.21.15.3:0/3337979524 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c00fa10 con 0x7f38ac127bb0 2023-10-03T10:09:40.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.080+0000 7f38b3aa0700 1 -- 172.21.15.3:0/3337979524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38ac127bb0 msgr2=0x7f38ac093640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:40.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.080+0000 7f38b3aa0700 1 --2- 172.21.15.3:0/3337979524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38ac127bb0 0x7f38ac093640 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f389c009830 tx=0x7f389c0090f0).stop 2023-10-03T10:09:40.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.080+0000 7f38b3aa0700 1 -- 172.21.15.3:0/3337979524 shutdown_connections 2023-10-03T10:09:40.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.080+0000 7f38b3aa0700 1 --2- 172.21.15.3:0/3337979524 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38ac124160 0x7f38ac1245e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:40.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.080+0000 7f38b3aa0700 1 --2- 172.21.15.3:0/3337979524 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38ac127630 0x7f38ac099810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:40.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.080+0000 7f38b3aa0700 1 --2- 172.21.15.3:0/3337979524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38ac127bb0 0x7f38ac093640 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:40.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.080+0000 7f38b3aa0700 1 -- 172.21.15.3:0/3337979524 >> 172.21.15.3:0/3337979524 conn(0x7f38ac0599a0 msgr2=0x7f38ac097460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:40.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.080+0000 7f38b3aa0700 1 -- 172.21.15.3:0/3337979524 shutdown_connections 2023-10-03T10:09:40.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.080+0000 7f38b3aa0700 1 -- 172.21.15.3:0/3337979524 wait complete. 2023-10-03T10:09:40.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.081+0000 7f38b3aa0700 1 Processor -- start 2023-10-03T10:09:40.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.082+0000 7f38b3aa0700 1 -- start start 2023-10-03T10:09:40.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.082+0000 7f38b3aa0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38ac124160 0x7f38ac115ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:40.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.082+0000 7f38b3aa0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38ac127630 0x7f38ac1163b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:40.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.082+0000 7f38b3aa0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38ac127bb0 0x7f38ac1168a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:40.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.082+0000 7f38b3aa0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac116d90 con 0x7f38ac127630 2023-10-03T10:09:40.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.082+0000 7f38b3aa0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac117330 con 0x7f38ac124160 2023-10-03T10:09:40.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.082+0000 7f38b3aa0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac119400 con 0x7f38ac127bb0 2023-10-03T10:09:40.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.082+0000 7f38b183c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38ac124160 0x7f38ac115ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:40.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.082+0000 7f38b103b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38ac127630 0x7f38ac1163b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:40.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.082+0000 7f38b183c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38ac124160 0x7f38ac115ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42112/0 (socket says 172.21.15.3:42112) 2023-10-03T10:09:40.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.082+0000 7f38b183c700 1 -- 172.21.15.3:0/3484847330 learned_addr learned my addr 172.21.15.3:0/3484847330 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:40.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.082+0000 7f38b203d700 1 --2- 172.21.15.3:0/3484847330 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38ac127bb0 0x7f38ac1168a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:40.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.083+0000 7f38b103b700 1 -- 172.21.15.3:0/3484847330 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38ac127bb0 msgr2=0x7f38ac1168a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:40.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.083+0000 7f38b103b700 1 --2- 172.21.15.3:0/3484847330 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38ac127bb0 0x7f38ac1168a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:40.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.083+0000 7f38b103b700 1 -- 172.21.15.3:0/3484847330 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38ac124160 msgr2=0x7f38ac115ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:40.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.083+0000 7f38b103b700 1 --2- 172.21.15.3:0/3484847330 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38ac124160 0x7f38ac115ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:40.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.083+0000 7f38b103b700 1 -- 172.21.15.3:0/3484847330 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a4009a20 con 0x7f38ac127630 2023-10-03T10:09:40.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.083+0000 7f38b183c700 1 --2- 172.21.15.3:0/3484847330 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38ac124160 0x7f38ac115ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:40.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.083+0000 7f38b103b700 1 --2- 172.21.15.3:0/3484847330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38ac127630 0x7f38ac1163b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f389c014000 tx=0x7f389c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:40.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.083+0000 7f38a1ffb700 1 -- 172.21.15.3:0/3484847330 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c006910 con 0x7f38ac127630 2023-10-03T10:09:40.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.083+0000 7f38a1ffb700 1 -- 172.21.15.3:0/3484847330 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c000bc0 con 0x7f38ac127630 2023-10-03T10:09:40.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.083+0000 7f38a1ffb700 1 -- 172.21.15.3:0/3484847330 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c024470 con 0x7f38ac127630 2023-10-03T10:09:40.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.083+0000 7f38b3aa0700 1 -- 172.21.15.3:0/3484847330 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a800b540 con 0x7f38ac127630 2023-10-03T10:09:40.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.083+0000 7f38b3aa0700 1 -- 172.21.15.3:0/3484847330 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f389c009370 con 0x7f38ac127630 2023-10-03T10:09:40.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.084+0000 7f38a1ffb700 1 -- 172.21.15.3:0/3484847330 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f389c021070 con 0x7f38ac127630 2023-10-03T10:09:40.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.084+0000 7f38a1ffb700 1 -- 172.21.15.3:0/3484847330 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f389c03c4e0 con 0x7f38ac127630 2023-10-03T10:09:40.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.085+0000 7f38b3aa0700 1 -- 172.21.15.3:0/3484847330 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38ac0635e0 con 0x7f38ac127630 2023-10-03T10:09:40.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.089+0000 7f38a1ffb700 1 -- 172.21.15.3:0/3484847330 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f389c029030 con 0x7f38ac127630 2023-10-03T10:09:40.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.265+0000 7f38b3aa0700 1 -- 172.21.15.3:0/3484847330 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f38ac0631e0 con 0x7f38ac127630 2023-10-03T10:09:40.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.265+0000 7f38a1ffb700 1 -- 172.21.15.3:0/3484847330 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v12) v1 ==== 110+0+10 (secure 0 0 0) 0x7f389c010990 con 0x7f38ac127630 2023-10-03T10:09:40.266 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-03T10:09:40.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.267+0000 7f38b3aa0700 1 -- 172.21.15.3:0/3484847330 >> v1:172.21.15.3:6816/118296 conn(0x7f389002bb60 legacy=0x7f389002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:40.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.267+0000 7f38b3aa0700 1 -- 172.21.15.3:0/3484847330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38ac127630 msgr2=0x7f38ac1163b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:40.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.267+0000 7f38b3aa0700 1 --2- 172.21.15.3:0/3484847330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38ac127630 0x7f38ac1163b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f389c014000 tx=0x7f389c015000).stop 2023-10-03T10:09:40.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.267+0000 7f38b3aa0700 1 -- 172.21.15.3:0/3484847330 shutdown_connections 2023-10-03T10:09:40.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.267+0000 7f38b3aa0700 1 --2- 172.21.15.3:0/3484847330 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38ac127bb0 0x7f38ac1168a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:40.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.267+0000 7f38b3aa0700 1 --2- 172.21.15.3:0/3484847330 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38ac124160 0x7f38ac115ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:40.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.267+0000 7f38b3aa0700 1 --2- 172.21.15.3:0/3484847330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38ac127630 0x7f38ac1163b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:40.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.267+0000 7f38b3aa0700 1 -- 172.21.15.3:0/3484847330 >> 172.21.15.3:0/3484847330 conn(0x7f38ac0599a0 msgr2=0x7f38ac097310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:40.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.267+0000 7f38b3aa0700 1 -- 172.21.15.3:0/3484847330 shutdown_connections 2023-10-03T10:09:40.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.267+0000 7f38b3aa0700 1 -- 172.21.15.3:0/3484847330 wait complete. 2023-10-03T10:09:40.281 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-03T10:09:40.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.414+0000 7f267500f700 1 Processor -- start 2023-10-03T10:09:40.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.414+0000 7f267500f700 1 -- start start 2023-10-03T10:09:40.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.414+0000 7f267500f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2670096440 0x7f2670096820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:40.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.414+0000 7f267500f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f267012e280 0x7f2670130670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:40.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.414+0000 7f267500f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2670098330 con 0x7f2670096d10 2023-10-03T10:09:40.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.414+0000 7f267500f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2670097800 con 0x7f2670096440 2023-10-03T10:09:40.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.414+0000 7f267500f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f267006ab40 con 0x7f267012e280 2023-10-03T10:09:40.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.414+0000 7f266f59e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f267012e280 0x7f2670130670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:40.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.414+0000 7f266ed9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2670096440 0x7f2670096820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:40.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.414+0000 7f266f59e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f267012e280 0x7f2670130670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38268/0 (socket says 172.21.15.3:38268) 2023-10-03T10:09:40.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.414+0000 7f266f59e700 1 -- 172.21.15.3:0/3015143091 learned_addr learned my addr 172.21.15.3:0/3015143091 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:40.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.414+0000 7f266e59c700 1 --1- 172.21.15.3:0/3015143091 >> v1:172.21.15.3:6789/0 conn(0x7f2670096d10 0x7f26700970f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58134/0 (socket says 172.21.15.3:58134) 2023-10-03T10:09:40.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.415+0000 7f266ed9d700 1 -- 172.21.15.3:0/3015143091 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f267012e280 msgr2=0x7f2670130670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:40.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.415+0000 7f266ed9d700 1 --2- 172.21.15.3:0/3015143091 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f267012e280 0x7f2670130670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:40.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.415+0000 7f266ed9d700 1 -- 172.21.15.3:0/3015143091 >> v1:172.21.15.3:6789/0 conn(0x7f2670096d10 legacy=0x7f26700970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:40.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.415+0000 7f266ed9d700 1 -- 172.21.15.3:0/3015143091 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2670068cf0 con 0x7f2670096440 2023-10-03T10:09:40.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.415+0000 7f266dd9b700 1 -- 172.21.15.3:0/3015143091 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 672678945 0 0) 0x7f2670098330 con 0x7f2670096d10 2023-10-03T10:09:40.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.415+0000 7f266f59e700 1 --2- 172.21.15.3:0/3015143091 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f267012e280 0x7f2670130670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:40.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.415+0000 7f266ed9d700 1 --2- 172.21.15.3:0/3015143091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2670096440 0x7f2670096820 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f2660009790 tx=0x7f2660009060).ready entity=mon.0 client_cookie=121cc382f45ce119 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:40.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.415+0000 7f266dd9b700 1 -- 172.21.15.3:0/3015143091 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2660009120 con 0x7f2670096440 2023-10-03T10:09:40.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.415+0000 7f266dd9b700 1 -- 172.21.15.3:0/3015143091 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266000f510 con 0x7f2670096440 2023-10-03T10:09:40.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.415+0000 7f266dd9b700 1 -- 172.21.15.3:0/3015143091 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266000f890 con 0x7f2670096440 2023-10-03T10:09:40.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.416+0000 7f267500f700 1 -- 172.21.15.3:0/3015143091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2670096440 msgr2=0x7f2670096820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:40.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.416+0000 7f267500f700 1 --2- 172.21.15.3:0/3015143091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2670096440 0x7f2670096820 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f2660009790 tx=0x7f2660009060).stop 2023-10-03T10:09:40.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.416+0000 7f267500f700 1 -- 172.21.15.3:0/3015143091 shutdown_connections 2023-10-03T10:09:40.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.416+0000 7f267500f700 1 --2- 172.21.15.3:0/3015143091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2670096440 0x7f2670096820 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:40.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.416+0000 7f267500f700 1 --2- 172.21.15.3:0/3015143091 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f267012e280 0x7f2670130670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:40.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.416+0000 7f267500f700 1 -- 172.21.15.3:0/3015143091 >> 172.21.15.3:0/3015143091 conn(0x7f2670059bb0 msgr2=0x7f267009ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:40.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.416+0000 7f267500f700 1 -- 172.21.15.3:0/3015143091 shutdown_connections 2023-10-03T10:09:40.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.416+0000 7f267500f700 1 -- 172.21.15.3:0/3015143091 wait complete. 2023-10-03T10:09:40.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.417+0000 7f267500f700 1 Processor -- start 2023-10-03T10:09:40.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.417+0000 7f267500f700 1 -- start start 2023-10-03T10:09:40.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.417+0000 7f267500f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2670096440 0x7f267011ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:40.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.417+0000 7f267500f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2670096d10 0x7f2670117e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:40.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.417+0000 7f267500f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f267012e280 0x7f2670118320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:40.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.417+0000 7f267500f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2670067dc0 con 0x7f2670096440 2023-10-03T10:09:40.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.417+0000 7f267500f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f267011b330 con 0x7f267012e280 2023-10-03T10:09:40.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.417+0000 7f267500f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f267011bdd0 con 0x7f2670096d10 2023-10-03T10:09:40.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.417+0000 7f266ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2670096440 0x7f267011ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:40.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.417+0000 7f266ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2670096440 0x7f267011ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56826/0 (socket says 172.21.15.3:56826) 2023-10-03T10:09:40.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.417+0000 7f266ed9d700 1 -- 172.21.15.3:0/569272337 learned_addr learned my addr 172.21.15.3:0/569272337 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:40.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.417+0000 7f266e59c700 1 --2- 172.21.15.3:0/569272337 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2670096d10 0x7f2670117e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:40.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.417+0000 7f266f59e700 1 --2- 172.21.15.3:0/569272337 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f267012e280 0x7f2670118320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:40.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.418+0000 7f266e59c700 1 -- 172.21.15.3:0/569272337 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f267012e280 msgr2=0x7f2670118320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:40.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.418+0000 7f266e59c700 1 --2- 172.21.15.3:0/569272337 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f267012e280 0x7f2670118320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:40.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.418+0000 7f266e59c700 1 -- 172.21.15.3:0/569272337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2670096440 msgr2=0x7f267011ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:40.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.418+0000 7f266e59c700 1 --2- 172.21.15.3:0/569272337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2670096440 0x7f267011ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:40.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.418+0000 7f266e59c700 1 -- 172.21.15.3:0/569272337 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2668009ae0 con 0x7f2670096d10 2023-10-03T10:09:40.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.418+0000 7f266e59c700 1 --2- 172.21.15.3:0/569272337 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2670096d10 0x7f2670117e30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f266400c490 tx=0x7f2664007d50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:40.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.418+0000 7f2656ffd700 1 -- 172.21.15.3:0/569272337 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26640070d0 con 0x7f2670096d10 2023-10-03T10:09:40.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.418+0000 7f2656ffd700 1 -- 172.21.15.3:0/569272337 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2664007230 con 0x7f2670096d10 2023-10-03T10:09:40.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.418+0000 7f2656ffd700 1 -- 172.21.15.3:0/569272337 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26640056e0 con 0x7f2670096d10 2023-10-03T10:09:40.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.418+0000 7f267500f700 1 -- 172.21.15.3:0/569272337 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26600092a0 con 0x7f2670096d10 2023-10-03T10:09:40.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.418+0000 7f267500f700 1 -- 172.21.15.3:0/569272337 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26701199c0 con 0x7f2670096d10 2023-10-03T10:09:40.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.419+0000 7f267500f700 1 -- 172.21.15.3:0/569272337 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f267009bf10 con 0x7f2670096d10 2023-10-03T10:09:40.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.420+0000 7f2656ffd700 1 -- 172.21.15.3:0/569272337 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f2664005a90 con 0x7f2670096d10 2023-10-03T10:09:40.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.420+0000 7f2656ffd700 1 -- 172.21.15.3:0/569272337 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f266403a870 con 0x7f2670096d10 2023-10-03T10:09:40.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.424+0000 7f2656ffd700 1 -- 172.21.15.3:0/569272337 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f266401c350 con 0x7f2670096d10 2023-10-03T10:09:40.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.602+0000 7f267500f700 1 -- 172.21.15.3:0/569272337 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f267005a260 con 0x7f2670096d10 2023-10-03T10:09:40.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.602+0000 7f2656ffd700 1 -- 172.21.15.3:0/569272337 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v13) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2664010ab0 con 0x7f2670096d10 2023-10-03T10:09:40.603 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-03T10:09:40.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.604+0000 7f267500f700 1 -- 172.21.15.3:0/569272337 >> v1:172.21.15.3:6816/118296 conn(0x7f265c02bb60 legacy=0x7f265c02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:40.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.604+0000 7f267500f700 1 -- 172.21.15.3:0/569272337 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2670096d10 msgr2=0x7f2670117e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:40.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.604+0000 7f267500f700 1 --2- 172.21.15.3:0/569272337 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2670096d10 0x7f2670117e30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f266400c490 tx=0x7f2664007d50).stop 2023-10-03T10:09:40.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.604+0000 7f267500f700 1 -- 172.21.15.3:0/569272337 shutdown_connections 2023-10-03T10:09:40.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.604+0000 7f267500f700 1 --2- 172.21.15.3:0/569272337 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f267012e280 0x7f2670118320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:40.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.604+0000 7f267500f700 1 --2- 172.21.15.3:0/569272337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2670096440 0x7f267011ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:40.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.604+0000 7f267500f700 1 --2- 172.21.15.3:0/569272337 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2670096d10 0x7f2670117e30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:40.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.605+0000 7f267500f700 1 -- 172.21.15.3:0/569272337 >> 172.21.15.3:0/569272337 conn(0x7f2670059bb0 msgr2=0x7f267009a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:40.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.605+0000 7f267500f700 1 -- 172.21.15.3:0/569272337 shutdown_connections 2023-10-03T10:09:40.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.605+0000 7f267500f700 1 -- 172.21.15.3:0/569272337 wait complete. 2023-10-03T10:09:40.618 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-03T10:09:40.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.749+0000 7f61af41f700 1 Processor -- start 2023-10-03T10:09:40.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.749+0000 7f61af41f700 1 -- start start 2023-10-03T10:09:40.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.750+0000 7f61af41f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f61a809ab50 0x7f61a8098970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:40.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.750+0000 7f61af41f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61a8098e60 0x7f61a8068f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:40.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.750+0000 7f61af41f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61a8096520 con 0x7f61a8069450 2023-10-03T10:09:40.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.750+0000 7f61af41f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a809af30 con 0x7f61a8098e60 2023-10-03T10:09:40.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.750+0000 7f61af41f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f61a8095830 con 0x7f61a809ab50 2023-10-03T10:09:40.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.750+0000 7f61ad9bc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f61a8069450 0x7f61a812fbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58140/0 (socket says 172.21.15.3:58140) 2023-10-03T10:09:40.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.750+0000 7f61ad9bc700 1 -- 172.21.15.3:0/754330889 learned_addr learned my addr 172.21.15.3:0/754330889 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:09:40.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.750+0000 7f61ad1bb700 1 --2- 172.21.15.3:0/754330889 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f61a809ab50 0x7f61a8098970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:40.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.750+0000 7f61ac9ba700 1 --2- 172.21.15.3:0/754330889 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61a8098e60 0x7f61a8068f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.750+0000 7f619ffff700 1 -- 172.21.15.3:0/754330889 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1134710901 0 0) 0x7f61a8096520 con 0x7f61a8069450 2023-10-03T10:09:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.750+0000 7f619ffff700 1 -- 172.21.15.3:0/754330889 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61940033f0 con 0x7f61a8069450 2023-10-03T10:09:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.750+0000 7f61ad1bb700 1 -- 172.21.15.3:0/754330889 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61a8098e60 msgr2=0x7f61a8068f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.750+0000 7f61ad1bb700 1 --2- 172.21.15.3:0/754330889 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61a8098e60 0x7f61a8068f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.750+0000 7f61ad1bb700 1 -- 172.21.15.3:0/754330889 >> v1:172.21.15.3:6789/0 conn(0x7f61a8069450 legacy=0x7f61a812fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.750+0000 7f61ad1bb700 1 -- 172.21.15.3:0/754330889 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a8093e60 con 0x7f61a809ab50 2023-10-03T10:09:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.750+0000 7f61ac9ba700 1 --2- 172.21.15.3:0/754330889 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61a8098e60 0x7f61a8068f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.751+0000 7f61ad1bb700 1 --2- 172.21.15.3:0/754330889 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f61a809ab50 0x7f61a8098970 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f61a40097c0 tx=0x7f61a4009060).ready entity=mon.2 client_cookie=e8affce5514c0a02 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.751+0000 7f619ffff700 1 -- 172.21.15.3:0/754330889 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a400d4a0 con 0x7f61a809ab50 2023-10-03T10:09:40.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.751+0000 7f619ffff700 1 -- 172.21.15.3:0/754330889 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a400d600 con 0x7f61a809ab50 2023-10-03T10:09:40.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.751+0000 7f619ffff700 1 -- 172.21.15.3:0/754330889 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a400d980 con 0x7f61a809ab50 2023-10-03T10:09:40.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.751+0000 7f61af41f700 1 -- 172.21.15.3:0/754330889 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f61a809ab50 msgr2=0x7f61a8098970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:40.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.751+0000 7f61af41f700 1 --2- 172.21.15.3:0/754330889 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f61a809ab50 0x7f61a8098970 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f61a40097c0 tx=0x7f61a4009060).stop 2023-10-03T10:09:40.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.751+0000 7f61af41f700 1 -- 172.21.15.3:0/754330889 shutdown_connections 2023-10-03T10:09:40.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.751+0000 7f61af41f700 1 --2- 172.21.15.3:0/754330889 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f61a809ab50 0x7f61a8098970 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:40.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.751+0000 7f61af41f700 1 --2- 172.21.15.3:0/754330889 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61a8098e60 0x7f61a8068f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:40.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.751+0000 7f61af41f700 1 -- 172.21.15.3:0/754330889 >> 172.21.15.3:0/754330889 conn(0x7f61a8063190 msgr2=0x7f61a809a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:40.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.751+0000 7f61af41f700 1 -- 172.21.15.3:0/754330889 shutdown_connections 2023-10-03T10:09:40.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.752+0000 7f61af41f700 1 -- 172.21.15.3:0/754330889 wait complete. 2023-10-03T10:09:40.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.752+0000 7f61af41f700 1 Processor -- start 2023-10-03T10:09:40.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.752+0000 7f61af41f700 1 -- start start 2023-10-03T10:09:40.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.753+0000 7f61af41f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f61a8069450 0x7f61a8138380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:40.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.753+0000 7f61af41f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61a8098e60 0x7f61a8138870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:40.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.753+0000 7f61af41f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61a809ab50 0x7f61a8116e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:40.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.753+0000 7f61af41f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a81182c0 con 0x7f61a8098e60 2023-10-03T10:09:40.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.753+0000 7f61af41f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a8094c40 con 0x7f61a809ab50 2023-10-03T10:09:40.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.753+0000 7f61af41f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f61a812d120 con 0x7f61a8069450 2023-10-03T10:09:40.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.753+0000 7f61ad1bb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f61a8069450 0x7f61a8138380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:40.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.753+0000 7f61ac9ba700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61a8098e60 0x7f61a8138870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:40.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.753+0000 7f61ad1bb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f61a8069450 0x7f61a8138380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38306/0 (socket says 172.21.15.3:38306) 2023-10-03T10:09:40.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.753+0000 7f61ad1bb700 1 -- 172.21.15.3:0/1913147906 learned_addr learned my addr 172.21.15.3:0/1913147906 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:40.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.753+0000 7f61ad9bc700 1 --2- 172.21.15.3:0/1913147906 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61a809ab50 0x7f61a8116e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:40.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.754+0000 7f61ad9bc700 1 -- 172.21.15.3:0/1913147906 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f61a8069450 msgr2=0x7f61a8138380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:40.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.754+0000 7f61ad9bc700 1 --2- 172.21.15.3:0/1913147906 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f61a8069450 0x7f61a8138380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:40.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.754+0000 7f61ad9bc700 1 -- 172.21.15.3:0/1913147906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61a8098e60 msgr2=0x7f61a8138870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:40.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.754+0000 7f61ad9bc700 1 --2- 172.21.15.3:0/1913147906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61a8098e60 0x7f61a8138870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:40.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.754+0000 7f61ad9bc700 1 -- 172.21.15.3:0/1913147906 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61980092a0 con 0x7f61a809ab50 2023-10-03T10:09:40.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.754+0000 7f61ad1bb700 1 --2- 172.21.15.3:0/1913147906 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f61a8069450 0x7f61a8138380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:40.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.754+0000 7f61ac9ba700 1 --2- 172.21.15.3:0/1913147906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61a8098e60 0x7f61a8138870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:40.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.754+0000 7f61ad9bc700 1 --2- 172.21.15.3:0/1913147906 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61a809ab50 0x7f61a8116e50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f61a000c710 tx=0x7f61a0007f10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:40.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.754+0000 7f619d7fa700 1 -- 172.21.15.3:0/1913147906 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a0005d30 con 0x7f61a809ab50 2023-10-03T10:09:40.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.754+0000 7f619d7fa700 1 -- 172.21.15.3:0/1913147906 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a0005e90 con 0x7f61a809ab50 2023-10-03T10:09:40.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.754+0000 7f61af41f700 1 -- 172.21.15.3:0/1913147906 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a40092a0 con 0x7f61a809ab50 2023-10-03T10:09:40.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.754+0000 7f619d7fa700 1 -- 172.21.15.3:0/1913147906 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a0011840 con 0x7f61a809ab50 2023-10-03T10:09:40.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.754+0000 7f61af41f700 1 -- 172.21.15.3:0/1913147906 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a811a270 con 0x7f61a809ab50 2023-10-03T10:09:40.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.755+0000 7f619d7fa700 1 -- 172.21.15.3:0/1913147906 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f61a0011c30 con 0x7f61a809ab50 2023-10-03T10:09:40.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.755+0000 7f619d7fa700 1 -- 172.21.15.3:0/1913147906 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f61a003e9b0 con 0x7f61a809ab50 2023-10-03T10:09:40.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.755+0000 7f61af41f700 1 -- 172.21.15.3:0/1913147906 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a811a470 con 0x7f61a809ab50 2023-10-03T10:09:40.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.760+0000 7f619d7fa700 1 -- 172.21.15.3:0/1913147906 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f61a811a470 con 0x7f61a809ab50 2023-10-03T10:09:40.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.979+0000 7f61af41f700 1 -- 172.21.15.3:0/1913147906 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61a811a470 con 0x7f61a809ab50 2023-10-03T10:09:40.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.979+0000 7f619d7fa700 1 -- 172.21.15.3:0/1913147906 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1194 (secure 0 0 0) 0x7f61a811a470 con 0x7f61a809ab50 2023-10-03T10:09:40.980 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:40.980 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4186,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6817/2582564779","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6817","nonce":2582564779}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-10-03T10:09:40.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.981+0000 7f61af41f700 1 -- 172.21.15.3:0/1913147906 >> v1:172.21.15.3:6816/118296 conn(0x7f618c02bb60 legacy=0x7f618c02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:40.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.981+0000 7f61af41f700 1 -- 172.21.15.3:0/1913147906 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61a809ab50 msgr2=0x7f61a8116e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:40.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.981+0000 7f61af41f700 1 --2- 172.21.15.3:0/1913147906 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61a809ab50 0x7f61a8116e50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f61a000c710 tx=0x7f61a0007f10).stop 2023-10-03T10:09:40.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.981+0000 7f61af41f700 1 -- 172.21.15.3:0/1913147906 shutdown_connections 2023-10-03T10:09:40.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.981+0000 7f61af41f700 1 --2- 172.21.15.3:0/1913147906 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61a809ab50 0x7f61a8116e50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:40.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.981+0000 7f61af41f700 1 --2- 172.21.15.3:0/1913147906 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f61a8069450 0x7f61a8138380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.981+0000 7f61af41f700 1 --2- 172.21.15.3:0/1913147906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61a8098e60 0x7f61a8138870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.981+0000 7f61af41f700 1 -- 172.21.15.3:0/1913147906 >> 172.21.15.3:0/1913147906 conn(0x7f61a8063190 msgr2=0x7f61a812bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.981+0000 7f61af41f700 1 -- 172.21.15.3:0/1913147906 shutdown_connections 2023-10-03T10:09:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:40.981+0000 7f61af41f700 1 -- 172.21.15.3:0/1913147906 wait complete. 2023-10-03T10:09:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 2 2023-10-03T10:09:40.995 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-03T10:09:41.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.125+0000 7f0f05954700 1 Processor -- start 2023-10-03T10:09:41.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.125+0000 7f0f05954700 1 -- start start 2023-10-03T10:09:41.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.125+0000 7f0f05954700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f00127360 0x7f0f0012fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:41.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.125+0000 7f0f05954700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f00127f70 0x7f0f00132140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:41.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.125+0000 7f0f05954700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f00091bc0 con 0x7f0f00062a90 2023-10-03T10:09:41.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.125+0000 7f0f05954700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f00062e70 con 0x7f0f00127360 2023-10-03T10:09:41.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.125+0000 7f0f05954700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f000968b0 con 0x7f0f00127f70 2023-10-03T10:09:41.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.125+0000 7f0eff7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f00127f70 0x7f0f00132140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:41.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.125+0000 7f0efeffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0f00062a90 0x7f0f00127860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58150/0 (socket says 172.21.15.3:58150) 2023-10-03T10:09:41.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.125+0000 7f0efeffd700 1 -- 172.21.15.3:0/2251434235 learned_addr learned my addr 172.21.15.3:0/2251434235 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:09:41.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.125+0000 7f0efe7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f00127360 0x7f0f0012fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:41.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.126+0000 7f0efdffb700 1 -- 172.21.15.3:0/2251434235 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1098641104 0 0) 0x7f0f00091bc0 con 0x7f0f00062a90 2023-10-03T10:09:41.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.126+0000 7f0efe7fc700 1 -- 172.21.15.3:0/2251434235 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f00127f70 msgr2=0x7f0f00132140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:41.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.126+0000 7f0efe7fc700 1 --2- 172.21.15.3:0/2251434235 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f00127f70 0x7f0f00132140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:41.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.126+0000 7f0efe7fc700 1 -- 172.21.15.3:0/2251434235 >> v1:172.21.15.3:6789/0 conn(0x7f0f00062a90 legacy=0x7f0f00127860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:41.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.126+0000 7f0efe7fc700 1 -- 172.21.15.3:0/2251434235 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f00092a70 con 0x7f0f00127360 2023-10-03T10:09:41.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.126+0000 7f0eff7fe700 1 --2- 172.21.15.3:0/2251434235 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f00127f70 0x7f0f00132140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:41.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.126+0000 7f0efe7fc700 1 --2- 172.21.15.3:0/2251434235 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f00127360 0x7f0f0012fc40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f0ef4009790 tx=0x7f0ef4009060).ready entity=mon.1 client_cookie=22ca57b5995fe1f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:41.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.126+0000 7f0efdffb700 1 -- 172.21.15.3:0/2251434235 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef40090d0 con 0x7f0f00127360 2023-10-03T10:09:41.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.126+0000 7f0efdffb700 1 -- 172.21.15.3:0/2251434235 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ef400f510 con 0x7f0f00127360 2023-10-03T10:09:41.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.126+0000 7f0efdffb700 1 -- 172.21.15.3:0/2251434235 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef400f890 con 0x7f0f00127360 2023-10-03T10:09:41.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.127+0000 7f0f05954700 1 -- 172.21.15.3:0/2251434235 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f00127360 msgr2=0x7f0f0012fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:41.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.127+0000 7f0f05954700 1 --2- 172.21.15.3:0/2251434235 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f00127360 0x7f0f0012fc40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f0ef4009790 tx=0x7f0ef4009060).stop 2023-10-03T10:09:41.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.127+0000 7f0f05954700 1 -- 172.21.15.3:0/2251434235 shutdown_connections 2023-10-03T10:09:41.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.127+0000 7f0f05954700 1 --2- 172.21.15.3:0/2251434235 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f00127f70 0x7f0f00132140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:41.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.127+0000 7f0f05954700 1 --2- 172.21.15.3:0/2251434235 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f00127360 0x7f0f0012fc40 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:41.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.127+0000 7f0f05954700 1 -- 172.21.15.3:0/2251434235 >> 172.21.15.3:0/2251434235 conn(0x7f0f00059bb0 msgr2=0x7f0f00097400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:41.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.127+0000 7f0f05954700 1 -- 172.21.15.3:0/2251434235 shutdown_connections 2023-10-03T10:09:41.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.127+0000 7f0f05954700 1 -- 172.21.15.3:0/2251434235 wait complete. 2023-10-03T10:09:41.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.128+0000 7f0f05954700 1 Processor -- start 2023-10-03T10:09:41.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.128+0000 7f0f05954700 1 -- start start 2023-10-03T10:09:41.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.128+0000 7f0f05954700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f00062a90 0x7f0f0012dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:41.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.128+0000 7f0f05954700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f00127360 0x7f0f0012e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:41.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.128+0000 7f0f05954700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f00127f70 0x7f0f0012e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:41.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.128+0000 7f0f05954700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f001313a0 con 0x7f0f00127360 2023-10-03T10:09:41.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.128+0000 7f0f05954700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f001320a0 con 0x7f0f00062a90 2023-10-03T10:09:41.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.128+0000 7f0f05954700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f00096e90 con 0x7f0f00127f70 2023-10-03T10:09:41.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.129+0000 7f0efeffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f00062a90 0x7f0f0012dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:41.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.129+0000 7f0efeffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f00062a90 0x7f0f0012dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42146/0 (socket says 172.21.15.3:42146) 2023-10-03T10:09:41.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.129+0000 7f0efeffd700 1 -- 172.21.15.3:0/2157085528 learned_addr learned my addr 172.21.15.3:0/2157085528 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:41.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.129+0000 7f0efe7fc700 1 --2- 172.21.15.3:0/2157085528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f00127360 0x7f0f0012e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:41.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.129+0000 7f0eff7fe700 1 --2- 172.21.15.3:0/2157085528 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f00127f70 0x7f0f0012e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:41.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.129+0000 7f0efeffd700 1 -- 172.21.15.3:0/2157085528 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f00127f70 msgr2=0x7f0f0012e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:41.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.129+0000 7f0efeffd700 1 --2- 172.21.15.3:0/2157085528 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f00127f70 0x7f0f0012e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:41.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.129+0000 7f0efeffd700 1 -- 172.21.15.3:0/2157085528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f00127360 msgr2=0x7f0f0012e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:41.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.129+0000 7f0efeffd700 1 --2- 172.21.15.3:0/2157085528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f00127360 0x7f0f0012e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:41.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.129+0000 7f0efeffd700 1 -- 172.21.15.3:0/2157085528 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef8009a20 con 0x7f0f00062a90 2023-10-03T10:09:41.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.129+0000 7f0efe7fc700 1 --2- 172.21.15.3:0/2157085528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f00127360 0x7f0f0012e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:41.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.129+0000 7f0efeffd700 1 --2- 172.21.15.3:0/2157085528 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f00062a90 0x7f0f0012dbd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f0ef000c4f0 tx=0x7f0ef0007c10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:41.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.129+0000 7f0ee6ffd700 1 -- 172.21.15.3:0/2157085528 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef00072e0 con 0x7f0f00062a90 2023-10-03T10:09:41.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.129+0000 7f0ee6ffd700 1 -- 172.21.15.3:0/2157085528 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ef0007460 con 0x7f0f00062a90 2023-10-03T10:09:41.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.129+0000 7f0ee6ffd700 1 -- 172.21.15.3:0/2157085528 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef00059f0 con 0x7f0f00062a90 2023-10-03T10:09:41.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.129+0000 7f0f05954700 1 -- 172.21.15.3:0/2157085528 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef40092a0 con 0x7f0f00062a90 2023-10-03T10:09:41.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.129+0000 7f0f05954700 1 -- 172.21.15.3:0/2157085528 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f00125060 con 0x7f0f00062a90 2023-10-03T10:09:41.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.130+0000 7f0ee6ffd700 1 -- 172.21.15.3:0/2157085528 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f0ef0013450 con 0x7f0f00062a90 2023-10-03T10:09:41.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.130+0000 7f0ee6ffd700 1 -- 172.21.15.3:0/2157085528 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f0ef003a320 con 0x7f0f00062a90 2023-10-03T10:09:41.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.130+0000 7f0f05954700 1 -- 172.21.15.3:0/2157085528 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f00091de0 con 0x7f0f00062a90 2023-10-03T10:09:41.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.135+0000 7f0ee6ffd700 1 -- 172.21.15.3:0/2157085528 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f0ef00075d0 con 0x7f0f00062a90 2023-10-03T10:09:41.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.311+0000 7f0f05954700 1 -- 172.21.15.3:0/2157085528 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f0005a260 con 0x7f0f00062a90 2023-10-03T10:09:41.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.312+0000 7f0ee6ffd700 1 -- 172.21.15.3:0/2157085528 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+8092 (secure 0 0 0) 0x7f0ef001c350 con 0x7f0f00062a90 2023-10-03T10:09:41.312 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:41.312 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":13,"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","created":"2023-10-03T10:09:28.703441+0000","modified":"2023-10-03T10:09:40.574956+0000","last_up_change":"2023-10-03T10:09:36.569401+0000","last_in_change":"2023-10-03T10:09:30.156546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-10-03T10:09:36.623971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-03T10:09:38.210614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"cdcdd930-3c5b-4486-a021-7132f90fb6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6808","nonce":118845}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6809","nonce":118845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6811","nonce":118845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6810","nonce":118845}]},"public_addr":"172.21.15.3:6808/118845","cluster_addr":"172.21.15.3:6809/118845","heartbeat_back_addr":"172.21.15.3:6811/118845","heartbeat_front_addr":"172.21.15.3:6810/118845","state":["exists","up"]},{"osd":1,"uuid":"c8e58877-6762-4741-a8c1-567ebc97b708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6804","nonce":118835}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6805","nonce":118835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6807","nonce":118835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6806","nonce":118835}]},"public_addr":"172.21.15.3:6804/118835","cluster_addr":"172.21.15.3:6805/118835","heartbeat_back_addr":"172.21.15.3:6807/118835","heartbeat_front_addr":"172.21.15.3:6806/118835","state":["exists","up"]},{"osd":2,"uuid":"68a2a21a-7a8e-4f82-8433-5201faa50777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6812","nonce":118864}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6813","nonce":118864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6815","nonce":118864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6814","nonce":118864}]},"public_addr":"172.21.15.3:6812/118864","cluster_addr":"172.21.15.3:6813/118864","heartbeat_back_addr":"172.21.15.3:6815/118864","heartbeat_front_addr":"172.21.15.3:6814/118864","state":["exists","up"]},{"osd":3,"uuid":"c4df2903-4e1e-4208-bf77-95b9fa749241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6800","nonce":118836}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6801","nonce":118836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6803","nonce":118836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6802","nonce":118836}]},"public_addr":"172.21.15.3:6800/118836","cluster_addr":"172.21.15.3:6801/118836","heartbeat_back_addr":"172.21.15.3:6803/118836","heartbeat_front_addr":"172.21.15.3:6802/118836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.369708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.257825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.337931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.085222+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-03T10:09:41.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.314+0000 7f0f05954700 1 -- 172.21.15.3:0/2157085528 >> v1:172.21.15.3:6816/118296 conn(0x7f0eec02bb90 legacy=0x7f0eec02e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:41.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.314+0000 7f0f05954700 1 -- 172.21.15.3:0/2157085528 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f00062a90 msgr2=0x7f0f0012dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:41.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.314+0000 7f0f05954700 1 --2- 172.21.15.3:0/2157085528 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f00062a90 0x7f0f0012dbd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f0ef000c4f0 tx=0x7f0ef0007c10).stop 2023-10-03T10:09:41.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.314+0000 7f0f05954700 1 -- 172.21.15.3:0/2157085528 shutdown_connections 2023-10-03T10:09:41.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.314+0000 7f0f05954700 1 --2- 172.21.15.3:0/2157085528 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f00127f70 0x7f0f0012e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:41.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.314+0000 7f0f05954700 1 --2- 172.21.15.3:0/2157085528 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f00062a90 0x7f0f0012dbd0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:41.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.314+0000 7f0f05954700 1 --2- 172.21.15.3:0/2157085528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f00127360 0x7f0f0012e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:41.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.314+0000 7f0f05954700 1 -- 172.21.15.3:0/2157085528 >> 172.21.15.3:0/2157085528 conn(0x7f0f00059bb0 msgr2=0x7f0f00126160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:41.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.314+0000 7f0f05954700 1 -- 172.21.15.3:0/2157085528 shutdown_connections 2023-10-03T10:09:41.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.314+0000 7f0f05954700 1 -- 172.21.15.3:0/2157085528 wait complete. 2023-10-03T10:09:41.327 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-10-03T10:09:36.623971+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-03T10:09:38.210614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-03T10:09:41.328 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-10-03T10:09:41.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.460+0000 7fa541fd7700 1 Processor -- start 2023-10-03T10:09:41.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.460+0000 7fa541fd7700 1 -- start start 2023-10-03T10:09:41.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.461+0000 7fa541fd7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa53c099810 0x7fa53c127bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:41.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.461+0000 7fa541fd7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa53c093640 0x7fa53c124870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:41.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.461+0000 7fa541fd7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa53c091a40 con 0x7fa53c127630 2023-10-03T10:09:41.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.461+0000 7fa541fd7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c099bf0 con 0x7fa53c099810 2023-10-03T10:09:41.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.461+0000 7fa541fd7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c094ef0 con 0x7fa53c093640 2023-10-03T10:09:41.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.461+0000 7fa53bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa53c093640 0x7fa53c124870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:41.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.461+0000 7fa53b7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa53c127630 0x7fa53c124160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58160/0 (socket says 172.21.15.3:58160) 2023-10-03T10:09:41.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.461+0000 7fa53affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa53c099810 0x7fa53c127bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:41.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.461+0000 7fa53b7fe700 1 -- 172.21.15.3:0/1218234780 learned_addr learned my addr 172.21.15.3:0/1218234780 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:09:41.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.461+0000 7fa53a7fc700 1 -- 172.21.15.3:0/1218234780 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1466896041 0 0) 0x7fa53c091a40 con 0x7fa53c127630 2023-10-03T10:09:41.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.461+0000 7fa53a7fc700 1 -- 172.21.15.3:0/1218234780 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5280033f0 con 0x7fa53c127630 2023-10-03T10:09:41.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.461+0000 7fa53bfff700 1 -- 172.21.15.3:0/1218234780 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa53c099810 msgr2=0x7fa53c127bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:41.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.461+0000 7fa53bfff700 1 --2- 172.21.15.3:0/1218234780 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa53c099810 0x7fa53c127bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:41.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.461+0000 7fa53bfff700 1 -- 172.21.15.3:0/1218234780 >> v1:172.21.15.3:6789/0 conn(0x7fa53c127630 legacy=0x7fa53c124160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:41.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.461+0000 7fa53bfff700 1 -- 172.21.15.3:0/1218234780 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa53c128190 con 0x7fa53c093640 2023-10-03T10:09:41.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.461+0000 7fa53affd700 1 --2- 172.21.15.3:0/1218234780 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa53c099810 0x7fa53c127bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.462+0000 7fa53bfff700 1 --2- 172.21.15.3:0/1218234780 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa53c093640 0x7fa53c124870 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fa534009ec0 tx=0x7fa534009780).ready entity=mon.2 client_cookie=94519f1424019b76 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.462+0000 7fa53a7fc700 1 -- 172.21.15.3:0/1218234780 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53400e070 con 0x7fa53c093640 2023-10-03T10:09:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.462+0000 7fa53a7fc700 1 -- 172.21.15.3:0/1218234780 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa53400bae0 con 0x7fa53c093640 2023-10-03T10:09:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.462+0000 7fa53a7fc700 1 -- 172.21.15.3:0/1218234780 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53400be60 con 0x7fa53c093640 2023-10-03T10:09:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.462+0000 7fa541fd7700 1 -- 172.21.15.3:0/1218234780 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa53c093640 msgr2=0x7fa53c124870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.462+0000 7fa541fd7700 1 --2- 172.21.15.3:0/1218234780 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa53c093640 0x7fa53c124870 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fa534009ec0 tx=0x7fa534009780).stop 2023-10-03T10:09:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.462+0000 7fa541fd7700 1 -- 172.21.15.3:0/1218234780 shutdown_connections 2023-10-03T10:09:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.462+0000 7fa541fd7700 1 --2- 172.21.15.3:0/1218234780 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa53c093640 0x7fa53c124870 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.462+0000 7fa541fd7700 1 --2- 172.21.15.3:0/1218234780 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa53c099810 0x7fa53c127bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.462+0000 7fa541fd7700 1 -- 172.21.15.3:0/1218234780 >> 172.21.15.3:0/1218234780 conn(0x7fa53c0599a0 msgr2=0x7fa53c097460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.462+0000 7fa541fd7700 1 -- 172.21.15.3:0/1218234780 shutdown_connections 2023-10-03T10:09:41.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.463+0000 7fa541fd7700 1 -- 172.21.15.3:0/1218234780 wait complete. 2023-10-03T10:09:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.464+0000 7fa541fd7700 1 Processor -- start 2023-10-03T10:09:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.464+0000 7fa541fd7700 1 -- start start 2023-10-03T10:09:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.464+0000 7fa541fd7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa53c093640 0x7fa53c062ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.464+0000 7fa541fd7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa53c099810 0x7fa53c114ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.464+0000 7fa541fd7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa53c127630 0x7fa53c114fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.464+0000 7fa541fd7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c123b60 con 0x7fa53c099810 2023-10-03T10:09:41.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.464+0000 7fa541fd7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c092350 con 0x7fa53c093640 2023-10-03T10:09:41.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.464+0000 7fa541fd7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c117540 con 0x7fa53c127630 2023-10-03T10:09:41.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.464+0000 7fa53b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa53c093640 0x7fa53c062ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:41.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.464+0000 7fa53b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa53c093640 0x7fa53c062ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42164/0 (socket says 172.21.15.3:42164) 2023-10-03T10:09:41.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.464+0000 7fa53b7fe700 1 -- 172.21.15.3:0/395737125 learned_addr learned my addr 172.21.15.3:0/395737125 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:41.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.464+0000 7fa53affd700 1 --2- 172.21.15.3:0/395737125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa53c099810 0x7fa53c114ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:41.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.464+0000 7fa53bfff700 1 --2- 172.21.15.3:0/395737125 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa53c127630 0x7fa53c114fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:41.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.465+0000 7fa53b7fe700 1 -- 172.21.15.3:0/395737125 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa53c127630 msgr2=0x7fa53c114fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:41.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.465+0000 7fa53b7fe700 1 --2- 172.21.15.3:0/395737125 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa53c127630 0x7fa53c114fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:41.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.465+0000 7fa53b7fe700 1 -- 172.21.15.3:0/395737125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa53c099810 msgr2=0x7fa53c114ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:41.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.465+0000 7fa53b7fe700 1 --2- 172.21.15.3:0/395737125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa53c099810 0x7fa53c114ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:41.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.465+0000 7fa53b7fe700 1 -- 172.21.15.3:0/395737125 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5240092a0 con 0x7fa53c093640 2023-10-03T10:09:41.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.465+0000 7fa53bfff700 1 --2- 172.21.15.3:0/395737125 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa53c127630 0x7fa53c114fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:41.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.465+0000 7fa53affd700 1 --2- 172.21.15.3:0/395737125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa53c099810 0x7fa53c114ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:41.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.465+0000 7fa53b7fe700 1 --2- 172.21.15.3:0/395737125 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa53c093640 0x7fa53c062ae0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fa52c00c590 tx=0x7fa52c007c80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:41.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.465+0000 7fa5337fe700 1 -- 172.21.15.3:0/395737125 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c0072e0 con 0x7fa53c093640 2023-10-03T10:09:41.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.465+0000 7fa5337fe700 1 -- 172.21.15.3:0/395737125 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52c007440 con 0x7fa53c093640 2023-10-03T10:09:41.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.465+0000 7fa541fd7700 1 -- 172.21.15.3:0/395737125 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa534009a20 con 0x7fa53c093640 2023-10-03T10:09:41.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.465+0000 7fa5337fe700 1 -- 172.21.15.3:0/395737125 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c0057a0 con 0x7fa53c093640 2023-10-03T10:09:41.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.465+0000 7fa541fd7700 1 -- 172.21.15.3:0/395737125 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa53c0638b0 con 0x7fa53c093640 2023-10-03T10:09:41.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.466+0000 7fa5337fe700 1 -- 172.21.15.3:0/395737125 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fa52c005bd0 con 0x7fa53c093640 2023-10-03T10:09:41.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.466+0000 7fa5337fe700 1 -- 172.21.15.3:0/395737125 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7fa52c03a060 con 0x7fa53c093640 2023-10-03T10:09:41.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.466+0000 7fa541fd7700 1 -- 172.21.15.3:0/395737125 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa53c125ea0 con 0x7fa53c093640 2023-10-03T10:09:41.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.471+0000 7fa5337fe700 1 -- 172.21.15.3:0/395737125 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fa52c018070 con 0x7fa53c093640 2023-10-03T10:09:41.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.648+0000 7fa541fd7700 1 -- 172.21.15.3:0/395737125 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa53c059db0 con 0x7fa53c093640 2023-10-03T10:09:41.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.648+0000 7fa5337fe700 1 -- 172.21.15.3:0/395737125 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v13) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa52c018020 con 0x7fa53c093640 2023-10-03T10:09:41.649 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-03T10:09:41.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.650+0000 7fa541fd7700 1 -- 172.21.15.3:0/395737125 >> v1:172.21.15.3:6816/118296 conn(0x7fa51c02bb60 legacy=0x7fa51c02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:41.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.650+0000 7fa541fd7700 1 -- 172.21.15.3:0/395737125 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa53c093640 msgr2=0x7fa53c062ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:41.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.650+0000 7fa541fd7700 1 --2- 172.21.15.3:0/395737125 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa53c093640 0x7fa53c062ae0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fa52c00c590 tx=0x7fa52c007c80).stop 2023-10-03T10:09:41.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.650+0000 7fa541fd7700 1 -- 172.21.15.3:0/395737125 shutdown_connections 2023-10-03T10:09:41.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.650+0000 7fa541fd7700 1 --2- 172.21.15.3:0/395737125 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa53c127630 0x7fa53c114fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:41.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.650+0000 7fa541fd7700 1 --2- 172.21.15.3:0/395737125 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa53c093640 0x7fa53c062ae0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:41.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.650+0000 7fa541fd7700 1 --2- 172.21.15.3:0/395737125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa53c099810 0x7fa53c114ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:41.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.650+0000 7fa541fd7700 1 -- 172.21.15.3:0/395737125 >> 172.21.15.3:0/395737125 conn(0x7fa53c0599a0 msgr2=0x7fa53c098730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:41.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.651+0000 7fa541fd7700 1 -- 172.21.15.3:0/395737125 shutdown_connections 2023-10-03T10:09:41.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.651+0000 7fa541fd7700 1 -- 172.21.15.3:0/395737125 wait complete. 2023-10-03T10:09:41.664 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-03T10:09:41.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.798+0000 7fb7d1bbd700 1 Processor -- start 2023-10-03T10:09:41.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.798+0000 7fb7d1bbd700 1 -- start start 2023-10-03T10:09:41.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.799+0000 7fb7d1bbd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7cc091a40 0x7fb7cc091e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:41.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.799+0000 7fb7d1bbd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb7cc128090 0x7fb7cc12fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:41.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.799+0000 7fb7d1bbd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7cc092c60 con 0x7fb7cc092310 2023-10-03T10:09:41.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.799+0000 7fb7d1bbd700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7cc0950b0 con 0x7fb7cc128090 2023-10-03T10:09:41.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.799+0000 7fb7d1bbd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc0947c0 con 0x7fb7cc091a40 2023-10-03T10:09:41.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.799+0000 7fb7cb7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7cc091a40 0x7fb7cc091e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:41.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.799+0000 7fb7cbfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb7cc128090 0x7fb7cc12fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:41.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.799+0000 7fb7caffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb7cc092310 0x7fb7cc127980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58162/0 (socket says 172.21.15.3:58162) 2023-10-03T10:09:41.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.799+0000 7fb7caffd700 1 -- 172.21.15.3:0/1166961897 learned_addr learned my addr 172.21.15.3:0/1166961897 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:09:41.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.799+0000 7fb7cb7fe700 1 -- 172.21.15.3:0/1166961897 >> v2:172.21.15.3:3300/0 conn(0x7fb7cc128090 msgr2=0x7fb7cc12fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:41.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.799+0000 7fb7cb7fe700 1 --2- 172.21.15.3:0/1166961897 >> v2:172.21.15.3:3300/0 conn(0x7fb7cc128090 0x7fb7cc12fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:41.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.799+0000 7fb7cb7fe700 1 -- 172.21.15.3:0/1166961897 >> v1:172.21.15.3:6789/0 conn(0x7fb7cc092310 legacy=0x7fb7cc127980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:41.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.800+0000 7fb7cb7fe700 1 -- 172.21.15.3:0/1166961897 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7cc094470 con 0x7fb7cc091a40 2023-10-03T10:09:41.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.800+0000 7fb7ca7fc700 1 -- 172.21.15.3:0/1166961897 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 598369632 0 0) 0x7fb7cc092c60 con 0x7fb7cc092310 2023-10-03T10:09:41.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.800+0000 7fb7cbfff700 1 --2- 172.21.15.3:0/1166961897 >> v2:172.21.15.3:3300/0 conn(0x7fb7cc128090 0x7fb7cc12fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:41.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.800+0000 7fb7cb7fe700 1 --2- 172.21.15.3:0/1166961897 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7cc091a40 0x7fb7cc091e20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fb7bc00ba30 tx=0x7fb7bc00b300).ready entity=mon.1 client_cookie=eb4c924eb30439af server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:41.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.800+0000 7fb7ca7fc700 1 -- 172.21.15.3:0/1166961897 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc00da30 con 0x7fb7cc091a40 2023-10-03T10:09:41.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.800+0000 7fb7ca7fc700 1 -- 172.21.15.3:0/1166961897 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7bc00db90 con 0x7fb7cc091a40 2023-10-03T10:09:41.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.800+0000 7fb7ca7fc700 1 -- 172.21.15.3:0/1166961897 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc010470 con 0x7fb7cc091a40 2023-10-03T10:09:41.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.800+0000 7fb7d1bbd700 1 -- 172.21.15.3:0/1166961897 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7cc091a40 msgr2=0x7fb7cc091e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:41.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.800+0000 7fb7d1bbd700 1 --2- 172.21.15.3:0/1166961897 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7cc091a40 0x7fb7cc091e20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fb7bc00ba30 tx=0x7fb7bc00b300).stop 2023-10-03T10:09:41.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.800+0000 7fb7d1bbd700 1 -- 172.21.15.3:0/1166961897 shutdown_connections 2023-10-03T10:09:41.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.800+0000 7fb7d1bbd700 1 --2- 172.21.15.3:0/1166961897 >> v2:172.21.15.3:3300/0 conn(0x7fb7cc128090 0x7fb7cc12fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:41.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.800+0000 7fb7d1bbd700 1 --2- 172.21.15.3:0/1166961897 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7cc091a40 0x7fb7cc091e20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:41.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.800+0000 7fb7d1bbd700 1 -- 172.21.15.3:0/1166961897 >> 172.21.15.3:0/1166961897 conn(0x7fb7cc063560 msgr2=0x7fb7cc09a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:41.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.801+0000 7fb7d1bbd700 1 -- 172.21.15.3:0/1166961897 shutdown_connections 2023-10-03T10:09:41.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.801+0000 7fb7d1bbd700 1 -- 172.21.15.3:0/1166961897 wait complete. 2023-10-03T10:09:41.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.802+0000 7fb7d1bbd700 1 Processor -- start 2023-10-03T10:09:41.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.802+0000 7fb7d1bbd700 1 -- start start 2023-10-03T10:09:41.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.802+0000 7fb7d1bbd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7cc091a40 0x7fb7cc119200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:41.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.802+0000 7fb7d1bbd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7cc092310 0x7fb7cc115320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:41.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.802+0000 7fb7d1bbd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7cc128090 0x7fb7cc115810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:41.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.802+0000 7fb7d1bbd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc118d70 con 0x7fb7cc091a40 2023-10-03T10:09:41.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.802+0000 7fb7d1bbd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc127880 con 0x7fb7cc092310 2023-10-03T10:09:41.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.802+0000 7fb7d1bbd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc116720 con 0x7fb7cc128090 2023-10-03T10:09:41.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.802+0000 7fb7cb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7cc091a40 0x7fb7cc119200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:41.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.802+0000 7fb7cb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7cc091a40 0x7fb7cc119200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56864/0 (socket says 172.21.15.3:56864) 2023-10-03T10:09:41.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.802+0000 7fb7cb7fe700 1 -- 172.21.15.3:0/3470245445 learned_addr learned my addr 172.21.15.3:0/3470245445 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:41.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.802+0000 7fb7cbfff700 1 --2- 172.21.15.3:0/3470245445 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7cc128090 0x7fb7cc115810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:41.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.802+0000 7fb7caffd700 1 --2- 172.21.15.3:0/3470245445 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7cc092310 0x7fb7cc115320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:41.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.803+0000 7fb7cb7fe700 1 -- 172.21.15.3:0/3470245445 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7cc128090 msgr2=0x7fb7cc115810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:41.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.803+0000 7fb7cb7fe700 1 --2- 172.21.15.3:0/3470245445 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7cc128090 0x7fb7cc115810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:41.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.803+0000 7fb7cb7fe700 1 -- 172.21.15.3:0/3470245445 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7cc092310 msgr2=0x7fb7cc115320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:41.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.803+0000 7fb7cb7fe700 1 --2- 172.21.15.3:0/3470245445 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7cc092310 0x7fb7cc115320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:41.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.803+0000 7fb7cb7fe700 1 -- 172.21.15.3:0/3470245445 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c4009a20 con 0x7fb7cc091a40 2023-10-03T10:09:41.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.803+0000 7fb7cbfff700 1 --2- 172.21.15.3:0/3470245445 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7cc128090 0x7fb7cc115810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:41.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.803+0000 7fb7caffd700 1 --2- 172.21.15.3:0/3470245445 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7cc092310 0x7fb7cc115320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:41.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.803+0000 7fb7cb7fe700 1 --2- 172.21.15.3:0/3470245445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7cc091a40 0x7fb7cc119200 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fb7bc00b340 tx=0x7fb7cc094b70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:41.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.803+0000 7fb7b37fe700 1 -- 172.21.15.3:0/3470245445 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc020070 con 0x7fb7cc091a40 2023-10-03T10:09:41.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.803+0000 7fb7b37fe700 1 -- 172.21.15.3:0/3470245445 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7bc019450 con 0x7fb7cc091a40 2023-10-03T10:09:41.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.803+0000 7fb7b37fe700 1 -- 172.21.15.3:0/3470245445 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc0197f0 con 0x7fb7cc091a40 2023-10-03T10:09:41.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.803+0000 7fb7d1bbd700 1 -- 172.21.15.3:0/3470245445 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7bc00b540 con 0x7fb7cc091a40 2023-10-03T10:09:41.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.803+0000 7fb7d1bbd700 1 -- 172.21.15.3:0/3470245445 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7cc118760 con 0x7fb7cc091a40 2023-10-03T10:09:41.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.804+0000 7fb7b37fe700 1 -- 172.21.15.3:0/3470245445 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fb7bc019b60 con 0x7fb7cc091a40 2023-10-03T10:09:41.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.804+0000 7fb7b37fe700 1 -- 172.21.15.3:0/3470245445 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7fb7bc03e5a0 con 0x7fb7cc091a40 2023-10-03T10:09:41.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.804+0000 7fb7d1bbd700 1 -- 172.21.15.3:0/3470245445 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7cc093dd0 con 0x7fb7cc091a40 2023-10-03T10:09:41.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.809+0000 7fb7b37fe700 1 -- 172.21.15.3:0/3470245445 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fb7bc02a080 con 0x7fb7cc091a40 2023-10-03T10:09:41.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.983+0000 7fb7d1bbd700 1 -- 172.21.15.3:0/3470245445 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb7cc05a260 con 0x7fb7cc091a40 2023-10-03T10:09:41.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.984+0000 7fb7b37fe700 1 -- 172.21.15.3:0/3470245445 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v13) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb7bc00f3d0 con 0x7fb7cc091a40 2023-10-03T10:09:41.984 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-03T10:09:41.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.986+0000 7fb7d1bbd700 1 -- 172.21.15.3:0/3470245445 >> v1:172.21.15.3:6816/118296 conn(0x7fb7b802bb60 legacy=0x7fb7b802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:41.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.986+0000 7fb7d1bbd700 1 -- 172.21.15.3:0/3470245445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7cc091a40 msgr2=0x7fb7cc119200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:41.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.986+0000 7fb7d1bbd700 1 --2- 172.21.15.3:0/3470245445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7cc091a40 0x7fb7cc119200 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fb7bc00b340 tx=0x7fb7cc094b70).stop 2023-10-03T10:09:41.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.986+0000 7fb7d1bbd700 1 -- 172.21.15.3:0/3470245445 shutdown_connections 2023-10-03T10:09:41.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.986+0000 7fb7d1bbd700 1 --2- 172.21.15.3:0/3470245445 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7cc128090 0x7fb7cc115810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:41.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.986+0000 7fb7d1bbd700 1 --2- 172.21.15.3:0/3470245445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7cc091a40 0x7fb7cc119200 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:41.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.986+0000 7fb7d1bbd700 1 --2- 172.21.15.3:0/3470245445 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7cc092310 0x7fb7cc115320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:41.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.986+0000 7fb7d1bbd700 1 -- 172.21.15.3:0/3470245445 >> 172.21.15.3:0/3470245445 conn(0x7fb7cc063560 msgr2=0x7fb7cc068ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:41.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.986+0000 7fb7d1bbd700 1 -- 172.21.15.3:0/3470245445 shutdown_connections 2023-10-03T10:09:41.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:41.986+0000 7fb7d1bbd700 1 -- 172.21.15.3:0/3470245445 wait complete. 2023-10-03T10:09:41.999 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-03T10:09:42.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.129+0000 7f59921f4700 1 Processor -- start 2023-10-03T10:09:42.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.129+0000 7f59921f4700 1 -- start start 2023-10-03T10:09:42.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.130+0000 7f59921f4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f598c093810 0x7f598c095300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:42.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.130+0000 7f59921f4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f598c0957f0 0x7f598c095bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:42.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.130+0000 7f59921f4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f598c093010 con 0x7f598c12c110 2023-10-03T10:09:42.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.130+0000 7f59921f4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f598c093bf0 con 0x7f598c093810 2023-10-03T10:09:42.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.130+0000 7f59921f4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598c092aa0 con 0x7f598c0957f0 2023-10-03T10:09:42.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.130+0000 7f598affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f598c0957f0 0x7f598c095bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:42.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.130+0000 7f598affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f598c0957f0 0x7f598c095bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42194/0 (socket says 172.21.15.3:42194) 2023-10-03T10:09:42.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.130+0000 7f598b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f598c093810 0x7f598c095300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:42.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.130+0000 7f598affd700 1 -- 172.21.15.3:0/3320842937 learned_addr learned my addr 172.21.15.3:0/3320842937 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:42.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.130+0000 7f598affd700 1 -- 172.21.15.3:0/3320842937 >> v2:172.21.15.3:3300/0 conn(0x7f598c093810 msgr2=0x7f598c095300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:42.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.130+0000 7f598affd700 1 --2- 172.21.15.3:0/3320842937 >> v2:172.21.15.3:3300/0 conn(0x7f598c093810 0x7f598c095300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:42.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.130+0000 7f598affd700 1 -- 172.21.15.3:0/3320842937 >> v1:172.21.15.3:6789/0 conn(0x7f598c12c110 legacy=0x7f598c12e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:42.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.130+0000 7f598a7fc700 1 -- 172.21.15.3:0/3320842937 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3433525979 0 0) 0x7f598c093010 con 0x7f598c12c110 2023-10-03T10:09:42.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.130+0000 7f598affd700 1 -- 172.21.15.3:0/3320842937 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598c096e90 con 0x7f598c0957f0 2023-10-03T10:09:42.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.130+0000 7f598b7fe700 1 --2- 172.21.15.3:0/3320842937 >> v2:172.21.15.3:3300/0 conn(0x7f598c093810 0x7f598c095300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:42.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.130+0000 7f598affd700 1 --2- 172.21.15.3:0/3320842937 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f598c0957f0 0x7f598c095bd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f5980009830 tx=0x7f59800090f0).ready entity=mon.1 client_cookie=ab48f0c7564e6d0e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:42.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.131+0000 7f598a7fc700 1 -- 172.21.15.3:0/3320842937 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598000b580 con 0x7f598c0957f0 2023-10-03T10:09:42.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.131+0000 7f598a7fc700 1 -- 172.21.15.3:0/3320842937 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598000b6e0 con 0x7f598c0957f0 2023-10-03T10:09:42.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.131+0000 7f598a7fc700 1 -- 172.21.15.3:0/3320842937 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598000ba60 con 0x7f598c0957f0 2023-10-03T10:09:42.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.131+0000 7f59921f4700 1 -- 172.21.15.3:0/3320842937 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f598c0957f0 msgr2=0x7f598c095bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:42.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.131+0000 7f59921f4700 1 --2- 172.21.15.3:0/3320842937 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f598c0957f0 0x7f598c095bd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f5980009830 tx=0x7f59800090f0).stop 2023-10-03T10:09:42.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.131+0000 7f59921f4700 1 -- 172.21.15.3:0/3320842937 shutdown_connections 2023-10-03T10:09:42.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.131+0000 7f59921f4700 1 --2- 172.21.15.3:0/3320842937 >> v2:172.21.15.3:3300/0 conn(0x7f598c093810 0x7f598c095300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:42.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.131+0000 7f59921f4700 1 --2- 172.21.15.3:0/3320842937 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f598c0957f0 0x7f598c095bd0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:42.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.131+0000 7f59921f4700 1 -- 172.21.15.3:0/3320842937 >> 172.21.15.3:0/3320842937 conn(0x7f598c059bb0 msgr2=0x7f598c097400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:42.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.131+0000 7f59921f4700 1 -- 172.21.15.3:0/3320842937 shutdown_connections 2023-10-03T10:09:42.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.131+0000 7f59921f4700 1 -- 172.21.15.3:0/3320842937 wait complete. 2023-10-03T10:09:42.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.133+0000 7f59921f4700 1 Processor -- start 2023-10-03T10:09:42.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.133+0000 7f59921f4700 1 -- start start 2023-10-03T10:09:42.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.133+0000 7f59921f4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f598c093810 0x7f598c12df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:42.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.133+0000 7f59921f4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f598c0957f0 0x7f598c12e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:42.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.133+0000 7f59921f4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f598c12c110 0x7f598c138110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:42.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.134+0000 7f59921f4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598c06a6a0 con 0x7f598c0957f0 2023-10-03T10:09:42.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.134+0000 7f59921f4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598c126c40 con 0x7f598c093810 2023-10-03T10:09:42.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.134+0000 7f59921f4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f598c126f00 con 0x7f598c12c110 2023-10-03T10:09:42.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.134+0000 7f598b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f598c093810 0x7f598c12df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:42.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.134+0000 7f598b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f598c093810 0x7f598c12df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42202/0 (socket says 172.21.15.3:42202) 2023-10-03T10:09:42.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.134+0000 7f598b7fe700 1 -- 172.21.15.3:0/3641775048 learned_addr learned my addr 172.21.15.3:0/3641775048 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:42.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.134+0000 7f598affd700 1 --2- 172.21.15.3:0/3641775048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f598c0957f0 0x7f598c12e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:42.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.134+0000 7f598bfff700 1 --2- 172.21.15.3:0/3641775048 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f598c12c110 0x7f598c138110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:42.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.134+0000 7f598b7fe700 1 -- 172.21.15.3:0/3641775048 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f598c12c110 msgr2=0x7f598c138110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:42.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.134+0000 7f598b7fe700 1 --2- 172.21.15.3:0/3641775048 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f598c12c110 0x7f598c138110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:42.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.134+0000 7f598b7fe700 1 -- 172.21.15.3:0/3641775048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f598c0957f0 msgr2=0x7f598c12e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:42.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.134+0000 7f598b7fe700 1 --2- 172.21.15.3:0/3641775048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f598c0957f0 0x7f598c12e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:42.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.134+0000 7f598b7fe700 1 -- 172.21.15.3:0/3641775048 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597c0092a0 con 0x7f598c093810 2023-10-03T10:09:42.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.134+0000 7f598affd700 1 --2- 172.21.15.3:0/3641775048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f598c0957f0 0x7f598c12e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:42.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.134+0000 7f598b7fe700 1 --2- 172.21.15.3:0/3641775048 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f598c093810 0x7f598c12df00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f597c00ea20 tx=0x7f597c009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:42.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.134+0000 7f59737fe700 1 -- 172.21.15.3:0/3641775048 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597c00ee60 con 0x7f598c093810 2023-10-03T10:09:42.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.134+0000 7f59737fe700 1 -- 172.21.15.3:0/3641775048 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f597c009600 con 0x7f598c093810 2023-10-03T10:09:42.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.134+0000 7f59921f4700 1 -- 172.21.15.3:0/3641775048 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5980009370 con 0x7f598c093810 2023-10-03T10:09:42.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.134+0000 7f59737fe700 1 -- 172.21.15.3:0/3641775048 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597c00ca60 con 0x7f598c093810 2023-10-03T10:09:42.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.134+0000 7f59921f4700 1 -- 172.21.15.3:0/3641775048 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f598c12dab0 con 0x7f598c093810 2023-10-03T10:09:42.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.135+0000 7f59737fe700 1 -- 172.21.15.3:0/3641775048 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f597c01b400 con 0x7f598c093810 2023-10-03T10:09:42.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.135+0000 7f59737fe700 1 -- 172.21.15.3:0/3641775048 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f597c03b130 con 0x7f598c093810 2023-10-03T10:09:42.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.135+0000 7f59921f4700 1 -- 172.21.15.3:0/3641775048 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f598c095cf0 con 0x7f598c093810 2023-10-03T10:09:42.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.140+0000 7f59737fe700 1 -- 172.21.15.3:0/3641775048 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f597c019310 con 0x7f598c093810 2023-10-03T10:09:42.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.314+0000 7f59921f4700 1 -- 172.21.15.3:0/3641775048 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f598c05a260 con 0x7f598c093810 2023-10-03T10:09:42.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.314+0000 7f59737fe700 1 -- 172.21.15.3:0/3641775048 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v13) v1 ==== 84+0+143 (secure 0 0 0) 0x7f597c029090 con 0x7f598c093810 2023-10-03T10:09:42.315 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:42.315 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2023-10-03T10:09:42.315 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:09:42.315 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2023-10-03T10:09:42.315 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": "device_health_metrics" 2023-10-03T10:09:42.315 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:09:42.315 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:09:42.315 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 2, 2023-10-03T10:09:42.315 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": "rbd" 2023-10-03T10:09:42.316 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:09:42.316 INFO:teuthology.orchestra.run.smithi003.stdout:] 2023-10-03T10:09:42.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.316+0000 7f59921f4700 1 -- 172.21.15.3:0/3641775048 >> v1:172.21.15.3:6816/118296 conn(0x7f597802bb60 legacy=0x7f597802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:42.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.316+0000 7f59921f4700 1 -- 172.21.15.3:0/3641775048 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f598c093810 msgr2=0x7f598c12df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:42.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.316+0000 7f59921f4700 1 --2- 172.21.15.3:0/3641775048 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f598c093810 0x7f598c12df00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f597c00ea20 tx=0x7f597c009060).stop 2023-10-03T10:09:42.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.316+0000 7f59921f4700 1 -- 172.21.15.3:0/3641775048 shutdown_connections 2023-10-03T10:09:42.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.316+0000 7f59921f4700 1 --2- 172.21.15.3:0/3641775048 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f598c12c110 0x7f598c138110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:42.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.316+0000 7f59921f4700 1 --2- 172.21.15.3:0/3641775048 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f598c093810 0x7f598c12df00 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:42.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.316+0000 7f59921f4700 1 --2- 172.21.15.3:0/3641775048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f598c0957f0 0x7f598c12e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:42.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.316+0000 7f59921f4700 1 -- 172.21.15.3:0/3641775048 >> 172.21.15.3:0/3641775048 conn(0x7f598c059bb0 msgr2=0x7f598c09a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:42.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.316+0000 7f59921f4700 1 -- 172.21.15.3:0/3641775048 shutdown_connections 2023-10-03T10:09:42.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.316+0000 7f59921f4700 1 -- 172.21.15.3:0/3641775048 wait complete. 2023-10-03T10:09:42.330 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-03T10:09:42.330 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-03T10:09:42.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.463+0000 7f31dd6c6700 1 Processor -- start 2023-10-03T10:09:42.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.464+0000 7f31dd6c6700 1 -- start start 2023-10-03T10:09:42.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.464+0000 7f31dd6c6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f31d80662b0 0x7f31d8097340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:42.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.464+0000 7f31dd6c6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31d812b280 0x7f31d81274f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:42.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.464+0000 7f31dd6c6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31d80657f0 con 0x7f31d812b680 2023-10-03T10:09:42.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.464+0000 7f31dd6c6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d8065ab0 con 0x7f31d80662b0 2023-10-03T10:09:42.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.464+0000 7f31dd6c6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d8064840 con 0x7f31d812b280 2023-10-03T10:09:42.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.464+0000 7f31d67fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31d812b280 0x7f31d81274f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:42.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.464+0000 7f31d67fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31d812b280 0x7f31d81274f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42216/0 (socket says 172.21.15.3:42216) 2023-10-03T10:09:42.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.464+0000 7f31d67fc700 1 -- 172.21.15.3:0/3659148092 learned_addr learned my addr 172.21.15.3:0/3659148092 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:42.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.464+0000 7f31d6ffd700 1 --2- 172.21.15.3:0/3659148092 >> v2:172.21.15.3:3300/0 conn(0x7f31d80662b0 0x7f31d8097340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:42.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.465+0000 7f31d67fc700 1 -- 172.21.15.3:0/3659148092 >> v2:172.21.15.3:3300/0 conn(0x7f31d80662b0 msgr2=0x7f31d8097340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:42.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.465+0000 7f31d67fc700 1 --2- 172.21.15.3:0/3659148092 >> v2:172.21.15.3:3300/0 conn(0x7f31d80662b0 0x7f31d8097340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:42.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.465+0000 7f31d67fc700 1 -- 172.21.15.3:0/3659148092 >> v1:172.21.15.3:6789/0 conn(0x7f31d812b680 legacy=0x7f31d81279e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:42.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.465+0000 7f31d67fc700 1 -- 172.21.15.3:0/3659148092 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d8064d70 con 0x7f31d812b280 2023-10-03T10:09:42.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.465+0000 7f31d67fc700 1 --2- 172.21.15.3:0/3659148092 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31d812b280 0x7f31d81274f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f31cc009830 tx=0x7f31cc0090f0).ready entity=mon.1 client_cookie=a66e29fafb564b76 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:42.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.465+0000 7f31d5ffb700 1 -- 172.21.15.3:0/3659148092 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31cc00b580 con 0x7f31d812b280 2023-10-03T10:09:42.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.465+0000 7f31d5ffb700 1 -- 172.21.15.3:0/3659148092 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31cc00b6e0 con 0x7f31d812b280 2023-10-03T10:09:42.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.465+0000 7f31d5ffb700 1 -- 172.21.15.3:0/3659148092 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31cc00ba80 con 0x7f31d812b280 2023-10-03T10:09:42.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.465+0000 7f31dd6c6700 1 -- 172.21.15.3:0/3659148092 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31d812b280 msgr2=0x7f31d81274f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:42.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.465+0000 7f31dd6c6700 1 --2- 172.21.15.3:0/3659148092 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31d812b280 0x7f31d81274f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f31cc009830 tx=0x7f31cc0090f0).stop 2023-10-03T10:09:42.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.466+0000 7f31dd6c6700 1 -- 172.21.15.3:0/3659148092 shutdown_connections 2023-10-03T10:09:42.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.466+0000 7f31dd6c6700 1 --2- 172.21.15.3:0/3659148092 >> v2:172.21.15.3:3300/0 conn(0x7f31d80662b0 0x7f31d8097340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:42.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.466+0000 7f31dd6c6700 1 --2- 172.21.15.3:0/3659148092 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31d812b280 0x7f31d81274f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:42.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.466+0000 7f31dd6c6700 1 -- 172.21.15.3:0/3659148092 >> 172.21.15.3:0/3659148092 conn(0x7f31d8098520 msgr2=0x7f31d8098930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:42.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.466+0000 7f31dd6c6700 1 -- 172.21.15.3:0/3659148092 shutdown_connections 2023-10-03T10:09:42.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.466+0000 7f31dd6c6700 1 -- 172.21.15.3:0/3659148092 wait complete. 2023-10-03T10:09:42.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.467+0000 7f31dd6c6700 1 Processor -- start 2023-10-03T10:09:42.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.467+0000 7f31dd6c6700 1 -- start start 2023-10-03T10:09:42.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.467+0000 7f31dd6c6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31d80662b0 0x7f31d8117ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:42.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.467+0000 7f31dd6c6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31d812b280 0x7f31d8113b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:42.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.467+0000 7f31dd6c6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31d812b680 0x7f31d8114020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:42.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.467+0000 7f31dd6c6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d8124c00 con 0x7f31d812b680 2023-10-03T10:09:42.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.467+0000 7f31dd6c6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d8065030 con 0x7f31d80662b0 2023-10-03T10:09:42.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.467+0000 7f31dd6c6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31d8117030 con 0x7f31d812b280 2023-10-03T10:09:42.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.467+0000 7f31d6ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31d80662b0 0x7f31d8117ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:42.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.467+0000 7f31d77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31d812b680 0x7f31d8114020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:42.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.467+0000 7f31d6ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31d80662b0 0x7f31d8117ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42226/0 (socket says 172.21.15.3:42226) 2023-10-03T10:09:42.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.467+0000 7f31d6ffd700 1 -- 172.21.15.3:0/1987100830 learned_addr learned my addr 172.21.15.3:0/1987100830 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:42.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.467+0000 7f31d67fc700 1 --2- 172.21.15.3:0/1987100830 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31d812b280 0x7f31d8113b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:42.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.468+0000 7f31d6ffd700 1 -- 172.21.15.3:0/1987100830 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31d812b280 msgr2=0x7f31d8113b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:42.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.468+0000 7f31d6ffd700 1 --2- 172.21.15.3:0/1987100830 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31d812b280 0x7f31d8113b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:42.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.468+0000 7f31d6ffd700 1 -- 172.21.15.3:0/1987100830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31d812b680 msgr2=0x7f31d8114020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:42.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.468+0000 7f31d6ffd700 1 --2- 172.21.15.3:0/1987100830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31d812b680 0x7f31d8114020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:42.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.468+0000 7f31d6ffd700 1 -- 172.21.15.3:0/1987100830 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31cc009370 con 0x7f31d80662b0 2023-10-03T10:09:42.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.468+0000 7f31d77fe700 1 --2- 172.21.15.3:0/1987100830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31d812b680 0x7f31d8114020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:42.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.468+0000 7f31d6ffd700 1 --2- 172.21.15.3:0/1987100830 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31d80662b0 0x7f31d8117ad0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f31c800e960 tx=0x7f31c8009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:42.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.468+0000 7f31beffd700 1 -- 172.21.15.3:0/1987100830 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8012600 con 0x7f31d80662b0 2023-10-03T10:09:42.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.468+0000 7f31beffd700 1 -- 172.21.15.3:0/1987100830 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c8012760 con 0x7f31d80662b0 2023-10-03T10:09:42.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.468+0000 7f31dd6c6700 1 -- 172.21.15.3:0/1987100830 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d81175d0 con 0x7f31d80662b0 2023-10-03T10:09:42.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.468+0000 7f31beffd700 1 -- 172.21.15.3:0/1987100830 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8012ae0 con 0x7f31d80662b0 2023-10-03T10:09:42.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.468+0000 7f31dd6c6700 1 -- 172.21.15.3:0/1987100830 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d8114f30 con 0x7f31d80662b0 2023-10-03T10:09:42.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.469+0000 7f31beffd700 1 -- 172.21.15.3:0/1987100830 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f31c801b400 con 0x7f31d80662b0 2023-10-03T10:09:42.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.469+0000 7f31beffd700 1 -- 172.21.15.3:0/1987100830 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f31c803b1a0 con 0x7f31d80662b0 2023-10-03T10:09:42.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.469+0000 7f31dd6c6700 1 -- 172.21.15.3:0/1987100830 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d805a260 con 0x7f31d80662b0 2023-10-03T10:09:42.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.474+0000 7f31beffd700 1 -- 172.21.15.3:0/1987100830 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f31c8019900 con 0x7f31d80662b0 2023-10-03T10:09:42.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.648+0000 7f31dd6c6700 1 -- 172.21.15.3:0/1987100830 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f31d812aef0 con 0x7f31d80662b0 2023-10-03T10:09:42.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.723+0000 7f31beffd700 1 -- 172.21.15.3:0/1987100830 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v14) v1 ==== 138+0+0 (secure 0 0 0) 0x7f31c8023350 con 0x7f31d80662b0 2023-10-03T10:09:42.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.726+0000 7f31dd6c6700 1 -- 172.21.15.3:0/1987100830 >> v1:172.21.15.3:6816/118296 conn(0x7f31c402bb60 legacy=0x7f31c402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:42.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.726+0000 7f31dd6c6700 1 -- 172.21.15.3:0/1987100830 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31d80662b0 msgr2=0x7f31d8117ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:42.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.726+0000 7f31dd6c6700 1 --2- 172.21.15.3:0/1987100830 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31d80662b0 0x7f31d8117ad0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f31c800e960 tx=0x7f31c8009060).stop 2023-10-03T10:09:42.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.726+0000 7f31dd6c6700 1 -- 172.21.15.3:0/1987100830 shutdown_connections 2023-10-03T10:09:42.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.726+0000 7f31dd6c6700 1 --2- 172.21.15.3:0/1987100830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31d812b680 0x7f31d8114020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:42.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.726+0000 7f31dd6c6700 1 --2- 172.21.15.3:0/1987100830 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31d80662b0 0x7f31d8117ad0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:42.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.726+0000 7f31dd6c6700 1 --2- 172.21.15.3:0/1987100830 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31d812b280 0x7f31d8113b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:42.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.726+0000 7f31dd6c6700 1 -- 172.21.15.3:0/1987100830 >> 172.21.15.3:0/1987100830 conn(0x7f31d8098520 msgr2=0x7f31d809bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:42.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.726+0000 7f31dd6c6700 1 -- 172.21.15.3:0/1987100830 shutdown_connections 2023-10-03T10:09:42.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.726+0000 7f31dd6c6700 1 -- 172.21.15.3:0/1987100830 wait complete. 2023-10-03T10:09:42.728 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2023-10-03T10:09:42.744 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-03T10:09:42.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.881+0000 7f848f52f700 1 Processor -- start 2023-10-03T10:09:42.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.881+0000 7f848f52f700 1 -- start start 2023-10-03T10:09:42.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.882+0000 7f848f52f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8488068ef0 0x7f84880692d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:42.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.882+0000 7f848f52f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f84880697c0 0x7f848812f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:42.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.882+0000 7f848f52f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8488091d30 con 0x7f848812f670 2023-10-03T10:09:42.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.882+0000 7f848f52f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8488093fc0 con 0x7f84880697c0 2023-10-03T10:09:42.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.882+0000 7f848f52f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f84880942a0 con 0x7f8488068ef0 2023-10-03T10:09:42.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.882+0000 7f848d2cb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8488068ef0 0x7f84880692d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:42.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.882+0000 7f848caca700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f84880697c0 0x7f848812f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:42.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.882+0000 7f848dacc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f848812f670 0x7f84881363d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58194/0 (socket says 172.21.15.3:58194) 2023-10-03T10:09:42.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.882+0000 7f848dacc700 1 -- 172.21.15.3:0/1905472094 learned_addr learned my addr 172.21.15.3:0/1905472094 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:09:42.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.882+0000 7f848d2cb700 1 -- 172.21.15.3:0/1905472094 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f84880697c0 msgr2=0x7f848812f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:42.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.882+0000 7f848d2cb700 1 --2- 172.21.15.3:0/1905472094 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f84880697c0 0x7f848812f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:42.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.882+0000 7f848d2cb700 1 -- 172.21.15.3:0/1905472094 >> v1:172.21.15.3:6789/0 conn(0x7f848812f670 legacy=0x7f84881363d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:42.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.882+0000 7f847ffff700 1 -- 172.21.15.3:0/1905472094 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084477243 0 0) 0x7f8488091d30 con 0x7f848812f670 2023-10-03T10:09:42.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.882+0000 7f848d2cb700 1 -- 172.21.15.3:0/1905472094 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8488092830 con 0x7f8488068ef0 2023-10-03T10:09:42.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.882+0000 7f848caca700 1 --2- 172.21.15.3:0/1905472094 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f84880697c0 0x7f848812f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:42.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.883+0000 7f848d2cb700 1 --2- 172.21.15.3:0/1905472094 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8488068ef0 0x7f84880692d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f84840097c0 tx=0x7f8484009060).ready entity=mon.2 client_cookie=f39f2c4a5fb0b911 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:42.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.883+0000 7f847ffff700 1 -- 172.21.15.3:0/1905472094 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84840090d0 con 0x7f8488068ef0 2023-10-03T10:09:42.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.883+0000 7f847ffff700 1 -- 172.21.15.3:0/1905472094 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848400f510 con 0x7f8488068ef0 2023-10-03T10:09:42.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.883+0000 7f847ffff700 1 -- 172.21.15.3:0/1905472094 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848400f890 con 0x7f8488068ef0 2023-10-03T10:09:42.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.883+0000 7f848f52f700 1 -- 172.21.15.3:0/1905472094 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8488068ef0 msgr2=0x7f84880692d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:42.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.883+0000 7f848f52f700 1 --2- 172.21.15.3:0/1905472094 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8488068ef0 0x7f84880692d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f84840097c0 tx=0x7f8484009060).stop 2023-10-03T10:09:42.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.883+0000 7f848f52f700 1 -- 172.21.15.3:0/1905472094 shutdown_connections 2023-10-03T10:09:42.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.883+0000 7f848f52f700 1 --2- 172.21.15.3:0/1905472094 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8488068ef0 0x7f84880692d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:42.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.883+0000 7f848f52f700 1 --2- 172.21.15.3:0/1905472094 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f84880697c0 0x7f848812f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:42.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.883+0000 7f848f52f700 1 -- 172.21.15.3:0/1905472094 >> 172.21.15.3:0/1905472094 conn(0x7f8488063560 msgr2=0x7f848809a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:42.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.884+0000 7f848f52f700 1 -- 172.21.15.3:0/1905472094 shutdown_connections 2023-10-03T10:09:42.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.884+0000 7f848f52f700 1 -- 172.21.15.3:0/1905472094 wait complete. 2023-10-03T10:09:42.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.884+0000 7f848f52f700 1 Processor -- start 2023-10-03T10:09:42.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.884+0000 7f848f52f700 1 -- start start 2023-10-03T10:09:42.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.885+0000 7f848f52f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8488068ef0 0x7f848811e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:42.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.885+0000 7f848f52f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84880697c0 0x7f848813e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:42.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.885+0000 7f848f52f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f848812f670 0x7f848813ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:42.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.885+0000 7f848f52f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8488092a60 con 0x7f84880697c0 2023-10-03T10:09:42.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.885+0000 7f848f52f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84881336f0 con 0x7f848812f670 2023-10-03T10:09:42.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.885+0000 7f848f52f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f848812f220 con 0x7f8488068ef0 2023-10-03T10:09:42.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.885+0000 7f848d2cb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8488068ef0 0x7f848811e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:42.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.885+0000 7f848caca700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84880697c0 0x7f848813e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:42.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.885+0000 7f848d2cb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8488068ef0 0x7f848811e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38380/0 (socket says 172.21.15.3:38380) 2023-10-03T10:09:42.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.885+0000 7f848dacc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f848812f670 0x7f848813ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:42.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.885+0000 7f848d2cb700 1 -- 172.21.15.3:0/19980180 learned_addr learned my addr 172.21.15.3:0/19980180 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:42.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.886+0000 7f848caca700 1 -- 172.21.15.3:0/19980180 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8488068ef0 msgr2=0x7f848811e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:42.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.886+0000 7f848caca700 1 --2- 172.21.15.3:0/19980180 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8488068ef0 0x7f848811e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:42.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.886+0000 7f848caca700 1 -- 172.21.15.3:0/19980180 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f848812f670 msgr2=0x7f848813ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:42.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.886+0000 7f848caca700 1 --2- 172.21.15.3:0/19980180 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f848812f670 0x7f848813ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:42.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.886+0000 7f848caca700 1 -- 172.21.15.3:0/19980180 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84780092a0 con 0x7f84880697c0 2023-10-03T10:09:42.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.886+0000 7f848d2cb700 1 --2- 172.21.15.3:0/19980180 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8488068ef0 0x7f848811e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:42.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.886+0000 7f848dacc700 1 --2- 172.21.15.3:0/19980180 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f848812f670 0x7f848813ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:42.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.886+0000 7f848caca700 1 --2- 172.21.15.3:0/19980180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84880697c0 0x7f848813e960 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f847800e950 tx=0x7f8478009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:42.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.886+0000 7f847d7fa700 1 -- 172.21.15.3:0/19980180 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478009580 con 0x7f84880697c0 2023-10-03T10:09:42.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.886+0000 7f847d7fa700 1 -- 172.21.15.3:0/19980180 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8478012600 con 0x7f84880697c0 2023-10-03T10:09:42.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.886+0000 7f847d7fa700 1 -- 172.21.15.3:0/19980180 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84780129a0 con 0x7f84880697c0 2023-10-03T10:09:42.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.886+0000 7f848f52f700 1 -- 172.21.15.3:0/19980180 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84840092a0 con 0x7f84880697c0 2023-10-03T10:09:42.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.886+0000 7f848f52f700 1 -- 172.21.15.3:0/19980180 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f848813f340 con 0x7f84880697c0 2023-10-03T10:09:42.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.887+0000 7f847d7fa700 1 -- 172.21.15.3:0/19980180 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f8478012d10 con 0x7f84880697c0 2023-10-03T10:09:42.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.887+0000 7f847d7fa700 1 -- 172.21.15.3:0/19980180 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3208+0+0 (secure 0 0 0) 0x7f847803b2a0 con 0x7f84880697c0 2023-10-03T10:09:42.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.887+0000 7f848f52f700 1 -- 172.21.15.3:0/19980180 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f848811dc10 con 0x7f84880697c0 2023-10-03T10:09:42.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:42.892+0000 7f847d7fa700 1 -- 172.21.15.3:0/19980180 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f8478011b40 con 0x7f84880697c0 2023-10-03T10:09:43.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.068+0000 7f848f52f700 1 -- 172.21.15.3:0/19980180 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f848811d7b0 con 0x7f84880697c0 2023-10-03T10:09:43.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.731+0000 7f847d7fa700 1 -- 172.21.15.3:0/19980180 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v15) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8478029030 con 0x7f84880697c0 2023-10-03T10:09:43.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.742+0000 7f848f52f700 1 -- 172.21.15.3:0/19980180 >> v1:172.21.15.3:6816/118296 conn(0x7f846c02bb60 legacy=0x7f846c02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:43.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.742+0000 7f848f52f700 1 -- 172.21.15.3:0/19980180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84880697c0 msgr2=0x7f848813e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:43.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.742+0000 7f848f52f700 1 --2- 172.21.15.3:0/19980180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84880697c0 0x7f848813e960 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f847800e950 tx=0x7f8478009060).stop 2023-10-03T10:09:43.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.758+0000 7f848f52f700 1 -- 172.21.15.3:0/19980180 shutdown_connections 2023-10-03T10:09:43.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.758+0000 7f848f52f700 1 --2- 172.21.15.3:0/19980180 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f848812f670 0x7f848813ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:43.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.758+0000 7f848f52f700 1 --2- 172.21.15.3:0/19980180 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8488068ef0 0x7f848811e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:43.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.758+0000 7f848f52f700 1 --2- 172.21.15.3:0/19980180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84880697c0 0x7f848813e960 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:43.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.758+0000 7f848f52f700 1 -- 172.21.15.3:0/19980180 >> 172.21.15.3:0/19980180 conn(0x7f8488063560 msgr2=0x7f84881321a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:43.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.759+0000 7f848f52f700 1 -- 172.21.15.3:0/19980180 shutdown_connections 2023-10-03T10:09:43.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.759+0000 7f848f52f700 1 -- 172.21.15.3:0/19980180 wait complete. 2023-10-03T10:09:43.761 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2023-10-03T10:09:43.775 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-03T10:09:43.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.910+0000 7f946bb15700 1 Processor -- start 2023-10-03T10:09:43.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.910+0000 7f946bb15700 1 -- start start 2023-10-03T10:09:43.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.911+0000 7f946bb15700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9464123940 0x7f946412c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:43.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.911+0000 7f946bb15700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9464124550 0x7f946412e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:43.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.911+0000 7f946bb15700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9464093630 con 0x7f9464068ef0 2023-10-03T10:09:43.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.911+0000 7f946bb15700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9464093b80 con 0x7f9464123940 2023-10-03T10:09:43.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.911+0000 7f946bb15700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94640938e0 con 0x7f9464124550 2023-10-03T10:09:43.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.911+0000 7f946a0b2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9464124550 0x7f946412e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:43.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.911+0000 7f94690b0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9464123940 0x7f946412c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:43.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.911+0000 7f946a0b2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9464124550 0x7f946412e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34468/0 (socket says 172.21.15.3:34468) 2023-10-03T10:09:43.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.911+0000 7f94690b0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9464123940 0x7f946412c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35362/0 (socket says 172.21.15.3:35362) 2023-10-03T10:09:43.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.911+0000 7f946a0b2700 1 -- 172.21.15.3:0/1864109867 learned_addr learned my addr 172.21.15.3:0/1864109867 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:43.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.911+0000 7f94698b1700 1 --1- 172.21.15.3:0/1864109867 >> v1:172.21.15.3:6789/0 conn(0x7f9464068ef0 0x7f9464123e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35006/0 (socket says 172.21.15.3:35006) 2023-10-03T10:09:43.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.911+0000 7f946a0b2700 1 -- 172.21.15.3:0/1864109867 >> v2:172.21.15.3:3300/0 conn(0x7f9464123940 msgr2=0x7f946412c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:43.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.911+0000 7f946a0b2700 1 --2- 172.21.15.3:0/1864109867 >> v2:172.21.15.3:3300/0 conn(0x7f9464123940 0x7f946412c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:43.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.911+0000 7f946a0b2700 1 -- 172.21.15.3:0/1864109867 >> v1:172.21.15.3:6789/0 conn(0x7f9464068ef0 legacy=0x7f9464123e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:43.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.911+0000 7f94688af700 1 -- 172.21.15.3:0/1864109867 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4016768582 0 0) 0x7f9464093630 con 0x7f9464068ef0 2023-10-03T10:09:43.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.911+0000 7f946a0b2700 1 -- 172.21.15.3:0/1864109867 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9464094770 con 0x7f9464124550 2023-10-03T10:09:43.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.911+0000 7f94690b0700 1 --2- 172.21.15.3:0/1864109867 >> v2:172.21.15.3:3300/0 conn(0x7f9464123940 0x7f946412c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:43.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.911+0000 7f946a0b2700 1 --2- 172.21.15.3:0/1864109867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9464124550 0x7f946412e6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f945c00d1a0 tx=0x7f945c00ba50).ready entity=mon.1 client_cookie=8fc43a517892ef12 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:43.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.912+0000 7f94688af700 1 -- 172.21.15.3:0/1864109867 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c010070 con 0x7f9464124550 2023-10-03T10:09:43.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.912+0000 7f94688af700 1 -- 172.21.15.3:0/1864109867 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945c00de10 con 0x7f9464124550 2023-10-03T10:09:43.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.912+0000 7f94688af700 1 -- 172.21.15.3:0/1864109867 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c00b2e0 con 0x7f9464124550 2023-10-03T10:09:43.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.912+0000 7f946bb15700 1 -- 172.21.15.3:0/1864109867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9464124550 msgr2=0x7f946412e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:43.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.912+0000 7f946bb15700 1 --2- 172.21.15.3:0/1864109867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9464124550 0x7f946412e6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f945c00d1a0 tx=0x7f945c00ba50).stop 2023-10-03T10:09:43.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.912+0000 7f946bb15700 1 -- 172.21.15.3:0/1864109867 shutdown_connections 2023-10-03T10:09:43.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.912+0000 7f946bb15700 1 --2- 172.21.15.3:0/1864109867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9464124550 0x7f946412e6c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:43.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.912+0000 7f946bb15700 1 --2- 172.21.15.3:0/1864109867 >> v2:172.21.15.3:3300/0 conn(0x7f9464123940 0x7f946412c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:43.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.912+0000 7f946bb15700 1 -- 172.21.15.3:0/1864109867 >> 172.21.15.3:0/1864109867 conn(0x7f9464063560 msgr2=0x7f946409a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:43.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.912+0000 7f946bb15700 1 -- 172.21.15.3:0/1864109867 shutdown_connections 2023-10-03T10:09:43.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.912+0000 7f946bb15700 1 -- 172.21.15.3:0/1864109867 wait complete. 2023-10-03T10:09:43.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.913+0000 7f946bb15700 1 Processor -- start 2023-10-03T10:09:43.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.913+0000 7f946bb15700 1 -- start start 2023-10-03T10:09:43.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.914+0000 7f946bb15700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9464068ef0 0x7f94641155f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:43.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.914+0000 7f946bb15700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9464123940 0x7f94641116c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:43.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.914+0000 7f946bb15700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9464124550 0x7f9464111bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:43.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.914+0000 7f946bb15700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9464068a90 con 0x7f9464124550 2023-10-03T10:09:43.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.914+0000 7f946bb15700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9464092e80 con 0x7f9464068ef0 2023-10-03T10:09:43.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.914+0000 7f946bb15700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94641150f0 con 0x7f9464123940 2023-10-03T10:09:43.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.914+0000 7f94698b1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9464068ef0 0x7f94641155f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:43.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.914+0000 7f946a0b2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9464124550 0x7f9464111bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:43.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.914+0000 7f94698b1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9464068ef0 0x7f94641155f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34472/0 (socket says 172.21.15.3:34472) 2023-10-03T10:09:43.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.914+0000 7f94690b0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9464123940 0x7f94641116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:43.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.914+0000 7f94698b1700 1 -- 172.21.15.3:0/2470175053 learned_addr learned my addr 172.21.15.3:0/2470175053 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:43.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.914+0000 7f94698b1700 1 -- 172.21.15.3:0/2470175053 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9464123940 msgr2=0x7f94641116c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:43.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.914+0000 7f94698b1700 1 --2- 172.21.15.3:0/2470175053 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9464123940 0x7f94641116c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:43.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.914+0000 7f94698b1700 1 -- 172.21.15.3:0/2470175053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9464124550 msgr2=0x7f9464111bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:43.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.914+0000 7f94698b1700 1 --2- 172.21.15.3:0/2470175053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9464124550 0x7f9464111bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:43.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.914+0000 7f94698b1700 1 -- 172.21.15.3:0/2470175053 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94540092a0 con 0x7f9464068ef0 2023-10-03T10:09:43.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.914+0000 7f946a0b2700 1 --2- 172.21.15.3:0/2470175053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9464124550 0x7f9464111bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:43.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.914+0000 7f94698b1700 1 --2- 172.21.15.3:0/2470175053 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9464068ef0 0x7f94641155f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f946000c4c0 tx=0x7f9460007c10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:43.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.915+0000 7f9459ffb700 1 -- 172.21.15.3:0/2470175053 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946000ce30 con 0x7f9464068ef0 2023-10-03T10:09:43.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.915+0000 7f9459ffb700 1 -- 172.21.15.3:0/2470175053 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94600072e0 con 0x7f9464068ef0 2023-10-03T10:09:43.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.915+0000 7f9459ffb700 1 -- 172.21.15.3:0/2470175053 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94600056e0 con 0x7f9464068ef0 2023-10-03T10:09:43.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.915+0000 7f946bb15700 1 -- 172.21.15.3:0/2470175053 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f945c00bcf0 con 0x7f9464068ef0 2023-10-03T10:09:43.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.915+0000 7f946bb15700 1 -- 172.21.15.3:0/2470175053 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94641120a0 con 0x7f9464068ef0 2023-10-03T10:09:43.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.915+0000 7f9459ffb700 1 -- 172.21.15.3:0/2470175053 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f9460005a90 con 0x7f9464068ef0 2023-10-03T10:09:43.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.916+0000 7f9459ffb700 1 -- 172.21.15.3:0/2470175053 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3590+0+0 (secure 0 0 0) 0x7f946003a580 con 0x7f9464068ef0 2023-10-03T10:09:43.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.916+0000 7f946bb15700 1 -- 172.21.15.3:0/2470175053 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9464114f30 con 0x7f9464068ef0 2023-10-03T10:09:43.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:43.921+0000 7f9459ffb700 1 -- 172.21.15.3:0/2470175053 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f9460013450 con 0x7f9464068ef0 2023-10-03T10:09:44.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.139+0000 7f946bb15700 1 -- 172.21.15.3:0/2470175053 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f946405a260 con 0x7f9464068ef0 2023-10-03T10:09:44.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.160+0000 7f9459ffb700 1 -- 172.21.15.3:0/2470175053 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v3) v1 ==== 172+0+0 (secure 0 0 0) 0x7f9460010af0 con 0x7f9464068ef0 2023-10-03T10:09:44.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.173+0000 7f946bb15700 1 -- 172.21.15.3:0/2470175053 >> v1:172.21.15.3:6816/118296 conn(0x7f944802bb30 legacy=0x7f944802dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:44.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.173+0000 7f946bb15700 1 -- 172.21.15.3:0/2470175053 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9464068ef0 msgr2=0x7f94641155f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:44.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.173+0000 7f946bb15700 1 --2- 172.21.15.3:0/2470175053 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9464068ef0 0x7f94641155f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f946000c4c0 tx=0x7f9460007c10).stop 2023-10-03T10:09:44.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.176+0000 7f946bb15700 1 -- 172.21.15.3:0/2470175053 shutdown_connections 2023-10-03T10:09:44.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.176+0000 7f946bb15700 1 --2- 172.21.15.3:0/2470175053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9464124550 0x7f9464111bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:44.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.176+0000 7f946bb15700 1 --2- 172.21.15.3:0/2470175053 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9464068ef0 0x7f94641155f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:44.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.176+0000 7f946bb15700 1 --2- 172.21.15.3:0/2470175053 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9464123940 0x7f94641116c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:44.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.176+0000 7f946bb15700 1 -- 172.21.15.3:0/2470175053 >> 172.21.15.3:0/2470175053 conn(0x7f9464063560 msgr2=0x7f9464067f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:44.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.178+0000 7f946bb15700 1 -- 172.21.15.3:0/2470175053 shutdown_connections 2023-10-03T10:09:44.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.180+0000 7f946bb15700 1 -- 172.21.15.3:0/2470175053 wait complete. 2023-10-03T10:09:44.184 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 3 and data pool 4 2023-10-03T10:09:44.203 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-03T10:09:44.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.332+0000 7fdf5c0c0700 1 Processor -- start 2023-10-03T10:09:44.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.333+0000 7fdf5c0c0700 1 -- start start 2023-10-03T10:09:44.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.333+0000 7fdf5c0c0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf54093810 0x7fdf54095300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:44.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.333+0000 7fdf5c0c0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf5412d120 0x7fdf5412f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:44.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.333+0000 7fdf5c0c0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf54093010 con 0x7fdf540957f0 2023-10-03T10:09:44.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.333+0000 7fdf5c0c0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf54093bf0 con 0x7fdf54093810 2023-10-03T10:09:44.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.333+0000 7fdf5c0c0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf54092aa0 con 0x7fdf5412d120 2023-10-03T10:09:44.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.333+0000 7fdf59e5c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf54093810 0x7fdf54095300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:44.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.333+0000 7fdf5965b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdf540957f0 0x7fdf54095bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35008/0 (socket says 172.21.15.3:35008) 2023-10-03T10:09:44.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.333+0000 7fdf5965b700 1 -- 172.21.15.3:0/1041643246 learned_addr learned my addr 172.21.15.3:0/1041643246 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:09:44.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.333+0000 7fdf5a65d700 1 --2- 172.21.15.3:0/1041643246 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf5412d120 0x7fdf5412f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:44.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.334+0000 7fdf58e5a700 1 -- 172.21.15.3:0/1041643246 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1709962434 0 0) 0x7fdf54093010 con 0x7fdf540957f0 2023-10-03T10:09:44.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.334+0000 7fdf58e5a700 1 -- 172.21.15.3:0/1041643246 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf400033f0 con 0x7fdf540957f0 2023-10-03T10:09:44.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.334+0000 7fdf59e5c700 1 -- 172.21.15.3:0/1041643246 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf5412d120 msgr2=0x7fdf5412f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:44.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.334+0000 7fdf59e5c700 1 --2- 172.21.15.3:0/1041643246 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf5412d120 0x7fdf5412f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:44.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.334+0000 7fdf59e5c700 1 -- 172.21.15.3:0/1041643246 >> v1:172.21.15.3:6789/0 conn(0x7fdf540957f0 legacy=0x7fdf54095bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:44.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.334+0000 7fdf59e5c700 1 -- 172.21.15.3:0/1041643246 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf54096e90 con 0x7fdf54093810 2023-10-03T10:09:44.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.334+0000 7fdf5a65d700 1 --2- 172.21.15.3:0/1041643246 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf5412d120 0x7fdf5412f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:44.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.334+0000 7fdf59e5c700 1 --2- 172.21.15.3:0/1041643246 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf54093810 0x7fdf54095300 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fdf50009790 tx=0x7fdf50009060).ready entity=mon.1 client_cookie=72959ceaae8d1910 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:44.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.334+0000 7fdf58e5a700 1 -- 172.21.15.3:0/1041643246 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5000f510 con 0x7fdf54093810 2023-10-03T10:09:44.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.334+0000 7fdf58e5a700 1 -- 172.21.15.3:0/1041643246 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5000f670 con 0x7fdf54093810 2023-10-03T10:09:44.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.334+0000 7fdf58e5a700 1 -- 172.21.15.3:0/1041643246 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5000f9f0 con 0x7fdf54093810 2023-10-03T10:09:44.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.335+0000 7fdf5c0c0700 1 -- 172.21.15.3:0/1041643246 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf54093810 msgr2=0x7fdf54095300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:44.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.335+0000 7fdf5c0c0700 1 --2- 172.21.15.3:0/1041643246 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf54093810 0x7fdf54095300 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fdf50009790 tx=0x7fdf50009060).stop 2023-10-03T10:09:44.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.335+0000 7fdf5c0c0700 1 -- 172.21.15.3:0/1041643246 shutdown_connections 2023-10-03T10:09:44.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.335+0000 7fdf5c0c0700 1 --2- 172.21.15.3:0/1041643246 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf54093810 0x7fdf54095300 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:44.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.335+0000 7fdf5c0c0700 1 --2- 172.21.15.3:0/1041643246 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf5412d120 0x7fdf5412f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:44.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.335+0000 7fdf5c0c0700 1 -- 172.21.15.3:0/1041643246 >> 172.21.15.3:0/1041643246 conn(0x7fdf54059bb0 msgr2=0x7fdf54097400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:44.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.335+0000 7fdf5c0c0700 1 -- 172.21.15.3:0/1041643246 shutdown_connections 2023-10-03T10:09:44.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.335+0000 7fdf5c0c0700 1 -- 172.21.15.3:0/1041643246 wait complete. 2023-10-03T10:09:44.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.336+0000 7fdf5c0c0700 1 Processor -- start 2023-10-03T10:09:44.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.336+0000 7fdf5c0c0700 1 -- start start 2023-10-03T10:09:44.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.336+0000 7fdf5c0c0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf54093810 0x7fdf5412ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:44.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.336+0000 7fdf5c0c0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf540957f0 0x7fdf5412bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:44.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.336+0000 7fdf5c0c0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf5412d120 0x7fdf5412c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:44.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.336+0000 7fdf5c0c0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5411c6e0 con 0x7fdf54093810 2023-10-03T10:09:44.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.336+0000 7fdf5c0c0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5406a3e0 con 0x7fdf540957f0 2023-10-03T10:09:44.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.336+0000 7fdf5c0c0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5406a6a0 con 0x7fdf5412d120 2023-10-03T10:09:44.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.336+0000 7fdf5965b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf540957f0 0x7fdf5412bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:44.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.336+0000 7fdf5a65d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf5412d120 0x7fdf5412c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:44.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.336+0000 7fdf5965b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf540957f0 0x7fdf5412bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34498/0 (socket says 172.21.15.3:34498) 2023-10-03T10:09:44.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.336+0000 7fdf5a65d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf5412d120 0x7fdf5412c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50958/0 (socket says 172.21.15.3:50958) 2023-10-03T10:09:44.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.336+0000 7fdf5965b700 1 -- 172.21.15.3:0/4139566247 learned_addr learned my addr 172.21.15.3:0/4139566247 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:44.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.337+0000 7fdf59e5c700 1 --2- 172.21.15.3:0/4139566247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf54093810 0x7fdf5412ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:44.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.337+0000 7fdf5965b700 1 -- 172.21.15.3:0/4139566247 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf5412d120 msgr2=0x7fdf5412c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:44.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.337+0000 7fdf5965b700 1 --2- 172.21.15.3:0/4139566247 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf5412d120 0x7fdf5412c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:44.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.337+0000 7fdf5965b700 1 -- 172.21.15.3:0/4139566247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf54093810 msgr2=0x7fdf5412ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:44.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.337+0000 7fdf5965b700 1 --2- 172.21.15.3:0/4139566247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf54093810 0x7fdf5412ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:44.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.337+0000 7fdf5965b700 1 -- 172.21.15.3:0/4139566247 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4c009a20 con 0x7fdf540957f0 2023-10-03T10:09:44.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.337+0000 7fdf59e5c700 1 --2- 172.21.15.3:0/4139566247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf54093810 0x7fdf5412ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:44.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.337+0000 7fdf5965b700 1 --2- 172.21.15.3:0/4139566247 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf540957f0 0x7fdf5412bf60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fdf4800c4c0 tx=0x7fdf48007c50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:44.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.337+0000 7fdf45ffb700 1 -- 172.21.15.3:0/4139566247 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf480072e0 con 0x7fdf540957f0 2023-10-03T10:09:44.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.337+0000 7fdf45ffb700 1 -- 172.21.15.3:0/4139566247 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf48007440 con 0x7fdf540957f0 2023-10-03T10:09:44.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.337+0000 7fdf5c0c0700 1 -- 172.21.15.3:0/4139566247 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf500092a0 con 0x7fdf540957f0 2023-10-03T10:09:44.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.337+0000 7fdf45ffb700 1 -- 172.21.15.3:0/4139566247 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf480057a0 con 0x7fdf540957f0 2023-10-03T10:09:44.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.337+0000 7fdf5c0c0700 1 -- 172.21.15.3:0/4139566247 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5411e230 con 0x7fdf540957f0 2023-10-03T10:09:44.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.338+0000 7fdf45ffb700 1 -- 172.21.15.3:0/4139566247 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fdf48005bd0 con 0x7fdf540957f0 2023-10-03T10:09:44.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.338+0000 7fdf45ffb700 1 -- 172.21.15.3:0/4139566247 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fdf4803a6c0 con 0x7fdf540957f0 2023-10-03T10:09:44.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.339+0000 7fdf5c0c0700 1 -- 172.21.15.3:0/4139566247 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf5405a260 con 0x7fdf540957f0 2023-10-03T10:09:44.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.343+0000 7fdf45ffb700 1 -- 172.21.15.3:0/4139566247 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fdf4801d3f0 con 0x7fdf540957f0 2023-10-03T10:09:44.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.517+0000 7fdf5c0c0700 1 -- 172.21.15.3:0/4139566247 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf54046c20 con 0x7fdf540957f0 2023-10-03T10:09:44.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.518+0000 7fdf45ffb700 1 -- 172.21.15.3:0/4139566247 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+11322 (secure 0 0 0) 0x7fdf48007d90 con 0x7fdf540957f0 2023-10-03T10:09:44.519 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:44.519 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":16,"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","created":"2023-10-03T10:09:28.703441+0000","modified":"2023-10-03T10:09:44.140902+0000","last_up_change":"2023-10-03T10:09:36.569401+0000","last_in_change":"2023-10-03T10:09:30.156546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-10-03T10:09:36.623971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-03T10:09:38.210614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-03T10:09:42.695873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-03T10:09:43.115588+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cdcdd930-3c5b-4486-a021-7132f90fb6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6808","nonce":118845}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6809","nonce":118845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6811","nonce":118845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6810","nonce":118845}]},"public_addr":"172.21.15.3:6808/118845","cluster_addr":"172.21.15.3:6809/118845","heartbeat_back_addr":"172.21.15.3:6811/118845","heartbeat_front_addr":"172.21.15.3:6810/118845","state":["exists","up"]},{"osd":1,"uuid":"c8e58877-6762-4741-a8c1-567ebc97b708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6804","nonce":118835}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6805","nonce":118835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6807","nonce":118835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6806","nonce":118835}]},"public_addr":"172.21.15.3:6804/118835","cluster_addr":"172.21.15.3:6805/118835","heartbeat_back_addr":"172.21.15.3:6807/118835","heartbeat_front_addr":"172.21.15.3:6806/118835","state":["exists","up"]},{"osd":2,"uuid":"68a2a21a-7a8e-4f82-8433-5201faa50777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6812","nonce":118864}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6813","nonce":118864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6815","nonce":118864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6814","nonce":118864}]},"public_addr":"172.21.15.3:6812/118864","cluster_addr":"172.21.15.3:6813/118864","heartbeat_back_addr":"172.21.15.3:6815/118864","heartbeat_front_addr":"172.21.15.3:6814/118864","state":["exists","up"]},{"osd":3,"uuid":"c4df2903-4e1e-4208-bf77-95b9fa749241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6800","nonce":118836}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6801","nonce":118836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6803","nonce":118836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6802","nonce":118836}]},"public_addr":"172.21.15.3:6800/118836","cluster_addr":"172.21.15.3:6801/118836","heartbeat_back_addr":"172.21.15.3:6803/118836","heartbeat_front_addr":"172.21.15.3:6802/118836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.369708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.257825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.337931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.085222+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-03T10:09:44.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.520+0000 7fdf5c0c0700 1 -- 172.21.15.3:0/4139566247 >> v1:172.21.15.3:6816/118296 conn(0x7fdf3802bb90 legacy=0x7fdf3802e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:44.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.520+0000 7fdf5c0c0700 1 -- 172.21.15.3:0/4139566247 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf540957f0 msgr2=0x7fdf5412bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:44.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.520+0000 7fdf5c0c0700 1 --2- 172.21.15.3:0/4139566247 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf540957f0 0x7fdf5412bf60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fdf4800c4c0 tx=0x7fdf48007c50).stop 2023-10-03T10:09:44.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.520+0000 7fdf5c0c0700 1 -- 172.21.15.3:0/4139566247 shutdown_connections 2023-10-03T10:09:44.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.520+0000 7fdf5c0c0700 1 --2- 172.21.15.3:0/4139566247 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf5412d120 0x7fdf5412c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:44.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.520+0000 7fdf5c0c0700 1 --2- 172.21.15.3:0/4139566247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf54093810 0x7fdf5412ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:44.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.520+0000 7fdf5c0c0700 1 --2- 172.21.15.3:0/4139566247 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf540957f0 0x7fdf5412bf60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:44.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.520+0000 7fdf5c0c0700 1 -- 172.21.15.3:0/4139566247 >> 172.21.15.3:0/4139566247 conn(0x7fdf54059bb0 msgr2=0x7fdf540696d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:44.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.520+0000 7fdf5c0c0700 1 -- 172.21.15.3:0/4139566247 shutdown_connections 2023-10-03T10:09:44.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.520+0000 7fdf5c0c0700 1 -- 172.21.15.3:0/4139566247 wait complete. 2023-10-03T10:09:44.534 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-03T10:09:44.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.663+0000 7f5d09279700 1 Processor -- start 2023-10-03T10:09:44.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.663+0000 7f5d09279700 1 -- start start 2023-10-03T10:09:44.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.664+0000 7f5d09279700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d04059070 0x7f5d040692c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:44.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.664+0000 7f5d09279700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d040697b0 0x7f5d04069bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:44.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.664+0000 7f5d09279700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d04092180 con 0x7f5d0412abd0 2023-10-03T10:09:44.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.664+0000 7f5d09279700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d04059450 con 0x7f5d040697b0 2023-10-03T10:09:44.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.664+0000 7f5d09279700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d040690d0 con 0x7f5d04059070 2023-10-03T10:09:44.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.664+0000 7f5d02ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d04059070 0x7f5d040692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:44.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.664+0000 7f5d027fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d040697b0 0x7f5d04069bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:44.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.664+0000 7f5d037fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5d0412abd0 0x7f5d0412b050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35018/0 (socket says 172.21.15.3:35018) 2023-10-03T10:09:44.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.664+0000 7f5d037fe700 1 -- 172.21.15.3:0/3898573328 learned_addr learned my addr 172.21.15.3:0/3898573328 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:09:44.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.665+0000 7f5d027fc700 1 -- 172.21.15.3:0/3898573328 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d04059070 msgr2=0x7f5d040692c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:44.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.665+0000 7f5d027fc700 1 --2- 172.21.15.3:0/3898573328 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d04059070 0x7f5d040692c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:44.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.665+0000 7f5d01ffb700 1 -- 172.21.15.3:0/3898573328 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3212844795 0 0) 0x7f5d04092180 con 0x7f5d0412abd0 2023-10-03T10:09:44.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.665+0000 7f5d027fc700 1 -- 172.21.15.3:0/3898573328 >> v1:172.21.15.3:6789/0 conn(0x7f5d0412abd0 legacy=0x7f5d0412b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:44.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.665+0000 7f5d027fc700 1 -- 172.21.15.3:0/3898573328 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d04092720 con 0x7f5d040697b0 2023-10-03T10:09:44.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.665+0000 7f5d02ffd700 1 --2- 172.21.15.3:0/3898573328 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d04059070 0x7f5d040692c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:44.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.665+0000 7f5d027fc700 1 --2- 172.21.15.3:0/3898573328 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d040697b0 0x7f5d04069bb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f5cf8009790 tx=0x7f5cf8009060).ready entity=mon.1 client_cookie=f44eb0ee634cfd2d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:44.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.665+0000 7f5d01ffb700 1 -- 172.21.15.3:0/3898573328 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf800f510 con 0x7f5d040697b0 2023-10-03T10:09:44.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.665+0000 7f5d01ffb700 1 -- 172.21.15.3:0/3898573328 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf800f670 con 0x7f5d040697b0 2023-10-03T10:09:44.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.665+0000 7f5d01ffb700 1 -- 172.21.15.3:0/3898573328 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf800f9f0 con 0x7f5d040697b0 2023-10-03T10:09:44.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.665+0000 7f5d09279700 1 -- 172.21.15.3:0/3898573328 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d040697b0 msgr2=0x7f5d04069bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:44.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.665+0000 7f5d09279700 1 --2- 172.21.15.3:0/3898573328 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d040697b0 0x7f5d04069bb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f5cf8009790 tx=0x7f5cf8009060).stop 2023-10-03T10:09:44.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.665+0000 7f5d09279700 1 -- 172.21.15.3:0/3898573328 shutdown_connections 2023-10-03T10:09:44.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.665+0000 7f5d09279700 1 --2- 172.21.15.3:0/3898573328 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d04059070 0x7f5d040692c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:44.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.665+0000 7f5d09279700 1 --2- 172.21.15.3:0/3898573328 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d040697b0 0x7f5d04069bb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:44.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.666+0000 7f5d09279700 1 -- 172.21.15.3:0/3898573328 >> 172.21.15.3:0/3898573328 conn(0x7f5d0409a220 msgr2=0x7f5d0409a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:44.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.666+0000 7f5d09279700 1 -- 172.21.15.3:0/3898573328 shutdown_connections 2023-10-03T10:09:44.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.666+0000 7f5d09279700 1 -- 172.21.15.3:0/3898573328 wait complete. 2023-10-03T10:09:44.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.667+0000 7f5d09279700 1 Processor -- start 2023-10-03T10:09:44.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.667+0000 7f5d09279700 1 -- start start 2023-10-03T10:09:44.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.667+0000 7f5d09279700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d04059070 0x7f5d0413a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:44.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.667+0000 7f5d09279700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d040697b0 0x7f5d0413ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:44.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.667+0000 7f5d09279700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d0412abd0 0x7f5d04114d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:44.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.667+0000 7f5d09279700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d04094530 con 0x7f5d04059070 2023-10-03T10:09:44.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.667+0000 7f5d09279700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d04118a40 con 0x7f5d040697b0 2023-10-03T10:09:44.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.667+0000 7f5d09279700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d041161d0 con 0x7f5d0412abd0 2023-10-03T10:09:44.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.667+0000 7f5d027fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d040697b0 0x7f5d0413ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:44.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.667+0000 7f5d037fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d0412abd0 0x7f5d04114d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:44.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.667+0000 7f5d02ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d04059070 0x7f5d0413a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:44.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.667+0000 7f5d027fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d040697b0 0x7f5d0413ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34518/0 (socket says 172.21.15.3:34518) 2023-10-03T10:09:44.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.667+0000 7f5d037fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d0412abd0 0x7f5d04114d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50968/0 (socket says 172.21.15.3:50968) 2023-10-03T10:09:44.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.667+0000 7f5d02ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d04059070 0x7f5d0413a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35394/0 (socket says 172.21.15.3:35394) 2023-10-03T10:09:44.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.667+0000 7f5d027fc700 1 -- 172.21.15.3:0/3300981060 learned_addr learned my addr 172.21.15.3:0/3300981060 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:44.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.668+0000 7f5d027fc700 1 -- 172.21.15.3:0/3300981060 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d0412abd0 msgr2=0x7f5d04114d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:44.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.668+0000 7f5d027fc700 1 --2- 172.21.15.3:0/3300981060 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d0412abd0 0x7f5d04114d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:44.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.668+0000 7f5d027fc700 1 -- 172.21.15.3:0/3300981060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d04059070 msgr2=0x7f5d0413a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:44.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.668+0000 7f5d027fc700 1 --2- 172.21.15.3:0/3300981060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d04059070 0x7f5d0413a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:44.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.668+0000 7f5d027fc700 1 -- 172.21.15.3:0/3300981060 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf40092a0 con 0x7f5d040697b0 2023-10-03T10:09:44.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.668+0000 7f5d02ffd700 1 --2- 172.21.15.3:0/3300981060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d04059070 0x7f5d0413a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:44.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.668+0000 7f5d037fe700 1 --2- 172.21.15.3:0/3300981060 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d0412abd0 0x7f5d04114d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:44.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.668+0000 7f5d027fc700 1 --2- 172.21.15.3:0/3300981060 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d040697b0 0x7f5d0413ab50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f5cf8009e30 tx=0x7f5cf8017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:44.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.668+0000 7f5ceaffd700 1 -- 172.21.15.3:0/3300981060 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf800fe80 con 0x7f5d040697b0 2023-10-03T10:09:44.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.668+0000 7f5ceaffd700 1 -- 172.21.15.3:0/3300981060 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf800f7e0 con 0x7f5d040697b0 2023-10-03T10:09:44.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.668+0000 7f5ceaffd700 1 -- 172.21.15.3:0/3300981060 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf801b640 con 0x7f5d040697b0 2023-10-03T10:09:44.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.668+0000 7f5d09279700 1 -- 172.21.15.3:0/3300981060 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf80092a0 con 0x7f5d040697b0 2023-10-03T10:09:44.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.668+0000 7f5d09279700 1 -- 172.21.15.3:0/3300981060 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d041172e0 con 0x7f5d040697b0 2023-10-03T10:09:44.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.669+0000 7f5ceaffd700 1 -- 172.21.15.3:0/3300981060 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f5cf801b9b0 con 0x7f5d040697b0 2023-10-03T10:09:44.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.669+0000 7f5ceaffd700 1 -- 172.21.15.3:0/3300981060 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f5cf803c880 con 0x7f5d040697b0 2023-10-03T10:09:44.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.669+0000 7f5d09279700 1 -- 172.21.15.3:0/3300981060 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d04069dd0 con 0x7f5d040697b0 2023-10-03T10:09:44.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.674+0000 7f5ceaffd700 1 -- 172.21.15.3:0/3300981060 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f5cf801bc60 con 0x7f5d040697b0 2023-10-03T10:09:44.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.853+0000 7f5d09279700 1 -- 172.21.15.3:0/3300981060 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d0405a260 con 0x7f5d040697b0 2023-10-03T10:09:44.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.854+0000 7f5ceaffd700 1 -- 172.21.15.3:0/3300981060 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+11322 (secure 0 0 0) 0x7f5cf802c350 con 0x7f5d040697b0 2023-10-03T10:09:44.854 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:44.854 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":16,"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","created":"2023-10-03T10:09:28.703441+0000","modified":"2023-10-03T10:09:44.140902+0000","last_up_change":"2023-10-03T10:09:36.569401+0000","last_in_change":"2023-10-03T10:09:30.156546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-10-03T10:09:36.623971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-03T10:09:38.210614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-03T10:09:42.695873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-03T10:09:43.115588+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cdcdd930-3c5b-4486-a021-7132f90fb6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6808","nonce":118845}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6809","nonce":118845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6811","nonce":118845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6810","nonce":118845}]},"public_addr":"172.21.15.3:6808/118845","cluster_addr":"172.21.15.3:6809/118845","heartbeat_back_addr":"172.21.15.3:6811/118845","heartbeat_front_addr":"172.21.15.3:6810/118845","state":["exists","up"]},{"osd":1,"uuid":"c8e58877-6762-4741-a8c1-567ebc97b708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6804","nonce":118835}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6805","nonce":118835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6807","nonce":118835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6806","nonce":118835}]},"public_addr":"172.21.15.3:6804/118835","cluster_addr":"172.21.15.3:6805/118835","heartbeat_back_addr":"172.21.15.3:6807/118835","heartbeat_front_addr":"172.21.15.3:6806/118835","state":["exists","up"]},{"osd":2,"uuid":"68a2a21a-7a8e-4f82-8433-5201faa50777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6812","nonce":118864}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6813","nonce":118864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6815","nonce":118864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6814","nonce":118864}]},"public_addr":"172.21.15.3:6812/118864","cluster_addr":"172.21.15.3:6813/118864","heartbeat_back_addr":"172.21.15.3:6815/118864","heartbeat_front_addr":"172.21.15.3:6814/118864","state":["exists","up"]},{"osd":3,"uuid":"c4df2903-4e1e-4208-bf77-95b9fa749241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6800","nonce":118836}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6801","nonce":118836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6803","nonce":118836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6802","nonce":118836}]},"public_addr":"172.21.15.3:6800/118836","cluster_addr":"172.21.15.3:6801/118836","heartbeat_back_addr":"172.21.15.3:6803/118836","heartbeat_front_addr":"172.21.15.3:6802/118836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.369708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.257825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.337931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.085222+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-03T10:09:44.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.856+0000 7f5d09279700 1 -- 172.21.15.3:0/3300981060 >> v1:172.21.15.3:6816/118296 conn(0x7f5cf002bb60 legacy=0x7f5cf002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:44.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.856+0000 7f5d09279700 1 -- 172.21.15.3:0/3300981060 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d040697b0 msgr2=0x7f5d0413ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:44.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.856+0000 7f5d09279700 1 --2- 172.21.15.3:0/3300981060 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d040697b0 0x7f5d0413ab50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f5cf8009e30 tx=0x7f5cf8017000).stop 2023-10-03T10:09:44.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.856+0000 7f5d09279700 1 -- 172.21.15.3:0/3300981060 shutdown_connections 2023-10-03T10:09:44.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.856+0000 7f5d09279700 1 --2- 172.21.15.3:0/3300981060 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d0412abd0 0x7f5d04114d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:44.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.856+0000 7f5d09279700 1 --2- 172.21.15.3:0/3300981060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d04059070 0x7f5d0413a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:44.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.856+0000 7f5d09279700 1 --2- 172.21.15.3:0/3300981060 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d040697b0 0x7f5d0413ab50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:44.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.856+0000 7f5d09279700 1 -- 172.21.15.3:0/3300981060 >> 172.21.15.3:0/3300981060 conn(0x7f5d0409a220 msgr2=0x7f5d0412e090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:44.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.856+0000 7f5d09279700 1 -- 172.21.15.3:0/3300981060 shutdown_connections 2023-10-03T10:09:44.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:44.856+0000 7f5d09279700 1 -- 172.21.15.3:0/3300981060 wait complete. 2023-10-03T10:09:44.869 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-03T10:09:45.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.002+0000 7fc55bcb7700 1 Processor -- start 2023-10-03T10:09:45.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.002+0000 7fc55bcb7700 1 -- start start 2023-10-03T10:09:45.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.002+0000 7fc55bcb7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc554062a90 0x7fc5541255c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:45.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.002+0000 7fc55bcb7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc554125ab0 0x7fc554125ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:45.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.002+0000 7fc55bcb7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc554093c50 0x7fc554094100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:45.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.002+0000 7fc55bcb7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc554099a30 con 0x7fc554093c50 2023-10-03T10:09:45.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.002+0000 7fc55bcb7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5540631c0 con 0x7fc554062a90 2023-10-03T10:09:45.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.002+0000 7fc55bcb7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc55409a350 con 0x7fc554125ab0 2023-10-03T10:09:45.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.003+0000 7fc55a254700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc554093c50 0x7fc554094100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:45.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.003+0000 7fc559252700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc554125ab0 0x7fc554125ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:45.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.003+0000 7fc559a53700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc554062a90 0x7fc5541255c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:45.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.003+0000 7fc559252700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc554125ab0 0x7fc554125ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50974/0 (socket says 172.21.15.3:50974) 2023-10-03T10:09:45.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.003+0000 7fc55a254700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc554093c50 0x7fc554094100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35396/0 (socket says 172.21.15.3:35396) 2023-10-03T10:09:45.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.003+0000 7fc559252700 1 -- 172.21.15.3:0/39016872 learned_addr learned my addr 172.21.15.3:0/39016872 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:45.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.003+0000 7fc559252700 1 -- 172.21.15.3:0/39016872 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc554062a90 msgr2=0x7fc5541255c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:45.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.003+0000 7fc559252700 1 --2- 172.21.15.3:0/39016872 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc554062a90 0x7fc5541255c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:45.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.003+0000 7fc559252700 1 -- 172.21.15.3:0/39016872 >> v2:172.21.15.3:3300/0 conn(0x7fc554093c50 msgr2=0x7fc554094100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:45.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.003+0000 7fc559252700 1 --2- 172.21.15.3:0/39016872 >> v2:172.21.15.3:3300/0 conn(0x7fc554093c50 0x7fc554094100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:45.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.003+0000 7fc559252700 1 -- 172.21.15.3:0/39016872 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc554099420 con 0x7fc554125ab0 2023-10-03T10:09:45.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.003+0000 7fc559a53700 1 --2- 172.21.15.3:0/39016872 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc554062a90 0x7fc5541255c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:45.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.003+0000 7fc55a254700 1 --2- 172.21.15.3:0/39016872 >> v2:172.21.15.3:3300/0 conn(0x7fc554093c50 0x7fc554094100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:45.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.003+0000 7fc559252700 1 --2- 172.21.15.3:0/39016872 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc554125ab0 0x7fc554125ed0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fc544006fd0 tx=0x7fc5440090f0).ready entity=mon.2 client_cookie=8362c21383de6673 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:45.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.003+0000 7fc558a51700 1 -- 172.21.15.3:0/39016872 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54400f510 con 0x7fc554125ab0 2023-10-03T10:09:45.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.003+0000 7fc558a51700 1 -- 172.21.15.3:0/39016872 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54400f670 con 0x7fc554125ab0 2023-10-03T10:09:45.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.003+0000 7fc558a51700 1 -- 172.21.15.3:0/39016872 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54400f9f0 con 0x7fc554125ab0 2023-10-03T10:09:45.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.004+0000 7fc55bcb7700 1 -- 172.21.15.3:0/39016872 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc554125ab0 msgr2=0x7fc554125ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:45.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.004+0000 7fc55bcb7700 1 --2- 172.21.15.3:0/39016872 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc554125ab0 0x7fc554125ed0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fc544006fd0 tx=0x7fc5440090f0).stop 2023-10-03T10:09:45.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.004+0000 7fc55bcb7700 1 -- 172.21.15.3:0/39016872 shutdown_connections 2023-10-03T10:09:45.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.004+0000 7fc55bcb7700 1 --2- 172.21.15.3:0/39016872 >> v2:172.21.15.3:3300/0 conn(0x7fc554093c50 0x7fc554094100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:45.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.004+0000 7fc55bcb7700 1 --2- 172.21.15.3:0/39016872 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc554062a90 0x7fc5541255c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:45.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.004+0000 7fc55bcb7700 1 --2- 172.21.15.3:0/39016872 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc554125ab0 0x7fc554125ed0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:45.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.004+0000 7fc55bcb7700 1 -- 172.21.15.3:0/39016872 >> 172.21.15.3:0/39016872 conn(0x7fc5540599a0 msgr2=0x7fc554097460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:45.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.004+0000 7fc55bcb7700 1 -- 172.21.15.3:0/39016872 shutdown_connections 2023-10-03T10:09:45.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.004+0000 7fc55bcb7700 1 -- 172.21.15.3:0/39016872 wait complete. 2023-10-03T10:09:45.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.005+0000 7fc55bcb7700 1 Processor -- start 2023-10-03T10:09:45.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.005+0000 7fc55bcb7700 1 -- start start 2023-10-03T10:09:45.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.005+0000 7fc55bcb7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc554062a90 0x7fc554091a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:45.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.005+0000 7fc55bcb7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc554093c50 0x7fc554091f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:45.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.006+0000 7fc55bcb7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc554125ab0 0x7fc554092420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:45.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.006+0000 7fc55bcb7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc554092910 con 0x7fc554125ab0 2023-10-03T10:09:45.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.006+0000 7fc55bcb7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc554098c40 con 0x7fc554062a90 2023-10-03T10:09:45.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.006+0000 7fc55bcb7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc5541173e0 con 0x7fc554093c50 2023-10-03T10:09:45.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.006+0000 7fc559252700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc554093c50 0x7fc554091f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:45.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.006+0000 7fc559a53700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc554062a90 0x7fc554091a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:45.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.006+0000 7fc559252700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc554093c50 0x7fc554091f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50976/0 (socket says 172.21.15.3:50976) 2023-10-03T10:09:45.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.006+0000 7fc559252700 1 -- 172.21.15.3:0/3350349069 learned_addr learned my addr 172.21.15.3:0/3350349069 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:45.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.006+0000 7fc55a254700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc554125ab0 0x7fc554092420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:45.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.006+0000 7fc559252700 1 -- 172.21.15.3:0/3350349069 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc554062a90 msgr2=0x7fc554091a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:45.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.006+0000 7fc559252700 1 --2- 172.21.15.3:0/3350349069 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc554062a90 0x7fc554091a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:45.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.006+0000 7fc559252700 1 -- 172.21.15.3:0/3350349069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc554125ab0 msgr2=0x7fc554092420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:45.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.006+0000 7fc559252700 1 --2- 172.21.15.3:0/3350349069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc554125ab0 0x7fc554092420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:45.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.006+0000 7fc559252700 1 -- 172.21.15.3:0/3350349069 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5500092a0 con 0x7fc554093c50 2023-10-03T10:09:45.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.006+0000 7fc559252700 1 --2- 172.21.15.3:0/3350349069 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc554093c50 0x7fc554091f30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fc554065e40 tx=0x7fc544017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:45.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.006+0000 7fc549ffb700 1 -- 172.21.15.3:0/3350349069 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54400f7e0 con 0x7fc554093c50 2023-10-03T10:09:45.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.006+0000 7fc549ffb700 1 -- 172.21.15.3:0/3350349069 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54400f940 con 0x7fc554093c50 2023-10-03T10:09:45.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.006+0000 7fc55bcb7700 1 -- 172.21.15.3:0/3350349069 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54c009a20 con 0x7fc554093c50 2023-10-03T10:09:45.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.006+0000 7fc549ffb700 1 -- 172.21.15.3:0/3350349069 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54401d680 con 0x7fc554093c50 2023-10-03T10:09:45.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.007+0000 7fc55bcb7700 1 -- 172.21.15.3:0/3350349069 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc544009370 con 0x7fc554093c50 2023-10-03T10:09:45.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.007+0000 7fc549ffb700 1 -- 172.21.15.3:0/3350349069 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fc54401da30 con 0x7fc554093c50 2023-10-03T10:09:45.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.007+0000 7fc549ffb700 1 -- 172.21.15.3:0/3350349069 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fc54403ccd0 con 0x7fc554093c50 2023-10-03T10:09:45.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.008+0000 7fc55bcb7700 1 -- 172.21.15.3:0/3350349069 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc554099050 con 0x7fc554093c50 2023-10-03T10:09:45.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.013+0000 7fc549ffb700 1 -- 172.21.15.3:0/3350349069 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fc544018350 con 0x7fc554093c50 2023-10-03T10:09:45.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:45.235+0000 7fc55bcb7700 1 -- 172.21.15.3:0/3350349069 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc554129020 con 0x7fc554093c50 2023-10-03T10:09:46.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.155+0000 7fc549ffb700 1 -- 172.21.15.3:0/3350349069 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc544010b90 con 0x7fc554093c50 2023-10-03T10:09:46.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.157+0000 7fc55bcb7700 1 -- 172.21.15.3:0/3350349069 >> v1:172.21.15.3:6816/118296 conn(0x7fc53802bb60 legacy=0x7fc53802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:46.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.157+0000 7fc55bcb7700 1 -- 172.21.15.3:0/3350349069 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc554093c50 msgr2=0x7fc554091f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:46.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.157+0000 7fc55bcb7700 1 --2- 172.21.15.3:0/3350349069 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc554093c50 0x7fc554091f30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fc554065e40 tx=0x7fc544017000).stop 2023-10-03T10:09:46.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.157+0000 7fc55bcb7700 1 -- 172.21.15.3:0/3350349069 shutdown_connections 2023-10-03T10:09:46.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.157+0000 7fc55bcb7700 1 --2- 172.21.15.3:0/3350349069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc554125ab0 0x7fc554092420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:46.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.157+0000 7fc55bcb7700 1 --2- 172.21.15.3:0/3350349069 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc554062a90 0x7fc554091a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:46.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.157+0000 7fc55bcb7700 1 --2- 172.21.15.3:0/3350349069 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc554093c50 0x7fc554091f30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:46.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.157+0000 7fc55bcb7700 1 -- 172.21.15.3:0/3350349069 >> 172.21.15.3:0/3350349069 conn(0x7fc5540599a0 msgr2=0x7fc554127a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:46.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.158+0000 7fc55bcb7700 1 -- 172.21.15.3:0/3350349069 shutdown_connections 2023-10-03T10:09:46.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.158+0000 7fc55bcb7700 1 -- 172.21.15.3:0/3350349069 wait complete. 2023-10-03T10:09:46.172 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-03T10:09:46.173 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-03T10:09:46.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.309+0000 7fcf59741700 1 Processor -- start 2023-10-03T10:09:46.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.309+0000 7fcf59741700 1 -- start start 2023-10-03T10:09:46.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.310+0000 7fcf59741700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcf54097d00 0x7fcf54098df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:46.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.310+0000 7fcf59741700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcf540992e0 0x7fcf541324b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:46.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.310+0000 7fcf59741700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf54098190 con 0x7fcf54065340 2023-10-03T10:09:46.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.310+0000 7fcf59741700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf54065720 con 0x7fcf54097d00 2023-10-03T10:09:46.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.310+0000 7fcf59741700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcf54096190 con 0x7fcf540992e0 2023-10-03T10:09:46.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.310+0000 7fcf537fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcf540992e0 0x7fcf541324b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:46.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.310+0000 7fcf527fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcf54097d00 0x7fcf54098df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:46.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.310+0000 7fcf527fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcf54097d00 0x7fcf54098df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35412/0 (socket says 172.21.15.3:35412) 2023-10-03T10:09:46.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.310+0000 7fcf52ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcf54065340 0x7fcf540986e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35034/0 (socket says 172.21.15.3:35034) 2023-10-03T10:09:46.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.310+0000 7fcf537fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcf540992e0 0x7fcf541324b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50992/0 (socket says 172.21.15.3:50992) 2023-10-03T10:09:46.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.310+0000 7fcf527fc700 1 -- 172.21.15.3:0/2870368418 learned_addr learned my addr 172.21.15.3:0/2870368418 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:46.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.311+0000 7fcf527fc700 1 -- 172.21.15.3:0/2870368418 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcf540992e0 msgr2=0x7fcf541324b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:46.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.311+0000 7fcf527fc700 1 --2- 172.21.15.3:0/2870368418 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcf540992e0 0x7fcf541324b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:46.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.311+0000 7fcf527fc700 1 -- 172.21.15.3:0/2870368418 >> v1:172.21.15.3:6789/0 conn(0x7fcf54065340 legacy=0x7fcf540986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:46.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.311+0000 7fcf51ffb700 1 -- 172.21.15.3:0/2870368418 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1107463847 0 0) 0x7fcf54098190 con 0x7fcf54065340 2023-10-03T10:09:46.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.311+0000 7fcf527fc700 1 -- 172.21.15.3:0/2870368418 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf54066010 con 0x7fcf54097d00 2023-10-03T10:09:46.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.311+0000 7fcf527fc700 1 --2- 172.21.15.3:0/2870368418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf54097d00 0x7fcf54098df0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fcf48006fd0 tx=0x7fcf480090f0).ready entity=mon.0 client_cookie=4afbeeca1805b7e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:46.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.311+0000 7fcf51ffb700 1 -- 172.21.15.3:0/2870368418 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4800b580 con 0x7fcf54097d00 2023-10-03T10:09:46.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.311+0000 7fcf51ffb700 1 -- 172.21.15.3:0/2870368418 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4800b6e0 con 0x7fcf54097d00 2023-10-03T10:09:46.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.311+0000 7fcf51ffb700 1 -- 172.21.15.3:0/2870368418 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4800ba80 con 0x7fcf54097d00 2023-10-03T10:09:46.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.311+0000 7fcf59741700 1 -- 172.21.15.3:0/2870368418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf54097d00 msgr2=0x7fcf54098df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:46.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.311+0000 7fcf59741700 1 --2- 172.21.15.3:0/2870368418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf54097d00 0x7fcf54098df0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fcf48006fd0 tx=0x7fcf480090f0).stop 2023-10-03T10:09:46.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.311+0000 7fcf59741700 1 -- 172.21.15.3:0/2870368418 shutdown_connections 2023-10-03T10:09:46.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.311+0000 7fcf59741700 1 --2- 172.21.15.3:0/2870368418 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcf540992e0 0x7fcf541324b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:46.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.311+0000 7fcf59741700 1 --2- 172.21.15.3:0/2870368418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf54097d00 0x7fcf54098df0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:46.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.311+0000 7fcf59741700 1 -- 172.21.15.3:0/2870368418 >> 172.21.15.3:0/2870368418 conn(0x7fcf5409dcc0 msgr2=0x7fcf5409e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:46.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.311+0000 7fcf59741700 1 -- 172.21.15.3:0/2870368418 shutdown_connections 2023-10-03T10:09:46.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.312+0000 7fcf59741700 1 -- 172.21.15.3:0/2870368418 wait complete. 2023-10-03T10:09:46.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.313+0000 7fcf59741700 1 Processor -- start 2023-10-03T10:09:46.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.313+0000 7fcf59741700 1 -- start start 2023-10-03T10:09:46.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.313+0000 7fcf59741700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcf54065340 0x7fcf5411e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:46.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.313+0000 7fcf59741700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcf540992e0 0x7fcf5411a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:46.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.313+0000 7fcf59741700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf5411a8d0 0x7fcf5411ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:46.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.313+0000 7fcf59741700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf541325b0 con 0x7fcf5411a8d0 2023-10-03T10:09:46.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.313+0000 7fcf59741700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5411caa0 con 0x7fcf54065340 2023-10-03T10:09:46.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.313+0000 7fcf59741700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcf54099ae0 con 0x7fcf540992e0 2023-10-03T10:09:46.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.313+0000 7fcf527fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcf540992e0 0x7fcf5411a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:46.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.313+0000 7fcf537fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf5411a8d0 0x7fcf5411ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:46.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.313+0000 7fcf52ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcf54065340 0x7fcf5411e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:46.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.313+0000 7fcf527fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcf540992e0 0x7fcf5411a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51006/0 (socket says 172.21.15.3:51006) 2023-10-03T10:09:46.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.313+0000 7fcf527fc700 1 -- 172.21.15.3:0/454520634 learned_addr learned my addr 172.21.15.3:0/454520634 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:46.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.313+0000 7fcf537fe700 1 -- 172.21.15.3:0/454520634 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcf540992e0 msgr2=0x7fcf5411a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:46.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.314+0000 7fcf537fe700 1 --2- 172.21.15.3:0/454520634 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcf540992e0 0x7fcf5411a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:46.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.314+0000 7fcf537fe700 1 -- 172.21.15.3:0/454520634 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcf54065340 msgr2=0x7fcf5411e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:46.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.314+0000 7fcf537fe700 1 --2- 172.21.15.3:0/454520634 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcf54065340 0x7fcf5411e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:46.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.314+0000 7fcf537fe700 1 -- 172.21.15.3:0/454520634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf48009370 con 0x7fcf5411a8d0 2023-10-03T10:09:46.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.314+0000 7fcf527fc700 1 --2- 172.21.15.3:0/454520634 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcf540992e0 0x7fcf5411a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:46.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.314+0000 7fcf52ffd700 1 --2- 172.21.15.3:0/454520634 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcf54065340 0x7fcf5411e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:46.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.314+0000 7fcf537fe700 1 --2- 172.21.15.3:0/454520634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf5411a8d0 0x7fcf5411ad30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fcf4c009e20 tx=0x7fcf4c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:46.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.314+0000 7fcf3affd700 1 -- 172.21.15.3:0/454520634 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c00caa0 con 0x7fcf5411a8d0 2023-10-03T10:09:46.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.314+0000 7fcf3affd700 1 -- 172.21.15.3:0/454520634 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4c00f040 con 0x7fcf5411a8d0 2023-10-03T10:09:46.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.314+0000 7fcf59741700 1 -- 172.21.15.3:0/454520634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf540665e0 con 0x7fcf5411a8d0 2023-10-03T10:09:46.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.314+0000 7fcf3affd700 1 -- 172.21.15.3:0/454520634 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c00ce10 con 0x7fcf5411a8d0 2023-10-03T10:09:46.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.314+0000 7fcf59741700 1 -- 172.21.15.3:0/454520634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5411b220 con 0x7fcf5411a8d0 2023-10-03T10:09:46.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.315+0000 7fcf3affd700 1 -- 172.21.15.3:0/454520634 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fcf4c013610 con 0x7fcf5411a8d0 2023-10-03T10:09:46.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.315+0000 7fcf3affd700 1 -- 172.21.15.3:0/454520634 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fcf4c03b750 con 0x7fcf5411a8d0 2023-10-03T10:09:46.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.315+0000 7fcf59741700 1 -- 172.21.15.3:0/454520634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf5405a260 con 0x7fcf5411a8d0 2023-10-03T10:09:46.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.320+0000 7fcf3affd700 1 -- 172.21.15.3:0/454520634 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fcf4c01d460 con 0x7fcf5411a8d0 2023-10-03T10:09:46.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:46.538+0000 7fcf59741700 1 -- 172.21.15.3:0/454520634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fcf54097470 con 0x7fcf5411a8d0 2023-10-03T10:09:47.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.157+0000 7fcf3affd700 1 -- 172.21.15.3:0/454520634 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcf4c007340 con 0x7fcf5411a8d0 2023-10-03T10:09:47.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.160+0000 7fcf59741700 1 -- 172.21.15.3:0/454520634 >> v1:172.21.15.3:6816/118296 conn(0x7fcf4002bb60 legacy=0x7fcf4002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:47.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.160+0000 7fcf59741700 1 -- 172.21.15.3:0/454520634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf5411a8d0 msgr2=0x7fcf5411ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:47.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.160+0000 7fcf59741700 1 --2- 172.21.15.3:0/454520634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf5411a8d0 0x7fcf5411ad30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fcf4c009e20 tx=0x7fcf4c00b040).stop 2023-10-03T10:09:47.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.160+0000 7fcf59741700 1 -- 172.21.15.3:0/454520634 shutdown_connections 2023-10-03T10:09:47.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.160+0000 7fcf59741700 1 --2- 172.21.15.3:0/454520634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf5411a8d0 0x7fcf5411ad30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:47.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.160+0000 7fcf59741700 1 --2- 172.21.15.3:0/454520634 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcf54065340 0x7fcf5411e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:47.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.160+0000 7fcf59741700 1 --2- 172.21.15.3:0/454520634 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcf540992e0 0x7fcf5411a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:47.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.160+0000 7fcf59741700 1 -- 172.21.15.3:0/454520634 >> 172.21.15.3:0/454520634 conn(0x7fcf5409dcc0 msgr2=0x7fcf5409bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:47.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.161+0000 7fcf59741700 1 -- 172.21.15.3:0/454520634 shutdown_connections 2023-10-03T10:09:47.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.161+0000 7fcf59741700 1 -- 172.21.15.3:0/454520634 wait complete. 2023-10-03T10:09:47.176 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-03T10:09:47.176 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-03T10:09:47.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.309+0000 7fddc85ed700 1 Processor -- start 2023-10-03T10:09:47.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.309+0000 7fddc85ed700 1 -- start start 2023-10-03T10:09:47.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.310+0000 7fddc85ed700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fddc0127360 0x7fddc012fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:47.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.310+0000 7fddc85ed700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fddc0127f70 0x7fddc0132140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:47.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.310+0000 7fddc85ed700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc0091bc0 con 0x7fddc0062a90 2023-10-03T10:09:47.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.310+0000 7fddc85ed700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc0062e70 con 0x7fddc0127360 2023-10-03T10:09:47.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.310+0000 7fddc85ed700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc00968b0 con 0x7fddc0127f70 2023-10-03T10:09:47.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.310+0000 7fddc5b88700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fddc0127360 0x7fddc012fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:47.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.310+0000 7fddc6b8a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fddc0127f70 0x7fddc0132140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:47.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.310+0000 7fddc6389700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fddc0062a90 0x7fddc0127860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35050/0 (socket says 172.21.15.3:35050) 2023-10-03T10:09:47.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.310+0000 7fddc6389700 1 -- 172.21.15.3:0/3132831307 learned_addr learned my addr 172.21.15.3:0/3132831307 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:09:47.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.310+0000 7fddc5b88700 1 -- 172.21.15.3:0/3132831307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fddc0127f70 msgr2=0x7fddc0132140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:47.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.310+0000 7fddc5b88700 1 --2- 172.21.15.3:0/3132831307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fddc0127f70 0x7fddc0132140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:47.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.310+0000 7fddc5b88700 1 -- 172.21.15.3:0/3132831307 >> v1:172.21.15.3:6789/0 conn(0x7fddc0062a90 legacy=0x7fddc0127860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:47.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.310+0000 7fddc5b88700 1 -- 172.21.15.3:0/3132831307 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc0092a70 con 0x7fddc0127360 2023-10-03T10:09:47.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.310+0000 7fddc5387700 1 -- 172.21.15.3:0/3132831307 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 697001264 0 0) 0x7fddc0091bc0 con 0x7fddc0062a90 2023-10-03T10:09:47.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.310+0000 7fddc6b8a700 1 --2- 172.21.15.3:0/3132831307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fddc0127f70 0x7fddc0132140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:47.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.311+0000 7fddc5b88700 1 --2- 172.21.15.3:0/3132831307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddc0127360 0x7fddc012fc40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fddb0009760 tx=0x7fddb0009060).ready entity=mon.0 client_cookie=bb83d04427b00ed6 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:47.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.311+0000 7fddc5387700 1 -- 172.21.15.3:0/3132831307 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb00090d0 con 0x7fddc0127360 2023-10-03T10:09:47.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.311+0000 7fddc5387700 1 -- 172.21.15.3:0/3132831307 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb000f510 con 0x7fddc0127360 2023-10-03T10:09:47.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.311+0000 7fddc5387700 1 -- 172.21.15.3:0/3132831307 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb000f890 con 0x7fddc0127360 2023-10-03T10:09:47.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.311+0000 7fddc85ed700 1 -- 172.21.15.3:0/3132831307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddc0127360 msgr2=0x7fddc012fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:47.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.311+0000 7fddc85ed700 1 --2- 172.21.15.3:0/3132831307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddc0127360 0x7fddc012fc40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fddb0009760 tx=0x7fddb0009060).stop 2023-10-03T10:09:47.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.311+0000 7fddc85ed700 1 -- 172.21.15.3:0/3132831307 shutdown_connections 2023-10-03T10:09:47.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.311+0000 7fddc85ed700 1 --2- 172.21.15.3:0/3132831307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fddc0127f70 0x7fddc0132140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:47.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.311+0000 7fddc85ed700 1 --2- 172.21.15.3:0/3132831307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddc0127360 0x7fddc012fc40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:47.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.311+0000 7fddc85ed700 1 -- 172.21.15.3:0/3132831307 >> 172.21.15.3:0/3132831307 conn(0x7fddc0059bb0 msgr2=0x7fddc0097400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:47.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.311+0000 7fddc85ed700 1 -- 172.21.15.3:0/3132831307 shutdown_connections 2023-10-03T10:09:47.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.311+0000 7fddc85ed700 1 -- 172.21.15.3:0/3132831307 wait complete. 2023-10-03T10:09:47.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.312+0000 7fddc85ed700 1 Processor -- start 2023-10-03T10:09:47.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.312+0000 7fddc85ed700 1 -- start start 2023-10-03T10:09:47.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.313+0000 7fddc85ed700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fddc0062a90 0x7fddc012daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:47.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.313+0000 7fddc85ed700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fddc0127360 0x7fddc012dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:47.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.313+0000 7fddc85ed700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddc0127f70 0x7fddc012e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:47.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.313+0000 7fddc85ed700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc012f630 con 0x7fddc0127f70 2023-10-03T10:09:47.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.313+0000 7fddc85ed700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc0131260 con 0x7fddc0062a90 2023-10-03T10:09:47.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.313+0000 7fddc85ed700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fddc01111b0 con 0x7fddc0127360 2023-10-03T10:09:47.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.313+0000 7fddc6389700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fddc0062a90 0x7fddc012daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:47.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.313+0000 7fddc6b8a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddc0127f70 0x7fddc012e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:47.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.313+0000 7fddc5b88700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fddc0127360 0x7fddc012dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:47.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.313+0000 7fddc6b8a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddc0127f70 0x7fddc012e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35442/0 (socket says 172.21.15.3:35442) 2023-10-03T10:09:47.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.313+0000 7fddc6389700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fddc0062a90 0x7fddc012daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34556/0 (socket says 172.21.15.3:34556) 2023-10-03T10:09:47.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.313+0000 7fddc6b8a700 1 -- 172.21.15.3:0/1234352396 learned_addr learned my addr 172.21.15.3:0/1234352396 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:47.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.313+0000 7fddc6b8a700 1 -- 172.21.15.3:0/1234352396 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fddc0127360 msgr2=0x7fddc012dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:47.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.313+0000 7fddc6b8a700 1 --2- 172.21.15.3:0/1234352396 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fddc0127360 0x7fddc012dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:47.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.313+0000 7fddc6b8a700 1 -- 172.21.15.3:0/1234352396 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fddc0062a90 msgr2=0x7fddc012daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:47.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.313+0000 7fddc6b8a700 1 --2- 172.21.15.3:0/1234352396 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fddc0062a90 0x7fddc012daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:47.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.313+0000 7fddc6b8a700 1 -- 172.21.15.3:0/1234352396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb8009a20 con 0x7fddc0127f70 2023-10-03T10:09:47.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.313+0000 7fddc6389700 1 --2- 172.21.15.3:0/1234352396 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fddc0062a90 0x7fddc012daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:47.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.313+0000 7fddc5b88700 1 --2- 172.21.15.3:0/1234352396 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fddc0127360 0x7fddc012dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:47.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.313+0000 7fddc6b8a700 1 --2- 172.21.15.3:0/1234352396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddc0127f70 0x7fddc012e4d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fddb8009fc0 tx=0x7fddb800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:47.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.314+0000 7fddb67fc700 1 -- 172.21.15.3:0/1234352396 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8009d20 con 0x7fddc0127f70 2023-10-03T10:09:47.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.314+0000 7fddb67fc700 1 -- 172.21.15.3:0/1234352396 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb800f040 con 0x7fddc0127f70 2023-10-03T10:09:47.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.314+0000 7fddc85ed700 1 -- 172.21.15.3:0/1234352396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb00092a0 con 0x7fddc0127f70 2023-10-03T10:09:47.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.314+0000 7fddb67fc700 1 -- 172.21.15.3:0/1234352396 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb800cde0 con 0x7fddc0127f70 2023-10-03T10:09:47.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.314+0000 7fddc85ed700 1 -- 172.21.15.3:0/1234352396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc0130fa0 con 0x7fddc0127f70 2023-10-03T10:09:47.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.315+0000 7fddb67fc700 1 -- 172.21.15.3:0/1234352396 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fddb8013610 con 0x7fddc0127f70 2023-10-03T10:09:47.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.315+0000 7fddb67fc700 1 -- 172.21.15.3:0/1234352396 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fddb803b870 con 0x7fddc0127f70 2023-10-03T10:09:47.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.315+0000 7fddc85ed700 1 -- 172.21.15.3:0/1234352396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc012f860 con 0x7fddc0127f70 2023-10-03T10:09:47.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.320+0000 7fddb67fc700 1 -- 172.21.15.3:0/1234352396 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fddb801d460 con 0x7fddc0127f70 2023-10-03T10:09:47.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.535+0000 7fddc85ed700 1 -- 172.21.15.3:0/1234352396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fddc005a260 con 0x7fddc0127f70 2023-10-03T10:09:47.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.535+0000 7fddb67fc700 1 -- 172.21.15.3:0/1234352396 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2154 (secure 0 0 0) 0x7fddb802c090 con 0x7fddc0127f70 2023-10-03T10:09:47.536 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:47.536 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-03T10:09:44.141304+0000","modified":"2023-10-03T10:09:47.154287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4186},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4186":{"gid":4186,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6817/2582564779","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6817","nonce":2582564779}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-03T10:09:47.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.537+0000 7fddc85ed700 1 -- 172.21.15.3:0/1234352396 >> v1:172.21.15.3:6816/118296 conn(0x7fdda402bb60 legacy=0x7fdda402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:47.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.537+0000 7fddc85ed700 1 -- 172.21.15.3:0/1234352396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddc0127f70 msgr2=0x7fddc012e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:47.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.537+0000 7fddc85ed700 1 --2- 172.21.15.3:0/1234352396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddc0127f70 0x7fddc012e4d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fddb8009fc0 tx=0x7fddb800b040).stop 2023-10-03T10:09:47.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.537+0000 7fddc85ed700 1 -- 172.21.15.3:0/1234352396 shutdown_connections 2023-10-03T10:09:47.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.537+0000 7fddc85ed700 1 --2- 172.21.15.3:0/1234352396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddc0127f70 0x7fddc012e4d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:47.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.537+0000 7fddc85ed700 1 --2- 172.21.15.3:0/1234352396 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fddc0062a90 0x7fddc012daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:47.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.537+0000 7fddc85ed700 1 --2- 172.21.15.3:0/1234352396 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fddc0127360 0x7fddc012dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:47.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.537+0000 7fddc85ed700 1 -- 172.21.15.3:0/1234352396 >> 172.21.15.3:0/1234352396 conn(0x7fddc0059bb0 msgr2=0x7fddc0126160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:47.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.537+0000 7fddc85ed700 1 -- 172.21.15.3:0/1234352396 shutdown_connections 2023-10-03T10:09:47.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.537+0000 7fddc85ed700 1 -- 172.21.15.3:0/1234352396 wait complete. 2023-10-03T10:09:47.539 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 6 2023-10-03T10:09:47.550 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-03T10:09:47.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.681+0000 7f63f0382700 1 Processor -- start 2023-10-03T10:09:47.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.681+0000 7f63f0382700 1 -- start start 2023-10-03T10:09:47.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.682+0000 7f63f0382700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f63e8062a90 0x7f63e8127860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:47.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.682+0000 7f63f0382700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63e8127360 0x7f63e8127d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:47.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.682+0000 7f63f0382700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63e80968b0 con 0x7f63e8095300 2023-10-03T10:09:47.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.682+0000 7f63f0382700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63e8062e70 con 0x7f63e8062a90 2023-10-03T10:09:47.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.682+0000 7f63f0382700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f63e8091bc0 con 0x7f63e8127360 2023-10-03T10:09:47.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.682+0000 7f63ed91d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63e8127360 0x7f63e8127d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:47.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.682+0000 7f63ee11e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f63e8062a90 0x7f63e8127860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:47.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.682+0000 7f63ed91d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63e8127360 0x7f63e8127d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51036/0 (socket says 172.21.15.3:51036) 2023-10-03T10:09:47.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.682+0000 7f63ed91d700 1 -- 172.21.15.3:0/2751778836 learned_addr learned my addr 172.21.15.3:0/2751778836 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:47.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.682+0000 7f63ee91f700 1 --1- 172.21.15.3:0/2751778836 >> v1:172.21.15.3:6789/0 conn(0x7f63e8095300 0x7f63e80957b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35056/0 (socket says 172.21.15.3:35056) 2023-10-03T10:09:47.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.682+0000 7f63ed91d700 1 -- 172.21.15.3:0/2751778836 >> v2:172.21.15.3:3300/0 conn(0x7f63e8062a90 msgr2=0x7f63e8127860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:47.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.682+0000 7f63ed91d700 1 --2- 172.21.15.3:0/2751778836 >> v2:172.21.15.3:3300/0 conn(0x7f63e8062a90 0x7f63e8127860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:47.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.682+0000 7f63ed91d700 1 -- 172.21.15.3:0/2751778836 >> v1:172.21.15.3:6789/0 conn(0x7f63e8095300 legacy=0x7f63e80957b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:47.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.682+0000 7f63ed91d700 1 -- 172.21.15.3:0/2751778836 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e8092a70 con 0x7f63e8127360 2023-10-03T10:09:47.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.682+0000 7f63ed11c700 1 -- 172.21.15.3:0/2751778836 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1946352174 0 0) 0x7f63e0001490 con 0x7f63e8095300 2023-10-03T10:09:47.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.682+0000 7f63ee11e700 1 --2- 172.21.15.3:0/2751778836 >> v2:172.21.15.3:3300/0 conn(0x7f63e8062a90 0x7f63e8127860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:47.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.682+0000 7f63ed91d700 1 --2- 172.21.15.3:0/2751778836 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63e8127360 0x7f63e8127d50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f63e4009830 tx=0x7f63e40090f0).ready entity=mon.2 client_cookie=a30ee107bf78e60 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:47.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.682+0000 7f63ed11c700 1 -- 172.21.15.3:0/2751778836 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e400b580 con 0x7f63e8127360 2023-10-03T10:09:47.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.683+0000 7f63ed11c700 1 -- 172.21.15.3:0/2751778836 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e400b6e0 con 0x7f63e8127360 2023-10-03T10:09:47.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.683+0000 7f63ed11c700 1 -- 172.21.15.3:0/2751778836 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e400ba80 con 0x7f63e8127360 2023-10-03T10:09:47.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.683+0000 7f63f0382700 1 -- 172.21.15.3:0/2751778836 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63e8127360 msgr2=0x7f63e8127d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:47.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.683+0000 7f63f0382700 1 --2- 172.21.15.3:0/2751778836 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63e8127360 0x7f63e8127d50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f63e4009830 tx=0x7f63e40090f0).stop 2023-10-03T10:09:47.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.683+0000 7f63f0382700 1 -- 172.21.15.3:0/2751778836 shutdown_connections 2023-10-03T10:09:47.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.683+0000 7f63f0382700 1 --2- 172.21.15.3:0/2751778836 >> v2:172.21.15.3:3300/0 conn(0x7f63e8062a90 0x7f63e8127860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:47.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.683+0000 7f63f0382700 1 --2- 172.21.15.3:0/2751778836 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63e8127360 0x7f63e8127d50 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:47.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.683+0000 7f63f0382700 1 -- 172.21.15.3:0/2751778836 >> 172.21.15.3:0/2751778836 conn(0x7f63e8059bb0 msgr2=0x7f63e8097400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:47.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.683+0000 7f63f0382700 1 -- 172.21.15.3:0/2751778836 shutdown_connections 2023-10-03T10:09:47.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.683+0000 7f63f0382700 1 -- 172.21.15.3:0/2751778836 wait complete. 2023-10-03T10:09:47.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.684+0000 7f63f0382700 1 Processor -- start 2023-10-03T10:09:47.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.684+0000 7f63f0382700 1 -- start start 2023-10-03T10:09:47.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.685+0000 7f63f0382700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f63e8062a90 0x7f63e8124b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:47.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.685+0000 7f63f0382700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63e8095300 0x7f63e8113b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:47.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.685+0000 7f63f0382700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63e8127360 0x7f63e8114000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:47.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.685+0000 7f63f0382700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e8117670 con 0x7f63e8127360 2023-10-03T10:09:47.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.685+0000 7f63f0382700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e8096e90 con 0x7f63e8062a90 2023-10-03T10:09:47.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.685+0000 7f63f0382700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f63e8125500 con 0x7f63e8095300 2023-10-03T10:09:47.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.685+0000 7f63ed91d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63e8095300 0x7f63e8113b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:47.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.685+0000 7f63ee91f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63e8127360 0x7f63e8114000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:47.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.685+0000 7f63ee11e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f63e8062a90 0x7f63e8124b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:47.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.685+0000 7f63ed91d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63e8095300 0x7f63e8113b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51040/0 (socket says 172.21.15.3:51040) 2023-10-03T10:09:47.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.685+0000 7f63ed91d700 1 -- 172.21.15.3:0/68263828 learned_addr learned my addr 172.21.15.3:0/68263828 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:47.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.685+0000 7f63ed91d700 1 -- 172.21.15.3:0/68263828 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f63e8062a90 msgr2=0x7f63e8124b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:47.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.685+0000 7f63ed91d700 1 --2- 172.21.15.3:0/68263828 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f63e8062a90 0x7f63e8124b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:47.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.685+0000 7f63ed91d700 1 -- 172.21.15.3:0/68263828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63e8127360 msgr2=0x7f63e8114000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:47.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.685+0000 7f63ed91d700 1 --2- 172.21.15.3:0/68263828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63e8127360 0x7f63e8114000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:47.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.685+0000 7f63ed91d700 1 -- 172.21.15.3:0/68263828 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d80092a0 con 0x7f63e8095300 2023-10-03T10:09:47.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.685+0000 7f63ee91f700 1 --2- 172.21.15.3:0/68263828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63e8127360 0x7f63e8114000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:47.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.685+0000 7f63ee11e700 1 --2- 172.21.15.3:0/68263828 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f63e8062a90 0x7f63e8124b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:47.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.685+0000 7f63ed91d700 1 --2- 172.21.15.3:0/68263828 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63e8095300 0x7f63e8113b10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f63e8091d00 tx=0x7f63e4017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:47.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.685+0000 7f63de7fc700 1 -- 172.21.15.3:0/68263828 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e4006d90 con 0x7f63e8095300 2023-10-03T10:09:47.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.685+0000 7f63de7fc700 1 -- 172.21.15.3:0/68263828 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e400be40 con 0x7f63e8095300 2023-10-03T10:09:47.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.685+0000 7f63f0382700 1 -- 172.21.15.3:0/68263828 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e4009370 con 0x7f63e8095300 2023-10-03T10:09:47.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.685+0000 7f63de7fc700 1 -- 172.21.15.3:0/68263828 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e400ba80 con 0x7f63e8095300 2023-10-03T10:09:47.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.686+0000 7f63f0382700 1 -- 172.21.15.3:0/68263828 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e8114d50 con 0x7f63e8095300 2023-10-03T10:09:47.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.686+0000 7f63de7fc700 1 -- 172.21.15.3:0/68263828 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f63e401d640 con 0x7f63e8095300 2023-10-03T10:09:47.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.686+0000 7f63de7fc700 1 -- 172.21.15.3:0/68263828 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f63e403c9d0 con 0x7f63e8095300 2023-10-03T10:09:47.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.687+0000 7f63f0382700 1 -- 172.21.15.3:0/68263828 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e8117180 con 0x7f63e8095300 2023-10-03T10:09:47.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.691+0000 7f63de7fc700 1 -- 172.21.15.3:0/68263828 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f63e8117180 con 0x7f63e8095300 2023-10-03T10:09:47.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.867+0000 7f63f0382700 1 -- 172.21.15.3:0/68263828 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63e8117180 con 0x7f63e8095300 2023-10-03T10:09:47.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.868+0000 7f63de7fc700 1 -- 172.21.15.3:0/68263828 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+11309 (secure 0 0 0) 0x7f63e8117180 con 0x7f63e8095300 2023-10-03T10:09:47.869 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:47.869 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":17,"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","created":"2023-10-03T10:09:28.703441+0000","modified":"2023-10-03T10:09:45.145435+0000","last_up_change":"2023-10-03T10:09:36.569401+0000","last_in_change":"2023-10-03T10:09:30.156546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-10-03T10:09:36.623971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-03T10:09:38.210614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-03T10:09:42.695873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-03T10:09:43.115588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cdcdd930-3c5b-4486-a021-7132f90fb6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6808","nonce":118845}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6809","nonce":118845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6811","nonce":118845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6810","nonce":118845}]},"public_addr":"172.21.15.3:6808/118845","cluster_addr":"172.21.15.3:6809/118845","heartbeat_back_addr":"172.21.15.3:6811/118845","heartbeat_front_addr":"172.21.15.3:6810/118845","state":["exists","up"]},{"osd":1,"uuid":"c8e58877-6762-4741-a8c1-567ebc97b708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6804","nonce":118835}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6805","nonce":118835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6807","nonce":118835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6806","nonce":118835}]},"public_addr":"172.21.15.3:6804/118835","cluster_addr":"172.21.15.3:6805/118835","heartbeat_back_addr":"172.21.15.3:6807/118835","heartbeat_front_addr":"172.21.15.3:6806/118835","state":["exists","up"]},{"osd":2,"uuid":"68a2a21a-7a8e-4f82-8433-5201faa50777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6812","nonce":118864}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6813","nonce":118864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6815","nonce":118864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6814","nonce":118864}]},"public_addr":"172.21.15.3:6812/118864","cluster_addr":"172.21.15.3:6813/118864","heartbeat_back_addr":"172.21.15.3:6815/118864","heartbeat_front_addr":"172.21.15.3:6814/118864","state":["exists","up"]},{"osd":3,"uuid":"c4df2903-4e1e-4208-bf77-95b9fa749241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6800","nonce":118836}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6801","nonce":118836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6803","nonce":118836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6802","nonce":118836}]},"public_addr":"172.21.15.3:6800/118836","cluster_addr":"172.21.15.3:6801/118836","heartbeat_back_addr":"172.21.15.3:6803/118836","heartbeat_front_addr":"172.21.15.3:6802/118836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.369708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.257825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.337931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.085222+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-03T10:09:47.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.870+0000 7f63f0382700 1 -- 172.21.15.3:0/68263828 >> v1:172.21.15.3:6816/118296 conn(0x7f63cc02bb90 legacy=0x7f63cc02e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:47.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.870+0000 7f63f0382700 1 -- 172.21.15.3:0/68263828 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63e8095300 msgr2=0x7f63e8113b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:47.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.870+0000 7f63f0382700 1 --2- 172.21.15.3:0/68263828 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63e8095300 0x7f63e8113b10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f63e8091d00 tx=0x7f63e4017000).stop 2023-10-03T10:09:47.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.870+0000 7f63f0382700 1 -- 172.21.15.3:0/68263828 shutdown_connections 2023-10-03T10:09:47.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.870+0000 7f63f0382700 1 --2- 172.21.15.3:0/68263828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63e8127360 0x7f63e8114000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:47.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.870+0000 7f63f0382700 1 --2- 172.21.15.3:0/68263828 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f63e8062a90 0x7f63e8124b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:47.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.870+0000 7f63f0382700 1 --2- 172.21.15.3:0/68263828 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63e8095300 0x7f63e8113b10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:47.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.870+0000 7f63f0382700 1 -- 172.21.15.3:0/68263828 >> 172.21.15.3:0/68263828 conn(0x7f63e8059bb0 msgr2=0x7f63e80946a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:47.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.870+0000 7f63f0382700 1 -- 172.21.15.3:0/68263828 shutdown_connections 2023-10-03T10:09:47.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:47.870+0000 7f63f0382700 1 -- 172.21.15.3:0/68263828 wait complete. 2023-10-03T10:09:47.884 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-03T10:09:47.884 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-03T10:09:48.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.013+0000 7fb8e236a700 1 Processor -- start 2023-10-03T10:09:48.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.013+0000 7fb8e236a700 1 -- start start 2023-10-03T10:09:48.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.014+0000 7fb8e236a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8dc0943a0 0x7fb8dc126e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:48.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.014+0000 7fb8e236a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb8dc12c1a0 0x7fb8dc12e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:48.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.014+0000 7fb8e236a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8dc093970 con 0x7fb8dc1273c0 2023-10-03T10:09:48.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.014+0000 7fb8e236a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8dc094780 con 0x7fb8dc12c1a0 2023-10-03T10:09:48.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.014+0000 7fb8e236a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc093ba0 con 0x7fb8dc0943a0 2023-10-03T10:09:48.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.014+0000 7fb8db7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb8dc1273c0 0x7fb8dc1277c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35062/0 (socket says 172.21.15.3:35062) 2023-10-03T10:09:48.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.014+0000 7fb8dbfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8dc0943a0 0x7fb8dc126e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:48.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.014+0000 7fb8db7fe700 1 -- 172.21.15.3:0/499332629 learned_addr learned my addr 172.21.15.3:0/499332629 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:09:48.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.014+0000 7fb8e0907700 1 --2- 172.21.15.3:0/499332629 >> v2:172.21.15.3:3300/0 conn(0x7fb8dc12c1a0 0x7fb8dc12e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:48.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.014+0000 7fb8daffd700 1 -- 172.21.15.3:0/499332629 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4273086302 0 0) 0x7fb8dc093970 con 0x7fb8dc1273c0 2023-10-03T10:09:48.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.014+0000 7fb8daffd700 1 -- 172.21.15.3:0/499332629 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8c80033f0 con 0x7fb8dc1273c0 2023-10-03T10:09:48.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.014+0000 7fb8dbfff700 1 -- 172.21.15.3:0/499332629 >> v2:172.21.15.3:3300/0 conn(0x7fb8dc12c1a0 msgr2=0x7fb8dc12e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:48.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.014+0000 7fb8dbfff700 1 --2- 172.21.15.3:0/499332629 >> v2:172.21.15.3:3300/0 conn(0x7fb8dc12c1a0 0x7fb8dc12e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:48.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.014+0000 7fb8dbfff700 1 -- 172.21.15.3:0/499332629 >> v1:172.21.15.3:6789/0 conn(0x7fb8dc1273c0 legacy=0x7fb8dc1277c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:48.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.014+0000 7fb8dbfff700 1 -- 172.21.15.3:0/499332629 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8dc093f10 con 0x7fb8dc0943a0 2023-10-03T10:09:48.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.014+0000 7fb8e0907700 1 --2- 172.21.15.3:0/499332629 >> v2:172.21.15.3:3300/0 conn(0x7fb8dc12c1a0 0x7fb8dc12e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:48.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.014+0000 7fb8dbfff700 1 --2- 172.21.15.3:0/499332629 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8dc0943a0 0x7fb8dc126e10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fb8c4009760 tx=0x7fb8c4009060).ready entity=mon.1 client_cookie=1f0b5c9849ab2f14 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:48.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.014+0000 7fb8daffd700 1 -- 172.21.15.3:0/499332629 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c40090d0 con 0x7fb8dc0943a0 2023-10-03T10:09:48.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.015+0000 7fb8daffd700 1 -- 172.21.15.3:0/499332629 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c400d4a0 con 0x7fb8dc0943a0 2023-10-03T10:09:48.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.015+0000 7fb8daffd700 1 -- 172.21.15.3:0/499332629 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c400d820 con 0x7fb8dc0943a0 2023-10-03T10:09:48.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.015+0000 7fb8e236a700 1 -- 172.21.15.3:0/499332629 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8dc0943a0 msgr2=0x7fb8dc126e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:48.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.015+0000 7fb8e236a700 1 --2- 172.21.15.3:0/499332629 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8dc0943a0 0x7fb8dc126e10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fb8c4009760 tx=0x7fb8c4009060).stop 2023-10-03T10:09:48.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.015+0000 7fb8e236a700 1 -- 172.21.15.3:0/499332629 shutdown_connections 2023-10-03T10:09:48.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.015+0000 7fb8e236a700 1 --2- 172.21.15.3:0/499332629 >> v2:172.21.15.3:3300/0 conn(0x7fb8dc12c1a0 0x7fb8dc12e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:48.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.015+0000 7fb8e236a700 1 --2- 172.21.15.3:0/499332629 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8dc0943a0 0x7fb8dc126e10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:48.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.015+0000 7fb8e236a700 1 -- 172.21.15.3:0/499332629 >> 172.21.15.3:0/499332629 conn(0x7fb8dc063190 msgr2=0x7fb8dc09a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:48.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.015+0000 7fb8e236a700 1 -- 172.21.15.3:0/499332629 shutdown_connections 2023-10-03T10:09:48.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.015+0000 7fb8e236a700 1 -- 172.21.15.3:0/499332629 wait complete. 2023-10-03T10:09:48.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.016+0000 7fb8e236a700 1 Processor -- start 2023-10-03T10:09:48.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.016+0000 7fb8e236a700 1 -- start start 2023-10-03T10:09:48.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.017+0000 7fb8e236a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb8dc0943a0 0x7fb8dc1387d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:48.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.017+0000 7fb8e236a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8dc1273c0 0x7fb8dc138cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:48.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.017+0000 7fb8e236a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8dc12c1a0 0x7fb8dc110a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:48.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.017+0000 7fb8e236a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc12c580 con 0x7fb8dc1273c0 2023-10-03T10:09:48.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.017+0000 7fb8e236a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc111f00 con 0x7fb8dc12c1a0 2023-10-03T10:09:48.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.017+0000 7fb8e236a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc113e50 con 0x7fb8dc0943a0 2023-10-03T10:09:48.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.017+0000 7fb8dbfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb8dc0943a0 0x7fb8dc1387d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:48.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.017+0000 7fb8db7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8dc1273c0 0x7fb8dc138cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:48.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.017+0000 7fb8dbfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb8dc0943a0 0x7fb8dc1387d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51054/0 (socket says 172.21.15.3:51054) 2023-10-03T10:09:48.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.017+0000 7fb8dbfff700 1 -- 172.21.15.3:0/467271174 learned_addr learned my addr 172.21.15.3:0/467271174 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:48.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.017+0000 7fb8e0907700 1 --2- 172.21.15.3:0/467271174 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8dc12c1a0 0x7fb8dc110a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:48.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.017+0000 7fb8dbfff700 1 -- 172.21.15.3:0/467271174 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8dc12c1a0 msgr2=0x7fb8dc110a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:48.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.017+0000 7fb8dbfff700 1 --2- 172.21.15.3:0/467271174 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8dc12c1a0 0x7fb8dc110a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:48.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.017+0000 7fb8dbfff700 1 -- 172.21.15.3:0/467271174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8dc1273c0 msgr2=0x7fb8dc138cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:48.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.017+0000 7fb8dbfff700 1 --2- 172.21.15.3:0/467271174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8dc1273c0 0x7fb8dc138cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:48.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.017+0000 7fb8dbfff700 1 -- 172.21.15.3:0/467271174 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d4009a20 con 0x7fb8dc0943a0 2023-10-03T10:09:48.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.017+0000 7fb8e0907700 1 --2- 172.21.15.3:0/467271174 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8dc12c1a0 0x7fb8dc110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:48.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.017+0000 7fb8dbfff700 1 --2- 172.21.15.3:0/467271174 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb8dc0943a0 0x7fb8dc1387d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fb8c400d940 tx=0x7fb8c4016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:48.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.018+0000 7fb8d3fff700 1 -- 172.21.15.3:0/467271174 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4020070 con 0x7fb8dc0943a0 2023-10-03T10:09:48.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.018+0000 7fb8d3fff700 1 -- 172.21.15.3:0/467271174 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c4006990 con 0x7fb8dc0943a0 2023-10-03T10:09:48.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.018+0000 7fb8e236a700 1 -- 172.21.15.3:0/467271174 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c40092a0 con 0x7fb8dc0943a0 2023-10-03T10:09:48.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.018+0000 7fb8d3fff700 1 -- 172.21.15.3:0/467271174 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c401be20 con 0x7fb8dc0943a0 2023-10-03T10:09:48.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.018+0000 7fb8e236a700 1 -- 172.21.15.3:0/467271174 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8dc0924d0 con 0x7fb8dc0943a0 2023-10-03T10:09:48.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.018+0000 7fb8d3fff700 1 -- 172.21.15.3:0/467271174 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fb8c4025670 con 0x7fb8dc0943a0 2023-10-03T10:09:48.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.018+0000 7fb8d3fff700 1 -- 172.21.15.3:0/467271174 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fb8c403fc20 con 0x7fb8dc0943a0 2023-10-03T10:09:48.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.019+0000 7fb8e236a700 1 -- 172.21.15.3:0/467271174 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8dc092e80 con 0x7fb8dc0943a0 2023-10-03T10:09:48.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.023+0000 7fb8d3fff700 1 -- 172.21.15.3:0/467271174 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fb8c402a070 con 0x7fb8dc0943a0 2023-10-03T10:09:48.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.196+0000 7fb8e236a700 1 -- 172.21.15.3:0/467271174 --> v1:172.21.15.3:6816/118296 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb8dc05a260 con 0x7fb8bc02bb30 2023-10-03T10:09:48.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.203+0000 7fb8d3fff700 1 -- 172.21.15.3:0/467271174 <== mgr.4100 v1:172.21.15.3:6816/118296 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248601 (unknown 2965378022 0 4201081181) 0x7fb8dc05a260 con 0x7fb8bc02bb30 2023-10-03T10:09:48.207 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:48.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.206+0000 7fb8e236a700 1 -- 172.21.15.3:0/467271174 >> v1:172.21.15.3:6816/118296 conn(0x7fb8bc02bb30 legacy=0x7fb8bc02dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:48.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.206+0000 7fb8e236a700 1 -- 172.21.15.3:0/467271174 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb8dc0943a0 msgr2=0x7fb8dc1387d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:48.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.206+0000 7fb8e236a700 1 --2- 172.21.15.3:0/467271174 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb8dc0943a0 0x7fb8dc1387d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fb8c400d940 tx=0x7fb8c4016000).stop 2023-10-03T10:09:48.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.207+0000 7fb8e236a700 1 -- 172.21.15.3:0/467271174 shutdown_connections 2023-10-03T10:09:48.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.207+0000 7fb8e236a700 1 --2- 172.21.15.3:0/467271174 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8dc12c1a0 0x7fb8dc110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:48.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.207+0000 7fb8e236a700 1 --2- 172.21.15.3:0/467271174 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb8dc0943a0 0x7fb8dc1387d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:48.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.207+0000 7fb8e236a700 1 --2- 172.21.15.3:0/467271174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8dc1273c0 0x7fb8dc138cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:48.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.207+0000 7fb8e236a700 1 -- 172.21.15.3:0/467271174 >> 172.21.15.3:0/467271174 conn(0x7fb8dc063190 msgr2=0x7fb8dc12a950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:48.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.207+0000 7fb8e236a700 1 -- 172.21.15.3:0/467271174 shutdown_connections 2023-10-03T10:09:48.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.207+0000 7fb8e236a700 1 -- 172.21.15.3:0/467271174 wait complete. 2023-10-03T10:09:48.210 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-03T10:09:48.220 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-10-03T10:09:47.529155+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":178,"acting":178,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1188972,"kb_used_data":588,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298388,"statfs":{"total":386547056640,"available":385329549312,"internally_reserved":0,"allocated":602112,"data_stored":171509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.190395+0000","last_change":"2023-10-03T10:09:44.183364+0000","last_active":"2023-10-03T10:09:44.190395+0000","last_peered":"2023-10-03T10:09:44.190395+0000","last_clean":"2023-10-03T10:09:44.190395+0000","last_became_active":"2023-10-03T10:09:44.183094+0000","last_became_peered":"2023-10-03T10:09:44.183094+0000","last_unstale":"2023-10-03T10:09:44.190395+0000","last_undegraded":"2023-10-03T10:09:44.190395+0000","last_fullsized":"2023-10-03T10:09:44.190395+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.190339+0000","last_change":"2023-10-03T10:09:44.183273+0000","last_active":"2023-10-03T10:09:44.190339+0000","last_peered":"2023-10-03T10:09:44.190339+0000","last_clean":"2023-10-03T10:09:44.190339+0000","last_became_active":"2023-10-03T10:09:44.182451+0000","last_became_peered":"2023-10-03T10:09:44.182451+0000","last_unstale":"2023-10-03T10:09:44.190339+0000","last_undegraded":"2023-10-03T10:09:44.190339+0000","last_fullsized":"2023-10-03T10:09:44.190339+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.174191+0000","last_change":"2023-10-03T10:09:44.168483+0000","last_active":"2023-10-03T10:09:44.174191+0000","last_peered":"2023-10-03T10:09:44.174191+0000","last_clean":"2023-10-03T10:09:44.174191+0000","last_became_active":"2023-10-03T10:09:44.168186+0000","last_became_peered":"2023-10-03T10:09:44.168186+0000","last_unstale":"2023-10-03T10:09:44.174191+0000","last_undegraded":"2023-10-03T10:09:44.174191+0000","last_fullsized":"2023-10-03T10:09:44.174191+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.198209+0000","last_change":"2023-10-03T10:09:44.192967+0000","last_active":"2023-10-03T10:09:44.198209+0000","last_peered":"2023-10-03T10:09:44.198209+0000","last_clean":"2023-10-03T10:09:44.198209+0000","last_became_active":"2023-10-03T10:09:44.190979+0000","last_became_peered":"2023-10-03T10:09:44.190979+0000","last_unstale":"2023-10-03T10:09:44.198209+0000","last_undegraded":"2023-10-03T10:09:44.198209+0000","last_fullsized":"2023-10-03T10:09:44.198209+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.192360+0000","last_change":"2023-10-03T10:09:44.183121+0000","last_active":"2023-10-03T10:09:44.192360+0000","last_peered":"2023-10-03T10:09:44.192360+0000","last_clean":"2023-10-03T10:09:44.192360+0000","last_became_active":"2023-10-03T10:09:44.182647+0000","last_became_peered":"2023-10-03T10:09:44.182647+0000","last_unstale":"2023-10-03T10:09:44.192360+0000","last_undegraded":"2023-10-03T10:09:44.192360+0000","last_fullsized":"2023-10-03T10:09:44.192360+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.187765+0000","last_change":"2023-10-03T10:09:44.173123+0000","last_active":"2023-10-03T10:09:44.187765+0000","last_peered":"2023-10-03T10:09:44.187765+0000","last_clean":"2023-10-03T10:09:44.187765+0000","last_became_active":"2023-10-03T10:09:44.172353+0000","last_became_peered":"2023-10-03T10:09:44.172353+0000","last_unstale":"2023-10-03T10:09:44.187765+0000","last_undegraded":"2023-10-03T10:09:44.187765+0000","last_fullsized":"2023-10-03T10:09:44.187765+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173973+0000","last_change":"2023-10-03T10:09:44.166758+0000","last_active":"2023-10-03T10:09:44.173973+0000","last_peered":"2023-10-03T10:09:44.173973+0000","last_clean":"2023-10-03T10:09:44.173973+0000","last_became_active":"2023-10-03T10:09:44.166415+0000","last_became_peered":"2023-10-03T10:09:44.166415+0000","last_unstale":"2023-10-03T10:09:44.173973+0000","last_undegraded":"2023-10-03T10:09:44.173973+0000","last_fullsized":"2023-10-03T10:09:44.173973+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180446+0000","last_change":"2023-10-03T10:09:44.176833+0000","last_active":"2023-10-03T10:09:44.180446+0000","last_peered":"2023-10-03T10:09:44.180446+0000","last_clean":"2023-10-03T10:09:44.180446+0000","last_became_active":"2023-10-03T10:09:44.176526+0000","last_became_peered":"2023-10-03T10:09:44.176526+0000","last_unstale":"2023-10-03T10:09:44.180446+0000","last_undegraded":"2023-10-03T10:09:44.180446+0000","last_fullsized":"2023-10-03T10:09:44.180446+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180114+0000","last_change":"2023-10-03T10:09:44.175868+0000","last_active":"2023-10-03T10:09:44.180114+0000","last_peered":"2023-10-03T10:09:44.180114+0000","last_clean":"2023-10-03T10:09:44.180114+0000","last_became_active":"2023-10-03T10:09:44.175414+0000","last_became_peered":"2023-10-03T10:09:44.175414+0000","last_unstale":"2023-10-03T10:09:44.180114+0000","last_undegraded":"2023-10-03T10:09:44.180114+0000","last_fullsized":"2023-10-03T10:09:44.180114+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180522+0000","last_change":"2023-10-03T10:09:44.176456+0000","last_active":"2023-10-03T10:09:44.180522+0000","last_peered":"2023-10-03T10:09:44.180522+0000","last_clean":"2023-10-03T10:09:44.180522+0000","last_became_active":"2023-10-03T10:09:44.176146+0000","last_became_peered":"2023-10-03T10:09:44.176146+0000","last_unstale":"2023-10-03T10:09:44.180522+0000","last_undegraded":"2023-10-03T10:09:44.180522+0000","last_fullsized":"2023-10-03T10:09:44.180522+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.178380+0000","last_change":"2023-10-03T10:09:44.176614+0000","last_active":"2023-10-03T10:09:44.178380+0000","last_peered":"2023-10-03T10:09:44.178380+0000","last_clean":"2023-10-03T10:09:44.178380+0000","last_became_active":"2023-10-03T10:09:44.176334+0000","last_became_peered":"2023-10-03T10:09:44.176334+0000","last_unstale":"2023-10-03T10:09:44.178380+0000","last_undegraded":"2023-10-03T10:09:44.178380+0000","last_fullsized":"2023-10-03T10:09:44.178380+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.192741+0000","last_change":"2023-10-03T10:09:44.179440+0000","last_active":"2023-10-03T10:09:44.192741+0000","last_peered":"2023-10-03T10:09:44.192741+0000","last_clean":"2023-10-03T10:09:44.192741+0000","last_became_active":"2023-10-03T10:09:44.179150+0000","last_became_peered":"2023-10-03T10:09:44.179150+0000","last_unstale":"2023-10-03T10:09:44.192741+0000","last_undegraded":"2023-10-03T10:09:44.192741+0000","last_fullsized":"2023-10-03T10:09:44.192741+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173996+0000","last_change":"2023-10-03T10:09:44.166901+0000","last_active":"2023-10-03T10:09:44.173996+0000","last_peered":"2023-10-03T10:09:44.173996+0000","last_clean":"2023-10-03T10:09:44.173996+0000","last_became_active":"2023-10-03T10:09:44.166610+0000","last_became_peered":"2023-10-03T10:09:44.166610+0000","last_unstale":"2023-10-03T10:09:44.173996+0000","last_undegraded":"2023-10-03T10:09:44.173996+0000","last_fullsized":"2023-10-03T10:09:44.173996+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173643+0000","last_change":"2023-10-03T10:09:44.165366+0000","last_active":"2023-10-03T10:09:44.173643+0000","last_peered":"2023-10-03T10:09:44.173643+0000","last_clean":"2023-10-03T10:09:44.173643+0000","last_became_active":"2023-10-03T10:09:44.165083+0000","last_became_peered":"2023-10-03T10:09:44.165083+0000","last_unstale":"2023-10-03T10:09:44.173643+0000","last_undegraded":"2023-10-03T10:09:44.173643+0000","last_fullsized":"2023-10-03T10:09:44.173643+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184209+0000","last_change":"2023-10-03T10:09:44.177178+0000","last_active":"2023-10-03T10:09:44.184209+0000","last_peered":"2023-10-03T10:09:44.184209+0000","last_clean":"2023-10-03T10:09:44.184209+0000","last_became_active":"2023-10-03T10:09:44.176686+0000","last_became_peered":"2023-10-03T10:09:44.176686+0000","last_unstale":"2023-10-03T10:09:44.184209+0000","last_undegraded":"2023-10-03T10:09:44.184209+0000","last_fullsized":"2023-10-03T10:09:44.184209+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180396+0000","last_change":"2023-10-03T10:09:44.177335+0000","last_active":"2023-10-03T10:09:44.180396+0000","last_peered":"2023-10-03T10:09:44.180396+0000","last_clean":"2023-10-03T10:09:44.180396+0000","last_became_active":"2023-10-03T10:09:44.175202+0000","last_became_peered":"2023-10-03T10:09:44.175202+0000","last_unstale":"2023-10-03T10:09:44.180396+0000","last_undegraded":"2023-10-03T10:09:44.180396+0000","last_fullsized":"2023-10-03T10:09:44.180396+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184702+0000","last_change":"2023-10-03T10:09:44.182207+0000","last_active":"2023-10-03T10:09:44.184702+0000","last_peered":"2023-10-03T10:09:44.184702+0000","last_clean":"2023-10-03T10:09:44.184702+0000","last_became_active":"2023-10-03T10:09:44.181944+0000","last_became_peered":"2023-10-03T10:09:44.181944+0000","last_unstale":"2023-10-03T10:09:44.184702+0000","last_undegraded":"2023-10-03T10:09:44.184702+0000","last_fullsized":"2023-10-03T10:09:44.184702+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.192336+0000","last_change":"2023-10-03T10:09:44.182610+0000","last_active":"2023-10-03T10:09:44.192336+0000","last_peered":"2023-10-03T10:09:44.192336+0000","last_clean":"2023-10-03T10:09:44.192336+0000","last_became_active":"2023-10-03T10:09:44.180732+0000","last_became_peered":"2023-10-03T10:09:44.180732+0000","last_unstale":"2023-10-03T10:09:44.192336+0000","last_undegraded":"2023-10-03T10:09:44.192336+0000","last_fullsized":"2023-10-03T10:09:44.192336+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.187788+0000","last_change":"2023-10-03T10:09:44.173182+0000","last_active":"2023-10-03T10:09:44.187788+0000","last_peered":"2023-10-03T10:09:44.187788+0000","last_clean":"2023-10-03T10:09:44.187788+0000","last_became_active":"2023-10-03T10:09:44.172685+0000","last_became_peered":"2023-10-03T10:09:44.172685+0000","last_unstale":"2023-10-03T10:09:44.187788+0000","last_undegraded":"2023-10-03T10:09:44.187788+0000","last_fullsized":"2023-10-03T10:09:44.187788+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180761+0000","last_change":"2023-10-03T10:09:44.177910+0000","last_active":"2023-10-03T10:09:44.180761+0000","last_peered":"2023-10-03T10:09:44.180761+0000","last_clean":"2023-10-03T10:09:44.180761+0000","last_became_active":"2023-10-03T10:09:44.177618+0000","last_became_peered":"2023-10-03T10:09:44.177618+0000","last_unstale":"2023-10-03T10:09:44.180761+0000","last_undegraded":"2023-10-03T10:09:44.180761+0000","last_fullsized":"2023-10-03T10:09:44.180761+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180424+0000","last_change":"2023-10-03T10:09:44.176287+0000","last_active":"2023-10-03T10:09:44.180424+0000","last_peered":"2023-10-03T10:09:44.180424+0000","last_clean":"2023-10-03T10:09:44.180424+0000","last_became_active":"2023-10-03T10:09:44.175320+0000","last_became_peered":"2023-10-03T10:09:44.175320+0000","last_unstale":"2023-10-03T10:09:44.180424+0000","last_undegraded":"2023-10-03T10:09:44.180424+0000","last_fullsized":"2023-10-03T10:09:44.180424+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.194745+0000","last_change":"2023-10-03T10:09:44.187148+0000","last_active":"2023-10-03T10:09:44.194745+0000","last_peered":"2023-10-03T10:09:44.194745+0000","last_clean":"2023-10-03T10:09:44.194745+0000","last_became_active":"2023-10-03T10:09:44.186523+0000","last_became_peered":"2023-10-03T10:09:44.186523+0000","last_unstale":"2023-10-03T10:09:44.194745+0000","last_undegraded":"2023-10-03T10:09:44.194745+0000","last_fullsized":"2023-10-03T10:09:44.194745+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.189715+0000","last_change":"2023-10-03T10:09:43.745046+0000","last_active":"2023-10-03T10:09:44.189715+0000","last_peered":"2023-10-03T10:09:44.189715+0000","last_clean":"2023-10-03T10:09:44.189715+0000","last_became_active":"2023-10-03T10:09:43.744724+0000","last_became_peered":"2023-10-03T10:09:43.744724+0000","last_unstale":"2023-10-03T10:09:44.189715+0000","last_undegraded":"2023-10-03T10:09:44.189715+0000","last_fullsized":"2023-10-03T10:09:44.189715+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195809+0000","last_change":"2023-10-03T10:09:44.190869+0000","last_active":"2023-10-03T10:09:44.195809+0000","last_peered":"2023-10-03T10:09:44.195809+0000","last_clean":"2023-10-03T10:09:44.195809+0000","last_became_active":"2023-10-03T10:09:44.190207+0000","last_became_peered":"2023-10-03T10:09:44.190207+0000","last_unstale":"2023-10-03T10:09:44.195809+0000","last_undegraded":"2023-10-03T10:09:44.195809+0000","last_fullsized":"2023-10-03T10:09:44.195809+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.152860+0000","last_change":"2023-10-03T10:09:43.750891+0000","last_active":"2023-10-03T10:09:44.152860+0000","last_peered":"2023-10-03T10:09:44.152860+0000","last_clean":"2023-10-03T10:09:44.152860+0000","last_became_active":"2023-10-03T10:09:43.750356+0000","last_became_peered":"2023-10-03T10:09:43.750356+0000","last_unstale":"2023-10-03T10:09:44.152860+0000","last_undegraded":"2023-10-03T10:09:44.152860+0000","last_fullsized":"2023-10-03T10:09:44.152860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184645+0000","last_change":"2023-10-03T10:09:44.183145+0000","last_active":"2023-10-03T10:09:44.184645+0000","last_peered":"2023-10-03T10:09:44.184645+0000","last_clean":"2023-10-03T10:09:44.184645+0000","last_became_active":"2023-10-03T10:09:44.182816+0000","last_became_peered":"2023-10-03T10:09:44.182816+0000","last_unstale":"2023-10-03T10:09:44.184645+0000","last_undegraded":"2023-10-03T10:09:44.184645+0000","last_fullsized":"2023-10-03T10:09:44.184645+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.163680+0000","last_change":"2023-10-03T10:09:43.749289+0000","last_active":"2023-10-03T10:09:44.163680+0000","last_peered":"2023-10-03T10:09:44.163680+0000","last_clean":"2023-10-03T10:09:44.163680+0000","last_became_active":"2023-10-03T10:09:43.748981+0000","last_became_peered":"2023-10-03T10:09:43.748981+0000","last_unstale":"2023-10-03T10:09:44.163680+0000","last_undegraded":"2023-10-03T10:09:44.163680+0000","last_fullsized":"2023-10-03T10:09:44.163680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.197002+0000","last_change":"2023-10-03T10:09:44.192692+0000","last_active":"2023-10-03T10:09:44.197002+0000","last_peered":"2023-10-03T10:09:44.197002+0000","last_clean":"2023-10-03T10:09:44.197002+0000","last_became_active":"2023-10-03T10:09:44.192390+0000","last_became_peered":"2023-10-03T10:09:44.192390+0000","last_unstale":"2023-10-03T10:09:44.197002+0000","last_undegraded":"2023-10-03T10:09:44.197002+0000","last_fullsized":"2023-10-03T10:09:44.197002+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.160877+0000","last_change":"2023-10-03T10:09:43.754802+0000","last_active":"2023-10-03T10:09:44.160877+0000","last_peered":"2023-10-03T10:09:44.160877+0000","last_clean":"2023-10-03T10:09:44.160877+0000","last_became_active":"2023-10-03T10:09:43.754512+0000","last_became_peered":"2023-10-03T10:09:43.754512+0000","last_unstale":"2023-10-03T10:09:44.160877+0000","last_undegraded":"2023-10-03T10:09:44.160877+0000","last_fullsized":"2023-10-03T10:09:44.160877+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196969+0000","last_change":"2023-10-03T10:09:44.192798+0000","last_active":"2023-10-03T10:09:44.196969+0000","last_peered":"2023-10-03T10:09:44.196969+0000","last_clean":"2023-10-03T10:09:44.196969+0000","last_became_active":"2023-10-03T10:09:44.190787+0000","last_became_peered":"2023-10-03T10:09:44.190787+0000","last_unstale":"2023-10-03T10:09:44.196969+0000","last_undegraded":"2023-10-03T10:09:44.196969+0000","last_fullsized":"2023-10-03T10:09:44.196969+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:42.718400+0000","last_change":"2023-10-03T10:09:42.718400+0000","last_active":"2023-10-03T10:09:42.718400+0000","last_peered":"2023-10-03T10:09:42.718400+0000","last_clean":"2023-10-03T10:09:42.718400+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:42.718400+0000","last_undegraded":"2023-10-03T10:09:42.718400+0000","last_fullsized":"2023-10-03T10:09:42.718400+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.194551+0000","last_change":"2023-10-03T10:09:44.184790+0000","last_active":"2023-10-03T10:09:44.194551+0000","last_peered":"2023-10-03T10:09:44.194551+0000","last_clean":"2023-10-03T10:09:44.194551+0000","last_became_active":"2023-10-03T10:09:44.184531+0000","last_became_peered":"2023-10-03T10:09:44.184531+0000","last_unstale":"2023-10-03T10:09:44.194551+0000","last_undegraded":"2023-10-03T10:09:44.194551+0000","last_fullsized":"2023-10-03T10:09:44.194551+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"16'3","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200426+0000","last_change":"2023-10-03T10:09:43.756598+0000","last_active":"2023-10-03T10:09:44.200426+0000","last_peered":"2023-10-03T10:09:44.200426+0000","last_clean":"2023-10-03T10:09:44.200426+0000","last_became_active":"2023-10-03T10:09:43.754974+0000","last_became_peered":"2023-10-03T10:09:43.754974+0000","last_unstale":"2023-10-03T10:09:44.200426+0000","last_undegraded":"2023-10-03T10:09:44.200426+0000","last_fullsized":"2023-10-03T10:09:44.200426+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180499+0000","last_change":"2023-10-03T10:09:44.175504+0000","last_active":"2023-10-03T10:09:44.180499+0000","last_peered":"2023-10-03T10:09:44.180499+0000","last_clean":"2023-10-03T10:09:44.180499+0000","last_became_active":"2023-10-03T10:09:44.175174+0000","last_became_peered":"2023-10-03T10:09:44.175174+0000","last_unstale":"2023-10-03T10:09:44.180499+0000","last_undegraded":"2023-10-03T10:09:44.180499+0000","last_fullsized":"2023-10-03T10:09:44.180499+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"16'6","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.203073+0000","last_change":"2023-10-03T10:09:43.758711+0000","last_active":"2023-10-03T10:09:44.203073+0000","last_peered":"2023-10-03T10:09:44.203073+0000","last_clean":"2023-10-03T10:09:44.203073+0000","last_became_active":"2023-10-03T10:09:43.758392+0000","last_became_peered":"2023-10-03T10:09:43.758392+0000","last_unstale":"2023-10-03T10:09:44.203073+0000","last_undegraded":"2023-10-03T10:09:44.203073+0000","last_fullsized":"2023-10-03T10:09:44.203073+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200762+0000","last_change":"2023-10-03T10:09:43.751877+0000","last_active":"2023-10-03T10:09:44.200762+0000","last_peered":"2023-10-03T10:09:44.200762+0000","last_clean":"2023-10-03T10:09:44.200762+0000","last_became_active":"2023-10-03T10:09:43.749870+0000","last_became_peered":"2023-10-03T10:09:43.749870+0000","last_unstale":"2023-10-03T10:09:44.200762+0000","last_undegraded":"2023-10-03T10:09:44.200762+0000","last_fullsized":"2023-10-03T10:09:44.200762+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195297+0000","last_change":"2023-10-03T10:09:44.190230+0000","last_active":"2023-10-03T10:09:44.195297+0000","last_peered":"2023-10-03T10:09:44.195297+0000","last_clean":"2023-10-03T10:09:44.195297+0000","last_became_active":"2023-10-03T10:09:44.189342+0000","last_became_peered":"2023-10-03T10:09:44.189342+0000","last_unstale":"2023-10-03T10:09:44.195297+0000","last_undegraded":"2023-10-03T10:09:44.195297+0000","last_fullsized":"2023-10-03T10:09:44.195297+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.162475+0000","last_change":"2023-10-03T10:09:43.755271+0000","last_active":"2023-10-03T10:09:44.162475+0000","last_peered":"2023-10-03T10:09:44.162475+0000","last_clean":"2023-10-03T10:09:44.162475+0000","last_became_active":"2023-10-03T10:09:43.754957+0000","last_became_peered":"2023-10-03T10:09:43.754957+0000","last_unstale":"2023-10-03T10:09:44.162475+0000","last_undegraded":"2023-10-03T10:09:44.162475+0000","last_fullsized":"2023-10-03T10:09:44.162475+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.194802+0000","last_change":"2023-10-03T10:09:44.187333+0000","last_active":"2023-10-03T10:09:44.194802+0000","last_peered":"2023-10-03T10:09:44.194802+0000","last_clean":"2023-10-03T10:09:44.194802+0000","last_became_active":"2023-10-03T10:09:44.186824+0000","last_became_peered":"2023-10-03T10:09:44.186824+0000","last_unstale":"2023-10-03T10:09:44.194802+0000","last_undegraded":"2023-10-03T10:09:44.194802+0000","last_fullsized":"2023-10-03T10:09:44.194802+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.163844+0000","last_change":"2023-10-03T10:09:43.758995+0000","last_active":"2023-10-03T10:09:44.163844+0000","last_peered":"2023-10-03T10:09:44.163844+0000","last_clean":"2023-10-03T10:09:44.163844+0000","last_became_active":"2023-10-03T10:09:43.758565+0000","last_became_peered":"2023-10-03T10:09:43.758565+0000","last_unstale":"2023-10-03T10:09:44.163844+0000","last_undegraded":"2023-10-03T10:09:44.163844+0000","last_fullsized":"2023-10-03T10:09:44.163844+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173655+0000","last_change":"2023-10-03T10:09:44.166557+0000","last_active":"2023-10-03T10:09:44.173655+0000","last_peered":"2023-10-03T10:09:44.173655+0000","last_clean":"2023-10-03T10:09:44.173655+0000","last_became_active":"2023-10-03T10:09:44.166300+0000","last_became_peered":"2023-10-03T10:09:44.166300+0000","last_unstale":"2023-10-03T10:09:44.173655+0000","last_undegraded":"2023-10-03T10:09:44.173655+0000","last_fullsized":"2023-10-03T10:09:44.173655+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"16'8","reported_seq":18,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.194737+0000","last_change":"2023-10-03T10:09:43.748084+0000","last_active":"2023-10-03T10:09:44.194737+0000","last_peered":"2023-10-03T10:09:44.194737+0000","last_clean":"2023-10-03T10:09:44.194737+0000","last_became_active":"2023-10-03T10:09:43.747377+0000","last_became_peered":"2023-10-03T10:09:43.747377+0000","last_unstale":"2023-10-03T10:09:44.194737+0000","last_undegraded":"2023-10-03T10:09:44.194737+0000","last_fullsized":"2023-10-03T10:09:44.194737+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195326+0000","last_change":"2023-10-03T10:09:44.190767+0000","last_active":"2023-10-03T10:09:44.195326+0000","last_peered":"2023-10-03T10:09:44.195326+0000","last_clean":"2023-10-03T10:09:44.195326+0000","last_became_active":"2023-10-03T10:09:44.190514+0000","last_became_peered":"2023-10-03T10:09:44.190514+0000","last_unstale":"2023-10-03T10:09:44.195326+0000","last_undegraded":"2023-10-03T10:09:44.195326+0000","last_fullsized":"2023-10-03T10:09:44.195326+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.183000+0000","last_change":"2023-10-03T10:09:43.748461+0000","last_active":"2023-10-03T10:09:44.183000+0000","last_peered":"2023-10-03T10:09:44.183000+0000","last_clean":"2023-10-03T10:09:44.183000+0000","last_became_active":"2023-10-03T10:09:43.747372+0000","last_became_peered":"2023-10-03T10:09:43.747372+0000","last_unstale":"2023-10-03T10:09:44.183000+0000","last_undegraded":"2023-10-03T10:09:44.183000+0000","last_fullsized":"2023-10-03T10:09:44.183000+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196810+0000","last_change":"2023-10-03T10:09:44.192036+0000","last_active":"2023-10-03T10:09:44.196810+0000","last_peered":"2023-10-03T10:09:44.196810+0000","last_clean":"2023-10-03T10:09:44.196810+0000","last_became_active":"2023-10-03T10:09:44.191093+0000","last_became_peered":"2023-10-03T10:09:44.191093+0000","last_unstale":"2023-10-03T10:09:44.196810+0000","last_undegraded":"2023-10-03T10:09:44.196810+0000","last_fullsized":"2023-10-03T10:09:44.196810+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:42.718400+0000","last_change":"2023-10-03T10:09:42.718400+0000","last_active":"2023-10-03T10:09:42.718400+0000","last_peered":"2023-10-03T10:09:42.718400+0000","last_clean":"2023-10-03T10:09:42.718400+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:42.718400+0000","last_undegraded":"2023-10-03T10:09:42.718400+0000","last_fullsized":"2023-10-03T10:09:42.718400+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.179874+0000","last_change":"2023-10-03T10:09:44.172400+0000","last_active":"2023-10-03T10:09:44.179874+0000","last_peered":"2023-10-03T10:09:44.179874+0000","last_clean":"2023-10-03T10:09:44.179874+0000","last_became_active":"2023-10-03T10:09:44.172095+0000","last_became_peered":"2023-10-03T10:09:44.172095+0000","last_unstale":"2023-10-03T10:09:44.179874+0000","last_undegraded":"2023-10-03T10:09:44.179874+0000","last_fullsized":"2023-10-03T10:09:44.179874+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195109+0000","last_change":"2023-10-03T10:09:43.760622+0000","last_active":"2023-10-03T10:09:44.195109+0000","last_peered":"2023-10-03T10:09:44.195109+0000","last_clean":"2023-10-03T10:09:44.195109+0000","last_became_active":"2023-10-03T10:09:43.760371+0000","last_became_peered":"2023-10-03T10:09:43.760371+0000","last_unstale":"2023-10-03T10:09:44.195109+0000","last_undegraded":"2023-10-03T10:09:44.195109+0000","last_fullsized":"2023-10-03T10:09:44.195109+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.157166+0000","last_change":"2023-10-03T10:09:43.756885+0000","last_active":"2023-10-03T10:09:44.157166+0000","last_peered":"2023-10-03T10:09:44.157166+0000","last_clean":"2023-10-03T10:09:44.157166+0000","last_became_active":"2023-10-03T10:09:43.756588+0000","last_became_peered":"2023-10-03T10:09:43.756588+0000","last_unstale":"2023-10-03T10:09:44.157166+0000","last_undegraded":"2023-10-03T10:09:44.157166+0000","last_fullsized":"2023-10-03T10:09:44.157166+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184514+0000","last_change":"2023-10-03T10:09:44.181603+0000","last_active":"2023-10-03T10:09:44.184514+0000","last_peered":"2023-10-03T10:09:44.184514+0000","last_clean":"2023-10-03T10:09:44.184514+0000","last_became_active":"2023-10-03T10:09:44.181125+0000","last_became_peered":"2023-10-03T10:09:44.181125+0000","last_unstale":"2023-10-03T10:09:44.184514+0000","last_undegraded":"2023-10-03T10:09:44.184514+0000","last_fullsized":"2023-10-03T10:09:44.184514+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200822+0000","last_change":"2023-10-03T10:09:43.751873+0000","last_active":"2023-10-03T10:09:44.200822+0000","last_peered":"2023-10-03T10:09:44.200822+0000","last_clean":"2023-10-03T10:09:44.200822+0000","last_became_active":"2023-10-03T10:09:43.751324+0000","last_became_peered":"2023-10-03T10:09:43.751324+0000","last_unstale":"2023-10-03T10:09:44.200822+0000","last_undegraded":"2023-10-03T10:09:44.200822+0000","last_fullsized":"2023-10-03T10:09:44.200822+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.174302+0000","last_change":"2023-10-03T10:09:44.170676+0000","last_active":"2023-10-03T10:09:44.174302+0000","last_peered":"2023-10-03T10:09:44.174302+0000","last_clean":"2023-10-03T10:09:44.174302+0000","last_became_active":"2023-10-03T10:09:44.170398+0000","last_became_peered":"2023-10-03T10:09:44.170398+0000","last_unstale":"2023-10-03T10:09:44.174302+0000","last_undegraded":"2023-10-03T10:09:44.174302+0000","last_fullsized":"2023-10-03T10:09:44.174302+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.166137+0000","last_change":"2023-10-03T10:09:43.760711+0000","last_active":"2023-10-03T10:09:44.166137+0000","last_peered":"2023-10-03T10:09:44.166137+0000","last_clean":"2023-10-03T10:09:44.166137+0000","last_became_active":"2023-10-03T10:09:43.760432+0000","last_became_peered":"2023-10-03T10:09:43.760432+0000","last_unstale":"2023-10-03T10:09:44.166137+0000","last_undegraded":"2023-10-03T10:09:44.166137+0000","last_fullsized":"2023-10-03T10:09:44.166137+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195829+0000","last_change":"2023-10-03T10:09:44.191073+0000","last_active":"2023-10-03T10:09:44.195829+0000","last_peered":"2023-10-03T10:09:44.195829+0000","last_clean":"2023-10-03T10:09:44.195829+0000","last_became_active":"2023-10-03T10:09:44.190218+0000","last_became_peered":"2023-10-03T10:09:44.190218+0000","last_unstale":"2023-10-03T10:09:44.195829+0000","last_undegraded":"2023-10-03T10:09:44.195829+0000","last_fullsized":"2023-10-03T10:09:44.195829+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.163521+0000","last_change":"2023-10-03T10:09:43.758846+0000","last_active":"2023-10-03T10:09:44.163521+0000","last_peered":"2023-10-03T10:09:44.163521+0000","last_clean":"2023-10-03T10:09:44.163521+0000","last_became_active":"2023-10-03T10:09:43.757973+0000","last_became_peered":"2023-10-03T10:09:43.757973+0000","last_unstale":"2023-10-03T10:09:44.163521+0000","last_undegraded":"2023-10-03T10:09:44.163521+0000","last_fullsized":"2023-10-03T10:09:44.163521+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.162993+0000","last_change":"2023-10-03T10:09:43.761068+0000","last_active":"2023-10-03T10:09:44.162993+0000","last_peered":"2023-10-03T10:09:44.162993+0000","last_clean":"2023-10-03T10:09:44.162993+0000","last_became_active":"2023-10-03T10:09:43.760825+0000","last_became_peered":"2023-10-03T10:09:43.760825+0000","last_unstale":"2023-10-03T10:09:44.162993+0000","last_undegraded":"2023-10-03T10:09:44.162993+0000","last_fullsized":"2023-10-03T10:09:44.162993+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180799+0000","last_change":"2023-10-03T10:09:44.178856+0000","last_active":"2023-10-03T10:09:44.180799+0000","last_peered":"2023-10-03T10:09:44.180799+0000","last_clean":"2023-10-03T10:09:44.180799+0000","last_became_active":"2023-10-03T10:09:44.178410+0000","last_became_peered":"2023-10-03T10:09:44.178410+0000","last_unstale":"2023-10-03T10:09:44.180799+0000","last_undegraded":"2023-10-03T10:09:44.180799+0000","last_fullsized":"2023-10-03T10:09:44.180799+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.154127+0000","last_change":"2023-10-03T10:09:39.589080+0000","last_active":"2023-10-03T10:09:44.154127+0000","last_peered":"2023-10-03T10:09:44.154127+0000","last_clean":"2023-10-03T10:09:44.154127+0000","last_became_active":"2023-10-03T10:09:39.588651+0000","last_became_peered":"2023-10-03T10:09:39.588651+0000","last_unstale":"2023-10-03T10:09:44.154127+0000","last_undegraded":"2023-10-03T10:09:44.154127+0000","last_fullsized":"2023-10-03T10:09:44.154127+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:42.718400+0000","last_change":"2023-10-03T10:09:42.718400+0000","last_active":"2023-10-03T10:09:42.718400+0000","last_peered":"2023-10-03T10:09:42.718400+0000","last_clean":"2023-10-03T10:09:42.718400+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:42.718400+0000","last_undegraded":"2023-10-03T10:09:42.718400+0000","last_fullsized":"2023-10-03T10:09:42.718400+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180543+0000","last_change":"2023-10-03T10:09:44.176964+0000","last_active":"2023-10-03T10:09:44.180543+0000","last_peered":"2023-10-03T10:09:44.180543+0000","last_clean":"2023-10-03T10:09:44.180543+0000","last_became_active":"2023-10-03T10:09:44.176671+0000","last_became_peered":"2023-10-03T10:09:44.176671+0000","last_unstale":"2023-10-03T10:09:44.180543+0000","last_undegraded":"2023-10-03T10:09:44.180543+0000","last_fullsized":"2023-10-03T10:09:44.180543+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.166922+0000","last_change":"2023-10-03T10:09:39.586735+0000","last_active":"2023-10-03T10:09:44.166922+0000","last_peered":"2023-10-03T10:09:44.166922+0000","last_clean":"2023-10-03T10:09:44.166922+0000","last_became_active":"2023-10-03T10:09:39.586440+0000","last_became_peered":"2023-10-03T10:09:39.586440+0000","last_unstale":"2023-10-03T10:09:44.166922+0000","last_undegraded":"2023-10-03T10:09:44.166922+0000","last_fullsized":"2023-10-03T10:09:44.166922+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.159439+0000","last_change":"2023-10-03T10:09:43.748458+0000","last_active":"2023-10-03T10:09:44.159439+0000","last_peered":"2023-10-03T10:09:44.159439+0000","last_clean":"2023-10-03T10:09:44.159439+0000","last_became_active":"2023-10-03T10:09:43.747721+0000","last_became_peered":"2023-10-03T10:09:43.747721+0000","last_unstale":"2023-10-03T10:09:44.159439+0000","last_undegraded":"2023-10-03T10:09:44.159439+0000","last_fullsized":"2023-10-03T10:09:44.159439+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184166+0000","last_change":"2023-10-03T10:09:44.176165+0000","last_active":"2023-10-03T10:09:44.184166+0000","last_peered":"2023-10-03T10:09:44.184166+0000","last_clean":"2023-10-03T10:09:44.184166+0000","last_became_active":"2023-10-03T10:09:44.175853+0000","last_became_peered":"2023-10-03T10:09:44.175853+0000","last_unstale":"2023-10-03T10:09:44.184166+0000","last_undegraded":"2023-10-03T10:09:44.184166+0000","last_fullsized":"2023-10-03T10:09:44.184166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-10-03T10:09:39.591637+0000","last_change":"2023-10-03T10:09:39.590114+0000","last_active":"2023-10-03T10:09:39.591637+0000","last_peered":"2023-10-03T10:09:39.591637+0000","last_clean":"2023-10-03T10:09:39.591637+0000","last_became_active":"2023-10-03T10:09:39.589663+0000","last_became_peered":"2023-10-03T10:09:39.589663+0000","last_unstale":"2023-10-03T10:09:39.591637+0000","last_undegraded":"2023-10-03T10:09:39.591637+0000","last_fullsized":"2023-10-03T10:09:39.591637+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.155654+0000","last_change":"2023-10-03T10:09:43.748267+0000","last_active":"2023-10-03T10:09:44.155654+0000","last_peered":"2023-10-03T10:09:44.155654+0000","last_clean":"2023-10-03T10:09:44.155654+0000","last_became_active":"2023-10-03T10:09:43.747789+0000","last_became_peered":"2023-10-03T10:09:43.747789+0000","last_unstale":"2023-10-03T10:09:44.155654+0000","last_undegraded":"2023-10-03T10:09:44.155654+0000","last_fullsized":"2023-10-03T10:09:44.155654+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180773+0000","last_change":"2023-10-03T10:09:44.178680+0000","last_active":"2023-10-03T10:09:44.180773+0000","last_peered":"2023-10-03T10:09:44.180773+0000","last_clean":"2023-10-03T10:09:44.180773+0000","last_became_active":"2023-10-03T10:09:44.178114+0000","last_became_peered":"2023-10-03T10:09:44.178114+0000","last_unstale":"2023-10-03T10:09:44.180773+0000","last_undegraded":"2023-10-03T10:09:44.180773+0000","last_fullsized":"2023-10-03T10:09:44.180773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.167768+0000","last_change":"2023-10-03T10:09:39.587728+0000","last_active":"2023-10-03T10:09:44.167768+0000","last_peered":"2023-10-03T10:09:44.167768+0000","last_clean":"2023-10-03T10:09:44.167768+0000","last_became_active":"2023-10-03T10:09:39.587441+0000","last_became_peered":"2023-10-03T10:09:39.587441+0000","last_unstale":"2023-10-03T10:09:44.167768+0000","last_undegraded":"2023-10-03T10:09:44.167768+0000","last_fullsized":"2023-10-03T10:09:44.167768+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.153191+0000","last_change":"2023-10-03T10:09:38.585668+0000","last_active":"2023-10-03T10:09:44.153191+0000","last_peered":"2023-10-03T10:09:44.153191+0000","last_clean":"2023-10-03T10:09:44.153191+0000","last_became_active":"2023-10-03T10:09:38.585324+0000","last_became_peered":"2023-10-03T10:09:38.585324+0000","last_unstale":"2023-10-03T10:09:44.153191+0000","last_undegraded":"2023-10-03T10:09:44.153191+0000","last_fullsized":"2023-10-03T10:09:44.153191+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:37.570571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:37.570571+0000","last_clean_scrub_stamp":"2023-10-03T10:09:37.570571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.165999+0000","last_change":"2023-10-03T10:09:43.758096+0000","last_active":"2023-10-03T10:09:44.165999+0000","last_peered":"2023-10-03T10:09:44.165999+0000","last_clean":"2023-10-03T10:09:44.165999+0000","last_became_active":"2023-10-03T10:09:43.757689+0000","last_became_peered":"2023-10-03T10:09:43.757689+0000","last_unstale":"2023-10-03T10:09:44.165999+0000","last_undegraded":"2023-10-03T10:09:44.165999+0000","last_fullsized":"2023-10-03T10:09:44.165999+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196949+0000","last_change":"2023-10-03T10:09:44.192781+0000","last_active":"2023-10-03T10:09:44.196949+0000","last_peered":"2023-10-03T10:09:44.196949+0000","last_clean":"2023-10-03T10:09:44.196949+0000","last_became_active":"2023-10-03T10:09:44.190597+0000","last_became_peered":"2023-10-03T10:09:44.190597+0000","last_unstale":"2023-10-03T10:09:44.196949+0000","last_undegraded":"2023-10-03T10:09:44.196949+0000","last_fullsized":"2023-10-03T10:09:44.196949+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.163365+0000","last_change":"2023-10-03T10:09:39.586806+0000","last_active":"2023-10-03T10:09:44.163365+0000","last_peered":"2023-10-03T10:09:44.163365+0000","last_clean":"2023-10-03T10:09:44.163365+0000","last_became_active":"2023-10-03T10:09:39.586481+0000","last_became_peered":"2023-10-03T10:09:39.586481+0000","last_unstale":"2023-10-03T10:09:44.163365+0000","last_undegraded":"2023-10-03T10:09:44.163365+0000","last_fullsized":"2023-10-03T10:09:44.163365+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.156596+0000","last_change":"2023-10-03T10:09:43.747783+0000","last_active":"2023-10-03T10:09:44.156596+0000","last_peered":"2023-10-03T10:09:44.156596+0000","last_clean":"2023-10-03T10:09:44.156596+0000","last_became_active":"2023-10-03T10:09:43.746009+0000","last_became_peered":"2023-10-03T10:09:43.746009+0000","last_unstale":"2023-10-03T10:09:44.156596+0000","last_undegraded":"2023-10-03T10:09:44.156596+0000","last_fullsized":"2023-10-03T10:09:44.156596+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184590+0000","last_change":"2023-10-03T10:09:44.181359+0000","last_active":"2023-10-03T10:09:44.184590+0000","last_peered":"2023-10-03T10:09:44.184590+0000","last_clean":"2023-10-03T10:09:44.184590+0000","last_became_active":"2023-10-03T10:09:44.181075+0000","last_became_peered":"2023-10-03T10:09:44.181075+0000","last_unstale":"2023-10-03T10:09:44.184590+0000","last_undegraded":"2023-10-03T10:09:44.184590+0000","last_fullsized":"2023-10-03T10:09:44.184590+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.168332+0000","last_change":"2023-10-03T10:09:39.588827+0000","last_active":"2023-10-03T10:09:44.168332+0000","last_peered":"2023-10-03T10:09:44.168332+0000","last_clean":"2023-10-03T10:09:44.168332+0000","last_became_active":"2023-10-03T10:09:39.588531+0000","last_became_peered":"2023-10-03T10:09:39.588531+0000","last_unstale":"2023-10-03T10:09:44.168332+0000","last_undegraded":"2023-10-03T10:09:44.168332+0000","last_fullsized":"2023-10-03T10:09:44.168332+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.152966+0000","last_change":"2023-10-03T10:09:43.746339+0000","last_active":"2023-10-03T10:09:44.152966+0000","last_peered":"2023-10-03T10:09:44.152966+0000","last_clean":"2023-10-03T10:09:44.152966+0000","last_became_active":"2023-10-03T10:09:43.746051+0000","last_became_peered":"2023-10-03T10:09:43.746051+0000","last_unstale":"2023-10-03T10:09:44.152966+0000","last_undegraded":"2023-10-03T10:09:44.152966+0000","last_fullsized":"2023-10-03T10:09:44.152966+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173149+0000","last_change":"2023-10-03T10:09:44.165684+0000","last_active":"2023-10-03T10:09:44.173149+0000","last_peered":"2023-10-03T10:09:44.173149+0000","last_clean":"2023-10-03T10:09:44.173149+0000","last_became_active":"2023-10-03T10:09:44.165422+0000","last_became_peered":"2023-10-03T10:09:44.165422+0000","last_unstale":"2023-10-03T10:09:44.173149+0000","last_undegraded":"2023-10-03T10:09:44.173149+0000","last_fullsized":"2023-10-03T10:09:44.173149+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.197330+0000","last_change":"2023-10-03T10:09:43.760807+0000","last_active":"2023-10-03T10:09:44.197330+0000","last_peered":"2023-10-03T10:09:44.197330+0000","last_clean":"2023-10-03T10:09:44.197330+0000","last_became_active":"2023-10-03T10:09:43.760517+0000","last_became_peered":"2023-10-03T10:09:43.760517+0000","last_unstale":"2023-10-03T10:09:44.197330+0000","last_undegraded":"2023-10-03T10:09:44.197330+0000","last_fullsized":"2023-10-03T10:09:44.197330+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.153815+0000","last_change":"2023-10-03T10:09:39.586702+0000","last_active":"2023-10-03T10:09:44.153815+0000","last_peered":"2023-10-03T10:09:44.153815+0000","last_clean":"2023-10-03T10:09:44.153815+0000","last_became_active":"2023-10-03T10:09:39.586153+0000","last_became_peered":"2023-10-03T10:09:39.586153+0000","last_unstale":"2023-10-03T10:09:44.153815+0000","last_undegraded":"2023-10-03T10:09:44.153815+0000","last_fullsized":"2023-10-03T10:09:44.153815+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196883+0000","last_change":"2023-10-03T10:09:44.192664+0000","last_active":"2023-10-03T10:09:44.196883+0000","last_peered":"2023-10-03T10:09:44.196883+0000","last_clean":"2023-10-03T10:09:44.196883+0000","last_became_active":"2023-10-03T10:09:44.190513+0000","last_became_peered":"2023-10-03T10:09:44.190513+0000","last_unstale":"2023-10-03T10:09:44.196883+0000","last_undegraded":"2023-10-03T10:09:44.196883+0000","last_fullsized":"2023-10-03T10:09:44.196883+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.167083+0000","last_change":"2023-10-03T10:09:39.588950+0000","last_active":"2023-10-03T10:09:44.167083+0000","last_peered":"2023-10-03T10:09:44.167083+0000","last_clean":"2023-10-03T10:09:44.167083+0000","last_became_active":"2023-10-03T10:09:39.588601+0000","last_became_peered":"2023-10-03T10:09:39.588601+0000","last_unstale":"2023-10-03T10:09:44.167083+0000","last_undegraded":"2023-10-03T10:09:44.167083+0000","last_fullsized":"2023-10-03T10:09:44.167083+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.160538+0000","last_change":"2023-10-03T10:09:43.752793+0000","last_active":"2023-10-03T10:09:44.160538+0000","last_peered":"2023-10-03T10:09:44.160538+0000","last_clean":"2023-10-03T10:09:44.160538+0000","last_became_active":"2023-10-03T10:09:43.752374+0000","last_became_peered":"2023-10-03T10:09:43.752374+0000","last_unstale":"2023-10-03T10:09:44.160538+0000","last_undegraded":"2023-10-03T10:09:44.160538+0000","last_fullsized":"2023-10-03T10:09:44.160538+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200135+0000","last_change":"2023-10-03T10:09:43.756460+0000","last_active":"2023-10-03T10:09:44.200135+0000","last_peered":"2023-10-03T10:09:44.200135+0000","last_clean":"2023-10-03T10:09:44.200135+0000","last_became_active":"2023-10-03T10:09:43.756164+0000","last_became_peered":"2023-10-03T10:09:43.756164+0000","last_unstale":"2023-10-03T10:09:44.200135+0000","last_undegraded":"2023-10-03T10:09:44.200135+0000","last_fullsized":"2023-10-03T10:09:44.200135+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.193051+0000","last_change":"2023-10-03T10:09:43.748380+0000","last_active":"2023-10-03T10:09:44.193051+0000","last_peered":"2023-10-03T10:09:44.193051+0000","last_clean":"2023-10-03T10:09:44.193051+0000","last_became_active":"2023-10-03T10:09:43.748104+0000","last_became_peered":"2023-10-03T10:09:43.748104+0000","last_unstale":"2023-10-03T10:09:44.193051+0000","last_undegraded":"2023-10-03T10:09:44.193051+0000","last_fullsized":"2023-10-03T10:09:44.193051+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.193435+0000","last_change":"2023-10-03T10:09:43.758701+0000","last_active":"2023-10-03T10:09:44.193435+0000","last_peered":"2023-10-03T10:09:44.193435+0000","last_clean":"2023-10-03T10:09:44.193435+0000","last_became_active":"2023-10-03T10:09:43.758376+0000","last_became_peered":"2023-10-03T10:09:43.758376+0000","last_unstale":"2023-10-03T10:09:44.193435+0000","last_undegraded":"2023-10-03T10:09:44.193435+0000","last_fullsized":"2023-10-03T10:09:44.193435+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.199237+0000","last_change":"2023-10-03T10:09:43.758373+0000","last_active":"2023-10-03T10:09:44.199237+0000","last_peered":"2023-10-03T10:09:44.199237+0000","last_clean":"2023-10-03T10:09:44.199237+0000","last_became_active":"2023-10-03T10:09:43.758061+0000","last_became_peered":"2023-10-03T10:09:43.758061+0000","last_unstale":"2023-10-03T10:09:44.199237+0000","last_undegraded":"2023-10-03T10:09:44.199237+0000","last_fullsized":"2023-10-03T10:09:44.199237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196006+0000","last_change":"2023-10-03T10:09:44.191564+0000","last_active":"2023-10-03T10:09:44.196006+0000","last_peered":"2023-10-03T10:09:44.196006+0000","last_clean":"2023-10-03T10:09:44.196006+0000","last_became_active":"2023-10-03T10:09:44.191220+0000","last_became_peered":"2023-10-03T10:09:44.191220+0000","last_unstale":"2023-10-03T10:09:44.196006+0000","last_undegraded":"2023-10-03T10:09:44.196006+0000","last_fullsized":"2023-10-03T10:09:44.196006+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"16'4","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.183675+0000","last_change":"2023-10-03T10:09:43.750463+0000","last_active":"2023-10-03T10:09:44.183675+0000","last_peered":"2023-10-03T10:09:44.183675+0000","last_clean":"2023-10-03T10:09:44.183675+0000","last_became_active":"2023-10-03T10:09:43.749780+0000","last_became_peered":"2023-10-03T10:09:43.749780+0000","last_unstale":"2023-10-03T10:09:44.183675+0000","last_undegraded":"2023-10-03T10:09:44.183675+0000","last_fullsized":"2023-10-03T10:09:44.183675+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.187808+0000","last_change":"2023-10-03T10:09:44.178067+0000","last_active":"2023-10-03T10:09:44.187808+0000","last_peered":"2023-10-03T10:09:44.187808+0000","last_clean":"2023-10-03T10:09:44.187808+0000","last_became_active":"2023-10-03T10:09:44.175124+0000","last_became_peered":"2023-10-03T10:09:44.175124+0000","last_unstale":"2023-10-03T10:09:44.187808+0000","last_undegraded":"2023-10-03T10:09:44.187808+0000","last_fullsized":"2023-10-03T10:09:44.187808+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.197083+0000","last_change":"2023-10-03T10:09:44.192156+0000","last_active":"2023-10-03T10:09:44.197083+0000","last_peered":"2023-10-03T10:09:44.197083+0000","last_clean":"2023-10-03T10:09:44.197083+0000","last_became_active":"2023-10-03T10:09:44.189296+0000","last_became_peered":"2023-10-03T10:09:44.189296+0000","last_unstale":"2023-10-03T10:09:44.197083+0000","last_undegraded":"2023-10-03T10:09:44.197083+0000","last_fullsized":"2023-10-03T10:09:44.197083+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.174106+0000","last_change":"2023-10-03T10:09:44.168473+0000","last_active":"2023-10-03T10:09:44.174106+0000","last_peered":"2023-10-03T10:09:44.174106+0000","last_clean":"2023-10-03T10:09:44.174106+0000","last_became_active":"2023-10-03T10:09:44.168155+0000","last_became_peered":"2023-10-03T10:09:44.168155+0000","last_unstale":"2023-10-03T10:09:44.174106+0000","last_undegraded":"2023-10-03T10:09:44.174106+0000","last_fullsized":"2023-10-03T10:09:44.174106+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200881+0000","last_change":"2023-10-03T10:09:44.197689+0000","last_active":"2023-10-03T10:09:44.200881+0000","last_peered":"2023-10-03T10:09:44.200881+0000","last_clean":"2023-10-03T10:09:44.200881+0000","last_became_active":"2023-10-03T10:09:44.197410+0000","last_became_peered":"2023-10-03T10:09:44.197410+0000","last_unstale":"2023-10-03T10:09:44.200881+0000","last_undegraded":"2023-10-03T10:09:44.200881+0000","last_fullsized":"2023-10-03T10:09:44.200881+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184190+0000","last_change":"2023-10-03T10:09:44.176966+0000","last_active":"2023-10-03T10:09:44.184190+0000","last_peered":"2023-10-03T10:09:44.184190+0000","last_clean":"2023-10-03T10:09:44.184190+0000","last_became_active":"2023-10-03T10:09:44.176386+0000","last_became_peered":"2023-10-03T10:09:44.176386+0000","last_unstale":"2023-10-03T10:09:44.184190+0000","last_undegraded":"2023-10-03T10:09:44.184190+0000","last_fullsized":"2023-10-03T10:09:44.184190+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":102,"acting":102,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":3076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":58,"acting":58,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705666,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297200,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074640,"statfs":{"total":96636764160,"available":96332431360,"internally_reserved":0,"allocated":106496,"data_stored":26836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]}]},{"osd":3,"up_from":9,"seq":38654705667,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297264,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074576,"statfs":{"total":96636764160,"available":96332365824,"internally_reserved":0,"allocated":172032,"data_stored":48929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]}]},{"osd":0,"up_from":9,"seq":38654705667,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297256,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074584,"statfs":{"total":96636764160,"available":96332374016,"internally_reserved":0,"allocated":163840,"data_stored":47899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]}]},{"osd":1,"up_from":9,"seq":38654705667,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297252,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074588,"statfs":{"total":96636764160,"available":96332378112,"internally_reserved":0,"allocated":159744,"data_stored":47845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-03T10:09:48.227 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-03T10:09:48.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.393+0000 7fbe862ae700 1 Processor -- start 2023-10-03T10:09:48.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.393+0000 7fbe862ae700 1 -- start start 2023-10-03T10:09:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.393+0000 7fbe862ae700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe80095a10 0x7fbe80095df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.393+0000 7fbe862ae700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbe8012d120 0x7fbe8012f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.393+0000 7fbe862ae700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe80093010 con 0x7fbe80093810 2023-10-03T10:09:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.393+0000 7fbe862ae700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe80093bf0 con 0x7fbe8012d120 2023-10-03T10:09:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.393+0000 7fbe862ae700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbe80092aa0 con 0x7fbe80095a10 2023-10-03T10:09:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.394+0000 7fbe7f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe80095a10 0x7fbe80095df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.394+0000 7fbe8484b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbe8012d120 0x7fbe8012f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.394+0000 7fbe7ffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbe80093810 0x7fbe80095300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35076/0 (socket says 172.21.15.3:35076) 2023-10-03T10:09:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.394+0000 7fbe7ffff700 1 -- 172.21.15.3:0/492192057 learned_addr learned my addr 172.21.15.3:0/492192057 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:09:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.394+0000 7fbe7f7fe700 1 -- 172.21.15.3:0/492192057 >> v2:172.21.15.3:3300/0 conn(0x7fbe8012d120 msgr2=0x7fbe8012f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.394+0000 7fbe7f7fe700 1 --2- 172.21.15.3:0/492192057 >> v2:172.21.15.3:3300/0 conn(0x7fbe8012d120 0x7fbe8012f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:48.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.394+0000 7fbe7f7fe700 1 -- 172.21.15.3:0/492192057 >> v1:172.21.15.3:6789/0 conn(0x7fbe80093810 legacy=0x7fbe80095300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:48.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.394+0000 7fbe7f7fe700 1 -- 172.21.15.3:0/492192057 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe80096e90 con 0x7fbe80095a10 2023-10-03T10:09:48.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.394+0000 7fbe7effd700 1 -- 172.21.15.3:0/492192057 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1510612510 0 0) 0x7fbe80093010 con 0x7fbe80093810 2023-10-03T10:09:48.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.394+0000 7fbe8484b700 1 --2- 172.21.15.3:0/492192057 >> v2:172.21.15.3:3300/0 conn(0x7fbe8012d120 0x7fbe8012f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:48.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.394+0000 7fbe7f7fe700 1 --2- 172.21.15.3:0/492192057 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe80095a10 0x7fbe80095df0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fbe74009760 tx=0x7fbe74009060).ready entity=mon.2 client_cookie=72057b79c3c6dc78 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:48.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.394+0000 7fbe7effd700 1 -- 172.21.15.3:0/492192057 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7400f510 con 0x7fbe80095a10 2023-10-03T10:09:48.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.394+0000 7fbe7effd700 1 -- 172.21.15.3:0/492192057 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7400f670 con 0x7fbe80095a10 2023-10-03T10:09:48.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.394+0000 7fbe7effd700 1 -- 172.21.15.3:0/492192057 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7400f9f0 con 0x7fbe80095a10 2023-10-03T10:09:48.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.395+0000 7fbe862ae700 1 -- 172.21.15.3:0/492192057 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe80095a10 msgr2=0x7fbe80095df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:48.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.395+0000 7fbe862ae700 1 --2- 172.21.15.3:0/492192057 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe80095a10 0x7fbe80095df0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fbe74009760 tx=0x7fbe74009060).stop 2023-10-03T10:09:48.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.395+0000 7fbe862ae700 1 -- 172.21.15.3:0/492192057 shutdown_connections 2023-10-03T10:09:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.395+0000 7fbe862ae700 1 --2- 172.21.15.3:0/492192057 >> v2:172.21.15.3:3300/0 conn(0x7fbe8012d120 0x7fbe8012f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.395+0000 7fbe862ae700 1 --2- 172.21.15.3:0/492192057 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe80095a10 0x7fbe80095df0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.395+0000 7fbe862ae700 1 -- 172.21.15.3:0/492192057 >> 172.21.15.3:0/492192057 conn(0x7fbe80059bb0 msgr2=0x7fbe80097400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.395+0000 7fbe862ae700 1 -- 172.21.15.3:0/492192057 shutdown_connections 2023-10-03T10:09:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.395+0000 7fbe862ae700 1 -- 172.21.15.3:0/492192057 wait complete. 2023-10-03T10:09:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.396+0000 7fbe862ae700 1 Processor -- start 2023-10-03T10:09:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.396+0000 7fbe862ae700 1 -- start start 2023-10-03T10:09:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.397+0000 7fbe862ae700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe80093810 0x7fbe80118e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.397+0000 7fbe862ae700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe80095a10 0x7fbe80119320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.397+0000 7fbe862ae700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe8012d120 0x7fbe80119810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.397+0000 7fbe862ae700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8006a740 con 0x7fbe80093810 2023-10-03T10:09:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.397+0000 7fbe862ae700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe80119d00 con 0x7fbe80095a10 2023-10-03T10:09:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.397+0000 7fbe862ae700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8011b390 con 0x7fbe8012d120 2023-10-03T10:09:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.397+0000 7fbe8484b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe8012d120 0x7fbe80119810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.397+0000 7fbe7f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe80095a10 0x7fbe80119320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.397+0000 7fbe8484b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe8012d120 0x7fbe80119810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51072/0 (socket says 172.21.15.3:51072) 2023-10-03T10:09:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.397+0000 7fbe8484b700 1 -- 172.21.15.3:0/4270168097 learned_addr learned my addr 172.21.15.3:0/4270168097 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.397+0000 7fbe7ffff700 1 --2- 172.21.15.3:0/4270168097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe80093810 0x7fbe80118e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.398+0000 7fbe8484b700 1 -- 172.21.15.3:0/4270168097 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe80095a10 msgr2=0x7fbe80119320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.398+0000 7fbe8484b700 1 --2- 172.21.15.3:0/4270168097 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe80095a10 0x7fbe80119320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.398+0000 7fbe8484b700 1 -- 172.21.15.3:0/4270168097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe80093810 msgr2=0x7fbe80118e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.398+0000 7fbe8484b700 1 --2- 172.21.15.3:0/4270168097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe80093810 0x7fbe80118e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.398+0000 7fbe8484b700 1 -- 172.21.15.3:0/4270168097 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe78009a20 con 0x7fbe8012d120 2023-10-03T10:09:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.398+0000 7fbe7ffff700 1 --2- 172.21.15.3:0/4270168097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe80093810 0x7fbe80118e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.398+0000 7fbe7f7fe700 1 --2- 172.21.15.3:0/4270168097 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe80095a10 0x7fbe80119320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:48.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.398+0000 7fbe8484b700 1 --2- 172.21.15.3:0/4270168097 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe8012d120 0x7fbe80119810 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fbe7800ef90 tx=0x7fbe7800b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:48.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.398+0000 7fbe67fff700 1 -- 172.21.15.3:0/4270168097 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7800cbd0 con 0x7fbe8012d120 2023-10-03T10:09:48.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.398+0000 7fbe67fff700 1 -- 172.21.15.3:0/4270168097 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7800f040 con 0x7fbe8012d120 2023-10-03T10:09:48.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.398+0000 7fbe862ae700 1 -- 172.21.15.3:0/4270168097 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe740092a0 con 0x7fbe8012d120 2023-10-03T10:09:48.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.398+0000 7fbe67fff700 1 -- 172.21.15.3:0/4270168097 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe78013400 con 0x7fbe8012d120 2023-10-03T10:09:48.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.398+0000 7fbe862ae700 1 -- 172.21.15.3:0/4270168097 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe8011a3d0 con 0x7fbe8012d120 2023-10-03T10:09:48.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.399+0000 7fbe67fff700 1 -- 172.21.15.3:0/4270168097 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fbe78013830 con 0x7fbe8012d120 2023-10-03T10:09:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.399+0000 7fbe67fff700 1 -- 172.21.15.3:0/4270168097 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fbe7803b8e0 con 0x7fbe8012d120 2023-10-03T10:09:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.399+0000 7fbe862ae700 1 -- 172.21.15.3:0/4270168097 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe8005a260 con 0x7fbe8012d120 2023-10-03T10:09:48.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.404+0000 7fbe67fff700 1 -- 172.21.15.3:0/4270168097 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fbe78007bc0 con 0x7fbe8012d120 2023-10-03T10:09:48.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.578+0000 7fbe862ae700 1 -- 172.21.15.3:0/4270168097 --> v1:172.21.15.3:6816/118296 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbe80091b60 con 0x7fbe6c02bb60 2023-10-03T10:09:48.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.584+0000 7fbe67fff700 1 -- 172.21.15.3:0/4270168097 <== mgr.4100 v1:172.21.15.3:6816/118296 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248601 (unknown 2965378022 0 4201081181) 0x7fbe80091b60 con 0x7fbe6c02bb60 2023-10-03T10:09:48.587 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:48.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.588+0000 7fbe862ae700 1 -- 172.21.15.3:0/4270168097 >> v1:172.21.15.3:6816/118296 conn(0x7fbe6c02bb60 legacy=0x7fbe6c02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:48.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.588+0000 7fbe862ae700 1 -- 172.21.15.3:0/4270168097 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe8012d120 msgr2=0x7fbe80119810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.588+0000 7fbe862ae700 1 --2- 172.21.15.3:0/4270168097 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe8012d120 0x7fbe80119810 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fbe7800ef90 tx=0x7fbe7800b040).stop 2023-10-03T10:09:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.588+0000 7fbe862ae700 1 -- 172.21.15.3:0/4270168097 shutdown_connections 2023-10-03T10:09:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.588+0000 7fbe862ae700 1 --2- 172.21.15.3:0/4270168097 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe8012d120 0x7fbe80119810 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.588+0000 7fbe862ae700 1 --2- 172.21.15.3:0/4270168097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe80093810 0x7fbe80118e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.588+0000 7fbe862ae700 1 --2- 172.21.15.3:0/4270168097 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe80095a10 0x7fbe80119320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.588+0000 7fbe862ae700 1 -- 172.21.15.3:0/4270168097 >> 172.21.15.3:0/4270168097 conn(0x7fbe80059bb0 msgr2=0x7fbe80110f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.588+0000 7fbe862ae700 1 -- 172.21.15.3:0/4270168097 shutdown_connections 2023-10-03T10:09:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.588+0000 7fbe862ae700 1 -- 172.21.15.3:0/4270168097 wait complete. 2023-10-03T10:09:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-03T10:09:48.601 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-10-03T10:09:47.529155+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":178,"acting":178,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1188972,"kb_used_data":588,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298388,"statfs":{"total":386547056640,"available":385329549312,"internally_reserved":0,"allocated":602112,"data_stored":171509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.190395+0000","last_change":"2023-10-03T10:09:44.183364+0000","last_active":"2023-10-03T10:09:44.190395+0000","last_peered":"2023-10-03T10:09:44.190395+0000","last_clean":"2023-10-03T10:09:44.190395+0000","last_became_active":"2023-10-03T10:09:44.183094+0000","last_became_peered":"2023-10-03T10:09:44.183094+0000","last_unstale":"2023-10-03T10:09:44.190395+0000","last_undegraded":"2023-10-03T10:09:44.190395+0000","last_fullsized":"2023-10-03T10:09:44.190395+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.190339+0000","last_change":"2023-10-03T10:09:44.183273+0000","last_active":"2023-10-03T10:09:44.190339+0000","last_peered":"2023-10-03T10:09:44.190339+0000","last_clean":"2023-10-03T10:09:44.190339+0000","last_became_active":"2023-10-03T10:09:44.182451+0000","last_became_peered":"2023-10-03T10:09:44.182451+0000","last_unstale":"2023-10-03T10:09:44.190339+0000","last_undegraded":"2023-10-03T10:09:44.190339+0000","last_fullsized":"2023-10-03T10:09:44.190339+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.174191+0000","last_change":"2023-10-03T10:09:44.168483+0000","last_active":"2023-10-03T10:09:44.174191+0000","last_peered":"2023-10-03T10:09:44.174191+0000","last_clean":"2023-10-03T10:09:44.174191+0000","last_became_active":"2023-10-03T10:09:44.168186+0000","last_became_peered":"2023-10-03T10:09:44.168186+0000","last_unstale":"2023-10-03T10:09:44.174191+0000","last_undegraded":"2023-10-03T10:09:44.174191+0000","last_fullsized":"2023-10-03T10:09:44.174191+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.198209+0000","last_change":"2023-10-03T10:09:44.192967+0000","last_active":"2023-10-03T10:09:44.198209+0000","last_peered":"2023-10-03T10:09:44.198209+0000","last_clean":"2023-10-03T10:09:44.198209+0000","last_became_active":"2023-10-03T10:09:44.190979+0000","last_became_peered":"2023-10-03T10:09:44.190979+0000","last_unstale":"2023-10-03T10:09:44.198209+0000","last_undegraded":"2023-10-03T10:09:44.198209+0000","last_fullsized":"2023-10-03T10:09:44.198209+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.192360+0000","last_change":"2023-10-03T10:09:44.183121+0000","last_active":"2023-10-03T10:09:44.192360+0000","last_peered":"2023-10-03T10:09:44.192360+0000","last_clean":"2023-10-03T10:09:44.192360+0000","last_became_active":"2023-10-03T10:09:44.182647+0000","last_became_peered":"2023-10-03T10:09:44.182647+0000","last_unstale":"2023-10-03T10:09:44.192360+0000","last_undegraded":"2023-10-03T10:09:44.192360+0000","last_fullsized":"2023-10-03T10:09:44.192360+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.187765+0000","last_change":"2023-10-03T10:09:44.173123+0000","last_active":"2023-10-03T10:09:44.187765+0000","last_peered":"2023-10-03T10:09:44.187765+0000","last_clean":"2023-10-03T10:09:44.187765+0000","last_became_active":"2023-10-03T10:09:44.172353+0000","last_became_peered":"2023-10-03T10:09:44.172353+0000","last_unstale":"2023-10-03T10:09:44.187765+0000","last_undegraded":"2023-10-03T10:09:44.187765+0000","last_fullsized":"2023-10-03T10:09:44.187765+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173973+0000","last_change":"2023-10-03T10:09:44.166758+0000","last_active":"2023-10-03T10:09:44.173973+0000","last_peered":"2023-10-03T10:09:44.173973+0000","last_clean":"2023-10-03T10:09:44.173973+0000","last_became_active":"2023-10-03T10:09:44.166415+0000","last_became_peered":"2023-10-03T10:09:44.166415+0000","last_unstale":"2023-10-03T10:09:44.173973+0000","last_undegraded":"2023-10-03T10:09:44.173973+0000","last_fullsized":"2023-10-03T10:09:44.173973+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180446+0000","last_change":"2023-10-03T10:09:44.176833+0000","last_active":"2023-10-03T10:09:44.180446+0000","last_peered":"2023-10-03T10:09:44.180446+0000","last_clean":"2023-10-03T10:09:44.180446+0000","last_became_active":"2023-10-03T10:09:44.176526+0000","last_became_peered":"2023-10-03T10:09:44.176526+0000","last_unstale":"2023-10-03T10:09:44.180446+0000","last_undegraded":"2023-10-03T10:09:44.180446+0000","last_fullsized":"2023-10-03T10:09:44.180446+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180114+0000","last_change":"2023-10-03T10:09:44.175868+0000","last_active":"2023-10-03T10:09:44.180114+0000","last_peered":"2023-10-03T10:09:44.180114+0000","last_clean":"2023-10-03T10:09:44.180114+0000","last_became_active":"2023-10-03T10:09:44.175414+0000","last_became_peered":"2023-10-03T10:09:44.175414+0000","last_unstale":"2023-10-03T10:09:44.180114+0000","last_undegraded":"2023-10-03T10:09:44.180114+0000","last_fullsized":"2023-10-03T10:09:44.180114+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180522+0000","last_change":"2023-10-03T10:09:44.176456+0000","last_active":"2023-10-03T10:09:44.180522+0000","last_peered":"2023-10-03T10:09:44.180522+0000","last_clean":"2023-10-03T10:09:44.180522+0000","last_became_active":"2023-10-03T10:09:44.176146+0000","last_became_peered":"2023-10-03T10:09:44.176146+0000","last_unstale":"2023-10-03T10:09:44.180522+0000","last_undegraded":"2023-10-03T10:09:44.180522+0000","last_fullsized":"2023-10-03T10:09:44.180522+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.178380+0000","last_change":"2023-10-03T10:09:44.176614+0000","last_active":"2023-10-03T10:09:44.178380+0000","last_peered":"2023-10-03T10:09:44.178380+0000","last_clean":"2023-10-03T10:09:44.178380+0000","last_became_active":"2023-10-03T10:09:44.176334+0000","last_became_peered":"2023-10-03T10:09:44.176334+0000","last_unstale":"2023-10-03T10:09:44.178380+0000","last_undegraded":"2023-10-03T10:09:44.178380+0000","last_fullsized":"2023-10-03T10:09:44.178380+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.192741+0000","last_change":"2023-10-03T10:09:44.179440+0000","last_active":"2023-10-03T10:09:44.192741+0000","last_peered":"2023-10-03T10:09:44.192741+0000","last_clean":"2023-10-03T10:09:44.192741+0000","last_became_active":"2023-10-03T10:09:44.179150+0000","last_became_peered":"2023-10-03T10:09:44.179150+0000","last_unstale":"2023-10-03T10:09:44.192741+0000","last_undegraded":"2023-10-03T10:09:44.192741+0000","last_fullsized":"2023-10-03T10:09:44.192741+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173996+0000","last_change":"2023-10-03T10:09:44.166901+0000","last_active":"2023-10-03T10:09:44.173996+0000","last_peered":"2023-10-03T10:09:44.173996+0000","last_clean":"2023-10-03T10:09:44.173996+0000","last_became_active":"2023-10-03T10:09:44.166610+0000","last_became_peered":"2023-10-03T10:09:44.166610+0000","last_unstale":"2023-10-03T10:09:44.173996+0000","last_undegraded":"2023-10-03T10:09:44.173996+0000","last_fullsized":"2023-10-03T10:09:44.173996+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173643+0000","last_change":"2023-10-03T10:09:44.165366+0000","last_active":"2023-10-03T10:09:44.173643+0000","last_peered":"2023-10-03T10:09:44.173643+0000","last_clean":"2023-10-03T10:09:44.173643+0000","last_became_active":"2023-10-03T10:09:44.165083+0000","last_became_peered":"2023-10-03T10:09:44.165083+0000","last_unstale":"2023-10-03T10:09:44.173643+0000","last_undegraded":"2023-10-03T10:09:44.173643+0000","last_fullsized":"2023-10-03T10:09:44.173643+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184209+0000","last_change":"2023-10-03T10:09:44.177178+0000","last_active":"2023-10-03T10:09:44.184209+0000","last_peered":"2023-10-03T10:09:44.184209+0000","last_clean":"2023-10-03T10:09:44.184209+0000","last_became_active":"2023-10-03T10:09:44.176686+0000","last_became_peered":"2023-10-03T10:09:44.176686+0000","last_unstale":"2023-10-03T10:09:44.184209+0000","last_undegraded":"2023-10-03T10:09:44.184209+0000","last_fullsized":"2023-10-03T10:09:44.184209+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180396+0000","last_change":"2023-10-03T10:09:44.177335+0000","last_active":"2023-10-03T10:09:44.180396+0000","last_peered":"2023-10-03T10:09:44.180396+0000","last_clean":"2023-10-03T10:09:44.180396+0000","last_became_active":"2023-10-03T10:09:44.175202+0000","last_became_peered":"2023-10-03T10:09:44.175202+0000","last_unstale":"2023-10-03T10:09:44.180396+0000","last_undegraded":"2023-10-03T10:09:44.180396+0000","last_fullsized":"2023-10-03T10:09:44.180396+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184702+0000","last_change":"2023-10-03T10:09:44.182207+0000","last_active":"2023-10-03T10:09:44.184702+0000","last_peered":"2023-10-03T10:09:44.184702+0000","last_clean":"2023-10-03T10:09:44.184702+0000","last_became_active":"2023-10-03T10:09:44.181944+0000","last_became_peered":"2023-10-03T10:09:44.181944+0000","last_unstale":"2023-10-03T10:09:44.184702+0000","last_undegraded":"2023-10-03T10:09:44.184702+0000","last_fullsized":"2023-10-03T10:09:44.184702+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.192336+0000","last_change":"2023-10-03T10:09:44.182610+0000","last_active":"2023-10-03T10:09:44.192336+0000","last_peered":"2023-10-03T10:09:44.192336+0000","last_clean":"2023-10-03T10:09:44.192336+0000","last_became_active":"2023-10-03T10:09:44.180732+0000","last_became_peered":"2023-10-03T10:09:44.180732+0000","last_unstale":"2023-10-03T10:09:44.192336+0000","last_undegraded":"2023-10-03T10:09:44.192336+0000","last_fullsized":"2023-10-03T10:09:44.192336+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.187788+0000","last_change":"2023-10-03T10:09:44.173182+0000","last_active":"2023-10-03T10:09:44.187788+0000","last_peered":"2023-10-03T10:09:44.187788+0000","last_clean":"2023-10-03T10:09:44.187788+0000","last_became_active":"2023-10-03T10:09:44.172685+0000","last_became_peered":"2023-10-03T10:09:44.172685+0000","last_unstale":"2023-10-03T10:09:44.187788+0000","last_undegraded":"2023-10-03T10:09:44.187788+0000","last_fullsized":"2023-10-03T10:09:44.187788+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180761+0000","last_change":"2023-10-03T10:09:44.177910+0000","last_active":"2023-10-03T10:09:44.180761+0000","last_peered":"2023-10-03T10:09:44.180761+0000","last_clean":"2023-10-03T10:09:44.180761+0000","last_became_active":"2023-10-03T10:09:44.177618+0000","last_became_peered":"2023-10-03T10:09:44.177618+0000","last_unstale":"2023-10-03T10:09:44.180761+0000","last_undegraded":"2023-10-03T10:09:44.180761+0000","last_fullsized":"2023-10-03T10:09:44.180761+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180424+0000","last_change":"2023-10-03T10:09:44.176287+0000","last_active":"2023-10-03T10:09:44.180424+0000","last_peered":"2023-10-03T10:09:44.180424+0000","last_clean":"2023-10-03T10:09:44.180424+0000","last_became_active":"2023-10-03T10:09:44.175320+0000","last_became_peered":"2023-10-03T10:09:44.175320+0000","last_unstale":"2023-10-03T10:09:44.180424+0000","last_undegraded":"2023-10-03T10:09:44.180424+0000","last_fullsized":"2023-10-03T10:09:44.180424+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.194745+0000","last_change":"2023-10-03T10:09:44.187148+0000","last_active":"2023-10-03T10:09:44.194745+0000","last_peered":"2023-10-03T10:09:44.194745+0000","last_clean":"2023-10-03T10:09:44.194745+0000","last_became_active":"2023-10-03T10:09:44.186523+0000","last_became_peered":"2023-10-03T10:09:44.186523+0000","last_unstale":"2023-10-03T10:09:44.194745+0000","last_undegraded":"2023-10-03T10:09:44.194745+0000","last_fullsized":"2023-10-03T10:09:44.194745+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.189715+0000","last_change":"2023-10-03T10:09:43.745046+0000","last_active":"2023-10-03T10:09:44.189715+0000","last_peered":"2023-10-03T10:09:44.189715+0000","last_clean":"2023-10-03T10:09:44.189715+0000","last_became_active":"2023-10-03T10:09:43.744724+0000","last_became_peered":"2023-10-03T10:09:43.744724+0000","last_unstale":"2023-10-03T10:09:44.189715+0000","last_undegraded":"2023-10-03T10:09:44.189715+0000","last_fullsized":"2023-10-03T10:09:44.189715+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195809+0000","last_change":"2023-10-03T10:09:44.190869+0000","last_active":"2023-10-03T10:09:44.195809+0000","last_peered":"2023-10-03T10:09:44.195809+0000","last_clean":"2023-10-03T10:09:44.195809+0000","last_became_active":"2023-10-03T10:09:44.190207+0000","last_became_peered":"2023-10-03T10:09:44.190207+0000","last_unstale":"2023-10-03T10:09:44.195809+0000","last_undegraded":"2023-10-03T10:09:44.195809+0000","last_fullsized":"2023-10-03T10:09:44.195809+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.152860+0000","last_change":"2023-10-03T10:09:43.750891+0000","last_active":"2023-10-03T10:09:44.152860+0000","last_peered":"2023-10-03T10:09:44.152860+0000","last_clean":"2023-10-03T10:09:44.152860+0000","last_became_active":"2023-10-03T10:09:43.750356+0000","last_became_peered":"2023-10-03T10:09:43.750356+0000","last_unstale":"2023-10-03T10:09:44.152860+0000","last_undegraded":"2023-10-03T10:09:44.152860+0000","last_fullsized":"2023-10-03T10:09:44.152860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184645+0000","last_change":"2023-10-03T10:09:44.183145+0000","last_active":"2023-10-03T10:09:44.184645+0000","last_peered":"2023-10-03T10:09:44.184645+0000","last_clean":"2023-10-03T10:09:44.184645+0000","last_became_active":"2023-10-03T10:09:44.182816+0000","last_became_peered":"2023-10-03T10:09:44.182816+0000","last_unstale":"2023-10-03T10:09:44.184645+0000","last_undegraded":"2023-10-03T10:09:44.184645+0000","last_fullsized":"2023-10-03T10:09:44.184645+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.163680+0000","last_change":"2023-10-03T10:09:43.749289+0000","last_active":"2023-10-03T10:09:44.163680+0000","last_peered":"2023-10-03T10:09:44.163680+0000","last_clean":"2023-10-03T10:09:44.163680+0000","last_became_active":"2023-10-03T10:09:43.748981+0000","last_became_peered":"2023-10-03T10:09:43.748981+0000","last_unstale":"2023-10-03T10:09:44.163680+0000","last_undegraded":"2023-10-03T10:09:44.163680+0000","last_fullsized":"2023-10-03T10:09:44.163680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.197002+0000","last_change":"2023-10-03T10:09:44.192692+0000","last_active":"2023-10-03T10:09:44.197002+0000","last_peered":"2023-10-03T10:09:44.197002+0000","last_clean":"2023-10-03T10:09:44.197002+0000","last_became_active":"2023-10-03T10:09:44.192390+0000","last_became_peered":"2023-10-03T10:09:44.192390+0000","last_unstale":"2023-10-03T10:09:44.197002+0000","last_undegraded":"2023-10-03T10:09:44.197002+0000","last_fullsized":"2023-10-03T10:09:44.197002+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.160877+0000","last_change":"2023-10-03T10:09:43.754802+0000","last_active":"2023-10-03T10:09:44.160877+0000","last_peered":"2023-10-03T10:09:44.160877+0000","last_clean":"2023-10-03T10:09:44.160877+0000","last_became_active":"2023-10-03T10:09:43.754512+0000","last_became_peered":"2023-10-03T10:09:43.754512+0000","last_unstale":"2023-10-03T10:09:44.160877+0000","last_undegraded":"2023-10-03T10:09:44.160877+0000","last_fullsized":"2023-10-03T10:09:44.160877+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196969+0000","last_change":"2023-10-03T10:09:44.192798+0000","last_active":"2023-10-03T10:09:44.196969+0000","last_peered":"2023-10-03T10:09:44.196969+0000","last_clean":"2023-10-03T10:09:44.196969+0000","last_became_active":"2023-10-03T10:09:44.190787+0000","last_became_peered":"2023-10-03T10:09:44.190787+0000","last_unstale":"2023-10-03T10:09:44.196969+0000","last_undegraded":"2023-10-03T10:09:44.196969+0000","last_fullsized":"2023-10-03T10:09:44.196969+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:42.718400+0000","last_change":"2023-10-03T10:09:42.718400+0000","last_active":"2023-10-03T10:09:42.718400+0000","last_peered":"2023-10-03T10:09:42.718400+0000","last_clean":"2023-10-03T10:09:42.718400+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:42.718400+0000","last_undegraded":"2023-10-03T10:09:42.718400+0000","last_fullsized":"2023-10-03T10:09:42.718400+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.194551+0000","last_change":"2023-10-03T10:09:44.184790+0000","last_active":"2023-10-03T10:09:44.194551+0000","last_peered":"2023-10-03T10:09:44.194551+0000","last_clean":"2023-10-03T10:09:44.194551+0000","last_became_active":"2023-10-03T10:09:44.184531+0000","last_became_peered":"2023-10-03T10:09:44.184531+0000","last_unstale":"2023-10-03T10:09:44.194551+0000","last_undegraded":"2023-10-03T10:09:44.194551+0000","last_fullsized":"2023-10-03T10:09:44.194551+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"16'3","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200426+0000","last_change":"2023-10-03T10:09:43.756598+0000","last_active":"2023-10-03T10:09:44.200426+0000","last_peered":"2023-10-03T10:09:44.200426+0000","last_clean":"2023-10-03T10:09:44.200426+0000","last_became_active":"2023-10-03T10:09:43.754974+0000","last_became_peered":"2023-10-03T10:09:43.754974+0000","last_unstale":"2023-10-03T10:09:44.200426+0000","last_undegraded":"2023-10-03T10:09:44.200426+0000","last_fullsized":"2023-10-03T10:09:44.200426+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180499+0000","last_change":"2023-10-03T10:09:44.175504+0000","last_active":"2023-10-03T10:09:44.180499+0000","last_peered":"2023-10-03T10:09:44.180499+0000","last_clean":"2023-10-03T10:09:44.180499+0000","last_became_active":"2023-10-03T10:09:44.175174+0000","last_became_peered":"2023-10-03T10:09:44.175174+0000","last_unstale":"2023-10-03T10:09:44.180499+0000","last_undegraded":"2023-10-03T10:09:44.180499+0000","last_fullsized":"2023-10-03T10:09:44.180499+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"16'6","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.203073+0000","last_change":"2023-10-03T10:09:43.758711+0000","last_active":"2023-10-03T10:09:44.203073+0000","last_peered":"2023-10-03T10:09:44.203073+0000","last_clean":"2023-10-03T10:09:44.203073+0000","last_became_active":"2023-10-03T10:09:43.758392+0000","last_became_peered":"2023-10-03T10:09:43.758392+0000","last_unstale":"2023-10-03T10:09:44.203073+0000","last_undegraded":"2023-10-03T10:09:44.203073+0000","last_fullsized":"2023-10-03T10:09:44.203073+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200762+0000","last_change":"2023-10-03T10:09:43.751877+0000","last_active":"2023-10-03T10:09:44.200762+0000","last_peered":"2023-10-03T10:09:44.200762+0000","last_clean":"2023-10-03T10:09:44.200762+0000","last_became_active":"2023-10-03T10:09:43.749870+0000","last_became_peered":"2023-10-03T10:09:43.749870+0000","last_unstale":"2023-10-03T10:09:44.200762+0000","last_undegraded":"2023-10-03T10:09:44.200762+0000","last_fullsized":"2023-10-03T10:09:44.200762+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195297+0000","last_change":"2023-10-03T10:09:44.190230+0000","last_active":"2023-10-03T10:09:44.195297+0000","last_peered":"2023-10-03T10:09:44.195297+0000","last_clean":"2023-10-03T10:09:44.195297+0000","last_became_active":"2023-10-03T10:09:44.189342+0000","last_became_peered":"2023-10-03T10:09:44.189342+0000","last_unstale":"2023-10-03T10:09:44.195297+0000","last_undegraded":"2023-10-03T10:09:44.195297+0000","last_fullsized":"2023-10-03T10:09:44.195297+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.162475+0000","last_change":"2023-10-03T10:09:43.755271+0000","last_active":"2023-10-03T10:09:44.162475+0000","last_peered":"2023-10-03T10:09:44.162475+0000","last_clean":"2023-10-03T10:09:44.162475+0000","last_became_active":"2023-10-03T10:09:43.754957+0000","last_became_peered":"2023-10-03T10:09:43.754957+0000","last_unstale":"2023-10-03T10:09:44.162475+0000","last_undegraded":"2023-10-03T10:09:44.162475+0000","last_fullsized":"2023-10-03T10:09:44.162475+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.194802+0000","last_change":"2023-10-03T10:09:44.187333+0000","last_active":"2023-10-03T10:09:44.194802+0000","last_peered":"2023-10-03T10:09:44.194802+0000","last_clean":"2023-10-03T10:09:44.194802+0000","last_became_active":"2023-10-03T10:09:44.186824+0000","last_became_peered":"2023-10-03T10:09:44.186824+0000","last_unstale":"2023-10-03T10:09:44.194802+0000","last_undegraded":"2023-10-03T10:09:44.194802+0000","last_fullsized":"2023-10-03T10:09:44.194802+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.163844+0000","last_change":"2023-10-03T10:09:43.758995+0000","last_active":"2023-10-03T10:09:44.163844+0000","last_peered":"2023-10-03T10:09:44.163844+0000","last_clean":"2023-10-03T10:09:44.163844+0000","last_became_active":"2023-10-03T10:09:43.758565+0000","last_became_peered":"2023-10-03T10:09:43.758565+0000","last_unstale":"2023-10-03T10:09:44.163844+0000","last_undegraded":"2023-10-03T10:09:44.163844+0000","last_fullsized":"2023-10-03T10:09:44.163844+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173655+0000","last_change":"2023-10-03T10:09:44.166557+0000","last_active":"2023-10-03T10:09:44.173655+0000","last_peered":"2023-10-03T10:09:44.173655+0000","last_clean":"2023-10-03T10:09:44.173655+0000","last_became_active":"2023-10-03T10:09:44.166300+0000","last_became_peered":"2023-10-03T10:09:44.166300+0000","last_unstale":"2023-10-03T10:09:44.173655+0000","last_undegraded":"2023-10-03T10:09:44.173655+0000","last_fullsized":"2023-10-03T10:09:44.173655+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"16'8","reported_seq":18,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.194737+0000","last_change":"2023-10-03T10:09:43.748084+0000","last_active":"2023-10-03T10:09:44.194737+0000","last_peered":"2023-10-03T10:09:44.194737+0000","last_clean":"2023-10-03T10:09:44.194737+0000","last_became_active":"2023-10-03T10:09:43.747377+0000","last_became_peered":"2023-10-03T10:09:43.747377+0000","last_unstale":"2023-10-03T10:09:44.194737+0000","last_undegraded":"2023-10-03T10:09:44.194737+0000","last_fullsized":"2023-10-03T10:09:44.194737+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195326+0000","last_change":"2023-10-03T10:09:44.190767+0000","last_active":"2023-10-03T10:09:44.195326+0000","last_peered":"2023-10-03T10:09:44.195326+0000","last_clean":"2023-10-03T10:09:44.195326+0000","last_became_active":"2023-10-03T10:09:44.190514+0000","last_became_peered":"2023-10-03T10:09:44.190514+0000","last_unstale":"2023-10-03T10:09:44.195326+0000","last_undegraded":"2023-10-03T10:09:44.195326+0000","last_fullsized":"2023-10-03T10:09:44.195326+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.183000+0000","last_change":"2023-10-03T10:09:43.748461+0000","last_active":"2023-10-03T10:09:44.183000+0000","last_peered":"2023-10-03T10:09:44.183000+0000","last_clean":"2023-10-03T10:09:44.183000+0000","last_became_active":"2023-10-03T10:09:43.747372+0000","last_became_peered":"2023-10-03T10:09:43.747372+0000","last_unstale":"2023-10-03T10:09:44.183000+0000","last_undegraded":"2023-10-03T10:09:44.183000+0000","last_fullsized":"2023-10-03T10:09:44.183000+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196810+0000","last_change":"2023-10-03T10:09:44.192036+0000","last_active":"2023-10-03T10:09:44.196810+0000","last_peered":"2023-10-03T10:09:44.196810+0000","last_clean":"2023-10-03T10:09:44.196810+0000","last_became_active":"2023-10-03T10:09:44.191093+0000","last_became_peered":"2023-10-03T10:09:44.191093+0000","last_unstale":"2023-10-03T10:09:44.196810+0000","last_undegraded":"2023-10-03T10:09:44.196810+0000","last_fullsized":"2023-10-03T10:09:44.196810+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:42.718400+0000","last_change":"2023-10-03T10:09:42.718400+0000","last_active":"2023-10-03T10:09:42.718400+0000","last_peered":"2023-10-03T10:09:42.718400+0000","last_clean":"2023-10-03T10:09:42.718400+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:42.718400+0000","last_undegraded":"2023-10-03T10:09:42.718400+0000","last_fullsized":"2023-10-03T10:09:42.718400+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.179874+0000","last_change":"2023-10-03T10:09:44.172400+0000","last_active":"2023-10-03T10:09:44.179874+0000","last_peered":"2023-10-03T10:09:44.179874+0000","last_clean":"2023-10-03T10:09:44.179874+0000","last_became_active":"2023-10-03T10:09:44.172095+0000","last_became_peered":"2023-10-03T10:09:44.172095+0000","last_unstale":"2023-10-03T10:09:44.179874+0000","last_undegraded":"2023-10-03T10:09:44.179874+0000","last_fullsized":"2023-10-03T10:09:44.179874+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195109+0000","last_change":"2023-10-03T10:09:43.760622+0000","last_active":"2023-10-03T10:09:44.195109+0000","last_peered":"2023-10-03T10:09:44.195109+0000","last_clean":"2023-10-03T10:09:44.195109+0000","last_became_active":"2023-10-03T10:09:43.760371+0000","last_became_peered":"2023-10-03T10:09:43.760371+0000","last_unstale":"2023-10-03T10:09:44.195109+0000","last_undegraded":"2023-10-03T10:09:44.195109+0000","last_fullsized":"2023-10-03T10:09:44.195109+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.157166+0000","last_change":"2023-10-03T10:09:43.756885+0000","last_active":"2023-10-03T10:09:44.157166+0000","last_peered":"2023-10-03T10:09:44.157166+0000","last_clean":"2023-10-03T10:09:44.157166+0000","last_became_active":"2023-10-03T10:09:43.756588+0000","last_became_peered":"2023-10-03T10:09:43.756588+0000","last_unstale":"2023-10-03T10:09:44.157166+0000","last_undegraded":"2023-10-03T10:09:44.157166+0000","last_fullsized":"2023-10-03T10:09:44.157166+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184514+0000","last_change":"2023-10-03T10:09:44.181603+0000","last_active":"2023-10-03T10:09:44.184514+0000","last_peered":"2023-10-03T10:09:44.184514+0000","last_clean":"2023-10-03T10:09:44.184514+0000","last_became_active":"2023-10-03T10:09:44.181125+0000","last_became_peered":"2023-10-03T10:09:44.181125+0000","last_unstale":"2023-10-03T10:09:44.184514+0000","last_undegraded":"2023-10-03T10:09:44.184514+0000","last_fullsized":"2023-10-03T10:09:44.184514+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200822+0000","last_change":"2023-10-03T10:09:43.751873+0000","last_active":"2023-10-03T10:09:44.200822+0000","last_peered":"2023-10-03T10:09:44.200822+0000","last_clean":"2023-10-03T10:09:44.200822+0000","last_became_active":"2023-10-03T10:09:43.751324+0000","last_became_peered":"2023-10-03T10:09:43.751324+0000","last_unstale":"2023-10-03T10:09:44.200822+0000","last_undegraded":"2023-10-03T10:09:44.200822+0000","last_fullsized":"2023-10-03T10:09:44.200822+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.174302+0000","last_change":"2023-10-03T10:09:44.170676+0000","last_active":"2023-10-03T10:09:44.174302+0000","last_peered":"2023-10-03T10:09:44.174302+0000","last_clean":"2023-10-03T10:09:44.174302+0000","last_became_active":"2023-10-03T10:09:44.170398+0000","last_became_peered":"2023-10-03T10:09:44.170398+0000","last_unstale":"2023-10-03T10:09:44.174302+0000","last_undegraded":"2023-10-03T10:09:44.174302+0000","last_fullsized":"2023-10-03T10:09:44.174302+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.166137+0000","last_change":"2023-10-03T10:09:43.760711+0000","last_active":"2023-10-03T10:09:44.166137+0000","last_peered":"2023-10-03T10:09:44.166137+0000","last_clean":"2023-10-03T10:09:44.166137+0000","last_became_active":"2023-10-03T10:09:43.760432+0000","last_became_peered":"2023-10-03T10:09:43.760432+0000","last_unstale":"2023-10-03T10:09:44.166137+0000","last_undegraded":"2023-10-03T10:09:44.166137+0000","last_fullsized":"2023-10-03T10:09:44.166137+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195829+0000","last_change":"2023-10-03T10:09:44.191073+0000","last_active":"2023-10-03T10:09:44.195829+0000","last_peered":"2023-10-03T10:09:44.195829+0000","last_clean":"2023-10-03T10:09:44.195829+0000","last_became_active":"2023-10-03T10:09:44.190218+0000","last_became_peered":"2023-10-03T10:09:44.190218+0000","last_unstale":"2023-10-03T10:09:44.195829+0000","last_undegraded":"2023-10-03T10:09:44.195829+0000","last_fullsized":"2023-10-03T10:09:44.195829+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.163521+0000","last_change":"2023-10-03T10:09:43.758846+0000","last_active":"2023-10-03T10:09:44.163521+0000","last_peered":"2023-10-03T10:09:44.163521+0000","last_clean":"2023-10-03T10:09:44.163521+0000","last_became_active":"2023-10-03T10:09:43.757973+0000","last_became_peered":"2023-10-03T10:09:43.757973+0000","last_unstale":"2023-10-03T10:09:44.163521+0000","last_undegraded":"2023-10-03T10:09:44.163521+0000","last_fullsized":"2023-10-03T10:09:44.163521+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.162993+0000","last_change":"2023-10-03T10:09:43.761068+0000","last_active":"2023-10-03T10:09:44.162993+0000","last_peered":"2023-10-03T10:09:44.162993+0000","last_clean":"2023-10-03T10:09:44.162993+0000","last_became_active":"2023-10-03T10:09:43.760825+0000","last_became_peered":"2023-10-03T10:09:43.760825+0000","last_unstale":"2023-10-03T10:09:44.162993+0000","last_undegraded":"2023-10-03T10:09:44.162993+0000","last_fullsized":"2023-10-03T10:09:44.162993+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180799+0000","last_change":"2023-10-03T10:09:44.178856+0000","last_active":"2023-10-03T10:09:44.180799+0000","last_peered":"2023-10-03T10:09:44.180799+0000","last_clean":"2023-10-03T10:09:44.180799+0000","last_became_active":"2023-10-03T10:09:44.178410+0000","last_became_peered":"2023-10-03T10:09:44.178410+0000","last_unstale":"2023-10-03T10:09:44.180799+0000","last_undegraded":"2023-10-03T10:09:44.180799+0000","last_fullsized":"2023-10-03T10:09:44.180799+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.154127+0000","last_change":"2023-10-03T10:09:39.589080+0000","last_active":"2023-10-03T10:09:44.154127+0000","last_peered":"2023-10-03T10:09:44.154127+0000","last_clean":"2023-10-03T10:09:44.154127+0000","last_became_active":"2023-10-03T10:09:39.588651+0000","last_became_peered":"2023-10-03T10:09:39.588651+0000","last_unstale":"2023-10-03T10:09:44.154127+0000","last_undegraded":"2023-10-03T10:09:44.154127+0000","last_fullsized":"2023-10-03T10:09:44.154127+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:42.718400+0000","last_change":"2023-10-03T10:09:42.718400+0000","last_active":"2023-10-03T10:09:42.718400+0000","last_peered":"2023-10-03T10:09:42.718400+0000","last_clean":"2023-10-03T10:09:42.718400+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:42.718400+0000","last_undegraded":"2023-10-03T10:09:42.718400+0000","last_fullsized":"2023-10-03T10:09:42.718400+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180543+0000","last_change":"2023-10-03T10:09:44.176964+0000","last_active":"2023-10-03T10:09:44.180543+0000","last_peered":"2023-10-03T10:09:44.180543+0000","last_clean":"2023-10-03T10:09:44.180543+0000","last_became_active":"2023-10-03T10:09:44.176671+0000","last_became_peered":"2023-10-03T10:09:44.176671+0000","last_unstale":"2023-10-03T10:09:44.180543+0000","last_undegraded":"2023-10-03T10:09:44.180543+0000","last_fullsized":"2023-10-03T10:09:44.180543+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.166922+0000","last_change":"2023-10-03T10:09:39.586735+0000","last_active":"2023-10-03T10:09:44.166922+0000","last_peered":"2023-10-03T10:09:44.166922+0000","last_clean":"2023-10-03T10:09:44.166922+0000","last_became_active":"2023-10-03T10:09:39.586440+0000","last_became_peered":"2023-10-03T10:09:39.586440+0000","last_unstale":"2023-10-03T10:09:44.166922+0000","last_undegraded":"2023-10-03T10:09:44.166922+0000","last_fullsized":"2023-10-03T10:09:44.166922+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.159439+0000","last_change":"2023-10-03T10:09:43.748458+0000","last_active":"2023-10-03T10:09:44.159439+0000","last_peered":"2023-10-03T10:09:44.159439+0000","last_clean":"2023-10-03T10:09:44.159439+0000","last_became_active":"2023-10-03T10:09:43.747721+0000","last_became_peered":"2023-10-03T10:09:43.747721+0000","last_unstale":"2023-10-03T10:09:44.159439+0000","last_undegraded":"2023-10-03T10:09:44.159439+0000","last_fullsized":"2023-10-03T10:09:44.159439+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184166+0000","last_change":"2023-10-03T10:09:44.176165+0000","last_active":"2023-10-03T10:09:44.184166+0000","last_peered":"2023-10-03T10:09:44.184166+0000","last_clean":"2023-10-03T10:09:44.184166+0000","last_became_active":"2023-10-03T10:09:44.175853+0000","last_became_peered":"2023-10-03T10:09:44.175853+0000","last_unstale":"2023-10-03T10:09:44.184166+0000","last_undegraded":"2023-10-03T10:09:44.184166+0000","last_fullsized":"2023-10-03T10:09:44.184166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-10-03T10:09:39.591637+0000","last_change":"2023-10-03T10:09:39.590114+0000","last_active":"2023-10-03T10:09:39.591637+0000","last_peered":"2023-10-03T10:09:39.591637+0000","last_clean":"2023-10-03T10:09:39.591637+0000","last_became_active":"2023-10-03T10:09:39.589663+0000","last_became_peered":"2023-10-03T10:09:39.589663+0000","last_unstale":"2023-10-03T10:09:39.591637+0000","last_undegraded":"2023-10-03T10:09:39.591637+0000","last_fullsized":"2023-10-03T10:09:39.591637+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.155654+0000","last_change":"2023-10-03T10:09:43.748267+0000","last_active":"2023-10-03T10:09:44.155654+0000","last_peered":"2023-10-03T10:09:44.155654+0000","last_clean":"2023-10-03T10:09:44.155654+0000","last_became_active":"2023-10-03T10:09:43.747789+0000","last_became_peered":"2023-10-03T10:09:43.747789+0000","last_unstale":"2023-10-03T10:09:44.155654+0000","last_undegraded":"2023-10-03T10:09:44.155654+0000","last_fullsized":"2023-10-03T10:09:44.155654+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180773+0000","last_change":"2023-10-03T10:09:44.178680+0000","last_active":"2023-10-03T10:09:44.180773+0000","last_peered":"2023-10-03T10:09:44.180773+0000","last_clean":"2023-10-03T10:09:44.180773+0000","last_became_active":"2023-10-03T10:09:44.178114+0000","last_became_peered":"2023-10-03T10:09:44.178114+0000","last_unstale":"2023-10-03T10:09:44.180773+0000","last_undegraded":"2023-10-03T10:09:44.180773+0000","last_fullsized":"2023-10-03T10:09:44.180773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.167768+0000","last_change":"2023-10-03T10:09:39.587728+0000","last_active":"2023-10-03T10:09:44.167768+0000","last_peered":"2023-10-03T10:09:44.167768+0000","last_clean":"2023-10-03T10:09:44.167768+0000","last_became_active":"2023-10-03T10:09:39.587441+0000","last_became_peered":"2023-10-03T10:09:39.587441+0000","last_unstale":"2023-10-03T10:09:44.167768+0000","last_undegraded":"2023-10-03T10:09:44.167768+0000","last_fullsized":"2023-10-03T10:09:44.167768+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.153191+0000","last_change":"2023-10-03T10:09:38.585668+0000","last_active":"2023-10-03T10:09:44.153191+0000","last_peered":"2023-10-03T10:09:44.153191+0000","last_clean":"2023-10-03T10:09:44.153191+0000","last_became_active":"2023-10-03T10:09:38.585324+0000","last_became_peered":"2023-10-03T10:09:38.585324+0000","last_unstale":"2023-10-03T10:09:44.153191+0000","last_undegraded":"2023-10-03T10:09:44.153191+0000","last_fullsized":"2023-10-03T10:09:44.153191+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:37.570571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:37.570571+0000","last_clean_scrub_stamp":"2023-10-03T10:09:37.570571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.165999+0000","last_change":"2023-10-03T10:09:43.758096+0000","last_active":"2023-10-03T10:09:44.165999+0000","last_peered":"2023-10-03T10:09:44.165999+0000","last_clean":"2023-10-03T10:09:44.165999+0000","last_became_active":"2023-10-03T10:09:43.757689+0000","last_became_peered":"2023-10-03T10:09:43.757689+0000","last_unstale":"2023-10-03T10:09:44.165999+0000","last_undegraded":"2023-10-03T10:09:44.165999+0000","last_fullsized":"2023-10-03T10:09:44.165999+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196949+0000","last_change":"2023-10-03T10:09:44.192781+0000","last_active":"2023-10-03T10:09:44.196949+0000","last_peered":"2023-10-03T10:09:44.196949+0000","last_clean":"2023-10-03T10:09:44.196949+0000","last_became_active":"2023-10-03T10:09:44.190597+0000","last_became_peered":"2023-10-03T10:09:44.190597+0000","last_unstale":"2023-10-03T10:09:44.196949+0000","last_undegraded":"2023-10-03T10:09:44.196949+0000","last_fullsized":"2023-10-03T10:09:44.196949+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.163365+0000","last_change":"2023-10-03T10:09:39.586806+0000","last_active":"2023-10-03T10:09:44.163365+0000","last_peered":"2023-10-03T10:09:44.163365+0000","last_clean":"2023-10-03T10:09:44.163365+0000","last_became_active":"2023-10-03T10:09:39.586481+0000","last_became_peered":"2023-10-03T10:09:39.586481+0000","last_unstale":"2023-10-03T10:09:44.163365+0000","last_undegraded":"2023-10-03T10:09:44.163365+0000","last_fullsized":"2023-10-03T10:09:44.163365+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.156596+0000","last_change":"2023-10-03T10:09:43.747783+0000","last_active":"2023-10-03T10:09:44.156596+0000","last_peered":"2023-10-03T10:09:44.156596+0000","last_clean":"2023-10-03T10:09:44.156596+0000","last_became_active":"2023-10-03T10:09:43.746009+0000","last_became_peered":"2023-10-03T10:09:43.746009+0000","last_unstale":"2023-10-03T10:09:44.156596+0000","last_undegraded":"2023-10-03T10:09:44.156596+0000","last_fullsized":"2023-10-03T10:09:44.156596+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184590+0000","last_change":"2023-10-03T10:09:44.181359+0000","last_active":"2023-10-03T10:09:44.184590+0000","last_peered":"2023-10-03T10:09:44.184590+0000","last_clean":"2023-10-03T10:09:44.184590+0000","last_became_active":"2023-10-03T10:09:44.181075+0000","last_became_peered":"2023-10-03T10:09:44.181075+0000","last_unstale":"2023-10-03T10:09:44.184590+0000","last_undegraded":"2023-10-03T10:09:44.184590+0000","last_fullsized":"2023-10-03T10:09:44.184590+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.168332+0000","last_change":"2023-10-03T10:09:39.588827+0000","last_active":"2023-10-03T10:09:44.168332+0000","last_peered":"2023-10-03T10:09:44.168332+0000","last_clean":"2023-10-03T10:09:44.168332+0000","last_became_active":"2023-10-03T10:09:39.588531+0000","last_became_peered":"2023-10-03T10:09:39.588531+0000","last_unstale":"2023-10-03T10:09:44.168332+0000","last_undegraded":"2023-10-03T10:09:44.168332+0000","last_fullsized":"2023-10-03T10:09:44.168332+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.152966+0000","last_change":"2023-10-03T10:09:43.746339+0000","last_active":"2023-10-03T10:09:44.152966+0000","last_peered":"2023-10-03T10:09:44.152966+0000","last_clean":"2023-10-03T10:09:44.152966+0000","last_became_active":"2023-10-03T10:09:43.746051+0000","last_became_peered":"2023-10-03T10:09:43.746051+0000","last_unstale":"2023-10-03T10:09:44.152966+0000","last_undegraded":"2023-10-03T10:09:44.152966+0000","last_fullsized":"2023-10-03T10:09:44.152966+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173149+0000","last_change":"2023-10-03T10:09:44.165684+0000","last_active":"2023-10-03T10:09:44.173149+0000","last_peered":"2023-10-03T10:09:44.173149+0000","last_clean":"2023-10-03T10:09:44.173149+0000","last_became_active":"2023-10-03T10:09:44.165422+0000","last_became_peered":"2023-10-03T10:09:44.165422+0000","last_unstale":"2023-10-03T10:09:44.173149+0000","last_undegraded":"2023-10-03T10:09:44.173149+0000","last_fullsized":"2023-10-03T10:09:44.173149+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.197330+0000","last_change":"2023-10-03T10:09:43.760807+0000","last_active":"2023-10-03T10:09:44.197330+0000","last_peered":"2023-10-03T10:09:44.197330+0000","last_clean":"2023-10-03T10:09:44.197330+0000","last_became_active":"2023-10-03T10:09:43.760517+0000","last_became_peered":"2023-10-03T10:09:43.760517+0000","last_unstale":"2023-10-03T10:09:44.197330+0000","last_undegraded":"2023-10-03T10:09:44.197330+0000","last_fullsized":"2023-10-03T10:09:44.197330+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.153815+0000","last_change":"2023-10-03T10:09:39.586702+0000","last_active":"2023-10-03T10:09:44.153815+0000","last_peered":"2023-10-03T10:09:44.153815+0000","last_clean":"2023-10-03T10:09:44.153815+0000","last_became_active":"2023-10-03T10:09:39.586153+0000","last_became_peered":"2023-10-03T10:09:39.586153+0000","last_unstale":"2023-10-03T10:09:44.153815+0000","last_undegraded":"2023-10-03T10:09:44.153815+0000","last_fullsized":"2023-10-03T10:09:44.153815+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196883+0000","last_change":"2023-10-03T10:09:44.192664+0000","last_active":"2023-10-03T10:09:44.196883+0000","last_peered":"2023-10-03T10:09:44.196883+0000","last_clean":"2023-10-03T10:09:44.196883+0000","last_became_active":"2023-10-03T10:09:44.190513+0000","last_became_peered":"2023-10-03T10:09:44.190513+0000","last_unstale":"2023-10-03T10:09:44.196883+0000","last_undegraded":"2023-10-03T10:09:44.196883+0000","last_fullsized":"2023-10-03T10:09:44.196883+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.167083+0000","last_change":"2023-10-03T10:09:39.588950+0000","last_active":"2023-10-03T10:09:44.167083+0000","last_peered":"2023-10-03T10:09:44.167083+0000","last_clean":"2023-10-03T10:09:44.167083+0000","last_became_active":"2023-10-03T10:09:39.588601+0000","last_became_peered":"2023-10-03T10:09:39.588601+0000","last_unstale":"2023-10-03T10:09:44.167083+0000","last_undegraded":"2023-10-03T10:09:44.167083+0000","last_fullsized":"2023-10-03T10:09:44.167083+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.160538+0000","last_change":"2023-10-03T10:09:43.752793+0000","last_active":"2023-10-03T10:09:44.160538+0000","last_peered":"2023-10-03T10:09:44.160538+0000","last_clean":"2023-10-03T10:09:44.160538+0000","last_became_active":"2023-10-03T10:09:43.752374+0000","last_became_peered":"2023-10-03T10:09:43.752374+0000","last_unstale":"2023-10-03T10:09:44.160538+0000","last_undegraded":"2023-10-03T10:09:44.160538+0000","last_fullsized":"2023-10-03T10:09:44.160538+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200135+0000","last_change":"2023-10-03T10:09:43.756460+0000","last_active":"2023-10-03T10:09:44.200135+0000","last_peered":"2023-10-03T10:09:44.200135+0000","last_clean":"2023-10-03T10:09:44.200135+0000","last_became_active":"2023-10-03T10:09:43.756164+0000","last_became_peered":"2023-10-03T10:09:43.756164+0000","last_unstale":"2023-10-03T10:09:44.200135+0000","last_undegraded":"2023-10-03T10:09:44.200135+0000","last_fullsized":"2023-10-03T10:09:44.200135+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.193051+0000","last_change":"2023-10-03T10:09:43.748380+0000","last_active":"2023-10-03T10:09:44.193051+0000","last_peered":"2023-10-03T10:09:44.193051+0000","last_clean":"2023-10-03T10:09:44.193051+0000","last_became_active":"2023-10-03T10:09:43.748104+0000","last_became_peered":"2023-10-03T10:09:43.748104+0000","last_unstale":"2023-10-03T10:09:44.193051+0000","last_undegraded":"2023-10-03T10:09:44.193051+0000","last_fullsized":"2023-10-03T10:09:44.193051+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.193435+0000","last_change":"2023-10-03T10:09:43.758701+0000","last_active":"2023-10-03T10:09:44.193435+0000","last_peered":"2023-10-03T10:09:44.193435+0000","last_clean":"2023-10-03T10:09:44.193435+0000","last_became_active":"2023-10-03T10:09:43.758376+0000","last_became_peered":"2023-10-03T10:09:43.758376+0000","last_unstale":"2023-10-03T10:09:44.193435+0000","last_undegraded":"2023-10-03T10:09:44.193435+0000","last_fullsized":"2023-10-03T10:09:44.193435+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.199237+0000","last_change":"2023-10-03T10:09:43.758373+0000","last_active":"2023-10-03T10:09:44.199237+0000","last_peered":"2023-10-03T10:09:44.199237+0000","last_clean":"2023-10-03T10:09:44.199237+0000","last_became_active":"2023-10-03T10:09:43.758061+0000","last_became_peered":"2023-10-03T10:09:43.758061+0000","last_unstale":"2023-10-03T10:09:44.199237+0000","last_undegraded":"2023-10-03T10:09:44.199237+0000","last_fullsized":"2023-10-03T10:09:44.199237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196006+0000","last_change":"2023-10-03T10:09:44.191564+0000","last_active":"2023-10-03T10:09:44.196006+0000","last_peered":"2023-10-03T10:09:44.196006+0000","last_clean":"2023-10-03T10:09:44.196006+0000","last_became_active":"2023-10-03T10:09:44.191220+0000","last_became_peered":"2023-10-03T10:09:44.191220+0000","last_unstale":"2023-10-03T10:09:44.196006+0000","last_undegraded":"2023-10-03T10:09:44.196006+0000","last_fullsized":"2023-10-03T10:09:44.196006+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"16'4","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.183675+0000","last_change":"2023-10-03T10:09:43.750463+0000","last_active":"2023-10-03T10:09:44.183675+0000","last_peered":"2023-10-03T10:09:44.183675+0000","last_clean":"2023-10-03T10:09:44.183675+0000","last_became_active":"2023-10-03T10:09:43.749780+0000","last_became_peered":"2023-10-03T10:09:43.749780+0000","last_unstale":"2023-10-03T10:09:44.183675+0000","last_undegraded":"2023-10-03T10:09:44.183675+0000","last_fullsized":"2023-10-03T10:09:44.183675+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.187808+0000","last_change":"2023-10-03T10:09:44.178067+0000","last_active":"2023-10-03T10:09:44.187808+0000","last_peered":"2023-10-03T10:09:44.187808+0000","last_clean":"2023-10-03T10:09:44.187808+0000","last_became_active":"2023-10-03T10:09:44.175124+0000","last_became_peered":"2023-10-03T10:09:44.175124+0000","last_unstale":"2023-10-03T10:09:44.187808+0000","last_undegraded":"2023-10-03T10:09:44.187808+0000","last_fullsized":"2023-10-03T10:09:44.187808+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.197083+0000","last_change":"2023-10-03T10:09:44.192156+0000","last_active":"2023-10-03T10:09:44.197083+0000","last_peered":"2023-10-03T10:09:44.197083+0000","last_clean":"2023-10-03T10:09:44.197083+0000","last_became_active":"2023-10-03T10:09:44.189296+0000","last_became_peered":"2023-10-03T10:09:44.189296+0000","last_unstale":"2023-10-03T10:09:44.197083+0000","last_undegraded":"2023-10-03T10:09:44.197083+0000","last_fullsized":"2023-10-03T10:09:44.197083+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.174106+0000","last_change":"2023-10-03T10:09:44.168473+0000","last_active":"2023-10-03T10:09:44.174106+0000","last_peered":"2023-10-03T10:09:44.174106+0000","last_clean":"2023-10-03T10:09:44.174106+0000","last_became_active":"2023-10-03T10:09:44.168155+0000","last_became_peered":"2023-10-03T10:09:44.168155+0000","last_unstale":"2023-10-03T10:09:44.174106+0000","last_undegraded":"2023-10-03T10:09:44.174106+0000","last_fullsized":"2023-10-03T10:09:44.174106+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200881+0000","last_change":"2023-10-03T10:09:44.197689+0000","last_active":"2023-10-03T10:09:44.200881+0000","last_peered":"2023-10-03T10:09:44.200881+0000","last_clean":"2023-10-03T10:09:44.200881+0000","last_became_active":"2023-10-03T10:09:44.197410+0000","last_became_peered":"2023-10-03T10:09:44.197410+0000","last_unstale":"2023-10-03T10:09:44.200881+0000","last_undegraded":"2023-10-03T10:09:44.200881+0000","last_fullsized":"2023-10-03T10:09:44.200881+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184190+0000","last_change":"2023-10-03T10:09:44.176966+0000","last_active":"2023-10-03T10:09:44.184190+0000","last_peered":"2023-10-03T10:09:44.184190+0000","last_clean":"2023-10-03T10:09:44.184190+0000","last_became_active":"2023-10-03T10:09:44.176386+0000","last_became_peered":"2023-10-03T10:09:44.176386+0000","last_unstale":"2023-10-03T10:09:44.184190+0000","last_undegraded":"2023-10-03T10:09:44.184190+0000","last_fullsized":"2023-10-03T10:09:44.184190+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":102,"acting":102,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":3076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":58,"acting":58,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705666,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297200,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074640,"statfs":{"total":96636764160,"available":96332431360,"internally_reserved":0,"allocated":106496,"data_stored":26836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]}]},{"osd":3,"up_from":9,"seq":38654705667,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297264,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074576,"statfs":{"total":96636764160,"available":96332365824,"internally_reserved":0,"allocated":172032,"data_stored":48929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]}]},{"osd":0,"up_from":9,"seq":38654705667,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297256,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074584,"statfs":{"total":96636764160,"available":96332374016,"internally_reserved":0,"allocated":163840,"data_stored":47899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]}]},{"osd":1,"up_from":9,"seq":38654705667,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297252,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074588,"statfs":{"total":96636764160,"available":96332378112,"internally_reserved":0,"allocated":159744,"data_stored":47845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-03T10:09:48.607 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-03T10:09:48.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.775+0000 7f3b69aea700 1 Processor -- start 2023-10-03T10:09:48.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.775+0000 7f3b69aea700 1 -- start start 2023-10-03T10:09:48.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.776+0000 7f3b69aea700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b64093810 0x7f3b64095300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:48.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.776+0000 7f3b69aea700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3b640957f0 0x7f3b64095bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:48.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.776+0000 7f3b69aea700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b6412c110 0x7f3b6412e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:48.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.776+0000 7f3b69aea700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b64093bf0 con 0x7f3b640957f0 2023-10-03T10:09:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.776+0000 7f3b69aea700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b64093010 con 0x7f3b64093810 2023-10-03T10:09:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.776+0000 7f3b69aea700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3b64096e90 con 0x7f3b6412c110 2023-10-03T10:09:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.776+0000 7f3b637fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b64093810 0x7f3b64095300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.776+0000 7f3b63fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b6412c110 0x7f3b6412e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.776+0000 7f3b62ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3b640957f0 0x7f3b64095bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.776+0000 7f3b637fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b64093810 0x7f3b64095300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34592/0 (socket says 172.21.15.3:34592) 2023-10-03T10:09:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.776+0000 7f3b63fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b6412c110 0x7f3b6412e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51086/0 (socket says 172.21.15.3:51086) 2023-10-03T10:09:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.776+0000 7f3b637fe700 1 -- 172.21.15.3:0/1045089780 learned_addr learned my addr 172.21.15.3:0/1045089780 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.776+0000 7f3b63fff700 1 -- 172.21.15.3:0/1045089780 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b64093810 msgr2=0x7f3b64095300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.776+0000 7f3b63fff700 1 --2- 172.21.15.3:0/1045089780 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b64093810 0x7f3b64095300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.776+0000 7f3b63fff700 1 -- 172.21.15.3:0/1045089780 >> v2:172.21.15.3:3300/0 conn(0x7f3b640957f0 msgr2=0x7f3b64095bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.776+0000 7f3b63fff700 1 --2- 172.21.15.3:0/1045089780 >> v2:172.21.15.3:3300/0 conn(0x7f3b640957f0 0x7f3b64095bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.776+0000 7f3b63fff700 1 -- 172.21.15.3:0/1045089780 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b64092aa0 con 0x7f3b6412c110 2023-10-03T10:09:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.776+0000 7f3b62ffd700 1 --2- 172.21.15.3:0/1045089780 >> v2:172.21.15.3:3300/0 conn(0x7f3b640957f0 0x7f3b64095bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.776+0000 7f3b63fff700 1 --2- 172.21.15.3:0/1045089780 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b6412c110 0x7f3b6412e500 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f3b5c009e90 tx=0x7f3b5c009780).ready entity=mon.2 client_cookie=cb237fe562b8c9b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.777+0000 7f3b627fc700 1 -- 172.21.15.3:0/1045089780 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c00e070 con 0x7f3b6412c110 2023-10-03T10:09:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.777+0000 7f3b627fc700 1 -- 172.21.15.3:0/1045089780 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b5c00ba70 con 0x7f3b6412c110 2023-10-03T10:09:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.777+0000 7f3b627fc700 1 -- 172.21.15.3:0/1045089780 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c00bdf0 con 0x7f3b6412c110 2023-10-03T10:09:48.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.777+0000 7f3b69aea700 1 -- 172.21.15.3:0/1045089780 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b6412c110 msgr2=0x7f3b6412e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:48.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.777+0000 7f3b69aea700 1 --2- 172.21.15.3:0/1045089780 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b6412c110 0x7f3b6412e500 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f3b5c009e90 tx=0x7f3b5c009780).stop 2023-10-03T10:09:48.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.777+0000 7f3b69aea700 1 -- 172.21.15.3:0/1045089780 shutdown_connections 2023-10-03T10:09:48.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.777+0000 7f3b69aea700 1 --2- 172.21.15.3:0/1045089780 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b6412c110 0x7f3b6412e500 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:48.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.777+0000 7f3b69aea700 1 --2- 172.21.15.3:0/1045089780 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b64093810 0x7f3b64095300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:48.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.777+0000 7f3b69aea700 1 --2- 172.21.15.3:0/1045089780 >> v2:172.21.15.3:3300/0 conn(0x7f3b640957f0 0x7f3b64095bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:48.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.777+0000 7f3b69aea700 1 -- 172.21.15.3:0/1045089780 >> 172.21.15.3:0/1045089780 conn(0x7f3b64059bb0 msgr2=0x7f3b64097400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:48.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.777+0000 7f3b69aea700 1 -- 172.21.15.3:0/1045089780 shutdown_connections 2023-10-03T10:09:48.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.777+0000 7f3b69aea700 1 -- 172.21.15.3:0/1045089780 wait complete. 2023-10-03T10:09:48.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.778+0000 7f3b69aea700 1 Processor -- start 2023-10-03T10:09:48.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.778+0000 7f3b69aea700 1 -- start start 2023-10-03T10:09:48.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.779+0000 7f3b69aea700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b64093810 0x7f3b64066690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:48.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.779+0000 7f3b69aea700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b640957f0 0x7f3b64066b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:48.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.779+0000 7f3b69aea700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b6412c110 0x7f3b64067070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:48.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.779+0000 7f3b69aea700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b64067560 con 0x7f3b64093810 2023-10-03T10:09:48.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.779+0000 7f3b69aea700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b64067b40 con 0x7f3b6412c110 2023-10-03T10:09:48.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.779+0000 7f3b69aea700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6411d380 con 0x7f3b640957f0 2023-10-03T10:09:48.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.779+0000 7f3b637fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b64093810 0x7f3b64066690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:48.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.779+0000 7f3b637fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b64093810 0x7f3b64066690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35514/0 (socket says 172.21.15.3:35514) 2023-10-03T10:09:48.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.779+0000 7f3b63fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b6412c110 0x7f3b64067070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:48.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.779+0000 7f3b637fe700 1 -- 172.21.15.3:0/298905022 learned_addr learned my addr 172.21.15.3:0/298905022 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:48.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.779+0000 7f3b62ffd700 1 --2- 172.21.15.3:0/298905022 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b640957f0 0x7f3b64066b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:48.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.779+0000 7f3b637fe700 1 -- 172.21.15.3:0/298905022 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b640957f0 msgr2=0x7f3b64066b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:48.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.779+0000 7f3b637fe700 1 --2- 172.21.15.3:0/298905022 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b640957f0 0x7f3b64066b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:48.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.779+0000 7f3b637fe700 1 -- 172.21.15.3:0/298905022 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b6412c110 msgr2=0x7f3b64067070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:48.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.779+0000 7f3b637fe700 1 --2- 172.21.15.3:0/298905022 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b6412c110 0x7f3b64067070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:48.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.779+0000 7f3b637fe700 1 -- 172.21.15.3:0/298905022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b580092a0 con 0x7f3b64093810 2023-10-03T10:09:48.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.779+0000 7f3b63fff700 1 --2- 172.21.15.3:0/298905022 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b6412c110 0x7f3b64067070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:48.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.779+0000 7f3b637fe700 1 --2- 172.21.15.3:0/298905022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b64093810 0x7f3b64066690 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f3b5400e900 tx=0x7f3b540090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:48.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.780+0000 7f3b4b7fe700 1 -- 172.21.15.3:0/298905022 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b54012600 con 0x7f3b64093810 2023-10-03T10:09:48.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.780+0000 7f3b4b7fe700 1 -- 172.21.15.3:0/298905022 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b54012760 con 0x7f3b64093810 2023-10-03T10:09:48.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.780+0000 7f3b4b7fe700 1 -- 172.21.15.3:0/298905022 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b54012ae0 con 0x7f3b64093810 2023-10-03T10:09:48.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.780+0000 7f3b69aea700 1 -- 172.21.15.3:0/298905022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b5c009a20 con 0x7f3b64093810 2023-10-03T10:09:48.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.780+0000 7f3b69aea700 1 -- 172.21.15.3:0/298905022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b6406a6a0 con 0x7f3b64093810 2023-10-03T10:09:48.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.780+0000 7f3b4b7fe700 1 -- 172.21.15.3:0/298905022 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f3b5401b400 con 0x7f3b64093810 2023-10-03T10:09:48.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.781+0000 7f3b4b7fe700 1 -- 172.21.15.3:0/298905022 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f3b5403b3b0 con 0x7f3b64093810 2023-10-03T10:09:48.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.781+0000 7f3b69aea700 1 -- 172.21.15.3:0/298905022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b6405a260 con 0x7f3b64093810 2023-10-03T10:09:48.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.786+0000 7f3b4b7fe700 1 -- 172.21.15.3:0/298905022 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f3b54019310 con 0x7f3b64093810 2023-10-03T10:09:48.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.960+0000 7f3b69aea700 1 -- 172.21.15.3:0/298905022 --> v1:172.21.15.3:6816/118296 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3b6411d880 con 0x7f3b5002bb60 2023-10-03T10:09:48.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.967+0000 7f3b4b7fe700 1 -- 172.21.15.3:0/298905022 <== mgr.4100 v1:172.21.15.3:6816/118296 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248601 (unknown 2965378022 0 4201081181) 0x7f3b58009830 con 0x7f3b5002bb60 2023-10-03T10:09:48.970 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:48.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.970+0000 7f3b69aea700 1 -- 172.21.15.3:0/298905022 >> v1:172.21.15.3:6816/118296 conn(0x7f3b5002bb60 legacy=0x7f3b5002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:48.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.970+0000 7f3b69aea700 1 -- 172.21.15.3:0/298905022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b64093810 msgr2=0x7f3b64066690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:48.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.970+0000 7f3b69aea700 1 --2- 172.21.15.3:0/298905022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b64093810 0x7f3b64066690 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f3b5400e900 tx=0x7f3b540090f0).stop 2023-10-03T10:09:48.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.970+0000 7f3b69aea700 1 -- 172.21.15.3:0/298905022 shutdown_connections 2023-10-03T10:09:48.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.970+0000 7f3b69aea700 1 --2- 172.21.15.3:0/298905022 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b6412c110 0x7f3b64067070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:48.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.970+0000 7f3b69aea700 1 --2- 172.21.15.3:0/298905022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b64093810 0x7f3b64066690 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:48.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.970+0000 7f3b69aea700 1 --2- 172.21.15.3:0/298905022 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b640957f0 0x7f3b64066b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:48.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.970+0000 7f3b69aea700 1 -- 172.21.15.3:0/298905022 >> 172.21.15.3:0/298905022 conn(0x7f3b64059bb0 msgr2=0x7f3b64098d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:48.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.970+0000 7f3b69aea700 1 -- 172.21.15.3:0/298905022 shutdown_connections 2023-10-03T10:09:48.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:48.970+0000 7f3b69aea700 1 -- 172.21.15.3:0/298905022 wait complete. 2023-10-03T10:09:48.973 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-03T10:09:48.983 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-10-03T10:09:47.529155+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":178,"acting":178,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1188972,"kb_used_data":588,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298388,"statfs":{"total":386547056640,"available":385329549312,"internally_reserved":0,"allocated":602112,"data_stored":171509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.190395+0000","last_change":"2023-10-03T10:09:44.183364+0000","last_active":"2023-10-03T10:09:44.190395+0000","last_peered":"2023-10-03T10:09:44.190395+0000","last_clean":"2023-10-03T10:09:44.190395+0000","last_became_active":"2023-10-03T10:09:44.183094+0000","last_became_peered":"2023-10-03T10:09:44.183094+0000","last_unstale":"2023-10-03T10:09:44.190395+0000","last_undegraded":"2023-10-03T10:09:44.190395+0000","last_fullsized":"2023-10-03T10:09:44.190395+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.190339+0000","last_change":"2023-10-03T10:09:44.183273+0000","last_active":"2023-10-03T10:09:44.190339+0000","last_peered":"2023-10-03T10:09:44.190339+0000","last_clean":"2023-10-03T10:09:44.190339+0000","last_became_active":"2023-10-03T10:09:44.182451+0000","last_became_peered":"2023-10-03T10:09:44.182451+0000","last_unstale":"2023-10-03T10:09:44.190339+0000","last_undegraded":"2023-10-03T10:09:44.190339+0000","last_fullsized":"2023-10-03T10:09:44.190339+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.174191+0000","last_change":"2023-10-03T10:09:44.168483+0000","last_active":"2023-10-03T10:09:44.174191+0000","last_peered":"2023-10-03T10:09:44.174191+0000","last_clean":"2023-10-03T10:09:44.174191+0000","last_became_active":"2023-10-03T10:09:44.168186+0000","last_became_peered":"2023-10-03T10:09:44.168186+0000","last_unstale":"2023-10-03T10:09:44.174191+0000","last_undegraded":"2023-10-03T10:09:44.174191+0000","last_fullsized":"2023-10-03T10:09:44.174191+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.198209+0000","last_change":"2023-10-03T10:09:44.192967+0000","last_active":"2023-10-03T10:09:44.198209+0000","last_peered":"2023-10-03T10:09:44.198209+0000","last_clean":"2023-10-03T10:09:44.198209+0000","last_became_active":"2023-10-03T10:09:44.190979+0000","last_became_peered":"2023-10-03T10:09:44.190979+0000","last_unstale":"2023-10-03T10:09:44.198209+0000","last_undegraded":"2023-10-03T10:09:44.198209+0000","last_fullsized":"2023-10-03T10:09:44.198209+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.192360+0000","last_change":"2023-10-03T10:09:44.183121+0000","last_active":"2023-10-03T10:09:44.192360+0000","last_peered":"2023-10-03T10:09:44.192360+0000","last_clean":"2023-10-03T10:09:44.192360+0000","last_became_active":"2023-10-03T10:09:44.182647+0000","last_became_peered":"2023-10-03T10:09:44.182647+0000","last_unstale":"2023-10-03T10:09:44.192360+0000","last_undegraded":"2023-10-03T10:09:44.192360+0000","last_fullsized":"2023-10-03T10:09:44.192360+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.187765+0000","last_change":"2023-10-03T10:09:44.173123+0000","last_active":"2023-10-03T10:09:44.187765+0000","last_peered":"2023-10-03T10:09:44.187765+0000","last_clean":"2023-10-03T10:09:44.187765+0000","last_became_active":"2023-10-03T10:09:44.172353+0000","last_became_peered":"2023-10-03T10:09:44.172353+0000","last_unstale":"2023-10-03T10:09:44.187765+0000","last_undegraded":"2023-10-03T10:09:44.187765+0000","last_fullsized":"2023-10-03T10:09:44.187765+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173973+0000","last_change":"2023-10-03T10:09:44.166758+0000","last_active":"2023-10-03T10:09:44.173973+0000","last_peered":"2023-10-03T10:09:44.173973+0000","last_clean":"2023-10-03T10:09:44.173973+0000","last_became_active":"2023-10-03T10:09:44.166415+0000","last_became_peered":"2023-10-03T10:09:44.166415+0000","last_unstale":"2023-10-03T10:09:44.173973+0000","last_undegraded":"2023-10-03T10:09:44.173973+0000","last_fullsized":"2023-10-03T10:09:44.173973+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180446+0000","last_change":"2023-10-03T10:09:44.176833+0000","last_active":"2023-10-03T10:09:44.180446+0000","last_peered":"2023-10-03T10:09:44.180446+0000","last_clean":"2023-10-03T10:09:44.180446+0000","last_became_active":"2023-10-03T10:09:44.176526+0000","last_became_peered":"2023-10-03T10:09:44.176526+0000","last_unstale":"2023-10-03T10:09:44.180446+0000","last_undegraded":"2023-10-03T10:09:44.180446+0000","last_fullsized":"2023-10-03T10:09:44.180446+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180114+0000","last_change":"2023-10-03T10:09:44.175868+0000","last_active":"2023-10-03T10:09:44.180114+0000","last_peered":"2023-10-03T10:09:44.180114+0000","last_clean":"2023-10-03T10:09:44.180114+0000","last_became_active":"2023-10-03T10:09:44.175414+0000","last_became_peered":"2023-10-03T10:09:44.175414+0000","last_unstale":"2023-10-03T10:09:44.180114+0000","last_undegraded":"2023-10-03T10:09:44.180114+0000","last_fullsized":"2023-10-03T10:09:44.180114+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180522+0000","last_change":"2023-10-03T10:09:44.176456+0000","last_active":"2023-10-03T10:09:44.180522+0000","last_peered":"2023-10-03T10:09:44.180522+0000","last_clean":"2023-10-03T10:09:44.180522+0000","last_became_active":"2023-10-03T10:09:44.176146+0000","last_became_peered":"2023-10-03T10:09:44.176146+0000","last_unstale":"2023-10-03T10:09:44.180522+0000","last_undegraded":"2023-10-03T10:09:44.180522+0000","last_fullsized":"2023-10-03T10:09:44.180522+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.178380+0000","last_change":"2023-10-03T10:09:44.176614+0000","last_active":"2023-10-03T10:09:44.178380+0000","last_peered":"2023-10-03T10:09:44.178380+0000","last_clean":"2023-10-03T10:09:44.178380+0000","last_became_active":"2023-10-03T10:09:44.176334+0000","last_became_peered":"2023-10-03T10:09:44.176334+0000","last_unstale":"2023-10-03T10:09:44.178380+0000","last_undegraded":"2023-10-03T10:09:44.178380+0000","last_fullsized":"2023-10-03T10:09:44.178380+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.192741+0000","last_change":"2023-10-03T10:09:44.179440+0000","last_active":"2023-10-03T10:09:44.192741+0000","last_peered":"2023-10-03T10:09:44.192741+0000","last_clean":"2023-10-03T10:09:44.192741+0000","last_became_active":"2023-10-03T10:09:44.179150+0000","last_became_peered":"2023-10-03T10:09:44.179150+0000","last_unstale":"2023-10-03T10:09:44.192741+0000","last_undegraded":"2023-10-03T10:09:44.192741+0000","last_fullsized":"2023-10-03T10:09:44.192741+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173996+0000","last_change":"2023-10-03T10:09:44.166901+0000","last_active":"2023-10-03T10:09:44.173996+0000","last_peered":"2023-10-03T10:09:44.173996+0000","last_clean":"2023-10-03T10:09:44.173996+0000","last_became_active":"2023-10-03T10:09:44.166610+0000","last_became_peered":"2023-10-03T10:09:44.166610+0000","last_unstale":"2023-10-03T10:09:44.173996+0000","last_undegraded":"2023-10-03T10:09:44.173996+0000","last_fullsized":"2023-10-03T10:09:44.173996+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173643+0000","last_change":"2023-10-03T10:09:44.165366+0000","last_active":"2023-10-03T10:09:44.173643+0000","last_peered":"2023-10-03T10:09:44.173643+0000","last_clean":"2023-10-03T10:09:44.173643+0000","last_became_active":"2023-10-03T10:09:44.165083+0000","last_became_peered":"2023-10-03T10:09:44.165083+0000","last_unstale":"2023-10-03T10:09:44.173643+0000","last_undegraded":"2023-10-03T10:09:44.173643+0000","last_fullsized":"2023-10-03T10:09:44.173643+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184209+0000","last_change":"2023-10-03T10:09:44.177178+0000","last_active":"2023-10-03T10:09:44.184209+0000","last_peered":"2023-10-03T10:09:44.184209+0000","last_clean":"2023-10-03T10:09:44.184209+0000","last_became_active":"2023-10-03T10:09:44.176686+0000","last_became_peered":"2023-10-03T10:09:44.176686+0000","last_unstale":"2023-10-03T10:09:44.184209+0000","last_undegraded":"2023-10-03T10:09:44.184209+0000","last_fullsized":"2023-10-03T10:09:44.184209+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180396+0000","last_change":"2023-10-03T10:09:44.177335+0000","last_active":"2023-10-03T10:09:44.180396+0000","last_peered":"2023-10-03T10:09:44.180396+0000","last_clean":"2023-10-03T10:09:44.180396+0000","last_became_active":"2023-10-03T10:09:44.175202+0000","last_became_peered":"2023-10-03T10:09:44.175202+0000","last_unstale":"2023-10-03T10:09:44.180396+0000","last_undegraded":"2023-10-03T10:09:44.180396+0000","last_fullsized":"2023-10-03T10:09:44.180396+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184702+0000","last_change":"2023-10-03T10:09:44.182207+0000","last_active":"2023-10-03T10:09:44.184702+0000","last_peered":"2023-10-03T10:09:44.184702+0000","last_clean":"2023-10-03T10:09:44.184702+0000","last_became_active":"2023-10-03T10:09:44.181944+0000","last_became_peered":"2023-10-03T10:09:44.181944+0000","last_unstale":"2023-10-03T10:09:44.184702+0000","last_undegraded":"2023-10-03T10:09:44.184702+0000","last_fullsized":"2023-10-03T10:09:44.184702+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.192336+0000","last_change":"2023-10-03T10:09:44.182610+0000","last_active":"2023-10-03T10:09:44.192336+0000","last_peered":"2023-10-03T10:09:44.192336+0000","last_clean":"2023-10-03T10:09:44.192336+0000","last_became_active":"2023-10-03T10:09:44.180732+0000","last_became_peered":"2023-10-03T10:09:44.180732+0000","last_unstale":"2023-10-03T10:09:44.192336+0000","last_undegraded":"2023-10-03T10:09:44.192336+0000","last_fullsized":"2023-10-03T10:09:44.192336+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.187788+0000","last_change":"2023-10-03T10:09:44.173182+0000","last_active":"2023-10-03T10:09:44.187788+0000","last_peered":"2023-10-03T10:09:44.187788+0000","last_clean":"2023-10-03T10:09:44.187788+0000","last_became_active":"2023-10-03T10:09:44.172685+0000","last_became_peered":"2023-10-03T10:09:44.172685+0000","last_unstale":"2023-10-03T10:09:44.187788+0000","last_undegraded":"2023-10-03T10:09:44.187788+0000","last_fullsized":"2023-10-03T10:09:44.187788+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180761+0000","last_change":"2023-10-03T10:09:44.177910+0000","last_active":"2023-10-03T10:09:44.180761+0000","last_peered":"2023-10-03T10:09:44.180761+0000","last_clean":"2023-10-03T10:09:44.180761+0000","last_became_active":"2023-10-03T10:09:44.177618+0000","last_became_peered":"2023-10-03T10:09:44.177618+0000","last_unstale":"2023-10-03T10:09:44.180761+0000","last_undegraded":"2023-10-03T10:09:44.180761+0000","last_fullsized":"2023-10-03T10:09:44.180761+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180424+0000","last_change":"2023-10-03T10:09:44.176287+0000","last_active":"2023-10-03T10:09:44.180424+0000","last_peered":"2023-10-03T10:09:44.180424+0000","last_clean":"2023-10-03T10:09:44.180424+0000","last_became_active":"2023-10-03T10:09:44.175320+0000","last_became_peered":"2023-10-03T10:09:44.175320+0000","last_unstale":"2023-10-03T10:09:44.180424+0000","last_undegraded":"2023-10-03T10:09:44.180424+0000","last_fullsized":"2023-10-03T10:09:44.180424+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.194745+0000","last_change":"2023-10-03T10:09:44.187148+0000","last_active":"2023-10-03T10:09:44.194745+0000","last_peered":"2023-10-03T10:09:44.194745+0000","last_clean":"2023-10-03T10:09:44.194745+0000","last_became_active":"2023-10-03T10:09:44.186523+0000","last_became_peered":"2023-10-03T10:09:44.186523+0000","last_unstale":"2023-10-03T10:09:44.194745+0000","last_undegraded":"2023-10-03T10:09:44.194745+0000","last_fullsized":"2023-10-03T10:09:44.194745+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.189715+0000","last_change":"2023-10-03T10:09:43.745046+0000","last_active":"2023-10-03T10:09:44.189715+0000","last_peered":"2023-10-03T10:09:44.189715+0000","last_clean":"2023-10-03T10:09:44.189715+0000","last_became_active":"2023-10-03T10:09:43.744724+0000","last_became_peered":"2023-10-03T10:09:43.744724+0000","last_unstale":"2023-10-03T10:09:44.189715+0000","last_undegraded":"2023-10-03T10:09:44.189715+0000","last_fullsized":"2023-10-03T10:09:44.189715+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195809+0000","last_change":"2023-10-03T10:09:44.190869+0000","last_active":"2023-10-03T10:09:44.195809+0000","last_peered":"2023-10-03T10:09:44.195809+0000","last_clean":"2023-10-03T10:09:44.195809+0000","last_became_active":"2023-10-03T10:09:44.190207+0000","last_became_peered":"2023-10-03T10:09:44.190207+0000","last_unstale":"2023-10-03T10:09:44.195809+0000","last_undegraded":"2023-10-03T10:09:44.195809+0000","last_fullsized":"2023-10-03T10:09:44.195809+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.152860+0000","last_change":"2023-10-03T10:09:43.750891+0000","last_active":"2023-10-03T10:09:44.152860+0000","last_peered":"2023-10-03T10:09:44.152860+0000","last_clean":"2023-10-03T10:09:44.152860+0000","last_became_active":"2023-10-03T10:09:43.750356+0000","last_became_peered":"2023-10-03T10:09:43.750356+0000","last_unstale":"2023-10-03T10:09:44.152860+0000","last_undegraded":"2023-10-03T10:09:44.152860+0000","last_fullsized":"2023-10-03T10:09:44.152860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184645+0000","last_change":"2023-10-03T10:09:44.183145+0000","last_active":"2023-10-03T10:09:44.184645+0000","last_peered":"2023-10-03T10:09:44.184645+0000","last_clean":"2023-10-03T10:09:44.184645+0000","last_became_active":"2023-10-03T10:09:44.182816+0000","last_became_peered":"2023-10-03T10:09:44.182816+0000","last_unstale":"2023-10-03T10:09:44.184645+0000","last_undegraded":"2023-10-03T10:09:44.184645+0000","last_fullsized":"2023-10-03T10:09:44.184645+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.163680+0000","last_change":"2023-10-03T10:09:43.749289+0000","last_active":"2023-10-03T10:09:44.163680+0000","last_peered":"2023-10-03T10:09:44.163680+0000","last_clean":"2023-10-03T10:09:44.163680+0000","last_became_active":"2023-10-03T10:09:43.748981+0000","last_became_peered":"2023-10-03T10:09:43.748981+0000","last_unstale":"2023-10-03T10:09:44.163680+0000","last_undegraded":"2023-10-03T10:09:44.163680+0000","last_fullsized":"2023-10-03T10:09:44.163680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.197002+0000","last_change":"2023-10-03T10:09:44.192692+0000","last_active":"2023-10-03T10:09:44.197002+0000","last_peered":"2023-10-03T10:09:44.197002+0000","last_clean":"2023-10-03T10:09:44.197002+0000","last_became_active":"2023-10-03T10:09:44.192390+0000","last_became_peered":"2023-10-03T10:09:44.192390+0000","last_unstale":"2023-10-03T10:09:44.197002+0000","last_undegraded":"2023-10-03T10:09:44.197002+0000","last_fullsized":"2023-10-03T10:09:44.197002+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.160877+0000","last_change":"2023-10-03T10:09:43.754802+0000","last_active":"2023-10-03T10:09:44.160877+0000","last_peered":"2023-10-03T10:09:44.160877+0000","last_clean":"2023-10-03T10:09:44.160877+0000","last_became_active":"2023-10-03T10:09:43.754512+0000","last_became_peered":"2023-10-03T10:09:43.754512+0000","last_unstale":"2023-10-03T10:09:44.160877+0000","last_undegraded":"2023-10-03T10:09:44.160877+0000","last_fullsized":"2023-10-03T10:09:44.160877+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196969+0000","last_change":"2023-10-03T10:09:44.192798+0000","last_active":"2023-10-03T10:09:44.196969+0000","last_peered":"2023-10-03T10:09:44.196969+0000","last_clean":"2023-10-03T10:09:44.196969+0000","last_became_active":"2023-10-03T10:09:44.190787+0000","last_became_peered":"2023-10-03T10:09:44.190787+0000","last_unstale":"2023-10-03T10:09:44.196969+0000","last_undegraded":"2023-10-03T10:09:44.196969+0000","last_fullsized":"2023-10-03T10:09:44.196969+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:42.718400+0000","last_change":"2023-10-03T10:09:42.718400+0000","last_active":"2023-10-03T10:09:42.718400+0000","last_peered":"2023-10-03T10:09:42.718400+0000","last_clean":"2023-10-03T10:09:42.718400+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:42.718400+0000","last_undegraded":"2023-10-03T10:09:42.718400+0000","last_fullsized":"2023-10-03T10:09:42.718400+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.194551+0000","last_change":"2023-10-03T10:09:44.184790+0000","last_active":"2023-10-03T10:09:44.194551+0000","last_peered":"2023-10-03T10:09:44.194551+0000","last_clean":"2023-10-03T10:09:44.194551+0000","last_became_active":"2023-10-03T10:09:44.184531+0000","last_became_peered":"2023-10-03T10:09:44.184531+0000","last_unstale":"2023-10-03T10:09:44.194551+0000","last_undegraded":"2023-10-03T10:09:44.194551+0000","last_fullsized":"2023-10-03T10:09:44.194551+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"16'3","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200426+0000","last_change":"2023-10-03T10:09:43.756598+0000","last_active":"2023-10-03T10:09:44.200426+0000","last_peered":"2023-10-03T10:09:44.200426+0000","last_clean":"2023-10-03T10:09:44.200426+0000","last_became_active":"2023-10-03T10:09:43.754974+0000","last_became_peered":"2023-10-03T10:09:43.754974+0000","last_unstale":"2023-10-03T10:09:44.200426+0000","last_undegraded":"2023-10-03T10:09:44.200426+0000","last_fullsized":"2023-10-03T10:09:44.200426+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180499+0000","last_change":"2023-10-03T10:09:44.175504+0000","last_active":"2023-10-03T10:09:44.180499+0000","last_peered":"2023-10-03T10:09:44.180499+0000","last_clean":"2023-10-03T10:09:44.180499+0000","last_became_active":"2023-10-03T10:09:44.175174+0000","last_became_peered":"2023-10-03T10:09:44.175174+0000","last_unstale":"2023-10-03T10:09:44.180499+0000","last_undegraded":"2023-10-03T10:09:44.180499+0000","last_fullsized":"2023-10-03T10:09:44.180499+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"16'6","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.203073+0000","last_change":"2023-10-03T10:09:43.758711+0000","last_active":"2023-10-03T10:09:44.203073+0000","last_peered":"2023-10-03T10:09:44.203073+0000","last_clean":"2023-10-03T10:09:44.203073+0000","last_became_active":"2023-10-03T10:09:43.758392+0000","last_became_peered":"2023-10-03T10:09:43.758392+0000","last_unstale":"2023-10-03T10:09:44.203073+0000","last_undegraded":"2023-10-03T10:09:44.203073+0000","last_fullsized":"2023-10-03T10:09:44.203073+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200762+0000","last_change":"2023-10-03T10:09:43.751877+0000","last_active":"2023-10-03T10:09:44.200762+0000","last_peered":"2023-10-03T10:09:44.200762+0000","last_clean":"2023-10-03T10:09:44.200762+0000","last_became_active":"2023-10-03T10:09:43.749870+0000","last_became_peered":"2023-10-03T10:09:43.749870+0000","last_unstale":"2023-10-03T10:09:44.200762+0000","last_undegraded":"2023-10-03T10:09:44.200762+0000","last_fullsized":"2023-10-03T10:09:44.200762+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195297+0000","last_change":"2023-10-03T10:09:44.190230+0000","last_active":"2023-10-03T10:09:44.195297+0000","last_peered":"2023-10-03T10:09:44.195297+0000","last_clean":"2023-10-03T10:09:44.195297+0000","last_became_active":"2023-10-03T10:09:44.189342+0000","last_became_peered":"2023-10-03T10:09:44.189342+0000","last_unstale":"2023-10-03T10:09:44.195297+0000","last_undegraded":"2023-10-03T10:09:44.195297+0000","last_fullsized":"2023-10-03T10:09:44.195297+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.162475+0000","last_change":"2023-10-03T10:09:43.755271+0000","last_active":"2023-10-03T10:09:44.162475+0000","last_peered":"2023-10-03T10:09:44.162475+0000","last_clean":"2023-10-03T10:09:44.162475+0000","last_became_active":"2023-10-03T10:09:43.754957+0000","last_became_peered":"2023-10-03T10:09:43.754957+0000","last_unstale":"2023-10-03T10:09:44.162475+0000","last_undegraded":"2023-10-03T10:09:44.162475+0000","last_fullsized":"2023-10-03T10:09:44.162475+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.194802+0000","last_change":"2023-10-03T10:09:44.187333+0000","last_active":"2023-10-03T10:09:44.194802+0000","last_peered":"2023-10-03T10:09:44.194802+0000","last_clean":"2023-10-03T10:09:44.194802+0000","last_became_active":"2023-10-03T10:09:44.186824+0000","last_became_peered":"2023-10-03T10:09:44.186824+0000","last_unstale":"2023-10-03T10:09:44.194802+0000","last_undegraded":"2023-10-03T10:09:44.194802+0000","last_fullsized":"2023-10-03T10:09:44.194802+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.163844+0000","last_change":"2023-10-03T10:09:43.758995+0000","last_active":"2023-10-03T10:09:44.163844+0000","last_peered":"2023-10-03T10:09:44.163844+0000","last_clean":"2023-10-03T10:09:44.163844+0000","last_became_active":"2023-10-03T10:09:43.758565+0000","last_became_peered":"2023-10-03T10:09:43.758565+0000","last_unstale":"2023-10-03T10:09:44.163844+0000","last_undegraded":"2023-10-03T10:09:44.163844+0000","last_fullsized":"2023-10-03T10:09:44.163844+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173655+0000","last_change":"2023-10-03T10:09:44.166557+0000","last_active":"2023-10-03T10:09:44.173655+0000","last_peered":"2023-10-03T10:09:44.173655+0000","last_clean":"2023-10-03T10:09:44.173655+0000","last_became_active":"2023-10-03T10:09:44.166300+0000","last_became_peered":"2023-10-03T10:09:44.166300+0000","last_unstale":"2023-10-03T10:09:44.173655+0000","last_undegraded":"2023-10-03T10:09:44.173655+0000","last_fullsized":"2023-10-03T10:09:44.173655+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"16'8","reported_seq":18,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.194737+0000","last_change":"2023-10-03T10:09:43.748084+0000","last_active":"2023-10-03T10:09:44.194737+0000","last_peered":"2023-10-03T10:09:44.194737+0000","last_clean":"2023-10-03T10:09:44.194737+0000","last_became_active":"2023-10-03T10:09:43.747377+0000","last_became_peered":"2023-10-03T10:09:43.747377+0000","last_unstale":"2023-10-03T10:09:44.194737+0000","last_undegraded":"2023-10-03T10:09:44.194737+0000","last_fullsized":"2023-10-03T10:09:44.194737+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195326+0000","last_change":"2023-10-03T10:09:44.190767+0000","last_active":"2023-10-03T10:09:44.195326+0000","last_peered":"2023-10-03T10:09:44.195326+0000","last_clean":"2023-10-03T10:09:44.195326+0000","last_became_active":"2023-10-03T10:09:44.190514+0000","last_became_peered":"2023-10-03T10:09:44.190514+0000","last_unstale":"2023-10-03T10:09:44.195326+0000","last_undegraded":"2023-10-03T10:09:44.195326+0000","last_fullsized":"2023-10-03T10:09:44.195326+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.183000+0000","last_change":"2023-10-03T10:09:43.748461+0000","last_active":"2023-10-03T10:09:44.183000+0000","last_peered":"2023-10-03T10:09:44.183000+0000","last_clean":"2023-10-03T10:09:44.183000+0000","last_became_active":"2023-10-03T10:09:43.747372+0000","last_became_peered":"2023-10-03T10:09:43.747372+0000","last_unstale":"2023-10-03T10:09:44.183000+0000","last_undegraded":"2023-10-03T10:09:44.183000+0000","last_fullsized":"2023-10-03T10:09:44.183000+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196810+0000","last_change":"2023-10-03T10:09:44.192036+0000","last_active":"2023-10-03T10:09:44.196810+0000","last_peered":"2023-10-03T10:09:44.196810+0000","last_clean":"2023-10-03T10:09:44.196810+0000","last_became_active":"2023-10-03T10:09:44.191093+0000","last_became_peered":"2023-10-03T10:09:44.191093+0000","last_unstale":"2023-10-03T10:09:44.196810+0000","last_undegraded":"2023-10-03T10:09:44.196810+0000","last_fullsized":"2023-10-03T10:09:44.196810+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:42.718400+0000","last_change":"2023-10-03T10:09:42.718400+0000","last_active":"2023-10-03T10:09:42.718400+0000","last_peered":"2023-10-03T10:09:42.718400+0000","last_clean":"2023-10-03T10:09:42.718400+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:42.718400+0000","last_undegraded":"2023-10-03T10:09:42.718400+0000","last_fullsized":"2023-10-03T10:09:42.718400+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.179874+0000","last_change":"2023-10-03T10:09:44.172400+0000","last_active":"2023-10-03T10:09:44.179874+0000","last_peered":"2023-10-03T10:09:44.179874+0000","last_clean":"2023-10-03T10:09:44.179874+0000","last_became_active":"2023-10-03T10:09:44.172095+0000","last_became_peered":"2023-10-03T10:09:44.172095+0000","last_unstale":"2023-10-03T10:09:44.179874+0000","last_undegraded":"2023-10-03T10:09:44.179874+0000","last_fullsized":"2023-10-03T10:09:44.179874+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195109+0000","last_change":"2023-10-03T10:09:43.760622+0000","last_active":"2023-10-03T10:09:44.195109+0000","last_peered":"2023-10-03T10:09:44.195109+0000","last_clean":"2023-10-03T10:09:44.195109+0000","last_became_active":"2023-10-03T10:09:43.760371+0000","last_became_peered":"2023-10-03T10:09:43.760371+0000","last_unstale":"2023-10-03T10:09:44.195109+0000","last_undegraded":"2023-10-03T10:09:44.195109+0000","last_fullsized":"2023-10-03T10:09:44.195109+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.157166+0000","last_change":"2023-10-03T10:09:43.756885+0000","last_active":"2023-10-03T10:09:44.157166+0000","last_peered":"2023-10-03T10:09:44.157166+0000","last_clean":"2023-10-03T10:09:44.157166+0000","last_became_active":"2023-10-03T10:09:43.756588+0000","last_became_peered":"2023-10-03T10:09:43.756588+0000","last_unstale":"2023-10-03T10:09:44.157166+0000","last_undegraded":"2023-10-03T10:09:44.157166+0000","last_fullsized":"2023-10-03T10:09:44.157166+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184514+0000","last_change":"2023-10-03T10:09:44.181603+0000","last_active":"2023-10-03T10:09:44.184514+0000","last_peered":"2023-10-03T10:09:44.184514+0000","last_clean":"2023-10-03T10:09:44.184514+0000","last_became_active":"2023-10-03T10:09:44.181125+0000","last_became_peered":"2023-10-03T10:09:44.181125+0000","last_unstale":"2023-10-03T10:09:44.184514+0000","last_undegraded":"2023-10-03T10:09:44.184514+0000","last_fullsized":"2023-10-03T10:09:44.184514+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200822+0000","last_change":"2023-10-03T10:09:43.751873+0000","last_active":"2023-10-03T10:09:44.200822+0000","last_peered":"2023-10-03T10:09:44.200822+0000","last_clean":"2023-10-03T10:09:44.200822+0000","last_became_active":"2023-10-03T10:09:43.751324+0000","last_became_peered":"2023-10-03T10:09:43.751324+0000","last_unstale":"2023-10-03T10:09:44.200822+0000","last_undegraded":"2023-10-03T10:09:44.200822+0000","last_fullsized":"2023-10-03T10:09:44.200822+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.174302+0000","last_change":"2023-10-03T10:09:44.170676+0000","last_active":"2023-10-03T10:09:44.174302+0000","last_peered":"2023-10-03T10:09:44.174302+0000","last_clean":"2023-10-03T10:09:44.174302+0000","last_became_active":"2023-10-03T10:09:44.170398+0000","last_became_peered":"2023-10-03T10:09:44.170398+0000","last_unstale":"2023-10-03T10:09:44.174302+0000","last_undegraded":"2023-10-03T10:09:44.174302+0000","last_fullsized":"2023-10-03T10:09:44.174302+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.166137+0000","last_change":"2023-10-03T10:09:43.760711+0000","last_active":"2023-10-03T10:09:44.166137+0000","last_peered":"2023-10-03T10:09:44.166137+0000","last_clean":"2023-10-03T10:09:44.166137+0000","last_became_active":"2023-10-03T10:09:43.760432+0000","last_became_peered":"2023-10-03T10:09:43.760432+0000","last_unstale":"2023-10-03T10:09:44.166137+0000","last_undegraded":"2023-10-03T10:09:44.166137+0000","last_fullsized":"2023-10-03T10:09:44.166137+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195829+0000","last_change":"2023-10-03T10:09:44.191073+0000","last_active":"2023-10-03T10:09:44.195829+0000","last_peered":"2023-10-03T10:09:44.195829+0000","last_clean":"2023-10-03T10:09:44.195829+0000","last_became_active":"2023-10-03T10:09:44.190218+0000","last_became_peered":"2023-10-03T10:09:44.190218+0000","last_unstale":"2023-10-03T10:09:44.195829+0000","last_undegraded":"2023-10-03T10:09:44.195829+0000","last_fullsized":"2023-10-03T10:09:44.195829+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.163521+0000","last_change":"2023-10-03T10:09:43.758846+0000","last_active":"2023-10-03T10:09:44.163521+0000","last_peered":"2023-10-03T10:09:44.163521+0000","last_clean":"2023-10-03T10:09:44.163521+0000","last_became_active":"2023-10-03T10:09:43.757973+0000","last_became_peered":"2023-10-03T10:09:43.757973+0000","last_unstale":"2023-10-03T10:09:44.163521+0000","last_undegraded":"2023-10-03T10:09:44.163521+0000","last_fullsized":"2023-10-03T10:09:44.163521+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.162993+0000","last_change":"2023-10-03T10:09:43.761068+0000","last_active":"2023-10-03T10:09:44.162993+0000","last_peered":"2023-10-03T10:09:44.162993+0000","last_clean":"2023-10-03T10:09:44.162993+0000","last_became_active":"2023-10-03T10:09:43.760825+0000","last_became_peered":"2023-10-03T10:09:43.760825+0000","last_unstale":"2023-10-03T10:09:44.162993+0000","last_undegraded":"2023-10-03T10:09:44.162993+0000","last_fullsized":"2023-10-03T10:09:44.162993+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180799+0000","last_change":"2023-10-03T10:09:44.178856+0000","last_active":"2023-10-03T10:09:44.180799+0000","last_peered":"2023-10-03T10:09:44.180799+0000","last_clean":"2023-10-03T10:09:44.180799+0000","last_became_active":"2023-10-03T10:09:44.178410+0000","last_became_peered":"2023-10-03T10:09:44.178410+0000","last_unstale":"2023-10-03T10:09:44.180799+0000","last_undegraded":"2023-10-03T10:09:44.180799+0000","last_fullsized":"2023-10-03T10:09:44.180799+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.154127+0000","last_change":"2023-10-03T10:09:39.589080+0000","last_active":"2023-10-03T10:09:44.154127+0000","last_peered":"2023-10-03T10:09:44.154127+0000","last_clean":"2023-10-03T10:09:44.154127+0000","last_became_active":"2023-10-03T10:09:39.588651+0000","last_became_peered":"2023-10-03T10:09:39.588651+0000","last_unstale":"2023-10-03T10:09:44.154127+0000","last_undegraded":"2023-10-03T10:09:44.154127+0000","last_fullsized":"2023-10-03T10:09:44.154127+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:42.718400+0000","last_change":"2023-10-03T10:09:42.718400+0000","last_active":"2023-10-03T10:09:42.718400+0000","last_peered":"2023-10-03T10:09:42.718400+0000","last_clean":"2023-10-03T10:09:42.718400+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:42.718400+0000","last_undegraded":"2023-10-03T10:09:42.718400+0000","last_fullsized":"2023-10-03T10:09:42.718400+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180543+0000","last_change":"2023-10-03T10:09:44.176964+0000","last_active":"2023-10-03T10:09:44.180543+0000","last_peered":"2023-10-03T10:09:44.180543+0000","last_clean":"2023-10-03T10:09:44.180543+0000","last_became_active":"2023-10-03T10:09:44.176671+0000","last_became_peered":"2023-10-03T10:09:44.176671+0000","last_unstale":"2023-10-03T10:09:44.180543+0000","last_undegraded":"2023-10-03T10:09:44.180543+0000","last_fullsized":"2023-10-03T10:09:44.180543+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.166922+0000","last_change":"2023-10-03T10:09:39.586735+0000","last_active":"2023-10-03T10:09:44.166922+0000","last_peered":"2023-10-03T10:09:44.166922+0000","last_clean":"2023-10-03T10:09:44.166922+0000","last_became_active":"2023-10-03T10:09:39.586440+0000","last_became_peered":"2023-10-03T10:09:39.586440+0000","last_unstale":"2023-10-03T10:09:44.166922+0000","last_undegraded":"2023-10-03T10:09:44.166922+0000","last_fullsized":"2023-10-03T10:09:44.166922+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.159439+0000","last_change":"2023-10-03T10:09:43.748458+0000","last_active":"2023-10-03T10:09:44.159439+0000","last_peered":"2023-10-03T10:09:44.159439+0000","last_clean":"2023-10-03T10:09:44.159439+0000","last_became_active":"2023-10-03T10:09:43.747721+0000","last_became_peered":"2023-10-03T10:09:43.747721+0000","last_unstale":"2023-10-03T10:09:44.159439+0000","last_undegraded":"2023-10-03T10:09:44.159439+0000","last_fullsized":"2023-10-03T10:09:44.159439+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184166+0000","last_change":"2023-10-03T10:09:44.176165+0000","last_active":"2023-10-03T10:09:44.184166+0000","last_peered":"2023-10-03T10:09:44.184166+0000","last_clean":"2023-10-03T10:09:44.184166+0000","last_became_active":"2023-10-03T10:09:44.175853+0000","last_became_peered":"2023-10-03T10:09:44.175853+0000","last_unstale":"2023-10-03T10:09:44.184166+0000","last_undegraded":"2023-10-03T10:09:44.184166+0000","last_fullsized":"2023-10-03T10:09:44.184166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-10-03T10:09:39.591637+0000","last_change":"2023-10-03T10:09:39.590114+0000","last_active":"2023-10-03T10:09:39.591637+0000","last_peered":"2023-10-03T10:09:39.591637+0000","last_clean":"2023-10-03T10:09:39.591637+0000","last_became_active":"2023-10-03T10:09:39.589663+0000","last_became_peered":"2023-10-03T10:09:39.589663+0000","last_unstale":"2023-10-03T10:09:39.591637+0000","last_undegraded":"2023-10-03T10:09:39.591637+0000","last_fullsized":"2023-10-03T10:09:39.591637+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.155654+0000","last_change":"2023-10-03T10:09:43.748267+0000","last_active":"2023-10-03T10:09:44.155654+0000","last_peered":"2023-10-03T10:09:44.155654+0000","last_clean":"2023-10-03T10:09:44.155654+0000","last_became_active":"2023-10-03T10:09:43.747789+0000","last_became_peered":"2023-10-03T10:09:43.747789+0000","last_unstale":"2023-10-03T10:09:44.155654+0000","last_undegraded":"2023-10-03T10:09:44.155654+0000","last_fullsized":"2023-10-03T10:09:44.155654+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180773+0000","last_change":"2023-10-03T10:09:44.178680+0000","last_active":"2023-10-03T10:09:44.180773+0000","last_peered":"2023-10-03T10:09:44.180773+0000","last_clean":"2023-10-03T10:09:44.180773+0000","last_became_active":"2023-10-03T10:09:44.178114+0000","last_became_peered":"2023-10-03T10:09:44.178114+0000","last_unstale":"2023-10-03T10:09:44.180773+0000","last_undegraded":"2023-10-03T10:09:44.180773+0000","last_fullsized":"2023-10-03T10:09:44.180773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.167768+0000","last_change":"2023-10-03T10:09:39.587728+0000","last_active":"2023-10-03T10:09:44.167768+0000","last_peered":"2023-10-03T10:09:44.167768+0000","last_clean":"2023-10-03T10:09:44.167768+0000","last_became_active":"2023-10-03T10:09:39.587441+0000","last_became_peered":"2023-10-03T10:09:39.587441+0000","last_unstale":"2023-10-03T10:09:44.167768+0000","last_undegraded":"2023-10-03T10:09:44.167768+0000","last_fullsized":"2023-10-03T10:09:44.167768+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.153191+0000","last_change":"2023-10-03T10:09:38.585668+0000","last_active":"2023-10-03T10:09:44.153191+0000","last_peered":"2023-10-03T10:09:44.153191+0000","last_clean":"2023-10-03T10:09:44.153191+0000","last_became_active":"2023-10-03T10:09:38.585324+0000","last_became_peered":"2023-10-03T10:09:38.585324+0000","last_unstale":"2023-10-03T10:09:44.153191+0000","last_undegraded":"2023-10-03T10:09:44.153191+0000","last_fullsized":"2023-10-03T10:09:44.153191+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:37.570571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:37.570571+0000","last_clean_scrub_stamp":"2023-10-03T10:09:37.570571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.165999+0000","last_change":"2023-10-03T10:09:43.758096+0000","last_active":"2023-10-03T10:09:44.165999+0000","last_peered":"2023-10-03T10:09:44.165999+0000","last_clean":"2023-10-03T10:09:44.165999+0000","last_became_active":"2023-10-03T10:09:43.757689+0000","last_became_peered":"2023-10-03T10:09:43.757689+0000","last_unstale":"2023-10-03T10:09:44.165999+0000","last_undegraded":"2023-10-03T10:09:44.165999+0000","last_fullsized":"2023-10-03T10:09:44.165999+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196949+0000","last_change":"2023-10-03T10:09:44.192781+0000","last_active":"2023-10-03T10:09:44.196949+0000","last_peered":"2023-10-03T10:09:44.196949+0000","last_clean":"2023-10-03T10:09:44.196949+0000","last_became_active":"2023-10-03T10:09:44.190597+0000","last_became_peered":"2023-10-03T10:09:44.190597+0000","last_unstale":"2023-10-03T10:09:44.196949+0000","last_undegraded":"2023-10-03T10:09:44.196949+0000","last_fullsized":"2023-10-03T10:09:44.196949+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.163365+0000","last_change":"2023-10-03T10:09:39.586806+0000","last_active":"2023-10-03T10:09:44.163365+0000","last_peered":"2023-10-03T10:09:44.163365+0000","last_clean":"2023-10-03T10:09:44.163365+0000","last_became_active":"2023-10-03T10:09:39.586481+0000","last_became_peered":"2023-10-03T10:09:39.586481+0000","last_unstale":"2023-10-03T10:09:44.163365+0000","last_undegraded":"2023-10-03T10:09:44.163365+0000","last_fullsized":"2023-10-03T10:09:44.163365+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.156596+0000","last_change":"2023-10-03T10:09:43.747783+0000","last_active":"2023-10-03T10:09:44.156596+0000","last_peered":"2023-10-03T10:09:44.156596+0000","last_clean":"2023-10-03T10:09:44.156596+0000","last_became_active":"2023-10-03T10:09:43.746009+0000","last_became_peered":"2023-10-03T10:09:43.746009+0000","last_unstale":"2023-10-03T10:09:44.156596+0000","last_undegraded":"2023-10-03T10:09:44.156596+0000","last_fullsized":"2023-10-03T10:09:44.156596+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184590+0000","last_change":"2023-10-03T10:09:44.181359+0000","last_active":"2023-10-03T10:09:44.184590+0000","last_peered":"2023-10-03T10:09:44.184590+0000","last_clean":"2023-10-03T10:09:44.184590+0000","last_became_active":"2023-10-03T10:09:44.181075+0000","last_became_peered":"2023-10-03T10:09:44.181075+0000","last_unstale":"2023-10-03T10:09:44.184590+0000","last_undegraded":"2023-10-03T10:09:44.184590+0000","last_fullsized":"2023-10-03T10:09:44.184590+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.168332+0000","last_change":"2023-10-03T10:09:39.588827+0000","last_active":"2023-10-03T10:09:44.168332+0000","last_peered":"2023-10-03T10:09:44.168332+0000","last_clean":"2023-10-03T10:09:44.168332+0000","last_became_active":"2023-10-03T10:09:39.588531+0000","last_became_peered":"2023-10-03T10:09:39.588531+0000","last_unstale":"2023-10-03T10:09:44.168332+0000","last_undegraded":"2023-10-03T10:09:44.168332+0000","last_fullsized":"2023-10-03T10:09:44.168332+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.152966+0000","last_change":"2023-10-03T10:09:43.746339+0000","last_active":"2023-10-03T10:09:44.152966+0000","last_peered":"2023-10-03T10:09:44.152966+0000","last_clean":"2023-10-03T10:09:44.152966+0000","last_became_active":"2023-10-03T10:09:43.746051+0000","last_became_peered":"2023-10-03T10:09:43.746051+0000","last_unstale":"2023-10-03T10:09:44.152966+0000","last_undegraded":"2023-10-03T10:09:44.152966+0000","last_fullsized":"2023-10-03T10:09:44.152966+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173149+0000","last_change":"2023-10-03T10:09:44.165684+0000","last_active":"2023-10-03T10:09:44.173149+0000","last_peered":"2023-10-03T10:09:44.173149+0000","last_clean":"2023-10-03T10:09:44.173149+0000","last_became_active":"2023-10-03T10:09:44.165422+0000","last_became_peered":"2023-10-03T10:09:44.165422+0000","last_unstale":"2023-10-03T10:09:44.173149+0000","last_undegraded":"2023-10-03T10:09:44.173149+0000","last_fullsized":"2023-10-03T10:09:44.173149+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.197330+0000","last_change":"2023-10-03T10:09:43.760807+0000","last_active":"2023-10-03T10:09:44.197330+0000","last_peered":"2023-10-03T10:09:44.197330+0000","last_clean":"2023-10-03T10:09:44.197330+0000","last_became_active":"2023-10-03T10:09:43.760517+0000","last_became_peered":"2023-10-03T10:09:43.760517+0000","last_unstale":"2023-10-03T10:09:44.197330+0000","last_undegraded":"2023-10-03T10:09:44.197330+0000","last_fullsized":"2023-10-03T10:09:44.197330+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.153815+0000","last_change":"2023-10-03T10:09:39.586702+0000","last_active":"2023-10-03T10:09:44.153815+0000","last_peered":"2023-10-03T10:09:44.153815+0000","last_clean":"2023-10-03T10:09:44.153815+0000","last_became_active":"2023-10-03T10:09:39.586153+0000","last_became_peered":"2023-10-03T10:09:39.586153+0000","last_unstale":"2023-10-03T10:09:44.153815+0000","last_undegraded":"2023-10-03T10:09:44.153815+0000","last_fullsized":"2023-10-03T10:09:44.153815+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196883+0000","last_change":"2023-10-03T10:09:44.192664+0000","last_active":"2023-10-03T10:09:44.196883+0000","last_peered":"2023-10-03T10:09:44.196883+0000","last_clean":"2023-10-03T10:09:44.196883+0000","last_became_active":"2023-10-03T10:09:44.190513+0000","last_became_peered":"2023-10-03T10:09:44.190513+0000","last_unstale":"2023-10-03T10:09:44.196883+0000","last_undegraded":"2023-10-03T10:09:44.196883+0000","last_fullsized":"2023-10-03T10:09:44.196883+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.167083+0000","last_change":"2023-10-03T10:09:39.588950+0000","last_active":"2023-10-03T10:09:44.167083+0000","last_peered":"2023-10-03T10:09:44.167083+0000","last_clean":"2023-10-03T10:09:44.167083+0000","last_became_active":"2023-10-03T10:09:39.588601+0000","last_became_peered":"2023-10-03T10:09:39.588601+0000","last_unstale":"2023-10-03T10:09:44.167083+0000","last_undegraded":"2023-10-03T10:09:44.167083+0000","last_fullsized":"2023-10-03T10:09:44.167083+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.160538+0000","last_change":"2023-10-03T10:09:43.752793+0000","last_active":"2023-10-03T10:09:44.160538+0000","last_peered":"2023-10-03T10:09:44.160538+0000","last_clean":"2023-10-03T10:09:44.160538+0000","last_became_active":"2023-10-03T10:09:43.752374+0000","last_became_peered":"2023-10-03T10:09:43.752374+0000","last_unstale":"2023-10-03T10:09:44.160538+0000","last_undegraded":"2023-10-03T10:09:44.160538+0000","last_fullsized":"2023-10-03T10:09:44.160538+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200135+0000","last_change":"2023-10-03T10:09:43.756460+0000","last_active":"2023-10-03T10:09:44.200135+0000","last_peered":"2023-10-03T10:09:44.200135+0000","last_clean":"2023-10-03T10:09:44.200135+0000","last_became_active":"2023-10-03T10:09:43.756164+0000","last_became_peered":"2023-10-03T10:09:43.756164+0000","last_unstale":"2023-10-03T10:09:44.200135+0000","last_undegraded":"2023-10-03T10:09:44.200135+0000","last_fullsized":"2023-10-03T10:09:44.200135+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.193051+0000","last_change":"2023-10-03T10:09:43.748380+0000","last_active":"2023-10-03T10:09:44.193051+0000","last_peered":"2023-10-03T10:09:44.193051+0000","last_clean":"2023-10-03T10:09:44.193051+0000","last_became_active":"2023-10-03T10:09:43.748104+0000","last_became_peered":"2023-10-03T10:09:43.748104+0000","last_unstale":"2023-10-03T10:09:44.193051+0000","last_undegraded":"2023-10-03T10:09:44.193051+0000","last_fullsized":"2023-10-03T10:09:44.193051+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.193435+0000","last_change":"2023-10-03T10:09:43.758701+0000","last_active":"2023-10-03T10:09:44.193435+0000","last_peered":"2023-10-03T10:09:44.193435+0000","last_clean":"2023-10-03T10:09:44.193435+0000","last_became_active":"2023-10-03T10:09:43.758376+0000","last_became_peered":"2023-10-03T10:09:43.758376+0000","last_unstale":"2023-10-03T10:09:44.193435+0000","last_undegraded":"2023-10-03T10:09:44.193435+0000","last_fullsized":"2023-10-03T10:09:44.193435+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.199237+0000","last_change":"2023-10-03T10:09:43.758373+0000","last_active":"2023-10-03T10:09:44.199237+0000","last_peered":"2023-10-03T10:09:44.199237+0000","last_clean":"2023-10-03T10:09:44.199237+0000","last_became_active":"2023-10-03T10:09:43.758061+0000","last_became_peered":"2023-10-03T10:09:43.758061+0000","last_unstale":"2023-10-03T10:09:44.199237+0000","last_undegraded":"2023-10-03T10:09:44.199237+0000","last_fullsized":"2023-10-03T10:09:44.199237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196006+0000","last_change":"2023-10-03T10:09:44.191564+0000","last_active":"2023-10-03T10:09:44.196006+0000","last_peered":"2023-10-03T10:09:44.196006+0000","last_clean":"2023-10-03T10:09:44.196006+0000","last_became_active":"2023-10-03T10:09:44.191220+0000","last_became_peered":"2023-10-03T10:09:44.191220+0000","last_unstale":"2023-10-03T10:09:44.196006+0000","last_undegraded":"2023-10-03T10:09:44.196006+0000","last_fullsized":"2023-10-03T10:09:44.196006+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"16'4","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.183675+0000","last_change":"2023-10-03T10:09:43.750463+0000","last_active":"2023-10-03T10:09:44.183675+0000","last_peered":"2023-10-03T10:09:44.183675+0000","last_clean":"2023-10-03T10:09:44.183675+0000","last_became_active":"2023-10-03T10:09:43.749780+0000","last_became_peered":"2023-10-03T10:09:43.749780+0000","last_unstale":"2023-10-03T10:09:44.183675+0000","last_undegraded":"2023-10-03T10:09:44.183675+0000","last_fullsized":"2023-10-03T10:09:44.183675+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-03T10:09:43.722422+0000","last_change":"2023-10-03T10:09:43.722422+0000","last_active":"2023-10-03T10:09:43.722422+0000","last_peered":"2023-10-03T10:09:43.722422+0000","last_clean":"2023-10-03T10:09:43.722422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-03T10:09:43.722422+0000","last_undegraded":"2023-10-03T10:09:43.722422+0000","last_fullsized":"2023-10-03T10:09:43.722422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.187808+0000","last_change":"2023-10-03T10:09:44.178067+0000","last_active":"2023-10-03T10:09:44.187808+0000","last_peered":"2023-10-03T10:09:44.187808+0000","last_clean":"2023-10-03T10:09:44.187808+0000","last_became_active":"2023-10-03T10:09:44.175124+0000","last_became_peered":"2023-10-03T10:09:44.175124+0000","last_unstale":"2023-10-03T10:09:44.187808+0000","last_undegraded":"2023-10-03T10:09:44.187808+0000","last_fullsized":"2023-10-03T10:09:44.187808+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.197083+0000","last_change":"2023-10-03T10:09:44.192156+0000","last_active":"2023-10-03T10:09:44.197083+0000","last_peered":"2023-10-03T10:09:44.197083+0000","last_clean":"2023-10-03T10:09:44.197083+0000","last_became_active":"2023-10-03T10:09:44.189296+0000","last_became_peered":"2023-10-03T10:09:44.189296+0000","last_unstale":"2023-10-03T10:09:44.197083+0000","last_undegraded":"2023-10-03T10:09:44.197083+0000","last_fullsized":"2023-10-03T10:09:44.197083+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.174106+0000","last_change":"2023-10-03T10:09:44.168473+0000","last_active":"2023-10-03T10:09:44.174106+0000","last_peered":"2023-10-03T10:09:44.174106+0000","last_clean":"2023-10-03T10:09:44.174106+0000","last_became_active":"2023-10-03T10:09:44.168155+0000","last_became_peered":"2023-10-03T10:09:44.168155+0000","last_unstale":"2023-10-03T10:09:44.174106+0000","last_undegraded":"2023-10-03T10:09:44.174106+0000","last_fullsized":"2023-10-03T10:09:44.174106+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200881+0000","last_change":"2023-10-03T10:09:44.197689+0000","last_active":"2023-10-03T10:09:44.200881+0000","last_peered":"2023-10-03T10:09:44.200881+0000","last_clean":"2023-10-03T10:09:44.200881+0000","last_became_active":"2023-10-03T10:09:44.197410+0000","last_became_peered":"2023-10-03T10:09:44.197410+0000","last_unstale":"2023-10-03T10:09:44.200881+0000","last_undegraded":"2023-10-03T10:09:44.200881+0000","last_fullsized":"2023-10-03T10:09:44.200881+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184190+0000","last_change":"2023-10-03T10:09:44.176966+0000","last_active":"2023-10-03T10:09:44.184190+0000","last_peered":"2023-10-03T10:09:44.184190+0000","last_clean":"2023-10-03T10:09:44.184190+0000","last_became_active":"2023-10-03T10:09:44.176386+0000","last_became_peered":"2023-10-03T10:09:44.176386+0000","last_unstale":"2023-10-03T10:09:44.184190+0000","last_undegraded":"2023-10-03T10:09:44.184190+0000","last_fullsized":"2023-10-03T10:09:44.184190+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":102,"acting":102,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":3076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":58,"acting":58,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705666,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297200,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074640,"statfs":{"total":96636764160,"available":96332431360,"internally_reserved":0,"allocated":106496,"data_stored":26836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]}]},{"osd":3,"up_from":9,"seq":38654705667,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297264,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074576,"statfs":{"total":96636764160,"available":96332365824,"internally_reserved":0,"allocated":172032,"data_stored":48929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]}]},{"osd":0,"up_from":9,"seq":38654705667,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297256,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074584,"statfs":{"total":96636764160,"available":96332374016,"internally_reserved":0,"allocated":163840,"data_stored":47899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]}]},{"osd":1,"up_from":9,"seq":38654705667,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297252,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074588,"statfs":{"total":96636764160,"available":96332378112,"internally_reserved":0,"allocated":159744,"data_stored":47845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-03T10:09:48.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3f is not active+clean 2023-10-03T10:09:48.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-03T10:09:43.722422+0000', 'last_change': '2023-10-03T10:09:43.722422+0000', 'last_active': '2023-10-03T10:09:43.722422+0000', 'last_peered': '2023-10-03T10:09:43.722422+0000', 'last_clean': '2023-10-03T10:09:43.722422+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-03T10:09:43.722422+0000', 'last_undegraded': '2023-10-03T10:09:43.722422+0000', 'last_fullsized': '2023-10-03T10:09:43.722422+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'last_clean_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-03T10:09:48.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3e is not active+clean 2023-10-03T10:09:48.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-03T10:09:43.722422+0000', 'last_change': '2023-10-03T10:09:43.722422+0000', 'last_active': '2023-10-03T10:09:43.722422+0000', 'last_peered': '2023-10-03T10:09:43.722422+0000', 'last_clean': '2023-10-03T10:09:43.722422+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-03T10:09:43.722422+0000', 'last_undegraded': '2023-10-03T10:09:43.722422+0000', 'last_fullsized': '2023-10-03T10:09:43.722422+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'last_clean_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-03T10:09:48.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.34 is not active+clean 2023-10-03T10:09:48.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-03T10:09:43.722422+0000', 'last_change': '2023-10-03T10:09:43.722422+0000', 'last_active': '2023-10-03T10:09:43.722422+0000', 'last_peered': '2023-10-03T10:09:43.722422+0000', 'last_clean': '2023-10-03T10:09:43.722422+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-03T10:09:43.722422+0000', 'last_undegraded': '2023-10-03T10:09:43.722422+0000', 'last_fullsized': '2023-10-03T10:09:43.722422+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'last_clean_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-03T10:09:48.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.33 is not active+clean 2023-10-03T10:09:48.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-03T10:09:43.722422+0000', 'last_change': '2023-10-03T10:09:43.722422+0000', 'last_active': '2023-10-03T10:09:43.722422+0000', 'last_peered': '2023-10-03T10:09:43.722422+0000', 'last_clean': '2023-10-03T10:09:43.722422+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-03T10:09:43.722422+0000', 'last_undegraded': '2023-10-03T10:09:43.722422+0000', 'last_fullsized': '2023-10-03T10:09:43.722422+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'last_clean_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-03T10:09:48.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2d is not active+clean 2023-10-03T10:09:48.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-03T10:09:43.722422+0000', 'last_change': '2023-10-03T10:09:43.722422+0000', 'last_active': '2023-10-03T10:09:43.722422+0000', 'last_peered': '2023-10-03T10:09:43.722422+0000', 'last_clean': '2023-10-03T10:09:43.722422+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-03T10:09:43.722422+0000', 'last_undegraded': '2023-10-03T10:09:43.722422+0000', 'last_fullsized': '2023-10-03T10:09:43.722422+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'last_clean_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-03T10:09:48.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.16 is not active+clean 2023-10-03T10:09:48.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-03T10:09:42.718400+0000', 'last_change': '2023-10-03T10:09:42.718400+0000', 'last_active': '2023-10-03T10:09:42.718400+0000', 'last_peered': '2023-10-03T10:09:42.718400+0000', 'last_clean': '2023-10-03T10:09:42.718400+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-03T10:09:42.718400+0000', 'last_undegraded': '2023-10-03T10:09:42.718400+0000', 'last_fullsized': '2023-10-03T10:09:42.718400+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-03T10:09:42.718400+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-03T10:09:42.718400+0000', 'last_clean_scrub_stamp': '2023-10-03T10:09:42.718400+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-03T10:09:48.990 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.13 is not active+clean 2023-10-03T10:09:48.990 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-03T10:09:43.722422+0000', 'last_change': '2023-10-03T10:09:43.722422+0000', 'last_active': '2023-10-03T10:09:43.722422+0000', 'last_peered': '2023-10-03T10:09:43.722422+0000', 'last_clean': '2023-10-03T10:09:43.722422+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-03T10:09:43.722422+0000', 'last_undegraded': '2023-10-03T10:09:43.722422+0000', 'last_fullsized': '2023-10-03T10:09:43.722422+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'last_clean_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-03T10:09:48.990 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.e is not active+clean 2023-10-03T10:09:48.990 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-03T10:09:42.718400+0000', 'last_change': '2023-10-03T10:09:42.718400+0000', 'last_active': '2023-10-03T10:09:42.718400+0000', 'last_peered': '2023-10-03T10:09:42.718400+0000', 'last_clean': '2023-10-03T10:09:42.718400+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-03T10:09:42.718400+0000', 'last_undegraded': '2023-10-03T10:09:42.718400+0000', 'last_fullsized': '2023-10-03T10:09:42.718400+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-03T10:09:42.718400+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-03T10:09:42.718400+0000', 'last_clean_scrub_stamp': '2023-10-03T10:09:42.718400+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-03T10:09:48.990 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.a is not active+clean 2023-10-03T10:09:48.990 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-03T10:09:43.722422+0000', 'last_change': '2023-10-03T10:09:43.722422+0000', 'last_active': '2023-10-03T10:09:43.722422+0000', 'last_peered': '2023-10-03T10:09:43.722422+0000', 'last_clean': '2023-10-03T10:09:43.722422+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-03T10:09:43.722422+0000', 'last_undegraded': '2023-10-03T10:09:43.722422+0000', 'last_fullsized': '2023-10-03T10:09:43.722422+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'last_clean_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-03T10:09:48.990 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.e is not active+clean 2023-10-03T10:09:48.990 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-03T10:09:43.722422+0000', 'last_change': '2023-10-03T10:09:43.722422+0000', 'last_active': '2023-10-03T10:09:43.722422+0000', 'last_peered': '2023-10-03T10:09:43.722422+0000', 'last_clean': '2023-10-03T10:09:43.722422+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-03T10:09:43.722422+0000', 'last_undegraded': '2023-10-03T10:09:43.722422+0000', 'last_fullsized': '2023-10-03T10:09:43.722422+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'last_clean_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-03T10:09:48.991 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.5 is not active+clean 2023-10-03T10:09:48.991 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-03T10:09:42.718400+0000', 'last_change': '2023-10-03T10:09:42.718400+0000', 'last_active': '2023-10-03T10:09:42.718400+0000', 'last_peered': '2023-10-03T10:09:42.718400+0000', 'last_clean': '2023-10-03T10:09:42.718400+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-03T10:09:42.718400+0000', 'last_undegraded': '2023-10-03T10:09:42.718400+0000', 'last_fullsized': '2023-10-03T10:09:42.718400+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-03T10:09:42.718400+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-03T10:09:42.718400+0000', 'last_clean_scrub_stamp': '2023-10-03T10:09:42.718400+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-03T10:09:48.991 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1 is not active+clean 2023-10-03T10:09:48.991 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-03T10:09:43.722422+0000', 'last_change': '2023-10-03T10:09:43.722422+0000', 'last_active': '2023-10-03T10:09:43.722422+0000', 'last_peered': '2023-10-03T10:09:43.722422+0000', 'last_clean': '2023-10-03T10:09:43.722422+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-03T10:09:43.722422+0000', 'last_undegraded': '2023-10-03T10:09:43.722422+0000', 'last_fullsized': '2023-10-03T10:09:43.722422+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'last_clean_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-03T10:09:48.991 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1c is not active+clean 2023-10-03T10:09:48.991 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-03T10:09:43.722422+0000', 'last_change': '2023-10-03T10:09:43.722422+0000', 'last_active': '2023-10-03T10:09:43.722422+0000', 'last_peered': '2023-10-03T10:09:43.722422+0000', 'last_clean': '2023-10-03T10:09:43.722422+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-03T10:09:43.722422+0000', 'last_undegraded': '2023-10-03T10:09:43.722422+0000', 'last_fullsized': '2023-10-03T10:09:43.722422+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'last_clean_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-03T10:09:48.991 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1b is not active+clean 2023-10-03T10:09:48.991 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-03T10:09:43.722422+0000', 'last_change': '2023-10-03T10:09:43.722422+0000', 'last_active': '2023-10-03T10:09:43.722422+0000', 'last_peered': '2023-10-03T10:09:43.722422+0000', 'last_clean': '2023-10-03T10:09:43.722422+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-03T10:09:43.722422+0000', 'last_undegraded': '2023-10-03T10:09:43.722422+0000', 'last_fullsized': '2023-10-03T10:09:43.722422+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'last_clean_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-03T10:09:48.991 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1a is not active+clean 2023-10-03T10:09:48.992 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-03T10:09:43.722422+0000', 'last_change': '2023-10-03T10:09:43.722422+0000', 'last_active': '2023-10-03T10:09:43.722422+0000', 'last_peered': '2023-10-03T10:09:43.722422+0000', 'last_clean': '2023-10-03T10:09:43.722422+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-03T10:09:43.722422+0000', 'last_undegraded': '2023-10-03T10:09:43.722422+0000', 'last_fullsized': '2023-10-03T10:09:43.722422+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'last_clean_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-03T10:09:48.992 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.18 is not active+clean 2023-10-03T10:09:48.992 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-03T10:09:43.722422+0000', 'last_change': '2023-10-03T10:09:43.722422+0000', 'last_active': '2023-10-03T10:09:43.722422+0000', 'last_peered': '2023-10-03T10:09:43.722422+0000', 'last_clean': '2023-10-03T10:09:43.722422+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-03T10:09:43.722422+0000', 'last_undegraded': '2023-10-03T10:09:43.722422+0000', 'last_fullsized': '2023-10-03T10:09:43.722422+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'last_clean_scrub_stamp': '2023-10-03T10:09:43.722422+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-03T10:09:48.992 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-03T10:09:49.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.154+0000 7f6aeff67700 1 Processor -- start 2023-10-03T10:09:49.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.154+0000 7f6aeff67700 1 -- start start 2023-10-03T10:09:49.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.155+0000 7f6aeff67700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6ae8096c40 0x7f6ae8097020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:49.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.155+0000 7f6aeff67700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ae812e310 0x7f6ae8130700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:49.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.155+0000 7f6aeff67700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae806aba0 con 0x7f6ae806a7c0 2023-10-03T10:09:49.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.155+0000 7f6aeff67700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ae8097980 con 0x7f6ae8096c40 2023-10-03T10:09:49.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.155+0000 7f6aeff67700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae8069d80 con 0x7f6ae812e310 2023-10-03T10:09:49.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.155+0000 7f6aee504700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ae812e310 0x7f6ae8130700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:49.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.155+0000 7f6aed502700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6ae8096c40 0x7f6ae8097020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:49.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.155+0000 7f6aedd03700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6ae806a7c0 0x7f6ae8096530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35082/0 (socket says 172.21.15.3:35082) 2023-10-03T10:09:49.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.155+0000 7f6aedd03700 1 -- 172.21.15.3:0/2679583244 learned_addr learned my addr 172.21.15.3:0/2679583244 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:09:49.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.155+0000 7f6aee504700 1 -- 172.21.15.3:0/2679583244 >> v2:172.21.15.3:3300/0 conn(0x7f6ae8096c40 msgr2=0x7f6ae8097020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:49.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.155+0000 7f6aee504700 1 --2- 172.21.15.3:0/2679583244 >> v2:172.21.15.3:3300/0 conn(0x7f6ae8096c40 0x7f6ae8097020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:49.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.155+0000 7f6aee504700 1 -- 172.21.15.3:0/2679583244 >> v1:172.21.15.3:6789/0 conn(0x7f6ae806a7c0 legacy=0x7f6ae8096530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:49.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.156+0000 7f6aecd01700 1 -- 172.21.15.3:0/2679583244 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 943987659 0 0) 0x7f6ae806aba0 con 0x7f6ae806a7c0 2023-10-03T10:09:49.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.156+0000 7f6aee504700 1 -- 172.21.15.3:0/2679583244 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae8068c00 con 0x7f6ae812e310 2023-10-03T10:09:49.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.156+0000 7f6aed502700 1 --2- 172.21.15.3:0/2679583244 >> v2:172.21.15.3:3300/0 conn(0x7f6ae8096c40 0x7f6ae8097020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:49.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.156+0000 7f6aee504700 1 --2- 172.21.15.3:0/2679583244 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ae812e310 0x7f6ae8130700 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f6ae0009e90 tx=0x7f6ae0009780).ready entity=mon.1 client_cookie=2da678b661e4fa61 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:49.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.156+0000 7f6aecd01700 1 -- 172.21.15.3:0/2679583244 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae000e070 con 0x7f6ae812e310 2023-10-03T10:09:49.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.156+0000 7f6aecd01700 1 -- 172.21.15.3:0/2679583244 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae000bb20 con 0x7f6ae812e310 2023-10-03T10:09:49.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.156+0000 7f6aecd01700 1 -- 172.21.15.3:0/2679583244 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae000bea0 con 0x7f6ae812e310 2023-10-03T10:09:49.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.156+0000 7f6aeff67700 1 -- 172.21.15.3:0/2679583244 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ae812e310 msgr2=0x7f6ae8130700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:49.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.156+0000 7f6aeff67700 1 --2- 172.21.15.3:0/2679583244 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ae812e310 0x7f6ae8130700 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f6ae0009e90 tx=0x7f6ae0009780).stop 2023-10-03T10:09:49.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.156+0000 7f6aeff67700 1 -- 172.21.15.3:0/2679583244 shutdown_connections 2023-10-03T10:09:49.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.156+0000 7f6aeff67700 1 --2- 172.21.15.3:0/2679583244 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ae812e310 0x7f6ae8130700 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:49.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.156+0000 7f6aeff67700 1 --2- 172.21.15.3:0/2679583244 >> v2:172.21.15.3:3300/0 conn(0x7f6ae8096c40 0x7f6ae8097020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:49.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.156+0000 7f6aeff67700 1 -- 172.21.15.3:0/2679583244 >> 172.21.15.3:0/2679583244 conn(0x7f6ae809bb10 msgr2=0x7f6ae809bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:49.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.157+0000 7f6aeff67700 1 -- 172.21.15.3:0/2679583244 shutdown_connections 2023-10-03T10:09:49.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.157+0000 7f6aeff67700 1 -- 172.21.15.3:0/2679583244 wait complete. 2023-10-03T10:09:49.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.157+0000 7f6aeff67700 1 Processor -- start 2023-10-03T10:09:49.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.158+0000 7f6aeff67700 1 -- start start 2023-10-03T10:09:49.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.158+0000 7f6aeff67700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ae806a7c0 0x7f6ae811dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:49.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.158+0000 7f6aeff67700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ae8096c40 0x7f6ae8064660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:49.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.158+0000 7f6aeff67700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ae812e310 0x7f6ae8064b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:49.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.158+0000 7f6aeff67700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae811d750 con 0x7f6ae8096c40 2023-10-03T10:09:49.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.158+0000 7f6aeff67700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae8099080 con 0x7f6ae812e310 2023-10-03T10:09:49.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.158+0000 7f6aeff67700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae8067f70 con 0x7f6ae806a7c0 2023-10-03T10:09:49.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.158+0000 7f6aed502700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ae8096c40 0x7f6ae8064660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:49.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.158+0000 7f6aed502700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ae8096c40 0x7f6ae8064660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35526/0 (socket says 172.21.15.3:35526) 2023-10-03T10:09:49.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.158+0000 7f6aed502700 1 -- 172.21.15.3:0/256353267 learned_addr learned my addr 172.21.15.3:0/256353267 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:49.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.158+0000 7f6aee504700 1 --2- 172.21.15.3:0/256353267 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ae812e310 0x7f6ae8064b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:49.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.158+0000 7f6aedd03700 1 --2- 172.21.15.3:0/256353267 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ae806a7c0 0x7f6ae811dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:49.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.158+0000 7f6aed502700 1 -- 172.21.15.3:0/256353267 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ae806a7c0 msgr2=0x7f6ae811dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:49.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.158+0000 7f6aed502700 1 --2- 172.21.15.3:0/256353267 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ae806a7c0 0x7f6ae811dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:49.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.158+0000 7f6aed502700 1 -- 172.21.15.3:0/256353267 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ae812e310 msgr2=0x7f6ae8064b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:49.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.158+0000 7f6aed502700 1 --2- 172.21.15.3:0/256353267 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ae812e310 0x7f6ae8064b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:49.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.158+0000 7f6aed502700 1 -- 172.21.15.3:0/256353267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6adc0092a0 con 0x7f6ae8096c40 2023-10-03T10:09:49.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.159+0000 7f6aed502700 1 --2- 172.21.15.3:0/256353267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ae8096c40 0x7f6ae8064660 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f6adc00ea50 tx=0x7f6adc009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:49.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.159+0000 7f6ad9ffb700 1 -- 172.21.15.3:0/256353267 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc00ee90 con 0x7f6ae8096c40 2023-10-03T10:09:49.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.159+0000 7f6ad9ffb700 1 -- 172.21.15.3:0/256353267 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6adc00c850 con 0x7f6ae8096c40 2023-10-03T10:09:49.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.159+0000 7f6ad9ffb700 1 -- 172.21.15.3:0/256353267 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc00cbd0 con 0x7f6ae8096c40 2023-10-03T10:09:49.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.159+0000 7f6aeff67700 1 -- 172.21.15.3:0/256353267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae0009a20 con 0x7f6ae8096c40 2023-10-03T10:09:49.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.159+0000 7f6aeff67700 1 -- 172.21.15.3:0/256353267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae812dce0 con 0x7f6ae8096c40 2023-10-03T10:09:49.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.160+0000 7f6ad9ffb700 1 -- 172.21.15.3:0/256353267 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f6adc01b400 con 0x7f6ae8096c40 2023-10-03T10:09:49.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.160+0000 7f6ad9ffb700 1 -- 172.21.15.3:0/256353267 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f6adc03b620 con 0x7f6ae8096c40 2023-10-03T10:09:49.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.160+0000 7f6aeff67700 1 -- 172.21.15.3:0/256353267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ae805a260 con 0x7f6ae8096c40 2023-10-03T10:09:49.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.165+0000 7f6ad9ffb700 1 -- 172.21.15.3:0/256353267 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f6adc01be60 con 0x7f6ae8096c40 2023-10-03T10:09:49.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.413+0000 7f6aeff67700 1 -- 172.21.15.3:0/256353267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f6ae8065460 con 0x7f6ae8096c40 2023-10-03T10:09:49.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.413+0000 7f6ad9ffb700 1 -- 172.21.15.3:0/256353267 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1025 (secure 0 0 0) 0x7f6adc01be60 con 0x7f6ae8096c40 2023-10-03T10:09:49.414 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:49.414 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":20,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1696327776,"num_in_osds":4,"osd_in_since":1696327770,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":89},{"state_name":"unknown","count":16}],"num_pgs":105,"num_pools":4,"num_objects":20,"data_bytes":2306,"bytes_used":1217507328,"bytes_avail":385329549312,"bytes_total":386547056640,"unknown_pgs_ratio":0.15238095819950104},"fsmap":{"epoch":6,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"a","status":"up:active","gid":4186}],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2023-10-03T10:09:28.704453+0000","services":{}},"progress_events":{}} 2023-10-03T10:09:49.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.415+0000 7f6aeff67700 1 -- 172.21.15.3:0/256353267 >> v1:172.21.15.3:6816/118296 conn(0x7f6acc02bb60 legacy=0x7f6acc02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:49.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.415+0000 7f6aeff67700 1 -- 172.21.15.3:0/256353267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ae8096c40 msgr2=0x7f6ae8064660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:49.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.415+0000 7f6aeff67700 1 --2- 172.21.15.3:0/256353267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ae8096c40 0x7f6ae8064660 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f6adc00ea50 tx=0x7f6adc009060).stop 2023-10-03T10:09:49.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.415+0000 7f6aeff67700 1 -- 172.21.15.3:0/256353267 shutdown_connections 2023-10-03T10:09:49.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.415+0000 7f6aeff67700 1 --2- 172.21.15.3:0/256353267 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ae812e310 0x7f6ae8064b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:49.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.415+0000 7f6aeff67700 1 --2- 172.21.15.3:0/256353267 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ae806a7c0 0x7f6ae811dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:49.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.415+0000 7f6aeff67700 1 --2- 172.21.15.3:0/256353267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ae8096c40 0x7f6ae8064660 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:49.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.415+0000 7f6aeff67700 1 -- 172.21.15.3:0/256353267 >> 172.21.15.3:0/256353267 conn(0x7f6ae809bb10 msgr2=0x7f6ae8094720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:49.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.415+0000 7f6aeff67700 1 -- 172.21.15.3:0/256353267 shutdown_connections 2023-10-03T10:09:49.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.415+0000 7f6aeff67700 1 -- 172.21.15.3:0/256353267 wait complete. 2023-10-03T10:09:49.428 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-03T10:09:49.429 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-03T10:09:49.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.561+0000 7fe3a87fd700 1 Processor -- start 2023-10-03T10:09:49.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.561+0000 7fe3a87fd700 1 -- start start 2023-10-03T10:09:49.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.561+0000 7fe3a87fd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a0062a90 0x7fe3a0127860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:49.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.562+0000 7fe3a87fd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe3a0127d50 0x7fe3a0095a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:49.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.562+0000 7fe3a87fd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a0091bc0 con 0x7fe3a0127360 2023-10-03T10:09:49.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.562+0000 7fe3a87fd700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a0062e70 con 0x7fe3a0127d50 2023-10-03T10:09:49.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.562+0000 7fe3a87fd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a00968b0 con 0x7fe3a0062a90 2023-10-03T10:09:49.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.562+0000 7fe3a6599700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a0062a90 0x7fe3a0127860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:49.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.562+0000 7fe3a6d9a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe3a0127d50 0x7fe3a0095a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:49.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.562+0000 7fe3a5d98700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe3a0127360 0x7fe3a0095300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35094/0 (socket says 172.21.15.3:35094) 2023-10-03T10:09:49.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.562+0000 7fe3a6599700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a0062a90 0x7fe3a0127860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34614/0 (socket says 172.21.15.3:34614) 2023-10-03T10:09:49.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.562+0000 7fe3a5d98700 1 -- 172.21.15.3:0/1856121876 learned_addr learned my addr 172.21.15.3:0/1856121876 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:09:49.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.562+0000 7fe3a6599700 1 -- 172.21.15.3:0/1856121876 >> v2:172.21.15.3:3300/0 conn(0x7fe3a0127d50 msgr2=0x7fe3a0095a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:49.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.562+0000 7fe3a6599700 1 --2- 172.21.15.3:0/1856121876 >> v2:172.21.15.3:3300/0 conn(0x7fe3a0127d50 0x7fe3a0095a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:49.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.562+0000 7fe3a6599700 1 -- 172.21.15.3:0/1856121876 >> v1:172.21.15.3:6789/0 conn(0x7fe3a0127360 legacy=0x7fe3a0095300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:49.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.562+0000 7fe3a5597700 1 -- 172.21.15.3:0/1856121876 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 763426209 0 0) 0x7fe3a0091bc0 con 0x7fe3a0127360 2023-10-03T10:09:49.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.562+0000 7fe3a6599700 1 -- 172.21.15.3:0/1856121876 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a0092a70 con 0x7fe3a0062a90 2023-10-03T10:09:49.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.562+0000 7fe3a6d9a700 1 --2- 172.21.15.3:0/1856121876 >> v2:172.21.15.3:3300/0 conn(0x7fe3a0127d50 0x7fe3a0095a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:49.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.562+0000 7fe3a6599700 1 --2- 172.21.15.3:0/1856121876 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a0062a90 0x7fe3a0127860 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fe39c009760 tx=0x7fe39c009060).ready entity=mon.1 client_cookie=d274b8e5105e098f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:49.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.562+0000 7fe3a5597700 1 -- 172.21.15.3:0/1856121876 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c0090d0 con 0x7fe3a0062a90 2023-10-03T10:09:49.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.562+0000 7fe3a5597700 1 -- 172.21.15.3:0/1856121876 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39c00f510 con 0x7fe3a0062a90 2023-10-03T10:09:49.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.562+0000 7fe3a5597700 1 -- 172.21.15.3:0/1856121876 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c00f890 con 0x7fe3a0062a90 2023-10-03T10:09:49.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.563+0000 7fe3a87fd700 1 -- 172.21.15.3:0/1856121876 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a0062a90 msgr2=0x7fe3a0127860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:49.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.563+0000 7fe3a87fd700 1 --2- 172.21.15.3:0/1856121876 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a0062a90 0x7fe3a0127860 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fe39c009760 tx=0x7fe39c009060).stop 2023-10-03T10:09:49.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.563+0000 7fe3a87fd700 1 -- 172.21.15.3:0/1856121876 shutdown_connections 2023-10-03T10:09:49.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.563+0000 7fe3a87fd700 1 --2- 172.21.15.3:0/1856121876 >> v2:172.21.15.3:3300/0 conn(0x7fe3a0127d50 0x7fe3a0095a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:49.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.563+0000 7fe3a87fd700 1 --2- 172.21.15.3:0/1856121876 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a0062a90 0x7fe3a0127860 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:49.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.563+0000 7fe3a87fd700 1 -- 172.21.15.3:0/1856121876 >> 172.21.15.3:0/1856121876 conn(0x7fe3a0059bb0 msgr2=0x7fe3a0097400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:49.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.563+0000 7fe3a87fd700 1 -- 172.21.15.3:0/1856121876 shutdown_connections 2023-10-03T10:09:49.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.563+0000 7fe3a87fd700 1 -- 172.21.15.3:0/1856121876 wait complete. 2023-10-03T10:09:49.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.564+0000 7fe3a87fd700 1 Processor -- start 2023-10-03T10:09:49.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.564+0000 7fe3a87fd700 1 -- start start 2023-10-03T10:09:49.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.565+0000 7fe3a87fd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a0062a90 0x7fe3a01148c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:49.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.565+0000 7fe3a87fd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3a0127360 0x7fe3a0114db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:49.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.565+0000 7fe3a87fd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3a0127d50 0x7fe3a01152a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:49.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.565+0000 7fe3a87fd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a0099fa0 con 0x7fe3a0127d50 2023-10-03T10:09:49.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.565+0000 7fe3a87fd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a0096e90 con 0x7fe3a0062a90 2023-10-03T10:09:49.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.565+0000 7fe3a87fd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a0117450 con 0x7fe3a0127360 2023-10-03T10:09:49.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.565+0000 7fe3a6599700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a0062a90 0x7fe3a01148c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:49.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.565+0000 7fe3a6599700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a0062a90 0x7fe3a01148c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34624/0 (socket says 172.21.15.3:34624) 2023-10-03T10:09:49.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.565+0000 7fe3a6599700 1 -- 172.21.15.3:0/2942488333 learned_addr learned my addr 172.21.15.3:0/2942488333 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:49.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.565+0000 7fe3a5d98700 1 --2- 172.21.15.3:0/2942488333 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3a0127360 0x7fe3a0114db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:49.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.565+0000 7fe3a6d9a700 1 --2- 172.21.15.3:0/2942488333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3a0127d50 0x7fe3a01152a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:49.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.565+0000 7fe3a6599700 1 -- 172.21.15.3:0/2942488333 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3a0127360 msgr2=0x7fe3a0114db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:49.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.565+0000 7fe3a6599700 1 --2- 172.21.15.3:0/2942488333 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3a0127360 0x7fe3a0114db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:49.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.565+0000 7fe3a6599700 1 -- 172.21.15.3:0/2942488333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3a0127d50 msgr2=0x7fe3a01152a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:49.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.565+0000 7fe3a6599700 1 --2- 172.21.15.3:0/2942488333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3a0127d50 0x7fe3a01152a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:49.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.565+0000 7fe3a6599700 1 -- 172.21.15.3:0/2942488333 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe398009a20 con 0x7fe3a0062a90 2023-10-03T10:09:49.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.565+0000 7fe3a6d9a700 1 --2- 172.21.15.3:0/2942488333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3a0127d50 0x7fe3a01152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:49.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.565+0000 7fe3a6599700 1 --2- 172.21.15.3:0/2942488333 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a0062a90 0x7fe3a01148c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fe39c009e00 tx=0x7fe39c017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:49.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.565+0000 7fe3967fc700 1 -- 172.21.15.3:0/2942488333 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c00f890 con 0x7fe3a0062a90 2023-10-03T10:09:49.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.565+0000 7fe3967fc700 1 -- 172.21.15.3:0/2942488333 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39c006910 con 0x7fe3a0062a90 2023-10-03T10:09:49.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.565+0000 7fe3a87fd700 1 -- 172.21.15.3:0/2942488333 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe39c0092a0 con 0x7fe3a0062a90 2023-10-03T10:09:49.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.565+0000 7fe3967fc700 1 -- 172.21.15.3:0/2942488333 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c00fea0 con 0x7fe3a0062a90 2023-10-03T10:09:49.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.565+0000 7fe3a87fd700 1 -- 172.21.15.3:0/2942488333 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a0118540 con 0x7fe3a0062a90 2023-10-03T10:09:49.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.566+0000 7fe3967fc700 1 -- 172.21.15.3:0/2942488333 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fe39c01b620 con 0x7fe3a0062a90 2023-10-03T10:09:49.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.566+0000 7fe3967fc700 1 -- 172.21.15.3:0/2942488333 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fe39c03c780 con 0x7fe3a0062a90 2023-10-03T10:09:49.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.567+0000 7fe3a87fd700 1 -- 172.21.15.3:0/2942488333 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a0118540 con 0x7fe3a0062a90 2023-10-03T10:09:49.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.571+0000 7fe3967fc700 1 -- 172.21.15.3:0/2942488333 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fe3a0118540 con 0x7fe3a0062a90 2023-10-03T10:09:49.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.747+0000 7fe3a87fd700 1 -- 172.21.15.3:0/2942488333 --> v1:172.21.15.3:6816/118296 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe3a0117920 con 0x7fe38402bb90 2023-10-03T10:09:49.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.753+0000 7fe3967fc700 1 -- 172.21.15.3:0/2942488333 <== mgr.4100 v1:172.21.15.3:6816/118296 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249995 (unknown 2965378022 0 3492090012) 0x7fe3900012b0 con 0x7fe38402bb90 2023-10-03T10:09:49.757 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:49.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.757+0000 7fe3a87fd700 1 -- 172.21.15.3:0/2942488333 >> v1:172.21.15.3:6816/118296 conn(0x7fe38402bb90 legacy=0x7fe38402e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:49.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.757+0000 7fe3a87fd700 1 -- 172.21.15.3:0/2942488333 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a0062a90 msgr2=0x7fe3a01148c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:49.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.757+0000 7fe3a87fd700 1 --2- 172.21.15.3:0/2942488333 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a0062a90 0x7fe3a01148c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fe39c009e00 tx=0x7fe39c017000).stop 2023-10-03T10:09:49.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.757+0000 7fe3a87fd700 1 -- 172.21.15.3:0/2942488333 shutdown_connections 2023-10-03T10:09:49.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.757+0000 7fe3a87fd700 1 --2- 172.21.15.3:0/2942488333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3a0127d50 0x7fe3a01152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:49.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.757+0000 7fe3a87fd700 1 --2- 172.21.15.3:0/2942488333 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a0062a90 0x7fe3a01148c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:49.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.757+0000 7fe3a87fd700 1 --2- 172.21.15.3:0/2942488333 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3a0127360 0x7fe3a0114db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:49.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.757+0000 7fe3a87fd700 1 -- 172.21.15.3:0/2942488333 >> 172.21.15.3:0/2942488333 conn(0x7fe3a0059bb0 msgr2=0x7fe3a012a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:49.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.757+0000 7fe3a87fd700 1 -- 172.21.15.3:0/2942488333 shutdown_connections 2023-10-03T10:09:49.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:49.757+0000 7fe3a87fd700 1 -- 172.21.15.3:0/2942488333 wait complete. 2023-10-03T10:09:49.760 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-03T10:09:49.770 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-10-03T10:09:49.529583+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1189024,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298336,"statfs":{"total":386547056640,"available":385329496064,"internally_reserved":0,"allocated":655360,"data_stored":193408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000427"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.199042+0000","last_change":"2023-10-03T10:09:44.195682+0000","last_active":"2023-10-03T10:09:44.199042+0000","last_peered":"2023-10-03T10:09:44.199042+0000","last_clean":"2023-10-03T10:09:44.199042+0000","last_became_active":"2023-10-03T10:09:44.195360+0000","last_became_peered":"2023-10-03T10:09:44.195360+0000","last_unstale":"2023-10-03T10:09:44.199042+0000","last_undegraded":"2023-10-03T10:09:44.199042+0000","last_fullsized":"2023-10-03T10:09:44.199042+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180036+0000","last_change":"2023-10-03T10:09:44.178263+0000","last_active":"2023-10-03T10:09:44.180036+0000","last_peered":"2023-10-03T10:09:44.180036+0000","last_clean":"2023-10-03T10:09:44.180036+0000","last_became_active":"2023-10-03T10:09:44.177977+0000","last_became_peered":"2023-10-03T10:09:44.177977+0000","last_unstale":"2023-10-03T10:09:44.180036+0000","last_undegraded":"2023-10-03T10:09:44.180036+0000","last_fullsized":"2023-10-03T10:09:44.180036+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.190395+0000","last_change":"2023-10-03T10:09:44.183364+0000","last_active":"2023-10-03T10:09:44.190395+0000","last_peered":"2023-10-03T10:09:44.190395+0000","last_clean":"2023-10-03T10:09:44.190395+0000","last_became_active":"2023-10-03T10:09:44.183094+0000","last_became_peered":"2023-10-03T10:09:44.183094+0000","last_unstale":"2023-10-03T10:09:44.190395+0000","last_undegraded":"2023-10-03T10:09:44.190395+0000","last_fullsized":"2023-10-03T10:09:44.190395+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.190339+0000","last_change":"2023-10-03T10:09:44.183273+0000","last_active":"2023-10-03T10:09:44.190339+0000","last_peered":"2023-10-03T10:09:44.190339+0000","last_clean":"2023-10-03T10:09:44.190339+0000","last_became_active":"2023-10-03T10:09:44.182451+0000","last_became_peered":"2023-10-03T10:09:44.182451+0000","last_unstale":"2023-10-03T10:09:44.190339+0000","last_undegraded":"2023-10-03T10:09:44.190339+0000","last_fullsized":"2023-10-03T10:09:44.190339+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.174191+0000","last_change":"2023-10-03T10:09:44.168483+0000","last_active":"2023-10-03T10:09:44.174191+0000","last_peered":"2023-10-03T10:09:44.174191+0000","last_clean":"2023-10-03T10:09:44.174191+0000","last_became_active":"2023-10-03T10:09:44.168186+0000","last_became_peered":"2023-10-03T10:09:44.168186+0000","last_unstale":"2023-10-03T10:09:44.174191+0000","last_undegraded":"2023-10-03T10:09:44.174191+0000","last_fullsized":"2023-10-03T10:09:44.174191+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.198209+0000","last_change":"2023-10-03T10:09:44.192967+0000","last_active":"2023-10-03T10:09:44.198209+0000","last_peered":"2023-10-03T10:09:44.198209+0000","last_clean":"2023-10-03T10:09:44.198209+0000","last_became_active":"2023-10-03T10:09:44.190979+0000","last_became_peered":"2023-10-03T10:09:44.190979+0000","last_unstale":"2023-10-03T10:09:44.198209+0000","last_undegraded":"2023-10-03T10:09:44.198209+0000","last_fullsized":"2023-10-03T10:09:44.198209+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.192360+0000","last_change":"2023-10-03T10:09:44.183121+0000","last_active":"2023-10-03T10:09:44.192360+0000","last_peered":"2023-10-03T10:09:44.192360+0000","last_clean":"2023-10-03T10:09:44.192360+0000","last_became_active":"2023-10-03T10:09:44.182647+0000","last_became_peered":"2023-10-03T10:09:44.182647+0000","last_unstale":"2023-10-03T10:09:44.192360+0000","last_undegraded":"2023-10-03T10:09:44.192360+0000","last_fullsized":"2023-10-03T10:09:44.192360+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.187765+0000","last_change":"2023-10-03T10:09:44.173123+0000","last_active":"2023-10-03T10:09:44.187765+0000","last_peered":"2023-10-03T10:09:44.187765+0000","last_clean":"2023-10-03T10:09:44.187765+0000","last_became_active":"2023-10-03T10:09:44.172353+0000","last_became_peered":"2023-10-03T10:09:44.172353+0000","last_unstale":"2023-10-03T10:09:44.187765+0000","last_undegraded":"2023-10-03T10:09:44.187765+0000","last_fullsized":"2023-10-03T10:09:44.187765+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173973+0000","last_change":"2023-10-03T10:09:44.166758+0000","last_active":"2023-10-03T10:09:44.173973+0000","last_peered":"2023-10-03T10:09:44.173973+0000","last_clean":"2023-10-03T10:09:44.173973+0000","last_became_active":"2023-10-03T10:09:44.166415+0000","last_became_peered":"2023-10-03T10:09:44.166415+0000","last_unstale":"2023-10-03T10:09:44.173973+0000","last_undegraded":"2023-10-03T10:09:44.173973+0000","last_fullsized":"2023-10-03T10:09:44.173973+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180446+0000","last_change":"2023-10-03T10:09:44.176833+0000","last_active":"2023-10-03T10:09:44.180446+0000","last_peered":"2023-10-03T10:09:44.180446+0000","last_clean":"2023-10-03T10:09:44.180446+0000","last_became_active":"2023-10-03T10:09:44.176526+0000","last_became_peered":"2023-10-03T10:09:44.176526+0000","last_unstale":"2023-10-03T10:09:44.180446+0000","last_undegraded":"2023-10-03T10:09:44.180446+0000","last_fullsized":"2023-10-03T10:09:44.180446+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180114+0000","last_change":"2023-10-03T10:09:44.175868+0000","last_active":"2023-10-03T10:09:44.180114+0000","last_peered":"2023-10-03T10:09:44.180114+0000","last_clean":"2023-10-03T10:09:44.180114+0000","last_became_active":"2023-10-03T10:09:44.175414+0000","last_became_peered":"2023-10-03T10:09:44.175414+0000","last_unstale":"2023-10-03T10:09:44.180114+0000","last_undegraded":"2023-10-03T10:09:44.180114+0000","last_fullsized":"2023-10-03T10:09:44.180114+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200283+0000","last_change":"2023-10-03T10:09:44.198581+0000","last_active":"2023-10-03T10:09:44.200283+0000","last_peered":"2023-10-03T10:09:44.200283+0000","last_clean":"2023-10-03T10:09:44.200283+0000","last_became_active":"2023-10-03T10:09:44.198314+0000","last_became_peered":"2023-10-03T10:09:44.198314+0000","last_unstale":"2023-10-03T10:09:44.200283+0000","last_undegraded":"2023-10-03T10:09:44.200283+0000","last_fullsized":"2023-10-03T10:09:44.200283+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200379+0000","last_change":"2023-10-03T10:09:44.197915+0000","last_active":"2023-10-03T10:09:44.200379+0000","last_peered":"2023-10-03T10:09:44.200379+0000","last_clean":"2023-10-03T10:09:44.200379+0000","last_became_active":"2023-10-03T10:09:44.197417+0000","last_became_peered":"2023-10-03T10:09:44.197417+0000","last_unstale":"2023-10-03T10:09:44.200379+0000","last_undegraded":"2023-10-03T10:09:44.200379+0000","last_fullsized":"2023-10-03T10:09:44.200379+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180522+0000","last_change":"2023-10-03T10:09:44.176456+0000","last_active":"2023-10-03T10:09:44.180522+0000","last_peered":"2023-10-03T10:09:44.180522+0000","last_clean":"2023-10-03T10:09:44.180522+0000","last_became_active":"2023-10-03T10:09:44.176146+0000","last_became_peered":"2023-10-03T10:09:44.176146+0000","last_unstale":"2023-10-03T10:09:44.180522+0000","last_undegraded":"2023-10-03T10:09:44.180522+0000","last_fullsized":"2023-10-03T10:09:44.180522+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.178380+0000","last_change":"2023-10-03T10:09:44.176614+0000","last_active":"2023-10-03T10:09:44.178380+0000","last_peered":"2023-10-03T10:09:44.178380+0000","last_clean":"2023-10-03T10:09:44.178380+0000","last_became_active":"2023-10-03T10:09:44.176334+0000","last_became_peered":"2023-10-03T10:09:44.176334+0000","last_unstale":"2023-10-03T10:09:44.178380+0000","last_undegraded":"2023-10-03T10:09:44.178380+0000","last_fullsized":"2023-10-03T10:09:44.178380+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.192741+0000","last_change":"2023-10-03T10:09:44.179440+0000","last_active":"2023-10-03T10:09:44.192741+0000","last_peered":"2023-10-03T10:09:44.192741+0000","last_clean":"2023-10-03T10:09:44.192741+0000","last_became_active":"2023-10-03T10:09:44.179150+0000","last_became_peered":"2023-10-03T10:09:44.179150+0000","last_unstale":"2023-10-03T10:09:44.192741+0000","last_undegraded":"2023-10-03T10:09:44.192741+0000","last_fullsized":"2023-10-03T10:09:44.192741+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173996+0000","last_change":"2023-10-03T10:09:44.166901+0000","last_active":"2023-10-03T10:09:44.173996+0000","last_peered":"2023-10-03T10:09:44.173996+0000","last_clean":"2023-10-03T10:09:44.173996+0000","last_became_active":"2023-10-03T10:09:44.166610+0000","last_became_peered":"2023-10-03T10:09:44.166610+0000","last_unstale":"2023-10-03T10:09:44.173996+0000","last_undegraded":"2023-10-03T10:09:44.173996+0000","last_fullsized":"2023-10-03T10:09:44.173996+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173643+0000","last_change":"2023-10-03T10:09:44.165366+0000","last_active":"2023-10-03T10:09:44.173643+0000","last_peered":"2023-10-03T10:09:44.173643+0000","last_clean":"2023-10-03T10:09:44.173643+0000","last_became_active":"2023-10-03T10:09:44.165083+0000","last_became_peered":"2023-10-03T10:09:44.165083+0000","last_unstale":"2023-10-03T10:09:44.173643+0000","last_undegraded":"2023-10-03T10:09:44.173643+0000","last_fullsized":"2023-10-03T10:09:44.173643+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.198990+0000","last_change":"2023-10-03T10:09:44.195696+0000","last_active":"2023-10-03T10:09:44.198990+0000","last_peered":"2023-10-03T10:09:44.198990+0000","last_clean":"2023-10-03T10:09:44.198990+0000","last_became_active":"2023-10-03T10:09:44.195402+0000","last_became_peered":"2023-10-03T10:09:44.195402+0000","last_unstale":"2023-10-03T10:09:44.198990+0000","last_undegraded":"2023-10-03T10:09:44.198990+0000","last_fullsized":"2023-10-03T10:09:44.198990+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184209+0000","last_change":"2023-10-03T10:09:44.177178+0000","last_active":"2023-10-03T10:09:44.184209+0000","last_peered":"2023-10-03T10:09:44.184209+0000","last_clean":"2023-10-03T10:09:44.184209+0000","last_became_active":"2023-10-03T10:09:44.176686+0000","last_became_peered":"2023-10-03T10:09:44.176686+0000","last_unstale":"2023-10-03T10:09:44.184209+0000","last_undegraded":"2023-10-03T10:09:44.184209+0000","last_fullsized":"2023-10-03T10:09:44.184209+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180396+0000","last_change":"2023-10-03T10:09:44.177335+0000","last_active":"2023-10-03T10:09:44.180396+0000","last_peered":"2023-10-03T10:09:44.180396+0000","last_clean":"2023-10-03T10:09:44.180396+0000","last_became_active":"2023-10-03T10:09:44.175202+0000","last_became_peered":"2023-10-03T10:09:44.175202+0000","last_unstale":"2023-10-03T10:09:44.180396+0000","last_undegraded":"2023-10-03T10:09:44.180396+0000","last_fullsized":"2023-10-03T10:09:44.180396+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184702+0000","last_change":"2023-10-03T10:09:44.182207+0000","last_active":"2023-10-03T10:09:44.184702+0000","last_peered":"2023-10-03T10:09:44.184702+0000","last_clean":"2023-10-03T10:09:44.184702+0000","last_became_active":"2023-10-03T10:09:44.181944+0000","last_became_peered":"2023-10-03T10:09:44.181944+0000","last_unstale":"2023-10-03T10:09:44.184702+0000","last_undegraded":"2023-10-03T10:09:44.184702+0000","last_fullsized":"2023-10-03T10:09:44.184702+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.192336+0000","last_change":"2023-10-03T10:09:44.182610+0000","last_active":"2023-10-03T10:09:44.192336+0000","last_peered":"2023-10-03T10:09:44.192336+0000","last_clean":"2023-10-03T10:09:44.192336+0000","last_became_active":"2023-10-03T10:09:44.180732+0000","last_became_peered":"2023-10-03T10:09:44.180732+0000","last_unstale":"2023-10-03T10:09:44.192336+0000","last_undegraded":"2023-10-03T10:09:44.192336+0000","last_fullsized":"2023-10-03T10:09:44.192336+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.187788+0000","last_change":"2023-10-03T10:09:44.173182+0000","last_active":"2023-10-03T10:09:44.187788+0000","last_peered":"2023-10-03T10:09:44.187788+0000","last_clean":"2023-10-03T10:09:44.187788+0000","last_became_active":"2023-10-03T10:09:44.172685+0000","last_became_peered":"2023-10-03T10:09:44.172685+0000","last_unstale":"2023-10-03T10:09:44.187788+0000","last_undegraded":"2023-10-03T10:09:44.187788+0000","last_fullsized":"2023-10-03T10:09:44.187788+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180761+0000","last_change":"2023-10-03T10:09:44.177910+0000","last_active":"2023-10-03T10:09:44.180761+0000","last_peered":"2023-10-03T10:09:44.180761+0000","last_clean":"2023-10-03T10:09:44.180761+0000","last_became_active":"2023-10-03T10:09:44.177618+0000","last_became_peered":"2023-10-03T10:09:44.177618+0000","last_unstale":"2023-10-03T10:09:44.180761+0000","last_undegraded":"2023-10-03T10:09:44.180761+0000","last_fullsized":"2023-10-03T10:09:44.180761+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180424+0000","last_change":"2023-10-03T10:09:44.176287+0000","last_active":"2023-10-03T10:09:44.180424+0000","last_peered":"2023-10-03T10:09:44.180424+0000","last_clean":"2023-10-03T10:09:44.180424+0000","last_became_active":"2023-10-03T10:09:44.175320+0000","last_became_peered":"2023-10-03T10:09:44.175320+0000","last_unstale":"2023-10-03T10:09:44.180424+0000","last_undegraded":"2023-10-03T10:09:44.180424+0000","last_fullsized":"2023-10-03T10:09:44.180424+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.194745+0000","last_change":"2023-10-03T10:09:44.187148+0000","last_active":"2023-10-03T10:09:44.194745+0000","last_peered":"2023-10-03T10:09:44.194745+0000","last_clean":"2023-10-03T10:09:44.194745+0000","last_became_active":"2023-10-03T10:09:44.186523+0000","last_became_peered":"2023-10-03T10:09:44.186523+0000","last_unstale":"2023-10-03T10:09:44.194745+0000","last_undegraded":"2023-10-03T10:09:44.194745+0000","last_fullsized":"2023-10-03T10:09:44.194745+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.189715+0000","last_change":"2023-10-03T10:09:43.745046+0000","last_active":"2023-10-03T10:09:44.189715+0000","last_peered":"2023-10-03T10:09:44.189715+0000","last_clean":"2023-10-03T10:09:44.189715+0000","last_became_active":"2023-10-03T10:09:43.744724+0000","last_became_peered":"2023-10-03T10:09:43.744724+0000","last_unstale":"2023-10-03T10:09:44.189715+0000","last_undegraded":"2023-10-03T10:09:44.189715+0000","last_fullsized":"2023-10-03T10:09:44.189715+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195809+0000","last_change":"2023-10-03T10:09:44.190869+0000","last_active":"2023-10-03T10:09:44.195809+0000","last_peered":"2023-10-03T10:09:44.195809+0000","last_clean":"2023-10-03T10:09:44.195809+0000","last_became_active":"2023-10-03T10:09:44.190207+0000","last_became_peered":"2023-10-03T10:09:44.190207+0000","last_unstale":"2023-10-03T10:09:44.195809+0000","last_undegraded":"2023-10-03T10:09:44.195809+0000","last_fullsized":"2023-10-03T10:09:44.195809+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.152860+0000","last_change":"2023-10-03T10:09:43.750891+0000","last_active":"2023-10-03T10:09:44.152860+0000","last_peered":"2023-10-03T10:09:44.152860+0000","last_clean":"2023-10-03T10:09:44.152860+0000","last_became_active":"2023-10-03T10:09:43.750356+0000","last_became_peered":"2023-10-03T10:09:43.750356+0000","last_unstale":"2023-10-03T10:09:44.152860+0000","last_undegraded":"2023-10-03T10:09:44.152860+0000","last_fullsized":"2023-10-03T10:09:44.152860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184645+0000","last_change":"2023-10-03T10:09:44.183145+0000","last_active":"2023-10-03T10:09:44.184645+0000","last_peered":"2023-10-03T10:09:44.184645+0000","last_clean":"2023-10-03T10:09:44.184645+0000","last_became_active":"2023-10-03T10:09:44.182816+0000","last_became_peered":"2023-10-03T10:09:44.182816+0000","last_unstale":"2023-10-03T10:09:44.184645+0000","last_undegraded":"2023-10-03T10:09:44.184645+0000","last_fullsized":"2023-10-03T10:09:44.184645+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.163680+0000","last_change":"2023-10-03T10:09:43.749289+0000","last_active":"2023-10-03T10:09:44.163680+0000","last_peered":"2023-10-03T10:09:44.163680+0000","last_clean":"2023-10-03T10:09:44.163680+0000","last_became_active":"2023-10-03T10:09:43.748981+0000","last_became_peered":"2023-10-03T10:09:43.748981+0000","last_unstale":"2023-10-03T10:09:44.163680+0000","last_undegraded":"2023-10-03T10:09:44.163680+0000","last_fullsized":"2023-10-03T10:09:44.163680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.197002+0000","last_change":"2023-10-03T10:09:44.192692+0000","last_active":"2023-10-03T10:09:44.197002+0000","last_peered":"2023-10-03T10:09:44.197002+0000","last_clean":"2023-10-03T10:09:44.197002+0000","last_became_active":"2023-10-03T10:09:44.192390+0000","last_became_peered":"2023-10-03T10:09:44.192390+0000","last_unstale":"2023-10-03T10:09:44.197002+0000","last_undegraded":"2023-10-03T10:09:44.197002+0000","last_fullsized":"2023-10-03T10:09:44.197002+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.160877+0000","last_change":"2023-10-03T10:09:43.754802+0000","last_active":"2023-10-03T10:09:44.160877+0000","last_peered":"2023-10-03T10:09:44.160877+0000","last_clean":"2023-10-03T10:09:44.160877+0000","last_became_active":"2023-10-03T10:09:43.754512+0000","last_became_peered":"2023-10-03T10:09:43.754512+0000","last_unstale":"2023-10-03T10:09:44.160877+0000","last_undegraded":"2023-10-03T10:09:44.160877+0000","last_fullsized":"2023-10-03T10:09:44.160877+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196969+0000","last_change":"2023-10-03T10:09:44.192798+0000","last_active":"2023-10-03T10:09:44.196969+0000","last_peered":"2023-10-03T10:09:44.196969+0000","last_clean":"2023-10-03T10:09:44.196969+0000","last_became_active":"2023-10-03T10:09:44.190787+0000","last_became_peered":"2023-10-03T10:09:44.190787+0000","last_unstale":"2023-10-03T10:09:44.196969+0000","last_undegraded":"2023-10-03T10:09:44.196969+0000","last_fullsized":"2023-10-03T10:09:44.196969+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.157966+0000","last_change":"2023-10-03T10:09:43.756958+0000","last_active":"2023-10-03T10:09:44.157966+0000","last_peered":"2023-10-03T10:09:44.157966+0000","last_clean":"2023-10-03T10:09:44.157966+0000","last_became_active":"2023-10-03T10:09:43.756463+0000","last_became_peered":"2023-10-03T10:09:43.756463+0000","last_unstale":"2023-10-03T10:09:44.157966+0000","last_undegraded":"2023-10-03T10:09:44.157966+0000","last_fullsized":"2023-10-03T10:09:44.157966+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.194551+0000","last_change":"2023-10-03T10:09:44.184790+0000","last_active":"2023-10-03T10:09:44.194551+0000","last_peered":"2023-10-03T10:09:44.194551+0000","last_clean":"2023-10-03T10:09:44.194551+0000","last_became_active":"2023-10-03T10:09:44.184531+0000","last_became_peered":"2023-10-03T10:09:44.184531+0000","last_unstale":"2023-10-03T10:09:44.194551+0000","last_undegraded":"2023-10-03T10:09:44.194551+0000","last_fullsized":"2023-10-03T10:09:44.194551+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"16'3","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200426+0000","last_change":"2023-10-03T10:09:43.756598+0000","last_active":"2023-10-03T10:09:44.200426+0000","last_peered":"2023-10-03T10:09:44.200426+0000","last_clean":"2023-10-03T10:09:44.200426+0000","last_became_active":"2023-10-03T10:09:43.754974+0000","last_became_peered":"2023-10-03T10:09:43.754974+0000","last_unstale":"2023-10-03T10:09:44.200426+0000","last_undegraded":"2023-10-03T10:09:44.200426+0000","last_fullsized":"2023-10-03T10:09:44.200426+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180499+0000","last_change":"2023-10-03T10:09:44.175504+0000","last_active":"2023-10-03T10:09:44.180499+0000","last_peered":"2023-10-03T10:09:44.180499+0000","last_clean":"2023-10-03T10:09:44.180499+0000","last_became_active":"2023-10-03T10:09:44.175174+0000","last_became_peered":"2023-10-03T10:09:44.175174+0000","last_unstale":"2023-10-03T10:09:44.180499+0000","last_undegraded":"2023-10-03T10:09:44.180499+0000","last_fullsized":"2023-10-03T10:09:44.180499+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"16'6","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.203073+0000","last_change":"2023-10-03T10:09:43.758711+0000","last_active":"2023-10-03T10:09:44.203073+0000","last_peered":"2023-10-03T10:09:44.203073+0000","last_clean":"2023-10-03T10:09:44.203073+0000","last_became_active":"2023-10-03T10:09:43.758392+0000","last_became_peered":"2023-10-03T10:09:43.758392+0000","last_unstale":"2023-10-03T10:09:44.203073+0000","last_undegraded":"2023-10-03T10:09:44.203073+0000","last_fullsized":"2023-10-03T10:09:44.203073+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200336+0000","last_change":"2023-10-03T10:09:44.196559+0000","last_active":"2023-10-03T10:09:44.200336+0000","last_peered":"2023-10-03T10:09:44.200336+0000","last_clean":"2023-10-03T10:09:44.200336+0000","last_became_active":"2023-10-03T10:09:44.196280+0000","last_became_peered":"2023-10-03T10:09:44.196280+0000","last_unstale":"2023-10-03T10:09:44.200336+0000","last_undegraded":"2023-10-03T10:09:44.200336+0000","last_fullsized":"2023-10-03T10:09:44.200336+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200762+0000","last_change":"2023-10-03T10:09:43.751877+0000","last_active":"2023-10-03T10:09:44.200762+0000","last_peered":"2023-10-03T10:09:44.200762+0000","last_clean":"2023-10-03T10:09:44.200762+0000","last_became_active":"2023-10-03T10:09:43.749870+0000","last_became_peered":"2023-10-03T10:09:43.749870+0000","last_unstale":"2023-10-03T10:09:44.200762+0000","last_undegraded":"2023-10-03T10:09:44.200762+0000","last_fullsized":"2023-10-03T10:09:44.200762+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195297+0000","last_change":"2023-10-03T10:09:44.190230+0000","last_active":"2023-10-03T10:09:44.195297+0000","last_peered":"2023-10-03T10:09:44.195297+0000","last_clean":"2023-10-03T10:09:44.195297+0000","last_became_active":"2023-10-03T10:09:44.189342+0000","last_became_peered":"2023-10-03T10:09:44.189342+0000","last_unstale":"2023-10-03T10:09:44.195297+0000","last_undegraded":"2023-10-03T10:09:44.195297+0000","last_fullsized":"2023-10-03T10:09:44.195297+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.162475+0000","last_change":"2023-10-03T10:09:43.755271+0000","last_active":"2023-10-03T10:09:44.162475+0000","last_peered":"2023-10-03T10:09:44.162475+0000","last_clean":"2023-10-03T10:09:44.162475+0000","last_became_active":"2023-10-03T10:09:43.754957+0000","last_became_peered":"2023-10-03T10:09:43.754957+0000","last_unstale":"2023-10-03T10:09:44.162475+0000","last_undegraded":"2023-10-03T10:09:44.162475+0000","last_fullsized":"2023-10-03T10:09:44.162475+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.194802+0000","last_change":"2023-10-03T10:09:44.187333+0000","last_active":"2023-10-03T10:09:44.194802+0000","last_peered":"2023-10-03T10:09:44.194802+0000","last_clean":"2023-10-03T10:09:44.194802+0000","last_became_active":"2023-10-03T10:09:44.186824+0000","last_became_peered":"2023-10-03T10:09:44.186824+0000","last_unstale":"2023-10-03T10:09:44.194802+0000","last_undegraded":"2023-10-03T10:09:44.194802+0000","last_fullsized":"2023-10-03T10:09:44.194802+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.163844+0000","last_change":"2023-10-03T10:09:43.758995+0000","last_active":"2023-10-03T10:09:44.163844+0000","last_peered":"2023-10-03T10:09:44.163844+0000","last_clean":"2023-10-03T10:09:44.163844+0000","last_became_active":"2023-10-03T10:09:43.758565+0000","last_became_peered":"2023-10-03T10:09:43.758565+0000","last_unstale":"2023-10-03T10:09:44.163844+0000","last_undegraded":"2023-10-03T10:09:44.163844+0000","last_fullsized":"2023-10-03T10:09:44.163844+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173655+0000","last_change":"2023-10-03T10:09:44.166557+0000","last_active":"2023-10-03T10:09:44.173655+0000","last_peered":"2023-10-03T10:09:44.173655+0000","last_clean":"2023-10-03T10:09:44.173655+0000","last_became_active":"2023-10-03T10:09:44.166300+0000","last_became_peered":"2023-10-03T10:09:44.166300+0000","last_unstale":"2023-10-03T10:09:44.173655+0000","last_undegraded":"2023-10-03T10:09:44.173655+0000","last_fullsized":"2023-10-03T10:09:44.173655+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"16'8","reported_seq":18,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.194737+0000","last_change":"2023-10-03T10:09:43.748084+0000","last_active":"2023-10-03T10:09:44.194737+0000","last_peered":"2023-10-03T10:09:44.194737+0000","last_clean":"2023-10-03T10:09:44.194737+0000","last_became_active":"2023-10-03T10:09:43.747377+0000","last_became_peered":"2023-10-03T10:09:43.747377+0000","last_unstale":"2023-10-03T10:09:44.194737+0000","last_undegraded":"2023-10-03T10:09:44.194737+0000","last_fullsized":"2023-10-03T10:09:44.194737+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195326+0000","last_change":"2023-10-03T10:09:44.190767+0000","last_active":"2023-10-03T10:09:44.195326+0000","last_peered":"2023-10-03T10:09:44.195326+0000","last_clean":"2023-10-03T10:09:44.195326+0000","last_became_active":"2023-10-03T10:09:44.190514+0000","last_became_peered":"2023-10-03T10:09:44.190514+0000","last_unstale":"2023-10-03T10:09:44.195326+0000","last_undegraded":"2023-10-03T10:09:44.195326+0000","last_fullsized":"2023-10-03T10:09:44.195326+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.183000+0000","last_change":"2023-10-03T10:09:43.748461+0000","last_active":"2023-10-03T10:09:44.183000+0000","last_peered":"2023-10-03T10:09:44.183000+0000","last_clean":"2023-10-03T10:09:44.183000+0000","last_became_active":"2023-10-03T10:09:43.747372+0000","last_became_peered":"2023-10-03T10:09:43.747372+0000","last_unstale":"2023-10-03T10:09:44.183000+0000","last_undegraded":"2023-10-03T10:09:44.183000+0000","last_fullsized":"2023-10-03T10:09:44.183000+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196810+0000","last_change":"2023-10-03T10:09:44.192036+0000","last_active":"2023-10-03T10:09:44.196810+0000","last_peered":"2023-10-03T10:09:44.196810+0000","last_clean":"2023-10-03T10:09:44.196810+0000","last_became_active":"2023-10-03T10:09:44.191093+0000","last_became_peered":"2023-10-03T10:09:44.191093+0000","last_unstale":"2023-10-03T10:09:44.196810+0000","last_undegraded":"2023-10-03T10:09:44.196810+0000","last_fullsized":"2023-10-03T10:09:44.196810+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.174713+0000","last_change":"2023-10-03T10:09:43.756751+0000","last_active":"2023-10-03T10:09:44.174713+0000","last_peered":"2023-10-03T10:09:44.174713+0000","last_clean":"2023-10-03T10:09:44.174713+0000","last_became_active":"2023-10-03T10:09:43.756079+0000","last_became_peered":"2023-10-03T10:09:43.756079+0000","last_unstale":"2023-10-03T10:09:44.174713+0000","last_undegraded":"2023-10-03T10:09:44.174713+0000","last_fullsized":"2023-10-03T10:09:44.174713+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.179874+0000","last_change":"2023-10-03T10:09:44.172400+0000","last_active":"2023-10-03T10:09:44.179874+0000","last_peered":"2023-10-03T10:09:44.179874+0000","last_clean":"2023-10-03T10:09:44.179874+0000","last_became_active":"2023-10-03T10:09:44.172095+0000","last_became_peered":"2023-10-03T10:09:44.172095+0000","last_unstale":"2023-10-03T10:09:44.179874+0000","last_undegraded":"2023-10-03T10:09:44.179874+0000","last_fullsized":"2023-10-03T10:09:44.179874+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195109+0000","last_change":"2023-10-03T10:09:43.760622+0000","last_active":"2023-10-03T10:09:44.195109+0000","last_peered":"2023-10-03T10:09:44.195109+0000","last_clean":"2023-10-03T10:09:44.195109+0000","last_became_active":"2023-10-03T10:09:43.760371+0000","last_became_peered":"2023-10-03T10:09:43.760371+0000","last_unstale":"2023-10-03T10:09:44.195109+0000","last_undegraded":"2023-10-03T10:09:44.195109+0000","last_fullsized":"2023-10-03T10:09:44.195109+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200471+0000","last_change":"2023-10-03T10:09:44.196062+0000","last_active":"2023-10-03T10:09:44.200471+0000","last_peered":"2023-10-03T10:09:44.200471+0000","last_clean":"2023-10-03T10:09:44.200471+0000","last_became_active":"2023-10-03T10:09:44.195841+0000","last_became_peered":"2023-10-03T10:09:44.195841+0000","last_unstale":"2023-10-03T10:09:44.200471+0000","last_undegraded":"2023-10-03T10:09:44.200471+0000","last_fullsized":"2023-10-03T10:09:44.200471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.157166+0000","last_change":"2023-10-03T10:09:43.756885+0000","last_active":"2023-10-03T10:09:44.157166+0000","last_peered":"2023-10-03T10:09:44.157166+0000","last_clean":"2023-10-03T10:09:44.157166+0000","last_became_active":"2023-10-03T10:09:43.756588+0000","last_became_peered":"2023-10-03T10:09:43.756588+0000","last_unstale":"2023-10-03T10:09:44.157166+0000","last_undegraded":"2023-10-03T10:09:44.157166+0000","last_fullsized":"2023-10-03T10:09:44.157166+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184514+0000","last_change":"2023-10-03T10:09:44.181603+0000","last_active":"2023-10-03T10:09:44.184514+0000","last_peered":"2023-10-03T10:09:44.184514+0000","last_clean":"2023-10-03T10:09:44.184514+0000","last_became_active":"2023-10-03T10:09:44.181125+0000","last_became_peered":"2023-10-03T10:09:44.181125+0000","last_unstale":"2023-10-03T10:09:44.184514+0000","last_undegraded":"2023-10-03T10:09:44.184514+0000","last_fullsized":"2023-10-03T10:09:44.184514+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200822+0000","last_change":"2023-10-03T10:09:43.751873+0000","last_active":"2023-10-03T10:09:44.200822+0000","last_peered":"2023-10-03T10:09:44.200822+0000","last_clean":"2023-10-03T10:09:44.200822+0000","last_became_active":"2023-10-03T10:09:43.751324+0000","last_became_peered":"2023-10-03T10:09:43.751324+0000","last_unstale":"2023-10-03T10:09:44.200822+0000","last_undegraded":"2023-10-03T10:09:44.200822+0000","last_fullsized":"2023-10-03T10:09:44.200822+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.174302+0000","last_change":"2023-10-03T10:09:44.170676+0000","last_active":"2023-10-03T10:09:44.174302+0000","last_peered":"2023-10-03T10:09:44.174302+0000","last_clean":"2023-10-03T10:09:44.174302+0000","last_became_active":"2023-10-03T10:09:44.170398+0000","last_became_peered":"2023-10-03T10:09:44.170398+0000","last_unstale":"2023-10-03T10:09:44.174302+0000","last_undegraded":"2023-10-03T10:09:44.174302+0000","last_fullsized":"2023-10-03T10:09:44.174302+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.166137+0000","last_change":"2023-10-03T10:09:43.760711+0000","last_active":"2023-10-03T10:09:44.166137+0000","last_peered":"2023-10-03T10:09:44.166137+0000","last_clean":"2023-10-03T10:09:44.166137+0000","last_became_active":"2023-10-03T10:09:43.760432+0000","last_became_peered":"2023-10-03T10:09:43.760432+0000","last_unstale":"2023-10-03T10:09:44.166137+0000","last_undegraded":"2023-10-03T10:09:44.166137+0000","last_fullsized":"2023-10-03T10:09:44.166137+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195829+0000","last_change":"2023-10-03T10:09:44.191073+0000","last_active":"2023-10-03T10:09:44.195829+0000","last_peered":"2023-10-03T10:09:44.195829+0000","last_clean":"2023-10-03T10:09:44.195829+0000","last_became_active":"2023-10-03T10:09:44.190218+0000","last_became_peered":"2023-10-03T10:09:44.190218+0000","last_unstale":"2023-10-03T10:09:44.195829+0000","last_undegraded":"2023-10-03T10:09:44.195829+0000","last_fullsized":"2023-10-03T10:09:44.195829+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.163521+0000","last_change":"2023-10-03T10:09:43.758846+0000","last_active":"2023-10-03T10:09:44.163521+0000","last_peered":"2023-10-03T10:09:44.163521+0000","last_clean":"2023-10-03T10:09:44.163521+0000","last_became_active":"2023-10-03T10:09:43.757973+0000","last_became_peered":"2023-10-03T10:09:43.757973+0000","last_unstale":"2023-10-03T10:09:44.163521+0000","last_undegraded":"2023-10-03T10:09:44.163521+0000","last_fullsized":"2023-10-03T10:09:44.163521+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200102+0000","last_change":"2023-10-03T10:09:44.196056+0000","last_active":"2023-10-03T10:09:44.200102+0000","last_peered":"2023-10-03T10:09:44.200102+0000","last_clean":"2023-10-03T10:09:44.200102+0000","last_became_active":"2023-10-03T10:09:44.195817+0000","last_became_peered":"2023-10-03T10:09:44.195817+0000","last_unstale":"2023-10-03T10:09:44.200102+0000","last_undegraded":"2023-10-03T10:09:44.200102+0000","last_fullsized":"2023-10-03T10:09:44.200102+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.162993+0000","last_change":"2023-10-03T10:09:43.761068+0000","last_active":"2023-10-03T10:09:44.162993+0000","last_peered":"2023-10-03T10:09:44.162993+0000","last_clean":"2023-10-03T10:09:44.162993+0000","last_became_active":"2023-10-03T10:09:43.760825+0000","last_became_peered":"2023-10-03T10:09:43.760825+0000","last_unstale":"2023-10-03T10:09:44.162993+0000","last_undegraded":"2023-10-03T10:09:44.162993+0000","last_fullsized":"2023-10-03T10:09:44.162993+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180799+0000","last_change":"2023-10-03T10:09:44.178856+0000","last_active":"2023-10-03T10:09:44.180799+0000","last_peered":"2023-10-03T10:09:44.180799+0000","last_clean":"2023-10-03T10:09:44.180799+0000","last_became_active":"2023-10-03T10:09:44.178410+0000","last_became_peered":"2023-10-03T10:09:44.178410+0000","last_unstale":"2023-10-03T10:09:44.180799+0000","last_undegraded":"2023-10-03T10:09:44.180799+0000","last_fullsized":"2023-10-03T10:09:44.180799+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.154127+0000","last_change":"2023-10-03T10:09:39.589080+0000","last_active":"2023-10-03T10:09:44.154127+0000","last_peered":"2023-10-03T10:09:44.154127+0000","last_clean":"2023-10-03T10:09:44.154127+0000","last_became_active":"2023-10-03T10:09:39.588651+0000","last_became_peered":"2023-10-03T10:09:39.588651+0000","last_unstale":"2023-10-03T10:09:44.154127+0000","last_undegraded":"2023-10-03T10:09:44.154127+0000","last_fullsized":"2023-10-03T10:09:44.154127+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.157236+0000","last_change":"2023-10-03T10:09:43.756053+0000","last_active":"2023-10-03T10:09:44.157236+0000","last_peered":"2023-10-03T10:09:44.157236+0000","last_clean":"2023-10-03T10:09:44.157236+0000","last_became_active":"2023-10-03T10:09:43.755708+0000","last_became_peered":"2023-10-03T10:09:43.755708+0000","last_unstale":"2023-10-03T10:09:44.157236+0000","last_undegraded":"2023-10-03T10:09:44.157236+0000","last_fullsized":"2023-10-03T10:09:44.157236+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180543+0000","last_change":"2023-10-03T10:09:44.176964+0000","last_active":"2023-10-03T10:09:44.180543+0000","last_peered":"2023-10-03T10:09:44.180543+0000","last_clean":"2023-10-03T10:09:44.180543+0000","last_became_active":"2023-10-03T10:09:44.176671+0000","last_became_peered":"2023-10-03T10:09:44.176671+0000","last_unstale":"2023-10-03T10:09:44.180543+0000","last_undegraded":"2023-10-03T10:09:44.180543+0000","last_fullsized":"2023-10-03T10:09:44.180543+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.166922+0000","last_change":"2023-10-03T10:09:39.586735+0000","last_active":"2023-10-03T10:09:44.166922+0000","last_peered":"2023-10-03T10:09:44.166922+0000","last_clean":"2023-10-03T10:09:44.166922+0000","last_became_active":"2023-10-03T10:09:39.586440+0000","last_became_peered":"2023-10-03T10:09:39.586440+0000","last_unstale":"2023-10-03T10:09:44.166922+0000","last_undegraded":"2023-10-03T10:09:44.166922+0000","last_fullsized":"2023-10-03T10:09:44.166922+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.159439+0000","last_change":"2023-10-03T10:09:43.748458+0000","last_active":"2023-10-03T10:09:44.159439+0000","last_peered":"2023-10-03T10:09:44.159439+0000","last_clean":"2023-10-03T10:09:44.159439+0000","last_became_active":"2023-10-03T10:09:43.747721+0000","last_became_peered":"2023-10-03T10:09:43.747721+0000","last_unstale":"2023-10-03T10:09:44.159439+0000","last_undegraded":"2023-10-03T10:09:44.159439+0000","last_fullsized":"2023-10-03T10:09:44.159439+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184166+0000","last_change":"2023-10-03T10:09:44.176165+0000","last_active":"2023-10-03T10:09:44.184166+0000","last_peered":"2023-10-03T10:09:44.184166+0000","last_clean":"2023-10-03T10:09:44.184166+0000","last_became_active":"2023-10-03T10:09:44.175853+0000","last_became_peered":"2023-10-03T10:09:44.175853+0000","last_unstale":"2023-10-03T10:09:44.184166+0000","last_undegraded":"2023-10-03T10:09:44.184166+0000","last_fullsized":"2023-10-03T10:09:44.184166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.156829+0000","last_change":"2023-10-03T10:09:39.590114+0000","last_active":"2023-10-03T10:09:44.156829+0000","last_peered":"2023-10-03T10:09:44.156829+0000","last_clean":"2023-10-03T10:09:44.156829+0000","last_became_active":"2023-10-03T10:09:39.589663+0000","last_became_peered":"2023-10-03T10:09:39.589663+0000","last_unstale":"2023-10-03T10:09:44.156829+0000","last_undegraded":"2023-10-03T10:09:44.156829+0000","last_fullsized":"2023-10-03T10:09:44.156829+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.155654+0000","last_change":"2023-10-03T10:09:43.748267+0000","last_active":"2023-10-03T10:09:44.155654+0000","last_peered":"2023-10-03T10:09:44.155654+0000","last_clean":"2023-10-03T10:09:44.155654+0000","last_became_active":"2023-10-03T10:09:43.747789+0000","last_became_peered":"2023-10-03T10:09:43.747789+0000","last_unstale":"2023-10-03T10:09:44.155654+0000","last_undegraded":"2023-10-03T10:09:44.155654+0000","last_fullsized":"2023-10-03T10:09:44.155654+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180773+0000","last_change":"2023-10-03T10:09:44.178680+0000","last_active":"2023-10-03T10:09:44.180773+0000","last_peered":"2023-10-03T10:09:44.180773+0000","last_clean":"2023-10-03T10:09:44.180773+0000","last_became_active":"2023-10-03T10:09:44.178114+0000","last_became_peered":"2023-10-03T10:09:44.178114+0000","last_unstale":"2023-10-03T10:09:44.180773+0000","last_undegraded":"2023-10-03T10:09:44.180773+0000","last_fullsized":"2023-10-03T10:09:44.180773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.167768+0000","last_change":"2023-10-03T10:09:39.587728+0000","last_active":"2023-10-03T10:09:44.167768+0000","last_peered":"2023-10-03T10:09:44.167768+0000","last_clean":"2023-10-03T10:09:44.167768+0000","last_became_active":"2023-10-03T10:09:39.587441+0000","last_became_peered":"2023-10-03T10:09:39.587441+0000","last_unstale":"2023-10-03T10:09:44.167768+0000","last_undegraded":"2023-10-03T10:09:44.167768+0000","last_fullsized":"2023-10-03T10:09:44.167768+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.153191+0000","last_change":"2023-10-03T10:09:38.585668+0000","last_active":"2023-10-03T10:09:44.153191+0000","last_peered":"2023-10-03T10:09:44.153191+0000","last_clean":"2023-10-03T10:09:44.153191+0000","last_became_active":"2023-10-03T10:09:38.585324+0000","last_became_peered":"2023-10-03T10:09:38.585324+0000","last_unstale":"2023-10-03T10:09:44.153191+0000","last_undegraded":"2023-10-03T10:09:44.153191+0000","last_fullsized":"2023-10-03T10:09:44.153191+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:37.570571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:37.570571+0000","last_clean_scrub_stamp":"2023-10-03T10:09:37.570571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.165999+0000","last_change":"2023-10-03T10:09:43.758096+0000","last_active":"2023-10-03T10:09:44.165999+0000","last_peered":"2023-10-03T10:09:44.165999+0000","last_clean":"2023-10-03T10:09:44.165999+0000","last_became_active":"2023-10-03T10:09:43.757689+0000","last_became_peered":"2023-10-03T10:09:43.757689+0000","last_unstale":"2023-10-03T10:09:44.165999+0000","last_undegraded":"2023-10-03T10:09:44.165999+0000","last_fullsized":"2023-10-03T10:09:44.165999+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196949+0000","last_change":"2023-10-03T10:09:44.192781+0000","last_active":"2023-10-03T10:09:44.196949+0000","last_peered":"2023-10-03T10:09:44.196949+0000","last_clean":"2023-10-03T10:09:44.196949+0000","last_became_active":"2023-10-03T10:09:44.190597+0000","last_became_peered":"2023-10-03T10:09:44.190597+0000","last_unstale":"2023-10-03T10:09:44.196949+0000","last_undegraded":"2023-10-03T10:09:44.196949+0000","last_fullsized":"2023-10-03T10:09:44.196949+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.163365+0000","last_change":"2023-10-03T10:09:39.586806+0000","last_active":"2023-10-03T10:09:44.163365+0000","last_peered":"2023-10-03T10:09:44.163365+0000","last_clean":"2023-10-03T10:09:44.163365+0000","last_became_active":"2023-10-03T10:09:39.586481+0000","last_became_peered":"2023-10-03T10:09:39.586481+0000","last_unstale":"2023-10-03T10:09:44.163365+0000","last_undegraded":"2023-10-03T10:09:44.163365+0000","last_fullsized":"2023-10-03T10:09:44.163365+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.156596+0000","last_change":"2023-10-03T10:09:43.747783+0000","last_active":"2023-10-03T10:09:44.156596+0000","last_peered":"2023-10-03T10:09:44.156596+0000","last_clean":"2023-10-03T10:09:44.156596+0000","last_became_active":"2023-10-03T10:09:43.746009+0000","last_became_peered":"2023-10-03T10:09:43.746009+0000","last_unstale":"2023-10-03T10:09:44.156596+0000","last_undegraded":"2023-10-03T10:09:44.156596+0000","last_fullsized":"2023-10-03T10:09:44.156596+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184590+0000","last_change":"2023-10-03T10:09:44.181359+0000","last_active":"2023-10-03T10:09:44.184590+0000","last_peered":"2023-10-03T10:09:44.184590+0000","last_clean":"2023-10-03T10:09:44.184590+0000","last_became_active":"2023-10-03T10:09:44.181075+0000","last_became_peered":"2023-10-03T10:09:44.181075+0000","last_unstale":"2023-10-03T10:09:44.184590+0000","last_undegraded":"2023-10-03T10:09:44.184590+0000","last_fullsized":"2023-10-03T10:09:44.184590+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.168332+0000","last_change":"2023-10-03T10:09:39.588827+0000","last_active":"2023-10-03T10:09:44.168332+0000","last_peered":"2023-10-03T10:09:44.168332+0000","last_clean":"2023-10-03T10:09:44.168332+0000","last_became_active":"2023-10-03T10:09:39.588531+0000","last_became_peered":"2023-10-03T10:09:39.588531+0000","last_unstale":"2023-10-03T10:09:44.168332+0000","last_undegraded":"2023-10-03T10:09:44.168332+0000","last_fullsized":"2023-10-03T10:09:44.168332+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.152966+0000","last_change":"2023-10-03T10:09:43.746339+0000","last_active":"2023-10-03T10:09:44.152966+0000","last_peered":"2023-10-03T10:09:44.152966+0000","last_clean":"2023-10-03T10:09:44.152966+0000","last_became_active":"2023-10-03T10:09:43.746051+0000","last_became_peered":"2023-10-03T10:09:43.746051+0000","last_unstale":"2023-10-03T10:09:44.152966+0000","last_undegraded":"2023-10-03T10:09:44.152966+0000","last_fullsized":"2023-10-03T10:09:44.152966+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173149+0000","last_change":"2023-10-03T10:09:44.165684+0000","last_active":"2023-10-03T10:09:44.173149+0000","last_peered":"2023-10-03T10:09:44.173149+0000","last_clean":"2023-10-03T10:09:44.173149+0000","last_became_active":"2023-10-03T10:09:44.165422+0000","last_became_peered":"2023-10-03T10:09:44.165422+0000","last_unstale":"2023-10-03T10:09:44.173149+0000","last_undegraded":"2023-10-03T10:09:44.173149+0000","last_fullsized":"2023-10-03T10:09:44.173149+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.197330+0000","last_change":"2023-10-03T10:09:43.760807+0000","last_active":"2023-10-03T10:09:44.197330+0000","last_peered":"2023-10-03T10:09:44.197330+0000","last_clean":"2023-10-03T10:09:44.197330+0000","last_became_active":"2023-10-03T10:09:43.760517+0000","last_became_peered":"2023-10-03T10:09:43.760517+0000","last_unstale":"2023-10-03T10:09:44.197330+0000","last_undegraded":"2023-10-03T10:09:44.197330+0000","last_fullsized":"2023-10-03T10:09:44.197330+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.153815+0000","last_change":"2023-10-03T10:09:39.586702+0000","last_active":"2023-10-03T10:09:44.153815+0000","last_peered":"2023-10-03T10:09:44.153815+0000","last_clean":"2023-10-03T10:09:44.153815+0000","last_became_active":"2023-10-03T10:09:39.586153+0000","last_became_peered":"2023-10-03T10:09:39.586153+0000","last_unstale":"2023-10-03T10:09:44.153815+0000","last_undegraded":"2023-10-03T10:09:44.153815+0000","last_fullsized":"2023-10-03T10:09:44.153815+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196883+0000","last_change":"2023-10-03T10:09:44.192664+0000","last_active":"2023-10-03T10:09:44.196883+0000","last_peered":"2023-10-03T10:09:44.196883+0000","last_clean":"2023-10-03T10:09:44.196883+0000","last_became_active":"2023-10-03T10:09:44.190513+0000","last_became_peered":"2023-10-03T10:09:44.190513+0000","last_unstale":"2023-10-03T10:09:44.196883+0000","last_undegraded":"2023-10-03T10:09:44.196883+0000","last_fullsized":"2023-10-03T10:09:44.196883+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.167083+0000","last_change":"2023-10-03T10:09:39.588950+0000","last_active":"2023-10-03T10:09:44.167083+0000","last_peered":"2023-10-03T10:09:44.167083+0000","last_clean":"2023-10-03T10:09:44.167083+0000","last_became_active":"2023-10-03T10:09:39.588601+0000","last_became_peered":"2023-10-03T10:09:39.588601+0000","last_unstale":"2023-10-03T10:09:44.167083+0000","last_undegraded":"2023-10-03T10:09:44.167083+0000","last_fullsized":"2023-10-03T10:09:44.167083+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.160538+0000","last_change":"2023-10-03T10:09:43.752793+0000","last_active":"2023-10-03T10:09:44.160538+0000","last_peered":"2023-10-03T10:09:44.160538+0000","last_clean":"2023-10-03T10:09:44.160538+0000","last_became_active":"2023-10-03T10:09:43.752374+0000","last_became_peered":"2023-10-03T10:09:43.752374+0000","last_unstale":"2023-10-03T10:09:44.160538+0000","last_undegraded":"2023-10-03T10:09:44.160538+0000","last_fullsized":"2023-10-03T10:09:44.160538+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200171+0000","last_change":"2023-10-03T10:09:44.196314+0000","last_active":"2023-10-03T10:09:44.200171+0000","last_peered":"2023-10-03T10:09:44.200171+0000","last_clean":"2023-10-03T10:09:44.200171+0000","last_became_active":"2023-10-03T10:09:44.195901+0000","last_became_peered":"2023-10-03T10:09:44.195901+0000","last_unstale":"2023-10-03T10:09:44.200171+0000","last_undegraded":"2023-10-03T10:09:44.200171+0000","last_fullsized":"2023-10-03T10:09:44.200171+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200135+0000","last_change":"2023-10-03T10:09:43.756460+0000","last_active":"2023-10-03T10:09:44.200135+0000","last_peered":"2023-10-03T10:09:44.200135+0000","last_clean":"2023-10-03T10:09:44.200135+0000","last_became_active":"2023-10-03T10:09:43.756164+0000","last_became_peered":"2023-10-03T10:09:43.756164+0000","last_unstale":"2023-10-03T10:09:44.200135+0000","last_undegraded":"2023-10-03T10:09:44.200135+0000","last_fullsized":"2023-10-03T10:09:44.200135+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.189031+0000","last_change":"2023-10-03T10:09:44.181915+0000","last_active":"2023-10-03T10:09:44.189031+0000","last_peered":"2023-10-03T10:09:44.189031+0000","last_clean":"2023-10-03T10:09:44.189031+0000","last_became_active":"2023-10-03T10:09:44.181645+0000","last_became_peered":"2023-10-03T10:09:44.181645+0000","last_unstale":"2023-10-03T10:09:44.189031+0000","last_undegraded":"2023-10-03T10:09:44.189031+0000","last_fullsized":"2023-10-03T10:09:44.189031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.193051+0000","last_change":"2023-10-03T10:09:43.748380+0000","last_active":"2023-10-03T10:09:44.193051+0000","last_peered":"2023-10-03T10:09:44.193051+0000","last_clean":"2023-10-03T10:09:44.193051+0000","last_became_active":"2023-10-03T10:09:43.748104+0000","last_became_peered":"2023-10-03T10:09:43.748104+0000","last_unstale":"2023-10-03T10:09:44.193051+0000","last_undegraded":"2023-10-03T10:09:44.193051+0000","last_fullsized":"2023-10-03T10:09:44.193051+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200359+0000","last_change":"2023-10-03T10:09:44.197710+0000","last_active":"2023-10-03T10:09:44.200359+0000","last_peered":"2023-10-03T10:09:44.200359+0000","last_clean":"2023-10-03T10:09:44.200359+0000","last_became_active":"2023-10-03T10:09:44.197064+0000","last_became_peered":"2023-10-03T10:09:44.197064+0000","last_unstale":"2023-10-03T10:09:44.200359+0000","last_undegraded":"2023-10-03T10:09:44.200359+0000","last_fullsized":"2023-10-03T10:09:44.200359+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.193435+0000","last_change":"2023-10-03T10:09:43.758701+0000","last_active":"2023-10-03T10:09:44.193435+0000","last_peered":"2023-10-03T10:09:44.193435+0000","last_clean":"2023-10-03T10:09:44.193435+0000","last_became_active":"2023-10-03T10:09:43.758376+0000","last_became_peered":"2023-10-03T10:09:43.758376+0000","last_unstale":"2023-10-03T10:09:44.193435+0000","last_undegraded":"2023-10-03T10:09:44.193435+0000","last_fullsized":"2023-10-03T10:09:44.193435+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200488+0000","last_change":"2023-10-03T10:09:44.197164+0000","last_active":"2023-10-03T10:09:44.200488+0000","last_peered":"2023-10-03T10:09:44.200488+0000","last_clean":"2023-10-03T10:09:44.200488+0000","last_became_active":"2023-10-03T10:09:44.196885+0000","last_became_peered":"2023-10-03T10:09:44.196885+0000","last_unstale":"2023-10-03T10:09:44.200488+0000","last_undegraded":"2023-10-03T10:09:44.200488+0000","last_fullsized":"2023-10-03T10:09:44.200488+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.199237+0000","last_change":"2023-10-03T10:09:43.758373+0000","last_active":"2023-10-03T10:09:44.199237+0000","last_peered":"2023-10-03T10:09:44.199237+0000","last_clean":"2023-10-03T10:09:44.199237+0000","last_became_active":"2023-10-03T10:09:43.758061+0000","last_became_peered":"2023-10-03T10:09:43.758061+0000","last_unstale":"2023-10-03T10:09:44.199237+0000","last_undegraded":"2023-10-03T10:09:44.199237+0000","last_fullsized":"2023-10-03T10:09:44.199237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196006+0000","last_change":"2023-10-03T10:09:44.191564+0000","last_active":"2023-10-03T10:09:44.196006+0000","last_peered":"2023-10-03T10:09:44.196006+0000","last_clean":"2023-10-03T10:09:44.196006+0000","last_became_active":"2023-10-03T10:09:44.191220+0000","last_became_peered":"2023-10-03T10:09:44.191220+0000","last_unstale":"2023-10-03T10:09:44.196006+0000","last_undegraded":"2023-10-03T10:09:44.196006+0000","last_fullsized":"2023-10-03T10:09:44.196006+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"16'4","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.183675+0000","last_change":"2023-10-03T10:09:43.750463+0000","last_active":"2023-10-03T10:09:44.183675+0000","last_peered":"2023-10-03T10:09:44.183675+0000","last_clean":"2023-10-03T10:09:44.183675+0000","last_became_active":"2023-10-03T10:09:43.749780+0000","last_became_peered":"2023-10-03T10:09:43.749780+0000","last_unstale":"2023-10-03T10:09:44.183675+0000","last_undegraded":"2023-10-03T10:09:44.183675+0000","last_fullsized":"2023-10-03T10:09:44.183675+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.188984+0000","last_change":"2023-10-03T10:09:44.180378+0000","last_active":"2023-10-03T10:09:44.188984+0000","last_peered":"2023-10-03T10:09:44.188984+0000","last_clean":"2023-10-03T10:09:44.188984+0000","last_became_active":"2023-10-03T10:09:44.180082+0000","last_became_peered":"2023-10-03T10:09:44.180082+0000","last_unstale":"2023-10-03T10:09:44.188984+0000","last_undegraded":"2023-10-03T10:09:44.188984+0000","last_fullsized":"2023-10-03T10:09:44.188984+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.187808+0000","last_change":"2023-10-03T10:09:44.178067+0000","last_active":"2023-10-03T10:09:44.187808+0000","last_peered":"2023-10-03T10:09:44.187808+0000","last_clean":"2023-10-03T10:09:44.187808+0000","last_became_active":"2023-10-03T10:09:44.175124+0000","last_became_peered":"2023-10-03T10:09:44.175124+0000","last_unstale":"2023-10-03T10:09:44.187808+0000","last_undegraded":"2023-10-03T10:09:44.187808+0000","last_fullsized":"2023-10-03T10:09:44.187808+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.197083+0000","last_change":"2023-10-03T10:09:44.192156+0000","last_active":"2023-10-03T10:09:44.197083+0000","last_peered":"2023-10-03T10:09:44.197083+0000","last_clean":"2023-10-03T10:09:44.197083+0000","last_became_active":"2023-10-03T10:09:44.189296+0000","last_became_peered":"2023-10-03T10:09:44.189296+0000","last_unstale":"2023-10-03T10:09:44.197083+0000","last_undegraded":"2023-10-03T10:09:44.197083+0000","last_fullsized":"2023-10-03T10:09:44.197083+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.174106+0000","last_change":"2023-10-03T10:09:44.168473+0000","last_active":"2023-10-03T10:09:44.174106+0000","last_peered":"2023-10-03T10:09:44.174106+0000","last_clean":"2023-10-03T10:09:44.174106+0000","last_became_active":"2023-10-03T10:09:44.168155+0000","last_became_peered":"2023-10-03T10:09:44.168155+0000","last_unstale":"2023-10-03T10:09:44.174106+0000","last_undegraded":"2023-10-03T10:09:44.174106+0000","last_fullsized":"2023-10-03T10:09:44.174106+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200881+0000","last_change":"2023-10-03T10:09:44.197689+0000","last_active":"2023-10-03T10:09:44.200881+0000","last_peered":"2023-10-03T10:09:44.200881+0000","last_clean":"2023-10-03T10:09:44.200881+0000","last_became_active":"2023-10-03T10:09:44.197410+0000","last_became_peered":"2023-10-03T10:09:44.197410+0000","last_unstale":"2023-10-03T10:09:44.200881+0000","last_undegraded":"2023-10-03T10:09:44.200881+0000","last_fullsized":"2023-10-03T10:09:44.200881+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184190+0000","last_change":"2023-10-03T10:09:44.176966+0000","last_active":"2023-10-03T10:09:44.184190+0000","last_peered":"2023-10-03T10:09:44.184190+0000","last_clean":"2023-10-03T10:09:44.184190+0000","last_became_active":"2023-10-03T10:09:44.176386+0000","last_became_peered":"2023-10-03T10:09:44.176386+0000","last_unstale":"2023-10-03T10:09:44.184190+0000","last_undegraded":"2023-10-03T10:09:44.184190+0000","last_fullsized":"2023-10-03T10:09:44.184190+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705667,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297252,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074588,"statfs":{"total":96636764160,"available":96332378112,"internally_reserved":0,"allocated":159744,"data_stored":48735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]}]},{"osd":3,"up_from":9,"seq":38654705667,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297264,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074576,"statfs":{"total":96636764160,"available":96332365824,"internally_reserved":0,"allocated":172032,"data_stored":48929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]}]},{"osd":0,"up_from":9,"seq":38654705667,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297256,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074584,"statfs":{"total":96636764160,"available":96332374016,"internally_reserved":0,"allocated":163840,"data_stored":47899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]}]},{"osd":1,"up_from":9,"seq":38654705667,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297252,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074588,"statfs":{"total":96636764160,"available":96332378112,"internally_reserved":0,"allocated":159744,"data_stored":47845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-03T10:09:52.775 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-03T10:09:52.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.913+0000 7f632840a700 1 Processor -- start 2023-10-03T10:09:52.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.913+0000 7f632840a700 1 -- start start 2023-10-03T10:09:52.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.914+0000 7f632840a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6320062a90 0x7f6320127860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:52.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.914+0000 7f632840a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6320127360 0x7f6320127d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:52.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.914+0000 7f632840a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6320095300 0x7f6320095780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:52.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.914+0000 7f632840a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6320062e70 con 0x7f6320127360 2023-10-03T10:09:52.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.914+0000 7f632840a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63200968b0 con 0x7f6320062a90 2023-10-03T10:09:52.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.914+0000 7f632840a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6320091bc0 con 0x7f6320095300 2023-10-03T10:09:52.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.914+0000 7f63259a5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6320127360 0x7f6320127d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:52.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.914+0000 7f63261a6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6320062a90 0x7f6320127860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:52.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.914+0000 7f63269a7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6320095300 0x7f6320095780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:52.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.914+0000 7f63259a5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6320127360 0x7f6320127d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35546/0 (socket says 172.21.15.3:35546) 2023-10-03T10:09:52.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.914+0000 7f63261a6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6320062a90 0x7f6320127860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34626/0 (socket says 172.21.15.3:34626) 2023-10-03T10:09:52.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.914+0000 7f63259a5700 1 -- 172.21.15.3:0/3130769867 learned_addr learned my addr 172.21.15.3:0/3130769867 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:52.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.915+0000 7f63269a7700 1 -- 172.21.15.3:0/3130769867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6320062a90 msgr2=0x7f6320127860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:52.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.915+0000 7f63269a7700 1 --2- 172.21.15.3:0/3130769867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6320062a90 0x7f6320127860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:52.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.915+0000 7f63269a7700 1 -- 172.21.15.3:0/3130769867 >> v2:172.21.15.3:3300/0 conn(0x7f6320127360 msgr2=0x7f6320127d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:52.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.915+0000 7f63269a7700 1 --2- 172.21.15.3:0/3130769867 >> v2:172.21.15.3:3300/0 conn(0x7f6320127360 0x7f6320127d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:52.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.915+0000 7f63269a7700 1 -- 172.21.15.3:0/3130769867 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6320092a70 con 0x7f6320095300 2023-10-03T10:09:52.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.915+0000 7f63261a6700 1 --2- 172.21.15.3:0/3130769867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6320062a90 0x7f6320127860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:52.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.915+0000 7f63259a5700 1 --2- 172.21.15.3:0/3130769867 >> v2:172.21.15.3:3300/0 conn(0x7f6320127360 0x7f6320127d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:52.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.915+0000 7f63269a7700 1 --2- 172.21.15.3:0/3130769867 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6320095300 0x7f6320095780 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f6318009e60 tx=0x7f6318009780).ready entity=mon.2 client_cookie=f88ee6b1795f54e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:52.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.915+0000 7f63251a4700 1 -- 172.21.15.3:0/3130769867 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631800e070 con 0x7f6320095300 2023-10-03T10:09:52.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.915+0000 7f63251a4700 1 -- 172.21.15.3:0/3130769867 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631800bab0 con 0x7f6320095300 2023-10-03T10:09:52.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.915+0000 7f63251a4700 1 -- 172.21.15.3:0/3130769867 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631800be30 con 0x7f6320095300 2023-10-03T10:09:52.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.915+0000 7f632840a700 1 -- 172.21.15.3:0/3130769867 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6320095300 msgr2=0x7f6320095780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:52.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.915+0000 7f632840a700 1 --2- 172.21.15.3:0/3130769867 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6320095300 0x7f6320095780 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f6318009e60 tx=0x7f6318009780).stop 2023-10-03T10:09:52.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.916+0000 7f632840a700 1 -- 172.21.15.3:0/3130769867 shutdown_connections 2023-10-03T10:09:52.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.916+0000 7f632840a700 1 --2- 172.21.15.3:0/3130769867 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6320095300 0x7f6320095780 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:52.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.916+0000 7f632840a700 1 --2- 172.21.15.3:0/3130769867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6320062a90 0x7f6320127860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:52.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.916+0000 7f632840a700 1 --2- 172.21.15.3:0/3130769867 >> v2:172.21.15.3:3300/0 conn(0x7f6320127360 0x7f6320127d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:52.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.916+0000 7f632840a700 1 -- 172.21.15.3:0/3130769867 >> 172.21.15.3:0/3130769867 conn(0x7f6320059bb0 msgr2=0x7f6320097400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:52.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.916+0000 7f632840a700 1 -- 172.21.15.3:0/3130769867 shutdown_connections 2023-10-03T10:09:52.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.916+0000 7f632840a700 1 -- 172.21.15.3:0/3130769867 wait complete. 2023-10-03T10:09:52.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.916+0000 7f632840a700 1 Processor -- start 2023-10-03T10:09:52.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.917+0000 7f632840a700 1 -- start start 2023-10-03T10:09:52.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.917+0000 7f632840a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6320062a90 0x7f6320114e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:52.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.917+0000 7f632840a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6320095300 0x7f6320115350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:52.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.917+0000 7f632840a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6320127360 0x7f6320139d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:52.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.917+0000 7f632840a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6320126070 con 0x7f6320062a90 2023-10-03T10:09:52.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.917+0000 7f632840a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6320096e90 con 0x7f6320095300 2023-10-03T10:09:52.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.917+0000 7f632840a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6320126a80 con 0x7f6320127360 2023-10-03T10:09:52.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.917+0000 7f63259a5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6320095300 0x7f6320115350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:52.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.917+0000 7f63261a6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6320062a90 0x7f6320114e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:52.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.917+0000 7f63269a7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6320127360 0x7f6320139d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:52.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.917+0000 7f63261a6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6320062a90 0x7f6320114e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35552/0 (socket says 172.21.15.3:35552) 2023-10-03T10:09:52.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.917+0000 7f63259a5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6320095300 0x7f6320115350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34638/0 (socket says 172.21.15.3:34638) 2023-10-03T10:09:52.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.917+0000 7f63261a6700 1 -- 172.21.15.3:0/4259214210 learned_addr learned my addr 172.21.15.3:0/4259214210 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:52.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.917+0000 7f63269a7700 1 -- 172.21.15.3:0/4259214210 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6320095300 msgr2=0x7f6320115350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:52.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.917+0000 7f63269a7700 1 --2- 172.21.15.3:0/4259214210 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6320095300 0x7f6320115350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:52.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.917+0000 7f63269a7700 1 -- 172.21.15.3:0/4259214210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6320062a90 msgr2=0x7f6320114e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:52.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.917+0000 7f63269a7700 1 --2- 172.21.15.3:0/4259214210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6320062a90 0x7f6320114e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:52.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.917+0000 7f63269a7700 1 -- 172.21.15.3:0/4259214210 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631c0092a0 con 0x7f6320127360 2023-10-03T10:09:52.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.917+0000 7f63261a6700 1 --2- 172.21.15.3:0/4259214210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6320062a90 0x7f6320114e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:52.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.918+0000 7f63269a7700 1 --2- 172.21.15.3:0/4259214210 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6320127360 0x7f6320139d70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f6318014000 tx=0x7f6320092580).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:52.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.918+0000 7f63167fc700 1 -- 172.21.15.3:0/4259214210 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6318013e40 con 0x7f6320127360 2023-10-03T10:09:52.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.918+0000 7f63167fc700 1 -- 172.21.15.3:0/4259214210 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631801d630 con 0x7f6320127360 2023-10-03T10:09:52.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.918+0000 7f632840a700 1 -- 172.21.15.3:0/4259214210 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6310009370 con 0x7f6320127360 2023-10-03T10:09:52.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.918+0000 7f63167fc700 1 -- 172.21.15.3:0/4259214210 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631801d9f0 con 0x7f6320127360 2023-10-03T10:09:52.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.918+0000 7f632840a700 1 -- 172.21.15.3:0/4259214210 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6318009a20 con 0x7f6320127360 2023-10-03T10:09:52.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.919+0000 7f63167fc700 1 -- 172.21.15.3:0/4259214210 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f6318003040 con 0x7f6320127360 2023-10-03T10:09:52.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.919+0000 7f63167fc700 1 -- 172.21.15.3:0/4259214210 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f631803cac0 con 0x7f6320127360 2023-10-03T10:09:52.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.919+0000 7f632840a700 1 -- 172.21.15.3:0/4259214210 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6320117b50 con 0x7f6320127360 2023-10-03T10:09:52.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:52.924+0000 7f63167fc700 1 -- 172.21.15.3:0/4259214210 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f631801dda0 con 0x7f6320127360 2023-10-03T10:09:53.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.097+0000 7f632840a700 1 -- 172.21.15.3:0/4259214210 --> v1:172.21.15.3:6816/118296 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6320117e10 con 0x7f630402bb60 2023-10-03T10:09:53.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.104+0000 7f63167fc700 1 -- 172.21.15.3:0/4259214210 <== mgr.4100 v1:172.21.15.3:6816/118296 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249995 (unknown 2965378022 0 3262723905) 0x7f631c009cf0 con 0x7f630402bb60 2023-10-03T10:09:53.107 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:53.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.107+0000 7f632840a700 1 -- 172.21.15.3:0/4259214210 >> v1:172.21.15.3:6816/118296 conn(0x7f630402bb60 legacy=0x7f630402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:53.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.107+0000 7f632840a700 1 -- 172.21.15.3:0/4259214210 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6320127360 msgr2=0x7f6320139d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:53.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.107+0000 7f632840a700 1 --2- 172.21.15.3:0/4259214210 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6320127360 0x7f6320139d70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f6318014000 tx=0x7f6320092580).stop 2023-10-03T10:09:53.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.107+0000 7f632840a700 1 -- 172.21.15.3:0/4259214210 shutdown_connections 2023-10-03T10:09:53.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.107+0000 7f632840a700 1 --2- 172.21.15.3:0/4259214210 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6320127360 0x7f6320139d70 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:53.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.107+0000 7f632840a700 1 --2- 172.21.15.3:0/4259214210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6320062a90 0x7f6320114e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:53.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.107+0000 7f632840a700 1 --2- 172.21.15.3:0/4259214210 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6320095300 0x7f6320115350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:53.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.107+0000 7f632840a700 1 -- 172.21.15.3:0/4259214210 >> 172.21.15.3:0/4259214210 conn(0x7f6320059bb0 msgr2=0x7f63200999a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:53.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.107+0000 7f632840a700 1 -- 172.21.15.3:0/4259214210 shutdown_connections 2023-10-03T10:09:53.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.107+0000 7f632840a700 1 -- 172.21.15.3:0/4259214210 wait complete. 2023-10-03T10:09:53.110 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-03T10:09:53.120 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-03T10:09:51.530019+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1189024,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298336,"statfs":{"total":386547056640,"available":385329496064,"internally_reserved":0,"allocated":655360,"data_stored":193408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000863"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.199042+0000","last_change":"2023-10-03T10:09:44.195682+0000","last_active":"2023-10-03T10:09:44.199042+0000","last_peered":"2023-10-03T10:09:44.199042+0000","last_clean":"2023-10-03T10:09:44.199042+0000","last_became_active":"2023-10-03T10:09:44.195360+0000","last_became_peered":"2023-10-03T10:09:44.195360+0000","last_unstale":"2023-10-03T10:09:44.199042+0000","last_undegraded":"2023-10-03T10:09:44.199042+0000","last_fullsized":"2023-10-03T10:09:44.199042+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180036+0000","last_change":"2023-10-03T10:09:44.178263+0000","last_active":"2023-10-03T10:09:44.180036+0000","last_peered":"2023-10-03T10:09:44.180036+0000","last_clean":"2023-10-03T10:09:44.180036+0000","last_became_active":"2023-10-03T10:09:44.177977+0000","last_became_peered":"2023-10-03T10:09:44.177977+0000","last_unstale":"2023-10-03T10:09:44.180036+0000","last_undegraded":"2023-10-03T10:09:44.180036+0000","last_fullsized":"2023-10-03T10:09:44.180036+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.190395+0000","last_change":"2023-10-03T10:09:44.183364+0000","last_active":"2023-10-03T10:09:44.190395+0000","last_peered":"2023-10-03T10:09:44.190395+0000","last_clean":"2023-10-03T10:09:44.190395+0000","last_became_active":"2023-10-03T10:09:44.183094+0000","last_became_peered":"2023-10-03T10:09:44.183094+0000","last_unstale":"2023-10-03T10:09:44.190395+0000","last_undegraded":"2023-10-03T10:09:44.190395+0000","last_fullsized":"2023-10-03T10:09:44.190395+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.190339+0000","last_change":"2023-10-03T10:09:44.183273+0000","last_active":"2023-10-03T10:09:44.190339+0000","last_peered":"2023-10-03T10:09:44.190339+0000","last_clean":"2023-10-03T10:09:44.190339+0000","last_became_active":"2023-10-03T10:09:44.182451+0000","last_became_peered":"2023-10-03T10:09:44.182451+0000","last_unstale":"2023-10-03T10:09:44.190339+0000","last_undegraded":"2023-10-03T10:09:44.190339+0000","last_fullsized":"2023-10-03T10:09:44.190339+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.174191+0000","last_change":"2023-10-03T10:09:44.168483+0000","last_active":"2023-10-03T10:09:44.174191+0000","last_peered":"2023-10-03T10:09:44.174191+0000","last_clean":"2023-10-03T10:09:44.174191+0000","last_became_active":"2023-10-03T10:09:44.168186+0000","last_became_peered":"2023-10-03T10:09:44.168186+0000","last_unstale":"2023-10-03T10:09:44.174191+0000","last_undegraded":"2023-10-03T10:09:44.174191+0000","last_fullsized":"2023-10-03T10:09:44.174191+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.198209+0000","last_change":"2023-10-03T10:09:44.192967+0000","last_active":"2023-10-03T10:09:44.198209+0000","last_peered":"2023-10-03T10:09:44.198209+0000","last_clean":"2023-10-03T10:09:44.198209+0000","last_became_active":"2023-10-03T10:09:44.190979+0000","last_became_peered":"2023-10-03T10:09:44.190979+0000","last_unstale":"2023-10-03T10:09:44.198209+0000","last_undegraded":"2023-10-03T10:09:44.198209+0000","last_fullsized":"2023-10-03T10:09:44.198209+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.192360+0000","last_change":"2023-10-03T10:09:44.183121+0000","last_active":"2023-10-03T10:09:44.192360+0000","last_peered":"2023-10-03T10:09:44.192360+0000","last_clean":"2023-10-03T10:09:44.192360+0000","last_became_active":"2023-10-03T10:09:44.182647+0000","last_became_peered":"2023-10-03T10:09:44.182647+0000","last_unstale":"2023-10-03T10:09:44.192360+0000","last_undegraded":"2023-10-03T10:09:44.192360+0000","last_fullsized":"2023-10-03T10:09:44.192360+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.187765+0000","last_change":"2023-10-03T10:09:44.173123+0000","last_active":"2023-10-03T10:09:44.187765+0000","last_peered":"2023-10-03T10:09:44.187765+0000","last_clean":"2023-10-03T10:09:44.187765+0000","last_became_active":"2023-10-03T10:09:44.172353+0000","last_became_peered":"2023-10-03T10:09:44.172353+0000","last_unstale":"2023-10-03T10:09:44.187765+0000","last_undegraded":"2023-10-03T10:09:44.187765+0000","last_fullsized":"2023-10-03T10:09:44.187765+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173973+0000","last_change":"2023-10-03T10:09:44.166758+0000","last_active":"2023-10-03T10:09:44.173973+0000","last_peered":"2023-10-03T10:09:44.173973+0000","last_clean":"2023-10-03T10:09:44.173973+0000","last_became_active":"2023-10-03T10:09:44.166415+0000","last_became_peered":"2023-10-03T10:09:44.166415+0000","last_unstale":"2023-10-03T10:09:44.173973+0000","last_undegraded":"2023-10-03T10:09:44.173973+0000","last_fullsized":"2023-10-03T10:09:44.173973+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180446+0000","last_change":"2023-10-03T10:09:44.176833+0000","last_active":"2023-10-03T10:09:44.180446+0000","last_peered":"2023-10-03T10:09:44.180446+0000","last_clean":"2023-10-03T10:09:44.180446+0000","last_became_active":"2023-10-03T10:09:44.176526+0000","last_became_peered":"2023-10-03T10:09:44.176526+0000","last_unstale":"2023-10-03T10:09:44.180446+0000","last_undegraded":"2023-10-03T10:09:44.180446+0000","last_fullsized":"2023-10-03T10:09:44.180446+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180114+0000","last_change":"2023-10-03T10:09:44.175868+0000","last_active":"2023-10-03T10:09:44.180114+0000","last_peered":"2023-10-03T10:09:44.180114+0000","last_clean":"2023-10-03T10:09:44.180114+0000","last_became_active":"2023-10-03T10:09:44.175414+0000","last_became_peered":"2023-10-03T10:09:44.175414+0000","last_unstale":"2023-10-03T10:09:44.180114+0000","last_undegraded":"2023-10-03T10:09:44.180114+0000","last_fullsized":"2023-10-03T10:09:44.180114+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200283+0000","last_change":"2023-10-03T10:09:44.198581+0000","last_active":"2023-10-03T10:09:44.200283+0000","last_peered":"2023-10-03T10:09:44.200283+0000","last_clean":"2023-10-03T10:09:44.200283+0000","last_became_active":"2023-10-03T10:09:44.198314+0000","last_became_peered":"2023-10-03T10:09:44.198314+0000","last_unstale":"2023-10-03T10:09:44.200283+0000","last_undegraded":"2023-10-03T10:09:44.200283+0000","last_fullsized":"2023-10-03T10:09:44.200283+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200379+0000","last_change":"2023-10-03T10:09:44.197915+0000","last_active":"2023-10-03T10:09:44.200379+0000","last_peered":"2023-10-03T10:09:44.200379+0000","last_clean":"2023-10-03T10:09:44.200379+0000","last_became_active":"2023-10-03T10:09:44.197417+0000","last_became_peered":"2023-10-03T10:09:44.197417+0000","last_unstale":"2023-10-03T10:09:44.200379+0000","last_undegraded":"2023-10-03T10:09:44.200379+0000","last_fullsized":"2023-10-03T10:09:44.200379+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180522+0000","last_change":"2023-10-03T10:09:44.176456+0000","last_active":"2023-10-03T10:09:44.180522+0000","last_peered":"2023-10-03T10:09:44.180522+0000","last_clean":"2023-10-03T10:09:44.180522+0000","last_became_active":"2023-10-03T10:09:44.176146+0000","last_became_peered":"2023-10-03T10:09:44.176146+0000","last_unstale":"2023-10-03T10:09:44.180522+0000","last_undegraded":"2023-10-03T10:09:44.180522+0000","last_fullsized":"2023-10-03T10:09:44.180522+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.178380+0000","last_change":"2023-10-03T10:09:44.176614+0000","last_active":"2023-10-03T10:09:44.178380+0000","last_peered":"2023-10-03T10:09:44.178380+0000","last_clean":"2023-10-03T10:09:44.178380+0000","last_became_active":"2023-10-03T10:09:44.176334+0000","last_became_peered":"2023-10-03T10:09:44.176334+0000","last_unstale":"2023-10-03T10:09:44.178380+0000","last_undegraded":"2023-10-03T10:09:44.178380+0000","last_fullsized":"2023-10-03T10:09:44.178380+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.192741+0000","last_change":"2023-10-03T10:09:44.179440+0000","last_active":"2023-10-03T10:09:44.192741+0000","last_peered":"2023-10-03T10:09:44.192741+0000","last_clean":"2023-10-03T10:09:44.192741+0000","last_became_active":"2023-10-03T10:09:44.179150+0000","last_became_peered":"2023-10-03T10:09:44.179150+0000","last_unstale":"2023-10-03T10:09:44.192741+0000","last_undegraded":"2023-10-03T10:09:44.192741+0000","last_fullsized":"2023-10-03T10:09:44.192741+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173996+0000","last_change":"2023-10-03T10:09:44.166901+0000","last_active":"2023-10-03T10:09:44.173996+0000","last_peered":"2023-10-03T10:09:44.173996+0000","last_clean":"2023-10-03T10:09:44.173996+0000","last_became_active":"2023-10-03T10:09:44.166610+0000","last_became_peered":"2023-10-03T10:09:44.166610+0000","last_unstale":"2023-10-03T10:09:44.173996+0000","last_undegraded":"2023-10-03T10:09:44.173996+0000","last_fullsized":"2023-10-03T10:09:44.173996+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173643+0000","last_change":"2023-10-03T10:09:44.165366+0000","last_active":"2023-10-03T10:09:44.173643+0000","last_peered":"2023-10-03T10:09:44.173643+0000","last_clean":"2023-10-03T10:09:44.173643+0000","last_became_active":"2023-10-03T10:09:44.165083+0000","last_became_peered":"2023-10-03T10:09:44.165083+0000","last_unstale":"2023-10-03T10:09:44.173643+0000","last_undegraded":"2023-10-03T10:09:44.173643+0000","last_fullsized":"2023-10-03T10:09:44.173643+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.198990+0000","last_change":"2023-10-03T10:09:44.195696+0000","last_active":"2023-10-03T10:09:44.198990+0000","last_peered":"2023-10-03T10:09:44.198990+0000","last_clean":"2023-10-03T10:09:44.198990+0000","last_became_active":"2023-10-03T10:09:44.195402+0000","last_became_peered":"2023-10-03T10:09:44.195402+0000","last_unstale":"2023-10-03T10:09:44.198990+0000","last_undegraded":"2023-10-03T10:09:44.198990+0000","last_fullsized":"2023-10-03T10:09:44.198990+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184209+0000","last_change":"2023-10-03T10:09:44.177178+0000","last_active":"2023-10-03T10:09:44.184209+0000","last_peered":"2023-10-03T10:09:44.184209+0000","last_clean":"2023-10-03T10:09:44.184209+0000","last_became_active":"2023-10-03T10:09:44.176686+0000","last_became_peered":"2023-10-03T10:09:44.176686+0000","last_unstale":"2023-10-03T10:09:44.184209+0000","last_undegraded":"2023-10-03T10:09:44.184209+0000","last_fullsized":"2023-10-03T10:09:44.184209+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180396+0000","last_change":"2023-10-03T10:09:44.177335+0000","last_active":"2023-10-03T10:09:44.180396+0000","last_peered":"2023-10-03T10:09:44.180396+0000","last_clean":"2023-10-03T10:09:44.180396+0000","last_became_active":"2023-10-03T10:09:44.175202+0000","last_became_peered":"2023-10-03T10:09:44.175202+0000","last_unstale":"2023-10-03T10:09:44.180396+0000","last_undegraded":"2023-10-03T10:09:44.180396+0000","last_fullsized":"2023-10-03T10:09:44.180396+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184702+0000","last_change":"2023-10-03T10:09:44.182207+0000","last_active":"2023-10-03T10:09:44.184702+0000","last_peered":"2023-10-03T10:09:44.184702+0000","last_clean":"2023-10-03T10:09:44.184702+0000","last_became_active":"2023-10-03T10:09:44.181944+0000","last_became_peered":"2023-10-03T10:09:44.181944+0000","last_unstale":"2023-10-03T10:09:44.184702+0000","last_undegraded":"2023-10-03T10:09:44.184702+0000","last_fullsized":"2023-10-03T10:09:44.184702+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.192336+0000","last_change":"2023-10-03T10:09:44.182610+0000","last_active":"2023-10-03T10:09:44.192336+0000","last_peered":"2023-10-03T10:09:44.192336+0000","last_clean":"2023-10-03T10:09:44.192336+0000","last_became_active":"2023-10-03T10:09:44.180732+0000","last_became_peered":"2023-10-03T10:09:44.180732+0000","last_unstale":"2023-10-03T10:09:44.192336+0000","last_undegraded":"2023-10-03T10:09:44.192336+0000","last_fullsized":"2023-10-03T10:09:44.192336+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.187788+0000","last_change":"2023-10-03T10:09:44.173182+0000","last_active":"2023-10-03T10:09:44.187788+0000","last_peered":"2023-10-03T10:09:44.187788+0000","last_clean":"2023-10-03T10:09:44.187788+0000","last_became_active":"2023-10-03T10:09:44.172685+0000","last_became_peered":"2023-10-03T10:09:44.172685+0000","last_unstale":"2023-10-03T10:09:44.187788+0000","last_undegraded":"2023-10-03T10:09:44.187788+0000","last_fullsized":"2023-10-03T10:09:44.187788+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180761+0000","last_change":"2023-10-03T10:09:44.177910+0000","last_active":"2023-10-03T10:09:44.180761+0000","last_peered":"2023-10-03T10:09:44.180761+0000","last_clean":"2023-10-03T10:09:44.180761+0000","last_became_active":"2023-10-03T10:09:44.177618+0000","last_became_peered":"2023-10-03T10:09:44.177618+0000","last_unstale":"2023-10-03T10:09:44.180761+0000","last_undegraded":"2023-10-03T10:09:44.180761+0000","last_fullsized":"2023-10-03T10:09:44.180761+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180424+0000","last_change":"2023-10-03T10:09:44.176287+0000","last_active":"2023-10-03T10:09:44.180424+0000","last_peered":"2023-10-03T10:09:44.180424+0000","last_clean":"2023-10-03T10:09:44.180424+0000","last_became_active":"2023-10-03T10:09:44.175320+0000","last_became_peered":"2023-10-03T10:09:44.175320+0000","last_unstale":"2023-10-03T10:09:44.180424+0000","last_undegraded":"2023-10-03T10:09:44.180424+0000","last_fullsized":"2023-10-03T10:09:44.180424+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.194745+0000","last_change":"2023-10-03T10:09:44.187148+0000","last_active":"2023-10-03T10:09:44.194745+0000","last_peered":"2023-10-03T10:09:44.194745+0000","last_clean":"2023-10-03T10:09:44.194745+0000","last_became_active":"2023-10-03T10:09:44.186523+0000","last_became_peered":"2023-10-03T10:09:44.186523+0000","last_unstale":"2023-10-03T10:09:44.194745+0000","last_undegraded":"2023-10-03T10:09:44.194745+0000","last_fullsized":"2023-10-03T10:09:44.194745+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.189715+0000","last_change":"2023-10-03T10:09:43.745046+0000","last_active":"2023-10-03T10:09:44.189715+0000","last_peered":"2023-10-03T10:09:44.189715+0000","last_clean":"2023-10-03T10:09:44.189715+0000","last_became_active":"2023-10-03T10:09:43.744724+0000","last_became_peered":"2023-10-03T10:09:43.744724+0000","last_unstale":"2023-10-03T10:09:44.189715+0000","last_undegraded":"2023-10-03T10:09:44.189715+0000","last_fullsized":"2023-10-03T10:09:44.189715+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195809+0000","last_change":"2023-10-03T10:09:44.190869+0000","last_active":"2023-10-03T10:09:44.195809+0000","last_peered":"2023-10-03T10:09:44.195809+0000","last_clean":"2023-10-03T10:09:44.195809+0000","last_became_active":"2023-10-03T10:09:44.190207+0000","last_became_peered":"2023-10-03T10:09:44.190207+0000","last_unstale":"2023-10-03T10:09:44.195809+0000","last_undegraded":"2023-10-03T10:09:44.195809+0000","last_fullsized":"2023-10-03T10:09:44.195809+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.152860+0000","last_change":"2023-10-03T10:09:43.750891+0000","last_active":"2023-10-03T10:09:44.152860+0000","last_peered":"2023-10-03T10:09:44.152860+0000","last_clean":"2023-10-03T10:09:44.152860+0000","last_became_active":"2023-10-03T10:09:43.750356+0000","last_became_peered":"2023-10-03T10:09:43.750356+0000","last_unstale":"2023-10-03T10:09:44.152860+0000","last_undegraded":"2023-10-03T10:09:44.152860+0000","last_fullsized":"2023-10-03T10:09:44.152860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184645+0000","last_change":"2023-10-03T10:09:44.183145+0000","last_active":"2023-10-03T10:09:44.184645+0000","last_peered":"2023-10-03T10:09:44.184645+0000","last_clean":"2023-10-03T10:09:44.184645+0000","last_became_active":"2023-10-03T10:09:44.182816+0000","last_became_peered":"2023-10-03T10:09:44.182816+0000","last_unstale":"2023-10-03T10:09:44.184645+0000","last_undegraded":"2023-10-03T10:09:44.184645+0000","last_fullsized":"2023-10-03T10:09:44.184645+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.163680+0000","last_change":"2023-10-03T10:09:43.749289+0000","last_active":"2023-10-03T10:09:44.163680+0000","last_peered":"2023-10-03T10:09:44.163680+0000","last_clean":"2023-10-03T10:09:44.163680+0000","last_became_active":"2023-10-03T10:09:43.748981+0000","last_became_peered":"2023-10-03T10:09:43.748981+0000","last_unstale":"2023-10-03T10:09:44.163680+0000","last_undegraded":"2023-10-03T10:09:44.163680+0000","last_fullsized":"2023-10-03T10:09:44.163680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.197002+0000","last_change":"2023-10-03T10:09:44.192692+0000","last_active":"2023-10-03T10:09:44.197002+0000","last_peered":"2023-10-03T10:09:44.197002+0000","last_clean":"2023-10-03T10:09:44.197002+0000","last_became_active":"2023-10-03T10:09:44.192390+0000","last_became_peered":"2023-10-03T10:09:44.192390+0000","last_unstale":"2023-10-03T10:09:44.197002+0000","last_undegraded":"2023-10-03T10:09:44.197002+0000","last_fullsized":"2023-10-03T10:09:44.197002+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.160877+0000","last_change":"2023-10-03T10:09:43.754802+0000","last_active":"2023-10-03T10:09:44.160877+0000","last_peered":"2023-10-03T10:09:44.160877+0000","last_clean":"2023-10-03T10:09:44.160877+0000","last_became_active":"2023-10-03T10:09:43.754512+0000","last_became_peered":"2023-10-03T10:09:43.754512+0000","last_unstale":"2023-10-03T10:09:44.160877+0000","last_undegraded":"2023-10-03T10:09:44.160877+0000","last_fullsized":"2023-10-03T10:09:44.160877+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196969+0000","last_change":"2023-10-03T10:09:44.192798+0000","last_active":"2023-10-03T10:09:44.196969+0000","last_peered":"2023-10-03T10:09:44.196969+0000","last_clean":"2023-10-03T10:09:44.196969+0000","last_became_active":"2023-10-03T10:09:44.190787+0000","last_became_peered":"2023-10-03T10:09:44.190787+0000","last_unstale":"2023-10-03T10:09:44.196969+0000","last_undegraded":"2023-10-03T10:09:44.196969+0000","last_fullsized":"2023-10-03T10:09:44.196969+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.157966+0000","last_change":"2023-10-03T10:09:43.756958+0000","last_active":"2023-10-03T10:09:44.157966+0000","last_peered":"2023-10-03T10:09:44.157966+0000","last_clean":"2023-10-03T10:09:44.157966+0000","last_became_active":"2023-10-03T10:09:43.756463+0000","last_became_peered":"2023-10-03T10:09:43.756463+0000","last_unstale":"2023-10-03T10:09:44.157966+0000","last_undegraded":"2023-10-03T10:09:44.157966+0000","last_fullsized":"2023-10-03T10:09:44.157966+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.194551+0000","last_change":"2023-10-03T10:09:44.184790+0000","last_active":"2023-10-03T10:09:44.194551+0000","last_peered":"2023-10-03T10:09:44.194551+0000","last_clean":"2023-10-03T10:09:44.194551+0000","last_became_active":"2023-10-03T10:09:44.184531+0000","last_became_peered":"2023-10-03T10:09:44.184531+0000","last_unstale":"2023-10-03T10:09:44.194551+0000","last_undegraded":"2023-10-03T10:09:44.194551+0000","last_fullsized":"2023-10-03T10:09:44.194551+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"16'3","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200426+0000","last_change":"2023-10-03T10:09:43.756598+0000","last_active":"2023-10-03T10:09:44.200426+0000","last_peered":"2023-10-03T10:09:44.200426+0000","last_clean":"2023-10-03T10:09:44.200426+0000","last_became_active":"2023-10-03T10:09:43.754974+0000","last_became_peered":"2023-10-03T10:09:43.754974+0000","last_unstale":"2023-10-03T10:09:44.200426+0000","last_undegraded":"2023-10-03T10:09:44.200426+0000","last_fullsized":"2023-10-03T10:09:44.200426+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180499+0000","last_change":"2023-10-03T10:09:44.175504+0000","last_active":"2023-10-03T10:09:44.180499+0000","last_peered":"2023-10-03T10:09:44.180499+0000","last_clean":"2023-10-03T10:09:44.180499+0000","last_became_active":"2023-10-03T10:09:44.175174+0000","last_became_peered":"2023-10-03T10:09:44.175174+0000","last_unstale":"2023-10-03T10:09:44.180499+0000","last_undegraded":"2023-10-03T10:09:44.180499+0000","last_fullsized":"2023-10-03T10:09:44.180499+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"16'6","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.203073+0000","last_change":"2023-10-03T10:09:43.758711+0000","last_active":"2023-10-03T10:09:44.203073+0000","last_peered":"2023-10-03T10:09:44.203073+0000","last_clean":"2023-10-03T10:09:44.203073+0000","last_became_active":"2023-10-03T10:09:43.758392+0000","last_became_peered":"2023-10-03T10:09:43.758392+0000","last_unstale":"2023-10-03T10:09:44.203073+0000","last_undegraded":"2023-10-03T10:09:44.203073+0000","last_fullsized":"2023-10-03T10:09:44.203073+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200336+0000","last_change":"2023-10-03T10:09:44.196559+0000","last_active":"2023-10-03T10:09:44.200336+0000","last_peered":"2023-10-03T10:09:44.200336+0000","last_clean":"2023-10-03T10:09:44.200336+0000","last_became_active":"2023-10-03T10:09:44.196280+0000","last_became_peered":"2023-10-03T10:09:44.196280+0000","last_unstale":"2023-10-03T10:09:44.200336+0000","last_undegraded":"2023-10-03T10:09:44.200336+0000","last_fullsized":"2023-10-03T10:09:44.200336+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200762+0000","last_change":"2023-10-03T10:09:43.751877+0000","last_active":"2023-10-03T10:09:44.200762+0000","last_peered":"2023-10-03T10:09:44.200762+0000","last_clean":"2023-10-03T10:09:44.200762+0000","last_became_active":"2023-10-03T10:09:43.749870+0000","last_became_peered":"2023-10-03T10:09:43.749870+0000","last_unstale":"2023-10-03T10:09:44.200762+0000","last_undegraded":"2023-10-03T10:09:44.200762+0000","last_fullsized":"2023-10-03T10:09:44.200762+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195297+0000","last_change":"2023-10-03T10:09:44.190230+0000","last_active":"2023-10-03T10:09:44.195297+0000","last_peered":"2023-10-03T10:09:44.195297+0000","last_clean":"2023-10-03T10:09:44.195297+0000","last_became_active":"2023-10-03T10:09:44.189342+0000","last_became_peered":"2023-10-03T10:09:44.189342+0000","last_unstale":"2023-10-03T10:09:44.195297+0000","last_undegraded":"2023-10-03T10:09:44.195297+0000","last_fullsized":"2023-10-03T10:09:44.195297+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.162475+0000","last_change":"2023-10-03T10:09:43.755271+0000","last_active":"2023-10-03T10:09:44.162475+0000","last_peered":"2023-10-03T10:09:44.162475+0000","last_clean":"2023-10-03T10:09:44.162475+0000","last_became_active":"2023-10-03T10:09:43.754957+0000","last_became_peered":"2023-10-03T10:09:43.754957+0000","last_unstale":"2023-10-03T10:09:44.162475+0000","last_undegraded":"2023-10-03T10:09:44.162475+0000","last_fullsized":"2023-10-03T10:09:44.162475+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.194802+0000","last_change":"2023-10-03T10:09:44.187333+0000","last_active":"2023-10-03T10:09:44.194802+0000","last_peered":"2023-10-03T10:09:44.194802+0000","last_clean":"2023-10-03T10:09:44.194802+0000","last_became_active":"2023-10-03T10:09:44.186824+0000","last_became_peered":"2023-10-03T10:09:44.186824+0000","last_unstale":"2023-10-03T10:09:44.194802+0000","last_undegraded":"2023-10-03T10:09:44.194802+0000","last_fullsized":"2023-10-03T10:09:44.194802+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.163844+0000","last_change":"2023-10-03T10:09:43.758995+0000","last_active":"2023-10-03T10:09:44.163844+0000","last_peered":"2023-10-03T10:09:44.163844+0000","last_clean":"2023-10-03T10:09:44.163844+0000","last_became_active":"2023-10-03T10:09:43.758565+0000","last_became_peered":"2023-10-03T10:09:43.758565+0000","last_unstale":"2023-10-03T10:09:44.163844+0000","last_undegraded":"2023-10-03T10:09:44.163844+0000","last_fullsized":"2023-10-03T10:09:44.163844+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173655+0000","last_change":"2023-10-03T10:09:44.166557+0000","last_active":"2023-10-03T10:09:44.173655+0000","last_peered":"2023-10-03T10:09:44.173655+0000","last_clean":"2023-10-03T10:09:44.173655+0000","last_became_active":"2023-10-03T10:09:44.166300+0000","last_became_peered":"2023-10-03T10:09:44.166300+0000","last_unstale":"2023-10-03T10:09:44.173655+0000","last_undegraded":"2023-10-03T10:09:44.173655+0000","last_fullsized":"2023-10-03T10:09:44.173655+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"16'8","reported_seq":18,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.194737+0000","last_change":"2023-10-03T10:09:43.748084+0000","last_active":"2023-10-03T10:09:44.194737+0000","last_peered":"2023-10-03T10:09:44.194737+0000","last_clean":"2023-10-03T10:09:44.194737+0000","last_became_active":"2023-10-03T10:09:43.747377+0000","last_became_peered":"2023-10-03T10:09:43.747377+0000","last_unstale":"2023-10-03T10:09:44.194737+0000","last_undegraded":"2023-10-03T10:09:44.194737+0000","last_fullsized":"2023-10-03T10:09:44.194737+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195326+0000","last_change":"2023-10-03T10:09:44.190767+0000","last_active":"2023-10-03T10:09:44.195326+0000","last_peered":"2023-10-03T10:09:44.195326+0000","last_clean":"2023-10-03T10:09:44.195326+0000","last_became_active":"2023-10-03T10:09:44.190514+0000","last_became_peered":"2023-10-03T10:09:44.190514+0000","last_unstale":"2023-10-03T10:09:44.195326+0000","last_undegraded":"2023-10-03T10:09:44.195326+0000","last_fullsized":"2023-10-03T10:09:44.195326+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.183000+0000","last_change":"2023-10-03T10:09:43.748461+0000","last_active":"2023-10-03T10:09:44.183000+0000","last_peered":"2023-10-03T10:09:44.183000+0000","last_clean":"2023-10-03T10:09:44.183000+0000","last_became_active":"2023-10-03T10:09:43.747372+0000","last_became_peered":"2023-10-03T10:09:43.747372+0000","last_unstale":"2023-10-03T10:09:44.183000+0000","last_undegraded":"2023-10-03T10:09:44.183000+0000","last_fullsized":"2023-10-03T10:09:44.183000+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196810+0000","last_change":"2023-10-03T10:09:44.192036+0000","last_active":"2023-10-03T10:09:44.196810+0000","last_peered":"2023-10-03T10:09:44.196810+0000","last_clean":"2023-10-03T10:09:44.196810+0000","last_became_active":"2023-10-03T10:09:44.191093+0000","last_became_peered":"2023-10-03T10:09:44.191093+0000","last_unstale":"2023-10-03T10:09:44.196810+0000","last_undegraded":"2023-10-03T10:09:44.196810+0000","last_fullsized":"2023-10-03T10:09:44.196810+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.174713+0000","last_change":"2023-10-03T10:09:43.756751+0000","last_active":"2023-10-03T10:09:44.174713+0000","last_peered":"2023-10-03T10:09:44.174713+0000","last_clean":"2023-10-03T10:09:44.174713+0000","last_became_active":"2023-10-03T10:09:43.756079+0000","last_became_peered":"2023-10-03T10:09:43.756079+0000","last_unstale":"2023-10-03T10:09:44.174713+0000","last_undegraded":"2023-10-03T10:09:44.174713+0000","last_fullsized":"2023-10-03T10:09:44.174713+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.179874+0000","last_change":"2023-10-03T10:09:44.172400+0000","last_active":"2023-10-03T10:09:44.179874+0000","last_peered":"2023-10-03T10:09:44.179874+0000","last_clean":"2023-10-03T10:09:44.179874+0000","last_became_active":"2023-10-03T10:09:44.172095+0000","last_became_peered":"2023-10-03T10:09:44.172095+0000","last_unstale":"2023-10-03T10:09:44.179874+0000","last_undegraded":"2023-10-03T10:09:44.179874+0000","last_fullsized":"2023-10-03T10:09:44.179874+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195109+0000","last_change":"2023-10-03T10:09:43.760622+0000","last_active":"2023-10-03T10:09:44.195109+0000","last_peered":"2023-10-03T10:09:44.195109+0000","last_clean":"2023-10-03T10:09:44.195109+0000","last_became_active":"2023-10-03T10:09:43.760371+0000","last_became_peered":"2023-10-03T10:09:43.760371+0000","last_unstale":"2023-10-03T10:09:44.195109+0000","last_undegraded":"2023-10-03T10:09:44.195109+0000","last_fullsized":"2023-10-03T10:09:44.195109+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200471+0000","last_change":"2023-10-03T10:09:44.196062+0000","last_active":"2023-10-03T10:09:44.200471+0000","last_peered":"2023-10-03T10:09:44.200471+0000","last_clean":"2023-10-03T10:09:44.200471+0000","last_became_active":"2023-10-03T10:09:44.195841+0000","last_became_peered":"2023-10-03T10:09:44.195841+0000","last_unstale":"2023-10-03T10:09:44.200471+0000","last_undegraded":"2023-10-03T10:09:44.200471+0000","last_fullsized":"2023-10-03T10:09:44.200471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.157166+0000","last_change":"2023-10-03T10:09:43.756885+0000","last_active":"2023-10-03T10:09:44.157166+0000","last_peered":"2023-10-03T10:09:44.157166+0000","last_clean":"2023-10-03T10:09:44.157166+0000","last_became_active":"2023-10-03T10:09:43.756588+0000","last_became_peered":"2023-10-03T10:09:43.756588+0000","last_unstale":"2023-10-03T10:09:44.157166+0000","last_undegraded":"2023-10-03T10:09:44.157166+0000","last_fullsized":"2023-10-03T10:09:44.157166+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184514+0000","last_change":"2023-10-03T10:09:44.181603+0000","last_active":"2023-10-03T10:09:44.184514+0000","last_peered":"2023-10-03T10:09:44.184514+0000","last_clean":"2023-10-03T10:09:44.184514+0000","last_became_active":"2023-10-03T10:09:44.181125+0000","last_became_peered":"2023-10-03T10:09:44.181125+0000","last_unstale":"2023-10-03T10:09:44.184514+0000","last_undegraded":"2023-10-03T10:09:44.184514+0000","last_fullsized":"2023-10-03T10:09:44.184514+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200822+0000","last_change":"2023-10-03T10:09:43.751873+0000","last_active":"2023-10-03T10:09:44.200822+0000","last_peered":"2023-10-03T10:09:44.200822+0000","last_clean":"2023-10-03T10:09:44.200822+0000","last_became_active":"2023-10-03T10:09:43.751324+0000","last_became_peered":"2023-10-03T10:09:43.751324+0000","last_unstale":"2023-10-03T10:09:44.200822+0000","last_undegraded":"2023-10-03T10:09:44.200822+0000","last_fullsized":"2023-10-03T10:09:44.200822+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.174302+0000","last_change":"2023-10-03T10:09:44.170676+0000","last_active":"2023-10-03T10:09:44.174302+0000","last_peered":"2023-10-03T10:09:44.174302+0000","last_clean":"2023-10-03T10:09:44.174302+0000","last_became_active":"2023-10-03T10:09:44.170398+0000","last_became_peered":"2023-10-03T10:09:44.170398+0000","last_unstale":"2023-10-03T10:09:44.174302+0000","last_undegraded":"2023-10-03T10:09:44.174302+0000","last_fullsized":"2023-10-03T10:09:44.174302+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.166137+0000","last_change":"2023-10-03T10:09:43.760711+0000","last_active":"2023-10-03T10:09:44.166137+0000","last_peered":"2023-10-03T10:09:44.166137+0000","last_clean":"2023-10-03T10:09:44.166137+0000","last_became_active":"2023-10-03T10:09:43.760432+0000","last_became_peered":"2023-10-03T10:09:43.760432+0000","last_unstale":"2023-10-03T10:09:44.166137+0000","last_undegraded":"2023-10-03T10:09:44.166137+0000","last_fullsized":"2023-10-03T10:09:44.166137+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195829+0000","last_change":"2023-10-03T10:09:44.191073+0000","last_active":"2023-10-03T10:09:44.195829+0000","last_peered":"2023-10-03T10:09:44.195829+0000","last_clean":"2023-10-03T10:09:44.195829+0000","last_became_active":"2023-10-03T10:09:44.190218+0000","last_became_peered":"2023-10-03T10:09:44.190218+0000","last_unstale":"2023-10-03T10:09:44.195829+0000","last_undegraded":"2023-10-03T10:09:44.195829+0000","last_fullsized":"2023-10-03T10:09:44.195829+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.163521+0000","last_change":"2023-10-03T10:09:43.758846+0000","last_active":"2023-10-03T10:09:44.163521+0000","last_peered":"2023-10-03T10:09:44.163521+0000","last_clean":"2023-10-03T10:09:44.163521+0000","last_became_active":"2023-10-03T10:09:43.757973+0000","last_became_peered":"2023-10-03T10:09:43.757973+0000","last_unstale":"2023-10-03T10:09:44.163521+0000","last_undegraded":"2023-10-03T10:09:44.163521+0000","last_fullsized":"2023-10-03T10:09:44.163521+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200102+0000","last_change":"2023-10-03T10:09:44.196056+0000","last_active":"2023-10-03T10:09:44.200102+0000","last_peered":"2023-10-03T10:09:44.200102+0000","last_clean":"2023-10-03T10:09:44.200102+0000","last_became_active":"2023-10-03T10:09:44.195817+0000","last_became_peered":"2023-10-03T10:09:44.195817+0000","last_unstale":"2023-10-03T10:09:44.200102+0000","last_undegraded":"2023-10-03T10:09:44.200102+0000","last_fullsized":"2023-10-03T10:09:44.200102+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.162993+0000","last_change":"2023-10-03T10:09:43.761068+0000","last_active":"2023-10-03T10:09:44.162993+0000","last_peered":"2023-10-03T10:09:44.162993+0000","last_clean":"2023-10-03T10:09:44.162993+0000","last_became_active":"2023-10-03T10:09:43.760825+0000","last_became_peered":"2023-10-03T10:09:43.760825+0000","last_unstale":"2023-10-03T10:09:44.162993+0000","last_undegraded":"2023-10-03T10:09:44.162993+0000","last_fullsized":"2023-10-03T10:09:44.162993+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180799+0000","last_change":"2023-10-03T10:09:44.178856+0000","last_active":"2023-10-03T10:09:44.180799+0000","last_peered":"2023-10-03T10:09:44.180799+0000","last_clean":"2023-10-03T10:09:44.180799+0000","last_became_active":"2023-10-03T10:09:44.178410+0000","last_became_peered":"2023-10-03T10:09:44.178410+0000","last_unstale":"2023-10-03T10:09:44.180799+0000","last_undegraded":"2023-10-03T10:09:44.180799+0000","last_fullsized":"2023-10-03T10:09:44.180799+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.154127+0000","last_change":"2023-10-03T10:09:39.589080+0000","last_active":"2023-10-03T10:09:44.154127+0000","last_peered":"2023-10-03T10:09:44.154127+0000","last_clean":"2023-10-03T10:09:44.154127+0000","last_became_active":"2023-10-03T10:09:39.588651+0000","last_became_peered":"2023-10-03T10:09:39.588651+0000","last_unstale":"2023-10-03T10:09:44.154127+0000","last_undegraded":"2023-10-03T10:09:44.154127+0000","last_fullsized":"2023-10-03T10:09:44.154127+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.157236+0000","last_change":"2023-10-03T10:09:43.756053+0000","last_active":"2023-10-03T10:09:44.157236+0000","last_peered":"2023-10-03T10:09:44.157236+0000","last_clean":"2023-10-03T10:09:44.157236+0000","last_became_active":"2023-10-03T10:09:43.755708+0000","last_became_peered":"2023-10-03T10:09:43.755708+0000","last_unstale":"2023-10-03T10:09:44.157236+0000","last_undegraded":"2023-10-03T10:09:44.157236+0000","last_fullsized":"2023-10-03T10:09:44.157236+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180543+0000","last_change":"2023-10-03T10:09:44.176964+0000","last_active":"2023-10-03T10:09:44.180543+0000","last_peered":"2023-10-03T10:09:44.180543+0000","last_clean":"2023-10-03T10:09:44.180543+0000","last_became_active":"2023-10-03T10:09:44.176671+0000","last_became_peered":"2023-10-03T10:09:44.176671+0000","last_unstale":"2023-10-03T10:09:44.180543+0000","last_undegraded":"2023-10-03T10:09:44.180543+0000","last_fullsized":"2023-10-03T10:09:44.180543+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.166922+0000","last_change":"2023-10-03T10:09:39.586735+0000","last_active":"2023-10-03T10:09:44.166922+0000","last_peered":"2023-10-03T10:09:44.166922+0000","last_clean":"2023-10-03T10:09:44.166922+0000","last_became_active":"2023-10-03T10:09:39.586440+0000","last_became_peered":"2023-10-03T10:09:39.586440+0000","last_unstale":"2023-10-03T10:09:44.166922+0000","last_undegraded":"2023-10-03T10:09:44.166922+0000","last_fullsized":"2023-10-03T10:09:44.166922+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.159439+0000","last_change":"2023-10-03T10:09:43.748458+0000","last_active":"2023-10-03T10:09:44.159439+0000","last_peered":"2023-10-03T10:09:44.159439+0000","last_clean":"2023-10-03T10:09:44.159439+0000","last_became_active":"2023-10-03T10:09:43.747721+0000","last_became_peered":"2023-10-03T10:09:43.747721+0000","last_unstale":"2023-10-03T10:09:44.159439+0000","last_undegraded":"2023-10-03T10:09:44.159439+0000","last_fullsized":"2023-10-03T10:09:44.159439+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184166+0000","last_change":"2023-10-03T10:09:44.176165+0000","last_active":"2023-10-03T10:09:44.184166+0000","last_peered":"2023-10-03T10:09:44.184166+0000","last_clean":"2023-10-03T10:09:44.184166+0000","last_became_active":"2023-10-03T10:09:44.175853+0000","last_became_peered":"2023-10-03T10:09:44.175853+0000","last_unstale":"2023-10-03T10:09:44.184166+0000","last_undegraded":"2023-10-03T10:09:44.184166+0000","last_fullsized":"2023-10-03T10:09:44.184166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.156829+0000","last_change":"2023-10-03T10:09:39.590114+0000","last_active":"2023-10-03T10:09:44.156829+0000","last_peered":"2023-10-03T10:09:44.156829+0000","last_clean":"2023-10-03T10:09:44.156829+0000","last_became_active":"2023-10-03T10:09:39.589663+0000","last_became_peered":"2023-10-03T10:09:39.589663+0000","last_unstale":"2023-10-03T10:09:44.156829+0000","last_undegraded":"2023-10-03T10:09:44.156829+0000","last_fullsized":"2023-10-03T10:09:44.156829+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.155654+0000","last_change":"2023-10-03T10:09:43.748267+0000","last_active":"2023-10-03T10:09:44.155654+0000","last_peered":"2023-10-03T10:09:44.155654+0000","last_clean":"2023-10-03T10:09:44.155654+0000","last_became_active":"2023-10-03T10:09:43.747789+0000","last_became_peered":"2023-10-03T10:09:43.747789+0000","last_unstale":"2023-10-03T10:09:44.155654+0000","last_undegraded":"2023-10-03T10:09:44.155654+0000","last_fullsized":"2023-10-03T10:09:44.155654+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180773+0000","last_change":"2023-10-03T10:09:44.178680+0000","last_active":"2023-10-03T10:09:44.180773+0000","last_peered":"2023-10-03T10:09:44.180773+0000","last_clean":"2023-10-03T10:09:44.180773+0000","last_became_active":"2023-10-03T10:09:44.178114+0000","last_became_peered":"2023-10-03T10:09:44.178114+0000","last_unstale":"2023-10-03T10:09:44.180773+0000","last_undegraded":"2023-10-03T10:09:44.180773+0000","last_fullsized":"2023-10-03T10:09:44.180773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.167768+0000","last_change":"2023-10-03T10:09:39.587728+0000","last_active":"2023-10-03T10:09:44.167768+0000","last_peered":"2023-10-03T10:09:44.167768+0000","last_clean":"2023-10-03T10:09:44.167768+0000","last_became_active":"2023-10-03T10:09:39.587441+0000","last_became_peered":"2023-10-03T10:09:39.587441+0000","last_unstale":"2023-10-03T10:09:44.167768+0000","last_undegraded":"2023-10-03T10:09:44.167768+0000","last_fullsized":"2023-10-03T10:09:44.167768+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.153191+0000","last_change":"2023-10-03T10:09:38.585668+0000","last_active":"2023-10-03T10:09:44.153191+0000","last_peered":"2023-10-03T10:09:44.153191+0000","last_clean":"2023-10-03T10:09:44.153191+0000","last_became_active":"2023-10-03T10:09:38.585324+0000","last_became_peered":"2023-10-03T10:09:38.585324+0000","last_unstale":"2023-10-03T10:09:44.153191+0000","last_undegraded":"2023-10-03T10:09:44.153191+0000","last_fullsized":"2023-10-03T10:09:44.153191+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:37.570571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:37.570571+0000","last_clean_scrub_stamp":"2023-10-03T10:09:37.570571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.165999+0000","last_change":"2023-10-03T10:09:43.758096+0000","last_active":"2023-10-03T10:09:44.165999+0000","last_peered":"2023-10-03T10:09:44.165999+0000","last_clean":"2023-10-03T10:09:44.165999+0000","last_became_active":"2023-10-03T10:09:43.757689+0000","last_became_peered":"2023-10-03T10:09:43.757689+0000","last_unstale":"2023-10-03T10:09:44.165999+0000","last_undegraded":"2023-10-03T10:09:44.165999+0000","last_fullsized":"2023-10-03T10:09:44.165999+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196949+0000","last_change":"2023-10-03T10:09:44.192781+0000","last_active":"2023-10-03T10:09:44.196949+0000","last_peered":"2023-10-03T10:09:44.196949+0000","last_clean":"2023-10-03T10:09:44.196949+0000","last_became_active":"2023-10-03T10:09:44.190597+0000","last_became_peered":"2023-10-03T10:09:44.190597+0000","last_unstale":"2023-10-03T10:09:44.196949+0000","last_undegraded":"2023-10-03T10:09:44.196949+0000","last_fullsized":"2023-10-03T10:09:44.196949+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.163365+0000","last_change":"2023-10-03T10:09:39.586806+0000","last_active":"2023-10-03T10:09:44.163365+0000","last_peered":"2023-10-03T10:09:44.163365+0000","last_clean":"2023-10-03T10:09:44.163365+0000","last_became_active":"2023-10-03T10:09:39.586481+0000","last_became_peered":"2023-10-03T10:09:39.586481+0000","last_unstale":"2023-10-03T10:09:44.163365+0000","last_undegraded":"2023-10-03T10:09:44.163365+0000","last_fullsized":"2023-10-03T10:09:44.163365+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.156596+0000","last_change":"2023-10-03T10:09:43.747783+0000","last_active":"2023-10-03T10:09:44.156596+0000","last_peered":"2023-10-03T10:09:44.156596+0000","last_clean":"2023-10-03T10:09:44.156596+0000","last_became_active":"2023-10-03T10:09:43.746009+0000","last_became_peered":"2023-10-03T10:09:43.746009+0000","last_unstale":"2023-10-03T10:09:44.156596+0000","last_undegraded":"2023-10-03T10:09:44.156596+0000","last_fullsized":"2023-10-03T10:09:44.156596+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184590+0000","last_change":"2023-10-03T10:09:44.181359+0000","last_active":"2023-10-03T10:09:44.184590+0000","last_peered":"2023-10-03T10:09:44.184590+0000","last_clean":"2023-10-03T10:09:44.184590+0000","last_became_active":"2023-10-03T10:09:44.181075+0000","last_became_peered":"2023-10-03T10:09:44.181075+0000","last_unstale":"2023-10-03T10:09:44.184590+0000","last_undegraded":"2023-10-03T10:09:44.184590+0000","last_fullsized":"2023-10-03T10:09:44.184590+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.168332+0000","last_change":"2023-10-03T10:09:39.588827+0000","last_active":"2023-10-03T10:09:44.168332+0000","last_peered":"2023-10-03T10:09:44.168332+0000","last_clean":"2023-10-03T10:09:44.168332+0000","last_became_active":"2023-10-03T10:09:39.588531+0000","last_became_peered":"2023-10-03T10:09:39.588531+0000","last_unstale":"2023-10-03T10:09:44.168332+0000","last_undegraded":"2023-10-03T10:09:44.168332+0000","last_fullsized":"2023-10-03T10:09:44.168332+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.152966+0000","last_change":"2023-10-03T10:09:43.746339+0000","last_active":"2023-10-03T10:09:44.152966+0000","last_peered":"2023-10-03T10:09:44.152966+0000","last_clean":"2023-10-03T10:09:44.152966+0000","last_became_active":"2023-10-03T10:09:43.746051+0000","last_became_peered":"2023-10-03T10:09:43.746051+0000","last_unstale":"2023-10-03T10:09:44.152966+0000","last_undegraded":"2023-10-03T10:09:44.152966+0000","last_fullsized":"2023-10-03T10:09:44.152966+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173149+0000","last_change":"2023-10-03T10:09:44.165684+0000","last_active":"2023-10-03T10:09:44.173149+0000","last_peered":"2023-10-03T10:09:44.173149+0000","last_clean":"2023-10-03T10:09:44.173149+0000","last_became_active":"2023-10-03T10:09:44.165422+0000","last_became_peered":"2023-10-03T10:09:44.165422+0000","last_unstale":"2023-10-03T10:09:44.173149+0000","last_undegraded":"2023-10-03T10:09:44.173149+0000","last_fullsized":"2023-10-03T10:09:44.173149+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.197330+0000","last_change":"2023-10-03T10:09:43.760807+0000","last_active":"2023-10-03T10:09:44.197330+0000","last_peered":"2023-10-03T10:09:44.197330+0000","last_clean":"2023-10-03T10:09:44.197330+0000","last_became_active":"2023-10-03T10:09:43.760517+0000","last_became_peered":"2023-10-03T10:09:43.760517+0000","last_unstale":"2023-10-03T10:09:44.197330+0000","last_undegraded":"2023-10-03T10:09:44.197330+0000","last_fullsized":"2023-10-03T10:09:44.197330+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.153815+0000","last_change":"2023-10-03T10:09:39.586702+0000","last_active":"2023-10-03T10:09:44.153815+0000","last_peered":"2023-10-03T10:09:44.153815+0000","last_clean":"2023-10-03T10:09:44.153815+0000","last_became_active":"2023-10-03T10:09:39.586153+0000","last_became_peered":"2023-10-03T10:09:39.586153+0000","last_unstale":"2023-10-03T10:09:44.153815+0000","last_undegraded":"2023-10-03T10:09:44.153815+0000","last_fullsized":"2023-10-03T10:09:44.153815+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196883+0000","last_change":"2023-10-03T10:09:44.192664+0000","last_active":"2023-10-03T10:09:44.196883+0000","last_peered":"2023-10-03T10:09:44.196883+0000","last_clean":"2023-10-03T10:09:44.196883+0000","last_became_active":"2023-10-03T10:09:44.190513+0000","last_became_peered":"2023-10-03T10:09:44.190513+0000","last_unstale":"2023-10-03T10:09:44.196883+0000","last_undegraded":"2023-10-03T10:09:44.196883+0000","last_fullsized":"2023-10-03T10:09:44.196883+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.167083+0000","last_change":"2023-10-03T10:09:39.588950+0000","last_active":"2023-10-03T10:09:44.167083+0000","last_peered":"2023-10-03T10:09:44.167083+0000","last_clean":"2023-10-03T10:09:44.167083+0000","last_became_active":"2023-10-03T10:09:39.588601+0000","last_became_peered":"2023-10-03T10:09:39.588601+0000","last_unstale":"2023-10-03T10:09:44.167083+0000","last_undegraded":"2023-10-03T10:09:44.167083+0000","last_fullsized":"2023-10-03T10:09:44.167083+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.160538+0000","last_change":"2023-10-03T10:09:43.752793+0000","last_active":"2023-10-03T10:09:44.160538+0000","last_peered":"2023-10-03T10:09:44.160538+0000","last_clean":"2023-10-03T10:09:44.160538+0000","last_became_active":"2023-10-03T10:09:43.752374+0000","last_became_peered":"2023-10-03T10:09:43.752374+0000","last_unstale":"2023-10-03T10:09:44.160538+0000","last_undegraded":"2023-10-03T10:09:44.160538+0000","last_fullsized":"2023-10-03T10:09:44.160538+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200171+0000","last_change":"2023-10-03T10:09:44.196314+0000","last_active":"2023-10-03T10:09:44.200171+0000","last_peered":"2023-10-03T10:09:44.200171+0000","last_clean":"2023-10-03T10:09:44.200171+0000","last_became_active":"2023-10-03T10:09:44.195901+0000","last_became_peered":"2023-10-03T10:09:44.195901+0000","last_unstale":"2023-10-03T10:09:44.200171+0000","last_undegraded":"2023-10-03T10:09:44.200171+0000","last_fullsized":"2023-10-03T10:09:44.200171+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200135+0000","last_change":"2023-10-03T10:09:43.756460+0000","last_active":"2023-10-03T10:09:44.200135+0000","last_peered":"2023-10-03T10:09:44.200135+0000","last_clean":"2023-10-03T10:09:44.200135+0000","last_became_active":"2023-10-03T10:09:43.756164+0000","last_became_peered":"2023-10-03T10:09:43.756164+0000","last_unstale":"2023-10-03T10:09:44.200135+0000","last_undegraded":"2023-10-03T10:09:44.200135+0000","last_fullsized":"2023-10-03T10:09:44.200135+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.189031+0000","last_change":"2023-10-03T10:09:44.181915+0000","last_active":"2023-10-03T10:09:44.189031+0000","last_peered":"2023-10-03T10:09:44.189031+0000","last_clean":"2023-10-03T10:09:44.189031+0000","last_became_active":"2023-10-03T10:09:44.181645+0000","last_became_peered":"2023-10-03T10:09:44.181645+0000","last_unstale":"2023-10-03T10:09:44.189031+0000","last_undegraded":"2023-10-03T10:09:44.189031+0000","last_fullsized":"2023-10-03T10:09:44.189031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.193051+0000","last_change":"2023-10-03T10:09:43.748380+0000","last_active":"2023-10-03T10:09:44.193051+0000","last_peered":"2023-10-03T10:09:44.193051+0000","last_clean":"2023-10-03T10:09:44.193051+0000","last_became_active":"2023-10-03T10:09:43.748104+0000","last_became_peered":"2023-10-03T10:09:43.748104+0000","last_unstale":"2023-10-03T10:09:44.193051+0000","last_undegraded":"2023-10-03T10:09:44.193051+0000","last_fullsized":"2023-10-03T10:09:44.193051+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200359+0000","last_change":"2023-10-03T10:09:44.197710+0000","last_active":"2023-10-03T10:09:44.200359+0000","last_peered":"2023-10-03T10:09:44.200359+0000","last_clean":"2023-10-03T10:09:44.200359+0000","last_became_active":"2023-10-03T10:09:44.197064+0000","last_became_peered":"2023-10-03T10:09:44.197064+0000","last_unstale":"2023-10-03T10:09:44.200359+0000","last_undegraded":"2023-10-03T10:09:44.200359+0000","last_fullsized":"2023-10-03T10:09:44.200359+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.193435+0000","last_change":"2023-10-03T10:09:43.758701+0000","last_active":"2023-10-03T10:09:44.193435+0000","last_peered":"2023-10-03T10:09:44.193435+0000","last_clean":"2023-10-03T10:09:44.193435+0000","last_became_active":"2023-10-03T10:09:43.758376+0000","last_became_peered":"2023-10-03T10:09:43.758376+0000","last_unstale":"2023-10-03T10:09:44.193435+0000","last_undegraded":"2023-10-03T10:09:44.193435+0000","last_fullsized":"2023-10-03T10:09:44.193435+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200488+0000","last_change":"2023-10-03T10:09:44.197164+0000","last_active":"2023-10-03T10:09:44.200488+0000","last_peered":"2023-10-03T10:09:44.200488+0000","last_clean":"2023-10-03T10:09:44.200488+0000","last_became_active":"2023-10-03T10:09:44.196885+0000","last_became_peered":"2023-10-03T10:09:44.196885+0000","last_unstale":"2023-10-03T10:09:44.200488+0000","last_undegraded":"2023-10-03T10:09:44.200488+0000","last_fullsized":"2023-10-03T10:09:44.200488+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.199237+0000","last_change":"2023-10-03T10:09:43.758373+0000","last_active":"2023-10-03T10:09:44.199237+0000","last_peered":"2023-10-03T10:09:44.199237+0000","last_clean":"2023-10-03T10:09:44.199237+0000","last_became_active":"2023-10-03T10:09:43.758061+0000","last_became_peered":"2023-10-03T10:09:43.758061+0000","last_unstale":"2023-10-03T10:09:44.199237+0000","last_undegraded":"2023-10-03T10:09:44.199237+0000","last_fullsized":"2023-10-03T10:09:44.199237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196006+0000","last_change":"2023-10-03T10:09:44.191564+0000","last_active":"2023-10-03T10:09:44.196006+0000","last_peered":"2023-10-03T10:09:44.196006+0000","last_clean":"2023-10-03T10:09:44.196006+0000","last_became_active":"2023-10-03T10:09:44.191220+0000","last_became_peered":"2023-10-03T10:09:44.191220+0000","last_unstale":"2023-10-03T10:09:44.196006+0000","last_undegraded":"2023-10-03T10:09:44.196006+0000","last_fullsized":"2023-10-03T10:09:44.196006+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"16'4","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.183675+0000","last_change":"2023-10-03T10:09:43.750463+0000","last_active":"2023-10-03T10:09:44.183675+0000","last_peered":"2023-10-03T10:09:44.183675+0000","last_clean":"2023-10-03T10:09:44.183675+0000","last_became_active":"2023-10-03T10:09:43.749780+0000","last_became_peered":"2023-10-03T10:09:43.749780+0000","last_unstale":"2023-10-03T10:09:44.183675+0000","last_undegraded":"2023-10-03T10:09:44.183675+0000","last_fullsized":"2023-10-03T10:09:44.183675+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.188984+0000","last_change":"2023-10-03T10:09:44.180378+0000","last_active":"2023-10-03T10:09:44.188984+0000","last_peered":"2023-10-03T10:09:44.188984+0000","last_clean":"2023-10-03T10:09:44.188984+0000","last_became_active":"2023-10-03T10:09:44.180082+0000","last_became_peered":"2023-10-03T10:09:44.180082+0000","last_unstale":"2023-10-03T10:09:44.188984+0000","last_undegraded":"2023-10-03T10:09:44.188984+0000","last_fullsized":"2023-10-03T10:09:44.188984+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.187808+0000","last_change":"2023-10-03T10:09:44.178067+0000","last_active":"2023-10-03T10:09:44.187808+0000","last_peered":"2023-10-03T10:09:44.187808+0000","last_clean":"2023-10-03T10:09:44.187808+0000","last_became_active":"2023-10-03T10:09:44.175124+0000","last_became_peered":"2023-10-03T10:09:44.175124+0000","last_unstale":"2023-10-03T10:09:44.187808+0000","last_undegraded":"2023-10-03T10:09:44.187808+0000","last_fullsized":"2023-10-03T10:09:44.187808+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.197083+0000","last_change":"2023-10-03T10:09:44.192156+0000","last_active":"2023-10-03T10:09:44.197083+0000","last_peered":"2023-10-03T10:09:44.197083+0000","last_clean":"2023-10-03T10:09:44.197083+0000","last_became_active":"2023-10-03T10:09:44.189296+0000","last_became_peered":"2023-10-03T10:09:44.189296+0000","last_unstale":"2023-10-03T10:09:44.197083+0000","last_undegraded":"2023-10-03T10:09:44.197083+0000","last_fullsized":"2023-10-03T10:09:44.197083+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.174106+0000","last_change":"2023-10-03T10:09:44.168473+0000","last_active":"2023-10-03T10:09:44.174106+0000","last_peered":"2023-10-03T10:09:44.174106+0000","last_clean":"2023-10-03T10:09:44.174106+0000","last_became_active":"2023-10-03T10:09:44.168155+0000","last_became_peered":"2023-10-03T10:09:44.168155+0000","last_unstale":"2023-10-03T10:09:44.174106+0000","last_undegraded":"2023-10-03T10:09:44.174106+0000","last_fullsized":"2023-10-03T10:09:44.174106+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200881+0000","last_change":"2023-10-03T10:09:44.197689+0000","last_active":"2023-10-03T10:09:44.200881+0000","last_peered":"2023-10-03T10:09:44.200881+0000","last_clean":"2023-10-03T10:09:44.200881+0000","last_became_active":"2023-10-03T10:09:44.197410+0000","last_became_peered":"2023-10-03T10:09:44.197410+0000","last_unstale":"2023-10-03T10:09:44.200881+0000","last_undegraded":"2023-10-03T10:09:44.200881+0000","last_fullsized":"2023-10-03T10:09:44.200881+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184190+0000","last_change":"2023-10-03T10:09:44.176966+0000","last_active":"2023-10-03T10:09:44.184190+0000","last_peered":"2023-10-03T10:09:44.184190+0000","last_clean":"2023-10-03T10:09:44.184190+0000","last_became_active":"2023-10-03T10:09:44.176386+0000","last_became_peered":"2023-10-03T10:09:44.176386+0000","last_unstale":"2023-10-03T10:09:44.184190+0000","last_undegraded":"2023-10-03T10:09:44.184190+0000","last_fullsized":"2023-10-03T10:09:44.184190+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705667,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297252,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074588,"statfs":{"total":96636764160,"available":96332378112,"internally_reserved":0,"allocated":159744,"data_stored":48735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]}]},{"osd":3,"up_from":9,"seq":38654705667,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297264,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074576,"statfs":{"total":96636764160,"available":96332365824,"internally_reserved":0,"allocated":172032,"data_stored":48929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]}]},{"osd":0,"up_from":9,"seq":38654705667,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297256,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074584,"statfs":{"total":96636764160,"available":96332374016,"internally_reserved":0,"allocated":163840,"data_stored":47899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]}]},{"osd":1,"up_from":9,"seq":38654705667,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297252,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074588,"statfs":{"total":96636764160,"available":96332378112,"internally_reserved":0,"allocated":159744,"data_stored":47845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-03T10:09:53.125 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-03T10:09:53.125 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-03T10:09:53.125 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-03T10:09:53.125 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-03T10:09:53.126 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-03T10:09:53.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.293+0000 7f0cd9563700 1 Processor -- start 2023-10-03T10:09:53.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.293+0000 7f0cd9563700 1 -- start start 2023-10-03T10:09:53.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.293+0000 7f0cd9563700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0cd4097460 0x7f0cd412fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:53.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.293+0000 7f0cd9563700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0cd4130230 0x7f0cd4132620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:53.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.293+0000 7f0cd9563700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd4097de0 con 0x7f0cd40661c0 2023-10-03T10:09:53.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.293+0000 7f0cd9563700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cd4097840 con 0x7f0cd4097460 2023-10-03T10:09:53.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.293+0000 7f0cd9563700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd406aaa0 con 0x7f0cd4130230 2023-10-03T10:09:53.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.293+0000 7f0cd27fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0cd4097460 0x7f0cd412fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:53.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.293+0000 7f0cd2ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0cd40661c0 0x7f0cd4127d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38444/0 (socket says 172.21.15.3:38444) 2023-10-03T10:09:53.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.294+0000 7f0cd2ffd700 1 -- 172.21.15.3:0/2882126940 learned_addr learned my addr 172.21.15.3:0/2882126940 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:09:53.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.293+0000 7f0cd37fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0cd4130230 0x7f0cd4132620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:53.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.294+0000 7f0cd1ffb700 1 -- 172.21.15.3:0/2882126940 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3924311704 0 0) 0x7f0cd4097de0 con 0x7f0cd40661c0 2023-10-03T10:09:53.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.294+0000 7f0cd1ffb700 1 -- 172.21.15.3:0/2882126940 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cbc0033f0 con 0x7f0cd40661c0 2023-10-03T10:09:53.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.294+0000 7f0cd37fe700 1 -- 172.21.15.3:0/2882126940 >> v2:172.21.15.3:3300/0 conn(0x7f0cd4097460 msgr2=0x7f0cd412fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:53.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.294+0000 7f0cd37fe700 1 --2- 172.21.15.3:0/2882126940 >> v2:172.21.15.3:3300/0 conn(0x7f0cd4097460 0x7f0cd412fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:53.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.294+0000 7f0cd37fe700 1 -- 172.21.15.3:0/2882126940 >> v1:172.21.15.3:6789/0 conn(0x7f0cd40661c0 legacy=0x7f0cd4127d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:53.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.294+0000 7f0cd37fe700 1 -- 172.21.15.3:0/2882126940 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd40665a0 con 0x7f0cd4130230 2023-10-03T10:09:53.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.294+0000 7f0cd37fe700 1 --2- 172.21.15.3:0/2882126940 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0cd4130230 0x7f0cd4132620 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f0ccc009e60 tx=0x7f0ccc009780).ready entity=mon.2 client_cookie=d8924469734d31a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:53.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.294+0000 7f0cd1ffb700 1 -- 172.21.15.3:0/2882126940 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc00e070 con 0x7f0cd4130230 2023-10-03T10:09:53.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.294+0000 7f0cd1ffb700 1 -- 172.21.15.3:0/2882126940 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ccc00bab0 con 0x7f0cd4130230 2023-10-03T10:09:53.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.294+0000 7f0cd1ffb700 1 -- 172.21.15.3:0/2882126940 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc00be30 con 0x7f0cd4130230 2023-10-03T10:09:53.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.295+0000 7f0cd9563700 1 -- 172.21.15.3:0/2882126940 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0cd4130230 msgr2=0x7f0cd4132620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:53.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.295+0000 7f0cd9563700 1 --2- 172.21.15.3:0/2882126940 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0cd4130230 0x7f0cd4132620 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f0ccc009e60 tx=0x7f0ccc009780).stop 2023-10-03T10:09:53.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.295+0000 7f0cd9563700 1 -- 172.21.15.3:0/2882126940 shutdown_connections 2023-10-03T10:09:53.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.295+0000 7f0cd9563700 1 --2- 172.21.15.3:0/2882126940 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0cd4130230 0x7f0cd4132620 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:53.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.295+0000 7f0cd9563700 1 --2- 172.21.15.3:0/2882126940 >> v2:172.21.15.3:3300/0 conn(0x7f0cd4097460 0x7f0cd412fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:53.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.295+0000 7f0cd9563700 1 -- 172.21.15.3:0/2882126940 >> 172.21.15.3:0/2882126940 conn(0x7f0cd4059c90 msgr2=0x7f0cd4098550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:53.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.295+0000 7f0cd9563700 1 -- 172.21.15.3:0/2882126940 shutdown_connections 2023-10-03T10:09:53.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.295+0000 7f0cd9563700 1 -- 172.21.15.3:0/2882126940 wait complete. 2023-10-03T10:09:53.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.296+0000 7f0cd9563700 1 Processor -- start 2023-10-03T10:09:53.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.296+0000 7f0cd9563700 1 -- start start 2023-10-03T10:09:53.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.296+0000 7f0cd9563700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0cd40661c0 0x7f0cd4113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:53.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.296+0000 7f0cd9563700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cd4097460 0x7f0cd412dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:53.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.296+0000 7f0cd9563700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cd4130230 0x7f0cd412e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:53.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.296+0000 7f0cd9563700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd41136f0 con 0x7f0cd4097460 2023-10-03T10:09:53.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.296+0000 7f0cd9563700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd4125f50 con 0x7f0cd4130230 2023-10-03T10:09:53.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.296+0000 7f0cd9563700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd40650e0 con 0x7f0cd40661c0 2023-10-03T10:09:53.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.297+0000 7f0cd2ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0cd40661c0 0x7f0cd4113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:53.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.297+0000 7f0cd27fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cd4097460 0x7f0cd412dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:53.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.297+0000 7f0cd2ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0cd40661c0 0x7f0cd4113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57240/0 (socket says 172.21.15.3:57240) 2023-10-03T10:09:53.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.297+0000 7f0cd2ffd700 1 -- 172.21.15.3:0/3424108034 learned_addr learned my addr 172.21.15.3:0/3424108034 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:53.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.297+0000 7f0cd37fe700 1 --2- 172.21.15.3:0/3424108034 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cd4130230 0x7f0cd412e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:53.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.297+0000 7f0cd2ffd700 1 -- 172.21.15.3:0/3424108034 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cd4130230 msgr2=0x7f0cd412e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:53.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.297+0000 7f0cd2ffd700 1 --2- 172.21.15.3:0/3424108034 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cd4130230 0x7f0cd412e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:53.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.297+0000 7f0cd2ffd700 1 -- 172.21.15.3:0/3424108034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cd4097460 msgr2=0x7f0cd412dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:53.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.297+0000 7f0cd2ffd700 1 --2- 172.21.15.3:0/3424108034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cd4097460 0x7f0cd412dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:53.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.297+0000 7f0cd2ffd700 1 -- 172.21.15.3:0/3424108034 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ccc009a20 con 0x7f0cd40661c0 2023-10-03T10:09:53.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.297+0000 7f0cd27fc700 1 --2- 172.21.15.3:0/3424108034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cd4097460 0x7f0cd412dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:53.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.297+0000 7f0cd37fe700 1 --2- 172.21.15.3:0/3424108034 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cd4130230 0x7f0cd412e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:53.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.297+0000 7f0cd2ffd700 1 --2- 172.21.15.3:0/3424108034 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0cd40661c0 0x7f0cd4113ed0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f0cc400c590 tx=0x7f0cc4007c80).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:53.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.297+0000 7f0cbaffd700 1 -- 172.21.15.3:0/3424108034 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc40072e0 con 0x7f0cd40661c0 2023-10-03T10:09:53.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.297+0000 7f0cbaffd700 1 -- 172.21.15.3:0/3424108034 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc4007440 con 0x7f0cd40661c0 2023-10-03T10:09:53.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.297+0000 7f0cd9563700 1 -- 172.21.15.3:0/3424108034 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd4113430 con 0x7f0cd40661c0 2023-10-03T10:09:53.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.297+0000 7f0cbaffd700 1 -- 172.21.15.3:0/3424108034 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc40057a0 con 0x7f0cd40661c0 2023-10-03T10:09:53.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.297+0000 7f0cd9563700 1 -- 172.21.15.3:0/3424108034 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd4112160 con 0x7f0cd40661c0 2023-10-03T10:09:53.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.298+0000 7f0cbaffd700 1 -- 172.21.15.3:0/3424108034 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f0cc4005bd0 con 0x7f0cd40661c0 2023-10-03T10:09:53.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.298+0000 7f0cbaffd700 1 -- 172.21.15.3:0/3424108034 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f0cc403a3d0 con 0x7f0cd40661c0 2023-10-03T10:09:53.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.298+0000 7f0cd9563700 1 -- 172.21.15.3:0/3424108034 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd4113840 con 0x7f0cd40661c0 2023-10-03T10:09:53.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.303+0000 7f0cbaffd700 1 -- 172.21.15.3:0/3424108034 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f0cc4018070 con 0x7f0cd40661c0 2023-10-03T10:09:53.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.526+0000 7f0cd9563700 1 -- 172.21.15.3:0/3424108034 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f0cd4096da0 con 0x7f0cd40661c0 2023-10-03T10:09:53.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.528+0000 7f0cbaffd700 1 -- 172.21.15.3:0/3424108034 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) v1 ==== 74+0+64126 (secure 0 0 0) 0x7f0cc4018020 con 0x7f0cd40661c0 2023-10-03T10:09:53.529 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:53.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.530+0000 7f0cd9563700 1 -- 172.21.15.3:0/3424108034 >> v1:172.21.15.3:6816/118296 conn(0x7f0cc002bb90 legacy=0x7f0cc002e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:53.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.530+0000 7f0cd9563700 1 -- 172.21.15.3:0/3424108034 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0cd40661c0 msgr2=0x7f0cd4113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:53.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.530+0000 7f0cd9563700 1 --2- 172.21.15.3:0/3424108034 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0cd40661c0 0x7f0cd4113ed0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f0cc400c590 tx=0x7f0cc4007c80).stop 2023-10-03T10:09:53.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.530+0000 7f0cd9563700 1 -- 172.21.15.3:0/3424108034 shutdown_connections 2023-10-03T10:09:53.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.530+0000 7f0cd9563700 1 --2- 172.21.15.3:0/3424108034 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cd4130230 0x7f0cd412e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:53.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.530+0000 7f0cd9563700 1 --2- 172.21.15.3:0/3424108034 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0cd40661c0 0x7f0cd4113ed0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:53.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.530+0000 7f0cd9563700 1 --2- 172.21.15.3:0/3424108034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cd4097460 0x7f0cd412dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:53.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.530+0000 7f0cd9563700 1 -- 172.21.15.3:0/3424108034 >> 172.21.15.3:0/3424108034 conn(0x7f0cd4059c90 msgr2=0x7f0cd409a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:53.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.530+0000 7f0cd9563700 1 -- 172.21.15.3:0/3424108034 shutdown_connections 2023-10-03T10:09:53.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.530+0000 7f0cd9563700 1 -- 172.21.15.3:0/3424108034 wait complete. 2023-10-03T10:09:53.543 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":4,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6816","nonce":118296}]},"active_addr":"172.21.15.3:6816/118296","active_change":"2023-10-03T10:09:33.519586+0000","active_mgr_features":4540138314316775423,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":3919724564}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":2091814084}]}]} 2023-10-03T10:09:53.545 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-03T10:09:53.545 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-03T10:09:53.546 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-03T10:09:53.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.676+0000 7fdd642e7700 1 Processor -- start 2023-10-03T10:09:53.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.676+0000 7fdd642e7700 1 -- start start 2023-10-03T10:09:53.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.676+0000 7fdd642e7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdd5c062a90 0x7fdd5c127860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:53.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.676+0000 7fdd642e7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdd5c127360 0x7fdd5c127d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:53.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.676+0000 7fdd642e7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdd5c095300 0x7fdd5c095780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:53.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.676+0000 7fdd642e7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd5c062e70 con 0x7fdd5c127360 2023-10-03T10:09:53.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.676+0000 7fdd642e7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c0968b0 con 0x7fdd5c095300 2023-10-03T10:09:53.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.676+0000 7fdd642e7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c091bc0 con 0x7fdd5c062a90 2023-10-03T10:09:53.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.677+0000 7fdd61882700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdd5c127360 0x7fdd5c127d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:53.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.677+0000 7fdd62083700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdd5c062a90 0x7fdd5c127860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:53.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.677+0000 7fdd62884700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdd5c095300 0x7fdd5c095780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:53.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.677+0000 7fdd61882700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdd5c127360 0x7fdd5c127d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47578/0 (socket says 172.21.15.3:47578) 2023-10-03T10:09:53.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.677+0000 7fdd62083700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdd5c062a90 0x7fdd5c127860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57252/0 (socket says 172.21.15.3:57252) 2023-10-03T10:09:53.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.677+0000 7fdd61882700 1 -- 172.21.15.3:0/1208799436 learned_addr learned my addr 172.21.15.3:0/1208799436 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:53.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.677+0000 7fdd61882700 1 -- 172.21.15.3:0/1208799436 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdd5c062a90 msgr2=0x7fdd5c127860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:53.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.677+0000 7fdd61882700 1 --2- 172.21.15.3:0/1208799436 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdd5c062a90 0x7fdd5c127860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:53.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.677+0000 7fdd61882700 1 -- 172.21.15.3:0/1208799436 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdd5c095300 msgr2=0x7fdd5c095780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:53.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.677+0000 7fdd61882700 1 --2- 172.21.15.3:0/1208799436 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdd5c095300 0x7fdd5c095780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:53.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.677+0000 7fdd61882700 1 -- 172.21.15.3:0/1208799436 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c092a70 con 0x7fdd5c127360 2023-10-03T10:09:53.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.677+0000 7fdd62083700 1 --2- 172.21.15.3:0/1208799436 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdd5c062a90 0x7fdd5c127860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:53.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.677+0000 7fdd62884700 1 --2- 172.21.15.3:0/1208799436 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdd5c095300 0x7fdd5c095780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:53.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.677+0000 7fdd61882700 1 --2- 172.21.15.3:0/1208799436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd5c127360 0x7fdd5c127d50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fdd4c009830 tx=0x7fdd4c0090f0).ready entity=mon.0 client_cookie=8117e13609280b92 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:53.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.677+0000 7fdd61081700 1 -- 172.21.15.3:0/1208799436 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c00f510 con 0x7fdd5c127360 2023-10-03T10:09:53.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.677+0000 7fdd61081700 1 -- 172.21.15.3:0/1208799436 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c00f670 con 0x7fdd5c127360 2023-10-03T10:09:53.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.677+0000 7fdd61081700 1 -- 172.21.15.3:0/1208799436 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c00fa10 con 0x7fdd5c127360 2023-10-03T10:09:53.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.678+0000 7fdd642e7700 1 -- 172.21.15.3:0/1208799436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd5c127360 msgr2=0x7fdd5c127d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:53.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.678+0000 7fdd642e7700 1 --2- 172.21.15.3:0/1208799436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd5c127360 0x7fdd5c127d50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fdd4c009830 tx=0x7fdd4c0090f0).stop 2023-10-03T10:09:53.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.678+0000 7fdd642e7700 1 -- 172.21.15.3:0/1208799436 shutdown_connections 2023-10-03T10:09:53.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.678+0000 7fdd642e7700 1 --2- 172.21.15.3:0/1208799436 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdd5c095300 0x7fdd5c095780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:53.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.678+0000 7fdd642e7700 1 --2- 172.21.15.3:0/1208799436 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdd5c062a90 0x7fdd5c127860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:53.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.678+0000 7fdd642e7700 1 --2- 172.21.15.3:0/1208799436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd5c127360 0x7fdd5c127d50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:53.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.678+0000 7fdd642e7700 1 -- 172.21.15.3:0/1208799436 >> 172.21.15.3:0/1208799436 conn(0x7fdd5c059bb0 msgr2=0x7fdd5c097400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:53.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.678+0000 7fdd642e7700 1 -- 172.21.15.3:0/1208799436 shutdown_connections 2023-10-03T10:09:53.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.678+0000 7fdd642e7700 1 -- 172.21.15.3:0/1208799436 wait complete. 2023-10-03T10:09:53.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.679+0000 7fdd642e7700 1 Processor -- start 2023-10-03T10:09:53.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.679+0000 7fdd642e7700 1 -- start start 2023-10-03T10:09:53.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.680+0000 7fdd642e7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd5c062a90 0x7fdd5c113fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:53.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.680+0000 7fdd642e7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdd5c095300 0x7fdd5c1144b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:53.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.680+0000 7fdd642e7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdd5c127360 0x7fdd5c1149a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:53.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.680+0000 7fdd642e7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c114e90 con 0x7fdd5c062a90 2023-10-03T10:09:53.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.680+0000 7fdd642e7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c117290 con 0x7fdd5c095300 2023-10-03T10:09:53.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.680+0000 7fdd642e7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c115be0 con 0x7fdd5c127360 2023-10-03T10:09:53.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.680+0000 7fdd62083700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd5c062a90 0x7fdd5c113fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:53.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.680+0000 7fdd62884700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdd5c127360 0x7fdd5c1149a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:53.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.680+0000 7fdd62083700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd5c062a90 0x7fdd5c113fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47584/0 (socket says 172.21.15.3:47584) 2023-10-03T10:09:53.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.680+0000 7fdd62083700 1 -- 172.21.15.3:0/3858477949 learned_addr learned my addr 172.21.15.3:0/3858477949 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:53.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.680+0000 7fdd61882700 1 --2- 172.21.15.3:0/3858477949 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdd5c095300 0x7fdd5c1144b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:53.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.680+0000 7fdd62083700 1 -- 172.21.15.3:0/3858477949 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdd5c127360 msgr2=0x7fdd5c1149a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:53.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.680+0000 7fdd62083700 1 --2- 172.21.15.3:0/3858477949 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdd5c127360 0x7fdd5c1149a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:53.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.680+0000 7fdd62083700 1 -- 172.21.15.3:0/3858477949 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdd5c095300 msgr2=0x7fdd5c1144b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:53.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.680+0000 7fdd62083700 1 --2- 172.21.15.3:0/3858477949 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdd5c095300 0x7fdd5c1144b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:53.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.680+0000 7fdd62083700 1 -- 172.21.15.3:0/3858477949 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd580092a0 con 0x7fdd5c062a90 2023-10-03T10:09:53.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.680+0000 7fdd62083700 1 --2- 172.21.15.3:0/3858477949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd5c062a90 0x7fdd5c113fc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fdd5800ea20 tx=0x7fdd58009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:53.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.680+0000 7fdd527fc700 1 -- 172.21.15.3:0/3858477949 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd58009600 con 0x7fdd5c062a90 2023-10-03T10:09:53.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.680+0000 7fdd527fc700 1 -- 172.21.15.3:0/3858477949 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5800c850 con 0x7fdd5c062a90 2023-10-03T10:09:53.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.680+0000 7fdd642e7700 1 -- 172.21.15.3:0/3858477949 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd54009a20 con 0x7fdd5c062a90 2023-10-03T10:09:53.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.680+0000 7fdd527fc700 1 -- 172.21.15.3:0/3858477949 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5800cbd0 con 0x7fdd5c062a90 2023-10-03T10:09:53.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.681+0000 7fdd642e7700 1 -- 172.21.15.3:0/3858477949 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd4c009370 con 0x7fdd5c062a90 2023-10-03T10:09:53.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.681+0000 7fdd527fc700 1 -- 172.21.15.3:0/3858477949 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fdd5801b400 con 0x7fdd5c062a90 2023-10-03T10:09:53.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.681+0000 7fdd527fc700 1 -- 172.21.15.3:0/3858477949 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fdd5803b5a0 con 0x7fdd5c062a90 2023-10-03T10:09:53.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.682+0000 7fdd642e7700 1 -- 172.21.15.3:0/3858477949 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd5c096050 con 0x7fdd5c062a90 2023-10-03T10:09:53.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.686+0000 7fdd527fc700 1 -- 172.21.15.3:0/3858477949 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fdd58016030 con 0x7fdd5c062a90 2023-10-03T10:09:53.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.864+0000 7fdd642e7700 1 -- 172.21.15.3:0/3858477949 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd5c092f80 con 0x7fdd5c062a90 2023-10-03T10:09:53.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.865+0000 7fdd527fc700 1 -- 172.21.15.3:0/3858477949 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+11309 (secure 0 0 0) 0x7fdd58016020 con 0x7fdd5c062a90 2023-10-03T10:09:53.865 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:53.865 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":17,"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","created":"2023-10-03T10:09:28.703441+0000","modified":"2023-10-03T10:09:45.145435+0000","last_up_change":"2023-10-03T10:09:36.569401+0000","last_in_change":"2023-10-03T10:09:30.156546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-10-03T10:09:36.623971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-03T10:09:38.210614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-03T10:09:42.695873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-03T10:09:43.115588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cdcdd930-3c5b-4486-a021-7132f90fb6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6808","nonce":118845}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6809","nonce":118845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6811","nonce":118845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6810","nonce":118845}]},"public_addr":"172.21.15.3:6808/118845","cluster_addr":"172.21.15.3:6809/118845","heartbeat_back_addr":"172.21.15.3:6811/118845","heartbeat_front_addr":"172.21.15.3:6810/118845","state":["exists","up"]},{"osd":1,"uuid":"c8e58877-6762-4741-a8c1-567ebc97b708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6804","nonce":118835}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6805","nonce":118835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6807","nonce":118835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6806","nonce":118835}]},"public_addr":"172.21.15.3:6804/118835","cluster_addr":"172.21.15.3:6805/118835","heartbeat_back_addr":"172.21.15.3:6807/118835","heartbeat_front_addr":"172.21.15.3:6806/118835","state":["exists","up"]},{"osd":2,"uuid":"68a2a21a-7a8e-4f82-8433-5201faa50777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6812","nonce":118864}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6813","nonce":118864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6815","nonce":118864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6814","nonce":118864}]},"public_addr":"172.21.15.3:6812/118864","cluster_addr":"172.21.15.3:6813/118864","heartbeat_back_addr":"172.21.15.3:6815/118864","heartbeat_front_addr":"172.21.15.3:6814/118864","state":["exists","up"]},{"osd":3,"uuid":"c4df2903-4e1e-4208-bf77-95b9fa749241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6800","nonce":118836}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6801","nonce":118836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6803","nonce":118836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6802","nonce":118836}]},"public_addr":"172.21.15.3:6800/118836","cluster_addr":"172.21.15.3:6801/118836","heartbeat_back_addr":"172.21.15.3:6803/118836","heartbeat_front_addr":"172.21.15.3:6802/118836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.369708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.257825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.337931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.085222+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-03T10:09:53.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.867+0000 7fdd642e7700 1 -- 172.21.15.3:0/3858477949 >> v1:172.21.15.3:6816/118296 conn(0x7fdd4002bb60 legacy=0x7fdd4002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:53.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.867+0000 7fdd642e7700 1 -- 172.21.15.3:0/3858477949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd5c062a90 msgr2=0x7fdd5c113fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:53.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.867+0000 7fdd642e7700 1 --2- 172.21.15.3:0/3858477949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd5c062a90 0x7fdd5c113fc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fdd5800ea20 tx=0x7fdd58009060).stop 2023-10-03T10:09:53.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.867+0000 7fdd642e7700 1 -- 172.21.15.3:0/3858477949 shutdown_connections 2023-10-03T10:09:53.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.867+0000 7fdd642e7700 1 --2- 172.21.15.3:0/3858477949 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdd5c127360 0x7fdd5c1149a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:53.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.867+0000 7fdd642e7700 1 --2- 172.21.15.3:0/3858477949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd5c062a90 0x7fdd5c113fc0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:53.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.867+0000 7fdd642e7700 1 --2- 172.21.15.3:0/3858477949 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdd5c095300 0x7fdd5c1144b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:53.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.867+0000 7fdd642e7700 1 -- 172.21.15.3:0/3858477949 >> 172.21.15.3:0/3858477949 conn(0x7fdd5c059bb0 msgr2=0x7fdd5c09a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:53.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.867+0000 7fdd642e7700 1 -- 172.21.15.3:0/3858477949 shutdown_connections 2023-10-03T10:09:53.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:53.867+0000 7fdd642e7700 1 -- 172.21.15.3:0/3858477949 wait complete. 2023-10-03T10:09:53.880 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-03T10:09:53.880 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-03T10:09:54.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.010+0000 7f21cd578700 1 Processor -- start 2023-10-03T10:09:54.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.010+0000 7f21cd578700 1 -- start start 2023-10-03T10:09:54.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.010+0000 7f21cd578700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21c80653b0 0x7f21c80986f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.010+0000 7f21cd578700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21c80992f0 0x7f21c81324a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.010+0000 7f21cd578700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21c8066070 con 0x7f21c8097cc0 2023-10-03T10:09:54.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.010+0000 7f21cd578700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c8065790 con 0x7f21c80992f0 2023-10-03T10:09:54.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.010+0000 7f21cd578700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f21c8095300 con 0x7f21c80653b0 2023-10-03T10:09:54.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.011+0000 7f21c6ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21c80653b0 0x7f21c80986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.011+0000 7f21c77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21c80992f0 0x7f21c81324a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.011+0000 7f21c67fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f21c8097cc0 0x7f21c8098be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38454/0 (socket says 172.21.15.3:38454) 2023-10-03T10:09:54.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.011+0000 7f21c77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21c80992f0 0x7f21c81324a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38964/0 (socket says 172.21.15.3:38964) 2023-10-03T10:09:54.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.011+0000 7f21c6ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21c80653b0 0x7f21c80986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57264/0 (socket says 172.21.15.3:57264) 2023-10-03T10:09:54.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.011+0000 7f21c67fc700 1 -- 172.21.15.3:0/2907537352 learned_addr learned my addr 172.21.15.3:0/2907537352 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:09:54.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.011+0000 7f21c77fe700 1 -- 172.21.15.3:0/2907537352 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21c80653b0 msgr2=0x7f21c80986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:54.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.011+0000 7f21c5ffb700 1 -- 172.21.15.3:0/2907537352 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1564849650 0 0) 0x7f21c8066070 con 0x7f21c8097cc0 2023-10-03T10:09:54.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.011+0000 7f21c77fe700 1 --2- 172.21.15.3:0/2907537352 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21c80653b0 0x7f21c80986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.011+0000 7f21c77fe700 1 -- 172.21.15.3:0/2907537352 >> v1:172.21.15.3:6789/0 conn(0x7f21c8097cc0 legacy=0x7f21c8098be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.011+0000 7f21c77fe700 1 -- 172.21.15.3:0/2907537352 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c8098150 con 0x7f21c80992f0 2023-10-03T10:09:54.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.011+0000 7f21c77fe700 1 --2- 172.21.15.3:0/2907537352 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21c80992f0 0x7f21c81324a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f21c0009e90 tx=0x7f21c0009780).ready entity=mon.1 client_cookie=2de41d11fed14f93 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:54.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.011+0000 7f21c5ffb700 1 -- 172.21.15.3:0/2907537352 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c000e070 con 0x7f21c80992f0 2023-10-03T10:09:54.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.011+0000 7f21c5ffb700 1 -- 172.21.15.3:0/2907537352 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c000bb60 con 0x7f21c80992f0 2023-10-03T10:09:54.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.011+0000 7f21c5ffb700 1 -- 172.21.15.3:0/2907537352 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c0012410 con 0x7f21c80992f0 2023-10-03T10:09:54.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.012+0000 7f21cd578700 1 -- 172.21.15.3:0/2907537352 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21c80992f0 msgr2=0x7f21c81324a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.012+0000 7f21cd578700 1 --2- 172.21.15.3:0/2907537352 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21c80992f0 0x7f21c81324a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f21c0009e90 tx=0x7f21c0009780).stop 2023-10-03T10:09:54.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.012+0000 7f21cd578700 1 -- 172.21.15.3:0/2907537352 shutdown_connections 2023-10-03T10:09:54.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.012+0000 7f21cd578700 1 --2- 172.21.15.3:0/2907537352 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21c80992f0 0x7f21c81324a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.012+0000 7f21cd578700 1 --2- 172.21.15.3:0/2907537352 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21c80653b0 0x7f21c80986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.012+0000 7f21cd578700 1 -- 172.21.15.3:0/2907537352 >> 172.21.15.3:0/2907537352 conn(0x7f21c8059cd0 msgr2=0x7f21c809a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:54.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.012+0000 7f21cd578700 1 -- 172.21.15.3:0/2907537352 shutdown_connections 2023-10-03T10:09:54.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.012+0000 7f21cd578700 1 -- 172.21.15.3:0/2907537352 wait complete. 2023-10-03T10:09:54.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.013+0000 7f21cd578700 1 Processor -- start 2023-10-03T10:09:54.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.013+0000 7f21cd578700 1 -- start start 2023-10-03T10:09:54.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.014+0000 7f21cd578700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21c80653b0 0x7f21c811e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.014+0000 7f21cd578700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21c80992f0 0x7f21c811a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.014+0000 7f21cd578700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21c811a8b0 0x7f21c811ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.014+0000 7f21cd578700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c811ca80 con 0x7f21c80653b0 2023-10-03T10:09:54.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.014+0000 7f21cd578700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c8132550 con 0x7f21c811a8b0 2023-10-03T10:09:54.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.014+0000 7f21cd578700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f21c8099a80 con 0x7f21c80992f0 2023-10-03T10:09:54.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.014+0000 7f21c6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21c80653b0 0x7f21c811e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.014+0000 7f21c77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21c811a8b0 0x7f21c811ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.014+0000 7f21c67fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21c80992f0 0x7f21c811a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.014+0000 7f21c6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21c80653b0 0x7f21c811e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47588/0 (socket says 172.21.15.3:47588) 2023-10-03T10:09:54.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.014+0000 7f21c77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21c811a8b0 0x7f21c811ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38976/0 (socket says 172.21.15.3:38976) 2023-10-03T10:09:54.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.014+0000 7f21c6ffd700 1 -- 172.21.15.3:0/2353643190 learned_addr learned my addr 172.21.15.3:0/2353643190 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:54.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.014+0000 7f21c6ffd700 1 -- 172.21.15.3:0/2353643190 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21c80992f0 msgr2=0x7f21c811a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.014+0000 7f21c6ffd700 1 --2- 172.21.15.3:0/2353643190 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21c80992f0 0x7f21c811a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.014+0000 7f21c6ffd700 1 -- 172.21.15.3:0/2353643190 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21c811a8b0 msgr2=0x7f21c811ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.014+0000 7f21c6ffd700 1 --2- 172.21.15.3:0/2353643190 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21c811a8b0 0x7f21c811ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.014+0000 7f21c6ffd700 1 -- 172.21.15.3:0/2353643190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c0009a20 con 0x7f21c80653b0 2023-10-03T10:09:54.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.014+0000 7f21c6ffd700 1 --2- 172.21.15.3:0/2353643190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21c80653b0 0x7f21c811e3b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f21b800e930 tx=0x7f21b8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:54.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.014+0000 7f21aeffd700 1 -- 172.21.15.3:0/2353643190 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b8012600 con 0x7f21c80653b0 2023-10-03T10:09:54.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.014+0000 7f21aeffd700 1 -- 172.21.15.3:0/2353643190 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b8012760 con 0x7f21c80653b0 2023-10-03T10:09:54.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.014+0000 7f21aeffd700 1 -- 172.21.15.3:0/2353643190 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b8012ae0 con 0x7f21c80653b0 2023-10-03T10:09:54.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.014+0000 7f21cd578700 1 -- 172.21.15.3:0/2353643190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c809c8e0 con 0x7f21c80653b0 2023-10-03T10:09:54.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.015+0000 7f21cd578700 1 -- 172.21.15.3:0/2353643190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c811d910 con 0x7f21c80653b0 2023-10-03T10:09:54.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.015+0000 7f21aeffd700 1 -- 172.21.15.3:0/2353643190 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f21b801b400 con 0x7f21c80653b0 2023-10-03T10:09:54.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.015+0000 7f21aeffd700 1 -- 172.21.15.3:0/2353643190 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f21b803b450 con 0x7f21c80653b0 2023-10-03T10:09:54.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.016+0000 7f21cd578700 1 -- 172.21.15.3:0/2353643190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c811dee0 con 0x7f21c80653b0 2023-10-03T10:09:54.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.020+0000 7f21aeffd700 1 -- 172.21.15.3:0/2353643190 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f21b8019900 con 0x7f21c80653b0 2023-10-03T10:09:54.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.197+0000 7f21cd578700 1 -- 172.21.15.3:0/2353643190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21c811dcd0 con 0x7f21c80653b0 2023-10-03T10:09:54.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.197+0000 7f21aeffd700 1 -- 172.21.15.3:0/2353643190 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+11309 (secure 0 0 0) 0x7f21b8019310 con 0x7f21c80653b0 2023-10-03T10:09:54.198 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:54.198 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":17,"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","created":"2023-10-03T10:09:28.703441+0000","modified":"2023-10-03T10:09:45.145435+0000","last_up_change":"2023-10-03T10:09:36.569401+0000","last_in_change":"2023-10-03T10:09:30.156546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-10-03T10:09:36.623971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-03T10:09:38.210614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-03T10:09:42.695873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-03T10:09:43.115588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cdcdd930-3c5b-4486-a021-7132f90fb6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6808","nonce":118845}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6809","nonce":118845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6811","nonce":118845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6810","nonce":118845}]},"public_addr":"172.21.15.3:6808/118845","cluster_addr":"172.21.15.3:6809/118845","heartbeat_back_addr":"172.21.15.3:6811/118845","heartbeat_front_addr":"172.21.15.3:6810/118845","state":["exists","up"]},{"osd":1,"uuid":"c8e58877-6762-4741-a8c1-567ebc97b708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6804","nonce":118835}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6805","nonce":118835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6807","nonce":118835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6806","nonce":118835}]},"public_addr":"172.21.15.3:6804/118835","cluster_addr":"172.21.15.3:6805/118835","heartbeat_back_addr":"172.21.15.3:6807/118835","heartbeat_front_addr":"172.21.15.3:6806/118835","state":["exists","up"]},{"osd":2,"uuid":"68a2a21a-7a8e-4f82-8433-5201faa50777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6812","nonce":118864}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6813","nonce":118864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6815","nonce":118864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6814","nonce":118864}]},"public_addr":"172.21.15.3:6812/118864","cluster_addr":"172.21.15.3:6813/118864","heartbeat_back_addr":"172.21.15.3:6815/118864","heartbeat_front_addr":"172.21.15.3:6814/118864","state":["exists","up"]},{"osd":3,"uuid":"c4df2903-4e1e-4208-bf77-95b9fa749241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6800","nonce":118836}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6801","nonce":118836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6803","nonce":118836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6802","nonce":118836}]},"public_addr":"172.21.15.3:6800/118836","cluster_addr":"172.21.15.3:6801/118836","heartbeat_back_addr":"172.21.15.3:6803/118836","heartbeat_front_addr":"172.21.15.3:6802/118836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.369708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.257825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.337931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.085222+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-03T10:09:54.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.199+0000 7f21cd578700 1 -- 172.21.15.3:0/2353643190 >> v1:172.21.15.3:6816/118296 conn(0x7f21b402bb60 legacy=0x7f21b402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.199+0000 7f21cd578700 1 -- 172.21.15.3:0/2353643190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21c80653b0 msgr2=0x7f21c811e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.199+0000 7f21cd578700 1 --2- 172.21.15.3:0/2353643190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21c80653b0 0x7f21c811e3b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f21b800e930 tx=0x7f21b8009060).stop 2023-10-03T10:09:54.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.199+0000 7f21cd578700 1 -- 172.21.15.3:0/2353643190 shutdown_connections 2023-10-03T10:09:54.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.199+0000 7f21cd578700 1 --2- 172.21.15.3:0/2353643190 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21c811a8b0 0x7f21c811ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.199+0000 7f21cd578700 1 --2- 172.21.15.3:0/2353643190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21c80653b0 0x7f21c811e3b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.199+0000 7f21cd578700 1 --2- 172.21.15.3:0/2353643190 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21c80992f0 0x7f21c811a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.199+0000 7f21cd578700 1 -- 172.21.15.3:0/2353643190 >> 172.21.15.3:0/2353643190 conn(0x7f21c8059cd0 msgr2=0x7f21c809dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:54.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.200+0000 7f21cd578700 1 -- 172.21.15.3:0/2353643190 shutdown_connections 2023-10-03T10:09:54.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.200+0000 7f21cd578700 1 -- 172.21.15.3:0/2353643190 wait complete. 2023-10-03T10:09:54.213 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-03T10:09:54.214 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-03T10:09:54.214 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-03T10:09:54.215 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-03T10:09:54.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.354+0000 7fd839db8700 1 Processor -- start 2023-10-03T10:09:54.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.354+0000 7fd839db8700 1 -- start start 2023-10-03T10:09:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.355+0000 7fd839db8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd834062a90 0x7fd83412f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.355+0000 7fd839db8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd83412ed70 0x7fd83412f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.355+0000 7fd839db8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd834099ee0 con 0x7fd834098820 2023-10-03T10:09:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.355+0000 7fd839db8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd834062e70 con 0x7fd83412ed70 2023-10-03T10:09:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.355+0000 7fd839db8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd834091ce0 con 0x7fd834062a90 2023-10-03T10:09:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.355+0000 7fd833fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd834098820 0x7fd834098c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38458/0 (socket says 172.21.15.3:38458) 2023-10-03T10:09:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.355+0000 7fd833fff700 1 -- 172.21.15.3:0/1982183374 learned_addr learned my addr 172.21.15.3:0/1982183374 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:09:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.355+0000 7fd832ffd700 1 --2- 172.21.15.3:0/1982183374 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd83412ed70 0x7fd83412f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.356+0000 7fd8337fe700 1 --2- 172.21.15.3:0/1982183374 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd834062a90 0x7fd83412f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.356+0000 7fd8327fc700 1 -- 172.21.15.3:0/1982183374 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516455395 0 0) 0x7fd82c001490 con 0x7fd834098820 2023-10-03T10:09:54.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.356+0000 7fd8327fc700 1 -- 172.21.15.3:0/1982183374 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd81c0033f0 con 0x7fd834098820 2023-10-03T10:09:54.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.356+0000 7fd832ffd700 1 -- 172.21.15.3:0/1982183374 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd834062a90 msgr2=0x7fd83412f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:54.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.356+0000 7fd832ffd700 1 --2- 172.21.15.3:0/1982183374 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd834062a90 0x7fd83412f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.356+0000 7fd832ffd700 1 -- 172.21.15.3:0/1982183374 >> v1:172.21.15.3:6789/0 conn(0x7fd834098820 legacy=0x7fd834098c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.356+0000 7fd832ffd700 1 -- 172.21.15.3:0/1982183374 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd834091a40 con 0x7fd83412ed70 2023-10-03T10:09:54.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.356+0000 7fd8337fe700 1 --2- 172.21.15.3:0/1982183374 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd834062a90 0x7fd83412f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:09:54.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.356+0000 7fd832ffd700 1 --2- 172.21.15.3:0/1982183374 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd83412ed70 0x7fd83412f6f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fd828009790 tx=0x7fd828009060).ready entity=mon.1 client_cookie=be346473ca8f35d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:54.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.356+0000 7fd8327fc700 1 -- 172.21.15.3:0/1982183374 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82800f510 con 0x7fd83412ed70 2023-10-03T10:09:54.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.356+0000 7fd8327fc700 1 -- 172.21.15.3:0/1982183374 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82800f670 con 0x7fd83412ed70 2023-10-03T10:09:54.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.356+0000 7fd8327fc700 1 -- 172.21.15.3:0/1982183374 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82800f9f0 con 0x7fd83412ed70 2023-10-03T10:09:54.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.357+0000 7fd839db8700 1 -- 172.21.15.3:0/1982183374 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd83412ed70 msgr2=0x7fd83412f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.357+0000 7fd839db8700 1 --2- 172.21.15.3:0/1982183374 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd83412ed70 0x7fd83412f6f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fd828009790 tx=0x7fd828009060).stop 2023-10-03T10:09:54.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.357+0000 7fd839db8700 1 -- 172.21.15.3:0/1982183374 shutdown_connections 2023-10-03T10:09:54.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.357+0000 7fd839db8700 1 --2- 172.21.15.3:0/1982183374 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd834062a90 0x7fd83412f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.357+0000 7fd839db8700 1 --2- 172.21.15.3:0/1982183374 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd83412ed70 0x7fd83412f6f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.357+0000 7fd839db8700 1 -- 172.21.15.3:0/1982183374 >> 172.21.15.3:0/1982183374 conn(0x7fd834097460 msgr2=0x7fd834097870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:54.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.357+0000 7fd839db8700 1 -- 172.21.15.3:0/1982183374 shutdown_connections 2023-10-03T10:09:54.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.357+0000 7fd839db8700 1 -- 172.21.15.3:0/1982183374 wait complete. 2023-10-03T10:09:54.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.357+0000 7f2c04917700 1 Processor -- start 2023-10-03T10:09:54.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.357+0000 7f878d8b1700 1 Processor -- start 2023-10-03T10:09:54.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.358+0000 7f878d8b1700 1 -- start start 2023-10-03T10:09:54.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.358+0000 7f878d8b1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8788094550 0x7f8788123b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.358+0000 7f878d8b1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8788124160 0x7f87881245e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.358+0000 7f878d8b1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8788091a40 0x7f8788091ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.358+0000 7f878d8b1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8788092fc0 con 0x7f8788124160 2023-10-03T10:09:54.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.358+0000 7f878d8b1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8788093ab0 con 0x7f8788094550 2023-10-03T10:09:54.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.358+0000 7f878d8b1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8788093d70 con 0x7f8788091a40 2023-10-03T10:09:54.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.358+0000 7fd839db8700 1 Processor -- start 2023-10-03T10:09:54.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.358+0000 7f8786ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8788094550 0x7f8788123b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.358+0000 7f8786ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8788094550 0x7f8788123b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38990/0 (socket says 172.21.15.3:38990) 2023-10-03T10:09:54.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.358+0000 7f8786ffd700 1 -- 172.21.15.3:0/1536862999 learned_addr learned my addr 172.21.15.3:0/1536862999 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:54.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.358+0000 7f87877fe700 1 --2- 172.21.15.3:0/1536862999 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8788091a40 0x7f8788091ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.359+0000 7f87867fc700 1 --2- 172.21.15.3:0/1536862999 >> v2:172.21.15.3:3300/0 conn(0x7f8788124160 0x7f87881245e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.358+0000 7f2c04917700 1 -- start start 2023-10-03T10:09:54.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.359+0000 7f2c04917700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c00068e70 0x7f2c00091d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.359+0000 7f2c04917700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2c00092240 0x7f2c00126e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.359+0000 7f2c04917700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c000698e0 con 0x7f2c00092680 2023-10-03T10:09:54.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.359+0000 7f2c04917700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c00069250 con 0x7f2c00092240 2023-10-03T10:09:54.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.359+0000 7f2c04917700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c00069ca0 con 0x7f2c00068e70 2023-10-03T10:09:54.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.359+0000 7f2bfe59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c00068e70 0x7f2c00091d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.359+0000 7f2bfe59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c00068e70 0x7f2c00091d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57308/0 (socket says 172.21.15.3:57308) 2023-10-03T10:09:54.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.359+0000 7f2bfe59c700 1 -- 172.21.15.3:0/1733078428 learned_addr learned my addr 172.21.15.3:0/1733078428 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:54.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.359+0000 7f87867fc700 1 -- 172.21.15.3:0/1536862999 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8788091a40 msgr2=0x7f8788091ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:54.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.359+0000 7f87867fc700 1 --2- 172.21.15.3:0/1536862999 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8788091a40 0x7f8788091ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.359+0000 7f87867fc700 1 -- 172.21.15.3:0/1536862999 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8788094550 msgr2=0x7f8788123b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:54.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.359+0000 7f87867fc700 1 --2- 172.21.15.3:0/1536862999 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8788094550 0x7f8788123b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.359+0000 7f87867fc700 1 -- 172.21.15.3:0/1536862999 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8788094930 con 0x7f8788124160 2023-10-03T10:09:54.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.359+0000 7f8786ffd700 1 --2- 172.21.15.3:0/1536862999 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8788094550 0x7f8788123b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:54.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.358+0000 7fd839db8700 1 -- start start 2023-10-03T10:09:54.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.359+0000 7fd839db8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd834062a90 0x7fd83411f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.359+0000 7fd839db8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd834098820 0x7fd83411b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.359+0000 7fd839db8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd83411b940 0x7fd83411bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.359+0000 7fd839db8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83411c8f0 con 0x7fd83411b940 2023-10-03T10:09:54.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.359+0000 7fd839db8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83411e9e0 con 0x7fd834062a90 2023-10-03T10:09:54.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.359+0000 7fd839db8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd83411c290 con 0x7fd834098820 2023-10-03T10:09:54.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.359+0000 7f87867fc700 1 --2- 172.21.15.3:0/1536862999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8788124160 0x7f87881245e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f877c00d8d0 tx=0x7f877c009060).ready entity=mon.0 client_cookie=319d8288f1548ccb server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:54.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.360+0000 7fd832ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd834098820 0x7fd83411b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.360+0000 7fd832ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd834098820 0x7fd83411b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57312/0 (socket says 172.21.15.3:57312) 2023-10-03T10:09:54.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.360+0000 7fd832ffd700 1 -- 172.21.15.3:0/3016902769 learned_addr learned my addr 172.21.15.3:0/3016902769 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:54.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.360+0000 7fd833fff700 1 --2- 172.21.15.3:0/3016902769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd83411b940 0x7fd83411bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.360+0000 7f2bfe59c700 1 -- 172.21.15.3:0/1733078428 >> v2:172.21.15.3:3300/0 conn(0x7f2c00092240 msgr2=0x7f2c00126e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:54.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.360+0000 7f2bfe59c700 1 --2- 172.21.15.3:0/1733078428 >> v2:172.21.15.3:3300/0 conn(0x7f2c00092240 0x7f2c00126e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-03T10:09:54.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.360+0000 7f2bfe59c700 1 -- 172.21.15.3:0/1733078428 >> v1:172.21.15.3:6789/0 conn(0x7f2c00092680 legacy=0x7f2c00127330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.360+0000 7f2bfe59c700 1 -- 172.21.15.3:0/1733078428 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c00069f10 con 0x7f2c00068e70 2023-10-03T10:09:54.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.360+0000 7f8785ffb700 1 -- 172.21.15.3:0/1536862999 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c00d270 con 0x7f8788124160 2023-10-03T10:09:54.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.360+0000 7f8785ffb700 1 -- 172.21.15.3:0/1536862999 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877c0096e0 con 0x7f8788124160 2023-10-03T10:09:54.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.360+0000 7f8785ffb700 1 -- 172.21.15.3:0/1536862999 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c009a60 con 0x7f8788124160 2023-10-03T10:09:54.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.360+0000 7f2bfe59c700 1 --2- 172.21.15.3:0/1733078428 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c00068e70 0x7f2c00091d50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f2bf000bac0 tx=0x7f2bf000b370).ready entity=mon.2 client_cookie=a2b28148dfc42c88 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:54.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.360+0000 7fd8337fe700 1 --2- 172.21.15.3:0/3016902769 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd834062a90 0x7fd83411f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.360+0000 7fd832ffd700 1 -- 172.21.15.3:0/3016902769 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd834062a90 msgr2=0x7fd83411f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.360+0000 7fd832ffd700 1 --2- 172.21.15.3:0/3016902769 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd834062a90 0x7fd83411f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.360+0000 7fd832ffd700 1 -- 172.21.15.3:0/3016902769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd83411b940 msgr2=0x7fd83411bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.360+0000 7fd832ffd700 1 --2- 172.21.15.3:0/3016902769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd83411b940 0x7fd83411bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.360+0000 7f9551f7c700 1 Processor -- start 2023-10-03T10:09:54.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.360+0000 7fd832ffd700 1 -- 172.21.15.3:0/3016902769 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8280092a0 con 0x7fd834098820 2023-10-03T10:09:54.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.360+0000 7fd833fff700 1 --2- 172.21.15.3:0/3016902769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd83411b940 0x7fd83411bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:54.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.360+0000 7f9551f7c700 1 -- start start 2023-10-03T10:09:54.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.360+0000 7fd832ffd700 1 --2- 172.21.15.3:0/3016902769 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd834098820 0x7fd83411b450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fd828009e30 tx=0x7fd828017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:54.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.361+0000 7f2bfd59a700 1 -- 172.21.15.3:0/1733078428 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf000db40 con 0x7f2c00068e70 2023-10-03T10:09:54.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.361+0000 7fd81b7fe700 1 -- 172.21.15.3:0/3016902769 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82800f510 con 0x7fd834098820 2023-10-03T10:09:54.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.361+0000 7fd81b7fe700 1 -- 172.21.15.3:0/3016902769 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82800f7e0 con 0x7fd834098820 2023-10-03T10:09:54.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.361+0000 7f2bfd59a700 1 -- 172.21.15.3:0/1733078428 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf000dca0 con 0x7f2c00068e70 2023-10-03T10:09:54.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.361+0000 7fd839db8700 1 -- 172.21.15.3:0/3016902769 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd83411e5a0 con 0x7fd834098820 2023-10-03T10:09:54.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.361+0000 7f2bfd59a700 1 -- 172.21.15.3:0/1733078428 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0016650 con 0x7f2c00068e70 2023-10-03T10:09:54.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.361+0000 7fd81b7fe700 1 -- 172.21.15.3:0/3016902769 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82801b440 con 0x7fd834098820 2023-10-03T10:09:54.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.361+0000 7fd839db8700 1 -- 172.21.15.3:0/3016902769 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd83409a080 con 0x7fd834098820 2023-10-03T10:09:54.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.361+0000 7f9551f7c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f954c127f20 0x7f954c12fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.361+0000 7f9551f7c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f954c130160 0x7f954c132550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.361+0000 7f9551f7c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f954c092c10 con 0x7f954c069870 2023-10-03T10:09:54.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.361+0000 7f9551f7c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f954c069c50 con 0x7f954c130160 2023-10-03T10:09:54.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.361+0000 7f9551f7c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954c128380 con 0x7f954c127f20 2023-10-03T10:09:54.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.361+0000 7f954b7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f954c069870 0x7f954c127810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38484/0 (socket says 172.21.15.3:38484) 2023-10-03T10:09:54.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.361+0000 7f954b7fe700 1 -- 172.21.15.3:0/1816010760 learned_addr learned my addr 172.21.15.3:0/1816010760 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:09:54.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.361+0000 7f878d8b1700 1 -- 172.21.15.3:0/1536862999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8788124160 msgr2=0x7f87881245e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.361+0000 7f878d8b1700 1 --2- 172.21.15.3:0/1536862999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8788124160 0x7f87881245e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f877c00d8d0 tx=0x7f877c009060).stop 2023-10-03T10:09:54.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.362+0000 7f2c04917700 1 -- 172.21.15.3:0/1733078428 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c00068e70 msgr2=0x7f2c00091d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.362+0000 7f2c04917700 1 --2- 172.21.15.3:0/1733078428 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c00068e70 0x7f2c00091d50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f2bf000bac0 tx=0x7f2bf000b370).stop 2023-10-03T10:09:54.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.362+0000 7f2c04917700 1 -- 172.21.15.3:0/1733078428 shutdown_connections 2023-10-03T10:09:54.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.362+0000 7f2c04917700 1 --2- 172.21.15.3:0/1733078428 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c00068e70 0x7f2c00091d50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.362+0000 7f2c04917700 1 --2- 172.21.15.3:0/1733078428 >> v2:172.21.15.3:3300/0 conn(0x7f2c00092240 0x7f2c00126e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-03T10:09:54.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.362+0000 7f2c04917700 1 -- 172.21.15.3:0/1733078428 >> 172.21.15.3:0/1733078428 conn(0x7f2c00063560 msgr2=0x7f2c0009a220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:54.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.362+0000 7fd839db8700 1 -- 172.21.15.3:0/3016902769 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd83412e5f0 con 0x7fd834098820 2023-10-03T10:09:54.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.362+0000 7f2c04917700 1 -- 172.21.15.3:0/1733078428 shutdown_connections 2023-10-03T10:09:54.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.362+0000 7f954bfff700 1 --2- 172.21.15.3:0/1816010760 >> v2:172.21.15.3:3300/0 conn(0x7f954c130160 0x7f954c132550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.362+0000 7f2c04917700 1 -- 172.21.15.3:0/1733078428 wait complete. 2023-10-03T10:09:54.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.362+0000 7f9542ffd700 1 --2- 172.21.15.3:0/1816010760 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f954c127f20 0x7f954c12fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.362+0000 7f954bfff700 1 -- 172.21.15.3:0/1816010760 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f954c127f20 msgr2=0x7f954c12fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:54.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.362+0000 7f954bfff700 1 --2- 172.21.15.3:0/1816010760 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f954c127f20 0x7f954c12fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.362+0000 7f954bfff700 1 -- 172.21.15.3:0/1816010760 >> v1:172.21.15.3:6789/0 conn(0x7f954c069870 legacy=0x7f954c127810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.362+0000 7f954bfff700 1 -- 172.21.15.3:0/1816010760 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954c095c70 con 0x7f954c130160 2023-10-03T10:09:54.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.362+0000 7f9542ffd700 1 --2- 172.21.15.3:0/1816010760 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f954c127f20 0x7f954c12fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:09:54.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.362+0000 7f954bfff700 1 --2- 172.21.15.3:0/1816010760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f954c130160 0x7f954c132550 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f9544009e60 tx=0x7f9544009780).ready entity=mon.0 client_cookie=11a7e5b885162498 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:54.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.363+0000 7f954affd700 1 -- 172.21.15.3:0/1816010760 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954400e070 con 0x7f954c130160 2023-10-03T10:09:54.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.363+0000 7f954affd700 1 -- 172.21.15.3:0/1816010760 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954400bc30 con 0x7f954c130160 2023-10-03T10:09:54.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.363+0000 7f954affd700 1 -- 172.21.15.3:0/1816010760 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9544012410 con 0x7f954c130160 2023-10-03T10:09:54.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.363+0000 7fd81b7fe700 1 -- 172.21.15.3:0/3016902769 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fd82801b7b0 con 0x7fd834098820 2023-10-03T10:09:54.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.363+0000 7fd81b7fe700 1 -- 172.21.15.3:0/3016902769 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fd82803ce90 con 0x7fd834098820 2023-10-03T10:09:54.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.363+0000 7f2c04917700 1 Processor -- start 2023-10-03T10:09:54.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.363+0000 7fd81b7fe700 1 -- 172.21.15.3:0/3016902769 --> v1:172.21.15.3:6800/118836 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd8200384e0 con 0x7fd8200347f0 2023-10-03T10:09:54.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.363+0000 7fd81b7fe700 1 -- 172.21.15.3:0/3016902769 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd82803d290 con 0x7fd834098820 2023-10-03T10:09:54.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.363+0000 7f2c04917700 1 -- start start 2023-10-03T10:09:54.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.363+0000 7f9551f7c700 1 -- 172.21.15.3:0/1816010760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f954c130160 msgr2=0x7f954c132550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.364+0000 7f9551f7c700 1 --2- 172.21.15.3:0/1816010760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f954c130160 0x7f954c132550 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f9544009e60 tx=0x7f9544009780).stop 2023-10-03T10:09:54.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.364+0000 7f878d8b1700 1 -- 172.21.15.3:0/1536862999 shutdown_connections 2023-10-03T10:09:54.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.364+0000 7f878d8b1700 1 --2- 172.21.15.3:0/1536862999 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8788091a40 0x7f8788091ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.364+0000 7f878d8b1700 1 --2- 172.21.15.3:0/1536862999 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8788094550 0x7f8788123b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.364+0000 7f878d8b1700 1 --2- 172.21.15.3:0/1536862999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8788124160 0x7f87881245e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.364+0000 7f878d8b1700 1 -- 172.21.15.3:0/1536862999 >> 172.21.15.3:0/1536862999 conn(0x7f878809a1b0 msgr2=0x7f878809a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:54.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.364+0000 7f9551f7c700 1 -- 172.21.15.3:0/1816010760 shutdown_connections 2023-10-03T10:09:54.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.364+0000 7f9551f7c700 1 --2- 172.21.15.3:0/1816010760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f954c130160 0x7f954c132550 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.364+0000 7f9551f7c700 1 --2- 172.21.15.3:0/1816010760 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f954c127f20 0x7f954c12fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.364+0000 7f9551f7c700 1 -- 172.21.15.3:0/1816010760 >> 172.21.15.3:0/1816010760 conn(0x7f954c059c10 msgr2=0x7f954c099790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:54.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.364+0000 7f878d8b1700 1 -- 172.21.15.3:0/1536862999 shutdown_connections 2023-10-03T10:09:54.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.364+0000 7f878d8b1700 1 -- 172.21.15.3:0/1536862999 wait complete. 2023-10-03T10:09:54.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.364+0000 7f2c04917700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c00068e70 0x7f2c00112a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.364+0000 7f2c04917700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c00092240 0x7f2c00112f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.364+0000 7f2c04917700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2c00092680 0x7f2c00113470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.364+0000 7f2c04917700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c00113960 con 0x7f2c00092240 2023-10-03T10:09:54.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.364+0000 7f2c04917700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c001146b0 con 0x7f2c00092680 2023-10-03T10:09:54.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.364+0000 7f2c04917700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c00113ef0 con 0x7f2c00068e70 2023-10-03T10:09:54.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.364+0000 7f2bfed9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2c00092680 0x7f2c00113470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.364+0000 7f2bfed9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2c00092680 0x7f2c00113470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39004/0 (socket says 172.21.15.3:39004) 2023-10-03T10:09:54.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.364+0000 7f2bfed9d700 1 -- 172.21.15.3:0/1002855658 learned_addr learned my addr 172.21.15.3:0/1002855658 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:54.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.364+0000 7f2bfdd9b700 1 --2- 172.21.15.3:0/1002855658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c00092240 0x7f2c00112f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.365+0000 7f2bfe59c700 1 --2- 172.21.15.3:0/1002855658 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c00068e70 0x7f2c00112a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.365+0000 7f2bfed9d700 1 -- 172.21.15.3:0/1002855658 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c00068e70 msgr2=0x7f2c00112a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.365+0000 7f2bfed9d700 1 --2- 172.21.15.3:0/1002855658 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c00068e70 0x7f2c00112a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.365+0000 7f2bfed9d700 1 -- 172.21.15.3:0/1002855658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c00092240 msgr2=0x7f2c00112f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.365+0000 7f2bfed9d700 1 --2- 172.21.15.3:0/1002855658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c00092240 0x7f2c00112f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.365+0000 7f2bfed9d700 1 -- 172.21.15.3:0/1002855658 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf000b6a0 con 0x7f2c00092680 2023-10-03T10:09:54.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.365+0000 7f2bfed9d700 1 --2- 172.21.15.3:0/1002855658 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2c00092680 0x7f2c00113470 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f2bf800c8b0 tx=0x7f2bf80016c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:54.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.364+0000 7f9551f7c700 1 -- 172.21.15.3:0/1816010760 shutdown_connections 2023-10-03T10:09:54.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.364+0000 7f9551f7c700 1 -- 172.21.15.3:0/1816010760 wait complete. 2023-10-03T10:09:54.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.365+0000 7fd81b7fe700 1 -- 172.21.15.3:0/3016902769 <== osd.3 v1:172.21.15.3:6800/118836 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7fd8200384e0 con 0x7fd8200347f0 2023-10-03T10:09:54.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.365+0000 7f878d8b1700 1 Processor -- start 2023-10-03T10:09:54.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.365+0000 7f878d8b1700 1 -- start start 2023-10-03T10:09:54.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.365+0000 7f9551f7c700 1 Processor -- start 2023-10-03T10:09:54.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.366+0000 7f878d8b1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8788091a40 0x7f8788110810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.366+0000 7f878d8b1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8788094550 0x7f8788110d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.366+0000 7f9551f7c700 1 -- start start 2023-10-03T10:09:54.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.366+0000 7f9551f7c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f954c069870 0x7f954c12ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.366+0000 7f9551f7c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f954c127f20 0x7f954c12f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.366+0000 7f9551f7c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f954c130160 0x7f954c12b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.366+0000 7f9551f7c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c095f10 con 0x7f954c130160 2023-10-03T10:09:54.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.366+0000 7f9551f7c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954c12ddf0 con 0x7f954c069870 2023-10-03T10:09:54.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.366+0000 7f9551f7c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f954c12e0b0 con 0x7f954c127f20 2023-10-03T10:09:54.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.366+0000 7f954b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f954c069870 0x7f954c12ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.366+0000 7f954b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f954c069870 0x7f954c12ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39014/0 (socket says 172.21.15.3:39014) 2023-10-03T10:09:54.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.366+0000 7f954b7fe700 1 -- 172.21.15.3:0/1854124368 learned_addr learned my addr 172.21.15.3:0/1854124368 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:54.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.367+0000 7f9542ffd700 1 --2- 172.21.15.3:0/1854124368 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f954c127f20 0x7f954c12f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.366+0000 7f878d8b1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8788124160 0x7f87881111f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.366+0000 7f878d8b1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87881116e0 con 0x7f8788091a40 2023-10-03T10:09:54.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.366+0000 7f878d8b1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8788068780 con 0x7f8788094550 2023-10-03T10:09:54.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.366+0000 7f878d8b1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f87880931f0 con 0x7f8788124160 2023-10-03T10:09:54.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.366+0000 7f87867fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8788094550 0x7f8788110d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.366+0000 7f87867fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8788094550 0x7f8788110d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39012/0 (socket says 172.21.15.3:39012) 2023-10-03T10:09:54.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.366+0000 7f87867fc700 1 -- 172.21.15.3:0/2137579659 learned_addr learned my addr 172.21.15.3:0/2137579659 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:54.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.367+0000 7f87877fe700 1 --2- 172.21.15.3:0/2137579659 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8788124160 0x7f87881111f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.367+0000 7f8786ffd700 1 --2- 172.21.15.3:0/2137579659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8788091a40 0x7f8788110810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.367+0000 7f87867fc700 1 -- 172.21.15.3:0/2137579659 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8788124160 msgr2=0x7f87881111f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.367+0000 7f87867fc700 1 --2- 172.21.15.3:0/2137579659 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8788124160 0x7f87881111f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.367+0000 7f87867fc700 1 -- 172.21.15.3:0/2137579659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8788091a40 msgr2=0x7f8788110810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.367+0000 7f87867fc700 1 --2- 172.21.15.3:0/2137579659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8788091a40 0x7f8788110810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.367+0000 7f87867fc700 1 -- 172.21.15.3:0/2137579659 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8778009370 con 0x7f8788094550 2023-10-03T10:09:54.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.367+0000 7f87867fc700 1 --2- 172.21.15.3:0/2137579659 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8788094550 0x7f8788110d00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f877c00d000 tx=0x7f877c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:54.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.367+0000 7f954bfff700 1 --2- 172.21.15.3:0/1854124368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f954c130160 0x7f954c12b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.367+0000 7f954b7fe700 1 -- 172.21.15.3:0/1854124368 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f954c127f20 msgr2=0x7f954c12f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.367+0000 7f954b7fe700 1 --2- 172.21.15.3:0/1854124368 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f954c127f20 0x7f954c12f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.367+0000 7f954b7fe700 1 -- 172.21.15.3:0/1854124368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f954c130160 msgr2=0x7f954c12b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.367+0000 7f954b7fe700 1 --2- 172.21.15.3:0/1854124368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f954c130160 0x7f954c12b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.367+0000 7f954b7fe700 1 -- 172.21.15.3:0/1854124368 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9544009a20 con 0x7f954c069870 2023-10-03T10:09:54.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.367+0000 7f9542ffd700 1 --2- 172.21.15.3:0/1854124368 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f954c127f20 0x7f954c12f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:09:54.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.367+0000 7f2bee7fc700 1 -- 172.21.15.3:0/1002855658 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf800a580 con 0x7f2c00092680 2023-10-03T10:09:54.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.367+0000 7f954bfff700 1 --2- 172.21.15.3:0/1854124368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f954c130160 0x7f954c12b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:09:54.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.367+0000 7f954b7fe700 1 --2- 172.21.15.3:0/1854124368 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f954c069870 0x7f954c12ef70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f953c001eb0 tx=0x7f953c007e50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.367+0000 7f2bee7fc700 1 -- 172.21.15.3:0/1002855658 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf800a6e0 con 0x7f2c00092680 2023-10-03T10:09:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.367+0000 7f2bee7fc700 1 -- 172.21.15.3:0/1002855658 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf800aaa0 con 0x7f2c00092680 2023-10-03T10:09:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.367+0000 7f9543fff700 1 -- 172.21.15.3:0/1854124368 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c0072e0 con 0x7f954c069870 2023-10-03T10:09:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.368+0000 7f9543fff700 1 -- 172.21.15.3:0/1854124368 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953c007440 con 0x7f954c069870 2023-10-03T10:09:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.368+0000 7f9543fff700 1 -- 172.21.15.3:0/1854124368 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c0057a0 con 0x7f954c069870 2023-10-03T10:09:54.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.367+0000 7f2c04917700 1 -- 172.21.15.3:0/1002855658 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c00116020 con 0x7f2c00092680 2023-10-03T10:09:54.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.367+0000 7f2c04917700 1 -- 172.21.15.3:0/1002855658 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c00092e20 con 0x7f2c00092680 2023-10-03T10:09:54.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.368+0000 7f9551f7c700 1 -- 172.21.15.3:0/1854124368 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f954c12db40 con 0x7f954c069870 2023-10-03T10:09:54.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.368+0000 7f9551f7c700 1 -- 172.21.15.3:0/1854124368 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f954c12f950 con 0x7f954c069870 2023-10-03T10:09:54.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.368+0000 7f2c04917700 1 -- 172.21.15.3:0/1002855658 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2c0005a260 con 0x7f2c00092680 2023-10-03T10:09:54.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.369+0000 7f9543fff700 1 -- 172.21.15.3:0/1854124368 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f953c005b10 con 0x7f954c069870 2023-10-03T10:09:54.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.369+0000 7f9543fff700 1 -- 172.21.15.3:0/1854124368 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f953c03a730 con 0x7f954c069870 2023-10-03T10:09:54.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.369+0000 7f2bee7fc700 1 -- 172.21.15.3:0/1002855658 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f2bf801b400 con 0x7f2c00092680 2023-10-03T10:09:54.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.369+0000 7f9551f7c700 1 -- 172.21.15.3:0/1854124368 --> v1:172.21.15.3:6804/118835 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f95300053b0 con 0x7f95300015c0 2023-10-03T10:09:54.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.369+0000 7f2bee7fc700 1 -- 172.21.15.3:0/1002855658 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f2bf803afe0 con 0x7f2c00092680 2023-10-03T10:09:54.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.370+0000 7f876effd700 1 -- 172.21.15.3:0/2137579659 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c009a60 con 0x7f8788094550 2023-10-03T10:09:54.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.370+0000 7f876effd700 1 -- 172.21.15.3:0/2137579659 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877c000cb0 con 0x7f8788094550 2023-10-03T10:09:54.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.370+0000 7f876effd700 1 -- 172.21.15.3:0/2137579659 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c01c640 con 0x7f8788094550 2023-10-03T10:09:54.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.370+0000 7f2bee7fc700 1 -- 172.21.15.3:0/1002855658 --> v1:172.21.15.3:6808/118845 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2be0038630 con 0x7f2be0034940 2023-10-03T10:09:54.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.370+0000 7f2bee7fc700 1 -- 172.21.15.3:0/1002855658 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bf803d580 con 0x7f2c00092680 2023-10-03T10:09:54.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.370+0000 7f878d8b1700 1 -- 172.21.15.3:0/2137579659 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f877c00d410 con 0x7f8788094550 2023-10-03T10:09:54.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.370+0000 7f878d8b1700 1 -- 172.21.15.3:0/2137579659 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8788113e10 con 0x7f8788094550 2023-10-03T10:09:54.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.371+0000 7f878d8b1700 1 -- 172.21.15.3:0/2137579659 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f878805a260 con 0x7f8788094550 2023-10-03T10:09:54.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.371+0000 7f876effd700 1 -- 172.21.15.3:0/2137579659 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f877c01ca30 con 0x7f8788094550 2023-10-03T10:09:54.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.371+0000 7f2bee7fc700 1 -- 172.21.15.3:0/1002855658 <== osd.0 v1:172.21.15.3:6808/118845 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7f2be0038630 con 0x7f2be0034940 2023-10-03T10:09:54.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.371+0000 7f9543fff700 1 -- 172.21.15.3:0/1854124368 <== osd.1 v1:172.21.15.3:6804/118835 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7f95300053b0 con 0x7f95300015c0 2023-10-03T10:09:54.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.371+0000 7f876effd700 1 -- 172.21.15.3:0/2137579659 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f877c0463a0 con 0x7f8788094550 2023-10-03T10:09:54.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.372+0000 7f876effd700 1 -- 172.21.15.3:0/2137579659 --> v1:172.21.15.3:6812/118864 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f87740385b0 con 0x7f87740348c0 2023-10-03T10:09:54.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.372+0000 7f876effd700 1 -- 172.21.15.3:0/2137579659 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f877c046640 con 0x7f8788094550 2023-10-03T10:09:54.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.373+0000 7f876effd700 1 -- 172.21.15.3:0/2137579659 <== osd.2 v1:172.21.15.3:6812/118864 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7f87740385b0 con 0x7f87740348c0 2023-10-03T10:09:54.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.383+0000 7fd839db8700 1 -- 172.21.15.3:0/3016902769 --> v1:172.21.15.3:6800/118836 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd83411eec0 con 0x7fd8200347f0 2023-10-03T10:09:54.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.384+0000 7fd81b7fe700 1 -- 172.21.15.3:0/3016902769 <== osd.3 v1:172.21.15.3:6800/118836 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 675830894) 0x7fd83411eec0 con 0x7fd8200347f0 2023-10-03T10:09:54.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.384+0000 7fd839db8700 1 -- 172.21.15.3:0/3016902769 >> v1:172.21.15.3:6800/118836 conn(0x7fd8200347f0 legacy=0x7fd820036c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.384+0000 7fd839db8700 1 -- 172.21.15.3:0/3016902769 >> v1:172.21.15.3:6816/118296 conn(0x7fd82002bb60 legacy=0x7fd82002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.384+0000 7fd839db8700 1 -- 172.21.15.3:0/3016902769 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd834098820 msgr2=0x7fd83411b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.384+0000 7fd839db8700 1 --2- 172.21.15.3:0/3016902769 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd834098820 0x7fd83411b450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fd828009e30 tx=0x7fd828017000).stop 2023-10-03T10:09:54.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.384+0000 7fd833fff700 1 -- 172.21.15.3:0/3016902769 reap_dead start 2023-10-03T10:09:54.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.384+0000 7fd839db8700 1 -- 172.21.15.3:0/3016902769 shutdown_connections 2023-10-03T10:09:54.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.384+0000 7fd839db8700 1 -- 172.21.15.3:0/3016902769 >> 172.21.15.3:0/3016902769 conn(0x7fd834097460 msgr2=0x7fd834096c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:54.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.384+0000 7fd839db8700 1 -- 172.21.15.3:0/3016902769 shutdown_connections 2023-10-03T10:09:54.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.384+0000 7fd839db8700 1 -- 172.21.15.3:0/3016902769 wait complete. 2023-10-03T10:09:54.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.388+0000 7f9551f7c700 1 -- 172.21.15.3:0/1854124368 --> v1:172.21.15.3:6804/118835 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9530006f20 con 0x7f95300015c0 2023-10-03T10:09:54.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.388+0000 7f9543fff700 1 -- 172.21.15.3:0/1854124368 <== osd.1 v1:172.21.15.3:6804/118835 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 675830894) 0x7f9530006f20 con 0x7f95300015c0 2023-10-03T10:09:54.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.388+0000 7f9551f7c700 1 -- 172.21.15.3:0/1854124368 >> v1:172.21.15.3:6804/118835 conn(0x7f95300015c0 legacy=0x7f9530003a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.389+0000 7f9551f7c700 1 -- 172.21.15.3:0/1854124368 >> v1:172.21.15.3:6816/118296 conn(0x7f952c02bb60 legacy=0x7f952c02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.389+0000 7f9551f7c700 1 -- 172.21.15.3:0/1854124368 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f954c069870 msgr2=0x7f954c12ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.389+0000 7f9551f7c700 1 --2- 172.21.15.3:0/1854124368 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f954c069870 0x7f954c12ef70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f953c001eb0 tx=0x7f953c007e50).stop 2023-10-03T10:09:54.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.389+0000 7f954bfff700 1 -- 172.21.15.3:0/1854124368 reap_dead start 2023-10-03T10:09:54.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.389+0000 7f9551f7c700 1 -- 172.21.15.3:0/1854124368 shutdown_connections 2023-10-03T10:09:54.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.389+0000 7f9551f7c700 1 -- 172.21.15.3:0/1854124368 >> 172.21.15.3:0/1854124368 conn(0x7f954c059c10 msgr2=0x7f954c092e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:54.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.389+0000 7f9551f7c700 1 -- 172.21.15.3:0/1854124368 shutdown_connections 2023-10-03T10:09:54.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.389+0000 7f9551f7c700 1 -- 172.21.15.3:0/1854124368 wait complete. 2023-10-03T10:09:54.401 INFO:teuthology.orchestra.run.smithi003.stdout:38654705669 2023-10-03T10:09:54.401 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:09:54.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.402+0000 7f2c04917700 1 -- 172.21.15.3:0/1002855658 --> v1:172.21.15.3:6808/118845 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2c00116520 con 0x7f2be0034940 2023-10-03T10:09:54.403 INFO:teuthology.orchestra.run.smithi003.stdout:38654705669 2023-10-03T10:09:54.403 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:09:54.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.402+0000 7f2bee7fc700 1 -- 172.21.15.3:0/1002855658 <== osd.0 v1:172.21.15.3:6808/118845 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 675830894) 0x7f2bf4000fe0 con 0x7f2be0034940 2023-10-03T10:09:54.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.403+0000 7f2c04917700 1 -- 172.21.15.3:0/1002855658 >> v1:172.21.15.3:6808/118845 conn(0x7f2be0034940 legacy=0x7f2be0036dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.403+0000 7f2c04917700 1 -- 172.21.15.3:0/1002855658 >> v1:172.21.15.3:6816/118296 conn(0x7f2be002bbb0 legacy=0x7f2be002e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.403+0000 7f2c04917700 1 -- 172.21.15.3:0/1002855658 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2c00092680 msgr2=0x7f2c00113470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.403+0000 7f2c04917700 1 --2- 172.21.15.3:0/1002855658 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2c00092680 0x7f2c00113470 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f2bf800c8b0 tx=0x7f2bf80016c0).stop 2023-10-03T10:09:54.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.403+0000 7f2bfed9d700 1 -- 172.21.15.3:0/1002855658 reap_dead start 2023-10-03T10:09:54.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.403+0000 7f2c04917700 1 -- 172.21.15.3:0/1002855658 shutdown_connections 2023-10-03T10:09:54.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.403+0000 7f2c04917700 1 -- 172.21.15.3:0/1002855658 >> 172.21.15.3:0/1002855658 conn(0x7f2c00063560 msgr2=0x7f2c00067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:54.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.403+0000 7f2c04917700 1 -- 172.21.15.3:0/1002855658 shutdown_connections 2023-10-03T10:09:54.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.403+0000 7f2c04917700 1 -- 172.21.15.3:0/1002855658 wait complete. 2023-10-03T10:09:54.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.404+0000 7f878d8b1700 1 -- 172.21.15.3:0/2137579659 --> v1:172.21.15.3:6812/118864 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8788068e50 con 0x7f87740348c0 2023-10-03T10:09:54.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.404+0000 7f876effd700 1 -- 172.21.15.3:0/2137579659 <== osd.2 v1:172.21.15.3:6812/118864 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 675830894) 0x7f8788068e50 con 0x7f87740348c0 2023-10-03T10:09:54.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.405+0000 7f878d8b1700 1 -- 172.21.15.3:0/2137579659 >> v1:172.21.15.3:6812/118864 conn(0x7f87740348c0 legacy=0x7f8774036d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.405+0000 7f878d8b1700 1 -- 172.21.15.3:0/2137579659 >> v1:172.21.15.3:6816/118296 conn(0x7f877402bbb0 legacy=0x7f877402e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.405+0000 7f878d8b1700 1 -- 172.21.15.3:0/2137579659 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8788094550 msgr2=0x7f8788110d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.405+0000 7f878d8b1700 1 --2- 172.21.15.3:0/2137579659 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8788094550 0x7f8788110d00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f877c00d000 tx=0x7f877c016000).stop 2023-10-03T10:09:54.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.405+0000 7f87877fe700 1 -- 172.21.15.3:0/2137579659 reap_dead start 2023-10-03T10:09:54.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.405+0000 7f878d8b1700 1 -- 172.21.15.3:0/2137579659 shutdown_connections 2023-10-03T10:09:54.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.405+0000 7f878d8b1700 1 -- 172.21.15.3:0/2137579659 >> 172.21.15.3:0/2137579659 conn(0x7f878809a1b0 msgr2=0x7f8788122880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:54.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.405+0000 7f878d8b1700 1 -- 172.21.15.3:0/2137579659 shutdown_connections 2023-10-03T10:09:54.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.405+0000 7f878d8b1700 1 -- 172.21.15.3:0/2137579659 wait complete. 2023-10-03T10:09:54.425 INFO:teuthology.orchestra.run.smithi003.stdout:38654705669 2023-10-03T10:09:54.425 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:09:54.427 INFO:teuthology.orchestra.run.smithi003.stdout:38654705669 2023-10-03T10:09:54.428 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:09:54.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.545+0000 7f8c46981700 1 Processor -- start 2023-10-03T10:09:54.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.545+0000 7f8c46981700 1 -- start start 2023-10-03T10:09:54.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.545+0000 7f8c46981700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8c40098820 0x7f8c40098c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.545+0000 7f8c46981700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c400990f0 0x7f8c400994d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.545+0000 7f8c46981700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c40092da0 con 0x7f8c400999c0 2023-10-03T10:09:54.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.545+0000 7f8c46981700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c4009a4d0 con 0x7f8c40098820 2023-10-03T10:09:54.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.545+0000 7f8c46981700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c40094a40 con 0x7f8c400990f0 2023-10-03T10:09:54.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.545+0000 7f8c3f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c400990f0 0x7f8c400994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.546+0000 7f8c3f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c400990f0 0x7f8c400994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39026/0 (socket says 172.21.15.3:39026) 2023-10-03T10:09:54.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.546+0000 7f8c3f7fe700 1 -- 172.21.15.3:0/667085858 learned_addr learned my addr 172.21.15.3:0/667085858 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:54.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.546+0000 7f8c3ffff700 1 --2- 172.21.15.3:0/667085858 >> v2:172.21.15.3:3300/0 conn(0x7f8c40098820 0x7f8c40098c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.546+0000 7f8c3ffff700 1 -- 172.21.15.3:0/667085858 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c400990f0 msgr2=0x7f8c400994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:54.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.546+0000 7f8c3ffff700 1 --2- 172.21.15.3:0/667085858 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c400990f0 0x7f8c400994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.546+0000 7f8c3ffff700 1 -- 172.21.15.3:0/667085858 >> v1:172.21.15.3:6789/0 conn(0x7f8c400999c0 legacy=0x7f8c4012e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.546+0000 7f8c3ffff700 1 -- 172.21.15.3:0/667085858 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c40091c40 con 0x7f8c40098820 2023-10-03T10:09:54.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.546+0000 7f8c3ffff700 1 --2- 172.21.15.3:0/667085858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c40098820 0x7f8c40098c00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f8c30009760 tx=0x7f8c30009060).ready entity=mon.0 client_cookie=810dae38ce1d95c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:54.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.546+0000 7f8c3effd700 1 -- 172.21.15.3:0/667085858 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c300090d0 con 0x7f8c40098820 2023-10-03T10:09:54.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.547+0000 7f8c3effd700 1 -- 172.21.15.3:0/667085858 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3000f510 con 0x7f8c40098820 2023-10-03T10:09:54.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.547+0000 7f8c3effd700 1 -- 172.21.15.3:0/667085858 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3000f890 con 0x7f8c40098820 2023-10-03T10:09:54.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.547+0000 7f8c46981700 1 -- 172.21.15.3:0/667085858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c40098820 msgr2=0x7f8c40098c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.547+0000 7f8c46981700 1 --2- 172.21.15.3:0/667085858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c40098820 0x7f8c40098c00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f8c30009760 tx=0x7f8c30009060).stop 2023-10-03T10:09:54.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.547+0000 7f8c46981700 1 -- 172.21.15.3:0/667085858 shutdown_connections 2023-10-03T10:09:54.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.547+0000 7f8c46981700 1 --2- 172.21.15.3:0/667085858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c40098820 0x7f8c40098c00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.547+0000 7f8c46981700 1 --2- 172.21.15.3:0/667085858 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c400990f0 0x7f8c400994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.547+0000 7f8c46981700 1 -- 172.21.15.3:0/667085858 >> 172.21.15.3:0/667085858 conn(0x7f8c40097460 msgr2=0x7f8c40097870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:54.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.547+0000 7f8c46981700 1 -- 172.21.15.3:0/667085858 shutdown_connections 2023-10-03T10:09:54.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.547+0000 7f8c46981700 1 -- 172.21.15.3:0/667085858 wait complete. 2023-10-03T10:09:54.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.549+0000 7f8c46981700 1 Processor -- start 2023-10-03T10:09:54.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.549+0000 7f1a2a50c700 1 Processor -- start 2023-10-03T10:09:54.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.549+0000 7f1a2a50c700 1 -- start start 2023-10-03T10:09:54.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.549+0000 7f8c46981700 1 -- start start 2023-10-03T10:09:54.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.549+0000 7f1a2a50c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a24093780 0x7f1a24096310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.550+0000 7f1a2a50c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1a24125680 0x7f1a24125aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.550+0000 7f1a2a50c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a24125f90 0x7f1a24126440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.550+0000 7f1a2a50c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a24091b60 con 0x7f1a24125680 2023-10-03T10:09:54.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.550+0000 7f1a2a50c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a24093b60 con 0x7f1a24093780 2023-10-03T10:09:54.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.550+0000 7f1a2a50c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a24127150 con 0x7f1a24125f90 2023-10-03T10:09:54.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.550+0000 7f8c46981700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c40098820 0x7f8c40135bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.550+0000 7f8c46981700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c400990f0 0x7f8c401360e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.550+0000 7f8c46981700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c400999c0 0x7f8c401365d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.550+0000 7f8c46981700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c40119960 con 0x7f8c40098820 2023-10-03T10:09:54.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.550+0000 7f8c46981700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c40092320 con 0x7f8c400999c0 2023-10-03T10:09:54.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.550+0000 7f8c46981700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c40096510 con 0x7f8c400990f0 2023-10-03T10:09:54.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.550+0000 7f8c44f1e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c400999c0 0x7f8c401365d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.550+0000 7f1a237fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1a24125680 0x7f1a24125aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.550+0000 7f8c44f1e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c400999c0 0x7f8c401365d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39050/0 (socket says 172.21.15.3:39050) 2023-10-03T10:09:54.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.550+0000 7f8c44f1e700 1 -- 172.21.15.3:0/763890501 learned_addr learned my addr 172.21.15.3:0/763890501 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:54.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.550+0000 7f1a237fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1a24125680 0x7f1a24125aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47666/0 (socket says 172.21.15.3:47666) 2023-10-03T10:09:54.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.550+0000 7f1a237fe700 1 -- 172.21.15.3:0/4118447356 learned_addr learned my addr 172.21.15.3:0/4118447356 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:54.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.550+0000 7f8c3ffff700 1 --2- 172.21.15.3:0/763890501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c40098820 0x7f8c40135bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.550+0000 7f8c44f1e700 1 -- 172.21.15.3:0/763890501 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c400990f0 msgr2=0x7f8c401360e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.550+0000 7f1a237fe700 1 -- 172.21.15.3:0/4118447356 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a24125f90 msgr2=0x7f1a24126440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:54.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.550+0000 7f1a237fe700 1 --2- 172.21.15.3:0/4118447356 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a24125f90 0x7f1a24126440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-03T10:09:54.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.550+0000 7f1a237fe700 1 -- 172.21.15.3:0/4118447356 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a24093780 msgr2=0x7f1a24096310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:54.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.550+0000 7f1a237fe700 1 --2- 172.21.15.3:0/4118447356 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a24093780 0x7f1a24096310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-03T10:09:54.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.550+0000 7f1a237fe700 1 -- 172.21.15.3:0/4118447356 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a24092cc0 con 0x7f1a24125680 2023-10-03T10:09:54.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.550+0000 7f8c3f7fe700 1 --2- 172.21.15.3:0/763890501 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c400990f0 0x7f8c401360e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.551+0000 7f8c44f1e700 1 --2- 172.21.15.3:0/763890501 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c400990f0 0x7f8c401360e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.551+0000 7f8c44f1e700 1 -- 172.21.15.3:0/763890501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c40098820 msgr2=0x7f8c40135bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.551+0000 7f8c44f1e700 1 --2- 172.21.15.3:0/763890501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c40098820 0x7f8c40135bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.551+0000 7f8c44f1e700 1 -- 172.21.15.3:0/763890501 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c300092a0 con 0x7f8c400999c0 2023-10-03T10:09:54.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.551+0000 7f1a237fe700 1 --2- 172.21.15.3:0/4118447356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a24125680 0x7f1a24125aa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f1a18006fd0 tx=0x7f1a180090f0).ready entity=mon.0 client_cookie=7efa3a274018d069 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:54.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.551+0000 7f8c3ffff700 1 --2- 172.21.15.3:0/763890501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c40098820 0x7f8c40135bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:54.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.551+0000 7f8c3f7fe700 1 --2- 172.21.15.3:0/763890501 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c400990f0 0x7f8c401360e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:09:54.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.551+0000 7f1a22ffd700 1 -- 172.21.15.3:0/4118447356 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1800b600 con 0x7f1a24125680 2023-10-03T10:09:54.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.551+0000 7f1a22ffd700 1 -- 172.21.15.3:0/4118447356 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1800b760 con 0x7f1a24125680 2023-10-03T10:09:54.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.551+0000 7f1a22ffd700 1 -- 172.21.15.3:0/4118447356 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1800bb00 con 0x7f1a24125680 2023-10-03T10:09:54.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.551+0000 7f8c44f1e700 1 --2- 172.21.15.3:0/763890501 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c400999c0 0x7f8c401365d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f8c3800c7f0 tx=0x7f8c38001530).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:54.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.551+0000 7f8c27fff700 1 -- 172.21.15.3:0/763890501 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38010510 con 0x7f8c400999c0 2023-10-03T10:09:54.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.551+0000 7f8c27fff700 1 -- 172.21.15.3:0/763890501 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c38010670 con 0x7f8c400999c0 2023-10-03T10:09:54.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.551+0000 7f8c46981700 1 -- 172.21.15.3:0/763890501 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c401183f0 con 0x7f8c400999c0 2023-10-03T10:09:54.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.551+0000 7f8c27fff700 1 -- 172.21.15.3:0/763890501 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c380109f0 con 0x7f8c400999c0 2023-10-03T10:09:54.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.551+0000 7f8c46981700 1 -- 172.21.15.3:0/763890501 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c40117a00 con 0x7f8c400999c0 2023-10-03T10:09:54.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.552+0000 7f1a2a50c700 1 -- 172.21.15.3:0/4118447356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a24125680 msgr2=0x7f1a24125aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.552+0000 7f1a2a50c700 1 --2- 172.21.15.3:0/4118447356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a24125680 0x7f1a24125aa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f1a18006fd0 tx=0x7f1a180090f0).stop 2023-10-03T10:09:54.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.552+0000 7f1a2a50c700 1 -- 172.21.15.3:0/4118447356 shutdown_connections 2023-10-03T10:09:54.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.552+0000 7f1a2a50c700 1 --2- 172.21.15.3:0/4118447356 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a24125f90 0x7f1a24126440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-03T10:09:54.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.552+0000 7f1a2a50c700 1 --2- 172.21.15.3:0/4118447356 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a24093780 0x7f1a24096310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-03T10:09:54.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.552+0000 7f1a2a50c700 1 --2- 172.21.15.3:0/4118447356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a24125680 0x7f1a24125aa0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.552+0000 7f1a2a50c700 1 -- 172.21.15.3:0/4118447356 >> 172.21.15.3:0/4118447356 conn(0x7f1a2409a8e0 msgr2=0x7f1a2409acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:54.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.552+0000 7f8c27fff700 1 -- 172.21.15.3:0/763890501 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f8c3801b3b0 con 0x7f8c400999c0 2023-10-03T10:09:54.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.552+0000 7f8c27fff700 1 -- 172.21.15.3:0/763890501 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f8c3803a9b0 con 0x7f8c400999c0 2023-10-03T10:09:54.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.553+0000 7f1a2a50c700 1 -- 172.21.15.3:0/4118447356 shutdown_connections 2023-10-03T10:09:54.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.553+0000 7f1a2a50c700 1 -- 172.21.15.3:0/4118447356 wait complete. 2023-10-03T10:09:54.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.553+0000 7f8c46981700 1 -- 172.21.15.3:0/763890501 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c40119570 con 0x7f8c400999c0 2023-10-03T10:09:54.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.554+0000 7f1a2a50c700 1 Processor -- start 2023-10-03T10:09:54.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.554+0000 7f1a2a50c700 1 -- start start 2023-10-03T10:09:54.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.554+0000 7f1a2a50c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a24093780 0x7f1a24062a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.554+0000 7f1a2a50c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a24125680 0x7f1a24123a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.554+0000 7f1a2a50c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a24125f90 0x7f1a24123f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.554+0000 7f1a2a50c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a24128160 con 0x7f1a24125f90 2023-10-03T10:09:54.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.554+0000 7f1a2a50c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a24092470 con 0x7f1a24093780 2023-10-03T10:09:54.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.554+0000 7f1a2a50c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2411a3d0 con 0x7f1a24125680 2023-10-03T10:09:54.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.555+0000 7f1a23fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a24093780 0x7f1a24062a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.555+0000 7f1a23fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a24093780 0x7f1a24062a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39064/0 (socket says 172.21.15.3:39064) 2023-10-03T10:09:54.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.555+0000 7f1a23fff700 1 -- 172.21.15.3:0/2280636591 learned_addr learned my addr 172.21.15.3:0/2280636591 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:54.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.555+0000 7f1a237fe700 1 --2- 172.21.15.3:0/2280636591 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a24125680 0x7f1a24123a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.555+0000 7f1a23fff700 1 -- 172.21.15.3:0/2280636591 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a24125680 msgr2=0x7f1a24123a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.555+0000 7f1a23fff700 1 --2- 172.21.15.3:0/2280636591 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a24125680 0x7f1a24123a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.555+0000 7f1a23fff700 1 -- 172.21.15.3:0/2280636591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a24125f90 msgr2=0x7f1a24123f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.555+0000 7f1a23fff700 1 --2- 172.21.15.3:0/2280636591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a24125f90 0x7f1a24123f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-10-03T10:09:54.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.555+0000 7f1a23fff700 1 -- 172.21.15.3:0/2280636591 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a18009370 con 0x7f1a24093780 2023-10-03T10:09:54.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.555+0000 7f1a23fff700 1 --2- 172.21.15.3:0/2280636591 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a24093780 0x7f1a24062a90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f1a1400b560 tx=0x7f1a140090f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:54.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.562+0000 7f1a0bfff700 1 -- 172.21.15.3:0/2280636591 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a14002320 con 0x7f1a24093780 2023-10-03T10:09:54.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.562+0000 7f1a0bfff700 1 -- 172.21.15.3:0/2280636591 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a140024a0 con 0x7f1a24093780 2023-10-03T10:09:54.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.562+0000 7f1a2a50c700 1 -- 172.21.15.3:0/2280636591 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a240635b0 con 0x7f1a24093780 2023-10-03T10:09:54.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.562+0000 7f1a0bfff700 1 -- 172.21.15.3:0/2280636591 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a14002860 con 0x7f1a24093780 2023-10-03T10:09:54.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.562+0000 7f1a2a50c700 1 -- 172.21.15.3:0/2280636591 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a241245d0 con 0x7f1a24093780 2023-10-03T10:09:54.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.562+0000 7f8c27fff700 1 -- 172.21.15.3:0/763890501 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f8c3801b660 con 0x7f8c400999c0 2023-10-03T10:09:54.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.563+0000 7f1a0bfff700 1 -- 172.21.15.3:0/2280636591 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f1a14002bd0 con 0x7f1a24093780 2023-10-03T10:09:54.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.563+0000 7f1a2a50c700 1 -- 172.21.15.3:0/2280636591 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a2405a260 con 0x7f1a24093780 2023-10-03T10:09:54.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.563+0000 7f1a0bfff700 1 -- 172.21.15.3:0/2280636591 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f1a1403adc0 con 0x7f1a24093780 2023-10-03T10:09:54.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.568+0000 7f1a0bfff700 1 -- 172.21.15.3:0/2280636591 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f1a1401b3e0 con 0x7f1a24093780 2023-10-03T10:09:54.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.617+0000 7f596df16700 1 Processor -- start 2023-10-03T10:09:54.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.617+0000 7f596df16700 1 -- start start 2023-10-03T10:09:54.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.617+0000 7f596df16700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5968099310 0x7f59680996f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.617+0000 7f596df16700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f596812d100 0x7f596812f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.617+0000 7f596df16700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5968092da0 con 0x7f5968098820 2023-10-03T10:09:54.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.617+0000 7f596df16700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f596809a4d0 con 0x7f5968099310 2023-10-03T10:09:54.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.617+0000 7f596df16700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5968094a40 con 0x7f596812d100 2023-10-03T10:09:54.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.618+0000 7f59677fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5968098820 0x7f5968098c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38498/0 (socket says 172.21.15.3:38498) 2023-10-03T10:09:54.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.618+0000 7f5966ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5968099310 0x7f59680996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.618+0000 7f59677fe700 1 -- 172.21.15.3:0/2676207473 learned_addr learned my addr 172.21.15.3:0/2676207473 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:09:54.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.618+0000 7f5967fff700 1 --2- 172.21.15.3:0/2676207473 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f596812d100 0x7f596812f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.618+0000 7f59667fc700 1 -- 172.21.15.3:0/2676207473 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1713560793 0 0) 0x7f5968092da0 con 0x7f5968098820 2023-10-03T10:09:54.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.618+0000 7f5967fff700 1 -- 172.21.15.3:0/2676207473 >> v2:172.21.15.3:3300/0 conn(0x7f5968099310 msgr2=0x7f59680996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:54.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.618+0000 7f5967fff700 1 --2- 172.21.15.3:0/2676207473 >> v2:172.21.15.3:3300/0 conn(0x7f5968099310 0x7f59680996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.618+0000 7f5967fff700 1 -- 172.21.15.3:0/2676207473 >> v1:172.21.15.3:6789/0 conn(0x7f5968098820 legacy=0x7f5968098c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.618+0000 7f5967fff700 1 -- 172.21.15.3:0/2676207473 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5968091c40 con 0x7f596812d100 2023-10-03T10:09:54.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.618+0000 7f5967fff700 1 --2- 172.21.15.3:0/2676207473 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f596812d100 0x7f596812f4f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f5960009e60 tx=0x7f5960009780).ready entity=mon.1 client_cookie=a64d4cd620ac6fa2 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:54.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.618+0000 7f59667fc700 1 -- 172.21.15.3:0/2676207473 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596000e070 con 0x7f596812d100 2023-10-03T10:09:54.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.618+0000 7f59667fc700 1 -- 172.21.15.3:0/2676207473 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596000bb60 con 0x7f596812d100 2023-10-03T10:09:54.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.618+0000 7f59667fc700 1 -- 172.21.15.3:0/2676207473 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5960012410 con 0x7f596812d100 2023-10-03T10:09:54.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.619+0000 7f596df16700 1 -- 172.21.15.3:0/2676207473 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f596812d100 msgr2=0x7f596812f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.619+0000 7f596df16700 1 --2- 172.21.15.3:0/2676207473 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f596812d100 0x7f596812f4f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f5960009e60 tx=0x7f5960009780).stop 2023-10-03T10:09:54.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.619+0000 7f596df16700 1 -- 172.21.15.3:0/2676207473 shutdown_connections 2023-10-03T10:09:54.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.619+0000 7f596df16700 1 --2- 172.21.15.3:0/2676207473 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f596812d100 0x7f596812f4f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.619+0000 7f596df16700 1 --2- 172.21.15.3:0/2676207473 >> v2:172.21.15.3:3300/0 conn(0x7f5968099310 0x7f59680996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.619+0000 7f596df16700 1 -- 172.21.15.3:0/2676207473 >> 172.21.15.3:0/2676207473 conn(0x7f5968097460 msgr2=0x7f5968097870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:54.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.619+0000 7f596df16700 1 -- 172.21.15.3:0/2676207473 shutdown_connections 2023-10-03T10:09:54.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.619+0000 7f596df16700 1 -- 172.21.15.3:0/2676207473 wait complete. 2023-10-03T10:09:54.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.621+0000 7f596df16700 1 Processor -- start 2023-10-03T10:09:54.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.621+0000 7f596df16700 1 -- start start 2023-10-03T10:09:54.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.622+0000 7f596df16700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5968098820 0x7f5968062a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.622+0000 7f596df16700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5968099310 0x7f5968116ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.622+0000 7f596df16700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f596812d100 0x7f5968116fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.622+0000 7f596df16700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5968118a00 con 0x7f5968098820 2023-10-03T10:09:54.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.622+0000 7f596df16700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596811d270 con 0x7f596812d100 2023-10-03T10:09:54.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.622+0000 7f596df16700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f596811d510 con 0x7f5968099310 2023-10-03T10:09:54.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.622+0000 7f59677fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5968098820 0x7f5968062a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.622+0000 7f5967fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f596812d100 0x7f5968116fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.622+0000 7f5967fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f596812d100 0x7f5968116fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39074/0 (socket says 172.21.15.3:39074) 2023-10-03T10:09:54.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.622+0000 7f59677fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5968098820 0x7f5968062a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47714/0 (socket says 172.21.15.3:47714) 2023-10-03T10:09:54.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.622+0000 7f5967fff700 1 -- 172.21.15.3:0/1781674460 learned_addr learned my addr 172.21.15.3:0/1781674460 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:54.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.622+0000 7f5967fff700 1 -- 172.21.15.3:0/1781674460 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5968099310 msgr2=0x7f5968116ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.622+0000 7f5967fff700 1 --2- 172.21.15.3:0/1781674460 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5968099310 0x7f5968116ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-10-03T10:09:54.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.622+0000 7f5967fff700 1 -- 172.21.15.3:0/1781674460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5968098820 msgr2=0x7f5968062a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.622+0000 7f5967fff700 1 --2- 172.21.15.3:0/1781674460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5968098820 0x7f5968062a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.622+0000 7f5967fff700 1 -- 172.21.15.3:0/1781674460 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5960009a20 con 0x7f596812d100 2023-10-03T10:09:54.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.622+0000 7f59677fe700 1 --2- 172.21.15.3:0/1781674460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5968098820 0x7f5968062a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:54.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.622+0000 7f5967fff700 1 --2- 172.21.15.3:0/1781674460 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f596812d100 0x7f5968116fb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f596000b6c0 tx=0x7f5968093760).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:54.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.623+0000 7f594f7fe700 1 -- 172.21.15.3:0/1781674460 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5960012410 con 0x7f596812d100 2023-10-03T10:09:54.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.623+0000 7f594f7fe700 1 -- 172.21.15.3:0/1781674460 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596001d630 con 0x7f596812d100 2023-10-03T10:09:54.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.623+0000 7f594f7fe700 1 -- 172.21.15.3:0/1781674460 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59600070e0 con 0x7f596812d100 2023-10-03T10:09:54.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.623+0000 7f596df16700 1 -- 172.21.15.3:0/1781674460 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5968117ec0 con 0x7f596812d100 2023-10-03T10:09:54.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.623+0000 7f596df16700 1 -- 172.21.15.3:0/1781674460 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59681174a0 con 0x7f596812d100 2023-10-03T10:09:54.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.624+0000 7f594f7fe700 1 -- 172.21.15.3:0/1781674460 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f5960003040 con 0x7f596812d100 2023-10-03T10:09:54.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.624+0000 7f594f7fe700 1 -- 172.21.15.3:0/1781674460 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f596003ccb0 con 0x7f596812d100 2023-10-03T10:09:54.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.624+0000 7f596df16700 1 -- 172.21.15.3:0/1781674460 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5968097bd0 con 0x7f596812d100 2023-10-03T10:09:54.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.629+0000 7f594f7fe700 1 -- 172.21.15.3:0/1781674460 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f5960025dc0 con 0x7f596812d100 2023-10-03T10:09:54.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.635+0000 7f3ad6804700 1 Processor -- start 2023-10-03T10:09:54.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.635+0000 7f3ad6804700 1 -- start start 2023-10-03T10:09:54.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.636+0000 7f3ad6804700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3ad0091a40 0x7f3ad0091e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.636+0000 7f3ad6804700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ad0092310 0x7f3ad01279f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.636+0000 7f3ad6804700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ad0094140 con 0x7f3ad0127ee0 2023-10-03T10:09:54.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.636+0000 7f3ad6804700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ad0093050 con 0x7f3ad0091a40 2023-10-03T10:09:54.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.636+0000 7f3ad6804700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad0094f90 con 0x7f3ad0092310 2023-10-03T10:09:54.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.636+0000 7f3acffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3ad0091a40 0x7f3ad0091e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.636+0000 7f3ad4da1700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3ad0127ee0 0x7f3ad012ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38506/0 (socket says 172.21.15.3:38506) 2023-10-03T10:09:54.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.636+0000 7f3ad4da1700 1 -- 172.21.15.3:0/1369643027 learned_addr learned my addr 172.21.15.3:0/1369643027 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:09:54.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.636+0000 7f3acf7fe700 1 --2- 172.21.15.3:0/1369643027 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ad0092310 0x7f3ad01279f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.637+0000 7f3acffff700 1 -- 172.21.15.3:0/1369643027 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ad0092310 msgr2=0x7f3ad01279f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:54.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.637+0000 7f3acffff700 1 --2- 172.21.15.3:0/1369643027 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ad0092310 0x7f3ad01279f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.637+0000 7f3acffff700 1 -- 172.21.15.3:0/1369643027 >> v1:172.21.15.3:6789/0 conn(0x7f3ad0127ee0 legacy=0x7f3ad012ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.637+0000 7f3aceffd700 1 -- 172.21.15.3:0/1369643027 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2780138742 0 0) 0x7f3ad0094140 con 0x7f3ad0127ee0 2023-10-03T10:09:54.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.637+0000 7f3acffff700 1 -- 172.21.15.3:0/1369643027 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad0099fd0 con 0x7f3ad0091a40 2023-10-03T10:09:54.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.637+0000 7f3acf7fe700 1 --2- 172.21.15.3:0/1369643027 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ad0092310 0x7f3ad01279f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:54.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.637+0000 7f3acffff700 1 --2- 172.21.15.3:0/1369643027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ad0091a40 0x7f3ad0091e20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f3ac0009760 tx=0x7f3ac0009060).ready entity=mon.0 client_cookie=da4a9747c4fd0337 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:54.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.637+0000 7f3aceffd700 1 -- 172.21.15.3:0/1369643027 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac000f510 con 0x7f3ad0091a40 2023-10-03T10:09:54.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.637+0000 7f3aceffd700 1 -- 172.21.15.3:0/1369643027 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac000f670 con 0x7f3ad0091a40 2023-10-03T10:09:54.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.637+0000 7f3aceffd700 1 -- 172.21.15.3:0/1369643027 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac000f9f0 con 0x7f3ad0091a40 2023-10-03T10:09:54.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.638+0000 7f3ad6804700 1 -- 172.21.15.3:0/1369643027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ad0091a40 msgr2=0x7f3ad0091e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.638+0000 7f3ad6804700 1 --2- 172.21.15.3:0/1369643027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ad0091a40 0x7f3ad0091e20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f3ac0009760 tx=0x7f3ac0009060).stop 2023-10-03T10:09:54.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.638+0000 7f3ad6804700 1 -- 172.21.15.3:0/1369643027 shutdown_connections 2023-10-03T10:09:54.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.638+0000 7f3ad6804700 1 --2- 172.21.15.3:0/1369643027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ad0091a40 0x7f3ad0091e20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.638+0000 7f3ad6804700 1 --2- 172.21.15.3:0/1369643027 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ad0092310 0x7f3ad01279f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.638+0000 7f3ad6804700 1 -- 172.21.15.3:0/1369643027 >> 172.21.15.3:0/1369643027 conn(0x7f3ad0063560 msgr2=0x7f3ad009a220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:54.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.638+0000 7f3ad6804700 1 -- 172.21.15.3:0/1369643027 shutdown_connections 2023-10-03T10:09:54.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.638+0000 7f3ad6804700 1 -- 172.21.15.3:0/1369643027 wait complete. 2023-10-03T10:09:54.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.640+0000 7f3ad6804700 1 Processor -- start 2023-10-03T10:09:54.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.640+0000 7f3ad6804700 1 -- start start 2023-10-03T10:09:54.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.641+0000 7f3ad6804700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ad0092310 0x7f3ad01108f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.641+0000 7f3ad6804700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ad0127ee0 0x7f3ad012cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.641+0000 7f3ad6804700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ad012d440 0x7f3ad012d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:54.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.641+0000 7f3ad6804700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad0093df0 con 0x7f3ad012d440 2023-10-03T10:09:54.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.641+0000 7f3ad6804700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad0127aa0 con 0x7f3ad0092310 2023-10-03T10:09:54.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.641+0000 7f3ad6804700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad012ddb0 con 0x7f3ad0127ee0 2023-10-03T10:09:54.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.641+0000 7f3acf7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ad0127ee0 0x7f3ad012cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.641+0000 7f3acf7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ad0127ee0 0x7f3ad012cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57396/0 (socket says 172.21.15.3:57396) 2023-10-03T10:09:54.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.641+0000 7f3acf7fe700 1 -- 172.21.15.3:0/948542732 learned_addr learned my addr 172.21.15.3:0/948542732 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:54.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.641+0000 7f3ad4da1700 1 --2- 172.21.15.3:0/948542732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ad012d440 0x7f3ad012d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.641+0000 7f3acffff700 1 --2- 172.21.15.3:0/948542732 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ad0092310 0x7f3ad01108f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:54.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.641+0000 7f3acf7fe700 1 -- 172.21.15.3:0/948542732 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ad0092310 msgr2=0x7f3ad01108f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.641+0000 7f3acf7fe700 1 --2- 172.21.15.3:0/948542732 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ad0092310 0x7f3ad01108f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.641+0000 7f3acf7fe700 1 -- 172.21.15.3:0/948542732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ad012d440 msgr2=0x7f3ad012d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.641+0000 7f3acf7fe700 1 --2- 172.21.15.3:0/948542732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ad012d440 0x7f3ad012d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.641+0000 7f3acf7fe700 1 -- 172.21.15.3:0/948542732 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac40092a0 con 0x7f3ad0127ee0 2023-10-03T10:09:54.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.642+0000 7f3acffff700 1 --2- 172.21.15.3:0/948542732 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ad0092310 0x7f3ad01108f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:09:54.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.642+0000 7f3acf7fe700 1 --2- 172.21.15.3:0/948542732 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ad0127ee0 0x7f3ad012cf50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f3ac400ea00 tx=0x7f3ac4009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:54.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.642+0000 7f3ab7fff700 1 -- 172.21.15.3:0/948542732 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac400c850 con 0x7f3ad0127ee0 2023-10-03T10:09:54.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.642+0000 7f3ab7fff700 1 -- 172.21.15.3:0/948542732 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac400c9b0 con 0x7f3ad0127ee0 2023-10-03T10:09:54.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.642+0000 7f3ab7fff700 1 -- 172.21.15.3:0/948542732 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac400cd30 con 0x7f3ad0127ee0 2023-10-03T10:09:54.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.642+0000 7f3ad6804700 1 -- 172.21.15.3:0/948542732 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac00092a0 con 0x7f3ad0127ee0 2023-10-03T10:09:54.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.642+0000 7f3ad6804700 1 -- 172.21.15.3:0/948542732 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad0130450 con 0x7f3ad0127ee0 2023-10-03T10:09:54.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.643+0000 7f3ab7fff700 1 -- 172.21.15.3:0/948542732 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f3ac4012600 con 0x7f3ad0127ee0 2023-10-03T10:09:54.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.644+0000 7f3ab7fff700 1 -- 172.21.15.3:0/948542732 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f3ac403b610 con 0x7f3ad0127ee0 2023-10-03T10:09:54.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.644+0000 7f3ad6804700 1 -- 172.21.15.3:0/948542732 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ad0068600 con 0x7f3ad0127ee0 2023-10-03T10:09:54.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.653+0000 7f3ab7fff700 1 -- 172.21.15.3:0/948542732 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f3ac4016070 con 0x7f3ad0127ee0 2023-10-03T10:09:54.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.752+0000 7f8c46981700 1 -- 172.21.15.3:0/763890501 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8c4005a260 con 0x7f8c400999c0 2023-10-03T10:09:54.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.753+0000 7f8c27fff700 1 -- 172.21.15.3:0/763890501 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8c38024070 con 0x7f8c400999c0 2023-10-03T10:09:54.753 INFO:teuthology.orchestra.run.smithi003.stdout:38654705668 2023-10-03T10:09:54.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.756+0000 7f8c46981700 1 -- 172.21.15.3:0/763890501 >> v1:172.21.15.3:6816/118296 conn(0x7f8c2c02bb60 legacy=0x7f8c2c02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.756+0000 7f8c46981700 1 -- 172.21.15.3:0/763890501 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c400999c0 msgr2=0x7f8c401365d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.756+0000 7f8c46981700 1 --2- 172.21.15.3:0/763890501 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c400999c0 0x7f8c401365d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f8c3800c7f0 tx=0x7f8c38001530).stop 2023-10-03T10:09:54.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.757+0000 7f8c46981700 1 -- 172.21.15.3:0/763890501 shutdown_connections 2023-10-03T10:09:54.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.757+0000 7f8c46981700 1 --2- 172.21.15.3:0/763890501 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c400999c0 0x7f8c401365d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.757+0000 7f8c46981700 1 --2- 172.21.15.3:0/763890501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c40098820 0x7f8c40135bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.757+0000 7f8c46981700 1 --2- 172.21.15.3:0/763890501 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c400990f0 0x7f8c401360e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.757+0000 7f8c46981700 1 -- 172.21.15.3:0/763890501 >> 172.21.15.3:0/763890501 conn(0x7f8c40097460 msgr2=0x7f8c40095300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:54.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.758+0000 7f8c46981700 1 -- 172.21.15.3:0/763890501 shutdown_connections 2023-10-03T10:09:54.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.758+0000 7f8c46981700 1 -- 172.21.15.3:0/763890501 wait complete. 2023-10-03T10:09:54.771 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.3 2023-10-03T10:09:54.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.795+0000 7f1a2a50c700 1 -- 172.21.15.3:0/2280636591 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1a24091d40 con 0x7f1a24093780 2023-10-03T10:09:54.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.795+0000 7f1a0bfff700 1 -- 172.21.15.3:0/2280636591 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1a14019300 con 0x7f1a24093780 2023-10-03T10:09:54.796 INFO:teuthology.orchestra.run.smithi003.stdout:38654705668 2023-10-03T10:09:54.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.798+0000 7f1a2a50c700 1 -- 172.21.15.3:0/2280636591 >> v1:172.21.15.3:6816/118296 conn(0x7f1a1002bb60 legacy=0x7f1a1002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.799+0000 7f1a2a50c700 1 -- 172.21.15.3:0/2280636591 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a24093780 msgr2=0x7f1a24062a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.799+0000 7f1a2a50c700 1 --2- 172.21.15.3:0/2280636591 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a24093780 0x7f1a24062a90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f1a1400b560 tx=0x7f1a140090f0).stop 2023-10-03T10:09:54.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.799+0000 7f1a2a50c700 1 -- 172.21.15.3:0/2280636591 shutdown_connections 2023-10-03T10:09:54.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.799+0000 7f1a2a50c700 1 --2- 172.21.15.3:0/2280636591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a24125f90 0x7f1a24123f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-10-03T10:09:54.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.799+0000 7f1a2a50c700 1 --2- 172.21.15.3:0/2280636591 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a24093780 0x7f1a24062a90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.799+0000 7f1a2a50c700 1 --2- 172.21.15.3:0/2280636591 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a24125680 0x7f1a24123a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.799+0000 7f1a2a50c700 1 -- 172.21.15.3:0/2280636591 >> 172.21.15.3:0/2280636591 conn(0x7f1a2409a8e0 msgr2=0x7f1a240991b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:54.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.799+0000 7f1a2a50c700 1 -- 172.21.15.3:0/2280636591 shutdown_connections 2023-10-03T10:09:54.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.799+0000 7f1a2a50c700 1 -- 172.21.15.3:0/2280636591 wait complete. 2023-10-03T10:09:54.812 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.1 2023-10-03T10:09:54.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.815+0000 7f596df16700 1 -- 172.21.15.3:0/1781674460 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f596805a260 con 0x7f596812d100 2023-10-03T10:09:54.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.816+0000 7f594f7fe700 1 -- 172.21.15.3:0/1781674460 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5960036090 con 0x7f596812d100 2023-10-03T10:09:54.816 INFO:teuthology.orchestra.run.smithi003.stdout:38654705668 2023-10-03T10:09:54.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.818+0000 7f596df16700 1 -- 172.21.15.3:0/1781674460 >> v1:172.21.15.3:6816/118296 conn(0x7f595402bb60 legacy=0x7f595402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.818+0000 7f596df16700 1 -- 172.21.15.3:0/1781674460 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f596812d100 msgr2=0x7f5968116fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.818+0000 7f596df16700 1 --2- 172.21.15.3:0/1781674460 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f596812d100 0x7f5968116fb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f596000b6c0 tx=0x7f5968093760).stop 2023-10-03T10:09:54.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.818+0000 7f596df16700 1 -- 172.21.15.3:0/1781674460 shutdown_connections 2023-10-03T10:09:54.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.818+0000 7f596df16700 1 --2- 172.21.15.3:0/1781674460 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f596812d100 0x7f5968116fb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.818+0000 7f596df16700 1 --2- 172.21.15.3:0/1781674460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5968098820 0x7f5968062a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.818+0000 7f596df16700 1 --2- 172.21.15.3:0/1781674460 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5968099310 0x7f5968116ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-10-03T10:09:54.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.818+0000 7f596df16700 1 -- 172.21.15.3:0/1781674460 >> 172.21.15.3:0/1781674460 conn(0x7f5968097460 msgr2=0x7f5968069560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:54.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.818+0000 7f596df16700 1 -- 172.21.15.3:0/1781674460 shutdown_connections 2023-10-03T10:09:54.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.818+0000 7f596df16700 1 -- 172.21.15.3:0/1781674460 wait complete. 2023-10-03T10:09:54.836 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.2 2023-10-03T10:09:54.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.859+0000 7f3ad6804700 1 -- 172.21.15.3:0/948542732 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3ad0130610 con 0x7f3ad0127ee0 2023-10-03T10:09:54.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.859+0000 7f3ab7fff700 1 -- 172.21.15.3:0/948542732 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3ac4016020 con 0x7f3ad0127ee0 2023-10-03T10:09:54.860 INFO:teuthology.orchestra.run.smithi003.stdout:38654705668 2023-10-03T10:09:54.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.861+0000 7f3ad6804700 1 -- 172.21.15.3:0/948542732 >> v1:172.21.15.3:6816/118296 conn(0x7f3abc02bbe0 legacy=0x7f3abc02e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.861+0000 7f3ad6804700 1 -- 172.21.15.3:0/948542732 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ad0127ee0 msgr2=0x7f3ad012cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:54.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.861+0000 7f3ad6804700 1 --2- 172.21.15.3:0/948542732 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ad0127ee0 0x7f3ad012cf50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f3ac400ea00 tx=0x7f3ac4009060).stop 2023-10-03T10:09:54.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.861+0000 7f3ad6804700 1 -- 172.21.15.3:0/948542732 shutdown_connections 2023-10-03T10:09:54.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.861+0000 7f3ad6804700 1 --2- 172.21.15.3:0/948542732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ad012d440 0x7f3ad012d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.861+0000 7f3ad6804700 1 --2- 172.21.15.3:0/948542732 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ad0092310 0x7f3ad01108f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.861+0000 7f3ad6804700 1 --2- 172.21.15.3:0/948542732 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ad0127ee0 0x7f3ad012cf50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:54.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.861+0000 7f3ad6804700 1 -- 172.21.15.3:0/948542732 >> 172.21.15.3:0/948542732 conn(0x7f3ad0063560 msgr2=0x7f3ad0066ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:54.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.861+0000 7f3ad6804700 1 -- 172.21.15.3:0/948542732 shutdown_connections 2023-10-03T10:09:54.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:54.861+0000 7f3ad6804700 1 -- 172.21.15.3:0/948542732 wait complete. 2023-10-03T10:09:54.874 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.0 2023-10-03T10:09:55.772 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:09:55.813 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:09:55.837 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:09:55.875 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:09:55.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.918+0000 7f49d2259700 1 Processor -- start 2023-10-03T10:09:55.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.919+0000 7f49d2259700 1 -- start start 2023-10-03T10:09:55.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.919+0000 7f49d2259700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f49cc065640 0x7f49cc098620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:55.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.919+0000 7f49d2259700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f49cc099220 0x7f49cc132450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:55.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.919+0000 7f49d2259700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49cc0660b0 con 0x7f49cc097cf0 2023-10-03T10:09:55.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.919+0000 7f49d2259700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49cc098180 con 0x7f49cc099220 2023-10-03T10:09:55.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.919+0000 7f49d2259700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc095620 con 0x7f49cc065640 2023-10-03T10:09:55.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.919+0000 7f49cbfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f49cc099220 0x7f49cc132450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:55.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.919+0000 7f49cb7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f49cc065640 0x7f49cc098620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:55.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.920+0000 7f49cb7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f49cc065640 0x7f49cc098620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57398/0 (socket says 172.21.15.3:57398) 2023-10-03T10:09:55.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.920+0000 7f49cb7fe700 1 -- 172.21.15.3:0/1031582411 learned_addr learned my addr 172.21.15.3:0/1031582411 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:55.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.920+0000 7f49cbfff700 1 -- 172.21.15.3:0/1031582411 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f49cc065640 msgr2=0x7f49cc098620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:55.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.920+0000 7f49cbfff700 1 --2- 172.21.15.3:0/1031582411 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f49cc065640 0x7f49cc098620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:55.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.920+0000 7f49cbfff700 1 -- 172.21.15.3:0/1031582411 >> v1:172.21.15.3:6789/0 conn(0x7f49cc097cf0 legacy=0x7f49cc098b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:55.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.920+0000 7f49cbfff700 1 -- 172.21.15.3:0/1031582411 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc096180 con 0x7f49cc099220 2023-10-03T10:09:55.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.920+0000 7f49cb7fe700 1 --2- 172.21.15.3:0/1031582411 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f49cc065640 0x7f49cc098620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:55.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.920+0000 7f49cbfff700 1 --2- 172.21.15.3:0/1031582411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49cc099220 0x7f49cc132450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f49c4009e90 tx=0x7f49c4009780).ready entity=mon.0 client_cookie=e83c0b35db1b5e50 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:55.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.922+0000 7f49ca7fc700 1 -- 172.21.15.3:0/1031582411 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c400e070 con 0x7f49cc099220 2023-10-03T10:09:55.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.922+0000 7f49ca7fc700 1 -- 172.21.15.3:0/1031582411 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c400bb20 con 0x7f49cc099220 2023-10-03T10:09:55.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.922+0000 7f49ca7fc700 1 -- 172.21.15.3:0/1031582411 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c400bea0 con 0x7f49cc099220 2023-10-03T10:09:55.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.923+0000 7f49d2259700 1 -- 172.21.15.3:0/1031582411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49cc099220 msgr2=0x7f49cc132450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:55.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.923+0000 7f49d2259700 1 --2- 172.21.15.3:0/1031582411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49cc099220 0x7f49cc132450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f49c4009e90 tx=0x7f49c4009780).stop 2023-10-03T10:09:55.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.923+0000 7f49d2259700 1 -- 172.21.15.3:0/1031582411 shutdown_connections 2023-10-03T10:09:55.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.923+0000 7f49d2259700 1 --2- 172.21.15.3:0/1031582411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49cc099220 0x7f49cc132450 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:55.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.923+0000 7f49d2259700 1 --2- 172.21.15.3:0/1031582411 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f49cc065640 0x7f49cc098620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:55.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.923+0000 7f49d2259700 1 -- 172.21.15.3:0/1031582411 >> 172.21.15.3:0/1031582411 conn(0x7f49cc09a720 msgr2=0x7f49cc09ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:55.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.923+0000 7f49d2259700 1 -- 172.21.15.3:0/1031582411 shutdown_connections 2023-10-03T10:09:55.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.923+0000 7f49d2259700 1 -- 172.21.15.3:0/1031582411 wait complete. 2023-10-03T10:09:55.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.925+0000 7f49d2259700 1 Processor -- start 2023-10-03T10:09:55.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.925+0000 7f49d2259700 1 -- start start 2023-10-03T10:09:55.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.925+0000 7f49d2259700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49cc065640 0x7f49cc11e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:55.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.925+0000 7f49d2259700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f49cc099220 0x7f49cc11a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:55.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.925+0000 7f49d2259700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f49cc11ab50 0x7f49cc11afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:55.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.925+0000 7f49d2259700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc11da90 con 0x7f49cc065640 2023-10-03T10:09:55.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.925+0000 7f49d2259700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc11b4a0 con 0x7f49cc11ab50 2023-10-03T10:09:55.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.925+0000 7f49d2259700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc09cb20 con 0x7f49cc099220 2023-10-03T10:09:55.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.926+0000 7f49cb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49cc065640 0x7f49cc11e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:55.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.926+0000 7f49cb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49cc065640 0x7f49cc11e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47738/0 (socket says 172.21.15.3:47738) 2023-10-03T10:09:55.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.926+0000 7f49cb7fe700 1 -- 172.21.15.3:0/3651375806 learned_addr learned my addr 172.21.15.3:0/3651375806 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:55.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.926+0000 7f49caffd700 1 --2- 172.21.15.3:0/3651375806 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f49cc099220 0x7f49cc11a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:55.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.926+0000 7f49cbfff700 1 --2- 172.21.15.3:0/3651375806 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f49cc11ab50 0x7f49cc11afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:55.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.926+0000 7f49cb7fe700 1 -- 172.21.15.3:0/3651375806 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f49cc099220 msgr2=0x7f49cc11a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:55.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.926+0000 7f49cb7fe700 1 --2- 172.21.15.3:0/3651375806 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f49cc099220 0x7f49cc11a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:55.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.926+0000 7f49cb7fe700 1 -- 172.21.15.3:0/3651375806 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f49cc11ab50 msgr2=0x7f49cc11afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:55.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.926+0000 7f49cb7fe700 1 --2- 172.21.15.3:0/3651375806 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f49cc11ab50 0x7f49cc11afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:55.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.926+0000 7f49cb7fe700 1 -- 172.21.15.3:0/3651375806 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc009370 con 0x7f49cc065640 2023-10-03T10:09:55.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.926+0000 7f49cbfff700 1 --2- 172.21.15.3:0/3651375806 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f49cc11ab50 0x7f49cc11afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:55.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.926+0000 7f49cb7fe700 1 --2- 172.21.15.3:0/3651375806 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49cc065640 0x7f49cc11e530 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f49bc00ea20 tx=0x7f49bc0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:55.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.927+0000 7f49b37fe700 1 -- 172.21.15.3:0/3651375806 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc009650 con 0x7f49cc065640 2023-10-03T10:09:55.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.927+0000 7f49b37fe700 1 -- 172.21.15.3:0/3651375806 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49bc00ee60 con 0x7f49cc065640 2023-10-03T10:09:55.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.927+0000 7f49d2259700 1 -- 172.21.15.3:0/3651375806 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c4009a20 con 0x7f49cc065640 2023-10-03T10:09:55.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.927+0000 7f49b37fe700 1 -- 172.21.15.3:0/3651375806 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc00ca60 con 0x7f49cc065640 2023-10-03T10:09:55.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.927+0000 7f49d2259700 1 -- 172.21.15.3:0/3651375806 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49cc11d6f0 con 0x7f49cc065640 2023-10-03T10:09:55.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.928+0000 7f49b37fe700 1 -- 172.21.15.3:0/3651375806 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f49bc01b400 con 0x7f49cc065640 2023-10-03T10:09:55.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.928+0000 7f49b37fe700 1 -- 172.21.15.3:0/3651375806 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f49bc03b460 con 0x7f49cc065640 2023-10-03T10:09:55.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.928+0000 7f49d2259700 1 -- 172.21.15.3:0/3651375806 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49cc09e990 con 0x7f49cc065640 2023-10-03T10:09:55.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.937+0000 7f49b37fe700 1 -- 172.21.15.3:0/3651375806 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f49bc016020 con 0x7f49cc065640 2023-10-03T10:09:55.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.968+0000 7f8d98e97700 1 Processor -- start 2023-10-03T10:09:55.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.968+0000 7f8d98e97700 1 -- start start 2023-10-03T10:09:55.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.969+0000 7f8d98e97700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d9412b2b0 0x7f8d9412fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:55.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.969+0000 7f8d98e97700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d9412b6f0 0x7f8d941322c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:55.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.969+0000 7f8d98e97700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d9406a090 con 0x7f8d94059070 2023-10-03T10:09:55.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.969+0000 7f8d98e97700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d94059450 con 0x7f8d9412b2b0 2023-10-03T10:09:55.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.969+0000 7f8d98e97700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9405fe30 con 0x7f8d9412b6f0 2023-10-03T10:09:55.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.969+0000 7f8d9259c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8d94059070 0x7f8d9412aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38522/0 (socket says 172.21.15.3:38522) 2023-10-03T10:09:55.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.969+0000 7f8d9259c700 1 -- 172.21.15.3:0/1686990855 learned_addr learned my addr 172.21.15.3:0/1686990855 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:09:55.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.969+0000 7f8d91d9b700 1 --2- 172.21.15.3:0/1686990855 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d9412b2b0 0x7f8d9412fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:55.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.969+0000 7f8d92d9d700 1 --2- 172.21.15.3:0/1686990855 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d9412b6f0 0x7f8d941322c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:55.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.969+0000 7f8d9159a700 1 -- 172.21.15.3:0/1686990855 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4051859887 0 0) 0x7f8d9406a090 con 0x7f8d94059070 2023-10-03T10:09:55.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.969+0000 7f8d9159a700 1 -- 172.21.15.3:0/1686990855 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d7c0033f0 con 0x7f8d94059070 2023-10-03T10:09:55.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.970+0000 7f8d92d9d700 1 -- 172.21.15.3:0/1686990855 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d9412b2b0 msgr2=0x7f8d9412fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:55.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.970+0000 7f8d92d9d700 1 --2- 172.21.15.3:0/1686990855 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d9412b2b0 0x7f8d9412fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:55.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.970+0000 7f8d92d9d700 1 -- 172.21.15.3:0/1686990855 >> v1:172.21.15.3:6789/0 conn(0x7f8d94059070 legacy=0x7f8d9412aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:55.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.970+0000 7f8d92d9d700 1 -- 172.21.15.3:0/1686990855 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d94093770 con 0x7f8d9412b6f0 2023-10-03T10:09:55.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.970+0000 7f8d91d9b700 1 --2- 172.21.15.3:0/1686990855 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d9412b2b0 0x7f8d9412fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:55.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.970+0000 7f8d92d9d700 1 --2- 172.21.15.3:0/1686990855 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d9412b6f0 0x7f8d941322c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f8d8c009ec0 tx=0x7f8d8c009780).ready entity=mon.2 client_cookie=f472cc16e7a29087 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:55.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.970+0000 7f8d9159a700 1 -- 172.21.15.3:0/1686990855 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c00e070 con 0x7f8d9412b6f0 2023-10-03T10:09:55.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.970+0000 7f8d9159a700 1 -- 172.21.15.3:0/1686990855 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d8c00bae0 con 0x7f8d9412b6f0 2023-10-03T10:09:55.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.970+0000 7f8d9159a700 1 -- 172.21.15.3:0/1686990855 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c00be60 con 0x7f8d9412b6f0 2023-10-03T10:09:55.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.971+0000 7f8d98e97700 1 -- 172.21.15.3:0/1686990855 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d9412b6f0 msgr2=0x7f8d941322c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:55.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.971+0000 7f8d98e97700 1 --2- 172.21.15.3:0/1686990855 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d9412b6f0 0x7f8d941322c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f8d8c009ec0 tx=0x7f8d8c009780).stop 2023-10-03T10:09:55.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.971+0000 7f8d98e97700 1 -- 172.21.15.3:0/1686990855 shutdown_connections 2023-10-03T10:09:55.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.971+0000 7f8d98e97700 1 --2- 172.21.15.3:0/1686990855 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d9412b6f0 0x7f8d941322c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:55.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.971+0000 7f8d98e97700 1 --2- 172.21.15.3:0/1686990855 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d9412b2b0 0x7f8d9412fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:55.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.971+0000 7f8d98e97700 1 -- 172.21.15.3:0/1686990855 >> 172.21.15.3:0/1686990855 conn(0x7f8d94063560 msgr2=0x7f8d9409a220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:55.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.971+0000 7f8d98e97700 1 -- 172.21.15.3:0/1686990855 shutdown_connections 2023-10-03T10:09:55.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.971+0000 7f8d98e97700 1 -- 172.21.15.3:0/1686990855 wait complete. 2023-10-03T10:09:55.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.973+0000 7f8d98e97700 1 Processor -- start 2023-10-03T10:09:55.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.973+0000 7f8d98e97700 1 -- start start 2023-10-03T10:09:55.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.973+0000 7f8d98e97700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d94059070 0x7f8d94131850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:55.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.974+0000 7f8d98e97700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d9412b2b0 0x7f8d9412c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:55.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.974+0000 7f8d98e97700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d9412b6f0 0x7f8d9412cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:55.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.974+0000 7f8d98e97700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9412e1d0 con 0x7f8d9412b6f0 2023-10-03T10:09:55.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.974+0000 7f8d98e97700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9412e3c0 con 0x7f8d94059070 2023-10-03T10:09:55.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.974+0000 7f8d98e97700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8d940939a0 con 0x7f8d9412b2b0 2023-10-03T10:09:55.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.974+0000 7f8d9259c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d94059070 0x7f8d94131850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:55.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.974+0000 7f8d9259c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d94059070 0x7f8d94131850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39102/0 (socket says 172.21.15.3:39102) 2023-10-03T10:09:55.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.974+0000 7f8d9259c700 1 -- 172.21.15.3:0/4244064115 learned_addr learned my addr 172.21.15.3:0/4244064115 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:55.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.974+0000 7f8d92d9d700 1 --2- 172.21.15.3:0/4244064115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d9412b6f0 0x7f8d9412cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:55.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.974+0000 7f8d91d9b700 1 --2- 172.21.15.3:0/4244064115 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d9412b2b0 0x7f8d9412c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:55.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.974+0000 7f8d91d9b700 1 -- 172.21.15.3:0/4244064115 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d94059070 msgr2=0x7f8d94131850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:55.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.974+0000 7f8d91d9b700 1 --2- 172.21.15.3:0/4244064115 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d94059070 0x7f8d94131850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:55.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.974+0000 7f8d91d9b700 1 -- 172.21.15.3:0/4244064115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d9412b6f0 msgr2=0x7f8d9412cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:55.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.974+0000 7f8d91d9b700 1 --2- 172.21.15.3:0/4244064115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d9412b6f0 0x7f8d9412cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:55.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.974+0000 7f8d91d9b700 1 -- 172.21.15.3:0/4244064115 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d880092a0 con 0x7f8d9412b2b0 2023-10-03T10:09:55.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.974+0000 7f8d91d9b700 1 --2- 172.21.15.3:0/4244064115 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d9412b2b0 0x7f8d9412c680 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f8d8800e950 tx=0x7f8d88009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:55.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.975+0000 7f8d827fc700 1 -- 172.21.15.3:0/4244064115 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d88009580 con 0x7f8d9412b2b0 2023-10-03T10:09:55.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.975+0000 7f8d98e97700 1 -- 172.21.15.3:0/4244064115 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8c009a20 con 0x7f8d9412b2b0 2023-10-03T10:09:55.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.975+0000 7f8d98e97700 1 -- 172.21.15.3:0/4244064115 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d94130cc0 con 0x7f8d9412b2b0 2023-10-03T10:09:55.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.975+0000 7f8d827fc700 1 -- 172.21.15.3:0/4244064115 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d88012810 con 0x7f8d9412b2b0 2023-10-03T10:09:55.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.975+0000 7f8d827fc700 1 -- 172.21.15.3:0/4244064115 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d88012980 con 0x7f8d9412b2b0 2023-10-03T10:09:55.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.976+0000 7f8d827fc700 1 -- 172.21.15.3:0/4244064115 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f8d88012cf0 con 0x7f8d9412b2b0 2023-10-03T10:09:55.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.976+0000 7f8d827fc700 1 -- 172.21.15.3:0/4244064115 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f8d8803b410 con 0x7f8d9412b2b0 2023-10-03T10:09:55.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.976+0000 7f8d98e97700 1 -- 172.21.15.3:0/4244064115 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d78005120 con 0x7f8d9412b2b0 2023-10-03T10:09:55.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:55.985+0000 7f8d827fc700 1 -- 172.21.15.3:0/4244064115 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f8d880196e0 con 0x7f8d9412b2b0 2023-10-03T10:09:56.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.007+0000 7f3a617fe700 1 Processor -- start 2023-10-03T10:09:56.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.007+0000 7f3a617fe700 1 -- start start 2023-10-03T10:09:56.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.007+0000 7f3a617fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3a5c092460 0x7f3a5c12c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:56.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.007+0000 7f3a617fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3a5c12c6a0 0x7f3a5c12ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:56.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.008+0000 7f3a617fe700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a5c0698e0 con 0x7f3a5c068e70 2023-10-03T10:09:56.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.008+0000 7f3a617fe700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a5c069250 con 0x7f3a5c12c6a0 2023-10-03T10:09:56.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.008+0000 7f3a617fe700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c069ca0 con 0x7f3a5c092460 2023-10-03T10:09:56.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.008+0000 7f3a5affd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3a5c068e70 0x7f3a5c091d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38530/0 (socket says 172.21.15.3:38530) 2023-10-03T10:09:56.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.008+0000 7f3a5affd700 1 -- 172.21.15.3:0/3204117964 learned_addr learned my addr 172.21.15.3:0/3204117964 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:09:56.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.008+0000 7f3a5b7fe700 1 --2- 172.21.15.3:0/3204117964 >> v2:172.21.15.3:3300/0 conn(0x7f3a5c12c6a0 0x7f3a5c12ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:56.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.008+0000 7f3a59ffb700 1 -- 172.21.15.3:0/3204117964 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3761778141 0 0) 0x7f3a5c0698e0 con 0x7f3a5c068e70 2023-10-03T10:09:56.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.008+0000 7f3a59ffb700 1 -- 172.21.15.3:0/3204117964 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a440033f0 con 0x7f3a5c068e70 2023-10-03T10:09:56.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.008+0000 7f3a5b7fe700 1 -- 172.21.15.3:0/3204117964 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3a5c092460 msgr2=0x7f3a5c12c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:56.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.008+0000 7f3a5b7fe700 1 --2- 172.21.15.3:0/3204117964 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3a5c092460 0x7f3a5c12c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-03T10:09:56.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.008+0000 7f3a5b7fe700 1 -- 172.21.15.3:0/3204117964 >> v1:172.21.15.3:6789/0 conn(0x7f3a5c068e70 legacy=0x7f3a5c091d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:56.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.008+0000 7f3a5b7fe700 1 -- 172.21.15.3:0/3204117964 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a5c069f10 con 0x7f3a5c12c6a0 2023-10-03T10:09:56.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.008+0000 7f3a5b7fe700 1 --2- 172.21.15.3:0/3204117964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a5c12c6a0 0x7f3a5c12ea90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f3a5400afd0 tx=0x7f3a5400a890).ready entity=mon.0 client_cookie=be1c13e3d24e3c1a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:56.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.009+0000 7f3a59ffb700 1 -- 172.21.15.3:0/3204117964 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5400f070 con 0x7f3a5c12c6a0 2023-10-03T10:09:56.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.009+0000 7f3a59ffb700 1 -- 172.21.15.3:0/3204117964 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a5400a2e0 con 0x7f3a5c12c6a0 2023-10-03T10:09:56.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.009+0000 7f3a59ffb700 1 -- 172.21.15.3:0/3204117964 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5400cc10 con 0x7f3a5c12c6a0 2023-10-03T10:09:56.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.009+0000 7f3a617fe700 1 -- 172.21.15.3:0/3204117964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a5c12c6a0 msgr2=0x7f3a5c12ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:56.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.009+0000 7f3a617fe700 1 --2- 172.21.15.3:0/3204117964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a5c12c6a0 0x7f3a5c12ea90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f3a5400afd0 tx=0x7f3a5400a890).stop 2023-10-03T10:09:56.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.009+0000 7f3a617fe700 1 -- 172.21.15.3:0/3204117964 shutdown_connections 2023-10-03T10:09:56.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.009+0000 7f3a617fe700 1 --2- 172.21.15.3:0/3204117964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a5c12c6a0 0x7f3a5c12ea90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:56.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.009+0000 7f3a617fe700 1 --2- 172.21.15.3:0/3204117964 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3a5c092460 0x7f3a5c12c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-03T10:09:56.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.009+0000 7f3a617fe700 1 -- 172.21.15.3:0/3204117964 >> 172.21.15.3:0/3204117964 conn(0x7f3a5c063560 msgr2=0x7f3a5c09a220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:56.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.010+0000 7f3a617fe700 1 -- 172.21.15.3:0/3204117964 shutdown_connections 2023-10-03T10:09:56.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.010+0000 7f3a617fe700 1 -- 172.21.15.3:0/3204117964 wait complete. 2023-10-03T10:09:56.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.011+0000 7f3a617fe700 1 Processor -- start 2023-10-03T10:09:56.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.011+0000 7f3a617fe700 1 -- start start 2023-10-03T10:09:56.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.011+0000 7f3a617fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a5c068e70 0x7f3a5c1161d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:56.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.011+0000 7f3a617fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3a5c092460 0x7f3a5c1121a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:56.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.011+0000 7f3a617fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3a5c12c6a0 0x7f3a5c112690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:56.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.011+0000 7f3a617fe700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c124310 con 0x7f3a5c068e70 2023-10-03T10:09:56.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.011+0000 7f3a617fe700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c115cd0 con 0x7f3a5c12c6a0 2023-10-03T10:09:56.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.011+0000 7f3a617fe700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c068180 con 0x7f3a5c092460 2023-10-03T10:09:56.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.011+0000 7f3a5affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a5c068e70 0x7f3a5c1161d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:56.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.011+0000 7f3a5affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a5c068e70 0x7f3a5c1161d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47758/0 (socket says 172.21.15.3:47758) 2023-10-03T10:09:56.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.011+0000 7f3a5affd700 1 -- 172.21.15.3:0/965140839 learned_addr learned my addr 172.21.15.3:0/965140839 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:56.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.012+0000 7f3a5b7fe700 1 --2- 172.21.15.3:0/965140839 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3a5c12c6a0 0x7f3a5c112690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:56.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.012+0000 7f3a5a7fc700 1 --2- 172.21.15.3:0/965140839 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3a5c092460 0x7f3a5c1121a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:56.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.012+0000 7f3a5affd700 1 -- 172.21.15.3:0/965140839 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3a5c092460 msgr2=0x7f3a5c1121a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:56.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.012+0000 7f3a5affd700 1 --2- 172.21.15.3:0/965140839 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3a5c092460 0x7f3a5c1121a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:56.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.012+0000 7f3a5affd700 1 -- 172.21.15.3:0/965140839 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3a5c12c6a0 msgr2=0x7f3a5c112690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:56.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.012+0000 7f3a5affd700 1 --2- 172.21.15.3:0/965140839 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3a5c12c6a0 0x7f3a5c112690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:56.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.012+0000 7f3a5affd700 1 -- 172.21.15.3:0/965140839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a5400ab30 con 0x7f3a5c068e70 2023-10-03T10:09:56.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.012+0000 7f3a5a7fc700 1 --2- 172.21.15.3:0/965140839 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3a5c092460 0x7f3a5c1121a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:09:56.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.012+0000 7f3a5affd700 1 --2- 172.21.15.3:0/965140839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a5c068e70 0x7f3a5c1161d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f3a4c00c590 tx=0x7f3a4c007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:56.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.012+0000 7f3a42ffd700 1 -- 172.21.15.3:0/965140839 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c007e40 con 0x7f3a5c068e70 2023-10-03T10:09:56.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.012+0000 7f3a42ffd700 1 -- 172.21.15.3:0/965140839 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a4c0072e0 con 0x7f3a5c068e70 2023-10-03T10:09:56.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.012+0000 7f3a617fe700 1 -- 172.21.15.3:0/965140839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a5c115730 con 0x7f3a5c068e70 2023-10-03T10:09:56.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.013+0000 7f3a617fe700 1 -- 172.21.15.3:0/965140839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a5c113f80 con 0x7f3a5c068e70 2023-10-03T10:09:56.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.014+0000 7f3a42ffd700 1 -- 172.21.15.3:0/965140839 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c0057a0 con 0x7f3a5c068e70 2023-10-03T10:09:56.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.014+0000 7f3a42ffd700 1 -- 172.21.15.3:0/965140839 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f3a4c005bd0 con 0x7f3a5c068e70 2023-10-03T10:09:56.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.014+0000 7f3a42ffd700 1 -- 172.21.15.3:0/965140839 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f3a4c03a6b0 con 0x7f3a5c068e70 2023-10-03T10:09:56.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.015+0000 7f3a617fe700 1 -- 172.21.15.3:0/965140839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a5c09a930 con 0x7f3a5c068e70 2023-10-03T10:09:56.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.024+0000 7f3a42ffd700 1 -- 172.21.15.3:0/965140839 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f3a4c018070 con 0x7f3a5c068e70 2023-10-03T10:09:56.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.047+0000 7f4035199700 1 Processor -- start 2023-10-03T10:09:56.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.047+0000 7f4035199700 1 -- start start 2023-10-03T10:09:56.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.047+0000 7f4035199700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f403012ed70 0x7f40301340a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:56.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.047+0000 7f4035199700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f403012f910 0x7f40301365a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:56.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.047+0000 7f4035199700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4030099ee0 con 0x7f4030062a90 2023-10-03T10:09:56.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.047+0000 7f4035199700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4030062e70 con 0x7f403012ed70 2023-10-03T10:09:56.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.047+0000 7f4035199700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4030091ce0 con 0x7f403012f910 2023-10-03T10:09:56.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.047+0000 7f402e59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f403012ed70 0x7f40301340a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:56.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.047+0000 7f402e59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f403012ed70 0x7f40301340a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39110/0 (socket says 172.21.15.3:39110) 2023-10-03T10:09:56.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.047+0000 7f402e59c700 1 -- 172.21.15.3:0/2440356726 learned_addr learned my addr 172.21.15.3:0/2440356726 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:56.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.048+0000 7f402f59e700 1 --2- 172.21.15.3:0/2440356726 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f403012f910 0x7f40301365a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:56.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.048+0000 7f402e59c700 1 -- 172.21.15.3:0/2440356726 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f403012f910 msgr2=0x7f40301365a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:56.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.048+0000 7f402e59c700 1 --2- 172.21.15.3:0/2440356726 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f403012f910 0x7f40301365a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:56.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.048+0000 7f402e59c700 1 -- 172.21.15.3:0/2440356726 >> v1:172.21.15.3:6789/0 conn(0x7f4030062a90 legacy=0x7f403012f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:56.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.048+0000 7f402e59c700 1 -- 172.21.15.3:0/2440356726 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4030091a40 con 0x7f403012ed70 2023-10-03T10:09:56.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.048+0000 7f402e59c700 1 --2- 172.21.15.3:0/2440356726 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f403012ed70 0x7f40301340a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f4024009830 tx=0x7f40240090f0).ready entity=mon.1 client_cookie=d382fddaabc79a1b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:56.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.048+0000 7f402dd9b700 1 -- 172.21.15.3:0/2440356726 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402400b580 con 0x7f403012ed70 2023-10-03T10:09:56.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.049+0000 7f402dd9b700 1 -- 172.21.15.3:0/2440356726 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402400b6e0 con 0x7f403012ed70 2023-10-03T10:09:56.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.049+0000 7f402dd9b700 1 -- 172.21.15.3:0/2440356726 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402400ba60 con 0x7f403012ed70 2023-10-03T10:09:56.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.049+0000 7f4035199700 1 -- 172.21.15.3:0/2440356726 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f403012ed70 msgr2=0x7f40301340a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:56.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.049+0000 7f4035199700 1 --2- 172.21.15.3:0/2440356726 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f403012ed70 0x7f40301340a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f4024009830 tx=0x7f40240090f0).stop 2023-10-03T10:09:56.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.049+0000 7f4035199700 1 -- 172.21.15.3:0/2440356726 shutdown_connections 2023-10-03T10:09:56.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.049+0000 7f4035199700 1 --2- 172.21.15.3:0/2440356726 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f403012f910 0x7f40301365a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:56.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.049+0000 7f4035199700 1 --2- 172.21.15.3:0/2440356726 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f403012ed70 0x7f40301340a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:56.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.049+0000 7f4035199700 1 -- 172.21.15.3:0/2440356726 >> 172.21.15.3:0/2440356726 conn(0x7f4030097460 msgr2=0x7f4030097870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:56.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.049+0000 7f4035199700 1 -- 172.21.15.3:0/2440356726 shutdown_connections 2023-10-03T10:09:56.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.049+0000 7f4035199700 1 -- 172.21.15.3:0/2440356726 wait complete. 2023-10-03T10:09:56.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.050+0000 7f4035199700 1 Processor -- start 2023-10-03T10:09:56.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.051+0000 7f4035199700 1 -- start start 2023-10-03T10:09:56.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.051+0000 7f4035199700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4030062a90 0x7f403012c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:56.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.051+0000 7f4035199700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f403012f910 0x7f4030130780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:56.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.051+0000 7f4035199700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4030130c70 0x7f40301310f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:56.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.051+0000 7f4035199700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40300947e0 con 0x7f403012f910 2023-10-03T10:09:56.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.051+0000 7f4035199700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403009a080 con 0x7f4030062a90 2023-10-03T10:09:56.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.051+0000 7f4035199700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4030131cc0 con 0x7f4030130c70 2023-10-03T10:09:56.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.052+0000 7f402ed9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4030062a90 0x7f403012c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:56.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.052+0000 7f402f59e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4030130c70 0x7f40301310f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:56.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.052+0000 7f402ed9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4030062a90 0x7f403012c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39126/0 (socket says 172.21.15.3:39126) 2023-10-03T10:09:56.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.052+0000 7f402ed9d700 1 -- 172.21.15.3:0/3463728524 learned_addr learned my addr 172.21.15.3:0/3463728524 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:56.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.052+0000 7f402e59c700 1 --2- 172.21.15.3:0/3463728524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f403012f910 0x7f4030130780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:56.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.052+0000 7f402ed9d700 1 -- 172.21.15.3:0/3463728524 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4030130c70 msgr2=0x7f40301310f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:56.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.052+0000 7f402ed9d700 1 --2- 172.21.15.3:0/3463728524 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4030130c70 0x7f40301310f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:56.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.052+0000 7f402ed9d700 1 -- 172.21.15.3:0/3463728524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f403012f910 msgr2=0x7f4030130780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:56.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.052+0000 7f402ed9d700 1 --2- 172.21.15.3:0/3463728524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f403012f910 0x7f4030130780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:56.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.052+0000 7f402ed9d700 1 -- 172.21.15.3:0/3463728524 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4024009370 con 0x7f4030062a90 2023-10-03T10:09:56.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.052+0000 7f402ed9d700 1 --2- 172.21.15.3:0/3463728524 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4030062a90 0x7f403012c110 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f402000c4c0 tx=0x7f4020007e00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:56.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.053+0000 7f4016ffd700 1 -- 172.21.15.3:0/3463728524 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4020007020 con 0x7f4030062a90 2023-10-03T10:09:56.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.053+0000 7f4016ffd700 1 -- 172.21.15.3:0/3463728524 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4020001e90 con 0x7f4030062a90 2023-10-03T10:09:56.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.053+0000 7f4035199700 1 -- 172.21.15.3:0/3463728524 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40301324b0 con 0x7f4030062a90 2023-10-03T10:09:56.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.053+0000 7f4016ffd700 1 -- 172.21.15.3:0/3463728524 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40200073b0 con 0x7f4030062a90 2023-10-03T10:09:56.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.053+0000 7f4035199700 1 -- 172.21.15.3:0/3463728524 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f403012c600 con 0x7f4030062a90 2023-10-03T10:09:56.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.054+0000 7f4016ffd700 1 -- 172.21.15.3:0/3463728524 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f40200068b0 con 0x7f4030062a90 2023-10-03T10:09:56.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.054+0000 7f4035199700 1 -- 172.21.15.3:0/3463728524 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4030133600 con 0x7f4030062a90 2023-10-03T10:09:56.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.054+0000 7f4016ffd700 1 -- 172.21.15.3:0/3463728524 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f402003a8e0 con 0x7f4030062a90 2023-10-03T10:09:56.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.063+0000 7f4016ffd700 1 -- 172.21.15.3:0/3463728524 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f402001abf0 con 0x7f4030062a90 2023-10-03T10:09:56.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.130+0000 7f49d2259700 1 -- 172.21.15.3:0/3651375806 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f49cc05a260 con 0x7f49cc065640 2023-10-03T10:09:56.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.130+0000 7f49b37fe700 1 -- 172.21.15.3:0/3651375806 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f49bc019310 con 0x7f49cc065640 2023-10-03T10:09:56.131 INFO:teuthology.orchestra.run.smithi003.stdout:38654705668 2023-10-03T10:09:56.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.132+0000 7f49d2259700 1 -- 172.21.15.3:0/3651375806 >> v1:172.21.15.3:6816/118296 conn(0x7f49b802bb60 legacy=0x7f49b802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:56.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.132+0000 7f49d2259700 1 -- 172.21.15.3:0/3651375806 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49cc065640 msgr2=0x7f49cc11e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:56.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.132+0000 7f49d2259700 1 --2- 172.21.15.3:0/3651375806 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49cc065640 0x7f49cc11e530 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f49bc00ea20 tx=0x7f49bc0090f0).stop 2023-10-03T10:09:56.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.132+0000 7f49d2259700 1 -- 172.21.15.3:0/3651375806 shutdown_connections 2023-10-03T10:09:56.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.132+0000 7f49d2259700 1 --2- 172.21.15.3:0/3651375806 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f49cc11ab50 0x7f49cc11afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:56.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.132+0000 7f49d2259700 1 --2- 172.21.15.3:0/3651375806 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49cc065640 0x7f49cc11e530 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:56.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.132+0000 7f49d2259700 1 --2- 172.21.15.3:0/3651375806 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f49cc099220 0x7f49cc11a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:56.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.132+0000 7f49d2259700 1 -- 172.21.15.3:0/3651375806 >> 172.21.15.3:0/3651375806 conn(0x7f49cc09a720 msgr2=0x7f49cc09baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:56.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.133+0000 7f49d2259700 1 -- 172.21.15.3:0/3651375806 shutdown_connections 2023-10-03T10:09:56.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.133+0000 7f49d2259700 1 -- 172.21.15.3:0/3651375806 wait complete. 2023-10-03T10:09:56.146 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.3 2023-10-03T10:09:56.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.178+0000 7f8d98e97700 1 -- 172.21.15.3:0/4244064115 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8d780057f0 con 0x7f8d9412b2b0 2023-10-03T10:09:56.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.178+0000 7f8d827fc700 1 -- 172.21.15.3:0/4244064115 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8d88023350 con 0x7f8d9412b2b0 2023-10-03T10:09:56.179 INFO:teuthology.orchestra.run.smithi003.stdout:38654705668 2023-10-03T10:09:56.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.181+0000 7f8d98e97700 1 -- 172.21.15.3:0/4244064115 >> v1:172.21.15.3:6816/118296 conn(0x7f8d7402bb60 legacy=0x7f8d7402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:56.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.181+0000 7f8d98e97700 1 -- 172.21.15.3:0/4244064115 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d9412b2b0 msgr2=0x7f8d9412c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:56.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.181+0000 7f8d98e97700 1 --2- 172.21.15.3:0/4244064115 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d9412b2b0 0x7f8d9412c680 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f8d8800e950 tx=0x7f8d88009060).stop 2023-10-03T10:09:56.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.181+0000 7f8d98e97700 1 -- 172.21.15.3:0/4244064115 shutdown_connections 2023-10-03T10:09:56.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.181+0000 7f8d98e97700 1 --2- 172.21.15.3:0/4244064115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d9412b6f0 0x7f8d9412cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:56.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.181+0000 7f8d98e97700 1 --2- 172.21.15.3:0/4244064115 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d94059070 0x7f8d94131850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:56.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.181+0000 7f8d98e97700 1 --2- 172.21.15.3:0/4244064115 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d9412b2b0 0x7f8d9412c680 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:56.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.181+0000 7f8d98e97700 1 -- 172.21.15.3:0/4244064115 >> 172.21.15.3:0/4244064115 conn(0x7f8d94063560 msgr2=0x7f8d94065e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:56.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.181+0000 7f8d98e97700 1 -- 172.21.15.3:0/4244064115 shutdown_connections 2023-10-03T10:09:56.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.181+0000 7f8d98e97700 1 -- 172.21.15.3:0/4244064115 wait complete. 2023-10-03T10:09:56.194 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.1 2023-10-03T10:09:56.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.205+0000 7f3a617fe700 1 -- 172.21.15.3:0/965140839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3a5c115af0 con 0x7f3a5c068e70 2023-10-03T10:09:56.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.206+0000 7f3a42ffd700 1 -- 172.21.15.3:0/965140839 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3a4c018020 con 0x7f3a5c068e70 2023-10-03T10:09:56.206 INFO:teuthology.orchestra.run.smithi003.stdout:38654705668 2023-10-03T10:09:56.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.208+0000 7f3a617fe700 1 -- 172.21.15.3:0/965140839 >> v1:172.21.15.3:6816/118296 conn(0x7f3a4802bbb0 legacy=0x7f3a4802e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:56.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.208+0000 7f3a617fe700 1 -- 172.21.15.3:0/965140839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a5c068e70 msgr2=0x7f3a5c1161d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:56.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.208+0000 7f3a617fe700 1 --2- 172.21.15.3:0/965140839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a5c068e70 0x7f3a5c1161d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f3a4c00c590 tx=0x7f3a4c007c80).stop 2023-10-03T10:09:56.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.208+0000 7f3a617fe700 1 -- 172.21.15.3:0/965140839 shutdown_connections 2023-10-03T10:09:56.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.208+0000 7f3a617fe700 1 --2- 172.21.15.3:0/965140839 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3a5c12c6a0 0x7f3a5c112690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:56.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.208+0000 7f3a617fe700 1 --2- 172.21.15.3:0/965140839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a5c068e70 0x7f3a5c1161d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:56.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.208+0000 7f3a617fe700 1 --2- 172.21.15.3:0/965140839 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3a5c092460 0x7f3a5c1121a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:56.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.208+0000 7f3a617fe700 1 -- 172.21.15.3:0/965140839 >> 172.21.15.3:0/965140839 conn(0x7f3a5c063560 msgr2=0x7f3a5c098e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:56.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.208+0000 7f3a617fe700 1 -- 172.21.15.3:0/965140839 shutdown_connections 2023-10-03T10:09:56.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.208+0000 7f3a617fe700 1 -- 172.21.15.3:0/965140839 wait complete. 2023-10-03T10:09:56.221 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.2 2023-10-03T10:09:56.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.243+0000 7f4035199700 1 -- 172.21.15.3:0/3463728524 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4030099ee0 con 0x7f4030062a90 2023-10-03T10:09:56.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.244+0000 7f4016ffd700 1 -- 172.21.15.3:0/3463728524 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4030099ee0 con 0x7f4030062a90 2023-10-03T10:09:56.244 INFO:teuthology.orchestra.run.smithi003.stdout:38654705668 2023-10-03T10:09:56.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.246+0000 7f4035199700 1 -- 172.21.15.3:0/3463728524 >> v1:172.21.15.3:6816/118296 conn(0x7f401c02bb90 legacy=0x7f401c02e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:56.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.246+0000 7f4035199700 1 -- 172.21.15.3:0/3463728524 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4030062a90 msgr2=0x7f403012c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:56.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.246+0000 7f4035199700 1 --2- 172.21.15.3:0/3463728524 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4030062a90 0x7f403012c110 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f402000c4c0 tx=0x7f4020007e00).stop 2023-10-03T10:09:56.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.246+0000 7f4035199700 1 -- 172.21.15.3:0/3463728524 shutdown_connections 2023-10-03T10:09:56.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.246+0000 7f4035199700 1 --2- 172.21.15.3:0/3463728524 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4030130c70 0x7f40301310f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:56.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.246+0000 7f4035199700 1 --2- 172.21.15.3:0/3463728524 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4030062a90 0x7f403012c110 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:56.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.246+0000 7f4035199700 1 --2- 172.21.15.3:0/3463728524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f403012f910 0x7f4030130780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:56.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.246+0000 7f4035199700 1 -- 172.21.15.3:0/3463728524 >> 172.21.15.3:0/3463728524 conn(0x7f4030097460 msgr2=0x7f4030093af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:56.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.246+0000 7f4035199700 1 -- 172.21.15.3:0/3463728524 shutdown_connections 2023-10-03T10:09:56.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:56.246+0000 7f4035199700 1 -- 172.21.15.3:0/3463728524 wait complete. 2023-10-03T10:09:56.259 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.0 2023-10-03T10:09:57.147 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:09:57.195 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:09:57.222 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:09:57.260 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:09:57.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.291+0000 7f2bc9d6a700 1 Processor -- start 2023-10-03T10:09:57.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.291+0000 7f2bc9d6a700 1 -- start start 2023-10-03T10:09:57.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.292+0000 7f2bc9d6a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2bc4068e70 0x7f2bc4069250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:57.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.292+0000 7f2bc9d6a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2bc4091ee0 0x7f2bc4069740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:57.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.292+0000 7f2bc9d6a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2bc4133210 0x7f2bc4135600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:57.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.292+0000 7f2bc9d6a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bc4091d00 con 0x7f2bc4091ee0 2023-10-03T10:09:57.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.292+0000 7f2bc9d6a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc4092880 con 0x7f2bc4133210 2023-10-03T10:09:57.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.292+0000 7f2bc9d6a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc4093320 con 0x7f2bc4068e70 2023-10-03T10:09:57.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.292+0000 7f2bc2ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2bc4091ee0 0x7f2bc4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:57.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.292+0000 7f2bc2ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2bc4091ee0 0x7f2bc4069740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47780/0 (socket says 172.21.15.3:47780) 2023-10-03T10:09:57.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.292+0000 7f2bc2ffd700 1 -- 172.21.15.3:0/439593729 learned_addr learned my addr 172.21.15.3:0/439593729 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:57.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.292+0000 7f2bc3fff700 1 --2- 172.21.15.3:0/439593729 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2bc4133210 0x7f2bc4135600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:57.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.292+0000 7f2bc37fe700 1 --2- 172.21.15.3:0/439593729 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2bc4068e70 0x7f2bc4069250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:57.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.293+0000 7f2bc2ffd700 1 -- 172.21.15.3:0/439593729 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2bc4068e70 msgr2=0x7f2bc4069250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:57.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.293+0000 7f2bc2ffd700 1 --2- 172.21.15.3:0/439593729 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2bc4068e70 0x7f2bc4069250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.293+0000 7f2bc2ffd700 1 -- 172.21.15.3:0/439593729 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2bc4133210 msgr2=0x7f2bc4135600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:57.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.293+0000 7f2bc2ffd700 1 --2- 172.21.15.3:0/439593729 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2bc4133210 0x7f2bc4135600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.293+0000 7f2bc2ffd700 1 -- 172.21.15.3:0/439593729 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc4092360 con 0x7f2bc4091ee0 2023-10-03T10:09:57.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.293+0000 7f2bc2ffd700 1 --2- 172.21.15.3:0/439593729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bc4091ee0 0x7f2bc4069740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f2bb8009830 tx=0x7f2bb80090f0).ready entity=mon.0 client_cookie=3e4fcfacda585239 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:57.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.293+0000 7f2bc27fc700 1 -- 172.21.15.3:0/439593729 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb800f510 con 0x7f2bc4091ee0 2023-10-03T10:09:57.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.293+0000 7f2bc27fc700 1 -- 172.21.15.3:0/439593729 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb800f670 con 0x7f2bc4091ee0 2023-10-03T10:09:57.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.293+0000 7f2bc27fc700 1 -- 172.21.15.3:0/439593729 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb800f9f0 con 0x7f2bc4091ee0 2023-10-03T10:09:57.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.294+0000 7f2bc9d6a700 1 -- 172.21.15.3:0/439593729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bc4091ee0 msgr2=0x7f2bc4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:57.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.294+0000 7f2bc9d6a700 1 --2- 172.21.15.3:0/439593729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bc4091ee0 0x7f2bc4069740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f2bb8009830 tx=0x7f2bb80090f0).stop 2023-10-03T10:09:57.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.294+0000 7f2bc9d6a700 1 -- 172.21.15.3:0/439593729 shutdown_connections 2023-10-03T10:09:57.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.294+0000 7f2bc9d6a700 1 --2- 172.21.15.3:0/439593729 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2bc4133210 0x7f2bc4135600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.294+0000 7f2bc9d6a700 1 --2- 172.21.15.3:0/439593729 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2bc4068e70 0x7f2bc4069250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.294+0000 7f2bc9d6a700 1 --2- 172.21.15.3:0/439593729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bc4091ee0 0x7f2bc4069740 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.294+0000 7f2bc9d6a700 1 -- 172.21.15.3:0/439593729 >> 172.21.15.3:0/439593729 conn(0x7f2bc4063560 msgr2=0x7f2bc409a220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:57.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.294+0000 7f2bc9d6a700 1 -- 172.21.15.3:0/439593729 shutdown_connections 2023-10-03T10:09:57.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.294+0000 7f2bc9d6a700 1 -- 172.21.15.3:0/439593729 wait complete. 2023-10-03T10:09:57.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.296+0000 7f2bc9d6a700 1 Processor -- start 2023-10-03T10:09:57.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.296+0000 7f2bc9d6a700 1 -- start start 2023-10-03T10:09:57.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.296+0000 7f2bc9d6a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2bc4068e70 0x7f2bc4114990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:57.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.296+0000 7f2bc9d6a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bc4091ee0 0x7f2bc4131740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:57.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.296+0000 7f2bc9d6a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2bc4133210 0x7f2bc4131c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:57.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.296+0000 7f2bc9d6a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc4115720 con 0x7f2bc4091ee0 2023-10-03T10:09:57.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.296+0000 7f2bc9d6a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc4116f10 con 0x7f2bc4068e70 2023-10-03T10:09:57.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.296+0000 7f2bc9d6a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc4132fe0 con 0x7f2bc4133210 2023-10-03T10:09:57.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.296+0000 7f2bc2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bc4091ee0 0x7f2bc4131740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:57.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.296+0000 7f2bc2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bc4091ee0 0x7f2bc4131740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47784/0 (socket says 172.21.15.3:47784) 2023-10-03T10:09:57.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.296+0000 7f2bc2ffd700 1 -- 172.21.15.3:0/1460091111 learned_addr learned my addr 172.21.15.3:0/1460091111 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:57.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.296+0000 7f2bc37fe700 1 --2- 172.21.15.3:0/1460091111 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2bc4068e70 0x7f2bc4114990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:57.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.296+0000 7f2bc3fff700 1 --2- 172.21.15.3:0/1460091111 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2bc4133210 0x7f2bc4131c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:57.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.297+0000 7f2bc37fe700 1 -- 172.21.15.3:0/1460091111 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2bc4133210 msgr2=0x7f2bc4131c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:57.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.297+0000 7f2bc37fe700 1 --2- 172.21.15.3:0/1460091111 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2bc4133210 0x7f2bc4131c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.297+0000 7f2bc37fe700 1 -- 172.21.15.3:0/1460091111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bc4091ee0 msgr2=0x7f2bc4131740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:57.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.297+0000 7f2bc37fe700 1 --2- 172.21.15.3:0/1460091111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bc4091ee0 0x7f2bc4131740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.297+0000 7f2bc37fe700 1 -- 172.21.15.3:0/1460091111 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb8009370 con 0x7f2bc4068e70 2023-10-03T10:09:57.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.297+0000 7f2bc2ffd700 1 --2- 172.21.15.3:0/1460091111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bc4091ee0 0x7f2bc4131740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:57.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.297+0000 7f2bc37fe700 1 --2- 172.21.15.3:0/1460091111 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2bc4068e70 0x7f2bc4114990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f2bb400e6f0 tx=0x7f2bb4009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:57.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.297+0000 7f2bab7fe700 1 -- 172.21.15.3:0/1460091111 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb4012600 con 0x7f2bc4068e70 2023-10-03T10:09:57.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.297+0000 7f2bab7fe700 1 -- 172.21.15.3:0/1460091111 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb4012760 con 0x7f2bc4068e70 2023-10-03T10:09:57.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.297+0000 7f2bab7fe700 1 -- 172.21.15.3:0/1460091111 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb4012ae0 con 0x7f2bc4068e70 2023-10-03T10:09:57.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.297+0000 7f2bc9d6a700 1 -- 172.21.15.3:0/1460091111 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc4068b60 con 0x7f2bc4068e70 2023-10-03T10:09:57.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.297+0000 7f2bc9d6a700 1 -- 172.21.15.3:0/1460091111 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc4117b90 con 0x7f2bc4068e70 2023-10-03T10:09:57.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.298+0000 7f2bc9d6a700 1 -- 172.21.15.3:0/1460091111 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc4066580 con 0x7f2bc4068e70 2023-10-03T10:09:57.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.299+0000 7f2bab7fe700 1 -- 172.21.15.3:0/1460091111 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f2bb401b400 con 0x7f2bc4068e70 2023-10-03T10:09:57.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.300+0000 7f2bab7fe700 1 -- 172.21.15.3:0/1460091111 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f2bb401b6b0 con 0x7f2bc4068e70 2023-10-03T10:09:57.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.304+0000 7f2bab7fe700 1 -- 172.21.15.3:0/1460091111 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f2bb4040060 con 0x7f2bc4068e70 2023-10-03T10:09:57.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.340+0000 7ff448cff700 1 Processor -- start 2023-10-03T10:09:57.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.340+0000 7ff448cff700 1 -- start start 2023-10-03T10:09:57.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.341+0000 7ff448cff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff444068e70 0x7ff444069250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:57.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.341+0000 7ff448cff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff444091ee0 0x7ff444069740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:57.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.341+0000 7ff448cff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff444133210 0x7ff444135600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:57.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.341+0000 7ff448cff700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff444091d00 con 0x7ff444068e70 2023-10-03T10:09:57.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.341+0000 7ff448cff700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff444092880 con 0x7ff444091ee0 2023-10-03T10:09:57.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.341+0000 7ff448cff700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff444093320 con 0x7ff444133210 2023-10-03T10:09:57.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.341+0000 7ff441d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff444091ee0 0x7ff444069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:57.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.342+0000 7ff441d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff444091ee0 0x7ff444069740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39144/0 (socket says 172.21.15.3:39144) 2023-10-03T10:09:57.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.342+0000 7ff441d9b700 1 -- 172.21.15.3:0/3736173871 learned_addr learned my addr 172.21.15.3:0/3736173871 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:57.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.342+0000 7ff44259c700 1 --2- 172.21.15.3:0/3736173871 >> v2:172.21.15.3:3300/0 conn(0x7ff444068e70 0x7ff444069250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:57.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.342+0000 7ff442d9d700 1 --2- 172.21.15.3:0/3736173871 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff444133210 0x7ff444135600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:57.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.342+0000 7ff442d9d700 1 -- 172.21.15.3:0/3736173871 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff444091ee0 msgr2=0x7ff444069740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:57.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.342+0000 7ff442d9d700 1 --2- 172.21.15.3:0/3736173871 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff444091ee0 0x7ff444069740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.342+0000 7ff442d9d700 1 -- 172.21.15.3:0/3736173871 >> v2:172.21.15.3:3300/0 conn(0x7ff444068e70 msgr2=0x7ff444069250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:57.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.342+0000 7ff442d9d700 1 --2- 172.21.15.3:0/3736173871 >> v2:172.21.15.3:3300/0 conn(0x7ff444068e70 0x7ff444069250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.342+0000 7ff442d9d700 1 -- 172.21.15.3:0/3736173871 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff444092360 con 0x7ff444133210 2023-10-03T10:09:57.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.342+0000 7ff44259c700 1 --2- 172.21.15.3:0/3736173871 >> v2:172.21.15.3:3300/0 conn(0x7ff444068e70 0x7ff444069250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:57.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.342+0000 7ff442d9d700 1 --2- 172.21.15.3:0/3736173871 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff444133210 0x7ff444135600 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7ff43c009e90 tx=0x7ff43c009780).ready entity=mon.2 client_cookie=16f38163bd99ec1a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:57.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.342+0000 7ff44159a700 1 -- 172.21.15.3:0/3736173871 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c00e070 con 0x7ff444133210 2023-10-03T10:09:57.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.343+0000 7ff44159a700 1 -- 172.21.15.3:0/3736173871 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43c00ba70 con 0x7ff444133210 2023-10-03T10:09:57.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.343+0000 7ff44159a700 1 -- 172.21.15.3:0/3736173871 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c00bdf0 con 0x7ff444133210 2023-10-03T10:09:57.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.344+0000 7ff448cff700 1 -- 172.21.15.3:0/3736173871 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff444133210 msgr2=0x7ff444135600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:57.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.344+0000 7ff448cff700 1 --2- 172.21.15.3:0/3736173871 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff444133210 0x7ff444135600 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7ff43c009e90 tx=0x7ff43c009780).stop 2023-10-03T10:09:57.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.344+0000 7ff448cff700 1 -- 172.21.15.3:0/3736173871 shutdown_connections 2023-10-03T10:09:57.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.344+0000 7ff448cff700 1 --2- 172.21.15.3:0/3736173871 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff444133210 0x7ff444135600 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.344+0000 7ff448cff700 1 --2- 172.21.15.3:0/3736173871 >> v2:172.21.15.3:3300/0 conn(0x7ff444068e70 0x7ff444069250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.344+0000 7ff448cff700 1 --2- 172.21.15.3:0/3736173871 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff444091ee0 0x7ff444069740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.344+0000 7ff448cff700 1 -- 172.21.15.3:0/3736173871 >> 172.21.15.3:0/3736173871 conn(0x7ff444063560 msgr2=0x7ff44409a220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:57.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.344+0000 7ff448cff700 1 -- 172.21.15.3:0/3736173871 shutdown_connections 2023-10-03T10:09:57.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.344+0000 7ff448cff700 1 -- 172.21.15.3:0/3736173871 wait complete. 2023-10-03T10:09:57.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.345+0000 7ff448cff700 1 Processor -- start 2023-10-03T10:09:57.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.345+0000 7ff448cff700 1 -- start start 2023-10-03T10:09:57.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.346+0000 7ff448cff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff444068e70 0x7ff44411e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:57.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.346+0000 7ff448cff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff444091ee0 0x7ff44411e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:57.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.346+0000 7ff448cff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff444133210 0x7ff44411e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:57.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.346+0000 7ff448cff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44411eed0 con 0x7ff444133210 2023-10-03T10:09:57.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.346+0000 7ff448cff700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4440686b0 con 0x7ff444068e70 2023-10-03T10:09:57.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.346+0000 7ff448cff700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff444092a40 con 0x7ff444091ee0 2023-10-03T10:09:57.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.346+0000 7ff442d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff444133210 0x7ff44411e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:57.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.346+0000 7ff442d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff444133210 0x7ff44411e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47800/0 (socket says 172.21.15.3:47800) 2023-10-03T10:09:57.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.346+0000 7ff442d9d700 1 -- 172.21.15.3:0/2806339242 learned_addr learned my addr 172.21.15.3:0/2806339242 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:57.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.346+0000 7ff441d9b700 1 --2- 172.21.15.3:0/2806339242 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff444091ee0 0x7ff44411e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:57.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.347+0000 7ff44259c700 1 --2- 172.21.15.3:0/2806339242 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff444068e70 0x7ff44411e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.347+0000 7ff441d9b700 1 -- 172.21.15.3:0/2806339242 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff444068e70 msgr2=0x7ff44411e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.347+0000 7ff441d9b700 1 --2- 172.21.15.3:0/2806339242 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff444068e70 0x7ff44411e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.347+0000 7ff441d9b700 1 -- 172.21.15.3:0/2806339242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff444133210 msgr2=0x7ff44411e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.347+0000 7ff441d9b700 1 --2- 172.21.15.3:0/2806339242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff444133210 0x7ff44411e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.347+0000 7ff441d9b700 1 -- 172.21.15.3:0/2806339242 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4340092a0 con 0x7ff444091ee0 2023-10-03T10:09:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.347+0000 7ff442d9d700 1 --2- 172.21.15.3:0/2806339242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff444133210 0x7ff44411e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.347+0000 7ff441d9b700 1 --2- 172.21.15.3:0/2806339242 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff444091ee0 0x7ff44411e4f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7ff43800e900 tx=0x7ff4380090f0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.347+0000 7ff4327fc700 1 -- 172.21.15.3:0/2806339242 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff438009410 con 0x7ff444091ee0 2023-10-03T10:09:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.347+0000 7ff4327fc700 1 -- 172.21.15.3:0/2806339242 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff438012600 con 0x7ff444091ee0 2023-10-03T10:09:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.347+0000 7ff448cff700 1 -- 172.21.15.3:0/2806339242 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff43c009a20 con 0x7ff444091ee0 2023-10-03T10:09:57.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.347+0000 7ff4327fc700 1 -- 172.21.15.3:0/2806339242 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff438012980 con 0x7ff444091ee0 2023-10-03T10:09:57.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.347+0000 7ff448cff700 1 -- 172.21.15.3:0/2806339242 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff444121b30 con 0x7ff444091ee0 2023-10-03T10:09:57.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.348+0000 7ff4327fc700 1 -- 172.21.15.3:0/2806339242 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7ff438012d30 con 0x7ff444091ee0 2023-10-03T10:09:57.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.348+0000 7ff4327fc700 1 -- 172.21.15.3:0/2806339242 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7ff43803b260 con 0x7ff444091ee0 2023-10-03T10:09:57.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.349+0000 7ff448cff700 1 -- 172.21.15.3:0/2806339242 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4441217b0 con 0x7ff444091ee0 2023-10-03T10:09:57.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.354+0000 7ff4327fc700 1 -- 172.21.15.3:0/2806339242 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7ff438016030 con 0x7ff444091ee0 2023-10-03T10:09:57.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.387+0000 7fc89469f700 1 Processor -- start 2023-10-03T10:09:57.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.387+0000 7fc89469f700 1 -- start start 2023-10-03T10:09:57.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.388+0000 7fc89469f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc88c062a90 0x7fc88c125670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:57.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.388+0000 7fc89469f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc88c12da30 0x7fc88c12fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:57.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.388+0000 7fc89469f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc88c0993f0 con 0x7fc88c125170 2023-10-03T10:09:57.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.388+0000 7fc89469f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc88c062e70 con 0x7fc88c12da30 2023-10-03T10:09:57.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.388+0000 7fc89469f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c099670 con 0x7fc88c062a90 2023-10-03T10:09:57.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.388+0000 7fc89243b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc88c062a90 0x7fc88c125670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:57.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.388+0000 7fc89243b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc88c062a90 0x7fc88c125670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57506/0 (socket says 172.21.15.3:57506) 2023-10-03T10:09:57.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.388+0000 7fc89243b700 1 -- 172.21.15.3:0/2850113318 learned_addr learned my addr 172.21.15.3:0/2850113318 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:57.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.388+0000 7fc892c3c700 1 --2- 172.21.15.3:0/2850113318 >> v2:172.21.15.3:3300/0 conn(0x7fc88c12da30 0x7fc88c12fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:57.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.389+0000 7fc89243b700 1 -- 172.21.15.3:0/2850113318 >> v2:172.21.15.3:3300/0 conn(0x7fc88c12da30 msgr2=0x7fc88c12fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:57.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.389+0000 7fc89243b700 1 --2- 172.21.15.3:0/2850113318 >> v2:172.21.15.3:3300/0 conn(0x7fc88c12da30 0x7fc88c12fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.389+0000 7fc89243b700 1 -- 172.21.15.3:0/2850113318 >> v1:172.21.15.3:6789/0 conn(0x7fc88c125170 legacy=0x7fc88c125b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:57.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.389+0000 7fc89243b700 1 -- 172.21.15.3:0/2850113318 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88c098bf0 con 0x7fc88c062a90 2023-10-03T10:09:57.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.389+0000 7fc891439700 1 -- 172.21.15.3:0/2850113318 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4108850535 0 0) 0x7fc88c0993f0 con 0x7fc88c125170 2023-10-03T10:09:57.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.389+0000 7fc89243b700 1 --2- 172.21.15.3:0/2850113318 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc88c062a90 0x7fc88c125670 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fc888006fd0 tx=0x7fc8880090f0).ready entity=mon.2 client_cookie=63e463dabf1f5b5e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:57.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.390+0000 7fc891439700 1 -- 172.21.15.3:0/2850113318 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88800b580 con 0x7fc88c062a90 2023-10-03T10:09:57.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.390+0000 7fc891439700 1 -- 172.21.15.3:0/2850113318 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88800b6e0 con 0x7fc88c062a90 2023-10-03T10:09:57.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.390+0000 7fc891439700 1 -- 172.21.15.3:0/2850113318 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88800ba60 con 0x7fc88c062a90 2023-10-03T10:09:57.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.391+0000 7fc89469f700 1 -- 172.21.15.3:0/2850113318 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc88c062a90 msgr2=0x7fc88c125670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:57.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.391+0000 7fc89469f700 1 --2- 172.21.15.3:0/2850113318 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc88c062a90 0x7fc88c125670 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fc888006fd0 tx=0x7fc8880090f0).stop 2023-10-03T10:09:57.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.391+0000 7fc89469f700 1 -- 172.21.15.3:0/2850113318 shutdown_connections 2023-10-03T10:09:57.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.391+0000 7fc89469f700 1 --2- 172.21.15.3:0/2850113318 >> v2:172.21.15.3:3300/0 conn(0x7fc88c12da30 0x7fc88c12fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.391+0000 7fc89469f700 1 --2- 172.21.15.3:0/2850113318 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc88c062a90 0x7fc88c125670 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.391+0000 7fc89469f700 1 -- 172.21.15.3:0/2850113318 >> 172.21.15.3:0/2850113318 conn(0x7fc88c097460 msgr2=0x7fc88c097870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:57.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.391+0000 7fc89469f700 1 -- 172.21.15.3:0/2850113318 shutdown_connections 2023-10-03T10:09:57.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.391+0000 7fc89469f700 1 -- 172.21.15.3:0/2850113318 wait complete. 2023-10-03T10:09:57.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.392+0000 7fc89469f700 1 Processor -- start 2023-10-03T10:09:57.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.393+0000 7fc89469f700 1 -- start start 2023-10-03T10:09:57.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.393+0000 7fc89469f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc88c062a90 0x7fc88c12d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:57.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.393+0000 7fc89469f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc88c125170 0x7fc88c12bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:57.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.393+0000 7fc89469f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc88c12da30 0x7fc88c12c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:57.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.393+0000 7fc89469f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c12cbc0 con 0x7fc88c062a90 2023-10-03T10:09:57.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.393+0000 7fc89469f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c12d8d0 con 0x7fc88c125170 2023-10-03T10:09:57.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.393+0000 7fc89469f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c124f60 con 0x7fc88c12da30 2023-10-03T10:09:57.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.393+0000 7fc89243b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc88c062a90 0x7fc88c12d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:57.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.393+0000 7fc89243b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc88c062a90 0x7fc88c12d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47820/0 (socket says 172.21.15.3:47820) 2023-10-03T10:09:57.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.393+0000 7fc89243b700 1 -- 172.21.15.3:0/3496289473 learned_addr learned my addr 172.21.15.3:0/3496289473 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:57.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.394+0000 7fc892c3c700 1 --2- 172.21.15.3:0/3496289473 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc88c12da30 0x7fc88c12c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:57.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.394+0000 7fc891c3a700 1 --2- 172.21.15.3:0/3496289473 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc88c125170 0x7fc88c12bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:57.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.394+0000 7fc892c3c700 1 -- 172.21.15.3:0/3496289473 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc88c125170 msgr2=0x7fc88c12bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:57.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.394+0000 7fc892c3c700 1 --2- 172.21.15.3:0/3496289473 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc88c125170 0x7fc88c12bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.394+0000 7fc892c3c700 1 -- 172.21.15.3:0/3496289473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc88c062a90 msgr2=0x7fc88c12d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:57.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.394+0000 7fc892c3c700 1 --2- 172.21.15.3:0/3496289473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc88c062a90 0x7fc88c12d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.394+0000 7fc892c3c700 1 -- 172.21.15.3:0/3496289473 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc888009370 con 0x7fc88c12da30 2023-10-03T10:09:57.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.394+0000 7fc892c3c700 1 --2- 172.21.15.3:0/3496289473 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc88c12da30 0x7fc88c12c030 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fc88c094c80 tx=0x7fc88400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:57.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.394+0000 7fc8827fc700 1 -- 172.21.15.3:0/3496289473 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88400caa0 con 0x7fc88c12da30 2023-10-03T10:09:57.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.394+0000 7fc8827fc700 1 -- 172.21.15.3:0/3496289473 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88400f040 con 0x7fc88c12da30 2023-10-03T10:09:57.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.394+0000 7fc89469f700 1 -- 172.21.15.3:0/3496289473 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc88c069e60 con 0x7fc88c12da30 2023-10-03T10:09:57.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.394+0000 7fc8827fc700 1 -- 172.21.15.3:0/3496289473 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88400ce10 con 0x7fc88c12da30 2023-10-03T10:09:57.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.394+0000 7fc89469f700 1 -- 172.21.15.3:0/3496289473 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc88c12b640 con 0x7fc88c12da30 2023-10-03T10:09:57.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.395+0000 7fc89469f700 1 -- 172.21.15.3:0/3496289473 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc88c12ad90 con 0x7fc88c12da30 2023-10-03T10:09:57.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.396+0000 7fc8827fc700 1 -- 172.21.15.3:0/3496289473 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fc884019070 con 0x7fc88c12da30 2023-10-03T10:09:57.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.396+0000 7fc8827fc700 1 -- 172.21.15.3:0/3496289473 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fc88403f8b0 con 0x7fc88c12da30 2023-10-03T10:09:57.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.404+0000 7fc8827fc700 1 -- 172.21.15.3:0/3496289473 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fc88401a960 con 0x7fc88c12da30 2023-10-03T10:09:57.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.431+0000 7f53da1a8700 1 Processor -- start 2023-10-03T10:09:57.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.431+0000 7f53da1a8700 1 -- start start 2023-10-03T10:09:57.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.431+0000 7f53da1a8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f53d4062a90 0x7f53d412f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:57.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.431+0000 7f53da1a8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53d412ed70 0x7f53d412f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:57.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.431+0000 7f53da1a8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d4099ee0 con 0x7f53d4098820 2023-10-03T10:09:57.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.431+0000 7f53da1a8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d4062e70 con 0x7f53d4062a90 2023-10-03T10:09:57.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.431+0000 7f53da1a8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53d4091ce0 con 0x7f53d412ed70 2023-10-03T10:09:57.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.432+0000 7f53d2ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53d412ed70 0x7f53d412f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:57.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.432+0000 7f53d37fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f53d4062a90 0x7f53d412f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:57.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.432+0000 7f53d2ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53d412ed70 0x7f53d412f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57528/0 (socket says 172.21.15.3:57528) 2023-10-03T10:09:57.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.432+0000 7f53d2ffd700 1 -- 172.21.15.3:0/3075203523 learned_addr learned my addr 172.21.15.3:0/3075203523 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:57.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.432+0000 7f53d37fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f53d4062a90 0x7f53d412f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47828/0 (socket says 172.21.15.3:47828) 2023-10-03T10:09:57.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.432+0000 7f53d2ffd700 1 -- 172.21.15.3:0/3075203523 >> v2:172.21.15.3:3300/0 conn(0x7f53d4062a90 msgr2=0x7f53d412f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:57.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.432+0000 7f53d2ffd700 1 --2- 172.21.15.3:0/3075203523 >> v2:172.21.15.3:3300/0 conn(0x7f53d4062a90 0x7f53d412f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.432+0000 7f53d2ffd700 1 -- 172.21.15.3:0/3075203523 >> v1:172.21.15.3:6789/0 conn(0x7f53d4098820 legacy=0x7f53d4098cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:57.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.432+0000 7f53d2ffd700 1 -- 172.21.15.3:0/3075203523 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d4091a40 con 0x7f53d412ed70 2023-10-03T10:09:57.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.432+0000 7f53d2ffd700 1 --2- 172.21.15.3:0/3075203523 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53d412ed70 0x7f53d412f6f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f53c8006fd0 tx=0x7f53c80090f0).ready entity=mon.2 client_cookie=975ea1592dedd72f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:57.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.432+0000 7f53d27fc700 1 -- 172.21.15.3:0/3075203523 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c800b580 con 0x7f53d412ed70 2023-10-03T10:09:57.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.433+0000 7f53d27fc700 1 -- 172.21.15.3:0/3075203523 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c800b6e0 con 0x7f53d412ed70 2023-10-03T10:09:57.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.433+0000 7f53d27fc700 1 -- 172.21.15.3:0/3075203523 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c800ba60 con 0x7f53d412ed70 2023-10-03T10:09:57.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.433+0000 7f53da1a8700 1 -- 172.21.15.3:0/3075203523 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53d412ed70 msgr2=0x7f53d412f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:57.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.433+0000 7f53da1a8700 1 --2- 172.21.15.3:0/3075203523 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53d412ed70 0x7f53d412f6f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f53c8006fd0 tx=0x7f53c80090f0).stop 2023-10-03T10:09:57.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.433+0000 7f53da1a8700 1 -- 172.21.15.3:0/3075203523 shutdown_connections 2023-10-03T10:09:57.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.433+0000 7f53da1a8700 1 --2- 172.21.15.3:0/3075203523 >> v2:172.21.15.3:3300/0 conn(0x7f53d4062a90 0x7f53d412f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.433+0000 7f53da1a8700 1 --2- 172.21.15.3:0/3075203523 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53d412ed70 0x7f53d412f6f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.433+0000 7f53da1a8700 1 -- 172.21.15.3:0/3075203523 >> 172.21.15.3:0/3075203523 conn(0x7f53d4097460 msgr2=0x7f53d4097870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:57.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.433+0000 7f53da1a8700 1 -- 172.21.15.3:0/3075203523 shutdown_connections 2023-10-03T10:09:57.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.433+0000 7f53da1a8700 1 -- 172.21.15.3:0/3075203523 wait complete. 2023-10-03T10:09:57.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.435+0000 7f53da1a8700 1 Processor -- start 2023-10-03T10:09:57.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.435+0000 7f53da1a8700 1 -- start start 2023-10-03T10:09:57.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.436+0000 7f53da1a8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53d4062a90 0x7f53d412c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:57.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.436+0000 7f53da1a8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f53d4098820 0x7f53d41375a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:57.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.436+0000 7f53da1a8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53d4137a90 0x7f53d412b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:57.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.436+0000 7f53da1a8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d4132db0 con 0x7f53d4137a90 2023-10-03T10:09:57.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.436+0000 7f53da1a8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d412e850 con 0x7f53d4098820 2023-10-03T10:09:57.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.436+0000 7f53da1a8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53d409a080 con 0x7f53d4062a90 2023-10-03T10:09:57.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.436+0000 7f53d37fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53d4062a90 0x7f53d412c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:57.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.436+0000 7f53d37fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53d4062a90 0x7f53d412c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57544/0 (socket says 172.21.15.3:57544) 2023-10-03T10:09:57.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.436+0000 7f53d37fe700 1 -- 172.21.15.3:0/3269498805 learned_addr learned my addr 172.21.15.3:0/3269498805 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:57.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.436+0000 7f53d3fff700 1 --2- 172.21.15.3:0/3269498805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53d4137a90 0x7f53d412b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:57.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.436+0000 7f53d37fe700 1 -- 172.21.15.3:0/3269498805 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f53d4098820 msgr2=0x7f53d41375a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:57.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.436+0000 7f53d37fe700 1 --2- 172.21.15.3:0/3269498805 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f53d4098820 0x7f53d41375a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-10-03T10:09:57.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.436+0000 7f53d37fe700 1 -- 172.21.15.3:0/3269498805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53d4137a90 msgr2=0x7f53d412b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:57.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.436+0000 7f53d37fe700 1 --2- 172.21.15.3:0/3269498805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53d4137a90 0x7f53d412b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.436+0000 7f53d37fe700 1 -- 172.21.15.3:0/3269498805 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c8009370 con 0x7f53d4062a90 2023-10-03T10:09:57.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.436+0000 7f53d3fff700 1 --2- 172.21.15.3:0/3269498805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53d4137a90 0x7f53d412b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:57.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.437+0000 7f53d37fe700 1 --2- 172.21.15.3:0/3269498805 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53d4062a90 0x7f53d412c3c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f53c400e930 tx=0x7f53c4009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:57.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.437+0000 7f53bb7fe700 1 -- 172.21.15.3:0/3269498805 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c400c670 con 0x7f53d4062a90 2023-10-03T10:09:57.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.437+0000 7f53bb7fe700 1 -- 172.21.15.3:0/3269498805 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c400c7d0 con 0x7f53d4062a90 2023-10-03T10:09:57.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.437+0000 7f53bb7fe700 1 -- 172.21.15.3:0/3269498805 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c400cb50 con 0x7f53d4062a90 2023-10-03T10:09:57.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.437+0000 7f53da1a8700 1 -- 172.21.15.3:0/3269498805 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d4137f10 con 0x7f53d4062a90 2023-10-03T10:09:57.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.437+0000 7f53da1a8700 1 -- 172.21.15.3:0/3269498805 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d4135c90 con 0x7f53d4062a90 2023-10-03T10:09:57.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.438+0000 7f53da1a8700 1 -- 172.21.15.3:0/3269498805 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d405a260 con 0x7f53d4062a90 2023-10-03T10:09:57.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.439+0000 7f53bb7fe700 1 -- 172.21.15.3:0/3269498805 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f53c401b400 con 0x7f53d4062a90 2023-10-03T10:09:57.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.439+0000 7f53bb7fe700 1 -- 172.21.15.3:0/3269498805 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f53c403bb90 con 0x7f53d4062a90 2023-10-03T10:09:57.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.447+0000 7f53bb7fe700 1 -- 172.21.15.3:0/3269498805 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f53c4011940 con 0x7f53d4062a90 2023-10-03T10:09:57.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.509+0000 7f2bc9d6a700 1 -- 172.21.15.3:0/1460091111 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2bc4117410 con 0x7f2bc4068e70 2023-10-03T10:09:57.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.509+0000 7f2bab7fe700 1 -- 172.21.15.3:0/1460091111 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2bb4040070 con 0x7f2bc4068e70 2023-10-03T10:09:57.510 INFO:teuthology.orchestra.run.smithi003.stdout:38654705669 2023-10-03T10:09:57.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.511+0000 7f2bc9d6a700 1 -- 172.21.15.3:0/1460091111 >> v1:172.21.15.3:6816/118296 conn(0x7f2bb002bb60 legacy=0x7f2bb002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:57.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.511+0000 7f2bc9d6a700 1 -- 172.21.15.3:0/1460091111 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2bc4068e70 msgr2=0x7f2bc4114990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:57.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.511+0000 7f2bc9d6a700 1 --2- 172.21.15.3:0/1460091111 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2bc4068e70 0x7f2bc4114990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f2bb400e6f0 tx=0x7f2bb4009060).stop 2023-10-03T10:09:57.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.512+0000 7f2bc9d6a700 1 -- 172.21.15.3:0/1460091111 shutdown_connections 2023-10-03T10:09:57.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.512+0000 7f2bc9d6a700 1 --2- 172.21.15.3:0/1460091111 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2bc4133210 0x7f2bc4131c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.512+0000 7f2bc9d6a700 1 --2- 172.21.15.3:0/1460091111 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2bc4068e70 0x7f2bc4114990 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.512+0000 7f2bc9d6a700 1 --2- 172.21.15.3:0/1460091111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bc4091ee0 0x7f2bc4131740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.512+0000 7f2bc9d6a700 1 -- 172.21.15.3:0/1460091111 >> 172.21.15.3:0/1460091111 conn(0x7f2bc4063560 msgr2=0x7f2bc4134480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:57.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.512+0000 7f2bc9d6a700 1 -- 172.21.15.3:0/1460091111 shutdown_connections 2023-10-03T10:09:57.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.512+0000 7f2bc9d6a700 1 -- 172.21.15.3:0/1460091111 wait complete. 2023-10-03T10:09:57.526 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.3 2023-10-03T10:09:57.526 DEBUG:teuthology.parallel:result is None 2023-10-03T10:09:57.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.557+0000 7ff448cff700 1 -- 172.21.15.3:0/2806339242 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff44405a260 con 0x7ff444091ee0 2023-10-03T10:09:57.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.557+0000 7ff4327fc700 1 -- 172.21.15.3:0/2806339242 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff438019310 con 0x7ff444091ee0 2023-10-03T10:09:57.558 INFO:teuthology.orchestra.run.smithi003.stdout:38654705669 2023-10-03T10:09:57.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.559+0000 7ff448cff700 1 -- 172.21.15.3:0/2806339242 >> v1:172.21.15.3:6816/118296 conn(0x7ff42402bb60 legacy=0x7ff42402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:57.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.559+0000 7ff448cff700 1 -- 172.21.15.3:0/2806339242 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff444091ee0 msgr2=0x7ff44411e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:57.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.559+0000 7ff448cff700 1 --2- 172.21.15.3:0/2806339242 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff444091ee0 0x7ff44411e4f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7ff43800e900 tx=0x7ff4380090f0).stop 2023-10-03T10:09:57.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.559+0000 7ff448cff700 1 -- 172.21.15.3:0/2806339242 shutdown_connections 2023-10-03T10:09:57.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.559+0000 7ff448cff700 1 --2- 172.21.15.3:0/2806339242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff444133210 0x7ff44411e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.559+0000 7ff448cff700 1 --2- 172.21.15.3:0/2806339242 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff444068e70 0x7ff44411e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.559+0000 7ff448cff700 1 --2- 172.21.15.3:0/2806339242 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff444091ee0 0x7ff44411e4f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.559+0000 7ff448cff700 1 -- 172.21.15.3:0/2806339242 >> 172.21.15.3:0/2806339242 conn(0x7ff444063560 msgr2=0x7ff444066ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:57.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.559+0000 7ff448cff700 1 -- 172.21.15.3:0/2806339242 shutdown_connections 2023-10-03T10:09:57.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.559+0000 7ff448cff700 1 -- 172.21.15.3:0/2806339242 wait complete. 2023-10-03T10:09:57.574 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.1 2023-10-03T10:09:57.574 DEBUG:teuthology.parallel:result is None 2023-10-03T10:09:57.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.595+0000 7fc89469f700 1 -- 172.21.15.3:0/3496289473 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc88c05a260 con 0x7fc88c12da30 2023-10-03T10:09:57.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.595+0000 7fc8827fc700 1 -- 172.21.15.3:0/3496289473 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc88401a370 con 0x7fc88c12da30 2023-10-03T10:09:57.596 INFO:teuthology.orchestra.run.smithi003.stdout:38654705669 2023-10-03T10:09:57.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.597+0000 7fc89469f700 1 -- 172.21.15.3:0/3496289473 >> v1:172.21.15.3:6816/118296 conn(0x7fc870034370 legacy=0x7fc870036830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:57.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.597+0000 7fc89469f700 1 -- 172.21.15.3:0/3496289473 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc88c12da30 msgr2=0x7fc88c12c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:57.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.597+0000 7fc89469f700 1 --2- 172.21.15.3:0/3496289473 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc88c12da30 0x7fc88c12c030 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fc88c094c80 tx=0x7fc88400b040).stop 2023-10-03T10:09:57.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.597+0000 7fc89469f700 1 -- 172.21.15.3:0/3496289473 shutdown_connections 2023-10-03T10:09:57.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.597+0000 7fc89469f700 1 --2- 172.21.15.3:0/3496289473 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc88c12da30 0x7fc88c12c030 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.597+0000 7fc89469f700 1 --2- 172.21.15.3:0/3496289473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc88c062a90 0x7fc88c12d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.597+0000 7fc89469f700 1 --2- 172.21.15.3:0/3496289473 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc88c125170 0x7fc88c12bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.597+0000 7fc89469f700 1 -- 172.21.15.3:0/3496289473 >> 172.21.15.3:0/3496289473 conn(0x7fc88c097460 msgr2=0x7fc88c12f590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:57.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.597+0000 7fc89469f700 1 -- 172.21.15.3:0/3496289473 shutdown_connections 2023-10-03T10:09:57.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.597+0000 7fc89469f700 1 -- 172.21.15.3:0/3496289473 wait complete. 2023-10-03T10:09:57.611 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.2 2023-10-03T10:09:57.611 DEBUG:teuthology.parallel:result is None 2023-10-03T10:09:57.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.633+0000 7f53da1a8700 1 -- 172.21.15.3:0/3269498805 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f53d4046c20 con 0x7f53d4062a90 2023-10-03T10:09:57.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.634+0000 7f53bb7fe700 1 -- 172.21.15.3:0/3269498805 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f53c4029070 con 0x7f53d4062a90 2023-10-03T10:09:57.634 INFO:teuthology.orchestra.run.smithi003.stdout:38654705670 2023-10-03T10:09:57.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.635+0000 7f53da1a8700 1 -- 172.21.15.3:0/3269498805 >> v1:172.21.15.3:6816/118296 conn(0x7f53c002bb60 legacy=0x7f53c002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:57.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.635+0000 7f53da1a8700 1 -- 172.21.15.3:0/3269498805 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53d4062a90 msgr2=0x7f53d412c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:57.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.635+0000 7f53da1a8700 1 --2- 172.21.15.3:0/3269498805 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53d4062a90 0x7f53d412c3c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f53c400e930 tx=0x7f53c4009060).stop 2023-10-03T10:09:57.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.636+0000 7f53da1a8700 1 -- 172.21.15.3:0/3269498805 shutdown_connections 2023-10-03T10:09:57.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.636+0000 7f53da1a8700 1 --2- 172.21.15.3:0/3269498805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53d4137a90 0x7f53d412b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.636+0000 7f53da1a8700 1 --2- 172.21.15.3:0/3269498805 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53d4062a90 0x7f53d412c3c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.636+0000 7f53da1a8700 1 --2- 172.21.15.3:0/3269498805 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f53d4098820 0x7f53d41375a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-10-03T10:09:57.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.636+0000 7f53da1a8700 1 -- 172.21.15.3:0/3269498805 >> 172.21.15.3:0/3269498805 conn(0x7f53d4097460 msgr2=0x7f53d4096830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:57.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.636+0000 7f53da1a8700 1 -- 172.21.15.3:0/3269498805 shutdown_connections 2023-10-03T10:09:57.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.636+0000 7f53da1a8700 1 -- 172.21.15.3:0/3269498805 wait complete. 2023-10-03T10:09:57.649 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705670 for osd.0 2023-10-03T10:09:57.649 DEBUG:teuthology.parallel:result is None 2023-10-03T10:09:57.649 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-03T10:09:57.649 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-03T10:09:57.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.822+0000 7f3543d62700 1 Processor -- start 2023-10-03T10:09:57.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.822+0000 7f3543d62700 1 -- start start 2023-10-03T10:09:57.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.823+0000 7f3543d62700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f353c062a90 0x7f353c127860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:57.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.823+0000 7f3543d62700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f353c127360 0x7f353c127d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:57.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.823+0000 7f3543d62700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f353c0968b0 con 0x7f353c095300 2023-10-03T10:09:57.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.823+0000 7f3543d62700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f353c062e70 con 0x7f353c127360 2023-10-03T10:09:57.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.823+0000 7f3543d62700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f353c091bc0 con 0x7f353c062a90 2023-10-03T10:09:57.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.823+0000 7f3541afe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f353c062a90 0x7f353c127860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:57.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.823+0000 7f35412fd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f353c127360 0x7f353c127d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:57.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.823+0000 7f3541afe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f353c062a90 0x7f353c127860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39184/0 (socket says 172.21.15.3:39184) 2023-10-03T10:09:57.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.823+0000 7f3541afe700 1 -- 172.21.15.3:0/4103922519 learned_addr learned my addr 172.21.15.3:0/4103922519 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:57.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.823+0000 7f35422ff700 1 --1- 172.21.15.3:0/4103922519 >> v1:172.21.15.3:6789/0 conn(0x7f353c095300 0x7f353c0957b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38558/0 (socket says 172.21.15.3:38558) 2023-10-03T10:09:57.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.823+0000 7f35412fd700 1 -- 172.21.15.3:0/4103922519 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f353c062a90 msgr2=0x7f353c127860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:57.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.823+0000 7f35412fd700 1 --2- 172.21.15.3:0/4103922519 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f353c062a90 0x7f353c127860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.823+0000 7f35412fd700 1 -- 172.21.15.3:0/4103922519 >> v1:172.21.15.3:6789/0 conn(0x7f353c095300 legacy=0x7f353c0957b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:57.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.823+0000 7f35412fd700 1 -- 172.21.15.3:0/4103922519 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f353c092a70 con 0x7f353c127360 2023-10-03T10:09:57.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.823+0000 7f3540afc700 1 -- 172.21.15.3:0/4103922519 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390235256 0 0) 0x7f3534001490 con 0x7f353c095300 2023-10-03T10:09:57.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.823+0000 7f35412fd700 1 --2- 172.21.15.3:0/4103922519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f353c127360 0x7f353c127d50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f352c009790 tx=0x7f352c009060).ready entity=mon.0 client_cookie=32c010fbf4369433 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:57.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.824+0000 7f3540afc700 1 -- 172.21.15.3:0/4103922519 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352c0090d0 con 0x7f353c127360 2023-10-03T10:09:57.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.824+0000 7f3540afc700 1 -- 172.21.15.3:0/4103922519 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352c00f510 con 0x7f353c127360 2023-10-03T10:09:57.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.824+0000 7f3540afc700 1 -- 172.21.15.3:0/4103922519 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352c00f890 con 0x7f353c127360 2023-10-03T10:09:57.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.824+0000 7f3543d62700 1 -- 172.21.15.3:0/4103922519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f353c127360 msgr2=0x7f353c127d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:57.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.824+0000 7f3543d62700 1 --2- 172.21.15.3:0/4103922519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f353c127360 0x7f353c127d50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f352c009790 tx=0x7f352c009060).stop 2023-10-03T10:09:57.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.824+0000 7f3543d62700 1 -- 172.21.15.3:0/4103922519 shutdown_connections 2023-10-03T10:09:57.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.824+0000 7f3543d62700 1 --2- 172.21.15.3:0/4103922519 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f353c062a90 0x7f353c127860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.824+0000 7f3543d62700 1 --2- 172.21.15.3:0/4103922519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f353c127360 0x7f353c127d50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.824+0000 7f3543d62700 1 -- 172.21.15.3:0/4103922519 >> 172.21.15.3:0/4103922519 conn(0x7f353c059bb0 msgr2=0x7f353c097400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:57.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.824+0000 7f3543d62700 1 -- 172.21.15.3:0/4103922519 shutdown_connections 2023-10-03T10:09:57.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.824+0000 7f3543d62700 1 -- 172.21.15.3:0/4103922519 wait complete. 2023-10-03T10:09:57.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.825+0000 7f3543d62700 1 Processor -- start 2023-10-03T10:09:57.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.825+0000 7f3543d62700 1 -- start start 2023-10-03T10:09:57.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.826+0000 7f3543d62700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f353c062a90 0x7f353c114f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:57.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.826+0000 7f3543d62700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f353c095300 0x7f353c115480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:57.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.826+0000 7f3543d62700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f353c127360 0x7f353c115970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:57.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.826+0000 7f3543d62700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353c115e60 con 0x7f353c062a90 2023-10-03T10:09:57.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.826+0000 7f3543d62700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f353c117e40 con 0x7f353c095300 2023-10-03T10:09:57.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.826+0000 7f3543d62700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f353c096e90 con 0x7f353c127360 2023-10-03T10:09:57.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.826+0000 7f3541afe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f353c062a90 0x7f353c114f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:57.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.826+0000 7f35412fd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f353c095300 0x7f353c115480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:57.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.826+0000 7f3541afe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f353c062a90 0x7f353c114f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47844/0 (socket says 172.21.15.3:47844) 2023-10-03T10:09:57.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.826+0000 7f35422ff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f353c127360 0x7f353c115970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:57.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.826+0000 7f3541afe700 1 -- 172.21.15.3:0/2025528301 learned_addr learned my addr 172.21.15.3:0/2025528301 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:57.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.826+0000 7f35412fd700 1 -- 172.21.15.3:0/2025528301 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f353c127360 msgr2=0x7f353c115970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:57.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.826+0000 7f35412fd700 1 --2- 172.21.15.3:0/2025528301 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f353c127360 0x7f353c115970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.826+0000 7f35412fd700 1 -- 172.21.15.3:0/2025528301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f353c062a90 msgr2=0x7f353c114f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:57.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.826+0000 7f35412fd700 1 --2- 172.21.15.3:0/2025528301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f353c062a90 0x7f353c114f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:57.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.826+0000 7f35412fd700 1 -- 172.21.15.3:0/2025528301 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352c0092a0 con 0x7f353c095300 2023-10-03T10:09:57.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.826+0000 7f3541afe700 1 --2- 172.21.15.3:0/2025528301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f353c062a90 0x7f353c114f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:57.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.826+0000 7f35412fd700 1 --2- 172.21.15.3:0/2025528301 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f353c095300 0x7f353c115480 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f352c014000 tx=0x7f352c017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:57.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.827+0000 7f3531ffb700 1 -- 172.21.15.3:0/2025528301 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352c00f890 con 0x7f353c095300 2023-10-03T10:09:57.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.827+0000 7f3531ffb700 1 -- 172.21.15.3:0/2025528301 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352c00fc50 con 0x7f353c095300 2023-10-03T10:09:57.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.827+0000 7f3543d62700 1 -- 172.21.15.3:0/2025528301 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f353c118070 con 0x7f353c095300 2023-10-03T10:09:57.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.827+0000 7f3531ffb700 1 -- 172.21.15.3:0/2025528301 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352c01b640 con 0x7f353c095300 2023-10-03T10:09:57.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.827+0000 7f3543d62700 1 -- 172.21.15.3:0/2025528301 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f353c118630 con 0x7f353c095300 2023-10-03T10:09:57.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.828+0000 7f3531ffb700 1 -- 172.21.15.3:0/2025528301 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f352c01ba70 con 0x7f353c095300 2023-10-03T10:09:57.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.828+0000 7f3531ffb700 1 -- 172.21.15.3:0/2025528301 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f352c03c7e0 con 0x7f353c095300 2023-10-03T10:09:57.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.828+0000 7f3543d62700 1 -- 172.21.15.3:0/2025528301 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f353c1268a0 con 0x7f353c095300 2023-10-03T10:09:57.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:57.833+0000 7f3531ffb700 1 -- 172.21.15.3:0/2025528301 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f352c029080 con 0x7f353c095300 2023-10-03T10:09:58.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.005+0000 7f3543d62700 1 -- 172.21.15.3:0/2025528301 --> v1:172.21.15.3:6816/118296 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f353c05a260 con 0x7f352002bb90 2023-10-03T10:09:58.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.012+0000 7f3531ffb700 1 -- 172.21.15.3:0/2025528301 <== mgr.4100 v1:172.21.15.3:6816/118296 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249934 (unknown 2965378022 0 955997573) 0x7f353c05a260 con 0x7f352002bb90 2023-10-03T10:09:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:58.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.015+0000 7f3543d62700 1 -- 172.21.15.3:0/2025528301 >> v1:172.21.15.3:6816/118296 conn(0x7f352002bb90 legacy=0x7f352002e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:58.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.015+0000 7f3543d62700 1 -- 172.21.15.3:0/2025528301 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f353c095300 msgr2=0x7f353c115480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:58.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.015+0000 7f3543d62700 1 --2- 172.21.15.3:0/2025528301 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f353c095300 0x7f353c115480 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f352c014000 tx=0x7f352c017000).stop 2023-10-03T10:09:58.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.015+0000 7f3543d62700 1 -- 172.21.15.3:0/2025528301 shutdown_connections 2023-10-03T10:09:58.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.015+0000 7f3543d62700 1 --2- 172.21.15.3:0/2025528301 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f353c127360 0x7f353c115970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:58.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.015+0000 7f3543d62700 1 --2- 172.21.15.3:0/2025528301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f353c062a90 0x7f353c114f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:58.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.015+0000 7f3543d62700 1 --2- 172.21.15.3:0/2025528301 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f353c095300 0x7f353c115480 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:58.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.015+0000 7f3543d62700 1 -- 172.21.15.3:0/2025528301 >> 172.21.15.3:0/2025528301 conn(0x7f353c059bb0 msgr2=0x7f353c094300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:58.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.015+0000 7f3543d62700 1 -- 172.21.15.3:0/2025528301 shutdown_connections 2023-10-03T10:09:58.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.015+0000 7f3543d62700 1 -- 172.21.15.3:0/2025528301 wait complete. 2023-10-03T10:09:58.018 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-03T10:09:58.028 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-03T10:09:57.531533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1189024,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298336,"statfs":{"total":386547056640,"available":385329496064,"internally_reserved":0,"allocated":655360,"data_stored":193408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002377"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.199042+0000","last_change":"2023-10-03T10:09:44.195682+0000","last_active":"2023-10-03T10:09:44.199042+0000","last_peered":"2023-10-03T10:09:44.199042+0000","last_clean":"2023-10-03T10:09:44.199042+0000","last_became_active":"2023-10-03T10:09:44.195360+0000","last_became_peered":"2023-10-03T10:09:44.195360+0000","last_unstale":"2023-10-03T10:09:44.199042+0000","last_undegraded":"2023-10-03T10:09:44.199042+0000","last_fullsized":"2023-10-03T10:09:44.199042+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180036+0000","last_change":"2023-10-03T10:09:44.178263+0000","last_active":"2023-10-03T10:09:44.180036+0000","last_peered":"2023-10-03T10:09:44.180036+0000","last_clean":"2023-10-03T10:09:44.180036+0000","last_became_active":"2023-10-03T10:09:44.177977+0000","last_became_peered":"2023-10-03T10:09:44.177977+0000","last_unstale":"2023-10-03T10:09:44.180036+0000","last_undegraded":"2023-10-03T10:09:44.180036+0000","last_fullsized":"2023-10-03T10:09:44.180036+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.190395+0000","last_change":"2023-10-03T10:09:44.183364+0000","last_active":"2023-10-03T10:09:44.190395+0000","last_peered":"2023-10-03T10:09:44.190395+0000","last_clean":"2023-10-03T10:09:44.190395+0000","last_became_active":"2023-10-03T10:09:44.183094+0000","last_became_peered":"2023-10-03T10:09:44.183094+0000","last_unstale":"2023-10-03T10:09:44.190395+0000","last_undegraded":"2023-10-03T10:09:44.190395+0000","last_fullsized":"2023-10-03T10:09:44.190395+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.190339+0000","last_change":"2023-10-03T10:09:44.183273+0000","last_active":"2023-10-03T10:09:44.190339+0000","last_peered":"2023-10-03T10:09:44.190339+0000","last_clean":"2023-10-03T10:09:44.190339+0000","last_became_active":"2023-10-03T10:09:44.182451+0000","last_became_peered":"2023-10-03T10:09:44.182451+0000","last_unstale":"2023-10-03T10:09:44.190339+0000","last_undegraded":"2023-10-03T10:09:44.190339+0000","last_fullsized":"2023-10-03T10:09:44.190339+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.174191+0000","last_change":"2023-10-03T10:09:44.168483+0000","last_active":"2023-10-03T10:09:44.174191+0000","last_peered":"2023-10-03T10:09:44.174191+0000","last_clean":"2023-10-03T10:09:44.174191+0000","last_became_active":"2023-10-03T10:09:44.168186+0000","last_became_peered":"2023-10-03T10:09:44.168186+0000","last_unstale":"2023-10-03T10:09:44.174191+0000","last_undegraded":"2023-10-03T10:09:44.174191+0000","last_fullsized":"2023-10-03T10:09:44.174191+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.198209+0000","last_change":"2023-10-03T10:09:44.192967+0000","last_active":"2023-10-03T10:09:44.198209+0000","last_peered":"2023-10-03T10:09:44.198209+0000","last_clean":"2023-10-03T10:09:44.198209+0000","last_became_active":"2023-10-03T10:09:44.190979+0000","last_became_peered":"2023-10-03T10:09:44.190979+0000","last_unstale":"2023-10-03T10:09:44.198209+0000","last_undegraded":"2023-10-03T10:09:44.198209+0000","last_fullsized":"2023-10-03T10:09:44.198209+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.192360+0000","last_change":"2023-10-03T10:09:44.183121+0000","last_active":"2023-10-03T10:09:44.192360+0000","last_peered":"2023-10-03T10:09:44.192360+0000","last_clean":"2023-10-03T10:09:44.192360+0000","last_became_active":"2023-10-03T10:09:44.182647+0000","last_became_peered":"2023-10-03T10:09:44.182647+0000","last_unstale":"2023-10-03T10:09:44.192360+0000","last_undegraded":"2023-10-03T10:09:44.192360+0000","last_fullsized":"2023-10-03T10:09:44.192360+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.187765+0000","last_change":"2023-10-03T10:09:44.173123+0000","last_active":"2023-10-03T10:09:44.187765+0000","last_peered":"2023-10-03T10:09:44.187765+0000","last_clean":"2023-10-03T10:09:44.187765+0000","last_became_active":"2023-10-03T10:09:44.172353+0000","last_became_peered":"2023-10-03T10:09:44.172353+0000","last_unstale":"2023-10-03T10:09:44.187765+0000","last_undegraded":"2023-10-03T10:09:44.187765+0000","last_fullsized":"2023-10-03T10:09:44.187765+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173973+0000","last_change":"2023-10-03T10:09:44.166758+0000","last_active":"2023-10-03T10:09:44.173973+0000","last_peered":"2023-10-03T10:09:44.173973+0000","last_clean":"2023-10-03T10:09:44.173973+0000","last_became_active":"2023-10-03T10:09:44.166415+0000","last_became_peered":"2023-10-03T10:09:44.166415+0000","last_unstale":"2023-10-03T10:09:44.173973+0000","last_undegraded":"2023-10-03T10:09:44.173973+0000","last_fullsized":"2023-10-03T10:09:44.173973+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180446+0000","last_change":"2023-10-03T10:09:44.176833+0000","last_active":"2023-10-03T10:09:44.180446+0000","last_peered":"2023-10-03T10:09:44.180446+0000","last_clean":"2023-10-03T10:09:44.180446+0000","last_became_active":"2023-10-03T10:09:44.176526+0000","last_became_peered":"2023-10-03T10:09:44.176526+0000","last_unstale":"2023-10-03T10:09:44.180446+0000","last_undegraded":"2023-10-03T10:09:44.180446+0000","last_fullsized":"2023-10-03T10:09:44.180446+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180114+0000","last_change":"2023-10-03T10:09:44.175868+0000","last_active":"2023-10-03T10:09:44.180114+0000","last_peered":"2023-10-03T10:09:44.180114+0000","last_clean":"2023-10-03T10:09:44.180114+0000","last_became_active":"2023-10-03T10:09:44.175414+0000","last_became_peered":"2023-10-03T10:09:44.175414+0000","last_unstale":"2023-10-03T10:09:44.180114+0000","last_undegraded":"2023-10-03T10:09:44.180114+0000","last_fullsized":"2023-10-03T10:09:44.180114+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200283+0000","last_change":"2023-10-03T10:09:44.198581+0000","last_active":"2023-10-03T10:09:44.200283+0000","last_peered":"2023-10-03T10:09:44.200283+0000","last_clean":"2023-10-03T10:09:44.200283+0000","last_became_active":"2023-10-03T10:09:44.198314+0000","last_became_peered":"2023-10-03T10:09:44.198314+0000","last_unstale":"2023-10-03T10:09:44.200283+0000","last_undegraded":"2023-10-03T10:09:44.200283+0000","last_fullsized":"2023-10-03T10:09:44.200283+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200379+0000","last_change":"2023-10-03T10:09:44.197915+0000","last_active":"2023-10-03T10:09:44.200379+0000","last_peered":"2023-10-03T10:09:44.200379+0000","last_clean":"2023-10-03T10:09:44.200379+0000","last_became_active":"2023-10-03T10:09:44.197417+0000","last_became_peered":"2023-10-03T10:09:44.197417+0000","last_unstale":"2023-10-03T10:09:44.200379+0000","last_undegraded":"2023-10-03T10:09:44.200379+0000","last_fullsized":"2023-10-03T10:09:44.200379+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180522+0000","last_change":"2023-10-03T10:09:44.176456+0000","last_active":"2023-10-03T10:09:44.180522+0000","last_peered":"2023-10-03T10:09:44.180522+0000","last_clean":"2023-10-03T10:09:44.180522+0000","last_became_active":"2023-10-03T10:09:44.176146+0000","last_became_peered":"2023-10-03T10:09:44.176146+0000","last_unstale":"2023-10-03T10:09:44.180522+0000","last_undegraded":"2023-10-03T10:09:44.180522+0000","last_fullsized":"2023-10-03T10:09:44.180522+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.178380+0000","last_change":"2023-10-03T10:09:44.176614+0000","last_active":"2023-10-03T10:09:44.178380+0000","last_peered":"2023-10-03T10:09:44.178380+0000","last_clean":"2023-10-03T10:09:44.178380+0000","last_became_active":"2023-10-03T10:09:44.176334+0000","last_became_peered":"2023-10-03T10:09:44.176334+0000","last_unstale":"2023-10-03T10:09:44.178380+0000","last_undegraded":"2023-10-03T10:09:44.178380+0000","last_fullsized":"2023-10-03T10:09:44.178380+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.192741+0000","last_change":"2023-10-03T10:09:44.179440+0000","last_active":"2023-10-03T10:09:44.192741+0000","last_peered":"2023-10-03T10:09:44.192741+0000","last_clean":"2023-10-03T10:09:44.192741+0000","last_became_active":"2023-10-03T10:09:44.179150+0000","last_became_peered":"2023-10-03T10:09:44.179150+0000","last_unstale":"2023-10-03T10:09:44.192741+0000","last_undegraded":"2023-10-03T10:09:44.192741+0000","last_fullsized":"2023-10-03T10:09:44.192741+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173996+0000","last_change":"2023-10-03T10:09:44.166901+0000","last_active":"2023-10-03T10:09:44.173996+0000","last_peered":"2023-10-03T10:09:44.173996+0000","last_clean":"2023-10-03T10:09:44.173996+0000","last_became_active":"2023-10-03T10:09:44.166610+0000","last_became_peered":"2023-10-03T10:09:44.166610+0000","last_unstale":"2023-10-03T10:09:44.173996+0000","last_undegraded":"2023-10-03T10:09:44.173996+0000","last_fullsized":"2023-10-03T10:09:44.173996+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173643+0000","last_change":"2023-10-03T10:09:44.165366+0000","last_active":"2023-10-03T10:09:44.173643+0000","last_peered":"2023-10-03T10:09:44.173643+0000","last_clean":"2023-10-03T10:09:44.173643+0000","last_became_active":"2023-10-03T10:09:44.165083+0000","last_became_peered":"2023-10-03T10:09:44.165083+0000","last_unstale":"2023-10-03T10:09:44.173643+0000","last_undegraded":"2023-10-03T10:09:44.173643+0000","last_fullsized":"2023-10-03T10:09:44.173643+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.198990+0000","last_change":"2023-10-03T10:09:44.195696+0000","last_active":"2023-10-03T10:09:44.198990+0000","last_peered":"2023-10-03T10:09:44.198990+0000","last_clean":"2023-10-03T10:09:44.198990+0000","last_became_active":"2023-10-03T10:09:44.195402+0000","last_became_peered":"2023-10-03T10:09:44.195402+0000","last_unstale":"2023-10-03T10:09:44.198990+0000","last_undegraded":"2023-10-03T10:09:44.198990+0000","last_fullsized":"2023-10-03T10:09:44.198990+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184209+0000","last_change":"2023-10-03T10:09:44.177178+0000","last_active":"2023-10-03T10:09:44.184209+0000","last_peered":"2023-10-03T10:09:44.184209+0000","last_clean":"2023-10-03T10:09:44.184209+0000","last_became_active":"2023-10-03T10:09:44.176686+0000","last_became_peered":"2023-10-03T10:09:44.176686+0000","last_unstale":"2023-10-03T10:09:44.184209+0000","last_undegraded":"2023-10-03T10:09:44.184209+0000","last_fullsized":"2023-10-03T10:09:44.184209+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180396+0000","last_change":"2023-10-03T10:09:44.177335+0000","last_active":"2023-10-03T10:09:44.180396+0000","last_peered":"2023-10-03T10:09:44.180396+0000","last_clean":"2023-10-03T10:09:44.180396+0000","last_became_active":"2023-10-03T10:09:44.175202+0000","last_became_peered":"2023-10-03T10:09:44.175202+0000","last_unstale":"2023-10-03T10:09:44.180396+0000","last_undegraded":"2023-10-03T10:09:44.180396+0000","last_fullsized":"2023-10-03T10:09:44.180396+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184702+0000","last_change":"2023-10-03T10:09:44.182207+0000","last_active":"2023-10-03T10:09:44.184702+0000","last_peered":"2023-10-03T10:09:44.184702+0000","last_clean":"2023-10-03T10:09:44.184702+0000","last_became_active":"2023-10-03T10:09:44.181944+0000","last_became_peered":"2023-10-03T10:09:44.181944+0000","last_unstale":"2023-10-03T10:09:44.184702+0000","last_undegraded":"2023-10-03T10:09:44.184702+0000","last_fullsized":"2023-10-03T10:09:44.184702+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.192336+0000","last_change":"2023-10-03T10:09:44.182610+0000","last_active":"2023-10-03T10:09:44.192336+0000","last_peered":"2023-10-03T10:09:44.192336+0000","last_clean":"2023-10-03T10:09:44.192336+0000","last_became_active":"2023-10-03T10:09:44.180732+0000","last_became_peered":"2023-10-03T10:09:44.180732+0000","last_unstale":"2023-10-03T10:09:44.192336+0000","last_undegraded":"2023-10-03T10:09:44.192336+0000","last_fullsized":"2023-10-03T10:09:44.192336+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.187788+0000","last_change":"2023-10-03T10:09:44.173182+0000","last_active":"2023-10-03T10:09:44.187788+0000","last_peered":"2023-10-03T10:09:44.187788+0000","last_clean":"2023-10-03T10:09:44.187788+0000","last_became_active":"2023-10-03T10:09:44.172685+0000","last_became_peered":"2023-10-03T10:09:44.172685+0000","last_unstale":"2023-10-03T10:09:44.187788+0000","last_undegraded":"2023-10-03T10:09:44.187788+0000","last_fullsized":"2023-10-03T10:09:44.187788+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180761+0000","last_change":"2023-10-03T10:09:44.177910+0000","last_active":"2023-10-03T10:09:44.180761+0000","last_peered":"2023-10-03T10:09:44.180761+0000","last_clean":"2023-10-03T10:09:44.180761+0000","last_became_active":"2023-10-03T10:09:44.177618+0000","last_became_peered":"2023-10-03T10:09:44.177618+0000","last_unstale":"2023-10-03T10:09:44.180761+0000","last_undegraded":"2023-10-03T10:09:44.180761+0000","last_fullsized":"2023-10-03T10:09:44.180761+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180424+0000","last_change":"2023-10-03T10:09:44.176287+0000","last_active":"2023-10-03T10:09:44.180424+0000","last_peered":"2023-10-03T10:09:44.180424+0000","last_clean":"2023-10-03T10:09:44.180424+0000","last_became_active":"2023-10-03T10:09:44.175320+0000","last_became_peered":"2023-10-03T10:09:44.175320+0000","last_unstale":"2023-10-03T10:09:44.180424+0000","last_undegraded":"2023-10-03T10:09:44.180424+0000","last_fullsized":"2023-10-03T10:09:44.180424+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.194745+0000","last_change":"2023-10-03T10:09:44.187148+0000","last_active":"2023-10-03T10:09:44.194745+0000","last_peered":"2023-10-03T10:09:44.194745+0000","last_clean":"2023-10-03T10:09:44.194745+0000","last_became_active":"2023-10-03T10:09:44.186523+0000","last_became_peered":"2023-10-03T10:09:44.186523+0000","last_unstale":"2023-10-03T10:09:44.194745+0000","last_undegraded":"2023-10-03T10:09:44.194745+0000","last_fullsized":"2023-10-03T10:09:44.194745+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.189715+0000","last_change":"2023-10-03T10:09:43.745046+0000","last_active":"2023-10-03T10:09:44.189715+0000","last_peered":"2023-10-03T10:09:44.189715+0000","last_clean":"2023-10-03T10:09:44.189715+0000","last_became_active":"2023-10-03T10:09:43.744724+0000","last_became_peered":"2023-10-03T10:09:43.744724+0000","last_unstale":"2023-10-03T10:09:44.189715+0000","last_undegraded":"2023-10-03T10:09:44.189715+0000","last_fullsized":"2023-10-03T10:09:44.189715+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195809+0000","last_change":"2023-10-03T10:09:44.190869+0000","last_active":"2023-10-03T10:09:44.195809+0000","last_peered":"2023-10-03T10:09:44.195809+0000","last_clean":"2023-10-03T10:09:44.195809+0000","last_became_active":"2023-10-03T10:09:44.190207+0000","last_became_peered":"2023-10-03T10:09:44.190207+0000","last_unstale":"2023-10-03T10:09:44.195809+0000","last_undegraded":"2023-10-03T10:09:44.195809+0000","last_fullsized":"2023-10-03T10:09:44.195809+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.152860+0000","last_change":"2023-10-03T10:09:43.750891+0000","last_active":"2023-10-03T10:09:44.152860+0000","last_peered":"2023-10-03T10:09:44.152860+0000","last_clean":"2023-10-03T10:09:44.152860+0000","last_became_active":"2023-10-03T10:09:43.750356+0000","last_became_peered":"2023-10-03T10:09:43.750356+0000","last_unstale":"2023-10-03T10:09:44.152860+0000","last_undegraded":"2023-10-03T10:09:44.152860+0000","last_fullsized":"2023-10-03T10:09:44.152860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184645+0000","last_change":"2023-10-03T10:09:44.183145+0000","last_active":"2023-10-03T10:09:44.184645+0000","last_peered":"2023-10-03T10:09:44.184645+0000","last_clean":"2023-10-03T10:09:44.184645+0000","last_became_active":"2023-10-03T10:09:44.182816+0000","last_became_peered":"2023-10-03T10:09:44.182816+0000","last_unstale":"2023-10-03T10:09:44.184645+0000","last_undegraded":"2023-10-03T10:09:44.184645+0000","last_fullsized":"2023-10-03T10:09:44.184645+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.163680+0000","last_change":"2023-10-03T10:09:43.749289+0000","last_active":"2023-10-03T10:09:44.163680+0000","last_peered":"2023-10-03T10:09:44.163680+0000","last_clean":"2023-10-03T10:09:44.163680+0000","last_became_active":"2023-10-03T10:09:43.748981+0000","last_became_peered":"2023-10-03T10:09:43.748981+0000","last_unstale":"2023-10-03T10:09:44.163680+0000","last_undegraded":"2023-10-03T10:09:44.163680+0000","last_fullsized":"2023-10-03T10:09:44.163680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.197002+0000","last_change":"2023-10-03T10:09:44.192692+0000","last_active":"2023-10-03T10:09:44.197002+0000","last_peered":"2023-10-03T10:09:44.197002+0000","last_clean":"2023-10-03T10:09:44.197002+0000","last_became_active":"2023-10-03T10:09:44.192390+0000","last_became_peered":"2023-10-03T10:09:44.192390+0000","last_unstale":"2023-10-03T10:09:44.197002+0000","last_undegraded":"2023-10-03T10:09:44.197002+0000","last_fullsized":"2023-10-03T10:09:44.197002+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.160877+0000","last_change":"2023-10-03T10:09:43.754802+0000","last_active":"2023-10-03T10:09:44.160877+0000","last_peered":"2023-10-03T10:09:44.160877+0000","last_clean":"2023-10-03T10:09:44.160877+0000","last_became_active":"2023-10-03T10:09:43.754512+0000","last_became_peered":"2023-10-03T10:09:43.754512+0000","last_unstale":"2023-10-03T10:09:44.160877+0000","last_undegraded":"2023-10-03T10:09:44.160877+0000","last_fullsized":"2023-10-03T10:09:44.160877+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196969+0000","last_change":"2023-10-03T10:09:44.192798+0000","last_active":"2023-10-03T10:09:44.196969+0000","last_peered":"2023-10-03T10:09:44.196969+0000","last_clean":"2023-10-03T10:09:44.196969+0000","last_became_active":"2023-10-03T10:09:44.190787+0000","last_became_peered":"2023-10-03T10:09:44.190787+0000","last_unstale":"2023-10-03T10:09:44.196969+0000","last_undegraded":"2023-10-03T10:09:44.196969+0000","last_fullsized":"2023-10-03T10:09:44.196969+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.157966+0000","last_change":"2023-10-03T10:09:43.756958+0000","last_active":"2023-10-03T10:09:44.157966+0000","last_peered":"2023-10-03T10:09:44.157966+0000","last_clean":"2023-10-03T10:09:44.157966+0000","last_became_active":"2023-10-03T10:09:43.756463+0000","last_became_peered":"2023-10-03T10:09:43.756463+0000","last_unstale":"2023-10-03T10:09:44.157966+0000","last_undegraded":"2023-10-03T10:09:44.157966+0000","last_fullsized":"2023-10-03T10:09:44.157966+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.194551+0000","last_change":"2023-10-03T10:09:44.184790+0000","last_active":"2023-10-03T10:09:44.194551+0000","last_peered":"2023-10-03T10:09:44.194551+0000","last_clean":"2023-10-03T10:09:44.194551+0000","last_became_active":"2023-10-03T10:09:44.184531+0000","last_became_peered":"2023-10-03T10:09:44.184531+0000","last_unstale":"2023-10-03T10:09:44.194551+0000","last_undegraded":"2023-10-03T10:09:44.194551+0000","last_fullsized":"2023-10-03T10:09:44.194551+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"16'3","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200426+0000","last_change":"2023-10-03T10:09:43.756598+0000","last_active":"2023-10-03T10:09:44.200426+0000","last_peered":"2023-10-03T10:09:44.200426+0000","last_clean":"2023-10-03T10:09:44.200426+0000","last_became_active":"2023-10-03T10:09:43.754974+0000","last_became_peered":"2023-10-03T10:09:43.754974+0000","last_unstale":"2023-10-03T10:09:44.200426+0000","last_undegraded":"2023-10-03T10:09:44.200426+0000","last_fullsized":"2023-10-03T10:09:44.200426+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180499+0000","last_change":"2023-10-03T10:09:44.175504+0000","last_active":"2023-10-03T10:09:44.180499+0000","last_peered":"2023-10-03T10:09:44.180499+0000","last_clean":"2023-10-03T10:09:44.180499+0000","last_became_active":"2023-10-03T10:09:44.175174+0000","last_became_peered":"2023-10-03T10:09:44.175174+0000","last_unstale":"2023-10-03T10:09:44.180499+0000","last_undegraded":"2023-10-03T10:09:44.180499+0000","last_fullsized":"2023-10-03T10:09:44.180499+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"16'6","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.203073+0000","last_change":"2023-10-03T10:09:43.758711+0000","last_active":"2023-10-03T10:09:44.203073+0000","last_peered":"2023-10-03T10:09:44.203073+0000","last_clean":"2023-10-03T10:09:44.203073+0000","last_became_active":"2023-10-03T10:09:43.758392+0000","last_became_peered":"2023-10-03T10:09:43.758392+0000","last_unstale":"2023-10-03T10:09:44.203073+0000","last_undegraded":"2023-10-03T10:09:44.203073+0000","last_fullsized":"2023-10-03T10:09:44.203073+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200336+0000","last_change":"2023-10-03T10:09:44.196559+0000","last_active":"2023-10-03T10:09:44.200336+0000","last_peered":"2023-10-03T10:09:44.200336+0000","last_clean":"2023-10-03T10:09:44.200336+0000","last_became_active":"2023-10-03T10:09:44.196280+0000","last_became_peered":"2023-10-03T10:09:44.196280+0000","last_unstale":"2023-10-03T10:09:44.200336+0000","last_undegraded":"2023-10-03T10:09:44.200336+0000","last_fullsized":"2023-10-03T10:09:44.200336+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200762+0000","last_change":"2023-10-03T10:09:43.751877+0000","last_active":"2023-10-03T10:09:44.200762+0000","last_peered":"2023-10-03T10:09:44.200762+0000","last_clean":"2023-10-03T10:09:44.200762+0000","last_became_active":"2023-10-03T10:09:43.749870+0000","last_became_peered":"2023-10-03T10:09:43.749870+0000","last_unstale":"2023-10-03T10:09:44.200762+0000","last_undegraded":"2023-10-03T10:09:44.200762+0000","last_fullsized":"2023-10-03T10:09:44.200762+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195297+0000","last_change":"2023-10-03T10:09:44.190230+0000","last_active":"2023-10-03T10:09:44.195297+0000","last_peered":"2023-10-03T10:09:44.195297+0000","last_clean":"2023-10-03T10:09:44.195297+0000","last_became_active":"2023-10-03T10:09:44.189342+0000","last_became_peered":"2023-10-03T10:09:44.189342+0000","last_unstale":"2023-10-03T10:09:44.195297+0000","last_undegraded":"2023-10-03T10:09:44.195297+0000","last_fullsized":"2023-10-03T10:09:44.195297+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.162475+0000","last_change":"2023-10-03T10:09:43.755271+0000","last_active":"2023-10-03T10:09:44.162475+0000","last_peered":"2023-10-03T10:09:44.162475+0000","last_clean":"2023-10-03T10:09:44.162475+0000","last_became_active":"2023-10-03T10:09:43.754957+0000","last_became_peered":"2023-10-03T10:09:43.754957+0000","last_unstale":"2023-10-03T10:09:44.162475+0000","last_undegraded":"2023-10-03T10:09:44.162475+0000","last_fullsized":"2023-10-03T10:09:44.162475+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.194802+0000","last_change":"2023-10-03T10:09:44.187333+0000","last_active":"2023-10-03T10:09:44.194802+0000","last_peered":"2023-10-03T10:09:44.194802+0000","last_clean":"2023-10-03T10:09:44.194802+0000","last_became_active":"2023-10-03T10:09:44.186824+0000","last_became_peered":"2023-10-03T10:09:44.186824+0000","last_unstale":"2023-10-03T10:09:44.194802+0000","last_undegraded":"2023-10-03T10:09:44.194802+0000","last_fullsized":"2023-10-03T10:09:44.194802+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.163844+0000","last_change":"2023-10-03T10:09:43.758995+0000","last_active":"2023-10-03T10:09:44.163844+0000","last_peered":"2023-10-03T10:09:44.163844+0000","last_clean":"2023-10-03T10:09:44.163844+0000","last_became_active":"2023-10-03T10:09:43.758565+0000","last_became_peered":"2023-10-03T10:09:43.758565+0000","last_unstale":"2023-10-03T10:09:44.163844+0000","last_undegraded":"2023-10-03T10:09:44.163844+0000","last_fullsized":"2023-10-03T10:09:44.163844+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173655+0000","last_change":"2023-10-03T10:09:44.166557+0000","last_active":"2023-10-03T10:09:44.173655+0000","last_peered":"2023-10-03T10:09:44.173655+0000","last_clean":"2023-10-03T10:09:44.173655+0000","last_became_active":"2023-10-03T10:09:44.166300+0000","last_became_peered":"2023-10-03T10:09:44.166300+0000","last_unstale":"2023-10-03T10:09:44.173655+0000","last_undegraded":"2023-10-03T10:09:44.173655+0000","last_fullsized":"2023-10-03T10:09:44.173655+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"16'8","reported_seq":18,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.194737+0000","last_change":"2023-10-03T10:09:43.748084+0000","last_active":"2023-10-03T10:09:44.194737+0000","last_peered":"2023-10-03T10:09:44.194737+0000","last_clean":"2023-10-03T10:09:44.194737+0000","last_became_active":"2023-10-03T10:09:43.747377+0000","last_became_peered":"2023-10-03T10:09:43.747377+0000","last_unstale":"2023-10-03T10:09:44.194737+0000","last_undegraded":"2023-10-03T10:09:44.194737+0000","last_fullsized":"2023-10-03T10:09:44.194737+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195326+0000","last_change":"2023-10-03T10:09:44.190767+0000","last_active":"2023-10-03T10:09:44.195326+0000","last_peered":"2023-10-03T10:09:44.195326+0000","last_clean":"2023-10-03T10:09:44.195326+0000","last_became_active":"2023-10-03T10:09:44.190514+0000","last_became_peered":"2023-10-03T10:09:44.190514+0000","last_unstale":"2023-10-03T10:09:44.195326+0000","last_undegraded":"2023-10-03T10:09:44.195326+0000","last_fullsized":"2023-10-03T10:09:44.195326+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.183000+0000","last_change":"2023-10-03T10:09:43.748461+0000","last_active":"2023-10-03T10:09:44.183000+0000","last_peered":"2023-10-03T10:09:44.183000+0000","last_clean":"2023-10-03T10:09:44.183000+0000","last_became_active":"2023-10-03T10:09:43.747372+0000","last_became_peered":"2023-10-03T10:09:43.747372+0000","last_unstale":"2023-10-03T10:09:44.183000+0000","last_undegraded":"2023-10-03T10:09:44.183000+0000","last_fullsized":"2023-10-03T10:09:44.183000+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196810+0000","last_change":"2023-10-03T10:09:44.192036+0000","last_active":"2023-10-03T10:09:44.196810+0000","last_peered":"2023-10-03T10:09:44.196810+0000","last_clean":"2023-10-03T10:09:44.196810+0000","last_became_active":"2023-10-03T10:09:44.191093+0000","last_became_peered":"2023-10-03T10:09:44.191093+0000","last_unstale":"2023-10-03T10:09:44.196810+0000","last_undegraded":"2023-10-03T10:09:44.196810+0000","last_fullsized":"2023-10-03T10:09:44.196810+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.174713+0000","last_change":"2023-10-03T10:09:43.756751+0000","last_active":"2023-10-03T10:09:44.174713+0000","last_peered":"2023-10-03T10:09:44.174713+0000","last_clean":"2023-10-03T10:09:44.174713+0000","last_became_active":"2023-10-03T10:09:43.756079+0000","last_became_peered":"2023-10-03T10:09:43.756079+0000","last_unstale":"2023-10-03T10:09:44.174713+0000","last_undegraded":"2023-10-03T10:09:44.174713+0000","last_fullsized":"2023-10-03T10:09:44.174713+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.179874+0000","last_change":"2023-10-03T10:09:44.172400+0000","last_active":"2023-10-03T10:09:44.179874+0000","last_peered":"2023-10-03T10:09:44.179874+0000","last_clean":"2023-10-03T10:09:44.179874+0000","last_became_active":"2023-10-03T10:09:44.172095+0000","last_became_peered":"2023-10-03T10:09:44.172095+0000","last_unstale":"2023-10-03T10:09:44.179874+0000","last_undegraded":"2023-10-03T10:09:44.179874+0000","last_fullsized":"2023-10-03T10:09:44.179874+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195109+0000","last_change":"2023-10-03T10:09:43.760622+0000","last_active":"2023-10-03T10:09:44.195109+0000","last_peered":"2023-10-03T10:09:44.195109+0000","last_clean":"2023-10-03T10:09:44.195109+0000","last_became_active":"2023-10-03T10:09:43.760371+0000","last_became_peered":"2023-10-03T10:09:43.760371+0000","last_unstale":"2023-10-03T10:09:44.195109+0000","last_undegraded":"2023-10-03T10:09:44.195109+0000","last_fullsized":"2023-10-03T10:09:44.195109+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200471+0000","last_change":"2023-10-03T10:09:44.196062+0000","last_active":"2023-10-03T10:09:44.200471+0000","last_peered":"2023-10-03T10:09:44.200471+0000","last_clean":"2023-10-03T10:09:44.200471+0000","last_became_active":"2023-10-03T10:09:44.195841+0000","last_became_peered":"2023-10-03T10:09:44.195841+0000","last_unstale":"2023-10-03T10:09:44.200471+0000","last_undegraded":"2023-10-03T10:09:44.200471+0000","last_fullsized":"2023-10-03T10:09:44.200471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.157166+0000","last_change":"2023-10-03T10:09:43.756885+0000","last_active":"2023-10-03T10:09:44.157166+0000","last_peered":"2023-10-03T10:09:44.157166+0000","last_clean":"2023-10-03T10:09:44.157166+0000","last_became_active":"2023-10-03T10:09:43.756588+0000","last_became_peered":"2023-10-03T10:09:43.756588+0000","last_unstale":"2023-10-03T10:09:44.157166+0000","last_undegraded":"2023-10-03T10:09:44.157166+0000","last_fullsized":"2023-10-03T10:09:44.157166+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184514+0000","last_change":"2023-10-03T10:09:44.181603+0000","last_active":"2023-10-03T10:09:44.184514+0000","last_peered":"2023-10-03T10:09:44.184514+0000","last_clean":"2023-10-03T10:09:44.184514+0000","last_became_active":"2023-10-03T10:09:44.181125+0000","last_became_peered":"2023-10-03T10:09:44.181125+0000","last_unstale":"2023-10-03T10:09:44.184514+0000","last_undegraded":"2023-10-03T10:09:44.184514+0000","last_fullsized":"2023-10-03T10:09:44.184514+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200822+0000","last_change":"2023-10-03T10:09:43.751873+0000","last_active":"2023-10-03T10:09:44.200822+0000","last_peered":"2023-10-03T10:09:44.200822+0000","last_clean":"2023-10-03T10:09:44.200822+0000","last_became_active":"2023-10-03T10:09:43.751324+0000","last_became_peered":"2023-10-03T10:09:43.751324+0000","last_unstale":"2023-10-03T10:09:44.200822+0000","last_undegraded":"2023-10-03T10:09:44.200822+0000","last_fullsized":"2023-10-03T10:09:44.200822+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.174302+0000","last_change":"2023-10-03T10:09:44.170676+0000","last_active":"2023-10-03T10:09:44.174302+0000","last_peered":"2023-10-03T10:09:44.174302+0000","last_clean":"2023-10-03T10:09:44.174302+0000","last_became_active":"2023-10-03T10:09:44.170398+0000","last_became_peered":"2023-10-03T10:09:44.170398+0000","last_unstale":"2023-10-03T10:09:44.174302+0000","last_undegraded":"2023-10-03T10:09:44.174302+0000","last_fullsized":"2023-10-03T10:09:44.174302+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.166137+0000","last_change":"2023-10-03T10:09:43.760711+0000","last_active":"2023-10-03T10:09:44.166137+0000","last_peered":"2023-10-03T10:09:44.166137+0000","last_clean":"2023-10-03T10:09:44.166137+0000","last_became_active":"2023-10-03T10:09:43.760432+0000","last_became_peered":"2023-10-03T10:09:43.760432+0000","last_unstale":"2023-10-03T10:09:44.166137+0000","last_undegraded":"2023-10-03T10:09:44.166137+0000","last_fullsized":"2023-10-03T10:09:44.166137+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195829+0000","last_change":"2023-10-03T10:09:44.191073+0000","last_active":"2023-10-03T10:09:44.195829+0000","last_peered":"2023-10-03T10:09:44.195829+0000","last_clean":"2023-10-03T10:09:44.195829+0000","last_became_active":"2023-10-03T10:09:44.190218+0000","last_became_peered":"2023-10-03T10:09:44.190218+0000","last_unstale":"2023-10-03T10:09:44.195829+0000","last_undegraded":"2023-10-03T10:09:44.195829+0000","last_fullsized":"2023-10-03T10:09:44.195829+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.163521+0000","last_change":"2023-10-03T10:09:43.758846+0000","last_active":"2023-10-03T10:09:44.163521+0000","last_peered":"2023-10-03T10:09:44.163521+0000","last_clean":"2023-10-03T10:09:44.163521+0000","last_became_active":"2023-10-03T10:09:43.757973+0000","last_became_peered":"2023-10-03T10:09:43.757973+0000","last_unstale":"2023-10-03T10:09:44.163521+0000","last_undegraded":"2023-10-03T10:09:44.163521+0000","last_fullsized":"2023-10-03T10:09:44.163521+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200102+0000","last_change":"2023-10-03T10:09:44.196056+0000","last_active":"2023-10-03T10:09:44.200102+0000","last_peered":"2023-10-03T10:09:44.200102+0000","last_clean":"2023-10-03T10:09:44.200102+0000","last_became_active":"2023-10-03T10:09:44.195817+0000","last_became_peered":"2023-10-03T10:09:44.195817+0000","last_unstale":"2023-10-03T10:09:44.200102+0000","last_undegraded":"2023-10-03T10:09:44.200102+0000","last_fullsized":"2023-10-03T10:09:44.200102+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.162993+0000","last_change":"2023-10-03T10:09:43.761068+0000","last_active":"2023-10-03T10:09:44.162993+0000","last_peered":"2023-10-03T10:09:44.162993+0000","last_clean":"2023-10-03T10:09:44.162993+0000","last_became_active":"2023-10-03T10:09:43.760825+0000","last_became_peered":"2023-10-03T10:09:43.760825+0000","last_unstale":"2023-10-03T10:09:44.162993+0000","last_undegraded":"2023-10-03T10:09:44.162993+0000","last_fullsized":"2023-10-03T10:09:44.162993+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180799+0000","last_change":"2023-10-03T10:09:44.178856+0000","last_active":"2023-10-03T10:09:44.180799+0000","last_peered":"2023-10-03T10:09:44.180799+0000","last_clean":"2023-10-03T10:09:44.180799+0000","last_became_active":"2023-10-03T10:09:44.178410+0000","last_became_peered":"2023-10-03T10:09:44.178410+0000","last_unstale":"2023-10-03T10:09:44.180799+0000","last_undegraded":"2023-10-03T10:09:44.180799+0000","last_fullsized":"2023-10-03T10:09:44.180799+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.154127+0000","last_change":"2023-10-03T10:09:39.589080+0000","last_active":"2023-10-03T10:09:44.154127+0000","last_peered":"2023-10-03T10:09:44.154127+0000","last_clean":"2023-10-03T10:09:44.154127+0000","last_became_active":"2023-10-03T10:09:39.588651+0000","last_became_peered":"2023-10-03T10:09:39.588651+0000","last_unstale":"2023-10-03T10:09:44.154127+0000","last_undegraded":"2023-10-03T10:09:44.154127+0000","last_fullsized":"2023-10-03T10:09:44.154127+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.157236+0000","last_change":"2023-10-03T10:09:43.756053+0000","last_active":"2023-10-03T10:09:44.157236+0000","last_peered":"2023-10-03T10:09:44.157236+0000","last_clean":"2023-10-03T10:09:44.157236+0000","last_became_active":"2023-10-03T10:09:43.755708+0000","last_became_peered":"2023-10-03T10:09:43.755708+0000","last_unstale":"2023-10-03T10:09:44.157236+0000","last_undegraded":"2023-10-03T10:09:44.157236+0000","last_fullsized":"2023-10-03T10:09:44.157236+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180543+0000","last_change":"2023-10-03T10:09:44.176964+0000","last_active":"2023-10-03T10:09:44.180543+0000","last_peered":"2023-10-03T10:09:44.180543+0000","last_clean":"2023-10-03T10:09:44.180543+0000","last_became_active":"2023-10-03T10:09:44.176671+0000","last_became_peered":"2023-10-03T10:09:44.176671+0000","last_unstale":"2023-10-03T10:09:44.180543+0000","last_undegraded":"2023-10-03T10:09:44.180543+0000","last_fullsized":"2023-10-03T10:09:44.180543+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.166922+0000","last_change":"2023-10-03T10:09:39.586735+0000","last_active":"2023-10-03T10:09:44.166922+0000","last_peered":"2023-10-03T10:09:44.166922+0000","last_clean":"2023-10-03T10:09:44.166922+0000","last_became_active":"2023-10-03T10:09:39.586440+0000","last_became_peered":"2023-10-03T10:09:39.586440+0000","last_unstale":"2023-10-03T10:09:44.166922+0000","last_undegraded":"2023-10-03T10:09:44.166922+0000","last_fullsized":"2023-10-03T10:09:44.166922+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.159439+0000","last_change":"2023-10-03T10:09:43.748458+0000","last_active":"2023-10-03T10:09:44.159439+0000","last_peered":"2023-10-03T10:09:44.159439+0000","last_clean":"2023-10-03T10:09:44.159439+0000","last_became_active":"2023-10-03T10:09:43.747721+0000","last_became_peered":"2023-10-03T10:09:43.747721+0000","last_unstale":"2023-10-03T10:09:44.159439+0000","last_undegraded":"2023-10-03T10:09:44.159439+0000","last_fullsized":"2023-10-03T10:09:44.159439+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184166+0000","last_change":"2023-10-03T10:09:44.176165+0000","last_active":"2023-10-03T10:09:44.184166+0000","last_peered":"2023-10-03T10:09:44.184166+0000","last_clean":"2023-10-03T10:09:44.184166+0000","last_became_active":"2023-10-03T10:09:44.175853+0000","last_became_peered":"2023-10-03T10:09:44.175853+0000","last_unstale":"2023-10-03T10:09:44.184166+0000","last_undegraded":"2023-10-03T10:09:44.184166+0000","last_fullsized":"2023-10-03T10:09:44.184166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.156829+0000","last_change":"2023-10-03T10:09:39.590114+0000","last_active":"2023-10-03T10:09:44.156829+0000","last_peered":"2023-10-03T10:09:44.156829+0000","last_clean":"2023-10-03T10:09:44.156829+0000","last_became_active":"2023-10-03T10:09:39.589663+0000","last_became_peered":"2023-10-03T10:09:39.589663+0000","last_unstale":"2023-10-03T10:09:44.156829+0000","last_undegraded":"2023-10-03T10:09:44.156829+0000","last_fullsized":"2023-10-03T10:09:44.156829+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.155654+0000","last_change":"2023-10-03T10:09:43.748267+0000","last_active":"2023-10-03T10:09:44.155654+0000","last_peered":"2023-10-03T10:09:44.155654+0000","last_clean":"2023-10-03T10:09:44.155654+0000","last_became_active":"2023-10-03T10:09:43.747789+0000","last_became_peered":"2023-10-03T10:09:43.747789+0000","last_unstale":"2023-10-03T10:09:44.155654+0000","last_undegraded":"2023-10-03T10:09:44.155654+0000","last_fullsized":"2023-10-03T10:09:44.155654+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180773+0000","last_change":"2023-10-03T10:09:44.178680+0000","last_active":"2023-10-03T10:09:44.180773+0000","last_peered":"2023-10-03T10:09:44.180773+0000","last_clean":"2023-10-03T10:09:44.180773+0000","last_became_active":"2023-10-03T10:09:44.178114+0000","last_became_peered":"2023-10-03T10:09:44.178114+0000","last_unstale":"2023-10-03T10:09:44.180773+0000","last_undegraded":"2023-10-03T10:09:44.180773+0000","last_fullsized":"2023-10-03T10:09:44.180773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.167768+0000","last_change":"2023-10-03T10:09:39.587728+0000","last_active":"2023-10-03T10:09:44.167768+0000","last_peered":"2023-10-03T10:09:44.167768+0000","last_clean":"2023-10-03T10:09:44.167768+0000","last_became_active":"2023-10-03T10:09:39.587441+0000","last_became_peered":"2023-10-03T10:09:39.587441+0000","last_unstale":"2023-10-03T10:09:44.167768+0000","last_undegraded":"2023-10-03T10:09:44.167768+0000","last_fullsized":"2023-10-03T10:09:44.167768+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.153191+0000","last_change":"2023-10-03T10:09:38.585668+0000","last_active":"2023-10-03T10:09:44.153191+0000","last_peered":"2023-10-03T10:09:44.153191+0000","last_clean":"2023-10-03T10:09:44.153191+0000","last_became_active":"2023-10-03T10:09:38.585324+0000","last_became_peered":"2023-10-03T10:09:38.585324+0000","last_unstale":"2023-10-03T10:09:44.153191+0000","last_undegraded":"2023-10-03T10:09:44.153191+0000","last_fullsized":"2023-10-03T10:09:44.153191+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:37.570571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:37.570571+0000","last_clean_scrub_stamp":"2023-10-03T10:09:37.570571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.165999+0000","last_change":"2023-10-03T10:09:43.758096+0000","last_active":"2023-10-03T10:09:44.165999+0000","last_peered":"2023-10-03T10:09:44.165999+0000","last_clean":"2023-10-03T10:09:44.165999+0000","last_became_active":"2023-10-03T10:09:43.757689+0000","last_became_peered":"2023-10-03T10:09:43.757689+0000","last_unstale":"2023-10-03T10:09:44.165999+0000","last_undegraded":"2023-10-03T10:09:44.165999+0000","last_fullsized":"2023-10-03T10:09:44.165999+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196949+0000","last_change":"2023-10-03T10:09:44.192781+0000","last_active":"2023-10-03T10:09:44.196949+0000","last_peered":"2023-10-03T10:09:44.196949+0000","last_clean":"2023-10-03T10:09:44.196949+0000","last_became_active":"2023-10-03T10:09:44.190597+0000","last_became_peered":"2023-10-03T10:09:44.190597+0000","last_unstale":"2023-10-03T10:09:44.196949+0000","last_undegraded":"2023-10-03T10:09:44.196949+0000","last_fullsized":"2023-10-03T10:09:44.196949+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.163365+0000","last_change":"2023-10-03T10:09:39.586806+0000","last_active":"2023-10-03T10:09:44.163365+0000","last_peered":"2023-10-03T10:09:44.163365+0000","last_clean":"2023-10-03T10:09:44.163365+0000","last_became_active":"2023-10-03T10:09:39.586481+0000","last_became_peered":"2023-10-03T10:09:39.586481+0000","last_unstale":"2023-10-03T10:09:44.163365+0000","last_undegraded":"2023-10-03T10:09:44.163365+0000","last_fullsized":"2023-10-03T10:09:44.163365+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.156596+0000","last_change":"2023-10-03T10:09:43.747783+0000","last_active":"2023-10-03T10:09:44.156596+0000","last_peered":"2023-10-03T10:09:44.156596+0000","last_clean":"2023-10-03T10:09:44.156596+0000","last_became_active":"2023-10-03T10:09:43.746009+0000","last_became_peered":"2023-10-03T10:09:43.746009+0000","last_unstale":"2023-10-03T10:09:44.156596+0000","last_undegraded":"2023-10-03T10:09:44.156596+0000","last_fullsized":"2023-10-03T10:09:44.156596+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184590+0000","last_change":"2023-10-03T10:09:44.181359+0000","last_active":"2023-10-03T10:09:44.184590+0000","last_peered":"2023-10-03T10:09:44.184590+0000","last_clean":"2023-10-03T10:09:44.184590+0000","last_became_active":"2023-10-03T10:09:44.181075+0000","last_became_peered":"2023-10-03T10:09:44.181075+0000","last_unstale":"2023-10-03T10:09:44.184590+0000","last_undegraded":"2023-10-03T10:09:44.184590+0000","last_fullsized":"2023-10-03T10:09:44.184590+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.168332+0000","last_change":"2023-10-03T10:09:39.588827+0000","last_active":"2023-10-03T10:09:44.168332+0000","last_peered":"2023-10-03T10:09:44.168332+0000","last_clean":"2023-10-03T10:09:44.168332+0000","last_became_active":"2023-10-03T10:09:39.588531+0000","last_became_peered":"2023-10-03T10:09:39.588531+0000","last_unstale":"2023-10-03T10:09:44.168332+0000","last_undegraded":"2023-10-03T10:09:44.168332+0000","last_fullsized":"2023-10-03T10:09:44.168332+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.152966+0000","last_change":"2023-10-03T10:09:43.746339+0000","last_active":"2023-10-03T10:09:44.152966+0000","last_peered":"2023-10-03T10:09:44.152966+0000","last_clean":"2023-10-03T10:09:44.152966+0000","last_became_active":"2023-10-03T10:09:43.746051+0000","last_became_peered":"2023-10-03T10:09:43.746051+0000","last_unstale":"2023-10-03T10:09:44.152966+0000","last_undegraded":"2023-10-03T10:09:44.152966+0000","last_fullsized":"2023-10-03T10:09:44.152966+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173149+0000","last_change":"2023-10-03T10:09:44.165684+0000","last_active":"2023-10-03T10:09:44.173149+0000","last_peered":"2023-10-03T10:09:44.173149+0000","last_clean":"2023-10-03T10:09:44.173149+0000","last_became_active":"2023-10-03T10:09:44.165422+0000","last_became_peered":"2023-10-03T10:09:44.165422+0000","last_unstale":"2023-10-03T10:09:44.173149+0000","last_undegraded":"2023-10-03T10:09:44.173149+0000","last_fullsized":"2023-10-03T10:09:44.173149+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.197330+0000","last_change":"2023-10-03T10:09:43.760807+0000","last_active":"2023-10-03T10:09:44.197330+0000","last_peered":"2023-10-03T10:09:44.197330+0000","last_clean":"2023-10-03T10:09:44.197330+0000","last_became_active":"2023-10-03T10:09:43.760517+0000","last_became_peered":"2023-10-03T10:09:43.760517+0000","last_unstale":"2023-10-03T10:09:44.197330+0000","last_undegraded":"2023-10-03T10:09:44.197330+0000","last_fullsized":"2023-10-03T10:09:44.197330+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.153815+0000","last_change":"2023-10-03T10:09:39.586702+0000","last_active":"2023-10-03T10:09:44.153815+0000","last_peered":"2023-10-03T10:09:44.153815+0000","last_clean":"2023-10-03T10:09:44.153815+0000","last_became_active":"2023-10-03T10:09:39.586153+0000","last_became_peered":"2023-10-03T10:09:39.586153+0000","last_unstale":"2023-10-03T10:09:44.153815+0000","last_undegraded":"2023-10-03T10:09:44.153815+0000","last_fullsized":"2023-10-03T10:09:44.153815+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196883+0000","last_change":"2023-10-03T10:09:44.192664+0000","last_active":"2023-10-03T10:09:44.196883+0000","last_peered":"2023-10-03T10:09:44.196883+0000","last_clean":"2023-10-03T10:09:44.196883+0000","last_became_active":"2023-10-03T10:09:44.190513+0000","last_became_peered":"2023-10-03T10:09:44.190513+0000","last_unstale":"2023-10-03T10:09:44.196883+0000","last_undegraded":"2023-10-03T10:09:44.196883+0000","last_fullsized":"2023-10-03T10:09:44.196883+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.167083+0000","last_change":"2023-10-03T10:09:39.588950+0000","last_active":"2023-10-03T10:09:44.167083+0000","last_peered":"2023-10-03T10:09:44.167083+0000","last_clean":"2023-10-03T10:09:44.167083+0000","last_became_active":"2023-10-03T10:09:39.588601+0000","last_became_peered":"2023-10-03T10:09:39.588601+0000","last_unstale":"2023-10-03T10:09:44.167083+0000","last_undegraded":"2023-10-03T10:09:44.167083+0000","last_fullsized":"2023-10-03T10:09:44.167083+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"17'1","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-10-03T10:09:45.154970+0000","last_change":"2023-10-03T10:09:43.752793+0000","last_active":"2023-10-03T10:09:45.154970+0000","last_peered":"2023-10-03T10:09:45.154970+0000","last_clean":"2023-10-03T10:09:45.154970+0000","last_became_active":"2023-10-03T10:09:43.752374+0000","last_became_peered":"2023-10-03T10:09:43.752374+0000","last_unstale":"2023-10-03T10:09:45.154970+0000","last_undegraded":"2023-10-03T10:09:45.154970+0000","last_fullsized":"2023-10-03T10:09:45.154970+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200171+0000","last_change":"2023-10-03T10:09:44.196314+0000","last_active":"2023-10-03T10:09:44.200171+0000","last_peered":"2023-10-03T10:09:44.200171+0000","last_clean":"2023-10-03T10:09:44.200171+0000","last_became_active":"2023-10-03T10:09:44.195901+0000","last_became_peered":"2023-10-03T10:09:44.195901+0000","last_unstale":"2023-10-03T10:09:44.200171+0000","last_undegraded":"2023-10-03T10:09:44.200171+0000","last_fullsized":"2023-10-03T10:09:44.200171+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200135+0000","last_change":"2023-10-03T10:09:43.756460+0000","last_active":"2023-10-03T10:09:44.200135+0000","last_peered":"2023-10-03T10:09:44.200135+0000","last_clean":"2023-10-03T10:09:44.200135+0000","last_became_active":"2023-10-03T10:09:43.756164+0000","last_became_peered":"2023-10-03T10:09:43.756164+0000","last_unstale":"2023-10-03T10:09:44.200135+0000","last_undegraded":"2023-10-03T10:09:44.200135+0000","last_fullsized":"2023-10-03T10:09:44.200135+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.189031+0000","last_change":"2023-10-03T10:09:44.181915+0000","last_active":"2023-10-03T10:09:44.189031+0000","last_peered":"2023-10-03T10:09:44.189031+0000","last_clean":"2023-10-03T10:09:44.189031+0000","last_became_active":"2023-10-03T10:09:44.181645+0000","last_became_peered":"2023-10-03T10:09:44.181645+0000","last_unstale":"2023-10-03T10:09:44.189031+0000","last_undegraded":"2023-10-03T10:09:44.189031+0000","last_fullsized":"2023-10-03T10:09:44.189031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.193051+0000","last_change":"2023-10-03T10:09:43.748380+0000","last_active":"2023-10-03T10:09:44.193051+0000","last_peered":"2023-10-03T10:09:44.193051+0000","last_clean":"2023-10-03T10:09:44.193051+0000","last_became_active":"2023-10-03T10:09:43.748104+0000","last_became_peered":"2023-10-03T10:09:43.748104+0000","last_unstale":"2023-10-03T10:09:44.193051+0000","last_undegraded":"2023-10-03T10:09:44.193051+0000","last_fullsized":"2023-10-03T10:09:44.193051+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200359+0000","last_change":"2023-10-03T10:09:44.197710+0000","last_active":"2023-10-03T10:09:44.200359+0000","last_peered":"2023-10-03T10:09:44.200359+0000","last_clean":"2023-10-03T10:09:44.200359+0000","last_became_active":"2023-10-03T10:09:44.197064+0000","last_became_peered":"2023-10-03T10:09:44.197064+0000","last_unstale":"2023-10-03T10:09:44.200359+0000","last_undegraded":"2023-10-03T10:09:44.200359+0000","last_fullsized":"2023-10-03T10:09:44.200359+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.193435+0000","last_change":"2023-10-03T10:09:43.758701+0000","last_active":"2023-10-03T10:09:44.193435+0000","last_peered":"2023-10-03T10:09:44.193435+0000","last_clean":"2023-10-03T10:09:44.193435+0000","last_became_active":"2023-10-03T10:09:43.758376+0000","last_became_peered":"2023-10-03T10:09:43.758376+0000","last_unstale":"2023-10-03T10:09:44.193435+0000","last_undegraded":"2023-10-03T10:09:44.193435+0000","last_fullsized":"2023-10-03T10:09:44.193435+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200488+0000","last_change":"2023-10-03T10:09:44.197164+0000","last_active":"2023-10-03T10:09:44.200488+0000","last_peered":"2023-10-03T10:09:44.200488+0000","last_clean":"2023-10-03T10:09:44.200488+0000","last_became_active":"2023-10-03T10:09:44.196885+0000","last_became_peered":"2023-10-03T10:09:44.196885+0000","last_unstale":"2023-10-03T10:09:44.200488+0000","last_undegraded":"2023-10-03T10:09:44.200488+0000","last_fullsized":"2023-10-03T10:09:44.200488+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.199237+0000","last_change":"2023-10-03T10:09:43.758373+0000","last_active":"2023-10-03T10:09:44.199237+0000","last_peered":"2023-10-03T10:09:44.199237+0000","last_clean":"2023-10-03T10:09:44.199237+0000","last_became_active":"2023-10-03T10:09:43.758061+0000","last_became_peered":"2023-10-03T10:09:43.758061+0000","last_unstale":"2023-10-03T10:09:44.199237+0000","last_undegraded":"2023-10-03T10:09:44.199237+0000","last_fullsized":"2023-10-03T10:09:44.199237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196006+0000","last_change":"2023-10-03T10:09:44.191564+0000","last_active":"2023-10-03T10:09:44.196006+0000","last_peered":"2023-10-03T10:09:44.196006+0000","last_clean":"2023-10-03T10:09:44.196006+0000","last_became_active":"2023-10-03T10:09:44.191220+0000","last_became_peered":"2023-10-03T10:09:44.191220+0000","last_unstale":"2023-10-03T10:09:44.196006+0000","last_undegraded":"2023-10-03T10:09:44.196006+0000","last_fullsized":"2023-10-03T10:09:44.196006+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"16'4","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.183675+0000","last_change":"2023-10-03T10:09:43.750463+0000","last_active":"2023-10-03T10:09:44.183675+0000","last_peered":"2023-10-03T10:09:44.183675+0000","last_clean":"2023-10-03T10:09:44.183675+0000","last_became_active":"2023-10-03T10:09:43.749780+0000","last_became_peered":"2023-10-03T10:09:43.749780+0000","last_unstale":"2023-10-03T10:09:44.183675+0000","last_undegraded":"2023-10-03T10:09:44.183675+0000","last_fullsized":"2023-10-03T10:09:44.183675+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.188984+0000","last_change":"2023-10-03T10:09:44.180378+0000","last_active":"2023-10-03T10:09:44.188984+0000","last_peered":"2023-10-03T10:09:44.188984+0000","last_clean":"2023-10-03T10:09:44.188984+0000","last_became_active":"2023-10-03T10:09:44.180082+0000","last_became_peered":"2023-10-03T10:09:44.180082+0000","last_unstale":"2023-10-03T10:09:44.188984+0000","last_undegraded":"2023-10-03T10:09:44.188984+0000","last_fullsized":"2023-10-03T10:09:44.188984+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.187808+0000","last_change":"2023-10-03T10:09:44.178067+0000","last_active":"2023-10-03T10:09:44.187808+0000","last_peered":"2023-10-03T10:09:44.187808+0000","last_clean":"2023-10-03T10:09:44.187808+0000","last_became_active":"2023-10-03T10:09:44.175124+0000","last_became_peered":"2023-10-03T10:09:44.175124+0000","last_unstale":"2023-10-03T10:09:44.187808+0000","last_undegraded":"2023-10-03T10:09:44.187808+0000","last_fullsized":"2023-10-03T10:09:44.187808+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.197083+0000","last_change":"2023-10-03T10:09:44.192156+0000","last_active":"2023-10-03T10:09:44.197083+0000","last_peered":"2023-10-03T10:09:44.197083+0000","last_clean":"2023-10-03T10:09:44.197083+0000","last_became_active":"2023-10-03T10:09:44.189296+0000","last_became_peered":"2023-10-03T10:09:44.189296+0000","last_unstale":"2023-10-03T10:09:44.197083+0000","last_undegraded":"2023-10-03T10:09:44.197083+0000","last_fullsized":"2023-10-03T10:09:44.197083+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.174106+0000","last_change":"2023-10-03T10:09:44.168473+0000","last_active":"2023-10-03T10:09:44.174106+0000","last_peered":"2023-10-03T10:09:44.174106+0000","last_clean":"2023-10-03T10:09:44.174106+0000","last_became_active":"2023-10-03T10:09:44.168155+0000","last_became_peered":"2023-10-03T10:09:44.168155+0000","last_unstale":"2023-10-03T10:09:44.174106+0000","last_undegraded":"2023-10-03T10:09:44.174106+0000","last_fullsized":"2023-10-03T10:09:44.174106+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200881+0000","last_change":"2023-10-03T10:09:44.197689+0000","last_active":"2023-10-03T10:09:44.200881+0000","last_peered":"2023-10-03T10:09:44.200881+0000","last_clean":"2023-10-03T10:09:44.200881+0000","last_became_active":"2023-10-03T10:09:44.197410+0000","last_became_peered":"2023-10-03T10:09:44.197410+0000","last_unstale":"2023-10-03T10:09:44.200881+0000","last_undegraded":"2023-10-03T10:09:44.200881+0000","last_fullsized":"2023-10-03T10:09:44.200881+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184190+0000","last_change":"2023-10-03T10:09:44.176966+0000","last_active":"2023-10-03T10:09:44.184190+0000","last_peered":"2023-10-03T10:09:44.184190+0000","last_clean":"2023-10-03T10:09:44.184190+0000","last_became_active":"2023-10-03T10:09:44.176386+0000","last_became_peered":"2023-10-03T10:09:44.176386+0000","last_unstale":"2023-10-03T10:09:44.184190+0000","last_undegraded":"2023-10-03T10:09:44.184190+0000","last_fullsized":"2023-10-03T10:09:44.184190+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297252,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074588,"statfs":{"total":96636764160,"available":96332378112,"internally_reserved":0,"allocated":159744,"data_stored":48735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]}]},{"osd":3,"up_from":9,"seq":38654705670,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297264,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074576,"statfs":{"total":96636764160,"available":96332365824,"internally_reserved":0,"allocated":172032,"data_stored":48929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]}]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297256,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074584,"statfs":{"total":96636764160,"available":96332374016,"internally_reserved":0,"allocated":163840,"data_stored":47899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]}]},{"osd":1,"up_from":9,"seq":38654705670,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297252,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074588,"statfs":{"total":96636764160,"available":96332378112,"internally_reserved":0,"allocated":159744,"data_stored":47845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-03T10:09:58.033 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-03T10:09:58.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.199+0000 7f24be80d700 1 Processor -- start 2023-10-03T10:09:58.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.199+0000 7f24be80d700 1 -- start start 2023-10-03T10:09:58.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.200+0000 7f24be80d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24b80661c0 0x7f24b8097540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:58.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.200+0000 7f24be80d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24b812fd40 0x7f24b8132130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:58.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.200+0000 7f24be80d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24b8064a00 con 0x7f24b812ada0 2023-10-03T10:09:58.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.200+0000 7f24be80d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b806abe0 con 0x7f24b80661c0 2023-10-03T10:09:58.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.200+0000 7f24be80d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24b8064e90 con 0x7f24b812fd40 2023-10-03T10:09:58.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.200+0000 7f24b7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24b80661c0 0x7f24b8097540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:58.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.200+0000 7f24b77fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f24b812ada0 0x7f24b812b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38564/0 (socket says 172.21.15.3:38564) 2023-10-03T10:09:58.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.200+0000 7f24b77fe700 1 -- 172.21.15.3:0/2158571257 learned_addr learned my addr 172.21.15.3:0/2158571257 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:09:58.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.200+0000 7f24bcdaa700 1 --2- 172.21.15.3:0/2158571257 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24b812fd40 0x7f24b8132130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:58.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.200+0000 7f24b6ffd700 1 -- 172.21.15.3:0/2158571257 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2598229327 0 0) 0x7f24b8064a00 con 0x7f24b812ada0 2023-10-03T10:09:58.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.200+0000 7f24bcdaa700 1 -- 172.21.15.3:0/2158571257 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24b80661c0 msgr2=0x7f24b8097540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:58.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.200+0000 7f24bcdaa700 1 --2- 172.21.15.3:0/2158571257 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24b80661c0 0x7f24b8097540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:58.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.200+0000 7f24bcdaa700 1 -- 172.21.15.3:0/2158571257 >> v1:172.21.15.3:6789/0 conn(0x7f24b812ada0 legacy=0x7f24b812b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:58.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.200+0000 7f24bcdaa700 1 -- 172.21.15.3:0/2158571257 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b8065700 con 0x7f24b812fd40 2023-10-03T10:09:58.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.200+0000 7f24b7fff700 1 --2- 172.21.15.3:0/2158571257 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24b80661c0 0x7f24b8097540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:58.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.200+0000 7f24bcdaa700 1 --2- 172.21.15.3:0/2158571257 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24b812fd40 0x7f24b8132130 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f24b0009e90 tx=0x7f24b0009780).ready entity=mon.2 client_cookie=95e2cd7fe2ba4c4a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:58.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.201+0000 7f24b6ffd700 1 -- 172.21.15.3:0/2158571257 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b000e070 con 0x7f24b812fd40 2023-10-03T10:09:58.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.201+0000 7f24b6ffd700 1 -- 172.21.15.3:0/2158571257 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b000bb60 con 0x7f24b812fd40 2023-10-03T10:09:58.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.201+0000 7f24b6ffd700 1 -- 172.21.15.3:0/2158571257 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b0012410 con 0x7f24b812fd40 2023-10-03T10:09:58.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.201+0000 7f24be80d700 1 -- 172.21.15.3:0/2158571257 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24b812fd40 msgr2=0x7f24b8132130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:58.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.201+0000 7f24be80d700 1 --2- 172.21.15.3:0/2158571257 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24b812fd40 0x7f24b8132130 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f24b0009e90 tx=0x7f24b0009780).stop 2023-10-03T10:09:58.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.201+0000 7f24be80d700 1 -- 172.21.15.3:0/2158571257 shutdown_connections 2023-10-03T10:09:58.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.201+0000 7f24be80d700 1 --2- 172.21.15.3:0/2158571257 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24b80661c0 0x7f24b8097540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:58.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.201+0000 7f24be80d700 1 --2- 172.21.15.3:0/2158571257 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24b812fd40 0x7f24b8132130 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:58.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.201+0000 7f24be80d700 1 -- 172.21.15.3:0/2158571257 >> 172.21.15.3:0/2158571257 conn(0x7f24b809bb10 msgr2=0x7f24b809bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:58.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.201+0000 7f24be80d700 1 -- 172.21.15.3:0/2158571257 shutdown_connections 2023-10-03T10:09:58.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.201+0000 7f24be80d700 1 -- 172.21.15.3:0/2158571257 wait complete. 2023-10-03T10:09:58.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.202+0000 7f24be80d700 1 Processor -- start 2023-10-03T10:09:58.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.203+0000 7f24be80d700 1 -- start start 2023-10-03T10:09:58.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.203+0000 7f24be80d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24b80661c0 0x7f24b812f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:58.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.203+0000 7f24be80d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24b812ada0 0x7f24b812bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:58.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.203+0000 7f24be80d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24b812fd40 0x7f24b812c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:58.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.203+0000 7f24be80d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b812ec50 con 0x7f24b812ada0 2023-10-03T10:09:58.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.203+0000 7f24be80d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b809b5f0 con 0x7f24b812fd40 2023-10-03T10:09:58.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.203+0000 7f24be80d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24b80650c0 con 0x7f24b80661c0 2023-10-03T10:09:58.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.203+0000 7f24b7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24b80661c0 0x7f24b812f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:58.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.203+0000 7f24b77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24b812ada0 0x7f24b812bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:58.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.203+0000 7f24bcdaa700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24b812fd40 0x7f24b812c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:58.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.203+0000 7f24b7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24b80661c0 0x7f24b812f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57568/0 (socket says 172.21.15.3:57568) 2023-10-03T10:09:58.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.203+0000 7f24b7fff700 1 -- 172.21.15.3:0/3324193805 learned_addr learned my addr 172.21.15.3:0/3324193805 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:58.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.204+0000 7f24b7fff700 1 -- 172.21.15.3:0/3324193805 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24b812fd40 msgr2=0x7f24b812c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:58.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.204+0000 7f24b7fff700 1 --2- 172.21.15.3:0/3324193805 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24b812fd40 0x7f24b812c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:58.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.204+0000 7f24b7fff700 1 -- 172.21.15.3:0/3324193805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24b812ada0 msgr2=0x7f24b812bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:58.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.204+0000 7f24b7fff700 1 --2- 172.21.15.3:0/3324193805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24b812ada0 0x7f24b812bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:58.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.204+0000 7f24b7fff700 1 -- 172.21.15.3:0/3324193805 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a80092a0 con 0x7f24b80661c0 2023-10-03T10:09:58.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.204+0000 7f24b77fe700 1 --2- 172.21.15.3:0/3324193805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24b812ada0 0x7f24b812bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:58.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.204+0000 7f24b7fff700 1 --2- 172.21.15.3:0/3324193805 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24b80661c0 0x7f24b812f750 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f24a800ea50 tx=0x7f24a8009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:58.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.204+0000 7f24affff700 1 -- 172.21.15.3:0/3324193805 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a800ee90 con 0x7f24b80661c0 2023-10-03T10:09:58.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.204+0000 7f24affff700 1 -- 172.21.15.3:0/3324193805 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a8009600 con 0x7f24b80661c0 2023-10-03T10:09:58.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.204+0000 7f24be80d700 1 -- 172.21.15.3:0/3324193805 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b0009a20 con 0x7f24b80661c0 2023-10-03T10:09:58.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.204+0000 7f24affff700 1 -- 172.21.15.3:0/3324193805 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a800ca60 con 0x7f24b80661c0 2023-10-03T10:09:58.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.204+0000 7f24be80d700 1 -- 172.21.15.3:0/3324193805 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b812e280 con 0x7f24b80661c0 2023-10-03T10:09:58.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.205+0000 7f24affff700 1 -- 172.21.15.3:0/3324193805 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f24a801b400 con 0x7f24b80661c0 2023-10-03T10:09:58.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.205+0000 7f24affff700 1 -- 172.21.15.3:0/3324193805 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f24a803b480 con 0x7f24b80661c0 2023-10-03T10:09:58.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.205+0000 7f24be80d700 1 -- 172.21.15.3:0/3324193805 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b805a260 con 0x7f24b80661c0 2023-10-03T10:09:58.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.210+0000 7f24affff700 1 -- 172.21.15.3:0/3324193805 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f24a8016070 con 0x7f24b80661c0 2023-10-03T10:09:58.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.381+0000 7f24be80d700 1 -- 172.21.15.3:0/3324193805 --> v1:172.21.15.3:6816/118296 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f24b812c910 con 0x7f249802bb90 2023-10-03T10:09:58.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.387+0000 7f24affff700 1 -- 172.21.15.3:0/3324193805 <== mgr.4100 v1:172.21.15.3:6816/118296 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249934 (unknown 2965378022 0 955997573) 0x7f24b812c910 con 0x7f249802bb90 2023-10-03T10:09:58.390 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:58.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.391+0000 7f24be80d700 1 -- 172.21.15.3:0/3324193805 >> v1:172.21.15.3:6816/118296 conn(0x7f249802bb90 legacy=0x7f249802e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:58.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.391+0000 7f24be80d700 1 -- 172.21.15.3:0/3324193805 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24b80661c0 msgr2=0x7f24b812f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:58.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.391+0000 7f24be80d700 1 --2- 172.21.15.3:0/3324193805 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24b80661c0 0x7f24b812f750 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f24a800ea50 tx=0x7f24a8009060).stop 2023-10-03T10:09:58.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.391+0000 7f24be80d700 1 -- 172.21.15.3:0/3324193805 shutdown_connections 2023-10-03T10:09:58.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.391+0000 7f24be80d700 1 --2- 172.21.15.3:0/3324193805 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24b812fd40 0x7f24b812c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:58.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.391+0000 7f24be80d700 1 --2- 172.21.15.3:0/3324193805 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24b80661c0 0x7f24b812f750 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:58.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.391+0000 7f24be80d700 1 --2- 172.21.15.3:0/3324193805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24b812ada0 0x7f24b812bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:58.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.391+0000 7f24be80d700 1 -- 172.21.15.3:0/3324193805 >> 172.21.15.3:0/3324193805 conn(0x7f24b809bb10 msgr2=0x7f24b81287e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:58.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.391+0000 7f24be80d700 1 -- 172.21.15.3:0/3324193805 shutdown_connections 2023-10-03T10:09:58.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.391+0000 7f24be80d700 1 -- 172.21.15.3:0/3324193805 wait complete. 2023-10-03T10:09:58.393 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-03T10:09:58.404 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-03T10:09:57.531533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1189024,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298336,"statfs":{"total":386547056640,"available":385329496064,"internally_reserved":0,"allocated":655360,"data_stored":193408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002377"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.199042+0000","last_change":"2023-10-03T10:09:44.195682+0000","last_active":"2023-10-03T10:09:44.199042+0000","last_peered":"2023-10-03T10:09:44.199042+0000","last_clean":"2023-10-03T10:09:44.199042+0000","last_became_active":"2023-10-03T10:09:44.195360+0000","last_became_peered":"2023-10-03T10:09:44.195360+0000","last_unstale":"2023-10-03T10:09:44.199042+0000","last_undegraded":"2023-10-03T10:09:44.199042+0000","last_fullsized":"2023-10-03T10:09:44.199042+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180036+0000","last_change":"2023-10-03T10:09:44.178263+0000","last_active":"2023-10-03T10:09:44.180036+0000","last_peered":"2023-10-03T10:09:44.180036+0000","last_clean":"2023-10-03T10:09:44.180036+0000","last_became_active":"2023-10-03T10:09:44.177977+0000","last_became_peered":"2023-10-03T10:09:44.177977+0000","last_unstale":"2023-10-03T10:09:44.180036+0000","last_undegraded":"2023-10-03T10:09:44.180036+0000","last_fullsized":"2023-10-03T10:09:44.180036+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.190395+0000","last_change":"2023-10-03T10:09:44.183364+0000","last_active":"2023-10-03T10:09:44.190395+0000","last_peered":"2023-10-03T10:09:44.190395+0000","last_clean":"2023-10-03T10:09:44.190395+0000","last_became_active":"2023-10-03T10:09:44.183094+0000","last_became_peered":"2023-10-03T10:09:44.183094+0000","last_unstale":"2023-10-03T10:09:44.190395+0000","last_undegraded":"2023-10-03T10:09:44.190395+0000","last_fullsized":"2023-10-03T10:09:44.190395+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.190339+0000","last_change":"2023-10-03T10:09:44.183273+0000","last_active":"2023-10-03T10:09:44.190339+0000","last_peered":"2023-10-03T10:09:44.190339+0000","last_clean":"2023-10-03T10:09:44.190339+0000","last_became_active":"2023-10-03T10:09:44.182451+0000","last_became_peered":"2023-10-03T10:09:44.182451+0000","last_unstale":"2023-10-03T10:09:44.190339+0000","last_undegraded":"2023-10-03T10:09:44.190339+0000","last_fullsized":"2023-10-03T10:09:44.190339+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.174191+0000","last_change":"2023-10-03T10:09:44.168483+0000","last_active":"2023-10-03T10:09:44.174191+0000","last_peered":"2023-10-03T10:09:44.174191+0000","last_clean":"2023-10-03T10:09:44.174191+0000","last_became_active":"2023-10-03T10:09:44.168186+0000","last_became_peered":"2023-10-03T10:09:44.168186+0000","last_unstale":"2023-10-03T10:09:44.174191+0000","last_undegraded":"2023-10-03T10:09:44.174191+0000","last_fullsized":"2023-10-03T10:09:44.174191+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.198209+0000","last_change":"2023-10-03T10:09:44.192967+0000","last_active":"2023-10-03T10:09:44.198209+0000","last_peered":"2023-10-03T10:09:44.198209+0000","last_clean":"2023-10-03T10:09:44.198209+0000","last_became_active":"2023-10-03T10:09:44.190979+0000","last_became_peered":"2023-10-03T10:09:44.190979+0000","last_unstale":"2023-10-03T10:09:44.198209+0000","last_undegraded":"2023-10-03T10:09:44.198209+0000","last_fullsized":"2023-10-03T10:09:44.198209+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.192360+0000","last_change":"2023-10-03T10:09:44.183121+0000","last_active":"2023-10-03T10:09:44.192360+0000","last_peered":"2023-10-03T10:09:44.192360+0000","last_clean":"2023-10-03T10:09:44.192360+0000","last_became_active":"2023-10-03T10:09:44.182647+0000","last_became_peered":"2023-10-03T10:09:44.182647+0000","last_unstale":"2023-10-03T10:09:44.192360+0000","last_undegraded":"2023-10-03T10:09:44.192360+0000","last_fullsized":"2023-10-03T10:09:44.192360+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.187765+0000","last_change":"2023-10-03T10:09:44.173123+0000","last_active":"2023-10-03T10:09:44.187765+0000","last_peered":"2023-10-03T10:09:44.187765+0000","last_clean":"2023-10-03T10:09:44.187765+0000","last_became_active":"2023-10-03T10:09:44.172353+0000","last_became_peered":"2023-10-03T10:09:44.172353+0000","last_unstale":"2023-10-03T10:09:44.187765+0000","last_undegraded":"2023-10-03T10:09:44.187765+0000","last_fullsized":"2023-10-03T10:09:44.187765+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173973+0000","last_change":"2023-10-03T10:09:44.166758+0000","last_active":"2023-10-03T10:09:44.173973+0000","last_peered":"2023-10-03T10:09:44.173973+0000","last_clean":"2023-10-03T10:09:44.173973+0000","last_became_active":"2023-10-03T10:09:44.166415+0000","last_became_peered":"2023-10-03T10:09:44.166415+0000","last_unstale":"2023-10-03T10:09:44.173973+0000","last_undegraded":"2023-10-03T10:09:44.173973+0000","last_fullsized":"2023-10-03T10:09:44.173973+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180446+0000","last_change":"2023-10-03T10:09:44.176833+0000","last_active":"2023-10-03T10:09:44.180446+0000","last_peered":"2023-10-03T10:09:44.180446+0000","last_clean":"2023-10-03T10:09:44.180446+0000","last_became_active":"2023-10-03T10:09:44.176526+0000","last_became_peered":"2023-10-03T10:09:44.176526+0000","last_unstale":"2023-10-03T10:09:44.180446+0000","last_undegraded":"2023-10-03T10:09:44.180446+0000","last_fullsized":"2023-10-03T10:09:44.180446+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180114+0000","last_change":"2023-10-03T10:09:44.175868+0000","last_active":"2023-10-03T10:09:44.180114+0000","last_peered":"2023-10-03T10:09:44.180114+0000","last_clean":"2023-10-03T10:09:44.180114+0000","last_became_active":"2023-10-03T10:09:44.175414+0000","last_became_peered":"2023-10-03T10:09:44.175414+0000","last_unstale":"2023-10-03T10:09:44.180114+0000","last_undegraded":"2023-10-03T10:09:44.180114+0000","last_fullsized":"2023-10-03T10:09:44.180114+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200283+0000","last_change":"2023-10-03T10:09:44.198581+0000","last_active":"2023-10-03T10:09:44.200283+0000","last_peered":"2023-10-03T10:09:44.200283+0000","last_clean":"2023-10-03T10:09:44.200283+0000","last_became_active":"2023-10-03T10:09:44.198314+0000","last_became_peered":"2023-10-03T10:09:44.198314+0000","last_unstale":"2023-10-03T10:09:44.200283+0000","last_undegraded":"2023-10-03T10:09:44.200283+0000","last_fullsized":"2023-10-03T10:09:44.200283+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200379+0000","last_change":"2023-10-03T10:09:44.197915+0000","last_active":"2023-10-03T10:09:44.200379+0000","last_peered":"2023-10-03T10:09:44.200379+0000","last_clean":"2023-10-03T10:09:44.200379+0000","last_became_active":"2023-10-03T10:09:44.197417+0000","last_became_peered":"2023-10-03T10:09:44.197417+0000","last_unstale":"2023-10-03T10:09:44.200379+0000","last_undegraded":"2023-10-03T10:09:44.200379+0000","last_fullsized":"2023-10-03T10:09:44.200379+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180522+0000","last_change":"2023-10-03T10:09:44.176456+0000","last_active":"2023-10-03T10:09:44.180522+0000","last_peered":"2023-10-03T10:09:44.180522+0000","last_clean":"2023-10-03T10:09:44.180522+0000","last_became_active":"2023-10-03T10:09:44.176146+0000","last_became_peered":"2023-10-03T10:09:44.176146+0000","last_unstale":"2023-10-03T10:09:44.180522+0000","last_undegraded":"2023-10-03T10:09:44.180522+0000","last_fullsized":"2023-10-03T10:09:44.180522+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.178380+0000","last_change":"2023-10-03T10:09:44.176614+0000","last_active":"2023-10-03T10:09:44.178380+0000","last_peered":"2023-10-03T10:09:44.178380+0000","last_clean":"2023-10-03T10:09:44.178380+0000","last_became_active":"2023-10-03T10:09:44.176334+0000","last_became_peered":"2023-10-03T10:09:44.176334+0000","last_unstale":"2023-10-03T10:09:44.178380+0000","last_undegraded":"2023-10-03T10:09:44.178380+0000","last_fullsized":"2023-10-03T10:09:44.178380+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.192741+0000","last_change":"2023-10-03T10:09:44.179440+0000","last_active":"2023-10-03T10:09:44.192741+0000","last_peered":"2023-10-03T10:09:44.192741+0000","last_clean":"2023-10-03T10:09:44.192741+0000","last_became_active":"2023-10-03T10:09:44.179150+0000","last_became_peered":"2023-10-03T10:09:44.179150+0000","last_unstale":"2023-10-03T10:09:44.192741+0000","last_undegraded":"2023-10-03T10:09:44.192741+0000","last_fullsized":"2023-10-03T10:09:44.192741+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173996+0000","last_change":"2023-10-03T10:09:44.166901+0000","last_active":"2023-10-03T10:09:44.173996+0000","last_peered":"2023-10-03T10:09:44.173996+0000","last_clean":"2023-10-03T10:09:44.173996+0000","last_became_active":"2023-10-03T10:09:44.166610+0000","last_became_peered":"2023-10-03T10:09:44.166610+0000","last_unstale":"2023-10-03T10:09:44.173996+0000","last_undegraded":"2023-10-03T10:09:44.173996+0000","last_fullsized":"2023-10-03T10:09:44.173996+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173643+0000","last_change":"2023-10-03T10:09:44.165366+0000","last_active":"2023-10-03T10:09:44.173643+0000","last_peered":"2023-10-03T10:09:44.173643+0000","last_clean":"2023-10-03T10:09:44.173643+0000","last_became_active":"2023-10-03T10:09:44.165083+0000","last_became_peered":"2023-10-03T10:09:44.165083+0000","last_unstale":"2023-10-03T10:09:44.173643+0000","last_undegraded":"2023-10-03T10:09:44.173643+0000","last_fullsized":"2023-10-03T10:09:44.173643+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.198990+0000","last_change":"2023-10-03T10:09:44.195696+0000","last_active":"2023-10-03T10:09:44.198990+0000","last_peered":"2023-10-03T10:09:44.198990+0000","last_clean":"2023-10-03T10:09:44.198990+0000","last_became_active":"2023-10-03T10:09:44.195402+0000","last_became_peered":"2023-10-03T10:09:44.195402+0000","last_unstale":"2023-10-03T10:09:44.198990+0000","last_undegraded":"2023-10-03T10:09:44.198990+0000","last_fullsized":"2023-10-03T10:09:44.198990+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184209+0000","last_change":"2023-10-03T10:09:44.177178+0000","last_active":"2023-10-03T10:09:44.184209+0000","last_peered":"2023-10-03T10:09:44.184209+0000","last_clean":"2023-10-03T10:09:44.184209+0000","last_became_active":"2023-10-03T10:09:44.176686+0000","last_became_peered":"2023-10-03T10:09:44.176686+0000","last_unstale":"2023-10-03T10:09:44.184209+0000","last_undegraded":"2023-10-03T10:09:44.184209+0000","last_fullsized":"2023-10-03T10:09:44.184209+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180396+0000","last_change":"2023-10-03T10:09:44.177335+0000","last_active":"2023-10-03T10:09:44.180396+0000","last_peered":"2023-10-03T10:09:44.180396+0000","last_clean":"2023-10-03T10:09:44.180396+0000","last_became_active":"2023-10-03T10:09:44.175202+0000","last_became_peered":"2023-10-03T10:09:44.175202+0000","last_unstale":"2023-10-03T10:09:44.180396+0000","last_undegraded":"2023-10-03T10:09:44.180396+0000","last_fullsized":"2023-10-03T10:09:44.180396+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184702+0000","last_change":"2023-10-03T10:09:44.182207+0000","last_active":"2023-10-03T10:09:44.184702+0000","last_peered":"2023-10-03T10:09:44.184702+0000","last_clean":"2023-10-03T10:09:44.184702+0000","last_became_active":"2023-10-03T10:09:44.181944+0000","last_became_peered":"2023-10-03T10:09:44.181944+0000","last_unstale":"2023-10-03T10:09:44.184702+0000","last_undegraded":"2023-10-03T10:09:44.184702+0000","last_fullsized":"2023-10-03T10:09:44.184702+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.192336+0000","last_change":"2023-10-03T10:09:44.182610+0000","last_active":"2023-10-03T10:09:44.192336+0000","last_peered":"2023-10-03T10:09:44.192336+0000","last_clean":"2023-10-03T10:09:44.192336+0000","last_became_active":"2023-10-03T10:09:44.180732+0000","last_became_peered":"2023-10-03T10:09:44.180732+0000","last_unstale":"2023-10-03T10:09:44.192336+0000","last_undegraded":"2023-10-03T10:09:44.192336+0000","last_fullsized":"2023-10-03T10:09:44.192336+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.187788+0000","last_change":"2023-10-03T10:09:44.173182+0000","last_active":"2023-10-03T10:09:44.187788+0000","last_peered":"2023-10-03T10:09:44.187788+0000","last_clean":"2023-10-03T10:09:44.187788+0000","last_became_active":"2023-10-03T10:09:44.172685+0000","last_became_peered":"2023-10-03T10:09:44.172685+0000","last_unstale":"2023-10-03T10:09:44.187788+0000","last_undegraded":"2023-10-03T10:09:44.187788+0000","last_fullsized":"2023-10-03T10:09:44.187788+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180761+0000","last_change":"2023-10-03T10:09:44.177910+0000","last_active":"2023-10-03T10:09:44.180761+0000","last_peered":"2023-10-03T10:09:44.180761+0000","last_clean":"2023-10-03T10:09:44.180761+0000","last_became_active":"2023-10-03T10:09:44.177618+0000","last_became_peered":"2023-10-03T10:09:44.177618+0000","last_unstale":"2023-10-03T10:09:44.180761+0000","last_undegraded":"2023-10-03T10:09:44.180761+0000","last_fullsized":"2023-10-03T10:09:44.180761+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180424+0000","last_change":"2023-10-03T10:09:44.176287+0000","last_active":"2023-10-03T10:09:44.180424+0000","last_peered":"2023-10-03T10:09:44.180424+0000","last_clean":"2023-10-03T10:09:44.180424+0000","last_became_active":"2023-10-03T10:09:44.175320+0000","last_became_peered":"2023-10-03T10:09:44.175320+0000","last_unstale":"2023-10-03T10:09:44.180424+0000","last_undegraded":"2023-10-03T10:09:44.180424+0000","last_fullsized":"2023-10-03T10:09:44.180424+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.194745+0000","last_change":"2023-10-03T10:09:44.187148+0000","last_active":"2023-10-03T10:09:44.194745+0000","last_peered":"2023-10-03T10:09:44.194745+0000","last_clean":"2023-10-03T10:09:44.194745+0000","last_became_active":"2023-10-03T10:09:44.186523+0000","last_became_peered":"2023-10-03T10:09:44.186523+0000","last_unstale":"2023-10-03T10:09:44.194745+0000","last_undegraded":"2023-10-03T10:09:44.194745+0000","last_fullsized":"2023-10-03T10:09:44.194745+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.189715+0000","last_change":"2023-10-03T10:09:43.745046+0000","last_active":"2023-10-03T10:09:44.189715+0000","last_peered":"2023-10-03T10:09:44.189715+0000","last_clean":"2023-10-03T10:09:44.189715+0000","last_became_active":"2023-10-03T10:09:43.744724+0000","last_became_peered":"2023-10-03T10:09:43.744724+0000","last_unstale":"2023-10-03T10:09:44.189715+0000","last_undegraded":"2023-10-03T10:09:44.189715+0000","last_fullsized":"2023-10-03T10:09:44.189715+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195809+0000","last_change":"2023-10-03T10:09:44.190869+0000","last_active":"2023-10-03T10:09:44.195809+0000","last_peered":"2023-10-03T10:09:44.195809+0000","last_clean":"2023-10-03T10:09:44.195809+0000","last_became_active":"2023-10-03T10:09:44.190207+0000","last_became_peered":"2023-10-03T10:09:44.190207+0000","last_unstale":"2023-10-03T10:09:44.195809+0000","last_undegraded":"2023-10-03T10:09:44.195809+0000","last_fullsized":"2023-10-03T10:09:44.195809+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.152860+0000","last_change":"2023-10-03T10:09:43.750891+0000","last_active":"2023-10-03T10:09:44.152860+0000","last_peered":"2023-10-03T10:09:44.152860+0000","last_clean":"2023-10-03T10:09:44.152860+0000","last_became_active":"2023-10-03T10:09:43.750356+0000","last_became_peered":"2023-10-03T10:09:43.750356+0000","last_unstale":"2023-10-03T10:09:44.152860+0000","last_undegraded":"2023-10-03T10:09:44.152860+0000","last_fullsized":"2023-10-03T10:09:44.152860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184645+0000","last_change":"2023-10-03T10:09:44.183145+0000","last_active":"2023-10-03T10:09:44.184645+0000","last_peered":"2023-10-03T10:09:44.184645+0000","last_clean":"2023-10-03T10:09:44.184645+0000","last_became_active":"2023-10-03T10:09:44.182816+0000","last_became_peered":"2023-10-03T10:09:44.182816+0000","last_unstale":"2023-10-03T10:09:44.184645+0000","last_undegraded":"2023-10-03T10:09:44.184645+0000","last_fullsized":"2023-10-03T10:09:44.184645+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.163680+0000","last_change":"2023-10-03T10:09:43.749289+0000","last_active":"2023-10-03T10:09:44.163680+0000","last_peered":"2023-10-03T10:09:44.163680+0000","last_clean":"2023-10-03T10:09:44.163680+0000","last_became_active":"2023-10-03T10:09:43.748981+0000","last_became_peered":"2023-10-03T10:09:43.748981+0000","last_unstale":"2023-10-03T10:09:44.163680+0000","last_undegraded":"2023-10-03T10:09:44.163680+0000","last_fullsized":"2023-10-03T10:09:44.163680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.197002+0000","last_change":"2023-10-03T10:09:44.192692+0000","last_active":"2023-10-03T10:09:44.197002+0000","last_peered":"2023-10-03T10:09:44.197002+0000","last_clean":"2023-10-03T10:09:44.197002+0000","last_became_active":"2023-10-03T10:09:44.192390+0000","last_became_peered":"2023-10-03T10:09:44.192390+0000","last_unstale":"2023-10-03T10:09:44.197002+0000","last_undegraded":"2023-10-03T10:09:44.197002+0000","last_fullsized":"2023-10-03T10:09:44.197002+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.160877+0000","last_change":"2023-10-03T10:09:43.754802+0000","last_active":"2023-10-03T10:09:44.160877+0000","last_peered":"2023-10-03T10:09:44.160877+0000","last_clean":"2023-10-03T10:09:44.160877+0000","last_became_active":"2023-10-03T10:09:43.754512+0000","last_became_peered":"2023-10-03T10:09:43.754512+0000","last_unstale":"2023-10-03T10:09:44.160877+0000","last_undegraded":"2023-10-03T10:09:44.160877+0000","last_fullsized":"2023-10-03T10:09:44.160877+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196969+0000","last_change":"2023-10-03T10:09:44.192798+0000","last_active":"2023-10-03T10:09:44.196969+0000","last_peered":"2023-10-03T10:09:44.196969+0000","last_clean":"2023-10-03T10:09:44.196969+0000","last_became_active":"2023-10-03T10:09:44.190787+0000","last_became_peered":"2023-10-03T10:09:44.190787+0000","last_unstale":"2023-10-03T10:09:44.196969+0000","last_undegraded":"2023-10-03T10:09:44.196969+0000","last_fullsized":"2023-10-03T10:09:44.196969+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.157966+0000","last_change":"2023-10-03T10:09:43.756958+0000","last_active":"2023-10-03T10:09:44.157966+0000","last_peered":"2023-10-03T10:09:44.157966+0000","last_clean":"2023-10-03T10:09:44.157966+0000","last_became_active":"2023-10-03T10:09:43.756463+0000","last_became_peered":"2023-10-03T10:09:43.756463+0000","last_unstale":"2023-10-03T10:09:44.157966+0000","last_undegraded":"2023-10-03T10:09:44.157966+0000","last_fullsized":"2023-10-03T10:09:44.157966+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.194551+0000","last_change":"2023-10-03T10:09:44.184790+0000","last_active":"2023-10-03T10:09:44.194551+0000","last_peered":"2023-10-03T10:09:44.194551+0000","last_clean":"2023-10-03T10:09:44.194551+0000","last_became_active":"2023-10-03T10:09:44.184531+0000","last_became_peered":"2023-10-03T10:09:44.184531+0000","last_unstale":"2023-10-03T10:09:44.194551+0000","last_undegraded":"2023-10-03T10:09:44.194551+0000","last_fullsized":"2023-10-03T10:09:44.194551+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"16'3","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200426+0000","last_change":"2023-10-03T10:09:43.756598+0000","last_active":"2023-10-03T10:09:44.200426+0000","last_peered":"2023-10-03T10:09:44.200426+0000","last_clean":"2023-10-03T10:09:44.200426+0000","last_became_active":"2023-10-03T10:09:43.754974+0000","last_became_peered":"2023-10-03T10:09:43.754974+0000","last_unstale":"2023-10-03T10:09:44.200426+0000","last_undegraded":"2023-10-03T10:09:44.200426+0000","last_fullsized":"2023-10-03T10:09:44.200426+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180499+0000","last_change":"2023-10-03T10:09:44.175504+0000","last_active":"2023-10-03T10:09:44.180499+0000","last_peered":"2023-10-03T10:09:44.180499+0000","last_clean":"2023-10-03T10:09:44.180499+0000","last_became_active":"2023-10-03T10:09:44.175174+0000","last_became_peered":"2023-10-03T10:09:44.175174+0000","last_unstale":"2023-10-03T10:09:44.180499+0000","last_undegraded":"2023-10-03T10:09:44.180499+0000","last_fullsized":"2023-10-03T10:09:44.180499+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"16'6","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.203073+0000","last_change":"2023-10-03T10:09:43.758711+0000","last_active":"2023-10-03T10:09:44.203073+0000","last_peered":"2023-10-03T10:09:44.203073+0000","last_clean":"2023-10-03T10:09:44.203073+0000","last_became_active":"2023-10-03T10:09:43.758392+0000","last_became_peered":"2023-10-03T10:09:43.758392+0000","last_unstale":"2023-10-03T10:09:44.203073+0000","last_undegraded":"2023-10-03T10:09:44.203073+0000","last_fullsized":"2023-10-03T10:09:44.203073+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200336+0000","last_change":"2023-10-03T10:09:44.196559+0000","last_active":"2023-10-03T10:09:44.200336+0000","last_peered":"2023-10-03T10:09:44.200336+0000","last_clean":"2023-10-03T10:09:44.200336+0000","last_became_active":"2023-10-03T10:09:44.196280+0000","last_became_peered":"2023-10-03T10:09:44.196280+0000","last_unstale":"2023-10-03T10:09:44.200336+0000","last_undegraded":"2023-10-03T10:09:44.200336+0000","last_fullsized":"2023-10-03T10:09:44.200336+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200762+0000","last_change":"2023-10-03T10:09:43.751877+0000","last_active":"2023-10-03T10:09:44.200762+0000","last_peered":"2023-10-03T10:09:44.200762+0000","last_clean":"2023-10-03T10:09:44.200762+0000","last_became_active":"2023-10-03T10:09:43.749870+0000","last_became_peered":"2023-10-03T10:09:43.749870+0000","last_unstale":"2023-10-03T10:09:44.200762+0000","last_undegraded":"2023-10-03T10:09:44.200762+0000","last_fullsized":"2023-10-03T10:09:44.200762+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195297+0000","last_change":"2023-10-03T10:09:44.190230+0000","last_active":"2023-10-03T10:09:44.195297+0000","last_peered":"2023-10-03T10:09:44.195297+0000","last_clean":"2023-10-03T10:09:44.195297+0000","last_became_active":"2023-10-03T10:09:44.189342+0000","last_became_peered":"2023-10-03T10:09:44.189342+0000","last_unstale":"2023-10-03T10:09:44.195297+0000","last_undegraded":"2023-10-03T10:09:44.195297+0000","last_fullsized":"2023-10-03T10:09:44.195297+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.162475+0000","last_change":"2023-10-03T10:09:43.755271+0000","last_active":"2023-10-03T10:09:44.162475+0000","last_peered":"2023-10-03T10:09:44.162475+0000","last_clean":"2023-10-03T10:09:44.162475+0000","last_became_active":"2023-10-03T10:09:43.754957+0000","last_became_peered":"2023-10-03T10:09:43.754957+0000","last_unstale":"2023-10-03T10:09:44.162475+0000","last_undegraded":"2023-10-03T10:09:44.162475+0000","last_fullsized":"2023-10-03T10:09:44.162475+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.194802+0000","last_change":"2023-10-03T10:09:44.187333+0000","last_active":"2023-10-03T10:09:44.194802+0000","last_peered":"2023-10-03T10:09:44.194802+0000","last_clean":"2023-10-03T10:09:44.194802+0000","last_became_active":"2023-10-03T10:09:44.186824+0000","last_became_peered":"2023-10-03T10:09:44.186824+0000","last_unstale":"2023-10-03T10:09:44.194802+0000","last_undegraded":"2023-10-03T10:09:44.194802+0000","last_fullsized":"2023-10-03T10:09:44.194802+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.163844+0000","last_change":"2023-10-03T10:09:43.758995+0000","last_active":"2023-10-03T10:09:44.163844+0000","last_peered":"2023-10-03T10:09:44.163844+0000","last_clean":"2023-10-03T10:09:44.163844+0000","last_became_active":"2023-10-03T10:09:43.758565+0000","last_became_peered":"2023-10-03T10:09:43.758565+0000","last_unstale":"2023-10-03T10:09:44.163844+0000","last_undegraded":"2023-10-03T10:09:44.163844+0000","last_fullsized":"2023-10-03T10:09:44.163844+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173655+0000","last_change":"2023-10-03T10:09:44.166557+0000","last_active":"2023-10-03T10:09:44.173655+0000","last_peered":"2023-10-03T10:09:44.173655+0000","last_clean":"2023-10-03T10:09:44.173655+0000","last_became_active":"2023-10-03T10:09:44.166300+0000","last_became_peered":"2023-10-03T10:09:44.166300+0000","last_unstale":"2023-10-03T10:09:44.173655+0000","last_undegraded":"2023-10-03T10:09:44.173655+0000","last_fullsized":"2023-10-03T10:09:44.173655+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"16'8","reported_seq":18,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.194737+0000","last_change":"2023-10-03T10:09:43.748084+0000","last_active":"2023-10-03T10:09:44.194737+0000","last_peered":"2023-10-03T10:09:44.194737+0000","last_clean":"2023-10-03T10:09:44.194737+0000","last_became_active":"2023-10-03T10:09:43.747377+0000","last_became_peered":"2023-10-03T10:09:43.747377+0000","last_unstale":"2023-10-03T10:09:44.194737+0000","last_undegraded":"2023-10-03T10:09:44.194737+0000","last_fullsized":"2023-10-03T10:09:44.194737+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195326+0000","last_change":"2023-10-03T10:09:44.190767+0000","last_active":"2023-10-03T10:09:44.195326+0000","last_peered":"2023-10-03T10:09:44.195326+0000","last_clean":"2023-10-03T10:09:44.195326+0000","last_became_active":"2023-10-03T10:09:44.190514+0000","last_became_peered":"2023-10-03T10:09:44.190514+0000","last_unstale":"2023-10-03T10:09:44.195326+0000","last_undegraded":"2023-10-03T10:09:44.195326+0000","last_fullsized":"2023-10-03T10:09:44.195326+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.183000+0000","last_change":"2023-10-03T10:09:43.748461+0000","last_active":"2023-10-03T10:09:44.183000+0000","last_peered":"2023-10-03T10:09:44.183000+0000","last_clean":"2023-10-03T10:09:44.183000+0000","last_became_active":"2023-10-03T10:09:43.747372+0000","last_became_peered":"2023-10-03T10:09:43.747372+0000","last_unstale":"2023-10-03T10:09:44.183000+0000","last_undegraded":"2023-10-03T10:09:44.183000+0000","last_fullsized":"2023-10-03T10:09:44.183000+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196810+0000","last_change":"2023-10-03T10:09:44.192036+0000","last_active":"2023-10-03T10:09:44.196810+0000","last_peered":"2023-10-03T10:09:44.196810+0000","last_clean":"2023-10-03T10:09:44.196810+0000","last_became_active":"2023-10-03T10:09:44.191093+0000","last_became_peered":"2023-10-03T10:09:44.191093+0000","last_unstale":"2023-10-03T10:09:44.196810+0000","last_undegraded":"2023-10-03T10:09:44.196810+0000","last_fullsized":"2023-10-03T10:09:44.196810+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.174713+0000","last_change":"2023-10-03T10:09:43.756751+0000","last_active":"2023-10-03T10:09:44.174713+0000","last_peered":"2023-10-03T10:09:44.174713+0000","last_clean":"2023-10-03T10:09:44.174713+0000","last_became_active":"2023-10-03T10:09:43.756079+0000","last_became_peered":"2023-10-03T10:09:43.756079+0000","last_unstale":"2023-10-03T10:09:44.174713+0000","last_undegraded":"2023-10-03T10:09:44.174713+0000","last_fullsized":"2023-10-03T10:09:44.174713+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.179874+0000","last_change":"2023-10-03T10:09:44.172400+0000","last_active":"2023-10-03T10:09:44.179874+0000","last_peered":"2023-10-03T10:09:44.179874+0000","last_clean":"2023-10-03T10:09:44.179874+0000","last_became_active":"2023-10-03T10:09:44.172095+0000","last_became_peered":"2023-10-03T10:09:44.172095+0000","last_unstale":"2023-10-03T10:09:44.179874+0000","last_undegraded":"2023-10-03T10:09:44.179874+0000","last_fullsized":"2023-10-03T10:09:44.179874+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195109+0000","last_change":"2023-10-03T10:09:43.760622+0000","last_active":"2023-10-03T10:09:44.195109+0000","last_peered":"2023-10-03T10:09:44.195109+0000","last_clean":"2023-10-03T10:09:44.195109+0000","last_became_active":"2023-10-03T10:09:43.760371+0000","last_became_peered":"2023-10-03T10:09:43.760371+0000","last_unstale":"2023-10-03T10:09:44.195109+0000","last_undegraded":"2023-10-03T10:09:44.195109+0000","last_fullsized":"2023-10-03T10:09:44.195109+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200471+0000","last_change":"2023-10-03T10:09:44.196062+0000","last_active":"2023-10-03T10:09:44.200471+0000","last_peered":"2023-10-03T10:09:44.200471+0000","last_clean":"2023-10-03T10:09:44.200471+0000","last_became_active":"2023-10-03T10:09:44.195841+0000","last_became_peered":"2023-10-03T10:09:44.195841+0000","last_unstale":"2023-10-03T10:09:44.200471+0000","last_undegraded":"2023-10-03T10:09:44.200471+0000","last_fullsized":"2023-10-03T10:09:44.200471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.157166+0000","last_change":"2023-10-03T10:09:43.756885+0000","last_active":"2023-10-03T10:09:44.157166+0000","last_peered":"2023-10-03T10:09:44.157166+0000","last_clean":"2023-10-03T10:09:44.157166+0000","last_became_active":"2023-10-03T10:09:43.756588+0000","last_became_peered":"2023-10-03T10:09:43.756588+0000","last_unstale":"2023-10-03T10:09:44.157166+0000","last_undegraded":"2023-10-03T10:09:44.157166+0000","last_fullsized":"2023-10-03T10:09:44.157166+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184514+0000","last_change":"2023-10-03T10:09:44.181603+0000","last_active":"2023-10-03T10:09:44.184514+0000","last_peered":"2023-10-03T10:09:44.184514+0000","last_clean":"2023-10-03T10:09:44.184514+0000","last_became_active":"2023-10-03T10:09:44.181125+0000","last_became_peered":"2023-10-03T10:09:44.181125+0000","last_unstale":"2023-10-03T10:09:44.184514+0000","last_undegraded":"2023-10-03T10:09:44.184514+0000","last_fullsized":"2023-10-03T10:09:44.184514+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200822+0000","last_change":"2023-10-03T10:09:43.751873+0000","last_active":"2023-10-03T10:09:44.200822+0000","last_peered":"2023-10-03T10:09:44.200822+0000","last_clean":"2023-10-03T10:09:44.200822+0000","last_became_active":"2023-10-03T10:09:43.751324+0000","last_became_peered":"2023-10-03T10:09:43.751324+0000","last_unstale":"2023-10-03T10:09:44.200822+0000","last_undegraded":"2023-10-03T10:09:44.200822+0000","last_fullsized":"2023-10-03T10:09:44.200822+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.174302+0000","last_change":"2023-10-03T10:09:44.170676+0000","last_active":"2023-10-03T10:09:44.174302+0000","last_peered":"2023-10-03T10:09:44.174302+0000","last_clean":"2023-10-03T10:09:44.174302+0000","last_became_active":"2023-10-03T10:09:44.170398+0000","last_became_peered":"2023-10-03T10:09:44.170398+0000","last_unstale":"2023-10-03T10:09:44.174302+0000","last_undegraded":"2023-10-03T10:09:44.174302+0000","last_fullsized":"2023-10-03T10:09:44.174302+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.166137+0000","last_change":"2023-10-03T10:09:43.760711+0000","last_active":"2023-10-03T10:09:44.166137+0000","last_peered":"2023-10-03T10:09:44.166137+0000","last_clean":"2023-10-03T10:09:44.166137+0000","last_became_active":"2023-10-03T10:09:43.760432+0000","last_became_peered":"2023-10-03T10:09:43.760432+0000","last_unstale":"2023-10-03T10:09:44.166137+0000","last_undegraded":"2023-10-03T10:09:44.166137+0000","last_fullsized":"2023-10-03T10:09:44.166137+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.195829+0000","last_change":"2023-10-03T10:09:44.191073+0000","last_active":"2023-10-03T10:09:44.195829+0000","last_peered":"2023-10-03T10:09:44.195829+0000","last_clean":"2023-10-03T10:09:44.195829+0000","last_became_active":"2023-10-03T10:09:44.190218+0000","last_became_peered":"2023-10-03T10:09:44.190218+0000","last_unstale":"2023-10-03T10:09:44.195829+0000","last_undegraded":"2023-10-03T10:09:44.195829+0000","last_fullsized":"2023-10-03T10:09:44.195829+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.163521+0000","last_change":"2023-10-03T10:09:43.758846+0000","last_active":"2023-10-03T10:09:44.163521+0000","last_peered":"2023-10-03T10:09:44.163521+0000","last_clean":"2023-10-03T10:09:44.163521+0000","last_became_active":"2023-10-03T10:09:43.757973+0000","last_became_peered":"2023-10-03T10:09:43.757973+0000","last_unstale":"2023-10-03T10:09:44.163521+0000","last_undegraded":"2023-10-03T10:09:44.163521+0000","last_fullsized":"2023-10-03T10:09:44.163521+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200102+0000","last_change":"2023-10-03T10:09:44.196056+0000","last_active":"2023-10-03T10:09:44.200102+0000","last_peered":"2023-10-03T10:09:44.200102+0000","last_clean":"2023-10-03T10:09:44.200102+0000","last_became_active":"2023-10-03T10:09:44.195817+0000","last_became_peered":"2023-10-03T10:09:44.195817+0000","last_unstale":"2023-10-03T10:09:44.200102+0000","last_undegraded":"2023-10-03T10:09:44.200102+0000","last_fullsized":"2023-10-03T10:09:44.200102+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.162993+0000","last_change":"2023-10-03T10:09:43.761068+0000","last_active":"2023-10-03T10:09:44.162993+0000","last_peered":"2023-10-03T10:09:44.162993+0000","last_clean":"2023-10-03T10:09:44.162993+0000","last_became_active":"2023-10-03T10:09:43.760825+0000","last_became_peered":"2023-10-03T10:09:43.760825+0000","last_unstale":"2023-10-03T10:09:44.162993+0000","last_undegraded":"2023-10-03T10:09:44.162993+0000","last_fullsized":"2023-10-03T10:09:44.162993+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180799+0000","last_change":"2023-10-03T10:09:44.178856+0000","last_active":"2023-10-03T10:09:44.180799+0000","last_peered":"2023-10-03T10:09:44.180799+0000","last_clean":"2023-10-03T10:09:44.180799+0000","last_became_active":"2023-10-03T10:09:44.178410+0000","last_became_peered":"2023-10-03T10:09:44.178410+0000","last_unstale":"2023-10-03T10:09:44.180799+0000","last_undegraded":"2023-10-03T10:09:44.180799+0000","last_fullsized":"2023-10-03T10:09:44.180799+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.154127+0000","last_change":"2023-10-03T10:09:39.589080+0000","last_active":"2023-10-03T10:09:44.154127+0000","last_peered":"2023-10-03T10:09:44.154127+0000","last_clean":"2023-10-03T10:09:44.154127+0000","last_became_active":"2023-10-03T10:09:39.588651+0000","last_became_peered":"2023-10-03T10:09:39.588651+0000","last_unstale":"2023-10-03T10:09:44.154127+0000","last_undegraded":"2023-10-03T10:09:44.154127+0000","last_fullsized":"2023-10-03T10:09:44.154127+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.157236+0000","last_change":"2023-10-03T10:09:43.756053+0000","last_active":"2023-10-03T10:09:44.157236+0000","last_peered":"2023-10-03T10:09:44.157236+0000","last_clean":"2023-10-03T10:09:44.157236+0000","last_became_active":"2023-10-03T10:09:43.755708+0000","last_became_peered":"2023-10-03T10:09:43.755708+0000","last_unstale":"2023-10-03T10:09:44.157236+0000","last_undegraded":"2023-10-03T10:09:44.157236+0000","last_fullsized":"2023-10-03T10:09:44.157236+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180543+0000","last_change":"2023-10-03T10:09:44.176964+0000","last_active":"2023-10-03T10:09:44.180543+0000","last_peered":"2023-10-03T10:09:44.180543+0000","last_clean":"2023-10-03T10:09:44.180543+0000","last_became_active":"2023-10-03T10:09:44.176671+0000","last_became_peered":"2023-10-03T10:09:44.176671+0000","last_unstale":"2023-10-03T10:09:44.180543+0000","last_undegraded":"2023-10-03T10:09:44.180543+0000","last_fullsized":"2023-10-03T10:09:44.180543+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.166922+0000","last_change":"2023-10-03T10:09:39.586735+0000","last_active":"2023-10-03T10:09:44.166922+0000","last_peered":"2023-10-03T10:09:44.166922+0000","last_clean":"2023-10-03T10:09:44.166922+0000","last_became_active":"2023-10-03T10:09:39.586440+0000","last_became_peered":"2023-10-03T10:09:39.586440+0000","last_unstale":"2023-10-03T10:09:44.166922+0000","last_undegraded":"2023-10-03T10:09:44.166922+0000","last_fullsized":"2023-10-03T10:09:44.166922+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.159439+0000","last_change":"2023-10-03T10:09:43.748458+0000","last_active":"2023-10-03T10:09:44.159439+0000","last_peered":"2023-10-03T10:09:44.159439+0000","last_clean":"2023-10-03T10:09:44.159439+0000","last_became_active":"2023-10-03T10:09:43.747721+0000","last_became_peered":"2023-10-03T10:09:43.747721+0000","last_unstale":"2023-10-03T10:09:44.159439+0000","last_undegraded":"2023-10-03T10:09:44.159439+0000","last_fullsized":"2023-10-03T10:09:44.159439+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184166+0000","last_change":"2023-10-03T10:09:44.176165+0000","last_active":"2023-10-03T10:09:44.184166+0000","last_peered":"2023-10-03T10:09:44.184166+0000","last_clean":"2023-10-03T10:09:44.184166+0000","last_became_active":"2023-10-03T10:09:44.175853+0000","last_became_peered":"2023-10-03T10:09:44.175853+0000","last_unstale":"2023-10-03T10:09:44.184166+0000","last_undegraded":"2023-10-03T10:09:44.184166+0000","last_fullsized":"2023-10-03T10:09:44.184166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.156829+0000","last_change":"2023-10-03T10:09:39.590114+0000","last_active":"2023-10-03T10:09:44.156829+0000","last_peered":"2023-10-03T10:09:44.156829+0000","last_clean":"2023-10-03T10:09:44.156829+0000","last_became_active":"2023-10-03T10:09:39.589663+0000","last_became_peered":"2023-10-03T10:09:39.589663+0000","last_unstale":"2023-10-03T10:09:44.156829+0000","last_undegraded":"2023-10-03T10:09:44.156829+0000","last_fullsized":"2023-10-03T10:09:44.156829+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.155654+0000","last_change":"2023-10-03T10:09:43.748267+0000","last_active":"2023-10-03T10:09:44.155654+0000","last_peered":"2023-10-03T10:09:44.155654+0000","last_clean":"2023-10-03T10:09:44.155654+0000","last_became_active":"2023-10-03T10:09:43.747789+0000","last_became_peered":"2023-10-03T10:09:43.747789+0000","last_unstale":"2023-10-03T10:09:44.155654+0000","last_undegraded":"2023-10-03T10:09:44.155654+0000","last_fullsized":"2023-10-03T10:09:44.155654+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.180773+0000","last_change":"2023-10-03T10:09:44.178680+0000","last_active":"2023-10-03T10:09:44.180773+0000","last_peered":"2023-10-03T10:09:44.180773+0000","last_clean":"2023-10-03T10:09:44.180773+0000","last_became_active":"2023-10-03T10:09:44.178114+0000","last_became_peered":"2023-10-03T10:09:44.178114+0000","last_unstale":"2023-10-03T10:09:44.180773+0000","last_undegraded":"2023-10-03T10:09:44.180773+0000","last_fullsized":"2023-10-03T10:09:44.180773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.167768+0000","last_change":"2023-10-03T10:09:39.587728+0000","last_active":"2023-10-03T10:09:44.167768+0000","last_peered":"2023-10-03T10:09:44.167768+0000","last_clean":"2023-10-03T10:09:44.167768+0000","last_became_active":"2023-10-03T10:09:39.587441+0000","last_became_peered":"2023-10-03T10:09:39.587441+0000","last_unstale":"2023-10-03T10:09:44.167768+0000","last_undegraded":"2023-10-03T10:09:44.167768+0000","last_fullsized":"2023-10-03T10:09:44.167768+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.153191+0000","last_change":"2023-10-03T10:09:38.585668+0000","last_active":"2023-10-03T10:09:44.153191+0000","last_peered":"2023-10-03T10:09:44.153191+0000","last_clean":"2023-10-03T10:09:44.153191+0000","last_became_active":"2023-10-03T10:09:38.585324+0000","last_became_peered":"2023-10-03T10:09:38.585324+0000","last_unstale":"2023-10-03T10:09:44.153191+0000","last_undegraded":"2023-10-03T10:09:44.153191+0000","last_fullsized":"2023-10-03T10:09:44.153191+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:37.570571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:37.570571+0000","last_clean_scrub_stamp":"2023-10-03T10:09:37.570571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.165999+0000","last_change":"2023-10-03T10:09:43.758096+0000","last_active":"2023-10-03T10:09:44.165999+0000","last_peered":"2023-10-03T10:09:44.165999+0000","last_clean":"2023-10-03T10:09:44.165999+0000","last_became_active":"2023-10-03T10:09:43.757689+0000","last_became_peered":"2023-10-03T10:09:43.757689+0000","last_unstale":"2023-10-03T10:09:44.165999+0000","last_undegraded":"2023-10-03T10:09:44.165999+0000","last_fullsized":"2023-10-03T10:09:44.165999+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196949+0000","last_change":"2023-10-03T10:09:44.192781+0000","last_active":"2023-10-03T10:09:44.196949+0000","last_peered":"2023-10-03T10:09:44.196949+0000","last_clean":"2023-10-03T10:09:44.196949+0000","last_became_active":"2023-10-03T10:09:44.190597+0000","last_became_peered":"2023-10-03T10:09:44.190597+0000","last_unstale":"2023-10-03T10:09:44.196949+0000","last_undegraded":"2023-10-03T10:09:44.196949+0000","last_fullsized":"2023-10-03T10:09:44.196949+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.163365+0000","last_change":"2023-10-03T10:09:39.586806+0000","last_active":"2023-10-03T10:09:44.163365+0000","last_peered":"2023-10-03T10:09:44.163365+0000","last_clean":"2023-10-03T10:09:44.163365+0000","last_became_active":"2023-10-03T10:09:39.586481+0000","last_became_peered":"2023-10-03T10:09:39.586481+0000","last_unstale":"2023-10-03T10:09:44.163365+0000","last_undegraded":"2023-10-03T10:09:44.163365+0000","last_fullsized":"2023-10-03T10:09:44.163365+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.156596+0000","last_change":"2023-10-03T10:09:43.747783+0000","last_active":"2023-10-03T10:09:44.156596+0000","last_peered":"2023-10-03T10:09:44.156596+0000","last_clean":"2023-10-03T10:09:44.156596+0000","last_became_active":"2023-10-03T10:09:43.746009+0000","last_became_peered":"2023-10-03T10:09:43.746009+0000","last_unstale":"2023-10-03T10:09:44.156596+0000","last_undegraded":"2023-10-03T10:09:44.156596+0000","last_fullsized":"2023-10-03T10:09:44.156596+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184590+0000","last_change":"2023-10-03T10:09:44.181359+0000","last_active":"2023-10-03T10:09:44.184590+0000","last_peered":"2023-10-03T10:09:44.184590+0000","last_clean":"2023-10-03T10:09:44.184590+0000","last_became_active":"2023-10-03T10:09:44.181075+0000","last_became_peered":"2023-10-03T10:09:44.181075+0000","last_unstale":"2023-10-03T10:09:44.184590+0000","last_undegraded":"2023-10-03T10:09:44.184590+0000","last_fullsized":"2023-10-03T10:09:44.184590+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.168332+0000","last_change":"2023-10-03T10:09:39.588827+0000","last_active":"2023-10-03T10:09:44.168332+0000","last_peered":"2023-10-03T10:09:44.168332+0000","last_clean":"2023-10-03T10:09:44.168332+0000","last_became_active":"2023-10-03T10:09:39.588531+0000","last_became_peered":"2023-10-03T10:09:39.588531+0000","last_unstale":"2023-10-03T10:09:44.168332+0000","last_undegraded":"2023-10-03T10:09:44.168332+0000","last_fullsized":"2023-10-03T10:09:44.168332+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.152966+0000","last_change":"2023-10-03T10:09:43.746339+0000","last_active":"2023-10-03T10:09:44.152966+0000","last_peered":"2023-10-03T10:09:44.152966+0000","last_clean":"2023-10-03T10:09:44.152966+0000","last_became_active":"2023-10-03T10:09:43.746051+0000","last_became_peered":"2023-10-03T10:09:43.746051+0000","last_unstale":"2023-10-03T10:09:44.152966+0000","last_undegraded":"2023-10-03T10:09:44.152966+0000","last_fullsized":"2023-10-03T10:09:44.152966+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.173149+0000","last_change":"2023-10-03T10:09:44.165684+0000","last_active":"2023-10-03T10:09:44.173149+0000","last_peered":"2023-10-03T10:09:44.173149+0000","last_clean":"2023-10-03T10:09:44.173149+0000","last_became_active":"2023-10-03T10:09:44.165422+0000","last_became_peered":"2023-10-03T10:09:44.165422+0000","last_unstale":"2023-10-03T10:09:44.173149+0000","last_undegraded":"2023-10-03T10:09:44.173149+0000","last_fullsized":"2023-10-03T10:09:44.173149+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.197330+0000","last_change":"2023-10-03T10:09:43.760807+0000","last_active":"2023-10-03T10:09:44.197330+0000","last_peered":"2023-10-03T10:09:44.197330+0000","last_clean":"2023-10-03T10:09:44.197330+0000","last_became_active":"2023-10-03T10:09:43.760517+0000","last_became_peered":"2023-10-03T10:09:43.760517+0000","last_unstale":"2023-10-03T10:09:44.197330+0000","last_undegraded":"2023-10-03T10:09:44.197330+0000","last_fullsized":"2023-10-03T10:09:44.197330+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.153815+0000","last_change":"2023-10-03T10:09:39.586702+0000","last_active":"2023-10-03T10:09:44.153815+0000","last_peered":"2023-10-03T10:09:44.153815+0000","last_clean":"2023-10-03T10:09:44.153815+0000","last_became_active":"2023-10-03T10:09:39.586153+0000","last_became_peered":"2023-10-03T10:09:39.586153+0000","last_unstale":"2023-10-03T10:09:44.153815+0000","last_undegraded":"2023-10-03T10:09:44.153815+0000","last_fullsized":"2023-10-03T10:09:44.153815+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196883+0000","last_change":"2023-10-03T10:09:44.192664+0000","last_active":"2023-10-03T10:09:44.196883+0000","last_peered":"2023-10-03T10:09:44.196883+0000","last_clean":"2023-10-03T10:09:44.196883+0000","last_became_active":"2023-10-03T10:09:44.190513+0000","last_became_peered":"2023-10-03T10:09:44.190513+0000","last_unstale":"2023-10-03T10:09:44.196883+0000","last_undegraded":"2023-10-03T10:09:44.196883+0000","last_fullsized":"2023-10-03T10:09:44.196883+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.167083+0000","last_change":"2023-10-03T10:09:39.588950+0000","last_active":"2023-10-03T10:09:44.167083+0000","last_peered":"2023-10-03T10:09:44.167083+0000","last_clean":"2023-10-03T10:09:44.167083+0000","last_became_active":"2023-10-03T10:09:39.588601+0000","last_became_peered":"2023-10-03T10:09:39.588601+0000","last_unstale":"2023-10-03T10:09:44.167083+0000","last_undegraded":"2023-10-03T10:09:44.167083+0000","last_fullsized":"2023-10-03T10:09:44.167083+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"17'1","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-10-03T10:09:45.154970+0000","last_change":"2023-10-03T10:09:43.752793+0000","last_active":"2023-10-03T10:09:45.154970+0000","last_peered":"2023-10-03T10:09:45.154970+0000","last_clean":"2023-10-03T10:09:45.154970+0000","last_became_active":"2023-10-03T10:09:43.752374+0000","last_became_peered":"2023-10-03T10:09:43.752374+0000","last_unstale":"2023-10-03T10:09:45.154970+0000","last_undegraded":"2023-10-03T10:09:45.154970+0000","last_fullsized":"2023-10-03T10:09:45.154970+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200171+0000","last_change":"2023-10-03T10:09:44.196314+0000","last_active":"2023-10-03T10:09:44.200171+0000","last_peered":"2023-10-03T10:09:44.200171+0000","last_clean":"2023-10-03T10:09:44.200171+0000","last_became_active":"2023-10-03T10:09:44.195901+0000","last_became_peered":"2023-10-03T10:09:44.195901+0000","last_unstale":"2023-10-03T10:09:44.200171+0000","last_undegraded":"2023-10-03T10:09:44.200171+0000","last_fullsized":"2023-10-03T10:09:44.200171+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200135+0000","last_change":"2023-10-03T10:09:43.756460+0000","last_active":"2023-10-03T10:09:44.200135+0000","last_peered":"2023-10-03T10:09:44.200135+0000","last_clean":"2023-10-03T10:09:44.200135+0000","last_became_active":"2023-10-03T10:09:43.756164+0000","last_became_peered":"2023-10-03T10:09:43.756164+0000","last_unstale":"2023-10-03T10:09:44.200135+0000","last_undegraded":"2023-10-03T10:09:44.200135+0000","last_fullsized":"2023-10-03T10:09:44.200135+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.189031+0000","last_change":"2023-10-03T10:09:44.181915+0000","last_active":"2023-10-03T10:09:44.189031+0000","last_peered":"2023-10-03T10:09:44.189031+0000","last_clean":"2023-10-03T10:09:44.189031+0000","last_became_active":"2023-10-03T10:09:44.181645+0000","last_became_peered":"2023-10-03T10:09:44.181645+0000","last_unstale":"2023-10-03T10:09:44.189031+0000","last_undegraded":"2023-10-03T10:09:44.189031+0000","last_fullsized":"2023-10-03T10:09:44.189031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.193051+0000","last_change":"2023-10-03T10:09:43.748380+0000","last_active":"2023-10-03T10:09:44.193051+0000","last_peered":"2023-10-03T10:09:44.193051+0000","last_clean":"2023-10-03T10:09:44.193051+0000","last_became_active":"2023-10-03T10:09:43.748104+0000","last_became_peered":"2023-10-03T10:09:43.748104+0000","last_unstale":"2023-10-03T10:09:44.193051+0000","last_undegraded":"2023-10-03T10:09:44.193051+0000","last_fullsized":"2023-10-03T10:09:44.193051+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200359+0000","last_change":"2023-10-03T10:09:44.197710+0000","last_active":"2023-10-03T10:09:44.200359+0000","last_peered":"2023-10-03T10:09:44.200359+0000","last_clean":"2023-10-03T10:09:44.200359+0000","last_became_active":"2023-10-03T10:09:44.197064+0000","last_became_peered":"2023-10-03T10:09:44.197064+0000","last_unstale":"2023-10-03T10:09:44.200359+0000","last_undegraded":"2023-10-03T10:09:44.200359+0000","last_fullsized":"2023-10-03T10:09:44.200359+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.193435+0000","last_change":"2023-10-03T10:09:43.758701+0000","last_active":"2023-10-03T10:09:44.193435+0000","last_peered":"2023-10-03T10:09:44.193435+0000","last_clean":"2023-10-03T10:09:44.193435+0000","last_became_active":"2023-10-03T10:09:43.758376+0000","last_became_peered":"2023-10-03T10:09:43.758376+0000","last_unstale":"2023-10-03T10:09:44.193435+0000","last_undegraded":"2023-10-03T10:09:44.193435+0000","last_fullsized":"2023-10-03T10:09:44.193435+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200488+0000","last_change":"2023-10-03T10:09:44.197164+0000","last_active":"2023-10-03T10:09:44.200488+0000","last_peered":"2023-10-03T10:09:44.200488+0000","last_clean":"2023-10-03T10:09:44.200488+0000","last_became_active":"2023-10-03T10:09:44.196885+0000","last_became_peered":"2023-10-03T10:09:44.196885+0000","last_unstale":"2023-10-03T10:09:44.200488+0000","last_undegraded":"2023-10-03T10:09:44.200488+0000","last_fullsized":"2023-10-03T10:09:44.200488+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.199237+0000","last_change":"2023-10-03T10:09:43.758373+0000","last_active":"2023-10-03T10:09:44.199237+0000","last_peered":"2023-10-03T10:09:44.199237+0000","last_clean":"2023-10-03T10:09:44.199237+0000","last_became_active":"2023-10-03T10:09:43.758061+0000","last_became_peered":"2023-10-03T10:09:43.758061+0000","last_unstale":"2023-10-03T10:09:44.199237+0000","last_undegraded":"2023-10-03T10:09:44.199237+0000","last_fullsized":"2023-10-03T10:09:44.199237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.196006+0000","last_change":"2023-10-03T10:09:44.191564+0000","last_active":"2023-10-03T10:09:44.196006+0000","last_peered":"2023-10-03T10:09:44.196006+0000","last_clean":"2023-10-03T10:09:44.196006+0000","last_became_active":"2023-10-03T10:09:44.191220+0000","last_became_peered":"2023-10-03T10:09:44.191220+0000","last_unstale":"2023-10-03T10:09:44.196006+0000","last_undegraded":"2023-10-03T10:09:44.196006+0000","last_fullsized":"2023-10-03T10:09:44.196006+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"16'4","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.183675+0000","last_change":"2023-10-03T10:09:43.750463+0000","last_active":"2023-10-03T10:09:44.183675+0000","last_peered":"2023-10-03T10:09:44.183675+0000","last_clean":"2023-10-03T10:09:44.183675+0000","last_became_active":"2023-10-03T10:09:43.749780+0000","last_became_peered":"2023-10-03T10:09:43.749780+0000","last_unstale":"2023-10-03T10:09:44.183675+0000","last_undegraded":"2023-10-03T10:09:44.183675+0000","last_fullsized":"2023-10-03T10:09:44.183675+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.188984+0000","last_change":"2023-10-03T10:09:44.180378+0000","last_active":"2023-10-03T10:09:44.188984+0000","last_peered":"2023-10-03T10:09:44.188984+0000","last_clean":"2023-10-03T10:09:44.188984+0000","last_became_active":"2023-10-03T10:09:44.180082+0000","last_became_peered":"2023-10-03T10:09:44.180082+0000","last_unstale":"2023-10-03T10:09:44.188984+0000","last_undegraded":"2023-10-03T10:09:44.188984+0000","last_fullsized":"2023-10-03T10:09:44.188984+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.187808+0000","last_change":"2023-10-03T10:09:44.178067+0000","last_active":"2023-10-03T10:09:44.187808+0000","last_peered":"2023-10-03T10:09:44.187808+0000","last_clean":"2023-10-03T10:09:44.187808+0000","last_became_active":"2023-10-03T10:09:44.175124+0000","last_became_peered":"2023-10-03T10:09:44.175124+0000","last_unstale":"2023-10-03T10:09:44.187808+0000","last_undegraded":"2023-10-03T10:09:44.187808+0000","last_fullsized":"2023-10-03T10:09:44.187808+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.197083+0000","last_change":"2023-10-03T10:09:44.192156+0000","last_active":"2023-10-03T10:09:44.197083+0000","last_peered":"2023-10-03T10:09:44.197083+0000","last_clean":"2023-10-03T10:09:44.197083+0000","last_became_active":"2023-10-03T10:09:44.189296+0000","last_became_peered":"2023-10-03T10:09:44.189296+0000","last_unstale":"2023-10-03T10:09:44.197083+0000","last_undegraded":"2023-10-03T10:09:44.197083+0000","last_fullsized":"2023-10-03T10:09:44.197083+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.174106+0000","last_change":"2023-10-03T10:09:44.168473+0000","last_active":"2023-10-03T10:09:44.174106+0000","last_peered":"2023-10-03T10:09:44.174106+0000","last_clean":"2023-10-03T10:09:44.174106+0000","last_became_active":"2023-10-03T10:09:44.168155+0000","last_became_peered":"2023-10-03T10:09:44.168155+0000","last_unstale":"2023-10-03T10:09:44.174106+0000","last_undegraded":"2023-10-03T10:09:44.174106+0000","last_fullsized":"2023-10-03T10:09:44.174106+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.200881+0000","last_change":"2023-10-03T10:09:44.197689+0000","last_active":"2023-10-03T10:09:44.200881+0000","last_peered":"2023-10-03T10:09:44.200881+0000","last_clean":"2023-10-03T10:09:44.200881+0000","last_became_active":"2023-10-03T10:09:44.197410+0000","last_became_peered":"2023-10-03T10:09:44.197410+0000","last_unstale":"2023-10-03T10:09:44.200881+0000","last_undegraded":"2023-10-03T10:09:44.200881+0000","last_fullsized":"2023-10-03T10:09:44.200881+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-03T10:09:44.184190+0000","last_change":"2023-10-03T10:09:44.176966+0000","last_active":"2023-10-03T10:09:44.184190+0000","last_peered":"2023-10-03T10:09:44.184190+0000","last_clean":"2023-10-03T10:09:44.184190+0000","last_became_active":"2023-10-03T10:09:44.176386+0000","last_became_peered":"2023-10-03T10:09:44.176386+0000","last_unstale":"2023-10-03T10:09:44.184190+0000","last_undegraded":"2023-10-03T10:09:44.184190+0000","last_fullsized":"2023-10-03T10:09:44.184190+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297252,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074588,"statfs":{"total":96636764160,"available":96332378112,"internally_reserved":0,"allocated":159744,"data_stored":48735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]}]},{"osd":3,"up_from":9,"seq":38654705670,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297264,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074576,"statfs":{"total":96636764160,"available":96332365824,"internally_reserved":0,"allocated":172032,"data_stored":48929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]}]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297256,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074584,"statfs":{"total":96636764160,"available":96332374016,"internally_reserved":0,"allocated":163840,"data_stored":47899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]}]},{"osd":1,"up_from":9,"seq":38654705670,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297252,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074588,"statfs":{"total":96636764160,"available":96332378112,"internally_reserved":0,"allocated":159744,"data_stored":47845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-03T10:09:58.409 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-03T10:09:58.410 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-03T10:09:58.410 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-03T10:09:58.410 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-03T10:09:58.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.576+0000 7f30284fb700 1 Processor -- start 2023-10-03T10:09:58.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.576+0000 7f30284fb700 1 -- start start 2023-10-03T10:09:58.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.577+0000 7f30284fb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f302009ab50 0x7f3020098970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:58.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.577+0000 7f30284fb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3020069670 0x7f3020130bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:58.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.577+0000 7f30284fb700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3020096520 con 0x7f3020098e60 2023-10-03T10:09:58.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.577+0000 7f30284fb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f302009af30 con 0x7f302009ab50 2023-10-03T10:09:58.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.577+0000 7f30284fb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3020095830 con 0x7f3020069670 2023-10-03T10:09:58.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.577+0000 7f3026a98700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3020069670 0x7f3020130bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:58.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.577+0000 7f3026297700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f302009ab50 0x7f3020098970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:58.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.577+0000 7f3025a96700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3020098e60 0x7f3020068f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38576/0 (socket says 172.21.15.3:38576) 2023-10-03T10:09:58.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.577+0000 7f3025a96700 1 -- 172.21.15.3:0/588370492 learned_addr learned my addr 172.21.15.3:0/588370492 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:09:58.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.577+0000 7f3025295700 1 -- 172.21.15.3:0/588370492 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1740989687 0 0) 0x7f3020096520 con 0x7f3020098e60 2023-10-03T10:09:58.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.577+0000 7f3025295700 1 -- 172.21.15.3:0/588370492 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f300c0033f0 con 0x7f3020098e60 2023-10-03T10:09:58.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.577+0000 7f3026297700 1 -- 172.21.15.3:0/588370492 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3020069670 msgr2=0x7f3020130bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:58.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.577+0000 7f3026297700 1 --2- 172.21.15.3:0/588370492 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3020069670 0x7f3020130bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:58.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.577+0000 7f3026297700 1 -- 172.21.15.3:0/588370492 >> v1:172.21.15.3:6789/0 conn(0x7f3020098e60 legacy=0x7f3020068f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:58.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.577+0000 7f3026297700 1 -- 172.21.15.3:0/588370492 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3020093e60 con 0x7f302009ab50 2023-10-03T10:09:58.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.577+0000 7f3026a98700 1 --2- 172.21.15.3:0/588370492 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3020069670 0x7f3020130bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:58.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.578+0000 7f3026297700 1 --2- 172.21.15.3:0/588370492 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f302009ab50 0x7f3020098970 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f301c0097c0 tx=0x7f301c009060).ready entity=mon.1 client_cookie=b1506d4f69320a8d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:58.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.578+0000 7f3025295700 1 -- 172.21.15.3:0/588370492 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301c00d4a0 con 0x7f302009ab50 2023-10-03T10:09:58.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.578+0000 7f3025295700 1 -- 172.21.15.3:0/588370492 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f301c00d600 con 0x7f302009ab50 2023-10-03T10:09:58.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.578+0000 7f3025295700 1 -- 172.21.15.3:0/588370492 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301c00d980 con 0x7f302009ab50 2023-10-03T10:09:58.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.578+0000 7f30284fb700 1 -- 172.21.15.3:0/588370492 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f302009ab50 msgr2=0x7f3020098970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:58.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.578+0000 7f30284fb700 1 --2- 172.21.15.3:0/588370492 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f302009ab50 0x7f3020098970 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f301c0097c0 tx=0x7f301c009060).stop 2023-10-03T10:09:58.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.578+0000 7f30284fb700 1 -- 172.21.15.3:0/588370492 shutdown_connections 2023-10-03T10:09:58.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.578+0000 7f30284fb700 1 --2- 172.21.15.3:0/588370492 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f302009ab50 0x7f3020098970 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:58.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.578+0000 7f30284fb700 1 --2- 172.21.15.3:0/588370492 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3020069670 0x7f3020130bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:58.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.578+0000 7f30284fb700 1 -- 172.21.15.3:0/588370492 >> 172.21.15.3:0/588370492 conn(0x7f3020063190 msgr2=0x7f302009a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:58.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.578+0000 7f30284fb700 1 -- 172.21.15.3:0/588370492 shutdown_connections 2023-10-03T10:09:58.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.578+0000 7f30284fb700 1 -- 172.21.15.3:0/588370492 wait complete. 2023-10-03T10:09:58.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.579+0000 7f30284fb700 1 Processor -- start 2023-10-03T10:09:58.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.579+0000 7f30284fb700 1 -- start start 2023-10-03T10:09:58.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.580+0000 7f30284fb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3020069670 0x7f3020117660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:58.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.580+0000 7f30284fb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3020098e60 0x7f3020117b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:58.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.580+0000 7f30284fb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f302009ab50 0x7f3020118040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:58.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.580+0000 7f30284fb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3020066b20 con 0x7f3020069670 2023-10-03T10:09:58.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.580+0000 7f30284fb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3020094c40 con 0x7f3020098e60 2023-10-03T10:09:58.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.580+0000 7f30284fb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3020119280 con 0x7f302009ab50 2023-10-03T10:09:58.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.580+0000 7f3026297700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3020069670 0x7f3020117660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:58.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.580+0000 7f3026297700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3020069670 0x7f3020117660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47868/0 (socket says 172.21.15.3:47868) 2023-10-03T10:09:58.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.580+0000 7f3025a96700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3020098e60 0x7f3020117b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:58.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.580+0000 7f3026297700 1 -- 172.21.15.3:0/2727318065 learned_addr learned my addr 172.21.15.3:0/2727318065 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:58.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.580+0000 7f3026a98700 1 --2- 172.21.15.3:0/2727318065 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f302009ab50 0x7f3020118040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:58.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.580+0000 7f3026297700 1 -- 172.21.15.3:0/2727318065 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f302009ab50 msgr2=0x7f3020118040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:58.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.580+0000 7f3026297700 1 --2- 172.21.15.3:0/2727318065 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f302009ab50 0x7f3020118040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:58.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.580+0000 7f3026297700 1 -- 172.21.15.3:0/2727318065 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3020098e60 msgr2=0x7f3020117b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:58.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.580+0000 7f3026297700 1 --2- 172.21.15.3:0/2727318065 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3020098e60 0x7f3020117b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:58.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.580+0000 7f3026297700 1 -- 172.21.15.3:0/2727318065 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3018009a20 con 0x7f3020069670 2023-10-03T10:09:58.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.580+0000 7f3026a98700 1 --2- 172.21.15.3:0/2727318065 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f302009ab50 0x7f3020118040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:58.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.580+0000 7f3025a96700 1 --2- 172.21.15.3:0/2727318065 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3020098e60 0x7f3020117b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:58.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.581+0000 7f3026297700 1 --2- 172.21.15.3:0/2727318065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3020069670 0x7f3020117660 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f301c009260 tx=0x7f301c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:58.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.581+0000 7f30167fc700 1 -- 172.21.15.3:0/2727318065 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301c000c90 con 0x7f3020069670 2023-10-03T10:09:58.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.581+0000 7f30167fc700 1 -- 172.21.15.3:0/2727318065 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f301c00d770 con 0x7f3020069670 2023-10-03T10:09:58.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.581+0000 7f30284fb700 1 -- 172.21.15.3:0/2727318065 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f301c0092a0 con 0x7f3020069670 2023-10-03T10:09:58.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.581+0000 7f30167fc700 1 -- 172.21.15.3:0/2727318065 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301c01d660 con 0x7f3020069670 2023-10-03T10:09:58.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.581+0000 7f30284fb700 1 -- 172.21.15.3:0/2727318065 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f302011b2b0 con 0x7f3020069670 2023-10-03T10:09:58.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.581+0000 7f30167fc700 1 -- 172.21.15.3:0/2727318065 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f301c01da90 con 0x7f3020069670 2023-10-03T10:09:58.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.582+0000 7f30167fc700 1 -- 172.21.15.3:0/2727318065 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f301c03c7b0 con 0x7f3020069670 2023-10-03T10:09:58.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.582+0000 7f30284fb700 1 -- 172.21.15.3:0/2727318065 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3020118530 con 0x7f3020069670 2023-10-03T10:09:58.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.587+0000 7f30167fc700 1 -- 172.21.15.3:0/2727318065 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f301c01dd40 con 0x7f3020069670 2023-10-03T10:09:58.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.801+0000 7f30284fb700 1 -- 172.21.15.3:0/2727318065 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f302011b2b0 con 0x7f3020069670 2023-10-03T10:09:58.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.801+0000 7f30167fc700 1 -- 172.21.15.3:0/2727318065 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7f302011b2b0 con 0x7f3020069670 2023-10-03T10:09:58.802 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:58.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.803+0000 7f30284fb700 1 -- 172.21.15.3:0/2727318065 >> v1:172.21.15.3:6816/118296 conn(0x7f300402bb60 legacy=0x7f300402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:58.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.803+0000 7f30284fb700 1 -- 172.21.15.3:0/2727318065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3020069670 msgr2=0x7f3020117660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:58.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.803+0000 7f30284fb700 1 --2- 172.21.15.3:0/2727318065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3020069670 0x7f3020117660 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f301c009260 tx=0x7f301c016000).stop 2023-10-03T10:09:58.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.803+0000 7f30284fb700 1 -- 172.21.15.3:0/2727318065 shutdown_connections 2023-10-03T10:09:58.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.803+0000 7f30284fb700 1 --2- 172.21.15.3:0/2727318065 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f302009ab50 0x7f3020118040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:58.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.803+0000 7f30284fb700 1 --2- 172.21.15.3:0/2727318065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3020069670 0x7f3020117660 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:58.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.803+0000 7f30284fb700 1 --2- 172.21.15.3:0/2727318065 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3020098e60 0x7f3020117b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:58.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.803+0000 7f30284fb700 1 -- 172.21.15.3:0/2727318065 >> 172.21.15.3:0/2727318065 conn(0x7f3020063190 msgr2=0x7f302012bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:58.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.803+0000 7f30284fb700 1 -- 172.21.15.3:0/2727318065 shutdown_connections 2023-10-03T10:09:58.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.804+0000 7f30284fb700 1 -- 172.21.15.3:0/2727318065 wait complete. 2023-10-03T10:09:58.817 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-03T10:09:58.817 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-03T10:09:58.817 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-03T10:09:58.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.947+0000 7f2ae46a8700 1 Processor -- start 2023-10-03T10:09:58.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.947+0000 7f2ae46a8700 1 -- start start 2023-10-03T10:09:58.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.947+0000 7f2ae46a8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2adc09ab50 0x7f2adc098970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:58.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.947+0000 7f2ae46a8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2adc069670 0x7f2adc130bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:58.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.947+0000 7f2ae46a8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2adc096520 con 0x7f2adc098e60 2023-10-03T10:09:58.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.948+0000 7f2ae46a8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc09af30 con 0x7f2adc069670 2023-10-03T10:09:58.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.948+0000 7f2ae46a8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc095830 con 0x7f2adc09ab50 2023-10-03T10:09:58.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.948+0000 7f2ae2444700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2adc09ab50 0x7f2adc098970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:58.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.948+0000 7f2ae2c45700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2adc069670 0x7f2adc130bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:58.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.948+0000 7f2ae1c43700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2adc098e60 0x7f2adc068f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38580/0 (socket says 172.21.15.3:38580) 2023-10-03T10:09:58.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.948+0000 7f2ae1c43700 1 -- 172.21.15.3:0/2788161217 learned_addr learned my addr 172.21.15.3:0/2788161217 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:09:58.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.948+0000 7f2ae1442700 1 -- 172.21.15.3:0/2788161217 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3749623885 0 0) 0x7f2adc096520 con 0x7f2adc098e60 2023-10-03T10:09:58.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.948+0000 7f2ae2c45700 1 -- 172.21.15.3:0/2788161217 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2adc09ab50 msgr2=0x7f2adc098970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:58.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.948+0000 7f2ae2c45700 1 --2- 172.21.15.3:0/2788161217 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2adc09ab50 0x7f2adc098970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:58.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.948+0000 7f2ae2c45700 1 -- 172.21.15.3:0/2788161217 >> v1:172.21.15.3:6789/0 conn(0x7f2adc098e60 legacy=0x7f2adc068f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:58.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.948+0000 7f2ae2c45700 1 -- 172.21.15.3:0/2788161217 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2adc093e60 con 0x7f2adc069670 2023-10-03T10:09:58.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.948+0000 7f2ae2444700 1 --2- 172.21.15.3:0/2788161217 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2adc09ab50 0x7f2adc098970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:58.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.948+0000 7f2ae2c45700 1 --2- 172.21.15.3:0/2788161217 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2adc069670 0x7f2adc130bf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f2ad4009ec0 tx=0x7f2ad4009780).ready entity=mon.1 client_cookie=712b66daec2f88a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:58.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.948+0000 7f2ae1442700 1 -- 172.21.15.3:0/2788161217 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad400e070 con 0x7f2adc069670 2023-10-03T10:09:58.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.948+0000 7f2ae1442700 1 -- 172.21.15.3:0/2788161217 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad400bbf0 con 0x7f2adc069670 2023-10-03T10:09:58.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.948+0000 7f2ae1442700 1 -- 172.21.15.3:0/2788161217 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4012410 con 0x7f2adc069670 2023-10-03T10:09:58.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.949+0000 7f2ae46a8700 1 -- 172.21.15.3:0/2788161217 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2adc069670 msgr2=0x7f2adc130bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:58.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.949+0000 7f2ae46a8700 1 --2- 172.21.15.3:0/2788161217 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2adc069670 0x7f2adc130bf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f2ad4009ec0 tx=0x7f2ad4009780).stop 2023-10-03T10:09:58.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.949+0000 7f2ae46a8700 1 -- 172.21.15.3:0/2788161217 shutdown_connections 2023-10-03T10:09:58.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.949+0000 7f2ae46a8700 1 --2- 172.21.15.3:0/2788161217 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2adc069670 0x7f2adc130bf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:58.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.949+0000 7f2ae46a8700 1 --2- 172.21.15.3:0/2788161217 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2adc09ab50 0x7f2adc098970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:58.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.949+0000 7f2ae46a8700 1 -- 172.21.15.3:0/2788161217 >> 172.21.15.3:0/2788161217 conn(0x7f2adc063190 msgr2=0x7f2adc09a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:58.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.949+0000 7f2ae46a8700 1 -- 172.21.15.3:0/2788161217 shutdown_connections 2023-10-03T10:09:58.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.949+0000 7f2ae46a8700 1 -- 172.21.15.3:0/2788161217 wait complete. 2023-10-03T10:09:58.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.950+0000 7f2ae46a8700 1 Processor -- start 2023-10-03T10:09:58.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.950+0000 7f2ae46a8700 1 -- start start 2023-10-03T10:09:58.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.951+0000 7f2ae46a8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2adc069670 0x7f2adc138220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:58.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.951+0000 7f2ae46a8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2adc098e60 0x7f2adc138710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:58.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.951+0000 7f2ae46a8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2adc09ab50 0x7f2adc117f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:58.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.951+0000 7f2ae46a8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc11bbc0 con 0x7f2adc098e60 2023-10-03T10:09:58.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.951+0000 7f2ae46a8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc12cf70 con 0x7f2adc09ab50 2023-10-03T10:09:58.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.951+0000 7f2ae46a8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc066960 con 0x7f2adc069670 2023-10-03T10:09:58.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.951+0000 7f2ae2444700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2adc069670 0x7f2adc138220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:58.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.951+0000 7f2ae2444700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2adc069670 0x7f2adc138220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57598/0 (socket says 172.21.15.3:57598) 2023-10-03T10:09:58.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.951+0000 7f2ae2444700 1 -- 172.21.15.3:0/840358485 learned_addr learned my addr 172.21.15.3:0/840358485 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:58.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.951+0000 7f2ae2c45700 1 --2- 172.21.15.3:0/840358485 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2adc09ab50 0x7f2adc117f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:58.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.951+0000 7f2ae1c43700 1 --2- 172.21.15.3:0/840358485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2adc098e60 0x7f2adc138710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:58.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.951+0000 7f2ae2c45700 1 -- 172.21.15.3:0/840358485 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2adc069670 msgr2=0x7f2adc138220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:58.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.951+0000 7f2ae2c45700 1 --2- 172.21.15.3:0/840358485 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2adc069670 0x7f2adc138220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:58.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.951+0000 7f2ae2c45700 1 -- 172.21.15.3:0/840358485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2adc098e60 msgr2=0x7f2adc138710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:58.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.951+0000 7f2ae2c45700 1 --2- 172.21.15.3:0/840358485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2adc098e60 0x7f2adc138710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:58.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.951+0000 7f2ae2c45700 1 -- 172.21.15.3:0/840358485 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad80092a0 con 0x7f2adc09ab50 2023-10-03T10:09:58.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.951+0000 7f2ae2444700 1 --2- 172.21.15.3:0/840358485 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2adc069670 0x7f2adc138220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:58.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.951+0000 7f2ae1c43700 1 --2- 172.21.15.3:0/840358485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2adc098e60 0x7f2adc138710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:58.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.951+0000 7f2ae2c45700 1 --2- 172.21.15.3:0/840358485 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2adc09ab50 0x7f2adc117f30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f2ad4009ec0 tx=0x7f2adc0954e0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:58.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.951+0000 7f2ad27fc700 1 -- 172.21.15.3:0/840358485 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad401d630 con 0x7f2adc09ab50 2023-10-03T10:09:58.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.951+0000 7f2ad27fc700 1 -- 172.21.15.3:0/840358485 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad401d790 con 0x7f2adc09ab50 2023-10-03T10:09:58.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.951+0000 7f2ae46a8700 1 -- 172.21.15.3:0/840358485 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad4009a20 con 0x7f2adc09ab50 2023-10-03T10:09:58.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.951+0000 7f2ad27fc700 1 -- 172.21.15.3:0/840358485 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad401db50 con 0x7f2adc09ab50 2023-10-03T10:09:58.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.951+0000 7f2ae46a8700 1 -- 172.21.15.3:0/840358485 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2adc11a460 con 0x7f2adc09ab50 2023-10-03T10:09:58.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.952+0000 7f2ad27fc700 1 -- 172.21.15.3:0/840358485 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f2ad4003040 con 0x7f2adc09ab50 2023-10-03T10:09:58.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.953+0000 7f2ad27fc700 1 -- 172.21.15.3:0/840358485 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f2ad403cbd0 con 0x7f2adc09ab50 2023-10-03T10:09:58.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.953+0000 7f2ae46a8700 1 -- 172.21.15.3:0/840358485 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2adc0663b0 con 0x7f2adc09ab50 2023-10-03T10:09:58.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:58.957+0000 7f2ad27fc700 1 -- 172.21.15.3:0/840358485 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f2ad4025b10 con 0x7f2adc09ab50 2023-10-03T10:09:59.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.133+0000 7f2ae46a8700 1 -- 172.21.15.3:0/840358485 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2adc05a260 con 0x7f2adc09ab50 2023-10-03T10:09:59.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.134+0000 7f2ad27fc700 1 -- 172.21.15.3:0/840358485 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+11309 (secure 0 0 0) 0x7f2ad4025b10 con 0x7f2adc09ab50 2023-10-03T10:09:59.135 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:09:59.135 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":17,"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","created":"2023-10-03T10:09:28.703441+0000","modified":"2023-10-03T10:09:45.145435+0000","last_up_change":"2023-10-03T10:09:36.569401+0000","last_in_change":"2023-10-03T10:09:30.156546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-10-03T10:09:36.623971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-03T10:09:38.210614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-03T10:09:42.695873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-03T10:09:43.115588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cdcdd930-3c5b-4486-a021-7132f90fb6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6808","nonce":118845}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6809","nonce":118845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6811","nonce":118845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6810","nonce":118845}]},"public_addr":"172.21.15.3:6808/118845","cluster_addr":"172.21.15.3:6809/118845","heartbeat_back_addr":"172.21.15.3:6811/118845","heartbeat_front_addr":"172.21.15.3:6810/118845","state":["exists","up"]},{"osd":1,"uuid":"c8e58877-6762-4741-a8c1-567ebc97b708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6804","nonce":118835}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6805","nonce":118835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6807","nonce":118835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6806","nonce":118835}]},"public_addr":"172.21.15.3:6804/118835","cluster_addr":"172.21.15.3:6805/118835","heartbeat_back_addr":"172.21.15.3:6807/118835","heartbeat_front_addr":"172.21.15.3:6806/118835","state":["exists","up"]},{"osd":2,"uuid":"68a2a21a-7a8e-4f82-8433-5201faa50777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6812","nonce":118864}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6813","nonce":118864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6815","nonce":118864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6814","nonce":118864}]},"public_addr":"172.21.15.3:6812/118864","cluster_addr":"172.21.15.3:6813/118864","heartbeat_back_addr":"172.21.15.3:6815/118864","heartbeat_front_addr":"172.21.15.3:6814/118864","state":["exists","up"]},{"osd":3,"uuid":"c4df2903-4e1e-4208-bf77-95b9fa749241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6800","nonce":118836}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6801","nonce":118836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6803","nonce":118836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6802","nonce":118836}]},"public_addr":"172.21.15.3:6800/118836","cluster_addr":"172.21.15.3:6801/118836","heartbeat_back_addr":"172.21.15.3:6803/118836","heartbeat_front_addr":"172.21.15.3:6802/118836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.369708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.257825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.337931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.085222+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-03T10:09:59.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.136+0000 7f2ae46a8700 1 -- 172.21.15.3:0/840358485 >> v1:172.21.15.3:6816/118296 conn(0x7f2ac002bb90 legacy=0x7f2ac002e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:59.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.136+0000 7f2ae46a8700 1 -- 172.21.15.3:0/840358485 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2adc09ab50 msgr2=0x7f2adc117f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:59.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.136+0000 7f2ae46a8700 1 --2- 172.21.15.3:0/840358485 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2adc09ab50 0x7f2adc117f30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f2ad4009ec0 tx=0x7f2adc0954e0).stop 2023-10-03T10:09:59.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.136+0000 7f2ae46a8700 1 -- 172.21.15.3:0/840358485 shutdown_connections 2023-10-03T10:09:59.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.136+0000 7f2ae46a8700 1 --2- 172.21.15.3:0/840358485 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2adc09ab50 0x7f2adc117f30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:59.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.136+0000 7f2ae46a8700 1 --2- 172.21.15.3:0/840358485 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2adc069670 0x7f2adc138220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:59.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.136+0000 7f2ae46a8700 1 --2- 172.21.15.3:0/840358485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2adc098e60 0x7f2adc138710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:59.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.136+0000 7f2ae46a8700 1 -- 172.21.15.3:0/840358485 >> 172.21.15.3:0/840358485 conn(0x7f2adc063190 msgr2=0x7f2adc0681b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:59.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.136+0000 7f2ae46a8700 1 -- 172.21.15.3:0/840358485 shutdown_connections 2023-10-03T10:09:59.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.136+0000 7f2ae46a8700 1 -- 172.21.15.3:0/840358485 wait complete. 2023-10-03T10:09:59.150 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-10-03T10:09:36.623971+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-03T10:09:38.210614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-03T10:09:42.695873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-03T10:09:43.115588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-03T10:09:59.150 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-10-03T10:09:59.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.284+0000 7fdc973a4700 1 Processor -- start 2023-10-03T10:09:59.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.284+0000 7fdc973a4700 1 -- start start 2023-10-03T10:09:59.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.284+0000 7fdc973a4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc90062a90 0x7fdc90098d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:59.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.285+0000 7fdc973a4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc90098800 0x7fdc90099230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:59.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.285+0000 7fdc973a4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc900928d0 con 0x7fdc9012a940 2023-10-03T10:09:59.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.285+0000 7fdc973a4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc90062e70 con 0x7fdc90098800 2023-10-03T10:09:59.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.285+0000 7fdc973a4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc90091a40 con 0x7fdc90062a90 2023-10-03T10:09:59.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.285+0000 7fdc9493f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc90098800 0x7fdc90099230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:59.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.285+0000 7fdc95941700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdc9012a940 0x7fdc9012adc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38584/0 (socket says 172.21.15.3:38584) 2023-10-03T10:09:59.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.285+0000 7fdc95140700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc90062a90 0x7fdc90098d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:59.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.285+0000 7fdc95941700 1 -- 172.21.15.3:0/2383632694 learned_addr learned my addr 172.21.15.3:0/2383632694 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:09:59.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.285+0000 7fdc9493f700 1 -- 172.21.15.3:0/2383632694 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc90062a90 msgr2=0x7fdc90098d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:59.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.285+0000 7fdc9493f700 1 --2- 172.21.15.3:0/2383632694 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc90062a90 0x7fdc90098d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:59.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.285+0000 7fdc9493f700 1 -- 172.21.15.3:0/2383632694 >> v1:172.21.15.3:6789/0 conn(0x7fdc9012a940 legacy=0x7fdc9012adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:59.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.285+0000 7fdc87fff700 1 -- 172.21.15.3:0/2383632694 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181852824 0 0) 0x7fdc900928d0 con 0x7fdc9012a940 2023-10-03T10:09:59.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.285+0000 7fdc9493f700 1 -- 172.21.15.3:0/2383632694 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc90099720 con 0x7fdc90098800 2023-10-03T10:09:59.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.285+0000 7fdc9493f700 1 --2- 172.21.15.3:0/2383632694 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc90098800 0x7fdc90099230 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fdc80009790 tx=0x7fdc80009060).ready entity=mon.1 client_cookie=2568ae1f615cdbb server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:59.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.285+0000 7fdc87fff700 1 -- 172.21.15.3:0/2383632694 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8000f510 con 0x7fdc90098800 2023-10-03T10:09:59.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.285+0000 7fdc87fff700 1 -- 172.21.15.3:0/2383632694 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8000f670 con 0x7fdc90098800 2023-10-03T10:09:59.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.285+0000 7fdc87fff700 1 -- 172.21.15.3:0/2383632694 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8000f9f0 con 0x7fdc90098800 2023-10-03T10:09:59.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.286+0000 7fdc973a4700 1 -- 172.21.15.3:0/2383632694 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc90098800 msgr2=0x7fdc90099230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:59.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.286+0000 7fdc973a4700 1 --2- 172.21.15.3:0/2383632694 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc90098800 0x7fdc90099230 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fdc80009790 tx=0x7fdc80009060).stop 2023-10-03T10:09:59.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.286+0000 7fdc973a4700 1 -- 172.21.15.3:0/2383632694 shutdown_connections 2023-10-03T10:09:59.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.286+0000 7fdc973a4700 1 --2- 172.21.15.3:0/2383632694 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc90062a90 0x7fdc90098d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:59.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.286+0000 7fdc973a4700 1 --2- 172.21.15.3:0/2383632694 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc90098800 0x7fdc90099230 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:59.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.286+0000 7fdc973a4700 1 -- 172.21.15.3:0/2383632694 >> 172.21.15.3:0/2383632694 conn(0x7fdc900599a0 msgr2=0x7fdc90097460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:59.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.286+0000 7fdc973a4700 1 -- 172.21.15.3:0/2383632694 shutdown_connections 2023-10-03T10:09:59.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.286+0000 7fdc973a4700 1 -- 172.21.15.3:0/2383632694 wait complete. 2023-10-03T10:09:59.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.287+0000 7fdc973a4700 1 Processor -- start 2023-10-03T10:09:59.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.287+0000 7fdc973a4700 1 -- start start 2023-10-03T10:09:59.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.288+0000 7fdc973a4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc90062a90 0x7fdc9011afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:59.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.288+0000 7fdc973a4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc90098800 0x7fdc90117050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:59.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.288+0000 7fdc973a4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc9012a940 0x7fdc90117540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:59.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.288+0000 7fdc973a4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc90119460 con 0x7fdc90098800 2023-10-03T10:09:59.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.288+0000 7fdc973a4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc90128030 con 0x7fdc90062a90 2023-10-03T10:09:59.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.288+0000 7fdc973a4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc900924e0 con 0x7fdc9012a940 2023-10-03T10:09:59.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.288+0000 7fdc95140700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc90062a90 0x7fdc9011afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:59.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.288+0000 7fdc95941700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc9012a940 0x7fdc90117540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:59.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.288+0000 7fdc95140700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc90062a90 0x7fdc9011afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39258/0 (socket says 172.21.15.3:39258) 2023-10-03T10:09:59.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.288+0000 7fdc95140700 1 -- 172.21.15.3:0/3601805060 learned_addr learned my addr 172.21.15.3:0/3601805060 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:59.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.288+0000 7fdc9493f700 1 --2- 172.21.15.3:0/3601805060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc90098800 0x7fdc90117050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:59.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.288+0000 7fdc95941700 1 -- 172.21.15.3:0/3601805060 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc90062a90 msgr2=0x7fdc9011afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:59.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.288+0000 7fdc95941700 1 --2- 172.21.15.3:0/3601805060 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc90062a90 0x7fdc9011afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:59.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.288+0000 7fdc95941700 1 -- 172.21.15.3:0/3601805060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc90098800 msgr2=0x7fdc90117050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:59.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.288+0000 7fdc95941700 1 --2- 172.21.15.3:0/3601805060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc90098800 0x7fdc90117050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:59.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.288+0000 7fdc95941700 1 -- 172.21.15.3:0/3601805060 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc800092a0 con 0x7fdc9012a940 2023-10-03T10:09:59.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.288+0000 7fdc95941700 1 --2- 172.21.15.3:0/3601805060 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc9012a940 0x7fdc90117540 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fdc8800c7e0 tx=0x7fdc88007eb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:59.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.288+0000 7fdc857fa700 1 -- 172.21.15.3:0/3601805060 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc88005d30 con 0x7fdc9012a940 2023-10-03T10:09:59.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.288+0000 7fdc857fa700 1 -- 172.21.15.3:0/3601805060 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc88005e90 con 0x7fdc9012a940 2023-10-03T10:09:59.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.289+0000 7fdc973a4700 1 -- 172.21.15.3:0/3601805060 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc901282c0 con 0x7fdc9012a940 2023-10-03T10:09:59.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.289+0000 7fdc857fa700 1 -- 172.21.15.3:0/3601805060 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc88007360 con 0x7fdc9012a940 2023-10-03T10:09:59.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.289+0000 7fdc973a4700 1 -- 172.21.15.3:0/3601805060 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc90118b90 con 0x7fdc9012a940 2023-10-03T10:09:59.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.289+0000 7fdc857fa700 1 -- 172.21.15.3:0/3601805060 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fdc88007710 con 0x7fdc9012a940 2023-10-03T10:09:59.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.289+0000 7fdc857fa700 1 -- 172.21.15.3:0/3601805060 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fdc8803ad90 con 0x7fdc9012a940 2023-10-03T10:09:59.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.290+0000 7fdc973a4700 1 -- 172.21.15.3:0/3601805060 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc90118d80 con 0x7fdc9012a940 2023-10-03T10:09:59.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.294+0000 7fdc857fa700 1 -- 172.21.15.3:0/3601805060 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fdc8801a300 con 0x7fdc9012a940 2023-10-03T10:09:59.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.466+0000 7fdc973a4700 1 -- 172.21.15.3:0/3601805060 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fdc90128260 con 0x7fdc9012a940 2023-10-03T10:09:59.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.467+0000 7fdc857fa700 1 -- 172.21.15.3:0/3601805060 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v17) v1 ==== 110+0+10 (secure 0 0 0) 0x7fdc88011b50 con 0x7fdc9012a940 2023-10-03T10:09:59.467 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-03T10:09:59.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.469+0000 7fdc973a4700 1 -- 172.21.15.3:0/3601805060 >> v1:172.21.15.3:6816/118296 conn(0x7fdc7402bb60 legacy=0x7fdc7402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:59.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.469+0000 7fdc973a4700 1 -- 172.21.15.3:0/3601805060 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc9012a940 msgr2=0x7fdc90117540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:59.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.469+0000 7fdc973a4700 1 --2- 172.21.15.3:0/3601805060 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc9012a940 0x7fdc90117540 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fdc8800c7e0 tx=0x7fdc88007eb0).stop 2023-10-03T10:09:59.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.469+0000 7fdc973a4700 1 -- 172.21.15.3:0/3601805060 shutdown_connections 2023-10-03T10:09:59.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.469+0000 7fdc973a4700 1 --2- 172.21.15.3:0/3601805060 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc9012a940 0x7fdc90117540 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:59.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.469+0000 7fdc973a4700 1 --2- 172.21.15.3:0/3601805060 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc90062a90 0x7fdc9011afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:59.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.469+0000 7fdc973a4700 1 --2- 172.21.15.3:0/3601805060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc90098800 0x7fdc90117050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:59.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.469+0000 7fdc973a4700 1 -- 172.21.15.3:0/3601805060 >> 172.21.15.3:0/3601805060 conn(0x7fdc900599a0 msgr2=0x7fdc90096c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:59.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.469+0000 7fdc973a4700 1 -- 172.21.15.3:0/3601805060 shutdown_connections 2023-10-03T10:09:59.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.469+0000 7fdc973a4700 1 -- 172.21.15.3:0/3601805060 wait complete. 2023-10-03T10:09:59.482 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-03T10:09:59.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.617+0000 7f7ec67f2700 1 Processor -- start 2023-10-03T10:09:59.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.618+0000 7f7ec67f2700 1 -- start start 2023-10-03T10:09:59.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.618+0000 7f7ec67f2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ec012b7d0 0x7f7ec012ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:59.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.618+0000 7f7ec67f2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7ec01304c0 0x7f7ec01328b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:59.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.618+0000 7f7ec67f2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec0098270 con 0x7f7ec0059070 2023-10-03T10:09:59.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.618+0000 7f7ec67f2700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ec0059450 con 0x7f7ec01304c0 2023-10-03T10:09:59.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.618+0000 7f7ec67f2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec012bbb0 con 0x7f7ec012b7d0 2023-10-03T10:09:59.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.618+0000 7f7ebf7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ec012b7d0 0x7f7ec012ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:59.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.618+0000 7f7ec4d8f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7ec01304c0 0x7f7ec01328b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:59.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.618+0000 7f7ebffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7ec0059070 0x7f7ec012b0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38588/0 (socket says 172.21.15.3:38588) 2023-10-03T10:09:59.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.618+0000 7f7ebffff700 1 -- 172.21.15.3:0/1201107564 learned_addr learned my addr 172.21.15.3:0/1201107564 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:09:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.619+0000 7f7ec4d8f700 1 -- 172.21.15.3:0/1201107564 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ec012b7d0 msgr2=0x7f7ec012ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.619+0000 7f7ec4d8f700 1 --2- 172.21.15.3:0/1201107564 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ec012b7d0 0x7f7ec012ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.619+0000 7f7ec4d8f700 1 -- 172.21.15.3:0/1201107564 >> v1:172.21.15.3:6789/0 conn(0x7f7ec0059070 legacy=0x7f7ec012b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.619+0000 7f7ebeffd700 1 -- 172.21.15.3:0/1201107564 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1382972979 0 0) 0x7f7ec0098270 con 0x7f7ec0059070 2023-10-03T10:09:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.619+0000 7f7ec4d8f700 1 -- 172.21.15.3:0/1201107564 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec00998e0 con 0x7f7ec01304c0 2023-10-03T10:09:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.619+0000 7f7ebf7fe700 1 --2- 172.21.15.3:0/1201107564 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ec012b7d0 0x7f7ec012ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.619+0000 7f7ec4d8f700 1 --2- 172.21.15.3:0/1201107564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ec01304c0 0x7f7ec01328b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f7eb8009e60 tx=0x7f7eb8009780).ready entity=mon.0 client_cookie=1da9706a47815e5d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.619+0000 7f7ebeffd700 1 -- 172.21.15.3:0/1201107564 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb800e070 con 0x7f7ec01304c0 2023-10-03T10:09:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.619+0000 7f7ebeffd700 1 -- 172.21.15.3:0/1201107564 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb800bb60 con 0x7f7ec01304c0 2023-10-03T10:09:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.619+0000 7f7ebeffd700 1 -- 172.21.15.3:0/1201107564 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8012410 con 0x7f7ec01304c0 2023-10-03T10:09:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.619+0000 7f7ec67f2700 1 -- 172.21.15.3:0/1201107564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ec01304c0 msgr2=0x7f7ec01328b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:59.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.619+0000 7f7ec67f2700 1 --2- 172.21.15.3:0/1201107564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ec01304c0 0x7f7ec01328b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f7eb8009e60 tx=0x7f7eb8009780).stop 2023-10-03T10:09:59.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.620+0000 7f7ec67f2700 1 -- 172.21.15.3:0/1201107564 shutdown_connections 2023-10-03T10:09:59.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.620+0000 7f7ec67f2700 1 --2- 172.21.15.3:0/1201107564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ec01304c0 0x7f7ec01328b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:59.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.620+0000 7f7ec67f2700 1 --2- 172.21.15.3:0/1201107564 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ec012b7d0 0x7f7ec012ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:59.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.620+0000 7f7ec67f2700 1 -- 172.21.15.3:0/1201107564 >> 172.21.15.3:0/1201107564 conn(0x7f7ec0063560 msgr2=0x7f7ec009a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:59.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.620+0000 7f7ec67f2700 1 -- 172.21.15.3:0/1201107564 shutdown_connections 2023-10-03T10:09:59.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.620+0000 7f7ec67f2700 1 -- 172.21.15.3:0/1201107564 wait complete. 2023-10-03T10:09:59.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.620+0000 7f7ec67f2700 1 Processor -- start 2023-10-03T10:09:59.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.621+0000 7f7ec67f2700 1 -- start start 2023-10-03T10:09:59.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.621+0000 7f7ec67f2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ec0059070 0x7f7ec01186d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:59.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.621+0000 7f7ec67f2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ec012b7d0 0x7f7ec0114780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:59.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.621+0000 7f7ec67f2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ec01304c0 0x7f7ec0114c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.621+0000 7f7ec67f2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec012a850 con 0x7f7ec01304c0 2023-10-03T10:09:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.621+0000 7f7ec67f2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec0116d60 con 0x7f7ec012b7d0 2023-10-03T10:09:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.621+0000 7f7ec67f2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec0098d70 con 0x7f7ec0059070 2023-10-03T10:09:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.621+0000 7f7ec4d8f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ec01304c0 0x7f7ec0114c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.621+0000 7f7ebffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ec0059070 0x7f7ec01186d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.621+0000 7f7ebffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ec0059070 0x7f7ec01186d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57626/0 (socket says 172.21.15.3:57626) 2023-10-03T10:09:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.621+0000 7f7ebffff700 1 -- 172.21.15.3:0/186108661 learned_addr learned my addr 172.21.15.3:0/186108661 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.621+0000 7f7ec4d8f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ec01304c0 0x7f7ec0114c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47904/0 (socket says 172.21.15.3:47904) 2023-10-03T10:09:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.621+0000 7f7ebf7fe700 1 --2- 172.21.15.3:0/186108661 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ec012b7d0 0x7f7ec0114780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.622+0000 7f7ebffff700 1 -- 172.21.15.3:0/186108661 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ec012b7d0 msgr2=0x7f7ec0114780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.622+0000 7f7ebffff700 1 --2- 172.21.15.3:0/186108661 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ec012b7d0 0x7f7ec0114780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.622+0000 7f7ebffff700 1 -- 172.21.15.3:0/186108661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ec01304c0 msgr2=0x7f7ec0114c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.622+0000 7f7ebffff700 1 --2- 172.21.15.3:0/186108661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ec01304c0 0x7f7ec0114c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.622+0000 7f7ebffff700 1 -- 172.21.15.3:0/186108661 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb00092a0 con 0x7f7ec0059070 2023-10-03T10:09:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.622+0000 7f7ec4d8f700 1 --2- 172.21.15.3:0/186108661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ec01304c0 0x7f7ec0114c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.622+0000 7f7ebffff700 1 --2- 172.21.15.3:0/186108661 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ec0059070 0x7f7ec01186d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f7ea800c4c0 tx=0x7f7ea8007c10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.622+0000 7f7eb7fff700 1 -- 172.21.15.3:0/186108661 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea800ce30 con 0x7f7ec0059070 2023-10-03T10:09:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.622+0000 7f7eb7fff700 1 -- 172.21.15.3:0/186108661 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea80072e0 con 0x7f7ec0059070 2023-10-03T10:09:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.622+0000 7f7ec67f2700 1 -- 172.21.15.3:0/186108661 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb8009a20 con 0x7f7ec0059070 2023-10-03T10:09:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.622+0000 7f7eb7fff700 1 -- 172.21.15.3:0/186108661 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea80057a0 con 0x7f7ec0059070 2023-10-03T10:09:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.622+0000 7f7ec67f2700 1 -- 172.21.15.3:0/186108661 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec0115b80 con 0x7f7ec0059070 2023-10-03T10:09:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.623+0000 7f7eb7fff700 1 -- 172.21.15.3:0/186108661 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f7ea8005bd0 con 0x7f7ec0059070 2023-10-03T10:09:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.623+0000 7f7eb7fff700 1 -- 172.21.15.3:0/186108661 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f7ea803a5c0 con 0x7f7ec0059070 2023-10-03T10:09:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.623+0000 7f7ec67f2700 1 -- 172.21.15.3:0/186108661 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec01180e0 con 0x7f7ec0059070 2023-10-03T10:09:59.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.628+0000 7f7eb7fff700 1 -- 172.21.15.3:0/186108661 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f7ea801d3f0 con 0x7f7ec0059070 2023-10-03T10:09:59.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.805+0000 7f7ec67f2700 1 -- 172.21.15.3:0/186108661 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7ec0115580 con 0x7f7ec0059070 2023-10-03T10:09:59.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.806+0000 7f7eb7fff700 1 -- 172.21.15.3:0/186108661 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7ea801c350 con 0x7f7ec0059070 2023-10-03T10:09:59.806 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-03T10:09:59.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.808+0000 7f7ec67f2700 1 -- 172.21.15.3:0/186108661 >> v1:172.21.15.3:6816/118296 conn(0x7f7ea002bb60 legacy=0x7f7ea002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:59.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.808+0000 7f7ec67f2700 1 -- 172.21.15.3:0/186108661 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ec0059070 msgr2=0x7f7ec01186d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:59.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.808+0000 7f7ec67f2700 1 --2- 172.21.15.3:0/186108661 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ec0059070 0x7f7ec01186d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f7ea800c4c0 tx=0x7f7ea8007c10).stop 2023-10-03T10:09:59.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.808+0000 7f7ec67f2700 1 -- 172.21.15.3:0/186108661 shutdown_connections 2023-10-03T10:09:59.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.808+0000 7f7ec67f2700 1 --2- 172.21.15.3:0/186108661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ec01304c0 0x7f7ec0114c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:59.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.808+0000 7f7ec67f2700 1 --2- 172.21.15.3:0/186108661 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ec0059070 0x7f7ec01186d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:59.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.808+0000 7f7ec67f2700 1 --2- 172.21.15.3:0/186108661 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ec012b7d0 0x7f7ec0114780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:59.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.808+0000 7f7ec67f2700 1 -- 172.21.15.3:0/186108661 >> 172.21.15.3:0/186108661 conn(0x7f7ec0063560 msgr2=0x7f7ec0066d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:59.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.808+0000 7f7ec67f2700 1 -- 172.21.15.3:0/186108661 shutdown_connections 2023-10-03T10:09:59.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.808+0000 7f7ec67f2700 1 -- 172.21.15.3:0/186108661 wait complete. 2023-10-03T10:09:59.821 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-03T10:09:59.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.954+0000 7f255f6f7700 1 Processor -- start 2023-10-03T10:09:59.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.954+0000 7f255f6f7700 1 -- start start 2023-10-03T10:09:59.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.954+0000 7f255f6f7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2558062a90 0x7f2558098800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:59.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.954+0000 7f255f6f7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2558098e20 0x7f2558099200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:59.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.954+0000 7f255f6f7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25580928d0 con 0x7f255812a940 2023-10-03T10:09:59.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.954+0000 7f255f6f7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2558062e70 con 0x7f2558098e20 2023-10-03T10:09:59.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.954+0000 7f255f6f7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2558091fc0 con 0x7f2558062a90 2023-10-03T10:09:59.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.955+0000 7f255d493700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2558062a90 0x7f2558098800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:59.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.955+0000 7f255cc92700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2558098e20 0x7f2558099200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:59.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.955+0000 7f255dc94700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f255812a940 0x7f255812adf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38596/0 (socket says 172.21.15.3:38596) 2023-10-03T10:09:59.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.955+0000 7f255dc94700 1 -- 172.21.15.3:0/1202210269 learned_addr learned my addr 172.21.15.3:0/1202210269 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:09:59.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.955+0000 7f255d493700 1 -- 172.21.15.3:0/1202210269 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2558098e20 msgr2=0x7f2558099200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:09:59.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.955+0000 7f254ffff700 1 -- 172.21.15.3:0/1202210269 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3948906768 0 0) 0x7f25580928d0 con 0x7f255812a940 2023-10-03T10:09:59.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.955+0000 7f255d493700 1 --2- 172.21.15.3:0/1202210269 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2558098e20 0x7f2558099200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:59.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.955+0000 7f255d493700 1 -- 172.21.15.3:0/1202210269 >> v1:172.21.15.3:6789/0 conn(0x7f255812a940 legacy=0x7f255812adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:59.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.955+0000 7f255d493700 1 -- 172.21.15.3:0/1202210269 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2558091a40 con 0x7f2558062a90 2023-10-03T10:09:59.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.955+0000 7f255cc92700 1 --2- 172.21.15.3:0/1202210269 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2558098e20 0x7f2558099200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:59.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.955+0000 7f255d493700 1 --2- 172.21.15.3:0/1202210269 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2558062a90 0x7f2558098800 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f25540097c0 tx=0x7f2554009060).ready entity=mon.2 client_cookie=520f158ff0664665 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:59.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.955+0000 7f254ffff700 1 -- 172.21.15.3:0/1202210269 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255400f510 con 0x7f2558062a90 2023-10-03T10:09:59.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.955+0000 7f254ffff700 1 -- 172.21.15.3:0/1202210269 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255400f670 con 0x7f2558062a90 2023-10-03T10:09:59.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.955+0000 7f254ffff700 1 -- 172.21.15.3:0/1202210269 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255400f9f0 con 0x7f2558062a90 2023-10-03T10:09:59.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.956+0000 7f255f6f7700 1 -- 172.21.15.3:0/1202210269 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2558062a90 msgr2=0x7f2558098800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:59.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.956+0000 7f255f6f7700 1 --2- 172.21.15.3:0/1202210269 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2558062a90 0x7f2558098800 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f25540097c0 tx=0x7f2554009060).stop 2023-10-03T10:09:59.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.956+0000 7f255f6f7700 1 -- 172.21.15.3:0/1202210269 shutdown_connections 2023-10-03T10:09:59.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.956+0000 7f255f6f7700 1 --2- 172.21.15.3:0/1202210269 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2558062a90 0x7f2558098800 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:59.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.956+0000 7f255f6f7700 1 --2- 172.21.15.3:0/1202210269 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2558098e20 0x7f2558099200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:59.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.956+0000 7f255f6f7700 1 -- 172.21.15.3:0/1202210269 >> 172.21.15.3:0/1202210269 conn(0x7f25580599a0 msgr2=0x7f2558097460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:09:59.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.956+0000 7f255f6f7700 1 -- 172.21.15.3:0/1202210269 shutdown_connections 2023-10-03T10:09:59.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.956+0000 7f255f6f7700 1 -- 172.21.15.3:0/1202210269 wait complete. 2023-10-03T10:09:59.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.957+0000 7f255f6f7700 1 Processor -- start 2023-10-03T10:09:59.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.957+0000 7f255f6f7700 1 -- start start 2023-10-03T10:09:59.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.957+0000 7f255f6f7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2558062a90 0x7f2558116f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:59.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.958+0000 7f255f6f7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2558098e20 0x7f2558117480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:59.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.958+0000 7f255f6f7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f255812a940 0x7f2558117970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:09:59.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.958+0000 7f255f6f7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25580974b0 con 0x7f2558062a90 2023-10-03T10:09:59.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.958+0000 7f255f6f7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2558099df0 con 0x7f255812a940 2023-10-03T10:09:59.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.958+0000 7f255f6f7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f255811aa30 con 0x7f2558098e20 2023-10-03T10:09:59.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.958+0000 7f255d493700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2558062a90 0x7f2558116f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:59.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.958+0000 7f255cc92700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2558098e20 0x7f2558117480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:59.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.958+0000 7f255d493700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2558062a90 0x7f2558116f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47920/0 (socket says 172.21.15.3:47920) 2023-10-03T10:09:59.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.958+0000 7f255d493700 1 -- 172.21.15.3:0/2225849310 learned_addr learned my addr 172.21.15.3:0/2225849310 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:09:59.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.958+0000 7f255dc94700 1 --2- 172.21.15.3:0/2225849310 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f255812a940 0x7f2558117970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:09:59.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.958+0000 7f255d493700 1 -- 172.21.15.3:0/2225849310 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2558098e20 msgr2=0x7f2558117480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:59.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.958+0000 7f255d493700 1 --2- 172.21.15.3:0/2225849310 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2558098e20 0x7f2558117480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:59.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.958+0000 7f255d493700 1 -- 172.21.15.3:0/2225849310 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f255812a940 msgr2=0x7f2558117970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:09:59.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.958+0000 7f255d493700 1 --2- 172.21.15.3:0/2225849310 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f255812a940 0x7f2558117970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:09:59.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.958+0000 7f255d493700 1 -- 172.21.15.3:0/2225849310 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25480092a0 con 0x7f2558062a90 2023-10-03T10:09:59.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.958+0000 7f255cc92700 1 --2- 172.21.15.3:0/2225849310 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2558098e20 0x7f2558117480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:09:59.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.958+0000 7f255d493700 1 --2- 172.21.15.3:0/2225849310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2558062a90 0x7f2558116f90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f2554009e60 tx=0x7f2554017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:09:59.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.958+0000 7f254d7fa700 1 -- 172.21.15.3:0/2225849310 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255400f7e0 con 0x7f2558062a90 2023-10-03T10:09:59.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.958+0000 7f254d7fa700 1 -- 172.21.15.3:0/2225849310 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255400f940 con 0x7f2558062a90 2023-10-03T10:09:59.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.958+0000 7f255f6f7700 1 -- 172.21.15.3:0/2225849310 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25540092a0 con 0x7f2558062a90 2023-10-03T10:09:59.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.958+0000 7f254d7fa700 1 -- 172.21.15.3:0/2225849310 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255401b640 con 0x7f2558062a90 2023-10-03T10:09:59.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.958+0000 7f255f6f7700 1 -- 172.21.15.3:0/2225849310 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2558128340 con 0x7f2558062a90 2023-10-03T10:09:59.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.959+0000 7f254d7fa700 1 -- 172.21.15.3:0/2225849310 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f255401ba30 con 0x7f2558062a90 2023-10-03T10:09:59.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.959+0000 7f254d7fa700 1 -- 172.21.15.3:0/2225849310 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f255403c740 con 0x7f2558062a90 2023-10-03T10:09:59.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.960+0000 7f255f6f7700 1 -- 172.21.15.3:0/2225849310 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2558128340 con 0x7f2558062a90 2023-10-03T10:09:59.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:09:59.964+0000 7f254d7fa700 1 -- 172.21.15.3:0/2225849310 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f2558128340 con 0x7f2558062a90 2023-10-03T10:10:00.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.137+0000 7f255f6f7700 1 -- 172.21.15.3:0/2225849310 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2558128340 con 0x7f2558062a90 2023-10-03T10:10:00.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.137+0000 7f254d7fa700 1 -- 172.21.15.3:0/2225849310 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v17) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2558128340 con 0x7f2558062a90 2023-10-03T10:10:00.138 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-10-03T10:10:00.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.139+0000 7f255f6f7700 1 -- 172.21.15.3:0/2225849310 >> v1:172.21.15.3:6816/118296 conn(0x7f253c02bb60 legacy=0x7f253c02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:00.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.139+0000 7f255f6f7700 1 -- 172.21.15.3:0/2225849310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2558062a90 msgr2=0x7f2558116f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:00.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.139+0000 7f255f6f7700 1 --2- 172.21.15.3:0/2225849310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2558062a90 0x7f2558116f90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f2554009e60 tx=0x7f2554017000).stop 2023-10-03T10:10:00.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.139+0000 7f255f6f7700 1 -- 172.21.15.3:0/2225849310 shutdown_connections 2023-10-03T10:10:00.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.139+0000 7f255f6f7700 1 --2- 172.21.15.3:0/2225849310 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f255812a940 0x7f2558117970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:00.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.139+0000 7f255f6f7700 1 --2- 172.21.15.3:0/2225849310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2558062a90 0x7f2558116f90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:00.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.139+0000 7f255f6f7700 1 --2- 172.21.15.3:0/2225849310 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2558098e20 0x7f2558117480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:00.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.139+0000 7f255f6f7700 1 -- 172.21.15.3:0/2225849310 >> 172.21.15.3:0/2225849310 conn(0x7f25580599a0 msgr2=0x7f2558096970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:00.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.139+0000 7f255f6f7700 1 -- 172.21.15.3:0/2225849310 shutdown_connections 2023-10-03T10:10:00.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.139+0000 7f255f6f7700 1 -- 172.21.15.3:0/2225849310 wait complete. 2023-10-03T10:10:00.153 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-03T10:10:00.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.291+0000 7f06166c4700 1 Processor -- start 2023-10-03T10:10:00.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.291+0000 7f06166c4700 1 -- start start 2023-10-03T10:10:00.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.291+0000 7f06166c4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0610068ef0 0x7f06100692d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:00.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.291+0000 7f06166c4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06101290a0 0x7f0610130e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:00.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.291+0000 7f06166c4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0610094590 con 0x7f06100697c0 2023-10-03T10:10:00.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.291+0000 7f06166c4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0610096210 con 0x7f06101290a0 2023-10-03T10:10:00.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.291+0000 7f06166c4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0610096410 con 0x7f0610068ef0 2023-10-03T10:10:00.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.292+0000 7f060f7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f06100697c0 0x7f0610128990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38612/0 (socket says 172.21.15.3:38612) 2023-10-03T10:10:00.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.292+0000 7f060f7fe700 1 -- 172.21.15.3:0/3714487307 learned_addr learned my addr 172.21.15.3:0/3714487307 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:10:00.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.292+0000 7f060ffff700 1 --2- 172.21.15.3:0/3714487307 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0610068ef0 0x7f06100692d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:00.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.292+0000 7f0614c61700 1 --2- 172.21.15.3:0/3714487307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06101290a0 0x7f0610130e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:00.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.292+0000 7f060effd700 1 -- 172.21.15.3:0/3714487307 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1640231900 0 0) 0x7f0610094590 con 0x7f06100697c0 2023-10-03T10:10:00.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.292+0000 7f060effd700 1 -- 172.21.15.3:0/3714487307 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05f80033f0 con 0x7f06100697c0 2023-10-03T10:10:00.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.292+0000 7f0614c61700 1 -- 172.21.15.3:0/3714487307 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0610068ef0 msgr2=0x7f06100692d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:00.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.292+0000 7f0614c61700 1 --2- 172.21.15.3:0/3714487307 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0610068ef0 0x7f06100692d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:00.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.292+0000 7f0614c61700 1 -- 172.21.15.3:0/3714487307 >> v1:172.21.15.3:6789/0 conn(0x7f06100697c0 legacy=0x7f0610128990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:00.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.292+0000 7f0614c61700 1 -- 172.21.15.3:0/3714487307 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0610093ed0 con 0x7f06101290a0 2023-10-03T10:10:00.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.292+0000 7f060ffff700 1 --2- 172.21.15.3:0/3714487307 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0610068ef0 0x7f06100692d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:00.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.292+0000 7f0614c61700 1 --2- 172.21.15.3:0/3714487307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06101290a0 0x7f0610130e00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f0608009e90 tx=0x7f0608009780).ready entity=mon.1 client_cookie=9c9d56c748e5985c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:00.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.292+0000 7f060effd700 1 -- 172.21.15.3:0/3714487307 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060800e070 con 0x7f06101290a0 2023-10-03T10:10:00.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.292+0000 7f060effd700 1 -- 172.21.15.3:0/3714487307 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060800bab0 con 0x7f06101290a0 2023-10-03T10:10:00.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.292+0000 7f060effd700 1 -- 172.21.15.3:0/3714487307 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060800be30 con 0x7f06101290a0 2023-10-03T10:10:00.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.293+0000 7f06166c4700 1 -- 172.21.15.3:0/3714487307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06101290a0 msgr2=0x7f0610130e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:00.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.293+0000 7f06166c4700 1 --2- 172.21.15.3:0/3714487307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06101290a0 0x7f0610130e00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f0608009e90 tx=0x7f0608009780).stop 2023-10-03T10:10:00.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.293+0000 7f06166c4700 1 -- 172.21.15.3:0/3714487307 shutdown_connections 2023-10-03T10:10:00.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.293+0000 7f06166c4700 1 --2- 172.21.15.3:0/3714487307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06101290a0 0x7f0610130e00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:00.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.293+0000 7f06166c4700 1 --2- 172.21.15.3:0/3714487307 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0610068ef0 0x7f06100692d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:00.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.293+0000 7f06166c4700 1 -- 172.21.15.3:0/3714487307 >> 172.21.15.3:0/3714487307 conn(0x7f0610063560 msgr2=0x7f061009a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:00.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.293+0000 7f06166c4700 1 -- 172.21.15.3:0/3714487307 shutdown_connections 2023-10-03T10:10:00.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.293+0000 7f06166c4700 1 -- 172.21.15.3:0/3714487307 wait complete. 2023-10-03T10:10:00.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.294+0000 7f06166c4700 1 Processor -- start 2023-10-03T10:10:00.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.294+0000 7f06166c4700 1 -- start start 2023-10-03T10:10:00.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.295+0000 7f06166c4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0610068ef0 0x7f0610091a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:00.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.295+0000 7f06166c4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06100697c0 0x7f0610091f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:00.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.295+0000 7f06166c4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06101290a0 0x7f0610092450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:00.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.295+0000 7f06166c4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061011b860 con 0x7f06100697c0 2023-10-03T10:10:00.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.295+0000 7f06166c4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0610094c70 con 0x7f06101290a0 2023-10-03T10:10:00.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.295+0000 7f06166c4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f061011c730 con 0x7f0610068ef0 2023-10-03T10:10:00.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.295+0000 7f060ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0610068ef0 0x7f0610091a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:00.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.295+0000 7f0614c61700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06101290a0 0x7f0610092450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:00.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.295+0000 7f060f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06100697c0 0x7f0610091f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:00.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.295+0000 7f0614c61700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06101290a0 0x7f0610092450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39304/0 (socket says 172.21.15.3:39304) 2023-10-03T10:10:00.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.295+0000 7f060f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06100697c0 0x7f0610091f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47926/0 (socket says 172.21.15.3:47926) 2023-10-03T10:10:00.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.295+0000 7f060ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0610068ef0 0x7f0610091a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57670/0 (socket says 172.21.15.3:57670) 2023-10-03T10:10:00.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.295+0000 7f0614c61700 1 -- 172.21.15.3:0/2899647801 learned_addr learned my addr 172.21.15.3:0/2899647801 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:00.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.295+0000 7f0614c61700 1 -- 172.21.15.3:0/2899647801 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0610068ef0 msgr2=0x7f0610091a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:00.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.295+0000 7f0614c61700 1 --2- 172.21.15.3:0/2899647801 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0610068ef0 0x7f0610091a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:00.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.295+0000 7f0614c61700 1 -- 172.21.15.3:0/2899647801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06100697c0 msgr2=0x7f0610091f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:00.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.295+0000 7f0614c61700 1 --2- 172.21.15.3:0/2899647801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06100697c0 0x7f0610091f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:00.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.295+0000 7f0614c61700 1 -- 172.21.15.3:0/2899647801 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06000092a0 con 0x7f06101290a0 2023-10-03T10:10:00.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.295+0000 7f060f7fe700 1 --2- 172.21.15.3:0/2899647801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06100697c0 0x7f0610091f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:00.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.295+0000 7f060ffff700 1 --2- 172.21.15.3:0/2899647801 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0610068ef0 0x7f0610091a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:00.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.295+0000 7f0614c61700 1 --2- 172.21.15.3:0/2899647801 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06101290a0 0x7f0610092450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f0608013cc0 tx=0x7f0610094720).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:00.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.295+0000 7f05f7fff700 1 -- 172.21.15.3:0/2899647801 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0608013ea0 con 0x7f06101290a0 2023-10-03T10:10:00.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.296+0000 7f05f7fff700 1 -- 172.21.15.3:0/2899647801 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060801d630 con 0x7f06101290a0 2023-10-03T10:10:00.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.296+0000 7f05f7fff700 1 -- 172.21.15.3:0/2899647801 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060801d9f0 con 0x7f06101290a0 2023-10-03T10:10:00.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.296+0000 7f06166c4700 1 -- 172.21.15.3:0/2899647801 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0608009a20 con 0x7f06101290a0 2023-10-03T10:10:00.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.296+0000 7f06166c4700 1 -- 172.21.15.3:0/2899647801 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06101281e0 con 0x7f06101290a0 2023-10-03T10:10:00.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.296+0000 7f05f7fff700 1 -- 172.21.15.3:0/2899647801 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f0608003040 con 0x7f06101290a0 2023-10-03T10:10:00.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.297+0000 7f05f7fff700 1 -- 172.21.15.3:0/2899647801 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f060803cc50 con 0x7f06101290a0 2023-10-03T10:10:00.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.297+0000 7f06166c4700 1 -- 172.21.15.3:0/2899647801 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f061011c870 con 0x7f06101290a0 2023-10-03T10:10:00.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.301+0000 7f05f7fff700 1 -- 172.21.15.3:0/2899647801 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f060801dda0 con 0x7f06101290a0 2023-10-03T10:10:00.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.475+0000 7f06166c4700 1 -- 172.21.15.3:0/2899647801 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f061005a260 con 0x7f06101290a0 2023-10-03T10:10:00.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.475+0000 7f05f7fff700 1 -- 172.21.15.3:0/2899647801 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v17) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0608025360 con 0x7f06101290a0 2023-10-03T10:10:00.475 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-10-03T10:10:00.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.477+0000 7f06166c4700 1 -- 172.21.15.3:0/2899647801 >> v1:172.21.15.3:6816/118296 conn(0x7f05fc02bb30 legacy=0x7f05fc02dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:00.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.477+0000 7f06166c4700 1 -- 172.21.15.3:0/2899647801 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06101290a0 msgr2=0x7f0610092450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:00.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.477+0000 7f06166c4700 1 --2- 172.21.15.3:0/2899647801 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06101290a0 0x7f0610092450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f0608013cc0 tx=0x7f0610094720).stop 2023-10-03T10:10:00.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.477+0000 7f06166c4700 1 -- 172.21.15.3:0/2899647801 shutdown_connections 2023-10-03T10:10:00.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.477+0000 7f06166c4700 1 --2- 172.21.15.3:0/2899647801 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06101290a0 0x7f0610092450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:00.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.477+0000 7f06166c4700 1 --2- 172.21.15.3:0/2899647801 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0610068ef0 0x7f0610091a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:00.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.477+0000 7f06166c4700 1 --2- 172.21.15.3:0/2899647801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06100697c0 0x7f0610091f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:00.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.477+0000 7f06166c4700 1 -- 172.21.15.3:0/2899647801 >> 172.21.15.3:0/2899647801 conn(0x7f0610063560 msgr2=0x7f0610110810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:00.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.477+0000 7f06166c4700 1 -- 172.21.15.3:0/2899647801 shutdown_connections 2023-10-03T10:10:00.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.477+0000 7f06166c4700 1 -- 172.21.15.3:0/2899647801 wait complete. 2023-10-03T10:10:00.491 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-03T10:10:00.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.622+0000 7f603e6d3700 1 Processor -- start 2023-10-03T10:10:00.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.622+0000 7f603e6d3700 1 -- start start 2023-10-03T10:10:00.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.623+0000 7f603e6d3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6038093810 0x7f6038095300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:00.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.623+0000 7f603e6d3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f603812d120 0x7f603812f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:00.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.623+0000 7f603e6d3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6038093010 con 0x7f60380957f0 2023-10-03T10:10:00.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.623+0000 7f603e6d3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6038093bf0 con 0x7f6038093810 2023-10-03T10:10:00.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.623+0000 7f603e6d3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6038092aa0 con 0x7f603812d120 2023-10-03T10:10:00.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.623+0000 7f603cc70700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f603812d120 0x7f603812f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:00.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.623+0000 7f6037fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6038093810 0x7f6038095300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:00.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.623+0000 7f60377fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f60380957f0 0x7f6038095c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38614/0 (socket says 172.21.15.3:38614) 2023-10-03T10:10:00.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.623+0000 7f60377fe700 1 -- 172.21.15.3:0/2762293311 learned_addr learned my addr 172.21.15.3:0/2762293311 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:10:00.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.623+0000 7f6037fff700 1 -- 172.21.15.3:0/2762293311 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f603812d120 msgr2=0x7f603812f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:00.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.623+0000 7f6037fff700 1 --2- 172.21.15.3:0/2762293311 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f603812d120 0x7f603812f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:00.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.623+0000 7f6037fff700 1 -- 172.21.15.3:0/2762293311 >> v1:172.21.15.3:6789/0 conn(0x7f60380957f0 legacy=0x7f6038095c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:00.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.623+0000 7f6037fff700 1 -- 172.21.15.3:0/2762293311 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6038096e90 con 0x7f6038093810 2023-10-03T10:10:00.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.623+0000 7f6036ffd700 1 -- 172.21.15.3:0/2762293311 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 767992948 0 0) 0x7f6038093010 con 0x7f60380957f0 2023-10-03T10:10:00.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.623+0000 7f603cc70700 1 --2- 172.21.15.3:0/2762293311 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f603812d120 0x7f603812f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:00.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.623+0000 7f6037fff700 1 --2- 172.21.15.3:0/2762293311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6038093810 0x7f6038095300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f6028009760 tx=0x7f6028009060).ready entity=mon.0 client_cookie=cfe9188960f2da00 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:00.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.624+0000 7f6036ffd700 1 -- 172.21.15.3:0/2762293311 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602800f510 con 0x7f6038093810 2023-10-03T10:10:00.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.624+0000 7f6036ffd700 1 -- 172.21.15.3:0/2762293311 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602800f670 con 0x7f6038093810 2023-10-03T10:10:00.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.624+0000 7f6036ffd700 1 -- 172.21.15.3:0/2762293311 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602800f9f0 con 0x7f6038093810 2023-10-03T10:10:00.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.624+0000 7f603e6d3700 1 -- 172.21.15.3:0/2762293311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6038093810 msgr2=0x7f6038095300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:00.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.624+0000 7f603e6d3700 1 --2- 172.21.15.3:0/2762293311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6038093810 0x7f6038095300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f6028009760 tx=0x7f6028009060).stop 2023-10-03T10:10:00.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.624+0000 7f603e6d3700 1 -- 172.21.15.3:0/2762293311 shutdown_connections 2023-10-03T10:10:00.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.624+0000 7f603e6d3700 1 --2- 172.21.15.3:0/2762293311 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f603812d120 0x7f603812f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:00.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.624+0000 7f603e6d3700 1 --2- 172.21.15.3:0/2762293311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6038093810 0x7f6038095300 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:00.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.624+0000 7f603e6d3700 1 -- 172.21.15.3:0/2762293311 >> 172.21.15.3:0/2762293311 conn(0x7f6038059bb0 msgr2=0x7f6038097400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:00.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.624+0000 7f603e6d3700 1 -- 172.21.15.3:0/2762293311 shutdown_connections 2023-10-03T10:10:00.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.624+0000 7f603e6d3700 1 -- 172.21.15.3:0/2762293311 wait complete. 2023-10-03T10:10:00.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.625+0000 7f603e6d3700 1 Processor -- start 2023-10-03T10:10:00.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.626+0000 7f603e6d3700 1 -- start start 2023-10-03T10:10:00.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.626+0000 7f603e6d3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6038093810 0x7f6038099d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:00.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.626+0000 7f603e6d3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f60380957f0 0x7f603809a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:00.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.626+0000 7f603e6d3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f603812d120 0x7f603809a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:00.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.626+0000 7f603e6d3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603811d6c0 con 0x7f603812d120 2023-10-03T10:10:00.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.626+0000 7f603e6d3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f603806a6a0 con 0x7f60380957f0 2023-10-03T10:10:00.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.626+0000 7f603e6d3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f60380670c0 con 0x7f6038093810 2023-10-03T10:10:00.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.626+0000 7f603cc70700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f603812d120 0x7f603809a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:00.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.626+0000 7f60377fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f60380957f0 0x7f603809a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:00.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.626+0000 7f6037fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6038093810 0x7f6038099d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:00.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.626+0000 7f603cc70700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f603812d120 0x7f603809a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47932/0 (socket says 172.21.15.3:47932) 2023-10-03T10:10:00.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.626+0000 7f603cc70700 1 -- 172.21.15.3:0/3649190835 learned_addr learned my addr 172.21.15.3:0/3649190835 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:00.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.626+0000 7f60377fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f60380957f0 0x7f603809a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39322/0 (socket says 172.21.15.3:39322) 2023-10-03T10:10:00.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.626+0000 7f6037fff700 1 --2- 172.21.15.3:0/3649190835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6038093810 0x7f6038099d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57678/0 (socket says 172.21.15.3:57678) 2023-10-03T10:10:00.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.626+0000 7f603cc70700 1 -- 172.21.15.3:0/3649190835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6038093810 msgr2=0x7f6038099d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:00.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.626+0000 7f603cc70700 1 --2- 172.21.15.3:0/3649190835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6038093810 0x7f6038099d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:00.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.626+0000 7f603cc70700 1 -- 172.21.15.3:0/3649190835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f60380957f0 msgr2=0x7f603809a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:00.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.627+0000 7f603cc70700 1 --2- 172.21.15.3:0/3649190835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f60380957f0 0x7f603809a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:00.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.627+0000 7f603cc70700 1 -- 172.21.15.3:0/3649190835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6030009a20 con 0x7f603812d120 2023-10-03T10:10:00.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.627+0000 7f6037fff700 1 --2- 172.21.15.3:0/3649190835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6038093810 0x7f6038099d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:00.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.627+0000 7f60377fe700 1 --2- 172.21.15.3:0/3649190835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f60380957f0 0x7f603809a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:00.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.627+0000 7f603cc70700 1 --2- 172.21.15.3:0/3649190835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f603812d120 0x7f603809a6f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f6030009fc0 tx=0x7f603000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:00.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.627+0000 7f601ffff700 1 -- 172.21.15.3:0/3649190835 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6030009d20 con 0x7f603812d120 2023-10-03T10:10:00.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.627+0000 7f601ffff700 1 -- 172.21.15.3:0/3649190835 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603000f040 con 0x7f603812d120 2023-10-03T10:10:00.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.627+0000 7f603e6d3700 1 -- 172.21.15.3:0/3649190835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60280092a0 con 0x7f603812d120 2023-10-03T10:10:00.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.627+0000 7f601ffff700 1 -- 172.21.15.3:0/3649190835 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603000cde0 con 0x7f603812d120 2023-10-03T10:10:00.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.627+0000 7f603e6d3700 1 -- 172.21.15.3:0/3649190835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603811e5f0 con 0x7f603812d120 2023-10-03T10:10:00.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.628+0000 7f601ffff700 1 -- 172.21.15.3:0/3649190835 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f6030013610 con 0x7f603812d120 2023-10-03T10:10:00.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.628+0000 7f601ffff700 1 -- 172.21.15.3:0/3649190835 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f603003b810 con 0x7f603812d120 2023-10-03T10:10:00.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.628+0000 7f603e6d3700 1 -- 172.21.15.3:0/3649190835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6038066c10 con 0x7f603812d120 2023-10-03T10:10:00.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.633+0000 7f601ffff700 1 -- 172.21.15.3:0/3649190835 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f603001d460 con 0x7f603812d120 2023-10-03T10:10:00.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.852+0000 7f603e6d3700 1 -- 172.21.15.3:0/3649190835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f603805a260 con 0x7f603812d120 2023-10-03T10:10:00.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.853+0000 7f601ffff700 1 -- 172.21.15.3:0/3649190835 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2154 (secure 0 0 0) 0x7f60300077b0 con 0x7f603812d120 2023-10-03T10:10:00.853 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:10:00.853 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-03T10:09:44.141304+0000","modified":"2023-10-03T10:09:47.154287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4186},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4186":{"gid":4186,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6817/2582564779","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6817","nonce":2582564779}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-03T10:10:00.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.855+0000 7f603e6d3700 1 -- 172.21.15.3:0/3649190835 >> v1:172.21.15.3:6816/118296 conn(0x7f602402bb60 legacy=0x7f602402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:00.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.855+0000 7f603e6d3700 1 -- 172.21.15.3:0/3649190835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f603812d120 msgr2=0x7f603809a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:00.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.855+0000 7f603e6d3700 1 --2- 172.21.15.3:0/3649190835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f603812d120 0x7f603809a6f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f6030009fc0 tx=0x7f603000b040).stop 2023-10-03T10:10:00.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.855+0000 7f603e6d3700 1 -- 172.21.15.3:0/3649190835 shutdown_connections 2023-10-03T10:10:00.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.855+0000 7f603e6d3700 1 --2- 172.21.15.3:0/3649190835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f603812d120 0x7f603809a6f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:00.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.855+0000 7f603e6d3700 1 --2- 172.21.15.3:0/3649190835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6038093810 0x7f6038099d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:00.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.855+0000 7f603e6d3700 1 --2- 172.21.15.3:0/3649190835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f60380957f0 0x7f603809a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:00.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.855+0000 7f603e6d3700 1 -- 172.21.15.3:0/3649190835 >> 172.21.15.3:0/3649190835 conn(0x7f6038059bb0 msgr2=0x7f6038062a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:00.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.855+0000 7f603e6d3700 1 -- 172.21.15.3:0/3649190835 shutdown_connections 2023-10-03T10:10:00.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.855+0000 7f603e6d3700 1 -- 172.21.15.3:0/3649190835 wait complete. 2023-10-03T10:10:00.856 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 6 2023-10-03T10:10:00.868 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-03T10:10:00.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.998+0000 7f0cee5a3700 1 Processor -- start 2023-10-03T10:10:00.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.998+0000 7f0cee5a3700 1 -- start start 2023-10-03T10:10:01.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.999+0000 7f0cee5a3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ce8093810 0x7f0ce8095300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:01.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.999+0000 7f0cee5a3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0ce80957f0 0x7f0ce8095bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:01.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.999+0000 7f0cee5a3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce8093010 con 0x7f0ce812c110 2023-10-03T10:10:01.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.999+0000 7f0cee5a3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce8093bf0 con 0x7f0ce80957f0 2023-10-03T10:10:01.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.999+0000 7f0cee5a3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce8092aa0 con 0x7f0ce8093810 2023-10-03T10:10:01.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.999+0000 7f0ce7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ce8093810 0x7f0ce8095300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:01.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.999+0000 7f0ce77fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0ce80957f0 0x7f0ce8095bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:01.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.999+0000 7f0ce7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ce8093810 0x7f0ce8095300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57682/0 (socket says 172.21.15.3:57682) 2023-10-03T10:10:01.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.999+0000 7f0cecb40700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0ce812c110 0x7f0ce812e500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38620/0 (socket says 172.21.15.3:38620) 2023-10-03T10:10:01.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.999+0000 7f0ce7fff700 1 -- 172.21.15.3:0/1772754983 learned_addr learned my addr 172.21.15.3:0/1772754983 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:01.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.999+0000 7f0ce7fff700 1 -- 172.21.15.3:0/1772754983 >> v2:172.21.15.3:3300/0 conn(0x7f0ce80957f0 msgr2=0x7f0ce8095bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:01.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.999+0000 7f0ce7fff700 1 --2- 172.21.15.3:0/1772754983 >> v2:172.21.15.3:3300/0 conn(0x7f0ce80957f0 0x7f0ce8095bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:01.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.999+0000 7f0ce7fff700 1 -- 172.21.15.3:0/1772754983 >> v1:172.21.15.3:6789/0 conn(0x7f0ce812c110 legacy=0x7f0ce812e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:01.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.999+0000 7f0ce7fff700 1 -- 172.21.15.3:0/1772754983 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce8096e90 con 0x7f0ce8093810 2023-10-03T10:10:01.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:00.999+0000 7f0ce77fe700 1 --2- 172.21.15.3:0/1772754983 >> v2:172.21.15.3:3300/0 conn(0x7f0ce80957f0 0x7f0ce8095bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:01.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.000+0000 7f0ce7fff700 1 --2- 172.21.15.3:0/1772754983 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ce8093810 0x7f0ce8095300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f0cd0009830 tx=0x7f0cd00090f0).ready entity=mon.2 client_cookie=9eb45130a5adf980 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:01.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.000+0000 7f0ce6ffd700 1 -- 172.21.15.3:0/1772754983 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd000b580 con 0x7f0ce8093810 2023-10-03T10:10:01.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.000+0000 7f0ce6ffd700 1 -- 172.21.15.3:0/1772754983 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd000b6e0 con 0x7f0ce8093810 2023-10-03T10:10:01.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.000+0000 7f0ce6ffd700 1 -- 172.21.15.3:0/1772754983 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd000ba60 con 0x7f0ce8093810 2023-10-03T10:10:01.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.000+0000 7f0cee5a3700 1 -- 172.21.15.3:0/1772754983 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ce8093810 msgr2=0x7f0ce8095300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:01.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.000+0000 7f0cee5a3700 1 --2- 172.21.15.3:0/1772754983 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ce8093810 0x7f0ce8095300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f0cd0009830 tx=0x7f0cd00090f0).stop 2023-10-03T10:10:01.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.000+0000 7f0cee5a3700 1 -- 172.21.15.3:0/1772754983 shutdown_connections 2023-10-03T10:10:01.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.000+0000 7f0cee5a3700 1 --2- 172.21.15.3:0/1772754983 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ce8093810 0x7f0ce8095300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:01.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.000+0000 7f0cee5a3700 1 --2- 172.21.15.3:0/1772754983 >> v2:172.21.15.3:3300/0 conn(0x7f0ce80957f0 0x7f0ce8095bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:01.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.000+0000 7f0cee5a3700 1 -- 172.21.15.3:0/1772754983 >> 172.21.15.3:0/1772754983 conn(0x7f0ce8059bb0 msgr2=0x7f0ce8097400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:01.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.000+0000 7f0cee5a3700 1 -- 172.21.15.3:0/1772754983 shutdown_connections 2023-10-03T10:10:01.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.000+0000 7f0cee5a3700 1 -- 172.21.15.3:0/1772754983 wait complete. 2023-10-03T10:10:01.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.002+0000 7f0cee5a3700 1 Processor -- start 2023-10-03T10:10:01.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.002+0000 7f0cee5a3700 1 -- start start 2023-10-03T10:10:01.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.003+0000 7f0cee5a3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ce8093810 0x7f0ce8111bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:01.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.003+0000 7f0cee5a3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0ce80957f0 0x7f0ce81120e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:01.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.003+0000 7f0cee5a3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ce812c110 0x7f0ce81125d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:01.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.003+0000 7f0cee5a3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce81119c0 con 0x7f0ce812c110 2023-10-03T10:10:01.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.003+0000 7f0cee5a3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce8063600 con 0x7f0ce80957f0 2023-10-03T10:10:01.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.003+0000 7f0cee5a3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce806a6a0 con 0x7f0ce8093810 2023-10-03T10:10:01.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.003+0000 7f0cecb40700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ce812c110 0x7f0ce81125d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:01.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.003+0000 7f0cecb40700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ce812c110 0x7f0ce81125d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47940/0 (socket says 172.21.15.3:47940) 2023-10-03T10:10:01.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.003+0000 7f0ce7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ce8093810 0x7f0ce8111bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:01.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.003+0000 7f0cecb40700 1 -- 172.21.15.3:0/1954335644 learned_addr learned my addr 172.21.15.3:0/1954335644 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:01.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.003+0000 7f0ce77fe700 1 --2- 172.21.15.3:0/1954335644 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0ce80957f0 0x7f0ce81120e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:01.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.003+0000 7f0cecb40700 1 -- 172.21.15.3:0/1954335644 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ce8093810 msgr2=0x7f0ce8111bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:01.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.003+0000 7f0cecb40700 1 --2- 172.21.15.3:0/1954335644 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ce8093810 0x7f0ce8111bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:01.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.003+0000 7f0cecb40700 1 -- 172.21.15.3:0/1954335644 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0ce80957f0 msgr2=0x7f0ce81120e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:01.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.003+0000 7f0cecb40700 1 --2- 172.21.15.3:0/1954335644 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0ce80957f0 0x7f0ce81120e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:01.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.003+0000 7f0cecb40700 1 -- 172.21.15.3:0/1954335644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd80092a0 con 0x7f0ce812c110 2023-10-03T10:10:01.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.003+0000 7f0ce7fff700 1 --2- 172.21.15.3:0/1954335644 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ce8093810 0x7f0ce8111bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:01.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.003+0000 7f0cecb40700 1 --2- 172.21.15.3:0/1954335644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ce812c110 0x7f0ce81125d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f0ce000c7c0 tx=0x7f0ce0001560).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:01.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.003+0000 7f0cdffff700 1 -- 172.21.15.3:0/1954335644 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0005c70 con 0x7f0ce812c110 2023-10-03T10:10:01.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.004+0000 7f0cdffff700 1 -- 172.21.15.3:0/1954335644 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce0005dd0 con 0x7f0ce812c110 2023-10-03T10:10:01.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.004+0000 7f0cee5a3700 1 -- 172.21.15.3:0/1954335644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd0009370 con 0x7f0ce812c110 2023-10-03T10:10:01.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.004+0000 7f0cdffff700 1 -- 172.21.15.3:0/1954335644 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce00072e0 con 0x7f0ce812c110 2023-10-03T10:10:01.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.004+0000 7f0cee5a3700 1 -- 172.21.15.3:0/1954335644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce81157d0 con 0x7f0ce812c110 2023-10-03T10:10:01.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.004+0000 7f0cdffff700 1 -- 172.21.15.3:0/1954335644 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f0ce00076d0 con 0x7f0ce812c110 2023-10-03T10:10:01.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.004+0000 7f0cdffff700 1 -- 172.21.15.3:0/1954335644 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f0ce003a810 con 0x7f0ce812c110 2023-10-03T10:10:01.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.005+0000 7f0cee5a3700 1 -- 172.21.15.3:0/1954335644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce805a260 con 0x7f0ce812c110 2023-10-03T10:10:01.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.009+0000 7f0cdffff700 1 -- 172.21.15.3:0/1954335644 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f0ce001b710 con 0x7f0ce812c110 2023-10-03T10:10:01.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.186+0000 7f0cee5a3700 1 -- 172.21.15.3:0/1954335644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ce8091b60 con 0x7f0ce812c110 2023-10-03T10:10:01.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.186+0000 7f0cdffff700 1 -- 172.21.15.3:0/1954335644 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+11309 (secure 0 0 0) 0x7f0ce001b300 con 0x7f0ce812c110 2023-10-03T10:10:01.187 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:10:01.187 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":17,"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","created":"2023-10-03T10:09:28.703441+0000","modified":"2023-10-03T10:09:45.145435+0000","last_up_change":"2023-10-03T10:09:36.569401+0000","last_in_change":"2023-10-03T10:09:30.156546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-10-03T10:09:36.623971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-03T10:09:38.210614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-03T10:09:42.695873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-03T10:09:43.115588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cdcdd930-3c5b-4486-a021-7132f90fb6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6808","nonce":118845}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6809","nonce":118845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6811","nonce":118845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6810","nonce":118845}]},"public_addr":"172.21.15.3:6808/118845","cluster_addr":"172.21.15.3:6809/118845","heartbeat_back_addr":"172.21.15.3:6811/118845","heartbeat_front_addr":"172.21.15.3:6810/118845","state":["exists","up"]},{"osd":1,"uuid":"c8e58877-6762-4741-a8c1-567ebc97b708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6804","nonce":118835}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6805","nonce":118835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6807","nonce":118835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6806","nonce":118835}]},"public_addr":"172.21.15.3:6804/118835","cluster_addr":"172.21.15.3:6805/118835","heartbeat_back_addr":"172.21.15.3:6807/118835","heartbeat_front_addr":"172.21.15.3:6806/118835","state":["exists","up"]},{"osd":2,"uuid":"68a2a21a-7a8e-4f82-8433-5201faa50777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6812","nonce":118864}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6813","nonce":118864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6815","nonce":118864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6814","nonce":118864}]},"public_addr":"172.21.15.3:6812/118864","cluster_addr":"172.21.15.3:6813/118864","heartbeat_back_addr":"172.21.15.3:6815/118864","heartbeat_front_addr":"172.21.15.3:6814/118864","state":["exists","up"]},{"osd":3,"uuid":"c4df2903-4e1e-4208-bf77-95b9fa749241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6800","nonce":118836}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6801","nonce":118836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6803","nonce":118836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6802","nonce":118836}]},"public_addr":"172.21.15.3:6800/118836","cluster_addr":"172.21.15.3:6801/118836","heartbeat_back_addr":"172.21.15.3:6803/118836","heartbeat_front_addr":"172.21.15.3:6802/118836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.369708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.257825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.337931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.085222+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-03T10:10:01.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.189+0000 7f0cee5a3700 1 -- 172.21.15.3:0/1954335644 >> v1:172.21.15.3:6816/118296 conn(0x7f0cc802bb60 legacy=0x7f0cc802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:01.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.189+0000 7f0cee5a3700 1 -- 172.21.15.3:0/1954335644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ce812c110 msgr2=0x7f0ce81125d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:01.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.189+0000 7f0cee5a3700 1 --2- 172.21.15.3:0/1954335644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ce812c110 0x7f0ce81125d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f0ce000c7c0 tx=0x7f0ce0001560).stop 2023-10-03T10:10:01.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.189+0000 7f0cee5a3700 1 -- 172.21.15.3:0/1954335644 shutdown_connections 2023-10-03T10:10:01.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.189+0000 7f0cee5a3700 1 --2- 172.21.15.3:0/1954335644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ce812c110 0x7f0ce81125d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:01.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.189+0000 7f0cee5a3700 1 --2- 172.21.15.3:0/1954335644 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ce8093810 0x7f0ce8111bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:01.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.189+0000 7f0cee5a3700 1 --2- 172.21.15.3:0/1954335644 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0ce80957f0 0x7f0ce81120e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:01.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.189+0000 7f0cee5a3700 1 -- 172.21.15.3:0/1954335644 >> 172.21.15.3:0/1954335644 conn(0x7f0ce8059bb0 msgr2=0x7f0ce8068c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:01.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.189+0000 7f0cee5a3700 1 -- 172.21.15.3:0/1954335644 shutdown_connections 2023-10-03T10:10:01.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.189+0000 7f0cee5a3700 1 -- 172.21.15.3:0/1954335644 wait complete. 2023-10-03T10:10:01.203 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-10-03T10:09:36.623971+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-03T10:09:38.210614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-03T10:09:42.695873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-03T10:09:43.115588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-03T10:10:01.203 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-10-03T10:10:01.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.336+0000 7f237844a700 1 Processor -- start 2023-10-03T10:10:01.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.336+0000 7f237844a700 1 -- start start 2023-10-03T10:10:01.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.337+0000 7f237844a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2370062a90 0x7f2370098800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:01.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.337+0000 7f237844a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2370098e20 0x7f2370099200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:01.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.337+0000 7f237844a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23700928d0 con 0x7f237012a940 2023-10-03T10:10:01.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.337+0000 7f237844a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2370062e70 con 0x7f2370098e20 2023-10-03T10:10:01.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.337+0000 7f237844a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2370091fc0 con 0x7f2370062a90 2023-10-03T10:10:01.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.337+0000 7f23761e6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2370062a90 0x7f2370098800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:01.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.337+0000 7f23759e5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2370098e20 0x7f2370099200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:01.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.337+0000 7f23761e6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2370062a90 0x7f2370098800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57698/0 (socket says 172.21.15.3:57698) 2023-10-03T10:10:01.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.337+0000 7f23761e6700 1 -- 172.21.15.3:0/4009187378 learned_addr learned my addr 172.21.15.3:0/4009187378 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:01.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.337+0000 7f23759e5700 1 -- 172.21.15.3:0/4009187378 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2370062a90 msgr2=0x7f2370098800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:01.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.337+0000 7f23759e5700 1 --2- 172.21.15.3:0/4009187378 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2370062a90 0x7f2370098800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:01.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.337+0000 7f23759e5700 1 -- 172.21.15.3:0/4009187378 >> v1:172.21.15.3:6789/0 conn(0x7f237012a940 legacy=0x7f237012ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:01.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.337+0000 7f23759e5700 1 -- 172.21.15.3:0/4009187378 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2370091a40 con 0x7f2370098e20 2023-10-03T10:10:01.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.337+0000 7f23751e4700 1 -- 172.21.15.3:0/4009187378 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2394815092 0 0) 0x7f23700928d0 con 0x7f237012a940 2023-10-03T10:10:01.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.337+0000 7f23761e6700 1 --2- 172.21.15.3:0/4009187378 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2370062a90 0x7f2370098800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:01.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.338+0000 7f23759e5700 1 --2- 172.21.15.3:0/4009187378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2370098e20 0x7f2370099200 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f2364011f80 tx=0x7f2364011900).ready entity=mon.0 client_cookie=3c7871259cb7e850 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:01.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.338+0000 7f23751e4700 1 -- 172.21.15.3:0/4009187378 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364016070 con 0x7f2370098e20 2023-10-03T10:10:01.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.338+0000 7f23751e4700 1 -- 172.21.15.3:0/4009187378 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2364013c90 con 0x7f2370098e20 2023-10-03T10:10:01.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.338+0000 7f23751e4700 1 -- 172.21.15.3:0/4009187378 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23640112e0 con 0x7f2370098e20 2023-10-03T10:10:01.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.338+0000 7f237844a700 1 -- 172.21.15.3:0/4009187378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2370098e20 msgr2=0x7f2370099200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:01.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.338+0000 7f237844a700 1 --2- 172.21.15.3:0/4009187378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2370098e20 0x7f2370099200 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f2364011f80 tx=0x7f2364011900).stop 2023-10-03T10:10:01.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.338+0000 7f237844a700 1 -- 172.21.15.3:0/4009187378 shutdown_connections 2023-10-03T10:10:01.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.338+0000 7f237844a700 1 --2- 172.21.15.3:0/4009187378 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2370062a90 0x7f2370098800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:01.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.338+0000 7f237844a700 1 --2- 172.21.15.3:0/4009187378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2370098e20 0x7f2370099200 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:01.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.338+0000 7f237844a700 1 -- 172.21.15.3:0/4009187378 >> 172.21.15.3:0/4009187378 conn(0x7f23700599a0 msgr2=0x7f2370097460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:01.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.338+0000 7f237844a700 1 -- 172.21.15.3:0/4009187378 shutdown_connections 2023-10-03T10:10:01.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.338+0000 7f237844a700 1 -- 172.21.15.3:0/4009187378 wait complete. 2023-10-03T10:10:01.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.339+0000 7f237844a700 1 Processor -- start 2023-10-03T10:10:01.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.339+0000 7f237844a700 1 -- start start 2023-10-03T10:10:01.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.340+0000 7f237844a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2370062a90 0x7f2370127940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:01.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.340+0000 7f237844a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2370098e20 0x7f23701127a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:01.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.340+0000 7f237844a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f237012a940 0x7f2370112c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:01.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.340+0000 7f237844a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23700977d0 con 0x7f237012a940 2023-10-03T10:10:01.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.340+0000 7f237844a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2370115d30 con 0x7f2370098e20 2023-10-03T10:10:01.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.340+0000 7f237844a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2370099d70 con 0x7f2370062a90 2023-10-03T10:10:01.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.340+0000 7f23769e7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f237012a940 0x7f2370112c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:01.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.340+0000 7f23761e6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2370062a90 0x7f2370127940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:01.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.340+0000 7f23759e5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2370098e20 0x7f23701127a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:01.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.340+0000 7f23761e6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2370062a90 0x7f2370127940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57712/0 (socket says 172.21.15.3:57712) 2023-10-03T10:10:01.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.340+0000 7f23769e7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f237012a940 0x7f2370112c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47954/0 (socket says 172.21.15.3:47954) 2023-10-03T10:10:01.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.340+0000 7f23761e6700 1 -- 172.21.15.3:0/916302506 learned_addr learned my addr 172.21.15.3:0/916302506 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:01.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.340+0000 7f23759e5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2370098e20 0x7f23701127a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39340/0 (socket says 172.21.15.3:39340) 2023-10-03T10:10:01.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.340+0000 7f23759e5700 1 -- 172.21.15.3:0/916302506 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2370062a90 msgr2=0x7f2370127940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:01.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.340+0000 7f23759e5700 1 --2- 172.21.15.3:0/916302506 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2370062a90 0x7f2370127940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:01.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.340+0000 7f23759e5700 1 -- 172.21.15.3:0/916302506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f237012a940 msgr2=0x7f2370112c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:01.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.340+0000 7f23759e5700 1 --2- 172.21.15.3:0/916302506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f237012a940 0x7f2370112c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:01.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.340+0000 7f23759e5700 1 -- 172.21.15.3:0/916302506 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c009370 con 0x7f2370098e20 2023-10-03T10:10:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.340+0000 7f23761e6700 1 --2- 172.21.15.3:0/916302506 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2370062a90 0x7f2370127940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.340+0000 7f23769e7700 1 --2- 172.21.15.3:0/916302506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f237012a940 0x7f2370112c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.341+0000 7f23759e5700 1 --2- 172.21.15.3:0/916302506 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2370098e20 0x7f23701127a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f2370098110 tx=0x7f2370091bd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.341+0000 7f23627fc700 1 -- 172.21.15.3:0/916302506 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364006cf0 con 0x7f2370098e20 2023-10-03T10:10:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.341+0000 7f23627fc700 1 -- 172.21.15.3:0/916302506 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23640110d0 con 0x7f2370098e20 2023-10-03T10:10:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.341+0000 7f23627fc700 1 -- 172.21.15.3:0/916302506 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236401b670 con 0x7f2370098e20 2023-10-03T10:10:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.341+0000 7f237844a700 1 -- 172.21.15.3:0/916302506 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2364011b40 con 0x7f2370098e20 2023-10-03T10:10:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.341+0000 7f237844a700 1 -- 172.21.15.3:0/916302506 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2370116b70 con 0x7f2370098e20 2023-10-03T10:10:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.341+0000 7f23627fc700 1 -- 172.21.15.3:0/916302506 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f236401b9e0 con 0x7f2370098e20 2023-10-03T10:10:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.342+0000 7f23627fc700 1 -- 172.21.15.3:0/916302506 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f23640447e0 con 0x7f2370098e20 2023-10-03T10:10:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.342+0000 7f237844a700 1 -- 172.21.15.3:0/916302506 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2370116b70 con 0x7f2370098e20 2023-10-03T10:10:01.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.347+0000 7f23627fc700 1 -- 172.21.15.3:0/916302506 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f2370116b70 con 0x7f2370098e20 2023-10-03T10:10:01.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.521+0000 7f237844a700 1 -- 172.21.15.3:0/916302506 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f2370116b70 con 0x7f2370098e20 2023-10-03T10:10:01.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.521+0000 7f23627fc700 1 -- 172.21.15.3:0/916302506 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v17) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2370116b70 con 0x7f2370098e20 2023-10-03T10:10:01.522 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-03T10:10:01.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.523+0000 7f237844a700 1 -- 172.21.15.3:0/916302506 >> v1:172.21.15.3:6816/118296 conn(0x7f235402bb60 legacy=0x7f235402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:01.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.523+0000 7f237844a700 1 -- 172.21.15.3:0/916302506 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2370098e20 msgr2=0x7f23701127a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:01.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.523+0000 7f237844a700 1 --2- 172.21.15.3:0/916302506 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2370098e20 0x7f23701127a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f2370098110 tx=0x7f2370091bd0).stop 2023-10-03T10:10:01.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.523+0000 7f237844a700 1 -- 172.21.15.3:0/916302506 shutdown_connections 2023-10-03T10:10:01.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.523+0000 7f237844a700 1 --2- 172.21.15.3:0/916302506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f237012a940 0x7f2370112c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:01.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.523+0000 7f237844a700 1 --2- 172.21.15.3:0/916302506 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2370062a90 0x7f2370127940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:01.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.523+0000 7f237844a700 1 --2- 172.21.15.3:0/916302506 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2370098e20 0x7f23701127a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:01.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.523+0000 7f237844a700 1 -- 172.21.15.3:0/916302506 >> 172.21.15.3:0/916302506 conn(0x7f23700599a0 msgr2=0x7f23700968e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:01.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.523+0000 7f237844a700 1 -- 172.21.15.3:0/916302506 shutdown_connections 2023-10-03T10:10:01.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.523+0000 7f237844a700 1 -- 172.21.15.3:0/916302506 wait complete. 2023-10-03T10:10:01.537 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-03T10:10:01.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.672+0000 7feebfe5b700 1 Processor -- start 2023-10-03T10:10:01.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.673+0000 7feebfe5b700 1 -- start start 2023-10-03T10:10:01.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.673+0000 7feebfe5b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feeb8065340 0x7feeb80986e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:01.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.673+0000 7feebfe5b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feeb8097c90 0x7feeb8098bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:01.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.673+0000 7feebfe5b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feeb80990c0 0x7feeb81314a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:01.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.673+0000 7feebfe5b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb8065720 con 0x7feeb8097c90 2023-10-03T10:10:01.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.673+0000 7feebfe5b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb8099d50 con 0x7feeb80990c0 2023-10-03T10:10:01.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.673+0000 7feebfe5b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feeb8095880 con 0x7feeb8065340 2023-10-03T10:10:01.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.673+0000 7feebd3f6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feeb8097c90 0x7feeb8098bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:01.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.673+0000 7feebe3f8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feeb80990c0 0x7feeb81314a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:01.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.673+0000 7feebdbf7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feeb8065340 0x7feeb80986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:01.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.673+0000 7feebd3f6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feeb8097c90 0x7feeb8098bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47956/0 (socket says 172.21.15.3:47956) 2023-10-03T10:10:01.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.673+0000 7feebdbf7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feeb8065340 0x7feeb80986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57726/0 (socket says 172.21.15.3:57726) 2023-10-03T10:10:01.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.673+0000 7feebe3f8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feeb80990c0 0x7feeb81314a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39352/0 (socket says 172.21.15.3:39352) 2023-10-03T10:10:01.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.673+0000 7feebd3f6700 1 -- 172.21.15.3:0/1028375848 learned_addr learned my addr 172.21.15.3:0/1028375848 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:01.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.674+0000 7feebd3f6700 1 -- 172.21.15.3:0/1028375848 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feeb8065340 msgr2=0x7feeb80986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:01.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.674+0000 7feebd3f6700 1 --2- 172.21.15.3:0/1028375848 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feeb8065340 0x7feeb80986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:01.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.674+0000 7feebd3f6700 1 -- 172.21.15.3:0/1028375848 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feeb80990c0 msgr2=0x7feeb81314a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:01.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.674+0000 7feebd3f6700 1 --2- 172.21.15.3:0/1028375848 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feeb80990c0 0x7feeb81314a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:01.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.674+0000 7feebd3f6700 1 -- 172.21.15.3:0/1028375848 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb8065fc0 con 0x7feeb8097c90 2023-10-03T10:10:01.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.674+0000 7feebe3f8700 1 --2- 172.21.15.3:0/1028375848 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feeb80990c0 0x7feeb81314a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:01.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.674+0000 7feebd3f6700 1 --2- 172.21.15.3:0/1028375848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feeb8097c90 0x7feeb8098bd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7feea8006fd0 tx=0x7feea80090f0).ready entity=mon.0 client_cookie=5b367788bceb0cc3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:01.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.674+0000 7feebcbf5700 1 -- 172.21.15.3:0/1028375848 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea800f510 con 0x7feeb8097c90 2023-10-03T10:10:01.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.674+0000 7feebcbf5700 1 -- 172.21.15.3:0/1028375848 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea800f670 con 0x7feeb8097c90 2023-10-03T10:10:01.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.674+0000 7feebcbf5700 1 -- 172.21.15.3:0/1028375848 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea800fa10 con 0x7feeb8097c90 2023-10-03T10:10:01.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.675+0000 7feebfe5b700 1 -- 172.21.15.3:0/1028375848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feeb8097c90 msgr2=0x7feeb8098bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:01.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.675+0000 7feebfe5b700 1 --2- 172.21.15.3:0/1028375848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feeb8097c90 0x7feeb8098bd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7feea8006fd0 tx=0x7feea80090f0).stop 2023-10-03T10:10:01.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.675+0000 7feebfe5b700 1 -- 172.21.15.3:0/1028375848 shutdown_connections 2023-10-03T10:10:01.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.675+0000 7feebfe5b700 1 --2- 172.21.15.3:0/1028375848 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feeb80990c0 0x7feeb81314a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:01.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.675+0000 7feebfe5b700 1 --2- 172.21.15.3:0/1028375848 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feeb8065340 0x7feeb80986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:01.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.675+0000 7feebfe5b700 1 --2- 172.21.15.3:0/1028375848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feeb8097c90 0x7feeb8098bd0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:01.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.675+0000 7feebfe5b700 1 -- 172.21.15.3:0/1028375848 >> 172.21.15.3:0/1028375848 conn(0x7feeb809dcc0 msgr2=0x7feeb809e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:01.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.675+0000 7feebfe5b700 1 -- 172.21.15.3:0/1028375848 shutdown_connections 2023-10-03T10:10:01.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.675+0000 7feebfe5b700 1 -- 172.21.15.3:0/1028375848 wait complete. 2023-10-03T10:10:01.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.676+0000 7feebfe5b700 1 Processor -- start 2023-10-03T10:10:01.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.676+0000 7feebfe5b700 1 -- start start 2023-10-03T10:10:01.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.676+0000 7feebfe5b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feeb8065340 0x7feeb813a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:01.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.676+0000 7feebfe5b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feeb8097c90 0x7feeb813a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:01.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.676+0000 7feebfe5b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feeb80990c0 0x7feeb813ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:01.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.676+0000 7feebfe5b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb811e000 con 0x7feeb8097c90 2023-10-03T10:10:01.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.676+0000 7feebfe5b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb8099a50 con 0x7feeb8065340 2023-10-03T10:10:01.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.676+0000 7feebfe5b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feeb809d110 con 0x7feeb80990c0 2023-10-03T10:10:01.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.676+0000 7feebe3f8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feeb80990c0 0x7feeb813ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:01.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.676+0000 7feebd3f6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feeb8097c90 0x7feeb813a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:01.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.676+0000 7feebe3f8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feeb80990c0 0x7feeb813ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57732/0 (socket says 172.21.15.3:57732) 2023-10-03T10:10:01.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.676+0000 7feebe3f8700 1 -- 172.21.15.3:0/2928638178 learned_addr learned my addr 172.21.15.3:0/2928638178 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:01.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.676+0000 7feebdbf7700 1 --2- 172.21.15.3:0/2928638178 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feeb8065340 0x7feeb813a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:01.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.677+0000 7feebd3f6700 1 -- 172.21.15.3:0/2928638178 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feeb80990c0 msgr2=0x7feeb813ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:01.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.677+0000 7feebd3f6700 1 --2- 172.21.15.3:0/2928638178 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feeb80990c0 0x7feeb813ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:01.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.677+0000 7feebd3f6700 1 -- 172.21.15.3:0/2928638178 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feeb8065340 msgr2=0x7feeb813a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:01.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.677+0000 7feebd3f6700 1 --2- 172.21.15.3:0/2928638178 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feeb8065340 0x7feeb813a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:01.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.677+0000 7feebd3f6700 1 -- 172.21.15.3:0/2928638178 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb0009a20 con 0x7feeb8097c90 2023-10-03T10:10:01.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.677+0000 7feebe3f8700 1 --2- 172.21.15.3:0/2928638178 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feeb80990c0 0x7feeb813ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:01.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.677+0000 7feebdbf7700 1 --2- 172.21.15.3:0/2928638178 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feeb8065340 0x7feeb813a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:01.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.677+0000 7feebd3f6700 1 --2- 172.21.15.3:0/2928638178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feeb8097c90 0x7feeb813a780 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7feea8014000 tx=0x7feea8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:01.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.677+0000 7feeadffb700 1 -- 172.21.15.3:0/2928638178 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea8006d90 con 0x7feeb8097c90 2023-10-03T10:10:01.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.677+0000 7feeadffb700 1 -- 172.21.15.3:0/2928638178 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea8006910 con 0x7feeb8097c90 2023-10-03T10:10:01.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.677+0000 7feebfe5b700 1 -- 172.21.15.3:0/2928638178 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea8009370 con 0x7feeb8097c90 2023-10-03T10:10:01.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.677+0000 7feeadffb700 1 -- 172.21.15.3:0/2928638178 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea8000bc0 con 0x7feeb8097c90 2023-10-03T10:10:01.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.677+0000 7feebfe5b700 1 -- 172.21.15.3:0/2928638178 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb811e810 con 0x7feeb8097c90 2023-10-03T10:10:01.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.678+0000 7feeadffb700 1 -- 172.21.15.3:0/2928638178 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7feea8022020 con 0x7feeb8097c90 2023-10-03T10:10:01.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.678+0000 7feeadffb700 1 -- 172.21.15.3:0/2928638178 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7feea803c540 con 0x7feeb8097c90 2023-10-03T10:10:01.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.678+0000 7feebfe5b700 1 -- 172.21.15.3:0/2928638178 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb811e810 con 0x7feeb8097c90 2023-10-03T10:10:01.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.683+0000 7feeadffb700 1 -- 172.21.15.3:0/2928638178 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7feeb811e810 con 0x7feeb8097c90 2023-10-03T10:10:01.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.857+0000 7feebfe5b700 1 -- 172.21.15.3:0/2928638178 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7feeb811e810 con 0x7feeb8097c90 2023-10-03T10:10:01.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.857+0000 7feeadffb700 1 -- 172.21.15.3:0/2928638178 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7feeb811e810 con 0x7feeb8097c90 2023-10-03T10:10:01.858 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-03T10:10:01.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.859+0000 7feebfe5b700 1 -- 172.21.15.3:0/2928638178 >> v1:172.21.15.3:6816/118296 conn(0x7fee9c02bb60 legacy=0x7fee9c02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:01.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.859+0000 7feebfe5b700 1 -- 172.21.15.3:0/2928638178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feeb8097c90 msgr2=0x7feeb813a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:01.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.859+0000 7feebfe5b700 1 --2- 172.21.15.3:0/2928638178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feeb8097c90 0x7feeb813a780 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7feea8014000 tx=0x7feea8017000).stop 2023-10-03T10:10:01.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.859+0000 7feebfe5b700 1 -- 172.21.15.3:0/2928638178 shutdown_connections 2023-10-03T10:10:01.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.859+0000 7feebfe5b700 1 --2- 172.21.15.3:0/2928638178 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feeb80990c0 0x7feeb813ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:01.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.859+0000 7feebfe5b700 1 --2- 172.21.15.3:0/2928638178 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feeb8065340 0x7feeb813a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:01.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.859+0000 7feebfe5b700 1 --2- 172.21.15.3:0/2928638178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feeb8097c90 0x7feeb813a780 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:01.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.859+0000 7feebfe5b700 1 -- 172.21.15.3:0/2928638178 >> 172.21.15.3:0/2928638178 conn(0x7feeb809dcc0 msgr2=0x7feeb8114300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:01.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.859+0000 7feebfe5b700 1 -- 172.21.15.3:0/2928638178 shutdown_connections 2023-10-03T10:10:01.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:01.859+0000 7feebfe5b700 1 -- 172.21.15.3:0/2928638178 wait complete. 2023-10-03T10:10:01.873 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-03T10:10:02.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.006+0000 7f1e79dd4700 1 Processor -- start 2023-10-03T10:10:02.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.006+0000 7f1e79dd4700 1 -- start start 2023-10-03T10:10:02.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.006+0000 7f1e79dd4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e74066070 0x7f1e7412b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:02.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.006+0000 7f1e79dd4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1e7412abf0 0x7f1e74127a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:02.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.006+0000 7f1e79dd4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e740655b0 con 0x7f1e7412a810 2023-10-03T10:10:02.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.006+0000 7f1e79dd4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e74066450 con 0x7f1e7412abf0 2023-10-03T10:10:02.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.006+0000 7f1e79dd4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e74064f70 con 0x7f1e74066070 2023-10-03T10:10:02.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.007+0000 7f1e737fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e74066070 0x7f1e7412b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:02.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.007+0000 7f1e73fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1e7412abf0 0x7f1e74127a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:02.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.007+0000 7f1e737fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e74066070 0x7f1e7412b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39364/0 (socket says 172.21.15.3:39364) 2023-10-03T10:10:02.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.007+0000 7f1e72ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1e7412a810 0x7f1e74127370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38650/0 (socket says 172.21.15.3:38650) 2023-10-03T10:10:02.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.007+0000 7f1e73fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1e7412abf0 0x7f1e74127a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47966/0 (socket says 172.21.15.3:47966) 2023-10-03T10:10:02.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.007+0000 7f1e737fe700 1 -- 172.21.15.3:0/2725823808 learned_addr learned my addr 172.21.15.3:0/2725823808 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:02.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.007+0000 7f1e73fff700 1 -- 172.21.15.3:0/2725823808 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e74066070 msgr2=0x7f1e7412b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:02.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.007+0000 7f1e73fff700 1 --2- 172.21.15.3:0/2725823808 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e74066070 0x7f1e7412b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:02.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.007+0000 7f1e73fff700 1 -- 172.21.15.3:0/2725823808 >> v1:172.21.15.3:6789/0 conn(0x7f1e7412a810 legacy=0x7f1e74127370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:02.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.007+0000 7f1e73fff700 1 -- 172.21.15.3:0/2725823808 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e74064b30 con 0x7f1e7412abf0 2023-10-03T10:10:02.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.007+0000 7f1e737fe700 1 --2- 172.21.15.3:0/2725823808 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e74066070 0x7f1e7412b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:02.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.007+0000 7f1e727fc700 1 -- 172.21.15.3:0/2725823808 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3213200310 0 0) 0x7f1e740655b0 con 0x7f1e7412a810 2023-10-03T10:10:02.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.007+0000 7f1e73fff700 1 --2- 172.21.15.3:0/2725823808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e7412abf0 0x7f1e74127a80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f1e6c009e60 tx=0x7f1e6c009780).ready entity=mon.0 client_cookie=1a90299a72f6730f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:02.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.007+0000 7f1e727fc700 1 -- 172.21.15.3:0/2725823808 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c00e070 con 0x7f1e7412abf0 2023-10-03T10:10:02.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.007+0000 7f1e727fc700 1 -- 172.21.15.3:0/2725823808 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6c00bb60 con 0x7f1e7412abf0 2023-10-03T10:10:02.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.007+0000 7f1e727fc700 1 -- 172.21.15.3:0/2725823808 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c012410 con 0x7f1e7412abf0 2023-10-03T10:10:02.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.008+0000 7f1e79dd4700 1 -- 172.21.15.3:0/2725823808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e7412abf0 msgr2=0x7f1e74127a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:02.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.008+0000 7f1e79dd4700 1 --2- 172.21.15.3:0/2725823808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e7412abf0 0x7f1e74127a80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f1e6c009e60 tx=0x7f1e6c009780).stop 2023-10-03T10:10:02.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.008+0000 7f1e79dd4700 1 -- 172.21.15.3:0/2725823808 shutdown_connections 2023-10-03T10:10:02.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.008+0000 7f1e79dd4700 1 --2- 172.21.15.3:0/2725823808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e7412abf0 0x7f1e74127a80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:02.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.008+0000 7f1e79dd4700 1 --2- 172.21.15.3:0/2725823808 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e74066070 0x7f1e7412b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:02.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.008+0000 7f1e79dd4700 1 -- 172.21.15.3:0/2725823808 >> 172.21.15.3:0/2725823808 conn(0x7f1e74059bb0 msgr2=0x7f1e7409ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:02.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.008+0000 7f1e79dd4700 1 -- 172.21.15.3:0/2725823808 shutdown_connections 2023-10-03T10:10:02.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.008+0000 7f1e79dd4700 1 -- 172.21.15.3:0/2725823808 wait complete. 2023-10-03T10:10:02.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.009+0000 7f1e79dd4700 1 Processor -- start 2023-10-03T10:10:02.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.009+0000 7f1e79dd4700 1 -- start start 2023-10-03T10:10:02.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.009+0000 7f1e79dd4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e74066070 0x7f1e74124b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:02.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.009+0000 7f1e79dd4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e7412a810 0x7f1e74113a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:02.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.009+0000 7f1e79dd4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e7412abf0 0x7f1e74113ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:02.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.009+0000 7f1e79dd4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e74064d60 con 0x7f1e7412a810 2023-10-03T10:10:02.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.009+0000 7f1e79dd4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e74117530 con 0x7f1e74066070 2023-10-03T10:10:02.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.009+0000 7f1e79dd4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e74116f90 con 0x7f1e7412abf0 2023-10-03T10:10:02.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.010+0000 7f1e737fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e74066070 0x7f1e74124b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:02.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.010+0000 7f1e737fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e74066070 0x7f1e74124b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39366/0 (socket says 172.21.15.3:39366) 2023-10-03T10:10:02.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.010+0000 7f1e737fe700 1 -- 172.21.15.3:0/3009687897 learned_addr learned my addr 172.21.15.3:0/3009687897 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:02.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.010+0000 7f1e72ffd700 1 --2- 172.21.15.3:0/3009687897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e7412a810 0x7f1e74113a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:02.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.010+0000 7f1e73fff700 1 --2- 172.21.15.3:0/3009687897 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e7412abf0 0x7f1e74113ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:02.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.010+0000 7f1e73fff700 1 -- 172.21.15.3:0/3009687897 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e74066070 msgr2=0x7f1e74124b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:02.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.010+0000 7f1e73fff700 1 --2- 172.21.15.3:0/3009687897 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e74066070 0x7f1e74124b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:02.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.010+0000 7f1e73fff700 1 -- 172.21.15.3:0/3009687897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e7412a810 msgr2=0x7f1e74113a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:02.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.010+0000 7f1e73fff700 1 --2- 172.21.15.3:0/3009687897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e7412a810 0x7f1e74113a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:02.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.010+0000 7f1e73fff700 1 -- 172.21.15.3:0/3009687897 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e64009370 con 0x7f1e7412abf0 2023-10-03T10:10:02.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.010+0000 7f1e72ffd700 1 --2- 172.21.15.3:0/3009687897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e7412a810 0x7f1e74113a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:02.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.010+0000 7f1e73fff700 1 --2- 172.21.15.3:0/3009687897 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e7412abf0 0x7f1e74113ef0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f1e7405a050 tx=0x7f1e74067040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:02.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.010+0000 7f1e5b7fe700 1 -- 172.21.15.3:0/3009687897 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c012410 con 0x7f1e7412abf0 2023-10-03T10:10:02.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.010+0000 7f1e5b7fe700 1 -- 172.21.15.3:0/3009687897 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6c01d630 con 0x7f1e7412abf0 2023-10-03T10:10:02.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.010+0000 7f1e79dd4700 1 -- 172.21.15.3:0/3009687897 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e6c009a20 con 0x7f1e7412abf0 2023-10-03T10:10:02.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.010+0000 7f1e5b7fe700 1 -- 172.21.15.3:0/3009687897 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c01d9f0 con 0x7f1e7412abf0 2023-10-03T10:10:02.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.010+0000 7f1e79dd4700 1 -- 172.21.15.3:0/3009687897 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e74117c10 con 0x7f1e7412abf0 2023-10-03T10:10:02.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.011+0000 7f1e5b7fe700 1 -- 172.21.15.3:0/3009687897 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f1e6c003040 con 0x7f1e7412abf0 2023-10-03T10:10:02.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.011+0000 7f1e5b7fe700 1 -- 172.21.15.3:0/3009687897 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f1e6c03cd40 con 0x7f1e7412abf0 2023-10-03T10:10:02.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.012+0000 7f1e79dd4700 1 -- 172.21.15.3:0/3009687897 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e74099040 con 0x7f1e7412abf0 2023-10-03T10:10:02.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.016+0000 7f1e5b7fe700 1 -- 172.21.15.3:0/3009687897 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f1e6c025dc0 con 0x7f1e7412abf0 2023-10-03T10:10:02.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.186+0000 7f1e79dd4700 1 -- 172.21.15.3:0/3009687897 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1e74098e40 con 0x7f1e7412abf0 2023-10-03T10:10:02.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.187+0000 7f1e5b7fe700 1 -- 172.21.15.3:0/3009687897 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v17) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1e6c025360 con 0x7f1e7412abf0 2023-10-03T10:10:02.187 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-10-03T10:10:02.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.188+0000 7f1e79dd4700 1 -- 172.21.15.3:0/3009687897 >> v1:172.21.15.3:6816/118296 conn(0x7f1e6002bb60 legacy=0x7f1e6002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:02.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.189+0000 7f1e79dd4700 1 -- 172.21.15.3:0/3009687897 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e7412abf0 msgr2=0x7f1e74113ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:02.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.189+0000 7f1e79dd4700 1 --2- 172.21.15.3:0/3009687897 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e7412abf0 0x7f1e74113ef0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f1e7405a050 tx=0x7f1e74067040).stop 2023-10-03T10:10:02.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.189+0000 7f1e79dd4700 1 -- 172.21.15.3:0/3009687897 shutdown_connections 2023-10-03T10:10:02.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.189+0000 7f1e79dd4700 1 --2- 172.21.15.3:0/3009687897 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e7412abf0 0x7f1e74113ef0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:02.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.189+0000 7f1e79dd4700 1 --2- 172.21.15.3:0/3009687897 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e74066070 0x7f1e74124b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:02.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.189+0000 7f1e79dd4700 1 --2- 172.21.15.3:0/3009687897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e7412a810 0x7f1e74113a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:02.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.189+0000 7f1e79dd4700 1 -- 172.21.15.3:0/3009687897 >> 172.21.15.3:0/3009687897 conn(0x7f1e74059bb0 msgr2=0x7f1e741272f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:02.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.189+0000 7f1e79dd4700 1 -- 172.21.15.3:0/3009687897 shutdown_connections 2023-10-03T10:10:02.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.189+0000 7f1e79dd4700 1 -- 172.21.15.3:0/3009687897 wait complete. 2023-10-03T10:10:02.202 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-03T10:10:02.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.335+0000 7fd661973700 1 Processor -- start 2023-10-03T10:10:02.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.335+0000 7fd661973700 1 -- start start 2023-10-03T10:10:02.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.336+0000 7fd661973700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd65c066150 0x7fd65c127460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:02.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.336+0000 7fd661973700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd65c1280c0 0x7fd65c12b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:02.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.336+0000 7fd661973700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65c098260 con 0x7fd65c127c80 2023-10-03T10:10:02.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.336+0000 7fd661973700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c066530 con 0x7fd65c1280c0 2023-10-03T10:10:02.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.336+0000 7fd661973700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c064cf0 con 0x7fd65c066150 2023-10-03T10:10:02.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.336+0000 7fd65affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd65c066150 0x7fd65c127460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:02.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.336+0000 7fd65a7fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd65c127c80 0x7fd65c12a9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38666/0 (socket says 172.21.15.3:38666) 2023-10-03T10:10:02.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.336+0000 7fd65b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd65c1280c0 0x7fd65c12b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:02.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.336+0000 7fd65a7fc700 1 -- 172.21.15.3:0/47013156 learned_addr learned my addr 172.21.15.3:0/47013156 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:10:02.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.336+0000 7fd659ffb700 1 -- 172.21.15.3:0/47013156 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2246543692 0 0) 0x7fd65c098260 con 0x7fd65c127c80 2023-10-03T10:10:02.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.336+0000 7fd65b7fe700 1 -- 172.21.15.3:0/47013156 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd65c066150 msgr2=0x7fd65c127460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:02.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.337+0000 7fd65b7fe700 1 --2- 172.21.15.3:0/47013156 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd65c066150 0x7fd65c127460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:02.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.337+0000 7fd65b7fe700 1 -- 172.21.15.3:0/47013156 >> v1:172.21.15.3:6789/0 conn(0x7fd65c127c80 legacy=0x7fd65c12a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:02.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.337+0000 7fd65b7fe700 1 -- 172.21.15.3:0/47013156 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c065950 con 0x7fd65c1280c0 2023-10-03T10:10:02.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.337+0000 7fd65affd700 1 --2- 172.21.15.3:0/47013156 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd65c066150 0x7fd65c127460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:02.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.337+0000 7fd65b7fe700 1 --2- 172.21.15.3:0/47013156 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd65c1280c0 0x7fd65c12b0c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fd654009e90 tx=0x7fd654009780).ready entity=mon.1 client_cookie=fed30331a3a559fa server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:02.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.337+0000 7fd659ffb700 1 -- 172.21.15.3:0/47013156 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65400e070 con 0x7fd65c1280c0 2023-10-03T10:10:02.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.337+0000 7fd659ffb700 1 -- 172.21.15.3:0/47013156 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65400bb60 con 0x7fd65c1280c0 2023-10-03T10:10:02.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.337+0000 7fd659ffb700 1 -- 172.21.15.3:0/47013156 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654012410 con 0x7fd65c1280c0 2023-10-03T10:10:02.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.337+0000 7fd661973700 1 -- 172.21.15.3:0/47013156 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd65c1280c0 msgr2=0x7fd65c12b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:02.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.337+0000 7fd661973700 1 --2- 172.21.15.3:0/47013156 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd65c1280c0 0x7fd65c12b0c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fd654009e90 tx=0x7fd654009780).stop 2023-10-03T10:10:02.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.337+0000 7fd661973700 1 -- 172.21.15.3:0/47013156 shutdown_connections 2023-10-03T10:10:02.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.337+0000 7fd661973700 1 --2- 172.21.15.3:0/47013156 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd65c1280c0 0x7fd65c12b0c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:02.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.337+0000 7fd661973700 1 --2- 172.21.15.3:0/47013156 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd65c066150 0x7fd65c127460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:02.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.337+0000 7fd661973700 1 -- 172.21.15.3:0/47013156 >> 172.21.15.3:0/47013156 conn(0x7fd65c059c60 msgr2=0x7fd65c09bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:02.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.338+0000 7fd661973700 1 -- 172.21.15.3:0/47013156 shutdown_connections 2023-10-03T10:10:02.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.338+0000 7fd661973700 1 -- 172.21.15.3:0/47013156 wait complete. 2023-10-03T10:10:02.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.339+0000 7fd661973700 1 Processor -- start 2023-10-03T10:10:02.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.339+0000 7fd661973700 1 -- start start 2023-10-03T10:10:02.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.339+0000 7fd661973700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd65c066150 0x7fd65c124a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:02.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.339+0000 7fd661973700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd65c127c80 0x7fd65c1139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:02.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.339+0000 7fd661973700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd65c1280c0 0x7fd65c113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:02.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.339+0000 7fd661973700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c115e00 con 0x7fd65c127c80 2023-10-03T10:10:02.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.339+0000 7fd661973700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c064e40 con 0x7fd65c066150 2023-10-03T10:10:02.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.339+0000 7fd661973700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c117470 con 0x7fd65c1280c0 2023-10-03T10:10:02.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.339+0000 7fd65a7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd65c127c80 0x7fd65c1139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:02.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.339+0000 7fd65affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd65c066150 0x7fd65c124a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:02.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.339+0000 7fd65affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd65c066150 0x7fd65c124a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39386/0 (socket says 172.21.15.3:39386) 2023-10-03T10:10:02.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.339+0000 7fd65affd700 1 -- 172.21.15.3:0/806540568 learned_addr learned my addr 172.21.15.3:0/806540568 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:02.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.339+0000 7fd65a7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd65c127c80 0x7fd65c1139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47980/0 (socket says 172.21.15.3:47980) 2023-10-03T10:10:02.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.339+0000 7fd65b7fe700 1 --2- 172.21.15.3:0/806540568 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd65c1280c0 0x7fd65c113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:02.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.340+0000 7fd65affd700 1 -- 172.21.15.3:0/806540568 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd65c1280c0 msgr2=0x7fd65c113e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:02.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.340+0000 7fd65affd700 1 --2- 172.21.15.3:0/806540568 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd65c1280c0 0x7fd65c113e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:02.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.340+0000 7fd65affd700 1 -- 172.21.15.3:0/806540568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd65c127c80 msgr2=0x7fd65c1139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:02.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.340+0000 7fd65affd700 1 --2- 172.21.15.3:0/806540568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd65c127c80 0x7fd65c1139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:02.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.340+0000 7fd65affd700 1 -- 172.21.15.3:0/806540568 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64c0092a0 con 0x7fd65c066150 2023-10-03T10:10:02.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.340+0000 7fd65affd700 1 --2- 172.21.15.3:0/806540568 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd65c066150 0x7fd65c124a70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fd64c00ea20 tx=0x7fd64c009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:02.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.340+0000 7fd642ffd700 1 -- 172.21.15.3:0/806540568 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c00ee60 con 0x7fd65c066150 2023-10-03T10:10:02.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.340+0000 7fd642ffd700 1 -- 172.21.15.3:0/806540568 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64c009600 con 0x7fd65c066150 2023-10-03T10:10:02.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.340+0000 7fd661973700 1 -- 172.21.15.3:0/806540568 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd654009a20 con 0x7fd65c066150 2023-10-03T10:10:02.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.340+0000 7fd642ffd700 1 -- 172.21.15.3:0/806540568 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c00ca60 con 0x7fd65c066150 2023-10-03T10:10:02.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.340+0000 7fd661973700 1 -- 172.21.15.3:0/806540568 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65c117b20 con 0x7fd65c066150 2023-10-03T10:10:02.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.341+0000 7fd642ffd700 1 -- 172.21.15.3:0/806540568 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fd64c01b400 con 0x7fd65c066150 2023-10-03T10:10:02.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.341+0000 7fd642ffd700 1 -- 172.21.15.3:0/806540568 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fd64c03b570 con 0x7fd65c066150 2023-10-03T10:10:02.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.341+0000 7fd661973700 1 -- 172.21.15.3:0/806540568 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd65c05a260 con 0x7fd65c066150 2023-10-03T10:10:02.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.346+0000 7fd642ffd700 1 -- 172.21.15.3:0/806540568 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fd64c016020 con 0x7fd65c066150 2023-10-03T10:10:02.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.522+0000 7fd661973700 1 -- 172.21.15.3:0/806540568 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd65c098dc0 con 0x7fd65c066150 2023-10-03T10:10:02.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.523+0000 7fd642ffd700 1 -- 172.21.15.3:0/806540568 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v17) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd64c011b00 con 0x7fd65c066150 2023-10-03T10:10:02.523 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-10-03T10:10:02.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.524+0000 7fd661973700 1 -- 172.21.15.3:0/806540568 >> v1:172.21.15.3:6816/118296 conn(0x7fd64802bb60 legacy=0x7fd64802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:02.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.524+0000 7fd661973700 1 -- 172.21.15.3:0/806540568 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd65c066150 msgr2=0x7fd65c124a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:02.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.524+0000 7fd661973700 1 --2- 172.21.15.3:0/806540568 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd65c066150 0x7fd65c124a70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fd64c00ea20 tx=0x7fd64c009060).stop 2023-10-03T10:10:02.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.524+0000 7fd661973700 1 -- 172.21.15.3:0/806540568 shutdown_connections 2023-10-03T10:10:02.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.524+0000 7fd661973700 1 --2- 172.21.15.3:0/806540568 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd65c1280c0 0x7fd65c113e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:02.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.524+0000 7fd661973700 1 --2- 172.21.15.3:0/806540568 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd65c066150 0x7fd65c124a70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:02.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.524+0000 7fd661973700 1 --2- 172.21.15.3:0/806540568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd65c127c80 0x7fd65c1139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:02.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.524+0000 7fd661973700 1 -- 172.21.15.3:0/806540568 >> 172.21.15.3:0/806540568 conn(0x7fd65c059c60 msgr2=0x7fd65c09ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:02.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.524+0000 7fd661973700 1 -- 172.21.15.3:0/806540568 shutdown_connections 2023-10-03T10:10:02.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.524+0000 7fd661973700 1 -- 172.21.15.3:0/806540568 wait complete. 2023-10-03T10:10:02.538 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-03T10:10:02.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.669+0000 7f6df651e700 1 Processor -- start 2023-10-03T10:10:02.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.669+0000 7f6df651e700 1 -- start start 2023-10-03T10:10:02.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.670+0000 7f6df651e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6df0093810 0x7f6df0095300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:02.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.670+0000 7f6df651e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6df00957f0 0x7f6df0095bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:02.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.670+0000 7f6df651e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df0093010 con 0x7f6df012c110 2023-10-03T10:10:02.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.670+0000 7f6df651e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df0093bf0 con 0x7f6df00957f0 2023-10-03T10:10:02.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.670+0000 7f6df651e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6df0092aa0 con 0x7f6df0093810 2023-10-03T10:10:02.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.670+0000 7f6deffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6df0093810 0x7f6df0095300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:02.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.670+0000 7f6def7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6df00957f0 0x7f6df0095bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:02.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.670+0000 7f6df4abb700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6df012c110 0x7f6df012e500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38680/0 (socket says 172.21.15.3:38680) 2023-10-03T10:10:02.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.670+0000 7f6df4abb700 1 -- 172.21.15.3:0/2853084121 learned_addr learned my addr 172.21.15.3:0/2853084121 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:10:02.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.670+0000 7f6deffff700 1 -- 172.21.15.3:0/2853084121 >> v2:172.21.15.3:3300/0 conn(0x7f6df00957f0 msgr2=0x7f6df0095bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:02.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.670+0000 7f6deffff700 1 --2- 172.21.15.3:0/2853084121 >> v2:172.21.15.3:3300/0 conn(0x7f6df00957f0 0x7f6df0095bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:02.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.670+0000 7f6deffff700 1 -- 172.21.15.3:0/2853084121 >> v1:172.21.15.3:6789/0 conn(0x7f6df012c110 legacy=0x7f6df012e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:02.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.670+0000 7f6deffff700 1 -- 172.21.15.3:0/2853084121 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df0096e90 con 0x7f6df0093810 2023-10-03T10:10:02.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.670+0000 7f6def7fe700 1 --2- 172.21.15.3:0/2853084121 >> v2:172.21.15.3:3300/0 conn(0x7f6df00957f0 0x7f6df0095bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:02.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.670+0000 7f6deffff700 1 --2- 172.21.15.3:0/2853084121 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6df0093810 0x7f6df0095300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f6dd8009790 tx=0x7f6dd8009060).ready entity=mon.2 client_cookie=3620b90264227d0b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:02.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.670+0000 7f6deeffd700 1 -- 172.21.15.3:0/2853084121 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd800f510 con 0x7f6df0093810 2023-10-03T10:10:02.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.670+0000 7f6deeffd700 1 -- 172.21.15.3:0/2853084121 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd800f670 con 0x7f6df0093810 2023-10-03T10:10:02.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.670+0000 7f6deeffd700 1 -- 172.21.15.3:0/2853084121 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd800f9f0 con 0x7f6df0093810 2023-10-03T10:10:02.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.671+0000 7f6df651e700 1 -- 172.21.15.3:0/2853084121 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6df0093810 msgr2=0x7f6df0095300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:02.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.671+0000 7f6df651e700 1 --2- 172.21.15.3:0/2853084121 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6df0093810 0x7f6df0095300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f6dd8009790 tx=0x7f6dd8009060).stop 2023-10-03T10:10:02.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.671+0000 7f6df651e700 1 -- 172.21.15.3:0/2853084121 shutdown_connections 2023-10-03T10:10:02.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.671+0000 7f6df651e700 1 --2- 172.21.15.3:0/2853084121 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6df0093810 0x7f6df0095300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:02.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.671+0000 7f6df651e700 1 --2- 172.21.15.3:0/2853084121 >> v2:172.21.15.3:3300/0 conn(0x7f6df00957f0 0x7f6df0095bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:02.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.671+0000 7f6df651e700 1 -- 172.21.15.3:0/2853084121 >> 172.21.15.3:0/2853084121 conn(0x7f6df0059bb0 msgr2=0x7f6df0097400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:02.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.671+0000 7f6df651e700 1 -- 172.21.15.3:0/2853084121 shutdown_connections 2023-10-03T10:10:02.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.671+0000 7f6df651e700 1 -- 172.21.15.3:0/2853084121 wait complete. 2023-10-03T10:10:02.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.672+0000 7f6df651e700 1 Processor -- start 2023-10-03T10:10:02.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.672+0000 7f6df651e700 1 -- start start 2023-10-03T10:10:02.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.673+0000 7f6df651e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6df0093810 0x7f6df01199a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:02.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.673+0000 7f6df651e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6df00957f0 0x7f6df0115970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:02.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.673+0000 7f6df651e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6df012c110 0x7f6df0115e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:02.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.673+0000 7f6df651e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df012bb20 con 0x7f6df012c110 2023-10-03T10:10:02.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.673+0000 7f6df651e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df01194a0 con 0x7f6df0093810 2023-10-03T10:10:02.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.673+0000 7f6df651e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6df0063600 con 0x7f6df00957f0 2023-10-03T10:10:02.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.673+0000 7f6deffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6df0093810 0x7f6df01199a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:02.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.673+0000 7f6def7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6df00957f0 0x7f6df0115970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:02.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.673+0000 7f6df4abb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6df012c110 0x7f6df0115e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:02.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.673+0000 7f6deffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6df0093810 0x7f6df01199a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39392/0 (socket says 172.21.15.3:39392) 2023-10-03T10:10:02.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.673+0000 7f6deffff700 1 -- 172.21.15.3:0/2595235945 learned_addr learned my addr 172.21.15.3:0/2595235945 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:02.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.673+0000 7f6df4abb700 1 -- 172.21.15.3:0/2595235945 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6df00957f0 msgr2=0x7f6df0115970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:02.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.673+0000 7f6df4abb700 1 --2- 172.21.15.3:0/2595235945 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6df00957f0 0x7f6df0115970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:02.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.673+0000 7f6df4abb700 1 -- 172.21.15.3:0/2595235945 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6df0093810 msgr2=0x7f6df01199a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:02.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.673+0000 7f6df4abb700 1 --2- 172.21.15.3:0/2595235945 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6df0093810 0x7f6df01199a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:02.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.673+0000 7f6df4abb700 1 -- 172.21.15.3:0/2595235945 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de00092a0 con 0x7f6df012c110 2023-10-03T10:10:02.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.673+0000 7f6df4abb700 1 --2- 172.21.15.3:0/2595235945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6df012c110 0x7f6df0115e60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f6de8005ad0 tx=0x7f6de8001530).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:02.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.674+0000 7f6de7fff700 1 -- 172.21.15.3:0/2595235945 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de8005d30 con 0x7f6df012c110 2023-10-03T10:10:02.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.674+0000 7f6de7fff700 1 -- 172.21.15.3:0/2595235945 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de8005e90 con 0x7f6df012c110 2023-10-03T10:10:02.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.674+0000 7f6df651e700 1 -- 172.21.15.3:0/2595235945 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dd80092a0 con 0x7f6df012c110 2023-10-03T10:10:02.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.674+0000 7f6de7fff700 1 -- 172.21.15.3:0/2595235945 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de8007300 con 0x7f6df012c110 2023-10-03T10:10:02.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.674+0000 7f6df651e700 1 -- 172.21.15.3:0/2595235945 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df0116d70 con 0x7f6df012c110 2023-10-03T10:10:02.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.674+0000 7f6de7fff700 1 -- 172.21.15.3:0/2595235945 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f6de80076b0 con 0x7f6df012c110 2023-10-03T10:10:02.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.674+0000 7f6de7fff700 1 -- 172.21.15.3:0/2595235945 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f6de803a8f0 con 0x7f6df012c110 2023-10-03T10:10:02.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.675+0000 7f6df651e700 1 -- 172.21.15.3:0/2595235945 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df01190c0 con 0x7f6df012c110 2023-10-03T10:10:02.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.679+0000 7f6de7fff700 1 -- 172.21.15.3:0/2595235945 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f6de8018030 con 0x7f6df012c110 2023-10-03T10:10:02.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.898+0000 7f6df651e700 1 -- 172.21.15.3:0/2595235945 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6df005a260 con 0x7f6df012c110 2023-10-03T10:10:02.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.898+0000 7f6de7fff700 1 -- 172.21.15.3:0/2595235945 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2154 (secure 0 0 0) 0x7f6de8018020 con 0x7f6df012c110 2023-10-03T10:10:02.899 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:10:02.899 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-03T10:09:44.141304+0000","modified":"2023-10-03T10:09:47.154287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4186},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4186":{"gid":4186,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6817/2582564779","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6817","nonce":2582564779}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-03T10:10:02.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.900+0000 7f6df651e700 1 -- 172.21.15.3:0/2595235945 >> v1:172.21.15.3:6816/118296 conn(0x7f6dd002bb60 legacy=0x7f6dd002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:02.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.900+0000 7f6df651e700 1 -- 172.21.15.3:0/2595235945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6df012c110 msgr2=0x7f6df0115e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:02.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.900+0000 7f6df651e700 1 --2- 172.21.15.3:0/2595235945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6df012c110 0x7f6df0115e60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f6de8005ad0 tx=0x7f6de8001530).stop 2023-10-03T10:10:02.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.901+0000 7f6df651e700 1 -- 172.21.15.3:0/2595235945 shutdown_connections 2023-10-03T10:10:02.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.901+0000 7f6df651e700 1 --2- 172.21.15.3:0/2595235945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6df012c110 0x7f6df0115e60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:02.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.901+0000 7f6df651e700 1 --2- 172.21.15.3:0/2595235945 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6df0093810 0x7f6df01199a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:02.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.901+0000 7f6df651e700 1 --2- 172.21.15.3:0/2595235945 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6df00957f0 0x7f6df0115970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:02.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.901+0000 7f6df651e700 1 -- 172.21.15.3:0/2595235945 >> 172.21.15.3:0/2595235945 conn(0x7f6df0059bb0 msgr2=0x7f6df0068c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:02.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.901+0000 7f6df651e700 1 -- 172.21.15.3:0/2595235945 shutdown_connections 2023-10-03T10:10:02.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:02.901+0000 7f6df651e700 1 -- 172.21.15.3:0/2595235945 wait complete. 2023-10-03T10:10:02.902 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 6 2023-10-03T10:10:02.914 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-03T10:10:03.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.043+0000 7fd385859700 1 Processor -- start 2023-10-03T10:10:03.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.044+0000 7fd385859700 1 -- start start 2023-10-03T10:10:03.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.044+0000 7fd385859700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd380097cc0 0x7fd380098e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:03.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.044+0000 7fd385859700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3800992f0 0x7fd3801324a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:03.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.044+0000 7fd385859700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd380066070 con 0x7fd3800653b0 2023-10-03T10:10:03.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.044+0000 7fd385859700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd380065790 con 0x7fd380097cc0 2023-10-03T10:10:03.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.044+0000 7fd385859700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd380095300 con 0x7fd3800992f0 2023-10-03T10:10:03.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.044+0000 7fd37e7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd380097cc0 0x7fd380098e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:03.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.044+0000 7fd37f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3800992f0 0x7fd3801324a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:03.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.044+0000 7fd37effd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd3800653b0 0x7fd3800986f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38686/0 (socket says 172.21.15.3:38686) 2023-10-03T10:10:03.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.044+0000 7fd37effd700 1 -- 172.21.15.3:0/1696228477 learned_addr learned my addr 172.21.15.3:0/1696228477 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:10:03.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.045+0000 7fd37f7fe700 1 -- 172.21.15.3:0/1696228477 >> v2:172.21.15.3:3300/0 conn(0x7fd380097cc0 msgr2=0x7fd380098e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:03.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.045+0000 7fd37f7fe700 1 --2- 172.21.15.3:0/1696228477 >> v2:172.21.15.3:3300/0 conn(0x7fd380097cc0 0x7fd380098e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.045+0000 7fd37f7fe700 1 -- 172.21.15.3:0/1696228477 >> v1:172.21.15.3:6789/0 conn(0x7fd3800653b0 legacy=0x7fd3800986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:03.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.045+0000 7fd37f7fe700 1 -- 172.21.15.3:0/1696228477 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd380098150 con 0x7fd3800992f0 2023-10-03T10:10:03.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.045+0000 7fd37e7fc700 1 --2- 172.21.15.3:0/1696228477 >> v2:172.21.15.3:3300/0 conn(0x7fd380097cc0 0x7fd380098e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:03.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.045+0000 7fd37f7fe700 1 --2- 172.21.15.3:0/1696228477 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3800992f0 0x7fd3801324a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fd378009e60 tx=0x7fd378009780).ready entity=mon.2 client_cookie=e9ae05899926b814 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:03.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.045+0000 7fd37dffb700 1 -- 172.21.15.3:0/1696228477 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37800e070 con 0x7fd3800992f0 2023-10-03T10:10:03.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.045+0000 7fd37dffb700 1 -- 172.21.15.3:0/1696228477 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37800bb60 con 0x7fd3800992f0 2023-10-03T10:10:03.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.045+0000 7fd37dffb700 1 -- 172.21.15.3:0/1696228477 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd378012410 con 0x7fd3800992f0 2023-10-03T10:10:03.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.045+0000 7fd385859700 1 -- 172.21.15.3:0/1696228477 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3800992f0 msgr2=0x7fd3801324a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:03.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.045+0000 7fd385859700 1 --2- 172.21.15.3:0/1696228477 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3800992f0 0x7fd3801324a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fd378009e60 tx=0x7fd378009780).stop 2023-10-03T10:10:03.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.046+0000 7fd385859700 1 -- 172.21.15.3:0/1696228477 shutdown_connections 2023-10-03T10:10:03.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.046+0000 7fd385859700 1 --2- 172.21.15.3:0/1696228477 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3800992f0 0x7fd3801324a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.046+0000 7fd385859700 1 --2- 172.21.15.3:0/1696228477 >> v2:172.21.15.3:3300/0 conn(0x7fd380097cc0 0x7fd380098e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.046+0000 7fd385859700 1 -- 172.21.15.3:0/1696228477 >> 172.21.15.3:0/1696228477 conn(0x7fd380059cd0 msgr2=0x7fd38009a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:03.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.046+0000 7fd385859700 1 -- 172.21.15.3:0/1696228477 shutdown_connections 2023-10-03T10:10:03.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.046+0000 7fd385859700 1 -- 172.21.15.3:0/1696228477 wait complete. 2023-10-03T10:10:03.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.047+0000 7fd385859700 1 Processor -- start 2023-10-03T10:10:03.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.047+0000 7fd385859700 1 -- start start 2023-10-03T10:10:03.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.047+0000 7fd385859700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3800653b0 0x7fd38011e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:03.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.047+0000 7fd385859700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3800992f0 0x7fd38011a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:03.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.047+0000 7fd385859700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd38011aae0 0x7fd38011af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:03.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.047+0000 7fd385859700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38011ba80 con 0x7fd3800992f0 2023-10-03T10:10:03.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.047+0000 7fd385859700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38011b450 con 0x7fd38011aae0 2023-10-03T10:10:03.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.047+0000 7fd385859700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd380099a80 con 0x7fd3800653b0 2023-10-03T10:10:03.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.047+0000 7fd37effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3800653b0 0x7fd38011e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:03.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.047+0000 7fd37e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3800992f0 0x7fd38011a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:03.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.047+0000 7fd37effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3800653b0 0x7fd38011e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57792/0 (socket says 172.21.15.3:57792) 2023-10-03T10:10:03.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.047+0000 7fd37effd700 1 -- 172.21.15.3:0/3464233747 learned_addr learned my addr 172.21.15.3:0/3464233747 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:03.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.047+0000 7fd37f7fe700 1 --2- 172.21.15.3:0/3464233747 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd38011aae0 0x7fd38011af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:03.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.048+0000 7fd37effd700 1 -- 172.21.15.3:0/3464233747 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd38011aae0 msgr2=0x7fd38011af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:03.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.048+0000 7fd37effd700 1 --2- 172.21.15.3:0/3464233747 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd38011aae0 0x7fd38011af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.048+0000 7fd37effd700 1 -- 172.21.15.3:0/3464233747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3800992f0 msgr2=0x7fd38011a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:03.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.048+0000 7fd37effd700 1 --2- 172.21.15.3:0/3464233747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3800992f0 0x7fd38011a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.048+0000 7fd37effd700 1 -- 172.21.15.3:0/3464233747 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3740092a0 con 0x7fd3800653b0 2023-10-03T10:10:03.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.048+0000 7fd37e7fc700 1 --2- 172.21.15.3:0/3464233747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3800992f0 0x7fd38011a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:03.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.048+0000 7fd37effd700 1 --2- 172.21.15.3:0/3464233747 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3800653b0 0x7fd38011e4f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fd370007fd0 tx=0x7fd370007df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:03.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.048+0000 7fd366ffd700 1 -- 172.21.15.3:0/3464233747 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37000ce10 con 0x7fd3800653b0 2023-10-03T10:10:03.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.048+0000 7fd366ffd700 1 -- 172.21.15.3:0/3464233747 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3700072e0 con 0x7fd3800653b0 2023-10-03T10:10:03.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.048+0000 7fd385859700 1 -- 172.21.15.3:0/3464233747 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd378009a20 con 0x7fd3800653b0 2023-10-03T10:10:03.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.048+0000 7fd366ffd700 1 -- 172.21.15.3:0/3464233747 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd370007660 con 0x7fd3800653b0 2023-10-03T10:10:03.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.048+0000 7fd385859700 1 -- 172.21.15.3:0/3464233747 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd380093af0 con 0x7fd3800653b0 2023-10-03T10:10:03.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.049+0000 7fd366ffd700 1 -- 172.21.15.3:0/3464233747 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fd3700059b0 con 0x7fd3800653b0 2023-10-03T10:10:03.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.049+0000 7fd366ffd700 1 -- 172.21.15.3:0/3464233747 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fd37003a710 con 0x7fd3800653b0 2023-10-03T10:10:03.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.049+0000 7fd385859700 1 -- 172.21.15.3:0/3464233747 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38011df30 con 0x7fd3800653b0 2023-10-03T10:10:03.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.054+0000 7fd366ffd700 1 -- 172.21.15.3:0/3464233747 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fd370018070 con 0x7fd3800653b0 2023-10-03T10:10:03.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.229+0000 7fd385859700 1 -- 172.21.15.3:0/3464233747 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd38005a260 con 0x7fd3800653b0 2023-10-03T10:10:03.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.229+0000 7fd366ffd700 1 -- 172.21.15.3:0/3464233747 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+11309 (secure 0 0 0) 0x7fd370018020 con 0x7fd3800653b0 2023-10-03T10:10:03.230 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:10:03.230 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":17,"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","created":"2023-10-03T10:09:28.703441+0000","modified":"2023-10-03T10:09:45.145435+0000","last_up_change":"2023-10-03T10:09:36.569401+0000","last_in_change":"2023-10-03T10:09:30.156546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-10-03T10:09:36.623971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-03T10:09:38.210614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-03T10:09:42.695873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-03T10:09:43.115588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cdcdd930-3c5b-4486-a021-7132f90fb6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6808","nonce":118845}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6809","nonce":118845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6811","nonce":118845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6810","nonce":118845}]},"public_addr":"172.21.15.3:6808/118845","cluster_addr":"172.21.15.3:6809/118845","heartbeat_back_addr":"172.21.15.3:6811/118845","heartbeat_front_addr":"172.21.15.3:6810/118845","state":["exists","up"]},{"osd":1,"uuid":"c8e58877-6762-4741-a8c1-567ebc97b708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6804","nonce":118835}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6805","nonce":118835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6807","nonce":118835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6806","nonce":118835}]},"public_addr":"172.21.15.3:6804/118835","cluster_addr":"172.21.15.3:6805/118835","heartbeat_back_addr":"172.21.15.3:6807/118835","heartbeat_front_addr":"172.21.15.3:6806/118835","state":["exists","up"]},{"osd":2,"uuid":"68a2a21a-7a8e-4f82-8433-5201faa50777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6812","nonce":118864}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6813","nonce":118864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6815","nonce":118864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6814","nonce":118864}]},"public_addr":"172.21.15.3:6812/118864","cluster_addr":"172.21.15.3:6813/118864","heartbeat_back_addr":"172.21.15.3:6815/118864","heartbeat_front_addr":"172.21.15.3:6814/118864","state":["exists","up"]},{"osd":3,"uuid":"c4df2903-4e1e-4208-bf77-95b9fa749241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6800","nonce":118836}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6801","nonce":118836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6803","nonce":118836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6802","nonce":118836}]},"public_addr":"172.21.15.3:6800/118836","cluster_addr":"172.21.15.3:6801/118836","heartbeat_back_addr":"172.21.15.3:6803/118836","heartbeat_front_addr":"172.21.15.3:6802/118836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.369708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.257825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.337931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.085222+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-03T10:10:03.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.231+0000 7fd385859700 1 -- 172.21.15.3:0/3464233747 >> v1:172.21.15.3:6816/118296 conn(0x7fd36c02bb90 legacy=0x7fd36c02e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:03.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.231+0000 7fd385859700 1 -- 172.21.15.3:0/3464233747 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3800653b0 msgr2=0x7fd38011e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:03.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.231+0000 7fd385859700 1 --2- 172.21.15.3:0/3464233747 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3800653b0 0x7fd38011e4f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fd370007fd0 tx=0x7fd370007df0).stop 2023-10-03T10:10:03.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.231+0000 7fd385859700 1 -- 172.21.15.3:0/3464233747 shutdown_connections 2023-10-03T10:10:03.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.231+0000 7fd385859700 1 --2- 172.21.15.3:0/3464233747 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd38011aae0 0x7fd38011af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.231+0000 7fd385859700 1 --2- 172.21.15.3:0/3464233747 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3800653b0 0x7fd38011e4f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.231+0000 7fd385859700 1 --2- 172.21.15.3:0/3464233747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3800992f0 0x7fd38011a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.231+0000 7fd385859700 1 -- 172.21.15.3:0/3464233747 >> 172.21.15.3:0/3464233747 conn(0x7fd380059cd0 msgr2=0x7fd380091a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:03.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.232+0000 7fd385859700 1 -- 172.21.15.3:0/3464233747 shutdown_connections 2023-10-03T10:10:03.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.232+0000 7fd385859700 1 -- 172.21.15.3:0/3464233747 wait complete. 2023-10-03T10:10:03.245 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-03T10:10:03.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.375+0000 7f9498806700 1 Processor -- start 2023-10-03T10:10:03.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.375+0000 7f9498806700 1 -- start start 2023-10-03T10:10:03.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.376+0000 7f9498806700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94940943a0 0x7f9494126e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:03.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.376+0000 7f9498806700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f949412c1a0 0x7f949412e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:03.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.376+0000 7f9498806700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9494093970 con 0x7f94941273c0 2023-10-03T10:10:03.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.376+0000 7f9498806700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9494094780 con 0x7f949412c1a0 2023-10-03T10:10:03.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.376+0000 7f9498806700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9494093ba0 con 0x7f94940943a0 2023-10-03T10:10:03.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.376+0000 7f949259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94940943a0 0x7f9494126e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:03.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.376+0000 7f9492d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f949412c1a0 0x7f949412e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:03.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.376+0000 7f949259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94940943a0 0x7f9494126e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:60242/0 (socket says 172.21.15.3:60242) 2023-10-03T10:10:03.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.376+0000 7f949259c700 1 -- 172.21.15.3:0/3921002609 learned_addr learned my addr 172.21.15.3:0/3921002609 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:03.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.376+0000 7f9492d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f949412c1a0 0x7f949412e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47350/0 (socket says 172.21.15.3:47350) 2023-10-03T10:10:03.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.376+0000 7f949259c700 1 -- 172.21.15.3:0/3921002609 >> v2:172.21.15.3:3300/0 conn(0x7f949412c1a0 msgr2=0x7f949412e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:03.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.376+0000 7f949259c700 1 --2- 172.21.15.3:0/3921002609 >> v2:172.21.15.3:3300/0 conn(0x7f949412c1a0 0x7f949412e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.376+0000 7f949259c700 1 -- 172.21.15.3:0/3921002609 >> v1:172.21.15.3:6789/0 conn(0x7f94941273c0 legacy=0x7f94941277c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:03.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.376+0000 7f949159a700 1 -- 172.21.15.3:0/3921002609 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2088515437 0 0) 0x7f9494093970 con 0x7f94941273c0 2023-10-03T10:10:03.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.376+0000 7f949259c700 1 -- 172.21.15.3:0/3921002609 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9494093f10 con 0x7f94940943a0 2023-10-03T10:10:03.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.376+0000 7f9492d9d700 1 --2- 172.21.15.3:0/3921002609 >> v2:172.21.15.3:3300/0 conn(0x7f949412c1a0 0x7f949412e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:03.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.376+0000 7f949259c700 1 --2- 172.21.15.3:0/3921002609 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94940943a0 0x7f9494126e10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f9484006fd0 tx=0x7f94840090f0).ready entity=mon.2 client_cookie=d93a77c7b636f11 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:03.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.377+0000 7f949159a700 1 -- 172.21.15.3:0/3921002609 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948400b580 con 0x7f94940943a0 2023-10-03T10:10:03.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.377+0000 7f949159a700 1 -- 172.21.15.3:0/3921002609 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948400b6e0 con 0x7f94940943a0 2023-10-03T10:10:03.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.377+0000 7f949159a700 1 -- 172.21.15.3:0/3921002609 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948400ba80 con 0x7f94940943a0 2023-10-03T10:10:03.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.377+0000 7f9498806700 1 -- 172.21.15.3:0/3921002609 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94940943a0 msgr2=0x7f9494126e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:03.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.377+0000 7f9498806700 1 --2- 172.21.15.3:0/3921002609 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94940943a0 0x7f9494126e10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f9484006fd0 tx=0x7f94840090f0).stop 2023-10-03T10:10:03.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.377+0000 7f9498806700 1 -- 172.21.15.3:0/3921002609 shutdown_connections 2023-10-03T10:10:03.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.377+0000 7f9498806700 1 --2- 172.21.15.3:0/3921002609 >> v2:172.21.15.3:3300/0 conn(0x7f949412c1a0 0x7f949412e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.377+0000 7f9498806700 1 --2- 172.21.15.3:0/3921002609 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94940943a0 0x7f9494126e10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.377+0000 7f9498806700 1 -- 172.21.15.3:0/3921002609 >> 172.21.15.3:0/3921002609 conn(0x7f9494063190 msgr2=0x7f949409a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:03.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.377+0000 7f9498806700 1 -- 172.21.15.3:0/3921002609 shutdown_connections 2023-10-03T10:10:03.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.377+0000 7f9498806700 1 -- 172.21.15.3:0/3921002609 wait complete. 2023-10-03T10:10:03.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.378+0000 7f9498806700 1 Processor -- start 2023-10-03T10:10:03.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.378+0000 7f9498806700 1 -- start start 2023-10-03T10:10:03.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.379+0000 7f9498806700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94940943a0 0x7f9494110810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:03.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.379+0000 7f9498806700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94941273c0 0x7f9494110d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:03.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.379+0000 7f9498806700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f949412c1a0 0x7f94941111f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:03.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.379+0000 7f9498806700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94941116e0 con 0x7f949412c1a0 2023-10-03T10:10:03.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.379+0000 7f9498806700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94940681c0 con 0x7f94941273c0 2023-10-03T10:10:03.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.379+0000 7f9498806700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9494114db0 con 0x7f94940943a0 2023-10-03T10:10:03.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.379+0000 7f949259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94940943a0 0x7f9494110810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:03.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.379+0000 7f9492d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f949412c1a0 0x7f94941111f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:03.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.379+0000 7f949259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94940943a0 0x7f9494110810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:60246/0 (socket says 172.21.15.3:60246) 2023-10-03T10:10:03.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.379+0000 7f949259c700 1 -- 172.21.15.3:0/3079910161 learned_addr learned my addr 172.21.15.3:0/3079910161 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:03.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.379+0000 7f9491d9b700 1 --2- 172.21.15.3:0/3079910161 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94941273c0 0x7f9494110d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:03.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.379+0000 7f9492d9d700 1 -- 172.21.15.3:0/3079910161 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94940943a0 msgr2=0x7f9494110810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:03.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.379+0000 7f9492d9d700 1 --2- 172.21.15.3:0/3079910161 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94940943a0 0x7f9494110810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.379+0000 7f9492d9d700 1 -- 172.21.15.3:0/3079910161 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94941273c0 msgr2=0x7f9494110d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:03.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.379+0000 7f9492d9d700 1 --2- 172.21.15.3:0/3079910161 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94941273c0 0x7f9494110d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.379+0000 7f9492d9d700 1 -- 172.21.15.3:0/3079910161 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948c009a20 con 0x7f949412c1a0 2023-10-03T10:10:03.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.379+0000 7f9492d9d700 1 --2- 172.21.15.3:0/3079910161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f949412c1a0 0x7f94941111f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f948c009f10 tx=0x7f948c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:03.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.379+0000 7f94827fc700 1 -- 172.21.15.3:0/3079910161 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c00cb70 con 0x7f949412c1a0 2023-10-03T10:10:03.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.379+0000 7f94827fc700 1 -- 172.21.15.3:0/3079910161 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948c00f040 con 0x7f949412c1a0 2023-10-03T10:10:03.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.379+0000 7f94827fc700 1 -- 172.21.15.3:0/3079910161 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c00cee0 con 0x7f949412c1a0 2023-10-03T10:10:03.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.379+0000 7f9498806700 1 -- 172.21.15.3:0/3079910161 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9484009370 con 0x7f949412c1a0 2023-10-03T10:10:03.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.380+0000 7f9498806700 1 -- 172.21.15.3:0/3079910161 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94941122e0 con 0x7f949412c1a0 2023-10-03T10:10:03.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.380+0000 7f94827fc700 1 -- 172.21.15.3:0/3079910161 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f948c013610 con 0x7f949412c1a0 2023-10-03T10:10:03.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.380+0000 7f94827fc700 1 -- 172.21.15.3:0/3079910161 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f948c03b7f0 con 0x7f949412c1a0 2023-10-03T10:10:03.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.381+0000 7f9498806700 1 -- 172.21.15.3:0/3079910161 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f949405a260 con 0x7f949412c1a0 2023-10-03T10:10:03.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.385+0000 7f94827fc700 1 -- 172.21.15.3:0/3079910161 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f948c01c410 con 0x7f949412c1a0 2023-10-03T10:10:03.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.601+0000 7f9498806700 1 -- 172.21.15.3:0/3079910161 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9494114190 con 0x7f949412c1a0 2023-10-03T10:10:03.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.602+0000 7f94827fc700 1 -- 172.21.15.3:0/3079910161 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2154 (secure 0 0 0) 0x7f948c02b090 con 0x7f949412c1a0 2023-10-03T10:10:03.602 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:10:03.603 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-03T10:09:44.141304+0000","modified":"2023-10-03T10:09:47.154287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4186},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4186":{"gid":4186,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6817/2582564779","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6817","nonce":2582564779}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-03T10:10:03.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.604+0000 7f9498806700 1 -- 172.21.15.3:0/3079910161 >> v1:172.21.15.3:6816/118296 conn(0x7f947402bb90 legacy=0x7f947402e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:03.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.604+0000 7f9498806700 1 -- 172.21.15.3:0/3079910161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f949412c1a0 msgr2=0x7f94941111f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:03.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.604+0000 7f9498806700 1 --2- 172.21.15.3:0/3079910161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f949412c1a0 0x7f94941111f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f948c009f10 tx=0x7f948c00b040).stop 2023-10-03T10:10:03.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.604+0000 7f9498806700 1 -- 172.21.15.3:0/3079910161 shutdown_connections 2023-10-03T10:10:03.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.604+0000 7f9498806700 1 --2- 172.21.15.3:0/3079910161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f949412c1a0 0x7f94941111f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.604+0000 7f9498806700 1 --2- 172.21.15.3:0/3079910161 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94940943a0 0x7f9494110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.604+0000 7f9498806700 1 --2- 172.21.15.3:0/3079910161 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94941273c0 0x7f9494110d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.604+0000 7f9498806700 1 -- 172.21.15.3:0/3079910161 >> 172.21.15.3:0/3079910161 conn(0x7f9494063190 msgr2=0x7f9494125ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:03.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.604+0000 7f9498806700 1 -- 172.21.15.3:0/3079910161 shutdown_connections 2023-10-03T10:10:03.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.604+0000 7f9498806700 1 -- 172.21.15.3:0/3079910161 wait complete. 2023-10-03T10:10:03.605 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 6 2023-10-03T10:10:03.719 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-03T10:09:44.141304+0000', 'modified': '2023-10-03T10:09:47.154287+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4186}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4186': {'gid': 4186, 'name': 'a', 'rank': 0, 'incarnation': 3, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.3:6817/2582564779', 'addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.3:6817', 'nonce': 2582564779}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-03T10:10:03.719 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-03T10:10:03.719 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-03T10:10:03.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.855+0000 7fc5e0967700 1 Processor -- start 2023-10-03T10:10:03.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.855+0000 7fc5e0967700 1 -- start start 2023-10-03T10:10:03.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.856+0000 7fc5e0967700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc5dc127ed0 0x7fc5dc12fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:03.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.856+0000 7fc5e0967700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5dc130130 0x7fc5dc132520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:03.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.856+0000 7fc5e0967700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5dc091f10 con 0x7fc5dc062a90 2023-10-03T10:10:03.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.856+0000 7fc5e0967700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5dc1282f0 con 0x7fc5dc127ed0 2023-10-03T10:10:03.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.856+0000 7fc5e0967700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc09a1a0 con 0x7fc5dc130130 2023-10-03T10:10:03.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.856+0000 7fc5dad9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5dc130130 0x7fc5dc132520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:03.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.856+0000 7fc5d9d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc5dc127ed0 0x7fc5dc12fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:03.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.856+0000 7fc5da59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc5dc062a90 0x7fc5dc1277c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47440/0 (socket says 172.21.15.3:47440) 2023-10-03T10:10:03.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.856+0000 7fc5dad9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5dc130130 0x7fc5dc132520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53766/0 (socket says 172.21.15.3:53766) 2023-10-03T10:10:03.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.856+0000 7fc5d9d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc5dc127ed0 0x7fc5dc12fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47370/0 (socket says 172.21.15.3:47370) 2023-10-03T10:10:03.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.856+0000 7fc5da59c700 1 -- 172.21.15.3:0/534018573 learned_addr learned my addr 172.21.15.3:0/534018573 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:10:03.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.856+0000 7fc5d959a700 1 -- 172.21.15.3:0/534018573 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 447598505 0 0) 0x7fc5dc091f10 con 0x7fc5dc062a90 2023-10-03T10:10:03.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.856+0000 7fc5dad9d700 1 -- 172.21.15.3:0/534018573 >> v2:172.21.15.3:3300/0 conn(0x7fc5dc127ed0 msgr2=0x7fc5dc12fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:03.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.856+0000 7fc5dad9d700 1 --2- 172.21.15.3:0/534018573 >> v2:172.21.15.3:3300/0 conn(0x7fc5dc127ed0 0x7fc5dc12fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.856+0000 7fc5dad9d700 1 -- 172.21.15.3:0/534018573 >> v1:172.21.15.3:6789/0 conn(0x7fc5dc062a90 legacy=0x7fc5dc1277c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:03.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.856+0000 7fc5dad9d700 1 -- 172.21.15.3:0/534018573 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5dc091bc0 con 0x7fc5dc130130 2023-10-03T10:10:03.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.856+0000 7fc5dad9d700 1 --2- 172.21.15.3:0/534018573 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5dc130130 0x7fc5dc132520 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fc5d4009e60 tx=0x7fc5d4009780).ready entity=mon.1 client_cookie=5466ed711142d2d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:03.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.857+0000 7fc5d959a700 1 -- 172.21.15.3:0/534018573 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d400e070 con 0x7fc5dc130130 2023-10-03T10:10:03.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.857+0000 7fc5d959a700 1 -- 172.21.15.3:0/534018573 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d400bb60 con 0x7fc5dc130130 2023-10-03T10:10:03.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.857+0000 7fc5d959a700 1 -- 172.21.15.3:0/534018573 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4012410 con 0x7fc5dc130130 2023-10-03T10:10:03.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.857+0000 7fc5e0967700 1 -- 172.21.15.3:0/534018573 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5dc130130 msgr2=0x7fc5dc132520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:03.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.857+0000 7fc5e0967700 1 --2- 172.21.15.3:0/534018573 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5dc130130 0x7fc5dc132520 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fc5d4009e60 tx=0x7fc5d4009780).stop 2023-10-03T10:10:03.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.857+0000 7fc5e0967700 1 -- 172.21.15.3:0/534018573 shutdown_connections 2023-10-03T10:10:03.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.857+0000 7fc5e0967700 1 --2- 172.21.15.3:0/534018573 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5dc130130 0x7fc5dc132520 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.857+0000 7fc5e0967700 1 --2- 172.21.15.3:0/534018573 >> v2:172.21.15.3:3300/0 conn(0x7fc5dc127ed0 0x7fc5dc12fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.857+0000 7fc5e0967700 1 -- 172.21.15.3:0/534018573 >> 172.21.15.3:0/534018573 conn(0x7fc5dc097460 msgr2=0x7fc5dc097870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:03.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.857+0000 7fc5e0967700 1 -- 172.21.15.3:0/534018573 shutdown_connections 2023-10-03T10:10:03.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.857+0000 7fc5e0967700 1 -- 172.21.15.3:0/534018573 wait complete. 2023-10-03T10:10:03.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.858+0000 7fc5e0967700 1 Processor -- start 2023-10-03T10:10:03.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.858+0000 7fc5e0967700 1 -- start start 2023-10-03T10:10:03.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.859+0000 7fc5e0967700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5dc062a90 0x7fc5dc124e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:03.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.859+0000 7fc5e0967700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5dc127ed0 0x7fc5dc12dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:03.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.859+0000 7fc5e0967700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5dc130130 0x7fc5dc12e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:03.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.859+0000 7fc5e0967700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc1129f0 con 0x7fc5dc062a90 2023-10-03T10:10:03.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.859+0000 7fc5e0967700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc112d80 con 0x7fc5dc130130 2023-10-03T10:10:03.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.859+0000 7fc5e0967700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc09a3d0 con 0x7fc5dc127ed0 2023-10-03T10:10:03.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.859+0000 7fc5dad9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5dc130130 0x7fc5dc12e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:03.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.859+0000 7fc5da59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5dc062a90 0x7fc5dc124e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:03.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.859+0000 7fc5dad9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5dc130130 0x7fc5dc12e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53776/0 (socket says 172.21.15.3:53776) 2023-10-03T10:10:03.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.859+0000 7fc5dad9d700 1 -- 172.21.15.3:0/2955291703 learned_addr learned my addr 172.21.15.3:0/2955291703 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:03.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.859+0000 7fc5d9d9b700 1 --2- 172.21.15.3:0/2955291703 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5dc127ed0 0x7fc5dc12dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:03.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.859+0000 7fc5dad9d700 1 -- 172.21.15.3:0/2955291703 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5dc127ed0 msgr2=0x7fc5dc12dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:03.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.859+0000 7fc5dad9d700 1 --2- 172.21.15.3:0/2955291703 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5dc127ed0 0x7fc5dc12dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.859+0000 7fc5dad9d700 1 -- 172.21.15.3:0/2955291703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5dc062a90 msgr2=0x7fc5dc124e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:03.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.859+0000 7fc5dad9d700 1 --2- 172.21.15.3:0/2955291703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5dc062a90 0x7fc5dc124e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.859+0000 7fc5dad9d700 1 -- 172.21.15.3:0/2955291703 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d4009a20 con 0x7fc5dc130130 2023-10-03T10:10:03.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.859+0000 7fc5d9d9b700 1 --2- 172.21.15.3:0/2955291703 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5dc127ed0 0x7fc5dc12dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:03.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.860+0000 7fc5dad9d700 1 --2- 172.21.15.3:0/2955291703 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5dc130130 0x7fc5dc12e1f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fc5dc09a2e0 tx=0x7fc5dc099b80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:03.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.860+0000 7fc5ca7fc700 1 -- 172.21.15.3:0/2955291703 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4012410 con 0x7fc5dc130130 2023-10-03T10:10:03.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.860+0000 7fc5ca7fc700 1 -- 172.21.15.3:0/2955291703 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d401d630 con 0x7fc5dc130130 2023-10-03T10:10:03.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.860+0000 7fc5ca7fc700 1 -- 172.21.15.3:0/2955291703 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d40070e0 con 0x7fc5dc130130 2023-10-03T10:10:03.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.860+0000 7fc5e0967700 1 -- 172.21.15.3:0/2955291703 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5dc1122e0 con 0x7fc5dc130130 2023-10-03T10:10:03.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.860+0000 7fc5e0967700 1 -- 172.21.15.3:0/2955291703 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5dc12f6f0 con 0x7fc5dc130130 2023-10-03T10:10:03.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.861+0000 7fc5ca7fc700 1 -- 172.21.15.3:0/2955291703 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fc5d4003040 con 0x7fc5dc130130 2023-10-03T10:10:03.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.861+0000 7fc5ca7fc700 1 -- 172.21.15.3:0/2955291703 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fc5d403cce0 con 0x7fc5dc130130 2023-10-03T10:10:03.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.864+0000 7fc5f231ab80 1 Processor -- start 2023-10-03T10:10:03.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.864+0000 7fc5f231ab80 1 -- start start 2023-10-03T10:10:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.865+0000 7fc5f231ab80 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56010a381170 0x56010a381550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.865+0000 7fc5f231ab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56010a381a40 0x56010a3ab0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.865+0000 7fc5f231ab80 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56010a3ab5b0 0x56010a3ad9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.865+0000 7fc5f231ab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010a34b610 con 0x56010a381a40 2023-10-03T10:10:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.865+0000 7fc5f231ab80 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x56010a2c1db0 con 0x56010a3ab5b0 2023-10-03T10:10:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.865+0000 7fc5f231ab80 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x56010a2d7030 con 0x56010a381170 2023-10-03T10:10:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.865+0000 7fc5d9d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56010a381a40 0x56010a3ab0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.865+0000 7fc5dad9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56010a3ab5b0 0x56010a3ad9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.865+0000 7fc5d9d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56010a381a40 0x56010a3ab0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47390/0 (socket says 172.21.15.3:47390) 2023-10-03T10:10:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.865+0000 7fc5d9d9b700 1 -- 172.21.15.3:0/2869524509 learned_addr learned my addr 172.21.15.3:0/2869524509 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.865+0000 7fc5da59c700 1 --2- 172.21.15.3:0/2869524509 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56010a381170 0x56010a381550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.865+0000 7fc5dad9d700 1 -- 172.21.15.3:0/2869524509 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56010a381170 msgr2=0x56010a381550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.865+0000 7fc5dad9d700 1 --2- 172.21.15.3:0/2869524509 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56010a381170 0x56010a381550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.865+0000 7fc5dad9d700 1 -- 172.21.15.3:0/2869524509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56010a381a40 msgr2=0x56010a3ab0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.865+0000 7fc5dad9d700 1 --2- 172.21.15.3:0/2869524509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56010a381a40 0x56010a3ab0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.865+0000 7fc5dad9d700 1 -- 172.21.15.3:0/2869524509 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56010a172710 con 0x56010a3ab5b0 2023-10-03T10:10:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.865+0000 7fc5d9d9b700 1 --2- 172.21.15.3:0/2869524509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56010a381a40 0x56010a3ab0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.865+0000 7fc5da59c700 1 --2- 172.21.15.3:0/2869524509 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56010a381170 0x56010a381550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.865+0000 7fc5dad9d700 1 --2- 172.21.15.3:0/2869524509 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56010a3ab5b0 0x56010a3ad9a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fc5d403b4d0 tx=0x7fc5d401d000).ready entity=mon.1 client_cookie=8559917961574c4d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.865+0000 7fc5bb7fe700 1 -- 172.21.15.3:0/2869524509 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4011c90 con 0x56010a3ab5b0 2023-10-03T10:10:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.866+0000 7fc5bb7fe700 1 -- 172.21.15.3:0/2869524509 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d40263c0 con 0x56010a3ab5b0 2023-10-03T10:10:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.866+0000 7fc5bb7fe700 1 -- 172.21.15.3:0/2869524509 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4025e70 con 0x56010a3ab5b0 2023-10-03T10:10:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.866+0000 7fc5f231ab80 1 -- 172.21.15.3:0/2869524509 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56010a3ab5b0 msgr2=0x56010a3ad9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.866+0000 7fc5f231ab80 1 --2- 172.21.15.3:0/2869524509 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56010a3ab5b0 0x56010a3ad9a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fc5d403b4d0 tx=0x7fc5d401d000).stop 2023-10-03T10:10:03.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.866+0000 7fc5f231ab80 1 -- 172.21.15.3:0/2869524509 shutdown_connections 2023-10-03T10:10:03.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.866+0000 7fc5f231ab80 1 --2- 172.21.15.3:0/2869524509 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56010a3ab5b0 0x56010a3ad9a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.866+0000 7fc5f231ab80 1 --2- 172.21.15.3:0/2869524509 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56010a381170 0x56010a381550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.866+0000 7fc5f231ab80 1 --2- 172.21.15.3:0/2869524509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56010a381a40 0x56010a3ab0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.866+0000 7fc5f231ab80 1 -- 172.21.15.3:0/2869524509 >> 172.21.15.3:0/2869524509 conn(0x56010a1ef940 msgr2=0x56010a3a1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:03.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.866+0000 7fc5f231ab80 1 -- 172.21.15.3:0/2869524509 shutdown_connections 2023-10-03T10:10:03.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.866+0000 7fc5f231ab80 1 -- 172.21.15.3:0/2869524509 wait complete. 2023-10-03T10:10:03.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.867+0000 7fc5f231ab80 1 -- start start 2023-10-03T10:10:03.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.867+0000 7fc5f231ab80 1 Processor -- start 2023-10-03T10:10:03.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.868+0000 7fc5f231ab80 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56010a381170 0x56010a3b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.868+0000 7fc5f231ab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56010a381a40 0x56010a3b1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.868+0000 7fc5f231ab80 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56010a3ab5b0 0x56010a3a6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.868+0000 7fc5f231ab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010a2d7170 con 0x56010a381a40 2023-10-03T10:10:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.868+0000 7fc5f231ab80 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x56010a2d73f0 con 0x56010a3ab5b0 2023-10-03T10:10:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.868+0000 7fc5f231ab80 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x56010a2a2920 con 0x56010a381170 2023-10-03T10:10:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.868+0000 7fc5f231ab80 10 client.0 ll_register_callbacks cb 0x56010a277ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-03T10:10:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.868+0000 7fc5d9d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56010a381a40 0x56010a3b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.868+0000 7fc5da59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56010a381170 0x56010a3b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.868+0000 7fc5d9d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56010a381a40 0x56010a3b1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47398/0 (socket says 172.21.15.3:47398) 2023-10-03T10:10:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.868+0000 7fc5d9d9b700 1 -- 172.21.15.3:0/1413643131 learned_addr learned my addr 172.21.15.3:0/1413643131 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.868+0000 7fc5dad9d700 1 --2- 172.21.15.3:0/1413643131 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56010a3ab5b0 0x56010a3a6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.868+0000 7fc5da59c700 1 -- 172.21.15.3:0/1413643131 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56010a3ab5b0 msgr2=0x56010a3a6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.868+0000 7fc5da59c700 1 --2- 172.21.15.3:0/1413643131 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56010a3ab5b0 0x56010a3a6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.868+0000 7fc5da59c700 1 -- 172.21.15.3:0/1413643131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56010a381a40 msgr2=0x56010a3b1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.868+0000 7fc5da59c700 1 --2- 172.21.15.3:0/1413643131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56010a381a40 0x56010a3b1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.868+0000 7fc5da59c700 1 -- 172.21.15.3:0/1413643131 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5dc12f6f0 con 0x56010a381170 2023-10-03T10:10:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.868+0000 7fc5dad9d700 1 --2- 172.21.15.3:0/1413643131 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56010a3ab5b0 0x56010a3a6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.868+0000 7fc5d9d9b700 1 --2- 172.21.15.3:0/1413643131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56010a381a40 0x56010a3b1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.868+0000 7fc5da59c700 1 --2- 172.21.15.3:0/1413643131 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56010a381170 0x56010a3b0b70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fc5cc000c00 tx=0x7fc5cc009040).ready entity=mon.2 client_cookie=83430fa3c26581f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.868+0000 7fc5b8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.3:3302/0 2023-10-03T10:10:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.868+0000 7fc5b8ff9700 1 -- 172.21.15.3:0/1413643131 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc016490 con 0x56010a381170 2023-10-03T10:10:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.869+0000 7fc5b8ff9700 1 -- 172.21.15.3:0/1413643131 <== mon.2 v2:172.21.15.3:3302/0 2 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fc5cc0165f0 con 0x56010a381170 2023-10-03T10:10:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.869+0000 7fc5f231ab80 1 -- 172.21.15.3:0/1413643131 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56010a379f20 con 0x56010a381170 2023-10-03T10:10:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.869+0000 7fc5b8ff9700 1 -- 172.21.15.3:0/1413643131 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5cc0169c0 con 0x56010a381170 2023-10-03T10:10:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.869+0000 7fc5f231ab80 10 client.4493 fetch_fsmap learned FSMap version 6 2023-10-03T10:10:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.869+0000 7fc5f231ab80 1 -- 172.21.15.3:0/1413643131 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x56010a2a9b00 con 0x56010a381170 2023-10-03T10:10:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.869+0000 7fc5b8ff9700 1 -- 172.21.15.3:0/1413643131 <== mon.2 v2:172.21.15.3:3302/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56010a379f20 con 0x56010a381170 2023-10-03T10:10:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.869+0000 7fc5b8ff9700 1 -- 172.21.15.3:0/1413643131 <== mon.2 v2:172.21.15.3:3302/0 5 ==== fsmap(e 6) v1 ==== 1430+0+0 (secure 0 0 0) 0x7fc5cc016b20 con 0x56010a381170 2023-10-03T10:10:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.869+0000 7fc5f231ab80 10 client.4493 fetch_fsmap finished waiting for FSMap version 6 2023-10-03T10:10:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.869+0000 7fc5f231ab80 10 client.4493 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-03T10:10:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.869+0000 7fc5f231ab80 20 client.4493 populate_metadata read hostname 'smithi003' 2023-10-03T10:10:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.869+0000 7fc5f231ab80 4 client.4493 mds_command: new command op to 4186 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-03T10:10:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.869+0000 7fc5f231ab80 1 -- 172.21.15.3:0/1413643131 --> v1:172.21.15.3:6817/2582564779 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56010a34f250 con 0x56010a3a7930 2023-10-03T10:10:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.869+0000 7fc5b8ff9700 10 client.4493 ms_handle_connect on v1:172.21.15.3:6817/2582564779 2023-10-03T10:10:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.870+0000 7fc5b8ff9700 1 -- 172.21.15.3:0/1413643131 <== mds.0 v1:172.21.15.3:6817/2582564779 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (unknown 0 0 3608088055) 0x56010a34f250 con 0x56010a3a7930 2023-10-03T10:10:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.870+0000 7fc5b8ff9700 10 client.4493 handle_command_reply: tid=0 2023-10-03T10:10:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.870+0000 7fc5f231ab80 1 client.4493 shutdown 2023-10-03T10:10:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.870+0000 7fc5f231ab80 1 -- 172.21.15.3:0/1413643131 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56010a381170 msgr2=0x56010a3b0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:03.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.870+0000 7fc5f231ab80 1 --2- 172.21.15.3:0/1413643131 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56010a381170 0x56010a3b0b70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fc5cc000c00 tx=0x7fc5cc009040).stop 2023-10-03T10:10:03.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.870+0000 7fc5f231ab80 1 -- 172.21.15.3:0/1413643131 shutdown_connections 2023-10-03T10:10:03.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.870+0000 7fc5f231ab80 1 --2- 172.21.15.3:0/1413643131 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56010a3ab5b0 0x56010a3a6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.870+0000 7fc5f231ab80 1 --2- 172.21.15.3:0/1413643131 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56010a381170 0x56010a3b0b70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.870+0000 7fc5f231ab80 1 --2- 172.21.15.3:0/1413643131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56010a381a40 0x56010a3b1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.870+0000 7fc5b8ff9700 0 client.4493 ms_handle_reset on v1:172.21.15.3:6817/2582564779 2023-10-03T10:10:03.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.870+0000 7fc5f231ab80 1 -- 172.21.15.3:0/1413643131 >> 172.21.15.3:0/1413643131 conn(0x56010a1ef940 msgr2=0x56010a3a1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:03.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.871+0000 7fc5f231ab80 1 -- 172.21.15.3:0/1413643131 shutdown_connections 2023-10-03T10:10:03.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.871+0000 7fc5f231ab80 1 -- 172.21.15.3:0/1413643131 wait complete. 2023-10-03T10:10:03.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.871+0000 7fc5f231ab80 20 client.4493 trim_cache size 0 max 16384 2023-10-03T10:10:03.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.888+0000 7fc5f231ab80 1 Processor -- start 2023-10-03T10:10:03.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.888+0000 7fc5f231ab80 1 -- start start 2023-10-03T10:10:03.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.888+0000 7fc5f231ab80 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56010a381170 0x56010a385650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:03.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.888+0000 7fc5f231ab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56010a381a40 0x56010a3a9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:03.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.888+0000 7fc5f231ab80 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56010a3ab5b0 0x56010a3bcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:03.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.888+0000 7fc5f231ab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010a39e190 con 0x56010a381a40 2023-10-03T10:10:03.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.888+0000 7fc5f231ab80 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x56010a383630 con 0x56010a381170 2023-10-03T10:10:03.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.888+0000 7fc5f231ab80 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x56010a3a2bc0 con 0x56010a3ab5b0 2023-10-03T10:10:03.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.888+0000 7fc5d9d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56010a381a40 0x56010a3a9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:03.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.888+0000 7fc5dad9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56010a3ab5b0 0x56010a3bcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:03.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.889+0000 7fc5da59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56010a381170 0x56010a385650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:03.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.889+0000 7fc5dad9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56010a3ab5b0 0x56010a3bcf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:60280/0 (socket says 172.21.15.3:60280) 2023-10-03T10:10:03.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.889+0000 7fc5d9d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56010a381a40 0x56010a3a9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47410/0 (socket says 172.21.15.3:47410) 2023-10-03T10:10:03.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.889+0000 7fc5dad9d700 1 -- 172.21.15.3:0/2128709326 learned_addr learned my addr 172.21.15.3:0/2128709326 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:03.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.889+0000 7fc5dad9d700 1 -- 172.21.15.3:0/2128709326 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56010a381170 msgr2=0x56010a385650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:03.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.889+0000 7fc5dad9d700 1 --2- 172.21.15.3:0/2128709326 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56010a381170 0x56010a385650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.889+0000 7fc5dad9d700 1 -- 172.21.15.3:0/2128709326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56010a381a40 msgr2=0x56010a3a9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.889+0000 7fc5dad9d700 1 --2- 172.21.15.3:0/2128709326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56010a381a40 0x56010a3a9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.889+0000 7fc5dad9d700 1 -- 172.21.15.3:0/2128709326 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56010a2cff70 con 0x56010a3ab5b0 2023-10-03T10:10:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.889+0000 7fc5d9d9b700 1 --2- 172.21.15.3:0/2128709326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56010a381a40 0x56010a3a9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.889+0000 7fc5dad9d700 1 --2- 172.21.15.3:0/2128709326 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56010a3ab5b0 0x56010a3bcf50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fc5d403b3b0 tx=0x7fc5d4035040).ready entity=mon.2 client_cookie=1ee9f61ba891d975 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.889+0000 7fc5bbfff700 1 -- 172.21.15.3:0/2128709326 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4041030 con 0x56010a3ab5b0 2023-10-03T10:10:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.889+0000 7fc5bbfff700 1 -- 172.21.15.3:0/2128709326 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d4042040 con 0x56010a3ab5b0 2023-10-03T10:10:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.889+0000 7fc5bbfff700 1 -- 172.21.15.3:0/2128709326 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4026d90 con 0x56010a3ab5b0 2023-10-03T10:10:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.890+0000 7fc5f231ab80 1 -- 172.21.15.3:0/2128709326 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56010a3ab5b0 msgr2=0x56010a3bcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.890+0000 7fc5f231ab80 1 --2- 172.21.15.3:0/2128709326 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56010a3ab5b0 0x56010a3bcf50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fc5d403b3b0 tx=0x7fc5d4035040).stop 2023-10-03T10:10:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.890+0000 7fc5f231ab80 1 -- 172.21.15.3:0/2128709326 shutdown_connections 2023-10-03T10:10:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.890+0000 7fc5f231ab80 1 --2- 172.21.15.3:0/2128709326 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56010a3ab5b0 0x56010a3bcf50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.890+0000 7fc5f231ab80 1 --2- 172.21.15.3:0/2128709326 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56010a381170 0x56010a385650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.890+0000 7fc5f231ab80 1 --2- 172.21.15.3:0/2128709326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56010a381a40 0x56010a3a9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.890+0000 7fc5f231ab80 1 -- 172.21.15.3:0/2128709326 >> 172.21.15.3:0/2128709326 conn(0x56010a1ef940 msgr2=0x56010a384f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:03.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.890+0000 7fc5f231ab80 1 -- 172.21.15.3:0/2128709326 shutdown_connections 2023-10-03T10:10:03.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.890+0000 7fc5f231ab80 1 -- 172.21.15.3:0/2128709326 wait complete. 2023-10-03T10:10:03.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.890+0000 7fc5f231ab80 1 -- start start 2023-10-03T10:10:03.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.890+0000 7fc5f231ab80 1 Processor -- start 2023-10-03T10:10:03.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.891+0000 7fc5f231ab80 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56010a381170 0x56010a3a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:03.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.891+0000 7fc5f231ab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56010a381a40 0x56010a3a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:03.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.891+0000 7fc5f231ab80 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56010a3ab5b0 0x56010a405520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:03.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.891+0000 7fc5f231ab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010a3e9d10 con 0x56010a381a40 2023-10-03T10:10:03.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.891+0000 7fc5f231ab80 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x56010a382f60 con 0x56010a381170 2023-10-03T10:10:03.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.891+0000 7fc5f231ab80 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x56010a3a7d10 con 0x56010a3ab5b0 2023-10-03T10:10:03.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.891+0000 7fc5f231ab80 10 client.0 ll_register_callbacks cb 0x56010a277ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-03T10:10:03.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.891+0000 7fc5d9d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56010a381a40 0x56010a3a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:03.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.891+0000 7fc5dad9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56010a3ab5b0 0x56010a405520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:03.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.891+0000 7fc5d9d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56010a381a40 0x56010a3a58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47412/0 (socket says 172.21.15.3:47412) 2023-10-03T10:10:03.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.891+0000 7fc5da59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56010a381170 0x56010a3a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:03.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.891+0000 7fc5d9d9b700 1 -- 172.21.15.3:0/1809394072 learned_addr learned my addr 172.21.15.3:0/1809394072 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:03.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.891+0000 7fc5da59c700 1 -- 172.21.15.3:0/1809394072 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56010a3ab5b0 msgr2=0x56010a405520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:03.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.891+0000 7fc5da59c700 1 --2- 172.21.15.3:0/1809394072 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56010a3ab5b0 0x56010a405520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.891+0000 7fc5da59c700 1 -- 172.21.15.3:0/1809394072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56010a381a40 msgr2=0x56010a3a58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:03.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.891+0000 7fc5da59c700 1 --2- 172.21.15.3:0/1809394072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56010a381a40 0x56010a3a58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.891+0000 7fc5da59c700 1 -- 172.21.15.3:0/1809394072 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d0014880 con 0x56010a381170 2023-10-03T10:10:03.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.892+0000 7fc5d9d9b700 1 --2- 172.21.15.3:0/1809394072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56010a381a40 0x56010a3a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:03.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.892+0000 7fc5da59c700 1 --2- 172.21.15.3:0/1809394072 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56010a381170 0x56010a3a53f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fc5cc001740 tx=0x7fc5cc020000).ready entity=mon.1 client_cookie=c2cd4b225ccd7622 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:03.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.892+0000 7fc5b8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.3:3301/0 2023-10-03T10:10:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.892+0000 7fc5b8ff9700 1 -- 172.21.15.3:0/1809394072 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc0169c0 con 0x56010a381170 2023-10-03T10:10:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.892+0000 7fc5b8ff9700 1 -- 172.21.15.3:0/1809394072 <== mon.1 v2:172.21.15.3:3301/0 2 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fc5cc016b20 con 0x56010a381170 2023-10-03T10:10:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.892+0000 7fc5f231ab80 1 -- 172.21.15.3:0/1809394072 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5d40263c0 con 0x56010a381170 2023-10-03T10:10:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.892+0000 7fc5b8ff9700 1 -- 172.21.15.3:0/1809394072 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5cc007240 con 0x56010a381170 2023-10-03T10:10:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.892+0000 7fc5f231ab80 10 client.4501 fetch_fsmap learned FSMap version 6 2023-10-03T10:10:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.892+0000 7fc5f231ab80 1 -- 172.21.15.3:0/1809394072 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x56010a172710 con 0x56010a381170 2023-10-03T10:10:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.892+0000 7fc5b8ff9700 1 -- 172.21.15.3:0/1809394072 <== mon.1 v2:172.21.15.3:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d40263c0 con 0x56010a381170 2023-10-03T10:10:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.892+0000 7fc5b8ff9700 1 -- 172.21.15.3:0/1809394072 <== mon.1 v2:172.21.15.3:3301/0 5 ==== fsmap(e 6) v1 ==== 1430+0+0 (secure 0 0 0) 0x7fc5cc033030 con 0x56010a381170 2023-10-03T10:10:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.892+0000 7fc5f231ab80 10 client.4501 fetch_fsmap finished waiting for FSMap version 6 2023-10-03T10:10:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.892+0000 7fc5f231ab80 10 client.4501 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-03T10:10:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.892+0000 7fc5f231ab80 20 client.4501 populate_metadata read hostname 'smithi003' 2023-10-03T10:10:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.892+0000 7fc5f231ab80 4 client.4501 mds_command: new command op to 4186 tid=0[{"prefix": "status"}] 2023-10-03T10:10:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.892+0000 7fc5f231ab80 1 -- 172.21.15.3:0/1809394072 --> v1:172.21.15.3:6817/2582564779 -- command(tid 0: {"prefix": "status"}) v1 -- 0x56010a13fef0 con 0x56010a405f40 2023-10-03T10:10:03.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.893+0000 7fc5b8ff9700 10 client.4501 ms_handle_connect on v1:172.21.15.3:6817/2582564779 2023-10-03T10:10:03.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.893+0000 7fc5b8ff9700 1 -- 172.21.15.3:0/1809394072 <== mds.0 v1:172.21.15.3:6817/2582564779 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (unknown 0 0 2427970782) 0x56010a13fef0 con 0x56010a405f40 2023-10-03T10:10:03.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.893+0000 7fc5b8ff9700 10 client.4501 handle_command_reply: tid=0 2023-10-03T10:10:03.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.893+0000 7fc5f231ab80 1 client.4501 shutdown 2023-10-03T10:10:03.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.893+0000 7fc5f231ab80 1 -- 172.21.15.3:0/1809394072 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56010a381170 msgr2=0x56010a3a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:03.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.893+0000 7fc5f231ab80 1 --2- 172.21.15.3:0/1809394072 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56010a381170 0x56010a3a53f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fc5cc001740 tx=0x7fc5cc020000).stop 2023-10-03T10:10:03.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.893+0000 7fc5f231ab80 1 -- 172.21.15.3:0/1809394072 shutdown_connections 2023-10-03T10:10:03.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.893+0000 7fc5f231ab80 1 --2- 172.21.15.3:0/1809394072 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56010a3ab5b0 0x56010a405520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.893+0000 7fc5f231ab80 1 --2- 172.21.15.3:0/1809394072 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56010a381170 0x56010a3a53f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.893+0000 7fc5f231ab80 1 --2- 172.21.15.3:0/1809394072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56010a381a40 0x56010a3a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.893+0000 7fc5b8ff9700 0 client.4501 ms_handle_reset on v1:172.21.15.3:6817/2582564779 2023-10-03T10:10:03.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.893+0000 7fc5f231ab80 1 -- 172.21.15.3:0/1809394072 >> 172.21.15.3:0/1809394072 conn(0x56010a1ef940 msgr2=0x56010a39e350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:03.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.893+0000 7fc5f231ab80 1 -- 172.21.15.3:0/1809394072 shutdown_connections 2023-10-03T10:10:03.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.893+0000 7fc5f231ab80 1 -- 172.21.15.3:0/1809394072 wait complete. 2023-10-03T10:10:03.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.893+0000 7fc5f231ab80 20 client.4501 trim_cache size 0 max 16384 2023-10-03T10:10:03.900 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-03T10:10:03.900 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "68e08e7d-a212-4fd3-bd42-88e2411aa299", 2023-10-03T10:10:03.900 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2023-10-03T10:10:03.900 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4186, 2023-10-03T10:10:03.900 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-10-03T10:10:03.900 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-10-03T10:10:03.900 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-10-03T10:10:03.900 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 19.745941027000001, 2023-10-03T10:10:03.901 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 6, 2023-10-03T10:10:03.901 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 17, 2023-10-03T10:10:03.901 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 17, 2023-10-03T10:10:03.901 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 24.201817104 2023-10-03T10:10:03.901 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-03T10:10:03.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.894+0000 7fc5e0967700 1 -- 172.21.15.3:0/2955291703 >> v1:172.21.15.3:6816/118296 conn(0x7fc5bc02bb60 legacy=0x7fc5bc02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:03.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.894+0000 7fc5e0967700 1 -- 172.21.15.3:0/2955291703 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5dc130130 msgr2=0x7fc5dc12e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:03.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.894+0000 7fc5e0967700 1 --2- 172.21.15.3:0/2955291703 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5dc130130 0x7fc5dc12e1f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fc5dc09a2e0 tx=0x7fc5dc099b80).stop 2023-10-03T10:10:03.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.894+0000 7fc5e0967700 1 -- 172.21.15.3:0/2955291703 shutdown_connections 2023-10-03T10:10:03.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.894+0000 7fc5e0967700 1 --2- 172.21.15.3:0/2955291703 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5dc130130 0x7fc5dc12e1f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.894+0000 7fc5e0967700 1 --2- 172.21.15.3:0/2955291703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5dc062a90 0x7fc5dc124e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.894+0000 7fc5e0967700 1 --2- 172.21.15.3:0/2955291703 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5dc127ed0 0x7fc5dc12dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:03.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.894+0000 7fc5e0967700 1 -- 172.21.15.3:0/2955291703 >> 172.21.15.3:0/2955291703 conn(0x7fc5dc097460 msgr2=0x7fc5dc098730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:03.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.894+0000 7fc5e0967700 1 -- 172.21.15.3:0/2955291703 shutdown_connections 2023-10-03T10:10:03.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:03.894+0000 7fc5e0967700 1 -- 172.21.15.3:0/2955291703 wait complete. 2023-10-03T10:10:03.907 INFO:teuthology.run_tasks:Running task exec... 2023-10-03T10:10:03.917 INFO:teuthology.task.exec:Executing custom commands... 2023-10-03T10:10:03.917 INFO:teuthology.task.exec:Running commands on role osd.0 host ubuntu@smithi003.front.sepia.ceph.com 2023-10-03T10:10:03.918 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd set-require-min-compat-client pacific' 2023-10-03T10:10:04.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.045+0000 7f2b691da700 1 Processor -- start 2023-10-03T10:10:04.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.046+0000 7f2b691da700 1 -- start start 2023-10-03T10:10:04.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.046+0000 7f2b691da700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2b64066230 0x7f2b64096480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:04.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.046+0000 7f2b691da700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b64130360 0x7f2b64132750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:04.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.046+0000 7f2b691da700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b640648a0 con 0x7f2b64096970 2023-10-03T10:10:04.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.046+0000 7f2b691da700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b64066610 con 0x7f2b64066230 2023-10-03T10:10:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.046+0000 7f2b691da700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6406a530 con 0x7f2b64130360 2023-10-03T10:10:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.046+0000 7f2b6359e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b64130360 0x7f2b64132750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.046+0000 7f2b62d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2b64066230 0x7f2b64096480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.046+0000 7f2b6259c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2b64096970 0x7f2b64096d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47442/0 (socket says 172.21.15.3:47442) 2023-10-03T10:10:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.046+0000 7f2b6359e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b64130360 0x7f2b64132750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:60296/0 (socket says 172.21.15.3:60296) 2023-10-03T10:10:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.046+0000 7f2b6259c700 1 -- 172.21.15.3:0/4213485276 learned_addr learned my addr 172.21.15.3:0/4213485276 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:10:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.047+0000 7f2b62d9d700 1 -- 172.21.15.3:0/4213485276 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b64130360 msgr2=0x7f2b64132750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.047+0000 7f2b62d9d700 1 --2- 172.21.15.3:0/4213485276 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b64130360 0x7f2b64132750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.047+0000 7f2b62d9d700 1 -- 172.21.15.3:0/4213485276 >> v1:172.21.15.3:6789/0 conn(0x7f2b64096970 legacy=0x7f2b64096d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.047+0000 7f2b61d9b700 1 -- 172.21.15.3:0/4213485276 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3532065043 0 0) 0x7f2b640648a0 con 0x7f2b64096970 2023-10-03T10:10:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.047+0000 7f2b62d9d700 1 -- 172.21.15.3:0/4213485276 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b64064f20 con 0x7f2b64066230 2023-10-03T10:10:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.047+0000 7f2b62d9d700 1 --2- 172.21.15.3:0/4213485276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b64066230 0x7f2b64096480 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f2b54009760 tx=0x7f2b54009060).ready entity=mon.0 client_cookie=6c8beff2fc3fb52 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.047+0000 7f2b61d9b700 1 -- 172.21.15.3:0/4213485276 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b540090d0 con 0x7f2b64066230 2023-10-03T10:10:04.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.047+0000 7f2b61d9b700 1 -- 172.21.15.3:0/4213485276 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5400f510 con 0x7f2b64066230 2023-10-03T10:10:04.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.047+0000 7f2b61d9b700 1 -- 172.21.15.3:0/4213485276 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5400f890 con 0x7f2b64066230 2023-10-03T10:10:04.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.047+0000 7f2b691da700 1 -- 172.21.15.3:0/4213485276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b64066230 msgr2=0x7f2b64096480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:04.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.047+0000 7f2b691da700 1 --2- 172.21.15.3:0/4213485276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b64066230 0x7f2b64096480 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f2b54009760 tx=0x7f2b54009060).stop 2023-10-03T10:10:04.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.048+0000 7f2b691da700 1 -- 172.21.15.3:0/4213485276 shutdown_connections 2023-10-03T10:10:04.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.048+0000 7f2b691da700 1 --2- 172.21.15.3:0/4213485276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b64066230 0x7f2b64096480 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:04.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.048+0000 7f2b691da700 1 --2- 172.21.15.3:0/4213485276 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b64130360 0x7f2b64132750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:04.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.048+0000 7f2b691da700 1 -- 172.21.15.3:0/4213485276 >> 172.21.15.3:0/4213485276 conn(0x7f2b6409bae0 msgr2=0x7f2b6409bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:04.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.048+0000 7f2b691da700 1 -- 172.21.15.3:0/4213485276 shutdown_connections 2023-10-03T10:10:04.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.048+0000 7f2b691da700 1 -- 172.21.15.3:0/4213485276 wait complete. 2023-10-03T10:10:04.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.049+0000 7f2b691da700 1 Processor -- start 2023-10-03T10:10:04.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.049+0000 7f2b691da700 1 -- start start 2023-10-03T10:10:04.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.049+0000 7f2b691da700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b64066230 0x7f2b6413a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:04.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.049+0000 7f2b691da700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b64096970 0x7f2b6413ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:04.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.049+0000 7f2b691da700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b64130360 0x7f2b64115d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:04.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.049+0000 7f2b691da700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b64119830 con 0x7f2b64130360 2023-10-03T10:10:04.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.049+0000 7f2b691da700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b64130740 con 0x7f2b64066230 2023-10-03T10:10:04.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.049+0000 7f2b691da700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b64065150 con 0x7f2b64096970 2023-10-03T10:10:04.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.049+0000 7f2b6259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b64096970 0x7f2b6413ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:04.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.049+0000 7f2b6259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b64096970 0x7f2b6413ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:60300/0 (socket says 172.21.15.3:60300) 2023-10-03T10:10:04.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.049+0000 7f2b6259c700 1 -- 172.21.15.3:0/1097217567 learned_addr learned my addr 172.21.15.3:0/1097217567 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:04.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.049+0000 7f2b6359e700 1 --2- 172.21.15.3:0/1097217567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b64130360 0x7f2b64115d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:04.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.049+0000 7f2b62d9d700 1 --2- 172.21.15.3:0/1097217567 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b64066230 0x7f2b6413a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:04.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.050+0000 7f2b6359e700 1 -- 172.21.15.3:0/1097217567 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b64096970 msgr2=0x7f2b6413ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:04.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.050+0000 7f2b6359e700 1 --2- 172.21.15.3:0/1097217567 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b64096970 0x7f2b6413ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:04.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.050+0000 7f2b6359e700 1 -- 172.21.15.3:0/1097217567 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b64066230 msgr2=0x7f2b6413a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:04.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.050+0000 7f2b6359e700 1 --2- 172.21.15.3:0/1097217567 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b64066230 0x7f2b6413a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:04.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.050+0000 7f2b6359e700 1 -- 172.21.15.3:0/1097217567 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b540092a0 con 0x7f2b64130360 2023-10-03T10:10:04.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.050+0000 7f2b6259c700 1 --2- 172.21.15.3:0/1097217567 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b64096970 0x7f2b6413ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:04.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.050+0000 7f2b62d9d700 1 --2- 172.21.15.3:0/1097217567 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b64066230 0x7f2b6413a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:04.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.050+0000 7f2b6359e700 1 --2- 172.21.15.3:0/1097217567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b64130360 0x7f2b64115d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f2b5c009e20 tx=0x7f2b5c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:04.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.050+0000 7f2b4affd700 1 -- 172.21.15.3:0/1097217567 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c00caa0 con 0x7f2b64130360 2023-10-03T10:10:04.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.050+0000 7f2b4affd700 1 -- 172.21.15.3:0/1097217567 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5c00f040 con 0x7f2b64130360 2023-10-03T10:10:04.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.050+0000 7f2b691da700 1 -- 172.21.15.3:0/1097217567 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b640998f0 con 0x7f2b64130360 2023-10-03T10:10:04.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.050+0000 7f2b4affd700 1 -- 172.21.15.3:0/1097217567 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c00ce10 con 0x7f2b64130360 2023-10-03T10:10:04.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.050+0000 7f2b691da700 1 -- 172.21.15.3:0/1097217567 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b641182d0 con 0x7f2b64130360 2023-10-03T10:10:04.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.051+0000 7f2b4affd700 1 -- 172.21.15.3:0/1097217567 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f2b5c013610 con 0x7f2b64130360 2023-10-03T10:10:04.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.051+0000 7f2b4affd700 1 -- 172.21.15.3:0/1097217567 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f2b5c03b7a0 con 0x7f2b64130360 2023-10-03T10:10:04.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.051+0000 7f2b691da700 1 -- 172.21.15.3:0/1097217567 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b641194b0 con 0x7f2b64130360 2023-10-03T10:10:04.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.056+0000 7f2b4affd700 1 -- 172.21.15.3:0/1097217567 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f2b5c01d460 con 0x7f2b64130360 2023-10-03T10:10:04.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.240+0000 7f2b691da700 1 -- 172.21.15.3:0/1097217567 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd set-require-min-compat-client", "version": "pacific"} v 0) v1 -- 0x7f2b64059d30 con 0x7f2b64130360 2023-10-03T10:10:04.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.614+0000 7f2b4affd700 1 -- 172.21.15.3:0/1097217567 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd set-require-min-compat-client", "version": "pacific"}]=0 set require_min_compat_client to pacific v18) v1 ==== 143+0+0 (secure 0 0 0) 0x7f2b5c0076e0 con 0x7f2b64130360 2023-10-03T10:10:04.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.620+0000 7f2b691da700 1 -- 172.21.15.3:0/1097217567 >> v1:172.21.15.3:6816/118296 conn(0x7f2b5002bb60 legacy=0x7f2b5002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:04.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.620+0000 7f2b691da700 1 -- 172.21.15.3:0/1097217567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b64130360 msgr2=0x7f2b64115d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:04.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.620+0000 7f2b691da700 1 --2- 172.21.15.3:0/1097217567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b64130360 0x7f2b64115d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f2b5c009e20 tx=0x7f2b5c00b040).stop 2023-10-03T10:10:04.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.620+0000 7f2b691da700 1 -- 172.21.15.3:0/1097217567 shutdown_connections 2023-10-03T10:10:04.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.620+0000 7f2b691da700 1 --2- 172.21.15.3:0/1097217567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b64130360 0x7f2b64115d90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:04.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.620+0000 7f2b691da700 1 --2- 172.21.15.3:0/1097217567 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b64066230 0x7f2b6413a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:04.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.620+0000 7f2b691da700 1 --2- 172.21.15.3:0/1097217567 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b64096970 0x7f2b6413ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:04.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.620+0000 7f2b691da700 1 -- 172.21.15.3:0/1097217567 >> 172.21.15.3:0/1097217567 conn(0x7f2b6409bae0 msgr2=0x7f2b64068440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:04.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.620+0000 7f2b691da700 1 -- 172.21.15.3:0/1097217567 shutdown_connections 2023-10-03T10:10:04.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:04.620+0000 7f2b691da700 1 -- 172.21.15.3:0/1097217567 wait complete. 2023-10-03T10:10:04.622 INFO:teuthology.orchestra.run.smithi003.stderr:set require_min_compat_client to pacific 2023-10-03T10:10:04.635 INFO:teuthology.run_tasks:Running task print... 2023-10-03T10:10:04.644 INFO:teuthology.task.print:**** done ceph 2023-10-03T10:10:04.644 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-10-03T10:10:04.768 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-10-03T10:10:04.768 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2023-10-03T10:10:04.768 INFO:tasks.ceph_fuse:client.0 config is {} 2023-10-03T10:10:04.769 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-03T10:10:04.769 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-03T10:10:04.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-03T10:10:04.769 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2023-10-03T10:10:04.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-03T10:10:04.794 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link delete ceph-brx 2023-10-03T10:10:04.872 INFO:teuthology.orchestra.run.smithi022.stderr:Cannot find device "ceph-brx" 2023-10-03T10:10:04.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-03T10:10:04.874 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-10-03T10:10:04.874 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-03T10:10:05.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.014+0000 7fbd7c7ef700 1 Processor -- start 2023-10-03T10:10:05.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.014+0000 7fbd7c7ef700 1 -- start start 2023-10-03T10:10:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.015+0000 7fbd7c7ef700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409aba0 0x7fbd74127980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.015+0000 7fbd7c7ef700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7412cdd0 0x7fbd7412f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.015+0000 7fbd7c7ef700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd74093030 con 0x7fbd74127e70 2023-10-03T10:10:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.015+0000 7fbd7c7ef700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd740938b0 con 0x7fbd7412cdd0 2023-10-03T10:10:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.015+0000 7fbd7c7ef700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd740952f0 con 0x7fbd7409aba0 2023-10-03T10:10:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.015+0000 7fbd7a58b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409aba0 0x7fbd74127980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.015+0000 7fbd7ad8c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7412cdd0 0x7fbd7412f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.015+0000 7fbd7a58b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409aba0 0x7fbd74127980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:60302/0 (socket says 172.21.15.3:60302) 2023-10-03T10:10:05.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.015+0000 7fbd79d8a700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbd74127e70 0x7fbd74128250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47444/0 (socket says 172.21.15.3:47444) 2023-10-03T10:10:05.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.015+0000 7fbd7ad8c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7412cdd0 0x7fbd7412f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53834/0 (socket says 172.21.15.3:53834) 2023-10-03T10:10:05.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.015+0000 7fbd7a58b700 1 -- 172.21.15.3:0/3798889505 learned_addr learned my addr 172.21.15.3:0/3798889505 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:05.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.015+0000 7fbd7a58b700 1 -- 172.21.15.3:0/3798889505 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7412cdd0 msgr2=0x7fbd7412f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:05.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.015+0000 7fbd79589700 1 -- 172.21.15.3:0/3798889505 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2147423313 0 0) 0x7fbd74093030 con 0x7fbd74127e70 2023-10-03T10:10:05.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.015+0000 7fbd7a58b700 1 --2- 172.21.15.3:0/3798889505 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7412cdd0 0x7fbd7412f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:05.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.015+0000 7fbd7a58b700 1 -- 172.21.15.3:0/3798889505 >> v1:172.21.15.3:6789/0 conn(0x7fbd74127e70 legacy=0x7fbd74128250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:05.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.015+0000 7fbd7a58b700 1 -- 172.21.15.3:0/3798889505 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd74094430 con 0x7fbd7409aba0 2023-10-03T10:10:05.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.015+0000 7fbd7ad8c700 1 --2- 172.21.15.3:0/3798889505 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7412cdd0 0x7fbd7412f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:05.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.015+0000 7fbd7a58b700 1 --2- 172.21.15.3:0/3798889505 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409aba0 0x7fbd74127980 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fbd70009830 tx=0x7fbd700090f0).ready entity=mon.2 client_cookie=82a827f3d10f7873 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:05.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.016+0000 7fbd79589700 1 -- 172.21.15.3:0/3798889505 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7000b580 con 0x7fbd7409aba0 2023-10-03T10:10:05.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.016+0000 7fbd79589700 1 -- 172.21.15.3:0/3798889505 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7000b6e0 con 0x7fbd7409aba0 2023-10-03T10:10:05.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.016+0000 7fbd79589700 1 -- 172.21.15.3:0/3798889505 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7000ba80 con 0x7fbd7409aba0 2023-10-03T10:10:05.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.016+0000 7fbd7c7ef700 1 -- 172.21.15.3:0/3798889505 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409aba0 msgr2=0x7fbd74127980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:05.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.016+0000 7fbd7c7ef700 1 --2- 172.21.15.3:0/3798889505 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409aba0 0x7fbd74127980 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fbd70009830 tx=0x7fbd700090f0).stop 2023-10-03T10:10:05.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.016+0000 7fbd7c7ef700 1 -- 172.21.15.3:0/3798889505 shutdown_connections 2023-10-03T10:10:05.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.016+0000 7fbd7c7ef700 1 --2- 172.21.15.3:0/3798889505 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7412cdd0 0x7fbd7412f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:05.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.016+0000 7fbd7c7ef700 1 --2- 172.21.15.3:0/3798889505 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409aba0 0x7fbd74127980 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:05.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.016+0000 7fbd7c7ef700 1 -- 172.21.15.3:0/3798889505 >> 172.21.15.3:0/3798889505 conn(0x7fbd74063190 msgr2=0x7fbd7409a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:05.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.016+0000 7fbd7c7ef700 1 -- 172.21.15.3:0/3798889505 shutdown_connections 2023-10-03T10:10:05.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.016+0000 7fbd7c7ef700 1 -- 172.21.15.3:0/3798889505 wait complete. 2023-10-03T10:10:05.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.017+0000 7fbd7c7ef700 1 Processor -- start 2023-10-03T10:10:05.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.017+0000 7fbd7c7ef700 1 -- start start 2023-10-03T10:10:05.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.018+0000 7fbd7c7ef700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd7409aba0 0x7fbd74126350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:05.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.018+0000 7fbd7c7ef700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd74127e70 0x7fbd74126840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:05.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.018+0000 7fbd7c7ef700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7412cdd0 0x7fbd74126d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:05.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.018+0000 7fbd7c7ef700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd741143a0 con 0x7fbd7409aba0 2023-10-03T10:10:05.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.018+0000 7fbd7c7ef700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd74068ad0 con 0x7fbd7412cdd0 2023-10-03T10:10:05.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.018+0000 7fbd7c7ef700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd74093b70 con 0x7fbd74127e70 2023-10-03T10:10:05.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.018+0000 7fbd79d8a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd74127e70 0x7fbd74126840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:05.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.018+0000 7fbd7ad8c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7412cdd0 0x7fbd74126d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:05.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.018+0000 7fbd7a58b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd7409aba0 0x7fbd74126350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:05.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.018+0000 7fbd79d8a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd74127e70 0x7fbd74126840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:60306/0 (socket says 172.21.15.3:60306) 2023-10-03T10:10:05.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.018+0000 7fbd7ad8c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7412cdd0 0x7fbd74126d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53850/0 (socket says 172.21.15.3:53850) 2023-10-03T10:10:05.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.018+0000 7fbd79d8a700 1 -- 172.21.15.3:0/2308062729 learned_addr learned my addr 172.21.15.3:0/2308062729 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:05.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.018+0000 7fbd7a58b700 1 -- 172.21.15.3:0/2308062729 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd74127e70 msgr2=0x7fbd74126840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:05.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.018+0000 7fbd7a58b700 1 --2- 172.21.15.3:0/2308062729 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd74127e70 0x7fbd74126840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:05.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.018+0000 7fbd7a58b700 1 -- 172.21.15.3:0/2308062729 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7412cdd0 msgr2=0x7fbd74126d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:05.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.018+0000 7fbd7a58b700 1 --2- 172.21.15.3:0/2308062729 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7412cdd0 0x7fbd74126d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.018+0000 7fbd7a58b700 1 -- 172.21.15.3:0/2308062729 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6c00bc30 con 0x7fbd7409aba0 2023-10-03T10:10:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.018+0000 7fbd79d8a700 1 --2- 172.21.15.3:0/2308062729 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd74127e70 0x7fbd74126840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.018+0000 7fbd7ad8c700 1 --2- 172.21.15.3:0/2308062729 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7412cdd0 0x7fbd74126d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.018+0000 7fbd7a58b700 1 --2- 172.21.15.3:0/2308062729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd7409aba0 0x7fbd74126350 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fbd70014000 tx=0x7fbd70017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.018+0000 7fbd667fc700 1 -- 172.21.15.3:0/2308062729 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd70021070 con 0x7fbd7409aba0 2023-10-03T10:10:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.019+0000 7fbd667fc700 1 -- 172.21.15.3:0/2308062729 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7000ba60 con 0x7fbd7409aba0 2023-10-03T10:10:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.019+0000 7fbd667fc700 1 -- 172.21.15.3:0/2308062729 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd70006d90 con 0x7fbd7409aba0 2023-10-03T10:10:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.019+0000 7fbd7c7ef700 1 -- 172.21.15.3:0/2308062729 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd70009370 con 0x7fbd7409aba0 2023-10-03T10:10:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.019+0000 7fbd7c7ef700 1 -- 172.21.15.3:0/2308062729 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd74114a10 con 0x7fbd7409aba0 2023-10-03T10:10:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.019+0000 7fbd667fc700 1 -- 172.21.15.3:0/2308062729 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fbd70011400 con 0x7fbd7409aba0 2023-10-03T10:10:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.019+0000 7fbd667fc700 1 -- 172.21.15.3:0/2308062729 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fbd7003fd00 con 0x7fbd7409aba0 2023-10-03T10:10:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.020+0000 7fbd7c7ef700 1 -- 172.21.15.3:0/2308062729 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd74113cf0 con 0x7fbd7409aba0 2023-10-03T10:10:05.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.024+0000 7fbd667fc700 1 -- 172.21.15.3:0/2308062729 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fbd7002a080 con 0x7fbd7409aba0 2023-10-03T10:10:05.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.244+0000 7fbd7c7ef700 1 -- 172.21.15.3:0/2308062729 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbd740697e0 con 0x7fbd7409aba0 2023-10-03T10:10:05.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.244+0000 7fbd667fc700 1 -- 172.21.15.3:0/2308062729 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbd700116b0 con 0x7fbd7409aba0 2023-10-03T10:10:05.245 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-03T10:10:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.246+0000 7fbd7c7ef700 1 -- 172.21.15.3:0/2308062729 >> v1:172.21.15.3:6816/118296 conn(0x7fbd5802bb60 legacy=0x7fbd5802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.246+0000 7fbd7c7ef700 1 -- 172.21.15.3:0/2308062729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd7409aba0 msgr2=0x7fbd74126350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.246+0000 7fbd7c7ef700 1 --2- 172.21.15.3:0/2308062729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd7409aba0 0x7fbd74126350 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fbd70014000 tx=0x7fbd70017000).stop 2023-10-03T10:10:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.246+0000 7fbd7c7ef700 1 -- 172.21.15.3:0/2308062729 shutdown_connections 2023-10-03T10:10:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.246+0000 7fbd7c7ef700 1 --2- 172.21.15.3:0/2308062729 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7412cdd0 0x7fbd74126d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.246+0000 7fbd7c7ef700 1 --2- 172.21.15.3:0/2308062729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd7409aba0 0x7fbd74126350 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:05.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.246+0000 7fbd7c7ef700 1 --2- 172.21.15.3:0/2308062729 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd74127e70 0x7fbd74126840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:05.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.246+0000 7fbd7c7ef700 1 -- 172.21.15.3:0/2308062729 >> 172.21.15.3:0/2308062729 conn(0x7fbd74063190 msgr2=0x7fbd74065f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:05.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.246+0000 7fbd7c7ef700 1 -- 172.21.15.3:0/2308062729 shutdown_connections 2023-10-03T10:10:05.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:05.246+0000 7fbd7c7ef700 1 -- 172.21.15.3:0/2308062729 wait complete. 2023-10-03T10:10:05.260 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-03T10:10:05.260 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-03T10:10:05.260 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2023-10-03T10:10:05.260 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-03T10:10:05.260 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-03T10:10:05.260 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-03T10:10:05.260 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-03T10:10:05.260 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-03T10:10:05.260 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-03T10:10:05.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-03T10:10:05.261 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2023-10-03T10:10:05.285 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-03T10:10:05.285 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-03T10:10:05.285 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2023-10-03T10:10:05.285 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-10-03T10:10:05.285 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2023-10-03T10:10:05.285 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-10-03T10:10:05.285 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-03T10:10:05.285 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2023-10-03T10:10:05.285 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2023-10-03T10:10:05.286 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-03T10:10:05.286 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2023-10-03T10:10:05.286 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2023-10-03T10:10:05.286 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-03T10:10:05.286 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2023-10-03T10:10:05.286 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2023-10-03T10:10:05.286 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-03T10:10:05.286 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42443sec preferred_lft 42443sec 2023-10-03T10:10:05.286 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2023-10-03T10:10:05.286 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-10-03T10:10:05.286 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-03T10:10:05.287 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2023-10-03T10:10:05.287 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2023-10-03T10:10:05.287 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-03T10:10:05.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-03T10:10:05.287 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-10-03T10:10:05.287 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-10-03T10:10:05.287 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add name ceph-brx type bridge 2023-10-03T10:10:05.288 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr flush dev ceph-brx 2023-10-03T10:10:05.288 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set ceph-brx up 2023-10-03T10:10:05.288 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-03T10:10:05.288 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-10-03T10:10:05.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-03T10:10:05.463 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-03T10:10:05.500 INFO:teuthology.orchestra.run.smithi022.stdout:1 2023-10-03T10:10:05.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-03T10:10:05.501 DEBUG:teuthology.orchestra.run.smithi022:> route 2023-10-03T10:10:05.587 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2023-10-03T10:10:05.587 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-03T10:10:05.587 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-03T10:10:05.587 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-03T10:10:05.587 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-03T10:10:05.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-03T10:10:05.588 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-10-03T10:10:05.588 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-10-03T10:10:05.588 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-03T10:10:05.588 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-03T10:10:05.588 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-03T10:10:05.588 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-10-03T10:10:05.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-03T10:10:05.828 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2023-10-03T10:10:05.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-03T10:10:05.850 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list-id 2023-10-03T10:10:05.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-03T10:10:05.913 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-10-03T10:10:05.913 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-10-03T10:10:05.913 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-03T10:10:05.913 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-03T10:10:05.913 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-10-03T10:10:06.024 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-03T10:10:06.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-03T10:10:06.024 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-10-03T10:10:06.024 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-10-03T10:10:06.024 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-03T10:10:06.024 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-03T10:10:06.024 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-03T10:10:06.024 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-03T10:10:06.025 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-03T10:10:06.025 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-10-03T10:10:06.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-03T10:10:06.237 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-10-03T10:10:06.237 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-10-03T10:10:06.237 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2023-10-03T10:10:06.238 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set dev brx.0 master ceph-brx 2023-10-03T10:10:06.238 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-10-03T10:10:06.349 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-03T10:10:06.349 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-03T10:10:06.349 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-03T10:10:06.369 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-03T10:10:06.369 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-03T10:10:06.370 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-03T10:10:06.431 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2023-10-03T10:10:06.507 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2023-10-03T10:10:06.571 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2023-10-03T10:10:06.571 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2023-10-03T10:10:06.571 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2023-10-03T10:10:06.571 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2023-10-03T10:10:06.571 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2023-10-03T10:10:06.571 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2023-10-03T10:10:06.572 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2023-10-03T10:10:06.572 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2023-10-03T10:10:06.572 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup/systemd 2023-10-03T10:10:06.572 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2023-10-03T10:10:06.572 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2023-10-03T10:10:06.572 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-10-03T10:10:06.572 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup/rdma 2023-10-03T10:10:06.572 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup/memory 2023-10-03T10:10:06.572 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup/freezer 2023-10-03T10:10:06.572 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup/pids 2023-10-03T10:10:06.572 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup/hugetlb 2023-10-03T10:10:06.572 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-10-03T10:10:06.572 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup/cpuset 2023-10-03T10:10:06.573 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup/blkio 2023-10-03T10:10:06.573 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup/perf_event 2023-10-03T10:10:06.573 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup/devices 2023-10-03T10:10:06.573 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2023-10-03T10:10:06.573 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2023-10-03T10:10:06.573 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2023-10-03T10:10:06.573 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/selinux 2023-10-03T10:10:06.573 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2023-10-03T10:10:06.573 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2023-10-03T10:10:06.573 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2023-10-03T10:10:06.573 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2023-10-03T10:10:06.573 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2023-10-03T10:10:06.573 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/nfs/rpc_pipefs 2023-10-03T10:10:06.574 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2023-10-03T10:10:06.574 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2023-10-03T10:10:06.574 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2023-10-03T10:10:06.574 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-03T10:10:06.574 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-03T10:10:06.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-03T10:10:06.575 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2023-10-03T10:10:06.638 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-03T10:10:06.638 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-03T10:10:06.680 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2023-10-03T10:10:06.715 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2023-10-03T10:10:06.775 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2023-10-03T10:10:06.775 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2023-10-03T10:10:06.776 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2023-10-03T10:10:06.776 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2023-10-03T10:10:06.776 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2023-10-03T10:10:06.776 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2023-10-03T10:10:06.776 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2023-10-03T10:10:06.776 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2023-10-03T10:10:06.776 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup/systemd 2023-10-03T10:10:06.776 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2023-10-03T10:10:06.776 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2023-10-03T10:10:06.776 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-10-03T10:10:06.776 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup/rdma 2023-10-03T10:10:06.776 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup/memory 2023-10-03T10:10:06.777 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup/freezer 2023-10-03T10:10:06.777 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup/pids 2023-10-03T10:10:06.777 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup/hugetlb 2023-10-03T10:10:06.777 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-10-03T10:10:06.777 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup/cpuset 2023-10-03T10:10:06.777 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup/blkio 2023-10-03T10:10:06.777 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup/perf_event 2023-10-03T10:10:06.777 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup/devices 2023-10-03T10:10:06.777 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2023-10-03T10:10:06.777 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2023-10-03T10:10:06.777 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2023-10-03T10:10:06.777 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/selinux 2023-10-03T10:10:06.777 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2023-10-03T10:10:06.778 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2023-10-03T10:10:06.778 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2023-10-03T10:10:06.778 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2023-10-03T10:10:06.778 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2023-10-03T10:10:06.778 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/nfs/rpc_pipefs 2023-10-03T10:10:06.778 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2023-10-03T10:10:06.778 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2023-10-03T10:10:06.778 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2023-10-03T10:10:06.778 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-03T10:10:06.778 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-03T10:10:06.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-03T10:10:06.779 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2023-10-03T10:10:06.794 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:2023-10-03T10:10:06.793+0000 7f35db8643c0 -1 init, newargv = 0x55a5fd8f2640 newargc=15 2023-10-03T10:10:06.794 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:ceph-fuse[112317]: starting ceph client 2023-10-03T10:10:06.800 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:ceph-fuse[112317]: starting fuse 2023-10-03T10:10:06.814 INFO:teuthology.orchestra.run.smithi022.stdout:47 2023-10-03T10:10:06.814 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-03T10:10:06.814 DEBUG:teuthology.orchestra.run.smithi022:> sudo stdin-killer -- python3 -c ' 2023-10-03T10:10:06.814 DEBUG:teuthology.orchestra.run.smithi022:> import glob 2023-10-03T10:10:06.815 DEBUG:teuthology.orchestra.run.smithi022:> import re 2023-10-03T10:10:06.815 DEBUG:teuthology.orchestra.run.smithi022:> import os 2023-10-03T10:10:06.815 DEBUG:teuthology.orchestra.run.smithi022:> import subprocess 2023-10-03T10:10:06.815 DEBUG:teuthology.orchestra.run.smithi022:> 2023-10-03T10:10:06.815 DEBUG:teuthology.orchestra.run.smithi022:> def _find_admin_socket(client_name): 2023-10-03T10:10:06.815 DEBUG:teuthology.orchestra.run.smithi022:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-03T10:10:06.815 DEBUG:teuthology.orchestra.run.smithi022:> files = glob.glob(asok_path) 2023-10-03T10:10:06.815 DEBUG:teuthology.orchestra.run.smithi022:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-03T10:10:06.815 DEBUG:teuthology.orchestra.run.smithi022:> 2023-10-03T10:10:06.815 DEBUG:teuthology.orchestra.run.smithi022:> # Given a non-glob path, it better be there 2023-10-03T10:10:06.815 DEBUG:teuthology.orchestra.run.smithi022:> if "*" not in asok_path: 2023-10-03T10:10:06.815 DEBUG:teuthology.orchestra.run.smithi022:> assert(len(files) == 1) 2023-10-03T10:10:06.815 DEBUG:teuthology.orchestra.run.smithi022:> return files[0] 2023-10-03T10:10:06.816 DEBUG:teuthology.orchestra.run.smithi022:> 2023-10-03T10:10:06.816 DEBUG:teuthology.orchestra.run.smithi022:> for f in files: 2023-10-03T10:10:06.816 DEBUG:teuthology.orchestra.run.smithi022:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-03T10:10:06.816 DEBUG:teuthology.orchestra.run.smithi022:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-03T10:10:06.816 DEBUG:teuthology.orchestra.run.smithi022:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-03T10:10:06.816 DEBUG:teuthology.orchestra.run.smithi022:> contents = proc_f.read() 2023-10-03T10:10:06.816 DEBUG:teuthology.orchestra.run.smithi022:> if mountpoint in contents: 2023-10-03T10:10:06.816 DEBUG:teuthology.orchestra.run.smithi022:> return f 2023-10-03T10:10:06.816 DEBUG:teuthology.orchestra.run.smithi022:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-03T10:10:06.816 DEBUG:teuthology.orchestra.run.smithi022:> 2023-10-03T10:10:06.816 DEBUG:teuthology.orchestra.run.smithi022:> print(_find_admin_socket("client.0")) 2023-10-03T10:10:06.816 DEBUG:teuthology.orchestra.run.smithi022:> ' 2023-10-03T10:10:06.952 INFO:teuthology.orchestra.run.smithi022.stdout:/var/run/ceph/ceph-client.0.112317.asok 2023-10-03T10:10:06.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-10-03T10:10:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-03T10:10:06.963 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.112317.asok 2023-10-03T10:10:06.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-03T10:10:06.963 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.112317.asok status 2023-10-03T10:10:07.131 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-10-03T10:10:07.131 INFO:teuthology.orchestra.run.smithi022.stdout: "metadata": { 2023-10-03T10:10:07.131 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_sha1": "eeb13ac974775bd0cce96a2c687e8bcc2faca43d", 2023-10-03T10:10:07.131 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_version": "ceph version 16.2.14-44-geeb13ac9 (eeb13ac974775bd0cce96a2c687e8bcc2faca43d) pacific (stable)", 2023-10-03T10:10:07.131 INFO:teuthology.orchestra.run.smithi022.stdout: "entity_id": "0", 2023-10-03T10:10:07.131 INFO:teuthology.orchestra.run.smithi022.stdout: "hostname": "smithi022", 2023-10-03T10:10:07.131 INFO:teuthology.orchestra.run.smithi022.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-03T10:10:07.131 INFO:teuthology.orchestra.run.smithi022.stdout: "pid": "112317", 2023-10-03T10:10:07.131 INFO:teuthology.orchestra.run.smithi022.stdout: "root": "/" 2023-10-03T10:10:07.132 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-10-03T10:10:07.132 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_count": 0, 2023-10-03T10:10:07.132 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_pinned_count": 0, 2023-10-03T10:10:07.132 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 4517, 2023-10-03T10:10:07.132 INFO:teuthology.orchestra.run.smithi022.stdout: "inst": { 2023-10-03T10:10:07.132 INFO:teuthology.orchestra.run.smithi022.stdout: "name": { 2023-10-03T10:10:07.132 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "client", 2023-10-03T10:10:07.132 INFO:teuthology.orchestra.run.smithi022.stdout: "num": 4517 2023-10-03T10:10:07.132 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-10-03T10:10:07.133 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2023-10-03T10:10:07.133 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2023-10-03T10:10:07.133 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:0", 2023-10-03T10:10:07.133 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 728839599 2023-10-03T10:10:07.133 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-10-03T10:10:07.133 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-10-03T10:10:07.133 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2023-10-03T10:10:07.133 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2023-10-03T10:10:07.133 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:0", 2023-10-03T10:10:07.133 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 728839599 2023-10-03T10:10:07.133 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-10-03T10:10:07.134 INFO:teuthology.orchestra.run.smithi022.stdout: "inst_str": "client.4517 172.21.15.22:0/728839599", 2023-10-03T10:10:07.134 INFO:teuthology.orchestra.run.smithi022.stdout: "addr_str": "172.21.15.22:0/728839599", 2023-10-03T10:10:07.134 INFO:teuthology.orchestra.run.smithi022.stdout: "inode_count": 1, 2023-10-03T10:10:07.134 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_epoch": 6, 2023-10-03T10:10:07.134 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch": 18, 2023-10-03T10:10:07.134 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch_barrier": 0, 2023-10-03T10:10:07.134 INFO:teuthology.orchestra.run.smithi022.stdout: "blocklisted": false, 2023-10-03T10:10:07.134 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs" 2023-10-03T10:10:07.134 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-10-03T10:10:07.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-03T10:10:07.142 DEBUG:teuthology.orchestra.run.smithi022:> stat --file-system '--printf=%T 2023-10-03T10:10:07.142 DEBUG:teuthology.orchestra.run.smithi022:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-03T10:10:07.163 INFO:teuthology.orchestra.run.smithi022.stdout:fuseblk 2023-10-03T10:10:07.163 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-03T10:10:07.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-03T10:10:07.164 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-03T10:10:07.237 INFO:teuthology.run_tasks:Running task mds_pre_upgrade... 2023-10-03T10:10:07.247 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-03T10:10:07.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.387+0000 7f251036d700 1 Processor -- start 2023-10-03T10:10:07.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.388+0000 7f251036d700 1 -- start start 2023-10-03T10:10:07.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.389+0000 7f251036d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f250809ab50 0x7f2508098970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:07.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.389+0000 7f251036d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2508069670 0x7f2508130bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:07.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.389+0000 7f251036d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2508096520 con 0x7f2508098e60 2023-10-03T10:10:07.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.389+0000 7f251036d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f250809af30 con 0x7f2508069670 2023-10-03T10:10:07.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.389+0000 7f251036d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2508095830 con 0x7f250809ab50 2023-10-03T10:10:07.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.389+0000 7f250e109700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f250809ab50 0x7f2508098970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:07.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.389+0000 7f250d908700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2508098e60 0x7f2508068f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47456/0 (socket says 172.21.15.3:47456) 2023-10-03T10:10:07.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.389+0000 7f250d908700 1 -- 172.21.15.3:0/868666714 learned_addr learned my addr 172.21.15.3:0/868666714 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:10:07.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.389+0000 7f250e90a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2508069670 0x7f2508130bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:07.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.389+0000 7f250e109700 1 -- 172.21.15.3:0/868666714 >> v2:172.21.15.3:3300/0 conn(0x7f2508069670 msgr2=0x7f2508130bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:07.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.389+0000 7f250e109700 1 --2- 172.21.15.3:0/868666714 >> v2:172.21.15.3:3300/0 conn(0x7f2508069670 0x7f2508130bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:07.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.389+0000 7f250e109700 1 -- 172.21.15.3:0/868666714 >> v1:172.21.15.3:6789/0 conn(0x7f2508098e60 legacy=0x7f2508068f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:07.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.389+0000 7f250d107700 1 -- 172.21.15.3:0/868666714 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3102587727 0 0) 0x7f2508096520 con 0x7f2508098e60 2023-10-03T10:10:07.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.389+0000 7f250e109700 1 -- 172.21.15.3:0/868666714 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2508093e60 con 0x7f250809ab50 2023-10-03T10:10:07.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.389+0000 7f250e90a700 1 --2- 172.21.15.3:0/868666714 >> v2:172.21.15.3:3300/0 conn(0x7f2508069670 0x7f2508130bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:07.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.389+0000 7f250e109700 1 --2- 172.21.15.3:0/868666714 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f250809ab50 0x7f2508098970 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f2504009790 tx=0x7f2504009060).ready entity=mon.2 client_cookie=f4a61d723919e7af server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:07.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.389+0000 7f250d107700 1 -- 172.21.15.3:0/868666714 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250400f510 con 0x7f250809ab50 2023-10-03T10:10:07.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.390+0000 7f250d107700 1 -- 172.21.15.3:0/868666714 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250400f670 con 0x7f250809ab50 2023-10-03T10:10:07.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.390+0000 7f250d107700 1 -- 172.21.15.3:0/868666714 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250400f9f0 con 0x7f250809ab50 2023-10-03T10:10:07.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.390+0000 7f251036d700 1 -- 172.21.15.3:0/868666714 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f250809ab50 msgr2=0x7f2508098970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:07.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.390+0000 7f251036d700 1 --2- 172.21.15.3:0/868666714 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f250809ab50 0x7f2508098970 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f2504009790 tx=0x7f2504009060).stop 2023-10-03T10:10:07.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.390+0000 7f251036d700 1 -- 172.21.15.3:0/868666714 shutdown_connections 2023-10-03T10:10:07.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.390+0000 7f251036d700 1 --2- 172.21.15.3:0/868666714 >> v2:172.21.15.3:3300/0 conn(0x7f2508069670 0x7f2508130bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:07.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.390+0000 7f251036d700 1 --2- 172.21.15.3:0/868666714 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f250809ab50 0x7f2508098970 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:07.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.390+0000 7f251036d700 1 -- 172.21.15.3:0/868666714 >> 172.21.15.3:0/868666714 conn(0x7f2508063190 msgr2=0x7f250809a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:07.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.390+0000 7f251036d700 1 -- 172.21.15.3:0/868666714 shutdown_connections 2023-10-03T10:10:07.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.390+0000 7f251036d700 1 -- 172.21.15.3:0/868666714 wait complete. 2023-10-03T10:10:07.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.391+0000 7f251036d700 1 Processor -- start 2023-10-03T10:10:07.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.391+0000 7f251036d700 1 -- start start 2023-10-03T10:10:07.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.392+0000 7f251036d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2508069670 0x7f2508092220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:07.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.392+0000 7f251036d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2508098e60 0x7f2508092710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:07.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.392+0000 7f251036d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f250809ab50 0x7f250813a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:07.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.392+0000 7f251036d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2508094c40 con 0x7f2508098e60 2023-10-03T10:10:07.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.392+0000 7f251036d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f250812ba10 con 0x7f250809ab50 2023-10-03T10:10:07.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.392+0000 7f251036d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f250811d590 con 0x7f2508069670 2023-10-03T10:10:07.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.392+0000 7f250e109700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2508069670 0x7f2508092220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:07.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.392+0000 7f250e109700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2508069670 0x7f2508092220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:60314/0 (socket says 172.21.15.3:60314) 2023-10-03T10:10:07.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.392+0000 7f250e109700 1 -- 172.21.15.3:0/2244667634 learned_addr learned my addr 172.21.15.3:0/2244667634 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:07.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.392+0000 7f250d908700 1 --2- 172.21.15.3:0/2244667634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2508098e60 0x7f2508092710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:07.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.392+0000 7f250e90a700 1 --2- 172.21.15.3:0/2244667634 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f250809ab50 0x7f250813a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:07.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.392+0000 7f250e109700 1 -- 172.21.15.3:0/2244667634 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f250809ab50 msgr2=0x7f250813a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:07.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.392+0000 7f250e109700 1 --2- 172.21.15.3:0/2244667634 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f250809ab50 0x7f250813a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:07.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.392+0000 7f250e109700 1 -- 172.21.15.3:0/2244667634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2508098e60 msgr2=0x7f2508092710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:07.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.392+0000 7f250e109700 1 --2- 172.21.15.3:0/2244667634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2508098e60 0x7f2508092710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:07.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.392+0000 7f250e109700 1 -- 172.21.15.3:0/2244667634 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2500009a20 con 0x7f2508069670 2023-10-03T10:10:07.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.392+0000 7f250e109700 1 --2- 172.21.15.3:0/2244667634 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2508069670 0x7f2508092220 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f2508093d40 tx=0x7f2504017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:07.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.392+0000 7f24fe7fc700 1 -- 172.21.15.3:0/2244667634 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250400f7e0 con 0x7f2508069670 2023-10-03T10:10:07.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.392+0000 7f24fe7fc700 1 -- 172.21.15.3:0/2244667634 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250400f940 con 0x7f2508069670 2023-10-03T10:10:07.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.392+0000 7f251036d700 1 -- 172.21.15.3:0/2244667634 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25040092a0 con 0x7f2508069670 2023-10-03T10:10:07.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.392+0000 7f24fe7fc700 1 -- 172.21.15.3:0/2244667634 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250401c670 con 0x7f2508069670 2023-10-03T10:10:07.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.393+0000 7f251036d700 1 -- 172.21.15.3:0/2244667634 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f250811e210 con 0x7f2508069670 2023-10-03T10:10:07.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.393+0000 7f24fe7fc700 1 -- 172.21.15.3:0/2244667634 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f250401c9e0 con 0x7f2508069670 2023-10-03T10:10:07.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.393+0000 7f24fe7fc700 1 -- 172.21.15.3:0/2244667634 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f250403c6c0 con 0x7f2508069670 2023-10-03T10:10:07.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.394+0000 7f251036d700 1 -- 172.21.15.3:0/2244667634 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f250811db80 con 0x7f2508069670 2023-10-03T10:10:07.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.398+0000 7f24fe7fc700 1 -- 172.21.15.3:0/2244667634 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f2504018940 con 0x7f2508069670 2023-10-03T10:10:07.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.575+0000 7f251036d700 1 -- 172.21.15.3:0/2244667634 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f250805a260 con 0x7f2508069670 2023-10-03T10:10:07.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.576+0000 7f24fe7fc700 1 -- 172.21.15.3:0/2244667634 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+11308 (secure 0 0 0) 0x7f2504018350 con 0x7f2508069670 2023-10-03T10:10:07.577 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:10:07.577 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":18,"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","created":"2023-10-03T10:09:28.703441+0000","modified":"2023-10-03T10:10:04.610398+0000","last_up_change":"2023-10-03T10:09:36.569401+0000","last_in_change":"2023-10-03T10:09:30.156546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-10-03T10:09:36.623971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-03T10:09:38.210614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-03T10:09:42.695873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-03T10:09:43.115588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cdcdd930-3c5b-4486-a021-7132f90fb6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6808","nonce":118845}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6809","nonce":118845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6811","nonce":118845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6810","nonce":118845}]},"public_addr":"172.21.15.3:6808/118845","cluster_addr":"172.21.15.3:6809/118845","heartbeat_back_addr":"172.21.15.3:6811/118845","heartbeat_front_addr":"172.21.15.3:6810/118845","state":["exists","up"]},{"osd":1,"uuid":"c8e58877-6762-4741-a8c1-567ebc97b708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6804","nonce":118835}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6805","nonce":118835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6807","nonce":118835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6806","nonce":118835}]},"public_addr":"172.21.15.3:6804/118835","cluster_addr":"172.21.15.3:6805/118835","heartbeat_back_addr":"172.21.15.3:6807/118835","heartbeat_front_addr":"172.21.15.3:6806/118835","state":["exists","up"]},{"osd":2,"uuid":"68a2a21a-7a8e-4f82-8433-5201faa50777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6812","nonce":118864}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6813","nonce":118864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6815","nonce":118864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6814","nonce":118864}]},"public_addr":"172.21.15.3:6812/118864","cluster_addr":"172.21.15.3:6813/118864","heartbeat_back_addr":"172.21.15.3:6815/118864","heartbeat_front_addr":"172.21.15.3:6814/118864","state":["exists","up"]},{"osd":3,"uuid":"c4df2903-4e1e-4208-bf77-95b9fa749241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6800","nonce":118836}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6801","nonce":118836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6803","nonce":118836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6802","nonce":118836}]},"public_addr":"172.21.15.3:6800/118836","cluster_addr":"172.21.15.3:6801/118836","heartbeat_back_addr":"172.21.15.3:6803/118836","heartbeat_front_addr":"172.21.15.3:6802/118836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.369708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.257825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.337931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.085222+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-03T10:10:07.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.578+0000 7f251036d700 1 -- 172.21.15.3:0/2244667634 >> v1:172.21.15.3:6816/118296 conn(0x7f24ec02bb90 legacy=0x7f24ec02e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:07.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.578+0000 7f251036d700 1 -- 172.21.15.3:0/2244667634 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2508069670 msgr2=0x7f2508092220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:07.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.578+0000 7f251036d700 1 --2- 172.21.15.3:0/2244667634 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2508069670 0x7f2508092220 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f2508093d40 tx=0x7f2504017000).stop 2023-10-03T10:10:07.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.578+0000 7f251036d700 1 -- 172.21.15.3:0/2244667634 shutdown_connections 2023-10-03T10:10:07.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.578+0000 7f251036d700 1 --2- 172.21.15.3:0/2244667634 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f250809ab50 0x7f250813a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:07.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.578+0000 7f251036d700 1 --2- 172.21.15.3:0/2244667634 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2508069670 0x7f2508092220 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:07.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.578+0000 7f251036d700 1 --2- 172.21.15.3:0/2244667634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2508098e60 0x7f2508092710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:07.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.578+0000 7f251036d700 1 -- 172.21.15.3:0/2244667634 >> 172.21.15.3:0/2244667634 conn(0x7f2508063190 msgr2=0x7f25080973b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:07.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.578+0000 7f251036d700 1 -- 172.21.15.3:0/2244667634 shutdown_connections 2023-10-03T10:10:07.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.578+0000 7f251036d700 1 -- 172.21.15.3:0/2244667634 wait complete. 2023-10-03T10:10:07.592 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-10-03T10:09:36.623971+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-03T10:09:38.210614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-03T10:09:42.695873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-03T10:09:43.115588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-03T10:10:07.592 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-10-03T10:10:07.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.727+0000 7f7df7c06700 1 Processor -- start 2023-10-03T10:10:07.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.727+0000 7f7df7c06700 1 -- start start 2023-10-03T10:10:07.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.728+0000 7f7df7c06700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7df0127630 0x7f7df0099810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:07.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.728+0000 7f7df7c06700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7df0127bb0 0x7f7df0093640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:07.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.728+0000 7f7df7c06700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7df0124160 0x7f7df01245e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:07.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.728+0000 7f7df7c06700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7df0127f90 con 0x7f7df0127630 2023-10-03T10:10:07.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.728+0000 7f7df7c06700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df0091a40 con 0x7f7df0127bb0 2023-10-03T10:10:07.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.728+0000 7f7df7c06700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7df0094ef0 con 0x7f7df0124160 2023-10-03T10:10:07.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.728+0000 7f7df51a1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7df0127bb0 0x7f7df0093640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:07.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.728+0000 7f7df61a3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7df0124160 0x7f7df01245e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:07.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.728+0000 7f7df59a2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7df0127630 0x7f7df0099810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:07.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.728+0000 7f7df61a3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7df0124160 0x7f7df01245e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:60318/0 (socket says 172.21.15.3:60318) 2023-10-03T10:10:07.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.728+0000 7f7df51a1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7df0127bb0 0x7f7df0093640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53858/0 (socket says 172.21.15.3:53858) 2023-10-03T10:10:07.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.728+0000 7f7df61a3700 1 -- 172.21.15.3:0/3084391406 learned_addr learned my addr 172.21.15.3:0/3084391406 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:07.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.728+0000 7f7df51a1700 1 -- 172.21.15.3:0/3084391406 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7df0124160 msgr2=0x7f7df01245e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:07.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.728+0000 7f7df51a1700 1 --2- 172.21.15.3:0/3084391406 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7df0124160 0x7f7df01245e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:07.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.728+0000 7f7df51a1700 1 -- 172.21.15.3:0/3084391406 >> v2:172.21.15.3:3300/0 conn(0x7f7df0127630 msgr2=0x7f7df0099810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:07.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.728+0000 7f7df51a1700 1 --2- 172.21.15.3:0/3084391406 >> v2:172.21.15.3:3300/0 conn(0x7f7df0127630 0x7f7df0099810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:07.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.728+0000 7f7df51a1700 1 -- 172.21.15.3:0/3084391406 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df0128190 con 0x7f7df0127bb0 2023-10-03T10:10:07.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.728+0000 7f7df59a2700 1 --2- 172.21.15.3:0/3084391406 >> v2:172.21.15.3:3300/0 conn(0x7f7df0127630 0x7f7df0099810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:07.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.729+0000 7f7df51a1700 1 --2- 172.21.15.3:0/3084391406 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7df0127bb0 0x7f7df0093640 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f7de4009760 tx=0x7f7de4009060).ready entity=mon.1 client_cookie=255248f4651e9c1c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:07.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.729+0000 7f7df49a0700 1 -- 172.21.15.3:0/3084391406 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de40090d0 con 0x7f7df0127bb0 2023-10-03T10:10:07.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.729+0000 7f7df49a0700 1 -- 172.21.15.3:0/3084391406 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de400d4a0 con 0x7f7df0127bb0 2023-10-03T10:10:07.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.729+0000 7f7df49a0700 1 -- 172.21.15.3:0/3084391406 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de400d820 con 0x7f7df0127bb0 2023-10-03T10:10:07.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.729+0000 7f7df7c06700 1 -- 172.21.15.3:0/3084391406 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7df0127bb0 msgr2=0x7f7df0093640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:07.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.729+0000 7f7df7c06700 1 --2- 172.21.15.3:0/3084391406 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7df0127bb0 0x7f7df0093640 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f7de4009760 tx=0x7f7de4009060).stop 2023-10-03T10:10:07.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.729+0000 7f7df7c06700 1 -- 172.21.15.3:0/3084391406 shutdown_connections 2023-10-03T10:10:07.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.729+0000 7f7df7c06700 1 --2- 172.21.15.3:0/3084391406 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7df0124160 0x7f7df01245e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:07.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.729+0000 7f7df7c06700 1 --2- 172.21.15.3:0/3084391406 >> v2:172.21.15.3:3300/0 conn(0x7f7df0127630 0x7f7df0099810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:07.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.729+0000 7f7df7c06700 1 --2- 172.21.15.3:0/3084391406 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7df0127bb0 0x7f7df0093640 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:07.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.729+0000 7f7df7c06700 1 -- 172.21.15.3:0/3084391406 >> 172.21.15.3:0/3084391406 conn(0x7f7df00599a0 msgr2=0x7f7df0097460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:07.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.729+0000 7f7df7c06700 1 -- 172.21.15.3:0/3084391406 shutdown_connections 2023-10-03T10:10:07.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.729+0000 7f7df7c06700 1 -- 172.21.15.3:0/3084391406 wait complete. 2023-10-03T10:10:07.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.731+0000 7f7df7c06700 1 Processor -- start 2023-10-03T10:10:07.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.731+0000 7f7df7c06700 1 -- start start 2023-10-03T10:10:07.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.731+0000 7f7df7c06700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7df0124160 0x7f7df0115650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:07.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.731+0000 7f7df7c06700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7df0127630 0x7f7df0111ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:07.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.731+0000 7f7df7c06700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df0127bb0 0x7f7df01123d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:07.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.731+0000 7f7df7c06700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df0092350 con 0x7f7df0127bb0 2023-10-03T10:10:07.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.731+0000 7f7df7c06700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df0114bb0 con 0x7f7df0127630 2023-10-03T10:10:07.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.731+0000 7f7df7c06700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7df0114e70 con 0x7f7df0124160 2023-10-03T10:10:07.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.732+0000 7f7df61a3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df0127bb0 0x7f7df01123d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:07.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.732+0000 7f7df59a2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7df0124160 0x7f7df0115650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:07.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.732+0000 7f7df61a3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df0127bb0 0x7f7df01123d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47486/0 (socket says 172.21.15.3:47486) 2023-10-03T10:10:07.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.732+0000 7f7df61a3700 1 -- 172.21.15.3:0/3950093408 learned_addr learned my addr 172.21.15.3:0/3950093408 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:07.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.732+0000 7f7df51a1700 1 --2- 172.21.15.3:0/3950093408 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7df0127630 0x7f7df0111ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:07.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.732+0000 7f7df59a2700 1 -- 172.21.15.3:0/3950093408 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7df0127630 msgr2=0x7f7df0111ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:07.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.732+0000 7f7df59a2700 1 --2- 172.21.15.3:0/3950093408 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7df0127630 0x7f7df0111ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:07.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.732+0000 7f7df59a2700 1 -- 172.21.15.3:0/3950093408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df0127bb0 msgr2=0x7f7df01123d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:07.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.732+0000 7f7df59a2700 1 --2- 172.21.15.3:0/3950093408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df0127bb0 0x7f7df01123d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:07.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.732+0000 7f7df59a2700 1 -- 172.21.15.3:0/3950093408 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dec0092a0 con 0x7f7df0124160 2023-10-03T10:10:07.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.732+0000 7f7df61a3700 1 --2- 172.21.15.3:0/3950093408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df0127bb0 0x7f7df01123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:07.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.732+0000 7f7df51a1700 1 --2- 172.21.15.3:0/3950093408 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7df0127630 0x7f7df0111ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:07.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.732+0000 7f7df59a2700 1 --2- 172.21.15.3:0/3950093408 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7df0124160 0x7f7df0115650 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f7dec00e980 tx=0x7f7dec009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:07.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.732+0000 7f7de1ffb700 1 -- 172.21.15.3:0/3950093408 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec012600 con 0x7f7df0124160 2023-10-03T10:10:07.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.732+0000 7f7de1ffb700 1 -- 172.21.15.3:0/3950093408 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dec012760 con 0x7f7df0124160 2023-10-03T10:10:07.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.732+0000 7f7df7c06700 1 -- 172.21.15.3:0/3950093408 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7de40092a0 con 0x7f7df0124160 2023-10-03T10:10:07.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.732+0000 7f7de1ffb700 1 -- 172.21.15.3:0/3950093408 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec012ae0 con 0x7f7df0124160 2023-10-03T10:10:07.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.733+0000 7f7df7c06700 1 -- 172.21.15.3:0/3950093408 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df01128c0 con 0x7f7df0124160 2023-10-03T10:10:07.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.733+0000 7f7de1ffb700 1 -- 172.21.15.3:0/3950093408 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f7dec01b400 con 0x7f7df0124160 2023-10-03T10:10:07.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.733+0000 7f7de1ffb700 1 -- 172.21.15.3:0/3950093408 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f7dec03b480 con 0x7f7df0124160 2023-10-03T10:10:07.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.734+0000 7f7df7c06700 1 -- 172.21.15.3:0/3950093408 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7df0112b10 con 0x7f7df0124160 2023-10-03T10:10:07.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.738+0000 7f7de1ffb700 1 -- 172.21.15.3:0/3950093408 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f7dec016070 con 0x7f7df0124160 2023-10-03T10:10:07.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.914+0000 7f7df7c06700 1 -- 172.21.15.3:0/3950093408 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7df01151a0 con 0x7f7df0124160 2023-10-03T10:10:07.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.915+0000 7f7de1ffb700 1 -- 172.21.15.3:0/3950093408 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v18) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7dec0290b0 con 0x7f7df0124160 2023-10-03T10:10:07.916 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-03T10:10:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.917+0000 7f7df7c06700 1 -- 172.21.15.3:0/3950093408 >> v1:172.21.15.3:6816/118296 conn(0x7f7dd402bb60 legacy=0x7f7dd402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.917+0000 7f7df7c06700 1 -- 172.21.15.3:0/3950093408 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7df0124160 msgr2=0x7f7df0115650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.917+0000 7f7df7c06700 1 --2- 172.21.15.3:0/3950093408 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7df0124160 0x7f7df0115650 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f7dec00e980 tx=0x7f7dec009060).stop 2023-10-03T10:10:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.917+0000 7f7df7c06700 1 -- 172.21.15.3:0/3950093408 shutdown_connections 2023-10-03T10:10:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.917+0000 7f7df7c06700 1 --2- 172.21.15.3:0/3950093408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df0127bb0 0x7f7df01123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.917+0000 7f7df7c06700 1 --2- 172.21.15.3:0/3950093408 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7df0124160 0x7f7df0115650 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.917+0000 7f7df7c06700 1 --2- 172.21.15.3:0/3950093408 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7df0127630 0x7f7df0111ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.917+0000 7f7df7c06700 1 -- 172.21.15.3:0/3950093408 >> 172.21.15.3:0/3950093408 conn(0x7f7df00599a0 msgr2=0x7f7df0125ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.917+0000 7f7df7c06700 1 -- 172.21.15.3:0/3950093408 shutdown_connections 2023-10-03T10:10:07.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:07.917+0000 7f7df7c06700 1 -- 172.21.15.3:0/3950093408 wait complete. 2023-10-03T10:10:07.931 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-03T10:10:08.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.064+0000 7f3f5ea68700 1 Processor -- start 2023-10-03T10:10:08.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.064+0000 7f3f5ea68700 1 -- start start 2023-10-03T10:10:08.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.064+0000 7f3f5ea68700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3f58096440 0x7f3f58096820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:08.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.064+0000 7f3f5ea68700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f5812e280 0x7f3f58130670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:08.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.064+0000 7f3f5ea68700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f58098330 con 0x7f3f58096d10 2023-10-03T10:10:08.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.064+0000 7f3f5ea68700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f58097800 con 0x7f3f58096440 2023-10-03T10:10:08.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.064+0000 7f3f5ea68700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5806ab40 con 0x7f3f5812e280 2023-10-03T10:10:08.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.064+0000 7f3f5d005700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f5812e280 0x7f3f58130670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:08.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.064+0000 7f3f5d005700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f5812e280 0x7f3f58130670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:60336/0 (socket says 172.21.15.3:60336) 2023-10-03T10:10:08.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.064+0000 7f3f57fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3f58096d10 0x7f3f580970f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47466/0 (socket says 172.21.15.3:47466) 2023-10-03T10:10:08.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.064+0000 7f3f5d005700 1 -- 172.21.15.3:0/1694270146 learned_addr learned my addr 172.21.15.3:0/1694270146 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:08.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.065+0000 7f3f5c804700 1 --2- 172.21.15.3:0/1694270146 >> v2:172.21.15.3:3300/0 conn(0x7f3f58096440 0x7f3f58096820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:08.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.065+0000 7f3f577fe700 1 -- 172.21.15.3:0/1694270146 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3496069013 0 0) 0x7f3f58098330 con 0x7f3f58096d10 2023-10-03T10:10:08.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.065+0000 7f3f577fe700 1 -- 172.21.15.3:0/1694270146 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f400033f0 con 0x7f3f58096d10 2023-10-03T10:10:08.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.065+0000 7f3f5c804700 1 -- 172.21.15.3:0/1694270146 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f5812e280 msgr2=0x7f3f58130670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:08.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.065+0000 7f3f5c804700 1 --2- 172.21.15.3:0/1694270146 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f5812e280 0x7f3f58130670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:08.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.065+0000 7f3f5c804700 1 -- 172.21.15.3:0/1694270146 >> v1:172.21.15.3:6789/0 conn(0x7f3f58096d10 legacy=0x7f3f580970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:08.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.065+0000 7f3f5c804700 1 -- 172.21.15.3:0/1694270146 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f58068cf0 con 0x7f3f58096440 2023-10-03T10:10:08.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.065+0000 7f3f5c804700 1 --2- 172.21.15.3:0/1694270146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f58096440 0x7f3f58096820 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f3f48009790 tx=0x7f3f48009060).ready entity=mon.0 client_cookie=9125009c6404a32a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:08.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.065+0000 7f3f577fe700 1 -- 172.21.15.3:0/1694270146 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f480090d0 con 0x7f3f58096440 2023-10-03T10:10:08.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.065+0000 7f3f577fe700 1 -- 172.21.15.3:0/1694270146 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4800d4a0 con 0x7f3f58096440 2023-10-03T10:10:08.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.065+0000 7f3f577fe700 1 -- 172.21.15.3:0/1694270146 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4800d820 con 0x7f3f58096440 2023-10-03T10:10:08.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.066+0000 7f3f5ea68700 1 -- 172.21.15.3:0/1694270146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f58096440 msgr2=0x7f3f58096820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:08.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.066+0000 7f3f5ea68700 1 --2- 172.21.15.3:0/1694270146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f58096440 0x7f3f58096820 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f3f48009790 tx=0x7f3f48009060).stop 2023-10-03T10:10:08.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.066+0000 7f3f5ea68700 1 -- 172.21.15.3:0/1694270146 shutdown_connections 2023-10-03T10:10:08.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.066+0000 7f3f5ea68700 1 --2- 172.21.15.3:0/1694270146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f58096440 0x7f3f58096820 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:08.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.066+0000 7f3f5ea68700 1 --2- 172.21.15.3:0/1694270146 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f5812e280 0x7f3f58130670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:08.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.066+0000 7f3f5ea68700 1 -- 172.21.15.3:0/1694270146 >> 172.21.15.3:0/1694270146 conn(0x7f3f58059bb0 msgr2=0x7f3f5809ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:08.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.066+0000 7f3f5ea68700 1 -- 172.21.15.3:0/1694270146 shutdown_connections 2023-10-03T10:10:08.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.066+0000 7f3f5ea68700 1 -- 172.21.15.3:0/1694270146 wait complete. 2023-10-03T10:10:08.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.067+0000 7f3f5ea68700 1 Processor -- start 2023-10-03T10:10:08.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.067+0000 7f3f5ea68700 1 -- start start 2023-10-03T10:10:08.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.068+0000 7f3f5ea68700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f58096440 0x7f3f5811bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:08.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.068+0000 7f3f5ea68700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3f58096d10 0x7f3f58117d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:08.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.068+0000 7f3f5ea68700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f5812e280 0x7f3f581181f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:08.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.068+0000 7f3f5ea68700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f58067dc0 con 0x7f3f5812e280 2023-10-03T10:10:08.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.068+0000 7f3f5ea68700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5812deb0 con 0x7f3f58096d10 2023-10-03T10:10:08.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.068+0000 7f3f5ea68700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5811b830 con 0x7f3f58096440 2023-10-03T10:10:08.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.068+0000 7f3f5c804700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f58096440 0x7f3f5811bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:08.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.068+0000 7f3f5d005700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f5812e280 0x7f3f581181f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:08.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.068+0000 7f3f57fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3f58096d10 0x7f3f58117d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:08.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.068+0000 7f3f5c804700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f58096440 0x7f3f5811bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:60348/0 (socket says 172.21.15.3:60348) 2023-10-03T10:10:08.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.068+0000 7f3f5c804700 1 -- 172.21.15.3:0/1407871571 learned_addr learned my addr 172.21.15.3:0/1407871571 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:08.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.068+0000 7f3f5c804700 1 -- 172.21.15.3:0/1407871571 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3f58096d10 msgr2=0x7f3f58117d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:08.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.068+0000 7f3f5c804700 1 --2- 172.21.15.3:0/1407871571 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3f58096d10 0x7f3f58117d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:08.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.068+0000 7f3f5c804700 1 -- 172.21.15.3:0/1407871571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f5812e280 msgr2=0x7f3f581181f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:08.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.068+0000 7f3f5c804700 1 --2- 172.21.15.3:0/1407871571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f5812e280 0x7f3f581181f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:08.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.068+0000 7f3f5c804700 1 -- 172.21.15.3:0/1407871571 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f480092a0 con 0x7f3f58096440 2023-10-03T10:10:08.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.068+0000 7f3f5d005700 1 --2- 172.21.15.3:0/1407871571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f5812e280 0x7f3f581181f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:08.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.068+0000 7f3f57fff700 1 --2- 172.21.15.3:0/1407871571 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3f58096d10 0x7f3f58117d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:08.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.068+0000 7f3f5c804700 1 --2- 172.21.15.3:0/1407871571 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f58096440 0x7f3f5811bd30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f3f4800d260 tx=0x7f3f48016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:08.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.068+0000 7f3f54ff9700 1 -- 172.21.15.3:0/1407871571 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48006910 con 0x7f3f58096440 2023-10-03T10:10:08.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.068+0000 7f3f54ff9700 1 -- 172.21.15.3:0/1407871571 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f480127f0 con 0x7f3f58096440 2023-10-03T10:10:08.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.068+0000 7f3f5ea68700 1 -- 172.21.15.3:0/1407871571 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f58091be0 con 0x7f3f58096440 2023-10-03T10:10:08.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.068+0000 7f3f54ff9700 1 -- 172.21.15.3:0/1407871571 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48012bb0 con 0x7f3f58096440 2023-10-03T10:10:08.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.069+0000 7f3f5ea68700 1 -- 172.21.15.3:0/1407871571 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f58119ae0 con 0x7f3f58096440 2023-10-03T10:10:08.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.069+0000 7f3f54ff9700 1 -- 172.21.15.3:0/1407871571 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f3f48026460 con 0x7f3f58096440 2023-10-03T10:10:08.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.069+0000 7f3f54ff9700 1 -- 172.21.15.3:0/1407871571 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f3f4803c2e0 con 0x7f3f58096440 2023-10-03T10:10:08.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.070+0000 7f3f5ea68700 1 -- 172.21.15.3:0/1407871571 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f5811b4b0 con 0x7f3f58096440 2023-10-03T10:10:08.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.074+0000 7f3f54ff9700 1 -- 172.21.15.3:0/1407871571 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f3f5811b4b0 con 0x7f3f58096440 2023-10-03T10:10:08.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.252+0000 7f3f5ea68700 1 -- 172.21.15.3:0/1407871571 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3f5811b4b0 con 0x7f3f58096440 2023-10-03T10:10:08.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.252+0000 7f3f54ff9700 1 -- 172.21.15.3:0/1407871571 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v18) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3f5811b4b0 con 0x7f3f58096440 2023-10-03T10:10:08.253 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-03T10:10:08.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.254+0000 7f3f5ea68700 1 -- 172.21.15.3:0/1407871571 >> v1:172.21.15.3:6816/118296 conn(0x7f3f4402bb60 legacy=0x7f3f4402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:08.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.254+0000 7f3f5ea68700 1 -- 172.21.15.3:0/1407871571 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f58096440 msgr2=0x7f3f5811bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:08.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.254+0000 7f3f5ea68700 1 --2- 172.21.15.3:0/1407871571 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f58096440 0x7f3f5811bd30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f3f4800d260 tx=0x7f3f48016000).stop 2023-10-03T10:10:08.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.254+0000 7f3f5ea68700 1 -- 172.21.15.3:0/1407871571 shutdown_connections 2023-10-03T10:10:08.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.254+0000 7f3f5ea68700 1 --2- 172.21.15.3:0/1407871571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f5812e280 0x7f3f581181f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:08.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.254+0000 7f3f5ea68700 1 --2- 172.21.15.3:0/1407871571 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f58096440 0x7f3f5811bd30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:08.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.254+0000 7f3f5ea68700 1 --2- 172.21.15.3:0/1407871571 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3f58096d10 0x7f3f58117d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:08.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.254+0000 7f3f5ea68700 1 -- 172.21.15.3:0/1407871571 >> 172.21.15.3:0/1407871571 conn(0x7f3f58059bb0 msgr2=0x7f3f58094bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:08.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.254+0000 7f3f5ea68700 1 -- 172.21.15.3:0/1407871571 shutdown_connections 2023-10-03T10:10:08.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.254+0000 7f3f5ea68700 1 -- 172.21.15.3:0/1407871571 wait complete. 2023-10-03T10:10:08.268 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-03T10:10:08.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.401+0000 7f088209e700 1 Processor -- start 2023-10-03T10:10:08.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.401+0000 7f088209e700 1 -- start start 2023-10-03T10:10:08.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.401+0000 7f088209e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f087c125d40 0x7f087c12dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:08.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.401+0000 7f088209e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f087c12dfc0 0x7f087c1303b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:08.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.401+0000 7f088209e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f087c09a370 con 0x7f087c062a90 2023-10-03T10:10:08.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.401+0000 7f088209e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f087c098a10 con 0x7f087c12dfc0 2023-10-03T10:10:08.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.401+0000 7f088209e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087c099420 con 0x7f087c125d40 2023-10-03T10:10:08.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.402+0000 7f087affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f087c125d40 0x7f087c12dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:08.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.402+0000 7f087bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f087c12dfc0 0x7f087c1303b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:08.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.402+0000 7f087affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f087c125d40 0x7f087c12dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53888/0 (socket says 172.21.15.3:53888) 2023-10-03T10:10:08.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.402+0000 7f087affd700 1 -- 172.21.15.3:0/2765769827 learned_addr learned my addr 172.21.15.3:0/2765769827 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:08.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.402+0000 7f087b7fe700 1 --1- 172.21.15.3:0/2765769827 >> v1:172.21.15.3:6789/0 conn(0x7f087c062a90 0x7f087c125630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47470/0 (socket says 172.21.15.3:47470) 2023-10-03T10:10:08.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.402+0000 7f087affd700 1 -- 172.21.15.3:0/2765769827 >> v2:172.21.15.3:3300/0 conn(0x7f087c12dfc0 msgr2=0x7f087c1303b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:08.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.402+0000 7f087affd700 1 --2- 172.21.15.3:0/2765769827 >> v2:172.21.15.3:3300/0 conn(0x7f087c12dfc0 0x7f087c1303b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:08.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.402+0000 7f087affd700 1 -- 172.21.15.3:0/2765769827 >> v1:172.21.15.3:6789/0 conn(0x7f087c062a90 legacy=0x7f087c125630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:08.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.402+0000 7f087affd700 1 -- 172.21.15.3:0/2765769827 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087c0631c0 con 0x7f087c125d40 2023-10-03T10:10:08.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.402+0000 7f087bfff700 1 --2- 172.21.15.3:0/2765769827 >> v2:172.21.15.3:3300/0 conn(0x7f087c12dfc0 0x7f087c1303b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:08.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.402+0000 7f087affd700 1 --2- 172.21.15.3:0/2765769827 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f087c125d40 0x7f087c12dad0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f0870009830 tx=0x7f08700090f0).ready entity=mon.1 client_cookie=991644bda59f817f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:08.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.402+0000 7f087a7fc700 1 -- 172.21.15.3:0/2765769827 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087000b580 con 0x7f087c125d40 2023-10-03T10:10:08.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.402+0000 7f087a7fc700 1 -- 172.21.15.3:0/2765769827 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087000b6e0 con 0x7f087c125d40 2023-10-03T10:10:08.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.402+0000 7f087a7fc700 1 -- 172.21.15.3:0/2765769827 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087000ba60 con 0x7f087c125d40 2023-10-03T10:10:08.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.403+0000 7f088209e700 1 -- 172.21.15.3:0/2765769827 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f087c125d40 msgr2=0x7f087c12dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:08.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.403+0000 7f088209e700 1 --2- 172.21.15.3:0/2765769827 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f087c125d40 0x7f087c12dad0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f0870009830 tx=0x7f08700090f0).stop 2023-10-03T10:10:08.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.403+0000 7f088209e700 1 -- 172.21.15.3:0/2765769827 shutdown_connections 2023-10-03T10:10:08.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.403+0000 7f088209e700 1 --2- 172.21.15.3:0/2765769827 >> v2:172.21.15.3:3300/0 conn(0x7f087c12dfc0 0x7f087c1303b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:08.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.403+0000 7f088209e700 1 --2- 172.21.15.3:0/2765769827 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f087c125d40 0x7f087c12dad0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:08.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.403+0000 7f088209e700 1 -- 172.21.15.3:0/2765769827 >> 172.21.15.3:0/2765769827 conn(0x7f087c0599a0 msgr2=0x7f087c097460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:08.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.403+0000 7f088209e700 1 -- 172.21.15.3:0/2765769827 shutdown_connections 2023-10-03T10:10:08.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.403+0000 7f088209e700 1 -- 172.21.15.3:0/2765769827 wait complete. 2023-10-03T10:10:08.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.404+0000 7f088209e700 1 Processor -- start 2023-10-03T10:10:08.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.404+0000 7f088209e700 1 -- start start 2023-10-03T10:10:08.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.405+0000 7f088209e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f087c062a90 0x7f087c129ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:08.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.405+0000 7f088209e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f087c125d40 0x7f087c12a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:08.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.405+0000 7f088209e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f087c12dfc0 0x7f087c12a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:08.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.405+0000 7f088209e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c12ab70 con 0x7f087c125d40 2023-10-03T10:10:08.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.405+0000 7f088209e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087c1115e0 con 0x7f087c062a90 2023-10-03T10:10:08.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.405+0000 7f088209e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f087c098c70 con 0x7f087c12dfc0 2023-10-03T10:10:08.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.405+0000 7f087b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f087c062a90 0x7f087c129ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:08.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.405+0000 7f087affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f087c125d40 0x7f087c12a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:08.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.405+0000 7f087b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f087c062a90 0x7f087c129ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53892/0 (socket says 172.21.15.3:53892) 2023-10-03T10:10:08.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.405+0000 7f087affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f087c125d40 0x7f087c12a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47518/0 (socket says 172.21.15.3:47518) 2023-10-03T10:10:08.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.405+0000 7f087b7fe700 1 -- 172.21.15.3:0/4157480602 learned_addr learned my addr 172.21.15.3:0/4157480602 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:08.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.405+0000 7f087bfff700 1 --2- 172.21.15.3:0/4157480602 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f087c12dfc0 0x7f087c12a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:08.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.405+0000 7f087affd700 1 -- 172.21.15.3:0/4157480602 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f087c12dfc0 msgr2=0x7f087c12a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:08.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.405+0000 7f087affd700 1 --2- 172.21.15.3:0/4157480602 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f087c12dfc0 0x7f087c12a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:08.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.405+0000 7f087affd700 1 -- 172.21.15.3:0/4157480602 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f087c062a90 msgr2=0x7f087c129ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:08.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.405+0000 7f087affd700 1 --2- 172.21.15.3:0/4157480602 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f087c062a90 0x7f087c129ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:08.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.405+0000 7f087affd700 1 -- 172.21.15.3:0/4157480602 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0874009a20 con 0x7f087c125d40 2023-10-03T10:10:08.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.405+0000 7f087b7fe700 1 --2- 172.21.15.3:0/4157480602 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f087c062a90 0x7f087c129ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:08.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.405+0000 7f087affd700 1 --2- 172.21.15.3:0/4157480602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f087c125d40 0x7f087c12a190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f0870014000 tx=0x7f0870017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:08.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.405+0000 7f08637fe700 1 -- 172.21.15.3:0/4157480602 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087000b850 con 0x7f087c125d40 2023-10-03T10:10:08.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.405+0000 7f08637fe700 1 -- 172.21.15.3:0/4157480602 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087000b9b0 con 0x7f087c125d40 2023-10-03T10:10:08.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.406+0000 7f088209e700 1 -- 172.21.15.3:0/4157480602 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0870009370 con 0x7f087c125d40 2023-10-03T10:10:08.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.406+0000 7f08637fe700 1 -- 172.21.15.3:0/4157480602 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087001c670 con 0x7f087c125d40 2023-10-03T10:10:08.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.406+0000 7f088209e700 1 -- 172.21.15.3:0/4157480602 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f087c12d040 con 0x7f087c125d40 2023-10-03T10:10:08.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.406+0000 7f08637fe700 1 -- 172.21.15.3:0/4157480602 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f087001caa0 con 0x7f087c125d40 2023-10-03T10:10:08.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.407+0000 7f08637fe700 1 -- 172.21.15.3:0/4157480602 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f087003cbf0 con 0x7f087c125d40 2023-10-03T10:10:08.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.407+0000 7f088209e700 1 -- 172.21.15.3:0/4157480602 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f087c12b730 con 0x7f087c125d40 2023-10-03T10:10:08.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.411+0000 7f08637fe700 1 -- 172.21.15.3:0/4157480602 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f0870020080 con 0x7f087c125d40 2023-10-03T10:10:08.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.587+0000 7f088209e700 1 -- 172.21.15.3:0/4157480602 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f087c125200 con 0x7f087c125d40 2023-10-03T10:10:08.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.588+0000 7f08637fe700 1 -- 172.21.15.3:0/4157480602 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v18) v1 ==== 104+0+11 (secure 0 0 0) 0x7f087001b3b0 con 0x7f087c125d40 2023-10-03T10:10:08.588 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-10-03T10:10:08.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.590+0000 7f088209e700 1 -- 172.21.15.3:0/4157480602 >> v1:172.21.15.3:6816/118296 conn(0x7f086802bb60 legacy=0x7f086802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:08.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.590+0000 7f088209e700 1 -- 172.21.15.3:0/4157480602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f087c125d40 msgr2=0x7f087c12a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:08.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.590+0000 7f088209e700 1 --2- 172.21.15.3:0/4157480602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f087c125d40 0x7f087c12a190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f0870014000 tx=0x7f0870017000).stop 2023-10-03T10:10:08.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.590+0000 7f088209e700 1 -- 172.21.15.3:0/4157480602 shutdown_connections 2023-10-03T10:10:08.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.590+0000 7f088209e700 1 --2- 172.21.15.3:0/4157480602 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f087c12dfc0 0x7f087c12a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:08.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.590+0000 7f088209e700 1 --2- 172.21.15.3:0/4157480602 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f087c062a90 0x7f087c129ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:08.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.590+0000 7f088209e700 1 --2- 172.21.15.3:0/4157480602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f087c125d40 0x7f087c12a190 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:08.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.590+0000 7f088209e700 1 -- 172.21.15.3:0/4157480602 >> 172.21.15.3:0/4157480602 conn(0x7f087c0599a0 msgr2=0x7f087c123870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:08.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.590+0000 7f088209e700 1 -- 172.21.15.3:0/4157480602 shutdown_connections 2023-10-03T10:10:08.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.590+0000 7f088209e700 1 -- 172.21.15.3:0/4157480602 wait complete. 2023-10-03T10:10:08.603 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-03T10:10:08.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.736+0000 7f0f21f28700 1 Processor -- start 2023-10-03T10:10:08.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.737+0000 7f0f21f28700 1 -- start start 2023-10-03T10:10:08.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.737+0000 7f0f21f28700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f1c096440 0x7f0f1c096820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:08.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.737+0000 7f0f21f28700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0f1c12e280 0x7f0f1c130670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:08.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.737+0000 7f0f21f28700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f1c098330 con 0x7f0f1c096d10 2023-10-03T10:10:08.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.737+0000 7f0f21f28700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f1c097800 con 0x7f0f1c12e280 2023-10-03T10:10:08.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.737+0000 7f0f21f28700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c06ab40 con 0x7f0f1c096440 2023-10-03T10:10:08.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.737+0000 7f0f1b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f1c096440 0x7f0f1c096820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:08.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.737+0000 7f0f1bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0f1c12e280 0x7f0f1c130670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:08.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.737+0000 7f0f1affd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0f1c096d10 0x7f0f1c0970f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47472/0 (socket says 172.21.15.3:47472) 2023-10-03T10:10:08.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.737+0000 7f0f1b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f1c096440 0x7f0f1c096820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53904/0 (socket says 172.21.15.3:53904) 2023-10-03T10:10:08.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.737+0000 7f0f1affd700 1 -- 172.21.15.3:0/1923144451 learned_addr learned my addr 172.21.15.3:0/1923144451 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:10:08.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.738+0000 7f0f1b7fe700 1 -- 172.21.15.3:0/1923144451 >> v2:172.21.15.3:3300/0 conn(0x7f0f1c12e280 msgr2=0x7f0f1c130670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:08.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.738+0000 7f0f1b7fe700 1 --2- 172.21.15.3:0/1923144451 >> v2:172.21.15.3:3300/0 conn(0x7f0f1c12e280 0x7f0f1c130670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:08.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.738+0000 7f0f1b7fe700 1 -- 172.21.15.3:0/1923144451 >> v1:172.21.15.3:6789/0 conn(0x7f0f1c096d10 legacy=0x7f0f1c0970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:08.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.738+0000 7f0f1b7fe700 1 -- 172.21.15.3:0/1923144451 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1c068cf0 con 0x7f0f1c096440 2023-10-03T10:10:08.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.738+0000 7f0f1a7fc700 1 -- 172.21.15.3:0/1923144451 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3895640288 0 0) 0x7f0f1c098330 con 0x7f0f1c096d10 2023-10-03T10:10:08.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.738+0000 7f0f1bfff700 1 --2- 172.21.15.3:0/1923144451 >> v2:172.21.15.3:3300/0 conn(0x7f0f1c12e280 0x7f0f1c130670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:08.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.738+0000 7f0f1b7fe700 1 --2- 172.21.15.3:0/1923144451 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f1c096440 0x7f0f1c096820 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f0f0c009760 tx=0x7f0f0c009060).ready entity=mon.1 client_cookie=d226ae320a555881 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:08.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.738+0000 7f0f1a7fc700 1 -- 172.21.15.3:0/1923144451 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c0090d0 con 0x7f0f1c096440 2023-10-03T10:10:08.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.738+0000 7f0f1a7fc700 1 -- 172.21.15.3:0/1923144451 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0c00f510 con 0x7f0f1c096440 2023-10-03T10:10:08.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.738+0000 7f0f1a7fc700 1 -- 172.21.15.3:0/1923144451 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c00f890 con 0x7f0f1c096440 2023-10-03T10:10:08.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.738+0000 7f0f21f28700 1 -- 172.21.15.3:0/1923144451 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f1c096440 msgr2=0x7f0f1c096820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:08.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.738+0000 7f0f21f28700 1 --2- 172.21.15.3:0/1923144451 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f1c096440 0x7f0f1c096820 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f0f0c009760 tx=0x7f0f0c009060).stop 2023-10-03T10:10:08.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.738+0000 7f0f21f28700 1 -- 172.21.15.3:0/1923144451 shutdown_connections 2023-10-03T10:10:08.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.738+0000 7f0f21f28700 1 --2- 172.21.15.3:0/1923144451 >> v2:172.21.15.3:3300/0 conn(0x7f0f1c12e280 0x7f0f1c130670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:08.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.738+0000 7f0f21f28700 1 --2- 172.21.15.3:0/1923144451 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f1c096440 0x7f0f1c096820 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:08.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.739+0000 7f0f21f28700 1 -- 172.21.15.3:0/1923144451 >> 172.21.15.3:0/1923144451 conn(0x7f0f1c059bb0 msgr2=0x7f0f1c09ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:08.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.739+0000 7f0f21f28700 1 -- 172.21.15.3:0/1923144451 shutdown_connections 2023-10-03T10:10:08.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.739+0000 7f0f21f28700 1 -- 172.21.15.3:0/1923144451 wait complete. 2023-10-03T10:10:08.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.740+0000 7f0f21f28700 1 Processor -- start 2023-10-03T10:10:08.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.740+0000 7f0f21f28700 1 -- start start 2023-10-03T10:10:08.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.740+0000 7f0f21f28700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f1c096440 0x7f0f1c064660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:08.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.740+0000 7f0f21f28700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f1c096d10 0x7f0f1c064b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:08.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.740+0000 7f0f21f28700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f1c12e280 0x7f0f1c065040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:08.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.740+0000 7f0f21f28700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c065530 con 0x7f0f1c096440 2023-10-03T10:10:08.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.740+0000 7f0f21f28700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c11e720 con 0x7f0f1c096d10 2023-10-03T10:10:08.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.740+0000 7f0f21f28700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c0661f0 con 0x7f0f1c12e280 2023-10-03T10:10:08.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.740+0000 7f0f1b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f1c096440 0x7f0f1c064660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:08.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.740+0000 7f0f1b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f1c096440 0x7f0f1c064660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47540/0 (socket says 172.21.15.3:47540) 2023-10-03T10:10:08.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.740+0000 7f0f1b7fe700 1 -- 172.21.15.3:0/4277386629 learned_addr learned my addr 172.21.15.3:0/4277386629 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:08.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.740+0000 7f0f1bfff700 1 --2- 172.21.15.3:0/4277386629 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f1c12e280 0x7f0f1c065040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:08.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.740+0000 7f0f1affd700 1 --2- 172.21.15.3:0/4277386629 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f1c096d10 0x7f0f1c064b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:08.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.741+0000 7f0f1b7fe700 1 -- 172.21.15.3:0/4277386629 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f1c12e280 msgr2=0x7f0f1c065040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:08.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.741+0000 7f0f1b7fe700 1 --2- 172.21.15.3:0/4277386629 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f1c12e280 0x7f0f1c065040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:08.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.741+0000 7f0f1b7fe700 1 -- 172.21.15.3:0/4277386629 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f1c096d10 msgr2=0x7f0f1c064b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:08.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.741+0000 7f0f1b7fe700 1 --2- 172.21.15.3:0/4277386629 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f1c096d10 0x7f0f1c064b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:08.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.741+0000 7f0f1b7fe700 1 -- 172.21.15.3:0/4277386629 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f14009a20 con 0x7f0f1c096440 2023-10-03T10:10:08.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.741+0000 7f0f1b7fe700 1 --2- 172.21.15.3:0/4277386629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f1c096440 0x7f0f1c064660 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f0f0c009150 tx=0x7f0f0c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:08.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.741+0000 7f0f037fe700 1 -- 172.21.15.3:0/4277386629 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c0069b0 con 0x7f0f1c096440 2023-10-03T10:10:08.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.741+0000 7f0f037fe700 1 -- 172.21.15.3:0/4277386629 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0c00fc50 con 0x7f0f1c096440 2023-10-03T10:10:08.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.741+0000 7f0f21f28700 1 -- 172.21.15.3:0/4277386629 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f0c0092a0 con 0x7f0f1c096440 2023-10-03T10:10:08.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.741+0000 7f0f037fe700 1 -- 172.21.15.3:0/4277386629 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c00f890 con 0x7f0f1c096440 2023-10-03T10:10:08.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.741+0000 7f0f21f28700 1 -- 172.21.15.3:0/4277386629 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f1c067dc0 con 0x7f0f1c096440 2023-10-03T10:10:08.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.742+0000 7f0f037fe700 1 -- 172.21.15.3:0/4277386629 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f0f0c01b650 con 0x7f0f1c096440 2023-10-03T10:10:08.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.742+0000 7f0f037fe700 1 -- 172.21.15.3:0/4277386629 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f0f0c03c5d0 con 0x7f0f1c096440 2023-10-03T10:10:08.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.742+0000 7f0f21f28700 1 -- 172.21.15.3:0/4277386629 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f1c098db0 con 0x7f0f1c096440 2023-10-03T10:10:08.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.747+0000 7f0f037fe700 1 -- 172.21.15.3:0/4277386629 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f0f0c028020 con 0x7f0f1c096440 2023-10-03T10:10:08.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.920+0000 7f0f21f28700 1 -- 172.21.15.3:0/4277386629 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0f1c11c680 con 0x7f0f1c096440 2023-10-03T10:10:08.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.920+0000 7f0f037fe700 1 -- 172.21.15.3:0/4277386629 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v18) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0f0c018900 con 0x7f0f1c096440 2023-10-03T10:10:08.921 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-10-03T10:10:08.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.922+0000 7f0f21f28700 1 -- 172.21.15.3:0/4277386629 >> v1:172.21.15.3:6816/118296 conn(0x7f0f0802bb90 legacy=0x7f0f0802e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:08.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.922+0000 7f0f21f28700 1 -- 172.21.15.3:0/4277386629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f1c096440 msgr2=0x7f0f1c064660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:08.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.922+0000 7f0f21f28700 1 --2- 172.21.15.3:0/4277386629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f1c096440 0x7f0f1c064660 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f0f0c009150 tx=0x7f0f0c017000).stop 2023-10-03T10:10:08.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.922+0000 7f0f21f28700 1 -- 172.21.15.3:0/4277386629 shutdown_connections 2023-10-03T10:10:08.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.922+0000 7f0f21f28700 1 --2- 172.21.15.3:0/4277386629 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f1c12e280 0x7f0f1c065040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:08.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.922+0000 7f0f21f28700 1 --2- 172.21.15.3:0/4277386629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f1c096440 0x7f0f1c064660 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:08.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.922+0000 7f0f21f28700 1 --2- 172.21.15.3:0/4277386629 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f1c096d10 0x7f0f1c064b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:08.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.922+0000 7f0f21f28700 1 -- 172.21.15.3:0/4277386629 >> 172.21.15.3:0/4277386629 conn(0x7f0f1c059bb0 msgr2=0x7f0f1c09a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:08.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.922+0000 7f0f21f28700 1 -- 172.21.15.3:0/4277386629 shutdown_connections 2023-10-03T10:10:08.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:08.922+0000 7f0f21f28700 1 -- 172.21.15.3:0/4277386629 wait complete. 2023-10-03T10:10:08.936 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-03T10:10:09.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.066+0000 7fb8f222b700 1 Processor -- start 2023-10-03T10:10:09.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.066+0000 7fb8f222b700 1 -- start start 2023-10-03T10:10:09.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.066+0000 7fb8f222b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb8ec062a90 0x7fb8ec127860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:09.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.066+0000 7fb8f222b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8ec127360 0x7fb8ec127d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:09.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.066+0000 7fb8f222b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8ec0968b0 con 0x7fb8ec095300 2023-10-03T10:10:09.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.066+0000 7fb8f222b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8ec062e70 con 0x7fb8ec062a90 2023-10-03T10:10:09.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.066+0000 7fb8f222b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec091bc0 con 0x7fb8ec127360 2023-10-03T10:10:09.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.066+0000 7fb8eb7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb8ec062a90 0x7fb8ec127860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:09.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.066+0000 7fb8ebfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb8ec095300 0x7fb8ec0957b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47488/0 (socket says 172.21.15.3:47488) 2023-10-03T10:10:09.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.066+0000 7fb8ebfff700 1 -- 172.21.15.3:0/2299647307 learned_addr learned my addr 172.21.15.3:0/2299647307 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:10:09.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.066+0000 7fb8eaffd700 1 --2- 172.21.15.3:0/2299647307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8ec127360 0x7fb8ec127d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:09.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.067+0000 7fb8eb7fe700 1 -- 172.21.15.3:0/2299647307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8ec127360 msgr2=0x7fb8ec127d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:09.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.067+0000 7fb8eb7fe700 1 --2- 172.21.15.3:0/2299647307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8ec127360 0x7fb8ec127d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:09.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.067+0000 7fb8eb7fe700 1 -- 172.21.15.3:0/2299647307 >> v1:172.21.15.3:6789/0 conn(0x7fb8ec095300 legacy=0x7fb8ec0957b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:09.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.067+0000 7fb8eb7fe700 1 -- 172.21.15.3:0/2299647307 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ec092a70 con 0x7fb8ec062a90 2023-10-03T10:10:09.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.067+0000 7fb8ea7fc700 1 -- 172.21.15.3:0/2299647307 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1436013711 0 0) 0x7fb8e4001490 con 0x7fb8ec095300 2023-10-03T10:10:09.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.067+0000 7fb8eb7fe700 1 --2- 172.21.15.3:0/2299647307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8ec062a90 0x7fb8ec127860 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fb8dc009790 tx=0x7fb8dc009060).ready entity=mon.0 client_cookie=172f3197da05775f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:09.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.067+0000 7fb8ea7fc700 1 -- 172.21.15.3:0/2299647307 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc0090d0 con 0x7fb8ec062a90 2023-10-03T10:10:09.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.067+0000 7fb8ea7fc700 1 -- 172.21.15.3:0/2299647307 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8dc00f510 con 0x7fb8ec062a90 2023-10-03T10:10:09.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.067+0000 7fb8ea7fc700 1 -- 172.21.15.3:0/2299647307 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc00f890 con 0x7fb8ec062a90 2023-10-03T10:10:09.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.068+0000 7fb8f222b700 1 -- 172.21.15.3:0/2299647307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8ec062a90 msgr2=0x7fb8ec127860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:09.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.068+0000 7fb8f222b700 1 --2- 172.21.15.3:0/2299647307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8ec062a90 0x7fb8ec127860 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fb8dc009790 tx=0x7fb8dc009060).stop 2023-10-03T10:10:09.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.068+0000 7fb8f222b700 1 -- 172.21.15.3:0/2299647307 shutdown_connections 2023-10-03T10:10:09.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.068+0000 7fb8f222b700 1 --2- 172.21.15.3:0/2299647307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8ec062a90 0x7fb8ec127860 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:09.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.068+0000 7fb8f222b700 1 --2- 172.21.15.3:0/2299647307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8ec127360 0x7fb8ec127d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:09.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.068+0000 7fb8f222b700 1 -- 172.21.15.3:0/2299647307 >> 172.21.15.3:0/2299647307 conn(0x7fb8ec059bb0 msgr2=0x7fb8ec097400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:09.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.068+0000 7fb8f222b700 1 -- 172.21.15.3:0/2299647307 shutdown_connections 2023-10-03T10:10:09.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.068+0000 7fb8f222b700 1 -- 172.21.15.3:0/2299647307 wait complete. 2023-10-03T10:10:09.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.069+0000 7fb8f222b700 1 Processor -- start 2023-10-03T10:10:09.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.069+0000 7fb8f222b700 1 -- start start 2023-10-03T10:10:09.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.069+0000 7fb8f222b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8ec062a90 0x7fb8ec117bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:09.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.069+0000 7fb8f222b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb8ec095300 0x7fb8ec113c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:09.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.069+0000 7fb8f222b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8ec127360 0x7fb8ec114120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:09.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.069+0000 7fb8f222b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec124d00 con 0x7fb8ec062a90 2023-10-03T10:10:09.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.069+0000 7fb8f222b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec096e90 con 0x7fb8ec127360 2023-10-03T10:10:09.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.069+0000 7fb8f222b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec117130 con 0x7fb8ec095300 2023-10-03T10:10:09.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.070+0000 7fb8eb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8ec062a90 0x7fb8ec117bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:09.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.070+0000 7fb8eaffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb8ec095300 0x7fb8ec113c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:09.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.070+0000 7fb8eb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8ec062a90 0x7fb8ec117bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47552/0 (socket says 172.21.15.3:47552) 2023-10-03T10:10:09.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.070+0000 7fb8eaffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb8ec095300 0x7fb8ec113c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:60376/0 (socket says 172.21.15.3:60376) 2023-10-03T10:10:09.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.070+0000 7fb8eb7fe700 1 -- 172.21.15.3:0/1781262198 learned_addr learned my addr 172.21.15.3:0/1781262198 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:09.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.070+0000 7fb8ebfff700 1 --2- 172.21.15.3:0/1781262198 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8ec127360 0x7fb8ec114120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:09.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.070+0000 7fb8eb7fe700 1 -- 172.21.15.3:0/1781262198 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb8ec095300 msgr2=0x7fb8ec113c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:09.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.070+0000 7fb8eb7fe700 1 --2- 172.21.15.3:0/1781262198 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb8ec095300 0x7fb8ec113c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:09.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.070+0000 7fb8eb7fe700 1 -- 172.21.15.3:0/1781262198 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8ec127360 msgr2=0x7fb8ec114120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:09.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.070+0000 7fb8eb7fe700 1 --2- 172.21.15.3:0/1781262198 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8ec127360 0x7fb8ec114120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:09.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.070+0000 7fb8eb7fe700 1 -- 172.21.15.3:0/1781262198 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8dc0092a0 con 0x7fb8ec062a90 2023-10-03T10:10:09.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.070+0000 7fb8eb7fe700 1 --2- 172.21.15.3:0/1781262198 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8ec062a90 0x7fb8ec117bd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fb8dc009e30 tx=0x7fb8dc017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:09.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.070+0000 7fb8d37fe700 1 -- 172.21.15.3:0/1781262198 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc00f890 con 0x7fb8ec062a90 2023-10-03T10:10:09.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.070+0000 7fb8d37fe700 1 -- 172.21.15.3:0/1781262198 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8dc00fc50 con 0x7fb8ec062a90 2023-10-03T10:10:09.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.070+0000 7fb8f222b700 1 -- 172.21.15.3:0/1781262198 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8ec1176d0 con 0x7fb8ec062a90 2023-10-03T10:10:09.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.070+0000 7fb8d37fe700 1 -- 172.21.15.3:0/1781262198 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc01b440 con 0x7fb8ec062a90 2023-10-03T10:10:09.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.070+0000 7fb8f222b700 1 -- 172.21.15.3:0/1781262198 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8ec1157c0 con 0x7fb8ec062a90 2023-10-03T10:10:09.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.071+0000 7fb8d37fe700 1 -- 172.21.15.3:0/1781262198 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fb8dc01b7f0 con 0x7fb8ec062a90 2023-10-03T10:10:09.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.071+0000 7fb8d37fe700 1 -- 172.21.15.3:0/1781262198 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fb8dc03c600 con 0x7fb8ec062a90 2023-10-03T10:10:09.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.071+0000 7fb8f222b700 1 -- 172.21.15.3:0/1781262198 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8ec1159b0 con 0x7fb8ec062a90 2023-10-03T10:10:09.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.076+0000 7fb8d37fe700 1 -- 172.21.15.3:0/1781262198 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fb8dc01a3a0 con 0x7fb8ec062a90 2023-10-03T10:10:09.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.295+0000 7fb8f222b700 1 -- 172.21.15.3:0/1781262198 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8ec0960c0 con 0x7fb8ec062a90 2023-10-03T10:10:09.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.296+0000 7fb8d37fe700 1 -- 172.21.15.3:0/1781262198 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2154 (secure 0 0 0) 0x7fb8dc018920 con 0x7fb8ec062a90 2023-10-03T10:10:09.296 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:10:09.296 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-03T10:09:44.141304+0000","modified":"2023-10-03T10:09:47.154287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4186},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4186":{"gid":4186,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6817/2582564779","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6817","nonce":2582564779}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-03T10:10:09.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.297+0000 7fb8f222b700 1 -- 172.21.15.3:0/1781262198 >> v1:172.21.15.3:6816/118296 conn(0x7fb8d802bb60 legacy=0x7fb8d802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:09.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.297+0000 7fb8f222b700 1 -- 172.21.15.3:0/1781262198 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8ec062a90 msgr2=0x7fb8ec117bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:09.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.298+0000 7fb8f222b700 1 --2- 172.21.15.3:0/1781262198 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8ec062a90 0x7fb8ec117bd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fb8dc009e30 tx=0x7fb8dc017000).stop 2023-10-03T10:10:09.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.298+0000 7fb8f222b700 1 -- 172.21.15.3:0/1781262198 shutdown_connections 2023-10-03T10:10:09.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.298+0000 7fb8f222b700 1 --2- 172.21.15.3:0/1781262198 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8ec127360 0x7fb8ec114120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:09.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.298+0000 7fb8f222b700 1 --2- 172.21.15.3:0/1781262198 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8ec062a90 0x7fb8ec117bd0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:09.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.298+0000 7fb8f222b700 1 --2- 172.21.15.3:0/1781262198 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb8ec095300 0x7fb8ec113c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:09.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.298+0000 7fb8f222b700 1 -- 172.21.15.3:0/1781262198 >> 172.21.15.3:0/1781262198 conn(0x7fb8ec059bb0 msgr2=0x7fb8ec09b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:09.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.298+0000 7fb8f222b700 1 -- 172.21.15.3:0/1781262198 shutdown_connections 2023-10-03T10:10:09.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.298+0000 7fb8f222b700 1 -- 172.21.15.3:0/1781262198 wait complete. 2023-10-03T10:10:09.299 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 6 2023-10-03T10:10:09.311 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-03T10:10:09.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.441+0000 7f2696a13700 1 Processor -- start 2023-10-03T10:10:09.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.441+0000 7f2696a13700 1 -- start start 2023-10-03T10:10:09.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.442+0000 7f2696a13700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2690096c40 0x7f2690097020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:09.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.442+0000 7f2696a13700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f269012e310 0x7f2690130700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:09.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.442+0000 7f2696a13700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f269006aba0 con 0x7f269006a7c0 2023-10-03T10:10:09.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.442+0000 7f2696a13700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2690097980 con 0x7f2690096c40 2023-10-03T10:10:09.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.442+0000 7f2696a13700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2690069d80 con 0x7f269012e310 2023-10-03T10:10:09.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.442+0000 7f2694fb0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f269012e310 0x7f2690130700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:09.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.442+0000 7f268f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2690096c40 0x7f2690097020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:09.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.442+0000 7f2694fb0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f269012e310 0x7f2690130700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53946/0 (socket says 172.21.15.3:53946) 2023-10-03T10:10:09.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.442+0000 7f2694fb0700 1 -- 172.21.15.3:0/3096390769 learned_addr learned my addr 172.21.15.3:0/3096390769 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:09.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.442+0000 7f2694fb0700 1 -- 172.21.15.3:0/3096390769 >> v2:172.21.15.3:3300/0 conn(0x7f2690096c40 msgr2=0x7f2690097020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:09.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.442+0000 7f2694fb0700 1 --2- 172.21.15.3:0/3096390769 >> v2:172.21.15.3:3300/0 conn(0x7f2690096c40 0x7f2690097020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:09.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.442+0000 7f2694fb0700 1 -- 172.21.15.3:0/3096390769 >> v1:172.21.15.3:6789/0 conn(0x7f269006a7c0 legacy=0x7f2690096530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:09.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.443+0000 7f2694fb0700 1 -- 172.21.15.3:0/3096390769 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2690068c00 con 0x7f269012e310 2023-10-03T10:10:09.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.443+0000 7f268effd700 1 -- 172.21.15.3:0/3096390769 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2896557520 0 0) 0x7f269006aba0 con 0x7f269006a7c0 2023-10-03T10:10:09.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.443+0000 7f268f7fe700 1 --2- 172.21.15.3:0/3096390769 >> v2:172.21.15.3:3300/0 conn(0x7f2690096c40 0x7f2690097020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:09.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.443+0000 7f2694fb0700 1 --2- 172.21.15.3:0/3096390769 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f269012e310 0x7f2690130700 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f2688009f50 tx=0x7f2688009840).ready entity=mon.1 client_cookie=b41adc144e64e15c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:09.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.443+0000 7f268effd700 1 -- 172.21.15.3:0/3096390769 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268800e070 con 0x7f269012e310 2023-10-03T10:10:09.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.443+0000 7f268effd700 1 -- 172.21.15.3:0/3096390769 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268800b9c0 con 0x7f269012e310 2023-10-03T10:10:09.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.443+0000 7f268effd700 1 -- 172.21.15.3:0/3096390769 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268800bd40 con 0x7f269012e310 2023-10-03T10:10:09.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.443+0000 7f2696a13700 1 -- 172.21.15.3:0/3096390769 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f269012e310 msgr2=0x7f2690130700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:09.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.443+0000 7f2696a13700 1 --2- 172.21.15.3:0/3096390769 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f269012e310 0x7f2690130700 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f2688009f50 tx=0x7f2688009840).stop 2023-10-03T10:10:09.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.443+0000 7f2696a13700 1 -- 172.21.15.3:0/3096390769 shutdown_connections 2023-10-03T10:10:09.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.443+0000 7f2696a13700 1 --2- 172.21.15.3:0/3096390769 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f269012e310 0x7f2690130700 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:09.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.443+0000 7f2696a13700 1 --2- 172.21.15.3:0/3096390769 >> v2:172.21.15.3:3300/0 conn(0x7f2690096c40 0x7f2690097020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:09.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.443+0000 7f2696a13700 1 -- 172.21.15.3:0/3096390769 >> 172.21.15.3:0/3096390769 conn(0x7f269009bb10 msgr2=0x7f269009bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:09.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.443+0000 7f2696a13700 1 -- 172.21.15.3:0/3096390769 shutdown_connections 2023-10-03T10:10:09.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.444+0000 7f2696a13700 1 -- 172.21.15.3:0/3096390769 wait complete. 2023-10-03T10:10:09.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.444+0000 7f2696a13700 1 Processor -- start 2023-10-03T10:10:09.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.445+0000 7f2696a13700 1 -- start start 2023-10-03T10:10:09.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.445+0000 7f2696a13700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f269006a7c0 0x7f269011ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:09.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.445+0000 7f2696a13700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2690096c40 0x7f2690064660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:09.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.445+0000 7f2696a13700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f269012e310 0x7f2690064b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:09.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.445+0000 7f2696a13700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269011d960 con 0x7f2690096c40 2023-10-03T10:10:09.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.445+0000 7f2696a13700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2690099110 con 0x7f269012e310 2023-10-03T10:10:09.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.445+0000 7f2696a13700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2690067fa0 con 0x7f269006a7c0 2023-10-03T10:10:09.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.445+0000 7f268ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f269006a7c0 0x7f269011ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:09.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.445+0000 7f268f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2690096c40 0x7f2690064660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:09.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.445+0000 7f268f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2690096c40 0x7f2690064660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47566/0 (socket says 172.21.15.3:47566) 2023-10-03T10:10:09.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.445+0000 7f268f7fe700 1 -- 172.21.15.3:0/2353467026 learned_addr learned my addr 172.21.15.3:0/2353467026 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:09.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.445+0000 7f268ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f269006a7c0 0x7f269011ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:60388/0 (socket says 172.21.15.3:60388) 2023-10-03T10:10:09.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.445+0000 7f2694fb0700 1 --2- 172.21.15.3:0/2353467026 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f269012e310 0x7f2690064b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:09.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.446+0000 7f268f7fe700 1 -- 172.21.15.3:0/2353467026 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f269006a7c0 msgr2=0x7f269011ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:09.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.446+0000 7f268f7fe700 1 --2- 172.21.15.3:0/2353467026 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f269006a7c0 0x7f269011ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:09.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.446+0000 7f268f7fe700 1 -- 172.21.15.3:0/2353467026 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f269012e310 msgr2=0x7f2690064b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:09.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.446+0000 7f268f7fe700 1 --2- 172.21.15.3:0/2353467026 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f269012e310 0x7f2690064b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:09.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.446+0000 7f268f7fe700 1 -- 172.21.15.3:0/2353467026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26840092a0 con 0x7f2690096c40 2023-10-03T10:10:09.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.446+0000 7f268f7fe700 1 --2- 172.21.15.3:0/2353467026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2690096c40 0x7f2690064660 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f268400ea20 tx=0x7f2684009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:09.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.446+0000 7f2677fff700 1 -- 172.21.15.3:0/2353467026 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268400ee60 con 0x7f2690096c40 2023-10-03T10:10:09.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.446+0000 7f2677fff700 1 -- 172.21.15.3:0/2353467026 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268400c850 con 0x7f2690096c40 2023-10-03T10:10:09.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.446+0000 7f2696a13700 1 -- 172.21.15.3:0/2353467026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2688009ae0 con 0x7f2690096c40 2023-10-03T10:10:09.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.446+0000 7f2677fff700 1 -- 172.21.15.3:0/2353467026 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268400cbd0 con 0x7f2690096c40 2023-10-03T10:10:09.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.446+0000 7f2696a13700 1 -- 172.21.15.3:0/2353467026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2690066280 con 0x7f2690096c40 2023-10-03T10:10:09.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.447+0000 7f2677fff700 1 -- 172.21.15.3:0/2353467026 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f268401b400 con 0x7f2690096c40 2023-10-03T10:10:09.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.447+0000 7f2677fff700 1 -- 172.21.15.3:0/2353467026 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f268403b4c0 con 0x7f2690096c40 2023-10-03T10:10:09.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.447+0000 7f2696a13700 1 -- 172.21.15.3:0/2353467026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f269009c360 con 0x7f2690096c40 2023-10-03T10:10:09.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.452+0000 7f2677fff700 1 -- 172.21.15.3:0/2353467026 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f2684019900 con 0x7f2690096c40 2023-10-03T10:10:09.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.628+0000 7f2696a13700 1 -- 172.21.15.3:0/2353467026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f269005a260 con 0x7f2690096c40 2023-10-03T10:10:09.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.628+0000 7f2677fff700 1 -- 172.21.15.3:0/2353467026 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+11308 (secure 0 0 0) 0x7f2684019310 con 0x7f2690096c40 2023-10-03T10:10:09.629 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:10:09.629 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":18,"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","created":"2023-10-03T10:09:28.703441+0000","modified":"2023-10-03T10:10:04.610398+0000","last_up_change":"2023-10-03T10:09:36.569401+0000","last_in_change":"2023-10-03T10:09:30.156546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-10-03T10:09:36.623971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-03T10:09:38.210614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-03T10:09:42.695873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-03T10:09:43.115588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cdcdd930-3c5b-4486-a021-7132f90fb6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6808","nonce":118845}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6809","nonce":118845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6811","nonce":118845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6810","nonce":118845}]},"public_addr":"172.21.15.3:6808/118845","cluster_addr":"172.21.15.3:6809/118845","heartbeat_back_addr":"172.21.15.3:6811/118845","heartbeat_front_addr":"172.21.15.3:6810/118845","state":["exists","up"]},{"osd":1,"uuid":"c8e58877-6762-4741-a8c1-567ebc97b708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6804","nonce":118835}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6805","nonce":118835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6807","nonce":118835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6806","nonce":118835}]},"public_addr":"172.21.15.3:6804/118835","cluster_addr":"172.21.15.3:6805/118835","heartbeat_back_addr":"172.21.15.3:6807/118835","heartbeat_front_addr":"172.21.15.3:6806/118835","state":["exists","up"]},{"osd":2,"uuid":"68a2a21a-7a8e-4f82-8433-5201faa50777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6812","nonce":118864}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6813","nonce":118864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6815","nonce":118864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6814","nonce":118864}]},"public_addr":"172.21.15.3:6812/118864","cluster_addr":"172.21.15.3:6813/118864","heartbeat_back_addr":"172.21.15.3:6815/118864","heartbeat_front_addr":"172.21.15.3:6814/118864","state":["exists","up"]},{"osd":3,"uuid":"c4df2903-4e1e-4208-bf77-95b9fa749241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6800","nonce":118836}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6801","nonce":118836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6803","nonce":118836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6802","nonce":118836}]},"public_addr":"172.21.15.3:6800/118836","cluster_addr":"172.21.15.3:6801/118836","heartbeat_back_addr":"172.21.15.3:6803/118836","heartbeat_front_addr":"172.21.15.3:6802/118836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.369708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.257825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.337931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.085222+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-03T10:10:09.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.631+0000 7f2696a13700 1 -- 172.21.15.3:0/2353467026 >> v1:172.21.15.3:6816/118296 conn(0x7f267c02bb90 legacy=0x7f267c02e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:09.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.631+0000 7f2696a13700 1 -- 172.21.15.3:0/2353467026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2690096c40 msgr2=0x7f2690064660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:09.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.631+0000 7f2696a13700 1 --2- 172.21.15.3:0/2353467026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2690096c40 0x7f2690064660 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f268400ea20 tx=0x7f2684009060).stop 2023-10-03T10:10:09.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.631+0000 7f2696a13700 1 -- 172.21.15.3:0/2353467026 shutdown_connections 2023-10-03T10:10:09.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.631+0000 7f2696a13700 1 --2- 172.21.15.3:0/2353467026 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f269012e310 0x7f2690064b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:09.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.631+0000 7f2696a13700 1 --2- 172.21.15.3:0/2353467026 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f269006a7c0 0x7f269011ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:09.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.631+0000 7f2696a13700 1 --2- 172.21.15.3:0/2353467026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2690096c40 0x7f2690064660 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:09.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.631+0000 7f2696a13700 1 -- 172.21.15.3:0/2353467026 >> 172.21.15.3:0/2353467026 conn(0x7f269009bb10 msgr2=0x7f2690094850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:09.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.631+0000 7f2696a13700 1 -- 172.21.15.3:0/2353467026 shutdown_connections 2023-10-03T10:10:09.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.631+0000 7f2696a13700 1 -- 172.21.15.3:0/2353467026 wait complete. 2023-10-03T10:10:09.645 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-03T10:10:09.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.779+0000 7fbc9d343700 1 Processor -- start 2023-10-03T10:10:09.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.779+0000 7fbc9d343700 1 -- start start 2023-10-03T10:10:09.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.780+0000 7fbc9d343700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc98097d00 0x7fbc98098df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:09.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.780+0000 7fbc9d343700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc980992e0 0x7fbc981324b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:09.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.780+0000 7fbc9d343700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc98098190 con 0x7fbc98065340 2023-10-03T10:10:09.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.780+0000 7fbc9d343700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc98065720 con 0x7fbc980992e0 2023-10-03T10:10:09.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.780+0000 7fbc9d343700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbc98096190 con 0x7fbc98097d00 2023-10-03T10:10:09.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.780+0000 7fbc967fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc98097d00 0x7fbc98098df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:09.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.780+0000 7fbc977fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc980992e0 0x7fbc981324b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:09.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.780+0000 7fbc967fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc98097d00 0x7fbc98098df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:60398/0 (socket says 172.21.15.3:60398) 2023-10-03T10:10:09.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.780+0000 7fbc967fc700 1 -- 172.21.15.3:0/1550270705 learned_addr learned my addr 172.21.15.3:0/1550270705 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:09.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.780+0000 7fbc967fc700 1 -- 172.21.15.3:0/1550270705 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc980992e0 msgr2=0x7fbc981324b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:09.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.780+0000 7fbc967fc700 1 --2- 172.21.15.3:0/1550270705 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc980992e0 0x7fbc981324b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:09.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.780+0000 7fbc95ffb700 1 -- 172.21.15.3:0/1550270705 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 719077929 0 0) 0x7fbc98098190 con 0x7fbc98065340 2023-10-03T10:10:09.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.780+0000 7fbc967fc700 1 -- 172.21.15.3:0/1550270705 >> v1:172.21.15.3:6789/0 conn(0x7fbc98065340 legacy=0x7fbc980986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:09.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.780+0000 7fbc967fc700 1 -- 172.21.15.3:0/1550270705 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc98066010 con 0x7fbc98097d00 2023-10-03T10:10:09.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.780+0000 7fbc977fe700 1 --2- 172.21.15.3:0/1550270705 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc980992e0 0x7fbc981324b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:09.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.781+0000 7fbc967fc700 1 --2- 172.21.15.3:0/1550270705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc98097d00 0x7fbc98098df0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fbc8c009830 tx=0x7fbc8c0090f0).ready entity=mon.2 client_cookie=8c4fafd0d60320e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:09.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.781+0000 7fbc95ffb700 1 -- 172.21.15.3:0/1550270705 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c00b580 con 0x7fbc98097d00 2023-10-03T10:10:09.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.781+0000 7fbc95ffb700 1 -- 172.21.15.3:0/1550270705 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc8c00b6e0 con 0x7fbc98097d00 2023-10-03T10:10:09.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.781+0000 7fbc95ffb700 1 -- 172.21.15.3:0/1550270705 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c00ba80 con 0x7fbc98097d00 2023-10-03T10:10:09.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.781+0000 7fbc9d343700 1 -- 172.21.15.3:0/1550270705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc98097d00 msgr2=0x7fbc98098df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:09.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.781+0000 7fbc9d343700 1 --2- 172.21.15.3:0/1550270705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc98097d00 0x7fbc98098df0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fbc8c009830 tx=0x7fbc8c0090f0).stop 2023-10-03T10:10:09.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.781+0000 7fbc9d343700 1 -- 172.21.15.3:0/1550270705 shutdown_connections 2023-10-03T10:10:09.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.781+0000 7fbc9d343700 1 --2- 172.21.15.3:0/1550270705 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc980992e0 0x7fbc981324b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:09.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.781+0000 7fbc9d343700 1 --2- 172.21.15.3:0/1550270705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc98097d00 0x7fbc98098df0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:09.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.781+0000 7fbc9d343700 1 -- 172.21.15.3:0/1550270705 >> 172.21.15.3:0/1550270705 conn(0x7fbc9809dcc0 msgr2=0x7fbc9809e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:09.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.781+0000 7fbc9d343700 1 -- 172.21.15.3:0/1550270705 shutdown_connections 2023-10-03T10:10:09.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.781+0000 7fbc9d343700 1 -- 172.21.15.3:0/1550270705 wait complete. 2023-10-03T10:10:09.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.782+0000 7fbc9d343700 1 Processor -- start 2023-10-03T10:10:09.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.783+0000 7fbc9d343700 1 -- start start 2023-10-03T10:10:09.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.783+0000 7fbc9d343700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc98065340 0x7fbc98117130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:09.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.783+0000 7fbc9d343700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc980992e0 0x7fbc98117620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:09.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.783+0000 7fbc9d343700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc98117b10 0x7fbc98140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:09.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.783+0000 7fbc9d343700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc98099ae0 con 0x7fbc98117b10 2023-10-03T10:10:09.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.783+0000 7fbc9d343700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9811a2f0 con 0x7fbc98065340 2023-10-03T10:10:09.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.783+0000 7fbc9d343700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbc98117f50 con 0x7fbc980992e0 2023-10-03T10:10:09.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.783+0000 7fbc96ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc98065340 0x7fbc98117130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:09.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.783+0000 7fbc96ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc98065340 0x7fbc98117130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53958/0 (socket says 172.21.15.3:53958) 2023-10-03T10:10:09.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.783+0000 7fbc96ffd700 1 -- 172.21.15.3:0/15922263 learned_addr learned my addr 172.21.15.3:0/15922263 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:09.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.783+0000 7fbc967fc700 1 --2- 172.21.15.3:0/15922263 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc980992e0 0x7fbc98117620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:09.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.783+0000 7fbc977fe700 1 --2- 172.21.15.3:0/15922263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc98117b10 0x7fbc98140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:09.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.783+0000 7fbc96ffd700 1 -- 172.21.15.3:0/15922263 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc980992e0 msgr2=0x7fbc98117620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:09.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.783+0000 7fbc96ffd700 1 --2- 172.21.15.3:0/15922263 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc980992e0 0x7fbc98117620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:09.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.784+0000 7fbc96ffd700 1 -- 172.21.15.3:0/15922263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc98117b10 msgr2=0x7fbc98140dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:09.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.784+0000 7fbc96ffd700 1 --2- 172.21.15.3:0/15922263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc98117b10 0x7fbc98140dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:09.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.784+0000 7fbc96ffd700 1 -- 172.21.15.3:0/15922263 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc90009a20 con 0x7fbc98065340 2023-10-03T10:10:09.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.784+0000 7fbc967fc700 1 --2- 172.21.15.3:0/15922263 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc980992e0 0x7fbc98117620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:09.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.784+0000 7fbc96ffd700 1 --2- 172.21.15.3:0/15922263 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc98065340 0x7fbc98117130 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fbc8800c490 tx=0x7fbc88007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:09.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.784+0000 7fbc7effd700 1 -- 172.21.15.3:0/15922263 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8800ce00 con 0x7fbc98065340 2023-10-03T10:10:09.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.784+0000 7fbc7effd700 1 -- 172.21.15.3:0/15922263 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc8800f470 con 0x7fbc98065340 2023-10-03T10:10:09.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.784+0000 7fbc9d343700 1 -- 172.21.15.3:0/15922263 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc8c009370 con 0x7fbc98065340 2023-10-03T10:10:09.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.784+0000 7fbc7effd700 1 -- 172.21.15.3:0/15922263 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8800f830 con 0x7fbc98065340 2023-10-03T10:10:09.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.784+0000 7fbc9d343700 1 -- 172.21.15.3:0/15922263 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc980665e0 con 0x7fbc98065340 2023-10-03T10:10:09.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.785+0000 7fbc9d343700 1 -- 172.21.15.3:0/15922263 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc9809b4a0 con 0x7fbc98065340 2023-10-03T10:10:09.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.785+0000 7fbc7effd700 1 -- 172.21.15.3:0/15922263 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fbc8800fbe0 con 0x7fbc98065340 2023-10-03T10:10:09.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.785+0000 7fbc7effd700 1 -- 172.21.15.3:0/15922263 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fbc8803a4d0 con 0x7fbc98065340 2023-10-03T10:10:09.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:09.790+0000 7fbc7effd700 1 -- 172.21.15.3:0/15922263 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fbc88018260 con 0x7fbc98065340 2023-10-03T10:10:10.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.011+0000 7fbc9d343700 1 -- 172.21.15.3:0/15922263 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fbc9811a5d0 con 0x7fbc98065340 2023-10-03T10:10:10.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.641+0000 7fbc7effd700 1 -- 172.21.15.3:0/15922263 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbc88010af0 con 0x7fbc98065340 2023-10-03T10:10:10.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.644+0000 7fbc9d343700 1 -- 172.21.15.3:0/15922263 >> v1:172.21.15.3:6816/118296 conn(0x7fbc8402bb60 legacy=0x7fbc8402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:10.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.644+0000 7fbc9d343700 1 -- 172.21.15.3:0/15922263 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc98065340 msgr2=0x7fbc98117130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:10.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.644+0000 7fbc9d343700 1 --2- 172.21.15.3:0/15922263 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc98065340 0x7fbc98117130 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fbc8800c490 tx=0x7fbc88007c20).stop 2023-10-03T10:10:10.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.644+0000 7fbc9d343700 1 -- 172.21.15.3:0/15922263 shutdown_connections 2023-10-03T10:10:10.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.644+0000 7fbc9d343700 1 --2- 172.21.15.3:0/15922263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc98117b10 0x7fbc98140dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:10.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.644+0000 7fbc9d343700 1 --2- 172.21.15.3:0/15922263 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc98065340 0x7fbc98117130 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:10.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.644+0000 7fbc9d343700 1 --2- 172.21.15.3:0/15922263 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc980992e0 0x7fbc98117620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:10.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.644+0000 7fbc9d343700 1 -- 172.21.15.3:0/15922263 >> 172.21.15.3:0/15922263 conn(0x7fbc9809dcc0 msgr2=0x7fbc9809bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:10.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.644+0000 7fbc9d343700 1 -- 172.21.15.3:0/15922263 shutdown_connections 2023-10-03T10:10:10.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.644+0000 7fbc9d343700 1 -- 172.21.15.3:0/15922263 wait complete. 2023-10-03T10:10:10.660 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 1 2023-10-03T10:10:10.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.796+0000 7f262c964700 1 Processor -- start 2023-10-03T10:10:10.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.796+0000 7f262c964700 1 -- start start 2023-10-03T10:10:10.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.797+0000 7f262c964700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2628091a40 0x7f2628091e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:10.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.797+0000 7f262c964700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2628092310 0x7f2628127980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:10.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.797+0000 7f262c964700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2628127e70 0x7f26281282f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:10.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.797+0000 7f262c964700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2628092c60 con 0x7f2628127e70 2023-10-03T10:10:10.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.797+0000 7f262c964700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26280950b0 con 0x7f2628092310 2023-10-03T10:10:10.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.797+0000 7f262c964700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f26280947c0 con 0x7f2628091a40 2023-10-03T10:10:10.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.797+0000 7f2626d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2628127e70 0x7f26281282f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:10.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.797+0000 7f2625d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2628092310 0x7f2628127980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:10.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.797+0000 7f262659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2628091a40 0x7f2628091e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:10.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.797+0000 7f2625d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2628092310 0x7f2628127980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53972/0 (socket says 172.21.15.3:53972) 2023-10-03T10:10:10.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.797+0000 7f262659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2628091a40 0x7f2628091e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:60408/0 (socket says 172.21.15.3:60408) 2023-10-03T10:10:10.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.797+0000 7f2626d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2628127e70 0x7f26281282f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47574/0 (socket says 172.21.15.3:47574) 2023-10-03T10:10:10.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.797+0000 7f2625d9b700 1 -- 172.21.15.3:0/184791648 learned_addr learned my addr 172.21.15.3:0/184791648 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:10.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.798+0000 7f2625d9b700 1 -- 172.21.15.3:0/184791648 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2628091a40 msgr2=0x7f2628091e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:10.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.798+0000 7f2625d9b700 1 --2- 172.21.15.3:0/184791648 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2628091a40 0x7f2628091e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:10.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.798+0000 7f2625d9b700 1 -- 172.21.15.3:0/184791648 >> v2:172.21.15.3:3300/0 conn(0x7f2628127e70 msgr2=0x7f26281282f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:10.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.798+0000 7f2625d9b700 1 --2- 172.21.15.3:0/184791648 >> v2:172.21.15.3:3300/0 conn(0x7f2628127e70 0x7f26281282f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:10.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.798+0000 7f2625d9b700 1 -- 172.21.15.3:0/184791648 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26281287e0 con 0x7f2628092310 2023-10-03T10:10:10.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.798+0000 7f2626d9d700 1 --2- 172.21.15.3:0/184791648 >> v2:172.21.15.3:3300/0 conn(0x7f2628127e70 0x7f26281282f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:10.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.798+0000 7f262659c700 1 --2- 172.21.15.3:0/184791648 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2628091a40 0x7f2628091e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:10.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.798+0000 7f2625d9b700 1 --2- 172.21.15.3:0/184791648 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2628092310 0x7f2628127980 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f261c009830 tx=0x7f261c0090f0).ready entity=mon.1 client_cookie=9544ac00c9e09ff2 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:10.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.798+0000 7f262559a700 1 -- 172.21.15.3:0/184791648 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c00f510 con 0x7f2628092310 2023-10-03T10:10:10.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.798+0000 7f262559a700 1 -- 172.21.15.3:0/184791648 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261c00f670 con 0x7f2628092310 2023-10-03T10:10:10.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.798+0000 7f262559a700 1 -- 172.21.15.3:0/184791648 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c00f9f0 con 0x7f2628092310 2023-10-03T10:10:10.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.799+0000 7f262c964700 1 -- 172.21.15.3:0/184791648 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2628092310 msgr2=0x7f2628127980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:10.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.799+0000 7f262c964700 1 --2- 172.21.15.3:0/184791648 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2628092310 0x7f2628127980 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f261c009830 tx=0x7f261c0090f0).stop 2023-10-03T10:10:10.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.799+0000 7f262c964700 1 -- 172.21.15.3:0/184791648 shutdown_connections 2023-10-03T10:10:10.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.799+0000 7f262c964700 1 --2- 172.21.15.3:0/184791648 >> v2:172.21.15.3:3300/0 conn(0x7f2628127e70 0x7f26281282f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:10.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.799+0000 7f262c964700 1 --2- 172.21.15.3:0/184791648 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2628091a40 0x7f2628091e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:10.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.799+0000 7f262c964700 1 --2- 172.21.15.3:0/184791648 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2628092310 0x7f2628127980 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:10.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.799+0000 7f262c964700 1 -- 172.21.15.3:0/184791648 >> 172.21.15.3:0/184791648 conn(0x7f2628063560 msgr2=0x7f262809a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:10.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.799+0000 7f262c964700 1 -- 172.21.15.3:0/184791648 shutdown_connections 2023-10-03T10:10:10.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.799+0000 7f262c964700 1 -- 172.21.15.3:0/184791648 wait complete. 2023-10-03T10:10:10.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.800+0000 7f262c964700 1 Processor -- start 2023-10-03T10:10:10.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.800+0000 7f262c964700 1 -- start start 2023-10-03T10:10:10.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.800+0000 7f262c964700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2628091a40 0x7f2628113360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:10.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.800+0000 7f262c964700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2628092310 0x7f2628113850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:10.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.800+0000 7f262c964700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2628127e70 0x7f2628113d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:10.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.800+0000 7f262c964700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2628113130 con 0x7f2628091a40 2023-10-03T10:10:10.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.800+0000 7f262c964700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2628116260 con 0x7f2628127e70 2023-10-03T10:10:10.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.800+0000 7f262c964700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2628069b10 con 0x7f2628092310 2023-10-03T10:10:10.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.801+0000 7f262659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2628091a40 0x7f2628113360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:10.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.801+0000 7f2626d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2628127e70 0x7f2628113d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:10.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.801+0000 7f262659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2628091a40 0x7f2628113360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47580/0 (socket says 172.21.15.3:47580) 2023-10-03T10:10:10.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.801+0000 7f262659c700 1 -- 172.21.15.3:0/2035738394 learned_addr learned my addr 172.21.15.3:0/2035738394 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:10.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.801+0000 7f2625d9b700 1 --2- 172.21.15.3:0/2035738394 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2628092310 0x7f2628113850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:10.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.801+0000 7f262659c700 1 -- 172.21.15.3:0/2035738394 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2628092310 msgr2=0x7f2628113850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:10.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.801+0000 7f262659c700 1 --2- 172.21.15.3:0/2035738394 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2628092310 0x7f2628113850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:10.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.801+0000 7f262659c700 1 -- 172.21.15.3:0/2035738394 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2628127e70 msgr2=0x7f2628113d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:10.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.801+0000 7f262659c700 1 --2- 172.21.15.3:0/2035738394 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2628127e70 0x7f2628113d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:10.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.801+0000 7f262659c700 1 -- 172.21.15.3:0/2035738394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26180092a0 con 0x7f2628091a40 2023-10-03T10:10:10.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.801+0000 7f2625d9b700 1 --2- 172.21.15.3:0/2035738394 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2628092310 0x7f2628113850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:10.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.801+0000 7f2626d9d700 1 --2- 172.21.15.3:0/2035738394 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2628127e70 0x7f2628113d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:10.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.801+0000 7f262659c700 1 --2- 172.21.15.3:0/2035738394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2628091a40 0x7f2628113360 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f261800ea20 tx=0x7f2618009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:10.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.801+0000 7f26167fc700 1 -- 172.21.15.3:0/2035738394 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261800c850 con 0x7f2628091a40 2023-10-03T10:10:10.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.801+0000 7f26167fc700 1 -- 172.21.15.3:0/2035738394 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261800c9b0 con 0x7f2628091a40 2023-10-03T10:10:10.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.801+0000 7f262c964700 1 -- 172.21.15.3:0/2035738394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f262000bc30 con 0x7f2628091a40 2023-10-03T10:10:10.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.801+0000 7f26167fc700 1 -- 172.21.15.3:0/2035738394 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261800cd30 con 0x7f2628091a40 2023-10-03T10:10:10.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.801+0000 7f262c964700 1 -- 172.21.15.3:0/2035738394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f261c009370 con 0x7f2628091a40 2023-10-03T10:10:10.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.802+0000 7f26167fc700 1 -- 172.21.15.3:0/2035738394 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f2618012600 con 0x7f2628091a40 2023-10-03T10:10:10.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.802+0000 7f26167fc700 1 -- 172.21.15.3:0/2035738394 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f261803b600 con 0x7f2628091a40 2023-10-03T10:10:10.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.802+0000 7f262c964700 1 -- 172.21.15.3:0/2035738394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f262805a260 con 0x7f2628091a40 2023-10-03T10:10:10.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:10.807+0000 7f26167fc700 1 -- 172.21.15.3:0/2035738394 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f261801b400 con 0x7f2628091a40 2023-10-03T10:10:11.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.023+0000 7f262c964700 1 -- 172.21.15.3:0/2035738394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7f2628117000 con 0x7f2628091a40 2023-10-03T10:10:11.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.650+0000 7f26167fc700 1 -- 172.21.15.3:0/2035738394 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v8) v1 ==== 105+0+0 (secure 0 0 0) 0x7f261802a0b0 con 0x7f2628091a40 2023-10-03T10:10:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.652+0000 7f262c964700 1 -- 172.21.15.3:0/2035738394 >> v1:172.21.15.3:6816/118296 conn(0x7f260802bb60 legacy=0x7f260802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.652+0000 7f262c964700 1 -- 172.21.15.3:0/2035738394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2628091a40 msgr2=0x7f2628113360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.652+0000 7f262c964700 1 --2- 172.21.15.3:0/2035738394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2628091a40 0x7f2628113360 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f261800ea20 tx=0x7f2618009060).stop 2023-10-03T10:10:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.652+0000 7f262c964700 1 -- 172.21.15.3:0/2035738394 shutdown_connections 2023-10-03T10:10:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.652+0000 7f262c964700 1 --2- 172.21.15.3:0/2035738394 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2628127e70 0x7f2628113d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.652+0000 7f262c964700 1 --2- 172.21.15.3:0/2035738394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2628091a40 0x7f2628113360 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.652+0000 7f262c964700 1 --2- 172.21.15.3:0/2035738394 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2628092310 0x7f2628113850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.652+0000 7f262c964700 1 -- 172.21.15.3:0/2035738394 >> 172.21.15.3:0/2035738394 conn(0x7f2628063560 msgr2=0x7f2628097cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:11.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.653+0000 7f262c964700 1 -- 172.21.15.3:0/2035738394 shutdown_connections 2023-10-03T10:10:11.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.653+0000 7f262c964700 1 -- 172.21.15.3:0/2035738394 wait complete. 2023-10-03T10:10:11.668 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-03T10:10:11.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.801+0000 7f66a1509700 1 Processor -- start 2023-10-03T10:10:11.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.802+0000 7f66a1509700 1 -- start start 2023-10-03T10:10:11.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.802+0000 7f66a1509700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f669c062a90 0x7f669c127730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:11.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.802+0000 7f66a1509700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f669c127c20 0x7f669c0987e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:11.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.802+0000 7f66a1509700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f669c098cd0 0x7f669c099150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:11.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.802+0000 7f66a1509700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f669c062e70 con 0x7f669c098cd0 2023-10-03T10:10:11.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.802+0000 7f66a1509700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f669c092b00 con 0x7f669c127c20 2023-10-03T10:10:11.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.802+0000 7f66a1509700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f669c099640 con 0x7f669c062a90 2023-10-03T10:10:11.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.802+0000 7f669a7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f669c127c20 0x7f669c0987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:11.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.802+0000 7f669b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f669c098cd0 0x7f669c099150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:11.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.802+0000 7f669affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f669c062a90 0x7f669c127730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:11.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.802+0000 7f669b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f669c098cd0 0x7f669c099150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47594/0 (socket says 172.21.15.3:47594) 2023-10-03T10:10:11.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.802+0000 7f669a7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f669c127c20 0x7f669c0987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53988/0 (socket says 172.21.15.3:53988) 2023-10-03T10:10:11.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.802+0000 7f669b7fe700 1 -- 172.21.15.3:0/1311560007 learned_addr learned my addr 172.21.15.3:0/1311560007 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:11.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.803+0000 7f669a7fc700 1 -- 172.21.15.3:0/1311560007 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f669c062a90 msgr2=0x7f669c127730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:11.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.803+0000 7f669a7fc700 1 --2- 172.21.15.3:0/1311560007 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f669c062a90 0x7f669c127730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:11.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.803+0000 7f669a7fc700 1 -- 172.21.15.3:0/1311560007 >> v2:172.21.15.3:3300/0 conn(0x7f669c098cd0 msgr2=0x7f669c099150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:11.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.803+0000 7f669a7fc700 1 --2- 172.21.15.3:0/1311560007 >> v2:172.21.15.3:3300/0 conn(0x7f669c098cd0 0x7f669c099150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:11.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.803+0000 7f669a7fc700 1 -- 172.21.15.3:0/1311560007 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669c091bc0 con 0x7f669c127c20 2023-10-03T10:10:11.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.803+0000 7f669b7fe700 1 --2- 172.21.15.3:0/1311560007 >> v2:172.21.15.3:3300/0 conn(0x7f669c098cd0 0x7f669c099150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:11.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.803+0000 7f669a7fc700 1 --2- 172.21.15.3:0/1311560007 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f669c127c20 0x7f669c0987e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f6690009760 tx=0x7f6690009060).ready entity=mon.1 client_cookie=11069e084ec85b13 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:11.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.803+0000 7f6699ffb700 1 -- 172.21.15.3:0/1311560007 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66900090d0 con 0x7f669c127c20 2023-10-03T10:10:11.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.803+0000 7f6699ffb700 1 -- 172.21.15.3:0/1311560007 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f669000d4a0 con 0x7f669c127c20 2023-10-03T10:10:11.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.803+0000 7f6699ffb700 1 -- 172.21.15.3:0/1311560007 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669000d820 con 0x7f669c127c20 2023-10-03T10:10:11.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.804+0000 7f66a1509700 1 -- 172.21.15.3:0/1311560007 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f669c127c20 msgr2=0x7f669c0987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:11.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.804+0000 7f66a1509700 1 --2- 172.21.15.3:0/1311560007 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f669c127c20 0x7f669c0987e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f6690009760 tx=0x7f6690009060).stop 2023-10-03T10:10:11.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.804+0000 7f66a1509700 1 -- 172.21.15.3:0/1311560007 shutdown_connections 2023-10-03T10:10:11.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.804+0000 7f66a1509700 1 --2- 172.21.15.3:0/1311560007 >> v2:172.21.15.3:3300/0 conn(0x7f669c098cd0 0x7f669c099150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:11.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.804+0000 7f66a1509700 1 --2- 172.21.15.3:0/1311560007 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f669c062a90 0x7f669c127730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:11.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.804+0000 7f66a1509700 1 --2- 172.21.15.3:0/1311560007 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f669c127c20 0x7f669c0987e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:11.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.804+0000 7f66a1509700 1 -- 172.21.15.3:0/1311560007 >> 172.21.15.3:0/1311560007 conn(0x7f669c059b50 msgr2=0x7f669c097460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:11.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.804+0000 7f66a1509700 1 -- 172.21.15.3:0/1311560007 shutdown_connections 2023-10-03T10:10:11.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.804+0000 7f66a1509700 1 -- 172.21.15.3:0/1311560007 wait complete. 2023-10-03T10:10:11.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.805+0000 7f66a1509700 1 Processor -- start 2023-10-03T10:10:11.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.805+0000 7f66a1509700 1 -- start start 2023-10-03T10:10:11.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.806+0000 7f66a1509700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f669c062a90 0x7f669c1138e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:11.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.806+0000 7f66a1509700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f669c098cd0 0x7f669c113dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:11.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.806+0000 7f66a1509700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f669c127c20 0x7f669c1142c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:11.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.806+0000 7f66a1509700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669c124a40 con 0x7f669c098cd0 2023-10-03T10:10:11.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.806+0000 7f66a1509700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f669c09a370 con 0x7f669c062a90 2023-10-03T10:10:11.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.806+0000 7f66a1509700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f669c117410 con 0x7f669c127c20 2023-10-03T10:10:11.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.806+0000 7f669affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f669c062a90 0x7f669c1138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:11.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.806+0000 7f669affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f669c062a90 0x7f669c1138e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53998/0 (socket says 172.21.15.3:53998) 2023-10-03T10:10:11.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.806+0000 7f669affd700 1 -- 172.21.15.3:0/239018829 learned_addr learned my addr 172.21.15.3:0/239018829 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:11.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.806+0000 7f669a7fc700 1 --2- 172.21.15.3:0/239018829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f669c098cd0 0x7f669c113dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:11.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.806+0000 7f669b7fe700 1 --2- 172.21.15.3:0/239018829 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f669c127c20 0x7f669c1142c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:11.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.806+0000 7f669affd700 1 -- 172.21.15.3:0/239018829 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f669c127c20 msgr2=0x7f669c1142c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:11.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.806+0000 7f669affd700 1 --2- 172.21.15.3:0/239018829 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f669c127c20 0x7f669c1142c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:11.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.806+0000 7f669affd700 1 -- 172.21.15.3:0/239018829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f669c098cd0 msgr2=0x7f669c113dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:11.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.806+0000 7f669affd700 1 --2- 172.21.15.3:0/239018829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f669c098cd0 0x7f669c113dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:11.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.806+0000 7f669affd700 1 -- 172.21.15.3:0/239018829 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6694009ae0 con 0x7f669c062a90 2023-10-03T10:10:11.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.806+0000 7f669b7fe700 1 --2- 172.21.15.3:0/239018829 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f669c127c20 0x7f669c1142c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:11.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.806+0000 7f669a7fc700 1 --2- 172.21.15.3:0/239018829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f669c098cd0 0x7f669c113dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:11.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.807+0000 7f669affd700 1 --2- 172.21.15.3:0/239018829 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f669c062a90 0x7f669c1138e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f668c00e930 tx=0x7f668c009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:11.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.807+0000 7f6682ffd700 1 -- 172.21.15.3:0/239018829 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c012600 con 0x7f669c062a90 2023-10-03T10:10:11.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.807+0000 7f6682ffd700 1 -- 172.21.15.3:0/239018829 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668c012760 con 0x7f669c062a90 2023-10-03T10:10:11.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.807+0000 7f66a1509700 1 -- 172.21.15.3:0/239018829 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66900092a0 con 0x7f669c062a90 2023-10-03T10:10:11.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.807+0000 7f6682ffd700 1 -- 172.21.15.3:0/239018829 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c012ae0 con 0x7f669c062a90 2023-10-03T10:10:11.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.807+0000 7f66a1509700 1 -- 172.21.15.3:0/239018829 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f669c117af0 con 0x7f669c062a90 2023-10-03T10:10:11.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.808+0000 7f6682ffd700 1 -- 172.21.15.3:0/239018829 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f668c01b400 con 0x7f669c062a90 2023-10-03T10:10:11.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.808+0000 7f6682ffd700 1 -- 172.21.15.3:0/239018829 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f668c03b4e0 con 0x7f669c062a90 2023-10-03T10:10:11.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.808+0000 7f66a1509700 1 -- 172.21.15.3:0/239018829 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f669c05a260 con 0x7f669c062a90 2023-10-03T10:10:11.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:11.813+0000 7f6682ffd700 1 -- 172.21.15.3:0/239018829 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f668c019900 con 0x7f669c062a90 2023-10-03T10:10:12.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.031+0000 7f66a1509700 1 -- 172.21.15.3:0/239018829 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f669c046c20 con 0x7f669c062a90 2023-10-03T10:10:12.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.031+0000 7f6682ffd700 1 -- 172.21.15.3:0/239018829 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+2154 (secure 0 0 0) 0x7f668c019310 con 0x7f669c062a90 2023-10-03T10:10:12.032 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:10:12.032 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-03T10:09:44.141304+0000","modified":"2023-10-03T10:10:11.643845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4186},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4186":{"gid":4186,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6817/2582564779","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6817","nonce":2582564779}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-03T10:10:12.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.033+0000 7f66a1509700 1 -- 172.21.15.3:0/239018829 >> v1:172.21.15.3:6816/118296 conn(0x7f668802bb60 legacy=0x7f668802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:12.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.033+0000 7f66a1509700 1 -- 172.21.15.3:0/239018829 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f669c062a90 msgr2=0x7f669c1138e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:12.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.033+0000 7f66a1509700 1 --2- 172.21.15.3:0/239018829 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f669c062a90 0x7f669c1138e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f668c00e930 tx=0x7f668c009060).stop 2023-10-03T10:10:12.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.033+0000 7f66a1509700 1 -- 172.21.15.3:0/239018829 shutdown_connections 2023-10-03T10:10:12.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.033+0000 7f66a1509700 1 --2- 172.21.15.3:0/239018829 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f669c127c20 0x7f669c1142c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.033+0000 7f66a1509700 1 --2- 172.21.15.3:0/239018829 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f669c062a90 0x7f669c1138e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.033+0000 7f66a1509700 1 --2- 172.21.15.3:0/239018829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f669c098cd0 0x7f669c113dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.033+0000 7f66a1509700 1 -- 172.21.15.3:0/239018829 >> 172.21.15.3:0/239018829 conn(0x7f669c059b50 msgr2=0x7f669c096730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:12.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.033+0000 7f66a1509700 1 -- 172.21.15.3:0/239018829 shutdown_connections 2023-10-03T10:10:12.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.033+0000 7f66a1509700 1 -- 172.21.15.3:0/239018829 wait complete. 2023-10-03T10:10:12.035 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2023-10-03T10:10:12.147 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-03T10:09:44.141304+0000', 'modified': '2023-10-03T10:10:11.643845+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4186}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4186': {'gid': 4186, 'name': 'a', 'rank': 0, 'incarnation': 3, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.3:6817/2582564779', 'addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.3:6817', 'nonce': 2582564779}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-03T10:10:12.147 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-03T10:10:12.148 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-03T10:10:12.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.282+0000 7fd3c0fc3700 1 Processor -- start 2023-10-03T10:10:12.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.282+0000 7fd3c0fc3700 1 -- start start 2023-10-03T10:10:12.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.283+0000 7fd3c0fc3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3bc06a620 0x7fd3bc125870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:12.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.283+0000 7fd3c0fc3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3bc094360 0x7fd3bc130a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:12.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.283+0000 7fd3c0fc3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3bc0994a0 con 0x7fd3bc125d60 2023-10-03T10:10:12.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.283+0000 7fd3c0fc3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc09a690 con 0x7fd3bc06a620 2023-10-03T10:10:12.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.283+0000 7fd3c0fc3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc099720 con 0x7fd3bc094360 2023-10-03T10:10:12.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.283+0000 7fd3bad9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3bc094360 0x7fd3bc130a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:12.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.283+0000 7fd3b9d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd3bc125d60 0x7fd3bc093c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47512/0 (socket says 172.21.15.3:47512) 2023-10-03T10:10:12.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.283+0000 7fd3ba59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3bc06a620 0x7fd3bc125870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:12.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.283+0000 7fd3b9d9b700 1 -- 172.21.15.3:0/66704093 learned_addr learned my addr 172.21.15.3:0/66704093 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:10:12.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.284+0000 7fd3b959a700 1 -- 172.21.15.3:0/66704093 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2737298804 0 0) 0x7fd3bc0994a0 con 0x7fd3bc125d60 2023-10-03T10:10:12.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.284+0000 7fd3bad9d700 1 -- 172.21.15.3:0/66704093 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3bc06a620 msgr2=0x7fd3bc125870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:12.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.284+0000 7fd3bad9d700 1 --2- 172.21.15.3:0/66704093 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3bc06a620 0x7fd3bc125870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.284+0000 7fd3bad9d700 1 -- 172.21.15.3:0/66704093 >> v1:172.21.15.3:6789/0 conn(0x7fd3bc125d60 legacy=0x7fd3bc093c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:12.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.284+0000 7fd3bad9d700 1 -- 172.21.15.3:0/66704093 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3bc06aa00 con 0x7fd3bc094360 2023-10-03T10:10:12.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.284+0000 7fd3ba59c700 1 --2- 172.21.15.3:0/66704093 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3bc06a620 0x7fd3bc125870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:12.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.284+0000 7fd3bad9d700 1 --2- 172.21.15.3:0/66704093 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3bc094360 0x7fd3bc130a70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fd3b4009e90 tx=0x7fd3b4009780).ready entity=mon.2 client_cookie=9f1ab31ea76312b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:12.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.284+0000 7fd3b959a700 1 -- 172.21.15.3:0/66704093 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b400e070 con 0x7fd3bc094360 2023-10-03T10:10:12.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.284+0000 7fd3b959a700 1 -- 172.21.15.3:0/66704093 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b400bb60 con 0x7fd3bc094360 2023-10-03T10:10:12.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.284+0000 7fd3b959a700 1 -- 172.21.15.3:0/66704093 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4012410 con 0x7fd3bc094360 2023-10-03T10:10:12.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.285+0000 7fd3c0fc3700 1 -- 172.21.15.3:0/66704093 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3bc094360 msgr2=0x7fd3bc130a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:12.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.285+0000 7fd3c0fc3700 1 --2- 172.21.15.3:0/66704093 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3bc094360 0x7fd3bc130a70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fd3b4009e90 tx=0x7fd3b4009780).stop 2023-10-03T10:10:12.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.285+0000 7fd3c0fc3700 1 -- 172.21.15.3:0/66704093 shutdown_connections 2023-10-03T10:10:12.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.285+0000 7fd3c0fc3700 1 --2- 172.21.15.3:0/66704093 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3bc06a620 0x7fd3bc125870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.285+0000 7fd3c0fc3700 1 --2- 172.21.15.3:0/66704093 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3bc094360 0x7fd3bc130a70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.285+0000 7fd3c0fc3700 1 -- 172.21.15.3:0/66704093 >> 172.21.15.3:0/66704093 conn(0x7fd3bc097460 msgr2=0x7fd3bc097870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:12.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.285+0000 7fd3c0fc3700 1 -- 172.21.15.3:0/66704093 shutdown_connections 2023-10-03T10:10:12.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.285+0000 7fd3c0fc3700 1 -- 172.21.15.3:0/66704093 wait complete. 2023-10-03T10:10:12.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.286+0000 7fd3c0fc3700 1 Processor -- start 2023-10-03T10:10:12.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.286+0000 7fd3c0fc3700 1 -- start start 2023-10-03T10:10:12.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.286+0000 7fd3c0fc3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3bc06a620 0x7fd3bc137f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:12.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.286+0000 7fd3c0fc3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3bc094360 0x7fd3bc138420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:12.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.286+0000 7fd3c0fc3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3bc125d60 0x7fd3bc0677f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:12.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.286+0000 7fd3c0fc3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc12ccf0 con 0x7fd3bc094360 2023-10-03T10:10:12.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.286+0000 7fd3c0fc3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc111c40 con 0x7fd3bc06a620 2023-10-03T10:10:12.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.286+0000 7fd3c0fc3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc0693c0 con 0x7fd3bc125d60 2023-10-03T10:10:12.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.286+0000 7fd3ba59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3bc06a620 0x7fd3bc137f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:12.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.286+0000 7fd3b9d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3bc094360 0x7fd3bc138420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:12.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.286+0000 7fd3bad9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3bc125d60 0x7fd3bc0677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:12.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.286+0000 7fd3ba59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3bc06a620 0x7fd3bc137f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:54008/0 (socket says 172.21.15.3:54008) 2023-10-03T10:10:12.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.286+0000 7fd3ba59c700 1 -- 172.21.15.3:0/3958972821 learned_addr learned my addr 172.21.15.3:0/3958972821 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:12.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.287+0000 7fd3ba59c700 1 -- 172.21.15.3:0/3958972821 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3bc125d60 msgr2=0x7fd3bc0677f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:12.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.287+0000 7fd3ba59c700 1 --2- 172.21.15.3:0/3958972821 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3bc125d60 0x7fd3bc0677f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.287+0000 7fd3ba59c700 1 -- 172.21.15.3:0/3958972821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3bc094360 msgr2=0x7fd3bc138420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:12.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.287+0000 7fd3ba59c700 1 --2- 172.21.15.3:0/3958972821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3bc094360 0x7fd3bc138420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.287+0000 7fd3ba59c700 1 -- 172.21.15.3:0/3958972821 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ac0092a0 con 0x7fd3bc06a620 2023-10-03T10:10:12.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.287+0000 7fd3b9d9b700 1 --2- 172.21.15.3:0/3958972821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3bc094360 0x7fd3bc138420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:12.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.287+0000 7fd3ba59c700 1 --2- 172.21.15.3:0/3958972821 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3bc06a620 0x7fd3bc137f30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fd3ac00ea20 tx=0x7fd3ac009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:12.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.287+0000 7fd3aa7fc700 1 -- 172.21.15.3:0/3958972821 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac00ee60 con 0x7fd3bc06a620 2023-10-03T10:10:12.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.287+0000 7fd3aa7fc700 1 -- 172.21.15.3:0/3958972821 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ac009600 con 0x7fd3bc06a620 2023-10-03T10:10:12.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.287+0000 7fd3aa7fc700 1 -- 172.21.15.3:0/3958972821 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac00ca60 con 0x7fd3bc06a620 2023-10-03T10:10:12.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.287+0000 7fd3c0fc3700 1 -- 172.21.15.3:0/3958972821 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3b4009a20 con 0x7fd3bc06a620 2023-10-03T10:10:12.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.287+0000 7fd3c0fc3700 1 -- 172.21.15.3:0/3958972821 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3bc098ed0 con 0x7fd3bc06a620 2023-10-03T10:10:12.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.288+0000 7fd3aa7fc700 1 -- 172.21.15.3:0/3958972821 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fd3ac01b400 con 0x7fd3bc06a620 2023-10-03T10:10:12.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.288+0000 7fd3aa7fc700 1 -- 172.21.15.3:0/3958972821 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fd3ac03b500 con 0x7fd3bc06a620 2023-10-03T10:10:12.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.291+0000 7fd3d2976b80 1 Processor -- start 2023-10-03T10:10:12.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.291+0000 7fd3d2976b80 1 -- start start 2023-10-03T10:10:12.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.292+0000 7fd3d2976b80 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x561f300ac170 0x561f300ac550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:12.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.292+0000 7fd3d2976b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561f300aca40 0x561f300d60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:12.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.292+0000 7fd3d2976b80 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x561f300d65b0 0x561f300d89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:12.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.292+0000 7fd3d2976b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f30076610 con 0x561f300aca40 2023-10-03T10:10:12.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.292+0000 7fd3d2976b80 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x561f2ffecdb0 con 0x561f300ac170 2023-10-03T10:10:12.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.292+0000 7fd3d2976b80 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x561f30002030 con 0x561f300d65b0 2023-10-03T10:10:12.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.292+0000 7fd3b9d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561f300aca40 0x561f300d60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:12.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.292+0000 7fd3b9d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561f300aca40 0x561f300d60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47616/0 (socket says 172.21.15.3:47616) 2023-10-03T10:10:12.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.292+0000 7fd3b9d9b700 1 -- 172.21.15.3:0/4093917639 learned_addr learned my addr 172.21.15.3:0/4093917639 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:12.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.292+0000 7fd3ba59c700 1 --2- 172.21.15.3:0/4093917639 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x561f300ac170 0x561f300ac550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:12.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.292+0000 7fd3bad9d700 1 --2- 172.21.15.3:0/4093917639 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x561f300d65b0 0x561f300d89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:12.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.292+0000 7fd3ba59c700 1 -- 172.21.15.3:0/4093917639 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x561f300d65b0 msgr2=0x561f300d89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:12.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.292+0000 7fd3ba59c700 1 --2- 172.21.15.3:0/4093917639 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x561f300d65b0 0x561f300d89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.292+0000 7fd3ba59c700 1 -- 172.21.15.3:0/4093917639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561f300aca40 msgr2=0x561f300d60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:12.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.292+0000 7fd3ba59c700 1 --2- 172.21.15.3:0/4093917639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561f300aca40 0x561f300d60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.292+0000 7fd3ba59c700 1 -- 172.21.15.3:0/4093917639 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f2fe9d710 con 0x561f300ac170 2023-10-03T10:10:12.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.292+0000 7fd3b9d9b700 1 --2- 172.21.15.3:0/4093917639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561f300aca40 0x561f300d60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:12.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.292+0000 7fd3bad9d700 1 --2- 172.21.15.3:0/4093917639 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x561f300d65b0 0x561f300d89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:10:12.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.293+0000 7fd3ba59c700 1 --2- 172.21.15.3:0/4093917639 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x561f300ac170 0x561f300ac550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fd3ac0094f0 tx=0x7fd3ac01a000).ready entity=mon.1 client_cookie=b5007b3d11369d7c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:12.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.293+0000 7fd39b7fe700 1 -- 172.21.15.3:0/4093917639 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac01b6b0 con 0x561f300ac170 2023-10-03T10:10:12.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.293+0000 7fd39b7fe700 1 -- 172.21.15.3:0/4093917639 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ac01b810 con 0x561f300ac170 2023-10-03T10:10:12.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.293+0000 7fd39b7fe700 1 -- 172.21.15.3:0/4093917639 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac01bde0 con 0x561f300ac170 2023-10-03T10:10:12.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.293+0000 7fd3d2976b80 1 -- 172.21.15.3:0/4093917639 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x561f300ac170 msgr2=0x561f300ac550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:12.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.293+0000 7fd3d2976b80 1 --2- 172.21.15.3:0/4093917639 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x561f300ac170 0x561f300ac550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fd3ac0094f0 tx=0x7fd3ac01a000).stop 2023-10-03T10:10:12.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.293+0000 7fd3d2976b80 1 -- 172.21.15.3:0/4093917639 shutdown_connections 2023-10-03T10:10:12.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.293+0000 7fd3d2976b80 1 --2- 172.21.15.3:0/4093917639 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x561f300d65b0 0x561f300d89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.293+0000 7fd3d2976b80 1 --2- 172.21.15.3:0/4093917639 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x561f300ac170 0x561f300ac550 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.293+0000 7fd3d2976b80 1 --2- 172.21.15.3:0/4093917639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561f300aca40 0x561f300d60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.293+0000 7fd3d2976b80 1 -- 172.21.15.3:0/4093917639 >> 172.21.15.3:0/4093917639 conn(0x561f2ff1a940 msgr2=0x561f300cccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:12.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.294+0000 7fd3d2976b80 1 -- 172.21.15.3:0/4093917639 shutdown_connections 2023-10-03T10:10:12.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.294+0000 7fd3d2976b80 1 -- 172.21.15.3:0/4093917639 wait complete. 2023-10-03T10:10:12.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.294+0000 7fd3d2976b80 1 -- start start 2023-10-03T10:10:12.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.294+0000 7fd3d2976b80 1 Processor -- start 2023-10-03T10:10:12.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.295+0000 7fd3d2976b80 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x561f300ac170 0x561f300d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:12.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.295+0000 7fd3d2976b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561f300aca40 0x561f300d20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:12.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.295+0000 7fd3d2976b80 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x561f300d65b0 0x561f300d25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:12.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.295+0000 7fd3d2976b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f30002170 con 0x561f300aca40 2023-10-03T10:10:12.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.295+0000 7fd3d2976b80 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x561f300023f0 con 0x561f300ac170 2023-10-03T10:10:12.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.295+0000 7fd3d2976b80 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x561f2ffcd920 con 0x561f300d65b0 2023-10-03T10:10:12.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.295+0000 7fd3d2976b80 10 client.0 ll_register_callbacks cb 0x561f2ffa2ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-03T10:10:12.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.295+0000 7fd3bad9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x561f300d65b0 0x561f300d25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:12.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.295+0000 7fd3b9d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561f300aca40 0x561f300d20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:12.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.295+0000 7fd3ba59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x561f300ac170 0x561f300d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:12.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.295+0000 7fd3bad9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x561f300d65b0 0x561f300d25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:60470/0 (socket says 172.21.15.3:60470) 2023-10-03T10:10:12.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.295+0000 7fd3bad9d700 1 -- 172.21.15.3:0/3999775647 learned_addr learned my addr 172.21.15.3:0/3999775647 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:12.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.295+0000 7fd3b9d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561f300aca40 0x561f300d20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47622/0 (socket says 172.21.15.3:47622) 2023-10-03T10:10:12.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.295+0000 7fd3ba59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x561f300ac170 0x561f300d1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:54036/0 (socket says 172.21.15.3:54036) 2023-10-03T10:10:12.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.295+0000 7fd3bad9d700 1 -- 172.21.15.3:0/3999775647 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x561f300ac170 msgr2=0x561f300d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:12.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.295+0000 7fd3bad9d700 1 --2- 172.21.15.3:0/3999775647 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x561f300ac170 0x561f300d1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.295+0000 7fd3bad9d700 1 -- 172.21.15.3:0/3999775647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561f300aca40 msgr2=0x561f300d20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:12.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.295+0000 7fd3bad9d700 1 --2- 172.21.15.3:0/3999775647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561f300aca40 0x561f300d20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.295+0000 7fd3bad9d700 1 -- 172.21.15.3:0/3999775647 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3bc098ed0 con 0x561f300d65b0 2023-10-03T10:10:12.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.295+0000 7fd3bad9d700 1 --2- 172.21.15.3:0/3999775647 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x561f300d65b0 0x561f300d25c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fd3bc09a7d0 tx=0x7fd3b4009780).ready entity=mon.2 client_cookie=5233c1abbf125aa3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:12.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.295+0000 7fd398ff9700 10 client.0 ms_handle_connect on v2:172.21.15.3:3302/0 2023-10-03T10:10:12.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.296+0000 7fd398ff9700 1 -- 172.21.15.3:0/3999775647 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b402f430 con 0x561f300d65b0 2023-10-03T10:10:12.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.296+0000 7fd398ff9700 1 -- 172.21.15.3:0/3999775647 <== mon.2 v2:172.21.15.3:3302/0 2 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fd3b402f590 con 0x561f300d65b0 2023-10-03T10:10:12.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.296+0000 7fd3d2976b80 1 -- 172.21.15.3:0/3999775647 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f300a4f20 con 0x561f300d65b0 2023-10-03T10:10:12.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.296+0000 7fd398ff9700 1 -- 172.21.15.3:0/3999775647 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3b402f960 con 0x561f300d65b0 2023-10-03T10:10:12.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.296+0000 7fd3d2976b80 10 client.4577 fetch_fsmap learned FSMap version 8 2023-10-03T10:10:12.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.296+0000 7fd3d2976b80 1 -- 172.21.15.3:0/3999775647 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x561f2ffd4b00 con 0x561f300d65b0 2023-10-03T10:10:12.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.296+0000 7fd398ff9700 1 -- 172.21.15.3:0/3999775647 <== mon.2 v2:172.21.15.3:3302/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561f300a4f20 con 0x561f300d65b0 2023-10-03T10:10:12.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.296+0000 7fd398ff9700 1 -- 172.21.15.3:0/3999775647 <== mon.2 v2:172.21.15.3:3302/0 5 ==== fsmap(e 8) v1 ==== 1430+0+0 (secure 0 0 0) 0x7fd3b402fac0 con 0x561f300d65b0 2023-10-03T10:10:12.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.296+0000 7fd3d2976b80 10 client.4577 fetch_fsmap finished waiting for FSMap version 8 2023-10-03T10:10:12.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.296+0000 7fd3d2976b80 10 client.4577 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-03T10:10:12.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.296+0000 7fd3d2976b80 20 client.4577 populate_metadata read hostname 'smithi003' 2023-10-03T10:10:12.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.296+0000 7fd3d2976b80 4 client.4577 mds_command: new command op to 4186 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-03T10:10:12.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.296+0000 7fd3d2976b80 1 -- 172.21.15.3:0/3999775647 --> v1:172.21.15.3:6817/2582564779 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f3007a250 con 0x561f300c6a30 2023-10-03T10:10:12.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.296+0000 7fd398ff9700 10 client.4577 ms_handle_connect on v1:172.21.15.3:6817/2582564779 2023-10-03T10:10:12.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.297+0000 7fd398ff9700 1 -- 172.21.15.3:0/3999775647 <== mds.0 v1:172.21.15.3:6817/2582564779 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (unknown 0 0 3608088055) 0x561f3007a250 con 0x561f300c6a30 2023-10-03T10:10:12.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.297+0000 7fd398ff9700 10 client.4577 handle_command_reply: tid=0 2023-10-03T10:10:12.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.297+0000 7fd3d2976b80 1 client.4577 shutdown 2023-10-03T10:10:12.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.297+0000 7fd3d2976b80 1 -- 172.21.15.3:0/3999775647 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x561f300d65b0 msgr2=0x561f300d25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:12.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.297+0000 7fd3d2976b80 1 --2- 172.21.15.3:0/3999775647 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x561f300d65b0 0x561f300d25c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fd3bc09a7d0 tx=0x7fd3b4009780).stop 2023-10-03T10:10:12.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.297+0000 7fd3d2976b80 1 -- 172.21.15.3:0/3999775647 shutdown_connections 2023-10-03T10:10:12.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.297+0000 7fd3d2976b80 1 --2- 172.21.15.3:0/3999775647 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x561f300d65b0 0x561f300d25c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.297+0000 7fd3d2976b80 1 --2- 172.21.15.3:0/3999775647 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x561f300ac170 0x561f300d1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.297+0000 7fd3d2976b80 1 --2- 172.21.15.3:0/3999775647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561f300aca40 0x561f300d20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.297+0000 7fd398ff9700 0 client.4577 ms_handle_reset on v1:172.21.15.3:6817/2582564779 2023-10-03T10:10:12.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.297+0000 7fd3d2976b80 1 -- 172.21.15.3:0/3999775647 >> 172.21.15.3:0/3999775647 conn(0x561f2ff1a940 msgr2=0x561f300ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:12.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.298+0000 7fd3d2976b80 1 -- 172.21.15.3:0/3999775647 shutdown_connections 2023-10-03T10:10:12.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.298+0000 7fd3d2976b80 1 -- 172.21.15.3:0/3999775647 wait complete. 2023-10-03T10:10:12.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.298+0000 7fd3d2976b80 20 client.4577 trim_cache size 0 max 16384 2023-10-03T10:10:12.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.315+0000 7fd3d2976b80 1 Processor -- start 2023-10-03T10:10:12.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.315+0000 7fd3d2976b80 1 -- start start 2023-10-03T10:10:12.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.315+0000 7fd3d2976b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561f300ac170 0x561f300ace20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:12.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.315+0000 7fd3d2976b80 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x561f300aca40 0x561f300ac550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:12.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.315+0000 7fd3d2976b80 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x561f300d65b0 0x561f300d44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:12.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.315+0000 7fd3d2976b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f300d4bb0 con 0x561f300ac170 2023-10-03T10:10:12.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.315+0000 7fd3d2976b80 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x561f300ad520 con 0x561f300d65b0 2023-10-03T10:10:12.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.315+0000 7fd3d2976b80 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x561f3007a250 con 0x561f300aca40 2023-10-03T10:10:12.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.316+0000 7fd3ba59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561f300ac170 0x561f300ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:12.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.316+0000 7fd3b9d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x561f300aca40 0x561f300ac550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:12.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.316+0000 7fd3ba59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561f300ac170 0x561f300ace20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47632/0 (socket says 172.21.15.3:47632) 2023-10-03T10:10:12.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.316+0000 7fd3b9d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x561f300aca40 0x561f300ac550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:60482/0 (socket says 172.21.15.3:60482) 2023-10-03T10:10:12.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.316+0000 7fd3ba59c700 1 -- 172.21.15.3:0/3037486590 learned_addr learned my addr 172.21.15.3:0/3037486590 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:12.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.316+0000 7fd3bad9d700 1 --2- 172.21.15.3:0/3037486590 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x561f300d65b0 0x561f300d44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:12.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.316+0000 7fd3ba59c700 1 -- 172.21.15.3:0/3037486590 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x561f300aca40 msgr2=0x561f300ac550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:12.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.316+0000 7fd3ba59c700 1 --2- 172.21.15.3:0/3037486590 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x561f300aca40 0x561f300ac550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.316+0000 7fd3ba59c700 1 -- 172.21.15.3:0/3037486590 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x561f300d65b0 msgr2=0x561f300d44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:12.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.316+0000 7fd3ba59c700 1 --2- 172.21.15.3:0/3037486590 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x561f300d65b0 0x561f300d44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.316+0000 7fd3ba59c700 1 -- 172.21.15.3:0/3037486590 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f2fffaf70 con 0x561f300ac170 2023-10-03T10:10:12.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.316+0000 7fd3ba59c700 1 --2- 172.21.15.3:0/3037486590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561f300ac170 0x561f300ace20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fd3ac011c70 tx=0x7fd3ac018000).ready entity=mon.0 client_cookie=bf2fc324b4aca5c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:12.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.316+0000 7fd39bfff700 1 -- 172.21.15.3:0/3037486590 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac00cc00 con 0x561f300ac170 2023-10-03T10:10:12.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.316+0000 7fd39bfff700 1 -- 172.21.15.3:0/3037486590 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ac01bbd0 con 0x561f300ac170 2023-10-03T10:10:12.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.316+0000 7fd39bfff700 1 -- 172.21.15.3:0/3037486590 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac01bd40 con 0x561f300ac170 2023-10-03T10:10:12.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.317+0000 7fd3d2976b80 1 -- 172.21.15.3:0/3037486590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561f300ac170 msgr2=0x561f300ace20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:12.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.317+0000 7fd3d2976b80 1 --2- 172.21.15.3:0/3037486590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561f300ac170 0x561f300ace20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fd3ac011c70 tx=0x7fd3ac018000).stop 2023-10-03T10:10:12.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.317+0000 7fd3d2976b80 1 -- 172.21.15.3:0/3037486590 shutdown_connections 2023-10-03T10:10:12.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.317+0000 7fd3d2976b80 1 --2- 172.21.15.3:0/3037486590 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x561f300d65b0 0x561f300d44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.317+0000 7fd3d2976b80 1 --2- 172.21.15.3:0/3037486590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561f300ac170 0x561f300ace20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.317+0000 7fd3d2976b80 1 --2- 172.21.15.3:0/3037486590 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x561f300aca40 0x561f300ac550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.317+0000 7fd3d2976b80 1 -- 172.21.15.3:0/3037486590 >> 172.21.15.3:0/3037486590 conn(0x561f2ff1a940 msgr2=0x561f300af1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:12.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.317+0000 7fd3d2976b80 1 -- 172.21.15.3:0/3037486590 shutdown_connections 2023-10-03T10:10:12.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.317+0000 7fd3d2976b80 1 -- 172.21.15.3:0/3037486590 wait complete. 2023-10-03T10:10:12.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.317+0000 7fd3d2976b80 1 -- start start 2023-10-03T10:10:12.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.317+0000 7fd3d2976b80 1 Processor -- start 2023-10-03T10:10:12.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.318+0000 7fd3d2976b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561f300ac170 0x561f300d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:12.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.318+0000 7fd3d2976b80 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x561f300aca40 0x561f300d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:12.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.318+0000 7fd3d2976b80 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x561f300d65b0 0x561f30130120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-03T10:10:12.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.318+0000 7fd3d2976b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f300d0b00 con 0x561f300ac170 2023-10-03T10:10:12.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.318+0000 7fd3d2976b80 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x561f300af710 con 0x561f300d65b0 2023-10-03T10:10:12.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.318+0000 7fd3d2976b80 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x561f30113a60 con 0x561f300aca40 2023-10-03T10:10:12.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.318+0000 7fd3d2976b80 10 client.0 ll_register_callbacks cb 0x561f2ffa2ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-03T10:10:12.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.318+0000 7fd3ba59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561f300ac170 0x561f300d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:12.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.318+0000 7fd3bad9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x561f300d65b0 0x561f30130120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-03T10:10:12.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.318+0000 7fd3ba59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561f300ac170 0x561f300d0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47638/0 (socket says 172.21.15.3:47638) 2023-10-03T10:10:12.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.318+0000 7fd3ba59c700 1 -- 172.21.15.3:0/3664754972 learned_addr learned my addr 172.21.15.3:0/3664754972 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:10:12.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.318+0000 7fd3bad9d700 1 -- 172.21.15.3:0/3664754972 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x561f300aca40 msgr2=0x561f300d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:12.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.318+0000 7fd3bad9d700 1 --2- 172.21.15.3:0/3664754972 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x561f300aca40 0x561f300d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-03T10:10:12.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.318+0000 7fd3bad9d700 1 -- 172.21.15.3:0/3664754972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561f300ac170 msgr2=0x561f300d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:10:12.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.318+0000 7fd3bad9d700 1 --2- 172.21.15.3:0/3664754972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561f300ac170 0x561f300d0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.318+0000 7fd3bad9d700 1 -- 172.21.15.3:0/3664754972 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f2fe9d710 con 0x561f300d65b0 2023-10-03T10:10:12.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.319+0000 7fd3bad9d700 1 --2- 172.21.15.3:0/3664754972 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x561f300d65b0 0x561f30130120 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fd3b4009e90 tx=0x7fd3b4029040).ready entity=mon.1 client_cookie=1c34a258164a95ab server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:10:12.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.319+0000 7fd398ff9700 10 client.0 ms_handle_connect on v2:172.21.15.3:3301/0 2023-10-03T10:10:12.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.319+0000 7fd398ff9700 1 -- 172.21.15.3:0/3664754972 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4012530 con 0x561f300d65b0 2023-10-03T10:10:12.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.319+0000 7fd398ff9700 1 -- 172.21.15.3:0/3664754972 <== mon.1 v2:172.21.15.3:3301/0 2 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fd3b4012690 con 0x561f300d65b0 2023-10-03T10:10:12.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.319+0000 7fd3d2976b80 1 -- 172.21.15.3:0/3664754972 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd3ac01b810 con 0x561f300d65b0 2023-10-03T10:10:12.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.319+0000 7fd398ff9700 1 -- 172.21.15.3:0/3664754972 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3b4012850 con 0x561f300d65b0 2023-10-03T10:10:12.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.319+0000 7fd3d2976b80 10 client.4585 fetch_fsmap learned FSMap version 8 2023-10-03T10:10:12.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.319+0000 7fd3d2976b80 1 -- 172.21.15.3:0/3664754972 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x561f2fff6580 con 0x561f300d65b0 2023-10-03T10:10:12.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.319+0000 7fd398ff9700 1 -- 172.21.15.3:0/3664754972 <== mon.1 v2:172.21.15.3:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ac01b810 con 0x561f300d65b0 2023-10-03T10:10:12.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.319+0000 7fd398ff9700 1 -- 172.21.15.3:0/3664754972 <== mon.1 v2:172.21.15.3:3301/0 5 ==== fsmap(e 8) v1 ==== 1430+0+0 (secure 0 0 0) 0x7fd3b403f070 con 0x561f300d65b0 2023-10-03T10:10:12.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.319+0000 7fd3d2976b80 10 client.4585 fetch_fsmap finished waiting for FSMap version 8 2023-10-03T10:10:12.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.319+0000 7fd3d2976b80 10 client.4585 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-03T10:10:12.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.319+0000 7fd3d2976b80 20 client.4585 populate_metadata read hostname 'smithi003' 2023-10-03T10:10:12.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.319+0000 7fd3d2976b80 4 client.4585 mds_command: new command op to 4186 tid=0[{"prefix": "status"}] 2023-10-03T10:10:12.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.319+0000 7fd3d2976b80 1 -- 172.21.15.3:0/3664754972 --> v1:172.21.15.3:6817/2582564779 -- command(tid 0: {"prefix": "status"}) v1 -- 0x561f2fe6aef0 con 0x561f30130990 2023-10-03T10:10:12.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.320+0000 7fd398ff9700 10 client.4585 ms_handle_connect on v1:172.21.15.3:6817/2582564779 2023-10-03T10:10:12.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.320+0000 7fd398ff9700 1 -- 172.21.15.3:0/3664754972 <== mds.0 v1:172.21.15.3:6817/2582564779 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (unknown 0 0 1612425714) 0x561f2fe6aef0 con 0x561f30130990 2023-10-03T10:10:12.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.320+0000 7fd398ff9700 10 client.4585 handle_command_reply: tid=0 2023-10-03T10:10:12.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.320+0000 7fd3d2976b80 1 client.4585 shutdown 2023-10-03T10:10:12.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.320+0000 7fd3d2976b80 1 -- 172.21.15.3:0/3664754972 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x561f300d65b0 msgr2=0x561f30130120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:12.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.320+0000 7fd3d2976b80 1 --2- 172.21.15.3:0/3664754972 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x561f300d65b0 0x561f30130120 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fd3b4009e90 tx=0x7fd3b4029040).stop 2023-10-03T10:10:12.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.320+0000 7fd3d2976b80 1 -- 172.21.15.3:0/3664754972 shutdown_connections 2023-10-03T10:10:12.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.320+0000 7fd3d2976b80 1 --2- 172.21.15.3:0/3664754972 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x561f300d65b0 0x561f30130120 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.320+0000 7fd3d2976b80 1 --2- 172.21.15.3:0/3664754972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561f300ac170 0x561f300d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.320+0000 7fd398ff9700 0 client.4585 ms_handle_reset on v1:172.21.15.3:6817/2582564779 2023-10-03T10:10:12.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.320+0000 7fd3d2976b80 1 --2- 172.21.15.3:0/3664754972 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x561f300aca40 0x561f300d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-03T10:10:12.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.320+0000 7fd3d2976b80 1 -- 172.21.15.3:0/3664754972 >> 172.21.15.3:0/3664754972 conn(0x561f2ff1a940 msgr2=0x561f300c9380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:12.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.320+0000 7fd3d2976b80 1 -- 172.21.15.3:0/3664754972 shutdown_connections 2023-10-03T10:10:12.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.320+0000 7fd3d2976b80 1 -- 172.21.15.3:0/3664754972 wait complete. 2023-10-03T10:10:12.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.320+0000 7fd3d2976b80 20 client.4585 trim_cache size 0 max 16384 2023-10-03T10:10:12.327 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-03T10:10:12.327 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "68e08e7d-a212-4fd3-bd42-88e2411aa299", 2023-10-03T10:10:12.327 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2023-10-03T10:10:12.327 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4186, 2023-10-03T10:10:12.327 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-10-03T10:10:12.327 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-10-03T10:10:12.327 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-10-03T10:10:12.328 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 28.172954459, 2023-10-03T10:10:12.328 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 8, 2023-10-03T10:10:12.328 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 18, 2023-10-03T10:10:12.328 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 18, 2023-10-03T10:10:12.328 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 32.628829783999997 2023-10-03T10:10:12.328 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-03T10:10:12.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.321+0000 7fd3c0fc3700 1 -- 172.21.15.3:0/3958972821 >> v1:172.21.15.3:6816/118296 conn(0x7fd39c02bb60 legacy=0x7fd39c02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:12.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.321+0000 7fd3c0fc3700 1 -- 172.21.15.3:0/3958972821 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3bc06a620 msgr2=0x7fd3bc137f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:10:12.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.321+0000 7fd3c0fc3700 1 --2- 172.21.15.3:0/3958972821 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3bc06a620 0x7fd3bc137f30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fd3ac00ea20 tx=0x7fd3ac009060).stop 2023-10-03T10:10:12.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.321+0000 7fd3c0fc3700 1 -- 172.21.15.3:0/3958972821 shutdown_connections 2023-10-03T10:10:12.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.321+0000 7fd3c0fc3700 1 --2- 172.21.15.3:0/3958972821 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3bc125d60 0x7fd3bc0677f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.321+0000 7fd3c0fc3700 1 --2- 172.21.15.3:0/3958972821 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3bc06a620 0x7fd3bc137f30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.321+0000 7fd3c0fc3700 1 --2- 172.21.15.3:0/3958972821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3bc094360 0x7fd3bc138420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-03T10:10:12.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.321+0000 7fd3c0fc3700 1 -- 172.21.15.3:0/3958972821 >> 172.21.15.3:0/3958972821 conn(0x7fd3bc097460 msgr2=0x7fd3bc126cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:10:12.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.321+0000 7fd3c0fc3700 1 -- 172.21.15.3:0/3958972821 shutdown_connections 2023-10-03T10:10:12.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:10:12.321+0000 7fd3c0fc3700 1 -- 172.21.15.3:0/3958972821 wait complete. 2023-10-03T10:10:12.334 INFO:teuthology.run_tasks:Running task print... 2023-10-03T10:10:12.344 INFO:teuthology.task.print:**** done mds pre-upgrade sequence 2023-10-03T10:10:12.344 INFO:teuthology.run_tasks:Running task install.upgrade... 2023-10-03T10:10:12.353 INFO:teuthology.task.install:project ceph config {'mon.a': {'branch': 'reef'}} overrides {'flavor': 'default', 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054'} 2023-10-03T10:10:12.353 INFO:teuthology.task.install:config contains sha1|tag|branch, removing those keys from override 2023-10-03T10:10:12.353 INFO:teuthology.task.install:remote ubuntu@smithi003.front.sepia.ceph.com config {'branch': 'reef', 'flavor': 'default'} 2023-10-03T10:10:12.353 INFO:teuthology.task.install:extra packages: [] 2023-10-03T10:10:12.358 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-03T10:10:12.359 INFO:teuthology.task.install:Upgrading ceph rpm packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd 2023-10-03T10:10:12.359 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q ceph-common --qf '%{VERSION}-%{RELEASE}' 2023-10-03T10:10:12.385 INFO:teuthology.orchestra.run.smithi003.stdout:16.2.14-44.geeb13ac9.el8 2023-10-03T10:10:12.385 INFO:teuthology.packaging:The installed version of ceph-common is 16.2.14-44.geeb13ac9.el8 2023-10-03T10:10:12.385 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=reef 2023-10-03T10:10:12.763 INFO:teuthology.task.install:Ceph rpm upgrade from 16.2.14-44.geeb13ac9.el8 to 18.2.0-507.g7cd99ce4 2023-10-03T10:10:12.763 INFO:teuthology.task.install.rpm:Host smithi003 is: centos 8 x86_64 2023-10-03T10:10:12.763 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=reef 2023-10-03T10:10:13.168 INFO:teuthology.task.install.rpm:Repo base URL: https://2.chacra.ceph.com/r/ceph/reef/7cd99ce428fa36b28345615983ecfd3d24e90eba/centos/8/flavors/default/ 2023-10-03T10:10:13.168 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f /etc/yum.repos.d/ceph.repo 2023-10-03T10:10:13.481 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/reef/7cd99ce428fa36b28345615983ecfd3d24e90eba/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/reef/7cd99ce428fa36b28345615983ecfd3d24e90eba/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/reef/7cd99ce428fa36b28345615983ecfd3d24e90eba/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-03T10:10:13.481 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-10-03T10:10:13.482 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-03T10:10:13.517 DEBUG:teuthology.orchestra.run.smithi003:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/reef/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-03T10:10:13.593 DEBUG:teuthology.orchestra.run.smithi003:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-03T10:10:13.670 DEBUG:teuthology.orchestra.run.smithi003:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-03T10:10:13.733 INFO:teuthology.orchestra.run.smithi003.stdout:check_obsoletes = 1 2023-10-03T10:10:13.747 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum clean all 2023-10-03T10:10:14.055 INFO:teuthology.orchestra.run.smithi003.stdout:71 files removed 2023-10-03T10:10:14.088 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y upgrade ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd 2023-10-03T10:10:14.974 INFO:teuthology.orchestra.run.smithi003.stdout:ceph packages for x86_64 198 kB/s | 80 kB 00:00 2023-10-03T10:10:16.409 INFO:teuthology.orchestra.run.smithi003.stdout:ceph noarch packages 6.8 kB/s | 9.4 kB 00:01 2023-10-03T10:10:16.984 INFO:teuthology.orchestra.run.smithi003.stdout:ceph source packages 2.7 kB/s | 1.5 kB 00:00 2023-10-03T10:10:18.026 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 8 - AppStream 33 MB/s | 33 MB 00:01 2023-10-03T10:10:27.126 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 8 - BaseOS 20 MB/s | 48 MB 00:02 2023-10-03T10:10:35.134 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 8 - Extras 55 kB/s | 18 kB 00:00 2023-10-03T10:10:35.669 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 8 - Extras common packages 21 kB/s | 6.9 kB 00:00 2023-10-03T10:10:36.345 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 8 - PowerTools 13 MB/s | 6.5 MB 00:00 2023-10-03T10:10:37.922 INFO:teuthology.orchestra.run.smithi003.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2023-10-03T10:10:39.098 INFO:teuthology.orchestra.run.smithi003.stdout:Extra Packages for Enterprise Linux 16 MB/s | 16 MB 00:00 2023-10-03T10:10:43.423 INFO:teuthology.orchestra.run.smithi003.stdout:lab-extras 355 kB/s | 24 kB 00:00 2023-10-03T10:10:48.115 INFO:teuthology.orchestra.run.smithi003.stderr:Package cephadm available, but not installed. 2023-10-03T10:10:48.115 INFO:teuthology.orchestra.run.smithi003.stdout:No match for argument: cephadm 2023-10-03T10:10:48.115 INFO:teuthology.orchestra.run.smithi003.stderr:Package ceph-immutable-object-cache available, but not installed. 2023-10-03T10:10:48.116 INFO:teuthology.orchestra.run.smithi003.stdout:No match for argument: ceph-immutable-object-cache 2023-10-03T10:10:48.130 INFO:teuthology.orchestra.run.smithi003.stderr:Package ceph-mgr-cephadm available, but not installed. 2023-10-03T10:10:48.131 INFO:teuthology.orchestra.run.smithi003.stdout:No match for argument: ceph-mgr-cephadm 2023-10-03T10:10:48.135 INFO:teuthology.orchestra.run.smithi003.stderr:Package ceph-volume available, but not installed. 2023-10-03T10:10:48.135 INFO:teuthology.orchestra.run.smithi003.stdout:No match for argument: ceph-volume 2023-10-03T10:10:48.446 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2023-10-03T10:10:48.452 INFO:teuthology.orchestra.run.smithi003.stdout:====================================================================================== 2023-10-03T10:10:48.452 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repository Size 2023-10-03T10:10:48.452 INFO:teuthology.orchestra.run.smithi003.stdout:====================================================================================== 2023-10-03T10:10:48.452 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrading: 2023-10-03T10:10:48.452 INFO:teuthology.orchestra.run.smithi003.stdout: ceph x86_64 2:18.2.0-507.g7cd99ce4.el8 ceph 6.4 k 2023-10-03T10:10:48.452 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base x86_64 2:18.2.0-507.g7cd99ce4.el8 ceph 5.4 M 2023-10-03T10:10:48.452 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-common x86_64 2:18.2.0-507.g7cd99ce4.el8 ceph 22 M 2023-10-03T10:10:48.452 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-fuse x86_64 2:18.2.0-507.g7cd99ce4.el8 ceph 847 k 2023-10-03T10:10:48.452 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-grafana-dashboards noarch 2:18.2.0-507.g7cd99ce4.el8 ceph-noarch 23 k 2023-10-03T10:10:48.452 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mds x86_64 2:18.2.0-507.g7cd99ce4.el8 ceph 2.3 M 2023-10-03T10:10:48.452 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr x86_64 2:18.2.0-507.g7cd99ce4.el8 ceph 1.5 M 2023-10-03T10:10:48.453 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard noarch 2:18.2.0-507.g7cd99ce4.el8 ceph-noarch 1.7 M 2023-10-03T10:10:48.453 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-507.g7cd99ce4.el8 ceph-noarch 7.4 M 2023-10-03T10:10:48.453 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core noarch 2:18.2.0-507.g7cd99ce4.el8 ceph-noarch 241 k 2023-10-03T10:10:48.453 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-rook noarch 2:18.2.0-507.g7cd99ce4.el8 ceph-noarch 47 k 2023-10-03T10:10:48.453 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mon x86_64 2:18.2.0-507.g7cd99ce4.el8 ceph 4.6 M 2023-10-03T10:10:48.453 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-osd x86_64 2:18.2.0-507.g7cd99ce4.el8 ceph 18 M 2023-10-03T10:10:48.453 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-prometheus-alerts noarch 2:18.2.0-507.g7cd99ce4.el8 ceph-noarch 15 k 2023-10-03T10:10:48.453 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-radosgw x86_64 2:18.2.0-507.g7cd99ce4.el8 ceph 13 M 2023-10-03T10:10:48.453 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-selinux x86_64 2:18.2.0-507.g7cd99ce4.el8 ceph 24 k 2023-10-03T10:10:48.453 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-test x86_64 2:18.2.0-507.g7cd99ce4.el8 ceph 47 M 2023-10-03T10:10:48.453 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-devel x86_64 2:18.2.0-507.g7cd99ce4.el8 ceph 30 k 2023-10-03T10:10:48.453 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2 x86_64 2:18.2.0-507.g7cd99ce4.el8 ceph 769 k 2023-10-03T10:10:48.454 INFO:teuthology.orchestra.run.smithi003.stdout: libcephsqlite x86_64 2:18.2.0-507.g7cd99ce4.el8 ceph 137 k 2023-10-03T10:10:48.454 INFO:teuthology.orchestra.run.smithi003.stdout: librados-devel x86_64 2:18.2.0-507.g7cd99ce4.el8 ceph 139 k 2023-10-03T10:10:48.454 INFO:teuthology.orchestra.run.smithi003.stdout: librados2 x86_64 2:18.2.0-507.g7cd99ce4.el8 ceph 3.7 M 2023-10-03T10:10:48.454 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1 x86_64 2:18.2.0-507.g7cd99ce4.el8 ceph 474 k 2023-10-03T10:10:48.454 INFO:teuthology.orchestra.run.smithi003.stdout: librbd1 x86_64 2:18.2.0-507.g7cd99ce4.el8 ceph 3.6 M 2023-10-03T10:10:48.454 INFO:teuthology.orchestra.run.smithi003.stdout: librgw2 x86_64 2:18.2.0-507.g7cd99ce4.el8 ceph 6.5 M 2023-10-03T10:10:48.454 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse x86_64 2:18.2.0-507.g7cd99ce4.el8 ceph 45 k 2023-10-03T10:10:48.454 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common x86_64 2:18.2.0-507.g7cd99ce4.el8 ceph 123 k 2023-10-03T10:10:48.454 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cephfs x86_64 2:18.2.0-507.g7cd99ce4.el8 ceph 190 k 2023-10-03T10:10:48.454 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados x86_64 2:18.2.0-507.g7cd99ce4.el8 ceph 370 k 2023-10-03T10:10:48.454 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rbd x86_64 2:18.2.0-507.g7cd99ce4.el8 ceph 364 k 2023-10-03T10:10:48.454 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rgw x86_64 2:18.2.0-507.g7cd99ce4.el8 ceph 107 k 2023-10-03T10:10:48.454 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-fuse x86_64 2:18.2.0-507.g7cd99ce4.el8 ceph 94 k 2023-10-03T10:10:48.455 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-mirror x86_64 2:18.2.0-507.g7cd99ce4.el8 ceph 3.4 M 2023-10-03T10:10:48.455 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-nbd x86_64 2:18.2.0-507.g7cd99ce4.el8 ceph 179 k 2023-10-03T10:10:48.455 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2023-10-03T10:10:48.455 INFO:teuthology.orchestra.run.smithi003.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-03T10:10:48.455 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:10:48.455 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2023-10-03T10:10:48.455 INFO:teuthology.orchestra.run.smithi003.stdout:====================================================================================== 2023-10-03T10:10:48.455 INFO:teuthology.orchestra.run.smithi003.stdout:Install 1 Package 2023-10-03T10:10:48.455 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrade 34 Packages 2023-10-03T10:10:48.455 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:10:48.456 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 145 M 2023-10-03T10:10:48.456 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2023-10-03T10:10:48.849 INFO:teuthology.orchestra.run.smithi003.stdout:(1/35): ceph-18.2.0-507.g7cd99ce4.el8.x86_64.rp 20 kB/s | 6.4 kB 00:00 2023-10-03T10:10:49.382 INFO:teuthology.orchestra.run.smithi003.stdout:(2/35): thrift-0.13.0-2.el8.x86_64.rpm 2.0 MB/s | 1.7 MB 00:00 2023-10-03T10:10:50.716 INFO:teuthology.orchestra.run.smithi003.stdout:(3/35): ceph-fuse-18.2.0-507.g7cd99ce4.el8.x86_ 636 kB/s | 847 kB 00:01 2023-10-03T10:10:51.641 INFO:teuthology.orchestra.run.smithi003.stdout:(4/35): ceph-base-18.2.0-507.g7cd99ce4.el8.x86_ 1.7 MB/s | 5.4 MB 00:03 2023-10-03T10:10:51.908 INFO:teuthology.orchestra.run.smithi003.stdout:(5/35): ceph-mds-18.2.0-507.g7cd99ce4.el8.x86_6 1.9 MB/s | 2.3 MB 00:01 2023-10-03T10:10:52.234 INFO:teuthology.orchestra.run.smithi003.stdout:(6/35): ceph-mgr-18.2.0-507.g7cd99ce4.el8.x86_6 2.6 MB/s | 1.5 MB 00:00 2023-10-03T10:10:52.792 INFO:teuthology.orchestra.run.smithi003.stdout:(7/35): ceph-mon-18.2.0-507.g7cd99ce4.el8.x86_6 5.2 MB/s | 4.6 MB 00:00 2023-10-03T10:10:53.056 INFO:teuthology.orchestra.run.smithi003.stdout:(8/35): ceph-common-18.2.0-507.g7cd99ce4.el8.x8 5.2 MB/s | 22 MB 00:04 2023-10-03T10:10:53.091 INFO:teuthology.orchestra.run.smithi003.stdout:(9/35): ceph-selinux-18.2.0-507.g7cd99ce4.el8.x 300 kB/s | 24 kB 00:00 2023-10-03T10:10:54.318 INFO:teuthology.orchestra.run.smithi003.stdout:(10/35): ceph-radosgw-18.2.0-507.g7cd99ce4.el8. 8.5 MB/s | 13 MB 00:01 2023-10-03T10:10:54.385 INFO:teuthology.orchestra.run.smithi003.stdout:(11/35): libcephfs-devel-18.2.0-507.g7cd99ce4.e 460 kB/s | 30 kB 00:00 2023-10-03T10:10:54.502 INFO:teuthology.orchestra.run.smithi003.stdout:(12/35): libcephfs2-18.2.0-507.g7cd99ce4.el8.x8 6.4 MB/s | 769 kB 00:00 2023-10-03T10:10:54.585 INFO:teuthology.orchestra.run.smithi003.stdout:(13/35): libcephsqlite-18.2.0-507.g7cd99ce4.el8 1.6 MB/s | 137 kB 00:00 2023-10-03T10:10:54.694 INFO:teuthology.orchestra.run.smithi003.stdout:(14/35): librados-devel-18.2.0-507.g7cd99ce4.el 1.3 MB/s | 139 kB 00:00 2023-10-03T10:10:55.069 INFO:teuthology.orchestra.run.smithi003.stdout:(15/35): librados2-18.2.0-507.g7cd99ce4.el8.x86 9.8 MB/s | 3.7 MB 00:00 2023-10-03T10:10:55.170 INFO:teuthology.orchestra.run.smithi003.stdout:(16/35): libradosstriper1-18.2.0-507.g7cd99ce4. 4.6 MB/s | 474 kB 00:00 2023-10-03T10:10:55.504 INFO:teuthology.orchestra.run.smithi003.stdout:(17/35): librbd1-18.2.0-507.g7cd99ce4.el8.x86_6 11 MB/s | 3.6 MB 00:00 2023-10-03T10:10:56.036 INFO:teuthology.orchestra.run.smithi003.stdout:(18/35): librgw2-18.2.0-507.g7cd99ce4.el8.x86_6 12 MB/s | 6.5 MB 00:00 2023-10-03T10:10:56.129 INFO:teuthology.orchestra.run.smithi003.stdout:(19/35): python3-ceph-argparse-18.2.0-507.g7cd9 482 kB/s | 45 kB 00:00 2023-10-03T10:10:56.204 INFO:teuthology.orchestra.run.smithi003.stdout:(20/35): python3-ceph-common-18.2.0-507.g7cd99c 1.6 MB/s | 123 kB 00:00 2023-10-03T10:10:56.296 INFO:teuthology.orchestra.run.smithi003.stdout:(21/35): python3-cephfs-18.2.0-507.g7cd99ce4.el 2.0 MB/s | 190 kB 00:00 2023-10-03T10:10:56.396 INFO:teuthology.orchestra.run.smithi003.stdout:(22/35): python3-rados-18.2.0-507.g7cd99ce4.el8 3.6 MB/s | 370 kB 00:00 2023-10-03T10:10:56.514 INFO:teuthology.orchestra.run.smithi003.stdout:(23/35): python3-rbd-18.2.0-507.g7cd99ce4.el8.x 3.0 MB/s | 364 kB 00:00 2023-10-03T10:10:56.872 INFO:teuthology.orchestra.run.smithi003.stdout:(24/35): python3-rgw-18.2.0-507.g7cd99ce4.el8.x 298 kB/s | 107 kB 00:00 2023-10-03T10:10:56.965 INFO:teuthology.orchestra.run.smithi003.stdout:(25/35): rbd-fuse-18.2.0-507.g7cd99ce4.el8.x86_ 1.0 MB/s | 94 kB 00:00 2023-10-03T10:10:57.323 INFO:teuthology.orchestra.run.smithi003.stdout:(26/35): rbd-mirror-18.2.0-507.g7cd99ce4.el8.x8 9.6 MB/s | 3.4 MB 00:00 2023-10-03T10:10:57.726 INFO:teuthology.orchestra.run.smithi003.stdout:(27/35): rbd-nbd-18.2.0-507.g7cd99ce4.el8.x86_6 1.9 MB/s | 179 kB 00:00 2023-10-03T10:10:57.726 INFO:teuthology.orchestra.run.smithi003.stdout:(28/35): ceph-grafana-dashboards-18.2.0-507.g7c 309 kB/s | 23 kB 00:00 2023-10-03T10:10:57.726 INFO:teuthology.orchestra.run.smithi003.stdout:(29/35): ceph-mgr-dashboard-18.2.0-507.g7cd99ce 7.7 MB/s | 1.7 MB 00:00 2023-10-03T10:10:58.541 INFO:teuthology.orchestra.run.smithi003.stdout:(30/35): ceph-mgr-diskprediction-local-18.2.0-5 9.0 MB/s | 7.4 MB 00:00 2023-10-03T10:10:58.633 INFO:teuthology.orchestra.run.smithi003.stdout:(31/35): ceph-mgr-modules-core-18.2.0-507.g7cd9 2.6 MB/s | 241 kB 00:00 2023-10-03T10:10:58.699 INFO:teuthology.orchestra.run.smithi003.stdout:(32/35): ceph-mgr-rook-18.2.0-507.g7cd99ce4.el8 715 kB/s | 47 kB 00:00 2023-10-03T10:10:58.783 INFO:teuthology.orchestra.run.smithi003.stdout:(33/35): ceph-prometheus-alerts-18.2.0-507.g7cd 174 kB/s | 15 kB 00:00 2023-10-03T10:10:59.025 INFO:teuthology.orchestra.run.smithi003.stdout:(34/35): ceph-osd-18.2.0-507.g7cd99ce4.el8.x86_ 2.6 MB/s | 18 MB 00:06 2023-10-03T10:11:01.800 INFO:teuthology.orchestra.run.smithi003.stdout:(35/35): ceph-test-18.2.0-507.g7cd99ce4.el8.x86 5.4 MB/s | 47 MB 00:08 2023-10-03T10:11:01.803 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2023-10-03T10:11:01.804 INFO:teuthology.orchestra.run.smithi003.stdout:Total 11 MB/s | 145 MB 00:13 2023-10-03T10:11:01.805 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2023-10-03T10:11:01.884 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2023-10-03T10:11:01.885 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2023-10-03T10:11:02.613 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2023-10-03T10:11:02.617 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2023-10-03T10:11:05.217 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2023-10-03T10:11:05.412 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: thrift-0.13.0-2.el8.x86_64 1/1 2023-10-03T10:11:05.981 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/69 2023-10-03T10:11:06.310 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librados2-2:18.2.0-507.g7cd99ce4.el8.x86_64 2/69 2023-10-03T10:11:06.753 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados2-2:18.2.0-507.g7cd99ce4.el8.x86_64 2/69 2023-10-03T10:11:06.968 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librbd1-2:18.2.0-507.g7cd99ce4.el8.x86_64 3/69 2023-10-03T10:11:07.102 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librbd1-2:18.2.0-507.g7cd99ce4.el8.x86_64 3/69 2023-10-03T10:11:07.310 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : libcephfs2-2:18.2.0-507.g7cd99ce4.el8.x86_64 4/69 2023-10-03T10:11:08.359 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephfs2-2:18.2.0-507.g7cd99ce4.el8.x86_64 4/69 2023-10-03T10:11:08.576 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librgw2-2:18.2.0-507.g7cd99ce4.el8.x86_64 5/69 2023-10-03T10:11:08.716 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librgw2-2:18.2.0-507.g7cd99ce4.el8.x86_64 5/69 2023-10-03T10:11:08.940 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : python3-rados-2:18.2.0-507.g7cd99ce4.el8.x86_64 6/69 2023-10-03T10:11:09.126 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : libcephsqlite-2:18.2.0-507.g7cd99ce4.el8.x86_64 7/69 2023-10-03T10:11:09.241 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephsqlite-2:18.2.0-507.g7cd99ce4.el8.x86_64 7/69 2023-10-03T10:11:09.434 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : libradosstriper1-2:18.2.0-507.g7cd99ce4.el8.x86_64 8/69 2023-10-03T10:11:09.511 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libradosstriper1-2:18.2.0-507.g7cd99ce4.el8.x86_64 8/69 2023-10-03T10:11:09.690 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : python3-ceph-argparse-2:18.2.0-507.g7cd99ce4.el8.x 9/69 2023-10-03T10:11:09.941 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : python3-cephfs-2:18.2.0-507.g7cd99ce4.el8.x86_64 10/69 2023-10-03T10:11:10.173 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : python3-rbd-2:18.2.0-507.g7cd99ce4.el8.x86_64 11/69 2023-10-03T10:11:10.401 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : python3-rgw-2:18.2.0-507.g7cd99ce4.el8.x86_64 12/69 2023-10-03T10:11:10.595 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librados-devel-2:18.2.0-507.g7cd99ce4.el8.x86_64 13/69 2023-10-03T10:11:10.690 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : ceph-prometheus-alerts-2:18.2.0-507.g7cd99ce4.el8. 14/69 2023-10-03T10:11:10.906 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : ceph-grafana-dashboards-2:18.2.0-507.g7cd99ce4.el8 15/69 2023-10-03T10:11:11.109 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : python3-ceph-common-2:18.2.0-507.g7cd99ce4.el8.x86 16/69 2023-10-03T10:11:13.654 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:18.2.0-507.g7cd99ce4.el8.x86_64 17/69 2023-10-03T10:11:13.918 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : ceph-common-2:18.2.0-507.g7cd99ce4.el8.x86_64 17/69 2023-10-03T10:11:14.636 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:18.2.0-507.g7cd99ce4.el8.x86_64 17/69 2023-10-03T10:11:14.913 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : ceph-base-2:18.2.0-507.g7cd99ce4.el8.x86_64 18/69 2023-10-03T10:11:14.985 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-base-2:18.2.0-507.g7cd99ce4.el8.x86_64 18/69 2023-10-03T10:11:15.175 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : ceph-selinux-2:18.2.0-507.g7cd99ce4.el8.x86_64 19/69 2023-10-03T10:11:35.506 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-selinux-2:18.2.0-507.g7cd99ce4.el8.x86_64 19/69 2023-10-03T10:11:35.734 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : ceph-mgr-dashboard-2:18.2.0-507.g7cd99ce4.el8.noar 20/69 2023-10-03T10:11:37.019 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-507.g7cd99ce4.el8.noar 20/69 2023-10-03T10:11:37.213 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : ceph-mgr-diskprediction-local-2:18.2.0-507.g7cd99c 21/69 2023-10-03T10:11:37.320 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-507.g7cd99c 21/69 2023-10-03T10:11:37.905 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : ceph-mgr-modules-core-2:18.2.0-507.g7cd99ce4.el8.n 22/69 2023-10-03T10:11:38.159 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : ceph-mgr-2:18.2.0-507.g7cd99ce4.el8.x86_64 23/69 2023-10-03T10:11:38.189 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-2:18.2.0-507.g7cd99ce4.el8.x86_64 23/69 2023-10-03T10:11:38.338 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : ceph-mgr-rook-2:18.2.0-507.g7cd99ce4.el8.noarch 24/69 2023-10-03T10:11:38.658 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-507.g7cd99ce4.el8.noarch 24/69 2023-10-03T10:11:38.864 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : ceph-mds-2:18.2.0-507.g7cd99ce4.el8.x86_64 25/69 2023-10-03T10:11:39.425 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mds-2:18.2.0-507.g7cd99ce4.el8.x86_64 25/69 2023-10-03T10:11:39.847 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : ceph-mon-2:18.2.0-507.g7cd99ce4.el8.x86_64 26/69 2023-10-03T10:11:41.834 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mon-2:18.2.0-507.g7cd99ce4.el8.x86_64 26/69 2023-10-03T10:11:42.083 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : ceph-osd-2:18.2.0-507.g7cd99ce4.el8.x86_64 27/69 2023-10-03T10:11:42.098 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-osd-2:18.2.0-507.g7cd99ce4.el8.x86_64 27/69 2023-10-03T10:11:43.707 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : ceph-2:18.2.0-507.g7cd99ce4.el8.x86_64 28/69 2023-10-03T10:11:43.975 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : ceph-radosgw-2:18.2.0-507.g7cd99ce4.el8.x86_64 29/69 2023-10-03T10:11:44.429 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-507.g7cd99ce4.el8.x86_64 29/69 2023-10-03T10:11:44.741 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : rbd-mirror-2:18.2.0-507.g7cd99ce4.el8.x86_64 30/69 2023-10-03T10:11:50.177 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: rbd-mirror-2:18.2.0-507.g7cd99ce4.el8.x86_64 30/69 2023-10-03T10:11:50.492 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : ceph-test-2:18.2.0-507.g7cd99ce4.el8.x86_64 31/69 2023-10-03T10:11:50.668 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : libcephfs-devel-2:18.2.0-507.g7cd99ce4.el8.x86_64 32/69 2023-10-03T10:11:50.871 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : rbd-fuse-2:18.2.0-507.g7cd99ce4.el8.x86_64 33/69 2023-10-03T10:11:51.352 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : rbd-nbd-2:18.2.0-507.g7cd99ce4.el8.x86_64 34/69 2023-10-03T10:11:51.580 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : ceph-fuse-2:18.2.0-507.g7cd99ce4.el8.x86_64 35/69 2023-10-03T10:11:51.733 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 36/69 2023-10-03T10:11:51.979 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 37/69 2023-10-03T10:11:51.989 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 38/69 2023-10-03T10:11:51.990 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 38/69 2023-10-03T10:11:52.275 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 38/69 2023-10-03T10:11:52.429 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 39/69 2023-10-03T10:11:52.429 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 40/69 2023-10-03T10:11:52.622 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 40/69 2023-10-03T10:11:52.629 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 41/69 2023-10-03T10:11:52.630 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 41/69 2023-10-03T10:11:52.871 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 41/69 2023-10-03T10:11:53.090 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8.n 42/69 2023-10-03T10:11:53.287 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 43/69 2023-10-03T10:11:53.295 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 44/69 2023-10-03T10:11:53.295 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 44/69 2023-10-03T10:11:53.521 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 44/69 2023-10-03T10:11:53.670 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 45/69 2023-10-03T10:11:53.670 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 46/69 2023-10-03T10:11:53.888 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 46/69 2023-10-03T10:11:54.070 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 47/69 2023-10-03T10:11:54.079 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 48/69 2023-10-03T10:11:54.080 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 48/69 2023-10-03T10:11:54.497 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 48/69 2023-10-03T10:11:54.497 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.noar 49/69 2023-10-03T10:11:54.767 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.noar 49/69 2023-10-03T10:11:54.775 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 50/69 2023-10-03T10:11:54.776 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 50/69 2023-10-03T10:11:54.971 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 50/69 2023-10-03T10:11:54.978 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 51/69 2023-10-03T10:11:54.979 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 51/69 2023-10-03T10:11:55.170 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 51/69 2023-10-03T10:11:55.171 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 52/69 2023-10-03T10:11:55.338 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 52/69 2023-10-03T10:11:55.369 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 53/69 2023-10-03T10:11:55.369 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 53/69 2023-10-03T10:11:55.654 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 53/69 2023-10-03T10:11:55.654 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 54/69 2023-10-03T10:11:55.879 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 54/69 2023-10-03T10:11:55.879 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_64 55/69 2023-10-03T10:11:56.096 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_64 55/69 2023-10-03T10:11:56.312 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 56/69 2023-10-03T10:11:56.520 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 57/69 2023-10-03T10:11:56.521 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 58/69 2023-10-03T10:11:56.779 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 58/69 2023-10-03T10:11:56.978 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 59/69 2023-10-03T10:11:56.978 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 60/69 2023-10-03T10:11:57.212 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 60/69 2023-10-03T10:11:57.370 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 61/69 2023-10-03T10:11:57.370 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 62/69 2023-10-03T10:11:57.672 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 62/69 2023-10-03T10:11:57.828 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 63/69 2023-10-03T10:11:57.971 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8.x 64/69 2023-10-03T10:11:58.178 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x86 65/69 2023-10-03T10:11:58.286 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.el8 66/69 2023-10-03T10:11:58.413 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el8. 67/69 2023-10-03T10:11:58.413 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : ceph-mgr-diskprediction-local-2:16.2.14-44.geeb13a 68/69 2023-10-03T10:11:58.596 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.14-44.geeb13a 68/69 2023-10-03T10:11:58.596 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 69/69 2023-10-03T10:11:59.418 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 69/69 2023-10-03T10:11:59.418 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 1/69 2023-10-03T10:11:59.418 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-2:18.2.0-507.g7cd99ce4.el8.x86_64 2/69 2023-10-03T10:11:59.419 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 3/69 2023-10-03T10:11:59.419 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-base-2:18.2.0-507.g7cd99ce4.el8.x86_64 4/69 2023-10-03T10:11:59.419 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 5/69 2023-10-03T10:11:59.419 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-common-2:18.2.0-507.g7cd99ce4.el8.x86_64 6/69 2023-10-03T10:11:59.419 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 7/69 2023-10-03T10:11:59.419 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-fuse-2:18.2.0-507.g7cd99ce4.el8.x86_64 8/69 2023-10-03T10:11:59.419 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 9/69 2023-10-03T10:11:59.419 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mds-2:18.2.0-507.g7cd99ce4.el8.x86_64 10/69 2023-10-03T10:11:59.419 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 11/69 2023-10-03T10:11:59.419 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-2:18.2.0-507.g7cd99ce4.el8.x86_64 12/69 2023-10-03T10:11:59.419 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 13/69 2023-10-03T10:11:59.419 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mon-2:18.2.0-507.g7cd99ce4.el8.x86_64 14/69 2023-10-03T10:11:59.420 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 15/69 2023-10-03T10:11:59.420 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-osd-2:18.2.0-507.g7cd99ce4.el8.x86_64 16/69 2023-10-03T10:11:59.420 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 17/69 2023-10-03T10:11:59.420 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-radosgw-2:18.2.0-507.g7cd99ce4.el8.x86_64 18/69 2023-10-03T10:11:59.420 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 19/69 2023-10-03T10:11:59.420 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-selinux-2:18.2.0-507.g7cd99ce4.el8.x86_64 20/69 2023-10-03T10:11:59.420 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 21/69 2023-10-03T10:11:59.420 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-test-2:18.2.0-507.g7cd99ce4.el8.x86_64 22/69 2023-10-03T10:11:59.420 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 23/69 2023-10-03T10:11:59.420 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs-devel-2:18.2.0-507.g7cd99ce4.el8.x86_64 24/69 2023-10-03T10:11:59.420 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 25/69 2023-10-03T10:11:59.420 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs2-2:18.2.0-507.g7cd99ce4.el8.x86_64 26/69 2023-10-03T10:11:59.421 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 27/69 2023-10-03T10:11:59.421 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephsqlite-2:18.2.0-507.g7cd99ce4.el8.x86_64 28/69 2023-10-03T10:11:59.421 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 29/69 2023-10-03T10:11:59.421 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados-devel-2:18.2.0-507.g7cd99ce4.el8.x86_64 30/69 2023-10-03T10:11:59.421 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 31/69 2023-10-03T10:11:59.421 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados2-2:18.2.0-507.g7cd99ce4.el8.x86_64 32/69 2023-10-03T10:11:59.421 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 33/69 2023-10-03T10:11:59.421 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libradosstriper1-2:18.2.0-507.g7cd99ce4.el8.x86_64 34/69 2023-10-03T10:11:59.421 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_64 35/69 2023-10-03T10:11:59.421 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librbd1-2:18.2.0-507.g7cd99ce4.el8.x86_64 36/69 2023-10-03T10:11:59.421 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 37/69 2023-10-03T10:11:59.421 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librgw2-2:18.2.0-507.g7cd99ce4.el8.x86_64 38/69 2023-10-03T10:11:59.422 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 39/69 2023-10-03T10:11:59.422 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-argparse-2:18.2.0-507.g7cd99ce4.el8.x 40/69 2023-10-03T10:11:59.422 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8.x 41/69 2023-10-03T10:11:59.422 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-common-2:18.2.0-507.g7cd99ce4.el8.x86 42/69 2023-10-03T10:11:59.422 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x86 43/69 2023-10-03T10:11:59.422 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cephfs-2:18.2.0-507.g7cd99ce4.el8.x86_64 44/69 2023-10-03T10:11:59.422 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 45/69 2023-10-03T10:11:59.422 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rados-2:18.2.0-507.g7cd99ce4.el8.x86_64 46/69 2023-10-03T10:11:59.422 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 47/69 2023-10-03T10:11:59.422 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rbd-2:18.2.0-507.g7cd99ce4.el8.x86_64 48/69 2023-10-03T10:11:59.422 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 49/69 2023-10-03T10:11:59.422 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rgw-2:18.2.0-507.g7cd99ce4.el8.x86_64 50/69 2023-10-03T10:11:59.422 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 51/69 2023-10-03T10:11:59.423 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-fuse-2:18.2.0-507.g7cd99ce4.el8.x86_64 52/69 2023-10-03T10:11:59.423 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 53/69 2023-10-03T10:11:59.423 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-mirror-2:18.2.0-507.g7cd99ce4.el8.x86_64 54/69 2023-10-03T10:11:59.423 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 55/69 2023-10-03T10:11:59.423 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-nbd-2:18.2.0-507.g7cd99ce4.el8.x86_64 56/69 2023-10-03T10:11:59.423 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 57/69 2023-10-03T10:11:59.423 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-507.g7cd99ce4.el8 58/69 2023-10-03T10:11:59.423 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.el8 59/69 2023-10-03T10:11:59.423 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-507.g7cd99ce4.el8.noar 60/69 2023-10-03T10:11:59.423 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.noar 61/69 2023-10-03T10:11:59.423 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-507.g7cd99c 62/69 2023-10-03T10:11:59.423 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.14-44.geeb13a 63/69 2023-10-03T10:11:59.423 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-507.g7cd99ce4.el8.n 64/69 2023-10-03T10:11:59.424 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8.n 65/69 2023-10-03T10:11:59.424 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-rook-2:18.2.0-507.g7cd99ce4.el8.noarch 66/69 2023-10-03T10:11:59.424 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 67/69 2023-10-03T10:11:59.424 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-507.g7cd99ce4.el8. 68/69 2023-10-03T10:12:01.484 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el8. 69/69 2023-10-03T10:12:01.484 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:12:01.484 INFO:teuthology.orchestra.run.smithi003.stdout:Upgraded: 2023-10-03T10:12:01.484 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-2:18.2.0-507.g7cd99ce4.el8.x86_64 2023-10-03T10:12:01.484 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base-2:18.2.0-507.g7cd99ce4.el8.x86_64 2023-10-03T10:12:01.484 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-common-2:18.2.0-507.g7cd99ce4.el8.x86_64 2023-10-03T10:12:01.485 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-fuse-2:18.2.0-507.g7cd99ce4.el8.x86_64 2023-10-03T10:12:01.485 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-grafana-dashboards-2:18.2.0-507.g7cd99ce4.el8.noarch 2023-10-03T10:12:01.485 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mds-2:18.2.0-507.g7cd99ce4.el8.x86_64 2023-10-03T10:12:01.485 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-2:18.2.0-507.g7cd99ce4.el8.x86_64 2023-10-03T10:12:01.485 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard-2:18.2.0-507.g7cd99ce4.el8.noarch 2023-10-03T10:12:01.485 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-diskprediction-local-2:18.2.0-507.g7cd99ce4.el8.noarch 2023-10-03T10:12:01.485 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core-2:18.2.0-507.g7cd99ce4.el8.noarch 2023-10-03T10:12:01.485 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-rook-2:18.2.0-507.g7cd99ce4.el8.noarch 2023-10-03T10:12:01.485 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mon-2:18.2.0-507.g7cd99ce4.el8.x86_64 2023-10-03T10:12:01.485 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-osd-2:18.2.0-507.g7cd99ce4.el8.x86_64 2023-10-03T10:12:01.485 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-prometheus-alerts-2:18.2.0-507.g7cd99ce4.el8.noarch 2023-10-03T10:12:01.485 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-radosgw-2:18.2.0-507.g7cd99ce4.el8.x86_64 2023-10-03T10:12:01.486 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-selinux-2:18.2.0-507.g7cd99ce4.el8.x86_64 2023-10-03T10:12:01.486 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-test-2:18.2.0-507.g7cd99ce4.el8.x86_64 2023-10-03T10:12:01.486 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-devel-2:18.2.0-507.g7cd99ce4.el8.x86_64 2023-10-03T10:12:01.486 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2-2:18.2.0-507.g7cd99ce4.el8.x86_64 2023-10-03T10:12:01.486 INFO:teuthology.orchestra.run.smithi003.stdout: libcephsqlite-2:18.2.0-507.g7cd99ce4.el8.x86_64 2023-10-03T10:12:01.486 INFO:teuthology.orchestra.run.smithi003.stdout: librados-devel-2:18.2.0-507.g7cd99ce4.el8.x86_64 2023-10-03T10:12:01.486 INFO:teuthology.orchestra.run.smithi003.stdout: librados2-2:18.2.0-507.g7cd99ce4.el8.x86_64 2023-10-03T10:12:01.486 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1-2:18.2.0-507.g7cd99ce4.el8.x86_64 2023-10-03T10:12:01.486 INFO:teuthology.orchestra.run.smithi003.stdout: librbd1-2:18.2.0-507.g7cd99ce4.el8.x86_64 2023-10-03T10:12:01.486 INFO:teuthology.orchestra.run.smithi003.stdout: librgw2-2:18.2.0-507.g7cd99ce4.el8.x86_64 2023-10-03T10:12:01.486 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse-2:18.2.0-507.g7cd99ce4.el8.x86_64 2023-10-03T10:12:01.486 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common-2:18.2.0-507.g7cd99ce4.el8.x86_64 2023-10-03T10:12:01.486 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cephfs-2:18.2.0-507.g7cd99ce4.el8.x86_64 2023-10-03T10:12:01.487 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados-2:18.2.0-507.g7cd99ce4.el8.x86_64 2023-10-03T10:12:01.487 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rbd-2:18.2.0-507.g7cd99ce4.el8.x86_64 2023-10-03T10:12:01.487 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rgw-2:18.2.0-507.g7cd99ce4.el8.x86_64 2023-10-03T10:12:01.487 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-fuse-2:18.2.0-507.g7cd99ce4.el8.x86_64 2023-10-03T10:12:01.487 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-mirror-2:18.2.0-507.g7cd99ce4.el8.x86_64 2023-10-03T10:12:01.487 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-nbd-2:18.2.0-507.g7cd99ce4.el8.x86_64 2023-10-03T10:12:01.487 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2023-10-03T10:12:01.487 INFO:teuthology.orchestra.run.smithi003.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-03T10:12:01.487 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:12:01.487 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2023-10-03T10:12:01.632 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=reef 2023-10-03T10:12:01.849 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-03T10:12:01.880 INFO:teuthology.orchestra.run.smithi003.stdout:18.2.0-507.g7cd99ce4.el8 2023-10-03T10:12:01.880 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-507.g7cd99ce4.el8 2023-10-03T10:12:01.880 INFO:teuthology.task.install:The correct ceph version 18.2.0-507.g7cd99ce4 is installed. 2023-10-03T10:12:01.881 INFO:teuthology.run_tasks:Running task print... 2023-10-03T10:12:01.889 INFO:teuthology.task.print:**** done install.upgrade the host 2023-10-03T10:12:01.890 INFO:teuthology.run_tasks:Running task ceph.restart... 2023-10-03T10:12:01.980 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-03T10:12:01.980 INFO:teuthology.orchestra.run:waiting for 300 2023-10-03T10:12:01.993 INFO:tasks.ceph.mon.a:Stopped 2023-10-03T10:12:01.993 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-03T10:12:01.994 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-03T10:12:02.036 INFO:tasks.ceph.mon.a:Started 2023-10-03T10:12:02.036 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-03T10:12:02.036 INFO:teuthology.orchestra.run:waiting for 300 2023-10-03T10:12:02.051 INFO:tasks.ceph.mon.b:Stopped 2023-10-03T10:12:02.051 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-03T10:12:02.051 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-03T10:12:02.093 INFO:tasks.ceph.mon.b:Started 2023-10-03T10:12:02.094 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-03T10:12:02.094 INFO:teuthology.orchestra.run:waiting for 300 2023-10-03T10:12:02.107 INFO:tasks.ceph.mon.c:Stopped 2023-10-03T10:12:02.107 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-03T10:12:02.107 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-03T10:12:02.149 INFO:tasks.ceph.mon.c:Started 2023-10-03T10:12:02.150 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-03T10:12:02.150 INFO:teuthology.orchestra.run:waiting for 300 2023-10-03T10:12:02.180 INFO:tasks.ceph.mgr.x:Stopped 2023-10-03T10:12:02.180 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-03T10:12:02.181 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-03T10:12:02.223 INFO:tasks.ceph.mgr.x:Started 2023-10-03T10:12:02.224 INFO:teuthology.run_tasks:Running task ceph.healthy... 2023-10-03T10:12:02.233 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-03T10:12:02.233 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-03T10:12:02.233 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-03T10:12:02.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.380+0000 7f3b3b570700 1 Processor -- start 2023-10-03T10:12:02.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.380+0000 7f3b3b570700 1 -- start start 2023-10-03T10:12:02.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.381+0000 7f3b3b570700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 0x7f3b34138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:02.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.381+0000 7f3b3b570700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b34139170 0x7f3b3413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:02.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.381+0000 7f3b3b570700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b3409d2f0 con 0x7f3b3409d6f0 2023-10-03T10:12:02.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.381+0000 7f3b3b570700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b3409b0f0 con 0x7f3b3409e1d0 2023-10-03T10:12:02.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.381+0000 7f3b3b570700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3409a810 con 0x7f3b34139170 2023-10-03T10:12:02.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.381+0000 7f3b3930c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3b3409d6f0 0x7f3b3409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57784/0 (socket says 172.21.15.3:57784) 2023-10-03T10:12:02.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.381+0000 7f3b38b0b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 0x7f3b34138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:02.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.381+0000 7f3b3930c700 1 -- 172.21.15.3:0/3133005781 learned_addr learned my addr 172.21.15.3:0/3133005781 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:02.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.381+0000 7f3b39b0d700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b34139170 0x7f3b3413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:02.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.381+0000 7f3b38b0b700 1 -- 172.21.15.3:0/3133005781 >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 msgr2=0x7f3b34138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-03T10:12:02.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.381+0000 7f3b38b0b700 1 -- 172.21.15.3:0/3133005781 >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 msgr2=0x7f3b34138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-03T10:12:02.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.381+0000 7f3b38b0b700 1 --2- 172.21.15.3:0/3133005781 >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 0x7f3b34138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-03T10:12:02.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.381+0000 7f3b38b0b700 1 --2- 172.21.15.3:0/3133005781 >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 0x7f3b34138c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-03T10:12:02.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.381+0000 7f3b39b0d700 1 -- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b34139170 msgr2=0x7f3b3413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-03T10:12:02.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.381+0000 7f3b39b0d700 1 -- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b34139170 msgr2=0x7f3b3413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-03T10:12:02.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.381+0000 7f3b39b0d700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b34139170 0x7f3b3413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-03T10:12:02.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.381+0000 7f3b39b0d700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b34139170 0x7f3b3413b550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-03T10:12:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.582+0000 7f3b38b0b700 1 --2- 172.21.15.3:0/3133005781 >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 0x7f3b34138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:02.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.582+0000 7f3b39b0d700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b34139170 0x7f3b3413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:02.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.582+0000 7f3b38b0b700 1 -- 172.21.15.3:0/3133005781 >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 msgr2=0x7f3b34138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-03T10:12:02.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.582+0000 7f3b38b0b700 1 -- 172.21.15.3:0/3133005781 >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 msgr2=0x7f3b34138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-03T10:12:02.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.582+0000 7f3b38b0b700 1 --2- 172.21.15.3:0/3133005781 >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 0x7f3b34138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-03T10:12:02.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.582+0000 7f3b38b0b700 1 --2- 172.21.15.3:0/3133005781 >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 0x7f3b34138c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-03T10:12:02.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.582+0000 7f3b39b0d700 1 -- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b34139170 msgr2=0x7f3b3413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-03T10:12:02.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.582+0000 7f3b39b0d700 1 -- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b34139170 msgr2=0x7f3b3413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-03T10:12:02.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.582+0000 7f3b39b0d700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b34139170 0x7f3b3413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-03T10:12:02.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.582+0000 7f3b39b0d700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b34139170 0x7f3b3413b550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-03T10:12:02.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.983+0000 7f3b39b0d700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b34139170 0x7f3b3413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:02.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.983+0000 7f3b38b0b700 1 --2- 172.21.15.3:0/3133005781 >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 0x7f3b34138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:02.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.983+0000 7f3b39b0d700 1 -- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b34139170 msgr2=0x7f3b3413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-03T10:12:02.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.983+0000 7f3b39b0d700 1 -- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b34139170 msgr2=0x7f3b3413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-03T10:12:02.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.983+0000 7f3b39b0d700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b34139170 0x7f3b3413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-03T10:12:02.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.983+0000 7f3b38b0b700 1 -- 172.21.15.3:0/3133005781 >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 msgr2=0x7f3b34138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-03T10:12:02.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.983+0000 7f3b39b0d700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b34139170 0x7f3b3413b550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-03T10:12:02.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.983+0000 7f3b38b0b700 1 -- 172.21.15.3:0/3133005781 >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 msgr2=0x7f3b34138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-03T10:12:02.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.983+0000 7f3b38b0b700 1 --2- 172.21.15.3:0/3133005781 >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 0x7f3b34138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-03T10:12:02.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:02.983+0000 7f3b38b0b700 1 --2- 172.21.15.3:0/3133005781 >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 0x7f3b34138c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-03T10:12:03.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:03.784+0000 7f3b39b0d700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b34139170 0x7f3b3413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:03.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:03.784+0000 7f3b38b0b700 1 --2- 172.21.15.3:0/3133005781 >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 0x7f3b34138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:03.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:03.784+0000 7f3b39b0d700 1 -- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b34139170 msgr2=0x7f3b3413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-03T10:12:03.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:03.784+0000 7f3b39b0d700 1 -- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b34139170 msgr2=0x7f3b3413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-03T10:12:03.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:03.784+0000 7f3b39b0d700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b34139170 0x7f3b3413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-03T10:12:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:03.784+0000 7f3b38b0b700 1 -- 172.21.15.3:0/3133005781 >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 msgr2=0x7f3b34138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-03T10:12:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:03.784+0000 7f3b38b0b700 1 -- 172.21.15.3:0/3133005781 >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 msgr2=0x7f3b34138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-03T10:12:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:03.784+0000 7f3b39b0d700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b34139170 0x7f3b3413b550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-03T10:12:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:03.784+0000 7f3b38b0b700 1 --2- 172.21.15.3:0/3133005781 >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 0x7f3b34138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-03T10:12:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:03.784+0000 7f3b38b0b700 1 --2- 172.21.15.3:0/3133005781 >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 0x7f3b34138c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-03T10:12:05.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:05.381+0000 7f3b26ffd700 1 -- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b34139170 msgr2=0x7f3b3413b550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-03T10:12:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:05.381+0000 7f3b26ffd700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b34139170 0x7f3b3413b550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:05.381+0000 7f3b26ffd700 1 -- 172.21.15.3:0/3133005781 >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 msgr2=0x7f3b34138c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-03T10:12:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:05.381+0000 7f3b26ffd700 1 --2- 172.21.15.3:0/3133005781 >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 0x7f3b34138c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:05.381+0000 7f3b26ffd700 1 -- 172.21.15.3:0/3133005781 >> v1:172.21.15.3:6789/0 conn(0x7f3b3409d6f0 legacy=0x7f3b3409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:05.381+0000 7f3b26ffd700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 0x7f3b18005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:05.381+0000 7f3b26ffd700 1 -- 172.21.15.3:0/3133005781 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3409b0f0 con 0x7f3b18003700 2023-10-03T10:12:05.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:05.381+0000 7f3b3930c700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 0x7f3b18005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:05.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:05.381+0000 7f3b3930c700 1 -- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 msgr2=0x7f3b18005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-03T10:12:05.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:05.381+0000 7f3b3930c700 1 -- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 msgr2=0x7f3b18005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-03T10:12:05.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:05.381+0000 7f3b3930c700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 0x7f3b18005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-03T10:12:05.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:05.381+0000 7f3b3930c700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 0x7f3b18005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-03T10:12:05.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:05.582+0000 7f3b3930c700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 0x7f3b18005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:05.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:05.582+0000 7f3b3930c700 1 -- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 msgr2=0x7f3b18005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-03T10:12:05.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:05.582+0000 7f3b3930c700 1 -- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 msgr2=0x7f3b18005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-03T10:12:05.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:05.582+0000 7f3b3930c700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 0x7f3b18005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-03T10:12:05.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:05.582+0000 7f3b3930c700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 0x7f3b18005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-03T10:12:05.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:05.983+0000 7f3b3930c700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 0x7f3b18005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:05.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:05.983+0000 7f3b3930c700 1 -- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 msgr2=0x7f3b18005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-03T10:12:05.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:05.983+0000 7f3b3930c700 1 -- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 msgr2=0x7f3b18005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-03T10:12:05.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:05.983+0000 7f3b3930c700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 0x7f3b18005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-03T10:12:05.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:05.983+0000 7f3b3930c700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 0x7f3b18005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-03T10:12:06.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:06.784+0000 7f3b3930c700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 0x7f3b18005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:06.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:06.784+0000 7f3b3930c700 1 -- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 msgr2=0x7f3b18005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-03T10:12:06.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:06.784+0000 7f3b3930c700 1 -- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 msgr2=0x7f3b18005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-03T10:12:06.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:06.784+0000 7f3b3930c700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 0x7f3b18005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-03T10:12:06.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:06.784+0000 7f3b3930c700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 0x7f3b18005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-03T10:12:07.579 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:07.579+0000 7f2cd33f8240 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-03T10:12:07.766 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:07.765+0000 7f2cd33f8240 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-03T10:12:08.047 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:08.046+0000 7f2cd33f8240 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-03T10:12:08.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.381+0000 7f3b26ffd700 1 -- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 msgr2=0x7f3b18005b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-03T10:12:08.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.381+0000 7f3b26ffd700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 0x7f3b18005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:08.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.381+0000 7f3b26ffd700 1 --2- 172.21.15.3:0/3133005781 >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 0x7f3b180084b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:08.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.381+0000 7f3b26ffd700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 0x7f3b18005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:08.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.381+0000 7f3b26ffd700 1 -- 172.21.15.3:0/3133005781 --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b3409a470 con 0x7f3b3409d6f0 2023-10-03T10:12:08.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.381+0000 7f3b26ffd700 1 -- 172.21.15.3:0/3133005781 --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b3409b0f0 con 0x7f3b3409e1d0 2023-10-03T10:12:08.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.381+0000 7f3b26ffd700 1 -- 172.21.15.3:0/3133005781 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3409a810 con 0x7f3b18003700 2023-10-03T10:12:08.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.381+0000 7f3b3930c700 1 --2- 172.21.15.3:0/3133005781 >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 0x7f3b180084b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:08.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.381+0000 7f3b38b0b700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 0x7f3b18005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:08.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.382+0000 7f3b38b0b700 1 -- 172.21.15.3:0/3133005781 >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 msgr2=0x7f3b180084b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:08.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.382+0000 7f3b38b0b700 1 --2- 172.21.15.3:0/3133005781 >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 0x7f3b180084b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:08.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.382+0000 7f3b38b0b700 1 -- 172.21.15.3:0/3133005781 >> v1:172.21.15.3:6789/0 conn(0x7f3b3409d6f0 legacy=0x7f3b1800aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:08.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.382+0000 7f3b38b0b700 1 -- 172.21.15.3:0/3133005781 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1800d290 con 0x7f3b18003700 2023-10-03T10:12:08.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.382+0000 7f3b27fff700 1 -- 172.21.15.3:0/3133005781 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090704270 0 0) 0x7f3b3409a470 con 0x7f3b3409d6f0 2023-10-03T10:12:08.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.382+0000 7f3b3930c700 1 --2- 172.21.15.3:0/3133005781 >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 0x7f3b180084b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:08.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.382+0000 7f3b38b0b700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 0x7f3b18005b80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009100 tx=0x7f3b28010c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afe24b8324109e78 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:08.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.382+0000 7f3b27fff700 1 -- 172.21.15.3:0/3133005781 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b28004020 con 0x7f3b18003700 2023-10-03T10:12:08.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.382+0000 7f3b27fff700 1 -- 172.21.15.3:0/3133005781 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2800c070 con 0x7f3b18003700 2023-10-03T10:12:08.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.382+0000 7f3b27fff700 1 -- 172.21.15.3:0/3133005781 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b28004380 con 0x7f3b18003700 2023-10-03T10:12:08.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.383+0000 7f3b3b570700 1 -- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 msgr2=0x7f3b18005b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:08.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.383+0000 7f3b3b570700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 0x7f3b18005b80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009100 tx=0x7f3b28010c50 comp rx=0 tx=0).stop 2023-10-03T10:12:08.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.383+0000 7f3b3b570700 1 -- 172.21.15.3:0/3133005781 shutdown_connections 2023-10-03T10:12:08.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.383+0000 7f3b3b570700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b18003700 0x7f3b18005b80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:08.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.383+0000 7f3b3b570700 1 --2- 172.21.15.3:0/3133005781 >> v2:172.21.15.3:3300/0 conn(0x7f3b3409e1d0 0x7f3b180084b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:08.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.383+0000 7f3b3b570700 1 --2- 172.21.15.3:0/3133005781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b34139170 0x7f3b3413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:08.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.383+0000 7f3b3b570700 1 -- 172.21.15.3:0/3133005781 >> 172.21.15.3:0/3133005781 conn(0x7f3b34066f50 msgr2=0x7f3b3409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:08.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.383+0000 7f3b3b570700 1 -- 172.21.15.3:0/3133005781 shutdown_connections 2023-10-03T10:12:08.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.384+0000 7f3b3b570700 1 -- 172.21.15.3:0/3133005781 wait complete. 2023-10-03T10:12:08.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.385+0000 7f3b3b570700 1 Processor -- start 2023-10-03T10:12:08.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.385+0000 7f3b3b570700 1 -- start start 2023-10-03T10:12:08.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.385+0000 7f3b3b570700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b18003700 0x7f3b34121400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:08.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.385+0000 7f3b3b570700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b3409d6f0 0x7f3b34121940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:08.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.385+0000 7f3b3b570700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b3409e1d0 0x7f3b34121e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:08.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.385+0000 7f3b3b570700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b34124d20 con 0x7f3b18003700 2023-10-03T10:12:08.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.385+0000 7f3b3b570700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b34123eb0 con 0x7f3b3409d6f0 2023-10-03T10:12:08.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.385+0000 7f3b3b570700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3b341242d0 con 0x7f3b3409e1d0 2023-10-03T10:12:08.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.385+0000 7f3b39b0d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b3409e1d0 0x7f3b34121e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:08.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.385+0000 7f3b38b0b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b3409d6f0 0x7f3b34121940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:08.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.385+0000 7f3b39b0d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b3409e1d0 0x7f3b34121e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:33966/0 (socket says 172.21.15.3:33966) 2023-10-03T10:12:08.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.385+0000 7f3b38b0b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b3409d6f0 0x7f3b34121940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52670/0 (socket says 172.21.15.3:52670) 2023-10-03T10:12:08.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.385+0000 7f3b39b0d700 1 -- 172.21.15.3:0/697053316 learned_addr learned my addr 172.21.15.3:0/697053316 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:08.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.385+0000 7f3b3930c700 1 --2- 172.21.15.3:0/697053316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b18003700 0x7f3b34121400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:08.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.386+0000 7f3b39b0d700 1 -- 172.21.15.3:0/697053316 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b3409e1d0 msgr2=0x7f3b34121e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2023-10-03T10:12:08.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.386+0000 7f3b39b0d700 1 -- 172.21.15.3:0/697053316 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b3409e1d0 msgr2=0x7f3b34121e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-03T10:12:08.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.386+0000 7f3b39b0d700 1 --2- 172.21.15.3:0/697053316 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b3409e1d0 0x7f3b34121e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-03T10:12:08.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.386+0000 7f3b39b0d700 1 --2- 172.21.15.3:0/697053316 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b3409e1d0 0x7f3b34121e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-03T10:12:08.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.386+0000 7f3b3930c700 1 -- 172.21.15.3:0/697053316 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b3409e1d0 msgr2=0x7f3b34121e80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:08.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.386+0000 7f3b3930c700 1 --2- 172.21.15.3:0/697053316 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b3409e1d0 0x7f3b34121e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:08.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.386+0000 7f3b3930c700 1 -- 172.21.15.3:0/697053316 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b3409d6f0 msgr2=0x7f3b34121940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:08.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.386+0000 7f3b3930c700 1 --2- 172.21.15.3:0/697053316 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b3409d6f0 0x7f3b34121940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:08.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.386+0000 7f3b3930c700 1 -- 172.21.15.3:0/697053316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3000f040 con 0x7f3b18003700 2023-10-03T10:12:08.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.386+0000 7f3b38b0b700 1 --2- 172.21.15.3:0/697053316 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b3409d6f0 0x7f3b34121940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:08.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.386+0000 7f3b3930c700 1 --2- 172.21.15.3:0/697053316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b18003700 0x7f3b34121400 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3b30001510 tx=0x7f3b30001820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:08.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.386+0000 7f3b25ffb700 1 -- 172.21.15.3:0/697053316 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b30004a80 con 0x7f3b18003700 2023-10-03T10:12:08.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.386+0000 7f3b25ffb700 1 -- 172.21.15.3:0/697053316 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b30004be0 con 0x7f3b18003700 2023-10-03T10:12:08.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.386+0000 7f3b25ffb700 1 -- 172.21.15.3:0/697053316 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b30004d70 con 0x7f3b18003700 2023-10-03T10:12:08.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.386+0000 7f3b3b570700 1 -- 172.21.15.3:0/697053316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b2800e040 con 0x7f3b18003700 2023-10-03T10:12:08.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.386+0000 7f3b3b570700 1 -- 172.21.15.3:0/697053316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b3409c030 con 0x7f3b18003700 2023-10-03T10:12:08.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.387+0000 7f3b25ffb700 1 -- 172.21.15.3:0/697053316 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f3b30005050 con 0x7f3b18003700 2023-10-03T10:12:08.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.387+0000 7f3b25ffb700 1 -- 172.21.15.3:0/697053316 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3951+0+0 (secure 0 0 0) 0x7f3b3003cbb0 con 0x7f3b18003700 2023-10-03T10:12:08.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.387+0000 7f3b38b0b700 1 -- 172.21.15.3:0/697053316 >> v1:172.21.15.3:6816/118296 conn(0x7f3b1802ca60 legacy=0x7f3b1802ef10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:08.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.387+0000 7f3b3b570700 1 -- 172.21.15.3:0/697053316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b34074ea0 con 0x7f3b18003700 2023-10-03T10:12:08.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.395+0000 7f3b25ffb700 1 -- 172.21.15.3:0/697053316 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f3b3001c030 con 0x7f3b18003700 2023-10-03T10:12:08.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.588+0000 7f3b38b0b700 1 -- 172.21.15.3:0/697053316 >> v1:172.21.15.3:6816/118296 conn(0x7f3b1802ca60 legacy=0x7f3b1802ef10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:08.618 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:08.617+0000 7f2cd33f8240 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-03T10:12:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.655+0000 7f3b3b570700 1 -- 172.21.15.3:0/697053316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f3b34124b60 con 0x7f3b18003700 2023-10-03T10:12:08.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.656+0000 7f3b25ffb700 1 -- 172.21.15.3:0/697053316 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+64278 (secure 0 0 0) 0x7f3b30005300 con 0x7f3b18003700 2023-10-03T10:12:08.658 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:12:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.659+0000 7f3b3b570700 1 -- 172.21.15.3:0/697053316 >> v1:172.21.15.3:6816/118296 conn(0x7f3b1802ca60 legacy=0x7f3b1802ef10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.659+0000 7f3b3b570700 1 -- 172.21.15.3:0/697053316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b18003700 msgr2=0x7f3b34121400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.659+0000 7f3b3b570700 1 --2- 172.21.15.3:0/697053316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b18003700 0x7f3b34121400 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3b30001510 tx=0x7f3b30001820 comp rx=0 tx=0).stop 2023-10-03T10:12:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.659+0000 7f3b3b570700 1 -- 172.21.15.3:0/697053316 shutdown_connections 2023-10-03T10:12:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.659+0000 7f3b3b570700 1 --2- 172.21.15.3:0/697053316 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b3409e1d0 0x7f3b34121e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.659+0000 7f3b3b570700 1 --2- 172.21.15.3:0/697053316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b18003700 0x7f3b34121400 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.659+0000 7f3b3b570700 1 --2- 172.21.15.3:0/697053316 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b3409d6f0 0x7f3b34121940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.659+0000 7f3b3b570700 1 -- 172.21.15.3:0/697053316 >> 172.21.15.3:0/697053316 conn(0x7f3b34066f50 msgr2=0x7f3b34070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.659+0000 7f3b3b570700 1 -- 172.21.15.3:0/697053316 shutdown_connections 2023-10-03T10:12:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.659+0000 7f3b3b570700 1 -- 172.21.15.3:0/697053316 wait complete. 2023-10-03T10:12:08.673 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":5,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6816","nonce":118296}]},"active_addr":"172.21.15.3:6816/118296","active_change":"2023-10-03T10:09:33.519586+0000","active_mgr_features":4540138314316775423,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":3919724564}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":2091814084}]}]} 2023-10-03T10:12:08.676 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-03T10:12:08.676 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-03T10:12:08.676 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-03T10:12:08.755 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:08.754+0000 7f2cd33f8240 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-03T10:12:08.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.814+0000 7fe7e0557700 1 Processor -- start 2023-10-03T10:12:08.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.814+0000 7fe7e0557700 1 -- start start 2023-10-03T10:12:08.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.815+0000 7fe7e0557700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7d80a0a30 0x7fe7d80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:08.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.815+0000 7fe7e0557700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe7d8134170 0x7fe7d813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:08.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.815+0000 7fe7e0557700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7d809edd0 con 0x7fe7d80a1340 2023-10-03T10:12:08.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.815+0000 7fe7e0557700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7d809e1f0 con 0x7fe7d8134170 2023-10-03T10:12:08.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.815+0000 7fe7e0557700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d809dc50 con 0x7fe7d80a0a30 2023-10-03T10:12:08.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.815+0000 7fe7deaf4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe7d8134170 0x7fe7d813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:08.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.815+0000 7fe7de2f3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7d80a0a30 0x7fe7d80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:08.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.815+0000 7fe7ddaf2700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe7d80a1340 0x7fe7d8133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44426/0 (socket says 172.21.15.3:44426) 2023-10-03T10:12:08.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.815+0000 7fe7deaf4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe7d8134170 0x7fe7d813bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49110/0 (socket says 172.21.15.3:49110) 2023-10-03T10:12:08.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.815+0000 7fe7ddaf2700 1 -- 172.21.15.3:0/1304537423 learned_addr learned my addr 172.21.15.3:0/1304537423 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:08.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.815+0000 7fe7de2f3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7d80a0a30 0x7fe7d80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:33974/0 (socket says 172.21.15.3:33974) 2023-10-03T10:12:08.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.815+0000 7fe7deaf4700 1 -- 172.21.15.3:0/1304537423 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7d80a0a30 msgr2=0x7fe7d80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:08.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.815+0000 7fe7deaf4700 1 --2- 172.21.15.3:0/1304537423 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7d80a0a30 0x7fe7d80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:08.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.815+0000 7fe7deaf4700 1 -- 172.21.15.3:0/1304537423 >> v1:172.21.15.3:6789/0 conn(0x7fe7d80a1340 legacy=0x7fe7d8133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:08.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.815+0000 7fe7dd2f1700 1 -- 172.21.15.3:0/1304537423 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 574923393 0 0) 0x7fe7d809edd0 con 0x7fe7d80a1340 2023-10-03T10:12:08.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.815+0000 7fe7deaf4700 1 -- 172.21.15.3:0/1304537423 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d80a1fe0 con 0x7fe7d8134170 2023-10-03T10:12:08.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.815+0000 7fe7de2f3700 1 --2- 172.21.15.3:0/1304537423 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7d80a0a30 0x7fe7d80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:08.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.815+0000 7fe7deaf4700 1 --2- 172.21.15.3:0/1304537423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7d8134170 0x7fe7d813bd30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0009840 tx=0x7fe7d0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43fdbf3a0035411b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:08.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.816+0000 7fe7dd2f1700 1 -- 172.21.15.3:0/1304537423 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0011830 con 0x7fe7d8134170 2023-10-03T10:12:08.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.816+0000 7fe7dd2f1700 1 -- 172.21.15.3:0/1304537423 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d0011990 con 0x7fe7d8134170 2023-10-03T10:12:08.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.816+0000 7fe7dd2f1700 1 -- 172.21.15.3:0/1304537423 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0011d20 con 0x7fe7d8134170 2023-10-03T10:12:08.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.816+0000 7fe7e0557700 1 -- 172.21.15.3:0/1304537423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7d8134170 msgr2=0x7fe7d813bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:08.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.816+0000 7fe7e0557700 1 --2- 172.21.15.3:0/1304537423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7d8134170 0x7fe7d813bd30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0009840 tx=0x7fe7d0009b50 comp rx=0 tx=0).stop 2023-10-03T10:12:08.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.816+0000 7fe7e0557700 1 -- 172.21.15.3:0/1304537423 shutdown_connections 2023-10-03T10:12:08.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.816+0000 7fe7e0557700 1 --2- 172.21.15.3:0/1304537423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7d8134170 0x7fe7d813bd30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:08.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.816+0000 7fe7e0557700 1 --2- 172.21.15.3:0/1304537423 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7d80a0a30 0x7fe7d80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:08.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.816+0000 7fe7e0557700 1 -- 172.21.15.3:0/1304537423 >> 172.21.15.3:0/1304537423 conn(0x7fe7d80a6550 msgr2=0x7fe7d80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:08.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.817+0000 7fe7e0557700 1 -- 172.21.15.3:0/1304537423 shutdown_connections 2023-10-03T10:12:08.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.817+0000 7fe7e0557700 1 -- 172.21.15.3:0/1304537423 wait complete. 2023-10-03T10:12:08.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.818+0000 7fe7e0557700 1 Processor -- start 2023-10-03T10:12:08.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.818+0000 7fe7e0557700 1 -- start start 2023-10-03T10:12:08.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.819+0000 7fe7e0557700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7d80a0a30 0x7fe7d8122780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:08.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.819+0000 7fe7e0557700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7d80a1340 0x7fe7d8122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:08.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.819+0000 7fe7e0557700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7d8134170 0x7fe7d8123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:08.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.819+0000 7fe7e0557700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d8125f30 con 0x7fe7d80a1340 2023-10-03T10:12:08.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.819+0000 7fe7e0557700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d80a4dc0 con 0x7fe7d8134170 2023-10-03T10:12:08.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.819+0000 7fe7e0557700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d80a0220 con 0x7fe7d80a0a30 2023-10-03T10:12:08.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.819+0000 7fe7de2f3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7d80a0a30 0x7fe7d8122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:08.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.819+0000 7fe7deaf4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7d8134170 0x7fe7d8123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:08.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.819+0000 7fe7de2f3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7d80a0a30 0x7fe7d8122780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:33986/0 (socket says 172.21.15.3:33986) 2023-10-03T10:12:08.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.819+0000 7fe7de2f3700 1 -- 172.21.15.3:0/760635954 learned_addr learned my addr 172.21.15.3:0/760635954 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:08.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.819+0000 7fe7ddaf2700 1 --2- 172.21.15.3:0/760635954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7d80a1340 0x7fe7d8122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:08.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.819+0000 7fe7deaf4700 1 -- 172.21.15.3:0/760635954 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7d80a0a30 msgr2=0x7fe7d8122780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:08.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.819+0000 7fe7deaf4700 1 --2- 172.21.15.3:0/760635954 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7d80a0a30 0x7fe7d8122780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:08.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.819+0000 7fe7deaf4700 1 -- 172.21.15.3:0/760635954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7d80a1340 msgr2=0x7fe7d8122cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:08.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.819+0000 7fe7deaf4700 1 --2- 172.21.15.3:0/760635954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7d80a1340 0x7fe7d8122cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:08.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.819+0000 7fe7deaf4700 1 -- 172.21.15.3:0/760635954 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d4009a70 con 0x7fe7d8134170 2023-10-03T10:12:08.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.819+0000 7fe7de2f3700 1 --2- 172.21.15.3:0/760635954 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7d80a0a30 0x7fe7d8122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:08.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.819+0000 7fe7ddaf2700 1 --2- 172.21.15.3:0/760635954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7d80a1340 0x7fe7d8122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:08.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.820+0000 7fe7deaf4700 1 --2- 172.21.15.3:0/760635954 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7d8134170 0x7fe7d8123200 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0016000 tx=0x7fe7d0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:08.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.820+0000 7fe7ceffd700 1 -- 172.21.15.3:0/760635954 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d00036a0 con 0x7fe7d8134170 2023-10-03T10:12:08.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.820+0000 7fe7ceffd700 1 -- 172.21.15.3:0/760635954 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d001f070 con 0x7fe7d8134170 2023-10-03T10:12:08.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.820+0000 7fe7e0557700 1 -- 172.21.15.3:0/760635954 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7d00094a0 con 0x7fe7d8134170 2023-10-03T10:12:08.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.820+0000 7fe7ceffd700 1 -- 172.21.15.3:0/760635954 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0003a40 con 0x7fe7d8134170 2023-10-03T10:12:08.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.820+0000 7fe7e0557700 1 -- 172.21.15.3:0/760635954 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7d81269a0 con 0x7fe7d8134170 2023-10-03T10:12:08.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.820+0000 7fe7ceffd700 1 -- 172.21.15.3:0/760635954 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fe7d0003ba0 con 0x7fe7d8134170 2023-10-03T10:12:08.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.821+0000 7fe7ceffd700 1 -- 172.21.15.3:0/760635954 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3951+0+0 (secure 0 0 0) 0x7fe7d0021070 con 0x7fe7d8134170 2023-10-03T10:12:08.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.821+0000 7fe7de2f3700 1 -- 172.21.15.3:0/760635954 >> v1:172.21.15.3:6816/118296 conn(0x7fe7bc02c1b0 legacy=0x7fe7bc02e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:08.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.821+0000 7fe7e0557700 1 -- 172.21.15.3:0/760635954 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7d8125bd0 con 0x7fe7d8134170 2023-10-03T10:12:08.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:08.825+0000 7fe7ceffd700 1 -- 172.21.15.3:0/760635954 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fe7d002b070 con 0x7fe7d8134170 2023-10-03T10:12:09.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.009+0000 7fe7e0557700 1 -- 172.21.15.3:0/760635954 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7d8074ea0 con 0x7fe7d8134170 2023-10-03T10:12:09.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.010+0000 7fe7ceffd700 1 -- 172.21.15.3:0/760635954 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+12750 (secure 0 0 0) 0x7fe7d002b030 con 0x7fe7d8134170 2023-10-03T10:12:09.010 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:12:09.010 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","created":"2023-10-03T10:09:28.703441+0000","modified":"2023-10-03T10:10:35.504810+0000","last_up_change":"2023-10-03T10:09:36.569401+0000","last_in_change":"2023-10-03T10:09:30.156546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-10-03T10:09:36.623971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-03T10:09:38.210614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-03T10:09:42.695873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-03T10:09:43.115588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cdcdd930-3c5b-4486-a021-7132f90fb6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6808","nonce":118845}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6809","nonce":118845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6811","nonce":118845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6810","nonce":118845}]},"public_addr":"172.21.15.3:6808/118845","cluster_addr":"172.21.15.3:6809/118845","heartbeat_back_addr":"172.21.15.3:6811/118845","heartbeat_front_addr":"172.21.15.3:6810/118845","state":["exists","up"]},{"osd":1,"uuid":"c8e58877-6762-4741-a8c1-567ebc97b708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6804","nonce":118835}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6805","nonce":118835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6807","nonce":118835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6806","nonce":118835}]},"public_addr":"172.21.15.3:6804/118835","cluster_addr":"172.21.15.3:6805/118835","heartbeat_back_addr":"172.21.15.3:6807/118835","heartbeat_front_addr":"172.21.15.3:6806/118835","state":["exists","up"]},{"osd":2,"uuid":"68a2a21a-7a8e-4f82-8433-5201faa50777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6812","nonce":118864}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6813","nonce":118864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6815","nonce":118864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6814","nonce":118864}]},"public_addr":"172.21.15.3:6812/118864","cluster_addr":"172.21.15.3:6813/118864","heartbeat_back_addr":"172.21.15.3:6815/118864","heartbeat_front_addr":"172.21.15.3:6814/118864","state":["exists","up"]},{"osd":3,"uuid":"c4df2903-4e1e-4208-bf77-95b9fa749241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6800","nonce":118836}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6801","nonce":118836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6803","nonce":118836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6802","nonce":118836}]},"public_addr":"172.21.15.3:6800/118836","cluster_addr":"172.21.15.3:6801/118836","heartbeat_back_addr":"172.21.15.3:6803/118836","heartbeat_front_addr":"172.21.15.3:6802/118836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.369708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.257825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.337931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.085222+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1","mappings":[{"from":3,"to":0}]},{"pgid":"4.8","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.19","mappings":[{"from":3,"to":0}]},{"pgid":"4.2a","mappings":[{"from":1,"to":2}]},{"pgid":"4.2d","mappings":[{"from":3,"to":0}]},{"pgid":"4.33","mappings":[{"from":3,"to":0}]},{"pgid":"4.39","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-03T10:12:09.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.012+0000 7fe7e0557700 1 -- 172.21.15.3:0/760635954 >> v1:172.21.15.3:6816/118296 conn(0x7fe7bc02c1b0 legacy=0x7fe7bc02e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:09.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.012+0000 7fe7e0557700 1 -- 172.21.15.3:0/760635954 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7d8134170 msgr2=0x7fe7d8123200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.012+0000 7fe7e0557700 1 --2- 172.21.15.3:0/760635954 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7d8134170 0x7fe7d8123200 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0016000 tx=0x7fe7d0004750 comp rx=0 tx=0).stop 2023-10-03T10:12:09.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.012+0000 7fe7e0557700 1 -- 172.21.15.3:0/760635954 shutdown_connections 2023-10-03T10:12:09.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.012+0000 7fe7e0557700 1 --2- 172.21.15.3:0/760635954 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7d8134170 0x7fe7d8123200 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.012+0000 7fe7e0557700 1 --2- 172.21.15.3:0/760635954 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7d80a0a30 0x7fe7d8122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.012+0000 7fe7e0557700 1 --2- 172.21.15.3:0/760635954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7d80a1340 0x7fe7d8122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.012+0000 7fe7e0557700 1 -- 172.21.15.3:0/760635954 >> 172.21.15.3:0/760635954 conn(0x7fe7d80a6550 msgr2=0x7fe7d809edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:09.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.012+0000 7fe7e0557700 1 -- 172.21.15.3:0/760635954 shutdown_connections 2023-10-03T10:12:09.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.012+0000 7fe7e0557700 1 -- 172.21.15.3:0/760635954 wait complete. 2023-10-03T10:12:09.026 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-03T10:12:09.027 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-03T10:12:09.126 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:09.126+0000 7f2cd33f8240 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-03T10:12:09.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.163+0000 7f8e0aaf1700 1 Processor -- start 2023-10-03T10:12:09.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.163+0000 7f8e0aaf1700 1 -- start start 2023-10-03T10:12:09.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.163+0000 7f8e0aaf1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8e0409e930 0x7f8e0409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.163+0000 7f8e0aaf1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8e0409f240 0x7f8e041317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.163+0000 7f8e0aaf1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e0409f650 0x7f8e04131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.163+0000 7f8e0aaf1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e04070940 con 0x7f8e0409f240 2023-10-03T10:12:09.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.163+0000 7f8e0aaf1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0406fd60 con 0x7f8e0409f650 2023-10-03T10:12:09.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.163+0000 7f8e0aaf1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0406fa80 con 0x7f8e0409e930 2023-10-03T10:12:09.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.163+0000 7f8e03fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8e0409f240 0x7f8e041317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.163+0000 7f8e0908e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e0409f650 0x7f8e04131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.163+0000 7f8e03fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8e0409f240 0x7f8e041317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49130/0 (socket says 172.21.15.3:49130) 2023-10-03T10:12:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.163+0000 7f8e03fff700 1 -- 172.21.15.3:0/1274813449 learned_addr learned my addr 172.21.15.3:0/1274813449 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.164+0000 7f8e0888d700 1 --2- 172.21.15.3:0/1274813449 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8e0409e930 0x7f8e0409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.164+0000 7f8e0888d700 1 -- 172.21.15.3:0/1274813449 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e0409f650 msgr2=0x7f8e04131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.164+0000 7f8e0888d700 1 --2- 172.21.15.3:0/1274813449 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e0409f650 0x7f8e04131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.164+0000 7f8e0888d700 1 -- 172.21.15.3:0/1274813449 >> v2:172.21.15.3:3300/0 conn(0x7f8e0409f240 msgr2=0x7f8e041317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.164+0000 7f8e0888d700 1 --2- 172.21.15.3:0/1274813449 >> v2:172.21.15.3:3300/0 conn(0x7f8e0409f240 0x7f8e041317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.164+0000 7f8e0888d700 1 -- 172.21.15.3:0/1274813449 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0409fb60 con 0x7f8e0409e930 2023-10-03T10:12:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.164+0000 7f8e0908e700 1 --2- 172.21.15.3:0/1274813449 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e0409f650 0x7f8e04131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.164+0000 7f8e03fff700 1 --2- 172.21.15.3:0/1274813449 >> v2:172.21.15.3:3300/0 conn(0x7f8e0409f240 0x7f8e041317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.164+0000 7f8e0888d700 1 --2- 172.21.15.3:0/1274813449 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8e0409e930 0x7f8e0409ed00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8df4009840 tx=0x7f8df4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab7c6f853f0c0889 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:09.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.164+0000 7f8e037fe700 1 -- 172.21.15.3:0/1274813449 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df4011830 con 0x7f8e0409e930 2023-10-03T10:12:09.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.164+0000 7f8e037fe700 1 -- 172.21.15.3:0/1274813449 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df4011990 con 0x7f8e0409e930 2023-10-03T10:12:09.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.164+0000 7f8e037fe700 1 -- 172.21.15.3:0/1274813449 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df4011d20 con 0x7f8e0409e930 2023-10-03T10:12:09.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.165+0000 7f8e0aaf1700 1 -- 172.21.15.3:0/1274813449 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8e0409e930 msgr2=0x7f8e0409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.165+0000 7f8e0aaf1700 1 --2- 172.21.15.3:0/1274813449 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8e0409e930 0x7f8e0409ed00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8df4009840 tx=0x7f8df4009b50 comp rx=0 tx=0).stop 2023-10-03T10:12:09.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.165+0000 7f8e0aaf1700 1 -- 172.21.15.3:0/1274813449 shutdown_connections 2023-10-03T10:12:09.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.165+0000 7f8e0aaf1700 1 --2- 172.21.15.3:0/1274813449 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e0409f650 0x7f8e04131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.165+0000 7f8e0aaf1700 1 --2- 172.21.15.3:0/1274813449 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8e0409e930 0x7f8e0409ed00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.165+0000 7f8e0aaf1700 1 --2- 172.21.15.3:0/1274813449 >> v2:172.21.15.3:3300/0 conn(0x7f8e0409f240 0x7f8e041317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.165+0000 7f8e0aaf1700 1 -- 172.21.15.3:0/1274813449 >> 172.21.15.3:0/1274813449 conn(0x7f8e040a4210 msgr2=0x7f8e040a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:09.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.165+0000 7f8e0aaf1700 1 -- 172.21.15.3:0/1274813449 shutdown_connections 2023-10-03T10:12:09.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.165+0000 7f8e0aaf1700 1 -- 172.21.15.3:0/1274813449 wait complete. 2023-10-03T10:12:09.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.166+0000 7f8e0aaf1700 1 Processor -- start 2023-10-03T10:12:09.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.166+0000 7f8e0aaf1700 1 -- start start 2023-10-03T10:12:09.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.167+0000 7f8e0aaf1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e0409e930 0x7f8e041a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.167+0000 7f8e0aaf1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8e0409f240 0x7f8e041a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.167+0000 7f8e0aaf1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e0409f650 0x7f8e041a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.167+0000 7f8e0aaf1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e041a5270 con 0x7f8e0409e930 2023-10-03T10:12:09.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.167+0000 7f8e0aaf1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e041a5ea0 con 0x7f8e0409f650 2023-10-03T10:12:09.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.167+0000 7f8e0aaf1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e040a3630 con 0x7f8e0409f240 2023-10-03T10:12:09.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.167+0000 7f8e0888d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e0409e930 0x7f8e041a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.167+0000 7f8e0908e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e0409f650 0x7f8e041a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.167+0000 7f8e0888d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e0409e930 0x7f8e041a3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49140/0 (socket says 172.21.15.3:49140) 2023-10-03T10:12:09.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.167+0000 7f8e0888d700 1 -- 172.21.15.3:0/149805589 learned_addr learned my addr 172.21.15.3:0/149805589 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:09.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.167+0000 7f8e03fff700 1 --2- 172.21.15.3:0/149805589 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8e0409f240 0x7f8e041a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.167+0000 7f8e0888d700 1 -- 172.21.15.3:0/149805589 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8e0409f240 msgr2=0x7f8e041a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.167+0000 7f8e0888d700 1 --2- 172.21.15.3:0/149805589 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8e0409f240 0x7f8e041a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.167+0000 7f8e0888d700 1 -- 172.21.15.3:0/149805589 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e0409f650 msgr2=0x7f8e041a4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.167+0000 7f8e0888d700 1 --2- 172.21.15.3:0/149805589 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e0409f650 0x7f8e041a4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.167+0000 7f8e0888d700 1 -- 172.21.15.3:0/149805589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df8009550 con 0x7f8e0409e930 2023-10-03T10:12:09.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.168+0000 7f8e0888d700 1 --2- 172.21.15.3:0/149805589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e0409e930 0x7f8e041a3de0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8df40090b0 tx=0x7f8df4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:09.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.168+0000 7f8e017fa700 1 -- 172.21.15.3:0/149805589 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df4004510 con 0x7f8e0409e930 2023-10-03T10:12:09.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.168+0000 7f8e017fa700 1 -- 172.21.15.3:0/149805589 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df401f070 con 0x7f8e0409e930 2023-10-03T10:12:09.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.168+0000 7f8e0aaf1700 1 -- 172.21.15.3:0/149805589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dfc009a70 con 0x7f8e0409e930 2023-10-03T10:12:09.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.168+0000 7f8e017fa700 1 -- 172.21.15.3:0/149805589 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df40038a0 con 0x7f8e0409e930 2023-10-03T10:12:09.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.168+0000 7f8e0aaf1700 1 -- 172.21.15.3:0/149805589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df40094a0 con 0x7f8e0409e930 2023-10-03T10:12:09.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.168+0000 7f8e017fa700 1 -- 172.21.15.3:0/149805589 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f8df4003a00 con 0x7f8e0409e930 2023-10-03T10:12:09.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.169+0000 7f8e017fa700 1 -- 172.21.15.3:0/149805589 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3951+0+0 (secure 0 0 0) 0x7f8df4021070 con 0x7f8e0409e930 2023-10-03T10:12:09.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.169+0000 7f8e03fff700 1 -- 172.21.15.3:0/149805589 >> v1:172.21.15.3:6816/118296 conn(0x7f8df002c1b0 legacy=0x7f8df002e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:09.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.169+0000 7f8e0aaf1700 1 -- 172.21.15.3:0/149805589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e040a3e20 con 0x7f8e0409e930 2023-10-03T10:12:09.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.174+0000 7f8e017fa700 1 -- 172.21.15.3:0/149805589 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f8df4015020 con 0x7f8e0409e930 2023-10-03T10:12:09.288 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:09.288+0000 7f2cd33f8240 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-03T10:12:09.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.356+0000 7f8e0aaf1700 1 -- 172.21.15.3:0/149805589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e04074ea0 con 0x7f8e0409e930 2023-10-03T10:12:09.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.357+0000 7f8e017fa700 1 -- 172.21.15.3:0/149805589 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+12750 (secure 0 0 0) 0x7f8df4024380 con 0x7f8e0409e930 2023-10-03T10:12:09.358 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:12:09.358 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","created":"2023-10-03T10:09:28.703441+0000","modified":"2023-10-03T10:10:35.504810+0000","last_up_change":"2023-10-03T10:09:36.569401+0000","last_in_change":"2023-10-03T10:09:30.156546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-10-03T10:09:36.623971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-03T10:09:38.210614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-03T10:09:42.695873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-03T10:09:43.115588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cdcdd930-3c5b-4486-a021-7132f90fb6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6808","nonce":118845}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6809","nonce":118845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6811","nonce":118845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6810","nonce":118845}]},"public_addr":"172.21.15.3:6808/118845","cluster_addr":"172.21.15.3:6809/118845","heartbeat_back_addr":"172.21.15.3:6811/118845","heartbeat_front_addr":"172.21.15.3:6810/118845","state":["exists","up"]},{"osd":1,"uuid":"c8e58877-6762-4741-a8c1-567ebc97b708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6804","nonce":118835}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6805","nonce":118835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6807","nonce":118835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6806","nonce":118835}]},"public_addr":"172.21.15.3:6804/118835","cluster_addr":"172.21.15.3:6805/118835","heartbeat_back_addr":"172.21.15.3:6807/118835","heartbeat_front_addr":"172.21.15.3:6806/118835","state":["exists","up"]},{"osd":2,"uuid":"68a2a21a-7a8e-4f82-8433-5201faa50777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6812","nonce":118864}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6813","nonce":118864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6815","nonce":118864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6814","nonce":118864}]},"public_addr":"172.21.15.3:6812/118864","cluster_addr":"172.21.15.3:6813/118864","heartbeat_back_addr":"172.21.15.3:6815/118864","heartbeat_front_addr":"172.21.15.3:6814/118864","state":["exists","up"]},{"osd":3,"uuid":"c4df2903-4e1e-4208-bf77-95b9fa749241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6800","nonce":118836}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6801","nonce":118836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6803","nonce":118836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6802","nonce":118836}]},"public_addr":"172.21.15.3:6800/118836","cluster_addr":"172.21.15.3:6801/118836","heartbeat_back_addr":"172.21.15.3:6803/118836","heartbeat_front_addr":"172.21.15.3:6802/118836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.369708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.257825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.337931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.085222+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1","mappings":[{"from":3,"to":0}]},{"pgid":"4.8","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.19","mappings":[{"from":3,"to":0}]},{"pgid":"4.2a","mappings":[{"from":1,"to":2}]},{"pgid":"4.2d","mappings":[{"from":3,"to":0}]},{"pgid":"4.33","mappings":[{"from":3,"to":0}]},{"pgid":"4.39","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-03T10:12:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.359+0000 7f8e0aaf1700 1 -- 172.21.15.3:0/149805589 >> v1:172.21.15.3:6816/118296 conn(0x7f8df002c1b0 legacy=0x7f8df002e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.359+0000 7f8e0aaf1700 1 -- 172.21.15.3:0/149805589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e0409e930 msgr2=0x7f8e041a3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.359+0000 7f8e0aaf1700 1 --2- 172.21.15.3:0/149805589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e0409e930 0x7f8e041a3de0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8df40090b0 tx=0x7f8df4004750 comp rx=0 tx=0).stop 2023-10-03T10:12:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.359+0000 7f8e0aaf1700 1 -- 172.21.15.3:0/149805589 shutdown_connections 2023-10-03T10:12:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.360+0000 7f8e0aaf1700 1 --2- 172.21.15.3:0/149805589 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e0409f650 0x7f8e041a4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.360+0000 7f8e0aaf1700 1 --2- 172.21.15.3:0/149805589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e0409e930 0x7f8e041a3de0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.360+0000 7f8e0aaf1700 1 --2- 172.21.15.3:0/149805589 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8e0409f240 0x7f8e041a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.360+0000 7f8e0aaf1700 1 -- 172.21.15.3:0/149805589 >> 172.21.15.3:0/149805589 conn(0x7f8e040a4210 msgr2=0x7f8e041b8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:09.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.360+0000 7f8e0aaf1700 1 -- 172.21.15.3:0/149805589 shutdown_connections 2023-10-03T10:12:09.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.360+0000 7f8e0aaf1700 1 -- 172.21.15.3:0/149805589 wait complete. 2023-10-03T10:12:09.372 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:09.372+0000 7f2cd33f8240 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-03T10:12:09.374 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-03T10:12:09.374 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-03T10:12:09.375 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-03T10:12:09.375 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-03T10:12:09.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.529+0000 7f74573fb700 1 Processor -- start 2023-10-03T10:12:09.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.529+0000 7f74573fb700 1 -- start start 2023-10-03T10:12:09.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.530+0000 7f74573fb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f745006c0e0 0x7f745009ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.530+0000 7f74573fb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7450134ec0 0x7f74501379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.530+0000 7f74573fb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7450137f20 0x7f74501383c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.530+0000 7f74573fb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f745006c4b0 con 0x7f7450134ec0 2023-10-03T10:12:09.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.530+0000 7f74573fb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74500a09c0 con 0x7f7450137f20 2023-10-03T10:12:09.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.530+0000 7f74573fb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7450134980 con 0x7f745006c0e0 2023-10-03T10:12:09.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.530+0000 7f7455998700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7450137f20 0x7f74501383c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.531+0000 7f7455998700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7450137f20 0x7f74501383c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52700/0 (socket says 172.21.15.3:52700) 2023-10-03T10:12:09.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.531+0000 7f7455998700 1 -- 172.21.15.3:0/1092486747 learned_addr learned my addr 172.21.15.3:0/1092486747 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:09.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.531+0000 7f7454996700 1 --2- 172.21.15.3:0/1092486747 >> v2:172.21.15.3:3300/0 conn(0x7f7450134ec0 0x7f74501379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.531+0000 7f7455197700 1 --2- 172.21.15.3:0/1092486747 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f745006c0e0 0x7f745009ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.531+0000 7f7454996700 1 -- 172.21.15.3:0/1092486747 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f745006c0e0 msgr2=0x7f745009ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:09.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.531+0000 7f7454996700 1 --2- 172.21.15.3:0/1092486747 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f745006c0e0 0x7f745009ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.531+0000 7f7454996700 1 -- 172.21.15.3:0/1092486747 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7450137f20 msgr2=0x7f74501383c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:09.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.531+0000 7f7454996700 1 --2- 172.21.15.3:0/1092486747 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7450137f20 0x7f74501383c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.531+0000 7f7454996700 1 -- 172.21.15.3:0/1092486747 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74500a0600 con 0x7f7450134ec0 2023-10-03T10:12:09.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.531+0000 7f7455998700 1 --2- 172.21.15.3:0/1092486747 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7450137f20 0x7f74501383c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:09.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.531+0000 7f7454996700 1 --2- 172.21.15.3:0/1092486747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7450134ec0 0x7f74501379e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7440009840 tx=0x7f7440009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9c8906935ea27d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:09.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.532+0000 7f7447fff700 1 -- 172.21.15.3:0/1092486747 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7440011830 con 0x7f7450134ec0 2023-10-03T10:12:09.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.532+0000 7f7447fff700 1 -- 172.21.15.3:0/1092486747 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7440011990 con 0x7f7450134ec0 2023-10-03T10:12:09.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.532+0000 7f7447fff700 1 -- 172.21.15.3:0/1092486747 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7440011d20 con 0x7f7450134ec0 2023-10-03T10:12:09.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.532+0000 7f74573fb700 1 -- 172.21.15.3:0/1092486747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7450134ec0 msgr2=0x7f74501379e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.532+0000 7f74573fb700 1 --2- 172.21.15.3:0/1092486747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7450134ec0 0x7f74501379e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7440009840 tx=0x7f7440009b50 comp rx=0 tx=0).stop 2023-10-03T10:12:09.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.532+0000 7f74573fb700 1 -- 172.21.15.3:0/1092486747 shutdown_connections 2023-10-03T10:12:09.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.532+0000 7f74573fb700 1 --2- 172.21.15.3:0/1092486747 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7450137f20 0x7f74501383c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.532+0000 7f74573fb700 1 --2- 172.21.15.3:0/1092486747 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f745006c0e0 0x7f745009ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.532+0000 7f74573fb700 1 --2- 172.21.15.3:0/1092486747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7450134ec0 0x7f74501379e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.532+0000 7f74573fb700 1 -- 172.21.15.3:0/1092486747 >> 172.21.15.3:0/1092486747 conn(0x7f74500a4210 msgr2=0x7f74500a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:09.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.533+0000 7f74573fb700 1 -- 172.21.15.3:0/1092486747 shutdown_connections 2023-10-03T10:12:09.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.533+0000 7f74573fb700 1 -- 172.21.15.3:0/1092486747 wait complete. 2023-10-03T10:12:09.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.534+0000 7f74573fb700 1 Processor -- start 2023-10-03T10:12:09.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.534+0000 7f74573fb700 1 -- start start 2023-10-03T10:12:09.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.535+0000 7f74573fb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f745006c0e0 0x7f7450133df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.535+0000 7f74573fb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7450134ec0 0x7f745011aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.535+0000 7f74573fb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7450137f20 0x7f745011b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.535+0000 7f74573fb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745011c970 con 0x7f7450137f20 2023-10-03T10:12:09.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.535+0000 7f74573fb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74500a03f0 con 0x7f745006c0e0 2023-10-03T10:12:09.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.535+0000 7f74573fb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f745011c450 con 0x7f7450134ec0 2023-10-03T10:12:09.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.535+0000 7f7455998700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7450137f20 0x7f745011b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.535+0000 7f7455998700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7450137f20 0x7f745011b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49156/0 (socket says 172.21.15.3:49156) 2023-10-03T10:12:09.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.535+0000 7f7455998700 1 -- 172.21.15.3:0/271997962 learned_addr learned my addr 172.21.15.3:0/271997962 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:09.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.535+0000 7f7455197700 1 --2- 172.21.15.3:0/271997962 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f745006c0e0 0x7f7450133df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.535+0000 7f7454996700 1 --2- 172.21.15.3:0/271997962 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7450134ec0 0x7f745011aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.536+0000 7f7455197700 1 -- 172.21.15.3:0/271997962 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7450134ec0 msgr2=0x7f745011aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.536+0000 7f7455197700 1 --2- 172.21.15.3:0/271997962 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7450134ec0 0x7f745011aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.536+0000 7f7455197700 1 -- 172.21.15.3:0/271997962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7450137f20 msgr2=0x7f745011b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.536+0000 7f7455197700 1 --2- 172.21.15.3:0/271997962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7450137f20 0x7f745011b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.536+0000 7f7455197700 1 -- 172.21.15.3:0/271997962 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7448009b30 con 0x7f745006c0e0 2023-10-03T10:12:09.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.536+0000 7f7455197700 1 --2- 172.21.15.3:0/271997962 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f745006c0e0 0x7f7450133df0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f744c00d5c0 tx=0x7f744c00d980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:09.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.536+0000 7f7445ffb700 1 -- 172.21.15.3:0/271997962 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744c0125f0 con 0x7f745006c0e0 2023-10-03T10:12:09.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.536+0000 7f7445ffb700 1 -- 172.21.15.3:0/271997962 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744c012750 con 0x7f745006c0e0 2023-10-03T10:12:09.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.536+0000 7f74573fb700 1 -- 172.21.15.3:0/271997962 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74400094a0 con 0x7f745006c0e0 2023-10-03T10:12:09.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.536+0000 7f7445ffb700 1 -- 172.21.15.3:0/271997962 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744c012ae0 con 0x7f745006c0e0 2023-10-03T10:12:09.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.537+0000 7f74573fb700 1 -- 172.21.15.3:0/271997962 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f745011e330 con 0x7f745006c0e0 2023-10-03T10:12:09.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.538+0000 7f74573fb700 1 -- 172.21.15.3:0/271997962 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7450134bb0 con 0x7f745006c0e0 2023-10-03T10:12:09.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.538+0000 7f7445ffb700 1 -- 172.21.15.3:0/271997962 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f744c012c40 con 0x7f745006c0e0 2023-10-03T10:12:09.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.539+0000 7f7445ffb700 1 -- 172.21.15.3:0/271997962 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3951+0+0 (secure 0 0 0) 0x7f744c03c2c0 con 0x7f745006c0e0 2023-10-03T10:12:09.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.539+0000 7f7454996700 1 -- 172.21.15.3:0/271997962 >> v1:172.21.15.3:6816/118296 conn(0x7f743402c1b0 legacy=0x7f743402e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:09.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.539+0000 7f7445ffb700 1 -- 172.21.15.3:0/271997962 --> v1:172.21.15.3:6804/118835 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7434038f10 con 0x7f7434035210 2023-10-03T10:12:09.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.539+0000 7f7445ffb700 1 -- 172.21.15.3:0/271997962 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f744c03c640 con 0x7f745006c0e0 2023-10-03T10:12:09.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.540+0000 7f7445ffb700 1 -- 172.21.15.3:0/271997962 <== osd.1 v1:172.21.15.3:6804/118835 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7f7434038f10 con 0x7f7434035210 2023-10-03T10:12:09.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.555+0000 7ff989deb700 1 Processor -- start 2023-10-03T10:12:09.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.555+0000 7ff989deb700 1 -- start start 2023-10-03T10:12:09.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.556+0000 7ff989deb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff98406a770 0x7ff98409d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.556+0000 7ff989deb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff98409dc30 0x7ff98409e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.556+0000 7ff989deb700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff98409a810 con 0x7ff98413c1e0 2023-10-03T10:12:09.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.556+0000 7ff989deb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff98406ab40 con 0x7ff98409dc30 2023-10-03T10:12:09.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.556+0000 7ff989deb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98409ebc0 con 0x7ff98406a770 2023-10-03T10:12:09.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.556+0000 7ff983fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff98413c1e0 0x7ff98413e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44432/0 (socket says 172.21.15.3:44432) 2023-10-03T10:12:09.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.556+0000 7ff983fff700 1 -- 172.21.15.3:0/1154658294 learned_addr learned my addr 172.21.15.3:0/1154658294 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:09.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.556+0000 7ff9837fe700 1 --2- 172.21.15.3:0/1154658294 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff98406a770 0x7ff98409d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.556+0000 7f74573fb700 1 -- 172.21.15.3:0/271997962 --> v1:172.21.15.3:6804/118835 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7450134570 con 0x7f7434035210 2023-10-03T10:12:09.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.556+0000 7ff9837fe700 1 -- 172.21.15.3:0/1154658294 >> v2:172.21.15.3:3300/0 conn(0x7ff98409dc30 msgr2=0x7ff98409e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:09.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.556+0000 7ff9837fe700 1 --2- 172.21.15.3:0/1154658294 >> v2:172.21.15.3:3300/0 conn(0x7ff98409dc30 0x7ff98409e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.556+0000 7ff9837fe700 1 -- 172.21.15.3:0/1154658294 >> v1:172.21.15.3:6789/0 conn(0x7ff98413c1e0 legacy=0x7ff98413e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.556+0000 7ff9837fe700 1 -- 172.21.15.3:0/1154658294 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98409e920 con 0x7ff98406a770 2023-10-03T10:12:09.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.557+0000 7ff9837fe700 1 --2- 172.21.15.3:0/1154658294 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff98406a770 0x7ff98409d6f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff974009870 tx=0x7ff974009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c00087b7c59f429 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:09.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.557+0000 7f7445ffb700 1 -- 172.21.15.3:0/271997962 <== osd.1 v1:172.21.15.3:6804/118835 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 2434951863) 0x7f7450134570 con 0x7f7434035210 2023-10-03T10:12:09.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.557+0000 7fac74812700 1 Processor -- start 2023-10-03T10:12:09.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.557+0000 7fac74812700 1 -- start start 2023-10-03T10:12:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.557+0000 7ff9827fc700 1 -- 172.21.15.3:0/1154658294 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff974011830 con 0x7ff98406a770 2023-10-03T10:12:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.557+0000 7ff9827fc700 1 -- 172.21.15.3:0/1154658294 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff974011990 con 0x7ff98406a770 2023-10-03T10:12:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.557+0000 7f74573fb700 1 -- 172.21.15.3:0/271997962 >> v1:172.21.15.3:6804/118835 conn(0x7f7434035210 legacy=0x7f74340376a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.558+0000 7f74573fb700 1 -- 172.21.15.3:0/271997962 >> v1:172.21.15.3:6816/118296 conn(0x7f743402c1b0 legacy=0x7f743402e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.558+0000 7f74573fb700 1 -- 172.21.15.3:0/271997962 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f745006c0e0 msgr2=0x7f7450133df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.558+0000 7f74573fb700 1 --2- 172.21.15.3:0/271997962 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f745006c0e0 0x7f7450133df0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f744c00d5c0 tx=0x7f744c00d980 comp rx=0 tx=0).stop 2023-10-03T10:12:09.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.558+0000 7f7455998700 1 -- 172.21.15.3:0/271997962 reap_dead start 2023-10-03T10:12:09.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.558+0000 7fac74812700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fac70130b30 0x7fac70138890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.558+0000 7fac74812700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fac70138dd0 0x7fac7013b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.558+0000 7fac74812700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac70099ef0 con 0x7fac70071b40 2023-10-03T10:12:09.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.558+0000 7fac74812700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac70099500 con 0x7fac70138dd0 2023-10-03T10:12:09.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.558+0000 7fac74812700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fac70099c10 con 0x7fac70130b30 2023-10-03T10:12:09.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.558+0000 7f74573fb700 1 -- 172.21.15.3:0/271997962 shutdown_connections 2023-10-03T10:12:09.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.558+0000 7f74573fb700 1 -- 172.21.15.3:0/271997962 >> 172.21.15.3:0/271997962 conn(0x7f74500a4210 msgr2=0x7f7450099c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:09.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.558+0000 7fac6e59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fac70071b40 0x7fac70130420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44446/0 (socket says 172.21.15.3:44446) 2023-10-03T10:12:09.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.558+0000 7fac6dd9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fac70130b30 0x7fac70138890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.558+0000 7fac6e59c700 1 -- 172.21.15.3:0/652761558 learned_addr learned my addr 172.21.15.3:0/652761558 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.558+0000 7f74573fb700 1 -- 172.21.15.3:0/271997962 shutdown_connections 2023-10-03T10:12:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.558+0000 7f74573fb700 1 -- 172.21.15.3:0/271997962 wait complete. 2023-10-03T10:12:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.557+0000 7ff9827fc700 1 -- 172.21.15.3:0/1154658294 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff974011d20 con 0x7ff98406a770 2023-10-03T10:12:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.558+0000 7fac6d59a700 1 -- 172.21.15.3:0/652761558 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2241300572 0 0) 0x7fac70099ef0 con 0x7fac70071b40 2023-10-03T10:12:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.558+0000 7fac6dd9b700 1 -- 172.21.15.3:0/652761558 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fac70138dd0 msgr2=0x7fac7013b1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-03T10:12:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.558+0000 7fac6dd9b700 1 --2- 172.21.15.3:0/652761558 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fac70138dd0 0x7fac7013b1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.558+0000 7fac6dd9b700 1 -- 172.21.15.3:0/652761558 >> v1:172.21.15.3:6789/0 conn(0x7fac70071b40 legacy=0x7fac70130420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.559+0000 7fac6dd9b700 1 -- 172.21.15.3:0/652761558 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac70130f20 con 0x7fac70130b30 2023-10-03T10:12:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.558+0000 7ff989deb700 1 -- 172.21.15.3:0/1154658294 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff98406a770 msgr2=0x7ff98409d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.559+0000 7ff989deb700 1 --2- 172.21.15.3:0/1154658294 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff98406a770 0x7ff98409d6f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff974009870 tx=0x7ff974009b80 comp rx=0 tx=0).stop 2023-10-03T10:12:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.559+0000 7ff989deb700 1 -- 172.21.15.3:0/1154658294 shutdown_connections 2023-10-03T10:12:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.559+0000 7ff989deb700 1 --2- 172.21.15.3:0/1154658294 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff98406a770 0x7ff98409d6f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.559+0000 7ff989deb700 1 --2- 172.21.15.3:0/1154658294 >> v2:172.21.15.3:3300/0 conn(0x7ff98409dc30 0x7ff98409e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.559+0000 7ff989deb700 1 -- 172.21.15.3:0/1154658294 >> 172.21.15.3:0/1154658294 conn(0x7ff984066f50 msgr2=0x7ff98409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:09.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.559+0000 7fac6dd9b700 1 --2- 172.21.15.3:0/652761558 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fac70130b30 0x7fac70138890 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fac60009870 tx=0x7fac60009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5ac7d4547fa94ba server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:09.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.559+0000 7ff989deb700 1 -- 172.21.15.3:0/1154658294 shutdown_connections 2023-10-03T10:12:09.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.559+0000 7ff989deb700 1 -- 172.21.15.3:0/1154658294 wait complete. 2023-10-03T10:12:09.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.559+0000 7fac6d59a700 1 -- 172.21.15.3:0/652761558 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac60011830 con 0x7fac70130b30 2023-10-03T10:12:09.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.559+0000 7fac6d59a700 1 -- 172.21.15.3:0/652761558 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac60011990 con 0x7fac70130b30 2023-10-03T10:12:09.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.559+0000 7fac6d59a700 1 -- 172.21.15.3:0/652761558 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac60011d20 con 0x7fac70130b30 2023-10-03T10:12:09.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.560+0000 7fac74812700 1 -- 172.21.15.3:0/652761558 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fac70130b30 msgr2=0x7fac70138890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.560+0000 7fac74812700 1 --2- 172.21.15.3:0/652761558 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fac70130b30 0x7fac70138890 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fac60009870 tx=0x7fac60009b80 comp rx=0 tx=0).stop 2023-10-03T10:12:09.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.560+0000 7ff989deb700 1 Processor -- start 2023-10-03T10:12:09.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.560+0000 7fac74812700 1 -- 172.21.15.3:0/652761558 shutdown_connections 2023-10-03T10:12:09.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.560+0000 7fac74812700 1 --2- 172.21.15.3:0/652761558 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fac70138dd0 0x7fac7013b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.560+0000 7fac74812700 1 --2- 172.21.15.3:0/652761558 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fac70130b30 0x7fac70138890 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.560+0000 7fac74812700 1 -- 172.21.15.3:0/652761558 >> 172.21.15.3:0/652761558 conn(0x7fac7009e7e0 msgr2=0x7fac7009ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:09.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.560+0000 7fac74812700 1 -- 172.21.15.3:0/652761558 shutdown_connections 2023-10-03T10:12:09.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.560+0000 7fac74812700 1 -- 172.21.15.3:0/652761558 wait complete. 2023-10-03T10:12:09.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.560+0000 7ff989deb700 1 -- start start 2023-10-03T10:12:09.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.561+0000 7ff989deb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff98406a770 0x7ff9841cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.561+0000 7ff989deb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff98409dc30 0x7ff9841ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.561+0000 7ff989deb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff98413c1e0 0x7ff9841cd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.561+0000 7ff989deb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff984128a50 con 0x7ff98406a770 2023-10-03T10:12:09.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.561+0000 7ff989deb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff984129e70 con 0x7ff98409dc30 2023-10-03T10:12:09.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.561+0000 7ff989deb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff98409f1a0 con 0x7ff98413c1e0 2023-10-03T10:12:09.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.562+0000 7ff982ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff98409dc30 0x7ff9841ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.562+0000 7ff982ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff98409dc30 0x7ff9841ccd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52720/0 (socket says 172.21.15.3:52720) 2023-10-03T10:12:09.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.562+0000 7ff982ffd700 1 -- 172.21.15.3:0/542937271 learned_addr learned my addr 172.21.15.3:0/542937271 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:09.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.562+0000 7fac74812700 1 Processor -- start 2023-10-03T10:12:09.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.562+0000 7ff983fff700 1 --2- 172.21.15.3:0/542937271 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff98413c1e0 0x7ff9841cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.562+0000 7ff9837fe700 1 --2- 172.21.15.3:0/542937271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff98406a770 0x7ff9841cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.562+0000 7ff982ffd700 1 -- 172.21.15.3:0/542937271 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff98413c1e0 msgr2=0x7ff9841cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.562+0000 7ff982ffd700 1 --2- 172.21.15.3:0/542937271 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff98413c1e0 0x7ff9841cd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.562+0000 7ff982ffd700 1 -- 172.21.15.3:0/542937271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff98406a770 msgr2=0x7ff9841cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.562+0000 7ff982ffd700 1 --2- 172.21.15.3:0/542937271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff98406a770 0x7ff9841cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.562+0000 7ff982ffd700 1 -- 172.21.15.3:0/542937271 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9740094a0 con 0x7ff98409dc30 2023-10-03T10:12:09.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.562+0000 7ff9837fe700 1 --2- 172.21.15.3:0/542937271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff98406a770 0x7ff9841cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:12:09.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.562+0000 7fac74812700 1 -- start start 2023-10-03T10:12:09.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.563+0000 7ff982ffd700 1 --2- 172.21.15.3:0/542937271 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff98409dc30 0x7ff9841ccd10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff97800b530 tx=0x7ff97800b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:09.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.563+0000 7ff96bfff700 1 -- 172.21.15.3:0/542937271 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff978011830 con 0x7ff98409dc30 2023-10-03T10:12:09.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.563+0000 7ff96bfff700 1 -- 172.21.15.3:0/542937271 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9780119b0 con 0x7ff98409dc30 2023-10-03T10:12:09.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.563+0000 7ff96bfff700 1 -- 172.21.15.3:0/542937271 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff978011d60 con 0x7ff98409dc30 2023-10-03T10:12:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.563+0000 7fac74812700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fac70071b40 0x7fac7006d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.563+0000 7fac74812700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac70130b30 0x7fac70137480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.563+0000 7fac74812700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fac70138dd0 0x7fac701379c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.563+0000 7fac74812700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac700719a0 con 0x7fac70130b30 2023-10-03T10:12:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.563+0000 7fac74812700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac700701a0 con 0x7fac70071b40 2023-10-03T10:12:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.563+0000 7fac74812700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fac7006f240 con 0x7fac70138dd0 2023-10-03T10:12:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.563+0000 7fac6e59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fac70071b40 0x7fac7006d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.563+0000 7fac6dd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac70130b30 0x7fac70137480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.563+0000 7fac6dd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac70130b30 0x7fac70137480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49186/0 (socket says 172.21.15.3:49186) 2023-10-03T10:12:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.563+0000 7fac6dd9b700 1 -- 172.21.15.3:0/2276084879 learned_addr learned my addr 172.21.15.3:0/2276084879 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.564+0000 7fac6ed9d700 1 --2- 172.21.15.3:0/2276084879 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fac70138dd0 0x7fac701379c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.564+0000 7fac6e59c700 1 -- 172.21.15.3:0/2276084879 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fac70138dd0 msgr2=0x7fac701379c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.564+0000 7fac6e59c700 1 --2- 172.21.15.3:0/2276084879 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fac70138dd0 0x7fac701379c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.564+0000 7fac6e59c700 1 -- 172.21.15.3:0/2276084879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac70130b30 msgr2=0x7fac70137480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.564+0000 7fac6e59c700 1 --2- 172.21.15.3:0/2276084879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac70130b30 0x7fac70137480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.564+0000 7fac6e59c700 1 -- 172.21.15.3:0/2276084879 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac600094a0 con 0x7fac70071b40 2023-10-03T10:12:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.564+0000 7fac6e59c700 1 --2- 172.21.15.3:0/2276084879 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fac70071b40 0x7fac7006d7c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fac5800c410 tx=0x7fac5800c720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.564+0000 7fac6dd9b700 1 --2- 172.21.15.3:0/2276084879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac70130b30 0x7fac70137480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.565+0000 7ff989deb700 1 -- 172.21.15.3:0/542937271 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9840a3a10 con 0x7ff98409dc30 2023-10-03T10:12:09.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.565+0000 7ff989deb700 1 -- 172.21.15.3:0/542937271 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff98412a800 con 0x7ff98409dc30 2023-10-03T10:12:09.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.565+0000 7fac66ffd700 1 -- 172.21.15.3:0/2276084879 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac58004c80 con 0x7fac70071b40 2023-10-03T10:12:09.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.565+0000 7fac66ffd700 1 -- 172.21.15.3:0/2276084879 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac58004de0 con 0x7fac70071b40 2023-10-03T10:12:09.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.565+0000 7fac66ffd700 1 -- 172.21.15.3:0/2276084879 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac580072d0 con 0x7fac70071b40 2023-10-03T10:12:09.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.565+0000 7fac74812700 1 -- 172.21.15.3:0/2276084879 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac7006f520 con 0x7fac70071b40 2023-10-03T10:12:09.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.565+0000 7fac74812700 1 -- 172.21.15.3:0/2276084879 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac70137f00 con 0x7fac70071b40 2023-10-03T10:12:09.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.566+0000 7fac74812700 1 -- 172.21.15.3:0/2276084879 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fac7013b6f0 con 0x7fac70071b40 2023-10-03T10:12:09.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.566+0000 7ff989deb700 1 -- 172.21.15.3:0/542937271 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff98409f620 con 0x7ff98409dc30 2023-10-03T10:12:09.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.566+0000 7ff96bfff700 1 -- 172.21.15.3:0/542937271 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7ff97801a400 con 0x7ff98409dc30 2023-10-03T10:12:09.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.567+0000 7fac66ffd700 1 -- 172.21.15.3:0/2276084879 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fac58007430 con 0x7fac70071b40 2023-10-03T10:12:09.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.567+0000 7fac66ffd700 1 -- 172.21.15.3:0/2276084879 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3951+0+0 (secure 0 0 0) 0x7fac5803bc70 con 0x7fac70071b40 2023-10-03T10:12:09.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.567+0000 7fac6dd9b700 1 -- 172.21.15.3:0/2276084879 >> v1:172.21.15.3:6816/118296 conn(0x7fac5002c230 legacy=0x7fac5002e6e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:09.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.567+0000 7fac66ffd700 1 -- 172.21.15.3:0/2276084879 --> v1:172.21.15.3:6812/118864 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fac50039010 con 0x7fac50035310 2023-10-03T10:12:09.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.567+0000 7fac66ffd700 1 -- 172.21.15.3:0/2276084879 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac5803bff0 con 0x7fac70071b40 2023-10-03T10:12:09.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.567+0000 7ff96bfff700 1 -- 172.21.15.3:0/542937271 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3951+0+0 (secure 0 0 0) 0x7ff97803bf50 con 0x7ff98409dc30 2023-10-03T10:12:09.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.567+0000 7ff9837fe700 1 -- 172.21.15.3:0/542937271 >> v1:172.21.15.3:6816/118296 conn(0x7ff97002c1b0 legacy=0x7ff97002e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:09.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.567+0000 7ff96bfff700 1 -- 172.21.15.3:0/542937271 --> v1:172.21.15.3:6800/118836 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff970038f10 con 0x7ff970035210 2023-10-03T10:12:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.567+0000 7ff96bfff700 1 -- 172.21.15.3:0/542937271 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff97803c2d0 con 0x7ff98409dc30 2023-10-03T10:12:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.568+0000 7fac66ffd700 1 -- 172.21.15.3:0/2276084879 <== osd.2 v1:172.21.15.3:6812/118864 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7fac50039010 con 0x7fac50035310 2023-10-03T10:12:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.569+0000 7ff96bfff700 1 -- 172.21.15.3:0/542937271 <== osd.3 v1:172.21.15.3:6800/118836 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7ff970038f10 con 0x7ff970035210 2023-10-03T10:12:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.576+0000 7f1812edb700 1 Processor -- start 2023-10-03T10:12:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.576+0000 7f1812edb700 1 -- start start 2023-10-03T10:12:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.577+0000 7f1812edb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f180c06e490 0x7f180c06e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.577+0000 7f1812edb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f180c134d20 0x7f180c13fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.577+0000 7f1812edb700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f180c06f8c0 con 0x7f180c06eda0 2023-10-03T10:12:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.577+0000 7f1812edb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f180c070ee0 con 0x7f180c06e490 2023-10-03T10:12:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.577+0000 7f1812edb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180c071980 con 0x7f180c134d20 2023-10-03T10:12:09.580 INFO:teuthology.orchestra.run.smithi003.stdout:38654705696 2023-10-03T10:12:09.581 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:12:09.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.578+0000 7f1811478700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f180c134d20 0x7f180c13fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.578+0000 7f1810c77700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f180c06e490 0x7f180c06e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.578+0000 7f180bfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f180c06eda0 0x7f180c134610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44456/0 (socket says 172.21.15.3:44456) 2023-10-03T10:12:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.578+0000 7f1811478700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f180c134d20 0x7f180c13fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52740/0 (socket says 172.21.15.3:52740) 2023-10-03T10:12:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.578+0000 7f180bfff700 1 -- 172.21.15.3:0/156337256 learned_addr learned my addr 172.21.15.3:0/156337256 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.578+0000 7f1810c77700 1 -- 172.21.15.3:0/156337256 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f180c134d20 msgr2=0x7f180c13fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.578+0000 7f1810c77700 1 --2- 172.21.15.3:0/156337256 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f180c134d20 0x7f180c13fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.578+0000 7f1810c77700 1 -- 172.21.15.3:0/156337256 >> v1:172.21.15.3:6789/0 conn(0x7f180c06eda0 legacy=0x7f180c134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.578+0000 7f1810c77700 1 -- 172.21.15.3:0/156337256 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180c071bd0 con 0x7f180c06e490 2023-10-03T10:12:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.578+0000 7f1810c77700 1 --2- 172.21.15.3:0/156337256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f180c06e490 0x7f180c06e860 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1800009840 tx=0x7f1800009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c58db6069f28c98 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.579+0000 7f180b7fe700 1 -- 172.21.15.3:0/156337256 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1800011830 con 0x7f180c06e490 2023-10-03T10:12:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.579+0000 7f180b7fe700 1 -- 172.21.15.3:0/156337256 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1800011990 con 0x7f180c06e490 2023-10-03T10:12:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.579+0000 7f180b7fe700 1 -- 172.21.15.3:0/156337256 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1800011d20 con 0x7f180c06e490 2023-10-03T10:12:09.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.580+0000 7f1812edb700 1 -- 172.21.15.3:0/156337256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f180c06e490 msgr2=0x7f180c06e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.580+0000 7f1812edb700 1 --2- 172.21.15.3:0/156337256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f180c06e490 0x7f180c06e860 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1800009840 tx=0x7f1800009b50 comp rx=0 tx=0).stop 2023-10-03T10:12:09.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.580+0000 7f1812edb700 1 -- 172.21.15.3:0/156337256 shutdown_connections 2023-10-03T10:12:09.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.580+0000 7f1812edb700 1 --2- 172.21.15.3:0/156337256 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f180c134d20 0x7f180c13fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.580+0000 7f1812edb700 1 --2- 172.21.15.3:0/156337256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f180c06e490 0x7f180c06e860 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.580+0000 7f1812edb700 1 -- 172.21.15.3:0/156337256 >> 172.21.15.3:0/156337256 conn(0x7f180c0a2aa0 msgr2=0x7f180c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:09.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.580+0000 7f1812edb700 1 -- 172.21.15.3:0/156337256 shutdown_connections 2023-10-03T10:12:09.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.580+0000 7f1812edb700 1 -- 172.21.15.3:0/156337256 wait complete. 2023-10-03T10:12:09.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.582+0000 7f1812edb700 1 Processor -- start 2023-10-03T10:12:09.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.582+0000 7f1812edb700 1 -- start start 2023-10-03T10:12:09.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.583+0000 7f1812edb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f180c06e490 0x7f180c1230d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.583+0000 7f1812edb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f180c06eda0 0x7f180c123610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.583+0000 7f180bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f180c06eda0 0x7f180c123610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.583+0000 7f1812edb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f180c134d20 0x7f180c123b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.583+0000 7f180bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f180c06eda0 0x7f180c123610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49196/0 (socket says 172.21.15.3:49196) 2023-10-03T10:12:09.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.583+0000 7f1810c77700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f180c06e490 0x7f180c1230d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.583+0000 7f1812edb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c125fa0 con 0x7f180c06eda0 2023-10-03T10:12:09.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.583+0000 7f1812edb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180c124530 con 0x7f180c06e490 2023-10-03T10:12:09.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.583+0000 7f180bfff700 1 -- 172.21.15.3:0/1912376061 learned_addr learned my addr 172.21.15.3:0/1912376061 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:09.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.583+0000 7f1812edb700 1 -- 172.21.15.3:0/1912376061 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f180c134620 con 0x7f180c134d20 2023-10-03T10:12:09.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.583+0000 7f1810c77700 1 -- 172.21.15.3:0/1912376061 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f180c134d20 msgr2=0x7f180c123b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.583+0000 7f1810c77700 1 --2- 172.21.15.3:0/1912376061 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f180c134d20 0x7f180c123b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.583+0000 7f1810c77700 1 -- 172.21.15.3:0/1912376061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f180c06eda0 msgr2=0x7f180c123610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.583+0000 7f1810c77700 1 --2- 172.21.15.3:0/1912376061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f180c06eda0 0x7f180c123610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.583+0000 7f1810c77700 1 -- 172.21.15.3:0/1912376061 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18000094a0 con 0x7f180c06e490 2023-10-03T10:12:09.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.584+0000 7f1810c77700 1 --2- 172.21.15.3:0/1912376061 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f180c06e490 0x7f180c1230d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1800009840 tx=0x7f1800004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:09.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.585+0000 7f18097fa700 1 -- 172.21.15.3:0/1912376061 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18000045d0 con 0x7f180c06e490 2023-10-03T10:12:09.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.585+0000 7f18097fa700 1 -- 172.21.15.3:0/1912376061 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180001f070 con 0x7f180c06e490 2023-10-03T10:12:09.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.585+0000 7f18097fa700 1 -- 172.21.15.3:0/1912376061 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18000038a0 con 0x7f180c06e490 2023-10-03T10:12:09.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.585+0000 7f1812edb700 1 -- 172.21.15.3:0/1912376061 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f180c125b70 con 0x7f180c06e490 2023-10-03T10:12:09.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.585+0000 7f1812edb700 1 -- 172.21.15.3:0/1912376061 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f180c126650 con 0x7f180c06e490 2023-10-03T10:12:09.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.586+0000 7f1812edb700 1 -- 172.21.15.3:0/1912376061 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f180c0700c0 con 0x7f180c06e490 2023-10-03T10:12:09.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.586+0000 7f18097fa700 1 -- 172.21.15.3:0/1912376061 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f1800003a00 con 0x7f180c06e490 2023-10-03T10:12:09.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.587+0000 7f18097fa700 1 -- 172.21.15.3:0/1912376061 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3951+0+0 (secure 0 0 0) 0x7f1800021070 con 0x7f180c06e490 2023-10-03T10:12:09.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.587+0000 7f180bfff700 1 -- 172.21.15.3:0/1912376061 >> v1:172.21.15.3:6816/118296 conn(0x7f17f002c200 legacy=0x7f17f002e6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:09.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.587+0000 7f18097fa700 1 -- 172.21.15.3:0/1912376061 --> v1:172.21.15.3:6808/118845 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f17f0038f60 con 0x7f17f0035260 2023-10-03T10:12:09.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.587+0000 7f18097fa700 1 -- 172.21.15.3:0/1912376061 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1800003cb0 con 0x7f180c06e490 2023-10-03T10:12:09.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.589+0000 7f18097fa700 1 -- 172.21.15.3:0/1912376061 <== osd.0 v1:172.21.15.3:6808/118845 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7f17f0038f60 con 0x7f17f0035260 2023-10-03T10:12:09.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.589+0000 7fac74812700 1 -- 172.21.15.3:0/2276084879 --> v1:172.21.15.3:6812/118864 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fac7012f9d0 con 0x7fac50035310 2023-10-03T10:12:09.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.592+0000 7ff989deb700 1 -- 172.21.15.3:0/542937271 --> v1:172.21.15.3:6800/118836 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff98406b470 con 0x7ff970035210 2023-10-03T10:12:09.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.592+0000 7ff96bfff700 1 -- 172.21.15.3:0/542937271 <== osd.3 v1:172.21.15.3:6800/118836 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 2434951863) 0x7ff97c0019c0 con 0x7ff970035210 2023-10-03T10:12:09.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.592+0000 7fac66ffd700 1 -- 172.21.15.3:0/2276084879 <== osd.2 v1:172.21.15.3:6812/118864 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 2188548419) 0x7fac7012f9d0 con 0x7fac50035310 2023-10-03T10:12:09.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.593+0000 7ff989deb700 1 -- 172.21.15.3:0/542937271 >> v1:172.21.15.3:6800/118836 conn(0x7ff970035210 legacy=0x7ff9700376a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.593+0000 7ff989deb700 1 -- 172.21.15.3:0/542937271 >> v1:172.21.15.3:6816/118296 conn(0x7ff97002c1b0 legacy=0x7ff97002e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:09.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.593+0000 7ff989deb700 1 -- 172.21.15.3:0/542937271 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff98409dc30 msgr2=0x7ff9841ccd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.593+0000 7ff989deb700 1 --2- 172.21.15.3:0/542937271 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff98409dc30 0x7ff9841ccd10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff97800b530 tx=0x7ff97800b8f0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.593+0000 7ff983fff700 1 -- 172.21.15.3:0/542937271 reap_dead start 2023-10-03T10:12:09.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.593+0000 7ff989deb700 1 -- 172.21.15.3:0/542937271 shutdown_connections 2023-10-03T10:12:09.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.593+0000 7ff989deb700 1 -- 172.21.15.3:0/542937271 >> 172.21.15.3:0/542937271 conn(0x7ff984066f50 msgr2=0x7ff9840a2110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:09.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.593+0000 7fac74812700 1 -- 172.21.15.3:0/2276084879 >> v1:172.21.15.3:6812/118864 conn(0x7fac50035310 legacy=0x7fac500377a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.593+0000 7fac74812700 1 -- 172.21.15.3:0/2276084879 >> v1:172.21.15.3:6816/118296 conn(0x7fac5002c230 legacy=0x7fac5002e6e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:09.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.593+0000 7fac74812700 1 -- 172.21.15.3:0/2276084879 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fac70071b40 msgr2=0x7fac7006d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.593+0000 7fac74812700 1 --2- 172.21.15.3:0/2276084879 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fac70071b40 0x7fac7006d7c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fac5800c410 tx=0x7fac5800c720 comp rx=0 tx=0).stop 2023-10-03T10:12:09.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.593+0000 7fac6ed9d700 1 -- 172.21.15.3:0/2276084879 reap_dead start 2023-10-03T10:12:09.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.593+0000 7ff989deb700 1 -- 172.21.15.3:0/542937271 shutdown_connections 2023-10-03T10:12:09.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.593+0000 7ff989deb700 1 -- 172.21.15.3:0/542937271 wait complete. 2023-10-03T10:12:09.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.596+0000 7fac74812700 1 -- 172.21.15.3:0/2276084879 shutdown_connections 2023-10-03T10:12:09.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.596+0000 7fac74812700 1 -- 172.21.15.3:0/2276084879 >> 172.21.15.3:0/2276084879 conn(0x7fac7009e7e0 msgr2=0x7fac700a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:09.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.596+0000 7fac74812700 1 -- 172.21.15.3:0/2276084879 shutdown_connections 2023-10-03T10:12:09.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.596+0000 7fac74812700 1 -- 172.21.15.3:0/2276084879 wait complete. 2023-10-03T10:12:09.606 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:09.605+0000 7f2cd33f8240 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-03T10:12:09.614 INFO:teuthology.orchestra.run.smithi003.stdout:38654705696 2023-10-03T10:12:09.614 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:12:09.618 INFO:teuthology.orchestra.run.smithi003.stdout:38654705695 2023-10-03T10:12:09.618 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:12:09.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.622+0000 7f1812edb700 1 -- 172.21.15.3:0/1912376061 --> v1:172.21.15.3:6808/118845 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f180c06f2c0 con 0x7f17f0035260 2023-10-03T10:12:09.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.622+0000 7f18097fa700 1 -- 172.21.15.3:0/1912376061 <== osd.0 v1:172.21.15.3:6808/118845 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 2434951863) 0x7f1804007330 con 0x7f17f0035260 2023-10-03T10:12:09.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.623+0000 7f1812edb700 1 -- 172.21.15.3:0/1912376061 >> v1:172.21.15.3:6808/118845 conn(0x7f17f0035260 legacy=0x7f17f00376f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.623+0000 7f1812edb700 1 -- 172.21.15.3:0/1912376061 >> v1:172.21.15.3:6816/118296 conn(0x7f17f002c200 legacy=0x7f17f002e6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:09.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.623+0000 7f1812edb700 1 -- 172.21.15.3:0/1912376061 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f180c06e490 msgr2=0x7f180c1230d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.623+0000 7f1812edb700 1 --2- 172.21.15.3:0/1912376061 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f180c06e490 0x7f180c1230d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1800009840 tx=0x7f1800004750 comp rx=0 tx=0).stop 2023-10-03T10:12:09.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.623+0000 7f1811478700 1 -- 172.21.15.3:0/1912376061 reap_dead start 2023-10-03T10:12:09.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.623+0000 7f1812edb700 1 -- 172.21.15.3:0/1912376061 shutdown_connections 2023-10-03T10:12:09.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.623+0000 7f1812edb700 1 -- 172.21.15.3:0/1912376061 >> 172.21.15.3:0/1912376061 conn(0x7f180c0a2aa0 msgr2=0x7f180c136e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:09.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.623+0000 7f1812edb700 1 -- 172.21.15.3:0/1912376061 shutdown_connections 2023-10-03T10:12:09.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.623+0000 7f1812edb700 1 -- 172.21.15.3:0/1912376061 wait complete. 2023-10-03T10:12:09.645 INFO:teuthology.orchestra.run.smithi003.stdout:38654705696 2023-10-03T10:12:09.646 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:12:09.712 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:09.712+0000 7f2cd33f8240 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-03T10:12:09.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.749+0000 7f9a7dd6a700 1 Processor -- start 2023-10-03T10:12:09.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.749+0000 7f9a7dd6a700 1 -- start start 2023-10-03T10:12:09.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.749+0000 7f9a7dd6a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a7806e570 0x7f9a7806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.750+0000 7f9a7dd6a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a7809eca0 0x7f9a7806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.750+0000 7f9a7dd6a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a78070ee0 con 0x7f9a78134610 2023-10-03T10:12:09.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.750+0000 7f9a7dd6a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a78071480 con 0x7f9a7806e570 2023-10-03T10:12:09.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.750+0000 7f9a7dd6a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7806f3c0 con 0x7f9a7809eca0 2023-10-03T10:12:09.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.750+0000 7f9a777fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a7806e570 0x7f9a7806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.750+0000 7f9a76ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a7809eca0 0x7f9a7806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.750+0000 7f9a777fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a7806e570 0x7f9a7806e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52748/0 (socket says 172.21.15.3:52748) 2023-10-03T10:12:09.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.750+0000 7f9a76ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a7809eca0 0x7f9a7806ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:34070/0 (socket says 172.21.15.3:34070) 2023-10-03T10:12:09.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.750+0000 7f9a777fe700 1 -- 172.21.15.3:0/724077892 learned_addr learned my addr 172.21.15.3:0/724077892 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:09.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.750+0000 7f9a777fe700 1 -- 172.21.15.3:0/724077892 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a7809eca0 msgr2=0x7f9a7806ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:09.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.750+0000 7f9a777fe700 1 --2- 172.21.15.3:0/724077892 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a7809eca0 0x7f9a7806ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.750+0000 7f9a777fe700 1 -- 172.21.15.3:0/724077892 >> v1:172.21.15.3:6789/0 conn(0x7f9a78134610 legacy=0x7f9a78134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:09.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.750+0000 7f9a777fe700 1 -- 172.21.15.3:0/724077892 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a78071b90 con 0x7f9a7806e570 2023-10-03T10:12:09.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.751+0000 7f9a76ffd700 1 --2- 172.21.15.3:0/724077892 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a7809eca0 0x7f9a7806ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:09.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.751+0000 7f9a777fe700 1 --2- 172.21.15.3:0/724077892 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a7806e570 0x7f9a7806e940 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009920 tx=0x7f9a68009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcbab15556b9761e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:09.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.751+0000 7f9a767fc700 1 -- 172.21.15.3:0/724077892 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a68004230 con 0x7f9a7806e570 2023-10-03T10:12:09.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.751+0000 7f9a767fc700 1 -- 172.21.15.3:0/724077892 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a68004390 con 0x7f9a7806e570 2023-10-03T10:12:09.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.751+0000 7f9a767fc700 1 -- 172.21.15.3:0/724077892 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a680036a0 con 0x7f9a7806e570 2023-10-03T10:12:09.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.752+0000 7f9a7dd6a700 1 -- 172.21.15.3:0/724077892 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a7806e570 msgr2=0x7f9a7806e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.752+0000 7f9a7dd6a700 1 --2- 172.21.15.3:0/724077892 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a7806e570 0x7f9a7806e940 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009920 tx=0x7f9a68009c30 comp rx=0 tx=0).stop 2023-10-03T10:12:09.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.752+0000 7f9a7dd6a700 1 -- 172.21.15.3:0/724077892 shutdown_connections 2023-10-03T10:12:09.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.752+0000 7f9a7dd6a700 1 --2- 172.21.15.3:0/724077892 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a7806e570 0x7f9a7806e940 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.752+0000 7f9a7dd6a700 1 --2- 172.21.15.3:0/724077892 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a7809eca0 0x7f9a7806ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.752+0000 7f9a7dd6a700 1 -- 172.21.15.3:0/724077892 >> 172.21.15.3:0/724077892 conn(0x7f9a780a2aa0 msgr2=0x7f9a780a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:09.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.752+0000 7f9a7dd6a700 1 -- 172.21.15.3:0/724077892 shutdown_connections 2023-10-03T10:12:09.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.752+0000 7f9a7dd6a700 1 -- 172.21.15.3:0/724077892 wait complete. 2023-10-03T10:12:09.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.754+0000 7f9a7dd6a700 1 Processor -- start 2023-10-03T10:12:09.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.754+0000 7f9a7dd6a700 1 -- start start 2023-10-03T10:12:09.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.755+0000 7f9a7dd6a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a7806e570 0x7f9a78123240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.755+0000 7f9a7dd6a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a78134610 0x7f9a78123780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.755+0000 7f9a7dd6a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a78123cc0 0x7f9a781cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.755+0000 7f9a7dd6a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a78126a30 con 0x7f9a7806e570 2023-10-03T10:12:09.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.755+0000 7f9a7dd6a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a78127990 con 0x7f9a78134610 2023-10-03T10:12:09.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.755+0000 7f9a7dd6a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a78133cf0 con 0x7f9a78123cc0 2023-10-03T10:12:09.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.756+0000 7f9a76ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a78134610 0x7f9a78123780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.756+0000 7f9a76ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a78134610 0x7f9a78123780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52754/0 (socket says 172.21.15.3:52754) 2023-10-03T10:12:09.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.756+0000 7f9a76ffd700 1 -- 172.21.15.3:0/2412386821 learned_addr learned my addr 172.21.15.3:0/2412386821 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:09.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.756+0000 7f9a77fff700 1 --2- 172.21.15.3:0/2412386821 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a78123cc0 0x7f9a781cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.756+0000 7f9a76ffd700 1 -- 172.21.15.3:0/2412386821 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a78123cc0 msgr2=0x7f9a781cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.756+0000 7f9a76ffd700 1 --2- 172.21.15.3:0/2412386821 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a78123cc0 0x7f9a781cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.756+0000 7f9a76ffd700 1 -- 172.21.15.3:0/2412386821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a7806e570 msgr2=0x7f9a78123240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.756+0000 7f9a76ffd700 1 --2- 172.21.15.3:0/2412386821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a7806e570 0x7f9a78123240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.756+0000 7f9a76ffd700 1 -- 172.21.15.3:0/2412386821 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a6c0094a0 con 0x7f9a78134610 2023-10-03T10:12:09.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.756+0000 7f9a76ffd700 1 --2- 172.21.15.3:0/2412386821 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a78134610 0x7f9a78123780 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00ebe0 tx=0x7f9a6c00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:09.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.756+0000 7f9a5ffff700 1 -- 172.21.15.3:0/2412386821 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c00c850 con 0x7f9a78134610 2023-10-03T10:12:09.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.757+0000 7f9a5ffff700 1 -- 172.21.15.3:0/2412386821 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6c00c9b0 con 0x7f9a78134610 2023-10-03T10:12:09.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.757+0000 7f9a7dd6a700 1 -- 172.21.15.3:0/2412386821 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a68009550 con 0x7f9a78134610 2023-10-03T10:12:09.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.757+0000 7f9a5ffff700 1 -- 172.21.15.3:0/2412386821 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c00cd60 con 0x7f9a78134610 2023-10-03T10:12:09.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.757+0000 7f9a7dd6a700 1 -- 172.21.15.3:0/2412386821 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a781274a0 con 0x7f9a78134610 2023-10-03T10:12:09.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.758+0000 7f9a5ffff700 1 -- 172.21.15.3:0/2412386821 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f9a6c013470 con 0x7f9a78134610 2023-10-03T10:12:09.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.758+0000 7f9a5ffff700 1 -- 172.21.15.3:0/2412386821 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3951+0+0 (secure 0 0 0) 0x7f9a6c03baa0 con 0x7f9a78134610 2023-10-03T10:12:09.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.758+0000 7f9a777fe700 1 -- 172.21.15.3:0/2412386821 >> v1:172.21.15.3:6816/118296 conn(0x7f9a6402c200 legacy=0x7f9a6402e6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:09.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.758+0000 7f9a7dd6a700 1 -- 172.21.15.3:0/2412386821 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a580052e0 con 0x7f9a78134610 2023-10-03T10:12:09.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.767+0000 7f9a5ffff700 1 -- 172.21.15.3:0/2412386821 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f9a6c0232f0 con 0x7f9a78134610 2023-10-03T10:12:09.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.783+0000 7f4e107fa700 1 Processor -- start 2023-10-03T10:12:09.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.783+0000 7f4e107fa700 1 -- start start 2023-10-03T10:12:09.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.784+0000 7f4e107fa700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4e0809d6f0 0x7f4e0809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.784+0000 7f4e107fa700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4e0809e410 0x7f4e08130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.784+0000 7f4e107fa700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e0809f130 con 0x7f4e0809e000 2023-10-03T10:12:09.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.784+0000 7f4e107fa700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e0809f520 con 0x7f4e0809d6f0 2023-10-03T10:12:09.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.784+0000 7f4e107fa700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0809e930 con 0x7f4e0809e410 2023-10-03T10:12:09.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.784+0000 7f4e0dd95700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4e0809e000 0x7f4e08130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44474/0 (socket says 172.21.15.3:44474) 2023-10-03T10:12:09.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.784+0000 7f4e0dd95700 1 -- 172.21.15.3:0/3977620058 learned_addr learned my addr 172.21.15.3:0/3977620058 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:09.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.784+0000 7f4e0e596700 1 --2- 172.21.15.3:0/3977620058 >> v2:172.21.15.3:3300/0 conn(0x7f4e0809d6f0 0x7f4e0809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.784+0000 7f4e0ed97700 1 --2- 172.21.15.3:0/3977620058 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4e0809e410 0x7f4e08130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.785+0000 7f4e0d594700 1 -- 172.21.15.3:0/3977620058 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3877757365 0 0) 0x7f4e0809f130 con 0x7f4e0809e000 2023-10-03T10:12:09.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.785+0000 7f4e0d594700 1 -- 172.21.15.3:0/3977620058 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4df40035b0 con 0x7f4e0809e000 2023-10-03T10:12:09.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.785+0000 7f4e0e596700 1 -- 172.21.15.3:0/3977620058 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4e0809e410 msgr2=0x7f4e08130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:09.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.785+0000 7f4e0e596700 1 --2- 172.21.15.3:0/3977620058 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4e0809e410 0x7f4e08130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.785+0000 7f4e0e596700 1 -- 172.21.15.3:0/3977620058 >> v1:172.21.15.3:6789/0 conn(0x7f4e0809e000 legacy=0x7f4e08130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.785+0000 7f4e0e596700 1 -- 172.21.15.3:0/3977620058 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0809b2d0 con 0x7f4e0809d6f0 2023-10-03T10:12:09.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.785+0000 7f4e0e596700 1 --2- 172.21.15.3:0/3977620058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e0809d6f0 0x7f4e0809dac0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009840 tx=0x7f4e04009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a438cf8a8aa2932d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:09.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.785+0000 7f4e0d594700 1 -- 172.21.15.3:0/3977620058 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e04011830 con 0x7f4e0809d6f0 2023-10-03T10:12:09.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.785+0000 7f4e0d594700 1 -- 172.21.15.3:0/3977620058 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e04011990 con 0x7f4e0809d6f0 2023-10-03T10:12:09.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.785+0000 7f4e0d594700 1 -- 172.21.15.3:0/3977620058 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e04011d00 con 0x7f4e0809d6f0 2023-10-03T10:12:09.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.786+0000 7f4e107fa700 1 -- 172.21.15.3:0/3977620058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e0809d6f0 msgr2=0x7f4e0809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.786+0000 7f4e107fa700 1 --2- 172.21.15.3:0/3977620058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e0809d6f0 0x7f4e0809dac0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009840 tx=0x7f4e04009b50 comp rx=0 tx=0).stop 2023-10-03T10:12:09.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.786+0000 7f4e107fa700 1 -- 172.21.15.3:0/3977620058 shutdown_connections 2023-10-03T10:12:09.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.786+0000 7f4e107fa700 1 --2- 172.21.15.3:0/3977620058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e0809d6f0 0x7f4e0809dac0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.786+0000 7f4e107fa700 1 --2- 172.21.15.3:0/3977620058 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4e0809e410 0x7f4e08130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.786+0000 7f4e107fa700 1 -- 172.21.15.3:0/3977620058 >> 172.21.15.3:0/3977620058 conn(0x7f4e08066f50 msgr2=0x7f4e0809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:09.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.786+0000 7f4e107fa700 1 -- 172.21.15.3:0/3977620058 shutdown_connections 2023-10-03T10:12:09.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.786+0000 7f4e107fa700 1 -- 172.21.15.3:0/3977620058 wait complete. 2023-10-03T10:12:09.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.787+0000 7f4e107fa700 1 Processor -- start 2023-10-03T10:12:09.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.788+0000 7f4e107fa700 1 -- start start 2023-10-03T10:12:09.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.788+0000 7f4e107fa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e0809d6f0 0x7f4e08139970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.788+0000 7f4e107fa700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4e0809e000 0x7f4e08139eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.788+0000 7f4e107fa700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4e0809e410 0x7f4e0813a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.788+0000 7f4e107fa700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e080a3940 con 0x7f4e0809d6f0 2023-10-03T10:12:09.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.788+0000 7f4e107fa700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0809c700 con 0x7f4e0809e000 2023-10-03T10:12:09.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.788+0000 7f4e107fa700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0813b620 con 0x7f4e0809e410 2023-10-03T10:12:09.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.788+0000 7f4e0dd95700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4e0809e000 0x7f4e08139eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.788+0000 7f4e0dd95700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4e0809e000 0x7f4e08139eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52762/0 (socket says 172.21.15.3:52762) 2023-10-03T10:12:09.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.788+0000 7f4e0dd95700 1 -- 172.21.15.3:0/3065002237 learned_addr learned my addr 172.21.15.3:0/3065002237 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:09.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.788+0000 7f4e0e596700 1 --2- 172.21.15.3:0/3065002237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e0809d6f0 0x7f4e08139970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.789+0000 7f4e0ed97700 1 --2- 172.21.15.3:0/3065002237 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4e0809e410 0x7f4e0813a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.789+0000 7f4e0dd95700 1 -- 172.21.15.3:0/3065002237 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4e0809e410 msgr2=0x7f4e0813a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.789+0000 7f4e0dd95700 1 --2- 172.21.15.3:0/3065002237 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4e0809e410 0x7f4e0813a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.789+0000 7f4e0dd95700 1 -- 172.21.15.3:0/3065002237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e0809d6f0 msgr2=0x7f4e08139970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.789+0000 7f4e0dd95700 1 --2- 172.21.15.3:0/3065002237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e0809d6f0 0x7f4e08139970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.789+0000 7f4e0dd95700 1 -- 172.21.15.3:0/3065002237 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e040094a0 con 0x7f4e0809e000 2023-10-03T10:12:09.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.789+0000 7f4e0dd95700 1 --2- 172.21.15.3:0/3065002237 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4e0809e000 0x7f4e08139eb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4df800c700 tx=0x7f4df800ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:09.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.790+0000 7f4dfeffd700 1 -- 172.21.15.3:0/3065002237 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df8005d70 con 0x7f4e0809e000 2023-10-03T10:12:09.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.790+0000 7f4dfeffd700 1 -- 172.21.15.3:0/3065002237 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df80070d0 con 0x7f4e0809e000 2023-10-03T10:12:09.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.790+0000 7f4e107fa700 1 -- 172.21.15.3:0/3065002237 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e08138dc0 con 0x7f4e0809e000 2023-10-03T10:12:09.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.790+0000 7f4e107fa700 1 -- 172.21.15.3:0/3065002237 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e081378a0 con 0x7f4e0809e000 2023-10-03T10:12:09.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.791+0000 7f4e107fa700 1 -- 172.21.15.3:0/3065002237 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e08137ba0 con 0x7f4e0809e000 2023-10-03T10:12:09.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.791+0000 7f4dfeffd700 1 -- 172.21.15.3:0/3065002237 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df8007460 con 0x7f4e0809e000 2023-10-03T10:12:09.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.792+0000 7f4dfeffd700 1 -- 172.21.15.3:0/3065002237 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f4df80075c0 con 0x7f4e0809e000 2023-10-03T10:12:09.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.792+0000 7f4dfeffd700 1 -- 172.21.15.3:0/3065002237 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3951+0+0 (secure 0 0 0) 0x7f4df803b250 con 0x7f4e0809e000 2023-10-03T10:12:09.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.795+0000 7f4e0e596700 1 -- 172.21.15.3:0/3065002237 >> v1:172.21.15.3:6816/118296 conn(0x7f4dec02c1b0 legacy=0x7f4dec02e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:09.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.795+0000 7fc925ab6700 1 Processor -- start 2023-10-03T10:12:09.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.796+0000 7fc925ab6700 1 -- start start 2023-10-03T10:12:09.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.796+0000 7fc925ab6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc9201338a0 0x7fc920133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.796+0000 7fc925ab6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc92009e930 0x7fc92013b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.796+0000 7fc925ab6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc92006ef30 con 0x7fc92009de50 2023-10-03T10:12:09.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.796+0000 7fc925ab6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc92006fef0 con 0x7fc9201338a0 2023-10-03T10:12:09.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.796+0000 7fc925ab6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9201334b0 con 0x7fc92009e930 2023-10-03T10:12:09.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.796+0000 7fc91ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc92009e930 0x7fc92013b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.796+0000 7fc91ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc92009e930 0x7fc92013b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52768/0 (socket says 172.21.15.3:52768) 2023-10-03T10:12:09.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.796+0000 7fc91ffff700 1 -- 172.21.15.3:0/361691187 learned_addr learned my addr 172.21.15.3:0/361691187 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:09.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.797+0000 7fc91ffff700 1 -- 172.21.15.3:0/361691187 >> v2:172.21.15.3:3300/0 conn(0x7fc9201338a0 msgr2=0x7fc920133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:09.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.797+0000 7fc91ffff700 1 --2- 172.21.15.3:0/361691187 >> v2:172.21.15.3:3300/0 conn(0x7fc9201338a0 0x7fc920133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.797+0000 7fc91ffff700 1 -- 172.21.15.3:0/361691187 >> v1:172.21.15.3:6789/0 conn(0x7fc92009de50 legacy=0x7fc92009e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:09.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.797+0000 7fc91ffff700 1 -- 172.21.15.3:0/361691187 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92006fc30 con 0x7fc92009e930 2023-10-03T10:12:09.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.797+0000 7fc91ffff700 1 --2- 172.21.15.3:0/361691187 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc92009e930 0x7fc92013b740 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc918009e50 tx=0x7fc91800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fa68dfb00cd2fbb server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:09.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.800+0000 7f4dfeffd700 1 -- 172.21.15.3:0/3065002237 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f4df800e280 con 0x7f4e0809e000 2023-10-03T10:12:09.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.800+0000 7fc91e7fc700 1 -- 172.21.15.3:0/361691187 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91800e070 con 0x7fc92009e930 2023-10-03T10:12:09.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.800+0000 7fc91e7fc700 1 -- 172.21.15.3:0/361691187 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc918003a20 con 0x7fc92009e930 2023-10-03T10:12:09.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.800+0000 7fc91e7fc700 1 -- 172.21.15.3:0/361691187 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc918003dd0 con 0x7fc92009e930 2023-10-03T10:12:09.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.801+0000 7fc925ab6700 1 -- 172.21.15.3:0/361691187 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc92009e930 msgr2=0x7fc92013b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.801+0000 7fc925ab6700 1 --2- 172.21.15.3:0/361691187 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc92009e930 0x7fc92013b740 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc918009e50 tx=0x7fc91800b330 comp rx=0 tx=0).stop 2023-10-03T10:12:09.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.802+0000 7fc925ab6700 1 -- 172.21.15.3:0/361691187 shutdown_connections 2023-10-03T10:12:09.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.802+0000 7fc925ab6700 1 --2- 172.21.15.3:0/361691187 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc92009e930 0x7fc92013b740 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.802+0000 7fc925ab6700 1 --2- 172.21.15.3:0/361691187 >> v2:172.21.15.3:3300/0 conn(0x7fc9201338a0 0x7fc920133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.802+0000 7fc925ab6700 1 -- 172.21.15.3:0/361691187 >> 172.21.15.3:0/361691187 conn(0x7fc9200a2aa0 msgr2=0x7fc9200a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:09.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.802+0000 7fc925ab6700 1 -- 172.21.15.3:0/361691187 shutdown_connections 2023-10-03T10:12:09.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.802+0000 7fc925ab6700 1 -- 172.21.15.3:0/361691187 wait complete. 2023-10-03T10:12:09.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.803+0000 7fc925ab6700 1 Processor -- start 2023-10-03T10:12:09.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.804+0000 7fc925ab6700 1 -- start start 2023-10-03T10:12:09.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.804+0000 7fc925ab6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc92009de50 0x7fc92011eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.804+0000 7fc925ab6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc92009e930 0x7fc92011f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.804+0000 7fc925ab6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9201338a0 0x7fc92011f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.804+0000 7fc925ab6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9201207f0 con 0x7fc92009de50 2023-10-03T10:12:09.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.804+0000 7fc925ab6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92006f640 con 0x7fc9201338a0 2023-10-03T10:12:09.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.804+0000 7fc925ab6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc92012fb00 con 0x7fc92009e930 2023-10-03T10:12:09.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.805+0000 7fc91effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc92009e930 0x7fc92011f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.805+0000 7fc91effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc92009e930 0x7fc92011f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:34118/0 (socket says 172.21.15.3:34118) 2023-10-03T10:12:09.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.805+0000 7fc91effd700 1 -- 172.21.15.3:0/665214207 learned_addr learned my addr 172.21.15.3:0/665214207 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:09.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.805+0000 7fc91ffff700 1 --2- 172.21.15.3:0/665214207 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9201338a0 0x7fc92011f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.805+0000 7fc91f7fe700 1 --2- 172.21.15.3:0/665214207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc92009de50 0x7fc92011eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.805+0000 7fc91effd700 1 -- 172.21.15.3:0/665214207 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9201338a0 msgr2=0x7fc92011f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.805+0000 7fc91effd700 1 --2- 172.21.15.3:0/665214207 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9201338a0 0x7fc92011f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.805+0000 7fc91effd700 1 -- 172.21.15.3:0/665214207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc92009de50 msgr2=0x7fc92011eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.805+0000 7fc91effd700 1 --2- 172.21.15.3:0/665214207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc92009de50 0x7fc92011eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.805+0000 7fc91effd700 1 -- 172.21.15.3:0/665214207 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc918009b30 con 0x7fc92009e930 2023-10-03T10:12:09.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.806+0000 7fc91effd700 1 --2- 172.21.15.3:0/665214207 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc92009e930 0x7fc92011f080 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc91400b470 tx=0x7fc91400b780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:09.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.806+0000 7fc91f7fe700 1 --2- 172.21.15.3:0/665214207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc92009de50 0x7fc92011eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:12:09.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.806+0000 7fc907fff700 1 -- 172.21.15.3:0/665214207 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc914014d70 con 0x7fc92009e930 2023-10-03T10:12:09.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.806+0000 7fc907fff700 1 -- 172.21.15.3:0/665214207 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc91401c440 con 0x7fc92009e930 2023-10-03T10:12:09.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.806+0000 7fc925ab6700 1 -- 172.21.15.3:0/665214207 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc92011ff10 con 0x7fc92009e930 2023-10-03T10:12:09.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.806+0000 7fc907fff700 1 -- 172.21.15.3:0/665214207 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91401c7f0 con 0x7fc92009e930 2023-10-03T10:12:09.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.806+0000 7fc925ab6700 1 -- 172.21.15.3:0/665214207 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92012f640 con 0x7fc92009e930 2023-10-03T10:12:09.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.809+0000 7fc907fff700 1 -- 172.21.15.3:0/665214207 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fc91401c950 con 0x7fc92009e930 2023-10-03T10:12:09.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.809+0000 7fc907fff700 1 -- 172.21.15.3:0/665214207 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3951+0+0 (secure 0 0 0) 0x7fc91403b3f0 con 0x7fc92009e930 2023-10-03T10:12:09.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.809+0000 7fc91f7fe700 1 -- 172.21.15.3:0/665214207 >> v1:172.21.15.3:6816/118296 conn(0x7fc90c03d1c0 legacy=0x7fc90c03f670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:09.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.809+0000 7fc925ab6700 1 -- 172.21.15.3:0/665214207 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc920074ea0 con 0x7fc92009e930 2023-10-03T10:12:09.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.817+0000 7fc907fff700 1 -- 172.21.15.3:0/665214207 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fc914021020 con 0x7fc92009e930 2023-10-03T10:12:09.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.881+0000 7f0823e46700 1 Processor -- start 2023-10-03T10:12:09.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.882+0000 7f0823e46700 1 -- start start 2023-10-03T10:12:09.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.882+0000 7f0823e46700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f081c09e1d0 0x7f081c135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.882+0000 7f0823e46700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f081c135e50 0x7f081c138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.882+0000 7f0823e46700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081c09f130 con 0x7f081c09d6f0 2023-10-03T10:12:09.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.882+0000 7f0823e46700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f081c09f520 con 0x7f081c09e1d0 2023-10-03T10:12:09.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.882+0000 7f0823e46700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081c09e930 con 0x7f081c135e50 2023-10-03T10:12:09.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.883+0000 7f0821be2700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f081c09d6f0 0x7f081c09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44492/0 (socket says 172.21.15.3:44492) 2023-10-03T10:12:09.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.883+0000 7f0821be2700 1 -- 172.21.15.3:0/2011585122 learned_addr learned my addr 172.21.15.3:0/2011585122 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:09.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.883+0000 7f08223e3700 1 --2- 172.21.15.3:0/2011585122 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f081c135e50 0x7f081c138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.883+0000 7f08223e3700 1 -- 172.21.15.3:0/2011585122 >> v2:172.21.15.3:3300/0 conn(0x7f081c09e1d0 msgr2=0x7f081c135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:09.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.883+0000 7f08223e3700 1 --2- 172.21.15.3:0/2011585122 >> v2:172.21.15.3:3300/0 conn(0x7f081c09e1d0 0x7f081c135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.883+0000 7f08223e3700 1 -- 172.21.15.3:0/2011585122 >> v1:172.21.15.3:6789/0 conn(0x7f081c09d6f0 legacy=0x7f081c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.883+0000 7f08223e3700 1 -- 172.21.15.3:0/2011585122 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081c09b2d0 con 0x7f081c135e50 2023-10-03T10:12:09.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.884+0000 7f08223e3700 1 --2- 172.21.15.3:0/2011585122 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f081c135e50 0x7f081c138230 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0814009d90 tx=0x7f081400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=324748b102d160e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:09.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.884+0000 7f0820be0700 1 -- 172.21.15.3:0/2011585122 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081400e070 con 0x7f081c135e50 2023-10-03T10:12:09.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.884+0000 7f0820be0700 1 -- 172.21.15.3:0/2011585122 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081400bbd0 con 0x7f081c135e50 2023-10-03T10:12:09.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.884+0000 7f0820be0700 1 -- 172.21.15.3:0/2011585122 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0814012430 con 0x7f081c135e50 2023-10-03T10:12:09.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.885+0000 7f0823e46700 1 -- 172.21.15.3:0/2011585122 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f081c135e50 msgr2=0x7f081c138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.885+0000 7f0823e46700 1 --2- 172.21.15.3:0/2011585122 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f081c135e50 0x7f081c138230 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0814009d90 tx=0x7f081400b330 comp rx=0 tx=0).stop 2023-10-03T10:12:09.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.885+0000 7f0823e46700 1 -- 172.21.15.3:0/2011585122 shutdown_connections 2023-10-03T10:12:09.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.885+0000 7f0823e46700 1 --2- 172.21.15.3:0/2011585122 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f081c135e50 0x7f081c138230 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.885+0000 7f0823e46700 1 --2- 172.21.15.3:0/2011585122 >> v2:172.21.15.3:3300/0 conn(0x7f081c09e1d0 0x7f081c135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.885+0000 7f0823e46700 1 -- 172.21.15.3:0/2011585122 >> 172.21.15.3:0/2011585122 conn(0x7f081c066f50 msgr2=0x7f081c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:09.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.886+0000 7f0823e46700 1 -- 172.21.15.3:0/2011585122 shutdown_connections 2023-10-03T10:12:09.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.886+0000 7f0823e46700 1 -- 172.21.15.3:0/2011585122 wait complete. 2023-10-03T10:12:09.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.887+0000 7f0823e46700 1 Processor -- start 2023-10-03T10:12:09.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.887+0000 7f0823e46700 1 -- start start 2023-10-03T10:12:09.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.888+0000 7f0823e46700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f081c09d6f0 0x7f081c132890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.888+0000 7f0823e46700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f081c09e1d0 0x7f081c132dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.888+0000 7f0823e46700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f081c135e50 0x7f081c133310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:09.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.888+0000 7f0823e46700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c1199e0 con 0x7f081c09d6f0 2023-10-03T10:12:09.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.888+0000 7f0823e46700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081c134a80 con 0x7f081c09e1d0 2023-10-03T10:12:09.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.888+0000 7f0823e46700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f081c135b40 con 0x7f081c135e50 2023-10-03T10:12:09.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.888+0000 7f0821be2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f081c09d6f0 0x7f081c132890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.888+0000 7f0821be2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f081c09d6f0 0x7f081c132890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49272/0 (socket says 172.21.15.3:49272) 2023-10-03T10:12:09.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.888+0000 7f0821be2700 1 -- 172.21.15.3:0/1975441608 learned_addr learned my addr 172.21.15.3:0/1975441608 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:09.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.888+0000 7f08213e1700 1 --2- 172.21.15.3:0/1975441608 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f081c09e1d0 0x7f081c132dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.889+0000 7f08223e3700 1 --2- 172.21.15.3:0/1975441608 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f081c135e50 0x7f081c133310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:09.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.889+0000 7f0821be2700 1 -- 172.21.15.3:0/1975441608 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f081c135e50 msgr2=0x7f081c133310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.889+0000 7f0821be2700 1 --2- 172.21.15.3:0/1975441608 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f081c135e50 0x7f081c133310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.889+0000 7f0821be2700 1 -- 172.21.15.3:0/1975441608 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f081c09e1d0 msgr2=0x7f081c132dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.889+0000 7f0821be2700 1 --2- 172.21.15.3:0/1975441608 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f081c09e1d0 0x7f081c132dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.889+0000 7f0821be2700 1 -- 172.21.15.3:0/1975441608 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0814009a70 con 0x7f081c09d6f0 2023-10-03T10:12:09.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.889+0000 7f08223e3700 1 --2- 172.21.15.3:0/1975441608 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f081c135e50 0x7f081c133310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:12:09.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.889+0000 7f08213e1700 1 --2- 172.21.15.3:0/1975441608 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f081c09e1d0 0x7f081c132dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:12:09.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.889+0000 7f0821be2700 1 --2- 172.21.15.3:0/1975441608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f081c09d6f0 0x7f081c132890 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f081800c4b0 tx=0x7f081800c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:09.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.890+0000 7f08127fc700 1 -- 172.21.15.3:0/1975441608 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0818005d70 con 0x7f081c09d6f0 2023-10-03T10:12:09.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.890+0000 7f08127fc700 1 -- 172.21.15.3:0/1975441608 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08180070d0 con 0x7f081c09d6f0 2023-10-03T10:12:09.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.890+0000 7f0823e46700 1 -- 172.21.15.3:0/1975441608 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f081c09c700 con 0x7f081c09d6f0 2023-10-03T10:12:09.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.890+0000 7f08127fc700 1 -- 172.21.15.3:0/1975441608 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0818007440 con 0x7f081c09d6f0 2023-10-03T10:12:09.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.890+0000 7f0823e46700 1 -- 172.21.15.3:0/1975441608 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f081c0a2360 con 0x7f081c09d6f0 2023-10-03T10:12:09.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.891+0000 7f08127fc700 1 -- 172.21.15.3:0/1975441608 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f08180075a0 con 0x7f081c09d6f0 2023-10-03T10:12:09.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.891+0000 7f08127fc700 1 -- 172.21.15.3:0/1975441608 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3951+0+0 (secure 0 0 0) 0x7f081803ada0 con 0x7f081c09d6f0 2023-10-03T10:12:09.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.891+0000 7f08213e1700 1 -- 172.21.15.3:0/1975441608 >> v1:172.21.15.3:6816/118296 conn(0x7f080002c1b0 legacy=0x7f080002e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:09.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.891+0000 7f0823e46700 1 -- 172.21.15.3:0/1975441608 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f081c071700 con 0x7f081c09d6f0 2023-10-03T10:12:09.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.896+0000 7f08127fc700 1 -- 172.21.15.3:0/1975441608 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f0818017070 con 0x7f081c09d6f0 2023-10-03T10:12:09.938 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:09.938+0000 7f2cd33f8240 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-03T10:12:09.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.959+0000 7f9a777fe700 1 -- 172.21.15.3:0/2412386821 >> v1:172.21.15.3:6816/118296 conn(0x7f9a6402c200 legacy=0x7f9a6402e6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:09.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.969+0000 7f9a7dd6a700 1 -- 172.21.15.3:0/2412386821 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9a58005590 con 0x7f9a78134610 2023-10-03T10:12:09.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.969+0000 7f9a5ffff700 1 -- 172.21.15.3:0/2412386821 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9a6c017070 con 0x7f9a78134610 2023-10-03T10:12:09.969 INFO:teuthology.orchestra.run.smithi003.stdout:38654705694 2023-10-03T10:12:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.971+0000 7f9a7dd6a700 1 -- 172.21.15.3:0/2412386821 >> v1:172.21.15.3:6816/118296 conn(0x7f9a6402c200 legacy=0x7f9a6402e6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.971+0000 7f9a7dd6a700 1 -- 172.21.15.3:0/2412386821 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a78134610 msgr2=0x7f9a78123780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.971+0000 7f9a7dd6a700 1 --2- 172.21.15.3:0/2412386821 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a78134610 0x7f9a78123780 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00ebe0 tx=0x7f9a6c00eef0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.971+0000 7f9a7dd6a700 1 -- 172.21.15.3:0/2412386821 shutdown_connections 2023-10-03T10:12:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.971+0000 7f9a7dd6a700 1 --2- 172.21.15.3:0/2412386821 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a78123cc0 0x7f9a781cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.971+0000 7f9a7dd6a700 1 --2- 172.21.15.3:0/2412386821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a7806e570 0x7f9a78123240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.971+0000 7f9a7dd6a700 1 --2- 172.21.15.3:0/2412386821 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a78134610 0x7f9a78123780 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.971+0000 7f9a7dd6a700 1 -- 172.21.15.3:0/2412386821 >> 172.21.15.3:0/2412386821 conn(0x7f9a780a2aa0 msgr2=0x7f9a780a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.972+0000 7f9a7dd6a700 1 -- 172.21.15.3:0/2412386821 shutdown_connections 2023-10-03T10:12:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.972+0000 7f9a7dd6a700 1 -- 172.21.15.3:0/2412386821 wait complete. 2023-10-03T10:12:09.985 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705696 got 38654705694 for osd.1 2023-10-03T10:12:09.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:09.996+0000 7f4e0e596700 1 -- 172.21.15.3:0/3065002237 >> v1:172.21.15.3:6816/118296 conn(0x7f4dec02c1b0 legacy=0x7f4dec02e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:10.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.010+0000 7fc91f7fe700 1 -- 172.21.15.3:0/665214207 >> v1:172.21.15.3:6816/118296 conn(0x7fc90c03d1c0 legacy=0x7fc90c03f670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:10.087 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:10.086+0000 7f2cd33f8240 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-03T10:12:10.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.088+0000 7fc925ab6700 1 -- 172.21.15.3:0/665214207 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc92006b9e0 con 0x7fc92009e930 2023-10-03T10:12:10.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.089+0000 7fc907fff700 1 -- 172.21.15.3:0/665214207 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc9140123d0 con 0x7fc92009e930 2023-10-03T10:12:10.089 INFO:teuthology.orchestra.run.smithi003.stdout:38654705694 2023-10-03T10:12:10.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.091+0000 7fc925ab6700 1 -- 172.21.15.3:0/665214207 >> v1:172.21.15.3:6816/118296 conn(0x7fc90c03d1c0 legacy=0x7fc90c03f670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:10.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.091+0000 7fc925ab6700 1 -- 172.21.15.3:0/665214207 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc92009e930 msgr2=0x7fc92011f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:10.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.091+0000 7fc925ab6700 1 --2- 172.21.15.3:0/665214207 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc92009e930 0x7fc92011f080 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc91400b470 tx=0x7fc91400b780 comp rx=0 tx=0).stop 2023-10-03T10:12:10.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.091+0000 7fc925ab6700 1 -- 172.21.15.3:0/665214207 shutdown_connections 2023-10-03T10:12:10.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.091+0000 7fc925ab6700 1 --2- 172.21.15.3:0/665214207 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9201338a0 0x7fc92011f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:10.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.091+0000 7fc925ab6700 1 --2- 172.21.15.3:0/665214207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc92009de50 0x7fc92011eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:10.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.091+0000 7fc925ab6700 1 --2- 172.21.15.3:0/665214207 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc92009e930 0x7fc92011f080 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:10.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.091+0000 7fc925ab6700 1 -- 172.21.15.3:0/665214207 >> 172.21.15.3:0/665214207 conn(0x7fc9200a2aa0 msgr2=0x7fc92009ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:10.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.091+0000 7fc925ab6700 1 -- 172.21.15.3:0/665214207 shutdown_connections 2023-10-03T10:12:10.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.091+0000 7fc925ab6700 1 -- 172.21.15.3:0/665214207 wait complete. 2023-10-03T10:12:10.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.092+0000 7f08213e1700 1 -- 172.21.15.3:0/1975441608 >> v1:172.21.15.3:6816/118296 conn(0x7f080002c1b0 legacy=0x7f080002e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:10.105 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705696 got 38654705694 for osd.3 2023-10-03T10:12:10.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.114+0000 7f4e107fa700 1 -- 172.21.15.3:0/3065002237 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4e08131250 con 0x7f4e0809e000 2023-10-03T10:12:10.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.115+0000 7f4dfeffd700 1 -- 172.21.15.3:0/3065002237 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4df8017030 con 0x7f4e0809e000 2023-10-03T10:12:10.115 INFO:teuthology.orchestra.run.smithi003.stdout:38654705694 2023-10-03T10:12:10.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.117+0000 7f4e107fa700 1 -- 172.21.15.3:0/3065002237 >> v1:172.21.15.3:6816/118296 conn(0x7f4dec02c1b0 legacy=0x7f4dec02e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:10.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.117+0000 7f4e107fa700 1 -- 172.21.15.3:0/3065002237 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4e0809e000 msgr2=0x7f4e08139eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:10.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.117+0000 7f4e107fa700 1 --2- 172.21.15.3:0/3065002237 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4e0809e000 0x7f4e08139eb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4df800c700 tx=0x7f4df800ca10 comp rx=0 tx=0).stop 2023-10-03T10:12:10.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.117+0000 7f4e107fa700 1 -- 172.21.15.3:0/3065002237 shutdown_connections 2023-10-03T10:12:10.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.117+0000 7f4e107fa700 1 --2- 172.21.15.3:0/3065002237 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4e0809e410 0x7f4e0813a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:10.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.117+0000 7f4e107fa700 1 --2- 172.21.15.3:0/3065002237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e0809d6f0 0x7f4e08139970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:10.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.117+0000 7f4e107fa700 1 --2- 172.21.15.3:0/3065002237 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4e0809e000 0x7f4e08139eb0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:10.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.117+0000 7f4e107fa700 1 -- 172.21.15.3:0/3065002237 >> 172.21.15.3:0/3065002237 conn(0x7f4e08066f50 msgr2=0x7f4e08070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:10.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.117+0000 7f4e107fa700 1 -- 172.21.15.3:0/3065002237 shutdown_connections 2023-10-03T10:12:10.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.117+0000 7f4e107fa700 1 -- 172.21.15.3:0/3065002237 wait complete. 2023-10-03T10:12:10.131 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705695 got 38654705694 for osd.2 2023-10-03T10:12:10.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.142+0000 7f0823e46700 1 -- 172.21.15.3:0/1975441608 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f081c074ea0 con 0x7f081c09d6f0 2023-10-03T10:12:10.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.143+0000 7f08127fc700 1 -- 172.21.15.3:0/1975441608 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0818017030 con 0x7f081c09d6f0 2023-10-03T10:12:10.143 INFO:teuthology.orchestra.run.smithi003.stdout:38654705694 2023-10-03T10:12:10.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.145+0000 7f0823e46700 1 -- 172.21.15.3:0/1975441608 >> v1:172.21.15.3:6816/118296 conn(0x7f080002c1b0 legacy=0x7f080002e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:10.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.145+0000 7f0823e46700 1 -- 172.21.15.3:0/1975441608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f081c09d6f0 msgr2=0x7f081c132890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:10.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.145+0000 7f0823e46700 1 --2- 172.21.15.3:0/1975441608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f081c09d6f0 0x7f081c132890 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f081800c4b0 tx=0x7f081800c870 comp rx=0 tx=0).stop 2023-10-03T10:12:10.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.145+0000 7f0823e46700 1 -- 172.21.15.3:0/1975441608 shutdown_connections 2023-10-03T10:12:10.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.145+0000 7f0823e46700 1 --2- 172.21.15.3:0/1975441608 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f081c135e50 0x7f081c133310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:10.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.145+0000 7f0823e46700 1 --2- 172.21.15.3:0/1975441608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f081c09d6f0 0x7f081c132890 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:10.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.145+0000 7f0823e46700 1 --2- 172.21.15.3:0/1975441608 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f081c09e1d0 0x7f081c132dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:10.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.145+0000 7f0823e46700 1 -- 172.21.15.3:0/1975441608 >> 172.21.15.3:0/1975441608 conn(0x7f081c066f50 msgr2=0x7f081c06e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:10.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.145+0000 7f0823e46700 1 -- 172.21.15.3:0/1975441608 shutdown_connections 2023-10-03T10:12:10.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:10.145+0000 7f0823e46700 1 -- 172.21.15.3:0/1975441608 wait complete. 2023-10-03T10:12:10.158 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705696 got 38654705694 for osd.0 2023-10-03T10:12:10.190 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:10.190+0000 7f2cd33f8240 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-03T10:12:10.309 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:10.309+0000 7f2cd33f8240 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-03T10:12:10.655 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:10.653+0000 7f2cd33f8240 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-03T10:12:10.748 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:10.748+0000 7f2cd33f8240 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-03T10:12:10.926 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:10.926+0000 7f2cd33f8240 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-03T10:12:10.987 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:12:11.009 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:11.009+0000 7f2cd33f8240 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-03T10:12:11.106 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:12:11.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.124+0000 7f6617e95700 1 Processor -- start 2023-10-03T10:12:11.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.124+0000 7f6617e95700 1 -- start start 2023-10-03T10:12:11.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.125+0000 7f6617e95700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f661006b250 0x7f6610136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:11.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.125+0000 7f6617e95700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6610136d80 0x7f6610139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:11.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.125+0000 7f6617e95700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f661009ece0 con 0x7f661006a770 2023-10-03T10:12:11.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.125+0000 7f6617e95700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f661006b640 con 0x7f6610136d80 2023-10-03T10:12:11.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.125+0000 7f6617e95700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661009e760 con 0x7f661006b250 2023-10-03T10:12:11.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.125+0000 7f6615c31700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f661006a770 0x7f661006ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44494/0 (socket says 172.21.15.3:44494) 2023-10-03T10:12:11.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.125+0000 7f6616432700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6610136d80 0x7f6610139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:11.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.125+0000 7f6615c31700 1 -- 172.21.15.3:0/155428655 learned_addr learned my addr 172.21.15.3:0/155428655 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:11.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.125+0000 7f6615430700 1 --2- 172.21.15.3:0/155428655 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f661006b250 0x7f6610136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:11.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.125+0000 7f6616432700 1 -- 172.21.15.3:0/155428655 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f661006b250 msgr2=0x7f6610136840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:11.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.125+0000 7f6616432700 1 --2- 172.21.15.3:0/155428655 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f661006b250 0x7f6610136840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.125+0000 7f6616432700 1 -- 172.21.15.3:0/155428655 >> v1:172.21.15.3:6789/0 conn(0x7f661006a770 legacy=0x7f661006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:11.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.126+0000 7f6616432700 1 -- 172.21.15.3:0/155428655 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661009e040 con 0x7f6610136d80 2023-10-03T10:12:11.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.126+0000 7f6615430700 1 --2- 172.21.15.3:0/155428655 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f661006b250 0x7f6610136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:11.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.126+0000 7f6616432700 1 --2- 172.21.15.3:0/155428655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6610136d80 0x7f6610139160 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6608009d90 tx=0x7f660800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b72f7b403fd764f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:11.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.126+0000 7f6614c2f700 1 -- 172.21.15.3:0/155428655 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660800e070 con 0x7f6610136d80 2023-10-03T10:12:11.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.126+0000 7f6614c2f700 1 -- 172.21.15.3:0/155428655 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660800bb20 con 0x7f6610136d80 2023-10-03T10:12:11.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.126+0000 7f6614c2f700 1 -- 172.21.15.3:0/155428655 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6608012430 con 0x7f6610136d80 2023-10-03T10:12:11.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.127+0000 7f6617e95700 1 -- 172.21.15.3:0/155428655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6610136d80 msgr2=0x7f6610139160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:11.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.127+0000 7f6617e95700 1 --2- 172.21.15.3:0/155428655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6610136d80 0x7f6610139160 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6608009d90 tx=0x7f660800b330 comp rx=0 tx=0).stop 2023-10-03T10:12:11.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.127+0000 7f6617e95700 1 -- 172.21.15.3:0/155428655 shutdown_connections 2023-10-03T10:12:11.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.127+0000 7f6617e95700 1 --2- 172.21.15.3:0/155428655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6610136d80 0x7f6610139160 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.127+0000 7f6617e95700 1 --2- 172.21.15.3:0/155428655 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f661006b250 0x7f6610136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.127+0000 7f6617e95700 1 -- 172.21.15.3:0/155428655 >> 172.21.15.3:0/155428655 conn(0x7f6610066f50 msgr2=0x7f661009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:11.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.127+0000 7f6617e95700 1 -- 172.21.15.3:0/155428655 shutdown_connections 2023-10-03T10:12:11.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.127+0000 7f6617e95700 1 -- 172.21.15.3:0/155428655 wait complete. 2023-10-03T10:12:11.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.128+0000 7f6617e95700 1 Processor -- start 2023-10-03T10:12:11.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.128+0000 7f6617e95700 1 -- start start 2023-10-03T10:12:11.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.128+0000 7f6617e95700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f661006a770 0x7f661009add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:11.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.128+0000 7f6617e95700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f661006b250 0x7f661009b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:11.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.128+0000 7f6617e95700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6610136d80 0x7f66101c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:11.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.128+0000 7f6617e95700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661009b850 con 0x7f661006a770 2023-10-03T10:12:11.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.128+0000 7f6617e95700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6610122380 con 0x7f661006b250 2023-10-03T10:12:11.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.128+0000 7f6617e95700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6610071e60 con 0x7f6610136d80 2023-10-03T10:12:11.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.129+0000 7f6615430700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f661006b250 0x7f661009b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:11.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.129+0000 7f6615430700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f661006b250 0x7f661009b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52816/0 (socket says 172.21.15.3:52816) 2023-10-03T10:12:11.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.129+0000 7f6615430700 1 -- 172.21.15.3:0/919992779 learned_addr learned my addr 172.21.15.3:0/919992779 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:11.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.129+0000 7f6615430700 1 -- 172.21.15.3:0/919992779 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6610136d80 msgr2=0x7f66101c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:11.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.129+0000 7f6615430700 1 --2- 172.21.15.3:0/919992779 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6610136d80 0x7f66101c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.129+0000 7f6615430700 1 -- 172.21.15.3:0/919992779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f661006a770 msgr2=0x7f661009add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:11.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.129+0000 7f6615430700 1 --2- 172.21.15.3:0/919992779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f661006a770 0x7f661009add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.129+0000 7f6615430700 1 -- 172.21.15.3:0/919992779 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66000094a0 con 0x7f661006b250 2023-10-03T10:12:11.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.129+0000 7f6615430700 1 --2- 172.21.15.3:0/919992779 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f661006b250 0x7f661009b310 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f660000ec10 tx=0x7f6600009ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:11.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.129+0000 7f66067fc700 1 -- 172.21.15.3:0/919992779 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660000c940 con 0x7f661006b250 2023-10-03T10:12:11.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.129+0000 7f66067fc700 1 -- 172.21.15.3:0/919992779 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660000caa0 con 0x7f661006b250 2023-10-03T10:12:11.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.129+0000 7f6617e95700 1 -- 172.21.15.3:0/919992779 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6608009a70 con 0x7f661006b250 2023-10-03T10:12:11.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.129+0000 7f66067fc700 1 -- 172.21.15.3:0/919992779 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660000ce10 con 0x7f661006b250 2023-10-03T10:12:11.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.129+0000 7f6617e95700 1 -- 172.21.15.3:0/919992779 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f661006e8e0 con 0x7f661006b250 2023-10-03T10:12:11.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.130+0000 7f6617e95700 1 -- 172.21.15.3:0/919992779 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6610074ea0 con 0x7f661006b250 2023-10-03T10:12:11.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.131+0000 7f66067fc700 1 -- 172.21.15.3:0/919992779 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f6600013470 con 0x7f661006b250 2023-10-03T10:12:11.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.131+0000 7f66067fc700 1 -- 172.21.15.3:0/919992779 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3951+0+0 (secure 0 0 0) 0x7f660003bdb0 con 0x7f661006b250 2023-10-03T10:12:11.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.131+0000 7f6615c31700 1 -- 172.21.15.3:0/919992779 >> v1:172.21.15.3:6816/118296 conn(0x7f65f402c1b0 legacy=0x7f65f402e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:11.136 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:12:11.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.135+0000 7f66067fc700 1 -- 172.21.15.3:0/919992779 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f660001b3b0 con 0x7f661006b250 2023-10-03T10:12:11.160 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:12:11.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.257+0000 7fcb005c1700 1 Processor -- start 2023-10-03T10:12:11.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.257+0000 7fcb005c1700 1 -- start start 2023-10-03T10:12:11.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.257+0000 7fcb005c1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcaf806e570 0x7fcaf806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:11.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.257+0000 7fcb005c1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcaf809eca0 0x7fcaf806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:11.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.257+0000 7fcb005c1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf8070ee0 con 0x7fcaf8134610 2023-10-03T10:12:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.257+0000 7fcb005c1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf8071480 con 0x7fcaf806e570 2023-10-03T10:12:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.257+0000 7fcb005c1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf806f3c0 con 0x7fcaf809eca0 2023-10-03T10:12:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.258+0000 7fcafdb5c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcaf809eca0 0x7fcaf806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.258+0000 7fcafdb5c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcaf809eca0 0x7fcaf806ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:34156/0 (socket says 172.21.15.3:34156) 2023-10-03T10:12:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.258+0000 7fcafdb5c700 1 -- 172.21.15.3:0/2372844269 learned_addr learned my addr 172.21.15.3:0/2372844269 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.258+0000 7fcafe35d700 1 --2- 172.21.15.3:0/2372844269 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcaf806e570 0x7fcaf806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:11.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.258+0000 7fcafdb5c700 1 -- 172.21.15.3:0/2372844269 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcaf806e570 msgr2=0x7fcaf806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:11.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.258+0000 7fcafdb5c700 1 --2- 172.21.15.3:0/2372844269 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcaf806e570 0x7fcaf806e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.258+0000 7fcafdb5c700 1 -- 172.21.15.3:0/2372844269 >> v1:172.21.15.3:6789/0 conn(0x7fcaf8134610 legacy=0x7fcaf8134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:11.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.258+0000 7fcafdb5c700 1 -- 172.21.15.3:0/2372844269 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf8071b90 con 0x7fcaf809eca0 2023-10-03T10:12:11.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.258+0000 7fcafdb5c700 1 --2- 172.21.15.3:0/2372844269 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcaf809eca0 0x7fcaf806ee80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcae8009920 tx=0x7fcae8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d068aaf9cde416e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:11.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.258+0000 7fcafd35b700 1 -- 172.21.15.3:0/2372844269 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae8004230 con 0x7fcaf809eca0 2023-10-03T10:12:11.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.259+0000 7fcafd35b700 1 -- 172.21.15.3:0/2372844269 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae8004390 con 0x7fcaf809eca0 2023-10-03T10:12:11.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.259+0000 7fcafd35b700 1 -- 172.21.15.3:0/2372844269 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae8009e70 con 0x7fcaf809eca0 2023-10-03T10:12:11.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.259+0000 7fcb005c1700 1 -- 172.21.15.3:0/2372844269 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcaf809eca0 msgr2=0x7fcaf806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:11.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.259+0000 7fcb005c1700 1 --2- 172.21.15.3:0/2372844269 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcaf809eca0 0x7fcaf806ee80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcae8009920 tx=0x7fcae8009c30 comp rx=0 tx=0).stop 2023-10-03T10:12:11.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.259+0000 7fcb005c1700 1 -- 172.21.15.3:0/2372844269 shutdown_connections 2023-10-03T10:12:11.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.259+0000 7fcb005c1700 1 --2- 172.21.15.3:0/2372844269 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcaf806e570 0x7fcaf806e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.259+0000 7fcb005c1700 1 --2- 172.21.15.3:0/2372844269 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcaf809eca0 0x7fcaf806ee80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.259+0000 7fcb005c1700 1 -- 172.21.15.3:0/2372844269 >> 172.21.15.3:0/2372844269 conn(0x7fcaf80a2aa0 msgr2=0x7fcaf80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:11.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.260+0000 7fcb005c1700 1 -- 172.21.15.3:0/2372844269 shutdown_connections 2023-10-03T10:12:11.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.260+0000 7fcb005c1700 1 -- 172.21.15.3:0/2372844269 wait complete. 2023-10-03T10:12:11.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.261+0000 7fcb005c1700 1 Processor -- start 2023-10-03T10:12:11.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.261+0000 7fcb005c1700 1 -- start start 2023-10-03T10:12:11.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.261+0000 7fcb005c1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaf806e570 0x7fcaf813ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:11.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.261+0000 7fcb005c1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcaf8134610 0x7fcaf813f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:11.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.261+0000 7fcb005c1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcaf813f880 0x7fcaf8132e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:11.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.261+0000 7fcb005c1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf813d560 con 0x7fcaf806e570 2023-10-03T10:12:11.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.261+0000 7fcb005c1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf80a2170 con 0x7fcaf813f880 2023-10-03T10:12:11.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.261+0000 7fcb005c1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf813c900 con 0x7fcaf8134610 2023-10-03T10:12:11.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.261+0000 7fcafeb5e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcaf813f880 0x7fcaf8132e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:11.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.261+0000 7fcafeb5e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcaf813f880 0x7fcaf8132e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52830/0 (socket says 172.21.15.3:52830) 2023-10-03T10:12:11.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.261+0000 7fcafeb5e700 1 -- 172.21.15.3:0/181944585 learned_addr learned my addr 172.21.15.3:0/181944585 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:11.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.262+0000 7fcafe35d700 1 --2- 172.21.15.3:0/181944585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaf806e570 0x7fcaf813ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:11.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.262+0000 7fcafdb5c700 1 --2- 172.21.15.3:0/181944585 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcaf8134610 0x7fcaf813f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:11.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.262+0000 7fcafeb5e700 1 -- 172.21.15.3:0/181944585 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcaf8134610 msgr2=0x7fcaf813f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:11.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.262+0000 7fcafeb5e700 1 --2- 172.21.15.3:0/181944585 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcaf8134610 0x7fcaf813f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.262+0000 7fcafeb5e700 1 -- 172.21.15.3:0/181944585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaf806e570 msgr2=0x7fcaf813ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:11.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.262+0000 7fcafeb5e700 1 --2- 172.21.15.3:0/181944585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaf806e570 0x7fcaf813ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.262+0000 7fcafeb5e700 1 -- 172.21.15.3:0/181944585 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae8009550 con 0x7fcaf813f880 2023-10-03T10:12:11.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.262+0000 7fcafeb5e700 1 --2- 172.21.15.3:0/181944585 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcaf813f880 0x7fcaf8132e70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000c7a0 tx=0x7fcaf000cab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:11.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.262+0000 7fcaeeffd700 1 -- 172.21.15.3:0/181944585 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0003e10 con 0x7fcaf813f880 2023-10-03T10:12:11.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.263+0000 7fcaeeffd700 1 -- 172.21.15.3:0/181944585 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf0005070 con 0x7fcaf813f880 2023-10-03T10:12:11.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.263+0000 7fcb005c1700 1 -- 172.21.15.3:0/181944585 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf8070610 con 0x7fcaf813f880 2023-10-03T10:12:11.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.263+0000 7fcaeeffd700 1 -- 172.21.15.3:0/181944585 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0005420 con 0x7fcaf813f880 2023-10-03T10:12:11.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.263+0000 7fcb005c1700 1 -- 172.21.15.3:0/181944585 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf813dc80 con 0x7fcaf813f880 2023-10-03T10:12:11.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.264+0000 7fcaeeffd700 1 -- 172.21.15.3:0/181944585 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fcaf0005580 con 0x7fcaf813f880 2023-10-03T10:12:11.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.264+0000 7fcb005c1700 1 -- 172.21.15.3:0/181944585 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf8074ea0 con 0x7fcaf813f880 2023-10-03T10:12:11.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.265+0000 7fcaeeffd700 1 -- 172.21.15.3:0/181944585 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3951+0+0 (secure 0 0 0) 0x7fcaf003afb0 con 0x7fcaf813f880 2023-10-03T10:12:11.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.265+0000 7fcafe35d700 1 -- 172.21.15.3:0/181944585 >> v1:172.21.15.3:6816/118296 conn(0x7fcadc02c200 legacy=0x7fcadc02e6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:11.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.269+0000 7fcaeeffd700 1 -- 172.21.15.3:0/181944585 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fcaf0005830 con 0x7fcaf813f880 2023-10-03T10:12:11.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.290+0000 7fdf6b3d7700 1 Processor -- start 2023-10-03T10:12:11.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.290+0000 7fdf6b3d7700 1 -- start start 2023-10-03T10:12:11.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.290+0000 7fdf6b3d7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf6409eca0 0x7fdf6413cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:11.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.291+0000 7fdf6b3d7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdf6406f050 0x7fdf6413f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:11.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.291+0000 7fdf6b3d7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf64070ee0 con 0x7fdf6406e570 2023-10-03T10:12:11.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.291+0000 7fdf6b3d7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf64071980 con 0x7fdf6406f050 2023-10-03T10:12:11.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.291+0000 7fdf6b3d7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf64071480 con 0x7fdf6409eca0 2023-10-03T10:12:11.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.291+0000 7fdf68972700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf6409eca0 0x7fdf6413cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:11.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.291+0000 7fdf68972700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf6409eca0 0x7fdf6413cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:34170/0 (socket says 172.21.15.3:34170) 2023-10-03T10:12:11.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.291+0000 7fdf68972700 1 -- 172.21.15.3:0/2477209319 learned_addr learned my addr 172.21.15.3:0/2477209319 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:11.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.291+0000 7fdf68972700 1 -- 172.21.15.3:0/2477209319 >> v2:172.21.15.3:3300/0 conn(0x7fdf6406f050 msgr2=0x7fdf6413f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:11.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.291+0000 7fdf68972700 1 --2- 172.21.15.3:0/2477209319 >> v2:172.21.15.3:3300/0 conn(0x7fdf6406f050 0x7fdf6413f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.291+0000 7fdf68972700 1 -- 172.21.15.3:0/2477209319 >> v1:172.21.15.3:6789/0 conn(0x7fdf6406e570 legacy=0x7fdf6406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:11.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.291+0000 7fdf68972700 1 -- 172.21.15.3:0/2477209319 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf64071b90 con 0x7fdf6409eca0 2023-10-03T10:12:11.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.292+0000 7fdf68972700 1 --2- 172.21.15.3:0/2477209319 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf6409eca0 0x7fdf6413cea0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdf540098f0 tx=0x7fdf54009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eab45978223a32 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:11.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.292+0000 7fdf5bfff700 1 -- 172.21.15.3:0/2477209319 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf54004230 con 0x7fdf6409eca0 2023-10-03T10:12:11.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.292+0000 7fdf5bfff700 1 -- 172.21.15.3:0/2477209319 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf54004390 con 0x7fdf6409eca0 2023-10-03T10:12:11.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.292+0000 7fdf5bfff700 1 -- 172.21.15.3:0/2477209319 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf540036a0 con 0x7fdf6409eca0 2023-10-03T10:12:11.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.293+0000 7fdf6b3d7700 1 -- 172.21.15.3:0/2477209319 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf6409eca0 msgr2=0x7fdf6413cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:11.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.293+0000 7fdf6b3d7700 1 --2- 172.21.15.3:0/2477209319 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf6409eca0 0x7fdf6413cea0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdf540098f0 tx=0x7fdf54009c00 comp rx=0 tx=0).stop 2023-10-03T10:12:11.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.293+0000 7fdf6b3d7700 1 -- 172.21.15.3:0/2477209319 shutdown_connections 2023-10-03T10:12:11.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.293+0000 7fdf6b3d7700 1 --2- 172.21.15.3:0/2477209319 >> v2:172.21.15.3:3300/0 conn(0x7fdf6406f050 0x7fdf6413f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.293+0000 7fdf6b3d7700 1 --2- 172.21.15.3:0/2477209319 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf6409eca0 0x7fdf6413cea0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.293+0000 7fdf6b3d7700 1 -- 172.21.15.3:0/2477209319 >> 172.21.15.3:0/2477209319 conn(0x7fdf640a2aa0 msgr2=0x7fdf640a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:11.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.293+0000 7fdf6b3d7700 1 -- 172.21.15.3:0/2477209319 shutdown_connections 2023-10-03T10:12:11.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.293+0000 7fdf6b3d7700 1 -- 172.21.15.3:0/2477209319 wait complete. 2023-10-03T10:12:11.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.295+0000 7fdf6b3d7700 1 Processor -- start 2023-10-03T10:12:11.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.295+0000 7fdf6b3d7700 1 -- start start 2023-10-03T10:12:11.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.295+0000 7fdf6b3d7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf6406e570 0x7fdf641214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:11.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.295+0000 7fdf6b3d7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf6406f050 0x7fdf64121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:11.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.295+0000 7fdf6b3d7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf64121f60 0x7fdf641cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:11.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.295+0000 7fdf6b3d7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf64123f70 con 0x7fdf64121f60 2023-10-03T10:12:11.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.295+0000 7fdf6b3d7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6413e7c0 con 0x7fdf6406e570 2023-10-03T10:12:11.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.295+0000 7fdf6b3d7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6409a300 con 0x7fdf6406f050 2023-10-03T10:12:11.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.296+0000 7fdf69173700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf6406e570 0x7fdf641214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:11.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.296+0000 7fdf69974700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf64121f60 0x7fdf641cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:11.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.296+0000 7fdf69173700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf6406e570 0x7fdf641214e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52836/0 (socket says 172.21.15.3:52836) 2023-10-03T10:12:11.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.296+0000 7fdf69173700 1 -- 172.21.15.3:0/2832416271 learned_addr learned my addr 172.21.15.3:0/2832416271 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:11.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.296+0000 7fdf68972700 1 --2- 172.21.15.3:0/2832416271 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf6406f050 0x7fdf64121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:11.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.296+0000 7fdf68972700 1 -- 172.21.15.3:0/2832416271 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf6406e570 msgr2=0x7fdf641214e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:11.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.296+0000 7fdf68972700 1 --2- 172.21.15.3:0/2832416271 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf6406e570 0x7fdf641214e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.296+0000 7fdf68972700 1 -- 172.21.15.3:0/2832416271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf64121f60 msgr2=0x7fdf641cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:11.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.296+0000 7fdf68972700 1 --2- 172.21.15.3:0/2832416271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf64121f60 0x7fdf641cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.296+0000 7fdf68972700 1 -- 172.21.15.3:0/2832416271 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf54009550 con 0x7fdf6406f050 2023-10-03T10:12:11.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.297+0000 7fdf68972700 1 --2- 172.21.15.3:0/2832416271 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf6406f050 0x7fdf64121a20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdf54015000 tx=0x7fdf54004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:11.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.297+0000 7fdf59ffb700 1 -- 172.21.15.3:0/2832416271 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf54003e60 con 0x7fdf6406f050 2023-10-03T10:12:11.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.297+0000 7fdf6b3d7700 1 -- 172.21.15.3:0/2832416271 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf641239d0 con 0x7fdf6406f050 2023-10-03T10:12:11.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.297+0000 7fdf59ffb700 1 -- 172.21.15.3:0/2832416271 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5400b660 con 0x7fdf6406f050 2023-10-03T10:12:11.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.297+0000 7fdf59ffb700 1 -- 172.21.15.3:0/2832416271 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5400b9f0 con 0x7fdf6406f050 2023-10-03T10:12:11.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.297+0000 7fdf6b3d7700 1 -- 172.21.15.3:0/2832416271 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf6413de80 con 0x7fdf6406f050 2023-10-03T10:12:11.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.298+0000 7fdf59ffb700 1 -- 172.21.15.3:0/2832416271 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fdf54022070 con 0x7fdf6406f050 2023-10-03T10:12:11.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.299+0000 7fdf59ffb700 1 -- 172.21.15.3:0/2832416271 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3951+0+0 (secure 0 0 0) 0x7fdf5403d410 con 0x7fdf6406f050 2023-10-03T10:12:11.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.299+0000 7fdf69173700 1 -- 172.21.15.3:0/2832416271 >> v1:172.21.15.3:6816/118296 conn(0x7fdf4802c1e0 legacy=0x7fdf4802e690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:11.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.299+0000 7fdf4f7fe700 1 -- 172.21.15.3:0/2832416271 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf3c0052e0 con 0x7fdf6406f050 2023-10-03T10:12:11.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.304+0000 7fdf59ffb700 1 -- 172.21.15.3:0/2832416271 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fdf54025470 con 0x7fdf6406f050 2023-10-03T10:12:11.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.332+0000 7f6615c31700 1 -- 172.21.15.3:0/919992779 >> v1:172.21.15.3:6816/118296 conn(0x7f65f402c1b0 legacy=0x7f65f402e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:11.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.378+0000 7fd2ed85e700 1 Processor -- start 2023-10-03T10:12:11.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.378+0000 7fd2ed85e700 1 -- start start 2023-10-03T10:12:11.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.379+0000 7fd2ed85e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd2e80a0a30 0x7fd2e80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:11.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.379+0000 7fd2ed85e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd2e80a1340 0x7fd2e8133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:11.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.379+0000 7fd2ed85e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2e809edd0 con 0x7fd2e8133fa0 2023-10-03T10:12:11.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.379+0000 7fd2ed85e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2e809e1f0 con 0x7fd2e80a0a30 2023-10-03T10:12:11.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.379+0000 7fd2ed85e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e809dc50 con 0x7fd2e80a1340 2023-10-03T10:12:11.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.380+0000 7fd2e67fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd2e80a1340 0x7fd2e8133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:11.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.380+0000 7fd2e67fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd2e80a1340 0x7fd2e8133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52846/0 (socket says 172.21.15.3:52846) 2023-10-03T10:12:11.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.380+0000 7fd2e67fc700 1 -- 172.21.15.3:0/3487986327 learned_addr learned my addr 172.21.15.3:0/3487986327 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:11.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.380+0000 7fd2e6ffd700 1 --2- 172.21.15.3:0/3487986327 >> v2:172.21.15.3:3300/0 conn(0x7fd2e80a0a30 0x7fd2e80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:11.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.380+0000 7fd2e67fc700 1 -- 172.21.15.3:0/3487986327 >> v2:172.21.15.3:3300/0 conn(0x7fd2e80a0a30 msgr2=0x7fd2e80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:11.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.380+0000 7fd2e67fc700 1 --2- 172.21.15.3:0/3487986327 >> v2:172.21.15.3:3300/0 conn(0x7fd2e80a0a30 0x7fd2e80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.380+0000 7fd2e67fc700 1 -- 172.21.15.3:0/3487986327 >> v1:172.21.15.3:6789/0 conn(0x7fd2e8133fa0 legacy=0x7fd2e813ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:11.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.380+0000 7fd2e5ffb700 1 -- 172.21.15.3:0/3487986327 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1182960283 0 0) 0x7fd2e809edd0 con 0x7fd2e8133fa0 2023-10-03T10:12:11.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.380+0000 7fd2e67fc700 1 -- 172.21.15.3:0/3487986327 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e80a1fe0 con 0x7fd2e80a1340 2023-10-03T10:12:11.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.380+0000 7fd2e67fc700 1 --2- 172.21.15.3:0/3487986327 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd2e80a1340 0x7fd2e8133a60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc0098f0 tx=0x7fd2dc009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=489de9a44c35c67d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:11.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.381+0000 7fd2e5ffb700 1 -- 172.21.15.3:0/3487986327 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc00b8a0 con 0x7fd2e80a1340 2023-10-03T10:12:11.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.381+0000 7fd2e5ffb700 1 -- 172.21.15.3:0/3487986327 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2dc00ba00 con 0x7fd2e80a1340 2023-10-03T10:12:11.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.381+0000 7fd2e5ffb700 1 -- 172.21.15.3:0/3487986327 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc00bd90 con 0x7fd2e80a1340 2023-10-03T10:12:11.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.381+0000 7f6617e95700 1 -- 172.21.15.3:0/919992779 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f661006c360 con 0x7f661006b250 2023-10-03T10:12:11.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.382+0000 7fd2ed85e700 1 -- 172.21.15.3:0/3487986327 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd2e80a1340 msgr2=0x7fd2e8133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:11.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.382+0000 7fd2ed85e700 1 --2- 172.21.15.3:0/3487986327 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd2e80a1340 0x7fd2e8133a60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc0098f0 tx=0x7fd2dc009c00 comp rx=0 tx=0).stop 2023-10-03T10:12:11.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.382+0000 7fd2ed85e700 1 -- 172.21.15.3:0/3487986327 shutdown_connections 2023-10-03T10:12:11.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.382+0000 7fd2ed85e700 1 --2- 172.21.15.3:0/3487986327 >> v2:172.21.15.3:3300/0 conn(0x7fd2e80a0a30 0x7fd2e80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.382+0000 7fd2ed85e700 1 --2- 172.21.15.3:0/3487986327 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd2e80a1340 0x7fd2e8133a60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.382+0000 7fd2ed85e700 1 -- 172.21.15.3:0/3487986327 >> 172.21.15.3:0/3487986327 conn(0x7fd2e80a6550 msgr2=0x7fd2e80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:11.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.382+0000 7fd2ed85e700 1 -- 172.21.15.3:0/3487986327 shutdown_connections 2023-10-03T10:12:11.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.382+0000 7fd2ed85e700 1 -- 172.21.15.3:0/3487986327 wait complete. 2023-10-03T10:12:11.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.383+0000 7f66067fc700 1 -- 172.21.15.3:0/919992779 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6600017020 con 0x7f661006b250 2023-10-03T10:12:11.384 INFO:teuthology.orchestra.run.smithi003.stdout:38654705694 2023-10-03T10:12:11.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.383+0000 7fd2ed85e700 1 Processor -- start 2023-10-03T10:12:11.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.384+0000 7fd2ed85e700 1 -- start start 2023-10-03T10:12:11.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.384+0000 7fd2ed85e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2e80a0a30 0x7fd2e811e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:11.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.384+0000 7fd2ed85e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd2e80a1340 0x7fd2e811ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:11.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.384+0000 7fd2ed85e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd2e8133fa0 0x7fd2e811ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:11.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.384+0000 7fd2ed85e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e8121f60 con 0x7fd2e80a0a30 2023-10-03T10:12:11.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.384+0000 7fd2ed85e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e8120670 con 0x7fd2e8133fa0 2023-10-03T10:12:11.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.384+0000 7fd2ed85e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e80a0220 con 0x7fd2e80a1340 2023-10-03T10:12:11.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.385+0000 7fd2e77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd2e8133fa0 0x7fd2e811ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:11.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.385+0000 7fd2e77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd2e8133fa0 0x7fd2e811ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52848/0 (socket says 172.21.15.3:52848) 2023-10-03T10:12:11.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.385+0000 7fd2e6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2e80a0a30 0x7fd2e811e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:11.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.385+0000 7fd2e77fe700 1 -- 172.21.15.3:0/1517775935 learned_addr learned my addr 172.21.15.3:0/1517775935 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:11.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.385+0000 7fd2e67fc700 1 --2- 172.21.15.3:0/1517775935 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd2e80a1340 0x7fd2e811ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:11.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.385+0000 7fd2e6ffd700 1 -- 172.21.15.3:0/1517775935 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd2e80a1340 msgr2=0x7fd2e811ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:11.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.385+0000 7fd2e6ffd700 1 --2- 172.21.15.3:0/1517775935 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd2e80a1340 0x7fd2e811ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.385+0000 7fd2e6ffd700 1 -- 172.21.15.3:0/1517775935 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd2e8133fa0 msgr2=0x7fd2e811ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:11.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.385+0000 7fd2e6ffd700 1 --2- 172.21.15.3:0/1517775935 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd2e8133fa0 0x7fd2e811ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.385+0000 7fd2e6ffd700 1 -- 172.21.15.3:0/1517775935 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2dc009550 con 0x7fd2e80a0a30 2023-10-03T10:12:11.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.385+0000 7fd2e77fe700 1 --2- 172.21.15.3:0/1517775935 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd2e8133fa0 0x7fd2e811ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:11.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.385+0000 7fd2e6ffd700 1 --2- 172.21.15.3:0/1517775935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2e80a0a30 0x7fd2e811e4d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800e7e0 tx=0x7fd2d800eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:11.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.386+0000 7fd2cffff700 1 -- 172.21.15.3:0/1517775935 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d800c070 con 0x7fd2e80a0a30 2023-10-03T10:12:11.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.386+0000 7fd2ed85e700 1 -- 172.21.15.3:0/1517775935 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e8120050 con 0x7fd2e80a0a30 2023-10-03T10:12:11.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.386+0000 7fd2cffff700 1 -- 172.21.15.3:0/1517775935 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d8013610 con 0x7fd2e80a0a30 2023-10-03T10:12:11.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.386+0000 7fd2cffff700 1 -- 172.21.15.3:0/1517775935 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d80139a0 con 0x7fd2e80a0a30 2023-10-03T10:12:11.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.386+0000 7fd2ed85e700 1 -- 172.21.15.3:0/1517775935 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e8121a20 con 0x7fd2e80a0a30 2023-10-03T10:12:11.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.387+0000 7fd2ed85e700 1 -- 172.21.15.3:0/1517775935 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e8133460 con 0x7fd2e80a0a30 2023-10-03T10:12:11.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.387+0000 7fd2cffff700 1 -- 172.21.15.3:0/1517775935 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fd2d8013b00 con 0x7fd2e80a0a30 2023-10-03T10:12:11.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.387+0000 7fd2e67fc700 1 -- 172.21.15.3:0/1517775935 >> v1:172.21.15.3:6816/118296 conn(0x7fd2d402c1b0 legacy=0x7fd2d402e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:11.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.387+0000 7fd2cffff700 1 -- 172.21.15.3:0/1517775935 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3951+0+0 (secure 0 0 0) 0x7fd2d803b7a0 con 0x7fd2e80a0a30 2023-10-03T10:12:11.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.388+0000 7f6617e95700 1 -- 172.21.15.3:0/919992779 >> v1:172.21.15.3:6816/118296 conn(0x7f65f402c1b0 legacy=0x7f65f402e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:11.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.388+0000 7f6617e95700 1 -- 172.21.15.3:0/919992779 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f661006b250 msgr2=0x7f661009b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:11.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.388+0000 7f6617e95700 1 --2- 172.21.15.3:0/919992779 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f661006b250 0x7f661009b310 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f660000ec10 tx=0x7f6600009ec0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.389+0000 7f6617e95700 1 -- 172.21.15.3:0/919992779 shutdown_connections 2023-10-03T10:12:11.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.389+0000 7f6617e95700 1 --2- 172.21.15.3:0/919992779 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6610136d80 0x7f66101c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.389+0000 7f6617e95700 1 --2- 172.21.15.3:0/919992779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f661006a770 0x7f661009add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.389+0000 7f6617e95700 1 --2- 172.21.15.3:0/919992779 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f661006b250 0x7f661009b310 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.389+0000 7f6617e95700 1 -- 172.21.15.3:0/919992779 >> 172.21.15.3:0/919992779 conn(0x7f6610066f50 msgr2=0x7f6610135e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:11.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.395+0000 7fd2cffff700 1 -- 172.21.15.3:0/1517775935 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fd2d8013e00 con 0x7fd2e80a0a30 2023-10-03T10:12:11.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.395+0000 7f6617e95700 1 -- 172.21.15.3:0/919992779 shutdown_connections 2023-10-03T10:12:11.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.395+0000 7f6617e95700 1 -- 172.21.15.3:0/919992779 wait complete. 2023-10-03T10:12:11.417 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705696 got 38654705694 for osd.1 2023-10-03T10:12:11.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.459+0000 7fcb005c1700 1 -- 172.21.15.3:0/181944585 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcaf8071100 con 0x7fcaf813f880 2023-10-03T10:12:11.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.460+0000 7fcaeeffd700 1 -- 172.21.15.3:0/181944585 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcaf0017070 con 0x7fcaf813f880 2023-10-03T10:12:11.460 INFO:teuthology.orchestra.run.smithi003.stdout:38654705694 2023-10-03T10:12:11.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.462+0000 7fcb005c1700 1 -- 172.21.15.3:0/181944585 >> v1:172.21.15.3:6816/118296 conn(0x7fcadc02c200 legacy=0x7fcadc02e6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:11.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.462+0000 7fcb005c1700 1 -- 172.21.15.3:0/181944585 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcaf813f880 msgr2=0x7fcaf8132e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:11.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.462+0000 7fcb005c1700 1 --2- 172.21.15.3:0/181944585 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcaf813f880 0x7fcaf8132e70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000c7a0 tx=0x7fcaf000cab0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.462+0000 7fcb005c1700 1 -- 172.21.15.3:0/181944585 shutdown_connections 2023-10-03T10:12:11.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.462+0000 7fcb005c1700 1 --2- 172.21.15.3:0/181944585 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcaf813f880 0x7fcaf8132e70 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.462+0000 7fcb005c1700 1 --2- 172.21.15.3:0/181944585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaf806e570 0x7fcaf813ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.462+0000 7fcb005c1700 1 --2- 172.21.15.3:0/181944585 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcaf8134610 0x7fcaf813f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.462+0000 7fcb005c1700 1 -- 172.21.15.3:0/181944585 >> 172.21.15.3:0/181944585 conn(0x7fcaf80a2aa0 msgr2=0x7fcaf8137cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:11.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.463+0000 7fcb005c1700 1 -- 172.21.15.3:0/181944585 shutdown_connections 2023-10-03T10:12:11.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.466+0000 7fcb005c1700 1 -- 172.21.15.3:0/181944585 wait complete. 2023-10-03T10:12:11.481 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705696 got 38654705694 for osd.3 2023-10-03T10:12:11.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.500+0000 7fdf69173700 1 -- 172.21.15.3:0/2832416271 >> v1:172.21.15.3:6816/118296 conn(0x7fdf4802c1e0 legacy=0x7fdf4802e690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:11.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.534+0000 7fdf4f7fe700 1 -- 172.21.15.3:0/2832416271 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fdf3c005590 con 0x7fdf6406f050 2023-10-03T10:12:11.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.534+0000 7fdf59ffb700 1 -- 172.21.15.3:0/2832416271 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdf5402fbe0 con 0x7fdf6406f050 2023-10-03T10:12:11.535 INFO:teuthology.orchestra.run.smithi003.stdout:38654705694 2023-10-03T10:12:11.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.536+0000 7fdf4f7fe700 1 -- 172.21.15.3:0/2832416271 >> v1:172.21.15.3:6816/118296 conn(0x7fdf4802c1e0 legacy=0x7fdf4802e690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:11.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.536+0000 7fdf4f7fe700 1 -- 172.21.15.3:0/2832416271 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf6406f050 msgr2=0x7fdf64121a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:11.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.536+0000 7fdf4f7fe700 1 --2- 172.21.15.3:0/2832416271 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf6406f050 0x7fdf64121a20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdf54015000 tx=0x7fdf54004750 comp rx=0 tx=0).stop 2023-10-03T10:12:11.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.536+0000 7fdf4f7fe700 1 -- 172.21.15.3:0/2832416271 shutdown_connections 2023-10-03T10:12:11.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.536+0000 7fdf4f7fe700 1 --2- 172.21.15.3:0/2832416271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf64121f60 0x7fdf641cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.536+0000 7fdf4f7fe700 1 --2- 172.21.15.3:0/2832416271 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf6406e570 0x7fdf641214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.536+0000 7fdf4f7fe700 1 --2- 172.21.15.3:0/2832416271 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf6406f050 0x7fdf64121a20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.536+0000 7fdf4f7fe700 1 -- 172.21.15.3:0/2832416271 >> 172.21.15.3:0/2832416271 conn(0x7fdf640a2aa0 msgr2=0x7fdf640a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:11.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.536+0000 7fdf4f7fe700 1 -- 172.21.15.3:0/2832416271 shutdown_connections 2023-10-03T10:12:11.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.536+0000 7fdf4f7fe700 1 -- 172.21.15.3:0/2832416271 wait complete. 2023-10-03T10:12:11.551 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705695 got 38654705694 for osd.2 2023-10-03T10:12:11.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.584+0000 7fd2ed85e700 1 -- 172.21.15.3:0/1517775935 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd2e8070950 con 0x7fd2e80a0a30 2023-10-03T10:12:11.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.585+0000 7fd2cffff700 1 -- 172.21.15.3:0/1517775935 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd2d8010540 con 0x7fd2e80a0a30 2023-10-03T10:12:11.585 INFO:teuthology.orchestra.run.smithi003.stdout:38654705694 2023-10-03T10:12:11.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.587+0000 7fd2ed85e700 1 -- 172.21.15.3:0/1517775935 >> v1:172.21.15.3:6816/118296 conn(0x7fd2d402c1b0 legacy=0x7fd2d402e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:11.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.587+0000 7fd2ed85e700 1 -- 172.21.15.3:0/1517775935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2e80a0a30 msgr2=0x7fd2e811e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:11.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.587+0000 7fd2ed85e700 1 --2- 172.21.15.3:0/1517775935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2e80a0a30 0x7fd2e811e4d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800e7e0 tx=0x7fd2d800eaf0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.587+0000 7fd2ed85e700 1 -- 172.21.15.3:0/1517775935 shutdown_connections 2023-10-03T10:12:11.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.587+0000 7fd2ed85e700 1 --2- 172.21.15.3:0/1517775935 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd2e8133fa0 0x7fd2e811ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.587+0000 7fd2ed85e700 1 --2- 172.21.15.3:0/1517775935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2e80a0a30 0x7fd2e811e4d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.587+0000 7fd2ed85e700 1 --2- 172.21.15.3:0/1517775935 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd2e80a1340 0x7fd2e811ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:11.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.587+0000 7fd2ed85e700 1 -- 172.21.15.3:0/1517775935 >> 172.21.15.3:0/1517775935 conn(0x7fd2e80a6550 msgr2=0x7fd2e806da90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:11.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.587+0000 7fd2ed85e700 1 -- 172.21.15.3:0/1517775935 shutdown_connections 2023-10-03T10:12:11.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:11.587+0000 7fd2ed85e700 1 -- 172.21.15.3:0/1517775935 wait complete. 2023-10-03T10:12:11.602 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705696 got 38654705694 for osd.0 2023-10-03T10:12:11.836 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:11.836+0000 7f2cd33f8240 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-03T10:12:11.975 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:11.975+0000 7f2cd33f8240 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-03T10:12:12.107 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:12.106+0000 7f2cd33f8240 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-03T10:12:12.418 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:12:12.482 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:12:12.552 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:12:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.563+0000 7efd300ba700 1 Processor -- start 2023-10-03T10:12:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.563+0000 7efd300ba700 1 -- start start 2023-10-03T10:12:12.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.564+0000 7efd300ba700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd2812cf60 0x7efd2806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:12.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.564+0000 7efd300ba700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd2812f8b0 0x7efd2812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:12.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.564+0000 7efd300ba700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd2812cd90 con 0x7efd28130260 2023-10-03T10:12:12.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.564+0000 7efd300ba700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd2812d330 con 0x7efd2812cf60 2023-10-03T10:12:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.564+0000 7efd300ba700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efd281306d0 con 0x7efd2812f8b0 2023-10-03T10:12:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.564+0000 7efd2d655700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd2812f8b0 0x7efd2812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.564+0000 7efd2e657700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7efd28130260 0x7efd28136c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44530/0 (socket says 172.21.15.3:44530) 2023-10-03T10:12:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.564+0000 7efd2e657700 1 -- 172.21.15.3:0/1765529467 learned_addr learned my addr 172.21.15.3:0/1765529467 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.564+0000 7efd2d655700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd2812f8b0 0x7efd2812fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:34198/0 (socket says 172.21.15.3:34198) 2023-10-03T10:12:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.564+0000 7efd2de56700 1 --2- 172.21.15.3:0/1765529467 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd2812cf60 0x7efd2806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:12.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.564+0000 7efd2ce54700 1 -- 172.21.15.3:0/1765529467 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 907205726 0 0) 0x7efd2812cd90 con 0x7efd28130260 2023-10-03T10:12:12.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.564+0000 7efd2ce54700 1 -- 172.21.15.3:0/1765529467 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd140035b0 con 0x7efd28130260 2023-10-03T10:12:12.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.564+0000 7efd2d655700 1 -- 172.21.15.3:0/1765529467 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd2812cf60 msgr2=0x7efd2806f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:12.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.564+0000 7efd2d655700 1 --2- 172.21.15.3:0/1765529467 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd2812cf60 0x7efd2806f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.564+0000 7efd2d655700 1 -- 172.21.15.3:0/1765529467 >> v1:172.21.15.3:6789/0 conn(0x7efd28130260 legacy=0x7efd28136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:12.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.565+0000 7efd2d655700 1 -- 172.21.15.3:0/1765529467 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd28071780 con 0x7efd2812f8b0 2023-10-03T10:12:12.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.565+0000 7efd2d655700 1 --2- 172.21.15.3:0/1765529467 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd2812f8b0 0x7efd2812fd20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efd18009870 tx=0x7efd18009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a85cd460ab857f48 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:12.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.565+0000 7efd2ce54700 1 -- 172.21.15.3:0/1765529467 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd18011830 con 0x7efd2812f8b0 2023-10-03T10:12:12.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.565+0000 7efd2ce54700 1 -- 172.21.15.3:0/1765529467 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd18011990 con 0x7efd2812f8b0 2023-10-03T10:12:12.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.565+0000 7efd2ce54700 1 -- 172.21.15.3:0/1765529467 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd18011d20 con 0x7efd2812f8b0 2023-10-03T10:12:12.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.566+0000 7efd300ba700 1 -- 172.21.15.3:0/1765529467 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd2812f8b0 msgr2=0x7efd2812fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:12.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.566+0000 7efd300ba700 1 --2- 172.21.15.3:0/1765529467 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd2812f8b0 0x7efd2812fd20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efd18009870 tx=0x7efd18009b80 comp rx=0 tx=0).stop 2023-10-03T10:12:12.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.566+0000 7efd300ba700 1 -- 172.21.15.3:0/1765529467 shutdown_connections 2023-10-03T10:12:12.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.566+0000 7efd300ba700 1 --2- 172.21.15.3:0/1765529467 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd2812cf60 0x7efd2806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.566+0000 7efd300ba700 1 --2- 172.21.15.3:0/1765529467 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd2812f8b0 0x7efd2812fd20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.566+0000 7efd300ba700 1 -- 172.21.15.3:0/1765529467 >> 172.21.15.3:0/1765529467 conn(0x7efd280a2aa0 msgr2=0x7efd280a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:12.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.566+0000 7efd300ba700 1 -- 172.21.15.3:0/1765529467 shutdown_connections 2023-10-03T10:12:12.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.566+0000 7efd300ba700 1 -- 172.21.15.3:0/1765529467 wait complete. 2023-10-03T10:12:12.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.568+0000 7efd300ba700 1 Processor -- start 2023-10-03T10:12:12.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.568+0000 7efd300ba700 1 -- start start 2023-10-03T10:12:12.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.569+0000 7efd300ba700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd2812cf60 0x7efd2809bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:12.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.569+0000 7efd300ba700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd2812f8b0 0x7efd2809c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:12.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.569+0000 7efd300ba700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd28130260 0x7efd281c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:12.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.569+0000 7efd300ba700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd28129fa0 con 0x7efd28130260 2023-10-03T10:12:12.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.569+0000 7efd300ba700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd2809eae0 con 0x7efd2812cf60 2023-10-03T10:12:12.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.569+0000 7efd300ba700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efd2809b6d0 con 0x7efd2812f8b0 2023-10-03T10:12:12.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.569+0000 7efd2de56700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd2812cf60 0x7efd2809bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:12.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.569+0000 7efd2de56700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd2812cf60 0x7efd2809bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52868/0 (socket says 172.21.15.3:52868) 2023-10-03T10:12:12.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.569+0000 7efd2de56700 1 -- 172.21.15.3:0/4250586705 learned_addr learned my addr 172.21.15.3:0/4250586705 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:12.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.569+0000 7efd2d655700 1 --2- 172.21.15.3:0/4250586705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd2812f8b0 0x7efd2809c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:12.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.569+0000 7efd2e657700 1 --2- 172.21.15.3:0/4250586705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd28130260 0x7efd281c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:12.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.569+0000 7efd2e657700 1 -- 172.21.15.3:0/4250586705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd2812f8b0 msgr2=0x7efd2809c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:12.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.569+0000 7efd2e657700 1 --2- 172.21.15.3:0/4250586705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd2812f8b0 0x7efd2809c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.569+0000 7efd2e657700 1 -- 172.21.15.3:0/4250586705 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd2812cf60 msgr2=0x7efd2809bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:12.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.569+0000 7efd2e657700 1 --2- 172.21.15.3:0/4250586705 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd2812cf60 0x7efd2809bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.569+0000 7efd2e657700 1 -- 172.21.15.3:0/4250586705 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd180094a0 con 0x7efd28130260 2023-10-03T10:12:12.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.569+0000 7efd2d655700 1 --2- 172.21.15.3:0/4250586705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd2812f8b0 0x7efd2809c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:12.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.569+0000 7efd2de56700 1 --2- 172.21.15.3:0/4250586705 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd2812cf60 0x7efd2809bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:12.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.570+0000 7efd2e657700 1 --2- 172.21.15.3:0/4250586705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd28130260 0x7efd281c3ee0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efd2000ca50 tx=0x7efd2000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:12.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.570+0000 7efd1e7fc700 1 -- 172.21.15.3:0/4250586705 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2000a9e0 con 0x7efd28130260 2023-10-03T10:12:12.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.570+0000 7efd1e7fc700 1 -- 172.21.15.3:0/4250586705 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2000ab40 con 0x7efd28130260 2023-10-03T10:12:12.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.570+0000 7efd300ba700 1 -- 172.21.15.3:0/4250586705 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd2809cdd0 con 0x7efd28130260 2023-10-03T10:12:12.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.570+0000 7efd1e7fc700 1 -- 172.21.15.3:0/4250586705 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd20012400 con 0x7efd28130260 2023-10-03T10:12:12.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.570+0000 7efd300ba700 1 -- 172.21.15.3:0/4250586705 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd28129b20 con 0x7efd28130260 2023-10-03T10:12:12.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.571+0000 7efd1e7fc700 1 -- 172.21.15.3:0/4250586705 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7efd2001b3b0 con 0x7efd28130260 2023-10-03T10:12:12.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.571+0000 7efd1e7fc700 1 -- 172.21.15.3:0/4250586705 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3951+0+0 (secure 0 0 0) 0x7efd2003b380 con 0x7efd28130260 2023-10-03T10:12:12.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.571+0000 7efd2de56700 1 -- 172.21.15.3:0/4250586705 >> v1:172.21.15.3:6816/118296 conn(0x7efd0c02c1b0 legacy=0x7efd0c02e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:12.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.572+0000 7efd300ba700 1 -- 172.21.15.3:0/4250586705 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd28074ea0 con 0x7efd28130260 2023-10-03T10:12:12.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.577+0000 7efd1e7fc700 1 -- 172.21.15.3:0/4250586705 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7efd20017080 con 0x7efd28130260 2023-10-03T10:12:12.603 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:12:12.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.634+0000 7f8d7cb47700 1 Processor -- start 2023-10-03T10:12:12.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.634+0000 7f8d7cb47700 1 -- start start 2023-10-03T10:12:12.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.635+0000 7f8d7cb47700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d78131810 0x7f8d78131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:12.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.635+0000 7f8d7cb47700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d7809a580 0x7f8d78139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:12.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.635+0000 7f8d7cb47700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d781313c0 con 0x7f8d78099a40 2023-10-03T10:12:12.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.635+0000 7f8d7cb47700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7806e8c0 con 0x7f8d7809a580 2023-10-03T10:12:12.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.635+0000 7f8d7cb47700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8d78071d20 con 0x7f8d78131810 2023-10-03T10:12:12.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.635+0000 7f8d75d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8d78099a40 0x7f8d78099e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44546/0 (socket says 172.21.15.3:44546) 2023-10-03T10:12:12.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.635+0000 7f8d75d9b700 1 -- 172.21.15.3:0/3686755425 learned_addr learned my addr 172.21.15.3:0/3686755425 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:12.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.636+0000 7f8d7659c700 1 --2- 172.21.15.3:0/3686755425 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d78131810 0x7f8d78131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:12.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.636+0000 7f8d7659c700 1 -- 172.21.15.3:0/3686755425 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d7809a580 msgr2=0x7f8d78139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:12.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.636+0000 7f8d7659c700 1 --2- 172.21.15.3:0/3686755425 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d7809a580 0x7f8d78139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.636+0000 7f8d7659c700 1 -- 172.21.15.3:0/3686755425 >> v1:172.21.15.3:6789/0 conn(0x7f8d78099a40 legacy=0x7f8d78099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:12.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.636+0000 7f8d7659c700 1 -- 172.21.15.3:0/3686755425 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d780721a0 con 0x7f8d78131810 2023-10-03T10:12:12.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.636+0000 7f8d7659c700 1 --2- 172.21.15.3:0/3686755425 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d78131810 0x7f8d78131be0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009870 tx=0x7f8d60009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cfe9893e9ad9f3d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:12.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.637+0000 7f8d7559a700 1 -- 172.21.15.3:0/3686755425 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d60004230 con 0x7f8d78131810 2023-10-03T10:12:12.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.637+0000 7f8d7559a700 1 -- 172.21.15.3:0/3686755425 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d60004390 con 0x7f8d78131810 2023-10-03T10:12:12.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.637+0000 7f8d7559a700 1 -- 172.21.15.3:0/3686755425 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d600036a0 con 0x7f8d78131810 2023-10-03T10:12:12.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.638+0000 7f8d7cb47700 1 -- 172.21.15.3:0/3686755425 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d78131810 msgr2=0x7f8d78131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:12.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.638+0000 7f8d7cb47700 1 --2- 172.21.15.3:0/3686755425 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d78131810 0x7f8d78131be0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009870 tx=0x7f8d60009b80 comp rx=0 tx=0).stop 2023-10-03T10:12:12.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.638+0000 7f8d7cb47700 1 -- 172.21.15.3:0/3686755425 shutdown_connections 2023-10-03T10:12:12.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.638+0000 7f8d7cb47700 1 --2- 172.21.15.3:0/3686755425 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d7809a580 0x7f8d78139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.638+0000 7f8d7cb47700 1 --2- 172.21.15.3:0/3686755425 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d78131810 0x7f8d78131be0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.638+0000 7f8d7cb47700 1 -- 172.21.15.3:0/3686755425 >> 172.21.15.3:0/3686755425 conn(0x7f8d780a2aa0 msgr2=0x7f8d780a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:12.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.638+0000 7f8d7cb47700 1 -- 172.21.15.3:0/3686755425 shutdown_connections 2023-10-03T10:12:12.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.638+0000 7f8d7cb47700 1 -- 172.21.15.3:0/3686755425 wait complete. 2023-10-03T10:12:12.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.639+0000 7f8d7cb47700 1 Processor -- start 2023-10-03T10:12:12.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.639+0000 7f8d7cb47700 1 -- start start 2023-10-03T10:12:12.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.640+0000 7f8d7cb47700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d78099a40 0x7f8d7811edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:12.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.640+0000 7f8d7cb47700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d7809a580 0x7f8d7811f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:12.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.640+0000 7f8d7cb47700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d78131810 0x7f8d7806f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:12.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.640+0000 7f8d7cb47700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7811c360 con 0x7f8d7809a580 2023-10-03T10:12:12.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.640+0000 7f8d7cb47700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7811bcf0 con 0x7f8d78131810 2023-10-03T10:12:12.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.640+0000 7f8d7cb47700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7811c9f0 con 0x7f8d78099a40 2023-10-03T10:12:12.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.640+0000 7f8d7659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d78099a40 0x7f8d7811edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:12.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.640+0000 7f8d7659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d78099a40 0x7f8d7811edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:34216/0 (socket says 172.21.15.3:34216) 2023-10-03T10:12:12.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.640+0000 7f8d7659c700 1 -- 172.21.15.3:0/3065171756 learned_addr learned my addr 172.21.15.3:0/3065171756 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:12.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.640+0000 7f8d76d9d700 1 --2- 172.21.15.3:0/3065171756 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d78131810 0x7f8d7806f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:12.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.640+0000 7f8d75d9b700 1 --2- 172.21.15.3:0/3065171756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d7809a580 0x7f8d7811f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:12.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.640+0000 7f8d7659c700 1 -- 172.21.15.3:0/3065171756 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d78131810 msgr2=0x7f8d7806f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:12.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.641+0000 7f8d7659c700 1 --2- 172.21.15.3:0/3065171756 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d78131810 0x7f8d7806f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.641+0000 7f8d7659c700 1 -- 172.21.15.3:0/3065171756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d7809a580 msgr2=0x7f8d7811f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:12.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.641+0000 7f8d7659c700 1 --2- 172.21.15.3:0/3065171756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d7809a580 0x7f8d7811f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.641+0000 7f8d7659c700 1 -- 172.21.15.3:0/3065171756 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d600094a0 con 0x7f8d78099a40 2023-10-03T10:12:12.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.641+0000 7f8d76d9d700 1 --2- 172.21.15.3:0/3065171756 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d78131810 0x7f8d7806f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:12.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.641+0000 7f8d75d9b700 1 --2- 172.21.15.3:0/3065171756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d7809a580 0x7f8d7811f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:12:12.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.641+0000 7f8d7659c700 1 --2- 172.21.15.3:0/3065171756 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d78099a40 0x7f8d7811edb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009840 tx=0x7f8d60004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:12.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.641+0000 7f8d6effd700 1 -- 172.21.15.3:0/3065171756 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d60003e00 con 0x7f8d78099a40 2023-10-03T10:12:12.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.641+0000 7f8d6effd700 1 -- 172.21.15.3:0/3065171756 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6001f070 con 0x7f8d78099a40 2023-10-03T10:12:12.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.641+0000 7f8d7cb47700 1 -- 172.21.15.3:0/3065171756 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d7806eb00 con 0x7f8d78099a40 2023-10-03T10:12:12.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.641+0000 7f8d7cb47700 1 -- 172.21.15.3:0/3065171756 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d7811d720 con 0x7f8d78099a40 2023-10-03T10:12:12.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.642+0000 7f8d6effd700 1 -- 172.21.15.3:0/3065171756 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d60011870 con 0x7f8d78099a40 2023-10-03T10:12:12.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.642+0000 7f8d6effd700 1 -- 172.21.15.3:0/3065171756 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f8d60021030 con 0x7f8d78099a40 2023-10-03T10:12:12.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.643+0000 7f8d7cb47700 1 -- 172.21.15.3:0/3065171756 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d5c0052e0 con 0x7f8d78099a40 2023-10-03T10:12:12.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.643+0000 7f8d75d9b700 1 -- 172.21.15.3:0/3065171756 >> v1:172.21.15.3:6816/118296 conn(0x7f8d5802c1b0 legacy=0x7f8d5802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:12.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.643+0000 7f8d6effd700 1 -- 172.21.15.3:0/3065171756 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3951+0+0 (secure 0 0 0) 0x7f8d600199b0 con 0x7f8d78099a40 2023-10-03T10:12:12.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.651+0000 7f8d6effd700 1 -- 172.21.15.3:0/3065171756 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f8d60042060 con 0x7f8d78099a40 2023-10-03T10:12:12.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.713+0000 7fa87cc65700 1 Processor -- start 2023-10-03T10:12:12.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.713+0000 7fa87cc65700 1 -- start start 2023-10-03T10:12:12.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.714+0000 7fa87cc65700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa87806f5d0 0x7fa87812ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:12.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.714+0000 7fa87cc65700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa878135c20 0x7fa878138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:12.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.714+0000 7fa87cc65700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa87806fa40 con 0x7fa87812cf60 2023-10-03T10:12:12.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.714+0000 7fa87cc65700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa87812cd90 con 0x7fa878135c20 2023-10-03T10:12:12.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.714+0000 7fa87cc65700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa878071780 con 0x7fa87806f5d0 2023-10-03T10:12:12.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.714+0000 7fa876d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa878135c20 0x7fa878138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:12.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.714+0000 7fa876d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa878135c20 0x7fa878138000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49348/0 (socket says 172.21.15.3:49348) 2023-10-03T10:12:12.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.714+0000 7fa876d9d700 1 -- 172.21.15.3:0/3416016613 learned_addr learned my addr 172.21.15.3:0/3416016613 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:12.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.715+0000 7fa875d9b700 1 --2- 172.21.15.3:0/3416016613 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa87806f5d0 0x7fa87812ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:12.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.715+0000 7fa875d9b700 1 -- 172.21.15.3:0/3416016613 >> v2:172.21.15.3:3300/0 conn(0x7fa878135c20 msgr2=0x7fa878138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:12.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.715+0000 7fa875d9b700 1 --2- 172.21.15.3:0/3416016613 >> v2:172.21.15.3:3300/0 conn(0x7fa878135c20 0x7fa878138000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.715+0000 7fa875d9b700 1 -- 172.21.15.3:0/3416016613 >> v1:172.21.15.3:6789/0 conn(0x7fa87812cf60 legacy=0x7fa87812f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:12.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.715+0000 7fa875d9b700 1 -- 172.21.15.3:0/3416016613 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa878071a40 con 0x7fa87806f5d0 2023-10-03T10:12:12.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.715+0000 7fa875d9b700 1 --2- 172.21.15.3:0/3416016613 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa87806f5d0 0x7fa87812ffc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009840 tx=0x7fa86c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8e58590a5b53de3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:12.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.715+0000 7fa87559a700 1 -- 172.21.15.3:0/3416016613 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c010800 con 0x7fa87806f5d0 2023-10-03T10:12:12.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.716+0000 7fa87559a700 1 -- 172.21.15.3:0/3416016613 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86c010960 con 0x7fa87806f5d0 2023-10-03T10:12:12.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.716+0000 7fa87559a700 1 -- 172.21.15.3:0/3416016613 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c010cf0 con 0x7fa87806f5d0 2023-10-03T10:12:12.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.716+0000 7fa87cc65700 1 -- 172.21.15.3:0/3416016613 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa87806f5d0 msgr2=0x7fa87812ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:12.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.716+0000 7fa87cc65700 1 --2- 172.21.15.3:0/3416016613 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa87806f5d0 0x7fa87812ffc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009840 tx=0x7fa86c009b50 comp rx=0 tx=0).stop 2023-10-03T10:12:12.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.716+0000 7fa87cc65700 1 -- 172.21.15.3:0/3416016613 shutdown_connections 2023-10-03T10:12:12.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.716+0000 7fa87cc65700 1 --2- 172.21.15.3:0/3416016613 >> v2:172.21.15.3:3300/0 conn(0x7fa878135c20 0x7fa878138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.716+0000 7fa87cc65700 1 --2- 172.21.15.3:0/3416016613 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa87806f5d0 0x7fa87812ffc0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.716+0000 7fa87cc65700 1 -- 172.21.15.3:0/3416016613 >> 172.21.15.3:0/3416016613 conn(0x7fa8780a2aa0 msgr2=0x7fa8780a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:12.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.717+0000 7fa87cc65700 1 -- 172.21.15.3:0/3416016613 shutdown_connections 2023-10-03T10:12:12.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.717+0000 7fa87cc65700 1 -- 172.21.15.3:0/3416016613 wait complete. 2023-10-03T10:12:12.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.718+0000 7fa87cc65700 1 Processor -- start 2023-10-03T10:12:12.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.718+0000 7fa87cc65700 1 -- start start 2023-10-03T10:12:12.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.718+0000 7fa87cc65700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa87806f5d0 0x7fa87809bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:12.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.718+0000 7fa87cc65700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa87812cf60 0x7fa87809c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:12.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.718+0000 7fa87cc65700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa878135c20 0x7fa87809c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:12.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.718+0000 7fa87cc65700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87809d040 con 0x7fa878135c20 2023-10-03T10:12:12.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.719+0000 7fa87cc65700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa87809ec00 con 0x7fa87806f5d0 2023-10-03T10:12:12.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.719+0000 7fa87cc65700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa87812bff0 con 0x7fa87812cf60 2023-10-03T10:12:12.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.719+0000 7fa875d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa87812cf60 0x7fa87809c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:12.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.719+0000 7fa875d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa87812cf60 0x7fa87809c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:34232/0 (socket says 172.21.15.3:34232) 2023-10-03T10:12:12.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.719+0000 7fa875d9b700 1 -- 172.21.15.3:0/4248239108 learned_addr learned my addr 172.21.15.3:0/4248239108 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:12.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.719+0000 7fa876d9d700 1 --2- 172.21.15.3:0/4248239108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa878135c20 0x7fa87809c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:12.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.719+0000 7fa875d9b700 1 -- 172.21.15.3:0/4248239108 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa87806f5d0 msgr2=0x7fa87809bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:12.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.719+0000 7fa875d9b700 1 --2- 172.21.15.3:0/4248239108 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa87806f5d0 0x7fa87809bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.719+0000 7fa875d9b700 1 -- 172.21.15.3:0/4248239108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa878135c20 msgr2=0x7fa87809c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:12.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.719+0000 7fa875d9b700 1 --2- 172.21.15.3:0/4248239108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa878135c20 0x7fa87809c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.719+0000 7fa875d9b700 1 -- 172.21.15.3:0/4248239108 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c0094a0 con 0x7fa87812cf60 2023-10-03T10:12:12.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.719+0000 7fa875d9b700 1 --2- 172.21.15.3:0/4248239108 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa87812cf60 0x7fa87809c4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa86c0090b0 tx=0x7fa86c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:12.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.720+0000 7fa866ffd700 1 -- 172.21.15.3:0/4248239108 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c003ab0 con 0x7fa87812cf60 2023-10-03T10:12:12.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.720+0000 7fa866ffd700 1 -- 172.21.15.3:0/4248239108 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86c003c30 con 0x7fa87812cf60 2023-10-03T10:12:12.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.720+0000 7fa866ffd700 1 -- 172.21.15.3:0/4248239108 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c004560 con 0x7fa87812cf60 2023-10-03T10:12:12.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.720+0000 7fa87cc65700 1 -- 172.21.15.3:0/4248239108 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa878129fa0 con 0x7fa87812cf60 2023-10-03T10:12:12.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.720+0000 7fa87cc65700 1 -- 172.21.15.3:0/4248239108 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa878135100 con 0x7fa87812cf60 2023-10-03T10:12:12.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.721+0000 7fa866ffd700 1 -- 172.21.15.3:0/4248239108 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fa86c018400 con 0x7fa87812cf60 2023-10-03T10:12:12.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.721+0000 7fa87cc65700 1 -- 172.21.15.3:0/4248239108 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa878074ea0 con 0x7fa87812cf60 2023-10-03T10:12:12.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.721+0000 7fa866ffd700 1 -- 172.21.15.3:0/4248239108 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3951+0+0 (secure 0 0 0) 0x7fa86c03c540 con 0x7fa87812cf60 2023-10-03T10:12:12.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.722+0000 7fa87659c700 1 -- 172.21.15.3:0/4248239108 >> v1:172.21.15.3:6816/118296 conn(0x7fa85802c1b0 legacy=0x7fa85802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:12.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.726+0000 7fa866ffd700 1 -- 172.21.15.3:0/4248239108 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fa86c026020 con 0x7fa87812cf60 2023-10-03T10:12:12.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.753+0000 7f7e7811e700 1 Processor -- start 2023-10-03T10:12:12.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.753+0000 7f7e7811e700 1 -- start start 2023-10-03T10:12:12.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.754+0000 7f7e7811e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e7009d6f0 0x7f7e7009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:12.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.754+0000 7f7e7811e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7e7009e000 0x7f7e701339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:12.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.754+0000 7f7e7811e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e7009b120 con 0x7f7e7009e410 2023-10-03T10:12:12.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.754+0000 7f7e7811e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e7006b460 con 0x7f7e7009e000 2023-10-03T10:12:12.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.754+0000 7f7e7811e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7009a7e0 con 0x7f7e7009d6f0 2023-10-03T10:12:12.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.754+0000 7f7e756b9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7e7009e000 0x7f7e701339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:12.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.754+0000 7f7e756b9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7e7009e000 0x7f7e701339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49370/0 (socket says 172.21.15.3:49370) 2023-10-03T10:12:12.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.754+0000 7f7e75eba700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e7009d6f0 0x7f7e7009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:12.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.754+0000 7f7e756b9700 1 -- 172.21.15.3:0/2516656458 learned_addr learned my addr 172.21.15.3:0/2516656458 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:12.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.755+0000 7f7e756b9700 1 -- 172.21.15.3:0/2516656458 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e7009d6f0 msgr2=0x7f7e7009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:12.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.755+0000 7f7e756b9700 1 --2- 172.21.15.3:0/2516656458 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e7009d6f0 0x7f7e7009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.755+0000 7f7e756b9700 1 -- 172.21.15.3:0/2516656458 >> v1:172.21.15.3:6789/0 conn(0x7f7e7009e410 legacy=0x7f7e70133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:12.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.755+0000 7f7e756b9700 1 -- 172.21.15.3:0/2516656458 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7009a500 con 0x7f7e7009e000 2023-10-03T10:12:12.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.755+0000 7f7e756b9700 1 --2- 172.21.15.3:0/2516656458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e7009e000 0x7f7e701339f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7e600098f0 tx=0x7f7e60009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214d3994a4ca47e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:12.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.755+0000 7f7e74eb8700 1 -- 172.21.15.3:0/2516656458 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e60011830 con 0x7f7e7009e000 2023-10-03T10:12:12.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.755+0000 7f7e74eb8700 1 -- 172.21.15.3:0/2516656458 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e60011990 con 0x7f7e7009e000 2023-10-03T10:12:12.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.755+0000 7f7e74eb8700 1 -- 172.21.15.3:0/2516656458 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e60011d20 con 0x7f7e7009e000 2023-10-03T10:12:12.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.756+0000 7f7e7811e700 1 -- 172.21.15.3:0/2516656458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e7009e000 msgr2=0x7f7e701339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:12.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.756+0000 7f7e7811e700 1 --2- 172.21.15.3:0/2516656458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e7009e000 0x7f7e701339f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7e600098f0 tx=0x7f7e60009c00 comp rx=0 tx=0).stop 2023-10-03T10:12:12.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.756+0000 7f7e7811e700 1 -- 172.21.15.3:0/2516656458 shutdown_connections 2023-10-03T10:12:12.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.756+0000 7f7e7811e700 1 --2- 172.21.15.3:0/2516656458 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e7009d6f0 0x7f7e7009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.756+0000 7f7e7811e700 1 --2- 172.21.15.3:0/2516656458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e7009e000 0x7f7e701339f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.756+0000 7f7e7811e700 1 -- 172.21.15.3:0/2516656458 >> 172.21.15.3:0/2516656458 conn(0x7f7e70066f50 msgr2=0x7f7e7009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:12.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.756+0000 7f7e7811e700 1 -- 172.21.15.3:0/2516656458 shutdown_connections 2023-10-03T10:12:12.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.756+0000 7f7e7811e700 1 -- 172.21.15.3:0/2516656458 wait complete. 2023-10-03T10:12:12.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.757+0000 7f7e7811e700 1 Processor -- start 2023-10-03T10:12:12.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.757+0000 7f7e7811e700 1 -- start start 2023-10-03T10:12:12.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.758+0000 7f7e7811e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e7009d6f0 0x7f7e701abb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:12.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.758+0000 7f7e7811e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e7009e000 0x7f7e701ac040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:12.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.758+0000 7f7e7811e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e7009e410 0x7f7e701ac580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:12.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.758+0000 7f7e7811e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e701af2b0 con 0x7f7e7009e000 2023-10-03T10:12:12.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.758+0000 7f7e7811e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e701af590 con 0x7f7e7009e410 2023-10-03T10:12:12.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.758+0000 7f7e7811e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e700a2fd0 con 0x7f7e7009d6f0 2023-10-03T10:12:12.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.758+0000 7f7e756b9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e7009e000 0x7f7e701ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:12.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.758+0000 7f7e756b9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e7009e000 0x7f7e701ac040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49376/0 (socket says 172.21.15.3:49376) 2023-10-03T10:12:12.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.758+0000 7f7e756b9700 1 -- 172.21.15.3:0/3150833699 learned_addr learned my addr 172.21.15.3:0/3150833699 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:12.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.758+0000 7f7e75eba700 1 --2- 172.21.15.3:0/3150833699 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e7009d6f0 0x7f7e701abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:12.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.758+0000 7f7e756b9700 1 -- 172.21.15.3:0/3150833699 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e7009d6f0 msgr2=0x7f7e701abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:12.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.758+0000 7f7e756b9700 1 --2- 172.21.15.3:0/3150833699 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e7009d6f0 0x7f7e701abb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.758+0000 7f7e756b9700 1 -- 172.21.15.3:0/3150833699 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e7009e410 msgr2=0x7f7e701ac580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:12.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.758+0000 7f7e756b9700 1 --2- 172.21.15.3:0/3150833699 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e7009e410 0x7f7e701ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.758+0000 7f7e756b9700 1 -- 172.21.15.3:0/3150833699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e60009550 con 0x7f7e7009e000 2023-10-03T10:12:12.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.759+0000 7f7e756b9700 1 --2- 172.21.15.3:0/3150833699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e7009e000 0x7f7e701ac040 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009c40 tx=0x7f7e60004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:12.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.759+0000 7f7e667fc700 1 -- 172.21.15.3:0/3150833699 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e600036a0 con 0x7f7e7009e000 2023-10-03T10:12:12.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.759+0000 7f7e7811e700 1 -- 172.21.15.3:0/3150833699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e7009c770 con 0x7f7e7009e000 2023-10-03T10:12:12.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.759+0000 7f7e7811e700 1 -- 172.21.15.3:0/3150833699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e701ade30 con 0x7f7e7009e000 2023-10-03T10:12:12.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.759+0000 7f7e667fc700 1 -- 172.21.15.3:0/3150833699 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6001f070 con 0x7f7e7009e000 2023-10-03T10:12:12.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.759+0000 7f7e667fc700 1 -- 172.21.15.3:0/3150833699 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e60006d00 con 0x7f7e7009e000 2023-10-03T10:12:12.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.760+0000 7f7e667fc700 1 -- 172.21.15.3:0/3150833699 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f7e60003a20 con 0x7f7e7009e000 2023-10-03T10:12:12.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.760+0000 7f7e7811e700 1 -- 172.21.15.3:0/3150833699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e580052e0 con 0x7f7e7009e000 2023-10-03T10:12:12.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.760+0000 7f7e667fc700 1 -- 172.21.15.3:0/3150833699 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3951+0+0 (secure 0 0 0) 0x7f7e60021070 con 0x7f7e7009e000 2023-10-03T10:12:12.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.760+0000 7f7e75eba700 1 -- 172.21.15.3:0/3150833699 >> v1:172.21.15.3:6816/118296 conn(0x7f7e5402c1b0 legacy=0x7f7e5402e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:12.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.765+0000 7f7e667fc700 1 -- 172.21.15.3:0/3150833699 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f7e6002a080 con 0x7f7e7009e000 2023-10-03T10:12:12.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.772+0000 7efd2de56700 1 -- 172.21.15.3:0/4250586705 >> v1:172.21.15.3:6816/118296 conn(0x7efd0c02c1b0 legacy=0x7efd0c02e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:12.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.844+0000 7f8d75d9b700 1 -- 172.21.15.3:0/3065171756 >> v1:172.21.15.3:6816/118296 conn(0x7f8d5802c1b0 legacy=0x7f8d5802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:12.854 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:12.854+0000 7f2cd33f8240 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-03T10:12:12.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.903+0000 7efd300ba700 1 -- 172.21.15.3:0/4250586705 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7efd280713a0 con 0x7efd28130260 2023-10-03T10:12:12.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.903+0000 7efd1e7fc700 1 -- 172.21.15.3:0/4250586705 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efd200192c0 con 0x7efd28130260 2023-10-03T10:12:12.904 INFO:teuthology.orchestra.run.smithi003.stdout:38654705694 2023-10-03T10:12:12.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.906+0000 7efd300ba700 1 -- 172.21.15.3:0/4250586705 >> v1:172.21.15.3:6816/118296 conn(0x7efd0c02c1b0 legacy=0x7efd0c02e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:12.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.906+0000 7efd300ba700 1 -- 172.21.15.3:0/4250586705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd28130260 msgr2=0x7efd281c3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:12.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.906+0000 7efd300ba700 1 --2- 172.21.15.3:0/4250586705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd28130260 0x7efd281c3ee0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efd2000ca50 tx=0x7efd2000cd60 comp rx=0 tx=0).stop 2023-10-03T10:12:12.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.907+0000 7efd300ba700 1 -- 172.21.15.3:0/4250586705 shutdown_connections 2023-10-03T10:12:12.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.907+0000 7efd300ba700 1 --2- 172.21.15.3:0/4250586705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd28130260 0x7efd281c3ee0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.907+0000 7efd300ba700 1 --2- 172.21.15.3:0/4250586705 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd2812cf60 0x7efd2809bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.907+0000 7efd300ba700 1 --2- 172.21.15.3:0/4250586705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd2812f8b0 0x7efd2809c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.907+0000 7efd300ba700 1 -- 172.21.15.3:0/4250586705 >> 172.21.15.3:0/4250586705 conn(0x7efd280a2aa0 msgr2=0x7efd2812a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:12.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.907+0000 7efd300ba700 1 -- 172.21.15.3:0/4250586705 shutdown_connections 2023-10-03T10:12:12.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.907+0000 7efd300ba700 1 -- 172.21.15.3:0/4250586705 wait complete. 2023-10-03T10:12:12.922 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705696 got 38654705694 for osd.1 2023-10-03T10:12:12.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.923+0000 7fa87659c700 1 -- 172.21.15.3:0/4248239108 >> v1:172.21.15.3:6816/118296 conn(0x7fa85802c1b0 legacy=0x7fa85802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/118296 2023-10-03T10:12:12.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.949+0000 7f7e7811e700 1 -- 172.21.15.3:0/3150833699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7e58005590 con 0x7f7e7009e000 2023-10-03T10:12:12.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.949+0000 7f7e667fc700 1 -- 172.21.15.3:0/3150833699 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7e60015030 con 0x7f7e7009e000 2023-10-03T10:12:12.950 INFO:teuthology.orchestra.run.smithi003.stdout:38654705694 2023-10-03T10:12:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.952+0000 7f7e7811e700 1 -- 172.21.15.3:0/3150833699 >> v1:172.21.15.3:6816/118296 conn(0x7f7e5402c1b0 legacy=0x7f7e5402e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.952+0000 7f7e7811e700 1 -- 172.21.15.3:0/3150833699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e7009e000 msgr2=0x7f7e701ac040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.952+0000 7f7e7811e700 1 --2- 172.21.15.3:0/3150833699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e7009e000 0x7f7e701ac040 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009c40 tx=0x7f7e60004750 comp rx=0 tx=0).stop 2023-10-03T10:12:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.952+0000 7f7e7811e700 1 -- 172.21.15.3:0/3150833699 shutdown_connections 2023-10-03T10:12:12.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.952+0000 7f7e7811e700 1 --2- 172.21.15.3:0/3150833699 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e7009e410 0x7f7e701ac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.952+0000 7f7e7811e700 1 --2- 172.21.15.3:0/3150833699 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e7009d6f0 0x7f7e701abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.952+0000 7f7e7811e700 1 --2- 172.21.15.3:0/3150833699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e7009e000 0x7f7e701ac040 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.952+0000 7f7e7811e700 1 -- 172.21.15.3:0/3150833699 >> 172.21.15.3:0/3150833699 conn(0x7f7e70066f50 msgr2=0x7f7e700711f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:12.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.953+0000 7f7e7811e700 1 -- 172.21.15.3:0/3150833699 shutdown_connections 2023-10-03T10:12:12.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.953+0000 7f7e7811e700 1 -- 172.21.15.3:0/3150833699 wait complete. 2023-10-03T10:12:12.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.962+0000 7f8d7cb47700 1 -- 172.21.15.3:0/3065171756 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8d5c005590 con 0x7f8d78099a40 2023-10-03T10:12:12.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.962+0000 7f8d6effd700 1 -- 172.21.15.3:0/3065171756 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8d6001a460 con 0x7f8d78099a40 2023-10-03T10:12:12.963 INFO:teuthology.orchestra.run.smithi003.stdout:38654705694 2023-10-03T10:12:12.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.964+0000 7f8d7cb47700 1 -- 172.21.15.3:0/3065171756 >> v1:172.21.15.3:6816/118296 conn(0x7f8d5802c1b0 legacy=0x7f8d5802e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:12.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.964+0000 7f8d7cb47700 1 -- 172.21.15.3:0/3065171756 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d78099a40 msgr2=0x7f8d7811edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:12.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.964+0000 7f8d7cb47700 1 --2- 172.21.15.3:0/3065171756 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d78099a40 0x7f8d7811edb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009840 tx=0x7f8d60004750 comp rx=0 tx=0).stop 2023-10-03T10:12:12.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.964+0000 7f8d7cb47700 1 -- 172.21.15.3:0/3065171756 shutdown_connections 2023-10-03T10:12:12.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.964+0000 7f8d7cb47700 1 --2- 172.21.15.3:0/3065171756 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d78131810 0x7f8d7806f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.964+0000 7f8d7cb47700 1 --2- 172.21.15.3:0/3065171756 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d78099a40 0x7f8d7811edb0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.964+0000 7f8d7cb47700 1 --2- 172.21.15.3:0/3065171756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d7809a580 0x7f8d7811f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:12.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.964+0000 7f8d7cb47700 1 -- 172.21.15.3:0/3065171756 >> 172.21.15.3:0/3065171756 conn(0x7f8d780a2aa0 msgr2=0x7f8d781b8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:12.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.964+0000 7f8d7cb47700 1 -- 172.21.15.3:0/3065171756 shutdown_connections 2023-10-03T10:12:12.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:12.965+0000 7f8d7cb47700 1 -- 172.21.15.3:0/3065171756 wait complete. 2023-10-03T10:12:12.967 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705696 got 38654705694 for osd.0 2023-10-03T10:12:12.979 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705696 got 38654705694 for osd.3 2023-10-03T10:12:12.997 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:12.996+0000 7f2cd33f8240 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-03T10:12:13.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:13.027+0000 7fa87cc65700 1 -- 172.21.15.3:0/4248239108 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa8780713a0 con 0x7fa87812cf60 2023-10-03T10:12:13.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:13.028+0000 7fa866ffd700 1 -- 172.21.15.3:0/4248239108 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa86c00e390 con 0x7fa87812cf60 2023-10-03T10:12:13.028 INFO:teuthology.orchestra.run.smithi003.stdout:38654705694 2023-10-03T10:12:13.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:13.030+0000 7fa87cc65700 1 -- 172.21.15.3:0/4248239108 >> v1:172.21.15.3:6816/118296 conn(0x7fa85802c1b0 legacy=0x7fa85802e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:13.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:13.030+0000 7fa87cc65700 1 -- 172.21.15.3:0/4248239108 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa87812cf60 msgr2=0x7fa87809c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:13.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:13.030+0000 7fa87cc65700 1 --2- 172.21.15.3:0/4248239108 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa87812cf60 0x7fa87809c4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa86c0090b0 tx=0x7fa86c004750 comp rx=0 tx=0).stop 2023-10-03T10:12:13.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:13.030+0000 7fa87cc65700 1 -- 172.21.15.3:0/4248239108 shutdown_connections 2023-10-03T10:12:13.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:13.030+0000 7fa87cc65700 1 --2- 172.21.15.3:0/4248239108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa878135c20 0x7fa87809c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:13.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:13.030+0000 7fa87cc65700 1 --2- 172.21.15.3:0/4248239108 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa87806f5d0 0x7fa87809bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:13.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:13.030+0000 7fa87cc65700 1 --2- 172.21.15.3:0/4248239108 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa87812cf60 0x7fa87809c4a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:13.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:13.030+0000 7fa87cc65700 1 -- 172.21.15.3:0/4248239108 >> 172.21.15.3:0/4248239108 conn(0x7fa8780a2aa0 msgr2=0x7fa87806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:13.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:13.030+0000 7fa87cc65700 1 -- 172.21.15.3:0/4248239108 shutdown_connections 2023-10-03T10:12:13.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:13.030+0000 7fa87cc65700 1 -- 172.21.15.3:0/4248239108 wait complete. 2023-10-03T10:12:13.043 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705695 got 38654705694 for osd.2 2023-10-03T10:12:13.173 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:13.173+0000 7f2cd33f8240 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-03T10:12:13.291 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:13.291+0000 7f2cd33f8240 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-03T10:12:13.374 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:13.372+0000 7f2c4f28b700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-03T10:12:13.374 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:13.372+0000 7f2c4f28b700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-03T10:12:13.374 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:13.372+0000 7f2c4f28b700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-03T10:12:13.374 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:13.372+0000 7f2c4f28b700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-03T10:12:13.374 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:13.372+0000 7f2c4f28b700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-03T10:12:13.375 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:13.373+0000 7f2c52291700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-03T10:12:13.375 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:13.373+0000 7f2c52291700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-03T10:12:13.375 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:13.373+0000 7f2c52291700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-03T10:12:13.375 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:13.373+0000 7f2c52291700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-03T10:12:13.375 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:13.373+0000 7f2c52291700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-03T10:12:13.924 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:12:13.968 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:12:13.980 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:12:14.044 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:12:14.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.073+0000 7f38246c3700 1 Processor -- start 2023-10-03T10:12:14.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.073+0000 7f38246c3700 1 -- start start 2023-10-03T10:12:14.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.074+0000 7f38246c3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f381c09d9f0 0x7f381c136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:14.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.074+0000 7f38246c3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f381c136a70 0x7f381c138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:14.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.074+0000 7f38246c3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f381c09e130 con 0x7f381c1316c0 2023-10-03T10:12:14.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.074+0000 7f38246c3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f381c1321a0 con 0x7f381c09d9f0 2023-10-03T10:12:14.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.074+0000 7f38246c3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381c1312b0 con 0x7f381c136a70 2023-10-03T10:12:14.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.074+0000 7f382245f700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f381c1316c0 0x7f381c131a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35618/0 (socket says 172.21.15.3:35618) 2023-10-03T10:12:14.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.074+0000 7f382245f700 1 -- 172.21.15.3:0/1723735957 learned_addr learned my addr 172.21.15.3:0/1723735957 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:14.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.074+0000 7f3821c5e700 1 --2- 172.21.15.3:0/1723735957 >> v2:172.21.15.3:3300/0 conn(0x7f381c09d9f0 0x7f381c136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:14.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.075+0000 7f382145d700 1 -- 172.21.15.3:0/1723735957 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 770624869 0 0) 0x7f3818001110 con 0x7f381c1316c0 2023-10-03T10:12:14.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.075+0000 7f3821c5e700 1 -- 172.21.15.3:0/1723735957 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f381c136a70 msgr2=0x7f381c138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:14.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.075+0000 7f3821c5e700 1 --2- 172.21.15.3:0/1723735957 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f381c136a70 0x7f381c138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.075+0000 7f3821c5e700 1 -- 172.21.15.3:0/1723735957 >> v1:172.21.15.3:6789/0 conn(0x7f381c1316c0 legacy=0x7f381c131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:14.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.075+0000 7f3821c5e700 1 -- 172.21.15.3:0/1723735957 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381c09f6e0 con 0x7f381c09d9f0 2023-10-03T10:12:14.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.075+0000 7f3821c5e700 1 --2- 172.21.15.3:0/1723735957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f381c09d9f0 0x7f381c136530 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f380c009840 tx=0x7f380c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad5eb44393c79ab server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:14.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.075+0000 7f382145d700 1 -- 172.21.15.3:0/1723735957 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c011830 con 0x7f381c09d9f0 2023-10-03T10:12:14.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.075+0000 7f382145d700 1 -- 172.21.15.3:0/1723735957 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380c011990 con 0x7f381c09d9f0 2023-10-03T10:12:14.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.075+0000 7f382145d700 1 -- 172.21.15.3:0/1723735957 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c011d00 con 0x7f381c09d9f0 2023-10-03T10:12:14.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.076+0000 7f38246c3700 1 -- 172.21.15.3:0/1723735957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f381c09d9f0 msgr2=0x7f381c136530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:14.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.076+0000 7f38246c3700 1 --2- 172.21.15.3:0/1723735957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f381c09d9f0 0x7f381c136530 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f380c009840 tx=0x7f380c009b50 comp rx=0 tx=0).stop 2023-10-03T10:12:14.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.076+0000 7f38246c3700 1 -- 172.21.15.3:0/1723735957 shutdown_connections 2023-10-03T10:12:14.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.076+0000 7f38246c3700 1 --2- 172.21.15.3:0/1723735957 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f381c136a70 0x7f381c138e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.076+0000 7f38246c3700 1 --2- 172.21.15.3:0/1723735957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f381c09d9f0 0x7f381c136530 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.076+0000 7f38246c3700 1 -- 172.21.15.3:0/1723735957 >> 172.21.15.3:0/1723735957 conn(0x7f381c066f50 msgr2=0x7f381c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:14.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.076+0000 7f38246c3700 1 -- 172.21.15.3:0/1723735957 shutdown_connections 2023-10-03T10:12:14.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.077+0000 7f38246c3700 1 -- 172.21.15.3:0/1723735957 wait complete. 2023-10-03T10:12:14.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.078+0000 7f38246c3700 1 Processor -- start 2023-10-03T10:12:14.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.078+0000 7f38246c3700 1 -- start start 2023-10-03T10:12:14.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.078+0000 7f38246c3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f381c09d9f0 0x7f381c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:14.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.078+0000 7f38246c3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f381c1316c0 0x7f381c071600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:14.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.078+0000 7f38246c3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f381c136a70 0x7f381c071b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:14.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.078+0000 7f38246c3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c136150 con 0x7f381c1316c0 2023-10-03T10:12:14.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.078+0000 7f38246c3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381c138e60 con 0x7f381c09d9f0 2023-10-03T10:12:14.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.078+0000 7f38246c3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f381c12b900 con 0x7f381c136a70 2023-10-03T10:12:14.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.078+0000 7f3821c5e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f381c1316c0 0x7f381c071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:14.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.078+0000 7f3821c5e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f381c1316c0 0x7f381c071600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44106/0 (socket says 172.21.15.3:44106) 2023-10-03T10:12:14.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.078+0000 7f3821c5e700 1 -- 172.21.15.3:0/1935597836 learned_addr learned my addr 172.21.15.3:0/1935597836 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:14.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.078+0000 7f3822c60700 1 --2- 172.21.15.3:0/1935597836 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f381c136a70 0x7f381c071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:14.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.079+0000 7f382245f700 1 --2- 172.21.15.3:0/1935597836 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f381c09d9f0 0x7f381c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:14.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.079+0000 7f3821c5e700 1 -- 172.21.15.3:0/1935597836 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f381c136a70 msgr2=0x7f381c071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:14.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.079+0000 7f3821c5e700 1 --2- 172.21.15.3:0/1935597836 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f381c136a70 0x7f381c071b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.079+0000 7f3821c5e700 1 -- 172.21.15.3:0/1935597836 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f381c09d9f0 msgr2=0x7f381c0710c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:14.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.079+0000 7f3821c5e700 1 --2- 172.21.15.3:0/1935597836 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f381c09d9f0 0x7f381c0710c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.079+0000 7f3821c5e700 1 -- 172.21.15.3:0/1935597836 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380c0094a0 con 0x7f381c1316c0 2023-10-03T10:12:14.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.079+0000 7f3821c5e700 1 --2- 172.21.15.3:0/1935597836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f381c1316c0 0x7f381c071600 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f380c000c00 tx=0x7f380c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:14.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.079+0000 7f3812ffd700 1 -- 172.21.15.3:0/1935597836 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c0038a0 con 0x7f381c1316c0 2023-10-03T10:12:14.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.079+0000 7f3812ffd700 1 -- 172.21.15.3:0/1935597836 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380c01f070 con 0x7f381c1316c0 2023-10-03T10:12:14.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.079+0000 7f38246c3700 1 -- 172.21.15.3:0/1935597836 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f381c06ff40 con 0x7f381c1316c0 2023-10-03T10:12:14.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.079+0000 7f3812ffd700 1 -- 172.21.15.3:0/1935597836 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c003c40 con 0x7f381c1316c0 2023-10-03T10:12:14.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.079+0000 7f38246c3700 1 -- 172.21.15.3:0/1935597836 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f381c072080 con 0x7f381c1316c0 2023-10-03T10:12:14.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.080+0000 7f3812ffd700 1 -- 172.21.15.3:0/1935597836 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 6) v1 ==== 35253+0+0 (secure 0 0 0) 0x7f380c01a430 con 0x7f381c1316c0 2023-10-03T10:12:14.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.081+0000 7f3812ffd700 1 -- 172.21.15.3:0/1935597836 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4084+0+0 (secure 0 0 0) 0x7f380c021070 con 0x7f381c1316c0 2023-10-03T10:12:14.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.081+0000 7f38246c3700 1 -- 172.21.15.3:0/1935597836 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f381c074ea0 con 0x7f381c1316c0 2023-10-03T10:12:14.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.086+0000 7f3812ffd700 1 -- 172.21.15.3:0/1935597836 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f380c02b070 con 0x7f381c1316c0 2023-10-03T10:12:14.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.123+0000 7f6e6e7ed700 1 Processor -- start 2023-10-03T10:12:14.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.123+0000 7f6e6e7ed700 1 -- start start 2023-10-03T10:12:14.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.124+0000 7f6e6e7ed700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6e680a09c0 0x7f6e680a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:14.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.124+0000 7f6e6e7ed700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6e680a12d0 0x7f6e68134030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:14.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.124+0000 7f6e6e7ed700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e6809ea10 con 0x7f6e6809d5c0 2023-10-03T10:12:14.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.124+0000 7f6e6e7ed700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e6809e210 con 0x7f6e680a12d0 2023-10-03T10:12:14.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.124+0000 7f6e6e7ed700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e680a26f0 con 0x7f6e680a09c0 2023-10-03T10:12:14.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.124+0000 7f6e677fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6e6809d5c0 0x7f6e68133920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35624/0 (socket says 172.21.15.3:35624) 2023-10-03T10:12:14.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.124+0000 7f6e677fe700 1 -- 172.21.15.3:0/1289657763 learned_addr learned my addr 172.21.15.3:0/1289657763 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:14.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.124+0000 7f6e6cd8a700 1 --2- 172.21.15.3:0/1289657763 >> v2:172.21.15.3:3300/0 conn(0x7f6e680a12d0 0x7f6e68134030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:14.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.124+0000 7f6e67fff700 1 --2- 172.21.15.3:0/1289657763 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6e680a09c0 0x7f6e680a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:14.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.125+0000 7f6e6cd8a700 1 -- 172.21.15.3:0/1289657763 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6e680a09c0 msgr2=0x7f6e680a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:14.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.125+0000 7f6e6cd8a700 1 --2- 172.21.15.3:0/1289657763 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6e680a09c0 0x7f6e680a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.125+0000 7f6e6cd8a700 1 -- 172.21.15.3:0/1289657763 >> v1:172.21.15.3:6789/0 conn(0x7f6e6809d5c0 legacy=0x7f6e68133920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:14.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.125+0000 7f6e66ffd700 1 -- 172.21.15.3:0/1289657763 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 863787140 0 0) 0x7f6e6809ea10 con 0x7f6e6809d5c0 2023-10-03T10:12:14.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.125+0000 7f6e6cd8a700 1 -- 172.21.15.3:0/1289657763 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6809db70 con 0x7f6e680a12d0 2023-10-03T10:12:14.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.125+0000 7f6e6cd8a700 1 --2- 172.21.15.3:0/1289657763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e680a12d0 0x7f6e68134030 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009d90 tx=0x7f6e6000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d97dccf63e5bb24 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:14.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.125+0000 7f6e66ffd700 1 -- 172.21.15.3:0/1289657763 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6000e070 con 0x7f6e680a12d0 2023-10-03T10:12:14.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.125+0000 7f6e66ffd700 1 -- 172.21.15.3:0/1289657763 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6000bb20 con 0x7f6e680a12d0 2023-10-03T10:12:14.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.125+0000 7f6e66ffd700 1 -- 172.21.15.3:0/1289657763 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e60012430 con 0x7f6e680a12d0 2023-10-03T10:12:14.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.126+0000 7f6e6e7ed700 1 -- 172.21.15.3:0/1289657763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e680a12d0 msgr2=0x7f6e68134030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:14.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.126+0000 7f6e6e7ed700 1 --2- 172.21.15.3:0/1289657763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e680a12d0 0x7f6e68134030 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009d90 tx=0x7f6e6000b330 comp rx=0 tx=0).stop 2023-10-03T10:12:14.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.126+0000 7f6e6e7ed700 1 -- 172.21.15.3:0/1289657763 shutdown_connections 2023-10-03T10:12:14.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.126+0000 7f6e6e7ed700 1 --2- 172.21.15.3:0/1289657763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e680a12d0 0x7f6e68134030 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.126+0000 7f6e6e7ed700 1 --2- 172.21.15.3:0/1289657763 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6e680a09c0 0x7f6e680a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.126+0000 7f6e6e7ed700 1 -- 172.21.15.3:0/1289657763 >> 172.21.15.3:0/1289657763 conn(0x7f6e680a2fe0 msgr2=0x7f6e680a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:14.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.126+0000 7f6e6e7ed700 1 -- 172.21.15.3:0/1289657763 shutdown_connections 2023-10-03T10:12:14.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.126+0000 7f6e6e7ed700 1 -- 172.21.15.3:0/1289657763 wait complete. 2023-10-03T10:12:14.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.127+0000 7f6e6e7ed700 1 Processor -- start 2023-10-03T10:12:14.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.128+0000 7f6e6e7ed700 1 -- start start 2023-10-03T10:12:14.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.128+0000 7f6e6e7ed700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6e6809d5c0 0x7f6e68122600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:14.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.128+0000 7f6e6e7ed700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e680a09c0 0x7f6e68122b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:14.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.129+0000 7f6e6e7ed700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6e680a12d0 0x7f6e68123080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:14.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.129+0000 7f6e6e7ed700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e68125740 con 0x7f6e680a09c0 2023-10-03T10:12:14.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.129+0000 7f6e6e7ed700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e68125da0 con 0x7f6e6809d5c0 2023-10-03T10:12:14.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.129+0000 7f6e6e7ed700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e68126010 con 0x7f6e680a12d0 2023-10-03T10:12:14.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.129+0000 7f6e677fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e680a09c0 0x7f6e68122b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:14.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.129+0000 7f6e677fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e680a09c0 0x7f6e68122b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44132/0 (socket says 172.21.15.3:44132) 2023-10-03T10:12:14.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.129+0000 7f6e677fe700 1 -- 172.21.15.3:0/2670833031 learned_addr learned my addr 172.21.15.3:0/2670833031 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:14.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.129+0000 7f6e67fff700 1 --2- 172.21.15.3:0/2670833031 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6e6809d5c0 0x7f6e68122600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:14.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.129+0000 7f6e6cd8a700 1 --2- 172.21.15.3:0/2670833031 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6e680a12d0 0x7f6e68123080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:14.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.129+0000 7f6e677fe700 1 -- 172.21.15.3:0/2670833031 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6e680a12d0 msgr2=0x7f6e68123080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:14.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.129+0000 7f6e677fe700 1 --2- 172.21.15.3:0/2670833031 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6e680a12d0 0x7f6e68123080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.129+0000 7f6e677fe700 1 -- 172.21.15.3:0/2670833031 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6e6809d5c0 msgr2=0x7f6e68122600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:14.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.129+0000 7f6e677fe700 1 --2- 172.21.15.3:0/2670833031 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6e6809d5c0 0x7f6e68122600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.129+0000 7f6e677fe700 1 -- 172.21.15.3:0/2670833031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e60009a70 con 0x7f6e680a09c0 2023-10-03T10:12:14.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.129+0000 7f6e677fe700 1 --2- 172.21.15.3:0/2670833031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e680a09c0 0x7f6e68122b40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6e58001eb0 tx=0x7f6e5800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:14.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.130+0000 7f6e64ff9700 1 -- 172.21.15.3:0/2670833031 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e58004c80 con 0x7f6e680a09c0 2023-10-03T10:12:14.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.130+0000 7f6e64ff9700 1 -- 172.21.15.3:0/2670833031 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e58004de0 con 0x7f6e680a09c0 2023-10-03T10:12:14.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.130+0000 7f6e64ff9700 1 -- 172.21.15.3:0/2670833031 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e580072f0 con 0x7f6e680a09c0 2023-10-03T10:12:14.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.130+0000 7f6e6e7ed700 1 -- 172.21.15.3:0/2670833031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e6809cd10 con 0x7f6e680a09c0 2023-10-03T10:12:14.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.130+0000 7f6e6e7ed700 1 -- 172.21.15.3:0/2670833031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e68125a70 con 0x7f6e680a09c0 2023-10-03T10:12:14.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.131+0000 7f6e64ff9700 1 -- 172.21.15.3:0/2670833031 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 6) v1 ==== 35253+0+0 (secure 0 0 0) 0x7f6e58007450 con 0x7f6e680a09c0 2023-10-03T10:12:14.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.131+0000 7f6e64ff9700 1 -- 172.21.15.3:0/2670833031 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4084+0+0 (secure 0 0 0) 0x7f6e5803f240 con 0x7f6e680a09c0 2023-10-03T10:12:14.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.131+0000 7f6e6e7ed700 1 -- 172.21.15.3:0/2670833031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e6809c100 con 0x7f6e680a09c0 2023-10-03T10:12:14.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.140+0000 7f6e64ff9700 1 -- 172.21.15.3:0/2670833031 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f6e58007700 con 0x7f6e680a09c0 2023-10-03T10:12:14.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.173+0000 7fd95d5b7700 1 Processor -- start 2023-10-03T10:12:14.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.174+0000 7fd95d5b7700 1 -- start start 2023-10-03T10:12:14.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.174+0000 7fd95d5b7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd95809d6f0 0x7fd95809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:14.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.174+0000 7fd95d5b7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd95809e000 0x7fd958130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:14.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.174+0000 7fd95d5b7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd95809f130 con 0x7fd95809e410 2023-10-03T10:12:14.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.174+0000 7fd95d5b7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd95809f520 con 0x7fd95809e000 2023-10-03T10:12:14.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.174+0000 7fd95d5b7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd95809e930 con 0x7fd95809d6f0 2023-10-03T10:12:14.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.175+0000 7fd956ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd95809d6f0 0x7fd95809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:14.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.175+0000 7fd9577fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd95809e410 0x7fd958130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35628/0 (socket says 172.21.15.3:35628) 2023-10-03T10:12:14.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.175+0000 7fd9577fe700 1 -- 172.21.15.3:0/2671235521 learned_addr learned my addr 172.21.15.3:0/2671235521 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:14.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.175+0000 7fd9567fc700 1 --2- 172.21.15.3:0/2671235521 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd95809e000 0x7fd958130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:14.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.175+0000 7fd956ffd700 1 -- 172.21.15.3:0/2671235521 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd95809e000 msgr2=0x7fd958130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:14.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.175+0000 7fd956ffd700 1 --2- 172.21.15.3:0/2671235521 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd95809e000 0x7fd958130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.175+0000 7fd956ffd700 1 -- 172.21.15.3:0/2671235521 >> v1:172.21.15.3:6789/0 conn(0x7fd95809e410 legacy=0x7fd958130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:14.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.175+0000 7fd956ffd700 1 -- 172.21.15.3:0/2671235521 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95809b2d0 con 0x7fd95809d6f0 2023-10-03T10:12:14.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.175+0000 7fd955ffb700 1 -- 172.21.15.3:0/2671235521 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 34579202 0 0) 0x7fd95809f130 con 0x7fd95809e410 2023-10-03T10:12:14.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.175+0000 7fd956ffd700 1 --2- 172.21.15.3:0/2671235521 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd95809d6f0 0x7fd95809dac0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd94800b950 tx=0x7fd94800bc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=189d5641209ece56 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:14.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.175+0000 7fd955ffb700 1 -- 172.21.15.3:0/2671235521 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94800ca30 con 0x7fd95809d6f0 2023-10-03T10:12:14.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.176+0000 7fd955ffb700 1 -- 172.21.15.3:0/2671235521 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd94800cb90 con 0x7fd95809d6f0 2023-10-03T10:12:14.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.176+0000 7fd955ffb700 1 -- 172.21.15.3:0/2671235521 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94800f460 con 0x7fd95809d6f0 2023-10-03T10:12:14.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.176+0000 7fd95d5b7700 1 -- 172.21.15.3:0/2671235521 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd95809d6f0 msgr2=0x7fd95809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:14.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.176+0000 7fd95d5b7700 1 --2- 172.21.15.3:0/2671235521 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd95809d6f0 0x7fd95809dac0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd94800b950 tx=0x7fd94800bc60 comp rx=0 tx=0).stop 2023-10-03T10:12:14.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.177+0000 7fd95d5b7700 1 -- 172.21.15.3:0/2671235521 shutdown_connections 2023-10-03T10:12:14.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.177+0000 7fd95d5b7700 1 --2- 172.21.15.3:0/2671235521 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd95809d6f0 0x7fd95809dac0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.177+0000 7fd95d5b7700 1 --2- 172.21.15.3:0/2671235521 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd95809e000 0x7fd958130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.177+0000 7fd95d5b7700 1 -- 172.21.15.3:0/2671235521 >> 172.21.15.3:0/2671235521 conn(0x7fd958066f50 msgr2=0x7fd95809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:14.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.177+0000 7fd95d5b7700 1 -- 172.21.15.3:0/2671235521 shutdown_connections 2023-10-03T10:12:14.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.177+0000 7fd95d5b7700 1 -- 172.21.15.3:0/2671235521 wait complete. 2023-10-03T10:12:14.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.178+0000 7fd95d5b7700 1 Processor -- start 2023-10-03T10:12:14.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.178+0000 7fd95d5b7700 1 -- start start 2023-10-03T10:12:14.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.179+0000 7fd95d5b7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd95809d6f0 0x7fd95811de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:14.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.179+0000 7fd95d5b7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd95809e000 0x7fd95811e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:14.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.179+0000 7fd95d5b7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd95809e410 0x7fd95811e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:14.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.179+0000 7fd95d5b7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95811f2d0 con 0x7fd95809e410 2023-10-03T10:12:14.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.179+0000 7fd95d5b7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd95811f9b0 con 0x7fd95809e000 2023-10-03T10:12:14.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.179+0000 7fd95d5b7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd95811ffd0 con 0x7fd95809d6f0 2023-10-03T10:12:14.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.179+0000 7fd9567fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd95809e000 0x7fd95811e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:14.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.179+0000 7fd9567fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd95809e000 0x7fd95811e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37340/0 (socket says 172.21.15.3:37340) 2023-10-03T10:12:14.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.179+0000 7fd9567fc700 1 -- 172.21.15.3:0/413098339 learned_addr learned my addr 172.21.15.3:0/413098339 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:14.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.180+0000 7fd956ffd700 1 --2- 172.21.15.3:0/413098339 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd95809d6f0 0x7fd95811de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:14.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.180+0000 7fd9577fe700 1 --2- 172.21.15.3:0/413098339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd95809e410 0x7fd95811e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:14.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.180+0000 7fd9567fc700 1 -- 172.21.15.3:0/413098339 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd95809d6f0 msgr2=0x7fd95811de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:14.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.180+0000 7fd9567fc700 1 --2- 172.21.15.3:0/413098339 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd95809d6f0 0x7fd95811de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.180+0000 7fd9567fc700 1 -- 172.21.15.3:0/413098339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd95809e410 msgr2=0x7fd95811e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:14.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.180+0000 7fd9567fc700 1 --2- 172.21.15.3:0/413098339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd95809e410 0x7fd95811e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.180+0000 7fd9567fc700 1 -- 172.21.15.3:0/413098339 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd94800b550 con 0x7fd95809e000 2023-10-03T10:12:14.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.180+0000 7fd9567fc700 1 --2- 172.21.15.3:0/413098339 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd95809e000 0x7fd95811e380 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd94c00dd40 tx=0x7fd94c0103a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:14.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.180+0000 7fd93f7fe700 1 -- 172.21.15.3:0/413098339 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94c00dbb0 con 0x7fd95809e000 2023-10-03T10:12:14.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.180+0000 7fd93f7fe700 1 -- 172.21.15.3:0/413098339 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd94c013040 con 0x7fd95809e000 2023-10-03T10:12:14.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.180+0000 7fd95d5b7700 1 -- 172.21.15.3:0/413098339 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9581215e0 con 0x7fd95809e000 2023-10-03T10:12:14.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.180+0000 7fd93f7fe700 1 -- 172.21.15.3:0/413098339 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94c010d00 con 0x7fd95809e000 2023-10-03T10:12:14.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.181+0000 7fd95d5b7700 1 -- 172.21.15.3:0/413098339 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd95809c700 con 0x7fd95809e000 2023-10-03T10:12:14.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.182+0000 7fd93f7fe700 1 -- 172.21.15.3:0/413098339 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 6) v1 ==== 35253+0+0 (secure 0 0 0) 0x7fd94c018480 con 0x7fd95809e000 2023-10-03T10:12:14.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.182+0000 7fd95d5b7700 1 -- 172.21.15.3:0/413098339 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd958074ea0 con 0x7fd95809e000 2023-10-03T10:12:14.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.182+0000 7fd93f7fe700 1 -- 172.21.15.3:0/413098339 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4084+0+0 (secure 0 0 0) 0x7fd94c043560 con 0x7fd95809e000 2023-10-03T10:12:14.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.187+0000 7fd93f7fe700 1 -- 172.21.15.3:0/413098339 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fd94c010e60 con 0x7fd95809e000 2023-10-03T10:12:14.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.207+0000 7fa3bbcdc700 1 Processor -- start 2023-10-03T10:12:14.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.207+0000 7fa3bbcdc700 1 -- start start 2023-10-03T10:12:14.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.208+0000 7fa3bbcdc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3b412cf60 0x7fa3b406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:14.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.208+0000 7fa3bbcdc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3b412f8b0 0x7fa3b412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:14.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.208+0000 7fa3bbcdc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3b4130260 0x7fa3b4136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:14.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.208+0000 7fa3bbcdc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3b412d330 con 0x7fa3b4130260 2023-10-03T10:12:14.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.208+0000 7fa3bbcdc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b412cd90 con 0x7fa3b412cf60 2023-10-03T10:12:14.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.208+0000 7fa3bbcdc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b41306d0 con 0x7fa3b412f8b0 2023-10-03T10:12:14.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.208+0000 7fa3b9277700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3b412f8b0 0x7fa3b412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:14.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.208+0000 7fa3b9277700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3b412f8b0 0x7fa3b412fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:58968/0 (socket says 172.21.15.3:58968) 2023-10-03T10:12:14.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.208+0000 7fa3ba279700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3b4130260 0x7fa3b4136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:14.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.208+0000 7fa3b9277700 1 -- 172.21.15.3:0/3632536644 learned_addr learned my addr 172.21.15.3:0/3632536644 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:14.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.208+0000 7fa3b9a78700 1 --2- 172.21.15.3:0/3632536644 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3b412cf60 0x7fa3b406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:14.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.208+0000 7fa3b9277700 1 -- 172.21.15.3:0/3632536644 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3b412cf60 msgr2=0x7fa3b406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:14.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.208+0000 7fa3b9277700 1 --2- 172.21.15.3:0/3632536644 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3b412cf60 0x7fa3b406f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.208+0000 7fa3b9277700 1 -- 172.21.15.3:0/3632536644 >> v2:172.21.15.3:3300/0 conn(0x7fa3b4130260 msgr2=0x7fa3b4136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:14.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.208+0000 7fa3b9277700 1 --2- 172.21.15.3:0/3632536644 >> v2:172.21.15.3:3300/0 conn(0x7fa3b4130260 0x7fa3b4136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.208+0000 7fa3b9277700 1 -- 172.21.15.3:0/3632536644 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b4071780 con 0x7fa3b412f8b0 2023-10-03T10:12:14.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.209+0000 7fa3b9277700 1 --2- 172.21.15.3:0/3632536644 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3b412f8b0 0x7fa3b412fd20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400b9a0 tx=0x7fa3a400bcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b935de361a4aa8c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:14.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.209+0000 7fa3b8a76700 1 -- 172.21.15.3:0/3632536644 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a400caa0 con 0x7fa3b412f8b0 2023-10-03T10:12:14.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.209+0000 7fa3b8a76700 1 -- 172.21.15.3:0/3632536644 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a400cc00 con 0x7fa3b412f8b0 2023-10-03T10:12:14.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.209+0000 7fa3b8a76700 1 -- 172.21.15.3:0/3632536644 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a400f460 con 0x7fa3b412f8b0 2023-10-03T10:12:14.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.210+0000 7fa3bbcdc700 1 -- 172.21.15.3:0/3632536644 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3b412f8b0 msgr2=0x7fa3b412fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:14.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.210+0000 7fa3bbcdc700 1 --2- 172.21.15.3:0/3632536644 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3b412f8b0 0x7fa3b412fd20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400b9a0 tx=0x7fa3a400bcb0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.210+0000 7fa3bbcdc700 1 -- 172.21.15.3:0/3632536644 shutdown_connections 2023-10-03T10:12:14.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.210+0000 7fa3bbcdc700 1 --2- 172.21.15.3:0/3632536644 >> v2:172.21.15.3:3300/0 conn(0x7fa3b4130260 0x7fa3b4136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.210+0000 7fa3bbcdc700 1 --2- 172.21.15.3:0/3632536644 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3b412cf60 0x7fa3b406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.210+0000 7fa3bbcdc700 1 --2- 172.21.15.3:0/3632536644 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3b412f8b0 0x7fa3b412fd20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.210+0000 7fa3bbcdc700 1 -- 172.21.15.3:0/3632536644 >> 172.21.15.3:0/3632536644 conn(0x7fa3b40a2aa0 msgr2=0x7fa3b40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:14.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.210+0000 7fa3bbcdc700 1 -- 172.21.15.3:0/3632536644 shutdown_connections 2023-10-03T10:12:14.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.210+0000 7fa3bbcdc700 1 -- 172.21.15.3:0/3632536644 wait complete. 2023-10-03T10:12:14.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.211+0000 7fa3bbcdc700 1 Processor -- start 2023-10-03T10:12:14.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.211+0000 7fa3bbcdc700 1 -- start start 2023-10-03T10:12:14.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.211+0000 7fa3bbcdc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3b412cf60 0x7fa3b409bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:14.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.211+0000 7fa3bbcdc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3b412f8b0 0x7fa3b409c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:14.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.211+0000 7fa3bbcdc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3b4130260 0x7fa3b41c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:14.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.211+0000 7fa3bbcdc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b409e620 con 0x7fa3b4130260 2023-10-03T10:12:14.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.211+0000 7fa3bbcdc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b409da00 con 0x7fa3b412f8b0 2023-10-03T10:12:14.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.212+0000 7fa3bbcdc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b4129a00 con 0x7fa3b412cf60 2023-10-03T10:12:14.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.212+0000 7fa3ba279700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3b4130260 0x7fa3b41c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:14.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.212+0000 7fa3ba279700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3b4130260 0x7fa3b41c3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44154/0 (socket says 172.21.15.3:44154) 2023-10-03T10:12:14.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.212+0000 7fa3ba279700 1 -- 172.21.15.3:0/445862381 learned_addr learned my addr 172.21.15.3:0/445862381 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:14.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.212+0000 7fa3b9277700 1 --2- 172.21.15.3:0/445862381 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3b412f8b0 0x7fa3b409c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:14.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.212+0000 7fa3b9a78700 1 --2- 172.21.15.3:0/445862381 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3b412cf60 0x7fa3b409bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:14.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.212+0000 7fa3ba279700 1 -- 172.21.15.3:0/445862381 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3b412cf60 msgr2=0x7fa3b409bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:14.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.212+0000 7fa3ba279700 1 --2- 172.21.15.3:0/445862381 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3b412cf60 0x7fa3b409bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.212+0000 7fa3ba279700 1 -- 172.21.15.3:0/445862381 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3b412f8b0 msgr2=0x7fa3b409c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:14.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.212+0000 7fa3ba279700 1 --2- 172.21.15.3:0/445862381 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3b412f8b0 0x7fa3b409c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.212+0000 7fa3ba279700 1 -- 172.21.15.3:0/445862381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a400b600 con 0x7fa3b4130260 2023-10-03T10:12:14.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.212+0000 7fa3b9a78700 1 --2- 172.21.15.3:0/445862381 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3b412cf60 0x7fa3b409bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:12:14.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.212+0000 7fa3b9277700 1 --2- 172.21.15.3:0/445862381 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3b412f8b0 0x7fa3b409c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:14.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.212+0000 7fa3ba279700 1 --2- 172.21.15.3:0/445862381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3b4130260 0x7fa3b41c3f20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00eea0 tx=0x7fa3ac013420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:14.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.213+0000 7fa3aa7fc700 1 -- 172.21.15.3:0/445862381 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac00ecc0 con 0x7fa3b4130260 2023-10-03T10:12:14.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.213+0000 7fa3aa7fc700 1 -- 172.21.15.3:0/445862381 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ac014040 con 0x7fa3b4130260 2023-10-03T10:12:14.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.213+0000 7fa3bbcdc700 1 -- 172.21.15.3:0/445862381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b409c610 con 0x7fa3b4130260 2023-10-03T10:12:14.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.213+0000 7fa3aa7fc700 1 -- 172.21.15.3:0/445862381 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac013bb0 con 0x7fa3b4130260 2023-10-03T10:12:14.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.213+0000 7fa3bbcdc700 1 -- 172.21.15.3:0/445862381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b412f500 con 0x7fa3b4130260 2023-10-03T10:12:14.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.214+0000 7fa3bbcdc700 1 -- 172.21.15.3:0/445862381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b409e840 con 0x7fa3b4130260 2023-10-03T10:12:14.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.215+0000 7fa3aa7fc700 1 -- 172.21.15.3:0/445862381 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 6) v1 ==== 35253+0+0 (secure 0 0 0) 0x7fa3ac013d10 con 0x7fa3b4130260 2023-10-03T10:12:14.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.215+0000 7fa3aa7fc700 1 -- 172.21.15.3:0/445862381 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4084+0+0 (secure 0 0 0) 0x7fa3ac044960 con 0x7fa3b4130260 2023-10-03T10:12:14.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.219+0000 7fa3aa7fc700 1 -- 172.21.15.3:0/445862381 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fa3ac00c4f0 con 0x7fa3b4130260 2023-10-03T10:12:14.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.290+0000 7f38246c3700 1 -- 172.21.15.3:0/1935597836 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f381c09e980 con 0x7f381c1316c0 2023-10-03T10:12:14.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.290+0000 7f3812ffd700 1 -- 172.21.15.3:0/1935597836 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f380c003da0 con 0x7f381c1316c0 2023-10-03T10:12:14.291 INFO:teuthology.orchestra.run.smithi003.stdout:38654705694 2023-10-03T10:12:14.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.293+0000 7f38246c3700 1 -- 172.21.15.3:0/1935597836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f381c1316c0 msgr2=0x7f381c071600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:14.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.293+0000 7f38246c3700 1 --2- 172.21.15.3:0/1935597836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f381c1316c0 0x7f381c071600 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f380c000c00 tx=0x7f380c004750 comp rx=0 tx=0).stop 2023-10-03T10:12:14.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.293+0000 7f38246c3700 1 -- 172.21.15.3:0/1935597836 shutdown_connections 2023-10-03T10:12:14.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.293+0000 7f38246c3700 1 --2- 172.21.15.3:0/1935597836 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f381c136a70 0x7f381c071b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.293+0000 7f38246c3700 1 --2- 172.21.15.3:0/1935597836 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f381c09d9f0 0x7f381c0710c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.293+0000 7f38246c3700 1 --2- 172.21.15.3:0/1935597836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f381c1316c0 0x7f381c071600 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.293+0000 7f38246c3700 1 -- 172.21.15.3:0/1935597836 >> 172.21.15.3:0/1935597836 conn(0x7f381c066f50 msgr2=0x7f381c136e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:14.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.293+0000 7f38246c3700 1 -- 172.21.15.3:0/1935597836 shutdown_connections 2023-10-03T10:12:14.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.293+0000 7f38246c3700 1 -- 172.21.15.3:0/1935597836 wait complete. 2023-10-03T10:12:14.312 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705696 got 38654705694 for osd.1 2023-10-03T10:12:14.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.315+0000 7fa3aa7fc700 1 -- 172.21.15.3:0/445862381 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 7) v1 ==== 35345+0+0 (secure 0 0 0) 0x7fa3ac00c910 con 0x7fa3b4130260 2023-10-03T10:12:14.318 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:14.318+0000 7f2c94813700 -1 mgr.server handle_report got status from non-daemon mon.a 2023-10-03T10:12:14.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.318+0000 7f6e64ff9700 1 -- 172.21.15.3:0/2670833031 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 7) v1 ==== 35345+0+0 (secure 0 0 0) 0x7f6e58013bc0 con 0x7f6e680a09c0 2023-10-03T10:12:14.322 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:14.322+0000 7f2c94813700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-03T10:12:14.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.323+0000 7fd93f7fe700 1 -- 172.21.15.3:0/413098339 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mgrmap(e 7) v1 ==== 35345+0+0 (secure 0 0 0) 0x7fd94c00b450 con 0x7fd95809e000 2023-10-03T10:12:14.328 INFO:tasks.ceph.mgr.x.smithi003.stderr:2023-10-03T10:12:14.327+0000 7f2c94813700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-03T10:12:14.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.361+0000 7f6e6e7ed700 1 -- 172.21.15.3:0/2670833031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6e6809cec0 con 0x7f6e680a09c0 2023-10-03T10:12:14.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.362+0000 7f6e64ff9700 1 -- 172.21.15.3:0/2670833031 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6e5801b890 con 0x7f6e680a09c0 2023-10-03T10:12:14.362 INFO:teuthology.orchestra.run.smithi003.stdout:38654705694 2023-10-03T10:12:14.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.364+0000 7f6e6e7ed700 1 -- 172.21.15.3:0/2670833031 >> v1:172.21.15.3:6816/127025 conn(0x7f6e480346f0 legacy=0x7f6e48036ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:14.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.364+0000 7f6e6e7ed700 1 -- 172.21.15.3:0/2670833031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e680a09c0 msgr2=0x7f6e68122b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:14.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.364+0000 7f6e6e7ed700 1 --2- 172.21.15.3:0/2670833031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e680a09c0 0x7f6e68122b40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6e58001eb0 tx=0x7f6e5800c800 comp rx=0 tx=0).stop 2023-10-03T10:12:14.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.364+0000 7f6e6e7ed700 1 -- 172.21.15.3:0/2670833031 shutdown_connections 2023-10-03T10:12:14.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.364+0000 7f6e6e7ed700 1 --2- 172.21.15.3:0/2670833031 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6e680a12d0 0x7f6e68123080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.364+0000 7f6e6e7ed700 1 --2- 172.21.15.3:0/2670833031 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6e6809d5c0 0x7f6e68122600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.364+0000 7f6e6e7ed700 1 --2- 172.21.15.3:0/2670833031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e680a09c0 0x7f6e68122b40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.364+0000 7f6e6e7ed700 1 -- 172.21.15.3:0/2670833031 >> 172.21.15.3:0/2670833031 conn(0x7f6e680a2fe0 msgr2=0x7f6e6811c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:14.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.364+0000 7f6e6e7ed700 1 -- 172.21.15.3:0/2670833031 shutdown_connections 2023-10-03T10:12:14.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.364+0000 7f6e6e7ed700 1 -- 172.21.15.3:0/2670833031 wait complete. 2023-10-03T10:12:14.378 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705696 got 38654705694 for osd.0 2023-10-03T10:12:14.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.387+0000 7fd95d5b7700 1 -- 172.21.15.3:0/413098339 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd95806c360 con 0x7fd95809e000 2023-10-03T10:12:14.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.388+0000 7fd93f7fe700 1 -- 172.21.15.3:0/413098339 <== mon.1 v2:172.21.15.3:3301/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd94c045030 con 0x7fd95809e000 2023-10-03T10:12:14.388 INFO:teuthology.orchestra.run.smithi003.stdout:38654705694 2023-10-03T10:12:14.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.390+0000 7fd95d5b7700 1 -- 172.21.15.3:0/413098339 >> v1:172.21.15.3:6816/127025 conn(0x7fd9440347c0 legacy=0x7fd944036ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:14.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.390+0000 7fd95d5b7700 1 -- 172.21.15.3:0/413098339 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd95809e000 msgr2=0x7fd95811e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:14.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.390+0000 7fd95d5b7700 1 --2- 172.21.15.3:0/413098339 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd95809e000 0x7fd95811e380 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd94c00dd40 tx=0x7fd94c0103a0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.390+0000 7fd95d5b7700 1 -- 172.21.15.3:0/413098339 shutdown_connections 2023-10-03T10:12:14.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.390+0000 7fd95d5b7700 1 --2- 172.21.15.3:0/413098339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd95809e410 0x7fd95811e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.390+0000 7fd95d5b7700 1 --2- 172.21.15.3:0/413098339 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd95809d6f0 0x7fd95811de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.390+0000 7fd95d5b7700 1 --2- 172.21.15.3:0/413098339 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd95809e000 0x7fd95811e380 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.390+0000 7fd95d5b7700 1 -- 172.21.15.3:0/413098339 >> 172.21.15.3:0/413098339 conn(0x7fd958066f50 msgr2=0x7fd958070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:14.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.390+0000 7fd95d5b7700 1 -- 172.21.15.3:0/413098339 shutdown_connections 2023-10-03T10:12:14.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.390+0000 7fd95d5b7700 1 -- 172.21.15.3:0/413098339 wait complete. 2023-10-03T10:12:14.404 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705696 got 38654705694 for osd.3 2023-10-03T10:12:14.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.414+0000 7fa3bbcdc700 1 -- 172.21.15.3:0/445862381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa3b4129fa0 con 0x7fa3b4130260 2023-10-03T10:12:14.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.415+0000 7fa3aa7fc700 1 -- 172.21.15.3:0/445862381 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa3ac018c90 con 0x7fa3b4130260 2023-10-03T10:12:14.415 INFO:teuthology.orchestra.run.smithi003.stdout:38654705694 2023-10-03T10:12:14.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.417+0000 7fa3bbcdc700 1 -- 172.21.15.3:0/445862381 >> v1:172.21.15.3:6816/127025 conn(0x7fa3980347e0 legacy=0x7fa398036bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:14.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.417+0000 7fa3bbcdc700 1 -- 172.21.15.3:0/445862381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3b4130260 msgr2=0x7fa3b41c3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:14.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.417+0000 7fa3bbcdc700 1 --2- 172.21.15.3:0/445862381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3b4130260 0x7fa3b41c3f20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00eea0 tx=0x7fa3ac013420 comp rx=0 tx=0).stop 2023-10-03T10:12:14.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.417+0000 7fa3bbcdc700 1 -- 172.21.15.3:0/445862381 shutdown_connections 2023-10-03T10:12:14.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.417+0000 7fa3bbcdc700 1 --2- 172.21.15.3:0/445862381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3b4130260 0x7fa3b41c3f20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.417+0000 7fa3bbcdc700 1 --2- 172.21.15.3:0/445862381 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3b412cf60 0x7fa3b409bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.417+0000 7fa3bbcdc700 1 --2- 172.21.15.3:0/445862381 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3b412f8b0 0x7fa3b409c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:14.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.417+0000 7fa3bbcdc700 1 -- 172.21.15.3:0/445862381 >> 172.21.15.3:0/445862381 conn(0x7fa3b40a2aa0 msgr2=0x7fa3b406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:14.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.417+0000 7fa3bbcdc700 1 -- 172.21.15.3:0/445862381 shutdown_connections 2023-10-03T10:12:14.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:14.417+0000 7fa3bbcdc700 1 -- 172.21.15.3:0/445862381 wait complete. 2023-10-03T10:12:14.430 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705695 got 38654705694 for osd.2 2023-10-03T10:12:15.313 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:12:15.379 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:12:15.405 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:12:15.431 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:12:15.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.502+0000 7f3a346a9700 1 Processor -- start 2023-10-03T10:12:15.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.502+0000 7f3a346a9700 1 -- start start 2023-10-03T10:12:15.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.503+0000 7f3a346a9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3a2c09c5e0 0x7f3a2c09c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.503+0000 7f3a346a9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3a2c09cef0 0x7f3a2c13bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.503+0000 7f3a346a9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a2c09a830 con 0x7f3a2c13c470 2023-10-03T10:12:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.503+0000 7f3a346a9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a2c09bff0 con 0x7f3a2c09cef0 2023-10-03T10:12:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.503+0000 7f3a346a9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c098ab0 con 0x7f3a2c09c5e0 2023-10-03T10:12:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.504+0000 7f3a31c44700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3a2c09cef0 0x7f3a2c13bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.504+0000 7f3a31c44700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3a2c09cef0 0x7f3a2c13bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44184/0 (socket says 172.21.15.3:44184) 2023-10-03T10:12:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.504+0000 7f3a31c44700 1 -- 172.21.15.3:0/4195176404 learned_addr learned my addr 172.21.15.3:0/4195176404 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.504+0000 7f3a32445700 1 --2- 172.21.15.3:0/4195176404 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3a2c09c5e0 0x7f3a2c09c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:15.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.504+0000 7f3a32445700 1 -- 172.21.15.3:0/4195176404 >> v2:172.21.15.3:3300/0 conn(0x7f3a2c09cef0 msgr2=0x7f3a2c13bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:15.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.504+0000 7f3a32445700 1 --2- 172.21.15.3:0/4195176404 >> v2:172.21.15.3:3300/0 conn(0x7f3a2c09cef0 0x7f3a2c13bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.504+0000 7f3a32445700 1 -- 172.21.15.3:0/4195176404 >> v1:172.21.15.3:6789/0 conn(0x7f3a2c13c470 legacy=0x7f3a2c13e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:15.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.505+0000 7f3a32445700 1 -- 172.21.15.3:0/4195176404 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2c09c1a0 con 0x7f3a2c09c5e0 2023-10-03T10:12:15.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.505+0000 7f3a31443700 1 -- 172.21.15.3:0/4195176404 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2843083358 0 0) 0x7f3a2c09a830 con 0x7f3a2c13c470 2023-10-03T10:12:15.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.505+0000 7f3a32445700 1 --2- 172.21.15.3:0/4195176404 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3a2c09c5e0 0x7f3a2c09c9b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009870 tx=0x7f3a1c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a31ee93ff870c14 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:15.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.506+0000 7f3a31443700 1 -- 172.21.15.3:0/4195176404 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c011830 con 0x7f3a2c09c5e0 2023-10-03T10:12:15.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.506+0000 7f3a31443700 1 -- 172.21.15.3:0/4195176404 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1c011990 con 0x7f3a2c09c5e0 2023-10-03T10:12:15.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.506+0000 7f3a31443700 1 -- 172.21.15.3:0/4195176404 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c011d20 con 0x7f3a2c09c5e0 2023-10-03T10:12:15.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.506+0000 7f3a346a9700 1 -- 172.21.15.3:0/4195176404 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3a2c09c5e0 msgr2=0x7f3a2c09c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:15.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.506+0000 7f3a346a9700 1 --2- 172.21.15.3:0/4195176404 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3a2c09c5e0 0x7f3a2c09c9b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009870 tx=0x7f3a1c009b80 comp rx=0 tx=0).stop 2023-10-03T10:12:15.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.507+0000 7f3a346a9700 1 -- 172.21.15.3:0/4195176404 shutdown_connections 2023-10-03T10:12:15.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.507+0000 7f3a346a9700 1 --2- 172.21.15.3:0/4195176404 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3a2c09c5e0 0x7f3a2c09c9b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.507+0000 7f3a346a9700 1 --2- 172.21.15.3:0/4195176404 >> v2:172.21.15.3:3300/0 conn(0x7f3a2c09cef0 0x7f3a2c13bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.507+0000 7f3a346a9700 1 -- 172.21.15.3:0/4195176404 >> 172.21.15.3:0/4195176404 conn(0x7f3a2c09e7e0 msgr2=0x7f3a2c09ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:15.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.507+0000 7f3a346a9700 1 -- 172.21.15.3:0/4195176404 shutdown_connections 2023-10-03T10:12:15.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.507+0000 7f3a346a9700 1 -- 172.21.15.3:0/4195176404 wait complete. 2023-10-03T10:12:15.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.508+0000 7f3a346a9700 1 Processor -- start 2023-10-03T10:12:15.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.508+0000 7f3a346a9700 1 -- start start 2023-10-03T10:12:15.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.509+0000 7f3a346a9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3a2c09c5e0 0x7f3a2c13b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:15.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.509+0000 7f3a346a9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a2c09cef0 0x7f3a2c13bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:15.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.509+0000 7f3a346a9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3a2c13c470 0x7f3a2c128ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:15.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.509+0000 7f3a346a9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c098130 con 0x7f3a2c09cef0 2023-10-03T10:12:15.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.509+0000 7f3a346a9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c06e5a0 con 0x7f3a2c09c5e0 2023-10-03T10:12:15.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.509+0000 7f3a346a9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c06de30 con 0x7f3a2c13c470 2023-10-03T10:12:15.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.509+0000 7f3a32c46700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3a2c13c470 0x7f3a2c128ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:15.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.509+0000 7f3a32c46700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3a2c13c470 0x7f3a2c128ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:59026/0 (socket says 172.21.15.3:59026) 2023-10-03T10:12:15.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.509+0000 7f3a32c46700 1 -- 172.21.15.3:0/2405576028 learned_addr learned my addr 172.21.15.3:0/2405576028 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:15.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.510+0000 7f3a32c46700 1 -- 172.21.15.3:0/2405576028 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3a2c09c5e0 msgr2=0x7f3a2c13b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:15.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.510+0000 7f3a32c46700 1 --2- 172.21.15.3:0/2405576028 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3a2c09c5e0 0x7f3a2c13b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.510+0000 7f3a32c46700 1 -- 172.21.15.3:0/2405576028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a2c09cef0 msgr2=0x7f3a2c13bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:15.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.510+0000 7f3a32c46700 1 --2- 172.21.15.3:0/2405576028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a2c09cef0 0x7f3a2c13bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.510+0000 7f3a32c46700 1 -- 172.21.15.3:0/2405576028 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1c0094a0 con 0x7f3a2c13c470 2023-10-03T10:12:15.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.510+0000 7f3a32c46700 1 --2- 172.21.15.3:0/2405576028 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3a2c13c470 0x7f3a2c128ff0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3a2800c4a0 tx=0x7f3a2800c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:15.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.510+0000 7f3a22ffd700 1 -- 172.21.15.3:0/2405576028 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a280054e0 con 0x7f3a2c13c470 2023-10-03T10:12:15.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.510+0000 7f3a22ffd700 1 -- 172.21.15.3:0/2405576028 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a28003680 con 0x7f3a2c13c470 2023-10-03T10:12:15.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.510+0000 7f3a22ffd700 1 -- 172.21.15.3:0/2405576028 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a28003a30 con 0x7f3a2c13c470 2023-10-03T10:12:15.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.511+0000 7f3a346a9700 1 -- 172.21.15.3:0/2405576028 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2c12ab40 con 0x7f3a2c13c470 2023-10-03T10:12:15.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.511+0000 7f3a346a9700 1 -- 172.21.15.3:0/2405576028 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2c06eaf0 con 0x7f3a2c13c470 2023-10-03T10:12:15.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.512+0000 7f3a346a9700 1 -- 172.21.15.3:0/2405576028 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a2c0747f0 con 0x7f3a2c13c470 2023-10-03T10:12:15.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.513+0000 7f3a22ffd700 1 -- 172.21.15.3:0/2405576028 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 7) v1 ==== 35345+0+0 (secure 0 0 0) 0x7f3a28003b90 con 0x7f3a2c13c470 2023-10-03T10:12:15.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.513+0000 7f3a22ffd700 1 -- 172.21.15.3:0/2405576028 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4078+0+0 (secure 0 0 0) 0x7f3a2803f160 con 0x7f3a2c13c470 2023-10-03T10:12:15.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.518+0000 7f3a22ffd700 1 -- 172.21.15.3:0/2405576028 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3a28003e40 con 0x7f3a2c13c470 2023-10-03T10:12:15.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.560+0000 7f1fad353700 1 Processor -- start 2023-10-03T10:12:15.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.560+0000 7f1fad353700 1 -- start start 2023-10-03T10:12:15.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.561+0000 7f1fad353700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1fa809eca0 0x7f1fa813cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:15.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.561+0000 7f1fad353700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1fa806f050 0x7f1fa813f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:15.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.561+0000 7f1fad353700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fa8070ee0 con 0x7f1fa806e570 2023-10-03T10:12:15.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.561+0000 7f1fad353700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fa8071980 con 0x7f1fa806f050 2023-10-03T10:12:15.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.561+0000 7f1fad353700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa8071480 con 0x7f1fa809eca0 2023-10-03T10:12:15.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.561+0000 7f1fa77fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1fa806f050 0x7f1fa813f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:15.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.561+0000 7f1fa77fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1fa806f050 0x7f1fa813f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44200/0 (socket says 172.21.15.3:44200) 2023-10-03T10:12:15.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.561+0000 7f1fa77fe700 1 -- 172.21.15.3:0/2938487538 learned_addr learned my addr 172.21.15.3:0/2938487538 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:15.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.561+0000 7f1fa67fc700 1 --2- 172.21.15.3:0/2938487538 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1fa809eca0 0x7f1fa813cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.562+0000 7f1fa67fc700 1 -- 172.21.15.3:0/2938487538 >> v2:172.21.15.3:3300/0 conn(0x7f1fa806f050 msgr2=0x7f1fa813f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.562+0000 7f1fa67fc700 1 --2- 172.21.15.3:0/2938487538 >> v2:172.21.15.3:3300/0 conn(0x7f1fa806f050 0x7f1fa813f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.562+0000 7f1fa67fc700 1 -- 172.21.15.3:0/2938487538 >> v1:172.21.15.3:6789/0 conn(0x7f1fa806e570 legacy=0x7f1fa806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.562+0000 7f1fa67fc700 1 -- 172.21.15.3:0/2938487538 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa8071b90 con 0x7f1fa809eca0 2023-10-03T10:12:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.562+0000 7f1fa67fc700 1 --2- 172.21.15.3:0/2938487538 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1fa809eca0 0x7f1fa813cea0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c009840 tx=0x7f1f9c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d77f27a4dc0e523b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.562+0000 7f1fa5ffb700 1 -- 172.21.15.3:0/2938487538 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c004230 con 0x7f1fa809eca0 2023-10-03T10:12:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.562+0000 7f1fa5ffb700 1 -- 172.21.15.3:0/2938487538 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9c004390 con 0x7f1fa809eca0 2023-10-03T10:12:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.562+0000 7f1fa5ffb700 1 -- 172.21.15.3:0/2938487538 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c0036a0 con 0x7f1fa809eca0 2023-10-03T10:12:15.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.563+0000 7f1fad353700 1 -- 172.21.15.3:0/2938487538 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1fa809eca0 msgr2=0x7f1fa813cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:15.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.563+0000 7f1fad353700 1 --2- 172.21.15.3:0/2938487538 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1fa809eca0 0x7f1fa813cea0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c009840 tx=0x7f1f9c009b50 comp rx=0 tx=0).stop 2023-10-03T10:12:15.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.563+0000 7f1fad353700 1 -- 172.21.15.3:0/2938487538 shutdown_connections 2023-10-03T10:12:15.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.563+0000 7f1fad353700 1 --2- 172.21.15.3:0/2938487538 >> v2:172.21.15.3:3300/0 conn(0x7f1fa806f050 0x7f1fa813f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.563+0000 7f1fad353700 1 --2- 172.21.15.3:0/2938487538 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1fa809eca0 0x7f1fa813cea0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.563+0000 7f1fad353700 1 -- 172.21.15.3:0/2938487538 >> 172.21.15.3:0/2938487538 conn(0x7f1fa80a2aa0 msgr2=0x7f1fa80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:15.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.563+0000 7f1fad353700 1 -- 172.21.15.3:0/2938487538 shutdown_connections 2023-10-03T10:12:15.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.564+0000 7f1fad353700 1 -- 172.21.15.3:0/2938487538 wait complete. 2023-10-03T10:12:15.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.565+0000 7f1fad353700 1 Processor -- start 2023-10-03T10:12:15.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.565+0000 7f1fad353700 1 -- start start 2023-10-03T10:12:15.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.565+0000 7f1fad353700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1fa806e570 0x7f1fa813cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:15.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.565+0000 7f1fad353700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fa806f050 0x7f1fa813d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:15.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.565+0000 7f1fad353700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1fa812e310 0x7f1fa813d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:15.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.565+0000 7f1fad353700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa8070610 con 0x7f1fa806f050 2023-10-03T10:12:15.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.565+0000 7f1fad353700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa813ec60 con 0x7f1fa806e570 2023-10-03T10:12:15.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.565+0000 7f1fad353700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa812d5a0 con 0x7f1fa812e310 2023-10-03T10:12:15.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.566+0000 7f1fa67fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fa806f050 0x7f1fa813d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:15.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.566+0000 7f1fa67fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fa806f050 0x7f1fa813d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44204/0 (socket says 172.21.15.3:44204) 2023-10-03T10:12:15.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.566+0000 7f1fa67fc700 1 -- 172.21.15.3:0/2921708888 learned_addr learned my addr 172.21.15.3:0/2921708888 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:15.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.566+0000 7f1fa6ffd700 1 --2- 172.21.15.3:0/2921708888 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1fa806e570 0x7f1fa813cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:15.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.566+0000 7f1fa67fc700 1 -- 172.21.15.3:0/2921708888 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1fa812e310 msgr2=0x7f1fa813d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:15.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.566+0000 7f1fa77fe700 1 --2- 172.21.15.3:0/2921708888 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1fa812e310 0x7f1fa813d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:15.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.566+0000 7f1fa67fc700 1 --2- 172.21.15.3:0/2921708888 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1fa812e310 0x7f1fa813d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.566+0000 7f1fa67fc700 1 -- 172.21.15.3:0/2921708888 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1fa806e570 msgr2=0x7f1fa813cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:15.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.566+0000 7f1fa67fc700 1 --2- 172.21.15.3:0/2921708888 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1fa806e570 0x7f1fa813cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.566+0000 7f1fa67fc700 1 -- 172.21.15.3:0/2921708888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9c0094a0 con 0x7f1fa806f050 2023-10-03T10:12:15.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.566+0000 7f1fa77fe700 1 --2- 172.21.15.3:0/2921708888 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1fa812e310 0x7f1fa813d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:12:15.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.566+0000 7f1fa67fc700 1 --2- 172.21.15.3:0/2921708888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fa806f050 0x7f1fa813d140 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c014040 tx=0x7f1f9c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:15.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.567+0000 7f1f8f7fe700 1 -- 172.21.15.3:0/2921708888 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c011830 con 0x7f1fa806f050 2023-10-03T10:12:15.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.567+0000 7f1f8f7fe700 1 -- 172.21.15.3:0/2921708888 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9c011990 con 0x7f1fa806f050 2023-10-03T10:12:15.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.567+0000 7f1fad353700 1 -- 172.21.15.3:0/2921708888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa812deb0 con 0x7f1fa806f050 2023-10-03T10:12:15.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.567+0000 7f1fad353700 1 -- 172.21.15.3:0/2921708888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa812db30 con 0x7f1fa806f050 2023-10-03T10:12:15.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.568+0000 7f1f8f7fe700 1 -- 172.21.15.3:0/2921708888 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c011d20 con 0x7f1fa806f050 2023-10-03T10:12:15.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.568+0000 7f1fad353700 1 -- 172.21.15.3:0/2921708888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa8074ea0 con 0x7f1fa806f050 2023-10-03T10:12:15.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.569+0000 7f1f8f7fe700 1 -- 172.21.15.3:0/2921708888 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) v1 ==== 35345+0+0 (secure 0 0 0) 0x7f1f9c018050 con 0x7f1fa806f050 2023-10-03T10:12:15.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.569+0000 7f1f8f7fe700 1 -- 172.21.15.3:0/2921708888 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4078+0+0 (secure 0 0 0) 0x7f1f9c0404e0 con 0x7f1fa806f050 2023-10-03T10:12:15.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.579+0000 7f1f8f7fe700 1 -- 172.21.15.3:0/2921708888 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1f9c01b410 con 0x7f1fa806f050 2023-10-03T10:12:15.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.590+0000 7fba162fd700 1 Processor -- start 2023-10-03T10:12:15.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.590+0000 7fba162fd700 1 -- start start 2023-10-03T10:12:15.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.591+0000 7fba162fd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fba10133420 0x7fba101337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:15.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.591+0000 7fba162fd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fba10133e30 0x7fba1012ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:15.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.591+0000 7fba162fd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba1006b990 con 0x7fba10134260 2023-10-03T10:12:15.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.591+0000 7fba162fd700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba100a0590 con 0x7fba10133e30 2023-10-03T10:12:15.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.591+0000 7fba162fd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1009fb80 con 0x7fba10133420 2023-10-03T10:12:15.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.591+0000 7fba0f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fba10133e30 0x7fba1012ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:15.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.591+0000 7fba0f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fba10133e30 0x7fba1012ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44218/0 (socket says 172.21.15.3:44218) 2023-10-03T10:12:15.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.591+0000 7fba0f7fe700 1 -- 172.21.15.3:0/695100040 learned_addr learned my addr 172.21.15.3:0/695100040 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:15.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.592+0000 7fba0f7fe700 1 -- 172.21.15.3:0/695100040 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fba10133420 msgr2=0x7fba101337f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-03T10:12:15.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.592+0000 7fba0f7fe700 1 --2- 172.21.15.3:0/695100040 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fba10133420 0x7fba101337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.592+0000 7fba0f7fe700 1 -- 172.21.15.3:0/695100040 >> v1:172.21.15.3:6789/0 conn(0x7fba10134260 legacy=0x7fba101304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:15.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.592+0000 7fba0f7fe700 1 -- 172.21.15.3:0/695100040 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba100a0030 con 0x7fba10133e30 2023-10-03T10:12:15.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.592+0000 7fba0f7fe700 1 --2- 172.21.15.3:0/695100040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba10133e30 0x7fba1012ff90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fba040098f0 tx=0x7fba04009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84b24763963f7fc server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:15.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.592+0000 7fba0effd700 1 -- 172.21.15.3:0/695100040 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba04004230 con 0x7fba10133e30 2023-10-03T10:12:15.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.592+0000 7fba0effd700 1 -- 172.21.15.3:0/695100040 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba04004390 con 0x7fba10133e30 2023-10-03T10:12:15.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.592+0000 7fba0effd700 1 -- 172.21.15.3:0/695100040 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba040036a0 con 0x7fba10133e30 2023-10-03T10:12:15.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.593+0000 7fba162fd700 1 -- 172.21.15.3:0/695100040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba10133e30 msgr2=0x7fba1012ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:15.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.593+0000 7fba162fd700 1 --2- 172.21.15.3:0/695100040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba10133e30 0x7fba1012ff90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fba040098f0 tx=0x7fba04009c00 comp rx=0 tx=0).stop 2023-10-03T10:12:15.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.593+0000 7fba162fd700 1 -- 172.21.15.3:0/695100040 shutdown_connections 2023-10-03T10:12:15.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.593+0000 7fba162fd700 1 --2- 172.21.15.3:0/695100040 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fba10133420 0x7fba101337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.593+0000 7fba162fd700 1 --2- 172.21.15.3:0/695100040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba10133e30 0x7fba1012ff90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.593+0000 7fba162fd700 1 -- 172.21.15.3:0/695100040 >> 172.21.15.3:0/695100040 conn(0x7fba100a4210 msgr2=0x7fba100a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:15.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.593+0000 7fba162fd700 1 -- 172.21.15.3:0/695100040 shutdown_connections 2023-10-03T10:12:15.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.593+0000 7fba162fd700 1 -- 172.21.15.3:0/695100040 wait complete. 2023-10-03T10:12:15.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.595+0000 7fba162fd700 1 Processor -- start 2023-10-03T10:12:15.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.595+0000 7fba162fd700 1 -- start start 2023-10-03T10:12:15.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.596+0000 7fba162fd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fba10133420 0x7fba1009ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:15.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.596+0000 7fba162fd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba10133e30 0x7fba1009bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:15.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.596+0000 7fba162fd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fba10134260 0x7fba1009c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:15.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.596+0000 7fba162fd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1012d040 con 0x7fba10133e30 2023-10-03T10:12:15.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.596+0000 7fba162fd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1009ce10 con 0x7fba10133420 2023-10-03T10:12:15.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.596+0000 7fba162fd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba1012dbf0 con 0x7fba10134260 2023-10-03T10:12:15.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.596+0000 7fba0f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba10133e30 0x7fba1009bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:15.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.596+0000 7fba0f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba10133e30 0x7fba1009bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44220/0 (socket says 172.21.15.3:44220) 2023-10-03T10:12:15.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.596+0000 7fba0f7fe700 1 -- 172.21.15.3:0/2466059741 learned_addr learned my addr 172.21.15.3:0/2466059741 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:15.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.596+0000 7fba1489a700 1 --2- 172.21.15.3:0/2466059741 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fba10134260 0x7fba1009c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:15.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.597+0000 7fba0f7fe700 1 -- 172.21.15.3:0/2466059741 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fba10134260 msgr2=0x7fba1009c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:15.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.597+0000 7fba0f7fe700 1 --2- 172.21.15.3:0/2466059741 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fba10134260 0x7fba1009c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.597+0000 7fba0f7fe700 1 -- 172.21.15.3:0/2466059741 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fba10133420 msgr2=0x7fba1009ba40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:15.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.597+0000 7fba0f7fe700 1 --2- 172.21.15.3:0/2466059741 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fba10133420 0x7fba1009ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.597+0000 7fba0f7fe700 1 -- 172.21.15.3:0/2466059741 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba04009550 con 0x7fba10133e30 2023-10-03T10:12:15.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.597+0000 7fba0f7fe700 1 --2- 172.21.15.3:0/2466059741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba10133e30 0x7fba1009bf80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fba04004000 tx=0x7fba04004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:15.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.597+0000 7fba0cff9700 1 -- 172.21.15.3:0/2466059741 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0400b660 con 0x7fba10133e30 2023-10-03T10:12:15.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.597+0000 7fba0cff9700 1 -- 172.21.15.3:0/2466059741 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0400b7c0 con 0x7fba10133e30 2023-10-03T10:12:15.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.597+0000 7fba0cff9700 1 -- 172.21.15.3:0/2466059741 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0400bb30 con 0x7fba10133e30 2023-10-03T10:12:15.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.597+0000 7fba162fd700 1 -- 172.21.15.3:0/2466059741 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba1011af50 con 0x7fba10133e30 2023-10-03T10:12:15.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.597+0000 7fba162fd700 1 -- 172.21.15.3:0/2466059741 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba1009e340 con 0x7fba10133e30 2023-10-03T10:12:15.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.599+0000 7fba0cff9700 1 -- 172.21.15.3:0/2466059741 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) v1 ==== 35345+0+0 (secure 0 0 0) 0x7fba04022020 con 0x7fba10133e30 2023-10-03T10:12:15.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.599+0000 7fba0cff9700 1 -- 172.21.15.3:0/2466059741 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4078+0+0 (secure 0 0 0) 0x7fba04040ee0 con 0x7fba10133e30 2023-10-03T10:12:15.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.601+0000 7fba162fd700 1 -- 172.21.15.3:0/2466059741 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba10074ea0 con 0x7fba10133e30 2023-10-03T10:12:15.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.606+0000 7fba0cff9700 1 -- 172.21.15.3:0/2466059741 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fba040247e0 con 0x7fba10133e30 2023-10-03T10:12:15.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.660+0000 7fa219d5b700 1 Processor -- start 2023-10-03T10:12:15.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.660+0000 7fa219d5b700 1 -- start start 2023-10-03T10:12:15.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.660+0000 7fa219d5b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa21406c0e0 0x7fa21409ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:15.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.660+0000 7fa219d5b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa214134ec0 0x7fa2141379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:15.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.660+0000 7fa219d5b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa214137f20 0x7fa214138390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:15.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.660+0000 7fa219d5b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa21406c4b0 con 0x7fa214137f20 2023-10-03T10:12:15.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.660+0000 7fa219d5b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2140a09c0 con 0x7fa214134ec0 2023-10-03T10:12:15.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.660+0000 7fa219d5b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa214134980 con 0x7fa21406c0e0 2023-10-03T10:12:15.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.661+0000 7fa213fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa214137f20 0x7fa214138390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:15.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.661+0000 7fa213fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa214137f20 0x7fa214138390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44224/0 (socket says 172.21.15.3:44224) 2023-10-03T10:12:15.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.661+0000 7fa213fff700 1 -- 172.21.15.3:0/2983877812 learned_addr learned my addr 172.21.15.3:0/2983877812 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:15.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.661+0000 7fa2137fe700 1 --2- 172.21.15.3:0/2983877812 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa21406c0e0 0x7fa21409ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:15.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.661+0000 7fa213fff700 1 -- 172.21.15.3:0/2983877812 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa21406c0e0 msgr2=0x7fa21409ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:15.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.661+0000 7fa213fff700 1 --2- 172.21.15.3:0/2983877812 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa21406c0e0 0x7fa21409ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.661+0000 7fa213fff700 1 -- 172.21.15.3:0/2983877812 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa214134ec0 msgr2=0x7fa2141379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:15.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.661+0000 7fa213fff700 1 --2- 172.21.15.3:0/2983877812 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa214134ec0 0x7fa2141379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.661+0000 7fa213fff700 1 -- 172.21.15.3:0/2983877812 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2140a0600 con 0x7fa214137f20 2023-10-03T10:12:15.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.661+0000 7fa213fff700 1 --2- 172.21.15.3:0/2983877812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa214137f20 0x7fa214138390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa20c009e80 tx=0x7fa20c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a53b3ebd145de01 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:15.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.662+0000 7fa2127fc700 1 -- 172.21.15.3:0/2983877812 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c00e070 con 0x7fa214137f20 2023-10-03T10:12:15.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.662+0000 7fa2127fc700 1 -- 172.21.15.3:0/2983877812 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa20c00b980 con 0x7fa214137f20 2023-10-03T10:12:15.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.662+0000 7fa2127fc700 1 -- 172.21.15.3:0/2983877812 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c00bd30 con 0x7fa214137f20 2023-10-03T10:12:15.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.663+0000 7fa219d5b700 1 -- 172.21.15.3:0/2983877812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa214137f20 msgr2=0x7fa214138390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:15.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.663+0000 7fa219d5b700 1 --2- 172.21.15.3:0/2983877812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa214137f20 0x7fa214138390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa20c009e80 tx=0x7fa20c00b330 comp rx=0 tx=0).stop 2023-10-03T10:12:15.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.663+0000 7fa219d5b700 1 -- 172.21.15.3:0/2983877812 shutdown_connections 2023-10-03T10:12:15.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.663+0000 7fa219d5b700 1 --2- 172.21.15.3:0/2983877812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa214137f20 0x7fa214138390 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.663+0000 7fa219d5b700 1 --2- 172.21.15.3:0/2983877812 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa21406c0e0 0x7fa21409ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.663+0000 7fa219d5b700 1 --2- 172.21.15.3:0/2983877812 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa214134ec0 0x7fa2141379e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.663+0000 7fa219d5b700 1 -- 172.21.15.3:0/2983877812 >> 172.21.15.3:0/2983877812 conn(0x7fa2140a4210 msgr2=0x7fa2140a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:15.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.663+0000 7fa219d5b700 1 -- 172.21.15.3:0/2983877812 shutdown_connections 2023-10-03T10:12:15.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.663+0000 7fa219d5b700 1 -- 172.21.15.3:0/2983877812 wait complete. 2023-10-03T10:12:15.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.664+0000 7fa219d5b700 1 Processor -- start 2023-10-03T10:12:15.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.664+0000 7fa219d5b700 1 -- start start 2023-10-03T10:12:15.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.665+0000 7fa219d5b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa21406c0e0 0x7fa21411a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:15.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.665+0000 7fa219d5b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa214134ec0 0x7fa21411ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:15.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.665+0000 7fa219d5b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa214137f20 0x7fa21411b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:15.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.665+0000 7fa219d5b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21411c380 con 0x7fa214137f20 2023-10-03T10:12:15.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.665+0000 7fa219d5b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21411df10 con 0x7fa214134ec0 2023-10-03T10:12:15.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.665+0000 7fa219d5b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa21406d280 con 0x7fa21406c0e0 2023-10-03T10:12:15.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.665+0000 7fa212ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa214134ec0 0x7fa21411ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:15.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.665+0000 7fa212ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa214134ec0 0x7fa21411ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37430/0 (socket says 172.21.15.3:37430) 2023-10-03T10:12:15.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.665+0000 7fa212ffd700 1 -- 172.21.15.3:0/2132964202 learned_addr learned my addr 172.21.15.3:0/2132964202 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:15.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.665+0000 7fa213fff700 1 --2- 172.21.15.3:0/2132964202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa214137f20 0x7fa21411b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:15.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.665+0000 7fa212ffd700 1 -- 172.21.15.3:0/2132964202 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa21406c0e0 msgr2=0x7fa21411a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:15.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.665+0000 7fa212ffd700 1 --2- 172.21.15.3:0/2132964202 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa21406c0e0 0x7fa21411a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.665+0000 7fa212ffd700 1 -- 172.21.15.3:0/2132964202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa214137f20 msgr2=0x7fa21411b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:15.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.665+0000 7fa212ffd700 1 --2- 172.21.15.3:0/2132964202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa214137f20 0x7fa21411b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.665+0000 7fa212ffd700 1 -- 172.21.15.3:0/2132964202 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20c009b30 con 0x7fa214134ec0 2023-10-03T10:12:15.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.666+0000 7fa212ffd700 1 --2- 172.21.15.3:0/2132964202 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa214134ec0 0x7fa21411ae20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa20800b500 tx=0x7fa20800b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:15.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.666+0000 7fa1fbfff700 1 -- 172.21.15.3:0/2132964202 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa208011830 con 0x7fa214134ec0 2023-10-03T10:12:15.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.666+0000 7fa1fbfff700 1 -- 172.21.15.3:0/2132964202 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2080119b0 con 0x7fa214134ec0 2023-10-03T10:12:15.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.666+0000 7fa1fbfff700 1 -- 172.21.15.3:0/2132964202 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa208011d60 con 0x7fa214134ec0 2023-10-03T10:12:15.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.667+0000 7fa219d5b700 1 -- 172.21.15.3:0/2132964202 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2140a03f0 con 0x7fa214134ec0 2023-10-03T10:12:15.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.667+0000 7fa219d5b700 1 -- 172.21.15.3:0/2132964202 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa214138d40 con 0x7fa214134ec0 2023-10-03T10:12:15.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.668+0000 7fa1fbfff700 1 -- 172.21.15.3:0/2132964202 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 7) v1 ==== 35345+0+0 (secure 0 0 0) 0x7fa20801a400 con 0x7fa214134ec0 2023-10-03T10:12:15.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.668+0000 7fa219d5b700 1 -- 172.21.15.3:0/2132964202 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa214074ea0 con 0x7fa214134ec0 2023-10-03T10:12:15.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.668+0000 7fa1fbfff700 1 -- 172.21.15.3:0/2132964202 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4078+0+0 (secure 0 0 0) 0x7fa20803f690 con 0x7fa214134ec0 2023-10-03T10:12:15.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.678+0000 7fa1fbfff700 1 -- 172.21.15.3:0/2132964202 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa20800f8d0 con 0x7fa214134ec0 2023-10-03T10:12:15.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.784+0000 7f3a346a9700 1 -- 172.21.15.3:0/2405576028 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3a2c06df70 con 0x7f3a2c13c470 2023-10-03T10:12:15.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.784+0000 7f3a22ffd700 1 -- 172.21.15.3:0/2405576028 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3a28017020 con 0x7f3a2c13c470 2023-10-03T10:12:15.787 INFO:teuthology.orchestra.run.smithi003.stdout:38654705697 2023-10-03T10:12:15.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.787+0000 7f3a346a9700 1 -- 172.21.15.3:0/2405576028 >> v1:172.21.15.3:6816/127025 conn(0x7f3a1002f240 legacy=0x7f3a100316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:15.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.787+0000 7f3a346a9700 1 -- 172.21.15.3:0/2405576028 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3a2c13c470 msgr2=0x7f3a2c128ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:15.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.787+0000 7f3a346a9700 1 --2- 172.21.15.3:0/2405576028 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3a2c13c470 0x7f3a2c128ff0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3a2800c4a0 tx=0x7f3a2800c860 comp rx=0 tx=0).stop 2023-10-03T10:12:15.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.787+0000 7f3a346a9700 1 -- 172.21.15.3:0/2405576028 shutdown_connections 2023-10-03T10:12:15.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.787+0000 7f3a346a9700 1 --2- 172.21.15.3:0/2405576028 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3a2c13c470 0x7f3a2c128ff0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.787+0000 7f3a346a9700 1 --2- 172.21.15.3:0/2405576028 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3a2c09c5e0 0x7f3a2c13b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.787+0000 7f3a346a9700 1 --2- 172.21.15.3:0/2405576028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a2c09cef0 0x7f3a2c13bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.787+0000 7f3a346a9700 1 -- 172.21.15.3:0/2405576028 >> 172.21.15.3:0/2405576028 conn(0x7f3a2c09e7e0 msgr2=0x7f3a2c12e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:15.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.787+0000 7f3a346a9700 1 -- 172.21.15.3:0/2405576028 shutdown_connections 2023-10-03T10:12:15.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.787+0000 7f3a346a9700 1 -- 172.21.15.3:0/2405576028 wait complete. 2023-10-03T10:12:15.802 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705696 got 38654705697 for osd.1 2023-10-03T10:12:15.803 DEBUG:teuthology.parallel:result is None 2023-10-03T10:12:15.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.839+0000 7fba162fd700 1 -- 172.21.15.3:0/2466059741 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fba1006b250 con 0x7fba10133e30 2023-10-03T10:12:15.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.840+0000 7fba0cff9700 1 -- 172.21.15.3:0/2466059741 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fba040193f0 con 0x7fba10133e30 2023-10-03T10:12:15.841 INFO:teuthology.orchestra.run.smithi003.stdout:38654705697 2023-10-03T10:12:15.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.842+0000 7fba162fd700 1 -- 172.21.15.3:0/2466059741 >> v1:172.21.15.3:6816/127025 conn(0x7fb9fc02f240 legacy=0x7fb9fc0316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:15.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.842+0000 7fba162fd700 1 -- 172.21.15.3:0/2466059741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba10133e30 msgr2=0x7fba1009bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:15.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.842+0000 7fba162fd700 1 --2- 172.21.15.3:0/2466059741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba10133e30 0x7fba1009bf80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fba04004000 tx=0x7fba04004750 comp rx=0 tx=0).stop 2023-10-03T10:12:15.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.842+0000 7fba162fd700 1 -- 172.21.15.3:0/2466059741 shutdown_connections 2023-10-03T10:12:15.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.842+0000 7fba162fd700 1 --2- 172.21.15.3:0/2466059741 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fba10134260 0x7fba1009c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.842+0000 7fba162fd700 1 --2- 172.21.15.3:0/2466059741 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fba10133420 0x7fba1009ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.842+0000 7fba162fd700 1 --2- 172.21.15.3:0/2466059741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba10133e30 0x7fba1009bf80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.842+0000 7fba162fd700 1 -- 172.21.15.3:0/2466059741 >> 172.21.15.3:0/2466059741 conn(0x7fba100a4210 msgr2=0x7fba1012fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:15.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.842+0000 7fba162fd700 1 -- 172.21.15.3:0/2466059741 shutdown_connections 2023-10-03T10:12:15.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.842+0000 7fba162fd700 1 -- 172.21.15.3:0/2466059741 wait complete. 2023-10-03T10:12:15.856 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705696 got 38654705697 for osd.3 2023-10-03T10:12:15.857 DEBUG:teuthology.parallel:result is None 2023-10-03T10:12:15.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.919+0000 7f1fad353700 1 -- 172.21.15.3:0/2921708888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1fa80711f0 con 0x7f1fa806f050 2023-10-03T10:12:15.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.919+0000 7f1f8f7fe700 1 -- 172.21.15.3:0/2921708888 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1f9c016070 con 0x7f1fa806f050 2023-10-03T10:12:15.920 INFO:teuthology.orchestra.run.smithi003.stdout:38654705697 2023-10-03T10:12:15.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.922+0000 7f1fad353700 1 -- 172.21.15.3:0/2921708888 >> v1:172.21.15.3:6816/127025 conn(0x7f1f9402f240 legacy=0x7f1f940316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:15.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.922+0000 7f1fad353700 1 -- 172.21.15.3:0/2921708888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fa806f050 msgr2=0x7f1fa813d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:15.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.922+0000 7f1fad353700 1 --2- 172.21.15.3:0/2921708888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fa806f050 0x7f1fa813d140 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c014040 tx=0x7f1f9c004750 comp rx=0 tx=0).stop 2023-10-03T10:12:15.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.922+0000 7f1fad353700 1 -- 172.21.15.3:0/2921708888 shutdown_connections 2023-10-03T10:12:15.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.922+0000 7f1fad353700 1 --2- 172.21.15.3:0/2921708888 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1fa812e310 0x7f1fa813d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.922+0000 7f1fad353700 1 --2- 172.21.15.3:0/2921708888 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1fa806e570 0x7f1fa813cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.922+0000 7f1fad353700 1 --2- 172.21.15.3:0/2921708888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fa806f050 0x7f1fa813d140 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:15.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.922+0000 7f1fad353700 1 -- 172.21.15.3:0/2921708888 >> 172.21.15.3:0/2921708888 conn(0x7f1fa80a2aa0 msgr2=0x7f1fa809cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:15.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.922+0000 7f1fad353700 1 -- 172.21.15.3:0/2921708888 shutdown_connections 2023-10-03T10:12:15.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:15.922+0000 7f1fad353700 1 -- 172.21.15.3:0/2921708888 wait complete. 2023-10-03T10:12:15.937 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705696 got 38654705697 for osd.0 2023-10-03T10:12:15.937 DEBUG:teuthology.parallel:result is None 2023-10-03T10:12:16.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.000+0000 7fa219d5b700 1 -- 172.21.15.3:0/2132964202 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa21406b250 con 0x7fa214134ec0 2023-10-03T10:12:16.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.000+0000 7fa1fbfff700 1 -- 172.21.15.3:0/2132964202 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa208015030 con 0x7fa214134ec0 2023-10-03T10:12:16.001 INFO:teuthology.orchestra.run.smithi003.stdout:38654705696 2023-10-03T10:12:16.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.002+0000 7fa219d5b700 1 -- 172.21.15.3:0/2132964202 >> v1:172.21.15.3:6816/127025 conn(0x7fa20002f240 legacy=0x7fa2000316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:16.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.002+0000 7fa219d5b700 1 -- 172.21.15.3:0/2132964202 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa214134ec0 msgr2=0x7fa21411ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:16.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.002+0000 7fa219d5b700 1 --2- 172.21.15.3:0/2132964202 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa214134ec0 0x7fa21411ae20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa20800b500 tx=0x7fa20800b810 comp rx=0 tx=0).stop 2023-10-03T10:12:16.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.002+0000 7fa219d5b700 1 -- 172.21.15.3:0/2132964202 shutdown_connections 2023-10-03T10:12:16.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.002+0000 7fa219d5b700 1 --2- 172.21.15.3:0/2132964202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa214137f20 0x7fa21411b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:16.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.002+0000 7fa219d5b700 1 --2- 172.21.15.3:0/2132964202 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa21406c0e0 0x7fa21411a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:16.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.002+0000 7fa219d5b700 1 --2- 172.21.15.3:0/2132964202 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa214134ec0 0x7fa21411ae20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:16.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.002+0000 7fa219d5b700 1 -- 172.21.15.3:0/2132964202 >> 172.21.15.3:0/2132964202 conn(0x7fa2140a4210 msgr2=0x7fa214099c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:16.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.002+0000 7fa219d5b700 1 -- 172.21.15.3:0/2132964202 shutdown_connections 2023-10-03T10:12:16.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.003+0000 7fa219d5b700 1 -- 172.21.15.3:0/2132964202 wait complete. 2023-10-03T10:12:16.016 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705695 got 38654705696 for osd.2 2023-10-03T10:12:16.017 DEBUG:teuthology.parallel:result is None 2023-10-03T10:12:16.017 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-03T10:12:16.017 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-03T10:12:16.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.156+0000 7f80d8bf8700 1 Processor -- start 2023-10-03T10:12:16.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.156+0000 7f80d8bf8700 1 -- start start 2023-10-03T10:12:16.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.156+0000 7f80d8bf8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f80d4133420 0x7f80d41337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:16.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.156+0000 7f80d8bf8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f80d4133e30 0x7f80d412ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:16.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.156+0000 7f80d8bf8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f80d4134260 0x7f80d41304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:16.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.156+0000 7f80d8bf8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80d406b990 con 0x7f80d4133420 2023-10-03T10:12:16.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.156+0000 7f80d8bf8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d40a0590 con 0x7f80d4134260 2023-10-03T10:12:16.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.156+0000 7f80d8bf8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f80d409fb80 con 0x7f80d4133e30 2023-10-03T10:12:16.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.157+0000 7f80d259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f80d4133420 0x7f80d41337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:16.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.157+0000 7f80d1d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f80d4133e30 0x7f80d412ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:16.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.157+0000 7f80d1d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f80d4133e30 0x7f80d412ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:59068/0 (socket says 172.21.15.3:59068) 2023-10-03T10:12:16.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.157+0000 7f80d1d9b700 1 -- 172.21.15.3:0/588160158 learned_addr learned my addr 172.21.15.3:0/588160158 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:16.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.157+0000 7f80d259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f80d4133420 0x7f80d41337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44252/0 (socket says 172.21.15.3:44252) 2023-10-03T10:12:16.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.157+0000 7f80d2d9d700 1 --2- 172.21.15.3:0/588160158 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f80d4134260 0x7f80d41304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:16.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.157+0000 7f80d259c700 1 -- 172.21.15.3:0/588160158 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f80d4133e30 msgr2=0x7f80d412ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:16.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.157+0000 7f80d259c700 1 --2- 172.21.15.3:0/588160158 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f80d4133e30 0x7f80d412ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:16.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.157+0000 7f80d259c700 1 -- 172.21.15.3:0/588160158 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f80d4134260 msgr2=0x7f80d41304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:16.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.157+0000 7f80d259c700 1 --2- 172.21.15.3:0/588160158 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f80d4134260 0x7f80d41304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:16.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.157+0000 7f80d259c700 1 -- 172.21.15.3:0/588160158 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d40a0030 con 0x7f80d4133420 2023-10-03T10:12:16.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.157+0000 7f80d1d9b700 1 --2- 172.21.15.3:0/588160158 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f80d4133e30 0x7f80d412ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:16.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.157+0000 7f80d259c700 1 --2- 172.21.15.3:0/588160158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80d4133420 0x7f80d41337f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009d90 tx=0x7f80c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba36d547e302584 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:16.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.157+0000 7f80d159a700 1 -- 172.21.15.3:0/588160158 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c400e070 con 0x7f80d4133420 2023-10-03T10:12:16.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.157+0000 7f80d159a700 1 -- 172.21.15.3:0/588160158 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c400bb20 con 0x7f80d4133420 2023-10-03T10:12:16.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.157+0000 7f80d159a700 1 -- 172.21.15.3:0/588160158 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c4012430 con 0x7f80d4133420 2023-10-03T10:12:16.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.158+0000 7f80d8bf8700 1 -- 172.21.15.3:0/588160158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80d4133420 msgr2=0x7f80d41337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:16.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.158+0000 7f80d8bf8700 1 --2- 172.21.15.3:0/588160158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80d4133420 0x7f80d41337f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009d90 tx=0x7f80c400b330 comp rx=0 tx=0).stop 2023-10-03T10:12:16.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.159+0000 7f80d8bf8700 1 -- 172.21.15.3:0/588160158 shutdown_connections 2023-10-03T10:12:16.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.159+0000 7f80d8bf8700 1 --2- 172.21.15.3:0/588160158 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f80d4134260 0x7f80d41304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:16.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.159+0000 7f80d8bf8700 1 --2- 172.21.15.3:0/588160158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80d4133420 0x7f80d41337f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:16.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.159+0000 7f80d8bf8700 1 --2- 172.21.15.3:0/588160158 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f80d4133e30 0x7f80d412ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:16.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.159+0000 7f80d8bf8700 1 -- 172.21.15.3:0/588160158 >> 172.21.15.3:0/588160158 conn(0x7f80d40a4210 msgr2=0x7f80d40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:16.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.159+0000 7f80d8bf8700 1 -- 172.21.15.3:0/588160158 shutdown_connections 2023-10-03T10:12:16.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.159+0000 7f80d8bf8700 1 -- 172.21.15.3:0/588160158 wait complete. 2023-10-03T10:12:16.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.160+0000 7f80d8bf8700 1 Processor -- start 2023-10-03T10:12:16.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.160+0000 7f80d8bf8700 1 -- start start 2023-10-03T10:12:16.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.161+0000 7f80d8bf8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f80d4133420 0x7f80d409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:16.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.161+0000 7f80d8bf8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80d4133e30 0x7f80d409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:16.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.161+0000 7f80d8bf8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f80d4134260 0x7f80d409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:16.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.161+0000 7f80d8bf8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d409d8e0 con 0x7f80d4133e30 2023-10-03T10:12:16.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.161+0000 7f80d8bf8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d409dcc0 con 0x7f80d4134260 2023-10-03T10:12:16.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.161+0000 7f80d8bf8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f80d4131490 con 0x7f80d4133420 2023-10-03T10:12:16.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.161+0000 7f80d2d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f80d4134260 0x7f80d409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:16.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.161+0000 7f80d1d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80d4133e30 0x7f80d409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:16.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.161+0000 7f80d2d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f80d4134260 0x7f80d409cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37474/0 (socket says 172.21.15.3:37474) 2023-10-03T10:12:16.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.161+0000 7f80d2d9d700 1 -- 172.21.15.3:0/3486426447 learned_addr learned my addr 172.21.15.3:0/3486426447 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:16.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.161+0000 7f80d259c700 1 --2- 172.21.15.3:0/3486426447 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f80d4133420 0x7f80d409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:16.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.162+0000 7f80d259c700 1 -- 172.21.15.3:0/3486426447 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f80d4134260 msgr2=0x7f80d409cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:16.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.162+0000 7f80d259c700 1 --2- 172.21.15.3:0/3486426447 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f80d4134260 0x7f80d409cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:16.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.162+0000 7f80d259c700 1 -- 172.21.15.3:0/3486426447 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80d4133e30 msgr2=0x7f80d409c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:16.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.162+0000 7f80d259c700 1 --2- 172.21.15.3:0/3486426447 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80d4133e30 0x7f80d409c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:16.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.162+0000 7f80d259c700 1 -- 172.21.15.3:0/3486426447 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80bc009550 con 0x7f80d4133420 2023-10-03T10:12:16.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.162+0000 7f80d1d9b700 1 --2- 172.21.15.3:0/3486426447 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80d4133e30 0x7f80d409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:16.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.162+0000 7f80d259c700 1 --2- 172.21.15.3:0/3486426447 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f80d4133420 0x7f80d409c480 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f80c4016000 tx=0x7f80c4004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:16.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.162+0000 7f80caffd700 1 -- 172.21.15.3:0/3486426447 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c4026070 con 0x7f80d4133420 2023-10-03T10:12:16.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.162+0000 7f80caffd700 1 -- 172.21.15.3:0/3486426447 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c40098f0 con 0x7f80d4133420 2023-10-03T10:12:16.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.162+0000 7f80d8bf8700 1 -- 172.21.15.3:0/3486426447 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c4009a70 con 0x7f80d4133420 2023-10-03T10:12:16.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.162+0000 7f80caffd700 1 -- 172.21.15.3:0/3486426447 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c4012430 con 0x7f80d4133420 2023-10-03T10:12:16.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.162+0000 7f80d8bf8700 1 -- 172.21.15.3:0/3486426447 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d411ff00 con 0x7f80d4133420 2023-10-03T10:12:16.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.163+0000 7f80caffd700 1 -- 172.21.15.3:0/3486426447 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 7) v1 ==== 35345+0+0 (secure 0 0 0) 0x7f80c402a070 con 0x7f80d4133420 2023-10-03T10:12:16.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.163+0000 7f80caffd700 1 -- 172.21.15.3:0/3486426447 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4078+0+0 (secure 0 0 0) 0x7f80c40489d0 con 0x7f80d4133420 2023-10-03T10:12:16.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.164+0000 7f80d8bf8700 1 -- 172.21.15.3:0/3486426447 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d4074ea0 con 0x7f80d4133420 2023-10-03T10:12:16.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.169+0000 7f80caffd700 1 -- 172.21.15.3:0/3486426447 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f80c4031080 con 0x7f80d4133420 2023-10-03T10:12:16.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.331+0000 7f80caffd700 1 -- 172.21.15.3:0/3486426447 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7f80c4012820 con 0x7f80d4133420 2023-10-03T10:12:16.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.385+0000 7f80d8bf8700 1 -- 172.21.15.3:0/3486426447 --> v1:172.21.15.3:6816/127025 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f80d406b7b0 con 0x7f80b402f270 2023-10-03T10:12:16.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.393+0000 7f80caffd700 1 -- 172.21.15.3:0/3486426447 <== mgr.14098 v1:172.21.15.3:6816/127025 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+270428 (unknown 2965378022 0 295861048) 0x7f80d406b7b0 con 0x7f80b402f270 2023-10-03T10:12:16.396 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:12:16.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.396+0000 7f80d8bf8700 1 -- 172.21.15.3:0/3486426447 >> v1:172.21.15.3:6816/127025 conn(0x7f80b402f270 legacy=0x7f80b4031720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:16.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.396+0000 7f80d8bf8700 1 -- 172.21.15.3:0/3486426447 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f80d4133420 msgr2=0x7f80d409c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:16.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.396+0000 7f80d8bf8700 1 --2- 172.21.15.3:0/3486426447 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f80d4133420 0x7f80d409c480 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f80c4016000 tx=0x7f80c4004ad0 comp rx=0 tx=0).stop 2023-10-03T10:12:16.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.396+0000 7f80d8bf8700 1 -- 172.21.15.3:0/3486426447 shutdown_connections 2023-10-03T10:12:16.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.396+0000 7f80d8bf8700 1 --2- 172.21.15.3:0/3486426447 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f80d4134260 0x7f80d409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:16.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.396+0000 7f80d8bf8700 1 --2- 172.21.15.3:0/3486426447 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f80d4133420 0x7f80d409c480 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:16.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.396+0000 7f80d8bf8700 1 --2- 172.21.15.3:0/3486426447 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80d4133e30 0x7f80d409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:16.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.396+0000 7f80d8bf8700 1 -- 172.21.15.3:0/3486426447 >> 172.21.15.3:0/3486426447 conn(0x7f80d40a4210 msgr2=0x7f80d412de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:16.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.396+0000 7f80d8bf8700 1 -- 172.21.15.3:0/3486426447 shutdown_connections 2023-10-03T10:12:16.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.396+0000 7f80d8bf8700 1 -- 172.21.15.3:0/3486426447 wait complete. 2023-10-03T10:12:16.399 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-03T10:12:16.410 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2023-10-03T10:12:15.327643+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4333,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1189132,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298228,"statfs":{"total":386547056640,"available":385329385472,"internally_reserved":0,"allocated":765952,"data_stored":259394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.998295"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.362278+0000","last_change":"2023-10-03T10:09:44.195682+0000","last_active":"2023-10-03T10:12:13.362278+0000","last_peered":"2023-10-03T10:12:13.362278+0000","last_clean":"2023-10-03T10:12:13.362278+0000","last_became_active":"2023-10-03T10:09:44.195360+0000","last_became_peered":"2023-10-03T10:09:44.195360+0000","last_unstale":"2023-10-03T10:12:13.362278+0000","last_undegraded":"2023-10-03T10:12:13.362278+0000","last_fullsized":"2023-10-03T10:12:13.362278+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.362172+0000","last_change":"2023-10-03T10:09:44.178263+0000","last_active":"2023-10-03T10:12:13.362172+0000","last_peered":"2023-10-03T10:12:13.362172+0000","last_clean":"2023-10-03T10:12:13.362172+0000","last_became_active":"2023-10-03T10:09:44.177977+0000","last_became_peered":"2023-10-03T10:09:44.177977+0000","last_unstale":"2023-10-03T10:12:13.362172+0000","last_undegraded":"2023-10-03T10:12:13.362172+0000","last_fullsized":"2023-10-03T10:12:13.362172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.515500+0000","last_change":"2023-10-03T10:09:44.183364+0000","last_active":"2023-10-03T10:10:35.515500+0000","last_peered":"2023-10-03T10:10:35.515500+0000","last_clean":"2023-10-03T10:10:35.515500+0000","last_became_active":"2023-10-03T10:09:44.183094+0000","last_became_peered":"2023-10-03T10:09:44.183094+0000","last_unstale":"2023-10-03T10:10:35.515500+0000","last_undegraded":"2023-10-03T10:10:35.515500+0000","last_fullsized":"2023-10-03T10:10:35.515500+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.514914+0000","last_change":"2023-10-03T10:09:44.183273+0000","last_active":"2023-10-03T10:10:35.514914+0000","last_peered":"2023-10-03T10:10:35.514914+0000","last_clean":"2023-10-03T10:10:35.514914+0000","last_became_active":"2023-10-03T10:09:44.182451+0000","last_became_peered":"2023-10-03T10:09:44.182451+0000","last_unstale":"2023-10-03T10:10:35.514914+0000","last_undegraded":"2023-10-03T10:10:35.514914+0000","last_fullsized":"2023-10-03T10:10:35.514914+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.514451+0000","last_change":"2023-10-03T10:09:44.168483+0000","last_active":"2023-10-03T10:10:35.514451+0000","last_peered":"2023-10-03T10:10:35.514451+0000","last_clean":"2023-10-03T10:10:35.514451+0000","last_became_active":"2023-10-03T10:09:44.168186+0000","last_became_peered":"2023-10-03T10:09:44.168186+0000","last_unstale":"2023-10-03T10:10:35.514451+0000","last_undegraded":"2023-10-03T10:10:35.514451+0000","last_fullsized":"2023-10-03T10:10:35.514451+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518314+0000","last_change":"2023-10-03T10:09:44.192967+0000","last_active":"2023-10-03T10:10:35.518314+0000","last_peered":"2023-10-03T10:10:35.518314+0000","last_clean":"2023-10-03T10:10:35.518314+0000","last_became_active":"2023-10-03T10:09:44.190979+0000","last_became_peered":"2023-10-03T10:09:44.190979+0000","last_unstale":"2023-10-03T10:10:35.518314+0000","last_undegraded":"2023-10-03T10:10:35.518314+0000","last_fullsized":"2023-10-03T10:10:35.518314+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.360752+0000","last_change":"2023-10-03T10:10:35.520559+0000","last_active":"2023-10-03T10:12:13.360752+0000","last_peered":"2023-10-03T10:12:13.360752+0000","last_clean":"2023-10-03T10:12:13.360752+0000","last_became_active":"2023-10-03T10:10:35.520227+0000","last_became_peered":"2023-10-03T10:10:35.520227+0000","last_unstale":"2023-10-03T10:12:13.360752+0000","last_undegraded":"2023-10-03T10:12:13.360752+0000","last_fullsized":"2023-10-03T10:12:13.360752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593090+0000","last_change":"2023-10-03T10:09:44.173123+0000","last_active":"2023-10-03T10:10:35.593090+0000","last_peered":"2023-10-03T10:10:35.593090+0000","last_clean":"2023-10-03T10:10:35.593090+0000","last_became_active":"2023-10-03T10:09:44.172353+0000","last_became_peered":"2023-10-03T10:09:44.172353+0000","last_unstale":"2023-10-03T10:10:35.593090+0000","last_undegraded":"2023-10-03T10:10:35.593090+0000","last_fullsized":"2023-10-03T10:10:35.593090+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.516273+0000","last_change":"2023-10-03T10:09:44.166758+0000","last_active":"2023-10-03T10:10:35.516273+0000","last_peered":"2023-10-03T10:10:35.516273+0000","last_clean":"2023-10-03T10:10:35.516273+0000","last_became_active":"2023-10-03T10:09:44.166415+0000","last_became_peered":"2023-10-03T10:09:44.166415+0000","last_unstale":"2023-10-03T10:10:35.516273+0000","last_undegraded":"2023-10-03T10:10:35.516273+0000","last_fullsized":"2023-10-03T10:10:35.516273+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593631+0000","last_change":"2023-10-03T10:09:44.176833+0000","last_active":"2023-10-03T10:10:35.593631+0000","last_peered":"2023-10-03T10:10:35.593631+0000","last_clean":"2023-10-03T10:10:35.593631+0000","last_became_active":"2023-10-03T10:09:44.176526+0000","last_became_peered":"2023-10-03T10:09:44.176526+0000","last_unstale":"2023-10-03T10:10:35.593631+0000","last_undegraded":"2023-10-03T10:10:35.593631+0000","last_fullsized":"2023-10-03T10:10:35.593631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593635+0000","last_change":"2023-10-03T10:09:44.175868+0000","last_active":"2023-10-03T10:10:35.593635+0000","last_peered":"2023-10-03T10:10:35.593635+0000","last_clean":"2023-10-03T10:10:35.593635+0000","last_became_active":"2023-10-03T10:09:44.175414+0000","last_became_peered":"2023-10-03T10:09:44.175414+0000","last_unstale":"2023-10-03T10:10:35.593635+0000","last_undegraded":"2023-10-03T10:10:35.593635+0000","last_fullsized":"2023-10-03T10:10:35.593635+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.362364+0000","last_change":"2023-10-03T10:09:44.198581+0000","last_active":"2023-10-03T10:12:13.362364+0000","last_peered":"2023-10-03T10:12:13.362364+0000","last_clean":"2023-10-03T10:12:13.362364+0000","last_became_active":"2023-10-03T10:09:44.198314+0000","last_became_peered":"2023-10-03T10:09:44.198314+0000","last_unstale":"2023-10-03T10:12:13.362364+0000","last_undegraded":"2023-10-03T10:12:13.362364+0000","last_fullsized":"2023-10-03T10:12:13.362364+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.360956+0000","last_change":"2023-10-03T10:10:35.521810+0000","last_active":"2023-10-03T10:12:13.360956+0000","last_peered":"2023-10-03T10:12:13.360956+0000","last_clean":"2023-10-03T10:12:13.360956+0000","last_became_active":"2023-10-03T10:10:35.521112+0000","last_became_peered":"2023-10-03T10:10:35.521112+0000","last_unstale":"2023-10-03T10:12:13.360956+0000","last_undegraded":"2023-10-03T10:12:13.360956+0000","last_fullsized":"2023-10-03T10:12:13.360956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.594065+0000","last_change":"2023-10-03T10:09:44.176456+0000","last_active":"2023-10-03T10:10:35.594065+0000","last_peered":"2023-10-03T10:10:35.594065+0000","last_clean":"2023-10-03T10:10:35.594065+0000","last_became_active":"2023-10-03T10:09:44.176146+0000","last_became_peered":"2023-10-03T10:09:44.176146+0000","last_unstale":"2023-10-03T10:10:35.594065+0000","last_undegraded":"2023-10-03T10:10:35.594065+0000","last_fullsized":"2023-10-03T10:10:35.594065+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.514008+0000","last_change":"2023-10-03T10:09:44.176614+0000","last_active":"2023-10-03T10:10:35.514008+0000","last_peered":"2023-10-03T10:10:35.514008+0000","last_clean":"2023-10-03T10:10:35.514008+0000","last_became_active":"2023-10-03T10:09:44.176334+0000","last_became_peered":"2023-10-03T10:09:44.176334+0000","last_unstale":"2023-10-03T10:10:35.514008+0000","last_undegraded":"2023-10-03T10:10:35.514008+0000","last_fullsized":"2023-10-03T10:10:35.514008+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.517267+0000","last_change":"2023-10-03T10:09:44.179440+0000","last_active":"2023-10-03T10:10:35.517267+0000","last_peered":"2023-10-03T10:10:35.517267+0000","last_clean":"2023-10-03T10:10:35.517267+0000","last_became_active":"2023-10-03T10:09:44.179150+0000","last_became_peered":"2023-10-03T10:09:44.179150+0000","last_unstale":"2023-10-03T10:10:35.517267+0000","last_undegraded":"2023-10-03T10:10:35.517267+0000","last_fullsized":"2023-10-03T10:10:35.517267+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.516183+0000","last_change":"2023-10-03T10:09:44.166901+0000","last_active":"2023-10-03T10:10:35.516183+0000","last_peered":"2023-10-03T10:10:35.516183+0000","last_clean":"2023-10-03T10:10:35.516183+0000","last_became_active":"2023-10-03T10:09:44.166610+0000","last_became_peered":"2023-10-03T10:09:44.166610+0000","last_unstale":"2023-10-03T10:10:35.516183+0000","last_undegraded":"2023-10-03T10:10:35.516183+0000","last_fullsized":"2023-10-03T10:10:35.516183+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.516077+0000","last_change":"2023-10-03T10:09:44.165366+0000","last_active":"2023-10-03T10:10:35.516077+0000","last_peered":"2023-10-03T10:10:35.516077+0000","last_clean":"2023-10-03T10:10:35.516077+0000","last_became_active":"2023-10-03T10:09:44.165083+0000","last_became_peered":"2023-10-03T10:09:44.165083+0000","last_unstale":"2023-10-03T10:10:35.516077+0000","last_undegraded":"2023-10-03T10:10:35.516077+0000","last_fullsized":"2023-10-03T10:10:35.516077+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.361831+0000","last_change":"2023-10-03T10:10:35.521376+0000","last_active":"2023-10-03T10:12:13.361831+0000","last_peered":"2023-10-03T10:12:13.361831+0000","last_clean":"2023-10-03T10:12:13.361831+0000","last_became_active":"2023-10-03T10:10:35.521029+0000","last_became_peered":"2023-10-03T10:10:35.521029+0000","last_unstale":"2023-10-03T10:12:13.361831+0000","last_undegraded":"2023-10-03T10:12:13.361831+0000","last_fullsized":"2023-10-03T10:12:13.361831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593713+0000","last_change":"2023-10-03T10:09:44.177178+0000","last_active":"2023-10-03T10:10:35.593713+0000","last_peered":"2023-10-03T10:10:35.593713+0000","last_clean":"2023-10-03T10:10:35.593713+0000","last_became_active":"2023-10-03T10:09:44.176686+0000","last_became_peered":"2023-10-03T10:09:44.176686+0000","last_unstale":"2023-10-03T10:10:35.593713+0000","last_undegraded":"2023-10-03T10:10:35.593713+0000","last_fullsized":"2023-10-03T10:10:35.593713+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593119+0000","last_change":"2023-10-03T10:09:44.177335+0000","last_active":"2023-10-03T10:10:35.593119+0000","last_peered":"2023-10-03T10:10:35.593119+0000","last_clean":"2023-10-03T10:10:35.593119+0000","last_became_active":"2023-10-03T10:09:44.175202+0000","last_became_peered":"2023-10-03T10:09:44.175202+0000","last_unstale":"2023-10-03T10:10:35.593119+0000","last_undegraded":"2023-10-03T10:10:35.593119+0000","last_fullsized":"2023-10-03T10:10:35.593119+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.525179+0000","last_change":"2023-10-03T10:10:35.521795+0000","last_active":"2023-10-03T10:10:35.525179+0000","last_peered":"2023-10-03T10:10:35.525179+0000","last_clean":"2023-10-03T10:10:35.525179+0000","last_became_active":"2023-10-03T10:10:35.521498+0000","last_became_peered":"2023-10-03T10:10:35.521498+0000","last_unstale":"2023-10-03T10:10:35.525179+0000","last_undegraded":"2023-10-03T10:10:35.525179+0000","last_fullsized":"2023-10-03T10:10:35.525179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.517982+0000","last_change":"2023-10-03T10:09:44.182610+0000","last_active":"2023-10-03T10:10:35.517982+0000","last_peered":"2023-10-03T10:10:35.517982+0000","last_clean":"2023-10-03T10:10:35.517982+0000","last_became_active":"2023-10-03T10:09:44.180732+0000","last_became_peered":"2023-10-03T10:09:44.180732+0000","last_unstale":"2023-10-03T10:10:35.517982+0000","last_undegraded":"2023-10-03T10:10:35.517982+0000","last_fullsized":"2023-10-03T10:10:35.517982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593219+0000","last_change":"2023-10-03T10:09:44.173182+0000","last_active":"2023-10-03T10:10:35.593219+0000","last_peered":"2023-10-03T10:10:35.593219+0000","last_clean":"2023-10-03T10:10:35.593219+0000","last_became_active":"2023-10-03T10:09:44.172685+0000","last_became_peered":"2023-10-03T10:09:44.172685+0000","last_unstale":"2023-10-03T10:10:35.593219+0000","last_undegraded":"2023-10-03T10:10:35.593219+0000","last_fullsized":"2023-10-03T10:10:35.593219+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593586+0000","last_change":"2023-10-03T10:09:44.177910+0000","last_active":"2023-10-03T10:10:35.593586+0000","last_peered":"2023-10-03T10:10:35.593586+0000","last_clean":"2023-10-03T10:10:35.593586+0000","last_became_active":"2023-10-03T10:09:44.177618+0000","last_became_peered":"2023-10-03T10:09:44.177618+0000","last_unstale":"2023-10-03T10:10:35.593586+0000","last_undegraded":"2023-10-03T10:10:35.593586+0000","last_fullsized":"2023-10-03T10:10:35.593586+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593571+0000","last_change":"2023-10-03T10:09:44.176287+0000","last_active":"2023-10-03T10:10:35.593571+0000","last_peered":"2023-10-03T10:10:35.593571+0000","last_clean":"2023-10-03T10:10:35.593571+0000","last_became_active":"2023-10-03T10:09:44.175320+0000","last_became_peered":"2023-10-03T10:09:44.175320+0000","last_unstale":"2023-10-03T10:10:35.593571+0000","last_undegraded":"2023-10-03T10:10:35.593571+0000","last_fullsized":"2023-10-03T10:10:35.593571+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518762+0000","last_change":"2023-10-03T10:09:44.187148+0000","last_active":"2023-10-03T10:10:35.518762+0000","last_peered":"2023-10-03T10:10:35.518762+0000","last_clean":"2023-10-03T10:10:35.518762+0000","last_became_active":"2023-10-03T10:09:44.186523+0000","last_became_peered":"2023-10-03T10:09:44.186523+0000","last_unstale":"2023-10-03T10:10:35.518762+0000","last_undegraded":"2023-10-03T10:10:35.518762+0000","last_fullsized":"2023-10-03T10:10:35.518762+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593909+0000","last_change":"2023-10-03T10:09:43.745046+0000","last_active":"2023-10-03T10:10:35.593909+0000","last_peered":"2023-10-03T10:10:35.593909+0000","last_clean":"2023-10-03T10:10:35.593909+0000","last_became_active":"2023-10-03T10:09:43.744724+0000","last_became_peered":"2023-10-03T10:09:43.744724+0000","last_unstale":"2023-10-03T10:10:35.593909+0000","last_undegraded":"2023-10-03T10:10:35.593909+0000","last_fullsized":"2023-10-03T10:10:35.593909+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518719+0000","last_change":"2023-10-03T10:09:44.190869+0000","last_active":"2023-10-03T10:10:35.518719+0000","last_peered":"2023-10-03T10:10:35.518719+0000","last_clean":"2023-10-03T10:10:35.518719+0000","last_became_active":"2023-10-03T10:09:44.190207+0000","last_became_peered":"2023-10-03T10:09:44.190207+0000","last_unstale":"2023-10-03T10:10:35.518719+0000","last_undegraded":"2023-10-03T10:10:35.518719+0000","last_fullsized":"2023-10-03T10:10:35.518719+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.592889+0000","last_change":"2023-10-03T10:09:43.750891+0000","last_active":"2023-10-03T10:10:35.592889+0000","last_peered":"2023-10-03T10:10:35.592889+0000","last_clean":"2023-10-03T10:10:35.592889+0000","last_became_active":"2023-10-03T10:09:43.750356+0000","last_became_peered":"2023-10-03T10:09:43.750356+0000","last_unstale":"2023-10-03T10:10:35.592889+0000","last_undegraded":"2023-10-03T10:10:35.592889+0000","last_fullsized":"2023-10-03T10:10:35.592889+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.515727+0000","last_change":"2023-10-03T10:09:44.183145+0000","last_active":"2023-10-03T10:10:35.515727+0000","last_peered":"2023-10-03T10:10:35.515727+0000","last_clean":"2023-10-03T10:10:35.515727+0000","last_became_active":"2023-10-03T10:09:44.182816+0000","last_became_peered":"2023-10-03T10:09:44.182816+0000","last_unstale":"2023-10-03T10:10:35.515727+0000","last_undegraded":"2023-10-03T10:10:35.515727+0000","last_fullsized":"2023-10-03T10:10:35.515727+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.517490+0000","last_change":"2023-10-03T10:09:43.749289+0000","last_active":"2023-10-03T10:10:35.517490+0000","last_peered":"2023-10-03T10:10:35.517490+0000","last_clean":"2023-10-03T10:10:35.517490+0000","last_became_active":"2023-10-03T10:09:43.748981+0000","last_became_peered":"2023-10-03T10:09:43.748981+0000","last_unstale":"2023-10-03T10:10:35.517490+0000","last_undegraded":"2023-10-03T10:10:35.517490+0000","last_fullsized":"2023-10-03T10:10:35.517490+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.519771+0000","last_change":"2023-10-03T10:09:44.192692+0000","last_active":"2023-10-03T10:10:35.519771+0000","last_peered":"2023-10-03T10:10:35.519771+0000","last_clean":"2023-10-03T10:10:35.519771+0000","last_became_active":"2023-10-03T10:09:44.192390+0000","last_became_peered":"2023-10-03T10:09:44.192390+0000","last_unstale":"2023-10-03T10:10:35.519771+0000","last_undegraded":"2023-10-03T10:10:35.519771+0000","last_fullsized":"2023-10-03T10:10:35.519771+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.515989+0000","last_change":"2023-10-03T10:09:43.754802+0000","last_active":"2023-10-03T10:10:35.515989+0000","last_peered":"2023-10-03T10:10:35.515989+0000","last_clean":"2023-10-03T10:10:35.515989+0000","last_became_active":"2023-10-03T10:09:43.754512+0000","last_became_peered":"2023-10-03T10:09:43.754512+0000","last_unstale":"2023-10-03T10:10:35.515989+0000","last_undegraded":"2023-10-03T10:10:35.515989+0000","last_fullsized":"2023-10-03T10:10:35.515989+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518307+0000","last_change":"2023-10-03T10:09:44.192798+0000","last_active":"2023-10-03T10:10:35.518307+0000","last_peered":"2023-10-03T10:10:35.518307+0000","last_clean":"2023-10-03T10:10:35.518307+0000","last_became_active":"2023-10-03T10:09:44.190787+0000","last_became_peered":"2023-10-03T10:09:44.190787+0000","last_unstale":"2023-10-03T10:10:35.518307+0000","last_undegraded":"2023-10-03T10:10:35.518307+0000","last_fullsized":"2023-10-03T10:10:35.518307+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.361838+0000","last_change":"2023-10-03T10:09:43.756958+0000","last_active":"2023-10-03T10:12:13.361838+0000","last_peered":"2023-10-03T10:12:13.361838+0000","last_clean":"2023-10-03T10:12:13.361838+0000","last_became_active":"2023-10-03T10:09:43.756463+0000","last_became_peered":"2023-10-03T10:09:43.756463+0000","last_unstale":"2023-10-03T10:12:13.361838+0000","last_undegraded":"2023-10-03T10:12:13.361838+0000","last_fullsized":"2023-10-03T10:12:13.361838+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518495+0000","last_change":"2023-10-03T10:09:44.184790+0000","last_active":"2023-10-03T10:10:35.518495+0000","last_peered":"2023-10-03T10:10:35.518495+0000","last_clean":"2023-10-03T10:10:35.518495+0000","last_became_active":"2023-10-03T10:09:44.184531+0000","last_became_peered":"2023-10-03T10:09:44.184531+0000","last_unstale":"2023-10-03T10:10:35.518495+0000","last_undegraded":"2023-10-03T10:10:35.518495+0000","last_fullsized":"2023-10-03T10:10:35.518495+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"16'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.515410+0000","last_change":"2023-10-03T10:09:43.756598+0000","last_active":"2023-10-03T10:10:35.515410+0000","last_peered":"2023-10-03T10:10:35.515410+0000","last_clean":"2023-10-03T10:10:35.515410+0000","last_became_active":"2023-10-03T10:09:43.754974+0000","last_became_peered":"2023-10-03T10:09:43.754974+0000","last_unstale":"2023-10-03T10:10:35.515410+0000","last_undegraded":"2023-10-03T10:10:35.515410+0000","last_fullsized":"2023-10-03T10:10:35.515410+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.594021+0000","last_change":"2023-10-03T10:09:44.175504+0000","last_active":"2023-10-03T10:10:35.594021+0000","last_peered":"2023-10-03T10:10:35.594021+0000","last_clean":"2023-10-03T10:10:35.594021+0000","last_became_active":"2023-10-03T10:09:44.175174+0000","last_became_peered":"2023-10-03T10:09:44.175174+0000","last_unstale":"2023-10-03T10:10:35.594021+0000","last_undegraded":"2023-10-03T10:10:35.594021+0000","last_fullsized":"2023-10-03T10:10:35.594021+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"21'12","reported_seq":26,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.380292+0000","last_change":"2023-10-03T10:09:43.758711+0000","last_active":"2023-10-03T10:12:13.380292+0000","last_peered":"2023-10-03T10:12:13.380292+0000","last_clean":"2023-10-03T10:12:13.380292+0000","last_became_active":"2023-10-03T10:09:43.758392+0000","last_became_peered":"2023-10-03T10:09:43.758392+0000","last_unstale":"2023-10-03T10:12:13.380292+0000","last_undegraded":"2023-10-03T10:12:13.380292+0000","last_fullsized":"2023-10-03T10:12:13.380292+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3563,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.361056+0000","last_change":"2023-10-03T10:09:44.196559+0000","last_active":"2023-10-03T10:12:13.361056+0000","last_peered":"2023-10-03T10:12:13.361056+0000","last_clean":"2023-10-03T10:12:13.361056+0000","last_became_active":"2023-10-03T10:09:44.196280+0000","last_became_peered":"2023-10-03T10:09:44.196280+0000","last_unstale":"2023-10-03T10:12:13.361056+0000","last_undegraded":"2023-10-03T10:12:13.361056+0000","last_fullsized":"2023-10-03T10:12:13.361056+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518819+0000","last_change":"2023-10-03T10:09:43.751877+0000","last_active":"2023-10-03T10:10:35.518819+0000","last_peered":"2023-10-03T10:10:35.518819+0000","last_clean":"2023-10-03T10:10:35.518819+0000","last_became_active":"2023-10-03T10:09:43.749870+0000","last_became_peered":"2023-10-03T10:09:43.749870+0000","last_unstale":"2023-10-03T10:10:35.518819+0000","last_undegraded":"2023-10-03T10:10:35.518819+0000","last_fullsized":"2023-10-03T10:10:35.518819+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.360943+0000","last_change":"2023-10-03T10:10:35.595943+0000","last_active":"2023-10-03T10:12:13.360943+0000","last_peered":"2023-10-03T10:12:13.360943+0000","last_clean":"2023-10-03T10:12:13.360943+0000","last_became_active":"2023-10-03T10:10:35.595469+0000","last_became_peered":"2023-10-03T10:10:35.595469+0000","last_unstale":"2023-10-03T10:12:13.360943+0000","last_undegraded":"2023-10-03T10:12:13.360943+0000","last_fullsized":"2023-10-03T10:12:13.360943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.513785+0000","last_change":"2023-10-03T10:09:43.755271+0000","last_active":"2023-10-03T10:10:35.513785+0000","last_peered":"2023-10-03T10:10:35.513785+0000","last_clean":"2023-10-03T10:10:35.513785+0000","last_became_active":"2023-10-03T10:09:43.754957+0000","last_became_peered":"2023-10-03T10:09:43.754957+0000","last_unstale":"2023-10-03T10:10:35.513785+0000","last_undegraded":"2023-10-03T10:10:35.513785+0000","last_fullsized":"2023-10-03T10:10:35.513785+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518823+0000","last_change":"2023-10-03T10:09:44.187333+0000","last_active":"2023-10-03T10:10:35.518823+0000","last_peered":"2023-10-03T10:10:35.518823+0000","last_clean":"2023-10-03T10:10:35.518823+0000","last_became_active":"2023-10-03T10:09:44.186824+0000","last_became_peered":"2023-10-03T10:09:44.186824+0000","last_unstale":"2023-10-03T10:10:35.518823+0000","last_undegraded":"2023-10-03T10:10:35.518823+0000","last_fullsized":"2023-10-03T10:10:35.518823+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518372+0000","last_change":"2023-10-03T10:09:43.758995+0000","last_active":"2023-10-03T10:10:35.518372+0000","last_peered":"2023-10-03T10:10:35.518372+0000","last_clean":"2023-10-03T10:10:35.518372+0000","last_became_active":"2023-10-03T10:09:43.758565+0000","last_became_peered":"2023-10-03T10:09:43.758565+0000","last_unstale":"2023-10-03T10:10:35.518372+0000","last_undegraded":"2023-10-03T10:10:35.518372+0000","last_fullsized":"2023-10-03T10:10:35.518372+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.515894+0000","last_change":"2023-10-03T10:09:44.166557+0000","last_active":"2023-10-03T10:10:35.515894+0000","last_peered":"2023-10-03T10:10:35.515894+0000","last_clean":"2023-10-03T10:10:35.515894+0000","last_became_active":"2023-10-03T10:09:44.166300+0000","last_became_peered":"2023-10-03T10:09:44.166300+0000","last_unstale":"2023-10-03T10:10:35.515894+0000","last_undegraded":"2023-10-03T10:10:35.515894+0000","last_fullsized":"2023-10-03T10:10:35.515894+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"16'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593762+0000","last_change":"2023-10-03T10:09:43.748084+0000","last_active":"2023-10-03T10:10:35.593762+0000","last_peered":"2023-10-03T10:10:35.593762+0000","last_clean":"2023-10-03T10:10:35.593762+0000","last_became_active":"2023-10-03T10:09:43.747377+0000","last_became_peered":"2023-10-03T10:09:43.747377+0000","last_unstale":"2023-10-03T10:10:35.593762+0000","last_undegraded":"2023-10-03T10:10:35.593762+0000","last_fullsized":"2023-10-03T10:10:35.593762+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.519686+0000","last_change":"2023-10-03T10:09:44.190767+0000","last_active":"2023-10-03T10:10:35.519686+0000","last_peered":"2023-10-03T10:10:35.519686+0000","last_clean":"2023-10-03T10:10:35.519686+0000","last_became_active":"2023-10-03T10:09:44.190514+0000","last_became_peered":"2023-10-03T10:09:44.190514+0000","last_unstale":"2023-10-03T10:10:35.519686+0000","last_undegraded":"2023-10-03T10:10:35.519686+0000","last_fullsized":"2023-10-03T10:10:35.519686+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.515759+0000","last_change":"2023-10-03T10:09:43.748461+0000","last_active":"2023-10-03T10:10:35.515759+0000","last_peered":"2023-10-03T10:10:35.515759+0000","last_clean":"2023-10-03T10:10:35.515759+0000","last_became_active":"2023-10-03T10:09:43.747372+0000","last_became_peered":"2023-10-03T10:09:43.747372+0000","last_unstale":"2023-10-03T10:10:35.515759+0000","last_undegraded":"2023-10-03T10:10:35.515759+0000","last_fullsized":"2023-10-03T10:10:35.515759+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.528421+0000","last_change":"2023-10-03T10:10:35.525502+0000","last_active":"2023-10-03T10:10:35.528421+0000","last_peered":"2023-10-03T10:10:35.528421+0000","last_clean":"2023-10-03T10:10:35.528421+0000","last_became_active":"2023-10-03T10:10:35.525214+0000","last_became_peered":"2023-10-03T10:10:35.525214+0000","last_unstale":"2023-10-03T10:10:35.528421+0000","last_undegraded":"2023-10-03T10:10:35.528421+0000","last_fullsized":"2023-10-03T10:10:35.528421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"16'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.361932+0000","last_change":"2023-10-03T10:09:43.756751+0000","last_active":"2023-10-03T10:12:13.361932+0000","last_peered":"2023-10-03T10:12:13.361932+0000","last_clean":"2023-10-03T10:12:13.361932+0000","last_became_active":"2023-10-03T10:09:43.756079+0000","last_became_peered":"2023-10-03T10:09:43.756079+0000","last_unstale":"2023-10-03T10:12:13.361932+0000","last_undegraded":"2023-10-03T10:12:13.361932+0000","last_fullsized":"2023-10-03T10:12:13.361932+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593032+0000","last_change":"2023-10-03T10:09:44.172400+0000","last_active":"2023-10-03T10:10:35.593032+0000","last_peered":"2023-10-03T10:10:35.593032+0000","last_clean":"2023-10-03T10:10:35.593032+0000","last_became_active":"2023-10-03T10:09:44.172095+0000","last_became_peered":"2023-10-03T10:09:44.172095+0000","last_unstale":"2023-10-03T10:10:35.593032+0000","last_undegraded":"2023-10-03T10:10:35.593032+0000","last_fullsized":"2023-10-03T10:10:35.593032+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518935+0000","last_change":"2023-10-03T10:09:43.760622+0000","last_active":"2023-10-03T10:10:35.518935+0000","last_peered":"2023-10-03T10:10:35.518935+0000","last_clean":"2023-10-03T10:10:35.518935+0000","last_became_active":"2023-10-03T10:09:43.760371+0000","last_became_peered":"2023-10-03T10:09:43.760371+0000","last_unstale":"2023-10-03T10:10:35.518935+0000","last_undegraded":"2023-10-03T10:10:35.518935+0000","last_fullsized":"2023-10-03T10:10:35.518935+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.360867+0000","last_change":"2023-10-03T10:09:44.196062+0000","last_active":"2023-10-03T10:12:13.360867+0000","last_peered":"2023-10-03T10:12:13.360867+0000","last_clean":"2023-10-03T10:12:13.360867+0000","last_became_active":"2023-10-03T10:09:44.195841+0000","last_became_peered":"2023-10-03T10:09:44.195841+0000","last_unstale":"2023-10-03T10:12:13.360867+0000","last_undegraded":"2023-10-03T10:12:13.360867+0000","last_fullsized":"2023-10-03T10:12:13.360867+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.515441+0000","last_change":"2023-10-03T10:09:43.756885+0000","last_active":"2023-10-03T10:10:35.515441+0000","last_peered":"2023-10-03T10:10:35.515441+0000","last_clean":"2023-10-03T10:10:35.515441+0000","last_became_active":"2023-10-03T10:09:43.756588+0000","last_became_peered":"2023-10-03T10:09:43.756588+0000","last_unstale":"2023-10-03T10:10:35.515441+0000","last_undegraded":"2023-10-03T10:10:35.515441+0000","last_fullsized":"2023-10-03T10:10:35.515441+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.514541+0000","last_change":"2023-10-03T10:09:44.181603+0000","last_active":"2023-10-03T10:10:35.514541+0000","last_peered":"2023-10-03T10:10:35.514541+0000","last_clean":"2023-10-03T10:10:35.514541+0000","last_became_active":"2023-10-03T10:09:44.181125+0000","last_became_peered":"2023-10-03T10:09:44.181125+0000","last_unstale":"2023-10-03T10:10:35.514541+0000","last_undegraded":"2023-10-03T10:10:35.514541+0000","last_fullsized":"2023-10-03T10:10:35.514541+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518701+0000","last_change":"2023-10-03T10:09:43.751873+0000","last_active":"2023-10-03T10:10:35.518701+0000","last_peered":"2023-10-03T10:10:35.518701+0000","last_clean":"2023-10-03T10:10:35.518701+0000","last_became_active":"2023-10-03T10:09:43.751324+0000","last_became_peered":"2023-10-03T10:09:43.751324+0000","last_unstale":"2023-10-03T10:10:35.518701+0000","last_undegraded":"2023-10-03T10:10:35.518701+0000","last_fullsized":"2023-10-03T10:10:35.518701+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.515356+0000","last_change":"2023-10-03T10:09:44.170676+0000","last_active":"2023-10-03T10:10:35.515356+0000","last_peered":"2023-10-03T10:10:35.515356+0000","last_clean":"2023-10-03T10:10:35.515356+0000","last_became_active":"2023-10-03T10:09:44.170398+0000","last_became_peered":"2023-10-03T10:09:44.170398+0000","last_unstale":"2023-10-03T10:10:35.515356+0000","last_undegraded":"2023-10-03T10:10:35.515356+0000","last_fullsized":"2023-10-03T10:10:35.515356+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518265+0000","last_change":"2023-10-03T10:09:43.760711+0000","last_active":"2023-10-03T10:10:35.518265+0000","last_peered":"2023-10-03T10:10:35.518265+0000","last_clean":"2023-10-03T10:10:35.518265+0000","last_became_active":"2023-10-03T10:09:43.760432+0000","last_became_peered":"2023-10-03T10:09:43.760432+0000","last_unstale":"2023-10-03T10:10:35.518265+0000","last_undegraded":"2023-10-03T10:10:35.518265+0000","last_fullsized":"2023-10-03T10:10:35.518265+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.361648+0000","last_change":"2023-10-03T10:10:35.595831+0000","last_active":"2023-10-03T10:12:13.361648+0000","last_peered":"2023-10-03T10:12:13.361648+0000","last_clean":"2023-10-03T10:12:13.361648+0000","last_became_active":"2023-10-03T10:10:35.595457+0000","last_became_peered":"2023-10-03T10:10:35.595457+0000","last_unstale":"2023-10-03T10:12:13.361648+0000","last_undegraded":"2023-10-03T10:12:13.361648+0000","last_fullsized":"2023-10-03T10:12:13.361648+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518180+0000","last_change":"2023-10-03T10:09:43.758846+0000","last_active":"2023-10-03T10:10:35.518180+0000","last_peered":"2023-10-03T10:10:35.518180+0000","last_clean":"2023-10-03T10:10:35.518180+0000","last_became_active":"2023-10-03T10:09:43.757973+0000","last_became_peered":"2023-10-03T10:09:43.757973+0000","last_unstale":"2023-10-03T10:10:35.518180+0000","last_undegraded":"2023-10-03T10:10:35.518180+0000","last_fullsized":"2023-10-03T10:10:35.518180+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.362091+0000","last_change":"2023-10-03T10:09:44.196056+0000","last_active":"2023-10-03T10:12:13.362091+0000","last_peered":"2023-10-03T10:12:13.362091+0000","last_clean":"2023-10-03T10:12:13.362091+0000","last_became_active":"2023-10-03T10:09:44.195817+0000","last_became_peered":"2023-10-03T10:09:44.195817+0000","last_unstale":"2023-10-03T10:12:13.362091+0000","last_undegraded":"2023-10-03T10:12:13.362091+0000","last_fullsized":"2023-10-03T10:12:13.362091+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.517314+0000","last_change":"2023-10-03T10:09:43.761068+0000","last_active":"2023-10-03T10:10:35.517314+0000","last_peered":"2023-10-03T10:10:35.517314+0000","last_clean":"2023-10-03T10:10:35.517314+0000","last_became_active":"2023-10-03T10:09:43.760825+0000","last_became_peered":"2023-10-03T10:09:43.760825+0000","last_unstale":"2023-10-03T10:10:35.517314+0000","last_undegraded":"2023-10-03T10:10:35.517314+0000","last_fullsized":"2023-10-03T10:10:35.517314+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593797+0000","last_change":"2023-10-03T10:09:44.178856+0000","last_active":"2023-10-03T10:10:35.593797+0000","last_peered":"2023-10-03T10:10:35.593797+0000","last_clean":"2023-10-03T10:10:35.593797+0000","last_became_active":"2023-10-03T10:09:44.178410+0000","last_became_peered":"2023-10-03T10:09:44.178410+0000","last_unstale":"2023-10-03T10:10:35.593797+0000","last_undegraded":"2023-10-03T10:10:35.593797+0000","last_fullsized":"2023-10-03T10:10:35.593797+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.592583+0000","last_change":"2023-10-03T10:09:39.589080+0000","last_active":"2023-10-03T10:10:35.592583+0000","last_peered":"2023-10-03T10:10:35.592583+0000","last_clean":"2023-10-03T10:10:35.592583+0000","last_became_active":"2023-10-03T10:09:39.588651+0000","last_became_peered":"2023-10-03T10:09:39.588651+0000","last_unstale":"2023-10-03T10:10:35.592583+0000","last_undegraded":"2023-10-03T10:10:35.592583+0000","last_fullsized":"2023-10-03T10:10:35.592583+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.361743+0000","last_change":"2023-10-03T10:09:43.756053+0000","last_active":"2023-10-03T10:12:13.361743+0000","last_peered":"2023-10-03T10:12:13.361743+0000","last_clean":"2023-10-03T10:12:13.361743+0000","last_became_active":"2023-10-03T10:09:43.755708+0000","last_became_peered":"2023-10-03T10:09:43.755708+0000","last_unstale":"2023-10-03T10:12:13.361743+0000","last_undegraded":"2023-10-03T10:12:13.361743+0000","last_fullsized":"2023-10-03T10:12:13.361743+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593942+0000","last_change":"2023-10-03T10:09:44.176964+0000","last_active":"2023-10-03T10:10:35.593942+0000","last_peered":"2023-10-03T10:10:35.593942+0000","last_clean":"2023-10-03T10:10:35.593942+0000","last_became_active":"2023-10-03T10:09:44.176671+0000","last_became_peered":"2023-10-03T10:09:44.176671+0000","last_unstale":"2023-10-03T10:10:35.593942+0000","last_undegraded":"2023-10-03T10:10:35.593942+0000","last_fullsized":"2023-10-03T10:10:35.593942+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518390+0000","last_change":"2023-10-03T10:09:39.586735+0000","last_active":"2023-10-03T10:10:35.518390+0000","last_peered":"2023-10-03T10:10:35.518390+0000","last_clean":"2023-10-03T10:10:35.518390+0000","last_became_active":"2023-10-03T10:09:39.586440+0000","last_became_peered":"2023-10-03T10:09:39.586440+0000","last_unstale":"2023-10-03T10:10:35.518390+0000","last_undegraded":"2023-10-03T10:10:35.518390+0000","last_fullsized":"2023-10-03T10:10:35.518390+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.514163+0000","last_change":"2023-10-03T10:09:43.748458+0000","last_active":"2023-10-03T10:10:35.514163+0000","last_peered":"2023-10-03T10:10:35.514163+0000","last_clean":"2023-10-03T10:10:35.514163+0000","last_became_active":"2023-10-03T10:09:43.747721+0000","last_became_peered":"2023-10-03T10:09:43.747721+0000","last_unstale":"2023-10-03T10:10:35.514163+0000","last_undegraded":"2023-10-03T10:10:35.514163+0000","last_fullsized":"2023-10-03T10:10:35.514163+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593338+0000","last_change":"2023-10-03T10:09:44.176165+0000","last_active":"2023-10-03T10:10:35.593338+0000","last_peered":"2023-10-03T10:10:35.593338+0000","last_clean":"2023-10-03T10:10:35.593338+0000","last_became_active":"2023-10-03T10:09:44.175853+0000","last_became_peered":"2023-10-03T10:09:44.175853+0000","last_unstale":"2023-10-03T10:10:35.593338+0000","last_undegraded":"2023-10-03T10:10:35.593338+0000","last_fullsized":"2023-10-03T10:10:35.593338+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.360681+0000","last_change":"2023-10-03T10:09:39.590114+0000","last_active":"2023-10-03T10:12:13.360681+0000","last_peered":"2023-10-03T10:12:13.360681+0000","last_clean":"2023-10-03T10:12:13.360681+0000","last_became_active":"2023-10-03T10:09:39.589663+0000","last_became_peered":"2023-10-03T10:09:39.589663+0000","last_unstale":"2023-10-03T10:12:13.360681+0000","last_undegraded":"2023-10-03T10:12:13.360681+0000","last_fullsized":"2023-10-03T10:12:13.360681+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593543+0000","last_change":"2023-10-03T10:09:43.748267+0000","last_active":"2023-10-03T10:10:35.593543+0000","last_peered":"2023-10-03T10:10:35.593543+0000","last_clean":"2023-10-03T10:10:35.593543+0000","last_became_active":"2023-10-03T10:09:43.747789+0000","last_became_peered":"2023-10-03T10:09:43.747789+0000","last_unstale":"2023-10-03T10:10:35.593543+0000","last_undegraded":"2023-10-03T10:10:35.593543+0000","last_fullsized":"2023-10-03T10:10:35.593543+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593698+0000","last_change":"2023-10-03T10:09:44.178680+0000","last_active":"2023-10-03T10:10:35.593698+0000","last_peered":"2023-10-03T10:10:35.593698+0000","last_clean":"2023-10-03T10:10:35.593698+0000","last_became_active":"2023-10-03T10:09:44.178114+0000","last_became_peered":"2023-10-03T10:09:44.178114+0000","last_unstale":"2023-10-03T10:10:35.593698+0000","last_undegraded":"2023-10-03T10:10:35.593698+0000","last_fullsized":"2023-10-03T10:10:35.593698+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518722+0000","last_change":"2023-10-03T10:09:39.587728+0000","last_active":"2023-10-03T10:10:35.518722+0000","last_peered":"2023-10-03T10:10:35.518722+0000","last_clean":"2023-10-03T10:10:35.518722+0000","last_became_active":"2023-10-03T10:09:39.587441+0000","last_became_peered":"2023-10-03T10:09:39.587441+0000","last_unstale":"2023-10-03T10:10:35.518722+0000","last_undegraded":"2023-10-03T10:10:35.518722+0000","last_fullsized":"2023-10-03T10:10:35.518722+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593334+0000","last_change":"2023-10-03T10:09:38.585668+0000","last_active":"2023-10-03T10:10:35.593334+0000","last_peered":"2023-10-03T10:10:35.593334+0000","last_clean":"2023-10-03T10:10:35.593334+0000","last_became_active":"2023-10-03T10:09:38.585324+0000","last_became_peered":"2023-10-03T10:09:38.585324+0000","last_unstale":"2023-10-03T10:10:35.593334+0000","last_undegraded":"2023-10-03T10:10:35.593334+0000","last_fullsized":"2023-10-03T10:10:35.593334+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:37.570571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:37.570571+0000","last_clean_scrub_stamp":"2023-10-03T10:09:37.570571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518216+0000","last_change":"2023-10-03T10:09:43.758096+0000","last_active":"2023-10-03T10:10:35.518216+0000","last_peered":"2023-10-03T10:10:35.518216+0000","last_clean":"2023-10-03T10:10:35.518216+0000","last_became_active":"2023-10-03T10:09:43.757689+0000","last_became_peered":"2023-10-03T10:09:43.757689+0000","last_unstale":"2023-10-03T10:10:35.518216+0000","last_undegraded":"2023-10-03T10:10:35.518216+0000","last_fullsized":"2023-10-03T10:10:35.518216+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.519138+0000","last_change":"2023-10-03T10:09:44.192781+0000","last_active":"2023-10-03T10:10:35.519138+0000","last_peered":"2023-10-03T10:10:35.519138+0000","last_clean":"2023-10-03T10:10:35.519138+0000","last_became_active":"2023-10-03T10:09:44.190597+0000","last_became_peered":"2023-10-03T10:09:44.190597+0000","last_unstale":"2023-10-03T10:10:35.519138+0000","last_undegraded":"2023-10-03T10:10:35.519138+0000","last_fullsized":"2023-10-03T10:10:35.519138+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.517402+0000","last_change":"2023-10-03T10:09:39.586806+0000","last_active":"2023-10-03T10:10:35.517402+0000","last_peered":"2023-10-03T10:10:35.517402+0000","last_clean":"2023-10-03T10:10:35.517402+0000","last_became_active":"2023-10-03T10:09:39.586481+0000","last_became_peered":"2023-10-03T10:09:39.586481+0000","last_unstale":"2023-10-03T10:10:35.517402+0000","last_undegraded":"2023-10-03T10:10:35.517402+0000","last_fullsized":"2023-10-03T10:10:35.517402+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.514120+0000","last_change":"2023-10-03T10:09:43.747783+0000","last_active":"2023-10-03T10:10:35.514120+0000","last_peered":"2023-10-03T10:10:35.514120+0000","last_clean":"2023-10-03T10:10:35.514120+0000","last_became_active":"2023-10-03T10:09:43.746009+0000","last_became_peered":"2023-10-03T10:09:43.746009+0000","last_unstale":"2023-10-03T10:10:35.514120+0000","last_undegraded":"2023-10-03T10:10:35.514120+0000","last_fullsized":"2023-10-03T10:10:35.514120+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.515983+0000","last_change":"2023-10-03T10:09:44.181359+0000","last_active":"2023-10-03T10:10:35.515983+0000","last_peered":"2023-10-03T10:10:35.515983+0000","last_clean":"2023-10-03T10:10:35.515983+0000","last_became_active":"2023-10-03T10:09:44.181075+0000","last_became_peered":"2023-10-03T10:09:44.181075+0000","last_unstale":"2023-10-03T10:10:35.515983+0000","last_undegraded":"2023-10-03T10:10:35.515983+0000","last_fullsized":"2023-10-03T10:10:35.515983+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.519002+0000","last_change":"2023-10-03T10:09:39.588827+0000","last_active":"2023-10-03T10:10:35.519002+0000","last_peered":"2023-10-03T10:10:35.519002+0000","last_clean":"2023-10-03T10:10:35.519002+0000","last_became_active":"2023-10-03T10:09:39.588531+0000","last_became_peered":"2023-10-03T10:09:39.588531+0000","last_unstale":"2023-10-03T10:10:35.519002+0000","last_undegraded":"2023-10-03T10:10:35.519002+0000","last_fullsized":"2023-10-03T10:10:35.519002+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593205+0000","last_change":"2023-10-03T10:09:43.746339+0000","last_active":"2023-10-03T10:10:35.593205+0000","last_peered":"2023-10-03T10:10:35.593205+0000","last_clean":"2023-10-03T10:10:35.593205+0000","last_became_active":"2023-10-03T10:09:43.746051+0000","last_became_peered":"2023-10-03T10:09:43.746051+0000","last_unstale":"2023-10-03T10:10:35.593205+0000","last_undegraded":"2023-10-03T10:10:35.593205+0000","last_fullsized":"2023-10-03T10:10:35.593205+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.514356+0000","last_change":"2023-10-03T10:09:44.165684+0000","last_active":"2023-10-03T10:10:35.514356+0000","last_peered":"2023-10-03T10:10:35.514356+0000","last_clean":"2023-10-03T10:10:35.514356+0000","last_became_active":"2023-10-03T10:09:44.165422+0000","last_became_peered":"2023-10-03T10:09:44.165422+0000","last_unstale":"2023-10-03T10:10:35.514356+0000","last_undegraded":"2023-10-03T10:10:35.514356+0000","last_fullsized":"2023-10-03T10:10:35.514356+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.519510+0000","last_change":"2023-10-03T10:09:43.760807+0000","last_active":"2023-10-03T10:10:35.519510+0000","last_peered":"2023-10-03T10:10:35.519510+0000","last_clean":"2023-10-03T10:10:35.519510+0000","last_became_active":"2023-10-03T10:09:43.760517+0000","last_became_peered":"2023-10-03T10:09:43.760517+0000","last_unstale":"2023-10-03T10:10:35.519510+0000","last_undegraded":"2023-10-03T10:10:35.519510+0000","last_fullsized":"2023-10-03T10:10:35.519510+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.592628+0000","last_change":"2023-10-03T10:09:39.586702+0000","last_active":"2023-10-03T10:10:35.592628+0000","last_peered":"2023-10-03T10:10:35.592628+0000","last_clean":"2023-10-03T10:10:35.592628+0000","last_became_active":"2023-10-03T10:09:39.586153+0000","last_became_peered":"2023-10-03T10:09:39.586153+0000","last_unstale":"2023-10-03T10:10:35.592628+0000","last_undegraded":"2023-10-03T10:10:35.592628+0000","last_fullsized":"2023-10-03T10:10:35.592628+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.517371+0000","last_change":"2023-10-03T10:09:44.192664+0000","last_active":"2023-10-03T10:10:35.517371+0000","last_peered":"2023-10-03T10:10:35.517371+0000","last_clean":"2023-10-03T10:10:35.517371+0000","last_became_active":"2023-10-03T10:09:44.190513+0000","last_became_peered":"2023-10-03T10:09:44.190513+0000","last_unstale":"2023-10-03T10:10:35.517371+0000","last_undegraded":"2023-10-03T10:10:35.517371+0000","last_fullsized":"2023-10-03T10:10:35.517371+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.519596+0000","last_change":"2023-10-03T10:09:39.588950+0000","last_active":"2023-10-03T10:10:35.519596+0000","last_peered":"2023-10-03T10:10:35.519596+0000","last_clean":"2023-10-03T10:10:35.519596+0000","last_became_active":"2023-10-03T10:09:39.588601+0000","last_became_peered":"2023-10-03T10:09:39.588601+0000","last_unstale":"2023-10-03T10:10:35.519596+0000","last_undegraded":"2023-10-03T10:10:35.519596+0000","last_fullsized":"2023-10-03T10:10:35.519596+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"17'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.516197+0000","last_change":"2023-10-03T10:09:43.752793+0000","last_active":"2023-10-03T10:10:35.516197+0000","last_peered":"2023-10-03T10:10:35.516197+0000","last_clean":"2023-10-03T10:10:35.516197+0000","last_became_active":"2023-10-03T10:09:43.752374+0000","last_became_peered":"2023-10-03T10:09:43.752374+0000","last_unstale":"2023-10-03T10:10:35.516197+0000","last_undegraded":"2023-10-03T10:10:35.516197+0000","last_fullsized":"2023-10-03T10:10:35.516197+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.360862+0000","last_change":"2023-10-03T10:10:35.521309+0000","last_active":"2023-10-03T10:12:13.360862+0000","last_peered":"2023-10-03T10:12:13.360862+0000","last_clean":"2023-10-03T10:12:13.360862+0000","last_became_active":"2023-10-03T10:10:35.520943+0000","last_became_peered":"2023-10-03T10:10:35.520943+0000","last_unstale":"2023-10-03T10:12:13.360862+0000","last_undegraded":"2023-10-03T10:12:13.360862+0000","last_fullsized":"2023-10-03T10:12:13.360862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.514259+0000","last_change":"2023-10-03T10:09:43.756460+0000","last_active":"2023-10-03T10:10:35.514259+0000","last_peered":"2023-10-03T10:10:35.514259+0000","last_clean":"2023-10-03T10:10:35.514259+0000","last_became_active":"2023-10-03T10:09:43.756164+0000","last_became_peered":"2023-10-03T10:09:43.756164+0000","last_unstale":"2023-10-03T10:10:35.514259+0000","last_undegraded":"2023-10-03T10:10:35.514259+0000","last_fullsized":"2023-10-03T10:10:35.514259+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.361562+0000","last_change":"2023-10-03T10:09:44.181915+0000","last_active":"2023-10-03T10:12:13.361562+0000","last_peered":"2023-10-03T10:12:13.361562+0000","last_clean":"2023-10-03T10:12:13.361562+0000","last_became_active":"2023-10-03T10:09:44.181645+0000","last_became_peered":"2023-10-03T10:09:44.181645+0000","last_unstale":"2023-10-03T10:12:13.361562+0000","last_undegraded":"2023-10-03T10:12:13.361562+0000","last_fullsized":"2023-10-03T10:12:13.361562+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593449+0000","last_change":"2023-10-03T10:09:43.748380+0000","last_active":"2023-10-03T10:10:35.593449+0000","last_peered":"2023-10-03T10:10:35.593449+0000","last_clean":"2023-10-03T10:10:35.593449+0000","last_became_active":"2023-10-03T10:09:43.748104+0000","last_became_peered":"2023-10-03T10:09:43.748104+0000","last_unstale":"2023-10-03T10:10:35.593449+0000","last_undegraded":"2023-10-03T10:10:35.593449+0000","last_fullsized":"2023-10-03T10:10:35.593449+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.360930+0000","last_change":"2023-10-03T10:09:44.197710+0000","last_active":"2023-10-03T10:12:13.360930+0000","last_peered":"2023-10-03T10:12:13.360930+0000","last_clean":"2023-10-03T10:12:13.360930+0000","last_became_active":"2023-10-03T10:09:44.197064+0000","last_became_peered":"2023-10-03T10:09:44.197064+0000","last_unstale":"2023-10-03T10:12:13.360930+0000","last_undegraded":"2023-10-03T10:12:13.360930+0000","last_fullsized":"2023-10-03T10:12:13.360930+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.519132+0000","last_change":"2023-10-03T10:09:43.758701+0000","last_active":"2023-10-03T10:10:35.519132+0000","last_peered":"2023-10-03T10:10:35.519132+0000","last_clean":"2023-10-03T10:10:35.519132+0000","last_became_active":"2023-10-03T10:09:43.758376+0000","last_became_peered":"2023-10-03T10:09:43.758376+0000","last_unstale":"2023-10-03T10:10:35.519132+0000","last_undegraded":"2023-10-03T10:10:35.519132+0000","last_fullsized":"2023-10-03T10:10:35.519132+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.361278+0000","last_change":"2023-10-03T10:09:44.197164+0000","last_active":"2023-10-03T10:12:13.361278+0000","last_peered":"2023-10-03T10:12:13.361278+0000","last_clean":"2023-10-03T10:12:13.361278+0000","last_became_active":"2023-10-03T10:09:44.196885+0000","last_became_peered":"2023-10-03T10:09:44.196885+0000","last_unstale":"2023-10-03T10:12:13.361278+0000","last_undegraded":"2023-10-03T10:12:13.361278+0000","last_fullsized":"2023-10-03T10:12:13.361278+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.519349+0000","last_change":"2023-10-03T10:09:43.758373+0000","last_active":"2023-10-03T10:10:35.519349+0000","last_peered":"2023-10-03T10:10:35.519349+0000","last_clean":"2023-10-03T10:10:35.519349+0000","last_became_active":"2023-10-03T10:09:43.758061+0000","last_became_peered":"2023-10-03T10:09:43.758061+0000","last_unstale":"2023-10-03T10:10:35.519349+0000","last_undegraded":"2023-10-03T10:10:35.519349+0000","last_fullsized":"2023-10-03T10:10:35.519349+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.520383+0000","last_change":"2023-10-03T10:10:35.517815+0000","last_active":"2023-10-03T10:10:35.520383+0000","last_peered":"2023-10-03T10:10:35.520383+0000","last_clean":"2023-10-03T10:10:35.520383+0000","last_became_active":"2023-10-03T10:10:35.517465+0000","last_became_peered":"2023-10-03T10:10:35.517465+0000","last_unstale":"2023-10-03T10:10:35.520383+0000","last_undegraded":"2023-10-03T10:10:35.520383+0000","last_fullsized":"2023-10-03T10:10:35.520383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"16'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.516092+0000","last_change":"2023-10-03T10:09:43.750463+0000","last_active":"2023-10-03T10:10:35.516092+0000","last_peered":"2023-10-03T10:10:35.516092+0000","last_clean":"2023-10-03T10:10:35.516092+0000","last_became_active":"2023-10-03T10:09:43.749780+0000","last_became_peered":"2023-10-03T10:09:43.749780+0000","last_unstale":"2023-10-03T10:10:35.516092+0000","last_undegraded":"2023-10-03T10:10:35.516092+0000","last_fullsized":"2023-10-03T10:10:35.516092+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.360700+0000","last_change":"2023-10-03T10:09:44.180378+0000","last_active":"2023-10-03T10:12:13.360700+0000","last_peered":"2023-10-03T10:12:13.360700+0000","last_clean":"2023-10-03T10:12:13.360700+0000","last_became_active":"2023-10-03T10:09:44.180082+0000","last_became_peered":"2023-10-03T10:09:44.180082+0000","last_unstale":"2023-10-03T10:12:13.360700+0000","last_undegraded":"2023-10-03T10:12:13.360700+0000","last_fullsized":"2023-10-03T10:12:13.360700+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593684+0000","last_change":"2023-10-03T10:09:44.178067+0000","last_active":"2023-10-03T10:10:35.593684+0000","last_peered":"2023-10-03T10:10:35.593684+0000","last_clean":"2023-10-03T10:10:35.593684+0000","last_became_active":"2023-10-03T10:09:44.175124+0000","last_became_peered":"2023-10-03T10:09:44.175124+0000","last_unstale":"2023-10-03T10:10:35.593684+0000","last_undegraded":"2023-10-03T10:10:35.593684+0000","last_fullsized":"2023-10-03T10:10:35.593684+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518308+0000","last_change":"2023-10-03T10:09:44.192156+0000","last_active":"2023-10-03T10:10:35.518308+0000","last_peered":"2023-10-03T10:10:35.518308+0000","last_clean":"2023-10-03T10:10:35.518308+0000","last_became_active":"2023-10-03T10:09:44.189296+0000","last_became_peered":"2023-10-03T10:09:44.189296+0000","last_unstale":"2023-10-03T10:10:35.518308+0000","last_undegraded":"2023-10-03T10:10:35.518308+0000","last_fullsized":"2023-10-03T10:10:35.518308+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.514071+0000","last_change":"2023-10-03T10:09:44.168473+0000","last_active":"2023-10-03T10:10:35.514071+0000","last_peered":"2023-10-03T10:10:35.514071+0000","last_clean":"2023-10-03T10:10:35.514071+0000","last_became_active":"2023-10-03T10:09:44.168155+0000","last_became_peered":"2023-10-03T10:09:44.168155+0000","last_unstale":"2023-10-03T10:10:35.514071+0000","last_undegraded":"2023-10-03T10:10:35.514071+0000","last_fullsized":"2023-10-03T10:10:35.514071+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518916+0000","last_change":"2023-10-03T10:09:44.197689+0000","last_active":"2023-10-03T10:10:35.518916+0000","last_peered":"2023-10-03T10:10:35.518916+0000","last_clean":"2023-10-03T10:10:35.518916+0000","last_became_active":"2023-10-03T10:09:44.197410+0000","last_became_peered":"2023-10-03T10:09:44.197410+0000","last_unstale":"2023-10-03T10:10:35.518916+0000","last_undegraded":"2023-10-03T10:10:35.518916+0000","last_fullsized":"2023-10-03T10:10:35.518916+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593682+0000","last_change":"2023-10-03T10:09:44.176966+0000","last_active":"2023-10-03T10:10:35.593682+0000","last_peered":"2023-10-03T10:10:35.593682+0000","last_clean":"2023-10-03T10:10:35.593682+0000","last_became_active":"2023-10-03T10:09:44.176386+0000","last_became_peered":"2023-10-03T10:09:44.176386+0000","last_unstale":"2023-10-03T10:10:35.593682+0000","last_undegraded":"2023-10-03T10:10:35.593682+0000","last_fullsized":"2023-10-03T10:10:35.593682+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4333,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705697,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297292,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074548,"statfs":{"total":96636764160,"available":96332337152,"internally_reserved":0,"allocated":200704,"data_stored":67449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 3 10:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.20000000000000001,"5min":0.159,"15min":0.159},"max":{"1min":0.41399999999999998,"5min":0.504,"15min":0.504},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.375,"5min":0.36799999999999999,"15min":0.36699999999999999},"min":{"1min":0.27200000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.49099999999999999,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.52200000000000002}]},{"osd":1,"last update":"Tue Oct 3 10:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.308,"15min":0.30599999999999999},"min":{"1min":0.23999999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.46600000000000003,"5min":0.498,"15min":0.498},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.32700000000000001,"15min":0.32400000000000001},"min":{"1min":0.23300000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.38300000000000001}]},{"osd":2,"last update":"Tue Oct 3 10:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.33800000000000002,"15min":0.34000000000000002},"min":{"1min":0.245,"5min":0.218,"15min":0.218},"max":{"1min":0.45100000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.29999999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.48899999999999999}]}]},{"osd":2,"up_from":9,"seq":38654705696,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297280,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074560,"statfs":{"total":96636764160,"available":96332349440,"internally_reserved":0,"allocated":188416,"data_stored":67255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 3 10:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.33300000000000002,"15min":0.33500000000000002},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.51900000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.35599999999999998,"15min":0.35799999999999998},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.36699999999999999}]},{"osd":1,"last update":"Tue Oct 3 10:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.311},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.35299999999999998,"15min":0.35699999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.47499999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"last":0.32700000000000001}]},{"osd":3,"last update":"Tue Oct 3 10:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.376,"15min":0.377},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.373,"15min":0.375},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.54800000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.29999999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705697,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297280,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074560,"statfs":{"total":96636764160,"available":96332349440,"internally_reserved":0,"allocated":188416,"data_stored":60352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 3 10:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.39700000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.33900000000000002,"15min":0.34100000000000003},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.39800000000000002,"5min":0.53500000000000003,"15min":0.53500000000000003},"last":0.35199999999999998}]},{"osd":2,"last update":"Tue Oct 3 10:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.374,"15min":0.38100000000000001},"min":{"1min":0.26100000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.41099999999999998,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.318},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.373,"15min":0.38100000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.41299999999999998,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.32900000000000001}]},{"osd":3,"last update":"Tue Oct 3 10:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.371,"15min":0.375},"min":{"1min":0.27700000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.42199999999999999,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.373},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.371,"15min":0.375},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.42099999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.34399999999999997}]}]},{"osd":1,"up_from":9,"seq":38654705697,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297280,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074560,"statfs":{"total":96636764160,"available":96332349440,"internally_reserved":0,"allocated":188416,"data_stored":64338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 3 10:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.32100000000000001,"15min":0.32300000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.39100000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.34599999999999997,"15min":0.34699999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.45600000000000002,"5min":0.504,"15min":0.504},"last":0.28799999999999998}]},{"osd":2,"last update":"Tue Oct 3 10:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.33400000000000002,"15min":0.33600000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.52200000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.35899999999999999,"15min":0.36099999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.44400000000000001,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.52700000000000002}]},{"osd":3,"last update":"Tue Oct 3 10:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.35499999999999998,"15min":0.35999999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.41099999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.372},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.38200000000000001,"15min":0.38500000000000001},"min":{"1min":0.31,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.47499999999999998,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.80000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-03T10:12:16.416 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-03T10:12:16.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.588+0000 7fc9a8eba700 1 Processor -- start 2023-10-03T10:12:16.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.588+0000 7fc9a8eba700 1 -- start start 2023-10-03T10:12:16.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.589+0000 7fc9a8eba700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9a406a770 0x7fc9a406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:16.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.589+0000 7fc9a8eba700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc9a406b080 0x7fc9a4131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:16.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.589+0000 7fc9a8eba700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9a409ece0 con 0x7fc9a4131b40 2023-10-03T10:12:16.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.589+0000 7fc9a8eba700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9a409e760 con 0x7fc9a406b080 2023-10-03T10:12:16.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.589+0000 7fc9a8eba700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a409e040 con 0x7fc9a406a770 2023-10-03T10:12:16.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.589+0000 7fc9a259c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9a406a770 0x7fc9a406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:16.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.589+0000 7fc9a1d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc9a406b080 0x7fc9a4131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:16.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.589+0000 7fc9a259c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9a406a770 0x7fc9a406ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37484/0 (socket says 172.21.15.3:37484) 2023-10-03T10:12:16.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.589+0000 7fc9a2d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc9a4131b40 0x7fc9a4138850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35666/0 (socket says 172.21.15.3:35666) 2023-10-03T10:12:16.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.589+0000 7fc9a259c700 1 -- 172.21.15.3:0/1288357780 learned_addr learned my addr 172.21.15.3:0/1288357780 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:16.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.590+0000 7fc9a259c700 1 -- 172.21.15.3:0/1288357780 >> v2:172.21.15.3:3300/0 conn(0x7fc9a406b080 msgr2=0x7fc9a4131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:16.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.590+0000 7fc9a259c700 1 --2- 172.21.15.3:0/1288357780 >> v2:172.21.15.3:3300/0 conn(0x7fc9a406b080 0x7fc9a4131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:16.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.590+0000 7fc9a259c700 1 -- 172.21.15.3:0/1288357780 >> v1:172.21.15.3:6789/0 conn(0x7fc9a4131b40 legacy=0x7fc9a4138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:16.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.590+0000 7fc9a259c700 1 -- 172.21.15.3:0/1288357780 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a409f630 con 0x7fc9a406a770 2023-10-03T10:12:16.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.590+0000 7fc9a1d9b700 1 --2- 172.21.15.3:0/1288357780 >> v2:172.21.15.3:3300/0 conn(0x7fc9a406b080 0x7fc9a4131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:16.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.590+0000 7fc9a159a700 1 -- 172.21.15.3:0/1288357780 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3567292082 0 0) 0x7fc9a409ece0 con 0x7fc9a4131b40 2023-10-03T10:12:16.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.590+0000 7fc9a259c700 1 --2- 172.21.15.3:0/1288357780 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9a406a770 0x7fc9a406ab40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc9940098f0 tx=0x7fc994009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63337a2b0466ff6e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:16.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.590+0000 7fc9a159a700 1 -- 172.21.15.3:0/1288357780 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc994011830 con 0x7fc9a406a770 2023-10-03T10:12:16.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.590+0000 7fc9a159a700 1 -- 172.21.15.3:0/1288357780 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc994011990 con 0x7fc9a406a770 2023-10-03T10:12:16.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.590+0000 7fc9a159a700 1 -- 172.21.15.3:0/1288357780 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc994011d20 con 0x7fc9a406a770 2023-10-03T10:12:16.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.590+0000 7fc9a8eba700 1 -- 172.21.15.3:0/1288357780 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9a406a770 msgr2=0x7fc9a406ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:16.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.590+0000 7fc9a8eba700 1 --2- 172.21.15.3:0/1288357780 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9a406a770 0x7fc9a406ab40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc9940098f0 tx=0x7fc994009c00 comp rx=0 tx=0).stop 2023-10-03T10:12:16.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.591+0000 7fc9a8eba700 1 -- 172.21.15.3:0/1288357780 shutdown_connections 2023-10-03T10:12:16.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.591+0000 7fc9a8eba700 1 --2- 172.21.15.3:0/1288357780 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9a406a770 0x7fc9a406ab40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:16.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.591+0000 7fc9a8eba700 1 --2- 172.21.15.3:0/1288357780 >> v2:172.21.15.3:3300/0 conn(0x7fc9a406b080 0x7fc9a4131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:16.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.591+0000 7fc9a8eba700 1 -- 172.21.15.3:0/1288357780 >> 172.21.15.3:0/1288357780 conn(0x7fc9a4066f50 msgr2=0x7fc9a409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:16.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.591+0000 7fc9a8eba700 1 -- 172.21.15.3:0/1288357780 shutdown_connections 2023-10-03T10:12:16.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.591+0000 7fc9a8eba700 1 -- 172.21.15.3:0/1288357780 wait complete. 2023-10-03T10:12:16.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.592+0000 7fc9a8eba700 1 Processor -- start 2023-10-03T10:12:16.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.592+0000 7fc9a8eba700 1 -- start start 2023-10-03T10:12:16.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.592+0000 7fc9a8eba700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9a406a770 0x7fc9a409ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:16.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.592+0000 7fc9a8eba700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9a406b080 0x7fc9a409b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:16.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.592+0000 7fc9a8eba700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9a4131b40 0x7fc9a41c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:16.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.592+0000 7fc9a8eba700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a4130fe0 con 0x7fc9a406b080 2023-10-03T10:12:16.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.592+0000 7fc9a8eba700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a409a910 con 0x7fc9a406a770 2023-10-03T10:12:16.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.592+0000 7fc9a8eba700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a4121bf0 con 0x7fc9a4131b40 2023-10-03T10:12:16.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.592+0000 7fc9a1d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9a406b080 0x7fc9a409b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:16.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.593+0000 7fc9a1d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9a406b080 0x7fc9a409b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44278/0 (socket says 172.21.15.3:44278) 2023-10-03T10:12:16.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.593+0000 7fc9a2d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9a4131b40 0x7fc9a41c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:16.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.593+0000 7fc9a1d9b700 1 -- 172.21.15.3:0/269075203 learned_addr learned my addr 172.21.15.3:0/269075203 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:16.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.593+0000 7fc9a259c700 1 --2- 172.21.15.3:0/269075203 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9a406a770 0x7fc9a409ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:16.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.593+0000 7fc9a259c700 1 -- 172.21.15.3:0/269075203 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9a4131b40 msgr2=0x7fc9a41c8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:16.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.593+0000 7fc9a259c700 1 --2- 172.21.15.3:0/269075203 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9a4131b40 0x7fc9a41c8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:16.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.593+0000 7fc9a259c700 1 -- 172.21.15.3:0/269075203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9a406b080 msgr2=0x7fc9a409b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:16.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.593+0000 7fc9a259c700 1 --2- 172.21.15.3:0/269075203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9a406b080 0x7fc9a409b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:16.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.593+0000 7fc9a259c700 1 -- 172.21.15.3:0/269075203 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc98c0094a0 con 0x7fc9a406a770 2023-10-03T10:12:16.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.593+0000 7fc9a1d9b700 1 --2- 172.21.15.3:0/269075203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9a406b080 0x7fc9a409b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:16.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.593+0000 7fc9a2d9d700 1 --2- 172.21.15.3:0/269075203 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9a4131b40 0x7fc9a41c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:16.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.593+0000 7fc9a259c700 1 --2- 172.21.15.3:0/269075203 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9a406a770 0x7fc9a409ae60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc994016000 tx=0x7fc994004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:16.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.593+0000 7fc99affd700 1 -- 172.21.15.3:0/269075203 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc994004300 con 0x7fc9a406a770 2023-10-03T10:12:16.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.593+0000 7fc99affd700 1 -- 172.21.15.3:0/269075203 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99401f070 con 0x7fc9a406a770 2023-10-03T10:12:16.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.593+0000 7fc9a8eba700 1 -- 172.21.15.3:0/269075203 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc994009550 con 0x7fc9a406a770 2023-10-03T10:12:16.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.593+0000 7fc99affd700 1 -- 172.21.15.3:0/269075203 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9940036a0 con 0x7fc9a406a770 2023-10-03T10:12:16.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.593+0000 7fc9a8eba700 1 -- 172.21.15.3:0/269075203 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a4071e60 con 0x7fc9a406a770 2023-10-03T10:12:16.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.594+0000 7fc99affd700 1 -- 172.21.15.3:0/269075203 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7fc994021030 con 0x7fc9a406a770 2023-10-03T10:12:16.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.595+0000 7fc9a8eba700 1 -- 172.21.15.3:0/269075203 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9a406aea0 con 0x7fc9a406a770 2023-10-03T10:12:16.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.595+0000 7fc99affd700 1 -- 172.21.15.3:0/269075203 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4078+0+0 (secure 0 0 0) 0x7fc994040ce0 con 0x7fc9a406a770 2023-10-03T10:12:16.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.600+0000 7fc99affd700 1 -- 172.21.15.3:0/269075203 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc994015070 con 0x7fc9a406a770 2023-10-03T10:12:16.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.799+0000 7fc9a8eba700 1 -- 172.21.15.3:0/269075203 --> v1:172.21.15.3:6816/127025 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc9a409e410 con 0x7fc98402f480 2023-10-03T10:12:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.806+0000 7fc99affd700 1 -- 172.21.15.3:0/269075203 <== mgr.14098 v1:172.21.15.3:6816/127025 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+270428 (unknown 2965378022 0 295861048) 0x7fc98c009da0 con 0x7fc98402f480 2023-10-03T10:12:16.809 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:12:16.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.809+0000 7fc9a8eba700 1 -- 172.21.15.3:0/269075203 >> v1:172.21.15.3:6816/127025 conn(0x7fc98402f480 legacy=0x7fc984031930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:16.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.809+0000 7fc9a8eba700 1 -- 172.21.15.3:0/269075203 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9a406a770 msgr2=0x7fc9a409ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:16.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.809+0000 7fc9a8eba700 1 --2- 172.21.15.3:0/269075203 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9a406a770 0x7fc9a409ae60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc994016000 tx=0x7fc994004750 comp rx=0 tx=0).stop 2023-10-03T10:12:16.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.809+0000 7fc9a8eba700 1 -- 172.21.15.3:0/269075203 shutdown_connections 2023-10-03T10:12:16.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.809+0000 7fc9a8eba700 1 --2- 172.21.15.3:0/269075203 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9a4131b40 0x7fc9a41c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:16.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.809+0000 7fc9a8eba700 1 --2- 172.21.15.3:0/269075203 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9a406a770 0x7fc9a409ae60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:16.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.809+0000 7fc9a8eba700 1 --2- 172.21.15.3:0/269075203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9a406b080 0x7fc9a409b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:16.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.810+0000 7fc9a8eba700 1 -- 172.21.15.3:0/269075203 >> 172.21.15.3:0/269075203 conn(0x7fc9a4066f50 msgr2=0x7fc9a406ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:16.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.810+0000 7fc9a8eba700 1 -- 172.21.15.3:0/269075203 shutdown_connections 2023-10-03T10:12:16.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:16.810+0000 7fc9a8eba700 1 -- 172.21.15.3:0/269075203 wait complete. 2023-10-03T10:12:16.813 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-10-03T10:12:16.824 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2023-10-03T10:12:15.327643+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4333,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1189132,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298228,"statfs":{"total":386547056640,"available":385329385472,"internally_reserved":0,"allocated":765952,"data_stored":259394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.998295"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.362278+0000","last_change":"2023-10-03T10:09:44.195682+0000","last_active":"2023-10-03T10:12:13.362278+0000","last_peered":"2023-10-03T10:12:13.362278+0000","last_clean":"2023-10-03T10:12:13.362278+0000","last_became_active":"2023-10-03T10:09:44.195360+0000","last_became_peered":"2023-10-03T10:09:44.195360+0000","last_unstale":"2023-10-03T10:12:13.362278+0000","last_undegraded":"2023-10-03T10:12:13.362278+0000","last_fullsized":"2023-10-03T10:12:13.362278+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.362172+0000","last_change":"2023-10-03T10:09:44.178263+0000","last_active":"2023-10-03T10:12:13.362172+0000","last_peered":"2023-10-03T10:12:13.362172+0000","last_clean":"2023-10-03T10:12:13.362172+0000","last_became_active":"2023-10-03T10:09:44.177977+0000","last_became_peered":"2023-10-03T10:09:44.177977+0000","last_unstale":"2023-10-03T10:12:13.362172+0000","last_undegraded":"2023-10-03T10:12:13.362172+0000","last_fullsized":"2023-10-03T10:12:13.362172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.515500+0000","last_change":"2023-10-03T10:09:44.183364+0000","last_active":"2023-10-03T10:10:35.515500+0000","last_peered":"2023-10-03T10:10:35.515500+0000","last_clean":"2023-10-03T10:10:35.515500+0000","last_became_active":"2023-10-03T10:09:44.183094+0000","last_became_peered":"2023-10-03T10:09:44.183094+0000","last_unstale":"2023-10-03T10:10:35.515500+0000","last_undegraded":"2023-10-03T10:10:35.515500+0000","last_fullsized":"2023-10-03T10:10:35.515500+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.514914+0000","last_change":"2023-10-03T10:09:44.183273+0000","last_active":"2023-10-03T10:10:35.514914+0000","last_peered":"2023-10-03T10:10:35.514914+0000","last_clean":"2023-10-03T10:10:35.514914+0000","last_became_active":"2023-10-03T10:09:44.182451+0000","last_became_peered":"2023-10-03T10:09:44.182451+0000","last_unstale":"2023-10-03T10:10:35.514914+0000","last_undegraded":"2023-10-03T10:10:35.514914+0000","last_fullsized":"2023-10-03T10:10:35.514914+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.514451+0000","last_change":"2023-10-03T10:09:44.168483+0000","last_active":"2023-10-03T10:10:35.514451+0000","last_peered":"2023-10-03T10:10:35.514451+0000","last_clean":"2023-10-03T10:10:35.514451+0000","last_became_active":"2023-10-03T10:09:44.168186+0000","last_became_peered":"2023-10-03T10:09:44.168186+0000","last_unstale":"2023-10-03T10:10:35.514451+0000","last_undegraded":"2023-10-03T10:10:35.514451+0000","last_fullsized":"2023-10-03T10:10:35.514451+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518314+0000","last_change":"2023-10-03T10:09:44.192967+0000","last_active":"2023-10-03T10:10:35.518314+0000","last_peered":"2023-10-03T10:10:35.518314+0000","last_clean":"2023-10-03T10:10:35.518314+0000","last_became_active":"2023-10-03T10:09:44.190979+0000","last_became_peered":"2023-10-03T10:09:44.190979+0000","last_unstale":"2023-10-03T10:10:35.518314+0000","last_undegraded":"2023-10-03T10:10:35.518314+0000","last_fullsized":"2023-10-03T10:10:35.518314+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.360752+0000","last_change":"2023-10-03T10:10:35.520559+0000","last_active":"2023-10-03T10:12:13.360752+0000","last_peered":"2023-10-03T10:12:13.360752+0000","last_clean":"2023-10-03T10:12:13.360752+0000","last_became_active":"2023-10-03T10:10:35.520227+0000","last_became_peered":"2023-10-03T10:10:35.520227+0000","last_unstale":"2023-10-03T10:12:13.360752+0000","last_undegraded":"2023-10-03T10:12:13.360752+0000","last_fullsized":"2023-10-03T10:12:13.360752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593090+0000","last_change":"2023-10-03T10:09:44.173123+0000","last_active":"2023-10-03T10:10:35.593090+0000","last_peered":"2023-10-03T10:10:35.593090+0000","last_clean":"2023-10-03T10:10:35.593090+0000","last_became_active":"2023-10-03T10:09:44.172353+0000","last_became_peered":"2023-10-03T10:09:44.172353+0000","last_unstale":"2023-10-03T10:10:35.593090+0000","last_undegraded":"2023-10-03T10:10:35.593090+0000","last_fullsized":"2023-10-03T10:10:35.593090+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.516273+0000","last_change":"2023-10-03T10:09:44.166758+0000","last_active":"2023-10-03T10:10:35.516273+0000","last_peered":"2023-10-03T10:10:35.516273+0000","last_clean":"2023-10-03T10:10:35.516273+0000","last_became_active":"2023-10-03T10:09:44.166415+0000","last_became_peered":"2023-10-03T10:09:44.166415+0000","last_unstale":"2023-10-03T10:10:35.516273+0000","last_undegraded":"2023-10-03T10:10:35.516273+0000","last_fullsized":"2023-10-03T10:10:35.516273+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593631+0000","last_change":"2023-10-03T10:09:44.176833+0000","last_active":"2023-10-03T10:10:35.593631+0000","last_peered":"2023-10-03T10:10:35.593631+0000","last_clean":"2023-10-03T10:10:35.593631+0000","last_became_active":"2023-10-03T10:09:44.176526+0000","last_became_peered":"2023-10-03T10:09:44.176526+0000","last_unstale":"2023-10-03T10:10:35.593631+0000","last_undegraded":"2023-10-03T10:10:35.593631+0000","last_fullsized":"2023-10-03T10:10:35.593631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593635+0000","last_change":"2023-10-03T10:09:44.175868+0000","last_active":"2023-10-03T10:10:35.593635+0000","last_peered":"2023-10-03T10:10:35.593635+0000","last_clean":"2023-10-03T10:10:35.593635+0000","last_became_active":"2023-10-03T10:09:44.175414+0000","last_became_peered":"2023-10-03T10:09:44.175414+0000","last_unstale":"2023-10-03T10:10:35.593635+0000","last_undegraded":"2023-10-03T10:10:35.593635+0000","last_fullsized":"2023-10-03T10:10:35.593635+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.362364+0000","last_change":"2023-10-03T10:09:44.198581+0000","last_active":"2023-10-03T10:12:13.362364+0000","last_peered":"2023-10-03T10:12:13.362364+0000","last_clean":"2023-10-03T10:12:13.362364+0000","last_became_active":"2023-10-03T10:09:44.198314+0000","last_became_peered":"2023-10-03T10:09:44.198314+0000","last_unstale":"2023-10-03T10:12:13.362364+0000","last_undegraded":"2023-10-03T10:12:13.362364+0000","last_fullsized":"2023-10-03T10:12:13.362364+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.360956+0000","last_change":"2023-10-03T10:10:35.521810+0000","last_active":"2023-10-03T10:12:13.360956+0000","last_peered":"2023-10-03T10:12:13.360956+0000","last_clean":"2023-10-03T10:12:13.360956+0000","last_became_active":"2023-10-03T10:10:35.521112+0000","last_became_peered":"2023-10-03T10:10:35.521112+0000","last_unstale":"2023-10-03T10:12:13.360956+0000","last_undegraded":"2023-10-03T10:12:13.360956+0000","last_fullsized":"2023-10-03T10:12:13.360956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.594065+0000","last_change":"2023-10-03T10:09:44.176456+0000","last_active":"2023-10-03T10:10:35.594065+0000","last_peered":"2023-10-03T10:10:35.594065+0000","last_clean":"2023-10-03T10:10:35.594065+0000","last_became_active":"2023-10-03T10:09:44.176146+0000","last_became_peered":"2023-10-03T10:09:44.176146+0000","last_unstale":"2023-10-03T10:10:35.594065+0000","last_undegraded":"2023-10-03T10:10:35.594065+0000","last_fullsized":"2023-10-03T10:10:35.594065+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.514008+0000","last_change":"2023-10-03T10:09:44.176614+0000","last_active":"2023-10-03T10:10:35.514008+0000","last_peered":"2023-10-03T10:10:35.514008+0000","last_clean":"2023-10-03T10:10:35.514008+0000","last_became_active":"2023-10-03T10:09:44.176334+0000","last_became_peered":"2023-10-03T10:09:44.176334+0000","last_unstale":"2023-10-03T10:10:35.514008+0000","last_undegraded":"2023-10-03T10:10:35.514008+0000","last_fullsized":"2023-10-03T10:10:35.514008+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.517267+0000","last_change":"2023-10-03T10:09:44.179440+0000","last_active":"2023-10-03T10:10:35.517267+0000","last_peered":"2023-10-03T10:10:35.517267+0000","last_clean":"2023-10-03T10:10:35.517267+0000","last_became_active":"2023-10-03T10:09:44.179150+0000","last_became_peered":"2023-10-03T10:09:44.179150+0000","last_unstale":"2023-10-03T10:10:35.517267+0000","last_undegraded":"2023-10-03T10:10:35.517267+0000","last_fullsized":"2023-10-03T10:10:35.517267+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.516183+0000","last_change":"2023-10-03T10:09:44.166901+0000","last_active":"2023-10-03T10:10:35.516183+0000","last_peered":"2023-10-03T10:10:35.516183+0000","last_clean":"2023-10-03T10:10:35.516183+0000","last_became_active":"2023-10-03T10:09:44.166610+0000","last_became_peered":"2023-10-03T10:09:44.166610+0000","last_unstale":"2023-10-03T10:10:35.516183+0000","last_undegraded":"2023-10-03T10:10:35.516183+0000","last_fullsized":"2023-10-03T10:10:35.516183+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.516077+0000","last_change":"2023-10-03T10:09:44.165366+0000","last_active":"2023-10-03T10:10:35.516077+0000","last_peered":"2023-10-03T10:10:35.516077+0000","last_clean":"2023-10-03T10:10:35.516077+0000","last_became_active":"2023-10-03T10:09:44.165083+0000","last_became_peered":"2023-10-03T10:09:44.165083+0000","last_unstale":"2023-10-03T10:10:35.516077+0000","last_undegraded":"2023-10-03T10:10:35.516077+0000","last_fullsized":"2023-10-03T10:10:35.516077+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.361831+0000","last_change":"2023-10-03T10:10:35.521376+0000","last_active":"2023-10-03T10:12:13.361831+0000","last_peered":"2023-10-03T10:12:13.361831+0000","last_clean":"2023-10-03T10:12:13.361831+0000","last_became_active":"2023-10-03T10:10:35.521029+0000","last_became_peered":"2023-10-03T10:10:35.521029+0000","last_unstale":"2023-10-03T10:12:13.361831+0000","last_undegraded":"2023-10-03T10:12:13.361831+0000","last_fullsized":"2023-10-03T10:12:13.361831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593713+0000","last_change":"2023-10-03T10:09:44.177178+0000","last_active":"2023-10-03T10:10:35.593713+0000","last_peered":"2023-10-03T10:10:35.593713+0000","last_clean":"2023-10-03T10:10:35.593713+0000","last_became_active":"2023-10-03T10:09:44.176686+0000","last_became_peered":"2023-10-03T10:09:44.176686+0000","last_unstale":"2023-10-03T10:10:35.593713+0000","last_undegraded":"2023-10-03T10:10:35.593713+0000","last_fullsized":"2023-10-03T10:10:35.593713+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593119+0000","last_change":"2023-10-03T10:09:44.177335+0000","last_active":"2023-10-03T10:10:35.593119+0000","last_peered":"2023-10-03T10:10:35.593119+0000","last_clean":"2023-10-03T10:10:35.593119+0000","last_became_active":"2023-10-03T10:09:44.175202+0000","last_became_peered":"2023-10-03T10:09:44.175202+0000","last_unstale":"2023-10-03T10:10:35.593119+0000","last_undegraded":"2023-10-03T10:10:35.593119+0000","last_fullsized":"2023-10-03T10:10:35.593119+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.525179+0000","last_change":"2023-10-03T10:10:35.521795+0000","last_active":"2023-10-03T10:10:35.525179+0000","last_peered":"2023-10-03T10:10:35.525179+0000","last_clean":"2023-10-03T10:10:35.525179+0000","last_became_active":"2023-10-03T10:10:35.521498+0000","last_became_peered":"2023-10-03T10:10:35.521498+0000","last_unstale":"2023-10-03T10:10:35.525179+0000","last_undegraded":"2023-10-03T10:10:35.525179+0000","last_fullsized":"2023-10-03T10:10:35.525179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.517982+0000","last_change":"2023-10-03T10:09:44.182610+0000","last_active":"2023-10-03T10:10:35.517982+0000","last_peered":"2023-10-03T10:10:35.517982+0000","last_clean":"2023-10-03T10:10:35.517982+0000","last_became_active":"2023-10-03T10:09:44.180732+0000","last_became_peered":"2023-10-03T10:09:44.180732+0000","last_unstale":"2023-10-03T10:10:35.517982+0000","last_undegraded":"2023-10-03T10:10:35.517982+0000","last_fullsized":"2023-10-03T10:10:35.517982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593219+0000","last_change":"2023-10-03T10:09:44.173182+0000","last_active":"2023-10-03T10:10:35.593219+0000","last_peered":"2023-10-03T10:10:35.593219+0000","last_clean":"2023-10-03T10:10:35.593219+0000","last_became_active":"2023-10-03T10:09:44.172685+0000","last_became_peered":"2023-10-03T10:09:44.172685+0000","last_unstale":"2023-10-03T10:10:35.593219+0000","last_undegraded":"2023-10-03T10:10:35.593219+0000","last_fullsized":"2023-10-03T10:10:35.593219+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593586+0000","last_change":"2023-10-03T10:09:44.177910+0000","last_active":"2023-10-03T10:10:35.593586+0000","last_peered":"2023-10-03T10:10:35.593586+0000","last_clean":"2023-10-03T10:10:35.593586+0000","last_became_active":"2023-10-03T10:09:44.177618+0000","last_became_peered":"2023-10-03T10:09:44.177618+0000","last_unstale":"2023-10-03T10:10:35.593586+0000","last_undegraded":"2023-10-03T10:10:35.593586+0000","last_fullsized":"2023-10-03T10:10:35.593586+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593571+0000","last_change":"2023-10-03T10:09:44.176287+0000","last_active":"2023-10-03T10:10:35.593571+0000","last_peered":"2023-10-03T10:10:35.593571+0000","last_clean":"2023-10-03T10:10:35.593571+0000","last_became_active":"2023-10-03T10:09:44.175320+0000","last_became_peered":"2023-10-03T10:09:44.175320+0000","last_unstale":"2023-10-03T10:10:35.593571+0000","last_undegraded":"2023-10-03T10:10:35.593571+0000","last_fullsized":"2023-10-03T10:10:35.593571+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518762+0000","last_change":"2023-10-03T10:09:44.187148+0000","last_active":"2023-10-03T10:10:35.518762+0000","last_peered":"2023-10-03T10:10:35.518762+0000","last_clean":"2023-10-03T10:10:35.518762+0000","last_became_active":"2023-10-03T10:09:44.186523+0000","last_became_peered":"2023-10-03T10:09:44.186523+0000","last_unstale":"2023-10-03T10:10:35.518762+0000","last_undegraded":"2023-10-03T10:10:35.518762+0000","last_fullsized":"2023-10-03T10:10:35.518762+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593909+0000","last_change":"2023-10-03T10:09:43.745046+0000","last_active":"2023-10-03T10:10:35.593909+0000","last_peered":"2023-10-03T10:10:35.593909+0000","last_clean":"2023-10-03T10:10:35.593909+0000","last_became_active":"2023-10-03T10:09:43.744724+0000","last_became_peered":"2023-10-03T10:09:43.744724+0000","last_unstale":"2023-10-03T10:10:35.593909+0000","last_undegraded":"2023-10-03T10:10:35.593909+0000","last_fullsized":"2023-10-03T10:10:35.593909+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518719+0000","last_change":"2023-10-03T10:09:44.190869+0000","last_active":"2023-10-03T10:10:35.518719+0000","last_peered":"2023-10-03T10:10:35.518719+0000","last_clean":"2023-10-03T10:10:35.518719+0000","last_became_active":"2023-10-03T10:09:44.190207+0000","last_became_peered":"2023-10-03T10:09:44.190207+0000","last_unstale":"2023-10-03T10:10:35.518719+0000","last_undegraded":"2023-10-03T10:10:35.518719+0000","last_fullsized":"2023-10-03T10:10:35.518719+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.592889+0000","last_change":"2023-10-03T10:09:43.750891+0000","last_active":"2023-10-03T10:10:35.592889+0000","last_peered":"2023-10-03T10:10:35.592889+0000","last_clean":"2023-10-03T10:10:35.592889+0000","last_became_active":"2023-10-03T10:09:43.750356+0000","last_became_peered":"2023-10-03T10:09:43.750356+0000","last_unstale":"2023-10-03T10:10:35.592889+0000","last_undegraded":"2023-10-03T10:10:35.592889+0000","last_fullsized":"2023-10-03T10:10:35.592889+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.515727+0000","last_change":"2023-10-03T10:09:44.183145+0000","last_active":"2023-10-03T10:10:35.515727+0000","last_peered":"2023-10-03T10:10:35.515727+0000","last_clean":"2023-10-03T10:10:35.515727+0000","last_became_active":"2023-10-03T10:09:44.182816+0000","last_became_peered":"2023-10-03T10:09:44.182816+0000","last_unstale":"2023-10-03T10:10:35.515727+0000","last_undegraded":"2023-10-03T10:10:35.515727+0000","last_fullsized":"2023-10-03T10:10:35.515727+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.517490+0000","last_change":"2023-10-03T10:09:43.749289+0000","last_active":"2023-10-03T10:10:35.517490+0000","last_peered":"2023-10-03T10:10:35.517490+0000","last_clean":"2023-10-03T10:10:35.517490+0000","last_became_active":"2023-10-03T10:09:43.748981+0000","last_became_peered":"2023-10-03T10:09:43.748981+0000","last_unstale":"2023-10-03T10:10:35.517490+0000","last_undegraded":"2023-10-03T10:10:35.517490+0000","last_fullsized":"2023-10-03T10:10:35.517490+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.519771+0000","last_change":"2023-10-03T10:09:44.192692+0000","last_active":"2023-10-03T10:10:35.519771+0000","last_peered":"2023-10-03T10:10:35.519771+0000","last_clean":"2023-10-03T10:10:35.519771+0000","last_became_active":"2023-10-03T10:09:44.192390+0000","last_became_peered":"2023-10-03T10:09:44.192390+0000","last_unstale":"2023-10-03T10:10:35.519771+0000","last_undegraded":"2023-10-03T10:10:35.519771+0000","last_fullsized":"2023-10-03T10:10:35.519771+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.515989+0000","last_change":"2023-10-03T10:09:43.754802+0000","last_active":"2023-10-03T10:10:35.515989+0000","last_peered":"2023-10-03T10:10:35.515989+0000","last_clean":"2023-10-03T10:10:35.515989+0000","last_became_active":"2023-10-03T10:09:43.754512+0000","last_became_peered":"2023-10-03T10:09:43.754512+0000","last_unstale":"2023-10-03T10:10:35.515989+0000","last_undegraded":"2023-10-03T10:10:35.515989+0000","last_fullsized":"2023-10-03T10:10:35.515989+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518307+0000","last_change":"2023-10-03T10:09:44.192798+0000","last_active":"2023-10-03T10:10:35.518307+0000","last_peered":"2023-10-03T10:10:35.518307+0000","last_clean":"2023-10-03T10:10:35.518307+0000","last_became_active":"2023-10-03T10:09:44.190787+0000","last_became_peered":"2023-10-03T10:09:44.190787+0000","last_unstale":"2023-10-03T10:10:35.518307+0000","last_undegraded":"2023-10-03T10:10:35.518307+0000","last_fullsized":"2023-10-03T10:10:35.518307+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.361838+0000","last_change":"2023-10-03T10:09:43.756958+0000","last_active":"2023-10-03T10:12:13.361838+0000","last_peered":"2023-10-03T10:12:13.361838+0000","last_clean":"2023-10-03T10:12:13.361838+0000","last_became_active":"2023-10-03T10:09:43.756463+0000","last_became_peered":"2023-10-03T10:09:43.756463+0000","last_unstale":"2023-10-03T10:12:13.361838+0000","last_undegraded":"2023-10-03T10:12:13.361838+0000","last_fullsized":"2023-10-03T10:12:13.361838+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518495+0000","last_change":"2023-10-03T10:09:44.184790+0000","last_active":"2023-10-03T10:10:35.518495+0000","last_peered":"2023-10-03T10:10:35.518495+0000","last_clean":"2023-10-03T10:10:35.518495+0000","last_became_active":"2023-10-03T10:09:44.184531+0000","last_became_peered":"2023-10-03T10:09:44.184531+0000","last_unstale":"2023-10-03T10:10:35.518495+0000","last_undegraded":"2023-10-03T10:10:35.518495+0000","last_fullsized":"2023-10-03T10:10:35.518495+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"16'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.515410+0000","last_change":"2023-10-03T10:09:43.756598+0000","last_active":"2023-10-03T10:10:35.515410+0000","last_peered":"2023-10-03T10:10:35.515410+0000","last_clean":"2023-10-03T10:10:35.515410+0000","last_became_active":"2023-10-03T10:09:43.754974+0000","last_became_peered":"2023-10-03T10:09:43.754974+0000","last_unstale":"2023-10-03T10:10:35.515410+0000","last_undegraded":"2023-10-03T10:10:35.515410+0000","last_fullsized":"2023-10-03T10:10:35.515410+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.594021+0000","last_change":"2023-10-03T10:09:44.175504+0000","last_active":"2023-10-03T10:10:35.594021+0000","last_peered":"2023-10-03T10:10:35.594021+0000","last_clean":"2023-10-03T10:10:35.594021+0000","last_became_active":"2023-10-03T10:09:44.175174+0000","last_became_peered":"2023-10-03T10:09:44.175174+0000","last_unstale":"2023-10-03T10:10:35.594021+0000","last_undegraded":"2023-10-03T10:10:35.594021+0000","last_fullsized":"2023-10-03T10:10:35.594021+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"21'12","reported_seq":26,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.380292+0000","last_change":"2023-10-03T10:09:43.758711+0000","last_active":"2023-10-03T10:12:13.380292+0000","last_peered":"2023-10-03T10:12:13.380292+0000","last_clean":"2023-10-03T10:12:13.380292+0000","last_became_active":"2023-10-03T10:09:43.758392+0000","last_became_peered":"2023-10-03T10:09:43.758392+0000","last_unstale":"2023-10-03T10:12:13.380292+0000","last_undegraded":"2023-10-03T10:12:13.380292+0000","last_fullsized":"2023-10-03T10:12:13.380292+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3563,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.361056+0000","last_change":"2023-10-03T10:09:44.196559+0000","last_active":"2023-10-03T10:12:13.361056+0000","last_peered":"2023-10-03T10:12:13.361056+0000","last_clean":"2023-10-03T10:12:13.361056+0000","last_became_active":"2023-10-03T10:09:44.196280+0000","last_became_peered":"2023-10-03T10:09:44.196280+0000","last_unstale":"2023-10-03T10:12:13.361056+0000","last_undegraded":"2023-10-03T10:12:13.361056+0000","last_fullsized":"2023-10-03T10:12:13.361056+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518819+0000","last_change":"2023-10-03T10:09:43.751877+0000","last_active":"2023-10-03T10:10:35.518819+0000","last_peered":"2023-10-03T10:10:35.518819+0000","last_clean":"2023-10-03T10:10:35.518819+0000","last_became_active":"2023-10-03T10:09:43.749870+0000","last_became_peered":"2023-10-03T10:09:43.749870+0000","last_unstale":"2023-10-03T10:10:35.518819+0000","last_undegraded":"2023-10-03T10:10:35.518819+0000","last_fullsized":"2023-10-03T10:10:35.518819+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.360943+0000","last_change":"2023-10-03T10:10:35.595943+0000","last_active":"2023-10-03T10:12:13.360943+0000","last_peered":"2023-10-03T10:12:13.360943+0000","last_clean":"2023-10-03T10:12:13.360943+0000","last_became_active":"2023-10-03T10:10:35.595469+0000","last_became_peered":"2023-10-03T10:10:35.595469+0000","last_unstale":"2023-10-03T10:12:13.360943+0000","last_undegraded":"2023-10-03T10:12:13.360943+0000","last_fullsized":"2023-10-03T10:12:13.360943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.513785+0000","last_change":"2023-10-03T10:09:43.755271+0000","last_active":"2023-10-03T10:10:35.513785+0000","last_peered":"2023-10-03T10:10:35.513785+0000","last_clean":"2023-10-03T10:10:35.513785+0000","last_became_active":"2023-10-03T10:09:43.754957+0000","last_became_peered":"2023-10-03T10:09:43.754957+0000","last_unstale":"2023-10-03T10:10:35.513785+0000","last_undegraded":"2023-10-03T10:10:35.513785+0000","last_fullsized":"2023-10-03T10:10:35.513785+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518823+0000","last_change":"2023-10-03T10:09:44.187333+0000","last_active":"2023-10-03T10:10:35.518823+0000","last_peered":"2023-10-03T10:10:35.518823+0000","last_clean":"2023-10-03T10:10:35.518823+0000","last_became_active":"2023-10-03T10:09:44.186824+0000","last_became_peered":"2023-10-03T10:09:44.186824+0000","last_unstale":"2023-10-03T10:10:35.518823+0000","last_undegraded":"2023-10-03T10:10:35.518823+0000","last_fullsized":"2023-10-03T10:10:35.518823+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518372+0000","last_change":"2023-10-03T10:09:43.758995+0000","last_active":"2023-10-03T10:10:35.518372+0000","last_peered":"2023-10-03T10:10:35.518372+0000","last_clean":"2023-10-03T10:10:35.518372+0000","last_became_active":"2023-10-03T10:09:43.758565+0000","last_became_peered":"2023-10-03T10:09:43.758565+0000","last_unstale":"2023-10-03T10:10:35.518372+0000","last_undegraded":"2023-10-03T10:10:35.518372+0000","last_fullsized":"2023-10-03T10:10:35.518372+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.515894+0000","last_change":"2023-10-03T10:09:44.166557+0000","last_active":"2023-10-03T10:10:35.515894+0000","last_peered":"2023-10-03T10:10:35.515894+0000","last_clean":"2023-10-03T10:10:35.515894+0000","last_became_active":"2023-10-03T10:09:44.166300+0000","last_became_peered":"2023-10-03T10:09:44.166300+0000","last_unstale":"2023-10-03T10:10:35.515894+0000","last_undegraded":"2023-10-03T10:10:35.515894+0000","last_fullsized":"2023-10-03T10:10:35.515894+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"16'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593762+0000","last_change":"2023-10-03T10:09:43.748084+0000","last_active":"2023-10-03T10:10:35.593762+0000","last_peered":"2023-10-03T10:10:35.593762+0000","last_clean":"2023-10-03T10:10:35.593762+0000","last_became_active":"2023-10-03T10:09:43.747377+0000","last_became_peered":"2023-10-03T10:09:43.747377+0000","last_unstale":"2023-10-03T10:10:35.593762+0000","last_undegraded":"2023-10-03T10:10:35.593762+0000","last_fullsized":"2023-10-03T10:10:35.593762+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.519686+0000","last_change":"2023-10-03T10:09:44.190767+0000","last_active":"2023-10-03T10:10:35.519686+0000","last_peered":"2023-10-03T10:10:35.519686+0000","last_clean":"2023-10-03T10:10:35.519686+0000","last_became_active":"2023-10-03T10:09:44.190514+0000","last_became_peered":"2023-10-03T10:09:44.190514+0000","last_unstale":"2023-10-03T10:10:35.519686+0000","last_undegraded":"2023-10-03T10:10:35.519686+0000","last_fullsized":"2023-10-03T10:10:35.519686+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.515759+0000","last_change":"2023-10-03T10:09:43.748461+0000","last_active":"2023-10-03T10:10:35.515759+0000","last_peered":"2023-10-03T10:10:35.515759+0000","last_clean":"2023-10-03T10:10:35.515759+0000","last_became_active":"2023-10-03T10:09:43.747372+0000","last_became_peered":"2023-10-03T10:09:43.747372+0000","last_unstale":"2023-10-03T10:10:35.515759+0000","last_undegraded":"2023-10-03T10:10:35.515759+0000","last_fullsized":"2023-10-03T10:10:35.515759+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.528421+0000","last_change":"2023-10-03T10:10:35.525502+0000","last_active":"2023-10-03T10:10:35.528421+0000","last_peered":"2023-10-03T10:10:35.528421+0000","last_clean":"2023-10-03T10:10:35.528421+0000","last_became_active":"2023-10-03T10:10:35.525214+0000","last_became_peered":"2023-10-03T10:10:35.525214+0000","last_unstale":"2023-10-03T10:10:35.528421+0000","last_undegraded":"2023-10-03T10:10:35.528421+0000","last_fullsized":"2023-10-03T10:10:35.528421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"16'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.361932+0000","last_change":"2023-10-03T10:09:43.756751+0000","last_active":"2023-10-03T10:12:13.361932+0000","last_peered":"2023-10-03T10:12:13.361932+0000","last_clean":"2023-10-03T10:12:13.361932+0000","last_became_active":"2023-10-03T10:09:43.756079+0000","last_became_peered":"2023-10-03T10:09:43.756079+0000","last_unstale":"2023-10-03T10:12:13.361932+0000","last_undegraded":"2023-10-03T10:12:13.361932+0000","last_fullsized":"2023-10-03T10:12:13.361932+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593032+0000","last_change":"2023-10-03T10:09:44.172400+0000","last_active":"2023-10-03T10:10:35.593032+0000","last_peered":"2023-10-03T10:10:35.593032+0000","last_clean":"2023-10-03T10:10:35.593032+0000","last_became_active":"2023-10-03T10:09:44.172095+0000","last_became_peered":"2023-10-03T10:09:44.172095+0000","last_unstale":"2023-10-03T10:10:35.593032+0000","last_undegraded":"2023-10-03T10:10:35.593032+0000","last_fullsized":"2023-10-03T10:10:35.593032+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518935+0000","last_change":"2023-10-03T10:09:43.760622+0000","last_active":"2023-10-03T10:10:35.518935+0000","last_peered":"2023-10-03T10:10:35.518935+0000","last_clean":"2023-10-03T10:10:35.518935+0000","last_became_active":"2023-10-03T10:09:43.760371+0000","last_became_peered":"2023-10-03T10:09:43.760371+0000","last_unstale":"2023-10-03T10:10:35.518935+0000","last_undegraded":"2023-10-03T10:10:35.518935+0000","last_fullsized":"2023-10-03T10:10:35.518935+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.360867+0000","last_change":"2023-10-03T10:09:44.196062+0000","last_active":"2023-10-03T10:12:13.360867+0000","last_peered":"2023-10-03T10:12:13.360867+0000","last_clean":"2023-10-03T10:12:13.360867+0000","last_became_active":"2023-10-03T10:09:44.195841+0000","last_became_peered":"2023-10-03T10:09:44.195841+0000","last_unstale":"2023-10-03T10:12:13.360867+0000","last_undegraded":"2023-10-03T10:12:13.360867+0000","last_fullsized":"2023-10-03T10:12:13.360867+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.515441+0000","last_change":"2023-10-03T10:09:43.756885+0000","last_active":"2023-10-03T10:10:35.515441+0000","last_peered":"2023-10-03T10:10:35.515441+0000","last_clean":"2023-10-03T10:10:35.515441+0000","last_became_active":"2023-10-03T10:09:43.756588+0000","last_became_peered":"2023-10-03T10:09:43.756588+0000","last_unstale":"2023-10-03T10:10:35.515441+0000","last_undegraded":"2023-10-03T10:10:35.515441+0000","last_fullsized":"2023-10-03T10:10:35.515441+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.514541+0000","last_change":"2023-10-03T10:09:44.181603+0000","last_active":"2023-10-03T10:10:35.514541+0000","last_peered":"2023-10-03T10:10:35.514541+0000","last_clean":"2023-10-03T10:10:35.514541+0000","last_became_active":"2023-10-03T10:09:44.181125+0000","last_became_peered":"2023-10-03T10:09:44.181125+0000","last_unstale":"2023-10-03T10:10:35.514541+0000","last_undegraded":"2023-10-03T10:10:35.514541+0000","last_fullsized":"2023-10-03T10:10:35.514541+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518701+0000","last_change":"2023-10-03T10:09:43.751873+0000","last_active":"2023-10-03T10:10:35.518701+0000","last_peered":"2023-10-03T10:10:35.518701+0000","last_clean":"2023-10-03T10:10:35.518701+0000","last_became_active":"2023-10-03T10:09:43.751324+0000","last_became_peered":"2023-10-03T10:09:43.751324+0000","last_unstale":"2023-10-03T10:10:35.518701+0000","last_undegraded":"2023-10-03T10:10:35.518701+0000","last_fullsized":"2023-10-03T10:10:35.518701+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.515356+0000","last_change":"2023-10-03T10:09:44.170676+0000","last_active":"2023-10-03T10:10:35.515356+0000","last_peered":"2023-10-03T10:10:35.515356+0000","last_clean":"2023-10-03T10:10:35.515356+0000","last_became_active":"2023-10-03T10:09:44.170398+0000","last_became_peered":"2023-10-03T10:09:44.170398+0000","last_unstale":"2023-10-03T10:10:35.515356+0000","last_undegraded":"2023-10-03T10:10:35.515356+0000","last_fullsized":"2023-10-03T10:10:35.515356+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518265+0000","last_change":"2023-10-03T10:09:43.760711+0000","last_active":"2023-10-03T10:10:35.518265+0000","last_peered":"2023-10-03T10:10:35.518265+0000","last_clean":"2023-10-03T10:10:35.518265+0000","last_became_active":"2023-10-03T10:09:43.760432+0000","last_became_peered":"2023-10-03T10:09:43.760432+0000","last_unstale":"2023-10-03T10:10:35.518265+0000","last_undegraded":"2023-10-03T10:10:35.518265+0000","last_fullsized":"2023-10-03T10:10:35.518265+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.361648+0000","last_change":"2023-10-03T10:10:35.595831+0000","last_active":"2023-10-03T10:12:13.361648+0000","last_peered":"2023-10-03T10:12:13.361648+0000","last_clean":"2023-10-03T10:12:13.361648+0000","last_became_active":"2023-10-03T10:10:35.595457+0000","last_became_peered":"2023-10-03T10:10:35.595457+0000","last_unstale":"2023-10-03T10:12:13.361648+0000","last_undegraded":"2023-10-03T10:12:13.361648+0000","last_fullsized":"2023-10-03T10:12:13.361648+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518180+0000","last_change":"2023-10-03T10:09:43.758846+0000","last_active":"2023-10-03T10:10:35.518180+0000","last_peered":"2023-10-03T10:10:35.518180+0000","last_clean":"2023-10-03T10:10:35.518180+0000","last_became_active":"2023-10-03T10:09:43.757973+0000","last_became_peered":"2023-10-03T10:09:43.757973+0000","last_unstale":"2023-10-03T10:10:35.518180+0000","last_undegraded":"2023-10-03T10:10:35.518180+0000","last_fullsized":"2023-10-03T10:10:35.518180+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.362091+0000","last_change":"2023-10-03T10:09:44.196056+0000","last_active":"2023-10-03T10:12:13.362091+0000","last_peered":"2023-10-03T10:12:13.362091+0000","last_clean":"2023-10-03T10:12:13.362091+0000","last_became_active":"2023-10-03T10:09:44.195817+0000","last_became_peered":"2023-10-03T10:09:44.195817+0000","last_unstale":"2023-10-03T10:12:13.362091+0000","last_undegraded":"2023-10-03T10:12:13.362091+0000","last_fullsized":"2023-10-03T10:12:13.362091+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.517314+0000","last_change":"2023-10-03T10:09:43.761068+0000","last_active":"2023-10-03T10:10:35.517314+0000","last_peered":"2023-10-03T10:10:35.517314+0000","last_clean":"2023-10-03T10:10:35.517314+0000","last_became_active":"2023-10-03T10:09:43.760825+0000","last_became_peered":"2023-10-03T10:09:43.760825+0000","last_unstale":"2023-10-03T10:10:35.517314+0000","last_undegraded":"2023-10-03T10:10:35.517314+0000","last_fullsized":"2023-10-03T10:10:35.517314+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593797+0000","last_change":"2023-10-03T10:09:44.178856+0000","last_active":"2023-10-03T10:10:35.593797+0000","last_peered":"2023-10-03T10:10:35.593797+0000","last_clean":"2023-10-03T10:10:35.593797+0000","last_became_active":"2023-10-03T10:09:44.178410+0000","last_became_peered":"2023-10-03T10:09:44.178410+0000","last_unstale":"2023-10-03T10:10:35.593797+0000","last_undegraded":"2023-10-03T10:10:35.593797+0000","last_fullsized":"2023-10-03T10:10:35.593797+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.592583+0000","last_change":"2023-10-03T10:09:39.589080+0000","last_active":"2023-10-03T10:10:35.592583+0000","last_peered":"2023-10-03T10:10:35.592583+0000","last_clean":"2023-10-03T10:10:35.592583+0000","last_became_active":"2023-10-03T10:09:39.588651+0000","last_became_peered":"2023-10-03T10:09:39.588651+0000","last_unstale":"2023-10-03T10:10:35.592583+0000","last_undegraded":"2023-10-03T10:10:35.592583+0000","last_fullsized":"2023-10-03T10:10:35.592583+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.361743+0000","last_change":"2023-10-03T10:09:43.756053+0000","last_active":"2023-10-03T10:12:13.361743+0000","last_peered":"2023-10-03T10:12:13.361743+0000","last_clean":"2023-10-03T10:12:13.361743+0000","last_became_active":"2023-10-03T10:09:43.755708+0000","last_became_peered":"2023-10-03T10:09:43.755708+0000","last_unstale":"2023-10-03T10:12:13.361743+0000","last_undegraded":"2023-10-03T10:12:13.361743+0000","last_fullsized":"2023-10-03T10:12:13.361743+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593942+0000","last_change":"2023-10-03T10:09:44.176964+0000","last_active":"2023-10-03T10:10:35.593942+0000","last_peered":"2023-10-03T10:10:35.593942+0000","last_clean":"2023-10-03T10:10:35.593942+0000","last_became_active":"2023-10-03T10:09:44.176671+0000","last_became_peered":"2023-10-03T10:09:44.176671+0000","last_unstale":"2023-10-03T10:10:35.593942+0000","last_undegraded":"2023-10-03T10:10:35.593942+0000","last_fullsized":"2023-10-03T10:10:35.593942+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518390+0000","last_change":"2023-10-03T10:09:39.586735+0000","last_active":"2023-10-03T10:10:35.518390+0000","last_peered":"2023-10-03T10:10:35.518390+0000","last_clean":"2023-10-03T10:10:35.518390+0000","last_became_active":"2023-10-03T10:09:39.586440+0000","last_became_peered":"2023-10-03T10:09:39.586440+0000","last_unstale":"2023-10-03T10:10:35.518390+0000","last_undegraded":"2023-10-03T10:10:35.518390+0000","last_fullsized":"2023-10-03T10:10:35.518390+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.514163+0000","last_change":"2023-10-03T10:09:43.748458+0000","last_active":"2023-10-03T10:10:35.514163+0000","last_peered":"2023-10-03T10:10:35.514163+0000","last_clean":"2023-10-03T10:10:35.514163+0000","last_became_active":"2023-10-03T10:09:43.747721+0000","last_became_peered":"2023-10-03T10:09:43.747721+0000","last_unstale":"2023-10-03T10:10:35.514163+0000","last_undegraded":"2023-10-03T10:10:35.514163+0000","last_fullsized":"2023-10-03T10:10:35.514163+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593338+0000","last_change":"2023-10-03T10:09:44.176165+0000","last_active":"2023-10-03T10:10:35.593338+0000","last_peered":"2023-10-03T10:10:35.593338+0000","last_clean":"2023-10-03T10:10:35.593338+0000","last_became_active":"2023-10-03T10:09:44.175853+0000","last_became_peered":"2023-10-03T10:09:44.175853+0000","last_unstale":"2023-10-03T10:10:35.593338+0000","last_undegraded":"2023-10-03T10:10:35.593338+0000","last_fullsized":"2023-10-03T10:10:35.593338+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.360681+0000","last_change":"2023-10-03T10:09:39.590114+0000","last_active":"2023-10-03T10:12:13.360681+0000","last_peered":"2023-10-03T10:12:13.360681+0000","last_clean":"2023-10-03T10:12:13.360681+0000","last_became_active":"2023-10-03T10:09:39.589663+0000","last_became_peered":"2023-10-03T10:09:39.589663+0000","last_unstale":"2023-10-03T10:12:13.360681+0000","last_undegraded":"2023-10-03T10:12:13.360681+0000","last_fullsized":"2023-10-03T10:12:13.360681+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593543+0000","last_change":"2023-10-03T10:09:43.748267+0000","last_active":"2023-10-03T10:10:35.593543+0000","last_peered":"2023-10-03T10:10:35.593543+0000","last_clean":"2023-10-03T10:10:35.593543+0000","last_became_active":"2023-10-03T10:09:43.747789+0000","last_became_peered":"2023-10-03T10:09:43.747789+0000","last_unstale":"2023-10-03T10:10:35.593543+0000","last_undegraded":"2023-10-03T10:10:35.593543+0000","last_fullsized":"2023-10-03T10:10:35.593543+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593698+0000","last_change":"2023-10-03T10:09:44.178680+0000","last_active":"2023-10-03T10:10:35.593698+0000","last_peered":"2023-10-03T10:10:35.593698+0000","last_clean":"2023-10-03T10:10:35.593698+0000","last_became_active":"2023-10-03T10:09:44.178114+0000","last_became_peered":"2023-10-03T10:09:44.178114+0000","last_unstale":"2023-10-03T10:10:35.593698+0000","last_undegraded":"2023-10-03T10:10:35.593698+0000","last_fullsized":"2023-10-03T10:10:35.593698+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518722+0000","last_change":"2023-10-03T10:09:39.587728+0000","last_active":"2023-10-03T10:10:35.518722+0000","last_peered":"2023-10-03T10:10:35.518722+0000","last_clean":"2023-10-03T10:10:35.518722+0000","last_became_active":"2023-10-03T10:09:39.587441+0000","last_became_peered":"2023-10-03T10:09:39.587441+0000","last_unstale":"2023-10-03T10:10:35.518722+0000","last_undegraded":"2023-10-03T10:10:35.518722+0000","last_fullsized":"2023-10-03T10:10:35.518722+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593334+0000","last_change":"2023-10-03T10:09:38.585668+0000","last_active":"2023-10-03T10:10:35.593334+0000","last_peered":"2023-10-03T10:10:35.593334+0000","last_clean":"2023-10-03T10:10:35.593334+0000","last_became_active":"2023-10-03T10:09:38.585324+0000","last_became_peered":"2023-10-03T10:09:38.585324+0000","last_unstale":"2023-10-03T10:10:35.593334+0000","last_undegraded":"2023-10-03T10:10:35.593334+0000","last_fullsized":"2023-10-03T10:10:35.593334+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:37.570571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:37.570571+0000","last_clean_scrub_stamp":"2023-10-03T10:09:37.570571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518216+0000","last_change":"2023-10-03T10:09:43.758096+0000","last_active":"2023-10-03T10:10:35.518216+0000","last_peered":"2023-10-03T10:10:35.518216+0000","last_clean":"2023-10-03T10:10:35.518216+0000","last_became_active":"2023-10-03T10:09:43.757689+0000","last_became_peered":"2023-10-03T10:09:43.757689+0000","last_unstale":"2023-10-03T10:10:35.518216+0000","last_undegraded":"2023-10-03T10:10:35.518216+0000","last_fullsized":"2023-10-03T10:10:35.518216+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.519138+0000","last_change":"2023-10-03T10:09:44.192781+0000","last_active":"2023-10-03T10:10:35.519138+0000","last_peered":"2023-10-03T10:10:35.519138+0000","last_clean":"2023-10-03T10:10:35.519138+0000","last_became_active":"2023-10-03T10:09:44.190597+0000","last_became_peered":"2023-10-03T10:09:44.190597+0000","last_unstale":"2023-10-03T10:10:35.519138+0000","last_undegraded":"2023-10-03T10:10:35.519138+0000","last_fullsized":"2023-10-03T10:10:35.519138+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.517402+0000","last_change":"2023-10-03T10:09:39.586806+0000","last_active":"2023-10-03T10:10:35.517402+0000","last_peered":"2023-10-03T10:10:35.517402+0000","last_clean":"2023-10-03T10:10:35.517402+0000","last_became_active":"2023-10-03T10:09:39.586481+0000","last_became_peered":"2023-10-03T10:09:39.586481+0000","last_unstale":"2023-10-03T10:10:35.517402+0000","last_undegraded":"2023-10-03T10:10:35.517402+0000","last_fullsized":"2023-10-03T10:10:35.517402+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.514120+0000","last_change":"2023-10-03T10:09:43.747783+0000","last_active":"2023-10-03T10:10:35.514120+0000","last_peered":"2023-10-03T10:10:35.514120+0000","last_clean":"2023-10-03T10:10:35.514120+0000","last_became_active":"2023-10-03T10:09:43.746009+0000","last_became_peered":"2023-10-03T10:09:43.746009+0000","last_unstale":"2023-10-03T10:10:35.514120+0000","last_undegraded":"2023-10-03T10:10:35.514120+0000","last_fullsized":"2023-10-03T10:10:35.514120+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.515983+0000","last_change":"2023-10-03T10:09:44.181359+0000","last_active":"2023-10-03T10:10:35.515983+0000","last_peered":"2023-10-03T10:10:35.515983+0000","last_clean":"2023-10-03T10:10:35.515983+0000","last_became_active":"2023-10-03T10:09:44.181075+0000","last_became_peered":"2023-10-03T10:09:44.181075+0000","last_unstale":"2023-10-03T10:10:35.515983+0000","last_undegraded":"2023-10-03T10:10:35.515983+0000","last_fullsized":"2023-10-03T10:10:35.515983+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.519002+0000","last_change":"2023-10-03T10:09:39.588827+0000","last_active":"2023-10-03T10:10:35.519002+0000","last_peered":"2023-10-03T10:10:35.519002+0000","last_clean":"2023-10-03T10:10:35.519002+0000","last_became_active":"2023-10-03T10:09:39.588531+0000","last_became_peered":"2023-10-03T10:09:39.588531+0000","last_unstale":"2023-10-03T10:10:35.519002+0000","last_undegraded":"2023-10-03T10:10:35.519002+0000","last_fullsized":"2023-10-03T10:10:35.519002+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593205+0000","last_change":"2023-10-03T10:09:43.746339+0000","last_active":"2023-10-03T10:10:35.593205+0000","last_peered":"2023-10-03T10:10:35.593205+0000","last_clean":"2023-10-03T10:10:35.593205+0000","last_became_active":"2023-10-03T10:09:43.746051+0000","last_became_peered":"2023-10-03T10:09:43.746051+0000","last_unstale":"2023-10-03T10:10:35.593205+0000","last_undegraded":"2023-10-03T10:10:35.593205+0000","last_fullsized":"2023-10-03T10:10:35.593205+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.514356+0000","last_change":"2023-10-03T10:09:44.165684+0000","last_active":"2023-10-03T10:10:35.514356+0000","last_peered":"2023-10-03T10:10:35.514356+0000","last_clean":"2023-10-03T10:10:35.514356+0000","last_became_active":"2023-10-03T10:09:44.165422+0000","last_became_peered":"2023-10-03T10:09:44.165422+0000","last_unstale":"2023-10-03T10:10:35.514356+0000","last_undegraded":"2023-10-03T10:10:35.514356+0000","last_fullsized":"2023-10-03T10:10:35.514356+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.519510+0000","last_change":"2023-10-03T10:09:43.760807+0000","last_active":"2023-10-03T10:10:35.519510+0000","last_peered":"2023-10-03T10:10:35.519510+0000","last_clean":"2023-10-03T10:10:35.519510+0000","last_became_active":"2023-10-03T10:09:43.760517+0000","last_became_peered":"2023-10-03T10:09:43.760517+0000","last_unstale":"2023-10-03T10:10:35.519510+0000","last_undegraded":"2023-10-03T10:10:35.519510+0000","last_fullsized":"2023-10-03T10:10:35.519510+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.592628+0000","last_change":"2023-10-03T10:09:39.586702+0000","last_active":"2023-10-03T10:10:35.592628+0000","last_peered":"2023-10-03T10:10:35.592628+0000","last_clean":"2023-10-03T10:10:35.592628+0000","last_became_active":"2023-10-03T10:09:39.586153+0000","last_became_peered":"2023-10-03T10:09:39.586153+0000","last_unstale":"2023-10-03T10:10:35.592628+0000","last_undegraded":"2023-10-03T10:10:35.592628+0000","last_fullsized":"2023-10-03T10:10:35.592628+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.517371+0000","last_change":"2023-10-03T10:09:44.192664+0000","last_active":"2023-10-03T10:10:35.517371+0000","last_peered":"2023-10-03T10:10:35.517371+0000","last_clean":"2023-10-03T10:10:35.517371+0000","last_became_active":"2023-10-03T10:09:44.190513+0000","last_became_peered":"2023-10-03T10:09:44.190513+0000","last_unstale":"2023-10-03T10:10:35.517371+0000","last_undegraded":"2023-10-03T10:10:35.517371+0000","last_fullsized":"2023-10-03T10:10:35.517371+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.519596+0000","last_change":"2023-10-03T10:09:39.588950+0000","last_active":"2023-10-03T10:10:35.519596+0000","last_peered":"2023-10-03T10:10:35.519596+0000","last_clean":"2023-10-03T10:10:35.519596+0000","last_became_active":"2023-10-03T10:09:39.588601+0000","last_became_peered":"2023-10-03T10:09:39.588601+0000","last_unstale":"2023-10-03T10:10:35.519596+0000","last_undegraded":"2023-10-03T10:10:35.519596+0000","last_fullsized":"2023-10-03T10:10:35.519596+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:38.571841+0000","last_clean_scrub_stamp":"2023-10-03T10:09:38.571841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"17'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.516197+0000","last_change":"2023-10-03T10:09:43.752793+0000","last_active":"2023-10-03T10:10:35.516197+0000","last_peered":"2023-10-03T10:10:35.516197+0000","last_clean":"2023-10-03T10:10:35.516197+0000","last_became_active":"2023-10-03T10:09:43.752374+0000","last_became_peered":"2023-10-03T10:09:43.752374+0000","last_unstale":"2023-10-03T10:10:35.516197+0000","last_undegraded":"2023-10-03T10:10:35.516197+0000","last_fullsized":"2023-10-03T10:10:35.516197+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.360862+0000","last_change":"2023-10-03T10:10:35.521309+0000","last_active":"2023-10-03T10:12:13.360862+0000","last_peered":"2023-10-03T10:12:13.360862+0000","last_clean":"2023-10-03T10:12:13.360862+0000","last_became_active":"2023-10-03T10:10:35.520943+0000","last_became_peered":"2023-10-03T10:10:35.520943+0000","last_unstale":"2023-10-03T10:12:13.360862+0000","last_undegraded":"2023-10-03T10:12:13.360862+0000","last_fullsized":"2023-10-03T10:12:13.360862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.514259+0000","last_change":"2023-10-03T10:09:43.756460+0000","last_active":"2023-10-03T10:10:35.514259+0000","last_peered":"2023-10-03T10:10:35.514259+0000","last_clean":"2023-10-03T10:10:35.514259+0000","last_became_active":"2023-10-03T10:09:43.756164+0000","last_became_peered":"2023-10-03T10:09:43.756164+0000","last_unstale":"2023-10-03T10:10:35.514259+0000","last_undegraded":"2023-10-03T10:10:35.514259+0000","last_fullsized":"2023-10-03T10:10:35.514259+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.361562+0000","last_change":"2023-10-03T10:09:44.181915+0000","last_active":"2023-10-03T10:12:13.361562+0000","last_peered":"2023-10-03T10:12:13.361562+0000","last_clean":"2023-10-03T10:12:13.361562+0000","last_became_active":"2023-10-03T10:09:44.181645+0000","last_became_peered":"2023-10-03T10:09:44.181645+0000","last_unstale":"2023-10-03T10:12:13.361562+0000","last_undegraded":"2023-10-03T10:12:13.361562+0000","last_fullsized":"2023-10-03T10:12:13.361562+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593449+0000","last_change":"2023-10-03T10:09:43.748380+0000","last_active":"2023-10-03T10:10:35.593449+0000","last_peered":"2023-10-03T10:10:35.593449+0000","last_clean":"2023-10-03T10:10:35.593449+0000","last_became_active":"2023-10-03T10:09:43.748104+0000","last_became_peered":"2023-10-03T10:09:43.748104+0000","last_unstale":"2023-10-03T10:10:35.593449+0000","last_undegraded":"2023-10-03T10:10:35.593449+0000","last_fullsized":"2023-10-03T10:10:35.593449+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.360930+0000","last_change":"2023-10-03T10:09:44.197710+0000","last_active":"2023-10-03T10:12:13.360930+0000","last_peered":"2023-10-03T10:12:13.360930+0000","last_clean":"2023-10-03T10:12:13.360930+0000","last_became_active":"2023-10-03T10:09:44.197064+0000","last_became_peered":"2023-10-03T10:09:44.197064+0000","last_unstale":"2023-10-03T10:12:13.360930+0000","last_undegraded":"2023-10-03T10:12:13.360930+0000","last_fullsized":"2023-10-03T10:12:13.360930+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.519132+0000","last_change":"2023-10-03T10:09:43.758701+0000","last_active":"2023-10-03T10:10:35.519132+0000","last_peered":"2023-10-03T10:10:35.519132+0000","last_clean":"2023-10-03T10:10:35.519132+0000","last_became_active":"2023-10-03T10:09:43.758376+0000","last_became_peered":"2023-10-03T10:09:43.758376+0000","last_unstale":"2023-10-03T10:10:35.519132+0000","last_undegraded":"2023-10-03T10:10:35.519132+0000","last_fullsized":"2023-10-03T10:10:35.519132+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.361278+0000","last_change":"2023-10-03T10:09:44.197164+0000","last_active":"2023-10-03T10:12:13.361278+0000","last_peered":"2023-10-03T10:12:13.361278+0000","last_clean":"2023-10-03T10:12:13.361278+0000","last_became_active":"2023-10-03T10:09:44.196885+0000","last_became_peered":"2023-10-03T10:09:44.196885+0000","last_unstale":"2023-10-03T10:12:13.361278+0000","last_undegraded":"2023-10-03T10:12:13.361278+0000","last_fullsized":"2023-10-03T10:12:13.361278+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.519349+0000","last_change":"2023-10-03T10:09:43.758373+0000","last_active":"2023-10-03T10:10:35.519349+0000","last_peered":"2023-10-03T10:10:35.519349+0000","last_clean":"2023-10-03T10:10:35.519349+0000","last_became_active":"2023-10-03T10:09:43.758061+0000","last_became_peered":"2023-10-03T10:09:43.758061+0000","last_unstale":"2023-10-03T10:10:35.519349+0000","last_undegraded":"2023-10-03T10:10:35.519349+0000","last_fullsized":"2023-10-03T10:10:35.519349+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.520383+0000","last_change":"2023-10-03T10:10:35.517815+0000","last_active":"2023-10-03T10:10:35.520383+0000","last_peered":"2023-10-03T10:10:35.520383+0000","last_clean":"2023-10-03T10:10:35.520383+0000","last_became_active":"2023-10-03T10:10:35.517465+0000","last_became_peered":"2023-10-03T10:10:35.517465+0000","last_unstale":"2023-10-03T10:10:35.520383+0000","last_undegraded":"2023-10-03T10:10:35.520383+0000","last_fullsized":"2023-10-03T10:10:35.520383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"16'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.516092+0000","last_change":"2023-10-03T10:09:43.750463+0000","last_active":"2023-10-03T10:10:35.516092+0000","last_peered":"2023-10-03T10:10:35.516092+0000","last_clean":"2023-10-03T10:10:35.516092+0000","last_became_active":"2023-10-03T10:09:43.749780+0000","last_became_peered":"2023-10-03T10:09:43.749780+0000","last_unstale":"2023-10-03T10:10:35.516092+0000","last_undegraded":"2023-10-03T10:10:35.516092+0000","last_fullsized":"2023-10-03T10:10:35.516092+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:42.718400+0000","last_clean_scrub_stamp":"2023-10-03T10:09:42.718400+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-03T10:12:13.360700+0000","last_change":"2023-10-03T10:09:44.180378+0000","last_active":"2023-10-03T10:12:13.360700+0000","last_peered":"2023-10-03T10:12:13.360700+0000","last_clean":"2023-10-03T10:12:13.360700+0000","last_became_active":"2023-10-03T10:09:44.180082+0000","last_became_peered":"2023-10-03T10:09:44.180082+0000","last_unstale":"2023-10-03T10:12:13.360700+0000","last_undegraded":"2023-10-03T10:12:13.360700+0000","last_fullsized":"2023-10-03T10:12:13.360700+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593684+0000","last_change":"2023-10-03T10:09:44.178067+0000","last_active":"2023-10-03T10:10:35.593684+0000","last_peered":"2023-10-03T10:10:35.593684+0000","last_clean":"2023-10-03T10:10:35.593684+0000","last_became_active":"2023-10-03T10:09:44.175124+0000","last_became_peered":"2023-10-03T10:09:44.175124+0000","last_unstale":"2023-10-03T10:10:35.593684+0000","last_undegraded":"2023-10-03T10:10:35.593684+0000","last_fullsized":"2023-10-03T10:10:35.593684+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518308+0000","last_change":"2023-10-03T10:09:44.192156+0000","last_active":"2023-10-03T10:10:35.518308+0000","last_peered":"2023-10-03T10:10:35.518308+0000","last_clean":"2023-10-03T10:10:35.518308+0000","last_became_active":"2023-10-03T10:09:44.189296+0000","last_became_peered":"2023-10-03T10:09:44.189296+0000","last_unstale":"2023-10-03T10:10:35.518308+0000","last_undegraded":"2023-10-03T10:10:35.518308+0000","last_fullsized":"2023-10-03T10:10:35.518308+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.514071+0000","last_change":"2023-10-03T10:09:44.168473+0000","last_active":"2023-10-03T10:10:35.514071+0000","last_peered":"2023-10-03T10:10:35.514071+0000","last_clean":"2023-10-03T10:10:35.514071+0000","last_became_active":"2023-10-03T10:09:44.168155+0000","last_became_peered":"2023-10-03T10:09:44.168155+0000","last_unstale":"2023-10-03T10:10:35.514071+0000","last_undegraded":"2023-10-03T10:10:35.514071+0000","last_fullsized":"2023-10-03T10:10:35.514071+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.518916+0000","last_change":"2023-10-03T10:09:44.197689+0000","last_active":"2023-10-03T10:10:35.518916+0000","last_peered":"2023-10-03T10:10:35.518916+0000","last_clean":"2023-10-03T10:10:35.518916+0000","last_became_active":"2023-10-03T10:09:44.197410+0000","last_became_peered":"2023-10-03T10:09:44.197410+0000","last_unstale":"2023-10-03T10:10:35.518916+0000","last_undegraded":"2023-10-03T10:10:35.518916+0000","last_fullsized":"2023-10-03T10:10:35.518916+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-03T10:10:35.593682+0000","last_change":"2023-10-03T10:09:44.176966+0000","last_active":"2023-10-03T10:10:35.593682+0000","last_peered":"2023-10-03T10:10:35.593682+0000","last_clean":"2023-10-03T10:10:35.593682+0000","last_became_active":"2023-10-03T10:09:44.176386+0000","last_became_peered":"2023-10-03T10:09:44.176386+0000","last_unstale":"2023-10-03T10:10:35.593682+0000","last_undegraded":"2023-10-03T10:10:35.593682+0000","last_fullsized":"2023-10-03T10:10:35.593682+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-03T10:09:43.722422+0000","last_clean_scrub_stamp":"2023-10-03T10:09:43.722422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4333,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705697,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297292,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074548,"statfs":{"total":96636764160,"available":96332337152,"internally_reserved":0,"allocated":200704,"data_stored":67449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 3 10:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.20000000000000001,"5min":0.159,"15min":0.159},"max":{"1min":0.41399999999999998,"5min":0.504,"15min":0.504},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.375,"5min":0.36799999999999999,"15min":0.36699999999999999},"min":{"1min":0.27200000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.49099999999999999,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.52200000000000002}]},{"osd":1,"last update":"Tue Oct 3 10:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.308,"15min":0.30599999999999999},"min":{"1min":0.23999999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.46600000000000003,"5min":0.498,"15min":0.498},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.32700000000000001,"15min":0.32400000000000001},"min":{"1min":0.23300000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.38300000000000001}]},{"osd":2,"last update":"Tue Oct 3 10:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.33800000000000002,"15min":0.34000000000000002},"min":{"1min":0.245,"5min":0.218,"15min":0.218},"max":{"1min":0.45100000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.29999999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.48899999999999999}]}]},{"osd":2,"up_from":9,"seq":38654705696,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297280,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074560,"statfs":{"total":96636764160,"available":96332349440,"internally_reserved":0,"allocated":188416,"data_stored":67255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 3 10:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.33300000000000002,"15min":0.33500000000000002},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.51900000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.35599999999999998,"15min":0.35799999999999998},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.36699999999999999}]},{"osd":1,"last update":"Tue Oct 3 10:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.311},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.35299999999999998,"15min":0.35699999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.47499999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"last":0.32700000000000001}]},{"osd":3,"last update":"Tue Oct 3 10:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.376,"15min":0.377},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.373,"15min":0.375},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.54800000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.29999999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705697,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297280,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074560,"statfs":{"total":96636764160,"available":96332349440,"internally_reserved":0,"allocated":188416,"data_stored":60352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 3 10:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.39700000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.33900000000000002,"15min":0.34100000000000003},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.39800000000000002,"5min":0.53500000000000003,"15min":0.53500000000000003},"last":0.35199999999999998}]},{"osd":2,"last update":"Tue Oct 3 10:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.374,"15min":0.38100000000000001},"min":{"1min":0.26100000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.41099999999999998,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.318},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.373,"15min":0.38100000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.41299999999999998,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.32900000000000001}]},{"osd":3,"last update":"Tue Oct 3 10:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.371,"15min":0.375},"min":{"1min":0.27700000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.42199999999999999,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.373},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.371,"15min":0.375},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.42099999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.34399999999999997}]}]},{"osd":1,"up_from":9,"seq":38654705697,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297280,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074560,"statfs":{"total":96636764160,"available":96332349440,"internally_reserved":0,"allocated":188416,"data_stored":64338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 3 10:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.32100000000000001,"15min":0.32300000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.39100000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.34599999999999997,"15min":0.34699999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.45600000000000002,"5min":0.504,"15min":0.504},"last":0.28799999999999998}]},{"osd":2,"last update":"Tue Oct 3 10:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.33400000000000002,"15min":0.33600000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.52200000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.35899999999999999,"15min":0.36099999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.44400000000000001,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.52700000000000002}]},{"osd":3,"last update":"Tue Oct 3 10:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.35499999999999998,"15min":0.35999999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.41099999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.372},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.38200000000000001,"15min":0.38500000000000001},"min":{"1min":0.31,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.47499999999999998,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.80000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-03T10:12:16.902 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-03T10:12:16.902 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-03T10:12:16.902 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-03T10:12:16.902 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-03T10:12:17.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.041+0000 7f241ff40700 1 Processor -- start 2023-10-03T10:12:17.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.041+0000 7f241ff40700 1 -- start start 2023-10-03T10:12:17.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.042+0000 7f241ff40700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f241809e1d0 0x7f2418135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:17.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.042+0000 7f241ff40700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2418135e50 0x7f2418138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:17.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.042+0000 7f241ff40700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f241809f130 con 0x7f241809d6f0 2023-10-03T10:12:17.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.042+0000 7f241ff40700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f241809f520 con 0x7f2418135e50 2023-10-03T10:12:17.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.042+0000 7f241ff40700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241809e930 con 0x7f241809e1d0 2023-10-03T10:12:17.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.042+0000 7f241e4dd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2418135e50 0x7f2418138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:17.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.042+0000 7f241dcdc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f241809d6f0 0x7f241809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35678/0 (socket says 172.21.15.3:35678) 2023-10-03T10:12:17.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.042+0000 7f241dcdc700 1 -- 172.21.15.3:0/937731861 learned_addr learned my addr 172.21.15.3:0/937731861 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:17.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.042+0000 7f241d4db700 1 --2- 172.21.15.3:0/937731861 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f241809e1d0 0x7f2418135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:17.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.042+0000 7f241d4db700 1 -- 172.21.15.3:0/937731861 >> v2:172.21.15.3:3300/0 conn(0x7f2418135e50 msgr2=0x7f2418138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:17.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.042+0000 7f241d4db700 1 --2- 172.21.15.3:0/937731861 >> v2:172.21.15.3:3300/0 conn(0x7f2418135e50 0x7f2418138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:17.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.042+0000 7f241d4db700 1 -- 172.21.15.3:0/937731861 >> v1:172.21.15.3:6789/0 conn(0x7f241809d6f0 legacy=0x7f241809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:17.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.042+0000 7f241d4db700 1 -- 172.21.15.3:0/937731861 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241809b2d0 con 0x7f241809e1d0 2023-10-03T10:12:17.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.042+0000 7f241e4dd700 1 --2- 172.21.15.3:0/937731861 >> v2:172.21.15.3:3300/0 conn(0x7f2418135e50 0x7f2418138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:17.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.043+0000 7f241d4db700 1 --2- 172.21.15.3:0/937731861 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f241809e1d0 0x7f2418135910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2408009870 tx=0x7f2408009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be704d8104223ad2 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:17.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.043+0000 7f241ccda700 1 -- 172.21.15.3:0/937731861 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2408011830 con 0x7f241809e1d0 2023-10-03T10:12:17.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.043+0000 7f241ccda700 1 -- 172.21.15.3:0/937731861 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2408011990 con 0x7f241809e1d0 2023-10-03T10:12:17.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.043+0000 7f241ccda700 1 -- 172.21.15.3:0/937731861 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2408011d20 con 0x7f241809e1d0 2023-10-03T10:12:17.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.043+0000 7f241ff40700 1 -- 172.21.15.3:0/937731861 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f241809e1d0 msgr2=0x7f2418135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:17.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.043+0000 7f241ff40700 1 --2- 172.21.15.3:0/937731861 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f241809e1d0 0x7f2418135910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2408009870 tx=0x7f2408009b80 comp rx=0 tx=0).stop 2023-10-03T10:12:17.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.043+0000 7f241ff40700 1 -- 172.21.15.3:0/937731861 shutdown_connections 2023-10-03T10:12:17.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.043+0000 7f241ff40700 1 --2- 172.21.15.3:0/937731861 >> v2:172.21.15.3:3300/0 conn(0x7f2418135e50 0x7f2418138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:17.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.043+0000 7f241ff40700 1 --2- 172.21.15.3:0/937731861 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f241809e1d0 0x7f2418135910 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:17.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.043+0000 7f241ff40700 1 -- 172.21.15.3:0/937731861 >> 172.21.15.3:0/937731861 conn(0x7f2418066f50 msgr2=0x7f241809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:17.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.043+0000 7f241ff40700 1 -- 172.21.15.3:0/937731861 shutdown_connections 2023-10-03T10:12:17.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.043+0000 7f241ff40700 1 -- 172.21.15.3:0/937731861 wait complete. 2023-10-03T10:12:17.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.044+0000 7f241ff40700 1 Processor -- start 2023-10-03T10:12:17.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.044+0000 7f241ff40700 1 -- start start 2023-10-03T10:12:17.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.045+0000 7f241ff40700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f241809d6f0 0x7f241806f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:17.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.045+0000 7f241ff40700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f241809e1d0 0x7f241806f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:17.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.045+0000 7f241ff40700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2418135e50 0x7f24181c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.045+0000 7f241ff40700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2418124460 con 0x7f241809e1d0 2023-10-03T10:12:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.045+0000 7f241ff40700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24181250a0 con 0x7f241809d6f0 2023-10-03T10:12:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.045+0000 7f241ff40700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f241806ed30 con 0x7f2418135e50 2023-10-03T10:12:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.045+0000 7f241d4db700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f241809e1d0 0x7f241806f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.045+0000 7f241e4dd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2418135e50 0x7f24181c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.045+0000 7f241dcdc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f241809d6f0 0x7f241806f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.045+0000 7f241d4db700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f241809e1d0 0x7f241806f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44300/0 (socket says 172.21.15.3:44300) 2023-10-03T10:12:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.045+0000 7f241d4db700 1 -- 172.21.15.3:0/2974218915 learned_addr learned my addr 172.21.15.3:0/2974218915 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.045+0000 7f241d4db700 1 -- 172.21.15.3:0/2974218915 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2418135e50 msgr2=0x7f24181c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.045+0000 7f241d4db700 1 --2- 172.21.15.3:0/2974218915 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2418135e50 0x7f24181c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.045+0000 7f241d4db700 1 -- 172.21.15.3:0/2974218915 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f241809d6f0 msgr2=0x7f241806f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.045+0000 7f241d4db700 1 --2- 172.21.15.3:0/2974218915 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f241809d6f0 0x7f241806f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.045+0000 7f241d4db700 1 -- 172.21.15.3:0/2974218915 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2410009a70 con 0x7f241809e1d0 2023-10-03T10:12:17.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.045+0000 7f241dcdc700 1 --2- 172.21.15.3:0/2974218915 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f241809d6f0 0x7f241806f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:17.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.046+0000 7f241d4db700 1 --2- 172.21.15.3:0/2974218915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f241809e1d0 0x7f241806f5e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2408009840 tx=0x7f2408004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:17.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.046+0000 7f240e7fc700 1 -- 172.21.15.3:0/2974218915 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24080036a0 con 0x7f241809e1d0 2023-10-03T10:12:17.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.046+0000 7f240e7fc700 1 -- 172.21.15.3:0/2974218915 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f240801f070 con 0x7f241809e1d0 2023-10-03T10:12:17.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.046+0000 7f241ff40700 1 -- 172.21.15.3:0/2974218915 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24080094a0 con 0x7f241809e1d0 2023-10-03T10:12:17.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.046+0000 7f240e7fc700 1 -- 172.21.15.3:0/2974218915 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2408003a60 con 0x7f241809e1d0 2023-10-03T10:12:17.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.046+0000 7f241ff40700 1 -- 172.21.15.3:0/2974218915 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24180a1f50 con 0x7f241809e1d0 2023-10-03T10:12:17.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.046+0000 7f240e7fc700 1 -- 172.21.15.3:0/2974218915 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7f2408003bc0 con 0x7f241809e1d0 2023-10-03T10:12:17.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.047+0000 7f240e7fc700 1 -- 172.21.15.3:0/2974218915 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4078+0+0 (secure 0 0 0) 0x7f2408021070 con 0x7f241809e1d0 2023-10-03T10:12:17.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.047+0000 7f241ff40700 1 -- 172.21.15.3:0/2974218915 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24181251e0 con 0x7f241809e1d0 2023-10-03T10:12:17.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.052+0000 7f240e7fc700 1 -- 172.21.15.3:0/2974218915 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f24181251e0 con 0x7f241809e1d0 2023-10-03T10:12:17.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.303+0000 7f241ff40700 1 -- 172.21.15.3:0/2974218915 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f24181251e0 con 0x7f241809e1d0 2023-10-03T10:12:17.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.303+0000 7f240e7fc700 1 -- 172.21.15.3:0/2974218915 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f24181251e0 con 0x7f241809e1d0 2023-10-03T10:12:17.304 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:12:17.304 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-03T10:12:17.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.305+0000 7f241ff40700 1 -- 172.21.15.3:0/2974218915 >> v1:172.21.15.3:6816/127025 conn(0x7f23fc02f480 legacy=0x7f23fc031930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:17.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.305+0000 7f241ff40700 1 -- 172.21.15.3:0/2974218915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f241809e1d0 msgr2=0x7f241806f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:17.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.305+0000 7f241ff40700 1 --2- 172.21.15.3:0/2974218915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f241809e1d0 0x7f241806f5e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2408009840 tx=0x7f2408004750 comp rx=0 tx=0).stop 2023-10-03T10:12:17.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.305+0000 7f241ff40700 1 -- 172.21.15.3:0/2974218915 shutdown_connections 2023-10-03T10:12:17.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.305+0000 7f241ff40700 1 --2- 172.21.15.3:0/2974218915 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2418135e50 0x7f24181c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:17.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.305+0000 7f241ff40700 1 --2- 172.21.15.3:0/2974218915 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f241809d6f0 0x7f241806f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:17.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.305+0000 7f241ff40700 1 --2- 172.21.15.3:0/2974218915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f241809e1d0 0x7f241806f5e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:17.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.305+0000 7f241ff40700 1 -- 172.21.15.3:0/2974218915 >> 172.21.15.3:0/2974218915 conn(0x7f2418066f50 msgr2=0x7f2418134840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:17.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.305+0000 7f241ff40700 1 -- 172.21.15.3:0/2974218915 shutdown_connections 2023-10-03T10:12:17.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.305+0000 7f241ff40700 1 -- 172.21.15.3:0/2974218915 wait complete. 2023-10-03T10:12:17.319 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-03T10:12:17.319 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-03T10:12:17.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.454+0000 7fb1c035c700 1 Processor -- start 2023-10-03T10:12:17.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.454+0000 7fb1c035c700 1 -- start start 2023-10-03T10:12:17.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.455+0000 7fb1c035c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1b809d6f0 0x7fb1b809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:17.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.455+0000 7fb1c035c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb1b809e000 0x7fb1b81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:17.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.455+0000 7fb1c035c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1b809b120 con 0x7fb1b809e410 2023-10-03T10:12:17.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.455+0000 7fb1c035c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b806b460 con 0x7fb1b809d6f0 2023-10-03T10:12:17.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.455+0000 7fb1c035c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b809a7e0 con 0x7fb1b809e000 2023-10-03T10:12:17.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.455+0000 7fb1be0f8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1b809d6f0 0x7fb1b809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:17.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.455+0000 7fb1be8f9700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb1b809e410 0x7fb1b8133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35694/0 (socket says 172.21.15.3:35694) 2023-10-03T10:12:17.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.455+0000 7fb1be8f9700 1 -- 172.21.15.3:0/3868300590 learned_addr learned my addr 172.21.15.3:0/3868300590 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:17.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.455+0000 7fb1bd8f7700 1 --2- 172.21.15.3:0/3868300590 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb1b809e000 0x7fb1b81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:17.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.456+0000 7fb1bd0f6700 1 -- 172.21.15.3:0/3868300590 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2596887248 0 0) 0x7fb1b809b120 con 0x7fb1b809e410 2023-10-03T10:12:17.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.456+0000 7fb1bd0f6700 1 -- 172.21.15.3:0/3868300590 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1a00035b0 con 0x7fb1b809e410 2023-10-03T10:12:17.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.456+0000 7fb1be0f8700 1 -- 172.21.15.3:0/3868300590 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb1b809e000 msgr2=0x7fb1b81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:17.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.456+0000 7fb1be0f8700 1 --2- 172.21.15.3:0/3868300590 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb1b809e000 0x7fb1b81339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.456+0000 7fb1be0f8700 1 -- 172.21.15.3:0/3868300590 >> v1:172.21.15.3:6789/0 conn(0x7fb1b809e410 legacy=0x7fb1b8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.456+0000 7fb1be0f8700 1 -- 172.21.15.3:0/3868300590 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b809a500 con 0x7fb1b809d6f0 2023-10-03T10:12:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.456+0000 7fb1bd8f7700 1 --2- 172.21.15.3:0/3868300590 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb1b809e000 0x7fb1b81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.456+0000 7fb1be0f8700 1 --2- 172.21.15.3:0/3868300590 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1b809d6f0 0x7fb1b809dac0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009870 tx=0x7fb1a8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bd6c4cbf2caabc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.456+0000 7fb1bd0f6700 1 -- 172.21.15.3:0/3868300590 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8011830 con 0x7fb1b809d6f0 2023-10-03T10:12:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.456+0000 7fb1bd0f6700 1 -- 172.21.15.3:0/3868300590 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a8011990 con 0x7fb1b809d6f0 2023-10-03T10:12:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.456+0000 7fb1bd0f6700 1 -- 172.21.15.3:0/3868300590 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8011d20 con 0x7fb1b809d6f0 2023-10-03T10:12:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.457+0000 7fb1c035c700 1 -- 172.21.15.3:0/3868300590 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1b809d6f0 msgr2=0x7fb1b809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.457+0000 7fb1c035c700 1 --2- 172.21.15.3:0/3868300590 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1b809d6f0 0x7fb1b809dac0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009870 tx=0x7fb1a8009b80 comp rx=0 tx=0).stop 2023-10-03T10:12:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.457+0000 7fb1c035c700 1 -- 172.21.15.3:0/3868300590 shutdown_connections 2023-10-03T10:12:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.457+0000 7fb1c035c700 1 --2- 172.21.15.3:0/3868300590 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1b809d6f0 0x7fb1b809dac0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.457+0000 7fb1c035c700 1 --2- 172.21.15.3:0/3868300590 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb1b809e000 0x7fb1b81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.457+0000 7fb1c035c700 1 -- 172.21.15.3:0/3868300590 >> 172.21.15.3:0/3868300590 conn(0x7fb1b8066f50 msgr2=0x7fb1b809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.457+0000 7fb1c035c700 1 -- 172.21.15.3:0/3868300590 shutdown_connections 2023-10-03T10:12:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.457+0000 7fb1c035c700 1 -- 172.21.15.3:0/3868300590 wait complete. 2023-10-03T10:12:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.458+0000 7fb1c035c700 1 Processor -- start 2023-10-03T10:12:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.458+0000 7fb1c035c700 1 -- start start 2023-10-03T10:12:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.458+0000 7fb1c035c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1b809d6f0 0x7fb1b8122340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.458+0000 7fb1c035c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1b809e000 0x7fb1b8122880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.458+0000 7fb1c035c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb1b809e410 0x7fb1b8122dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.458+0000 7fb1c035c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b8133680 con 0x7fb1b809e000 2023-10-03T10:12:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.458+0000 7fb1c035c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b81237a0 con 0x7fb1b809d6f0 2023-10-03T10:12:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.458+0000 7fb1c035c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b8124d50 con 0x7fb1b809e410 2023-10-03T10:12:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.458+0000 7fb1be0f8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1b809d6f0 0x7fb1b8122340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.458+0000 7fb1be8f9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb1b809e410 0x7fb1b8122dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.458+0000 7fb1be0f8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1b809d6f0 0x7fb1b8122340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37522/0 (socket says 172.21.15.3:37522) 2023-10-03T10:12:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.458+0000 7fb1be0f8700 1 -- 172.21.15.3:0/3914123330 learned_addr learned my addr 172.21.15.3:0/3914123330 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.458+0000 7fb1bd8f7700 1 --2- 172.21.15.3:0/3914123330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1b809e000 0x7fb1b8122880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.459+0000 7fb1be0f8700 1 -- 172.21.15.3:0/3914123330 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb1b809e410 msgr2=0x7fb1b8122dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.459+0000 7fb1be0f8700 1 --2- 172.21.15.3:0/3914123330 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb1b809e410 0x7fb1b8122dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.459+0000 7fb1be0f8700 1 -- 172.21.15.3:0/3914123330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1b809e000 msgr2=0x7fb1b8122880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.459+0000 7fb1be0f8700 1 --2- 172.21.15.3:0/3914123330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1b809e000 0x7fb1b8122880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:17.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.459+0000 7fb1be0f8700 1 -- 172.21.15.3:0/3914123330 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ac0094a0 con 0x7fb1b809d6f0 2023-10-03T10:12:17.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.459+0000 7fb1be8f9700 1 --2- 172.21.15.3:0/3914123330 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb1b809e410 0x7fb1b8122dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:17.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.459+0000 7fb1be0f8700 1 --2- 172.21.15.3:0/3914123330 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1b809d6f0 0x7fb1b8122340 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009ed0 tx=0x7fb1a8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:17.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.459+0000 7fb1b6ffd700 1 -- 172.21.15.3:0/3914123330 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8004450 con 0x7fb1b809d6f0 2023-10-03T10:12:17.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.459+0000 7fb1b6ffd700 1 -- 172.21.15.3:0/3914123330 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a801f070 con 0x7fb1b809d6f0 2023-10-03T10:12:17.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.459+0000 7fb1b6ffd700 1 -- 172.21.15.3:0/3914123330 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a80038e0 con 0x7fb1b809d6f0 2023-10-03T10:12:17.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.459+0000 7fb1c035c700 1 -- 172.21.15.3:0/3914123330 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a80094a0 con 0x7fb1b809d6f0 2023-10-03T10:12:17.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.459+0000 7fb1c035c700 1 -- 172.21.15.3:0/3914123330 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b8125560 con 0x7fb1b809d6f0 2023-10-03T10:12:17.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.460+0000 7fb1b6ffd700 1 -- 172.21.15.3:0/3914123330 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7fb1a8003a40 con 0x7fb1b809d6f0 2023-10-03T10:12:17.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.460+0000 7fb1b6ffd700 1 -- 172.21.15.3:0/3914123330 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4078+0+0 (secure 0 0 0) 0x7fb1a8021070 con 0x7fb1b809d6f0 2023-10-03T10:12:17.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.460+0000 7fb1c035c700 1 -- 172.21.15.3:0/3914123330 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1b8125110 con 0x7fb1b809d6f0 2023-10-03T10:12:17.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.466+0000 7fb1b6ffd700 1 -- 172.21.15.3:0/3914123330 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb1a802b080 con 0x7fb1b809d6f0 2023-10-03T10:12:17.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.670+0000 7fb1c035c700 1 -- 172.21.15.3:0/3914123330 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1b811f030 con 0x7fb1b809d6f0 2023-10-03T10:12:17.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.671+0000 7fb1b6ffd700 1 -- 172.21.15.3:0/3914123330 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+12923 (secure 0 0 0) 0x7fb1a8015020 con 0x7fb1b809d6f0 2023-10-03T10:12:17.672 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:12:17.672 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","created":"2023-10-03T10:09:28.703441+0000","modified":"2023-10-03T10:12:15.315657+0000","last_up_change":"2023-10-03T10:09:36.569401+0000","last_in_change":"2023-10-03T10:09:30.156546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-03T10:09:36.623971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-03T10:09:38.210614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-03T10:09:42.695873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-03T10:09:43.115588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cdcdd930-3c5b-4486-a021-7132f90fb6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6808","nonce":118845}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6809","nonce":118845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6811","nonce":118845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6810","nonce":118845}]},"public_addr":"172.21.15.3:6808/118845","cluster_addr":"172.21.15.3:6809/118845","heartbeat_back_addr":"172.21.15.3:6811/118845","heartbeat_front_addr":"172.21.15.3:6810/118845","state":["exists","up"]},{"osd":1,"uuid":"c8e58877-6762-4741-a8c1-567ebc97b708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6804","nonce":118835}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6805","nonce":118835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6807","nonce":118835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6806","nonce":118835}]},"public_addr":"172.21.15.3:6804/118835","cluster_addr":"172.21.15.3:6805/118835","heartbeat_back_addr":"172.21.15.3:6807/118835","heartbeat_front_addr":"172.21.15.3:6806/118835","state":["exists","up"]},{"osd":2,"uuid":"68a2a21a-7a8e-4f82-8433-5201faa50777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6812","nonce":118864}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6813","nonce":118864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6815","nonce":118864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6814","nonce":118864}]},"public_addr":"172.21.15.3:6812/118864","cluster_addr":"172.21.15.3:6813/118864","heartbeat_back_addr":"172.21.15.3:6815/118864","heartbeat_front_addr":"172.21.15.3:6814/118864","state":["exists","up"]},{"osd":3,"uuid":"c4df2903-4e1e-4208-bf77-95b9fa749241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6800","nonce":118836}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6801","nonce":118836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6803","nonce":118836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6802","nonce":118836}]},"public_addr":"172.21.15.3:6800/118836","cluster_addr":"172.21.15.3:6801/118836","heartbeat_back_addr":"172.21.15.3:6803/118836","heartbeat_front_addr":"172.21.15.3:6802/118836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.369708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.257825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.337931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.085222+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1","mappings":[{"from":3,"to":0}]},{"pgid":"4.8","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.19","mappings":[{"from":3,"to":0}]},{"pgid":"4.2a","mappings":[{"from":1,"to":2}]},{"pgid":"4.2d","mappings":[{"from":3,"to":0}]},{"pgid":"4.33","mappings":[{"from":3,"to":0}]},{"pgid":"4.39","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/3919724564":"2023-10-04T10:12:13.305411+0000","172.21.15.3:0/2091814084":"2023-10-04T10:12:13.305411+0000","172.21.15.3:6816/118296":"2023-10-04T10:12:13.305411+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-03T10:12:17.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.673+0000 7fb1c035c700 1 -- 172.21.15.3:0/3914123330 >> v1:172.21.15.3:6816/127025 conn(0x7fb1a402f4b0 legacy=0x7fb1a4031960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:17.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.673+0000 7fb1c035c700 1 -- 172.21.15.3:0/3914123330 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1b809d6f0 msgr2=0x7fb1b8122340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:17.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.673+0000 7fb1c035c700 1 --2- 172.21.15.3:0/3914123330 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1b809d6f0 0x7fb1b8122340 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009ed0 tx=0x7fb1a8004750 comp rx=0 tx=0).stop 2023-10-03T10:12:17.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.673+0000 7fb1c035c700 1 -- 172.21.15.3:0/3914123330 shutdown_connections 2023-10-03T10:12:17.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.673+0000 7fb1c035c700 1 --2- 172.21.15.3:0/3914123330 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb1b809e410 0x7fb1b8122dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:17.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.673+0000 7fb1c035c700 1 --2- 172.21.15.3:0/3914123330 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1b809d6f0 0x7fb1b8122340 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:17.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.673+0000 7fb1c035c700 1 --2- 172.21.15.3:0/3914123330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1b809e000 0x7fb1b8122880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:17.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.673+0000 7fb1c035c700 1 -- 172.21.15.3:0/3914123330 >> 172.21.15.3:0/3914123330 conn(0x7fb1b8066f50 msgr2=0x7fb1b8133e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:17.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.673+0000 7fb1c035c700 1 -- 172.21.15.3:0/3914123330 shutdown_connections 2023-10-03T10:12:17.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.673+0000 7fb1c035c700 1 -- 172.21.15.3:0/3914123330 wait complete. 2023-10-03T10:12:17.687 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-03T10:09:36.623971+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}, 'mgr_devicehealth': {}}, 'read_balance': {'score_acting': 4, 'score_stable': 4, 'optimal_score': 0.5, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-03T10:09:38.210614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-03T10:09:42.695873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-03T10:09:43.115588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.059999942779541, 'score_stable': 1.059999942779541, 'optimal_score': 1, 'raw_score_acting': 1.059999942779541, 'raw_score_stable': 1.059999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-03T10:12:17.688 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-03T10:12:17.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.826+0000 7f863cb70700 1 Processor -- start 2023-10-03T10:12:17.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.826+0000 7f863cb70700 1 -- start start 2023-10-03T10:12:17.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.827+0000 7f863cb70700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8638071cd0 0x7f86380720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:17.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.827+0000 7f863cb70700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f863809f350 0x7f863813bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:17.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.827+0000 7f863cb70700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f863806bcc0 con 0x7f863809e850 2023-10-03T10:12:17.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.827+0000 7f863cb70700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f863806ba00 con 0x7f863809f350 2023-10-03T10:12:17.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.827+0000 7f863cb70700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f863809f7f0 con 0x7f8638071cd0 2023-10-03T10:12:17.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.827+0000 7f863659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8638071cd0 0x7f86380720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:17.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.827+0000 7f8636d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f863809f350 0x7f863813bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:17.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.827+0000 7f8635d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f863809e850 0x7f863809ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35702/0 (socket says 172.21.15.3:35702) 2023-10-03T10:12:17.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.827+0000 7f8635d9b700 1 -- 172.21.15.3:0/2584206638 learned_addr learned my addr 172.21.15.3:0/2584206638 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:17.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.827+0000 7f8636d9d700 1 -- 172.21.15.3:0/2584206638 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8638071cd0 msgr2=0x7f86380720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:17.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.827+0000 7f8636d9d700 1 --2- 172.21.15.3:0/2584206638 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8638071cd0 0x7f86380720a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:17.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.827+0000 7f8636d9d700 1 -- 172.21.15.3:0/2584206638 >> v1:172.21.15.3:6789/0 conn(0x7f863809e850 legacy=0x7f863809ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:17.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.827+0000 7f8636d9d700 1 -- 172.21.15.3:0/2584206638 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f863806c0f0 con 0x7f863809f350 2023-10-03T10:12:17.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.827+0000 7f863659c700 1 --2- 172.21.15.3:0/2584206638 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8638071cd0 0x7f86380720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:17.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.827+0000 7f863559a700 1 -- 172.21.15.3:0/2584206638 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2926781370 0 0) 0x7f863806bcc0 con 0x7f863809e850 2023-10-03T10:12:17.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.828+0000 7f8636d9d700 1 --2- 172.21.15.3:0/2584206638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f863809f350 0x7f863813bbb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8630009d90 tx=0x7f863000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b20c0f2fa4e52d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:17.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.828+0000 7f863559a700 1 -- 172.21.15.3:0/2584206638 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863000e070 con 0x7f863809f350 2023-10-03T10:12:17.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.828+0000 7f863559a700 1 -- 172.21.15.3:0/2584206638 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f863000bb20 con 0x7f863809f350 2023-10-03T10:12:17.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.828+0000 7f863559a700 1 -- 172.21.15.3:0/2584206638 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8630012430 con 0x7f863809f350 2023-10-03T10:12:17.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.828+0000 7f863cb70700 1 -- 172.21.15.3:0/2584206638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f863809f350 msgr2=0x7f863813bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:17.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.828+0000 7f863cb70700 1 --2- 172.21.15.3:0/2584206638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f863809f350 0x7f863813bbb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8630009d90 tx=0x7f863000b330 comp rx=0 tx=0).stop 2023-10-03T10:12:17.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.828+0000 7f863cb70700 1 -- 172.21.15.3:0/2584206638 shutdown_connections 2023-10-03T10:12:17.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.828+0000 7f863cb70700 1 --2- 172.21.15.3:0/2584206638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f863809f350 0x7f863813bbb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:17.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.828+0000 7f863cb70700 1 --2- 172.21.15.3:0/2584206638 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8638071cd0 0x7f86380720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:17.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.828+0000 7f863cb70700 1 -- 172.21.15.3:0/2584206638 >> 172.21.15.3:0/2584206638 conn(0x7f86380a4130 msgr2=0x7f86380a4530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:17.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.828+0000 7f863cb70700 1 -- 172.21.15.3:0/2584206638 shutdown_connections 2023-10-03T10:12:17.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.829+0000 7f863cb70700 1 -- 172.21.15.3:0/2584206638 wait complete. 2023-10-03T10:12:17.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.830+0000 7f863cb70700 1 Processor -- start 2023-10-03T10:12:17.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.830+0000 7f863cb70700 1 -- start start 2023-10-03T10:12:17.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.830+0000 7f863cb70700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8638071cd0 0x7f86381a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:17.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.830+0000 7f863cb70700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f863809e850 0x7f86381a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:17.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.830+0000 7f863cb70700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f863809f350 0x7f86381a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:17.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.830+0000 7f863cb70700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86381a8310 con 0x7f8638071cd0 2023-10-03T10:12:17.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.830+0000 7f863cb70700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f863806cda0 con 0x7f863809e850 2023-10-03T10:12:17.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.830+0000 7f863cb70700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f86380a1370 con 0x7f863809f350 2023-10-03T10:12:17.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.830+0000 7f863659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8638071cd0 0x7f86381a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:17.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.830+0000 7f8635d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f863809e850 0x7f86381a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:17.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.830+0000 7f863659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8638071cd0 0x7f86381a5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44328/0 (socket says 172.21.15.3:44328) 2023-10-03T10:12:17.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.830+0000 7f863659c700 1 -- 172.21.15.3:0/3845932046 learned_addr learned my addr 172.21.15.3:0/3845932046 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:17.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.830+0000 7f8636d9d700 1 --2- 172.21.15.3:0/3845932046 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f863809f350 0x7f86381a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:17.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.831+0000 7f863659c700 1 -- 172.21.15.3:0/3845932046 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f863809f350 msgr2=0x7f86381a5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:17.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.831+0000 7f863659c700 1 --2- 172.21.15.3:0/3845932046 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f863809f350 0x7f86381a5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:17.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.831+0000 7f863659c700 1 -- 172.21.15.3:0/3845932046 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f863809e850 msgr2=0x7f86381a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:17.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.831+0000 7f863659c700 1 --2- 172.21.15.3:0/3845932046 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f863809e850 0x7f86381a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:17.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.831+0000 7f863659c700 1 -- 172.21.15.3:0/3845932046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86280094a0 con 0x7f8638071cd0 2023-10-03T10:12:17.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.831+0000 7f8636d9d700 1 --2- 172.21.15.3:0/3845932046 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f863809f350 0x7f86381a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:17.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.831+0000 7f8635d9b700 1 --2- 172.21.15.3:0/3845932046 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f863809e850 0x7f86381a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:17.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.831+0000 7f863659c700 1 --2- 172.21.15.3:0/3845932046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8638071cd0 0x7f86381a5060 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f862800ebe0 tx=0x7f862800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:17.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.831+0000 7f862effd700 1 -- 172.21.15.3:0/3845932046 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8628009760 con 0x7f8638071cd0 2023-10-03T10:12:17.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.831+0000 7f862effd700 1 -- 172.21.15.3:0/3845932046 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862800c850 con 0x7f8638071cd0 2023-10-03T10:12:17.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.831+0000 7f863cb70700 1 -- 172.21.15.3:0/3845932046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8630009a70 con 0x7f8638071cd0 2023-10-03T10:12:17.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.831+0000 7f862effd700 1 -- 172.21.15.3:0/3845932046 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862800cbe0 con 0x7f8638071cd0 2023-10-03T10:12:17.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.831+0000 7f863cb70700 1 -- 172.21.15.3:0/3845932046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86381a7ec0 con 0x7f8638071cd0 2023-10-03T10:12:17.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.832+0000 7f862effd700 1 -- 172.21.15.3:0/3845932046 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7f862800cd40 con 0x7f8638071cd0 2023-10-03T10:12:17.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.832+0000 7f862effd700 1 -- 172.21.15.3:0/3845932046 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4078+0+0 (secure 0 0 0) 0x7f862803fb10 con 0x7f8638071cd0 2023-10-03T10:12:17.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.832+0000 7f863cb70700 1 -- 172.21.15.3:0/3845932046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86381a8720 con 0x7f8638071cd0 2023-10-03T10:12:17.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:17.837+0000 7f862effd700 1 -- 172.21.15.3:0/3845932046 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8628017080 con 0x7f8638071cd0 2023-10-03T10:12:18.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.040+0000 7f863cb70700 1 -- 172.21.15.3:0/3845932046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f86381a8540 con 0x7f8638071cd0 2023-10-03T10:12:18.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.040+0000 7f862effd700 1 -- 172.21.15.3:0/3845932046 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8628021030 con 0x7f8638071cd0 2023-10-03T10:12:18.041 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-03T10:12:18.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.043+0000 7f863cb70700 1 -- 172.21.15.3:0/3845932046 >> v1:172.21.15.3:6816/127025 conn(0x7f861802f480 legacy=0x7f8618031930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:18.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.043+0000 7f863cb70700 1 -- 172.21.15.3:0/3845932046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8638071cd0 msgr2=0x7f86381a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:18.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.043+0000 7f863cb70700 1 --2- 172.21.15.3:0/3845932046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8638071cd0 0x7f86381a5060 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f862800ebe0 tx=0x7f862800eef0 comp rx=0 tx=0).stop 2023-10-03T10:12:18.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.043+0000 7f863cb70700 1 -- 172.21.15.3:0/3845932046 shutdown_connections 2023-10-03T10:12:18.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.043+0000 7f863cb70700 1 --2- 172.21.15.3:0/3845932046 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f863809f350 0x7f86381a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:18.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.043+0000 7f863cb70700 1 --2- 172.21.15.3:0/3845932046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8638071cd0 0x7f86381a5060 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:18.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.043+0000 7f863cb70700 1 --2- 172.21.15.3:0/3845932046 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f863809e850 0x7f86381a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:18.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.043+0000 7f863cb70700 1 -- 172.21.15.3:0/3845932046 >> 172.21.15.3:0/3845932046 conn(0x7f86380a4130 msgr2=0x7f86381ba050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:18.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.043+0000 7f863cb70700 1 -- 172.21.15.3:0/3845932046 shutdown_connections 2023-10-03T10:12:18.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.043+0000 7f863cb70700 1 -- 172.21.15.3:0/3845932046 wait complete. 2023-10-03T10:12:18.057 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-03T10:12:18.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.200+0000 7f391e405700 1 Processor -- start 2023-10-03T10:12:18.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.200+0000 7f391e405700 1 -- start start 2023-10-03T10:12:18.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.200+0000 7f391e405700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f39181349a0 0x7f3918136d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:18.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.200+0000 7f391e405700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f39181372c0 0x7f39181396a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:18.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.200+0000 7f391e405700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39180a01c0 con 0x7f3918131b30 2023-10-03T10:12:18.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.200+0000 7f391e405700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3918132610 con 0x7f39181349a0 2023-10-03T10:12:18.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.200+0000 7f391e405700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f391809f860 con 0x7f39181372c0 2023-10-03T10:12:18.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.200+0000 7f391c9a2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f39181372c0 0x7f39181396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:18.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.200+0000 7f39177fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f39181349a0 0x7f3918136d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:18.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.200+0000 7f3917fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3918131b30 0x7f3918131f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35710/0 (socket says 172.21.15.3:35710) 2023-10-03T10:12:18.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.200+0000 7f3917fff700 1 -- 172.21.15.3:0/1088986940 learned_addr learned my addr 172.21.15.3:0/1088986940 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:18.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.201+0000 7f3916ffd700 1 -- 172.21.15.3:0/1088986940 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3161259939 0 0) 0x7f39180a01c0 con 0x7f3918131b30 2023-10-03T10:12:18.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.201+0000 7f39177fe700 1 -- 172.21.15.3:0/1088986940 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f39181372c0 msgr2=0x7f39181396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:18.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.201+0000 7f39177fe700 1 --2- 172.21.15.3:0/1088986940 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f39181372c0 0x7f39181396a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:18.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.201+0000 7f39177fe700 1 -- 172.21.15.3:0/1088986940 >> v1:172.21.15.3:6789/0 conn(0x7f3918131b30 legacy=0x7f3918131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:18.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.201+0000 7f39177fe700 1 -- 172.21.15.3:0/1088986940 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f391809b560 con 0x7f39181349a0 2023-10-03T10:12:18.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.201+0000 7f391c9a2700 1 --2- 172.21.15.3:0/1088986940 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f39181372c0 0x7f39181396a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:18.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.201+0000 7f39177fe700 1 --2- 172.21.15.3:0/1088986940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39181349a0 0x7f3918136d80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f390c009840 tx=0x7f390c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02f23ba8cec8bea server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:18.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.201+0000 7f3916ffd700 1 -- 172.21.15.3:0/1088986940 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c011830 con 0x7f39181349a0 2023-10-03T10:12:18.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.201+0000 7f3916ffd700 1 -- 172.21.15.3:0/1088986940 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390c011990 con 0x7f39181349a0 2023-10-03T10:12:18.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.201+0000 7f3916ffd700 1 -- 172.21.15.3:0/1088986940 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c011d20 con 0x7f39181349a0 2023-10-03T10:12:18.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.202+0000 7f391e405700 1 -- 172.21.15.3:0/1088986940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39181349a0 msgr2=0x7f3918136d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:18.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.202+0000 7f391e405700 1 --2- 172.21.15.3:0/1088986940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39181349a0 0x7f3918136d80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f390c009840 tx=0x7f390c009b50 comp rx=0 tx=0).stop 2023-10-03T10:12:18.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.202+0000 7f391e405700 1 -- 172.21.15.3:0/1088986940 shutdown_connections 2023-10-03T10:12:18.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.202+0000 7f391e405700 1 --2- 172.21.15.3:0/1088986940 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f39181372c0 0x7f39181396a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:18.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.202+0000 7f391e405700 1 --2- 172.21.15.3:0/1088986940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39181349a0 0x7f3918136d80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:18.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.202+0000 7f391e405700 1 -- 172.21.15.3:0/1088986940 >> 172.21.15.3:0/1088986940 conn(0x7f39180a4130 msgr2=0x7f39180a4530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:18.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.202+0000 7f391e405700 1 -- 172.21.15.3:0/1088986940 shutdown_connections 2023-10-03T10:12:18.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.202+0000 7f391e405700 1 -- 172.21.15.3:0/1088986940 wait complete. 2023-10-03T10:12:18.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.203+0000 7f391e405700 1 Processor -- start 2023-10-03T10:12:18.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.203+0000 7f391e405700 1 -- start start 2023-10-03T10:12:18.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.204+0000 7f391e405700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3918131b30 0x7f391811b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:18.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.204+0000 7f391e405700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f39181349a0 0x7f391811b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:18.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.204+0000 7f391e405700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f39181372c0 0x7f391811bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:18.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.204+0000 7f391e405700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391811e770 con 0x7f3918131b30 2023-10-03T10:12:18.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.204+0000 7f391e405700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f391811ec90 con 0x7f39181349a0 2023-10-03T10:12:18.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.204+0000 7f391e405700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3918137090 con 0x7f39181372c0 2023-10-03T10:12:18.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.204+0000 7f39177fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f39181349a0 0x7f391811b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:18.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.204+0000 7f391c9a2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f39181372c0 0x7f391811bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:18.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.204+0000 7f39177fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f39181349a0 0x7f391811b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37546/0 (socket says 172.21.15.3:37546) 2023-10-03T10:12:18.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.204+0000 7f39177fe700 1 -- 172.21.15.3:0/1808211586 learned_addr learned my addr 172.21.15.3:0/1808211586 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:18.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.204+0000 7f3917fff700 1 --2- 172.21.15.3:0/1808211586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3918131b30 0x7f391811b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:18.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.205+0000 7f39177fe700 1 -- 172.21.15.3:0/1808211586 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f39181372c0 msgr2=0x7f391811bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:18.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.205+0000 7f39177fe700 1 --2- 172.21.15.3:0/1808211586 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f39181372c0 0x7f391811bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:18.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.205+0000 7f39177fe700 1 -- 172.21.15.3:0/1808211586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3918131b30 msgr2=0x7f391811b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:18.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.205+0000 7f39177fe700 1 --2- 172.21.15.3:0/1808211586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3918131b30 0x7f391811b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:18.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.205+0000 7f39177fe700 1 -- 172.21.15.3:0/1808211586 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3910009a70 con 0x7f39181349a0 2023-10-03T10:12:18.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.205+0000 7f39177fe700 1 --2- 172.21.15.3:0/1808211586 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f39181349a0 0x7f391811b9e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f390c009fd0 tx=0x7f390c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:18.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.205+0000 7f3914ff9700 1 -- 172.21.15.3:0/1808211586 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c0045d0 con 0x7f39181349a0 2023-10-03T10:12:18.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.205+0000 7f3914ff9700 1 -- 172.21.15.3:0/1808211586 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390c01f070 con 0x7f39181349a0 2023-10-03T10:12:18.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.205+0000 7f391e405700 1 -- 172.21.15.3:0/1808211586 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390c0094a0 con 0x7f39181349a0 2023-10-03T10:12:18.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.205+0000 7f3914ff9700 1 -- 172.21.15.3:0/1808211586 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c0038a0 con 0x7f39181349a0 2023-10-03T10:12:18.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.205+0000 7f391e405700 1 -- 172.21.15.3:0/1808211586 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f391809da70 con 0x7f39181349a0 2023-10-03T10:12:18.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.206+0000 7f3914ff9700 1 -- 172.21.15.3:0/1808211586 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f390c003a00 con 0x7f39181349a0 2023-10-03T10:12:18.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.206+0000 7f3914ff9700 1 -- 172.21.15.3:0/1808211586 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4078+0+0 (secure 0 0 0) 0x7f390c021070 con 0x7f39181349a0 2023-10-03T10:12:18.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.206+0000 7f391e405700 1 -- 172.21.15.3:0/1808211586 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f391811f7c0 con 0x7f39181349a0 2023-10-03T10:12:18.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.211+0000 7f3914ff9700 1 -- 172.21.15.3:0/1808211586 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f390c015080 con 0x7f39181349a0 2023-10-03T10:12:18.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.415+0000 7f391e405700 1 -- 172.21.15.3:0/1808211586 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f391804f2a0 con 0x7f39181349a0 2023-10-03T10:12:18.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.416+0000 7f3914ff9700 1 -- 172.21.15.3:0/1808211586 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f390c02b030 con 0x7f39181349a0 2023-10-03T10:12:18.416 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-03T10:12:18.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.418+0000 7f391e405700 1 -- 172.21.15.3:0/1808211586 >> v1:172.21.15.3:6816/127025 conn(0x7f390402f510 legacy=0x7f39040319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:18.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.418+0000 7f391e405700 1 -- 172.21.15.3:0/1808211586 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f39181349a0 msgr2=0x7f391811b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:18.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.418+0000 7f391e405700 1 --2- 172.21.15.3:0/1808211586 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f39181349a0 0x7f391811b9e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f390c009fd0 tx=0x7f390c004750 comp rx=0 tx=0).stop 2023-10-03T10:12:18.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.418+0000 7f391e405700 1 -- 172.21.15.3:0/1808211586 shutdown_connections 2023-10-03T10:12:18.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.418+0000 7f391e405700 1 --2- 172.21.15.3:0/1808211586 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f39181372c0 0x7f391811bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:18.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.418+0000 7f391e405700 1 --2- 172.21.15.3:0/1808211586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3918131b30 0x7f391811b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:18.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.418+0000 7f391e405700 1 --2- 172.21.15.3:0/1808211586 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f39181349a0 0x7f391811b9e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:18.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.418+0000 7f391e405700 1 -- 172.21.15.3:0/1808211586 >> 172.21.15.3:0/1808211586 conn(0x7f39180a4130 msgr2=0x7f391809c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:18.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.418+0000 7f391e405700 1 -- 172.21.15.3:0/1808211586 shutdown_connections 2023-10-03T10:12:18.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.418+0000 7f391e405700 1 -- 172.21.15.3:0/1808211586 wait complete. 2023-10-03T10:12:18.432 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-03T10:12:18.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.571+0000 7fbf55eab700 1 Processor -- start 2023-10-03T10:12:18.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.571+0000 7fbf55eab700 1 -- start start 2023-10-03T10:12:18.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.571+0000 7fbf55eab700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf50131b30 0x7fbf50131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:18.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.571+0000 7fbf55eab700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbf5009f390 0x7fbf501399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:18.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.571+0000 7fbf55eab700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf500a01c0 con 0x7fbf5009e850 2023-10-03T10:12:18.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.571+0000 7fbf55eab700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf5009b560 con 0x7fbf5009f390 2023-10-03T10:12:18.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.571+0000 7fbf55eab700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5009f810 con 0x7fbf50131b30 2023-10-03T10:12:18.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.571+0000 7fbf4effd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbf5009e850 0x7fbf5009ec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35714/0 (socket says 172.21.15.3:35714) 2023-10-03T10:12:18.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.572+0000 7fbf4effd700 1 -- 172.21.15.3:0/459990464 learned_addr learned my addr 172.21.15.3:0/459990464 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:18.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.572+0000 7fbf4f7fe700 1 --2- 172.21.15.3:0/459990464 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf50131b30 0x7fbf50131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:18.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.572+0000 7fbf4ffff700 1 --2- 172.21.15.3:0/459990464 >> v2:172.21.15.3:3300/0 conn(0x7fbf5009f390 0x7fbf501399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:18.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.572+0000 7fbf4e7fc700 1 -- 172.21.15.3:0/459990464 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2572154831 0 0) 0x7fbf500a01c0 con 0x7fbf5009e850 2023-10-03T10:12:18.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.572+0000 7fbf4f7fe700 1 -- 172.21.15.3:0/459990464 >> v2:172.21.15.3:3300/0 conn(0x7fbf5009f390 msgr2=0x7fbf501399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:18.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.572+0000 7fbf4f7fe700 1 --2- 172.21.15.3:0/459990464 >> v2:172.21.15.3:3300/0 conn(0x7fbf5009f390 0x7fbf501399d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:18.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.572+0000 7fbf4f7fe700 1 -- 172.21.15.3:0/459990464 >> v1:172.21.15.3:6789/0 conn(0x7fbf5009e850 legacy=0x7fbf5009ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:18.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.572+0000 7fbf4f7fe700 1 -- 172.21.15.3:0/459990464 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5009a260 con 0x7fbf50131b30 2023-10-03T10:12:18.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.572+0000 7fbf4ffff700 1 --2- 172.21.15.3:0/459990464 >> v2:172.21.15.3:3300/0 conn(0x7fbf5009f390 0x7fbf501399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:18.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.572+0000 7fbf4f7fe700 1 --2- 172.21.15.3:0/459990464 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf50131b30 0x7fbf50131f00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009840 tx=0x7fbf38009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fba76a6369df23d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:18.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.572+0000 7fbf4e7fc700 1 -- 172.21.15.3:0/459990464 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf38011830 con 0x7fbf50131b30 2023-10-03T10:12:18.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.572+0000 7fbf4e7fc700 1 -- 172.21.15.3:0/459990464 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf38011990 con 0x7fbf50131b30 2023-10-03T10:12:18.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.572+0000 7fbf4e7fc700 1 -- 172.21.15.3:0/459990464 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf38011d20 con 0x7fbf50131b30 2023-10-03T10:12:18.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.573+0000 7fbf55eab700 1 -- 172.21.15.3:0/459990464 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf50131b30 msgr2=0x7fbf50131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:18.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.573+0000 7fbf55eab700 1 --2- 172.21.15.3:0/459990464 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf50131b30 0x7fbf50131f00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009840 tx=0x7fbf38009b50 comp rx=0 tx=0).stop 2023-10-03T10:12:18.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.573+0000 7fbf55eab700 1 -- 172.21.15.3:0/459990464 shutdown_connections 2023-10-03T10:12:18.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.573+0000 7fbf55eab700 1 --2- 172.21.15.3:0/459990464 >> v2:172.21.15.3:3300/0 conn(0x7fbf5009f390 0x7fbf501399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:18.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.573+0000 7fbf55eab700 1 --2- 172.21.15.3:0/459990464 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf50131b30 0x7fbf50131f00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:18.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.573+0000 7fbf55eab700 1 -- 172.21.15.3:0/459990464 >> 172.21.15.3:0/459990464 conn(0x7fbf500a4130 msgr2=0x7fbf500a4530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:18.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.573+0000 7fbf55eab700 1 -- 172.21.15.3:0/459990464 shutdown_connections 2023-10-03T10:12:18.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.573+0000 7fbf55eab700 1 -- 172.21.15.3:0/459990464 wait complete. 2023-10-03T10:12:18.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.574+0000 7fbf55eab700 1 Processor -- start 2023-10-03T10:12:18.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.574+0000 7fbf55eab700 1 -- start start 2023-10-03T10:12:18.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.575+0000 7fbf55eab700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf5009e850 0x7fbf5006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:18.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.575+0000 7fbf55eab700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf5009f390 0x7fbf5006bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:18.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.575+0000 7fbf55eab700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf50131b30 0x7fbf5006c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:18.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.575+0000 7fbf55eab700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5006cd80 con 0x7fbf5009f390 2023-10-03T10:12:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.575+0000 7fbf55eab700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf50123c80 con 0x7fbf50131b30 2023-10-03T10:12:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.575+0000 7fbf55eab700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf50122c20 con 0x7fbf5009e850 2023-10-03T10:12:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.575+0000 7fbf4f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf5009e850 0x7fbf5006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.575+0000 7fbf4ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf50131b30 0x7fbf5006c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.575+0000 7fbf4effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf5009f390 0x7fbf5006bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.575+0000 7fbf4ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf50131b30 0x7fbf5006c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37552/0 (socket says 172.21.15.3:37552) 2023-10-03T10:12:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.575+0000 7fbf4f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf5009e850 0x7fbf5006b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:59166/0 (socket says 172.21.15.3:59166) 2023-10-03T10:12:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.575+0000 7fbf4ffff700 1 -- 172.21.15.3:0/2579792755 learned_addr learned my addr 172.21.15.3:0/2579792755 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.575+0000 7fbf4ffff700 1 -- 172.21.15.3:0/2579792755 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf5009e850 msgr2=0x7fbf5006b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.575+0000 7fbf4ffff700 1 --2- 172.21.15.3:0/2579792755 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf5009e850 0x7fbf5006b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.575+0000 7fbf4ffff700 1 -- 172.21.15.3:0/2579792755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf5009f390 msgr2=0x7fbf5006bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.575+0000 7fbf4ffff700 1 --2- 172.21.15.3:0/2579792755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf5009f390 0x7fbf5006bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.575+0000 7fbf4ffff700 1 -- 172.21.15.3:0/2579792755 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf48009a70 con 0x7fbf50131b30 2023-10-03T10:12:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.575+0000 7fbf4f7fe700 1 --2- 172.21.15.3:0/2579792755 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf5009e850 0x7fbf5006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.575+0000 7fbf4effd700 1 --2- 172.21.15.3:0/2579792755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf5009f390 0x7fbf5006bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.575+0000 7fbf4ffff700 1 --2- 172.21.15.3:0/2579792755 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf50131b30 0x7fbf5006c330 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbf4800efc0 tx=0x7fbf4800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.576+0000 7fbf47fff700 1 -- 172.21.15.3:0/2579792755 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4800ca90 con 0x7fbf50131b30 2023-10-03T10:12:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.576+0000 7fbf47fff700 1 -- 172.21.15.3:0/2579792755 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4800f040 con 0x7fbf50131b30 2023-10-03T10:12:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.576+0000 7fbf55eab700 1 -- 172.21.15.3:0/2579792755 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf380094a0 con 0x7fbf50131b30 2023-10-03T10:12:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.576+0000 7fbf47fff700 1 -- 172.21.15.3:0/2579792755 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4800ce10 con 0x7fbf50131b30 2023-10-03T10:12:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.576+0000 7fbf55eab700 1 -- 172.21.15.3:0/2579792755 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf50123730 con 0x7fbf50131b30 2023-10-03T10:12:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.576+0000 7fbf47fff700 1 -- 172.21.15.3:0/2579792755 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fbf48014480 con 0x7fbf50131b30 2023-10-03T10:12:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.577+0000 7fbf47fff700 1 -- 172.21.15.3:0/2579792755 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4078+0+0 (secure 0 0 0) 0x7fbf480402f0 con 0x7fbf50131b30 2023-10-03T10:12:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.577+0000 7fbf55eab700 1 -- 172.21.15.3:0/2579792755 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf5004f2a0 con 0x7fbf50131b30 2023-10-03T10:12:18.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.582+0000 7fbf47fff700 1 -- 172.21.15.3:0/2579792755 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbf48018020 con 0x7fbf50131b30 2023-10-03T10:12:18.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.787+0000 7fbf55eab700 1 -- 172.21.15.3:0/2579792755 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbf50131530 con 0x7fbf50131b30 2023-10-03T10:12:18.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.787+0000 7fbf47fff700 1 -- 172.21.15.3:0/2579792755 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbf48012930 con 0x7fbf50131b30 2023-10-03T10:12:18.788 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-10-03T10:12:18.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.790+0000 7fbf55eab700 1 -- 172.21.15.3:0/2579792755 >> v1:172.21.15.3:6816/127025 conn(0x7fbf3002f510 legacy=0x7fbf300319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:18.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.790+0000 7fbf55eab700 1 -- 172.21.15.3:0/2579792755 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf50131b30 msgr2=0x7fbf5006c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:18.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.790+0000 7fbf55eab700 1 --2- 172.21.15.3:0/2579792755 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf50131b30 0x7fbf5006c330 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbf4800efc0 tx=0x7fbf4800c3a0 comp rx=0 tx=0).stop 2023-10-03T10:12:18.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.790+0000 7fbf55eab700 1 -- 172.21.15.3:0/2579792755 shutdown_connections 2023-10-03T10:12:18.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.790+0000 7fbf55eab700 1 --2- 172.21.15.3:0/2579792755 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf50131b30 0x7fbf5006c330 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:18.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.790+0000 7fbf55eab700 1 --2- 172.21.15.3:0/2579792755 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf5009e850 0x7fbf5006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:18.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.790+0000 7fbf55eab700 1 --2- 172.21.15.3:0/2579792755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf5009f390 0x7fbf5006bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:18.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.790+0000 7fbf55eab700 1 -- 172.21.15.3:0/2579792755 >> 172.21.15.3:0/2579792755 conn(0x7fbf500a4130 msgr2=0x7fbf501b76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:18.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.790+0000 7fbf55eab700 1 -- 172.21.15.3:0/2579792755 shutdown_connections 2023-10-03T10:12:18.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.790+0000 7fbf55eab700 1 -- 172.21.15.3:0/2579792755 wait complete. 2023-10-03T10:12:18.804 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-03T10:12:18.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.944+0000 7f0abced3700 1 Processor -- start 2023-10-03T10:12:18.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.944+0000 7f0abced3700 1 -- start start 2023-10-03T10:12:18.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.944+0000 7f0abced3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0ab809d7d0 0x7f0ab809dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:18.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.944+0000 7f0abced3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ab809e0e0 0x7f0ab8130770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:18.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.944+0000 7f0abced3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab809b800 con 0x7f0ab8130cb0 2023-10-03T10:12:18.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.944+0000 7f0abced3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab809b620 con 0x7f0ab809d7d0 2023-10-03T10:12:18.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.944+0000 7f0abced3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab809f370 con 0x7f0ab809e0e0 2023-10-03T10:12:18.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.944+0000 7f0ab5d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ab809e0e0 0x7f0ab8130770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:18.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.945+0000 7f0ab659c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0ab809d7d0 0x7f0ab809dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:18.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.945+0000 7f0ab6d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0ab8130cb0 0x7f0ab8137a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35726/0 (socket says 172.21.15.3:35726) 2023-10-03T10:12:18.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.945+0000 7f0ab5d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ab809e0e0 0x7f0ab8130770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:59168/0 (socket says 172.21.15.3:59168) 2023-10-03T10:12:18.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.945+0000 7f0ab6d9d700 1 -- 172.21.15.3:0/54458835 learned_addr learned my addr 172.21.15.3:0/54458835 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:18.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.945+0000 7f0ab659c700 1 -- 172.21.15.3:0/54458835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ab809e0e0 msgr2=0x7f0ab8130770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:18.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.945+0000 7f0ab659c700 1 --2- 172.21.15.3:0/54458835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ab809e0e0 0x7f0ab8130770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:18.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.945+0000 7f0ab659c700 1 -- 172.21.15.3:0/54458835 >> v1:172.21.15.3:6789/0 conn(0x7f0ab8130cb0 legacy=0x7f0ab8137a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:18.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.945+0000 7f0ab659c700 1 -- 172.21.15.3:0/54458835 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab809ef80 con 0x7f0ab809d7d0 2023-10-03T10:12:18.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.945+0000 7f0ab559a700 1 -- 172.21.15.3:0/54458835 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3696130786 0 0) 0x7f0ab809b800 con 0x7f0ab8130cb0 2023-10-03T10:12:18.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.945+0000 7f0ab5d9b700 1 --2- 172.21.15.3:0/54458835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ab809e0e0 0x7f0ab8130770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:18.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.945+0000 7f0ab659c700 1 --2- 172.21.15.3:0/54458835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ab809d7d0 0x7f0ab809dba0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009840 tx=0x7f0aa8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ef45d61fea19b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:18.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.945+0000 7f0ab559a700 1 -- 172.21.15.3:0/54458835 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8011830 con 0x7f0ab809d7d0 2023-10-03T10:12:18.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.945+0000 7f0ab559a700 1 -- 172.21.15.3:0/54458835 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa8011990 con 0x7f0ab809d7d0 2023-10-03T10:12:18.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.945+0000 7f0ab559a700 1 -- 172.21.15.3:0/54458835 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8011d20 con 0x7f0ab809d7d0 2023-10-03T10:12:18.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.946+0000 7f0abced3700 1 -- 172.21.15.3:0/54458835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ab809d7d0 msgr2=0x7f0ab809dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:18.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.946+0000 7f0abced3700 1 --2- 172.21.15.3:0/54458835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ab809d7d0 0x7f0ab809dba0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009840 tx=0x7f0aa8009b50 comp rx=0 tx=0).stop 2023-10-03T10:12:18.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.946+0000 7f0abced3700 1 -- 172.21.15.3:0/54458835 shutdown_connections 2023-10-03T10:12:18.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.946+0000 7f0abced3700 1 --2- 172.21.15.3:0/54458835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ab809d7d0 0x7f0ab809dba0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:18.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.946+0000 7f0abced3700 1 --2- 172.21.15.3:0/54458835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ab809e0e0 0x7f0ab8130770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:18.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.946+0000 7f0abced3700 1 -- 172.21.15.3:0/54458835 >> 172.21.15.3:0/54458835 conn(0x7f0ab8066d70 msgr2=0x7f0ab80a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:18.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.946+0000 7f0abced3700 1 -- 172.21.15.3:0/54458835 shutdown_connections 2023-10-03T10:12:18.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.946+0000 7f0abced3700 1 -- 172.21.15.3:0/54458835 wait complete. 2023-10-03T10:12:18.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.947+0000 7f0abced3700 1 Processor -- start 2023-10-03T10:12:18.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.947+0000 7f0abced3700 1 -- start start 2023-10-03T10:12:18.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.948+0000 7f0abced3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ab809d7d0 0x7f0ab81c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:18.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.948+0000 7f0abced3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ab809e0e0 0x7f0ab81c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:18.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.948+0000 7f0abced3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0ab8130cb0 0x7f0ab81c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:18.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.948+0000 7f0abced3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab8124420 con 0x7f0ab809d7d0 2023-10-03T10:12:18.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.948+0000 7f0abced3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab80719e0 con 0x7f0ab8130cb0 2023-10-03T10:12:18.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.948+0000 7f0abced3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab809c720 con 0x7f0ab809e0e0 2023-10-03T10:12:18.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.948+0000 7f0ab659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ab809d7d0 0x7f0ab81c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:18.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.948+0000 7f0ab5d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ab809e0e0 0x7f0ab81c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:18.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.948+0000 7f0ab659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ab809d7d0 0x7f0ab81c80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44370/0 (socket says 172.21.15.3:44370) 2023-10-03T10:12:18.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.948+0000 7f0ab659c700 1 -- 172.21.15.3:0/3840805742 learned_addr learned my addr 172.21.15.3:0/3840805742 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:18.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.948+0000 7f0ab6d9d700 1 --2- 172.21.15.3:0/3840805742 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0ab8130cb0 0x7f0ab81c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:18.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.948+0000 7f0ab659c700 1 -- 172.21.15.3:0/3840805742 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ab809e0e0 msgr2=0x7f0ab81c8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:18.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.948+0000 7f0ab659c700 1 --2- 172.21.15.3:0/3840805742 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ab809e0e0 0x7f0ab81c8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:18.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.948+0000 7f0ab659c700 1 -- 172.21.15.3:0/3840805742 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0ab8130cb0 msgr2=0x7f0ab81c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:18.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.948+0000 7f0ab659c700 1 --2- 172.21.15.3:0/3840805742 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0ab8130cb0 0x7f0ab81c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:18.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.948+0000 7f0ab659c700 1 -- 172.21.15.3:0/3840805742 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa00094a0 con 0x7f0ab809d7d0 2023-10-03T10:12:18.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.948+0000 7f0ab659c700 1 --2- 172.21.15.3:0/3840805742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ab809d7d0 0x7f0ab81c80d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009fd0 tx=0x7f0aa8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:18.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.949+0000 7f0aaeffd700 1 -- 172.21.15.3:0/3840805742 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8004550 con 0x7f0ab809d7d0 2023-10-03T10:12:18.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.949+0000 7f0aaeffd700 1 -- 172.21.15.3:0/3840805742 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa801f070 con 0x7f0ab809d7d0 2023-10-03T10:12:18.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.949+0000 7f0abced3700 1 -- 172.21.15.3:0/3840805742 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa80094a0 con 0x7f0ab809d7d0 2023-10-03T10:12:18.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.949+0000 7f0aaeffd700 1 -- 172.21.15.3:0/3840805742 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa80038e0 con 0x7f0ab809d7d0 2023-10-03T10:12:18.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.949+0000 7f0abced3700 1 -- 172.21.15.3:0/3840805742 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab8124c10 con 0x7f0ab809d7d0 2023-10-03T10:12:18.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.949+0000 7f0aaeffd700 1 -- 172.21.15.3:0/3840805742 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f0aa8003a40 con 0x7f0ab809d7d0 2023-10-03T10:12:18.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.950+0000 7f0aaeffd700 1 -- 172.21.15.3:0/3840805742 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4078+0+0 (secure 0 0 0) 0x7f0aa8021070 con 0x7f0ab809d7d0 2023-10-03T10:12:18.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.950+0000 7f0abced3700 1 -- 172.21.15.3:0/3840805742 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab8125f70 con 0x7f0ab809d7d0 2023-10-03T10:12:18.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:18.955+0000 7f0aaeffd700 1 -- 172.21.15.3:0/3840805742 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0aa801a430 con 0x7f0ab809d7d0 2023-10-03T10:12:19.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.156+0000 7f0abced3700 1 -- 172.21.15.3:0/3840805742 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0ab809f180 con 0x7f0ab809d7d0 2023-10-03T10:12:19.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.156+0000 7f0aaeffd700 1 -- 172.21.15.3:0/3840805742 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0aa8015070 con 0x7f0ab809d7d0 2023-10-03T10:12:19.157 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-10-03T10:12:19.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.158+0000 7f0abced3700 1 -- 172.21.15.3:0/3840805742 >> v1:172.21.15.3:6816/127025 conn(0x7f0a9802f510 legacy=0x7f0a980319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:19.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.158+0000 7f0abced3700 1 -- 172.21.15.3:0/3840805742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ab809d7d0 msgr2=0x7f0ab81c80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:19.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.158+0000 7f0abced3700 1 --2- 172.21.15.3:0/3840805742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ab809d7d0 0x7f0ab81c80d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009fd0 tx=0x7f0aa8004750 comp rx=0 tx=0).stop 2023-10-03T10:12:19.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.158+0000 7f0abced3700 1 -- 172.21.15.3:0/3840805742 shutdown_connections 2023-10-03T10:12:19.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.158+0000 7f0abced3700 1 --2- 172.21.15.3:0/3840805742 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0ab8130cb0 0x7f0ab81c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:19.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.158+0000 7f0abced3700 1 --2- 172.21.15.3:0/3840805742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ab809d7d0 0x7f0ab81c80d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:19.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.158+0000 7f0abced3700 1 --2- 172.21.15.3:0/3840805742 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ab809e0e0 0x7f0ab81c8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:19.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.159+0000 7f0abced3700 1 -- 172.21.15.3:0/3840805742 >> 172.21.15.3:0/3840805742 conn(0x7f0ab8066d70 msgr2=0x7f0ab80a0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:19.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.159+0000 7f0abced3700 1 -- 172.21.15.3:0/3840805742 shutdown_connections 2023-10-03T10:12:19.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.159+0000 7f0abced3700 1 -- 172.21.15.3:0/3840805742 wait complete. 2023-10-03T10:12:19.172 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-03T10:12:19.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.309+0000 7f50f9043700 1 Processor -- start 2023-10-03T10:12:19.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.309+0000 7f50f9043700 1 -- start start 2023-10-03T10:12:19.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.310+0000 7f50f9043700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50f406a770 0x7f50f406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:19.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.310+0000 7f50f9043700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f50f4131d10 0x7f50f4139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:19.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.310+0000 7f50f9043700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50f409ece0 con 0x7f50f406b080 2023-10-03T10:12:19.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.310+0000 7f50f9043700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f409e760 con 0x7f50f4131d10 2023-10-03T10:12:19.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.310+0000 7f50f9043700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f409e040 con 0x7f50f406a770 2023-10-03T10:12:19.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.310+0000 7f50f359e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f50f4131d10 0x7f50f4139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:19.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.310+0000 7f50f259c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f50f406b080 0x7f50f4131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35732/0 (socket says 172.21.15.3:35732) 2023-10-03T10:12:19.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.310+0000 7f50f259c700 1 -- 172.21.15.3:0/823067864 learned_addr learned my addr 172.21.15.3:0/823067864 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:19.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.310+0000 7f50f2d9d700 1 --2- 172.21.15.3:0/823067864 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50f406a770 0x7f50f406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:19.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.310+0000 7f50f2d9d700 1 -- 172.21.15.3:0/823067864 >> v2:172.21.15.3:3300/0 conn(0x7f50f4131d10 msgr2=0x7f50f4139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:19.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.310+0000 7f50f2d9d700 1 --2- 172.21.15.3:0/823067864 >> v2:172.21.15.3:3300/0 conn(0x7f50f4131d10 0x7f50f4139860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:19.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.310+0000 7f50f2d9d700 1 -- 172.21.15.3:0/823067864 >> v1:172.21.15.3:6789/0 conn(0x7f50f406b080 legacy=0x7f50f4131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:19.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.310+0000 7f50f2d9d700 1 -- 172.21.15.3:0/823067864 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f409f630 con 0x7f50f406a770 2023-10-03T10:12:19.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.310+0000 7f50f1d9b700 1 -- 172.21.15.3:0/823067864 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2796151957 0 0) 0x7f50f409ece0 con 0x7f50f406b080 2023-10-03T10:12:19.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.310+0000 7f50f359e700 1 --2- 172.21.15.3:0/823067864 >> v2:172.21.15.3:3300/0 conn(0x7f50f4131d10 0x7f50f4139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:19.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.310+0000 7f50f2d9d700 1 --2- 172.21.15.3:0/823067864 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50f406a770 0x7f50f406ab40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009840 tx=0x7f50e4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fb8db6ac3b896b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:19.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.311+0000 7f50f1d9b700 1 -- 172.21.15.3:0/823067864 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e4011830 con 0x7f50f406a770 2023-10-03T10:12:19.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.311+0000 7f50f1d9b700 1 -- 172.21.15.3:0/823067864 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e4011990 con 0x7f50f406a770 2023-10-03T10:12:19.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.311+0000 7f50f1d9b700 1 -- 172.21.15.3:0/823067864 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e4011d20 con 0x7f50f406a770 2023-10-03T10:12:19.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.311+0000 7f50f9043700 1 -- 172.21.15.3:0/823067864 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50f406a770 msgr2=0x7f50f406ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:19.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.311+0000 7f50f9043700 1 --2- 172.21.15.3:0/823067864 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50f406a770 0x7f50f406ab40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009840 tx=0x7f50e4009b50 comp rx=0 tx=0).stop 2023-10-03T10:12:19.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.311+0000 7f50f9043700 1 -- 172.21.15.3:0/823067864 shutdown_connections 2023-10-03T10:12:19.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.311+0000 7f50f9043700 1 --2- 172.21.15.3:0/823067864 >> v2:172.21.15.3:3300/0 conn(0x7f50f4131d10 0x7f50f4139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:19.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.311+0000 7f50f9043700 1 --2- 172.21.15.3:0/823067864 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50f406a770 0x7f50f406ab40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:19.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.311+0000 7f50f9043700 1 -- 172.21.15.3:0/823067864 >> 172.21.15.3:0/823067864 conn(0x7f50f4066f50 msgr2=0x7f50f409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:19.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.311+0000 7f50f9043700 1 -- 172.21.15.3:0/823067864 shutdown_connections 2023-10-03T10:12:19.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.311+0000 7f50f9043700 1 -- 172.21.15.3:0/823067864 wait complete. 2023-10-03T10:12:19.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.313+0000 7f50f9043700 1 Processor -- start 2023-10-03T10:12:19.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.313+0000 7f50f9043700 1 -- start start 2023-10-03T10:12:19.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.313+0000 7f50f9043700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50f406a770 0x7f50f409abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:19.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.313+0000 7f50f9043700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f50f406b080 0x7f50f409b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:19.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.313+0000 7f50f9043700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50f4131d10 0x7f50f41c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:19.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.313+0000 7f50f9043700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f41311d0 con 0x7f50f406a770 2023-10-03T10:12:19.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.313+0000 7f50f9043700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f4071e60 con 0x7f50f4131d10 2023-10-03T10:12:19.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.313+0000 7f50f9043700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f50f4124ed0 con 0x7f50f406b080 2023-10-03T10:12:19.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.313+0000 7f50f2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50f406a770 0x7f50f409abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:19.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.313+0000 7f50f359e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50f4131d10 0x7f50f41c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:19.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.313+0000 7f50f2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50f406a770 0x7f50f409abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44390/0 (socket says 172.21.15.3:44390) 2023-10-03T10:12:19.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.313+0000 7f50f2d9d700 1 -- 172.21.15.3:0/3791704273 learned_addr learned my addr 172.21.15.3:0/3791704273 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:19.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.313+0000 7f50f259c700 1 --2- 172.21.15.3:0/3791704273 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f50f406b080 0x7f50f409b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:19.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.314+0000 7f50f359e700 1 -- 172.21.15.3:0/3791704273 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f50f406b080 msgr2=0x7f50f409b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:19.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.314+0000 7f50f359e700 1 --2- 172.21.15.3:0/3791704273 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f50f406b080 0x7f50f409b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:19.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.314+0000 7f50f359e700 1 -- 172.21.15.3:0/3791704273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50f406a770 msgr2=0x7f50f409abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:19.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.314+0000 7f50f359e700 1 --2- 172.21.15.3:0/3791704273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50f406a770 0x7f50f409abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:19.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.314+0000 7f50f359e700 1 -- 172.21.15.3:0/3791704273 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ec009a70 con 0x7f50f4131d10 2023-10-03T10:12:19.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.314+0000 7f50f259c700 1 --2- 172.21.15.3:0/3791704273 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f50f406b080 0x7f50f409b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:12:19.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.314+0000 7f50f359e700 1 --2- 172.21.15.3:0/3791704273 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50f4131d10 0x7f50f41c8040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009e50 tx=0x7f50ec00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:19.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.314+0000 7f50eb7fe700 1 -- 172.21.15.3:0/3791704273 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec00cb40 con 0x7f50f4131d10 2023-10-03T10:12:19.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.314+0000 7f50eb7fe700 1 -- 172.21.15.3:0/3791704273 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50ec00f040 con 0x7f50f4131d10 2023-10-03T10:12:19.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.314+0000 7f50f9043700 1 -- 172.21.15.3:0/3791704273 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e40094a0 con 0x7f50f4131d10 2023-10-03T10:12:19.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.314+0000 7f50eb7fe700 1 -- 172.21.15.3:0/3791704273 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec003a20 con 0x7f50f4131d10 2023-10-03T10:12:19.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.314+0000 7f50f9043700 1 -- 172.21.15.3:0/3791704273 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f4125c30 con 0x7f50f4131d10 2023-10-03T10:12:19.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.315+0000 7f50eb7fe700 1 -- 172.21.15.3:0/3791704273 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f50ec014480 con 0x7f50f4131d10 2023-10-03T10:12:19.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.315+0000 7f50eb7fe700 1 -- 172.21.15.3:0/3791704273 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4078+0+0 (secure 0 0 0) 0x7f50ec040360 con 0x7f50f4131d10 2023-10-03T10:12:19.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.315+0000 7f50f9043700 1 -- 172.21.15.3:0/3791704273 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f4131580 con 0x7f50f4131d10 2023-10-03T10:12:19.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.320+0000 7f50eb7fe700 1 -- 172.21.15.3:0/3791704273 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f50ec018080 con 0x7f50f4131d10 2023-10-03T10:12:19.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.570+0000 7f50f9043700 1 -- 172.21.15.3:0/3791704273 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50f4074ea0 con 0x7f50f4131d10 2023-10-03T10:12:19.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.571+0000 7f50eb7fe700 1 -- 172.21.15.3:0/3791704273 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+2313 (secure 0 0 0) 0x7f50ec01bd80 con 0x7f50f4131d10 2023-10-03T10:12:19.571 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:12:19.571 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-03T10:09:44.141304+0000","modified":"2023-10-03T10:10:11.643845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4186},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4186":{"gid":4186,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6817/2582564779","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6817","nonce":2582564779}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-03T10:12:19.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.573+0000 7f50f9043700 1 -- 172.21.15.3:0/3791704273 >> v1:172.21.15.3:6816/127025 conn(0x7f50d402f510 legacy=0x7f50d40319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:19.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.573+0000 7f50f9043700 1 -- 172.21.15.3:0/3791704273 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50f4131d10 msgr2=0x7f50f41c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:19.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.573+0000 7f50f9043700 1 --2- 172.21.15.3:0/3791704273 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50f4131d10 0x7f50f41c8040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009e50 tx=0x7f50ec00c3a0 comp rx=0 tx=0).stop 2023-10-03T10:12:19.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.573+0000 7f50f9043700 1 -- 172.21.15.3:0/3791704273 shutdown_connections 2023-10-03T10:12:19.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.573+0000 7f50f9043700 1 --2- 172.21.15.3:0/3791704273 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50f4131d10 0x7f50f41c8040 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:19.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.573+0000 7f50f9043700 1 --2- 172.21.15.3:0/3791704273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50f406a770 0x7f50f409abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:19.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.573+0000 7f50f9043700 1 --2- 172.21.15.3:0/3791704273 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f50f406b080 0x7f50f409b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:19.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.573+0000 7f50f9043700 1 -- 172.21.15.3:0/3791704273 >> 172.21.15.3:0/3791704273 conn(0x7f50f4066f50 msgr2=0x7f50f4070000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:19.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.573+0000 7f50f9043700 1 -- 172.21.15.3:0/3791704273 shutdown_connections 2023-10-03T10:12:19.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.573+0000 7f50f9043700 1 -- 172.21.15.3:0/3791704273 wait complete. 2023-10-03T10:12:19.574 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2023-10-03T10:12:19.587 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-03T10:12:19.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.723+0000 7fa9dd95b700 1 Processor -- start 2023-10-03T10:12:19.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.723+0000 7fa9dd95b700 1 -- start start 2023-10-03T10:12:19.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.724+0000 7fa9dd95b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9d80a09c0 0x7fa9d80a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:19.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.724+0000 7fa9dd95b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa9d809d5c0 0x7fa9d80a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:19.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.724+0000 7fa9dd95b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9d8133920 0x7fa9d8133dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:19.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.724+0000 7fa9dd95b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9d80a1810 con 0x7fa9d809d5c0 2023-10-03T10:12:19.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.724+0000 7fa9dd95b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d809ea10 con 0x7fa9d8133920 2023-10-03T10:12:19.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.724+0000 7fa9dd95b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d809e210 con 0x7fa9d80a09c0 2023-10-03T10:12:19.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.724+0000 7fa9d67fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa9d809d5c0 0x7fa9d80a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:19.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.724+0000 7fa9d77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9d8133920 0x7fa9d8133dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:19.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.724+0000 7fa9d67fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa9d809d5c0 0x7fa9d80a12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44398/0 (socket says 172.21.15.3:44398) 2023-10-03T10:12:19.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.724+0000 7fa9d6ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9d80a09c0 0x7fa9d80a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:19.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.724+0000 7fa9d67fc700 1 -- 172.21.15.3:0/4050625050 learned_addr learned my addr 172.21.15.3:0/4050625050 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:19.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.724+0000 7fa9d77fe700 1 -- 172.21.15.3:0/4050625050 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9d80a09c0 msgr2=0x7fa9d80a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:19.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.724+0000 7fa9d77fe700 1 --2- 172.21.15.3:0/4050625050 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9d80a09c0 0x7fa9d80a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:19.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.724+0000 7fa9d77fe700 1 -- 172.21.15.3:0/4050625050 >> v2:172.21.15.3:3300/0 conn(0x7fa9d809d5c0 msgr2=0x7fa9d80a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:19.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.724+0000 7fa9d77fe700 1 --2- 172.21.15.3:0/4050625050 >> v2:172.21.15.3:3300/0 conn(0x7fa9d809d5c0 0x7fa9d80a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:19.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.724+0000 7fa9d77fe700 1 -- 172.21.15.3:0/4050625050 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d80a26f0 con 0x7fa9d8133920 2023-10-03T10:12:19.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.724+0000 7fa9d67fc700 1 --2- 172.21.15.3:0/4050625050 >> v2:172.21.15.3:3300/0 conn(0x7fa9d809d5c0 0x7fa9d80a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:19.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.724+0000 7fa9d6ffd700 1 --2- 172.21.15.3:0/4050625050 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9d80a09c0 0x7fa9d80a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:19.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.724+0000 7fa9d77fe700 1 --2- 172.21.15.3:0/4050625050 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9d8133920 0x7fa9d8133dc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0009d90 tx=0x7fa9d000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c617f964878fe03c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:19.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.725+0000 7fa9d5ffb700 1 -- 172.21.15.3:0/4050625050 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d000e070 con 0x7fa9d8133920 2023-10-03T10:12:19.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.725+0000 7fa9d5ffb700 1 -- 172.21.15.3:0/4050625050 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d000bb20 con 0x7fa9d8133920 2023-10-03T10:12:19.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.725+0000 7fa9d5ffb700 1 -- 172.21.15.3:0/4050625050 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0012430 con 0x7fa9d8133920 2023-10-03T10:12:19.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.725+0000 7fa9dd95b700 1 -- 172.21.15.3:0/4050625050 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9d8133920 msgr2=0x7fa9d8133dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:19.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.725+0000 7fa9dd95b700 1 --2- 172.21.15.3:0/4050625050 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9d8133920 0x7fa9d8133dc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0009d90 tx=0x7fa9d000b330 comp rx=0 tx=0).stop 2023-10-03T10:12:19.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.725+0000 7fa9dd95b700 1 -- 172.21.15.3:0/4050625050 shutdown_connections 2023-10-03T10:12:19.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.725+0000 7fa9dd95b700 1 --2- 172.21.15.3:0/4050625050 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9d8133920 0x7fa9d8133dc0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:19.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.725+0000 7fa9dd95b700 1 --2- 172.21.15.3:0/4050625050 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9d80a09c0 0x7fa9d80a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:19.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.725+0000 7fa9dd95b700 1 --2- 172.21.15.3:0/4050625050 >> v2:172.21.15.3:3300/0 conn(0x7fa9d809d5c0 0x7fa9d80a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:19.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.725+0000 7fa9dd95b700 1 -- 172.21.15.3:0/4050625050 >> 172.21.15.3:0/4050625050 conn(0x7fa9d80a2fe0 msgr2=0x7fa9d80a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:19.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.725+0000 7fa9dd95b700 1 -- 172.21.15.3:0/4050625050 shutdown_connections 2023-10-03T10:12:19.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.725+0000 7fa9dd95b700 1 -- 172.21.15.3:0/4050625050 wait complete. 2023-10-03T10:12:19.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.726+0000 7fa9dd95b700 1 Processor -- start 2023-10-03T10:12:19.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.726+0000 7fa9dd95b700 1 -- start start 2023-10-03T10:12:19.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.727+0000 7fa9dd95b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9d809d5c0 0x7fa9d81c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:19.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.727+0000 7fa9dd95b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9d80a09c0 0x7fa9d81c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:19.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.727+0000 7fa9dd95b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9d8133920 0x7fa9d81c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:19.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.727+0000 7fa9dd95b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d8120680 con 0x7fa9d809d5c0 2023-10-03T10:12:19.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.727+0000 7fa9dd95b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d8121ee0 con 0x7fa9d80a09c0 2023-10-03T10:12:19.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.727+0000 7fa9dd95b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d809cec0 con 0x7fa9d8133920 2023-10-03T10:12:19.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.727+0000 7fa9d67fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9d80a09c0 0x7fa9d81c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:19.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.727+0000 7fa9d6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9d809d5c0 0x7fa9d81c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:19.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.727+0000 7fa9d77fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9d8133920 0x7fa9d81c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:19.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.727+0000 7fa9d67fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9d80a09c0 0x7fa9d81c85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37584/0 (socket says 172.21.15.3:37584) 2023-10-03T10:12:19.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.727+0000 7fa9d77fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9d8133920 0x7fa9d81c8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:59196/0 (socket says 172.21.15.3:59196) 2023-10-03T10:12:19.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.727+0000 7fa9d67fc700 1 -- 172.21.15.3:0/3570068618 learned_addr learned my addr 172.21.15.3:0/3570068618 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:19.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.727+0000 7fa9d77fe700 1 -- 172.21.15.3:0/3570068618 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9d80a09c0 msgr2=0x7fa9d81c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:19.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.727+0000 7fa9d77fe700 1 --2- 172.21.15.3:0/3570068618 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9d80a09c0 0x7fa9d81c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:19.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.727+0000 7fa9d77fe700 1 -- 172.21.15.3:0/3570068618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9d809d5c0 msgr2=0x7fa9d81c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:19.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.727+0000 7fa9d77fe700 1 --2- 172.21.15.3:0/3570068618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9d809d5c0 0x7fa9d81c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:19.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.727+0000 7fa9d77fe700 1 -- 172.21.15.3:0/3570068618 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c0009550 con 0x7fa9d8133920 2023-10-03T10:12:19.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.727+0000 7fa9d67fc700 1 --2- 172.21.15.3:0/3570068618 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9d80a09c0 0x7fa9d81c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:19.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.728+0000 7fa9d77fe700 1 --2- 172.21.15.3:0/3570068618 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9d8133920 0x7fa9d81c8b00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0016000 tx=0x7fa9d0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:19.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.728+0000 7fa9cffff700 1 -- 172.21.15.3:0/3570068618 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0012430 con 0x7fa9d8133920 2023-10-03T10:12:19.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.728+0000 7fa9cffff700 1 -- 172.21.15.3:0/3570068618 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d0020070 con 0x7fa9d8133920 2023-10-03T10:12:19.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.728+0000 7fa9dd95b700 1 -- 172.21.15.3:0/3570068618 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9c80094a0 con 0x7fa9d8133920 2023-10-03T10:12:19.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.728+0000 7fa9cffff700 1 -- 172.21.15.3:0/3570068618 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0004280 con 0x7fa9d8133920 2023-10-03T10:12:19.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.728+0000 7fa9dd95b700 1 -- 172.21.15.3:0/3570068618 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d0009a70 con 0x7fa9d8133920 2023-10-03T10:12:19.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.729+0000 7fa9cffff700 1 -- 172.21.15.3:0/3570068618 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fa9d00043e0 con 0x7fa9d8133920 2023-10-03T10:12:19.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.729+0000 7fa9dd95b700 1 -- 172.21.15.3:0/3570068618 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d80747f0 con 0x7fa9d8133920 2023-10-03T10:12:19.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.729+0000 7fa9cffff700 1 -- 172.21.15.3:0/3570068618 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4078+0+0 (secure 0 0 0) 0x7fa9d0022070 con 0x7fa9d8133920 2023-10-03T10:12:19.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.734+0000 7fa9cffff700 1 -- 172.21.15.3:0/3570068618 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa9d0003e30 con 0x7fa9d8133920 2023-10-03T10:12:19.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.939+0000 7fa9dd95b700 1 -- 172.21.15.3:0/3570068618 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9d8072980 con 0x7fa9d8133920 2023-10-03T10:12:19.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.940+0000 7fa9cffff700 1 -- 172.21.15.3:0/3570068618 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+12923 (secure 0 0 0) 0x7fa9d00263e0 con 0x7fa9d8133920 2023-10-03T10:12:19.941 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:12:19.941 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","created":"2023-10-03T10:09:28.703441+0000","modified":"2023-10-03T10:12:15.315657+0000","last_up_change":"2023-10-03T10:09:36.569401+0000","last_in_change":"2023-10-03T10:09:30.156546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-03T10:09:36.623971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-03T10:09:38.210614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-03T10:09:42.695873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-03T10:09:43.115588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cdcdd930-3c5b-4486-a021-7132f90fb6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6808","nonce":118845}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6809","nonce":118845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6811","nonce":118845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6810","nonce":118845}]},"public_addr":"172.21.15.3:6808/118845","cluster_addr":"172.21.15.3:6809/118845","heartbeat_back_addr":"172.21.15.3:6811/118845","heartbeat_front_addr":"172.21.15.3:6810/118845","state":["exists","up"]},{"osd":1,"uuid":"c8e58877-6762-4741-a8c1-567ebc97b708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6804","nonce":118835}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6805","nonce":118835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6807","nonce":118835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6806","nonce":118835}]},"public_addr":"172.21.15.3:6804/118835","cluster_addr":"172.21.15.3:6805/118835","heartbeat_back_addr":"172.21.15.3:6807/118835","heartbeat_front_addr":"172.21.15.3:6806/118835","state":["exists","up"]},{"osd":2,"uuid":"68a2a21a-7a8e-4f82-8433-5201faa50777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6812","nonce":118864}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6813","nonce":118864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6815","nonce":118864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6814","nonce":118864}]},"public_addr":"172.21.15.3:6812/118864","cluster_addr":"172.21.15.3:6813/118864","heartbeat_back_addr":"172.21.15.3:6815/118864","heartbeat_front_addr":"172.21.15.3:6814/118864","state":["exists","up"]},{"osd":3,"uuid":"c4df2903-4e1e-4208-bf77-95b9fa749241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6800","nonce":118836}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6801","nonce":118836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6803","nonce":118836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6802","nonce":118836}]},"public_addr":"172.21.15.3:6800/118836","cluster_addr":"172.21.15.3:6801/118836","heartbeat_back_addr":"172.21.15.3:6803/118836","heartbeat_front_addr":"172.21.15.3:6802/118836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.369708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.257825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.337931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.085222+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1","mappings":[{"from":3,"to":0}]},{"pgid":"4.8","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.19","mappings":[{"from":3,"to":0}]},{"pgid":"4.2a","mappings":[{"from":1,"to":2}]},{"pgid":"4.2d","mappings":[{"from":3,"to":0}]},{"pgid":"4.33","mappings":[{"from":3,"to":0}]},{"pgid":"4.39","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/3919724564":"2023-10-04T10:12:13.305411+0000","172.21.15.3:0/2091814084":"2023-10-04T10:12:13.305411+0000","172.21.15.3:6816/118296":"2023-10-04T10:12:13.305411+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-03T10:12:19.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.943+0000 7fa9dd95b700 1 -- 172.21.15.3:0/3570068618 >> v1:172.21.15.3:6816/127025 conn(0x7fa9b802f510 legacy=0x7fa9b80319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:19.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.943+0000 7fa9dd95b700 1 -- 172.21.15.3:0/3570068618 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9d8133920 msgr2=0x7fa9d81c8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:19.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.943+0000 7fa9dd95b700 1 --2- 172.21.15.3:0/3570068618 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9d8133920 0x7fa9d81c8b00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0016000 tx=0x7fa9d0004ad0 comp rx=0 tx=0).stop 2023-10-03T10:12:19.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.943+0000 7fa9dd95b700 1 -- 172.21.15.3:0/3570068618 shutdown_connections 2023-10-03T10:12:19.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.943+0000 7fa9dd95b700 1 --2- 172.21.15.3:0/3570068618 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9d8133920 0x7fa9d81c8b00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:19.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.943+0000 7fa9dd95b700 1 --2- 172.21.15.3:0/3570068618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9d809d5c0 0x7fa9d81c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:19.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.943+0000 7fa9dd95b700 1 --2- 172.21.15.3:0/3570068618 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9d80a09c0 0x7fa9d81c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:19.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.943+0000 7fa9dd95b700 1 -- 172.21.15.3:0/3570068618 >> 172.21.15.3:0/3570068618 conn(0x7fa9d80a2fe0 msgr2=0x7fa9d81bbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:19.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.943+0000 7fa9dd95b700 1 -- 172.21.15.3:0/3570068618 shutdown_connections 2023-10-03T10:12:19.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:19.943+0000 7fa9dd95b700 1 -- 172.21.15.3:0/3570068618 wait complete. 2023-10-03T10:12:19.957 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-03T10:09:36.623971+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}, 'mgr_devicehealth': {}}, 'read_balance': {'score_acting': 4, 'score_stable': 4, 'optimal_score': 0.5, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-03T10:09:38.210614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-03T10:09:42.695873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-03T10:09:43.115588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.059999942779541, 'score_stable': 1.059999942779541, 'optimal_score': 1, 'raw_score_acting': 1.059999942779541, 'raw_score_stable': 1.059999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-03T10:12:19.957 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-03T10:12:20.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.096+0000 7fc408037700 1 Processor -- start 2023-10-03T10:12:20.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.096+0000 7fc408037700 1 -- start start 2023-10-03T10:12:20.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.097+0000 7fc408037700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc4000a37f0 0x7fc4001338b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:20.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.097+0000 7fc408037700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc400133df0 0x7fc40006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:20.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.097+0000 7fc408037700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc40006acb0 0x7fc40006b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:20.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.097+0000 7fc408037700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc40009a7b0 con 0x7fc40006acb0 2023-10-03T10:12:20.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.097+0000 7fc408037700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc40009dec0 con 0x7fc400133df0 2023-10-03T10:12:20.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.097+0000 7fc408037700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc40009e900 con 0x7fc4000a37f0 2023-10-03T10:12:20.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.097+0000 7fc405dd3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc4000a37f0 0x7fc4001338b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:20.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.097+0000 7fc4055d2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc400133df0 0x7fc40006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:20.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.097+0000 7fc4065d4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc40006acb0 0x7fc40006b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:20.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.097+0000 7fc405dd3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc4000a37f0 0x7fc4001338b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:59208/0 (socket says 172.21.15.3:59208) 2023-10-03T10:12:20.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.097+0000 7fc4055d2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc400133df0 0x7fc40006a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37598/0 (socket says 172.21.15.3:37598) 2023-10-03T10:12:20.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.097+0000 7fc405dd3700 1 -- 172.21.15.3:0/2305034695 learned_addr learned my addr 172.21.15.3:0/2305034695 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:20.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.097+0000 7fc405dd3700 1 -- 172.21.15.3:0/2305034695 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc400133df0 msgr2=0x7fc40006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:20.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.097+0000 7fc405dd3700 1 --2- 172.21.15.3:0/2305034695 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc400133df0 0x7fc40006a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:20.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.097+0000 7fc405dd3700 1 -- 172.21.15.3:0/2305034695 >> v2:172.21.15.3:3300/0 conn(0x7fc40006acb0 msgr2=0x7fc40006b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:20.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.097+0000 7fc405dd3700 1 --2- 172.21.15.3:0/2305034695 >> v2:172.21.15.3:3300/0 conn(0x7fc40006acb0 0x7fc40006b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:20.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.097+0000 7fc405dd3700 1 -- 172.21.15.3:0/2305034695 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc40009f600 con 0x7fc4000a37f0 2023-10-03T10:12:20.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.097+0000 7fc4065d4700 1 --2- 172.21.15.3:0/2305034695 >> v2:172.21.15.3:3300/0 conn(0x7fc40006acb0 0x7fc40006b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:20.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.097+0000 7fc405dd3700 1 --2- 172.21.15.3:0/2305034695 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc4000a37f0 0x7fc4001338b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc0098f0 tx=0x7fc3fc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49a2734769f43b4c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:20.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.098+0000 7fc404dd1700 1 -- 172.21.15.3:0/2305034695 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc011830 con 0x7fc4000a37f0 2023-10-03T10:12:20.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.098+0000 7fc404dd1700 1 -- 172.21.15.3:0/2305034695 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3fc011990 con 0x7fc4000a37f0 2023-10-03T10:12:20.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.098+0000 7fc404dd1700 1 -- 172.21.15.3:0/2305034695 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc011d20 con 0x7fc4000a37f0 2023-10-03T10:12:20.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.098+0000 7fc408037700 1 -- 172.21.15.3:0/2305034695 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc4000a37f0 msgr2=0x7fc4001338b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:20.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.098+0000 7fc408037700 1 --2- 172.21.15.3:0/2305034695 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc4000a37f0 0x7fc4001338b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc0098f0 tx=0x7fc3fc009c00 comp rx=0 tx=0).stop 2023-10-03T10:12:20.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.098+0000 7fc408037700 1 -- 172.21.15.3:0/2305034695 shutdown_connections 2023-10-03T10:12:20.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.098+0000 7fc408037700 1 --2- 172.21.15.3:0/2305034695 >> v2:172.21.15.3:3300/0 conn(0x7fc40006acb0 0x7fc40006b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:20.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.098+0000 7fc408037700 1 --2- 172.21.15.3:0/2305034695 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc4000a37f0 0x7fc4001338b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:20.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.098+0000 7fc408037700 1 --2- 172.21.15.3:0/2305034695 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc400133df0 0x7fc40006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:20.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.098+0000 7fc408037700 1 -- 172.21.15.3:0/2305034695 >> 172.21.15.3:0/2305034695 conn(0x7fc400066d70 msgr2=0x7fc4000a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:20.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.098+0000 7fc408037700 1 -- 172.21.15.3:0/2305034695 shutdown_connections 2023-10-03T10:12:20.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.098+0000 7fc408037700 1 -- 172.21.15.3:0/2305034695 wait complete. 2023-10-03T10:12:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.099+0000 7fc408037700 1 Processor -- start 2023-10-03T10:12:20.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.099+0000 7fc408037700 1 -- start start 2023-10-03T10:12:20.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.100+0000 7fc408037700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc40006acb0 0x7fc40011d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:20.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.100+0000 7fc408037700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4000a37f0 0x7fc40011de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:20.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.100+0000 7fc408037700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc400133df0 0x7fc40011e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:20.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.100+0000 7fc408037700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40012d810 con 0x7fc4000a37f0 2023-10-03T10:12:20.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.100+0000 7fc408037700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc400071f60 con 0x7fc40006acb0 2023-10-03T10:12:20.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.100+0000 7fc408037700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc40011ead0 con 0x7fc400133df0 2023-10-03T10:12:20.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.100+0000 7fc405dd3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc40006acb0 0x7fc40011d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:20.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.100+0000 7fc405dd3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc40006acb0 0x7fc40011d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37600/0 (socket says 172.21.15.3:37600) 2023-10-03T10:12:20.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.100+0000 7fc405dd3700 1 -- 172.21.15.3:0/2259421906 learned_addr learned my addr 172.21.15.3:0/2259421906 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:20.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.100+0000 7fc4055d2700 1 --2- 172.21.15.3:0/2259421906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4000a37f0 0x7fc40011de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.100+0000 7fc4065d4700 1 --2- 172.21.15.3:0/2259421906 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc400133df0 0x7fc40011e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.100+0000 7fc405dd3700 1 -- 172.21.15.3:0/2259421906 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc400133df0 msgr2=0x7fc40011e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.100+0000 7fc405dd3700 1 --2- 172.21.15.3:0/2259421906 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc400133df0 0x7fc40011e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.100+0000 7fc405dd3700 1 -- 172.21.15.3:0/2259421906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4000a37f0 msgr2=0x7fc40011de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.100+0000 7fc405dd3700 1 --2- 172.21.15.3:0/2259421906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4000a37f0 0x7fc40011de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.100+0000 7fc405dd3700 1 -- 172.21.15.3:0/2259421906 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f8009a70 con 0x7fc40006acb0 2023-10-03T10:12:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.101+0000 7fc405dd3700 1 --2- 172.21.15.3:0/2259421906 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc40006acb0 0x7fc40011d930 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc009f50 tx=0x7fc3fc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.101+0000 7fc3f27fc700 1 -- 172.21.15.3:0/2259421906 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc004450 con 0x7fc40006acb0 2023-10-03T10:12:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.101+0000 7fc3f27fc700 1 -- 172.21.15.3:0/2259421906 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3fc0045b0 con 0x7fc40006acb0 2023-10-03T10:12:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.101+0000 7fc408037700 1 -- 172.21.15.3:0/2259421906 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3fc009550 con 0x7fc40006acb0 2023-10-03T10:12:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.101+0000 7fc3f27fc700 1 -- 172.21.15.3:0/2259421906 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc0038a0 con 0x7fc40006acb0 2023-10-03T10:12:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.101+0000 7fc408037700 1 -- 172.21.15.3:0/2259421906 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc40009b420 con 0x7fc40006acb0 2023-10-03T10:12:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.102+0000 7fc3f27fc700 1 -- 172.21.15.3:0/2259421906 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fc3fc021070 con 0x7fc40006acb0 2023-10-03T10:12:20.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.102+0000 7fc3f27fc700 1 -- 172.21.15.3:0/2259421906 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4078+0+0 (secure 0 0 0) 0x7fc3fc040d10 con 0x7fc40006acb0 2023-10-03T10:12:20.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.102+0000 7fc408037700 1 -- 172.21.15.3:0/2259421906 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc40012dc50 con 0x7fc40006acb0 2023-10-03T10:12:20.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.107+0000 7fc3f27fc700 1 -- 172.21.15.3:0/2259421906 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc3fc003a00 con 0x7fc40006acb0 2023-10-03T10:12:20.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.311+0000 7fc408037700 1 -- 172.21.15.3:0/2259421906 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc40004f2a0 con 0x7fc40006acb0 2023-10-03T10:12:20.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.311+0000 7fc3f27fc700 1 -- 172.21.15.3:0/2259421906 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc3fc032070 con 0x7fc40006acb0 2023-10-03T10:12:20.312 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-03T10:12:20.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.313+0000 7fc408037700 1 -- 172.21.15.3:0/2259421906 >> v1:172.21.15.3:6816/127025 conn(0x7fc3e402f540 legacy=0x7fc3e40319f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:20.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.313+0000 7fc408037700 1 -- 172.21.15.3:0/2259421906 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc40006acb0 msgr2=0x7fc40011d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:20.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.313+0000 7fc408037700 1 --2- 172.21.15.3:0/2259421906 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc40006acb0 0x7fc40011d930 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc009f50 tx=0x7fc3fc004750 comp rx=0 tx=0).stop 2023-10-03T10:12:20.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.313+0000 7fc408037700 1 -- 172.21.15.3:0/2259421906 shutdown_connections 2023-10-03T10:12:20.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.313+0000 7fc408037700 1 --2- 172.21.15.3:0/2259421906 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc400133df0 0x7fc40011e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:20.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.313+0000 7fc408037700 1 --2- 172.21.15.3:0/2259421906 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc40006acb0 0x7fc40011d930 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:20.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.313+0000 7fc408037700 1 --2- 172.21.15.3:0/2259421906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4000a37f0 0x7fc40011de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:20.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.313+0000 7fc408037700 1 -- 172.21.15.3:0/2259421906 >> 172.21.15.3:0/2259421906 conn(0x7fc400066d70 msgr2=0x7fc400132290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:20.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.313+0000 7fc408037700 1 -- 172.21.15.3:0/2259421906 shutdown_connections 2023-10-03T10:12:20.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.313+0000 7fc408037700 1 -- 172.21.15.3:0/2259421906 wait complete. 2023-10-03T10:12:20.327 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-03T10:12:20.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.465+0000 7f91417d8700 1 Processor -- start 2023-10-03T10:12:20.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.465+0000 7f91417d8700 1 -- start start 2023-10-03T10:12:20.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.466+0000 7f91417d8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f913c09d7d0 0x7f913c09dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:20.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.466+0000 7f91417d8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f913c09e0e0 0x7f913c130770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:20.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.466+0000 7f91417d8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f913c130cb0 0x7f913c131150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:20.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.466+0000 7f91417d8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f913c09b620 con 0x7f913c09e0e0 2023-10-03T10:12:20.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.466+0000 7f91417d8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913c09f370 con 0x7f913c130cb0 2023-10-03T10:12:20.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.466+0000 7f91417d8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f913c09b800 con 0x7f913c09d7d0 2023-10-03T10:12:20.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.466+0000 7f913affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f913c09d7d0 0x7f913c09dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:20.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.466+0000 7f913a7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f913c09e0e0 0x7f913c130770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:20.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.466+0000 7f913b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f913c130cb0 0x7f913c131150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:20.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.466+0000 7f913affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f913c09d7d0 0x7f913c09dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:59230/0 (socket says 172.21.15.3:59230) 2023-10-03T10:12:20.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.466+0000 7f913affd700 1 -- 172.21.15.3:0/1145031830 learned_addr learned my addr 172.21.15.3:0/1145031830 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:20.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.466+0000 7f913affd700 1 -- 172.21.15.3:0/1145031830 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f913c130cb0 msgr2=0x7f913c131150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:20.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.466+0000 7f913affd700 1 --2- 172.21.15.3:0/1145031830 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f913c130cb0 0x7f913c131150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:20.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.466+0000 7f913affd700 1 -- 172.21.15.3:0/1145031830 >> v2:172.21.15.3:3300/0 conn(0x7f913c09e0e0 msgr2=0x7f913c130770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:20.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.466+0000 7f913affd700 1 --2- 172.21.15.3:0/1145031830 >> v2:172.21.15.3:3300/0 conn(0x7f913c09e0e0 0x7f913c130770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:20.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.467+0000 7f913affd700 1 -- 172.21.15.3:0/1145031830 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c09ef80 con 0x7f913c09d7d0 2023-10-03T10:12:20.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.467+0000 7f913b7fe700 1 --2- 172.21.15.3:0/1145031830 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f913c130cb0 0x7f913c131150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:20.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.467+0000 7f913a7fc700 1 --2- 172.21.15.3:0/1145031830 >> v2:172.21.15.3:3300/0 conn(0x7f913c09e0e0 0x7f913c130770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:20.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.467+0000 7f913affd700 1 --2- 172.21.15.3:0/1145031830 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f913c09d7d0 0x7f913c09dba0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f912c009920 tx=0x7f912c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca15d60816d95617 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:20.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.467+0000 7f9139ffb700 1 -- 172.21.15.3:0/1145031830 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c011830 con 0x7f913c09d7d0 2023-10-03T10:12:20.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.467+0000 7f9139ffb700 1 -- 172.21.15.3:0/1145031830 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912c011990 con 0x7f913c09d7d0 2023-10-03T10:12:20.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.467+0000 7f9139ffb700 1 -- 172.21.15.3:0/1145031830 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c011d20 con 0x7f913c09d7d0 2023-10-03T10:12:20.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.467+0000 7f91417d8700 1 -- 172.21.15.3:0/1145031830 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f913c09d7d0 msgr2=0x7f913c09dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:20.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.467+0000 7f91417d8700 1 --2- 172.21.15.3:0/1145031830 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f913c09d7d0 0x7f913c09dba0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f912c009920 tx=0x7f912c009c30 comp rx=0 tx=0).stop 2023-10-03T10:12:20.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.468+0000 7f91417d8700 1 -- 172.21.15.3:0/1145031830 shutdown_connections 2023-10-03T10:12:20.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.468+0000 7f91417d8700 1 --2- 172.21.15.3:0/1145031830 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f913c130cb0 0x7f913c131150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:20.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.468+0000 7f91417d8700 1 --2- 172.21.15.3:0/1145031830 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f913c09d7d0 0x7f913c09dba0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:20.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.468+0000 7f91417d8700 1 --2- 172.21.15.3:0/1145031830 >> v2:172.21.15.3:3300/0 conn(0x7f913c09e0e0 0x7f913c130770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:20.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.468+0000 7f91417d8700 1 -- 172.21.15.3:0/1145031830 >> 172.21.15.3:0/1145031830 conn(0x7f913c066d70 msgr2=0x7f913c0a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:20.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.468+0000 7f91417d8700 1 -- 172.21.15.3:0/1145031830 shutdown_connections 2023-10-03T10:12:20.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.468+0000 7f91417d8700 1 -- 172.21.15.3:0/1145031830 wait complete. 2023-10-03T10:12:20.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.469+0000 7f91417d8700 1 Processor -- start 2023-10-03T10:12:20.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.469+0000 7f91417d8700 1 -- start start 2023-10-03T10:12:20.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.469+0000 7f91417d8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f913c09d7d0 0x7f913c120210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:20.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.469+0000 7f91417d8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f913c09e0e0 0x7f913c120750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:20.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.469+0000 7f91417d8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f913c130cb0 0x7f913c120c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:20.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.469+0000 7f91417d8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c1216e0 con 0x7f913c09e0e0 2023-10-03T10:12:20.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.469+0000 7f91417d8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913c122470 con 0x7f913c09d7d0 2023-10-03T10:12:20.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.469+0000 7f91417d8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f913c0a0460 con 0x7f913c130cb0 2023-10-03T10:12:20.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.469+0000 7f913affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f913c09d7d0 0x7f913c120210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:20.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.469+0000 7f913b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f913c130cb0 0x7f913c120c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:20.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.469+0000 7f913a7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f913c09e0e0 0x7f913c120750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:20.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.469+0000 7f913affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f913c09d7d0 0x7f913c120210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37618/0 (socket says 172.21.15.3:37618) 2023-10-03T10:12:20.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.469+0000 7f913affd700 1 -- 172.21.15.3:0/1114672801 learned_addr learned my addr 172.21.15.3:0/1114672801 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:20.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.470+0000 7f913b7fe700 1 -- 172.21.15.3:0/1114672801 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f913c09d7d0 msgr2=0x7f913c120210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:20.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.470+0000 7f913b7fe700 1 --2- 172.21.15.3:0/1114672801 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f913c09d7d0 0x7f913c120210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:20.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.470+0000 7f913b7fe700 1 -- 172.21.15.3:0/1114672801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f913c09e0e0 msgr2=0x7f913c120750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:20.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.470+0000 7f913b7fe700 1 --2- 172.21.15.3:0/1114672801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f913c09e0e0 0x7f913c120750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:20.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.470+0000 7f913b7fe700 1 -- 172.21.15.3:0/1114672801 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91240094a0 con 0x7f913c130cb0 2023-10-03T10:12:20.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.470+0000 7f913affd700 1 --2- 172.21.15.3:0/1114672801 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f913c09d7d0 0x7f913c120210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:20.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.470+0000 7f913a7fc700 1 --2- 172.21.15.3:0/1114672801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f913c09e0e0 0x7f913c120750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:20.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.470+0000 7f913b7fe700 1 --2- 172.21.15.3:0/1114672801 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f913c130cb0 0x7f913c120c90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9134009e50 tx=0x7f913400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:20.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.470+0000 7f91337fe700 1 -- 172.21.15.3:0/1114672801 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913400ca90 con 0x7f913c130cb0 2023-10-03T10:12:20.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.470+0000 7f91337fe700 1 -- 172.21.15.3:0/1114672801 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913400f040 con 0x7f913c130cb0 2023-10-03T10:12:20.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.470+0000 7f91337fe700 1 -- 172.21.15.3:0/1114672801 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913400ce50 con 0x7f913c130cb0 2023-10-03T10:12:20.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.470+0000 7f91417d8700 1 -- 172.21.15.3:0/1114672801 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9134009a70 con 0x7f913c130cb0 2023-10-03T10:12:20.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.470+0000 7f91417d8700 1 -- 172.21.15.3:0/1114672801 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f912c009550 con 0x7f913c130cb0 2023-10-03T10:12:20.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.471+0000 7f91337fe700 1 -- 172.21.15.3:0/1114672801 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f9134014480 con 0x7f913c130cb0 2023-10-03T10:12:20.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.471+0000 7f91417d8700 1 -- 172.21.15.3:0/1114672801 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f913c123e30 con 0x7f913c130cb0 2023-10-03T10:12:20.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.471+0000 7f91337fe700 1 -- 172.21.15.3:0/1114672801 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4078+0+0 (secure 0 0 0) 0x7f9134040200 con 0x7f913c130cb0 2023-10-03T10:12:20.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.476+0000 7f91337fe700 1 -- 172.21.15.3:0/1114672801 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9134018070 con 0x7f913c130cb0 2023-10-03T10:12:20.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.681+0000 7f91417d8700 1 -- 172.21.15.3:0/1114672801 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f913c04f2a0 con 0x7f913c130cb0 2023-10-03T10:12:20.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.681+0000 7f91337fe700 1 -- 172.21.15.3:0/1114672801 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9134012930 con 0x7f913c130cb0 2023-10-03T10:12:20.681 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-03T10:12:20.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.683+0000 7f91417d8700 1 -- 172.21.15.3:0/1114672801 >> v1:172.21.15.3:6816/127025 conn(0x7f911c02f510 legacy=0x7f911c0319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:20.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.683+0000 7f91417d8700 1 -- 172.21.15.3:0/1114672801 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f913c130cb0 msgr2=0x7f913c120c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:20.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.683+0000 7f91417d8700 1 --2- 172.21.15.3:0/1114672801 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f913c130cb0 0x7f913c120c90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9134009e50 tx=0x7f913400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:12:20.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.683+0000 7f91417d8700 1 -- 172.21.15.3:0/1114672801 shutdown_connections 2023-10-03T10:12:20.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.683+0000 7f91417d8700 1 --2- 172.21.15.3:0/1114672801 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f913c130cb0 0x7f913c120c90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:20.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.683+0000 7f91417d8700 1 --2- 172.21.15.3:0/1114672801 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f913c09d7d0 0x7f913c120210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:20.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.683+0000 7f91417d8700 1 --2- 172.21.15.3:0/1114672801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f913c09e0e0 0x7f913c120750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:20.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.683+0000 7f91417d8700 1 -- 172.21.15.3:0/1114672801 >> 172.21.15.3:0/1114672801 conn(0x7f913c066d70 msgr2=0x7f913c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:20.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.684+0000 7f91417d8700 1 -- 172.21.15.3:0/1114672801 shutdown_connections 2023-10-03T10:12:20.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.684+0000 7f91417d8700 1 -- 172.21.15.3:0/1114672801 wait complete. 2023-10-03T10:12:20.697 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-03T10:12:20.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.838+0000 7fabc8190700 1 Processor -- start 2023-10-03T10:12:20.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.838+0000 7fabc8190700 1 -- start start 2023-10-03T10:12:20.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.838+0000 7fabc8190700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fabc00a37f0 0x7fabc01338b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:20.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.838+0000 7fabc8190700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fabc0133df0 0x7fabc006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:20.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.838+0000 7fabc8190700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fabc006acb0 0x7fabc006b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:20.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.838+0000 7fabc8190700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc009a7b0 con 0x7fabc0133df0 2023-10-03T10:12:20.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.839+0000 7fabc8190700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc009dec0 con 0x7fabc006acb0 2023-10-03T10:12:20.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.839+0000 7fabc8190700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabc009e900 con 0x7fabc00a37f0 2023-10-03T10:12:20.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.839+0000 7fabc572b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fabc0133df0 0x7fabc006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:20.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.839+0000 7fabc672d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fabc006acb0 0x7fabc006b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:20.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.839+0000 7fabc5f2c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fabc00a37f0 0x7fabc01338b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:20.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.839+0000 7fabc672d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fabc006acb0 0x7fabc006b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37620/0 (socket says 172.21.15.3:37620) 2023-10-03T10:12:20.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.839+0000 7fabc572b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fabc0133df0 0x7fabc006a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44456/0 (socket says 172.21.15.3:44456) 2023-10-03T10:12:20.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.839+0000 7fabc672d700 1 -- 172.21.15.3:0/2742586509 learned_addr learned my addr 172.21.15.3:0/2742586509 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:20.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.839+0000 7fabc5f2c700 1 -- 172.21.15.3:0/2742586509 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fabc006acb0 msgr2=0x7fabc006b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:20.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.839+0000 7fabc5f2c700 1 --2- 172.21.15.3:0/2742586509 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fabc006acb0 0x7fabc006b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:20.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.839+0000 7fabc5f2c700 1 -- 172.21.15.3:0/2742586509 >> v2:172.21.15.3:3300/0 conn(0x7fabc0133df0 msgr2=0x7fabc006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:20.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.839+0000 7fabc5f2c700 1 --2- 172.21.15.3:0/2742586509 >> v2:172.21.15.3:3300/0 conn(0x7fabc0133df0 0x7fabc006a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:20.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.839+0000 7fabc5f2c700 1 -- 172.21.15.3:0/2742586509 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc009f600 con 0x7fabc00a37f0 2023-10-03T10:12:20.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.839+0000 7fabc572b700 1 --2- 172.21.15.3:0/2742586509 >> v2:172.21.15.3:3300/0 conn(0x7fabc0133df0 0x7fabc006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:20.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.839+0000 7fabc5f2c700 1 --2- 172.21.15.3:0/2742586509 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fabc00a37f0 0x7fabc01338b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fabbc009840 tx=0x7fabbc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5cd834dd79bf849 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:20.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.839+0000 7fabc4f2a700 1 -- 172.21.15.3:0/2742586509 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc011830 con 0x7fabc00a37f0 2023-10-03T10:12:20.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.839+0000 7fabc4f2a700 1 -- 172.21.15.3:0/2742586509 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabbc011990 con 0x7fabc00a37f0 2023-10-03T10:12:20.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.839+0000 7fabc4f2a700 1 -- 172.21.15.3:0/2742586509 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc011d20 con 0x7fabc00a37f0 2023-10-03T10:12:20.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.840+0000 7fabc8190700 1 -- 172.21.15.3:0/2742586509 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fabc00a37f0 msgr2=0x7fabc01338b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:20.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.840+0000 7fabc8190700 1 --2- 172.21.15.3:0/2742586509 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fabc00a37f0 0x7fabc01338b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fabbc009840 tx=0x7fabbc009b50 comp rx=0 tx=0).stop 2023-10-03T10:12:20.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.840+0000 7fabc8190700 1 -- 172.21.15.3:0/2742586509 shutdown_connections 2023-10-03T10:12:20.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.840+0000 7fabc8190700 1 --2- 172.21.15.3:0/2742586509 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fabc006acb0 0x7fabc006b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:20.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.840+0000 7fabc8190700 1 --2- 172.21.15.3:0/2742586509 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fabc00a37f0 0x7fabc01338b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:20.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.840+0000 7fabc8190700 1 --2- 172.21.15.3:0/2742586509 >> v2:172.21.15.3:3300/0 conn(0x7fabc0133df0 0x7fabc006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:20.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.840+0000 7fabc8190700 1 -- 172.21.15.3:0/2742586509 >> 172.21.15.3:0/2742586509 conn(0x7fabc0066d70 msgr2=0x7fabc00a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:20.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.840+0000 7fabc8190700 1 -- 172.21.15.3:0/2742586509 shutdown_connections 2023-10-03T10:12:20.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.840+0000 7fabc8190700 1 -- 172.21.15.3:0/2742586509 wait complete. 2023-10-03T10:12:20.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.841+0000 7fabc8190700 1 Processor -- start 2023-10-03T10:12:20.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.842+0000 7fabc8190700 1 -- start start 2023-10-03T10:12:20.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.842+0000 7fabc8190700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabc006acb0 0x7fabc011d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:20.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.842+0000 7fabc8190700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fabc00a37f0 0x7fabc011de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:20.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.842+0000 7fabc8190700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fabc0133df0 0x7fabc011e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:20.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.842+0000 7fabc8190700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc012dbf0 con 0x7fabc006acb0 2023-10-03T10:12:20.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.842+0000 7fabc8190700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc00a0240 con 0x7fabc00a37f0 2023-10-03T10:12:20.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.842+0000 7fabc8190700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabc0120170 con 0x7fabc0133df0 2023-10-03T10:12:20.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.842+0000 7fabc5f2c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabc006acb0 0x7fabc011d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:20.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.842+0000 7fabc572b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fabc00a37f0 0x7fabc011de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:20.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.842+0000 7fabc5f2c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabc006acb0 0x7fabc011d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44466/0 (socket says 172.21.15.3:44466) 2023-10-03T10:12:20.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.842+0000 7fabc5f2c700 1 -- 172.21.15.3:0/3775867054 learned_addr learned my addr 172.21.15.3:0/3775867054 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:20.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.842+0000 7fabc672d700 1 --2- 172.21.15.3:0/3775867054 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fabc0133df0 0x7fabc011e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:20.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.843+0000 7fabc572b700 1 -- 172.21.15.3:0/3775867054 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fabc0133df0 msgr2=0x7fabc011e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:20.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.843+0000 7fabc572b700 1 --2- 172.21.15.3:0/3775867054 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fabc0133df0 0x7fabc011e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:20.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.843+0000 7fabc572b700 1 -- 172.21.15.3:0/3775867054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabc006acb0 msgr2=0x7fabc011d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:20.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.843+0000 7fabc572b700 1 --2- 172.21.15.3:0/3775867054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabc006acb0 0x7fabc011d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:20.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.843+0000 7fabc572b700 1 -- 172.21.15.3:0/3775867054 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb40094a0 con 0x7fabc00a37f0 2023-10-03T10:12:20.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.843+0000 7fabc5f2c700 1 --2- 172.21.15.3:0/3775867054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabc006acb0 0x7fabc011d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:20.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.843+0000 7fabc672d700 1 --2- 172.21.15.3:0/3775867054 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fabc0133df0 0x7fabc011e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:20.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.843+0000 7fabc572b700 1 --2- 172.21.15.3:0/3775867054 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fabc00a37f0 0x7fabc011de70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fabb400e9b0 tx=0x7fabb400ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:20.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.843+0000 7fabb27fc700 1 -- 172.21.15.3:0/3775867054 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb400ee50 con 0x7fabc00a37f0 2023-10-03T10:12:20.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.843+0000 7fabb27fc700 1 -- 172.21.15.3:0/3775867054 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb400c850 con 0x7fabc00a37f0 2023-10-03T10:12:20.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.843+0000 7fabb27fc700 1 -- 172.21.15.3:0/3775867054 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb400cbe0 con 0x7fabc00a37f0 2023-10-03T10:12:20.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.843+0000 7fabc8190700 1 -- 172.21.15.3:0/3775867054 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabbc0094a0 con 0x7fabc00a37f0 2023-10-03T10:12:20.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.843+0000 7fabc8190700 1 -- 172.21.15.3:0/3775867054 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc011fd80 con 0x7fabc00a37f0 2023-10-03T10:12:20.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.844+0000 7fabb27fc700 1 -- 172.21.15.3:0/3775867054 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fabb400cd40 con 0x7fabc00a37f0 2023-10-03T10:12:20.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.844+0000 7fabb27fc700 1 -- 172.21.15.3:0/3775867054 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4078+0+0 (secure 0 0 0) 0x7fabb403fe90 con 0x7fabc00a37f0 2023-10-03T10:12:20.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.844+0000 7fabc8190700 1 -- 172.21.15.3:0/3775867054 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc004f2a0 con 0x7fabc00a37f0 2023-10-03T10:12:20.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:20.849+0000 7fabb27fc700 1 -- 172.21.15.3:0/3775867054 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fabb406b750 con 0x7fabc00a37f0 2023-10-03T10:12:21.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.053+0000 7fabc8190700 1 -- 172.21.15.3:0/3775867054 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fabc011fbc0 con 0x7fabc00a37f0 2023-10-03T10:12:21.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.053+0000 7fabb27fc700 1 -- 172.21.15.3:0/3775867054 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fabb402a0a0 con 0x7fabc00a37f0 2023-10-03T10:12:21.054 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-10-03T10:12:21.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.055+0000 7fabc8190700 1 -- 172.21.15.3:0/3775867054 >> v1:172.21.15.3:6816/127025 conn(0x7faba402f510 legacy=0x7faba40319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:21.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.055+0000 7fabc8190700 1 -- 172.21.15.3:0/3775867054 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fabc00a37f0 msgr2=0x7fabc011de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:21.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.055+0000 7fabc8190700 1 --2- 172.21.15.3:0/3775867054 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fabc00a37f0 0x7fabc011de70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fabb400e9b0 tx=0x7fabb400ecc0 comp rx=0 tx=0).stop 2023-10-03T10:12:21.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.055+0000 7fabc8190700 1 -- 172.21.15.3:0/3775867054 shutdown_connections 2023-10-03T10:12:21.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.055+0000 7fabc8190700 1 --2- 172.21.15.3:0/3775867054 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fabc0133df0 0x7fabc011e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:21.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.055+0000 7fabc8190700 1 --2- 172.21.15.3:0/3775867054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabc006acb0 0x7fabc011d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:21.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.055+0000 7fabc8190700 1 --2- 172.21.15.3:0/3775867054 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fabc00a37f0 0x7fabc011de70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:21.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.055+0000 7fabc8190700 1 -- 172.21.15.3:0/3775867054 >> 172.21.15.3:0/3775867054 conn(0x7fabc0066d70 msgr2=0x7fabc00a1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:21.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.056+0000 7fabc8190700 1 -- 172.21.15.3:0/3775867054 shutdown_connections 2023-10-03T10:12:21.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.056+0000 7fabc8190700 1 -- 172.21.15.3:0/3775867054 wait complete. 2023-10-03T10:12:21.070 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-03T10:12:21.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.212+0000 7f5f78fca700 1 Processor -- start 2023-10-03T10:12:21.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.212+0000 7f5f78fca700 1 -- start start 2023-10-03T10:12:21.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.213+0000 7f5f78fca700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f7409e2b0 0x7f5f74135a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:21.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.213+0000 7f5f78fca700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f74135f60 0x7f5f74138340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:21.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.213+0000 7f5f78fca700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7409b800 con 0x7f5f7409d7d0 2023-10-03T10:12:21.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.213+0000 7f5f78fca700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7409b620 con 0x7f5f7409e2b0 2023-10-03T10:12:21.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.213+0000 7f5f78fca700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7409f370 con 0x7f5f74135f60 2023-10-03T10:12:21.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.213+0000 7f5f7259c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5f7409d7d0 0x7f5f7409dba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35744/0 (socket says 172.21.15.3:35744) 2023-10-03T10:12:21.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.213+0000 7f5f71d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f7409e2b0 0x7f5f74135a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:21.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.213+0000 7f5f72d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f74135f60 0x7f5f74138340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:21.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.213+0000 7f5f7259c700 1 -- 172.21.15.3:0/1917675231 learned_addr learned my addr 172.21.15.3:0/1917675231 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:21.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.213+0000 7f5f7159a700 1 -- 172.21.15.3:0/1917675231 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3730873286 0 0) 0x7f5f7409b800 con 0x7f5f7409d7d0 2023-10-03T10:12:21.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.213+0000 7f5f72d9d700 1 -- 172.21.15.3:0/1917675231 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f7409e2b0 msgr2=0x7f5f74135a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:21.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.213+0000 7f5f72d9d700 1 --2- 172.21.15.3:0/1917675231 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f7409e2b0 0x7f5f74135a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:21.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.213+0000 7f5f72d9d700 1 -- 172.21.15.3:0/1917675231 >> v1:172.21.15.3:6789/0 conn(0x7f5f7409d7d0 legacy=0x7f5f7409dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:21.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.213+0000 7f5f72d9d700 1 -- 172.21.15.3:0/1917675231 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7409ef80 con 0x7f5f74135f60 2023-10-03T10:12:21.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.213+0000 7f5f71d9b700 1 --2- 172.21.15.3:0/1917675231 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f7409e2b0 0x7f5f74135a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:21.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.214+0000 7f5f72d9d700 1 --2- 172.21.15.3:0/1917675231 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f74135f60 0x7f5f74138340 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c009dc0 tx=0x7f5f6c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed49ae89cba0d76d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:21.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.214+0000 7f5f7159a700 1 -- 172.21.15.3:0/1917675231 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c00e070 con 0x7f5f74135f60 2023-10-03T10:12:21.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.214+0000 7f5f7159a700 1 -- 172.21.15.3:0/1917675231 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6c00bac0 con 0x7f5f74135f60 2023-10-03T10:12:21.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.214+0000 7f5f7159a700 1 -- 172.21.15.3:0/1917675231 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c00be70 con 0x7f5f74135f60 2023-10-03T10:12:21.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.214+0000 7f5f78fca700 1 -- 172.21.15.3:0/1917675231 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f74135f60 msgr2=0x7f5f74138340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:21.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.214+0000 7f5f78fca700 1 --2- 172.21.15.3:0/1917675231 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f74135f60 0x7f5f74138340 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c009dc0 tx=0x7f5f6c00b330 comp rx=0 tx=0).stop 2023-10-03T10:12:21.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.214+0000 7f5f78fca700 1 -- 172.21.15.3:0/1917675231 shutdown_connections 2023-10-03T10:12:21.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.214+0000 7f5f78fca700 1 --2- 172.21.15.3:0/1917675231 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f74135f60 0x7f5f74138340 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:21.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.214+0000 7f5f78fca700 1 --2- 172.21.15.3:0/1917675231 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f7409e2b0 0x7f5f74135a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:21.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.214+0000 7f5f78fca700 1 -- 172.21.15.3:0/1917675231 >> 172.21.15.3:0/1917675231 conn(0x7f5f74066d70 msgr2=0x7f5f740a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:21.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.214+0000 7f5f78fca700 1 -- 172.21.15.3:0/1917675231 shutdown_connections 2023-10-03T10:12:21.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.215+0000 7f5f78fca700 1 -- 172.21.15.3:0/1917675231 wait complete. 2023-10-03T10:12:21.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.216+0000 7f5f78fca700 1 Processor -- start 2023-10-03T10:12:21.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.216+0000 7f5f78fca700 1 -- start start 2023-10-03T10:12:21.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.216+0000 7f5f78fca700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f7409d7d0 0x7f5f741205c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:21.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.216+0000 7f5f78fca700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f7409e2b0 0x7f5f74120b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:21.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.216+0000 7f5f78fca700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f74135f60 0x7f5f74121040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:21.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.216+0000 7f5f78fca700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f74121a90 con 0x7f5f7409d7d0 2023-10-03T10:12:21.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.216+0000 7f5f78fca700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f741240e0 con 0x7f5f74135f60 2023-10-03T10:12:21.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.216+0000 7f5f78fca700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f741221d0 con 0x7f5f7409e2b0 2023-10-03T10:12:21.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.216+0000 7f5f7259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f7409d7d0 0x7f5f741205c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:21.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.216+0000 7f5f71d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f7409e2b0 0x7f5f74120b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:21.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.216+0000 7f5f7259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f7409d7d0 0x7f5f741205c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44478/0 (socket says 172.21.15.3:44478) 2023-10-03T10:12:21.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.216+0000 7f5f7259c700 1 -- 172.21.15.3:0/2639550654 learned_addr learned my addr 172.21.15.3:0/2639550654 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:21.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.216+0000 7f5f72d9d700 1 --2- 172.21.15.3:0/2639550654 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f74135f60 0x7f5f74121040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:21.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.217+0000 7f5f71d9b700 1 -- 172.21.15.3:0/2639550654 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f74135f60 msgr2=0x7f5f74121040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:21.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.217+0000 7f5f71d9b700 1 --2- 172.21.15.3:0/2639550654 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f74135f60 0x7f5f74121040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:21.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.217+0000 7f5f71d9b700 1 -- 172.21.15.3:0/2639550654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f7409d7d0 msgr2=0x7f5f741205c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:21.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.217+0000 7f5f71d9b700 1 --2- 172.21.15.3:0/2639550654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f7409d7d0 0x7f5f741205c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:21.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.217+0000 7f5f71d9b700 1 -- 172.21.15.3:0/2639550654 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f680094a0 con 0x7f5f7409e2b0 2023-10-03T10:12:21.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.217+0000 7f5f71d9b700 1 --2- 172.21.15.3:0/2639550654 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f7409e2b0 0x7f5f74120b00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800e9e0 tx=0x7f5f6800ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:21.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.217+0000 7f5f62ffd700 1 -- 172.21.15.3:0/2639550654 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6800ee80 con 0x7f5f7409e2b0 2023-10-03T10:12:21.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.217+0000 7f5f62ffd700 1 -- 172.21.15.3:0/2639550654 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6800c850 con 0x7f5f7409e2b0 2023-10-03T10:12:21.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.217+0000 7f5f78fca700 1 -- 172.21.15.3:0/2639550654 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f6c009a70 con 0x7f5f7409e2b0 2023-10-03T10:12:21.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.217+0000 7f5f62ffd700 1 -- 172.21.15.3:0/2639550654 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6800cc00 con 0x7f5f7409e2b0 2023-10-03T10:12:21.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.217+0000 7f5f78fca700 1 -- 172.21.15.3:0/2639550654 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f74123b40 con 0x7f5f7409e2b0 2023-10-03T10:12:21.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.218+0000 7f5f62ffd700 1 -- 172.21.15.3:0/2639550654 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f5f6800cd60 con 0x7f5f7409e2b0 2023-10-03T10:12:21.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.218+0000 7f5f78fca700 1 -- 172.21.15.3:0/2639550654 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f7404f2a0 con 0x7f5f7409e2b0 2023-10-03T10:12:21.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.218+0000 7f5f62ffd700 1 -- 172.21.15.3:0/2639550654 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4078+0+0 (secure 0 0 0) 0x7f5f68040030 con 0x7f5f7409e2b0 2023-10-03T10:12:21.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.223+0000 7f5f62ffd700 1 -- 172.21.15.3:0/2639550654 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5f68017020 con 0x7f5f7409e2b0 2023-10-03T10:12:21.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.429+0000 7f5f78fca700 1 -- 172.21.15.3:0/2639550654 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5f741229a0 con 0x7f5f7409e2b0 2023-10-03T10:12:21.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.429+0000 7f5f62ffd700 1 -- 172.21.15.3:0/2639550654 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5f68011d80 con 0x7f5f7409e2b0 2023-10-03T10:12:21.430 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-10-03T10:12:21.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.431+0000 7f5f78fca700 1 -- 172.21.15.3:0/2639550654 >> v1:172.21.15.3:6816/127025 conn(0x7f5f5402f510 legacy=0x7f5f540319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:21.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.431+0000 7f5f78fca700 1 -- 172.21.15.3:0/2639550654 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f7409e2b0 msgr2=0x7f5f74120b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:21.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.431+0000 7f5f78fca700 1 --2- 172.21.15.3:0/2639550654 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f7409e2b0 0x7f5f74120b00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800e9e0 tx=0x7f5f6800ecf0 comp rx=0 tx=0).stop 2023-10-03T10:12:21.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.431+0000 7f5f78fca700 1 -- 172.21.15.3:0/2639550654 shutdown_connections 2023-10-03T10:12:21.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.431+0000 7f5f78fca700 1 --2- 172.21.15.3:0/2639550654 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f74135f60 0x7f5f74121040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:21.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.431+0000 7f5f78fca700 1 --2- 172.21.15.3:0/2639550654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f7409d7d0 0x7f5f741205c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:21.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.431+0000 7f5f78fca700 1 --2- 172.21.15.3:0/2639550654 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f7409e2b0 0x7f5f74120b00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:21.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.431+0000 7f5f78fca700 1 -- 172.21.15.3:0/2639550654 >> 172.21.15.3:0/2639550654 conn(0x7f5f74066d70 msgr2=0x7f5f740a2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:21.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.432+0000 7f5f78fca700 1 -- 172.21.15.3:0/2639550654 shutdown_connections 2023-10-03T10:12:21.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.432+0000 7f5f78fca700 1 -- 172.21.15.3:0/2639550654 wait complete. 2023-10-03T10:12:21.446 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-03T10:12:21.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.581+0000 7fcd3741b700 1 Processor -- start 2023-10-03T10:12:21.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.581+0000 7fcd3741b700 1 -- start start 2023-10-03T10:12:21.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.581+0000 7fcd3741b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd3009d6f0 0x7fcd3009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:21.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.581+0000 7fcd3741b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd3009e000 0x7fcd30130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:21.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.581+0000 7fcd3741b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd3009f130 con 0x7fcd3009e410 2023-10-03T10:12:21.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.581+0000 7fcd3741b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3009f520 con 0x7fcd3009e000 2023-10-03T10:12:21.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.581+0000 7fcd3741b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3009e930 con 0x7fcd3009d6f0 2023-10-03T10:12:21.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.581+0000 7fcd349b6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd3009e000 0x7fcd30130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:21.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.581+0000 7fcd351b7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd3009d6f0 0x7fcd3009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:21.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.581+0000 7fcd359b8700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcd3009e410 0x7fcd30130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35756/0 (socket says 172.21.15.3:35756) 2023-10-03T10:12:21.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.582+0000 7fcd359b8700 1 -- 172.21.15.3:0/1594788831 learned_addr learned my addr 172.21.15.3:0/1594788831 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:21.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.582+0000 7fcd27fff700 1 -- 172.21.15.3:0/1594788831 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4133477354 0 0) 0x7fcd3009f130 con 0x7fcd3009e410 2023-10-03T10:12:21.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.582+0000 7fcd349b6700 1 -- 172.21.15.3:0/1594788831 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd3009d6f0 msgr2=0x7fcd3009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:21.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.582+0000 7fcd349b6700 1 --2- 172.21.15.3:0/1594788831 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd3009d6f0 0x7fcd3009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:21.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.582+0000 7fcd349b6700 1 -- 172.21.15.3:0/1594788831 >> v1:172.21.15.3:6789/0 conn(0x7fcd3009e410 legacy=0x7fcd30130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:21.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.582+0000 7fcd349b6700 1 -- 172.21.15.3:0/1594788831 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3009b2d0 con 0x7fcd3009e000 2023-10-03T10:12:21.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.582+0000 7fcd351b7700 1 --2- 172.21.15.3:0/1594788831 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd3009d6f0 0x7fcd3009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:21.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.582+0000 7fcd349b6700 1 --2- 172.21.15.3:0/1594788831 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd3009e000 0x7fcd30130660 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcd20009870 tx=0x7fcd20009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4f337b023d0e5db server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:21.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.582+0000 7fcd27fff700 1 -- 172.21.15.3:0/1594788831 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd20011830 con 0x7fcd3009e000 2023-10-03T10:12:21.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.582+0000 7fcd27fff700 1 -- 172.21.15.3:0/1594788831 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd20011990 con 0x7fcd3009e000 2023-10-03T10:12:21.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.582+0000 7fcd27fff700 1 -- 172.21.15.3:0/1594788831 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd20011d20 con 0x7fcd3009e000 2023-10-03T10:12:21.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.583+0000 7fcd3741b700 1 -- 172.21.15.3:0/1594788831 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd3009e000 msgr2=0x7fcd30130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:21.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.583+0000 7fcd3741b700 1 --2- 172.21.15.3:0/1594788831 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd3009e000 0x7fcd30130660 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcd20009870 tx=0x7fcd20009b80 comp rx=0 tx=0).stop 2023-10-03T10:12:21.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.583+0000 7fcd3741b700 1 -- 172.21.15.3:0/1594788831 shutdown_connections 2023-10-03T10:12:21.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.583+0000 7fcd3741b700 1 --2- 172.21.15.3:0/1594788831 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd3009d6f0 0x7fcd3009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:21.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.583+0000 7fcd3741b700 1 --2- 172.21.15.3:0/1594788831 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd3009e000 0x7fcd30130660 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:21.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.583+0000 7fcd3741b700 1 -- 172.21.15.3:0/1594788831 >> 172.21.15.3:0/1594788831 conn(0x7fcd30066f50 msgr2=0x7fcd3009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:21.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.583+0000 7fcd3741b700 1 -- 172.21.15.3:0/1594788831 shutdown_connections 2023-10-03T10:12:21.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.583+0000 7fcd3741b700 1 -- 172.21.15.3:0/1594788831 wait complete. 2023-10-03T10:12:21.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.584+0000 7fcd3741b700 1 Processor -- start 2023-10-03T10:12:21.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.584+0000 7fcd3741b700 1 -- start start 2023-10-03T10:12:21.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.584+0000 7fcd3741b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd3009d6f0 0x7fcd3011f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:21.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.585+0000 7fcd3741b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd3009e000 0x7fcd3011f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:21.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.585+0000 7fcd3741b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd3009e410 0x7fcd3011fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:21.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.585+0000 7fcd3741b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd30126790 con 0x7fcd3009e000 2023-10-03T10:12:21.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.585+0000 7fcd3741b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3006b1f0 con 0x7fcd3009e410 2023-10-03T10:12:21.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.585+0000 7fcd3741b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3006eb10 con 0x7fcd3009d6f0 2023-10-03T10:12:21.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.585+0000 7fcd351b7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd3009d6f0 0x7fcd3011f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:21.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.585+0000 7fcd351b7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd3009d6f0 0x7fcd3011f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:59286/0 (socket says 172.21.15.3:59286) 2023-10-03T10:12:21.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.585+0000 7fcd349b6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd3009e000 0x7fcd3011f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:21.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.585+0000 7fcd359b8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd3009e410 0x7fcd3011fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:21.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.585+0000 7fcd351b7700 1 -- 172.21.15.3:0/2510702282 learned_addr learned my addr 172.21.15.3:0/2510702282 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:21.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.585+0000 7fcd351b7700 1 -- 172.21.15.3:0/2510702282 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd3009e410 msgr2=0x7fcd3011fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:21.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.585+0000 7fcd351b7700 1 --2- 172.21.15.3:0/2510702282 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd3009e410 0x7fcd3011fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:21.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.585+0000 7fcd351b7700 1 -- 172.21.15.3:0/2510702282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd3009e000 msgr2=0x7fcd3011f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:21.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.585+0000 7fcd351b7700 1 --2- 172.21.15.3:0/2510702282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd3009e000 0x7fcd3011f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:21.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.585+0000 7fcd351b7700 1 -- 172.21.15.3:0/2510702282 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd2c0094a0 con 0x7fcd3009d6f0 2023-10-03T10:12:21.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.585+0000 7fcd349b6700 1 --2- 172.21.15.3:0/2510702282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd3009e000 0x7fcd3011f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:21.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.585+0000 7fcd351b7700 1 --2- 172.21.15.3:0/2510702282 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd3009d6f0 0x7fcd3011f0d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00ebe0 tx=0x7fcd2c00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:21.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.585+0000 7fcd25ffb700 1 -- 172.21.15.3:0/2510702282 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c00c850 con 0x7fcd3009d6f0 2023-10-03T10:12:21.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.585+0000 7fcd25ffb700 1 -- 172.21.15.3:0/2510702282 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2c00c9b0 con 0x7fcd3009d6f0 2023-10-03T10:12:21.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.585+0000 7fcd3741b700 1 -- 172.21.15.3:0/2510702282 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd200094a0 con 0x7fcd3009d6f0 2023-10-03T10:12:21.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.585+0000 7fcd25ffb700 1 -- 172.21.15.3:0/2510702282 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c00cd40 con 0x7fcd3009d6f0 2023-10-03T10:12:21.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.586+0000 7fcd3741b700 1 -- 172.21.15.3:0/2510702282 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd30125880 con 0x7fcd3009d6f0 2023-10-03T10:12:21.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.586+0000 7fcd25ffb700 1 -- 172.21.15.3:0/2510702282 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fcd2c013470 con 0x7fcd3009d6f0 2023-10-03T10:12:21.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.587+0000 7fcd25ffb700 1 -- 172.21.15.3:0/2510702282 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4078+0+0 (secure 0 0 0) 0x7fcd2c03ffe0 con 0x7fcd3009d6f0 2023-10-03T10:12:21.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.587+0000 7fcd3741b700 1 -- 172.21.15.3:0/2510702282 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd30125a70 con 0x7fcd3009d6f0 2023-10-03T10:12:21.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.592+0000 7fcd25ffb700 1 -- 172.21.15.3:0/2510702282 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcd2c017020 con 0x7fcd3009d6f0 2023-10-03T10:12:21.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.837+0000 7fcd3741b700 1 -- 172.21.15.3:0/2510702282 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd301302d0 con 0x7fcd3009d6f0 2023-10-03T10:12:21.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.838+0000 7fcd25ffb700 1 -- 172.21.15.3:0/2510702282 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+2313 (secure 0 0 0) 0x7fcd2c013720 con 0x7fcd3009d6f0 2023-10-03T10:12:21.838 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:12:21.839 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-03T10:09:44.141304+0000","modified":"2023-10-03T10:10:11.643845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4186},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4186":{"gid":4186,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6817/2582564779","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6817","nonce":2582564779}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-03T10:12:21.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.840+0000 7fcd3741b700 1 -- 172.21.15.3:0/2510702282 >> v1:172.21.15.3:6816/127025 conn(0x7fcd1402f510 legacy=0x7fcd140319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:21.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.840+0000 7fcd3741b700 1 -- 172.21.15.3:0/2510702282 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd3009d6f0 msgr2=0x7fcd3011f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:21.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.840+0000 7fcd3741b700 1 --2- 172.21.15.3:0/2510702282 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd3009d6f0 0x7fcd3011f0d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00ebe0 tx=0x7fcd2c00eef0 comp rx=0 tx=0).stop 2023-10-03T10:12:21.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.840+0000 7fcd3741b700 1 -- 172.21.15.3:0/2510702282 shutdown_connections 2023-10-03T10:12:21.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.840+0000 7fcd3741b700 1 --2- 172.21.15.3:0/2510702282 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd3009e410 0x7fcd3011fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:21.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.840+0000 7fcd3741b700 1 --2- 172.21.15.3:0/2510702282 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd3009d6f0 0x7fcd3011f0d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:21.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.840+0000 7fcd3741b700 1 --2- 172.21.15.3:0/2510702282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd3009e000 0x7fcd3011f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:21.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.840+0000 7fcd3741b700 1 -- 172.21.15.3:0/2510702282 >> 172.21.15.3:0/2510702282 conn(0x7fcd30066f50 msgr2=0x7fcd30119480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:21.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.841+0000 7fcd3741b700 1 -- 172.21.15.3:0/2510702282 shutdown_connections 2023-10-03T10:12:21.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.841+0000 7fcd3741b700 1 -- 172.21.15.3:0/2510702282 wait complete. 2023-10-03T10:12:21.842 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2023-10-03T10:12:21.854 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-03T10:12:21.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.989+0000 7f8b9f670700 1 Processor -- start 2023-10-03T10:12:21.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.990+0000 7f8b9f670700 1 -- start start 2023-10-03T10:12:21.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.990+0000 7f8b9f670700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b98133420 0x7f8b981337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:21.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.990+0000 7f8b9f670700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8b98133e30 0x7f8b9812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:21.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.990+0000 7f8b9f670700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b9806b990 con 0x7f8b98134260 2023-10-03T10:12:21.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.990+0000 7f8b9f670700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b980a0590 con 0x7f8b98133e30 2023-10-03T10:12:21.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.990+0000 7f8b9f670700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9809fb80 con 0x7f8b98133420 2023-10-03T10:12:21.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.990+0000 7f8b9d40c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b98133420 0x7f8b981337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:21.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.990+0000 7f8b9cc0b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8b98133e30 0x7f8b9812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:21.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.990+0000 7f8b9d40c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b98133420 0x7f8b981337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37684/0 (socket says 172.21.15.3:37684) 2023-10-03T10:12:21.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.990+0000 7f8b9dc0d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8b98134260 0x7f8b981304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35768/0 (socket says 172.21.15.3:35768) 2023-10-03T10:12:21.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.990+0000 7f8b9d40c700 1 -- 172.21.15.3:0/971568121 learned_addr learned my addr 172.21.15.3:0/971568121 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:21.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.991+0000 7f8b9d40c700 1 -- 172.21.15.3:0/971568121 >> v2:172.21.15.3:3300/0 conn(0x7f8b98133e30 msgr2=0x7f8b9812ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:21.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.991+0000 7f8b9d40c700 1 --2- 172.21.15.3:0/971568121 >> v2:172.21.15.3:3300/0 conn(0x7f8b98133e30 0x7f8b9812ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:21.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.991+0000 7f8b9d40c700 1 -- 172.21.15.3:0/971568121 >> v1:172.21.15.3:6789/0 conn(0x7f8b98134260 legacy=0x7f8b981304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:21.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.991+0000 7f8b9d40c700 1 -- 172.21.15.3:0/971568121 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b980a0030 con 0x7f8b98133420 2023-10-03T10:12:21.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.991+0000 7f8b8ffff700 1 -- 172.21.15.3:0/971568121 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 443085203 0 0) 0x7f8b9806b990 con 0x7f8b98134260 2023-10-03T10:12:21.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.991+0000 7f8b9cc0b700 1 --2- 172.21.15.3:0/971568121 >> v2:172.21.15.3:3300/0 conn(0x7f8b98133e30 0x7f8b9812ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:21.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.991+0000 7f8b9d40c700 1 --2- 172.21.15.3:0/971568121 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b98133420 0x7f8b981337f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8b940098f0 tx=0x7f8b94009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaac88cb9fe3702d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:21.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.991+0000 7f8b8ffff700 1 -- 172.21.15.3:0/971568121 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b94011830 con 0x7f8b98133420 2023-10-03T10:12:21.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.991+0000 7f8b8ffff700 1 -- 172.21.15.3:0/971568121 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b94011990 con 0x7f8b98133420 2023-10-03T10:12:21.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.991+0000 7f8b8ffff700 1 -- 172.21.15.3:0/971568121 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b94011d20 con 0x7f8b98133420 2023-10-03T10:12:21.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.992+0000 7f8b9f670700 1 -- 172.21.15.3:0/971568121 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b98133420 msgr2=0x7f8b981337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:21.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.992+0000 7f8b9f670700 1 --2- 172.21.15.3:0/971568121 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b98133420 0x7f8b981337f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8b940098f0 tx=0x7f8b94009c00 comp rx=0 tx=0).stop 2023-10-03T10:12:21.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.992+0000 7f8b9f670700 1 -- 172.21.15.3:0/971568121 shutdown_connections 2023-10-03T10:12:21.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.992+0000 7f8b9f670700 1 --2- 172.21.15.3:0/971568121 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b98133420 0x7f8b981337f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:21.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.992+0000 7f8b9f670700 1 --2- 172.21.15.3:0/971568121 >> v2:172.21.15.3:3300/0 conn(0x7f8b98133e30 0x7f8b9812ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:21.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.992+0000 7f8b9f670700 1 -- 172.21.15.3:0/971568121 >> 172.21.15.3:0/971568121 conn(0x7f8b980a4210 msgr2=0x7f8b980a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:21.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.992+0000 7f8b9f670700 1 -- 172.21.15.3:0/971568121 shutdown_connections 2023-10-03T10:12:21.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.992+0000 7f8b9f670700 1 -- 172.21.15.3:0/971568121 wait complete. 2023-10-03T10:12:21.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.993+0000 7f8b9f670700 1 Processor -- start 2023-10-03T10:12:21.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.993+0000 7f8b9f670700 1 -- start start 2023-10-03T10:12:21.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.993+0000 7f8b9f670700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b98133420 0x7f8b9809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:21.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.993+0000 7f8b9f670700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b98133e30 0x7f8b9809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:21.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.993+0000 7f8b9f670700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b98134260 0x7f8b9809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:21.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.993+0000 7f8b9f670700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9811fad0 con 0x7f8b98133e30 2023-10-03T10:12:21.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.993+0000 7f8b9f670700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9812d850 con 0x7f8b98133420 2023-10-03T10:12:21.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.993+0000 7f8b9f670700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9811ee40 con 0x7f8b98134260 2023-10-03T10:12:21.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.993+0000 7f8b9d40c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b98133420 0x7f8b9809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:21.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.993+0000 7f8b9cc0b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b98133e30 0x7f8b9809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:21.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.993+0000 7f8b9dc0d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b98134260 0x7f8b9809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:21.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.993+0000 7f8b9d40c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b98133420 0x7f8b9809c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37692/0 (socket says 172.21.15.3:37692) 2023-10-03T10:12:21.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.993+0000 7f8b9d40c700 1 -- 172.21.15.3:0/3321244115 learned_addr learned my addr 172.21.15.3:0/3321244115 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:21.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.994+0000 7f8b9d40c700 1 -- 172.21.15.3:0/3321244115 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b98134260 msgr2=0x7f8b9809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:21.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.994+0000 7f8b9d40c700 1 --2- 172.21.15.3:0/3321244115 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b98134260 0x7f8b9809cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.994+0000 7f8b9d40c700 1 -- 172.21.15.3:0/3321244115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b98133e30 msgr2=0x7f8b9809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:22.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.994+0000 7f8b9d40c700 1 --2- 172.21.15.3:0/3321244115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b98133e30 0x7f8b9809c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.994+0000 7f8b9d40c700 1 -- 172.21.15.3:0/3321244115 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b880094a0 con 0x7f8b98133420 2023-10-03T10:12:22.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.994+0000 7f8b9cc0b700 1 --2- 172.21.15.3:0/3321244115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b98133e30 0x7f8b9809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:22.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.994+0000 7f8b9d40c700 1 --2- 172.21.15.3:0/3321244115 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b98133420 0x7f8b9809c480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8b94009c40 tx=0x7f8b94004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:22.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.994+0000 7f8b8dffb700 1 -- 172.21.15.3:0/3321244115 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b94004510 con 0x7f8b98133420 2023-10-03T10:12:22.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.994+0000 7f8b8dffb700 1 -- 172.21.15.3:0/3321244115 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b940036a0 con 0x7f8b98133420 2023-10-03T10:12:22.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.994+0000 7f8b9f670700 1 -- 172.21.15.3:0/3321244115 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b94009550 con 0x7f8b98133420 2023-10-03T10:12:22.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.994+0000 7f8b8dffb700 1 -- 172.21.15.3:0/3321244115 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b94003a50 con 0x7f8b98133420 2023-10-03T10:12:22.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.995+0000 7f8b9f670700 1 -- 172.21.15.3:0/3321244115 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b9811fd20 con 0x7f8b98133420 2023-10-03T10:12:22.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.995+0000 7f8b8dffb700 1 -- 172.21.15.3:0/3321244115 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f8b94021070 con 0x7f8b98133420 2023-10-03T10:12:22.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.995+0000 7f8b8dffb700 1 -- 172.21.15.3:0/3321244115 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4078+0+0 (secure 0 0 0) 0x7f8b94040fe0 con 0x7f8b98133420 2023-10-03T10:12:22.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:21.995+0000 7f8b9f670700 1 -- 172.21.15.3:0/3321244115 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b98074ea0 con 0x7f8b98133420 2023-10-03T10:12:22.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.000+0000 7f8b8dffb700 1 -- 172.21.15.3:0/3321244115 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8b94029020 con 0x7f8b98133420 2023-10-03T10:12:22.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.205+0000 7f8b9f670700 1 -- 172.21.15.3:0/3321244115 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b98120030 con 0x7f8b98133420 2023-10-03T10:12:22.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.206+0000 7f8b8dffb700 1 -- 172.21.15.3:0/3321244115 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+12923 (secure 0 0 0) 0x7f8b94015070 con 0x7f8b98133420 2023-10-03T10:12:22.207 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:12:22.207 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","created":"2023-10-03T10:09:28.703441+0000","modified":"2023-10-03T10:12:15.315657+0000","last_up_change":"2023-10-03T10:09:36.569401+0000","last_in_change":"2023-10-03T10:09:30.156546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-03T10:09:36.623971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-03T10:09:38.210614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-03T10:09:42.695873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-03T10:09:43.115588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cdcdd930-3c5b-4486-a021-7132f90fb6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6808","nonce":118845}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6809","nonce":118845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6811","nonce":118845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6810","nonce":118845}]},"public_addr":"172.21.15.3:6808/118845","cluster_addr":"172.21.15.3:6809/118845","heartbeat_back_addr":"172.21.15.3:6811/118845","heartbeat_front_addr":"172.21.15.3:6810/118845","state":["exists","up"]},{"osd":1,"uuid":"c8e58877-6762-4741-a8c1-567ebc97b708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6804","nonce":118835}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6805","nonce":118835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6807","nonce":118835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6806","nonce":118835}]},"public_addr":"172.21.15.3:6804/118835","cluster_addr":"172.21.15.3:6805/118835","heartbeat_back_addr":"172.21.15.3:6807/118835","heartbeat_front_addr":"172.21.15.3:6806/118835","state":["exists","up"]},{"osd":2,"uuid":"68a2a21a-7a8e-4f82-8433-5201faa50777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6812","nonce":118864}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6813","nonce":118864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6815","nonce":118864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6814","nonce":118864}]},"public_addr":"172.21.15.3:6812/118864","cluster_addr":"172.21.15.3:6813/118864","heartbeat_back_addr":"172.21.15.3:6815/118864","heartbeat_front_addr":"172.21.15.3:6814/118864","state":["exists","up"]},{"osd":3,"uuid":"c4df2903-4e1e-4208-bf77-95b9fa749241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6800","nonce":118836}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6801","nonce":118836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6803","nonce":118836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6802","nonce":118836}]},"public_addr":"172.21.15.3:6800/118836","cluster_addr":"172.21.15.3:6801/118836","heartbeat_back_addr":"172.21.15.3:6803/118836","heartbeat_front_addr":"172.21.15.3:6802/118836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.369708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.257825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.337931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.085222+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1","mappings":[{"from":3,"to":0}]},{"pgid":"4.8","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.19","mappings":[{"from":3,"to":0}]},{"pgid":"4.2a","mappings":[{"from":1,"to":2}]},{"pgid":"4.2d","mappings":[{"from":3,"to":0}]},{"pgid":"4.33","mappings":[{"from":3,"to":0}]},{"pgid":"4.39","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/3919724564":"2023-10-04T10:12:13.305411+0000","172.21.15.3:0/2091814084":"2023-10-04T10:12:13.305411+0000","172.21.15.3:6816/118296":"2023-10-04T10:12:13.305411+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-03T10:12:22.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.209+0000 7f8b9f670700 1 -- 172.21.15.3:0/3321244115 >> v1:172.21.15.3:6816/127025 conn(0x7f8b7c02f510 legacy=0x7f8b7c0319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:22.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.209+0000 7f8b9f670700 1 -- 172.21.15.3:0/3321244115 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b98133420 msgr2=0x7f8b9809c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:22.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.209+0000 7f8b9f670700 1 --2- 172.21.15.3:0/3321244115 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b98133420 0x7f8b9809c480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8b94009c40 tx=0x7f8b94004750 comp rx=0 tx=0).stop 2023-10-03T10:12:22.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.209+0000 7f8b9f670700 1 -- 172.21.15.3:0/3321244115 shutdown_connections 2023-10-03T10:12:22.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.209+0000 7f8b9f670700 1 --2- 172.21.15.3:0/3321244115 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b98134260 0x7f8b9809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.209+0000 7f8b9f670700 1 --2- 172.21.15.3:0/3321244115 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b98133420 0x7f8b9809c480 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.209+0000 7f8b9f670700 1 --2- 172.21.15.3:0/3321244115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b98133e30 0x7f8b9809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.209+0000 7f8b9f670700 1 -- 172.21.15.3:0/3321244115 >> 172.21.15.3:0/3321244115 conn(0x7f8b980a4210 msgr2=0x7f8b980a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:22.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.209+0000 7f8b9f670700 1 -- 172.21.15.3:0/3321244115 shutdown_connections 2023-10-03T10:12:22.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.209+0000 7f8b9f670700 1 -- 172.21.15.3:0/3321244115 wait complete. 2023-10-03T10:12:22.223 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-03T10:12:22.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.359+0000 7f29ad6f0700 1 Processor -- start 2023-10-03T10:12:22.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.359+0000 7f29ad6f0700 1 -- start start 2023-10-03T10:12:22.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.360+0000 7f29ad6f0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f29a8133420 0x7f29a81337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:22.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.360+0000 7f29ad6f0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29a8134260 0x7f29a81306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:22.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.360+0000 7f29ad6f0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a806b990 con 0x7f29a8133e30 2023-10-03T10:12:22.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.360+0000 7f29ad6f0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a80a0590 con 0x7f29a8133420 2023-10-03T10:12:22.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.360+0000 7f29ad6f0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a809fb80 con 0x7f29a8134260 2023-10-03T10:12:22.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.360+0000 7f29a77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29a8134260 0x7f29a81306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:22.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.360+0000 7f29a6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f29a8133420 0x7f29a81337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:22.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.360+0000 7f29a67fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f29a8133e30 0x7f29a812ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35782/0 (socket says 172.21.15.3:35782) 2023-10-03T10:12:22.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.360+0000 7f29a67fc700 1 -- 172.21.15.3:0/507503129 learned_addr learned my addr 172.21.15.3:0/507503129 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:22.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.360+0000 7f29a5ffb700 1 -- 172.21.15.3:0/507503129 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1040301496 0 0) 0x7f29a806b990 con 0x7f29a8133e30 2023-10-03T10:12:22.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.360+0000 7f29a5ffb700 1 -- 172.21.15.3:0/507503129 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29900035b0 con 0x7f29a8133e30 2023-10-03T10:12:22.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.360+0000 7f29a6ffd700 1 -- 172.21.15.3:0/507503129 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29a8134260 msgr2=0x7f29a81306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:22.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.360+0000 7f29a6ffd700 1 --2- 172.21.15.3:0/507503129 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29a8134260 0x7f29a81306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.360+0000 7f29a6ffd700 1 -- 172.21.15.3:0/507503129 >> v1:172.21.15.3:6789/0 conn(0x7f29a8133e30 legacy=0x7f29a812ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:22.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.360+0000 7f29a6ffd700 1 -- 172.21.15.3:0/507503129 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a80a0030 con 0x7f29a8133420 2023-10-03T10:12:22.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.360+0000 7f29a77fe700 1 --2- 172.21.15.3:0/507503129 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29a8134260 0x7f29a81306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:22.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.361+0000 7f29a6ffd700 1 --2- 172.21.15.3:0/507503129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29a8133420 0x7f29a81337f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2998009840 tx=0x7f2998009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26654b70ef179cb server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:22.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.361+0000 7f29a5ffb700 1 -- 172.21.15.3:0/507503129 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2998011830 con 0x7f29a8133420 2023-10-03T10:12:22.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.361+0000 7f29a5ffb700 1 -- 172.21.15.3:0/507503129 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2998011990 con 0x7f29a8133420 2023-10-03T10:12:22.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.361+0000 7f29a5ffb700 1 -- 172.21.15.3:0/507503129 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2998011d20 con 0x7f29a8133420 2023-10-03T10:12:22.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.361+0000 7f29ad6f0700 1 -- 172.21.15.3:0/507503129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29a8133420 msgr2=0x7f29a81337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:22.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.361+0000 7f29ad6f0700 1 --2- 172.21.15.3:0/507503129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29a8133420 0x7f29a81337f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2998009840 tx=0x7f2998009b50 comp rx=0 tx=0).stop 2023-10-03T10:12:22.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.361+0000 7f29ad6f0700 1 -- 172.21.15.3:0/507503129 shutdown_connections 2023-10-03T10:12:22.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.361+0000 7f29ad6f0700 1 --2- 172.21.15.3:0/507503129 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29a8134260 0x7f29a81306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.361+0000 7f29ad6f0700 1 --2- 172.21.15.3:0/507503129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29a8133420 0x7f29a81337f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.361+0000 7f29ad6f0700 1 -- 172.21.15.3:0/507503129 >> 172.21.15.3:0/507503129 conn(0x7f29a80a4210 msgr2=0x7f29a80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:22.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.362+0000 7f29ad6f0700 1 -- 172.21.15.3:0/507503129 shutdown_connections 2023-10-03T10:12:22.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.362+0000 7f29ad6f0700 1 -- 172.21.15.3:0/507503129 wait complete. 2023-10-03T10:12:22.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.363+0000 7f29ad6f0700 1 Processor -- start 2023-10-03T10:12:22.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.363+0000 7f29ad6f0700 1 -- start start 2023-10-03T10:12:22.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.363+0000 7f29ad6f0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29a8133420 0x7f29a809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:22.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.363+0000 7f29ad6f0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29a8133e30 0x7f29a809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:22.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.363+0000 7f29ad6f0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29a8134260 0x7f29a809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:22.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.363+0000 7f29ad6f0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a809df00 con 0x7f29a8133420 2023-10-03T10:12:22.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.363+0000 7f29ad6f0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a812dba0 con 0x7f29a8133e30 2023-10-03T10:12:22.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.363+0000 7f29ad6f0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29a811fcc0 con 0x7f29a8134260 2023-10-03T10:12:22.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.363+0000 7f29a6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29a8133420 0x7f29a809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:22.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.363+0000 7f29a6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29a8133420 0x7f29a809c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44530/0 (socket says 172.21.15.3:44530) 2023-10-03T10:12:22.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.363+0000 7f29a6ffd700 1 -- 172.21.15.3:0/2339442186 learned_addr learned my addr 172.21.15.3:0/2339442186 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:22.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.363+0000 7f29a77fe700 1 --2- 172.21.15.3:0/2339442186 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29a8134260 0x7f29a809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:22.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.363+0000 7f29a67fc700 1 --2- 172.21.15.3:0/2339442186 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29a8133e30 0x7f29a809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:22.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.364+0000 7f29a6ffd700 1 -- 172.21.15.3:0/2339442186 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29a8134260 msgr2=0x7f29a809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:22.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.364+0000 7f29a6ffd700 1 --2- 172.21.15.3:0/2339442186 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29a8134260 0x7f29a809cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.364+0000 7f29a6ffd700 1 -- 172.21.15.3:0/2339442186 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29a8133e30 msgr2=0x7f29a809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:22.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.364+0000 7f29a6ffd700 1 --2- 172.21.15.3:0/2339442186 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29a8133e30 0x7f29a809c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.364+0000 7f29a6ffd700 1 -- 172.21.15.3:0/2339442186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a0009a70 con 0x7f29a8133420 2023-10-03T10:12:22.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.364+0000 7f29a67fc700 1 --2- 172.21.15.3:0/2339442186 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29a8133e30 0x7f29a809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:22.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.364+0000 7f29a6ffd700 1 --2- 172.21.15.3:0/2339442186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29a8133420 0x7f29a809c480 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f29980090b0 tx=0x7f2998004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:22.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.364+0000 7f298f7fe700 1 -- 172.21.15.3:0/2339442186 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2998004380 con 0x7f29a8133420 2023-10-03T10:12:22.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.364+0000 7f298f7fe700 1 -- 172.21.15.3:0/2339442186 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f299801f070 con 0x7f29a8133420 2023-10-03T10:12:22.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.364+0000 7f29ad6f0700 1 -- 172.21.15.3:0/2339442186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29980094a0 con 0x7f29a8133420 2023-10-03T10:12:22.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.364+0000 7f298f7fe700 1 -- 172.21.15.3:0/2339442186 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29980036a0 con 0x7f29a8133420 2023-10-03T10:12:22.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.364+0000 7f29ad6f0700 1 -- 172.21.15.3:0/2339442186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a81209f0 con 0x7f29a8133420 2023-10-03T10:12:22.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.365+0000 7f298f7fe700 1 -- 172.21.15.3:0/2339442186 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f2998021030 con 0x7f29a8133420 2023-10-03T10:12:22.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.365+0000 7f29ad6f0700 1 -- 172.21.15.3:0/2339442186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a811fee0 con 0x7f29a8133420 2023-10-03T10:12:22.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.365+0000 7f298f7fe700 1 -- 172.21.15.3:0/2339442186 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4078+0+0 (secure 0 0 0) 0x7f2998040bf0 con 0x7f29a8133420 2023-10-03T10:12:22.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.371+0000 7f298f7fe700 1 -- 172.21.15.3:0/2339442186 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f29980239c0 con 0x7f29a8133420 2023-10-03T10:12:22.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.619+0000 7f29ad6f0700 1 -- 172.21.15.3:0/2339442186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29a8074ea0 con 0x7f29a8133420 2023-10-03T10:12:22.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.620+0000 7f298f7fe700 1 -- 172.21.15.3:0/2339442186 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+2313 (secure 0 0 0) 0x7f299801b430 con 0x7f29a8133420 2023-10-03T10:12:22.620 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:12:22.621 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-03T10:09:44.141304+0000","modified":"2023-10-03T10:10:11.643845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4186},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4186":{"gid":4186,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6817/2582564779","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6817","nonce":2582564779}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-03T10:12:22.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.622+0000 7f29ad6f0700 1 -- 172.21.15.3:0/2339442186 >> v1:172.21.15.3:6816/127025 conn(0x7f299402f510 legacy=0x7f29940319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:22.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.622+0000 7f29ad6f0700 1 -- 172.21.15.3:0/2339442186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29a8133420 msgr2=0x7f29a809c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:22.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.622+0000 7f29ad6f0700 1 --2- 172.21.15.3:0/2339442186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29a8133420 0x7f29a809c480 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f29980090b0 tx=0x7f2998004750 comp rx=0 tx=0).stop 2023-10-03T10:12:22.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.622+0000 7f29ad6f0700 1 -- 172.21.15.3:0/2339442186 shutdown_connections 2023-10-03T10:12:22.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.622+0000 7f29ad6f0700 1 --2- 172.21.15.3:0/2339442186 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29a8134260 0x7f29a809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.622+0000 7f29ad6f0700 1 --2- 172.21.15.3:0/2339442186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29a8133420 0x7f29a809c480 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.622+0000 7f29ad6f0700 1 --2- 172.21.15.3:0/2339442186 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29a8133e30 0x7f29a809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.622+0000 7f29ad6f0700 1 -- 172.21.15.3:0/2339442186 >> 172.21.15.3:0/2339442186 conn(0x7f29a80a4210 msgr2=0x7f29a8130480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:22.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.622+0000 7f29ad6f0700 1 -- 172.21.15.3:0/2339442186 shutdown_connections 2023-10-03T10:12:22.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.622+0000 7f29ad6f0700 1 -- 172.21.15.3:0/2339442186 wait complete. 2023-10-03T10:12:22.624 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2023-10-03T10:12:22.737 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-03T10:09:44.141304+0000', 'modified': '2023-10-03T10:10:11.643845+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4186}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4186': {'gid': 4186, 'name': 'a', 'rank': 0, 'incarnation': 3, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.3:6817/2582564779', 'addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.3:6817', 'nonce': 2582564779}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-03T10:12:22.737 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-03T10:12:22.737 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-03T10:12:22.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.878+0000 7fcef65a6700 1 Processor -- start 2023-10-03T10:12:22.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.878+0000 7fcef65a6700 1 -- start start 2023-10-03T10:12:22.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.879+0000 7fcef65a6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcef01316c0 0x7fcef0131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:22.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.879+0000 7fcef65a6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcef006ae80 0x7fcef0139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:22.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.879+0000 7fcef65a6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef009e130 con 0x7fcef009d9f0 2023-10-03T10:12:22.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.879+0000 7fcef65a6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef01312b0 con 0x7fcef01316c0 2023-10-03T10:12:22.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.879+0000 7fcef65a6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef009f6e0 con 0x7fcef006ae80 2023-10-03T10:12:22.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.879+0000 7fcef4b43700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcef006ae80 0x7fcef0139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:22.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.879+0000 7fceeffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcef01316c0 0x7fcef0131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:22.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.879+0000 7fcef4b43700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcef006ae80 0x7fcef0139550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37722/0 (socket says 172.21.15.3:37722) 2023-10-03T10:12:22.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.879+0000 7fcef4b43700 1 -- 172.21.15.3:0/117409043 learned_addr learned my addr 172.21.15.3:0/117409043 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:22.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.879+0000 7fceef7fe700 1 --1- 172.21.15.3:0/117409043 >> v1:172.21.15.3:6789/0 conn(0x7fcef009d9f0 0x7fcef006a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35792/0 (socket says 172.21.15.3:35792) 2023-10-03T10:12:22.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.879+0000 7fcef4b43700 1 -- 172.21.15.3:0/117409043 >> v2:172.21.15.3:3300/0 conn(0x7fcef01316c0 msgr2=0x7fcef0131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:22.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.879+0000 7fcef4b43700 1 --2- 172.21.15.3:0/117409043 >> v2:172.21.15.3:3300/0 conn(0x7fcef01316c0 0x7fcef0131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.879+0000 7fcef4b43700 1 -- 172.21.15.3:0/117409043 >> v1:172.21.15.3:6789/0 conn(0x7fcef009d9f0 legacy=0x7fcef006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:22.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.879+0000 7fcef4b43700 1 -- 172.21.15.3:0/117409043 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef009e760 con 0x7fcef006ae80 2023-10-03T10:12:22.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.880+0000 7fceeffff700 1 --2- 172.21.15.3:0/117409043 >> v2:172.21.15.3:3300/0 conn(0x7fcef01316c0 0x7fcef0131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:22.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.879+0000 7fceeeffd700 1 -- 172.21.15.3:0/117409043 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4284763211 0 0) 0x7fcee4001110 con 0x7fcef009d9f0 2023-10-03T10:12:22.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.880+0000 7fcef4b43700 1 --2- 172.21.15.3:0/117409043 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcef006ae80 0x7fcef0139550 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcee8009e50 tx=0x7fcee800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9784a09aaa8824c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:22.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.880+0000 7fceeeffd700 1 -- 172.21.15.3:0/117409043 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee800e070 con 0x7fcef006ae80 2023-10-03T10:12:22.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.880+0000 7fceeeffd700 1 -- 172.21.15.3:0/117409043 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee800b910 con 0x7fcef006ae80 2023-10-03T10:12:22.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.880+0000 7fceeeffd700 1 -- 172.21.15.3:0/117409043 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee800bcc0 con 0x7fcef006ae80 2023-10-03T10:12:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.880+0000 7fcef65a6700 1 -- 172.21.15.3:0/117409043 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcef006ae80 msgr2=0x7fcef0139550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.880+0000 7fcef65a6700 1 --2- 172.21.15.3:0/117409043 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcef006ae80 0x7fcef0139550 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcee8009e50 tx=0x7fcee800b330 comp rx=0 tx=0).stop 2023-10-03T10:12:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.881+0000 7fcef65a6700 1 -- 172.21.15.3:0/117409043 shutdown_connections 2023-10-03T10:12:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.881+0000 7fcef65a6700 1 --2- 172.21.15.3:0/117409043 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcef006ae80 0x7fcef0139550 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.881+0000 7fcef65a6700 1 --2- 172.21.15.3:0/117409043 >> v2:172.21.15.3:3300/0 conn(0x7fcef01316c0 0x7fcef0131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.881+0000 7fcef65a6700 1 -- 172.21.15.3:0/117409043 >> 172.21.15.3:0/117409043 conn(0x7fcef0066f50 msgr2=0x7fcef009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.881+0000 7fcef65a6700 1 -- 172.21.15.3:0/117409043 shutdown_connections 2023-10-03T10:12:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.881+0000 7fcef65a6700 1 -- 172.21.15.3:0/117409043 wait complete. 2023-10-03T10:12:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.882+0000 7fcef65a6700 1 Processor -- start 2023-10-03T10:12:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.882+0000 7fcef65a6700 1 -- start start 2023-10-03T10:12:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.882+0000 7fcef65a6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcef006ae80 0x7fcef0070980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.882+0000 7fcef65a6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcef009d9f0 0x7fcef0070ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:22.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.882+0000 7fcef65a6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcef01316c0 0x7fcef0071400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:22.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.882+0000 7fcef65a6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef0119d90 con 0x7fcef01316c0 2023-10-03T10:12:22.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.882+0000 7fcef65a6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef0128710 con 0x7fcef009d9f0 2023-10-03T10:12:22.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.882+0000 7fcef65a6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcef00a3540 con 0x7fcef006ae80 2023-10-03T10:12:22.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.882+0000 7fceef7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcef009d9f0 0x7fcef0070ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:22.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.882+0000 7fcef4b43700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcef01316c0 0x7fcef0071400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:22.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.882+0000 7fceeffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcef006ae80 0x7fcef0070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:22.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.882+0000 7fceef7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcef009d9f0 0x7fcef0070ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37738/0 (socket says 172.21.15.3:37738) 2023-10-03T10:12:22.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.882+0000 7fceef7fe700 1 -- 172.21.15.3:0/3632278435 learned_addr learned my addr 172.21.15.3:0/3632278435 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:22.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.882+0000 7fcef4b43700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcef01316c0 0x7fcef0071400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44548/0 (socket says 172.21.15.3:44548) 2023-10-03T10:12:22.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.883+0000 7fceef7fe700 1 -- 172.21.15.3:0/3632278435 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcef006ae80 msgr2=0x7fcef0070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:22.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.883+0000 7fceef7fe700 1 --2- 172.21.15.3:0/3632278435 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcef006ae80 0x7fcef0070980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.883+0000 7fceef7fe700 1 -- 172.21.15.3:0/3632278435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcef01316c0 msgr2=0x7fcef0071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:22.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.883+0000 7fceef7fe700 1 --2- 172.21.15.3:0/3632278435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcef01316c0 0x7fcef0071400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.883+0000 7fceef7fe700 1 -- 172.21.15.3:0/3632278435 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee00094a0 con 0x7fcef009d9f0 2023-10-03T10:12:22.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.883+0000 7fceeffff700 1 --2- 172.21.15.3:0/3632278435 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcef006ae80 0x7fcef0070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:22.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.883+0000 7fcef4b43700 1 --2- 172.21.15.3:0/3632278435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcef01316c0 0x7fcef0071400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:22.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.883+0000 7fceef7fe700 1 --2- 172.21.15.3:0/3632278435 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcef009d9f0 0x7fcef0070ec0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcee400c520 tx=0x7fcee400c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:22.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.883+0000 7fceecff9700 1 -- 172.21.15.3:0/3632278435 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee4004c80 con 0x7fcef009d9f0 2023-10-03T10:12:22.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.883+0000 7fceecff9700 1 -- 172.21.15.3:0/3632278435 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee4004de0 con 0x7fcef009d9f0 2023-10-03T10:12:22.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.883+0000 7fcef65a6700 1 -- 172.21.15.3:0/3632278435 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee8009b30 con 0x7fcef009d9f0 2023-10-03T10:12:22.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.883+0000 7fceecff9700 1 -- 172.21.15.3:0/3632278435 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee4007310 con 0x7fcef009d9f0 2023-10-03T10:12:22.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.883+0000 7fcef65a6700 1 -- 172.21.15.3:0/3632278435 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef011aaa0 con 0x7fcef009d9f0 2023-10-03T10:12:22.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.884+0000 7fceecff9700 1 -- 172.21.15.3:0/3632278435 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fcee4007470 con 0x7fcef009d9f0 2023-10-03T10:12:22.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.884+0000 7fceecff9700 1 -- 172.21.15.3:0/3632278435 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4078+0+0 (secure 0 0 0) 0x7fcee403f440 con 0x7fcef009d9f0 2023-10-03T10:12:22.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.887+0000 7fcf02749b80 1 Processor -- start 2023-10-03T10:12:22.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.888+0000 7fcf02749b80 1 -- start start 2023-10-03T10:12:22.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.888+0000 7fcf02749b80 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56295b548c50 0x56295b549020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:22.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.888+0000 7fcf02749b80 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56295b565100 0x56295b549560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:22.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.888+0000 7fcf02749b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56295b55f8f0 0x56295b55fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:22.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.888+0000 7fcf02749b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56295b4abc60 con 0x56295b55f8f0 2023-10-03T10:12:22.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.888+0000 7fcf02749b80 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x56295b4a6a10 con 0x56295b548c50 2023-10-03T10:12:22.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.888+0000 7fcf02749b80 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x56295b4a4210 con 0x56295b565100 2023-10-03T10:12:22.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.888+0000 7fceeffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56295b548c50 0x56295b549020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:22.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.888+0000 7fceeffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56295b548c50 0x56295b549020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37752/0 (socket says 172.21.15.3:37752) 2023-10-03T10:12:22.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.888+0000 7fceeffff700 1 -- 172.21.15.3:0/3861323343 learned_addr learned my addr 172.21.15.3:0/3861323343 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:22.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.888+0000 7fcef4b43700 1 --2- 172.21.15.3:0/3861323343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56295b55f8f0 0x56295b55fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:22.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.888+0000 7fceef7fe700 1 --2- 172.21.15.3:0/3861323343 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56295b565100 0x56295b549560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:22.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.889+0000 7fcef4b43700 1 -- 172.21.15.3:0/3861323343 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56295b565100 msgr2=0x56295b549560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:22.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.889+0000 7fcef4b43700 1 --2- 172.21.15.3:0/3861323343 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56295b565100 0x56295b549560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.889+0000 7fcef4b43700 1 -- 172.21.15.3:0/3861323343 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56295b548c50 msgr2=0x56295b549020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:22.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.889+0000 7fcef4b43700 1 --2- 172.21.15.3:0/3861323343 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56295b548c50 0x56295b549020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.889+0000 7fcef4b43700 1 -- 172.21.15.3:0/3861323343 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56295b293790 con 0x56295b55f8f0 2023-10-03T10:12:22.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.889+0000 7fceeffff700 1 --2- 172.21.15.3:0/3861323343 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56295b548c50 0x56295b549020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:22.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.889+0000 7fceef7fe700 1 --2- 172.21.15.3:0/3861323343 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56295b565100 0x56295b549560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:22.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.889+0000 7fcef4b43700 1 --2- 172.21.15.3:0/3861323343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56295b55f8f0 0x56295b55fd60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcef009d810 tx=0x7fcee8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d939c9e95d2098c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:22.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.889+0000 7fced5ffb700 1 -- 172.21.15.3:0/3861323343 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee800bc80 con 0x56295b55f8f0 2023-10-03T10:12:22.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.889+0000 7fced5ffb700 1 -- 172.21.15.3:0/3861323343 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee8004040 con 0x56295b55f8f0 2023-10-03T10:12:22.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.889+0000 7fced5ffb700 1 -- 172.21.15.3:0/3861323343 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee80043b0 con 0x56295b55f8f0 2023-10-03T10:12:22.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.890+0000 7fcf02749b80 1 -- 172.21.15.3:0/3861323343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56295b55f8f0 msgr2=0x56295b55fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:22.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.890+0000 7fcf02749b80 1 --2- 172.21.15.3:0/3861323343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56295b55f8f0 0x56295b55fd60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcef009d810 tx=0x7fcee8010000 comp rx=0 tx=0).stop 2023-10-03T10:12:22.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.890+0000 7fcf02749b80 1 -- 172.21.15.3:0/3861323343 shutdown_connections 2023-10-03T10:12:22.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.890+0000 7fcf02749b80 1 --2- 172.21.15.3:0/3861323343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56295b55f8f0 0x56295b55fd60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.890+0000 7fcf02749b80 1 --2- 172.21.15.3:0/3861323343 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56295b548c50 0x56295b549020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.890+0000 7fcf02749b80 1 --2- 172.21.15.3:0/3861323343 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56295b565100 0x56295b549560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.890+0000 7fcf02749b80 1 -- 172.21.15.3:0/3861323343 >> 172.21.15.3:0/3861323343 conn(0x56295b54d0f0 msgr2=0x56295b54c690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:22.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.890+0000 7fcf02749b80 1 -- 172.21.15.3:0/3861323343 shutdown_connections 2023-10-03T10:12:22.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.890+0000 7fcf02749b80 1 -- 172.21.15.3:0/3861323343 wait complete. 2023-10-03T10:12:22.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.890+0000 7fcf02749b80 1 -- start start 2023-10-03T10:12:22.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.891+0000 7fcf02749b80 1 Processor -- start 2023-10-03T10:12:22.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.892+0000 7fcf02749b80 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56295b548c50 0x56295b568b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:22.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.892+0000 7fcf02749b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56295b55f8f0 0x56295b5690b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:22.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.892+0000 7fcf02749b80 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56295b565100 0x56295b5695f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:22.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.892+0000 7fcf02749b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56295b49fb00 con 0x56295b55f8f0 2023-10-03T10:12:22.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.892+0000 7fcf02749b80 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x56295b43b770 con 0x56295b565100 2023-10-03T10:12:22.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.892+0000 7fcf02749b80 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x56295b45e6f0 con 0x56295b548c50 2023-10-03T10:12:22.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.892+0000 7fcf02749b80 10 client.0 _ll_register_callbacks cb 0x56295b2a75a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-03T10:12:22.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.892+0000 7fcef4b43700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56295b565100 0x56295b5695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:22.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.892+0000 7fceef7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56295b55f8f0 0x56295b5690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:22.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.892+0000 7fcef4b43700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56295b565100 0x56295b5695f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37766/0 (socket says 172.21.15.3:37766) 2023-10-03T10:12:22.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.892+0000 7fcef4b43700 1 -- 172.21.15.3:0/3525900200 learned_addr learned my addr 172.21.15.3:0/3525900200 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:22.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.892+0000 7fceeffff700 1 --2- 172.21.15.3:0/3525900200 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56295b548c50 0x56295b568b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:22.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.892+0000 7fceef7fe700 1 -- 172.21.15.3:0/3525900200 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56295b548c50 msgr2=0x56295b568b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:22.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.892+0000 7fceef7fe700 1 --2- 172.21.15.3:0/3525900200 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56295b548c50 0x56295b568b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.892+0000 7fceef7fe700 1 -- 172.21.15.3:0/3525900200 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56295b565100 msgr2=0x56295b5695f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:22.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.892+0000 7fceef7fe700 1 --2- 172.21.15.3:0/3525900200 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56295b565100 0x56295b5695f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.892+0000 7fceef7fe700 1 -- 172.21.15.3:0/3525900200 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee8009b30 con 0x56295b55f8f0 2023-10-03T10:12:22.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.892+0000 7fcef4b43700 1 --2- 172.21.15.3:0/3525900200 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56295b565100 0x56295b5695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:22.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.892+0000 7fceef7fe700 1 --2- 172.21.15.3:0/3525900200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56295b55f8f0 0x56295b5690b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcee403df90 tx=0x7fcee4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f3b27fd8cc30d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:22.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.892+0000 7fceceffd700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-10-03T10:12:22.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.892+0000 7fceceffd700 1 -- 172.21.15.3:0/3525900200 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee401c590 con 0x56295b55f8f0 2023-10-03T10:12:22.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.892+0000 7fceceffd700 1 -- 172.21.15.3:0/3525900200 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 4078+0+0 (secure 0 0 0) 0x7fcee401cab0 con 0x56295b55f8f0 2023-10-03T10:12:22.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.892+0000 7fcf02749b80 1 -- 172.21.15.3:0/3525900200 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56295b5412b0 con 0x56295b55f8f0 2023-10-03T10:12:22.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.893+0000 7fceceffd700 1 -- 172.21.15.3:0/3525900200 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcee401cec0 con 0x56295b55f8f0 2023-10-03T10:12:22.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.893+0000 7fcf02749b80 10 client.14442 fetch_fsmap learned FSMap version 8 2023-10-03T10:12:22.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.893+0000 7fcf02749b80 1 -- 172.21.15.3:0/3525900200 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=3+}) v3 -- 0x7fcee800be70 con 0x56295b55f8f0 2023-10-03T10:12:22.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.893+0000 7fceceffd700 1 -- 172.21.15.3:0/3525900200 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56295b5412b0 con 0x56295b55f8f0 2023-10-03T10:12:22.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.893+0000 7fceceffd700 1 -- 172.21.15.3:0/3525900200 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 8) v1 ==== 1436+0+0 (secure 0 0 0) 0x7fcee401b340 con 0x56295b55f8f0 2023-10-03T10:12:22.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.893+0000 7fcf02749b80 10 client.14442 fetch_fsmap finished waiting for FSMap version 8 2023-10-03T10:12:22.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.893+0000 7fcf02749b80 10 client.14442 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-03T10:12:22.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.893+0000 7fcf02749b80 20 client.14442 populate_metadata read hostname 'smithi003' 2023-10-03T10:12:22.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.893+0000 7fcf02749b80 4 client.14442 mds_command: new command op to 4186 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-03T10:12:22.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.893+0000 7fcf02749b80 1 -- 172.21.15.3:0/3525900200 --> v1:172.21.15.3:6817/2582564779 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56295b51df40 con 0x56295b55f1e0 2023-10-03T10:12:22.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.893+0000 7fceceffd700 10 client.14442 ms_handle_connect on v1:172.21.15.3:6817/2582564779 2023-10-03T10:12:22.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.894+0000 7fceceffd700 1 -- 172.21.15.3:0/3525900200 <== mds.0 v1:172.21.15.3:6817/2582564779 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (unknown 0 0 3608088055) 0x56295b51df40 con 0x56295b55f1e0 2023-10-03T10:12:22.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.894+0000 7fceceffd700 10 client.14442 handle_command_reply: tid=0 2023-10-03T10:12:22.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.894+0000 7fcf02749b80 1 client.14442 shutdown 2023-10-03T10:12:22.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.894+0000 7fcf02749b80 1 -- 172.21.15.3:0/3525900200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56295b55f8f0 msgr2=0x56295b5690b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:22.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.894+0000 7fcf02749b80 1 --2- 172.21.15.3:0/3525900200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56295b55f8f0 0x56295b5690b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcee403df90 tx=0x7fcee4000ed0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.894+0000 7fcf02749b80 1 -- 172.21.15.3:0/3525900200 shutdown_connections 2023-10-03T10:12:22.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.894+0000 7fcf02749b80 1 --2- 172.21.15.3:0/3525900200 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56295b565100 0x56295b5695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.894+0000 7fcf02749b80 1 --2- 172.21.15.3:0/3525900200 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56295b548c50 0x56295b568b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.894+0000 7fceceffd700 0 client.14442 ms_handle_reset on v1:172.21.15.3:6817/2582564779 2023-10-03T10:12:22.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.894+0000 7fcf02749b80 1 --2- 172.21.15.3:0/3525900200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56295b55f8f0 0x56295b5690b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.894+0000 7fcf02749b80 1 -- 172.21.15.3:0/3525900200 >> 172.21.15.3:0/3525900200 conn(0x56295b54d0f0 msgr2=0x56295b549230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:22.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.894+0000 7fcf02749b80 1 -- 172.21.15.3:0/3525900200 shutdown_connections 2023-10-03T10:12:22.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.894+0000 7fcf02749b80 1 -- 172.21.15.3:0/3525900200 wait complete. 2023-10-03T10:12:22.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.895+0000 7fcf02749b80 20 client.14442 trim_cache size 0 max 16384 2023-10-03T10:12:22.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.912+0000 7fcf02749b80 1 Processor -- start 2023-10-03T10:12:22.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.912+0000 7fcf02749b80 1 -- start start 2023-10-03T10:12:22.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.913+0000 7fcf02749b80 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56295b548c50 0x56295b569a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:22.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.913+0000 7fcf02749b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56295b55f8f0 0x56295b589ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:22.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.913+0000 7fcf02749b80 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56295b565100 0x56295b55fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:22.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.913+0000 7fcf02749b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56295b294390 con 0x56295b55f8f0 2023-10-03T10:12:22.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.913+0000 7fcf02749b80 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x56295b54a100 con 0x56295b548c50 2023-10-03T10:12:22.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.913+0000 7fcf02749b80 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x56295b56ecb0 con 0x56295b565100 2023-10-03T10:12:22.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.913+0000 7fceef7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56295b55f8f0 0x56295b589ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:22.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.913+0000 7fceeffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56295b548c50 0x56295b569a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:22.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.913+0000 7fceef7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56295b55f8f0 0x56295b589ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44564/0 (socket says 172.21.15.3:44564) 2023-10-03T10:12:22.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.913+0000 7fceeffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56295b548c50 0x56295b569a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37778/0 (socket says 172.21.15.3:37778) 2023-10-03T10:12:22.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.913+0000 7fceef7fe700 1 -- 172.21.15.3:0/3395747677 learned_addr learned my addr 172.21.15.3:0/3395747677 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:22.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.913+0000 7fcef4b43700 1 --2- 172.21.15.3:0/3395747677 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56295b565100 0x56295b55fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:22.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.913+0000 7fceef7fe700 1 -- 172.21.15.3:0/3395747677 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56295b565100 msgr2=0x56295b55fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:22.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.913+0000 7fceef7fe700 1 --2- 172.21.15.3:0/3395747677 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56295b565100 0x56295b55fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.913+0000 7fceef7fe700 1 -- 172.21.15.3:0/3395747677 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56295b548c50 msgr2=0x56295b569a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:22.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.913+0000 7fceef7fe700 1 --2- 172.21.15.3:0/3395747677 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56295b548c50 0x56295b569a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.913+0000 7fceef7fe700 1 -- 172.21.15.3:0/3395747677 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56295b3800a0 con 0x56295b55f8f0 2023-10-03T10:12:22.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.913+0000 7fcef4b43700 1 --2- 172.21.15.3:0/3395747677 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56295b565100 0x56295b55fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:22.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.913+0000 7fceeffff700 1 --2- 172.21.15.3:0/3395747677 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56295b548c50 0x56295b569a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:22.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.913+0000 7fceef7fe700 1 --2- 172.21.15.3:0/3395747677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56295b55f8f0 0x56295b589ae0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcee403f080 tx=0x7fcee403c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59ea52cae648d7f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:22.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.914+0000 7fced67fc700 1 -- 172.21.15.3:0/3395747677 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee4011e50 con 0x56295b55f8f0 2023-10-03T10:12:22.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.914+0000 7fced67fc700 1 -- 172.21.15.3:0/3395747677 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee4007df0 con 0x56295b55f8f0 2023-10-03T10:12:22.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.914+0000 7fced67fc700 1 -- 172.21.15.3:0/3395747677 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee401cab0 con 0x56295b55f8f0 2023-10-03T10:12:22.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.914+0000 7fcf02749b80 1 -- 172.21.15.3:0/3395747677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56295b55f8f0 msgr2=0x56295b589ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:22.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.914+0000 7fcf02749b80 1 --2- 172.21.15.3:0/3395747677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56295b55f8f0 0x56295b589ae0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcee403f080 tx=0x7fcee403c000 comp rx=0 tx=0).stop 2023-10-03T10:12:22.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.914+0000 7fcf02749b80 1 -- 172.21.15.3:0/3395747677 shutdown_connections 2023-10-03T10:12:22.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.914+0000 7fcf02749b80 1 --2- 172.21.15.3:0/3395747677 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56295b565100 0x56295b55fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.914+0000 7fcf02749b80 1 --2- 172.21.15.3:0/3395747677 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56295b548c50 0x56295b569a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.914+0000 7fcf02749b80 1 --2- 172.21.15.3:0/3395747677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56295b55f8f0 0x56295b589ae0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.914+0000 7fcf02749b80 1 -- 172.21.15.3:0/3395747677 >> 172.21.15.3:0/3395747677 conn(0x56295b54d0f0 msgr2=0x56295b549230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:22.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.914+0000 7fcf02749b80 1 -- 172.21.15.3:0/3395747677 shutdown_connections 2023-10-03T10:12:22.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.914+0000 7fcf02749b80 1 -- 172.21.15.3:0/3395747677 wait complete. 2023-10-03T10:12:22.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.915+0000 7fcf02749b80 1 -- start start 2023-10-03T10:12:22.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.915+0000 7fcf02749b80 1 Processor -- start 2023-10-03T10:12:22.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.916+0000 7fcf02749b80 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56295b548c50 0x56295b5cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:22.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.916+0000 7fcf02749b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56295b55f8f0 0x56295b5cb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:22.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.916+0000 7fcf02749b80 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56295b565100 0x56295b57c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:22.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.916+0000 7fcf02749b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56295b54a980 con 0x56295b55f8f0 2023-10-03T10:12:22.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.916+0000 7fcf02749b80 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x56295b57d990 con 0x56295b548c50 2023-10-03T10:12:22.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.916+0000 7fcf02749b80 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x56295b57e5f0 con 0x56295b565100 2023-10-03T10:12:22.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.916+0000 7fcf02749b80 10 client.0 _ll_register_callbacks cb 0x7fcee401b860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-03T10:12:22.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.916+0000 7fceeffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56295b548c50 0x56295b5cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:22.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.916+0000 7fceeffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56295b548c50 0x56295b5cace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37780/0 (socket says 172.21.15.3:37780) 2023-10-03T10:12:22.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.916+0000 7fceeffff700 1 -- 172.21.15.3:0/512262449 learned_addr learned my addr 172.21.15.3:0/512262449 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:22.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.916+0000 7fcef4b43700 1 --2- 172.21.15.3:0/512262449 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56295b565100 0x56295b57c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:22.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.916+0000 7fceef7fe700 1 --2- 172.21.15.3:0/512262449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56295b55f8f0 0x56295b5cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:22.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.916+0000 7fceeffff700 1 -- 172.21.15.3:0/512262449 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56295b565100 msgr2=0x56295b57c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:22.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.916+0000 7fceeffff700 1 --2- 172.21.15.3:0/512262449 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56295b565100 0x56295b57c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.916+0000 7fceeffff700 1 -- 172.21.15.3:0/512262449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56295b55f8f0 msgr2=0x56295b5cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:22.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.916+0000 7fceeffff700 1 --2- 172.21.15.3:0/512262449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56295b55f8f0 0x56295b5cb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.916+0000 7fceeffff700 1 -- 172.21.15.3:0/512262449 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee0011780 con 0x56295b548c50 2023-10-03T10:12:22.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.916+0000 7fceeffff700 1 --2- 172.21.15.3:0/512262449 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56295b548c50 0x56295b5cace0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcee0009fd0 tx=0x7fcee0009210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b3bafd8ca3ef871 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:22.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.916+0000 7fceceffd700 10 client.0 ms_handle_connect on v2:172.21.15.3:3301/0 2023-10-03T10:12:22.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.916+0000 7fceceffd700 1 -- 172.21.15.3:0/512262449 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee001e040 con 0x56295b548c50 2023-10-03T10:12:22.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.916+0000 7fceceffd700 1 -- 172.21.15.3:0/512262449 <== mon.1 v2:172.21.15.3:3301/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 4078+0+0 (secure 0 0 0) 0x7fcee0017b30 con 0x56295b548c50 2023-10-03T10:12:22.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.916+0000 7fcf02749b80 1 -- 172.21.15.3:0/512262449 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56295b498ce0 con 0x56295b548c50 2023-10-03T10:12:22.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.917+0000 7fceceffd700 1 -- 172.21.15.3:0/512262449 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcee00172e0 con 0x56295b548c50 2023-10-03T10:12:22.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.917+0000 7fcf02749b80 10 client.14359 fetch_fsmap learned FSMap version 8 2023-10-03T10:12:22.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.917+0000 7fcf02749b80 1 -- 172.21.15.3:0/512262449 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,fsmap=8,monmap=3+}) v3 -- 0x56295b293790 con 0x56295b548c50 2023-10-03T10:12:22.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.917+0000 7fceceffd700 1 -- 172.21.15.3:0/512262449 <== mon.1 v2:172.21.15.3:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56295b498ce0 con 0x56295b548c50 2023-10-03T10:12:22.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.917+0000 7fceceffd700 1 -- 172.21.15.3:0/512262449 <== mon.1 v2:172.21.15.3:3301/0 5 ==== fsmap(e 8) v1 ==== 1436+0+0 (secure 0 0 0) 0x7fcee0017cf0 con 0x56295b548c50 2023-10-03T10:12:22.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.917+0000 7fcf02749b80 10 client.14359 fetch_fsmap finished waiting for FSMap version 8 2023-10-03T10:12:22.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.917+0000 7fcf02749b80 10 client.14359 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-03T10:12:22.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.917+0000 7fcf02749b80 20 client.14359 populate_metadata read hostname 'smithi003' 2023-10-03T10:12:22.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.917+0000 7fcf02749b80 4 client.14359 mds_command: new command op to 4186 tid=0[{"prefix": "status"}] 2023-10-03T10:12:22.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.917+0000 7fcf02749b80 1 -- 172.21.15.3:0/512262449 --> v1:172.21.15.3:6817/2582564779 -- command(tid 0: {"prefix": "status"}) v1 -- 0x56295b2eeef0 con 0x56295b57d170 2023-10-03T10:12:22.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.917+0000 7fceceffd700 10 client.14359 ms_handle_connect on v1:172.21.15.3:6817/2582564779 2023-10-03T10:12:22.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.918+0000 7fceceffd700 1 -- 172.21.15.3:0/512262449 <== mds.0 v1:172.21.15.3:6817/2582564779 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (unknown 0 0 1051829685) 0x56295b2eeef0 con 0x56295b57d170 2023-10-03T10:12:22.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.918+0000 7fceceffd700 10 client.14359 handle_command_reply: tid=0 2023-10-03T10:12:22.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.918+0000 7fcf02749b80 1 client.14359 shutdown 2023-10-03T10:12:22.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.918+0000 7fcf02749b80 1 -- 172.21.15.3:0/512262449 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56295b548c50 msgr2=0x56295b5cace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:22.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.918+0000 7fcf02749b80 1 --2- 172.21.15.3:0/512262449 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56295b548c50 0x56295b5cace0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcee0009fd0 tx=0x7fcee0009210 comp rx=0 tx=0).stop 2023-10-03T10:12:22.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.918+0000 7fcf02749b80 1 -- 172.21.15.3:0/512262449 shutdown_connections 2023-10-03T10:12:22.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.918+0000 7fcf02749b80 1 --2- 172.21.15.3:0/512262449 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x56295b565100 0x56295b57c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.918+0000 7fcf02749b80 1 --2- 172.21.15.3:0/512262449 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x56295b548c50 0x56295b5cace0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.918+0000 7fcf02749b80 1 --2- 172.21.15.3:0/512262449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56295b55f8f0 0x56295b5cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.918+0000 7fceceffd700 0 client.14359 ms_handle_reset on v1:172.21.15.3:6817/2582564779 2023-10-03T10:12:22.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.918+0000 7fcf02749b80 1 -- 172.21.15.3:0/512262449 >> 172.21.15.3:0/512262449 conn(0x56295b54d0f0 msgr2=0x56295b547920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:22.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.918+0000 7fcf02749b80 1 -- 172.21.15.3:0/512262449 shutdown_connections 2023-10-03T10:12:22.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.918+0000 7fcf02749b80 1 -- 172.21.15.3:0/512262449 wait complete. 2023-10-03T10:12:22.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.918+0000 7fcf02749b80 20 client.14359 trim_cache size 0 max 16384 2023-10-03T10:12:22.923 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-03T10:12:22.924 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "68e08e7d-a212-4fd3-bd42-88e2411aa299", 2023-10-03T10:12:22.924 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2023-10-03T10:12:22.924 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4186, 2023-10-03T10:12:22.924 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-10-03T10:12:22.924 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-10-03T10:12:22.924 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-10-03T10:12:22.924 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 158.77075268199999, 2023-10-03T10:12:22.924 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 8, 2023-10-03T10:12:22.924 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 23, 2023-10-03T10:12:22.924 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 18, 2023-10-03T10:12:22.924 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 163.22662675399999 2023-10-03T10:12:22.924 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-03T10:12:22.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.919+0000 7fcef65a6700 1 -- 172.21.15.3:0/3632278435 >> v1:172.21.15.3:6816/127025 conn(0x7fcedc02f510 legacy=0x7fcedc0319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:22.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.919+0000 7fcef65a6700 1 -- 172.21.15.3:0/3632278435 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcef009d9f0 msgr2=0x7fcef0070ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:22.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.919+0000 7fcef65a6700 1 --2- 172.21.15.3:0/3632278435 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcef009d9f0 0x7fcef0070ec0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcee400c520 tx=0x7fcee400c830 comp rx=0 tx=0).stop 2023-10-03T10:12:22.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.919+0000 7fcef65a6700 1 -- 172.21.15.3:0/3632278435 shutdown_connections 2023-10-03T10:12:22.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.919+0000 7fcef65a6700 1 --2- 172.21.15.3:0/3632278435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcef01316c0 0x7fcef0071400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.919+0000 7fcef65a6700 1 --2- 172.21.15.3:0/3632278435 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcef006ae80 0x7fcef0070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.919+0000 7fcef65a6700 1 --2- 172.21.15.3:0/3632278435 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcef009d9f0 0x7fcef0070ec0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:22.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.919+0000 7fcef65a6700 1 -- 172.21.15.3:0/3632278435 >> 172.21.15.3:0/3632278435 conn(0x7fcef0066f50 msgr2=0x7fcef012b070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:22.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.919+0000 7fcef65a6700 1 -- 172.21.15.3:0/3632278435 shutdown_connections 2023-10-03T10:12:22.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:22.919+0000 7fcef65a6700 1 -- 172.21.15.3:0/3632278435 wait complete. 2023-10-03T10:12:22.934 INFO:teuthology.run_tasks:Running task ceph.restart... 2023-10-03T10:12:22.946 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-03T10:12:22.947 INFO:teuthology.orchestra.run:waiting for 300 2023-10-03T10:12:22.980 INFO:tasks.ceph.osd.0:Stopped 2023-10-03T10:12:22.980 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 2023-10-03T10:12:23.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.162+0000 7f3672483700 1 Processor -- start 2023-10-03T10:12:23.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.162+0000 7f3672483700 1 -- start start 2023-10-03T10:12:23.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.163+0000 7f3672483700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f366c09d6f0 0x7f366c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:23.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.163+0000 7f3672483700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f366c09e440 0x7f366c134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:23.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.163+0000 7f3672483700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f366c09b120 con 0x7f366c09e000 2023-10-03T10:12:23.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.163+0000 7f3672483700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f366c06b460 con 0x7f366c09d6f0 2023-10-03T10:12:23.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.163+0000 7f3672483700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f366c09a7e0 con 0x7f366c09e440 2023-10-03T10:12:23.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.163+0000 7f366bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f366c09d6f0 0x7f366c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:23.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.163+0000 7f366b7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f366c09e000 0x7f366c1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40740/0 (socket says 172.21.15.3:40740) 2023-10-03T10:12:23.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.163+0000 7f3670a20700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f366c09e440 0x7f366c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:23.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.163+0000 7f366b7fe700 1 -- 172.21.15.3:0/2971985117 learned_addr learned my addr 172.21.15.3:0/2971985117 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:23.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.163+0000 7f366affd700 1 -- 172.21.15.3:0/2971985117 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2538255280 0 0) 0x7f366c09b120 con 0x7f366c09e000 2023-10-03T10:12:23.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.163+0000 7f366affd700 1 -- 172.21.15.3:0/2971985117 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36540035b0 con 0x7f366c09e000 2023-10-03T10:12:23.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.163+0000 7f3670a20700 1 -- 172.21.15.3:0/2971985117 >> v2:172.21.15.3:3300/0 conn(0x7f366c09d6f0 msgr2=0x7f366c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:23.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.163+0000 7f3670a20700 1 --2- 172.21.15.3:0/2971985117 >> v2:172.21.15.3:3300/0 conn(0x7f366c09d6f0 0x7f366c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:23.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.163+0000 7f3670a20700 1 -- 172.21.15.3:0/2971985117 >> v1:172.21.15.3:6789/0 conn(0x7f366c09e000 legacy=0x7f366c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:23.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.163+0000 7f3670a20700 1 -- 172.21.15.3:0/2971985117 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f366c09a500 con 0x7f366c09e440 2023-10-03T10:12:23.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.163+0000 7f366bfff700 1 --2- 172.21.15.3:0/2971985117 >> v2:172.21.15.3:3300/0 conn(0x7f366c09d6f0 0x7f366c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:23.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.164+0000 7f3670a20700 1 --2- 172.21.15.3:0/2971985117 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f366c09e440 0x7f366c134100 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3664009dc0 tx=0x7f366400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55f022eab94b41a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:23.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.164+0000 7f366affd700 1 -- 172.21.15.3:0/2971985117 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366400e070 con 0x7f366c09e440 2023-10-03T10:12:23.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.164+0000 7f366affd700 1 -- 172.21.15.3:0/2971985117 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366400bae0 con 0x7f366c09e440 2023-10-03T10:12:23.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.164+0000 7f366affd700 1 -- 172.21.15.3:0/2971985117 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366400be90 con 0x7f366c09e440 2023-10-03T10:12:23.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.164+0000 7f3672483700 1 -- 172.21.15.3:0/2971985117 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f366c09e440 msgr2=0x7f366c134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:23.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.164+0000 7f3672483700 1 --2- 172.21.15.3:0/2971985117 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f366c09e440 0x7f366c134100 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3664009dc0 tx=0x7f366400b330 comp rx=0 tx=0).stop 2023-10-03T10:12:23.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.164+0000 7f3672483700 1 -- 172.21.15.3:0/2971985117 shutdown_connections 2023-10-03T10:12:23.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.164+0000 7f3672483700 1 --2- 172.21.15.3:0/2971985117 >> v2:172.21.15.3:3300/0 conn(0x7f366c09d6f0 0x7f366c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:23.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.164+0000 7f3672483700 1 --2- 172.21.15.3:0/2971985117 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f366c09e440 0x7f366c134100 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:23.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.164+0000 7f3672483700 1 -- 172.21.15.3:0/2971985117 >> 172.21.15.3:0/2971985117 conn(0x7f366c066f50 msgr2=0x7f366c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:23.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.164+0000 7f3672483700 1 -- 172.21.15.3:0/2971985117 shutdown_connections 2023-10-03T10:12:23.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.164+0000 7f3672483700 1 -- 172.21.15.3:0/2971985117 wait complete. 2023-10-03T10:12:23.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.165+0000 7f3672483700 1 Processor -- start 2023-10-03T10:12:23.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.166+0000 7f3672483700 1 -- start start 2023-10-03T10:12:23.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.166+0000 7f3672483700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f366c09d6f0 0x7f366c1a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:23.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.166+0000 7f3672483700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f366c09e000 0x7f366c1a3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:23.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.166+0000 7f3672483700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f366c09e440 0x7f366c1a40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:23.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.166+0000 7f3672483700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366c09c780 con 0x7f366c09d6f0 2023-10-03T10:12:23.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.166+0000 7f3672483700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f366c1a57d0 con 0x7f366c09e440 2023-10-03T10:12:23.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.166+0000 7f3672483700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f366c1a6e00 con 0x7f366c09e000 2023-10-03T10:12:23.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.166+0000 7f366b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f366c09e000 0x7f366c1a3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:23.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.166+0000 7f366bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f366c09d6f0 0x7f366c1a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:23.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.166+0000 7f366bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f366c09d6f0 0x7f366c1a3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39548/0 (socket says 172.21.15.3:39548) 2023-10-03T10:12:23.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.166+0000 7f366bfff700 1 -- 172.21.15.3:0/3388664734 learned_addr learned my addr 172.21.15.3:0/3388664734 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:23.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.166+0000 7f366b7fe700 1 --2- 172.21.15.3:0/3388664734 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f366c09e000 0x7f366c1a3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:49588/0 (socket says 172.21.15.3:49588) 2023-10-03T10:12:23.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.166+0000 7f3670a20700 1 --2- 172.21.15.3:0/3388664734 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f366c09e440 0x7f366c1a40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:23.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.166+0000 7f366b7fe700 1 -- 172.21.15.3:0/3388664734 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f366c09e440 msgr2=0x7f366c1a40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:23.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.166+0000 7f366b7fe700 1 --2- 172.21.15.3:0/3388664734 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f366c09e440 0x7f366c1a40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:23.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.166+0000 7f366b7fe700 1 -- 172.21.15.3:0/3388664734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f366c09d6f0 msgr2=0x7f366c1a3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:23.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.166+0000 7f366b7fe700 1 --2- 172.21.15.3:0/3388664734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f366c09d6f0 0x7f366c1a3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:23.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.167+0000 7f366b7fe700 1 -- 172.21.15.3:0/3388664734 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365c0094a0 con 0x7f366c09e000 2023-10-03T10:12:23.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.167+0000 7f366bfff700 1 --2- 172.21.15.3:0/3388664734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f366c09d6f0 0x7f366c1a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:23.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.167+0000 7f366b7fe700 1 --2- 172.21.15.3:0/3388664734 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f366c09e000 0x7f366c1a3b90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f366000c490 tx=0x7f366000c7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:23.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.167+0000 7f3668ff9700 1 -- 172.21.15.3:0/3388664734 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3660005d70 con 0x7f366c09e000 2023-10-03T10:12:23.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.167+0000 7f3668ff9700 1 -- 172.21.15.3:0/3388664734 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36600070d0 con 0x7f366c09e000 2023-10-03T10:12:23.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.167+0000 7f3672483700 1 -- 172.21.15.3:0/3388664734 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3664009a70 con 0x7f366c09e000 2023-10-03T10:12:23.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.167+0000 7f3668ff9700 1 -- 172.21.15.3:0/3388664734 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3660007440 con 0x7f366c09e000 2023-10-03T10:12:23.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.167+0000 7f3672483700 1 -- 172.21.15.3:0/3388664734 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f366c1a6b40 con 0x7f366c09e000 2023-10-03T10:12:23.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.168+0000 7f3668ff9700 1 -- 172.21.15.3:0/3388664734 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f36600075a0 con 0x7f366c09e000 2023-10-03T10:12:23.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.168+0000 7f3668ff9700 1 -- 172.21.15.3:0/3388664734 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4078+0+0 (secure 0 0 0) 0x7f366003ed10 con 0x7f366c09e000 2023-10-03T10:12:23.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.168+0000 7f3672483700 1 -- 172.21.15.3:0/3388664734 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f366c074ea0 con 0x7f366c09e000 2023-10-03T10:12:23.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.173+0000 7f3668ff9700 1 -- 172.21.15.3:0/3388664734 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3660017020 con 0x7f366c09e000 2023-10-03T10:12:23.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.396+0000 7f3672483700 1 -- 172.21.15.3:0/3388664734 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd down", "ids": ["0"]} v 0) v1 -- 0x7f366c09f420 con 0x7f366c09e000 2023-10-03T10:12:23.397 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-10-03T10:12:23.397+0000 7ff45f291700 -1 mon.a@0(leader).osd e24 definitely_dead 0 2023-10-03T10:12:23.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.397+0000 7f3668ff9700 1 -- 172.21.15.3:0/3388664734 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["0"]}]=0 osd.0 is already down. v24) v1 ==== 93+0+0 (secure 0 0 0) 0x7f3660018af0 con 0x7f366c09e000 2023-10-03T10:12:23.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.399+0000 7f3672483700 1 -- 172.21.15.3:0/3388664734 >> v1:172.21.15.3:6816/127025 conn(0x7f365802f510 legacy=0x7f36580319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:23.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.399+0000 7f3672483700 1 -- 172.21.15.3:0/3388664734 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f366c09e000 msgr2=0x7f366c1a3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:23.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.399+0000 7f3672483700 1 --2- 172.21.15.3:0/3388664734 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f366c09e000 0x7f366c1a3b90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f366000c490 tx=0x7f366000c7a0 comp rx=0 tx=0).stop 2023-10-03T10:12:23.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.399+0000 7f3672483700 1 -- 172.21.15.3:0/3388664734 shutdown_connections 2023-10-03T10:12:23.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.399+0000 7f3672483700 1 --2- 172.21.15.3:0/3388664734 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f366c09e440 0x7f366c1a40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:23.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.400+0000 7f3672483700 1 --2- 172.21.15.3:0/3388664734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f366c09d6f0 0x7f366c1a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:23.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.400+0000 7f3672483700 1 --2- 172.21.15.3:0/3388664734 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f366c09e000 0x7f366c1a3b90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:23.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.400+0000 7f3672483700 1 -- 172.21.15.3:0/3388664734 >> 172.21.15.3:0/3388664734 conn(0x7f366c066f50 msgr2=0x7f366c1bbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:23.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.400+0000 7f3672483700 1 -- 172.21.15.3:0/3388664734 shutdown_connections 2023-10-03T10:12:23.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.400+0000 7f3672483700 1 -- 172.21.15.3:0/3388664734 wait complete. 2023-10-03T10:12:23.401 INFO:teuthology.orchestra.run.smithi003.stderr:osd.0 is already down. 2023-10-03T10:12:23.413 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-03T10:12:23.414 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-03T10:12:23.416 INFO:tasks.ceph.osd.0:Started 2023-10-03T10:12:23.416 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-03T10:12:23.416 INFO:teuthology.orchestra.run:waiting for 300 2023-10-03T10:12:23.456 INFO:tasks.ceph.osd.1:Stopped 2023-10-03T10:12:23.456 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 1 2023-10-03T10:12:23.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.636+0000 7f3c00584700 1 Processor -- start 2023-10-03T10:12:23.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.636+0000 7f3c00584700 1 -- start start 2023-10-03T10:12:23.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.637+0000 7f3c00584700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3bf8133a80 0x7f3bf8133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:23.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.637+0000 7f3c00584700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3bf812ff20 0x7f3bf8130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:23.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.637+0000 7f3c00584700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf806bc00 con 0x7f3bf8130870 2023-10-03T10:12:23.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.637+0000 7f3c00584700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf806c570 con 0x7f3bf8133a80 2023-10-03T10:12:23.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.637+0000 7f3c00584700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf80a0040 con 0x7f3bf812ff20 2023-10-03T10:12:23.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.637+0000 7f3bfdb1f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3bf812ff20 0x7f3bf8130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:23.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.637+0000 7f3bfeb21700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3bf8130870 0x7f3bf813a840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40758/0 (socket says 172.21.15.3:40758) 2023-10-03T10:12:23.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.637+0000 7f3bfe320700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3bf8133a80 0x7f3bf8133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:23.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.637+0000 7f3bfeb21700 1 -- 172.21.15.3:0/3169653585 learned_addr learned my addr 172.21.15.3:0/3169653585 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:23.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.637+0000 7f3bfd31e700 1 -- 172.21.15.3:0/3169653585 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2104842958 0 0) 0x7f3bf806bc00 con 0x7f3bf8130870 2023-10-03T10:12:23.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.637+0000 7f3bfd31e700 1 -- 172.21.15.3:0/3169653585 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3be40035b0 con 0x7f3bf8130870 2023-10-03T10:12:23.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.637+0000 7f3bfdb1f700 1 -- 172.21.15.3:0/3169653585 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3bf8133a80 msgr2=0x7f3bf8133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:23.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.637+0000 7f3bfdb1f700 1 --2- 172.21.15.3:0/3169653585 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3bf8133a80 0x7f3bf8133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:23.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.637+0000 7f3bfdb1f700 1 -- 172.21.15.3:0/3169653585 >> v1:172.21.15.3:6789/0 conn(0x7f3bf8130870 legacy=0x7f3bf813a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:23.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.637+0000 7f3bfdb1f700 1 -- 172.21.15.3:0/3169653585 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf8133860 con 0x7f3bf812ff20 2023-10-03T10:12:23.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.637+0000 7f3bfe320700 1 --2- 172.21.15.3:0/3169653585 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3bf8133a80 0x7f3bf8133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:23.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.637+0000 7f3bfdb1f700 1 --2- 172.21.15.3:0/3169653585 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3bf812ff20 0x7f3bf8130330 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4009870 tx=0x7f3bf4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfd956d52c452c32 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:23.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.638+0000 7f3bfd31e700 1 -- 172.21.15.3:0/3169653585 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4011830 con 0x7f3bf812ff20 2023-10-03T10:12:23.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.638+0000 7f3bfd31e700 1 -- 172.21.15.3:0/3169653585 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf4011990 con 0x7f3bf812ff20 2023-10-03T10:12:23.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.638+0000 7f3bfd31e700 1 -- 172.21.15.3:0/3169653585 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4011d20 con 0x7f3bf812ff20 2023-10-03T10:12:23.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.638+0000 7f3c00584700 1 -- 172.21.15.3:0/3169653585 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3bf812ff20 msgr2=0x7f3bf8130330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:23.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.638+0000 7f3c00584700 1 --2- 172.21.15.3:0/3169653585 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3bf812ff20 0x7f3bf8130330 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4009870 tx=0x7f3bf4009b80 comp rx=0 tx=0).stop 2023-10-03T10:12:23.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.638+0000 7f3c00584700 1 -- 172.21.15.3:0/3169653585 shutdown_connections 2023-10-03T10:12:23.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.638+0000 7f3c00584700 1 --2- 172.21.15.3:0/3169653585 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3bf8133a80 0x7f3bf8133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:23.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.638+0000 7f3c00584700 1 --2- 172.21.15.3:0/3169653585 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3bf812ff20 0x7f3bf8130330 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:23.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.638+0000 7f3c00584700 1 -- 172.21.15.3:0/3169653585 >> 172.21.15.3:0/3169653585 conn(0x7f3bf80a4210 msgr2=0x7f3bf80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:23.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.638+0000 7f3c00584700 1 -- 172.21.15.3:0/3169653585 shutdown_connections 2023-10-03T10:12:23.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.638+0000 7f3c00584700 1 -- 172.21.15.3:0/3169653585 wait complete. 2023-10-03T10:12:23.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.640+0000 7f3c00584700 1 Processor -- start 2023-10-03T10:12:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.640+0000 7f3c00584700 1 -- start start 2023-10-03T10:12:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.640+0000 7f3c00584700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3bf812ff20 0x7f3bf809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.640+0000 7f3c00584700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3bf8130870 0x7f3bf809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.640+0000 7f3c00584700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bf8133a80 0x7f3bf809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.640+0000 7f3c00584700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf812d490 con 0x7f3bf8133a80 2023-10-03T10:12:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.640+0000 7f3c00584700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf80a1f30 con 0x7f3bf8130870 2023-10-03T10:12:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.640+0000 7f3c00584700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf811fe10 con 0x7f3bf812ff20 2023-10-03T10:12:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.640+0000 7f3bfe320700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3bf812ff20 0x7f3bf809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.641+0000 7f3bfe320700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3bf812ff20 0x7f3bf809c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:49600/0 (socket says 172.21.15.3:49600) 2023-10-03T10:12:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.641+0000 7f3bfe320700 1 -- 172.21.15.3:0/13931394 learned_addr learned my addr 172.21.15.3:0/13931394 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.641+0000 7f3bfdb1f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3bf8130870 0x7f3bf809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.641+0000 7f3bfeb21700 1 --2- 172.21.15.3:0/13931394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bf8133a80 0x7f3bf809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:23.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.641+0000 7f3bfdb1f700 1 -- 172.21.15.3:0/13931394 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3bf812ff20 msgr2=0x7f3bf809c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:23.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.641+0000 7f3bfdb1f700 1 --2- 172.21.15.3:0/13931394 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3bf812ff20 0x7f3bf809c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:23.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.641+0000 7f3bfdb1f700 1 -- 172.21.15.3:0/13931394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bf8133a80 msgr2=0x7f3bf809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:23.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.641+0000 7f3bfdb1f700 1 --2- 172.21.15.3:0/13931394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bf8133a80 0x7f3bf809cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:23.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.641+0000 7f3bfdb1f700 1 -- 172.21.15.3:0/13931394 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bec0094a0 con 0x7f3bf8130870 2023-10-03T10:12:23.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.641+0000 7f3bfe320700 1 --2- 172.21.15.3:0/13931394 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3bf812ff20 0x7f3bf809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:23.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.641+0000 7f3bfdb1f700 1 --2- 172.21.15.3:0/13931394 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3bf8130870 0x7f3bf809c9c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4016000 tx=0x7f3bf4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:23.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.641+0000 7f3beaffd700 1 -- 172.21.15.3:0/13931394 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4004450 con 0x7f3bf8130870 2023-10-03T10:12:23.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.641+0000 7f3beaffd700 1 -- 172.21.15.3:0/13931394 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf401f070 con 0x7f3bf8130870 2023-10-03T10:12:23.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.641+0000 7f3c00584700 1 -- 172.21.15.3:0/13931394 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf40094a0 con 0x7f3bf8130870 2023-10-03T10:12:23.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.641+0000 7f3beaffd700 1 -- 172.21.15.3:0/13931394 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf40038e0 con 0x7f3bf8130870 2023-10-03T10:12:23.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.641+0000 7f3c00584700 1 -- 172.21.15.3:0/13931394 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf812db20 con 0x7f3bf8130870 2023-10-03T10:12:23.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.642+0000 7f3beaffd700 1 -- 172.21.15.3:0/13931394 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f3bf4003a40 con 0x7f3bf8130870 2023-10-03T10:12:23.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.642+0000 7f3c00584700 1 -- 172.21.15.3:0/13931394 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf811fc60 con 0x7f3bf8130870 2023-10-03T10:12:23.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.642+0000 7f3beaffd700 1 -- 172.21.15.3:0/13931394 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 4078+0+0 (secure 0 0 0) 0x7f3bf4021070 con 0x7f3bf8130870 2023-10-03T10:12:23.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.648+0000 7f3beaffd700 1 -- 172.21.15.3:0/13931394 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3bf40253e0 con 0x7f3bf8130870 2023-10-03T10:12:23.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:23.865+0000 7f3c00584700 1 -- 172.21.15.3:0/13931394 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd down", "ids": ["1"]} v 0) v1 -- 0x7f3bf80a31d0 con 0x7f3bf8130870 2023-10-03T10:12:23.866 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-10-03T10:12:23.866+0000 7ff45f291700 -1 mon.a@0(leader).osd e24 definitely_dead 0 2023-10-03T10:12:24.019 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-10-03T10:12:24.019+0000 7f79182e3680 -1 Falling back to public interface 2023-10-03T10:12:24.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.207+0000 7f3beaffd700 1 -- 172.21.15.3:0/13931394 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["1"]}]=0 marked down osd.1. v25) v1 ==== 89+0+0 (secure 0 0 0) 0x7f3bf4010b70 con 0x7f3bf8130870 2023-10-03T10:12:24.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.212+0000 7f3c00584700 1 -- 172.21.15.3:0/13931394 >> v1:172.21.15.3:6816/127025 conn(0x7f3bdc02f510 legacy=0x7f3bdc0319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:24.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.212+0000 7f3c00584700 1 -- 172.21.15.3:0/13931394 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3bf8130870 msgr2=0x7f3bf809c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:24.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.212+0000 7f3c00584700 1 --2- 172.21.15.3:0/13931394 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3bf8130870 0x7f3bf809c9c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4016000 tx=0x7f3bf4004750 comp rx=0 tx=0).stop 2023-10-03T10:12:24.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.214+0000 7f3c00584700 1 -- 172.21.15.3:0/13931394 shutdown_connections 2023-10-03T10:12:24.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.214+0000 7f3c00584700 1 --2- 172.21.15.3:0/13931394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bf8133a80 0x7f3bf809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:24.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.214+0000 7f3c00584700 1 --2- 172.21.15.3:0/13931394 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3bf812ff20 0x7f3bf809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:24.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.214+0000 7f3c00584700 1 --2- 172.21.15.3:0/13931394 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3bf8130870 0x7f3bf809c9c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:24.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.214+0000 7f3c00584700 1 -- 172.21.15.3:0/13931394 >> 172.21.15.3:0/13931394 conn(0x7f3bf80a4210 msgr2=0x7f3bf809e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:24.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.215+0000 7f3c00584700 1 -- 172.21.15.3:0/13931394 shutdown_connections 2023-10-03T10:12:24.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.216+0000 7f3c00584700 1 -- 172.21.15.3:0/13931394 wait complete. 2023-10-03T10:12:24.219 INFO:teuthology.orchestra.run.smithi003.stderr:marked down osd.1. 2023-10-03T10:12:24.235 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-03T10:12:24.236 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-03T10:12:24.238 INFO:tasks.ceph.osd.1:Started 2023-10-03T10:12:24.238 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-03T10:12:24.238 INFO:teuthology.orchestra.run:waiting for 300 2023-10-03T10:12:24.275 INFO:tasks.ceph.osd.2:Stopped 2023-10-03T10:12:24.275 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 2 2023-10-03T10:12:24.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.453+0000 7f452acd0700 1 Processor -- start 2023-10-03T10:12:24.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.453+0000 7f452acd0700 1 -- start start 2023-10-03T10:12:24.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.453+0000 7f452acd0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f452409d6f0 0x7f452409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:24.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.453+0000 7f452acd0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f452409e000 0x7f45241339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:24.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.453+0000 7f452acd0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f452409b120 con 0x7f452409e410 2023-10-03T10:12:24.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.453+0000 7f452acd0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f452406b460 con 0x7f452409d6f0 2023-10-03T10:12:24.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.453+0000 7f452acd0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452409a7e0 con 0x7f452409e000 2023-10-03T10:12:24.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.453+0000 7f4528a6c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f452409d6f0 0x7f452409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:24.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.453+0000 7f4523fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f452409e000 0x7f45241339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:24.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.453+0000 7f452926d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f452409e410 0x7f4524133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40764/0 (socket says 172.21.15.3:40764) 2023-10-03T10:12:24.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.453+0000 7f452926d700 1 -- 172.21.15.3:0/1846768955 learned_addr learned my addr 172.21.15.3:0/1846768955 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:24.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.454+0000 7f4528a6c700 1 -- 172.21.15.3:0/1846768955 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f452409e000 msgr2=0x7f45241339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:24.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.454+0000 7f4528a6c700 1 --2- 172.21.15.3:0/1846768955 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f452409e000 0x7f45241339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:24.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.454+0000 7f4528a6c700 1 -- 172.21.15.3:0/1846768955 >> v1:172.21.15.3:6789/0 conn(0x7f452409e410 legacy=0x7f4524133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:24.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.454+0000 7f4528a6c700 1 -- 172.21.15.3:0/1846768955 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452409a500 con 0x7f452409d6f0 2023-10-03T10:12:24.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.454+0000 7f45237fe700 1 -- 172.21.15.3:0/1846768955 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1236686127 0 0) 0x7f452409b120 con 0x7f452409e410 2023-10-03T10:12:24.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.454+0000 7f4523fff700 1 --2- 172.21.15.3:0/1846768955 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f452409e000 0x7f45241339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:24.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.454+0000 7f4528a6c700 1 --2- 172.21.15.3:0/1846768955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f452409d6f0 0x7f452409dac0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4514009840 tx=0x7f4514009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44ffcb82d748f0b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:24.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.454+0000 7f45237fe700 1 -- 172.21.15.3:0/1846768955 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4514011830 con 0x7f452409d6f0 2023-10-03T10:12:24.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.454+0000 7f45237fe700 1 -- 172.21.15.3:0/1846768955 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4514011990 con 0x7f452409d6f0 2023-10-03T10:12:24.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.454+0000 7f45237fe700 1 -- 172.21.15.3:0/1846768955 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4514011d20 con 0x7f452409d6f0 2023-10-03T10:12:24.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.455+0000 7f452acd0700 1 -- 172.21.15.3:0/1846768955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f452409d6f0 msgr2=0x7f452409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:24.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.455+0000 7f452acd0700 1 --2- 172.21.15.3:0/1846768955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f452409d6f0 0x7f452409dac0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4514009840 tx=0x7f4514009b50 comp rx=0 tx=0).stop 2023-10-03T10:12:24.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.455+0000 7f452acd0700 1 -- 172.21.15.3:0/1846768955 shutdown_connections 2023-10-03T10:12:24.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.455+0000 7f452acd0700 1 --2- 172.21.15.3:0/1846768955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f452409d6f0 0x7f452409dac0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:24.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.455+0000 7f452acd0700 1 --2- 172.21.15.3:0/1846768955 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f452409e000 0x7f45241339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:24.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.455+0000 7f452acd0700 1 -- 172.21.15.3:0/1846768955 >> 172.21.15.3:0/1846768955 conn(0x7f4524066f50 msgr2=0x7f452409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:24.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.455+0000 7f452acd0700 1 -- 172.21.15.3:0/1846768955 shutdown_connections 2023-10-03T10:12:24.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.455+0000 7f452acd0700 1 -- 172.21.15.3:0/1846768955 wait complete. 2023-10-03T10:12:24.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.456+0000 7f452acd0700 1 Processor -- start 2023-10-03T10:12:24.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.456+0000 7f452acd0700 1 -- start start 2023-10-03T10:12:24.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.456+0000 7f452acd0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f452409d6f0 0x7f45241c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:24.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.456+0000 7f452acd0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f452409e000 0x7f45241c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:24.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.456+0000 7f452acd0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f452409e410 0x7f45241c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:24.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.456+0000 7f452acd0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45241a7680 con 0x7f452409d6f0 2023-10-03T10:12:24.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.456+0000 7f452acd0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452409c710 con 0x7f452409e410 2023-10-03T10:12:24.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.456+0000 7f452acd0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f45240a2ec0 con 0x7f452409e000 2023-10-03T10:12:24.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.457+0000 7f4528a6c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f452409d6f0 0x7f45241c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:24.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.457+0000 7f452926d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f452409e410 0x7f45241c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:24.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.457+0000 7f4528a6c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f452409d6f0 0x7f45241c81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39594/0 (socket says 172.21.15.3:39594) 2023-10-03T10:12:24.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.457+0000 7f4528a6c700 1 -- 172.21.15.3:0/1517211371 learned_addr learned my addr 172.21.15.3:0/1517211371 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:24.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.457+0000 7f4523fff700 1 --2- 172.21.15.3:0/1517211371 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f452409e000 0x7f45241c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:24.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.457+0000 7f4523fff700 1 -- 172.21.15.3:0/1517211371 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f452409e410 msgr2=0x7f45241c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:24.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.457+0000 7f4523fff700 1 --2- 172.21.15.3:0/1517211371 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f452409e410 0x7f45241c8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:24.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.457+0000 7f4523fff700 1 -- 172.21.15.3:0/1517211371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f452409d6f0 msgr2=0x7f45241c81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:24.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.457+0000 7f4523fff700 1 --2- 172.21.15.3:0/1517211371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f452409d6f0 0x7f45241c81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:24.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.457+0000 7f4523fff700 1 -- 172.21.15.3:0/1517211371 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45180094a0 con 0x7f452409e000 2023-10-03T10:12:24.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.457+0000 7f4528a6c700 1 --2- 172.21.15.3:0/1517211371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f452409d6f0 0x7f45241c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:24.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.457+0000 7f452926d700 1 --2- 172.21.15.3:0/1517211371 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f452409e410 0x7f45241c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:24.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.457+0000 7f4523fff700 1 --2- 172.21.15.3:0/1517211371 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f452409e000 0x7f45241c8700 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f451800e9b0 tx=0x7f451800ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:24.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.457+0000 7f45217fa700 1 -- 172.21.15.3:0/1517211371 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451800ee50 con 0x7f452409e000 2023-10-03T10:12:24.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.457+0000 7f45217fa700 1 -- 172.21.15.3:0/1517211371 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451800c850 con 0x7f452409e000 2023-10-03T10:12:24.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.457+0000 7f45217fa700 1 -- 172.21.15.3:0/1517211371 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451800cc00 con 0x7f452409e000 2023-10-03T10:12:24.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.457+0000 7f452acd0700 1 -- 172.21.15.3:0/1517211371 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45140094a0 con 0x7f452409e000 2023-10-03T10:12:24.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.457+0000 7f452acd0700 1 -- 172.21.15.3:0/1517211371 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45241a8d10 con 0x7f452409e000 2023-10-03T10:12:24.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.458+0000 7f45217fa700 1 -- 172.21.15.3:0/1517211371 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f4518013470 con 0x7f452409e000 2023-10-03T10:12:24.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.459+0000 7f45217fa700 1 -- 172.21.15.3:0/1517211371 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4078+0+0 (secure 0 0 0) 0x7f451803fbb0 con 0x7f452409e000 2023-10-03T10:12:24.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.459+0000 7f452acd0700 1 -- 172.21.15.3:0/1517211371 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f452404ea50 con 0x7f452409e000 2023-10-03T10:12:24.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.464+0000 7f45217fa700 1 -- 172.21.15.3:0/1517211371 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4518021080 con 0x7f452409e000 2023-10-03T10:12:24.592 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-10-03T10:12:24.591+0000 7fb804114680 -1 Falling back to public interface 2023-10-03T10:12:24.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:24.681+0000 7f452acd0700 1 -- 172.21.15.3:0/1517211371 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd down", "ids": ["2"]} v 0) v1 -- 0x7f45241a8200 con 0x7f452409e000 2023-10-03T10:12:24.682 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-10-03T10:12:24.682+0000 7ff45f291700 -1 mon.a@0(leader).osd e25 definitely_dead 0 2023-10-03T10:12:25.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.209+0000 7f45217fa700 1 -- 172.21.15.3:0/1517211371 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["2"]}]=0 marked down osd.2. v26) v1 ==== 89+0+0 (secure 0 0 0) 0x7f45241a8200 con 0x7f452409e000 2023-10-03T10:12:25.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.215+0000 7f452acd0700 1 -- 172.21.15.3:0/1517211371 >> v1:172.21.15.3:6816/127025 conn(0x7f451002f510 legacy=0x7f45100319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:25.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.215+0000 7f452acd0700 1 -- 172.21.15.3:0/1517211371 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f452409e000 msgr2=0x7f45241c8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:25.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.215+0000 7f452acd0700 1 --2- 172.21.15.3:0/1517211371 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f452409e000 0x7f45241c8700 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f451800e9b0 tx=0x7f451800ecc0 comp rx=0 tx=0).stop 2023-10-03T10:12:25.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.216+0000 7f452acd0700 1 -- 172.21.15.3:0/1517211371 shutdown_connections 2023-10-03T10:12:25.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.216+0000 7f452acd0700 1 --2- 172.21.15.3:0/1517211371 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f452409e410 0x7f45241c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:25.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.216+0000 7f452acd0700 1 --2- 172.21.15.3:0/1517211371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f452409d6f0 0x7f45241c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:25.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.216+0000 7f452acd0700 1 --2- 172.21.15.3:0/1517211371 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f452409e000 0x7f45241c8700 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:25.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.216+0000 7f452acd0700 1 -- 172.21.15.3:0/1517211371 >> 172.21.15.3:0/1517211371 conn(0x7f4524066f50 msgr2=0x7f45241bba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:25.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.217+0000 7f452acd0700 1 -- 172.21.15.3:0/1517211371 shutdown_connections 2023-10-03T10:12:25.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.217+0000 7f452acd0700 1 -- 172.21.15.3:0/1517211371 wait complete. 2023-10-03T10:12:25.218 INFO:teuthology.orchestra.run.smithi003.stderr:marked down osd.2. 2023-10-03T10:12:25.235 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-03T10:12:25.235 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-03T10:12:25.237 INFO:tasks.ceph.osd.2:Started 2023-10-03T10:12:25.237 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-03T10:12:25.237 INFO:teuthology.orchestra.run:waiting for 300 2023-10-03T10:12:25.266 INFO:tasks.ceph.osd.3:Stopped 2023-10-03T10:12:25.266 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 3 2023-10-03T10:12:25.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.444+0000 7f8412ebf700 1 Processor -- start 2023-10-03T10:12:25.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.444+0000 7f8412ebf700 1 -- start start 2023-10-03T10:12:25.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.444+0000 7f8412ebf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f840c09d6f0 0x7f840c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:25.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.444+0000 7f8412ebf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f840c09e000 0x7f840c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:25.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.444+0000 7f8412ebf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f840c09e410 0x7f840c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:25.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.444+0000 7f8412ebf700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c09f520 con 0x7f840c09e410 2023-10-03T10:12:25.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.444+0000 7f8412ebf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c09f130 con 0x7f840c09e000 2023-10-03T10:12:25.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.444+0000 7f8412ebf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f840c09e930 con 0x7f840c09d6f0 2023-10-03T10:12:25.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.444+0000 7f8410c5b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f840c09d6f0 0x7f840c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:25.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.444+0000 7f840bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f840c09e000 0x7f840c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:25.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.444+0000 7f8410c5b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f840c09d6f0 0x7f840c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:49642/0 (socket says 172.21.15.3:49642) 2023-10-03T10:12:25.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.445+0000 7f8410c5b700 1 -- 172.21.15.3:0/412301614 learned_addr learned my addr 172.21.15.3:0/412301614 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:25.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.445+0000 7f841145c700 1 --2- 172.21.15.3:0/412301614 >> v2:172.21.15.3:3300/0 conn(0x7f840c09e410 0x7f840c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:25.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.445+0000 7f840bfff700 1 -- 172.21.15.3:0/412301614 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f840c09d6f0 msgr2=0x7f840c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:25.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.445+0000 7f840bfff700 1 --2- 172.21.15.3:0/412301614 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f840c09d6f0 0x7f840c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:25.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.445+0000 7f840bfff700 1 -- 172.21.15.3:0/412301614 >> v2:172.21.15.3:3300/0 conn(0x7f840c09e410 msgr2=0x7f840c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:25.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.445+0000 7f840bfff700 1 --2- 172.21.15.3:0/412301614 >> v2:172.21.15.3:3300/0 conn(0x7f840c09e410 0x7f840c130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:25.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.445+0000 7f840bfff700 1 -- 172.21.15.3:0/412301614 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840c09b2d0 con 0x7f840c09e000 2023-10-03T10:12:25.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.445+0000 7f840bfff700 1 --2- 172.21.15.3:0/412301614 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f840c09e000 0x7f840c130660 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f83f8009870 tx=0x7f83f8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d220451140cbca9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:25.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.445+0000 7f840b7fe700 1 -- 172.21.15.3:0/412301614 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f8011830 con 0x7f840c09e000 2023-10-03T10:12:25.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.445+0000 7f840b7fe700 1 -- 172.21.15.3:0/412301614 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f8011990 con 0x7f840c09e000 2023-10-03T10:12:25.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.445+0000 7f840b7fe700 1 -- 172.21.15.3:0/412301614 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f8011d20 con 0x7f840c09e000 2023-10-03T10:12:25.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.446+0000 7f8412ebf700 1 -- 172.21.15.3:0/412301614 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f840c09e000 msgr2=0x7f840c130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:25.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.446+0000 7f8412ebf700 1 --2- 172.21.15.3:0/412301614 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f840c09e000 0x7f840c130660 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f83f8009870 tx=0x7f83f8009b80 comp rx=0 tx=0).stop 2023-10-03T10:12:25.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.446+0000 7f8412ebf700 1 -- 172.21.15.3:0/412301614 shutdown_connections 2023-10-03T10:12:25.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.446+0000 7f8412ebf700 1 --2- 172.21.15.3:0/412301614 >> v2:172.21.15.3:3300/0 conn(0x7f840c09e410 0x7f840c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:25.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.446+0000 7f8412ebf700 1 --2- 172.21.15.3:0/412301614 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f840c09d6f0 0x7f840c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:25.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.446+0000 7f8412ebf700 1 --2- 172.21.15.3:0/412301614 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f840c09e000 0x7f840c130660 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:25.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.446+0000 7f8412ebf700 1 -- 172.21.15.3:0/412301614 >> 172.21.15.3:0/412301614 conn(0x7f840c066f50 msgr2=0x7f840c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:25.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.446+0000 7f8412ebf700 1 -- 172.21.15.3:0/412301614 shutdown_connections 2023-10-03T10:12:25.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.446+0000 7f8412ebf700 1 -- 172.21.15.3:0/412301614 wait complete. 2023-10-03T10:12:25.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.447+0000 7f8412ebf700 1 Processor -- start 2023-10-03T10:12:25.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.447+0000 7f8412ebf700 1 -- start start 2023-10-03T10:12:25.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.448+0000 7f8412ebf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f840c09d6f0 0x7f840c11eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:25.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.448+0000 7f8412ebf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f840c09e000 0x7f840c11f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:25.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.448+0000 7f8412ebf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f840c09e410 0x7f840c11fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:25.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.448+0000 7f8412ebf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c122b10 con 0x7f840c09d6f0 2023-10-03T10:12:25.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.448+0000 7f8412ebf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c121a90 con 0x7f840c09e410 2023-10-03T10:12:25.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.448+0000 7f8412ebf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f840c122570 con 0x7f840c09e000 2023-10-03T10:12:25.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.448+0000 7f8410c5b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f840c09d6f0 0x7f840c11eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:25.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.448+0000 7f8410c5b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f840c09d6f0 0x7f840c11eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39606/0 (socket says 172.21.15.3:39606) 2023-10-03T10:12:25.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.448+0000 7f8410c5b700 1 -- 172.21.15.3:0/23448250 learned_addr learned my addr 172.21.15.3:0/23448250 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:25.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.448+0000 7f841145c700 1 --2- 172.21.15.3:0/23448250 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f840c09e410 0x7f840c11fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:25.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.448+0000 7f840bfff700 1 --2- 172.21.15.3:0/23448250 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f840c09e000 0x7f840c11f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:25.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.449+0000 7f8410c5b700 1 -- 172.21.15.3:0/23448250 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f840c09e000 msgr2=0x7f840c11f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:25.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.449+0000 7f8410c5b700 1 --2- 172.21.15.3:0/23448250 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f840c09e000 0x7f840c11f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:25.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.449+0000 7f8410c5b700 1 -- 172.21.15.3:0/23448250 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f840c09e410 msgr2=0x7f840c11fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:25.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.449+0000 7f8410c5b700 1 --2- 172.21.15.3:0/23448250 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f840c09e410 0x7f840c11fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:25.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.449+0000 7f8410c5b700 1 -- 172.21.15.3:0/23448250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f80094a0 con 0x7f840c09d6f0 2023-10-03T10:12:25.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.449+0000 7f841145c700 1 --2- 172.21.15.3:0/23448250 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f840c09e410 0x7f840c11fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:25.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.449+0000 7f8410c5b700 1 --2- 172.21.15.3:0/23448250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f840c09d6f0 0x7f840c11eff0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f840000e990 tx=0x7f840000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:25.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.449+0000 7f84097fa700 1 -- 172.21.15.3:0/23448250 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8400013680 con 0x7f840c09d6f0 2023-10-03T10:12:25.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.449+0000 7f84097fa700 1 -- 172.21.15.3:0/23448250 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84000137e0 con 0x7f840c09d6f0 2023-10-03T10:12:25.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.449+0000 7f8412ebf700 1 -- 172.21.15.3:0/23448250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f840c0a2a00 con 0x7f840c09d6f0 2023-10-03T10:12:25.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.449+0000 7f84097fa700 1 -- 172.21.15.3:0/23448250 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8400013b50 con 0x7f840c09d6f0 2023-10-03T10:12:25.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.449+0000 7f8412ebf700 1 -- 172.21.15.3:0/23448250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f840c09c750 con 0x7f840c09d6f0 2023-10-03T10:12:25.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.450+0000 7f84097fa700 1 -- 172.21.15.3:0/23448250 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f8400013cb0 con 0x7f840c09d6f0 2023-10-03T10:12:25.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.450+0000 7f84097fa700 1 -- 172.21.15.3:0/23448250 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4078+0+0 (secure 0 0 0) 0x7f840003fed0 con 0x7f840c09d6f0 2023-10-03T10:12:25.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.450+0000 7f8412ebf700 1 -- 172.21.15.3:0/23448250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f840c1223c0 con 0x7f840c09d6f0 2023-10-03T10:12:25.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.456+0000 7f84097fa700 1 -- 172.21.15.3:0/23448250 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8400017080 con 0x7f840c09d6f0 2023-10-03T10:12:25.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:25.671+0000 7f8412ebf700 1 -- 172.21.15.3:0/23448250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd down", "ids": ["3"]} v 0) v1 -- 0x7f840c074ea0 con 0x7f840c09d6f0 2023-10-03T10:12:25.672 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-10-03T10:12:25.672+0000 7ff45f291700 -1 mon.a@0(leader).osd e26 definitely_dead 0 2023-10-03T10:12:25.839 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-10-03T10:12:25.839+0000 7fc1af63e680 -1 Falling back to public interface 2023-10-03T10:12:26.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.219+0000 7f84097fa700 1 -- 172.21.15.3:0/23448250 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["3"]}]=0 marked down osd.3. v27) v1 ==== 89+0+0 (secure 0 0 0) 0x7f8400021030 con 0x7f840c09d6f0 2023-10-03T10:12:26.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.222+0000 7f8412ebf700 1 -- 172.21.15.3:0/23448250 >> v1:172.21.15.3:6816/127025 conn(0x7f83f002f510 legacy=0x7f83f00319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:26.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.222+0000 7f8412ebf700 1 -- 172.21.15.3:0/23448250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f840c09d6f0 msgr2=0x7f840c11eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:26.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.222+0000 7f8412ebf700 1 --2- 172.21.15.3:0/23448250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f840c09d6f0 0x7f840c11eff0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f840000e990 tx=0x7f840000eca0 comp rx=0 tx=0).stop 2023-10-03T10:12:26.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.222+0000 7f8412ebf700 1 -- 172.21.15.3:0/23448250 shutdown_connections 2023-10-03T10:12:26.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.222+0000 7f8412ebf700 1 --2- 172.21.15.3:0/23448250 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f840c09e410 0x7f840c11fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:26.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.222+0000 7f8412ebf700 1 --2- 172.21.15.3:0/23448250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f840c09d6f0 0x7f840c11eff0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:26.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.222+0000 7f8412ebf700 1 --2- 172.21.15.3:0/23448250 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f840c09e000 0x7f840c11f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:26.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.222+0000 7f8412ebf700 1 -- 172.21.15.3:0/23448250 >> 172.21.15.3:0/23448250 conn(0x7f840c066f50 msgr2=0x7f840c119480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:26.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.222+0000 7f8412ebf700 1 -- 172.21.15.3:0/23448250 shutdown_connections 2023-10-03T10:12:26.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.222+0000 7f8412ebf700 1 -- 172.21.15.3:0/23448250 wait complete. 2023-10-03T10:12:26.224 INFO:teuthology.orchestra.run.smithi003.stderr:marked down osd.3. 2023-10-03T10:12:26.238 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-03T10:12:26.238 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-03T10:12:26.241 INFO:tasks.ceph.osd.3:Started 2023-10-03T10:12:26.241 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-03T10:12:26.241 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-03T10:12:26.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.380+0000 7f1af8790700 1 Processor -- start 2023-10-03T10:12:26.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.380+0000 7f1af8790700 1 -- start start 2023-10-03T10:12:26.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.381+0000 7f1af8790700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1af009e930 0x7f1af009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:26.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.381+0000 7f1af8790700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af009f240 0x7f1af01317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:26.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.381+0000 7f1af8790700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af0070940 con 0x7f1af009f650 2023-10-03T10:12:26.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.381+0000 7f1af8790700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1af006fd60 con 0x7f1af009e930 2023-10-03T10:12:26.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.381+0000 7f1af8790700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af006fa80 con 0x7f1af009f240 2023-10-03T10:12:26.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.381+0000 7f1af5d2b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af009f240 0x7f1af01317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:26.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.381+0000 7f1af5d2b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af009f240 0x7f1af01317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:45018/0 (socket says 172.21.15.3:45018) 2023-10-03T10:12:26.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.381+0000 7f1af5d2b700 1 -- 172.21.15.3:0/4037315148 learned_addr learned my addr 172.21.15.3:0/4037315148 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:26.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.381+0000 7f1af652c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1af009e930 0x7f1af009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:26.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.381+0000 7f1af5d2b700 1 -- 172.21.15.3:0/4037315148 >> v2:172.21.15.3:3300/0 conn(0x7f1af009e930 msgr2=0x7f1af009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:26.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.381+0000 7f1af5d2b700 1 --2- 172.21.15.3:0/4037315148 >> v2:172.21.15.3:3300/0 conn(0x7f1af009e930 0x7f1af009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:26.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.381+0000 7f1af5d2b700 1 -- 172.21.15.3:0/4037315148 >> v1:172.21.15.3:6789/0 conn(0x7f1af009f650 legacy=0x7f1af0131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:26.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.381+0000 7f1af5d2b700 1 -- 172.21.15.3:0/4037315148 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af009fb60 con 0x7f1af009f240 2023-10-03T10:12:26.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.381+0000 7f1af652c700 1 --2- 172.21.15.3:0/4037315148 >> v2:172.21.15.3:3300/0 conn(0x7f1af009e930 0x7f1af009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:26.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.381+0000 7f1af5d2b700 1 --2- 172.21.15.3:0/4037315148 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af009f240 0x7f1af01317f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1ae00098f0 tx=0x7f1ae0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3afdfeb606c3826b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:26.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.381+0000 7f1af552a700 1 -- 172.21.15.3:0/4037315148 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0011830 con 0x7f1af009f240 2023-10-03T10:12:26.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.382+0000 7f1af552a700 1 -- 172.21.15.3:0/4037315148 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae0011990 con 0x7f1af009f240 2023-10-03T10:12:26.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.382+0000 7f1af552a700 1 -- 172.21.15.3:0/4037315148 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0011d20 con 0x7f1af009f240 2023-10-03T10:12:26.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.382+0000 7f1af8790700 1 -- 172.21.15.3:0/4037315148 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af009f240 msgr2=0x7f1af01317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:26.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.382+0000 7f1af8790700 1 --2- 172.21.15.3:0/4037315148 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af009f240 0x7f1af01317f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1ae00098f0 tx=0x7f1ae0009c00 comp rx=0 tx=0).stop 2023-10-03T10:12:26.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.382+0000 7f1af8790700 1 -- 172.21.15.3:0/4037315148 shutdown_connections 2023-10-03T10:12:26.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.382+0000 7f1af8790700 1 --2- 172.21.15.3:0/4037315148 >> v2:172.21.15.3:3300/0 conn(0x7f1af009e930 0x7f1af009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:26.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.382+0000 7f1af8790700 1 --2- 172.21.15.3:0/4037315148 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af009f240 0x7f1af01317f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:26.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.382+0000 7f1af8790700 1 -- 172.21.15.3:0/4037315148 >> 172.21.15.3:0/4037315148 conn(0x7f1af00a4210 msgr2=0x7f1af00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:26.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.382+0000 7f1af8790700 1 -- 172.21.15.3:0/4037315148 shutdown_connections 2023-10-03T10:12:26.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.382+0000 7f1af8790700 1 -- 172.21.15.3:0/4037315148 wait complete. 2023-10-03T10:12:26.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.383+0000 7f1af8790700 1 Processor -- start 2023-10-03T10:12:26.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.383+0000 7f1af8790700 1 -- start start 2023-10-03T10:12:26.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.384+0000 7f1af8790700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af009e930 0x7f1af006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:26.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.384+0000 7f1af8790700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af009f240 0x7f1af006bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:26.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.384+0000 7f1af8790700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1af009f650 0x7f1af006c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:26.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.384+0000 7f1af8790700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af0071d20 con 0x7f1af009e930 2023-10-03T10:12:26.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.384+0000 7f1af8790700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af006d580 con 0x7f1af009f240 2023-10-03T10:12:26.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.384+0000 7f1af8790700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1af01ab710 con 0x7f1af009f650 2023-10-03T10:12:26.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.384+0000 7f1af5d2b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af009f240 0x7f1af006bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:26.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.384+0000 7f1af5d2b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af009f240 0x7f1af006bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:45024/0 (socket says 172.21.15.3:45024) 2023-10-03T10:12:26.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.384+0000 7f1af5d2b700 1 -- 172.21.15.3:0/1875501465 learned_addr learned my addr 172.21.15.3:0/1875501465 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:26.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.384+0000 7f1af6d2d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1af009f650 0x7f1af006c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:26.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.384+0000 7f1af652c700 1 --2- 172.21.15.3:0/1875501465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af009e930 0x7f1af006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:26.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.384+0000 7f1af5d2b700 1 -- 172.21.15.3:0/1875501465 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1af009f650 msgr2=0x7f1af006c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:26.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.384+0000 7f1af5d2b700 1 --2- 172.21.15.3:0/1875501465 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1af009f650 0x7f1af006c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:26.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.384+0000 7f1af5d2b700 1 -- 172.21.15.3:0/1875501465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af009e930 msgr2=0x7f1af006b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:26.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.384+0000 7f1af5d2b700 1 --2- 172.21.15.3:0/1875501465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af009e930 0x7f1af006b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:26.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.384+0000 7f1af5d2b700 1 -- 172.21.15.3:0/1875501465 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aec0094a0 con 0x7f1af009f240 2023-10-03T10:12:26.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.384+0000 7f1af6d2d700 1 --2- 172.21.15.3:0/1875501465 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1af009f650 0x7f1af006c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:26.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.384+0000 7f1af652c700 1 --2- 172.21.15.3:0/1875501465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af009e930 0x7f1af006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:26.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.384+0000 7f1af5d2b700 1 --2- 172.21.15.3:0/1875501465 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af009f240 0x7f1af006bdf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0000c00 tx=0x7f1ae0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:26.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.385+0000 7f1ae6ffd700 1 -- 172.21.15.3:0/1875501465 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae00036a0 con 0x7f1af009f240 2023-10-03T10:12:26.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.385+0000 7f1ae6ffd700 1 -- 172.21.15.3:0/1875501465 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae0003800 con 0x7f1af009f240 2023-10-03T10:12:26.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.385+0000 7f1af8790700 1 -- 172.21.15.3:0/1875501465 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae0009550 con 0x7f1af009f240 2023-10-03T10:12:26.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.385+0000 7f1ae6ffd700 1 -- 172.21.15.3:0/1875501465 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0003bb0 con 0x7f1af009f240 2023-10-03T10:12:26.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.385+0000 7f1af8790700 1 -- 172.21.15.3:0/1875501465 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af00a0190 con 0x7f1af009f240 2023-10-03T10:12:26.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.385+0000 7f1ae6ffd700 1 -- 172.21.15.3:0/1875501465 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f1ae0021070 con 0x7f1af009f240 2023-10-03T10:12:26.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.386+0000 7f1ae6ffd700 1 -- 172.21.15.3:0/1875501465 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4078+0+0 (secure 0 0 0) 0x7f1ae0041030 con 0x7f1af009f240 2023-10-03T10:12:26.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.386+0000 7f1af8790700 1 -- 172.21.15.3:0/1875501465 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af01a2b40 con 0x7f1af009f240 2023-10-03T10:12:26.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.391+0000 7f1ae6ffd700 1 -- 172.21.15.3:0/1875501465 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1ae0015070 con 0x7f1af009f240 2023-10-03T10:12:26.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.598+0000 7f1af8790700 1 -- 172.21.15.3:0/1875501465 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1af01aa330 con 0x7f1af009f240 2023-10-03T10:12:26.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.599+0000 7f1ae6ffd700 1 -- 172.21.15.3:0/1875501465 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+11959 (secure 0 0 0) 0x7f1ae000b3e0 con 0x7f1af009f240 2023-10-03T10:12:26.599 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:12:26.599 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","created":"2023-10-03T10:09:28.703441+0000","modified":"2023-10-03T10:12:26.212430+0000","last_up_change":"2023-10-03T10:12:26.212430+0000","last_in_change":"2023-10-03T10:09:30.156546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-03T10:09:36.623971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-03T10:09:38.210614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-03T10:09:42.695873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-03T10:09:43.115588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cdcdd930-3c5b-4486-a021-7132f90fb6e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6808","nonce":118845}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6809","nonce":118845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6811","nonce":118845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6810","nonce":118845}]},"public_addr":"172.21.15.3:6808/118845","cluster_addr":"172.21.15.3:6809/118845","heartbeat_back_addr":"172.21.15.3:6811/118845","heartbeat_front_addr":"172.21.15.3:6810/118845","state":["exists"]},{"osd":1,"uuid":"c8e58877-6762-4741-a8c1-567ebc97b708","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6804","nonce":118835}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6805","nonce":118835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6807","nonce":118835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6806","nonce":118835}]},"public_addr":"172.21.15.3:6804/118835","cluster_addr":"172.21.15.3:6805/118835","heartbeat_back_addr":"172.21.15.3:6807/118835","heartbeat_front_addr":"172.21.15.3:6806/118835","state":["exists"]},{"osd":2,"uuid":"68a2a21a-7a8e-4f82-8433-5201faa50777","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":25,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6812","nonce":118864}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6813","nonce":118864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6815","nonce":118864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6814","nonce":118864}]},"public_addr":"172.21.15.3:6812/118864","cluster_addr":"172.21.15.3:6813/118864","heartbeat_back_addr":"172.21.15.3:6815/118864","heartbeat_front_addr":"172.21.15.3:6814/118864","state":["exists"]},{"osd":3,"uuid":"c4df2903-4e1e-4208-bf77-95b9fa749241","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":26,"down_at":27,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6800","nonce":118836}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6801","nonce":118836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6803","nonce":118836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6802","nonce":118836}]},"public_addr":"172.21.15.3:6800/118836","cluster_addr":"172.21.15.3:6801/118836","heartbeat_back_addr":"172.21.15.3:6803/118836","heartbeat_front_addr":"172.21.15.3:6802/118836","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.369708+0000","dead_epoch":24},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.257825+0000","dead_epoch":25},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.337931+0000","dead_epoch":26},{"osd":3,"down_stamp":"2023-10-03T10:12:26.212430+0000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.085222+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1","mappings":[{"from":3,"to":0}]},{"pgid":"4.8","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.19","mappings":[{"from":3,"to":0}]},{"pgid":"4.2a","mappings":[{"from":1,"to":2}]},{"pgid":"4.2d","mappings":[{"from":3,"to":0}]},{"pgid":"4.33","mappings":[{"from":3,"to":0}]},{"pgid":"4.39","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/3919724564":"2023-10-04T10:12:13.305411+0000","172.21.15.3:0/2091814084":"2023-10-04T10:12:13.305411+0000","172.21.15.3:6816/118296":"2023-10-04T10:12:13.305411+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-03T10:12:26.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.601+0000 7f1af8790700 1 -- 172.21.15.3:0/1875501465 >> v1:172.21.15.3:6816/127025 conn(0x7f1ad402f510 legacy=0x7f1ad40319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:26.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.601+0000 7f1af8790700 1 -- 172.21.15.3:0/1875501465 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af009f240 msgr2=0x7f1af006bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:26.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.601+0000 7f1af8790700 1 --2- 172.21.15.3:0/1875501465 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af009f240 0x7f1af006bdf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0000c00 tx=0x7f1ae0004750 comp rx=0 tx=0).stop 2023-10-03T10:12:26.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.601+0000 7f1af8790700 1 -- 172.21.15.3:0/1875501465 shutdown_connections 2023-10-03T10:12:26.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.601+0000 7f1af8790700 1 --2- 172.21.15.3:0/1875501465 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1af009f650 0x7f1af006c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:26.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.601+0000 7f1af8790700 1 --2- 172.21.15.3:0/1875501465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af009e930 0x7f1af006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:26.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.601+0000 7f1af8790700 1 --2- 172.21.15.3:0/1875501465 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af009f240 0x7f1af006bdf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:26.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.601+0000 7f1af8790700 1 -- 172.21.15.3:0/1875501465 >> 172.21.15.3:0/1875501465 conn(0x7f1af00a4210 msgr2=0x7f1af009dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:26.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.601+0000 7f1af8790700 1 -- 172.21.15.3:0/1875501465 shutdown_connections 2023-10-03T10:12:26.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:26.601+0000 7f1af8790700 1 -- 172.21.15.3:0/1875501465 wait complete. 2023-10-03T10:12:26.844 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-10-03T10:12:26.843+0000 7f349086a680 -1 Falling back to public interface 2023-10-03T10:12:27.798 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-10-03T10:12:27.798+0000 7f79182e3680 -1 osd.0 23 log_to_monitors true 2023-10-03T10:12:28.412 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-10-03T10:12:28.412+0000 7fb804114680 -1 osd.1 23 log_to_monitors true 2023-10-03T10:12:29.188 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-10-03T10:12:29.187+0000 7f790425e700 -1 osd.0 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-03T10:12:29.419 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-10-03T10:12:29.419+0000 7fc1af63e680 -1 osd.2 25 log_to_monitors true 2023-10-03T10:12:29.616 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-03T10:12:29.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.785+0000 7f49896dc700 1 Processor -- start 2023-10-03T10:12:29.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.785+0000 7f49896dc700 1 -- start start 2023-10-03T10:12:29.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.786+0000 7f49896dc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f498412cc20 0x7f498412f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:29.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.786+0000 7f49896dc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f498412c2f0 0x7f498412fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:29.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.786+0000 7f49896dc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f498412cff0 0x7f49841303b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:29.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.786+0000 7f49896dc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f498409a510 con 0x7f498412c2f0 2023-10-03T10:12:29.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.786+0000 7f49896dc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498412c7d0 con 0x7f498412cc20 2023-10-03T10:12:29.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.786+0000 7f49896dc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f498409c190 con 0x7f498412cff0 2023-10-03T10:12:29.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.786+0000 7f4982ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f498412cc20 0x7f498412f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:29.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.786+0000 7f4982ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f498412cc20 0x7f498412f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:45034/0 (socket says 172.21.15.3:45034) 2023-10-03T10:12:29.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.786+0000 7f4982ffd700 1 -- 172.21.15.3:0/3915612699 learned_addr learned my addr 172.21.15.3:0/3915612699 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:29.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.786+0000 7f49837fe700 1 --2- 172.21.15.3:0/3915612699 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f498412cff0 0x7f49841303b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:29.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.787+0000 7f4982ffd700 1 -- 172.21.15.3:0/3915612699 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f498412cff0 msgr2=0x7f49841303b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:29.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.787+0000 7f4982ffd700 1 --2- 172.21.15.3:0/3915612699 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f498412cff0 0x7f49841303b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:29.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.787+0000 7f4982ffd700 1 -- 172.21.15.3:0/3915612699 >> v2:172.21.15.3:3300/0 conn(0x7f498412c2f0 msgr2=0x7f498412fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:29.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.787+0000 7f4982ffd700 1 --2- 172.21.15.3:0/3915612699 >> v2:172.21.15.3:3300/0 conn(0x7f498412c2f0 0x7f498412fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:29.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.787+0000 7f4982ffd700 1 -- 172.21.15.3:0/3915612699 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498409b540 con 0x7f498412cc20 2023-10-03T10:12:29.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.787+0000 7f4982ffd700 1 --2- 172.21.15.3:0/3915612699 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f498412cc20 0x7f498412f930 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f49740098f0 tx=0x7f4974009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=482eec836098f927 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:29.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.787+0000 7f4981ffb700 1 -- 172.21.15.3:0/3915612699 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497400b8a0 con 0x7f498412cc20 2023-10-03T10:12:29.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.787+0000 7f4981ffb700 1 -- 172.21.15.3:0/3915612699 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497400ba00 con 0x7f498412cc20 2023-10-03T10:12:29.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.787+0000 7f4981ffb700 1 -- 172.21.15.3:0/3915612699 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497400bd90 con 0x7f498412cc20 2023-10-03T10:12:29.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.788+0000 7f49896dc700 1 -- 172.21.15.3:0/3915612699 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f498412cc20 msgr2=0x7f498412f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:29.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.788+0000 7f49896dc700 1 --2- 172.21.15.3:0/3915612699 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f498412cc20 0x7f498412f930 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f49740098f0 tx=0x7f4974009c00 comp rx=0 tx=0).stop 2023-10-03T10:12:29.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.788+0000 7f49896dc700 1 -- 172.21.15.3:0/3915612699 shutdown_connections 2023-10-03T10:12:29.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.788+0000 7f49896dc700 1 --2- 172.21.15.3:0/3915612699 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f498412cff0 0x7f49841303b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:29.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.788+0000 7f49896dc700 1 --2- 172.21.15.3:0/3915612699 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f498412cc20 0x7f498412f930 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:29.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.788+0000 7f49896dc700 1 --2- 172.21.15.3:0/3915612699 >> v2:172.21.15.3:3300/0 conn(0x7f498412c2f0 0x7f498412fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:29.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.788+0000 7f49896dc700 1 -- 172.21.15.3:0/3915612699 >> 172.21.15.3:0/3915612699 conn(0x7f498409f470 msgr2=0x7f49840a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:29.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.788+0000 7f49896dc700 1 -- 172.21.15.3:0/3915612699 shutdown_connections 2023-10-03T10:12:29.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.788+0000 7f49896dc700 1 -- 172.21.15.3:0/3915612699 wait complete. 2023-10-03T10:12:29.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.790+0000 7f49896dc700 1 Processor -- start 2023-10-03T10:12:29.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.790+0000 7f49896dc700 1 -- start start 2023-10-03T10:12:29.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.790+0000 7f49896dc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f498412c2f0 0x7f498406f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:29.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.790+0000 7f49896dc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f498412cc20 0x7f498406fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:29.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.790+0000 7f49896dc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f498412cff0 0x7f49841c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:29.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.790+0000 7f49896dc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498406f0d0 con 0x7f498412c2f0 2023-10-03T10:12:29.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.790+0000 7f49896dc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498406ffc0 con 0x7f498412cff0 2023-10-03T10:12:29.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.790+0000 7f49896dc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4984129f10 con 0x7f498412cc20 2023-10-03T10:12:29.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.790+0000 7f4982ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f498412c2f0 0x7f498406f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:29.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.790+0000 7f4982ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f498412c2f0 0x7f498406f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39662/0 (socket says 172.21.15.3:39662) 2023-10-03T10:12:29.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.790+0000 7f4982ffd700 1 -- 172.21.15.3:0/2043534267 learned_addr learned my addr 172.21.15.3:0/2043534267 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:29.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.790+0000 7f49827fc700 1 --2- 172.21.15.3:0/2043534267 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f498412cc20 0x7f498406fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:29.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.791+0000 7f49837fe700 1 --2- 172.21.15.3:0/2043534267 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f498412cff0 0x7f49841c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:29.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.791+0000 7f49827fc700 1 -- 172.21.15.3:0/2043534267 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f498412cff0 msgr2=0x7f49841c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:29.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.791+0000 7f49827fc700 1 --2- 172.21.15.3:0/2043534267 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f498412cff0 0x7f49841c8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:29.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.791+0000 7f49827fc700 1 -- 172.21.15.3:0/2043534267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f498412c2f0 msgr2=0x7f498406f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:29.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.791+0000 7f49827fc700 1 --2- 172.21.15.3:0/2043534267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f498412c2f0 0x7f498406f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:29.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.791+0000 7f49827fc700 1 -- 172.21.15.3:0/2043534267 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4974009550 con 0x7f498412cc20 2023-10-03T10:12:29.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.791+0000 7f4982ffd700 1 --2- 172.21.15.3:0/2043534267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f498412c2f0 0x7f498406f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:29.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.791+0000 7f49837fe700 1 --2- 172.21.15.3:0/2043534267 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f498412cff0 0x7f49841c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:29.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.791+0000 7f49827fc700 1 --2- 172.21.15.3:0/2043534267 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f498412cc20 0x7f498406fa80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f497800b450 tx=0x7f497800b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:29.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.791+0000 7f496b7fe700 1 -- 172.21.15.3:0/2043534267 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4978011830 con 0x7f498412cc20 2023-10-03T10:12:29.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.792+0000 7f496b7fe700 1 -- 172.21.15.3:0/2043534267 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4978011990 con 0x7f498412cc20 2023-10-03T10:12:29.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.792+0000 7f49896dc700 1 -- 172.21.15.3:0/2043534267 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4984071e90 con 0x7f498412cc20 2023-10-03T10:12:29.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.792+0000 7f496b7fe700 1 -- 172.21.15.3:0/2043534267 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4978011d40 con 0x7f498412cc20 2023-10-03T10:12:29.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.792+0000 7f49896dc700 1 -- 172.21.15.3:0/2043534267 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4984098e20 con 0x7f498412cc20 2023-10-03T10:12:29.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.793+0000 7f496b7fe700 1 -- 172.21.15.3:0/2043534267 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f497801a400 con 0x7f498412cc20 2023-10-03T10:12:29.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.793+0000 7f49896dc700 1 -- 172.21.15.3:0/2043534267 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49840747f0 con 0x7f498412cc20 2023-10-03T10:12:29.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.793+0000 7f496b7fe700 1 -- 172.21.15.3:0/2043534267 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4078+0+0 (secure 0 0 0) 0x7f497803fae0 con 0x7f498412cc20 2023-10-03T10:12:29.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:29.800+0000 7f496b7fe700 1 -- 172.21.15.3:0/2043534267 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f497801a6b0 con 0x7f498412cc20 2023-10-03T10:12:30.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:30.063+0000 7f49896dc700 1 -- 172.21.15.3:0/2043534267 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4984072390 con 0x7f498412cc20 2023-10-03T10:12:30.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:30.064+0000 7f496b7fe700 1 -- 172.21.15.3:0/2043534267 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+12747 (secure 0 0 0) 0x7f4978015030 con 0x7f498412cc20 2023-10-03T10:12:30.065 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:12:30.065 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":29,"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","created":"2023-10-03T10:09:28.703441+0000","modified":"2023-10-03T10:12:29.225383+0000","last_up_change":"2023-10-03T10:12:29.225383+0000","last_in_change":"2023-10-03T10:09:30.156546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-03T10:09:36.623971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.25,"raw_score_acting":1,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-03T10:09:38.210614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"error_message":"pool 2 has some PGs where all OSDs are with primary_affinity 0 (at least pg 2.0), read balance score may not be reliable","score_acting":0,"score_stable":0}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-03T10:09:42.695873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"error_message":"pool 3 has some PGs where all OSDs are with primary_affinity 0 (at least pg 3.0), read balance score may not be reliable","score_acting":0,"score_stable":0}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-03T10:09:43.115588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"error_message":"pool 4 has some PGs where all OSDs are with primary_affinity 0 (at least pg 4.0), read balance score may not be reliable","score_acting":0,"score_stable":0}}],"osds":[{"osd":0,"uuid":"cdcdd930-3c5b-4486-a021-7132f90fb6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":29,"up_thru":19,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6804","nonce":129556}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6805","nonce":129556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6807","nonce":129556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6806","nonce":129556}]},"public_addr":"172.21.15.3:6804/129556","cluster_addr":"172.21.15.3:6805/129556","heartbeat_back_addr":"172.21.15.3:6807/129556","heartbeat_front_addr":"172.21.15.3:6806/129556","state":["exists","up"]},{"osd":1,"uuid":"c8e58877-6762-4741-a8c1-567ebc97b708","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6804","nonce":118835}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6805","nonce":118835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6807","nonce":118835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6806","nonce":118835}]},"public_addr":"172.21.15.3:6804/118835","cluster_addr":"172.21.15.3:6805/118835","heartbeat_back_addr":"172.21.15.3:6807/118835","heartbeat_front_addr":"172.21.15.3:6806/118835","state":["exists"]},{"osd":2,"uuid":"68a2a21a-7a8e-4f82-8433-5201faa50777","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":25,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6812","nonce":118864}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6813","nonce":118864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6815","nonce":118864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6814","nonce":118864}]},"public_addr":"172.21.15.3:6812/118864","cluster_addr":"172.21.15.3:6813/118864","heartbeat_back_addr":"172.21.15.3:6815/118864","heartbeat_front_addr":"172.21.15.3:6814/118864","state":["exists"]},{"osd":3,"uuid":"c4df2903-4e1e-4208-bf77-95b9fa749241","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":26,"down_at":27,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6800","nonce":118836}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6801","nonce":118836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6803","nonce":118836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6802","nonce":118836}]},"public_addr":"172.21.15.3:6800/118836","cluster_addr":"172.21.15.3:6801/118836","heartbeat_back_addr":"172.21.15.3:6803/118836","heartbeat_front_addr":"172.21.15.3:6802/118836","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.369708+0000","dead_epoch":24},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.257825+0000","dead_epoch":25},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.337931+0000","dead_epoch":26},{"osd":3,"down_stamp":"2023-10-03T10:12:26.212430+0000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.085222+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1","mappings":[{"from":3,"to":0}]},{"pgid":"4.8","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.19","mappings":[{"from":3,"to":0}]},{"pgid":"4.2a","mappings":[{"from":1,"to":2}]},{"pgid":"4.2d","mappings":[{"from":3,"to":0}]},{"pgid":"4.33","mappings":[{"from":3,"to":0}]},{"pgid":"4.39","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/3919724564":"2023-10-04T10:12:13.305411+0000","172.21.15.3:0/2091814084":"2023-10-04T10:12:13.305411+0000","172.21.15.3:6816/118296":"2023-10-04T10:12:13.305411+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-03T10:12:30.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:30.066+0000 7f49896dc700 1 -- 172.21.15.3:0/2043534267 >> v1:172.21.15.3:6816/127025 conn(0x7f497002f510 legacy=0x7f49700319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:30.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:30.067+0000 7f49896dc700 1 -- 172.21.15.3:0/2043534267 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f498412cc20 msgr2=0x7f498406fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:30.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:30.067+0000 7f49896dc700 1 --2- 172.21.15.3:0/2043534267 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f498412cc20 0x7f498406fa80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f497800b450 tx=0x7f497800b760 comp rx=0 tx=0).stop 2023-10-03T10:12:30.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:30.067+0000 7f49896dc700 1 -- 172.21.15.3:0/2043534267 shutdown_connections 2023-10-03T10:12:30.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:30.067+0000 7f49896dc700 1 --2- 172.21.15.3:0/2043534267 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f498412cff0 0x7f49841c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:30.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:30.067+0000 7f49896dc700 1 --2- 172.21.15.3:0/2043534267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f498412c2f0 0x7f498406f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:30.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:30.067+0000 7f49896dc700 1 --2- 172.21.15.3:0/2043534267 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f498412cc20 0x7f498406fa80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:30.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:30.067+0000 7f49896dc700 1 -- 172.21.15.3:0/2043534267 >> 172.21.15.3:0/2043534267 conn(0x7f498409f470 msgr2=0x7f498412a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:30.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:30.067+0000 7f49896dc700 1 -- 172.21.15.3:0/2043534267 shutdown_connections 2023-10-03T10:12:30.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:30.067+0000 7f49896dc700 1 -- 172.21.15.3:0/2043534267 wait complete. 2023-10-03T10:12:30.248 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-10-03T10:12:30.248+0000 7fb7f008f700 -1 osd.1 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-03T10:12:30.657 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-10-03T10:12:30.657+0000 7f349086a680 -1 osd.3 26 log_to_monitors true 2023-10-03T10:12:31.186 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-10-03T10:12:31.185+0000 7fc19b5b9700 -1 osd.2 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-03T10:12:32.226 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-10-03T10:12:32.226+0000 7f347c7e5700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-03T10:12:33.084 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-03T10:12:33.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.229+0000 7f30ce0b9700 1 Processor -- start 2023-10-03T10:12:33.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.229+0000 7f30ce0b9700 1 -- start start 2023-10-03T10:12:33.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.230+0000 7f30ce0b9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30c809e930 0x7f30c809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:33.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.230+0000 7f30ce0b9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f30c809f650 0x7f30c8131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:33.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.230+0000 7f30ce0b9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c8070940 con 0x7f30c809f240 2023-10-03T10:12:33.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.230+0000 7f30ce0b9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c806fd60 con 0x7f30c809f650 2023-10-03T10:12:33.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.230+0000 7f30ce0b9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f30c806fa80 con 0x7f30c809e930 2023-10-03T10:12:33.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.230+0000 7f30c77fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30c809e930 0x7f30c809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:33.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.230+0000 7f30c7fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f30c809f650 0x7f30c8131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:33.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.230+0000 7f30c7fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f30c809f650 0x7f30c8131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52762/0 (socket says 172.21.15.3:52762) 2023-10-03T10:12:33.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.230+0000 7f30c77fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30c809e930 0x7f30c809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56710/0 (socket says 172.21.15.3:56710) 2023-10-03T10:12:33.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.230+0000 7f30c7fff700 1 -- 172.21.15.3:0/2213680194 learned_addr learned my addr 172.21.15.3:0/2213680194 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:33.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.230+0000 7f30c6ffd700 1 --1- 172.21.15.3:0/2213680194 >> v1:172.21.15.3:6789/0 conn(0x7f30c809f240 0x7f30c81317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59188/0 (socket says 172.21.15.3:59188) 2023-10-03T10:12:33.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.230+0000 7f30c7fff700 1 -- 172.21.15.3:0/2213680194 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30c809e930 msgr2=0x7f30c809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:33.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.230+0000 7f30c7fff700 1 --2- 172.21.15.3:0/2213680194 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30c809e930 0x7f30c809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:33.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.230+0000 7f30c7fff700 1 -- 172.21.15.3:0/2213680194 >> v1:172.21.15.3:6789/0 conn(0x7f30c809f240 legacy=0x7f30c81317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:33.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.230+0000 7f30c67fc700 1 -- 172.21.15.3:0/2213680194 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172636713 0 0) 0x7f30c8070940 con 0x7f30c809f240 2023-10-03T10:12:33.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.230+0000 7f30c7fff700 1 -- 172.21.15.3:0/2213680194 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c809fb60 con 0x7f30c809f650 2023-10-03T10:12:33.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.231+0000 7f30c7fff700 1 --2- 172.21.15.3:0/2213680194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30c809f650 0x7f30c8131f00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f30c0009e50 tx=0x7f30c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3615d98650a16f28 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:33.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.231+0000 7f30c67fc700 1 -- 172.21.15.3:0/2213680194 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c000e070 con 0x7f30c809f650 2023-10-03T10:12:33.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.231+0000 7f30c67fc700 1 -- 172.21.15.3:0/2213680194 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c000b910 con 0x7f30c809f650 2023-10-03T10:12:33.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.231+0000 7f30c67fc700 1 -- 172.21.15.3:0/2213680194 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c000bcc0 con 0x7f30c809f650 2023-10-03T10:12:33.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.231+0000 7f30ce0b9700 1 -- 172.21.15.3:0/2213680194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30c809f650 msgr2=0x7f30c8131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:33.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.231+0000 7f30ce0b9700 1 --2- 172.21.15.3:0/2213680194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30c809f650 0x7f30c8131f00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f30c0009e50 tx=0x7f30c000b330 comp rx=0 tx=0).stop 2023-10-03T10:12:33.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.231+0000 7f30ce0b9700 1 -- 172.21.15.3:0/2213680194 shutdown_connections 2023-10-03T10:12:33.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.231+0000 7f30ce0b9700 1 --2- 172.21.15.3:0/2213680194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30c809f650 0x7f30c8131f00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:33.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.231+0000 7f30ce0b9700 1 --2- 172.21.15.3:0/2213680194 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30c809e930 0x7f30c809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:33.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.231+0000 7f30ce0b9700 1 -- 172.21.15.3:0/2213680194 >> 172.21.15.3:0/2213680194 conn(0x7f30c80a4210 msgr2=0x7f30c80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:33.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.231+0000 7f30ce0b9700 1 -- 172.21.15.3:0/2213680194 shutdown_connections 2023-10-03T10:12:33.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.231+0000 7f30ce0b9700 1 -- 172.21.15.3:0/2213680194 wait complete. 2023-10-03T10:12:33.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.232+0000 7f30ce0b9700 1 Processor -- start 2023-10-03T10:12:33.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.233+0000 7f30ce0b9700 1 -- start start 2023-10-03T10:12:33.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.233+0000 7f30ce0b9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30c809e930 0x7f30c806c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:33.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.233+0000 7f30ce0b9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f30c809f240 0x7f30c806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:33.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.233+0000 7f30ce0b9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30c809f650 0x7f30c81c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:33.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.233+0000 7f30ce0b9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c806d630 con 0x7f30c809e930 2023-10-03T10:12:33.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.233+0000 7f30ce0b9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c806d440 con 0x7f30c809f240 2023-10-03T10:12:33.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.233+0000 7f30ce0b9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f30c81b7240 con 0x7f30c809f650 2023-10-03T10:12:33.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.233+0000 7f30c7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30c809f650 0x7f30c81c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:33.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.233+0000 7f30c6ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f30c809f240 0x7f30c806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:33.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.233+0000 7f30c77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30c809e930 0x7f30c806c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:33.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.233+0000 7f30c7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30c809f650 0x7f30c81c8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56712/0 (socket says 172.21.15.3:56712) 2023-10-03T10:12:33.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.233+0000 7f30c7fff700 1 -- 172.21.15.3:0/1482841177 learned_addr learned my addr 172.21.15.3:0/1482841177 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:33.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.233+0000 7f30c6ffd700 1 -- 172.21.15.3:0/1482841177 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30c809f650 msgr2=0x7f30c81c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:33.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.234+0000 7f30c6ffd700 1 --2- 172.21.15.3:0/1482841177 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30c809f650 0x7f30c81c8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:33.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.234+0000 7f30c6ffd700 1 -- 172.21.15.3:0/1482841177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30c809e930 msgr2=0x7f30c806c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:33.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.234+0000 7f30c6ffd700 1 --2- 172.21.15.3:0/1482841177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30c809e930 0x7f30c806c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:33.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.234+0000 7f30c6ffd700 1 -- 172.21.15.3:0/1482841177 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c0009b30 con 0x7f30c809f240 2023-10-03T10:12:33.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.234+0000 7f30c7fff700 1 --2- 172.21.15.3:0/1482841177 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30c809f650 0x7f30c81c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:33.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.234+0000 7f30c77fe700 1 --2- 172.21.15.3:0/1482841177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30c809e930 0x7f30c806c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:33.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.234+0000 7f30c6ffd700 1 --2- 172.21.15.3:0/1482841177 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f30c809f240 0x7f30c806c6a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00c470 tx=0x7f30bc00c780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:33.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.234+0000 7f30affff700 1 -- 172.21.15.3:0/1482841177 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc004c80 con 0x7f30c809f240 2023-10-03T10:12:33.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.234+0000 7f30affff700 1 -- 172.21.15.3:0/1482841177 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30bc004de0 con 0x7f30c809f240 2023-10-03T10:12:33.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.234+0000 7f30affff700 1 -- 172.21.15.3:0/1482841177 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc007310 con 0x7f30c809f240 2023-10-03T10:12:33.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.234+0000 7f30ce0b9700 1 -- 172.21.15.3:0/1482841177 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c80a0190 con 0x7f30c809f240 2023-10-03T10:12:33.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.234+0000 7f30ce0b9700 1 -- 172.21.15.3:0/1482841177 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c8071d70 con 0x7f30c809f240 2023-10-03T10:12:33.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.235+0000 7f30ce0b9700 1 -- 172.21.15.3:0/1482841177 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c8074ea0 con 0x7f30c809f240 2023-10-03T10:12:33.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.235+0000 7f30affff700 1 -- 172.21.15.3:0/1482841177 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f30bc007470 con 0x7f30c809f240 2023-10-03T10:12:33.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.236+0000 7f30affff700 1 -- 172.21.15.3:0/1482841177 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4078+0+0 (secure 0 0 0) 0x7f30bc03fdd0 con 0x7f30c809f240 2023-10-03T10:12:33.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.240+0000 7f30affff700 1 -- 172.21.15.3:0/1482841177 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f30bc007720 con 0x7f30c809f240 2023-10-03T10:12:33.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.467+0000 7f30ce0b9700 1 -- 172.21.15.3:0/1482841177 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30c81a81d0 con 0x7f30c809f240 2023-10-03T10:12:33.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.468+0000 7f30affff700 1 -- 172.21.15.3:0/1482841177 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+12961 (secure 0 0 0) 0x7f30bc01b950 con 0x7f30c809f240 2023-10-03T10:12:33.469 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:12:33.469 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":33,"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","created":"2023-10-03T10:09:28.703441+0000","modified":"2023-10-03T10:12:33.264731+0000","last_up_change":"2023-10-03T10:12:32.260880+0000","last_in_change":"2023-10-03T10:09:30.156546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-03T10:09:36.623971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-03T10:09:38.210614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-03T10:09:42.695873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-03T10:09:43.115588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cdcdd930-3c5b-4486-a021-7132f90fb6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":29,"up_thru":32,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6804","nonce":129556}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6805","nonce":129556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6807","nonce":129556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6806","nonce":129556}]},"public_addr":"172.21.15.3:6804/129556","cluster_addr":"172.21.15.3:6805/129556","heartbeat_back_addr":"172.21.15.3:6807/129556","heartbeat_front_addr":"172.21.15.3:6806/129556","state":["exists","up"]},{"osd":1,"uuid":"c8e58877-6762-4741-a8c1-567ebc97b708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":31,"up_thru":32,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6808","nonce":129654}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6809","nonce":129654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6811","nonce":129654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6810","nonce":129654}]},"public_addr":"172.21.15.3:6808/129654","cluster_addr":"172.21.15.3:6809/129654","heartbeat_back_addr":"172.21.15.3:6811/129654","heartbeat_front_addr":"172.21.15.3:6810/129654","state":["exists","up"]},{"osd":2,"uuid":"68a2a21a-7a8e-4f82-8433-5201faa50777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":25,"up_from":31,"up_thru":32,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6800","nonce":129764}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6801","nonce":129764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6803","nonce":129764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6802","nonce":129764}]},"public_addr":"172.21.15.3:6800/129764","cluster_addr":"172.21.15.3:6801/129764","heartbeat_back_addr":"172.21.15.3:6803/129764","heartbeat_front_addr":"172.21.15.3:6802/129764","state":["exists","up"]},{"osd":3,"uuid":"c4df2903-4e1e-4208-bf77-95b9fa749241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":26,"up_from":32,"up_thru":32,"down_at":27,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6812","nonce":129912}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6813","nonce":129912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6815","nonce":129912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6814","nonce":129912}]},"public_addr":"172.21.15.3:6812/129912","cluster_addr":"172.21.15.3:6813/129912","heartbeat_back_addr":"172.21.15.3:6815/129912","heartbeat_front_addr":"172.21.15.3:6814/129912","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.369708+0000","dead_epoch":24},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.257825+0000","dead_epoch":25},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.337931+0000","dead_epoch":26},{"osd":3,"down_stamp":"2023-10-03T10:12:26.212430+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.085222+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1","mappings":[{"from":3,"to":0}]},{"pgid":"4.8","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.19","mappings":[{"from":3,"to":0}]},{"pgid":"4.2a","mappings":[{"from":1,"to":2}]},{"pgid":"4.2d","mappings":[{"from":3,"to":0}]},{"pgid":"4.33","mappings":[{"from":3,"to":0}]},{"pgid":"4.39","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/3919724564":"2023-10-04T10:12:13.305411+0000","172.21.15.3:0/2091814084":"2023-10-04T10:12:13.305411+0000","172.21.15.3:6816/118296":"2023-10-04T10:12:13.305411+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-03T10:12:33.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.470+0000 7f30ce0b9700 1 -- 172.21.15.3:0/1482841177 >> v1:172.21.15.3:6816/127025 conn(0x7f30b402f510 legacy=0x7f30b40319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:33.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.470+0000 7f30ce0b9700 1 -- 172.21.15.3:0/1482841177 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f30c809f240 msgr2=0x7f30c806c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:33.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.470+0000 7f30ce0b9700 1 --2- 172.21.15.3:0/1482841177 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f30c809f240 0x7f30c806c6a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00c470 tx=0x7f30bc00c780 comp rx=0 tx=0).stop 2023-10-03T10:12:33.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.470+0000 7f30ce0b9700 1 -- 172.21.15.3:0/1482841177 shutdown_connections 2023-10-03T10:12:33.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.470+0000 7f30ce0b9700 1 --2- 172.21.15.3:0/1482841177 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30c809f650 0x7f30c81c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:33.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.471+0000 7f30ce0b9700 1 --2- 172.21.15.3:0/1482841177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30c809e930 0x7f30c806c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:33.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.471+0000 7f30ce0b9700 1 --2- 172.21.15.3:0/1482841177 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f30c809f240 0x7f30c806c6a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:33.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.471+0000 7f30ce0b9700 1 -- 172.21.15.3:0/1482841177 >> 172.21.15.3:0/1482841177 conn(0x7f30c80a4210 msgr2=0x7f30c81b98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:33.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.471+0000 7f30ce0b9700 1 -- 172.21.15.3:0/1482841177 shutdown_connections 2023-10-03T10:12:33.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:33.471+0000 7f30ce0b9700 1 -- 172.21.15.3:0/1482841177 wait complete. 2023-10-03T10:12:33.485 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-03T10:12:33.485 INFO:teuthology.run_tasks:Running task ceph.stop... 2023-10-03T10:12:33.495 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-03T10:12:33.495 INFO:teuthology.orchestra.run:waiting for 300 2023-10-03T10:12:33.506 INFO:tasks.ceph.mds.a:Stopped 2023-10-03T10:12:33.607 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-10-03T10:12:38.608 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog finished 2023-10-03T10:12:38.608 INFO:teuthology.run_tasks:Running task ceph.restart... 2023-10-03T10:12:38.620 ERROR:tasks.ceph.mds.a:tried to stop a non-running daemon 2023-10-03T10:12:38.620 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-03T10:12:38.620 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-03T10:12:38.623 INFO:tasks.ceph.mds.a:Started 2023-10-03T10:12:38.623 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-03T10:12:38.623 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-03T10:12:38.718 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2023-10-03T10:12:38.734 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-10-03T10:12:38.734+0000 7ff45f291700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-03T10:12:38.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.776+0000 7f11a721c700 1 Processor -- start 2023-10-03T10:12:38.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.776+0000 7f11a721c700 1 -- start start 2023-10-03T10:12:38.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.777+0000 7f11a721c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f11a009d6f0 0x7f11a009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:38.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.777+0000 7f11a721c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f11a009e000 0x7f11a01339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:38.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.777+0000 7f11a721c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f11a009e410 0x7f11a0133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:38.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.777+0000 7f11a721c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11a006b460 con 0x7f11a009e410 2023-10-03T10:12:38.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.777+0000 7f11a721c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a009b120 con 0x7f11a009e000 2023-10-03T10:12:38.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.777+0000 7f11a721c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11a009a7e0 con 0x7f11a009d6f0 2023-10-03T10:12:38.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.777+0000 7f11a4fb8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f11a009d6f0 0x7f11a009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.777+0000 7f11a57b9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f11a009e410 0x7f11a0133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.777+0000 7f1197fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f11a009e000 0x7f11a01339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.777+0000 7f11a57b9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f11a009e410 0x7f11a0133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52796/0 (socket says 172.21.15.3:52796) 2023-10-03T10:12:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.777+0000 7f1197fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f11a009e000 0x7f11a01339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40540/0 (socket says 172.21.15.3:40540) 2023-10-03T10:12:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.777+0000 7f11a57b9700 1 -- 172.21.15.3:0/3629967612 learned_addr learned my addr 172.21.15.3:0/3629967612 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.777+0000 7f11a4fb8700 1 --2- 172.21.15.3:0/3629967612 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f11a009d6f0 0x7f11a009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56740/0 (socket says 172.21.15.3:56740) 2023-10-03T10:12:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.777+0000 7f11a57b9700 1 -- 172.21.15.3:0/3629967612 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f11a009d6f0 msgr2=0x7f11a009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.777+0000 7f11a57b9700 1 --2- 172.21.15.3:0/3629967612 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f11a009d6f0 0x7f11a009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.778+0000 7f11a57b9700 1 -- 172.21.15.3:0/3629967612 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f11a009e000 msgr2=0x7f11a01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.778+0000 7f11a57b9700 1 --2- 172.21.15.3:0/3629967612 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f11a009e000 0x7f11a01339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.778+0000 7f11a57b9700 1 -- 172.21.15.3:0/3629967612 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a009a500 con 0x7f11a009e410 2023-10-03T10:12:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.778+0000 7f11a4fb8700 1 --2- 172.21.15.3:0/3629967612 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f11a009d6f0 0x7f11a009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:38.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.778+0000 7f1197fff700 1 --2- 172.21.15.3:0/3629967612 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f11a009e000 0x7f11a01339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:38.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.778+0000 7f11a57b9700 1 --2- 172.21.15.3:0/3629967612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11a009e410 0x7f11a0133f30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1198009e50 tx=0x7f119800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2c5529086758b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:38.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.778+0000 7f11977fe700 1 -- 172.21.15.3:0/3629967612 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119800e070 con 0x7f11a009e410 2023-10-03T10:12:38.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.778+0000 7f11977fe700 1 -- 172.21.15.3:0/3629967612 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f119800b910 con 0x7f11a009e410 2023-10-03T10:12:38.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.778+0000 7f11977fe700 1 -- 172.21.15.3:0/3629967612 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119800bcc0 con 0x7f11a009e410 2023-10-03T10:12:38.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.778+0000 7f11a721c700 1 -- 172.21.15.3:0/3629967612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11a009e410 msgr2=0x7f11a0133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:38.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.778+0000 7f11a721c700 1 --2- 172.21.15.3:0/3629967612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11a009e410 0x7f11a0133f30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1198009e50 tx=0x7f119800b330 comp rx=0 tx=0).stop 2023-10-03T10:12:38.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.779+0000 7f11a721c700 1 -- 172.21.15.3:0/3629967612 shutdown_connections 2023-10-03T10:12:38.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.779+0000 7f11a721c700 1 --2- 172.21.15.3:0/3629967612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11a009e410 0x7f11a0133f30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:38.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.779+0000 7f11a721c700 1 --2- 172.21.15.3:0/3629967612 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f11a009d6f0 0x7f11a009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:38.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.779+0000 7f11a721c700 1 --2- 172.21.15.3:0/3629967612 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f11a009e000 0x7f11a01339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:38.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.779+0000 7f11a721c700 1 -- 172.21.15.3:0/3629967612 >> 172.21.15.3:0/3629967612 conn(0x7f11a0066f50 msgr2=0x7f11a009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:38.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.779+0000 7f11a721c700 1 -- 172.21.15.3:0/3629967612 shutdown_connections 2023-10-03T10:12:38.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.779+0000 7f11a721c700 1 -- 172.21.15.3:0/3629967612 wait complete. 2023-10-03T10:12:38.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.780+0000 7f11a721c700 1 Processor -- start 2023-10-03T10:12:38.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.780+0000 7f11a721c700 1 -- start start 2023-10-03T10:12:38.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.780+0000 7f11a721c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f11a009d6f0 0x7f11a01a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:38.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.780+0000 7f11a721c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f11a009e000 0x7f11a01a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:38.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.780+0000 7f11a721c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11a009e410 0x7f11a01c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:38.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.780+0000 7f11a721c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a01a5c90 con 0x7f11a009e410 2023-10-03T10:12:38.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.780+0000 7f11a721c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a01b9140 con 0x7f11a009d6f0 2023-10-03T10:12:38.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.780+0000 7f11a721c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11a01a4d20 con 0x7f11a009e000 2023-10-03T10:12:38.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.780+0000 7f1197fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f11a009e000 0x7f11a01a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:38.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.780+0000 7f11a57b9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11a009e410 0x7f11a01c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:38.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.780+0000 7f11a4fb8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f11a009d6f0 0x7f11a01a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:38.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.780+0000 7f1197fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f11a009e000 0x7f11a01a5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56746/0 (socket says 172.21.15.3:56746) 2023-10-03T10:12:38.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.780+0000 7f11a57b9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11a009e410 0x7f11a01c8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52804/0 (socket says 172.21.15.3:52804) 2023-10-03T10:12:38.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.780+0000 7f1197fff700 1 -- 172.21.15.3:0/2024190726 learned_addr learned my addr 172.21.15.3:0/2024190726 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:38.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.781+0000 7f1197fff700 1 -- 172.21.15.3:0/2024190726 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f11a009d6f0 msgr2=0x7f11a01a5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:38.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.781+0000 7f1197fff700 1 --2- 172.21.15.3:0/2024190726 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f11a009d6f0 0x7f11a01a5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:38.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.781+0000 7f1197fff700 1 -- 172.21.15.3:0/2024190726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11a009e410 msgr2=0x7f11a01c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:38.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.781+0000 7f1197fff700 1 --2- 172.21.15.3:0/2024190726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11a009e410 0x7f11a01c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:38.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.781+0000 7f1197fff700 1 -- 172.21.15.3:0/2024190726 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f119c0094a0 con 0x7f11a009e000 2023-10-03T10:12:38.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.781+0000 7f11a57b9700 1 --2- 172.21.15.3:0/2024190726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11a009e410 0x7f11a01c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:38.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.781+0000 7f11a4fb8700 1 --2- 172.21.15.3:0/2024190726 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f11a009d6f0 0x7f11a01a5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:38.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.781+0000 7f1197fff700 1 --2- 172.21.15.3:0/2024190726 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f11a009e000 0x7f11a01a5750 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f119000ebe0 tx=0x7f119000eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:38.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.781+0000 7f11957fa700 1 -- 172.21.15.3:0/2024190726 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119000c850 con 0x7f11a009e000 2023-10-03T10:12:38.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.781+0000 7f11957fa700 1 -- 172.21.15.3:0/2024190726 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f119000c9b0 con 0x7f11a009e000 2023-10-03T10:12:38.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.781+0000 7f11a721c700 1 -- 172.21.15.3:0/2024190726 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11900094a0 con 0x7f11a009e000 2023-10-03T10:12:38.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.781+0000 7f11957fa700 1 -- 172.21.15.3:0/2024190726 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119000cd40 con 0x7f11a009e000 2023-10-03T10:12:38.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.781+0000 7f11a721c700 1 -- 172.21.15.3:0/2024190726 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1198009b30 con 0x7f11a009e000 2023-10-03T10:12:38.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.782+0000 7f11957fa700 1 -- 172.21.15.3:0/2024190726 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f1190013470 con 0x7f11a009e000 2023-10-03T10:12:38.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.782+0000 7f11957fa700 1 -- 172.21.15.3:0/2024190726 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f119003ffc0 con 0x7f11a009e000 2023-10-03T10:12:38.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.782+0000 7f11a721c700 1 -- 172.21.15.3:0/2024190726 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a0074ea0 con 0x7f11a009e000 2023-10-03T10:12:38.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.788+0000 7f11957fa700 1 -- 172.21.15.3:0/2024190726 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1190013720 con 0x7f11a009e000 2023-10-03T10:12:38.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.992+0000 7f11a721c700 1 -- 172.21.15.3:0/2024190726 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11a01a7030 con 0x7f11a009e000 2023-10-03T10:12:38.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.993+0000 7f11957fa700 1 -- 172.21.15.3:0/2024190726 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+13025 (secure 0 0 0) 0x7f1190011b40 con 0x7f11a009e000 2023-10-03T10:12:38.993 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:12:38.994 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":34,"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","created":"2023-10-03T10:09:28.703441+0000","modified":"2023-10-03T10:12:38.734716+0000","last_up_change":"2023-10-03T10:12:32.260880+0000","last_in_change":"2023-10-03T10:09:30.156546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-03T10:09:36.623971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-03T10:09:38.210614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-03T10:09:42.695873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-03T10:09:43.115588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cdcdd930-3c5b-4486-a021-7132f90fb6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":29,"up_thru":32,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6804","nonce":129556}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6805","nonce":129556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6807","nonce":129556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6806","nonce":129556}]},"public_addr":"172.21.15.3:6804/129556","cluster_addr":"172.21.15.3:6805/129556","heartbeat_back_addr":"172.21.15.3:6807/129556","heartbeat_front_addr":"172.21.15.3:6806/129556","state":["exists","up"]},{"osd":1,"uuid":"c8e58877-6762-4741-a8c1-567ebc97b708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":31,"up_thru":32,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6808","nonce":129654}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6809","nonce":129654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6811","nonce":129654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6810","nonce":129654}]},"public_addr":"172.21.15.3:6808/129654","cluster_addr":"172.21.15.3:6809/129654","heartbeat_back_addr":"172.21.15.3:6811/129654","heartbeat_front_addr":"172.21.15.3:6810/129654","state":["exists","up"]},{"osd":2,"uuid":"68a2a21a-7a8e-4f82-8433-5201faa50777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":25,"up_from":31,"up_thru":32,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6800","nonce":129764}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6801","nonce":129764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6803","nonce":129764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6802","nonce":129764}]},"public_addr":"172.21.15.3:6800/129764","cluster_addr":"172.21.15.3:6801/129764","heartbeat_back_addr":"172.21.15.3:6803/129764","heartbeat_front_addr":"172.21.15.3:6802/129764","state":["exists","up"]},{"osd":3,"uuid":"c4df2903-4e1e-4208-bf77-95b9fa749241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":26,"up_from":32,"up_thru":32,"down_at":27,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6812","nonce":129912}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6813","nonce":129912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6815","nonce":129912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6814","nonce":129912}]},"public_addr":"172.21.15.3:6812/129912","cluster_addr":"172.21.15.3:6813/129912","heartbeat_back_addr":"172.21.15.3:6815/129912","heartbeat_front_addr":"172.21.15.3:6814/129912","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.369708+0000","dead_epoch":24},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.257825+0000","dead_epoch":25},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.337931+0000","dead_epoch":26},{"osd":3,"down_stamp":"2023-10-03T10:12:26.212430+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.085222+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1","mappings":[{"from":3,"to":0}]},{"pgid":"4.8","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.19","mappings":[{"from":3,"to":0}]},{"pgid":"4.2a","mappings":[{"from":1,"to":2}]},{"pgid":"4.2d","mappings":[{"from":3,"to":0}]},{"pgid":"4.33","mappings":[{"from":3,"to":0}]},{"pgid":"4.39","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/3919724564":"2023-10-04T10:12:13.305411+0000","172.21.15.3:6817/2582564779":"2023-10-04T10:12:38.734689+0000","172.21.15.3:0/2091814084":"2023-10-04T10:12:13.305411+0000","172.21.15.3:6816/118296":"2023-10-04T10:12:13.305411+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-03T10:12:38.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.995+0000 7f11a721c700 1 -- 172.21.15.3:0/2024190726 >> v1:172.21.15.3:6816/127025 conn(0x7f118c02f510 legacy=0x7f118c0319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:38.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.995+0000 7f11a721c700 1 -- 172.21.15.3:0/2024190726 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f11a009e000 msgr2=0x7f11a01a5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:38.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.995+0000 7f11a721c700 1 --2- 172.21.15.3:0/2024190726 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f11a009e000 0x7f11a01a5750 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f119000ebe0 tx=0x7f119000eef0 comp rx=0 tx=0).stop 2023-10-03T10:12:38.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.996+0000 7f11a721c700 1 -- 172.21.15.3:0/2024190726 shutdown_connections 2023-10-03T10:12:38.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.996+0000 7f11a721c700 1 --2- 172.21.15.3:0/2024190726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11a009e410 0x7f11a01c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:38.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.996+0000 7f11a721c700 1 --2- 172.21.15.3:0/2024190726 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f11a009d6f0 0x7f11a01a5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:38.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.996+0000 7f11a721c700 1 --2- 172.21.15.3:0/2024190726 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f11a009e000 0x7f11a01a5750 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:38.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.996+0000 7f11a721c700 1 -- 172.21.15.3:0/2024190726 >> 172.21.15.3:0/2024190726 conn(0x7f11a0066f50 msgr2=0x7f11a01bbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:38.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.996+0000 7f11a721c700 1 -- 172.21.15.3:0/2024190726 shutdown_connections 2023-10-03T10:12:38.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:38.996+0000 7f11a721c700 1 -- 172.21.15.3:0/2024190726 wait complete. 2023-10-03T10:12:39.009 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-03T10:12:39.010 INFO:teuthology.run_tasks:Running task exec... 2023-10-03T10:12:39.019 INFO:teuthology.task.exec:Executing custom commands... 2023-10-03T10:12:39.020 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2023-10-03T10:12:39.020 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd dump -f json-pretty' 2023-10-03T10:12:39.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.153+0000 7fdb2458b700 1 Processor -- start 2023-10-03T10:12:39.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.153+0000 7fdb2458b700 1 -- start start 2023-10-03T10:12:39.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.154+0000 7fdb2458b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb1c09abf0 0x7fdb1c09f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:39.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.154+0000 7fdb2458b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb1c138de0 0x7fdb1c13b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:39.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.154+0000 7fdb2458b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb1c06d9d0 con 0x7fdb1c09e9d0 2023-10-03T10:12:39.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.154+0000 7fdb2458b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c06d760 con 0x7fdb1c09abf0 2023-10-03T10:12:39.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.154+0000 7fdb2458b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c09a9f0 con 0x7fdb1c138de0 2023-10-03T10:12:39.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.154+0000 7fdb21b26700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb1c09abf0 0x7fdb1c09f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:39.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.154+0000 7fdb22327700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdb1c09e9d0 0x7fdb1c09eda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59210/0 (socket says 172.21.15.3:59210) 2023-10-03T10:12:39.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.154+0000 7fdb22327700 1 -- 172.21.15.3:0/2683010676 learned_addr learned my addr 172.21.15.3:0/2683010676 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:39.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.154+0000 7fdb22b28700 1 --2- 172.21.15.3:0/2683010676 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb1c138de0 0x7fdb1c13b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:39.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.155+0000 7fdb21325700 1 -- 172.21.15.3:0/2683010676 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2011425807 0 0) 0x7fdb1c06d9d0 con 0x7fdb1c09e9d0 2023-10-03T10:12:39.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.155+0000 7fdb21325700 1 -- 172.21.15.3:0/2683010676 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb080035b0 con 0x7fdb1c09e9d0 2023-10-03T10:12:39.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.155+0000 7fdb22b28700 1 -- 172.21.15.3:0/2683010676 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb1c09abf0 msgr2=0x7fdb1c09f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:39.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.155+0000 7fdb22b28700 1 --2- 172.21.15.3:0/2683010676 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb1c09abf0 0x7fdb1c09f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:39.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.155+0000 7fdb22b28700 1 -- 172.21.15.3:0/2683010676 >> v1:172.21.15.3:6789/0 conn(0x7fdb1c09e9d0 legacy=0x7fdb1c09eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:39.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.155+0000 7fdb22b28700 1 -- 172.21.15.3:0/2683010676 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb1c06ced0 con 0x7fdb1c138de0 2023-10-03T10:12:39.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.155+0000 7fdb21b26700 1 --2- 172.21.15.3:0/2683010676 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb1c09abf0 0x7fdb1c09f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:39.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.155+0000 7fdb22b28700 1 --2- 172.21.15.3:0/2683010676 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb1c138de0 0x7fdb1c13b1c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009dc0 tx=0x7fdb1400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81ec1100daf27c74 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:39.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.155+0000 7fdb21325700 1 -- 172.21.15.3:0/2683010676 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1400e070 con 0x7fdb1c138de0 2023-10-03T10:12:39.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.155+0000 7fdb21325700 1 -- 172.21.15.3:0/2683010676 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb1400bac0 con 0x7fdb1c138de0 2023-10-03T10:12:39.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.155+0000 7fdb21325700 1 -- 172.21.15.3:0/2683010676 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1400be30 con 0x7fdb1c138de0 2023-10-03T10:12:39.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.156+0000 7fdb2458b700 1 -- 172.21.15.3:0/2683010676 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb1c138de0 msgr2=0x7fdb1c13b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:39.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.156+0000 7fdb2458b700 1 --2- 172.21.15.3:0/2683010676 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb1c138de0 0x7fdb1c13b1c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009dc0 tx=0x7fdb1400b330 comp rx=0 tx=0).stop 2023-10-03T10:12:39.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.156+0000 7fdb2458b700 1 -- 172.21.15.3:0/2683010676 shutdown_connections 2023-10-03T10:12:39.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.156+0000 7fdb2458b700 1 --2- 172.21.15.3:0/2683010676 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb1c138de0 0x7fdb1c13b1c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:39.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.156+0000 7fdb2458b700 1 --2- 172.21.15.3:0/2683010676 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb1c09abf0 0x7fdb1c09f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:39.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.156+0000 7fdb2458b700 1 -- 172.21.15.3:0/2683010676 >> 172.21.15.3:0/2683010676 conn(0x7fdb1c0a43f0 msgr2=0x7fdb1c0a47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:39.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.156+0000 7fdb2458b700 1 -- 172.21.15.3:0/2683010676 shutdown_connections 2023-10-03T10:12:39.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.156+0000 7fdb2458b700 1 -- 172.21.15.3:0/2683010676 wait complete. 2023-10-03T10:12:39.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.157+0000 7fdb2458b700 1 Processor -- start 2023-10-03T10:12:39.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.157+0000 7fdb2458b700 1 -- start start 2023-10-03T10:12:39.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.157+0000 7fdb2458b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb1c09abf0 0x7fdb1c1206d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:39.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.157+0000 7fdb2458b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb1c09e9d0 0x7fdb1c120c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:39.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.157+0000 7fdb2458b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb1c138de0 0x7fdb1c121150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:39.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.157+0000 7fdb2458b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c138ac0 con 0x7fdb1c09abf0 2023-10-03T10:12:39.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.157+0000 7fdb2458b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c1234e0 con 0x7fdb1c138de0 2023-10-03T10:12:39.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.157+0000 7fdb2458b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c122af0 con 0x7fdb1c09e9d0 2023-10-03T10:12:39.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.157+0000 7fdb22327700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb1c09abf0 0x7fdb1c1206d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:39.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.157+0000 7fdb21b26700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb1c09e9d0 0x7fdb1c120c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:39.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.157+0000 7fdb21b26700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb1c09e9d0 0x7fdb1c120c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56770/0 (socket says 172.21.15.3:56770) 2023-10-03T10:12:39.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.157+0000 7fdb22327700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb1c09abf0 0x7fdb1c1206d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52816/0 (socket says 172.21.15.3:52816) 2023-10-03T10:12:39.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.157+0000 7fdb21b26700 1 -- 172.21.15.3:0/481959842 learned_addr learned my addr 172.21.15.3:0/481959842 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:39.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.157+0000 7fdb22b28700 1 --2- 172.21.15.3:0/481959842 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb1c138de0 0x7fdb1c121150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:39.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.158+0000 7fdb21b26700 1 -- 172.21.15.3:0/481959842 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb1c138de0 msgr2=0x7fdb1c121150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:39.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.158+0000 7fdb21b26700 1 --2- 172.21.15.3:0/481959842 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb1c138de0 0x7fdb1c121150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:39.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.158+0000 7fdb21b26700 1 -- 172.21.15.3:0/481959842 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb1c09abf0 msgr2=0x7fdb1c1206d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:39.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.158+0000 7fdb21b26700 1 --2- 172.21.15.3:0/481959842 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb1c09abf0 0x7fdb1c1206d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.158+0000 7fdb21b26700 1 -- 172.21.15.3:0/481959842 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0c0094a0 con 0x7fdb1c09e9d0 2023-10-03T10:12:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.158+0000 7fdb21b26700 1 --2- 172.21.15.3:0/481959842 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb1c09e9d0 0x7fdb1c120c10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00ebb0 tx=0x7fdb0c00eec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.158+0000 7fdb12ffd700 1 -- 172.21.15.3:0/481959842 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c00c850 con 0x7fdb1c09e9d0 2023-10-03T10:12:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.158+0000 7fdb12ffd700 1 -- 172.21.15.3:0/481959842 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0c00c9b0 con 0x7fdb1c09e9d0 2023-10-03T10:12:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.158+0000 7fdb2458b700 1 -- 172.21.15.3:0/481959842 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb14009a70 con 0x7fdb1c09e9d0 2023-10-03T10:12:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.158+0000 7fdb12ffd700 1 -- 172.21.15.3:0/481959842 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c00cd40 con 0x7fdb1c09e9d0 2023-10-03T10:12:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.158+0000 7fdb2458b700 1 -- 172.21.15.3:0/481959842 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb1c06dd10 con 0x7fdb1c09e9d0 2023-10-03T10:12:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.159+0000 7fdb12ffd700 1 -- 172.21.15.3:0/481959842 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fdb0c013470 con 0x7fdb1c09e9d0 2023-10-03T10:12:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.159+0000 7fdb12ffd700 1 -- 172.21.15.3:0/481959842 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4121+0+0 (secure 0 0 0) 0x7fdb0c03fcb0 con 0x7fdb1c09e9d0 2023-10-03T10:12:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.159+0000 7fdb2458b700 1 -- 172.21.15.3:0/481959842 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb1c04f2a0 con 0x7fdb1c09e9d0 2023-10-03T10:12:39.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.164+0000 7fdb12ffd700 1 -- 172.21.15.3:0/481959842 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdb0c017020 con 0x7fdb1c09e9d0 2023-10-03T10:12:39.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.370+0000 7fdb2458b700 1 -- 172.21.15.3:0/481959842 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fdb1c123d90 con 0x7fdb1c09e9d0 2023-10-03T10:12:39.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.371+0000 7fdb12ffd700 1 -- 172.21.15.3:0/481959842 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v34) v1 ==== 81+0+24489 (secure 0 0 0) 0x7fdb0c0232f0 con 0x7fdb1c09e9d0 2023-10-03T10:12:39.372 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:12:39.372 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-03T10:12:39.372 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 34, 2023-10-03T10:12:39.372 INFO:teuthology.orchestra.run.smithi003.stdout: "fsid": "68e08e7d-a212-4fd3-bd42-88e2411aa299", 2023-10-03T10:12:39.372 INFO:teuthology.orchestra.run.smithi003.stdout: "created": "2023-10-03T10:09:28.703441+0000", 2023-10-03T10:12:39.372 INFO:teuthology.orchestra.run.smithi003.stdout: "modified": "2023-10-03T10:12:38.734716+0000", 2023-10-03T10:12:39.372 INFO:teuthology.orchestra.run.smithi003.stdout: "last_up_change": "2023-10-03T10:12:32.260880+0000", 2023-10-03T10:12:39.372 INFO:teuthology.orchestra.run.smithi003.stdout: "last_in_change": "2023-10-03T10:09:30.156546+0000", 2023-10-03T10:12:39.372 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-03T10:12:39.372 INFO:teuthology.orchestra.run.smithi003.stdout: "flags_num": 5799936, 2023-10-03T10:12:39.373 INFO:teuthology.orchestra.run.smithi003.stdout: "flags_set": [ 2023-10-03T10:12:39.373 INFO:teuthology.orchestra.run.smithi003.stdout: "pglog_hardlimit", 2023-10-03T10:12:39.373 INFO:teuthology.orchestra.run.smithi003.stdout: "purged_snapdirs", 2023-10-03T10:12:39.373 INFO:teuthology.orchestra.run.smithi003.stdout: "recovery_deletes", 2023-10-03T10:12:39.373 INFO:teuthology.orchestra.run.smithi003.stdout: "sortbitwise" 2023-10-03T10:12:39.373 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-10-03T10:12:39.373 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_version": 8, 2023-10-03T10:12:39.373 INFO:teuthology.orchestra.run.smithi003.stdout: "full_ratio": 0.94999998807907104, 2023-10-03T10:12:39.373 INFO:teuthology.orchestra.run.smithi003.stdout: "backfillfull_ratio": 0.89999997615814209, 2023-10-03T10:12:39.373 INFO:teuthology.orchestra.run.smithi003.stdout: "nearfull_ratio": 0.85000002384185791, 2023-10-03T10:12:39.373 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_snapshot": "", 2023-10-03T10:12:39.373 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_max": 4, 2023-10-03T10:12:39.373 INFO:teuthology.orchestra.run.smithi003.stdout: "max_osd": 4, 2023-10-03T10:12:39.374 INFO:teuthology.orchestra.run.smithi003.stdout: "require_min_compat_client": "pacific", 2023-10-03T10:12:39.374 INFO:teuthology.orchestra.run.smithi003.stdout: "min_compat_client": "luminous", 2023-10-03T10:12:39.374 INFO:teuthology.orchestra.run.smithi003.stdout: "require_osd_release": "pacific", 2023-10-03T10:12:39.374 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_crimson": false, 2023-10-03T10:12:39.374 INFO:teuthology.orchestra.run.smithi003.stdout: "pools": [ 2023-10-03T10:12:39.374 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.374 INFO:teuthology.orchestra.run.smithi003.stdout: "pool": 1, 2023-10-03T10:12:39.374 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_name": ".mgr", 2023-10-03T10:12:39.374 INFO:teuthology.orchestra.run.smithi003.stdout: "create_time": "2023-10-03T10:09:36.623971+0000", 2023-10-03T10:12:39.374 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-10-03T10:12:39.374 INFO:teuthology.orchestra.run.smithi003.stdout: "flags_names": "hashpspool", 2023-10-03T10:12:39.374 INFO:teuthology.orchestra.run.smithi003.stdout: "type": 1, 2023-10-03T10:12:39.374 INFO:teuthology.orchestra.run.smithi003.stdout: "size": 2, 2023-10-03T10:12:39.375 INFO:teuthology.orchestra.run.smithi003.stdout: "min_size": 1, 2023-10-03T10:12:39.375 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_rule": 0, 2023-10-03T10:12:39.375 INFO:teuthology.orchestra.run.smithi003.stdout: "peering_crush_bucket_count": 0, 2023-10-03T10:12:39.375 INFO:teuthology.orchestra.run.smithi003.stdout: "peering_crush_bucket_target": 0, 2023-10-03T10:12:39.375 INFO:teuthology.orchestra.run.smithi003.stdout: "peering_crush_bucket_barrier": 0, 2023-10-03T10:12:39.375 INFO:teuthology.orchestra.run.smithi003.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-03T10:12:39.375 INFO:teuthology.orchestra.run.smithi003.stdout: "object_hash": 2, 2023-10-03T10:12:39.375 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscale_mode": "off", 2023-10-03T10:12:39.375 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num": 1, 2023-10-03T10:12:39.375 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_placement_num": 1, 2023-10-03T10:12:39.375 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_placement_num_target": 1, 2023-10-03T10:12:39.375 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num_target": 1, 2023-10-03T10:12:39.376 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num_pending": 1, 2023-10-03T10:12:39.376 INFO:teuthology.orchestra.run.smithi003.stdout: "last_pg_merge_meta": { 2023-10-03T10:12:39.376 INFO:teuthology.orchestra.run.smithi003.stdout: "source_pgid": "0.0", 2023-10-03T10:12:39.376 INFO:teuthology.orchestra.run.smithi003.stdout: "ready_epoch": 0, 2023-10-03T10:12:39.376 INFO:teuthology.orchestra.run.smithi003.stdout: "last_epoch_started": 0, 2023-10-03T10:12:39.376 INFO:teuthology.orchestra.run.smithi003.stdout: "last_epoch_clean": 0, 2023-10-03T10:12:39.376 INFO:teuthology.orchestra.run.smithi003.stdout: "source_version": "0'0", 2023-10-03T10:12:39.376 INFO:teuthology.orchestra.run.smithi003.stdout: "target_version": "0'0" 2023-10-03T10:12:39.376 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.376 INFO:teuthology.orchestra.run.smithi003.stdout: "last_change": "23", 2023-10-03T10:12:39.376 INFO:teuthology.orchestra.run.smithi003.stdout: "last_force_op_resend": "0", 2023-10-03T10:12:39.376 INFO:teuthology.orchestra.run.smithi003.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-03T10:12:39.376 INFO:teuthology.orchestra.run.smithi003.stdout: "last_force_op_resend_preluminous": "0", 2023-10-03T10:12:39.377 INFO:teuthology.orchestra.run.smithi003.stdout: "auid": 0, 2023-10-03T10:12:39.377 INFO:teuthology.orchestra.run.smithi003.stdout: "snap_mode": "selfmanaged", 2023-10-03T10:12:39.377 INFO:teuthology.orchestra.run.smithi003.stdout: "snap_seq": 0, 2023-10-03T10:12:39.377 INFO:teuthology.orchestra.run.smithi003.stdout: "snap_epoch": 0, 2023-10-03T10:12:39.377 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_snaps": [], 2023-10-03T10:12:39.377 INFO:teuthology.orchestra.run.smithi003.stdout: "removed_snaps": "[]", 2023-10-03T10:12:39.377 INFO:teuthology.orchestra.run.smithi003.stdout: "quota_max_bytes": 0, 2023-10-03T10:12:39.377 INFO:teuthology.orchestra.run.smithi003.stdout: "quota_max_objects": 0, 2023-10-03T10:12:39.377 INFO:teuthology.orchestra.run.smithi003.stdout: "tiers": [], 2023-10-03T10:12:39.377 INFO:teuthology.orchestra.run.smithi003.stdout: "tier_of": -1, 2023-10-03T10:12:39.377 INFO:teuthology.orchestra.run.smithi003.stdout: "read_tier": -1, 2023-10-03T10:12:39.377 INFO:teuthology.orchestra.run.smithi003.stdout: "write_tier": -1, 2023-10-03T10:12:39.377 INFO:teuthology.orchestra.run.smithi003.stdout: "cache_mode": "none", 2023-10-03T10:12:39.378 INFO:teuthology.orchestra.run.smithi003.stdout: "target_max_bytes": 0, 2023-10-03T10:12:39.378 INFO:teuthology.orchestra.run.smithi003.stdout: "target_max_objects": 0, 2023-10-03T10:12:39.378 INFO:teuthology.orchestra.run.smithi003.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-03T10:12:39.378 INFO:teuthology.orchestra.run.smithi003.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-03T10:12:39.378 INFO:teuthology.orchestra.run.smithi003.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-03T10:12:39.378 INFO:teuthology.orchestra.run.smithi003.stdout: "cache_min_flush_age": 0, 2023-10-03T10:12:39.378 INFO:teuthology.orchestra.run.smithi003.stdout: "cache_min_evict_age": 0, 2023-10-03T10:12:39.378 INFO:teuthology.orchestra.run.smithi003.stdout: "erasure_code_profile": "", 2023-10-03T10:12:39.378 INFO:teuthology.orchestra.run.smithi003.stdout: "hit_set_params": { 2023-10-03T10:12:39.378 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "none" 2023-10-03T10:12:39.378 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.378 INFO:teuthology.orchestra.run.smithi003.stdout: "hit_set_period": 0, 2023-10-03T10:12:39.378 INFO:teuthology.orchestra.run.smithi003.stdout: "hit_set_count": 0, 2023-10-03T10:12:39.379 INFO:teuthology.orchestra.run.smithi003.stdout: "use_gmt_hitset": true, 2023-10-03T10:12:39.379 INFO:teuthology.orchestra.run.smithi003.stdout: "min_read_recency_for_promote": 0, 2023-10-03T10:12:39.379 INFO:teuthology.orchestra.run.smithi003.stdout: "min_write_recency_for_promote": 0, 2023-10-03T10:12:39.379 INFO:teuthology.orchestra.run.smithi003.stdout: "hit_set_grade_decay_rate": 0, 2023-10-03T10:12:39.379 INFO:teuthology.orchestra.run.smithi003.stdout: "hit_set_search_last_n": 0, 2023-10-03T10:12:39.379 INFO:teuthology.orchestra.run.smithi003.stdout: "grade_table": [], 2023-10-03T10:12:39.379 INFO:teuthology.orchestra.run.smithi003.stdout: "stripe_width": 0, 2023-10-03T10:12:39.379 INFO:teuthology.orchestra.run.smithi003.stdout: "expected_num_objects": 0, 2023-10-03T10:12:39.379 INFO:teuthology.orchestra.run.smithi003.stdout: "fast_read": false, 2023-10-03T10:12:39.379 INFO:teuthology.orchestra.run.smithi003.stdout: "options": { 2023-10-03T10:12:39.379 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num_max": 32, 2023-10-03T10:12:39.379 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num_min": 1 2023-10-03T10:12:39.379 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.380 INFO:teuthology.orchestra.run.smithi003.stdout: "application_metadata": { 2023-10-03T10:12:39.380 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": {}, 2023-10-03T10:12:39.380 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr_devicehealth": {} 2023-10-03T10:12:39.380 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.380 INFO:teuthology.orchestra.run.smithi003.stdout: "read_balance": { 2023-10-03T10:12:39.380 INFO:teuthology.orchestra.run.smithi003.stdout: "score_acting": 4, 2023-10-03T10:12:39.380 INFO:teuthology.orchestra.run.smithi003.stdout: "score_stable": 4, 2023-10-03T10:12:39.380 INFO:teuthology.orchestra.run.smithi003.stdout: "optimal_score": 0.5, 2023-10-03T10:12:39.380 INFO:teuthology.orchestra.run.smithi003.stdout: "raw_score_acting": 2, 2023-10-03T10:12:39.380 INFO:teuthology.orchestra.run.smithi003.stdout: "raw_score_stable": 2, 2023-10-03T10:12:39.380 INFO:teuthology.orchestra.run.smithi003.stdout: "primary_affinity_weighted": 1, 2023-10-03T10:12:39.380 INFO:teuthology.orchestra.run.smithi003.stdout: "average_primary_affinity": 1, 2023-10-03T10:12:39.380 INFO:teuthology.orchestra.run.smithi003.stdout: "average_primary_affinity_weighted": 1 2023-10-03T10:12:39.381 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.381 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.381 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.381 INFO:teuthology.orchestra.run.smithi003.stdout: "pool": 2, 2023-10-03T10:12:39.381 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_name": "rbd", 2023-10-03T10:12:39.381 INFO:teuthology.orchestra.run.smithi003.stdout: "create_time": "2023-10-03T10:09:38.210614+0000", 2023-10-03T10:12:39.381 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-10-03T10:12:39.381 INFO:teuthology.orchestra.run.smithi003.stdout: "flags_names": "hashpspool", 2023-10-03T10:12:39.381 INFO:teuthology.orchestra.run.smithi003.stdout: "type": 1, 2023-10-03T10:12:39.381 INFO:teuthology.orchestra.run.smithi003.stdout: "size": 2, 2023-10-03T10:12:39.381 INFO:teuthology.orchestra.run.smithi003.stdout: "min_size": 1, 2023-10-03T10:12:39.381 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_rule": 0, 2023-10-03T10:12:39.381 INFO:teuthology.orchestra.run.smithi003.stdout: "peering_crush_bucket_count": 0, 2023-10-03T10:12:39.382 INFO:teuthology.orchestra.run.smithi003.stdout: "peering_crush_bucket_target": 0, 2023-10-03T10:12:39.382 INFO:teuthology.orchestra.run.smithi003.stdout: "peering_crush_bucket_barrier": 0, 2023-10-03T10:12:39.382 INFO:teuthology.orchestra.run.smithi003.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-03T10:12:39.382 INFO:teuthology.orchestra.run.smithi003.stdout: "object_hash": 2, 2023-10-03T10:12:39.382 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscale_mode": "off", 2023-10-03T10:12:39.382 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num": 8, 2023-10-03T10:12:39.382 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_placement_num": 8, 2023-10-03T10:12:39.382 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_placement_num_target": 8, 2023-10-03T10:12:39.382 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num_target": 8, 2023-10-03T10:12:39.382 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num_pending": 8, 2023-10-03T10:12:39.382 INFO:teuthology.orchestra.run.smithi003.stdout: "last_pg_merge_meta": { 2023-10-03T10:12:39.382 INFO:teuthology.orchestra.run.smithi003.stdout: "source_pgid": "0.0", 2023-10-03T10:12:39.382 INFO:teuthology.orchestra.run.smithi003.stdout: "ready_epoch": 0, 2023-10-03T10:12:39.383 INFO:teuthology.orchestra.run.smithi003.stdout: "last_epoch_started": 0, 2023-10-03T10:12:39.383 INFO:teuthology.orchestra.run.smithi003.stdout: "last_epoch_clean": 0, 2023-10-03T10:12:39.383 INFO:teuthology.orchestra.run.smithi003.stdout: "source_version": "0'0", 2023-10-03T10:12:39.383 INFO:teuthology.orchestra.run.smithi003.stdout: "target_version": "0'0" 2023-10-03T10:12:39.383 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.383 INFO:teuthology.orchestra.run.smithi003.stdout: "last_change": "13", 2023-10-03T10:12:39.383 INFO:teuthology.orchestra.run.smithi003.stdout: "last_force_op_resend": "0", 2023-10-03T10:12:39.383 INFO:teuthology.orchestra.run.smithi003.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-03T10:12:39.383 INFO:teuthology.orchestra.run.smithi003.stdout: "last_force_op_resend_preluminous": "0", 2023-10-03T10:12:39.383 INFO:teuthology.orchestra.run.smithi003.stdout: "auid": 0, 2023-10-03T10:12:39.383 INFO:teuthology.orchestra.run.smithi003.stdout: "snap_mode": "selfmanaged", 2023-10-03T10:12:39.383 INFO:teuthology.orchestra.run.smithi003.stdout: "snap_seq": 0, 2023-10-03T10:12:39.383 INFO:teuthology.orchestra.run.smithi003.stdout: "snap_epoch": 0, 2023-10-03T10:12:39.384 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_snaps": [], 2023-10-03T10:12:39.384 INFO:teuthology.orchestra.run.smithi003.stdout: "removed_snaps": "[]", 2023-10-03T10:12:39.384 INFO:teuthology.orchestra.run.smithi003.stdout: "quota_max_bytes": 0, 2023-10-03T10:12:39.384 INFO:teuthology.orchestra.run.smithi003.stdout: "quota_max_objects": 0, 2023-10-03T10:12:39.384 INFO:teuthology.orchestra.run.smithi003.stdout: "tiers": [], 2023-10-03T10:12:39.384 INFO:teuthology.orchestra.run.smithi003.stdout: "tier_of": -1, 2023-10-03T10:12:39.384 INFO:teuthology.orchestra.run.smithi003.stdout: "read_tier": -1, 2023-10-03T10:12:39.384 INFO:teuthology.orchestra.run.smithi003.stdout: "write_tier": -1, 2023-10-03T10:12:39.384 INFO:teuthology.orchestra.run.smithi003.stdout: "cache_mode": "none", 2023-10-03T10:12:39.384 INFO:teuthology.orchestra.run.smithi003.stdout: "target_max_bytes": 0, 2023-10-03T10:12:39.384 INFO:teuthology.orchestra.run.smithi003.stdout: "target_max_objects": 0, 2023-10-03T10:12:39.384 INFO:teuthology.orchestra.run.smithi003.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-03T10:12:39.384 INFO:teuthology.orchestra.run.smithi003.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-03T10:12:39.385 INFO:teuthology.orchestra.run.smithi003.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-03T10:12:39.385 INFO:teuthology.orchestra.run.smithi003.stdout: "cache_min_flush_age": 0, 2023-10-03T10:12:39.385 INFO:teuthology.orchestra.run.smithi003.stdout: "cache_min_evict_age": 0, 2023-10-03T10:12:39.385 INFO:teuthology.orchestra.run.smithi003.stdout: "erasure_code_profile": "", 2023-10-03T10:12:39.385 INFO:teuthology.orchestra.run.smithi003.stdout: "hit_set_params": { 2023-10-03T10:12:39.385 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "none" 2023-10-03T10:12:39.385 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.385 INFO:teuthology.orchestra.run.smithi003.stdout: "hit_set_period": 0, 2023-10-03T10:12:39.385 INFO:teuthology.orchestra.run.smithi003.stdout: "hit_set_count": 0, 2023-10-03T10:12:39.385 INFO:teuthology.orchestra.run.smithi003.stdout: "use_gmt_hitset": true, 2023-10-03T10:12:39.385 INFO:teuthology.orchestra.run.smithi003.stdout: "min_read_recency_for_promote": 0, 2023-10-03T10:12:39.385 INFO:teuthology.orchestra.run.smithi003.stdout: "min_write_recency_for_promote": 0, 2023-10-03T10:12:39.385 INFO:teuthology.orchestra.run.smithi003.stdout: "hit_set_grade_decay_rate": 0, 2023-10-03T10:12:39.385 INFO:teuthology.orchestra.run.smithi003.stdout: "hit_set_search_last_n": 0, 2023-10-03T10:12:39.386 INFO:teuthology.orchestra.run.smithi003.stdout: "grade_table": [], 2023-10-03T10:12:39.386 INFO:teuthology.orchestra.run.smithi003.stdout: "stripe_width": 0, 2023-10-03T10:12:39.386 INFO:teuthology.orchestra.run.smithi003.stdout: "expected_num_objects": 0, 2023-10-03T10:12:39.386 INFO:teuthology.orchestra.run.smithi003.stdout: "fast_read": false, 2023-10-03T10:12:39.386 INFO:teuthology.orchestra.run.smithi003.stdout: "options": {}, 2023-10-03T10:12:39.386 INFO:teuthology.orchestra.run.smithi003.stdout: "application_metadata": { 2023-10-03T10:12:39.386 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd": {} 2023-10-03T10:12:39.386 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.386 INFO:teuthology.orchestra.run.smithi003.stdout: "read_balance": { 2023-10-03T10:12:39.386 INFO:teuthology.orchestra.run.smithi003.stdout: "score_acting": 2.5, 2023-10-03T10:12:39.386 INFO:teuthology.orchestra.run.smithi003.stdout: "score_stable": 2.5, 2023-10-03T10:12:39.386 INFO:teuthology.orchestra.run.smithi003.stdout: "optimal_score": 1, 2023-10-03T10:12:39.387 INFO:teuthology.orchestra.run.smithi003.stdout: "raw_score_acting": 2.5, 2023-10-03T10:12:39.387 INFO:teuthology.orchestra.run.smithi003.stdout: "raw_score_stable": 2.5, 2023-10-03T10:12:39.387 INFO:teuthology.orchestra.run.smithi003.stdout: "primary_affinity_weighted": 1, 2023-10-03T10:12:39.387 INFO:teuthology.orchestra.run.smithi003.stdout: "average_primary_affinity": 1, 2023-10-03T10:12:39.387 INFO:teuthology.orchestra.run.smithi003.stdout: "average_primary_affinity_weighted": 1 2023-10-03T10:12:39.387 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.387 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.387 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.387 INFO:teuthology.orchestra.run.smithi003.stdout: "pool": 3, 2023-10-03T10:12:39.387 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_name": "cephfs_metadata", 2023-10-03T10:12:39.387 INFO:teuthology.orchestra.run.smithi003.stdout: "create_time": "2023-10-03T10:09:42.695873+0000", 2023-10-03T10:12:39.387 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-10-03T10:12:39.387 INFO:teuthology.orchestra.run.smithi003.stdout: "flags_names": "hashpspool", 2023-10-03T10:12:39.387 INFO:teuthology.orchestra.run.smithi003.stdout: "type": 1, 2023-10-03T10:12:39.388 INFO:teuthology.orchestra.run.smithi003.stdout: "size": 2, 2023-10-03T10:12:39.388 INFO:teuthology.orchestra.run.smithi003.stdout: "min_size": 1, 2023-10-03T10:12:39.388 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_rule": 0, 2023-10-03T10:12:39.388 INFO:teuthology.orchestra.run.smithi003.stdout: "peering_crush_bucket_count": 0, 2023-10-03T10:12:39.388 INFO:teuthology.orchestra.run.smithi003.stdout: "peering_crush_bucket_target": 0, 2023-10-03T10:12:39.388 INFO:teuthology.orchestra.run.smithi003.stdout: "peering_crush_bucket_barrier": 0, 2023-10-03T10:12:39.388 INFO:teuthology.orchestra.run.smithi003.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-03T10:12:39.388 INFO:teuthology.orchestra.run.smithi003.stdout: "object_hash": 2, 2023-10-03T10:12:39.388 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscale_mode": "off", 2023-10-03T10:12:39.388 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num": 32, 2023-10-03T10:12:39.388 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_placement_num": 32, 2023-10-03T10:12:39.388 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_placement_num_target": 32, 2023-10-03T10:12:39.388 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num_target": 32, 2023-10-03T10:12:39.389 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num_pending": 32, 2023-10-03T10:12:39.389 INFO:teuthology.orchestra.run.smithi003.stdout: "last_pg_merge_meta": { 2023-10-03T10:12:39.389 INFO:teuthology.orchestra.run.smithi003.stdout: "source_pgid": "0.0", 2023-10-03T10:12:39.389 INFO:teuthology.orchestra.run.smithi003.stdout: "ready_epoch": 0, 2023-10-03T10:12:39.389 INFO:teuthology.orchestra.run.smithi003.stdout: "last_epoch_started": 0, 2023-10-03T10:12:39.389 INFO:teuthology.orchestra.run.smithi003.stdout: "last_epoch_clean": 0, 2023-10-03T10:12:39.389 INFO:teuthology.orchestra.run.smithi003.stdout: "source_version": "0'0", 2023-10-03T10:12:39.389 INFO:teuthology.orchestra.run.smithi003.stdout: "target_version": "0'0" 2023-10-03T10:12:39.389 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.389 INFO:teuthology.orchestra.run.smithi003.stdout: "last_change": "16", 2023-10-03T10:12:39.389 INFO:teuthology.orchestra.run.smithi003.stdout: "last_force_op_resend": "0", 2023-10-03T10:12:39.389 INFO:teuthology.orchestra.run.smithi003.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-03T10:12:39.389 INFO:teuthology.orchestra.run.smithi003.stdout: "last_force_op_resend_preluminous": "0", 2023-10-03T10:12:39.389 INFO:teuthology.orchestra.run.smithi003.stdout: "auid": 0, 2023-10-03T10:12:39.390 INFO:teuthology.orchestra.run.smithi003.stdout: "snap_mode": "selfmanaged", 2023-10-03T10:12:39.390 INFO:teuthology.orchestra.run.smithi003.stdout: "snap_seq": 0, 2023-10-03T10:12:39.390 INFO:teuthology.orchestra.run.smithi003.stdout: "snap_epoch": 0, 2023-10-03T10:12:39.390 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_snaps": [], 2023-10-03T10:12:39.390 INFO:teuthology.orchestra.run.smithi003.stdout: "removed_snaps": "[]", 2023-10-03T10:12:39.390 INFO:teuthology.orchestra.run.smithi003.stdout: "quota_max_bytes": 0, 2023-10-03T10:12:39.390 INFO:teuthology.orchestra.run.smithi003.stdout: "quota_max_objects": 0, 2023-10-03T10:12:39.390 INFO:teuthology.orchestra.run.smithi003.stdout: "tiers": [], 2023-10-03T10:12:39.390 INFO:teuthology.orchestra.run.smithi003.stdout: "tier_of": -1, 2023-10-03T10:12:39.390 INFO:teuthology.orchestra.run.smithi003.stdout: "read_tier": -1, 2023-10-03T10:12:39.390 INFO:teuthology.orchestra.run.smithi003.stdout: "write_tier": -1, 2023-10-03T10:12:39.390 INFO:teuthology.orchestra.run.smithi003.stdout: "cache_mode": "none", 2023-10-03T10:12:39.391 INFO:teuthology.orchestra.run.smithi003.stdout: "target_max_bytes": 0, 2023-10-03T10:12:39.391 INFO:teuthology.orchestra.run.smithi003.stdout: "target_max_objects": 0, 2023-10-03T10:12:39.391 INFO:teuthology.orchestra.run.smithi003.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-03T10:12:39.391 INFO:teuthology.orchestra.run.smithi003.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-03T10:12:39.391 INFO:teuthology.orchestra.run.smithi003.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-03T10:12:39.391 INFO:teuthology.orchestra.run.smithi003.stdout: "cache_min_flush_age": 0, 2023-10-03T10:12:39.391 INFO:teuthology.orchestra.run.smithi003.stdout: "cache_min_evict_age": 0, 2023-10-03T10:12:39.391 INFO:teuthology.orchestra.run.smithi003.stdout: "erasure_code_profile": "", 2023-10-03T10:12:39.391 INFO:teuthology.orchestra.run.smithi003.stdout: "hit_set_params": { 2023-10-03T10:12:39.391 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "none" 2023-10-03T10:12:39.391 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.391 INFO:teuthology.orchestra.run.smithi003.stdout: "hit_set_period": 0, 2023-10-03T10:12:39.391 INFO:teuthology.orchestra.run.smithi003.stdout: "hit_set_count": 0, 2023-10-03T10:12:39.391 INFO:teuthology.orchestra.run.smithi003.stdout: "use_gmt_hitset": true, 2023-10-03T10:12:39.392 INFO:teuthology.orchestra.run.smithi003.stdout: "min_read_recency_for_promote": 0, 2023-10-03T10:12:39.392 INFO:teuthology.orchestra.run.smithi003.stdout: "min_write_recency_for_promote": 0, 2023-10-03T10:12:39.392 INFO:teuthology.orchestra.run.smithi003.stdout: "hit_set_grade_decay_rate": 0, 2023-10-03T10:12:39.392 INFO:teuthology.orchestra.run.smithi003.stdout: "hit_set_search_last_n": 0, 2023-10-03T10:12:39.392 INFO:teuthology.orchestra.run.smithi003.stdout: "grade_table": [], 2023-10-03T10:12:39.392 INFO:teuthology.orchestra.run.smithi003.stdout: "stripe_width": 0, 2023-10-03T10:12:39.392 INFO:teuthology.orchestra.run.smithi003.stdout: "expected_num_objects": 0, 2023-10-03T10:12:39.392 INFO:teuthology.orchestra.run.smithi003.stdout: "fast_read": false, 2023-10-03T10:12:39.392 INFO:teuthology.orchestra.run.smithi003.stdout: "options": { 2023-10-03T10:12:39.392 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscale_bias": 4, 2023-10-03T10:12:39.392 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num_min": 16, 2023-10-03T10:12:39.392 INFO:teuthology.orchestra.run.smithi003.stdout: "recovery_priority": 5 2023-10-03T10:12:39.393 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.393 INFO:teuthology.orchestra.run.smithi003.stdout: "application_metadata": { 2023-10-03T10:12:39.393 INFO:teuthology.orchestra.run.smithi003.stdout: "cephfs": { 2023-10-03T10:12:39.393 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": "cephfs" 2023-10-03T10:12:39.393 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.393 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.393 INFO:teuthology.orchestra.run.smithi003.stdout: "read_balance": { 2023-10-03T10:12:39.393 INFO:teuthology.orchestra.run.smithi003.stdout: "score_acting": 1.6299999952316284, 2023-10-03T10:12:39.393 INFO:teuthology.orchestra.run.smithi003.stdout: "score_stable": 1.6299999952316284, 2023-10-03T10:12:39.393 INFO:teuthology.orchestra.run.smithi003.stdout: "optimal_score": 1, 2023-10-03T10:12:39.393 INFO:teuthology.orchestra.run.smithi003.stdout: "raw_score_acting": 1.6299999952316284, 2023-10-03T10:12:39.393 INFO:teuthology.orchestra.run.smithi003.stdout: "raw_score_stable": 1.6299999952316284, 2023-10-03T10:12:39.393 INFO:teuthology.orchestra.run.smithi003.stdout: "primary_affinity_weighted": 1, 2023-10-03T10:12:39.393 INFO:teuthology.orchestra.run.smithi003.stdout: "average_primary_affinity": 1, 2023-10-03T10:12:39.394 INFO:teuthology.orchestra.run.smithi003.stdout: "average_primary_affinity_weighted": 1 2023-10-03T10:12:39.394 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.394 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.394 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.394 INFO:teuthology.orchestra.run.smithi003.stdout: "pool": 4, 2023-10-03T10:12:39.394 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_name": "cephfs_data", 2023-10-03T10:12:39.394 INFO:teuthology.orchestra.run.smithi003.stdout: "create_time": "2023-10-03T10:09:43.115588+0000", 2023-10-03T10:12:39.394 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-10-03T10:12:39.394 INFO:teuthology.orchestra.run.smithi003.stdout: "flags_names": "hashpspool", 2023-10-03T10:12:39.394 INFO:teuthology.orchestra.run.smithi003.stdout: "type": 1, 2023-10-03T10:12:39.394 INFO:teuthology.orchestra.run.smithi003.stdout: "size": 2, 2023-10-03T10:12:39.394 INFO:teuthology.orchestra.run.smithi003.stdout: "min_size": 1, 2023-10-03T10:12:39.394 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_rule": 0, 2023-10-03T10:12:39.395 INFO:teuthology.orchestra.run.smithi003.stdout: "peering_crush_bucket_count": 0, 2023-10-03T10:12:39.395 INFO:teuthology.orchestra.run.smithi003.stdout: "peering_crush_bucket_target": 0, 2023-10-03T10:12:39.395 INFO:teuthology.orchestra.run.smithi003.stdout: "peering_crush_bucket_barrier": 0, 2023-10-03T10:12:39.395 INFO:teuthology.orchestra.run.smithi003.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-03T10:12:39.395 INFO:teuthology.orchestra.run.smithi003.stdout: "object_hash": 2, 2023-10-03T10:12:39.395 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscale_mode": "off", 2023-10-03T10:12:39.395 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num": 64, 2023-10-03T10:12:39.395 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_placement_num": 64, 2023-10-03T10:12:39.395 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_placement_num_target": 64, 2023-10-03T10:12:39.395 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num_target": 64, 2023-10-03T10:12:39.395 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num_pending": 64, 2023-10-03T10:12:39.395 INFO:teuthology.orchestra.run.smithi003.stdout: "last_pg_merge_meta": { 2023-10-03T10:12:39.395 INFO:teuthology.orchestra.run.smithi003.stdout: "source_pgid": "0.0", 2023-10-03T10:12:39.396 INFO:teuthology.orchestra.run.smithi003.stdout: "ready_epoch": 0, 2023-10-03T10:12:39.396 INFO:teuthology.orchestra.run.smithi003.stdout: "last_epoch_started": 0, 2023-10-03T10:12:39.396 INFO:teuthology.orchestra.run.smithi003.stdout: "last_epoch_clean": 0, 2023-10-03T10:12:39.396 INFO:teuthology.orchestra.run.smithi003.stdout: "source_version": "0'0", 2023-10-03T10:12:39.396 INFO:teuthology.orchestra.run.smithi003.stdout: "target_version": "0'0" 2023-10-03T10:12:39.396 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.396 INFO:teuthology.orchestra.run.smithi003.stdout: "last_change": "17", 2023-10-03T10:12:39.396 INFO:teuthology.orchestra.run.smithi003.stdout: "last_force_op_resend": "0", 2023-10-03T10:12:39.396 INFO:teuthology.orchestra.run.smithi003.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-03T10:12:39.396 INFO:teuthology.orchestra.run.smithi003.stdout: "last_force_op_resend_preluminous": "0", 2023-10-03T10:12:39.396 INFO:teuthology.orchestra.run.smithi003.stdout: "auid": 0, 2023-10-03T10:12:39.396 INFO:teuthology.orchestra.run.smithi003.stdout: "snap_mode": "selfmanaged", 2023-10-03T10:12:39.397 INFO:teuthology.orchestra.run.smithi003.stdout: "snap_seq": 0, 2023-10-03T10:12:39.397 INFO:teuthology.orchestra.run.smithi003.stdout: "snap_epoch": 0, 2023-10-03T10:12:39.397 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_snaps": [], 2023-10-03T10:12:39.397 INFO:teuthology.orchestra.run.smithi003.stdout: "removed_snaps": "[]", 2023-10-03T10:12:39.397 INFO:teuthology.orchestra.run.smithi003.stdout: "quota_max_bytes": 0, 2023-10-03T10:12:39.397 INFO:teuthology.orchestra.run.smithi003.stdout: "quota_max_objects": 0, 2023-10-03T10:12:39.397 INFO:teuthology.orchestra.run.smithi003.stdout: "tiers": [], 2023-10-03T10:12:39.397 INFO:teuthology.orchestra.run.smithi003.stdout: "tier_of": -1, 2023-10-03T10:12:39.397 INFO:teuthology.orchestra.run.smithi003.stdout: "read_tier": -1, 2023-10-03T10:12:39.397 INFO:teuthology.orchestra.run.smithi003.stdout: "write_tier": -1, 2023-10-03T10:12:39.397 INFO:teuthology.orchestra.run.smithi003.stdout: "cache_mode": "none", 2023-10-03T10:12:39.397 INFO:teuthology.orchestra.run.smithi003.stdout: "target_max_bytes": 0, 2023-10-03T10:12:39.397 INFO:teuthology.orchestra.run.smithi003.stdout: "target_max_objects": 0, 2023-10-03T10:12:39.398 INFO:teuthology.orchestra.run.smithi003.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-03T10:12:39.398 INFO:teuthology.orchestra.run.smithi003.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-03T10:12:39.398 INFO:teuthology.orchestra.run.smithi003.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-03T10:12:39.398 INFO:teuthology.orchestra.run.smithi003.stdout: "cache_min_flush_age": 0, 2023-10-03T10:12:39.398 INFO:teuthology.orchestra.run.smithi003.stdout: "cache_min_evict_age": 0, 2023-10-03T10:12:39.398 INFO:teuthology.orchestra.run.smithi003.stdout: "erasure_code_profile": "", 2023-10-03T10:12:39.398 INFO:teuthology.orchestra.run.smithi003.stdout: "hit_set_params": { 2023-10-03T10:12:39.398 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "none" 2023-10-03T10:12:39.398 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.398 INFO:teuthology.orchestra.run.smithi003.stdout: "hit_set_period": 0, 2023-10-03T10:12:39.398 INFO:teuthology.orchestra.run.smithi003.stdout: "hit_set_count": 0, 2023-10-03T10:12:39.398 INFO:teuthology.orchestra.run.smithi003.stdout: "use_gmt_hitset": true, 2023-10-03T10:12:39.398 INFO:teuthology.orchestra.run.smithi003.stdout: "min_read_recency_for_promote": 0, 2023-10-03T10:12:39.399 INFO:teuthology.orchestra.run.smithi003.stdout: "min_write_recency_for_promote": 0, 2023-10-03T10:12:39.399 INFO:teuthology.orchestra.run.smithi003.stdout: "hit_set_grade_decay_rate": 0, 2023-10-03T10:12:39.399 INFO:teuthology.orchestra.run.smithi003.stdout: "hit_set_search_last_n": 0, 2023-10-03T10:12:39.399 INFO:teuthology.orchestra.run.smithi003.stdout: "grade_table": [], 2023-10-03T10:12:39.399 INFO:teuthology.orchestra.run.smithi003.stdout: "stripe_width": 0, 2023-10-03T10:12:39.399 INFO:teuthology.orchestra.run.smithi003.stdout: "expected_num_objects": 0, 2023-10-03T10:12:39.399 INFO:teuthology.orchestra.run.smithi003.stdout: "fast_read": false, 2023-10-03T10:12:39.399 INFO:teuthology.orchestra.run.smithi003.stdout: "options": { 2023-10-03T10:12:39.399 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num_min": 64, 2023-10-03T10:12:39.399 INFO:teuthology.orchestra.run.smithi003.stdout: "target_size_ratio": 0.89999997615814209 2023-10-03T10:12:39.399 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.399 INFO:teuthology.orchestra.run.smithi003.stdout: "application_metadata": { 2023-10-03T10:12:39.399 INFO:teuthology.orchestra.run.smithi003.stdout: "cephfs": { 2023-10-03T10:12:39.400 INFO:teuthology.orchestra.run.smithi003.stdout: "data": "cephfs" 2023-10-03T10:12:39.400 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.400 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.400 INFO:teuthology.orchestra.run.smithi003.stdout: "read_balance": { 2023-10-03T10:12:39.400 INFO:teuthology.orchestra.run.smithi003.stdout: "score_acting": 1.059999942779541, 2023-10-03T10:12:39.400 INFO:teuthology.orchestra.run.smithi003.stdout: "score_stable": 1.059999942779541, 2023-10-03T10:12:39.400 INFO:teuthology.orchestra.run.smithi003.stdout: "optimal_score": 1, 2023-10-03T10:12:39.400 INFO:teuthology.orchestra.run.smithi003.stdout: "raw_score_acting": 1.059999942779541, 2023-10-03T10:12:39.400 INFO:teuthology.orchestra.run.smithi003.stdout: "raw_score_stable": 1.059999942779541, 2023-10-03T10:12:39.400 INFO:teuthology.orchestra.run.smithi003.stdout: "primary_affinity_weighted": 1, 2023-10-03T10:12:39.400 INFO:teuthology.orchestra.run.smithi003.stdout: "average_primary_affinity": 1, 2023-10-03T10:12:39.400 INFO:teuthology.orchestra.run.smithi003.stdout: "average_primary_affinity_weighted": 1 2023-10-03T10:12:39.400 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.401 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.401 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-10-03T10:12:39.401 INFO:teuthology.orchestra.run.smithi003.stdout: "osds": [ 2023-10-03T10:12:39.401 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.401 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": 0, 2023-10-03T10:12:39.401 INFO:teuthology.orchestra.run.smithi003.stdout: "uuid": "cdcdd930-3c5b-4486-a021-7132f90fb6e6", 2023-10-03T10:12:39.401 INFO:teuthology.orchestra.run.smithi003.stdout: "up": 1, 2023-10-03T10:12:39.401 INFO:teuthology.orchestra.run.smithi003.stdout: "in": 1, 2023-10-03T10:12:39.401 INFO:teuthology.orchestra.run.smithi003.stdout: "weight": 1, 2023-10-03T10:12:39.401 INFO:teuthology.orchestra.run.smithi003.stdout: "primary_affinity": 1, 2023-10-03T10:12:39.401 INFO:teuthology.orchestra.run.smithi003.stdout: "last_clean_begin": 9, 2023-10-03T10:12:39.401 INFO:teuthology.orchestra.run.smithi003.stdout: "last_clean_end": 23, 2023-10-03T10:12:39.401 INFO:teuthology.orchestra.run.smithi003.stdout: "up_from": 29, 2023-10-03T10:12:39.402 INFO:teuthology.orchestra.run.smithi003.stdout: "up_thru": 32, 2023-10-03T10:12:39.402 INFO:teuthology.orchestra.run.smithi003.stdout: "down_at": 24, 2023-10-03T10:12:39.402 INFO:teuthology.orchestra.run.smithi003.stdout: "lost_at": 0, 2023-10-03T10:12:39.402 INFO:teuthology.orchestra.run.smithi003.stdout: "public_addrs": { 2023-10-03T10:12:39.402 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2023-10-03T10:12:39.402 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.402 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-03T10:12:39.402 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6804", 2023-10-03T10:12:39.402 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 129556 2023-10-03T10:12:39.402 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.402 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.402 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.402 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_addrs": { 2023-10-03T10:12:39.403 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2023-10-03T10:12:39.403 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.403 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-03T10:12:39.403 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6805", 2023-10-03T10:12:39.403 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 129556 2023-10-03T10:12:39.403 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.403 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.403 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.403 INFO:teuthology.orchestra.run.smithi003.stdout: "heartbeat_back_addrs": { 2023-10-03T10:12:39.403 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2023-10-03T10:12:39.403 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.403 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-03T10:12:39.403 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6807", 2023-10-03T10:12:39.404 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 129556 2023-10-03T10:12:39.404 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.404 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.404 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.404 INFO:teuthology.orchestra.run.smithi003.stdout: "heartbeat_front_addrs": { 2023-10-03T10:12:39.404 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2023-10-03T10:12:39.404 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.404 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-03T10:12:39.404 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6806", 2023-10-03T10:12:39.404 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 129556 2023-10-03T10:12:39.404 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.404 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.404 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.405 INFO:teuthology.orchestra.run.smithi003.stdout: "public_addr": "172.21.15.3:6804/129556", 2023-10-03T10:12:39.405 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_addr": "172.21.15.3:6805/129556", 2023-10-03T10:12:39.405 INFO:teuthology.orchestra.run.smithi003.stdout: "heartbeat_back_addr": "172.21.15.3:6807/129556", 2023-10-03T10:12:39.405 INFO:teuthology.orchestra.run.smithi003.stdout: "heartbeat_front_addr": "172.21.15.3:6806/129556", 2023-10-03T10:12:39.405 INFO:teuthology.orchestra.run.smithi003.stdout: "state": [ 2023-10-03T10:12:39.405 INFO:teuthology.orchestra.run.smithi003.stdout: "exists", 2023-10-03T10:12:39.405 INFO:teuthology.orchestra.run.smithi003.stdout: "up" 2023-10-03T10:12:39.405 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.405 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.405 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.405 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": 1, 2023-10-03T10:12:39.405 INFO:teuthology.orchestra.run.smithi003.stdout: "uuid": "c8e58877-6762-4741-a8c1-567ebc97b708", 2023-10-03T10:12:39.405 INFO:teuthology.orchestra.run.smithi003.stdout: "up": 1, 2023-10-03T10:12:39.406 INFO:teuthology.orchestra.run.smithi003.stdout: "in": 1, 2023-10-03T10:12:39.406 INFO:teuthology.orchestra.run.smithi003.stdout: "weight": 1, 2023-10-03T10:12:39.406 INFO:teuthology.orchestra.run.smithi003.stdout: "primary_affinity": 1, 2023-10-03T10:12:39.406 INFO:teuthology.orchestra.run.smithi003.stdout: "last_clean_begin": 9, 2023-10-03T10:12:39.406 INFO:teuthology.orchestra.run.smithi003.stdout: "last_clean_end": 23, 2023-10-03T10:12:39.406 INFO:teuthology.orchestra.run.smithi003.stdout: "up_from": 31, 2023-10-03T10:12:39.406 INFO:teuthology.orchestra.run.smithi003.stdout: "up_thru": 32, 2023-10-03T10:12:39.406 INFO:teuthology.orchestra.run.smithi003.stdout: "down_at": 25, 2023-10-03T10:12:39.406 INFO:teuthology.orchestra.run.smithi003.stdout: "lost_at": 0, 2023-10-03T10:12:39.406 INFO:teuthology.orchestra.run.smithi003.stdout: "public_addrs": { 2023-10-03T10:12:39.406 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2023-10-03T10:12:39.406 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.406 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-03T10:12:39.407 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6808", 2023-10-03T10:12:39.407 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 129654 2023-10-03T10:12:39.407 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.407 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.407 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.407 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_addrs": { 2023-10-03T10:12:39.407 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2023-10-03T10:12:39.407 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.407 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-03T10:12:39.407 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6809", 2023-10-03T10:12:39.407 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 129654 2023-10-03T10:12:39.407 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.407 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.408 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.408 INFO:teuthology.orchestra.run.smithi003.stdout: "heartbeat_back_addrs": { 2023-10-03T10:12:39.408 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2023-10-03T10:12:39.408 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.408 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-03T10:12:39.408 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6811", 2023-10-03T10:12:39.408 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 129654 2023-10-03T10:12:39.408 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.408 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.408 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.408 INFO:teuthology.orchestra.run.smithi003.stdout: "heartbeat_front_addrs": { 2023-10-03T10:12:39.408 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2023-10-03T10:12:39.408 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.408 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-03T10:12:39.409 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6810", 2023-10-03T10:12:39.409 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 129654 2023-10-03T10:12:39.409 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.409 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.409 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.409 INFO:teuthology.orchestra.run.smithi003.stdout: "public_addr": "172.21.15.3:6808/129654", 2023-10-03T10:12:39.409 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_addr": "172.21.15.3:6809/129654", 2023-10-03T10:12:39.409 INFO:teuthology.orchestra.run.smithi003.stdout: "heartbeat_back_addr": "172.21.15.3:6811/129654", 2023-10-03T10:12:39.409 INFO:teuthology.orchestra.run.smithi003.stdout: "heartbeat_front_addr": "172.21.15.3:6810/129654", 2023-10-03T10:12:39.409 INFO:teuthology.orchestra.run.smithi003.stdout: "state": [ 2023-10-03T10:12:39.409 INFO:teuthology.orchestra.run.smithi003.stdout: "exists", 2023-10-03T10:12:39.409 INFO:teuthology.orchestra.run.smithi003.stdout: "up" 2023-10-03T10:12:39.409 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.410 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.410 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.410 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": 2, 2023-10-03T10:12:39.410 INFO:teuthology.orchestra.run.smithi003.stdout: "uuid": "68a2a21a-7a8e-4f82-8433-5201faa50777", 2023-10-03T10:12:39.410 INFO:teuthology.orchestra.run.smithi003.stdout: "up": 1, 2023-10-03T10:12:39.410 INFO:teuthology.orchestra.run.smithi003.stdout: "in": 1, 2023-10-03T10:12:39.410 INFO:teuthology.orchestra.run.smithi003.stdout: "weight": 1, 2023-10-03T10:12:39.410 INFO:teuthology.orchestra.run.smithi003.stdout: "primary_affinity": 1, 2023-10-03T10:12:39.410 INFO:teuthology.orchestra.run.smithi003.stdout: "last_clean_begin": 9, 2023-10-03T10:12:39.410 INFO:teuthology.orchestra.run.smithi003.stdout: "last_clean_end": 25, 2023-10-03T10:12:39.410 INFO:teuthology.orchestra.run.smithi003.stdout: "up_from": 31, 2023-10-03T10:12:39.410 INFO:teuthology.orchestra.run.smithi003.stdout: "up_thru": 32, 2023-10-03T10:12:39.411 INFO:teuthology.orchestra.run.smithi003.stdout: "down_at": 26, 2023-10-03T10:12:39.411 INFO:teuthology.orchestra.run.smithi003.stdout: "lost_at": 0, 2023-10-03T10:12:39.411 INFO:teuthology.orchestra.run.smithi003.stdout: "public_addrs": { 2023-10-03T10:12:39.411 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2023-10-03T10:12:39.411 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.411 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-03T10:12:39.411 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6800", 2023-10-03T10:12:39.411 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 129764 2023-10-03T10:12:39.411 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.411 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.411 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.411 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_addrs": { 2023-10-03T10:12:39.411 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2023-10-03T10:12:39.412 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.412 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-03T10:12:39.412 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6801", 2023-10-03T10:12:39.412 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 129764 2023-10-03T10:12:39.412 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.412 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.412 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.412 INFO:teuthology.orchestra.run.smithi003.stdout: "heartbeat_back_addrs": { 2023-10-03T10:12:39.412 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2023-10-03T10:12:39.412 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.412 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-03T10:12:39.412 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6803", 2023-10-03T10:12:39.412 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 129764 2023-10-03T10:12:39.412 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.413 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.413 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.413 INFO:teuthology.orchestra.run.smithi003.stdout: "heartbeat_front_addrs": { 2023-10-03T10:12:39.413 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2023-10-03T10:12:39.413 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.413 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-03T10:12:39.413 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6802", 2023-10-03T10:12:39.413 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 129764 2023-10-03T10:12:39.413 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.413 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.413 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.413 INFO:teuthology.orchestra.run.smithi003.stdout: "public_addr": "172.21.15.3:6800/129764", 2023-10-03T10:12:39.414 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_addr": "172.21.15.3:6801/129764", 2023-10-03T10:12:39.414 INFO:teuthology.orchestra.run.smithi003.stdout: "heartbeat_back_addr": "172.21.15.3:6803/129764", 2023-10-03T10:12:39.414 INFO:teuthology.orchestra.run.smithi003.stdout: "heartbeat_front_addr": "172.21.15.3:6802/129764", 2023-10-03T10:12:39.414 INFO:teuthology.orchestra.run.smithi003.stdout: "state": [ 2023-10-03T10:12:39.414 INFO:teuthology.orchestra.run.smithi003.stdout: "exists", 2023-10-03T10:12:39.414 INFO:teuthology.orchestra.run.smithi003.stdout: "up" 2023-10-03T10:12:39.414 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.414 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.414 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.414 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": 3, 2023-10-03T10:12:39.414 INFO:teuthology.orchestra.run.smithi003.stdout: "uuid": "c4df2903-4e1e-4208-bf77-95b9fa749241", 2023-10-03T10:12:39.414 INFO:teuthology.orchestra.run.smithi003.stdout: "up": 1, 2023-10-03T10:12:39.414 INFO:teuthology.orchestra.run.smithi003.stdout: "in": 1, 2023-10-03T10:12:39.415 INFO:teuthology.orchestra.run.smithi003.stdout: "weight": 1, 2023-10-03T10:12:39.415 INFO:teuthology.orchestra.run.smithi003.stdout: "primary_affinity": 1, 2023-10-03T10:12:39.415 INFO:teuthology.orchestra.run.smithi003.stdout: "last_clean_begin": 9, 2023-10-03T10:12:39.415 INFO:teuthology.orchestra.run.smithi003.stdout: "last_clean_end": 26, 2023-10-03T10:12:39.415 INFO:teuthology.orchestra.run.smithi003.stdout: "up_from": 32, 2023-10-03T10:12:39.415 INFO:teuthology.orchestra.run.smithi003.stdout: "up_thru": 32, 2023-10-03T10:12:39.415 INFO:teuthology.orchestra.run.smithi003.stdout: "down_at": 27, 2023-10-03T10:12:39.415 INFO:teuthology.orchestra.run.smithi003.stdout: "lost_at": 0, 2023-10-03T10:12:39.415 INFO:teuthology.orchestra.run.smithi003.stdout: "public_addrs": { 2023-10-03T10:12:39.415 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2023-10-03T10:12:39.415 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.415 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-03T10:12:39.415 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6812", 2023-10-03T10:12:39.416 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 129912 2023-10-03T10:12:39.416 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.416 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.416 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.416 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_addrs": { 2023-10-03T10:12:39.416 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2023-10-03T10:12:39.416 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.416 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-03T10:12:39.416 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6813", 2023-10-03T10:12:39.416 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 129912 2023-10-03T10:12:39.416 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.416 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.416 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.417 INFO:teuthology.orchestra.run.smithi003.stdout: "heartbeat_back_addrs": { 2023-10-03T10:12:39.417 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2023-10-03T10:12:39.417 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.417 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-03T10:12:39.417 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6815", 2023-10-03T10:12:39.417 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 129912 2023-10-03T10:12:39.417 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.417 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.417 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.417 INFO:teuthology.orchestra.run.smithi003.stdout: "heartbeat_front_addrs": { 2023-10-03T10:12:39.417 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2023-10-03T10:12:39.417 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.417 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-10-03T10:12:39.418 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6814", 2023-10-03T10:12:39.418 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 129912 2023-10-03T10:12:39.418 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.418 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.418 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.418 INFO:teuthology.orchestra.run.smithi003.stdout: "public_addr": "172.21.15.3:6812/129912", 2023-10-03T10:12:39.418 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_addr": "172.21.15.3:6813/129912", 2023-10-03T10:12:39.418 INFO:teuthology.orchestra.run.smithi003.stdout: "heartbeat_back_addr": "172.21.15.3:6815/129912", 2023-10-03T10:12:39.418 INFO:teuthology.orchestra.run.smithi003.stdout: "heartbeat_front_addr": "172.21.15.3:6814/129912", 2023-10-03T10:12:39.418 INFO:teuthology.orchestra.run.smithi003.stdout: "state": [ 2023-10-03T10:12:39.418 INFO:teuthology.orchestra.run.smithi003.stdout: "exists", 2023-10-03T10:12:39.418 INFO:teuthology.orchestra.run.smithi003.stdout: "up" 2023-10-03T10:12:39.419 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.419 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.419 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-10-03T10:12:39.419 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_xinfo": [ 2023-10-03T10:12:39.419 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.419 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": 0, 2023-10-03T10:12:39.419 INFO:teuthology.orchestra.run.smithi003.stdout: "down_stamp": "0.000000", 2023-10-03T10:12:39.419 INFO:teuthology.orchestra.run.smithi003.stdout: "laggy_probability": 0, 2023-10-03T10:12:39.419 INFO:teuthology.orchestra.run.smithi003.stdout: "laggy_interval": 0, 2023-10-03T10:12:39.419 INFO:teuthology.orchestra.run.smithi003.stdout: "features": 4540138322906710015, 2023-10-03T10:12:39.419 INFO:teuthology.orchestra.run.smithi003.stdout: "old_weight": 0, 2023-10-03T10:12:39.419 INFO:teuthology.orchestra.run.smithi003.stdout: "last_purged_snaps_scrub": "2023-10-03T10:09:35.369708+0000", 2023-10-03T10:12:39.419 INFO:teuthology.orchestra.run.smithi003.stdout: "dead_epoch": 24 2023-10-03T10:12:39.420 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.420 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.420 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": 1, 2023-10-03T10:12:39.420 INFO:teuthology.orchestra.run.smithi003.stdout: "down_stamp": "0.000000", 2023-10-03T10:12:39.420 INFO:teuthology.orchestra.run.smithi003.stdout: "laggy_probability": 0, 2023-10-03T10:12:39.420 INFO:teuthology.orchestra.run.smithi003.stdout: "laggy_interval": 0, 2023-10-03T10:12:39.420 INFO:teuthology.orchestra.run.smithi003.stdout: "features": 4540138322906710015, 2023-10-03T10:12:39.420 INFO:teuthology.orchestra.run.smithi003.stdout: "old_weight": 0, 2023-10-03T10:12:39.420 INFO:teuthology.orchestra.run.smithi003.stdout: "last_purged_snaps_scrub": "2023-10-03T10:09:35.257825+0000", 2023-10-03T10:12:39.420 INFO:teuthology.orchestra.run.smithi003.stdout: "dead_epoch": 25 2023-10-03T10:12:39.420 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.420 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.421 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": 2, 2023-10-03T10:12:39.421 INFO:teuthology.orchestra.run.smithi003.stdout: "down_stamp": "0.000000", 2023-10-03T10:12:39.421 INFO:teuthology.orchestra.run.smithi003.stdout: "laggy_probability": 0, 2023-10-03T10:12:39.421 INFO:teuthology.orchestra.run.smithi003.stdout: "laggy_interval": 0, 2023-10-03T10:12:39.421 INFO:teuthology.orchestra.run.smithi003.stdout: "features": 4540138322906710015, 2023-10-03T10:12:39.421 INFO:teuthology.orchestra.run.smithi003.stdout: "old_weight": 0, 2023-10-03T10:12:39.421 INFO:teuthology.orchestra.run.smithi003.stdout: "last_purged_snaps_scrub": "2023-10-03T10:09:35.337931+0000", 2023-10-03T10:12:39.421 INFO:teuthology.orchestra.run.smithi003.stdout: "dead_epoch": 26 2023-10-03T10:12:39.421 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.421 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.421 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": 3, 2023-10-03T10:12:39.421 INFO:teuthology.orchestra.run.smithi003.stdout: "down_stamp": "2023-10-03T10:12:26.212430+0000", 2023-10-03T10:12:39.421 INFO:teuthology.orchestra.run.smithi003.stdout: "laggy_probability": 0, 2023-10-03T10:12:39.422 INFO:teuthology.orchestra.run.smithi003.stdout: "laggy_interval": 0, 2023-10-03T10:12:39.422 INFO:teuthology.orchestra.run.smithi003.stdout: "features": 4540138322906710015, 2023-10-03T10:12:39.422 INFO:teuthology.orchestra.run.smithi003.stdout: "old_weight": 0, 2023-10-03T10:12:39.422 INFO:teuthology.orchestra.run.smithi003.stdout: "last_purged_snaps_scrub": "2023-10-03T10:09:35.085222+0000", 2023-10-03T10:12:39.422 INFO:teuthology.orchestra.run.smithi003.stdout: "dead_epoch": 0 2023-10-03T10:12:39.422 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.422 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-10-03T10:12:39.422 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_upmap": [], 2023-10-03T10:12:39.422 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_upmap_items": [ 2023-10-03T10:12:39.422 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.422 INFO:teuthology.orchestra.run.smithi003.stdout: "pgid": "4.1", 2023-10-03T10:12:39.422 INFO:teuthology.orchestra.run.smithi003.stdout: "mappings": [ 2023-10-03T10:12:39.422 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.423 INFO:teuthology.orchestra.run.smithi003.stdout: "from": 3, 2023-10-03T10:12:39.423 INFO:teuthology.orchestra.run.smithi003.stdout: "to": 0 2023-10-03T10:12:39.423 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.423 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.423 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.423 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.423 INFO:teuthology.orchestra.run.smithi003.stdout: "pgid": "4.8", 2023-10-03T10:12:39.423 INFO:teuthology.orchestra.run.smithi003.stdout: "mappings": [ 2023-10-03T10:12:39.423 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.423 INFO:teuthology.orchestra.run.smithi003.stdout: "from": 1, 2023-10-03T10:12:39.423 INFO:teuthology.orchestra.run.smithi003.stdout: "to": 2 2023-10-03T10:12:39.423 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.424 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.424 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.424 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.424 INFO:teuthology.orchestra.run.smithi003.stdout: "pgid": "4.d", 2023-10-03T10:12:39.424 INFO:teuthology.orchestra.run.smithi003.stdout: "mappings": [ 2023-10-03T10:12:39.424 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.424 INFO:teuthology.orchestra.run.smithi003.stdout: "from": 3, 2023-10-03T10:12:39.424 INFO:teuthology.orchestra.run.smithi003.stdout: "to": 2 2023-10-03T10:12:39.424 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.424 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.424 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.424 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.424 INFO:teuthology.orchestra.run.smithi003.stdout: "pgid": "4.14", 2023-10-03T10:12:39.424 INFO:teuthology.orchestra.run.smithi003.stdout: "mappings": [ 2023-10-03T10:12:39.425 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.425 INFO:teuthology.orchestra.run.smithi003.stdout: "from": 3, 2023-10-03T10:12:39.425 INFO:teuthology.orchestra.run.smithi003.stdout: "to": 2 2023-10-03T10:12:39.425 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.425 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.425 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.425 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.425 INFO:teuthology.orchestra.run.smithi003.stdout: "pgid": "4.19", 2023-10-03T10:12:39.425 INFO:teuthology.orchestra.run.smithi003.stdout: "mappings": [ 2023-10-03T10:12:39.425 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.425 INFO:teuthology.orchestra.run.smithi003.stdout: "from": 3, 2023-10-03T10:12:39.425 INFO:teuthology.orchestra.run.smithi003.stdout: "to": 0 2023-10-03T10:12:39.425 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.426 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.426 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.426 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.426 INFO:teuthology.orchestra.run.smithi003.stdout: "pgid": "4.2a", 2023-10-03T10:12:39.426 INFO:teuthology.orchestra.run.smithi003.stdout: "mappings": [ 2023-10-03T10:12:39.426 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.426 INFO:teuthology.orchestra.run.smithi003.stdout: "from": 1, 2023-10-03T10:12:39.426 INFO:teuthology.orchestra.run.smithi003.stdout: "to": 2 2023-10-03T10:12:39.426 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.426 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.426 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.426 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.426 INFO:teuthology.orchestra.run.smithi003.stdout: "pgid": "4.2d", 2023-10-03T10:12:39.427 INFO:teuthology.orchestra.run.smithi003.stdout: "mappings": [ 2023-10-03T10:12:39.427 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.427 INFO:teuthology.orchestra.run.smithi003.stdout: "from": 3, 2023-10-03T10:12:39.427 INFO:teuthology.orchestra.run.smithi003.stdout: "to": 0 2023-10-03T10:12:39.427 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.427 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.427 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.427 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.427 INFO:teuthology.orchestra.run.smithi003.stdout: "pgid": "4.33", 2023-10-03T10:12:39.427 INFO:teuthology.orchestra.run.smithi003.stdout: "mappings": [ 2023-10-03T10:12:39.427 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.427 INFO:teuthology.orchestra.run.smithi003.stdout: "from": 3, 2023-10-03T10:12:39.427 INFO:teuthology.orchestra.run.smithi003.stdout: "to": 0 2023-10-03T10:12:39.428 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.428 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.428 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.428 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.428 INFO:teuthology.orchestra.run.smithi003.stdout: "pgid": "4.39", 2023-10-03T10:12:39.428 INFO:teuthology.orchestra.run.smithi003.stdout: "mappings": [ 2023-10-03T10:12:39.428 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-10-03T10:12:39.428 INFO:teuthology.orchestra.run.smithi003.stdout: "from": 3, 2023-10-03T10:12:39.428 INFO:teuthology.orchestra.run.smithi003.stdout: "to": 2 2023-10-03T10:12:39.428 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.428 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-10-03T10:12:39.428 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.428 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-10-03T10:12:39.429 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_upmap_primaries": [], 2023-10-03T10:12:39.429 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_temp": [], 2023-10-03T10:12:39.429 INFO:teuthology.orchestra.run.smithi003.stdout: "primary_temp": [], 2023-10-03T10:12:39.429 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklist": { 2023-10-03T10:12:39.429 INFO:teuthology.orchestra.run.smithi003.stdout: "172.21.15.3:0/3919724564": "2023-10-04T10:12:13.305411+0000", 2023-10-03T10:12:39.429 INFO:teuthology.orchestra.run.smithi003.stdout: "172.21.15.3:6817/2582564779": "2023-10-04T10:12:38.734689+0000", 2023-10-03T10:12:39.429 INFO:teuthology.orchestra.run.smithi003.stdout: "172.21.15.3:0/2091814084": "2023-10-04T10:12:13.305411+0000", 2023-10-03T10:12:39.429 INFO:teuthology.orchestra.run.smithi003.stdout: "172.21.15.3:6816/118296": "2023-10-04T10:12:13.305411+0000" 2023-10-03T10:12:39.429 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.429 INFO:teuthology.orchestra.run.smithi003.stdout: "range_blocklist": {}, 2023-10-03T10:12:39.429 INFO:teuthology.orchestra.run.smithi003.stdout: "erasure_code_profiles": { 2023-10-03T10:12:39.429 INFO:teuthology.orchestra.run.smithi003.stdout: "default": { 2023-10-03T10:12:39.429 INFO:teuthology.orchestra.run.smithi003.stdout: "crush-failure-domain": "osd", 2023-10-03T10:12:39.430 INFO:teuthology.orchestra.run.smithi003.stdout: "k": "2", 2023-10-03T10:12:39.430 INFO:teuthology.orchestra.run.smithi003.stdout: "m": "1", 2023-10-03T10:12:39.430 INFO:teuthology.orchestra.run.smithi003.stdout: "plugin": "jerasure", 2023-10-03T10:12:39.430 INFO:teuthology.orchestra.run.smithi003.stdout: "technique": "reed_sol_van" 2023-10-03T10:12:39.430 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.430 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.430 INFO:teuthology.orchestra.run.smithi003.stdout: "removed_snaps_queue": [], 2023-10-03T10:12:39.430 INFO:teuthology.orchestra.run.smithi003.stdout: "new_removed_snaps": [], 2023-10-03T10:12:39.430 INFO:teuthology.orchestra.run.smithi003.stdout: "new_purged_snaps": [], 2023-10-03T10:12:39.430 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_node_flags": {}, 2023-10-03T10:12:39.430 INFO:teuthology.orchestra.run.smithi003.stdout: "device_class_flags": {}, 2023-10-03T10:12:39.430 INFO:teuthology.orchestra.run.smithi003.stdout: "stretch_mode": { 2023-10-03T10:12:39.430 INFO:teuthology.orchestra.run.smithi003.stdout: "stretch_mode_enabled": false, 2023-10-03T10:12:39.431 INFO:teuthology.orchestra.run.smithi003.stdout: "stretch_bucket_count": 0, 2023-10-03T10:12:39.431 INFO:teuthology.orchestra.run.smithi003.stdout: "degraded_stretch_mode": 0, 2023-10-03T10:12:39.431 INFO:teuthology.orchestra.run.smithi003.stdout: "recovering_stretch_mode": 0, 2023-10-03T10:12:39.431 INFO:teuthology.orchestra.run.smithi003.stdout: "stretch_mode_bucket": 0 2023-10-03T10:12:39.431 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.431 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-03T10:12:39.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.373+0000 7fdb2458b700 1 -- 172.21.15.3:0/481959842 >> v1:172.21.15.3:6816/127025 conn(0x7fdb0002f510 legacy=0x7fdb000319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:39.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.373+0000 7fdb2458b700 1 -- 172.21.15.3:0/481959842 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb1c09e9d0 msgr2=0x7fdb1c120c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:39.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.373+0000 7fdb2458b700 1 --2- 172.21.15.3:0/481959842 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb1c09e9d0 0x7fdb1c120c10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00ebb0 tx=0x7fdb0c00eec0 comp rx=0 tx=0).stop 2023-10-03T10:12:39.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.373+0000 7fdb2458b700 1 -- 172.21.15.3:0/481959842 shutdown_connections 2023-10-03T10:12:39.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.373+0000 7fdb2458b700 1 --2- 172.21.15.3:0/481959842 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb1c138de0 0x7fdb1c121150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:39.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.373+0000 7fdb2458b700 1 --2- 172.21.15.3:0/481959842 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb1c09abf0 0x7fdb1c1206d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:39.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.373+0000 7fdb2458b700 1 --2- 172.21.15.3:0/481959842 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb1c09e9d0 0x7fdb1c120c10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:39.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.373+0000 7fdb2458b700 1 -- 172.21.15.3:0/481959842 >> 172.21.15.3:0/481959842 conn(0x7fdb1c0a43f0 msgr2=0x7fdb1c0a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:39.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.373+0000 7fdb2458b700 1 -- 172.21.15.3:0/481959842 shutdown_connections 2023-10-03T10:12:39.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.373+0000 7fdb2458b700 1 -- 172.21.15.3:0/481959842 wait complete. 2023-10-03T10:12:39.433 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph versions' 2023-10-03T10:12:39.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.568+0000 7f18f40b4700 1 Processor -- start 2023-10-03T10:12:39.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.569+0000 7f18f40b4700 1 -- start start 2023-10-03T10:12:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.569+0000 7f18f40b4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f18ec09e430 0x7f18ec135b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.569+0000 7f18f40b4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f18ec1360a0 0x7f18ec138480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.569+0000 7f18f40b4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18ec09b9b0 con 0x7f18ec09d950 2023-10-03T10:12:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.569+0000 7f18f40b4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18ec09b320 con 0x7f18ec1360a0 2023-10-03T10:12:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.569+0000 7f18f40b4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec09e820 con 0x7f18ec09e430 2023-10-03T10:12:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.569+0000 7f18f164f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f18ec09e430 0x7f18ec135b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.569+0000 7f18f2651700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f18ec1360a0 0x7f18ec138480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.569+0000 7f18f164f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f18ec09e430 0x7f18ec135b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40596/0 (socket says 172.21.15.3:40596) 2023-10-03T10:12:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.569+0000 7f18f1e50700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f18ec09d950 0x7f18ec09dd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59216/0 (socket says 172.21.15.3:59216) 2023-10-03T10:12:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.569+0000 7f18f2651700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f18ec1360a0 0x7f18ec138480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52826/0 (socket says 172.21.15.3:52826) 2023-10-03T10:12:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.569+0000 7f18f164f700 1 -- 172.21.15.3:0/2374246602 learned_addr learned my addr 172.21.15.3:0/2374246602 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.570+0000 7f18f164f700 1 -- 172.21.15.3:0/2374246602 >> v2:172.21.15.3:3300/0 conn(0x7f18ec1360a0 msgr2=0x7f18ec138480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.570+0000 7f18f164f700 1 --2- 172.21.15.3:0/2374246602 >> v2:172.21.15.3:3300/0 conn(0x7f18ec1360a0 0x7f18ec138480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.570+0000 7f18f164f700 1 -- 172.21.15.3:0/2374246602 >> v1:172.21.15.3:6789/0 conn(0x7f18ec09d950 legacy=0x7f18ec09dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.570+0000 7f18f164f700 1 -- 172.21.15.3:0/2374246602 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec09a350 con 0x7f18ec09e430 2023-10-03T10:12:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.570+0000 7f18f2651700 1 --2- 172.21.15.3:0/2374246602 >> v2:172.21.15.3:3300/0 conn(0x7f18ec1360a0 0x7f18ec138480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.570+0000 7f18f164f700 1 --2- 172.21.15.3:0/2374246602 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f18ec09e430 0x7f18ec135b60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f18dc006fd0 tx=0x7f18dc00be70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54f481aaad095a2c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.570+0000 7f18f0e4e700 1 -- 172.21.15.3:0/2374246602 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc005e30 con 0x7f18ec09e430 2023-10-03T10:12:39.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.570+0000 7f18f0e4e700 1 -- 172.21.15.3:0/2374246602 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18dc004030 con 0x7f18ec09e430 2023-10-03T10:12:39.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.570+0000 7f18f0e4e700 1 -- 172.21.15.3:0/2374246602 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc0043e0 con 0x7f18ec09e430 2023-10-03T10:12:39.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.571+0000 7f18f40b4700 1 -- 172.21.15.3:0/2374246602 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f18ec09e430 msgr2=0x7f18ec135b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:39.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.571+0000 7f18f40b4700 1 --2- 172.21.15.3:0/2374246602 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f18ec09e430 0x7f18ec135b60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f18dc006fd0 tx=0x7f18dc00be70 comp rx=0 tx=0).stop 2023-10-03T10:12:39.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.571+0000 7f18f40b4700 1 -- 172.21.15.3:0/2374246602 shutdown_connections 2023-10-03T10:12:39.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.571+0000 7f18f40b4700 1 --2- 172.21.15.3:0/2374246602 >> v2:172.21.15.3:3300/0 conn(0x7f18ec1360a0 0x7f18ec138480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:39.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.571+0000 7f18f40b4700 1 --2- 172.21.15.3:0/2374246602 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f18ec09e430 0x7f18ec135b60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:39.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.571+0000 7f18f40b4700 1 -- 172.21.15.3:0/2374246602 >> 172.21.15.3:0/2374246602 conn(0x7f18ec0a2f50 msgr2=0x7f18ec0a3350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:39.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.571+0000 7f18f40b4700 1 -- 172.21.15.3:0/2374246602 shutdown_connections 2023-10-03T10:12:39.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.571+0000 7f18f40b4700 1 -- 172.21.15.3:0/2374246602 wait complete. 2023-10-03T10:12:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.572+0000 7f18f40b4700 1 Processor -- start 2023-10-03T10:12:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.572+0000 7f18f40b4700 1 -- start start 2023-10-03T10:12:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.572+0000 7f18f40b4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18ec09d950 0x7f18ec11e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.572+0000 7f18f40b4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f18ec09e430 0x7f18ec11eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.572+0000 7f18f40b4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18ec1360a0 0x7f18ec11f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.572+0000 7f18f40b4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec1203d0 con 0x7f18ec09d950 2023-10-03T10:12:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.572+0000 7f18f40b4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec1201e0 con 0x7f18ec09e430 2023-10-03T10:12:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.572+0000 7f18f40b4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec120ab0 con 0x7f18ec1360a0 2023-10-03T10:12:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.572+0000 7f18f1e50700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18ec09d950 0x7f18ec11e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.572+0000 7f18f164f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f18ec09e430 0x7f18ec11eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.572+0000 7f18f1e50700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18ec09d950 0x7f18ec11e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52842/0 (socket says 172.21.15.3:52842) 2023-10-03T10:12:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.572+0000 7f18f164f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f18ec09e430 0x7f18ec11eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40612/0 (socket says 172.21.15.3:40612) 2023-10-03T10:12:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.572+0000 7f18f1e50700 1 -- 172.21.15.3:0/2402217306 learned_addr learned my addr 172.21.15.3:0/2402217306 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.572+0000 7f18f2651700 1 --2- 172.21.15.3:0/2402217306 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18ec1360a0 0x7f18ec11f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.573+0000 7f18f164f700 1 -- 172.21.15.3:0/2402217306 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18ec1360a0 msgr2=0x7f18ec11f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.573+0000 7f18f164f700 1 --2- 172.21.15.3:0/2402217306 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18ec1360a0 0x7f18ec11f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.573+0000 7f18f164f700 1 -- 172.21.15.3:0/2402217306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18ec09d950 msgr2=0x7f18ec11e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.573+0000 7f18f164f700 1 --2- 172.21.15.3:0/2402217306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18ec09d950 0x7f18ec11e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.573+0000 7f18f164f700 1 -- 172.21.15.3:0/2402217306 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e4009a70 con 0x7f18ec09e430 2023-10-03T10:12:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.573+0000 7f18f1e50700 1 --2- 172.21.15.3:0/2402217306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18ec09d950 0x7f18ec11e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.573+0000 7f18f2651700 1 --2- 172.21.15.3:0/2402217306 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18ec1360a0 0x7f18ec11f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.573+0000 7f18f164f700 1 --2- 172.21.15.3:0/2402217306 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f18ec09e430 0x7f18ec11eb90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f18dc016000 tx=0x7f18dc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.573+0000 7f18e27fc700 1 -- 172.21.15.3:0/2402217306 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc023070 con 0x7f18ec09e430 2023-10-03T10:12:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.573+0000 7f18e27fc700 1 -- 172.21.15.3:0/2402217306 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18dc0038b0 con 0x7f18ec09e430 2023-10-03T10:12:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.573+0000 7f18f40b4700 1 -- 172.21.15.3:0/2402217306 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18dc00b7a0 con 0x7f18ec09e430 2023-10-03T10:12:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.573+0000 7f18e27fc700 1 -- 172.21.15.3:0/2402217306 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc003c60 con 0x7f18ec09e430 2023-10-03T10:12:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.573+0000 7f18f40b4700 1 -- 172.21.15.3:0/2402217306 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ec121b50 con 0x7f18ec09e430 2023-10-03T10:12:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.574+0000 7f18e27fc700 1 -- 172.21.15.3:0/2402217306 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f18dc01e450 con 0x7f18ec09e430 2023-10-03T10:12:39.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.574+0000 7f18e27fc700 1 -- 172.21.15.3:0/2402217306 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f18dc044f40 con 0x7f18ec09e430 2023-10-03T10:12:39.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.574+0000 7f18f40b4700 1 -- 172.21.15.3:0/2402217306 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18ec121880 con 0x7f18ec09e430 2023-10-03T10:12:39.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.579+0000 7f18e27fc700 1 -- 172.21.15.3:0/2402217306 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f18dc02c080 con 0x7f18ec09e430 2023-10-03T10:12:39.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.870+0000 7f18f40b4700 1 -- 172.21.15.3:0/2402217306 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f18ec04f2a0 con 0x7f18ec09e430 2023-10-03T10:12:39.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.870+0000 7f18e27fc700 1 -- 172.21.15.3:0/2402217306 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+627 (secure 0 0 0) 0x7f18dc00e770 con 0x7f18ec09e430 2023-10-03T10:12:39.871 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-10-03T10:12:39.871 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": { 2023-10-03T10:12:39.871 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph version 18.2.0-507-g7cd99ce4 (7cd99ce428fa36b28345615983ecfd3d24e90eba) reef (stable)": 3 2023-10-03T10:12:39.871 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.871 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": { 2023-10-03T10:12:39.871 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph version 18.2.0-507-g7cd99ce4 (7cd99ce428fa36b28345615983ecfd3d24e90eba) reef (stable)": 1 2023-10-03T10:12:39.872 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.872 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": { 2023-10-03T10:12:39.872 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph version 18.2.0-507-g7cd99ce4 (7cd99ce428fa36b28345615983ecfd3d24e90eba) reef (stable)": 4 2023-10-03T10:12:39.872 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.872 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": { 2023-10-03T10:12:39.872 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph version 18.2.0-507-g7cd99ce4 (7cd99ce428fa36b28345615983ecfd3d24e90eba) reef (stable)": 1 2023-10-03T10:12:39.872 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-10-03T10:12:39.872 INFO:teuthology.orchestra.run.smithi003.stdout: "overall": { 2023-10-03T10:12:39.872 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph version 18.2.0-507-g7cd99ce4 (7cd99ce428fa36b28345615983ecfd3d24e90eba) reef (stable)": 9 2023-10-03T10:12:39.872 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-10-03T10:12:39.872 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-10-03T10:12:39.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.873+0000 7f18f40b4700 1 -- 172.21.15.3:0/2402217306 >> v1:172.21.15.3:6816/127025 conn(0x7f18d002f510 legacy=0x7f18d00319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:39.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.873+0000 7f18f40b4700 1 -- 172.21.15.3:0/2402217306 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f18ec09e430 msgr2=0x7f18ec11eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:39.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.873+0000 7f18f40b4700 1 --2- 172.21.15.3:0/2402217306 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f18ec09e430 0x7f18ec11eb90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f18dc016000 tx=0x7f18dc004750 comp rx=0 tx=0).stop 2023-10-03T10:12:39.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.873+0000 7f18f40b4700 1 -- 172.21.15.3:0/2402217306 shutdown_connections 2023-10-03T10:12:39.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.873+0000 7f18f40b4700 1 --2- 172.21.15.3:0/2402217306 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18ec1360a0 0x7f18ec11f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:39.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.873+0000 7f18f40b4700 1 --2- 172.21.15.3:0/2402217306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18ec09d950 0x7f18ec11e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:39.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.873+0000 7f18f40b4700 1 --2- 172.21.15.3:0/2402217306 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f18ec09e430 0x7f18ec11eb90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:39.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.873+0000 7f18f40b4700 1 -- 172.21.15.3:0/2402217306 >> 172.21.15.3:0/2402217306 conn(0x7f18ec0a2f50 msgr2=0x7f18ec06e400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:39.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.873+0000 7f18f40b4700 1 -- 172.21.15.3:0/2402217306 shutdown_connections 2023-10-03T10:12:39.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:39.873+0000 7f18f40b4700 1 -- 172.21.15.3:0/2402217306 wait complete. 2023-10-03T10:12:39.886 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done' 2023-10-03T10:12:40.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.019+0000 7f2ca91ba700 1 Processor -- start 2023-10-03T10:12:40.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.019+0000 7f2ca91ba700 1 -- start start 2023-10-03T10:12:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.020+0000 7f2ca91ba700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ca409e9d0 0x7f2ca409eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.020+0000 7f2ca91ba700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ca409f2e0 0x7f2ca409f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.020+0000 7f2ca91ba700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ca409ab50 con 0x7f2ca41317b0 2023-10-03T10:12:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.020+0000 7f2ca91ba700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca40a0140 con 0x7f2ca409f2e0 2023-10-03T10:12:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.020+0000 7f2ca91ba700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca40a0330 con 0x7f2ca409e9d0 2023-10-03T10:12:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.020+0000 7f2ca2d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ca409e9d0 0x7f2ca409eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.020+0000 7f2ca259c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ca409f2e0 0x7f2ca409f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.020+0000 7f2ca359e700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2ca41317b0 0x7f2ca4131c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59230/0 (socket says 172.21.15.3:59230) 2023-10-03T10:12:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.020+0000 7f2ca359e700 1 -- 172.21.15.3:0/1663780863 learned_addr learned my addr 172.21.15.3:0/1663780863 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.020+0000 7f2ca2d9d700 1 -- 172.21.15.3:0/1663780863 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ca409f2e0 msgr2=0x7f2ca409f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.020+0000 7f2ca2d9d700 1 --2- 172.21.15.3:0/1663780863 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ca409f2e0 0x7f2ca409f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.020+0000 7f2ca2d9d700 1 -- 172.21.15.3:0/1663780863 >> v1:172.21.15.3:6789/0 conn(0x7f2ca41317b0 legacy=0x7f2ca4131c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.020+0000 7f2ca2d9d700 1 -- 172.21.15.3:0/1663780863 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca406ef60 con 0x7f2ca409e9d0 2023-10-03T10:12:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.020+0000 7f2ca259c700 1 --2- 172.21.15.3:0/1663780863 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ca409f2e0 0x7f2ca409f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.020+0000 7f2ca1d9b700 1 -- 172.21.15.3:0/1663780863 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3595383434 0 0) 0x7f2ca409ab50 con 0x7f2ca41317b0 2023-10-03T10:12:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.021+0000 7f2ca2d9d700 1 --2- 172.21.15.3:0/1663780863 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ca409e9d0 0x7f2ca409eda0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2c94009870 tx=0x7f2c94009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f042a6348e54b142 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.021+0000 7f2ca1d9b700 1 -- 172.21.15.3:0/1663780863 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c94011830 con 0x7f2ca409e9d0 2023-10-03T10:12:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.021+0000 7f2ca1d9b700 1 -- 172.21.15.3:0/1663780863 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c94011990 con 0x7f2ca409e9d0 2023-10-03T10:12:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.021+0000 7f2ca1d9b700 1 -- 172.21.15.3:0/1663780863 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c94011d20 con 0x7f2ca409e9d0 2023-10-03T10:12:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.021+0000 7f2ca91ba700 1 -- 172.21.15.3:0/1663780863 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ca409e9d0 msgr2=0x7f2ca409eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.021+0000 7f2ca91ba700 1 --2- 172.21.15.3:0/1663780863 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ca409e9d0 0x7f2ca409eda0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2c94009870 tx=0x7f2c94009b80 comp rx=0 tx=0).stop 2023-10-03T10:12:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.021+0000 7f2ca91ba700 1 -- 172.21.15.3:0/1663780863 shutdown_connections 2023-10-03T10:12:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.021+0000 7f2ca91ba700 1 --2- 172.21.15.3:0/1663780863 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ca409e9d0 0x7f2ca409eda0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:40.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.021+0000 7f2ca91ba700 1 --2- 172.21.15.3:0/1663780863 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ca409f2e0 0x7f2ca409f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:40.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.021+0000 7f2ca91ba700 1 -- 172.21.15.3:0/1663780863 >> 172.21.15.3:0/1663780863 conn(0x7f2ca40a43f0 msgr2=0x7f2ca40a47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:40.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.021+0000 7f2ca91ba700 1 -- 172.21.15.3:0/1663780863 shutdown_connections 2023-10-03T10:12:40.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.021+0000 7f2ca91ba700 1 -- 172.21.15.3:0/1663780863 wait complete. 2023-10-03T10:12:40.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.023+0000 7f2ca91ba700 1 Processor -- start 2023-10-03T10:12:40.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.023+0000 7f2ca91ba700 1 -- start start 2023-10-03T10:12:40.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.023+0000 7f2ca91ba700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ca409e9d0 0x7f2ca406d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:40.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.023+0000 7f2ca91ba700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ca409f2e0 0x7f2ca406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:40.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.023+0000 7f2ca91ba700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ca41317b0 0x7f2ca41c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:40.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.023+0000 7f2ca91ba700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca4122320 con 0x7f2ca409e9d0 2023-10-03T10:12:40.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.023+0000 7f2ca91ba700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca41315f0 con 0x7f2ca409f2e0 2023-10-03T10:12:40.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.023+0000 7f2ca91ba700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca40a4a50 con 0x7f2ca41317b0 2023-10-03T10:12:40.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.023+0000 7f2ca2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ca409e9d0 0x7f2ca406d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:40.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.023+0000 7f2ca259c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ca409f2e0 0x7f2ca406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:40.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.023+0000 7f2ca359e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ca41317b0 0x7f2ca41c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:40.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.023+0000 7f2ca2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ca409e9d0 0x7f2ca406d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52858/0 (socket says 172.21.15.3:52858) 2023-10-03T10:12:40.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.023+0000 7f2ca2d9d700 1 -- 172.21.15.3:0/3731009951 learned_addr learned my addr 172.21.15.3:0/3731009951 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:40.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.024+0000 7f2ca2d9d700 1 -- 172.21.15.3:0/3731009951 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ca41317b0 msgr2=0x7f2ca41c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:40.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.024+0000 7f2ca2d9d700 1 --2- 172.21.15.3:0/3731009951 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ca41317b0 0x7f2ca41c80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:40.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.024+0000 7f2ca2d9d700 1 -- 172.21.15.3:0/3731009951 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ca409f2e0 msgr2=0x7f2ca406d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:40.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.024+0000 7f2ca2d9d700 1 --2- 172.21.15.3:0/3731009951 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ca409f2e0 0x7f2ca406d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:40.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.024+0000 7f2ca2d9d700 1 -- 172.21.15.3:0/3731009951 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8c0094a0 con 0x7f2ca409e9d0 2023-10-03T10:12:40.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.024+0000 7f2ca259c700 1 --2- 172.21.15.3:0/3731009951 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ca409f2e0 0x7f2ca406d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:40.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.024+0000 7f2ca2d9d700 1 --2- 172.21.15.3:0/3731009951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ca409e9d0 0x7f2ca406d240 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2c94009840 tx=0x7f2c94004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:40.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.024+0000 7f2c9b7fe700 1 -- 172.21.15.3:0/3731009951 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c94004510 con 0x7f2ca409e9d0 2023-10-03T10:12:40.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.024+0000 7f2c9b7fe700 1 -- 172.21.15.3:0/3731009951 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c9401f070 con 0x7f2ca409e9d0 2023-10-03T10:12:40.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.024+0000 7f2ca91ba700 1 -- 172.21.15.3:0/3731009951 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c940094a0 con 0x7f2ca409e9d0 2023-10-03T10:12:40.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.024+0000 7f2c9b7fe700 1 -- 172.21.15.3:0/3731009951 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c940038e0 con 0x7f2ca409e9d0 2023-10-03T10:12:40.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.024+0000 7f2ca91ba700 1 -- 172.21.15.3:0/3731009951 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ca41225c0 con 0x7f2ca409e9d0 2023-10-03T10:12:40.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.025+0000 7f2c9b7fe700 1 -- 172.21.15.3:0/3731009951 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f2c94003a40 con 0x7f2ca409e9d0 2023-10-03T10:12:40.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.025+0000 7f2c9b7fe700 1 -- 172.21.15.3:0/3731009951 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f2c94021070 con 0x7f2ca409e9d0 2023-10-03T10:12:40.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.025+0000 7f2ca91ba700 1 -- 172.21.15.3:0/3731009951 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ca41225c0 con 0x7f2ca409e9d0 2023-10-03T10:12:40.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.030+0000 7f2c9b7fe700 1 -- 172.21.15.3:0/3731009951 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2ca41225c0 con 0x7f2ca409e9d0 2023-10-03T10:12:40.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.233+0000 7f2ca91ba700 1 -- 172.21.15.3:0/3731009951 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool ls"} v 0) v1 -- 0x7f2ca41225c0 con 0x7f2ca409e9d0 2023-10-03T10:12:40.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.234+0000 7f2c9b7fe700 1 -- 172.21.15.3:0/3731009951 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool ls"}]=0 v34) v1 ==== 59+0+37 (secure 0 0 0) 0x7f2ca41225c0 con 0x7f2ca409e9d0 2023-10-03T10:12:40.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.236+0000 7f2ca91ba700 1 -- 172.21.15.3:0/3731009951 >> v1:172.21.15.3:6816/127025 conn(0x7f2c8402f510 legacy=0x7f2c840319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:40.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.236+0000 7f2ca91ba700 1 -- 172.21.15.3:0/3731009951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ca409e9d0 msgr2=0x7f2ca406d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:40.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.236+0000 7f2ca91ba700 1 --2- 172.21.15.3:0/3731009951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ca409e9d0 0x7f2ca406d240 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2c94009840 tx=0x7f2c94004750 comp rx=0 tx=0).stop 2023-10-03T10:12:40.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.236+0000 7f2ca91ba700 1 -- 172.21.15.3:0/3731009951 shutdown_connections 2023-10-03T10:12:40.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.236+0000 7f2ca91ba700 1 --2- 172.21.15.3:0/3731009951 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ca41317b0 0x7f2ca41c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:40.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.236+0000 7f2ca91ba700 1 --2- 172.21.15.3:0/3731009951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ca409e9d0 0x7f2ca406d240 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:40.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.236+0000 7f2ca91ba700 1 --2- 172.21.15.3:0/3731009951 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ca409f2e0 0x7f2ca406d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:40.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.236+0000 7f2ca91ba700 1 -- 172.21.15.3:0/3731009951 >> 172.21.15.3:0/3731009951 conn(0x7f2ca40a43f0 msgr2=0x7f2ca409d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:40.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.236+0000 7f2ca91ba700 1 -- 172.21.15.3:0/3731009951 shutdown_connections 2023-10-03T10:12:40.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.236+0000 7f2ca91ba700 1 -- 172.21.15.3:0/3731009951 wait complete. 2023-10-03T10:12:40.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.361+0000 7f57ad217700 1 Processor -- start 2023-10-03T10:12:40.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.361+0000 7f57ad217700 1 -- start start 2023-10-03T10:12:40.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.361+0000 7f57ad217700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57a809e9d0 0x7f57a809eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:40.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.361+0000 7f57ad217700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57a809f2e0 0x7f57a809f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:40.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.361+0000 7f57ad217700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f57a81317b0 0x7f57a8131c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:40.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.361+0000 7f57ad217700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a809abd0 con 0x7f57a81317b0 2023-10-03T10:12:40.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.361+0000 7f57ad217700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a806fd90 con 0x7f57a809f2e0 2023-10-03T10:12:40.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.361+0000 7f57ad217700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f57a8071e80 con 0x7f57a809e9d0 2023-10-03T10:12:40.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.361+0000 7f57a759e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f57a81317b0 0x7f57a8131c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:40.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.361+0000 7f57a659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57a809f2e0 0x7f57a809f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:40.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.361+0000 7f57a759e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f57a81317b0 0x7f57a8131c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52862/0 (socket says 172.21.15.3:52862) 2023-10-03T10:12:40.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.361+0000 7f57a659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57a809f2e0 0x7f57a809f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40650/0 (socket says 172.21.15.3:40650) 2023-10-03T10:12:40.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.361+0000 7f57a6d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57a809e9d0 0x7f57a809eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:40.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.361+0000 7f57a759e700 1 -- 172.21.15.3:0/1209371259 learned_addr learned my addr 172.21.15.3:0/1209371259 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:40.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.362+0000 7f57a659c700 1 -- 172.21.15.3:0/1209371259 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57a809e9d0 msgr2=0x7f57a809eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:40.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.362+0000 7f57a659c700 1 --2- 172.21.15.3:0/1209371259 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57a809e9d0 0x7f57a809eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:40.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.362+0000 7f57a659c700 1 -- 172.21.15.3:0/1209371259 >> v2:172.21.15.3:3300/0 conn(0x7f57a81317b0 msgr2=0x7f57a8131c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:40.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.362+0000 7f57a659c700 1 --2- 172.21.15.3:0/1209371259 >> v2:172.21.15.3:3300/0 conn(0x7f57a81317b0 0x7f57a8131c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:40.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.362+0000 7f57a659c700 1 -- 172.21.15.3:0/1209371259 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a80a0240 con 0x7f57a809f2e0 2023-10-03T10:12:40.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.362+0000 7f57a759e700 1 --2- 172.21.15.3:0/1209371259 >> v2:172.21.15.3:3300/0 conn(0x7f57a81317b0 0x7f57a8131c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:40.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.362+0000 7f57a659c700 1 --2- 172.21.15.3:0/1209371259 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57a809f2e0 0x7f57a809f6b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f579c009840 tx=0x7f579c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a2a35cc89bc8549 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:40.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.362+0000 7f57a5d9b700 1 -- 172.21.15.3:0/1209371259 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c011830 con 0x7f57a809f2e0 2023-10-03T10:12:40.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.362+0000 7f57a5d9b700 1 -- 172.21.15.3:0/1209371259 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579c011990 con 0x7f57a809f2e0 2023-10-03T10:12:40.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.362+0000 7f57a5d9b700 1 -- 172.21.15.3:0/1209371259 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c011d20 con 0x7f57a809f2e0 2023-10-03T10:12:40.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.363+0000 7f57ad217700 1 -- 172.21.15.3:0/1209371259 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57a809f2e0 msgr2=0x7f57a809f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:40.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.363+0000 7f57ad217700 1 --2- 172.21.15.3:0/1209371259 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57a809f2e0 0x7f57a809f6b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f579c009840 tx=0x7f579c009b50 comp rx=0 tx=0).stop 2023-10-03T10:12:40.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.363+0000 7f57ad217700 1 -- 172.21.15.3:0/1209371259 shutdown_connections 2023-10-03T10:12:40.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.363+0000 7f57ad217700 1 --2- 172.21.15.3:0/1209371259 >> v2:172.21.15.3:3300/0 conn(0x7f57a81317b0 0x7f57a8131c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:40.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.363+0000 7f57ad217700 1 --2- 172.21.15.3:0/1209371259 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57a809e9d0 0x7f57a809eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:40.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.363+0000 7f57ad217700 1 --2- 172.21.15.3:0/1209371259 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57a809f2e0 0x7f57a809f6b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:40.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.363+0000 7f57ad217700 1 -- 172.21.15.3:0/1209371259 >> 172.21.15.3:0/1209371259 conn(0x7f57a80a43c0 msgr2=0x7f57a80a47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:40.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.363+0000 7f57ad217700 1 -- 172.21.15.3:0/1209371259 shutdown_connections 2023-10-03T10:12:40.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.363+0000 7f57ad217700 1 -- 172.21.15.3:0/1209371259 wait complete. 2023-10-03T10:12:40.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.364+0000 7f57ad217700 1 Processor -- start 2023-10-03T10:12:40.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.364+0000 7f57ad217700 1 -- start start 2023-10-03T10:12:40.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.365+0000 7f57ad217700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57a809e9d0 0x7f57a81c6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:40.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.365+0000 7f57ad217700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57a809f2e0 0x7f57a81c6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:40.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.365+0000 7f57ad217700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a81317b0 0x7f57a81c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:40.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.365+0000 7f57ad217700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a811f370 con 0x7f57a81317b0 2023-10-03T10:12:40.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.365+0000 7f57ad217700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a811f550 con 0x7f57a809e9d0 2023-10-03T10:12:40.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.365+0000 7f57ad217700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f57a81c6e90 con 0x7f57a809f2e0 2023-10-03T10:12:40.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.365+0000 7f57a659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57a809f2e0 0x7f57a81c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:40.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.365+0000 7f57a6d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57a809e9d0 0x7f57a81c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:40.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.365+0000 7f57a759e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a81317b0 0x7f57a81c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:40.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.365+0000 7f57a6d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57a809e9d0 0x7f57a81c6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40666/0 (socket says 172.21.15.3:40666) 2023-10-03T10:12:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.365+0000 7f57a6d9d700 1 -- 172.21.15.3:0/3314194364 learned_addr learned my addr 172.21.15.3:0/3314194364 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.365+0000 7f57a759e700 1 -- 172.21.15.3:0/3314194364 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57a809f2e0 msgr2=0x7f57a81c6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.365+0000 7f57a759e700 1 --2- 172.21.15.3:0/3314194364 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57a809f2e0 0x7f57a81c6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.365+0000 7f57a759e700 1 -- 172.21.15.3:0/3314194364 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57a809e9d0 msgr2=0x7f57a81c6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.365+0000 7f57a759e700 1 --2- 172.21.15.3:0/3314194364 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57a809e9d0 0x7f57a81c6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.365+0000 7f57a759e700 1 -- 172.21.15.3:0/3314194364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a0009b30 con 0x7f57a81317b0 2023-10-03T10:12:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.365+0000 7f57a659c700 1 --2- 172.21.15.3:0/3314194364 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57a809f2e0 0x7f57a81c6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.365+0000 7f57a6d9d700 1 --2- 172.21.15.3:0/3314194364 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57a809e9d0 0x7f57a81c6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.366+0000 7f57a759e700 1 --2- 172.21.15.3:0/3314194364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a81317b0 0x7f57a81c29a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f57a0009e50 tx=0x7f57a000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.366+0000 7f578f7fe700 1 -- 172.21.15.3:0/3314194364 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a0018070 con 0x7f57a81317b0 2023-10-03T10:12:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.366+0000 7f578f7fe700 1 -- 172.21.15.3:0/3314194364 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a000f040 con 0x7f57a81317b0 2023-10-03T10:12:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.366+0000 7f57ad217700 1 -- 172.21.15.3:0/3314194364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f579c0094a0 con 0x7f57a81317b0 2023-10-03T10:12:40.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.366+0000 7f578f7fe700 1 -- 172.21.15.3:0/3314194364 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a0003c40 con 0x7f57a81317b0 2023-10-03T10:12:40.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.366+0000 7f57ad217700 1 -- 172.21.15.3:0/3314194364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a80a20d0 con 0x7f57a81317b0 2023-10-03T10:12:40.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.367+0000 7f57ad217700 1 -- 172.21.15.3:0/3314194364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a804f2a0 con 0x7f57a81317b0 2023-10-03T10:12:40.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.367+0000 7f578f7fe700 1 -- 172.21.15.3:0/3314194364 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f57a000c880 con 0x7f57a81317b0 2023-10-03T10:12:40.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.367+0000 7f578f7fe700 1 -- 172.21.15.3:0/3314194364 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f57a0040a60 con 0x7f57a81317b0 2023-10-03T10:12:40.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.372+0000 7f578f7fe700 1 -- 172.21.15.3:0/3314194364 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f57a0013bd0 con 0x7f57a81317b0 2023-10-03T10:12:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:40.574+0000 7f57ad217700 1 -- 172.21.15.3:0/3314194364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool set", "pool": ".mgr", "var": "pg_autoscale_mode", "val": "off"} v 0) v1 -- 0x7f57a811f1c0 con 0x7f57a81317b0 2023-10-03T10:12:40.756 INFO:tasks.ceph.mgr.x.smithi003.stderr:terminate called after throwing an instance of 'ceph::buffer::v15_2_0::end_of_buffer' 2023-10-03T10:12:40.757 INFO:tasks.ceph.mgr.x.smithi003.stderr:terminate called recursively 2023-10-03T10:12:40.757 INFO:tasks.ceph.mgr.x.smithi003.stderr: what(): End of buffer [buffer:2] 2023-10-03T10:12:40.757 INFO:tasks.ceph.mgr.x.smithi003.stderr:*** Caught signal (Aborted) ** 2023-10-03T10:12:40.757 INFO:tasks.ceph.mgr.x.smithi003.stderr: in thread 7f2c43273700 thread_name:ms_dispatch 2023-10-03T10:12:41.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.024+0000 7f57a6d9d700 1 -- 172.21.15.3:0/3314194364 >> v1:172.21.15.3:6816/127025 conn(0x7f579402f510 legacy=0x7f57940319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2023-10-03T10:12:41.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.024+0000 7f57a6d9d700 1 -- 172.21.15.3:0/3314194364 >> v1:172.21.15.3:6816/127025 conn(0x7f579402f510 legacy=0x7f57940319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-03T10:12:41.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.024+0000 7f57a6d9d700 1 --1- 172.21.15.3:0/3314194364 >> v1:172.21.15.3:6816/127025 conn(0x7f579402f510 0x7f57940319c0 :-1 s=OPENED pgs=55 cs=1 l=1).handle_message read tag failed 2023-10-03T10:12:41.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.024+0000 7f57a6d9d700 1 --1- 172.21.15.3:0/3314194364 >> v1:172.21.15.3:6816/127025 conn(0x7f579402f510 0x7f57940319c0 :-1 s=OPENED pgs=55 cs=1 l=1).fault on lossy channel, failing 2023-10-03T10:12:41.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.024+0000 7f578f7fe700 1 -- 172.21.15.3:0/3314194364 >> v1:172.21.15.3:6816/127025 conn(0x7f579402f510 legacy=0x7f57940319c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2023-10-03T10:12:41.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.307+0000 7f578f7fe700 1 -- 172.21.15.3:0/3314194364 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_autoscale_mode", "val": "off"}]=0 set pool 1 pg_autoscale_mode to off v35) v1 ==== 153+0+0 (secure 0 0 0) 0x7f57a0013bd0 con 0x7f57a81317b0 2023-10-03T10:12:41.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.311+0000 7f57ad217700 1 -- 172.21.15.3:0/3314194364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a81317b0 msgr2=0x7f57a81c29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:41.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.311+0000 7f57ad217700 1 --2- 172.21.15.3:0/3314194364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a81317b0 0x7f57a81c29a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f57a0009e50 tx=0x7f57a000c3a0 comp rx=0 tx=0).stop 2023-10-03T10:12:41.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.311+0000 7f57ad217700 1 -- 172.21.15.3:0/3314194364 shutdown_connections 2023-10-03T10:12:41.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.311+0000 7f57ad217700 1 --2- 172.21.15.3:0/3314194364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a81317b0 0x7f57a81c29a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:41.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.311+0000 7f57ad217700 1 --2- 172.21.15.3:0/3314194364 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57a809e9d0 0x7f57a81c6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:41.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.311+0000 7f57ad217700 1 --2- 172.21.15.3:0/3314194364 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57a809f2e0 0x7f57a81c6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:41.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.311+0000 7f57ad217700 1 -- 172.21.15.3:0/3314194364 >> 172.21.15.3:0/3314194364 conn(0x7f57a80a43c0 msgr2=0x7f57a8130d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:41.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.311+0000 7f57ad217700 1 -- 172.21.15.3:0/3314194364 shutdown_connections 2023-10-03T10:12:41.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.311+0000 7f57ad217700 1 -- 172.21.15.3:0/3314194364 wait complete. 2023-10-03T10:12:41.312 INFO:teuthology.orchestra.run.smithi003.stderr:set pool 1 pg_autoscale_mode to off 2023-10-03T10:12:41.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.434+0000 7f8547eec700 1 Processor -- start 2023-10-03T10:12:41.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.434+0000 7f8547eec700 1 -- start start 2023-10-03T10:12:41.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.435+0000 7f8547eec700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f854009d950 0x7f854009dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:41.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.435+0000 7f8547eec700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f854009e6a0 0x7f8540138620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:41.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.435+0000 7f8547eec700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f854009ab70 con 0x7f854009e260 2023-10-03T10:12:41.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.435+0000 7f8547eec700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f854009b550 con 0x7f854009d950 2023-10-03T10:12:41.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.435+0000 7f8547eec700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f854009ef60 con 0x7f854009e6a0 2023-10-03T10:12:41.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.435+0000 7f8546489700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f854009e6a0 0x7f8540138620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:41.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.435+0000 7f8545c88700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f854009d950 0x7f854009dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:41.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.435+0000 7f8546489700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f854009e6a0 0x7f8540138620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56816/0 (socket says 172.21.15.3:56816) 2023-10-03T10:12:41.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.435+0000 7f8545487700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f854009e260 0x7f8540137f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59238/0 (socket says 172.21.15.3:59238) 2023-10-03T10:12:41.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.435+0000 7f8545c88700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f854009d950 0x7f854009dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52880/0 (socket says 172.21.15.3:52880) 2023-10-03T10:12:41.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.435+0000 7f8546489700 1 -- 172.21.15.3:0/3475497226 learned_addr learned my addr 172.21.15.3:0/3475497226 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:41.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.435+0000 7f8545c88700 1 -- 172.21.15.3:0/3475497226 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f854009e6a0 msgr2=0x7f8540138620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:41.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.435+0000 7f8545c88700 1 --2- 172.21.15.3:0/3475497226 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f854009e6a0 0x7f8540138620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:41.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.436+0000 7f8545c88700 1 -- 172.21.15.3:0/3475497226 >> v1:172.21.15.3:6789/0 conn(0x7f854009e260 legacy=0x7f8540137f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:41.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.436+0000 7f8545c88700 1 -- 172.21.15.3:0/3475497226 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f854006b640 con 0x7f854009d950 2023-10-03T10:12:41.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.436+0000 7f8544c86700 1 -- 172.21.15.3:0/3475497226 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2129519334 0 0) 0x7f854009ab70 con 0x7f854009e260 2023-10-03T10:12:41.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.436+0000 7f8545c88700 1 --2- 172.21.15.3:0/3475497226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f854009d950 0x7f854009dd20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f853c009840 tx=0x7f853c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f87db8022addf8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:41.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.436+0000 7f8544c86700 1 -- 172.21.15.3:0/3475497226 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853c011830 con 0x7f854009d950 2023-10-03T10:12:41.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.436+0000 7f8544c86700 1 -- 172.21.15.3:0/3475497226 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853c011990 con 0x7f854009d950 2023-10-03T10:12:41.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.436+0000 7f8544c86700 1 -- 172.21.15.3:0/3475497226 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853c011d20 con 0x7f854009d950 2023-10-03T10:12:41.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.436+0000 7f8547eec700 1 -- 172.21.15.3:0/3475497226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f854009d950 msgr2=0x7f854009dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:41.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.436+0000 7f8547eec700 1 --2- 172.21.15.3:0/3475497226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f854009d950 0x7f854009dd20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f853c009840 tx=0x7f853c009b50 comp rx=0 tx=0).stop 2023-10-03T10:12:41.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.437+0000 7f8547eec700 1 -- 172.21.15.3:0/3475497226 shutdown_connections 2023-10-03T10:12:41.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.437+0000 7f8547eec700 1 --2- 172.21.15.3:0/3475497226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f854009d950 0x7f854009dd20 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:41.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.437+0000 7f8547eec700 1 --2- 172.21.15.3:0/3475497226 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f854009e6a0 0x7f8540138620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:41.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.437+0000 7f8547eec700 1 -- 172.21.15.3:0/3475497226 >> 172.21.15.3:0/3475497226 conn(0x7f85400a2f50 msgr2=0x7f85400a3350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:41.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.437+0000 7f8547eec700 1 -- 172.21.15.3:0/3475497226 shutdown_connections 2023-10-03T10:12:41.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.437+0000 7f8547eec700 1 -- 172.21.15.3:0/3475497226 wait complete. 2023-10-03T10:12:41.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.438+0000 7f8547eec700 1 Processor -- start 2023-10-03T10:12:41.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.438+0000 7f8547eec700 1 -- start start 2023-10-03T10:12:41.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.439+0000 7f8547eec700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f854009d950 0x7f8540126a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:41.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.439+0000 7f8547eec700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f854009e260 0x7f8540126fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:41.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.439+0000 7f8547eec700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f854009e6a0 0x7f85401274e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:41.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.439+0000 7f8547eec700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854012a470 con 0x7f854009d950 2023-10-03T10:12:41.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.439+0000 7f8547eec700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85400a1e60 con 0x7f854009e260 2023-10-03T10:12:41.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.439+0000 7f8547eec700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f854009f760 con 0x7f854009e6a0 2023-10-03T10:12:41.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.439+0000 7f8545c88700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f854009d950 0x7f8540126a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:41.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.439+0000 7f8545c88700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f854009d950 0x7f8540126a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52882/0 (socket says 172.21.15.3:52882) 2023-10-03T10:12:41.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.439+0000 7f8545487700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f854009e260 0x7f8540126fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:41.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.439+0000 7f8545c88700 1 -- 172.21.15.3:0/1468064902 learned_addr learned my addr 172.21.15.3:0/1468064902 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:41.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.439+0000 7f8546489700 1 --2- 172.21.15.3:0/1468064902 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f854009e6a0 0x7f85401274e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:41.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.439+0000 7f8545c88700 1 -- 172.21.15.3:0/1468064902 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f854009e6a0 msgr2=0x7f85401274e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:41.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.439+0000 7f8545c88700 1 --2- 172.21.15.3:0/1468064902 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f854009e6a0 0x7f85401274e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:41.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.439+0000 7f8545c88700 1 -- 172.21.15.3:0/1468064902 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f854009e260 msgr2=0x7f8540126fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:41.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.439+0000 7f8545c88700 1 --2- 172.21.15.3:0/1468064902 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f854009e260 0x7f8540126fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:41.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.439+0000 7f8545c88700 1 -- 172.21.15.3:0/1468064902 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853c0094a0 con 0x7f854009d950 2023-10-03T10:12:41.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.439+0000 7f8545c88700 1 --2- 172.21.15.3:0/1468064902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f854009d950 0x7f8540126a60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f853c009fd0 tx=0x7f853c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:41.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.439+0000 7f85367fc700 1 -- 172.21.15.3:0/1468064902 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853c0036a0 con 0x7f854009d950 2023-10-03T10:12:41.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.439+0000 7f85367fc700 1 -- 172.21.15.3:0/1468064902 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853c01f070 con 0x7f854009d950 2023-10-03T10:12:41.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.439+0000 7f8547eec700 1 -- 172.21.15.3:0/1468064902 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8540137c20 con 0x7f854009d950 2023-10-03T10:12:41.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.440+0000 7f85367fc700 1 -- 172.21.15.3:0/1468064902 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853c003a60 con 0x7f854009d950 2023-10-03T10:12:41.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.440+0000 7f8547eec700 1 -- 172.21.15.3:0/1468064902 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f854012ab70 con 0x7f854009d950 2023-10-03T10:12:41.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.440+0000 7f85367fc700 1 -- 172.21.15.3:0/1468064902 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f853c003bc0 con 0x7f854009d950 2023-10-03T10:12:41.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.440+0000 7f85367fc700 1 -- 172.21.15.3:0/1468064902 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f853c021070 con 0x7f854009d950 2023-10-03T10:12:41.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.440+0000 7f8545487700 1 -- 172.21.15.3:0/1468064902 >> v1:172.21.15.3:6816/127025 conn(0x7f852402f510 legacy=0x7f85240319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:41.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.441+0000 7f8547eec700 1 -- 172.21.15.3:0/1468064902 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f854009d340 con 0x7f854009d950 2023-10-03T10:12:41.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.446+0000 7f85367fc700 1 -- 172.21.15.3:0/1468064902 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f853c0253e0 con 0x7f854009d950 2023-10-03T10:12:41.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.641+0000 7f8545487700 1 -- 172.21.15.3:0/1468064902 >> v1:172.21.15.3:6816/127025 conn(0x7f852402f510 legacy=0x7f85240319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:41.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:41.652+0000 7f8547eec700 1 -- 172.21.15.3:0/1468064902 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "rbd", "var": "pg_autoscale_mode", "val": "off"} v 0) v1 -- 0x7f854004f2a0 con 0x7f854009d950 2023-10-03T10:12:42.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.042+0000 7f8545487700 1 -- 172.21.15.3:0/1468064902 >> v1:172.21.15.3:6816/127025 conn(0x7f852402f510 legacy=0x7f85240319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:42.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.308+0000 7f85367fc700 1 -- 172.21.15.3:0/1468064902 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "rbd", "var": "pg_autoscale_mode", "val": "off"}]=0 set pool 2 pg_autoscale_mode to off v36) v1 ==== 152+0+0 (secure 0 0 0) 0x7f853c010b70 con 0x7f854009d950 2023-10-03T10:12:42.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.313+0000 7f8547eec700 1 -- 172.21.15.3:0/1468064902 >> v1:172.21.15.3:6816/127025 conn(0x7f852402f510 legacy=0x7f85240319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:42.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.313+0000 7f8547eec700 1 -- 172.21.15.3:0/1468064902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f854009d950 msgr2=0x7f8540126a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:42.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.313+0000 7f8547eec700 1 --2- 172.21.15.3:0/1468064902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f854009d950 0x7f8540126a60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f853c009fd0 tx=0x7f853c004750 comp rx=0 tx=0).stop 2023-10-03T10:12:42.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.313+0000 7f8547eec700 1 -- 172.21.15.3:0/1468064902 shutdown_connections 2023-10-03T10:12:42.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.313+0000 7f8547eec700 1 --2- 172.21.15.3:0/1468064902 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f854009e6a0 0x7f85401274e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:42.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.313+0000 7f8547eec700 1 --2- 172.21.15.3:0/1468064902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f854009d950 0x7f8540126a60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:42.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.313+0000 7f8547eec700 1 --2- 172.21.15.3:0/1468064902 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f854009e260 0x7f8540126fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:42.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.313+0000 7f8547eec700 1 -- 172.21.15.3:0/1468064902 >> 172.21.15.3:0/1468064902 conn(0x7f85400a2f50 msgr2=0x7f85400a0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:42.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.313+0000 7f8547eec700 1 -- 172.21.15.3:0/1468064902 shutdown_connections 2023-10-03T10:12:42.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.313+0000 7f8547eec700 1 -- 172.21.15.3:0/1468064902 wait complete. 2023-10-03T10:12:42.314 INFO:teuthology.orchestra.run.smithi003.stderr:set pool 2 pg_autoscale_mode to off 2023-10-03T10:12:42.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.438+0000 7fea2371d700 1 Processor -- start 2023-10-03T10:12:42.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.438+0000 7fea2371d700 1 -- start start 2023-10-03T10:12:42.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.438+0000 7fea2371d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea1c09c910 0x7fea1c09cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:42.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.438+0000 7fea2371d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fea1c09d850 0x7fea1c09dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:42.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.438+0000 7fea2371d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea1c09f200 con 0x7fea1c09e1c0 2023-10-03T10:12:42.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.438+0000 7fea2371d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea1c09ed90 con 0x7fea1c09d850 2023-10-03T10:12:42.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.438+0000 7fea2371d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c0a0880 con 0x7fea1c09c910 2023-10-03T10:12:42.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.438+0000 7fea20cb8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fea1c09d850 0x7fea1c09dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:42.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.438+0000 7fea214b9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea1c09c910 0x7fea1c09cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:42.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.438+0000 7fea20cb8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fea1c09d850 0x7fea1c09dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52886/0 (socket says 172.21.15.3:52886) 2023-10-03T10:12:42.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.438+0000 7fea21cba700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fea1c09e1c0 0x7fea1c13ad00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59242/0 (socket says 172.21.15.3:59242) 2023-10-03T10:12:42.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.438+0000 7fea20cb8700 1 -- 172.21.15.3:0/4222747628 learned_addr learned my addr 172.21.15.3:0/4222747628 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:42.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.438+0000 7fea214b9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea1c09c910 0x7fea1c09cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40678/0 (socket says 172.21.15.3:40678) 2023-10-03T10:12:42.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.439+0000 7fea214b9700 1 -- 172.21.15.3:0/4222747628 >> v2:172.21.15.3:3300/0 conn(0x7fea1c09d850 msgr2=0x7fea1c09dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:42.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.439+0000 7fea214b9700 1 --2- 172.21.15.3:0/4222747628 >> v2:172.21.15.3:3300/0 conn(0x7fea1c09d850 0x7fea1c09dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:42.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.439+0000 7fea214b9700 1 -- 172.21.15.3:0/4222747628 >> v1:172.21.15.3:6789/0 conn(0x7fea1c09e1c0 legacy=0x7fea1c13ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:42.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.439+0000 7fea214b9700 1 -- 172.21.15.3:0/4222747628 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1c0a05e0 con 0x7fea1c09c910 2023-10-03T10:12:42.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.439+0000 7fea13fff700 1 -- 172.21.15.3:0/4222747628 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 93178928 0 0) 0x7fea1c09f200 con 0x7fea1c09e1c0 2023-10-03T10:12:42.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.439+0000 7fea20cb8700 1 --2- 172.21.15.3:0/4222747628 >> v2:172.21.15.3:3300/0 conn(0x7fea1c09d850 0x7fea1c09dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:42.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.439+0000 7fea214b9700 1 --2- 172.21.15.3:0/4222747628 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea1c09c910 0x7fea1c09cce0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fea18009850 tx=0x7fea18009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34cd17a03660af5b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:42.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.439+0000 7fea13fff700 1 -- 172.21.15.3:0/4222747628 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea18005e30 con 0x7fea1c09c910 2023-10-03T10:12:42.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.439+0000 7fea13fff700 1 -- 172.21.15.3:0/4222747628 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea18006030 con 0x7fea1c09c910 2023-10-03T10:12:42.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.439+0000 7fea13fff700 1 -- 172.21.15.3:0/4222747628 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea180063c0 con 0x7fea1c09c910 2023-10-03T10:12:42.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.440+0000 7fea2371d700 1 -- 172.21.15.3:0/4222747628 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea1c09c910 msgr2=0x7fea1c09cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:42.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.440+0000 7fea2371d700 1 --2- 172.21.15.3:0/4222747628 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea1c09c910 0x7fea1c09cce0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fea18009850 tx=0x7fea18009b60 comp rx=0 tx=0).stop 2023-10-03T10:12:42.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.440+0000 7fea2371d700 1 -- 172.21.15.3:0/4222747628 shutdown_connections 2023-10-03T10:12:42.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.440+0000 7fea2371d700 1 --2- 172.21.15.3:0/4222747628 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea1c09c910 0x7fea1c09cce0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:42.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.440+0000 7fea2371d700 1 --2- 172.21.15.3:0/4222747628 >> v2:172.21.15.3:3300/0 conn(0x7fea1c09d850 0x7fea1c09dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:42.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.440+0000 7fea2371d700 1 -- 172.21.15.3:0/4222747628 >> 172.21.15.3:0/4222747628 conn(0x7fea1c0a2e50 msgr2=0x7fea1c0a3250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:42.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.440+0000 7fea2371d700 1 -- 172.21.15.3:0/4222747628 shutdown_connections 2023-10-03T10:12:42.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.440+0000 7fea2371d700 1 -- 172.21.15.3:0/4222747628 wait complete. 2023-10-03T10:12:42.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.441+0000 7fea2371d700 1 Processor -- start 2023-10-03T10:12:42.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.441+0000 7fea2371d700 1 -- start start 2023-10-03T10:12:42.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.442+0000 7fea2371d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea1c09c910 0x7fea1c1c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:42.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.442+0000 7fea2371d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea1c09d850 0x7fea1c1c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:42.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.442+0000 7fea2371d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea1c09e1c0 0x7fea1c1c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:42.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.442+0000 7fea2371d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c06eaf0 con 0x7fea1c09e1c0 2023-10-03T10:12:42.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.442+0000 7fea2371d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c0a1190 con 0x7fea1c09c910 2023-10-03T10:12:42.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.442+0000 7fea2371d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c09eed0 con 0x7fea1c09d850 2023-10-03T10:12:42.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.442+0000 7fea214b9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea1c09c910 0x7fea1c1c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:42.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.442+0000 7fea20cb8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea1c09d850 0x7fea1c1c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:42.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.442+0000 7fea214b9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea1c09c910 0x7fea1c1c5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40682/0 (socket says 172.21.15.3:40682) 2023-10-03T10:12:42.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.442+0000 7fea214b9700 1 -- 172.21.15.3:0/3632669968 learned_addr learned my addr 172.21.15.3:0/3632669968 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:42.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.442+0000 7fea21cba700 1 --2- 172.21.15.3:0/3632669968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea1c09e1c0 0x7fea1c1c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:42.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.443+0000 7fea214b9700 1 -- 172.21.15.3:0/3632669968 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea1c09d850 msgr2=0x7fea1c1c5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:42.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.443+0000 7fea214b9700 1 --2- 172.21.15.3:0/3632669968 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea1c09d850 0x7fea1c1c5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:42.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.443+0000 7fea214b9700 1 -- 172.21.15.3:0/3632669968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea1c09e1c0 msgr2=0x7fea1c1c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:42.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.443+0000 7fea214b9700 1 --2- 172.21.15.3:0/3632669968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea1c09e1c0 0x7fea1c1c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:42.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.443+0000 7fea214b9700 1 -- 172.21.15.3:0/3632669968 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea0c009550 con 0x7fea1c09c910 2023-10-03T10:12:42.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.443+0000 7fea214b9700 1 --2- 172.21.15.3:0/3632669968 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea1c09c910 0x7fea1c1c5220 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fea18009ba0 tx=0x7fea18004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:42.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.443+0000 7fea11ffb700 1 -- 172.21.15.3:0/3632669968 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea18004040 con 0x7fea1c09c910 2023-10-03T10:12:42.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.443+0000 7fea11ffb700 1 -- 172.21.15.3:0/3632669968 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea180041c0 con 0x7fea1c09c910 2023-10-03T10:12:42.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.443+0000 7fea11ffb700 1 -- 172.21.15.3:0/3632669968 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea18004570 con 0x7fea1c09c910 2023-10-03T10:12:42.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.443+0000 7fea2371d700 1 -- 172.21.15.3:0/3632669968 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea18009430 con 0x7fea1c09c910 2023-10-03T10:12:42.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.443+0000 7fea2371d700 1 -- 172.21.15.3:0/3632669968 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea1c06df80 con 0x7fea1c09c910 2023-10-03T10:12:42.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.444+0000 7fea11ffb700 1 -- 172.21.15.3:0/3632669968 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fea18022020 con 0x7fea1c09c910 2023-10-03T10:12:42.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.444+0000 7fea11ffb700 1 -- 172.21.15.3:0/3632669968 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4121+0+0 (secure 0 0 0) 0x7fea18040dd0 con 0x7fea1c09c910 2023-10-03T10:12:42.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.444+0000 7fea20cb8700 1 -- 172.21.15.3:0/3632669968 >> v1:172.21.15.3:6816/127025 conn(0x7fea0002f510 legacy=0x7fea000319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:42.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.444+0000 7fea2371d700 1 -- 172.21.15.3:0/3632669968 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea1c06baa0 con 0x7fea1c09c910 2023-10-03T10:12:42.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.449+0000 7fea11ffb700 1 -- 172.21.15.3:0/3632669968 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fea1802a070 con 0x7fea1c09c910 2023-10-03T10:12:42.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.645+0000 7fea20cb8700 1 -- 172.21.15.3:0/3632669968 >> v1:172.21.15.3:6816/127025 conn(0x7fea0002f510 legacy=0x7fea000319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:42.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:42.654+0000 7fea2371d700 1 -- 172.21.15.3:0/3632669968 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_metadata", "var": "pg_autoscale_mode", "val": "off"} v 0) v1 -- 0x7fea1c067ae0 con 0x7fea1c09c910 2023-10-03T10:12:43.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.046+0000 7fea20cb8700 1 -- 172.21.15.3:0/3632669968 >> v1:172.21.15.3:6816/127025 conn(0x7fea0002f510 legacy=0x7fea000319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:43.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.312+0000 7fea11ffb700 1 -- 172.21.15.3:0/3632669968 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_metadata", "var": "pg_autoscale_mode", "val": "off"}]=0 set pool 3 pg_autoscale_mode to off v37) v1 ==== 164+0+0 (secure 0 0 0) 0x7fea18011420 con 0x7fea1c09c910 2023-10-03T10:12:43.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.315+0000 7fea2371d700 1 -- 172.21.15.3:0/3632669968 >> v1:172.21.15.3:6816/127025 conn(0x7fea0002f510 legacy=0x7fea000319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:43.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.315+0000 7fea2371d700 1 -- 172.21.15.3:0/3632669968 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea1c09c910 msgr2=0x7fea1c1c5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:43.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.315+0000 7fea2371d700 1 --2- 172.21.15.3:0/3632669968 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea1c09c910 0x7fea1c1c5220 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fea18009ba0 tx=0x7fea18004750 comp rx=0 tx=0).stop 2023-10-03T10:12:43.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.317+0000 7fea2371d700 1 -- 172.21.15.3:0/3632669968 shutdown_connections 2023-10-03T10:12:43.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.317+0000 7fea2371d700 1 --2- 172.21.15.3:0/3632669968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea1c09e1c0 0x7fea1c1c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:43.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.317+0000 7fea2371d700 1 --2- 172.21.15.3:0/3632669968 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea1c09c910 0x7fea1c1c5220 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:43.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.317+0000 7fea2371d700 1 --2- 172.21.15.3:0/3632669968 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea1c09d850 0x7fea1c1c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:43.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.317+0000 7fea2371d700 1 -- 172.21.15.3:0/3632669968 >> 172.21.15.3:0/3632669968 conn(0x7fea1c0a2e50 msgr2=0x7fea1c1bb0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:43.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.317+0000 7fea2371d700 1 -- 172.21.15.3:0/3632669968 shutdown_connections 2023-10-03T10:12:43.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.317+0000 7fea2371d700 1 -- 172.21.15.3:0/3632669968 wait complete. 2023-10-03T10:12:43.319 INFO:teuthology.orchestra.run.smithi003.stderr:set pool 3 pg_autoscale_mode to off 2023-10-03T10:12:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.441+0000 7f286e2b6700 1 Processor -- start 2023-10-03T10:12:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.441+0000 7f286e2b6700 1 -- start start 2023-10-03T10:12:43.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.441+0000 7f286e2b6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f286809d950 0x7f286809dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:43.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.441+0000 7f286e2b6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2868133e50 0x7f286813bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:43.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.442+0000 7f286e2b6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f286806b5a0 con 0x7f286809e260 2023-10-03T10:12:43.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.442+0000 7f286e2b6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f286809ab70 con 0x7f2868133e50 2023-10-03T10:12:43.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.442+0000 7f286e2b6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f286809f3d0 con 0x7f286809d950 2023-10-03T10:12:43.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.442+0000 7f2867fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f286809d950 0x7f286809dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:43.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.442+0000 7f286c853700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2868133e50 0x7f286813bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:43.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.442+0000 7f286c853700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2868133e50 0x7f286813bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55726/0 (socket says 172.21.15.3:55726) 2023-10-03T10:12:43.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.442+0000 7f28677fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f286809e260 0x7f2868133740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50592/0 (socket says 172.21.15.3:50592) 2023-10-03T10:12:43.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.442+0000 7f2867fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f286809d950 0x7f286809dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38390/0 (socket says 172.21.15.3:38390) 2023-10-03T10:12:43.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.442+0000 7f286c853700 1 -- 172.21.15.3:0/2709518987 learned_addr learned my addr 172.21.15.3:0/2709518987 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:43.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.442+0000 7f286c853700 1 -- 172.21.15.3:0/2709518987 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f286809d950 msgr2=0x7f286809dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:43.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.442+0000 7f286c853700 1 --2- 172.21.15.3:0/2709518987 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f286809d950 0x7f286809dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:43.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.442+0000 7f286c853700 1 -- 172.21.15.3:0/2709518987 >> v1:172.21.15.3:6789/0 conn(0x7f286809e260 legacy=0x7f2868133740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:43.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.442+0000 7f286c853700 1 -- 172.21.15.3:0/2709518987 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f286809a560 con 0x7f2868133e50 2023-10-03T10:12:43.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.442+0000 7f2867fff700 1 --2- 172.21.15.3:0/2709518987 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f286809d950 0x7f286809dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:43.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.442+0000 7f2866ffd700 1 -- 172.21.15.3:0/2709518987 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 572894835 0 0) 0x7f286806b5a0 con 0x7f286809e260 2023-10-03T10:12:43.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.442+0000 7f286c853700 1 --2- 172.21.15.3:0/2709518987 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2868133e50 0x7f286813bc20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2860009e50 tx=0x7f286000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5cc9405fce54787 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:43.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.442+0000 7f2866ffd700 1 -- 172.21.15.3:0/2709518987 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286000e070 con 0x7f2868133e50 2023-10-03T10:12:43.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.442+0000 7f2866ffd700 1 -- 172.21.15.3:0/2709518987 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286000b910 con 0x7f2868133e50 2023-10-03T10:12:43.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.442+0000 7f2866ffd700 1 -- 172.21.15.3:0/2709518987 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286000bcc0 con 0x7f2868133e50 2023-10-03T10:12:43.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.443+0000 7f286e2b6700 1 -- 172.21.15.3:0/2709518987 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2868133e50 msgr2=0x7f286813bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:43.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.443+0000 7f286e2b6700 1 --2- 172.21.15.3:0/2709518987 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2868133e50 0x7f286813bc20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2860009e50 tx=0x7f286000b330 comp rx=0 tx=0).stop 2023-10-03T10:12:43.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.443+0000 7f286e2b6700 1 -- 172.21.15.3:0/2709518987 shutdown_connections 2023-10-03T10:12:43.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.443+0000 7f286e2b6700 1 --2- 172.21.15.3:0/2709518987 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2868133e50 0x7f286813bc20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:43.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.443+0000 7f286e2b6700 1 --2- 172.21.15.3:0/2709518987 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f286809d950 0x7f286809dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:43.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.443+0000 7f286e2b6700 1 -- 172.21.15.3:0/2709518987 >> 172.21.15.3:0/2709518987 conn(0x7f28680a2f50 msgr2=0x7f28680a3350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:43.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.443+0000 7f286e2b6700 1 -- 172.21.15.3:0/2709518987 shutdown_connections 2023-10-03T10:12:43.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.443+0000 7f286e2b6700 1 -- 172.21.15.3:0/2709518987 wait complete. 2023-10-03T10:12:43.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.444+0000 7f286e2b6700 1 Processor -- start 2023-10-03T10:12:43.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.445+0000 7f286e2b6700 1 -- start start 2023-10-03T10:12:43.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.445+0000 7f286e2b6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f286809d950 0x7f286811cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:43.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.445+0000 7f286e2b6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f286809e260 0x7f286811d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:43.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.445+0000 7f286e2b6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2868133e50 0x7f286811da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:43.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.445+0000 7f286e2b6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2868121480 con 0x7f2868133e50 2023-10-03T10:12:43.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.445+0000 7f286e2b6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f286811ebb0 con 0x7f286809d950 2023-10-03T10:12:43.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.445+0000 7f286e2b6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f286811fe90 con 0x7f286809e260 2023-10-03T10:12:43.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.445+0000 7f2867fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f286809d950 0x7f286811cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:43.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.445+0000 7f2867fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f286809d950 0x7f286811cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59780/0 (socket says 172.21.15.3:59780) 2023-10-03T10:12:43.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.445+0000 7f28677fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f286809e260 0x7f286811d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:43.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.445+0000 7f2867fff700 1 -- 172.21.15.3:0/1834029436 learned_addr learned my addr 172.21.15.3:0/1834029436 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:43.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.446+0000 7f286c853700 1 --2- 172.21.15.3:0/1834029436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2868133e50 0x7f286811da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:43.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.446+0000 7f2867fff700 1 -- 172.21.15.3:0/1834029436 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f286809e260 msgr2=0x7f286811d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:43.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.446+0000 7f2867fff700 1 --2- 172.21.15.3:0/1834029436 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f286809e260 0x7f286811d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:43.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.446+0000 7f2867fff700 1 -- 172.21.15.3:0/1834029436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2868133e50 msgr2=0x7f286811da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:43.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.446+0000 7f2867fff700 1 --2- 172.21.15.3:0/1834029436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2868133e50 0x7f286811da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:43.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.446+0000 7f2867fff700 1 -- 172.21.15.3:0/1834029436 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2858009430 con 0x7f286809d950 2023-10-03T10:12:43.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.446+0000 7f2867fff700 1 --2- 172.21.15.3:0/1834029436 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f286809d950 0x7f286811cfa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f285800ec10 tx=0x7f285800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:43.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.446+0000 7f2864ff9700 1 -- 172.21.15.3:0/1834029436 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2858009740 con 0x7f286809d950 2023-10-03T10:12:43.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.446+0000 7f2864ff9700 1 -- 172.21.15.3:0/1834029436 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2858005a20 con 0x7f286809d950 2023-10-03T10:12:43.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.446+0000 7f286e2b6700 1 -- 172.21.15.3:0/1834029436 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2860009b30 con 0x7f286809d950 2023-10-03T10:12:43.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.446+0000 7f2864ff9700 1 -- 172.21.15.3:0/1834029436 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2858005d90 con 0x7f286809d950 2023-10-03T10:12:43.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.446+0000 7f286e2b6700 1 -- 172.21.15.3:0/1834029436 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f286809cbd0 con 0x7f286809d950 2023-10-03T10:12:43.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.447+0000 7f2864ff9700 1 -- 172.21.15.3:0/1834029436 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f285800c850 con 0x7f286809d950 2023-10-03T10:12:43.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.447+0000 7f2864ff9700 1 -- 172.21.15.3:0/1834029436 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f285803fdc0 con 0x7f286809d950 2023-10-03T10:12:43.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.447+0000 7f28677fe700 1 -- 172.21.15.3:0/1834029436 >> v1:172.21.15.3:6816/127025 conn(0x7f285402f510 legacy=0x7f28540319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:43.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.447+0000 7f286e2b6700 1 -- 172.21.15.3:0/1834029436 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f286804f2a0 con 0x7f286809d950 2023-10-03T10:12:43.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.452+0000 7f2864ff9700 1 -- 172.21.15.3:0/1834029436 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2858006030 con 0x7f286809d950 2023-10-03T10:12:43.547 INFO:tasks.ceph.mgr.x.smithi003.stderr:daemon-helper: command crashed with signal 6 2023-10-03T10:12:43.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.648+0000 7f28677fe700 1 -- 172.21.15.3:0/1834029436 >> v1:172.21.15.3:6816/127025 conn(0x7f285402f510 legacy=0x7f28540319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:43.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:43.658+0000 7f286e2b6700 1 -- 172.21.15.3:0/1834029436 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data", "var": "pg_autoscale_mode", "val": "off"} v 0) v1 -- 0x7f28680727b0 con 0x7f286809d950 2023-10-03T10:12:44.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.049+0000 7f28677fe700 1 -- 172.21.15.3:0/1834029436 >> v1:172.21.15.3:6816/127025 conn(0x7f285402f510 legacy=0x7f28540319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:44.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.321+0000 7f2864ff9700 1 -- 172.21.15.3:0/1834029436 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data", "var": "pg_autoscale_mode", "val": "off"}]=0 set pool 4 pg_autoscale_mode to off v38) v1 ==== 160+0+0 (secure 0 0 0) 0x7f2858011b40 con 0x7f286809d950 2023-10-03T10:12:44.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.325+0000 7f286e2b6700 1 -- 172.21.15.3:0/1834029436 >> v1:172.21.15.3:6816/127025 conn(0x7f285402f510 legacy=0x7f28540319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:44.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.325+0000 7f286e2b6700 1 -- 172.21.15.3:0/1834029436 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f286809d950 msgr2=0x7f286811cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:44.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.325+0000 7f286e2b6700 1 --2- 172.21.15.3:0/1834029436 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f286809d950 0x7f286811cfa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f285800ec10 tx=0x7f285800ef20 comp rx=0 tx=0).stop 2023-10-03T10:12:44.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.325+0000 7f286e2b6700 1 -- 172.21.15.3:0/1834029436 shutdown_connections 2023-10-03T10:12:44.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.325+0000 7f286e2b6700 1 --2- 172.21.15.3:0/1834029436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2868133e50 0x7f286811da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:44.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.325+0000 7f286e2b6700 1 --2- 172.21.15.3:0/1834029436 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f286809d950 0x7f286811cfa0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:44.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.325+0000 7f286e2b6700 1 --2- 172.21.15.3:0/1834029436 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f286809e260 0x7f286811d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:44.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.325+0000 7f286e2b6700 1 -- 172.21.15.3:0/1834029436 >> 172.21.15.3:0/1834029436 conn(0x7f28680a2f50 msgr2=0x7f28681320b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:44.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.325+0000 7f286e2b6700 1 -- 172.21.15.3:0/1834029436 shutdown_connections 2023-10-03T10:12:44.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.325+0000 7f286e2b6700 1 -- 172.21.15.3:0/1834029436 wait complete. 2023-10-03T10:12:44.327 INFO:teuthology.orchestra.run.smithi003.stderr:set pool 4 pg_autoscale_mode to off 2023-10-03T10:12:44.341 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd require-osd-release reef' 2023-10-03T10:12:44.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.480+0000 7f6750d21700 1 Processor -- start 2023-10-03T10:12:44.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.480+0000 7f6750d21700 1 -- start start 2023-10-03T10:12:44.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.481+0000 7f6750d21700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f674c133b00 0x7f674c133ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:44.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.481+0000 7f6750d21700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f674c134410 0x7f674c09d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:44.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.481+0000 7f6750d21700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f674c09f640 con 0x7f674c09de90 2023-10-03T10:12:44.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.481+0000 7f6750d21700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674c09a790 con 0x7f674c134410 2023-10-03T10:12:44.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.481+0000 7f6750d21700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f674c09a4f0 con 0x7f674c133b00 2023-10-03T10:12:44.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.481+0000 7f674a59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f674c133b00 0x7f674c133ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:44.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.481+0000 7f674ad9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f674c09de90 0x7f674c13aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50596/0 (socket says 172.21.15.3:50596) 2023-10-03T10:12:44.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.481+0000 7f6749d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f674c134410 0x7f674c09d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:44.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.481+0000 7f674ad9d700 1 -- 172.21.15.3:0/3455299776 learned_addr learned my addr 172.21.15.3:0/3455299776 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:44.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.482+0000 7f674a59c700 1 -- 172.21.15.3:0/3455299776 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f674c134410 msgr2=0x7f674c09d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:44.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.482+0000 7f674a59c700 1 --2- 172.21.15.3:0/3455299776 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f674c134410 0x7f674c09d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.482+0000 7f674959a700 1 -- 172.21.15.3:0/3455299776 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2436502794 0 0) 0x7f674c09f640 con 0x7f674c09de90 2023-10-03T10:12:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.482+0000 7f674a59c700 1 -- 172.21.15.3:0/3455299776 >> v1:172.21.15.3:6789/0 conn(0x7f674c09de90 legacy=0x7f674c13aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.482+0000 7f674a59c700 1 -- 172.21.15.3:0/3455299776 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f674c09eb20 con 0x7f674c133b00 2023-10-03T10:12:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.482+0000 7f6749d9b700 1 --2- 172.21.15.3:0/3455299776 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f674c134410 0x7f674c09d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.482+0000 7f674a59c700 1 --2- 172.21.15.3:0/3455299776 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f674c133b00 0x7f674c133ed0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f673c0098a0 tx=0x7f673c009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e88ab54032e45b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.482+0000 7f674959a700 1 -- 172.21.15.3:0/3455299776 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c011830 con 0x7f674c133b00 2023-10-03T10:12:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.482+0000 7f674959a700 1 -- 172.21.15.3:0/3455299776 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673c011990 con 0x7f674c133b00 2023-10-03T10:12:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.482+0000 7f674959a700 1 -- 172.21.15.3:0/3455299776 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c011d20 con 0x7f674c133b00 2023-10-03T10:12:44.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.483+0000 7f6750d21700 1 -- 172.21.15.3:0/3455299776 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f674c133b00 msgr2=0x7f674c133ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:44.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.483+0000 7f6750d21700 1 --2- 172.21.15.3:0/3455299776 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f674c133b00 0x7f674c133ed0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f673c0098a0 tx=0x7f673c009bb0 comp rx=0 tx=0).stop 2023-10-03T10:12:44.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.483+0000 7f6750d21700 1 -- 172.21.15.3:0/3455299776 shutdown_connections 2023-10-03T10:12:44.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.483+0000 7f6750d21700 1 --2- 172.21.15.3:0/3455299776 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f674c133b00 0x7f674c133ed0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:44.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.483+0000 7f6750d21700 1 --2- 172.21.15.3:0/3455299776 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f674c134410 0x7f674c09d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:44.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.483+0000 7f6750d21700 1 -- 172.21.15.3:0/3455299776 >> 172.21.15.3:0/3455299776 conn(0x7f674c0a2f50 msgr2=0x7f674c0a3350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:44.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.483+0000 7f6750d21700 1 -- 172.21.15.3:0/3455299776 shutdown_connections 2023-10-03T10:12:44.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.483+0000 7f6750d21700 1 -- 172.21.15.3:0/3455299776 wait complete. 2023-10-03T10:12:44.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.484+0000 7f6750d21700 1 Processor -- start 2023-10-03T10:12:44.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.484+0000 7f6750d21700 1 -- start start 2023-10-03T10:12:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.484+0000 7f6750d21700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f674c09de90 0x7f674c11e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.484+0000 7f6750d21700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f674c133b00 0x7f674c11e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.484+0000 7f6750d21700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f674c134410 0x7f674c11ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.484+0000 7f6750d21700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674c121ae0 con 0x7f674c09de90 2023-10-03T10:12:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.484+0000 7f6750d21700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674c1201b0 con 0x7f674c134410 2023-10-03T10:12:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.484+0000 7f6750d21700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f674c121270 con 0x7f674c133b00 2023-10-03T10:12:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.485+0000 7f6749d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f674c133b00 0x7f674c11e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.485+0000 7f674a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f674c09de90 0x7f674c11e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.485+0000 7f674ad9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f674c134410 0x7f674c11ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.485+0000 7f6749d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f674c133b00 0x7f674c11e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38422/0 (socket says 172.21.15.3:38422) 2023-10-03T10:12:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.485+0000 7f6749d9b700 1 -- 172.21.15.3:0/1257678492 learned_addr learned my addr 172.21.15.3:0/1257678492 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.485+0000 7f674a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f674c09de90 0x7f674c11e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55736/0 (socket says 172.21.15.3:55736) 2023-10-03T10:12:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.485+0000 7f6749d9b700 1 -- 172.21.15.3:0/1257678492 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f674c134410 msgr2=0x7f674c11ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.485+0000 7f6749d9b700 1 --2- 172.21.15.3:0/1257678492 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f674c134410 0x7f674c11ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.485+0000 7f6749d9b700 1 -- 172.21.15.3:0/1257678492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f674c09de90 msgr2=0x7f674c11e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.485+0000 7f6749d9b700 1 --2- 172.21.15.3:0/1257678492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f674c09de90 0x7f674c11e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.485+0000 7f6749d9b700 1 -- 172.21.15.3:0/1257678492 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6740009430 con 0x7f674c133b00 2023-10-03T10:12:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.485+0000 7f674ad9d700 1 --2- 172.21.15.3:0/1257678492 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f674c134410 0x7f674c11ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.485+0000 7f6749d9b700 1 --2- 172.21.15.3:0/1257678492 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f674c133b00 0x7f674c11e590 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f674000ec10 tx=0x7f674000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.485+0000 7f673affd700 1 -- 172.21.15.3:0/1257678492 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6740005a20 con 0x7f674c133b00 2023-10-03T10:12:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.485+0000 7f673affd700 1 -- 172.21.15.3:0/1257678492 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6740005b80 con 0x7f674c133b00 2023-10-03T10:12:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.485+0000 7f6750d21700 1 -- 172.21.15.3:0/1257678492 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f673c0094a0 con 0x7f674c133b00 2023-10-03T10:12:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.485+0000 7f673affd700 1 -- 172.21.15.3:0/1257678492 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674000c850 con 0x7f674c133b00 2023-10-03T10:12:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.485+0000 7f6750d21700 1 -- 172.21.15.3:0/1257678492 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f674c09ccb0 con 0x7f674c133b00 2023-10-03T10:12:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.486+0000 7f673affd700 1 -- 172.21.15.3:0/1257678492 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f674000c9b0 con 0x7f674c133b00 2023-10-03T10:12:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.486+0000 7f673affd700 1 -- 172.21.15.3:0/1257678492 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f674003fdb0 con 0x7f674c133b00 2023-10-03T10:12:44.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.486+0000 7f674a59c700 1 -- 172.21.15.3:0/1257678492 >> v1:172.21.15.3:6816/127025 conn(0x7f672c02f510 legacy=0x7f672c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:44.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.486+0000 7f6750d21700 1 -- 172.21.15.3:0/1257678492 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f674c11ad40 con 0x7f674c133b00 2023-10-03T10:12:44.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.492+0000 7f673affd700 1 -- 172.21.15.3:0/1257678492 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6740020020 con 0x7f674c133b00 2023-10-03T10:12:44.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.687+0000 7f674a59c700 1 -- 172.21.15.3:0/1257678492 >> v1:172.21.15.3:6816/127025 conn(0x7f672c02f510 legacy=0x7f672c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:44.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:44.707+0000 7f6750d21700 1 -- 172.21.15.3:0/1257678492 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd require-osd-release", "release": "reef"} v 0) v1 -- 0x7f674c121660 con 0x7f674c133b00 2023-10-03T10:12:45.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.088+0000 7f674a59c700 1 -- 172.21.15.3:0/1257678492 >> v1:172.21.15.3:6816/127025 conn(0x7f672c02f510 legacy=0x7f672c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:45.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.330+0000 7f673affd700 1 -- 172.21.15.3:0/1257678492 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd require-osd-release", "release": "reef"}]=0 v39) v1 ==== 90+0+0 (secure 0 0 0) 0x7f674c121660 con 0x7f674c133b00 2023-10-03T10:12:45.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.336+0000 7f6750d21700 1 -- 172.21.15.3:0/1257678492 >> v1:172.21.15.3:6816/127025 conn(0x7f672c02f510 legacy=0x7f672c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:45.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.336+0000 7f6750d21700 1 -- 172.21.15.3:0/1257678492 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f674c133b00 msgr2=0x7f674c11e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:45.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.336+0000 7f6750d21700 1 --2- 172.21.15.3:0/1257678492 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f674c133b00 0x7f674c11e590 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f674000ec10 tx=0x7f674000ef20 comp rx=0 tx=0).stop 2023-10-03T10:12:45.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.337+0000 7f6750d21700 1 -- 172.21.15.3:0/1257678492 shutdown_connections 2023-10-03T10:12:45.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.337+0000 7f6750d21700 1 --2- 172.21.15.3:0/1257678492 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f674c134410 0x7f674c11ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:45.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.337+0000 7f6750d21700 1 --2- 172.21.15.3:0/1257678492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f674c09de90 0x7f674c11e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:45.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.337+0000 7f6750d21700 1 --2- 172.21.15.3:0/1257678492 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f674c133b00 0x7f674c11e590 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:45.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.337+0000 7f6750d21700 1 -- 172.21.15.3:0/1257678492 >> 172.21.15.3:0/1257678492 conn(0x7f674c0a2f50 msgr2=0x7f674c132e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:45.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.337+0000 7f6750d21700 1 -- 172.21.15.3:0/1257678492 shutdown_connections 2023-10-03T10:12:45.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.337+0000 7f6750d21700 1 -- 172.21.15.3:0/1257678492 wait complete. 2023-10-03T10:12:45.352 INFO:teuthology.run_tasks:Running task ceph.healthy... 2023-10-03T10:12:45.363 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-03T10:12:45.363 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-03T10:12:45.363 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-03T10:12:45.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.505+0000 7efe68c7e700 1 Processor -- start 2023-10-03T10:12:45.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.505+0000 7efe68c7e700 1 -- start start 2023-10-03T10:12:45.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.505+0000 7efe68c7e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efe6409d6f0 0x7efe6409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:45.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.506+0000 7efe68c7e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efe6409e000 0x7efe64130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:45.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.506+0000 7efe68c7e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efe6409e410 0x7efe64130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:45.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.506+0000 7efe68c7e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe6409f520 con 0x7efe6409e410 2023-10-03T10:12:45.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.506+0000 7efe68c7e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe6409f130 con 0x7efe6409d6f0 2023-10-03T10:12:45.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.506+0000 7efe68c7e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efe6409e930 con 0x7efe6409e000 2023-10-03T10:12:45.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.506+0000 7efe61d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efe6409e000 0x7efe64130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:45.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.506+0000 7efe62d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efe6409e410 0x7efe64130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:45.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.506+0000 7efe62d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efe6409e410 0x7efe64130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55738/0 (socket says 172.21.15.3:55738) 2023-10-03T10:12:45.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.506+0000 7efe61d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efe6409e000 0x7efe64130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38436/0 (socket says 172.21.15.3:38436) 2023-10-03T10:12:45.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.506+0000 7efe62d9d700 1 -- 172.21.15.3:0/1425941975 learned_addr learned my addr 172.21.15.3:0/1425941975 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:45.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.506+0000 7efe6259c700 1 --2- 172.21.15.3:0/1425941975 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efe6409d6f0 0x7efe6409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:45.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.506+0000 7efe62d9d700 1 -- 172.21.15.3:0/1425941975 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efe6409e000 msgr2=0x7efe64130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:45.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.506+0000 7efe62d9d700 1 --2- 172.21.15.3:0/1425941975 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efe6409e000 0x7efe64130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:45.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.506+0000 7efe62d9d700 1 -- 172.21.15.3:0/1425941975 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efe6409d6f0 msgr2=0x7efe6409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:45.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.506+0000 7efe62d9d700 1 --2- 172.21.15.3:0/1425941975 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efe6409d6f0 0x7efe6409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:45.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.506+0000 7efe62d9d700 1 -- 172.21.15.3:0/1425941975 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6409b2d0 con 0x7efe6409e410 2023-10-03T10:12:45.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.506+0000 7efe61d9b700 1 --2- 172.21.15.3:0/1425941975 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efe6409e000 0x7efe64130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:45.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.506+0000 7efe6259c700 1 --2- 172.21.15.3:0/1425941975 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efe6409d6f0 0x7efe6409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:45.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.506+0000 7efe62d9d700 1 --2- 172.21.15.3:0/1425941975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe6409e410 0x7efe64130ba0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00d150 tx=0x7efe5c00d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca42ab0f14d17f4b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:45.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.506+0000 7efe6159a700 1 -- 172.21.15.3:0/1425941975 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c010070 con 0x7efe6409e410 2023-10-03T10:12:45.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.506+0000 7efe6159a700 1 -- 172.21.15.3:0/1425941975 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe5c00dd10 con 0x7efe6409e410 2023-10-03T10:12:45.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.506+0000 7efe6159a700 1 -- 172.21.15.3:0/1425941975 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c00b310 con 0x7efe6409e410 2023-10-03T10:12:45.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.507+0000 7efe68c7e700 1 -- 172.21.15.3:0/1425941975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe6409e410 msgr2=0x7efe64130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:45.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.507+0000 7efe68c7e700 1 --2- 172.21.15.3:0/1425941975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe6409e410 0x7efe64130ba0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00d150 tx=0x7efe5c00d460 comp rx=0 tx=0).stop 2023-10-03T10:12:45.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.507+0000 7efe68c7e700 1 -- 172.21.15.3:0/1425941975 shutdown_connections 2023-10-03T10:12:45.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.507+0000 7efe68c7e700 1 --2- 172.21.15.3:0/1425941975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe6409e410 0x7efe64130ba0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:45.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.507+0000 7efe68c7e700 1 --2- 172.21.15.3:0/1425941975 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efe6409d6f0 0x7efe6409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:45.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.507+0000 7efe68c7e700 1 --2- 172.21.15.3:0/1425941975 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efe6409e000 0x7efe64130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:45.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.507+0000 7efe68c7e700 1 -- 172.21.15.3:0/1425941975 >> 172.21.15.3:0/1425941975 conn(0x7efe64066f50 msgr2=0x7efe6409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:45.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.507+0000 7efe68c7e700 1 -- 172.21.15.3:0/1425941975 shutdown_connections 2023-10-03T10:12:45.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.507+0000 7efe68c7e700 1 -- 172.21.15.3:0/1425941975 wait complete. 2023-10-03T10:12:45.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.508+0000 7efe68c7e700 1 Processor -- start 2023-10-03T10:12:45.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.509+0000 7efe68c7e700 1 -- start start 2023-10-03T10:12:45.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.509+0000 7efe68c7e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efe6409d6f0 0x7efe6411de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:45.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.509+0000 7efe68c7e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe6409e000 0x7efe6411e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:45.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.509+0000 7efe68c7e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efe6409e410 0x7efe6411e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:45.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.509+0000 7efe68c7e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe64121960 con 0x7efe6409e000 2023-10-03T10:12:45.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.509+0000 7efe68c7e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe64120d30 con 0x7efe6409e410 2023-10-03T10:12:45.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.509+0000 7efe68c7e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efe640a15d0 con 0x7efe6409d6f0 2023-10-03T10:12:45.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.509+0000 7efe6259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efe6409d6f0 0x7efe6411de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:45.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.509+0000 7efe62d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efe6409e410 0x7efe6411e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:45.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.509+0000 7efe61d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe6409e000 0x7efe6411e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:45.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.509+0000 7efe6259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efe6409d6f0 0x7efe6411de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38448/0 (socket says 172.21.15.3:38448) 2023-10-03T10:12:45.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.509+0000 7efe6259c700 1 -- 172.21.15.3:0/4021855709 learned_addr learned my addr 172.21.15.3:0/4021855709 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:45.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.509+0000 7efe62d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efe6409e410 0x7efe6411e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59816/0 (socket says 172.21.15.3:59816) 2023-10-03T10:12:45.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.509+0000 7efe6259c700 1 -- 172.21.15.3:0/4021855709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efe6409e410 msgr2=0x7efe6411e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.509+0000 7efe6259c700 1 --2- 172.21.15.3:0/4021855709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efe6409e410 0x7efe6411e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.509+0000 7efe6259c700 1 -- 172.21.15.3:0/4021855709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe6409e000 msgr2=0x7efe6411e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.509+0000 7efe6259c700 1 --2- 172.21.15.3:0/4021855709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe6409e000 0x7efe6411e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.509+0000 7efe6259c700 1 -- 172.21.15.3:0/4021855709 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe540094a0 con 0x7efe6409d6f0 2023-10-03T10:12:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.509+0000 7efe61d9b700 1 --2- 172.21.15.3:0/4021855709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe6409e000 0x7efe6411e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.510+0000 7efe6259c700 1 --2- 172.21.15.3:0/4021855709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efe6409d6f0 0x7efe6411de90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efe5400ebe0 tx=0x7efe5400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.510+0000 7efe52ffd700 1 -- 172.21.15.3:0/4021855709 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe54009780 con 0x7efe6409d6f0 2023-10-03T10:12:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.510+0000 7efe52ffd700 1 -- 172.21.15.3:0/4021855709 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe5400c850 con 0x7efe6409d6f0 2023-10-03T10:12:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.510+0000 7efe52ffd700 1 -- 172.21.15.3:0/4021855709 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5400cbe0 con 0x7efe6409d6f0 2023-10-03T10:12:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.510+0000 7efe68c7e700 1 -- 172.21.15.3:0/4021855709 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe580094a0 con 0x7efe6409d6f0 2023-10-03T10:12:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.510+0000 7efe68c7e700 1 -- 172.21.15.3:0/4021855709 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe5c00bd40 con 0x7efe6409d6f0 2023-10-03T10:12:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.511+0000 7efe52ffd700 1 -- 172.21.15.3:0/4021855709 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7efe5400cd40 con 0x7efe6409d6f0 2023-10-03T10:12:45.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.511+0000 7efe68c7e700 1 -- 172.21.15.3:0/4021855709 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe64074ea0 con 0x7efe6409d6f0 2023-10-03T10:12:45.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.511+0000 7efe61d9b700 1 -- 172.21.15.3:0/4021855709 >> v1:172.21.15.3:6816/127025 conn(0x7efe4402f510 legacy=0x7efe440319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:45.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.511+0000 7efe52ffd700 1 -- 172.21.15.3:0/4021855709 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7efe5403fff0 con 0x7efe6409d6f0 2023-10-03T10:12:45.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.516+0000 7efe52ffd700 1 -- 172.21.15.3:0/4021855709 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efe54071750 con 0x7efe6409d6f0 2023-10-03T10:12:45.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.712+0000 7efe61d9b700 1 -- 172.21.15.3:0/4021855709 >> v1:172.21.15.3:6816/127025 conn(0x7efe4402f510 legacy=0x7efe440319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:45.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.775+0000 7efe68c7e700 1 -- 172.21.15.3:0/4021855709 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7efe6406c360 con 0x7efe6409d6f0 2023-10-03T10:12:45.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.777+0000 7efe52ffd700 1 -- 172.21.15.3:0/4021855709 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v9) v1 ==== 74+0+70476 (secure 0 0 0) 0x7efe54013990 con 0x7efe6409d6f0 2023-10-03T10:12:45.778 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:12:45.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.779+0000 7efe68c7e700 1 -- 172.21.15.3:0/4021855709 >> v1:172.21.15.3:6816/127025 conn(0x7efe4402f510 legacy=0x7efe440319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:45.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.779+0000 7efe68c7e700 1 -- 172.21.15.3:0/4021855709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efe6409d6f0 msgr2=0x7efe6411de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:45.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.779+0000 7efe68c7e700 1 --2- 172.21.15.3:0/4021855709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efe6409d6f0 0x7efe6411de90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efe5400ebe0 tx=0x7efe5400eef0 comp rx=0 tx=0).stop 2023-10-03T10:12:45.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.779+0000 7efe68c7e700 1 -- 172.21.15.3:0/4021855709 shutdown_connections 2023-10-03T10:12:45.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.779+0000 7efe68c7e700 1 --2- 172.21.15.3:0/4021855709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efe6409e410 0x7efe6411e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:45.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.779+0000 7efe68c7e700 1 --2- 172.21.15.3:0/4021855709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efe6409d6f0 0x7efe6411de90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:45.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.779+0000 7efe68c7e700 1 --2- 172.21.15.3:0/4021855709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe6409e000 0x7efe6411e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:45.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.779+0000 7efe68c7e700 1 -- 172.21.15.3:0/4021855709 >> 172.21.15.3:0/4021855709 conn(0x7efe64066f50 msgr2=0x7efe64070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:45.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.779+0000 7efe68c7e700 1 -- 172.21.15.3:0/4021855709 shutdown_connections 2023-10-03T10:12:45.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.779+0000 7efe68c7e700 1 -- 172.21.15.3:0/4021855709 wait complete. 2023-10-03T10:12:45.793 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":9,"active_gid":14098,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6816","nonce":127025}]},"active_addr":"172.21.15.3:6816/127025","active_change":"2023-10-03T10:12:13.305595+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":21,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":3571619151}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":2059330378}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":3289486158}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":3264604155}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":3246155339}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":1911521769}]}]} 2023-10-03T10:12:45.795 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-03T10:12:45.795 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-03T10:12:45.795 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-03T10:12:45.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.932+0000 7f6507f05700 1 Processor -- start 2023-10-03T10:12:45.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.933+0000 7f6507f05700 1 -- start start 2023-10-03T10:12:45.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.933+0000 7f6507f05700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f650009e1d0 0x7f6500138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:45.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.933+0000 7f6507f05700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6500139170 0x7f650013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:45.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.933+0000 7f6507f05700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f650009b120 con 0x7f650009d6f0 2023-10-03T10:12:45.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.933+0000 7f6507f05700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f650006b460 con 0x7f6500139170 2023-10-03T10:12:45.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.933+0000 7f6507f05700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650009a7e0 con 0x7f650009e1d0 2023-10-03T10:12:45.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.933+0000 7f65054a0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f650009e1d0 0x7f6500138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:45.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.933+0000 7f65064a2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6500139170 0x7f650013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:45.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.933+0000 7f65054a0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f650009e1d0 0x7f6500138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59828/0 (socket says 172.21.15.3:59828) 2023-10-03T10:12:45.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.933+0000 7f65064a2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6500139170 0x7f650013b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55754/0 (socket says 172.21.15.3:55754) 2023-10-03T10:12:45.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.933+0000 7f65064a2700 1 -- 172.21.15.3:0/2006802579 learned_addr learned my addr 172.21.15.3:0/2006802579 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:45.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.933+0000 7f6505ca1700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f650009d6f0 0x7f650009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50598/0 (socket says 172.21.15.3:50598) 2023-10-03T10:12:45.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.934+0000 7f65064a2700 1 -- 172.21.15.3:0/2006802579 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f650009e1d0 msgr2=0x7f6500138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:45.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.934+0000 7f65064a2700 1 --2- 172.21.15.3:0/2006802579 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f650009e1d0 0x7f6500138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:45.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.934+0000 7f65064a2700 1 -- 172.21.15.3:0/2006802579 >> v1:172.21.15.3:6789/0 conn(0x7f650009d6f0 legacy=0x7f650009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:45.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.934+0000 7f65064a2700 1 -- 172.21.15.3:0/2006802579 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f650009a500 con 0x7f6500139170 2023-10-03T10:12:45.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.934+0000 7f6504c9f700 1 -- 172.21.15.3:0/2006802579 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 33167776 0 0) 0x7f650009b120 con 0x7f650009d6f0 2023-10-03T10:12:45.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.934+0000 7f65054a0700 1 --2- 172.21.15.3:0/2006802579 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f650009e1d0 0x7f6500138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:45.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.934+0000 7f65064a2700 1 --2- 172.21.15.3:0/2006802579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6500139170 0x7f650013b550 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f64f80098f0 tx=0x7f64f8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b748a66b2842b98 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:45.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.934+0000 7f6504c9f700 1 -- 172.21.15.3:0/2006802579 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f8011830 con 0x7f6500139170 2023-10-03T10:12:45.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.934+0000 7f6504c9f700 1 -- 172.21.15.3:0/2006802579 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f8011990 con 0x7f6500139170 2023-10-03T10:12:45.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.934+0000 7f6504c9f700 1 -- 172.21.15.3:0/2006802579 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f8011d20 con 0x7f6500139170 2023-10-03T10:12:45.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.935+0000 7f6507f05700 1 -- 172.21.15.3:0/2006802579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6500139170 msgr2=0x7f650013b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:45.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.935+0000 7f6507f05700 1 --2- 172.21.15.3:0/2006802579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6500139170 0x7f650013b550 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f64f80098f0 tx=0x7f64f8009c00 comp rx=0 tx=0).stop 2023-10-03T10:12:45.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.935+0000 7f6507f05700 1 -- 172.21.15.3:0/2006802579 shutdown_connections 2023-10-03T10:12:45.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.935+0000 7f6507f05700 1 --2- 172.21.15.3:0/2006802579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6500139170 0x7f650013b550 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:45.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.935+0000 7f6507f05700 1 --2- 172.21.15.3:0/2006802579 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f650009e1d0 0x7f6500138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:45.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.935+0000 7f6507f05700 1 -- 172.21.15.3:0/2006802579 >> 172.21.15.3:0/2006802579 conn(0x7f6500066f50 msgr2=0x7f650009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:45.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.935+0000 7f6507f05700 1 -- 172.21.15.3:0/2006802579 shutdown_connections 2023-10-03T10:12:45.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.935+0000 7f6507f05700 1 -- 172.21.15.3:0/2006802579 wait complete. 2023-10-03T10:12:45.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.936+0000 7f6507f05700 1 Processor -- start 2023-10-03T10:12:45.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.936+0000 7f6507f05700 1 -- start start 2023-10-03T10:12:45.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.936+0000 7f6507f05700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f650009d6f0 0x7f65001218a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:45.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.936+0000 7f6507f05700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f650009e1d0 0x7f6500121de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:45.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.936+0000 7f6507f05700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6500139170 0x7f6500122320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:45.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.936+0000 7f6507f05700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6500124f00 con 0x7f650009e1d0 2023-10-03T10:12:45.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.936+0000 7f6507f05700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65001250e0 con 0x7f6500139170 2023-10-03T10:12:45.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.936+0000 7f6507f05700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f650009c710 con 0x7f650009d6f0 2023-10-03T10:12:45.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.936+0000 7f65054a0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f650009e1d0 0x7f6500121de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:45.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.936+0000 7f6505ca1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f650009d6f0 0x7f65001218a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:45.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.936+0000 7f65054a0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f650009e1d0 0x7f6500121de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55768/0 (socket says 172.21.15.3:55768) 2023-10-03T10:12:45.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.936+0000 7f65064a2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6500139170 0x7f6500122320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:45.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.936+0000 7f65054a0700 1 -- 172.21.15.3:0/43599625 learned_addr learned my addr 172.21.15.3:0/43599625 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:45.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.937+0000 7f65054a0700 1 -- 172.21.15.3:0/43599625 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f650009d6f0 msgr2=0x7f65001218a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:45.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.937+0000 7f65054a0700 1 --2- 172.21.15.3:0/43599625 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f650009d6f0 0x7f65001218a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:45.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.937+0000 7f65054a0700 1 -- 172.21.15.3:0/43599625 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6500139170 msgr2=0x7f6500122320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:45.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.937+0000 7f65054a0700 1 --2- 172.21.15.3:0/43599625 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6500139170 0x7f6500122320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:45.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.937+0000 7f65054a0700 1 -- 172.21.15.3:0/43599625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f00094a0 con 0x7f650009e1d0 2023-10-03T10:12:45.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.937+0000 7f65064a2700 1 --2- 172.21.15.3:0/43599625 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6500139170 0x7f6500122320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:45.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.937+0000 7f65054a0700 1 --2- 172.21.15.3:0/43599625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f650009e1d0 0x7f6500121de0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f64f000ec10 tx=0x7f64f000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:45.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.937+0000 7f64f67fc700 1 -- 172.21.15.3:0/43599625 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f000c850 con 0x7f650009e1d0 2023-10-03T10:12:45.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.937+0000 7f64f67fc700 1 -- 172.21.15.3:0/43599625 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f000c9b0 con 0x7f650009e1d0 2023-10-03T10:12:45.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.937+0000 7f64f67fc700 1 -- 172.21.15.3:0/43599625 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f000cd20 con 0x7f650009e1d0 2023-10-03T10:12:45.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.937+0000 7f6507f05700 1 -- 172.21.15.3:0/43599625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f8009550 con 0x7f650009e1d0 2023-10-03T10:12:45.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.937+0000 7f6507f05700 1 -- 172.21.15.3:0/43599625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6500072150 con 0x7f650009e1d0 2023-10-03T10:12:45.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.938+0000 7f64f67fc700 1 -- 172.21.15.3:0/43599625 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f64f0013470 con 0x7f650009e1d0 2023-10-03T10:12:45.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.938+0000 7f64f67fc700 1 -- 172.21.15.3:0/43599625 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f64f0040030 con 0x7f650009e1d0 2023-10-03T10:12:45.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.938+0000 7f6505ca1700 1 -- 172.21.15.3:0/43599625 >> v1:172.21.15.3:6816/127025 conn(0x7f64e402f510 legacy=0x7f64e40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:45.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.938+0000 7f6507f05700 1 -- 172.21.15.3:0/43599625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6500125c10 con 0x7f650009e1d0 2023-10-03T10:12:45.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:45.944+0000 7f64f67fc700 1 -- 172.21.15.3:0/43599625 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f64f0021020 con 0x7f650009e1d0 2023-10-03T10:12:46.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.139+0000 7f6505ca1700 1 -- 172.21.15.3:0/43599625 >> v1:172.21.15.3:6816/127025 conn(0x7f64e402f510 legacy=0x7f64e40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:46.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.150+0000 7f6507f05700 1 -- 172.21.15.3:0/43599625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6500125220 con 0x7f650009e1d0 2023-10-03T10:12:46.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.151+0000 7f64f67fc700 1 -- 172.21.15.3:0/43599625 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+13022 (secure 0 0 0) 0x7f6500125220 con 0x7f650009e1d0 2023-10-03T10:12:46.151 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:12:46.151 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":39,"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","created":"2023-10-03T10:09:28.703441+0000","modified":"2023-10-03T10:12:45.322787+0000","last_up_change":"2023-10-03T10:12:32.260880+0000","last_in_change":"2023-10-03T10:09:30.156546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-03T10:09:36.623971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-03T10:09:38.210614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-03T10:09:42.695873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-03T10:09:43.115588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cdcdd930-3c5b-4486-a021-7132f90fb6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":29,"up_thru":32,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6804","nonce":129556}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6805","nonce":129556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6807","nonce":129556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6806","nonce":129556}]},"public_addr":"172.21.15.3:6804/129556","cluster_addr":"172.21.15.3:6805/129556","heartbeat_back_addr":"172.21.15.3:6807/129556","heartbeat_front_addr":"172.21.15.3:6806/129556","state":["exists","up"]},{"osd":1,"uuid":"c8e58877-6762-4741-a8c1-567ebc97b708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":31,"up_thru":32,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6808","nonce":129654}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6809","nonce":129654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6811","nonce":129654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6810","nonce":129654}]},"public_addr":"172.21.15.3:6808/129654","cluster_addr":"172.21.15.3:6809/129654","heartbeat_back_addr":"172.21.15.3:6811/129654","heartbeat_front_addr":"172.21.15.3:6810/129654","state":["exists","up"]},{"osd":2,"uuid":"68a2a21a-7a8e-4f82-8433-5201faa50777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":25,"up_from":31,"up_thru":32,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6800","nonce":129764}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6801","nonce":129764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6803","nonce":129764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6802","nonce":129764}]},"public_addr":"172.21.15.3:6800/129764","cluster_addr":"172.21.15.3:6801/129764","heartbeat_back_addr":"172.21.15.3:6803/129764","heartbeat_front_addr":"172.21.15.3:6802/129764","state":["exists","up"]},{"osd":3,"uuid":"c4df2903-4e1e-4208-bf77-95b9fa749241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":26,"up_from":32,"up_thru":32,"down_at":27,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6812","nonce":129912}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6813","nonce":129912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6815","nonce":129912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6814","nonce":129912}]},"public_addr":"172.21.15.3:6812/129912","cluster_addr":"172.21.15.3:6813/129912","heartbeat_back_addr":"172.21.15.3:6815/129912","heartbeat_front_addr":"172.21.15.3:6814/129912","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.369708+0000","dead_epoch":24},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.257825+0000","dead_epoch":25},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.337931+0000","dead_epoch":26},{"osd":3,"down_stamp":"2023-10-03T10:12:26.212430+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.085222+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1","mappings":[{"from":3,"to":0}]},{"pgid":"4.8","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.19","mappings":[{"from":3,"to":0}]},{"pgid":"4.2a","mappings":[{"from":1,"to":2}]},{"pgid":"4.2d","mappings":[{"from":3,"to":0}]},{"pgid":"4.33","mappings":[{"from":3,"to":0}]},{"pgid":"4.39","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/3919724564":"2023-10-04T10:12:13.305411+0000","172.21.15.3:6817/2582564779":"2023-10-04T10:12:38.734689+0000","172.21.15.3:0/2091814084":"2023-10-04T10:12:13.305411+0000","172.21.15.3:6816/118296":"2023-10-04T10:12:13.305411+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-03T10:12:46.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.153+0000 7f6507f05700 1 -- 172.21.15.3:0/43599625 >> v1:172.21.15.3:6816/127025 conn(0x7f64e402f510 legacy=0x7f64e40319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:46.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.153+0000 7f6507f05700 1 -- 172.21.15.3:0/43599625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f650009e1d0 msgr2=0x7f6500121de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.153+0000 7f6507f05700 1 --2- 172.21.15.3:0/43599625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f650009e1d0 0x7f6500121de0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f64f000ec10 tx=0x7f64f000ef20 comp rx=0 tx=0).stop 2023-10-03T10:12:46.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.153+0000 7f6507f05700 1 -- 172.21.15.3:0/43599625 shutdown_connections 2023-10-03T10:12:46.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.153+0000 7f6507f05700 1 --2- 172.21.15.3:0/43599625 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6500139170 0x7f6500122320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.153+0000 7f6507f05700 1 --2- 172.21.15.3:0/43599625 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f650009d6f0 0x7f65001218a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.153+0000 7f6507f05700 1 --2- 172.21.15.3:0/43599625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f650009e1d0 0x7f6500121de0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.153+0000 7f6507f05700 1 -- 172.21.15.3:0/43599625 >> 172.21.15.3:0/43599625 conn(0x7f6500066f50 msgr2=0x7f6500070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:46.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.153+0000 7f6507f05700 1 -- 172.21.15.3:0/43599625 shutdown_connections 2023-10-03T10:12:46.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.153+0000 7f6507f05700 1 -- 172.21.15.3:0/43599625 wait complete. 2023-10-03T10:12:46.167 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-03T10:12:46.168 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-03T10:12:46.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.303+0000 7f848fa92700 1 Processor -- start 2023-10-03T10:12:46.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.304+0000 7f848fa92700 1 -- start start 2023-10-03T10:12:46.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.304+0000 7f848fa92700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f848809e930 0x7f848809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.304+0000 7f848fa92700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f848809f240 0x7f84881317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.304+0000 7f848fa92700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8488070940 con 0x7f848809f650 2023-10-03T10:12:46.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.304+0000 7f848fa92700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f848806fd60 con 0x7f848809f240 2023-10-03T10:12:46.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.304+0000 7f848fa92700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f848806fa80 con 0x7f848809e930 2023-10-03T10:12:46.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.304+0000 7f848d82e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f848809e930 0x7f848809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.304+0000 7f848d02d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f848809f240 0x7f84881317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.304+0000 7f848e02f700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f848809f650 0x7f8488131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50606/0 (socket says 172.21.15.3:50606) 2023-10-03T10:12:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.304+0000 7f848e02f700 1 -- 172.21.15.3:0/3324618170 learned_addr learned my addr 172.21.15.3:0/3324618170 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.304+0000 7f848c82c700 1 -- 172.21.15.3:0/3324618170 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1753143378 0 0) 0x7f8488070940 con 0x7f848809f650 2023-10-03T10:12:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.305+0000 7f848c82c700 1 -- 172.21.15.3:0/3324618170 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84740035b0 con 0x7f848809f650 2023-10-03T10:12:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.305+0000 7f848d02d700 1 -- 172.21.15.3:0/3324618170 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f848809e930 msgr2=0x7f848809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.305+0000 7f848d02d700 1 --2- 172.21.15.3:0/3324618170 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f848809e930 0x7f848809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.305+0000 7f848d02d700 1 -- 172.21.15.3:0/3324618170 >> v1:172.21.15.3:6789/0 conn(0x7f848809f650 legacy=0x7f8488131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.305+0000 7f848d02d700 1 -- 172.21.15.3:0/3324618170 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848809fb60 con 0x7f848809f240 2023-10-03T10:12:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.305+0000 7f848d82e700 1 --2- 172.21.15.3:0/3324618170 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f848809e930 0x7f848809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.305+0000 7f848d02d700 1 --2- 172.21.15.3:0/3324618170 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f848809f240 0x7f84881317f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8478009870 tx=0x7f8478009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=732b2be361f1d8a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.305+0000 7f848c82c700 1 -- 172.21.15.3:0/3324618170 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478011830 con 0x7f848809f240 2023-10-03T10:12:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.305+0000 7f848c82c700 1 -- 172.21.15.3:0/3324618170 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8478011990 con 0x7f848809f240 2023-10-03T10:12:46.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.305+0000 7f848c82c700 1 -- 172.21.15.3:0/3324618170 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478011d20 con 0x7f848809f240 2023-10-03T10:12:46.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.306+0000 7f848fa92700 1 -- 172.21.15.3:0/3324618170 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f848809f240 msgr2=0x7f84881317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.306+0000 7f848fa92700 1 --2- 172.21.15.3:0/3324618170 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f848809f240 0x7f84881317f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8478009870 tx=0x7f8478009b80 comp rx=0 tx=0).stop 2023-10-03T10:12:46.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.306+0000 7f848fa92700 1 -- 172.21.15.3:0/3324618170 shutdown_connections 2023-10-03T10:12:46.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.306+0000 7f848fa92700 1 --2- 172.21.15.3:0/3324618170 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f848809e930 0x7f848809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.306+0000 7f848fa92700 1 --2- 172.21.15.3:0/3324618170 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f848809f240 0x7f84881317f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.306+0000 7f848fa92700 1 -- 172.21.15.3:0/3324618170 >> 172.21.15.3:0/3324618170 conn(0x7f84880a4210 msgr2=0x7f84880a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:46.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.306+0000 7f848fa92700 1 -- 172.21.15.3:0/3324618170 shutdown_connections 2023-10-03T10:12:46.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.306+0000 7f848fa92700 1 -- 172.21.15.3:0/3324618170 wait complete. 2023-10-03T10:12:46.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.307+0000 7f848fa92700 1 Processor -- start 2023-10-03T10:12:46.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.307+0000 7f848fa92700 1 -- start start 2023-10-03T10:12:46.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.307+0000 7f848fa92700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f848809e930 0x7f848806b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.307+0000 7f848fa92700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f848809f240 0x7f848806bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.307+0000 7f848fa92700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f848809f650 0x7f848806c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.307+0000 7f848fa92700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84881b7230 con 0x7f848809e930 2023-10-03T10:12:46.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.307+0000 7f848fa92700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84881a93e0 con 0x7f848809f650 2023-10-03T10:12:46.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.307+0000 7f848fa92700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f84881a8900 con 0x7f848809f240 2023-10-03T10:12:46.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.307+0000 7f848d82e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f848809e930 0x7f848806b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.307+0000 7f848d82e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f848809e930 0x7f848806b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55774/0 (socket says 172.21.15.3:55774) 2023-10-03T10:12:46.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.307+0000 7f848e02f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f848809f650 0x7f848806c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.307+0000 7f848d82e700 1 -- 172.21.15.3:0/540083098 learned_addr learned my addr 172.21.15.3:0/540083098 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:46.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.307+0000 7f848d02d700 1 --2- 172.21.15.3:0/540083098 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f848809f240 0x7f848806bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.308+0000 7f848d02d700 1 -- 172.21.15.3:0/540083098 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f848809f650 msgr2=0x7f848806c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.308+0000 7f848d02d700 1 --2- 172.21.15.3:0/540083098 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f848809f650 0x7f848806c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.308+0000 7f848d02d700 1 -- 172.21.15.3:0/540083098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f848809e930 msgr2=0x7f848806b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.308+0000 7f848d02d700 1 --2- 172.21.15.3:0/540083098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f848809e930 0x7f848806b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.308+0000 7f848d02d700 1 -- 172.21.15.3:0/540083098 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84840094a0 con 0x7f848809f240 2023-10-03T10:12:46.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.308+0000 7f848e02f700 1 --2- 172.21.15.3:0/540083098 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f848809f650 0x7f848806c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:46.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.308+0000 7f848d02d700 1 --2- 172.21.15.3:0/540083098 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f848809f240 0x7f848806bdf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8488072a80 tx=0x7f8478004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:46.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.308+0000 7f847e7fc700 1 -- 172.21.15.3:0/540083098 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478004450 con 0x7f848809f240 2023-10-03T10:12:46.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.308+0000 7f847e7fc700 1 -- 172.21.15.3:0/540083098 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847801f070 con 0x7f848809f240 2023-10-03T10:12:46.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.308+0000 7f848fa92700 1 -- 172.21.15.3:0/540083098 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84780094a0 con 0x7f848809f240 2023-10-03T10:12:46.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.308+0000 7f847e7fc700 1 -- 172.21.15.3:0/540083098 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84780038e0 con 0x7f848809f240 2023-10-03T10:12:46.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.308+0000 7f848fa92700 1 -- 172.21.15.3:0/540083098 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84880a0190 con 0x7f848809f240 2023-10-03T10:12:46.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.309+0000 7f847e7fc700 1 -- 172.21.15.3:0/540083098 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f8478003a40 con 0x7f848809f240 2023-10-03T10:12:46.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.309+0000 7f847e7fc700 1 -- 172.21.15.3:0/540083098 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f8478021070 con 0x7f848809f240 2023-10-03T10:12:46.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.309+0000 7f848d82e700 1 -- 172.21.15.3:0/540083098 >> v1:172.21.15.3:6816/127025 conn(0x7f846c02f510 legacy=0x7f846c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:46.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.309+0000 7f848fa92700 1 -- 172.21.15.3:0/540083098 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84881a0b30 con 0x7f848809f240 2023-10-03T10:12:46.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.314+0000 7f847e7fc700 1 -- 172.21.15.3:0/540083098 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f847801a7b0 con 0x7f848809f240 2023-10-03T10:12:46.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.510+0000 7f848d82e700 1 -- 172.21.15.3:0/540083098 >> v1:172.21.15.3:6816/127025 conn(0x7f846c02f510 legacy=0x7f846c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:46.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.524+0000 7f848fa92700 1 -- 172.21.15.3:0/540083098 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84881a9110 con 0x7f848809f240 2023-10-03T10:12:46.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.526+0000 7f847e7fc700 1 -- 172.21.15.3:0/540083098 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+13022 (secure 0 0 0) 0x7f8478024b70 con 0x7f848809f240 2023-10-03T10:12:46.526 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:12:46.526 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":39,"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","created":"2023-10-03T10:09:28.703441+0000","modified":"2023-10-03T10:12:45.322787+0000","last_up_change":"2023-10-03T10:12:32.260880+0000","last_in_change":"2023-10-03T10:09:30.156546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-03T10:09:36.623971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-03T10:09:38.210614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-03T10:09:42.695873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-03T10:09:43.115588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cdcdd930-3c5b-4486-a021-7132f90fb6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":29,"up_thru":32,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6804","nonce":129556}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6805","nonce":129556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6807","nonce":129556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6806","nonce":129556}]},"public_addr":"172.21.15.3:6804/129556","cluster_addr":"172.21.15.3:6805/129556","heartbeat_back_addr":"172.21.15.3:6807/129556","heartbeat_front_addr":"172.21.15.3:6806/129556","state":["exists","up"]},{"osd":1,"uuid":"c8e58877-6762-4741-a8c1-567ebc97b708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":31,"up_thru":32,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6808","nonce":129654}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6809","nonce":129654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6811","nonce":129654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6810","nonce":129654}]},"public_addr":"172.21.15.3:6808/129654","cluster_addr":"172.21.15.3:6809/129654","heartbeat_back_addr":"172.21.15.3:6811/129654","heartbeat_front_addr":"172.21.15.3:6810/129654","state":["exists","up"]},{"osd":2,"uuid":"68a2a21a-7a8e-4f82-8433-5201faa50777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":25,"up_from":31,"up_thru":32,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6800","nonce":129764}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6801","nonce":129764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6803","nonce":129764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6802","nonce":129764}]},"public_addr":"172.21.15.3:6800/129764","cluster_addr":"172.21.15.3:6801/129764","heartbeat_back_addr":"172.21.15.3:6803/129764","heartbeat_front_addr":"172.21.15.3:6802/129764","state":["exists","up"]},{"osd":3,"uuid":"c4df2903-4e1e-4208-bf77-95b9fa749241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":26,"up_from":32,"up_thru":32,"down_at":27,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6812","nonce":129912}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6813","nonce":129912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6815","nonce":129912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6814","nonce":129912}]},"public_addr":"172.21.15.3:6812/129912","cluster_addr":"172.21.15.3:6813/129912","heartbeat_back_addr":"172.21.15.3:6815/129912","heartbeat_front_addr":"172.21.15.3:6814/129912","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.369708+0000","dead_epoch":24},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.257825+0000","dead_epoch":25},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.337931+0000","dead_epoch":26},{"osd":3,"down_stamp":"2023-10-03T10:12:26.212430+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.085222+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1","mappings":[{"from":3,"to":0}]},{"pgid":"4.8","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.19","mappings":[{"from":3,"to":0}]},{"pgid":"4.2a","mappings":[{"from":1,"to":2}]},{"pgid":"4.2d","mappings":[{"from":3,"to":0}]},{"pgid":"4.33","mappings":[{"from":3,"to":0}]},{"pgid":"4.39","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/3919724564":"2023-10-04T10:12:13.305411+0000","172.21.15.3:6817/2582564779":"2023-10-04T10:12:38.734689+0000","172.21.15.3:0/2091814084":"2023-10-04T10:12:13.305411+0000","172.21.15.3:6816/118296":"2023-10-04T10:12:13.305411+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-03T10:12:46.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.528+0000 7f848fa92700 1 -- 172.21.15.3:0/540083098 >> v1:172.21.15.3:6816/127025 conn(0x7f846c02f510 legacy=0x7f846c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:46.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.528+0000 7f848fa92700 1 -- 172.21.15.3:0/540083098 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f848809f240 msgr2=0x7f848806bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.528+0000 7f848fa92700 1 --2- 172.21.15.3:0/540083098 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f848809f240 0x7f848806bdf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8488072a80 tx=0x7f8478004750 comp rx=0 tx=0).stop 2023-10-03T10:12:46.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.528+0000 7f848fa92700 1 -- 172.21.15.3:0/540083098 shutdown_connections 2023-10-03T10:12:46.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.528+0000 7f848fa92700 1 --2- 172.21.15.3:0/540083098 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f848809f650 0x7f848806c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.528+0000 7f848fa92700 1 --2- 172.21.15.3:0/540083098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f848809e930 0x7f848806b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.528+0000 7f848fa92700 1 --2- 172.21.15.3:0/540083098 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f848809f240 0x7f848806bdf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.528+0000 7f848fa92700 1 -- 172.21.15.3:0/540083098 >> 172.21.15.3:0/540083098 conn(0x7f84880a4210 msgr2=0x7f84881b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:46.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.528+0000 7f848fa92700 1 -- 172.21.15.3:0/540083098 shutdown_connections 2023-10-03T10:12:46.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.528+0000 7f848fa92700 1 -- 172.21.15.3:0/540083098 wait complete. 2023-10-03T10:12:46.543 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-03T10:12:46.543 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-03T10:12:46.544 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-03T10:12:46.544 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-03T10:12:46.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.690+0000 7f7083e68700 1 Processor -- start 2023-10-03T10:12:46.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.690+0000 7f7083e68700 1 -- start start 2023-10-03T10:12:46.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.691+0000 7f7083e68700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f707c09f410 0x7f707c136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.691+0000 7f7083e68700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f707c136fe0 0x7f707c1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.691+0000 7f7083e68700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f707c070940 con 0x7f707c09e930 2023-10-03T10:12:46.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.691+0000 7f7083e68700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f707c06fd60 con 0x7f707c09f410 2023-10-03T10:12:46.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.691+0000 7f7083e68700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f707c06fa80 con 0x7f707c136fe0 2023-10-03T10:12:46.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.691+0000 7f7082405700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f707c136fe0 0x7f707c1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.691+0000 7f7082405700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f707c136fe0 0x7f707c1393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59864/0 (socket says 172.21.15.3:59864) 2023-10-03T10:12:46.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.691+0000 7f7082405700 1 -- 172.21.15.3:0/12714746 learned_addr learned my addr 172.21.15.3:0/12714746 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:46.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.692+0000 7f7081403700 1 --2- 172.21.15.3:0/12714746 >> v2:172.21.15.3:3300/0 conn(0x7f707c09f410 0x7f707c136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.692+0000 7f7082405700 1 -- 172.21.15.3:0/12714746 >> v2:172.21.15.3:3300/0 conn(0x7f707c09f410 msgr2=0x7f707c136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:46.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.692+0000 7f7082405700 1 --2- 172.21.15.3:0/12714746 >> v2:172.21.15.3:3300/0 conn(0x7f707c09f410 0x7f707c136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.692+0000 7f7082405700 1 -- 172.21.15.3:0/12714746 >> v1:172.21.15.3:6789/0 conn(0x7f707c09e930 legacy=0x7f707c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.692+0000 7f7080c02700 1 -- 172.21.15.3:0/12714746 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3954680295 0 0) 0x7f707c070940 con 0x7f707c09e930 2023-10-03T10:12:46.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.692+0000 7f7082405700 1 -- 172.21.15.3:0/12714746 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c09fb60 con 0x7f707c136fe0 2023-10-03T10:12:46.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.692+0000 7f7081403700 1 --2- 172.21.15.3:0/12714746 >> v2:172.21.15.3:3300/0 conn(0x7f707c09f410 0x7f707c136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:12:46.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.692+0000 7f7082405700 1 --2- 172.21.15.3:0/12714746 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f707c136fe0 0x7f707c1393c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7074009e50 tx=0x7f707400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48f0e4d75abd7c90 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:46.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.693+0000 7f7080c02700 1 -- 172.21.15.3:0/12714746 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707400e070 con 0x7f707c136fe0 2023-10-03T10:12:46.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.693+0000 7f7080c02700 1 -- 172.21.15.3:0/12714746 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707400b9c0 con 0x7f707c136fe0 2023-10-03T10:12:46.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.693+0000 7f7080c02700 1 -- 172.21.15.3:0/12714746 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707400bd70 con 0x7f707c136fe0 2023-10-03T10:12:46.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.693+0000 7fbb7b5e2700 1 Processor -- start 2023-10-03T10:12:46.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.693+0000 7fbb7b5e2700 1 -- start start 2023-10-03T10:12:46.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.694+0000 7f7083e68700 1 -- 172.21.15.3:0/12714746 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f707c136fe0 msgr2=0x7f707c1393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.694+0000 7f7083e68700 1 --2- 172.21.15.3:0/12714746 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f707c136fe0 0x7f707c1393c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7074009e50 tx=0x7f707400b330 comp rx=0 tx=0).stop 2023-10-03T10:12:46.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.694+0000 7f7083e68700 1 -- 172.21.15.3:0/12714746 shutdown_connections 2023-10-03T10:12:46.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.694+0000 7f7083e68700 1 --2- 172.21.15.3:0/12714746 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f707c136fe0 0x7f707c1393c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.694+0000 7f7083e68700 1 --2- 172.21.15.3:0/12714746 >> v2:172.21.15.3:3300/0 conn(0x7f707c09f410 0x7f707c136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.694+0000 7f7083e68700 1 -- 172.21.15.3:0/12714746 >> 172.21.15.3:0/12714746 conn(0x7f707c0a4210 msgr2=0x7f707c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:46.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.694+0000 7f7083e68700 1 -- 172.21.15.3:0/12714746 shutdown_connections 2023-10-03T10:12:46.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.694+0000 7fbb7b5e2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb7409e1d0 0x7fbb74135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.694+0000 7fbb7b5e2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb74135e50 0x7fbb74138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.694+0000 7fbb7b5e2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb7409f130 con 0x7fbb7409d6f0 2023-10-03T10:12:46.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.694+0000 7fbb7b5e2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7409f520 con 0x7fbb7409e1d0 2023-10-03T10:12:46.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.694+0000 7fbb7b5e2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7409e930 con 0x7fbb74135e50 2023-10-03T10:12:46.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.694+0000 7f7083e68700 1 -- 172.21.15.3:0/12714746 wait complete. 2023-10-03T10:12:46.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.694+0000 7fbb79b7f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb74135e50 0x7fbb74138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.694+0000 7fbb79b7f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb74135e50 0x7fbb74138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38474/0 (socket says 172.21.15.3:38474) 2023-10-03T10:12:46.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.694+0000 7fbb79b7f700 1 -- 172.21.15.3:0/3402753574 learned_addr learned my addr 172.21.15.3:0/3402753574 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:46.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.694+0000 7fbb78b7d700 1 --2- 172.21.15.3:0/3402753574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb7409e1d0 0x7fbb74135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.695+0000 7fbb79b7f700 1 -- 172.21.15.3:0/3402753574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb7409e1d0 msgr2=0x7fbb74135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:46.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.695+0000 7fbb79b7f700 1 --2- 172.21.15.3:0/3402753574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb7409e1d0 0x7fbb74135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.695+0000 7fbb79b7f700 1 -- 172.21.15.3:0/3402753574 >> v1:172.21.15.3:6789/0 conn(0x7fbb7409d6f0 legacy=0x7fbb7409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.695+0000 7fbb79b7f700 1 -- 172.21.15.3:0/3402753574 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7409b2d0 con 0x7fbb74135e50 2023-10-03T10:12:46.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.695+0000 7fbb78b7d700 1 --2- 172.21.15.3:0/3402753574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb7409e1d0 0x7fbb74135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:46.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.695+0000 7fbb79b7f700 1 --2- 172.21.15.3:0/3402753574 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb74135e50 0x7fbb74138230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009e80 tx=0x7fbb6c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1dfa7ee9d98fe35 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:46.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.695+0000 7fbb6bfff700 1 -- 172.21.15.3:0/3402753574 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c00e070 con 0x7fbb74135e50 2023-10-03T10:12:46.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.695+0000 7fbb6bfff700 1 -- 172.21.15.3:0/3402753574 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6c00b910 con 0x7fbb74135e50 2023-10-03T10:12:46.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.695+0000 7fbb6bfff700 1 -- 172.21.15.3:0/3402753574 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c00bcc0 con 0x7fbb74135e50 2023-10-03T10:12:46.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.695+0000 7f7083e68700 1 Processor -- start 2023-10-03T10:12:46.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.695+0000 7f7083e68700 1 -- start start 2023-10-03T10:12:46.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.696+0000 7f7083e68700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f707c09e930 0x7f707c11e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.696+0000 7f7083e68700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f707c09f410 0x7f707c11eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.696+0000 7f7083e68700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f707c136fe0 0x7f707c11f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.696+0000 7f7083e68700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c1201e0 con 0x7f707c136fe0 2023-10-03T10:12:46.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.696+0000 7f7083e68700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f707c120800 con 0x7f707c09f410 2023-10-03T10:12:46.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.696+0000 7f7083e68700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f707c071d90 con 0x7f707c09e930 2023-10-03T10:12:46.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.696+0000 7f7081403700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f707c09f410 0x7f707c11eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.696+0000 7f7081403700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f707c09f410 0x7f707c11eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59884/0 (socket says 172.21.15.3:59884) 2023-10-03T10:12:46.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.696+0000 7f7081403700 1 -- 172.21.15.3:0/2511446692 learned_addr learned my addr 172.21.15.3:0/2511446692 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:46.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.696+0000 7f7082405700 1 --2- 172.21.15.3:0/2511446692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f707c136fe0 0x7f707c11f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.696+0000 7fbb7b5e2700 1 -- 172.21.15.3:0/3402753574 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb74135e50 msgr2=0x7fbb74138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.696+0000 7fbb7b5e2700 1 --2- 172.21.15.3:0/3402753574 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb74135e50 0x7fbb74138230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009e80 tx=0x7fbb6c00b330 comp rx=0 tx=0).stop 2023-10-03T10:12:46.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.696+0000 7fbb7b5e2700 1 -- 172.21.15.3:0/3402753574 shutdown_connections 2023-10-03T10:12:46.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.696+0000 7fbb7b5e2700 1 --2- 172.21.15.3:0/3402753574 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb74135e50 0x7fbb74138230 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.696+0000 7fbb7b5e2700 1 --2- 172.21.15.3:0/3402753574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb7409e1d0 0x7fbb74135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.696+0000 7fbb7b5e2700 1 -- 172.21.15.3:0/3402753574 >> 172.21.15.3:0/3402753574 conn(0x7fbb74066f50 msgr2=0x7fbb7409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:46.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.697+0000 7fbb7b5e2700 1 -- 172.21.15.3:0/3402753574 shutdown_connections 2023-10-03T10:12:46.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.697+0000 7fbb7b5e2700 1 -- 172.21.15.3:0/3402753574 wait complete. 2023-10-03T10:12:46.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.697+0000 7f7081403700 1 -- 172.21.15.3:0/2511446692 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f707c09e930 msgr2=0x7f707c11e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.697+0000 7f7081403700 1 --2- 172.21.15.3:0/2511446692 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f707c09e930 0x7f707c11e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.697+0000 7f7081403700 1 -- 172.21.15.3:0/2511446692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f707c136fe0 msgr2=0x7f707c11f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.697+0000 7f7081403700 1 --2- 172.21.15.3:0/2511446692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f707c136fe0 0x7f707c11f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.697+0000 7f7081403700 1 -- 172.21.15.3:0/2511446692 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7074009b30 con 0x7f707c09f410 2023-10-03T10:12:46.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.697+0000 7f7081403700 1 --2- 172.21.15.3:0/2511446692 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f707c09f410 0x7f707c11eb10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f706c009f60 tx=0x7f706c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:46.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.697+0000 7f70727fc700 1 -- 172.21.15.3:0/2511446692 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c004230 con 0x7f707c09f410 2023-10-03T10:12:46.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.697+0000 7f70727fc700 1 -- 172.21.15.3:0/2511446692 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706c004390 con 0x7f707c09f410 2023-10-03T10:12:46.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.697+0000 7f70727fc700 1 -- 172.21.15.3:0/2511446692 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c013920 con 0x7f707c09f410 2023-10-03T10:12:46.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.698+0000 7f7083e68700 1 -- 172.21.15.3:0/2511446692 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f707c121e10 con 0x7f707c09f410 2023-10-03T10:12:46.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.698+0000 7f7083e68700 1 -- 172.21.15.3:0/2511446692 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f707c0a31d0 con 0x7f707c09f410 2023-10-03T10:12:46.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.698+0000 7fbb7b5e2700 1 Processor -- start 2023-10-03T10:12:46.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.698+0000 7fbb7b5e2700 1 -- start start 2023-10-03T10:12:46.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.699+0000 7fbb7b5e2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb7409d6f0 0x7fbb741c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.699+0000 7fbb7b5e2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb7409e1d0 0x7fbb741c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.699+0000 7fbb7b5e2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb74135e50 0x7fbb741c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.699+0000 7fbb7b5e2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7411d080 con 0x7fbb7409e1d0 2023-10-03T10:12:46.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.699+0000 7fbb7b5e2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb74122220 con 0x7fbb7409d6f0 2023-10-03T10:12:46.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.699+0000 7fbb7b5e2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7406b1e0 con 0x7fbb74135e50 2023-10-03T10:12:46.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.699+0000 7f7083e68700 1 -- 172.21.15.3:0/2511446692 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f707c120a90 con 0x7f707c09f410 2023-10-03T10:12:46.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.699+0000 7fbb78b7d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb7409e1d0 0x7fbb741c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.699+0000 7fbb78b7d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb7409e1d0 0x7fbb741c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55792/0 (socket says 172.21.15.3:55792) 2023-10-03T10:12:46.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.699+0000 7fbb78b7d700 1 -- 172.21.15.3:0/641004557 learned_addr learned my addr 172.21.15.3:0/641004557 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:46.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.699+0000 7fbb7937e700 1 --2- 172.21.15.3:0/641004557 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb7409d6f0 0x7fbb741c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.699+0000 7fbb79b7f700 1 --2- 172.21.15.3:0/641004557 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb74135e50 0x7fbb741c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.699+0000 7fbb78b7d700 1 -- 172.21.15.3:0/641004557 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb74135e50 msgr2=0x7fbb741c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.699+0000 7fbb78b7d700 1 --2- 172.21.15.3:0/641004557 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb74135e50 0x7fbb741c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.699+0000 7fbb78b7d700 1 -- 172.21.15.3:0/641004557 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb7409d6f0 msgr2=0x7fbb741c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.699+0000 7fbb78b7d700 1 --2- 172.21.15.3:0/641004557 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb7409d6f0 0x7fbb741c83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.699+0000 7fbb78b7d700 1 -- 172.21.15.3:0/641004557 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb640094a0 con 0x7fbb7409e1d0 2023-10-03T10:12:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.699+0000 7fbb79b7f700 1 --2- 172.21.15.3:0/641004557 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb74135e50 0x7fbb741c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:12:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.700+0000 7fbb78b7d700 1 --2- 172.21.15.3:0/641004557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb7409e1d0 0x7fbb741c8930 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400ec10 tx=0x7fbb6400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.700+0000 7fbb69ffb700 1 -- 172.21.15.3:0/641004557 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6400c910 con 0x7fbb7409e1d0 2023-10-03T10:12:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.700+0000 7fbb7b5e2700 1 -- 172.21.15.3:0/641004557 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb6c009b30 con 0x7fbb7409e1d0 2023-10-03T10:12:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.700+0000 7fbb7b5e2700 1 -- 172.21.15.3:0/641004557 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7409c700 con 0x7fbb7409e1d0 2023-10-03T10:12:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.700+0000 7fbb69ffb700 1 -- 172.21.15.3:0/641004557 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6400ca70 con 0x7fbb7409e1d0 2023-10-03T10:12:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.700+0000 7fbb69ffb700 1 -- 172.21.15.3:0/641004557 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6400cde0 con 0x7fbb7409e1d0 2023-10-03T10:12:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.701+0000 7fbb7b5e2700 1 -- 172.21.15.3:0/641004557 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbb7409c000 con 0x7fbb7409e1d0 2023-10-03T10:12:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.701+0000 7fbb69ffb700 1 -- 172.21.15.3:0/641004557 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fbb64013470 con 0x7fbb7409e1d0 2023-10-03T10:12:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.701+0000 7fbb69ffb700 1 -- 172.21.15.3:0/641004557 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7fbb64040220 con 0x7fbb7409e1d0 2023-10-03T10:12:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.701+0000 7fbb69ffb700 1 -- 172.21.15.3:0/641004557 --> v1:172.21.15.3:6812/129912 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb5803c490 con 0x7fbb58038790 2023-10-03T10:12:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.701+0000 7fbb69ffb700 1 -- 172.21.15.3:0/641004557 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb640427c0 con 0x7fbb7409e1d0 2023-10-03T10:12:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.701+0000 7fbb7937e700 1 -- 172.21.15.3:0/641004557 >> v1:172.21.15.3:6816/127025 conn(0x7fbb5802f540 legacy=0x7fbb580319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.702+0000 7f70727fc700 1 -- 172.21.15.3:0/2511446692 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f706c013a80 con 0x7f707c09f410 2023-10-03T10:12:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.702+0000 7f70727fc700 1 -- 172.21.15.3:0/2511446692 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f706c0437d0 con 0x7f707c09f410 2023-10-03T10:12:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.703+0000 7f70727fc700 1 -- 172.21.15.3:0/2511446692 --> v1:172.21.15.3:6804/129556 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f706003c410 con 0x7f7060038710 2023-10-03T10:12:46.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.703+0000 7fbb69ffb700 1 -- 172.21.15.3:0/641004557 <== osd.3 v1:172.21.15.3:6812/129912 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (unknown 0 0 75535769) 0x7fbb5803c490 con 0x7fbb58038790 2023-10-03T10:12:46.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.703+0000 7f70727fc700 1 -- 172.21.15.3:0/2511446692 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f706c0036a0 con 0x7f707c09f410 2023-10-03T10:12:46.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.704+0000 7f70727fc700 1 -- 172.21.15.3:0/2511446692 <== osd.0 v1:172.21.15.3:6804/129556 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (unknown 0 0 75535769) 0x7f706003c410 con 0x7f7060038710 2023-10-03T10:12:46.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.706+0000 7f7081c04700 1 -- 172.21.15.3:0/2511446692 >> v1:172.21.15.3:6816/127025 conn(0x7f706002f540 legacy=0x7f70600319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.710+0000 7fce40367700 1 Processor -- start 2023-10-03T10:12:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.710+0000 7fce40367700 1 -- start start 2023-10-03T10:12:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.711+0000 7fce40367700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fce3806e570 0x7fce3806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.711+0000 7fce40367700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce3806ee80 0x7fce38134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.711+0000 7fce40367700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce38070ee0 con 0x7fce3809eca0 2023-10-03T10:12:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.711+0000 7fce40367700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce38071480 con 0x7fce3806e570 2023-10-03T10:12:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.711+0000 7fce40367700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce38071b90 con 0x7fce3806ee80 2023-10-03T10:12:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.712+0000 7fce3e904700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce3806ee80 0x7fce38134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.712+0000 7fce3e904700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce3806ee80 0x7fce38134d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38494/0 (socket says 172.21.15.3:38494) 2023-10-03T10:12:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.712+0000 7fce3e904700 1 -- 172.21.15.3:0/4040414391 learned_addr learned my addr 172.21.15.3:0/4040414391 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:46.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.712+0000 7fce3e904700 1 -- 172.21.15.3:0/4040414391 >> v2:172.21.15.3:3300/0 conn(0x7fce3806e570 msgr2=0x7fce3806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:46.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.712+0000 7fce3e904700 1 --2- 172.21.15.3:0/4040414391 >> v2:172.21.15.3:3300/0 conn(0x7fce3806e570 0x7fce3806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.712+0000 7fce3e904700 1 -- 172.21.15.3:0/4040414391 >> v1:172.21.15.3:6789/0 conn(0x7fce3809eca0 legacy=0x7fce38134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.712+0000 7fce3e904700 1 -- 172.21.15.3:0/4040414391 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce380711a0 con 0x7fce3806ee80 2023-10-03T10:12:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.712+0000 7fce3e904700 1 --2- 172.21.15.3:0/4040414391 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce3806ee80 0x7fce38134d20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fce30009e50 tx=0x7fce3000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ce4e245b9f50297 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.712+0000 7fce3d101700 1 -- 172.21.15.3:0/4040414391 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3000e070 con 0x7fce3806ee80 2023-10-03T10:12:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.712+0000 7fce3d101700 1 -- 172.21.15.3:0/4040414391 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce30003a20 con 0x7fce3806ee80 2023-10-03T10:12:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.712+0000 7fce3d101700 1 -- 172.21.15.3:0/4040414391 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce30003dd0 con 0x7fce3806ee80 2023-10-03T10:12:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.713+0000 7fce40367700 1 -- 172.21.15.3:0/4040414391 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce3806ee80 msgr2=0x7fce38134d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.713+0000 7fce40367700 1 --2- 172.21.15.3:0/4040414391 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce3806ee80 0x7fce38134d20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fce30009e50 tx=0x7fce3000b330 comp rx=0 tx=0).stop 2023-10-03T10:12:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.713+0000 7fce40367700 1 -- 172.21.15.3:0/4040414391 shutdown_connections 2023-10-03T10:12:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.713+0000 7fce40367700 1 --2- 172.21.15.3:0/4040414391 >> v2:172.21.15.3:3300/0 conn(0x7fce3806e570 0x7fce3806e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.713+0000 7fce40367700 1 --2- 172.21.15.3:0/4040414391 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce3806ee80 0x7fce38134d20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.713+0000 7fce40367700 1 -- 172.21.15.3:0/4040414391 >> 172.21.15.3:0/4040414391 conn(0x7fce380a2aa0 msgr2=0x7fce380a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:46.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.713+0000 7fce40367700 1 -- 172.21.15.3:0/4040414391 shutdown_connections 2023-10-03T10:12:46.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.713+0000 7fce40367700 1 -- 172.21.15.3:0/4040414391 wait complete. 2023-10-03T10:12:46.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.714+0000 7fce40367700 1 Processor -- start 2023-10-03T10:12:46.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.714+0000 7fce40367700 1 -- start start 2023-10-03T10:12:46.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.715+0000 7fce40367700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce3806e570 0x7fce38134980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.715+0000 7fce40367700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce3806ee80 0x7fce381389f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.715+0000 7fce40367700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce38138f30 0x7fce381393d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.715+0000 7fce40367700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce381405f0 con 0x7fce3806ee80 2023-10-03T10:12:46.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.715+0000 7fce40367700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce38135950 con 0x7fce38138f30 2023-10-03T10:12:46.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.715+0000 7fce40367700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce3813b600 con 0x7fce3806e570 2023-10-03T10:12:46.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.715+0000 7fce3e904700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce38138f30 0x7fce381393d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.715+0000 7fce3e904700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce38138f30 0x7fce381393d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59912/0 (socket says 172.21.15.3:59912) 2023-10-03T10:12:46.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.715+0000 7fce3e904700 1 -- 172.21.15.3:0/1386716294 learned_addr learned my addr 172.21.15.3:0/1386716294 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:46.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.715+0000 7fce3d902700 1 --2- 172.21.15.3:0/1386716294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce3806ee80 0x7fce381389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.715+0000 7f9d42eca700 1 Processor -- start 2023-10-03T10:12:46.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.715+0000 7fce3e904700 1 -- 172.21.15.3:0/1386716294 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce3806e570 msgr2=0x7fce38134980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.715+0000 7fce3e904700 1 --2- 172.21.15.3:0/1386716294 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce3806e570 0x7fce38134980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.715+0000 7fce3e904700 1 -- 172.21.15.3:0/1386716294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce3806ee80 msgr2=0x7fce381389f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.715+0000 7fce3e904700 1 --2- 172.21.15.3:0/1386716294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce3806ee80 0x7fce381389f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.715+0000 7fce3e904700 1 -- 172.21.15.3:0/1386716294 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce30009b30 con 0x7fce38138f30 2023-10-03T10:12:46.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.716+0000 7f9d42eca700 1 -- start start 2023-10-03T10:12:46.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.716+0000 7fce3e904700 1 --2- 172.21.15.3:0/1386716294 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce38138f30 0x7fce381393d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fce38071cd0 tx=0x7fce30004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:46.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.716+0000 7fce2effd700 1 -- 172.21.15.3:0/1386716294 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce30004440 con 0x7fce38138f30 2023-10-03T10:12:46.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.716+0000 7fce2effd700 1 -- 172.21.15.3:0/1386716294 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce300045c0 con 0x7fce38138f30 2023-10-03T10:12:46.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.716+0000 7fce2effd700 1 -- 172.21.15.3:0/1386716294 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce30004970 con 0x7fce38138f30 2023-10-03T10:12:46.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.716+0000 7f9d42eca700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d3c09d6f0 0x7f9d3c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.716+0000 7f9d42eca700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9d3c09e410 0x7f9d3c134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.716+0000 7f9d42eca700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d3c09b120 con 0x7f9d3c09e000 2023-10-03T10:12:46.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.716+0000 7f9d42eca700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d3c06b460 con 0x7f9d3c09e410 2023-10-03T10:12:46.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.716+0000 7f9d42eca700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c09a7e0 con 0x7f9d3c09d6f0 2023-10-03T10:12:46.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.716+0000 7f9d3bfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9d3c09e000 0x7f9d3c1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50638/0 (socket says 172.21.15.3:50638) 2023-10-03T10:12:46.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.716+0000 7f9d3bfff700 1 -- 172.21.15.3:0/1478766355 learned_addr learned my addr 172.21.15.3:0/1478766355 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:46.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.717+0000 7fce40367700 1 -- 172.21.15.3:0/1386716294 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce3809e980 con 0x7fce38138f30 2023-10-03T10:12:46.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.717+0000 7fce40367700 1 -- 172.21.15.3:0/1386716294 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce38139ee0 con 0x7fce38138f30 2023-10-03T10:12:46.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.717+0000 7f9d3b7fe700 1 -- 172.21.15.3:0/1478766355 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1178142228 0 0) 0x7f9d3c09b120 con 0x7f9d3c09e000 2023-10-03T10:12:46.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.717+0000 7f9d3b7fe700 1 -- 172.21.15.3:0/1478766355 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d2c0035b0 con 0x7f9d3c09e000 2023-10-03T10:12:46.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.717+0000 7f9d40c66700 1 --2- 172.21.15.3:0/1478766355 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d3c09d6f0 0x7f9d3c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.717+0000 7f9d3b7fe700 1 -- 172.21.15.3:0/1478766355 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3479614560 0 0) 0x7f9d2c0035b0 con 0x7f9d3c09e000 2023-10-03T10:12:46.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.717+0000 7f9d3b7fe700 1 -- 172.21.15.3:0/1478766355 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d3c09d6f0 msgr2=0x7f9d3c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:46.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.717+0000 7f9d3b7fe700 1 --2- 172.21.15.3:0/1478766355 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d3c09d6f0 0x7f9d3c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.717+0000 7f9d3b7fe700 1 -- 172.21.15.3:0/1478766355 >> v2:172.21.15.3:3300/0 conn(0x7f9d3c09e410 msgr2=0x7f9d3c134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:46.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.717+0000 7f9d3b7fe700 1 --2- 172.21.15.3:0/1478766355 >> v2:172.21.15.3:3300/0 conn(0x7f9d3c09e410 0x7f9d3c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.717+0000 7f9d3b7fe700 1 -- 172.21.15.3:0/1478766355 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3c09a500 con 0x7f9d3c09e000 2023-10-03T10:12:46.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.717+0000 7f9d3b7fe700 1 -- 172.21.15.3:0/1478766355 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1156569835 0 0) 0x7f9d28001670 con 0x7f9d3c09e000 2023-10-03T10:12:46.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.717+0000 7f9d3b7fe700 1 -- 172.21.15.3:0/1478766355 >> v1:172.21.15.3:6789/0 conn(0x7f9d3c09e000 legacy=0x7f9d3c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.717+0000 7f9d3b7fe700 1 --2- 172.21.15.3:0/1478766355 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d2c003a30 0x7f9d2c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.717+0000 7f9d3b7fe700 1 --2- 172.21.15.3:0/1478766355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d2c0043c0 0x7f9d2c004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.717+0000 7f9d3b7fe700 1 --2- 172.21.15.3:0/1478766355 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d2c005090 0x7f9d2c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.717+0000 7f9d3b7fe700 1 -- 172.21.15.3:0/1478766355 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c06b460 con 0x7f9d2c0043c0 2023-10-03T10:12:46.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.717+0000 7f9d3b7fe700 1 -- 172.21.15.3:0/1478766355 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c09a7e0 con 0x7f9d2c005090 2023-10-03T10:12:46.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.717+0000 7f9d3b7fe700 1 -- 172.21.15.3:0/1478766355 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c00c050 con 0x7f9d2c003a30 2023-10-03T10:12:46.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.718+0000 7f9d3bfff700 1 --2- 172.21.15.3:0/1478766355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d2c0043c0 0x7f9d2c004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.718+0000 7f9d40c66700 1 --2- 172.21.15.3:0/1478766355 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d2c003a30 0x7f9d2c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.718+0000 7fce40367700 1 -- 172.21.15.3:0/1386716294 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fce380722d0 con 0x7fce38138f30 2023-10-03T10:12:46.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.718+0000 7f9d3bfff700 1 -- 172.21.15.3:0/1478766355 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d2c003a30 msgr2=0x7f9d2c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:46.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.718+0000 7f9d3bfff700 1 --2- 172.21.15.3:0/1478766355 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d2c003a30 0x7f9d2c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.718+0000 7f9d3bfff700 1 -- 172.21.15.3:0/1478766355 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d2c005090 msgr2=0x7f9d2c00ba10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-03T10:12:46.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.718+0000 7f9d3bfff700 1 --2- 172.21.15.3:0/1478766355 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d2c005090 0x7f9d2c00ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.718+0000 7f9d3bfff700 1 -- 172.21.15.3:0/1478766355 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3c09a500 con 0x7f9d2c0043c0 2023-10-03T10:12:46.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.718+0000 7fce2effd700 1 -- 172.21.15.3:0/1386716294 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fce30012430 con 0x7fce38138f30 2023-10-03T10:12:46.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.718+0000 7f9d3bfff700 1 --2- 172.21.15.3:0/1478766355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d2c0043c0 0x7f9d2c004890 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9d2800c340 tx=0x7f9d2c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c469b9b160989a1f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:46.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.718+0000 7fce2effd700 1 -- 172.21.15.3:0/1386716294 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7fce300411e0 con 0x7fce38138f30 2023-10-03T10:12:46.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.718+0000 7fce3e103700 1 -- 172.21.15.3:0/1386716294 >> v1:172.21.15.3:6816/127025 conn(0x7fce1c02f540 legacy=0x7fce1c0319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:46.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.718+0000 7f9d3b7fe700 1 -- 172.21.15.3:0/1478766355 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2800cdc0 con 0x7f9d2c0043c0 2023-10-03T10:12:46.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.718+0000 7fce2effd700 1 -- 172.21.15.3:0/1386716294 --> v1:172.21.15.3:6800/129764 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fce1c03c410 con 0x7fce1c038710 2023-10-03T10:12:46.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.718+0000 7fce2effd700 1 -- 172.21.15.3:0/1386716294 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce300437f0 con 0x7fce38138f30 2023-10-03T10:12:46.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.718+0000 7f9d3b7fe700 1 -- 172.21.15.3:0/1478766355 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d28005960 con 0x7f9d2c0043c0 2023-10-03T10:12:46.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.718+0000 7f9d3b7fe700 1 -- 172.21.15.3:0/1478766355 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d28005cf0 con 0x7f9d2c0043c0 2023-10-03T10:12:46.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.719+0000 7f9d42eca700 1 -- 172.21.15.3:0/1478766355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d2c0043c0 msgr2=0x7f9d2c004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.719+0000 7f9d42eca700 1 --2- 172.21.15.3:0/1478766355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d2c0043c0 0x7f9d2c004890 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9d2800c340 tx=0x7f9d2c005880 comp rx=0 tx=0).stop 2023-10-03T10:12:46.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.719+0000 7f9d41467700 1 -- 172.21.15.3:0/1478766355 reap_dead start 2023-10-03T10:12:46.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.719+0000 7f9d42eca700 1 -- 172.21.15.3:0/1478766355 shutdown_connections 2023-10-03T10:12:46.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.719+0000 7f9d42eca700 1 -- 172.21.15.3:0/1478766355 >> 172.21.15.3:0/1478766355 conn(0x7f9d3c066f50 msgr2=0x7f9d3c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:46.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.720+0000 7fce2effd700 1 -- 172.21.15.3:0/1386716294 <== osd.2 v1:172.21.15.3:6800/129764 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (unknown 0 0 75535769) 0x7fce1c03c410 con 0x7fce1c038710 2023-10-03T10:12:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.720+0000 7f9d42eca700 1 -- 172.21.15.3:0/1478766355 shutdown_connections 2023-10-03T10:12:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.720+0000 7f9d42eca700 1 -- 172.21.15.3:0/1478766355 wait complete. 2023-10-03T10:12:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.721+0000 7f9d42eca700 1 Processor -- start 2023-10-03T10:12:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.721+0000 7f9d42eca700 1 -- start start 2023-10-03T10:12:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.722+0000 7f9d42eca700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d2c0043c0 0x7f9d3c1a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.722+0000 7f9d42eca700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d3c1a10e0 0x7f9d3c1a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.722+0000 7f9d42eca700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d3c1c8620 0x7f9d3c1caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.722+0000 7f9d42eca700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c1a2d70 con 0x7f9d2c0043c0 2023-10-03T10:12:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.722+0000 7f9d42eca700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c1a4630 con 0x7f9d3c1c8620 2023-10-03T10:12:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.722+0000 7f9d42eca700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c1a3a00 con 0x7f9d3c1a10e0 2023-10-03T10:12:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.722+0000 7f9d41467700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d3c1c8620 0x7f9d3c1caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.722+0000 7f9d41467700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d3c1c8620 0x7f9d3c1caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59928/0 (socket says 172.21.15.3:59928) 2023-10-03T10:12:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.722+0000 7f9d41467700 1 -- 172.21.15.3:0/736422136 learned_addr learned my addr 172.21.15.3:0/736422136 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.723+0000 7f9d40c66700 1 --2- 172.21.15.3:0/736422136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d2c0043c0 0x7f9d3c1a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.723+0000 7f9d3bfff700 1 --2- 172.21.15.3:0/736422136 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d3c1a10e0 0x7f9d3c1a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.723+0000 7f9d40c66700 1 -- 172.21.15.3:0/736422136 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d3c1a10e0 msgr2=0x7f9d3c1a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.723+0000 7f9d40c66700 1 --2- 172.21.15.3:0/736422136 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d3c1a10e0 0x7f9d3c1a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.723+0000 7f9d40c66700 1 -- 172.21.15.3:0/736422136 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d3c1c8620 msgr2=0x7f9d3c1caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.723+0000 7f9d40c66700 1 --2- 172.21.15.3:0/736422136 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d3c1c8620 0x7f9d3c1caa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.723+0000 7f9d40c66700 1 -- 172.21.15.3:0/736422136 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3c1a2000 con 0x7f9d2c0043c0 2023-10-03T10:12:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.723+0000 7f9d41467700 1 --2- 172.21.15.3:0/736422136 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d3c1c8620 0x7f9d3c1caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.724+0000 7f9d40c66700 1 --2- 172.21.15.3:0/736422136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d2c0043c0 0x7f9d3c1a0ba0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c067350 tx=0x7f9d30011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.724+0000 7f9d397fa700 1 -- 172.21.15.3:0/736422136 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d30004240 con 0x7f9d2c0043c0 2023-10-03T10:12:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.724+0000 7f9d397fa700 1 -- 172.21.15.3:0/736422136 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d300043c0 con 0x7f9d2c0043c0 2023-10-03T10:12:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.724+0000 7f9d42eca700 1 -- 172.21.15.3:0/736422136 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d3c0a2300 con 0x7f9d2c0043c0 2023-10-03T10:12:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.724+0000 7f9d397fa700 1 -- 172.21.15.3:0/736422136 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d300038e0 con 0x7f9d2c0043c0 2023-10-03T10:12:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.724+0000 7f9d42eca700 1 -- 172.21.15.3:0/736422136 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d3c1a2720 con 0x7f9d2c0043c0 2023-10-03T10:12:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.725+0000 7f9d397fa700 1 -- 172.21.15.3:0/736422136 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f9d30003a40 con 0x7f9d2c0043c0 2023-10-03T10:12:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.725+0000 7f9d397fa700 1 -- 172.21.15.3:0/736422136 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f9d30018030 con 0x7f9d2c0043c0 2023-10-03T10:12:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.725+0000 7f9d3bfff700 1 -- 172.21.15.3:0/736422136 >> v1:172.21.15.3:6816/127025 conn(0x7f9d2002f510 legacy=0x7f9d200319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.728+0000 7fbb7b5e2700 1 -- 172.21.15.3:0/641004557 --> v1:172.21.15.3:6812/129912 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbb7409c450 con 0x7fbb58038790 2023-10-03T10:12:46.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.728+0000 7f7083e68700 1 -- 172.21.15.3:0/2511446692 --> v1:172.21.15.3:6804/129556 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f707c0704f0 con 0x7f7060038710 2023-10-03T10:12:46.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.728+0000 7fbb69ffb700 1 -- 172.21.15.3:0/641004557 <== osd.3 v1:172.21.15.3:6812/129912 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 3191560019) 0x7fbb6c0094f0 con 0x7fbb58038790 2023-10-03T10:12:46.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.729+0000 7f70727fc700 1 -- 172.21.15.3:0/2511446692 <== osd.0 v1:172.21.15.3:6804/129556 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 277563570) 0x7f707400bb30 con 0x7f7060038710 2023-10-03T10:12:46.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.729+0000 7f7083e68700 1 -- 172.21.15.3:0/2511446692 >> v1:172.21.15.3:6804/129556 conn(0x7f7060038710 legacy=0x7f706003aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.729+0000 7f7083e68700 1 -- 172.21.15.3:0/2511446692 >> v1:172.21.15.3:6816/127025 conn(0x7f706002f540 legacy=0x7f70600319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:46.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.729+0000 7f7083e68700 1 -- 172.21.15.3:0/2511446692 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f707c09f410 msgr2=0x7f707c11eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.729+0000 7f7083e68700 1 --2- 172.21.15.3:0/2511446692 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f707c09f410 0x7f707c11eb10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f706c009f60 tx=0x7f706c009f90 comp rx=0 tx=0).stop 2023-10-03T10:12:46.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.729+0000 7f9d42eca700 1 -- 172.21.15.3:0/736422136 --> v1:172.21.15.3:6808/129654 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9d3c19f1a0 con 0x7f9d3c061190 2023-10-03T10:12:46.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.730+0000 7f7082405700 1 -- 172.21.15.3:0/2511446692 reap_dead start 2023-10-03T10:12:46.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.730+0000 7f7083e68700 1 -- 172.21.15.3:0/2511446692 shutdown_connections 2023-10-03T10:12:46.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.730+0000 7f7083e68700 1 -- 172.21.15.3:0/2511446692 >> 172.21.15.3:0/2511446692 conn(0x7f707c0a4210 msgr2=0x7f707c09b430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:46.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.729+0000 7fbb7b5e2700 1 -- 172.21.15.3:0/641004557 >> v1:172.21.15.3:6812/129912 conn(0x7fbb58038790 legacy=0x7fbb5803ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.730+0000 7fbb7b5e2700 1 -- 172.21.15.3:0/641004557 >> v1:172.21.15.3:6816/127025 conn(0x7fbb5802f540 legacy=0x7fbb580319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:46.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.730+0000 7fbb7b5e2700 1 -- 172.21.15.3:0/641004557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb7409e1d0 msgr2=0x7fbb741c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.730+0000 7fbb7b5e2700 1 --2- 172.21.15.3:0/641004557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb7409e1d0 0x7fbb741c8930 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400ec10 tx=0x7fbb6400ef20 comp rx=0 tx=0).stop 2023-10-03T10:12:46.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.730+0000 7f7083e68700 1 -- 172.21.15.3:0/2511446692 shutdown_connections 2023-10-03T10:12:46.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.730+0000 7fbb79b7f700 1 -- 172.21.15.3:0/641004557 reap_dead start 2023-10-03T10:12:46.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.730+0000 7fbb7b5e2700 1 -- 172.21.15.3:0/641004557 shutdown_connections 2023-10-03T10:12:46.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.730+0000 7fbb7b5e2700 1 -- 172.21.15.3:0/641004557 >> 172.21.15.3:0/641004557 conn(0x7fbb74066f50 msgr2=0x7fbb74133820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:46.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.730+0000 7f7083e68700 1 -- 172.21.15.3:0/2511446692 wait complete. 2023-10-03T10:12:46.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.730+0000 7fbb7b5e2700 1 -- 172.21.15.3:0/641004557 shutdown_connections 2023-10-03T10:12:46.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.730+0000 7fbb7b5e2700 1 -- 172.21.15.3:0/641004557 wait complete. 2023-10-03T10:12:46.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.731+0000 7f9d397fa700 1 -- 172.21.15.3:0/736422136 <== osd.1 v1:172.21.15.3:6808/129654 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (unknown 0 0 75535769) 0x7f9d3c19f1a0 con 0x7f9d3c061190 2023-10-03T10:12:46.745 INFO:teuthology.orchestra.run.smithi003.stdout:124554051588 2023-10-03T10:12:46.746 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:12:46.746 INFO:teuthology.orchestra.run.smithi003.stdout:137438953475 2023-10-03T10:12:46.747 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:12:46.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.746+0000 7fce40367700 1 -- 172.21.15.3:0/1386716294 --> v1:172.21.15.3:6800/129764 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fce3806f320 con 0x7fce1c038710 2023-10-03T10:12:46.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.747+0000 7fce2effd700 1 -- 172.21.15.3:0/1386716294 <== osd.2 v1:172.21.15.3:6800/129764 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 3918910179) 0x7fce280012c0 con 0x7fce1c038710 2023-10-03T10:12:46.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.747+0000 7fce40367700 1 -- 172.21.15.3:0/1386716294 >> v1:172.21.15.3:6800/129764 conn(0x7fce1c038710 legacy=0x7fce1c03aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.747+0000 7fce40367700 1 -- 172.21.15.3:0/1386716294 >> v1:172.21.15.3:6816/127025 conn(0x7fce1c02f540 legacy=0x7fce1c0319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:46.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.747+0000 7fce40367700 1 -- 172.21.15.3:0/1386716294 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce38138f30 msgr2=0x7fce381393d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.747+0000 7fce40367700 1 --2- 172.21.15.3:0/1386716294 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce38138f30 0x7fce381393d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fce38071cd0 tx=0x7fce30004ad0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.747+0000 7fce3e904700 1 -- 172.21.15.3:0/1386716294 reap_dead start 2023-10-03T10:12:46.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.747+0000 7fce40367700 1 -- 172.21.15.3:0/1386716294 shutdown_connections 2023-10-03T10:12:46.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.747+0000 7fce40367700 1 -- 172.21.15.3:0/1386716294 >> 172.21.15.3:0/1386716294 conn(0x7fce380a2aa0 msgr2=0x7fce380a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:46.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.748+0000 7fce40367700 1 -- 172.21.15.3:0/1386716294 shutdown_connections 2023-10-03T10:12:46.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.748+0000 7fce40367700 1 -- 172.21.15.3:0/1386716294 wait complete. 2023-10-03T10:12:46.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.756+0000 7f9d42eca700 1 -- 172.21.15.3:0/736422136 --> v1:172.21.15.3:6808/129654 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9d3c09f450 con 0x7f9d3c061190 2023-10-03T10:12:46.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.757+0000 7f9d397fa700 1 -- 172.21.15.3:0/736422136 <== osd.1 v1:172.21.15.3:6808/129654 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 3918910179) 0x7f9d34001e40 con 0x7f9d3c061190 2023-10-03T10:12:46.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.757+0000 7f9d42eca700 1 -- 172.21.15.3:0/736422136 >> v1:172.21.15.3:6808/129654 conn(0x7f9d3c061190 legacy=0x7f9d3c061560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.757+0000 7f9d42eca700 1 -- 172.21.15.3:0/736422136 >> v1:172.21.15.3:6816/127025 conn(0x7f9d2002f510 legacy=0x7f9d200319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:46.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.757+0000 7f9d42eca700 1 -- 172.21.15.3:0/736422136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d2c0043c0 msgr2=0x7f9d3c1a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.757+0000 7f9d42eca700 1 --2- 172.21.15.3:0/736422136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d2c0043c0 0x7f9d3c1a0ba0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c067350 tx=0x7f9d30011b40 comp rx=0 tx=0).stop 2023-10-03T10:12:46.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.757+0000 7f9d41467700 1 -- 172.21.15.3:0/736422136 reap_dead start 2023-10-03T10:12:46.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.758+0000 7f9d42eca700 1 -- 172.21.15.3:0/736422136 shutdown_connections 2023-10-03T10:12:46.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.758+0000 7f9d42eca700 1 -- 172.21.15.3:0/736422136 >> 172.21.15.3:0/736422136 conn(0x7f9d3c066f50 msgr2=0x7f9d3c1b6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:46.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.758+0000 7f9d42eca700 1 -- 172.21.15.3:0/736422136 shutdown_connections 2023-10-03T10:12:46.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.758+0000 7f9d42eca700 1 -- 172.21.15.3:0/736422136 wait complete. 2023-10-03T10:12:46.764 INFO:teuthology.orchestra.run.smithi003.stdout:133143986180 2023-10-03T10:12:46.764 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:12:46.775 INFO:teuthology.orchestra.run.smithi003.stdout:133143986180 2023-10-03T10:12:46.775 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:12:46.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.898+0000 7f617e5b3700 1 Processor -- start 2023-10-03T10:12:46.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.898+0000 7f617e5b3700 1 -- start start 2023-10-03T10:12:46.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.899+0000 7f617e5b3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f617809f410 0x7f6178136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.899+0000 7f617e5b3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6178136fe0 0x7f61781393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.899+0000 7f617e5b3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6178070940 con 0x7f617809e930 2023-10-03T10:12:46.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.899+0000 7f617e5b3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f617806fd60 con 0x7f617809f410 2023-10-03T10:12:46.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.899+0000 7f617e5b3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f617806fa80 con 0x7f6178136fe0 2023-10-03T10:12:46.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.899+0000 7f6177fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f617809e930 0x7f617809ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50648/0 (socket says 172.21.15.3:50648) 2023-10-03T10:12:46.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.899+0000 7f6177fff700 1 -- 172.21.15.3:0/48355431 learned_addr learned my addr 172.21.15.3:0/48355431 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:46.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.900+0000 7f61777fe700 1 --2- 172.21.15.3:0/48355431 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f617809f410 0x7f6178136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.900+0000 7f617cb50700 1 --2- 172.21.15.3:0/48355431 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6178136fe0 0x7f61781393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.900+0000 7f6176ffd700 1 -- 172.21.15.3:0/48355431 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3163616364 0 0) 0x7f6178070940 con 0x7f617809e930 2023-10-03T10:12:46.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.900+0000 7f6176ffd700 1 -- 172.21.15.3:0/48355431 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61600035b0 con 0x7f617809e930 2023-10-03T10:12:46.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.900+0000 7f61777fe700 1 -- 172.21.15.3:0/48355431 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6178136fe0 msgr2=0x7f61781393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:46.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.900+0000 7f61777fe700 1 --2- 172.21.15.3:0/48355431 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6178136fe0 0x7f61781393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.900+0000 7f61777fe700 1 -- 172.21.15.3:0/48355431 >> v1:172.21.15.3:6789/0 conn(0x7f617809e930 legacy=0x7f617809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.900+0000 7f61777fe700 1 -- 172.21.15.3:0/48355431 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f617809fb60 con 0x7f617809f410 2023-10-03T10:12:46.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.900+0000 7f617cb50700 1 --2- 172.21.15.3:0/48355431 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6178136fe0 0x7f61781393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:46.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.900+0000 7f61777fe700 1 --2- 172.21.15.3:0/48355431 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f617809f410 0x7f6178136aa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f616c009870 tx=0x7f616c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c1f4723a2736180 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:46.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.901+0000 7f6176ffd700 1 -- 172.21.15.3:0/48355431 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616c011830 con 0x7f617809f410 2023-10-03T10:12:46.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.901+0000 7f6176ffd700 1 -- 172.21.15.3:0/48355431 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616c011990 con 0x7f617809f410 2023-10-03T10:12:46.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.901+0000 7f6176ffd700 1 -- 172.21.15.3:0/48355431 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616c011d20 con 0x7f617809f410 2023-10-03T10:12:46.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.902+0000 7f617e5b3700 1 -- 172.21.15.3:0/48355431 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f617809f410 msgr2=0x7f6178136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.902+0000 7f617e5b3700 1 --2- 172.21.15.3:0/48355431 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f617809f410 0x7f6178136aa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f616c009870 tx=0x7f616c009b80 comp rx=0 tx=0).stop 2023-10-03T10:12:46.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.902+0000 7f617e5b3700 1 -- 172.21.15.3:0/48355431 shutdown_connections 2023-10-03T10:12:46.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.902+0000 7f617e5b3700 1 --2- 172.21.15.3:0/48355431 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6178136fe0 0x7f61781393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.902+0000 7f617e5b3700 1 --2- 172.21.15.3:0/48355431 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f617809f410 0x7f6178136aa0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.902+0000 7f617e5b3700 1 -- 172.21.15.3:0/48355431 >> 172.21.15.3:0/48355431 conn(0x7f61780a4210 msgr2=0x7f61780a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:46.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.902+0000 7f617e5b3700 1 -- 172.21.15.3:0/48355431 shutdown_connections 2023-10-03T10:12:46.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.902+0000 7f617e5b3700 1 -- 172.21.15.3:0/48355431 wait complete. 2023-10-03T10:12:46.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.903+0000 7f617e5b3700 1 Processor -- start 2023-10-03T10:12:46.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.904+0000 7f617e5b3700 1 -- start start 2023-10-03T10:12:46.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.904+0000 7f617e5b3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f617809e930 0x7f617811c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.904+0000 7f617e5b3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f617809f410 0x7f617811cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.904+0000 7f617e5b3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6178136fe0 0x7f617811d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.904+0000 7f617e5b3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617811e140 con 0x7f617809f410 2023-10-03T10:12:46.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.904+0000 7f617e5b3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6178131e60 con 0x7f617809e930 2023-10-03T10:12:46.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.904+0000 7f617e5b3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f617811e9d0 con 0x7f6178136fe0 2023-10-03T10:12:46.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.905+0000 7f61777fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f617809f410 0x7f617811cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.905+0000 7f6177fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f617809e930 0x7f617811c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.905+0000 7f61777fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f617809f410 0x7f617811cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55836/0 (socket says 172.21.15.3:55836) 2023-10-03T10:12:46.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.905+0000 7f61777fe700 1 -- 172.21.15.3:0/1093979838 learned_addr learned my addr 172.21.15.3:0/1093979838 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:46.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.905+0000 7f617cb50700 1 --2- 172.21.15.3:0/1093979838 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6178136fe0 0x7f617811d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.905+0000 7f61777fe700 1 -- 172.21.15.3:0/1093979838 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6178136fe0 msgr2=0x7f617811d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.905+0000 7f61777fe700 1 --2- 172.21.15.3:0/1093979838 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6178136fe0 0x7f617811d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.905+0000 7f61777fe700 1 -- 172.21.15.3:0/1093979838 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f617809e930 msgr2=0x7f617811c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.905+0000 7f61777fe700 1 --2- 172.21.15.3:0/1093979838 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f617809e930 0x7f617811c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.905+0000 7f61777fe700 1 -- 172.21.15.3:0/1093979838 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6170009a70 con 0x7f617809f410 2023-10-03T10:12:46.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.905+0000 7f61777fe700 1 --2- 172.21.15.3:0/1093979838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f617809f410 0x7f617811cc60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f616c000c00 tx=0x7f616c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.905+0000 7f6174ff9700 1 -- 172.21.15.3:0/1093979838 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616c0044d0 con 0x7f617809f410 2023-10-03T10:12:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.906+0000 7f6174ff9700 1 -- 172.21.15.3:0/1093979838 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616c01f070 con 0x7f617809f410 2023-10-03T10:12:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.906+0000 7f617e5b3700 1 -- 172.21.15.3:0/1093979838 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f616c0094a0 con 0x7f617809f410 2023-10-03T10:12:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.906+0000 7f6174ff9700 1 -- 172.21.15.3:0/1093979838 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616c0038e0 con 0x7f617809f410 2023-10-03T10:12:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.906+0000 7f617e5b3700 1 -- 172.21.15.3:0/1093979838 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f617811f800 con 0x7f617809f410 2023-10-03T10:12:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.907+0000 7f6174ff9700 1 -- 172.21.15.3:0/1093979838 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f616c003a40 con 0x7f617809f410 2023-10-03T10:12:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.907+0000 7f6174ff9700 1 -- 172.21.15.3:0/1093979838 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f616c021070 con 0x7f617809f410 2023-10-03T10:12:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.907+0000 7f6177fff700 1 -- 172.21.15.3:0/1093979838 >> v1:172.21.15.3:6816/127025 conn(0x7f616402f540 legacy=0x7f61640319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.907+0000 7f617e5b3700 1 -- 172.21.15.3:0/1093979838 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6178074ea0 con 0x7f617809f410 2023-10-03T10:12:46.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.908+0000 7f0b59c6d700 1 Processor -- start 2023-10-03T10:12:46.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.909+0000 7f0b59c6d700 1 -- start start 2023-10-03T10:12:46.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.909+0000 7f0b59c6d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b54131810 0x7f0b54131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.909+0000 7f0b59c6d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b5409a580 0x7f0b54139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.909+0000 7f0b59c6d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b541313c0 con 0x7f0b54099a40 2023-10-03T10:12:46.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.909+0000 7f0b59c6d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5406e8c0 con 0x7f0b54131810 2023-10-03T10:12:46.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.909+0000 7f0b59c6d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b54071d20 con 0x7f0b5409a580 2023-10-03T10:12:46.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.909+0000 7f0b52ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0b54099a40 0x7f0b54099e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50654/0 (socket says 172.21.15.3:50654) 2023-10-03T10:12:46.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.909+0000 7f0b52ffd700 1 -- 172.21.15.3:0/1426182071 learned_addr learned my addr 172.21.15.3:0/1426182071 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:46.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.909+0000 7f0b537fe700 1 --2- 172.21.15.3:0/1426182071 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b54131810 0x7f0b54131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.910+0000 7f0b537fe700 1 -- 172.21.15.3:0/1426182071 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b5409a580 msgr2=0x7f0b54139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:46.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.910+0000 7f0b537fe700 1 --2- 172.21.15.3:0/1426182071 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b5409a580 0x7f0b54139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.910+0000 7f0b537fe700 1 -- 172.21.15.3:0/1426182071 >> v1:172.21.15.3:6789/0 conn(0x7f0b54099a40 legacy=0x7f0b54099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.910+0000 7f0b537fe700 1 -- 172.21.15.3:0/1426182071 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b540721a0 con 0x7f0b54131810 2023-10-03T10:12:46.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.910+0000 7f0b537fe700 1 --2- 172.21.15.3:0/1426182071 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b54131810 0x7f0b54131be0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0b44009870 tx=0x7f0b44009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fbace26237e615d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:46.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.910+0000 7f0b527fc700 1 -- 172.21.15.3:0/1426182071 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44011830 con 0x7f0b54131810 2023-10-03T10:12:46.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.910+0000 7f0b527fc700 1 -- 172.21.15.3:0/1426182071 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b44011990 con 0x7f0b54131810 2023-10-03T10:12:46.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.910+0000 7f0b527fc700 1 -- 172.21.15.3:0/1426182071 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44011d20 con 0x7f0b54131810 2023-10-03T10:12:46.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.911+0000 7f0b59c6d700 1 -- 172.21.15.3:0/1426182071 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b54131810 msgr2=0x7f0b54131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.911+0000 7f0b59c6d700 1 --2- 172.21.15.3:0/1426182071 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b54131810 0x7f0b54131be0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0b44009870 tx=0x7f0b44009b80 comp rx=0 tx=0).stop 2023-10-03T10:12:46.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.911+0000 7f0b59c6d700 1 -- 172.21.15.3:0/1426182071 shutdown_connections 2023-10-03T10:12:46.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.911+0000 7f0b59c6d700 1 --2- 172.21.15.3:0/1426182071 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b54131810 0x7f0b54131be0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.911+0000 7f0b59c6d700 1 --2- 172.21.15.3:0/1426182071 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b5409a580 0x7f0b54139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.911+0000 7f0b59c6d700 1 -- 172.21.15.3:0/1426182071 >> 172.21.15.3:0/1426182071 conn(0x7f0b540a2aa0 msgr2=0x7f0b540a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:46.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.911+0000 7f0b59c6d700 1 -- 172.21.15.3:0/1426182071 shutdown_connections 2023-10-03T10:12:46.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.911+0000 7f0b59c6d700 1 -- 172.21.15.3:0/1426182071 wait complete. 2023-10-03T10:12:46.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.912+0000 7f0b59c6d700 1 Processor -- start 2023-10-03T10:12:46.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.912+0000 7f0b59c6d700 1 -- start start 2023-10-03T10:12:46.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.913+0000 7f0b59c6d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b54099a40 0x7f0b541cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.913+0000 7f0b59c6d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b5409a580 0x7f0b541cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.913+0000 7f0b59c6d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b54131810 0x7f0b541cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.913+0000 7f0b59c6d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b54126560 con 0x7f0b54099a40 2023-10-03T10:12:46.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.913+0000 7f0b59c6d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5406eb00 con 0x7f0b54131810 2023-10-03T10:12:46.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.913+0000 7f0b59c6d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540a1fc0 con 0x7f0b5409a580 2023-10-03T10:12:46.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.913+0000 7f0b52ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b5409a580 0x7f0b541cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.913+0000 7f0b537fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b54099a40 0x7f0b541cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.913+0000 7f0b52ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b5409a580 0x7f0b541cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38530/0 (socket says 172.21.15.3:38530) 2023-10-03T10:12:46.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.913+0000 7f0b52ffd700 1 -- 172.21.15.3:0/4183087065 learned_addr learned my addr 172.21.15.3:0/4183087065 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:46.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.913+0000 7f0b53fff700 1 --2- 172.21.15.3:0/4183087065 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b54131810 0x7f0b541cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.913+0000 7f6174ff9700 1 -- 172.21.15.3:0/1093979838 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f616c015020 con 0x7f617809f410 2023-10-03T10:12:46.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.913+0000 7f0b52ffd700 1 -- 172.21.15.3:0/4183087065 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b54131810 msgr2=0x7f0b541cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.913+0000 7f0b52ffd700 1 --2- 172.21.15.3:0/4183087065 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b54131810 0x7f0b541cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.913+0000 7f0b52ffd700 1 -- 172.21.15.3:0/4183087065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b54099a40 msgr2=0x7f0b541cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.913+0000 7f0b52ffd700 1 --2- 172.21.15.3:0/4183087065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b54099a40 0x7f0b541cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.913+0000 7f0b52ffd700 1 -- 172.21.15.3:0/4183087065 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b440094a0 con 0x7f0b5409a580 2023-10-03T10:12:46.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.914+0000 7f0b537fe700 1 --2- 172.21.15.3:0/4183087065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b54099a40 0x7f0b541cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:46.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.914+0000 7f0b52ffd700 1 --2- 172.21.15.3:0/4183087065 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b5409a580 0x7f0b541cccb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800c500 tx=0x7f0b4800c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:46.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.914+0000 7f0b3bfff700 1 -- 172.21.15.3:0/4183087065 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b48005d70 con 0x7f0b5409a580 2023-10-03T10:12:46.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.914+0000 7f0b3bfff700 1 -- 172.21.15.3:0/4183087065 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b480070d0 con 0x7f0b5409a580 2023-10-03T10:12:46.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.914+0000 7f0b59c6d700 1 -- 172.21.15.3:0/4183087065 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b5411bc20 con 0x7f0b5409a580 2023-10-03T10:12:46.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.914+0000 7f0b3bfff700 1 -- 172.21.15.3:0/4183087065 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b48007440 con 0x7f0b5409a580 2023-10-03T10:12:46.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.914+0000 7f0b59c6d700 1 -- 172.21.15.3:0/4183087065 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b54126ef0 con 0x7f0b5409a580 2023-10-03T10:12:46.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.915+0000 7f0b3bfff700 1 -- 172.21.15.3:0/4183087065 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f0b480075a0 con 0x7f0b5409a580 2023-10-03T10:12:46.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.915+0000 7f0b3bfff700 1 -- 172.21.15.3:0/4183087065 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f0b4803ef60 con 0x7f0b5409a580 2023-10-03T10:12:46.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.915+0000 7f0b537fe700 1 -- 172.21.15.3:0/4183087065 >> v1:172.21.15.3:6816/127025 conn(0x7f0b4002f510 legacy=0x7f0b400319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:46.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.915+0000 7f0b59c6d700 1 -- 172.21.15.3:0/4183087065 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b54074ea0 con 0x7f0b5409a580 2023-10-03T10:12:46.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.921+0000 7f0b3bfff700 1 -- 172.21.15.3:0/4183087065 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0b48017020 con 0x7f0b5409a580 2023-10-03T10:12:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.970+0000 7f8cfe69f700 1 Processor -- start 2023-10-03T10:12:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.970+0000 7f8cfe69f700 1 -- start start 2023-10-03T10:12:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.971+0000 7f8cfe69f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cf81338a0 0x7f8cf8133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.971+0000 7f8cfe69f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8cf809de50 0x7f8cf809e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.971+0000 7f8cfe69f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cf806ef30 con 0x7f8cf809e760 2023-10-03T10:12:46.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.971+0000 7f8cfe69f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf806fef0 con 0x7f8cf809de50 2023-10-03T10:12:46.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.971+0000 7f8cfe69f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf81334b0 con 0x7f8cf81338a0 2023-10-03T10:12:46.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.971+0000 7f8cf77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8cf809de50 0x7f8cf809e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.971+0000 7f8cf7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cf81338a0 0x7f8cf8133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.971+0000 7f8cf77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8cf809de50 0x7f8cf809e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59978/0 (socket says 172.21.15.3:59978) 2023-10-03T10:12:46.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.971+0000 7f8cf77fe700 1 -- 172.21.15.3:0/2983247180 learned_addr learned my addr 172.21.15.3:0/2983247180 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:46.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.972+0000 7f8cf77fe700 1 -- 172.21.15.3:0/2983247180 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cf81338a0 msgr2=0x7f8cf8133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:46.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.972+0000 7f8cf77fe700 1 --2- 172.21.15.3:0/2983247180 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cf81338a0 0x7f8cf8133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.972+0000 7f8cf77fe700 1 -- 172.21.15.3:0/2983247180 >> v1:172.21.15.3:6789/0 conn(0x7f8cf809e760 legacy=0x7f8cf813a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:46.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.972+0000 7f8cf77fe700 1 -- 172.21.15.3:0/2983247180 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf806fc30 con 0x7f8cf809de50 2023-10-03T10:12:46.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.972+0000 7f8cf7fff700 1 --2- 172.21.15.3:0/2983247180 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cf81338a0 0x7f8cf8133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:46.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.972+0000 7f8cf77fe700 1 --2- 172.21.15.3:0/2983247180 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8cf809de50 0x7f8cf809e220 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8cec009920 tx=0x7f8cec009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c9d2e5cb42aa53 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:46.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.972+0000 7f8cf6ffd700 1 -- 172.21.15.3:0/2983247180 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec004230 con 0x7f8cf809de50 2023-10-03T10:12:46.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.972+0000 7f8cf6ffd700 1 -- 172.21.15.3:0/2983247180 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cec004390 con 0x7f8cf809de50 2023-10-03T10:12:46.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.972+0000 7f8cf6ffd700 1 -- 172.21.15.3:0/2983247180 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec0036a0 con 0x7f8cf809de50 2023-10-03T10:12:46.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.973+0000 7f8cfe69f700 1 -- 172.21.15.3:0/2983247180 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8cf809de50 msgr2=0x7f8cf809e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.973+0000 7f8cfe69f700 1 --2- 172.21.15.3:0/2983247180 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8cf809de50 0x7f8cf809e220 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8cec009920 tx=0x7f8cec009c30 comp rx=0 tx=0).stop 2023-10-03T10:12:46.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.973+0000 7f8cfe69f700 1 -- 172.21.15.3:0/2983247180 shutdown_connections 2023-10-03T10:12:46.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.973+0000 7f8cfe69f700 1 --2- 172.21.15.3:0/2983247180 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cf81338a0 0x7f8cf8133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.973+0000 7f8cfe69f700 1 --2- 172.21.15.3:0/2983247180 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8cf809de50 0x7f8cf809e220 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.973+0000 7f8cfe69f700 1 -- 172.21.15.3:0/2983247180 >> 172.21.15.3:0/2983247180 conn(0x7f8cf80a2aa0 msgr2=0x7f8cf80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:46.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.973+0000 7f8cfe69f700 1 -- 172.21.15.3:0/2983247180 shutdown_connections 2023-10-03T10:12:46.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.973+0000 7f8cfe69f700 1 -- 172.21.15.3:0/2983247180 wait complete. 2023-10-03T10:12:46.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.974+0000 7f8cfe69f700 1 Processor -- start 2023-10-03T10:12:46.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.975+0000 7f8cfe69f700 1 -- start start 2023-10-03T10:12:46.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.975+0000 7f8cfe69f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cf809de50 0x7f8cf811eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.975+0000 7f8cfe69f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8cf809e760 0x7f8cf811f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.976+0000 7f8cfe69f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cf81338a0 0x7f8cf811f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.976+0000 7f8cfe69f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf8121ea0 con 0x7f8cf81338a0 2023-10-03T10:12:46.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.976+0000 7f8cfe69f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf8122080 con 0x7f8cf809e760 2023-10-03T10:12:46.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.976+0000 7f8cfe69f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf812f690 con 0x7f8cf809de50 2023-10-03T10:12:46.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.976+0000 7f8cf77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8cf809e760 0x7f8cf811f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.976+0000 7f8cf77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8cf809e760 0x7f8cf811f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59988/0 (socket says 172.21.15.3:59988) 2023-10-03T10:12:46.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.976+0000 7f8cf77fe700 1 -- 172.21.15.3:0/2833960605 learned_addr learned my addr 172.21.15.3:0/2833960605 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:46.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.976+0000 7f8cf7fff700 1 --2- 172.21.15.3:0/2833960605 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cf809de50 0x7f8cf811eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.976+0000 7f8cf77fe700 1 -- 172.21.15.3:0/2833960605 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cf809de50 msgr2=0x7f8cf811eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.976+0000 7f8cf77fe700 1 --2- 172.21.15.3:0/2833960605 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cf809de50 0x7f8cf811eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.976+0000 7f8cf77fe700 1 -- 172.21.15.3:0/2833960605 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cf81338a0 msgr2=0x7f8cf811f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.976+0000 7f8cf77fe700 1 --2- 172.21.15.3:0/2833960605 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cf81338a0 0x7f8cf811f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.976+0000 7f8cf77fe700 1 -- 172.21.15.3:0/2833960605 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce80094a0 con 0x7f8cf809e760 2023-10-03T10:12:46.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.976+0000 7f8cf77fe700 1 --2- 172.21.15.3:0/2833960605 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8cf809e760 0x7f8cf811f0d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0098f0 tx=0x7f8cec004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:46.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.977+0000 7f8cf4ff9700 1 -- 172.21.15.3:0/2833960605 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec003e60 con 0x7f8cf809e760 2023-10-03T10:12:46.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.977+0000 7f8cfe69f700 1 -- 172.21.15.3:0/2833960605 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cec009550 con 0x7f8cf809e760 2023-10-03T10:12:46.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.977+0000 7f8cf4ff9700 1 -- 172.21.15.3:0/2833960605 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cec011630 con 0x7f8cf809e760 2023-10-03T10:12:46.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.977+0000 7f8cf4ff9700 1 -- 172.21.15.3:0/2833960605 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec0119c0 con 0x7f8cf809e760 2023-10-03T10:12:46.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.977+0000 7f8cfe69f700 1 -- 172.21.15.3:0/2833960605 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf806f640 con 0x7f8cf809e760 2023-10-03T10:12:46.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.978+0000 7f8cfe69f700 1 -- 172.21.15.3:0/2833960605 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cf8074ea0 con 0x7f8cf809e760 2023-10-03T10:12:46.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.979+0000 7f8cf4ff9700 1 -- 172.21.15.3:0/2833960605 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f8cec011b20 con 0x7f8cf809e760 2023-10-03T10:12:46.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.979+0000 7f8cf7fff700 1 -- 172.21.15.3:0/2833960605 >> v1:172.21.15.3:6816/127025 conn(0x7f8ce402f510 legacy=0x7f8ce40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:46.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.979+0000 7f8cf4ff9700 1 -- 172.21.15.3:0/2833960605 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f8cec041410 con 0x7f8cf809e760 2023-10-03T10:12:46.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.988+0000 7f8cf4ff9700 1 -- 172.21.15.3:0/2833960605 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8cec010380 con 0x7f8cf809e760 2023-10-03T10:12:46.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.990+0000 7f81997c1700 1 Processor -- start 2023-10-03T10:12:46.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.990+0000 7f81997c1700 1 -- start start 2023-10-03T10:12:46.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.990+0000 7f81997c1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f819409d6f0 0x7f819409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.990+0000 7f81997c1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f819409e000 0x7f8194130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.990+0000 7f81997c1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f819409e410 0x7f8194130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.990+0000 7f81997c1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f819409f520 con 0x7f819409e410 2023-10-03T10:12:46.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.990+0000 7f81997c1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f819409f130 con 0x7f819409e000 2023-10-03T10:12:46.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.990+0000 7f81997c1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f819409e930 con 0x7f819409d6f0 2023-10-03T10:12:46.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.990+0000 7f8192ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f819409d6f0 0x7f819409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.990+0000 7f81927fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f819409e000 0x7f8194130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.991+0000 7f8192ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f819409d6f0 0x7f819409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38546/0 (socket says 172.21.15.3:38546) 2023-10-03T10:12:46.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.991+0000 7f81927fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f819409e000 0x7f8194130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60002/0 (socket says 172.21.15.3:60002) 2023-10-03T10:12:46.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.991+0000 7f8192ffd700 1 -- 172.21.15.3:0/2501289157 learned_addr learned my addr 172.21.15.3:0/2501289157 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:46.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.991+0000 7f81937fe700 1 --2- 172.21.15.3:0/2501289157 >> v2:172.21.15.3:3300/0 conn(0x7f819409e410 0x7f8194130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.991+0000 7f8192ffd700 1 -- 172.21.15.3:0/2501289157 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f819409e000 msgr2=0x7f8194130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:46.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.991+0000 7f8192ffd700 1 --2- 172.21.15.3:0/2501289157 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f819409e000 0x7f8194130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.991+0000 7f8192ffd700 1 -- 172.21.15.3:0/2501289157 >> v2:172.21.15.3:3300/0 conn(0x7f819409e410 msgr2=0x7f8194130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:46.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.991+0000 7f8192ffd700 1 --2- 172.21.15.3:0/2501289157 >> v2:172.21.15.3:3300/0 conn(0x7f819409e410 0x7f8194130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.991+0000 7f8192ffd700 1 -- 172.21.15.3:0/2501289157 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819409b2d0 con 0x7f819409d6f0 2023-10-03T10:12:46.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.991+0000 7f8192ffd700 1 --2- 172.21.15.3:0/2501289157 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f819409d6f0 0x7f819409dac0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8184009920 tx=0x7f8184009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc697e89b27c9a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:46.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.992+0000 7f8191ffb700 1 -- 172.21.15.3:0/2501289157 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8184011830 con 0x7f819409d6f0 2023-10-03T10:12:46.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.992+0000 7f8191ffb700 1 -- 172.21.15.3:0/2501289157 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8184011990 con 0x7f819409d6f0 2023-10-03T10:12:46.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.992+0000 7f8191ffb700 1 -- 172.21.15.3:0/2501289157 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8184011d20 con 0x7f819409d6f0 2023-10-03T10:12:46.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.992+0000 7f81997c1700 1 -- 172.21.15.3:0/2501289157 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f819409d6f0 msgr2=0x7f819409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.992+0000 7f81997c1700 1 --2- 172.21.15.3:0/2501289157 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f819409d6f0 0x7f819409dac0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8184009920 tx=0x7f8184009c30 comp rx=0 tx=0).stop 2023-10-03T10:12:46.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.992+0000 7f81997c1700 1 -- 172.21.15.3:0/2501289157 shutdown_connections 2023-10-03T10:12:46.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.992+0000 7f81997c1700 1 --2- 172.21.15.3:0/2501289157 >> v2:172.21.15.3:3300/0 conn(0x7f819409e410 0x7f8194130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.992+0000 7f81997c1700 1 --2- 172.21.15.3:0/2501289157 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f819409d6f0 0x7f819409dac0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.992+0000 7f81997c1700 1 --2- 172.21.15.3:0/2501289157 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f819409e000 0x7f8194130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.992+0000 7f81997c1700 1 -- 172.21.15.3:0/2501289157 >> 172.21.15.3:0/2501289157 conn(0x7f8194066f50 msgr2=0x7f819409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:46.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.993+0000 7f81997c1700 1 -- 172.21.15.3:0/2501289157 shutdown_connections 2023-10-03T10:12:46.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.993+0000 7f81997c1700 1 -- 172.21.15.3:0/2501289157 wait complete. 2023-10-03T10:12:46.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.994+0000 7f81997c1700 1 Processor -- start 2023-10-03T10:12:46.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.994+0000 7f81997c1700 1 -- start start 2023-10-03T10:12:46.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.994+0000 7f81997c1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f819409d6f0 0x7f819412fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.994+0000 7f81997c1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f819409e000 0x7f81941302b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.994+0000 7f81997c1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f819409e410 0x7f81941307f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:46.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.994+0000 7f81997c1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81941bcf60 con 0x7f819409e410 2023-10-03T10:12:46.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.994+0000 7f81997c1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81940a3b60 con 0x7f819409d6f0 2023-10-03T10:12:46.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.994+0000 7f81997c1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f819409c700 con 0x7f819409e000 2023-10-03T10:12:46.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.995+0000 7f8192ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f819409d6f0 0x7f819412fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.995+0000 7f8192ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f819409d6f0 0x7f819412fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60008/0 (socket says 172.21.15.3:60008) 2023-10-03T10:12:46.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.995+0000 7f8192ffd700 1 -- 172.21.15.3:0/142312691 learned_addr learned my addr 172.21.15.3:0/142312691 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:46.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.995+0000 7f81927fc700 1 --2- 172.21.15.3:0/142312691 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f819409e000 0x7f81941302b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.995+0000 7f81937fe700 1 --2- 172.21.15.3:0/142312691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f819409e410 0x7f81941307f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:46.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.995+0000 7f8192ffd700 1 -- 172.21.15.3:0/142312691 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f819409e000 msgr2=0x7f81941302b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.995+0000 7f8192ffd700 1 --2- 172.21.15.3:0/142312691 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f819409e000 0x7f81941302b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.995+0000 7f8192ffd700 1 -- 172.21.15.3:0/142312691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f819409e410 msgr2=0x7f81941307f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:46.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.995+0000 7f8192ffd700 1 --2- 172.21.15.3:0/142312691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f819409e410 0x7f81941307f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:46.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.995+0000 7f8192ffd700 1 -- 172.21.15.3:0/142312691 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8184009550 con 0x7f819409d6f0 2023-10-03T10:12:46.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.995+0000 7f81937fe700 1 --2- 172.21.15.3:0/142312691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f819409e410 0x7f81941307f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:12:46.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.995+0000 7f81927fc700 1 --2- 172.21.15.3:0/142312691 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f819409e000 0x7f81941302b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:12:46.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.995+0000 7f8192ffd700 1 --2- 172.21.15.3:0/142312691 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f819409d6f0 0x7f819412fd70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8184000c00 tx=0x7f8184004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:46.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.996+0000 7f817b7fe700 1 -- 172.21.15.3:0/142312691 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8184004300 con 0x7f819409d6f0 2023-10-03T10:12:46.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.996+0000 7f817b7fe700 1 -- 172.21.15.3:0/142312691 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818401f070 con 0x7f819409d6f0 2023-10-03T10:12:46.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.996+0000 7f817b7fe700 1 -- 172.21.15.3:0/142312691 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81840036a0 con 0x7f819409d6f0 2023-10-03T10:12:46.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.996+0000 7f81997c1700 1 -- 172.21.15.3:0/142312691 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81941c11f0 con 0x7f819409d6f0 2023-10-03T10:12:46.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.996+0000 7f81997c1700 1 -- 172.21.15.3:0/142312691 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81941bf570 con 0x7f819409d6f0 2023-10-03T10:12:46.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.997+0000 7f81997c1700 1 -- 172.21.15.3:0/142312691 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8194074ea0 con 0x7f819409d6f0 2023-10-03T10:12:46.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.998+0000 7f817b7fe700 1 -- 172.21.15.3:0/142312691 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f8184021030 con 0x7f819409d6f0 2023-10-03T10:12:46.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.998+0000 7f817b7fe700 1 -- 172.21.15.3:0/142312691 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f8184041400 con 0x7f819409d6f0 2023-10-03T10:12:46.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:46.998+0000 7f81927fc700 1 -- 172.21.15.3:0/142312691 >> v1:172.21.15.3:6816/127025 conn(0x7f818002f510 legacy=0x7f81800319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:47.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.003+0000 7f817b7fe700 1 -- 172.21.15.3:0/142312691 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f81840239c0 con 0x7f819409d6f0 2023-10-03T10:12:47.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.108+0000 7f6177fff700 1 -- 172.21.15.3:0/1093979838 >> v1:172.21.15.3:6816/127025 conn(0x7f616402f540 legacy=0x7f61640319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:47.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.116+0000 7f0b537fe700 1 -- 172.21.15.3:0/4183087065 >> v1:172.21.15.3:6816/127025 conn(0x7f0b4002f510 legacy=0x7f0b400319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:47.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.127+0000 7f617e5b3700 1 -- 172.21.15.3:0/1093979838 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f617806b250 con 0x7f617809f410 2023-10-03T10:12:47.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.127+0000 7f6174ff9700 1 -- 172.21.15.3:0/1093979838 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f616c01a950 con 0x7f617809f410 2023-10-03T10:12:47.127 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:12:47.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.130+0000 7f617e5b3700 1 -- 172.21.15.3:0/1093979838 >> v1:172.21.15.3:6816/127025 conn(0x7f616402f540 legacy=0x7f61640319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:47.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.130+0000 7f617e5b3700 1 -- 172.21.15.3:0/1093979838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f617809f410 msgr2=0x7f617811cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:47.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.130+0000 7f617e5b3700 1 --2- 172.21.15.3:0/1093979838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f617809f410 0x7f617811cc60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f616c000c00 tx=0x7f616c004750 comp rx=0 tx=0).stop 2023-10-03T10:12:47.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.131+0000 7f617e5b3700 1 -- 172.21.15.3:0/1093979838 shutdown_connections 2023-10-03T10:12:47.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.131+0000 7f617e5b3700 1 --2- 172.21.15.3:0/1093979838 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6178136fe0 0x7f617811d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:47.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.131+0000 7f617e5b3700 1 --2- 172.21.15.3:0/1093979838 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f617809e930 0x7f617811c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:47.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.131+0000 7f617e5b3700 1 --2- 172.21.15.3:0/1093979838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f617809f410 0x7f617811cc60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:47.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.131+0000 7f617e5b3700 1 -- 172.21.15.3:0/1093979838 >> 172.21.15.3:0/1093979838 conn(0x7f61780a4210 msgr2=0x7f61780a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:47.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.131+0000 7f617e5b3700 1 -- 172.21.15.3:0/1093979838 shutdown_connections 2023-10-03T10:12:47.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.131+0000 7f617e5b3700 1 -- 172.21.15.3:0/1093979838 wait complete. 2023-10-03T10:12:47.145 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:12:47.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.158+0000 7f0b59c6d700 1 -- 172.21.15.3:0/4183087065 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0b54126350 con 0x7f0b5409a580 2023-10-03T10:12:47.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.158+0000 7f0b3bfff700 1 -- 172.21.15.3:0/4183087065 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0b48019d20 con 0x7f0b5409a580 2023-10-03T10:12:47.159 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:12:47.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.160+0000 7f0b59c6d700 1 -- 172.21.15.3:0/4183087065 >> v1:172.21.15.3:6816/127025 conn(0x7f0b4002f510 legacy=0x7f0b400319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:47.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.160+0000 7f0b59c6d700 1 -- 172.21.15.3:0/4183087065 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b5409a580 msgr2=0x7f0b541cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:47.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.160+0000 7f0b59c6d700 1 --2- 172.21.15.3:0/4183087065 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b5409a580 0x7f0b541cccb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800c500 tx=0x7f0b4800c810 comp rx=0 tx=0).stop 2023-10-03T10:12:47.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.160+0000 7f0b59c6d700 1 -- 172.21.15.3:0/4183087065 shutdown_connections 2023-10-03T10:12:47.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.160+0000 7f0b59c6d700 1 --2- 172.21.15.3:0/4183087065 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b54131810 0x7f0b541cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:47.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.160+0000 7f0b59c6d700 1 --2- 172.21.15.3:0/4183087065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b54099a40 0x7f0b541cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:47.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.160+0000 7f0b59c6d700 1 --2- 172.21.15.3:0/4183087065 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b5409a580 0x7f0b541cccb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:47.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.160+0000 7f0b59c6d700 1 -- 172.21.15.3:0/4183087065 >> 172.21.15.3:0/4183087065 conn(0x7f0b540a2aa0 msgr2=0x7f0b541306a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:47.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.161+0000 7f0b59c6d700 1 -- 172.21.15.3:0/4183087065 shutdown_connections 2023-10-03T10:12:47.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.161+0000 7f0b59c6d700 1 -- 172.21.15.3:0/4183087065 wait complete. 2023-10-03T10:12:47.175 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:12:47.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.180+0000 7f8cf7fff700 1 -- 172.21.15.3:0/2833960605 >> v1:172.21.15.3:6816/127025 conn(0x7f8ce402f510 legacy=0x7f8ce40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:47.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.199+0000 7f81927fc700 1 -- 172.21.15.3:0/142312691 >> v1:172.21.15.3:6816/127025 conn(0x7f818002f510 legacy=0x7f81800319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:47.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.200+0000 7f8cfe69f700 1 -- 172.21.15.3:0/2833960605 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8cf8122400 con 0x7f8cf809e760 2023-10-03T10:12:47.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.201+0000 7f8cf4ff9700 1 -- 172.21.15.3:0/2833960605 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8cec02bb40 con 0x7f8cf809e760 2023-10-03T10:12:47.201 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:12:47.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.203+0000 7f8cfe69f700 1 -- 172.21.15.3:0/2833960605 >> v1:172.21.15.3:6816/127025 conn(0x7f8ce402f510 legacy=0x7f8ce40319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:47.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.203+0000 7f8cfe69f700 1 -- 172.21.15.3:0/2833960605 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8cf809e760 msgr2=0x7f8cf811f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:47.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.203+0000 7f8cfe69f700 1 --2- 172.21.15.3:0/2833960605 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8cf809e760 0x7f8cf811f0d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0098f0 tx=0x7f8cec004750 comp rx=0 tx=0).stop 2023-10-03T10:12:47.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.203+0000 7f8cfe69f700 1 -- 172.21.15.3:0/2833960605 shutdown_connections 2023-10-03T10:12:47.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.203+0000 7f8cfe69f700 1 --2- 172.21.15.3:0/2833960605 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cf81338a0 0x7f8cf811f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:47.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.203+0000 7f8cfe69f700 1 --2- 172.21.15.3:0/2833960605 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cf809de50 0x7f8cf811eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:47.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.203+0000 7f8cfe69f700 1 --2- 172.21.15.3:0/2833960605 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8cf809e760 0x7f8cf811f0d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:47.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.203+0000 7f8cfe69f700 1 -- 172.21.15.3:0/2833960605 >> 172.21.15.3:0/2833960605 conn(0x7f8cf80a2aa0 msgr2=0x7f8cf809ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:47.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.203+0000 7f8cfe69f700 1 -- 172.21.15.3:0/2833960605 shutdown_connections 2023-10-03T10:12:47.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.203+0000 7f8cfe69f700 1 -- 172.21.15.3:0/2833960605 wait complete. 2023-10-03T10:12:47.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.209+0000 7f81997c1700 1 -- 172.21.15.3:0/142312691 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f819406c360 con 0x7f819409d6f0 2023-10-03T10:12:47.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.210+0000 7f817b7fe700 1 -- 172.21.15.3:0/142312691 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8184015070 con 0x7f819409d6f0 2023-10-03T10:12:47.210 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:12:47.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.212+0000 7f81997c1700 1 -- 172.21.15.3:0/142312691 >> v1:172.21.15.3:6816/127025 conn(0x7f818002f510 legacy=0x7f81800319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:47.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.212+0000 7f81997c1700 1 -- 172.21.15.3:0/142312691 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f819409d6f0 msgr2=0x7f819412fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:47.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.212+0000 7f81997c1700 1 --2- 172.21.15.3:0/142312691 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f819409d6f0 0x7f819412fd70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8184000c00 tx=0x7f8184004750 comp rx=0 tx=0).stop 2023-10-03T10:12:47.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.212+0000 7f81997c1700 1 -- 172.21.15.3:0/142312691 shutdown_connections 2023-10-03T10:12:47.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.212+0000 7f81997c1700 1 --2- 172.21.15.3:0/142312691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f819409e410 0x7f81941307f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:47.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.212+0000 7f81997c1700 1 --2- 172.21.15.3:0/142312691 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f819409d6f0 0x7f819412fd70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:47.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.212+0000 7f81997c1700 1 --2- 172.21.15.3:0/142312691 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f819409e000 0x7f81941302b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:47.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.212+0000 7f81997c1700 1 -- 172.21.15.3:0/142312691 >> 172.21.15.3:0/142312691 conn(0x7f8194066f50 msgr2=0x7f8194070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:47.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.212+0000 7f81997c1700 1 -- 172.21.15.3:0/142312691 shutdown_connections 2023-10-03T10:12:47.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:47.212+0000 7f81997c1700 1 -- 172.21.15.3:0/142312691 wait complete. 2023-10-03T10:12:47.218 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:12:47.226 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:12:48.147 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:12:48.175 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:12:48.219 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:12:48.227 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:12:48.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.310+0000 7fec7b8c4700 1 Processor -- start 2023-10-03T10:12:48.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.310+0000 7fec7b8c4700 1 -- start start 2023-10-03T10:12:48.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.311+0000 7fec7b8c4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fec741338a0 0x7fec74133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:48.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.311+0000 7fec7b8c4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fec7409de50 0x7fec7409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.311+0000 7fec7b8c4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fec7409e760 0x7fec7413a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.311+0000 7fec7b8c4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec7406fef0 con 0x7fec741338a0 2023-10-03T10:12:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.311+0000 7fec7b8c4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7406ef30 con 0x7fec7409e760 2023-10-03T10:12:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.311+0000 7fec7b8c4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fec741334b0 con 0x7fec7409de50 2023-10-03T10:12:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.311+0000 7fec79e61700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fec7409e760 0x7fec7413a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.311+0000 7fec79660700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fec741338a0 0x7fec74133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.311+0000 7fec79660700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fec741338a0 0x7fec74133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55874/0 (socket says 172.21.15.3:55874) 2023-10-03T10:12:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.311+0000 7fec79e61700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fec7409e760 0x7fec7413a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60010/0 (socket says 172.21.15.3:60010) 2023-10-03T10:12:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.311+0000 7fec79660700 1 -- 172.21.15.3:0/1266104024 learned_addr learned my addr 172.21.15.3:0/1266104024 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.311+0000 7fec78e5f700 1 --2- 172.21.15.3:0/1266104024 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fec7409de50 0x7fec7409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:48.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.312+0000 7fec79660700 1 -- 172.21.15.3:0/1266104024 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fec7409de50 msgr2=0x7fec7409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:48.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.312+0000 7fec79660700 1 --2- 172.21.15.3:0/1266104024 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fec7409de50 0x7fec7409e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.312+0000 7fec79660700 1 -- 172.21.15.3:0/1266104024 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fec7409e760 msgr2=0x7fec7413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:48.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.312+0000 7fec79660700 1 --2- 172.21.15.3:0/1266104024 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fec7409e760 0x7fec7413a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.312+0000 7fec79660700 1 -- 172.21.15.3:0/1266104024 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7406fc30 con 0x7fec741338a0 2023-10-03T10:12:48.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.312+0000 7fec79e61700 1 --2- 172.21.15.3:0/1266104024 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fec7409e760 0x7fec7413a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:48.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.312+0000 7fec79660700 1 --2- 172.21.15.3:0/1266104024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec741338a0 0x7fec74133c70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fec700098f0 tx=0x7fec70009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c5262e3b88f356 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:48.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.312+0000 7fec6bfff700 1 -- 172.21.15.3:0/1266104024 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec70011830 con 0x7fec741338a0 2023-10-03T10:12:48.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.312+0000 7fec6bfff700 1 -- 172.21.15.3:0/1266104024 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec70011990 con 0x7fec741338a0 2023-10-03T10:12:48.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.312+0000 7fec6bfff700 1 -- 172.21.15.3:0/1266104024 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec70011d20 con 0x7fec741338a0 2023-10-03T10:12:48.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.313+0000 7fec7b8c4700 1 -- 172.21.15.3:0/1266104024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec741338a0 msgr2=0x7fec74133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:48.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.313+0000 7fec7b8c4700 1 --2- 172.21.15.3:0/1266104024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec741338a0 0x7fec74133c70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fec700098f0 tx=0x7fec70009c00 comp rx=0 tx=0).stop 2023-10-03T10:12:48.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.313+0000 7fec7b8c4700 1 -- 172.21.15.3:0/1266104024 shutdown_connections 2023-10-03T10:12:48.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.313+0000 7fec7b8c4700 1 --2- 172.21.15.3:0/1266104024 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fec7409e760 0x7fec7413a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.313+0000 7fec7b8c4700 1 --2- 172.21.15.3:0/1266104024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec741338a0 0x7fec74133c70 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.313+0000 7fec7b8c4700 1 --2- 172.21.15.3:0/1266104024 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fec7409de50 0x7fec7409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.313+0000 7fec7b8c4700 1 -- 172.21.15.3:0/1266104024 >> 172.21.15.3:0/1266104024 conn(0x7fec740a2aa0 msgr2=0x7fec740a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:48.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.313+0000 7fec7b8c4700 1 -- 172.21.15.3:0/1266104024 shutdown_connections 2023-10-03T10:12:48.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.313+0000 7fec7b8c4700 1 -- 172.21.15.3:0/1266104024 wait complete. 2023-10-03T10:12:48.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.314+0000 7fec7b8c4700 1 Processor -- start 2023-10-03T10:12:48.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.314+0000 7fec7b8c4700 1 -- start start 2023-10-03T10:12:48.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.315+0000 7fec7b8c4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec7409de50 0x7fec7411b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:48.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.315+0000 7fec7b8c4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fec7409e760 0x7fec7411ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:48.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.315+0000 7fec7b8c4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fec741338a0 0x7fec7411bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:48.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.315+0000 7fec7b8c4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7411d740 con 0x7fec7409de50 2023-10-03T10:12:48.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.315+0000 7fec7b8c4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7411e800 con 0x7fec7409e760 2023-10-03T10:12:48.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.315+0000 7fec7b8c4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fec7406f640 con 0x7fec741338a0 2023-10-03T10:12:48.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.315+0000 7fec79660700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec7409de50 0x7fec7411b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:48.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.315+0000 7fec79660700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec7409de50 0x7fec7411b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55878/0 (socket says 172.21.15.3:55878) 2023-10-03T10:12:48.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.315+0000 7fec79660700 1 -- 172.21.15.3:0/925339675 learned_addr learned my addr 172.21.15.3:0/925339675 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:48.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.315+0000 7fec78e5f700 1 --2- 172.21.15.3:0/925339675 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fec7409e760 0x7fec7411ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:48.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.315+0000 7fec79e61700 1 --2- 172.21.15.3:0/925339675 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fec741338a0 0x7fec7411bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:48.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.315+0000 7fec79660700 1 -- 172.21.15.3:0/925339675 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fec741338a0 msgr2=0x7fec7411bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:48.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.315+0000 7fec79660700 1 --2- 172.21.15.3:0/925339675 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fec741338a0 0x7fec7411bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.315+0000 7fec79660700 1 -- 172.21.15.3:0/925339675 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fec7409e760 msgr2=0x7fec7411ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:48.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.315+0000 7fec79660700 1 --2- 172.21.15.3:0/925339675 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fec7409e760 0x7fec7411ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.315+0000 7fec79660700 1 -- 172.21.15.3:0/925339675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6c009a70 con 0x7fec7409de50 2023-10-03T10:12:48.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.316+0000 7fec79660700 1 --2- 172.21.15.3:0/925339675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec7409de50 0x7fec7411b550 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fec70009c40 tx=0x7fec70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:48.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.316+0000 7fec69ffb700 1 -- 172.21.15.3:0/925339675 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec700044d0 con 0x7fec7409de50 2023-10-03T10:12:48.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.316+0000 7fec69ffb700 1 -- 172.21.15.3:0/925339675 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7001f070 con 0x7fec7409de50 2023-10-03T10:12:48.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.316+0000 7fec7b8c4700 1 -- 172.21.15.3:0/925339675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec70009550 con 0x7fec7409de50 2023-10-03T10:12:48.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.316+0000 7fec69ffb700 1 -- 172.21.15.3:0/925339675 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec700038e0 con 0x7fec7409de50 2023-10-03T10:12:48.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.316+0000 7fec7b8c4700 1 -- 172.21.15.3:0/925339675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec7411d160 con 0x7fec7409de50 2023-10-03T10:12:48.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.317+0000 7fec69ffb700 1 -- 172.21.15.3:0/925339675 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fec70003a40 con 0x7fec7409de50 2023-10-03T10:12:48.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.317+0000 7fec69ffb700 1 -- 172.21.15.3:0/925339675 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7fec70021070 con 0x7fec7409de50 2023-10-03T10:12:48.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.317+0000 7fec78e5f700 1 -- 172.21.15.3:0/925339675 >> v1:172.21.15.3:6816/127025 conn(0x7fec5802f510 legacy=0x7fec580319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:48.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.318+0000 7fec7b8c4700 1 -- 172.21.15.3:0/925339675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec7411ebf0 con 0x7fec7409de50 2023-10-03T10:12:48.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.327+0000 7fec69ffb700 1 -- 172.21.15.3:0/925339675 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fec7411ebf0 con 0x7fec7409de50 2023-10-03T10:12:48.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.346+0000 7f783a3df700 1 Processor -- start 2023-10-03T10:12:48.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.346+0000 7f783a3df700 1 -- start start 2023-10-03T10:12:48.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.346+0000 7f783a3df700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f783409ac90 0x7f783413cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:48.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.346+0000 7f783a3df700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f783413d240 0x7f783413f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:48.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.346+0000 7f783a3df700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7834134980 con 0x7f783406c0e0 2023-10-03T10:12:48.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.346+0000 7f783a3df700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f783406c4b0 con 0x7f783409ac90 2023-10-03T10:12:48.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.346+0000 7f783a3df700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f783409b0e0 con 0x7f783413d240 2023-10-03T10:12:48.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.347+0000 7f783897c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f783413d240 0x7f783413f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:48.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.347+0000 7f78337fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f783409ac90 0x7f783413cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:48.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.347+0000 7f7833fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f783406c0e0 0x7f7834134ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50684/0 (socket says 172.21.15.3:50684) 2023-10-03T10:12:48.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.347+0000 7f7833fff700 1 -- 172.21.15.3:0/4070792861 learned_addr learned my addr 172.21.15.3:0/4070792861 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:48.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.347+0000 7f78337fe700 1 -- 172.21.15.3:0/4070792861 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f783413d240 msgr2=0x7f783413f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:48.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.347+0000 7f78337fe700 1 --2- 172.21.15.3:0/4070792861 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f783413d240 0x7f783413f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.347+0000 7f78337fe700 1 -- 172.21.15.3:0/4070792861 >> v1:172.21.15.3:6789/0 conn(0x7f783406c0e0 legacy=0x7f7834134ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:48.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.347+0000 7f78337fe700 1 -- 172.21.15.3:0/4070792861 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78340a0600 con 0x7f783409ac90 2023-10-03T10:12:48.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.347+0000 7f7832ffd700 1 -- 172.21.15.3:0/4070792861 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3559085802 0 0) 0x7f7834134980 con 0x7f783406c0e0 2023-10-03T10:12:48.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.347+0000 7f78337fe700 1 --2- 172.21.15.3:0/4070792861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f783409ac90 0x7f783413cd00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7828009840 tx=0x7f7828009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61cade1e144f3b3a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:48.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.347+0000 7f7832ffd700 1 -- 172.21.15.3:0/4070792861 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7828011830 con 0x7f783409ac90 2023-10-03T10:12:48.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.348+0000 7f7832ffd700 1 -- 172.21.15.3:0/4070792861 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7828011990 con 0x7f783409ac90 2023-10-03T10:12:48.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.348+0000 7f7832ffd700 1 -- 172.21.15.3:0/4070792861 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7828011d20 con 0x7f783409ac90 2023-10-03T10:12:48.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.348+0000 7f783a3df700 1 -- 172.21.15.3:0/4070792861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f783409ac90 msgr2=0x7f783413cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:48.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.348+0000 7f783a3df700 1 --2- 172.21.15.3:0/4070792861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f783409ac90 0x7f783413cd00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7828009840 tx=0x7f7828009b50 comp rx=0 tx=0).stop 2023-10-03T10:12:48.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.348+0000 7f783a3df700 1 -- 172.21.15.3:0/4070792861 shutdown_connections 2023-10-03T10:12:48.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.348+0000 7f783a3df700 1 --2- 172.21.15.3:0/4070792861 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f783413d240 0x7f783413f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.348+0000 7f783a3df700 1 --2- 172.21.15.3:0/4070792861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f783409ac90 0x7f783413cd00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.348+0000 7f783a3df700 1 -- 172.21.15.3:0/4070792861 >> 172.21.15.3:0/4070792861 conn(0x7f78340a4210 msgr2=0x7f78340a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:48.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.348+0000 7f783a3df700 1 -- 172.21.15.3:0/4070792861 shutdown_connections 2023-10-03T10:12:48.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.348+0000 7f783a3df700 1 -- 172.21.15.3:0/4070792861 wait complete. 2023-10-03T10:12:48.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.350+0000 7f783a3df700 1 Processor -- start 2023-10-03T10:12:48.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.350+0000 7f783a3df700 1 -- start start 2023-10-03T10:12:48.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.351+0000 7f783a3df700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f783406c0e0 0x7f783409b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:48.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.351+0000 7f783a3df700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f783409ac90 0x7f783409b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:48.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.351+0000 7f783a3df700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f783413d240 0x7f783409beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:48.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.351+0000 7f783a3df700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783413c990 con 0x7f783406c0e0 2023-10-03T10:12:48.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.351+0000 7f783a3df700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f783413f630 con 0x7f783413d240 2023-10-03T10:12:48.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.351+0000 7f783a3df700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f783409de40 con 0x7f783409ac90 2023-10-03T10:12:48.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.351+0000 7f7833fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f783406c0e0 0x7f783409b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:48.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.351+0000 7f7833fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f783406c0e0 0x7f783409b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55908/0 (socket says 172.21.15.3:55908) 2023-10-03T10:12:48.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.351+0000 7f7833fff700 1 -- 172.21.15.3:0/231659338 learned_addr learned my addr 172.21.15.3:0/231659338 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:48.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.351+0000 7f783897c700 1 --2- 172.21.15.3:0/231659338 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f783413d240 0x7f783409beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:48.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.351+0000 7f78337fe700 1 --2- 172.21.15.3:0/231659338 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f783409ac90 0x7f783409b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:48.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.351+0000 7f783897c700 1 -- 172.21.15.3:0/231659338 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f783409ac90 msgr2=0x7f783409b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:48.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.351+0000 7f783897c700 1 --2- 172.21.15.3:0/231659338 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f783409ac90 0x7f783409b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.351+0000 7f783897c700 1 -- 172.21.15.3:0/231659338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f783406c0e0 msgr2=0x7f783409b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:48.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.351+0000 7f783897c700 1 --2- 172.21.15.3:0/231659338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f783406c0e0 0x7f783409b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.351+0000 7f783897c700 1 -- 172.21.15.3:0/231659338 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78280094a0 con 0x7f783413d240 2023-10-03T10:12:48.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.352+0000 7f783897c700 1 --2- 172.21.15.3:0/231659338 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f783413d240 0x7f783409beb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f782c00efc0 tx=0x7f782c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:48.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.352+0000 7f7830ff9700 1 -- 172.21.15.3:0/231659338 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782c00cc90 con 0x7f783413d240 2023-10-03T10:12:48.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.352+0000 7f7830ff9700 1 -- 172.21.15.3:0/231659338 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782c00f040 con 0x7f783413d240 2023-10-03T10:12:48.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.352+0000 7f7830ff9700 1 -- 172.21.15.3:0/231659338 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782c009e60 con 0x7f783413d240 2023-10-03T10:12:48.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.352+0000 7f783a3df700 1 -- 172.21.15.3:0/231659338 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7834134370 con 0x7f783413d240 2023-10-03T10:12:48.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.352+0000 7f783a3df700 1 -- 172.21.15.3:0/231659338 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7834138f00 con 0x7f783413d240 2023-10-03T10:12:48.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.353+0000 7f7830ff9700 1 -- 172.21.15.3:0/231659338 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f782c014480 con 0x7f783413d240 2023-10-03T10:12:48.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.353+0000 7f7830ff9700 1 -- 172.21.15.3:0/231659338 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f782c040500 con 0x7f783413d240 2023-10-03T10:12:48.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.353+0000 7f7833fff700 1 -- 172.21.15.3:0/231659338 >> v1:172.21.15.3:6816/127025 conn(0x7f782002f510 legacy=0x7f78200319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:48.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.353+0000 7f783a3df700 1 -- 172.21.15.3:0/231659338 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f783409df80 con 0x7f783413d240 2023-10-03T10:12:48.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.359+0000 7f7830ff9700 1 -- 172.21.15.3:0/231659338 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f783409df80 con 0x7f783413d240 2023-10-03T10:12:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.395+0000 7f9bd36bc700 1 Processor -- start 2023-10-03T10:12:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.395+0000 7f9bd36bc700 1 -- start start 2023-10-03T10:12:48.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.396+0000 7f9bd36bc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9bcc09e1d0 0x7f9bcc138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:48.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.396+0000 7f9bd36bc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9bcc139170 0x7f9bcc13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:48.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.396+0000 7f9bd36bc700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bcc09b120 con 0x7f9bcc09d6f0 2023-10-03T10:12:48.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.396+0000 7f9bd36bc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bcc06b460 con 0x7f9bcc09e1d0 2023-10-03T10:12:48.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.396+0000 7f9bd36bc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc09a7e0 con 0x7f9bcc139170 2023-10-03T10:12:48.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.396+0000 7f9bd0c57700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9bcc09e1d0 0x7f9bcc138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:48.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.396+0000 7f9bd0c57700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9bcc09e1d0 0x7f9bcc138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55918/0 (socket says 172.21.15.3:55918) 2023-10-03T10:12:48.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.396+0000 7f9bd0c57700 1 -- 172.21.15.3:0/228510657 learned_addr learned my addr 172.21.15.3:0/228510657 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:48.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.396+0000 7f9bd1458700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9bcc09d6f0 0x7f9bcc09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50686/0 (socket says 172.21.15.3:50686) 2023-10-03T10:12:48.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.396+0000 7f9bd0c57700 1 -- 172.21.15.3:0/228510657 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9bcc139170 msgr2=0x7f9bcc13b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:48.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.396+0000 7f9bd1c59700 1 --2- 172.21.15.3:0/228510657 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9bcc139170 0x7f9bcc13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:48.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.396+0000 7f9bd0c57700 1 --2- 172.21.15.3:0/228510657 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9bcc139170 0x7f9bcc13b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.396+0000 7f9bc3fff700 1 -- 172.21.15.3:0/228510657 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 327611727 0 0) 0x7f9bcc09b120 con 0x7f9bcc09d6f0 2023-10-03T10:12:48.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.396+0000 7f9bd0c57700 1 -- 172.21.15.3:0/228510657 >> v1:172.21.15.3:6789/0 conn(0x7f9bcc09d6f0 legacy=0x7f9bcc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:48.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.396+0000 7f9bd0c57700 1 -- 172.21.15.3:0/228510657 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bcc09a500 con 0x7f9bcc09e1d0 2023-10-03T10:12:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.397+0000 7f9bd1c59700 1 --2- 172.21.15.3:0/228510657 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9bcc139170 0x7f9bcc13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:12:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.397+0000 7f9bd0c57700 1 --2- 172.21.15.3:0/228510657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9bcc09e1d0 0x7f9bcc138c30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc009920 tx=0x7f9bbc009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3ed0a5baee639 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.397+0000 7f9bc3fff700 1 -- 172.21.15.3:0/228510657 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc00ba20 con 0x7f9bcc09e1d0 2023-10-03T10:12:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.397+0000 7f9bc3fff700 1 -- 172.21.15.3:0/228510657 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bbc00bb80 con 0x7f9bcc09e1d0 2023-10-03T10:12:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.397+0000 7f9bc3fff700 1 -- 172.21.15.3:0/228510657 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc009e70 con 0x7f9bcc09e1d0 2023-10-03T10:12:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.398+0000 7f9bd36bc700 1 -- 172.21.15.3:0/228510657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9bcc09e1d0 msgr2=0x7f9bcc138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.398+0000 7f9bd36bc700 1 --2- 172.21.15.3:0/228510657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9bcc09e1d0 0x7f9bcc138c30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc009920 tx=0x7f9bbc009c30 comp rx=0 tx=0).stop 2023-10-03T10:12:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.398+0000 7f9bd36bc700 1 -- 172.21.15.3:0/228510657 shutdown_connections 2023-10-03T10:12:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.398+0000 7f9bd36bc700 1 --2- 172.21.15.3:0/228510657 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9bcc139170 0x7f9bcc13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.398+0000 7f9bd36bc700 1 --2- 172.21.15.3:0/228510657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9bcc09e1d0 0x7f9bcc138c30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.398+0000 7f9bd36bc700 1 -- 172.21.15.3:0/228510657 >> 172.21.15.3:0/228510657 conn(0x7f9bcc066f50 msgr2=0x7f9bcc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.398+0000 7f9bd36bc700 1 -- 172.21.15.3:0/228510657 shutdown_connections 2023-10-03T10:12:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.398+0000 7f9bd36bc700 1 -- 172.21.15.3:0/228510657 wait complete. 2023-10-03T10:12:48.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.399+0000 7f9bd36bc700 1 Processor -- start 2023-10-03T10:12:48.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.399+0000 7f9bd36bc700 1 -- start start 2023-10-03T10:12:48.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.399+0000 7f9bd36bc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9bcc09d6f0 0x7f9bcc1c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:48.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.399+0000 7f9bd36bc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9bcc09e1d0 0x7f9bcc1c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.399+0000 7f9bd36bc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9bcc139170 0x7f9bcc1c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.399+0000 7f9bd36bc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc123670 con 0x7f9bcc09d6f0 2023-10-03T10:12:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.399+0000 7f9bd36bc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc123c90 con 0x7f9bcc09e1d0 2023-10-03T10:12:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.399+0000 7f9bd36bc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc1252a0 con 0x7f9bcc139170 2023-10-03T10:12:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.399+0000 7f9bd1458700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9bcc09d6f0 0x7f9bcc1c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.399+0000 7f9bd1458700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9bcc09d6f0 0x7f9bcc1c8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55934/0 (socket says 172.21.15.3:55934) 2023-10-03T10:12:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.400+0000 7f9bd1458700 1 -- 172.21.15.3:0/683429103 learned_addr learned my addr 172.21.15.3:0/683429103 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.400+0000 7f9bd0c57700 1 --2- 172.21.15.3:0/683429103 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9bcc09e1d0 0x7f9bcc1c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.400+0000 7f9bd1c59700 1 --2- 172.21.15.3:0/683429103 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9bcc139170 0x7f9bcc1c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.400+0000 7f9bd0c57700 1 -- 172.21.15.3:0/683429103 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9bcc139170 msgr2=0x7f9bcc1c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.400+0000 7f9bd0c57700 1 --2- 172.21.15.3:0/683429103 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9bcc139170 0x7f9bcc1c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.400+0000 7f9bd0c57700 1 -- 172.21.15.3:0/683429103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9bcc09d6f0 msgr2=0x7f9bcc1c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:48.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.400+0000 7f9bd0c57700 1 --2- 172.21.15.3:0/683429103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9bcc09d6f0 0x7f9bcc1c8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.400+0000 7f9bd0c57700 1 -- 172.21.15.3:0/683429103 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bbc009550 con 0x7f9bcc09e1d0 2023-10-03T10:12:48.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.400+0000 7f9bd1458700 1 --2- 172.21.15.3:0/683429103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9bcc09d6f0 0x7f9bcc1c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:48.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.400+0000 7f9bd0c57700 1 --2- 172.21.15.3:0/683429103 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9bcc09e1d0 0x7f9bcc1c88a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc016000 tx=0x7f9bbc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:48.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.400+0000 7f9bc1ffb700 1 -- 172.21.15.3:0/683429103 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc004400 con 0x7f9bcc09e1d0 2023-10-03T10:12:48.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.400+0000 7f9bc1ffb700 1 -- 172.21.15.3:0/683429103 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bbc01f070 con 0x7f9bcc09e1d0 2023-10-03T10:12:48.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.400+0000 7f9bd36bc700 1 -- 172.21.15.3:0/683429103 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bcc0a20b0 con 0x7f9bcc09e1d0 2023-10-03T10:12:48.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.400+0000 7f9bc1ffb700 1 -- 172.21.15.3:0/683429103 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc0038e0 con 0x7f9bcc09e1d0 2023-10-03T10:12:48.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.401+0000 7f9bd36bc700 1 -- 172.21.15.3:0/683429103 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bcc123e60 con 0x7f9bcc09e1d0 2023-10-03T10:12:48.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.401+0000 7f9bc1ffb700 1 -- 172.21.15.3:0/683429103 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f9bbc021030 con 0x7f9bcc09e1d0 2023-10-03T10:12:48.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.402+0000 7f9bc1ffb700 1 -- 172.21.15.3:0/683429103 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f9bbc040f40 con 0x7f9bcc09e1d0 2023-10-03T10:12:48.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.402+0000 7f9bd1458700 1 -- 172.21.15.3:0/683429103 >> v1:172.21.15.3:6816/127025 conn(0x7f9bb002f510 legacy=0x7f9bb00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:48.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.402+0000 7f9bd36bc700 1 -- 172.21.15.3:0/683429103 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bcc074ea0 con 0x7f9bcc09e1d0 2023-10-03T10:12:48.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.407+0000 7f9bc1ffb700 1 -- 172.21.15.3:0/683429103 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9bbc015080 con 0x7f9bcc09e1d0 2023-10-03T10:12:48.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.414+0000 7f8d46376700 1 Processor -- start 2023-10-03T10:12:48.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.414+0000 7f8d46376700 1 -- start start 2023-10-03T10:12:48.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.414+0000 7f8d46376700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d4006e570 0x7f8d4006e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:48.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.414+0000 7f8d46376700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8d4009eca0 0x7f8d4006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:48.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.414+0000 7f8d46376700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d40070ee0 con 0x7f8d40134610 2023-10-03T10:12:48.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.414+0000 7f8d46376700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d40071480 con 0x7f8d4009eca0 2023-10-03T10:12:48.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.414+0000 7f8d46376700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4006f3c0 con 0x7f8d4006e570 2023-10-03T10:12:48.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.415+0000 7f8d44913700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8d40134610 0x7f8d40134a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50698/0 (socket says 172.21.15.3:50698) 2023-10-03T10:12:48.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.415+0000 7f8d44913700 1 -- 172.21.15.3:0/2322973700 learned_addr learned my addr 172.21.15.3:0/2322973700 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:48.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.415+0000 7f8d3ffff700 1 --2- 172.21.15.3:0/2322973700 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d4006e570 0x7f8d4006e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.415+0000 7f8d3f7fe700 1 --2- 172.21.15.3:0/2322973700 >> v2:172.21.15.3:3300/0 conn(0x7f8d4009eca0 0x7f8d4006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.415+0000 7f8d3effd700 1 -- 172.21.15.3:0/2322973700 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1589192249 0 0) 0x7f8d40070ee0 con 0x7f8d40134610 2023-10-03T10:12:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.415+0000 7f8d3effd700 1 -- 172.21.15.3:0/2322973700 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d280035b0 con 0x7f8d40134610 2023-10-03T10:12:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.415+0000 7f8d3ffff700 1 -- 172.21.15.3:0/2322973700 >> v2:172.21.15.3:3300/0 conn(0x7f8d4009eca0 msgr2=0x7f8d4006ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.415+0000 7f8d3ffff700 1 --2- 172.21.15.3:0/2322973700 >> v2:172.21.15.3:3300/0 conn(0x7f8d4009eca0 0x7f8d4006ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.415+0000 7f8d3ffff700 1 -- 172.21.15.3:0/2322973700 >> v1:172.21.15.3:6789/0 conn(0x7f8d40134610 legacy=0x7f8d40134a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.415+0000 7f8d3ffff700 1 -- 172.21.15.3:0/2322973700 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d40071b90 con 0x7f8d4006e570 2023-10-03T10:12:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.415+0000 7f8d3f7fe700 1 --2- 172.21.15.3:0/2322973700 >> v2:172.21.15.3:3300/0 conn(0x7f8d4009eca0 0x7f8d4006ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:12:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.415+0000 7f8d3ffff700 1 --2- 172.21.15.3:0/2322973700 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d4006e570 0x7f8d4006e940 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009840 tx=0x7f8d30009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebd1250f306f275c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.416+0000 7f8d3effd700 1 -- 172.21.15.3:0/2322973700 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30011830 con 0x7f8d4006e570 2023-10-03T10:12:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.416+0000 7f8d3effd700 1 -- 172.21.15.3:0/2322973700 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d30011990 con 0x7f8d4006e570 2023-10-03T10:12:48.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.416+0000 7f8d3effd700 1 -- 172.21.15.3:0/2322973700 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30011d20 con 0x7f8d4006e570 2023-10-03T10:12:48.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.416+0000 7f8d46376700 1 -- 172.21.15.3:0/2322973700 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d4006e570 msgr2=0x7f8d4006e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:48.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.416+0000 7f8d46376700 1 --2- 172.21.15.3:0/2322973700 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d4006e570 0x7f8d4006e940 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009840 tx=0x7f8d30009b50 comp rx=0 tx=0).stop 2023-10-03T10:12:48.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.416+0000 7f8d46376700 1 -- 172.21.15.3:0/2322973700 shutdown_connections 2023-10-03T10:12:48.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.416+0000 7f8d46376700 1 --2- 172.21.15.3:0/2322973700 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d4006e570 0x7f8d4006e940 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.416+0000 7f8d46376700 1 --2- 172.21.15.3:0/2322973700 >> v2:172.21.15.3:3300/0 conn(0x7f8d4009eca0 0x7f8d4006ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.416+0000 7f8d46376700 1 -- 172.21.15.3:0/2322973700 >> 172.21.15.3:0/2322973700 conn(0x7f8d400a2aa0 msgr2=0x7f8d400a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:48.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.417+0000 7f8d46376700 1 -- 172.21.15.3:0/2322973700 shutdown_connections 2023-10-03T10:12:48.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.417+0000 7f8d46376700 1 -- 172.21.15.3:0/2322973700 wait complete. 2023-10-03T10:12:48.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.418+0000 7f8d46376700 1 Processor -- start 2023-10-03T10:12:48.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.418+0000 7f8d46376700 1 -- start start 2023-10-03T10:12:48.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.419+0000 7f8d46376700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d4006e570 0x7f8d401214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:48.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.419+0000 7f8d46376700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d40134610 0x7f8d40121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:48.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.419+0000 7f8d46376700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d4009e310 0x7f8d40121f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:48.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.419+0000 7f8d46376700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d40122f50 con 0x7f8d4006e570 2023-10-03T10:12:48.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.419+0000 7f8d46376700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d40124010 con 0x7f8d40134610 2023-10-03T10:12:48.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.419+0000 7f8d46376700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8d40070610 con 0x7f8d4009e310 2023-10-03T10:12:48.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.419+0000 7f8d3ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d4006e570 0x7f8d401214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:48.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.419+0000 7f8d3ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d4006e570 0x7f8d401214e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55952/0 (socket says 172.21.15.3:55952) 2023-10-03T10:12:48.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.419+0000 7f8d3ffff700 1 -- 172.21.15.3:0/1037310283 learned_addr learned my addr 172.21.15.3:0/1037310283 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:48.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.419+0000 7f8d44913700 1 --2- 172.21.15.3:0/1037310283 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d4009e310 0x7f8d40121f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:48.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.420+0000 7f8d3f7fe700 1 --2- 172.21.15.3:0/1037310283 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d40134610 0x7f8d40121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:48.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.420+0000 7f8d3ffff700 1 -- 172.21.15.3:0/1037310283 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d4009e310 msgr2=0x7f8d40121f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:48.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.420+0000 7f8d3ffff700 1 --2- 172.21.15.3:0/1037310283 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d4009e310 0x7f8d40121f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.420+0000 7f8d3ffff700 1 -- 172.21.15.3:0/1037310283 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d40134610 msgr2=0x7f8d40121a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:48.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.420+0000 7f8d3ffff700 1 --2- 172.21.15.3:0/1037310283 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d40134610 0x7f8d40121a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.420+0000 7f8d3ffff700 1 -- 172.21.15.3:0/1037310283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d300094a0 con 0x7f8d4006e570 2023-10-03T10:12:48.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.420+0000 7f8d3ffff700 1 --2- 172.21.15.3:0/1037310283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d4006e570 0x7f8d401214e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009fd0 tx=0x7f8d30004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:48.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.420+0000 7f8d3cff9700 1 -- 172.21.15.3:0/1037310283 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d300036a0 con 0x7f8d4006e570 2023-10-03T10:12:48.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.420+0000 7f8d3cff9700 1 -- 172.21.15.3:0/1037310283 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3001f070 con 0x7f8d4006e570 2023-10-03T10:12:48.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.420+0000 7f8d3cff9700 1 -- 172.21.15.3:0/1037310283 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30003a40 con 0x7f8d4006e570 2023-10-03T10:12:48.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.420+0000 7f8d46376700 1 -- 172.21.15.3:0/1037310283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d400a1e80 con 0x7f8d4006e570 2023-10-03T10:12:48.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.420+0000 7f8d46376700 1 -- 172.21.15.3:0/1037310283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d40123bc0 con 0x7f8d4006e570 2023-10-03T10:12:48.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.422+0000 7f8d3cff9700 1 -- 172.21.15.3:0/1037310283 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f8d30003ba0 con 0x7f8d4006e570 2023-10-03T10:12:48.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.422+0000 7f8d46376700 1 -- 172.21.15.3:0/1037310283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d40124240 con 0x7f8d4006e570 2023-10-03T10:12:48.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.422+0000 7f8d3cff9700 1 -- 172.21.15.3:0/1037310283 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f8d30021070 con 0x7f8d4006e570 2023-10-03T10:12:48.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.424+0000 7f8d3f7fe700 1 -- 172.21.15.3:0/1037310283 >> v1:172.21.15.3:6816/127025 conn(0x7f8d2c02f510 legacy=0x7f8d2c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:48.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.427+0000 7f8d3cff9700 1 -- 172.21.15.3:0/1037310283 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8d40124240 con 0x7f8d4006e570 2023-10-03T10:12:48.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.519+0000 7fec78e5f700 1 -- 172.21.15.3:0/925339675 >> v1:172.21.15.3:6816/127025 conn(0x7fec5802f510 legacy=0x7fec580319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:48.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.554+0000 7f7833fff700 1 -- 172.21.15.3:0/231659338 >> v1:172.21.15.3:6816/127025 conn(0x7f782002f510 legacy=0x7f78200319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:48.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.572+0000 7f783a3df700 1 -- 172.21.15.3:0/231659338 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f783409df80 con 0x7f783413d240 2023-10-03T10:12:48.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.572+0000 7f7830ff9700 1 -- 172.21.15.3:0/231659338 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f783409df80 con 0x7f783413d240 2023-10-03T10:12:48.572 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:12:48.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.574+0000 7f783a3df700 1 -- 172.21.15.3:0/231659338 >> v1:172.21.15.3:6816/127025 conn(0x7f782002f510 legacy=0x7f78200319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:48.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.574+0000 7f783a3df700 1 -- 172.21.15.3:0/231659338 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f783413d240 msgr2=0x7f783409beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:48.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.574+0000 7f783a3df700 1 --2- 172.21.15.3:0/231659338 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f783413d240 0x7f783409beb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f782c00efc0 tx=0x7f782c00c3a0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.575+0000 7f783a3df700 1 -- 172.21.15.3:0/231659338 shutdown_connections 2023-10-03T10:12:48.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.575+0000 7f783a3df700 1 --2- 172.21.15.3:0/231659338 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f783413d240 0x7f783409beb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.575+0000 7f783a3df700 1 --2- 172.21.15.3:0/231659338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f783406c0e0 0x7f783409b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.575+0000 7f783a3df700 1 --2- 172.21.15.3:0/231659338 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f783409ac90 0x7f783409b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.575+0000 7f783a3df700 1 -- 172.21.15.3:0/231659338 >> 172.21.15.3:0/231659338 conn(0x7f78340a4210 msgr2=0x7f78340a0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:48.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.575+0000 7f783a3df700 1 -- 172.21.15.3:0/231659338 shutdown_connections 2023-10-03T10:12:48.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.575+0000 7f783a3df700 1 -- 172.21.15.3:0/231659338 wait complete. 2023-10-03T10:12:48.589 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:12:48.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.602+0000 7fec7b8c4700 1 -- 172.21.15.3:0/925339675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fec7411ebf0 con 0x7fec7409de50 2023-10-03T10:12:48.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.602+0000 7fec69ffb700 1 -- 172.21.15.3:0/925339675 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fec7411ebf0 con 0x7fec7409de50 2023-10-03T10:12:48.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.603+0000 7f9bd1458700 1 -- 172.21.15.3:0/683429103 >> v1:172.21.15.3:6816/127025 conn(0x7f9bb002f510 legacy=0x7f9bb00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:48.603 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:12:48.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.605+0000 7fec7b8c4700 1 -- 172.21.15.3:0/925339675 >> v1:172.21.15.3:6816/127025 conn(0x7fec5802f510 legacy=0x7fec580319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:48.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.605+0000 7fec7b8c4700 1 -- 172.21.15.3:0/925339675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec7409de50 msgr2=0x7fec7411b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:48.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.605+0000 7fec7b8c4700 1 --2- 172.21.15.3:0/925339675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec7409de50 0x7fec7411b550 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fec70009c40 tx=0x7fec70004750 comp rx=0 tx=0).stop 2023-10-03T10:12:48.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.605+0000 7fec7b8c4700 1 -- 172.21.15.3:0/925339675 shutdown_connections 2023-10-03T10:12:48.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.605+0000 7fec7b8c4700 1 --2- 172.21.15.3:0/925339675 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fec741338a0 0x7fec7411bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.605+0000 7fec7b8c4700 1 --2- 172.21.15.3:0/925339675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec7409de50 0x7fec7411b550 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.605+0000 7fec7b8c4700 1 --2- 172.21.15.3:0/925339675 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fec7409e760 0x7fec7411ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.605+0000 7fec7b8c4700 1 -- 172.21.15.3:0/925339675 >> 172.21.15.3:0/925339675 conn(0x7fec740a2aa0 msgr2=0x7fec74130510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:48.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.605+0000 7fec7b8c4700 1 -- 172.21.15.3:0/925339675 shutdown_connections 2023-10-03T10:12:48.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.605+0000 7fec7b8c4700 1 -- 172.21.15.3:0/925339675 wait complete. 2023-10-03T10:12:48.619 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:12:48.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.619+0000 7f9bd36bc700 1 -- 172.21.15.3:0/683429103 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9bcc09f420 con 0x7f9bcc09e1d0 2023-10-03T10:12:48.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.620+0000 7f9bc1ffb700 1 -- 172.21.15.3:0/683429103 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9bbc0199a0 con 0x7f9bcc09e1d0 2023-10-03T10:12:48.620 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:12:48.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.622+0000 7f9bd36bc700 1 -- 172.21.15.3:0/683429103 >> v1:172.21.15.3:6816/127025 conn(0x7f9bb002f510 legacy=0x7f9bb00319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:48.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.622+0000 7f9bd36bc700 1 -- 172.21.15.3:0/683429103 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9bcc09e1d0 msgr2=0x7f9bcc1c88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:48.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.622+0000 7f9bd36bc700 1 --2- 172.21.15.3:0/683429103 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9bcc09e1d0 0x7f9bcc1c88a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc016000 tx=0x7f9bbc004750 comp rx=0 tx=0).stop 2023-10-03T10:12:48.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.622+0000 7f9bd36bc700 1 -- 172.21.15.3:0/683429103 shutdown_connections 2023-10-03T10:12:48.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.622+0000 7f9bd36bc700 1 --2- 172.21.15.3:0/683429103 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9bcc139170 0x7f9bcc1c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.622+0000 7f9bd36bc700 1 --2- 172.21.15.3:0/683429103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9bcc09d6f0 0x7f9bcc1c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.622+0000 7f9bd36bc700 1 --2- 172.21.15.3:0/683429103 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9bcc09e1d0 0x7f9bcc1c88a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.622+0000 7f9bd36bc700 1 -- 172.21.15.3:0/683429103 >> 172.21.15.3:0/683429103 conn(0x7f9bcc066f50 msgr2=0x7f9bcc070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:48.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.622+0000 7f9bd36bc700 1 -- 172.21.15.3:0/683429103 shutdown_connections 2023-10-03T10:12:48.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.622+0000 7f9bd36bc700 1 -- 172.21.15.3:0/683429103 wait complete. 2023-10-03T10:12:48.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.625+0000 7f8d3f7fe700 1 -- 172.21.15.3:0/1037310283 >> v1:172.21.15.3:6816/127025 conn(0x7f8d2c02f510 legacy=0x7f8d2c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:48.638 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:12:48.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.658+0000 7f8d46376700 1 -- 172.21.15.3:0/1037310283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8d4009b1e0 con 0x7f8d4006e570 2023-10-03T10:12:48.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.659+0000 7f8d3cff9700 1 -- 172.21.15.3:0/1037310283 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8d3002b020 con 0x7f8d4006e570 2023-10-03T10:12:48.659 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:12:48.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.661+0000 7f8d46376700 1 -- 172.21.15.3:0/1037310283 >> v1:172.21.15.3:6816/127025 conn(0x7f8d2c02f510 legacy=0x7f8d2c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:48.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.661+0000 7f8d46376700 1 -- 172.21.15.3:0/1037310283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d4006e570 msgr2=0x7f8d401214e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:48.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.661+0000 7f8d46376700 1 --2- 172.21.15.3:0/1037310283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d4006e570 0x7f8d401214e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009fd0 tx=0x7f8d30004750 comp rx=0 tx=0).stop 2023-10-03T10:12:48.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.661+0000 7f8d46376700 1 -- 172.21.15.3:0/1037310283 shutdown_connections 2023-10-03T10:12:48.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.661+0000 7f8d46376700 1 --2- 172.21.15.3:0/1037310283 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d4009e310 0x7f8d40121f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.661+0000 7f8d46376700 1 --2- 172.21.15.3:0/1037310283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d4006e570 0x7f8d401214e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.661+0000 7f8d46376700 1 --2- 172.21.15.3:0/1037310283 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d40134610 0x7f8d40121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:48.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.661+0000 7f8d46376700 1 -- 172.21.15.3:0/1037310283 >> 172.21.15.3:0/1037310283 conn(0x7f8d400a2aa0 msgr2=0x7f8d401bf080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:48.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.661+0000 7f8d46376700 1 -- 172.21.15.3:0/1037310283 shutdown_connections 2023-10-03T10:12:48.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:48.661+0000 7f8d46376700 1 -- 172.21.15.3:0/1037310283 wait complete. 2023-10-03T10:12:48.675 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:12:49.590 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:12:49.620 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:12:49.639 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:12:49.676 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:12:49.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.754+0000 7f38b9b10700 1 Processor -- start 2023-10-03T10:12:49.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.754+0000 7f38b9b10700 1 -- start start 2023-10-03T10:12:49.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.755+0000 7f38b9b10700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38b409e930 0x7f38b409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:49.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.755+0000 7f38b9b10700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f38b409f650 0x7f38b4131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:49.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.755+0000 7f38b9b10700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b4070940 con 0x7f38b409f240 2023-10-03T10:12:49.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.755+0000 7f38b9b10700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b406fd60 con 0x7f38b409f650 2023-10-03T10:12:49.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.755+0000 7f38b9b10700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b406fa80 con 0x7f38b409e930 2023-10-03T10:12:49.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.755+0000 7f38b2ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f38b409f240 0x7f38b41317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50700/0 (socket says 172.21.15.3:50700) 2023-10-03T10:12:49.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.755+0000 7f38b2ffd700 1 -- 172.21.15.3:0/3313327631 learned_addr learned my addr 172.21.15.3:0/3313327631 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:49.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.755+0000 7f38b37fe700 1 --2- 172.21.15.3:0/3313327631 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38b409e930 0x7f38b409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:49.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.755+0000 7f38b3fff700 1 --2- 172.21.15.3:0/3313327631 >> v2:172.21.15.3:3300/0 conn(0x7f38b409f650 0x7f38b4131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:49.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.755+0000 7f38b37fe700 1 -- 172.21.15.3:0/3313327631 >> v2:172.21.15.3:3300/0 conn(0x7f38b409f650 msgr2=0x7f38b4131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:49.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.755+0000 7f38b37fe700 1 --2- 172.21.15.3:0/3313327631 >> v2:172.21.15.3:3300/0 conn(0x7f38b409f650 0x7f38b4131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:49.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.755+0000 7f38b37fe700 1 -- 172.21.15.3:0/3313327631 >> v1:172.21.15.3:6789/0 conn(0x7f38b409f240 legacy=0x7f38b41317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:49.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.755+0000 7f38b37fe700 1 -- 172.21.15.3:0/3313327631 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b409fb60 con 0x7f38b409e930 2023-10-03T10:12:49.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.756+0000 7f38b37fe700 1 --2- 172.21.15.3:0/3313327631 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38b409e930 0x7f38b409ed00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f38a4009840 tx=0x7f38a4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6374f7c59ad2c1f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:49.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.756+0000 7f38b27fc700 1 -- 172.21.15.3:0/3313327631 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a4011830 con 0x7f38b409e930 2023-10-03T10:12:49.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.756+0000 7f38b27fc700 1 -- 172.21.15.3:0/3313327631 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a4011990 con 0x7f38b409e930 2023-10-03T10:12:49.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.756+0000 7f38b27fc700 1 -- 172.21.15.3:0/3313327631 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a4011d20 con 0x7f38b409e930 2023-10-03T10:12:49.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.757+0000 7f38b9b10700 1 -- 172.21.15.3:0/3313327631 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38b409e930 msgr2=0x7f38b409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:49.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.757+0000 7f38b9b10700 1 --2- 172.21.15.3:0/3313327631 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38b409e930 0x7f38b409ed00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f38a4009840 tx=0x7f38a4009b50 comp rx=0 tx=0).stop 2023-10-03T10:12:49.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.757+0000 7f38b9b10700 1 -- 172.21.15.3:0/3313327631 shutdown_connections 2023-10-03T10:12:49.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.757+0000 7f38b9b10700 1 --2- 172.21.15.3:0/3313327631 >> v2:172.21.15.3:3300/0 conn(0x7f38b409f650 0x7f38b4131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:49.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.757+0000 7f38b9b10700 1 --2- 172.21.15.3:0/3313327631 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38b409e930 0x7f38b409ed00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:49.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.757+0000 7f38b9b10700 1 -- 172.21.15.3:0/3313327631 >> 172.21.15.3:0/3313327631 conn(0x7f38b40a4210 msgr2=0x7f38b40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:49.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.757+0000 7f38b9b10700 1 -- 172.21.15.3:0/3313327631 shutdown_connections 2023-10-03T10:12:49.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.757+0000 7f38b9b10700 1 -- 172.21.15.3:0/3313327631 wait complete. 2023-10-03T10:12:49.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.758+0000 7f38b9b10700 1 Processor -- start 2023-10-03T10:12:49.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.758+0000 7f38b9b10700 1 -- start start 2023-10-03T10:12:49.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.759+0000 7f38b9b10700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38b409e930 0x7f38b41c0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:49.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.759+0000 7f38b9b10700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38b409f240 0x7f38b41c1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:49.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.759+0000 7f38b9b10700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38b409f650 0x7f38b41c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:49.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.759+0000 7f38b9b10700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b41a5750 con 0x7f38b409e930 2023-10-03T10:12:49.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.759+0000 7f38b9b10700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b41a53d0 con 0x7f38b409f240 2023-10-03T10:12:49.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.759+0000 7f38b9b10700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38b41c1860 con 0x7f38b409f650 2023-10-03T10:12:49.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.759+0000 7f38b3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38b409f650 0x7f38b41c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:49.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.759+0000 7f38b37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38b409e930 0x7f38b41c0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:49.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.759+0000 7f38b37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38b409e930 0x7f38b41c0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55956/0 (socket says 172.21.15.3:55956) 2023-10-03T10:12:49.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.759+0000 7f38b3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38b409f650 0x7f38b41c3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38628/0 (socket says 172.21.15.3:38628) 2023-10-03T10:12:49.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.759+0000 7f38b37fe700 1 -- 172.21.15.3:0/246141547 learned_addr learned my addr 172.21.15.3:0/246141547 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:49.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.759+0000 7f38b2ffd700 1 --2- 172.21.15.3:0/246141547 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38b409f240 0x7f38b41c1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:49.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.759+0000 7f38b37fe700 1 -- 172.21.15.3:0/246141547 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38b409f650 msgr2=0x7f38b41c3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:49.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.759+0000 7f38b37fe700 1 --2- 172.21.15.3:0/246141547 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38b409f650 0x7f38b41c3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:49.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.759+0000 7f38b37fe700 1 -- 172.21.15.3:0/246141547 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38b409f240 msgr2=0x7f38b41c1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:49.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.759+0000 7f38b37fe700 1 --2- 172.21.15.3:0/246141547 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38b409f240 0x7f38b41c1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:49.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.759+0000 7f38b37fe700 1 -- 172.21.15.3:0/246141547 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a40094a0 con 0x7f38b409e930 2023-10-03T10:12:49.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.760+0000 7f38b37fe700 1 --2- 172.21.15.3:0/246141547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38b409e930 0x7f38b41c0de0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f38a40090b0 tx=0x7f38a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:49.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.760+0000 7f389bfff700 1 -- 172.21.15.3:0/246141547 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a40036a0 con 0x7f38b409e930 2023-10-03T10:12:49.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.760+0000 7f38b9b10700 1 -- 172.21.15.3:0/246141547 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b40a0190 con 0x7f38b409e930 2023-10-03T10:12:49.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.760+0000 7f389bfff700 1 -- 172.21.15.3:0/246141547 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a4027070 con 0x7f38b409e930 2023-10-03T10:12:49.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.760+0000 7f389bfff700 1 -- 172.21.15.3:0/246141547 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a4003a60 con 0x7f38b409e930 2023-10-03T10:12:49.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.760+0000 7f38b9b10700 1 -- 172.21.15.3:0/246141547 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b4071d20 con 0x7f38b409e930 2023-10-03T10:12:49.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.762+0000 7f38b9b10700 1 -- 172.21.15.3:0/246141547 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b40a3530 con 0x7f38b409e930 2023-10-03T10:12:49.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.762+0000 7f389bfff700 1 -- 172.21.15.3:0/246141547 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f38a4003bc0 con 0x7f38b409e930 2023-10-03T10:12:49.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.762+0000 7f389bfff700 1 -- 172.21.15.3:0/246141547 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f38a4029070 con 0x7f38b409e930 2023-10-03T10:12:49.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.762+0000 7f38b2ffd700 1 -- 172.21.15.3:0/246141547 >> v1:172.21.15.3:6816/127025 conn(0x7f38a002f510 legacy=0x7f38a00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:49.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.771+0000 7f389bfff700 1 -- 172.21.15.3:0/246141547 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f38a4037d20 con 0x7f38b409e930 2023-10-03T10:12:49.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.796+0000 7f35187cb700 1 Processor -- start 2023-10-03T10:12:49.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.796+0000 7f35187cb700 1 -- start start 2023-10-03T10:12:49.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.797+0000 7f35187cb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f351009e930 0x7f351009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:49.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.797+0000 7f35187cb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f351009f240 0x7f35101317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:49.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.797+0000 7f35187cb700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3510070940 con 0x7f351009f650 2023-10-03T10:12:49.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.797+0000 7f35187cb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f351006fd60 con 0x7f351009f240 2023-10-03T10:12:49.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.797+0000 7f35187cb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f351006fa80 con 0x7f351009e930 2023-10-03T10:12:49.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.797+0000 7f3516567700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f351009e930 0x7f351009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:49.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.797+0000 7f3516d68700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f351009f650 0x7f3510131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50716/0 (socket says 172.21.15.3:50716) 2023-10-03T10:12:49.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.797+0000 7f3516d68700 1 -- 172.21.15.3:0/4010413677 learned_addr learned my addr 172.21.15.3:0/4010413677 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:49.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.797+0000 7f3515d66700 1 --2- 172.21.15.3:0/4010413677 >> v2:172.21.15.3:3300/0 conn(0x7f351009f240 0x7f35101317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:49.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.797+0000 7f3516567700 1 -- 172.21.15.3:0/4010413677 >> v2:172.21.15.3:3300/0 conn(0x7f351009f240 msgr2=0x7f35101317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:49.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.797+0000 7f3516567700 1 --2- 172.21.15.3:0/4010413677 >> v2:172.21.15.3:3300/0 conn(0x7f351009f240 0x7f35101317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:49.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.797+0000 7f3516567700 1 -- 172.21.15.3:0/4010413677 >> v1:172.21.15.3:6789/0 conn(0x7f351009f650 legacy=0x7f3510131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:49.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.798+0000 7f3515565700 1 -- 172.21.15.3:0/4010413677 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 876348520 0 0) 0x7f3510070940 con 0x7f351009f650 2023-10-03T10:12:49.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.798+0000 7f3516567700 1 -- 172.21.15.3:0/4010413677 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f351009fb60 con 0x7f351009e930 2023-10-03T10:12:49.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.798+0000 7f3516567700 1 --2- 172.21.15.3:0/4010413677 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f351009e930 0x7f351009ed00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f350c009840 tx=0x7f350c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77bc1a355dcab707 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:49.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.798+0000 7f3515565700 1 -- 172.21.15.3:0/4010413677 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c011830 con 0x7f351009e930 2023-10-03T10:12:49.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.798+0000 7f3515565700 1 -- 172.21.15.3:0/4010413677 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350c011990 con 0x7f351009e930 2023-10-03T10:12:49.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.798+0000 7f3515565700 1 -- 172.21.15.3:0/4010413677 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c011d20 con 0x7f351009e930 2023-10-03T10:12:49.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.799+0000 7f35187cb700 1 -- 172.21.15.3:0/4010413677 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f351009e930 msgr2=0x7f351009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:49.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.799+0000 7f35187cb700 1 --2- 172.21.15.3:0/4010413677 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f351009e930 0x7f351009ed00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f350c009840 tx=0x7f350c009b50 comp rx=0 tx=0).stop 2023-10-03T10:12:49.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.799+0000 7f35187cb700 1 -- 172.21.15.3:0/4010413677 shutdown_connections 2023-10-03T10:12:49.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.799+0000 7f35187cb700 1 --2- 172.21.15.3:0/4010413677 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f351009e930 0x7f351009ed00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:49.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.799+0000 7f35187cb700 1 --2- 172.21.15.3:0/4010413677 >> v2:172.21.15.3:3300/0 conn(0x7f351009f240 0x7f35101317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:49.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.799+0000 7f35187cb700 1 -- 172.21.15.3:0/4010413677 >> 172.21.15.3:0/4010413677 conn(0x7f35100a4210 msgr2=0x7f35100a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:49.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.799+0000 7f35187cb700 1 -- 172.21.15.3:0/4010413677 shutdown_connections 2023-10-03T10:12:49.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.799+0000 7f35187cb700 1 -- 172.21.15.3:0/4010413677 wait complete. 2023-10-03T10:12:49.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.800+0000 7f35187cb700 1 Processor -- start 2023-10-03T10:12:49.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.800+0000 7f35187cb700 1 -- start start 2023-10-03T10:12:49.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.801+0000 7f35187cb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f351009e930 0x7f35101a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:49.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.801+0000 7f35187cb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f351009f240 0x7f35101a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:49.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.801+0000 7f35187cb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f351009f650 0x7f35101c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:49.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.801+0000 7f35187cb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35101a2830 con 0x7f351009f650 2023-10-03T10:12:49.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.801+0000 7f35187cb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35101a1a40 con 0x7f351009e930 2023-10-03T10:12:49.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.801+0000 7f35187cb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f35101b4f60 con 0x7f351009f240 2023-10-03T10:12:49.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.801+0000 7f3516567700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f351009e930 0x7f35101a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:49.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.801+0000 7f3516567700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f351009e930 0x7f35101a0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60066/0 (socket says 172.21.15.3:60066) 2023-10-03T10:12:49.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.801+0000 7f3516567700 1 -- 172.21.15.3:0/549145209 learned_addr learned my addr 172.21.15.3:0/549145209 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:49.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.801+0000 7f3516d68700 1 --2- 172.21.15.3:0/549145209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f351009f650 0x7f35101c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:49.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.801+0000 7f3515d66700 1 --2- 172.21.15.3:0/549145209 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f351009f240 0x7f35101a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:49.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.801+0000 7f3516d68700 1 -- 172.21.15.3:0/549145209 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f351009f240 msgr2=0x7f35101a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:49.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.801+0000 7f3516d68700 1 --2- 172.21.15.3:0/549145209 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f351009f240 0x7f35101a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:49.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.801+0000 7f3516d68700 1 -- 172.21.15.3:0/549145209 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f351009e930 msgr2=0x7f35101a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:49.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.801+0000 7f3516d68700 1 --2- 172.21.15.3:0/549145209 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f351009e930 0x7f35101a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:49.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.801+0000 7f3516d68700 1 -- 172.21.15.3:0/549145209 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f350c0094a0 con 0x7f351009f650 2023-10-03T10:12:49.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.801+0000 7f3515d66700 1 --2- 172.21.15.3:0/549145209 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f351009f240 0x7f35101a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:12:49.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.801+0000 7f3516567700 1 --2- 172.21.15.3:0/549145209 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f351009e930 0x7f35101a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:49.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.802+0000 7f3516d68700 1 --2- 172.21.15.3:0/549145209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f351009f650 0x7f35101c3d20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f350800cc10 tx=0x7f350800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:49.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.802+0000 7f3506ffd700 1 -- 172.21.15.3:0/549145209 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350800aab0 con 0x7f351009f650 2023-10-03T10:12:49.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.802+0000 7f3506ffd700 1 -- 172.21.15.3:0/549145209 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350800ac30 con 0x7f351009f650 2023-10-03T10:12:49.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.802+0000 7f3506ffd700 1 -- 172.21.15.3:0/549145209 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35080070d0 con 0x7f351009f650 2023-10-03T10:12:49.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.802+0000 7f35187cb700 1 -- 172.21.15.3:0/549145209 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35100a0190 con 0x7f351009f650 2023-10-03T10:12:49.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.802+0000 7f35187cb700 1 -- 172.21.15.3:0/549145209 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35101a3e10 con 0x7f351009f650 2023-10-03T10:12:49.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.803+0000 7f35187cb700 1 -- 172.21.15.3:0/549145209 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35101a3ab0 con 0x7f351009f650 2023-10-03T10:12:49.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.803+0000 7f3506ffd700 1 -- 172.21.15.3:0/549145209 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f3508007230 con 0x7f351009f650 2023-10-03T10:12:49.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.804+0000 7f3516567700 1 -- 172.21.15.3:0/549145209 >> v1:172.21.15.3:6816/127025 conn(0x7f34f402f510 legacy=0x7f34f40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:49.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.804+0000 7f3506ffd700 1 -- 172.21.15.3:0/549145209 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f35080074e0 con 0x7f351009f650 2023-10-03T10:12:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.809+0000 7f3506ffd700 1 -- 172.21.15.3:0/549145209 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f35101a3ab0 con 0x7f351009f650 2023-10-03T10:12:49.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.827+0000 7f22abbb8700 1 Processor -- start 2023-10-03T10:12:49.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.827+0000 7f22abbb8700 1 -- start start 2023-10-03T10:12:49.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.827+0000 7f22abbb8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f22a409d6f0 0x7f22a409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:49.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.827+0000 7f22abbb8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f22a409e410 0x7f22a4130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:49.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.828+0000 7f22abbb8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22a409f130 con 0x7f22a409e000 2023-10-03T10:12:49.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.828+0000 7f22abbb8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a409f520 con 0x7f22a409e410 2023-10-03T10:12:49.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.828+0000 7f22abbb8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a409e930 con 0x7f22a409d6f0 2023-10-03T10:12:49.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.828+0000 7f22a9954700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f22a409d6f0 0x7f22a409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:49.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.828+0000 7f22a9954700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f22a409d6f0 0x7f22a409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60080/0 (socket says 172.21.15.3:60080) 2023-10-03T10:12:49.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.828+0000 7f22a9954700 1 -- 172.21.15.3:0/165711244 learned_addr learned my addr 172.21.15.3:0/165711244 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:49.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.828+0000 7f22aa155700 1 --2- 172.21.15.3:0/165711244 >> v2:172.21.15.3:3300/0 conn(0x7f22a409e410 0x7f22a4130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:49.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.828+0000 7f22a9954700 1 -- 172.21.15.3:0/165711244 >> v2:172.21.15.3:3300/0 conn(0x7f22a409e410 msgr2=0x7f22a4130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:49.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.828+0000 7f22a9954700 1 --2- 172.21.15.3:0/165711244 >> v2:172.21.15.3:3300/0 conn(0x7f22a409e410 0x7f22a4130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:49.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.828+0000 7f22a9954700 1 -- 172.21.15.3:0/165711244 >> v1:172.21.15.3:6789/0 conn(0x7f22a409e000 legacy=0x7f22a4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:49.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.828+0000 7f22a9954700 1 -- 172.21.15.3:0/165711244 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a409b2d0 con 0x7f22a409d6f0 2023-10-03T10:12:49.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.828+0000 7f22a9954700 1 --2- 172.21.15.3:0/165711244 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f22a409d6f0 0x7f22a409dac0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f22a000bb90 tx=0x7f22a000bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e34f2a633796a4ea server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:49.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.829+0000 7f22a8952700 1 -- 172.21.15.3:0/165711244 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a000dd30 con 0x7f22a409d6f0 2023-10-03T10:12:49.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.829+0000 7f22a8952700 1 -- 172.21.15.3:0/165711244 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a000de90 con 0x7f22a409d6f0 2023-10-03T10:12:49.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.829+0000 7f22a8952700 1 -- 172.21.15.3:0/165711244 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a0010690 con 0x7f22a409d6f0 2023-10-03T10:12:49.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.829+0000 7f22abbb8700 1 -- 172.21.15.3:0/165711244 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f22a409d6f0 msgr2=0x7f22a409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:49.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.829+0000 7f22abbb8700 1 --2- 172.21.15.3:0/165711244 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f22a409d6f0 0x7f22a409dac0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f22a000bb90 tx=0x7f22a000bea0 comp rx=0 tx=0).stop 2023-10-03T10:12:49.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.830+0000 7f22abbb8700 1 -- 172.21.15.3:0/165711244 shutdown_connections 2023-10-03T10:12:49.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.830+0000 7f22abbb8700 1 --2- 172.21.15.3:0/165711244 >> v2:172.21.15.3:3300/0 conn(0x7f22a409e410 0x7f22a4130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:49.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.830+0000 7f22abbb8700 1 --2- 172.21.15.3:0/165711244 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f22a409d6f0 0x7f22a409dac0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:49.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.830+0000 7f22abbb8700 1 -- 172.21.15.3:0/165711244 >> 172.21.15.3:0/165711244 conn(0x7f22a4066f50 msgr2=0x7f22a409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:49.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.830+0000 7f22abbb8700 1 -- 172.21.15.3:0/165711244 shutdown_connections 2023-10-03T10:12:49.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.830+0000 7f22abbb8700 1 -- 172.21.15.3:0/165711244 wait complete. 2023-10-03T10:12:49.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.831+0000 7f22abbb8700 1 Processor -- start 2023-10-03T10:12:49.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.831+0000 7f22abbb8700 1 -- start start 2023-10-03T10:12:49.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.831+0000 7f22abbb8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f22a409d6f0 0x7f22a4122450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:49.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.831+0000 7f22abbb8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22a409e000 0x7f22a4122990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:49.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.831+0000 7f22abbb8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f22a409e410 0x7f22a4122ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:49.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.831+0000 7f22abbb8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a4125f20 con 0x7f22a409e000 2023-10-03T10:12:49.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.831+0000 7f22abbb8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a4124010 con 0x7f22a409e410 2023-10-03T10:12:49.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.831+0000 7f22abbb8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22a40a1e90 con 0x7f22a409d6f0 2023-10-03T10:12:49.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.832+0000 7f22a9954700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f22a409d6f0 0x7f22a4122450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:49.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.832+0000 7f22a9954700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f22a409d6f0 0x7f22a4122450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38654/0 (socket says 172.21.15.3:38654) 2023-10-03T10:12:49.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.832+0000 7f22a9954700 1 -- 172.21.15.3:0/243764884 learned_addr learned my addr 172.21.15.3:0/243764884 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:49.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.832+0000 7f22aa155700 1 --2- 172.21.15.3:0/243764884 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f22a409e410 0x7f22a4122ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:49.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.832+0000 7f22a9153700 1 --2- 172.21.15.3:0/243764884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22a409e000 0x7f22a4122990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:49.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.832+0000 7f22a9954700 1 -- 172.21.15.3:0/243764884 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f22a409e410 msgr2=0x7f22a4122ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:49.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.832+0000 7f22a9954700 1 --2- 172.21.15.3:0/243764884 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f22a409e410 0x7f22a4122ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:49.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.832+0000 7f22a9954700 1 -- 172.21.15.3:0/243764884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22a409e000 msgr2=0x7f22a4122990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:49.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.832+0000 7f22a9954700 1 --2- 172.21.15.3:0/243764884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22a409e000 0x7f22a4122990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:49.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.832+0000 7f22a9954700 1 -- 172.21.15.3:0/243764884 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a000b7f0 con 0x7f22a409d6f0 2023-10-03T10:12:49.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.832+0000 7f22a9954700 1 --2- 172.21.15.3:0/243764884 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f22a409d6f0 0x7f22a4122450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f22a0000c00 tx=0x7f22a0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:49.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.832+0000 7f229a7fc700 1 -- 172.21.15.3:0/243764884 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a0003b70 con 0x7f22a409d6f0 2023-10-03T10:12:49.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.832+0000 7f229a7fc700 1 -- 172.21.15.3:0/243764884 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a0003cd0 con 0x7f22a409d6f0 2023-10-03T10:12:49.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.832+0000 7f22abbb8700 1 -- 172.21.15.3:0/243764884 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a409c700 con 0x7f22a409d6f0 2023-10-03T10:12:49.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.833+0000 7f229a7fc700 1 -- 172.21.15.3:0/243764884 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a00106b0 con 0x7f22a409d6f0 2023-10-03T10:12:49.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.833+0000 7f22abbb8700 1 -- 172.21.15.3:0/243764884 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a41247e0 con 0x7f22a409d6f0 2023-10-03T10:12:49.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.833+0000 7f229a7fc700 1 -- 172.21.15.3:0/243764884 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f22a0019400 con 0x7f22a409d6f0 2023-10-03T10:12:49.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.834+0000 7f229a7fc700 1 -- 172.21.15.3:0/243764884 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f22a0040fd0 con 0x7f22a409d6f0 2023-10-03T10:12:49.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.834+0000 7f22a9153700 1 -- 172.21.15.3:0/243764884 >> v1:172.21.15.3:6816/127025 conn(0x7f228802f510 legacy=0x7f22880319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:49.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.834+0000 7f22abbb8700 1 -- 172.21.15.3:0/243764884 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a4074ea0 con 0x7f22a409d6f0 2023-10-03T10:12:49.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.839+0000 7f229a7fc700 1 -- 172.21.15.3:0/243764884 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f22a0003e40 con 0x7f22a409d6f0 2023-10-03T10:12:49.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.849+0000 7f0ce0f01700 1 Processor -- start 2023-10-03T10:12:49.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.849+0000 7f0ce0f01700 1 -- start start 2023-10-03T10:12:49.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.850+0000 7f0ce0f01700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cdc09e930 0x7f0cdc09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:49.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.850+0000 7f0ce0f01700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0cdc09f240 0x7f0cdc1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:49.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.850+0000 7f0ce0f01700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cdc070940 con 0x7f0cdc09f650 2023-10-03T10:12:49.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.850+0000 7f0ce0f01700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cdc06fd60 con 0x7f0cdc09f240 2023-10-03T10:12:49.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.850+0000 7f0ce0f01700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc06fa80 con 0x7f0cdc09e930 2023-10-03T10:12:49.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.850+0000 7f0cdad9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0cdc09f650 0x7f0cdc131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50738/0 (socket says 172.21.15.3:50738) 2023-10-03T10:12:49.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.850+0000 7f0cdad9d700 1 -- 172.21.15.3:0/4279860228 learned_addr learned my addr 172.21.15.3:0/4279860228 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:49.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.850+0000 7f0cd9d9b700 1 --2- 172.21.15.3:0/4279860228 >> v2:172.21.15.3:3300/0 conn(0x7f0cdc09f240 0x7f0cdc1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:49.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.850+0000 7f0cda59c700 1 --2- 172.21.15.3:0/4279860228 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cdc09e930 0x7f0cdc09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:49.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.850+0000 7f0cd9d9b700 1 -- 172.21.15.3:0/4279860228 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cdc09e930 msgr2=0x7f0cdc09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:49.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.850+0000 7f0cd9d9b700 1 --2- 172.21.15.3:0/4279860228 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cdc09e930 0x7f0cdc09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:49.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.850+0000 7f0cd9d9b700 1 -- 172.21.15.3:0/4279860228 >> v1:172.21.15.3:6789/0 conn(0x7f0cdc09f650 legacy=0x7f0cdc131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.850+0000 7f0cd9d9b700 1 -- 172.21.15.3:0/4279860228 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cdc09fb60 con 0x7f0cdc09f240 2023-10-03T10:12:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.851+0000 7f0cd9d9b700 1 --2- 172.21.15.3:0/4279860228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cdc09f240 0x7f0cdc1317f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009840 tx=0x7f0cd0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b05ef1fd93faff server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.851+0000 7f0cd959a700 1 -- 172.21.15.3:0/4279860228 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0011830 con 0x7f0cdc09f240 2023-10-03T10:12:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.851+0000 7f0cd959a700 1 -- 172.21.15.3:0/4279860228 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd0011990 con 0x7f0cdc09f240 2023-10-03T10:12:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.851+0000 7f0cd959a700 1 -- 172.21.15.3:0/4279860228 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0011d20 con 0x7f0cdc09f240 2023-10-03T10:12:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.852+0000 7f0ce0f01700 1 -- 172.21.15.3:0/4279860228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cdc09f240 msgr2=0x7f0cdc1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.852+0000 7f0ce0f01700 1 --2- 172.21.15.3:0/4279860228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cdc09f240 0x7f0cdc1317f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009840 tx=0x7f0cd0009b50 comp rx=0 tx=0).stop 2023-10-03T10:12:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.852+0000 7f0ce0f01700 1 -- 172.21.15.3:0/4279860228 shutdown_connections 2023-10-03T10:12:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.852+0000 7f0ce0f01700 1 --2- 172.21.15.3:0/4279860228 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cdc09e930 0x7f0cdc09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:49.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.852+0000 7f0ce0f01700 1 --2- 172.21.15.3:0/4279860228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cdc09f240 0x7f0cdc1317f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:49.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.852+0000 7f0ce0f01700 1 -- 172.21.15.3:0/4279860228 >> 172.21.15.3:0/4279860228 conn(0x7f0cdc0a4210 msgr2=0x7f0cdc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:49.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.852+0000 7f0ce0f01700 1 -- 172.21.15.3:0/4279860228 shutdown_connections 2023-10-03T10:12:49.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.852+0000 7f0ce0f01700 1 -- 172.21.15.3:0/4279860228 wait complete. 2023-10-03T10:12:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.853+0000 7f0ce0f01700 1 Processor -- start 2023-10-03T10:12:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.853+0000 7f0ce0f01700 1 -- start start 2023-10-03T10:12:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.854+0000 7f0ce0f01700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0cdc09e930 0x7f0cdc1abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.854+0000 7f0ce0f01700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cdc09f240 0x7f0cdc1ac4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:49.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.854+0000 7f0ce0f01700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cdc09f650 0x7f0cdc1aca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:49.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.854+0000 7f0ce0f01700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc1ba6e0 con 0x7f0cdc09f240 2023-10-03T10:12:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.854+0000 7f0ce0f01700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc0a3630 con 0x7f0cdc09f650 2023-10-03T10:12:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.854+0000 7f0ce0f01700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc1af160 con 0x7f0cdc09e930 2023-10-03T10:12:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.854+0000 7f0cd9d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cdc09f240 0x7f0cdc1ac4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.854+0000 7f0cd9d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cdc09f240 0x7f0cdc1ac4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56010/0 (socket says 172.21.15.3:56010) 2023-10-03T10:12:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.854+0000 7f0cd9d9b700 1 -- 172.21.15.3:0/940518844 learned_addr learned my addr 172.21.15.3:0/940518844 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.855+0000 7f0cd9d9b700 1 -- 172.21.15.3:0/940518844 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0cdc09e930 msgr2=0x7f0cdc1abfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.855+0000 7f0cd9d9b700 1 --2- 172.21.15.3:0/940518844 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0cdc09e930 0x7f0cdc1abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.855+0000 7f0cdad9d700 1 --2- 172.21.15.3:0/940518844 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cdc09f650 0x7f0cdc1aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.855+0000 7f0cd9d9b700 1 -- 172.21.15.3:0/940518844 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cdc09f650 msgr2=0x7f0cdc1aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.855+0000 7f0cd9d9b700 1 --2- 172.21.15.3:0/940518844 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cdc09f650 0x7f0cdc1aca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.855+0000 7f0cd9d9b700 1 -- 172.21.15.3:0/940518844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd00094a0 con 0x7f0cdc09f240 2023-10-03T10:12:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.855+0000 7f0cd9d9b700 1 --2- 172.21.15.3:0/940518844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cdc09f240 0x7f0cdc1ac4f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0cd00090b0 tx=0x7f0cd0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.855+0000 7f0cdad9d700 1 --2- 172.21.15.3:0/940518844 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cdc09f650 0x7f0cdc1aca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:12:49.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.855+0000 7f0ccaffd700 1 -- 172.21.15.3:0/940518844 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd001a890 con 0x7f0cdc09f240 2023-10-03T10:12:49.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.855+0000 7f0ccaffd700 1 -- 172.21.15.3:0/940518844 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd001a9f0 con 0x7f0cdc09f240 2023-10-03T10:12:49.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.855+0000 7f0ce0f01700 1 -- 172.21.15.3:0/940518844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cdc0a0190 con 0x7f0cdc09f240 2023-10-03T10:12:49.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.855+0000 7f0ccaffd700 1 -- 172.21.15.3:0/940518844 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd001ad80 con 0x7f0cdc09f240 2023-10-03T10:12:49.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.855+0000 7f0ce0f01700 1 -- 172.21.15.3:0/940518844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cdc1afe70 con 0x7f0cdc09f240 2023-10-03T10:12:49.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.857+0000 7f0ccaffd700 1 -- 172.21.15.3:0/940518844 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f0cd0022070 con 0x7f0cdc09f240 2023-10-03T10:12:49.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.857+0000 7f0ccaffd700 1 -- 172.21.15.3:0/940518844 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f0cd0040a20 con 0x7f0cdc09f240 2023-10-03T10:12:49.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.858+0000 7f0cda59c700 1 -- 172.21.15.3:0/940518844 >> v1:172.21.15.3:6816/127025 conn(0x7f0cbc02f510 legacy=0x7f0cbc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:49.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.858+0000 7f0ce0f01700 1 -- 172.21.15.3:0/940518844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cdc06b9a0 con 0x7f0cdc09f240 2023-10-03T10:12:49.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.868+0000 7f0ccaffd700 1 -- 172.21.15.3:0/940518844 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0cd0015080 con 0x7f0cdc09f240 2023-10-03T10:12:49.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:49.963+0000 7f38b2ffd700 1 -- 172.21.15.3:0/246141547 >> v1:172.21.15.3:6816/127025 conn(0x7f38a002f510 legacy=0x7f38a00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:50.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.005+0000 7f3516567700 1 -- 172.21.15.3:0/549145209 >> v1:172.21.15.3:6816/127025 conn(0x7f34f402f510 legacy=0x7f34f40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:50.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.020+0000 7f35187cb700 1 -- 172.21.15.3:0/549145209 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f35101a3ab0 con 0x7f351009f650 2023-10-03T10:12:50.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.020+0000 7f38b9b10700 1 -- 172.21.15.3:0/246141547 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f38b4074ea0 con 0x7f38b409e930 2023-10-03T10:12:50.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.020+0000 7f3506ffd700 1 -- 172.21.15.3:0/549145209 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f35101a3ab0 con 0x7f351009f650 2023-10-03T10:12:50.021 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:12:50.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.021+0000 7f389bfff700 1 -- 172.21.15.3:0/246141547 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f38a4015020 con 0x7f38b409e930 2023-10-03T10:12:50.022 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:12:50.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.023+0000 7f38b9b10700 1 -- 172.21.15.3:0/246141547 >> v1:172.21.15.3:6816/127025 conn(0x7f38a002f510 legacy=0x7f38a00319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:50.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.023+0000 7f38b9b10700 1 -- 172.21.15.3:0/246141547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38b409e930 msgr2=0x7f38b41c0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:50.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.023+0000 7f38b9b10700 1 --2- 172.21.15.3:0/246141547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38b409e930 0x7f38b41c0de0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f38a40090b0 tx=0x7f38a4004750 comp rx=0 tx=0).stop 2023-10-03T10:12:50.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.023+0000 7f38b9b10700 1 -- 172.21.15.3:0/246141547 shutdown_connections 2023-10-03T10:12:50.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.023+0000 7f38b9b10700 1 --2- 172.21.15.3:0/246141547 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38b409f650 0x7f38b41c3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:50.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.023+0000 7f38b9b10700 1 --2- 172.21.15.3:0/246141547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38b409e930 0x7f38b41c0de0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:50.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.023+0000 7f38b9b10700 1 --2- 172.21.15.3:0/246141547 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38b409f240 0x7f38b41c1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:50.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.023+0000 7f38b9b10700 1 -- 172.21.15.3:0/246141547 >> 172.21.15.3:0/246141547 conn(0x7f38b40a4210 msgr2=0x7f38b41b6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:50.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.023+0000 7f38b9b10700 1 -- 172.21.15.3:0/246141547 shutdown_connections 2023-10-03T10:12:50.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.023+0000 7f38b9b10700 1 -- 172.21.15.3:0/246141547 wait complete. 2023-10-03T10:12:50.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.024+0000 7f35187cb700 1 -- 172.21.15.3:0/549145209 >> v1:172.21.15.3:6816/127025 conn(0x7f34f402f510 legacy=0x7f34f40319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:50.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.024+0000 7f35187cb700 1 -- 172.21.15.3:0/549145209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f351009f650 msgr2=0x7f35101c3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:50.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.024+0000 7f35187cb700 1 --2- 172.21.15.3:0/549145209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f351009f650 0x7f35101c3d20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f350800cc10 tx=0x7f350800cf20 comp rx=0 tx=0).stop 2023-10-03T10:12:50.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.024+0000 7f35187cb700 1 -- 172.21.15.3:0/549145209 shutdown_connections 2023-10-03T10:12:50.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.024+0000 7f35187cb700 1 --2- 172.21.15.3:0/549145209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f351009f650 0x7f35101c3d20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:50.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.024+0000 7f35187cb700 1 --2- 172.21.15.3:0/549145209 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f351009e930 0x7f35101a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:50.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.024+0000 7f35187cb700 1 --2- 172.21.15.3:0/549145209 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f351009f240 0x7f35101a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:50.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.024+0000 7f35187cb700 1 -- 172.21.15.3:0/549145209 >> 172.21.15.3:0/549145209 conn(0x7f35100a4210 msgr2=0x7f351006c320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:50.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.024+0000 7f35187cb700 1 -- 172.21.15.3:0/549145209 shutdown_connections 2023-10-03T10:12:50.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.024+0000 7f35187cb700 1 -- 172.21.15.3:0/549145209 wait complete. 2023-10-03T10:12:50.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.035+0000 7f22a9153700 1 -- 172.21.15.3:0/243764884 >> v1:172.21.15.3:6816/127025 conn(0x7f228802f510 legacy=0x7f22880319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:50.038 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:12:50.045 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:12:50.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.052+0000 7f22abbb8700 1 -- 172.21.15.3:0/243764884 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f22a406c360 con 0x7f22a409d6f0 2023-10-03T10:12:50.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.052+0000 7f229a7fc700 1 -- 172.21.15.3:0/243764884 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f22a0018360 con 0x7f22a409d6f0 2023-10-03T10:12:50.053 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:12:50.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.055+0000 7f22abbb8700 1 -- 172.21.15.3:0/243764884 >> v1:172.21.15.3:6816/127025 conn(0x7f228802f510 legacy=0x7f22880319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:50.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.055+0000 7f22abbb8700 1 -- 172.21.15.3:0/243764884 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f22a409d6f0 msgr2=0x7f22a4122450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:50.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.055+0000 7f22abbb8700 1 --2- 172.21.15.3:0/243764884 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f22a409d6f0 0x7f22a4122450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f22a0000c00 tx=0x7f22a0004750 comp rx=0 tx=0).stop 2023-10-03T10:12:50.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.055+0000 7f22abbb8700 1 -- 172.21.15.3:0/243764884 shutdown_connections 2023-10-03T10:12:50.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.055+0000 7f22abbb8700 1 --2- 172.21.15.3:0/243764884 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f22a409e410 0x7f22a4122ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:50.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.055+0000 7f22abbb8700 1 --2- 172.21.15.3:0/243764884 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f22a409d6f0 0x7f22a4122450 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:50.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.055+0000 7f22abbb8700 1 --2- 172.21.15.3:0/243764884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22a409e000 0x7f22a4122990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:50.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.055+0000 7f22abbb8700 1 -- 172.21.15.3:0/243764884 >> 172.21.15.3:0/243764884 conn(0x7f22a4066f50 msgr2=0x7f22a40a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:50.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.055+0000 7f22abbb8700 1 -- 172.21.15.3:0/243764884 shutdown_connections 2023-10-03T10:12:50.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.055+0000 7f22abbb8700 1 -- 172.21.15.3:0/243764884 wait complete. 2023-10-03T10:12:50.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.059+0000 7f0cda59c700 1 -- 172.21.15.3:0/940518844 >> v1:172.21.15.3:6816/127025 conn(0x7f0cbc02f510 legacy=0x7f0cbc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:50.069 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:12:50.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.086+0000 7f0ce0f01700 1 -- 172.21.15.3:0/940518844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0cdc074ea0 con 0x7f0cdc09f240 2023-10-03T10:12:50.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.086+0000 7f0ccaffd700 1 -- 172.21.15.3:0/940518844 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0cd0029020 con 0x7f0cdc09f240 2023-10-03T10:12:50.086 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:12:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.088+0000 7f0ce0f01700 1 -- 172.21.15.3:0/940518844 >> v1:172.21.15.3:6816/127025 conn(0x7f0cbc02f510 legacy=0x7f0cbc0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.088+0000 7f0ce0f01700 1 -- 172.21.15.3:0/940518844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cdc09f240 msgr2=0x7f0cdc1ac4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.088+0000 7f0ce0f01700 1 --2- 172.21.15.3:0/940518844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cdc09f240 0x7f0cdc1ac4f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0cd00090b0 tx=0x7f0cd0004750 comp rx=0 tx=0).stop 2023-10-03T10:12:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.088+0000 7f0ce0f01700 1 -- 172.21.15.3:0/940518844 shutdown_connections 2023-10-03T10:12:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.088+0000 7f0ce0f01700 1 --2- 172.21.15.3:0/940518844 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cdc09f650 0x7f0cdc1aca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.088+0000 7f0ce0f01700 1 --2- 172.21.15.3:0/940518844 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0cdc09e930 0x7f0cdc1abfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.088+0000 7f0ce0f01700 1 --2- 172.21.15.3:0/940518844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cdc09f240 0x7f0cdc1ac4f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.088+0000 7f0ce0f01700 1 -- 172.21.15.3:0/940518844 >> 172.21.15.3:0/940518844 conn(0x7f0cdc0a4210 msgr2=0x7f0cdc06c610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.088+0000 7f0ce0f01700 1 -- 172.21.15.3:0/940518844 shutdown_connections 2023-10-03T10:12:50.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:50.088+0000 7f0ce0f01700 1 -- 172.21.15.3:0/940518844 wait complete. 2023-10-03T10:12:50.102 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:12:51.039 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:12:51.047 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:12:51.069 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:12:51.103 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:12:51.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.205+0000 7f71aa105700 1 Processor -- start 2023-10-03T10:12:51.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.205+0000 7f71aa105700 1 -- start start 2023-10-03T10:12:51.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.206+0000 7f71aa105700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f71a409d6f0 0x7f71a409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:51.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.206+0000 7f71aa105700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f71a409e000 0x7f71a41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:51.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.206+0000 7f71aa105700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71a409e410 0x7f71a4133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:51.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.206+0000 7f71aa105700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71a406b460 con 0x7f71a409d6f0 2023-10-03T10:12:51.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.206+0000 7f71aa105700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a409b120 con 0x7f71a409e000 2023-10-03T10:12:51.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.206+0000 7f71aa105700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71a409a7e0 con 0x7f71a409e410 2023-10-03T10:12:51.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.206+0000 7f71a2ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f71a409e000 0x7f71a41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:51.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.206+0000 7f71a2ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f71a409e000 0x7f71a41339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60126/0 (socket says 172.21.15.3:60126) 2023-10-03T10:12:51.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.206+0000 7f71a3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71a409e410 0x7f71a4133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:51.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.206+0000 7f71a2ffd700 1 -- 172.21.15.3:0/4154009781 learned_addr learned my addr 172.21.15.3:0/4154009781 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:51.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.207+0000 7f71a37fe700 1 --2- 172.21.15.3:0/4154009781 >> v2:172.21.15.3:3300/0 conn(0x7f71a409d6f0 0x7f71a409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:51.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.207+0000 7f71a3fff700 1 -- 172.21.15.3:0/4154009781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f71a409e000 msgr2=0x7f71a41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:51.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.207+0000 7f71a3fff700 1 --2- 172.21.15.3:0/4154009781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f71a409e000 0x7f71a41339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.207+0000 7f71a3fff700 1 -- 172.21.15.3:0/4154009781 >> v2:172.21.15.3:3300/0 conn(0x7f71a409d6f0 msgr2=0x7f71a409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:51.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.207+0000 7f71a3fff700 1 --2- 172.21.15.3:0/4154009781 >> v2:172.21.15.3:3300/0 conn(0x7f71a409d6f0 0x7f71a409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.207+0000 7f71a3fff700 1 -- 172.21.15.3:0/4154009781 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a409a500 con 0x7f71a409e410 2023-10-03T10:12:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.207+0000 7f71a3fff700 1 --2- 172.21.15.3:0/4154009781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71a409e410 0x7f71a4133f30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f719c009dc0 tx=0x7f719c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36aba022208e6b34 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.207+0000 7f71a27fc700 1 -- 172.21.15.3:0/4154009781 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c00e070 con 0x7f71a409e410 2023-10-03T10:12:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.207+0000 7f71a27fc700 1 -- 172.21.15.3:0/4154009781 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719c00bae0 con 0x7f71a409e410 2023-10-03T10:12:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.207+0000 7f71a27fc700 1 -- 172.21.15.3:0/4154009781 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c00be90 con 0x7f71a409e410 2023-10-03T10:12:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.208+0000 7f71aa105700 1 -- 172.21.15.3:0/4154009781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71a409e410 msgr2=0x7f71a4133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.208+0000 7f71aa105700 1 --2- 172.21.15.3:0/4154009781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71a409e410 0x7f71a4133f30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f719c009dc0 tx=0x7f719c00b330 comp rx=0 tx=0).stop 2023-10-03T10:12:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.208+0000 7f71aa105700 1 -- 172.21.15.3:0/4154009781 shutdown_connections 2023-10-03T10:12:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.208+0000 7f71aa105700 1 --2- 172.21.15.3:0/4154009781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71a409e410 0x7f71a4133f30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.208+0000 7f71aa105700 1 --2- 172.21.15.3:0/4154009781 >> v2:172.21.15.3:3300/0 conn(0x7f71a409d6f0 0x7f71a409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.208+0000 7f71aa105700 1 --2- 172.21.15.3:0/4154009781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f71a409e000 0x7f71a41339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.208+0000 7f71aa105700 1 -- 172.21.15.3:0/4154009781 >> 172.21.15.3:0/4154009781 conn(0x7f71a4066f50 msgr2=0x7f71a409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.209+0000 7f71aa105700 1 -- 172.21.15.3:0/4154009781 shutdown_connections 2023-10-03T10:12:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.209+0000 7f71aa105700 1 -- 172.21.15.3:0/4154009781 wait complete. 2023-10-03T10:12:51.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.210+0000 7f71aa105700 1 Processor -- start 2023-10-03T10:12:51.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.210+0000 7f71aa105700 1 -- start start 2023-10-03T10:12:51.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.211+0000 7f71aa105700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f71a409d6f0 0x7f71a41a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:51.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.211+0000 7f71aa105700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a409e000 0x7f71a41a5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:51.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.211+0000 7f71aa105700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71a409e410 0x7f71a41c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:51.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.211+0000 7f71aa105700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a41a6a00 con 0x7f71a409e000 2023-10-03T10:12:51.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.211+0000 7f71aa105700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a41b9360 con 0x7f71a409d6f0 2023-10-03T10:12:51.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.211+0000 7f71aa105700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71a41a7e30 con 0x7f71a409e410 2023-10-03T10:12:51.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.211+0000 7f71a37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f71a409d6f0 0x7f71a41a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:51.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.211+0000 7f71a2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a409e000 0x7f71a41a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:51.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.211+0000 7f71a37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f71a409d6f0 0x7f71a41a5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60138/0 (socket says 172.21.15.3:60138) 2023-10-03T10:12:51.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.211+0000 7f71a37fe700 1 -- 172.21.15.3:0/3521324687 learned_addr learned my addr 172.21.15.3:0/3521324687 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:51.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.212+0000 7f71a3fff700 1 --2- 172.21.15.3:0/3521324687 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71a409e410 0x7f71a41c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:51.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.212+0000 7f71a37fe700 1 -- 172.21.15.3:0/3521324687 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71a409e410 msgr2=0x7f71a41c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:51.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.212+0000 7f71a37fe700 1 --2- 172.21.15.3:0/3521324687 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71a409e410 0x7f71a41c7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.212+0000 7f71a37fe700 1 -- 172.21.15.3:0/3521324687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a409e000 msgr2=0x7f71a41a5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:51.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.212+0000 7f71a37fe700 1 --2- 172.21.15.3:0/3521324687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a409e000 0x7f71a41a5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.212+0000 7f71a37fe700 1 -- 172.21.15.3:0/3521324687 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719c009a70 con 0x7f71a409d6f0 2023-10-03T10:12:51.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.212+0000 7f71a2ffd700 1 --2- 172.21.15.3:0/3521324687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a409e000 0x7f71a41a5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:51.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.212+0000 7f71a37fe700 1 --2- 172.21.15.3:0/3521324687 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f71a409d6f0 0x7f71a41a5230 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f719400e9b0 tx=0x7f719400ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:51.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.212+0000 7f718bfff700 1 -- 172.21.15.3:0/3521324687 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7194009710 con 0x7f71a409d6f0 2023-10-03T10:12:51.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.213+0000 7f718bfff700 1 -- 172.21.15.3:0/3521324687 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719400ee50 con 0x7f71a409d6f0 2023-10-03T10:12:51.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.213+0000 7f718bfff700 1 -- 172.21.15.3:0/3521324687 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719400ca50 con 0x7f71a409d6f0 2023-10-03T10:12:51.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.213+0000 7f71aa105700 1 -- 172.21.15.3:0/3521324687 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a409c710 con 0x7f71a409d6f0 2023-10-03T10:12:51.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.213+0000 7f71aa105700 1 -- 172.21.15.3:0/3521324687 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a41a4e30 con 0x7f71a409d6f0 2023-10-03T10:12:51.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.214+0000 7f718bfff700 1 -- 172.21.15.3:0/3521324687 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f719400cbb0 con 0x7f71a409d6f0 2023-10-03T10:12:51.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.214+0000 7f71aa105700 1 -- 172.21.15.3:0/3521324687 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a4074ea0 con 0x7f71a409d6f0 2023-10-03T10:12:51.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.214+0000 7f71a2ffd700 1 -- 172.21.15.3:0/3521324687 >> v1:172.21.15.3:6816/127025 conn(0x7f719002f510 legacy=0x7f71900319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:51.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.214+0000 7f718bfff700 1 -- 172.21.15.3:0/3521324687 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f7194040090 con 0x7f71a409d6f0 2023-10-03T10:12:51.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.219+0000 7f718bfff700 1 -- 172.21.15.3:0/3521324687 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7194013990 con 0x7f71a409d6f0 2023-10-03T10:12:51.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.236+0000 7ff2ee887700 1 Processor -- start 2023-10-03T10:12:51.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.236+0000 7ff2ee887700 1 -- start start 2023-10-03T10:12:51.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.237+0000 7ff2ee887700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2e809eca0 0x7ff2e813cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:51.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.237+0000 7ff2ee887700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2e806f050 0x7ff2e813f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:51.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.237+0000 7ff2ee887700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e8070ee0 con 0x7ff2e806e570 2023-10-03T10:12:51.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.237+0000 7ff2ee887700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e8071980 con 0x7ff2e806f050 2023-10-03T10:12:51.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.237+0000 7ff2ee887700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e8071480 con 0x7ff2e809eca0 2023-10-03T10:12:51.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.237+0000 7ff2ece24700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2e806f050 0x7ff2e813f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:51.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.237+0000 7ff2ece24700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2e806f050 0x7ff2e813f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60154/0 (socket says 172.21.15.3:60154) 2023-10-03T10:12:51.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.237+0000 7ff2ece24700 1 -- 172.21.15.3:0/517984885 learned_addr learned my addr 172.21.15.3:0/517984885 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:51.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.237+0000 7ff2e77fe700 1 --2- 172.21.15.3:0/517984885 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2e809eca0 0x7ff2e813cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:51.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.238+0000 7ff2e77fe700 1 -- 172.21.15.3:0/517984885 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2e806f050 msgr2=0x7ff2e813f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:51.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.238+0000 7ff2e77fe700 1 --2- 172.21.15.3:0/517984885 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2e806f050 0x7ff2e813f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.238+0000 7ff2e77fe700 1 -- 172.21.15.3:0/517984885 >> v1:172.21.15.3:6789/0 conn(0x7ff2e806e570 legacy=0x7ff2e806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:51.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.238+0000 7ff2e6ffd700 1 -- 172.21.15.3:0/517984885 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199629138 0 0) 0x7ff2e8070ee0 con 0x7ff2e806e570 2023-10-03T10:12:51.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.238+0000 7ff2e77fe700 1 -- 172.21.15.3:0/517984885 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e8071b90 con 0x7ff2e809eca0 2023-10-03T10:12:51.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.238+0000 7ff2ece24700 1 --2- 172.21.15.3:0/517984885 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2e806f050 0x7ff2e813f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:51.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.238+0000 7ff2e77fe700 1 --2- 172.21.15.3:0/517984885 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2e809eca0 0x7ff2e813cea0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc009870 tx=0x7ff2dc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a04ae775aa1aba13 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:51.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.238+0000 7ff2e6ffd700 1 -- 172.21.15.3:0/517984885 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc011830 con 0x7ff2e809eca0 2023-10-03T10:12:51.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.238+0000 7ff2e6ffd700 1 -- 172.21.15.3:0/517984885 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2dc011990 con 0x7ff2e809eca0 2023-10-03T10:12:51.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.238+0000 7ff2e6ffd700 1 -- 172.21.15.3:0/517984885 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc011d20 con 0x7ff2e809eca0 2023-10-03T10:12:51.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.239+0000 7ff2ee887700 1 -- 172.21.15.3:0/517984885 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2e809eca0 msgr2=0x7ff2e813cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:51.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.239+0000 7ff2ee887700 1 --2- 172.21.15.3:0/517984885 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2e809eca0 0x7ff2e813cea0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc009870 tx=0x7ff2dc009b80 comp rx=0 tx=0).stop 2023-10-03T10:12:51.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.239+0000 7ff2ee887700 1 -- 172.21.15.3:0/517984885 shutdown_connections 2023-10-03T10:12:51.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.239+0000 7ff2ee887700 1 --2- 172.21.15.3:0/517984885 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2e806f050 0x7ff2e813f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.239+0000 7ff2ee887700 1 --2- 172.21.15.3:0/517984885 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2e809eca0 0x7ff2e813cea0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.239+0000 7ff2ee887700 1 -- 172.21.15.3:0/517984885 >> 172.21.15.3:0/517984885 conn(0x7ff2e80a2aa0 msgr2=0x7ff2e80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:51.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.239+0000 7ff2ee887700 1 -- 172.21.15.3:0/517984885 shutdown_connections 2023-10-03T10:12:51.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.239+0000 7ff2ee887700 1 -- 172.21.15.3:0/517984885 wait complete. 2023-10-03T10:12:51.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.240+0000 7f6258d4a700 1 Processor -- start 2023-10-03T10:12:51.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.241+0000 7f6258d4a700 1 -- start start 2023-10-03T10:12:51.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.241+0000 7ff2ee887700 1 Processor -- start 2023-10-03T10:12:51.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.241+0000 7ff2ee887700 1 -- start start 2023-10-03T10:12:51.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.242+0000 7ff2ee887700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2e806e570 0x7ff2e81265d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:51.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.242+0000 7ff2ee887700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2e806f050 0x7ff2e8126b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:51.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.242+0000 7ff2ee887700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2e8127050 0x7ff2e81d2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:51.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.242+0000 7ff2ee887700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e8129ec0 con 0x7ff2e806f050 2023-10-03T10:12:51.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.242+0000 7ff2ee887700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e812a0a0 con 0x7ff2e806e570 2023-10-03T10:12:51.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.242+0000 7ff2ee887700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e8129030 con 0x7ff2e8127050 2023-10-03T10:12:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.242+0000 7f6258d4a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6254136710 0x7f6254138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.242+0000 7f6258d4a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6254139030 0x7f625413b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.242+0000 7f6258d4a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f625406ef30 con 0x7f62541338a0 2023-10-03T10:12:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.242+0000 7f6258d4a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f625406fef0 con 0x7f6254139030 2023-10-03T10:12:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.242+0000 7f6258d4a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62541334b0 con 0x7f6254136710 2023-10-03T10:12:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.242+0000 7ff2ece24700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2e8127050 0x7ff2e81d2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.242+0000 7ff2ece24700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2e8127050 0x7ff2e81d2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38690/0 (socket says 172.21.15.3:38690) 2023-10-03T10:12:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.242+0000 7ff2ece24700 1 -- 172.21.15.3:0/787115072 learned_addr learned my addr 172.21.15.3:0/787115072 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.242+0000 7ff2e7fff700 1 --2- 172.21.15.3:0/787115072 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2e806e570 0x7ff2e81265d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.242+0000 7ff2e77fe700 1 --2- 172.21.15.3:0/787115072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2e806f050 0x7ff2e8126b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.242+0000 7f625259c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f62541338a0 0x7f6254133c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50758/0 (socket says 172.21.15.3:50758) 2023-10-03T10:12:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.242+0000 7f625259c700 1 -- 172.21.15.3:0/3364664170 learned_addr learned my addr 172.21.15.3:0/3364664170 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.242+0000 7f6252d9d700 1 --2- 172.21.15.3:0/3364664170 >> v2:172.21.15.3:3300/0 conn(0x7f6254139030 0x7f625413b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.242+0000 7f6251d9b700 1 --2- 172.21.15.3:0/3364664170 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6254136710 0x7f6254138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.242+0000 7ff2ece24700 1 -- 172.21.15.3:0/787115072 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2e806e570 msgr2=0x7ff2e81265d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.242+0000 7ff2ece24700 1 --2- 172.21.15.3:0/787115072 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2e806e570 0x7ff2e81265d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.242+0000 7ff2ece24700 1 -- 172.21.15.3:0/787115072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2e806f050 msgr2=0x7ff2e8126b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.242+0000 7ff2ece24700 1 --2- 172.21.15.3:0/787115072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2e806f050 0x7ff2e8126b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.243+0000 7ff2ece24700 1 -- 172.21.15.3:0/787115072 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e0009b30 con 0x7ff2e8127050 2023-10-03T10:12:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.243+0000 7f6252d9d700 1 -- 172.21.15.3:0/3364664170 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6254136710 msgr2=0x7f6254138af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.243+0000 7f6252d9d700 1 --2- 172.21.15.3:0/3364664170 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6254136710 0x7f6254138af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.243+0000 7f6252d9d700 1 -- 172.21.15.3:0/3364664170 >> v1:172.21.15.3:6789/0 conn(0x7f62541338a0 legacy=0x7f6254133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.243+0000 7f6252d9d700 1 -- 172.21.15.3:0/3364664170 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f625406fc30 con 0x7f6254139030 2023-10-03T10:12:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.243+0000 7ff2e77fe700 1 --2- 172.21.15.3:0/787115072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2e806f050 0x7ff2e8126b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:12:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.243+0000 7ff2ece24700 1 --2- 172.21.15.3:0/787115072 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2e8127050 0x7ff2e81d2a70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000efc0 tx=0x7ff2e000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.243+0000 7f6252d9d700 1 --2- 172.21.15.3:0/3364664170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6254139030 0x7f625413b410 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f624c009d90 tx=0x7f624c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e780514194b0446 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.243+0000 7ff2e4ff9700 1 -- 172.21.15.3:0/787115072 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0018070 con 0x7ff2e8127050 2023-10-03T10:12:51.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.243+0000 7f625159a700 1 -- 172.21.15.3:0/3364664170 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c00e070 con 0x7f6254139030 2023-10-03T10:12:51.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.243+0000 7ff2e4ff9700 1 -- 172.21.15.3:0/787115072 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e000f040 con 0x7ff2e8127050 2023-10-03T10:12:51.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.243+0000 7ff2ee887700 1 -- 172.21.15.3:0/787115072 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2dc0094a0 con 0x7ff2e8127050 2023-10-03T10:12:51.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.243+0000 7f625159a700 1 -- 172.21.15.3:0/3364664170 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624c00bb20 con 0x7f6254139030 2023-10-03T10:12:51.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.243+0000 7f625159a700 1 -- 172.21.15.3:0/3364664170 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c012430 con 0x7f6254139030 2023-10-03T10:12:51.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.243+0000 7ff2e4ff9700 1 -- 172.21.15.3:0/787115072 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0003c20 con 0x7ff2e8127050 2023-10-03T10:12:51.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.243+0000 7ff2ee887700 1 -- 172.21.15.3:0/787115072 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e8070610 con 0x7ff2e8127050 2023-10-03T10:12:51.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.244+0000 7f6258d4a700 1 -- 172.21.15.3:0/3364664170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6254139030 msgr2=0x7f625413b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:51.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.244+0000 7f6258d4a700 1 --2- 172.21.15.3:0/3364664170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6254139030 0x7f625413b410 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f624c009d90 tx=0x7f624c00b330 comp rx=0 tx=0).stop 2023-10-03T10:12:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.245+0000 7ff2ee887700 1 -- 172.21.15.3:0/787115072 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2e8122c80 con 0x7ff2e8127050 2023-10-03T10:12:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.245+0000 7ff2e4ff9700 1 -- 172.21.15.3:0/787115072 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7ff2e000c930 con 0x7ff2e8127050 2023-10-03T10:12:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.245+0000 7f6258d4a700 1 -- 172.21.15.3:0/3364664170 shutdown_connections 2023-10-03T10:12:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.245+0000 7f6258d4a700 1 --2- 172.21.15.3:0/3364664170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6254139030 0x7f625413b410 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.245+0000 7f6258d4a700 1 --2- 172.21.15.3:0/3364664170 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6254136710 0x7f6254138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.245+0000 7f6258d4a700 1 -- 172.21.15.3:0/3364664170 >> 172.21.15.3:0/3364664170 conn(0x7f62540a2aa0 msgr2=0x7f62540a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.245+0000 7ff2e7fff700 1 -- 172.21.15.3:0/787115072 >> v1:172.21.15.3:6816/127025 conn(0x7ff2d402f510 legacy=0x7ff2d40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.245+0000 7f6258d4a700 1 -- 172.21.15.3:0/3364664170 shutdown_connections 2023-10-03T10:12:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.245+0000 7f6258d4a700 1 -- 172.21.15.3:0/3364664170 wait complete. 2023-10-03T10:12:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.246+0000 7ff2e4ff9700 1 -- 172.21.15.3:0/787115072 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7ff2e00092e0 con 0x7ff2e8127050 2023-10-03T10:12:51.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.247+0000 7f6258d4a700 1 Processor -- start 2023-10-03T10:12:51.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.247+0000 7f6258d4a700 1 -- start start 2023-10-03T10:12:51.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.248+0000 7f6258d4a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62541338a0 0x7f62541c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:51.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.248+0000 7f6258d4a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6254136710 0x7f62541c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:51.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.248+0000 7f6258d4a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6254139030 0x7f625411acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:51.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.248+0000 7f6258d4a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625411e5d0 con 0x7f62541338a0 2023-10-03T10:12:51.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.248+0000 7f6258d4a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f625411c8c0 con 0x7f6254136710 2023-10-03T10:12:51.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.248+0000 7f6258d4a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f625411e230 con 0x7f6254139030 2023-10-03T10:12:51.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.248+0000 7f625259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62541338a0 0x7f62541c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:51.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.248+0000 7f6252d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6254139030 0x7f625411acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:51.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.248+0000 7f625259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62541338a0 0x7f62541c8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56062/0 (socket says 172.21.15.3:56062) 2023-10-03T10:12:51.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.248+0000 7f625259c700 1 -- 172.21.15.3:0/2541912369 learned_addr learned my addr 172.21.15.3:0/2541912369 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:51.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.249+0000 7f625259c700 1 -- 172.21.15.3:0/2541912369 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6254139030 msgr2=0x7f625411acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:51.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.249+0000 7f625259c700 1 --2- 172.21.15.3:0/2541912369 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6254139030 0x7f625411acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.249+0000 7f625259c700 1 -- 172.21.15.3:0/2541912369 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6254136710 msgr2=0x7f62541c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:51.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.249+0000 7f625259c700 1 --2- 172.21.15.3:0/2541912369 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6254136710 0x7f62541c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.249+0000 7f625259c700 1 -- 172.21.15.3:0/2541912369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f624c009a70 con 0x7f62541338a0 2023-10-03T10:12:51.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.249+0000 7f6252d9d700 1 --2- 172.21.15.3:0/2541912369 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6254139030 0x7f625411acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:51.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.249+0000 7f625259c700 1 --2- 172.21.15.3:0/2541912369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62541338a0 0x7f62541c8390 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f624400c520 tx=0x7f624400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:51.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.249+0000 7f6242ffd700 1 -- 172.21.15.3:0/2541912369 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6244005d70 con 0x7f62541338a0 2023-10-03T10:12:51.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.249+0000 7f6242ffd700 1 -- 172.21.15.3:0/2541912369 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62440070d0 con 0x7f62541338a0 2023-10-03T10:12:51.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.249+0000 7f6258d4a700 1 -- 172.21.15.3:0/2541912369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f625406f640 con 0x7f62541338a0 2023-10-03T10:12:51.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.249+0000 7f6258d4a700 1 -- 172.21.15.3:0/2541912369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f625411e7d0 con 0x7f62541338a0 2023-10-03T10:12:51.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.250+0000 7f6242ffd700 1 -- 172.21.15.3:0/2541912369 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6244007440 con 0x7f62541338a0 2023-10-03T10:12:51.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.250+0000 7f6242ffd700 1 -- 172.21.15.3:0/2541912369 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f62440075a0 con 0x7f62541338a0 2023-10-03T10:12:51.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.251+0000 7f6242ffd700 1 -- 172.21.15.3:0/2541912369 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f624403f050 con 0x7f62541338a0 2023-10-03T10:12:51.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.251+0000 7f6251d9b700 1 -- 172.21.15.3:0/2541912369 >> v1:172.21.15.3:6816/127025 conn(0x7f623402f510 legacy=0x7f62340319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:51.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.251+0000 7f6258d4a700 1 -- 172.21.15.3:0/2541912369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62380052e0 con 0x7f62541338a0 2023-10-03T10:12:51.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.252+0000 7ff2e4ff9700 1 -- 172.21.15.3:0/787115072 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff2e0045060 con 0x7ff2e8127050 2023-10-03T10:12:51.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.259+0000 7f6242ffd700 1 -- 172.21.15.3:0/2541912369 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6244017030 con 0x7f62541338a0 2023-10-03T10:12:51.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.277+0000 7f5755eb1700 1 Processor -- start 2023-10-03T10:12:51.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.277+0000 7f5755eb1700 1 -- start start 2023-10-03T10:12:51.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.278+0000 7f5755eb1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57500a0a30 0x7f57500a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:51.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.278+0000 7f5755eb1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5750134170 0x7f575013bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:51.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.278+0000 7f5755eb1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f575009edd0 con 0x7f57500a1340 2023-10-03T10:12:51.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.278+0000 7f5755eb1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f575009e1f0 con 0x7f5750134170 2023-10-03T10:12:51.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.278+0000 7f5755eb1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f575009dc50 con 0x7f57500a0a30 2023-10-03T10:12:51.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.278+0000 7f574f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57500a0a30 0x7f57500a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:51.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.278+0000 7f574f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57500a0a30 0x7f57500a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38720/0 (socket says 172.21.15.3:38720) 2023-10-03T10:12:51.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.278+0000 7f574f7fe700 1 -- 172.21.15.3:0/12088324 learned_addr learned my addr 172.21.15.3:0/12088324 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:51.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.279+0000 7f574e7fc700 1 -- 172.21.15.3:0/12088324 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1460631017 0 0) 0x7f575009edd0 con 0x7f57500a1340 2023-10-03T10:12:51.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.279+0000 7f574f7fe700 1 -- 172.21.15.3:0/12088324 >> v2:172.21.15.3:3300/0 conn(0x7f5750134170 msgr2=0x7f575013bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:51.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.279+0000 7f574f7fe700 1 --2- 172.21.15.3:0/12088324 >> v2:172.21.15.3:3300/0 conn(0x7f5750134170 0x7f575013bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.279+0000 7f574f7fe700 1 -- 172.21.15.3:0/12088324 >> v1:172.21.15.3:6789/0 conn(0x7f57500a1340 legacy=0x7f5750133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:51.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.279+0000 7f574f7fe700 1 -- 172.21.15.3:0/12088324 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57500a1fe0 con 0x7f57500a0a30 2023-10-03T10:12:51.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.279+0000 7f574f7fe700 1 --2- 172.21.15.3:0/12088324 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57500a0a30 0x7f57500a0e00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5740009e50 tx=0x7f574000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=706dfb50c76922e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:51.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.279+0000 7f574e7fc700 1 -- 172.21.15.3:0/12088324 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574000e070 con 0x7f57500a0a30 2023-10-03T10:12:51.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.279+0000 7f574e7fc700 1 -- 172.21.15.3:0/12088324 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f574000b9c0 con 0x7f57500a0a30 2023-10-03T10:12:51.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.279+0000 7f574e7fc700 1 -- 172.21.15.3:0/12088324 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574000bd70 con 0x7f57500a0a30 2023-10-03T10:12:51.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.280+0000 7f5755eb1700 1 -- 172.21.15.3:0/12088324 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57500a0a30 msgr2=0x7f57500a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:51.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.280+0000 7f5755eb1700 1 --2- 172.21.15.3:0/12088324 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57500a0a30 0x7f57500a0e00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5740009e50 tx=0x7f574000b330 comp rx=0 tx=0).stop 2023-10-03T10:12:51.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.280+0000 7f5755eb1700 1 -- 172.21.15.3:0/12088324 shutdown_connections 2023-10-03T10:12:51.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.280+0000 7f5755eb1700 1 --2- 172.21.15.3:0/12088324 >> v2:172.21.15.3:3300/0 conn(0x7f5750134170 0x7f575013bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.280+0000 7f5755eb1700 1 --2- 172.21.15.3:0/12088324 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57500a0a30 0x7f57500a0e00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.280+0000 7f5755eb1700 1 -- 172.21.15.3:0/12088324 >> 172.21.15.3:0/12088324 conn(0x7f57500a6550 msgr2=0x7f57500a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:51.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.280+0000 7f5755eb1700 1 -- 172.21.15.3:0/12088324 shutdown_connections 2023-10-03T10:12:51.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.280+0000 7f5755eb1700 1 -- 172.21.15.3:0/12088324 wait complete. 2023-10-03T10:12:51.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.281+0000 7f5755eb1700 1 Processor -- start 2023-10-03T10:12:51.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.282+0000 7f5755eb1700 1 -- start start 2023-10-03T10:12:51.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.282+0000 7f5755eb1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57500a0a30 0x7f5750122920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:51.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.282+0000 7f5755eb1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57500a1340 0x7f5750122e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:51.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.282+0000 7f5755eb1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5750134170 0x7f57501233a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:51.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.283+0000 7f5755eb1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5750123df0 con 0x7f5750134170 2023-10-03T10:12:51.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.283+0000 7f5755eb1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5750124530 con 0x7f57500a0a30 2023-10-03T10:12:51.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.283+0000 7f5755eb1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5750126440 con 0x7f57500a1340 2023-10-03T10:12:51.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.283+0000 7f574f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57500a0a30 0x7f5750122920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:51.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.283+0000 7f574f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57500a0a30 0x7f5750122920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60212/0 (socket says 172.21.15.3:60212) 2023-10-03T10:12:51.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.283+0000 7f574f7fe700 1 -- 172.21.15.3:0/847750813 learned_addr learned my addr 172.21.15.3:0/847750813 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:51.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.283+0000 7f574effd700 1 --2- 172.21.15.3:0/847750813 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57500a1340 0x7f5750122e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:51.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.283+0000 7f574ffff700 1 --2- 172.21.15.3:0/847750813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5750134170 0x7f57501233a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:51.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.283+0000 7f574ffff700 1 -- 172.21.15.3:0/847750813 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57500a1340 msgr2=0x7f5750122e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:51.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.283+0000 7f574ffff700 1 --2- 172.21.15.3:0/847750813 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57500a1340 0x7f5750122e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.283+0000 7f574ffff700 1 -- 172.21.15.3:0/847750813 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57500a0a30 msgr2=0x7f5750122920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:51.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.283+0000 7f574ffff700 1 --2- 172.21.15.3:0/847750813 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57500a0a30 0x7f5750122920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.283+0000 7f574ffff700 1 -- 172.21.15.3:0/847750813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5740009b30 con 0x7f5750134170 2023-10-03T10:12:51.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.283+0000 7f574effd700 1 --2- 172.21.15.3:0/847750813 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57500a1340 0x7f5750122e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:51.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.283+0000 7f574f7fe700 1 --2- 172.21.15.3:0/847750813 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57500a0a30 0x7f5750122920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:51.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.283+0000 7f574ffff700 1 --2- 172.21.15.3:0/847750813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5750134170 0x7f57501233a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f574800b3f0 tx=0x7f574800b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:51.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.284+0000 7f5737fff700 1 -- 172.21.15.3:0/847750813 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5748011630 con 0x7f5750134170 2023-10-03T10:12:51.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.284+0000 7f5737fff700 1 -- 172.21.15.3:0/847750813 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5748011790 con 0x7f5750134170 2023-10-03T10:12:51.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.284+0000 7f5755eb1700 1 -- 172.21.15.3:0/847750813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f575009bd90 con 0x7f5750134170 2023-10-03T10:12:51.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.284+0000 7f5737fff700 1 -- 172.21.15.3:0/847750813 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5748011b40 con 0x7f5750134170 2023-10-03T10:12:51.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.284+0000 7f5755eb1700 1 -- 172.21.15.3:0/847750813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57500a0270 con 0x7f5750134170 2023-10-03T10:12:51.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.285+0000 7f5737fff700 1 -- 172.21.15.3:0/847750813 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f5748011ca0 con 0x7f5750134170 2023-10-03T10:12:51.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.285+0000 7f5755eb1700 1 -- 172.21.15.3:0/847750813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5750124b40 con 0x7f5750134170 2023-10-03T10:12:51.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.285+0000 7f5737fff700 1 -- 172.21.15.3:0/847750813 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f574803fac0 con 0x7f5750134170 2023-10-03T10:12:51.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.285+0000 7f574f7fe700 1 -- 172.21.15.3:0/847750813 >> v1:172.21.15.3:6816/127025 conn(0x7f573c02f510 legacy=0x7f573c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:51.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.290+0000 7f5737fff700 1 -- 172.21.15.3:0/847750813 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5750124b40 con 0x7f5750134170 2023-10-03T10:12:51.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.415+0000 7f71a2ffd700 1 -- 172.21.15.3:0/3521324687 >> v1:172.21.15.3:6816/127025 conn(0x7f719002f510 legacy=0x7f71900319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:51.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.446+0000 7ff2e7fff700 1 -- 172.21.15.3:0/787115072 >> v1:172.21.15.3:6816/127025 conn(0x7ff2d402f510 legacy=0x7ff2d40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:51.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.452+0000 7f6251d9b700 1 -- 172.21.15.3:0/2541912369 >> v1:172.21.15.3:6816/127025 conn(0x7f623402f510 legacy=0x7f62340319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.485+0000 7f6258d4a700 1 -- 172.21.15.3:0/2541912369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6238005860 con 0x7f62541338a0 2023-10-03T10:12:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.485+0000 7f6242ffd700 1 -- 172.21.15.3:0/2541912369 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6244017030 con 0x7f62541338a0 2023-10-03T10:12:51.486 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:12:51.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.486+0000 7f574f7fe700 1 -- 172.21.15.3:0/847750813 >> v1:172.21.15.3:6816/127025 conn(0x7f573c02f510 legacy=0x7f573c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.487+0000 7f6258d4a700 1 -- 172.21.15.3:0/2541912369 >> v1:172.21.15.3:6816/127025 conn(0x7f623402f510 legacy=0x7f62340319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.488+0000 7f6258d4a700 1 -- 172.21.15.3:0/2541912369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62541338a0 msgr2=0x7f62541c8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.488+0000 7f6258d4a700 1 --2- 172.21.15.3:0/2541912369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62541338a0 0x7f62541c8390 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f624400c520 tx=0x7f624400c830 comp rx=0 tx=0).stop 2023-10-03T10:12:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.488+0000 7f6258d4a700 1 -- 172.21.15.3:0/2541912369 shutdown_connections 2023-10-03T10:12:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.488+0000 7f6258d4a700 1 --2- 172.21.15.3:0/2541912369 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6254139030 0x7f625411acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.488+0000 7f6258d4a700 1 --2- 172.21.15.3:0/2541912369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62541338a0 0x7f62541c8390 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.488+0000 7f6258d4a700 1 --2- 172.21.15.3:0/2541912369 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6254136710 0x7f62541c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.488+0000 7f6258d4a700 1 -- 172.21.15.3:0/2541912369 >> 172.21.15.3:0/2541912369 conn(0x7f62540a2aa0 msgr2=0x7f62541344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:51.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.488+0000 7f6258d4a700 1 -- 172.21.15.3:0/2541912369 shutdown_connections 2023-10-03T10:12:51.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.488+0000 7f6258d4a700 1 -- 172.21.15.3:0/2541912369 wait complete. 2023-10-03T10:12:51.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.490+0000 7f71aa105700 1 -- 172.21.15.3:0/3521324687 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f71a406c360 con 0x7f71a409d6f0 2023-10-03T10:12:51.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.490+0000 7f718bfff700 1 -- 172.21.15.3:0/3521324687 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7194021030 con 0x7f71a409d6f0 2023-10-03T10:12:51.491 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:12:51.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.494+0000 7f71aa105700 1 -- 172.21.15.3:0/3521324687 >> v1:172.21.15.3:6816/127025 conn(0x7f719002f510 legacy=0x7f71900319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:51.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.494+0000 7f71aa105700 1 -- 172.21.15.3:0/3521324687 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f71a409d6f0 msgr2=0x7f71a41a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:51.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.494+0000 7f71aa105700 1 --2- 172.21.15.3:0/3521324687 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f71a409d6f0 0x7f71a41a5230 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f719400e9b0 tx=0x7f719400ecc0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.494+0000 7f71aa105700 1 -- 172.21.15.3:0/3521324687 shutdown_connections 2023-10-03T10:12:51.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.494+0000 7f71aa105700 1 --2- 172.21.15.3:0/3521324687 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71a409e410 0x7f71a41c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.494+0000 7f71aa105700 1 --2- 172.21.15.3:0/3521324687 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f71a409d6f0 0x7f71a41a5230 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.494+0000 7f71aa105700 1 --2- 172.21.15.3:0/3521324687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a409e000 0x7f71a41a5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.494+0000 7f71aa105700 1 -- 172.21.15.3:0/3521324687 >> 172.21.15.3:0/3521324687 conn(0x7f71a4066f50 msgr2=0x7f71a40a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:51.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.494+0000 7f71aa105700 1 -- 172.21.15.3:0/3521324687 shutdown_connections 2023-10-03T10:12:51.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.494+0000 7f71aa105700 1 -- 172.21.15.3:0/3521324687 wait complete. 2023-10-03T10:12:51.504 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:12:51.514 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:12:51.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.515+0000 7ff2ee887700 1 -- 172.21.15.3:0/787115072 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff2e81294b0 con 0x7ff2e8127050 2023-10-03T10:12:51.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.515+0000 7ff2e4ff9700 1 -- 172.21.15.3:0/787115072 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff2e0045020 con 0x7ff2e8127050 2023-10-03T10:12:51.516 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:12:51.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.518+0000 7ff2ee887700 1 -- 172.21.15.3:0/787115072 >> v1:172.21.15.3:6816/127025 conn(0x7ff2d402f510 legacy=0x7ff2d40319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:51.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.518+0000 7ff2ee887700 1 -- 172.21.15.3:0/787115072 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2e8127050 msgr2=0x7ff2e81d2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:51.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.518+0000 7ff2ee887700 1 --2- 172.21.15.3:0/787115072 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2e8127050 0x7ff2e81d2a70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000efc0 tx=0x7ff2e000c3a0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.518+0000 7ff2ee887700 1 -- 172.21.15.3:0/787115072 shutdown_connections 2023-10-03T10:12:51.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.518+0000 7ff2ee887700 1 --2- 172.21.15.3:0/787115072 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2e8127050 0x7ff2e81d2a70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.518+0000 7ff2ee887700 1 --2- 172.21.15.3:0/787115072 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2e806e570 0x7ff2e81265d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.518+0000 7ff2ee887700 1 --2- 172.21.15.3:0/787115072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2e806f050 0x7ff2e8126b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.518+0000 7ff2ee887700 1 -- 172.21.15.3:0/787115072 >> 172.21.15.3:0/787115072 conn(0x7ff2e80a2aa0 msgr2=0x7ff2e813ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:51.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.518+0000 7ff2ee887700 1 -- 172.21.15.3:0/787115072 shutdown_connections 2023-10-03T10:12:51.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.518+0000 7ff2ee887700 1 -- 172.21.15.3:0/787115072 wait complete. 2023-10-03T10:12:51.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.523+0000 7f5755eb1700 1 -- 172.21.15.3:0/847750813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5750125e00 con 0x7f5750134170 2023-10-03T10:12:51.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.524+0000 7f5737fff700 1 -- 172.21.15.3:0/847750813 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5750125e00 con 0x7f5750134170 2023-10-03T10:12:51.524 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:12:51.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.526+0000 7f5755eb1700 1 -- 172.21.15.3:0/847750813 >> v1:172.21.15.3:6816/127025 conn(0x7f573c02f510 legacy=0x7f573c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:51.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.526+0000 7f5755eb1700 1 -- 172.21.15.3:0/847750813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5750134170 msgr2=0x7f57501233a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:51.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.526+0000 7f5755eb1700 1 --2- 172.21.15.3:0/847750813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5750134170 0x7f57501233a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f574800b3f0 tx=0x7f574800b700 comp rx=0 tx=0).stop 2023-10-03T10:12:51.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.526+0000 7f5755eb1700 1 -- 172.21.15.3:0/847750813 shutdown_connections 2023-10-03T10:12:51.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.526+0000 7f5755eb1700 1 --2- 172.21.15.3:0/847750813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5750134170 0x7f57501233a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.526+0000 7f5755eb1700 1 --2- 172.21.15.3:0/847750813 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57500a0a30 0x7f5750122920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.526+0000 7f5755eb1700 1 --2- 172.21.15.3:0/847750813 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57500a1340 0x7f5750122e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:51.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.526+0000 7f5755eb1700 1 -- 172.21.15.3:0/847750813 >> 172.21.15.3:0/847750813 conn(0x7f57500a6550 msgr2=0x7f575009edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:51.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.527+0000 7f5755eb1700 1 -- 172.21.15.3:0/847750813 shutdown_connections 2023-10-03T10:12:51.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:51.527+0000 7f5755eb1700 1 -- 172.21.15.3:0/847750813 wait complete. 2023-10-03T10:12:51.533 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:12:51.541 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:12:52.506 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:12:52.515 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:12:52.533 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:12:52.542 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:12:52.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.677+0000 7f0957844700 1 Processor -- start 2023-10-03T10:12:52.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.677+0000 7f0957844700 1 -- start start 2023-10-03T10:12:52.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.678+0000 7f0957844700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f095009e1d0 0x7f0950135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:52.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.678+0000 7f0957844700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0950135e50 0x7f0950138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:52.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.678+0000 7f0957844700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f095009f130 con 0x7f095009d6f0 2023-10-03T10:12:52.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.678+0000 7f0957844700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f095009f520 con 0x7f095009e1d0 2023-10-03T10:12:52.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.678+0000 7f0957844700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f095009e930 con 0x7f0950135e50 2023-10-03T10:12:52.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.678+0000 7f0954ddf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f095009e1d0 0x7f0950135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:52.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.678+0000 7f0955de1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0950135e50 0x7f0950138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:52.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.678+0000 7f0954ddf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f095009e1d0 0x7f0950135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56092/0 (socket says 172.21.15.3:56092) 2023-10-03T10:12:52.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.678+0000 7f0954ddf700 1 -- 172.21.15.3:0/4246488063 learned_addr learned my addr 172.21.15.3:0/4246488063 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:52.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.678+0000 7f0955de1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0950135e50 0x7f0950138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38746/0 (socket says 172.21.15.3:38746) 2023-10-03T10:12:52.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.679+0000 7f0954ddf700 1 -- 172.21.15.3:0/4246488063 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0950135e50 msgr2=0x7f0950138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:52.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.679+0000 7f0954ddf700 1 --2- 172.21.15.3:0/4246488063 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0950135e50 0x7f0950138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.679+0000 7f0954ddf700 1 -- 172.21.15.3:0/4246488063 >> v1:172.21.15.3:6789/0 conn(0x7f095009d6f0 legacy=0x7f095009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:52.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.679+0000 7f0954ddf700 1 -- 172.21.15.3:0/4246488063 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095009b2d0 con 0x7f095009e1d0 2023-10-03T10:12:52.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.679+0000 7f0955de1700 1 --2- 172.21.15.3:0/4246488063 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0950135e50 0x7f0950138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:52.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.679+0000 7f0954ddf700 1 --2- 172.21.15.3:0/4246488063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f095009e1d0 0x7f0950135910 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f09400098f0 tx=0x7f0940009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f0a2062e7ba7c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:52.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.679+0000 7f0947fff700 1 -- 172.21.15.3:0/4246488063 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0940011830 con 0x7f095009e1d0 2023-10-03T10:12:52.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.679+0000 7f0947fff700 1 -- 172.21.15.3:0/4246488063 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0940011990 con 0x7f095009e1d0 2023-10-03T10:12:52.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.679+0000 7f0947fff700 1 -- 172.21.15.3:0/4246488063 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0940011d20 con 0x7f095009e1d0 2023-10-03T10:12:52.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.680+0000 7f0957844700 1 -- 172.21.15.3:0/4246488063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f095009e1d0 msgr2=0x7f0950135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:52.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.680+0000 7f0957844700 1 --2- 172.21.15.3:0/4246488063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f095009e1d0 0x7f0950135910 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f09400098f0 tx=0x7f0940009c00 comp rx=0 tx=0).stop 2023-10-03T10:12:52.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.680+0000 7f0957844700 1 -- 172.21.15.3:0/4246488063 shutdown_connections 2023-10-03T10:12:52.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.680+0000 7f0957844700 1 --2- 172.21.15.3:0/4246488063 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0950135e50 0x7f0950138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.680+0000 7f0957844700 1 --2- 172.21.15.3:0/4246488063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f095009e1d0 0x7f0950135910 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.680+0000 7f0957844700 1 -- 172.21.15.3:0/4246488063 >> 172.21.15.3:0/4246488063 conn(0x7f0950066f50 msgr2=0x7f095009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:52.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.680+0000 7f0957844700 1 -- 172.21.15.3:0/4246488063 shutdown_connections 2023-10-03T10:12:52.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.680+0000 7f0957844700 1 -- 172.21.15.3:0/4246488063 wait complete. 2023-10-03T10:12:52.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.682+0000 7f0957844700 1 Processor -- start 2023-10-03T10:12:52.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.682+0000 7f0957844700 1 -- start start 2023-10-03T10:12:52.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.683+0000 7f0957844700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f095009d6f0 0x7f09501c4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:52.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.683+0000 7f0957844700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f095009e1d0 0x7f0950119900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:52.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.683+0000 7f0957844700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0950135e50 0x7f0950119e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:52.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.683+0000 7f0957844700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095011d4e0 con 0x7f0950135e50 2023-10-03T10:12:52.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.683+0000 7f0957844700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095011d050 con 0x7f095009e1d0 2023-10-03T10:12:52.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.683+0000 7f0957844700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f095011b4a0 con 0x7f095009d6f0 2023-10-03T10:12:52.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.683+0000 7f0954ddf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f095009e1d0 0x7f0950119900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:52.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.683+0000 7f0954ddf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f095009e1d0 0x7f0950119900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60218/0 (socket says 172.21.15.3:60218) 2023-10-03T10:12:52.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.683+0000 7f09555e0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f095009d6f0 0x7f09501c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:52.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.683+0000 7f0954ddf700 1 -- 172.21.15.3:0/1661425628 learned_addr learned my addr 172.21.15.3:0/1661425628 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:52.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.683+0000 7f0955de1700 1 --2- 172.21.15.3:0/1661425628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0950135e50 0x7f0950119e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:52.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.683+0000 7f0954ddf700 1 -- 172.21.15.3:0/1661425628 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f095009d6f0 msgr2=0x7f09501c4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:52.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.683+0000 7f0954ddf700 1 --2- 172.21.15.3:0/1661425628 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f095009d6f0 0x7f09501c4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.683+0000 7f0954ddf700 1 -- 172.21.15.3:0/1661425628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0950135e50 msgr2=0x7f0950119e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:52.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.683+0000 7f0954ddf700 1 --2- 172.21.15.3:0/1661425628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0950135e50 0x7f0950119e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.683+0000 7f0954ddf700 1 -- 172.21.15.3:0/1661425628 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094800bc80 con 0x7f095009e1d0 2023-10-03T10:12:52.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.684+0000 7f0954ddf700 1 --2- 172.21.15.3:0/1661425628 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f095009e1d0 0x7f0950119900 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0940009c40 tx=0x7f0940004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:52.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.684+0000 7f0945ffb700 1 -- 172.21.15.3:0/1661425628 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0940004510 con 0x7f095009e1d0 2023-10-03T10:12:52.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.684+0000 7f0945ffb700 1 -- 172.21.15.3:0/1661425628 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094001e070 con 0x7f095009e1d0 2023-10-03T10:12:52.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.684+0000 7f0957844700 1 -- 172.21.15.3:0/1661425628 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0940009550 con 0x7f095009e1d0 2023-10-03T10:12:52.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.684+0000 7f0945ffb700 1 -- 172.21.15.3:0/1661425628 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09400038e0 con 0x7f095009e1d0 2023-10-03T10:12:52.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.684+0000 7f0957844700 1 -- 172.21.15.3:0/1661425628 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f095009c700 con 0x7f095009e1d0 2023-10-03T10:12:52.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.686+0000 7f0945ffb700 1 -- 172.21.15.3:0/1661425628 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f0940003a40 con 0x7f095009e1d0 2023-10-03T10:12:52.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.686+0000 7f0945ffb700 1 -- 172.21.15.3:0/1661425628 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f0940040aa0 con 0x7f095009e1d0 2023-10-03T10:12:52.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.686+0000 7f09555e0700 1 -- 172.21.15.3:0/1661425628 >> v1:172.21.15.3:6816/127025 conn(0x7f093402f510 legacy=0x7f09340319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:52.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.686+0000 7f0957844700 1 -- 172.21.15.3:0/1661425628 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0950074ea0 con 0x7f095009e1d0 2023-10-03T10:12:52.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.692+0000 7f0945ffb700 1 -- 172.21.15.3:0/1661425628 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0940015020 con 0x7f095009e1d0 2023-10-03T10:12:52.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.700+0000 7f0a63292700 1 Processor -- start 2023-10-03T10:12:52.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.700+0000 7f0a63292700 1 -- start start 2023-10-03T10:12:52.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.701+0000 7f0a63292700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0a5c09e930 0x7f0a5c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:52.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.701+0000 7f0a63292700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0a5c09f240 0x7f0a5c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:52.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.701+0000 7f0a63292700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0a5c09f650 0x7f0a5c131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:52.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.701+0000 7f0a63292700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a5c070940 con 0x7f0a5c09f650 2023-10-03T10:12:52.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.701+0000 7f0a63292700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c06fd60 con 0x7f0a5c09f240 2023-10-03T10:12:52.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.701+0000 7f0a63292700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c06fa80 con 0x7f0a5c09e930 2023-10-03T10:12:52.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.701+0000 7f0a6102e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0a5c09e930 0x7f0a5c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:52.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.701+0000 7f0a6102e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0a5c09e930 0x7f0a5c09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38772/0 (socket says 172.21.15.3:38772) 2023-10-03T10:12:52.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.701+0000 7f0a6102e700 1 -- 172.21.15.3:0/2969760158 learned_addr learned my addr 172.21.15.3:0/2969760158 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:52.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.701+0000 7f0a6082d700 1 --2- 172.21.15.3:0/2969760158 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0a5c09f240 0x7f0a5c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:52.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.701+0000 7f0a6182f700 1 --2- 172.21.15.3:0/2969760158 >> v2:172.21.15.3:3300/0 conn(0x7f0a5c09f650 0x7f0a5c131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:52.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.701+0000 7f0a6102e700 1 -- 172.21.15.3:0/2969760158 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0a5c09f240 msgr2=0x7f0a5c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:52.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.701+0000 7f0a6102e700 1 --2- 172.21.15.3:0/2969760158 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0a5c09f240 0x7f0a5c1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.701+0000 7f0a6102e700 1 -- 172.21.15.3:0/2969760158 >> v2:172.21.15.3:3300/0 conn(0x7f0a5c09f650 msgr2=0x7f0a5c131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:52.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.702+0000 7f0a6102e700 1 --2- 172.21.15.3:0/2969760158 >> v2:172.21.15.3:3300/0 conn(0x7f0a5c09f650 0x7f0a5c131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.702+0000 7f0a6102e700 1 -- 172.21.15.3:0/2969760158 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5c09fb60 con 0x7f0a5c09e930 2023-10-03T10:12:52.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.702+0000 7f0a6082d700 1 --2- 172.21.15.3:0/2969760158 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0a5c09f240 0x7f0a5c1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:52.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.702+0000 7f0a6102e700 1 --2- 172.21.15.3:0/2969760158 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0a5c09e930 0x7f0a5c09ed00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0a580098f0 tx=0x7f0a58009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f6ab1e5ede1e96d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:52.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.702+0000 7f0a53fff700 1 -- 172.21.15.3:0/2969760158 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a58011830 con 0x7f0a5c09e930 2023-10-03T10:12:52.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.702+0000 7f0a53fff700 1 -- 172.21.15.3:0/2969760158 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a58011990 con 0x7f0a5c09e930 2023-10-03T10:12:52.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.702+0000 7f0a53fff700 1 -- 172.21.15.3:0/2969760158 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a58011d20 con 0x7f0a5c09e930 2023-10-03T10:12:52.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.703+0000 7f0a63292700 1 -- 172.21.15.3:0/2969760158 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0a5c09e930 msgr2=0x7f0a5c09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:52.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.703+0000 7f0a63292700 1 --2- 172.21.15.3:0/2969760158 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0a5c09e930 0x7f0a5c09ed00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0a580098f0 tx=0x7f0a58009c00 comp rx=0 tx=0).stop 2023-10-03T10:12:52.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.703+0000 7f0a63292700 1 -- 172.21.15.3:0/2969760158 shutdown_connections 2023-10-03T10:12:52.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.703+0000 7f0a63292700 1 --2- 172.21.15.3:0/2969760158 >> v2:172.21.15.3:3300/0 conn(0x7f0a5c09f650 0x7f0a5c131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.703+0000 7f0a63292700 1 --2- 172.21.15.3:0/2969760158 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0a5c09e930 0x7f0a5c09ed00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.703+0000 7f0a63292700 1 --2- 172.21.15.3:0/2969760158 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0a5c09f240 0x7f0a5c1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.703+0000 7f0a63292700 1 -- 172.21.15.3:0/2969760158 >> 172.21.15.3:0/2969760158 conn(0x7f0a5c0a4210 msgr2=0x7f0a5c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:52.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.703+0000 7f0a63292700 1 -- 172.21.15.3:0/2969760158 shutdown_connections 2023-10-03T10:12:52.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.703+0000 7f0a63292700 1 -- 172.21.15.3:0/2969760158 wait complete. 2023-10-03T10:12:52.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.704+0000 7f0a63292700 1 Processor -- start 2023-10-03T10:12:52.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.704+0000 7f0a63292700 1 -- start start 2023-10-03T10:12:52.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.705+0000 7f0a63292700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0a5c09e930 0x7f0a5c1a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:52.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.705+0000 7f0a63292700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0a5c09f240 0x7f0a5c1a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:52.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.705+0000 7f0a63292700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a5c09f650 0x7f0a5c1a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:52.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.705+0000 7f0a63292700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c1a5220 con 0x7f0a5c09f650 2023-10-03T10:12:52.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.705+0000 7f0a63292700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c1a5980 con 0x7f0a5c09f240 2023-10-03T10:12:52.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.705+0000 7f0a63292700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c1a5fa0 con 0x7f0a5c09e930 2023-10-03T10:12:52.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.705+0000 7f0a6102e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0a5c09e930 0x7f0a5c1a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:52.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.705+0000 7f0a6082d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0a5c09f240 0x7f0a5c1a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:52.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.705+0000 7f0a6102e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0a5c09e930 0x7f0a5c1a3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38788/0 (socket says 172.21.15.3:38788) 2023-10-03T10:12:52.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.705+0000 7f0a6102e700 1 -- 172.21.15.3:0/1489243306 learned_addr learned my addr 172.21.15.3:0/1489243306 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:52.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.705+0000 7f0a6182f700 1 --2- 172.21.15.3:0/1489243306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a5c09f650 0x7f0a5c1a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:52.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.705+0000 7f0a6102e700 1 -- 172.21.15.3:0/1489243306 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0a5c09f240 msgr2=0x7f0a5c1a42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:52.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.705+0000 7f0a6102e700 1 --2- 172.21.15.3:0/1489243306 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0a5c09f240 0x7f0a5c1a42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.705+0000 7f0a6102e700 1 -- 172.21.15.3:0/1489243306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a5c09f650 msgr2=0x7f0a5c1a47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:52.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.705+0000 7f0a6102e700 1 --2- 172.21.15.3:0/1489243306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a5c09f650 0x7f0a5c1a47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.705+0000 7f0a6102e700 1 -- 172.21.15.3:0/1489243306 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a4c0094a0 con 0x7f0a5c09e930 2023-10-03T10:12:52.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.705+0000 7f0a6102e700 1 --2- 172.21.15.3:0/1489243306 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0a5c09e930 0x7f0a5c1a3d70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0a58009c40 tx=0x7f0a58004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:52.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.706+0000 7f0a51ffb700 1 -- 172.21.15.3:0/1489243306 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a58004240 con 0x7f0a5c09e930 2023-10-03T10:12:52.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.706+0000 7f0a51ffb700 1 -- 172.21.15.3:0/1489243306 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a5801f070 con 0x7f0a5c09e930 2023-10-03T10:12:52.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.706+0000 7f0a63292700 1 -- 172.21.15.3:0/1489243306 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a58009550 con 0x7f0a5c09e930 2023-10-03T10:12:52.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.706+0000 7f0a51ffb700 1 -- 172.21.15.3:0/1489243306 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a580045e0 con 0x7f0a5c09e930 2023-10-03T10:12:52.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.706+0000 7f0a63292700 1 -- 172.21.15.3:0/1489243306 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a5c0a0190 con 0x7f0a5c09e930 2023-10-03T10:12:52.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.707+0000 7f0a51ffb700 1 -- 172.21.15.3:0/1489243306 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f0a58021030 con 0x7f0a5c09e930 2023-10-03T10:12:52.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.707+0000 7f0a51ffb700 1 -- 172.21.15.3:0/1489243306 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f0a58040f90 con 0x7f0a5c09e930 2023-10-03T10:12:52.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.707+0000 7f0a477fe700 1 -- 172.21.15.3:0/1489243306 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a340052e0 con 0x7f0a5c09e930 2023-10-03T10:12:52.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.708+0000 7f0a6082d700 1 -- 172.21.15.3:0/1489243306 >> v1:172.21.15.3:6816/127025 conn(0x7f0a4002f510 legacy=0x7f0a400319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:52.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.713+0000 7f0a51ffb700 1 -- 172.21.15.3:0/1489243306 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0a58015070 con 0x7f0a5c09e930 2023-10-03T10:12:52.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.715+0000 7fdc2016e700 1 Processor -- start 2023-10-03T10:12:52.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.715+0000 7fdc2016e700 1 -- start start 2023-10-03T10:12:52.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.716+0000 7fdc2016e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc1806a770 0x7fdc1809d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:52.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.716+0000 7fdc2016e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdc1809dc30 0x7fdc1809e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:52.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.716+0000 7fdc2016e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc1809a810 con 0x7fdc1813c1e0 2023-10-03T10:12:52.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.716+0000 7fdc2016e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1806ab40 con 0x7fdc1809dc30 2023-10-03T10:12:52.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.716+0000 7fdc2016e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1809ebc0 con 0x7fdc1806a770 2023-10-03T10:12:52.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.716+0000 7fdc1df0a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc1806a770 0x7fdc1809d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:52.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.716+0000 7fdc1df0a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc1806a770 0x7fdc1809d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38796/0 (socket says 172.21.15.3:38796) 2023-10-03T10:12:52.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.716+0000 7fdc1df0a700 1 -- 172.21.15.3:0/1117651584 learned_addr learned my addr 172.21.15.3:0/1117651584 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:52.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.716+0000 7fdc1d709700 1 --2- 172.21.15.3:0/1117651584 >> v2:172.21.15.3:3300/0 conn(0x7fdc1809dc30 0x7fdc1809e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:52.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.716+0000 7fdc1df0a700 1 -- 172.21.15.3:0/1117651584 >> v2:172.21.15.3:3300/0 conn(0x7fdc1809dc30 msgr2=0x7fdc1809e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:52.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.716+0000 7fdc1df0a700 1 --2- 172.21.15.3:0/1117651584 >> v2:172.21.15.3:3300/0 conn(0x7fdc1809dc30 0x7fdc1809e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.716+0000 7fdc1df0a700 1 -- 172.21.15.3:0/1117651584 >> v1:172.21.15.3:6789/0 conn(0x7fdc1813c1e0 legacy=0x7fdc1813e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:52.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.716+0000 7fdc1df0a700 1 -- 172.21.15.3:0/1117651584 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1809e920 con 0x7fdc1806a770 2023-10-03T10:12:52.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.716+0000 7fdc1d709700 1 --2- 172.21.15.3:0/1117651584 >> v2:172.21.15.3:3300/0 conn(0x7fdc1809dc30 0x7fdc1809e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:12:52.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.717+0000 7fdc1df0a700 1 --2- 172.21.15.3:0/1117651584 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc1806a770 0x7fdc1809d6f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009920 tx=0x7fdc14009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f80287357bef467f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:52.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.717+0000 7fdc1cf08700 1 -- 172.21.15.3:0/1117651584 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1400ba20 con 0x7fdc1806a770 2023-10-03T10:12:52.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.717+0000 7fdc1cf08700 1 -- 172.21.15.3:0/1117651584 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1400bb80 con 0x7fdc1806a770 2023-10-03T10:12:52.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.717+0000 7fdc1cf08700 1 -- 172.21.15.3:0/1117651584 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc14009e70 con 0x7fdc1806a770 2023-10-03T10:12:52.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.718+0000 7fdc2016e700 1 -- 172.21.15.3:0/1117651584 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc1806a770 msgr2=0x7fdc1809d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:52.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.718+0000 7fdc2016e700 1 --2- 172.21.15.3:0/1117651584 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc1806a770 0x7fdc1809d6f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009920 tx=0x7fdc14009c30 comp rx=0 tx=0).stop 2023-10-03T10:12:52.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.718+0000 7fdc2016e700 1 -- 172.21.15.3:0/1117651584 shutdown_connections 2023-10-03T10:12:52.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.718+0000 7fdc2016e700 1 --2- 172.21.15.3:0/1117651584 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc1806a770 0x7fdc1809d6f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.718+0000 7fdc2016e700 1 --2- 172.21.15.3:0/1117651584 >> v2:172.21.15.3:3300/0 conn(0x7fdc1809dc30 0x7fdc1809e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.718+0000 7fdc2016e700 1 -- 172.21.15.3:0/1117651584 >> 172.21.15.3:0/1117651584 conn(0x7fdc18066f50 msgr2=0x7fdc1809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:52.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.718+0000 7fdc2016e700 1 -- 172.21.15.3:0/1117651584 shutdown_connections 2023-10-03T10:12:52.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.718+0000 7fdc2016e700 1 -- 172.21.15.3:0/1117651584 wait complete. 2023-10-03T10:12:52.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.719+0000 7fdc2016e700 1 Processor -- start 2023-10-03T10:12:52.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.719+0000 7fdc2016e700 1 -- start start 2023-10-03T10:12:52.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.719+0000 7fdc2016e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc1809dc30 0x7fdc1813e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:52.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.719+0000 7fdc2016e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc1813c1e0 0x7fdc1813a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:52.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.719+0000 7fdc2016e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc1813ac60 0x7fdc1813b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:52.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.719+0000 7fdc2016e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1813ef70 con 0x7fdc1813ac60 2023-10-03T10:12:52.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.719+0000 7fdc2016e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1811dba0 con 0x7fdc1809dc30 2023-10-03T10:12:52.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.719+0000 7fdc2016e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1813bf20 con 0x7fdc1813c1e0 2023-10-03T10:12:52.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.720+0000 7fdc1d709700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc1813c1e0 0x7fdc1813a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:52.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.720+0000 7fdc1d709700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc1813c1e0 0x7fdc1813a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38798/0 (socket says 172.21.15.3:38798) 2023-10-03T10:12:52.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.720+0000 7fdc1d709700 1 -- 172.21.15.3:0/3224404164 learned_addr learned my addr 172.21.15.3:0/3224404164 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:52.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.720+0000 7fdc1e70b700 1 --2- 172.21.15.3:0/3224404164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc1813ac60 0x7fdc1813b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:52.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.720+0000 7fdc1d709700 1 -- 172.21.15.3:0/3224404164 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc1809dc30 msgr2=0x7fdc1813e370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:52.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.720+0000 7fdc1d709700 1 --2- 172.21.15.3:0/3224404164 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc1809dc30 0x7fdc1813e370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.720+0000 7fdc1d709700 1 -- 172.21.15.3:0/3224404164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc1813ac60 msgr2=0x7fdc1813b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:52.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.720+0000 7fdc1d709700 1 --2- 172.21.15.3:0/3224404164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc1813ac60 0x7fdc1813b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.720+0000 7fdc1d709700 1 -- 172.21.15.3:0/3224404164 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc14009550 con 0x7fdc1813c1e0 2023-10-03T10:12:52.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.720+0000 7fdc1d709700 1 --2- 172.21.15.3:0/3224404164 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc1813c1e0 0x7fdc1813a720 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800e7e0 tx=0x7fdc0800eaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:52.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.720+0000 7fdc0e7fc700 1 -- 172.21.15.3:0/3224404164 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc08013610 con 0x7fdc1813c1e0 2023-10-03T10:12:52.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.720+0000 7fdc2016e700 1 -- 172.21.15.3:0/3224404164 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1809f1a0 con 0x7fdc1813c1e0 2023-10-03T10:12:52.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.721+0000 7fdc2016e700 1 -- 172.21.15.3:0/3224404164 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1811ddd0 con 0x7fdc1813c1e0 2023-10-03T10:12:52.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.721+0000 7fdc0e7fc700 1 -- 172.21.15.3:0/3224404164 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc08013770 con 0x7fdc1813c1e0 2023-10-03T10:12:52.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.721+0000 7fdc0e7fc700 1 -- 172.21.15.3:0/3224404164 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc08013ae0 con 0x7fdc1813c1e0 2023-10-03T10:12:52.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.722+0000 7fdc0e7fc700 1 -- 172.21.15.3:0/3224404164 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fdc08013c40 con 0x7fdc1813c1e0 2023-10-03T10:12:52.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.722+0000 7fdc2016e700 1 -- 172.21.15.3:0/3224404164 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc000052e0 con 0x7fdc1813c1e0 2023-10-03T10:12:52.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.722+0000 7fdc1df0a700 1 -- 172.21.15.3:0/3224404164 >> v1:172.21.15.3:6816/127025 conn(0x7fdbfc02f510 legacy=0x7fdbfc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:52.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.722+0000 7fdc0e7fc700 1 -- 172.21.15.3:0/3224404164 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7fdc0803feb0 con 0x7fdc1813c1e0 2023-10-03T10:12:52.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.727+0000 7fdc0e7fc700 1 -- 172.21.15.3:0/3224404164 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdc08017070 con 0x7fdc1813c1e0 2023-10-03T10:12:52.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.730+0000 7fb991816700 1 Processor -- start 2023-10-03T10:12:52.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.730+0000 7fb991816700 1 -- start start 2023-10-03T10:12:52.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.731+0000 7fb991816700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb98c136830 0x7fb98c138c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:52.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.731+0000 7fb991816700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb98c139150 0x7fb98c13b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:52.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.731+0000 7fb991816700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb98c06f720 con 0x7fb98c1338c0 2023-10-03T10:12:52.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.731+0000 7fb991816700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb98c070c80 con 0x7fb98c139150 2023-10-03T10:12:52.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.731+0000 7fb991816700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c06fce0 con 0x7fb98c136830 2023-10-03T10:12:52.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.731+0000 7fb98affd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb98c1338c0 0x7fb98c133c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50780/0 (socket says 172.21.15.3:50780) 2023-10-03T10:12:52.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.731+0000 7fb98affd700 1 -- 172.21.15.3:0/2364453153 learned_addr learned my addr 172.21.15.3:0/2364453153 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:52.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.731+0000 7fb98a7fc700 1 --2- 172.21.15.3:0/2364453153 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb98c136830 0x7fb98c138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:52.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.731+0000 7fb98b7fe700 1 --2- 172.21.15.3:0/2364453153 >> v2:172.21.15.3:3300/0 conn(0x7fb98c139150 0x7fb98c13b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:52.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.732+0000 7fb98a7fc700 1 -- 172.21.15.3:0/2364453153 >> v2:172.21.15.3:3300/0 conn(0x7fb98c139150 msgr2=0x7fb98c13b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:52.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.732+0000 7fb98a7fc700 1 --2- 172.21.15.3:0/2364453153 >> v2:172.21.15.3:3300/0 conn(0x7fb98c139150 0x7fb98c13b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.732+0000 7fb98a7fc700 1 -- 172.21.15.3:0/2364453153 >> v1:172.21.15.3:6789/0 conn(0x7fb98c1338c0 legacy=0x7fb98c133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:52.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.732+0000 7fb98a7fc700 1 -- 172.21.15.3:0/2364453153 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98c1336a0 con 0x7fb98c136830 2023-10-03T10:12:52.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.732+0000 7fb98a7fc700 1 --2- 172.21.15.3:0/2364453153 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb98c136830 0x7fb98c138c10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb980009840 tx=0x7fb980009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff8eb52fe5fd107 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:52.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.732+0000 7fb989ffb700 1 -- 172.21.15.3:0/2364453153 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb980004230 con 0x7fb98c136830 2023-10-03T10:12:52.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.733+0000 7fb989ffb700 1 -- 172.21.15.3:0/2364453153 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb980004390 con 0x7fb98c136830 2023-10-03T10:12:52.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.733+0000 7fb989ffb700 1 -- 172.21.15.3:0/2364453153 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9800036a0 con 0x7fb98c136830 2023-10-03T10:12:52.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.733+0000 7fb991816700 1 -- 172.21.15.3:0/2364453153 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb98c136830 msgr2=0x7fb98c138c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:52.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.733+0000 7fb991816700 1 --2- 172.21.15.3:0/2364453153 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb98c136830 0x7fb98c138c10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb980009840 tx=0x7fb980009b50 comp rx=0 tx=0).stop 2023-10-03T10:12:52.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.733+0000 7fb991816700 1 -- 172.21.15.3:0/2364453153 shutdown_connections 2023-10-03T10:12:52.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.733+0000 7fb991816700 1 --2- 172.21.15.3:0/2364453153 >> v2:172.21.15.3:3300/0 conn(0x7fb98c139150 0x7fb98c13b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.733+0000 7fb991816700 1 --2- 172.21.15.3:0/2364453153 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb98c136830 0x7fb98c138c10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.733+0000 7fb991816700 1 -- 172.21.15.3:0/2364453153 >> 172.21.15.3:0/2364453153 conn(0x7fb98c0a2aa0 msgr2=0x7fb98c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:52.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.733+0000 7fb991816700 1 -- 172.21.15.3:0/2364453153 shutdown_connections 2023-10-03T10:12:52.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.734+0000 7fb991816700 1 -- 172.21.15.3:0/2364453153 wait complete. 2023-10-03T10:12:52.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.735+0000 7fb991816700 1 Processor -- start 2023-10-03T10:12:52.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.735+0000 7fb991816700 1 -- start start 2023-10-03T10:12:52.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.735+0000 7fb991816700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb98c1338c0 0x7fb98c1c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:52.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.735+0000 7fb991816700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb98c136830 0x7fb98c1c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:52.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.735+0000 7fb991816700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb98c139150 0x7fb98c119f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:52.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.735+0000 7fb991816700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c1be2d0 con 0x7fb98c1338c0 2023-10-03T10:12:52.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.735+0000 7fb991816700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c12fd10 con 0x7fb98c136830 2023-10-03T10:12:52.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.735+0000 7fb991816700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c06f960 con 0x7fb98c139150 2023-10-03T10:12:52.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.735+0000 7fb98affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb98c1338c0 0x7fb98c1c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:52.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.736+0000 7fb98affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb98c1338c0 0x7fb98c1c8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56136/0 (socket says 172.21.15.3:56136) 2023-10-03T10:12:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.736+0000 7fb98affd700 1 -- 172.21.15.3:0/377889809 learned_addr learned my addr 172.21.15.3:0/377889809 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.736+0000 7fb98b7fe700 1 --2- 172.21.15.3:0/377889809 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb98c139150 0x7fb98c119f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.736+0000 7fb98a7fc700 1 --2- 172.21.15.3:0/377889809 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb98c136830 0x7fb98c1c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.736+0000 7fb98affd700 1 -- 172.21.15.3:0/377889809 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb98c139150 msgr2=0x7fb98c119f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:52.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.736+0000 7fb98affd700 1 --2- 172.21.15.3:0/377889809 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb98c139150 0x7fb98c119f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.736+0000 7fb98affd700 1 -- 172.21.15.3:0/377889809 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb98c136830 msgr2=0x7fb98c1c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:52.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.736+0000 7fb98affd700 1 --2- 172.21.15.3:0/377889809 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb98c136830 0x7fb98c1c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.736+0000 7fb98affd700 1 -- 172.21.15.3:0/377889809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9800094a0 con 0x7fb98c1338c0 2023-10-03T10:12:52.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.736+0000 7fb98affd700 1 --2- 172.21.15.3:0/377889809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb98c1338c0 0x7fb98c1c8630 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00c780 tx=0x7fb97c00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:52.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.736+0000 7fb973fff700 1 -- 172.21.15.3:0/377889809 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97c004c80 con 0x7fb98c1338c0 2023-10-03T10:12:52.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.736+0000 7fb973fff700 1 -- 172.21.15.3:0/377889809 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97c004de0 con 0x7fb98c1338c0 2023-10-03T10:12:52.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.736+0000 7fb973fff700 1 -- 172.21.15.3:0/377889809 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97c0072d0 con 0x7fb98c1338c0 2023-10-03T10:12:52.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.737+0000 7fb991816700 1 -- 172.21.15.3:0/377889809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb98c11d3c0 con 0x7fb98c1338c0 2023-10-03T10:12:52.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.737+0000 7fb991816700 1 -- 172.21.15.3:0/377889809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb98c11e0f0 con 0x7fb98c1338c0 2023-10-03T10:12:52.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.738+0000 7fb973fff700 1 -- 172.21.15.3:0/377889809 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fb97c007430 con 0x7fb98c1338c0 2023-10-03T10:12:52.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.739+0000 7fb973fff700 1 -- 172.21.15.3:0/377889809 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7fb97c044020 con 0x7fb98c1338c0 2023-10-03T10:12:52.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.739+0000 7fb98a7fc700 1 -- 172.21.15.3:0/377889809 >> v1:172.21.15.3:6816/127025 conn(0x7fb97802f510 legacy=0x7fb9780319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:52.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.739+0000 7fb991816700 1 -- 172.21.15.3:0/377889809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb98c074ea0 con 0x7fb98c1338c0 2023-10-03T10:12:52.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.749+0000 7fb973fff700 1 -- 172.21.15.3:0/377889809 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb97c0076e0 con 0x7fb98c1338c0 2023-10-03T10:12:52.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.887+0000 7f09555e0700 1 -- 172.21.15.3:0/1661425628 >> v1:172.21.15.3:6816/127025 conn(0x7f093402f510 legacy=0x7f09340319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:52.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.909+0000 7f0a6082d700 1 -- 172.21.15.3:0/1489243306 >> v1:172.21.15.3:6816/127025 conn(0x7f0a4002f510 legacy=0x7f0a400319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:52.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.923+0000 7fdc1df0a700 1 -- 172.21.15.3:0/3224404164 >> v1:172.21.15.3:6816/127025 conn(0x7fdbfc02f510 legacy=0x7fdbfc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:52.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.933+0000 7f0a477fe700 1 -- 172.21.15.3:0/1489243306 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0a34005860 con 0x7f0a5c09e930 2023-10-03T10:12:52.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.933+0000 7f0a51ffb700 1 -- 172.21.15.3:0/1489243306 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0a580197f0 con 0x7f0a5c09e930 2023-10-03T10:12:52.934 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:12:52.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.936+0000 7f0a477fe700 1 -- 172.21.15.3:0/1489243306 >> v1:172.21.15.3:6816/127025 conn(0x7f0a4002f510 legacy=0x7f0a400319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:52.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.936+0000 7f0a477fe700 1 -- 172.21.15.3:0/1489243306 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0a5c09e930 msgr2=0x7f0a5c1a3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:52.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.936+0000 7f0a477fe700 1 --2- 172.21.15.3:0/1489243306 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0a5c09e930 0x7f0a5c1a3d70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0a58009c40 tx=0x7f0a58004750 comp rx=0 tx=0).stop 2023-10-03T10:12:52.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.936+0000 7f0a477fe700 1 -- 172.21.15.3:0/1489243306 shutdown_connections 2023-10-03T10:12:52.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.936+0000 7f0a477fe700 1 --2- 172.21.15.3:0/1489243306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a5c09f650 0x7f0a5c1a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.936+0000 7f0a477fe700 1 --2- 172.21.15.3:0/1489243306 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0a5c09e930 0x7f0a5c1a3d70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.936+0000 7f0a477fe700 1 --2- 172.21.15.3:0/1489243306 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0a5c09f240 0x7f0a5c1a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.936+0000 7f0a477fe700 1 -- 172.21.15.3:0/1489243306 >> 172.21.15.3:0/1489243306 conn(0x7f0a5c0a4210 msgr2=0x7f0a5c1b8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:52.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.936+0000 7f0a477fe700 1 -- 172.21.15.3:0/1489243306 shutdown_connections 2023-10-03T10:12:52.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.936+0000 7f0a477fe700 1 -- 172.21.15.3:0/1489243306 wait complete. 2023-10-03T10:12:52.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.938+0000 7fdc2016e700 1 -- 172.21.15.3:0/3224404164 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdc00005860 con 0x7fdc1813c1e0 2023-10-03T10:12:52.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.938+0000 7fdc0e7fc700 1 -- 172.21.15.3:0/3224404164 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdc0802b090 con 0x7fdc1813c1e0 2023-10-03T10:12:52.938 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:12:52.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.940+0000 7fb98a7fc700 1 -- 172.21.15.3:0/377889809 >> v1:172.21.15.3:6816/127025 conn(0x7fb97802f510 legacy=0x7fb9780319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:52.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.941+0000 7fdc2016e700 1 -- 172.21.15.3:0/3224404164 >> v1:172.21.15.3:6816/127025 conn(0x7fdbfc02f510 legacy=0x7fdbfc0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:52.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.941+0000 7fdc2016e700 1 -- 172.21.15.3:0/3224404164 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc1813c1e0 msgr2=0x7fdc1813a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:52.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.941+0000 7fdc2016e700 1 --2- 172.21.15.3:0/3224404164 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc1813c1e0 0x7fdc1813a720 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800e7e0 tx=0x7fdc0800eaf0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.941+0000 7fdc2016e700 1 -- 172.21.15.3:0/3224404164 shutdown_connections 2023-10-03T10:12:52.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.941+0000 7fdc2016e700 1 --2- 172.21.15.3:0/3224404164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc1813ac60 0x7fdc1813b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.941+0000 7fdc2016e700 1 --2- 172.21.15.3:0/3224404164 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc1809dc30 0x7fdc1813e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.941+0000 7fdc2016e700 1 --2- 172.21.15.3:0/3224404164 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc1813c1e0 0x7fdc1813a720 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.941+0000 7fdc2016e700 1 -- 172.21.15.3:0/3224404164 >> 172.21.15.3:0/3224404164 conn(0x7fdc18066f50 msgr2=0x7fdc180a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:52.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.941+0000 7fdc2016e700 1 -- 172.21.15.3:0/3224404164 shutdown_connections 2023-10-03T10:12:52.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.941+0000 7fdc2016e700 1 -- 172.21.15.3:0/3224404164 wait complete. 2023-10-03T10:12:52.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.944+0000 7f0957844700 1 -- 172.21.15.3:0/1661425628 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f09501c21b0 con 0x7f095009e1d0 2023-10-03T10:12:52.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.944+0000 7f0945ffb700 1 -- 172.21.15.3:0/1661425628 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f094002d7d0 con 0x7f095009e1d0 2023-10-03T10:12:52.944 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:12:52.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.946+0000 7f0957844700 1 -- 172.21.15.3:0/1661425628 >> v1:172.21.15.3:6816/127025 conn(0x7f093402f510 legacy=0x7f09340319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:52.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.946+0000 7f0957844700 1 -- 172.21.15.3:0/1661425628 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f095009e1d0 msgr2=0x7f0950119900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:52.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.946+0000 7f0957844700 1 --2- 172.21.15.3:0/1661425628 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f095009e1d0 0x7f0950119900 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0940009c40 tx=0x7f0940004750 comp rx=0 tx=0).stop 2023-10-03T10:12:52.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.946+0000 7f0957844700 1 -- 172.21.15.3:0/1661425628 shutdown_connections 2023-10-03T10:12:52.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.946+0000 7f0957844700 1 --2- 172.21.15.3:0/1661425628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0950135e50 0x7f0950119e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.946+0000 7f0957844700 1 --2- 172.21.15.3:0/1661425628 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f095009d6f0 0x7f09501c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.946+0000 7f0957844700 1 --2- 172.21.15.3:0/1661425628 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f095009e1d0 0x7f0950119900 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.947+0000 7f0957844700 1 -- 172.21.15.3:0/1661425628 >> 172.21.15.3:0/1661425628 conn(0x7f0950066f50 msgr2=0x7f0950132ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:52.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.947+0000 7f0957844700 1 -- 172.21.15.3:0/1661425628 shutdown_connections 2023-10-03T10:12:52.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.947+0000 7f0957844700 1 -- 172.21.15.3:0/1661425628 wait complete. 2023-10-03T10:12:52.958 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:12:52.959 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:12:52.962 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:12:52.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.996+0000 7fb991816700 1 -- 172.21.15.3:0/377889809 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb98c06b9e0 con 0x7fb98c1338c0 2023-10-03T10:12:52.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.996+0000 7fb973fff700 1 -- 172.21.15.3:0/377889809 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb97c0102c0 con 0x7fb98c1338c0 2023-10-03T10:12:52.996 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:12:52.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.998+0000 7fb991816700 1 -- 172.21.15.3:0/377889809 >> v1:172.21.15.3:6816/127025 conn(0x7fb97802f510 legacy=0x7fb9780319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:52.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.998+0000 7fb991816700 1 -- 172.21.15.3:0/377889809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb98c1338c0 msgr2=0x7fb98c1c8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:52.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.998+0000 7fb991816700 1 --2- 172.21.15.3:0/377889809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb98c1338c0 0x7fb98c1c8630 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00c780 tx=0x7fb97c00ca90 comp rx=0 tx=0).stop 2023-10-03T10:12:52.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.998+0000 7fb991816700 1 -- 172.21.15.3:0/377889809 shutdown_connections 2023-10-03T10:12:52.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.998+0000 7fb991816700 1 --2- 172.21.15.3:0/377889809 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb98c139150 0x7fb98c119f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.998+0000 7fb991816700 1 --2- 172.21.15.3:0/377889809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb98c1338c0 0x7fb98c1c8630 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.998+0000 7fb991816700 1 --2- 172.21.15.3:0/377889809 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb98c136830 0x7fb98c1c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:52.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.998+0000 7fb991816700 1 -- 172.21.15.3:0/377889809 >> 172.21.15.3:0/377889809 conn(0x7fb98c0a2aa0 msgr2=0x7fb98c09de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:52.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.998+0000 7fb991816700 1 -- 172.21.15.3:0/377889809 shutdown_connections 2023-10-03T10:12:52.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:52.998+0000 7fb991816700 1 -- 172.21.15.3:0/377889809 wait complete. 2023-10-03T10:12:53.012 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:12:53.959 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:12:53.961 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:12:53.963 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:12:54.013 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:12:54.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.114+0000 7f5163784700 1 Processor -- start 2023-10-03T10:12:54.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.115+0000 7f144e8c5700 1 Processor -- start 2023-10-03T10:12:54.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.115+0000 7f5163784700 1 -- start start 2023-10-03T10:12:54.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.115+0000 7f5163784700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f515c09e1d0 0x7f515c135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:54.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.115+0000 7f5163784700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f515c135e50 0x7f515c138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:54.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.115+0000 7f5163784700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f515c09f130 con 0x7f515c09d6f0 2023-10-03T10:12:54.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.115+0000 7f5163784700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f515c09f520 con 0x7f515c135e50 2023-10-03T10:12:54.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.115+0000 7f5163784700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f515c09e930 con 0x7f515c09e1d0 2023-10-03T10:12:54.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.115+0000 7f144e8c5700 1 -- start start 2023-10-03T10:12:54.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.116+0000 7f5161d21700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f515c135e50 0x7f515c138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:54.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.116+0000 7f5161d21700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f515c135e50 0x7f515c138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42820/0 (socket says 172.21.15.3:42820) 2023-10-03T10:12:54.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.116+0000 7f5161d21700 1 -- 172.21.15.3:0/2314173837 learned_addr learned my addr 172.21.15.3:0/2314173837 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:54.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.116+0000 7f5160d1f700 1 --2- 172.21.15.3:0/2314173837 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f515c09e1d0 0x7f515c135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:54.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.116+0000 7f5160d1f700 1 -- 172.21.15.3:0/2314173837 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f515c135e50 msgr2=0x7f515c138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:54.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.116+0000 7f5160d1f700 1 --2- 172.21.15.3:0/2314173837 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f515c135e50 0x7f515c138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.116+0000 7f5160d1f700 1 -- 172.21.15.3:0/2314173837 >> v1:172.21.15.3:6789/0 conn(0x7f515c09d6f0 legacy=0x7f515c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:54.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.116+0000 7f5160d1f700 1 -- 172.21.15.3:0/2314173837 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f515c09b2d0 con 0x7f515c09e1d0 2023-10-03T10:12:54.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.116+0000 7f5160d1f700 1 --2- 172.21.15.3:0/2314173837 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f515c09e1d0 0x7f515c135910 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f514c009870 tx=0x7f514c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93208b142e95d23d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:54.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.116+0000 7f144e8c5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f144809eca0 0x7f144813cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:54.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.116+0000 7f144e8c5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f144806f050 0x7f144813f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:54.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.116+0000 7f144e8c5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1448070ee0 con 0x7f144806e570 2023-10-03T10:12:54.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.116+0000 7f144e8c5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1448071980 con 0x7f144809eca0 2023-10-03T10:12:54.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.116+0000 7f144e8c5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1448071480 con 0x7f144806f050 2023-10-03T10:12:54.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.116+0000 7f144ce62700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f144806f050 0x7f144813f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:54.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.116+0000 7f144ce62700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f144806f050 0x7f144813f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54566/0 (socket says 172.21.15.3:54566) 2023-10-03T10:12:54.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.116+0000 7f144ce62700 1 -- 172.21.15.3:0/2545317205 learned_addr learned my addr 172.21.15.3:0/2545317205 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:54.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.117+0000 7f5153fff700 1 -- 172.21.15.3:0/2314173837 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c010a90 con 0x7f515c09e1d0 2023-10-03T10:12:54.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.117+0000 7f144ce62700 1 -- 172.21.15.3:0/2545317205 >> v2:172.21.15.3:3300/0 conn(0x7f144809eca0 msgr2=0x7f144813cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:54.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.117+0000 7f144ce62700 1 --2- 172.21.15.3:0/2545317205 >> v2:172.21.15.3:3300/0 conn(0x7f144809eca0 0x7f144813cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.117+0000 7f144ce62700 1 -- 172.21.15.3:0/2545317205 >> v1:172.21.15.3:6789/0 conn(0x7f144806e570 legacy=0x7f144806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:54.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.117+0000 7f144ce62700 1 -- 172.21.15.3:0/2545317205 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1448071b90 con 0x7f144806f050 2023-10-03T10:12:54.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.117+0000 7f5153fff700 1 -- 172.21.15.3:0/2314173837 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514c010bf0 con 0x7f515c09e1d0 2023-10-03T10:12:54.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.117+0000 7f5153fff700 1 -- 172.21.15.3:0/2314173837 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c018440 con 0x7f515c09e1d0 2023-10-03T10:12:54.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.117+0000 7f144ce62700 1 --2- 172.21.15.3:0/2545317205 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f144806f050 0x7f144813f3f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1440009e50 tx=0x7f144000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca62f06d21a4a698 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:54.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.117+0000 7f1446ffd700 1 -- 172.21.15.3:0/2545317205 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144000e070 con 0x7f144806f050 2023-10-03T10:12:54.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.117+0000 7f1446ffd700 1 -- 172.21.15.3:0/2545317205 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1440003a20 con 0x7f144806f050 2023-10-03T10:12:54.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.117+0000 7f1446ffd700 1 -- 172.21.15.3:0/2545317205 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1440003dd0 con 0x7f144806f050 2023-10-03T10:12:54.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.118+0000 7f5163784700 1 -- 172.21.15.3:0/2314173837 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f515c09e1d0 msgr2=0x7f515c135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:54.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.118+0000 7f5163784700 1 --2- 172.21.15.3:0/2314173837 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f515c09e1d0 0x7f515c135910 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f514c009870 tx=0x7f514c009b80 comp rx=0 tx=0).stop 2023-10-03T10:12:54.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.118+0000 7f144e8c5700 1 -- 172.21.15.3:0/2545317205 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f144806f050 msgr2=0x7f144813f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:54.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.118+0000 7f144e8c5700 1 --2- 172.21.15.3:0/2545317205 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f144806f050 0x7f144813f3f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1440009e50 tx=0x7f144000b330 comp rx=0 tx=0).stop 2023-10-03T10:12:54.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.118+0000 7f5163784700 1 -- 172.21.15.3:0/2314173837 shutdown_connections 2023-10-03T10:12:54.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.118+0000 7f5163784700 1 --2- 172.21.15.3:0/2314173837 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f515c135e50 0x7f515c138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.118+0000 7f5163784700 1 --2- 172.21.15.3:0/2314173837 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f515c09e1d0 0x7f515c135910 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.118+0000 7f5163784700 1 -- 172.21.15.3:0/2314173837 >> 172.21.15.3:0/2314173837 conn(0x7f515c066f50 msgr2=0x7f515c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:54.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.118+0000 7f144e8c5700 1 -- 172.21.15.3:0/2545317205 shutdown_connections 2023-10-03T10:12:54.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.118+0000 7f144e8c5700 1 --2- 172.21.15.3:0/2545317205 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f144806f050 0x7f144813f3f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.118+0000 7f144e8c5700 1 --2- 172.21.15.3:0/2545317205 >> v2:172.21.15.3:3300/0 conn(0x7f144809eca0 0x7f144813cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.118+0000 7f144e8c5700 1 -- 172.21.15.3:0/2545317205 >> 172.21.15.3:0/2545317205 conn(0x7f14480a2aa0 msgr2=0x7f14480a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:54.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.118+0000 7f144e8c5700 1 -- 172.21.15.3:0/2545317205 shutdown_connections 2023-10-03T10:12:54.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.118+0000 7f144e8c5700 1 -- 172.21.15.3:0/2545317205 wait complete. 2023-10-03T10:12:54.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.118+0000 7f5163784700 1 -- 172.21.15.3:0/2314173837 shutdown_connections 2023-10-03T10:12:54.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.118+0000 7f5163784700 1 -- 172.21.15.3:0/2314173837 wait complete. 2023-10-03T10:12:54.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.119+0000 7f5163784700 1 Processor -- start 2023-10-03T10:12:54.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.119+0000 7f5163784700 1 -- start start 2023-10-03T10:12:54.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.119+0000 7f144e8c5700 1 Processor -- start 2023-10-03T10:12:54.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.120+0000 7f5163784700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f515c09d6f0 0x7f515c119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:54.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.120+0000 7f5163784700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f515c09e1d0 0x7f515c1199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:54.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.120+0000 7f5163784700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f515c135e50 0x7f515c119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:54.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.120+0000 7f5163784700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515c0a1d70 con 0x7f515c09e1d0 2023-10-03T10:12:54.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.120+0000 7f5163784700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f515c11c8b0 con 0x7f515c135e50 2023-10-03T10:12:54.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.120+0000 7f5163784700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f515c130110 con 0x7f515c09d6f0 2023-10-03T10:12:54.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.120+0000 7f5161520700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f515c09d6f0 0x7f515c119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:54.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.120+0000 7f5161520700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f515c09d6f0 0x7f515c119480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54576/0 (socket says 172.21.15.3:54576) 2023-10-03T10:12:54.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.120+0000 7f5161520700 1 -- 172.21.15.3:0/3076193130 learned_addr learned my addr 172.21.15.3:0/3076193130 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:54.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.120+0000 7f5161d21700 1 --2- 172.21.15.3:0/3076193130 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f515c135e50 0x7f515c119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:54.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.120+0000 7f5160d1f700 1 --2- 172.21.15.3:0/3076193130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f515c09e1d0 0x7f515c1199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:54.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.120+0000 7f5161520700 1 -- 172.21.15.3:0/3076193130 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f515c135e50 msgr2=0x7f515c119f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:54.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.120+0000 7f5161520700 1 --2- 172.21.15.3:0/3076193130 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f515c135e50 0x7f515c119f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.120+0000 7f5161520700 1 -- 172.21.15.3:0/3076193130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f515c09e1d0 msgr2=0x7f515c1199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:54.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.120+0000 7f5161520700 1 --2- 172.21.15.3:0/3076193130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f515c09e1d0 0x7f515c1199c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.120+0000 7f5161520700 1 -- 172.21.15.3:0/3076193130 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514c0094a0 con 0x7f515c09d6f0 2023-10-03T10:12:54.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.120+0000 7f144e8c5700 1 -- start start 2023-10-03T10:12:54.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.120+0000 7f144e8c5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f144806e570 0x7f14481214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:54.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.120+0000 7f144e8c5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f144806f050 0x7f1448121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:54.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.120+0000 7f144e8c5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1448121f60 0x7f1448136df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:54.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.120+0000 7f144e8c5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144809a450 con 0x7f144806e570 2023-10-03T10:12:54.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.120+0000 7f144e8c5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1448123880 con 0x7f1448121f60 2023-10-03T10:12:54.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.120+0000 7f144e8c5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1448124190 con 0x7f144806f050 2023-10-03T10:12:54.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.121+0000 7f144ce62700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1448121f60 0x7f1448136df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:54.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.121+0000 7f144ce62700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1448121f60 0x7f1448136df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42830/0 (socket says 172.21.15.3:42830) 2023-10-03T10:12:54.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.121+0000 7f144ce62700 1 -- 172.21.15.3:0/1086133829 learned_addr learned my addr 172.21.15.3:0/1086133829 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:54.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.121+0000 7f1447fff700 1 --2- 172.21.15.3:0/1086133829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f144806e570 0x7f14481214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:54.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.121+0000 7f5161520700 1 --2- 172.21.15.3:0/3076193130 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f515c09d6f0 0x7f515c119480 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5158001c20 tx=0x7f5158014330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:54.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.121+0000 7f5151ffb700 1 -- 172.21.15.3:0/3076193130 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5158012470 con 0x7f515c09d6f0 2023-10-03T10:12:54.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.121+0000 7fcab7a4a700 1 Processor -- start 2023-10-03T10:12:54.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.121+0000 7f1447fff700 1 -- 172.21.15.3:0/1086133829 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f144806f050 msgr2=0x7f1448121a20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:54.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.121+0000 7f1447fff700 1 --2- 172.21.15.3:0/1086133829 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f144806f050 0x7f1448121a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.121+0000 7f1447fff700 1 -- 172.21.15.3:0/1086133829 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1448121f60 msgr2=0x7f1448136df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:54.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.121+0000 7f1447fff700 1 --2- 172.21.15.3:0/1086133829 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1448121f60 0x7f1448136df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.121+0000 7f1447fff700 1 -- 172.21.15.3:0/1086133829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1440009b30 con 0x7f144806e570 2023-10-03T10:12:54.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.121+0000 7fcab7a4a700 1 -- start start 2023-10-03T10:12:54.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.122+0000 7f1447fff700 1 --2- 172.21.15.3:0/1086133829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f144806e570 0x7f14481214e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f143800c3e0 tx=0x7f143800c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:54.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.122+0000 7f1444ff9700 1 -- 172.21.15.3:0/1086133829 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143800cea0 con 0x7f144806e570 2023-10-03T10:12:54.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.122+0000 7f1444ff9700 1 -- 172.21.15.3:0/1086133829 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1438005500 con 0x7f144806e570 2023-10-03T10:12:54.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.122+0000 7f1444ff9700 1 -- 172.21.15.3:0/1086133829 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14380038c0 con 0x7f144806e570 2023-10-03T10:12:54.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.122+0000 7f144e8c5700 1 -- 172.21.15.3:0/1086133829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14481229d0 con 0x7f144806e570 2023-10-03T10:12:54.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.122+0000 7f144e8c5700 1 -- 172.21.15.3:0/1086133829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f144813ded0 con 0x7f144806e570 2023-10-03T10:12:54.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.123+0000 7f1444ff9700 1 -- 172.21.15.3:0/1086133829 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f1438003a20 con 0x7f144806e570 2023-10-03T10:12:54.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.121+0000 7f5151ffb700 1 -- 172.21.15.3:0/3076193130 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51580125d0 con 0x7f515c09d6f0 2023-10-03T10:12:54.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.121+0000 7f5163784700 1 -- 172.21.15.3:0/3076193130 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f515c09c700 con 0x7f515c09d6f0 2023-10-03T10:12:54.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.121+0000 7f5151ffb700 1 -- 172.21.15.3:0/3076193130 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51580052f0 con 0x7f515c09d6f0 2023-10-03T10:12:54.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.121+0000 7f5163784700 1 -- 172.21.15.3:0/3076193130 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f515c11c680 con 0x7f515c09d6f0 2023-10-03T10:12:54.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.122+0000 7f5151ffb700 1 -- 172.21.15.3:0/3076193130 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f515800f070 con 0x7f515c09d6f0 2023-10-03T10:12:54.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.124+0000 7f1444ff9700 1 -- 172.21.15.3:0/1086133829 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f143803ec40 con 0x7f144806e570 2023-10-03T10:12:54.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.124+0000 7f14477fe700 1 -- 172.21.15.3:0/1086133829 >> v1:172.21.15.3:6816/127025 conn(0x7f143402f540 legacy=0x7f14340319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:54.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.122+0000 7fcab7a4a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcab0066410 0x7fcab0135d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:54.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.124+0000 7f5151ffb700 1 -- 172.21.15.3:0/3076193130 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f51580421a0 con 0x7f515c09d6f0 2023-10-03T10:12:54.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.124+0000 7f5160d1f700 1 -- 172.21.15.3:0/3076193130 >> v1:172.21.15.3:6816/127025 conn(0x7f514002f560 legacy=0x7f5140031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:54.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.122+0000 7fcab7a4a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcab0066880 0x7fcab0130bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:54.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.122+0000 7fcab7a4a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab006f9a0 con 0x7fcab006f5d0 2023-10-03T10:12:54.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.122+0000 7fcab7a4a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab0071840 con 0x7fcab0066410 2023-10-03T10:12:54.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.122+0000 7fcab7a4a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcab0071a40 con 0x7fcab0066880 2023-10-03T10:12:54.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.122+0000 7fcab57e6700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcab006f5d0 0x7fcab012cd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45324/0 (socket says 172.21.15.3:45324) 2023-10-03T10:12:54.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.122+0000 7fcab57e6700 1 -- 172.21.15.3:0/960799686 learned_addr learned my addr 172.21.15.3:0/960799686 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:54.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.123+0000 7fcab5fe7700 1 --2- 172.21.15.3:0/960799686 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcab0066880 0x7fcab0130bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:54.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.123+0000 7fcab4fe5700 1 --2- 172.21.15.3:0/960799686 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcab0066410 0x7fcab0135d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:54.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.123+0000 7fcaa7fff700 1 -- 172.21.15.3:0/960799686 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2678767253 0 0) 0x7fcab006f9a0 con 0x7fcab006f5d0 2023-10-03T10:12:54.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.123+0000 7fcab5fe7700 1 -- 172.21.15.3:0/960799686 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcab0066410 msgr2=0x7fcab0135d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:54.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.123+0000 7fcab5fe7700 1 --2- 172.21.15.3:0/960799686 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcab0066410 0x7fcab0135d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.123+0000 7fcab5fe7700 1 -- 172.21.15.3:0/960799686 >> v1:172.21.15.3:6789/0 conn(0x7fcab006f5d0 legacy=0x7fcab012cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:54.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.123+0000 7fcab5fe7700 1 -- 172.21.15.3:0/960799686 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab0072220 con 0x7fcab0066880 2023-10-03T10:12:54.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.124+0000 7fcab5fe7700 1 --2- 172.21.15.3:0/960799686 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcab0066880 0x7fcab0130bf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcaac0098a0 tx=0x7fcaac009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1002f7b9259f988d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:54.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.124+0000 7f144e8c5700 1 -- 172.21.15.3:0/1086133829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1448122360 con 0x7f144806e570 2023-10-03T10:12:54.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.124+0000 7fcaa7fff700 1 -- 172.21.15.3:0/960799686 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac011830 con 0x7fcab0066880 2023-10-03T10:12:54.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.124+0000 7fcaa7fff700 1 -- 172.21.15.3:0/960799686 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaac011990 con 0x7fcab0066880 2023-10-03T10:12:54.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.124+0000 7fcaa7fff700 1 -- 172.21.15.3:0/960799686 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac011d20 con 0x7fcab0066880 2023-10-03T10:12:54.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.125+0000 7fcab7a4a700 1 -- 172.21.15.3:0/960799686 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcab0066880 msgr2=0x7fcab0130bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:54.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.125+0000 7fcab7a4a700 1 --2- 172.21.15.3:0/960799686 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcab0066880 0x7fcab0130bf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcaac0098a0 tx=0x7fcaac009bb0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.124+0000 7f5163784700 1 -- 172.21.15.3:0/3076193130 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f515c074ea0 con 0x7f515c09d6f0 2023-10-03T10:12:54.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.125+0000 7fcab7a4a700 1 -- 172.21.15.3:0/960799686 shutdown_connections 2023-10-03T10:12:54.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.125+0000 7fcab7a4a700 1 --2- 172.21.15.3:0/960799686 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcab0066880 0x7fcab0130bf0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.125+0000 7fcab7a4a700 1 --2- 172.21.15.3:0/960799686 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcab0066410 0x7fcab0135d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.125+0000 7fcab7a4a700 1 -- 172.21.15.3:0/960799686 >> 172.21.15.3:0/960799686 conn(0x7fcab00a2aa0 msgr2=0x7fcab00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:54.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.125+0000 7fcab7a4a700 1 -- 172.21.15.3:0/960799686 shutdown_connections 2023-10-03T10:12:54.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.125+0000 7fcab7a4a700 1 -- 172.21.15.3:0/960799686 wait complete. 2023-10-03T10:12:54.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.126+0000 7fcab7a4a700 1 Processor -- start 2023-10-03T10:12:54.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.127+0000 7fcab7a4a700 1 -- start start 2023-10-03T10:12:54.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.127+0000 7fcab7a4a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab0066410 0x7fcab0128de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:54.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.127+0000 7fcab7a4a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcab0066880 0x7fcab0129320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:54.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.127+0000 7fcab7a4a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcab006f5d0 0x7fcab01c3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:54.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.127+0000 7fcab7a4a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab009cf60 con 0x7fcab0066410 2023-10-03T10:12:54.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.127+0000 7fcab7a4a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab009cd70 con 0x7fcab0066880 2023-10-03T10:12:54.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.127+0000 7fcab7a4a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcab009d230 con 0x7fcab006f5d0 2023-10-03T10:12:54.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.127+0000 7fcab57e6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab0066410 0x7fcab0128de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:54.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.127+0000 7fcab57e6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab0066410 0x7fcab0128de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38968/0 (socket says 172.21.15.3:38968) 2023-10-03T10:12:54.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.128+0000 7fcab57e6700 1 -- 172.21.15.3:0/2687888675 learned_addr learned my addr 172.21.15.3:0/2687888675 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:54.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.128+0000 7fcab57e6700 1 -- 172.21.15.3:0/2687888675 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcab006f5d0 msgr2=0x7fcab01c3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:54.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.128+0000 7fcab57e6700 1 --2- 172.21.15.3:0/2687888675 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcab006f5d0 0x7fcab01c3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.128+0000 7fcab57e6700 1 -- 172.21.15.3:0/2687888675 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcab0066880 msgr2=0x7fcab0129320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:54.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.128+0000 7fcab57e6700 1 --2- 172.21.15.3:0/2687888675 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcab0066880 0x7fcab0129320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.128+0000 7fcab57e6700 1 -- 172.21.15.3:0/2687888675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaac0094a0 con 0x7fcab0066410 2023-10-03T10:12:54.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.128+0000 7fcab57e6700 1 --2- 172.21.15.3:0/2687888675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab0066410 0x7fcab0128de0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcaa800cc80 tx=0x7fcaa800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:54.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.131+0000 7fcaa5ffb700 1 -- 172.21.15.3:0/2687888675 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa800ade0 con 0x7fcab0066410 2023-10-03T10:12:54.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.131+0000 7f1444ff9700 1 -- 172.21.15.3:0/1086133829 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1438017070 con 0x7f144806e570 2023-10-03T10:12:54.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.131+0000 7fcab7a4a700 1 -- 172.21.15.3:0/2687888675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab0129860 con 0x7fcab0066410 2023-10-03T10:12:54.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.131+0000 7fcaa5ffb700 1 -- 172.21.15.3:0/2687888675 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa800ce80 con 0x7fcab0066410 2023-10-03T10:12:54.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.131+0000 7fcab7a4a700 1 -- 172.21.15.3:0/2687888675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab009e730 con 0x7fcab0066410 2023-10-03T10:12:54.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.132+0000 7f5151ffb700 1 -- 172.21.15.3:0/3076193130 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5158044060 con 0x7f515c09d6f0 2023-10-03T10:12:54.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.132+0000 7fcaa5ffb700 1 -- 172.21.15.3:0/2687888675 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa80072d0 con 0x7fcab0066410 2023-10-03T10:12:54.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.132+0000 7fcab7a4a700 1 -- 172.21.15.3:0/2687888675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab0074ea0 con 0x7fcab0066410 2023-10-03T10:12:54.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.133+0000 7fcaa5ffb700 1 -- 172.21.15.3:0/2687888675 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fcaa8007430 con 0x7fcab0066410 2023-10-03T10:12:54.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.133+0000 7fcaa5ffb700 1 -- 172.21.15.3:0/2687888675 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7fcaa803fa80 con 0x7fcab0066410 2023-10-03T10:12:54.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.133+0000 7fcab4fe5700 1 -- 172.21.15.3:0/2687888675 >> v1:172.21.15.3:6816/127025 conn(0x7fca9402f510 legacy=0x7fca940319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.138+0000 7fcaa5ffb700 1 -- 172.21.15.3:0/2687888675 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcaa801c3b0 con 0x7fcab0066410 2023-10-03T10:12:54.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.183+0000 7f5234695700 1 Processor -- start 2023-10-03T10:12:54.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.183+0000 7f5234695700 1 -- start start 2023-10-03T10:12:54.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.184+0000 7f5234695700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f522c09e850 0x7f522c09ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:54.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.184+0000 7f5234695700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f522c09f160 0x7f522c133bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:54.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.184+0000 7f5234695700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f522c09f960 con 0x7f522c134130 2023-10-03T10:12:54.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.184+0000 7f5234695700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f522c09ae60 con 0x7f522c09f160 2023-10-03T10:12:54.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.184+0000 7f5234695700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f522c06cb10 con 0x7f522c09e850 2023-10-03T10:12:54.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.184+0000 7f5231c30700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f522c09f160 0x7f522c133bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:54.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.184+0000 7f5231c30700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f522c09f160 0x7f522c133bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38976/0 (socket says 172.21.15.3:38976) 2023-10-03T10:12:54.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.185+0000 7f5231c30700 1 -- 172.21.15.3:0/1437222591 learned_addr learned my addr 172.21.15.3:0/1437222591 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:54.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.185+0000 7f5232431700 1 --2- 172.21.15.3:0/1437222591 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f522c09e850 0x7f522c09ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:54.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.185+0000 7f5231c30700 1 -- 172.21.15.3:0/1437222591 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f522c09e850 msgr2=0x7f522c09ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:54.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.185+0000 7f5231c30700 1 --2- 172.21.15.3:0/1437222591 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f522c09e850 0x7f522c09ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.185+0000 7f5231c30700 1 -- 172.21.15.3:0/1437222591 >> v1:172.21.15.3:6789/0 conn(0x7f522c134130 legacy=0x7f522c13aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:54.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.185+0000 7f523142f700 1 -- 172.21.15.3:0/1437222591 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3235147899 0 0) 0x7f522c09f960 con 0x7f522c134130 2023-10-03T10:12:54.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.185+0000 7f5231c30700 1 -- 172.21.15.3:0/1437222591 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f522c06b990 con 0x7f522c09f160 2023-10-03T10:12:54.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.185+0000 7f5232431700 1 --2- 172.21.15.3:0/1437222591 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f522c09e850 0x7f522c09ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:54.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.185+0000 7f5231c30700 1 --2- 172.21.15.3:0/1437222591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f522c09f160 0x7f522c133bf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f521c0098f0 tx=0x7f521c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908c39d08a20c795 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:54.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.185+0000 7f523142f700 1 -- 172.21.15.3:0/1437222591 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c011830 con 0x7f522c09f160 2023-10-03T10:12:54.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.185+0000 7f523142f700 1 -- 172.21.15.3:0/1437222591 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521c011990 con 0x7f522c09f160 2023-10-03T10:12:54.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.185+0000 7f523142f700 1 -- 172.21.15.3:0/1437222591 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c011d20 con 0x7f522c09f160 2023-10-03T10:12:54.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.186+0000 7f5234695700 1 -- 172.21.15.3:0/1437222591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f522c09f160 msgr2=0x7f522c133bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:54.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.186+0000 7f5234695700 1 --2- 172.21.15.3:0/1437222591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f522c09f160 0x7f522c133bf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f521c0098f0 tx=0x7f521c009c00 comp rx=0 tx=0).stop 2023-10-03T10:12:54.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.186+0000 7f5234695700 1 -- 172.21.15.3:0/1437222591 shutdown_connections 2023-10-03T10:12:54.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.187+0000 7f5234695700 1 --2- 172.21.15.3:0/1437222591 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f522c09e850 0x7f522c09ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.187+0000 7f5234695700 1 --2- 172.21.15.3:0/1437222591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f522c09f160 0x7f522c133bf0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.187+0000 7f5234695700 1 -- 172.21.15.3:0/1437222591 >> 172.21.15.3:0/1437222591 conn(0x7f522c0a3f70 msgr2=0x7f522c0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:54.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.187+0000 7f5234695700 1 -- 172.21.15.3:0/1437222591 shutdown_connections 2023-10-03T10:12:54.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.187+0000 7f5234695700 1 -- 172.21.15.3:0/1437222591 wait complete. 2023-10-03T10:12:54.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.188+0000 7f5234695700 1 Processor -- start 2023-10-03T10:12:54.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.188+0000 7f5234695700 1 -- start start 2023-10-03T10:12:54.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.189+0000 7f5234695700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f522c09e850 0x7f522c122a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:54.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.189+0000 7f5234695700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f522c09f160 0x7f522c122f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:54.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.189+0000 7f5234695700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f522c134130 0x7f522c1234d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:54.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.189+0000 7f5234695700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522c1271f0 con 0x7f522c09e850 2023-10-03T10:12:54.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.189+0000 7f5234695700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f522c125fd0 con 0x7f522c134130 2023-10-03T10:12:54.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.189+0000 7f5234695700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f522c124c60 con 0x7f522c09f160 2023-10-03T10:12:54.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.189+0000 7f5232431700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f522c09e850 0x7f522c122a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:54.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.189+0000 7f5232431700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f522c09e850 0x7f522c122a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38990/0 (socket says 172.21.15.3:38990) 2023-10-03T10:12:54.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.189+0000 7f5232431700 1 -- 172.21.15.3:0/344690146 learned_addr learned my addr 172.21.15.3:0/344690146 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:54.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.189+0000 7f5232c32700 1 --2- 172.21.15.3:0/344690146 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f522c134130 0x7f522c1234d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:54.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.189+0000 7f5232431700 1 -- 172.21.15.3:0/344690146 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f522c09f160 msgr2=0x7f522c122f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:54.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.189+0000 7f5232431700 1 --2- 172.21.15.3:0/344690146 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f522c09f160 0x7f522c122f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.189+0000 7f5232431700 1 -- 172.21.15.3:0/344690146 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f522c134130 msgr2=0x7f522c1234d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:54.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.189+0000 7f5232431700 1 --2- 172.21.15.3:0/344690146 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f522c134130 0x7f522c1234d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.189+0000 7f5232431700 1 -- 172.21.15.3:0/344690146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52280094a0 con 0x7f522c09e850 2023-10-03T10:12:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.190+0000 7f5232431700 1 --2- 172.21.15.3:0/344690146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f522c09e850 0x7f522c122a50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f522800ec10 tx=0x7f5228009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.190+0000 7f5222ffd700 1 -- 172.21.15.3:0/344690146 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522800c940 con 0x7f522c09e850 2023-10-03T10:12:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.190+0000 7f5222ffd700 1 -- 172.21.15.3:0/344690146 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522800caa0 con 0x7f522c09e850 2023-10-03T10:12:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.190+0000 7f5222ffd700 1 -- 172.21.15.3:0/344690146 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522800ce10 con 0x7f522c09e850 2023-10-03T10:12:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.190+0000 7f5234695700 1 -- 172.21.15.3:0/344690146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f521c009550 con 0x7f522c09e850 2023-10-03T10:12:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.190+0000 7f5234695700 1 -- 172.21.15.3:0/344690146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f522c124e50 con 0x7f522c09e850 2023-10-03T10:12:54.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.191+0000 7f5222ffd700 1 -- 172.21.15.3:0/344690146 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f5228013470 con 0x7f522c09e850 2023-10-03T10:12:54.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.192+0000 7f5234695700 1 -- 172.21.15.3:0/344690146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f522c0747f0 con 0x7f522c09e850 2023-10-03T10:12:54.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.192+0000 7f5222ffd700 1 -- 172.21.15.3:0/344690146 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f522803ff60 con 0x7f522c09e850 2023-10-03T10:12:54.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.194+0000 7f5231c30700 1 -- 172.21.15.3:0/344690146 >> v1:172.21.15.3:6816/127025 conn(0x7f521002f510 legacy=0x7f52100319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:54.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.201+0000 7f5222ffd700 1 -- 172.21.15.3:0/344690146 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f52280232f0 con 0x7f522c09e850 2023-10-03T10:12:54.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.325+0000 7f14477fe700 1 -- 172.21.15.3:0/1086133829 >> v1:172.21.15.3:6816/127025 conn(0x7f143402f540 legacy=0x7f14340319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:54.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.325+0000 7f5160d1f700 1 -- 172.21.15.3:0/3076193130 >> v1:172.21.15.3:6816/127025 conn(0x7f514002f560 legacy=0x7f5140031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:54.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.334+0000 7fcab4fe5700 1 -- 172.21.15.3:0/2687888675 >> v1:172.21.15.3:6816/127025 conn(0x7fca9402f510 legacy=0x7fca940319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:54.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.342+0000 7f5163784700 1 -- 172.21.15.3:0/3076193130 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f515c06c360 con 0x7f515c09d6f0 2023-10-03T10:12:54.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.342+0000 7f5151ffb700 1 -- 172.21.15.3:0/3076193130 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5158035090 con 0x7f515c09d6f0 2023-10-03T10:12:54.343 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:12:54.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.345+0000 7f5163784700 1 -- 172.21.15.3:0/3076193130 >> v1:172.21.15.3:6816/127025 conn(0x7f514002f560 legacy=0x7f5140031a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:54.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.345+0000 7f5163784700 1 -- 172.21.15.3:0/3076193130 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f515c09d6f0 msgr2=0x7f515c119480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:54.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.345+0000 7f5163784700 1 --2- 172.21.15.3:0/3076193130 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f515c09d6f0 0x7f515c119480 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5158001c20 tx=0x7f5158014330 comp rx=0 tx=0).stop 2023-10-03T10:12:54.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.345+0000 7f5163784700 1 -- 172.21.15.3:0/3076193130 shutdown_connections 2023-10-03T10:12:54.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.345+0000 7f5163784700 1 --2- 172.21.15.3:0/3076193130 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f515c135e50 0x7f515c119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.345+0000 7f5163784700 1 --2- 172.21.15.3:0/3076193130 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f515c09d6f0 0x7f515c119480 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.345+0000 7f5163784700 1 --2- 172.21.15.3:0/3076193130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f515c09e1d0 0x7f515c1199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.345+0000 7f5163784700 1 -- 172.21.15.3:0/3076193130 >> 172.21.15.3:0/3076193130 conn(0x7f515c066f50 msgr2=0x7f515c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:54.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.345+0000 7f5163784700 1 -- 172.21.15.3:0/3076193130 shutdown_connections 2023-10-03T10:12:54.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.345+0000 7f5163784700 1 -- 172.21.15.3:0/3076193130 wait complete. 2023-10-03T10:12:54.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.353+0000 7fcab7a4a700 1 -- 172.21.15.3:0/2687888675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fcab0129860 con 0x7fcab0066410 2023-10-03T10:12:54.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.353+0000 7fcaa5ffb700 1 -- 172.21.15.3:0/2687888675 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcaa8011360 con 0x7fcab0066410 2023-10-03T10:12:54.354 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:12:54.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.356+0000 7fcab7a4a700 1 -- 172.21.15.3:0/2687888675 >> v1:172.21.15.3:6816/127025 conn(0x7fca9402f510 legacy=0x7fca940319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:54.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.356+0000 7fcab7a4a700 1 -- 172.21.15.3:0/2687888675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab0066410 msgr2=0x7fcab0128de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.356+0000 7fcab7a4a700 1 --2- 172.21.15.3:0/2687888675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab0066410 0x7fcab0128de0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcaa800cc80 tx=0x7fcaa800a340 comp rx=0 tx=0).stop 2023-10-03T10:12:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.356+0000 7fcab7a4a700 1 -- 172.21.15.3:0/2687888675 shutdown_connections 2023-10-03T10:12:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.356+0000 7fcab7a4a700 1 --2- 172.21.15.3:0/2687888675 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcab006f5d0 0x7fcab01c3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.356+0000 7fcab7a4a700 1 --2- 172.21.15.3:0/2687888675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab0066410 0x7fcab0128de0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.356+0000 7fcab7a4a700 1 --2- 172.21.15.3:0/2687888675 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcab0066880 0x7fcab0129320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.356+0000 7fcab7a4a700 1 -- 172.21.15.3:0/2687888675 >> 172.21.15.3:0/2687888675 conn(0x7fcab00a2aa0 msgr2=0x7fcab006e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.356+0000 7fcab7a4a700 1 -- 172.21.15.3:0/2687888675 shutdown_connections 2023-10-03T10:12:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.356+0000 7fcab7a4a700 1 -- 172.21.15.3:0/2687888675 wait complete. 2023-10-03T10:12:54.359 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:12:54.375 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:12:54.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.395+0000 7f5231c30700 1 -- 172.21.15.3:0/344690146 >> v1:172.21.15.3:6816/127025 conn(0x7f521002f510 legacy=0x7f52100319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:54.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.398+0000 7f144e8c5700 1 -- 172.21.15.3:0/1086133829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1448123c50 con 0x7f144806e570 2023-10-03T10:12:54.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.399+0000 7f1444ff9700 1 -- 172.21.15.3:0/1086133829 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1438018cd0 con 0x7f144806e570 2023-10-03T10:12:54.399 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:12:54.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.401+0000 7f144e8c5700 1 -- 172.21.15.3:0/1086133829 >> v1:172.21.15.3:6816/127025 conn(0x7f143402f540 legacy=0x7f14340319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:54.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.401+0000 7f144e8c5700 1 -- 172.21.15.3:0/1086133829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f144806e570 msgr2=0x7f14481214e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:54.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.401+0000 7f144e8c5700 1 --2- 172.21.15.3:0/1086133829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f144806e570 0x7f14481214e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f143800c3e0 tx=0x7f143800c6f0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.402+0000 7f144e8c5700 1 -- 172.21.15.3:0/1086133829 shutdown_connections 2023-10-03T10:12:54.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.402+0000 7f144e8c5700 1 --2- 172.21.15.3:0/1086133829 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1448121f60 0x7f1448136df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.402+0000 7f144e8c5700 1 --2- 172.21.15.3:0/1086133829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f144806e570 0x7f14481214e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.402+0000 7f144e8c5700 1 --2- 172.21.15.3:0/1086133829 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f144806f050 0x7f1448121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.402+0000 7f144e8c5700 1 -- 172.21.15.3:0/1086133829 >> 172.21.15.3:0/1086133829 conn(0x7f14480a2aa0 msgr2=0x7f144809d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:54.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.402+0000 7f144e8c5700 1 -- 172.21.15.3:0/1086133829 shutdown_connections 2023-10-03T10:12:54.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.402+0000 7f144e8c5700 1 -- 172.21.15.3:0/1086133829 wait complete. 2023-10-03T10:12:54.416 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:12:54.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.452+0000 7f5234695700 1 -- 172.21.15.3:0/344690146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f522c06bbb0 con 0x7f522c09e850 2023-10-03T10:12:54.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.453+0000 7f5222ffd700 1 -- 172.21.15.3:0/344690146 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5228017030 con 0x7f522c09e850 2023-10-03T10:12:54.453 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:12:54.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.455+0000 7f5234695700 1 -- 172.21.15.3:0/344690146 >> v1:172.21.15.3:6816/127025 conn(0x7f521002f510 legacy=0x7f52100319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:54.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.455+0000 7f5234695700 1 -- 172.21.15.3:0/344690146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f522c09e850 msgr2=0x7f522c122a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:54.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.455+0000 7f5234695700 1 --2- 172.21.15.3:0/344690146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f522c09e850 0x7f522c122a50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f522800ec10 tx=0x7f5228009ec0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.455+0000 7f5234695700 1 -- 172.21.15.3:0/344690146 shutdown_connections 2023-10-03T10:12:54.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.455+0000 7f5234695700 1 --2- 172.21.15.3:0/344690146 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f522c134130 0x7f522c1234d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.455+0000 7f5234695700 1 --2- 172.21.15.3:0/344690146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f522c09e850 0x7f522c122a50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.455+0000 7f5234695700 1 --2- 172.21.15.3:0/344690146 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f522c09f160 0x7f522c122f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:54.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.455+0000 7f5234695700 1 -- 172.21.15.3:0/344690146 >> 172.21.15.3:0/344690146 conn(0x7f522c0a3f70 msgr2=0x7f522c070dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:54.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.455+0000 7f5234695700 1 -- 172.21.15.3:0/344690146 shutdown_connections 2023-10-03T10:12:54.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:54.455+0000 7f5234695700 1 -- 172.21.15.3:0/344690146 wait complete. 2023-10-03T10:12:54.469 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:12:55.360 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:12:55.376 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:12:55.416 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:12:55.470 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:12:55.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.516+0000 7f927a247700 1 Processor -- start 2023-10-03T10:12:55.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.516+0000 7f927a247700 1 -- start start 2023-10-03T10:12:55.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.517+0000 7f927a247700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f92741338c0 0x7f9274133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:55.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.517+0000 7f927a247700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f927409e560 0x7f927413b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:55.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.517+0000 7f927a247700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f927406f720 con 0x7f92741341d0 2023-10-03T10:12:55.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.517+0000 7f927a247700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9274070c80 con 0x7f927409e560 2023-10-03T10:12:55.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.517+0000 7f927a247700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f927406fce0 con 0x7f92741338c0 2023-10-03T10:12:55.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.517+0000 7f9272ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f92741341d0 0x7f927409de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45352/0 (socket says 172.21.15.3:45352) 2023-10-03T10:12:55.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.517+0000 7f9272ffd700 1 -- 172.21.15.3:0/216061627 learned_addr learned my addr 172.21.15.3:0/216061627 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:55.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.517+0000 7f92737fe700 1 --2- 172.21.15.3:0/216061627 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f92741338c0 0x7f9274133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:55.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.517+0000 7f9273fff700 1 --2- 172.21.15.3:0/216061627 >> v2:172.21.15.3:3300/0 conn(0x7f927409e560 0x7f927413b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:55.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.517+0000 7f92737fe700 1 -- 172.21.15.3:0/216061627 >> v2:172.21.15.3:3300/0 conn(0x7f927409e560 msgr2=0x7f927413b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:55.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.517+0000 7f92737fe700 1 --2- 172.21.15.3:0/216061627 >> v2:172.21.15.3:3300/0 conn(0x7f927409e560 0x7f927413b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.517+0000 7f92737fe700 1 -- 172.21.15.3:0/216061627 >> v1:172.21.15.3:6789/0 conn(0x7f92741341d0 legacy=0x7f927409de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:55.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.517+0000 7f92737fe700 1 -- 172.21.15.3:0/216061627 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92741336a0 con 0x7f92741338c0 2023-10-03T10:12:55.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.518+0000 7f92737fe700 1 --2- 172.21.15.3:0/216061627 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f92741338c0 0x7f9274133c90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9264009840 tx=0x7f9264009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16d54e83c0d6973b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:55.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.518+0000 7f92727fc700 1 -- 172.21.15.3:0/216061627 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264011830 con 0x7f92741338c0 2023-10-03T10:12:55.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.518+0000 7f92727fc700 1 -- 172.21.15.3:0/216061627 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9264011990 con 0x7f92741338c0 2023-10-03T10:12:55.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.518+0000 7f92727fc700 1 -- 172.21.15.3:0/216061627 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264011d20 con 0x7f92741338c0 2023-10-03T10:12:55.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.519+0000 7f927a247700 1 -- 172.21.15.3:0/216061627 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f92741338c0 msgr2=0x7f9274133c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.519+0000 7f927a247700 1 --2- 172.21.15.3:0/216061627 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f92741338c0 0x7f9274133c90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9264009840 tx=0x7f9264009b50 comp rx=0 tx=0).stop 2023-10-03T10:12:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.519+0000 7f927a247700 1 -- 172.21.15.3:0/216061627 shutdown_connections 2023-10-03T10:12:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.519+0000 7f927a247700 1 --2- 172.21.15.3:0/216061627 >> v2:172.21.15.3:3300/0 conn(0x7f927409e560 0x7f927413b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.519+0000 7f927a247700 1 --2- 172.21.15.3:0/216061627 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f92741338c0 0x7f9274133c90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.519+0000 7f927a247700 1 -- 172.21.15.3:0/216061627 >> 172.21.15.3:0/216061627 conn(0x7f92740a2aa0 msgr2=0x7f92740a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.519+0000 7f927a247700 1 -- 172.21.15.3:0/216061627 shutdown_connections 2023-10-03T10:12:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.519+0000 7f927a247700 1 -- 172.21.15.3:0/216061627 wait complete. 2023-10-03T10:12:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.521+0000 7f927a247700 1 Processor -- start 2023-10-03T10:12:55.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.521+0000 7f927a247700 1 -- start start 2023-10-03T10:12:55.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.522+0000 7f927a247700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f927409e560 0x7f927411fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:55.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.522+0000 7f927a247700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f92741338c0 0x7f9274120090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:55.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.522+0000 7f927a247700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f92741341d0 0x7f92741205d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:55.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.522+0000 7f927a247700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927406f960 con 0x7f927409e560 2023-10-03T10:12:55.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.522+0000 7f927a247700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9274123730 con 0x7f92741338c0 2023-10-03T10:12:55.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.522+0000 7f927a247700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92741217c0 con 0x7f92741341d0 2023-10-03T10:12:55.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.522+0000 7f9272ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f92741338c0 0x7f9274120090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:55.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.522+0000 7f9273fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f92741341d0 0x7f92741205d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:55.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.522+0000 7f9272ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f92741338c0 0x7f9274120090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42866/0 (socket says 172.21.15.3:42866) 2023-10-03T10:12:55.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.522+0000 7f9272ffd700 1 -- 172.21.15.3:0/3865247617 learned_addr learned my addr 172.21.15.3:0/3865247617 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:55.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.522+0000 7f92737fe700 1 --2- 172.21.15.3:0/3865247617 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f927409e560 0x7f927411fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:55.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.522+0000 7f9273fff700 1 -- 172.21.15.3:0/3865247617 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f92741338c0 msgr2=0x7f9274120090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:55.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.522+0000 7f9273fff700 1 --2- 172.21.15.3:0/3865247617 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f92741338c0 0x7f9274120090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.522+0000 7f9273fff700 1 -- 172.21.15.3:0/3865247617 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f927409e560 msgr2=0x7f927411fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:55.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.522+0000 7f9273fff700 1 --2- 172.21.15.3:0/3865247617 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f927409e560 0x7f927411fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.522+0000 7f9273fff700 1 -- 172.21.15.3:0/3865247617 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92640094a0 con 0x7f92741341d0 2023-10-03T10:12:55.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.523+0000 7f9273fff700 1 --2- 172.21.15.3:0/3865247617 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f92741341d0 0x7f92741205d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f926c007d90 tx=0x7f926c00e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:55.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.523+0000 7f925bfff700 1 -- 172.21.15.3:0/3865247617 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c018070 con 0x7f92741341d0 2023-10-03T10:12:55.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.523+0000 7f925bfff700 1 -- 172.21.15.3:0/3865247617 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f926c00eb30 con 0x7f92741341d0 2023-10-03T10:12:55.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.523+0000 7f927a247700 1 -- 172.21.15.3:0/3865247617 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9274123090 con 0x7f92741341d0 2023-10-03T10:12:55.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.523+0000 7f927a247700 1 -- 172.21.15.3:0/3865247617 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9274122da0 con 0x7f92741341d0 2023-10-03T10:12:55.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.523+0000 7f925bfff700 1 -- 172.21.15.3:0/3865247617 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c003a20 con 0x7f92741341d0 2023-10-03T10:12:55.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.524+0000 7f925bfff700 1 -- 172.21.15.3:0/3865247617 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f926c014420 con 0x7f92741341d0 2023-10-03T10:12:55.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.524+0000 7f925bfff700 1 -- 172.21.15.3:0/3865247617 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f926c03fb10 con 0x7f92741341d0 2023-10-03T10:12:55.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.524+0000 7f92737fe700 1 -- 172.21.15.3:0/3865247617 >> v1:172.21.15.3:6816/127025 conn(0x7f926002f510 legacy=0x7f92600319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:55.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.524+0000 7f927a247700 1 -- 172.21.15.3:0/3865247617 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92540052e0 con 0x7f92741341d0 2023-10-03T10:12:55.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.530+0000 7f925bfff700 1 -- 172.21.15.3:0/3865247617 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f926c0128d0 con 0x7f92741341d0 2023-10-03T10:12:55.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.568+0000 7f65c2997700 1 Processor -- start 2023-10-03T10:12:55.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.568+0000 7f65c2997700 1 -- start start 2023-10-03T10:12:55.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.568+0000 7f65c2997700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65bc130540 0x7f65bc130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:55.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.568+0000 7f65c2997700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f65bc130e50 0x7f65bc06f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:55.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.568+0000 7f65c2997700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65bc06fb20 0x7f65bc06ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:55.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.568+0000 7f65c2997700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65bc0704b0 con 0x7f65bc130e50 2023-10-03T10:12:55.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.568+0000 7f65c2997700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc09a390 con 0x7f65bc06fb20 2023-10-03T10:12:55.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.568+0000 7f65c2997700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc0716a0 con 0x7f65bc130540 2023-10-03T10:12:55.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.569+0000 7f65bbfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65bc130540 0x7f65bc130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:55.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.569+0000 7f65bbfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65bc130540 0x7f65bc130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54630/0 (socket says 172.21.15.3:54630) 2023-10-03T10:12:55.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.569+0000 7f65bbfff700 1 -- 172.21.15.3:0/1445696379 learned_addr learned my addr 172.21.15.3:0/1445696379 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:55.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.569+0000 7f65bb7fe700 1 --2- 172.21.15.3:0/1445696379 >> v2:172.21.15.3:3300/0 conn(0x7f65bc130e50 0x7f65bc06f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:55.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.569+0000 7f65c0f34700 1 --2- 172.21.15.3:0/1445696379 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65bc06fb20 0x7f65bc06ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:55.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.569+0000 7f65bbfff700 1 -- 172.21.15.3:0/1445696379 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65bc06fb20 msgr2=0x7f65bc06ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:55.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.569+0000 7f65bbfff700 1 --2- 172.21.15.3:0/1445696379 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65bc06fb20 0x7f65bc06ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.569+0000 7f65bbfff700 1 -- 172.21.15.3:0/1445696379 >> v2:172.21.15.3:3300/0 conn(0x7f65bc130e50 msgr2=0x7f65bc06f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:55.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.569+0000 7f65bbfff700 1 --2- 172.21.15.3:0/1445696379 >> v2:172.21.15.3:3300/0 conn(0x7f65bc130e50 0x7f65bc06f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.569+0000 7f65bbfff700 1 -- 172.21.15.3:0/1445696379 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65bc071290 con 0x7f65bc130540 2023-10-03T10:12:55.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.569+0000 7f65bbfff700 1 --2- 172.21.15.3:0/1445696379 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65bc130540 0x7f65bc130910 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009920 tx=0x7f65ac009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a9db926a758404b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:55.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.570+0000 7f65baffd700 1 -- 172.21.15.3:0/1445696379 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac014e90 con 0x7f65bc130540 2023-10-03T10:12:55.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.570+0000 7f65baffd700 1 -- 172.21.15.3:0/1445696379 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ac01c440 con 0x7f65bc130540 2023-10-03T10:12:55.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.570+0000 7f65baffd700 1 -- 172.21.15.3:0/1445696379 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac01c7d0 con 0x7f65bc130540 2023-10-03T10:12:55.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.571+0000 7f65c2997700 1 -- 172.21.15.3:0/1445696379 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65bc130540 msgr2=0x7f65bc130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:55.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.571+0000 7f65c2997700 1 --2- 172.21.15.3:0/1445696379 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65bc130540 0x7f65bc130910 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009920 tx=0x7f65ac009c30 comp rx=0 tx=0).stop 2023-10-03T10:12:55.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.571+0000 7f65c2997700 1 -- 172.21.15.3:0/1445696379 shutdown_connections 2023-10-03T10:12:55.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.571+0000 7f65c2997700 1 --2- 172.21.15.3:0/1445696379 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65bc06fb20 0x7f65bc06ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.571+0000 7f65c2997700 1 --2- 172.21.15.3:0/1445696379 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65bc130540 0x7f65bc130910 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.571+0000 7f65c2997700 1 --2- 172.21.15.3:0/1445696379 >> v2:172.21.15.3:3300/0 conn(0x7f65bc130e50 0x7f65bc06f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.571+0000 7f65c2997700 1 -- 172.21.15.3:0/1445696379 >> 172.21.15.3:0/1445696379 conn(0x7f65bc0a2aa0 msgr2=0x7f65bc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:55.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.571+0000 7f65c2997700 1 -- 172.21.15.3:0/1445696379 shutdown_connections 2023-10-03T10:12:55.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.571+0000 7f65c2997700 1 -- 172.21.15.3:0/1445696379 wait complete. 2023-10-03T10:12:55.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.572+0000 7f65c2997700 1 Processor -- start 2023-10-03T10:12:55.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.572+0000 7f65c2997700 1 -- start start 2023-10-03T10:12:55.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.573+0000 7f65c2997700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65bc06fb20 0x7f65bc118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:55.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.573+0000 7f65c2997700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65bc130540 0x7f65bc119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:55.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.573+0000 7f65c2997700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65bc130e50 0x7f65bc119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:55.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.573+0000 7f65c2997700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc11a1a0 con 0x7f65bc06fb20 2023-10-03T10:12:55.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.573+0000 7f65c2997700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc11a7a0 con 0x7f65bc130540 2023-10-03T10:12:55.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.573+0000 7f65c2997700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc11b3d0 con 0x7f65bc130e50 2023-10-03T10:12:55.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.573+0000 7f65bbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65bc06fb20 0x7f65bc118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:55.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.573+0000 7f65bbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65bc06fb20 0x7f65bc118cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39010/0 (socket says 172.21.15.3:39010) 2023-10-03T10:12:55.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.573+0000 7f65bbfff700 1 -- 172.21.15.3:0/671668860 learned_addr learned my addr 172.21.15.3:0/671668860 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:55.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.573+0000 7f65bb7fe700 1 --2- 172.21.15.3:0/671668860 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65bc130540 0x7f65bc119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:55.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.573+0000 7f65c0f34700 1 --2- 172.21.15.3:0/671668860 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65bc130e50 0x7f65bc119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:55.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.573+0000 7f65bbfff700 1 -- 172.21.15.3:0/671668860 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65bc130e50 msgr2=0x7f65bc119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:55.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.573+0000 7f65bbfff700 1 --2- 172.21.15.3:0/671668860 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65bc130e50 0x7f65bc119750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.574+0000 7f65bbfff700 1 -- 172.21.15.3:0/671668860 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65bc130540 msgr2=0x7f65bc119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:55.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.574+0000 7f65bbfff700 1 --2- 172.21.15.3:0/671668860 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65bc130540 0x7f65bc119210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.574+0000 7f65bbfff700 1 -- 172.21.15.3:0/671668860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ac009550 con 0x7f65bc06fb20 2023-10-03T10:12:55.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.574+0000 7f65c0f34700 1 --2- 172.21.15.3:0/671668860 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65bc130e50 0x7f65bc119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:55.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.574+0000 7f65bb7fe700 1 --2- 172.21.15.3:0/671668860 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65bc130540 0x7f65bc119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:55.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.574+0000 7f65bbfff700 1 --2- 172.21.15.3:0/671668860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65bc06fb20 0x7f65bc118cd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009130 tx=0x7f65ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:55.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.574+0000 7f65b8ff9700 1 -- 172.21.15.3:0/671668860 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac004450 con 0x7f65bc06fb20 2023-10-03T10:12:55.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.574+0000 7f65b8ff9700 1 -- 172.21.15.3:0/671668860 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ac01ce80 con 0x7f65bc06fb20 2023-10-03T10:12:55.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.574+0000 7f65b8ff9700 1 -- 172.21.15.3:0/671668860 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac0142b0 con 0x7f65bc06fb20 2023-10-03T10:12:55.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.574+0000 7f65c2997700 1 -- 172.21.15.3:0/671668860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65bc0a0c40 con 0x7f65bc06fb20 2023-10-03T10:12:55.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.574+0000 7f65c2997700 1 -- 172.21.15.3:0/671668860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65bc11b820 con 0x7f65bc06fb20 2023-10-03T10:12:55.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.575+0000 7f65a27fc700 1 -- 172.21.15.3:0/671668860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65940052e0 con 0x7f65bc06fb20 2023-10-03T10:12:55.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.576+0000 7f65b8ff9700 1 -- 172.21.15.3:0/671668860 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f65ac014410 con 0x7f65bc06fb20 2023-10-03T10:12:55.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.576+0000 7f65b8ff9700 1 -- 172.21.15.3:0/671668860 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f65ac0412b0 con 0x7f65bc06fb20 2023-10-03T10:12:55.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.576+0000 7f65bb7fe700 1 -- 172.21.15.3:0/671668860 >> v1:172.21.15.3:6816/127025 conn(0x7f65a802f510 legacy=0x7f65a80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:55.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.585+0000 7f65b8ff9700 1 -- 172.21.15.3:0/671668860 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f65ac01a340 con 0x7f65bc06fb20 2023-10-03T10:12:55.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.611+0000 7fe0d1543700 1 Processor -- start 2023-10-03T10:12:55.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.611+0000 7fe0d1543700 1 -- start start 2023-10-03T10:12:55.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.611+0000 7fe0d1543700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe0cc09eca0 0x7fe0cc13cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:55.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.611+0000 7fe0d1543700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe0cc06f050 0x7fe0cc13f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:55.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.612+0000 7fe0d1543700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0cc070ee0 con 0x7fe0cc06e570 2023-10-03T10:12:55.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.612+0000 7fe0d1543700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0cc071980 con 0x7fe0cc09eca0 2023-10-03T10:12:55.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.612+0000 7fe0d1543700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe0cc071480 con 0x7fe0cc06f050 2023-10-03T10:12:55.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.612+0000 7fe0ca7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe0cc09eca0 0x7fe0cc13cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:55.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.612+0000 7fe0cb7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe0cc06f050 0x7fe0cc13f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:55.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.612+0000 7fe0caffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe0cc06e570 0x7fe0cc06e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45364/0 (socket says 172.21.15.3:45364) 2023-10-03T10:12:55.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.612+0000 7fe0caffd700 1 -- 172.21.15.3:0/4046213533 learned_addr learned my addr 172.21.15.3:0/4046213533 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:55.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.612+0000 7fe0ca7fc700 1 -- 172.21.15.3:0/4046213533 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe0cc06f050 msgr2=0x7fe0cc13f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:55.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.612+0000 7fe0ca7fc700 1 --2- 172.21.15.3:0/4046213533 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe0cc06f050 0x7fe0cc13f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.612+0000 7fe0ca7fc700 1 -- 172.21.15.3:0/4046213533 >> v1:172.21.15.3:6789/0 conn(0x7fe0cc06e570 legacy=0x7fe0cc06e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:55.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.612+0000 7fe0ca7fc700 1 -- 172.21.15.3:0/4046213533 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0cc071b90 con 0x7fe0cc09eca0 2023-10-03T10:12:55.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.612+0000 7fe0cb7fe700 1 --2- 172.21.15.3:0/4046213533 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe0cc06f050 0x7fe0cc13f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:55.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.612+0000 7fe0ca7fc700 1 --2- 172.21.15.3:0/4046213533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0cc09eca0 0x7fe0cc13cea0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009840 tx=0x7fe0c0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1960c147d91b4f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:55.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.613+0000 7fe0c9ffb700 1 -- 172.21.15.3:0/4046213533 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0011830 con 0x7fe0cc09eca0 2023-10-03T10:12:55.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.613+0000 7fe0c9ffb700 1 -- 172.21.15.3:0/4046213533 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c0011990 con 0x7fe0cc09eca0 2023-10-03T10:12:55.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.613+0000 7fe0c9ffb700 1 -- 172.21.15.3:0/4046213533 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0011d20 con 0x7fe0cc09eca0 2023-10-03T10:12:55.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.614+0000 7fe0d1543700 1 -- 172.21.15.3:0/4046213533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0cc09eca0 msgr2=0x7fe0cc13cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:55.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.614+0000 7fe0d1543700 1 --2- 172.21.15.3:0/4046213533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0cc09eca0 0x7fe0cc13cea0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009840 tx=0x7fe0c0009b50 comp rx=0 tx=0).stop 2023-10-03T10:12:55.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.614+0000 7fe0d1543700 1 -- 172.21.15.3:0/4046213533 shutdown_connections 2023-10-03T10:12:55.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.614+0000 7fe0d1543700 1 --2- 172.21.15.3:0/4046213533 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe0cc06f050 0x7fe0cc13f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.614+0000 7fe0d1543700 1 --2- 172.21.15.3:0/4046213533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0cc09eca0 0x7fe0cc13cea0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.614+0000 7fe0d1543700 1 -- 172.21.15.3:0/4046213533 >> 172.21.15.3:0/4046213533 conn(0x7fe0cc0a2aa0 msgr2=0x7fe0cc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:55.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.614+0000 7fe0d1543700 1 -- 172.21.15.3:0/4046213533 shutdown_connections 2023-10-03T10:12:55.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.614+0000 7fe0d1543700 1 -- 172.21.15.3:0/4046213533 wait complete. 2023-10-03T10:12:55.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.616+0000 7fe0d1543700 1 Processor -- start 2023-10-03T10:12:55.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.616+0000 7fe0d1543700 1 -- start start 2023-10-03T10:12:55.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.616+0000 7fe0d1543700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0cc06e570 0x7fe0cc1333e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:55.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.616+0000 7fe0d1543700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe0cc06f050 0x7fe0cc133920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:55.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.616+0000 7fe0d1543700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe0cc13e720 0x7fe0cc133e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:55.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.616+0000 7fe0d1543700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0cc135190 con 0x7fe0cc06e570 2023-10-03T10:12:55.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.616+0000 7fe0d1543700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0cc134fa0 con 0x7fe0cc13e720 2023-10-03T10:12:55.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.616+0000 7fe0d1543700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe0cc070610 con 0x7fe0cc06f050 2023-10-03T10:12:55.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.617+0000 7fe0ca7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe0cc06f050 0x7fe0cc133920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:55.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.617+0000 7fe0ca7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe0cc06f050 0x7fe0cc133920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54652/0 (socket says 172.21.15.3:54652) 2023-10-03T10:12:55.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.617+0000 7fe0ca7fc700 1 -- 172.21.15.3:0/440459299 learned_addr learned my addr 172.21.15.3:0/440459299 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:55.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.617+0000 7fe0caffd700 1 --2- 172.21.15.3:0/440459299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0cc06e570 0x7fe0cc1333e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:55.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.617+0000 7fe0ca7fc700 1 -- 172.21.15.3:0/440459299 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe0cc13e720 msgr2=0x7fe0cc133e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:55.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.617+0000 7fe0ca7fc700 1 --2- 172.21.15.3:0/440459299 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe0cc13e720 0x7fe0cc133e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.617+0000 7fe0ca7fc700 1 -- 172.21.15.3:0/440459299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0cc06e570 msgr2=0x7fe0cc1333e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:55.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.617+0000 7fe0ca7fc700 1 --2- 172.21.15.3:0/440459299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0cc06e570 0x7fe0cc1333e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.617+0000 7fe0ca7fc700 1 -- 172.21.15.3:0/440459299 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c4009a70 con 0x7fe0cc06f050 2023-10-03T10:12:55.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.617+0000 7fe0ca7fc700 1 --2- 172.21.15.3:0/440459299 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe0cc06f050 0x7fe0cc133920 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0000c00 tx=0x7fe0c0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:55.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.617+0000 7fe0b37fe700 1 -- 172.21.15.3:0/440459299 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c00036a0 con 0x7fe0cc06f050 2023-10-03T10:12:55.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.617+0000 7fe0d1543700 1 -- 172.21.15.3:0/440459299 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c00094a0 con 0x7fe0cc06f050 2023-10-03T10:12:55.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.617+0000 7fe0b37fe700 1 -- 172.21.15.3:0/440459299 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c001f070 con 0x7fe0cc06f050 2023-10-03T10:12:55.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.618+0000 7fe0b37fe700 1 -- 172.21.15.3:0/440459299 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0003a60 con 0x7fe0cc06f050 2023-10-03T10:12:55.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.618+0000 7fe0d1543700 1 -- 172.21.15.3:0/440459299 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0cc136960 con 0x7fe0cc06f050 2023-10-03T10:12:55.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.619+0000 7fe0d1543700 1 -- 172.21.15.3:0/440459299 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0cc135e00 con 0x7fe0cc06f050 2023-10-03T10:12:55.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.619+0000 7fe0b37fe700 1 -- 172.21.15.3:0/440459299 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fe0c0003bc0 con 0x7fe0cc06f050 2023-10-03T10:12:55.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.619+0000 7fe0caffd700 1 -- 172.21.15.3:0/440459299 >> v1:172.21.15.3:6816/127025 conn(0x7fe0b802f510 legacy=0x7fe0b80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:55.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.619+0000 7fe0b37fe700 1 -- 172.21.15.3:0/440459299 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7fe0c0021070 con 0x7fe0cc06f050 2023-10-03T10:12:55.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.629+0000 7fe0b37fe700 1 -- 172.21.15.3:0/440459299 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe0c0046060 con 0x7fe0cc06f050 2023-10-03T10:12:55.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.654+0000 7f3ea7a30700 1 Processor -- start 2023-10-03T10:12:55.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.654+0000 7f3ea7a30700 1 -- start start 2023-10-03T10:12:55.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.655+0000 7f3ea7a30700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ea009e930 0x7f3ea009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:55.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.655+0000 7f3ea7a30700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ea009f240 0x7f3ea01317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:55.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.655+0000 7f3ea7a30700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea0070940 con 0x7f3ea009f650 2023-10-03T10:12:55.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.655+0000 7f3ea7a30700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea006fd60 con 0x7f3ea009f240 2023-10-03T10:12:55.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.655+0000 7f3ea7a30700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea006fa80 con 0x7f3ea009e930 2023-10-03T10:12:55.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.655+0000 7f3ea57cc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ea009e930 0x7f3ea009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:55.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.655+0000 7f3ea4fcb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ea009f240 0x7f3ea01317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:55.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.655+0000 7f3ea57cc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ea009e930 0x7f3ea009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54660/0 (socket says 172.21.15.3:54660) 2023-10-03T10:12:55.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.655+0000 7f3ea57cc700 1 -- 172.21.15.3:0/628789910 learned_addr learned my addr 172.21.15.3:0/628789910 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:55.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.655+0000 7f3ea4fcb700 1 -- 172.21.15.3:0/628789910 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ea009e930 msgr2=0x7f3ea009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:55.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.655+0000 7f3ea4fcb700 1 --2- 172.21.15.3:0/628789910 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ea009e930 0x7f3ea009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.655+0000 7f3ea4fcb700 1 -- 172.21.15.3:0/628789910 >> v1:172.21.15.3:6789/0 conn(0x7f3ea009f650 legacy=0x7f3ea0131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:55.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.655+0000 7f3ea4fcb700 1 -- 172.21.15.3:0/628789910 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea009fb60 con 0x7f3ea009f240 2023-10-03T10:12:55.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.656+0000 7f3ea4fcb700 1 --2- 172.21.15.3:0/628789910 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ea009f240 0x7f3ea01317f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3e90009870 tx=0x7f3e90009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd6497a5b8b19e0a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:55.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.656+0000 7f3e97fff700 1 -- 172.21.15.3:0/628789910 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90011830 con 0x7f3ea009f240 2023-10-03T10:12:55.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.656+0000 7f3e97fff700 1 -- 172.21.15.3:0/628789910 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e90011990 con 0x7f3ea009f240 2023-10-03T10:12:55.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.656+0000 7f3e97fff700 1 -- 172.21.15.3:0/628789910 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90011d20 con 0x7f3ea009f240 2023-10-03T10:12:55.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.657+0000 7f3ea7a30700 1 -- 172.21.15.3:0/628789910 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ea009f240 msgr2=0x7f3ea01317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:55.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.657+0000 7f3ea7a30700 1 --2- 172.21.15.3:0/628789910 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ea009f240 0x7f3ea01317f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3e90009870 tx=0x7f3e90009b80 comp rx=0 tx=0).stop 2023-10-03T10:12:55.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.657+0000 7f3ea7a30700 1 -- 172.21.15.3:0/628789910 shutdown_connections 2023-10-03T10:12:55.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.657+0000 7f3ea7a30700 1 --2- 172.21.15.3:0/628789910 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ea009e930 0x7f3ea009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.657+0000 7f3ea7a30700 1 --2- 172.21.15.3:0/628789910 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ea009f240 0x7f3ea01317f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.657+0000 7f3ea7a30700 1 -- 172.21.15.3:0/628789910 >> 172.21.15.3:0/628789910 conn(0x7f3ea00a4210 msgr2=0x7f3ea00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:55.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.657+0000 7f3ea7a30700 1 -- 172.21.15.3:0/628789910 shutdown_connections 2023-10-03T10:12:55.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.657+0000 7f3ea7a30700 1 -- 172.21.15.3:0/628789910 wait complete. 2023-10-03T10:12:55.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.658+0000 7f3ea7a30700 1 Processor -- start 2023-10-03T10:12:55.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.658+0000 7f3ea7a30700 1 -- start start 2023-10-03T10:12:55.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.659+0000 7f3ea7a30700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ea009e930 0x7f3ea01c0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:55.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.659+0000 7f3ea7a30700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ea009f240 0x7f3ea01c1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:55.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.659+0000 7f3ea7a30700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ea009f650 0x7f3ea01c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:55.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.659+0000 7f3ea7a30700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea01a3080 con 0x7f3ea009f650 2023-10-03T10:12:55.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.659+0000 7f3ea7a30700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea01c18b0 con 0x7f3ea009e930 2023-10-03T10:12:55.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.659+0000 7f3ea7a30700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea01a5660 con 0x7f3ea009f240 2023-10-03T10:12:55.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.659+0000 7f3ea4fcb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ea009f240 0x7f3ea01c1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:55.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.659+0000 7f3ea57cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ea009e930 0x7f3ea01c0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:55.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.659+0000 7f3ea4fcb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ea009f240 0x7f3ea01c1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54668/0 (socket says 172.21.15.3:54668) 2023-10-03T10:12:55.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.659+0000 7f3ea57cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ea009e930 0x7f3ea01c0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42904/0 (socket says 172.21.15.3:42904) 2023-10-03T10:12:55.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.659+0000 7f3ea4fcb700 1 -- 172.21.15.3:0/4209199893 learned_addr learned my addr 172.21.15.3:0/4209199893 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:55.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.659+0000 7f3ea5fcd700 1 --2- 172.21.15.3:0/4209199893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ea009f650 0x7f3ea01c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:55.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.660+0000 7f3ea5fcd700 1 -- 172.21.15.3:0/4209199893 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ea009f240 msgr2=0x7f3ea01c1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:55.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.660+0000 7f3ea5fcd700 1 --2- 172.21.15.3:0/4209199893 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ea009f240 0x7f3ea01c1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.660+0000 7f3ea5fcd700 1 -- 172.21.15.3:0/4209199893 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ea009e930 msgr2=0x7f3ea01c0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:55.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.660+0000 7f3ea5fcd700 1 --2- 172.21.15.3:0/4209199893 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ea009e930 0x7f3ea01c0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.660+0000 7f3ea5fcd700 1 -- 172.21.15.3:0/4209199893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e900094a0 con 0x7f3ea009f650 2023-10-03T10:12:55.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.660+0000 7f3ea57cc700 1 --2- 172.21.15.3:0/4209199893 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ea009e930 0x7f3ea01c0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:55.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.660+0000 7f3ea4fcb700 1 --2- 172.21.15.3:0/4209199893 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ea009f240 0x7f3ea01c1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:55.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.660+0000 7f3ea5fcd700 1 --2- 172.21.15.3:0/4209199893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ea009f650 0x7f3ea01c3e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3ea009faf0 tx=0x7f3e9800ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:55.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.660+0000 7f3e95ffb700 1 -- 172.21.15.3:0/4209199893 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e98003c00 con 0x7f3ea009f650 2023-10-03T10:12:55.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.660+0000 7f3e95ffb700 1 -- 172.21.15.3:0/4209199893 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e98003d60 con 0x7f3ea009f650 2023-10-03T10:12:55.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.660+0000 7f3ea7a30700 1 -- 172.21.15.3:0/4209199893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea00a0190 con 0x7f3ea009f650 2023-10-03T10:12:55.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.661+0000 7f3e95ffb700 1 -- 172.21.15.3:0/4209199893 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9800a9c0 con 0x7f3ea009f650 2023-10-03T10:12:55.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.661+0000 7f3ea7a30700 1 -- 172.21.15.3:0/4209199893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea01a49a0 con 0x7f3ea009f650 2023-10-03T10:12:55.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.662+0000 7f3ea7a30700 1 -- 172.21.15.3:0/4209199893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea0074ea0 con 0x7f3ea009f650 2023-10-03T10:12:55.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.662+0000 7f3e95ffb700 1 -- 172.21.15.3:0/4209199893 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f3e9800ab20 con 0x7f3ea009f650 2023-10-03T10:12:55.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.663+0000 7f3e95ffb700 1 -- 172.21.15.3:0/4209199893 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f3e980434b0 con 0x7f3ea009f650 2023-10-03T10:12:55.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.663+0000 7f3ea57cc700 1 -- 172.21.15.3:0/4209199893 >> v1:172.21.15.3:6816/127025 conn(0x7f3e8402f510 legacy=0x7f3e840319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:55.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.668+0000 7f3e95ffb700 1 -- 172.21.15.3:0/4209199893 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3e98019710 con 0x7f3ea009f650 2023-10-03T10:12:55.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.725+0000 7f92737fe700 1 -- 172.21.15.3:0/3865247617 >> v1:172.21.15.3:6816/127025 conn(0x7f926002f510 legacy=0x7f92600319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:55.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.761+0000 7f927a247700 1 -- 172.21.15.3:0/3865247617 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9254005150 con 0x7f92741341d0 2023-10-03T10:12:55.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.762+0000 7f925bfff700 1 -- 172.21.15.3:0/3865247617 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f926c0122e0 con 0x7f92741341d0 2023-10-03T10:12:55.762 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:12:55.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.764+0000 7f927a247700 1 -- 172.21.15.3:0/3865247617 >> v1:172.21.15.3:6816/127025 conn(0x7f926002f510 legacy=0x7f92600319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:55.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.764+0000 7f927a247700 1 -- 172.21.15.3:0/3865247617 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f92741341d0 msgr2=0x7f92741205d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:55.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.764+0000 7f927a247700 1 --2- 172.21.15.3:0/3865247617 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f92741341d0 0x7f92741205d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f926c007d90 tx=0x7f926c00e3c0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.764+0000 7f927a247700 1 -- 172.21.15.3:0/3865247617 shutdown_connections 2023-10-03T10:12:55.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.764+0000 7f927a247700 1 --2- 172.21.15.3:0/3865247617 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f92741341d0 0x7f92741205d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.764+0000 7f927a247700 1 --2- 172.21.15.3:0/3865247617 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f927409e560 0x7f927411fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.764+0000 7f927a247700 1 --2- 172.21.15.3:0/3865247617 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f92741338c0 0x7f9274120090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.764+0000 7f927a247700 1 -- 172.21.15.3:0/3865247617 >> 172.21.15.3:0/3865247617 conn(0x7f92740a2aa0 msgr2=0x7f927409fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:55.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.764+0000 7f927a247700 1 -- 172.21.15.3:0/3865247617 shutdown_connections 2023-10-03T10:12:55.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.764+0000 7f927a247700 1 -- 172.21.15.3:0/3865247617 wait complete. 2023-10-03T10:12:55.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.778+0000 7f65bb7fe700 1 -- 172.21.15.3:0/671668860 >> v1:172.21.15.3:6816/127025 conn(0x7f65a802f510 legacy=0x7f65a80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:55.779 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:12:55.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.820+0000 7fe0caffd700 1 -- 172.21.15.3:0/440459299 >> v1:172.21.15.3:6816/127025 conn(0x7fe0b802f510 legacy=0x7fe0b80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:55.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.834+0000 7f65a27fc700 1 -- 172.21.15.3:0/671668860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6594005860 con 0x7f65bc06fb20 2023-10-03T10:12:55.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.834+0000 7f65b8ff9700 1 -- 172.21.15.3:0/671668860 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f65ac021070 con 0x7f65bc06fb20 2023-10-03T10:12:55.835 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:12:55.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.837+0000 7f65a27fc700 1 -- 172.21.15.3:0/671668860 >> v1:172.21.15.3:6816/127025 conn(0x7f65a802f510 legacy=0x7f65a80319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:55.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.837+0000 7f65a27fc700 1 -- 172.21.15.3:0/671668860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65bc06fb20 msgr2=0x7f65bc118cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:55.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.837+0000 7f65a27fc700 1 --2- 172.21.15.3:0/671668860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65bc06fb20 0x7f65bc118cd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009130 tx=0x7f65ac004750 comp rx=0 tx=0).stop 2023-10-03T10:12:55.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.837+0000 7f65a27fc700 1 -- 172.21.15.3:0/671668860 shutdown_connections 2023-10-03T10:12:55.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.837+0000 7f65a27fc700 1 --2- 172.21.15.3:0/671668860 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65bc130e50 0x7f65bc119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.837+0000 7f65a27fc700 1 --2- 172.21.15.3:0/671668860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65bc06fb20 0x7f65bc118cd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.837+0000 7f65a27fc700 1 --2- 172.21.15.3:0/671668860 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65bc130540 0x7f65bc119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.837+0000 7f65a27fc700 1 -- 172.21.15.3:0/671668860 >> 172.21.15.3:0/671668860 conn(0x7f65bc0a2aa0 msgr2=0x7f65bc12d550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:55.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.837+0000 7f65a27fc700 1 -- 172.21.15.3:0/671668860 shutdown_connections 2023-10-03T10:12:55.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.837+0000 7f65a27fc700 1 -- 172.21.15.3:0/671668860 wait complete. 2023-10-03T10:12:55.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.840+0000 7fe0d1543700 1 -- 172.21.15.3:0/440459299 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe0cc074ea0 con 0x7fe0cc06f050 2023-10-03T10:12:55.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.841+0000 7fe0b37fe700 1 -- 172.21.15.3:0/440459299 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe0c0046020 con 0x7fe0cc06f050 2023-10-03T10:12:55.841 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:12:55.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.844+0000 7fe0d1543700 1 -- 172.21.15.3:0/440459299 >> v1:172.21.15.3:6816/127025 conn(0x7fe0b802f510 legacy=0x7fe0b80319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:55.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.844+0000 7fe0d1543700 1 -- 172.21.15.3:0/440459299 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe0cc06f050 msgr2=0x7fe0cc133920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:55.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.844+0000 7fe0d1543700 1 --2- 172.21.15.3:0/440459299 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe0cc06f050 0x7fe0cc133920 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0000c00 tx=0x7fe0c0004750 comp rx=0 tx=0).stop 2023-10-03T10:12:55.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.844+0000 7fe0d1543700 1 -- 172.21.15.3:0/440459299 shutdown_connections 2023-10-03T10:12:55.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.844+0000 7fe0d1543700 1 --2- 172.21.15.3:0/440459299 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe0cc13e720 0x7fe0cc133e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.844+0000 7fe0d1543700 1 --2- 172.21.15.3:0/440459299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0cc06e570 0x7fe0cc1333e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.844+0000 7fe0d1543700 1 --2- 172.21.15.3:0/440459299 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe0cc06f050 0x7fe0cc133920 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.844+0000 7fe0d1543700 1 -- 172.21.15.3:0/440459299 >> 172.21.15.3:0/440459299 conn(0x7fe0cc0a2aa0 msgr2=0x7fe0cc131050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:55.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.844+0000 7fe0d1543700 1 -- 172.21.15.3:0/440459299 shutdown_connections 2023-10-03T10:12:55.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.844+0000 7fe0d1543700 1 -- 172.21.15.3:0/440459299 wait complete. 2023-10-03T10:12:55.852 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:12:55.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.864+0000 7f3ea57cc700 1 -- 172.21.15.3:0/4209199893 >> v1:172.21.15.3:6816/127025 conn(0x7f3e8402f510 legacy=0x7f3e840319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:55.865 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:12:55.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.911+0000 7f3ea7a30700 1 -- 172.21.15.3:0/4209199893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3ea006b250 con 0x7f3ea009f650 2023-10-03T10:12:55.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.912+0000 7f3e95ffb700 1 -- 172.21.15.3:0/4209199893 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3e98020020 con 0x7f3ea009f650 2023-10-03T10:12:55.912 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:12:55.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.914+0000 7f3ea7a30700 1 -- 172.21.15.3:0/4209199893 >> v1:172.21.15.3:6816/127025 conn(0x7f3e8402f510 legacy=0x7f3e840319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:55.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.914+0000 7f3ea7a30700 1 -- 172.21.15.3:0/4209199893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ea009f650 msgr2=0x7f3ea01c3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:55.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.914+0000 7f3ea7a30700 1 --2- 172.21.15.3:0/4209199893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ea009f650 0x7f3ea01c3e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3ea009faf0 tx=0x7f3e9800ce70 comp rx=0 tx=0).stop 2023-10-03T10:12:55.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.914+0000 7f3ea7a30700 1 -- 172.21.15.3:0/4209199893 shutdown_connections 2023-10-03T10:12:55.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.914+0000 7f3ea7a30700 1 --2- 172.21.15.3:0/4209199893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ea009f650 0x7f3ea01c3e20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.914+0000 7f3ea7a30700 1 --2- 172.21.15.3:0/4209199893 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ea009e930 0x7f3ea01c0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.914+0000 7f3ea7a30700 1 --2- 172.21.15.3:0/4209199893 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ea009f240 0x7f3ea01c1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:55.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.914+0000 7f3ea7a30700 1 -- 172.21.15.3:0/4209199893 >> 172.21.15.3:0/4209199893 conn(0x7f3ea00a4210 msgr2=0x7f3ea01b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:55.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.914+0000 7f3ea7a30700 1 -- 172.21.15.3:0/4209199893 shutdown_connections 2023-10-03T10:12:55.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:55.914+0000 7f3ea7a30700 1 -- 172.21.15.3:0/4209199893 wait complete. 2023-10-03T10:12:55.928 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:12:56.780 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:12:56.854 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:12:56.865 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:12:56.928 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:12:56.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.931+0000 7ffb043f6700 1 Processor -- start 2023-10-03T10:12:56.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.931+0000 7ffb043f6700 1 -- start start 2023-10-03T10:12:56.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.931+0000 7ffb043f6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffafc134630 0x7ffafc136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:56.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.931+0000 7ffb043f6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffafc136f50 0x7ffafc139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:56.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.931+0000 7ffb043f6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffafc09b4f0 con 0x7ffafc131810 2023-10-03T10:12:56.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.931+0000 7ffb043f6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffafc1313c0 con 0x7ffafc136f50 2023-10-03T10:12:56.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.931+0000 7ffb043f6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc06e8c0 con 0x7ffafc134630 2023-10-03T10:12:56.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.932+0000 7ffb01991700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffafc134630 0x7ffafc136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:56.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.932+0000 7ffb01991700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffafc134630 0x7ffafc136a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54684/0 (socket says 172.21.15.3:54684) 2023-10-03T10:12:56.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.932+0000 7ffb02993700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffafc136f50 0x7ffafc139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:56.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.932+0000 7ffb01991700 1 -- 172.21.15.3:0/2913653306 learned_addr learned my addr 172.21.15.3:0/2913653306 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:56.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.932+0000 7ffb02192700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ffafc131810 0x7ffafc131be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45376/0 (socket says 172.21.15.3:45376) 2023-10-03T10:12:56.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.932+0000 7ffb02993700 1 -- 172.21.15.3:0/2913653306 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffafc134630 msgr2=0x7ffafc136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:56.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.932+0000 7ffb02993700 1 --2- 172.21.15.3:0/2913653306 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffafc134630 0x7ffafc136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:56.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.932+0000 7ffb02993700 1 -- 172.21.15.3:0/2913653306 >> v1:172.21.15.3:6789/0 conn(0x7ffafc131810 legacy=0x7ffafc131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:56.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.932+0000 7ffb01190700 1 -- 172.21.15.3:0/2913653306 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4159802124 0 0) 0x7ffafc09b4f0 con 0x7ffafc131810 2023-10-03T10:12:56.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.932+0000 7ffb02993700 1 -- 172.21.15.3:0/2913653306 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffafc071d20 con 0x7ffafc136f50 2023-10-03T10:12:56.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.932+0000 7ffb02993700 1 --2- 172.21.15.3:0/2913653306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffafc136f50 0x7ffafc139330 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009d90 tx=0x7ffaf400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19c95b657e3222a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:56.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.932+0000 7ffb01190700 1 -- 172.21.15.3:0/2913653306 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf400e070 con 0x7ffafc136f50 2023-10-03T10:12:56.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.933+0000 7ffb01190700 1 -- 172.21.15.3:0/2913653306 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf400bb20 con 0x7ffafc136f50 2023-10-03T10:12:56.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.933+0000 7ffb01190700 1 -- 172.21.15.3:0/2913653306 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4012430 con 0x7ffafc136f50 2023-10-03T10:12:56.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.933+0000 7ffb043f6700 1 -- 172.21.15.3:0/2913653306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffafc136f50 msgr2=0x7ffafc139330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:56.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.933+0000 7ffb043f6700 1 --2- 172.21.15.3:0/2913653306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffafc136f50 0x7ffafc139330 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009d90 tx=0x7ffaf400b330 comp rx=0 tx=0).stop 2023-10-03T10:12:56.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.933+0000 7ffb043f6700 1 -- 172.21.15.3:0/2913653306 shutdown_connections 2023-10-03T10:12:56.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.933+0000 7ffb043f6700 1 --2- 172.21.15.3:0/2913653306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffafc136f50 0x7ffafc139330 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:56.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.933+0000 7ffb043f6700 1 --2- 172.21.15.3:0/2913653306 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffafc134630 0x7ffafc136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:56.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.933+0000 7ffb043f6700 1 -- 172.21.15.3:0/2913653306 >> 172.21.15.3:0/2913653306 conn(0x7ffafc0a2aa0 msgr2=0x7ffafc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:56.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.934+0000 7ffb043f6700 1 -- 172.21.15.3:0/2913653306 shutdown_connections 2023-10-03T10:12:56.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.934+0000 7ffb043f6700 1 -- 172.21.15.3:0/2913653306 wait complete. 2023-10-03T10:12:56.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.935+0000 7ffb043f6700 1 Processor -- start 2023-10-03T10:12:56.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.935+0000 7ffb043f6700 1 -- start start 2023-10-03T10:12:56.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.935+0000 7ffb043f6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffafc131810 0x7ffafc09c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:56.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.935+0000 7ffb043f6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffafc134630 0x7ffafc09caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:56.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.935+0000 7ffb043f6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffafc136f50 0x7ffafc09d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:56.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.935+0000 7ffb043f6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc11a330 con 0x7ffafc131810 2023-10-03T10:12:56.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.935+0000 7ffb043f6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc0a2340 con 0x7ffafc136f50 2023-10-03T10:12:56.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.935+0000 7ffb043f6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc06eb00 con 0x7ffafc134630 2023-10-03T10:12:56.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.935+0000 7ffb02192700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffafc131810 0x7ffafc09c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:56.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.935+0000 7ffb02192700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffafc131810 0x7ffafc09c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39064/0 (socket says 172.21.15.3:39064) 2023-10-03T10:12:56.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.935+0000 7ffb02192700 1 -- 172.21.15.3:0/475124081 learned_addr learned my addr 172.21.15.3:0/475124081 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:56.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.936+0000 7ffb02993700 1 --2- 172.21.15.3:0/475124081 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffafc136f50 0x7ffafc09d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:56.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.936+0000 7ffb02192700 1 -- 172.21.15.3:0/475124081 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffafc134630 msgr2=0x7ffafc09caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:56.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.936+0000 7ffb02192700 1 --2- 172.21.15.3:0/475124081 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffafc134630 0x7ffafc09caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:56.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.936+0000 7ffb02192700 1 -- 172.21.15.3:0/475124081 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffafc136f50 msgr2=0x7ffafc09d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:56.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.936+0000 7ffb02192700 1 --2- 172.21.15.3:0/475124081 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffafc136f50 0x7ffafc09d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:56.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.936+0000 7ffb02192700 1 -- 172.21.15.3:0/475124081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf4009a70 con 0x7ffafc131810 2023-10-03T10:12:56.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.936+0000 7ffb02192700 1 --2- 172.21.15.3:0/475124081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffafc131810 0x7ffafc09c5b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ffaf800c520 tx=0x7ffaf800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:56.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.936+0000 7ffaf2ffd700 1 -- 172.21.15.3:0/475124081 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8004c80 con 0x7ffafc131810 2023-10-03T10:12:56.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.936+0000 7ffaf2ffd700 1 -- 172.21.15.3:0/475124081 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf8004e20 con 0x7ffafc131810 2023-10-03T10:12:56.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.936+0000 7ffb043f6700 1 -- 172.21.15.3:0/475124081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffafc118ef0 con 0x7ffafc131810 2023-10-03T10:12:56.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.936+0000 7ffaf2ffd700 1 -- 172.21.15.3:0/475124081 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8007310 con 0x7ffafc131810 2023-10-03T10:12:56.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.936+0000 7ffb043f6700 1 -- 172.21.15.3:0/475124081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffafc11a610 con 0x7ffafc131810 2023-10-03T10:12:56.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.937+0000 7ffaf2ffd700 1 -- 172.21.15.3:0/475124081 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7ffaf8007470 con 0x7ffafc131810 2023-10-03T10:12:56.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.938+0000 7ffb043f6700 1 -- 172.21.15.3:0/475124081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffafc074ea0 con 0x7ffafc131810 2023-10-03T10:12:56.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.938+0000 7ffaf2ffd700 1 -- 172.21.15.3:0/475124081 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7ffaf803f8a0 con 0x7ffafc131810 2023-10-03T10:12:56.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.938+0000 7ffb01991700 1 -- 172.21.15.3:0/475124081 >> v1:172.21.15.3:6816/127025 conn(0x7ffae002f510 legacy=0x7ffae00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:56.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:56.947+0000 7ffaf2ffd700 1 -- 172.21.15.3:0/475124081 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffaf80106d0 con 0x7ffafc131810 2023-10-03T10:12:57.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.002+0000 7f6b68685700 1 Processor -- start 2023-10-03T10:12:57.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.002+0000 7f6b68685700 1 -- start start 2023-10-03T10:12:57.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.003+0000 7f6b68685700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6b6009d6f0 0x7f6b6009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:57.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.003+0000 7f6b68685700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6b6009e000 0x7f6b601339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:57.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.003+0000 7f6b68685700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6b6009e410 0x7f6b60133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:57.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.003+0000 7f6b68685700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b6006b460 con 0x7f6b6009e410 2023-10-03T10:12:57.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.003+0000 7f6b68685700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6009b120 con 0x7f6b6009d6f0 2023-10-03T10:12:57.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.003+0000 7f6b68685700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6009a7e0 con 0x7f6b6009e000 2023-10-03T10:12:57.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.003+0000 7f6b65c20700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6b6009e000 0x7f6b601339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:57.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.003+0000 7f6b65c20700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6b6009e000 0x7f6b601339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54700/0 (socket says 172.21.15.3:54700) 2023-10-03T10:12:57.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.003+0000 7f6b65c20700 1 -- 172.21.15.3:0/3695344960 learned_addr learned my addr 172.21.15.3:0/3695344960 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:57.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.003+0000 7f6b66421700 1 --2- 172.21.15.3:0/3695344960 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6b6009d6f0 0x7f6b6009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:57.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.003+0000 7f6b66c22700 1 --2- 172.21.15.3:0/3695344960 >> v2:172.21.15.3:3300/0 conn(0x7f6b6009e410 0x7f6b60133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:57.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.004+0000 7f6b65c20700 1 -- 172.21.15.3:0/3695344960 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6b6009d6f0 msgr2=0x7f6b6009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:57.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.004+0000 7f6b65c20700 1 --2- 172.21.15.3:0/3695344960 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6b6009d6f0 0x7f6b6009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.004+0000 7f6b65c20700 1 -- 172.21.15.3:0/3695344960 >> v2:172.21.15.3:3300/0 conn(0x7f6b6009e410 msgr2=0x7f6b60133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:57.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.004+0000 7f6b65c20700 1 --2- 172.21.15.3:0/3695344960 >> v2:172.21.15.3:3300/0 conn(0x7f6b6009e410 0x7f6b60133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.004+0000 7f6b65c20700 1 -- 172.21.15.3:0/3695344960 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6009a500 con 0x7f6b6009e000 2023-10-03T10:12:57.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.004+0000 7f6b65c20700 1 --2- 172.21.15.3:0/3695344960 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6b6009e000 0x7f6b601339f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009920 tx=0x7f6b50009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5f53d3117379197 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:57.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.004+0000 7f6b6541f700 1 -- 172.21.15.3:0/3695344960 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b50011830 con 0x7f6b6009e000 2023-10-03T10:12:57.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.004+0000 7f6b6541f700 1 -- 172.21.15.3:0/3695344960 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b50011990 con 0x7f6b6009e000 2023-10-03T10:12:57.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.004+0000 7f6b6541f700 1 -- 172.21.15.3:0/3695344960 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b50011d20 con 0x7f6b6009e000 2023-10-03T10:12:57.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.005+0000 7f6b68685700 1 -- 172.21.15.3:0/3695344960 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6b6009e000 msgr2=0x7f6b601339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:57.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.005+0000 7f6b68685700 1 --2- 172.21.15.3:0/3695344960 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6b6009e000 0x7f6b601339f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009920 tx=0x7f6b50009c30 comp rx=0 tx=0).stop 2023-10-03T10:12:57.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.005+0000 7f6b68685700 1 -- 172.21.15.3:0/3695344960 shutdown_connections 2023-10-03T10:12:57.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.005+0000 7f6b68685700 1 --2- 172.21.15.3:0/3695344960 >> v2:172.21.15.3:3300/0 conn(0x7f6b6009e410 0x7f6b60133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.005+0000 7f6b68685700 1 --2- 172.21.15.3:0/3695344960 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6b6009d6f0 0x7f6b6009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.005+0000 7f6b68685700 1 --2- 172.21.15.3:0/3695344960 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6b6009e000 0x7f6b601339f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.005+0000 7f6b68685700 1 -- 172.21.15.3:0/3695344960 >> 172.21.15.3:0/3695344960 conn(0x7f6b60066f50 msgr2=0x7f6b6009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:57.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.005+0000 7f6b68685700 1 -- 172.21.15.3:0/3695344960 shutdown_connections 2023-10-03T10:12:57.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.005+0000 7f6b68685700 1 -- 172.21.15.3:0/3695344960 wait complete. 2023-10-03T10:12:57.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.007+0000 7f6b68685700 1 Processor -- start 2023-10-03T10:12:57.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.007+0000 7f6b68685700 1 -- start start 2023-10-03T10:12:57.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.008+0000 7f6b68685700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6b6009d6f0 0x7f6b601b91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:57.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.008+0000 7f6b68685700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6b6009e000 0x7f6b601a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:57.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.008+0000 7f6b68685700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b6009e410 0x7f6b601a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:57.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.008+0000 7f6b68685700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b601aa2c0 con 0x7f6b6009e410 2023-10-03T10:12:57.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.008+0000 7f6b68685700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b601a9900 con 0x7f6b6009d6f0 2023-10-03T10:12:57.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.008+0000 7f6b68685700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6b601a9090 con 0x7f6b6009e000 2023-10-03T10:12:57.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.008+0000 7f6b65c20700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6b6009e000 0x7f6b601a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:57.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.008+0000 7f6b65c20700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6b6009e000 0x7f6b601a5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54708/0 (socket says 172.21.15.3:54708) 2023-10-03T10:12:57.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.008+0000 7f6b65c20700 1 -- 172.21.15.3:0/2218150945 learned_addr learned my addr 172.21.15.3:0/2218150945 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:57.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.008+0000 7f6b66c22700 1 --2- 172.21.15.3:0/2218150945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b6009e410 0x7f6b601a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:57.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.008+0000 7f6b65c20700 1 -- 172.21.15.3:0/2218150945 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6b6009d6f0 msgr2=0x7f6b601b91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:57.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.008+0000 7f6b65c20700 1 --2- 172.21.15.3:0/2218150945 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6b6009d6f0 0x7f6b601b91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.008+0000 7f6b65c20700 1 -- 172.21.15.3:0/2218150945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b6009e410 msgr2=0x7f6b601a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:57.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.008+0000 7f6b65c20700 1 --2- 172.21.15.3:0/2218150945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b6009e410 0x7f6b601a6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.008+0000 7f6b65c20700 1 -- 172.21.15.3:0/2218150945 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b50009550 con 0x7f6b6009e000 2023-10-03T10:12:57.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.008+0000 7f6b66c22700 1 --2- 172.21.15.3:0/2218150945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b6009e410 0x7f6b601a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:12:57.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.009+0000 7f6b65c20700 1 --2- 172.21.15.3:0/2218150945 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6b6009e000 0x7f6b601a5de0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009c70 tx=0x7f6b50004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:57.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.009+0000 7f6b56ffd700 1 -- 172.21.15.3:0/2218150945 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b50004400 con 0x7f6b6009e000 2023-10-03T10:12:57.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.009+0000 7f6b56ffd700 1 -- 172.21.15.3:0/2218150945 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5001f070 con 0x7f6b6009e000 2023-10-03T10:12:57.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.009+0000 7f6b56ffd700 1 -- 172.21.15.3:0/2218150945 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b500038e0 con 0x7f6b6009e000 2023-10-03T10:12:57.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.009+0000 7f6b68685700 1 -- 172.21.15.3:0/2218150945 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b601a8c40 con 0x7f6b6009e000 2023-10-03T10:12:57.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.009+0000 7f6b68685700 1 -- 172.21.15.3:0/2218150945 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b601a79f0 con 0x7f6b6009e000 2023-10-03T10:12:57.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.010+0000 7f6b56ffd700 1 -- 172.21.15.3:0/2218150945 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f6b50021030 con 0x7f6b6009e000 2023-10-03T10:12:57.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.010+0000 7f6b56ffd700 1 -- 172.21.15.3:0/2218150945 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f6b50040f10 con 0x7f6b6009e000 2023-10-03T10:12:57.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.010+0000 7f6b66421700 1 -- 172.21.15.3:0/2218150945 >> v1:172.21.15.3:6816/127025 conn(0x7f6b4402f510 legacy=0x7f6b440319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:57.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.011+0000 7f6b68685700 1 -- 172.21.15.3:0/2218150945 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b60074ea0 con 0x7f6b6009e000 2023-10-03T10:12:57.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.020+0000 7f6b56ffd700 1 -- 172.21.15.3:0/2218150945 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6b50015020 con 0x7f6b6009e000 2023-10-03T10:12:57.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.045+0000 7f761250a700 1 Processor -- start 2023-10-03T10:12:57.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.046+0000 7f761250a700 1 -- start start 2023-10-03T10:12:57.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.046+0000 7f761250a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f760c09d6f0 0x7f760c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:57.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.046+0000 7f761250a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f760c09e000 0x7f760c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:57.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.046+0000 7f761250a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f760c09e410 0x7f760c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:57.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.046+0000 7f761250a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f760c09f520 con 0x7f760c09e410 2023-10-03T10:12:57.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.046+0000 7f761250a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f760c09f130 con 0x7f760c09e000 2023-10-03T10:12:57.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.046+0000 7f761250a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f760c09e930 con 0x7f760c09d6f0 2023-10-03T10:12:57.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.046+0000 7f760bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f760c09d6f0 0x7f760c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:57.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.046+0000 7f7610aa7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f760c09e410 0x7f760c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:57.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.046+0000 7f760b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f760c09e000 0x7f760c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:57.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.046+0000 7f760bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f760c09d6f0 0x7f760c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54716/0 (socket says 172.21.15.3:54716) 2023-10-03T10:12:57.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.046+0000 7f7610aa7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f760c09e410 0x7f760c130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39090/0 (socket says 172.21.15.3:39090) 2023-10-03T10:12:57.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.046+0000 7f760bfff700 1 -- 172.21.15.3:0/36380438 learned_addr learned my addr 172.21.15.3:0/36380438 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:57.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.047+0000 7f760b7fe700 1 -- 172.21.15.3:0/36380438 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f760c09d6f0 msgr2=0x7f760c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:57.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.047+0000 7f760b7fe700 1 --2- 172.21.15.3:0/36380438 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f760c09d6f0 0x7f760c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.047+0000 7f760b7fe700 1 -- 172.21.15.3:0/36380438 >> v2:172.21.15.3:3300/0 conn(0x7f760c09e410 msgr2=0x7f760c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:57.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.047+0000 7f760b7fe700 1 --2- 172.21.15.3:0/36380438 >> v2:172.21.15.3:3300/0 conn(0x7f760c09e410 0x7f760c130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.047+0000 7f760b7fe700 1 -- 172.21.15.3:0/36380438 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f760c09b2d0 con 0x7f760c09e000 2023-10-03T10:12:57.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.047+0000 7f760b7fe700 1 --2- 172.21.15.3:0/36380438 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f760c09e000 0x7f760c130660 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f760000dc90 tx=0x7f7600008050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0293f2cc743b005 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:57.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.047+0000 7f760affd700 1 -- 172.21.15.3:0/36380438 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760000b070 con 0x7f760c09e000 2023-10-03T10:12:57.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.047+0000 7f760affd700 1 -- 172.21.15.3:0/36380438 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7600008a00 con 0x7f760c09e000 2023-10-03T10:12:57.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.047+0000 7f760affd700 1 -- 172.21.15.3:0/36380438 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7600008d90 con 0x7f760c09e000 2023-10-03T10:12:57.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.048+0000 7f761250a700 1 -- 172.21.15.3:0/36380438 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f760c09e000 msgr2=0x7f760c130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:57.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.048+0000 7f761250a700 1 --2- 172.21.15.3:0/36380438 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f760c09e000 0x7f760c130660 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f760000dc90 tx=0x7f7600008050 comp rx=0 tx=0).stop 2023-10-03T10:12:57.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.048+0000 7f761250a700 1 -- 172.21.15.3:0/36380438 shutdown_connections 2023-10-03T10:12:57.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.048+0000 7f761250a700 1 --2- 172.21.15.3:0/36380438 >> v2:172.21.15.3:3300/0 conn(0x7f760c09e410 0x7f760c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.048+0000 7f761250a700 1 --2- 172.21.15.3:0/36380438 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f760c09d6f0 0x7f760c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.048+0000 7f761250a700 1 --2- 172.21.15.3:0/36380438 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f760c09e000 0x7f760c130660 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.048+0000 7f761250a700 1 -- 172.21.15.3:0/36380438 >> 172.21.15.3:0/36380438 conn(0x7f760c066f50 msgr2=0x7f760c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:57.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.048+0000 7f761250a700 1 -- 172.21.15.3:0/36380438 shutdown_connections 2023-10-03T10:12:57.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.048+0000 7f761250a700 1 -- 172.21.15.3:0/36380438 wait complete. 2023-10-03T10:12:57.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.049+0000 7f761250a700 1 Processor -- start 2023-10-03T10:12:57.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.049+0000 7f761250a700 1 -- start start 2023-10-03T10:12:57.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.050+0000 7f761250a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f760c09d6f0 0x7f760c11e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:57.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.050+0000 7f761250a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f760c09e000 0x7f760c11e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:57.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.050+0000 7f761250a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f760c09e410 0x7f760c11eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:57.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.050+0000 7f761250a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760c120560 con 0x7f760c09e410 2023-10-03T10:12:57.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.050+0000 7f761250a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f760c120240 con 0x7f760c09d6f0 2023-10-03T10:12:57.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.050+0000 7f761250a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f760c072590 con 0x7f760c09e000 2023-10-03T10:12:57.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.050+0000 7f760b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f760c09e000 0x7f760c11e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:57.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.050+0000 7f760b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f760c09e000 0x7f760c11e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54724/0 (socket says 172.21.15.3:54724) 2023-10-03T10:12:57.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.050+0000 7f760b7fe700 1 -- 172.21.15.3:0/2287877080 learned_addr learned my addr 172.21.15.3:0/2287877080 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:57.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.050+0000 7f7610aa7700 1 --2- 172.21.15.3:0/2287877080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f760c09e410 0x7f760c11eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:57.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.050+0000 7f760bfff700 1 --2- 172.21.15.3:0/2287877080 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f760c09d6f0 0x7f760c11e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:57.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.051+0000 7f760b7fe700 1 -- 172.21.15.3:0/2287877080 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f760c09d6f0 msgr2=0x7f760c11e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:57.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.051+0000 7f760b7fe700 1 --2- 172.21.15.3:0/2287877080 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f760c09d6f0 0x7f760c11e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.051+0000 7f760b7fe700 1 -- 172.21.15.3:0/2287877080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f760c09e410 msgr2=0x7f760c11eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:57.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.051+0000 7f760b7fe700 1 --2- 172.21.15.3:0/2287877080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f760c09e410 0x7f760c11eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.051+0000 7f760b7fe700 1 -- 172.21.15.3:0/2287877080 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f760000d940 con 0x7f760c09e000 2023-10-03T10:12:57.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.051+0000 7f7610aa7700 1 --2- 172.21.15.3:0/2287877080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f760c09e410 0x7f760c11eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:57.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.051+0000 7f760b7fe700 1 --2- 172.21.15.3:0/2287877080 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f760c09e000 0x7f760c11e610 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7600008090 tx=0x7f7600004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:57.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.051+0000 7f7608ff9700 1 -- 172.21.15.3:0/2287877080 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760001b070 con 0x7f760c09e000 2023-10-03T10:12:57.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.051+0000 7f7608ff9700 1 -- 172.21.15.3:0/2287877080 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7600008800 con 0x7f760c09e000 2023-10-03T10:12:57.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.051+0000 7f761250a700 1 -- 172.21.15.3:0/2287877080 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f760c121330 con 0x7f760c09e000 2023-10-03T10:12:57.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.051+0000 7f7608ff9700 1 -- 172.21.15.3:0/2287877080 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760000b070 con 0x7f760c09e000 2023-10-03T10:12:57.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.051+0000 7f761250a700 1 -- 172.21.15.3:0/2287877080 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f760c1218c0 con 0x7f760c09e000 2023-10-03T10:12:57.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.052+0000 7f7608ff9700 1 -- 172.21.15.3:0/2287877080 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f7600004030 con 0x7f760c09e000 2023-10-03T10:12:57.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.052+0000 7f761250a700 1 -- 172.21.15.3:0/2287877080 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f760c121700 con 0x7f760c09e000 2023-10-03T10:12:57.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.053+0000 7f7608ff9700 1 -- 172.21.15.3:0/2287877080 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f7600040890 con 0x7f760c09e000 2023-10-03T10:12:57.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.053+0000 7f760bfff700 1 -- 172.21.15.3:0/2287877080 >> v1:172.21.15.3:6816/127025 conn(0x7f75f802f510 legacy=0x7f75f80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:57.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.058+0000 7f7608ff9700 1 -- 172.21.15.3:0/2287877080 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f76000036a0 con 0x7f760c09e000 2023-10-03T10:12:57.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.086+0000 7fcb66b2a700 1 Processor -- start 2023-10-03T10:12:57.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.086+0000 7fcb66b2a700 1 -- start start 2023-10-03T10:12:57.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.087+0000 7fcb66b2a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb6006a770 0x7fcb6009d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:57.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.087+0000 7fcb66b2a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb6009dc30 0x7fcb6009e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:57.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.087+0000 7fcb66b2a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcb6013c1e0 0x7fcb6013e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:57.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.087+0000 7fcb66b2a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb6006ab40 con 0x7fcb6013c1e0 2023-10-03T10:12:57.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.087+0000 7fcb66b2a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6009ebc0 con 0x7fcb6006a770 2023-10-03T10:12:57.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.087+0000 7fcb66b2a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6009a810 con 0x7fcb6009dc30 2023-10-03T10:12:57.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.087+0000 7fcb5ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb6009dc30 0x7fcb6009e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:57.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.087+0000 7fcb5ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb6009dc30 0x7fcb6009e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54726/0 (socket says 172.21.15.3:54726) 2023-10-03T10:12:57.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.087+0000 7fcb5ffff700 1 -- 172.21.15.3:0/2391338928 learned_addr learned my addr 172.21.15.3:0/2391338928 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:57.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.087+0000 7fcb650c7700 1 --2- 172.21.15.3:0/2391338928 >> v2:172.21.15.3:3300/0 conn(0x7fcb6013c1e0 0x7fcb6013e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:57.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.087+0000 7fcb5ffff700 1 -- 172.21.15.3:0/2391338928 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb6006a770 msgr2=0x7fcb6009d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:57.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.087+0000 7fcb5ffff700 1 --2- 172.21.15.3:0/2391338928 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb6006a770 0x7fcb6009d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.087+0000 7fcb5ffff700 1 -- 172.21.15.3:0/2391338928 >> v2:172.21.15.3:3300/0 conn(0x7fcb6013c1e0 msgr2=0x7fcb6013e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:57.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.087+0000 7fcb5ffff700 1 --2- 172.21.15.3:0/2391338928 >> v2:172.21.15.3:3300/0 conn(0x7fcb6013c1e0 0x7fcb6013e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.087+0000 7fcb5ffff700 1 -- 172.21.15.3:0/2391338928 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6009e920 con 0x7fcb6009dc30 2023-10-03T10:12:57.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.088+0000 7fcb650c7700 1 --2- 172.21.15.3:0/2391338928 >> v2:172.21.15.3:3300/0 conn(0x7fcb6013c1e0 0x7fcb6013e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:57.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.088+0000 7fcb5ffff700 1 --2- 172.21.15.3:0/2391338928 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb6009dc30 0x7fcb6009e080 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009920 tx=0x7fcb4c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a72a8f9919cb85e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:57.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.088+0000 7fcb5f7fe700 1 -- 172.21.15.3:0/2391338928 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c011830 con 0x7fcb6009dc30 2023-10-03T10:12:57.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.088+0000 7fcb5f7fe700 1 -- 172.21.15.3:0/2391338928 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4c011990 con 0x7fcb6009dc30 2023-10-03T10:12:57.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.088+0000 7fcb5f7fe700 1 -- 172.21.15.3:0/2391338928 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c011d00 con 0x7fcb6009dc30 2023-10-03T10:12:57.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.089+0000 7fcb66b2a700 1 -- 172.21.15.3:0/2391338928 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb6009dc30 msgr2=0x7fcb6009e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:57.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.089+0000 7fcb66b2a700 1 --2- 172.21.15.3:0/2391338928 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb6009dc30 0x7fcb6009e080 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009920 tx=0x7fcb4c009c30 comp rx=0 tx=0).stop 2023-10-03T10:12:57.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.089+0000 7fcb66b2a700 1 -- 172.21.15.3:0/2391338928 shutdown_connections 2023-10-03T10:12:57.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.089+0000 7fcb66b2a700 1 --2- 172.21.15.3:0/2391338928 >> v2:172.21.15.3:3300/0 conn(0x7fcb6013c1e0 0x7fcb6013e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.089+0000 7fcb66b2a700 1 --2- 172.21.15.3:0/2391338928 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb6006a770 0x7fcb6009d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.089+0000 7fcb66b2a700 1 --2- 172.21.15.3:0/2391338928 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb6009dc30 0x7fcb6009e080 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.089+0000 7fcb66b2a700 1 -- 172.21.15.3:0/2391338928 >> 172.21.15.3:0/2391338928 conn(0x7fcb60066f50 msgr2=0x7fcb6009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:57.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.089+0000 7fcb66b2a700 1 -- 172.21.15.3:0/2391338928 shutdown_connections 2023-10-03T10:12:57.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.089+0000 7fcb66b2a700 1 -- 172.21.15.3:0/2391338928 wait complete. 2023-10-03T10:12:57.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.090+0000 7fcb66b2a700 1 Processor -- start 2023-10-03T10:12:57.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.090+0000 7fcb66b2a700 1 -- start start 2023-10-03T10:12:57.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.091+0000 7fcb66b2a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb6006a770 0x7fcb6013a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:57.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.091+0000 7fcb66b2a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb6009dc30 0x7fcb6013abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:57.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.091+0000 7fcb66b2a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb6013c1e0 0x7fcb6013b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:57.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.091+0000 7fcb66b2a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb600724d0 con 0x7fcb6013c1e0 2023-10-03T10:12:57.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.091+0000 7fcb66b2a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6009f1a0 con 0x7fcb6006a770 2023-10-03T10:12:57.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.091+0000 7fcb66b2a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6011f490 con 0x7fcb6009dc30 2023-10-03T10:12:57.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.091+0000 7fcb650c7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb6013c1e0 0x7fcb6013b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:57.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.091+0000 7fcb650c7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb6013c1e0 0x7fcb6013b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39120/0 (socket says 172.21.15.3:39120) 2023-10-03T10:12:57.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.091+0000 7fcb650c7700 1 -- 172.21.15.3:0/3511851812 learned_addr learned my addr 172.21.15.3:0/3511851812 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:57.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.091+0000 7fcb648c6700 1 --2- 172.21.15.3:0/3511851812 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb6006a770 0x7fcb6013a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:57.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.091+0000 7fcb5ffff700 1 --2- 172.21.15.3:0/3511851812 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb6009dc30 0x7fcb6013abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:57.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.091+0000 7fcb650c7700 1 -- 172.21.15.3:0/3511851812 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb6009dc30 msgr2=0x7fcb6013abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:57.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.091+0000 7fcb650c7700 1 --2- 172.21.15.3:0/3511851812 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb6009dc30 0x7fcb6013abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.091+0000 7fcb650c7700 1 -- 172.21.15.3:0/3511851812 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb6006a770 msgr2=0x7fcb6013a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:57.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.091+0000 7fcb650c7700 1 --2- 172.21.15.3:0/3511851812 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb6006a770 0x7fcb6013a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.091+0000 7fcb650c7700 1 -- 172.21.15.3:0/3511851812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb58009a70 con 0x7fcb6013c1e0 2023-10-03T10:12:57.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.092+0000 7fcb650c7700 1 --2- 172.21.15.3:0/3511851812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb6013c1e0 0x7fcb6013b120 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800efc0 tx=0x7fcb5800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:57.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.092+0000 7fcb5d7fa700 1 -- 172.21.15.3:0/3511851812 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5800ca90 con 0x7fcb6013c1e0 2023-10-03T10:12:57.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.092+0000 7fcb5d7fa700 1 -- 172.21.15.3:0/3511851812 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5800f040 con 0x7fcb6013c1e0 2023-10-03T10:12:57.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.092+0000 7fcb66b2a700 1 -- 172.21.15.3:0/3511851812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb4c009550 con 0x7fcb6013c1e0 2023-10-03T10:12:57.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.092+0000 7fcb5d7fa700 1 -- 172.21.15.3:0/3511851812 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5800ce10 con 0x7fcb6013c1e0 2023-10-03T10:12:57.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.092+0000 7fcb66b2a700 1 -- 172.21.15.3:0/3511851812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6013bf60 con 0x7fcb6013c1e0 2023-10-03T10:12:57.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.093+0000 7fcb5d7fa700 1 -- 172.21.15.3:0/3511851812 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fcb58014480 con 0x7fcb6013c1e0 2023-10-03T10:12:57.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.093+0000 7fcb648c6700 1 -- 172.21.15.3:0/3511851812 >> v1:172.21.15.3:6816/127025 conn(0x7fcb4402f510 legacy=0x7fcb440319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:57.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.093+0000 7fcb5d7fa700 1 -- 172.21.15.3:0/3511851812 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7fcb58040470 con 0x7fcb6013c1e0 2023-10-03T10:12:57.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.094+0000 7fcb66b2a700 1 -- 172.21.15.3:0/3511851812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb6011cc30 con 0x7fcb6013c1e0 2023-10-03T10:12:57.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.103+0000 7fcb5d7fa700 1 -- 172.21.15.3:0/3511851812 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcb58018080 con 0x7fcb6013c1e0 2023-10-03T10:12:57.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.139+0000 7ffb01991700 1 -- 172.21.15.3:0/475124081 >> v1:172.21.15.3:6816/127025 conn(0x7ffae002f510 legacy=0x7ffae00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:57.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.167+0000 7ffb043f6700 1 -- 172.21.15.3:0/475124081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ffafc11a470 con 0x7ffafc131810 2023-10-03T10:12:57.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.168+0000 7ffaf2ffd700 1 -- 172.21.15.3:0/475124081 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffafc11a470 con 0x7ffafc131810 2023-10-03T10:12:57.168 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:12:57.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.171+0000 7ffb043f6700 1 -- 172.21.15.3:0/475124081 >> v1:172.21.15.3:6816/127025 conn(0x7ffae002f510 legacy=0x7ffae00319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:57.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.171+0000 7ffb043f6700 1 -- 172.21.15.3:0/475124081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffafc131810 msgr2=0x7ffafc09c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:57.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.171+0000 7ffb043f6700 1 --2- 172.21.15.3:0/475124081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffafc131810 0x7ffafc09c5b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ffaf800c520 tx=0x7ffaf800c8e0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.172+0000 7ffb043f6700 1 -- 172.21.15.3:0/475124081 shutdown_connections 2023-10-03T10:12:57.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.172+0000 7ffb043f6700 1 --2- 172.21.15.3:0/475124081 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffafc136f50 0x7ffafc09d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.172+0000 7ffb043f6700 1 --2- 172.21.15.3:0/475124081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffafc131810 0x7ffafc09c5b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.172+0000 7ffb043f6700 1 --2- 172.21.15.3:0/475124081 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffafc134630 0x7ffafc09caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.172+0000 7ffb043f6700 1 -- 172.21.15.3:0/475124081 >> 172.21.15.3:0/475124081 conn(0x7ffafc0a2aa0 msgr2=0x7ffafc0a0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:57.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.172+0000 7ffb043f6700 1 -- 172.21.15.3:0/475124081 shutdown_connections 2023-10-03T10:12:57.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.172+0000 7ffb043f6700 1 -- 172.21.15.3:0/475124081 wait complete. 2023-10-03T10:12:57.187 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:12:57.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.212+0000 7f6b66421700 1 -- 172.21.15.3:0/2218150945 >> v1:172.21.15.3:6816/127025 conn(0x7f6b4402f510 legacy=0x7f6b440319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:57.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.254+0000 7f760bfff700 1 -- 172.21.15.3:0/2287877080 >> v1:172.21.15.3:6816/127025 conn(0x7f75f802f510 legacy=0x7f75f80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:57.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.271+0000 7f761250a700 1 -- 172.21.15.3:0/2287877080 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f760c121520 con 0x7f760c09e000 2023-10-03T10:12:57.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.271+0000 7f7608ff9700 1 -- 172.21.15.3:0/2287877080 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7600024070 con 0x7f760c09e000 2023-10-03T10:12:57.271 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:12:57.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.273+0000 7f761250a700 1 -- 172.21.15.3:0/2287877080 >> v1:172.21.15.3:6816/127025 conn(0x7f75f802f510 legacy=0x7f75f80319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:57.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.273+0000 7f761250a700 1 -- 172.21.15.3:0/2287877080 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f760c09e000 msgr2=0x7f760c11e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:57.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.273+0000 7f761250a700 1 --2- 172.21.15.3:0/2287877080 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f760c09e000 0x7f760c11e610 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7600008090 tx=0x7f7600004750 comp rx=0 tx=0).stop 2023-10-03T10:12:57.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.273+0000 7f761250a700 1 -- 172.21.15.3:0/2287877080 shutdown_connections 2023-10-03T10:12:57.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.273+0000 7f761250a700 1 --2- 172.21.15.3:0/2287877080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f760c09e410 0x7f760c11eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.273+0000 7f761250a700 1 --2- 172.21.15.3:0/2287877080 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f760c09d6f0 0x7f760c11e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.273+0000 7f761250a700 1 --2- 172.21.15.3:0/2287877080 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f760c09e000 0x7f760c11e610 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.273+0000 7f761250a700 1 -- 172.21.15.3:0/2287877080 >> 172.21.15.3:0/2287877080 conn(0x7f760c066f50 msgr2=0x7f760c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:57.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.274+0000 7f761250a700 1 -- 172.21.15.3:0/2287877080 shutdown_connections 2023-10-03T10:12:57.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.274+0000 7f761250a700 1 -- 172.21.15.3:0/2287877080 wait complete. 2023-10-03T10:12:57.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.274+0000 7f6b68685700 1 -- 172.21.15.3:0/2218150945 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6b6006c360 con 0x7f6b6009e000 2023-10-03T10:12:57.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.275+0000 7f6b56ffd700 1 -- 172.21.15.3:0/2218150945 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6b50004580 con 0x7f6b6009e000 2023-10-03T10:12:57.276 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:12:57.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.277+0000 7f6b68685700 1 -- 172.21.15.3:0/2218150945 >> v1:172.21.15.3:6816/127025 conn(0x7f6b4402f510 legacy=0x7f6b440319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:57.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.277+0000 7f6b68685700 1 -- 172.21.15.3:0/2218150945 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6b6009e000 msgr2=0x7f6b601a5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:57.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.277+0000 7f6b68685700 1 --2- 172.21.15.3:0/2218150945 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6b6009e000 0x7f6b601a5de0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009c70 tx=0x7f6b50004750 comp rx=0 tx=0).stop 2023-10-03T10:12:57.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.277+0000 7f6b68685700 1 -- 172.21.15.3:0/2218150945 shutdown_connections 2023-10-03T10:12:57.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.277+0000 7f6b68685700 1 --2- 172.21.15.3:0/2218150945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b6009e410 0x7f6b601a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.277+0000 7f6b68685700 1 --2- 172.21.15.3:0/2218150945 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6b6009d6f0 0x7f6b601b91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.277+0000 7f6b68685700 1 --2- 172.21.15.3:0/2218150945 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6b6009e000 0x7f6b601a5de0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.277+0000 7f6b68685700 1 -- 172.21.15.3:0/2218150945 >> 172.21.15.3:0/2218150945 conn(0x7f6b60066f50 msgr2=0x7f6b601badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:57.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.277+0000 7f6b68685700 1 -- 172.21.15.3:0/2218150945 shutdown_connections 2023-10-03T10:12:57.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.277+0000 7f6b68685700 1 -- 172.21.15.3:0/2218150945 wait complete. 2023-10-03T10:12:57.290 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:12:57.294 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:12:57.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.294+0000 7fcb648c6700 1 -- 172.21.15.3:0/3511851812 >> v1:172.21.15.3:6816/127025 conn(0x7fcb4402f510 legacy=0x7fcb440319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:57.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.342+0000 7fcb66b2a700 1 -- 172.21.15.3:0/3511851812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcb6011fc70 con 0x7fcb6013c1e0 2023-10-03T10:12:57.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.343+0000 7fcb5d7fa700 1 -- 172.21.15.3:0/3511851812 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcb58012930 con 0x7fcb6013c1e0 2023-10-03T10:12:57.343 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:12:57.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.345+0000 7fcb66b2a700 1 -- 172.21.15.3:0/3511851812 >> v1:172.21.15.3:6816/127025 conn(0x7fcb4402f510 legacy=0x7fcb440319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:57.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.345+0000 7fcb66b2a700 1 -- 172.21.15.3:0/3511851812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb6013c1e0 msgr2=0x7fcb6013b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:57.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.345+0000 7fcb66b2a700 1 --2- 172.21.15.3:0/3511851812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb6013c1e0 0x7fcb6013b120 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800efc0 tx=0x7fcb5800c3a0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.345+0000 7fcb66b2a700 1 -- 172.21.15.3:0/3511851812 shutdown_connections 2023-10-03T10:12:57.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.345+0000 7fcb66b2a700 1 --2- 172.21.15.3:0/3511851812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb6013c1e0 0x7fcb6013b120 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.345+0000 7fcb66b2a700 1 --2- 172.21.15.3:0/3511851812 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb6006a770 0x7fcb6013a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.345+0000 7fcb66b2a700 1 --2- 172.21.15.3:0/3511851812 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb6009dc30 0x7fcb6013abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:57.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.345+0000 7fcb66b2a700 1 -- 172.21.15.3:0/3511851812 >> 172.21.15.3:0/3511851812 conn(0x7fcb60066f50 msgr2=0x7fcb600a1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:57.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.345+0000 7fcb66b2a700 1 -- 172.21.15.3:0/3511851812 shutdown_connections 2023-10-03T10:12:57.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:57.345+0000 7fcb66b2a700 1 -- 172.21.15.3:0/3511851812 wait complete. 2023-10-03T10:12:57.359 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:12:58.188 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:12:58.290 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:12:58.295 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:12:58.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.336+0000 7fef652d3700 1 Processor -- start 2023-10-03T10:12:58.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.336+0000 7fef652d3700 1 -- start start 2023-10-03T10:12:58.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.336+0000 7fef652d3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fef6009e1d0 0x7fef60138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:58.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.336+0000 7fef652d3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fef60139170 0x7fef6013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:58.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.336+0000 7fef652d3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef6009b120 con 0x7fef6009d6f0 2023-10-03T10:12:58.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.336+0000 7fef652d3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef6006b460 con 0x7fef60139170 2023-10-03T10:12:58.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.336+0000 7fef652d3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef6009a7e0 con 0x7fef6009e1d0 2023-10-03T10:12:58.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.336+0000 7fef5e7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fef6009e1d0 0x7fef60138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:58.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.337+0000 7fef5f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fef60139170 0x7fef6013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:58.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.337+0000 7fef5e7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fef6009e1d0 0x7fef60138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42992/0 (socket says 172.21.15.3:42992) 2023-10-03T10:12:58.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.337+0000 7fef5e7fc700 1 -- 172.21.15.3:0/1851405253 learned_addr learned my addr 172.21.15.3:0/1851405253 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:58.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.337+0000 7fef5e7fc700 1 -- 172.21.15.3:0/1851405253 >> v2:172.21.15.3:3300/0 conn(0x7fef60139170 msgr2=0x7fef6013b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:58.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.337+0000 7fef5e7fc700 1 --2- 172.21.15.3:0/1851405253 >> v2:172.21.15.3:3300/0 conn(0x7fef60139170 0x7fef6013b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.337+0000 7fef5e7fc700 1 -- 172.21.15.3:0/1851405253 >> v1:172.21.15.3:6789/0 conn(0x7fef6009d6f0 legacy=0x7fef6009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:58.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.337+0000 7fef5e7fc700 1 -- 172.21.15.3:0/1851405253 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6009a500 con 0x7fef6009e1d0 2023-10-03T10:12:58.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.337+0000 7fef5e7fc700 1 --2- 172.21.15.3:0/1851405253 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fef6009e1d0 0x7fef60138c30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fef54009920 tx=0x7fef54009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68a305502ad92cb3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:58.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.337+0000 7fef5dffb700 1 -- 172.21.15.3:0/1851405253 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef54011830 con 0x7fef6009e1d0 2023-10-03T10:12:58.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.337+0000 7fef5dffb700 1 -- 172.21.15.3:0/1851405253 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef54011990 con 0x7fef6009e1d0 2023-10-03T10:12:58.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.337+0000 7fef5dffb700 1 -- 172.21.15.3:0/1851405253 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef54011d20 con 0x7fef6009e1d0 2023-10-03T10:12:58.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.338+0000 7fef652d3700 1 -- 172.21.15.3:0/1851405253 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fef6009e1d0 msgr2=0x7fef60138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:58.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.338+0000 7fef652d3700 1 --2- 172.21.15.3:0/1851405253 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fef6009e1d0 0x7fef60138c30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fef54009920 tx=0x7fef54009c30 comp rx=0 tx=0).stop 2023-10-03T10:12:58.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.338+0000 7fef652d3700 1 -- 172.21.15.3:0/1851405253 shutdown_connections 2023-10-03T10:12:58.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.338+0000 7fef652d3700 1 --2- 172.21.15.3:0/1851405253 >> v2:172.21.15.3:3300/0 conn(0x7fef60139170 0x7fef6013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.338+0000 7fef652d3700 1 --2- 172.21.15.3:0/1851405253 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fef6009e1d0 0x7fef60138c30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.338+0000 7fef652d3700 1 -- 172.21.15.3:0/1851405253 >> 172.21.15.3:0/1851405253 conn(0x7fef60066f50 msgr2=0x7fef6009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:58.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.338+0000 7fef652d3700 1 -- 172.21.15.3:0/1851405253 shutdown_connections 2023-10-03T10:12:58.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.338+0000 7fef652d3700 1 -- 172.21.15.3:0/1851405253 wait complete. 2023-10-03T10:12:58.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.340+0000 7fef652d3700 1 Processor -- start 2023-10-03T10:12:58.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.340+0000 7fef652d3700 1 -- start start 2023-10-03T10:12:58.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.340+0000 7fef652d3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fef6009d6f0 0x7fef601c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:58.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.340+0000 7fef652d3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fef6009e1d0 0x7fef601c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:58.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.340+0000 7fef652d3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef60139170 0x7fef601c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:58.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.340+0000 7fef652d3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef60072750 con 0x7fef60139170 2023-10-03T10:12:58.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.340+0000 7fef652d3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef6009c710 con 0x7fef6009e1d0 2023-10-03T10:12:58.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.340+0000 7fef652d3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef601207b0 con 0x7fef6009d6f0 2023-10-03T10:12:58.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.340+0000 7fef5e7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fef6009e1d0 0x7fef601c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:58.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.340+0000 7fef5effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fef6009d6f0 0x7fef601c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:58.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.340+0000 7fef5e7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fef6009e1d0 0x7fef601c89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:43004/0 (socket says 172.21.15.3:43004) 2023-10-03T10:12:58.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.340+0000 7fef5effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fef6009d6f0 0x7fef601c8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54740/0 (socket says 172.21.15.3:54740) 2023-10-03T10:12:58.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.340+0000 7fef5e7fc700 1 -- 172.21.15.3:0/848220533 learned_addr learned my addr 172.21.15.3:0/848220533 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:58.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.340+0000 7fef5f7fe700 1 --2- 172.21.15.3:0/848220533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef60139170 0x7fef601c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:58.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.341+0000 7fef5f7fe700 1 -- 172.21.15.3:0/848220533 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fef6009d6f0 msgr2=0x7fef601c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:58.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.341+0000 7fef5f7fe700 1 --2- 172.21.15.3:0/848220533 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fef6009d6f0 0x7fef601c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.341+0000 7fef5f7fe700 1 -- 172.21.15.3:0/848220533 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fef6009e1d0 msgr2=0x7fef601c89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:58.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.341+0000 7fef5f7fe700 1 --2- 172.21.15.3:0/848220533 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fef6009e1d0 0x7fef601c89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.341+0000 7fef5f7fe700 1 -- 172.21.15.3:0/848220533 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef54009550 con 0x7fef60139170 2023-10-03T10:12:58.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.341+0000 7fef5effd700 1 --2- 172.21.15.3:0/848220533 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fef6009d6f0 0x7fef601c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:58.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.341+0000 7fef5e7fc700 1 --2- 172.21.15.3:0/848220533 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fef6009e1d0 0x7fef601c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:58.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.341+0000 7fef5f7fe700 1 --2- 172.21.15.3:0/848220533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef60139170 0x7fef601c8ee0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fef5800efc0 tx=0x7fef5800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:58.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.341+0000 7fef477fe700 1 -- 172.21.15.3:0/848220533 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5800cc90 con 0x7fef60139170 2023-10-03T10:12:58.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.341+0000 7fef652d3700 1 -- 172.21.15.3:0/848220533 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef60122010 con 0x7fef60139170 2023-10-03T10:12:58.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.341+0000 7fef652d3700 1 -- 172.21.15.3:0/848220533 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef60121310 con 0x7fef60139170 2023-10-03T10:12:58.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.341+0000 7fef477fe700 1 -- 172.21.15.3:0/848220533 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5800f040 con 0x7fef60139170 2023-10-03T10:12:58.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.342+0000 7fef477fe700 1 -- 172.21.15.3:0/848220533 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef58009e60 con 0x7fef60139170 2023-10-03T10:12:58.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.342+0000 7fef477fe700 1 -- 172.21.15.3:0/848220533 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fef58014480 con 0x7fef60139170 2023-10-03T10:12:58.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.343+0000 7fef5effd700 1 -- 172.21.15.3:0/848220533 >> v1:172.21.15.3:6816/127025 conn(0x7fef4c02f510 legacy=0x7fef4c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:58.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.343+0000 7fef477fe700 1 -- 172.21.15.3:0/848220533 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7fef58040880 con 0x7fef60139170 2023-10-03T10:12:58.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.343+0000 7fef652d3700 1 -- 172.21.15.3:0/848220533 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef400052e0 con 0x7fef60139170 2023-10-03T10:12:58.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.352+0000 7fef477fe700 1 -- 172.21.15.3:0/848220533 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fef58018020 con 0x7fef60139170 2023-10-03T10:12:58.360 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:12:58.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.438+0000 7f32f0a90700 1 Processor -- start 2023-10-03T10:12:58.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.438+0000 7f32f0a90700 1 -- start start 2023-10-03T10:12:58.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.439+0000 7f32f0a90700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f32ec130670 0x7f32ec130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:58.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.439+0000 7f32f0a90700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f32ec06f680 0x7f32ec06fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:58.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.439+0000 7f32f0a90700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f32ec06ffd0 0x7f32ec137530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:58.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.439+0000 7f32f0a90700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32ec09a350 con 0x7f32ec06f680 2023-10-03T10:12:58.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.439+0000 7f32f0a90700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec130260 con 0x7f32ec06ffd0 2023-10-03T10:12:58.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.439+0000 7f32f0a90700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec071ff0 con 0x7f32ec130670 2023-10-03T10:12:58.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.440+0000 7f32ea59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f32ec130670 0x7f32ec130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:58.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.440+0000 7f32ea59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f32ec130670 0x7f32ec130a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54754/0 (socket says 172.21.15.3:54754) 2023-10-03T10:12:58.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.440+0000 7f32ea59c700 1 -- 172.21.15.3:0/1684408811 learned_addr learned my addr 172.21.15.3:0/1684408811 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:58.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.440+0000 7f32e9d9b700 1 --2- 172.21.15.3:0/1684408811 >> v2:172.21.15.3:3300/0 conn(0x7f32ec06f680 0x7f32ec06fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:58.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.440+0000 7f32ead9d700 1 --2- 172.21.15.3:0/1684408811 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f32ec06ffd0 0x7f32ec137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:58.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.440+0000 7f32ea59c700 1 -- 172.21.15.3:0/1684408811 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f32ec06ffd0 msgr2=0x7f32ec137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:58.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.440+0000 7f32ea59c700 1 --2- 172.21.15.3:0/1684408811 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f32ec06ffd0 0x7f32ec137530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.440+0000 7f32ea59c700 1 -- 172.21.15.3:0/1684408811 >> v2:172.21.15.3:3300/0 conn(0x7f32ec06f680 msgr2=0x7f32ec06fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:58.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.440+0000 7f32ea59c700 1 --2- 172.21.15.3:0/1684408811 >> v2:172.21.15.3:3300/0 conn(0x7f32ec06f680 0x7f32ec06fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.440+0000 7f32ea59c700 1 -- 172.21.15.3:0/1684408811 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ec070470 con 0x7f32ec130670 2023-10-03T10:12:58.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.440+0000 7f32e9d9b700 1 --2- 172.21.15.3:0/1684408811 >> v2:172.21.15.3:3300/0 conn(0x7f32ec06f680 0x7f32ec06fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:58.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.441+0000 7f32ea59c700 1 --2- 172.21.15.3:0/1684408811 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f32ec130670 0x7f32ec130a40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f32dc009920 tx=0x7f32dc009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5604aead42f023b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:58.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.441+0000 7f32e959a700 1 -- 172.21.15.3:0/1684408811 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc011830 con 0x7f32ec130670 2023-10-03T10:12:58.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.441+0000 7f32e959a700 1 -- 172.21.15.3:0/1684408811 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32dc011990 con 0x7f32ec130670 2023-10-03T10:12:58.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.441+0000 7f32e959a700 1 -- 172.21.15.3:0/1684408811 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc011d20 con 0x7f32ec130670 2023-10-03T10:12:58.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.442+0000 7f32f0a90700 1 -- 172.21.15.3:0/1684408811 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f32ec130670 msgr2=0x7f32ec130a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:58.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.442+0000 7f32f0a90700 1 --2- 172.21.15.3:0/1684408811 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f32ec130670 0x7f32ec130a40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f32dc009920 tx=0x7f32dc009c30 comp rx=0 tx=0).stop 2023-10-03T10:12:58.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.442+0000 7f32f0a90700 1 -- 172.21.15.3:0/1684408811 shutdown_connections 2023-10-03T10:12:58.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.442+0000 7f32f0a90700 1 --2- 172.21.15.3:0/1684408811 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f32ec06ffd0 0x7f32ec137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.442+0000 7f32f0a90700 1 --2- 172.21.15.3:0/1684408811 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f32ec130670 0x7f32ec130a40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.442+0000 7f32f0a90700 1 --2- 172.21.15.3:0/1684408811 >> v2:172.21.15.3:3300/0 conn(0x7f32ec06f680 0x7f32ec06fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.442+0000 7f32f0a90700 1 -- 172.21.15.3:0/1684408811 >> 172.21.15.3:0/1684408811 conn(0x7f32ec0a2aa0 msgr2=0x7f32ec0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:58.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.442+0000 7f32f0a90700 1 -- 172.21.15.3:0/1684408811 shutdown_connections 2023-10-03T10:12:58.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.442+0000 7f32f0a90700 1 -- 172.21.15.3:0/1684408811 wait complete. 2023-10-03T10:12:58.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.443+0000 7f32f0a90700 1 Processor -- start 2023-10-03T10:12:58.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.444+0000 7f32f0a90700 1 -- start start 2023-10-03T10:12:58.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.444+0000 7f32f0a90700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f32ec06f680 0x7f32ec138560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:58.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.444+0000 7f32f0a90700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f32ec06ffd0 0x7f32ec138aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:58.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.444+0000 7f32f0a90700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32ec130670 0x7f32ec1c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:58.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.444+0000 7f32f0a90700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec135d20 con 0x7f32ec130670 2023-10-03T10:12:58.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.444+0000 7f32f0a90700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec1361c0 con 0x7f32ec06ffd0 2023-10-03T10:12:58.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.444+0000 7f32f0a90700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec071870 con 0x7f32ec06f680 2023-10-03T10:12:58.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.444+0000 7f32ea59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f32ec06f680 0x7f32ec138560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:58.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.444+0000 7f32ea59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f32ec06f680 0x7f32ec138560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54766/0 (socket says 172.21.15.3:54766) 2023-10-03T10:12:58.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.444+0000 7f32ea59c700 1 -- 172.21.15.3:0/3114807680 learned_addr learned my addr 172.21.15.3:0/3114807680 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:58.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.444+0000 7f32ead9d700 1 --2- 172.21.15.3:0/3114807680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32ec130670 0x7f32ec1c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:58.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.445+0000 7f32ea59c700 1 -- 172.21.15.3:0/3114807680 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f32ec06ffd0 msgr2=0x7f32ec138aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:58.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.445+0000 7f32ea59c700 1 --2- 172.21.15.3:0/3114807680 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f32ec06ffd0 0x7f32ec138aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.445+0000 7f32ea59c700 1 -- 172.21.15.3:0/3114807680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32ec130670 msgr2=0x7f32ec1c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:58.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.445+0000 7f32ea59c700 1 --2- 172.21.15.3:0/3114807680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32ec130670 0x7f32ec1c80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.445+0000 7f32ea59c700 1 -- 172.21.15.3:0/3114807680 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e00094a0 con 0x7f32ec06f680 2023-10-03T10:12:58.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.445+0000 7f32ea59c700 1 --2- 172.21.15.3:0/3114807680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f32ec06f680 0x7f32ec138560 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f32dc000c00 tx=0x7f32dc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:58.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.445+0000 7f32daffd700 1 -- 172.21.15.3:0/3114807680 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc004450 con 0x7f32ec06f680 2023-10-03T10:12:58.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.445+0000 7f32daffd700 1 -- 172.21.15.3:0/3114807680 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32dc0045b0 con 0x7f32ec06f680 2023-10-03T10:12:58.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.445+0000 7f32f0a90700 1 -- 172.21.15.3:0/3114807680 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32dc009550 con 0x7f32ec06f680 2023-10-03T10:12:58.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.445+0000 7f32daffd700 1 -- 172.21.15.3:0/3114807680 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0038e0 con 0x7f32ec06f680 2023-10-03T10:12:58.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.445+0000 7f32f0a90700 1 -- 172.21.15.3:0/3114807680 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32ec135450 con 0x7f32ec06f680 2023-10-03T10:12:58.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.447+0000 7f32daffd700 1 -- 172.21.15.3:0/3114807680 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f32dc003a40 con 0x7f32ec06f680 2023-10-03T10:12:58.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.447+0000 7f32f0a90700 1 -- 172.21.15.3:0/3114807680 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32ec135640 con 0x7f32ec06f680 2023-10-03T10:12:58.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.447+0000 7f32e9d9b700 1 -- 172.21.15.3:0/3114807680 >> v1:172.21.15.3:6816/127025 conn(0x7f32cc02f510 legacy=0x7f32cc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:58.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.447+0000 7f32daffd700 1 -- 172.21.15.3:0/3114807680 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f32dc040ba0 con 0x7f32ec06f680 2023-10-03T10:12:58.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.456+0000 7f32daffd700 1 -- 172.21.15.3:0/3114807680 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f32dc01a420 con 0x7f32ec06f680 2023-10-03T10:12:58.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.483+0000 7fb87fbb3700 1 Processor -- start 2023-10-03T10:12:58.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.483+0000 7fb87fbb3700 1 -- start start 2023-10-03T10:12:58.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.483+0000 7fb87fbb3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb87806a770 0x7fb87809d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:58.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.483+0000 7fb87fbb3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb87809dc30 0x7fb87809e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:58.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.483+0000 7fb87fbb3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb87813c1e0 0x7fb87813e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:58.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.484+0000 7fb87fbb3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb87806ab40 con 0x7fb87809dc30 2023-10-03T10:12:58.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.484+0000 7fb87fbb3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87809ebc0 con 0x7fb87813c1e0 2023-10-03T10:12:58.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.484+0000 7fb87fbb3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb87809a810 con 0x7fb87806a770 2023-10-03T10:12:58.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.484+0000 7fb87d14e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb87809dc30 0x7fb87809e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:58.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.484+0000 7fb87d14e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb87809dc30 0x7fb87809e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39164/0 (socket says 172.21.15.3:39164) 2023-10-03T10:12:58.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.484+0000 7fb87d14e700 1 -- 172.21.15.3:0/3022367591 learned_addr learned my addr 172.21.15.3:0/3022367591 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:58.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.484+0000 7fb87d94f700 1 --2- 172.21.15.3:0/3022367591 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb87806a770 0x7fb87809d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:58.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.484+0000 7fb87e150700 1 --2- 172.21.15.3:0/3022367591 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb87813c1e0 0x7fb87813e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:58.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.484+0000 7fb87d14e700 1 -- 172.21.15.3:0/3022367591 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb87806a770 msgr2=0x7fb87809d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:58.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.484+0000 7fb87d14e700 1 --2- 172.21.15.3:0/3022367591 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb87806a770 0x7fb87809d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.484+0000 7fb87d14e700 1 -- 172.21.15.3:0/3022367591 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb87813c1e0 msgr2=0x7fb87813e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:58.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.484+0000 7fb87d14e700 1 --2- 172.21.15.3:0/3022367591 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb87813c1e0 0x7fb87813e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.484+0000 7fb87d14e700 1 -- 172.21.15.3:0/3022367591 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87809e920 con 0x7fb87809dc30 2023-10-03T10:12:58.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.484+0000 7fb87d14e700 1 --2- 172.21.15.3:0/3022367591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb87809dc30 0x7fb87809e080 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb868009920 tx=0x7fb868009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf97cbdaaf4bf4d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:58.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.485+0000 7fb87c94d700 1 -- 172.21.15.3:0/3022367591 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868011830 con 0x7fb87809dc30 2023-10-03T10:12:58.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.485+0000 7fb87c94d700 1 -- 172.21.15.3:0/3022367591 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb868011990 con 0x7fb87809dc30 2023-10-03T10:12:58.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.485+0000 7fb87c94d700 1 -- 172.21.15.3:0/3022367591 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868011d20 con 0x7fb87809dc30 2023-10-03T10:12:58.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.485+0000 7fb87fbb3700 1 -- 172.21.15.3:0/3022367591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb87809dc30 msgr2=0x7fb87809e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:58.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.485+0000 7fb87fbb3700 1 --2- 172.21.15.3:0/3022367591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb87809dc30 0x7fb87809e080 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb868009920 tx=0x7fb868009c30 comp rx=0 tx=0).stop 2023-10-03T10:12:58.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.486+0000 7fb87fbb3700 1 -- 172.21.15.3:0/3022367591 shutdown_connections 2023-10-03T10:12:58.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.486+0000 7fb87fbb3700 1 --2- 172.21.15.3:0/3022367591 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb87813c1e0 0x7fb87813e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.486+0000 7fb87fbb3700 1 --2- 172.21.15.3:0/3022367591 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb87806a770 0x7fb87809d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.486+0000 7fb87fbb3700 1 --2- 172.21.15.3:0/3022367591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb87809dc30 0x7fb87809e080 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.486+0000 7fb87fbb3700 1 -- 172.21.15.3:0/3022367591 >> 172.21.15.3:0/3022367591 conn(0x7fb878066f50 msgr2=0x7fb87809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:58.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.486+0000 7fb87fbb3700 1 -- 172.21.15.3:0/3022367591 shutdown_connections 2023-10-03T10:12:58.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.486+0000 7fb87fbb3700 1 -- 172.21.15.3:0/3022367591 wait complete. 2023-10-03T10:12:58.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.487+0000 7fb87fbb3700 1 Processor -- start 2023-10-03T10:12:58.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.487+0000 7fb87fbb3700 1 -- start start 2023-10-03T10:12:58.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.487+0000 7fb87fbb3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb87806a770 0x7fb87811d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:58.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.487+0000 7fb87fbb3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb87809dc30 0x7fb87811ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:58.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.487+0000 7fb87fbb3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb87813c1e0 0x7fb87811e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:58.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.487+0000 7fb87fbb3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8780a38b0 con 0x7fb87813c1e0 2023-10-03T10:12:58.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.487+0000 7fb87fbb3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87811fd20 con 0x7fb87806a770 2023-10-03T10:12:58.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.487+0000 7fb87fbb3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb87809f1a0 con 0x7fb87809dc30 2023-10-03T10:12:58.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.487+0000 7fb87d14e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb87809dc30 0x7fb87811ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:58.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.488+0000 7fb87d14e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb87809dc30 0x7fb87811ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54776/0 (socket says 172.21.15.3:54776) 2023-10-03T10:12:58.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.488+0000 7fb87d14e700 1 -- 172.21.15.3:0/2266242879 learned_addr learned my addr 172.21.15.3:0/2266242879 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:58.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.488+0000 7fb87e150700 1 --2- 172.21.15.3:0/2266242879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb87813c1e0 0x7fb87811e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:58.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.488+0000 7fb87d94f700 1 --2- 172.21.15.3:0/2266242879 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb87806a770 0x7fb87811d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:58.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.488+0000 7fb87d14e700 1 -- 172.21.15.3:0/2266242879 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb87806a770 msgr2=0x7fb87811d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:58.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.488+0000 7fb87d14e700 1 --2- 172.21.15.3:0/2266242879 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb87806a770 0x7fb87811d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.488+0000 7fb87d14e700 1 -- 172.21.15.3:0/2266242879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb87813c1e0 msgr2=0x7fb87811e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:58.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.488+0000 7fb87d14e700 1 --2- 172.21.15.3:0/2266242879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb87813c1e0 0x7fb87811e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.488+0000 7fb87d14e700 1 -- 172.21.15.3:0/2266242879 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb868009550 con 0x7fb87809dc30 2023-10-03T10:12:58.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.488+0000 7fb87e150700 1 --2- 172.21.15.3:0/2266242879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb87813c1e0 0x7fb87811e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:58.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.488+0000 7fb87d14e700 1 --2- 172.21.15.3:0/2266242879 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb87809dc30 0x7fb87811ddd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb868009c70 tx=0x7fb868004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:58.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.488+0000 7fb86e7fc700 1 -- 172.21.15.3:0/2266242879 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868004300 con 0x7fb87809dc30 2023-10-03T10:12:58.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.489+0000 7fb86e7fc700 1 -- 172.21.15.3:0/2266242879 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86801f070 con 0x7fb87809dc30 2023-10-03T10:12:58.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.489+0000 7fb86e7fc700 1 -- 172.21.15.3:0/2266242879 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8680036a0 con 0x7fb87809dc30 2023-10-03T10:12:58.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.489+0000 7fb87fbb3700 1 -- 172.21.15.3:0/2266242879 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb878138e80 con 0x7fb87809dc30 2023-10-03T10:12:58.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.489+0000 7fb87fbb3700 1 -- 172.21.15.3:0/2266242879 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb87811fa60 con 0x7fb87809dc30 2023-10-03T10:12:58.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.490+0000 7fb86e7fc700 1 -- 172.21.15.3:0/2266242879 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fb868021030 con 0x7fb87809dc30 2023-10-03T10:12:58.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.491+0000 7fb86e7fc700 1 -- 172.21.15.3:0/2266242879 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7fb868040e10 con 0x7fb87809dc30 2023-10-03T10:12:58.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.490+0000 7fb87fbb3700 1 -- 172.21.15.3:0/2266242879 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb87813b980 con 0x7fb87809dc30 2023-10-03T10:12:58.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.491+0000 7fb87d94f700 1 -- 172.21.15.3:0/2266242879 >> v1:172.21.15.3:6816/127025 conn(0x7fb85c02f510 legacy=0x7fb85c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:58.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.496+0000 7fb86e7fc700 1 -- 172.21.15.3:0/2266242879 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb87813b980 con 0x7fb87809dc30 2023-10-03T10:12:58.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.515+0000 7f9e05320700 1 Processor -- start 2023-10-03T10:12:58.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.515+0000 7f9e05320700 1 -- start start 2023-10-03T10:12:58.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.516+0000 7f9e05320700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9e00133420 0x7f9e001337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:58.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.516+0000 7f9e05320700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e00134260 0x7f9e001306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:58.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.516+0000 7f9e05320700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e0006b990 con 0x7f9e00133e30 2023-10-03T10:12:58.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.516+0000 7f9e05320700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e000a0590 con 0x7f9e00133420 2023-10-03T10:12:58.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.516+0000 7f9e05320700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0009fb80 con 0x7f9e00134260 2023-10-03T10:12:58.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.516+0000 7f9dff7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e00134260 0x7f9e001306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:58.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.516+0000 7f9dfeffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9e00133420 0x7f9e001337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:58.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.516+0000 7f9dff7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e00134260 0x7f9e001306a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:43050/0 (socket says 172.21.15.3:43050) 2023-10-03T10:12:58.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.516+0000 7f9dff7fe700 1 -- 172.21.15.3:0/1689374758 learned_addr learned my addr 172.21.15.3:0/1689374758 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:58.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.516+0000 7f9dff7fe700 1 -- 172.21.15.3:0/1689374758 >> v2:172.21.15.3:3300/0 conn(0x7f9e00133420 msgr2=0x7f9e001337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:58.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.516+0000 7f9dff7fe700 1 --2- 172.21.15.3:0/1689374758 >> v2:172.21.15.3:3300/0 conn(0x7f9e00133420 0x7f9e001337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.516+0000 7f9dff7fe700 1 -- 172.21.15.3:0/1689374758 >> v1:172.21.15.3:6789/0 conn(0x7f9e00133e30 legacy=0x7f9e0012ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:58.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.516+0000 7f9dff7fe700 1 -- 172.21.15.3:0/1689374758 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e000a0030 con 0x7f9e00134260 2023-10-03T10:12:58.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.516+0000 7f9dfeffd700 1 --2- 172.21.15.3:0/1689374758 >> v2:172.21.15.3:3300/0 conn(0x7f9e00133420 0x7f9e001337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:58.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.517+0000 7f9dff7fe700 1 --2- 172.21.15.3:0/1689374758 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e00134260 0x7f9e001306a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9df8009e50 tx=0x7f9df800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a0ae6053adbbade server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:58.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.517+0000 7f9dfdffb700 1 -- 172.21.15.3:0/1689374758 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df800e070 con 0x7f9e00134260 2023-10-03T10:12:58.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.517+0000 7f9dfdffb700 1 -- 172.21.15.3:0/1689374758 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df800b910 con 0x7f9e00134260 2023-10-03T10:12:58.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.517+0000 7f9dfdffb700 1 -- 172.21.15.3:0/1689374758 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df800bcc0 con 0x7f9e00134260 2023-10-03T10:12:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.518+0000 7f9e05320700 1 -- 172.21.15.3:0/1689374758 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e00134260 msgr2=0x7f9e001306a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.518+0000 7f9e05320700 1 --2- 172.21.15.3:0/1689374758 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e00134260 0x7f9e001306a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9df8009e50 tx=0x7f9df800b330 comp rx=0 tx=0).stop 2023-10-03T10:12:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.518+0000 7f9e05320700 1 -- 172.21.15.3:0/1689374758 shutdown_connections 2023-10-03T10:12:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.518+0000 7f9e05320700 1 --2- 172.21.15.3:0/1689374758 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e00134260 0x7f9e001306a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.518+0000 7f9e05320700 1 --2- 172.21.15.3:0/1689374758 >> v2:172.21.15.3:3300/0 conn(0x7f9e00133420 0x7f9e001337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.518+0000 7f9e05320700 1 -- 172.21.15.3:0/1689374758 >> 172.21.15.3:0/1689374758 conn(0x7f9e000a4210 msgr2=0x7f9e000a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:58.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.518+0000 7f9e05320700 1 -- 172.21.15.3:0/1689374758 shutdown_connections 2023-10-03T10:12:58.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.518+0000 7f9e05320700 1 -- 172.21.15.3:0/1689374758 wait complete. 2023-10-03T10:12:58.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.520+0000 7f9e05320700 1 Processor -- start 2023-10-03T10:12:58.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.520+0000 7f9e05320700 1 -- start start 2023-10-03T10:12:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.521+0000 7f9e05320700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e00133420 0x7f9e0009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.521+0000 7f9e05320700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e00133e30 0x7f9e0009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.521+0000 7f9e05320700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e00134260 0x7f9e0009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.521+0000 7f9e05320700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0011fd30 con 0x7f9e00133420 2023-10-03T10:12:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.521+0000 7f9e05320700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0011fa90 con 0x7f9e00134260 2023-10-03T10:12:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.521+0000 7f9e05320700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0009e050 con 0x7f9e00133e30 2023-10-03T10:12:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.521+0000 7f9dff7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e00134260 0x7f9e0009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.521+0000 7f9dff7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e00134260 0x7f9e0009cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:43066/0 (socket says 172.21.15.3:43066) 2023-10-03T10:12:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.521+0000 7f9dff7fe700 1 -- 172.21.15.3:0/1703200120 learned_addr learned my addr 172.21.15.3:0/1703200120 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.521+0000 7f9dfeffd700 1 --2- 172.21.15.3:0/1703200120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e00133420 0x7f9e0009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:58.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.521+0000 7f9dff7fe700 1 -- 172.21.15.3:0/1703200120 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e00133e30 msgr2=0x7f9e0009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:58.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.521+0000 7f9dff7fe700 1 --2- 172.21.15.3:0/1703200120 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e00133e30 0x7f9e0009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.521+0000 7f9dff7fe700 1 -- 172.21.15.3:0/1703200120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e00133420 msgr2=0x7f9e0009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:58.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.521+0000 7f9dff7fe700 1 --2- 172.21.15.3:0/1703200120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e00133420 0x7f9e0009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.521+0000 7f9dff7fe700 1 -- 172.21.15.3:0/1703200120 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df00094a0 con 0x7f9e00134260 2023-10-03T10:12:58.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.522+0000 7f9dff7fe700 1 --2- 172.21.15.3:0/1703200120 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e00134260 0x7f9e0009cf00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9df8012000 tx=0x7f9df8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:58.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.522+0000 7f9df77fe700 1 -- 172.21.15.3:0/1703200120 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df80048b0 con 0x7f9e00134260 2023-10-03T10:12:58.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.522+0000 7f9df77fe700 1 -- 172.21.15.3:0/1703200120 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df80099b0 con 0x7f9e00134260 2023-10-03T10:12:58.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.522+0000 7f9df77fe700 1 -- 172.21.15.3:0/1703200120 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df801c680 con 0x7f9e00134260 2023-10-03T10:12:58.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.522+0000 7f9e05320700 1 -- 172.21.15.3:0/1703200120 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df8009b30 con 0x7f9e00134260 2023-10-03T10:12:58.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.522+0000 7f9e05320700 1 -- 172.21.15.3:0/1703200120 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0012db30 con 0x7f9e00134260 2023-10-03T10:12:58.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.523+0000 7f9df77fe700 1 -- 172.21.15.3:0/1703200120 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f9df801c7e0 con 0x7f9e00134260 2023-10-03T10:12:58.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.523+0000 7f9e05320700 1 -- 172.21.15.3:0/1703200120 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e00074ea0 con 0x7f9e00134260 2023-10-03T10:12:58.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.523+0000 7f9dfeffd700 1 -- 172.21.15.3:0/1703200120 >> v1:172.21.15.3:6816/127025 conn(0x7f9de002f510 legacy=0x7f9de00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:58.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.526+0000 7f9df77fe700 1 -- 172.21.15.3:0/1703200120 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f9df801bc30 con 0x7f9e00134260 2023-10-03T10:12:58.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.533+0000 7f9df77fe700 1 -- 172.21.15.3:0/1703200120 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9df80257d0 con 0x7f9e00134260 2023-10-03T10:12:58.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.544+0000 7fef5effd700 1 -- 172.21.15.3:0/848220533 >> v1:172.21.15.3:6816/127025 conn(0x7fef4c02f510 legacy=0x7fef4c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:58.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.578+0000 7fef652d3700 1 -- 172.21.15.3:0/848220533 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fef40005860 con 0x7fef60139170 2023-10-03T10:12:58.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.578+0000 7fef477fe700 1 -- 172.21.15.3:0/848220533 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fef5802b090 con 0x7fef60139170 2023-10-03T10:12:58.579 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:12:58.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.581+0000 7fef652d3700 1 -- 172.21.15.3:0/848220533 >> v1:172.21.15.3:6816/127025 conn(0x7fef4c02f510 legacy=0x7fef4c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:58.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.581+0000 7fef652d3700 1 -- 172.21.15.3:0/848220533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef60139170 msgr2=0x7fef601c8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:58.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.581+0000 7fef652d3700 1 --2- 172.21.15.3:0/848220533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef60139170 0x7fef601c8ee0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fef5800efc0 tx=0x7fef5800c3a0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.581+0000 7fef652d3700 1 -- 172.21.15.3:0/848220533 shutdown_connections 2023-10-03T10:12:58.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.581+0000 7fef652d3700 1 --2- 172.21.15.3:0/848220533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef60139170 0x7fef601c8ee0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.581+0000 7fef652d3700 1 --2- 172.21.15.3:0/848220533 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fef6009d6f0 0x7fef601c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.581+0000 7fef652d3700 1 --2- 172.21.15.3:0/848220533 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fef6009e1d0 0x7fef601c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.581+0000 7fef652d3700 1 -- 172.21.15.3:0/848220533 >> 172.21.15.3:0/848220533 conn(0x7fef60066f50 msgr2=0x7fef600a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:58.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.582+0000 7fef652d3700 1 -- 172.21.15.3:0/848220533 shutdown_connections 2023-10-03T10:12:58.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.582+0000 7fef652d3700 1 -- 172.21.15.3:0/848220533 wait complete. 2023-10-03T10:12:58.602 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:12:58.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.648+0000 7f32e9d9b700 1 -- 172.21.15.3:0/3114807680 >> v1:172.21.15.3:6816/127025 conn(0x7f32cc02f510 legacy=0x7f32cc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:58.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.685+0000 7f32f0a90700 1 -- 172.21.15.3:0/3114807680 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f32ec137180 con 0x7f32ec06f680 2023-10-03T10:12:58.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.685+0000 7f32daffd700 1 -- 172.21.15.3:0/3114807680 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f32dc015090 con 0x7f32ec06f680 2023-10-03T10:12:58.685 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:12:58.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.687+0000 7f32f0a90700 1 -- 172.21.15.3:0/3114807680 >> v1:172.21.15.3:6816/127025 conn(0x7f32cc02f510 legacy=0x7f32cc0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:58.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.687+0000 7f32f0a90700 1 -- 172.21.15.3:0/3114807680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f32ec06f680 msgr2=0x7f32ec138560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:58.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.687+0000 7f32f0a90700 1 --2- 172.21.15.3:0/3114807680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f32ec06f680 0x7f32ec138560 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f32dc000c00 tx=0x7f32dc004750 comp rx=0 tx=0).stop 2023-10-03T10:12:58.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.687+0000 7f32f0a90700 1 -- 172.21.15.3:0/3114807680 shutdown_connections 2023-10-03T10:12:58.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.687+0000 7f32f0a90700 1 --2- 172.21.15.3:0/3114807680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32ec130670 0x7f32ec1c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.687+0000 7f32f0a90700 1 --2- 172.21.15.3:0/3114807680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f32ec06f680 0x7f32ec138560 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.687+0000 7f32f0a90700 1 --2- 172.21.15.3:0/3114807680 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f32ec06ffd0 0x7f32ec138aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.687+0000 7f32f0a90700 1 -- 172.21.15.3:0/3114807680 >> 172.21.15.3:0/3114807680 conn(0x7f32ec0a2aa0 msgr2=0x7f32ec0a1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:58.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.688+0000 7f32f0a90700 1 -- 172.21.15.3:0/3114807680 shutdown_connections 2023-10-03T10:12:58.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.688+0000 7f32f0a90700 1 -- 172.21.15.3:0/3114807680 wait complete. 2023-10-03T10:12:58.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.692+0000 7fb87d94f700 1 -- 172.21.15.3:0/2266242879 >> v1:172.21.15.3:6816/127025 conn(0x7fb85c02f510 legacy=0x7fb85c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:58.702 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:12:58.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.720+0000 7fb87fbb3700 1 -- 172.21.15.3:0/2266242879 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb87811cc30 con 0x7fb87809dc30 2023-10-03T10:12:58.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.721+0000 7fb86e7fc700 1 -- 172.21.15.3:0/2266242879 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb868015070 con 0x7fb87809dc30 2023-10-03T10:12:58.721 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:12:58.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.723+0000 7fb87fbb3700 1 -- 172.21.15.3:0/2266242879 >> v1:172.21.15.3:6816/127025 conn(0x7fb85c02f510 legacy=0x7fb85c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:58.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.723+0000 7fb87fbb3700 1 -- 172.21.15.3:0/2266242879 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb87809dc30 msgr2=0x7fb87811ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:58.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.723+0000 7fb87fbb3700 1 --2- 172.21.15.3:0/2266242879 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb87809dc30 0x7fb87811ddd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb868009c70 tx=0x7fb868004750 comp rx=0 tx=0).stop 2023-10-03T10:12:58.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.723+0000 7fb87fbb3700 1 -- 172.21.15.3:0/2266242879 shutdown_connections 2023-10-03T10:12:58.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.723+0000 7fb87fbb3700 1 --2- 172.21.15.3:0/2266242879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb87813c1e0 0x7fb87811e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.723+0000 7fb87fbb3700 1 --2- 172.21.15.3:0/2266242879 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb87806a770 0x7fb87811d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.723+0000 7fb87fbb3700 1 --2- 172.21.15.3:0/2266242879 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb87809dc30 0x7fb87811ddd0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.723+0000 7fb87fbb3700 1 -- 172.21.15.3:0/2266242879 >> 172.21.15.3:0/2266242879 conn(0x7fb878066f50 msgr2=0x7fb8780a2120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:58.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.723+0000 7fb87fbb3700 1 -- 172.21.15.3:0/2266242879 shutdown_connections 2023-10-03T10:12:58.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.723+0000 7fb87fbb3700 1 -- 172.21.15.3:0/2266242879 wait complete. 2023-10-03T10:12:58.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.724+0000 7f9dfeffd700 1 -- 172.21.15.3:0/1703200120 >> v1:172.21.15.3:6816/127025 conn(0x7f9de002f510 legacy=0x7f9de00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:58.737 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:12:58.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.745+0000 7f9e05320700 1 -- 172.21.15.3:0/1703200120 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9e0006b250 con 0x7f9e00134260 2023-10-03T10:12:58.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.745+0000 7f9df77fe700 1 -- 172.21.15.3:0/1703200120 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9df802db40 con 0x7f9e00134260 2023-10-03T10:12:58.746 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:12:58.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.748+0000 7f9e05320700 1 -- 172.21.15.3:0/1703200120 >> v1:172.21.15.3:6816/127025 conn(0x7f9de002f510 legacy=0x7f9de00319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:58.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.748+0000 7f9e05320700 1 -- 172.21.15.3:0/1703200120 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e00134260 msgr2=0x7f9e0009cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:58.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.748+0000 7f9e05320700 1 --2- 172.21.15.3:0/1703200120 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e00134260 0x7f9e0009cf00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9df8012000 tx=0x7f9df8004ad0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.748+0000 7f9e05320700 1 -- 172.21.15.3:0/1703200120 shutdown_connections 2023-10-03T10:12:58.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.748+0000 7f9e05320700 1 --2- 172.21.15.3:0/1703200120 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e00134260 0x7f9e0009cf00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.748+0000 7f9e05320700 1 --2- 172.21.15.3:0/1703200120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e00133420 0x7f9e0009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.748+0000 7f9e05320700 1 --2- 172.21.15.3:0/1703200120 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e00133e30 0x7f9e0009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:58.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.748+0000 7f9e05320700 1 -- 172.21.15.3:0/1703200120 >> 172.21.15.3:0/1703200120 conn(0x7f9e000a4210 msgr2=0x7f9e000a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:58.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.748+0000 7f9e05320700 1 -- 172.21.15.3:0/1703200120 shutdown_connections 2023-10-03T10:12:58.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:58.748+0000 7f9e05320700 1 -- 172.21.15.3:0/1703200120 wait complete. 2023-10-03T10:12:58.762 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:12:59.602 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:12:59.703 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:12:59.738 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:12:59.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.749+0000 7f74f6b50700 1 Processor -- start 2023-10-03T10:12:59.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.749+0000 7f74f6b50700 1 -- start start 2023-10-03T10:12:59.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.750+0000 7f74f6b50700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74f01338a0 0x7f74f0133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:59.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.750+0000 7f74f6b50700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f74f009e930 0x7f74f013b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:59.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.750+0000 7f74f6b50700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74f006ef30 con 0x7f74f009de50 2023-10-03T10:12:59.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.750+0000 7f74f6b50700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74f006fef0 con 0x7f74f009e930 2023-10-03T10:12:59.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.750+0000 7f74f6b50700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74f01334b0 con 0x7f74f01338a0 2023-10-03T10:12:59.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.750+0000 7f74f50ed700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f74f009e930 0x7f74f013b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:59.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.750+0000 7f74f50ed700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f74f009e930 0x7f74f013b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39206/0 (socket says 172.21.15.3:39206) 2023-10-03T10:12:59.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.750+0000 7f74f50ed700 1 -- 172.21.15.3:0/1560944824 learned_addr learned my addr 172.21.15.3:0/1560944824 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:59.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.750+0000 7f74f48ec700 1 --2- 172.21.15.3:0/1560944824 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74f01338a0 0x7f74f0133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:59.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.750+0000 7f74f50ed700 1 -- 172.21.15.3:0/1560944824 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74f01338a0 msgr2=0x7f74f0133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:59.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.750+0000 7f74f50ed700 1 --2- 172.21.15.3:0/1560944824 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74f01338a0 0x7f74f0133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:59.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.750+0000 7f74f50ed700 1 -- 172.21.15.3:0/1560944824 >> v1:172.21.15.3:6789/0 conn(0x7f74f009de50 legacy=0x7f74f009e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:59.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.750+0000 7f74ef7fe700 1 -- 172.21.15.3:0/1560944824 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4074229511 0 0) 0x7f74f006ef30 con 0x7f74f009de50 2023-10-03T10:12:59.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.751+0000 7f74f50ed700 1 -- 172.21.15.3:0/1560944824 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f006fc30 con 0x7f74f009e930 2023-10-03T10:12:59.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.751+0000 7f74f48ec700 1 --2- 172.21.15.3:0/1560944824 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74f01338a0 0x7f74f0133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:12:59.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.751+0000 7f74f50ed700 1 --2- 172.21.15.3:0/1560944824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f009e930 0x7f74f013b740 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009e50 tx=0x7f74e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34faa9d6955e9525 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:59.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.751+0000 7f74ef7fe700 1 -- 172.21.15.3:0/1560944824 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e800e070 con 0x7f74f009e930 2023-10-03T10:12:59.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.751+0000 7f74ef7fe700 1 -- 172.21.15.3:0/1560944824 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e800b9c0 con 0x7f74f009e930 2023-10-03T10:12:59.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.751+0000 7f74ef7fe700 1 -- 172.21.15.3:0/1560944824 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e800bd70 con 0x7f74f009e930 2023-10-03T10:12:59.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.752+0000 7f74f6b50700 1 -- 172.21.15.3:0/1560944824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f009e930 msgr2=0x7f74f013b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:59.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.752+0000 7f74f6b50700 1 --2- 172.21.15.3:0/1560944824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f009e930 0x7f74f013b740 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009e50 tx=0x7f74e800b330 comp rx=0 tx=0).stop 2023-10-03T10:12:59.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.752+0000 7f74f6b50700 1 -- 172.21.15.3:0/1560944824 shutdown_connections 2023-10-03T10:12:59.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.752+0000 7f74f6b50700 1 --2- 172.21.15.3:0/1560944824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f009e930 0x7f74f013b740 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:59.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.752+0000 7f74f6b50700 1 --2- 172.21.15.3:0/1560944824 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74f01338a0 0x7f74f0133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:59.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.752+0000 7f74f6b50700 1 -- 172.21.15.3:0/1560944824 >> 172.21.15.3:0/1560944824 conn(0x7f74f00a2aa0 msgr2=0x7f74f00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:59.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.752+0000 7f74f6b50700 1 -- 172.21.15.3:0/1560944824 shutdown_connections 2023-10-03T10:12:59.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.752+0000 7f74f6b50700 1 -- 172.21.15.3:0/1560944824 wait complete. 2023-10-03T10:12:59.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.753+0000 7f74f6b50700 1 Processor -- start 2023-10-03T10:12:59.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.753+0000 7f74f6b50700 1 -- start start 2023-10-03T10:12:59.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.754+0000 7f74f6b50700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74f009de50 0x7f74f011a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:59.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.754+0000 7f74f6b50700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f009e930 0x7f74f011aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:59.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.754+0000 7f74f6b50700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74f01338a0 0x7f74f011afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:59.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.754+0000 7f74f6b50700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f011d860 con 0x7f74f009e930 2023-10-03T10:12:59.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.754+0000 7f74f6b50700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f011d330 con 0x7f74f01338a0 2023-10-03T10:12:59.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.754+0000 7f74f6b50700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74f011c180 con 0x7f74f009de50 2023-10-03T10:12:59.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.754+0000 7f74effff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f009e930 0x7f74f011aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:59.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.754+0000 7f74effff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f009e930 0x7f74f011aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39208/0 (socket says 172.21.15.3:39208) 2023-10-03T10:12:59.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.754+0000 7f74effff700 1 -- 172.21.15.3:0/1813163928 learned_addr learned my addr 172.21.15.3:0/1813163928 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:59.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.754+0000 7f74f48ec700 1 --2- 172.21.15.3:0/1813163928 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74f009de50 0x7f74f011a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:59.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.754+0000 7f74f50ed700 1 --2- 172.21.15.3:0/1813163928 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74f01338a0 0x7f74f011afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:59.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.754+0000 7f74f50ed700 1 -- 172.21.15.3:0/1813163928 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74f009de50 msgr2=0x7f74f011a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:59.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.754+0000 7f74f50ed700 1 --2- 172.21.15.3:0/1813163928 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74f009de50 0x7f74f011a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:59.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.755+0000 7f74f50ed700 1 -- 172.21.15.3:0/1813163928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f009e930 msgr2=0x7f74f011aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:59.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.755+0000 7f74f50ed700 1 --2- 172.21.15.3:0/1813163928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f009e930 0x7f74f011aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:59.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.755+0000 7f74f50ed700 1 -- 172.21.15.3:0/1813163928 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e8009b30 con 0x7f74f01338a0 2023-10-03T10:12:59.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.755+0000 7f74f48ec700 1 --2- 172.21.15.3:0/1813163928 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74f009de50 0x7f74f011a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:59.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.755+0000 7f74effff700 1 --2- 172.21.15.3:0/1813163928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f009e930 0x7f74f011aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:59.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.755+0000 7f74f50ed700 1 --2- 172.21.15.3:0/1813163928 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74f01338a0 0x7f74f011afa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f74f006dc90 tx=0x7f74e8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:59.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.755+0000 7f74ed7fa700 1 -- 172.21.15.3:0/1813163928 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e8003ea0 con 0x7f74f01338a0 2023-10-03T10:12:59.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.755+0000 7f74ed7fa700 1 -- 172.21.15.3:0/1813163928 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e80048a0 con 0x7f74f01338a0 2023-10-03T10:12:59.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.755+0000 7f74f6b50700 1 -- 172.21.15.3:0/1813163928 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f011dd10 con 0x7f74f01338a0 2023-10-03T10:12:59.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.755+0000 7f74ed7fa700 1 -- 172.21.15.3:0/1813163928 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e801c680 con 0x7f74f01338a0 2023-10-03T10:12:59.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.755+0000 7f74f6b50700 1 -- 172.21.15.3:0/1813163928 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f006f640 con 0x7f74f01338a0 2023-10-03T10:12:59.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.756+0000 7f74ed7fa700 1 -- 172.21.15.3:0/1813163928 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f74e801c7e0 con 0x7f74f01338a0 2023-10-03T10:12:59.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.756+0000 7f74f6b50700 1 -- 172.21.15.3:0/1813163928 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f0074ea0 con 0x7f74f01338a0 2023-10-03T10:12:59.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.756+0000 7f74f48ec700 1 -- 172.21.15.3:0/1813163928 >> v1:172.21.15.3:6816/127025 conn(0x7f74d402f510 legacy=0x7f74d40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:59.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.756+0000 7f74ed7fa700 1 -- 172.21.15.3:0/1813163928 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f74e80414a0 con 0x7f74f01338a0 2023-10-03T10:12:59.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.762+0000 7f74ed7fa700 1 -- 172.21.15.3:0/1813163928 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f74e80249a0 con 0x7f74f01338a0 2023-10-03T10:12:59.762 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:12:59.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.858+0000 7ff78b2dd700 1 Processor -- start 2023-10-03T10:12:59.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.858+0000 7ff78b2dd700 1 -- start start 2023-10-03T10:12:59.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.859+0000 7ff78b2dd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff78409e930 0x7ff78409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:59.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.859+0000 7ff78b2dd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78409f240 0x7ff7841317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:59.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.859+0000 7ff78b2dd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78409f650 0x7ff784131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:59.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.859+0000 7ff78b2dd700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff784070940 con 0x7ff78409e930 2023-10-03T10:12:59.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.859+0000 7ff78b2dd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78406fd60 con 0x7ff78409f240 2023-10-03T10:12:59.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.859+0000 7ff78b2dd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff78406fa80 con 0x7ff78409f650 2023-10-03T10:12:59.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.859+0000 7ff789079700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff78409e930 0x7ff78409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:59.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.859+0000 7ff788878700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78409f240 0x7ff7841317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:59.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.859+0000 7ff788878700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78409f240 0x7ff7841317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:43084/0 (socket says 172.21.15.3:43084) 2023-10-03T10:12:59.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.859+0000 7ff788878700 1 -- 172.21.15.3:0/809060711 learned_addr learned my addr 172.21.15.3:0/809060711 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:59.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.859+0000 7ff789079700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff78409e930 0x7ff78409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39210/0 (socket says 172.21.15.3:39210) 2023-10-03T10:12:59.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.859+0000 7ff78987a700 1 --2- 172.21.15.3:0/809060711 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78409f650 0x7ff784131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:59.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.859+0000 7ff788878700 1 -- 172.21.15.3:0/809060711 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78409f650 msgr2=0x7ff784131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:59.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.859+0000 7ff788878700 1 --2- 172.21.15.3:0/809060711 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78409f650 0x7ff784131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:59.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.859+0000 7ff788878700 1 -- 172.21.15.3:0/809060711 >> v2:172.21.15.3:3300/0 conn(0x7ff78409e930 msgr2=0x7ff78409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:59.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.859+0000 7ff788878700 1 --2- 172.21.15.3:0/809060711 >> v2:172.21.15.3:3300/0 conn(0x7ff78409e930 0x7ff78409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:59.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.860+0000 7ff788878700 1 -- 172.21.15.3:0/809060711 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78409fb60 con 0x7ff78409f240 2023-10-03T10:12:59.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.860+0000 7ff789079700 1 --2- 172.21.15.3:0/809060711 >> v2:172.21.15.3:3300/0 conn(0x7ff78409e930 0x7ff78409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:59.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.860+0000 7ff788878700 1 --2- 172.21.15.3:0/809060711 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78409f240 0x7ff7841317f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff7740098f0 tx=0x7ff774009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a72db8f138dd8156 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:59.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.860+0000 7ff77bfff700 1 -- 172.21.15.3:0/809060711 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff774010a90 con 0x7ff78409f240 2023-10-03T10:12:59.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.860+0000 7ff77bfff700 1 -- 172.21.15.3:0/809060711 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff774010bf0 con 0x7ff78409f240 2023-10-03T10:12:59.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.860+0000 7ff77bfff700 1 -- 172.21.15.3:0/809060711 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff774018440 con 0x7ff78409f240 2023-10-03T10:12:59.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.861+0000 7ff78b2dd700 1 -- 172.21.15.3:0/809060711 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78409f240 msgr2=0x7ff7841317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:59.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.861+0000 7ff78b2dd700 1 --2- 172.21.15.3:0/809060711 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78409f240 0x7ff7841317f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff7740098f0 tx=0x7ff774009c00 comp rx=0 tx=0).stop 2023-10-03T10:12:59.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.861+0000 7ff78b2dd700 1 -- 172.21.15.3:0/809060711 shutdown_connections 2023-10-03T10:12:59.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.861+0000 7ff78b2dd700 1 --2- 172.21.15.3:0/809060711 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78409f650 0x7ff784131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:59.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.861+0000 7ff78b2dd700 1 --2- 172.21.15.3:0/809060711 >> v2:172.21.15.3:3300/0 conn(0x7ff78409e930 0x7ff78409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:59.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.861+0000 7ff78b2dd700 1 --2- 172.21.15.3:0/809060711 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78409f240 0x7ff7841317f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:59.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.861+0000 7ff78b2dd700 1 -- 172.21.15.3:0/809060711 >> 172.21.15.3:0/809060711 conn(0x7ff7840a4210 msgr2=0x7ff7840a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:59.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.861+0000 7ff78b2dd700 1 -- 172.21.15.3:0/809060711 shutdown_connections 2023-10-03T10:12:59.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.861+0000 7ff78b2dd700 1 -- 172.21.15.3:0/809060711 wait complete. 2023-10-03T10:12:59.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.863+0000 7ff78b2dd700 1 Processor -- start 2023-10-03T10:12:59.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.863+0000 7ff78b2dd700 1 -- start start 2023-10-03T10:12:59.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.864+0000 7ff78b2dd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78409e930 0x7ff78411d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:59.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.864+0000 7ff78b2dd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78409f240 0x7ff78411d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:59.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.864+0000 7ff78b2dd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78409f650 0x7ff78411deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:59.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.864+0000 7ff78b2dd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78411f4c0 con 0x7ff78409e930 2023-10-03T10:12:59.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.864+0000 7ff78b2dd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7840a2f40 con 0x7ff78409f650 2023-10-03T10:12:59.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.864+0000 7ff78b2dd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff784071d20 con 0x7ff78409f240 2023-10-03T10:12:59.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.864+0000 7ff789079700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78409e930 0x7ff78411d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:59.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.864+0000 7ff789079700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78409e930 0x7ff78411d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39216/0 (socket says 172.21.15.3:39216) 2023-10-03T10:12:59.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.864+0000 7ff789079700 1 -- 172.21.15.3:0/976873873 learned_addr learned my addr 172.21.15.3:0/976873873 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:59.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.864+0000 7ff78987a700 1 --2- 172.21.15.3:0/976873873 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78409f650 0x7ff78411deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:59.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.864+0000 7ff788878700 1 --2- 172.21.15.3:0/976873873 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78409f240 0x7ff78411d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:59.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.864+0000 7ff78987a700 1 -- 172.21.15.3:0/976873873 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78409f240 msgr2=0x7ff78411d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:59.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.864+0000 7ff78987a700 1 --2- 172.21.15.3:0/976873873 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78409f240 0x7ff78411d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:59.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.864+0000 7ff78987a700 1 -- 172.21.15.3:0/976873873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78409e930 msgr2=0x7ff78411d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:59.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.864+0000 7ff78987a700 1 --2- 172.21.15.3:0/976873873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78409e930 0x7ff78411d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:59.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.865+0000 7ff78987a700 1 -- 172.21.15.3:0/976873873 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7800094a0 con 0x7ff78409f650 2023-10-03T10:12:59.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.865+0000 7ff788878700 1 --2- 172.21.15.3:0/976873873 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78409f240 0x7ff78411d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:59.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.865+0000 7ff789079700 1 --2- 172.21.15.3:0/976873873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78409e930 0x7ff78411d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:12:59.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.865+0000 7ff78987a700 1 --2- 172.21.15.3:0/976873873 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78409f650 0x7ff78411deb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00efc0 tx=0x7ff77c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:59.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.865+0000 7ff779ffb700 1 -- 172.21.15.3:0/976873873 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c00cc90 con 0x7ff78409f650 2023-10-03T10:12:59.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.865+0000 7ff779ffb700 1 -- 172.21.15.3:0/976873873 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77c00f040 con 0x7ff78409f650 2023-10-03T10:12:59.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.865+0000 7ff779ffb700 1 -- 172.21.15.3:0/976873873 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c0146c0 con 0x7ff78409f650 2023-10-03T10:12:59.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.865+0000 7ff78b2dd700 1 -- 172.21.15.3:0/976873873 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff774009550 con 0x7ff78409f650 2023-10-03T10:12:59.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.865+0000 7ff78b2dd700 1 -- 172.21.15.3:0/976873873 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7840a0190 con 0x7ff78409f650 2023-10-03T10:12:59.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.867+0000 7ff779ffb700 1 -- 172.21.15.3:0/976873873 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7ff77c014820 con 0x7ff78409f650 2023-10-03T10:12:59.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.867+0000 7ff78b2dd700 1 -- 172.21.15.3:0/976873873 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff784120790 con 0x7ff78409f650 2023-10-03T10:12:59.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.867+0000 7ff789079700 1 -- 172.21.15.3:0/976873873 >> v1:172.21.15.3:6816/127025 conn(0x7ff76802f510 legacy=0x7ff7680319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:59.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.867+0000 7ff779ffb700 1 -- 172.21.15.3:0/976873873 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7ff77c040810 con 0x7ff78409f650 2023-10-03T10:12:59.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.875+0000 7ff779ffb700 1 -- 172.21.15.3:0/976873873 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff77c009e60 con 0x7ff78409f650 2023-10-03T10:12:59.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.895+0000 7ff653e58700 1 Processor -- start 2023-10-03T10:12:59.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.896+0000 7ff653e58700 1 -- start start 2023-10-03T10:12:59.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.896+0000 7ff653e58700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff64c0a0a30 0x7ff64c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:59.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.896+0000 7ff653e58700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff64c134170 0x7ff64c13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:59.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.896+0000 7ff653e58700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff64c09edd0 con 0x7ff64c0a1340 2023-10-03T10:12:59.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.896+0000 7ff653e58700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c09e1f0 con 0x7ff64c134170 2023-10-03T10:12:59.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.896+0000 7ff653e58700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c09dc50 con 0x7ff64c0a0a30 2023-10-03T10:12:59.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.896+0000 7ff6513f3700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff64c0a1340 0x7ff64c133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45414/0 (socket says 172.21.15.3:45414) 2023-10-03T10:12:59.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.897+0000 7ff6513f3700 1 -- 172.21.15.3:0/3170645709 learned_addr learned my addr 172.21.15.3:0/3170645709 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:59.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.897+0000 7ff651bf4700 1 --2- 172.21.15.3:0/3170645709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff64c0a0a30 0x7ff64c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:59.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.897+0000 7ff6523f5700 1 --2- 172.21.15.3:0/3170645709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff64c134170 0x7ff64c13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:59.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.897+0000 7ff650bf2700 1 -- 172.21.15.3:0/3170645709 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336567339 0 0) 0x7ff64c09edd0 con 0x7ff64c0a1340 2023-10-03T10:12:59.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.897+0000 7ff650bf2700 1 -- 172.21.15.3:0/3170645709 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6380035b0 con 0x7ff64c0a1340 2023-10-03T10:12:59.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.897+0000 7ff6523f5700 1 -- 172.21.15.3:0/3170645709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff64c0a0a30 msgr2=0x7ff64c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:59.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.897+0000 7ff6523f5700 1 --2- 172.21.15.3:0/3170645709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff64c0a0a30 0x7ff64c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:59.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.897+0000 7ff6523f5700 1 -- 172.21.15.3:0/3170645709 >> v1:172.21.15.3:6789/0 conn(0x7ff64c0a1340 legacy=0x7ff64c133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:59.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.897+0000 7ff6523f5700 1 -- 172.21.15.3:0/3170645709 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64c0a1fe0 con 0x7ff64c134170 2023-10-03T10:12:59.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.897+0000 7ff6523f5700 1 --2- 172.21.15.3:0/3170645709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff64c134170 0x7ff64c13bd30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff644009dc0 tx=0x7ff64400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcbb59aa2d0d9db1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:59.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.898+0000 7ff650bf2700 1 -- 172.21.15.3:0/3170645709 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64400e070 con 0x7ff64c134170 2023-10-03T10:12:59.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.898+0000 7ff650bf2700 1 -- 172.21.15.3:0/3170645709 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff64400bb20 con 0x7ff64c134170 2023-10-03T10:12:59.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.898+0000 7ff650bf2700 1 -- 172.21.15.3:0/3170645709 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64400be90 con 0x7ff64c134170 2023-10-03T10:12:59.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.898+0000 7ff653e58700 1 -- 172.21.15.3:0/3170645709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff64c134170 msgr2=0x7ff64c13bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:59.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.898+0000 7ff653e58700 1 --2- 172.21.15.3:0/3170645709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff64c134170 0x7ff64c13bd30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff644009dc0 tx=0x7ff64400b330 comp rx=0 tx=0).stop 2023-10-03T10:12:59.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.898+0000 7ff653e58700 1 -- 172.21.15.3:0/3170645709 shutdown_connections 2023-10-03T10:12:59.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.898+0000 7ff653e58700 1 --2- 172.21.15.3:0/3170645709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff64c134170 0x7ff64c13bd30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:59.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.898+0000 7ff653e58700 1 --2- 172.21.15.3:0/3170645709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff64c0a0a30 0x7ff64c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:59.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.898+0000 7ff653e58700 1 -- 172.21.15.3:0/3170645709 >> 172.21.15.3:0/3170645709 conn(0x7ff64c0a6550 msgr2=0x7ff64c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:59.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.899+0000 7ff653e58700 1 -- 172.21.15.3:0/3170645709 shutdown_connections 2023-10-03T10:12:59.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.899+0000 7ff653e58700 1 -- 172.21.15.3:0/3170645709 wait complete. 2023-10-03T10:12:59.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.899+0000 7ff653e58700 1 Processor -- start 2023-10-03T10:12:59.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.900+0000 7ff653e58700 1 -- start start 2023-10-03T10:12:59.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.900+0000 7ff653e58700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff64c0a0a30 0x7ff64c122780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:59.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.900+0000 7ff653e58700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff64c0a1340 0x7ff64c122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:59.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.900+0000 7ff653e58700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff64c134170 0x7ff64c123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:59.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.900+0000 7ff653e58700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c126200 con 0x7ff64c134170 2023-10-03T10:12:59.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.900+0000 7ff653e58700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c1242f0 con 0x7ff64c0a0a30 2023-10-03T10:12:59.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.900+0000 7ff653e58700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c0a4d50 con 0x7ff64c0a1340 2023-10-03T10:12:59.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.900+0000 7ff651bf4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff64c0a0a30 0x7ff64c122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:59.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.900+0000 7ff651bf4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff64c0a0a30 0x7ff64c122780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:43098/0 (socket says 172.21.15.3:43098) 2023-10-03T10:12:59.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.900+0000 7ff651bf4700 1 -- 172.21.15.3:0/722067750 learned_addr learned my addr 172.21.15.3:0/722067750 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:59.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.900+0000 7ff6523f5700 1 --2- 172.21.15.3:0/722067750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff64c134170 0x7ff64c123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:59.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.900+0000 7ff6513f3700 1 --2- 172.21.15.3:0/722067750 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff64c0a1340 0x7ff64c122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:59.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.901+0000 7ff6523f5700 1 -- 172.21.15.3:0/722067750 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff64c0a1340 msgr2=0x7ff64c122cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:59.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.901+0000 7ff6523f5700 1 --2- 172.21.15.3:0/722067750 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff64c0a1340 0x7ff64c122cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:59.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.901+0000 7ff6523f5700 1 -- 172.21.15.3:0/722067750 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff64c0a0a30 msgr2=0x7ff64c122780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:59.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.901+0000 7ff6523f5700 1 --2- 172.21.15.3:0/722067750 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff64c0a0a30 0x7ff64c122780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:59.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.901+0000 7ff6523f5700 1 -- 172.21.15.3:0/722067750 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff644009a70 con 0x7ff64c134170 2023-10-03T10:12:59.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.901+0000 7ff6513f3700 1 --2- 172.21.15.3:0/722067750 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff64c0a1340 0x7ff64c122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:12:59.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.901+0000 7ff6523f5700 1 --2- 172.21.15.3:0/722067750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff64c134170 0x7ff64c123200 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff644016000 tx=0x7ff644004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:59.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.901+0000 7ff6427fc700 1 -- 172.21.15.3:0/722067750 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff644004930 con 0x7ff64c134170 2023-10-03T10:12:59.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.901+0000 7ff6427fc700 1 -- 172.21.15.3:0/722067750 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff644020070 con 0x7ff64c134170 2023-10-03T10:12:59.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.901+0000 7ff6427fc700 1 -- 172.21.15.3:0/722067750 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6440276a0 con 0x7ff64c134170 2023-10-03T10:12:59.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.901+0000 7ff653e58700 1 -- 172.21.15.3:0/722067750 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff64c0a25e0 con 0x7ff64c134170 2023-10-03T10:12:59.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.902+0000 7ff653e58700 1 -- 172.21.15.3:0/722067750 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff64c133200 con 0x7ff64c134170 2023-10-03T10:12:59.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.903+0000 7ff653e58700 1 -- 172.21.15.3:0/722067750 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff64c125bc0 con 0x7ff64c134170 2023-10-03T10:12:59.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.903+0000 7ff6427fc700 1 -- 172.21.15.3:0/722067750 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7ff644031430 con 0x7ff64c134170 2023-10-03T10:12:59.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.904+0000 7ff6427fc700 1 -- 172.21.15.3:0/722067750 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7ff644022070 con 0x7ff64c134170 2023-10-03T10:12:59.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.905+0000 7ff651bf4700 1 -- 172.21.15.3:0/722067750 >> v1:172.21.15.3:6816/127025 conn(0x7ff63002f510 legacy=0x7ff6300319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:59.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.912+0000 7ff6427fc700 1 -- 172.21.15.3:0/722067750 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff6440316e0 con 0x7ff64c134170 2023-10-03T10:12:59.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.924+0000 7f9dc4e51700 1 Processor -- start 2023-10-03T10:12:59.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.924+0000 7f9dc4e51700 1 -- start start 2023-10-03T10:12:59.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.925+0000 7f9dc4e51700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9dc009e930 0x7f9dc009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:59.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.925+0000 7f9dc4e51700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9dc009f650 0x7f9dc0131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:59.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.925+0000 7f9dc4e51700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc0070940 con 0x7f9dc009f240 2023-10-03T10:12:59.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.925+0000 7f9dc4e51700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc006fd60 con 0x7f9dc009f650 2023-10-03T10:12:59.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.925+0000 7f9dc4e51700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc006fa80 con 0x7f9dc009e930 2023-10-03T10:12:59.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.925+0000 7f9dbdd9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9dc009f240 0x7f9dc01317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45426/0 (socket says 172.21.15.3:45426) 2023-10-03T10:12:59.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.925+0000 7f9dbdd9b700 1 -- 172.21.15.3:0/1587771408 learned_addr learned my addr 172.21.15.3:0/1587771408 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:12:59.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.925+0000 7f9dbe59c700 1 --2- 172.21.15.3:0/1587771408 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9dc009e930 0x7f9dc009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:59.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.925+0000 7f9dbd59a700 1 -- 172.21.15.3:0/1587771408 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 74907885 0 0) 0x7f9dc0070940 con 0x7f9dc009f240 2023-10-03T10:12:59.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.925+0000 7f9dbd59a700 1 -- 172.21.15.3:0/1587771408 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9da80035b0 con 0x7f9dc009f240 2023-10-03T10:12:59.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.925+0000 7f9dbe59c700 1 -- 172.21.15.3:0/1587771408 >> v2:172.21.15.3:3300/0 conn(0x7f9dc009f650 msgr2=0x7f9dc0131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:12:59.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.925+0000 7f9dbe59c700 1 --2- 172.21.15.3:0/1587771408 >> v2:172.21.15.3:3300/0 conn(0x7f9dc009f650 0x7f9dc0131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:59.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.925+0000 7f9dbe59c700 1 -- 172.21.15.3:0/1587771408 >> v1:172.21.15.3:6789/0 conn(0x7f9dc009f240 legacy=0x7f9dc01317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:59.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.925+0000 7f9dbe59c700 1 -- 172.21.15.3:0/1587771408 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc009fb60 con 0x7f9dc009e930 2023-10-03T10:12:59.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.926+0000 7f9dbe59c700 1 --2- 172.21.15.3:0/1587771408 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9dc009e930 0x7f9dc009ed00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009840 tx=0x7f9db0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc055c8297e79b84 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:59.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.926+0000 7f9dbd59a700 1 -- 172.21.15.3:0/1587771408 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db0011830 con 0x7f9dc009e930 2023-10-03T10:12:59.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.926+0000 7f9dbd59a700 1 -- 172.21.15.3:0/1587771408 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db0011990 con 0x7f9dc009e930 2023-10-03T10:12:59.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.926+0000 7f9dbd59a700 1 -- 172.21.15.3:0/1587771408 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db0011d20 con 0x7f9dc009e930 2023-10-03T10:12:59.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.927+0000 7f9dc4e51700 1 -- 172.21.15.3:0/1587771408 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9dc009e930 msgr2=0x7f9dc009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:59.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.927+0000 7f9dc4e51700 1 --2- 172.21.15.3:0/1587771408 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9dc009e930 0x7f9dc009ed00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009840 tx=0x7f9db0009b50 comp rx=0 tx=0).stop 2023-10-03T10:12:59.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.927+0000 7f9dc4e51700 1 -- 172.21.15.3:0/1587771408 shutdown_connections 2023-10-03T10:12:59.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.927+0000 7f9dc4e51700 1 --2- 172.21.15.3:0/1587771408 >> v2:172.21.15.3:3300/0 conn(0x7f9dc009f650 0x7f9dc0131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:59.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.927+0000 7f9dc4e51700 1 --2- 172.21.15.3:0/1587771408 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9dc009e930 0x7f9dc009ed00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:59.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.927+0000 7f9dc4e51700 1 -- 172.21.15.3:0/1587771408 >> 172.21.15.3:0/1587771408 conn(0x7f9dc00a4210 msgr2=0x7f9dc00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:59.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.927+0000 7f9dc4e51700 1 -- 172.21.15.3:0/1587771408 shutdown_connections 2023-10-03T10:12:59.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.927+0000 7f9dc4e51700 1 -- 172.21.15.3:0/1587771408 wait complete. 2023-10-03T10:12:59.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.928+0000 7f9dc4e51700 1 Processor -- start 2023-10-03T10:12:59.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.928+0000 7f9dc4e51700 1 -- start start 2023-10-03T10:12:59.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.929+0000 7f9dc4e51700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9dc009e930 0x7f9dc006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:59.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.929+0000 7f9dc4e51700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dc009f240 0x7f9dc006bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:59.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.929+0000 7f9dc4e51700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9dc009f650 0x7f9dc006c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:12:59.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.929+0000 7f9dc4e51700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc01a98e0 con 0x7f9dc009f240 2023-10-03T10:12:59.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.929+0000 7f9dc4e51700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc006cd10 con 0x7f9dc009f650 2023-10-03T10:12:59.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.929+0000 7f9dc4e51700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc0071d20 con 0x7f9dc009e930 2023-10-03T10:12:59.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.929+0000 7f9dbed9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9dc009f650 0x7f9dc006c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:59.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.929+0000 7f9dbed9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9dc009f650 0x7f9dc006c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:43112/0 (socket says 172.21.15.3:43112) 2023-10-03T10:12:59.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.929+0000 7f9dbdd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dc009f240 0x7f9dc006bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:12:59.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.929+0000 7f9dbed9d700 1 -- 172.21.15.3:0/3774950567 learned_addr learned my addr 172.21.15.3:0/3774950567 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:12:59.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.929+0000 7f9dbed9d700 1 -- 172.21.15.3:0/3774950567 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9dc009e930 msgr2=0x7f9dc006b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:59.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.929+0000 7f9dbed9d700 1 --2- 172.21.15.3:0/3774950567 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9dc009e930 0x7f9dc006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:59.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.929+0000 7f9dbed9d700 1 -- 172.21.15.3:0/3774950567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dc009f240 msgr2=0x7f9dc006bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:59.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.929+0000 7f9dbed9d700 1 --2- 172.21.15.3:0/3774950567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dc009f240 0x7f9dc006bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:59.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.929+0000 7f9dbed9d700 1 -- 172.21.15.3:0/3774950567 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db00094a0 con 0x7f9dc009f650 2023-10-03T10:12:59.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.930+0000 7f9dbed9d700 1 --2- 172.21.15.3:0/3774950567 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9dc009f650 0x7f9dc006c330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9db800b9a0 tx=0x7f9db800bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:12:59.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.930+0000 7f9daeffd700 1 -- 172.21.15.3:0/3774950567 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db800c930 con 0x7f9dc009f650 2023-10-03T10:12:59.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.930+0000 7f9daeffd700 1 -- 172.21.15.3:0/3774950567 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db800cab0 con 0x7f9dc009f650 2023-10-03T10:12:59.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.930+0000 7f9dc4e51700 1 -- 172.21.15.3:0/3774950567 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc00a0190 con 0x7f9dc009f650 2023-10-03T10:12:59.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.930+0000 7f9daeffd700 1 -- 172.21.15.3:0/3774950567 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db800ce60 con 0x7f9dc009f650 2023-10-03T10:12:59.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.930+0000 7f9dc4e51700 1 -- 172.21.15.3:0/3774950567 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc01b6bc0 con 0x7f9dc009f650 2023-10-03T10:12:59.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.931+0000 7f9daeffd700 1 -- 172.21.15.3:0/3774950567 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f9db801d3b0 con 0x7f9dc009f650 2023-10-03T10:12:59.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.932+0000 7f9daeffd700 1 -- 172.21.15.3:0/3774950567 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f9db8006e40 con 0x7f9dc009f650 2023-10-03T10:12:59.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.932+0000 7f9dc4e51700 1 -- 172.21.15.3:0/3774950567 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc009ef10 con 0x7f9dc009f650 2023-10-03T10:12:59.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.932+0000 7f9dbe59c700 1 -- 172.21.15.3:0/3774950567 >> v1:172.21.15.3:6816/127025 conn(0x7f9da002f510 legacy=0x7f9da00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:59.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.937+0000 7f9daeffd700 1 -- 172.21.15.3:0/3774950567 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9db8007370 con 0x7f9dc009f650 2023-10-03T10:12:59.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.957+0000 7f74f48ec700 1 -- 172.21.15.3:0/1813163928 >> v1:172.21.15.3:6816/127025 conn(0x7f74d402f510 legacy=0x7f74d40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:12:59.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.984+0000 7f74f6b50700 1 -- 172.21.15.3:0/1813163928 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f74f011e050 con 0x7f74f01338a0 2023-10-03T10:12:59.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.985+0000 7f74ed7fa700 1 -- 172.21.15.3:0/1813163928 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f74e80243b0 con 0x7f74f01338a0 2023-10-03T10:12:59.986 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:12:59.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.988+0000 7f74f6b50700 1 -- 172.21.15.3:0/1813163928 >> v1:172.21.15.3:6816/127025 conn(0x7f74d402f510 legacy=0x7f74d40319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:12:59.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.988+0000 7f74f6b50700 1 -- 172.21.15.3:0/1813163928 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74f01338a0 msgr2=0x7f74f011afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:12:59.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.988+0000 7f74f6b50700 1 --2- 172.21.15.3:0/1813163928 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74f01338a0 0x7f74f011afa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f74f006dc90 tx=0x7f74e8004ad0 comp rx=0 tx=0).stop 2023-10-03T10:12:59.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.988+0000 7f74f6b50700 1 -- 172.21.15.3:0/1813163928 shutdown_connections 2023-10-03T10:12:59.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.988+0000 7f74f6b50700 1 --2- 172.21.15.3:0/1813163928 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74f01338a0 0x7f74f011afa0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:59.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.988+0000 7f74f6b50700 1 --2- 172.21.15.3:0/1813163928 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74f009de50 0x7f74f011a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:59.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.988+0000 7f74f6b50700 1 --2- 172.21.15.3:0/1813163928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74f009e930 0x7f74f011aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:12:59.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.988+0000 7f74f6b50700 1 -- 172.21.15.3:0/1813163928 >> 172.21.15.3:0/1813163928 conn(0x7f74f00a2aa0 msgr2=0x7f74f0130730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:12:59.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.988+0000 7f74f6b50700 1 -- 172.21.15.3:0/1813163928 shutdown_connections 2023-10-03T10:12:59.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:12:59.988+0000 7f74f6b50700 1 -- 172.21.15.3:0/1813163928 wait complete. 2023-10-03T10:13:00.008 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:00.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.068+0000 7ff789079700 1 -- 172.21.15.3:0/976873873 >> v1:172.21.15.3:6816/127025 conn(0x7ff76802f510 legacy=0x7ff7680319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:00.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.088+0000 7ff78b2dd700 1 -- 172.21.15.3:0/976873873 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff784074ea0 con 0x7ff78409f650 2023-10-03T10:13:00.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.088+0000 7ff779ffb700 1 -- 172.21.15.3:0/976873873 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff77c007330 con 0x7ff78409f650 2023-10-03T10:13:00.088 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:00.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.090+0000 7ff78b2dd700 1 -- 172.21.15.3:0/976873873 >> v1:172.21.15.3:6816/127025 conn(0x7ff76802f510 legacy=0x7ff7680319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:00.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.090+0000 7ff78b2dd700 1 -- 172.21.15.3:0/976873873 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78409f650 msgr2=0x7ff78411deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:00.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.090+0000 7ff78b2dd700 1 --2- 172.21.15.3:0/976873873 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78409f650 0x7ff78411deb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00efc0 tx=0x7ff77c00c3a0 comp rx=0 tx=0).stop 2023-10-03T10:13:00.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.090+0000 7ff78b2dd700 1 -- 172.21.15.3:0/976873873 shutdown_connections 2023-10-03T10:13:00.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.090+0000 7ff78b2dd700 1 --2- 172.21.15.3:0/976873873 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78409f650 0x7ff78411deb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:00.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.090+0000 7ff78b2dd700 1 --2- 172.21.15.3:0/976873873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78409e930 0x7ff78411d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:00.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.090+0000 7ff78b2dd700 1 --2- 172.21.15.3:0/976873873 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78409f240 0x7ff78411d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:00.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.090+0000 7ff78b2dd700 1 -- 172.21.15.3:0/976873873 >> 172.21.15.3:0/976873873 conn(0x7ff7840a4210 msgr2=0x7ff78406c270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:00.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.090+0000 7ff78b2dd700 1 -- 172.21.15.3:0/976873873 shutdown_connections 2023-10-03T10:13:00.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.090+0000 7ff78b2dd700 1 -- 172.21.15.3:0/976873873 wait complete. 2023-10-03T10:13:00.106 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:00.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.106+0000 7ff651bf4700 1 -- 172.21.15.3:0/722067750 >> v1:172.21.15.3:6816/127025 conn(0x7ff63002f510 legacy=0x7ff6300319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:00.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.126+0000 7ff653e58700 1 -- 172.21.15.3:0/722067750 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff64c125bc0 con 0x7ff64c134170 2023-10-03T10:13:00.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.126+0000 7ff6427fc700 1 -- 172.21.15.3:0/722067750 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff64c125bc0 con 0x7ff64c134170 2023-10-03T10:13:00.126 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:00.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.128+0000 7ff653e58700 1 -- 172.21.15.3:0/722067750 >> v1:172.21.15.3:6816/127025 conn(0x7ff63002f510 legacy=0x7ff6300319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:00.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.128+0000 7ff653e58700 1 -- 172.21.15.3:0/722067750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff64c134170 msgr2=0x7ff64c123200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:00.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.128+0000 7ff653e58700 1 --2- 172.21.15.3:0/722067750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff64c134170 0x7ff64c123200 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff644016000 tx=0x7ff644004ad0 comp rx=0 tx=0).stop 2023-10-03T10:13:00.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.128+0000 7ff653e58700 1 -- 172.21.15.3:0/722067750 shutdown_connections 2023-10-03T10:13:00.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.128+0000 7ff653e58700 1 --2- 172.21.15.3:0/722067750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff64c134170 0x7ff64c123200 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:00.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.128+0000 7ff653e58700 1 --2- 172.21.15.3:0/722067750 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff64c0a0a30 0x7ff64c122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:00.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.128+0000 7ff653e58700 1 --2- 172.21.15.3:0/722067750 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff64c0a1340 0x7ff64c122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:00.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.128+0000 7ff653e58700 1 -- 172.21.15.3:0/722067750 >> 172.21.15.3:0/722067750 conn(0x7ff64c0a6550 msgr2=0x7ff64c09edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:00.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.128+0000 7ff653e58700 1 -- 172.21.15.3:0/722067750 shutdown_connections 2023-10-03T10:13:00.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.128+0000 7ff653e58700 1 -- 172.21.15.3:0/722067750 wait complete. 2023-10-03T10:13:00.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.133+0000 7f9dbe59c700 1 -- 172.21.15.3:0/3774950567 >> v1:172.21.15.3:6816/127025 conn(0x7f9da002f510 legacy=0x7f9da00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:00.142 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:00.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.154+0000 7f9dc4e51700 1 -- 172.21.15.3:0/3774950567 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9dc0074ea0 con 0x7f9dc009f650 2023-10-03T10:13:00.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.154+0000 7f9daeffd700 1 -- 172.21.15.3:0/3774950567 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9db8019030 con 0x7f9dc009f650 2023-10-03T10:13:00.154 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:00.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.156+0000 7f9dc4e51700 1 -- 172.21.15.3:0/3774950567 >> v1:172.21.15.3:6816/127025 conn(0x7f9da002f510 legacy=0x7f9da00319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:00.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.156+0000 7f9dc4e51700 1 -- 172.21.15.3:0/3774950567 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9dc009f650 msgr2=0x7f9dc006c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:00.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.156+0000 7f9dc4e51700 1 --2- 172.21.15.3:0/3774950567 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9dc009f650 0x7f9dc006c330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9db800b9a0 tx=0x7f9db800bcb0 comp rx=0 tx=0).stop 2023-10-03T10:13:00.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.156+0000 7f9dc4e51700 1 -- 172.21.15.3:0/3774950567 shutdown_connections 2023-10-03T10:13:00.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.156+0000 7f9dc4e51700 1 --2- 172.21.15.3:0/3774950567 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9dc009f650 0x7f9dc006c330 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:00.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.156+0000 7f9dc4e51700 1 --2- 172.21.15.3:0/3774950567 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9dc009e930 0x7f9dc006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:00.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.156+0000 7f9dc4e51700 1 --2- 172.21.15.3:0/3774950567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dc009f240 0x7f9dc006bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:00.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.156+0000 7f9dc4e51700 1 -- 172.21.15.3:0/3774950567 >> 172.21.15.3:0/3774950567 conn(0x7f9dc00a4210 msgr2=0x7f9dc01b8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:00.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.156+0000 7f9dc4e51700 1 -- 172.21.15.3:0/3774950567 shutdown_connections 2023-10-03T10:13:00.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:00.156+0000 7f9dc4e51700 1 -- 172.21.15.3:0/3774950567 wait complete. 2023-10-03T10:13:00.170 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:01.009 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:01.106 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:01.143 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:01.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.156+0000 7fc5ab4a5700 1 Processor -- start 2023-10-03T10:13:01.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.156+0000 7fc5ab4a5700 1 -- start start 2023-10-03T10:13:01.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.157+0000 7fc5ab4a5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5a4134630 0x7fc5a4136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:01.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.157+0000 7fc5ab4a5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5a4136f50 0x7fc5a4139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:01.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.157+0000 7fc5ab4a5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a409b4f0 con 0x7fc5a4131810 2023-10-03T10:13:01.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.157+0000 7fc5ab4a5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a41313c0 con 0x7fc5a4134630 2023-10-03T10:13:01.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.157+0000 7fc5ab4a5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a406e8c0 con 0x7fc5a4136f50 2023-10-03T10:13:01.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.157+0000 7fc5a9a42700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5a4136f50 0x7fc5a4139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:01.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.157+0000 7fc5a8a40700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5a4134630 0x7fc5a4136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:01.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.157+0000 7fc5a9241700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc5a4131810 0x7fc5a4131be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45428/0 (socket says 172.21.15.3:45428) 2023-10-03T10:13:01.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.157+0000 7fc5a9a42700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5a4136f50 0x7fc5a4139330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54840/0 (socket says 172.21.15.3:54840) 2023-10-03T10:13:01.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.157+0000 7fc5a9241700 1 -- 172.21.15.3:0/3261387350 learned_addr learned my addr 172.21.15.3:0/3261387350 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:01.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.157+0000 7fc5a9a42700 1 -- 172.21.15.3:0/3261387350 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5a4134630 msgr2=0x7fc5a4136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:01.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.157+0000 7fc5a9a42700 1 --2- 172.21.15.3:0/3261387350 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5a4134630 0x7fc5a4136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.157+0000 7fc5a9a42700 1 -- 172.21.15.3:0/3261387350 >> v1:172.21.15.3:6789/0 conn(0x7fc5a4131810 legacy=0x7fc5a4131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:01.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.157+0000 7fc5a9a42700 1 -- 172.21.15.3:0/3261387350 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a4071d20 con 0x7fc5a4136f50 2023-10-03T10:13:01.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.157+0000 7fc59bfff700 1 -- 172.21.15.3:0/3261387350 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1149817445 0 0) 0x7fc5a409b4f0 con 0x7fc5a4131810 2023-10-03T10:13:01.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.157+0000 7fc5a8a40700 1 --2- 172.21.15.3:0/3261387350 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5a4134630 0x7fc5a4136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:01.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.158+0000 7fc5a9a42700 1 --2- 172.21.15.3:0/3261387350 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5a4136f50 0x7fc5a4139330 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009dc0 tx=0x7fc59c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2506f1ed41cfe57 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:01.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.158+0000 7fc59bfff700 1 -- 172.21.15.3:0/3261387350 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c00e070 con 0x7fc5a4136f50 2023-10-03T10:13:01.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.158+0000 7fc59bfff700 1 -- 172.21.15.3:0/3261387350 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59c00bb20 con 0x7fc5a4136f50 2023-10-03T10:13:01.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.158+0000 7fc59bfff700 1 -- 172.21.15.3:0/3261387350 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c012430 con 0x7fc5a4136f50 2023-10-03T10:13:01.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.158+0000 7fc5ab4a5700 1 -- 172.21.15.3:0/3261387350 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5a4136f50 msgr2=0x7fc5a4139330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:01.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.158+0000 7fc5ab4a5700 1 --2- 172.21.15.3:0/3261387350 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5a4136f50 0x7fc5a4139330 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009dc0 tx=0x7fc59c00b330 comp rx=0 tx=0).stop 2023-10-03T10:13:01.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.159+0000 7fc5ab4a5700 1 -- 172.21.15.3:0/3261387350 shutdown_connections 2023-10-03T10:13:01.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.159+0000 7fc5ab4a5700 1 --2- 172.21.15.3:0/3261387350 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5a4136f50 0x7fc5a4139330 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.159+0000 7fc5ab4a5700 1 --2- 172.21.15.3:0/3261387350 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5a4134630 0x7fc5a4136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.159+0000 7fc5ab4a5700 1 -- 172.21.15.3:0/3261387350 >> 172.21.15.3:0/3261387350 conn(0x7fc5a40a2aa0 msgr2=0x7fc5a40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:01.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.159+0000 7fc5ab4a5700 1 -- 172.21.15.3:0/3261387350 shutdown_connections 2023-10-03T10:13:01.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.159+0000 7fc5ab4a5700 1 -- 172.21.15.3:0/3261387350 wait complete. 2023-10-03T10:13:01.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.160+0000 7fc5ab4a5700 1 Processor -- start 2023-10-03T10:13:01.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.160+0000 7fc5ab4a5700 1 -- start start 2023-10-03T10:13:01.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.160+0000 7fc5ab4a5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5a4131810 0x7fc5a41c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:01.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.160+0000 7fc5ab4a5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5a4134630 0x7fc5a41c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:01.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.160+0000 7fc5ab4a5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5a4136f50 0x7fc5a409c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:01.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.160+0000 7fc5ab4a5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a411e6b0 con 0x7fc5a4134630 2023-10-03T10:13:01.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.160+0000 7fc5ab4a5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a41c8f10 con 0x7fc5a4136f50 2023-10-03T10:13:01.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.160+0000 7fc5ab4a5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a409e000 con 0x7fc5a4131810 2023-10-03T10:13:01.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.160+0000 7fc5a9241700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5a4131810 0x7fc5a41c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:01.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.160+0000 7fc5a8a40700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5a4134630 0x7fc5a41c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:01.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.160+0000 7fc5a9241700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5a4131810 0x7fc5a41c8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54856/0 (socket says 172.21.15.3:54856) 2023-10-03T10:13:01.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.160+0000 7fc5a9241700 1 -- 172.21.15.3:0/713591066 learned_addr learned my addr 172.21.15.3:0/713591066 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:01.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.160+0000 7fc5a9a42700 1 --2- 172.21.15.3:0/713591066 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5a4136f50 0x7fc5a409c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:01.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.161+0000 7fc5a8a40700 1 -- 172.21.15.3:0/713591066 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5a4131810 msgr2=0x7fc5a41c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:01.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.161+0000 7fc5a8a40700 1 --2- 172.21.15.3:0/713591066 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5a4131810 0x7fc5a41c8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.161+0000 7fc5a8a40700 1 -- 172.21.15.3:0/713591066 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5a4136f50 msgr2=0x7fc5a409c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:01.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.161+0000 7fc5a8a40700 1 --2- 172.21.15.3:0/713591066 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5a4136f50 0x7fc5a409c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.161+0000 7fc5a8a40700 1 -- 172.21.15.3:0/713591066 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5940094a0 con 0x7fc5a4134630 2023-10-03T10:13:01.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.161+0000 7fc5a9a42700 1 --2- 172.21.15.3:0/713591066 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5a4136f50 0x7fc5a409c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:01.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.161+0000 7fc5a9241700 1 --2- 172.21.15.3:0/713591066 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5a4131810 0x7fc5a41c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:01.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.161+0000 7fc5a8a40700 1 --2- 172.21.15.3:0/713591066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5a4134630 0x7fc5a41c89d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc59400e9b0 tx=0x7fc59400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:01.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.161+0000 7fc599ffb700 1 -- 172.21.15.3:0/713591066 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59400ee50 con 0x7fc5a4134630 2023-10-03T10:13:01.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.161+0000 7fc599ffb700 1 -- 172.21.15.3:0/713591066 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59400c850 con 0x7fc5a4134630 2023-10-03T10:13:01.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.161+0000 7fc5ab4a5700 1 -- 172.21.15.3:0/713591066 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc59c009a70 con 0x7fc5a4134630 2023-10-03T10:13:01.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.161+0000 7fc599ffb700 1 -- 172.21.15.3:0/713591066 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59400cc00 con 0x7fc5a4134630 2023-10-03T10:13:01.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.162+0000 7fc5ab4a5700 1 -- 172.21.15.3:0/713591066 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a406eb00 con 0x7fc5a4134630 2023-10-03T10:13:01.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.163+0000 7fc599ffb700 1 -- 172.21.15.3:0/713591066 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fc594013470 con 0x7fc5a4134630 2023-10-03T10:13:01.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.163+0000 7fc599ffb700 1 -- 172.21.15.3:0/713591066 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7fc59403ffe0 con 0x7fc5a4134630 2023-10-03T10:13:01.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.163+0000 7fc5ab4a5700 1 -- 172.21.15.3:0/713591066 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a4074ea0 con 0x7fc5a4134630 2023-10-03T10:13:01.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.163+0000 7fc5a9241700 1 -- 172.21.15.3:0/713591066 >> v1:172.21.15.3:6816/127025 conn(0x7fc58802f510 legacy=0x7fc5880319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:01.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.168+0000 7fc599ffb700 1 -- 172.21.15.3:0/713591066 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc594017020 con 0x7fc5a4134630 2023-10-03T10:13:01.171 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:01.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.256+0000 7f35af73c700 1 Processor -- start 2023-10-03T10:13:01.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.256+0000 7f35af73c700 1 -- start start 2023-10-03T10:13:01.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.257+0000 7f35af73c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35a806c0e0 0x7f35a809ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:01.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.257+0000 7f35af73c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f35a8134ec0 0x7f35a81379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:01.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.257+0000 7f35af73c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35a8134980 con 0x7f35a8137f20 2023-10-03T10:13:01.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.257+0000 7f35af73c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35a806c4b0 con 0x7f35a8134ec0 2023-10-03T10:13:01.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.257+0000 7f35af73c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a80a09c0 con 0x7f35a806c0e0 2023-10-03T10:13:01.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.257+0000 7f35accd7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f35a8134ec0 0x7f35a81379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:01.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.257+0000 7f35accd7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f35a8134ec0 0x7f35a81379e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39240/0 (socket says 172.21.15.3:39240) 2023-10-03T10:13:01.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.257+0000 7f35accd7700 1 -- 172.21.15.3:0/770128670 learned_addr learned my addr 172.21.15.3:0/770128670 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:01.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.257+0000 7f35ad4d8700 1 --2- 172.21.15.3:0/770128670 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35a806c0e0 0x7f35a809ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:01.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.257+0000 7f35accd7700 1 -- 172.21.15.3:0/770128670 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35a806c0e0 msgr2=0x7f35a809ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:01.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.257+0000 7f35accd7700 1 --2- 172.21.15.3:0/770128670 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35a806c0e0 0x7f35a809ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.257+0000 7f35accd7700 1 -- 172.21.15.3:0/770128670 >> v1:172.21.15.3:6789/0 conn(0x7f35a8137f20 legacy=0x7f35a813ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:01.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.257+0000 7f35accd7700 1 -- 172.21.15.3:0/770128670 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a80a0600 con 0x7f35a8134ec0 2023-10-03T10:13:01.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.258+0000 7f35accd7700 1 --2- 172.21.15.3:0/770128670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35a8134ec0 0x7f35a81379e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f35980098f0 tx=0x7f3598009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a8a10145245dd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:01.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.258+0000 7f359ffff700 1 -- 172.21.15.3:0/770128670 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3598011830 con 0x7f35a8134ec0 2023-10-03T10:13:01.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.258+0000 7f359ffff700 1 -- 172.21.15.3:0/770128670 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3598011990 con 0x7f35a8134ec0 2023-10-03T10:13:01.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.258+0000 7f359ffff700 1 -- 172.21.15.3:0/770128670 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3598011d20 con 0x7f35a8134ec0 2023-10-03T10:13:01.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.259+0000 7f35af73c700 1 -- 172.21.15.3:0/770128670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35a8134ec0 msgr2=0x7f35a81379e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:01.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.259+0000 7f35af73c700 1 --2- 172.21.15.3:0/770128670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35a8134ec0 0x7f35a81379e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f35980098f0 tx=0x7f3598009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:01.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.259+0000 7f35af73c700 1 -- 172.21.15.3:0/770128670 shutdown_connections 2023-10-03T10:13:01.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.259+0000 7f35af73c700 1 --2- 172.21.15.3:0/770128670 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35a806c0e0 0x7f35a809ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.259+0000 7f35af73c700 1 --2- 172.21.15.3:0/770128670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35a8134ec0 0x7f35a81379e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.259+0000 7f35af73c700 1 -- 172.21.15.3:0/770128670 >> 172.21.15.3:0/770128670 conn(0x7f35a80a4210 msgr2=0x7f35a80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:01.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.259+0000 7f35af73c700 1 -- 172.21.15.3:0/770128670 shutdown_connections 2023-10-03T10:13:01.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.259+0000 7f35af73c700 1 -- 172.21.15.3:0/770128670 wait complete. 2023-10-03T10:13:01.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.260+0000 7f35af73c700 1 Processor -- start 2023-10-03T10:13:01.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.260+0000 7f35af73c700 1 -- start start 2023-10-03T10:13:01.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.261+0000 7f35af73c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35a806c0e0 0x7f35a81ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:01.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.261+0000 7f35af73c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35a8134ec0 0x7f35a81cd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:01.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.261+0000 7f35af73c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35a8137f20 0x7f35a81cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:01.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.261+0000 7f35af73c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a80a03f0 con 0x7f35a8137f20 2023-10-03T10:13:01.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.261+0000 7f35af73c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a81273b0 con 0x7f35a8134ec0 2023-10-03T10:13:01.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.261+0000 7f35af73c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f35a8137900 con 0x7f35a806c0e0 2023-10-03T10:13:01.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.261+0000 7f35adcd9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35a8137f20 0x7f35a81cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:01.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.261+0000 7f35adcd9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35a8137f20 0x7f35a81cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39250/0 (socket says 172.21.15.3:39250) 2023-10-03T10:13:01.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.261+0000 7f35adcd9700 1 -- 172.21.15.3:0/1653060581 learned_addr learned my addr 172.21.15.3:0/1653060581 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:01.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.261+0000 7f35ad4d8700 1 --2- 172.21.15.3:0/1653060581 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35a806c0e0 0x7f35a81ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:01.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.261+0000 7f35accd7700 1 --2- 172.21.15.3:0/1653060581 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35a8134ec0 0x7f35a81cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:01.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.261+0000 7f35adcd9700 1 -- 172.21.15.3:0/1653060581 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35a806c0e0 msgr2=0x7f35a81ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:01.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.261+0000 7f35adcd9700 1 --2- 172.21.15.3:0/1653060581 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35a806c0e0 0x7f35a81ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.261+0000 7f35adcd9700 1 -- 172.21.15.3:0/1653060581 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35a8134ec0 msgr2=0x7f35a81cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:01.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.261+0000 7f35adcd9700 1 --2- 172.21.15.3:0/1653060581 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35a8134ec0 0x7f35a81cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.262+0000 7f35adcd9700 1 -- 172.21.15.3:0/1653060581 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3598009550 con 0x7f35a8137f20 2023-10-03T10:13:01.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.262+0000 7f35accd7700 1 --2- 172.21.15.3:0/1653060581 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35a8134ec0 0x7f35a81cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:01.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.262+0000 7f35adcd9700 1 --2- 172.21.15.3:0/1653060581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35a8137f20 0x7f35a81cd560 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f35a000cc50 tx=0x7f35a000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:01.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.262+0000 7f359dffb700 1 -- 172.21.15.3:0/1653060581 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a0003c00 con 0x7f35a8137f20 2023-10-03T10:13:01.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.262+0000 7f359dffb700 1 -- 172.21.15.3:0/1653060581 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a0003d60 con 0x7f35a8137f20 2023-10-03T10:13:01.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.262+0000 7f359dffb700 1 -- 172.21.15.3:0/1653060581 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a000ab60 con 0x7f35a8137f20 2023-10-03T10:13:01.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.262+0000 7f35af73c700 1 -- 172.21.15.3:0/1653060581 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a8133a70 con 0x7f35a8137f20 2023-10-03T10:13:01.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.262+0000 7f35af73c700 1 -- 172.21.15.3:0/1653060581 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a811a680 con 0x7f35a8137f20 2023-10-03T10:13:01.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.263+0000 7f35af73c700 1 -- 172.21.15.3:0/1653060581 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a8074ea0 con 0x7f35a8137f20 2023-10-03T10:13:01.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.263+0000 7f359dffb700 1 -- 172.21.15.3:0/1653060581 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f35a000acc0 con 0x7f35a8137f20 2023-10-03T10:13:01.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.264+0000 7f359dffb700 1 -- 172.21.15.3:0/1653060581 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f35a003fbe0 con 0x7f35a8137f20 2023-10-03T10:13:01.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.264+0000 7f35ad4d8700 1 -- 172.21.15.3:0/1653060581 >> v1:172.21.15.3:6816/127025 conn(0x7f358c02f510 legacy=0x7f358c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:01.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.273+0000 7f359dffb700 1 -- 172.21.15.3:0/1653060581 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f35a00198f0 con 0x7f35a8137f20 2023-10-03T10:13:01.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.300+0000 7fad643a8700 1 Processor -- start 2023-10-03T10:13:01.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.300+0000 7fad643a8700 1 -- start start 2023-10-03T10:13:01.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.301+0000 7fad643a8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fad5c09f410 0x7fad5c136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:01.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.301+0000 7fad643a8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fad5c136fe0 0x7fad5c1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:01.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.301+0000 7fad643a8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad5c070940 con 0x7fad5c09e930 2023-10-03T10:13:01.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.301+0000 7fad643a8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad5c06fd60 con 0x7fad5c136fe0 2023-10-03T10:13:01.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.301+0000 7fad643a8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c06fa80 con 0x7fad5c09f410 2023-10-03T10:13:01.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.301+0000 7fad62945700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fad5c136fe0 0x7fad5c1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:01.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.301+0000 7fad62945700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fad5c136fe0 0x7fad5c1393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39258/0 (socket says 172.21.15.3:39258) 2023-10-03T10:13:01.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.301+0000 7fad62945700 1 -- 172.21.15.3:0/4272950212 learned_addr learned my addr 172.21.15.3:0/4272950212 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:01.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.301+0000 7fad61943700 1 --2- 172.21.15.3:0/4272950212 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fad5c09f410 0x7fad5c136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:01.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.302+0000 7fad62945700 1 -- 172.21.15.3:0/4272950212 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fad5c09f410 msgr2=0x7fad5c136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:01.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.302+0000 7fad62945700 1 --2- 172.21.15.3:0/4272950212 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fad5c09f410 0x7fad5c136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.302+0000 7fad62945700 1 -- 172.21.15.3:0/4272950212 >> v1:172.21.15.3:6789/0 conn(0x7fad5c09e930 legacy=0x7fad5c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:01.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.302+0000 7fad62945700 1 -- 172.21.15.3:0/4272950212 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5c09fb60 con 0x7fad5c136fe0 2023-10-03T10:13:01.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.302+0000 7fad62945700 1 --2- 172.21.15.3:0/4272950212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad5c136fe0 0x7fad5c1393c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fad54009e50 tx=0x7fad5400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dae07426206e9e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:01.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.302+0000 7fad61142700 1 -- 172.21.15.3:0/4272950212 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5400e070 con 0x7fad5c136fe0 2023-10-03T10:13:01.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.302+0000 7fad61142700 1 -- 172.21.15.3:0/4272950212 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5400b910 con 0x7fad5c136fe0 2023-10-03T10:13:01.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.302+0000 7fad61142700 1 -- 172.21.15.3:0/4272950212 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5400bcc0 con 0x7fad5c136fe0 2023-10-03T10:13:01.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.303+0000 7fad643a8700 1 -- 172.21.15.3:0/4272950212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad5c136fe0 msgr2=0x7fad5c1393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:01.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.303+0000 7fad643a8700 1 --2- 172.21.15.3:0/4272950212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad5c136fe0 0x7fad5c1393c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fad54009e50 tx=0x7fad5400b330 comp rx=0 tx=0).stop 2023-10-03T10:13:01.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.303+0000 7fad643a8700 1 -- 172.21.15.3:0/4272950212 shutdown_connections 2023-10-03T10:13:01.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.303+0000 7fad643a8700 1 --2- 172.21.15.3:0/4272950212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad5c136fe0 0x7fad5c1393c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.303+0000 7fad643a8700 1 --2- 172.21.15.3:0/4272950212 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fad5c09f410 0x7fad5c136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.303+0000 7fad643a8700 1 -- 172.21.15.3:0/4272950212 >> 172.21.15.3:0/4272950212 conn(0x7fad5c0a4210 msgr2=0x7fad5c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:01.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.303+0000 7fad643a8700 1 -- 172.21.15.3:0/4272950212 shutdown_connections 2023-10-03T10:13:01.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.303+0000 7fad643a8700 1 -- 172.21.15.3:0/4272950212 wait complete. 2023-10-03T10:13:01.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.305+0000 7fad643a8700 1 Processor -- start 2023-10-03T10:13:01.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.305+0000 7fad643a8700 1 -- start start 2023-10-03T10:13:01.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.305+0000 7fad643a8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad5c09e930 0x7fad5c11e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:01.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.305+0000 7fad643a8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fad5c09f410 0x7fad5c11eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:01.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.305+0000 7fad643a8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fad5c136fe0 0x7fad5c11f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:01.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.305+0000 7fad643a8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c133ef0 con 0x7fad5c09e930 2023-10-03T10:13:01.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.305+0000 7fad643a8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c1201d0 con 0x7fad5c136fe0 2023-10-03T10:13:01.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.305+0000 7fad643a8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c1344b0 con 0x7fad5c09f410 2023-10-03T10:13:01.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.306+0000 7fad62144700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad5c09e930 0x7fad5c11e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:01.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.306+0000 7fad62144700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad5c09e930 0x7fad5c11e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39266/0 (socket says 172.21.15.3:39266) 2023-10-03T10:13:01.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.306+0000 7fad62144700 1 -- 172.21.15.3:0/703334622 learned_addr learned my addr 172.21.15.3:0/703334622 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:01.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.306+0000 7fad62945700 1 --2- 172.21.15.3:0/703334622 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fad5c136fe0 0x7fad5c11f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:01.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.306+0000 7fad61943700 1 --2- 172.21.15.3:0/703334622 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fad5c09f410 0x7fad5c11eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:01.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.306+0000 7fad62144700 1 -- 172.21.15.3:0/703334622 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fad5c09f410 msgr2=0x7fad5c11eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:01.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.306+0000 7fad62144700 1 --2- 172.21.15.3:0/703334622 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fad5c09f410 0x7fad5c11eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.306+0000 7fad62144700 1 -- 172.21.15.3:0/703334622 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fad5c136fe0 msgr2=0x7fad5c11f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:01.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.306+0000 7fad62144700 1 --2- 172.21.15.3:0/703334622 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fad5c136fe0 0x7fad5c11f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.306+0000 7fad62144700 1 -- 172.21.15.3:0/703334622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad54009b30 con 0x7fad5c09e930 2023-10-03T10:13:01.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.306+0000 7fad62144700 1 --2- 172.21.15.3:0/703334622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad5c09e930 0x7fad5c11e5c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fad5800c510 tx=0x7fad5800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:01.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.306+0000 7fad52ffd700 1 -- 172.21.15.3:0/703334622 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad580052d0 con 0x7fad5c09e930 2023-10-03T10:13:01.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.306+0000 7fad52ffd700 1 -- 172.21.15.3:0/703334622 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad58005430 con 0x7fad5c09e930 2023-10-03T10:13:01.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.307+0000 7fad643a8700 1 -- 172.21.15.3:0/703334622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad5c071d20 con 0x7fad5c09e930 2023-10-03T10:13:01.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.307+0000 7fad52ffd700 1 -- 172.21.15.3:0/703334622 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad580038c0 con 0x7fad5c09e930 2023-10-03T10:13:01.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.307+0000 7fad643a8700 1 -- 172.21.15.3:0/703334622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad5c121870 con 0x7fad5c09e930 2023-10-03T10:13:01.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.308+0000 7fad643a8700 1 -- 172.21.15.3:0/703334622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad5c0a2f40 con 0x7fad5c09e930 2023-10-03T10:13:01.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.308+0000 7fad52ffd700 1 -- 172.21.15.3:0/703334622 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fad58003a20 con 0x7fad5c09e930 2023-10-03T10:13:01.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.309+0000 7fad52ffd700 1 -- 172.21.15.3:0/703334622 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7fad5803ef80 con 0x7fad5c09e930 2023-10-03T10:13:01.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.309+0000 7fad61943700 1 -- 172.21.15.3:0/703334622 >> v1:172.21.15.3:6816/127025 conn(0x7fad4002f510 legacy=0x7fad400319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:01.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.314+0000 7fad52ffd700 1 -- 172.21.15.3:0/703334622 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fad58019d20 con 0x7fad5c09e930 2023-10-03T10:13:01.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.319+0000 7f2a31f82700 1 Processor -- start 2023-10-03T10:13:01.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.319+0000 7f2a31f82700 1 -- start start 2023-10-03T10:13:01.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.320+0000 7f2a31f82700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2a2c09d6f0 0x7f2a2c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:01.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.320+0000 7f2a31f82700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2a2c09e000 0x7f2a2c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:01.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.320+0000 7f2a31f82700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2a2c09e410 0x7f2a2c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:01.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.320+0000 7f2a31f82700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a2c09f520 con 0x7f2a2c09e000 2023-10-03T10:13:01.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.320+0000 7f2a31f82700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c09f130 con 0x7f2a2c09d6f0 2023-10-03T10:13:01.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.320+0000 7f2a31f82700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c09e930 con 0x7f2a2c09e410 2023-10-03T10:13:01.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.320+0000 7f2a2bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2a2c09e410 0x7f2a2c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:01.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.320+0000 7f2a2bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2a2c09e410 0x7f2a2c130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54882/0 (socket says 172.21.15.3:54882) 2023-10-03T10:13:01.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.320+0000 7f2a2bfff700 1 -- 172.21.15.3:0/4092062489 learned_addr learned my addr 172.21.15.3:0/4092062489 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:01.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.320+0000 7f2a2b7fe700 1 --2- 172.21.15.3:0/4092062489 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2a2c09d6f0 0x7f2a2c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:01.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.320+0000 7f2a2affd700 1 --2- 172.21.15.3:0/4092062489 >> v2:172.21.15.3:3300/0 conn(0x7f2a2c09e000 0x7f2a2c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:01.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.321+0000 7f2a2bfff700 1 -- 172.21.15.3:0/4092062489 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2a2c09d6f0 msgr2=0x7f2a2c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:01.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.321+0000 7f2a2bfff700 1 --2- 172.21.15.3:0/4092062489 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2a2c09d6f0 0x7f2a2c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.321+0000 7f2a2bfff700 1 -- 172.21.15.3:0/4092062489 >> v2:172.21.15.3:3300/0 conn(0x7f2a2c09e000 msgr2=0x7f2a2c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:01.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.321+0000 7f2a2bfff700 1 --2- 172.21.15.3:0/4092062489 >> v2:172.21.15.3:3300/0 conn(0x7f2a2c09e000 0x7f2a2c130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.321+0000 7f2a2bfff700 1 -- 172.21.15.3:0/4092062489 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2c09b2d0 con 0x7f2a2c09e410 2023-10-03T10:13:01.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.321+0000 7f2a2bfff700 1 --2- 172.21.15.3:0/4092062489 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2a2c09e410 0x7f2a2c130ba0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2a2400bf60 tx=0x7f2a2400bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7d68b404d48b9d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:01.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.321+0000 7f2a2a7fc700 1 -- 172.21.15.3:0/4092062489 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2400b4e0 con 0x7f2a2c09e410 2023-10-03T10:13:01.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.321+0000 7f2a2a7fc700 1 -- 172.21.15.3:0/4092062489 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2400d9f0 con 0x7f2a2c09e410 2023-10-03T10:13:01.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.321+0000 7f2a2a7fc700 1 -- 172.21.15.3:0/4092062489 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2400dda0 con 0x7f2a2c09e410 2023-10-03T10:13:01.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.322+0000 7f2a31f82700 1 -- 172.21.15.3:0/4092062489 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2a2c09e410 msgr2=0x7f2a2c130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:01.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.322+0000 7f2a31f82700 1 --2- 172.21.15.3:0/4092062489 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2a2c09e410 0x7f2a2c130ba0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2a2400bf60 tx=0x7f2a2400bf90 comp rx=0 tx=0).stop 2023-10-03T10:13:01.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.322+0000 7f2a31f82700 1 -- 172.21.15.3:0/4092062489 shutdown_connections 2023-10-03T10:13:01.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.322+0000 7f2a31f82700 1 --2- 172.21.15.3:0/4092062489 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2a2c09e410 0x7f2a2c130ba0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.322+0000 7f2a31f82700 1 --2- 172.21.15.3:0/4092062489 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2a2c09d6f0 0x7f2a2c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.322+0000 7f2a31f82700 1 --2- 172.21.15.3:0/4092062489 >> v2:172.21.15.3:3300/0 conn(0x7f2a2c09e000 0x7f2a2c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.322+0000 7f2a31f82700 1 -- 172.21.15.3:0/4092062489 >> 172.21.15.3:0/4092062489 conn(0x7f2a2c066f50 msgr2=0x7f2a2c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:01.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.322+0000 7f2a31f82700 1 -- 172.21.15.3:0/4092062489 shutdown_connections 2023-10-03T10:13:01.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.322+0000 7f2a31f82700 1 -- 172.21.15.3:0/4092062489 wait complete. 2023-10-03T10:13:01.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.323+0000 7f2a31f82700 1 Processor -- start 2023-10-03T10:13:01.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.323+0000 7f2a31f82700 1 -- start start 2023-10-03T10:13:01.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.324+0000 7f2a31f82700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2a2c09d6f0 0x7f2a2c11dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:01.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.324+0000 7f2a31f82700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a2c09e000 0x7f2a2c11e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:01.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.324+0000 7f2a31f82700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2a2c09e410 0x7f2a2c11e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:01.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.324+0000 7f2a31f82700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c11f220 con 0x7f2a2c09e000 2023-10-03T10:13:01.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.324+0000 7f2a31f82700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c11f980 con 0x7f2a2c09d6f0 2023-10-03T10:13:01.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.324+0000 7f2a31f82700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c120bd0 con 0x7f2a2c09e410 2023-10-03T10:13:01.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.324+0000 7f2a2b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2a2c09d6f0 0x7f2a2c11dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:01.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.324+0000 7f2a2b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2a2c09d6f0 0x7f2a2c11dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:43204/0 (socket says 172.21.15.3:43204) 2023-10-03T10:13:01.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.324+0000 7f2a2b7fe700 1 -- 172.21.15.3:0/2113539215 learned_addr learned my addr 172.21.15.3:0/2113539215 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:01.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.324+0000 7f2a2affd700 1 --2- 172.21.15.3:0/2113539215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a2c09e000 0x7f2a2c11e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:01.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.324+0000 7f2a2bfff700 1 --2- 172.21.15.3:0/2113539215 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2a2c09e410 0x7f2a2c11e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:01.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.325+0000 7f2a2b7fe700 1 -- 172.21.15.3:0/2113539215 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2a2c09e410 msgr2=0x7f2a2c11e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:01.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.325+0000 7f2a2b7fe700 1 --2- 172.21.15.3:0/2113539215 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2a2c09e410 0x7f2a2c11e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.325+0000 7f2a2b7fe700 1 -- 172.21.15.3:0/2113539215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a2c09e000 msgr2=0x7f2a2c11e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:01.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.325+0000 7f2a2b7fe700 1 --2- 172.21.15.3:0/2113539215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a2c09e000 0x7f2a2c11e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.325+0000 7f2a2b7fe700 1 -- 172.21.15.3:0/2113539215 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2400bd40 con 0x7f2a2c09d6f0 2023-10-03T10:13:01.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.325+0000 7f2a2bfff700 1 --2- 172.21.15.3:0/2113539215 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2a2c09e410 0x7f2a2c11e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:01.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.325+0000 7f2a2b7fe700 1 --2- 172.21.15.3:0/2113539215 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2a2c09d6f0 0x7f2a2c11dd70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c00e930 tx=0x7f2a1c00ec40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:01.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.325+0000 7f2a13fff700 1 -- 172.21.15.3:0/2113539215 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c00ee80 con 0x7f2a2c09d6f0 2023-10-03T10:13:01.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.325+0000 7f2a13fff700 1 -- 172.21.15.3:0/2113539215 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a1c00c850 con 0x7f2a2c09d6f0 2023-10-03T10:13:01.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.325+0000 7f2a31f82700 1 -- 172.21.15.3:0/2113539215 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2c1215b0 con 0x7f2a2c09d6f0 2023-10-03T10:13:01.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.325+0000 7f2a13fff700 1 -- 172.21.15.3:0/2113539215 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c00cbe0 con 0x7f2a2c09d6f0 2023-10-03T10:13:01.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.325+0000 7f2a31f82700 1 -- 172.21.15.3:0/2113539215 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a2c09c700 con 0x7f2a2c09d6f0 2023-10-03T10:13:01.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.326+0000 7f2a13fff700 1 -- 172.21.15.3:0/2113539215 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f2a1c00cd40 con 0x7f2a2c09d6f0 2023-10-03T10:13:01.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.327+0000 7f2a31f82700 1 -- 172.21.15.3:0/2113539215 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a2c11ac20 con 0x7f2a2c09d6f0 2023-10-03T10:13:01.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.327+0000 7f2a2affd700 1 -- 172.21.15.3:0/2113539215 >> v1:172.21.15.3:6816/127025 conn(0x7f2a1802f510 legacy=0x7f2a180319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:01.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.327+0000 7f2a13fff700 1 -- 172.21.15.3:0/2113539215 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f2a1c03fc90 con 0x7f2a2c09d6f0 2023-10-03T10:13:01.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.336+0000 7f2a13fff700 1 -- 172.21.15.3:0/2113539215 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2a1c01bbe0 con 0x7f2a2c09d6f0 2023-10-03T10:13:01.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.364+0000 7fc5a9241700 1 -- 172.21.15.3:0/713591066 >> v1:172.21.15.3:6816/127025 conn(0x7fc58802f510 legacy=0x7fc5880319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:01.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.393+0000 7fc5ab4a5700 1 -- 172.21.15.3:0/713591066 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc5a412ec40 con 0x7fc5a4134630 2023-10-03T10:13:01.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.393+0000 7fc599ffb700 1 -- 172.21.15.3:0/713591066 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc594013720 con 0x7fc5a4134630 2023-10-03T10:13:01.393 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:01.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.396+0000 7fc5ab4a5700 1 -- 172.21.15.3:0/713591066 >> v1:172.21.15.3:6816/127025 conn(0x7fc58802f510 legacy=0x7fc5880319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:01.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.396+0000 7fc5ab4a5700 1 -- 172.21.15.3:0/713591066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5a4134630 msgr2=0x7fc5a41c89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:01.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.396+0000 7fc5ab4a5700 1 --2- 172.21.15.3:0/713591066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5a4134630 0x7fc5a41c89d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc59400e9b0 tx=0x7fc59400ecc0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.396+0000 7fc5ab4a5700 1 -- 172.21.15.3:0/713591066 shutdown_connections 2023-10-03T10:13:01.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.396+0000 7fc5ab4a5700 1 --2- 172.21.15.3:0/713591066 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5a4136f50 0x7fc5a409c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.396+0000 7fc5ab4a5700 1 --2- 172.21.15.3:0/713591066 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5a4131810 0x7fc5a41c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.396+0000 7fc5ab4a5700 1 --2- 172.21.15.3:0/713591066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5a4134630 0x7fc5a41c89d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.396+0000 7fc5ab4a5700 1 -- 172.21.15.3:0/713591066 >> 172.21.15.3:0/713591066 conn(0x7fc5a40a2aa0 msgr2=0x7fc5a40a0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:01.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.396+0000 7fc5ab4a5700 1 -- 172.21.15.3:0/713591066 shutdown_connections 2023-10-03T10:13:01.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.396+0000 7fc5ab4a5700 1 -- 172.21.15.3:0/713591066 wait complete. 2023-10-03T10:13:01.410 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:01.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.465+0000 7f35ad4d8700 1 -- 172.21.15.3:0/1653060581 >> v1:172.21.15.3:6816/127025 conn(0x7f358c02f510 legacy=0x7f358c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:01.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.490+0000 7f35af73c700 1 -- 172.21.15.3:0/1653060581 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f35a811dd10 con 0x7f35a8137f20 2023-10-03T10:13:01.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.490+0000 7f359dffb700 1 -- 172.21.15.3:0/1653060581 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f35a0019300 con 0x7f35a8137f20 2023-10-03T10:13:01.491 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:01.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.492+0000 7f35af73c700 1 -- 172.21.15.3:0/1653060581 >> v1:172.21.15.3:6816/127025 conn(0x7f358c02f510 legacy=0x7f358c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:01.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.492+0000 7f35af73c700 1 -- 172.21.15.3:0/1653060581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35a8137f20 msgr2=0x7f35a81cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:01.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.492+0000 7f35af73c700 1 --2- 172.21.15.3:0/1653060581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35a8137f20 0x7f35a81cd560 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f35a000cc50 tx=0x7f35a000cf60 comp rx=0 tx=0).stop 2023-10-03T10:13:01.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.492+0000 7f35af73c700 1 -- 172.21.15.3:0/1653060581 shutdown_connections 2023-10-03T10:13:01.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.492+0000 7f35af73c700 1 --2- 172.21.15.3:0/1653060581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35a8137f20 0x7f35a81cd560 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.492+0000 7f35af73c700 1 --2- 172.21.15.3:0/1653060581 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35a806c0e0 0x7f35a81ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.492+0000 7f35af73c700 1 --2- 172.21.15.3:0/1653060581 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35a8134ec0 0x7f35a81cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.493+0000 7f35af73c700 1 -- 172.21.15.3:0/1653060581 >> 172.21.15.3:0/1653060581 conn(0x7f35a80a4210 msgr2=0x7f35a806c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:01.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.493+0000 7f35af73c700 1 -- 172.21.15.3:0/1653060581 shutdown_connections 2023-10-03T10:13:01.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.493+0000 7f35af73c700 1 -- 172.21.15.3:0/1653060581 wait complete. 2023-10-03T10:13:01.507 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:01.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.510+0000 7fad61943700 1 -- 172.21.15.3:0/703334622 >> v1:172.21.15.3:6816/127025 conn(0x7fad4002f510 legacy=0x7fad400319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:01.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.528+0000 7f2a2affd700 1 -- 172.21.15.3:0/2113539215 >> v1:172.21.15.3:6816/127025 conn(0x7f2a1802f510 legacy=0x7f2a180319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:01.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.528+0000 7fad643a8700 1 -- 172.21.15.3:0/703334622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fad5c074ea0 con 0x7fad5c09e930 2023-10-03T10:13:01.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.529+0000 7fad52ffd700 1 -- 172.21.15.3:0/703334622 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fad58017030 con 0x7fad5c09e930 2023-10-03T10:13:01.529 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:01.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.531+0000 7fad643a8700 1 -- 172.21.15.3:0/703334622 >> v1:172.21.15.3:6816/127025 conn(0x7fad4002f510 legacy=0x7fad400319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:01.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.531+0000 7fad643a8700 1 -- 172.21.15.3:0/703334622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad5c09e930 msgr2=0x7fad5c11e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:01.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.531+0000 7fad643a8700 1 --2- 172.21.15.3:0/703334622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad5c09e930 0x7fad5c11e5c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fad5800c510 tx=0x7fad5800c820 comp rx=0 tx=0).stop 2023-10-03T10:13:01.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.531+0000 7fad643a8700 1 -- 172.21.15.3:0/703334622 shutdown_connections 2023-10-03T10:13:01.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.531+0000 7fad643a8700 1 --2- 172.21.15.3:0/703334622 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fad5c136fe0 0x7fad5c11f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.531+0000 7fad643a8700 1 --2- 172.21.15.3:0/703334622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad5c09e930 0x7fad5c11e5c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.531+0000 7fad643a8700 1 --2- 172.21.15.3:0/703334622 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fad5c09f410 0x7fad5c11eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.531+0000 7fad643a8700 1 -- 172.21.15.3:0/703334622 >> 172.21.15.3:0/703334622 conn(0x7fad5c0a4210 msgr2=0x7fad5c0a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:01.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.531+0000 7fad643a8700 1 -- 172.21.15.3:0/703334622 shutdown_connections 2023-10-03T10:13:01.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.531+0000 7fad643a8700 1 -- 172.21.15.3:0/703334622 wait complete. 2023-10-03T10:13:01.545 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:01.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.546+0000 7f2a31f82700 1 -- 172.21.15.3:0/2113539215 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2a2c074ea0 con 0x7f2a2c09d6f0 2023-10-03T10:13:01.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.546+0000 7f2a13fff700 1 -- 172.21.15.3:0/2113539215 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2a1c021090 con 0x7f2a2c09d6f0 2023-10-03T10:13:01.546 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:01.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.548+0000 7f2a31f82700 1 -- 172.21.15.3:0/2113539215 >> v1:172.21.15.3:6816/127025 conn(0x7f2a1802f510 legacy=0x7f2a180319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:01.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.548+0000 7f2a31f82700 1 -- 172.21.15.3:0/2113539215 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2a2c09d6f0 msgr2=0x7f2a2c11dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:01.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.548+0000 7f2a31f82700 1 --2- 172.21.15.3:0/2113539215 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2a2c09d6f0 0x7f2a2c11dd70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c00e930 tx=0x7f2a1c00ec40 comp rx=0 tx=0).stop 2023-10-03T10:13:01.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.548+0000 7f2a31f82700 1 -- 172.21.15.3:0/2113539215 shutdown_connections 2023-10-03T10:13:01.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.548+0000 7f2a31f82700 1 --2- 172.21.15.3:0/2113539215 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2a2c09e410 0x7f2a2c11e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.548+0000 7f2a31f82700 1 --2- 172.21.15.3:0/2113539215 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2a2c09d6f0 0x7f2a2c11dd70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.548+0000 7f2a31f82700 1 --2- 172.21.15.3:0/2113539215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a2c09e000 0x7f2a2c11e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:01.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.548+0000 7f2a31f82700 1 -- 172.21.15.3:0/2113539215 >> 172.21.15.3:0/2113539215 conn(0x7f2a2c066f50 msgr2=0x7f2a2c130800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:01.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.549+0000 7f2a31f82700 1 -- 172.21.15.3:0/2113539215 shutdown_connections 2023-10-03T10:13:01.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:01.549+0000 7f2a31f82700 1 -- 172.21.15.3:0/2113539215 wait complete. 2023-10-03T10:13:01.562 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:02.411 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:02.595 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:02.595 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:02.595 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:02.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.749+0000 7f2945099700 1 Processor -- start 2023-10-03T10:13:02.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.749+0000 7f2945099700 1 -- start start 2023-10-03T10:13:02.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.749+0000 7effadec5700 1 Processor -- start 2023-10-03T10:13:02.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.749+0000 7effadec5700 1 -- start start 2023-10-03T10:13:02.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.750+0000 7f2945099700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f294009d6d0 0x7f294009daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:02.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.750+0000 7f2945099700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f294009dfe0 0x7f294013bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:02.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.750+0000 7f2945099700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f294013c2f0 0x7f294013e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:02.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.750+0000 7f2945099700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f294009f0d0 con 0x7f294009dfe0 2023-10-03T10:13:02.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.750+0000 7f2945099700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294009a2d0 con 0x7f294013c2f0 2023-10-03T10:13:02.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.750+0000 7f2945099700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f294009a570 con 0x7f294009d6d0 2023-10-03T10:13:02.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.750+0000 7f293ed9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f294009d6d0 0x7f294009daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:02.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.750+0000 7effadec5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7effa80a0a30 0x7effa80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:02.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.750+0000 7effadec5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effa809d630 0x7effa80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:02.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.750+0000 7effadec5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7effa8133930 0x7effa8133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:02.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.750+0000 7effadec5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa80a2a70 con 0x7effa80a0a30 2023-10-03T10:13:02.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.750+0000 7effadec5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa80a01d0 con 0x7effa8133930 2023-10-03T10:13:02.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.750+0000 7effadec5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7effa809e2b0 con 0x7effa809d630 2023-10-03T10:13:02.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.750+0000 7effa6ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effa809d630 0x7effa80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:02.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.750+0000 7f293ed9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f294009d6d0 0x7f294009daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54904/0 (socket says 172.21.15.3:54904) 2023-10-03T10:13:02.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.750+0000 7f293ed9d700 1 -- 172.21.15.3:0/2347899877 learned_addr learned my addr 172.21.15.3:0/2347899877 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:02.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.750+0000 7effa6ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effa809d630 0x7effa80a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54910/0 (socket says 172.21.15.3:54910) 2023-10-03T10:13:02.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.750+0000 7effa6ffd700 1 -- 172.21.15.3:0/3834356986 learned_addr learned my addr 172.21.15.3:0/3834356986 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:02.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.750+0000 7f293ed9d700 1 -- 172.21.15.3:0/2347899877 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f294013c2f0 msgr2=0x7f294013e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:02.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.750+0000 7f293ed9d700 1 --2- 172.21.15.3:0/2347899877 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f294013c2f0 0x7f294013e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:02.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.750+0000 7f293ed9d700 1 -- 172.21.15.3:0/2347899877 >> v2:172.21.15.3:3300/0 conn(0x7f294009dfe0 msgr2=0x7f294013bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:02.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.750+0000 7f293ed9d700 1 --2- 172.21.15.3:0/2347899877 >> v2:172.21.15.3:3300/0 conn(0x7f294009dfe0 0x7f294013bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:02.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.750+0000 7f293ed9d700 1 -- 172.21.15.3:0/2347899877 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294009edc0 con 0x7f294009d6d0 2023-10-03T10:13:02.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.750+0000 7effa7fff700 1 --2- 172.21.15.3:0/3834356986 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7effa8133930 0x7effa8133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:02.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.750+0000 7f293ed9d700 1 --2- 172.21.15.3:0/2347899877 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f294009d6d0 0x7f294009daa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f29300098f0 tx=0x7f2930009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41c12387d703e5b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:02.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.751+0000 7effa6ffd700 1 -- 172.21.15.3:0/3834356986 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7effa8133930 msgr2=0x7effa8133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:02.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.751+0000 7effa6ffd700 1 --2- 172.21.15.3:0/3834356986 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7effa8133930 0x7effa8133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:02.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.751+0000 7effa6ffd700 1 -- 172.21.15.3:0/3834356986 >> v2:172.21.15.3:3300/0 conn(0x7effa80a0a30 msgr2=0x7effa80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:02.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.751+0000 7effa6ffd700 1 --2- 172.21.15.3:0/3834356986 >> v2:172.21.15.3:3300/0 conn(0x7effa80a0a30 0x7effa80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:02.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.751+0000 7effa6ffd700 1 -- 172.21.15.3:0/3834356986 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa809dc50 con 0x7effa809d630 2023-10-03T10:13:02.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.751+0000 7effa7fff700 1 --2- 172.21.15.3:0/3834356986 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7effa8133930 0x7effa8133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:13:02.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.751+0000 7effa6ffd700 1 --2- 172.21.15.3:0/3834356986 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effa809d630 0x7effa80a1340 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0098f0 tx=0x7eff9c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25a581c613407f26 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:02.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.751+0000 7f293dd9b700 1 -- 172.21.15.3:0/2347899877 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293000b8a0 con 0x7f294009d6d0 2023-10-03T10:13:02.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.751+0000 7f293dd9b700 1 -- 172.21.15.3:0/2347899877 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293000ba00 con 0x7f294009d6d0 2023-10-03T10:13:02.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.751+0000 7f293dd9b700 1 -- 172.21.15.3:0/2347899877 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293000bd90 con 0x7f294009d6d0 2023-10-03T10:13:02.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.751+0000 7effa67fc700 1 -- 172.21.15.3:0/3834356986 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c00b8a0 con 0x7effa809d630 2023-10-03T10:13:02.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.751+0000 7effa67fc700 1 -- 172.21.15.3:0/3834356986 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9c00ba00 con 0x7effa809d630 2023-10-03T10:13:02.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.751+0000 7effa67fc700 1 -- 172.21.15.3:0/3834356986 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c00bd90 con 0x7effa809d630 2023-10-03T10:13:02.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.752+0000 7f2945099700 1 -- 172.21.15.3:0/2347899877 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f294009d6d0 msgr2=0x7f294009daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:02.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.752+0000 7f2945099700 1 --2- 172.21.15.3:0/2347899877 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f294009d6d0 0x7f294009daa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f29300098f0 tx=0x7f2930009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:02.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.752+0000 7f2945099700 1 -- 172.21.15.3:0/2347899877 shutdown_connections 2023-10-03T10:13:02.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.752+0000 7f2945099700 1 --2- 172.21.15.3:0/2347899877 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f294013c2f0 0x7f294013e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:02.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.752+0000 7f2945099700 1 --2- 172.21.15.3:0/2347899877 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f294009d6d0 0x7f294009daa0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:02.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.752+0000 7f2945099700 1 --2- 172.21.15.3:0/2347899877 >> v2:172.21.15.3:3300/0 conn(0x7f294009dfe0 0x7f294013bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:02.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.752+0000 7f2945099700 1 -- 172.21.15.3:0/2347899877 >> 172.21.15.3:0/2347899877 conn(0x7f2940099c20 msgr2=0x7f29400a2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:02.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.752+0000 7f2945099700 1 -- 172.21.15.3:0/2347899877 shutdown_connections 2023-10-03T10:13:02.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.752+0000 7f2945099700 1 -- 172.21.15.3:0/2347899877 wait complete. 2023-10-03T10:13:02.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.752+0000 7effadec5700 1 -- 172.21.15.3:0/3834356986 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effa809d630 msgr2=0x7effa80a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:02.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.752+0000 7effadec5700 1 --2- 172.21.15.3:0/3834356986 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effa809d630 0x7effa80a1340 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0098f0 tx=0x7eff9c009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:02.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.752+0000 7effadec5700 1 -- 172.21.15.3:0/3834356986 shutdown_connections 2023-10-03T10:13:02.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.752+0000 7effadec5700 1 --2- 172.21.15.3:0/3834356986 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7effa8133930 0x7effa8133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:02.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.752+0000 7effadec5700 1 --2- 172.21.15.3:0/3834356986 >> v2:172.21.15.3:3300/0 conn(0x7effa80a0a30 0x7effa80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:02.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.752+0000 7effadec5700 1 --2- 172.21.15.3:0/3834356986 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effa809d630 0x7effa80a1340 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:02.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.752+0000 7effadec5700 1 -- 172.21.15.3:0/3834356986 >> 172.21.15.3:0/3834356986 conn(0x7effa80a3050 msgr2=0x7effa80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:03.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.752+0000 7effadec5700 1 -- 172.21.15.3:0/3834356986 shutdown_connections 2023-10-03T10:13:03.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.752+0000 7effadec5700 1 -- 172.21.15.3:0/3834356986 wait complete. 2023-10-03T10:13:03.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.753+0000 7f2945099700 1 Processor -- start 2023-10-03T10:13:03.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.753+0000 7effadec5700 1 Processor -- start 2023-10-03T10:13:03.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.753+0000 7f2945099700 1 -- start start 2023-10-03T10:13:03.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.754+0000 7effadec5700 1 -- start start 2023-10-03T10:13:03.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.754+0000 7f2945099700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f294009d6d0 0x7f29401cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:03.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.754+0000 7f2945099700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f294009dfe0 0x7f29401ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:03.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.754+0000 7f2945099700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f294013c2f0 0x7f29401cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:03.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.754+0000 7f2945099700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2940125bd0 con 0x7f294013c2f0 2023-10-03T10:13:03.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.754+0000 7f2945099700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29401273c0 con 0x7f294009dfe0 2023-10-03T10:13:03.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.754+0000 7f2945099700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29400a21a0 con 0x7f294009d6d0 2023-10-03T10:13:03.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.754+0000 7f293ed9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f294009d6d0 0x7f29401cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:03.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.754+0000 7f293ed9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f294009d6d0 0x7f29401cc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54914/0 (socket says 172.21.15.3:54914) 2023-10-03T10:13:03.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.754+0000 7f293ed9d700 1 -- 172.21.15.3:0/1568963027 learned_addr learned my addr 172.21.15.3:0/1568963027 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:03.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.754+0000 7f293e59c700 1 --2- 172.21.15.3:0/1568963027 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f294009dfe0 0x7f29401ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:03.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.754+0000 7effadec5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effa809d630 0x7effa81cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:03.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.754+0000 7effadec5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effa80a0a30 0x7effa81cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:03.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.754+0000 7effadec5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7effa8133930 0x7effa81ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:03.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.754+0000 7effadec5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa8071ab0 con 0x7effa809d630 2023-10-03T10:13:03.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.754+0000 7effadec5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa80a0410 con 0x7effa8133930 2023-10-03T10:13:03.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.754+0000 7effadec5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7effa8135ae0 con 0x7effa80a0a30 2023-10-03T10:13:03.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.755+0000 7effa6ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effa80a0a30 0x7effa81cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:03.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.755+0000 7effa6ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effa80a0a30 0x7effa81cc8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54928/0 (socket says 172.21.15.3:54928) 2023-10-03T10:13:03.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.755+0000 7effa6ffd700 1 -- 172.21.15.3:0/3758187973 learned_addr learned my addr 172.21.15.3:0/3758187973 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:03.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.755+0000 7f293e59c700 1 -- 172.21.15.3:0/1568963027 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f294009d6d0 msgr2=0x7f29401cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:03.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.755+0000 7f293e59c700 1 --2- 172.21.15.3:0/1568963027 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f294009d6d0 0x7f29401cc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:03.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.755+0000 7f293e59c700 1 -- 172.21.15.3:0/1568963027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f294013c2f0 msgr2=0x7f29401cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:03.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.755+0000 7f293e59c700 1 --2- 172.21.15.3:0/1568963027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f294013c2f0 0x7f29401cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:03.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.755+0000 7f293e59c700 1 -- 172.21.15.3:0/1568963027 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2930009550 con 0x7f294009dfe0 2023-10-03T10:13:03.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.755+0000 7f293e59c700 1 --2- 172.21.15.3:0/1568963027 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f294009dfe0 0x7f29401ccb30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2940072c00 tx=0x7f293400b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:03.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.755+0000 7effa6ffd700 1 -- 172.21.15.3:0/3758187973 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7effa8133930 msgr2=0x7effa81ccde0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:03.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.755+0000 7effa6ffd700 1 --2- 172.21.15.3:0/3758187973 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7effa8133930 0x7effa81ccde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:03.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.755+0000 7effa6ffd700 1 -- 172.21.15.3:0/3758187973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effa809d630 msgr2=0x7effa81cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:03.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.755+0000 7effa6ffd700 1 --2- 172.21.15.3:0/3758187973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effa809d630 0x7effa81cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:03.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.755+0000 7effa6ffd700 1 -- 172.21.15.3:0/3758187973 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9c009550 con 0x7effa80a0a30 2023-10-03T10:13:03.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.755+0000 7f29277fe700 1 -- 172.21.15.3:0/1568963027 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2934011830 con 0x7f294009dfe0 2023-10-03T10:13:03.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.755+0000 7f2945099700 1 -- 172.21.15.3:0/1568963027 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294009c6f0 con 0x7f294009dfe0 2023-10-03T10:13:03.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.755+0000 7f2945099700 1 -- 172.21.15.3:0/1568963027 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29401388f0 con 0x7f294009dfe0 2023-10-03T10:13:03.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.755+0000 7f29277fe700 1 -- 172.21.15.3:0/1568963027 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2934011990 con 0x7f294009dfe0 2023-10-03T10:13:03.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.755+0000 7f29277fe700 1 -- 172.21.15.3:0/1568963027 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2934011d00 con 0x7f294009dfe0 2023-10-03T10:13:03.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.755+0000 7effa6ffd700 1 --2- 172.21.15.3:0/3758187973 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effa80a0a30 0x7effa81cc8a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7eff9c000c00 tx=0x7eff9c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:03.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.756+0000 7eff8ffff700 1 -- 172.21.15.3:0/3758187973 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c0036a0 con 0x7effa80a0a30 2023-10-03T10:13:03.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.756+0000 7eff8ffff700 1 -- 172.21.15.3:0/3758187973 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9c003800 con 0x7effa80a0a30 2023-10-03T10:13:03.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.756+0000 7eff8ffff700 1 -- 172.21.15.3:0/3758187973 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c003b90 con 0x7effa80a0a30 2023-10-03T10:13:03.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.756+0000 7effadec5700 1 -- 172.21.15.3:0/3758187973 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa81390a0 con 0x7effa80a0a30 2023-10-03T10:13:03.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.757+0000 7f2945099700 1 -- 172.21.15.3:0/1568963027 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29200052e0 con 0x7f294009dfe0 2023-10-03T10:13:03.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.756+0000 7effadec5700 1 -- 172.21.15.3:0/3758187973 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa8137470 con 0x7effa80a0a30 2023-10-03T10:13:03.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.757+0000 7eff8ffff700 1 -- 172.21.15.3:0/3758187973 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7eff9c003cf0 con 0x7effa80a0a30 2023-10-03T10:13:03.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.757+0000 7f29277fe700 1 -- 172.21.15.3:0/1568963027 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f293401a400 con 0x7f294009dfe0 2023-10-03T10:13:03.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.757+0000 7f293ed9d700 1 -- 172.21.15.3:0/1568963027 >> v1:172.21.15.3:6816/127025 conn(0x7f292c02f510 legacy=0x7f292c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:03.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.757+0000 7effadec5700 1 -- 172.21.15.3:0/3758187973 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa8138d40 con 0x7effa80a0a30 2023-10-03T10:13:03.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.757+0000 7effa77fe700 1 -- 172.21.15.3:0/3758187973 >> v1:172.21.15.3:6816/127025 conn(0x7eff9402f510 legacy=0x7eff940319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:03.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.761+0000 7f29277fe700 1 -- 172.21.15.3:0/1568963027 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f2934010380 con 0x7f294009dfe0 2023-10-03T10:13:03.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.762+0000 7f29277fe700 1 -- 172.21.15.3:0/1568963027 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f293401a6b0 con 0x7f294009dfe0 2023-10-03T10:13:03.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.763+0000 7eff8ffff700 1 -- 172.21.15.3:0/3758187973 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7eff9c021030 con 0x7effa80a0a30 2023-10-03T10:13:03.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.767+0000 7eff8ffff700 1 -- 172.21.15.3:0/3758187973 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7effa8138d40 con 0x7effa80a0a30 2023-10-03T10:13:03.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.798+0000 7f0dc160b700 1 Processor -- start 2023-10-03T10:13:03.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.798+0000 7f0dc160b700 1 -- start start 2023-10-03T10:13:03.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.798+0000 7f0dc160b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dbc06a770 0x7f0dbc06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:03.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.798+0000 7f0dc160b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0dbc06b080 0x7f0dbc131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:03.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.798+0000 7f0dc160b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dbc09ece0 con 0x7f0dbc131b40 2023-10-03T10:13:03.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.798+0000 7f0dc160b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dbc09e760 con 0x7f0dbc06b080 2023-10-03T10:13:03.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.798+0000 7f0dc160b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc09e040 con 0x7f0dbc06a770 2023-10-03T10:13:03.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.799+0000 7f0dbaffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dbc06a770 0x7f0dbc06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:03.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.799+0000 7f0dba7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0dbc06b080 0x7f0dbc131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:03.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.799+0000 7f0dba7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0dbc06b080 0x7f0dbc131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39332/0 (socket says 172.21.15.3:39332) 2023-10-03T10:13:03.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.799+0000 7f0dbaffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dbc06a770 0x7f0dbc06ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54934/0 (socket says 172.21.15.3:54934) 2023-10-03T10:13:03.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.799+0000 7f0dba7fc700 1 -- 172.21.15.3:0/3314645659 learned_addr learned my addr 172.21.15.3:0/3314645659 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:03.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.799+0000 7f0dbaffd700 1 -- 172.21.15.3:0/3314645659 >> v2:172.21.15.3:3300/0 conn(0x7f0dbc06b080 msgr2=0x7f0dbc131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:03.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.799+0000 7f0dbaffd700 1 --2- 172.21.15.3:0/3314645659 >> v2:172.21.15.3:3300/0 conn(0x7f0dbc06b080 0x7f0dbc131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:03.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.799+0000 7f0dbaffd700 1 -- 172.21.15.3:0/3314645659 >> v1:172.21.15.3:6789/0 conn(0x7f0dbc131b40 legacy=0x7f0dbc138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:03.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.799+0000 7f0dbaffd700 1 -- 172.21.15.3:0/3314645659 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dbc09f630 con 0x7f0dbc06a770 2023-10-03T10:13:03.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.799+0000 7f0dbaffd700 1 --2- 172.21.15.3:0/3314645659 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dbc06a770 0x7f0dbc06ab40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0dac009840 tx=0x7f0dac009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10b46884a5a25696 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:03.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.799+0000 7f0db9ffb700 1 -- 172.21.15.3:0/3314645659 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dac004230 con 0x7f0dbc06a770 2023-10-03T10:13:03.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.799+0000 7f0db9ffb700 1 -- 172.21.15.3:0/3314645659 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dac004390 con 0x7f0dbc06a770 2023-10-03T10:13:03.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.799+0000 7f0db9ffb700 1 -- 172.21.15.3:0/3314645659 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dac0036a0 con 0x7f0dbc06a770 2023-10-03T10:13:03.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.800+0000 7f0dc160b700 1 -- 172.21.15.3:0/3314645659 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dbc06a770 msgr2=0x7f0dbc06ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:03.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.800+0000 7f0dc160b700 1 --2- 172.21.15.3:0/3314645659 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dbc06a770 0x7f0dbc06ab40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0dac009840 tx=0x7f0dac009b50 comp rx=0 tx=0).stop 2023-10-03T10:13:03.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.800+0000 7f0dc160b700 1 -- 172.21.15.3:0/3314645659 shutdown_connections 2023-10-03T10:13:03.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.800+0000 7f0dc160b700 1 --2- 172.21.15.3:0/3314645659 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dbc06a770 0x7f0dbc06ab40 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0dac009840 tx=0x7f0dac009b50 comp rx=0 tx=0).stop 2023-10-03T10:13:03.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.800+0000 7f0dc160b700 1 --2- 172.21.15.3:0/3314645659 >> v2:172.21.15.3:3300/0 conn(0x7f0dbc06b080 0x7f0dbc131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:03.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.800+0000 7f0dc160b700 1 -- 172.21.15.3:0/3314645659 >> 172.21.15.3:0/3314645659 conn(0x7f0dbc066f50 msgr2=0x7f0dbc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:03.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.800+0000 7f0dc160b700 1 -- 172.21.15.3:0/3314645659 shutdown_connections 2023-10-03T10:13:03.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.800+0000 7f0dc160b700 1 -- 172.21.15.3:0/3314645659 wait complete. 2023-10-03T10:13:03.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.801+0000 7f0dc160b700 1 Processor -- start 2023-10-03T10:13:03.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.801+0000 7f0dc160b700 1 -- start start 2023-10-03T10:13:03.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.802+0000 7f0dc160b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dbc06b080 0x7f0dbc1c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:03.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.802+0000 7f0dc160b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dbc131b40 0x7f0dbc1c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:03.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.802+0000 7f0dc160b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dbc1c8ed0 0x7f0dbc130280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:03.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.802+0000 7f0dc160b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc121e30 con 0x7f0dbc1c8ed0 2023-10-03T10:13:03.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.802+0000 7f0dc160b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc1213b0 con 0x7f0dbc06b080 2023-10-03T10:13:03.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.802+0000 7f0dc160b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc071e60 con 0x7f0dbc131b40 2023-10-03T10:13:03.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.802+0000 7f0dbb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dbc1c8ed0 0x7f0dbc130280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:03.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.802+0000 7f0dba7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dbc131b40 0x7f0dbc1c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:03.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.802+0000 7f0dbb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dbc1c8ed0 0x7f0dbc130280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39348/0 (socket says 172.21.15.3:39348) 2023-10-03T10:13:03.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.802+0000 7f0dbb7fe700 1 -- 172.21.15.3:0/3430457611 learned_addr learned my addr 172.21.15.3:0/3430457611 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:03.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.802+0000 7f0dbaffd700 1 --2- 172.21.15.3:0/3430457611 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dbc06b080 0x7f0dbc1c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:03.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.803+0000 7f0dbb7fe700 1 -- 172.21.15.3:0/3430457611 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dbc131b40 msgr2=0x7f0dbc1c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:03.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.803+0000 7f0dbb7fe700 1 --2- 172.21.15.3:0/3430457611 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dbc131b40 0x7f0dbc1c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:03.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.803+0000 7f0dbb7fe700 1 -- 172.21.15.3:0/3430457611 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dbc06b080 msgr2=0x7f0dbc1c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:03.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.803+0000 7f0dbb7fe700 1 --2- 172.21.15.3:0/3430457611 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dbc06b080 0x7f0dbc1c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:03.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.803+0000 7f0dbb7fe700 1 -- 172.21.15.3:0/3430457611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dac0094a0 con 0x7f0dbc1c8ed0 2023-10-03T10:13:03.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.803+0000 7f0dbaffd700 1 --2- 172.21.15.3:0/3430457611 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dbc06b080 0x7f0dbc1c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:03.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.803+0000 7f0dbb7fe700 1 --2- 172.21.15.3:0/3430457611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dbc1c8ed0 0x7f0dbc130280 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0db400caf0 tx=0x7f0db400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:03.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.803+0000 7f0da37fe700 1 -- 172.21.15.3:0/3430457611 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db4003e30 con 0x7f0dbc1c8ed0 2023-10-03T10:13:03.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.803+0000 7f0da37fe700 1 -- 172.21.15.3:0/3430457611 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db400ce70 con 0x7f0dbc1c8ed0 2023-10-03T10:13:03.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.803+0000 7f0da37fe700 1 -- 172.21.15.3:0/3430457611 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db4005290 con 0x7f0dbc1c8ed0 2023-10-03T10:13:03.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.803+0000 7f0dc160b700 1 -- 172.21.15.3:0/3430457611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dbc09c010 con 0x7f0dbc1c8ed0 2023-10-03T10:13:03.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.803+0000 7f0dc160b700 1 -- 172.21.15.3:0/3430457611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dbc122ad0 con 0x7f0dbc1c8ed0 2023-10-03T10:13:03.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.803+0000 7f2407d6b700 1 Processor -- start 2023-10-03T10:13:03.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.804+0000 7f2407d6b700 1 -- start start 2023-10-03T10:13:03.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.804+0000 7f0dc160b700 1 -- 172.21.15.3:0/3430457611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dbc074ea0 con 0x7f0dbc1c8ed0 2023-10-03T10:13:03.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.804+0000 7f2407d6b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2400071b40 0x7f2400130420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:03.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.805+0000 7f2407d6b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2400130d50 0x7f2400133c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:03.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.805+0000 7f2407d6b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2400099ef0 con 0x7f2400130960 2023-10-03T10:13:03.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.805+0000 7f2407d6b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2400099500 con 0x7f2400130d50 2023-10-03T10:13:03.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.805+0000 7f2407d6b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2400099c10 con 0x7f2400071b40 2023-10-03T10:13:03.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.805+0000 7f2405b07700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2400071b40 0x7f2400130420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:03.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.805+0000 7f2406308700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2400130d50 0x7f2400133c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:03.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.805+0000 7f2405b07700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2400071b40 0x7f2400130420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54952/0 (socket says 172.21.15.3:54952) 2023-10-03T10:13:03.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.805+0000 7f2405b07700 1 -- 172.21.15.3:0/832891542 learned_addr learned my addr 172.21.15.3:0/832891542 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:03.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.805+0000 7f2406308700 1 -- 172.21.15.3:0/832891542 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2400071b40 msgr2=0x7f2400130420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:03.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.805+0000 7f2406308700 1 --2- 172.21.15.3:0/832891542 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2400071b40 0x7f2400130420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:03.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.805+0000 7f2406308700 1 -- 172.21.15.3:0/832891542 >> v1:172.21.15.3:6789/0 conn(0x7f2400130960 legacy=0x7f2400133570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:03.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.805+0000 7f2406308700 1 -- 172.21.15.3:0/832891542 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f240009e370 con 0x7f2400130d50 2023-10-03T10:13:03.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.805+0000 7f0da37fe700 1 -- 172.21.15.3:0/3430457611 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f0db40053f0 con 0x7f0dbc1c8ed0 2023-10-03T10:13:03.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.805+0000 7f2406308700 1 --2- 172.21.15.3:0/832891542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2400130d50 0x7f2400133c80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f23f8009d90 tx=0x7f23f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31efcd75a5753afc server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:03.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.806+0000 7f0dbaffd700 1 -- 172.21.15.3:0/3430457611 >> v1:172.21.15.3:6816/127025 conn(0x7f0da802f540 legacy=0x7f0da80319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:03.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.806+0000 7f0da37fe700 1 -- 172.21.15.3:0/3430457611 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f0db40404f0 con 0x7f0dbc1c8ed0 2023-10-03T10:13:03.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.810+0000 7f2404b05700 1 -- 172.21.15.3:0/832891542 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f800e070 con 0x7f2400130d50 2023-10-03T10:13:03.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.810+0000 7f2404b05700 1 -- 172.21.15.3:0/832891542 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f800bbd0 con 0x7f2400130d50 2023-10-03T10:13:03.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.810+0000 7f2404b05700 1 -- 172.21.15.3:0/832891542 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f8012430 con 0x7f2400130d50 2023-10-03T10:13:03.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.810+0000 7f0da37fe700 1 -- 172.21.15.3:0/3430457611 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0db4024bd0 con 0x7f0dbc1c8ed0 2023-10-03T10:13:03.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.811+0000 7f2407d6b700 1 -- 172.21.15.3:0/832891542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2400130d50 msgr2=0x7f2400133c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:03.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.811+0000 7f2407d6b700 1 --2- 172.21.15.3:0/832891542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2400130d50 0x7f2400133c80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f23f8009d90 tx=0x7f23f800b330 comp rx=0 tx=0).stop 2023-10-03T10:13:03.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.811+0000 7f2407d6b700 1 -- 172.21.15.3:0/832891542 shutdown_connections 2023-10-03T10:13:03.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.811+0000 7f2407d6b700 1 --2- 172.21.15.3:0/832891542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2400130d50 0x7f2400133c80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:03.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.811+0000 7f2407d6b700 1 --2- 172.21.15.3:0/832891542 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2400071b40 0x7f2400130420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:03.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.811+0000 7f2407d6b700 1 -- 172.21.15.3:0/832891542 >> 172.21.15.3:0/832891542 conn(0x7f240009e7e0 msgr2=0x7f240009ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:03.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.811+0000 7f2407d6b700 1 -- 172.21.15.3:0/832891542 shutdown_connections 2023-10-03T10:13:03.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.811+0000 7f2407d6b700 1 -- 172.21.15.3:0/832891542 wait complete. 2023-10-03T10:13:03.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.812+0000 7f2407d6b700 1 Processor -- start 2023-10-03T10:13:03.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.812+0000 7f2407d6b700 1 -- start start 2023-10-03T10:13:03.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.813+0000 7f2407d6b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2400071b40 0x7f2400126e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:03.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.813+0000 7f2407d6b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2400130960 0x7f2400127340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:03.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.813+0000 7f2407d6b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2400130d50 0x7f2400127880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:03.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.813+0000 7f2407d6b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24001284f0 con 0x7f2400130d50 2023-10-03T10:13:03.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.813+0000 7f2407d6b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2400128d30 con 0x7f2400071b40 2023-10-03T10:13:03.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.813+0000 7f2407d6b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24001297e0 con 0x7f2400130960 2023-10-03T10:13:03.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.813+0000 7f2406308700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2400130d50 0x7f2400127880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:03.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.813+0000 7f2406308700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2400130d50 0x7f2400127880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39358/0 (socket says 172.21.15.3:39358) 2023-10-03T10:13:03.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.813+0000 7f2406308700 1 -- 172.21.15.3:0/2644552079 learned_addr learned my addr 172.21.15.3:0/2644552079 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:03.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.813+0000 7f2405306700 1 --2- 172.21.15.3:0/2644552079 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2400130960 0x7f2400127340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:03.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.813+0000 7f2406308700 1 -- 172.21.15.3:0/2644552079 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2400130960 msgr2=0x7f2400127340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:03.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.813+0000 7f2406308700 1 --2- 172.21.15.3:0/2644552079 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2400130960 0x7f2400127340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:03.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.813+0000 7f2406308700 1 -- 172.21.15.3:0/2644552079 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2400071b40 msgr2=0x7f2400126e00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:03.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.813+0000 7f2406308700 1 --2- 172.21.15.3:0/2644552079 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2400071b40 0x7f2400126e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:03.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.813+0000 7f2406308700 1 -- 172.21.15.3:0/2644552079 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f8009a70 con 0x7f2400130d50 2023-10-03T10:13:03.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.814+0000 7f2406308700 1 --2- 172.21.15.3:0/2644552079 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2400130d50 0x7f2400127880 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f23f800b370 tx=0x7f23f8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:03.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.814+0000 7f23f67fc700 1 -- 172.21.15.3:0/2644552079 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f8004830 con 0x7f2400130d50 2023-10-03T10:13:03.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.814+0000 7f23f67fc700 1 -- 172.21.15.3:0/2644552079 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f8020070 con 0x7f2400130d50 2023-10-03T10:13:03.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.814+0000 7f23f67fc700 1 -- 172.21.15.3:0/2644552079 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f80276a0 con 0x7f2400130d50 2023-10-03T10:13:03.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.814+0000 7f2407d6b700 1 -- 172.21.15.3:0/2644552079 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2400129000 con 0x7f2400130d50 2023-10-03T10:13:03.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.814+0000 7f2407d6b700 1 -- 172.21.15.3:0/2644552079 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f240012fd70 con 0x7f2400130d50 2023-10-03T10:13:03.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.815+0000 7f23f67fc700 1 -- 172.21.15.3:0/2644552079 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f23f8012430 con 0x7f2400130d50 2023-10-03T10:13:03.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.815+0000 7f2407d6b700 1 -- 172.21.15.3:0/2644552079 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24000747f0 con 0x7f2400130d50 2023-10-03T10:13:03.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.815+0000 7f23f67fc700 1 -- 172.21.15.3:0/2644552079 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f23f8022070 con 0x7f2400130d50 2023-10-03T10:13:03.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.815+0000 7f2405b07700 1 -- 172.21.15.3:0/2644552079 >> v1:172.21.15.3:6816/127025 conn(0x7f23e402f510 legacy=0x7f23e40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:03.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.825+0000 7f23f67fc700 1 -- 172.21.15.3:0/2644552079 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f23f802c020 con 0x7f2400130d50 2023-10-03T10:13:03.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.958+0000 7f293ed9d700 1 -- 172.21.15.3:0/1568963027 >> v1:172.21.15.3:6816/127025 conn(0x7f292c02f510 legacy=0x7f292c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:03.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.958+0000 7effa77fe700 1 -- 172.21.15.3:0/3758187973 >> v1:172.21.15.3:6816/127025 conn(0x7eff9402f510 legacy=0x7eff940319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:03.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.982+0000 7effadec5700 1 -- 172.21.15.3:0/3758187973 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7effa8138d40 con 0x7effa80a0a30 2023-10-03T10:13:03.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.982+0000 7eff8ffff700 1 -- 172.21.15.3:0/3758187973 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7effa8138d40 con 0x7effa80a0a30 2023-10-03T10:13:03.100 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:03.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.984+0000 7f2945099700 1 -- 172.21.15.3:0/1568963027 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2920005860 con 0x7f294009dfe0 2023-10-03T10:13:03.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.984+0000 7effadec5700 1 -- 172.21.15.3:0/3758187973 >> v1:172.21.15.3:6816/127025 conn(0x7eff9402f510 legacy=0x7eff940319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:03.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.984+0000 7effadec5700 1 -- 172.21.15.3:0/3758187973 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effa80a0a30 msgr2=0x7effa81cc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:03.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.984+0000 7effadec5700 1 --2- 172.21.15.3:0/3758187973 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effa80a0a30 0x7effa81cc8a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7eff9c000c00 tx=0x7eff9c004750 comp rx=0 tx=0).stop 2023-10-03T10:13:03.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.984+0000 7effadec5700 1 -- 172.21.15.3:0/3758187973 shutdown_connections 2023-10-03T10:13:03.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.984+0000 7effadec5700 1 --2- 172.21.15.3:0/3758187973 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7effa8133930 0x7effa81ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:03.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.984+0000 7effadec5700 1 --2- 172.21.15.3:0/3758187973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effa809d630 0x7effa81cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:03.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.984+0000 7effadec5700 1 --2- 172.21.15.3:0/3758187973 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effa80a0a30 0x7effa81cc8a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:03.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.984+0000 7effadec5700 1 -- 172.21.15.3:0/3758187973 >> 172.21.15.3:0/3758187973 conn(0x7effa80a3050 msgr2=0x7effa809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:03.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.985+0000 7effadec5700 1 -- 172.21.15.3:0/3758187973 shutdown_connections 2023-10-03T10:13:03.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.985+0000 7f29277fe700 1 -- 172.21.15.3:0/1568963027 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2934028020 con 0x7f294009dfe0 2023-10-03T10:13:03.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.985+0000 7effadec5700 1 -- 172.21.15.3:0/3758187973 wait complete. 2023-10-03T10:13:03.103 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:03.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.987+0000 7f2945099700 1 -- 172.21.15.3:0/1568963027 >> v1:172.21.15.3:6816/127025 conn(0x7f292c02f510 legacy=0x7f292c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:03.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.987+0000 7f2945099700 1 -- 172.21.15.3:0/1568963027 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f294009dfe0 msgr2=0x7f29401ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:03.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.987+0000 7f2945099700 1 --2- 172.21.15.3:0/1568963027 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f294009dfe0 0x7f29401ccb30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2940072c00 tx=0x7f293400b730 comp rx=0 tx=0).stop 2023-10-03T10:13:03.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.987+0000 7f2945099700 1 -- 172.21.15.3:0/1568963027 shutdown_connections 2023-10-03T10:13:03.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.987+0000 7f2945099700 1 --2- 172.21.15.3:0/1568963027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f294013c2f0 0x7f29401cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:03.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.987+0000 7f2945099700 1 --2- 172.21.15.3:0/1568963027 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f294009d6d0 0x7f29401cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:03.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.987+0000 7f2945099700 1 --2- 172.21.15.3:0/1568963027 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f294009dfe0 0x7f29401ccb30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:03.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.987+0000 7f2945099700 1 -- 172.21.15.3:0/1568963027 >> 172.21.15.3:0/1568963027 conn(0x7f2940099c20 msgr2=0x7f29400a0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:03.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.987+0000 7f2945099700 1 -- 172.21.15.3:0/1568963027 shutdown_connections 2023-10-03T10:13:03.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:02.987+0000 7f2945099700 1 -- 172.21.15.3:0/1568963027 wait complete. 2023-10-03T10:13:03.105 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:03.106 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:03.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:03.007+0000 7f0dbaffd700 1 -- 172.21.15.3:0/3430457611 >> v1:172.21.15.3:6816/127025 conn(0x7f0da802f540 legacy=0x7f0da80319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:03.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:03.016+0000 7f2405b07700 1 -- 172.21.15.3:0/2644552079 >> v1:172.21.15.3:6816/127025 conn(0x7f23e402f510 legacy=0x7f23e40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:03.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:03.020+0000 7f0dc160b700 1 -- 172.21.15.3:0/3430457611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0dbc06c360 con 0x7f0dbc1c8ed0 2023-10-03T10:13:03.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:03.021+0000 7f0da37fe700 1 -- 172.21.15.3:0/3430457611 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0db4024bd0 con 0x7f0dbc1c8ed0 2023-10-03T10:13:03.107 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:03.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:03.023+0000 7f0dc160b700 1 -- 172.21.15.3:0/3430457611 >> v1:172.21.15.3:6816/127025 conn(0x7f0da802f540 legacy=0x7f0da80319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:03.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:03.023+0000 7f0dc160b700 1 -- 172.21.15.3:0/3430457611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dbc1c8ed0 msgr2=0x7f0dbc130280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:03.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:03.023+0000 7f0dc160b700 1 --2- 172.21.15.3:0/3430457611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dbc1c8ed0 0x7f0dbc130280 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0db400caf0 tx=0x7f0db400ce00 comp rx=0 tx=0).stop 2023-10-03T10:13:03.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:03.023+0000 7f0dc160b700 1 -- 172.21.15.3:0/3430457611 shutdown_connections 2023-10-03T10:13:03.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:03.023+0000 7f0dc160b700 1 --2- 172.21.15.3:0/3430457611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dbc1c8ed0 0x7f0dbc130280 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:03.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:03.023+0000 7f0dc160b700 1 --2- 172.21.15.3:0/3430457611 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dbc06b080 0x7f0dbc1c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:03.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:03.023+0000 7f0dc160b700 1 --2- 172.21.15.3:0/3430457611 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dbc131b40 0x7f0dbc1c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:03.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:03.023+0000 7f0dc160b700 1 -- 172.21.15.3:0/3430457611 >> 172.21.15.3:0/3430457611 conn(0x7f0dbc066f50 msgr2=0x7f0dbc0a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:03.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:03.023+0000 7f0dc160b700 1 -- 172.21.15.3:0/3430457611 shutdown_connections 2023-10-03T10:13:03.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:03.023+0000 7f0dc160b700 1 -- 172.21.15.3:0/3430457611 wait complete. 2023-10-03T10:13:03.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:03.033+0000 7f2407d6b700 1 -- 172.21.15.3:0/2644552079 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f240006b250 con 0x7f2400130d50 2023-10-03T10:13:03.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:03.034+0000 7f23f67fc700 1 -- 172.21.15.3:0/2644552079 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f23f801a970 con 0x7f2400130d50 2023-10-03T10:13:03.109 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:03.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:03.036+0000 7f2407d6b700 1 -- 172.21.15.3:0/2644552079 >> v1:172.21.15.3:6816/127025 conn(0x7f23e402f510 legacy=0x7f23e40319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:03.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:03.036+0000 7f2407d6b700 1 -- 172.21.15.3:0/2644552079 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2400130d50 msgr2=0x7f2400127880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:03.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:03.036+0000 7f2407d6b700 1 --2- 172.21.15.3:0/2644552079 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2400130d50 0x7f2400127880 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f23f800b370 tx=0x7f23f8004ad0 comp rx=0 tx=0).stop 2023-10-03T10:13:03.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:03.036+0000 7f2407d6b700 1 -- 172.21.15.3:0/2644552079 shutdown_connections 2023-10-03T10:13:03.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:03.036+0000 7f2407d6b700 1 --2- 172.21.15.3:0/2644552079 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2400130d50 0x7f2400127880 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:03.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:03.036+0000 7f2407d6b700 1 --2- 172.21.15.3:0/2644552079 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2400071b40 0x7f2400126e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:03.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:03.036+0000 7f2407d6b700 1 --2- 172.21.15.3:0/2644552079 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2400130960 0x7f2400127340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:03.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:03.036+0000 7f2407d6b700 1 -- 172.21.15.3:0/2644552079 >> 172.21.15.3:0/2644552079 conn(0x7f240009e7e0 msgr2=0x7f240009c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:03.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:03.036+0000 7f2407d6b700 1 -- 172.21.15.3:0/2644552079 shutdown_connections 2023-10-03T10:13:03.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:03.036+0000 7f2407d6b700 1 -- 172.21.15.3:0/2644552079 wait complete. 2023-10-03T10:13:03.111 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:03.112 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:04.106 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:04.107 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:04.112 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:04.113 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:04.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.259+0000 7f35235aa700 1 Processor -- start 2023-10-03T10:13:04.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.259+0000 7f35235aa700 1 -- start start 2023-10-03T10:13:04.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.260+0000 7f35235aa700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f351c0a1510 0x7f351c138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:04.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.260+0000 7f35235aa700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f351c139250 0x7f351c13b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:04.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.260+0000 7f35235aa700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f351c09edd0 con 0x7f351c0a0a30 2023-10-03T10:13:04.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.260+0000 7f35235aa700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f351c0a18e0 con 0x7f351c0a1510 2023-10-03T10:13:04.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.260+0000 7f35235aa700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f351c09e1f0 con 0x7f351c139250 2023-10-03T10:13:04.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.260+0000 7f3521b47700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f351c139250 0x7f351c13b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:04.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.260+0000 7f3521b47700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f351c139250 0x7f351c13b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40744/0 (socket says 172.21.15.3:40744) 2023-10-03T10:13:04.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.260+0000 7f3521b47700 1 -- 172.21.15.3:0/6245831 learned_addr learned my addr 172.21.15.3:0/6245831 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:04.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.260+0000 7f3521b47700 1 -- 172.21.15.3:0/6245831 >> v2:172.21.15.3:3300/0 conn(0x7f351c0a1510 msgr2=0x7f351c138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:04.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.260+0000 7f3521b47700 1 --2- 172.21.15.3:0/6245831 >> v2:172.21.15.3:3300/0 conn(0x7f351c0a1510 0x7f351c138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.260+0000 7f3521b47700 1 -- 172.21.15.3:0/6245831 >> v1:172.21.15.3:6789/0 conn(0x7f351c0a0a30 legacy=0x7f351c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:04.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.260+0000 7f3521b47700 1 -- 172.21.15.3:0/6245831 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f351c09dc50 con 0x7f351c139250 2023-10-03T10:13:04.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.260+0000 7f3521b47700 1 --2- 172.21.15.3:0/6245831 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f351c139250 0x7f351c13b630 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3514009e50 tx=0x7f351400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5748ae7f6c4f0f7c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:04.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.261+0000 7f3513fff700 1 -- 172.21.15.3:0/6245831 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351400e070 con 0x7f351c139250 2023-10-03T10:13:04.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.261+0000 7f3513fff700 1 -- 172.21.15.3:0/6245831 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3514003a20 con 0x7f351c139250 2023-10-03T10:13:04.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.261+0000 7f3513fff700 1 -- 172.21.15.3:0/6245831 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3514003dd0 con 0x7f351c139250 2023-10-03T10:13:04.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.261+0000 7f35235aa700 1 -- 172.21.15.3:0/6245831 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f351c139250 msgr2=0x7f351c13b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:04.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.261+0000 7f35235aa700 1 --2- 172.21.15.3:0/6245831 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f351c139250 0x7f351c13b630 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3514009e50 tx=0x7f351400b330 comp rx=0 tx=0).stop 2023-10-03T10:13:04.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.261+0000 7f35235aa700 1 -- 172.21.15.3:0/6245831 shutdown_connections 2023-10-03T10:13:04.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.261+0000 7f35235aa700 1 --2- 172.21.15.3:0/6245831 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f351c139250 0x7f351c13b630 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.261+0000 7f35235aa700 1 --2- 172.21.15.3:0/6245831 >> v2:172.21.15.3:3300/0 conn(0x7f351c0a1510 0x7f351c138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.261+0000 7f35235aa700 1 -- 172.21.15.3:0/6245831 >> 172.21.15.3:0/6245831 conn(0x7f351c0a6550 msgr2=0x7f351c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:04.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.262+0000 7f35235aa700 1 -- 172.21.15.3:0/6245831 shutdown_connections 2023-10-03T10:13:04.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.262+0000 7f35235aa700 1 -- 172.21.15.3:0/6245831 wait complete. 2023-10-03T10:13:04.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.263+0000 7f35235aa700 1 Processor -- start 2023-10-03T10:13:04.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.263+0000 7f35235aa700 1 -- start start 2023-10-03T10:13:04.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.264+0000 7f35235aa700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f351c0a0a30 0x7f351c122510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:04.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.264+0000 7f35235aa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f351c0a1510 0x7f351c122a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:04.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.264+0000 7f35235aa700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f351c139250 0x7f351c1c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:04.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.264+0000 7f35235aa700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351c1251d0 con 0x7f351c0a1510 2023-10-03T10:13:04.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.264+0000 7f35235aa700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f351c0a0220 con 0x7f351c0a0a30 2023-10-03T10:13:04.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.264+0000 7f35235aa700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f351c122050 con 0x7f351c139250 2023-10-03T10:13:04.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.264+0000 7f3520b45700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f351c0a1510 0x7f351c122a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:04.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.264+0000 7f3521346700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f351c0a0a30 0x7f351c122510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:04.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.264+0000 7f3521346700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f351c0a0a30 0x7f351c122510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40752/0 (socket says 172.21.15.3:40752) 2023-10-03T10:13:04.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.264+0000 7f3520b45700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f351c0a1510 0x7f351c122a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48812/0 (socket says 172.21.15.3:48812) 2023-10-03T10:13:04.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.264+0000 7f3521346700 1 -- 172.21.15.3:0/1313183673 learned_addr learned my addr 172.21.15.3:0/1313183673 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:04.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.265+0000 7f3521346700 1 -- 172.21.15.3:0/1313183673 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f351c139250 msgr2=0x7f351c1c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:04.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.265+0000 7f3521346700 1 --2- 172.21.15.3:0/1313183673 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f351c139250 0x7f351c1c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.265+0000 7f3521346700 1 -- 172.21.15.3:0/1313183673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f351c0a1510 msgr2=0x7f351c122a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:04.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.265+0000 7f3521346700 1 --2- 172.21.15.3:0/1313183673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f351c0a1510 0x7f351c122a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.265+0000 7f3521346700 1 -- 172.21.15.3:0/1313183673 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3514009b30 con 0x7f351c0a0a30 2023-10-03T10:13:04.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.265+0000 7f3520b45700 1 --2- 172.21.15.3:0/1313183673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f351c0a1510 0x7f351c122a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:04.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.265+0000 7f3521346700 1 --2- 172.21.15.3:0/1313183673 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f351c0a0a30 0x7f351c122510 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f351800c4a0 tx=0x7f351800c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:04.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.265+0000 7f3511ffb700 1 -- 172.21.15.3:0/1313183673 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35180054e0 con 0x7f351c0a0a30 2023-10-03T10:13:04.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.265+0000 7f3511ffb700 1 -- 172.21.15.3:0/1313183673 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3518003680 con 0x7f351c0a0a30 2023-10-03T10:13:04.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.265+0000 7f35235aa700 1 -- 172.21.15.3:0/1313183673 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f351c124210 con 0x7f351c0a0a30 2023-10-03T10:13:04.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.265+0000 7f3511ffb700 1 -- 172.21.15.3:0/1313183673 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518003a30 con 0x7f351c0a0a30 2023-10-03T10:13:04.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.265+0000 7f35235aa700 1 -- 172.21.15.3:0/1313183673 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f351c125c40 con 0x7f351c0a0a30 2023-10-03T10:13:04.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.267+0000 7f3511ffb700 1 -- 172.21.15.3:0/1313183673 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f3518003b90 con 0x7f351c0a0a30 2023-10-03T10:13:04.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.267+0000 7f35235aa700 1 -- 172.21.15.3:0/1313183673 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f351c125320 con 0x7f351c0a0a30 2023-10-03T10:13:04.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.267+0000 7f3511ffb700 1 -- 172.21.15.3:0/1313183673 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f351803f0a0 con 0x7f351c0a0a30 2023-10-03T10:13:04.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.267+0000 7f3520b45700 1 -- 172.21.15.3:0/1313183673 >> v1:172.21.15.3:6816/127025 conn(0x7f350002f510 legacy=0x7f35000319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:04.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.269+0000 7f725d9cc700 1 Processor -- start 2023-10-03T10:13:04.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.269+0000 7f725d9cc700 1 -- start start 2023-10-03T10:13:04.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.270+0000 7f725d9cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f725809d6f0 0x7f725809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:04.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.270+0000 7f725d9cc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f725809e000 0x7f72581339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:04.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.270+0000 7f725d9cc700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f725809b120 con 0x7f725809e410 2023-10-03T10:13:04.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.270+0000 7f725d9cc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f725806b460 con 0x7f725809e000 2023-10-03T10:13:04.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.270+0000 7f725d9cc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725809a7e0 con 0x7f725809d6f0 2023-10-03T10:13:04.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.270+0000 7f72577fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f725809e410 0x7f7258133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49462/0 (socket says 172.21.15.3:49462) 2023-10-03T10:13:04.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.271+0000 7f724ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f725809e000 0x7f72581339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:04.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.271+0000 7f72577fe700 1 -- 172.21.15.3:0/1168305124 learned_addr learned my addr 172.21.15.3:0/1168305124 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:04.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.271+0000 7f724ffff700 1 -- 172.21.15.3:0/1168305124 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f725809d6f0 msgr2=0x7f725809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:04.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.271+0000 7f724ffff700 1 --2- 172.21.15.3:0/1168305124 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f725809d6f0 0x7f725809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.271+0000 7f724ffff700 1 -- 172.21.15.3:0/1168305124 >> v1:172.21.15.3:6789/0 conn(0x7f725809e410 legacy=0x7f7258133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:04.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.271+0000 7f724ffff700 1 -- 172.21.15.3:0/1168305124 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725809a500 con 0x7f725809e000 2023-10-03T10:13:04.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.271+0000 7f724ffff700 1 --2- 172.21.15.3:0/1168305124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f725809e000 0x7f72581339f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7240009840 tx=0x7f7240009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ccf3b8fff6298e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:04.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.271+0000 7f72567fc700 1 -- 172.21.15.3:0/1168305124 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7240011830 con 0x7f725809e000 2023-10-03T10:13:04.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.271+0000 7f72567fc700 1 -- 172.21.15.3:0/1168305124 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7240011990 con 0x7f725809e000 2023-10-03T10:13:04.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.271+0000 7f72567fc700 1 -- 172.21.15.3:0/1168305124 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7240011d20 con 0x7f725809e000 2023-10-03T10:13:04.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.272+0000 7f725d9cc700 1 -- 172.21.15.3:0/1168305124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f725809e000 msgr2=0x7f72581339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:04.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.272+0000 7f725d9cc700 1 --2- 172.21.15.3:0/1168305124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f725809e000 0x7f72581339f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7240009840 tx=0x7f7240009b50 comp rx=0 tx=0).stop 2023-10-03T10:13:04.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.272+0000 7f725d9cc700 1 -- 172.21.15.3:0/1168305124 shutdown_connections 2023-10-03T10:13:04.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.272+0000 7f725d9cc700 1 --2- 172.21.15.3:0/1168305124 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f725809d6f0 0x7f725809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.272+0000 7f725d9cc700 1 --2- 172.21.15.3:0/1168305124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f725809e000 0x7f72581339f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.272+0000 7f725d9cc700 1 -- 172.21.15.3:0/1168305124 >> 172.21.15.3:0/1168305124 conn(0x7f7258066f50 msgr2=0x7f725809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:04.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.273+0000 7f725d9cc700 1 -- 172.21.15.3:0/1168305124 shutdown_connections 2023-10-03T10:13:04.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.273+0000 7f725d9cc700 1 -- 172.21.15.3:0/1168305124 wait complete. 2023-10-03T10:13:04.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.274+0000 7f725d9cc700 1 Processor -- start 2023-10-03T10:13:04.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.274+0000 7f725d9cc700 1 -- start start 2023-10-03T10:13:04.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.274+0000 7f725d9cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f725809d6f0 0x7f72581abbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:04.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.274+0000 7f725d9cc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f725809e000 0x7f72581ac0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:04.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.274+0000 7f725d9cc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f725809e410 0x7f72581ac630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:04.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.274+0000 7f725d9cc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72581af440 con 0x7f725809e410 2023-10-03T10:13:04.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.274+0000 7f725d9cc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72581ad730 con 0x7f725809d6f0 2023-10-03T10:13:04.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.274+0000 7f725d9cc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f725809c710 con 0x7f725809e000 2023-10-03T10:13:04.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.274+0000 7f724ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f725809e000 0x7f72581ac0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:04.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.275+0000 7f724ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f725809e000 0x7f72581ac0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48768/0 (socket says 172.21.15.3:48768) 2023-10-03T10:13:04.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.275+0000 7f724ffff700 1 -- 172.21.15.3:0/2284179006 learned_addr learned my addr 172.21.15.3:0/2284179006 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:04.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.274+0000 7f3511ffb700 1 -- 172.21.15.3:0/1313183673 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3518003e40 con 0x7f351c0a0a30 2023-10-03T10:13:04.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.275+0000 7f7256ffd700 1 --2- 172.21.15.3:0/2284179006 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f725809d6f0 0x7f72581abbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:04.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.275+0000 7f72577fe700 1 --2- 172.21.15.3:0/2284179006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f725809e410 0x7f72581ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:04.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.275+0000 7f7256ffd700 1 -- 172.21.15.3:0/2284179006 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f725809e000 msgr2=0x7f72581ac0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:04.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.275+0000 7f7256ffd700 1 --2- 172.21.15.3:0/2284179006 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f725809e000 0x7f72581ac0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.275+0000 7f7256ffd700 1 -- 172.21.15.3:0/2284179006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f725809e410 msgr2=0x7f72581ac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:04.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.275+0000 7f7256ffd700 1 --2- 172.21.15.3:0/2284179006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f725809e410 0x7f72581ac630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.275+0000 7f7256ffd700 1 -- 172.21.15.3:0/2284179006 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72400094a0 con 0x7f725809d6f0 2023-10-03T10:13:04.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.275+0000 7f724ffff700 1 --2- 172.21.15.3:0/2284179006 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f725809e000 0x7f72581ac0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:04.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.275+0000 7f7256ffd700 1 --2- 172.21.15.3:0/2284179006 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f725809d6f0 0x7f72581abbb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f724800b450 tx=0x7f724800b760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:04.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.276+0000 7f7254ff9700 1 -- 172.21.15.3:0/2284179006 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7248011830 con 0x7f725809d6f0 2023-10-03T10:13:04.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.276+0000 7f7254ff9700 1 -- 172.21.15.3:0/2284179006 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7248011990 con 0x7f725809d6f0 2023-10-03T10:13:04.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.276+0000 7f7254ff9700 1 -- 172.21.15.3:0/2284179006 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7248011d40 con 0x7f725809d6f0 2023-10-03T10:13:04.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.276+0000 7f725d9cc700 1 -- 172.21.15.3:0/2284179006 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72581af0a0 con 0x7f725809d6f0 2023-10-03T10:13:04.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.276+0000 7f725d9cc700 1 -- 172.21.15.3:0/2284179006 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72581adec0 con 0x7f725809d6f0 2023-10-03T10:13:04.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.277+0000 7f7254ff9700 1 -- 172.21.15.3:0/2284179006 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f724801a400 con 0x7f725809d6f0 2023-10-03T10:13:04.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.277+0000 7f725d9cc700 1 -- 172.21.15.3:0/2284179006 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7258074ea0 con 0x7f725809d6f0 2023-10-03T10:13:04.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.277+0000 7f724ffff700 1 -- 172.21.15.3:0/2284179006 >> v1:172.21.15.3:6816/127025 conn(0x7f723802f510 legacy=0x7f72380319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:04.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.277+0000 7f7254ff9700 1 -- 172.21.15.3:0/2284179006 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f724800fdc0 con 0x7f725809d6f0 2023-10-03T10:13:04.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.283+0000 7f7254ff9700 1 -- 172.21.15.3:0/2284179006 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7248015030 con 0x7f725809d6f0 2023-10-03T10:13:04.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.306+0000 7f785017b700 1 Processor -- start 2023-10-03T10:13:04.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.307+0000 7f785017b700 1 -- start start 2023-10-03T10:13:04.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.308+0000 7f785017b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f784809c5e0 0x7f784809c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:04.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.308+0000 7f785017b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f784809cef0 0x7f784813bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:04.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.308+0000 7f785017b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f784813c470 0x7f784813e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:04.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.308+0000 7f785017b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f784809bff0 con 0x7f784813c470 2023-10-03T10:13:04.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.308+0000 7f785017b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7848098ab0 con 0x7f784809c5e0 2023-10-03T10:13:04.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.308+0000 7f785017b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f784809a830 con 0x7f784809cef0 2023-10-03T10:13:04.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.308+0000 7f784e718700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f784813c470 0x7f784813e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:04.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.308+0000 7f784e718700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f784813c470 0x7f784813e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48830/0 (socket says 172.21.15.3:48830) 2023-10-03T10:13:04.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.308+0000 7f784e718700 1 -- 172.21.15.3:0/2686868414 learned_addr learned my addr 172.21.15.3:0/2686868414 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:04.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.308+0000 7f784d716700 1 --2- 172.21.15.3:0/2686868414 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f784809cef0 0x7f784813bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:04.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.309+0000 7f784e718700 1 -- 172.21.15.3:0/2686868414 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f784809cef0 msgr2=0x7f784813bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:04.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.309+0000 7f784e718700 1 --2- 172.21.15.3:0/2686868414 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f784809cef0 0x7f784813bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.309+0000 7f784e718700 1 -- 172.21.15.3:0/2686868414 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f784809c5e0 msgr2=0x7f784809c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:04.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.309+0000 7f784e718700 1 --2- 172.21.15.3:0/2686868414 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f784809c5e0 0x7f784809c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.309+0000 7f784e718700 1 -- 172.21.15.3:0/2686868414 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f784809c1a0 con 0x7f784813c470 2023-10-03T10:13:04.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.309+0000 7f784e718700 1 --2- 172.21.15.3:0/2686868414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f784813c470 0x7f784813e850 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7840009e80 tx=0x7f784000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e8ac6de380dc3f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:04.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.309+0000 7f784cf15700 1 -- 172.21.15.3:0/2686868414 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784000e070 con 0x7f784813c470 2023-10-03T10:13:04.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.309+0000 7f784cf15700 1 -- 172.21.15.3:0/2686868414 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784000b8d0 con 0x7f784813c470 2023-10-03T10:13:04.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.309+0000 7f784cf15700 1 -- 172.21.15.3:0/2686868414 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784000bc60 con 0x7f784813c470 2023-10-03T10:13:04.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.310+0000 7f0dfcc9b700 1 Processor -- start 2023-10-03T10:13:04.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.310+0000 7f0dfcc9b700 1 -- start start 2023-10-03T10:13:04.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.310+0000 7f785017b700 1 -- 172.21.15.3:0/2686868414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f784813c470 msgr2=0x7f784813e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:04.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.310+0000 7f785017b700 1 --2- 172.21.15.3:0/2686868414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f784813c470 0x7f784813e850 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7840009e80 tx=0x7f784000b330 comp rx=0 tx=0).stop 2023-10-03T10:13:04.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.310+0000 7f0dfcc9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0df8071b40 0x7f0df8130420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:04.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.311+0000 7f785017b700 1 -- 172.21.15.3:0/2686868414 shutdown_connections 2023-10-03T10:13:04.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.311+0000 7f785017b700 1 --2- 172.21.15.3:0/2686868414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f784813c470 0x7f784813e850 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.311+0000 7f785017b700 1 --2- 172.21.15.3:0/2686868414 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f784809c5e0 0x7f784809c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.311+0000 7f785017b700 1 --2- 172.21.15.3:0/2686868414 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f784809cef0 0x7f784813bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.311+0000 7f785017b700 1 -- 172.21.15.3:0/2686868414 >> 172.21.15.3:0/2686868414 conn(0x7f784809e7e0 msgr2=0x7f784809ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:04.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.310+0000 7f0dfcc9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0df8130d80 0x7f0df8133c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:04.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.310+0000 7f0dfcc9b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0df8099ef0 con 0x7f0df8130960 2023-10-03T10:13:04.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.310+0000 7f0dfcc9b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0df8099500 con 0x7f0df8071b40 2023-10-03T10:13:04.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.310+0000 7f0dfcc9b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df8099c10 con 0x7f0df8130d80 2023-10-03T10:13:04.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.311+0000 7f0df5d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0df8130960 0x7f0df8133570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49478/0 (socket says 172.21.15.3:49478) 2023-10-03T10:13:04.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.311+0000 7f0df5d9b700 1 -- 172.21.15.3:0/3944218845 learned_addr learned my addr 172.21.15.3:0/3944218845 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:04.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.311+0000 7f785017b700 1 -- 172.21.15.3:0/2686868414 shutdown_connections 2023-10-03T10:13:04.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.311+0000 7f0df659c700 1 --2- 172.21.15.3:0/3944218845 >> v2:172.21.15.3:3300/0 conn(0x7f0df8071b40 0x7f0df8130420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:04.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.311+0000 7f785017b700 1 -- 172.21.15.3:0/2686868414 wait complete. 2023-10-03T10:13:04.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.311+0000 7f0df559a700 1 -- 172.21.15.3:0/3944218845 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 133999976 0 0) 0x7f0df8099ef0 con 0x7f0df8130960 2023-10-03T10:13:04.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.311+0000 7f0df559a700 1 -- 172.21.15.3:0/3944218845 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0de00035b0 con 0x7f0df8130960 2023-10-03T10:13:04.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.311+0000 7f0df659c700 1 -- 172.21.15.3:0/3944218845 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0df8130d80 msgr2=0x7f0df8133c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:04.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.311+0000 7f0df659c700 1 --2- 172.21.15.3:0/3944218845 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0df8130d80 0x7f0df8133c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.311+0000 7f0df659c700 1 -- 172.21.15.3:0/3944218845 >> v1:172.21.15.3:6789/0 conn(0x7f0df8130960 legacy=0x7f0df8133570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:04.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.311+0000 7f0df659c700 1 -- 172.21.15.3:0/3944218845 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df809e370 con 0x7f0df8071b40 2023-10-03T10:13:04.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.311+0000 7f0df659c700 1 --2- 172.21.15.3:0/3944218845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0df8071b40 0x7f0df8130420 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0de8009840 tx=0x7f0de8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2300eaab968d476 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:04.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.312+0000 7f0df559a700 1 -- 172.21.15.3:0/3944218845 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de8011830 con 0x7f0df8071b40 2023-10-03T10:13:04.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.312+0000 7f0df559a700 1 -- 172.21.15.3:0/3944218845 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de8011990 con 0x7f0df8071b40 2023-10-03T10:13:04.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.312+0000 7f0df559a700 1 -- 172.21.15.3:0/3944218845 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de8011d20 con 0x7f0df8071b40 2023-10-03T10:13:04.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.312+0000 7f785017b700 1 Processor -- start 2023-10-03T10:13:04.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.312+0000 7f785017b700 1 -- start start 2023-10-03T10:13:04.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.313+0000 7f785017b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f784809c5e0 0x7f784813bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:04.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.313+0000 7f785017b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f784809cef0 0x7f7848070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:04.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.313+0000 7f785017b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f784813c470 0x7f7848071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.313+0000 7f785017b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78481284f0 con 0x7f784809c5e0 2023-10-03T10:13:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.313+0000 7f785017b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7848072630 con 0x7f784809cef0 2023-10-03T10:13:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.313+0000 7f785017b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7848128720 con 0x7f784813c470 2023-10-03T10:13:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.312+0000 7f0dfcc9b700 1 -- 172.21.15.3:0/3944218845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0df8071b40 msgr2=0x7f0df8130420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.312+0000 7f0dfcc9b700 1 --2- 172.21.15.3:0/3944218845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0df8071b40 0x7f0df8130420 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0de8009840 tx=0x7f0de8009b50 comp rx=0 tx=0).stop 2023-10-03T10:13:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.313+0000 7f0dfcc9b700 1 -- 172.21.15.3:0/3944218845 shutdown_connections 2023-10-03T10:13:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.313+0000 7f0dfcc9b700 1 --2- 172.21.15.3:0/3944218845 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0df8130d80 0x7f0df8133c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.313+0000 7f0dfcc9b700 1 --2- 172.21.15.3:0/3944218845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0df8071b40 0x7f0df8130420 secure :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0de8009840 tx=0x7f0de8009b50 comp rx=0 tx=0).stop 2023-10-03T10:13:04.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.313+0000 7f0dfcc9b700 1 -- 172.21.15.3:0/3944218845 >> 172.21.15.3:0/3944218845 conn(0x7f0df809e7e0 msgr2=0x7f0df809ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:04.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.313+0000 7f0dfcc9b700 1 -- 172.21.15.3:0/3944218845 shutdown_connections 2023-10-03T10:13:04.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.313+0000 7f0dfcc9b700 1 -- 172.21.15.3:0/3944218845 wait complete. 2023-10-03T10:13:04.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.313+0000 7f784e718700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f784813c470 0x7f7848071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:04.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.313+0000 7f784e718700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f784813c470 0x7f7848071020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48794/0 (socket says 172.21.15.3:48794) 2023-10-03T10:13:04.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.313+0000 7f784e718700 1 -- 172.21.15.3:0/787891720 learned_addr learned my addr 172.21.15.3:0/787891720 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:04.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.313+0000 7f784df17700 1 --2- 172.21.15.3:0/787891720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f784809c5e0 0x7f784813bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:04.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.313+0000 7f784e718700 1 -- 172.21.15.3:0/787891720 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f784809cef0 msgr2=0x7f7848070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:04.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.313+0000 7f784e718700 1 --2- 172.21.15.3:0/787891720 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f784809cef0 0x7f7848070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.313+0000 7f784e718700 1 -- 172.21.15.3:0/787891720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f784809c5e0 msgr2=0x7f784813bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:04.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.313+0000 7f784e718700 1 --2- 172.21.15.3:0/787891720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f784809c5e0 0x7f784813bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.313+0000 7f784e718700 1 -- 172.21.15.3:0/787891720 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7840009b30 con 0x7f784813c470 2023-10-03T10:13:04.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.314+0000 7f784e718700 1 --2- 172.21.15.3:0/787891720 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f784813c470 0x7f7848071020 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f784809b1f0 tx=0x7f7840004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:04.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.314+0000 7f783e7fc700 1 -- 172.21.15.3:0/787891720 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7840004250 con 0x7f784813c470 2023-10-03T10:13:04.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.314+0000 7f783e7fc700 1 -- 172.21.15.3:0/787891720 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78400043b0 con 0x7f784813c470 2023-10-03T10:13:04.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.314+0000 7f783e7fc700 1 -- 172.21.15.3:0/787891720 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7840004740 con 0x7f784813c470 2023-10-03T10:13:04.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.314+0000 7f785017b700 1 -- 172.21.15.3:0/787891720 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7848127760 con 0x7f784813c470 2023-10-03T10:13:04.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.314+0000 7f785017b700 1 -- 172.21.15.3:0/787891720 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7848129190 con 0x7f784813c470 2023-10-03T10:13:04.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.315+0000 7f0dfcc9b700 1 Processor -- start 2023-10-03T10:13:04.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.315+0000 7f0dfcc9b700 1 -- start start 2023-10-03T10:13:04.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.315+0000 7f0dfcc9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0df8130960 0x7f0df81398e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:04.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.315+0000 7f0dfcc9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0df8098480 0x7f0df8098870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:04.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.315+0000 7f0dfcc9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0df8139f30 0x7f0df813a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:04.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.315+0000 7f0dfcc9b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df8139070 con 0x7f0df8139f30 2023-10-03T10:13:04.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.315+0000 7f0dfcc9b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df809c150 con 0x7f0df8098480 2023-10-03T10:13:04.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.315+0000 7f0dfcc9b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0df8135b90 con 0x7f0df8130960 2023-10-03T10:13:04.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.315+0000 7f0df659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0df8130960 0x7f0df81398e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:04.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.315+0000 7f0df659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0df8130960 0x7f0df81398e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48802/0 (socket says 172.21.15.3:48802) 2023-10-03T10:13:04.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.315+0000 7f0df659c700 1 -- 172.21.15.3:0/4068857821 learned_addr learned my addr 172.21.15.3:0/4068857821 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:04.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.315+0000 7f0df6d9d700 1 --2- 172.21.15.3:0/4068857821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0df8139f30 0x7f0df813a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:04.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.316+0000 7f0df659c700 1 -- 172.21.15.3:0/4068857821 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0df8098480 msgr2=0x7f0df8098870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:04.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.316+0000 7f0df659c700 1 --2- 172.21.15.3:0/4068857821 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0df8098480 0x7f0df8098870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.316+0000 7f0df659c700 1 -- 172.21.15.3:0/4068857821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0df8139f30 msgr2=0x7f0df813a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:04.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.316+0000 7f0df659c700 1 --2- 172.21.15.3:0/4068857821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0df8139f30 0x7f0df813a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.316+0000 7f0df659c700 1 -- 172.21.15.3:0/4068857821 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de80094a0 con 0x7f0df8130960 2023-10-03T10:13:04.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.316+0000 7f0df659c700 1 --2- 172.21.15.3:0/4068857821 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0df8130960 0x7f0df81398e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0de8009fd0 tx=0x7f0de8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:04.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.316+0000 7f0de6ffd700 1 -- 172.21.15.3:0/4068857821 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de80036a0 con 0x7f0df8130960 2023-10-03T10:13:04.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.316+0000 7f0de6ffd700 1 -- 172.21.15.3:0/4068857821 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de801f070 con 0x7f0df8130960 2023-10-03T10:13:04.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.316+0000 7f0dfcc9b700 1 -- 172.21.15.3:0/4068857821 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0df8138db0 con 0x7f0df8130960 2023-10-03T10:13:04.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.316+0000 7f783e7fc700 1 -- 172.21.15.3:0/787891720 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f784001c440 con 0x7f784813c470 2023-10-03T10:13:04.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.316+0000 7f0de6ffd700 1 -- 172.21.15.3:0/4068857821 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de8003a60 con 0x7f0df8130960 2023-10-03T10:13:04.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.317+0000 7f783e7fc700 1 -- 172.21.15.3:0/787891720 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f7840041240 con 0x7f784813c470 2023-10-03T10:13:04.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.317+0000 7f784df17700 1 -- 172.21.15.3:0/787891720 >> v1:172.21.15.3:6816/127025 conn(0x7f782c02f560 legacy=0x7f782c031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:04.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.316+0000 7f0dfcc9b700 1 -- 172.21.15.3:0/4068857821 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0df8098db0 con 0x7f0df8130960 2023-10-03T10:13:04.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.318+0000 7f0de6ffd700 1 -- 172.21.15.3:0/4068857821 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f0de8003bc0 con 0x7f0df8130960 2023-10-03T10:13:04.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.318+0000 7f0dfcc9b700 1 -- 172.21.15.3:0/4068857821 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0df81391c0 con 0x7f0df8130960 2023-10-03T10:13:04.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.318+0000 7f0df5d9b700 1 -- 172.21.15.3:0/4068857821 >> v1:172.21.15.3:6816/127025 conn(0x7f0dd802f510 legacy=0x7f0dd80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:04.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.318+0000 7f0de6ffd700 1 -- 172.21.15.3:0/4068857821 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f0de8021070 con 0x7f0df8130960 2023-10-03T10:13:04.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.319+0000 7f785017b700 1 -- 172.21.15.3:0/787891720 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78300052e0 con 0x7f784813c470 2023-10-03T10:13:04.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.323+0000 7f0de6ffd700 1 -- 172.21.15.3:0/4068857821 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0de8046060 con 0x7f0df8130960 2023-10-03T10:13:04.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.331+0000 7f783e7fc700 1 -- 172.21.15.3:0/787891720 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7840020080 con 0x7f784813c470 2023-10-03T10:13:04.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.468+0000 7f3520b45700 1 -- 172.21.15.3:0/1313183673 >> v1:172.21.15.3:6816/127025 conn(0x7f350002f510 legacy=0x7f35000319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:04.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.478+0000 7f724ffff700 1 -- 172.21.15.3:0/2284179006 >> v1:172.21.15.3:6816/127025 conn(0x7f723802f510 legacy=0x7f72380319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:04.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.491+0000 7f35235aa700 1 -- 172.21.15.3:0/1313183673 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f351c074ea0 con 0x7f351c0a0a30 2023-10-03T10:13:04.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.492+0000 7f3511ffb700 1 -- 172.21.15.3:0/1313183673 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3518017020 con 0x7f351c0a0a30 2023-10-03T10:13:04.492 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:04.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.494+0000 7f35235aa700 1 -- 172.21.15.3:0/1313183673 >> v1:172.21.15.3:6816/127025 conn(0x7f350002f510 legacy=0x7f35000319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:04.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.494+0000 7f35235aa700 1 -- 172.21.15.3:0/1313183673 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f351c0a0a30 msgr2=0x7f351c122510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:04.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.494+0000 7f35235aa700 1 --2- 172.21.15.3:0/1313183673 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f351c0a0a30 0x7f351c122510 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f351800c4a0 tx=0x7f351800c7b0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.494+0000 7f35235aa700 1 -- 172.21.15.3:0/1313183673 shutdown_connections 2023-10-03T10:13:04.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.494+0000 7f35235aa700 1 --2- 172.21.15.3:0/1313183673 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f351c139250 0x7f351c1c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.494+0000 7f35235aa700 1 --2- 172.21.15.3:0/1313183673 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f351c0a0a30 0x7f351c122510 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.494+0000 7f35235aa700 1 --2- 172.21.15.3:0/1313183673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f351c0a1510 0x7f351c122a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.494+0000 7f35235aa700 1 -- 172.21.15.3:0/1313183673 >> 172.21.15.3:0/1313183673 conn(0x7f351c0a6550 msgr2=0x7f351c137b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:04.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.494+0000 7f35235aa700 1 -- 172.21.15.3:0/1313183673 shutdown_connections 2023-10-03T10:13:04.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.494+0000 7f35235aa700 1 -- 172.21.15.3:0/1313183673 wait complete. 2023-10-03T10:13:04.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.505+0000 7f725d9cc700 1 -- 172.21.15.3:0/2284179006 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f72581af290 con 0x7f725809d6f0 2023-10-03T10:13:04.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.506+0000 7f7254ff9700 1 -- 172.21.15.3:0/2284179006 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7248015030 con 0x7f725809d6f0 2023-10-03T10:13:04.506 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:04.509 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:04.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.508+0000 7f725d9cc700 1 -- 172.21.15.3:0/2284179006 >> v1:172.21.15.3:6816/127025 conn(0x7f723802f510 legacy=0x7f72380319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:04.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.508+0000 7f725d9cc700 1 -- 172.21.15.3:0/2284179006 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f725809d6f0 msgr2=0x7f72581abbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:04.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.508+0000 7f725d9cc700 1 --2- 172.21.15.3:0/2284179006 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f725809d6f0 0x7f72581abbb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f724800b450 tx=0x7f724800b760 comp rx=0 tx=0).stop 2023-10-03T10:13:04.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.508+0000 7f725d9cc700 1 -- 172.21.15.3:0/2284179006 shutdown_connections 2023-10-03T10:13:04.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.508+0000 7f725d9cc700 1 --2- 172.21.15.3:0/2284179006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f725809e410 0x7f72581ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.508+0000 7f725d9cc700 1 --2- 172.21.15.3:0/2284179006 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f725809d6f0 0x7f72581abbb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.508+0000 7f725d9cc700 1 --2- 172.21.15.3:0/2284179006 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f725809e000 0x7f72581ac0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.508+0000 7f725d9cc700 1 -- 172.21.15.3:0/2284179006 >> 172.21.15.3:0/2284179006 conn(0x7f7258066f50 msgr2=0x7f72580712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:04.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.508+0000 7f725d9cc700 1 -- 172.21.15.3:0/2284179006 shutdown_connections 2023-10-03T10:13:04.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.508+0000 7f725d9cc700 1 -- 172.21.15.3:0/2284179006 wait complete. 2023-10-03T10:13:04.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.518+0000 7f784df17700 1 -- 172.21.15.3:0/787891720 >> v1:172.21.15.3:6816/127025 conn(0x7f782c02f560 legacy=0x7f782c031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:04.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.519+0000 7f0df5d9b700 1 -- 172.21.15.3:0/4068857821 >> v1:172.21.15.3:6816/127025 conn(0x7f0dd802f510 legacy=0x7f0dd80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:04.523 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:04.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.535+0000 7f0dfcc9b700 1 -- 172.21.15.3:0/4068857821 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0df8139430 con 0x7f0df8130960 2023-10-03T10:13:04.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.536+0000 7f0de6ffd700 1 -- 172.21.15.3:0/4068857821 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0de8046020 con 0x7f0df8130960 2023-10-03T10:13:04.536 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:04.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.538+0000 7f0dfcc9b700 1 -- 172.21.15.3:0/4068857821 >> v1:172.21.15.3:6816/127025 conn(0x7f0dd802f510 legacy=0x7f0dd80319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:04.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.538+0000 7f0dfcc9b700 1 -- 172.21.15.3:0/4068857821 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0df8130960 msgr2=0x7f0df81398e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:04.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.538+0000 7f0dfcc9b700 1 --2- 172.21.15.3:0/4068857821 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0df8130960 0x7f0df81398e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0de8009fd0 tx=0x7f0de8004750 comp rx=0 tx=0).stop 2023-10-03T10:13:04.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.538+0000 7f0dfcc9b700 1 -- 172.21.15.3:0/4068857821 shutdown_connections 2023-10-03T10:13:04.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.538+0000 7f0dfcc9b700 1 --2- 172.21.15.3:0/4068857821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0df8139f30 0x7f0df813a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.538+0000 7f0dfcc9b700 1 --2- 172.21.15.3:0/4068857821 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0df8130960 0x7f0df81398e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.538+0000 7f0dfcc9b700 1 --2- 172.21.15.3:0/4068857821 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0df8098480 0x7f0df8098870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.538+0000 7f0dfcc9b700 1 -- 172.21.15.3:0/4068857821 >> 172.21.15.3:0/4068857821 conn(0x7f0df809e7e0 msgr2=0x7f0df8133480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:04.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.538+0000 7f0dfcc9b700 1 -- 172.21.15.3:0/4068857821 shutdown_connections 2023-10-03T10:13:04.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.538+0000 7f0dfcc9b700 1 -- 172.21.15.3:0/4068857821 wait complete. 2023-10-03T10:13:04.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.541+0000 7f785017b700 1 -- 172.21.15.3:0/787891720 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7830005860 con 0x7f784813c470 2023-10-03T10:13:04.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.541+0000 7f783e7fc700 1 -- 172.21.15.3:0/787891720 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7840019350 con 0x7f784813c470 2023-10-03T10:13:04.541 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:04.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.543+0000 7f785017b700 1 -- 172.21.15.3:0/787891720 >> v1:172.21.15.3:6816/127025 conn(0x7f782c02f560 legacy=0x7f782c031a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:04.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.543+0000 7f785017b700 1 -- 172.21.15.3:0/787891720 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f784813c470 msgr2=0x7f7848071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:04.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.543+0000 7f785017b700 1 --2- 172.21.15.3:0/787891720 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f784813c470 0x7f7848071020 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f784809b1f0 tx=0x7f7840004ad0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.543+0000 7f785017b700 1 -- 172.21.15.3:0/787891720 shutdown_connections 2023-10-03T10:13:04.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.543+0000 7f785017b700 1 --2- 172.21.15.3:0/787891720 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f784813c470 0x7f7848071020 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.543+0000 7f785017b700 1 --2- 172.21.15.3:0/787891720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f784809c5e0 0x7f784813bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.543+0000 7f785017b700 1 --2- 172.21.15.3:0/787891720 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f784809cef0 0x7f7848070ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:04.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.543+0000 7f785017b700 1 -- 172.21.15.3:0/787891720 >> 172.21.15.3:0/787891720 conn(0x7f784809e7e0 msgr2=0x7f78480a0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:04.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.544+0000 7f785017b700 1 -- 172.21.15.3:0/787891720 shutdown_connections 2023-10-03T10:13:04.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:04.544+0000 7f785017b700 1 -- 172.21.15.3:0/787891720 wait complete. 2023-10-03T10:13:04.555 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:04.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:05.509 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:05.523 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:05.556 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:05.559 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:05.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.673+0000 7fb4549fc700 1 Processor -- start 2023-10-03T10:13:05.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.674+0000 7fb4549fc700 1 -- start start 2023-10-03T10:13:05.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.675+0000 7fb4549fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb45009d6f0 0x7fb45009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:05.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.675+0000 7fb4549fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb45009e000 0x7fb4501339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:05.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.675+0000 7fb4549fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb45009e410 0x7fb450133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:05.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.675+0000 7fb4549fc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb45006b460 con 0x7fb45009d6f0 2023-10-03T10:13:05.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.675+0000 7fb4549fc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb45009b120 con 0x7fb45009e000 2023-10-03T10:13:05.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.675+0000 7fb4549fc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb45009a7e0 con 0x7fb45009e410 2023-10-03T10:13:05.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.675+0000 7fb44e59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb45009d6f0 0x7fb45009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.675+0000 7fb44ed9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb45009e410 0x7fb450133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.675+0000 7fb44e59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb45009d6f0 0x7fb45009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48870/0 (socket says 172.21.15.3:48870) 2023-10-03T10:13:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.675+0000 7fb44e59c700 1 -- 172.21.15.3:0/1476658708 learned_addr learned my addr 172.21.15.3:0/1476658708 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.675+0000 7fb44dd9b700 1 --2- 172.21.15.3:0/1476658708 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb45009e000 0x7fb4501339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.675+0000 7fb44e59c700 1 -- 172.21.15.3:0/1476658708 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb45009e410 msgr2=0x7fb450133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.675+0000 7fb44e59c700 1 --2- 172.21.15.3:0/1476658708 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb45009e410 0x7fb450133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.675+0000 7fb44e59c700 1 -- 172.21.15.3:0/1476658708 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb45009e000 msgr2=0x7fb4501339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.675+0000 7fb44e59c700 1 --2- 172.21.15.3:0/1476658708 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb45009e000 0x7fb4501339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.676+0000 7fb44e59c700 1 -- 172.21.15.3:0/1476658708 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45009a500 con 0x7fb45009d6f0 2023-10-03T10:13:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.676+0000 7fb44ed9d700 1 --2- 172.21.15.3:0/1476658708 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb45009e410 0x7fb450133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:05.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.676+0000 7fb44dd9b700 1 --2- 172.21.15.3:0/1476658708 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb45009e000 0x7fb4501339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:05.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.676+0000 7fb44e59c700 1 --2- 172.21.15.3:0/1476658708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45009d6f0 0x7fb45009dac0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb4400098f0 tx=0x7fb440009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b94f6409347515 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:05.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.676+0000 7fb44d59a700 1 -- 172.21.15.3:0/1476658708 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb440011830 con 0x7fb45009d6f0 2023-10-03T10:13:05.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.676+0000 7fb44d59a700 1 -- 172.21.15.3:0/1476658708 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb440011990 con 0x7fb45009d6f0 2023-10-03T10:13:05.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.676+0000 7fb44d59a700 1 -- 172.21.15.3:0/1476658708 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb440011d20 con 0x7fb45009d6f0 2023-10-03T10:13:05.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.677+0000 7fb4549fc700 1 -- 172.21.15.3:0/1476658708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45009d6f0 msgr2=0x7fb45009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:05.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.677+0000 7fb4549fc700 1 --2- 172.21.15.3:0/1476658708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45009d6f0 0x7fb45009dac0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb4400098f0 tx=0x7fb440009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:05.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.677+0000 7fb4549fc700 1 -- 172.21.15.3:0/1476658708 shutdown_connections 2023-10-03T10:13:05.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.677+0000 7fb4549fc700 1 --2- 172.21.15.3:0/1476658708 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb45009e410 0x7fb450133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.677+0000 7fb4549fc700 1 --2- 172.21.15.3:0/1476658708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45009d6f0 0x7fb45009dac0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.677+0000 7fb4549fc700 1 --2- 172.21.15.3:0/1476658708 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb45009e000 0x7fb4501339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.677+0000 7fb4549fc700 1 -- 172.21.15.3:0/1476658708 >> 172.21.15.3:0/1476658708 conn(0x7fb450066f50 msgr2=0x7fb45009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:05.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.677+0000 7fb4549fc700 1 -- 172.21.15.3:0/1476658708 shutdown_connections 2023-10-03T10:13:05.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.677+0000 7fb4549fc700 1 -- 172.21.15.3:0/1476658708 wait complete. 2023-10-03T10:13:05.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.679+0000 7fb4549fc700 1 Processor -- start 2023-10-03T10:13:05.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.679+0000 7fb4549fc700 1 -- start start 2023-10-03T10:13:05.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.680+0000 7fb4549fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45009d6f0 0x7fb4501c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:05.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.680+0000 7fb4549fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb45009e000 0x7fb4501c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:05.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.680+0000 7fb4549fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb45009e410 0x7fb4501c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:05.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.680+0000 7fb4549fc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4501a7190 con 0x7fb45009d6f0 2023-10-03T10:13:05.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.680+0000 7fb4549fc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4501b9320 con 0x7fb45009e000 2023-10-03T10:13:05.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.680+0000 7fb4549fc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb4501a6520 con 0x7fb45009e410 2023-10-03T10:13:05.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.680+0000 7fb44e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45009d6f0 0x7fb4501c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:05.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.680+0000 7fb44e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45009d6f0 0x7fb4501c80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48878/0 (socket says 172.21.15.3:48878) 2023-10-03T10:13:05.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.680+0000 7fb44e59c700 1 -- 172.21.15.3:0/323398835 learned_addr learned my addr 172.21.15.3:0/323398835 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:05.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.680+0000 7fb44dd9b700 1 --2- 172.21.15.3:0/323398835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb45009e000 0x7fb4501c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:05.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.680+0000 7fb44ed9d700 1 --2- 172.21.15.3:0/323398835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb45009e410 0x7fb4501c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:05.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.681+0000 7fb44e59c700 1 -- 172.21.15.3:0/323398835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb45009e410 msgr2=0x7fb4501c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:05.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.681+0000 7fb44e59c700 1 --2- 172.21.15.3:0/323398835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb45009e410 0x7fb4501c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.681+0000 7fb44e59c700 1 -- 172.21.15.3:0/323398835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb45009e000 msgr2=0x7fb4501c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:05.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.681+0000 7fb44e59c700 1 --2- 172.21.15.3:0/323398835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb45009e000 0x7fb4501c85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.681+0000 7fb44e59c700 1 -- 172.21.15.3:0/323398835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4440094a0 con 0x7fb45009d6f0 2023-10-03T10:13:05.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.681+0000 7fb44e59c700 1 --2- 172.21.15.3:0/323398835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45009d6f0 0x7fb4501c80b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb440011db0 tx=0x7fb440004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:05.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.681+0000 7fb44dd9b700 1 --2- 172.21.15.3:0/323398835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb45009e000 0x7fb4501c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:05.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.681+0000 7fb43effd700 1 -- 172.21.15.3:0/323398835 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb440004240 con 0x7fb45009d6f0 2023-10-03T10:13:05.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.681+0000 7fb43effd700 1 -- 172.21.15.3:0/323398835 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44001f070 con 0x7fb45009d6f0 2023-10-03T10:13:05.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.681+0000 7fb4549fc700 1 -- 172.21.15.3:0/323398835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb448009a70 con 0x7fb45009d6f0 2023-10-03T10:13:05.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.681+0000 7fb43effd700 1 -- 172.21.15.3:0/323398835 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4400045a0 con 0x7fb45009d6f0 2023-10-03T10:13:05.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.681+0000 7fb4549fc700 1 -- 172.21.15.3:0/323398835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb440009550 con 0x7fb45009d6f0 2023-10-03T10:13:05.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.682+0000 7fb43effd700 1 -- 172.21.15.3:0/323398835 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fb440021030 con 0x7fb45009d6f0 2023-10-03T10:13:05.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.682+0000 7fb4549fc700 1 -- 172.21.15.3:0/323398835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb450074ea0 con 0x7fb45009d6f0 2023-10-03T10:13:05.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.683+0000 7fb44dd9b700 1 -- 172.21.15.3:0/323398835 >> v1:172.21.15.3:6816/127025 conn(0x7fb43002f510 legacy=0x7fb4300319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:05.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.683+0000 7fb43effd700 1 -- 172.21.15.3:0/323398835 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7fb440040cd0 con 0x7fb45009d6f0 2023-10-03T10:13:05.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.688+0000 7fb43effd700 1 -- 172.21.15.3:0/323398835 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb4400199a0 con 0x7fb45009d6f0 2023-10-03T10:13:05.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.709+0000 7f8dbd674700 1 Processor -- start 2023-10-03T10:13:05.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.709+0000 7f8dbd674700 1 -- start start 2023-10-03T10:13:05.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.710+0000 7f8dbd674700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8db8066410 0x7f8db806e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:05.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.710+0000 7f8dbd674700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8db806eb80 0x7f8db806ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:05.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.710+0000 7f8dbd674700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8db8099b50 con 0x7f8db812f320 2023-10-03T10:13:05.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.710+0000 7f8dbd674700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db80667e0 con 0x7f8db8066410 2023-10-03T10:13:05.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.710+0000 7f8dbd674700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db8099f60 con 0x7f8db806eb80 2023-10-03T10:13:05.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.710+0000 7f8db6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8db8066410 0x7f8db806e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:05.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.710+0000 7f8db6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8db8066410 0x7f8db806e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48880/0 (socket says 172.21.15.3:48880) 2023-10-03T10:13:05.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.710+0000 7f8db67fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8db806eb80 0x7f8db806ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:05.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.710+0000 7f8db6ffd700 1 -- 172.21.15.3:0/224005930 learned_addr learned my addr 172.21.15.3:0/224005930 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:05.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.711+0000 7f8db67fc700 1 -- 172.21.15.3:0/224005930 >> v2:172.21.15.3:3300/0 conn(0x7f8db8066410 msgr2=0x7f8db806e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:05.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.711+0000 7f8db67fc700 1 --2- 172.21.15.3:0/224005930 >> v2:172.21.15.3:3300/0 conn(0x7f8db8066410 0x7f8db806e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.711+0000 7f8db67fc700 1 -- 172.21.15.3:0/224005930 >> v1:172.21.15.3:6789/0 conn(0x7f8db812f320 legacy=0x7f8db812f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:05.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.711+0000 7f8db67fc700 1 -- 172.21.15.3:0/224005930 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db809a920 con 0x7f8db806eb80 2023-10-03T10:13:05.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.711+0000 7f8db6ffd700 1 --2- 172.21.15.3:0/224005930 >> v2:172.21.15.3:3300/0 conn(0x7f8db8066410 0x7f8db806e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:05.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.711+0000 7f8db67fc700 1 --2- 172.21.15.3:0/224005930 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8db806eb80 0x7f8db806ef70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8dac009840 tx=0x7f8dac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca620260a78f717a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:05.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.711+0000 7f8db5ffb700 1 -- 172.21.15.3:0/224005930 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac010a90 con 0x7f8db806eb80 2023-10-03T10:13:05.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.711+0000 7f8db5ffb700 1 -- 172.21.15.3:0/224005930 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dac010bf0 con 0x7f8db806eb80 2023-10-03T10:13:05.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.711+0000 7f8db5ffb700 1 -- 172.21.15.3:0/224005930 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac018440 con 0x7f8db806eb80 2023-10-03T10:13:05.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.712+0000 7f8dbd674700 1 -- 172.21.15.3:0/224005930 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8db806eb80 msgr2=0x7f8db806ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:05.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.712+0000 7f8dbd674700 1 --2- 172.21.15.3:0/224005930 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8db806eb80 0x7f8db806ef70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8dac009840 tx=0x7f8dac009b50 comp rx=0 tx=0).stop 2023-10-03T10:13:05.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.712+0000 7f8dbd674700 1 -- 172.21.15.3:0/224005930 shutdown_connections 2023-10-03T10:13:05.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.712+0000 7f8dbd674700 1 --2- 172.21.15.3:0/224005930 >> v2:172.21.15.3:3300/0 conn(0x7f8db8066410 0x7f8db806e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.712+0000 7f8dbd674700 1 --2- 172.21.15.3:0/224005930 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8db806eb80 0x7f8db806ef70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.712+0000 7f8dbd674700 1 -- 172.21.15.3:0/224005930 >> 172.21.15.3:0/224005930 conn(0x7f8db80a2aa0 msgr2=0x7f8db80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:05.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.712+0000 7f8dbd674700 1 -- 172.21.15.3:0/224005930 shutdown_connections 2023-10-03T10:13:05.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.712+0000 7f8dbd674700 1 -- 172.21.15.3:0/224005930 wait complete. 2023-10-03T10:13:05.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.714+0000 7f8dbd674700 1 Processor -- start 2023-10-03T10:13:05.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.714+0000 7f8dbd674700 1 -- start start 2023-10-03T10:13:05.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.714+0000 7f8dbd674700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8db8066410 0x7f8db8118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:05.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.714+0000 7f8dbd674700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8db806eb80 0x7f8db8119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:05.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.714+0000 7f8dbd674700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8db812f320 0x7f8db8119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:05.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.714+0000 7f8dbd674700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db811a2b0 con 0x7f8db8066410 2023-10-03T10:13:05.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.714+0000 7f8dbd674700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db811b500 con 0x7f8db806eb80 2023-10-03T10:13:05.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.714+0000 7f8dbd674700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8db811bf80 con 0x7f8db812f320 2023-10-03T10:13:05.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.715+0000 7f8db67fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8db806eb80 0x7f8db8119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:05.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.715+0000 7f8db67fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8db806eb80 0x7f8db8119210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40824/0 (socket says 172.21.15.3:40824) 2023-10-03T10:13:05.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.715+0000 7f8db67fc700 1 -- 172.21.15.3:0/2777641979 learned_addr learned my addr 172.21.15.3:0/2777641979 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:05.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.715+0000 7f8db77fe700 1 --2- 172.21.15.3:0/2777641979 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8db812f320 0x7f8db8119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:05.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.715+0000 7f8db6ffd700 1 --2- 172.21.15.3:0/2777641979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8db8066410 0x7f8db8118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:05.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.715+0000 7f8db77fe700 1 -- 172.21.15.3:0/2777641979 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8db806eb80 msgr2=0x7f8db8119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:05.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.715+0000 7f8db77fe700 1 --2- 172.21.15.3:0/2777641979 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8db806eb80 0x7f8db8119210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.715+0000 7f8db77fe700 1 -- 172.21.15.3:0/2777641979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8db8066410 msgr2=0x7f8db8118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:05.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.715+0000 7f8db77fe700 1 --2- 172.21.15.3:0/2777641979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8db8066410 0x7f8db8118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.715+0000 7f8db77fe700 1 -- 172.21.15.3:0/2777641979 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da8009550 con 0x7f8db812f320 2023-10-03T10:13:05.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.715+0000 7f8db67fc700 1 --2- 172.21.15.3:0/2777641979 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8db806eb80 0x7f8db8119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:05.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.716+0000 7f8db77fe700 1 --2- 172.21.15.3:0/2777641979 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8db812f320 0x7f8db8119750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8db000cb90 tx=0x7f8db000cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:05.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.716+0000 7f8d9f7fe700 1 -- 172.21.15.3:0/2777641979 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db0003c00 con 0x7f8db812f320 2023-10-03T10:13:05.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.716+0000 7f8d9f7fe700 1 -- 172.21.15.3:0/2777641979 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db0003d60 con 0x7f8db812f320 2023-10-03T10:13:05.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.716+0000 7f8dbd674700 1 -- 172.21.15.3:0/2777641979 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dac0094a0 con 0x7f8db812f320 2023-10-03T10:13:05.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.716+0000 7f8d9f7fe700 1 -- 172.21.15.3:0/2777641979 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db000aa80 con 0x7f8db812f320 2023-10-03T10:13:05.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.716+0000 7f8dbd674700 1 -- 172.21.15.3:0/2777641979 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db809fb50 con 0x7f8db812f320 2023-10-03T10:13:05.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.717+0000 7f8dbd674700 1 -- 172.21.15.3:0/2777641979 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db809a1e0 con 0x7f8db812f320 2023-10-03T10:13:05.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.718+0000 7f8d9f7fe700 1 -- 172.21.15.3:0/2777641979 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f8db000abe0 con 0x7f8db812f320 2023-10-03T10:13:05.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.718+0000 7f8d9f7fe700 1 -- 172.21.15.3:0/2777641979 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f8db0040770 con 0x7f8db812f320 2023-10-03T10:13:05.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.718+0000 7f8db6ffd700 1 -- 172.21.15.3:0/2777641979 >> v1:172.21.15.3:6816/127025 conn(0x7f8da402f510 legacy=0x7f8da40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:05.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.726+0000 7f8d9f7fe700 1 -- 172.21.15.3:0/2777641979 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8db0005090 con 0x7f8db812f320 2023-10-03T10:13:05.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.750+0000 7f121fd63700 1 Processor -- start 2023-10-03T10:13:05.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.751+0000 7f121fd63700 1 -- start start 2023-10-03T10:13:05.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.751+0000 7f121fd63700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1218133a80 0x7f1218133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:05.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.751+0000 7f121fd63700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f121812ff20 0x7f1218130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:05.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.751+0000 7f121fd63700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1218130870 0x7f121813a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:05.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.751+0000 7f121fd63700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f121806c570 con 0x7f1218133a80 2023-10-03T10:13:05.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.751+0000 7f121fd63700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f121806bc00 con 0x7f121812ff20 2023-10-03T10:13:05.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.751+0000 7f121fd63700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12180a0040 con 0x7f1218130870 2023-10-03T10:13:05.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.751+0000 7f121d2fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f121812ff20 0x7f1218130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:05.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.751+0000 7f121daff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1218133a80 0x7f1218133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:05.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.751+0000 7f121d2fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f121812ff20 0x7f1218130330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40840/0 (socket says 172.21.15.3:40840) 2023-10-03T10:13:05.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.751+0000 7f121d2fe700 1 -- 172.21.15.3:0/3041101881 learned_addr learned my addr 172.21.15.3:0/3041101881 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:05.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.752+0000 7f121e300700 1 --2- 172.21.15.3:0/3041101881 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1218130870 0x7f121813a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:05.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.752+0000 7f121d2fe700 1 -- 172.21.15.3:0/3041101881 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1218130870 msgr2=0x7f121813a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:05.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.752+0000 7f121d2fe700 1 --2- 172.21.15.3:0/3041101881 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1218130870 0x7f121813a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.752+0000 7f121d2fe700 1 -- 172.21.15.3:0/3041101881 >> v2:172.21.15.3:3300/0 conn(0x7f1218133a80 msgr2=0x7f1218133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:05.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.752+0000 7f121d2fe700 1 --2- 172.21.15.3:0/3041101881 >> v2:172.21.15.3:3300/0 conn(0x7f1218133a80 0x7f1218133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.752+0000 7f121d2fe700 1 -- 172.21.15.3:0/3041101881 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1218133860 con 0x7f121812ff20 2023-10-03T10:13:05.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.752+0000 7f121e300700 1 --2- 172.21.15.3:0/3041101881 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1218130870 0x7f121813a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:13:05.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.752+0000 7f121d2fe700 1 --2- 172.21.15.3:0/3041101881 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f121812ff20 0x7f1218130330 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f12080098f0 tx=0x7f1208009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7650a0d5f59dc177 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:05.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.752+0000 7f121cafd700 1 -- 172.21.15.3:0/3041101881 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1208011830 con 0x7f121812ff20 2023-10-03T10:13:05.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.752+0000 7f121cafd700 1 -- 172.21.15.3:0/3041101881 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1208011990 con 0x7f121812ff20 2023-10-03T10:13:05.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.752+0000 7f121cafd700 1 -- 172.21.15.3:0/3041101881 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1208011d20 con 0x7f121812ff20 2023-10-03T10:13:05.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.753+0000 7f121fd63700 1 -- 172.21.15.3:0/3041101881 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f121812ff20 msgr2=0x7f1218130330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:05.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.753+0000 7f121fd63700 1 --2- 172.21.15.3:0/3041101881 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f121812ff20 0x7f1218130330 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f12080098f0 tx=0x7f1208009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:05.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.753+0000 7f121fd63700 1 -- 172.21.15.3:0/3041101881 shutdown_connections 2023-10-03T10:13:05.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.753+0000 7f121fd63700 1 --2- 172.21.15.3:0/3041101881 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1218130870 0x7f121813a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.753+0000 7f121fd63700 1 --2- 172.21.15.3:0/3041101881 >> v2:172.21.15.3:3300/0 conn(0x7f1218133a80 0x7f1218133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.753+0000 7f121fd63700 1 --2- 172.21.15.3:0/3041101881 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f121812ff20 0x7f1218130330 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.753+0000 7f121fd63700 1 -- 172.21.15.3:0/3041101881 >> 172.21.15.3:0/3041101881 conn(0x7f12180a4210 msgr2=0x7f12180a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:05.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.753+0000 7f121fd63700 1 -- 172.21.15.3:0/3041101881 shutdown_connections 2023-10-03T10:13:05.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.753+0000 7f121fd63700 1 -- 172.21.15.3:0/3041101881 wait complete. 2023-10-03T10:13:05.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.754+0000 7f121fd63700 1 Processor -- start 2023-10-03T10:13:05.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.755+0000 7f121fd63700 1 -- start start 2023-10-03T10:13:05.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.755+0000 7f121fd63700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f121812ff20 0x7f121809c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:05.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.755+0000 7f121fd63700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1218130870 0x7f121809cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:05.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.755+0000 7f121fd63700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1218133a80 0x7f121809d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:05.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.755+0000 7f121fd63700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121811ff00 con 0x7f1218133a80 2023-10-03T10:13:05.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.755+0000 7f121fd63700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12181202a0 con 0x7f1218130870 2023-10-03T10:13:05.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.755+0000 7f121fd63700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12180a2130 con 0x7f121812ff20 2023-10-03T10:13:05.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.755+0000 7f121d2fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1218130870 0x7f121809cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:05.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.755+0000 7f121d2fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1218130870 0x7f121809cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40848/0 (socket says 172.21.15.3:40848) 2023-10-03T10:13:05.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.755+0000 7f121d2fe700 1 -- 172.21.15.3:0/664148097 learned_addr learned my addr 172.21.15.3:0/664148097 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:05.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.755+0000 7f121daff700 1 --2- 172.21.15.3:0/664148097 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f121812ff20 0x7f121809c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:05.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.755+0000 7f121e300700 1 --2- 172.21.15.3:0/664148097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1218133a80 0x7f121809d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:05.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.756+0000 7f121d2fe700 1 -- 172.21.15.3:0/664148097 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f121812ff20 msgr2=0x7f121809c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:05.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.756+0000 7f121d2fe700 1 --2- 172.21.15.3:0/664148097 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f121812ff20 0x7f121809c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.756+0000 7f121d2fe700 1 -- 172.21.15.3:0/664148097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1218133a80 msgr2=0x7f121809d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:05.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.756+0000 7f121d2fe700 1 --2- 172.21.15.3:0/664148097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1218133a80 0x7f121809d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.756+0000 7f121d2fe700 1 -- 172.21.15.3:0/664148097 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1208009550 con 0x7f1218130870 2023-10-03T10:13:05.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.756+0000 7f121d2fe700 1 --2- 172.21.15.3:0/664148097 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1218130870 0x7f121809cc20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1208000c00 tx=0x7f1208004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:05.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.756+0000 7f120e7fc700 1 -- 172.21.15.3:0/664148097 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1208004300 con 0x7f1218130870 2023-10-03T10:13:05.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.756+0000 7f120e7fc700 1 -- 172.21.15.3:0/664148097 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120801f070 con 0x7f1218130870 2023-10-03T10:13:05.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.756+0000 7f121fd63700 1 -- 172.21.15.3:0/664148097 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12181204a0 con 0x7f1218130870 2023-10-03T10:13:05.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.756+0000 7f121fd63700 1 -- 172.21.15.3:0/664148097 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f121812d4a0 con 0x7f1218130870 2023-10-03T10:13:05.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.756+0000 7f120e7fc700 1 -- 172.21.15.3:0/664148097 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12080036a0 con 0x7f1218130870 2023-10-03T10:13:05.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.758+0000 7f120e7fc700 1 -- 172.21.15.3:0/664148097 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f1208021030 con 0x7f1218130870 2023-10-03T10:13:05.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.758+0000 7f120e7fc700 1 -- 172.21.15.3:0/664148097 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f1208040dc0 con 0x7f1218130870 2023-10-03T10:13:05.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.758+0000 7f121fd63700 1 -- 172.21.15.3:0/664148097 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12000052e0 con 0x7f1218130870 2023-10-03T10:13:05.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.758+0000 7f121daff700 1 -- 172.21.15.3:0/664148097 >> v1:172.21.15.3:6816/127025 conn(0x7f11fc02f510 legacy=0x7f11fc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:05.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.761+0000 7f3716b3c700 1 Processor -- start 2023-10-03T10:13:05.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.761+0000 7f3716b3c700 1 -- start start 2023-10-03T10:13:05.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.762+0000 7f3716b3c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f371009d6f0 0x7f371009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:05.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.762+0000 7f3716b3c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f371009e410 0x7f3710134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:05.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.762+0000 7f3716b3c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f371009b120 con 0x7f371009e000 2023-10-03T10:13:05.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.762+0000 7f3716b3c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f371006b460 con 0x7f371009e410 2023-10-03T10:13:05.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.762+0000 7f3716b3c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f371009a7e0 con 0x7f371009d6f0 2023-10-03T10:13:05.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.762+0000 7f37150d9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f371009e410 0x7f3710134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:05.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.762+0000 7f37150d9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f371009e410 0x7f3710134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48926/0 (socket says 172.21.15.3:48926) 2023-10-03T10:13:05.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.762+0000 7f37150d9700 1 -- 172.21.15.3:0/1953228371 learned_addr learned my addr 172.21.15.3:0/1953228371 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:05.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.763+0000 7f37150d9700 1 -- 172.21.15.3:0/1953228371 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f371009d6f0 msgr2=0x7f371009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:05.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.763+0000 7f37150d9700 1 --2- 172.21.15.3:0/1953228371 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f371009d6f0 0x7f371009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.763+0000 7f37150d9700 1 -- 172.21.15.3:0/1953228371 >> v1:172.21.15.3:6789/0 conn(0x7f371009e000 legacy=0x7f37101339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:05.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.763+0000 7f37150d9700 1 -- 172.21.15.3:0/1953228371 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371009a500 con 0x7f371009e410 2023-10-03T10:13:05.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.763+0000 7f37150d9700 1 --2- 172.21.15.3:0/1953228371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f371009e410 0x7f3710134100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3708009e50 tx=0x7f370800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aaa33b78dab9f9c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:05.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.763+0000 7f370f7fe700 1 -- 172.21.15.3:0/1953228371 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370800e070 con 0x7f371009e410 2023-10-03T10:13:05.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.763+0000 7f120e7fc700 1 -- 172.21.15.3:0/664148097 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1208015020 con 0x7f1218130870 2023-10-03T10:13:05.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.763+0000 7f370f7fe700 1 -- 172.21.15.3:0/1953228371 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370800b9c0 con 0x7f371009e410 2023-10-03T10:13:05.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.763+0000 7f370f7fe700 1 -- 172.21.15.3:0/1953228371 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370800bd70 con 0x7f371009e410 2023-10-03T10:13:05.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.764+0000 7f3716b3c700 1 -- 172.21.15.3:0/1953228371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f371009e410 msgr2=0x7f3710134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:05.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.764+0000 7f3716b3c700 1 --2- 172.21.15.3:0/1953228371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f371009e410 0x7f3710134100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3708009e50 tx=0x7f370800b330 comp rx=0 tx=0).stop 2023-10-03T10:13:05.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.764+0000 7f3716b3c700 1 -- 172.21.15.3:0/1953228371 shutdown_connections 2023-10-03T10:13:05.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.764+0000 7f3716b3c700 1 --2- 172.21.15.3:0/1953228371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f371009e410 0x7f3710134100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.764+0000 7f3716b3c700 1 --2- 172.21.15.3:0/1953228371 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f371009d6f0 0x7f371009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.764+0000 7f3716b3c700 1 -- 172.21.15.3:0/1953228371 >> 172.21.15.3:0/1953228371 conn(0x7f3710066f50 msgr2=0x7f371009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:05.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.765+0000 7f3716b3c700 1 -- 172.21.15.3:0/1953228371 shutdown_connections 2023-10-03T10:13:05.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.765+0000 7f3716b3c700 1 -- 172.21.15.3:0/1953228371 wait complete. 2023-10-03T10:13:05.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.767+0000 7f3716b3c700 1 Processor -- start 2023-10-03T10:13:05.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.767+0000 7f3716b3c700 1 -- start start 2023-10-03T10:13:05.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.768+0000 7f3716b3c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f371009d6f0 0x7f371011b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:05.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.768+0000 7f3716b3c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f371009e000 0x7f371011b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:05.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.768+0000 7f3716b3c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f371009e410 0x7f37101d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:05.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.768+0000 7f3716b3c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37101a3610 con 0x7f371009e410 2023-10-03T10:13:05.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.768+0000 7f3716b3c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f371011af70 con 0x7f371009d6f0 2023-10-03T10:13:05.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.768+0000 7f3716b3c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f37100a31a0 con 0x7f371009e000 2023-10-03T10:13:05.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.768+0000 7f37150d9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f371009e410 0x7f37101d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:05.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.768+0000 7f37150d9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f371009e410 0x7f37101d4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48928/0 (socket says 172.21.15.3:48928) 2023-10-03T10:13:05.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.768+0000 7f37150d9700 1 -- 172.21.15.3:0/3939192776 learned_addr learned my addr 172.21.15.3:0/3939192776 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:05.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.768+0000 7f370ffff700 1 --2- 172.21.15.3:0/3939192776 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f371009e000 0x7f371011b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:05.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.768+0000 7f37148d8700 1 --2- 172.21.15.3:0/3939192776 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f371009d6f0 0x7f371011b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:05.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.769+0000 7f370ffff700 1 -- 172.21.15.3:0/3939192776 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f371009d6f0 msgr2=0x7f371011b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.769+0000 7f370ffff700 1 --2- 172.21.15.3:0/3939192776 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f371009d6f0 0x7f371011b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.769+0000 7f370ffff700 1 -- 172.21.15.3:0/3939192776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f371009e410 msgr2=0x7f37101d4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.769+0000 7f370ffff700 1 --2- 172.21.15.3:0/3939192776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f371009e410 0x7f37101d4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.769+0000 7f370ffff700 1 -- 172.21.15.3:0/3939192776 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3708009b30 con 0x7f371009e000 2023-10-03T10:13:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.769+0000 7f37150d9700 1 --2- 172.21.15.3:0/3939192776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f371009e410 0x7f37101d4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.769+0000 7f370ffff700 1 --2- 172.21.15.3:0/3939192776 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f371009e000 0x7f371011b8a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3704000f60 tx=0x7f370400c730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.769+0000 7f370d7fa700 1 -- 172.21.15.3:0/3939192776 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37040052d0 con 0x7f371009e000 2023-10-03T10:13:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.770+0000 7f370d7fa700 1 -- 172.21.15.3:0/3939192776 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3704005430 con 0x7f371009e000 2023-10-03T10:13:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.770+0000 7f370d7fa700 1 -- 172.21.15.3:0/3939192776 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37040038c0 con 0x7f371009e000 2023-10-03T10:13:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.770+0000 7f3716b3c700 1 -- 172.21.15.3:0/3939192776 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f371009c710 con 0x7f371009e000 2023-10-03T10:13:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.770+0000 7f3716b3c700 1 -- 172.21.15.3:0/3939192776 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f371011c5c0 con 0x7f371009e000 2023-10-03T10:13:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.771+0000 7f370d7fa700 1 -- 172.21.15.3:0/3939192776 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f3704003a20 con 0x7f371009e000 2023-10-03T10:13:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.771+0000 7f37148d8700 1 -- 172.21.15.3:0/3939192776 >> v1:172.21.15.3:6816/127025 conn(0x7f36fc02f510 legacy=0x7f36fc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.771+0000 7f370d7fa700 1 -- 172.21.15.3:0/3939192776 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f3704010db0 con 0x7f371009e000 2023-10-03T10:13:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.771+0000 7f3716b3c700 1 -- 172.21.15.3:0/3939192776 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37101a3270 con 0x7f371009e000 2023-10-03T10:13:05.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.776+0000 7f370d7fa700 1 -- 172.21.15.3:0/3939192776 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3704044060 con 0x7f371009e000 2023-10-03T10:13:05.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.884+0000 7fb44dd9b700 1 -- 172.21.15.3:0/323398835 >> v1:172.21.15.3:6816/127025 conn(0x7fb43002f510 legacy=0x7fb4300319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.919+0000 7f8db6ffd700 1 -- 172.21.15.3:0/2777641979 >> v1:172.21.15.3:6816/127025 conn(0x7f8da402f510 legacy=0x7f8da40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:05.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.937+0000 7fb4549fc700 1 -- 172.21.15.3:0/323398835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb45006c360 con 0x7fb45009d6f0 2023-10-03T10:13:05.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.937+0000 7fb43effd700 1 -- 172.21.15.3:0/323398835 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb440015020 con 0x7fb45009d6f0 2023-10-03T10:13:05.938 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:05.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.940+0000 7fb4549fc700 1 -- 172.21.15.3:0/323398835 >> v1:172.21.15.3:6816/127025 conn(0x7fb43002f510 legacy=0x7fb4300319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:05.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.940+0000 7fb4549fc700 1 -- 172.21.15.3:0/323398835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45009d6f0 msgr2=0x7fb4501c80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:05.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.940+0000 7fb4549fc700 1 --2- 172.21.15.3:0/323398835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45009d6f0 0x7fb4501c80b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb440011db0 tx=0x7fb440004750 comp rx=0 tx=0).stop 2023-10-03T10:13:05.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.941+0000 7fb4549fc700 1 -- 172.21.15.3:0/323398835 shutdown_connections 2023-10-03T10:13:05.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.941+0000 7fb4549fc700 1 --2- 172.21.15.3:0/323398835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb45009e410 0x7fb4501c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.941+0000 7fb4549fc700 1 --2- 172.21.15.3:0/323398835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45009d6f0 0x7fb4501c80b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.941+0000 7fb4549fc700 1 --2- 172.21.15.3:0/323398835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb45009e000 0x7fb4501c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.941+0000 7fb4549fc700 1 -- 172.21.15.3:0/323398835 >> 172.21.15.3:0/323398835 conn(0x7fb450066f50 msgr2=0x7fb4501bbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:05.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.941+0000 7fb4549fc700 1 -- 172.21.15.3:0/323398835 shutdown_connections 2023-10-03T10:13:05.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.941+0000 7fb4549fc700 1 -- 172.21.15.3:0/323398835 wait complete. 2023-10-03T10:13:05.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.953+0000 7f8dbd674700 1 -- 172.21.15.3:0/2777641979 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8db8074ea0 con 0x7f8db812f320 2023-10-03T10:13:05.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.954+0000 7f8d9f7fe700 1 -- 172.21.15.3:0/2777641979 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8db0011380 con 0x7f8db812f320 2023-10-03T10:13:05.954 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:05.955 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:05.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.956+0000 7f8dbd674700 1 -- 172.21.15.3:0/2777641979 >> v1:172.21.15.3:6816/127025 conn(0x7f8da402f510 legacy=0x7f8da40319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:05.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.956+0000 7f8dbd674700 1 -- 172.21.15.3:0/2777641979 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8db812f320 msgr2=0x7f8db8119750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:05.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.956+0000 7f8dbd674700 1 --2- 172.21.15.3:0/2777641979 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8db812f320 0x7f8db8119750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8db000cb90 tx=0x7f8db000cea0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.956+0000 7f8dbd674700 1 -- 172.21.15.3:0/2777641979 shutdown_connections 2023-10-03T10:13:05.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.956+0000 7f8dbd674700 1 --2- 172.21.15.3:0/2777641979 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8db812f320 0x7f8db8119750 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.956+0000 7f8dbd674700 1 --2- 172.21.15.3:0/2777641979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8db8066410 0x7f8db8118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.957+0000 7f8dbd674700 1 --2- 172.21.15.3:0/2777641979 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8db806eb80 0x7f8db8119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.957+0000 7f8dbd674700 1 -- 172.21.15.3:0/2777641979 >> 172.21.15.3:0/2777641979 conn(0x7f8db80a2aa0 msgr2=0x7f8db812d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:05.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.957+0000 7f8dbd674700 1 -- 172.21.15.3:0/2777641979 shutdown_connections 2023-10-03T10:13:05.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.957+0000 7f8dbd674700 1 -- 172.21.15.3:0/2777641979 wait complete. 2023-10-03T10:13:05.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.959+0000 7f121daff700 1 -- 172.21.15.3:0/664148097 >> v1:172.21.15.3:6816/127025 conn(0x7f11fc02f510 legacy=0x7f11fc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:05.971 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:05.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.972+0000 7f37148d8700 1 -- 172.21.15.3:0/3939192776 >> v1:172.21.15.3:6816/127025 conn(0x7f36fc02f510 legacy=0x7f36fc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:05.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.984+0000 7f121fd63700 1 -- 172.21.15.3:0/664148097 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1200005860 con 0x7f1218130870 2023-10-03T10:13:05.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.984+0000 7f120e7fc700 1 -- 172.21.15.3:0/664148097 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f12080199a0 con 0x7f1218130870 2023-10-03T10:13:05.985 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:05.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.987+0000 7f121fd63700 1 -- 172.21.15.3:0/664148097 >> v1:172.21.15.3:6816/127025 conn(0x7f11fc02f510 legacy=0x7f11fc0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:05.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.987+0000 7f121fd63700 1 -- 172.21.15.3:0/664148097 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1218130870 msgr2=0x7f121809cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:05.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.987+0000 7f121fd63700 1 --2- 172.21.15.3:0/664148097 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1218130870 0x7f121809cc20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1208000c00 tx=0x7f1208004750 comp rx=0 tx=0).stop 2023-10-03T10:13:05.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.987+0000 7f121fd63700 1 -- 172.21.15.3:0/664148097 shutdown_connections 2023-10-03T10:13:05.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.987+0000 7f121fd63700 1 --2- 172.21.15.3:0/664148097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1218133a80 0x7f121809d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.987+0000 7f121fd63700 1 --2- 172.21.15.3:0/664148097 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f121812ff20 0x7f121809c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.987+0000 7f121fd63700 1 --2- 172.21.15.3:0/664148097 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1218130870 0x7f121809cc20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:05.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.987+0000 7f121fd63700 1 -- 172.21.15.3:0/664148097 >> 172.21.15.3:0/664148097 conn(0x7f12180a4210 msgr2=0x7f121809e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:05.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.987+0000 7f121fd63700 1 -- 172.21.15.3:0/664148097 shutdown_connections 2023-10-03T10:13:05.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:05.987+0000 7f121fd63700 1 -- 172.21.15.3:0/664148097 wait complete. 2023-10-03T10:13:06.002 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:06.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:06.009+0000 7f3716b3c700 1 -- 172.21.15.3:0/3939192776 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f371006c360 con 0x7f371009e000 2023-10-03T10:13:06.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:06.010+0000 7f370d7fa700 1 -- 172.21.15.3:0/3939192776 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f370402e070 con 0x7f371009e000 2023-10-03T10:13:06.010 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:06.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:06.012+0000 7f3716b3c700 1 -- 172.21.15.3:0/3939192776 >> v1:172.21.15.3:6816/127025 conn(0x7f36fc02f510 legacy=0x7f36fc0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:06.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:06.012+0000 7f3716b3c700 1 -- 172.21.15.3:0/3939192776 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f371009e000 msgr2=0x7f371011b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:06.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:06.012+0000 7f3716b3c700 1 --2- 172.21.15.3:0/3939192776 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f371009e000 0x7f371011b8a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3704000f60 tx=0x7f370400c730 comp rx=0 tx=0).stop 2023-10-03T10:13:06.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:06.012+0000 7f3716b3c700 1 -- 172.21.15.3:0/3939192776 shutdown_connections 2023-10-03T10:13:06.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:06.012+0000 7f3716b3c700 1 --2- 172.21.15.3:0/3939192776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f371009e410 0x7f37101d4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:06.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:06.012+0000 7f3716b3c700 1 --2- 172.21.15.3:0/3939192776 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f371009d6f0 0x7f371011b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:06.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:06.012+0000 7f3716b3c700 1 --2- 172.21.15.3:0/3939192776 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f371009e000 0x7f371011b8a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:06.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:06.012+0000 7f3716b3c700 1 -- 172.21.15.3:0/3939192776 >> 172.21.15.3:0/3939192776 conn(0x7f3710066f50 msgr2=0x7f37101baab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:06.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:06.012+0000 7f3716b3c700 1 -- 172.21.15.3:0/3939192776 shutdown_connections 2023-10-03T10:13:06.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:06.012+0000 7f3716b3c700 1 -- 172.21.15.3:0/3939192776 wait complete. 2023-10-03T10:13:06.026 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:06.956 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:06.971 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:07.002 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:07.027 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:07.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.112+0000 7fe73ad35700 1 Processor -- start 2023-10-03T10:13:07.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.112+0000 7fe73ad35700 1 -- start start 2023-10-03T10:13:07.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.112+0000 7fe73ad35700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe73409e1d0 0x7fe734138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:07.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.112+0000 7fe73ad35700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe734139170 0x7fe73413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:07.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.112+0000 7fe73ad35700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe73409b120 con 0x7fe73409d6f0 2023-10-03T10:13:07.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.112+0000 7fe73ad35700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe73406b460 con 0x7fe734139170 2023-10-03T10:13:07.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.112+0000 7fe73ad35700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe73409a7e0 con 0x7fe73409e1d0 2023-10-03T10:13:07.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.113+0000 7fe7392d2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe734139170 0x7fe73413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:07.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.113+0000 7fe733fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe73409e1d0 0x7fe734138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:07.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.113+0000 7fe738ad1700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe73409d6f0 0x7fe73409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49520/0 (socket says 172.21.15.3:49520) 2023-10-03T10:13:07.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.113+0000 7fe738ad1700 1 -- 172.21.15.3:0/2273031925 learned_addr learned my addr 172.21.15.3:0/2273031925 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:07.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.113+0000 7fe7392d2700 1 -- 172.21.15.3:0/2273031925 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe73409e1d0 msgr2=0x7fe734138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:07.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.113+0000 7fe7392d2700 1 --2- 172.21.15.3:0/2273031925 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe73409e1d0 0x7fe734138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.113+0000 7fe7392d2700 1 -- 172.21.15.3:0/2273031925 >> v1:172.21.15.3:6789/0 conn(0x7fe73409d6f0 legacy=0x7fe73409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:07.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.113+0000 7fe7392d2700 1 -- 172.21.15.3:0/2273031925 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe73409a500 con 0x7fe734139170 2023-10-03T10:13:07.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.113+0000 7fe7337fe700 1 -- 172.21.15.3:0/2273031925 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2558315682 0 0) 0x7fe73409b120 con 0x7fe73409d6f0 2023-10-03T10:13:07.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.113+0000 7fe733fff700 1 --2- 172.21.15.3:0/2273031925 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe73409e1d0 0x7fe734138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:07.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.113+0000 7fe7392d2700 1 --2- 172.21.15.3:0/2273031925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe734139170 0x7fe73413b550 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe72c009d90 tx=0x7fe72c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486aad5405896729 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:07.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.114+0000 7fe7337fe700 1 -- 172.21.15.3:0/2273031925 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c00e070 con 0x7fe734139170 2023-10-03T10:13:07.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.114+0000 7fe7337fe700 1 -- 172.21.15.3:0/2273031925 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72c00bb20 con 0x7fe734139170 2023-10-03T10:13:07.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.114+0000 7fe7337fe700 1 -- 172.21.15.3:0/2273031925 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c012430 con 0x7fe734139170 2023-10-03T10:13:07.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.114+0000 7fe73ad35700 1 -- 172.21.15.3:0/2273031925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe734139170 msgr2=0x7fe73413b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:07.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.114+0000 7fe73ad35700 1 --2- 172.21.15.3:0/2273031925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe734139170 0x7fe73413b550 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe72c009d90 tx=0x7fe72c00b330 comp rx=0 tx=0).stop 2023-10-03T10:13:07.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.115+0000 7fe73ad35700 1 -- 172.21.15.3:0/2273031925 shutdown_connections 2023-10-03T10:13:07.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.115+0000 7fe73ad35700 1 --2- 172.21.15.3:0/2273031925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe734139170 0x7fe73413b550 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.115+0000 7fe73ad35700 1 --2- 172.21.15.3:0/2273031925 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe73409e1d0 0x7fe734138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.115+0000 7fe73ad35700 1 -- 172.21.15.3:0/2273031925 >> 172.21.15.3:0/2273031925 conn(0x7fe734066f50 msgr2=0x7fe73409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:07.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.115+0000 7fe73ad35700 1 -- 172.21.15.3:0/2273031925 shutdown_connections 2023-10-03T10:13:07.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.115+0000 7fe73ad35700 1 -- 172.21.15.3:0/2273031925 wait complete. 2023-10-03T10:13:07.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.116+0000 7fe73ad35700 1 Processor -- start 2023-10-03T10:13:07.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.116+0000 7fe73ad35700 1 -- start start 2023-10-03T10:13:07.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.117+0000 7fe73ad35700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe73409d6f0 0x7fe73411e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:07.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.117+0000 7fe73ad35700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe73409e1d0 0x7fe73411eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:07.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.117+0000 7fe73ad35700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe734139170 0x7fe73411f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:07.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.117+0000 7fe73ad35700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe734120150 con 0x7fe73409e1d0 2023-10-03T10:13:07.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.117+0000 7fe73ad35700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7341214a0 con 0x7fe73409d6f0 2023-10-03T10:13:07.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.117+0000 7fe73ad35700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe73411fa80 con 0x7fe734139170 2023-10-03T10:13:07.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.117+0000 7fe738ad1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe73409d6f0 0x7fe73411e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:07.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.117+0000 7fe738ad1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe73409d6f0 0x7fe73411e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40886/0 (socket says 172.21.15.3:40886) 2023-10-03T10:13:07.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.117+0000 7fe738ad1700 1 -- 172.21.15.3:0/1059071041 learned_addr learned my addr 172.21.15.3:0/1059071041 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:07.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.117+0000 7fe7392d2700 1 --2- 172.21.15.3:0/1059071041 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe734139170 0x7fe73411f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:07.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.118+0000 7fe733fff700 1 --2- 172.21.15.3:0/1059071041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe73409e1d0 0x7fe73411eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:07.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.118+0000 7fe738ad1700 1 -- 172.21.15.3:0/1059071041 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe734139170 msgr2=0x7fe73411f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:07.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.118+0000 7fe738ad1700 1 --2- 172.21.15.3:0/1059071041 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe734139170 0x7fe73411f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.118+0000 7fe738ad1700 1 -- 172.21.15.3:0/1059071041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe73409e1d0 msgr2=0x7fe73411eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:07.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.118+0000 7fe738ad1700 1 --2- 172.21.15.3:0/1059071041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe73409e1d0 0x7fe73411eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.118+0000 7fe738ad1700 1 -- 172.21.15.3:0/1059071041 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7200094a0 con 0x7fe73409d6f0 2023-10-03T10:13:07.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.118+0000 7fe733fff700 1 --2- 172.21.15.3:0/1059071041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe73409e1d0 0x7fe73411eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:07.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.118+0000 7fe7392d2700 1 --2- 172.21.15.3:0/1059071041 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe734139170 0x7fe73411f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:07.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.118+0000 7fe738ad1700 1 --2- 172.21.15.3:0/1059071041 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe73409d6f0 0x7fe73411e620 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe72800c550 tx=0x7fe72800c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:07.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.119+0000 7fe7317fa700 1 -- 172.21.15.3:0/1059071041 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe728004c80 con 0x7fe73409d6f0 2023-10-03T10:13:07.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.119+0000 7fe7317fa700 1 -- 172.21.15.3:0/1059071041 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe728004e00 con 0x7fe73409d6f0 2023-10-03T10:13:07.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.119+0000 7fe7317fa700 1 -- 172.21.15.3:0/1059071041 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe728007310 con 0x7fe73409d6f0 2023-10-03T10:13:07.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.119+0000 7fe73ad35700 1 -- 172.21.15.3:0/1059071041 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe72c009a70 con 0x7fe73409d6f0 2023-10-03T10:13:07.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.119+0000 7fe73ad35700 1 -- 172.21.15.3:0/1059071041 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7341220b0 con 0x7fe73409d6f0 2023-10-03T10:13:07.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.120+0000 7fe73ad35700 1 -- 172.21.15.3:0/1059071041 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe734074ea0 con 0x7fe73409d6f0 2023-10-03T10:13:07.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.120+0000 7fe7317fa700 1 -- 172.21.15.3:0/1059071041 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fe728007470 con 0x7fe73409d6f0 2023-10-03T10:13:07.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.120+0000 7fe7317fa700 1 -- 172.21.15.3:0/1059071041 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7fe72803f640 con 0x7fe73409d6f0 2023-10-03T10:13:07.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.120+0000 7fe733fff700 1 -- 172.21.15.3:0/1059071041 >> v1:172.21.15.3:6816/127025 conn(0x7fe71802f510 legacy=0x7fe7180319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:07.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.129+0000 7fe7317fa700 1 -- 172.21.15.3:0/1059071041 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe7280102c0 con 0x7fe73409d6f0 2023-10-03T10:13:07.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.161+0000 7f19c5b09700 1 Processor -- start 2023-10-03T10:13:07.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.161+0000 7f19c5b09700 1 -- start start 2023-10-03T10:13:07.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.162+0000 7f19c5b09700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19c0133a80 0x7f19c0133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:07.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.162+0000 7f19c5b09700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19c0130a40 0x7f19c013b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:07.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.162+0000 7f19c5b09700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19c006bc00 con 0x7f19c012ff20 2023-10-03T10:13:07.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.162+0000 7f19c5b09700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c006c570 con 0x7f19c0130a40 2023-10-03T10:13:07.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.162+0000 7f19c5b09700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f19c00a0040 con 0x7f19c0133a80 2023-10-03T10:13:07.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.162+0000 7f19bffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19c0130a40 0x7f19c013b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:07.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.162+0000 7f19bffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19c0130a40 0x7f19c013b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40900/0 (socket says 172.21.15.3:40900) 2023-10-03T10:13:07.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.162+0000 7f19bffff700 1 -- 172.21.15.3:0/1994389515 learned_addr learned my addr 172.21.15.3:0/1994389515 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:07.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.163+0000 7f19bf7fe700 1 --2- 172.21.15.3:0/1994389515 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19c0133a80 0x7f19c0133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:07.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.163+0000 7f19bffff700 1 -- 172.21.15.3:0/1994389515 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19c0133a80 msgr2=0x7f19c0133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:07.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.163+0000 7f19bffff700 1 --2- 172.21.15.3:0/1994389515 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19c0133a80 0x7f19c0133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.163+0000 7f19bffff700 1 -- 172.21.15.3:0/1994389515 >> v1:172.21.15.3:6789/0 conn(0x7f19c012ff20 legacy=0x7f19c0130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:07.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.163+0000 7f19be7fc700 1 -- 172.21.15.3:0/1994389515 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3761262001 0 0) 0x7f19c006bc00 con 0x7f19c012ff20 2023-10-03T10:13:07.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.163+0000 7f19bffff700 1 -- 172.21.15.3:0/1994389515 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c0133860 con 0x7f19c0130a40 2023-10-03T10:13:07.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.163+0000 7f19bffff700 1 --2- 172.21.15.3:0/1994389515 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19c0130a40 0x7f19c013b850 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f19b8009e80 tx=0x7f19b800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96a544289da8d346 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:07.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.163+0000 7f19be7fc700 1 -- 172.21.15.3:0/1994389515 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b800e070 con 0x7f19c0130a40 2023-10-03T10:13:07.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.164+0000 7f19be7fc700 1 -- 172.21.15.3:0/1994389515 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b800b910 con 0x7f19c0130a40 2023-10-03T10:13:07.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.164+0000 7f19be7fc700 1 -- 172.21.15.3:0/1994389515 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b800bcc0 con 0x7f19c0130a40 2023-10-03T10:13:07.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.165+0000 7f19c5b09700 1 -- 172.21.15.3:0/1994389515 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19c0130a40 msgr2=0x7f19c013b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:07.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.165+0000 7f19c5b09700 1 --2- 172.21.15.3:0/1994389515 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19c0130a40 0x7f19c013b850 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f19b8009e80 tx=0x7f19b800b330 comp rx=0 tx=0).stop 2023-10-03T10:13:07.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.165+0000 7f19c5b09700 1 -- 172.21.15.3:0/1994389515 shutdown_connections 2023-10-03T10:13:07.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.165+0000 7f19c5b09700 1 --2- 172.21.15.3:0/1994389515 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19c0130a40 0x7f19c013b850 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.165+0000 7f19c5b09700 1 --2- 172.21.15.3:0/1994389515 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19c0133a80 0x7f19c0133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.165+0000 7f19c5b09700 1 -- 172.21.15.3:0/1994389515 >> 172.21.15.3:0/1994389515 conn(0x7f19c00a4210 msgr2=0x7f19c00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:07.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.165+0000 7f19c5b09700 1 -- 172.21.15.3:0/1994389515 shutdown_connections 2023-10-03T10:13:07.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.165+0000 7f19c5b09700 1 -- 172.21.15.3:0/1994389515 wait complete. 2023-10-03T10:13:07.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.166+0000 7f19c5b09700 1 Processor -- start 2023-10-03T10:13:07.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.166+0000 7f19c5b09700 1 -- start start 2023-10-03T10:13:07.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.167+0000 7f19c5b09700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19c012ff20 0x7f19c009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:07.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.167+0000 7f19c5b09700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19c0130a40 0x7f19c009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:07.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.167+0000 7f19c5b09700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19c0133a80 0x7f19c009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:07.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.167+0000 7f19c5b09700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c012d140 con 0x7f19c012ff20 2023-10-03T10:13:07.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.167+0000 7f19c5b09700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c00a2150 con 0x7f19c0133a80 2023-10-03T10:13:07.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.167+0000 7f19c5b09700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f19c009df00 con 0x7f19c0130a40 2023-10-03T10:13:07.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.167+0000 7f19bf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19c012ff20 0x7f19c009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:07.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.167+0000 7f19bf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19c012ff20 0x7f19c009c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48954/0 (socket says 172.21.15.3:48954) 2023-10-03T10:13:07.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.167+0000 7f19bf7fe700 1 -- 172.21.15.3:0/3811211813 learned_addr learned my addr 172.21.15.3:0/3811211813 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:07.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.168+0000 7f19beffd700 1 --2- 172.21.15.3:0/3811211813 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19c0130a40 0x7f19c009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:07.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.168+0000 7f19bffff700 1 --2- 172.21.15.3:0/3811211813 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19c0133a80 0x7f19c009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:07.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.168+0000 7f19beffd700 1 -- 172.21.15.3:0/3811211813 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19c0133a80 msgr2=0x7f19c009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:07.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.168+0000 7f19beffd700 1 --2- 172.21.15.3:0/3811211813 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19c0133a80 0x7f19c009cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.168+0000 7f19beffd700 1 -- 172.21.15.3:0/3811211813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19c012ff20 msgr2=0x7f19c009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:07.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.168+0000 7f19beffd700 1 --2- 172.21.15.3:0/3811211813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19c012ff20 0x7f19c009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.168+0000 7f19beffd700 1 -- 172.21.15.3:0/3811211813 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b8009b30 con 0x7f19c0130a40 2023-10-03T10:13:07.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.168+0000 7f19beffd700 1 --2- 172.21.15.3:0/3811211813 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19c0130a40 0x7f19c009c9c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f19c009ac40 tx=0x7f19b400c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:07.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.168+0000 7f19a7fff700 1 -- 172.21.15.3:0/3811211813 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b40054e0 con 0x7f19c0130a40 2023-10-03T10:13:07.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.169+0000 7f19a7fff700 1 -- 172.21.15.3:0/3811211813 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b400ce40 con 0x7f19c0130a40 2023-10-03T10:13:07.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.169+0000 7f19a7fff700 1 -- 172.21.15.3:0/3811211813 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b40038c0 con 0x7f19c0130a40 2023-10-03T10:13:07.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.169+0000 7f19c5b09700 1 -- 172.21.15.3:0/3811211813 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c011fc50 con 0x7f19c0130a40 2023-10-03T10:13:07.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.169+0000 7f19c5b09700 1 -- 172.21.15.3:0/3811211813 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c011f8b0 con 0x7f19c0130a40 2023-10-03T10:13:07.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.170+0000 7f19a7fff700 1 -- 172.21.15.3:0/3811211813 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f19b4003a20 con 0x7f19c0130a40 2023-10-03T10:13:07.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.170+0000 7f19bf7fe700 1 -- 172.21.15.3:0/3811211813 >> v1:172.21.15.3:6816/127025 conn(0x7f19ac02f540 legacy=0x7f19ac0319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:07.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.170+0000 7f19a7fff700 1 -- 172.21.15.3:0/3811211813 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f19b403eef0 con 0x7f19c0130a40 2023-10-03T10:13:07.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.170+0000 7f19c5b09700 1 -- 172.21.15.3:0/3811211813 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19c0074ea0 con 0x7f19c0130a40 2023-10-03T10:13:07.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.180+0000 7f19a7fff700 1 -- 172.21.15.3:0/3811211813 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f19b4017020 con 0x7f19c0130a40 2023-10-03T10:13:07.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.196+0000 7f21c0837700 1 Processor -- start 2023-10-03T10:13:07.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.197+0000 7f21c0837700 1 -- start start 2023-10-03T10:13:07.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.198+0000 7f21c0837700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21bc09d9f0 0x7f21bc136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:07.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.198+0000 7f21c0837700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21bc136a70 0x7f21bc138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:07.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.198+0000 7f21c0837700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21bc09e130 con 0x7f21bc1316c0 2023-10-03T10:13:07.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.198+0000 7f21c0837700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc1321a0 con 0x7f21bc09d9f0 2023-10-03T10:13:07.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.198+0000 7f21c0837700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc1312b0 con 0x7f21bc136a70 2023-10-03T10:13:07.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.198+0000 7f21ba59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f21bc1316c0 0x7f21bc131a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49534/0 (socket says 172.21.15.3:49534) 2023-10-03T10:13:07.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.198+0000 7f21ba59c700 1 -- 172.21.15.3:0/917960616 learned_addr learned my addr 172.21.15.3:0/917960616 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:07.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.198+0000 7f21b9d9b700 1 --2- 172.21.15.3:0/917960616 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21bc09d9f0 0x7f21bc136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:07.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.198+0000 7f21bad9d700 1 --2- 172.21.15.3:0/917960616 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21bc136a70 0x7f21bc138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:07.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.198+0000 7f21b9d9b700 1 -- 172.21.15.3:0/917960616 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21bc136a70 msgr2=0x7f21bc138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:07.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.198+0000 7f21b9d9b700 1 --2- 172.21.15.3:0/917960616 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21bc136a70 0x7f21bc138e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.198+0000 7f21b9d9b700 1 -- 172.21.15.3:0/917960616 >> v1:172.21.15.3:6789/0 conn(0x7f21bc1316c0 legacy=0x7f21bc131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:07.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.198+0000 7f21b9d9b700 1 -- 172.21.15.3:0/917960616 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21bc09f6e0 con 0x7f21bc09d9f0 2023-10-03T10:13:07.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.199+0000 7f21b9d9b700 1 --2- 172.21.15.3:0/917960616 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21bc09d9f0 0x7f21bc136530 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009870 tx=0x7f21b0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=819f5c44844a09ce server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:07.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.200+0000 7f21b959a700 1 -- 172.21.15.3:0/917960616 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b0004230 con 0x7f21bc09d9f0 2023-10-03T10:13:07.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.200+0000 7f21b959a700 1 -- 172.21.15.3:0/917960616 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b0004390 con 0x7f21bc09d9f0 2023-10-03T10:13:07.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.200+0000 7f21b959a700 1 -- 172.21.15.3:0/917960616 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b00036a0 con 0x7f21bc09d9f0 2023-10-03T10:13:07.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.200+0000 7f21c0837700 1 -- 172.21.15.3:0/917960616 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21bc09d9f0 msgr2=0x7f21bc136530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:07.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.200+0000 7f21c0837700 1 --2- 172.21.15.3:0/917960616 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21bc09d9f0 0x7f21bc136530 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009870 tx=0x7f21b0009b80 comp rx=0 tx=0).stop 2023-10-03T10:13:07.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.200+0000 7f21c0837700 1 -- 172.21.15.3:0/917960616 shutdown_connections 2023-10-03T10:13:07.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.201+0000 7f21c0837700 1 --2- 172.21.15.3:0/917960616 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21bc136a70 0x7f21bc138e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.201+0000 7f21c0837700 1 --2- 172.21.15.3:0/917960616 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21bc09d9f0 0x7f21bc136530 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.201+0000 7f21c0837700 1 -- 172.21.15.3:0/917960616 >> 172.21.15.3:0/917960616 conn(0x7f21bc066f50 msgr2=0x7f21bc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:07.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.201+0000 7f21c0837700 1 -- 172.21.15.3:0/917960616 shutdown_connections 2023-10-03T10:13:07.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.201+0000 7f21c0837700 1 -- 172.21.15.3:0/917960616 wait complete. 2023-10-03T10:13:07.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.202+0000 7f21c0837700 1 Processor -- start 2023-10-03T10:13:07.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.202+0000 7f21c0837700 1 -- start start 2023-10-03T10:13:07.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.203+0000 7f21c0837700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21bc09d9f0 0x7f21bc070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:07.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.203+0000 7f21c0837700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21bc1316c0 0x7f21bc071340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:07.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.203+0000 7f21c0837700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21bc136a70 0x7f21bc071880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:07.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.203+0000 7f21c0837700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc135600 con 0x7f21bc09d9f0 2023-10-03T10:13:07.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.203+0000 7f21c0837700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc06f8f0 con 0x7f21bc136a70 2023-10-03T10:13:07.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.203+0000 7f21c0837700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc0722f0 con 0x7f21bc1316c0 2023-10-03T10:13:07.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.203+0000 7f21ba59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21bc09d9f0 0x7f21bc070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:07.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.203+0000 7f21ba59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21bc09d9f0 0x7f21bc070e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48956/0 (socket says 172.21.15.3:48956) 2023-10-03T10:13:07.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.203+0000 7f21ba59c700 1 -- 172.21.15.3:0/508963432 learned_addr learned my addr 172.21.15.3:0/508963432 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:07.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.203+0000 7f21b9d9b700 1 --2- 172.21.15.3:0/508963432 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21bc1316c0 0x7f21bc071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:07.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.204+0000 7f21bad9d700 1 --2- 172.21.15.3:0/508963432 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21bc136a70 0x7f21bc071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:07.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.204+0000 7f21b9d9b700 1 -- 172.21.15.3:0/508963432 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21bc136a70 msgr2=0x7f21bc071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:07.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.204+0000 7f21b9d9b700 1 --2- 172.21.15.3:0/508963432 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21bc136a70 0x7f21bc071880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.204+0000 7f21b9d9b700 1 -- 172.21.15.3:0/508963432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21bc09d9f0 msgr2=0x7f21bc070e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:07.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.204+0000 7f21b9d9b700 1 --2- 172.21.15.3:0/508963432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21bc09d9f0 0x7f21bc070e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.204+0000 7f21b9d9b700 1 -- 172.21.15.3:0/508963432 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b00094a0 con 0x7f21bc1316c0 2023-10-03T10:13:07.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.204+0000 7f21b9d9b700 1 --2- 172.21.15.3:0/508963432 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21bc1316c0 0x7f21bc071340 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009870 tx=0x7f21b0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:07.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.204+0000 7f21aaffd700 1 -- 172.21.15.3:0/508963432 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b0022070 con 0x7f21bc1316c0 2023-10-03T10:13:07.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.204+0000 7f21aaffd700 1 -- 172.21.15.3:0/508963432 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b00045d0 con 0x7f21bc1316c0 2023-10-03T10:13:07.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.204+0000 7f21c0837700 1 -- 172.21.15.3:0/508963432 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21bc06fc80 con 0x7f21bc1316c0 2023-10-03T10:13:07.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.204+0000 7f21aaffd700 1 -- 172.21.15.3:0/508963432 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b001dca0 con 0x7f21bc1316c0 2023-10-03T10:13:07.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.204+0000 7f21c0837700 1 -- 172.21.15.3:0/508963432 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21bc0709b0 con 0x7f21bc1316c0 2023-10-03T10:13:07.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.205+0000 7f21aaffd700 1 -- 172.21.15.3:0/508963432 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f21b0026070 con 0x7f21bc1316c0 2023-10-03T10:13:07.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.206+0000 7f21aaffd700 1 -- 172.21.15.3:0/508963432 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f21b0044630 con 0x7f21bc1316c0 2023-10-03T10:13:07.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.206+0000 7f21ba59c700 1 -- 172.21.15.3:0/508963432 >> v1:172.21.15.3:6816/127025 conn(0x7f219c02f510 legacy=0x7f219c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:07.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.206+0000 7f21c0837700 1 -- 172.21.15.3:0/508963432 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21bc074ea0 con 0x7f21bc1316c0 2023-10-03T10:13:07.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.211+0000 7f70e4ad0700 1 Processor -- start 2023-10-03T10:13:07.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.212+0000 7f21aaffd700 1 -- 172.21.15.3:0/508963432 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f21b002d020 con 0x7f21bc1316c0 2023-10-03T10:13:07.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.211+0000 7f70e4ad0700 1 -- start start 2023-10-03T10:13:07.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.212+0000 7f70e4ad0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70e006f5d0 0x7f70e012ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:07.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.212+0000 7f70e4ad0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f70e0135c20 0x7f70e0138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:07.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.212+0000 7f70e4ad0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70e006fa40 con 0x7f70e012cf60 2023-10-03T10:13:07.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.212+0000 7f70e4ad0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e012cd90 con 0x7f70e0135c20 2023-10-03T10:13:07.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.212+0000 7f70ddd9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70e006f5d0 0x7f70e012ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:07.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.212+0000 7f70de59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f70e012cf60 0x7f70e012f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49550/0 (socket says 172.21.15.3:49550) 2023-10-03T10:13:07.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.212+0000 7f70e4ad0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f70e0071780 con 0x7f70e006f5d0 2023-10-03T10:13:07.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.212+0000 7f70de59c700 1 -- 172.21.15.3:0/1315091188 learned_addr learned my addr 172.21.15.3:0/1315091188 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:07.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.212+0000 7f70ded9d700 1 --2- 172.21.15.3:0/1315091188 >> v2:172.21.15.3:3300/0 conn(0x7f70e0135c20 0x7f70e0138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:07.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.212+0000 7f70ded9d700 1 -- 172.21.15.3:0/1315091188 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70e006f5d0 msgr2=0x7f70e012ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:07.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.212+0000 7f70ded9d700 1 --2- 172.21.15.3:0/1315091188 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70e006f5d0 0x7f70e012ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.213+0000 7f70ded9d700 1 -- 172.21.15.3:0/1315091188 >> v1:172.21.15.3:6789/0 conn(0x7f70e012cf60 legacy=0x7f70e012f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:07.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.213+0000 7f70ded9d700 1 -- 172.21.15.3:0/1315091188 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e0071a40 con 0x7f70e0135c20 2023-10-03T10:13:07.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.213+0000 7f70ded9d700 1 --2- 172.21.15.3:0/1315091188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e0135c20 0x7f70e0138000 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009dc0 tx=0x7f70d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c07bb2b2f0c425 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:07.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.213+0000 7f70dd59a700 1 -- 172.21.15.3:0/1315091188 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d800e070 con 0x7f70e0135c20 2023-10-03T10:13:07.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.213+0000 7f70dd59a700 1 -- 172.21.15.3:0/1315091188 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d800bae0 con 0x7f70e0135c20 2023-10-03T10:13:07.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.213+0000 7f70dd59a700 1 -- 172.21.15.3:0/1315091188 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d800be90 con 0x7f70e0135c20 2023-10-03T10:13:07.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.214+0000 7f70e4ad0700 1 -- 172.21.15.3:0/1315091188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e0135c20 msgr2=0x7f70e0138000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:07.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.214+0000 7f70e4ad0700 1 --2- 172.21.15.3:0/1315091188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e0135c20 0x7f70e0138000 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009dc0 tx=0x7f70d800b330 comp rx=0 tx=0).stop 2023-10-03T10:13:07.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.214+0000 7f70e4ad0700 1 -- 172.21.15.3:0/1315091188 shutdown_connections 2023-10-03T10:13:07.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.214+0000 7f70e4ad0700 1 --2- 172.21.15.3:0/1315091188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e0135c20 0x7f70e0138000 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.214+0000 7f70e4ad0700 1 --2- 172.21.15.3:0/1315091188 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70e006f5d0 0x7f70e012ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.214+0000 7f70e4ad0700 1 -- 172.21.15.3:0/1315091188 >> 172.21.15.3:0/1315091188 conn(0x7f70e00a2aa0 msgr2=0x7f70e00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:07.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.214+0000 7f70e4ad0700 1 -- 172.21.15.3:0/1315091188 shutdown_connections 2023-10-03T10:13:07.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.214+0000 7f70e4ad0700 1 -- 172.21.15.3:0/1315091188 wait complete. 2023-10-03T10:13:07.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.216+0000 7f70e4ad0700 1 Processor -- start 2023-10-03T10:13:07.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.216+0000 7f70e4ad0700 1 -- start start 2023-10-03T10:13:07.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.217+0000 7f70e4ad0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e006f5d0 0x7f70e0130920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:07.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.217+0000 7f70e4ad0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70e012cf60 0x7f70e0130e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:07.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.217+0000 7f70e4ad0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70e0135c20 0x7f70e01313a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:07.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.217+0000 7f70e4ad0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e009e8c0 con 0x7f70e006f5d0 2023-10-03T10:13:07.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.217+0000 7f70e4ad0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e01323c0 con 0x7f70e0135c20 2023-10-03T10:13:07.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.217+0000 7f70e4ad0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f70e0133a30 con 0x7f70e012cf60 2023-10-03T10:13:07.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.217+0000 7f70ddd9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70e012cf60 0x7f70e0130e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:07.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.217+0000 7f70ddd9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70e012cf60 0x7f70e0130e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48918/0 (socket says 172.21.15.3:48918) 2023-10-03T10:13:07.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.217+0000 7f70ddd9b700 1 -- 172.21.15.3:0/672974938 learned_addr learned my addr 172.21.15.3:0/672974938 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:07.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.217+0000 7f70ded9d700 1 --2- 172.21.15.3:0/672974938 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70e0135c20 0x7f70e01313a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:07.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.217+0000 7f70de59c700 1 --2- 172.21.15.3:0/672974938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e006f5d0 0x7f70e0130920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:07.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.217+0000 7f70ddd9b700 1 -- 172.21.15.3:0/672974938 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70e0135c20 msgr2=0x7f70e01313a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:07.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.217+0000 7f70ddd9b700 1 --2- 172.21.15.3:0/672974938 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70e0135c20 0x7f70e01313a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.217+0000 7f70ddd9b700 1 -- 172.21.15.3:0/672974938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e006f5d0 msgr2=0x7f70e0130920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:07.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.217+0000 7f70ddd9b700 1 --2- 172.21.15.3:0/672974938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e006f5d0 0x7f70e0130920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.217+0000 7f70ddd9b700 1 -- 172.21.15.3:0/672974938 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d8009a70 con 0x7f70e012cf60 2023-10-03T10:13:07.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.217+0000 7f70ded9d700 1 --2- 172.21.15.3:0/672974938 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70e0135c20 0x7f70e01313a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:07.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.218+0000 7f70ddd9b700 1 --2- 172.21.15.3:0/672974938 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70e012cf60 0x7f70e0130e60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f70d400e9b0 tx=0x7f70d400ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:07.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.218+0000 7f70ceffd700 1 -- 172.21.15.3:0/672974938 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4009710 con 0x7f70e012cf60 2023-10-03T10:13:07.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.218+0000 7f70ceffd700 1 -- 172.21.15.3:0/672974938 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d400c850 con 0x7f70e012cf60 2023-10-03T10:13:07.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.218+0000 7f70ceffd700 1 -- 172.21.15.3:0/672974938 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d400cbc0 con 0x7f70e012cf60 2023-10-03T10:13:07.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.218+0000 7f70e4ad0700 1 -- 172.21.15.3:0/672974938 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e012a000 con 0x7f70e012cf60 2023-10-03T10:13:07.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.218+0000 7f70e4ad0700 1 -- 172.21.15.3:0/672974938 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e009ddb0 con 0x7f70e012cf60 2023-10-03T10:13:07.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.219+0000 7f70e4ad0700 1 -- 172.21.15.3:0/672974938 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e0074ea0 con 0x7f70e012cf60 2023-10-03T10:13:07.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.221+0000 7f70ceffd700 1 -- 172.21.15.3:0/672974938 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f70d400cd20 con 0x7f70e012cf60 2023-10-03T10:13:07.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.221+0000 7f70de59c700 1 -- 172.21.15.3:0/672974938 >> v1:172.21.15.3:6816/127025 conn(0x7f70c002f540 legacy=0x7f70c00319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:07.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.221+0000 7f70ceffd700 1 -- 172.21.15.3:0/672974938 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f70d403fe10 con 0x7f70e012cf60 2023-10-03T10:13:07.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.231+0000 7f70ceffd700 1 -- 172.21.15.3:0/672974938 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f70d40232f0 con 0x7f70e012cf60 2023-10-03T10:13:07.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.321+0000 7fe733fff700 1 -- 172.21.15.3:0/1059071041 >> v1:172.21.15.3:6816/127025 conn(0x7fe71802f510 legacy=0x7fe7180319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:07.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.348+0000 7fe73ad35700 1 -- 172.21.15.3:0/1059071041 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe7341219a0 con 0x7fe73409d6f0 2023-10-03T10:13:07.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.349+0000 7fe7317fa700 1 -- 172.21.15.3:0/1059071041 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe728011360 con 0x7fe73409d6f0 2023-10-03T10:13:07.349 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:07.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.351+0000 7fe73ad35700 1 -- 172.21.15.3:0/1059071041 >> v1:172.21.15.3:6816/127025 conn(0x7fe71802f510 legacy=0x7fe7180319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:07.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.351+0000 7fe73ad35700 1 -- 172.21.15.3:0/1059071041 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe73409d6f0 msgr2=0x7fe73411e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:07.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.351+0000 7fe73ad35700 1 --2- 172.21.15.3:0/1059071041 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe73409d6f0 0x7fe73411e620 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe72800c550 tx=0x7fe72800c860 comp rx=0 tx=0).stop 2023-10-03T10:13:07.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.351+0000 7fe73ad35700 1 -- 172.21.15.3:0/1059071041 shutdown_connections 2023-10-03T10:13:07.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.351+0000 7fe73ad35700 1 --2- 172.21.15.3:0/1059071041 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe734139170 0x7fe73411f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.351+0000 7fe73ad35700 1 --2- 172.21.15.3:0/1059071041 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe73409d6f0 0x7fe73411e620 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.351+0000 7fe73ad35700 1 --2- 172.21.15.3:0/1059071041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe73409e1d0 0x7fe73411eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.351+0000 7fe73ad35700 1 -- 172.21.15.3:0/1059071041 >> 172.21.15.3:0/1059071041 conn(0x7fe734066f50 msgr2=0x7fe734136b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:07.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.351+0000 7fe73ad35700 1 -- 172.21.15.3:0/1059071041 shutdown_connections 2023-10-03T10:13:07.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.351+0000 7fe73ad35700 1 -- 172.21.15.3:0/1059071041 wait complete. 2023-10-03T10:13:07.365 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:07.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.371+0000 7f19bf7fe700 1 -- 172.21.15.3:0/3811211813 >> v1:172.21.15.3:6816/127025 conn(0x7f19ac02f540 legacy=0x7f19ac0319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:07.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.394+0000 7f19c5b09700 1 -- 172.21.15.3:0/3811211813 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f19c006b250 con 0x7f19c0130a40 2023-10-03T10:13:07.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.395+0000 7f19a7fff700 1 -- 172.21.15.3:0/3811211813 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f19b4019b40 con 0x7f19c0130a40 2023-10-03T10:13:07.395 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:07.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.397+0000 7f19c5b09700 1 -- 172.21.15.3:0/3811211813 >> v1:172.21.15.3:6816/127025 conn(0x7f19ac02f540 legacy=0x7f19ac0319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:07.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.397+0000 7f19c5b09700 1 -- 172.21.15.3:0/3811211813 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19c0130a40 msgr2=0x7f19c009c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:07.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.397+0000 7f19c5b09700 1 --2- 172.21.15.3:0/3811211813 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19c0130a40 0x7f19c009c9c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f19c009ac40 tx=0x7f19b400c800 comp rx=0 tx=0).stop 2023-10-03T10:13:07.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.397+0000 7f19c5b09700 1 -- 172.21.15.3:0/3811211813 shutdown_connections 2023-10-03T10:13:07.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.397+0000 7f19c5b09700 1 --2- 172.21.15.3:0/3811211813 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19c0133a80 0x7f19c009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.397+0000 7f19c5b09700 1 --2- 172.21.15.3:0/3811211813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19c012ff20 0x7f19c009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.397+0000 7f19c5b09700 1 --2- 172.21.15.3:0/3811211813 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19c0130a40 0x7f19c009c9c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.397+0000 7f19c5b09700 1 -- 172.21.15.3:0/3811211813 >> 172.21.15.3:0/3811211813 conn(0x7f19c00a4210 msgr2=0x7f19c012f680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:07.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.397+0000 7f19c5b09700 1 -- 172.21.15.3:0/3811211813 shutdown_connections 2023-10-03T10:13:07.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.397+0000 7f19c5b09700 1 -- 172.21.15.3:0/3811211813 wait complete. 2023-10-03T10:13:07.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.407+0000 7f21ba59c700 1 -- 172.21.15.3:0/508963432 >> v1:172.21.15.3:6816/127025 conn(0x7f219c02f510 legacy=0x7f219c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:07.411 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:07.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.422+0000 7f70de59c700 1 -- 172.21.15.3:0/672974938 >> v1:172.21.15.3:6816/127025 conn(0x7f70c002f540 legacy=0x7f70c00319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:07.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.445+0000 7f21c0837700 1 -- 172.21.15.3:0/508963432 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f21bc09e980 con 0x7f21bc1316c0 2023-10-03T10:13:07.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.445+0000 7f21aaffd700 1 -- 172.21.15.3:0/508963432 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f21b0003bc0 con 0x7f21bc1316c0 2023-10-03T10:13:07.446 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:07.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.449+0000 7f21c0837700 1 -- 172.21.15.3:0/508963432 >> v1:172.21.15.3:6816/127025 conn(0x7f219c02f510 legacy=0x7f219c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:07.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.449+0000 7f21c0837700 1 -- 172.21.15.3:0/508963432 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21bc1316c0 msgr2=0x7f21bc071340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:07.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.449+0000 7f21c0837700 1 --2- 172.21.15.3:0/508963432 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21bc1316c0 0x7f21bc071340 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009870 tx=0x7f21b0004750 comp rx=0 tx=0).stop 2023-10-03T10:13:07.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.449+0000 7f21c0837700 1 -- 172.21.15.3:0/508963432 shutdown_connections 2023-10-03T10:13:07.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.449+0000 7f21c0837700 1 --2- 172.21.15.3:0/508963432 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21bc136a70 0x7f21bc071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.449+0000 7f21c0837700 1 --2- 172.21.15.3:0/508963432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21bc09d9f0 0x7f21bc070e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.449+0000 7f21c0837700 1 --2- 172.21.15.3:0/508963432 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21bc1316c0 0x7f21bc071340 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.449+0000 7f21c0837700 1 -- 172.21.15.3:0/508963432 >> 172.21.15.3:0/508963432 conn(0x7f21bc066f50 msgr2=0x7f21bc138730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:07.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.449+0000 7f21c0837700 1 -- 172.21.15.3:0/508963432 shutdown_connections 2023-10-03T10:13:07.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.449+0000 7f21c0837700 1 -- 172.21.15.3:0/508963432 wait complete. 2023-10-03T10:13:07.468 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.469+0000 7f70e4ad0700 1 -- 172.21.15.3:0/672974938 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f70e00713a0 con 0x7f70e012cf60 2023-10-03T10:13:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.469+0000 7f70ceffd700 1 -- 172.21.15.3:0/672974938 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f70d4021070 con 0x7f70e012cf60 2023-10-03T10:13:07.469 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:07.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.471+0000 7f70e4ad0700 1 -- 172.21.15.3:0/672974938 >> v1:172.21.15.3:6816/127025 conn(0x7f70c002f540 legacy=0x7f70c00319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:07.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.471+0000 7f70e4ad0700 1 -- 172.21.15.3:0/672974938 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70e012cf60 msgr2=0x7f70e0130e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:07.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.471+0000 7f70e4ad0700 1 --2- 172.21.15.3:0/672974938 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70e012cf60 0x7f70e0130e60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f70d400e9b0 tx=0x7f70d400ed70 comp rx=0 tx=0).stop 2023-10-03T10:13:07.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.472+0000 7f70e4ad0700 1 -- 172.21.15.3:0/672974938 shutdown_connections 2023-10-03T10:13:07.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.472+0000 7f70e4ad0700 1 --2- 172.21.15.3:0/672974938 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70e0135c20 0x7f70e01313a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.472+0000 7f70e4ad0700 1 --2- 172.21.15.3:0/672974938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e006f5d0 0x7f70e0130920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.472+0000 7f70e4ad0700 1 --2- 172.21.15.3:0/672974938 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70e012cf60 0x7f70e0130e60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:07.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.472+0000 7f70e4ad0700 1 -- 172.21.15.3:0/672974938 >> 172.21.15.3:0/672974938 conn(0x7f70e00a2aa0 msgr2=0x7f70e00a1910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:07.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.472+0000 7f70e4ad0700 1 -- 172.21.15.3:0/672974938 shutdown_connections 2023-10-03T10:13:07.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:07.472+0000 7f70e4ad0700 1 -- 172.21.15.3:0/672974938 wait complete. 2023-10-03T10:13:07.485 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:08.366 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:08.412 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:08.468 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:08.486 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:08.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.518+0000 7f335835d700 1 Processor -- start 2023-10-03T10:13:08.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.519+0000 7f335835d700 1 -- start start 2023-10-03T10:13:08.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.520+0000 7f335835d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f335009d6f0 0x7f335009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:08.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.520+0000 7f335835d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f335009e000 0x7f33501339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:08.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.520+0000 7f335835d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f335009b120 con 0x7f335009e410 2023-10-03T10:13:08.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.520+0000 7f335835d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f335006b460 con 0x7f335009e000 2023-10-03T10:13:08.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.520+0000 7f335835d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f335009a7e0 con 0x7f335009d6f0 2023-10-03T10:13:08.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.520+0000 7f33560f9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f335009d6f0 0x7f335009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:08.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.520+0000 7f33560f9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f335009d6f0 0x7f335009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48920/0 (socket says 172.21.15.3:48920) 2023-10-03T10:13:08.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.520+0000 7f33560f9700 1 -- 172.21.15.3:0/2821335007 learned_addr learned my addr 172.21.15.3:0/2821335007 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:08.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.520+0000 7f33558f8700 1 --2- 172.21.15.3:0/2821335007 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f335009e000 0x7f33501339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:08.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.520+0000 7f33550f7700 1 -- 172.21.15.3:0/2821335007 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 446794571 0 0) 0x7f335009b120 con 0x7f335009e410 2023-10-03T10:13:08.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.521+0000 7f33558f8700 1 -- 172.21.15.3:0/2821335007 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f335009d6f0 msgr2=0x7f335009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:08.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.521+0000 7f33558f8700 1 --2- 172.21.15.3:0/2821335007 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f335009d6f0 0x7f335009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.521+0000 7f33558f8700 1 -- 172.21.15.3:0/2821335007 >> v1:172.21.15.3:6789/0 conn(0x7f335009e410 legacy=0x7f3350133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:08.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.521+0000 7f33558f8700 1 -- 172.21.15.3:0/2821335007 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335009a500 con 0x7f335009e000 2023-10-03T10:13:08.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.521+0000 7f33560f9700 1 --2- 172.21.15.3:0/2821335007 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f335009d6f0 0x7f335009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:08.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.521+0000 7f33558f8700 1 --2- 172.21.15.3:0/2821335007 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f335009e000 0x7f33501339f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3340009870 tx=0x7f3340009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6e1ecc5d97d8f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:08.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.521+0000 7f33550f7700 1 -- 172.21.15.3:0/2821335007 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3340011830 con 0x7f335009e000 2023-10-03T10:13:08.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.521+0000 7f33550f7700 1 -- 172.21.15.3:0/2821335007 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3340011990 con 0x7f335009e000 2023-10-03T10:13:08.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.521+0000 7f33550f7700 1 -- 172.21.15.3:0/2821335007 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3340011d20 con 0x7f335009e000 2023-10-03T10:13:08.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.522+0000 7f335835d700 1 -- 172.21.15.3:0/2821335007 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f335009e000 msgr2=0x7f33501339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:08.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.522+0000 7f335835d700 1 --2- 172.21.15.3:0/2821335007 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f335009e000 0x7f33501339f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3340009870 tx=0x7f3340009b80 comp rx=0 tx=0).stop 2023-10-03T10:13:08.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.522+0000 7f335835d700 1 -- 172.21.15.3:0/2821335007 shutdown_connections 2023-10-03T10:13:08.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.522+0000 7f335835d700 1 --2- 172.21.15.3:0/2821335007 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f335009d6f0 0x7f335009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.522+0000 7f335835d700 1 --2- 172.21.15.3:0/2821335007 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f335009e000 0x7f33501339f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.522+0000 7f335835d700 1 -- 172.21.15.3:0/2821335007 >> 172.21.15.3:0/2821335007 conn(0x7f3350066f50 msgr2=0x7f335009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:08.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.522+0000 7f335835d700 1 -- 172.21.15.3:0/2821335007 shutdown_connections 2023-10-03T10:13:08.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.522+0000 7f335835d700 1 -- 172.21.15.3:0/2821335007 wait complete. 2023-10-03T10:13:08.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.524+0000 7f335835d700 1 Processor -- start 2023-10-03T10:13:08.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.524+0000 7f335835d700 1 -- start start 2023-10-03T10:13:08.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.524+0000 7f335835d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f335009d6f0 0x7f33501224a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:08.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.524+0000 7f335835d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f335009e000 0x7f33501229e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:08.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.524+0000 7f335835d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f335009e410 0x7f3350122f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:08.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.524+0000 7f335835d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3350133790 con 0x7f335009e410 2023-10-03T10:13:08.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.524+0000 7f335835d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3350125c60 con 0x7f335009e000 2023-10-03T10:13:08.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.524+0000 7f335835d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f33500a2b90 con 0x7f335009d6f0 2023-10-03T10:13:08.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.525+0000 7f33560f9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f335009d6f0 0x7f33501224a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:08.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.525+0000 7f33568fa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f335009e410 0x7f3350122f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:08.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.525+0000 7f33560f9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f335009d6f0 0x7f33501224a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48936/0 (socket says 172.21.15.3:48936) 2023-10-03T10:13:08.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.525+0000 7f33560f9700 1 -- 172.21.15.3:0/1492767586 learned_addr learned my addr 172.21.15.3:0/1492767586 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:08.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.525+0000 7f33568fa700 1 -- 172.21.15.3:0/1492767586 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f335009d6f0 msgr2=0x7f33501224a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:08.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.525+0000 7f33568fa700 1 --2- 172.21.15.3:0/1492767586 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f335009d6f0 0x7f33501224a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.525+0000 7f33568fa700 1 -- 172.21.15.3:0/1492767586 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f335009e000 msgr2=0x7f33501229e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:08.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.525+0000 7f33568fa700 1 --2- 172.21.15.3:0/1492767586 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f335009e000 0x7f33501229e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.525+0000 7f33568fa700 1 -- 172.21.15.3:0/1492767586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334c009550 con 0x7f335009e410 2023-10-03T10:13:08.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.525+0000 7f33568fa700 1 --2- 172.21.15.3:0/1492767586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f335009e410 0x7f3350122f20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f335009e8b0 tx=0x7f334800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:08.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.526+0000 7f3346ffd700 1 -- 172.21.15.3:0/1492767586 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3348003e30 con 0x7f335009e410 2023-10-03T10:13:08.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.526+0000 7f3346ffd700 1 -- 172.21.15.3:0/1492767586 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3348005070 con 0x7f335009e410 2023-10-03T10:13:08.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.526+0000 7f335835d700 1 -- 172.21.15.3:0/1492767586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33400094a0 con 0x7f335009e410 2023-10-03T10:13:08.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.526+0000 7f3346ffd700 1 -- 172.21.15.3:0/1492767586 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33480053e0 con 0x7f335009e410 2023-10-03T10:13:08.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.526+0000 7f335835d700 1 -- 172.21.15.3:0/1492767586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33501266d0 con 0x7f335009e410 2023-10-03T10:13:08.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.527+0000 7f335835d700 1 -- 172.21.15.3:0/1492767586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3350074ea0 con 0x7f335009e410 2023-10-03T10:13:08.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.527+0000 7f3346ffd700 1 -- 172.21.15.3:0/1492767586 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f3348005540 con 0x7f335009e410 2023-10-03T10:13:08.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.528+0000 7f3346ffd700 1 -- 172.21.15.3:0/1492767586 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f334803f730 con 0x7f335009e410 2023-10-03T10:13:08.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.528+0000 7f33560f9700 1 -- 172.21.15.3:0/1492767586 >> v1:172.21.15.3:6816/127025 conn(0x7f333402f510 legacy=0x7f33340319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:08.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.534+0000 7f3346ffd700 1 -- 172.21.15.3:0/1492767586 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3348010300 con 0x7f335009e410 2023-10-03T10:13:08.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.560+0000 7f3679c17700 1 Processor -- start 2023-10-03T10:13:08.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.560+0000 7f3679c17700 1 -- start start 2023-10-03T10:13:08.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.561+0000 7f3679c17700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3674131810 0x7f3674131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:08.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.561+0000 7f3679c17700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3674099a40 0x7f3674099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:08.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.561+0000 7f3679c17700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36741313c0 con 0x7f367409a3b0 2023-10-03T10:13:08.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.561+0000 7f3679c17700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f367409b580 con 0x7f3674131810 2023-10-03T10:13:08.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.561+0000 7f3679c17700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f367406e8c0 con 0x7f3674099a40 2023-10-03T10:13:08.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.561+0000 7f36737fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3674131810 0x7f3674131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:08.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.561+0000 7f36737fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3674131810 0x7f3674131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40974/0 (socket says 172.21.15.3:40974) 2023-10-03T10:13:08.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.561+0000 7f36737fe700 1 -- 172.21.15.3:0/731274337 learned_addr learned my addr 172.21.15.3:0/731274337 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:08.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.561+0000 7f3673fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f367409a3b0 0x7f3674138650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49564/0 (socket says 172.21.15.3:49564) 2023-10-03T10:13:08.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.561+0000 7f36737fe700 1 -- 172.21.15.3:0/731274337 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3674099a40 msgr2=0x7f3674099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:08.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.561+0000 7f36737fe700 1 --2- 172.21.15.3:0/731274337 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3674099a40 0x7f3674099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.561+0000 7f36737fe700 1 -- 172.21.15.3:0/731274337 >> v1:172.21.15.3:6789/0 conn(0x7f367409a3b0 legacy=0x7f3674138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:08.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.561+0000 7f36727fc700 1 -- 172.21.15.3:0/731274337 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3606012117 0 0) 0x7f36741313c0 con 0x7f367409a3b0 2023-10-03T10:13:08.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.561+0000 7f36737fe700 1 -- 172.21.15.3:0/731274337 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f367409a820 con 0x7f3674131810 2023-10-03T10:13:08.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.561+0000 7f36737fe700 1 --2- 172.21.15.3:0/731274337 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3674131810 0x7f3674131be0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3664009920 tx=0x7f3664009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c3129e32ea6995e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:08.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.562+0000 7f36727fc700 1 -- 172.21.15.3:0/731274337 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366400ba60 con 0x7f3674131810 2023-10-03T10:13:08.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.562+0000 7f36727fc700 1 -- 172.21.15.3:0/731274337 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366400bbc0 con 0x7f3674131810 2023-10-03T10:13:08.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.562+0000 7f36727fc700 1 -- 172.21.15.3:0/731274337 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3664009e70 con 0x7f3674131810 2023-10-03T10:13:08.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.562+0000 7f3679c17700 1 -- 172.21.15.3:0/731274337 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3674131810 msgr2=0x7f3674131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:08.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.562+0000 7f3679c17700 1 --2- 172.21.15.3:0/731274337 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3674131810 0x7f3674131be0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3664009920 tx=0x7f3664009c30 comp rx=0 tx=0).stop 2023-10-03T10:13:08.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.563+0000 7f3679c17700 1 -- 172.21.15.3:0/731274337 shutdown_connections 2023-10-03T10:13:08.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.563+0000 7f3679c17700 1 --2- 172.21.15.3:0/731274337 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3674131810 0x7f3674131be0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.563+0000 7f3679c17700 1 --2- 172.21.15.3:0/731274337 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3674099a40 0x7f3674099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.563+0000 7f3679c17700 1 -- 172.21.15.3:0/731274337 >> 172.21.15.3:0/731274337 conn(0x7f36740a2aa0 msgr2=0x7f36740a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:08.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.563+0000 7f3679c17700 1 -- 172.21.15.3:0/731274337 shutdown_connections 2023-10-03T10:13:08.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.563+0000 7f3679c17700 1 -- 172.21.15.3:0/731274337 wait complete. 2023-10-03T10:13:08.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.564+0000 7f3679c17700 1 Processor -- start 2023-10-03T10:13:08.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.564+0000 7f3679c17700 1 -- start start 2023-10-03T10:13:08.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.565+0000 7f3679c17700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3674099a40 0x7f367411b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:08.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.565+0000 7f3679c17700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f367409a3b0 0x7f367411ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:08.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.565+0000 7f3679c17700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3674131810 0x7f367411bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:08.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.565+0000 7f3679c17700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367411d2d0 con 0x7f367409a3b0 2023-10-03T10:13:08.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.565+0000 7f3679c17700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f367411e410 con 0x7f3674131810 2023-10-03T10:13:08.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.565+0000 7f3679c17700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f367406eb00 con 0x7f3674099a40 2023-10-03T10:13:08.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.565+0000 7f36737fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3674099a40 0x7f367411b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:08.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.565+0000 7f36737fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3674099a40 0x7f367411b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48954/0 (socket says 172.21.15.3:48954) 2023-10-03T10:13:08.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.565+0000 7f36737fe700 1 -- 172.21.15.3:0/1476962974 learned_addr learned my addr 172.21.15.3:0/1476962974 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:08.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.566+0000 7f3673fff700 1 --2- 172.21.15.3:0/1476962974 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3674131810 0x7f367411bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:08.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.566+0000 7f3672ffd700 1 --2- 172.21.15.3:0/1476962974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f367409a3b0 0x7f367411ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:08.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.566+0000 7f36737fe700 1 -- 172.21.15.3:0/1476962974 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3674131810 msgr2=0x7f367411bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:08.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.566+0000 7f36737fe700 1 --2- 172.21.15.3:0/1476962974 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3674131810 0x7f367411bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.566+0000 7f36737fe700 1 -- 172.21.15.3:0/1476962974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f367409a3b0 msgr2=0x7f367411ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:08.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.566+0000 7f36737fe700 1 --2- 172.21.15.3:0/1476962974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f367409a3b0 0x7f367411ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.566+0000 7f36737fe700 1 -- 172.21.15.3:0/1476962974 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3664009550 con 0x7f3674099a40 2023-10-03T10:13:08.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.566+0000 7f36737fe700 1 --2- 172.21.15.3:0/1476962974 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3674099a40 0x7f367411b540 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3664000c00 tx=0x7f3664004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:08.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.566+0000 7f365bfff700 1 -- 172.21.15.3:0/1476962974 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3664004550 con 0x7f3674099a40 2023-10-03T10:13:08.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.566+0000 7f365bfff700 1 -- 172.21.15.3:0/1476962974 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36640036a0 con 0x7f3674099a40 2023-10-03T10:13:08.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.567+0000 7f3679c17700 1 -- 172.21.15.3:0/1476962974 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f367411eac0 con 0x7f3674099a40 2023-10-03T10:13:08.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.567+0000 7f365bfff700 1 -- 172.21.15.3:0/1476962974 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3664003a10 con 0x7f3674099a40 2023-10-03T10:13:08.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.567+0000 7f3679c17700 1 -- 172.21.15.3:0/1476962974 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f367411f060 con 0x7f3674099a40 2023-10-03T10:13:08.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.567+0000 7f365bfff700 1 -- 172.21.15.3:0/1476962974 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f3664003b70 con 0x7f3674099a40 2023-10-03T10:13:08.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.568+0000 7f365bfff700 1 -- 172.21.15.3:0/1476962974 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f3664040f90 con 0x7f3674099a40 2023-10-03T10:13:08.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.568+0000 7f3672ffd700 1 -- 172.21.15.3:0/1476962974 >> v1:172.21.15.3:6816/127025 conn(0x7f366002f540 legacy=0x7f36600319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:08.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.568+0000 7f3679c17700 1 -- 172.21.15.3:0/1476962974 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f367411ec50 con 0x7f3674099a40 2023-10-03T10:13:08.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.577+0000 7f365bfff700 1 -- 172.21.15.3:0/1476962974 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f367411ec50 con 0x7f3674099a40 2023-10-03T10:13:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.658+0000 7f77ecfc1700 1 Processor -- start 2023-10-03T10:13:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.658+0000 7f77ecfc1700 1 -- start start 2023-10-03T10:13:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.659+0000 7f77ecfc1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77e8133420 0x7f77e81337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.659+0000 7f77ecfc1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77e8133e30 0x7f77e812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.659+0000 7f77ecfc1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f77e8134260 0x7f77e81304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.659+0000 7f77ecfc1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77e806b990 con 0x7f77e8134260 2023-10-03T10:13:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.659+0000 7f77ecfc1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e80a0590 con 0x7f77e8133420 2023-10-03T10:13:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.659+0000 7f77ecfc1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77e809fb80 con 0x7f77e8133e30 2023-10-03T10:13:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.659+0000 7f77e6d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f77e8134260 0x7f77e81304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.659+0000 7f77e6d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f77e8134260 0x7f77e81304d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49004/0 (socket says 172.21.15.3:49004) 2023-10-03T10:13:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.659+0000 7f77e6d9d700 1 -- 172.21.15.3:0/1735139721 learned_addr learned my addr 172.21.15.3:0/1735139721 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.659+0000 7f77e659c700 1 --2- 172.21.15.3:0/1735139721 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77e8133420 0x7f77e81337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.659+0000 7f77e5d9b700 1 --2- 172.21.15.3:0/1735139721 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77e8133e30 0x7f77e812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.660+0000 7f77e6d9d700 1 -- 172.21.15.3:0/1735139721 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77e8133e30 msgr2=0x7f77e812ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.660+0000 7f77e6d9d700 1 --2- 172.21.15.3:0/1735139721 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77e8133e30 0x7f77e812ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.660+0000 7f77e6d9d700 1 -- 172.21.15.3:0/1735139721 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77e8133420 msgr2=0x7f77e81337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.660+0000 7f77e6d9d700 1 --2- 172.21.15.3:0/1735139721 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77e8133420 0x7f77e81337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.660+0000 7f77e6d9d700 1 -- 172.21.15.3:0/1735139721 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e80a0030 con 0x7f77e8134260 2023-10-03T10:13:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.660+0000 7f77e659c700 1 --2- 172.21.15.3:0/1735139721 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77e8133420 0x7f77e81337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.660+0000 7f77e6d9d700 1 --2- 172.21.15.3:0/1735139721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77e8134260 0x7f77e81304d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009e50 tx=0x7f77e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc527c7ba58f44cd server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.660+0000 7f77e559a700 1 -- 172.21.15.3:0/1735139721 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e000e070 con 0x7f77e8134260 2023-10-03T10:13:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.660+0000 7f77e559a700 1 -- 172.21.15.3:0/1735139721 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e000b910 con 0x7f77e8134260 2023-10-03T10:13:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.660+0000 7f77e559a700 1 -- 172.21.15.3:0/1735139721 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e000bcc0 con 0x7f77e8134260 2023-10-03T10:13:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.661+0000 7f77ecfc1700 1 -- 172.21.15.3:0/1735139721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77e8134260 msgr2=0x7f77e81304d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.661+0000 7f77ecfc1700 1 --2- 172.21.15.3:0/1735139721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77e8134260 0x7f77e81304d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009e50 tx=0x7f77e000b330 comp rx=0 tx=0).stop 2023-10-03T10:13:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.661+0000 7f77ecfc1700 1 -- 172.21.15.3:0/1735139721 shutdown_connections 2023-10-03T10:13:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.661+0000 7f77ecfc1700 1 --2- 172.21.15.3:0/1735139721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77e8134260 0x7f77e81304d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.661+0000 7f77ecfc1700 1 --2- 172.21.15.3:0/1735139721 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77e8133420 0x7f77e81337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.661+0000 7f77ecfc1700 1 --2- 172.21.15.3:0/1735139721 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77e8133e30 0x7f77e812ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.661+0000 7f77ecfc1700 1 -- 172.21.15.3:0/1735139721 >> 172.21.15.3:0/1735139721 conn(0x7f77e80a4210 msgr2=0x7f77e80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.661+0000 7f77ecfc1700 1 -- 172.21.15.3:0/1735139721 shutdown_connections 2023-10-03T10:13:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.661+0000 7f77ecfc1700 1 -- 172.21.15.3:0/1735139721 wait complete. 2023-10-03T10:13:08.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.663+0000 7f77ecfc1700 1 Processor -- start 2023-10-03T10:13:08.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.663+0000 7f77ecfc1700 1 -- start start 2023-10-03T10:13:08.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.664+0000 7f77ecfc1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77e8133420 0x7f77e809c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:08.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.664+0000 7f712db86700 1 Processor -- start 2023-10-03T10:13:08.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.664+0000 7f77ecfc1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77e8133e30 0x7f77e809cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:08.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.664+0000 7f77ecfc1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77e8134260 0x7f77e809d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:08.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.664+0000 7f77ecfc1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e81210a0 con 0x7f77e8134260 2023-10-03T10:13:08.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.664+0000 7f77ecfc1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e809dbb0 con 0x7f77e8133420 2023-10-03T10:13:08.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.664+0000 7f77ecfc1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77e812db70 con 0x7f77e8133e30 2023-10-03T10:13:08.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.664+0000 7f712db86700 1 -- start start 2023-10-03T10:13:08.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.664+0000 7f77e6d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77e8134260 0x7f77e809d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:08.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.664+0000 7f77e6d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77e8134260 0x7f77e809d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49008/0 (socket says 172.21.15.3:49008) 2023-10-03T10:13:08.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.664+0000 7f77e6d9d700 1 -- 172.21.15.3:0/1622033006 learned_addr learned my addr 172.21.15.3:0/1622033006 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:08.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.664+0000 7f77e5d9b700 1 --2- 172.21.15.3:0/1622033006 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77e8133e30 0x7f77e809cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:08.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.665+0000 7f77e6d9d700 1 -- 172.21.15.3:0/1622033006 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77e8133e30 msgr2=0x7f77e809cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:08.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.665+0000 7f77e6d9d700 1 --2- 172.21.15.3:0/1622033006 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77e8133e30 0x7f77e809cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.665+0000 7f77e6d9d700 1 -- 172.21.15.3:0/1622033006 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77e8133420 msgr2=0x7f77e809c6e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:08.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.665+0000 7f77e6d9d700 1 --2- 172.21.15.3:0/1622033006 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77e8133420 0x7f77e809c6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.665+0000 7f77e6d9d700 1 -- 172.21.15.3:0/1622033006 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d80094a0 con 0x7f77e8134260 2023-10-03T10:13:08.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.665+0000 7f712db86700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f712806a770 0x7f712806ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:08.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.665+0000 7f712db86700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f712806b080 0x7f712813c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:08.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.665+0000 7f77e6d9d700 1 --2- 172.21.15.3:0/1622033006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77e8134260 0x7f77e809d160 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f77e806bad0 tx=0x7f77e0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:08.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.665+0000 7f712db86700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f712813c560 0x7f712813e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:08.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.665+0000 7f712db86700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71280a6030 con 0x7f712813c560 2023-10-03T10:13:08.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.665+0000 7f712db86700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712809e1c0 con 0x7f712806a770 2023-10-03T10:13:08.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.665+0000 7f712db86700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f712809dfc0 con 0x7f712806b080 2023-10-03T10:13:08.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.665+0000 7f7127fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f712813c560 0x7f712813e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:08.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.665+0000 7f7127fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f712813c560 0x7f712813e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49024/0 (socket says 172.21.15.3:49024) 2023-10-03T10:13:08.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.665+0000 7f7127fff700 1 -- 172.21.15.3:0/2041486680 learned_addr learned my addr 172.21.15.3:0/2041486680 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:08.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.665+0000 7f71277fe700 1 --2- 172.21.15.3:0/2041486680 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f712806a770 0x7f712806ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:08.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.665+0000 7f77d6ffd700 1 -- 172.21.15.3:0/1622033006 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e0004450 con 0x7f77e8134260 2023-10-03T10:13:08.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.665+0000 7f77d6ffd700 1 -- 172.21.15.3:0/1622033006 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e00045b0 con 0x7f77e8134260 2023-10-03T10:13:08.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.665+0000 7f77d6ffd700 1 -- 172.21.15.3:0/1622033006 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e0004960 con 0x7f77e8134260 2023-10-03T10:13:08.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.665+0000 7f77ecfc1700 1 -- 172.21.15.3:0/1622033006 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e0009b30 con 0x7f77e8134260 2023-10-03T10:13:08.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.665+0000 7f77ecfc1700 1 -- 172.21.15.3:0/1622033006 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e8120140 con 0x7f77e8134260 2023-10-03T10:13:08.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.665+0000 7f7127fff700 1 -- 172.21.15.3:0/2041486680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f712806b080 msgr2=0x7f712813c020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-03T10:13:08.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.665+0000 7f7127fff700 1 --2- 172.21.15.3:0/2041486680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f712806b080 0x7f712813c020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.665+0000 7f7127fff700 1 -- 172.21.15.3:0/2041486680 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f712806a770 msgr2=0x7f712806ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:08.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.665+0000 7f7127fff700 1 --2- 172.21.15.3:0/2041486680 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f712806a770 0x7f712806ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.665+0000 7f7127fff700 1 -- 172.21.15.3:0/2041486680 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71280a4f80 con 0x7f712813c560 2023-10-03T10:13:08.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.666+0000 7f7127fff700 1 --2- 172.21.15.3:0/2041486680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f712813c560 0x7f712813e940 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7120009e80 tx=0x7f712000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5962053eff09baa server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:08.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.666+0000 7f71267fc700 1 -- 172.21.15.3:0/2041486680 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712000e070 con 0x7f712813c560 2023-10-03T10:13:08.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.666+0000 7f71267fc700 1 -- 172.21.15.3:0/2041486680 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712000b8d0 con 0x7f712813c560 2023-10-03T10:13:08.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.666+0000 7f71267fc700 1 -- 172.21.15.3:0/2041486680 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712000bc80 con 0x7f712813c560 2023-10-03T10:13:08.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.667+0000 7f77ecfc1700 1 -- 172.21.15.3:0/1622033006 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e806d280 con 0x7f77e8134260 2023-10-03T10:13:08.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.667+0000 7f712db86700 1 -- 172.21.15.3:0/2041486680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f712813c560 msgr2=0x7f712813e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:08.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.667+0000 7f712db86700 1 --2- 172.21.15.3:0/2041486680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f712813c560 0x7f712813e940 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7120009e80 tx=0x7f712000b330 comp rx=0 tx=0).stop 2023-10-03T10:13:08.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.667+0000 7f712db86700 1 -- 172.21.15.3:0/2041486680 shutdown_connections 2023-10-03T10:13:08.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.667+0000 7f712db86700 1 --2- 172.21.15.3:0/2041486680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f712813c560 0x7f712813e940 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.667+0000 7f712db86700 1 --2- 172.21.15.3:0/2041486680 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f712806a770 0x7f712806ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.667+0000 7f712db86700 1 --2- 172.21.15.3:0/2041486680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f712806b080 0x7f712813c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.667+0000 7f712db86700 1 -- 172.21.15.3:0/2041486680 >> 172.21.15.3:0/2041486680 conn(0x7f7128066d30 msgr2=0x7f712809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:08.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.667+0000 7f712db86700 1 -- 172.21.15.3:0/2041486680 shutdown_connections 2023-10-03T10:13:08.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.667+0000 7f712db86700 1 -- 172.21.15.3:0/2041486680 wait complete. 2023-10-03T10:13:08.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.667+0000 7f77d6ffd700 1 -- 172.21.15.3:0/1622033006 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f77e0025460 con 0x7f77e8134260 2023-10-03T10:13:08.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.668+0000 7f77d6ffd700 1 -- 172.21.15.3:0/1622033006 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f77e0041c60 con 0x7f77e8134260 2023-10-03T10:13:08.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.668+0000 7f77e659c700 1 -- 172.21.15.3:0/1622033006 >> v1:172.21.15.3:6816/127025 conn(0x7f77c802f560 legacy=0x7f77c8031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:08.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.669+0000 7f712db86700 1 Processor -- start 2023-10-03T10:13:08.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.669+0000 7f712db86700 1 -- start start 2023-10-03T10:13:08.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.670+0000 7f712db86700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f712806a770 0x7f7128126de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:08.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.670+0000 7f712db86700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f712806b080 0x7f7128127320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:08.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.670+0000 7f712db86700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f712813c560 0x7f7128127860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:08.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.670+0000 7f712db86700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712812a070 con 0x7f712813c560 2023-10-03T10:13:08.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.670+0000 7f712db86700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712813b890 con 0x7f712806a770 2023-10-03T10:13:08.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.670+0000 7f712db86700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7128128a90 con 0x7f712806b080 2023-10-03T10:13:08.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.673+0000 7f7127fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f712813c560 0x7f7128127860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:08.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.673+0000 7f7127fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f712813c560 0x7f7128127860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49026/0 (socket says 172.21.15.3:49026) 2023-10-03T10:13:08.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.673+0000 7f7127fff700 1 -- 172.21.15.3:0/3265759370 learned_addr learned my addr 172.21.15.3:0/3265759370 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:08.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.673+0000 7f7127fff700 1 -- 172.21.15.3:0/3265759370 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f712806b080 msgr2=0x7f7128127320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:08.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.673+0000 7f7127fff700 1 --2- 172.21.15.3:0/3265759370 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f712806b080 0x7f7128127320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.673+0000 7f7127fff700 1 -- 172.21.15.3:0/3265759370 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f712806a770 msgr2=0x7f7128126de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:08.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.673+0000 7f7127fff700 1 --2- 172.21.15.3:0/3265759370 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f712806a770 0x7f7128126de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.673+0000 7f7127fff700 1 -- 172.21.15.3:0/3265759370 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7120009b30 con 0x7f712813c560 2023-10-03T10:13:08.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.673+0000 7f7127fff700 1 --2- 172.21.15.3:0/3265759370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f712813c560 0x7f7128127860 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f712000b680 tx=0x7f7120004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:08.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.674+0000 7f710ffff700 1 -- 172.21.15.3:0/3265759370 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7120004450 con 0x7f712813c560 2023-10-03T10:13:08.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.674+0000 7f710ffff700 1 -- 172.21.15.3:0/3265759370 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71200045b0 con 0x7f712813c560 2023-10-03T10:13:08.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.674+0000 7f712db86700 1 -- 172.21.15.3:0/3265759370 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f712813bd50 con 0x7f712813c560 2023-10-03T10:13:08.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.674+0000 7f77d6ffd700 1 -- 172.21.15.3:0/1622033006 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f77e001b3a0 con 0x7f77e8134260 2023-10-03T10:13:08.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.674+0000 7f710ffff700 1 -- 172.21.15.3:0/3265759370 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7120004920 con 0x7f712813c560 2023-10-03T10:13:08.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.674+0000 7f712db86700 1 -- 172.21.15.3:0/3265759370 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71280a5410 con 0x7f712813c560 2023-10-03T10:13:08.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.675+0000 7f710ffff700 1 -- 172.21.15.3:0/3265759370 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f712001c440 con 0x7f712813c560 2023-10-03T10:13:08.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.675+0000 7f710dffb700 1 -- 172.21.15.3:0/3265759370 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7128074ea0 con 0x7f712813c560 2023-10-03T10:13:08.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.675+0000 7f710ffff700 1 -- 172.21.15.3:0/3265759370 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f71200415d0 con 0x7f712813c560 2023-10-03T10:13:08.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.675+0000 7f71277fe700 1 -- 172.21.15.3:0/3265759370 >> v1:172.21.15.3:6816/127025 conn(0x7f711402f510 legacy=0x7f71140319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:08.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.680+0000 7f710ffff700 1 -- 172.21.15.3:0/3265759370 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7120023770 con 0x7f712813c560 2023-10-03T10:13:08.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.729+0000 7f33560f9700 1 -- 172.21.15.3:0/1492767586 >> v1:172.21.15.3:6816/127025 conn(0x7f333402f510 legacy=0x7f33340319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:08.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.769+0000 7f3672ffd700 1 -- 172.21.15.3:0/1476962974 >> v1:172.21.15.3:6816/127025 conn(0x7f366002f540 legacy=0x7f36600319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:08.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.788+0000 7f3679c17700 1 -- 172.21.15.3:0/1476962974 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f367411ec50 con 0x7f3674099a40 2023-10-03T10:13:08.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.789+0000 7f365bfff700 1 -- 172.21.15.3:0/1476962974 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f367411ec50 con 0x7f3674099a40 2023-10-03T10:13:08.789 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:08.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.791+0000 7f3679c17700 1 -- 172.21.15.3:0/1476962974 >> v1:172.21.15.3:6816/127025 conn(0x7f366002f540 legacy=0x7f36600319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:08.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.791+0000 7f3679c17700 1 -- 172.21.15.3:0/1476962974 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3674099a40 msgr2=0x7f367411b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:08.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.791+0000 7f3679c17700 1 --2- 172.21.15.3:0/1476962974 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3674099a40 0x7f367411b540 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3664000c00 tx=0x7f3664004750 comp rx=0 tx=0).stop 2023-10-03T10:13:08.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.791+0000 7f3679c17700 1 -- 172.21.15.3:0/1476962974 shutdown_connections 2023-10-03T10:13:08.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.791+0000 7f3679c17700 1 --2- 172.21.15.3:0/1476962974 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3674131810 0x7f367411bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.791+0000 7f3679c17700 1 --2- 172.21.15.3:0/1476962974 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3674099a40 0x7f367411b540 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.791+0000 7f3679c17700 1 --2- 172.21.15.3:0/1476962974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f367409a3b0 0x7f367411ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.791+0000 7f3679c17700 1 -- 172.21.15.3:0/1476962974 >> 172.21.15.3:0/1476962974 conn(0x7f36740a2aa0 msgr2=0x7f36741b8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:08.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.791+0000 7f3679c17700 1 -- 172.21.15.3:0/1476962974 shutdown_connections 2023-10-03T10:13:08.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.791+0000 7f3679c17700 1 -- 172.21.15.3:0/1476962974 wait complete. 2023-10-03T10:13:08.806 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:08.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.823+0000 7f335835d700 1 -- 172.21.15.3:0/1492767586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3350125900 con 0x7f335009e410 2023-10-03T10:13:08.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.823+0000 7f3346ffd700 1 -- 172.21.15.3:0/1492767586 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3348019e90 con 0x7f335009e410 2023-10-03T10:13:08.823 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:08.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.826+0000 7f335835d700 1 -- 172.21.15.3:0/1492767586 >> v1:172.21.15.3:6816/127025 conn(0x7f333402f510 legacy=0x7f33340319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:08.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.826+0000 7f335835d700 1 -- 172.21.15.3:0/1492767586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f335009e410 msgr2=0x7f3350122f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:08.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.826+0000 7f335835d700 1 --2- 172.21.15.3:0/1492767586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f335009e410 0x7f3350122f20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f335009e8b0 tx=0x7f334800ca10 comp rx=0 tx=0).stop 2023-10-03T10:13:08.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.826+0000 7f335835d700 1 -- 172.21.15.3:0/1492767586 shutdown_connections 2023-10-03T10:13:08.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.826+0000 7f335835d700 1 --2- 172.21.15.3:0/1492767586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f335009e410 0x7f3350122f20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.826+0000 7f335835d700 1 --2- 172.21.15.3:0/1492767586 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f335009d6f0 0x7f33501224a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.826+0000 7f335835d700 1 --2- 172.21.15.3:0/1492767586 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f335009e000 0x7f33501229e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.826+0000 7f335835d700 1 -- 172.21.15.3:0/1492767586 >> 172.21.15.3:0/1492767586 conn(0x7f3350066f50 msgr2=0x7f33500a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:08.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.826+0000 7f335835d700 1 -- 172.21.15.3:0/1492767586 shutdown_connections 2023-10-03T10:13:08.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.826+0000 7f335835d700 1 -- 172.21.15.3:0/1492767586 wait complete. 2023-10-03T10:13:08.840 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:08.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.869+0000 7f77e659c700 1 -- 172.21.15.3:0/1622033006 >> v1:172.21.15.3:6816/127025 conn(0x7f77c802f560 legacy=0x7f77c8031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:08.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.877+0000 7f71277fe700 1 -- 172.21.15.3:0/3265759370 >> v1:172.21.15.3:6816/127025 conn(0x7f711402f510 legacy=0x7f71140319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:08.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.888+0000 7f710dffb700 1 -- 172.21.15.3:0/3265759370 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f71280a4290 con 0x7f712813c560 2023-10-03T10:13:08.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.888+0000 7f710ffff700 1 -- 172.21.15.3:0/3265759370 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7120013e80 con 0x7f712813c560 2023-10-03T10:13:08.889 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:08.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.891+0000 7f710dffb700 1 -- 172.21.15.3:0/3265759370 >> v1:172.21.15.3:6816/127025 conn(0x7f711402f510 legacy=0x7f71140319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:08.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.891+0000 7f710dffb700 1 -- 172.21.15.3:0/3265759370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f712813c560 msgr2=0x7f7128127860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:08.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.891+0000 7f710dffb700 1 --2- 172.21.15.3:0/3265759370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f712813c560 0x7f7128127860 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f712000b680 tx=0x7f7120004ad0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.891+0000 7f710dffb700 1 -- 172.21.15.3:0/3265759370 shutdown_connections 2023-10-03T10:13:08.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.891+0000 7f710dffb700 1 --2- 172.21.15.3:0/3265759370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f712813c560 0x7f7128127860 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.891+0000 7f710dffb700 1 --2- 172.21.15.3:0/3265759370 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f712806a770 0x7f7128126de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.891+0000 7f710dffb700 1 --2- 172.21.15.3:0/3265759370 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f712806b080 0x7f7128127320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.891+0000 7f710dffb700 1 -- 172.21.15.3:0/3265759370 >> 172.21.15.3:0/3265759370 conn(0x7f7128066d30 msgr2=0x7f71280a20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:08.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.891+0000 7f710dffb700 1 -- 172.21.15.3:0/3265759370 shutdown_connections 2023-10-03T10:13:08.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.891+0000 7f710dffb700 1 -- 172.21.15.3:0/3265759370 wait complete. 2023-10-03T10:13:08.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.900+0000 7f77ecfc1700 1 -- 172.21.15.3:0/1622033006 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f77e8074ea0 con 0x7f77e8134260 2023-10-03T10:13:08.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.900+0000 7f77d6ffd700 1 -- 172.21.15.3:0/1622033006 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f77e001abb0 con 0x7f77e8134260 2023-10-03T10:13:08.901 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:08.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.903+0000 7f77ecfc1700 1 -- 172.21.15.3:0/1622033006 >> v1:172.21.15.3:6816/127025 conn(0x7f77c802f560 legacy=0x7f77c8031a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:08.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.903+0000 7f77ecfc1700 1 -- 172.21.15.3:0/1622033006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77e8134260 msgr2=0x7f77e809d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:08.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.903+0000 7f77ecfc1700 1 --2- 172.21.15.3:0/1622033006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77e8134260 0x7f77e809d160 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f77e806bad0 tx=0x7f77e0004ad0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.903+0000 7f77ecfc1700 1 -- 172.21.15.3:0/1622033006 shutdown_connections 2023-10-03T10:13:08.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.903+0000 7f77ecfc1700 1 --2- 172.21.15.3:0/1622033006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77e8134260 0x7f77e809d160 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.903+0000 7f77ecfc1700 1 --2- 172.21.15.3:0/1622033006 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77e8133420 0x7f77e809c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.903+0000 7f77ecfc1700 1 --2- 172.21.15.3:0/1622033006 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77e8133e30 0x7f77e809cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:08.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.903+0000 7f77ecfc1700 1 -- 172.21.15.3:0/1622033006 >> 172.21.15.3:0/1622033006 conn(0x7f77e80a4210 msgr2=0x7f77e80a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:08.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.903+0000 7f77ecfc1700 1 -- 172.21.15.3:0/1622033006 shutdown_connections 2023-10-03T10:13:08.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:08.903+0000 7f77ecfc1700 1 -- 172.21.15.3:0/1622033006 wait complete. 2023-10-03T10:13:08.905 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:08.918 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:09.806 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:09.840 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:09.906 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:09.918 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:09.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.958+0000 7f9c3a13d700 1 Processor -- start 2023-10-03T10:13:09.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.958+0000 7f9c3a13d700 1 -- start start 2023-10-03T10:13:09.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.959+0000 7f9c3a13d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9c341338c0 0x7f9c34133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:09.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.959+0000 7f9c3a13d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c3409e560 0x7f9c3413b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:09.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.959+0000 7f9c3a13d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c3406f720 con 0x7f9c341341d0 2023-10-03T10:13:09.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.959+0000 7f9c3a13d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c34070c80 con 0x7f9c341338c0 2023-10-03T10:13:09.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.959+0000 7f9c3a13d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3406fce0 con 0x7f9c3409e560 2023-10-03T10:13:09.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.959+0000 7f9c337fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9c341338c0 0x7f9c34133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:09.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.959+0000 7f9c33fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c3409e560 0x7f9c3413b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:09.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.959+0000 7f9c32ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9c341341d0 0x7f9c3409de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49566/0 (socket says 172.21.15.3:49566) 2023-10-03T10:13:09.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.959+0000 7f9c32ffd700 1 -- 172.21.15.3:0/806443823 learned_addr learned my addr 172.21.15.3:0/806443823 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:09.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.959+0000 7f9c337fe700 1 -- 172.21.15.3:0/806443823 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c3409e560 msgr2=0x7f9c3413b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:09.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.959+0000 7f9c337fe700 1 --2- 172.21.15.3:0/806443823 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c3409e560 0x7f9c3413b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:09.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.959+0000 7f9c337fe700 1 -- 172.21.15.3:0/806443823 >> v1:172.21.15.3:6789/0 conn(0x7f9c341341d0 legacy=0x7f9c3409de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:09.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.959+0000 7f9c337fe700 1 -- 172.21.15.3:0/806443823 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c341336a0 con 0x7f9c341338c0 2023-10-03T10:13:09.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.960+0000 7f9c337fe700 1 --2- 172.21.15.3:0/806443823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c341338c0 0x7f9c34133c90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009840 tx=0x7f9c24009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8590dd4fbd55c91 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:09.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.960+0000 7f9c327fc700 1 -- 172.21.15.3:0/806443823 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c24011830 con 0x7f9c341338c0 2023-10-03T10:13:09.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.960+0000 7f9c327fc700 1 -- 172.21.15.3:0/806443823 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c24011990 con 0x7f9c341338c0 2023-10-03T10:13:09.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.960+0000 7f9c327fc700 1 -- 172.21.15.3:0/806443823 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c24011d20 con 0x7f9c341338c0 2023-10-03T10:13:09.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.961+0000 7f9c3a13d700 1 -- 172.21.15.3:0/806443823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c341338c0 msgr2=0x7f9c34133c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:09.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.961+0000 7f9c3a13d700 1 --2- 172.21.15.3:0/806443823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c341338c0 0x7f9c34133c90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009840 tx=0x7f9c24009b50 comp rx=0 tx=0).stop 2023-10-03T10:13:09.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.961+0000 7f9c3a13d700 1 -- 172.21.15.3:0/806443823 shutdown_connections 2023-10-03T10:13:09.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.961+0000 7f9c3a13d700 1 --2- 172.21.15.3:0/806443823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c341338c0 0x7f9c34133c90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:09.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.961+0000 7f9c3a13d700 1 --2- 172.21.15.3:0/806443823 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c3409e560 0x7f9c3413b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:09.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.961+0000 7f9c3a13d700 1 -- 172.21.15.3:0/806443823 >> 172.21.15.3:0/806443823 conn(0x7f9c340a2aa0 msgr2=0x7f9c340a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:09.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.961+0000 7f9c3a13d700 1 -- 172.21.15.3:0/806443823 shutdown_connections 2023-10-03T10:13:09.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.962+0000 7f9c3a13d700 1 -- 172.21.15.3:0/806443823 wait complete. 2023-10-03T10:13:09.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.963+0000 7f9c3a13d700 1 Processor -- start 2023-10-03T10:13:09.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.963+0000 7f9c3a13d700 1 -- start start 2023-10-03T10:13:09.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.964+0000 7f9c3a13d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c3409e560 0x7f9c3411fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:09.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.964+0000 7f9c3a13d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c341338c0 0x7f9c341200c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:09.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.964+0000 7f9c3a13d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c341341d0 0x7f9c34120600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:09.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.964+0000 7f9c3a13d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c34122e10 con 0x7f9c341341d0 2023-10-03T10:13:09.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.964+0000 7f9c3a13d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c34130730 con 0x7f9c3409e560 2023-10-03T10:13:09.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.964+0000 7f9c3a13d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c34123190 con 0x7f9c341338c0 2023-10-03T10:13:09.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.964+0000 7f9c32ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c341338c0 0x7f9c341200c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:09.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.964+0000 7f9c32ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c341338c0 0x7f9c341200c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48996/0 (socket says 172.21.15.3:48996) 2023-10-03T10:13:09.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.964+0000 7f9c32ffd700 1 -- 172.21.15.3:0/3245309399 learned_addr learned my addr 172.21.15.3:0/3245309399 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:09.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.964+0000 7f9c337fe700 1 --2- 172.21.15.3:0/3245309399 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c3409e560 0x7f9c3411fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:09.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.964+0000 7f9c33fff700 1 --2- 172.21.15.3:0/3245309399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c341341d0 0x7f9c34120600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:09.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.965+0000 7f9c32ffd700 1 -- 172.21.15.3:0/3245309399 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c3409e560 msgr2=0x7f9c3411fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:09.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.965+0000 7f9c32ffd700 1 --2- 172.21.15.3:0/3245309399 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c3409e560 0x7f9c3411fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:09.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.965+0000 7f9c32ffd700 1 -- 172.21.15.3:0/3245309399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c341341d0 msgr2=0x7f9c34120600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:09.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.965+0000 7f9c32ffd700 1 --2- 172.21.15.3:0/3245309399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c341341d0 0x7f9c34120600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:09.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.965+0000 7f9c32ffd700 1 -- 172.21.15.3:0/3245309399 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c240094a0 con 0x7f9c341338c0 2023-10-03T10:13:09.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.965+0000 7f9c32ffd700 1 --2- 172.21.15.3:0/3245309399 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c341338c0 0x7f9c341200c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9c2800c550 tx=0x7f9c2800c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:09.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.966+0000 7f9c1bfff700 1 -- 172.21.15.3:0/3245309399 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c28005d70 con 0x7f9c341338c0 2023-10-03T10:13:09.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.966+0000 7f9c1bfff700 1 -- 172.21.15.3:0/3245309399 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c280070d0 con 0x7f9c341338c0 2023-10-03T10:13:09.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.966+0000 7f9c1bfff700 1 -- 172.21.15.3:0/3245309399 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c28007440 con 0x7f9c341338c0 2023-10-03T10:13:09.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.966+0000 7f9c3a13d700 1 -- 172.21.15.3:0/3245309399 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c34072660 con 0x7f9c341338c0 2023-10-03T10:13:09.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.966+0000 7f9c3a13d700 1 -- 172.21.15.3:0/3245309399 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c34121790 con 0x7f9c341338c0 2023-10-03T10:13:09.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.967+0000 7f9c1bfff700 1 -- 172.21.15.3:0/3245309399 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f9c280075a0 con 0x7f9c341338c0 2023-10-03T10:13:09.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.967+0000 7f9c1bfff700 1 -- 172.21.15.3:0/3245309399 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f9c2803efc0 con 0x7f9c341338c0 2023-10-03T10:13:09.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.967+0000 7f9c337fe700 1 -- 172.21.15.3:0/3245309399 >> v1:172.21.15.3:6816/127025 conn(0x7f9c2002f510 legacy=0x7f9c200319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:09.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.968+0000 7f9c3a13d700 1 -- 172.21.15.3:0/3245309399 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c34074ea0 con 0x7f9c341338c0 2023-10-03T10:13:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:09.973+0000 7f9c1bfff700 1 -- 172.21.15.3:0/3245309399 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9c28017070 con 0x7f9c341338c0 2023-10-03T10:13:09.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.000+0000 7f162562e700 1 Processor -- start 2023-10-03T10:13:10.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.000+0000 7f162562e700 1 -- start start 2023-10-03T10:13:10.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.001+0000 7f162562e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f162006c0e0 0x7f162009ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:10.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.001+0000 7f162562e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1620134ec0 0x7f16201379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:10.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.001+0000 7f162562e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1620134980 con 0x7f1620137f20 2023-10-03T10:13:10.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.001+0000 7f162562e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f162006c4b0 con 0x7f162006c0e0 2023-10-03T10:13:10.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.001+0000 7f162562e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16200a09c0 con 0x7f1620134ec0 2023-10-03T10:13:10.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.001+0000 7f161e7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1620134ec0 0x7f16201379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:10.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.001+0000 7f161e7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1620134ec0 0x7f16201379e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41044/0 (socket says 172.21.15.3:41044) 2023-10-03T10:13:10.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.001+0000 7f161e7fc700 1 -- 172.21.15.3:0/196070658 learned_addr learned my addr 172.21.15.3:0/196070658 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:10.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.001+0000 7f161effd700 1 --2- 172.21.15.3:0/196070658 >> v2:172.21.15.3:3300/0 conn(0x7f162006c0e0 0x7f162009ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:10.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.001+0000 7f161e7fc700 1 -- 172.21.15.3:0/196070658 >> v2:172.21.15.3:3300/0 conn(0x7f162006c0e0 msgr2=0x7f162009ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:10.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.001+0000 7f161e7fc700 1 --2- 172.21.15.3:0/196070658 >> v2:172.21.15.3:3300/0 conn(0x7f162006c0e0 0x7f162009ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.001+0000 7f161e7fc700 1 -- 172.21.15.3:0/196070658 >> v1:172.21.15.3:6789/0 conn(0x7f1620137f20 legacy=0x7f162013ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:10.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.001+0000 7f161dffb700 1 -- 172.21.15.3:0/196070658 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 410806605 0 0) 0x7f1620134980 con 0x7f1620137f20 2023-10-03T10:13:10.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.001+0000 7f161e7fc700 1 -- 172.21.15.3:0/196070658 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16200a0600 con 0x7f1620134ec0 2023-10-03T10:13:10.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.002+0000 7f161e7fc700 1 --2- 172.21.15.3:0/196070658 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1620134ec0 0x7f16201379e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f16140098f0 tx=0x7f1614009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93ddefa24fc9cfb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:10.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.002+0000 7f161dffb700 1 -- 172.21.15.3:0/196070658 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1614011830 con 0x7f1620134ec0 2023-10-03T10:13:10.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.002+0000 7f161dffb700 1 -- 172.21.15.3:0/196070658 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1614011990 con 0x7f1620134ec0 2023-10-03T10:13:10.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.002+0000 7f161dffb700 1 -- 172.21.15.3:0/196070658 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1614011d20 con 0x7f1620134ec0 2023-10-03T10:13:10.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.003+0000 7f162562e700 1 -- 172.21.15.3:0/196070658 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1620134ec0 msgr2=0x7f16201379e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:10.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.003+0000 7f162562e700 1 --2- 172.21.15.3:0/196070658 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1620134ec0 0x7f16201379e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f16140098f0 tx=0x7f1614009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:10.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.003+0000 7f162562e700 1 -- 172.21.15.3:0/196070658 shutdown_connections 2023-10-03T10:13:10.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.003+0000 7f162562e700 1 --2- 172.21.15.3:0/196070658 >> v2:172.21.15.3:3300/0 conn(0x7f162006c0e0 0x7f162009ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.003+0000 7f162562e700 1 --2- 172.21.15.3:0/196070658 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1620134ec0 0x7f16201379e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.003+0000 7f162562e700 1 -- 172.21.15.3:0/196070658 >> 172.21.15.3:0/196070658 conn(0x7f16200a4210 msgr2=0x7f16200a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:10.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.003+0000 7f162562e700 1 -- 172.21.15.3:0/196070658 shutdown_connections 2023-10-03T10:13:10.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.003+0000 7f162562e700 1 -- 172.21.15.3:0/196070658 wait complete. 2023-10-03T10:13:10.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.004+0000 7f162562e700 1 Processor -- start 2023-10-03T10:13:10.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.004+0000 7f162562e700 1 -- start start 2023-10-03T10:13:10.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.005+0000 7f162562e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f162006c0e0 0x7f162011a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:10.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.005+0000 7f162562e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1620134ec0 0x7f162011abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:10.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.005+0000 7f162562e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1620137f20 0x7f162011b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:10.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.005+0000 7f162562e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16200a03f0 con 0x7f1620134ec0 2023-10-03T10:13:10.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.005+0000 7f162562e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f162011cfb0 con 0x7f162006c0e0 2023-10-03T10:13:10.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.005+0000 7f162562e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f162009a5d0 con 0x7f1620137f20 2023-10-03T10:13:10.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.005+0000 7f161e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1620134ec0 0x7f162011abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:10.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.005+0000 7f161e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1620134ec0 0x7f162011abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49062/0 (socket says 172.21.15.3:49062) 2023-10-03T10:13:10.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.005+0000 7f161e7fc700 1 -- 172.21.15.3:0/332922143 learned_addr learned my addr 172.21.15.3:0/332922143 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:10.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.005+0000 7f161f7fe700 1 --2- 172.21.15.3:0/332922143 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1620137f20 0x7f162011b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:10.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.005+0000 7f161effd700 1 --2- 172.21.15.3:0/332922143 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f162006c0e0 0x7f162011a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:10.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.005+0000 7f161f7fe700 1 -- 172.21.15.3:0/332922143 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f162006c0e0 msgr2=0x7f162011a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:10.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.005+0000 7f161f7fe700 1 --2- 172.21.15.3:0/332922143 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f162006c0e0 0x7f162011a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.005+0000 7f161f7fe700 1 -- 172.21.15.3:0/332922143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1620134ec0 msgr2=0x7f162011abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:10.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.005+0000 7f161f7fe700 1 --2- 172.21.15.3:0/332922143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1620134ec0 0x7f162011abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.005+0000 7f161f7fe700 1 -- 172.21.15.3:0/332922143 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1614009550 con 0x7f1620137f20 2023-10-03T10:13:10.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.005+0000 7f161e7fc700 1 --2- 172.21.15.3:0/332922143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1620134ec0 0x7f162011abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:10.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.006+0000 7f161f7fe700 1 --2- 172.21.15.3:0/332922143 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1620137f20 0x7f162011b100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f161800c7c0 tx=0x7f161800cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:10.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.007+0000 7f16077fe700 1 -- 172.21.15.3:0/332922143 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1618003e30 con 0x7f1620137f20 2023-10-03T10:13:10.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.007+0000 7f16077fe700 1 -- 172.21.15.3:0/332922143 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1618005070 con 0x7f1620137f20 2023-10-03T10:13:10.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.007+0000 7f16077fe700 1 -- 172.21.15.3:0/332922143 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1618005420 con 0x7f1620137f20 2023-10-03T10:13:10.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.007+0000 7f162562e700 1 -- 172.21.15.3:0/332922143 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f162011cb80 con 0x7f1620137f20 2023-10-03T10:13:10.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.007+0000 7f162562e700 1 -- 172.21.15.3:0/332922143 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f162011e0e0 con 0x7f1620137f20 2023-10-03T10:13:10.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.008+0000 7f162562e700 1 -- 172.21.15.3:0/332922143 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f162011d740 con 0x7f1620137f20 2023-10-03T10:13:10.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.009+0000 7f16077fe700 1 -- 172.21.15.3:0/332922143 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f1618005580 con 0x7f1620137f20 2023-10-03T10:13:10.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.009+0000 7f16077fe700 1 -- 172.21.15.3:0/332922143 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f161803f250 con 0x7f1620137f20 2023-10-03T10:13:10.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.010+0000 7f161effd700 1 -- 172.21.15.3:0/332922143 >> v1:172.21.15.3:6816/127025 conn(0x7f160c02f510 legacy=0x7f160c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:10.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.018+0000 7f16077fe700 1 -- 172.21.15.3:0/332922143 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1618010b60 con 0x7f1620137f20 2023-10-03T10:13:10.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.066+0000 7f019546d700 1 Processor -- start 2023-10-03T10:13:10.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.066+0000 7f019546d700 1 -- start start 2023-10-03T10:13:10.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.067+0000 7f019546d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f019009d6f0 0x7f019009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:10.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.067+0000 7f019546d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f019009e000 0x7f01901339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:10.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.067+0000 7f019546d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f019009e410 0x7f0190133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:10.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.067+0000 7f019546d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f019006b460 con 0x7f019009e410 2023-10-03T10:13:10.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.067+0000 7f019546d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f019009b120 con 0x7f019009e000 2023-10-03T10:13:10.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.067+0000 7f019546d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f019009a7e0 con 0x7f019009d6f0 2023-10-03T10:13:10.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.067+0000 7f018effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f019009d6f0 0x7f019009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:10.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.068+0000 7f018effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f019009d6f0 0x7f019009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:49016/0 (socket says 172.21.15.3:49016) 2023-10-03T10:13:10.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.068+0000 7f018effd700 1 -- 172.21.15.3:0/3621307371 learned_addr learned my addr 172.21.15.3:0/3621307371 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:10.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.068+0000 7f018f7fe700 1 --2- 172.21.15.3:0/3621307371 >> v2:172.21.15.3:3300/0 conn(0x7f019009e410 0x7f0190133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:10.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.068+0000 7f018e7fc700 1 --2- 172.21.15.3:0/3621307371 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f019009e000 0x7f01901339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:10.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.068+0000 7f018effd700 1 -- 172.21.15.3:0/3621307371 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f019009e000 msgr2=0x7f01901339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:10.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.068+0000 7f018effd700 1 --2- 172.21.15.3:0/3621307371 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f019009e000 0x7f01901339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.068+0000 7f018effd700 1 -- 172.21.15.3:0/3621307371 >> v2:172.21.15.3:3300/0 conn(0x7f019009e410 msgr2=0x7f0190133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:10.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.068+0000 7f018effd700 1 --2- 172.21.15.3:0/3621307371 >> v2:172.21.15.3:3300/0 conn(0x7f019009e410 0x7f0190133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.068+0000 7f018effd700 1 -- 172.21.15.3:0/3621307371 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019009a500 con 0x7f019009d6f0 2023-10-03T10:13:10.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.068+0000 7f018effd700 1 --2- 172.21.15.3:0/3621307371 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f019009d6f0 0x7f019009dac0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0180009920 tx=0x7f0180009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd40feb577235048 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:10.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.068+0000 7f018dffb700 1 -- 172.21.15.3:0/3621307371 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0180011830 con 0x7f019009d6f0 2023-10-03T10:13:10.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.068+0000 7f018dffb700 1 -- 172.21.15.3:0/3621307371 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0180011990 con 0x7f019009d6f0 2023-10-03T10:13:10.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.068+0000 7f018dffb700 1 -- 172.21.15.3:0/3621307371 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0180011d20 con 0x7f019009d6f0 2023-10-03T10:13:10.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.069+0000 7f019546d700 1 -- 172.21.15.3:0/3621307371 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f019009d6f0 msgr2=0x7f019009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:10.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.069+0000 7f019546d700 1 --2- 172.21.15.3:0/3621307371 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f019009d6f0 0x7f019009dac0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0180009920 tx=0x7f0180009c30 comp rx=0 tx=0).stop 2023-10-03T10:13:10.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.069+0000 7f019546d700 1 -- 172.21.15.3:0/3621307371 shutdown_connections 2023-10-03T10:13:10.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.069+0000 7f019546d700 1 --2- 172.21.15.3:0/3621307371 >> v2:172.21.15.3:3300/0 conn(0x7f019009e410 0x7f0190133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.069+0000 7f019546d700 1 --2- 172.21.15.3:0/3621307371 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f019009d6f0 0x7f019009dac0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.069+0000 7f019546d700 1 --2- 172.21.15.3:0/3621307371 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f019009e000 0x7f01901339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.069+0000 7f019546d700 1 -- 172.21.15.3:0/3621307371 >> 172.21.15.3:0/3621307371 conn(0x7f0190066f50 msgr2=0x7f019009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:10.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.069+0000 7f019546d700 1 -- 172.21.15.3:0/3621307371 shutdown_connections 2023-10-03T10:13:10.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.069+0000 7f019546d700 1 -- 172.21.15.3:0/3621307371 wait complete. 2023-10-03T10:13:10.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.071+0000 7f019546d700 1 Processor -- start 2023-10-03T10:13:10.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.071+0000 7f019546d700 1 -- start start 2023-10-03T10:13:10.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.072+0000 7f019546d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f019009d6f0 0x7f01901a6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:10.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.072+0000 7f019546d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f019009e000 0x7f01901a6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:10.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.072+0000 7f019546d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f019009e410 0x7f01901a6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:10.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.072+0000 7f019546d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01901a9a60 con 0x7f019009d6f0 2023-10-03T10:13:10.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.072+0000 7f019546d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01901a8130 con 0x7f019009e000 2023-10-03T10:13:10.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.072+0000 7f019546d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f01901a91f0 con 0x7f019009e410 2023-10-03T10:13:10.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.072+0000 7f018effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f019009d6f0 0x7f01901a6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:10.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.072+0000 7f018effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f019009d6f0 0x7f01901a6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49084/0 (socket says 172.21.15.3:49084) 2023-10-03T10:13:10.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.072+0000 7f018effd700 1 -- 172.21.15.3:0/591954962 learned_addr learned my addr 172.21.15.3:0/591954962 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:10.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.072+0000 7f018e7fc700 1 --2- 172.21.15.3:0/591954962 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f019009e000 0x7f01901a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:10.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.072+0000 7f018f7fe700 1 --2- 172.21.15.3:0/591954962 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f019009e410 0x7f01901a6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:10.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.072+0000 7f018effd700 1 -- 172.21.15.3:0/591954962 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f019009e410 msgr2=0x7f01901a6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:10.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.072+0000 7f018effd700 1 --2- 172.21.15.3:0/591954962 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f019009e410 0x7f01901a6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.073+0000 7f018effd700 1 -- 172.21.15.3:0/591954962 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f019009e000 msgr2=0x7f01901a6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:10.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.073+0000 7f018effd700 1 --2- 172.21.15.3:0/591954962 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f019009e000 0x7f01901a6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.073+0000 7f018effd700 1 -- 172.21.15.3:0/591954962 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0180009550 con 0x7f019009d6f0 2023-10-03T10:13:10.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.073+0000 7f018e7fc700 1 --2- 172.21.15.3:0/591954962 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f019009e000 0x7f01901a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:10.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.073+0000 7f018effd700 1 --2- 172.21.15.3:0/591954962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f019009d6f0 0x7f01901a6020 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0180000c00 tx=0x7f0180004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:10.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.073+0000 7f01777fe700 1 -- 172.21.15.3:0/591954962 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0180004240 con 0x7f019009d6f0 2023-10-03T10:13:10.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.073+0000 7f01777fe700 1 -- 172.21.15.3:0/591954962 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f018001f070 con 0x7f019009d6f0 2023-10-03T10:13:10.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.073+0000 7f01777fe700 1 -- 172.21.15.3:0/591954962 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01800036a0 con 0x7f019009d6f0 2023-10-03T10:13:10.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.073+0000 7f019546d700 1 -- 172.21.15.3:0/591954962 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f019009c710 con 0x7f019009d6f0 2023-10-03T10:13:10.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.073+0000 7f019546d700 1 -- 172.21.15.3:0/591954962 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01901a7b50 con 0x7f019009d6f0 2023-10-03T10:13:10.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.074+0000 7f01777fe700 1 -- 172.21.15.3:0/591954962 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f0180021030 con 0x7f019009d6f0 2023-10-03T10:13:10.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.075+0000 7f019546d700 1 -- 172.21.15.3:0/591954962 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01901a8e00 con 0x7f019009d6f0 2023-10-03T10:13:10.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.075+0000 7f018e7fc700 1 -- 172.21.15.3:0/591954962 >> v1:172.21.15.3:6816/127025 conn(0x7f017c02f510 legacy=0x7f017c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:10.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.075+0000 7f01777fe700 1 -- 172.21.15.3:0/591954962 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f0180019570 con 0x7f019009d6f0 2023-10-03T10:13:10.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.084+0000 7f01777fe700 1 -- 172.21.15.3:0/591954962 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0180015020 con 0x7f019009d6f0 2023-10-03T10:13:10.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.113+0000 7f70d1c35700 1 Processor -- start 2023-10-03T10:13:10.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.113+0000 7f70d1c35700 1 -- start start 2023-10-03T10:13:10.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.114+0000 7f70d1c35700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70cc09d6f0 0x7f70cc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:10.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.114+0000 7f70d1c35700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70cc09e000 0x7f70cc1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:10.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.114+0000 7f70d1c35700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f70cc09e410 0x7f70cc133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:10.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.114+0000 7f70d1c35700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70cc06b460 con 0x7f70cc09e410 2023-10-03T10:13:10.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.114+0000 7f70d1c35700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc09b120 con 0x7f70cc09e000 2023-10-03T10:13:10.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.114+0000 7f70d1c35700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc09a7e0 con 0x7f70cc09d6f0 2023-10-03T10:13:10.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.114+0000 7f70cb7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70cc09d6f0 0x7f70cc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:10.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.114+0000 7f70caffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70cc09e000 0x7f70cc1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:10.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.114+0000 7f70cb7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70cc09d6f0 0x7f70cc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:49040/0 (socket says 172.21.15.3:49040) 2023-10-03T10:13:10.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.114+0000 7f70cb7fe700 1 -- 172.21.15.3:0/3285637156 learned_addr learned my addr 172.21.15.3:0/3285637156 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:10.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.114+0000 7f70cbfff700 1 --2- 172.21.15.3:0/3285637156 >> v2:172.21.15.3:3300/0 conn(0x7f70cc09e410 0x7f70cc133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:10.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.115+0000 7f70caffd700 1 -- 172.21.15.3:0/3285637156 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70cc09d6f0 msgr2=0x7f70cc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:10.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.115+0000 7f70caffd700 1 --2- 172.21.15.3:0/3285637156 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70cc09d6f0 0x7f70cc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.115+0000 7f70caffd700 1 -- 172.21.15.3:0/3285637156 >> v2:172.21.15.3:3300/0 conn(0x7f70cc09e410 msgr2=0x7f70cc133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:10.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.115+0000 7f70caffd700 1 --2- 172.21.15.3:0/3285637156 >> v2:172.21.15.3:3300/0 conn(0x7f70cc09e410 0x7f70cc133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.115+0000 7f70caffd700 1 -- 172.21.15.3:0/3285637156 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70cc09a500 con 0x7f70cc09e000 2023-10-03T10:13:10.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.115+0000 7f70cb7fe700 1 --2- 172.21.15.3:0/3285637156 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70cc09d6f0 0x7f70cc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:10.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.115+0000 7f70caffd700 1 --2- 172.21.15.3:0/3285637156 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70cc09e000 0x7f70cc1339f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f70c0009870 tx=0x7f70c0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f39e154bd09bd7ff server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:10.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.115+0000 7f70ca7fc700 1 -- 172.21.15.3:0/3285637156 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c0011830 con 0x7f70cc09e000 2023-10-03T10:13:10.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.115+0000 7f70ca7fc700 1 -- 172.21.15.3:0/3285637156 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c0011990 con 0x7f70cc09e000 2023-10-03T10:13:10.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.115+0000 7f70ca7fc700 1 -- 172.21.15.3:0/3285637156 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c0011d20 con 0x7f70cc09e000 2023-10-03T10:13:10.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.116+0000 7f70d1c35700 1 -- 172.21.15.3:0/3285637156 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70cc09e000 msgr2=0x7f70cc1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:10.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.116+0000 7f70d1c35700 1 --2- 172.21.15.3:0/3285637156 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70cc09e000 0x7f70cc1339f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f70c0009870 tx=0x7f70c0009b80 comp rx=0 tx=0).stop 2023-10-03T10:13:10.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.116+0000 7f70d1c35700 1 -- 172.21.15.3:0/3285637156 shutdown_connections 2023-10-03T10:13:10.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.116+0000 7f70d1c35700 1 --2- 172.21.15.3:0/3285637156 >> v2:172.21.15.3:3300/0 conn(0x7f70cc09e410 0x7f70cc133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.116+0000 7f70d1c35700 1 --2- 172.21.15.3:0/3285637156 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70cc09d6f0 0x7f70cc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.116+0000 7f70d1c35700 1 --2- 172.21.15.3:0/3285637156 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70cc09e000 0x7f70cc1339f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.116+0000 7f70d1c35700 1 -- 172.21.15.3:0/3285637156 >> 172.21.15.3:0/3285637156 conn(0x7f70cc066f50 msgr2=0x7f70cc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:10.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.116+0000 7f70d1c35700 1 -- 172.21.15.3:0/3285637156 shutdown_connections 2023-10-03T10:13:10.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.116+0000 7f70d1c35700 1 -- 172.21.15.3:0/3285637156 wait complete. 2023-10-03T10:13:10.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.117+0000 7f70d1c35700 1 Processor -- start 2023-10-03T10:13:10.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.118+0000 7f70d1c35700 1 -- start start 2023-10-03T10:13:10.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.118+0000 7f70d1c35700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70cc09d6f0 0x7f70cc1a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:10.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.118+0000 7f70d1c35700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70cc09e000 0x7f70cc1a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:10.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.118+0000 7f70d1c35700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70cc09e410 0x7f70cc1a6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:10.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.118+0000 7f70d1c35700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc1a96c0 con 0x7f70cc09e410 2023-10-03T10:13:10.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.118+0000 7f70d1c35700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc1a9910 con 0x7f70cc09d6f0 2023-10-03T10:13:10.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.118+0000 7f70d1c35700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc09c6d0 con 0x7f70cc09e000 2023-10-03T10:13:10.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.119+0000 7f70cbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70cc09e410 0x7f70cc1a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:10.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.119+0000 7f70cbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70cc09e410 0x7f70cc1a6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49112/0 (socket says 172.21.15.3:49112) 2023-10-03T10:13:10.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.119+0000 7f70cbfff700 1 -- 172.21.15.3:0/3882841106 learned_addr learned my addr 172.21.15.3:0/3882841106 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:10.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.119+0000 7f70caffd700 1 --2- 172.21.15.3:0/3882841106 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70cc09e000 0x7f70cc1a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:10.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.119+0000 7f70cb7fe700 1 --2- 172.21.15.3:0/3882841106 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70cc09d6f0 0x7f70cc1a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:10.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.119+0000 7f70cbfff700 1 -- 172.21.15.3:0/3882841106 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70cc09e000 msgr2=0x7f70cc1a6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:10.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.119+0000 7f70cbfff700 1 --2- 172.21.15.3:0/3882841106 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70cc09e000 0x7f70cc1a6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.119+0000 7f70cbfff700 1 -- 172.21.15.3:0/3882841106 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70cc09d6f0 msgr2=0x7f70cc1a5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:10.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.119+0000 7f70cbfff700 1 --2- 172.21.15.3:0/3882841106 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70cc09d6f0 0x7f70cc1a5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.119+0000 7f70cbfff700 1 -- 172.21.15.3:0/3882841106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70bc009550 con 0x7f70cc09e410 2023-10-03T10:13:10.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.119+0000 7f70cbfff700 1 --2- 172.21.15.3:0/3882841106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70cc09e410 0x7f70cc1a6940 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f70c400efc0 tx=0x7f70c400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:10.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.120+0000 7f70b3fff700 1 -- 172.21.15.3:0/3882841106 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c400cd40 con 0x7f70cc09e410 2023-10-03T10:13:10.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.120+0000 7f70b3fff700 1 -- 172.21.15.3:0/3882841106 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c400f040 con 0x7f70cc09e410 2023-10-03T10:13:10.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.120+0000 7f70d1c35700 1 -- 172.21.15.3:0/3882841106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70c00094a0 con 0x7f70cc09e410 2023-10-03T10:13:10.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.120+0000 7f70b3fff700 1 -- 172.21.15.3:0/3882841106 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c400cea0 con 0x7f70cc09e410 2023-10-03T10:13:10.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.120+0000 7f70d1c35700 1 -- 172.21.15.3:0/3882841106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70cc0a30e0 con 0x7f70cc09e410 2023-10-03T10:13:10.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.121+0000 7f70b3fff700 1 -- 172.21.15.3:0/3882841106 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f70c40146a0 con 0x7f70cc09e410 2023-10-03T10:13:10.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.121+0000 7f70b3fff700 1 -- 172.21.15.3:0/3882841106 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f70c4040b90 con 0x7f70cc09e410 2023-10-03T10:13:10.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.121+0000 7f70cb7fe700 1 -- 172.21.15.3:0/3882841106 >> v1:172.21.15.3:6816/127025 conn(0x7f70b802f510 legacy=0x7f70b80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:10.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.121+0000 7f70d1c35700 1 -- 172.21.15.3:0/3882841106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70cc1a7b00 con 0x7f70cc09e410 2023-10-03T10:13:10.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.131+0000 7f70b3fff700 1 -- 172.21.15.3:0/3882841106 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f70c4012520 con 0x7f70cc09e410 2023-10-03T10:13:10.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.168+0000 7f9c337fe700 1 -- 172.21.15.3:0/3245309399 >> v1:172.21.15.3:6816/127025 conn(0x7f9c2002f510 legacy=0x7f9c200319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:10.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.204+0000 7f9c3a13d700 1 -- 172.21.15.3:0/3245309399 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9c34131100 con 0x7f9c341338c0 2023-10-03T10:13:10.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.205+0000 7f9c1bfff700 1 -- 172.21.15.3:0/3245309399 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9c28011d70 con 0x7f9c341338c0 2023-10-03T10:13:10.204 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:10.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.207+0000 7f9c3a13d700 1 -- 172.21.15.3:0/3245309399 >> v1:172.21.15.3:6816/127025 conn(0x7f9c2002f510 legacy=0x7f9c200319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:10.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.207+0000 7f9c3a13d700 1 -- 172.21.15.3:0/3245309399 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c341338c0 msgr2=0x7f9c341200c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:10.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.207+0000 7f9c3a13d700 1 --2- 172.21.15.3:0/3245309399 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c341338c0 0x7f9c341200c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9c2800c550 tx=0x7f9c2800c860 comp rx=0 tx=0).stop 2023-10-03T10:13:10.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.207+0000 7f9c3a13d700 1 -- 172.21.15.3:0/3245309399 shutdown_connections 2023-10-03T10:13:10.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.207+0000 7f9c3a13d700 1 --2- 172.21.15.3:0/3245309399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c341341d0 0x7f9c34120600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.207+0000 7f9c3a13d700 1 --2- 172.21.15.3:0/3245309399 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c3409e560 0x7f9c3411fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.207+0000 7f9c3a13d700 1 --2- 172.21.15.3:0/3245309399 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c341338c0 0x7f9c341200c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.207+0000 7f9c3a13d700 1 -- 172.21.15.3:0/3245309399 >> 172.21.15.3:0/3245309399 conn(0x7f9c340a2aa0 msgr2=0x7f9c341314c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:10.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.207+0000 7f9c3a13d700 1 -- 172.21.15.3:0/3245309399 shutdown_connections 2023-10-03T10:13:10.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.207+0000 7f9c3a13d700 1 -- 172.21.15.3:0/3245309399 wait complete. 2023-10-03T10:13:10.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.211+0000 7f161effd700 1 -- 172.21.15.3:0/332922143 >> v1:172.21.15.3:6816/127025 conn(0x7f160c02f510 legacy=0x7f160c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:10.221 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:10.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.246+0000 7f162562e700 1 -- 172.21.15.3:0/332922143 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f162011d550 con 0x7f1620137f20 2023-10-03T10:13:10.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.247+0000 7f16077fe700 1 -- 172.21.15.3:0/332922143 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1618017090 con 0x7f1620137f20 2023-10-03T10:13:10.246 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:10.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.249+0000 7f162562e700 1 -- 172.21.15.3:0/332922143 >> v1:172.21.15.3:6816/127025 conn(0x7f160c02f510 legacy=0x7f160c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:10.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.249+0000 7f162562e700 1 -- 172.21.15.3:0/332922143 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1620137f20 msgr2=0x7f162011b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:10.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.249+0000 7f162562e700 1 --2- 172.21.15.3:0/332922143 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1620137f20 0x7f162011b100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f161800c7c0 tx=0x7f161800cad0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.249+0000 7f162562e700 1 -- 172.21.15.3:0/332922143 shutdown_connections 2023-10-03T10:13:10.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.249+0000 7f162562e700 1 --2- 172.21.15.3:0/332922143 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1620137f20 0x7f162011b100 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.249+0000 7f162562e700 1 --2- 172.21.15.3:0/332922143 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f162006c0e0 0x7f162011a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.249+0000 7f162562e700 1 --2- 172.21.15.3:0/332922143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1620134ec0 0x7f162011abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.249+0000 7f162562e700 1 -- 172.21.15.3:0/332922143 >> 172.21.15.3:0/332922143 conn(0x7f16200a4210 msgr2=0x7f16200a4670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:10.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.249+0000 7f162562e700 1 -- 172.21.15.3:0/332922143 shutdown_connections 2023-10-03T10:13:10.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.249+0000 7f162562e700 1 -- 172.21.15.3:0/332922143 wait complete. 2023-10-03T10:13:10.264 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:10.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.276+0000 7f018e7fc700 1 -- 172.21.15.3:0/591954962 >> v1:172.21.15.3:6816/127025 conn(0x7f017c02f510 legacy=0x7f017c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:10.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.307+0000 7f019546d700 1 -- 172.21.15.3:0/591954962 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0190074ea0 con 0x7f019009d6f0 2023-10-03T10:13:10.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.308+0000 7f01777fe700 1 -- 172.21.15.3:0/591954962 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0180046060 con 0x7f019009d6f0 2023-10-03T10:13:10.307 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:10.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.310+0000 7f019546d700 1 -- 172.21.15.3:0/591954962 >> v1:172.21.15.3:6816/127025 conn(0x7f017c02f510 legacy=0x7f017c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:10.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.310+0000 7f019546d700 1 -- 172.21.15.3:0/591954962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f019009d6f0 msgr2=0x7f01901a6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:10.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.310+0000 7f019546d700 1 --2- 172.21.15.3:0/591954962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f019009d6f0 0x7f01901a6020 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0180000c00 tx=0x7f0180004750 comp rx=0 tx=0).stop 2023-10-03T10:13:10.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.310+0000 7f019546d700 1 -- 172.21.15.3:0/591954962 shutdown_connections 2023-10-03T10:13:10.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.310+0000 7f019546d700 1 --2- 172.21.15.3:0/591954962 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f019009e410 0x7f01901a6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.310+0000 7f019546d700 1 --2- 172.21.15.3:0/591954962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f019009d6f0 0x7f01901a6020 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.310+0000 7f019546d700 1 --2- 172.21.15.3:0/591954962 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f019009e000 0x7f01901a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.310+0000 7f019546d700 1 -- 172.21.15.3:0/591954962 >> 172.21.15.3:0/591954962 conn(0x7f0190066f50 msgr2=0x7f01901bafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:10.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.310+0000 7f019546d700 1 -- 172.21.15.3:0/591954962 shutdown_connections 2023-10-03T10:13:10.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.310+0000 7f019546d700 1 -- 172.21.15.3:0/591954962 wait complete. 2023-10-03T10:13:10.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.322+0000 7f70cb7fe700 1 -- 172.21.15.3:0/3882841106 >> v1:172.21.15.3:6816/127025 conn(0x7f70b802f510 legacy=0x7f70b80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:10.323 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:10.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.343+0000 7f70d1c35700 1 -- 172.21.15.3:0/3882841106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f70cc074ea0 con 0x7f70cc09e410 2023-10-03T10:13:10.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.344+0000 7f70b3fff700 1 -- 172.21.15.3:0/3882841106 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f70c402c090 con 0x7f70cc09e410 2023-10-03T10:13:10.343 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:10.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.346+0000 7f70d1c35700 1 -- 172.21.15.3:0/3882841106 >> v1:172.21.15.3:6816/127025 conn(0x7f70b802f510 legacy=0x7f70b80319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:10.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.346+0000 7f70d1c35700 1 -- 172.21.15.3:0/3882841106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70cc09e410 msgr2=0x7f70cc1a6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:10.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.346+0000 7f70d1c35700 1 --2- 172.21.15.3:0/3882841106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70cc09e410 0x7f70cc1a6940 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f70c400efc0 tx=0x7f70c400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.346+0000 7f70d1c35700 1 -- 172.21.15.3:0/3882841106 shutdown_connections 2023-10-03T10:13:10.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.346+0000 7f70d1c35700 1 --2- 172.21.15.3:0/3882841106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70cc09e410 0x7f70cc1a6940 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.346+0000 7f70d1c35700 1 --2- 172.21.15.3:0/3882841106 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70cc09d6f0 0x7f70cc1a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.346+0000 7f70d1c35700 1 --2- 172.21.15.3:0/3882841106 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70cc09e000 0x7f70cc1a6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:10.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.346+0000 7f70d1c35700 1 -- 172.21.15.3:0/3882841106 >> 172.21.15.3:0/3882841106 conn(0x7f70cc066f50 msgr2=0x7f70cc1baeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:10.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.346+0000 7f70d1c35700 1 -- 172.21.15.3:0/3882841106 shutdown_connections 2023-10-03T10:13:10.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:10.346+0000 7f70d1c35700 1 -- 172.21.15.3:0/3882841106 wait complete. 2023-10-03T10:13:10.359 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:11.221 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:11.265 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:11.324 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:11.359 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:11.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.383+0000 7f308b3e1700 1 Processor -- start 2023-10-03T10:13:11.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.383+0000 7f308b3e1700 1 -- start start 2023-10-03T10:13:11.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.384+0000 7f308b3e1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f308409d6f0 0x7f308409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:11.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.384+0000 7f308b3e1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f308409e000 0x7f30841339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:11.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.384+0000 7f308b3e1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f308409e410 0x7f3084133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:11.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.384+0000 7f308b3e1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f308406b460 con 0x7f308409d6f0 2023-10-03T10:13:11.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.384+0000 7f308b3e1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f308409b120 con 0x7f308409e000 2023-10-03T10:13:11.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.384+0000 7f308b3e1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f308409a7e0 con 0x7f308409e410 2023-10-03T10:13:11.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.385+0000 7f308997e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f308409e410 0x7f3084133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:11.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.385+0000 7f308917d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f308409d6f0 0x7f308409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:11.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.385+0000 7f308917d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f308409d6f0 0x7f308409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49116/0 (socket says 172.21.15.3:49116) 2023-10-03T10:13:11.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.385+0000 7f308997e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f308409e410 0x7f3084133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:49056/0 (socket says 172.21.15.3:49056) 2023-10-03T10:13:11.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.385+0000 7f308917d700 1 -- 172.21.15.3:0/694239297 learned_addr learned my addr 172.21.15.3:0/694239297 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:11.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.385+0000 7f308897c700 1 --2- 172.21.15.3:0/694239297 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f308409e000 0x7f30841339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:11.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.385+0000 7f308917d700 1 -- 172.21.15.3:0/694239297 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f308409e410 msgr2=0x7f3084133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:11.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.385+0000 7f308917d700 1 --2- 172.21.15.3:0/694239297 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f308409e410 0x7f3084133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.385+0000 7f308917d700 1 -- 172.21.15.3:0/694239297 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f308409e000 msgr2=0x7f30841339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:11.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.385+0000 7f308917d700 1 --2- 172.21.15.3:0/694239297 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f308409e000 0x7f30841339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.385+0000 7f308917d700 1 -- 172.21.15.3:0/694239297 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308409a500 con 0x7f308409d6f0 2023-10-03T10:13:11.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.385+0000 7f308897c700 1 --2- 172.21.15.3:0/694239297 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f308409e000 0x7f30841339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:11.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.385+0000 7f308917d700 1 --2- 172.21.15.3:0/694239297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308409d6f0 0x7f308409dac0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f30800098f0 tx=0x7f3080009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14426973d7292df7 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:11.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.385+0000 7f307bfff700 1 -- 172.21.15.3:0/694239297 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3080011830 con 0x7f308409d6f0 2023-10-03T10:13:11.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.386+0000 7f307bfff700 1 -- 172.21.15.3:0/694239297 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3080011990 con 0x7f308409d6f0 2023-10-03T10:13:11.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.386+0000 7f307bfff700 1 -- 172.21.15.3:0/694239297 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3080011d20 con 0x7f308409d6f0 2023-10-03T10:13:11.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.386+0000 7f308b3e1700 1 -- 172.21.15.3:0/694239297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308409d6f0 msgr2=0x7f308409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:11.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.386+0000 7f308b3e1700 1 --2- 172.21.15.3:0/694239297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308409d6f0 0x7f308409dac0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f30800098f0 tx=0x7f3080009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:11.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.386+0000 7f308b3e1700 1 -- 172.21.15.3:0/694239297 shutdown_connections 2023-10-03T10:13:11.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.386+0000 7f308b3e1700 1 --2- 172.21.15.3:0/694239297 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f308409e410 0x7f3084133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.386+0000 7f308b3e1700 1 --2- 172.21.15.3:0/694239297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308409d6f0 0x7f308409dac0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.386+0000 7f308b3e1700 1 --2- 172.21.15.3:0/694239297 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f308409e000 0x7f30841339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.386+0000 7f308b3e1700 1 -- 172.21.15.3:0/694239297 >> 172.21.15.3:0/694239297 conn(0x7f3084066f50 msgr2=0x7f308409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:11.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.387+0000 7f308b3e1700 1 -- 172.21.15.3:0/694239297 shutdown_connections 2023-10-03T10:13:11.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.387+0000 7f308b3e1700 1 -- 172.21.15.3:0/694239297 wait complete. 2023-10-03T10:13:11.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.388+0000 7f308b3e1700 1 Processor -- start 2023-10-03T10:13:11.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.388+0000 7f308b3e1700 1 -- start start 2023-10-03T10:13:11.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.388+0000 7f308b3e1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308409d6f0 0x7f30841a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:11.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.388+0000 7f308b3e1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f308409e000 0x7f30841a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:11.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.388+0000 7f308b3e1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f308409e410 0x7f30841c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:11.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.388+0000 7f308b3e1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30840a1f20 con 0x7f308409d6f0 2023-10-03T10:13:11.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.388+0000 7f308b3e1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30841a80a0 con 0x7f308409e000 2023-10-03T10:13:11.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.388+0000 7f308b3e1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f30841b91f0 con 0x7f308409e410 2023-10-03T10:13:11.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.388+0000 7f308917d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308409d6f0 0x7f30841a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:11.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.388+0000 7f308917d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308409d6f0 0x7f30841a5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49118/0 (socket says 172.21.15.3:49118) 2023-10-03T10:13:11.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.388+0000 7f308917d700 1 -- 172.21.15.3:0/2435429305 learned_addr learned my addr 172.21.15.3:0/2435429305 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:11.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.388+0000 7f308897c700 1 --2- 172.21.15.3:0/2435429305 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f308409e000 0x7f30841a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:11.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.388+0000 7f308997e700 1 --2- 172.21.15.3:0/2435429305 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f308409e410 0x7f30841c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:11.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.389+0000 7f308917d700 1 -- 172.21.15.3:0/2435429305 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f308409e410 msgr2=0x7f30841c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:11.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.389+0000 7f308917d700 1 --2- 172.21.15.3:0/2435429305 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f308409e410 0x7f30841c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.389+0000 7f308917d700 1 -- 172.21.15.3:0/2435429305 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f308409e000 msgr2=0x7f30841a5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:11.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.389+0000 7f308917d700 1 --2- 172.21.15.3:0/2435429305 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f308409e000 0x7f30841a5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.389+0000 7f308917d700 1 -- 172.21.15.3:0/2435429305 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30740094a0 con 0x7f308409d6f0 2023-10-03T10:13:11.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.389+0000 7f308997e700 1 --2- 172.21.15.3:0/2435429305 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f308409e410 0x7f30841c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:11.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.389+0000 7f308897c700 1 --2- 172.21.15.3:0/2435429305 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f308409e000 0x7f30841a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:11.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.389+0000 7f308917d700 1 --2- 172.21.15.3:0/2435429305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308409d6f0 0x7f30841a5130 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3080011de0 tx=0x7f3080004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:11.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.389+0000 7f3079ffb700 1 -- 172.21.15.3:0/2435429305 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3080004300 con 0x7f308409d6f0 2023-10-03T10:13:11.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.389+0000 7f3079ffb700 1 -- 172.21.15.3:0/2435429305 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308001f070 con 0x7f308409d6f0 2023-10-03T10:13:11.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.389+0000 7f3079ffb700 1 -- 172.21.15.3:0/2435429305 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30800036a0 con 0x7f308409d6f0 2023-10-03T10:13:11.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.390+0000 7f308b3e1700 1 -- 172.21.15.3:0/2435429305 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3080009550 con 0x7f308409d6f0 2023-10-03T10:13:11.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.390+0000 7f308b3e1700 1 -- 172.21.15.3:0/2435429305 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30841a6c60 con 0x7f308409d6f0 2023-10-03T10:13:11.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.391+0000 7f3079ffb700 1 -- 172.21.15.3:0/2435429305 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f3080021030 con 0x7f308409d6f0 2023-10-03T10:13:11.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.391+0000 7f3079ffb700 1 -- 172.21.15.3:0/2435429305 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f3080040de0 con 0x7f308409d6f0 2023-10-03T10:13:11.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.391+0000 7f308897c700 1 -- 172.21.15.3:0/2435429305 >> v1:172.21.15.3:6816/127025 conn(0x7f306802f510 legacy=0x7f30680319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:11.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.391+0000 7f308b3e1700 1 -- 172.21.15.3:0/2435429305 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30841a64d0 con 0x7f308409d6f0 2023-10-03T10:13:11.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.400+0000 7f3079ffb700 1 -- 172.21.15.3:0/2435429305 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3080015080 con 0x7f308409d6f0 2023-10-03T10:13:11.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.424+0000 7fb60065c700 1 Processor -- start 2023-10-03T10:13:11.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.424+0000 7fb60065c700 1 -- start start 2023-10-03T10:13:11.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.424+0000 7fb60065c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5f80a0a30 0x7fb5f80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:11.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.424+0000 7fb60065c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb5f80a1340 0x7fb5f8133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:11.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.424+0000 7fb60065c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5f809edd0 con 0x7fb5f8133fa0 2023-10-03T10:13:11.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.424+0000 7fb60065c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f809e1f0 con 0x7fb5f80a1340 2023-10-03T10:13:11.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.424+0000 7fb60065c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f809dc50 con 0x7fb5f80a0a30 2023-10-03T10:13:11.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.425+0000 7fb5fe3f8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5f80a0a30 0x7fb5f80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:11.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.425+0000 7fb5fe3f8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5f80a0a30 0x7fb5f80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:49066/0 (socket says 172.21.15.3:49066) 2023-10-03T10:13:11.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.425+0000 7fb5fe3f8700 1 -- 172.21.15.3:0/56195072 learned_addr learned my addr 172.21.15.3:0/56195072 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:11.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.425+0000 7fb5fdbf7700 1 --2- 172.21.15.3:0/56195072 >> v2:172.21.15.3:3300/0 conn(0x7fb5f80a1340 0x7fb5f8133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:11.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.425+0000 7fb5fe3f8700 1 -- 172.21.15.3:0/56195072 >> v2:172.21.15.3:3300/0 conn(0x7fb5f80a1340 msgr2=0x7fb5f8133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:11.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.425+0000 7fb5fe3f8700 1 --2- 172.21.15.3:0/56195072 >> v2:172.21.15.3:3300/0 conn(0x7fb5f80a1340 0x7fb5f8133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.425+0000 7fb5fe3f8700 1 -- 172.21.15.3:0/56195072 >> v1:172.21.15.3:6789/0 conn(0x7fb5f8133fa0 legacy=0x7fb5f813ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:11.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.425+0000 7fb5fe3f8700 1 -- 172.21.15.3:0/56195072 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f80a1fe0 con 0x7fb5f80a0a30 2023-10-03T10:13:11.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.425+0000 7fb5fe3f8700 1 --2- 172.21.15.3:0/56195072 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5f80a0a30 0x7fb5f80a0e00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40098f0 tx=0x7fb5f4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1f84f1102405840 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:11.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.426+0000 7fb5fd3f6700 1 -- 172.21.15.3:0/56195072 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4004230 con 0x7fb5f80a0a30 2023-10-03T10:13:11.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.426+0000 7fb5fd3f6700 1 -- 172.21.15.3:0/56195072 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f4004390 con 0x7fb5f80a0a30 2023-10-03T10:13:11.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.426+0000 7fb5fd3f6700 1 -- 172.21.15.3:0/56195072 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f40036a0 con 0x7fb5f80a0a30 2023-10-03T10:13:11.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.427+0000 7fb60065c700 1 -- 172.21.15.3:0/56195072 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5f80a0a30 msgr2=0x7fb5f80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:11.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.427+0000 7fb60065c700 1 --2- 172.21.15.3:0/56195072 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5f80a0a30 0x7fb5f80a0e00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40098f0 tx=0x7fb5f4009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:11.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.427+0000 7fb60065c700 1 -- 172.21.15.3:0/56195072 shutdown_connections 2023-10-03T10:13:11.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.427+0000 7fb60065c700 1 --2- 172.21.15.3:0/56195072 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5f80a0a30 0x7fb5f80a0e00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.427+0000 7fb60065c700 1 --2- 172.21.15.3:0/56195072 >> v2:172.21.15.3:3300/0 conn(0x7fb5f80a1340 0x7fb5f8133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.427+0000 7fb60065c700 1 -- 172.21.15.3:0/56195072 >> 172.21.15.3:0/56195072 conn(0x7fb5f80a6550 msgr2=0x7fb5f80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:11.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.427+0000 7fb60065c700 1 -- 172.21.15.3:0/56195072 shutdown_connections 2023-10-03T10:13:11.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.427+0000 7fb60065c700 1 -- 172.21.15.3:0/56195072 wait complete. 2023-10-03T10:13:11.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.428+0000 7fb60065c700 1 Processor -- start 2023-10-03T10:13:11.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.429+0000 7fb60065c700 1 -- start start 2023-10-03T10:13:11.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.429+0000 7fb60065c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5f80a0a30 0x7fb5f81c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:11.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.429+0000 7fb60065c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5f80a1340 0x7fb5f81c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:11.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.429+0000 7fb60065c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5f8133fa0 0x7fb5f81c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:11.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.429+0000 7fb60065c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f8120830 con 0x7fb5f80a0a30 2023-10-03T10:13:11.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.429+0000 7fb60065c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f8120640 con 0x7fb5f8133fa0 2023-10-03T10:13:11.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.429+0000 7fb60065c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f80a37a0 con 0x7fb5f80a1340 2023-10-03T10:13:11.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.429+0000 7fb5fe3f8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5f80a0a30 0x7fb5f81c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:11.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.429+0000 7fb5fe3f8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5f80a0a30 0x7fb5f81c84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49128/0 (socket says 172.21.15.3:49128) 2023-10-03T10:13:11.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.429+0000 7fb5fe3f8700 1 -- 172.21.15.3:0/2041615203 learned_addr learned my addr 172.21.15.3:0/2041615203 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:11.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.430+0000 7fb5febf9700 1 --2- 172.21.15.3:0/2041615203 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5f8133fa0 0x7fb5f81c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:11.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.430+0000 7fb5fdbf7700 1 --2- 172.21.15.3:0/2041615203 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5f80a1340 0x7fb5f81c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:11.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.430+0000 7fb5fdbf7700 1 -- 172.21.15.3:0/2041615203 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5f8133fa0 msgr2=0x7fb5f81c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:11.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.430+0000 7fb5fdbf7700 1 --2- 172.21.15.3:0/2041615203 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5f8133fa0 0x7fb5f81c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.430+0000 7fb5fdbf7700 1 -- 172.21.15.3:0/2041615203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5f80a0a30 msgr2=0x7fb5f81c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:11.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.430+0000 7fb5fdbf7700 1 --2- 172.21.15.3:0/2041615203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5f80a0a30 0x7fb5f81c84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.430+0000 7fb5fdbf7700 1 -- 172.21.15.3:0/2041615203 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f4009550 con 0x7fb5f80a1340 2023-10-03T10:13:11.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.430+0000 7fb5fe3f8700 1 --2- 172.21.15.3:0/2041615203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5f80a0a30 0x7fb5f81c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:11.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.430+0000 7fb5fdbf7700 1 --2- 172.21.15.3:0/2041615203 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5f80a1340 0x7fb5f81c8a30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800e930 tx=0x7fb5e800ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:11.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.430+0000 7fb5eeffd700 1 -- 172.21.15.3:0/2041615203 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8013680 con 0x7fb5f80a1340 2023-10-03T10:13:11.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.431+0000 7fb5eeffd700 1 -- 172.21.15.3:0/2041615203 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e80137e0 con 0x7fb5f80a1340 2023-10-03T10:13:11.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.431+0000 7fb60065c700 1 -- 172.21.15.3:0/2041615203 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f80a0220 con 0x7fb5f80a1340 2023-10-03T10:13:11.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.431+0000 7fb5eeffd700 1 -- 172.21.15.3:0/2041615203 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8013b50 con 0x7fb5f80a1340 2023-10-03T10:13:11.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.431+0000 7fb60065c700 1 -- 172.21.15.3:0/2041615203 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f81212b0 con 0x7fb5f80a1340 2023-10-03T10:13:11.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.432+0000 7fb5eeffd700 1 -- 172.21.15.3:0/2041615203 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fb5e8013cb0 con 0x7fb5f80a1340 2023-10-03T10:13:11.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.432+0000 7fb5fe3f8700 1 -- 172.21.15.3:0/2041615203 >> v1:172.21.15.3:6816/127025 conn(0x7fb5dc02f510 legacy=0x7fb5dc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:11.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.432+0000 7fb60065c700 1 -- 172.21.15.3:0/2041615203 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f80a1920 con 0x7fb5f80a1340 2023-10-03T10:13:11.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.432+0000 7fb5eeffd700 1 -- 172.21.15.3:0/2041615203 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7fb5e801be60 con 0x7fb5f80a1340 2023-10-03T10:13:11.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.437+0000 7fb5eeffd700 1 -- 172.21.15.3:0/2041615203 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb5e8017030 con 0x7fb5f80a1340 2023-10-03T10:13:11.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.481+0000 7f73bfae5700 1 Processor -- start 2023-10-03T10:13:11.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.482+0000 7f73bfae5700 1 -- start start 2023-10-03T10:13:11.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.482+0000 7f73bfae5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73b809d6f0 0x7f73b809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:11.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.482+0000 7f73bfae5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73b809e000 0x7f73b81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:11.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.483+0000 7f73bfae5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73b809b120 con 0x7f73b809e410 2023-10-03T10:13:11.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.483+0000 7f73bfae5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b806b460 con 0x7f73b809d6f0 2023-10-03T10:13:11.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.483+0000 7f73bfae5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73b809a7e0 con 0x7f73b809e000 2023-10-03T10:13:11.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.483+0000 7f73bd080700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73b809e000 0x7f73b81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:11.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.483+0000 7f73be082700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f73b809e410 0x7f73b8133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49592/0 (socket says 172.21.15.3:49592) 2023-10-03T10:13:11.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.483+0000 7f73be082700 1 -- 172.21.15.3:0/3271605924 learned_addr learned my addr 172.21.15.3:0/3271605924 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:11.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.483+0000 7f73bd881700 1 --2- 172.21.15.3:0/3271605924 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73b809d6f0 0x7f73b809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:11.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.483+0000 7f73bd080700 1 -- 172.21.15.3:0/3271605924 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73b809d6f0 msgr2=0x7f73b809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:11.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.483+0000 7f73bd080700 1 --2- 172.21.15.3:0/3271605924 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73b809d6f0 0x7f73b809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.483+0000 7f73bd080700 1 -- 172.21.15.3:0/3271605924 >> v1:172.21.15.3:6789/0 conn(0x7f73b809e410 legacy=0x7f73b8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:11.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.483+0000 7f73bc87f700 1 -- 172.21.15.3:0/3271605924 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804678031 0 0) 0x7f73b809b120 con 0x7f73b809e410 2023-10-03T10:13:11.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.483+0000 7f73bd080700 1 -- 172.21.15.3:0/3271605924 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b809a500 con 0x7f73b809e000 2023-10-03T10:13:11.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.483+0000 7f73bd881700 1 --2- 172.21.15.3:0/3271605924 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73b809d6f0 0x7f73b809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:11.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.483+0000 7f73bd080700 1 --2- 172.21.15.3:0/3271605924 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73b809e000 0x7f73b81339f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f73a80098a0 tx=0x7f73a8009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddd8f5e1268cebbc server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:11.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.484+0000 7f73bc87f700 1 -- 172.21.15.3:0/3271605924 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a8011830 con 0x7f73b809e000 2023-10-03T10:13:11.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.484+0000 7f73bc87f700 1 -- 172.21.15.3:0/3271605924 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a8011990 con 0x7f73b809e000 2023-10-03T10:13:11.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.484+0000 7f73bc87f700 1 -- 172.21.15.3:0/3271605924 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a8011d40 con 0x7f73b809e000 2023-10-03T10:13:11.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.485+0000 7f73bfae5700 1 -- 172.21.15.3:0/3271605924 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73b809e000 msgr2=0x7f73b81339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:11.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.485+0000 7f73bfae5700 1 --2- 172.21.15.3:0/3271605924 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73b809e000 0x7f73b81339f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f73a80098a0 tx=0x7f73a8009bb0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.485+0000 7f73bfae5700 1 -- 172.21.15.3:0/3271605924 shutdown_connections 2023-10-03T10:13:11.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.485+0000 7f73bfae5700 1 --2- 172.21.15.3:0/3271605924 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73b809d6f0 0x7f73b809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.485+0000 7f73bfae5700 1 --2- 172.21.15.3:0/3271605924 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73b809e000 0x7f73b81339f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.485+0000 7f73bfae5700 1 -- 172.21.15.3:0/3271605924 >> 172.21.15.3:0/3271605924 conn(0x7f73b8066f50 msgr2=0x7f73b809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:11.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.485+0000 7f73bfae5700 1 -- 172.21.15.3:0/3271605924 shutdown_connections 2023-10-03T10:13:11.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.485+0000 7f73bfae5700 1 -- 172.21.15.3:0/3271605924 wait complete. 2023-10-03T10:13:11.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.487+0000 7f73bfae5700 1 Processor -- start 2023-10-03T10:13:11.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.487+0000 7f73bfae5700 1 -- start start 2023-10-03T10:13:11.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.487+0000 7f73bfae5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73b809d6f0 0x7f73b8122500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:11.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.487+0000 7f73bfae5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73b809e000 0x7f73b8122a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:11.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.487+0000 7f73bfae5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73b809e410 0x7f73b8122f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.487+0000 7f73bfae5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b81257b0 con 0x7f73b809e000 2023-10-03T10:13:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.487+0000 7f73bfae5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b80a20b0 con 0x7f73b809d6f0 2023-10-03T10:13:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.487+0000 7f73bfae5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73b8124f30 con 0x7f73b809e410 2023-10-03T10:13:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.488+0000 7f73bd881700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73b809d6f0 0x7f73b8122500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.488+0000 7f73be082700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73b809e410 0x7f73b8122f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.488+0000 7f73bd881700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73b809d6f0 0x7f73b8122500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41138/0 (socket says 172.21.15.3:41138) 2023-10-03T10:13:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.488+0000 7f73bd881700 1 -- 172.21.15.3:0/21393534 learned_addr learned my addr 172.21.15.3:0/21393534 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.488+0000 7f73bd080700 1 --2- 172.21.15.3:0/21393534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73b809e000 0x7f73b8122a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.488+0000 7f73bd881700 1 -- 172.21.15.3:0/21393534 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73b809e410 msgr2=0x7f73b8122f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.488+0000 7f73bd881700 1 --2- 172.21.15.3:0/21393534 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73b809e410 0x7f73b8122f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.488+0000 7f73bd881700 1 -- 172.21.15.3:0/21393534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73b809e000 msgr2=0x7f73b8122a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.488+0000 7f73bd881700 1 --2- 172.21.15.3:0/21393534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73b809e000 0x7f73b8122a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.488+0000 7f73bd881700 1 -- 172.21.15.3:0/21393534 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a80094a0 con 0x7f73b809d6f0 2023-10-03T10:13:11.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.488+0000 7f73bd881700 1 --2- 172.21.15.3:0/21393534 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73b809d6f0 0x7f73b8122500 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f73b400e810 tx=0x7f73b400eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:11.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.489+0000 7f73ae7fc700 1 -- 172.21.15.3:0/21393534 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b4013610 con 0x7f73b809d6f0 2023-10-03T10:13:11.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.489+0000 7f73ae7fc700 1 -- 172.21.15.3:0/21393534 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b4013770 con 0x7f73b809d6f0 2023-10-03T10:13:11.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.489+0000 7f73ae7fc700 1 -- 172.21.15.3:0/21393534 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b4013b20 con 0x7f73b809d6f0 2023-10-03T10:13:11.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.489+0000 7f73bfae5700 1 -- 172.21.15.3:0/21393534 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b8125d40 con 0x7f73b809d6f0 2023-10-03T10:13:11.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.489+0000 7f73bfae5700 1 -- 172.21.15.3:0/21393534 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b8126700 con 0x7f73b809d6f0 2023-10-03T10:13:11.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.490+0000 7f73bfae5700 1 -- 172.21.15.3:0/21393534 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b8074ea0 con 0x7f73b809d6f0 2023-10-03T10:13:11.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.490+0000 7f73ae7fc700 1 -- 172.21.15.3:0/21393534 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f73b4013c80 con 0x7f73b809d6f0 2023-10-03T10:13:11.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.491+0000 7f73ae7fc700 1 -- 172.21.15.3:0/21393534 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f73b403fd00 con 0x7f73b809d6f0 2023-10-03T10:13:11.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.497+0000 7f73bd080700 1 -- 172.21.15.3:0/21393534 >> v1:172.21.15.3:6816/127025 conn(0x7f739c02f510 legacy=0x7f739c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:11.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.499+0000 7f73ae7fc700 1 -- 172.21.15.3:0/21393534 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f73b4026430 con 0x7f73b809d6f0 2023-10-03T10:13:11.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.539+0000 7f2bb1718700 1 Processor -- start 2023-10-03T10:13:11.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.539+0000 7f2bb1718700 1 -- start start 2023-10-03T10:13:11.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.539+0000 7f2bb1718700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2bac09de00 0x7f2bac09e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:11.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.539+0000 7f2bb1718700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2bac13d1f0 0x7f2bac13f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:11.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.539+0000 7f2bb1718700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bac09a810 con 0x7f2bac06a770 2023-10-03T10:13:11.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.539+0000 7f2bb1718700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bac06ab40 con 0x7f2bac09de00 2023-10-03T10:13:11.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.539+0000 7f2bb1718700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac09ebc0 con 0x7f2bac13d1f0 2023-10-03T10:13:11.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.540+0000 7f2baa7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2bac09de00 0x7f2bac09e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:11.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.540+0000 7f2baa7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2bac09de00 0x7f2bac09e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49160/0 (socket says 172.21.15.3:49160) 2023-10-03T10:13:11.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.540+0000 7f2baa7fc700 1 -- 172.21.15.3:0/2404911503 learned_addr learned my addr 172.21.15.3:0/2404911503 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:11.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.540+0000 7f2bab7fe700 1 --2- 172.21.15.3:0/2404911503 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2bac13d1f0 0x7f2bac13f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:11.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.540+0000 7f2baa7fc700 1 -- 172.21.15.3:0/2404911503 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2bac13d1f0 msgr2=0x7f2bac13f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:11.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.540+0000 7f2baa7fc700 1 --2- 172.21.15.3:0/2404911503 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2bac13d1f0 0x7f2bac13f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.540+0000 7f2baa7fc700 1 -- 172.21.15.3:0/2404911503 >> v1:172.21.15.3:6789/0 conn(0x7f2bac06a770 legacy=0x7f2bac09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:11.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.540+0000 7f2baa7fc700 1 -- 172.21.15.3:0/2404911503 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bac09e920 con 0x7f2bac09de00 2023-10-03T10:13:11.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.540+0000 7f2bab7fe700 1 --2- 172.21.15.3:0/2404911503 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2bac13d1f0 0x7f2bac13f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:11.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.540+0000 7f2baa7fc700 1 --2- 172.21.15.3:0/2404911503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bac09de00 0x7f2bac09e250 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2ba00098f0 tx=0x7f2ba0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0699c73899a1ca0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:11.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.541+0000 7f2ba9ffb700 1 -- 172.21.15.3:0/2404911503 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba000b8a0 con 0x7f2bac09de00 2023-10-03T10:13:11.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.541+0000 7f2ba9ffb700 1 -- 172.21.15.3:0/2404911503 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba000ba00 con 0x7f2bac09de00 2023-10-03T10:13:11.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.541+0000 7f2ba9ffb700 1 -- 172.21.15.3:0/2404911503 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba000bd90 con 0x7f2bac09de00 2023-10-03T10:13:11.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.541+0000 7f2bb1718700 1 -- 172.21.15.3:0/2404911503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bac09de00 msgr2=0x7f2bac09e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:11.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.541+0000 7f2bb1718700 1 --2- 172.21.15.3:0/2404911503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bac09de00 0x7f2bac09e250 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2ba00098f0 tx=0x7f2ba0009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:11.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.541+0000 7f2bb1718700 1 -- 172.21.15.3:0/2404911503 shutdown_connections 2023-10-03T10:13:11.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.541+0000 7f2bb1718700 1 --2- 172.21.15.3:0/2404911503 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2bac13d1f0 0x7f2bac13f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.541+0000 7f2bb1718700 1 --2- 172.21.15.3:0/2404911503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bac09de00 0x7f2bac09e250 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.541+0000 7f2bb1718700 1 -- 172.21.15.3:0/2404911503 >> 172.21.15.3:0/2404911503 conn(0x7f2bac066f50 msgr2=0x7f2bac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:11.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.541+0000 7f2bb1718700 1 -- 172.21.15.3:0/2404911503 shutdown_connections 2023-10-03T10:13:11.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.541+0000 7f2bb1718700 1 -- 172.21.15.3:0/2404911503 wait complete. 2023-10-03T10:13:11.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.543+0000 7f2bb1718700 1 Processor -- start 2023-10-03T10:13:11.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.543+0000 7f2bb1718700 1 -- start start 2023-10-03T10:13:11.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.543+0000 7f2bb1718700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bac06a770 0x7f2bac071740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:11.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.543+0000 7f2bb1718700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2bac09de00 0x7f2bac13b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:11.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.543+0000 7f2bb1718700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2bac13d1f0 0x7f2bac13bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:11.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.543+0000 7f2bb1718700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac072340 con 0x7f2bac06a770 2023-10-03T10:13:11.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.543+0000 7f2bb1718700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac11f0a0 con 0x7f2bac13d1f0 2023-10-03T10:13:11.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.543+0000 7f2bb1718700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac11eb00 con 0x7f2bac09de00 2023-10-03T10:13:11.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.543+0000 7f2bab7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2bac13d1f0 0x7f2bac13bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:11.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.543+0000 7f2bab7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2bac13d1f0 0x7f2bac13bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41148/0 (socket says 172.21.15.3:41148) 2023-10-03T10:13:11.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.543+0000 7f2bab7fe700 1 -- 172.21.15.3:0/1735254091 learned_addr learned my addr 172.21.15.3:0/1735254091 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:11.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.543+0000 7f2baaffd700 1 --2- 172.21.15.3:0/1735254091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bac06a770 0x7f2bac071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:11.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.544+0000 7f2bab7fe700 1 -- 172.21.15.3:0/1735254091 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2bac09de00 msgr2=0x7f2bac13b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:11.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.544+0000 7f2baa7fc700 1 --2- 172.21.15.3:0/1735254091 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2bac09de00 0x7f2bac13b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:11.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.544+0000 7f2bab7fe700 1 --2- 172.21.15.3:0/1735254091 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2bac09de00 0x7f2bac13b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.544+0000 7f2bab7fe700 1 -- 172.21.15.3:0/1735254091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bac06a770 msgr2=0x7f2bac071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:11.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.544+0000 7f2bab7fe700 1 --2- 172.21.15.3:0/1735254091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bac06a770 0x7f2bac071740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.544+0000 7f2bab7fe700 1 -- 172.21.15.3:0/1735254091 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba0009550 con 0x7f2bac13d1f0 2023-10-03T10:13:11.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.544+0000 7f2baa7fc700 1 --2- 172.21.15.3:0/1735254091 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2bac09de00 0x7f2bac13b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:13:11.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.544+0000 7f2bab7fe700 1 --2- 172.21.15.3:0/1735254091 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2bac13d1f0 0x7f2bac13bc50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009f70 tx=0x7f2ba400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:11.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.544+0000 7f2b937fe700 1 -- 172.21.15.3:0/1735254091 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba400de30 con 0x7f2bac13d1f0 2023-10-03T10:13:11.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.544+0000 7f2b937fe700 1 -- 172.21.15.3:0/1735254091 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba400f040 con 0x7f2bac13d1f0 2023-10-03T10:13:11.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.544+0000 7f2b937fe700 1 -- 172.21.15.3:0/1735254091 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4003c60 con 0x7f2bac13d1f0 2023-10-03T10:13:11.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.545+0000 7f2bb1718700 1 -- 172.21.15.3:0/1735254091 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bac13cea0 con 0x7f2bac13d1f0 2023-10-03T10:13:11.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.545+0000 7f2bb1718700 1 -- 172.21.15.3:0/1735254091 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bac11e3e0 con 0x7f2bac13d1f0 2023-10-03T10:13:11.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.546+0000 7f2bb1718700 1 -- 172.21.15.3:0/1735254091 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bac074ea0 con 0x7f2bac13d1f0 2023-10-03T10:13:11.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.546+0000 7f2b937fe700 1 -- 172.21.15.3:0/1735254091 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f2ba4004030 con 0x7f2bac13d1f0 2023-10-03T10:13:11.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.546+0000 7f2b937fe700 1 -- 172.21.15.3:0/1735254091 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4121+0+0 (secure 0 0 0) 0x7f2ba403fea0 con 0x7f2bac13d1f0 2023-10-03T10:13:11.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.546+0000 7f2baaffd700 1 -- 172.21.15.3:0/1735254091 >> v1:172.21.15.3:6816/127025 conn(0x7f2b9802f510 legacy=0x7f2b980319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.555+0000 7f2b937fe700 1 -- 172.21.15.3:0/1735254091 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2ba401e3b0 con 0x7f2bac13d1f0 2023-10-03T10:13:11.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.592+0000 7f308897c700 1 -- 172.21.15.3:0/2435429305 >> v1:172.21.15.3:6816/127025 conn(0x7f306802f510 legacy=0x7f30680319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:11.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.637+0000 7f308b3e1700 1 -- 172.21.15.3:0/2435429305 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3084074ea0 con 0x7f308409d6f0 2023-10-03T10:13:11.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.637+0000 7f3079ffb700 1 -- 172.21.15.3:0/2435429305 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3080019980 con 0x7f308409d6f0 2023-10-03T10:13:11.637 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:11.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.641+0000 7f308b3e1700 1 -- 172.21.15.3:0/2435429305 >> v1:172.21.15.3:6816/127025 conn(0x7f306802f510 legacy=0x7f30680319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:11.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.641+0000 7f308b3e1700 1 -- 172.21.15.3:0/2435429305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308409d6f0 msgr2=0x7f30841a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:11.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.641+0000 7f308b3e1700 1 --2- 172.21.15.3:0/2435429305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308409d6f0 0x7f30841a5130 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3080011de0 tx=0x7f3080004750 comp rx=0 tx=0).stop 2023-10-03T10:13:11.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.642+0000 7f308b3e1700 1 -- 172.21.15.3:0/2435429305 shutdown_connections 2023-10-03T10:13:11.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.642+0000 7f308b3e1700 1 --2- 172.21.15.3:0/2435429305 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f308409e410 0x7f30841c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.642+0000 7f308b3e1700 1 --2- 172.21.15.3:0/2435429305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308409d6f0 0x7f30841a5130 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.642+0000 7f308b3e1700 1 --2- 172.21.15.3:0/2435429305 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f308409e000 0x7f30841a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.642+0000 7f308b3e1700 1 -- 172.21.15.3:0/2435429305 >> 172.21.15.3:0/2435429305 conn(0x7f3084066f50 msgr2=0x7f30841bbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:11.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.642+0000 7f308b3e1700 1 -- 172.21.15.3:0/2435429305 shutdown_connections 2023-10-03T10:13:11.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.642+0000 7f308b3e1700 1 -- 172.21.15.3:0/2435429305 wait complete. 2023-10-03T10:13:11.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.651+0000 7fb60065c700 1 -- 172.21.15.3:0/2041615203 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb5f8072420 con 0x7fb5f80a1340 2023-10-03T10:13:11.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.651+0000 7fb5eeffd700 1 -- 172.21.15.3:0/2041615203 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb5f8072420 con 0x7fb5f80a1340 2023-10-03T10:13:11.650 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.654+0000 7fb60065c700 1 -- 172.21.15.3:0/2041615203 >> v1:172.21.15.3:6816/127025 conn(0x7fb5dc02f510 legacy=0x7fb5dc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-03T10:13:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.654+0000 7fb60065c700 1 -- 172.21.15.3:0/2041615203 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5f80a1340 msgr2=0x7fb5f81c8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.654+0000 7fb60065c700 1 --2- 172.21.15.3:0/2041615203 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5f80a1340 0x7fb5f81c8a30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800e930 tx=0x7fb5e800ec40 comp rx=0 tx=0).stop 2023-10-03T10:13:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.654+0000 7fb60065c700 1 -- 172.21.15.3:0/2041615203 shutdown_connections 2023-10-03T10:13:11.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.654+0000 7fb60065c700 1 --2- 172.21.15.3:0/2041615203 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5f8133fa0 0x7fb5f81c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.654+0000 7fb60065c700 1 --2- 172.21.15.3:0/2041615203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5f80a0a30 0x7fb5f81c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.654+0000 7fb60065c700 1 --2- 172.21.15.3:0/2041615203 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5f80a1340 0x7fb5f81c8a30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.654+0000 7fb60065c700 1 -- 172.21.15.3:0/2041615203 >> 172.21.15.3:0/2041615203 conn(0x7fb5f80a6550 msgr2=0x7fb5f81b9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:11.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.654+0000 7fb60065c700 1 -- 172.21.15.3:0/2041615203 shutdown_connections 2023-10-03T10:13:11.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.654+0000 7fb60065c700 1 -- 172.21.15.3:0/2041615203 wait complete. 2023-10-03T10:13:11.655 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:11.667 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:11.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.698+0000 7f73bd080700 1 -- 172.21.15.3:0/21393534 >> v1:172.21.15.3:6816/127025 conn(0x7f739c02f510 legacy=0x7f739c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:11.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.713+0000 7f73bfae5700 1 -- 172.21.15.3:0/21393534 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f73b809c710 con 0x7f73b809d6f0 2023-10-03T10:13:11.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.714+0000 7f73ae7fc700 1 -- 172.21.15.3:0/21393534 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f73b4021070 con 0x7f73b809d6f0 2023-10-03T10:13:11.713 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:11.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.716+0000 7f73bfae5700 1 -- 172.21.15.3:0/21393534 >> v1:172.21.15.3:6816/127025 conn(0x7f739c02f510 legacy=0x7f739c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:11.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.716+0000 7f73bfae5700 1 -- 172.21.15.3:0/21393534 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73b809d6f0 msgr2=0x7f73b8122500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:11.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.716+0000 7f73bfae5700 1 --2- 172.21.15.3:0/21393534 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73b809d6f0 0x7f73b8122500 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f73b400e810 tx=0x7f73b400eb20 comp rx=0 tx=0).stop 2023-10-03T10:13:11.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.716+0000 7f73bfae5700 1 -- 172.21.15.3:0/21393534 shutdown_connections 2023-10-03T10:13:11.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.716+0000 7f73bfae5700 1 --2- 172.21.15.3:0/21393534 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73b809e410 0x7f73b8122f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.716+0000 7f73bfae5700 1 --2- 172.21.15.3:0/21393534 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73b809d6f0 0x7f73b8122500 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.716+0000 7f73bfae5700 1 --2- 172.21.15.3:0/21393534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73b809e000 0x7f73b8122a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.716+0000 7f73bfae5700 1 -- 172.21.15.3:0/21393534 >> 172.21.15.3:0/21393534 conn(0x7f73b8066f50 msgr2=0x7f73b8133f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:11.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.716+0000 7f73bfae5700 1 -- 172.21.15.3:0/21393534 shutdown_connections 2023-10-03T10:13:11.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.716+0000 7f73bfae5700 1 -- 172.21.15.3:0/21393534 wait complete. 2023-10-03T10:13:11.729 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:11.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.747+0000 7f2baaffd700 1 -- 172.21.15.3:0/1735254091 >> v1:172.21.15.3:6816/127025 conn(0x7f2b9802f510 legacy=0x7f2b980319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.3:6816/127025 2023-10-03T10:13:11.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.765+0000 7f2bb1718700 1 -- 172.21.15.3:0/1735254091 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2bac09a3d0 con 0x7f2bac13d1f0 2023-10-03T10:13:11.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.765+0000 7f2b937fe700 1 -- 172.21.15.3:0/1735254091 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2ba4012350 con 0x7f2bac13d1f0 2023-10-03T10:13:11.764 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:11.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.768+0000 7f2bb1718700 1 -- 172.21.15.3:0/1735254091 >> v1:172.21.15.3:6816/127025 conn(0x7f2b9802f510 legacy=0x7f2b980319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:11.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.768+0000 7f2bb1718700 1 -- 172.21.15.3:0/1735254091 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2bac13d1f0 msgr2=0x7f2bac13bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:11.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.768+0000 7f2bb1718700 1 --2- 172.21.15.3:0/1735254091 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2bac13d1f0 0x7f2bac13bc50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009f70 tx=0x7f2ba400d3b0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.768+0000 7f2bb1718700 1 -- 172.21.15.3:0/1735254091 shutdown_connections 2023-10-03T10:13:11.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.768+0000 7f2bb1718700 1 --2- 172.21.15.3:0/1735254091 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2bac13d1f0 0x7f2bac13bc50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.768+0000 7f2bb1718700 1 --2- 172.21.15.3:0/1735254091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bac06a770 0x7f2bac071740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.768+0000 7f2bb1718700 1 --2- 172.21.15.3:0/1735254091 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2bac09de00 0x7f2bac13b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:11.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.768+0000 7f2bb1718700 1 -- 172.21.15.3:0/1735254091 >> 172.21.15.3:0/1735254091 conn(0x7f2bac066f50 msgr2=0x7f2bac09b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:11.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.768+0000 7f2bb1718700 1 -- 172.21.15.3:0/1735254091 shutdown_connections 2023-10-03T10:13:11.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:11.768+0000 7f2bb1718700 1 -- 172.21.15.3:0/1735254091 wait complete. 2023-10-03T10:13:11.781 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:12.657 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:12.668 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:12.730 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:12.781 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:12.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.819+0000 7f5c31e61700 1 Processor -- start 2023-10-03T10:13:12.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.819+0000 7f5c31e61700 1 -- start start 2023-10-03T10:13:12.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.819+0000 7f5c31e61700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c2c06a770 0x7f5c2c09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:12.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.819+0000 7f5c31e61700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c2c09dc30 0x7f5c2c09e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:12.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.819+0000 7f5c31e61700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c2c13c1e0 0x7f5c2c13e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:12.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.820+0000 7f5c31e61700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c2c06ab40 con 0x7f5c2c06a770 2023-10-03T10:13:12.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.820+0000 7f5c31e61700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c09ebc0 con 0x7f5c2c09dc30 2023-10-03T10:13:12.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.820+0000 7f5c31e61700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c09a810 con 0x7f5c2c13c1e0 2023-10-03T10:13:12.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.820+0000 7f5c2affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c2c09dc30 0x7f5c2c09e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:12.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.820+0000 7f5c2affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c2c09dc30 0x7f5c2c09e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41154/0 (socket says 172.21.15.3:41154) 2023-10-03T10:13:12.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.820+0000 7f5c2affd700 1 -- 172.21.15.3:0/2275475232 learned_addr learned my addr 172.21.15.3:0/2275475232 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:12.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.820+0000 7f5c2b7fe700 1 --2- 172.21.15.3:0/2275475232 >> v2:172.21.15.3:3300/0 conn(0x7f5c2c06a770 0x7f5c2c09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:12.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.820+0000 7f5c2bfff700 1 --2- 172.21.15.3:0/2275475232 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c2c13c1e0 0x7f5c2c13e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:12.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.820+0000 7f5c2affd700 1 -- 172.21.15.3:0/2275475232 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c2c13c1e0 msgr2=0x7f5c2c13e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:12.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.820+0000 7f5c2affd700 1 --2- 172.21.15.3:0/2275475232 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c2c13c1e0 0x7f5c2c13e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:12.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.820+0000 7f5c2affd700 1 -- 172.21.15.3:0/2275475232 >> v2:172.21.15.3:3300/0 conn(0x7f5c2c06a770 msgr2=0x7f5c2c09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:12.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.820+0000 7f5c2affd700 1 --2- 172.21.15.3:0/2275475232 >> v2:172.21.15.3:3300/0 conn(0x7f5c2c06a770 0x7f5c2c09d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:12.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.820+0000 7f5c2affd700 1 -- 172.21.15.3:0/2275475232 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c2c09e920 con 0x7f5c2c09dc30 2023-10-03T10:13:12.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.820+0000 7f5c2bfff700 1 --2- 172.21.15.3:0/2275475232 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c2c13c1e0 0x7f5c2c13e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:13:12.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.820+0000 7f5c2affd700 1 --2- 172.21.15.3:0/2275475232 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c2c09dc30 0x7f5c2c09e080 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5c200098f0 tx=0x7f5c20009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1eb559ebad70235f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:12.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.821+0000 7f5c2a7fc700 1 -- 172.21.15.3:0/2275475232 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c20011830 con 0x7f5c2c09dc30 2023-10-03T10:13:12.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.821+0000 7f5c2a7fc700 1 -- 172.21.15.3:0/2275475232 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c20011990 con 0x7f5c2c09dc30 2023-10-03T10:13:12.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.821+0000 7f5c2a7fc700 1 -- 172.21.15.3:0/2275475232 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c20011d20 con 0x7f5c2c09dc30 2023-10-03T10:13:12.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.822+0000 7f5c31e61700 1 -- 172.21.15.3:0/2275475232 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c2c09dc30 msgr2=0x7f5c2c09e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:12.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.822+0000 7f5c31e61700 1 --2- 172.21.15.3:0/2275475232 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c2c09dc30 0x7f5c2c09e080 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5c200098f0 tx=0x7f5c20009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:12.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.822+0000 7f5c31e61700 1 -- 172.21.15.3:0/2275475232 shutdown_connections 2023-10-03T10:13:12.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.822+0000 7f5c31e61700 1 --2- 172.21.15.3:0/2275475232 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c2c13c1e0 0x7f5c2c13e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:12.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.822+0000 7f5c31e61700 1 --2- 172.21.15.3:0/2275475232 >> v2:172.21.15.3:3300/0 conn(0x7f5c2c06a770 0x7f5c2c09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:12.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.822+0000 7f5c31e61700 1 --2- 172.21.15.3:0/2275475232 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c2c09dc30 0x7f5c2c09e080 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:12.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.822+0000 7f5c31e61700 1 -- 172.21.15.3:0/2275475232 >> 172.21.15.3:0/2275475232 conn(0x7f5c2c066f50 msgr2=0x7f5c2c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:12.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.822+0000 7f5c31e61700 1 -- 172.21.15.3:0/2275475232 shutdown_connections 2023-10-03T10:13:12.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.822+0000 7f5c31e61700 1 -- 172.21.15.3:0/2275475232 wait complete. 2023-10-03T10:13:12.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.823+0000 7f5c31e61700 1 Processor -- start 2023-10-03T10:13:12.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.824+0000 7f5c31e61700 1 -- start start 2023-10-03T10:13:12.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.824+0000 7f5c31e61700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c2c06a770 0x7f5c2c126da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:12.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.824+0000 7f5c31e61700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c2c09dc30 0x7f5c2c1272e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:12.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.824+0000 7f5c31e61700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c2c13c1e0 0x7f5c2c1cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:12.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.824+0000 7f5c31e61700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c13ab10 con 0x7f5c2c06a770 2023-10-03T10:13:12.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.824+0000 7f5c31e61700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c127820 con 0x7f5c2c13c1e0 2023-10-03T10:13:12.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.824+0000 7f5c31e61700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c126800 con 0x7f5c2c09dc30 2023-10-03T10:13:12.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.825+0000 7f5c2b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c2c06a770 0x7f5c2c126da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:12.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.825+0000 7f5c2b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c2c06a770 0x7f5c2c126da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49180/0 (socket says 172.21.15.3:49180) 2023-10-03T10:13:12.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.825+0000 7f5c2b7fe700 1 -- 172.21.15.3:0/1192630856 learned_addr learned my addr 172.21.15.3:0/1192630856 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:12.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.825+0000 7f5c2affd700 1 --2- 172.21.15.3:0/1192630856 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c2c09dc30 0x7f5c2c1272e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:12.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.825+0000 7f5c2bfff700 1 --2- 172.21.15.3:0/1192630856 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c2c13c1e0 0x7f5c2c1cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:12.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.825+0000 7f5c2affd700 1 -- 172.21.15.3:0/1192630856 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c2c13c1e0 msgr2=0x7f5c2c1cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:12.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.825+0000 7f5c2affd700 1 --2- 172.21.15.3:0/1192630856 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c2c13c1e0 0x7f5c2c1cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:12.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.825+0000 7f5c2affd700 1 -- 172.21.15.3:0/1192630856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c2c06a770 msgr2=0x7f5c2c126da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:12.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.825+0000 7f5c2affd700 1 --2- 172.21.15.3:0/1192630856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c2c06a770 0x7f5c2c126da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:12.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.825+0000 7f5c2affd700 1 -- 172.21.15.3:0/1192630856 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c20009550 con 0x7f5c2c09dc30 2023-10-03T10:13:12.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.825+0000 7f5c2bfff700 1 --2- 172.21.15.3:0/1192630856 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c2c13c1e0 0x7f5c2c1cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:12.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.826+0000 7f5c2affd700 1 --2- 172.21.15.3:0/1192630856 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c2c09dc30 0x7f5c2c1272e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5c200098f0 tx=0x7f5c20004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:12.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.826+0000 7f5c13fff700 1 -- 172.21.15.3:0/1192630856 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c20004240 con 0x7f5c2c09dc30 2023-10-03T10:13:12.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.826+0000 7f5c13fff700 1 -- 172.21.15.3:0/1192630856 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2001f070 con 0x7f5c2c09dc30 2023-10-03T10:13:12.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.826+0000 7f5c31e61700 1 -- 172.21.15.3:0/1192630856 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c2c1299c0 con 0x7f5c2c09dc30 2023-10-03T10:13:12.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.826+0000 7f5c13fff700 1 -- 172.21.15.3:0/1192630856 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c200045a0 con 0x7f5c2c09dc30 2023-10-03T10:13:12.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.826+0000 7f5c31e61700 1 -- 172.21.15.3:0/1192630856 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c2c1282e0 con 0x7f5c2c09dc30 2023-10-03T10:13:12.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.827+0000 7f5c13fff700 1 -- 172.21.15.3:0/1192630856 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5c20021030 con 0x7f5c2c09dc30 2023-10-03T10:13:12.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.827+0000 7f5c13fff700 1 -- 172.21.15.3:0/1192630856 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5c20040980 con 0x7f5c2c09dc30 2023-10-03T10:13:12.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.827+0000 7f5c31e61700 1 -- 172.21.15.3:0/1192630856 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c2c09f1a0 con 0x7f5c2c09dc30 2023-10-03T10:13:12.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.833+0000 7f5c13fff700 1 -- 172.21.15.3:0/1192630856 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5c20006d80 con 0x7f5c2c09dc30 2023-10-03T10:13:12.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.854+0000 7ff7b2551700 1 Processor -- start 2023-10-03T10:13:12.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.854+0000 7ff7b2551700 1 -- start start 2023-10-03T10:13:12.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.855+0000 7ff7b2551700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7ac133420 0x7ff7ac1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:12.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.855+0000 7ff7b2551700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff7ac134260 0x7ff7ac1306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:12.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.855+0000 7ff7b2551700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7ac06b990 con 0x7ff7ac133e30 2023-10-03T10:13:12.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.855+0000 7ff7b2551700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac0a0590 con 0x7ff7ac133420 2023-10-03T10:13:12.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.855+0000 7ff7b2551700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac09fb80 con 0x7ff7ac134260 2023-10-03T10:13:12.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.855+0000 7ff7ab7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff7ac133e30 0x7ff7ac12ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49620/0 (socket says 172.21.15.3:49620) 2023-10-03T10:13:12.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.855+0000 7ff7ab7fe700 1 -- 172.21.15.3:0/1063893995 learned_addr learned my addr 172.21.15.3:0/1063893995 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:12.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.855+0000 7ff7abfff700 1 --2- 172.21.15.3:0/1063893995 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7ac133420 0x7ff7ac1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:12.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.855+0000 7ff7b0aee700 1 --2- 172.21.15.3:0/1063893995 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff7ac134260 0x7ff7ac1306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:12.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.855+0000 7ff7aaffd700 1 -- 172.21.15.3:0/1063893995 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 974058366 0 0) 0x7ff7ac06b990 con 0x7ff7ac133e30 2023-10-03T10:13:12.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.855+0000 7ff7aaffd700 1 -- 172.21.15.3:0/1063893995 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7940035b0 con 0x7ff7ac133e30 2023-10-03T10:13:12.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.856+0000 7ff7abfff700 1 -- 172.21.15.3:0/1063893995 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff7ac134260 msgr2=0x7ff7ac1306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:12.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.856+0000 7ff7abfff700 1 --2- 172.21.15.3:0/1063893995 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff7ac134260 0x7ff7ac1306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:12.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.856+0000 7ff7abfff700 1 -- 172.21.15.3:0/1063893995 >> v1:172.21.15.3:6789/0 conn(0x7ff7ac133e30 legacy=0x7ff7ac12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:12.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.856+0000 7ff7abfff700 1 -- 172.21.15.3:0/1063893995 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ac0a0030 con 0x7ff7ac133420 2023-10-03T10:13:12.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.856+0000 7ff7abfff700 1 --2- 172.21.15.3:0/1063893995 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7ac133420 0x7ff7ac1337f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff79c009870 tx=0x7ff79c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7993bc5c3e198d31 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:12.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.856+0000 7ff7aaffd700 1 -- 172.21.15.3:0/1063893995 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79c011830 con 0x7ff7ac133420 2023-10-03T10:13:12.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.856+0000 7ff7aaffd700 1 -- 172.21.15.3:0/1063893995 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff79c011990 con 0x7ff7ac133420 2023-10-03T10:13:12.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.856+0000 7ff7aaffd700 1 -- 172.21.15.3:0/1063893995 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79c011d20 con 0x7ff7ac133420 2023-10-03T10:13:12.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.857+0000 7ff7b2551700 1 -- 172.21.15.3:0/1063893995 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7ac133420 msgr2=0x7ff7ac1337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:12.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.857+0000 7ff7b2551700 1 --2- 172.21.15.3:0/1063893995 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7ac133420 0x7ff7ac1337f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff79c009870 tx=0x7ff79c009b80 comp rx=0 tx=0).stop 2023-10-03T10:13:12.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.857+0000 7ff7b2551700 1 -- 172.21.15.3:0/1063893995 shutdown_connections 2023-10-03T10:13:12.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.857+0000 7ff7b2551700 1 --2- 172.21.15.3:0/1063893995 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7ac133420 0x7ff7ac1337f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:12.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.857+0000 7ff7b2551700 1 --2- 172.21.15.3:0/1063893995 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff7ac134260 0x7ff7ac1306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:12.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.857+0000 7ff7b2551700 1 -- 172.21.15.3:0/1063893995 >> 172.21.15.3:0/1063893995 conn(0x7ff7ac0a4210 msgr2=0x7ff7ac0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:12.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.857+0000 7ff7b2551700 1 -- 172.21.15.3:0/1063893995 shutdown_connections 2023-10-03T10:13:12.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.857+0000 7ff7b2551700 1 -- 172.21.15.3:0/1063893995 wait complete. 2023-10-03T10:13:12.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.858+0000 7ff7b2551700 1 Processor -- start 2023-10-03T10:13:12.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.859+0000 7ff7b2551700 1 -- start start 2023-10-03T10:13:12.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.859+0000 7ff7b2551700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7ac133420 0x7ff7ac09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:12.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.859+0000 7ff7b2551700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff7ac133e30 0x7ff7ac09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:12.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.859+0000 7ff7b2551700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7ac134260 0x7ff7ac09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:12.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.859+0000 7ff7b2551700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac131690 con 0x7ff7ac133420 2023-10-03T10:13:12.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.859+0000 7ff7b2551700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac120630 con 0x7ff7ac134260 2023-10-03T10:13:12.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.859+0000 7ff7b2551700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac11f490 con 0x7ff7ac133e30 2023-10-03T10:13:12.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.859+0000 7ff7abfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7ac133420 0x7ff7ac09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:12.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.859+0000 7ff7abfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7ac133420 0x7ff7ac09c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49182/0 (socket says 172.21.15.3:49182) 2023-10-03T10:13:12.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.859+0000 7ff7abfff700 1 -- 172.21.15.3:0/3401826313 learned_addr learned my addr 172.21.15.3:0/3401826313 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:12.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.859+0000 7ff7b0aee700 1 --2- 172.21.15.3:0/3401826313 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7ac134260 0x7ff7ac09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:12.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.859+0000 7ff7ab7fe700 1 --2- 172.21.15.3:0/3401826313 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff7ac133e30 0x7ff7ac09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:12.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.860+0000 7ff7abfff700 1 -- 172.21.15.3:0/3401826313 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff7ac133e30 msgr2=0x7ff7ac09c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:12.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.860+0000 7ff7abfff700 1 --2- 172.21.15.3:0/3401826313 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff7ac133e30 0x7ff7ac09c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:12.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.860+0000 7ff7abfff700 1 -- 172.21.15.3:0/3401826313 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7ac134260 msgr2=0x7ff7ac09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:12.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.860+0000 7ff7abfff700 1 --2- 172.21.15.3:0/3401826313 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7ac134260 0x7ff7ac09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:12.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.860+0000 7ff7abfff700 1 -- 172.21.15.3:0/3401826313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79c0094a0 con 0x7ff7ac133420 2023-10-03T10:13:12.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.860+0000 7ff7ab7fe700 1 --2- 172.21.15.3:0/3401826313 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff7ac133e30 0x7ff7ac09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:12.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.860+0000 7ff7abfff700 1 --2- 172.21.15.3:0/3401826313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7ac133420 0x7ff7ac09c480 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff79c000c00 tx=0x7ff79c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:12.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.860+0000 7ff7a8ff9700 1 -- 172.21.15.3:0/3401826313 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79c004240 con 0x7ff7ac133420 2023-10-03T10:13:12.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.860+0000 7ff7a8ff9700 1 -- 172.21.15.3:0/3401826313 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff79c01f070 con 0x7ff7ac133420 2023-10-03T10:13:12.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.860+0000 7ff7b2551700 1 -- 172.21.15.3:0/3401826313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7ac11fb90 con 0x7ff7ac133420 2023-10-03T10:13:12.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.860+0000 7ff7a8ff9700 1 -- 172.21.15.3:0/3401826313 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79c0045e0 con 0x7ff7ac133420 2023-10-03T10:13:12.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.860+0000 7ff7b2551700 1 -- 172.21.15.3:0/3401826313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7ac120130 con 0x7ff7ac133420 2023-10-03T10:13:12.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.861+0000 7ff7a8ff9700 1 -- 172.21.15.3:0/3401826313 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff79c021030 con 0x7ff7ac133420 2023-10-03T10:13:12.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.861+0000 7ff7b2551700 1 -- 172.21.15.3:0/3401826313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7ac074ea0 con 0x7ff7ac133420 2023-10-03T10:13:12.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.861+0000 7ff7a8ff9700 1 -- 172.21.15.3:0/3401826313 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff79c040700 con 0x7ff7ac133420 2023-10-03T10:13:12.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.867+0000 7ff7a8ff9700 1 -- 172.21.15.3:0/3401826313 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff79c0197c0 con 0x7ff7ac133420 2023-10-03T10:13:12.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.903+0000 7f7921747700 1 Processor -- start 2023-10-03T10:13:12.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.903+0000 7f7921747700 1 -- start start 2023-10-03T10:13:12.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.904+0000 7f7921747700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f791c09d6f0 0x7f791c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:12.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.904+0000 7f7921747700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f791c09e000 0x7f791c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:12.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.904+0000 7f7921747700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f791c09e410 0x7f791c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:12.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.904+0000 7f7921747700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f791c09f520 con 0x7f791c09e000 2023-10-03T10:13:12.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.904+0000 7f7921747700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791c09f130 con 0x7f791c09e410 2023-10-03T10:13:12.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.904+0000 7f7921747700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f791c09e930 con 0x7f791c09d6f0 2023-10-03T10:13:12.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.904+0000 7f791affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f791c09d6f0 0x7f791c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:12.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.905+0000 7f791affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f791c09d6f0 0x7f791c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:49144/0 (socket says 172.21.15.3:49144) 2023-10-03T10:13:12.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.905+0000 7f791affd700 1 -- 172.21.15.3:0/1784279568 learned_addr learned my addr 172.21.15.3:0/1784279568 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:12.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.905+0000 7f791b7fe700 1 --2- 172.21.15.3:0/1784279568 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f791c09e410 0x7f791c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:12.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.905+0000 7f791a7fc700 1 --2- 172.21.15.3:0/1784279568 >> v2:172.21.15.3:3300/0 conn(0x7f791c09e000 0x7f791c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:12.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.905+0000 7f791b7fe700 1 -- 172.21.15.3:0/1784279568 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f791c09d6f0 msgr2=0x7f791c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:12.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.905+0000 7f791b7fe700 1 --2- 172.21.15.3:0/1784279568 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f791c09d6f0 0x7f791c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:12.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.905+0000 7f791b7fe700 1 -- 172.21.15.3:0/1784279568 >> v2:172.21.15.3:3300/0 conn(0x7f791c09e000 msgr2=0x7f791c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:12.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.905+0000 7f791b7fe700 1 --2- 172.21.15.3:0/1784279568 >> v2:172.21.15.3:3300/0 conn(0x7f791c09e000 0x7f791c130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:12.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.905+0000 7f791b7fe700 1 -- 172.21.15.3:0/1784279568 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791c09b2d0 con 0x7f791c09e410 2023-10-03T10:13:12.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.905+0000 7f791b7fe700 1 --2- 172.21.15.3:0/1784279568 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f791c09e410 0x7f791c130ba0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7914009d90 tx=0x7f791400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=100781def9a0f98e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:12.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.905+0000 7f7919ffb700 1 -- 172.21.15.3:0/1784279568 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791400e070 con 0x7f791c09e410 2023-10-03T10:13:12.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.905+0000 7f7919ffb700 1 -- 172.21.15.3:0/1784279568 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f791400bb20 con 0x7f791c09e410 2023-10-03T10:13:12.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.905+0000 7f7919ffb700 1 -- 172.21.15.3:0/1784279568 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7914012430 con 0x7f791c09e410 2023-10-03T10:13:12.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.906+0000 7f7921747700 1 -- 172.21.15.3:0/1784279568 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f791c09e410 msgr2=0x7f791c130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:12.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.906+0000 7f7921747700 1 --2- 172.21.15.3:0/1784279568 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f791c09e410 0x7f791c130ba0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7914009d90 tx=0x7f791400b330 comp rx=0 tx=0).stop 2023-10-03T10:13:12.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.906+0000 7f7921747700 1 -- 172.21.15.3:0/1784279568 shutdown_connections 2023-10-03T10:13:12.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.906+0000 7f7921747700 1 --2- 172.21.15.3:0/1784279568 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f791c09e410 0x7f791c130ba0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:12.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.906+0000 7f7921747700 1 --2- 172.21.15.3:0/1784279568 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f791c09d6f0 0x7f791c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:12.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.906+0000 7f7921747700 1 --2- 172.21.15.3:0/1784279568 >> v2:172.21.15.3:3300/0 conn(0x7f791c09e000 0x7f791c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:12.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.906+0000 7f7921747700 1 -- 172.21.15.3:0/1784279568 >> 172.21.15.3:0/1784279568 conn(0x7f791c066f50 msgr2=0x7f791c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:12.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.906+0000 7f7921747700 1 -- 172.21.15.3:0/1784279568 shutdown_connections 2023-10-03T10:13:12.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.906+0000 7f7921747700 1 -- 172.21.15.3:0/1784279568 wait complete. 2023-10-03T10:13:12.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.908+0000 7f7921747700 1 Processor -- start 2023-10-03T10:13:12.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.908+0000 7f7921747700 1 -- start start 2023-10-03T10:13:12.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.909+0000 7f7921747700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f791c09d6f0 0x7f791c1221b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:12.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.909+0000 7f7921747700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f791c09e000 0x7f791c1226f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:12.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.909+0000 7f7921747700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f791c09e410 0x7f791c122c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:12.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.909+0000 7f7921747700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791c124410 con 0x7f791c09e410 2023-10-03T10:13:12.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.909+0000 7f7921747700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791c09c700 con 0x7f791c09d6f0 2023-10-03T10:13:12.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.909+0000 7f7921747700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f791c123ed0 con 0x7f791c09e000 2023-10-03T10:13:12.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.909+0000 7f791affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f791c09d6f0 0x7f791c1221b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:12.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.909+0000 7f791affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f791c09d6f0 0x7f791c1221b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41194/0 (socket says 172.21.15.3:41194) 2023-10-03T10:13:12.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.909+0000 7f791affd700 1 -- 172.21.15.3:0/3431332555 learned_addr learned my addr 172.21.15.3:0/3431332555 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:12.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.909+0000 7f791b7fe700 1 --2- 172.21.15.3:0/3431332555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f791c09e410 0x7f791c122c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:12.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.909+0000 7f791a7fc700 1 --2- 172.21.15.3:0/3431332555 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f791c09e000 0x7f791c1226f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:12.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.909+0000 7f791affd700 1 -- 172.21.15.3:0/3431332555 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f791c09e000 msgr2=0x7f791c1226f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:12.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.909+0000 7f791affd700 1 --2- 172.21.15.3:0/3431332555 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f791c09e000 0x7f791c1226f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:12.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.909+0000 7f791affd700 1 -- 172.21.15.3:0/3431332555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f791c09e410 msgr2=0x7f791c122c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:12.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.909+0000 7f791affd700 1 --2- 172.21.15.3:0/3431332555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f791c09e410 0x7f791c122c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:12.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.909+0000 7f791affd700 1 -- 172.21.15.3:0/3431332555 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7914009a70 con 0x7f791c09d6f0 2023-10-03T10:13:12.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.909+0000 7f791a7fc700 1 --2- 172.21.15.3:0/3431332555 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f791c09e000 0x7f791c1226f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:12.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.910+0000 7f791affd700 1 --2- 172.21.15.3:0/3431332555 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f791c09d6f0 0x7f791c1221b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f790c00ed40 tx=0x7f790c00d630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:12.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.910+0000 7f79037fe700 1 -- 172.21.15.3:0/3431332555 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c00de40 con 0x7f791c09d6f0 2023-10-03T10:13:12.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.910+0000 7f79037fe700 1 -- 172.21.15.3:0/3431332555 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790c010440 con 0x7f791c09d6f0 2023-10-03T10:13:12.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.910+0000 7f79037fe700 1 -- 172.21.15.3:0/3431332555 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c0107b0 con 0x7f791c09d6f0 2023-10-03T10:13:12.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.910+0000 7f7921747700 1 -- 172.21.15.3:0/3431332555 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f791c1259f0 con 0x7f791c09d6f0 2023-10-03T10:13:12.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.910+0000 7f7921747700 1 -- 172.21.15.3:0/3431332555 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f791c126460 con 0x7f791c09d6f0 2023-10-03T10:13:12.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.911+0000 7f7921747700 1 -- 172.21.15.3:0/3431332555 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f791c074ea0 con 0x7f791c09d6f0 2023-10-03T10:13:12.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.911+0000 7f79037fe700 1 -- 172.21.15.3:0/3431332555 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f790c010910 con 0x7f791c09d6f0 2023-10-03T10:13:12.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.911+0000 7f79037fe700 1 -- 172.21.15.3:0/3431332555 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f790c014070 con 0x7f791c09d6f0 2023-10-03T10:13:12.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.921+0000 7f79037fe700 1 -- 172.21.15.3:0/3431332555 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f790c016500 con 0x7f791c09d6f0 2023-10-03T10:13:12.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.948+0000 7f3f9c3ab700 1 Processor -- start 2023-10-03T10:13:12.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.948+0000 7f3f9c3ab700 1 -- start start 2023-10-03T10:13:12.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.948+0000 7f3f9c3ab700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f9409d6f0 0x7f3f9409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:12.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.948+0000 7f3f9c3ab700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3f9409e000 0x7f3f94130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:12.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.948+0000 7f3f9c3ab700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3f9409e410 0x7f3f94130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:12.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.948+0000 7f3f9c3ab700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f9409f520 con 0x7f3f9409e000 2023-10-03T10:13:12.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.948+0000 7f3f9c3ab700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9409f130 con 0x7f3f9409e410 2023-10-03T10:13:12.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.948+0000 7f3f9c3ab700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9409e930 con 0x7f3f9409d6f0 2023-10-03T10:13:12.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.949+0000 7f3f99946700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3f9409e000 0x7f3f94130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:12.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.949+0000 7f3f99946700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3f9409e000 0x7f3f94130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49198/0 (socket says 172.21.15.3:49198) 2023-10-03T10:13:12.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.949+0000 7f3f99946700 1 -- 172.21.15.3:0/159171096 learned_addr learned my addr 172.21.15.3:0/159171096 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:12.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.949+0000 7f3f9a147700 1 --2- 172.21.15.3:0/159171096 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f9409d6f0 0x7f3f9409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:12.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.949+0000 7f3f9a948700 1 --2- 172.21.15.3:0/159171096 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3f9409e410 0x7f3f94130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:12.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.949+0000 7f3f99946700 1 -- 172.21.15.3:0/159171096 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f9409d6f0 msgr2=0x7f3f9409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:12.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.949+0000 7f3f99946700 1 --2- 172.21.15.3:0/159171096 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f9409d6f0 0x7f3f9409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:12.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.949+0000 7f3f99946700 1 -- 172.21.15.3:0/159171096 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3f9409e410 msgr2=0x7f3f94130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:12.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.949+0000 7f3f99946700 1 --2- 172.21.15.3:0/159171096 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3f9409e410 0x7f3f94130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:12.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.949+0000 7f3f99946700 1 -- 172.21.15.3:0/159171096 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f9409b2d0 con 0x7f3f9409e000 2023-10-03T10:13:12.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.949+0000 7f3f9a147700 1 --2- 172.21.15.3:0/159171096 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f9409d6f0 0x7f3f9409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:12.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.949+0000 7f3f99946700 1 --2- 172.21.15.3:0/159171096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f9409e000 0x7f3f94130660 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3f8400bbc0 tx=0x7f3f8400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d2948d44e4ca7f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:12.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.950+0000 7f3f99145700 1 -- 172.21.15.3:0/159171096 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8400dcf0 con 0x7f3f9409e000 2023-10-03T10:13:12.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.950+0000 7f3f99145700 1 -- 172.21.15.3:0/159171096 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f8400de50 con 0x7f3f9409e000 2023-10-03T10:13:12.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.950+0000 7f3f99145700 1 -- 172.21.15.3:0/159171096 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f84010690 con 0x7f3f9409e000 2023-10-03T10:13:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.951+0000 7f3f9c3ab700 1 -- 172.21.15.3:0/159171096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f9409e000 msgr2=0x7f3f94130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.951+0000 7f3f9c3ab700 1 --2- 172.21.15.3:0/159171096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f9409e000 0x7f3f94130660 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3f8400bbc0 tx=0x7f3f8400bed0 comp rx=0 tx=0).stop 2023-10-03T10:13:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.951+0000 7f3f9c3ab700 1 -- 172.21.15.3:0/159171096 shutdown_connections 2023-10-03T10:13:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.951+0000 7f3f9c3ab700 1 --2- 172.21.15.3:0/159171096 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3f9409e410 0x7f3f94130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.951+0000 7f3f9c3ab700 1 --2- 172.21.15.3:0/159171096 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f9409d6f0 0x7f3f9409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.951+0000 7f3f9c3ab700 1 --2- 172.21.15.3:0/159171096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f9409e000 0x7f3f94130660 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.951+0000 7f3f9c3ab700 1 -- 172.21.15.3:0/159171096 >> 172.21.15.3:0/159171096 conn(0x7f3f94066f50 msgr2=0x7f3f9409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.951+0000 7f3f9c3ab700 1 -- 172.21.15.3:0/159171096 shutdown_connections 2023-10-03T10:13:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.951+0000 7f3f9c3ab700 1 -- 172.21.15.3:0/159171096 wait complete. 2023-10-03T10:13:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.952+0000 7f3f9c3ab700 1 Processor -- start 2023-10-03T10:13:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.952+0000 7f3f9c3ab700 1 -- start start 2023-10-03T10:13:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.953+0000 7f3f9c3ab700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f9409d6f0 0x7f3f941225a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.953+0000 7f3f9c3ab700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f9409e000 0x7f3f94122ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.953+0000 7f3f9c3ab700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3f9409e410 0x7f3f94123020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.953+0000 7f3f9c3ab700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f94123a50 con 0x7f3f9409d6f0 2023-10-03T10:13:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.953+0000 7f3f9c3ab700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f94124230 con 0x7f3f9409e410 2023-10-03T10:13:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.953+0000 7f3f9c3ab700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3f94130000 con 0x7f3f9409e000 2023-10-03T10:13:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.953+0000 7f3f9a147700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f9409d6f0 0x7f3f941225a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.953+0000 7f3f99946700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f9409e000 0x7f3f94122ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.953+0000 7f3f9a147700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f9409d6f0 0x7f3f941225a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49200/0 (socket says 172.21.15.3:49200) 2023-10-03T10:13:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.953+0000 7f3f9a147700 1 -- 172.21.15.3:0/975703574 learned_addr learned my addr 172.21.15.3:0/975703574 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.953+0000 7f3f9a948700 1 --2- 172.21.15.3:0/975703574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3f9409e410 0x7f3f94123020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.954+0000 7f3f99946700 1 -- 172.21.15.3:0/975703574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3f9409e410 msgr2=0x7f3f94123020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.954+0000 7f3f99946700 1 --2- 172.21.15.3:0/975703574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3f9409e410 0x7f3f94123020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.954+0000 7f3f99946700 1 -- 172.21.15.3:0/975703574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f9409d6f0 msgr2=0x7f3f941225a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.954+0000 7f3f99946700 1 --2- 172.21.15.3:0/975703574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f9409d6f0 0x7f3f941225a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.954+0000 7f3f99946700 1 -- 172.21.15.3:0/975703574 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f900094a0 con 0x7f3f9409e000 2023-10-03T10:13:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.954+0000 7f3f9a948700 1 --2- 172.21.15.3:0/975703574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3f9409e410 0x7f3f94123020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.954+0000 7f3f99946700 1 --2- 172.21.15.3:0/975703574 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f9409e000 0x7f3f94122ae0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3f84018000 tx=0x7f3f84004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:12.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.954+0000 7f3f8affd700 1 -- 172.21.15.3:0/975703574 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f84003b70 con 0x7f3f9409e000 2023-10-03T10:13:12.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.955+0000 7f3f8affd700 1 -- 172.21.15.3:0/975703574 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f84003cd0 con 0x7f3f9409e000 2023-10-03T10:13:12.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.955+0000 7f3f9c3ab700 1 -- 172.21.15.3:0/975703574 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f8400b7f0 con 0x7f3f9409e000 2023-10-03T10:13:12.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.955+0000 7f3f8affd700 1 -- 172.21.15.3:0/975703574 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f84010690 con 0x7f3f9409e000 2023-10-03T10:13:12.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.955+0000 7f3f9c3ab700 1 -- 172.21.15.3:0/975703574 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f94125d50 con 0x7f3f9409e000 2023-10-03T10:13:12.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.955+0000 7f3f8affd700 1 -- 172.21.15.3:0/975703574 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3f84019400 con 0x7f3f9409e000 2023-10-03T10:13:12.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.956+0000 7f3f8affd700 1 -- 172.21.15.3:0/975703574 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3f84040d30 con 0x7f3f9409e000 2023-10-03T10:13:12.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.956+0000 7f3f88ff9700 1 -- 172.21.15.3:0/975703574 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f700052e0 con 0x7f3f9409e000 2023-10-03T10:13:12.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:12.961+0000 7f3f8affd700 1 -- 172.21.15.3:0/975703574 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3f8400f3d0 con 0x7f3f9409e000 2023-10-03T10:13:13.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.073+0000 7f5c31e61700 1 -- 172.21.15.3:0/1192630856 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5c2c129800 con 0x7f5c2c09dc30 2023-10-03T10:13:13.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.074+0000 7f5c13fff700 1 -- 172.21.15.3:0/1192630856 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5c2001b430 con 0x7f5c2c09dc30 2023-10-03T10:13:13.074 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:13.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.077+0000 7f5c31e61700 1 -- 172.21.15.3:0/1192630856 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c2c09dc30 msgr2=0x7f5c2c1272e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:13.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.077+0000 7f5c31e61700 1 --2- 172.21.15.3:0/1192630856 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c2c09dc30 0x7f5c2c1272e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5c200098f0 tx=0x7f5c20004750 comp rx=0 tx=0).stop 2023-10-03T10:13:13.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.077+0000 7f5c31e61700 1 -- 172.21.15.3:0/1192630856 shutdown_connections 2023-10-03T10:13:13.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.077+0000 7f5c31e61700 1 --2- 172.21.15.3:0/1192630856 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c2c13c1e0 0x7f5c2c1cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:13.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.077+0000 7f5c31e61700 1 --2- 172.21.15.3:0/1192630856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c2c06a770 0x7f5c2c126da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:13.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.077+0000 7f5c31e61700 1 --2- 172.21.15.3:0/1192630856 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c2c09dc30 0x7f5c2c1272e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:13.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.077+0000 7f5c31e61700 1 -- 172.21.15.3:0/1192630856 >> 172.21.15.3:0/1192630856 conn(0x7f5c2c066f50 msgr2=0x7f5c2c13b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:13.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.077+0000 7f5c31e61700 1 -- 172.21.15.3:0/1192630856 shutdown_connections 2023-10-03T10:13:13.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.078+0000 7f5c31e61700 1 -- 172.21.15.3:0/1192630856 wait complete. 2023-10-03T10:13:13.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.095+0000 7ff7b2551700 1 -- 172.21.15.3:0/3401826313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff7ac06b250 con 0x7ff7ac133420 2023-10-03T10:13:13.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.095+0000 7ff7a8ff9700 1 -- 172.21.15.3:0/3401826313 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff79c0233d0 con 0x7ff7ac133420 2023-10-03T10:13:13.096 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:13.097 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:13.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.098+0000 7ff7b2551700 1 -- 172.21.15.3:0/3401826313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7ac133420 msgr2=0x7ff7ac09c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:13.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.098+0000 7ff7b2551700 1 --2- 172.21.15.3:0/3401826313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7ac133420 0x7ff7ac09c480 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff79c000c00 tx=0x7ff79c004750 comp rx=0 tx=0).stop 2023-10-03T10:13:13.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.099+0000 7ff7b2551700 1 -- 172.21.15.3:0/3401826313 shutdown_connections 2023-10-03T10:13:13.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.099+0000 7ff7b2551700 1 --2- 172.21.15.3:0/3401826313 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7ac134260 0x7ff7ac09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:13.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.099+0000 7ff7b2551700 1 --2- 172.21.15.3:0/3401826313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7ac133420 0x7ff7ac09c480 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:13.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.099+0000 7ff7b2551700 1 --2- 172.21.15.3:0/3401826313 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff7ac133e30 0x7ff7ac09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:13.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.099+0000 7ff7b2551700 1 -- 172.21.15.3:0/3401826313 >> 172.21.15.3:0/3401826313 conn(0x7ff7ac0a4210 msgr2=0x7ff7ac0a30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:13.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.099+0000 7ff7b2551700 1 -- 172.21.15.3:0/3401826313 shutdown_connections 2023-10-03T10:13:13.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.099+0000 7ff7b2551700 1 -- 172.21.15.3:0/3401826313 wait complete. 2023-10-03T10:13:13.113 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:13.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.138+0000 7f7921747700 1 -- 172.21.15.3:0/3431332555 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f791c06c360 con 0x7f791c09d6f0 2023-10-03T10:13:13.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.138+0000 7f79037fe700 1 -- 172.21.15.3:0/3431332555 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f790c016320 con 0x7f791c09d6f0 2023-10-03T10:13:13.138 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:13.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.140+0000 7f7921747700 1 -- 172.21.15.3:0/3431332555 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f791c09d6f0 msgr2=0x7f791c1221b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:13.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.140+0000 7f7921747700 1 --2- 172.21.15.3:0/3431332555 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f791c09d6f0 0x7f791c1221b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f790c00ed40 tx=0x7f790c00d630 comp rx=0 tx=0).stop 2023-10-03T10:13:13.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.140+0000 7f7921747700 1 -- 172.21.15.3:0/3431332555 shutdown_connections 2023-10-03T10:13:13.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.140+0000 7f7921747700 1 --2- 172.21.15.3:0/3431332555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f791c09e410 0x7f791c122c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:13.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.140+0000 7f7921747700 1 --2- 172.21.15.3:0/3431332555 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f791c09d6f0 0x7f791c1221b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:13.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.140+0000 7f7921747700 1 --2- 172.21.15.3:0/3431332555 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f791c09e000 0x7f791c1226f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:13.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.140+0000 7f7921747700 1 -- 172.21.15.3:0/3431332555 >> 172.21.15.3:0/3431332555 conn(0x7f791c066f50 msgr2=0x7f791c130800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:13.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.141+0000 7f7921747700 1 -- 172.21.15.3:0/3431332555 shutdown_connections 2023-10-03T10:13:13.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.141+0000 7f7921747700 1 -- 172.21.15.3:0/3431332555 wait complete. 2023-10-03T10:13:13.155 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:13.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.169+0000 7f3f88ff9700 1 -- 172.21.15.3:0/975703574 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3f70005590 con 0x7f3f9409e000 2023-10-03T10:13:13.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.169+0000 7f3f8affd700 1 -- 172.21.15.3:0/975703574 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3f84017300 con 0x7f3f9409e000 2023-10-03T10:13:13.169 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:13.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.171+0000 7f3f88ff9700 1 -- 172.21.15.3:0/975703574 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f9409e000 msgr2=0x7f3f94122ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:13.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.171+0000 7f3f88ff9700 1 --2- 172.21.15.3:0/975703574 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f9409e000 0x7f3f94122ae0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3f84018000 tx=0x7f3f84004750 comp rx=0 tx=0).stop 2023-10-03T10:13:13.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.171+0000 7f3f88ff9700 1 -- 172.21.15.3:0/975703574 shutdown_connections 2023-10-03T10:13:13.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.171+0000 7f3f88ff9700 1 --2- 172.21.15.3:0/975703574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3f9409e410 0x7f3f94123020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:13.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.171+0000 7f3f88ff9700 1 --2- 172.21.15.3:0/975703574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f9409d6f0 0x7f3f941225a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:13.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.171+0000 7f3f88ff9700 1 --2- 172.21.15.3:0/975703574 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f9409e000 0x7f3f94122ae0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:13.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.171+0000 7f3f88ff9700 1 -- 172.21.15.3:0/975703574 >> 172.21.15.3:0/975703574 conn(0x7f3f94066f50 msgr2=0x7f3f940a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:13.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.171+0000 7f3f88ff9700 1 -- 172.21.15.3:0/975703574 shutdown_connections 2023-10-03T10:13:13.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:13.171+0000 7f3f88ff9700 1 -- 172.21.15.3:0/975703574 wait complete. 2023-10-03T10:13:13.185 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:14.098 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:14.114 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:14.156 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:14.185 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:14.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.262+0000 7f471677f700 1 Processor -- start 2023-10-03T10:13:14.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.262+0000 7f471677f700 1 -- start start 2023-10-03T10:13:14.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.263+0000 7f471677f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f471009e1d0 0x7f4710138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:14.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.263+0000 7f471677f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4710139170 0x7f471013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:14.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.263+0000 7f471677f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f471009b120 con 0x7f471009d6f0 2023-10-03T10:13:14.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.263+0000 7f471677f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f471006b460 con 0x7f471009e1d0 2023-10-03T10:13:14.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.263+0000 7f471677f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f471009a7e0 con 0x7f4710139170 2023-10-03T10:13:14.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.263+0000 7f4714d1c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4710139170 0x7f471013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:14.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.263+0000 7f4714d1c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4710139170 0x7f471013b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36516/0 (socket says 172.21.15.3:36516) 2023-10-03T10:13:14.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.263+0000 7f4714d1c700 1 -- 172.21.15.3:0/3125443807 learned_addr learned my addr 172.21.15.3:0/3125443807 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:14.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.263+0000 7f470f7fe700 1 --2- 172.21.15.3:0/3125443807 >> v2:172.21.15.3:3300/0 conn(0x7f471009e1d0 0x7f4710138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:14.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.264+0000 7f4714d1c700 1 -- 172.21.15.3:0/3125443807 >> v2:172.21.15.3:3300/0 conn(0x7f471009e1d0 msgr2=0x7f4710138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:14.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.264+0000 7f4714d1c700 1 --2- 172.21.15.3:0/3125443807 >> v2:172.21.15.3:3300/0 conn(0x7f471009e1d0 0x7f4710138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.264+0000 7f4714d1c700 1 -- 172.21.15.3:0/3125443807 >> v1:172.21.15.3:6789/0 conn(0x7f471009d6f0 legacy=0x7f471009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:14.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.264+0000 7f4714d1c700 1 -- 172.21.15.3:0/3125443807 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471009a500 con 0x7f4710139170 2023-10-03T10:13:14.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.264+0000 7f4714d1c700 1 --2- 172.21.15.3:0/3125443807 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4710139170 0x7f471013b550 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4708009e80 tx=0x7f470800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38e2ad66f042926c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:14.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.264+0000 7f470effd700 1 -- 172.21.15.3:0/3125443807 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470800e070 con 0x7f4710139170 2023-10-03T10:13:14.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.264+0000 7f470effd700 1 -- 172.21.15.3:0/3125443807 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470800b8d0 con 0x7f4710139170 2023-10-03T10:13:14.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.264+0000 7f470effd700 1 -- 172.21.15.3:0/3125443807 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470800bc80 con 0x7f4710139170 2023-10-03T10:13:14.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.265+0000 7f471677f700 1 -- 172.21.15.3:0/3125443807 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4710139170 msgr2=0x7f471013b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:14.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.265+0000 7f471677f700 1 --2- 172.21.15.3:0/3125443807 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4710139170 0x7f471013b550 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4708009e80 tx=0x7f470800b330 comp rx=0 tx=0).stop 2023-10-03T10:13:14.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.265+0000 7f471677f700 1 -- 172.21.15.3:0/3125443807 shutdown_connections 2023-10-03T10:13:14.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.265+0000 7f471677f700 1 --2- 172.21.15.3:0/3125443807 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4710139170 0x7f471013b550 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.265+0000 7f471677f700 1 --2- 172.21.15.3:0/3125443807 >> v2:172.21.15.3:3300/0 conn(0x7f471009e1d0 0x7f4710138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.265+0000 7f471677f700 1 -- 172.21.15.3:0/3125443807 >> 172.21.15.3:0/3125443807 conn(0x7f4710066f50 msgr2=0x7f471009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:14.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.265+0000 7f471677f700 1 -- 172.21.15.3:0/3125443807 shutdown_connections 2023-10-03T10:13:14.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.265+0000 7f471677f700 1 -- 172.21.15.3:0/3125443807 wait complete. 2023-10-03T10:13:14.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.267+0000 7f471677f700 1 Processor -- start 2023-10-03T10:13:14.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.267+0000 7f471677f700 1 -- start start 2023-10-03T10:13:14.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.267+0000 7f471677f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f471009d6f0 0x7f47101c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:14.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.268+0000 7f471677f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f471009e1d0 0x7f47101c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:14.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.268+0000 7f471677f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4710139170 0x7f471011e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:14.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.268+0000 7f471677f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4710121fe0 con 0x7f4710139170 2023-10-03T10:13:14.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.268+0000 7f471677f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47101be250 con 0x7f471009d6f0 2023-10-03T10:13:14.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.268+0000 7f471677f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4710122230 con 0x7f471009e1d0 2023-10-03T10:13:14.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.268+0000 7f470f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f471009e1d0 0x7f47101c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:14.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.268+0000 7f470f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f471009e1d0 0x7f47101c8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55348/0 (socket says 172.21.15.3:55348) 2023-10-03T10:13:14.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.268+0000 7f470f7fe700 1 -- 172.21.15.3:0/2603247839 learned_addr learned my addr 172.21.15.3:0/2603247839 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:14.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.268+0000 7f470ffff700 1 --2- 172.21.15.3:0/2603247839 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f471009d6f0 0x7f47101c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:14.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.268+0000 7f4714d1c700 1 --2- 172.21.15.3:0/2603247839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4710139170 0x7f471011e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:14.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.268+0000 7f470f7fe700 1 -- 172.21.15.3:0/2603247839 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f471009d6f0 msgr2=0x7f47101c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:14.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.268+0000 7f470f7fe700 1 --2- 172.21.15.3:0/2603247839 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f471009d6f0 0x7f47101c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.268+0000 7f470f7fe700 1 -- 172.21.15.3:0/2603247839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4710139170 msgr2=0x7f471011e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:14.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.268+0000 7f470f7fe700 1 --2- 172.21.15.3:0/2603247839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4710139170 0x7f471011e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.268+0000 7f470f7fe700 1 -- 172.21.15.3:0/2603247839 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4708009b30 con 0x7f471009e1d0 2023-10-03T10:13:14.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.269+0000 7f470f7fe700 1 --2- 172.21.15.3:0/2603247839 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f471009e1d0 0x7f47101c8a50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f470400e9b0 tx=0x7f470400ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:14.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.269+0000 7f470cff9700 1 -- 172.21.15.3:0/2603247839 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704009710 con 0x7f471009e1d0 2023-10-03T10:13:14.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.269+0000 7f470cff9700 1 -- 172.21.15.3:0/2603247839 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470400ee50 con 0x7f471009e1d0 2023-10-03T10:13:14.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.269+0000 7f471677f700 1 -- 172.21.15.3:0/2603247839 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4710120320 con 0x7f471009e1d0 2023-10-03T10:13:14.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.269+0000 7f470cff9700 1 -- 172.21.15.3:0/2603247839 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470400ca50 con 0x7f471009e1d0 2023-10-03T10:13:14.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.269+0000 7f471677f700 1 -- 172.21.15.3:0/2603247839 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47101345e0 con 0x7f471009e1d0 2023-10-03T10:13:14.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.270+0000 7f470cff9700 1 -- 172.21.15.3:0/2603247839 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f470400cbb0 con 0x7f471009e1d0 2023-10-03T10:13:14.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.270+0000 7f470cff9700 1 -- 172.21.15.3:0/2603247839 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f470401a360 con 0x7f471009e1d0 2023-10-03T10:13:14.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.271+0000 7f471677f700 1 -- 172.21.15.3:0/2603247839 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4710074ea0 con 0x7f471009e1d0 2023-10-03T10:13:14.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.280+0000 7f470cff9700 1 -- 172.21.15.3:0/2603247839 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4704017020 con 0x7f471009e1d0 2023-10-03T10:13:14.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.294+0000 7f897faf0700 1 Processor -- start 2023-10-03T10:13:14.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.295+0000 7f897faf0700 1 -- start start 2023-10-03T10:13:14.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.295+0000 7f897faf0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f897806a770 0x7f897809d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:14.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.295+0000 7f897faf0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f897809dc30 0x7f897809e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:14.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.295+0000 7f897faf0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f897809a810 con 0x7f897813c1e0 2023-10-03T10:13:14.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.295+0000 7f897faf0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897806ab40 con 0x7f897809dc30 2023-10-03T10:13:14.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.295+0000 7f897faf0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f897809ebc0 con 0x7f897806a770 2023-10-03T10:13:14.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.296+0000 7f897d08b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f897809dc30 0x7f897809e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:14.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.296+0000 7f897d08b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f897809dc30 0x7f897809e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36520/0 (socket says 172.21.15.3:36520) 2023-10-03T10:13:14.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.296+0000 7f897d08b700 1 -- 172.21.15.3:0/2989084406 learned_addr learned my addr 172.21.15.3:0/2989084406 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:14.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.296+0000 7f897d88c700 1 --2- 172.21.15.3:0/2989084406 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f897806a770 0x7f897809d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:14.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.296+0000 7f897d08b700 1 -- 172.21.15.3:0/2989084406 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f897806a770 msgr2=0x7f897809d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:14.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.296+0000 7f897d08b700 1 --2- 172.21.15.3:0/2989084406 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f897806a770 0x7f897809d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.296+0000 7f897d08b700 1 -- 172.21.15.3:0/2989084406 >> v1:172.21.15.3:6789/0 conn(0x7f897813c1e0 legacy=0x7f897813e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:14.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.296+0000 7f897d08b700 1 -- 172.21.15.3:0/2989084406 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897809e920 con 0x7f897809dc30 2023-10-03T10:13:14.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.296+0000 7f897d08b700 1 --2- 172.21.15.3:0/2989084406 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f897809dc30 0x7f897809e080 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8968009920 tx=0x7f8968009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=889796180da20297 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:14.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.296+0000 7f897c88a700 1 -- 172.21.15.3:0/2989084406 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8968011830 con 0x7f897809dc30 2023-10-03T10:13:14.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.297+0000 7f897c88a700 1 -- 172.21.15.3:0/2989084406 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8968011990 con 0x7f897809dc30 2023-10-03T10:13:14.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.297+0000 7f897c88a700 1 -- 172.21.15.3:0/2989084406 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8968011d20 con 0x7f897809dc30 2023-10-03T10:13:14.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.297+0000 7f897faf0700 1 -- 172.21.15.3:0/2989084406 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f897809dc30 msgr2=0x7f897809e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:14.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.297+0000 7f897faf0700 1 --2- 172.21.15.3:0/2989084406 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f897809dc30 0x7f897809e080 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8968009920 tx=0x7f8968009c30 comp rx=0 tx=0).stop 2023-10-03T10:13:14.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.297+0000 7f897faf0700 1 -- 172.21.15.3:0/2989084406 shutdown_connections 2023-10-03T10:13:14.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.297+0000 7f897faf0700 1 --2- 172.21.15.3:0/2989084406 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f897806a770 0x7f897809d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.297+0000 7f897faf0700 1 --2- 172.21.15.3:0/2989084406 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f897809dc30 0x7f897809e080 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.297+0000 7f897faf0700 1 -- 172.21.15.3:0/2989084406 >> 172.21.15.3:0/2989084406 conn(0x7f8978066f50 msgr2=0x7f897809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:14.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.298+0000 7f897faf0700 1 -- 172.21.15.3:0/2989084406 shutdown_connections 2023-10-03T10:13:14.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.298+0000 7f897faf0700 1 -- 172.21.15.3:0/2989084406 wait complete. 2023-10-03T10:13:14.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.299+0000 7f897faf0700 1 Processor -- start 2023-10-03T10:13:14.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.299+0000 7f897faf0700 1 -- start start 2023-10-03T10:13:14.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.299+0000 7f897faf0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f897806a770 0x7f897811b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:14.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.299+0000 7f897faf0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f897809dc30 0x7f897813a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:14.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.299+0000 7f897faf0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f897813c1e0 0x7f897813ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:14.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.299+0000 7f897faf0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897811d1b0 con 0x7f897809dc30 2023-10-03T10:13:14.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.299+0000 7f897faf0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897811bf90 con 0x7f897813c1e0 2023-10-03T10:13:14.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.299+0000 7f897faf0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f897813bee0 con 0x7f897806a770 2023-10-03T10:13:14.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.299+0000 7f897e08d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f897813c1e0 0x7f897813ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:14.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.300+0000 7f897e08d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f897813c1e0 0x7f897813ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36526/0 (socket says 172.21.15.3:36526) 2023-10-03T10:13:14.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.300+0000 7f897e08d700 1 -- 172.21.15.3:0/1377306293 learned_addr learned my addr 172.21.15.3:0/1377306293 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:14.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.300+0000 7f897d88c700 1 --2- 172.21.15.3:0/1377306293 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f897806a770 0x7f897811b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:14.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.300+0000 7f897d08b700 1 --2- 172.21.15.3:0/1377306293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f897809dc30 0x7f897813a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:14.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.300+0000 7f897e08d700 1 -- 172.21.15.3:0/1377306293 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f897806a770 msgr2=0x7f897811b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:14.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.300+0000 7f897e08d700 1 --2- 172.21.15.3:0/1377306293 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f897806a770 0x7f897811b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.300+0000 7f897e08d700 1 -- 172.21.15.3:0/1377306293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f897809dc30 msgr2=0x7f897813a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:14.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.300+0000 7f897e08d700 1 --2- 172.21.15.3:0/1377306293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f897809dc30 0x7f897813a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.300+0000 7f897e08d700 1 -- 172.21.15.3:0/1377306293 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8968009550 con 0x7f897813c1e0 2023-10-03T10:13:14.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.300+0000 7f897e08d700 1 --2- 172.21.15.3:0/1377306293 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f897813c1e0 0x7f897813ac00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f897000cc50 tx=0x7f897000cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:14.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.300+0000 7f896e7fc700 1 -- 172.21.15.3:0/1377306293 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8970003c00 con 0x7f897813c1e0 2023-10-03T10:13:14.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.300+0000 7f896e7fc700 1 -- 172.21.15.3:0/1377306293 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8970003d60 con 0x7f897813c1e0 2023-10-03T10:13:14.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.300+0000 7f897faf0700 1 -- 172.21.15.3:0/1377306293 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f897811e300 con 0x7f897813c1e0 2023-10-03T10:13:14.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.300+0000 7f896e7fc700 1 -- 172.21.15.3:0/1377306293 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897000ab40 con 0x7f897813c1e0 2023-10-03T10:13:14.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.300+0000 7f897faf0700 1 -- 172.21.15.3:0/1377306293 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897811d3f0 con 0x7f897813c1e0 2023-10-03T10:13:14.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.302+0000 7f897faf0700 1 -- 172.21.15.3:0/1377306293 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f897811df40 con 0x7f897813c1e0 2023-10-03T10:13:14.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.303+0000 7f896e7fc700 1 -- 172.21.15.3:0/1377306293 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f897000aca0 con 0x7f897813c1e0 2023-10-03T10:13:14.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.303+0000 7f896e7fc700 1 -- 172.21.15.3:0/1377306293 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f89700401b0 con 0x7f897813c1e0 2023-10-03T10:13:14.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.311+0000 7f896e7fc700 1 -- 172.21.15.3:0/1377306293 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8970011970 con 0x7f897813c1e0 2023-10-03T10:13:14.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.337+0000 7f7ac60b0700 1 Processor -- start 2023-10-03T10:13:14.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.337+0000 7f7ac60b0700 1 -- start start 2023-10-03T10:13:14.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.338+0000 7f7ac60b0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7ac009d6f0 0x7f7ac009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:14.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.338+0000 7f7ac60b0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ac009e000 0x7f7ac01339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:14.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.338+0000 7f7ac60b0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ac009e410 0x7f7ac0133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:14.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.338+0000 7f7ac60b0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ac006b460 con 0x7f7ac009d6f0 2023-10-03T10:13:14.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.338+0000 7f7ac60b0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac009b120 con 0x7f7ac009e000 2023-10-03T10:13:14.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.338+0000 7f7ac60b0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac009a7e0 con 0x7f7ac009e410 2023-10-03T10:13:14.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.338+0000 7f7abffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ac009e410 0x7f7ac0133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:14.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.338+0000 7f7abffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ac009e410 0x7f7ac0133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55380/0 (socket says 172.21.15.3:55380) 2023-10-03T10:13:14.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.338+0000 7f7abffff700 1 -- 172.21.15.3:0/2835289400 learned_addr learned my addr 172.21.15.3:0/2835289400 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:14.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.338+0000 7f7abeffd700 1 --2- 172.21.15.3:0/2835289400 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ac009e000 0x7f7ac01339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:14.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.338+0000 7f7abf7fe700 1 --2- 172.21.15.3:0/2835289400 >> v2:172.21.15.3:3300/0 conn(0x7f7ac009d6f0 0x7f7ac009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:14.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.338+0000 7f7abf7fe700 1 -- 172.21.15.3:0/2835289400 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ac009e410 msgr2=0x7f7ac0133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:14.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.338+0000 7f7abf7fe700 1 --2- 172.21.15.3:0/2835289400 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ac009e410 0x7f7ac0133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.338+0000 7f7abf7fe700 1 -- 172.21.15.3:0/2835289400 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ac009e000 msgr2=0x7f7ac01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:14.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.338+0000 7f7abf7fe700 1 --2- 172.21.15.3:0/2835289400 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ac009e000 0x7f7ac01339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.338+0000 7f7abf7fe700 1 -- 172.21.15.3:0/2835289400 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac009a500 con 0x7f7ac009d6f0 2023-10-03T10:13:14.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.338+0000 7f7abffff700 1 --2- 172.21.15.3:0/2835289400 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ac009e410 0x7f7ac0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:14.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.338+0000 7f7abeffd700 1 --2- 172.21.15.3:0/2835289400 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ac009e000 0x7f7ac01339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:14.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.339+0000 7f7abf7fe700 1 --2- 172.21.15.3:0/2835289400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ac009d6f0 0x7f7ac009dac0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0009840 tx=0x7f7ab0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71fd43218cba512 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:14.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.339+0000 7f7abe7fc700 1 -- 172.21.15.3:0/2835289400 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0011830 con 0x7f7ac009d6f0 2023-10-03T10:13:14.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.339+0000 7f7abe7fc700 1 -- 172.21.15.3:0/2835289400 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab0011990 con 0x7f7ac009d6f0 2023-10-03T10:13:14.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.339+0000 7f7abe7fc700 1 -- 172.21.15.3:0/2835289400 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0011d20 con 0x7f7ac009d6f0 2023-10-03T10:13:14.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.339+0000 7f7ac60b0700 1 -- 172.21.15.3:0/2835289400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ac009d6f0 msgr2=0x7f7ac009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:14.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.339+0000 7f7ac60b0700 1 --2- 172.21.15.3:0/2835289400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ac009d6f0 0x7f7ac009dac0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0009840 tx=0x7f7ab0009b50 comp rx=0 tx=0).stop 2023-10-03T10:13:14.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.340+0000 7f7ac60b0700 1 -- 172.21.15.3:0/2835289400 shutdown_connections 2023-10-03T10:13:14.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.340+0000 7f7ac60b0700 1 --2- 172.21.15.3:0/2835289400 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ac009e410 0x7f7ac0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.340+0000 7f7ac60b0700 1 --2- 172.21.15.3:0/2835289400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ac009d6f0 0x7f7ac009dac0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.340+0000 7f7ac60b0700 1 --2- 172.21.15.3:0/2835289400 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ac009e000 0x7f7ac01339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.340+0000 7f7ac60b0700 1 -- 172.21.15.3:0/2835289400 >> 172.21.15.3:0/2835289400 conn(0x7f7ac0066f50 msgr2=0x7f7ac009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:14.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.340+0000 7f7ac60b0700 1 -- 172.21.15.3:0/2835289400 shutdown_connections 2023-10-03T10:13:14.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.340+0000 7f7ac60b0700 1 -- 172.21.15.3:0/2835289400 wait complete. 2023-10-03T10:13:14.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.341+0000 7f7ac60b0700 1 Processor -- start 2023-10-03T10:13:14.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.341+0000 7f7ac60b0700 1 -- start start 2023-10-03T10:13:14.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.342+0000 7f7ac60b0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ac009d6f0 0x7f7ac01c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:14.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.342+0000 7f7ac60b0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ac009e000 0x7f7ac01c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:14.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.342+0000 7f7ac60b0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ac009e410 0x7f7ac01c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:14.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.342+0000 7f7ac60b0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac01a6660 con 0x7f7ac009e000 2023-10-03T10:13:14.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.342+0000 7f7ac60b0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac01a6430 con 0x7f7ac009d6f0 2023-10-03T10:13:14.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.342+0000 7f7ac60b0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac01a70a0 con 0x7f7ac009e410 2023-10-03T10:13:14.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.342+0000 7f7abf7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ac009d6f0 0x7f7ac01c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:14.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.342+0000 7f7abf7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ac009d6f0 0x7f7ac01c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36554/0 (socket says 172.21.15.3:36554) 2023-10-03T10:13:14.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.342+0000 7f7abf7fe700 1 -- 172.21.15.3:0/149315644 learned_addr learned my addr 172.21.15.3:0/149315644 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:14.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.342+0000 7f7abffff700 1 --2- 172.21.15.3:0/149315644 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ac009e410 0x7f7ac01c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:14.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.342+0000 7f7abeffd700 1 --2- 172.21.15.3:0/149315644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ac009e000 0x7f7ac01c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:14.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.343+0000 7f7abf7fe700 1 -- 172.21.15.3:0/149315644 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ac009e410 msgr2=0x7f7ac01c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:14.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.343+0000 7f7abf7fe700 1 --2- 172.21.15.3:0/149315644 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ac009e410 0x7f7ac01c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.343+0000 7f7abf7fe700 1 -- 172.21.15.3:0/149315644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ac009e000 msgr2=0x7f7ac01c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:14.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.343+0000 7f7abf7fe700 1 --2- 172.21.15.3:0/149315644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ac009e000 0x7f7ac01c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.343+0000 7f7abf7fe700 1 -- 172.21.15.3:0/149315644 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab40094a0 con 0x7f7ac009d6f0 2023-10-03T10:13:14.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.343+0000 7f7abeffd700 1 --2- 172.21.15.3:0/149315644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ac009e000 0x7f7ac01c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:14.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.343+0000 7f7abf7fe700 1 --2- 172.21.15.3:0/149315644 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ac009d6f0 0x7f7ac01c8090 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0009fd0 tx=0x7f7ab0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:14.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.343+0000 7f7aa7fff700 1 -- 172.21.15.3:0/149315644 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0004510 con 0x7f7ac009d6f0 2023-10-03T10:13:14.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.343+0000 7f7aa7fff700 1 -- 172.21.15.3:0/149315644 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab001f070 con 0x7f7ac009d6f0 2023-10-03T10:13:14.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.343+0000 7f7ac60b0700 1 -- 172.21.15.3:0/149315644 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab8009b30 con 0x7f7ac009d6f0 2023-10-03T10:13:14.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.343+0000 7f7aa7fff700 1 -- 172.21.15.3:0/149315644 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab00038e0 con 0x7f7ac009d6f0 2023-10-03T10:13:14.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.343+0000 7f7ac60b0700 1 -- 172.21.15.3:0/149315644 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab00094a0 con 0x7f7ac009d6f0 2023-10-03T10:13:14.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.345+0000 7f7ac60b0700 1 -- 172.21.15.3:0/149315644 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ac009c770 con 0x7f7ac009d6f0 2023-10-03T10:13:14.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.345+0000 7f7aa7fff700 1 -- 172.21.15.3:0/149315644 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7ab0003a40 con 0x7f7ac009d6f0 2023-10-03T10:13:14.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.345+0000 7f7aa7fff700 1 -- 172.21.15.3:0/149315644 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7ab0021070 con 0x7f7ac009d6f0 2023-10-03T10:13:14.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.350+0000 7f7aa7fff700 1 -- 172.21.15.3:0/149315644 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7ab00253e0 con 0x7f7ac009d6f0 2023-10-03T10:13:14.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.356+0000 7fdc6eb11700 1 Processor -- start 2023-10-03T10:13:14.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.357+0000 7fdc6eb11700 1 -- start start 2023-10-03T10:13:14.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.357+0000 7fdc6eb11700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc6809e1d0 0x7fdc6813cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:14.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.357+0000 7fdc6eb11700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdc6813d480 0x7fdc6813f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:14.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.357+0000 7fdc6eb11700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc680992c0 con 0x7fdc6809d6f0 2023-10-03T10:13:14.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.357+0000 7fdc6eb11700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc68099520 con 0x7fdc6813d480 2023-10-03T10:13:14.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.357+0000 7fdc6eb11700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc680997e0 con 0x7fdc6809e1d0 2023-10-03T10:13:14.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.357+0000 7fdc67fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc6809e1d0 0x7fdc6813cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:14.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.357+0000 7fdc67fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc6809e1d0 0x7fdc6813cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36556/0 (socket says 172.21.15.3:36556) 2023-10-03T10:13:14.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.357+0000 7fdc67fff700 1 -- 172.21.15.3:0/3640251281 learned_addr learned my addr 172.21.15.3:0/3640251281 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:14.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.357+0000 7fdc6d0ae700 1 --2- 172.21.15.3:0/3640251281 >> v2:172.21.15.3:3300/0 conn(0x7fdc6813d480 0x7fdc6813f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:14.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.358+0000 7fdc6d0ae700 1 -- 172.21.15.3:0/3640251281 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc6809e1d0 msgr2=0x7fdc6813cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:14.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.358+0000 7fdc6d0ae700 1 --2- 172.21.15.3:0/3640251281 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc6809e1d0 0x7fdc6813cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.358+0000 7fdc6d0ae700 1 -- 172.21.15.3:0/3640251281 >> v1:172.21.15.3:6789/0 conn(0x7fdc6809d6f0 legacy=0x7fdc6809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:14.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.358+0000 7fdc6d0ae700 1 -- 172.21.15.3:0/3640251281 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc68099b30 con 0x7fdc6813d480 2023-10-03T10:13:14.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.358+0000 7fdc67fff700 1 --2- 172.21.15.3:0/3640251281 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc6809e1d0 0x7fdc6813cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:14.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.358+0000 7fdc6d0ae700 1 --2- 172.21.15.3:0/3640251281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc6813d480 0x7fdc6813f860 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdc60009d90 tx=0x7fdc6000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158ab095b274251c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:14.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.358+0000 7fdc677fe700 1 -- 172.21.15.3:0/3640251281 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6000e070 con 0x7fdc6813d480 2023-10-03T10:13:14.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.358+0000 7fdc677fe700 1 -- 172.21.15.3:0/3640251281 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6000bb20 con 0x7fdc6813d480 2023-10-03T10:13:14.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.358+0000 7fdc677fe700 1 -- 172.21.15.3:0/3640251281 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc60012430 con 0x7fdc6813d480 2023-10-03T10:13:14.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.359+0000 7fdc6eb11700 1 -- 172.21.15.3:0/3640251281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc6813d480 msgr2=0x7fdc6813f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:14.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.359+0000 7fdc6eb11700 1 --2- 172.21.15.3:0/3640251281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc6813d480 0x7fdc6813f860 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdc60009d90 tx=0x7fdc6000b330 comp rx=0 tx=0).stop 2023-10-03T10:13:14.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.359+0000 7fdc6eb11700 1 -- 172.21.15.3:0/3640251281 shutdown_connections 2023-10-03T10:13:14.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.359+0000 7fdc6eb11700 1 --2- 172.21.15.3:0/3640251281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc6813d480 0x7fdc6813f860 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.359+0000 7fdc6eb11700 1 --2- 172.21.15.3:0/3640251281 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc6809e1d0 0x7fdc6813cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.359+0000 7fdc6eb11700 1 -- 172.21.15.3:0/3640251281 >> 172.21.15.3:0/3640251281 conn(0x7fdc6809f470 msgr2=0x7fdc680a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:14.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.359+0000 7fdc6eb11700 1 -- 172.21.15.3:0/3640251281 shutdown_connections 2023-10-03T10:13:14.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.359+0000 7fdc6eb11700 1 -- 172.21.15.3:0/3640251281 wait complete. 2023-10-03T10:13:14.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.360+0000 7fdc6eb11700 1 Processor -- start 2023-10-03T10:13:14.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.361+0000 7fdc6eb11700 1 -- start start 2023-10-03T10:13:14.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.361+0000 7fdc6eb11700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc6809d6f0 0x7fdc68070780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:14.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.361+0000 7fdc6eb11700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc6809e1d0 0x7fdc68070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:14.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.361+0000 7fdc6eb11700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc6813d480 0x7fdc68071200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:14.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.361+0000 7fdc6eb11700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc68127bc0 con 0x7fdc6809d6f0 2023-10-03T10:13:14.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.361+0000 7fdc6eb11700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6813d1a0 con 0x7fdc6813d480 2023-10-03T10:13:14.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.361+0000 7fdc6eb11700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc681285a0 con 0x7fdc6809e1d0 2023-10-03T10:13:14.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.362+0000 7fdc6c8ad700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc6809d6f0 0x7fdc68070780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:14.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.362+0000 7fdc67fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc6809e1d0 0x7fdc68070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:14.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.362+0000 7fdc6c8ad700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc6809d6f0 0x7fdc68070780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50476/0 (socket says 172.21.15.3:50476) 2023-10-03T10:13:14.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.362+0000 7fdc6c8ad700 1 -- 172.21.15.3:0/1635631324 learned_addr learned my addr 172.21.15.3:0/1635631324 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:14.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.362+0000 7fdc6d0ae700 1 --2- 172.21.15.3:0/1635631324 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc6813d480 0x7fdc68071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:14.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.362+0000 7fdc6c8ad700 1 -- 172.21.15.3:0/1635631324 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc6809e1d0 msgr2=0x7fdc68070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:14.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.362+0000 7fdc6c8ad700 1 --2- 172.21.15.3:0/1635631324 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc6809e1d0 0x7fdc68070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.362+0000 7fdc6c8ad700 1 -- 172.21.15.3:0/1635631324 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc6813d480 msgr2=0x7fdc68071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:14.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.362+0000 7fdc6c8ad700 1 --2- 172.21.15.3:0/1635631324 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc6813d480 0x7fdc68071200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.362+0000 7fdc6c8ad700 1 -- 172.21.15.3:0/1635631324 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc54009550 con 0x7fdc6809d6f0 2023-10-03T10:13:14.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.362+0000 7fdc67fff700 1 --2- 172.21.15.3:0/1635631324 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc6809e1d0 0x7fdc68070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:14.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.362+0000 7fdc6d0ae700 1 --2- 172.21.15.3:0/1635631324 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc6813d480 0x7fdc68071200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:14.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.362+0000 7fdc6c8ad700 1 --2- 172.21.15.3:0/1635631324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc6809d6f0 0x7fdc68070780 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c00c510 tx=0x7fdc5c00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:14.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.362+0000 7fdc657fa700 1 -- 172.21.15.3:0/1635631324 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c0052d0 con 0x7fdc6809d6f0 2023-10-03T10:13:14.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.363+0000 7fdc657fa700 1 -- 172.21.15.3:0/1635631324 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5c005430 con 0x7fdc6809d6f0 2023-10-03T10:13:14.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.363+0000 7fdc6eb11700 1 -- 172.21.15.3:0/1635631324 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc60009a70 con 0x7fdc6809d6f0 2023-10-03T10:13:14.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.363+0000 7fdc657fa700 1 -- 172.21.15.3:0/1635631324 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c0038c0 con 0x7fdc6809d6f0 2023-10-03T10:13:14.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.363+0000 7fdc6eb11700 1 -- 172.21.15.3:0/1635631324 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc68128060 con 0x7fdc6809d6f0 2023-10-03T10:13:14.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.364+0000 7fdc6eb11700 1 -- 172.21.15.3:0/1635631324 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc680747f0 con 0x7fdc6809d6f0 2023-10-03T10:13:14.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.364+0000 7fdc657fa700 1 -- 172.21.15.3:0/1635631324 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdc5c003a20 con 0x7fdc6809d6f0 2023-10-03T10:13:14.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.364+0000 7fdc657fa700 1 -- 172.21.15.3:0/1635631324 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fdc5c03eb30 con 0x7fdc6809d6f0 2023-10-03T10:13:14.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.369+0000 7fdc657fa700 1 -- 172.21.15.3:0/1635631324 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdc5c019e60 con 0x7fdc6809d6f0 2023-10-03T10:13:14.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.520+0000 7f471677f700 1 -- 172.21.15.3:0/2603247839 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f471006c360 con 0x7f471009e1d0 2023-10-03T10:13:14.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.520+0000 7f470cff9700 1 -- 172.21.15.3:0/2603247839 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4704044060 con 0x7f471009e1d0 2023-10-03T10:13:14.521 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:14.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.523+0000 7f471677f700 1 -- 172.21.15.3:0/2603247839 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f471009e1d0 msgr2=0x7f47101c8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:14.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.523+0000 7f471677f700 1 --2- 172.21.15.3:0/2603247839 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f471009e1d0 0x7f47101c8a50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f470400e9b0 tx=0x7f470400ecc0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.523+0000 7f471677f700 1 -- 172.21.15.3:0/2603247839 shutdown_connections 2023-10-03T10:13:14.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.523+0000 7f471677f700 1 --2- 172.21.15.3:0/2603247839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4710139170 0x7f471011e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.523+0000 7f471677f700 1 --2- 172.21.15.3:0/2603247839 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f471009d6f0 0x7f47101c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.523+0000 7f471677f700 1 --2- 172.21.15.3:0/2603247839 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f471009e1d0 0x7f47101c8a50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.523+0000 7f471677f700 1 -- 172.21.15.3:0/2603247839 >> 172.21.15.3:0/2603247839 conn(0x7f4710066f50 msgr2=0x7f47100a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:14.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.523+0000 7f471677f700 1 -- 172.21.15.3:0/2603247839 shutdown_connections 2023-10-03T10:13:14.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.523+0000 7f471677f700 1 -- 172.21.15.3:0/2603247839 wait complete. 2023-10-03T10:13:14.537 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:14.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.587+0000 7f7ac60b0700 1 -- 172.21.15.3:0/149315644 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7ac0074ea0 con 0x7f7ac009d6f0 2023-10-03T10:13:14.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.587+0000 7f7aa7fff700 1 -- 172.21.15.3:0/149315644 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7ab0010b30 con 0x7f7ac009d6f0 2023-10-03T10:13:14.589 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:14.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.590+0000 7f7ac60b0700 1 -- 172.21.15.3:0/149315644 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ac009d6f0 msgr2=0x7f7ac01c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:14.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.590+0000 7f7ac60b0700 1 --2- 172.21.15.3:0/149315644 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ac009d6f0 0x7f7ac01c8090 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0009fd0 tx=0x7f7ab0004750 comp rx=0 tx=0).stop 2023-10-03T10:13:14.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.590+0000 7f7ac60b0700 1 -- 172.21.15.3:0/149315644 shutdown_connections 2023-10-03T10:13:14.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.590+0000 7f7ac60b0700 1 --2- 172.21.15.3:0/149315644 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ac009e410 0x7f7ac01c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.590+0000 7f7ac60b0700 1 --2- 172.21.15.3:0/149315644 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ac009d6f0 0x7f7ac01c8090 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.590+0000 7f7ac60b0700 1 --2- 172.21.15.3:0/149315644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ac009e000 0x7f7ac01c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.590+0000 7f7ac60b0700 1 -- 172.21.15.3:0/149315644 >> 172.21.15.3:0/149315644 conn(0x7f7ac0066f50 msgr2=0x7f7ac006f350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:14.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.590+0000 7f7ac60b0700 1 -- 172.21.15.3:0/149315644 shutdown_connections 2023-10-03T10:13:14.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.590+0000 7f7ac60b0700 1 -- 172.21.15.3:0/149315644 wait complete. 2023-10-03T10:13:14.604 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:14.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.654+0000 7f897faf0700 1 -- 172.21.15.3:0/1377306293 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f897811df40 con 0x7f897813c1e0 2023-10-03T10:13:14.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.655+0000 7f896e7fc700 1 -- 172.21.15.3:0/1377306293 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f897811df40 con 0x7f897813c1e0 2023-10-03T10:13:14.655 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:14.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.657+0000 7f897faf0700 1 -- 172.21.15.3:0/1377306293 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f897813c1e0 msgr2=0x7f897813ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:14.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.657+0000 7f897faf0700 1 --2- 172.21.15.3:0/1377306293 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f897813c1e0 0x7f897813ac00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f897000cc50 tx=0x7f897000cf60 comp rx=0 tx=0).stop 2023-10-03T10:13:14.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.657+0000 7f897faf0700 1 -- 172.21.15.3:0/1377306293 shutdown_connections 2023-10-03T10:13:14.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.657+0000 7f897faf0700 1 --2- 172.21.15.3:0/1377306293 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f897813c1e0 0x7f897813ac00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.657+0000 7f897faf0700 1 --2- 172.21.15.3:0/1377306293 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f897806a770 0x7f897811b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.657+0000 7f897faf0700 1 --2- 172.21.15.3:0/1377306293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f897809dc30 0x7f897813a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.657+0000 7f897faf0700 1 -- 172.21.15.3:0/1377306293 >> 172.21.15.3:0/1377306293 conn(0x7f8978066f50 msgr2=0x7f89780a15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:14.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.657+0000 7f897faf0700 1 -- 172.21.15.3:0/1377306293 shutdown_connections 2023-10-03T10:13:14.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.657+0000 7f897faf0700 1 -- 172.21.15.3:0/1377306293 wait complete. 2023-10-03T10:13:14.671 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:14.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.700+0000 7fdc6eb11700 1 -- 172.21.15.3:0/1635631324 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdc68097de0 con 0x7fdc6809d6f0 2023-10-03T10:13:14.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.700+0000 7fdc657fa700 1 -- 172.21.15.3:0/1635631324 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdc5c017030 con 0x7fdc6809d6f0 2023-10-03T10:13:14.700 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:14.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.702+0000 7fdc6eb11700 1 -- 172.21.15.3:0/1635631324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc6809d6f0 msgr2=0x7fdc68070780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:14.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.702+0000 7fdc6eb11700 1 --2- 172.21.15.3:0/1635631324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc6809d6f0 0x7fdc68070780 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c00c510 tx=0x7fdc5c00c820 comp rx=0 tx=0).stop 2023-10-03T10:13:14.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.702+0000 7fdc6eb11700 1 -- 172.21.15.3:0/1635631324 shutdown_connections 2023-10-03T10:13:14.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.702+0000 7fdc6eb11700 1 --2- 172.21.15.3:0/1635631324 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc6813d480 0x7fdc68071200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.702+0000 7fdc6eb11700 1 --2- 172.21.15.3:0/1635631324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc6809d6f0 0x7fdc68070780 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.702+0000 7fdc6eb11700 1 --2- 172.21.15.3:0/1635631324 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc6809e1d0 0x7fdc68070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:14.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.703+0000 7fdc6eb11700 1 -- 172.21.15.3:0/1635631324 >> 172.21.15.3:0/1635631324 conn(0x7fdc6809f470 msgr2=0x7fdc6813d850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:14.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.703+0000 7fdc6eb11700 1 -- 172.21.15.3:0/1635631324 shutdown_connections 2023-10-03T10:13:14.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:14.703+0000 7fdc6eb11700 1 -- 172.21.15.3:0/1635631324 wait complete. 2023-10-03T10:13:14.716 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:15.538 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:15.605 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:15.672 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:15.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.687+0000 7f9190ed3700 1 Processor -- start 2023-10-03T10:13:15.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.687+0000 7f9190ed3700 1 -- start start 2023-10-03T10:13:15.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.688+0000 7f9190ed3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f918c09e930 0x7f918c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:15.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.688+0000 7f9190ed3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f918c09f240 0x7f918c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:15.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.688+0000 7f9190ed3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f918c09f650 0x7f918c131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:15.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.688+0000 7f9190ed3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f918c070940 con 0x7f918c09f650 2023-10-03T10:13:15.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.688+0000 7f9190ed3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f918c06fd60 con 0x7f918c09f240 2023-10-03T10:13:15.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.688+0000 7f9190ed3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f918c06fa80 con 0x7f918c09e930 2023-10-03T10:13:15.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.688+0000 7f918a59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f918c09e930 0x7f918c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:15.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.688+0000 7f918a59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f918c09e930 0x7f918c09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55396/0 (socket says 172.21.15.3:55396) 2023-10-03T10:13:15.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.688+0000 7f918a59c700 1 -- 172.21.15.3:0/2850694867 learned_addr learned my addr 172.21.15.3:0/2850694867 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:15.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.688+0000 7f918ad9d700 1 --2- 172.21.15.3:0/2850694867 >> v2:172.21.15.3:3300/0 conn(0x7f918c09f650 0x7f918c131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:15.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.688+0000 7f9189d9b700 1 --2- 172.21.15.3:0/2850694867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f918c09f240 0x7f918c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:15.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.688+0000 7f918a59c700 1 -- 172.21.15.3:0/2850694867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f918c09f240 msgr2=0x7f918c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:15.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.688+0000 7f918a59c700 1 --2- 172.21.15.3:0/2850694867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f918c09f240 0x7f918c1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:15.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.688+0000 7f918a59c700 1 -- 172.21.15.3:0/2850694867 >> v2:172.21.15.3:3300/0 conn(0x7f918c09f650 msgr2=0x7f918c131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:15.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.688+0000 7f918a59c700 1 --2- 172.21.15.3:0/2850694867 >> v2:172.21.15.3:3300/0 conn(0x7f918c09f650 0x7f918c131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:15.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.688+0000 7f918a59c700 1 -- 172.21.15.3:0/2850694867 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918c09fb60 con 0x7f918c09e930 2023-10-03T10:13:15.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.688+0000 7f918a59c700 1 --2- 172.21.15.3:0/2850694867 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f918c09e930 0x7f918c09ed00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f917c0098f0 tx=0x7f917c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be7e8f64d3f8f2c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:15.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.689+0000 7f918959a700 1 -- 172.21.15.3:0/2850694867 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c011830 con 0x7f918c09e930 2023-10-03T10:13:15.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.689+0000 7f918959a700 1 -- 172.21.15.3:0/2850694867 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917c011990 con 0x7f918c09e930 2023-10-03T10:13:15.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.689+0000 7f918959a700 1 -- 172.21.15.3:0/2850694867 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c011d20 con 0x7f918c09e930 2023-10-03T10:13:15.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.689+0000 7f9190ed3700 1 -- 172.21.15.3:0/2850694867 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f918c09e930 msgr2=0x7f918c09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:15.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.689+0000 7f9190ed3700 1 --2- 172.21.15.3:0/2850694867 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f918c09e930 0x7f918c09ed00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f917c0098f0 tx=0x7f917c009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:15.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.690+0000 7f9190ed3700 1 -- 172.21.15.3:0/2850694867 shutdown_connections 2023-10-03T10:13:15.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.690+0000 7f9190ed3700 1 --2- 172.21.15.3:0/2850694867 >> v2:172.21.15.3:3300/0 conn(0x7f918c09f650 0x7f918c131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:15.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.690+0000 7f9190ed3700 1 --2- 172.21.15.3:0/2850694867 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f918c09e930 0x7f918c09ed00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:15.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.690+0000 7f9190ed3700 1 --2- 172.21.15.3:0/2850694867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f918c09f240 0x7f918c1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:15.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.690+0000 7f9190ed3700 1 -- 172.21.15.3:0/2850694867 >> 172.21.15.3:0/2850694867 conn(0x7f918c0a4210 msgr2=0x7f918c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:15.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.690+0000 7f9190ed3700 1 -- 172.21.15.3:0/2850694867 shutdown_connections 2023-10-03T10:13:15.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.690+0000 7f9190ed3700 1 -- 172.21.15.3:0/2850694867 wait complete. 2023-10-03T10:13:15.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.691+0000 7f9190ed3700 1 Processor -- start 2023-10-03T10:13:15.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.691+0000 7f9190ed3700 1 -- start start 2023-10-03T10:13:15.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.692+0000 7f9190ed3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f918c09e930 0x7f918c1a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:15.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.692+0000 7f9190ed3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f918c09f240 0x7f918c1a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:15.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.692+0000 7f9190ed3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f918c09f650 0x7f918c1a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:15.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.692+0000 7f9190ed3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918c1a5220 con 0x7f918c09e930 2023-10-03T10:13:15.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.692+0000 7f9190ed3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f918c1a5980 con 0x7f918c09f240 2023-10-03T10:13:15.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.692+0000 7f9190ed3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f918c1a5fa0 con 0x7f918c09f650 2023-10-03T10:13:15.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.692+0000 7f9189d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f918c09f240 0x7f918c1a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:15.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.692+0000 7f9189d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f918c09f240 0x7f918c1a42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36580/0 (socket says 172.21.15.3:36580) 2023-10-03T10:13:15.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.692+0000 7f9189d9b700 1 -- 172.21.15.3:0/2464460512 learned_addr learned my addr 172.21.15.3:0/2464460512 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:15.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.692+0000 7f918a59c700 1 --2- 172.21.15.3:0/2464460512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f918c09e930 0x7f918c1a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:15.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.692+0000 7f918ad9d700 1 --2- 172.21.15.3:0/2464460512 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f918c09f650 0x7f918c1a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:15.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.692+0000 7f9189d9b700 1 -- 172.21.15.3:0/2464460512 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f918c09f650 msgr2=0x7f918c1a47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:15.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.692+0000 7f9189d9b700 1 --2- 172.21.15.3:0/2464460512 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f918c09f650 0x7f918c1a47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:15.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.692+0000 7f9189d9b700 1 -- 172.21.15.3:0/2464460512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f918c09e930 msgr2=0x7f918c1a3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:15.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.692+0000 7f9189d9b700 1 --2- 172.21.15.3:0/2464460512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f918c09e930 0x7f918c1a3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:15.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.692+0000 7f9189d9b700 1 -- 172.21.15.3:0/2464460512 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917c009550 con 0x7f918c09f240 2023-10-03T10:13:15.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.692+0000 7f9189d9b700 1 --2- 172.21.15.3:0/2464460512 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f918c09f240 0x7f918c1a42b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f918000e810 tx=0x7f918000eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:15.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.693+0000 7f917affd700 1 -- 172.21.15.3:0/2464460512 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180004230 con 0x7f918c09f240 2023-10-03T10:13:15.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.693+0000 7f9190ed3700 1 -- 172.21.15.3:0/2464460512 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f918c0a0190 con 0x7f918c09f240 2023-10-03T10:13:15.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.693+0000 7f917affd700 1 -- 172.21.15.3:0/2464460512 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9180004390 con 0x7f918c09f240 2023-10-03T10:13:15.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.693+0000 7f917affd700 1 -- 172.21.15.3:0/2464460512 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91800036a0 con 0x7f918c09f240 2023-10-03T10:13:15.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.693+0000 7f9190ed3700 1 -- 172.21.15.3:0/2464460512 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f918c1a7610 con 0x7f918c09f240 2023-10-03T10:13:15.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.694+0000 7f917affd700 1 -- 172.21.15.3:0/2464460512 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9180003800 con 0x7f918c09f240 2023-10-03T10:13:15.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.694+0000 7f9190ed3700 1 -- 172.21.15.3:0/2464460512 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f918c071d20 con 0x7f918c09f240 2023-10-03T10:13:15.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.694+0000 7f917affd700 1 -- 172.21.15.3:0/2464460512 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f918003f000 con 0x7f918c09f240 2023-10-03T10:13:15.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.702+0000 7f917affd700 1 -- 172.21.15.3:0/2464460512 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f918001b720 con 0x7f918c09f240 2023-10-03T10:13:15.716 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:15.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.774+0000 7f4a3dbfc700 1 Processor -- start 2023-10-03T10:13:15.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.774+0000 7f4a3dbfc700 1 -- start start 2023-10-03T10:13:15.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.775+0000 7f4a3dbfc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a3809d6f0 0x7f4a3809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:15.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.775+0000 7f4a3dbfc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a3809e000 0x7f4a381339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:15.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.775+0000 7f4a3dbfc700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a3809b120 con 0x7f4a3809e410 2023-10-03T10:13:15.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.775+0000 7f4a3dbfc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3806b460 con 0x7f4a3809d6f0 2023-10-03T10:13:15.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.775+0000 7f4a3dbfc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3809a7e0 con 0x7f4a3809e000 2023-10-03T10:13:15.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.775+0000 7f4a36ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a3809e000 0x7f4a381339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:15.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.775+0000 7f4a37fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4a3809e410 0x7f4a38133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59294/0 (socket says 172.21.15.3:59294) 2023-10-03T10:13:15.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.776+0000 7f4a37fff700 1 -- 172.21.15.3:0/894686706 learned_addr learned my addr 172.21.15.3:0/894686706 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:15.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.776+0000 7f4a377fe700 1 --2- 172.21.15.3:0/894686706 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a3809d6f0 0x7f4a3809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:15.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.776+0000 7f4a377fe700 1 -- 172.21.15.3:0/894686706 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a3809e000 msgr2=0x7f4a381339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:15.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.776+0000 7f4a367fc700 1 -- 172.21.15.3:0/894686706 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4220200852 0 0) 0x7f4a3809b120 con 0x7f4a3809e410 2023-10-03T10:13:15.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.776+0000 7f4a377fe700 1 --2- 172.21.15.3:0/894686706 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a3809e000 0x7f4a381339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:15.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.776+0000 7f4a377fe700 1 -- 172.21.15.3:0/894686706 >> v1:172.21.15.3:6789/0 conn(0x7f4a3809e410 legacy=0x7f4a38133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:15.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.776+0000 7f4a377fe700 1 -- 172.21.15.3:0/894686706 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a3809a500 con 0x7f4a3809d6f0 2023-10-03T10:13:15.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.776+0000 7f4a377fe700 1 --2- 172.21.15.3:0/894686706 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a3809d6f0 0x7f4a3809dac0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4a28009870 tx=0x7f4a28009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13928e3927249d34 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:15.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.776+0000 7f4a367fc700 1 -- 172.21.15.3:0/894686706 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a28011830 con 0x7f4a3809d6f0 2023-10-03T10:13:15.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.776+0000 7f4a367fc700 1 -- 172.21.15.3:0/894686706 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a28011990 con 0x7f4a3809d6f0 2023-10-03T10:13:15.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.776+0000 7f4a367fc700 1 -- 172.21.15.3:0/894686706 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a28011d20 con 0x7f4a3809d6f0 2023-10-03T10:13:15.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.777+0000 7f4a3dbfc700 1 -- 172.21.15.3:0/894686706 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a3809d6f0 msgr2=0x7f4a3809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:15.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.777+0000 7f4a3dbfc700 1 --2- 172.21.15.3:0/894686706 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a3809d6f0 0x7f4a3809dac0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4a28009870 tx=0x7f4a28009b80 comp rx=0 tx=0).stop 2023-10-03T10:13:15.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.777+0000 7f4a3dbfc700 1 -- 172.21.15.3:0/894686706 shutdown_connections 2023-10-03T10:13:15.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.777+0000 7f4a3dbfc700 1 --2- 172.21.15.3:0/894686706 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a3809d6f0 0x7f4a3809dac0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:15.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.777+0000 7f4a3dbfc700 1 --2- 172.21.15.3:0/894686706 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a3809e000 0x7f4a381339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:15.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.777+0000 7f4a3dbfc700 1 -- 172.21.15.3:0/894686706 >> 172.21.15.3:0/894686706 conn(0x7f4a38066f50 msgr2=0x7f4a3809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:15.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.777+0000 7f4a3dbfc700 1 -- 172.21.15.3:0/894686706 shutdown_connections 2023-10-03T10:13:15.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.777+0000 7f4a3dbfc700 1 -- 172.21.15.3:0/894686706 wait complete. 2023-10-03T10:13:15.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.779+0000 7f4a3dbfc700 1 Processor -- start 2023-10-03T10:13:15.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.779+0000 7f4a3dbfc700 1 -- start start 2023-10-03T10:13:15.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.780+0000 7f4a3dbfc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a3809d6f0 0x7f4a381c3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:15.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.780+0000 7f4a3dbfc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a3809e000 0x7f4a381c4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:15.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.780+0000 7f4a3dbfc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a3809e410 0x7f4a381c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:15.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.780+0000 7f4a3dbfc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a381a2060 con 0x7f4a3809e000 2023-10-03T10:13:15.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.780+0000 7f4a3dbfc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a381a38c0 con 0x7f4a3809e410 2023-10-03T10:13:15.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.780+0000 7f4a3dbfc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3809c710 con 0x7f4a3809d6f0 2023-10-03T10:13:15.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.780+0000 7f4a37fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a3809e410 0x7f4a381c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:15.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.780+0000 7f4a36ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a3809e000 0x7f4a381c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:15.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.780+0000 7f4a37fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a3809e410 0x7f4a381c46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36592/0 (socket says 172.21.15.3:36592) 2023-10-03T10:13:15.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.780+0000 7f4a37fff700 1 -- 172.21.15.3:0/628291945 learned_addr learned my addr 172.21.15.3:0/628291945 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:15.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.780+0000 7f4a377fe700 1 --2- 172.21.15.3:0/628291945 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a3809d6f0 0x7f4a381c3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:15.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.781+0000 7f4a377fe700 1 -- 172.21.15.3:0/628291945 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a3809e410 msgr2=0x7f4a381c46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:15.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.781+0000 7f4a377fe700 1 --2- 172.21.15.3:0/628291945 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a3809e410 0x7f4a381c46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:15.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.781+0000 7f4a377fe700 1 -- 172.21.15.3:0/628291945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a3809e000 msgr2=0x7f4a381c4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:15.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.781+0000 7f4a377fe700 1 --2- 172.21.15.3:0/628291945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a3809e000 0x7f4a381c4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:15.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.781+0000 7f4a377fe700 1 -- 172.21.15.3:0/628291945 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a280094a0 con 0x7f4a3809d6f0 2023-10-03T10:13:15.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.781+0000 7f4a36ffd700 1 --2- 172.21.15.3:0/628291945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a3809e000 0x7f4a381c4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:15.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.781+0000 7f4a377fe700 1 --2- 172.21.15.3:0/628291945 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a3809d6f0 0x7f4a381c3c30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4a28009870 tx=0x7f4a28004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:15.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.781+0000 7f4a1ffff700 1 -- 172.21.15.3:0/628291945 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a280045d0 con 0x7f4a3809d6f0 2023-10-03T10:13:15.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.781+0000 7f4a1ffff700 1 -- 172.21.15.3:0/628291945 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a2801f070 con 0x7f4a3809d6f0 2023-10-03T10:13:15.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.781+0000 7f4a3dbfc700 1 -- 172.21.15.3:0/628291945 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a381a45f0 con 0x7f4a3809d6f0 2023-10-03T10:13:15.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.781+0000 7f4a1ffff700 1 -- 172.21.15.3:0/628291945 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a280038a0 con 0x7f4a3809d6f0 2023-10-03T10:13:15.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.781+0000 7f4a3dbfc700 1 -- 172.21.15.3:0/628291945 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a381b4df0 con 0x7f4a3809d6f0 2023-10-03T10:13:15.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.782+0000 7f4a1ffff700 1 -- 172.21.15.3:0/628291945 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4a28003a00 con 0x7f4a3809d6f0 2023-10-03T10:13:15.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.783+0000 7f4a1ffff700 1 -- 172.21.15.3:0/628291945 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4a28021070 con 0x7f4a3809d6f0 2023-10-03T10:13:15.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.783+0000 7f4a3dbfc700 1 -- 172.21.15.3:0/628291945 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a381a3ae0 con 0x7f4a3809d6f0 2023-10-03T10:13:15.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.788+0000 7f4a1ffff700 1 -- 172.21.15.3:0/628291945 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4a28015080 con 0x7f4a3809d6f0 2023-10-03T10:13:15.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.822+0000 7fb5be49d700 1 Processor -- start 2023-10-03T10:13:15.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.822+0000 7fb5be49d700 1 -- start start 2023-10-03T10:13:15.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.823+0000 7fb5be49d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5b80a0a30 0x7fb5b80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:15.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.823+0000 7fb5be49d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb5b80a1340 0x7fb5b8133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:15.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.823+0000 7fb5be49d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5b8133fa0 0x7fb5b8134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:15.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.823+0000 7fb5be49d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5b809edd0 con 0x7fb5b80a1340 2023-10-03T10:13:15.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.823+0000 7fb5be49d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b809e1f0 con 0x7fb5b80a0a30 2023-10-03T10:13:15.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.823+0000 7fb5be49d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b809dc50 con 0x7fb5b8133fa0 2023-10-03T10:13:15.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.823+0000 7fb5b77fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb5b80a1340 0x7fb5b8133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:15.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.823+0000 7fb5b7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5b80a0a30 0x7fb5b80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:15.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.823+0000 7fb5b77fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb5b80a1340 0x7fb5b8133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50514/0 (socket says 172.21.15.3:50514) 2023-10-03T10:13:15.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.823+0000 7fb5b77fe700 1 -- 172.21.15.3:0/3294095648 learned_addr learned my addr 172.21.15.3:0/3294095648 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:15.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.823+0000 7fb5b77fe700 1 -- 172.21.15.3:0/3294095648 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5b8133fa0 msgr2=0x7fb5b8134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:15.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.823+0000 7fb5b77fe700 1 --2- 172.21.15.3:0/3294095648 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5b8133fa0 0x7fb5b8134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:15.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.823+0000 7fb5b77fe700 1 -- 172.21.15.3:0/3294095648 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5b80a0a30 msgr2=0x7fb5b80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:15.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.823+0000 7fb5b77fe700 1 --2- 172.21.15.3:0/3294095648 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5b80a0a30 0x7fb5b80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:15.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.823+0000 7fb5b77fe700 1 -- 172.21.15.3:0/3294095648 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b80a1fe0 con 0x7fb5b80a1340 2023-10-03T10:13:15.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.823+0000 7fb5b7fff700 1 --2- 172.21.15.3:0/3294095648 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5b80a0a30 0x7fb5b80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:15.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.824+0000 7fb5b77fe700 1 --2- 172.21.15.3:0/3294095648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5b80a1340 0x7fb5b8133a60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac0098f0 tx=0x7fb5ac009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a7f9c33f605ac server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:15.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.824+0000 7fb5b6ffd700 1 -- 172.21.15.3:0/3294095648 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac011830 con 0x7fb5b80a1340 2023-10-03T10:13:15.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.824+0000 7fb5b6ffd700 1 -- 172.21.15.3:0/3294095648 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ac011990 con 0x7fb5b80a1340 2023-10-03T10:13:15.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.824+0000 7fb5b6ffd700 1 -- 172.21.15.3:0/3294095648 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac011d20 con 0x7fb5b80a1340 2023-10-03T10:13:15.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.824+0000 7fb5be49d700 1 -- 172.21.15.3:0/3294095648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5b80a1340 msgr2=0x7fb5b8133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:15.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.824+0000 7fb5be49d700 1 --2- 172.21.15.3:0/3294095648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5b80a1340 0x7fb5b8133a60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac0098f0 tx=0x7fb5ac009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:15.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.825+0000 7fb5be49d700 1 -- 172.21.15.3:0/3294095648 shutdown_connections 2023-10-03T10:13:15.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.825+0000 7fb5be49d700 1 --2- 172.21.15.3:0/3294095648 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5b8133fa0 0x7fb5b8134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:15.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.825+0000 7fb5be49d700 1 --2- 172.21.15.3:0/3294095648 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5b80a0a30 0x7fb5b80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:15.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.825+0000 7fb5be49d700 1 --2- 172.21.15.3:0/3294095648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5b80a1340 0x7fb5b8133a60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:15.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.825+0000 7fb5be49d700 1 -- 172.21.15.3:0/3294095648 >> 172.21.15.3:0/3294095648 conn(0x7fb5b80a6550 msgr2=0x7fb5b80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:15.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.825+0000 7fb5be49d700 1 -- 172.21.15.3:0/3294095648 shutdown_connections 2023-10-03T10:13:15.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.825+0000 7fb5be49d700 1 -- 172.21.15.3:0/3294095648 wait complete. 2023-10-03T10:13:15.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.826+0000 7fb5be49d700 1 Processor -- start 2023-10-03T10:13:15.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.826+0000 7fb5be49d700 1 -- start start 2023-10-03T10:13:15.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.827+0000 7fb5be49d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5b80a0a30 0x7fb5b811d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:15.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.827+0000 7fb5be49d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5b80a1340 0x7fb5b811d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:15.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.827+0000 7fb5be49d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5b8133fa0 0x7fb5b811dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:15.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.827+0000 7fb5be49d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b81208d0 con 0x7fb5b80a1340 2023-10-03T10:13:15.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.827+0000 7fb5be49d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b811f9c0 con 0x7fb5b8133fa0 2023-10-03T10:13:15.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.827+0000 7fb5be49d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b811fe80 con 0x7fb5b80a0a30 2023-10-03T10:13:15.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.827+0000 7fb5b77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5b80a1340 0x7fb5b811d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:15.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.827+0000 7fb5b77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5b80a1340 0x7fb5b811d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50530/0 (socket says 172.21.15.3:50530) 2023-10-03T10:13:15.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.827+0000 7fb5b77fe700 1 -- 172.21.15.3:0/3346010816 learned_addr learned my addr 172.21.15.3:0/3346010816 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:15.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.828+0000 7fb5b7fff700 1 --2- 172.21.15.3:0/3346010816 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5b80a0a30 0x7fb5b811d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:15.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.828+0000 7fb5bca3a700 1 --2- 172.21.15.3:0/3346010816 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5b8133fa0 0x7fb5b811dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:15.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.828+0000 7fb5b77fe700 1 -- 172.21.15.3:0/3346010816 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5b80a0a30 msgr2=0x7fb5b811d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:15.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.828+0000 7fb5b77fe700 1 --2- 172.21.15.3:0/3346010816 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5b80a0a30 0x7fb5b811d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:15.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.828+0000 7fb5b77fe700 1 -- 172.21.15.3:0/3346010816 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5b8133fa0 msgr2=0x7fb5b811dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:15.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.828+0000 7fb5b77fe700 1 --2- 172.21.15.3:0/3346010816 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5b8133fa0 0x7fb5b811dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:15.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.828+0000 7fb5b77fe700 1 -- 172.21.15.3:0/3346010816 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a80094a0 con 0x7fb5b80a1340 2023-10-03T10:13:15.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.828+0000 7fb5b77fe700 1 --2- 172.21.15.3:0/3346010816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5b80a1340 0x7fb5b811d590 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac009c40 tx=0x7fb5ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:15.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.828+0000 7fb5b4ff9700 1 -- 172.21.15.3:0/3346010816 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac004240 con 0x7fb5b80a1340 2023-10-03T10:13:15.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.828+0000 7fb5b4ff9700 1 -- 172.21.15.3:0/3346010816 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ac01f070 con 0x7fb5b80a1340 2023-10-03T10:13:15.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.828+0000 7fb5be49d700 1 -- 172.21.15.3:0/3346010816 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5ac009550 con 0x7fb5b80a1340 2023-10-03T10:13:15.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.828+0000 7fb5b4ff9700 1 -- 172.21.15.3:0/3346010816 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac0045a0 con 0x7fb5b80a1340 2023-10-03T10:13:15.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.829+0000 7fb5be49d700 1 -- 172.21.15.3:0/3346010816 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5b811ec60 con 0x7fb5b80a1340 2023-10-03T10:13:15.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.830+0000 7fb5be49d700 1 -- 172.21.15.3:0/3346010816 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5b8074ea0 con 0x7fb5b80a1340 2023-10-03T10:13:15.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.830+0000 7fb5b4ff9700 1 -- 172.21.15.3:0/3346010816 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb5ac021030 con 0x7fb5b80a1340 2023-10-03T10:13:15.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.830+0000 7fb5b4ff9700 1 -- 172.21.15.3:0/3346010816 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb5ac040cb0 con 0x7fb5b80a1340 2023-10-03T10:13:15.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.835+0000 7fb5b4ff9700 1 -- 172.21.15.3:0/3346010816 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb5ac0036a0 con 0x7fb5b80a1340 2023-10-03T10:13:15.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.868+0000 7f73b1153700 1 Processor -- start 2023-10-03T10:13:15.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.868+0000 7f73b1153700 1 -- start start 2023-10-03T10:13:15.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.869+0000 7f73b1153700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73ac0a0a30 0x7f73ac0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:15.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.869+0000 7f73b1153700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73ac09d630 0x7f73ac0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:15.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.869+0000 7f73aa59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73ac09d630 0x7f73ac0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:15.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.869+0000 7f73b1153700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73ac0a2a70 con 0x7f73ac133930 2023-10-03T10:13:15.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.869+0000 7f73b1153700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac0a01d0 con 0x7f73ac09d630 2023-10-03T10:13:15.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.869+0000 7f73aad9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73ac0a0a30 0x7f73ac0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:15.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.869+0000 7f73aa59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73ac09d630 0x7f73ac0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36628/0 (socket says 172.21.15.3:36628) 2023-10-03T10:13:15.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.869+0000 7f73aa59c700 1 -- 172.21.15.3:0/1330060909 learned_addr learned my addr 172.21.15.3:0/1330060909 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:15.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.869+0000 7f73b1153700 1 -- 172.21.15.3:0/1330060909 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac09e2b0 con 0x7f73ac0a0a30 2023-10-03T10:13:15.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.870+0000 7f73aa59c700 1 -- 172.21.15.3:0/1330060909 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73ac0a0a30 msgr2=0x7f73ac0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:15.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.870+0000 7f73aa59c700 1 --2- 172.21.15.3:0/1330060909 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73ac0a0a30 0x7f73ac0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:15.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.870+0000 7f73aa59c700 1 -- 172.21.15.3:0/1330060909 >> v1:172.21.15.3:6789/0 conn(0x7f73ac133930 legacy=0x7f73ac133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:15.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.870+0000 7f73aa59c700 1 -- 172.21.15.3:0/1330060909 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ac09dc50 con 0x7f73ac09d630 2023-10-03T10:13:15.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.870+0000 7f73aa59c700 1 --2- 172.21.15.3:0/1330060909 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73ac09d630 0x7f73ac0a1340 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009920 tx=0x7f73a0009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fa9aee45b8dda75 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:15.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.870+0000 7f73a9d9b700 1 -- 172.21.15.3:0/1330060909 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a0011830 con 0x7f73ac09d630 2023-10-03T10:13:15.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.870+0000 7f73a9d9b700 1 -- 172.21.15.3:0/1330060909 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a0011990 con 0x7f73ac09d630 2023-10-03T10:13:15.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.870+0000 7f73a9d9b700 1 -- 172.21.15.3:0/1330060909 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a0011d20 con 0x7f73ac09d630 2023-10-03T10:13:15.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.871+0000 7f73b1153700 1 -- 172.21.15.3:0/1330060909 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73ac09d630 msgr2=0x7f73ac0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:15.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.871+0000 7f73b1153700 1 --2- 172.21.15.3:0/1330060909 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73ac09d630 0x7f73ac0a1340 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009920 tx=0x7f73a0009c30 comp rx=0 tx=0).stop 2023-10-03T10:13:15.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.871+0000 7f73b1153700 1 -- 172.21.15.3:0/1330060909 shutdown_connections 2023-10-03T10:13:15.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.871+0000 7f73b1153700 1 --2- 172.21.15.3:0/1330060909 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73ac0a0a30 0x7f73ac0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:15.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.871+0000 7f73b1153700 1 --2- 172.21.15.3:0/1330060909 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73ac09d630 0x7f73ac0a1340 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:15.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.871+0000 7f73b1153700 1 -- 172.21.15.3:0/1330060909 >> 172.21.15.3:0/1330060909 conn(0x7f73ac0a3050 msgr2=0x7f73ac0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:15.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.871+0000 7f73b1153700 1 -- 172.21.15.3:0/1330060909 shutdown_connections 2023-10-03T10:13:15.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.871+0000 7f73b1153700 1 -- 172.21.15.3:0/1330060909 wait complete. 2023-10-03T10:13:15.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.873+0000 7f73b1153700 1 Processor -- start 2023-10-03T10:13:15.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.873+0000 7f73b1153700 1 -- start start 2023-10-03T10:13:15.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.874+0000 7f73b1153700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73ac09d630 0x7f73ac11f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:15.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.874+0000 7f73b1153700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73ac0a0a30 0x7f73ac11f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:15.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.874+0000 7f73b1153700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73ac133930 0x7f73ac11fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:15.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.874+0000 7f73b1153700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac120f00 con 0x7f73ac0a0a30 2023-10-03T10:13:15.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.874+0000 7f73b1153700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac120d10 con 0x7f73ac09d630 2023-10-03T10:13:15.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.874+0000 7f73b1153700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac0a0410 con 0x7f73ac133930 2023-10-03T10:13:15.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.874+0000 7f73ab59e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73ac133930 0x7f73ac11fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:15.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.874+0000 7f73ab59e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73ac133930 0x7f73ac11fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55438/0 (socket says 172.21.15.3:55438) 2023-10-03T10:13:15.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.874+0000 7f73aad9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73ac09d630 0x7f73ac11f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:15.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.874+0000 7f73ab59e700 1 -- 172.21.15.3:0/2422169459 learned_addr learned my addr 172.21.15.3:0/2422169459 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:15.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.874+0000 7f73aa59c700 1 --2- 172.21.15.3:0/2422169459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73ac0a0a30 0x7f73ac11f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:15.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.874+0000 7f73aad9d700 1 -- 172.21.15.3:0/2422169459 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73ac133930 msgr2=0x7f73ac11fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:15.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.874+0000 7f73aad9d700 1 --2- 172.21.15.3:0/2422169459 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73ac133930 0x7f73ac11fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:15.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.874+0000 7f73aad9d700 1 -- 172.21.15.3:0/2422169459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73ac0a0a30 msgr2=0x7f73ac11f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:15.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.874+0000 7f73aad9d700 1 --2- 172.21.15.3:0/2422169459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73ac0a0a30 0x7f73ac11f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:15.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.874+0000 7f73aad9d700 1 -- 172.21.15.3:0/2422169459 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a0009550 con 0x7f73ac09d630 2023-10-03T10:13:15.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.874+0000 7f73aa59c700 1 --2- 172.21.15.3:0/2422169459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73ac0a0a30 0x7f73ac11f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:15.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.874+0000 7f73ab59e700 1 --2- 172.21.15.3:0/2422169459 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73ac133930 0x7f73ac11fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:15.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.875+0000 7f73aad9d700 1 --2- 172.21.15.3:0/2422169459 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73ac09d630 0x7f73ac11f2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f739c00e900 tx=0x7f739c00ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:15.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.875+0000 7f73937fe700 1 -- 172.21.15.3:0/2422169459 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c00ee50 con 0x7f73ac09d630 2023-10-03T10:13:15.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.875+0000 7f73937fe700 1 -- 172.21.15.3:0/2422169459 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f739c00c850 con 0x7f73ac09d630 2023-10-03T10:13:15.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.875+0000 7f73b1153700 1 -- 172.21.15.3:0/2422169459 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73ac122680 con 0x7f73ac09d630 2023-10-03T10:13:15.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.875+0000 7f73937fe700 1 -- 172.21.15.3:0/2422169459 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c00cbc0 con 0x7f73ac09d630 2023-10-03T10:13:15.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.875+0000 7f73b1153700 1 -- 172.21.15.3:0/2422169459 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73ac122c20 con 0x7f73ac09d630 2023-10-03T10:13:15.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.876+0000 7f73937fe700 1 -- 172.21.15.3:0/2422169459 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f739c00cd20 con 0x7f73ac09d630 2023-10-03T10:13:15.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.876+0000 7f73b1153700 1 -- 172.21.15.3:0/2422169459 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73ac09c900 con 0x7f73ac09d630 2023-10-03T10:13:15.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.876+0000 7f73937fe700 1 -- 172.21.15.3:0/2422169459 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f739c03f810 con 0x7f73ac09d630 2023-10-03T10:13:15.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:15.881+0000 7f73937fe700 1 -- 172.21.15.3:0/2422169459 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f739c0232f0 con 0x7f73ac09d630 2023-10-03T10:13:16.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.003+0000 7f9190ed3700 1 -- 172.21.15.3:0/2464460512 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f918c074ea0 con 0x7f918c09f240 2023-10-03T10:13:16.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.004+0000 7f917affd700 1 -- 172.21.15.3:0/2464460512 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f918001c3c0 con 0x7f918c09f240 2023-10-03T10:13:16.005 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:16.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.007+0000 7f9190ed3700 1 -- 172.21.15.3:0/2464460512 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f918c09f240 msgr2=0x7f918c1a42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:16.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.007+0000 7f9190ed3700 1 --2- 172.21.15.3:0/2464460512 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f918c09f240 0x7f918c1a42b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f918000e810 tx=0x7f918000eb20 comp rx=0 tx=0).stop 2023-10-03T10:13:16.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.007+0000 7f9190ed3700 1 -- 172.21.15.3:0/2464460512 shutdown_connections 2023-10-03T10:13:16.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.007+0000 7f9190ed3700 1 --2- 172.21.15.3:0/2464460512 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f918c09f650 0x7f918c1a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:16.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.007+0000 7f9190ed3700 1 --2- 172.21.15.3:0/2464460512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f918c09e930 0x7f918c1a3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:16.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.007+0000 7f9190ed3700 1 --2- 172.21.15.3:0/2464460512 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f918c09f240 0x7f918c1a42b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:16.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.007+0000 7f9190ed3700 1 -- 172.21.15.3:0/2464460512 >> 172.21.15.3:0/2464460512 conn(0x7f918c0a4210 msgr2=0x7f918c1b8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:16.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.007+0000 7f9190ed3700 1 -- 172.21.15.3:0/2464460512 shutdown_connections 2023-10-03T10:13:16.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.007+0000 7f9190ed3700 1 -- 172.21.15.3:0/2464460512 wait complete. 2023-10-03T10:13:16.027 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:16.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.053+0000 7fb5be49d700 1 -- 172.21.15.3:0/3346010816 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb5b80a28e0 con 0x7fb5b80a1340 2023-10-03T10:13:16.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.053+0000 7fb5b4ff9700 1 -- 172.21.15.3:0/3346010816 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb5ac015030 con 0x7fb5b80a1340 2023-10-03T10:13:16.054 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:16.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.056+0000 7f4a3dbfc700 1 -- 172.21.15.3:0/628291945 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4a38074ea0 con 0x7f4a3809d6f0 2023-10-03T10:13:16.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.056+0000 7fb5be49d700 1 -- 172.21.15.3:0/3346010816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5b80a1340 msgr2=0x7fb5b811d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:16.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.056+0000 7fb5be49d700 1 --2- 172.21.15.3:0/3346010816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5b80a1340 0x7fb5b811d590 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac009c40 tx=0x7fb5ac004750 comp rx=0 tx=0).stop 2023-10-03T10:13:16.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.056+0000 7fb5be49d700 1 -- 172.21.15.3:0/3346010816 shutdown_connections 2023-10-03T10:13:16.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.056+0000 7fb5be49d700 1 --2- 172.21.15.3:0/3346010816 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5b8133fa0 0x7fb5b811dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:16.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.056+0000 7fb5be49d700 1 --2- 172.21.15.3:0/3346010816 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5b80a0a30 0x7fb5b811d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:16.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.056+0000 7fb5be49d700 1 --2- 172.21.15.3:0/3346010816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5b80a1340 0x7fb5b811d590 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:16.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.056+0000 7fb5be49d700 1 -- 172.21.15.3:0/3346010816 >> 172.21.15.3:0/3346010816 conn(0x7fb5b80a6550 msgr2=0x7fb5b81331f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:16.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.056+0000 7fb5be49d700 1 -- 172.21.15.3:0/3346010816 shutdown_connections 2023-10-03T10:13:16.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.056+0000 7f4a1ffff700 1 -- 172.21.15.3:0/628291945 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4a2802b020 con 0x7f4a3809d6f0 2023-10-03T10:13:16.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.056+0000 7fb5be49d700 1 -- 172.21.15.3:0/3346010816 wait complete. 2023-10-03T10:13:16.058 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:16.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.058+0000 7f4a3dbfc700 1 -- 172.21.15.3:0/628291945 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a3809d6f0 msgr2=0x7f4a381c3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:16.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.058+0000 7f4a3dbfc700 1 --2- 172.21.15.3:0/628291945 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a3809d6f0 0x7f4a381c3c30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4a28009870 tx=0x7f4a28004750 comp rx=0 tx=0).stop 2023-10-03T10:13:16.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.058+0000 7f4a3dbfc700 1 -- 172.21.15.3:0/628291945 shutdown_connections 2023-10-03T10:13:16.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.058+0000 7f4a3dbfc700 1 --2- 172.21.15.3:0/628291945 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a3809e410 0x7f4a381c46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:16.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.059+0000 7f4a3dbfc700 1 --2- 172.21.15.3:0/628291945 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a3809d6f0 0x7f4a381c3c30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:16.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.059+0000 7f4a3dbfc700 1 --2- 172.21.15.3:0/628291945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a3809e000 0x7f4a381c4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:16.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.059+0000 7f4a3dbfc700 1 -- 172.21.15.3:0/628291945 >> 172.21.15.3:0/628291945 conn(0x7f4a38066f50 msgr2=0x7f4a380a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:16.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.059+0000 7f4a3dbfc700 1 -- 172.21.15.3:0/628291945 shutdown_connections 2023-10-03T10:13:16.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.059+0000 7f4a3dbfc700 1 -- 172.21.15.3:0/628291945 wait complete. 2023-10-03T10:13:16.071 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:16.074 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:16.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.089+0000 7f73b1153700 1 -- 172.21.15.3:0/2422169459 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f73ac074770 con 0x7f73ac09d630 2023-10-03T10:13:16.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.090+0000 7f73937fe700 1 -- 172.21.15.3:0/2422169459 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f739c021090 con 0x7f73ac09d630 2023-10-03T10:13:16.090 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:16.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.092+0000 7f73b1153700 1 -- 172.21.15.3:0/2422169459 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73ac09d630 msgr2=0x7f73ac11f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:16.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.092+0000 7f73b1153700 1 --2- 172.21.15.3:0/2422169459 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73ac09d630 0x7f73ac11f2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f739c00e900 tx=0x7f739c00ec10 comp rx=0 tx=0).stop 2023-10-03T10:13:16.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.092+0000 7f73b1153700 1 -- 172.21.15.3:0/2422169459 shutdown_connections 2023-10-03T10:13:16.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.092+0000 7f73b1153700 1 --2- 172.21.15.3:0/2422169459 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73ac133930 0x7f73ac11fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:16.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.092+0000 7f73b1153700 1 --2- 172.21.15.3:0/2422169459 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73ac09d630 0x7f73ac11f2f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:16.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.092+0000 7f73b1153700 1 --2- 172.21.15.3:0/2422169459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73ac0a0a30 0x7f73ac11f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:16.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.092+0000 7f73b1153700 1 -- 172.21.15.3:0/2422169459 >> 172.21.15.3:0/2422169459 conn(0x7f73ac0a3050 msgr2=0x7f73ac099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:16.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.092+0000 7f73b1153700 1 -- 172.21.15.3:0/2422169459 shutdown_connections 2023-10-03T10:13:16.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:16.092+0000 7f73b1153700 1 -- 172.21.15.3:0/2422169459 wait complete. 2023-10-03T10:13:16.106 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:17.027 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:17.072 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:17.074 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:17.107 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:17.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.186+0000 7f96cc689700 1 Processor -- start 2023-10-03T10:13:17.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.187+0000 7f96cc689700 1 -- start start 2023-10-03T10:13:17.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.188+0000 7f96cc689700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f96c41367c0 0x7f96c4138ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:17.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.188+0000 7f96cc689700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f96c41390e0 0x7f96c413b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:17.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.188+0000 7f96cc689700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96c41334b0 con 0x7f96c4133870 2023-10-03T10:13:17.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.188+0000 7f96cc689700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96c409fa40 con 0x7f96c41390e0 2023-10-03T10:13:17.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.188+0000 7f96cc689700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c406f250 con 0x7f96c41367c0 2023-10-03T10:13:17.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.188+0000 7f96cac26700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f96c41390e0 0x7f96c413b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:17.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.188+0000 7f96cac26700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f96c41390e0 0x7f96c413b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50548/0 (socket says 172.21.15.3:50548) 2023-10-03T10:13:17.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.188+0000 7f96cac26700 1 -- 172.21.15.3:0/2911417686 learned_addr learned my addr 172.21.15.3:0/2911417686 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:17.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.188+0000 7f96c9c24700 1 --2- 172.21.15.3:0/2911417686 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f96c41367c0 0x7f96c4138ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:17.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.188+0000 7f96cac26700 1 -- 172.21.15.3:0/2911417686 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f96c41367c0 msgr2=0x7f96c4138ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:17.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.188+0000 7f96cac26700 1 --2- 172.21.15.3:0/2911417686 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f96c41367c0 0x7f96c4138ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.188+0000 7f96cac26700 1 -- 172.21.15.3:0/2911417686 >> v1:172.21.15.3:6789/0 conn(0x7f96c4133870 legacy=0x7f96c4133c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:17.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.188+0000 7f96cac26700 1 -- 172.21.15.3:0/2911417686 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c406fb20 con 0x7f96c41390e0 2023-10-03T10:13:17.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.188+0000 7f96cac26700 1 --2- 172.21.15.3:0/2911417686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96c41390e0 0x7f96c413b4c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f96bc009e50 tx=0x7f96bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce8cb031b2eefd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:17.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.189+0000 7f96c9423700 1 -- 172.21.15.3:0/2911417686 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96bc00e070 con 0x7f96c41390e0 2023-10-03T10:13:17.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.189+0000 7f96c9423700 1 -- 172.21.15.3:0/2911417686 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96bc00b910 con 0x7f96c41390e0 2023-10-03T10:13:17.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.189+0000 7f96c9423700 1 -- 172.21.15.3:0/2911417686 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96bc00bcc0 con 0x7f96c41390e0 2023-10-03T10:13:17.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.189+0000 7f96cc689700 1 -- 172.21.15.3:0/2911417686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96c41390e0 msgr2=0x7f96c413b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:17.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.189+0000 7f96cc689700 1 --2- 172.21.15.3:0/2911417686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96c41390e0 0x7f96c413b4c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f96bc009e50 tx=0x7f96bc00b330 comp rx=0 tx=0).stop 2023-10-03T10:13:17.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.190+0000 7f96cc689700 1 -- 172.21.15.3:0/2911417686 shutdown_connections 2023-10-03T10:13:17.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.190+0000 7f96cc689700 1 --2- 172.21.15.3:0/2911417686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96c41390e0 0x7f96c413b4c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.190+0000 7f96cc689700 1 --2- 172.21.15.3:0/2911417686 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f96c41367c0 0x7f96c4138ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.190+0000 7f96cc689700 1 -- 172.21.15.3:0/2911417686 >> 172.21.15.3:0/2911417686 conn(0x7f96c40a2610 msgr2=0x7f96c40a2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:17.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.190+0000 7f96cc689700 1 -- 172.21.15.3:0/2911417686 shutdown_connections 2023-10-03T10:13:17.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.190+0000 7f96cc689700 1 -- 172.21.15.3:0/2911417686 wait complete. 2023-10-03T10:13:17.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.191+0000 7f96cc689700 1 Processor -- start 2023-10-03T10:13:17.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.191+0000 7f96cc689700 1 -- start start 2023-10-03T10:13:17.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.191+0000 7f96cc689700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f96c4133870 0x7f96c4119b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:17.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.191+0000 7f96cc689700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f96c41367c0 0x7f96c411a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:17.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.191+0000 7f96cc689700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96c41390e0 0x7f96c411a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:17.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.191+0000 7f96cc689700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c411d010 con 0x7f96c41390e0 2023-10-03T10:13:17.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.191+0000 7f96cc689700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c4138cb0 con 0x7f96c4133870 2023-10-03T10:13:17.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.191+0000 7f96cc689700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f96c411b7b0 con 0x7f96c41367c0 2023-10-03T10:13:17.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.192+0000 7f96ca425700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f96c4133870 0x7f96c4119b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:17.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.192+0000 7f96ca425700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f96c4133870 0x7f96c4119b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36668/0 (socket says 172.21.15.3:36668) 2023-10-03T10:13:17.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.192+0000 7f96ca425700 1 -- 172.21.15.3:0/827019201 learned_addr learned my addr 172.21.15.3:0/827019201 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:17.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.192+0000 7f96cac26700 1 --2- 172.21.15.3:0/827019201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96c41390e0 0x7f96c411a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:17.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.192+0000 7f96ca425700 1 -- 172.21.15.3:0/827019201 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f96c41367c0 msgr2=0x7f96c411a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:17.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.192+0000 7f96ca425700 1 --2- 172.21.15.3:0/827019201 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f96c41367c0 0x7f96c411a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.192+0000 7f96ca425700 1 -- 172.21.15.3:0/827019201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96c41390e0 msgr2=0x7f96c411a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:17.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.192+0000 7f96ca425700 1 --2- 172.21.15.3:0/827019201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96c41390e0 0x7f96c411a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.192+0000 7f96ca425700 1 -- 172.21.15.3:0/827019201 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96bc009b30 con 0x7f96c4133870 2023-10-03T10:13:17.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.192+0000 7f96ca425700 1 --2- 172.21.15.3:0/827019201 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f96c4133870 0x7f96c4119b20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f96c000c4e0 tx=0x7f96c000c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:17.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.192+0000 7f96baffd700 1 -- 172.21.15.3:0/827019201 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c00052d0 con 0x7f96c4133870 2023-10-03T10:13:17.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.192+0000 7f96baffd700 1 -- 172.21.15.3:0/827019201 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c0005430 con 0x7f96c4133870 2023-10-03T10:13:17.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.192+0000 7f96cc689700 1 -- 172.21.15.3:0/827019201 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c412f960 con 0x7f96c4133870 2023-10-03T10:13:17.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.193+0000 7f96cc689700 1 -- 172.21.15.3:0/827019201 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c411dd40 con 0x7f96c4133870 2023-10-03T10:13:17.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.193+0000 7f96baffd700 1 -- 172.21.15.3:0/827019201 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c00038c0 con 0x7f96c4133870 2023-10-03T10:13:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.194+0000 7f96baffd700 1 -- 172.21.15.3:0/827019201 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f96c0003a20 con 0x7f96c4133870 2023-10-03T10:13:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.194+0000 7f96cc689700 1 -- 172.21.15.3:0/827019201 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96ac0052e0 con 0x7f96c4133870 2023-10-03T10:13:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.194+0000 7f96baffd700 1 -- 172.21.15.3:0/827019201 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f96c003ec30 con 0x7f96c4133870 2023-10-03T10:13:17.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.203+0000 7f96baffd700 1 -- 172.21.15.3:0/827019201 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f96c0045080 con 0x7f96c4133870 2023-10-03T10:13:17.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.223+0000 7f732b5e0700 1 Processor -- start 2023-10-03T10:13:17.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.223+0000 7f732b5e0700 1 -- start start 2023-10-03T10:13:17.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.224+0000 7f732b5e0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f732409e930 0x7f732409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:17.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.224+0000 7f732b5e0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f732409f240 0x7f73241317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:17.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.224+0000 7f732b5e0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7324070940 con 0x7f732409f650 2023-10-03T10:13:17.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.224+0000 7f732b5e0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f732406fd60 con 0x7f732409e930 2023-10-03T10:13:17.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.224+0000 7f732b5e0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732406fa80 con 0x7f732409f240 2023-10-03T10:13:17.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.224+0000 7f732937c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f732409e930 0x7f732409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:17.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.224+0000 7f732937c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f732409e930 0x7f732409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50562/0 (socket says 172.21.15.3:50562) 2023-10-03T10:13:17.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.224+0000 7f732937c700 1 -- 172.21.15.3:0/1517874709 learned_addr learned my addr 172.21.15.3:0/1517874709 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:17.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.224+0000 7f732937c700 1 -- 172.21.15.3:0/1517874709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f732409f240 msgr2=0x7f73241317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:17.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.224+0000 7f732937c700 1 --2- 172.21.15.3:0/1517874709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f732409f240 0x7f73241317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.224+0000 7f732937c700 1 -- 172.21.15.3:0/1517874709 >> v1:172.21.15.3:6789/0 conn(0x7f732409f650 legacy=0x7f7324131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:17.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.224+0000 7f732937c700 1 -- 172.21.15.3:0/1517874709 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732409fb60 con 0x7f732409e930 2023-10-03T10:13:17.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.225+0000 7f732937c700 1 --2- 172.21.15.3:0/1517874709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f732409e930 0x7f732409ed00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f73200098f0 tx=0x7f7320009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea5cdd69c428bf87 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:17.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.225+0000 7f731bfff700 1 -- 172.21.15.3:0/1517874709 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732000b8a0 con 0x7f732409e930 2023-10-03T10:13:17.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.225+0000 7f731bfff700 1 -- 172.21.15.3:0/1517874709 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732000ba00 con 0x7f732409e930 2023-10-03T10:13:17.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.225+0000 7f731bfff700 1 -- 172.21.15.3:0/1517874709 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732000bd90 con 0x7f732409e930 2023-10-03T10:13:17.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.225+0000 7f732b5e0700 1 -- 172.21.15.3:0/1517874709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f732409e930 msgr2=0x7f732409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:17.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.225+0000 7f732b5e0700 1 --2- 172.21.15.3:0/1517874709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f732409e930 0x7f732409ed00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f73200098f0 tx=0x7f7320009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:17.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.225+0000 7f732b5e0700 1 -- 172.21.15.3:0/1517874709 shutdown_connections 2023-10-03T10:13:17.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.225+0000 7f732b5e0700 1 --2- 172.21.15.3:0/1517874709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f732409e930 0x7f732409ed00 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.225+0000 7f732b5e0700 1 --2- 172.21.15.3:0/1517874709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f732409f240 0x7f73241317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.225+0000 7f732b5e0700 1 -- 172.21.15.3:0/1517874709 >> 172.21.15.3:0/1517874709 conn(0x7f73240a4210 msgr2=0x7f73240a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:17.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.226+0000 7f732b5e0700 1 -- 172.21.15.3:0/1517874709 shutdown_connections 2023-10-03T10:13:17.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.226+0000 7f732b5e0700 1 -- 172.21.15.3:0/1517874709 wait complete. 2023-10-03T10:13:17.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.227+0000 7f732b5e0700 1 Processor -- start 2023-10-03T10:13:17.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.227+0000 7f732b5e0700 1 -- start start 2023-10-03T10:13:17.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.227+0000 7fe79a8b2700 1 Processor -- start 2023-10-03T10:13:17.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.227+0000 7fe79a8b2700 1 -- start start 2023-10-03T10:13:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.227+0000 7f732b5e0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f732409e930 0x7f73241a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.227+0000 7f732b5e0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f732409f240 0x7f73241a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.228+0000 7f732b5e0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f732409f650 0x7f73241a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.228+0000 7f732b5e0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73241a70a0 con 0x7f732409e930 2023-10-03T10:13:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.228+0000 7f732b5e0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7324071d20 con 0x7f732409f240 2023-10-03T10:13:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.228+0000 7f732b5e0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73240a3460 con 0x7f732409f650 2023-10-03T10:13:17.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.228+0000 7fe79a8b2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe7941338c0 0x7fe794133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:17.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.228+0000 7fe79a8b2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe79409e560 0x7fe79413b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:17.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.228+0000 7fe79a8b2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe79406f720 con 0x7fe7941341d0 2023-10-03T10:13:17.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.228+0000 7fe79a8b2700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe794070c80 con 0x7fe7941338c0 2023-10-03T10:13:17.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.228+0000 7fe79a8b2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe79406fce0 con 0x7fe79409e560 2023-10-03T10:13:17.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.228+0000 7fe793fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe7941338c0 0x7fe794133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:17.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.228+0000 7fe793fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe7941338c0 0x7fe794133c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50576/0 (socket says 172.21.15.3:50576) 2023-10-03T10:13:17.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.228+0000 7fe793fff700 1 -- 172.21.15.3:0/213968131 learned_addr learned my addr 172.21.15.3:0/213968131 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:17.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.228+0000 7f732937c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f732409e930 0x7f73241a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:17.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.228+0000 7f732937c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f732409e930 0x7f73241a3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50568/0 (socket says 172.21.15.3:50568) 2023-10-03T10:13:17.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.228+0000 7f732937c700 1 -- 172.21.15.3:0/2076966283 learned_addr learned my addr 172.21.15.3:0/2076966283 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:17.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.228+0000 7f7329b7d700 1 --2- 172.21.15.3:0/2076966283 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f732409f650 0x7f73241a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.228+0000 7f7329b7d700 1 -- 172.21.15.3:0/2076966283 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f732409f240 msgr2=0x7f73241a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.229+0000 7f7329b7d700 1 --2- 172.21.15.3:0/2076966283 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f732409f240 0x7f73241a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.229+0000 7f7329b7d700 1 -- 172.21.15.3:0/2076966283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f732409e930 msgr2=0x7f73241a3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.229+0000 7f7329b7d700 1 --2- 172.21.15.3:0/2076966283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f732409e930 0x7f73241a3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.229+0000 7f7329b7d700 1 -- 172.21.15.3:0/2076966283 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7320009550 con 0x7f732409f650 2023-10-03T10:13:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.228+0000 7fe798e4f700 1 --2- 172.21.15.3:0/213968131 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe79409e560 0x7fe79413b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:17.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.228+0000 7fe793fff700 1 -- 172.21.15.3:0/213968131 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe79409e560 msgr2=0x7fe79413b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:17.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.228+0000 7fe793fff700 1 --2- 172.21.15.3:0/213968131 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe79409e560 0x7fe79413b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.228+0000 7fe793fff700 1 -- 172.21.15.3:0/213968131 >> v1:172.21.15.3:6789/0 conn(0x7fe7941341d0 legacy=0x7fe79409de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:17.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.228+0000 7fe793fff700 1 -- 172.21.15.3:0/213968131 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7941336a0 con 0x7fe7941338c0 2023-10-03T10:13:17.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.229+0000 7f7329b7d700 1 --2- 172.21.15.3:0/2076966283 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f732409f650 0x7f73241a4950 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f731c00ef30 tx=0x7f731c010330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:17.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.229+0000 7fe793fff700 1 --2- 172.21.15.3:0/213968131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7941338c0 0x7fe794133c90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe78c009e50 tx=0x7fe78c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0bb03ee0939bd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:17.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.229+0000 7f7319ffb700 1 -- 172.21.15.3:0/2076966283 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c013070 con 0x7f732409f650 2023-10-03T10:13:17.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.229+0000 7f7319ffb700 1 -- 172.21.15.3:0/2076966283 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731c00e2e0 con 0x7f732409f650 2023-10-03T10:13:17.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.229+0000 7f732b5e0700 1 -- 172.21.15.3:0/2076966283 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73240a0190 con 0x7f732409f650 2023-10-03T10:13:17.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.229+0000 7fe792ffd700 1 -- 172.21.15.3:0/213968131 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c00e070 con 0x7fe7941338c0 2023-10-03T10:13:17.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.229+0000 7fe792ffd700 1 -- 172.21.15.3:0/213968131 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78c00b9c0 con 0x7fe7941338c0 2023-10-03T10:13:17.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.229+0000 7fe792ffd700 1 -- 172.21.15.3:0/213968131 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c00bd70 con 0x7fe7941338c0 2023-10-03T10:13:17.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.230+0000 7f7319ffb700 1 -- 172.21.15.3:0/2076966283 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c010c40 con 0x7f732409f650 2023-10-03T10:13:17.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.230+0000 7f732b5e0700 1 -- 172.21.15.3:0/2076966283 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73241a6c50 con 0x7f732409f650 2023-10-03T10:13:17.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.230+0000 7fe79a8b2700 1 -- 172.21.15.3:0/213968131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7941338c0 msgr2=0x7fe794133c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:17.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.230+0000 7fe79a8b2700 1 --2- 172.21.15.3:0/213968131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7941338c0 0x7fe794133c90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe78c009e50 tx=0x7fe78c00b330 comp rx=0 tx=0).stop 2023-10-03T10:13:17.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.231+0000 7f7319ffb700 1 -- 172.21.15.3:0/2076966283 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f731c003a00 con 0x7f732409f650 2023-10-03T10:13:17.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.231+0000 7f732b5e0700 1 -- 172.21.15.3:0/2076966283 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73241a72d0 con 0x7f732409f650 2023-10-03T10:13:17.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.231+0000 7fe79a8b2700 1 -- 172.21.15.3:0/213968131 shutdown_connections 2023-10-03T10:13:17.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.231+0000 7fe79a8b2700 1 --2- 172.21.15.3:0/213968131 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe79409e560 0x7fe79413b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.231+0000 7fe79a8b2700 1 --2- 172.21.15.3:0/213968131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7941338c0 0x7fe794133c90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.231+0000 7fe79a8b2700 1 -- 172.21.15.3:0/213968131 >> 172.21.15.3:0/213968131 conn(0x7fe7940a2aa0 msgr2=0x7fe7940a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:17.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.231+0000 7f7319ffb700 1 -- 172.21.15.3:0/2076966283 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f731c04b3b0 con 0x7f732409f650 2023-10-03T10:13:17.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.231+0000 7fe79a8b2700 1 -- 172.21.15.3:0/213968131 shutdown_connections 2023-10-03T10:13:17.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.234+0000 7fe79a8b2700 1 -- 172.21.15.3:0/213968131 wait complete. 2023-10-03T10:13:17.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.235+0000 7fe79a8b2700 1 Processor -- start 2023-10-03T10:13:17.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.235+0000 7fe79a8b2700 1 -- start start 2023-10-03T10:13:17.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.235+0000 7fe79a8b2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe79409e560 0x7fe7941ce680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:17.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.235+0000 7fe79a8b2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7941338c0 0x7fe7941cebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:17.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.235+0000 7fe79a8b2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7941341d0 0x7fe7941cf100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:17.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.235+0000 7fe79a8b2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79412f0b0 con 0x7fe7941341d0 2023-10-03T10:13:17.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.235+0000 7fe79a8b2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe79412f470 con 0x7fe79409e560 2023-10-03T10:13:17.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.235+0000 7fe79a8b2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe794130db0 con 0x7fe7941338c0 2023-10-03T10:13:17.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.236+0000 7fe793fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe79409e560 0x7fe7941ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:17.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.236+0000 7fe793fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe79409e560 0x7fe7941ce680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36708/0 (socket says 172.21.15.3:36708) 2023-10-03T10:13:17.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.236+0000 7fe793fff700 1 -- 172.21.15.3:0/1263523123 learned_addr learned my addr 172.21.15.3:0/1263523123 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:17.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.236+0000 7fe793fff700 1 -- 172.21.15.3:0/1263523123 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7941338c0 msgr2=0x7fe7941cebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:17.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.236+0000 7fe793fff700 1 --2- 172.21.15.3:0/1263523123 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7941338c0 0x7fe7941cebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.236+0000 7fe793fff700 1 -- 172.21.15.3:0/1263523123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7941341d0 msgr2=0x7fe7941cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:17.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.236+0000 7fe793fff700 1 --2- 172.21.15.3:0/1263523123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7941341d0 0x7fe7941cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.236+0000 7fe793fff700 1 -- 172.21.15.3:0/1263523123 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78c009b30 con 0x7fe79409e560 2023-10-03T10:13:17.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.236+0000 7fe793fff700 1 --2- 172.21.15.3:0/1263523123 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe79409e560 0x7fe7941ce680 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe79406f8b0 tx=0x7fe78c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:17.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.236+0000 7fe790ff9700 1 -- 172.21.15.3:0/1263523123 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c0042d0 con 0x7fe79409e560 2023-10-03T10:13:17.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.237+0000 7fe790ff9700 1 -- 172.21.15.3:0/1263523123 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78c004430 con 0x7fe79409e560 2023-10-03T10:13:17.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.237+0000 7fe790ff9700 1 -- 172.21.15.3:0/1263523123 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c0047e0 con 0x7fe79409e560 2023-10-03T10:13:17.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.237+0000 7fe79a8b2700 1 -- 172.21.15.3:0/1263523123 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79412fab0 con 0x7fe79409e560 2023-10-03T10:13:17.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.237+0000 7fe79a8b2700 1 -- 172.21.15.3:0/1263523123 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7941307e0 con 0x7fe79409e560 2023-10-03T10:13:17.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.238+0000 7fe79a8b2700 1 -- 172.21.15.3:0/1263523123 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe794074ea0 con 0x7fe79409e560 2023-10-03T10:13:17.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.238+0000 7f7319ffb700 1 -- 172.21.15.3:0/2076966283 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f73241a72d0 con 0x7f732409f650 2023-10-03T10:13:17.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.238+0000 7fe790ff9700 1 -- 172.21.15.3:0/1263523123 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe78c01c440 con 0x7fe79409e560 2023-10-03T10:13:17.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.238+0000 7fe790ff9700 1 -- 172.21.15.3:0/1263523123 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe78c041260 con 0x7fe79409e560 2023-10-03T10:13:17.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.243+0000 7fe790ff9700 1 -- 172.21.15.3:0/1263523123 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe78c011980 con 0x7fe79409e560 2023-10-03T10:13:17.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.275+0000 7fb900cea700 1 Processor -- start 2023-10-03T10:13:17.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.275+0000 7fb900cea700 1 -- start start 2023-10-03T10:13:17.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.276+0000 7fb900cea700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb8fc09d6f0 0x7fb8fc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:17.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.276+0000 7fb900cea700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8fc09e440 0x7fb8fc134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:17.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.276+0000 7fb900cea700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8fc09b120 con 0x7fb8fc09e000 2023-10-03T10:13:17.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.276+0000 7fb900cea700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8fc06b460 con 0x7fb8fc09d6f0 2023-10-03T10:13:17.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.276+0000 7fb900cea700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc09a7e0 con 0x7fb8fc09e440 2023-10-03T10:13:17.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.276+0000 7fb8fad9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8fc09e440 0x7fb8fc134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:17.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.277+0000 7fb8fad9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8fc09e440 0x7fb8fc134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36724/0 (socket says 172.21.15.3:36724) 2023-10-03T10:13:17.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.277+0000 7fb8fad9d700 1 -- 172.21.15.3:0/3748085798 learned_addr learned my addr 172.21.15.3:0/3748085798 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:17.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.277+0000 7fb8fa59c700 1 --2- 172.21.15.3:0/3748085798 >> v2:172.21.15.3:3300/0 conn(0x7fb8fc09d6f0 0x7fb8fc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:17.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.277+0000 7fb8fad9d700 1 -- 172.21.15.3:0/3748085798 >> v2:172.21.15.3:3300/0 conn(0x7fb8fc09d6f0 msgr2=0x7fb8fc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:17.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.277+0000 7fb8fad9d700 1 --2- 172.21.15.3:0/3748085798 >> v2:172.21.15.3:3300/0 conn(0x7fb8fc09d6f0 0x7fb8fc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.277+0000 7fb8fad9d700 1 -- 172.21.15.3:0/3748085798 >> v1:172.21.15.3:6789/0 conn(0x7fb8fc09e000 legacy=0x7fb8fc1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:17.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.277+0000 7fb8fad9d700 1 -- 172.21.15.3:0/3748085798 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8fc09a500 con 0x7fb8fc09e440 2023-10-03T10:13:17.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.277+0000 7fb8fad9d700 1 --2- 172.21.15.3:0/3748085798 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8fc09e440 0x7fb8fc134100 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009e80 tx=0x7fb8f400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fbbe4edf276ab0f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:17.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.277+0000 7fb8f959a700 1 -- 172.21.15.3:0/3748085798 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f400e070 con 0x7fb8fc09e440 2023-10-03T10:13:17.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.277+0000 7fb8f959a700 1 -- 172.21.15.3:0/3748085798 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f400b980 con 0x7fb8fc09e440 2023-10-03T10:13:17.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.277+0000 7fb8f959a700 1 -- 172.21.15.3:0/3748085798 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f400bd30 con 0x7fb8fc09e440 2023-10-03T10:13:17.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.278+0000 7fb900cea700 1 -- 172.21.15.3:0/3748085798 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8fc09e440 msgr2=0x7fb8fc134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:17.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.278+0000 7fb900cea700 1 --2- 172.21.15.3:0/3748085798 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8fc09e440 0x7fb8fc134100 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009e80 tx=0x7fb8f400b330 comp rx=0 tx=0).stop 2023-10-03T10:13:17.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.278+0000 7fb900cea700 1 -- 172.21.15.3:0/3748085798 shutdown_connections 2023-10-03T10:13:17.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.278+0000 7fb900cea700 1 --2- 172.21.15.3:0/3748085798 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8fc09e440 0x7fb8fc134100 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.278+0000 7fb900cea700 1 --2- 172.21.15.3:0/3748085798 >> v2:172.21.15.3:3300/0 conn(0x7fb8fc09d6f0 0x7fb8fc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.278+0000 7fb900cea700 1 -- 172.21.15.3:0/3748085798 >> 172.21.15.3:0/3748085798 conn(0x7fb8fc066f50 msgr2=0x7fb8fc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:17.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.278+0000 7fb900cea700 1 -- 172.21.15.3:0/3748085798 shutdown_connections 2023-10-03T10:13:17.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.278+0000 7fb900cea700 1 -- 172.21.15.3:0/3748085798 wait complete. 2023-10-03T10:13:17.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.280+0000 7fb900cea700 1 Processor -- start 2023-10-03T10:13:17.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.280+0000 7fb900cea700 1 -- start start 2023-10-03T10:13:17.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.280+0000 7fb900cea700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb8fc09d6f0 0x7fb8fc1a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:17.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.280+0000 7fb900cea700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8fc09e000 0x7fb8fc1a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:17.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.280+0000 7fb900cea700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8fc09e440 0x7fb8fc1a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:17.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.280+0000 7fb900cea700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc1a81f0 con 0x7fb8fc09e440 2023-10-03T10:13:17.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.280+0000 7fb900cea700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc1a6b10 con 0x7fb8fc09e000 2023-10-03T10:13:17.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.280+0000 7fb900cea700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc1a86a0 con 0x7fb8fc09d6f0 2023-10-03T10:13:17.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.280+0000 7fb8fa59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb8fc09d6f0 0x7fb8fc1a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:17.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.280+0000 7fb8f9d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8fc09e000 0x7fb8fc1a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:17.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.280+0000 7fb8fa59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb8fc09d6f0 0x7fb8fc1a4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55468/0 (socket says 172.21.15.3:55468) 2023-10-03T10:13:17.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.280+0000 7fb8fa59c700 1 -- 172.21.15.3:0/3223818740 learned_addr learned my addr 172.21.15.3:0/3223818740 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:17.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.280+0000 7fb8f9d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8fc09e000 0x7fb8fc1a53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36736/0 (socket says 172.21.15.3:36736) 2023-10-03T10:13:17.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.281+0000 7fb8fad9d700 1 --2- 172.21.15.3:0/3223818740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8fc09e440 0x7fb8fc1a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:17.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.281+0000 7fb8f9d9b700 1 -- 172.21.15.3:0/3223818740 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb8fc09d6f0 msgr2=0x7fb8fc1a4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:17.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.281+0000 7fb8f9d9b700 1 --2- 172.21.15.3:0/3223818740 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb8fc09d6f0 0x7fb8fc1a4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.281+0000 7fb8f9d9b700 1 -- 172.21.15.3:0/3223818740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8fc09e440 msgr2=0x7fb8fc1a58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:17.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.281+0000 7fb8f9d9b700 1 --2- 172.21.15.3:0/3223818740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8fc09e440 0x7fb8fc1a58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.281+0000 7fb8f9d9b700 1 -- 172.21.15.3:0/3223818740 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f4009b30 con 0x7fb8fc09e000 2023-10-03T10:13:17.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.281+0000 7fb8f9d9b700 1 --2- 172.21.15.3:0/3223818740 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8fc09e000 0x7fb8fc1a53a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0000f60 tx=0x7fb8f000c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:17.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.281+0000 7fb8eaffd700 1 -- 172.21.15.3:0/3223818740 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f00052d0 con 0x7fb8fc09e000 2023-10-03T10:13:17.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.281+0000 7fb8eaffd700 1 -- 172.21.15.3:0/3223818740 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f0005430 con 0x7fb8fc09e000 2023-10-03T10:13:17.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.281+0000 7fb900cea700 1 -- 172.21.15.3:0/3223818740 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8fc0a1fb0 con 0x7fb8fc09e000 2023-10-03T10:13:17.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.281+0000 7fb8eaffd700 1 -- 172.21.15.3:0/3223818740 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f00038a0 con 0x7fb8fc09e000 2023-10-03T10:13:17.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.282+0000 7fb900cea700 1 -- 172.21.15.3:0/3223818740 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8fc1a8980 con 0x7fb8fc09e000 2023-10-03T10:13:17.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.283+0000 7fb900cea700 1 -- 172.21.15.3:0/3223818740 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8fc074ea0 con 0x7fb8fc09e000 2023-10-03T10:13:17.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.283+0000 7fb8eaffd700 1 -- 172.21.15.3:0/3223818740 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb8f0003a00 con 0x7fb8fc09e000 2023-10-03T10:13:17.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.283+0000 7fb8eaffd700 1 -- 172.21.15.3:0/3223818740 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb8f003e950 con 0x7fb8fc09e000 2023-10-03T10:13:17.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.292+0000 7fb8eaffd700 1 -- 172.21.15.3:0/3223818740 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb8f0003cb0 con 0x7fb8fc09e000 2023-10-03T10:13:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.430+0000 7f96cc689700 1 -- 172.21.15.3:0/827019201 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f96ac005590 con 0x7f96c4133870 2023-10-03T10:13:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.431+0000 7f96baffd700 1 -- 172.21.15.3:0/827019201 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f96c0017070 con 0x7f96c4133870 2023-10-03T10:13:17.431 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:17.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.435+0000 7f96cc689700 1 -- 172.21.15.3:0/827019201 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f96c4133870 msgr2=0x7f96c4119b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:17.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.435+0000 7f96cc689700 1 --2- 172.21.15.3:0/827019201 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f96c4133870 0x7f96c4119b20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f96c000c4e0 tx=0x7f96c000c8a0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.435+0000 7f96cc689700 1 -- 172.21.15.3:0/827019201 shutdown_connections 2023-10-03T10:13:17.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.435+0000 7f96cc689700 1 --2- 172.21.15.3:0/827019201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96c41390e0 0x7f96c411a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.435+0000 7f96cc689700 1 --2- 172.21.15.3:0/827019201 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f96c4133870 0x7f96c4119b20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.435+0000 7f96cc689700 1 --2- 172.21.15.3:0/827019201 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f96c41367c0 0x7f96c411a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.435+0000 7f96cc689700 1 -- 172.21.15.3:0/827019201 >> 172.21.15.3:0/827019201 conn(0x7f96c40a2610 msgr2=0x7f96c40709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:17.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.435+0000 7f96cc689700 1 -- 172.21.15.3:0/827019201 shutdown_connections 2023-10-03T10:13:17.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.435+0000 7f96cc689700 1 -- 172.21.15.3:0/827019201 wait complete. 2023-10-03T10:13:17.450 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:17.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.451+0000 7f732b5e0700 1 -- 172.21.15.3:0/2076966283 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f73241a72d0 con 0x7f732409f650 2023-10-03T10:13:17.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.452+0000 7f7319ffb700 1 -- 172.21.15.3:0/2076966283 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f73241a72d0 con 0x7f732409f650 2023-10-03T10:13:17.452 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:17.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.454+0000 7f732b5e0700 1 -- 172.21.15.3:0/2076966283 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f732409f650 msgr2=0x7f73241a4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:17.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.454+0000 7f732b5e0700 1 --2- 172.21.15.3:0/2076966283 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f732409f650 0x7f73241a4950 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f731c00ef30 tx=0x7f731c010330 comp rx=0 tx=0).stop 2023-10-03T10:13:17.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.454+0000 7f732b5e0700 1 -- 172.21.15.3:0/2076966283 shutdown_connections 2023-10-03T10:13:17.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.454+0000 7f732b5e0700 1 --2- 172.21.15.3:0/2076966283 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f732409f650 0x7f73241a4950 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.454+0000 7f732b5e0700 1 --2- 172.21.15.3:0/2076966283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f732409e930 0x7f73241a3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.454+0000 7f732b5e0700 1 --2- 172.21.15.3:0/2076966283 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f732409f240 0x7f73241a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.454+0000 7f732b5e0700 1 -- 172.21.15.3:0/2076966283 >> 172.21.15.3:0/2076966283 conn(0x7f73240a4210 msgr2=0x7f73241b8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:17.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.454+0000 7f732b5e0700 1 -- 172.21.15.3:0/2076966283 shutdown_connections 2023-10-03T10:13:17.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.454+0000 7f732b5e0700 1 -- 172.21.15.3:0/2076966283 wait complete. 2023-10-03T10:13:17.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.458+0000 7fe79a8b2700 1 -- 172.21.15.3:0/1263523123 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe79406b9e0 con 0x7fe79409e560 2023-10-03T10:13:17.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.458+0000 7fe790ff9700 1 -- 172.21.15.3:0/1263523123 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe78c011390 con 0x7fe79409e560 2023-10-03T10:13:17.458 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.460+0000 7fe79a8b2700 1 -- 172.21.15.3:0/1263523123 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe79409e560 msgr2=0x7fe7941ce680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.460+0000 7fe79a8b2700 1 --2- 172.21.15.3:0/1263523123 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe79409e560 0x7fe7941ce680 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe79406f8b0 tx=0x7fe78c004ad0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.461+0000 7fe79a8b2700 1 -- 172.21.15.3:0/1263523123 shutdown_connections 2023-10-03T10:13:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.461+0000 7fe79a8b2700 1 --2- 172.21.15.3:0/1263523123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7941341d0 0x7fe7941cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.461+0000 7fe79a8b2700 1 --2- 172.21.15.3:0/1263523123 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe79409e560 0x7fe7941ce680 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.461+0000 7fe79a8b2700 1 --2- 172.21.15.3:0/1263523123 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7941338c0 0x7fe7941cebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.461+0000 7fe79a8b2700 1 -- 172.21.15.3:0/1263523123 >> 172.21.15.3:0/1263523123 conn(0x7fe7940a2aa0 msgr2=0x7fe7940a0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.461+0000 7fe79a8b2700 1 -- 172.21.15.3:0/1263523123 shutdown_connections 2023-10-03T10:13:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.461+0000 7fe79a8b2700 1 -- 172.21.15.3:0/1263523123 wait complete. 2023-10-03T10:13:17.470 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:17.475 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:17.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.500+0000 7fb900cea700 1 -- 172.21.15.3:0/3223818740 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb8fc09f420 con 0x7fb8fc09e000 2023-10-03T10:13:17.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.501+0000 7fb8eaffd700 1 -- 172.21.15.3:0/3223818740 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb8f0017030 con 0x7fb8fc09e000 2023-10-03T10:13:17.501 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:17.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.503+0000 7fb900cea700 1 -- 172.21.15.3:0/3223818740 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8fc09e000 msgr2=0x7fb8fc1a53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:17.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.503+0000 7fb900cea700 1 --2- 172.21.15.3:0/3223818740 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8fc09e000 0x7fb8fc1a53a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0000f60 tx=0x7fb8f000c730 comp rx=0 tx=0).stop 2023-10-03T10:13:17.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.503+0000 7fb900cea700 1 -- 172.21.15.3:0/3223818740 shutdown_connections 2023-10-03T10:13:17.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.503+0000 7fb900cea700 1 --2- 172.21.15.3:0/3223818740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8fc09e440 0x7fb8fc1a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.503+0000 7fb900cea700 1 --2- 172.21.15.3:0/3223818740 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb8fc09d6f0 0x7fb8fc1a4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.503+0000 7fb900cea700 1 --2- 172.21.15.3:0/3223818740 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb8fc09e000 0x7fb8fc1a53a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:17.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.503+0000 7fb900cea700 1 -- 172.21.15.3:0/3223818740 >> 172.21.15.3:0/3223818740 conn(0x7fb8fc066f50 msgr2=0x7fb8fc1b9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:17.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.503+0000 7fb900cea700 1 -- 172.21.15.3:0/3223818740 shutdown_connections 2023-10-03T10:13:17.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:17.503+0000 7fb900cea700 1 -- 172.21.15.3:0/3223818740 wait complete. 2023-10-03T10:13:17.516 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:18.451 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:18.470 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:18.476 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:18.517 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:18.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.609+0000 7f82f7e61700 1 Processor -- start 2023-10-03T10:13:18.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.609+0000 7f82f7e61700 1 -- start start 2023-10-03T10:13:18.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.610+0000 7f82f7e61700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82f009e1d0 0x7f82f0138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:18.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.610+0000 7f82f7e61700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f82f0139170 0x7f82f013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:18.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.610+0000 7f82f7e61700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f009b120 con 0x7f82f009d6f0 2023-10-03T10:13:18.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.610+0000 7f82f7e61700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f006b460 con 0x7f82f0139170 2023-10-03T10:13:18.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.610+0000 7f82f7e61700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f82f009a7e0 con 0x7f82f009e1d0 2023-10-03T10:13:18.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.610+0000 7f82f63fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f82f0139170 0x7f82f013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:18.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.610+0000 7f82f63fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f82f0139170 0x7f82f013b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50618/0 (socket says 172.21.15.3:50618) 2023-10-03T10:13:18.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.610+0000 7f82f63fe700 1 -- 172.21.15.3:0/3808688811 learned_addr learned my addr 172.21.15.3:0/3808688811 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:18.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.610+0000 7f82f53fc700 1 --2- 172.21.15.3:0/3808688811 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82f009e1d0 0x7f82f0138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:18.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.611+0000 7f82f63fe700 1 -- 172.21.15.3:0/3808688811 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82f009e1d0 msgr2=0x7f82f0138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:18.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.611+0000 7f82f63fe700 1 --2- 172.21.15.3:0/3808688811 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82f009e1d0 0x7f82f0138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.611+0000 7f82f63fe700 1 -- 172.21.15.3:0/3808688811 >> v1:172.21.15.3:6789/0 conn(0x7f82f009d6f0 legacy=0x7f82f009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:18.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.611+0000 7f82f63fe700 1 -- 172.21.15.3:0/3808688811 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f009a500 con 0x7f82f0139170 2023-10-03T10:13:18.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.611+0000 7f82f63fe700 1 --2- 172.21.15.3:0/3808688811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f0139170 0x7f82f013b550 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009e50 tx=0x7f82e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5ce2231a94a93b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:18.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.611+0000 7f82f4bfb700 1 -- 172.21.15.3:0/3808688811 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e800e070 con 0x7f82f0139170 2023-10-03T10:13:18.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.611+0000 7f82f4bfb700 1 -- 172.21.15.3:0/3808688811 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e8003a20 con 0x7f82f0139170 2023-10-03T10:13:18.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.611+0000 7f82f4bfb700 1 -- 172.21.15.3:0/3808688811 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8003dd0 con 0x7f82f0139170 2023-10-03T10:13:18.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.612+0000 7f82f7e61700 1 -- 172.21.15.3:0/3808688811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f0139170 msgr2=0x7f82f013b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:18.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.612+0000 7f82f7e61700 1 --2- 172.21.15.3:0/3808688811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f0139170 0x7f82f013b550 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009e50 tx=0x7f82e800b330 comp rx=0 tx=0).stop 2023-10-03T10:13:18.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.612+0000 7f82f7e61700 1 -- 172.21.15.3:0/3808688811 shutdown_connections 2023-10-03T10:13:18.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.612+0000 7f82f7e61700 1 --2- 172.21.15.3:0/3808688811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f0139170 0x7f82f013b550 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.612+0000 7f82f7e61700 1 --2- 172.21.15.3:0/3808688811 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82f009e1d0 0x7f82f0138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.612+0000 7f82f7e61700 1 -- 172.21.15.3:0/3808688811 >> 172.21.15.3:0/3808688811 conn(0x7f82f0066f50 msgr2=0x7f82f009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:18.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.612+0000 7f82f7e61700 1 -- 172.21.15.3:0/3808688811 shutdown_connections 2023-10-03T10:13:18.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.612+0000 7f82f7e61700 1 -- 172.21.15.3:0/3808688811 wait complete. 2023-10-03T10:13:18.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.613+0000 7f82f7e61700 1 Processor -- start 2023-10-03T10:13:18.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.613+0000 7f82f7e61700 1 -- start start 2023-10-03T10:13:18.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.614+0000 7f82f7e61700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82f009d6f0 0x7f82f01215c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:18.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.614+0000 7f82f7e61700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f009e1d0 0x7f82f0121b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:18.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.614+0000 7f82f7e61700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f82f0139170 0x7f82f01c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:18.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.614+0000 7f82f7e61700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f01368d0 con 0x7f82f009e1d0 2023-10-03T10:13:18.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.614+0000 7f82f7e61700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f00a3040 con 0x7f82f0139170 2023-10-03T10:13:18.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.614+0000 7f82f7e61700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f82f009c710 con 0x7f82f009d6f0 2023-10-03T10:13:18.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.614+0000 7f82f53fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f009e1d0 0x7f82f0121b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:18.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.614+0000 7f82f53fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f009e1d0 0x7f82f0121b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50626/0 (socket says 172.21.15.3:50626) 2023-10-03T10:13:18.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.614+0000 7f82f5bfd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82f009d6f0 0x7f82f01215c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:18.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.614+0000 7f82f53fc700 1 -- 172.21.15.3:0/2381579011 learned_addr learned my addr 172.21.15.3:0/2381579011 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:18.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.614+0000 7f82f63fe700 1 --2- 172.21.15.3:0/2381579011 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f82f0139170 0x7f82f01c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:18.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.615+0000 7f82f53fc700 1 -- 172.21.15.3:0/2381579011 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82f009d6f0 msgr2=0x7f82f01215c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:18.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.615+0000 7f82f53fc700 1 --2- 172.21.15.3:0/2381579011 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82f009d6f0 0x7f82f01215c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.615+0000 7f82f53fc700 1 -- 172.21.15.3:0/2381579011 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f82f0139170 msgr2=0x7f82f01c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:18.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.615+0000 7f82f53fc700 1 --2- 172.21.15.3:0/2381579011 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f82f0139170 0x7f82f01c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.615+0000 7f82f53fc700 1 -- 172.21.15.3:0/2381579011 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e8009b30 con 0x7f82f009e1d0 2023-10-03T10:13:18.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.615+0000 7f82f63fe700 1 --2- 172.21.15.3:0/2381579011 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f82f0139170 0x7f82f01c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:18.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.615+0000 7f82f5bfd700 1 --2- 172.21.15.3:0/2381579011 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82f009d6f0 0x7f82f01215c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:18.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.615+0000 7f82f53fc700 1 --2- 172.21.15.3:0/2381579011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f009e1d0 0x7f82f0121b00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f82e000e9b0 tx=0x7f82e000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:18.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.615+0000 7f82e67fc700 1 -- 172.21.15.3:0/2381579011 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e0009710 con 0x7f82f009e1d0 2023-10-03T10:13:18.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.615+0000 7f82e67fc700 1 -- 172.21.15.3:0/2381579011 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e000c850 con 0x7f82f009e1d0 2023-10-03T10:13:18.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.615+0000 7f82e67fc700 1 -- 172.21.15.3:0/2381579011 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e000cbc0 con 0x7f82f009e1d0 2023-10-03T10:13:18.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.615+0000 7f82f7e61700 1 -- 172.21.15.3:0/2381579011 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f0121210 con 0x7f82f009e1d0 2023-10-03T10:13:18.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.615+0000 7f82f7e61700 1 -- 172.21.15.3:0/2381579011 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f0122040 con 0x7f82f009e1d0 2023-10-03T10:13:18.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.616+0000 7f82f7e61700 1 -- 172.21.15.3:0/2381579011 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f009e920 con 0x7f82f009e1d0 2023-10-03T10:13:18.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.616+0000 7f82e67fc700 1 -- 172.21.15.3:0/2381579011 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f82e000cd20 con 0x7f82f009e1d0 2023-10-03T10:13:18.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.617+0000 7f82e67fc700 1 -- 172.21.15.3:0/2381579011 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f82e003f830 con 0x7f82f009e1d0 2023-10-03T10:13:18.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.622+0000 7f82e67fc700 1 -- 172.21.15.3:0/2381579011 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f82e00232f0 con 0x7f82f009e1d0 2023-10-03T10:13:18.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.647+0000 7f88304d1700 1 Processor -- start 2023-10-03T10:13:18.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.647+0000 7f88304d1700 1 -- start start 2023-10-03T10:13:18.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.647+0000 7f4d75d31700 1 Processor -- start 2023-10-03T10:13:18.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.647+0000 7f4d75d31700 1 -- start start 2023-10-03T10:13:18.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.648+0000 7f88304d1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8828136820 0x7f8828138c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:18.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.648+0000 7f88304d1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8828139140 0x7f882813b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:18.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.648+0000 7f88304d1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f882806bc00 con 0x7f8828133a80 2023-10-03T10:13:18.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.648+0000 7f88304d1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f882806c570 con 0x7f8828139140 2023-10-03T10:13:18.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.648+0000 7f88304d1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f88280a0040 con 0x7f8828136820 2023-10-03T10:13:18.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.648+0000 7f882e26d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8828133a80 0x7f8828133e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59360/0 (socket says 172.21.15.3:59360) 2023-10-03T10:13:18.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.648+0000 7f882e26d700 1 -- 172.21.15.3:0/3100598448 learned_addr learned my addr 172.21.15.3:0/3100598448 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:18.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.648+0000 7f4d75d31700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d7009f410 0x7f4d70136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:18.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.648+0000 7f4d75d31700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d70136fe0 0x7f4d701393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:18.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.648+0000 7f4d75d31700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d70070940 con 0x7f4d7009e930 2023-10-03T10:13:18.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.648+0000 7f4d75d31700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d7006fd60 con 0x7f4d70136fe0 2023-10-03T10:13:18.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.648+0000 7f4d75d31700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7006fa80 con 0x7f4d7009f410 2023-10-03T10:13:18.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.648+0000 7f4d6ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d70136fe0 0x7f4d701393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:18.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.648+0000 7f4d6ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d70136fe0 0x7f4d701393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50628/0 (socket says 172.21.15.3:50628) 2023-10-03T10:13:18.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.648+0000 7f4d6ffff700 1 -- 172.21.15.3:0/2598213392 learned_addr learned my addr 172.21.15.3:0/2598213392 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:18.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.648+0000 7f4d6effd700 1 --2- 172.21.15.3:0/2598213392 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d7009f410 0x7f4d70136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.648+0000 7f882ea6e700 1 --2- 172.21.15.3:0/3100598448 >> v2:172.21.15.3:3300/0 conn(0x7f8828139140 0x7f882813b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.648+0000 7f882da6c700 1 --2- 172.21.15.3:0/3100598448 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8828136820 0x7f8828138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.649+0000 7f882ea6e700 1 -- 172.21.15.3:0/3100598448 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8828136820 msgr2=0x7f8828138c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.649+0000 7f882ea6e700 1 --2- 172.21.15.3:0/3100598448 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8828136820 0x7f8828138c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.649+0000 7f882ea6e700 1 -- 172.21.15.3:0/3100598448 >> v1:172.21.15.3:6789/0 conn(0x7f8828133a80 legacy=0x7f8828133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.649+0000 7f882ea6e700 1 -- 172.21.15.3:0/3100598448 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8828133860 con 0x7f8828139140 2023-10-03T10:13:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.649+0000 7f4d6ffff700 1 -- 172.21.15.3:0/2598213392 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d7009f410 msgr2=0x7f4d70136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.649+0000 7f4d6ffff700 1 --2- 172.21.15.3:0/2598213392 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d7009f410 0x7f4d70136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.649+0000 7f4d6ffff700 1 -- 172.21.15.3:0/2598213392 >> v1:172.21.15.3:6789/0 conn(0x7f4d7009e930 legacy=0x7f4d7009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.649+0000 7f4d6ffff700 1 -- 172.21.15.3:0/2598213392 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7009fb60 con 0x7f4d70136fe0 2023-10-03T10:13:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.649+0000 7f882ea6e700 1 --2- 172.21.15.3:0/3100598448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8828139140 0x7f882813b520 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8820009d90 tx=0x7f882000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd6dd403f54b6d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:18.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.649+0000 7f4d6ffff700 1 --2- 172.21.15.3:0/2598213392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d70136fe0 0x7f4d701393c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4d68009e50 tx=0x7f4d6800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eccd292def643655 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:18.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.649+0000 7f882d26b700 1 -- 172.21.15.3:0/3100598448 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882000e070 con 0x7f8828139140 2023-10-03T10:13:18.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.649+0000 7f882d26b700 1 -- 172.21.15.3:0/3100598448 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882000bb20 con 0x7f8828139140 2023-10-03T10:13:18.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.649+0000 7f882d26b700 1 -- 172.21.15.3:0/3100598448 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8820012430 con 0x7f8828139140 2023-10-03T10:13:18.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.649+0000 7f4d6e7fc700 1 -- 172.21.15.3:0/2598213392 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6800e070 con 0x7f4d70136fe0 2023-10-03T10:13:18.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.649+0000 7f4d6e7fc700 1 -- 172.21.15.3:0/2598213392 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d68003a20 con 0x7f4d70136fe0 2023-10-03T10:13:18.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.649+0000 7f4d6e7fc700 1 -- 172.21.15.3:0/2598213392 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d68003dd0 con 0x7f4d70136fe0 2023-10-03T10:13:18.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.650+0000 7f88304d1700 1 -- 172.21.15.3:0/3100598448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8828139140 msgr2=0x7f882813b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:18.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.650+0000 7f88304d1700 1 --2- 172.21.15.3:0/3100598448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8828139140 0x7f882813b520 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8820009d90 tx=0x7f882000b330 comp rx=0 tx=0).stop 2023-10-03T10:13:18.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.650+0000 7f4d75d31700 1 -- 172.21.15.3:0/2598213392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d70136fe0 msgr2=0x7f4d701393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:18.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.650+0000 7f4d75d31700 1 --2- 172.21.15.3:0/2598213392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d70136fe0 0x7f4d701393c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4d68009e50 tx=0x7f4d6800b330 comp rx=0 tx=0).stop 2023-10-03T10:13:18.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.650+0000 7f88304d1700 1 -- 172.21.15.3:0/3100598448 shutdown_connections 2023-10-03T10:13:18.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.650+0000 7f88304d1700 1 --2- 172.21.15.3:0/3100598448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8828139140 0x7f882813b520 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.650+0000 7f88304d1700 1 --2- 172.21.15.3:0/3100598448 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8828136820 0x7f8828138c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.650+0000 7f88304d1700 1 -- 172.21.15.3:0/3100598448 >> 172.21.15.3:0/3100598448 conn(0x7f88280a4210 msgr2=0x7f88280a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:18.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.650+0000 7f88304d1700 1 -- 172.21.15.3:0/3100598448 shutdown_connections 2023-10-03T10:13:18.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.650+0000 7f4d75d31700 1 -- 172.21.15.3:0/2598213392 shutdown_connections 2023-10-03T10:13:18.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.650+0000 7f4d75d31700 1 --2- 172.21.15.3:0/2598213392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d70136fe0 0x7f4d701393c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.650+0000 7f4d75d31700 1 --2- 172.21.15.3:0/2598213392 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d7009f410 0x7f4d70136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.650+0000 7f4d75d31700 1 -- 172.21.15.3:0/2598213392 >> 172.21.15.3:0/2598213392 conn(0x7f4d700a4210 msgr2=0x7f4d700a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:18.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.650+0000 7f88304d1700 1 -- 172.21.15.3:0/3100598448 wait complete. 2023-10-03T10:13:18.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.650+0000 7f4d75d31700 1 -- 172.21.15.3:0/2598213392 shutdown_connections 2023-10-03T10:13:18.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.651+0000 7f4d75d31700 1 -- 172.21.15.3:0/2598213392 wait complete. 2023-10-03T10:13:18.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.651+0000 7f88304d1700 1 Processor -- start 2023-10-03T10:13:18.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.651+0000 7f4d75d31700 1 Processor -- start 2023-10-03T10:13:18.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.652+0000 7f88304d1700 1 -- start start 2023-10-03T10:13:18.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.652+0000 7f4d75d31700 1 -- start start 2023-10-03T10:13:18.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.652+0000 7f88304d1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8828133a80 0x7f88281c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:18.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.652+0000 7f88304d1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8828136820 0x7f88281c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:18.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.652+0000 7f88304d1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8828139140 0x7f882809b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:18.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.652+0000 7f88304d1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882809e200 con 0x7f8828136820 2023-10-03T10:13:18.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.652+0000 7f88304d1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f882809c700 con 0x7f8828133a80 2023-10-03T10:13:18.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.652+0000 7f88304d1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f882809c4d0 con 0x7f8828139140 2023-10-03T10:13:18.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.652+0000 7f882ea6e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8828139140 0x7f882809b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:18.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.652+0000 7f882e26d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8828133a80 0x7f88281c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:18.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.652+0000 7f4d75d31700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d7009e930 0x7f4d70131ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:18.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.652+0000 7f4d75d31700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d7009f410 0x7f4d70132000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:18.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.652+0000 7f4d75d31700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d70136fe0 0x7f4d70132540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:18.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.652+0000 7f4d75d31700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d70135540 con 0x7f4d7009f410 2023-10-03T10:13:18.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.652+0000 7f4d75d31700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d70134880 con 0x7f4d70136fe0 2023-10-03T10:13:18.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.652+0000 7f4d75d31700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7009b220 con 0x7f4d7009e930 2023-10-03T10:13:18.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.652+0000 7f882ea6e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8828139140 0x7f882809b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55516/0 (socket says 172.21.15.3:55516) 2023-10-03T10:13:18.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.652+0000 7f882ea6e700 1 -- 172.21.15.3:0/4218474241 learned_addr learned my addr 172.21.15.3:0/4218474241 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:18.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.653+0000 7f882da6c700 1 --2- 172.21.15.3:0/4218474241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8828136820 0x7f88281c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:18.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.653+0000 7f882ea6e700 1 -- 172.21.15.3:0/4218474241 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8828133a80 msgr2=0x7f88281c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:18.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.653+0000 7f882ea6e700 1 --2- 172.21.15.3:0/4218474241 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8828133a80 0x7f88281c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.653+0000 7f882ea6e700 1 -- 172.21.15.3:0/4218474241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8828136820 msgr2=0x7f88281c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:18.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.653+0000 7f882ea6e700 1 --2- 172.21.15.3:0/4218474241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8828136820 0x7f88281c8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.653+0000 7f882ea6e700 1 -- 172.21.15.3:0/4218474241 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8820009a70 con 0x7f8828139140 2023-10-03T10:13:18.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.653+0000 7f882da6c700 1 --2- 172.21.15.3:0/4218474241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8828136820 0x7f88281c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:13:18.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.653+0000 7f882ea6e700 1 --2- 172.21.15.3:0/4218474241 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8828139140 0x7f882809b430 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8820013000 tx=0x7f8820004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:18.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.653+0000 7f881effd700 1 -- 172.21.15.3:0/4218474241 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8820004830 con 0x7f8828139140 2023-10-03T10:13:18.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.653+0000 7f881effd700 1 -- 172.21.15.3:0/4218474241 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8820020070 con 0x7f8828139140 2023-10-03T10:13:18.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.653+0000 7f881effd700 1 -- 172.21.15.3:0/4218474241 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88200276c0 con 0x7f8828139140 2023-10-03T10:13:18.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.653+0000 7f88304d1700 1 -- 172.21.15.3:0/4218474241 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f882809ca80 con 0x7f8828139140 2023-10-03T10:13:18.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.654+0000 7f4d6ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d70136fe0 0x7f4d70132540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:18.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.654+0000 7f4d6ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d70136fe0 0x7f4d70132540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36760/0 (socket says 172.21.15.3:36760) 2023-10-03T10:13:18.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.654+0000 7f4d6ffff700 1 -- 172.21.15.3:0/1320993763 learned_addr learned my addr 172.21.15.3:0/1320993763 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:18.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.653+0000 7f88304d1700 1 -- 172.21.15.3:0/4218474241 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88281c85b0 con 0x7f8828139140 2023-10-03T10:13:18.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.654+0000 7f4d6f7fe700 1 --2- 172.21.15.3:0/1320993763 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d7009e930 0x7f4d70131ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:18.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.654+0000 7f4d6effd700 1 --2- 172.21.15.3:0/1320993763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d7009f410 0x7f4d70132000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:18.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.654+0000 7f88304d1700 1 -- 172.21.15.3:0/4218474241 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f882809e060 con 0x7f8828139140 2023-10-03T10:13:18.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.655+0000 7f4d6f7fe700 1 -- 172.21.15.3:0/1320993763 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d70136fe0 msgr2=0x7f4d70132540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:18.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.655+0000 7f4d6f7fe700 1 --2- 172.21.15.3:0/1320993763 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d70136fe0 0x7f4d70132540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.655+0000 7f4d6f7fe700 1 -- 172.21.15.3:0/1320993763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d7009f410 msgr2=0x7f4d70132000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:18.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.655+0000 7f4d6f7fe700 1 --2- 172.21.15.3:0/1320993763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d7009f410 0x7f4d70132000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.655+0000 7f4d6f7fe700 1 -- 172.21.15.3:0/1320993763 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d68009b30 con 0x7f4d7009e930 2023-10-03T10:13:18.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.655+0000 7f4d6f7fe700 1 --2- 172.21.15.3:0/1320993763 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d7009e930 0x7f4d70131ac0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000c3e0 tx=0x7f4d6000c7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:18.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.655+0000 7f881effd700 1 -- 172.21.15.3:0/4218474241 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8820012430 con 0x7f8828139140 2023-10-03T10:13:18.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.655+0000 7f881effd700 1 -- 172.21.15.3:0/4218474241 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8820022070 con 0x7f8828139140 2023-10-03T10:13:18.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.656+0000 7f4d6effd700 1 --2- 172.21.15.3:0/1320993763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d7009f410 0x7f4d70132000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:13:18.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.660+0000 7f4d57fff700 1 -- 172.21.15.3:0/1320993763 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d600054e0 con 0x7f4d7009e930 2023-10-03T10:13:18.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.660+0000 7f4d57fff700 1 -- 172.21.15.3:0/1320993763 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d60003680 con 0x7f4d7009e930 2023-10-03T10:13:18.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.660+0000 7f4d57fff700 1 -- 172.21.15.3:0/1320993763 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d60003a30 con 0x7f4d7009e930 2023-10-03T10:13:18.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.660+0000 7f881effd700 1 -- 172.21.15.3:0/4218474241 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8820012e40 con 0x7f8828139140 2023-10-03T10:13:18.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.661+0000 7f4d75d31700 1 -- 172.21.15.3:0/1320993763 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d70133e90 con 0x7f4d7009e930 2023-10-03T10:13:18.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.661+0000 7f4d75d31700 1 -- 172.21.15.3:0/1320993763 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d70134fa0 con 0x7f4d7009e930 2023-10-03T10:13:18.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.662+0000 7f4d75d31700 1 -- 172.21.15.3:0/1320993763 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d70136120 con 0x7f4d7009e930 2023-10-03T10:13:18.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.662+0000 7f4d57fff700 1 -- 172.21.15.3:0/1320993763 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4d60003b90 con 0x7f4d7009e930 2023-10-03T10:13:18.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.662+0000 7f4d57fff700 1 -- 172.21.15.3:0/1320993763 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4d6003ec70 con 0x7f4d7009e930 2023-10-03T10:13:18.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.667+0000 7f4d57fff700 1 -- 172.21.15.3:0/1320993763 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4d60017080 con 0x7f4d7009e930 2023-10-03T10:13:18.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.692+0000 7fe248d52700 1 Processor -- start 2023-10-03T10:13:18.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.692+0000 7fe248d52700 1 -- start start 2023-10-03T10:13:18.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.693+0000 7fe248d52700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe24409e1d0 0x7fe244138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:18.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.693+0000 7fe248d52700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe244139170 0x7fe24413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:18.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.693+0000 7fe248d52700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe24409b120 con 0x7fe24409d6f0 2023-10-03T10:13:18.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.693+0000 7fe248d52700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe24406b460 con 0x7fe24409e1d0 2023-10-03T10:13:18.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.693+0000 7fe248d52700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe24409a7e0 con 0x7fe244139170 2023-10-03T10:13:18.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.693+0000 7fe242d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe244139170 0x7fe24413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:18.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.693+0000 7fe241d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe24409e1d0 0x7fe244138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:18.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.693+0000 7fe242d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe244139170 0x7fe24413b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55546/0 (socket says 172.21.15.3:55546) 2023-10-03T10:13:18.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.693+0000 7fe242d9d700 1 -- 172.21.15.3:0/2326475664 learned_addr learned my addr 172.21.15.3:0/2326475664 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:18.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.693+0000 7fe242d9d700 1 -- 172.21.15.3:0/2326475664 >> v2:172.21.15.3:3300/0 conn(0x7fe24409e1d0 msgr2=0x7fe244138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:18.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.693+0000 7fe242d9d700 1 --2- 172.21.15.3:0/2326475664 >> v2:172.21.15.3:3300/0 conn(0x7fe24409e1d0 0x7fe244138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.693+0000 7fe242d9d700 1 -- 172.21.15.3:0/2326475664 >> v1:172.21.15.3:6789/0 conn(0x7fe24409d6f0 legacy=0x7fe24409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:18.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.693+0000 7fe242d9d700 1 -- 172.21.15.3:0/2326475664 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24409a500 con 0x7fe244139170 2023-10-03T10:13:18.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.694+0000 7fe242d9d700 1 --2- 172.21.15.3:0/2326475664 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe244139170 0x7fe24413b550 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe23c009e80 tx=0x7fe23c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fe660161bc5b991 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:18.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.694+0000 7fe24159a700 1 -- 172.21.15.3:0/2326475664 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c00e070 con 0x7fe244139170 2023-10-03T10:13:18.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.694+0000 7fe24159a700 1 -- 172.21.15.3:0/2326475664 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23c00b8d0 con 0x7fe244139170 2023-10-03T10:13:18.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.694+0000 7fe24159a700 1 -- 172.21.15.3:0/2326475664 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c00bc80 con 0x7fe244139170 2023-10-03T10:13:18.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.694+0000 7fe248d52700 1 -- 172.21.15.3:0/2326475664 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe244139170 msgr2=0x7fe24413b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:18.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.694+0000 7fe248d52700 1 --2- 172.21.15.3:0/2326475664 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe244139170 0x7fe24413b550 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe23c009e80 tx=0x7fe23c00b330 comp rx=0 tx=0).stop 2023-10-03T10:13:18.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.695+0000 7fe248d52700 1 -- 172.21.15.3:0/2326475664 shutdown_connections 2023-10-03T10:13:18.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.695+0000 7fe248d52700 1 --2- 172.21.15.3:0/2326475664 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe244139170 0x7fe24413b550 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.695+0000 7fe248d52700 1 --2- 172.21.15.3:0/2326475664 >> v2:172.21.15.3:3300/0 conn(0x7fe24409e1d0 0x7fe244138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.695+0000 7fe248d52700 1 -- 172.21.15.3:0/2326475664 >> 172.21.15.3:0/2326475664 conn(0x7fe244066f50 msgr2=0x7fe24409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:18.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.695+0000 7fe248d52700 1 -- 172.21.15.3:0/2326475664 shutdown_connections 2023-10-03T10:13:18.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.695+0000 7fe248d52700 1 -- 172.21.15.3:0/2326475664 wait complete. 2023-10-03T10:13:18.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.696+0000 7fe248d52700 1 Processor -- start 2023-10-03T10:13:18.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.696+0000 7fe248d52700 1 -- start start 2023-10-03T10:13:18.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.696+0000 7fe248d52700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe24409d6f0 0x7fe2441c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:18.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.696+0000 7fe248d52700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe24409e1d0 0x7fe2441c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:18.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.696+0000 7fe248d52700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe244139170 0x7fe24411e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:18.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.696+0000 7fe248d52700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe244122110 con 0x7fe24409d6f0 2023-10-03T10:13:18.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.696+0000 7fe248d52700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe244120400 con 0x7fe244139170 2023-10-03T10:13:18.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.696+0000 7fe248d52700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe244121d70 con 0x7fe24409e1d0 2023-10-03T10:13:18.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.697+0000 7fe24259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe24409d6f0 0x7fe2441c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:18.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.697+0000 7fe24259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe24409d6f0 0x7fe2441c84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50680/0 (socket says 172.21.15.3:50680) 2023-10-03T10:13:18.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.697+0000 7fe24259c700 1 -- 172.21.15.3:0/829758939 learned_addr learned my addr 172.21.15.3:0/829758939 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:18.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.697+0000 7fe242d9d700 1 --2- 172.21.15.3:0/829758939 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe244139170 0x7fe24411e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:18.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.697+0000 7fe241d9b700 1 --2- 172.21.15.3:0/829758939 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe24409e1d0 0x7fe2441c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:18.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.697+0000 7fe24259c700 1 -- 172.21.15.3:0/829758939 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe24409e1d0 msgr2=0x7fe2441c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:18.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.697+0000 7fe24259c700 1 --2- 172.21.15.3:0/829758939 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe24409e1d0 0x7fe2441c8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.697+0000 7fe24259c700 1 -- 172.21.15.3:0/829758939 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe244139170 msgr2=0x7fe24411e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:18.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.697+0000 7fe24259c700 1 --2- 172.21.15.3:0/829758939 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe244139170 0x7fe24411e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.697+0000 7fe24259c700 1 -- 172.21.15.3:0/829758939 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe23c009b30 con 0x7fe24409d6f0 2023-10-03T10:13:18.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.697+0000 7fe24259c700 1 --2- 172.21.15.3:0/829758939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe24409d6f0 0x7fe2441c84d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe23400c4e0 tx=0x7fe23400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:18.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.697+0000 7fe232ffd700 1 -- 172.21.15.3:0/829758939 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2340052d0 con 0x7fe24409d6f0 2023-10-03T10:13:18.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.697+0000 7fe232ffd700 1 -- 172.21.15.3:0/829758939 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe234005430 con 0x7fe24409d6f0 2023-10-03T10:13:18.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.697+0000 7fe232ffd700 1 -- 172.21.15.3:0/829758939 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2340038c0 con 0x7fe24409d6f0 2023-10-03T10:13:18.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.697+0000 7fe248d52700 1 -- 172.21.15.3:0/829758939 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe24409c710 con 0x7fe24409d6f0 2023-10-03T10:13:18.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.698+0000 7fe248d52700 1 -- 172.21.15.3:0/829758939 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe244122310 con 0x7fe24409d6f0 2023-10-03T10:13:18.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.698+0000 7fe232ffd700 1 -- 172.21.15.3:0/829758939 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe234003a20 con 0x7fe24409d6f0 2023-10-03T10:13:18.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.699+0000 7fe232ffd700 1 -- 172.21.15.3:0/829758939 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe23403eb70 con 0x7fe24409d6f0 2023-10-03T10:13:18.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.699+0000 7fe248d52700 1 -- 172.21.15.3:0/829758939 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe244074ea0 con 0x7fe24409d6f0 2023-10-03T10:13:18.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.709+0000 7fe232ffd700 1 -- 172.21.15.3:0/829758939 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe234017020 con 0x7fe24409d6f0 2023-10-03T10:13:18.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.840+0000 7f82f7e61700 1 -- 172.21.15.3:0/2381579011 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f82f00a2c60 con 0x7f82f009e1d0 2023-10-03T10:13:18.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.840+0000 7f82e67fc700 1 -- 172.21.15.3:0/2381579011 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f82e0021030 con 0x7f82f009e1d0 2023-10-03T10:13:18.841 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:18.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.843+0000 7f82f7e61700 1 -- 172.21.15.3:0/2381579011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f009e1d0 msgr2=0x7f82f0121b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:18.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.843+0000 7f82f7e61700 1 --2- 172.21.15.3:0/2381579011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f009e1d0 0x7f82f0121b00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f82e000e9b0 tx=0x7f82e000ed70 comp rx=0 tx=0).stop 2023-10-03T10:13:18.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.843+0000 7f82f7e61700 1 -- 172.21.15.3:0/2381579011 shutdown_connections 2023-10-03T10:13:18.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.843+0000 7f82f7e61700 1 --2- 172.21.15.3:0/2381579011 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f82f0139170 0x7f82f01c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.843+0000 7f82f7e61700 1 --2- 172.21.15.3:0/2381579011 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82f009d6f0 0x7f82f01215c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.843+0000 7f82f7e61700 1 --2- 172.21.15.3:0/2381579011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f009e1d0 0x7f82f0121b00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.843+0000 7f82f7e61700 1 -- 172.21.15.3:0/2381579011 >> 172.21.15.3:0/2381579011 conn(0x7f82f0066f50 msgr2=0x7f82f0070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:18.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.843+0000 7f82f7e61700 1 -- 172.21.15.3:0/2381579011 shutdown_connections 2023-10-03T10:13:18.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.843+0000 7f82f7e61700 1 -- 172.21.15.3:0/2381579011 wait complete. 2023-10-03T10:13:18.858 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:18.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.869+0000 7f88304d1700 1 -- 172.21.15.3:0/4218474241 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f882809e060 con 0x7f8828139140 2023-10-03T10:13:18.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.870+0000 7f881effd700 1 -- 172.21.15.3:0/4218474241 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f882809e060 con 0x7f8828139140 2023-10-03T10:13:18.870 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:18.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.872+0000 7f88304d1700 1 -- 172.21.15.3:0/4218474241 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8828139140 msgr2=0x7f882809b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:18.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.872+0000 7f88304d1700 1 --2- 172.21.15.3:0/4218474241 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8828139140 0x7f882809b430 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8820013000 tx=0x7f8820004ad0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.872+0000 7f88304d1700 1 -- 172.21.15.3:0/4218474241 shutdown_connections 2023-10-03T10:13:18.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.872+0000 7f88304d1700 1 --2- 172.21.15.3:0/4218474241 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8828139140 0x7f882809b430 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.872+0000 7f88304d1700 1 --2- 172.21.15.3:0/4218474241 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8828133a80 0x7f88281c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.872+0000 7f88304d1700 1 --2- 172.21.15.3:0/4218474241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8828136820 0x7f88281c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.872+0000 7f88304d1700 1 -- 172.21.15.3:0/4218474241 >> 172.21.15.3:0/4218474241 conn(0x7f88280a4210 msgr2=0x7f88280a0920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:18.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.872+0000 7f88304d1700 1 -- 172.21.15.3:0/4218474241 shutdown_connections 2023-10-03T10:13:18.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.872+0000 7f88304d1700 1 -- 172.21.15.3:0/4218474241 wait complete. 2023-10-03T10:13:18.886 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:18.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.940+0000 7f4d75d31700 1 -- 172.21.15.3:0/1320993763 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4d70074ea0 con 0x7f4d7009e930 2023-10-03T10:13:18.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.940+0000 7f4d57fff700 1 -- 172.21.15.3:0/1320993763 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4d60011930 con 0x7f4d7009e930 2023-10-03T10:13:18.941 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:18.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.943+0000 7f4d75d31700 1 -- 172.21.15.3:0/1320993763 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d7009e930 msgr2=0x7f4d70131ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:18.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.943+0000 7f4d75d31700 1 --2- 172.21.15.3:0/1320993763 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d7009e930 0x7f4d70131ac0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000c3e0 tx=0x7f4d6000c7a0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.943+0000 7f4d75d31700 1 -- 172.21.15.3:0/1320993763 shutdown_connections 2023-10-03T10:13:18.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.943+0000 7f4d75d31700 1 --2- 172.21.15.3:0/1320993763 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d70136fe0 0x7f4d70132540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.943+0000 7f4d75d31700 1 --2- 172.21.15.3:0/1320993763 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d7009e930 0x7f4d70131ac0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.943+0000 7f4d75d31700 1 --2- 172.21.15.3:0/1320993763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d7009f410 0x7f4d70132000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.944+0000 7f4d75d31700 1 -- 172.21.15.3:0/1320993763 >> 172.21.15.3:0/1320993763 conn(0x7f4d700a4210 msgr2=0x7f4d700a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:18.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.944+0000 7f4d75d31700 1 -- 172.21.15.3:0/1320993763 shutdown_connections 2023-10-03T10:13:18.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.944+0000 7f4d75d31700 1 -- 172.21.15.3:0/1320993763 wait complete. 2023-10-03T10:13:18.963 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:18.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.983+0000 7fe248d52700 1 -- 172.21.15.3:0/829758939 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe24409f420 con 0x7fe24409d6f0 2023-10-03T10:13:18.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.984+0000 7fe232ffd700 1 -- 172.21.15.3:0/829758939 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe234019e60 con 0x7fe24409d6f0 2023-10-03T10:13:18.984 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:18.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.986+0000 7fe248d52700 1 -- 172.21.15.3:0/829758939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe24409d6f0 msgr2=0x7fe2441c84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:18.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.986+0000 7fe248d52700 1 --2- 172.21.15.3:0/829758939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe24409d6f0 0x7fe2441c84d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe23400c4e0 tx=0x7fe23400c7f0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.986+0000 7fe248d52700 1 -- 172.21.15.3:0/829758939 shutdown_connections 2023-10-03T10:13:18.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.986+0000 7fe248d52700 1 --2- 172.21.15.3:0/829758939 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe244139170 0x7fe24411e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.986+0000 7fe248d52700 1 --2- 172.21.15.3:0/829758939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe24409d6f0 0x7fe2441c84d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.986+0000 7fe248d52700 1 --2- 172.21.15.3:0/829758939 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe24409e1d0 0x7fe2441c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:18.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.986+0000 7fe248d52700 1 -- 172.21.15.3:0/829758939 >> 172.21.15.3:0/829758939 conn(0x7fe244066f50 msgr2=0x7fe2440a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:18.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.986+0000 7fe248d52700 1 -- 172.21.15.3:0/829758939 shutdown_connections 2023-10-03T10:13:18.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:18.986+0000 7fe248d52700 1 -- 172.21.15.3:0/829758939 wait complete. 2023-10-03T10:13:18.999 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:19.859 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:19.887 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:19.963 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:20.000 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:20.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.014+0000 7f7d00b8e700 1 Processor -- start 2023-10-03T10:13:20.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.014+0000 7f7d00b8e700 1 -- start start 2023-10-03T10:13:20.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.015+0000 7f7d00b8e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cfc09d6d0 0x7f7cfc09daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:20.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.015+0000 7f7d00b8e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cfc133cd0 0x7f7cfc13ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:20.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.015+0000 7f7d00b8e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cfc09a570 con 0x7f7cfc09dfe0 2023-10-03T10:13:20.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.015+0000 7f7d00b8e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc09b1c0 con 0x7f7cfc133cd0 2023-10-03T10:13:20.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.015+0000 7f7d00b8e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc09f240 con 0x7f7cfc09d6d0 2023-10-03T10:13:20.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.015+0000 7f7cf9d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7cfc09dfe0 0x7f7cfc1335c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59398/0 (socket says 172.21.15.3:59398) 2023-10-03T10:13:20.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.015+0000 7f7cf9d9b700 1 -- 172.21.15.3:0/2188840521 learned_addr learned my addr 172.21.15.3:0/2188840521 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:20.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.015+0000 7f7cfa59c700 1 --2- 172.21.15.3:0/2188840521 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cfc09d6d0 0x7f7cfc09daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:20.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.015+0000 7f7cf959a700 1 -- 172.21.15.3:0/2188840521 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 766466804 0 0) 0x7f7cfc09a570 con 0x7f7cfc09dfe0 2023-10-03T10:13:20.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.015+0000 7f7cf959a700 1 -- 172.21.15.3:0/2188840521 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ce40035b0 con 0x7f7cfc09dfe0 2023-10-03T10:13:20.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.016+0000 7f7cfad9d700 1 --2- 172.21.15.3:0/2188840521 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cfc133cd0 0x7f7cfc13ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:20.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.016+0000 7f7cf959a700 1 -- 172.21.15.3:0/2188840521 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2705899105 0 0) 0x7f7ce40035b0 con 0x7f7cfc09dfe0 2023-10-03T10:13:20.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.016+0000 7f7cf959a700 1 -- 172.21.15.3:0/2188840521 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cfc09d6d0 msgr2=0x7f7cfc09daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:20.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.016+0000 7f7cf959a700 1 --2- 172.21.15.3:0/2188840521 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cfc09d6d0 0x7f7cfc09daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.016+0000 7f7cf959a700 1 -- 172.21.15.3:0/2188840521 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cfc133cd0 msgr2=0x7f7cfc13ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:20.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.016+0000 7f7cf959a700 1 --2- 172.21.15.3:0/2188840521 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cfc133cd0 0x7f7cfc13ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.016+0000 7f7cf959a700 1 -- 172.21.15.3:0/2188840521 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc09a2d0 con 0x7f7cfc09dfe0 2023-10-03T10:13:20.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.016+0000 7f7cf959a700 1 -- 172.21.15.3:0/2188840521 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1156569835 0 0) 0x7f7cf0001670 con 0x7f7cfc09dfe0 2023-10-03T10:13:20.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.016+0000 7f7cf959a700 1 -- 172.21.15.3:0/2188840521 >> v1:172.21.15.3:6789/0 conn(0x7f7cfc09dfe0 legacy=0x7f7cfc1335c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:20.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.016+0000 7f7cf959a700 1 --2- 172.21.15.3:0/2188840521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ce4003b10 0x7f7ce4003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:20.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.016+0000 7f7cf959a700 1 --2- 172.21.15.3:0/2188840521 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cfc09d6d0 0x7f7ce4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:20.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.016+0000 7f7cf959a700 1 --2- 172.21.15.3:0/2188840521 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cfc133cd0 0x7f7ce4004d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:20.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.016+0000 7f7cf959a700 1 -- 172.21.15.3:0/2188840521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc09b1c0 con 0x7f7ce4003b10 2023-10-03T10:13:20.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.016+0000 7f7cf959a700 1 -- 172.21.15.3:0/2188840521 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc09f240 con 0x7f7cfc133cd0 2023-10-03T10:13:20.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.016+0000 7f7cf959a700 1 -- 172.21.15.3:0/2188840521 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce4005420 con 0x7f7cfc09d6d0 2023-10-03T10:13:20.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.016+0000 7f7cf9d9b700 1 --2- 172.21.15.3:0/2188840521 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cfc09d6d0 0x7f7ce4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:20.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.016+0000 7f7cf9d9b700 1 -- 172.21.15.3:0/2188840521 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cfc133cd0 msgr2=0x7f7ce4004d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:20.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.016+0000 7f7cf9d9b700 1 --2- 172.21.15.3:0/2188840521 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cfc133cd0 0x7f7ce4004d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.016+0000 7f7cf9d9b700 1 -- 172.21.15.3:0/2188840521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ce4003b10 msgr2=0x7f7ce4003f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-03T10:13:20.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.016+0000 7f7cf9d9b700 1 --2- 172.21.15.3:0/2188840521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ce4003b10 0x7f7ce4003f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.016+0000 7f7cf9d9b700 1 -- 172.21.15.3:0/2188840521 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce40035b0 con 0x7f7cfc09d6d0 2023-10-03T10:13:20.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.017+0000 7f7cf9d9b700 1 --2- 172.21.15.3:0/2188840521 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cfc09d6d0 0x7f7ce4004850 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7cf000c160 tx=0x7f7ce4005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7cb3c8600bbd0a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:20.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.017+0000 7f7cf959a700 1 -- 172.21.15.3:0/2188840521 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0007b30 con 0x7f7cfc09d6d0 2023-10-03T10:13:20.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.017+0000 7f7cf959a700 1 -- 172.21.15.3:0/2188840521 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf000cce0 con 0x7f7cfc09d6d0 2023-10-03T10:13:20.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.017+0000 7f7cf959a700 1 -- 172.21.15.3:0/2188840521 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0005ba0 con 0x7f7cfc09d6d0 2023-10-03T10:13:20.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.018+0000 7f7d00b8e700 1 -- 172.21.15.3:0/2188840521 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cfc09d6d0 msgr2=0x7f7ce4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:20.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.018+0000 7f7d00b8e700 1 --2- 172.21.15.3:0/2188840521 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cfc09d6d0 0x7f7ce4004850 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7cf000c160 tx=0x7f7ce4005880 comp rx=0 tx=0).stop 2023-10-03T10:13:20.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.018+0000 7f7d00b8e700 1 -- 172.21.15.3:0/2188840521 shutdown_connections 2023-10-03T10:13:20.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.018+0000 7f7d00b8e700 1 --2- 172.21.15.3:0/2188840521 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cfc133cd0 0x7f7ce4004d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.018+0000 7f7d00b8e700 1 --2- 172.21.15.3:0/2188840521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ce4003b10 0x7f7ce4003f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.018+0000 7f7d00b8e700 1 --2- 172.21.15.3:0/2188840521 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cfc09d6d0 0x7f7ce4004850 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.018+0000 7f7d00b8e700 1 -- 172.21.15.3:0/2188840521 >> 172.21.15.3:0/2188840521 conn(0x7f7cfc099c20 msgr2=0x7f7cfc0a2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:20.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.018+0000 7f7d00b8e700 1 -- 172.21.15.3:0/2188840521 shutdown_connections 2023-10-03T10:13:20.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.018+0000 7f7d00b8e700 1 -- 172.21.15.3:0/2188840521 wait complete. 2023-10-03T10:13:20.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.019+0000 7f7d00b8e700 1 Processor -- start 2023-10-03T10:13:20.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.020+0000 7f7d00b8e700 1 -- start start 2023-10-03T10:13:20.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.020+0000 7f7d00b8e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ce4003b10 0x7f7cfc1c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:20.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.020+0000 7f7d00b8e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cfc09d6d0 0x7f7cfc1c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:20.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.020+0000 7f7d00b8e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cfc09dfe0 0x7f7cfc1c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:20.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.020+0000 7f7d00b8e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc120cd0 con 0x7f7cfc09d6d0 2023-10-03T10:13:20.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.020+0000 7f7d00b8e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc11f360 con 0x7f7ce4003b10 2023-10-03T10:13:20.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.020+0000 7f7d00b8e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc121350 con 0x7f7cfc09dfe0 2023-10-03T10:13:20.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.020+0000 7f7cf9d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cfc09d6d0 0x7f7cfc1c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:20.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.020+0000 7f7cf9d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cfc09d6d0 0x7f7cfc1c82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50696/0 (socket says 172.21.15.3:50696) 2023-10-03T10:13:20.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.020+0000 7f7cf9d9b700 1 -- 172.21.15.3:0/670830393 learned_addr learned my addr 172.21.15.3:0/670830393 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:20.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.020+0000 7f7cfad9d700 1 --2- 172.21.15.3:0/670830393 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cfc09dfe0 0x7f7cfc1c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:20.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.021+0000 7f7cfa59c700 1 --2- 172.21.15.3:0/670830393 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ce4003b10 0x7f7cfc1c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:20.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.021+0000 7f7cf9d9b700 1 -- 172.21.15.3:0/670830393 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cfc09dfe0 msgr2=0x7f7cfc1c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:20.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.021+0000 7f7cf9d9b700 1 --2- 172.21.15.3:0/670830393 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cfc09dfe0 0x7f7cfc1c8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.021+0000 7f7cf9d9b700 1 -- 172.21.15.3:0/670830393 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ce4003b10 msgr2=0x7f7cfc1c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:20.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.021+0000 7f7cf9d9b700 1 --2- 172.21.15.3:0/670830393 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ce4003b10 0x7f7cfc1c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.021+0000 7f7cf9d9b700 1 -- 172.21.15.3:0/670830393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc0a06b0 con 0x7f7cfc09d6d0 2023-10-03T10:13:20.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.021+0000 7f7cfa59c700 1 --2- 172.21.15.3:0/670830393 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ce4003b10 0x7f7cfc1c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:13:20.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.021+0000 7f7cf9d9b700 1 --2- 172.21.15.3:0/670830393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cfc09d6d0 0x7f7cfc1c82d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7cf000c750 tx=0x7f7cfc06c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:20.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.021+0000 7f7ceaffd700 1 -- 172.21.15.3:0/670830393 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0004c80 con 0x7f7cfc09d6d0 2023-10-03T10:13:20.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.021+0000 7f7ceaffd700 1 -- 172.21.15.3:0/670830393 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf0004e00 con 0x7f7cfc09d6d0 2023-10-03T10:13:20.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.021+0000 7f7d00b8e700 1 -- 172.21.15.3:0/670830393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cfc09c900 con 0x7f7cfc09d6d0 2023-10-03T10:13:20.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.021+0000 7f7ceaffd700 1 -- 172.21.15.3:0/670830393 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0007310 con 0x7f7cfc09d6d0 2023-10-03T10:13:20.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.021+0000 7f7d00b8e700 1 -- 172.21.15.3:0/670830393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cfc121a00 con 0x7f7cfc09d6d0 2023-10-03T10:13:20.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.022+0000 7f7ceaffd700 1 -- 172.21.15.3:0/670830393 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7cf0007470 con 0x7f7cfc09d6d0 2023-10-03T10:13:20.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.023+0000 7f7ceaffd700 1 -- 172.21.15.3:0/670830393 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7cf00406d0 con 0x7f7cfc09d6d0 2023-10-03T10:13:20.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.023+0000 7f7d00b8e700 1 -- 172.21.15.3:0/670830393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cfc04f2a0 con 0x7f7cfc09d6d0 2023-10-03T10:13:20.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.028+0000 7f7ceaffd700 1 -- 172.21.15.3:0/670830393 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7cf0017080 con 0x7f7cfc09d6d0 2023-10-03T10:13:20.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.053+0000 7f06d3863700 1 Processor -- start 2023-10-03T10:13:20.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.053+0000 7f06d3863700 1 -- start start 2023-10-03T10:13:20.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.054+0000 7f06d3863700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06cc09d6f0 0x7f06cc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:20.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.054+0000 7f06d3863700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f06cc09e000 0x7f06cc130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:20.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.054+0000 7f06d3863700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06cc09f130 con 0x7f06cc09e410 2023-10-03T10:13:20.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.054+0000 7f06d3863700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc09f520 con 0x7f06cc09d6f0 2023-10-03T10:13:20.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.054+0000 7f06d3863700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc09e930 con 0x7f06cc09e000 2023-10-03T10:13:20.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.054+0000 7f06d15ff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06cc09d6f0 0x7f06cc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:20.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.054+0000 7f06d15ff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06cc09d6f0 0x7f06cc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36802/0 (socket says 172.21.15.3:36802) 2023-10-03T10:13:20.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.054+0000 7f06d15ff700 1 -- 172.21.15.3:0/4209162342 learned_addr learned my addr 172.21.15.3:0/4209162342 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:20.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.054+0000 7f06d0dfe700 1 --2- 172.21.15.3:0/4209162342 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f06cc09e000 0x7f06cc130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:20.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.055+0000 7f06d15ff700 1 -- 172.21.15.3:0/4209162342 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f06cc09e000 msgr2=0x7f06cc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:20.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.055+0000 7f06d15ff700 1 --2- 172.21.15.3:0/4209162342 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f06cc09e000 0x7f06cc130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.055+0000 7f06d15ff700 1 -- 172.21.15.3:0/4209162342 >> v1:172.21.15.3:6789/0 conn(0x7f06cc09e410 legacy=0x7f06cc130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:20.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.055+0000 7f06d15ff700 1 -- 172.21.15.3:0/4209162342 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06cc09b2d0 con 0x7f06cc09d6f0 2023-10-03T10:13:20.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.055+0000 7f06d15ff700 1 --2- 172.21.15.3:0/4209162342 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06cc09d6f0 0x7f06cc09dac0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f06c8009920 tx=0x7f06c8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c331d3eaebdbdbb server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:20.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.055+0000 7f06c3fff700 1 -- 172.21.15.3:0/4209162342 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c8011830 con 0x7f06cc09d6f0 2023-10-03T10:13:20.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.055+0000 7f06c3fff700 1 -- 172.21.15.3:0/4209162342 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c8011990 con 0x7f06cc09d6f0 2023-10-03T10:13:20.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.055+0000 7f06c3fff700 1 -- 172.21.15.3:0/4209162342 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c8011d20 con 0x7f06cc09d6f0 2023-10-03T10:13:20.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.056+0000 7f06d3863700 1 -- 172.21.15.3:0/4209162342 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06cc09d6f0 msgr2=0x7f06cc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:20.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.056+0000 7f06d3863700 1 --2- 172.21.15.3:0/4209162342 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06cc09d6f0 0x7f06cc09dac0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f06c8009920 tx=0x7f06c8009c30 comp rx=0 tx=0).stop 2023-10-03T10:13:20.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.056+0000 7f06d3863700 1 -- 172.21.15.3:0/4209162342 shutdown_connections 2023-10-03T10:13:20.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.056+0000 7f06d3863700 1 --2- 172.21.15.3:0/4209162342 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06cc09d6f0 0x7f06cc09dac0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.056+0000 7f06d3863700 1 --2- 172.21.15.3:0/4209162342 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f06cc09e000 0x7f06cc130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.056+0000 7f06d3863700 1 -- 172.21.15.3:0/4209162342 >> 172.21.15.3:0/4209162342 conn(0x7f06cc066f50 msgr2=0x7f06cc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:20.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.056+0000 7f06d3863700 1 -- 172.21.15.3:0/4209162342 shutdown_connections 2023-10-03T10:13:20.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.056+0000 7f06d3863700 1 -- 172.21.15.3:0/4209162342 wait complete. 2023-10-03T10:13:20.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.057+0000 7f06d3863700 1 Processor -- start 2023-10-03T10:13:20.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.057+0000 7f06d3863700 1 -- start start 2023-10-03T10:13:20.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.058+0000 7f06d3863700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06cc09d6f0 0x7f06cc122460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:20.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.058+0000 7f06d3863700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06cc09e000 0x7f06cc1229a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:20.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.058+0000 7f06d3863700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f06cc09e410 0x7f06cc122ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:20.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.058+0000 7f06d3863700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc123f10 con 0x7f06cc09d6f0 2023-10-03T10:13:20.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.058+0000 7f06d3863700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc125e90 con 0x7f06cc09e000 2023-10-03T10:13:20.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.058+0000 7f06d3863700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc125510 con 0x7f06cc09e410 2023-10-03T10:13:20.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.058+0000 7f06d0dfe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06cc09e000 0x7f06cc1229a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:20.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.058+0000 7f06d15ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06cc09d6f0 0x7f06cc122460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:20.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.058+0000 7f06d0dfe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06cc09e000 0x7f06cc1229a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36806/0 (socket says 172.21.15.3:36806) 2023-10-03T10:13:20.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.058+0000 7f06d0dfe700 1 -- 172.21.15.3:0/1188134125 learned_addr learned my addr 172.21.15.3:0/1188134125 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:20.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.058+0000 7f06d1e00700 1 --2- 172.21.15.3:0/1188134125 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f06cc09e410 0x7f06cc122ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:20.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.058+0000 7f06d0dfe700 1 -- 172.21.15.3:0/1188134125 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f06cc09e410 msgr2=0x7f06cc122ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:20.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.058+0000 7f06d0dfe700 1 --2- 172.21.15.3:0/1188134125 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f06cc09e410 0x7f06cc122ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.058+0000 7f06d0dfe700 1 -- 172.21.15.3:0/1188134125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06cc09d6f0 msgr2=0x7f06cc122460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:20.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.058+0000 7f06d0dfe700 1 --2- 172.21.15.3:0/1188134125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06cc09d6f0 0x7f06cc122460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.058+0000 7f06d0dfe700 1 -- 172.21.15.3:0/1188134125 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c8009550 con 0x7f06cc09e000 2023-10-03T10:13:20.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.058+0000 7f06d15ff700 1 --2- 172.21.15.3:0/1188134125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06cc09d6f0 0x7f06cc122460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:20.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.059+0000 7f06d0dfe700 1 --2- 172.21.15.3:0/1188134125 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06cc09e000 0x7f06cc1229a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00e980 tx=0x7f06bc00ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:20.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.059+0000 7f06c1ffb700 1 -- 172.21.15.3:0/1188134125 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc009710 con 0x7f06cc09e000 2023-10-03T10:13:20.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.059+0000 7f06c1ffb700 1 -- 172.21.15.3:0/1188134125 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06bc00eed0 con 0x7f06cc09e000 2023-10-03T10:13:20.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.059+0000 7f06d3863700 1 -- 172.21.15.3:0/1188134125 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06cc0a1d00 con 0x7f06cc09e000 2023-10-03T10:13:20.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.059+0000 7f06c1ffb700 1 -- 172.21.15.3:0/1188134125 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc00ca70 con 0x7f06cc09e000 2023-10-03T10:13:20.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.059+0000 7f06d3863700 1 -- 172.21.15.3:0/1188134125 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06cc1258f0 con 0x7f06cc09e000 2023-10-03T10:13:20.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.060+0000 7f06c1ffb700 1 -- 172.21.15.3:0/1188134125 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f06bc00cbd0 con 0x7f06cc09e000 2023-10-03T10:13:20.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.061+0000 7f06c1ffb700 1 -- 172.21.15.3:0/1188134125 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f06bc045060 con 0x7f06cc09e000 2023-10-03T10:13:20.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.061+0000 7f06d3863700 1 -- 172.21.15.3:0/1188134125 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06cc125ae0 con 0x7f06cc09e000 2023-10-03T10:13:20.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.070+0000 7f06c1ffb700 1 -- 172.21.15.3:0/1188134125 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f06bc017070 con 0x7f06cc09e000 2023-10-03T10:13:20.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.114+0000 7f0531ac6700 1 Processor -- start 2023-10-03T10:13:20.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.114+0000 7f0531ac6700 1 -- start start 2023-10-03T10:13:20.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.115+0000 7f0531ac6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f052c09eca0 0x7f052c13cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:20.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.115+0000 7f0531ac6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f052c06f050 0x7f052c13f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:20.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.115+0000 7f0531ac6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f052c070ee0 con 0x7f052c06e570 2023-10-03T10:13:20.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.115+0000 7f0531ac6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052c071980 con 0x7f052c06f050 2023-10-03T10:13:20.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.115+0000 7f0531ac6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f052c071480 con 0x7f052c09eca0 2023-10-03T10:13:20.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.115+0000 7f052affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f052c09eca0 0x7f052c13cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:20.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.115+0000 7f052b7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f052c06e570 0x7f052c06e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59410/0 (socket says 172.21.15.3:59410) 2023-10-03T10:13:20.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.115+0000 7f052b7fe700 1 -- 172.21.15.3:0/1386834107 learned_addr learned my addr 172.21.15.3:0/1386834107 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:20.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.115+0000 7f052bfff700 1 --2- 172.21.15.3:0/1386834107 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f052c06f050 0x7f052c13f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:20.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.115+0000 7f052bfff700 1 -- 172.21.15.3:0/1386834107 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f052c09eca0 msgr2=0x7f052c13cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:20.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.115+0000 7f052bfff700 1 --2- 172.21.15.3:0/1386834107 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f052c09eca0 0x7f052c13cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.115+0000 7f052bfff700 1 -- 172.21.15.3:0/1386834107 >> v1:172.21.15.3:6789/0 conn(0x7f052c06e570 legacy=0x7f052c06e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:20.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.115+0000 7f052bfff700 1 -- 172.21.15.3:0/1386834107 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052c071b90 con 0x7f052c06f050 2023-10-03T10:13:20.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.116+0000 7f052bfff700 1 --2- 172.21.15.3:0/1386834107 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f052c06f050 0x7f052c13f3f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0524009dc0 tx=0x7f052400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca63ef53245e0715 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:20.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.116+0000 7f052a7fc700 1 -- 172.21.15.3:0/1386834107 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052400e070 con 0x7f052c06f050 2023-10-03T10:13:20.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.116+0000 7f052a7fc700 1 -- 172.21.15.3:0/1386834107 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0524003a20 con 0x7f052c06f050 2023-10-03T10:13:20.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.116+0000 7f052a7fc700 1 -- 172.21.15.3:0/1386834107 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524003dd0 con 0x7f052c06f050 2023-10-03T10:13:20.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.117+0000 7f0531ac6700 1 -- 172.21.15.3:0/1386834107 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f052c06f050 msgr2=0x7f052c13f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:20.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.117+0000 7f0531ac6700 1 --2- 172.21.15.3:0/1386834107 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f052c06f050 0x7f052c13f3f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0524009dc0 tx=0x7f052400b330 comp rx=0 tx=0).stop 2023-10-03T10:13:20.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.117+0000 7f0531ac6700 1 -- 172.21.15.3:0/1386834107 shutdown_connections 2023-10-03T10:13:20.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.117+0000 7f0531ac6700 1 --2- 172.21.15.3:0/1386834107 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f052c06f050 0x7f052c13f3f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.117+0000 7f0531ac6700 1 --2- 172.21.15.3:0/1386834107 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f052c09eca0 0x7f052c13cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.117+0000 7f0531ac6700 1 -- 172.21.15.3:0/1386834107 >> 172.21.15.3:0/1386834107 conn(0x7f052c0a2aa0 msgr2=0x7f052c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:20.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.117+0000 7f0531ac6700 1 -- 172.21.15.3:0/1386834107 shutdown_connections 2023-10-03T10:13:20.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.117+0000 7f0531ac6700 1 -- 172.21.15.3:0/1386834107 wait complete. 2023-10-03T10:13:20.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.118+0000 7f0531ac6700 1 Processor -- start 2023-10-03T10:13:20.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.119+0000 7f0531ac6700 1 -- start start 2023-10-03T10:13:20.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.119+0000 7f0531ac6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f052c06e570 0x7f052c123160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:20.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.119+0000 7f0531ac6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f052c06f050 0x7f052c1236a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:20.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.119+0000 7f0531ac6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f052c123be0 0x7f052c1d2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:20.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.119+0000 7f0531ac6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c125610 con 0x7f052c123be0 2023-10-03T10:13:20.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.119+0000 7f0531ac6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052c126030 con 0x7f052c06f050 2023-10-03T10:13:20.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.119+0000 7f0531ac6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f052c09a3a0 con 0x7f052c06e570 2023-10-03T10:13:20.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.119+0000 7f052b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f052c06e570 0x7f052c123160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:20.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.119+0000 7f052b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f052c06e570 0x7f052c123160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55626/0 (socket says 172.21.15.3:55626) 2023-10-03T10:13:20.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.119+0000 7f052b7fe700 1 -- 172.21.15.3:0/3066449881 learned_addr learned my addr 172.21.15.3:0/3066449881 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:20.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.119+0000 7f052bfff700 1 --2- 172.21.15.3:0/3066449881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f052c123be0 0x7f052c1d2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:20.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.119+0000 7f052affd700 1 --2- 172.21.15.3:0/3066449881 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f052c06f050 0x7f052c1236a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:20.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.120+0000 7f052b7fe700 1 -- 172.21.15.3:0/3066449881 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f052c06f050 msgr2=0x7f052c1236a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:20.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.120+0000 7f052b7fe700 1 --2- 172.21.15.3:0/3066449881 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f052c06f050 0x7f052c1236a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.120+0000 7f052b7fe700 1 -- 172.21.15.3:0/3066449881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f052c123be0 msgr2=0x7f052c1d2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:20.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.120+0000 7f052b7fe700 1 --2- 172.21.15.3:0/3066449881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f052c123be0 0x7f052c1d2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.120+0000 7f052b7fe700 1 -- 172.21.15.3:0/3066449881 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0524009a70 con 0x7f052c06e570 2023-10-03T10:13:20.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.120+0000 7f052affd700 1 --2- 172.21.15.3:0/3066449881 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f052c06f050 0x7f052c1236a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:20.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.120+0000 7f052b7fe700 1 --2- 172.21.15.3:0/3066449881 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f052c06e570 0x7f052c123160 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f051c00c780 tx=0x7f051c00ca90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:20.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.120+0000 7f0513fff700 1 -- 172.21.15.3:0/3066449881 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c004c80 con 0x7f052c06e570 2023-10-03T10:13:20.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.120+0000 7f0513fff700 1 -- 172.21.15.3:0/3066449881 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051c004de0 con 0x7f052c06e570 2023-10-03T10:13:20.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.120+0000 7f0531ac6700 1 -- 172.21.15.3:0/3066449881 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052c070610 con 0x7f052c06e570 2023-10-03T10:13:20.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.120+0000 7f0513fff700 1 -- 172.21.15.3:0/3066449881 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c007310 con 0x7f052c06e570 2023-10-03T10:13:20.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.120+0000 7f0531ac6700 1 -- 172.21.15.3:0/3066449881 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052c127410 con 0x7f052c06e570 2023-10-03T10:13:20.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.121+0000 7f0531ac6700 1 -- 172.21.15.3:0/3066449881 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f052c13d030 con 0x7f052c06e570 2023-10-03T10:13:20.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.122+0000 7f0513fff700 1 -- 172.21.15.3:0/3066449881 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f051c007470 con 0x7f052c06e570 2023-10-03T10:13:20.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.122+0000 7f0513fff700 1 -- 172.21.15.3:0/3066449881 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f051c03ef10 con 0x7f052c06e570 2023-10-03T10:13:20.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.131+0000 7f0513fff700 1 -- 172.21.15.3:0/3066449881 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f051c007720 con 0x7f052c06e570 2023-10-03T10:13:20.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.163+0000 7fea2d435700 1 Processor -- start 2023-10-03T10:13:20.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.163+0000 7fea2d435700 1 -- start start 2023-10-03T10:13:20.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.163+0000 7fea2d435700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea280a0a30 0x7fea280a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:20.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.163+0000 7fea2d435700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea2809d630 0x7fea280a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:20.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.163+0000 7fea2d435700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea280a2a70 con 0x7fea28133930 2023-10-03T10:13:20.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.163+0000 7fea2d435700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea280a01d0 con 0x7fea2809d630 2023-10-03T10:13:20.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.163+0000 7fea2d435700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea2809e2b0 con 0x7fea280a0a30 2023-10-03T10:13:20.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.164+0000 7fea277fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fea28133930 0x7fea28133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59416/0 (socket says 172.21.15.3:59416) 2023-10-03T10:13:20.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.164+0000 7fea277fe700 1 -- 172.21.15.3:0/250300081 learned_addr learned my addr 172.21.15.3:0/250300081 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:20.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.164+0000 7fea267fc700 1 --2- 172.21.15.3:0/250300081 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea2809d630 0x7fea280a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:20.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.164+0000 7fea26ffd700 1 --2- 172.21.15.3:0/250300081 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea280a0a30 0x7fea280a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:20.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.164+0000 7fea25ffb700 1 -- 172.21.15.3:0/250300081 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1063440625 0 0) 0x7fea280a2a70 con 0x7fea28133930 2023-10-03T10:13:20.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.164+0000 7fea25ffb700 1 -- 172.21.15.3:0/250300081 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea100035b0 con 0x7fea28133930 2023-10-03T10:13:20.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.164+0000 7fea267fc700 1 -- 172.21.15.3:0/250300081 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea280a0a30 msgr2=0x7fea280a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:20.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.164+0000 7fea267fc700 1 --2- 172.21.15.3:0/250300081 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea280a0a30 0x7fea280a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.164+0000 7fea267fc700 1 -- 172.21.15.3:0/250300081 >> v1:172.21.15.3:6789/0 conn(0x7fea28133930 legacy=0x7fea28133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:20.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.164+0000 7fea267fc700 1 -- 172.21.15.3:0/250300081 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2809dc50 con 0x7fea2809d630 2023-10-03T10:13:20.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.164+0000 7fea267fc700 1 --2- 172.21.15.3:0/250300081 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea2809d630 0x7fea280a1340 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fea1c009870 tx=0x7fea1c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dd3c9271f27cc94 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:20.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.165+0000 7fea25ffb700 1 -- 172.21.15.3:0/250300081 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c011830 con 0x7fea2809d630 2023-10-03T10:13:20.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.165+0000 7fea25ffb700 1 -- 172.21.15.3:0/250300081 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1c011990 con 0x7fea2809d630 2023-10-03T10:13:20.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.165+0000 7fea25ffb700 1 -- 172.21.15.3:0/250300081 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c011d20 con 0x7fea2809d630 2023-10-03T10:13:20.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.165+0000 7fea2d435700 1 -- 172.21.15.3:0/250300081 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea2809d630 msgr2=0x7fea280a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:20.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.165+0000 7fea2d435700 1 --2- 172.21.15.3:0/250300081 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea2809d630 0x7fea280a1340 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fea1c009870 tx=0x7fea1c009b80 comp rx=0 tx=0).stop 2023-10-03T10:13:20.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.165+0000 7fea2d435700 1 -- 172.21.15.3:0/250300081 shutdown_connections 2023-10-03T10:13:20.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.165+0000 7fea2d435700 1 --2- 172.21.15.3:0/250300081 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea280a0a30 0x7fea280a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.165+0000 7fea2d435700 1 --2- 172.21.15.3:0/250300081 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea2809d630 0x7fea280a1340 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.165+0000 7fea2d435700 1 -- 172.21.15.3:0/250300081 >> 172.21.15.3:0/250300081 conn(0x7fea280a3050 msgr2=0x7fea280a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:20.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.165+0000 7fea2d435700 1 -- 172.21.15.3:0/250300081 shutdown_connections 2023-10-03T10:13:20.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.165+0000 7fea2d435700 1 -- 172.21.15.3:0/250300081 wait complete. 2023-10-03T10:13:20.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.167+0000 7fea2d435700 1 Processor -- start 2023-10-03T10:13:20.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.167+0000 7fea2d435700 1 -- start start 2023-10-03T10:13:20.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.167+0000 7fea2d435700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea2809d630 0x7fea2811e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:20.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.167+0000 7fea2d435700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea280a0a30 0x7fea2811eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:20.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.167+0000 7fea2d435700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea28133930 0x7fea2811f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:20.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.167+0000 7fea2d435700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea280a53d0 con 0x7fea2809d630 2023-10-03T10:13:20.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.167+0000 7fea2d435700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea280a0410 con 0x7fea28133930 2023-10-03T10:13:20.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.167+0000 7fea2d435700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea28121ce0 con 0x7fea280a0a30 2023-10-03T10:13:20.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.167+0000 7fea267fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea280a0a30 0x7fea2811eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:20.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.167+0000 7fea267fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea280a0a30 0x7fea2811eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55646/0 (socket says 172.21.15.3:55646) 2023-10-03T10:13:20.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.167+0000 7fea267fc700 1 -- 172.21.15.3:0/1655652779 learned_addr learned my addr 172.21.15.3:0/1655652779 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:20.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.167+0000 7fea277fe700 1 --2- 172.21.15.3:0/1655652779 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea28133930 0x7fea2811f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:20.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.168+0000 7fea267fc700 1 -- 172.21.15.3:0/1655652779 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea28133930 msgr2=0x7fea2811f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:20.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.168+0000 7fea267fc700 1 --2- 172.21.15.3:0/1655652779 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea28133930 0x7fea2811f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.168+0000 7fea267fc700 1 -- 172.21.15.3:0/1655652779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea2809d630 msgr2=0x7fea2811e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:20.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.168+0000 7fea267fc700 1 --2- 172.21.15.3:0/1655652779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea2809d630 0x7fea2811e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.168+0000 7fea267fc700 1 -- 172.21.15.3:0/1655652779 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1c0094a0 con 0x7fea280a0a30 2023-10-03T10:13:20.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.168+0000 7fea267fc700 1 --2- 172.21.15.3:0/1655652779 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea280a0a30 0x7fea2811eca0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fea1c000c00 tx=0x7fea1c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:20.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.168+0000 7fea0f7fe700 1 -- 172.21.15.3:0/1655652779 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c004510 con 0x7fea280a0a30 2023-10-03T10:13:20.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.168+0000 7fea0f7fe700 1 -- 172.21.15.3:0/1655652779 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1c01f070 con 0x7fea280a0a30 2023-10-03T10:13:20.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.168+0000 7fea2d435700 1 -- 172.21.15.3:0/1655652779 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea28122280 con 0x7fea280a0a30 2023-10-03T10:13:20.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.168+0000 7fea0f7fe700 1 -- 172.21.15.3:0/1655652779 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c0038e0 con 0x7fea280a0a30 2023-10-03T10:13:20.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.168+0000 7fea2d435700 1 -- 172.21.15.3:0/1655652779 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea28133210 con 0x7fea280a0a30 2023-10-03T10:13:20.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.169+0000 7fea0f7fe700 1 -- 172.21.15.3:0/1655652779 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fea1c003a40 con 0x7fea280a0a30 2023-10-03T10:13:20.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.169+0000 7fea0f7fe700 1 -- 172.21.15.3:0/1655652779 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fea1c021070 con 0x7fea280a0a30 2023-10-03T10:13:20.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.170+0000 7fea2d435700 1 -- 172.21.15.3:0/1655652779 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea28122ad0 con 0x7fea280a0a30 2023-10-03T10:13:20.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.179+0000 7fea0f7fe700 1 -- 172.21.15.3:0/1655652779 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fea1c015080 con 0x7fea280a0a30 2023-10-03T10:13:20.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.280+0000 7f7d00b8e700 1 -- 172.21.15.3:0/670830393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7cfc06c230 con 0x7f7cfc09d6d0 2023-10-03T10:13:20.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.280+0000 7f7ceaffd700 1 -- 172.21.15.3:0/670830393 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7cf002a020 con 0x7f7cfc09d6d0 2023-10-03T10:13:20.281 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:20.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.283+0000 7f7d00b8e700 1 -- 172.21.15.3:0/670830393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cfc09d6d0 msgr2=0x7f7cfc1c82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:20.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.283+0000 7f7d00b8e700 1 --2- 172.21.15.3:0/670830393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cfc09d6d0 0x7f7cfc1c82d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7cf000c750 tx=0x7f7cfc06c550 comp rx=0 tx=0).stop 2023-10-03T10:13:20.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.283+0000 7f7d00b8e700 1 -- 172.21.15.3:0/670830393 shutdown_connections 2023-10-03T10:13:20.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.283+0000 7f7d00b8e700 1 --2- 172.21.15.3:0/670830393 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cfc09dfe0 0x7f7cfc1c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.283+0000 7f7d00b8e700 1 --2- 172.21.15.3:0/670830393 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ce4003b10 0x7f7cfc1c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.283+0000 7f7d00b8e700 1 --2- 172.21.15.3:0/670830393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cfc09d6d0 0x7f7cfc1c82d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.283+0000 7f7d00b8e700 1 -- 172.21.15.3:0/670830393 >> 172.21.15.3:0/670830393 conn(0x7f7cfc099c20 msgr2=0x7f7cfc0a0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:20.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.284+0000 7f7d00b8e700 1 -- 172.21.15.3:0/670830393 shutdown_connections 2023-10-03T10:13:20.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.286+0000 7f7d00b8e700 1 -- 172.21.15.3:0/670830393 wait complete. 2023-10-03T10:13:20.306 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:20.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.327+0000 7f06d3863700 1 -- 172.21.15.3:0/1188134125 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f06cc074ea0 con 0x7f06cc09e000 2023-10-03T10:13:20.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.328+0000 7f06c1ffb700 1 -- 172.21.15.3:0/1188134125 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f06bc017030 con 0x7f06cc09e000 2023-10-03T10:13:20.328 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.330+0000 7f06d3863700 1 -- 172.21.15.3:0/1188134125 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06cc09e000 msgr2=0x7f06cc1229a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.330+0000 7f06d3863700 1 --2- 172.21.15.3:0/1188134125 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06cc09e000 0x7f06cc1229a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00e980 tx=0x7f06bc00ec90 comp rx=0 tx=0).stop 2023-10-03T10:13:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.330+0000 7f06d3863700 1 -- 172.21.15.3:0/1188134125 shutdown_connections 2023-10-03T10:13:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.330+0000 7f06d3863700 1 --2- 172.21.15.3:0/1188134125 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f06cc09e410 0x7f06cc122ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.330+0000 7f06d3863700 1 --2- 172.21.15.3:0/1188134125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06cc09d6f0 0x7f06cc122460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.330+0000 7f06d3863700 1 --2- 172.21.15.3:0/1188134125 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06cc09e000 0x7f06cc1229a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.330+0000 7f06d3863700 1 -- 172.21.15.3:0/1188134125 >> 172.21.15.3:0/1188134125 conn(0x7f06cc066f50 msgr2=0x7f06cc071010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.330+0000 7f06d3863700 1 -- 172.21.15.3:0/1188134125 shutdown_connections 2023-10-03T10:13:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.330+0000 7f06d3863700 1 -- 172.21.15.3:0/1188134125 wait complete. 2023-10-03T10:13:20.345 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:20.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.352+0000 7f0531ac6700 1 -- 172.21.15.3:0/3066449881 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f052c074ea0 con 0x7f052c06e570 2023-10-03T10:13:20.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.352+0000 7f0513fff700 1 -- 172.21.15.3:0/3066449881 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f051c018030 con 0x7f052c06e570 2023-10-03T10:13:20.353 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:20.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.355+0000 7f0531ac6700 1 -- 172.21.15.3:0/3066449881 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f052c06e570 msgr2=0x7f052c123160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:20.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.355+0000 7f0531ac6700 1 --2- 172.21.15.3:0/3066449881 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f052c06e570 0x7f052c123160 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f051c00c780 tx=0x7f051c00ca90 comp rx=0 tx=0).stop 2023-10-03T10:13:20.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.355+0000 7f0531ac6700 1 -- 172.21.15.3:0/3066449881 shutdown_connections 2023-10-03T10:13:20.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.355+0000 7f0531ac6700 1 --2- 172.21.15.3:0/3066449881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f052c123be0 0x7f052c1d2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.355+0000 7f0531ac6700 1 --2- 172.21.15.3:0/3066449881 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f052c06e570 0x7f052c123160 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.355+0000 7f0531ac6700 1 --2- 172.21.15.3:0/3066449881 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f052c06f050 0x7f052c1236a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.355+0000 7f0531ac6700 1 -- 172.21.15.3:0/3066449881 >> 172.21.15.3:0/3066449881 conn(0x7f052c0a2aa0 msgr2=0x7f052c09a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:20.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.355+0000 7f0531ac6700 1 -- 172.21.15.3:0/3066449881 shutdown_connections 2023-10-03T10:13:20.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.355+0000 7f0531ac6700 1 -- 172.21.15.3:0/3066449881 wait complete. 2023-10-03T10:13:20.369 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:20.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.394+0000 7fea2d435700 1 -- 172.21.15.3:0/1655652779 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fea280747f0 con 0x7fea280a0a30 2023-10-03T10:13:20.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.394+0000 7fea0f7fe700 1 -- 172.21.15.3:0/1655652779 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fea1c0253e0 con 0x7fea280a0a30 2023-10-03T10:13:20.394 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:20.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.396+0000 7fea2d435700 1 -- 172.21.15.3:0/1655652779 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea280a0a30 msgr2=0x7fea2811eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:20.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.396+0000 7fea2d435700 1 --2- 172.21.15.3:0/1655652779 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea280a0a30 0x7fea2811eca0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fea1c000c00 tx=0x7fea1c004750 comp rx=0 tx=0).stop 2023-10-03T10:13:20.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.396+0000 7fea2d435700 1 -- 172.21.15.3:0/1655652779 shutdown_connections 2023-10-03T10:13:20.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.396+0000 7fea2d435700 1 --2- 172.21.15.3:0/1655652779 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea28133930 0x7fea2811f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.397+0000 7fea2d435700 1 --2- 172.21.15.3:0/1655652779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea2809d630 0x7fea2811e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.397+0000 7fea2d435700 1 --2- 172.21.15.3:0/1655652779 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea280a0a30 0x7fea2811eca0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:20.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.397+0000 7fea2d435700 1 -- 172.21.15.3:0/1655652779 >> 172.21.15.3:0/1655652779 conn(0x7fea280a3050 msgr2=0x7fea281bbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:20.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.397+0000 7fea2d435700 1 -- 172.21.15.3:0/1655652779 shutdown_connections 2023-10-03T10:13:20.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:20.397+0000 7fea2d435700 1 -- 172.21.15.3:0/1655652779 wait complete. 2023-10-03T10:13:20.410 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:21.307 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:21.346 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:21.370 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:21.412 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:21.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.459+0000 7f85a0e41700 1 Processor -- start 2023-10-03T10:13:21.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.459+0000 7f85a0e41700 1 -- start start 2023-10-03T10:13:21.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.460+0000 7f85a0e41700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f859c09d6f0 0x7f859c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:21.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.460+0000 7f85a0e41700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f859c09e000 0x7f859c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:21.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.460+0000 7f85a0e41700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f859c09f130 con 0x7f859c09e410 2023-10-03T10:13:21.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.460+0000 7f85a0e41700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f859c09f520 con 0x7f859c09e000 2023-10-03T10:13:21.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.460+0000 7f85a0e41700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f859c09e930 con 0x7f859c09d6f0 2023-10-03T10:13:21.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.460+0000 7f859a59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f859c09d6f0 0x7f859c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:21.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.460+0000 7f859a59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f859c09d6f0 0x7f859c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36830/0 (socket says 172.21.15.3:36830) 2023-10-03T10:13:21.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.460+0000 7f859a59c700 1 -- 172.21.15.3:0/1819212692 learned_addr learned my addr 172.21.15.3:0/1819212692 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:21.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.460+0000 7f8599d9b700 1 --2- 172.21.15.3:0/1819212692 >> v2:172.21.15.3:3300/0 conn(0x7f859c09e000 0x7f859c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:21.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.460+0000 7f859a59c700 1 -- 172.21.15.3:0/1819212692 >> v2:172.21.15.3:3300/0 conn(0x7f859c09e000 msgr2=0x7f859c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:21.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.460+0000 7f859a59c700 1 --2- 172.21.15.3:0/1819212692 >> v2:172.21.15.3:3300/0 conn(0x7f859c09e000 0x7f859c130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.460+0000 7f859a59c700 1 -- 172.21.15.3:0/1819212692 >> v1:172.21.15.3:6789/0 conn(0x7f859c09e410 legacy=0x7f859c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:21.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.461+0000 7f859a59c700 1 -- 172.21.15.3:0/1819212692 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f859c09b2d0 con 0x7f859c09d6f0 2023-10-03T10:13:21.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.461+0000 7f8599d9b700 1 --2- 172.21.15.3:0/1819212692 >> v2:172.21.15.3:3300/0 conn(0x7f859c09e000 0x7f859c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:21.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.461+0000 7f859a59c700 1 --2- 172.21.15.3:0/1819212692 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f859c09d6f0 0x7f859c09dac0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f858c00bb90 tx=0x7f858c00bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b0e5a8fca7f5209 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:21.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.461+0000 7f859959a700 1 -- 172.21.15.3:0/1819212692 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c00dd10 con 0x7f859c09d6f0 2023-10-03T10:13:21.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.461+0000 7f859959a700 1 -- 172.21.15.3:0/1819212692 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f858c00de70 con 0x7f859c09d6f0 2023-10-03T10:13:21.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.461+0000 7f859959a700 1 -- 172.21.15.3:0/1819212692 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c004230 con 0x7f859c09d6f0 2023-10-03T10:13:21.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.462+0000 7f85a0e41700 1 -- 172.21.15.3:0/1819212692 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f859c09d6f0 msgr2=0x7f859c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:21.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.462+0000 7f85a0e41700 1 --2- 172.21.15.3:0/1819212692 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f859c09d6f0 0x7f859c09dac0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f858c00bb90 tx=0x7f858c00bea0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.462+0000 7f85a0e41700 1 -- 172.21.15.3:0/1819212692 shutdown_connections 2023-10-03T10:13:21.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.462+0000 7f85a0e41700 1 --2- 172.21.15.3:0/1819212692 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f859c09d6f0 0x7f859c09dac0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.462+0000 7f85a0e41700 1 --2- 172.21.15.3:0/1819212692 >> v2:172.21.15.3:3300/0 conn(0x7f859c09e000 0x7f859c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.462+0000 7f85a0e41700 1 -- 172.21.15.3:0/1819212692 >> 172.21.15.3:0/1819212692 conn(0x7f859c066f50 msgr2=0x7f859c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:21.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.462+0000 7f85a0e41700 1 -- 172.21.15.3:0/1819212692 shutdown_connections 2023-10-03T10:13:21.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.462+0000 7f85a0e41700 1 -- 172.21.15.3:0/1819212692 wait complete. 2023-10-03T10:13:21.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.463+0000 7f85a0e41700 1 Processor -- start 2023-10-03T10:13:21.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.463+0000 7f85a0e41700 1 -- start start 2023-10-03T10:13:21.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.464+0000 7f85a0e41700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f859c09d6f0 0x7f859c11e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:21.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.464+0000 7f85a0e41700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f859c09e000 0x7f859c11e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:21.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.464+0000 7f85a0e41700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c09e410 0x7f859c11eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:21.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.464+0000 7f85a0e41700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859c1218c0 con 0x7f859c09e410 2023-10-03T10:13:21.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.464+0000 7f85a0e41700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f859c09c700 con 0x7f859c09e000 2023-10-03T10:13:21.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.464+0000 7f85a0e41700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f859c130320 con 0x7f859c09d6f0 2023-10-03T10:13:21.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.464+0000 7f8599d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f859c09e000 0x7f859c11e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:21.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.464+0000 7f859a59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f859c09d6f0 0x7f859c11e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:21.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.465+0000 7f8599d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f859c09e000 0x7f859c11e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36840/0 (socket says 172.21.15.3:36840) 2023-10-03T10:13:21.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.465+0000 7f8599d9b700 1 -- 172.21.15.3:0/3653898849 learned_addr learned my addr 172.21.15.3:0/3653898849 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:21.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.465+0000 7f859ad9d700 1 --2- 172.21.15.3:0/3653898849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c09e410 0x7f859c11eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:21.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.465+0000 7f8599d9b700 1 -- 172.21.15.3:0/3653898849 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f859c09d6f0 msgr2=0x7f859c11e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:21.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.465+0000 7f8599d9b700 1 --2- 172.21.15.3:0/3653898849 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f859c09d6f0 0x7f859c11e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.465+0000 7f8599d9b700 1 -- 172.21.15.3:0/3653898849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c09e410 msgr2=0x7f859c11eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:21.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.465+0000 7f8599d9b700 1 --2- 172.21.15.3:0/3653898849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c09e410 0x7f859c11eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.465+0000 7f8599d9b700 1 -- 172.21.15.3:0/3653898849 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858c00b7f0 con 0x7f859c09e000 2023-10-03T10:13:21.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.465+0000 7f859ad9d700 1 --2- 172.21.15.3:0/3653898849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c09e410 0x7f859c11eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:21.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.465+0000 7f8599d9b700 1 --2- 172.21.15.3:0/3653898849 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f859c09e000 0x7f859c11e560 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f859000e840 tx=0x7f859000eb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:21.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.465+0000 7f858affd700 1 -- 172.21.15.3:0/3653898849 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8590013610 con 0x7f859c09e000 2023-10-03T10:13:21.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.465+0000 7f858affd700 1 -- 172.21.15.3:0/3653898849 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8590013770 con 0x7f859c09e000 2023-10-03T10:13:21.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.465+0000 7f85a0e41700 1 -- 172.21.15.3:0/3653898849 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f859c121b20 con 0x7f859c09e000 2023-10-03T10:13:21.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.465+0000 7f858affd700 1 -- 172.21.15.3:0/3653898849 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8590013b20 con 0x7f859c09e000 2023-10-03T10:13:21.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.466+0000 7f85a0e41700 1 -- 172.21.15.3:0/3653898849 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f859c122250 con 0x7f859c09e000 2023-10-03T10:13:21.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.467+0000 7f858affd700 1 -- 172.21.15.3:0/3653898849 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8590013c80 con 0x7f859c09e000 2023-10-03T10:13:21.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.467+0000 7f858affd700 1 -- 172.21.15.3:0/3653898849 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f859003f380 con 0x7f859c09e000 2023-10-03T10:13:21.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.467+0000 7f85a0e41700 1 -- 172.21.15.3:0/3653898849 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f859c11ac20 con 0x7f859c09e000 2023-10-03T10:13:21.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.477+0000 7f858affd700 1 -- 172.21.15.3:0/3653898849 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8590021020 con 0x7f859c09e000 2023-10-03T10:13:21.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.504+0000 7fa341646700 1 Processor -- start 2023-10-03T10:13:21.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.504+0000 7fa341646700 1 -- start start 2023-10-03T10:13:21.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.504+0000 7fa341646700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa33c09e1d0 0x7fa33c138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:21.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.504+0000 7fa341646700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa33c139170 0x7fa33c13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:21.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.504+0000 7fa341646700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa33c09d2f0 con 0x7fa33c09d6f0 2023-10-03T10:13:21.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.504+0000 7fa341646700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c09b0f0 con 0x7fa33c139170 2023-10-03T10:13:21.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.504+0000 7fa341646700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c09a810 con 0x7fa33c09e1d0 2023-10-03T10:13:21.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.505+0000 7fa33b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa33c139170 0x7fa33c13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:21.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.505+0000 7fa33b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa33c139170 0x7fa33c13b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36852/0 (socket says 172.21.15.3:36852) 2023-10-03T10:13:21.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.505+0000 7fa33b7fe700 1 -- 172.21.15.3:0/1780370926 learned_addr learned my addr 172.21.15.3:0/1780370926 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:21.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.505+0000 7fa33a7fc700 1 --2- 172.21.15.3:0/1780370926 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa33c09e1d0 0x7fa33c138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:21.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.505+0000 7fa33b7fe700 1 -- 172.21.15.3:0/1780370926 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa33c09e1d0 msgr2=0x7fa33c138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:21.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.505+0000 7fa33b7fe700 1 --2- 172.21.15.3:0/1780370926 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa33c09e1d0 0x7fa33c138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.505+0000 7fa33b7fe700 1 -- 172.21.15.3:0/1780370926 >> v1:172.21.15.3:6789/0 conn(0x7fa33c09d6f0 legacy=0x7fa33c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:21.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.505+0000 7fa33b7fe700 1 -- 172.21.15.3:0/1780370926 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa33c09a470 con 0x7fa33c139170 2023-10-03T10:13:21.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.505+0000 7fa33b7fe700 1 --2- 172.21.15.3:0/1780370926 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa33c139170 0x7fa33c13b550 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa334009eb0 tx=0x7fa33400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d37385353cabae9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:21.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.506+0000 7fa339ffb700 1 -- 172.21.15.3:0/1780370926 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33400e070 con 0x7fa33c139170 2023-10-03T10:13:21.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.506+0000 7fa339ffb700 1 -- 172.21.15.3:0/1780370926 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33400b980 con 0x7fa33c139170 2023-10-03T10:13:21.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.506+0000 7fa339ffb700 1 -- 172.21.15.3:0/1780370926 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33400bd30 con 0x7fa33c139170 2023-10-03T10:13:21.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.506+0000 7fa341646700 1 -- 172.21.15.3:0/1780370926 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa33c139170 msgr2=0x7fa33c13b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:21.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.506+0000 7fa341646700 1 --2- 172.21.15.3:0/1780370926 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa33c139170 0x7fa33c13b550 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa334009eb0 tx=0x7fa33400b330 comp rx=0 tx=0).stop 2023-10-03T10:13:21.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.507+0000 7fa341646700 1 -- 172.21.15.3:0/1780370926 shutdown_connections 2023-10-03T10:13:21.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.507+0000 7fa341646700 1 --2- 172.21.15.3:0/1780370926 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa33c139170 0x7fa33c13b550 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.507+0000 7fa341646700 1 --2- 172.21.15.3:0/1780370926 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa33c09e1d0 0x7fa33c138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.507+0000 7fa341646700 1 -- 172.21.15.3:0/1780370926 >> 172.21.15.3:0/1780370926 conn(0x7fa33c066f50 msgr2=0x7fa33c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:21.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.507+0000 7fa341646700 1 -- 172.21.15.3:0/1780370926 shutdown_connections 2023-10-03T10:13:21.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.507+0000 7fa341646700 1 -- 172.21.15.3:0/1780370926 wait complete. 2023-10-03T10:13:21.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.508+0000 7fa341646700 1 Processor -- start 2023-10-03T10:13:21.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.508+0000 7fa341646700 1 -- start start 2023-10-03T10:13:21.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.509+0000 7fa341646700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa33c09d6f0 0x7fa33c11e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:21.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.509+0000 7fa341646700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa33c09e1d0 0x7fa33c11ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:21.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.509+0000 7fa341646700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa33c139170 0x7fa33c11f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:21.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.509+0000 7fa341646700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c121b00 con 0x7fa33c09e1d0 2023-10-03T10:13:21.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.509+0000 7fa341646700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c09c640 con 0x7fa33c09d6f0 2023-10-03T10:13:21.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.509+0000 7fa341646700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c120570 con 0x7fa33c139170 2023-10-03T10:13:21.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.509+0000 7fa33a7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa33c09e1d0 0x7fa33c11ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:21.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.509+0000 7fa33b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa33c139170 0x7fa33c11f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:21.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.509+0000 7fa33a7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa33c09e1d0 0x7fa33c11ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50764/0 (socket says 172.21.15.3:50764) 2023-10-03T10:13:21.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.509+0000 7fa33a7fc700 1 -- 172.21.15.3:0/488402291 learned_addr learned my addr 172.21.15.3:0/488402291 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:21.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.510+0000 7fa33affd700 1 --2- 172.21.15.3:0/488402291 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa33c09d6f0 0x7fa33c11e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:21.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.510+0000 7fa33a7fc700 1 -- 172.21.15.3:0/488402291 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa33c139170 msgr2=0x7fa33c11f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:21.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.510+0000 7fa33a7fc700 1 --2- 172.21.15.3:0/488402291 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa33c139170 0x7fa33c11f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.510+0000 7fa33a7fc700 1 -- 172.21.15.3:0/488402291 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa33c09d6f0 msgr2=0x7fa33c11e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:21.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.510+0000 7fa33a7fc700 1 --2- 172.21.15.3:0/488402291 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa33c09d6f0 0x7fa33c11e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.510+0000 7fa33a7fc700 1 -- 172.21.15.3:0/488402291 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa334009b30 con 0x7fa33c09e1d0 2023-10-03T10:13:21.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.510+0000 7fa33a7fc700 1 --2- 172.21.15.3:0/488402291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa33c09e1d0 0x7fa33c11ee50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa33000e810 tx=0x7fa33000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:21.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.510+0000 7fa3237fe700 1 -- 172.21.15.3:0/488402291 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33000c070 con 0x7fa33c09e1d0 2023-10-03T10:13:21.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.510+0000 7fa3237fe700 1 -- 172.21.15.3:0/488402291 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa330013610 con 0x7fa33c09e1d0 2023-10-03T10:13:21.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.510+0000 7fa341646700 1 -- 172.21.15.3:0/488402291 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa33c121dd0 con 0x7fa33c09e1d0 2023-10-03T10:13:21.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.510+0000 7fa3237fe700 1 -- 172.21.15.3:0/488402291 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3300139c0 con 0x7fa33c09e1d0 2023-10-03T10:13:21.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.510+0000 7fa341646700 1 -- 172.21.15.3:0/488402291 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa33c122b00 con 0x7fa33c09e1d0 2023-10-03T10:13:21.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.512+0000 7fa3237fe700 1 -- 172.21.15.3:0/488402291 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa330013b20 con 0x7fa33c09e1d0 2023-10-03T10:13:21.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.512+0000 7fa3237fe700 1 -- 172.21.15.3:0/488402291 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa33003f450 con 0x7fa33c09e1d0 2023-10-03T10:13:21.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.512+0000 7fa341646700 1 -- 172.21.15.3:0/488402291 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa33c122120 con 0x7fa33c09e1d0 2023-10-03T10:13:21.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.521+0000 7fa3237fe700 1 -- 172.21.15.3:0/488402291 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa330010540 con 0x7fa33c09e1d0 2023-10-03T10:13:21.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.550+0000 7f5fe4ab8700 1 Processor -- start 2023-10-03T10:13:21.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.550+0000 7f5fe4ab8700 1 -- start start 2023-10-03T10:13:21.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.550+0000 7f5fe4ab8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fe00a0a30 0x7f5fe00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:21.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.550+0000 7f5fe4ab8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5fe0138d80 0x7f5fe013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:21.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.550+0000 7f5fe4ab8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fe00a2a70 con 0x7f5fe009d630 2023-10-03T10:13:21.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.550+0000 7f5fe4ab8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe00a01d0 con 0x7f5fe0138d80 2023-10-03T10:13:21.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.550+0000 7f5fe4ab8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe009e2b0 con 0x7f5fe00a0a30 2023-10-03T10:13:21.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.550+0000 7f5fde59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fe00a0a30 0x7f5fe00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:21.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.550+0000 7f5fde59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fe00a0a30 0x7f5fe00a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36868/0 (socket says 172.21.15.3:36868) 2023-10-03T10:13:21.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.550+0000 7f5fde59c700 1 -- 172.21.15.3:0/3318232182 learned_addr learned my addr 172.21.15.3:0/3318232182 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:21.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.551+0000 7f5fded9d700 1 --2- 172.21.15.3:0/3318232182 >> v2:172.21.15.3:3300/0 conn(0x7f5fe0138d80 0x7f5fe013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:21.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.551+0000 7f5fde59c700 1 -- 172.21.15.3:0/3318232182 >> v2:172.21.15.3:3300/0 conn(0x7f5fe0138d80 msgr2=0x7f5fe013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:21.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.551+0000 7f5fde59c700 1 --2- 172.21.15.3:0/3318232182 >> v2:172.21.15.3:3300/0 conn(0x7f5fe0138d80 0x7f5fe013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.551+0000 7f5fde59c700 1 -- 172.21.15.3:0/3318232182 >> v1:172.21.15.3:6789/0 conn(0x7f5fe009d630 legacy=0x7f5fe00a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:21.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.551+0000 7f5fde59c700 1 -- 172.21.15.3:0/3318232182 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe009dc50 con 0x7f5fe00a0a30 2023-10-03T10:13:21.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.551+0000 7f5fded9d700 1 --2- 172.21.15.3:0/3318232182 >> v2:172.21.15.3:3300/0 conn(0x7f5fe0138d80 0x7f5fe013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:21.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.551+0000 7f5fde59c700 1 --2- 172.21.15.3:0/3318232182 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fe00a0a30 0x7f5fe00a0e00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5fd00098f0 tx=0x7f5fd0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c142be1704b7f53 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:21.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.551+0000 7f5fddd9b700 1 -- 172.21.15.3:0/3318232182 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0004230 con 0x7f5fe00a0a30 2023-10-03T10:13:21.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.551+0000 7f5fddd9b700 1 -- 172.21.15.3:0/3318232182 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd0004390 con 0x7f5fe00a0a30 2023-10-03T10:13:21.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.551+0000 7f5fddd9b700 1 -- 172.21.15.3:0/3318232182 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd00036a0 con 0x7f5fe00a0a30 2023-10-03T10:13:21.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.552+0000 7f5fe4ab8700 1 -- 172.21.15.3:0/3318232182 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fe00a0a30 msgr2=0x7f5fe00a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:21.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.552+0000 7f5fe4ab8700 1 --2- 172.21.15.3:0/3318232182 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fe00a0a30 0x7f5fe00a0e00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5fd00098f0 tx=0x7f5fd0009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:21.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.552+0000 7f5fe4ab8700 1 -- 172.21.15.3:0/3318232182 shutdown_connections 2023-10-03T10:13:21.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.552+0000 7f5fe4ab8700 1 --2- 172.21.15.3:0/3318232182 >> v2:172.21.15.3:3300/0 conn(0x7f5fe0138d80 0x7f5fe013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.552+0000 7f5fe4ab8700 1 --2- 172.21.15.3:0/3318232182 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fe00a0a30 0x7f5fe00a0e00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.552+0000 7f5fe4ab8700 1 -- 172.21.15.3:0/3318232182 >> 172.21.15.3:0/3318232182 conn(0x7f5fe00a3050 msgr2=0x7f5fe00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:21.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.552+0000 7f5fe4ab8700 1 -- 172.21.15.3:0/3318232182 shutdown_connections 2023-10-03T10:13:21.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.552+0000 7f5fe4ab8700 1 -- 172.21.15.3:0/3318232182 wait complete. 2023-10-03T10:13:21.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.554+0000 7f5fe4ab8700 1 Processor -- start 2023-10-03T10:13:21.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.554+0000 7f5fe4ab8700 1 -- start start 2023-10-03T10:13:21.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.554+0000 7f5fe4ab8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fe009d630 0x7f5fe01207c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:21.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.554+0000 7f5fe4ab8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fe00a0a30 0x7f5fe0120d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:21.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.554+0000 7f5fe4ab8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fe0138d80 0x7f5fe0121240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:21.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.554+0000 7f5fe4ab8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe0123140 con 0x7f5fe009d630 2023-10-03T10:13:21.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.554+0000 7f5fe4ab8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe0122340 con 0x7f5fe0138d80 2023-10-03T10:13:21.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.554+0000 7f5fe4ab8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe0123690 con 0x7f5fe00a0a30 2023-10-03T10:13:21.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.554+0000 7f5fd5d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fe00a0a30 0x7f5fe0120d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:21.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.554+0000 7f5fde59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fe009d630 0x7f5fe01207c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:21.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.554+0000 7f5fd5d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fe00a0a30 0x7f5fe0120d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55698/0 (socket says 172.21.15.3:55698) 2023-10-03T10:13:21.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.554+0000 7f5fde59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fe009d630 0x7f5fe01207c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50776/0 (socket says 172.21.15.3:50776) 2023-10-03T10:13:21.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.554+0000 7f5fd5d9b700 1 -- 172.21.15.3:0/3573323250 learned_addr learned my addr 172.21.15.3:0/3573323250 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:21.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.555+0000 7f5fded9d700 1 --2- 172.21.15.3:0/3573323250 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fe0138d80 0x7f5fe0121240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:21.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.555+0000 7f5fde59c700 1 -- 172.21.15.3:0/3573323250 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fe00a0a30 msgr2=0x7f5fe0120d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:21.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.555+0000 7f5fde59c700 1 --2- 172.21.15.3:0/3573323250 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fe00a0a30 0x7f5fe0120d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.555+0000 7f5fde59c700 1 -- 172.21.15.3:0/3573323250 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fe0138d80 msgr2=0x7f5fe0121240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:21.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.555+0000 7f5fde59c700 1 --2- 172.21.15.3:0/3573323250 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fe0138d80 0x7f5fe0121240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.555+0000 7f5fde59c700 1 -- 172.21.15.3:0/3573323250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd8009a70 con 0x7f5fe009d630 2023-10-03T10:13:21.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.555+0000 7f5fded9d700 1 --2- 172.21.15.3:0/3573323250 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fe0138d80 0x7f5fe0121240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:21.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.555+0000 7f5fde59c700 1 --2- 172.21.15.3:0/3573323250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fe009d630 0x7f5fe01207c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5fe00672b0 tx=0x7f5fd0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:21.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.555+0000 7f5fd77fe700 1 -- 172.21.15.3:0/3573323250 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0003ae0 con 0x7f5fe009d630 2023-10-03T10:13:21.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.555+0000 7f5fd77fe700 1 -- 172.21.15.3:0/3573323250 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd001f070 con 0x7f5fe009d630 2023-10-03T10:13:21.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.555+0000 7f5fe4ab8700 1 -- 172.21.15.3:0/3573323250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd0009550 con 0x7f5fe009d630 2023-10-03T10:13:21.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.555+0000 7f5fd77fe700 1 -- 172.21.15.3:0/3573323250 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0003e40 con 0x7f5fe009d630 2023-10-03T10:13:21.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.556+0000 7f5fe4ab8700 1 -- 172.21.15.3:0/3573323250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe00a0410 con 0x7f5fe009d630 2023-10-03T10:13:21.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.556+0000 7f5fd77fe700 1 -- 172.21.15.3:0/3573323250 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5fd0021030 con 0x7f5fe009d630 2023-10-03T10:13:21.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.557+0000 7f5fd77fe700 1 -- 172.21.15.3:0/3573323250 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5fd00409d0 con 0x7f5fe009d630 2023-10-03T10:13:21.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.557+0000 7f5fe4ab8700 1 -- 172.21.15.3:0/3573323250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe0123b90 con 0x7f5fe009d630 2023-10-03T10:13:21.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.562+0000 7f5fd77fe700 1 -- 172.21.15.3:0/3573323250 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5fd00239c0 con 0x7f5fe009d630 2023-10-03T10:13:21.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.615+0000 7fb4c8fa6700 1 Processor -- start 2023-10-03T10:13:21.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.615+0000 7fb4c8fa6700 1 -- start start 2023-10-03T10:13:21.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.615+0000 7fb4c8fa6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb4c409f410 0x7fb4c4136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:21.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.615+0000 7fb4c8fa6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4c4136fe0 0x7fb4c41393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:21.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.615+0000 7fb4c8fa6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c4070940 con 0x7fb4c409e930 2023-10-03T10:13:21.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.615+0000 7fb4c8fa6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c406fd60 con 0x7fb4c4136fe0 2023-10-03T10:13:21.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.615+0000 7fb4c8fa6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c406fa80 con 0x7fb4c409f410 2023-10-03T10:13:21.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.616+0000 7fb4c259c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb4c409e930 0x7fb4c409ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59440/0 (socket says 172.21.15.3:59440) 2023-10-03T10:13:21.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.616+0000 7fb4c259c700 1 -- 172.21.15.3:0/3469087969 learned_addr learned my addr 172.21.15.3:0/3469087969 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:21.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.616+0000 7fb4c1d9b700 1 --2- 172.21.15.3:0/3469087969 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb4c409f410 0x7fb4c4136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:21.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.616+0000 7fb4c2d9d700 1 --2- 172.21.15.3:0/3469087969 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4c4136fe0 0x7fb4c41393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:21.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.616+0000 7fb4c159a700 1 -- 172.21.15.3:0/3469087969 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4272743258 0 0) 0x7fb4c4070940 con 0x7fb4c409e930 2023-10-03T10:13:21.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.616+0000 7fb4c159a700 1 -- 172.21.15.3:0/3469087969 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4ac0035b0 con 0x7fb4c409e930 2023-10-03T10:13:21.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.616+0000 7fb4c2d9d700 1 -- 172.21.15.3:0/3469087969 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb4c409f410 msgr2=0x7fb4c4136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:21.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.616+0000 7fb4c2d9d700 1 --2- 172.21.15.3:0/3469087969 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb4c409f410 0x7fb4c4136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.616+0000 7fb4c2d9d700 1 -- 172.21.15.3:0/3469087969 >> v1:172.21.15.3:6789/0 conn(0x7fb4c409e930 legacy=0x7fb4c409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:21.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.616+0000 7fb4c2d9d700 1 -- 172.21.15.3:0/3469087969 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c409fb60 con 0x7fb4c4136fe0 2023-10-03T10:13:21.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.617+0000 7fb4c2d9d700 1 --2- 172.21.15.3:0/3469087969 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4c4136fe0 0x7fb4c41393c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009dc0 tx=0x7fb4bc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b04190b08008d9a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:21.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.617+0000 7fb4c159a700 1 -- 172.21.15.3:0/3469087969 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc00e070 con 0x7fb4c4136fe0 2023-10-03T10:13:21.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.617+0000 7fb4c159a700 1 -- 172.21.15.3:0/3469087969 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4bc00bb20 con 0x7fb4c4136fe0 2023-10-03T10:13:21.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.617+0000 7fb4c159a700 1 -- 172.21.15.3:0/3469087969 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc012430 con 0x7fb4c4136fe0 2023-10-03T10:13:21.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.618+0000 7fb4c8fa6700 1 -- 172.21.15.3:0/3469087969 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4c4136fe0 msgr2=0x7fb4c41393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:21.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.618+0000 7fb4c8fa6700 1 --2- 172.21.15.3:0/3469087969 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4c4136fe0 0x7fb4c41393c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009dc0 tx=0x7fb4bc00b330 comp rx=0 tx=0).stop 2023-10-03T10:13:21.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.618+0000 7fb4c8fa6700 1 -- 172.21.15.3:0/3469087969 shutdown_connections 2023-10-03T10:13:21.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.618+0000 7fb4c8fa6700 1 --2- 172.21.15.3:0/3469087969 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4c4136fe0 0x7fb4c41393c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.618+0000 7fb4c8fa6700 1 --2- 172.21.15.3:0/3469087969 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb4c409f410 0x7fb4c4136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.618+0000 7fb4c8fa6700 1 -- 172.21.15.3:0/3469087969 >> 172.21.15.3:0/3469087969 conn(0x7fb4c40a4210 msgr2=0x7fb4c40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:21.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.618+0000 7fb4c8fa6700 1 -- 172.21.15.3:0/3469087969 shutdown_connections 2023-10-03T10:13:21.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.618+0000 7fb4c8fa6700 1 -- 172.21.15.3:0/3469087969 wait complete. 2023-10-03T10:13:21.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.619+0000 7fb4c8fa6700 1 Processor -- start 2023-10-03T10:13:21.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.620+0000 7fb4c8fa6700 1 -- start start 2023-10-03T10:13:21.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.620+0000 7fb4c8fa6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb4c409e930 0x7fb4c411e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:21.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.620+0000 7fb4c8fa6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4c409f410 0x7fb4c411ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:21.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.620+0000 7fb4c8fa6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4c4136fe0 0x7fb4c411f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:21.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.620+0000 7fb4c8fa6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c4134540 con 0x7fb4c4136fe0 2023-10-03T10:13:21.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.620+0000 7fb4c8fa6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c40a3310 con 0x7fb4c409f410 2023-10-03T10:13:21.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.620+0000 7fb4c8fa6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c4071d20 con 0x7fb4c409e930 2023-10-03T10:13:21.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.621+0000 7fb4c2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4c4136fe0 0x7fb4c411f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:21.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.621+0000 7fb4c2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4c4136fe0 0x7fb4c411f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50790/0 (socket says 172.21.15.3:50790) 2023-10-03T10:13:21.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.621+0000 7fb4c2d9d700 1 -- 172.21.15.3:0/2207501088 learned_addr learned my addr 172.21.15.3:0/2207501088 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:21.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.621+0000 7fb4c259c700 1 --2- 172.21.15.3:0/2207501088 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb4c409e930 0x7fb4c411e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:21.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.621+0000 7fb4c1d9b700 1 --2- 172.21.15.3:0/2207501088 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4c409f410 0x7fb4c411ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:21.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.621+0000 7fb4c2d9d700 1 -- 172.21.15.3:0/2207501088 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb4c409e930 msgr2=0x7fb4c411e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:21.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.621+0000 7fb4c2d9d700 1 --2- 172.21.15.3:0/2207501088 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb4c409e930 0x7fb4c411e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.621+0000 7fb4c2d9d700 1 -- 172.21.15.3:0/2207501088 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4c409f410 msgr2=0x7fb4c411ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:21.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.621+0000 7fb4c2d9d700 1 --2- 172.21.15.3:0/2207501088 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4c409f410 0x7fb4c411ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.621+0000 7fb4c2d9d700 1 -- 172.21.15.3:0/2207501088 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4bc009a70 con 0x7fb4c4136fe0 2023-10-03T10:13:21.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.621+0000 7fb4c1d9b700 1 --2- 172.21.15.3:0/2207501088 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4c409f410 0x7fb4c411ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:21.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.621+0000 7fb4c2d9d700 1 --2- 172.21.15.3:0/2207501088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4c4136fe0 0x7fb4c411f1f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc014040 tx=0x7fb4bc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:21.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.622+0000 7fb4b2ffd700 1 -- 172.21.15.3:0/2207501088 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc0048b0 con 0x7fb4c4136fe0 2023-10-03T10:13:21.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.622+0000 7fb4b2ffd700 1 -- 172.21.15.3:0/2207501088 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4bc020070 con 0x7fb4c4136fe0 2023-10-03T10:13:21.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.622+0000 7fb4b2ffd700 1 -- 172.21.15.3:0/2207501088 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc0276c0 con 0x7fb4c4136fe0 2023-10-03T10:13:21.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.622+0000 7fb4c8fa6700 1 -- 172.21.15.3:0/2207501088 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c4120ac0 con 0x7fb4c4136fe0 2023-10-03T10:13:21.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.622+0000 7fb4c8fa6700 1 -- 172.21.15.3:0/2207501088 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c4121e90 con 0x7fb4c4136fe0 2023-10-03T10:13:21.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.623+0000 7fb4b2ffd700 1 -- 172.21.15.3:0/2207501088 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb4bc012430 con 0x7fb4c4136fe0 2023-10-03T10:13:21.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.623+0000 7fb4b2ffd700 1 -- 172.21.15.3:0/2207501088 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb4bc022070 con 0x7fb4c4136fe0 2023-10-03T10:13:21.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.623+0000 7fb4c8fa6700 1 -- 172.21.15.3:0/2207501088 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c4122080 con 0x7fb4c4136fe0 2023-10-03T10:13:21.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.628+0000 7fb4b2ffd700 1 -- 172.21.15.3:0/2207501088 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb4bc02c080 con 0x7fb4c4136fe0 2023-10-03T10:13:21.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.695+0000 7f85a0e41700 1 -- 172.21.15.3:0/3653898849 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f859c074ea0 con 0x7f859c09e000 2023-10-03T10:13:21.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.695+0000 7f858affd700 1 -- 172.21.15.3:0/3653898849 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8590012370 con 0x7f859c09e000 2023-10-03T10:13:21.696 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:21.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.698+0000 7f85a0e41700 1 -- 172.21.15.3:0/3653898849 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f859c09e000 msgr2=0x7f859c11e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:21.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.698+0000 7f85a0e41700 1 --2- 172.21.15.3:0/3653898849 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f859c09e000 0x7f859c11e560 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f859000e840 tx=0x7f859000eb50 comp rx=0 tx=0).stop 2023-10-03T10:13:21.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.698+0000 7f85a0e41700 1 -- 172.21.15.3:0/3653898849 shutdown_connections 2023-10-03T10:13:21.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.698+0000 7f85a0e41700 1 --2- 172.21.15.3:0/3653898849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c09e410 0x7f859c11eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.698+0000 7f85a0e41700 1 --2- 172.21.15.3:0/3653898849 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f859c09d6f0 0x7f859c11e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.698+0000 7f85a0e41700 1 --2- 172.21.15.3:0/3653898849 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f859c09e000 0x7f859c11e560 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.698+0000 7f85a0e41700 1 -- 172.21.15.3:0/3653898849 >> 172.21.15.3:0/3653898849 conn(0x7f859c066f50 msgr2=0x7f859c130b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:21.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.698+0000 7f85a0e41700 1 -- 172.21.15.3:0/3653898849 shutdown_connections 2023-10-03T10:13:21.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.698+0000 7f85a0e41700 1 -- 172.21.15.3:0/3653898849 wait complete. 2023-10-03T10:13:21.712 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:21.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.771+0000 7fa341646700 1 -- 172.21.15.3:0/488402291 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa33c074ea0 con 0x7fa33c09e1d0 2023-10-03T10:13:21.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.772+0000 7fa3237fe700 1 -- 172.21.15.3:0/488402291 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa33002b090 con 0x7fa33c09e1d0 2023-10-03T10:13:21.772 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:21.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.774+0000 7fa341646700 1 -- 172.21.15.3:0/488402291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa33c09e1d0 msgr2=0x7fa33c11ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:21.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.774+0000 7fa341646700 1 --2- 172.21.15.3:0/488402291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa33c09e1d0 0x7fa33c11ee50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa33000e810 tx=0x7fa33000eb20 comp rx=0 tx=0).stop 2023-10-03T10:13:21.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.774+0000 7fa341646700 1 -- 172.21.15.3:0/488402291 shutdown_connections 2023-10-03T10:13:21.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.774+0000 7fa341646700 1 --2- 172.21.15.3:0/488402291 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa33c139170 0x7fa33c11f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.774+0000 7fa341646700 1 --2- 172.21.15.3:0/488402291 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa33c09d6f0 0x7fa33c11e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.774+0000 7fa341646700 1 --2- 172.21.15.3:0/488402291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa33c09e1d0 0x7fa33c11ee50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.774+0000 7fa341646700 1 -- 172.21.15.3:0/488402291 >> 172.21.15.3:0/488402291 conn(0x7fa33c066f50 msgr2=0x7fa33c136cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:21.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.774+0000 7f5fe4ab8700 1 -- 172.21.15.3:0/3573323250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5fe0123b90 con 0x7f5fe009d630 2023-10-03T10:13:21.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.774+0000 7fa341646700 1 -- 172.21.15.3:0/488402291 shutdown_connections 2023-10-03T10:13:21.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.774+0000 7fa341646700 1 -- 172.21.15.3:0/488402291 wait complete. 2023-10-03T10:13:21.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.775+0000 7f5fd77fe700 1 -- 172.21.15.3:0/3573323250 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5fd001b440 con 0x7f5fe009d630 2023-10-03T10:13:21.776 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:21.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.777+0000 7f5fe4ab8700 1 -- 172.21.15.3:0/3573323250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fe009d630 msgr2=0x7f5fe01207c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:21.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.777+0000 7f5fe4ab8700 1 --2- 172.21.15.3:0/3573323250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fe009d630 0x7f5fe01207c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5fe00672b0 tx=0x7f5fd0004750 comp rx=0 tx=0).stop 2023-10-03T10:13:21.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.777+0000 7f5fe4ab8700 1 -- 172.21.15.3:0/3573323250 shutdown_connections 2023-10-03T10:13:21.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.777+0000 7f5fe4ab8700 1 --2- 172.21.15.3:0/3573323250 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fe0138d80 0x7f5fe0121240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.777+0000 7f5fe4ab8700 1 --2- 172.21.15.3:0/3573323250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fe009d630 0x7f5fe01207c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.777+0000 7f5fe4ab8700 1 --2- 172.21.15.3:0/3573323250 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fe00a0a30 0x7f5fe0120d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.777+0000 7f5fe4ab8700 1 -- 172.21.15.3:0/3573323250 >> 172.21.15.3:0/3573323250 conn(0x7f5fe00a3050 msgr2=0x7f5fe009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:21.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.777+0000 7f5fe4ab8700 1 -- 172.21.15.3:0/3573323250 shutdown_connections 2023-10-03T10:13:21.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.777+0000 7f5fe4ab8700 1 -- 172.21.15.3:0/3573323250 wait complete. 2023-10-03T10:13:21.790 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:21.793 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:21.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.838+0000 7fb4c8fa6700 1 -- 172.21.15.3:0/2207501088 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb4c4074ea0 con 0x7fb4c4136fe0 2023-10-03T10:13:21.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.839+0000 7fb4b2ffd700 1 -- 172.21.15.3:0/2207501088 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb4bc003e30 con 0x7fb4c4136fe0 2023-10-03T10:13:21.839 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:21.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.841+0000 7fb4c8fa6700 1 -- 172.21.15.3:0/2207501088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4c4136fe0 msgr2=0x7fb4c411f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:21.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.841+0000 7fb4c8fa6700 1 --2- 172.21.15.3:0/2207501088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4c4136fe0 0x7fb4c411f1f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc014040 tx=0x7fb4bc004ad0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.841+0000 7fb4c8fa6700 1 -- 172.21.15.3:0/2207501088 shutdown_connections 2023-10-03T10:13:21.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.841+0000 7fb4c8fa6700 1 --2- 172.21.15.3:0/2207501088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4c4136fe0 0x7fb4c411f1f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.841+0000 7fb4c8fa6700 1 --2- 172.21.15.3:0/2207501088 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb4c409e930 0x7fb4c411e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.841+0000 7fb4c8fa6700 1 --2- 172.21.15.3:0/2207501088 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4c409f410 0x7fb4c411ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:21.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.841+0000 7fb4c8fa6700 1 -- 172.21.15.3:0/2207501088 >> 172.21.15.3:0/2207501088 conn(0x7fb4c40a4210 msgr2=0x7fb4c4134900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:21.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.841+0000 7fb4c8fa6700 1 -- 172.21.15.3:0/2207501088 shutdown_connections 2023-10-03T10:13:21.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:21.841+0000 7fb4c8fa6700 1 -- 172.21.15.3:0/2207501088 wait complete. 2023-10-03T10:13:21.855 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:22.713 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:22.792 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:22.794 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:22.855 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:22.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.864+0000 7fe176899700 1 Processor -- start 2023-10-03T10:13:22.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.864+0000 7fe176899700 1 -- start start 2023-10-03T10:13:22.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.864+0000 7fe176899700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe170131810 0x7fe170131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:22.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.864+0000 7fe176899700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe170099a40 0x7fe170099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:22.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.864+0000 7fe176899700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe17009a3b0 0x7fe170138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:22.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.864+0000 7fe176899700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1701313c0 con 0x7fe170131810 2023-10-03T10:13:22.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.864+0000 7fe176899700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17006e8c0 con 0x7fe170099a40 2023-10-03T10:13:22.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.864+0000 7fe176899700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe170071d20 con 0x7fe17009a3b0 2023-10-03T10:13:22.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.865+0000 7fe174e36700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe17009a3b0 0x7fe170138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:22.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.865+0000 7fe174e36700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe17009a3b0 0x7fe170138650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55712/0 (socket says 172.21.15.3:55712) 2023-10-03T10:13:22.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.865+0000 7fe174e36700 1 -- 172.21.15.3:0/1801010254 learned_addr learned my addr 172.21.15.3:0/1801010254 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:22.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.865+0000 7fe174e36700 1 -- 172.21.15.3:0/1801010254 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe170099a40 msgr2=0x7fe170099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:22.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.865+0000 7fe174e36700 1 --2- 172.21.15.3:0/1801010254 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe170099a40 0x7fe170099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:22.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.865+0000 7fe174e36700 1 -- 172.21.15.3:0/1801010254 >> v2:172.21.15.3:3300/0 conn(0x7fe170131810 msgr2=0x7fe170131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:22.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.865+0000 7fe174e36700 1 --2- 172.21.15.3:0/1801010254 >> v2:172.21.15.3:3300/0 conn(0x7fe170131810 0x7fe170131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:22.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.865+0000 7fe174e36700 1 -- 172.21.15.3:0/1801010254 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1700721a0 con 0x7fe17009a3b0 2023-10-03T10:13:22.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.865+0000 7fe174e36700 1 --2- 172.21.15.3:0/1801010254 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe17009a3b0 0x7fe170138650 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe168009e50 tx=0x7fe16800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=428bd3254978428d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:22.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.865+0000 7fe16effd700 1 -- 172.21.15.3:0/1801010254 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16800e070 con 0x7fe17009a3b0 2023-10-03T10:13:22.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.866+0000 7fe16effd700 1 -- 172.21.15.3:0/1801010254 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16800b9c0 con 0x7fe17009a3b0 2023-10-03T10:13:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.866+0000 7fe16effd700 1 -- 172.21.15.3:0/1801010254 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16800bd70 con 0x7fe17009a3b0 2023-10-03T10:13:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.866+0000 7fe176899700 1 -- 172.21.15.3:0/1801010254 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe17009a3b0 msgr2=0x7fe170138650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.866+0000 7fe176899700 1 --2- 172.21.15.3:0/1801010254 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe17009a3b0 0x7fe170138650 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe168009e50 tx=0x7fe16800b330 comp rx=0 tx=0).stop 2023-10-03T10:13:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.867+0000 7fe176899700 1 -- 172.21.15.3:0/1801010254 shutdown_connections 2023-10-03T10:13:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.867+0000 7fe176899700 1 --2- 172.21.15.3:0/1801010254 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe17009a3b0 0x7fe170138650 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.867+0000 7fe176899700 1 --2- 172.21.15.3:0/1801010254 >> v2:172.21.15.3:3300/0 conn(0x7fe170131810 0x7fe170131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.867+0000 7fe176899700 1 --2- 172.21.15.3:0/1801010254 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe170099a40 0x7fe170099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.867+0000 7fe176899700 1 -- 172.21.15.3:0/1801010254 >> 172.21.15.3:0/1801010254 conn(0x7fe1700a2aa0 msgr2=0x7fe1700a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.867+0000 7fe176899700 1 -- 172.21.15.3:0/1801010254 shutdown_connections 2023-10-03T10:13:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.867+0000 7fe176899700 1 -- 172.21.15.3:0/1801010254 wait complete. 2023-10-03T10:13:22.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.868+0000 7fe176899700 1 Processor -- start 2023-10-03T10:13:22.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.868+0000 7fe176899700 1 -- start start 2023-10-03T10:13:22.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.869+0000 7fe176899700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe170099a40 0x7fe17006f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:22.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.869+0000 7fe176899700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe17009a3b0 0x7fe17006fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:22.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.869+0000 7fe176899700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe170131810 0x7fe170070060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:22.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.869+0000 7fe176899700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17011d610 con 0x7fe170099a40 2023-10-03T10:13:22.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.869+0000 7fe176899700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17006eb00 con 0x7fe170131810 2023-10-03T10:13:22.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.869+0000 7fe176899700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe17011dd00 con 0x7fe17009a3b0 2023-10-03T10:13:22.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.869+0000 7fe174e36700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe170131810 0x7fe170070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:22.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.869+0000 7fe16ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe170099a40 0x7fe17006f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:22.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.869+0000 7fe174e36700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe170131810 0x7fe170070060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36898/0 (socket says 172.21.15.3:36898) 2023-10-03T10:13:22.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.869+0000 7fe174e36700 1 -- 172.21.15.3:0/1442580996 learned_addr learned my addr 172.21.15.3:0/1442580996 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:22.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.870+0000 7fe16f7fe700 1 --2- 172.21.15.3:0/1442580996 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe17009a3b0 0x7fe17006fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:22.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.870+0000 7fe16ffff700 1 -- 172.21.15.3:0/1442580996 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe17009a3b0 msgr2=0x7fe17006fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:22.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.870+0000 7fe16ffff700 1 --2- 172.21.15.3:0/1442580996 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe17009a3b0 0x7fe17006fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:22.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.870+0000 7fe16ffff700 1 -- 172.21.15.3:0/1442580996 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe170131810 msgr2=0x7fe170070060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:22.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.870+0000 7fe16ffff700 1 --2- 172.21.15.3:0/1442580996 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe170131810 0x7fe170070060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:22.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.870+0000 7fe16ffff700 1 -- 172.21.15.3:0/1442580996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe168009b30 con 0x7fe170099a40 2023-10-03T10:13:22.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.870+0000 7fe174e36700 1 --2- 172.21.15.3:0/1442580996 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe170131810 0x7fe170070060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:22.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.870+0000 7fe16f7fe700 1 --2- 172.21.15.3:0/1442580996 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe17009a3b0 0x7fe17006fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:22.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.870+0000 7fe16ffff700 1 --2- 172.21.15.3:0/1442580996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe170099a40 0x7fe17006f5e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe16000b480 tx=0x7fe16000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.871+0000 7fe16cff9700 1 -- 172.21.15.3:0/1442580996 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe160011830 con 0x7fe170099a40 2023-10-03T10:13:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.871+0000 7fe16cff9700 1 -- 172.21.15.3:0/1442580996 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe160011990 con 0x7fe170099a40 2023-10-03T10:13:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.871+0000 7fe16cff9700 1 -- 172.21.15.3:0/1442580996 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe160011d40 con 0x7fe170099a40 2023-10-03T10:13:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.871+0000 7fe176899700 1 -- 172.21.15.3:0/1442580996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe170070c20 con 0x7fe170099a40 2023-10-03T10:13:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.871+0000 7fe176899700 1 -- 172.21.15.3:0/1442580996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe17011ec10 con 0x7fe170099a40 2023-10-03T10:13:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.873+0000 7fe16cff9700 1 -- 172.21.15.3:0/1442580996 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe16001a400 con 0x7fe170099a40 2023-10-03T10:13:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.873+0000 7fe176899700 1 -- 172.21.15.3:0/1442580996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe170074ea0 con 0x7fe170099a40 2023-10-03T10:13:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.873+0000 7fe16cff9700 1 -- 172.21.15.3:0/1442580996 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe16003f590 con 0x7fe170099a40 2023-10-03T10:13:22.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.882+0000 7fe16cff9700 1 -- 172.21.15.3:0/1442580996 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe160009280 con 0x7fe170099a40 2023-10-03T10:13:22.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.941+0000 7fb7e95a8700 1 Processor -- start 2023-10-03T10:13:22.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.941+0000 7fb7e95a8700 1 -- start start 2023-10-03T10:13:22.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.941+0000 7fb7e95a8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb7e409d6f0 0x7fb7e409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:22.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.942+0000 7fb7e95a8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7e409e000 0x7fb7e41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:22.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.942+0000 7fb7e95a8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7e409e410 0x7fb7e4133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:22.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.942+0000 7fb7e95a8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7e406b460 con 0x7fb7e409d6f0 2023-10-03T10:13:22.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.942+0000 7fb7e95a8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e409b120 con 0x7fb7e409e410 2023-10-03T10:13:22.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.942+0000 7fb7e95a8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e409a7e0 con 0x7fb7e409e000 2023-10-03T10:13:22.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.942+0000 7fb7e27fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7e409e000 0x7fb7e41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:22.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.942+0000 7fb7e37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7e409e410 0x7fb7e4133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:22.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.942+0000 7fb7e27fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7e409e000 0x7fb7e41339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55734/0 (socket says 172.21.15.3:55734) 2023-10-03T10:13:22.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.942+0000 7fb7e37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7e409e410 0x7fb7e4133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36914/0 (socket says 172.21.15.3:36914) 2023-10-03T10:13:22.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.942+0000 7fb7e27fc700 1 -- 172.21.15.3:0/3224938455 learned_addr learned my addr 172.21.15.3:0/3224938455 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:22.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.942+0000 7fb7e2ffd700 1 --2- 172.21.15.3:0/3224938455 >> v2:172.21.15.3:3300/0 conn(0x7fb7e409d6f0 0x7fb7e409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:22.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.942+0000 7fb7e27fc700 1 -- 172.21.15.3:0/3224938455 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7e409e410 msgr2=0x7fb7e4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:22.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.942+0000 7fb7e27fc700 1 --2- 172.21.15.3:0/3224938455 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7e409e410 0x7fb7e4133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:22.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.942+0000 7fb7e27fc700 1 -- 172.21.15.3:0/3224938455 >> v2:172.21.15.3:3300/0 conn(0x7fb7e409d6f0 msgr2=0x7fb7e409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:22.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.942+0000 7fb7e27fc700 1 --2- 172.21.15.3:0/3224938455 >> v2:172.21.15.3:3300/0 conn(0x7fb7e409d6f0 0x7fb7e409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:22.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.942+0000 7fb7e27fc700 1 -- 172.21.15.3:0/3224938455 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e409a500 con 0x7fb7e409e000 2023-10-03T10:13:22.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.942+0000 7fb7e2ffd700 1 --2- 172.21.15.3:0/3224938455 >> v2:172.21.15.3:3300/0 conn(0x7fb7e409d6f0 0x7fb7e409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:22.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.942+0000 7fb7e37fe700 1 --2- 172.21.15.3:0/3224938455 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7e409e410 0x7fb7e4133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:22.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.943+0000 7fb7e27fc700 1 --2- 172.21.15.3:0/3224938455 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7e409e000 0x7fb7e41339f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8009920 tx=0x7fb7d8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c22f4701530360f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:22.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.943+0000 7fb7e1ffb700 1 -- 172.21.15.3:0/3224938455 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8011830 con 0x7fb7e409e000 2023-10-03T10:13:22.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.943+0000 7fb7e1ffb700 1 -- 172.21.15.3:0/3224938455 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d8011990 con 0x7fb7e409e000 2023-10-03T10:13:22.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.943+0000 7fb7e1ffb700 1 -- 172.21.15.3:0/3224938455 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8011d20 con 0x7fb7e409e000 2023-10-03T10:13:22.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.944+0000 7fb7e95a8700 1 -- 172.21.15.3:0/3224938455 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7e409e000 msgr2=0x7fb7e41339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:22.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.944+0000 7fb7e95a8700 1 --2- 172.21.15.3:0/3224938455 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7e409e000 0x7fb7e41339f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8009920 tx=0x7fb7d8009c30 comp rx=0 tx=0).stop 2023-10-03T10:13:22.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.944+0000 7fb7e95a8700 1 -- 172.21.15.3:0/3224938455 shutdown_connections 2023-10-03T10:13:22.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.944+0000 7fb7e95a8700 1 --2- 172.21.15.3:0/3224938455 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7e409e410 0x7fb7e4133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:22.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.944+0000 7fb7e95a8700 1 --2- 172.21.15.3:0/3224938455 >> v2:172.21.15.3:3300/0 conn(0x7fb7e409d6f0 0x7fb7e409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:22.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.944+0000 7fb7e95a8700 1 --2- 172.21.15.3:0/3224938455 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7e409e000 0x7fb7e41339f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:22.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.944+0000 7fb7e95a8700 1 -- 172.21.15.3:0/3224938455 >> 172.21.15.3:0/3224938455 conn(0x7fb7e4066f50 msgr2=0x7fb7e409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:22.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.944+0000 7fb7e95a8700 1 -- 172.21.15.3:0/3224938455 shutdown_connections 2023-10-03T10:13:22.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.944+0000 7fb7e95a8700 1 -- 172.21.15.3:0/3224938455 wait complete. 2023-10-03T10:13:22.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.945+0000 7fb7e95a8700 1 Processor -- start 2023-10-03T10:13:22.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.945+0000 7fb7e95a8700 1 -- start start 2023-10-03T10:13:22.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.946+0000 7fb7e95a8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7e409d6f0 0x7fb7e41abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:22.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.946+0000 7fb7e95a8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7e409e000 0x7fb7e41ac130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:22.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.946+0000 7fb7e95a8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7e409e410 0x7fb7e41ac670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:22.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.946+0000 7fb7e95a8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e41af710 con 0x7fb7e409e410 2023-10-03T10:13:22.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.946+0000 7fb7e95a8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e41ad030 con 0x7fb7e409e000 2023-10-03T10:13:22.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.946+0000 7fb7e95a8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e41ad800 con 0x7fb7e409d6f0 2023-10-03T10:13:22.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.946+0000 7fb7e2ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7e409d6f0 0x7fb7e41abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:22.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.946+0000 7fb7e2ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7e409d6f0 0x7fb7e41abbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55746/0 (socket says 172.21.15.3:55746) 2023-10-03T10:13:22.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.946+0000 7fb7e2ffd700 1 -- 172.21.15.3:0/129257924 learned_addr learned my addr 172.21.15.3:0/129257924 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:22.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.946+0000 7fb7e37fe700 1 --2- 172.21.15.3:0/129257924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7e409e410 0x7fb7e41ac670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:22.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.946+0000 7fb7e27fc700 1 --2- 172.21.15.3:0/129257924 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7e409e000 0x7fb7e41ac130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:22.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.946+0000 7fb7e37fe700 1 -- 172.21.15.3:0/129257924 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7e409d6f0 msgr2=0x7fb7e41abbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:22.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.946+0000 7fb7e37fe700 1 --2- 172.21.15.3:0/129257924 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7e409d6f0 0x7fb7e41abbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:22.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.946+0000 7fb7e37fe700 1 -- 172.21.15.3:0/129257924 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7e409e000 msgr2=0x7fb7e41ac130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:22.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.946+0000 7fb7e37fe700 1 --2- 172.21.15.3:0/129257924 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7e409e000 0x7fb7e41ac130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:22.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.946+0000 7fb7e37fe700 1 -- 172.21.15.3:0/129257924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d40094a0 con 0x7fb7e409e410 2023-10-03T10:13:22.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.946+0000 7fb7e2ffd700 1 --2- 172.21.15.3:0/129257924 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7e409d6f0 0x7fb7e41abbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:22.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.947+0000 7fb7e37fe700 1 --2- 172.21.15.3:0/129257924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7e409e410 0x7fb7e41ac670 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc009e50 tx=0x7fb7dc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:22.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.947+0000 7fb7cb7fe700 1 -- 172.21.15.3:0/129257924 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc00ca90 con 0x7fb7e409e410 2023-10-03T10:13:22.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.947+0000 7fb7cb7fe700 1 -- 172.21.15.3:0/129257924 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7dc00f040 con 0x7fb7e409e410 2023-10-03T10:13:22.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.947+0000 7fb7e95a8700 1 -- 172.21.15.3:0/129257924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7dc009a70 con 0x7fb7e409e410 2023-10-03T10:13:22.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.947+0000 7fb7cb7fe700 1 -- 172.21.15.3:0/129257924 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc00ce30 con 0x7fb7e409e410 2023-10-03T10:13:22.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.947+0000 7fb7e95a8700 1 -- 172.21.15.3:0/129257924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d8009550 con 0x7fb7e409e410 2023-10-03T10:13:22.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.948+0000 7fb7cb7fe700 1 -- 172.21.15.3:0/129257924 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb7dc014480 con 0x7fb7e409e410 2023-10-03T10:13:22.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.948+0000 7fb7cb7fe700 1 -- 172.21.15.3:0/129257924 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb7dc03fc30 con 0x7fb7e409e410 2023-10-03T10:13:22.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.948+0000 7fb7e95a8700 1 -- 172.21.15.3:0/129257924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e40a39b0 con 0x7fb7e409e410 2023-10-03T10:13:22.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.956+0000 7f33f9fd8700 1 Processor -- start 2023-10-03T10:13:22.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.956+0000 7f33f9fd8700 1 -- start start 2023-10-03T10:13:22.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.956+0000 7fb7cb7fe700 1 -- 172.21.15.3:0/129257924 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb7dc012930 con 0x7fb7e409e410 2023-10-03T10:13:22.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.957+0000 7f33f9fd8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f33f4130540 0x7f33f4130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:22.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.957+0000 7f33f9fd8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33f4130e50 0x7f33f406f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:22.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.957+0000 7f33f9fd8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f33f406fb20 0x7f33f406ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:22.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.957+0000 7f33f9fd8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33f409a390 con 0x7f33f406fb20 2023-10-03T10:13:22.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.957+0000 7f33f9fd8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f40716a0 con 0x7f33f4130e50 2023-10-03T10:13:22.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.957+0000 7f33f9fd8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f33f4071290 con 0x7f33f4130540 2023-10-03T10:13:22.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.957+0000 7f33f37fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f33f4130540 0x7f33f4130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:22.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.957+0000 7f33f37fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f33f4130540 0x7f33f4130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55758/0 (socket says 172.21.15.3:55758) 2023-10-03T10:13:22.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.957+0000 7f33f2ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33f4130e50 0x7f33f406f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:22.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.957+0000 7f33f37fe700 1 -- 172.21.15.3:0/1895084548 learned_addr learned my addr 172.21.15.3:0/1895084548 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:22.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.957+0000 7f33f3fff700 1 --2- 172.21.15.3:0/1895084548 >> v2:172.21.15.3:3300/0 conn(0x7f33f406fb20 0x7f33f406ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:22.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.958+0000 7f33f2ffd700 1 -- 172.21.15.3:0/1895084548 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f33f4130540 msgr2=0x7f33f4130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:22.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.958+0000 7f33f2ffd700 1 --2- 172.21.15.3:0/1895084548 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f33f4130540 0x7f33f4130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:22.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.958+0000 7f33f2ffd700 1 -- 172.21.15.3:0/1895084548 >> v2:172.21.15.3:3300/0 conn(0x7f33f406fb20 msgr2=0x7f33f406ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:22.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.958+0000 7f33f2ffd700 1 --2- 172.21.15.3:0/1895084548 >> v2:172.21.15.3:3300/0 conn(0x7f33f406fb20 0x7f33f406ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:22.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.958+0000 7f33f2ffd700 1 -- 172.21.15.3:0/1895084548 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f4130340 con 0x7f33f4130e50 2023-10-03T10:13:22.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.958+0000 7f33f37fe700 1 --2- 172.21.15.3:0/1895084548 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f33f4130540 0x7f33f4130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:22.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.958+0000 7f33f2ffd700 1 --2- 172.21.15.3:0/1895084548 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33f4130e50 0x7f33f406f5e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f33e800dc90 tx=0x7f33e800f050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd227be2716019e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:22.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.958+0000 7f33f27fc700 1 -- 172.21.15.3:0/1895084548 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e8012070 con 0x7f33f4130e50 2023-10-03T10:13:22.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.958+0000 7f33f27fc700 1 -- 172.21.15.3:0/1895084548 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e800d2e0 con 0x7f33f4130e50 2023-10-03T10:13:22.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.958+0000 7f33f27fc700 1 -- 172.21.15.3:0/1895084548 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e800f8b0 con 0x7f33f4130e50 2023-10-03T10:13:22.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.959+0000 7f33f9fd8700 1 -- 172.21.15.3:0/1895084548 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33f4130e50 msgr2=0x7f33f406f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:22.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.959+0000 7f33f9fd8700 1 --2- 172.21.15.3:0/1895084548 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33f4130e50 0x7f33f406f5e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f33e800dc90 tx=0x7f33e800f050 comp rx=0 tx=0).stop 2023-10-03T10:13:22.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.959+0000 7f33f9fd8700 1 -- 172.21.15.3:0/1895084548 shutdown_connections 2023-10-03T10:13:22.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.959+0000 7f33f9fd8700 1 --2- 172.21.15.3:0/1895084548 >> v2:172.21.15.3:3300/0 conn(0x7f33f406fb20 0x7f33f406ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:22.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.959+0000 7f33f9fd8700 1 --2- 172.21.15.3:0/1895084548 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f33f4130540 0x7f33f4130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:22.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.959+0000 7f33f9fd8700 1 --2- 172.21.15.3:0/1895084548 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33f4130e50 0x7f33f406f5e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:22.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.959+0000 7f33f9fd8700 1 -- 172.21.15.3:0/1895084548 >> 172.21.15.3:0/1895084548 conn(0x7f33f40a2aa0 msgr2=0x7f33f40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:22.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.959+0000 7f33f9fd8700 1 -- 172.21.15.3:0/1895084548 shutdown_connections 2023-10-03T10:13:22.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.959+0000 7f33f9fd8700 1 -- 172.21.15.3:0/1895084548 wait complete. 2023-10-03T10:13:22.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.961+0000 7f33f9fd8700 1 Processor -- start 2023-10-03T10:13:22.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.961+0000 7f33f9fd8700 1 -- start start 2023-10-03T10:13:22.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.962+0000 7f33f9fd8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f33f406fb20 0x7f33f4123880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:22.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.962+0000 7f33f9fd8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33f4130540 0x7f33f4123dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:22.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.962+0000 7f33f9fd8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33f4130e50 0x7f33f41cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:22.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.962+0000 7f33f9fd8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f40a0a70 con 0x7f33f4130e50 2023-10-03T10:13:22.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.962+0000 7f33f9fd8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f40718e0 con 0x7f33f4130540 2023-10-03T10:13:22.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.962+0000 7f33f9fd8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f33f412c9b0 con 0x7f33f406fb20 2023-10-03T10:13:22.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.962+0000 7f33f3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33f4130e50 0x7f33f41cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:22.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.962+0000 7f33f3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33f4130e50 0x7f33f41cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50830/0 (socket says 172.21.15.3:50830) 2023-10-03T10:13:22.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.962+0000 7f33f3fff700 1 -- 172.21.15.3:0/4128141518 learned_addr learned my addr 172.21.15.3:0/4128141518 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:22.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.962+0000 7f33f37fe700 1 --2- 172.21.15.3:0/4128141518 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f33f406fb20 0x7f33f4123880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:22.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.962+0000 7f33f2ffd700 1 --2- 172.21.15.3:0/4128141518 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33f4130540 0x7f33f4123dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:22.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.962+0000 7f33f2ffd700 1 -- 172.21.15.3:0/4128141518 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f33f406fb20 msgr2=0x7f33f4123880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:22.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.962+0000 7f33f2ffd700 1 --2- 172.21.15.3:0/4128141518 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f33f406fb20 0x7f33f4123880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:22.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.962+0000 7f33f2ffd700 1 -- 172.21.15.3:0/4128141518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33f4130e50 msgr2=0x7f33f41cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:22.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.962+0000 7f33f2ffd700 1 --2- 172.21.15.3:0/4128141518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33f4130e50 0x7f33f41cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:22.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.962+0000 7f33f2ffd700 1 -- 172.21.15.3:0/4128141518 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e400b600 con 0x7f33f4130540 2023-10-03T10:13:22.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.962+0000 7f33f3fff700 1 --2- 172.21.15.3:0/4128141518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33f4130e50 0x7f33f41cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:22.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.963+0000 7f33f2ffd700 1 --2- 172.21.15.3:0/4128141518 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33f4130540 0x7f33f4123dc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f33e8019000 tx=0x7f33e8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:22.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.963+0000 7f33dbfff700 1 -- 172.21.15.3:0/4128141518 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e800b070 con 0x7f33f4130540 2023-10-03T10:13:22.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.963+0000 7f33f9fd8700 1 -- 172.21.15.3:0/4128141518 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e800d940 con 0x7f33f4130540 2023-10-03T10:13:22.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.963+0000 7f33dbfff700 1 -- 172.21.15.3:0/4128141518 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e800f6b0 con 0x7f33f4130540 2023-10-03T10:13:22.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.963+0000 7f33dbfff700 1 -- 172.21.15.3:0/4128141518 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e800fa60 con 0x7f33f4130540 2023-10-03T10:13:22.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.963+0000 7f33f9fd8700 1 -- 172.21.15.3:0/4128141518 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f4124b30 con 0x7f33f4130540 2023-10-03T10:13:22.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.964+0000 7f33dbfff700 1 -- 172.21.15.3:0/4128141518 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f33e800fbc0 con 0x7f33f4130540 2023-10-03T10:13:22.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.964+0000 7f33d9ffb700 1 -- 172.21.15.3:0/4128141518 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33d40052e0 con 0x7f33f4130540 2023-10-03T10:13:22.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.965+0000 7f33dbfff700 1 -- 172.21.15.3:0/4128141518 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f33e803fcf0 con 0x7f33f4130540 2023-10-03T10:13:22.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:22.970+0000 7f33dbfff700 1 -- 172.21.15.3:0/4128141518 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f33e8027410 con 0x7f33f4130540 2023-10-03T10:13:23.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.020+0000 7f6e8a44f700 1 Processor -- start 2023-10-03T10:13:23.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.020+0000 7f6e8a44f700 1 -- start start 2023-10-03T10:13:23.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.021+0000 7f6e8a44f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6e8409e1d0 0x7f6e84135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:23.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.021+0000 7f6e8a44f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6e84135e50 0x7f6e84138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:23.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.021+0000 7f6e8a44f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e8409f130 con 0x7f6e8409d6f0 2023-10-03T10:13:23.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.021+0000 7f6e8a44f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e8409f520 con 0x7f6e8409e1d0 2023-10-03T10:13:23.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.021+0000 7f6e8a44f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8409e930 con 0x7f6e84135e50 2023-10-03T10:13:23.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.021+0000 7f6e889ec700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6e84135e50 0x7f6e84138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:23.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.021+0000 7f6e889ec700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6e84135e50 0x7f6e84138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55772/0 (socket says 172.21.15.3:55772) 2023-10-03T10:13:23.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.021+0000 7f6e83fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6e8409d6f0 0x7f6e8409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59456/0 (socket says 172.21.15.3:59456) 2023-10-03T10:13:23.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.021+0000 7f6e889ec700 1 -- 172.21.15.3:0/809748277 learned_addr learned my addr 172.21.15.3:0/809748277 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:23.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.021+0000 7f6e837fe700 1 --2- 172.21.15.3:0/809748277 >> v2:172.21.15.3:3300/0 conn(0x7f6e8409e1d0 0x7f6e84135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:23.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.022+0000 7f6e889ec700 1 -- 172.21.15.3:0/809748277 >> v2:172.21.15.3:3300/0 conn(0x7f6e8409e1d0 msgr2=0x7f6e84135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:23.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.022+0000 7f6e889ec700 1 --2- 172.21.15.3:0/809748277 >> v2:172.21.15.3:3300/0 conn(0x7f6e8409e1d0 0x7f6e84135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:23.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.022+0000 7f6e889ec700 1 -- 172.21.15.3:0/809748277 >> v1:172.21.15.3:6789/0 conn(0x7f6e8409d6f0 legacy=0x7f6e8409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:23.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.022+0000 7f6e889ec700 1 -- 172.21.15.3:0/809748277 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e8409b2d0 con 0x7f6e84135e50 2023-10-03T10:13:23.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.022+0000 7f6e889ec700 1 --2- 172.21.15.3:0/809748277 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6e84135e50 0x7f6e84138230 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c00bef0 tx=0x7f6e7c009370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5be8464b6d1e5932 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:23.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.022+0000 7f6e82ffd700 1 -- 172.21.15.3:0/809748277 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c00b480 con 0x7f6e84135e50 2023-10-03T10:13:23.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.022+0000 7f6e82ffd700 1 -- 172.21.15.3:0/809748277 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e7c00c950 con 0x7f6e84135e50 2023-10-03T10:13:23.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.022+0000 7f6e82ffd700 1 -- 172.21.15.3:0/809748277 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c00cd00 con 0x7f6e84135e50 2023-10-03T10:13:23.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.023+0000 7f6e8a44f700 1 -- 172.21.15.3:0/809748277 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6e84135e50 msgr2=0x7f6e84138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:23.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.023+0000 7f6e8a44f700 1 --2- 172.21.15.3:0/809748277 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6e84135e50 0x7f6e84138230 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c00bef0 tx=0x7f6e7c009370 comp rx=0 tx=0).stop 2023-10-03T10:13:23.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.023+0000 7f6e8a44f700 1 -- 172.21.15.3:0/809748277 shutdown_connections 2023-10-03T10:13:23.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.023+0000 7f6e8a44f700 1 --2- 172.21.15.3:0/809748277 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6e84135e50 0x7f6e84138230 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:23.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.023+0000 7f6e8a44f700 1 --2- 172.21.15.3:0/809748277 >> v2:172.21.15.3:3300/0 conn(0x7f6e8409e1d0 0x7f6e84135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:23.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.023+0000 7f6e8a44f700 1 -- 172.21.15.3:0/809748277 >> 172.21.15.3:0/809748277 conn(0x7f6e84066f50 msgr2=0x7f6e8409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:23.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.023+0000 7f6e8a44f700 1 -- 172.21.15.3:0/809748277 shutdown_connections 2023-10-03T10:13:23.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.023+0000 7f6e8a44f700 1 -- 172.21.15.3:0/809748277 wait complete. 2023-10-03T10:13:23.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.024+0000 7f6e8a44f700 1 Processor -- start 2023-10-03T10:13:23.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.024+0000 7f6e8a44f700 1 -- start start 2023-10-03T10:13:23.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.025+0000 7f6e8a44f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e8409d6f0 0x7f6e84121f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:23.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.025+0000 7f6e8a44f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6e8409e1d0 0x7f6e84122440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:23.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.025+0000 7f6e8a44f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6e84135e50 0x7f6e84122980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:23.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.025+0000 7f6e8a44f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e841242b0 con 0x7f6e8409d6f0 2023-10-03T10:13:23.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.025+0000 7f6e8a44f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e84125070 con 0x7f6e84135e50 2023-10-03T10:13:23.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.025+0000 7f6e8a44f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e840a1f20 con 0x7f6e8409e1d0 2023-10-03T10:13:23.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.025+0000 7f6e83fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e8409d6f0 0x7f6e84121f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:23.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.025+0000 7f6e83fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e8409d6f0 0x7f6e84121f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50842/0 (socket says 172.21.15.3:50842) 2023-10-03T10:13:23.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.025+0000 7f6e83fff700 1 -- 172.21.15.3:0/3162474721 learned_addr learned my addr 172.21.15.3:0/3162474721 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:23.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.025+0000 7f6e837fe700 1 --2- 172.21.15.3:0/3162474721 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6e8409e1d0 0x7f6e84122440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:23.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.025+0000 7f6e83fff700 1 -- 172.21.15.3:0/3162474721 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6e8409e1d0 msgr2=0x7f6e84122440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:23.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.025+0000 7f6e83fff700 1 --2- 172.21.15.3:0/3162474721 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6e8409e1d0 0x7f6e84122440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:23.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.025+0000 7f6e83fff700 1 -- 172.21.15.3:0/3162474721 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6e84135e50 msgr2=0x7f6e84122980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:23.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.025+0000 7f6e83fff700 1 --2- 172.21.15.3:0/3162474721 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6e84135e50 0x7f6e84122980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:23.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.025+0000 7f6e83fff700 1 -- 172.21.15.3:0/3162474721 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c00bb50 con 0x7f6e8409d6f0 2023-10-03T10:13:23.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.025+0000 7f6e837fe700 1 --2- 172.21.15.3:0/3162474721 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6e8409e1d0 0x7f6e84122440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:23.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.026+0000 7f6e83fff700 1 --2- 172.21.15.3:0/3162474721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e8409d6f0 0x7f6e84121f00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400c500 tx=0x7f6e7400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:23.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.026+0000 7f6e80ff9700 1 -- 172.21.15.3:0/3162474721 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74005cb0 con 0x7f6e8409d6f0 2023-10-03T10:13:23.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.026+0000 7f6e80ff9700 1 -- 172.21.15.3:0/3162474721 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e74005e10 con 0x7f6e8409d6f0 2023-10-03T10:13:23.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.026+0000 7f6e80ff9700 1 -- 172.21.15.3:0/3162474721 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e740072d0 con 0x7f6e8409d6f0 2023-10-03T10:13:23.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.026+0000 7f6e8a44f700 1 -- 172.21.15.3:0/3162474721 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e8409c700 con 0x7f6e8409d6f0 2023-10-03T10:13:23.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.026+0000 7f6e8a44f700 1 -- 172.21.15.3:0/3162474721 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e84125340 con 0x7f6e8409d6f0 2023-10-03T10:13:23.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.028+0000 7f6e8a44f700 1 -- 172.21.15.3:0/3162474721 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e84074ea0 con 0x7f6e8409d6f0 2023-10-03T10:13:23.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.029+0000 7f6e80ff9700 1 -- 172.21.15.3:0/3162474721 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6e74007430 con 0x7f6e8409d6f0 2023-10-03T10:13:23.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.029+0000 7f6e80ff9700 1 -- 172.21.15.3:0/3162474721 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6e7403eb10 con 0x7f6e8409d6f0 2023-10-03T10:13:23.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.038+0000 7f6e80ff9700 1 -- 172.21.15.3:0/3162474721 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6e7401b3b0 con 0x7f6e8409d6f0 2023-10-03T10:13:23.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.100+0000 7fe176899700 1 -- 172.21.15.3:0/1442580996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe17011e080 con 0x7fe170099a40 2023-10-03T10:13:23.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.100+0000 7fe16cff9700 1 -- 172.21.15.3:0/1442580996 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe160021af0 con 0x7fe170099a40 2023-10-03T10:13:23.101 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:23.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.104+0000 7fe176899700 1 -- 172.21.15.3:0/1442580996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe170099a40 msgr2=0x7fe17006f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:23.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.104+0000 7fe176899700 1 --2- 172.21.15.3:0/1442580996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe170099a40 0x7fe17006f5e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe16000b480 tx=0x7fe16000b790 comp rx=0 tx=0).stop 2023-10-03T10:13:23.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.104+0000 7fe176899700 1 -- 172.21.15.3:0/1442580996 shutdown_connections 2023-10-03T10:13:23.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.104+0000 7fe176899700 1 --2- 172.21.15.3:0/1442580996 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe170131810 0x7fe170070060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:23.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.104+0000 7fe176899700 1 --2- 172.21.15.3:0/1442580996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe170099a40 0x7fe17006f5e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:23.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.104+0000 7fe176899700 1 --2- 172.21.15.3:0/1442580996 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe17009a3b0 0x7fe17006fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:23.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.104+0000 7fe176899700 1 -- 172.21.15.3:0/1442580996 >> 172.21.15.3:0/1442580996 conn(0x7fe1700a2aa0 msgr2=0x7fe17012d360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:23.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.104+0000 7fe176899700 1 -- 172.21.15.3:0/1442580996 shutdown_connections 2023-10-03T10:13:23.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.104+0000 7fe176899700 1 -- 172.21.15.3:0/1442580996 wait complete. 2023-10-03T10:13:23.119 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:23.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.189+0000 7fb7e95a8700 1 -- 172.21.15.3:0/129257924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb7e4074ea0 con 0x7fb7e409e410 2023-10-03T10:13:23.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.189+0000 7fb7cb7fe700 1 -- 172.21.15.3:0/129257924 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb7dc012340 con 0x7fb7e409e410 2023-10-03T10:13:23.189 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:23.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.191+0000 7fb7e95a8700 1 -- 172.21.15.3:0/129257924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7e409e410 msgr2=0x7fb7e41ac670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:23.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.191+0000 7fb7e95a8700 1 --2- 172.21.15.3:0/129257924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7e409e410 0x7fb7e41ac670 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc009e50 tx=0x7fb7dc00c3a0 comp rx=0 tx=0).stop 2023-10-03T10:13:23.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.191+0000 7fb7e95a8700 1 -- 172.21.15.3:0/129257924 shutdown_connections 2023-10-03T10:13:23.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.191+0000 7fb7e95a8700 1 --2- 172.21.15.3:0/129257924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7e409e410 0x7fb7e41ac670 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:23.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.191+0000 7fb7e95a8700 1 --2- 172.21.15.3:0/129257924 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7e409d6f0 0x7fb7e41abbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:23.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.191+0000 7fb7e95a8700 1 --2- 172.21.15.3:0/129257924 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7e409e000 0x7fb7e41ac130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:23.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.191+0000 7fb7e95a8700 1 -- 172.21.15.3:0/129257924 >> 172.21.15.3:0/129257924 conn(0x7fb7e4066f50 msgr2=0x7fb7e40a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:23.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.191+0000 7fb7e95a8700 1 -- 172.21.15.3:0/129257924 shutdown_connections 2023-10-03T10:13:23.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.191+0000 7fb7e95a8700 1 -- 172.21.15.3:0/129257924 wait complete. 2023-10-03T10:13:23.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.198+0000 7f33d9ffb700 1 -- 172.21.15.3:0/4128141518 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f33d4005590 con 0x7f33f4130540 2023-10-03T10:13:23.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.198+0000 7f33dbfff700 1 -- 172.21.15.3:0/4128141518 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f33e8026370 con 0x7f33f4130540 2023-10-03T10:13:23.198 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:23.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.200+0000 7f33d9ffb700 1 -- 172.21.15.3:0/4128141518 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33f4130540 msgr2=0x7f33f4123dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:23.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.200+0000 7f33d9ffb700 1 --2- 172.21.15.3:0/4128141518 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33f4130540 0x7f33f4123dc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f33e8019000 tx=0x7f33e8004750 comp rx=0 tx=0).stop 2023-10-03T10:13:23.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.200+0000 7f33d9ffb700 1 -- 172.21.15.3:0/4128141518 shutdown_connections 2023-10-03T10:13:23.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.200+0000 7f33d9ffb700 1 --2- 172.21.15.3:0/4128141518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33f4130e50 0x7f33f41cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:23.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.200+0000 7f33d9ffb700 1 --2- 172.21.15.3:0/4128141518 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f33f406fb20 0x7f33f4123880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:23.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.200+0000 7f33d9ffb700 1 --2- 172.21.15.3:0/4128141518 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33f4130540 0x7f33f4123dc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:23.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.200+0000 7f33d9ffb700 1 -- 172.21.15.3:0/4128141518 >> 172.21.15.3:0/4128141518 conn(0x7f33f40a2aa0 msgr2=0x7f33f412d690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:23.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.200+0000 7f33d9ffb700 1 -- 172.21.15.3:0/4128141518 shutdown_connections 2023-10-03T10:13:23.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.201+0000 7f33d9ffb700 1 -- 172.21.15.3:0/4128141518 wait complete. 2023-10-03T10:13:23.207 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:23.215 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:23.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.268+0000 7f6e8a44f700 1 -- 172.21.15.3:0/3162474721 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6e8406c360 con 0x7f6e8409d6f0 2023-10-03T10:13:23.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.269+0000 7f6e80ff9700 1 -- 172.21.15.3:0/3162474721 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6e740192c0 con 0x7f6e8409d6f0 2023-10-03T10:13:23.269 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:23.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.271+0000 7f6e8a44f700 1 -- 172.21.15.3:0/3162474721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e8409d6f0 msgr2=0x7f6e84121f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.271+0000 7f6e8a44f700 1 --2- 172.21.15.3:0/3162474721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e8409d6f0 0x7f6e84121f00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400c500 tx=0x7f6e7400c810 comp rx=0 tx=0).stop 2023-10-03T10:13:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.271+0000 7f6e8a44f700 1 -- 172.21.15.3:0/3162474721 shutdown_connections 2023-10-03T10:13:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.271+0000 7f6e8a44f700 1 --2- 172.21.15.3:0/3162474721 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6e84135e50 0x7f6e84122980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.271+0000 7f6e8a44f700 1 --2- 172.21.15.3:0/3162474721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e8409d6f0 0x7f6e84121f00 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.271+0000 7f6e8a44f700 1 --2- 172.21.15.3:0/3162474721 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6e8409e1d0 0x7f6e84122440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.271+0000 7f6e8a44f700 1 -- 172.21.15.3:0/3162474721 >> 172.21.15.3:0/3162474721 conn(0x7f6e84066f50 msgr2=0x7f6e84134830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.271+0000 7f6e8a44f700 1 -- 172.21.15.3:0/3162474721 shutdown_connections 2023-10-03T10:13:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:23.271+0000 7f6e8a44f700 1 -- 172.21.15.3:0/3162474721 wait complete. 2023-10-03T10:13:23.285 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:24.120 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:24.208 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:24.216 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:24.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.268+0000 7f2ea34e7700 1 Processor -- start 2023-10-03T10:13:24.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.269+0000 7f2ea34e7700 1 -- start start 2023-10-03T10:13:24.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.269+0000 7f2ea34e7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e9c136710 0x7f2e9c138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:24.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.269+0000 7f2ea34e7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2e9c139030 0x7f2e9c13b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:24.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.269+0000 7f2ea34e7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e9c06ef30 con 0x7f2e9c1338a0 2023-10-03T10:13:24.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.269+0000 7f2ea34e7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e9c06fef0 con 0x7f2e9c139030 2023-10-03T10:13:24.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.269+0000 7f2ea34e7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c1334b0 con 0x7f2e9c136710 2023-10-03T10:13:24.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.269+0000 7f2ea1a84700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2e9c139030 0x7f2e9c13b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:24.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.269+0000 7f2ea0a82700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e9c136710 0x7f2e9c138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:24.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.269+0000 7f2ea1a84700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2e9c139030 0x7f2e9c13b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53522/0 (socket says 172.21.15.3:53522) 2023-10-03T10:13:24.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.269+0000 7f2ea1a84700 1 -- 172.21.15.3:0/1062614196 learned_addr learned my addr 172.21.15.3:0/1062614196 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:24.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.270+0000 7f2ea0a82700 1 -- 172.21.15.3:0/1062614196 >> v2:172.21.15.3:3300/0 conn(0x7f2e9c139030 msgr2=0x7f2e9c13b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:24.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.270+0000 7f2ea0a82700 1 --2- 172.21.15.3:0/1062614196 >> v2:172.21.15.3:3300/0 conn(0x7f2e9c139030 0x7f2e9c13b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.270+0000 7f2ea0a82700 1 -- 172.21.15.3:0/1062614196 >> v1:172.21.15.3:6789/0 conn(0x7f2e9c1338a0 legacy=0x7f2e9c133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:24.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.270+0000 7f2e93fff700 1 -- 172.21.15.3:0/1062614196 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 59067070 0 0) 0x7f2e9c06ef30 con 0x7f2e9c1338a0 2023-10-03T10:13:24.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.270+0000 7f2ea0a82700 1 -- 172.21.15.3:0/1062614196 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c06fc30 con 0x7f2e9c136710 2023-10-03T10:13:24.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.270+0000 7f2ea1a84700 1 --2- 172.21.15.3:0/1062614196 >> v2:172.21.15.3:3300/0 conn(0x7f2e9c139030 0x7f2e9c13b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:24.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.270+0000 7f2ea0a82700 1 --2- 172.21.15.3:0/1062614196 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e9c136710 0x7f2e9c138af0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009840 tx=0x7f2e8c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1072052f199cdf1b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:24.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.270+0000 7f2e93fff700 1 -- 172.21.15.3:0/1062614196 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c011830 con 0x7f2e9c136710 2023-10-03T10:13:24.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.270+0000 7f2e93fff700 1 -- 172.21.15.3:0/1062614196 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8c011990 con 0x7f2e9c136710 2023-10-03T10:13:24.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.270+0000 7f2e93fff700 1 -- 172.21.15.3:0/1062614196 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c011d20 con 0x7f2e9c136710 2023-10-03T10:13:24.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.271+0000 7f2ea34e7700 1 -- 172.21.15.3:0/1062614196 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e9c136710 msgr2=0x7f2e9c138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:24.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.271+0000 7f2ea34e7700 1 --2- 172.21.15.3:0/1062614196 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e9c136710 0x7f2e9c138af0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009840 tx=0x7f2e8c009b50 comp rx=0 tx=0).stop 2023-10-03T10:13:24.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.271+0000 7f2ea34e7700 1 -- 172.21.15.3:0/1062614196 shutdown_connections 2023-10-03T10:13:24.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.271+0000 7f2ea34e7700 1 --2- 172.21.15.3:0/1062614196 >> v2:172.21.15.3:3300/0 conn(0x7f2e9c139030 0x7f2e9c13b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.271+0000 7f2ea34e7700 1 --2- 172.21.15.3:0/1062614196 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e9c136710 0x7f2e9c138af0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.271+0000 7f2ea34e7700 1 -- 172.21.15.3:0/1062614196 >> 172.21.15.3:0/1062614196 conn(0x7f2e9c0a2aa0 msgr2=0x7f2e9c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:24.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.272+0000 7f2ea34e7700 1 -- 172.21.15.3:0/1062614196 shutdown_connections 2023-10-03T10:13:24.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.272+0000 7f2ea34e7700 1 -- 172.21.15.3:0/1062614196 wait complete. 2023-10-03T10:13:24.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.273+0000 7f2ea34e7700 1 Processor -- start 2023-10-03T10:13:24.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.273+0000 7f2ea34e7700 1 -- start start 2023-10-03T10:13:24.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.274+0000 7f2ea34e7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e9c1338a0 0x7f2e9c11ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:24.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.274+0000 7f2ea34e7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e9c136710 0x7f2e9c11b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:24.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.274+0000 7f2ea34e7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e9c139030 0x7f2e9c11b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:24.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.274+0000 7f2ea34e7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c130590 con 0x7f2e9c136710 2023-10-03T10:13:24.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.274+0000 7f2ea34e7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c11cea0 con 0x7f2e9c1338a0 2023-10-03T10:13:24.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.274+0000 7f2ea34e7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c11c980 con 0x7f2e9c139030 2023-10-03T10:13:24.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.274+0000 7f2ea1283700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e9c1338a0 0x7f2e9c11ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:24.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.274+0000 7f2ea0a82700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e9c136710 0x7f2e9c11b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:24.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.274+0000 7f2ea0a82700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e9c136710 0x7f2e9c11b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53538/0 (socket says 172.21.15.3:53538) 2023-10-03T10:13:24.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.274+0000 7f2ea0a82700 1 -- 172.21.15.3:0/2918691583 learned_addr learned my addr 172.21.15.3:0/2918691583 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:24.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.274+0000 7f2ea1283700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e9c1338a0 0x7f2e9c11ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56722/0 (socket says 172.21.15.3:56722) 2023-10-03T10:13:24.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.274+0000 7f2ea1a84700 1 --2- 172.21.15.3:0/2918691583 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e9c139030 0x7f2e9c11b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:24.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.274+0000 7f2ea0a82700 1 -- 172.21.15.3:0/2918691583 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e9c139030 msgr2=0x7f2e9c11b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:24.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.274+0000 7f2ea0a82700 1 --2- 172.21.15.3:0/2918691583 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e9c139030 0x7f2e9c11b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.274+0000 7f2ea0a82700 1 -- 172.21.15.3:0/2918691583 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e9c1338a0 msgr2=0x7f2e9c11ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:24.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.274+0000 7f2ea0a82700 1 --2- 172.21.15.3:0/2918691583 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e9c1338a0 0x7f2e9c11ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.274+0000 7f2ea0a82700 1 -- 172.21.15.3:0/2918691583 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e94009b30 con 0x7f2e9c136710 2023-10-03T10:13:24.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.274+0000 7f2ea1283700 1 --2- 172.21.15.3:0/2918691583 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e9c1338a0 0x7f2e9c11ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:24.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.275+0000 7f2ea0a82700 1 --2- 172.21.15.3:0/2918691583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e9c136710 0x7f2e9c11b1a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0090b0 tx=0x7f2e8c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:24.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.275+0000 7f2e91ffb700 1 -- 172.21.15.3:0/2918691583 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c0036a0 con 0x7f2e9c136710 2023-10-03T10:13:24.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.275+0000 7f2e91ffb700 1 -- 172.21.15.3:0/2918691583 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8c01f070 con 0x7f2e9c136710 2023-10-03T10:13:24.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.275+0000 7f2ea34e7700 1 -- 172.21.15.3:0/2918691583 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e8c0094a0 con 0x7f2e9c136710 2023-10-03T10:13:24.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.275+0000 7f2e91ffb700 1 -- 172.21.15.3:0/2918691583 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c003a60 con 0x7f2e9c136710 2023-10-03T10:13:24.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.275+0000 7f2ea34e7700 1 -- 172.21.15.3:0/2918691583 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e9c11d060 con 0x7f2e9c136710 2023-10-03T10:13:24.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.276+0000 7f2e91ffb700 1 -- 172.21.15.3:0/2918691583 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2e8c003bc0 con 0x7f2e9c136710 2023-10-03T10:13:24.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.276+0000 7f2e91ffb700 1 -- 172.21.15.3:0/2918691583 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2e8c021070 con 0x7f2e9c136710 2023-10-03T10:13:24.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.276+0000 7f2ea34e7700 1 -- 172.21.15.3:0/2918691583 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e9c074ea0 con 0x7f2e9c136710 2023-10-03T10:13:24.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.282+0000 7f2e91ffb700 1 -- 172.21.15.3:0/2918691583 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2e8c015020 con 0x7f2e9c136710 2023-10-03T10:13:24.286 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:24.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.367+0000 7f314d81c700 1 Processor -- start 2023-10-03T10:13:24.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.367+0000 7f314d81c700 1 -- start start 2023-10-03T10:13:24.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.368+0000 7f314d81c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f314809d9f0 0x7f3148136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:24.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.368+0000 7f314d81c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3148136a70 0x7f3148138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:24.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.368+0000 7f314d81c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f314809e130 con 0x7f31481316c0 2023-10-03T10:13:24.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.368+0000 7f314d81c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31481321a0 con 0x7f3148136a70 2023-10-03T10:13:24.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.368+0000 7f314d81c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31481312b0 con 0x7f314809d9f0 2023-10-03T10:13:24.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.368+0000 7f31477fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3148136a70 0x7f3148138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:24.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.368+0000 7f3146ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f31481316c0 0x7f3148131a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48710/0 (socket says 172.21.15.3:48710) 2023-10-03T10:13:24.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.368+0000 7f31467fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f314809d9f0 0x7f3148136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:24.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.368+0000 7f3146ffd700 1 -- 172.21.15.3:0/323226882 learned_addr learned my addr 172.21.15.3:0/323226882 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:24.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.369+0000 7f31477fe700 1 -- 172.21.15.3:0/323226882 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f314809d9f0 msgr2=0x7f3148136530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:24.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.369+0000 7f31477fe700 1 --2- 172.21.15.3:0/323226882 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f314809d9f0 0x7f3148136530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.369+0000 7f31477fe700 1 -- 172.21.15.3:0/323226882 >> v1:172.21.15.3:6789/0 conn(0x7f31481316c0 legacy=0x7f3148131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:24.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.369+0000 7f31477fe700 1 -- 172.21.15.3:0/323226882 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f314809f6e0 con 0x7f3148136a70 2023-10-03T10:13:24.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.369+0000 7f3145ffb700 1 -- 172.21.15.3:0/323226882 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2310154796 0 0) 0x7f3138001110 con 0x7f31481316c0 2023-10-03T10:13:24.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.369+0000 7f31477fe700 1 --2- 172.21.15.3:0/323226882 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3148136a70 0x7f3148138e50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3140009dc0 tx=0x7f314000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=930e1f013bf51d52 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:24.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.369+0000 7f3145ffb700 1 -- 172.21.15.3:0/323226882 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314000e070 con 0x7f3148136a70 2023-10-03T10:13:24.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.369+0000 7f3145ffb700 1 -- 172.21.15.3:0/323226882 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314000bb20 con 0x7f3148136a70 2023-10-03T10:13:24.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.369+0000 7f3145ffb700 1 -- 172.21.15.3:0/323226882 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3140012430 con 0x7f3148136a70 2023-10-03T10:13:24.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.370+0000 7f314d81c700 1 -- 172.21.15.3:0/323226882 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3148136a70 msgr2=0x7f3148138e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:24.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.370+0000 7f314d81c700 1 --2- 172.21.15.3:0/323226882 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3148136a70 0x7f3148138e50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3140009dc0 tx=0x7f314000b330 comp rx=0 tx=0).stop 2023-10-03T10:13:24.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.370+0000 7f314d81c700 1 -- 172.21.15.3:0/323226882 shutdown_connections 2023-10-03T10:13:24.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.370+0000 7f314d81c700 1 --2- 172.21.15.3:0/323226882 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3148136a70 0x7f3148138e50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.370+0000 7f314d81c700 1 --2- 172.21.15.3:0/323226882 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f314809d9f0 0x7f3148136530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.370+0000 7f314d81c700 1 -- 172.21.15.3:0/323226882 >> 172.21.15.3:0/323226882 conn(0x7f3148066f50 msgr2=0x7f314809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:24.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.371+0000 7f314d81c700 1 -- 172.21.15.3:0/323226882 shutdown_connections 2023-10-03T10:13:24.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.371+0000 7f314d81c700 1 -- 172.21.15.3:0/323226882 wait complete. 2023-10-03T10:13:24.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.372+0000 7f314d81c700 1 Processor -- start 2023-10-03T10:13:24.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.372+0000 7f314d81c700 1 -- start start 2023-10-03T10:13:24.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.373+0000 7f314d81c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f314809d9f0 0x7f314812ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:24.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.373+0000 7f314d81c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31481316c0 0x7f314812b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:24.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.373+0000 7f314d81c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3148136a70 0x7f31481342f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:24.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.373+0000 7f314d81c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314812b970 con 0x7f31481316c0 2023-10-03T10:13:24.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.373+0000 7f314d81c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f314806e980 con 0x7f314809d9f0 2023-10-03T10:13:24.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.373+0000 7f314d81c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3148072090 con 0x7f3148136a70 2023-10-03T10:13:24.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.373+0000 7f31467fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31481316c0 0x7f314812b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:24.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.373+0000 7f3146ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f314809d9f0 0x7f314812ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:24.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.373+0000 7f31467fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31481316c0 0x7f314812b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53540/0 (socket says 172.21.15.3:53540) 2023-10-03T10:13:24.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.373+0000 7f31467fc700 1 -- 172.21.15.3:0/3778825547 learned_addr learned my addr 172.21.15.3:0/3778825547 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:24.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.373+0000 7f31477fe700 1 --2- 172.21.15.3:0/3778825547 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3148136a70 0x7f31481342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:24.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.373+0000 7f3146ffd700 1 -- 172.21.15.3:0/3778825547 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3148136a70 msgr2=0x7f31481342f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:24.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.373+0000 7f3146ffd700 1 --2- 172.21.15.3:0/3778825547 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3148136a70 0x7f31481342f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.373+0000 7f3146ffd700 1 -- 172.21.15.3:0/3778825547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31481316c0 msgr2=0x7f314812b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:24.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.373+0000 7f3146ffd700 1 --2- 172.21.15.3:0/3778825547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31481316c0 0x7f314812b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.373+0000 7f3146ffd700 1 -- 172.21.15.3:0/3778825547 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3140009a70 con 0x7f314809d9f0 2023-10-03T10:13:24.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.374+0000 7f3146ffd700 1 --2- 172.21.15.3:0/3778825547 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f314809d9f0 0x7f314812ae00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3138000c00 tx=0x7f313800ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:24.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.374+0000 7f312ffff700 1 -- 172.21.15.3:0/3778825547 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3138004c80 con 0x7f314809d9f0 2023-10-03T10:13:24.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.374+0000 7f312ffff700 1 -- 172.21.15.3:0/3778825547 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3138004e00 con 0x7f314809d9f0 2023-10-03T10:13:24.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.374+0000 7f314d81c700 1 -- 172.21.15.3:0/3778825547 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f314809b870 con 0x7f314809d9f0 2023-10-03T10:13:24.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.374+0000 7f312ffff700 1 -- 172.21.15.3:0/3778825547 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3138007310 con 0x7f314809d9f0 2023-10-03T10:13:24.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.374+0000 7f314d81c700 1 -- 172.21.15.3:0/3778825547 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f314809be10 con 0x7f314809d9f0 2023-10-03T10:13:24.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.375+0000 7f312ffff700 1 -- 172.21.15.3:0/3778825547 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3138007470 con 0x7f314809d9f0 2023-10-03T10:13:24.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.375+0000 7f312ffff700 1 -- 172.21.15.3:0/3778825547 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f313803f020 con 0x7f314809d9f0 2023-10-03T10:13:24.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.376+0000 7f314d81c700 1 -- 172.21.15.3:0/3778825547 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3148074ea0 con 0x7f314809d9f0 2023-10-03T10:13:24.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.385+0000 7f312ffff700 1 -- 172.21.15.3:0/3778825547 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f313801bda0 con 0x7f314809d9f0 2023-10-03T10:13:24.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.408+0000 7fd0624a1700 1 Processor -- start 2023-10-03T10:13:24.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.408+0000 7fd0624a1700 1 -- start start 2023-10-03T10:13:24.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.409+0000 7fd0624a1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd05c0a0a30 0x7fd05c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.409+0000 7fd0624a1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd05c09d630 0x7fd05c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.409+0000 7fd0624a1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd05c0a2a70 con 0x7fd05c133930 2023-10-03T10:13:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.409+0000 7fd0624a1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd05c0a01d0 con 0x7fd05c0a0a30 2023-10-03T10:13:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.409+0000 7fd0624a1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c09e2b0 con 0x7fd05c09d630 2023-10-03T10:13:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.409+0000 7fd05b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd05c09d630 0x7fd05c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.409+0000 7fd05bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd05c0a0a30 0x7fd05c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.409+0000 7fd05b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd05c09d630 0x7fd05c0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53790/0 (socket says 172.21.15.3:53790) 2023-10-03T10:13:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.409+0000 7fd05b7fe700 1 -- 172.21.15.3:0/142702420 learned_addr learned my addr 172.21.15.3:0/142702420 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.410+0000 7fd05bfff700 1 -- 172.21.15.3:0/142702420 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd05c09d630 msgr2=0x7fd05c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.410+0000 7fd05bfff700 1 --2- 172.21.15.3:0/142702420 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd05c09d630 0x7fd05c0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.410+0000 7fd05bfff700 1 -- 172.21.15.3:0/142702420 >> v1:172.21.15.3:6789/0 conn(0x7fd05c133930 legacy=0x7fd05c133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.410+0000 7fd05bfff700 1 -- 172.21.15.3:0/142702420 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd05c09dc50 con 0x7fd05c0a0a30 2023-10-03T10:13:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.410+0000 7fd05bfff700 1 --2- 172.21.15.3:0/142702420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd05c0a0a30 0x7fd05c0a0e00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009840 tx=0x7fd04c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a9c28c01a2351e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.410+0000 7fd05affd700 1 -- 172.21.15.3:0/142702420 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c011830 con 0x7fd05c0a0a30 2023-10-03T10:13:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.410+0000 7fd05affd700 1 -- 172.21.15.3:0/142702420 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04c011990 con 0x7fd05c0a0a30 2023-10-03T10:13:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.410+0000 7fd05affd700 1 -- 172.21.15.3:0/142702420 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c011d20 con 0x7fd05c0a0a30 2023-10-03T10:13:24.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.411+0000 7fd0624a1700 1 -- 172.21.15.3:0/142702420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd05c0a0a30 msgr2=0x7fd05c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:24.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.411+0000 7fd0624a1700 1 --2- 172.21.15.3:0/142702420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd05c0a0a30 0x7fd05c0a0e00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009840 tx=0x7fd04c009b50 comp rx=0 tx=0).stop 2023-10-03T10:13:24.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.411+0000 7fd0624a1700 1 -- 172.21.15.3:0/142702420 shutdown_connections 2023-10-03T10:13:24.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.411+0000 7fd0624a1700 1 --2- 172.21.15.3:0/142702420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd05c0a0a30 0x7fd05c0a0e00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.411+0000 7fd0624a1700 1 --2- 172.21.15.3:0/142702420 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd05c09d630 0x7fd05c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.411+0000 7fd0624a1700 1 -- 172.21.15.3:0/142702420 >> 172.21.15.3:0/142702420 conn(0x7fd05c0a3050 msgr2=0x7fd05c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:24.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.411+0000 7fd0624a1700 1 -- 172.21.15.3:0/142702420 shutdown_connections 2023-10-03T10:13:24.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.411+0000 7fd0624a1700 1 -- 172.21.15.3:0/142702420 wait complete. 2023-10-03T10:13:24.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.412+0000 7fd0624a1700 1 Processor -- start 2023-10-03T10:13:24.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.412+0000 7fd0624a1700 1 -- start start 2023-10-03T10:13:24.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.413+0000 7fd0624a1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd05c09d630 0x7fd05c1c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:24.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.413+0000 7fd0624a1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd05c0a0a30 0x7fd05c1c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:24.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.413+0000 7fd0624a1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd05c133930 0x7fd05c1c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:24.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.413+0000 7fd0624a1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c11eea0 con 0x7fd05c133930 2023-10-03T10:13:24.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.413+0000 7fd0624a1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c1207f0 con 0x7fd05c0a0a30 2023-10-03T10:13:24.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.413+0000 7fd0624a1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c120600 con 0x7fd05c09d630 2023-10-03T10:13:24.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.413+0000 7fd05b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd05c0a0a30 0x7fd05c1c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:24.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.413+0000 7fd05b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd05c0a0a30 0x7fd05c1c8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56738/0 (socket says 172.21.15.3:56738) 2023-10-03T10:13:24.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.413+0000 7fd05b7fe700 1 -- 172.21.15.3:0/3191284680 learned_addr learned my addr 172.21.15.3:0/3191284680 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:24.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.413+0000 7fd05bfff700 1 --2- 172.21.15.3:0/3191284680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd05c09d630 0x7fd05c1c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:24.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.413+0000 7fd060a3e700 1 --2- 172.21.15.3:0/3191284680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd05c133930 0x7fd05c1c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:24.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.413+0000 7fd05b7fe700 1 -- 172.21.15.3:0/3191284680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd05c09d630 msgr2=0x7fd05c1c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:24.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.413+0000 7fd05b7fe700 1 --2- 172.21.15.3:0/3191284680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd05c09d630 0x7fd05c1c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.413+0000 7fd05b7fe700 1 -- 172.21.15.3:0/3191284680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd05c133930 msgr2=0x7fd05c1c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:24.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.413+0000 7fd05b7fe700 1 --2- 172.21.15.3:0/3191284680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd05c133930 0x7fd05c1c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.413+0000 7fd05b7fe700 1 -- 172.21.15.3:0/3191284680 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c0094a0 con 0x7fd05c0a0a30 2023-10-03T10:13:24.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.414+0000 7fd05b7fe700 1 --2- 172.21.15.3:0/3191284680 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd05c0a0a30 0x7fd05c1c8660 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd05000e910 tx=0x7fd05000ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:24.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.414+0000 7fd058ff9700 1 -- 172.21.15.3:0/3191284680 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0500096a0 con 0x7fd05c0a0a30 2023-10-03T10:13:24.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.414+0000 7fd058ff9700 1 -- 172.21.15.3:0/3191284680 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd050013680 con 0x7fd05c0a0a30 2023-10-03T10:13:24.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.414+0000 7fd058ff9700 1 -- 172.21.15.3:0/3191284680 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd050013a30 con 0x7fd05c0a0a30 2023-10-03T10:13:24.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.415+0000 7fd0624a1700 1 -- 172.21.15.3:0/3191284680 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd05c122510 con 0x7fd05c0a0a30 2023-10-03T10:13:24.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.415+0000 7fd0624a1700 1 -- 172.21.15.3:0/3191284680 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd05c122230 con 0x7fd05c0a0a30 2023-10-03T10:13:24.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.415+0000 7fd058ff9700 1 -- 172.21.15.3:0/3191284680 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd050013b90 con 0x7fd05c0a0a30 2023-10-03T10:13:24.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.416+0000 7fd0624a1700 1 -- 172.21.15.3:0/3191284680 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd05c120e50 con 0x7fd05c0a0a30 2023-10-03T10:13:24.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.416+0000 7fd058ff9700 1 -- 172.21.15.3:0/3191284680 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd05003f660 con 0x7fd05c0a0a30 2023-10-03T10:13:24.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.421+0000 7fd058ff9700 1 -- 172.21.15.3:0/3191284680 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd050017070 con 0x7fd05c0a0a30 2023-10-03T10:13:24.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.459+0000 7f55b4f31700 1 Processor -- start 2023-10-03T10:13:24.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.459+0000 7f55b4f31700 1 -- start start 2023-10-03T10:13:24.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.460+0000 7f55b4f31700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f55b009d630 0x7f55b00a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:24.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.460+0000 7f55b4f31700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f55b0138d80 0x7f55b013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:24.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.460+0000 7f55b4f31700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55b00a2a70 con 0x7f55b00a0a30 2023-10-03T10:13:24.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.460+0000 7f55b4f31700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55b00a01d0 con 0x7f55b0138d80 2023-10-03T10:13:24.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.460+0000 7f55b4f31700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b009e2b0 con 0x7f55b009d630 2023-10-03T10:13:24.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.460+0000 7f55add9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f55b009d630 0x7f55b00a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:24.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.460+0000 7f55ae59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f55b00a0a30 0x7f55b00a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48732/0 (socket says 172.21.15.3:48732) 2023-10-03T10:13:24.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.460+0000 7f55ae59c700 1 -- 172.21.15.3:0/2148716206 learned_addr learned my addr 172.21.15.3:0/2148716206 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:24.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.460+0000 7f55aed9d700 1 --2- 172.21.15.3:0/2148716206 >> v2:172.21.15.3:3300/0 conn(0x7f55b0138d80 0x7f55b013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:24.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.461+0000 7f55add9b700 1 -- 172.21.15.3:0/2148716206 >> v2:172.21.15.3:3300/0 conn(0x7f55b0138d80 msgr2=0x7f55b013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:24.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.461+0000 7f55add9b700 1 --2- 172.21.15.3:0/2148716206 >> v2:172.21.15.3:3300/0 conn(0x7f55b0138d80 0x7f55b013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.461+0000 7f55add9b700 1 -- 172.21.15.3:0/2148716206 >> v1:172.21.15.3:6789/0 conn(0x7f55b00a0a30 legacy=0x7f55b00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:24.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.461+0000 7f55add9b700 1 -- 172.21.15.3:0/2148716206 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b009dc50 con 0x7f55b009d630 2023-10-03T10:13:24.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.461+0000 7f55add9b700 1 --2- 172.21.15.3:0/2148716206 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f55b009d630 0x7f55b00a1510 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f55a4009840 tx=0x7f55a4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2babebaa194acd3c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:24.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.461+0000 7f55ad59a700 1 -- 172.21.15.3:0/2148716206 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a4011830 con 0x7f55b009d630 2023-10-03T10:13:24.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.461+0000 7f55ad59a700 1 -- 172.21.15.3:0/2148716206 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a4011990 con 0x7f55b009d630 2023-10-03T10:13:24.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.461+0000 7f55ad59a700 1 -- 172.21.15.3:0/2148716206 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a4011d20 con 0x7f55b009d630 2023-10-03T10:13:24.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.462+0000 7f55b4f31700 1 -- 172.21.15.3:0/2148716206 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f55b009d630 msgr2=0x7f55b00a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:24.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.462+0000 7f55b4f31700 1 --2- 172.21.15.3:0/2148716206 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f55b009d630 0x7f55b00a1510 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f55a4009840 tx=0x7f55a4009b50 comp rx=0 tx=0).stop 2023-10-03T10:13:24.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.462+0000 7f55b4f31700 1 -- 172.21.15.3:0/2148716206 shutdown_connections 2023-10-03T10:13:24.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.462+0000 7f55b4f31700 1 --2- 172.21.15.3:0/2148716206 >> v2:172.21.15.3:3300/0 conn(0x7f55b0138d80 0x7f55b013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.462+0000 7f55b4f31700 1 --2- 172.21.15.3:0/2148716206 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f55b009d630 0x7f55b00a1510 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.462+0000 7f55b4f31700 1 -- 172.21.15.3:0/2148716206 >> 172.21.15.3:0/2148716206 conn(0x7f55b00a3050 msgr2=0x7f55b00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:24.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.462+0000 7f55b4f31700 1 -- 172.21.15.3:0/2148716206 shutdown_connections 2023-10-03T10:13:24.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.462+0000 7f55b4f31700 1 -- 172.21.15.3:0/2148716206 wait complete. 2023-10-03T10:13:24.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.463+0000 7f55b4f31700 1 Processor -- start 2023-10-03T10:13:24.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.464+0000 7f55b4f31700 1 -- start start 2023-10-03T10:13:24.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.464+0000 7f55b4f31700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f55b009d630 0x7f55b0133b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:24.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.464+0000 7f55b4f31700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f55b00a0a30 0x7f55b01340b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:24.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.464+0000 7f55b4f31700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55b0138d80 0x7f55b01345f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:24.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.464+0000 7f55b4f31700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b0137560 con 0x7f55b0138d80 2023-10-03T10:13:24.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.464+0000 7f55b4f31700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b013af80 con 0x7f55b009d630 2023-10-03T10:13:24.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.464+0000 7f55b4f31700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f55b00a0410 con 0x7f55b00a0a30 2023-10-03T10:13:24.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.465+0000 7f55aed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55b0138d80 0x7f55b01345f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:24.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.465+0000 7f55add9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f55b00a0a30 0x7f55b01340b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:24.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.465+0000 7f55aed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55b0138d80 0x7f55b01345f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53586/0 (socket says 172.21.15.3:53586) 2023-10-03T10:13:24.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.465+0000 7f55aed9d700 1 -- 172.21.15.3:0/3498484390 learned_addr learned my addr 172.21.15.3:0/3498484390 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:24.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.465+0000 7f55add9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f55b00a0a30 0x7f55b01340b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53812/0 (socket says 172.21.15.3:53812) 2023-10-03T10:13:24.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.465+0000 7f55ae59c700 1 --2- 172.21.15.3:0/3498484390 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f55b009d630 0x7f55b0133b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:24.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.465+0000 7f55aed9d700 1 -- 172.21.15.3:0/3498484390 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f55b00a0a30 msgr2=0x7f55b01340b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:24.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.465+0000 7f55aed9d700 1 --2- 172.21.15.3:0/3498484390 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f55b00a0a30 0x7f55b01340b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.465+0000 7f55aed9d700 1 -- 172.21.15.3:0/3498484390 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f55b009d630 msgr2=0x7f55b0133b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:24.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.465+0000 7f55aed9d700 1 --2- 172.21.15.3:0/3498484390 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f55b009d630 0x7f55b0133b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.465+0000 7f55aed9d700 1 -- 172.21.15.3:0/3498484390 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a40094a0 con 0x7f55b0138d80 2023-10-03T10:13:24.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.465+0000 7f55ae59c700 1 --2- 172.21.15.3:0/3498484390 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f55b009d630 0x7f55b0133b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:24.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.465+0000 7f55aed9d700 1 --2- 172.21.15.3:0/3498484390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55b0138d80 0x7f55b01345f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f55a800efc0 tx=0x7f55a800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:24.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.466+0000 7f559effd700 1 -- 172.21.15.3:0/3498484390 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a800cc90 con 0x7f55b0138d80 2023-10-03T10:13:24.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.466+0000 7f559effd700 1 -- 172.21.15.3:0/3498484390 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a800f040 con 0x7f55b0138d80 2023-10-03T10:13:24.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.466+0000 7f55b4f31700 1 -- 172.21.15.3:0/3498484390 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b01381e0 con 0x7f55b0138d80 2023-10-03T10:13:24.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.466+0000 7f559effd700 1 -- 172.21.15.3:0/3498484390 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a80146c0 con 0x7f55b0138d80 2023-10-03T10:13:24.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.466+0000 7f55b4f31700 1 -- 172.21.15.3:0/3498484390 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b0136cb0 con 0x7f55b0138d80 2023-10-03T10:13:24.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.467+0000 7f559effd700 1 -- 172.21.15.3:0/3498484390 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f55a8014820 con 0x7f55b0138d80 2023-10-03T10:13:24.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.467+0000 7f559effd700 1 -- 172.21.15.3:0/3498484390 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f55a80403d0 con 0x7f55b0138d80 2023-10-03T10:13:24.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.467+0000 7f55b4f31700 1 -- 172.21.15.3:0/3498484390 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55b00747f0 con 0x7f55b0138d80 2023-10-03T10:13:24.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.472+0000 7f559effd700 1 -- 172.21.15.3:0/3498484390 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f55a8018030 con 0x7f55b0138d80 2023-10-03T10:13:24.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.514+0000 7f2ea34e7700 1 -- 172.21.15.3:0/2918691583 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2e9c11ef70 con 0x7f2e9c136710 2023-10-03T10:13:24.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.515+0000 7f2e91ffb700 1 -- 172.21.15.3:0/2918691583 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2e8c024790 con 0x7f2e9c136710 2023-10-03T10:13:24.515 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:24.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.518+0000 7f2ea34e7700 1 -- 172.21.15.3:0/2918691583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e9c136710 msgr2=0x7f2e9c11b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:24.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.518+0000 7f2ea34e7700 1 --2- 172.21.15.3:0/2918691583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e9c136710 0x7f2e9c11b1a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0090b0 tx=0x7f2e8c004750 comp rx=0 tx=0).stop 2023-10-03T10:13:24.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.519+0000 7f2ea34e7700 1 -- 172.21.15.3:0/2918691583 shutdown_connections 2023-10-03T10:13:24.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.519+0000 7f2ea34e7700 1 --2- 172.21.15.3:0/2918691583 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e9c139030 0x7f2e9c11b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.519+0000 7f2ea34e7700 1 --2- 172.21.15.3:0/2918691583 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e9c1338a0 0x7f2e9c11ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.519+0000 7f2ea34e7700 1 --2- 172.21.15.3:0/2918691583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e9c136710 0x7f2e9c11b1a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.519+0000 7f2ea34e7700 1 -- 172.21.15.3:0/2918691583 >> 172.21.15.3:0/2918691583 conn(0x7f2e9c0a2aa0 msgr2=0x7f2e9c1344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:24.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.519+0000 7f2ea34e7700 1 -- 172.21.15.3:0/2918691583 shutdown_connections 2023-10-03T10:13:24.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.519+0000 7f2ea34e7700 1 -- 172.21.15.3:0/2918691583 wait complete. 2023-10-03T10:13:24.538 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:24.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.630+0000 7f314d81c700 1 -- 172.21.15.3:0/3778825547 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f314809e980 con 0x7f314809d9f0 2023-10-03T10:13:24.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.631+0000 7f312ffff700 1 -- 172.21.15.3:0/3778825547 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f313801b340 con 0x7f314809d9f0 2023-10-03T10:13:24.631 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:24.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.633+0000 7fd0624a1700 1 -- 172.21.15.3:0/3191284680 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd05c0747f0 con 0x7fd05c0a0a30 2023-10-03T10:13:24.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.633+0000 7f314d81c700 1 -- 172.21.15.3:0/3778825547 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f314809d9f0 msgr2=0x7f314812ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:24.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.633+0000 7f314d81c700 1 --2- 172.21.15.3:0/3778825547 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f314809d9f0 0x7f314812ae00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3138000c00 tx=0x7f313800ca30 comp rx=0 tx=0).stop 2023-10-03T10:13:24.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.633+0000 7f314d81c700 1 -- 172.21.15.3:0/3778825547 shutdown_connections 2023-10-03T10:13:24.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.633+0000 7f314d81c700 1 --2- 172.21.15.3:0/3778825547 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3148136a70 0x7f31481342f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.633+0000 7f314d81c700 1 --2- 172.21.15.3:0/3778825547 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f314809d9f0 0x7f314812ae00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.633+0000 7f314d81c700 1 --2- 172.21.15.3:0/3778825547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31481316c0 0x7f314812b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.633+0000 7f314d81c700 1 -- 172.21.15.3:0/3778825547 >> 172.21.15.3:0/3778825547 conn(0x7f3148066f50 msgr2=0x7f3148136e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:24.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.633+0000 7fd058ff9700 1 -- 172.21.15.3:0/3191284680 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd05001bdf0 con 0x7fd05c0a0a30 2023-10-03T10:13:24.634 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:24.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.633+0000 7f314d81c700 1 -- 172.21.15.3:0/3778825547 shutdown_connections 2023-10-03T10:13:24.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.633+0000 7f314d81c700 1 -- 172.21.15.3:0/3778825547 wait complete. 2023-10-03T10:13:24.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.636+0000 7fd0624a1700 1 -- 172.21.15.3:0/3191284680 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd05c0a0a30 msgr2=0x7fd05c1c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:24.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.636+0000 7fd0624a1700 1 --2- 172.21.15.3:0/3191284680 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd05c0a0a30 0x7fd05c1c8660 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd05000e910 tx=0x7fd05000ec20 comp rx=0 tx=0).stop 2023-10-03T10:13:24.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.637+0000 7fd0624a1700 1 -- 172.21.15.3:0/3191284680 shutdown_connections 2023-10-03T10:13:24.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.637+0000 7fd0624a1700 1 --2- 172.21.15.3:0/3191284680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd05c133930 0x7fd05c1c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.637+0000 7fd0624a1700 1 --2- 172.21.15.3:0/3191284680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd05c09d630 0x7fd05c1c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.637+0000 7fd0624a1700 1 --2- 172.21.15.3:0/3191284680 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd05c0a0a30 0x7fd05c1c8660 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.637+0000 7fd0624a1700 1 -- 172.21.15.3:0/3191284680 >> 172.21.15.3:0/3191284680 conn(0x7fd05c0a3050 msgr2=0x7fd05c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:24.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.637+0000 7fd0624a1700 1 -- 172.21.15.3:0/3191284680 shutdown_connections 2023-10-03T10:13:24.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.637+0000 7fd0624a1700 1 -- 172.21.15.3:0/3191284680 wait complete. 2023-10-03T10:13:24.653 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:24.656 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:24.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.704+0000 7f55b4f31700 1 -- 172.21.15.3:0/3498484390 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f55b01371e0 con 0x7f55b0138d80 2023-10-03T10:13:24.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.704+0000 7f559effd700 1 -- 172.21.15.3:0/3498484390 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f55b01371e0 con 0x7f55b0138d80 2023-10-03T10:13:24.705 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:24.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.706+0000 7f55b4f31700 1 -- 172.21.15.3:0/3498484390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55b0138d80 msgr2=0x7f55b01345f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:24.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.706+0000 7f55b4f31700 1 --2- 172.21.15.3:0/3498484390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55b0138d80 0x7f55b01345f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f55a800efc0 tx=0x7f55a800c3a0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.706+0000 7f55b4f31700 1 -- 172.21.15.3:0/3498484390 shutdown_connections 2023-10-03T10:13:24.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.706+0000 7f55b4f31700 1 --2- 172.21.15.3:0/3498484390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55b0138d80 0x7f55b01345f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.706+0000 7f55b4f31700 1 --2- 172.21.15.3:0/3498484390 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f55b009d630 0x7f55b0133b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.706+0000 7f55b4f31700 1 --2- 172.21.15.3:0/3498484390 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f55b00a0a30 0x7f55b01340b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:24.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.706+0000 7f55b4f31700 1 -- 172.21.15.3:0/3498484390 >> 172.21.15.3:0/3498484390 conn(0x7f55b00a3050 msgr2=0x7f55b01399e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:24.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.707+0000 7f55b4f31700 1 -- 172.21.15.3:0/3498484390 shutdown_connections 2023-10-03T10:13:24.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:24.707+0000 7f55b4f31700 1 -- 172.21.15.3:0/3498484390 wait complete. 2023-10-03T10:13:24.720 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:25.540 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:25.655 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:25.656 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:25.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.694+0000 7fe3d10d9700 1 Processor -- start 2023-10-03T10:13:25.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.695+0000 7fe3d10d9700 1 -- start start 2023-10-03T10:13:25.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.695+0000 7fe3d10d9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe3cc1338a0 0x7fe3cc133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:25.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.695+0000 7fe3d10d9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3cc09de50 0x7fe3cc09e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:25.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.695+0000 7fe3d10d9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3cc09e760 0x7fe3cc13a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:25.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.695+0000 7fe3d10d9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3cc06fef0 con 0x7fe3cc1338a0 2023-10-03T10:13:25.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.695+0000 7fe3d10d9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc06ef30 con 0x7fe3cc09de50 2023-10-03T10:13:25.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.695+0000 7fe3d10d9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc1334b0 con 0x7fe3cc09e760 2023-10-03T10:13:25.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.695+0000 7fe3cb59e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3cc09e760 0x7fe3cc13a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:25.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.695+0000 7fe3cb59e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3cc09e760 0x7fe3cc13a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53828/0 (socket says 172.21.15.3:53828) 2023-10-03T10:13:25.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.695+0000 7fe3cb59e700 1 -- 172.21.15.3:0/3572733151 learned_addr learned my addr 172.21.15.3:0/3572733151 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:25.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.695+0000 7fe3cad9d700 1 --2- 172.21.15.3:0/3572733151 >> v2:172.21.15.3:3300/0 conn(0x7fe3cc1338a0 0x7fe3cc133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:25.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.695+0000 7fe3ca59c700 1 --2- 172.21.15.3:0/3572733151 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3cc09de50 0x7fe3cc09e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:25.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.696+0000 7fe3cb59e700 1 -- 172.21.15.3:0/3572733151 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3cc09de50 msgr2=0x7fe3cc09e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:25.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.696+0000 7fe3cb59e700 1 --2- 172.21.15.3:0/3572733151 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3cc09de50 0x7fe3cc09e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:25.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.696+0000 7fe3cb59e700 1 -- 172.21.15.3:0/3572733151 >> v2:172.21.15.3:3300/0 conn(0x7fe3cc1338a0 msgr2=0x7fe3cc133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:25.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.696+0000 7fe3cb59e700 1 --2- 172.21.15.3:0/3572733151 >> v2:172.21.15.3:3300/0 conn(0x7fe3cc1338a0 0x7fe3cc133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:25.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.696+0000 7fe3cb59e700 1 -- 172.21.15.3:0/3572733151 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3cc06fc30 con 0x7fe3cc09e760 2023-10-03T10:13:25.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.696+0000 7fe3cad9d700 1 --2- 172.21.15.3:0/3572733151 >> v2:172.21.15.3:3300/0 conn(0x7fe3cc1338a0 0x7fe3cc133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:25.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.696+0000 7fe3ca59c700 1 --2- 172.21.15.3:0/3572733151 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3cc09de50 0x7fe3cc09e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:25.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.696+0000 7fe3cb59e700 1 --2- 172.21.15.3:0/3572733151 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3cc09e760 0x7fe3cc13a730 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4009e50 tx=0x7fe3c400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e426bd4bc62d39 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:25.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.696+0000 7fe3c9d9b700 1 -- 172.21.15.3:0/3572733151 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c400e070 con 0x7fe3cc09e760 2023-10-03T10:13:25.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.696+0000 7fe3c9d9b700 1 -- 172.21.15.3:0/3572733151 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c400b910 con 0x7fe3cc09e760 2023-10-03T10:13:25.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.696+0000 7fe3c9d9b700 1 -- 172.21.15.3:0/3572733151 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c400bcc0 con 0x7fe3cc09e760 2023-10-03T10:13:25.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.697+0000 7fe3d10d9700 1 -- 172.21.15.3:0/3572733151 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3cc09e760 msgr2=0x7fe3cc13a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:25.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.697+0000 7fe3d10d9700 1 --2- 172.21.15.3:0/3572733151 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3cc09e760 0x7fe3cc13a730 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4009e50 tx=0x7fe3c400b330 comp rx=0 tx=0).stop 2023-10-03T10:13:25.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.697+0000 7fe3d10d9700 1 -- 172.21.15.3:0/3572733151 shutdown_connections 2023-10-03T10:13:25.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.697+0000 7fe3d10d9700 1 --2- 172.21.15.3:0/3572733151 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3cc09e760 0x7fe3cc13a730 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:25.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.697+0000 7fe3d10d9700 1 --2- 172.21.15.3:0/3572733151 >> v2:172.21.15.3:3300/0 conn(0x7fe3cc1338a0 0x7fe3cc133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:25.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.697+0000 7fe3d10d9700 1 --2- 172.21.15.3:0/3572733151 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3cc09de50 0x7fe3cc09e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:25.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.697+0000 7fe3d10d9700 1 -- 172.21.15.3:0/3572733151 >> 172.21.15.3:0/3572733151 conn(0x7fe3cc0a2aa0 msgr2=0x7fe3cc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:25.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.697+0000 7fe3d10d9700 1 -- 172.21.15.3:0/3572733151 shutdown_connections 2023-10-03T10:13:25.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.697+0000 7fe3d10d9700 1 -- 172.21.15.3:0/3572733151 wait complete. 2023-10-03T10:13:25.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.699+0000 7fe3d10d9700 1 Processor -- start 2023-10-03T10:13:25.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.699+0000 7fe3d10d9700 1 -- start start 2023-10-03T10:13:25.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.700+0000 7fe3d10d9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3cc09de50 0x7fe3cc11a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:25.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.700+0000 7fe3d10d9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3cc09e760 0x7fe3cc11aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:25.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.700+0000 7fe3d10d9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3cc1338a0 0x7fe3cc11b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:25.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.700+0000 7fe3d10d9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc11e9f0 con 0x7fe3cc09de50 2023-10-03T10:13:25.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.700+0000 7fe3d10d9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc11e030 con 0x7fe3cc09e760 2023-10-03T10:13:25.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.700+0000 7fe3d10d9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc11d370 con 0x7fe3cc1338a0 2023-10-03T10:13:25.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.700+0000 7fe3cad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3cc09de50 0x7fe3cc11a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:25.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.700+0000 7fe3cad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3cc09de50 0x7fe3cc11a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53592/0 (socket says 172.21.15.3:53592) 2023-10-03T10:13:25.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.700+0000 7fe3cad9d700 1 -- 172.21.15.3:0/883507703 learned_addr learned my addr 172.21.15.3:0/883507703 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:25.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.700+0000 7fe3ca59c700 1 --2- 172.21.15.3:0/883507703 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3cc09e760 0x7fe3cc11aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:25.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.700+0000 7fe3cb59e700 1 --2- 172.21.15.3:0/883507703 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3cc1338a0 0x7fe3cc11b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:25.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.700+0000 7fe3cad9d700 1 -- 172.21.15.3:0/883507703 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3cc1338a0 msgr2=0x7fe3cc11b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:25.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.700+0000 7fe3cad9d700 1 --2- 172.21.15.3:0/883507703 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3cc1338a0 0x7fe3cc11b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:25.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.700+0000 7fe3cad9d700 1 -- 172.21.15.3:0/883507703 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3cc09e760 msgr2=0x7fe3cc11aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:25.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.700+0000 7fe3cad9d700 1 --2- 172.21.15.3:0/883507703 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3cc09e760 0x7fe3cc11aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:25.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.700+0000 7fe3cad9d700 1 -- 172.21.15.3:0/883507703 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c00094a0 con 0x7fe3cc09de50 2023-10-03T10:13:25.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.701+0000 7fe3cad9d700 1 --2- 172.21.15.3:0/883507703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3cc09de50 0x7fe3cc11a5a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00ebe0 tx=0x7fe3bc00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:25.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.701+0000 7fe3b37fe700 1 -- 172.21.15.3:0/883507703 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc00c850 con 0x7fe3cc09de50 2023-10-03T10:13:25.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.701+0000 7fe3b37fe700 1 -- 172.21.15.3:0/883507703 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3bc00c9b0 con 0x7fe3cc09de50 2023-10-03T10:13:25.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.701+0000 7fe3d10d9700 1 -- 172.21.15.3:0/883507703 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3bc0094a0 con 0x7fe3cc09de50 2023-10-03T10:13:25.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.701+0000 7fe3b37fe700 1 -- 172.21.15.3:0/883507703 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc00cd20 con 0x7fe3cc09de50 2023-10-03T10:13:25.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.701+0000 7fe3d10d9700 1 -- 172.21.15.3:0/883507703 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c4009b30 con 0x7fe3cc09de50 2023-10-03T10:13:25.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.702+0000 7fe3b37fe700 1 -- 172.21.15.3:0/883507703 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe3bc013470 con 0x7fe3cc09de50 2023-10-03T10:13:25.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.702+0000 7fe3b37fe700 1 -- 172.21.15.3:0/883507703 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe3bc03fab0 con 0x7fe3cc09de50 2023-10-03T10:13:25.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.702+0000 7fe3d10d9700 1 -- 172.21.15.3:0/883507703 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3cc11dbd0 con 0x7fe3cc09de50 2023-10-03T10:13:25.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.711+0000 7fe3b37fe700 1 -- 172.21.15.3:0/883507703 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe3bc013720 con 0x7fe3cc09de50 2023-10-03T10:13:25.721 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:25.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.804+0000 7f9af0c2a700 1 Processor -- start 2023-10-03T10:13:25.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.804+0000 7f9af0c2a700 1 -- start start 2023-10-03T10:13:25.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.805+0000 7f0e0d4bf700 1 Processor -- start 2023-10-03T10:13:25.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.805+0000 7f0e0d4bf700 1 -- start start 2023-10-03T10:13:25.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.805+0000 7f9af0c2a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9aec09ac90 0x7f9aec13cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:25.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.805+0000 7f9af0c2a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9aec13d240 0x7f9aec13f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:25.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.805+0000 7f9af0c2a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aec134980 con 0x7f9aec06c0e0 2023-10-03T10:13:25.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.805+0000 7f9af0c2a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9aec06c4b0 con 0x7f9aec13d240 2023-10-03T10:13:25.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.805+0000 7f9af0c2a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec09b0e0 con 0x7f9aec09ac90 2023-10-03T10:13:25.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.805+0000 7f9ae9d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9aec09ac90 0x7f9aec13cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:25.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.805+0000 7f9ae9d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9aec09ac90 0x7f9aec13cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53852/0 (socket says 172.21.15.3:53852) 2023-10-03T10:13:25.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.805+0000 7f9ae9d9b700 1 -- 172.21.15.3:0/3090629896 learned_addr learned my addr 172.21.15.3:0/3090629896 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:25.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.805+0000 7f0e0d4bf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e0809d6f0 0x7f0e0809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:25.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.805+0000 7f0e0d4bf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0e0809e410 0x7f0e08134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:25.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.805+0000 7f0e0d4bf700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e0809b120 con 0x7f0e0809e000 2023-10-03T10:13:25.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.805+0000 7f0e0d4bf700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e0806b460 con 0x7f0e0809e410 2023-10-03T10:13:25.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.805+0000 7f0e0d4bf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0809a7e0 con 0x7f0e0809d6f0 2023-10-03T10:13:25.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.806+0000 7f0e06ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e0809d6f0 0x7f0e0809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:25.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.806+0000 7f0e06ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e0809d6f0 0x7f0e0809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53868/0 (socket says 172.21.15.3:53868) 2023-10-03T10:13:25.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.806+0000 7f0e06ffd700 1 -- 172.21.15.3:0/1918071986 learned_addr learned my addr 172.21.15.3:0/1918071986 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:25.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.805+0000 7f9ae9d9b700 1 -- 172.21.15.3:0/3090629896 >> v2:172.21.15.3:3300/0 conn(0x7f9aec13d240 msgr2=0x7f9aec13f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:25.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.805+0000 7f9ae9d9b700 1 --2- 172.21.15.3:0/3090629896 >> v2:172.21.15.3:3300/0 conn(0x7f9aec13d240 0x7f9aec13f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:25.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.805+0000 7f9ae9d9b700 1 -- 172.21.15.3:0/3090629896 >> v1:172.21.15.3:6789/0 conn(0x7f9aec06c0e0 legacy=0x7f9aec134ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:25.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.806+0000 7f9ae9d9b700 1 -- 172.21.15.3:0/3090629896 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aec0a0600 con 0x7f9aec09ac90 2023-10-03T10:13:25.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.806+0000 7f9ae9d9b700 1 --2- 172.21.15.3:0/3090629896 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9aec09ac90 0x7f9aec13cd00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00098f0 tx=0x7f9ae0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5dcc34529c01c64 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:25.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.806+0000 7f0e077fe700 1 --2- 172.21.15.3:0/1918071986 >> v2:172.21.15.3:3300/0 conn(0x7f0e0809e410 0x7f0e08134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:25.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.806+0000 7f9ae959a700 1 -- 172.21.15.3:0/3090629896 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0004230 con 0x7f9aec09ac90 2023-10-03T10:13:25.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.806+0000 7f9ae959a700 1 -- 172.21.15.3:0/3090629896 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae0004390 con 0x7f9aec09ac90 2023-10-03T10:13:25.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.806+0000 7f9ae959a700 1 -- 172.21.15.3:0/3090629896 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae00036a0 con 0x7f9aec09ac90 2023-10-03T10:13:25.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.806+0000 7f0e077fe700 1 -- 172.21.15.3:0/1918071986 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e0809d6f0 msgr2=0x7f0e0809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:25.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.806+0000 7f0e077fe700 1 --2- 172.21.15.3:0/1918071986 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e0809d6f0 0x7f0e0809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:25.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.806+0000 7f0e077fe700 1 -- 172.21.15.3:0/1918071986 >> v1:172.21.15.3:6789/0 conn(0x7f0e0809e000 legacy=0x7f0e081339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:25.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.806+0000 7f0e077fe700 1 -- 172.21.15.3:0/1918071986 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0809a500 con 0x7f0e0809e410 2023-10-03T10:13:25.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.806+0000 7f0e077fe700 1 --2- 172.21.15.3:0/1918071986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e0809e410 0x7f0e08134100 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009d90 tx=0x7f0e0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88803499102edee server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:25.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.807+0000 7f0e05ffb700 1 -- 172.21.15.3:0/1918071986 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0000e070 con 0x7f0e0809e410 2023-10-03T10:13:25.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.807+0000 7f0e05ffb700 1 -- 172.21.15.3:0/1918071986 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0000bae0 con 0x7f0e0809e410 2023-10-03T10:13:25.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.807+0000 7f0e05ffb700 1 -- 172.21.15.3:0/1918071986 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0000be70 con 0x7f0e0809e410 2023-10-03T10:13:25.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.807+0000 7f9af0c2a700 1 -- 172.21.15.3:0/3090629896 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9aec09ac90 msgr2=0x7f9aec13cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:25.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.807+0000 7f9af0c2a700 1 --2- 172.21.15.3:0/3090629896 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9aec09ac90 0x7f9aec13cd00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00098f0 tx=0x7f9ae0009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:25.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.807+0000 7f9af0c2a700 1 -- 172.21.15.3:0/3090629896 shutdown_connections 2023-10-03T10:13:25.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.807+0000 7f9af0c2a700 1 --2- 172.21.15.3:0/3090629896 >> v2:172.21.15.3:3300/0 conn(0x7f9aec13d240 0x7f9aec13f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:25.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.807+0000 7f9af0c2a700 1 --2- 172.21.15.3:0/3090629896 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9aec09ac90 0x7f9aec13cd00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:25.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.807+0000 7f9af0c2a700 1 -- 172.21.15.3:0/3090629896 >> 172.21.15.3:0/3090629896 conn(0x7f9aec0a4210 msgr2=0x7f9aec0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:25.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.807+0000 7f9af0c2a700 1 -- 172.21.15.3:0/3090629896 shutdown_connections 2023-10-03T10:13:25.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.807+0000 7f9af0c2a700 1 -- 172.21.15.3:0/3090629896 wait complete. 2023-10-03T10:13:25.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.807+0000 7f0e0d4bf700 1 -- 172.21.15.3:0/1918071986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e0809e410 msgr2=0x7f0e08134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:25.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.808+0000 7f0e0d4bf700 1 --2- 172.21.15.3:0/1918071986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e0809e410 0x7f0e08134100 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009d90 tx=0x7f0e0000b330 comp rx=0 tx=0).stop 2023-10-03T10:13:25.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.808+0000 7f0e0d4bf700 1 -- 172.21.15.3:0/1918071986 shutdown_connections 2023-10-03T10:13:25.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.808+0000 7f0e0d4bf700 1 --2- 172.21.15.3:0/1918071986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e0809e410 0x7f0e08134100 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:25.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.808+0000 7f0e0d4bf700 1 --2- 172.21.15.3:0/1918071986 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e0809d6f0 0x7f0e0809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:25.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.808+0000 7f0e0d4bf700 1 -- 172.21.15.3:0/1918071986 >> 172.21.15.3:0/1918071986 conn(0x7f0e08066f50 msgr2=0x7f0e0809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:25.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.808+0000 7f0e0d4bf700 1 -- 172.21.15.3:0/1918071986 shutdown_connections 2023-10-03T10:13:25.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.808+0000 7f0e0d4bf700 1 -- 172.21.15.3:0/1918071986 wait complete. 2023-10-03T10:13:25.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.808+0000 7f9af0c2a700 1 Processor -- start 2023-10-03T10:13:25.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.809+0000 7f9af0c2a700 1 -- start start 2023-10-03T10:13:25.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.809+0000 7f9af0c2a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9aec06c0e0 0x7f9aec130330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:25.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.809+0000 7f0e0d4bf700 1 Processor -- start 2023-10-03T10:13:25.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.809+0000 7f9af0c2a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9aec09ac90 0x7f9aec130870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:25.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.809+0000 7f9af0c2a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9aec13d240 0x7f9aec130db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:25.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.809+0000 7f9af0c2a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec09e0d0 con 0x7f9aec13d240 2023-10-03T10:13:25.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.809+0000 7f9af0c2a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec09daf0 con 0x7f9aec09ac90 2023-10-03T10:13:25.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.809+0000 7f9af0c2a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec136860 con 0x7f9aec06c0e0 2023-10-03T10:13:25.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.810+0000 7f9aea59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9aec06c0e0 0x7f9aec130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:25.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.810+0000 7f9aea59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9aec06c0e0 0x7f9aec130330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53876/0 (socket says 172.21.15.3:53876) 2023-10-03T10:13:25.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.810+0000 7f9aea59c700 1 -- 172.21.15.3:0/2283730287 learned_addr learned my addr 172.21.15.3:0/2283730287 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:25.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.810+0000 7f9ae9d9b700 1 --2- 172.21.15.3:0/2283730287 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9aec09ac90 0x7f9aec130870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:25.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.809+0000 7f0e0d4bf700 1 -- start start 2023-10-03T10:13:25.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.810+0000 7f0e0d4bf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0e0809d6f0 0x7f0e081cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:25.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.810+0000 7f0e0d4bf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e0809e000 0x7f0e081cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:25.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.810+0000 7f0e0d4bf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e0809e410 0x7f0e081194b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:25.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.810+0000 7f0e0d4bf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0811ccf0 con 0x7f0e0809e000 2023-10-03T10:13:25.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.810+0000 7f0e0d4bf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e081ae8d0 con 0x7f0e0809d6f0 2023-10-03T10:13:25.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.810+0000 7f0e0d4bf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e080a3140 con 0x7f0e0809e410 2023-10-03T10:13:25.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.810+0000 7f0e06ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0e0809d6f0 0x7f0e081cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:25.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.810+0000 7f0e06ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0e0809d6f0 0x7f0e081cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56790/0 (socket says 172.21.15.3:56790) 2023-10-03T10:13:25.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.810+0000 7f0e06ffd700 1 -- 172.21.15.3:0/2305019718 learned_addr learned my addr 172.21.15.3:0/2305019718 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:25.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.810+0000 7f0e077fe700 1 --2- 172.21.15.3:0/2305019718 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e0809e410 0x7f0e081194b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:25.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.810+0000 7f0e067fc700 1 --2- 172.21.15.3:0/2305019718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e0809e000 0x7f0e081cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:25.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.810+0000 7f9aea59c700 1 -- 172.21.15.3:0/2283730287 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9aec09ac90 msgr2=0x7f9aec130870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:25.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.810+0000 7f9aea59c700 1 --2- 172.21.15.3:0/2283730287 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9aec09ac90 0x7f9aec130870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:25.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.810+0000 7f9aea59c700 1 -- 172.21.15.3:0/2283730287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9aec13d240 msgr2=0x7f9aec130db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:25.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.810+0000 7f9aea59c700 1 --2- 172.21.15.3:0/2283730287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9aec13d240 0x7f9aec130db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:25.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.810+0000 7f9aea59c700 1 -- 172.21.15.3:0/2283730287 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae0009550 con 0x7f9aec06c0e0 2023-10-03T10:13:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.811+0000 7f0e06ffd700 1 -- 172.21.15.3:0/2305019718 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e0809e410 msgr2=0x7f0e081194b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.811+0000 7f0e06ffd700 1 --2- 172.21.15.3:0/2305019718 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e0809e410 0x7f0e081194b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.811+0000 7f0e06ffd700 1 -- 172.21.15.3:0/2305019718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e0809e000 msgr2=0x7f0e081cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.811+0000 7f0e06ffd700 1 --2- 172.21.15.3:0/2305019718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e0809e000 0x7f0e081cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.811+0000 7f0e06ffd700 1 -- 172.21.15.3:0/2305019718 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e00009a70 con 0x7f0e0809d6f0 2023-10-03T10:13:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.811+0000 7f0e067fc700 1 --2- 172.21.15.3:0/2305019718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e0809e000 0x7f0e081cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:25.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.811+0000 7f9aea59c700 1 --2- 172.21.15.3:0/2283730287 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9aec06c0e0 0x7f9aec130330 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00b470 tx=0x7f9adc00b780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:25.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.811+0000 7f9adaffd700 1 -- 172.21.15.3:0/2283730287 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc011830 con 0x7f9aec06c0e0 2023-10-03T10:13:25.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.811+0000 7f9af0c2a700 1 -- 172.21.15.3:0/2283730287 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aec0a03f0 con 0x7f9aec06c0e0 2023-10-03T10:13:25.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.811+0000 7f0e06ffd700 1 --2- 172.21.15.3:0/2305019718 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0e0809d6f0 0x7f0e081cccc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0df800e910 tx=0x7f0df800ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:25.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.811+0000 7f9af0c2a700 1 -- 172.21.15.3:0/2283730287 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aec09d770 con 0x7f9aec06c0e0 2023-10-03T10:13:25.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.811+0000 7f9adaffd700 1 -- 172.21.15.3:0/2283730287 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9adc011990 con 0x7f9aec06c0e0 2023-10-03T10:13:25.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.811+0000 7f9adaffd700 1 -- 172.21.15.3:0/2283730287 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc011d00 con 0x7f9aec06c0e0 2023-10-03T10:13:25.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.812+0000 7f0def7fe700 1 -- 172.21.15.3:0/2305019718 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df80096a0 con 0x7f0e0809d6f0 2023-10-03T10:13:25.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.812+0000 7f9af0c2a700 1 -- 172.21.15.3:0/2283730287 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ad00052e0 con 0x7f9aec06c0e0 2023-10-03T10:13:25.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.812+0000 7f0def7fe700 1 -- 172.21.15.3:0/2305019718 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df800ee60 con 0x7f0e0809d6f0 2023-10-03T10:13:25.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.812+0000 7f0def7fe700 1 -- 172.21.15.3:0/2305019718 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df800ca90 con 0x7f0e0809d6f0 2023-10-03T10:13:25.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.813+0000 7f0e0d4bf700 1 -- 172.21.15.3:0/2305019718 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e0811bab0 con 0x7f0e0809d6f0 2023-10-03T10:13:25.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.813+0000 7f0e0d4bf700 1 -- 172.21.15.3:0/2305019718 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e081cd740 con 0x7f0e0809d6f0 2023-10-03T10:13:25.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.813+0000 7f9adaffd700 1 -- 172.21.15.3:0/2283730287 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9adc01a400 con 0x7f9aec06c0e0 2023-10-03T10:13:25.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.813+0000 7f9adaffd700 1 -- 172.21.15.3:0/2283730287 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9adc03fd20 con 0x7f9aec06c0e0 2023-10-03T10:13:25.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.814+0000 7f0e0d4bf700 1 -- 172.21.15.3:0/2305019718 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e08074ea0 con 0x7f0e0809d6f0 2023-10-03T10:13:25.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.815+0000 7f0def7fe700 1 -- 172.21.15.3:0/2305019718 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0df800cbf0 con 0x7f0e0809d6f0 2023-10-03T10:13:25.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.815+0000 7f0def7fe700 1 -- 172.21.15.3:0/2305019718 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0df803ff50 con 0x7f0e0809d6f0 2023-10-03T10:13:25.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.819+0000 7f0def7fe700 1 -- 172.21.15.3:0/2305019718 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0df8011350 con 0x7f0e0809d6f0 2023-10-03T10:13:25.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.822+0000 7f9adaffd700 1 -- 172.21.15.3:0/2283730287 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9adc021710 con 0x7f9aec06c0e0 2023-10-03T10:13:25.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.872+0000 7f8c762f3700 1 Processor -- start 2023-10-03T10:13:25.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.872+0000 7f8c762f3700 1 -- start start 2023-10-03T10:13:25.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.872+0000 7f8c762f3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8c700a0a30 0x7f8c700a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:25.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.872+0000 7f8c762f3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c7009d630 0x7f8c700a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:25.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.872+0000 7f8c762f3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c700a2a70 con 0x7f8c70133930 2023-10-03T10:13:25.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.872+0000 7f8c762f3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c700a01d0 con 0x7f8c700a0a30 2023-10-03T10:13:25.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.872+0000 7f8c762f3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7009e2b0 con 0x7f8c7009d630 2023-10-03T10:13:25.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.873+0000 7f8c6f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c7009d630 0x7f8c700a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:25.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.873+0000 7f8c6ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8c700a0a30 0x7f8c700a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:25.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.873+0000 7f8c6f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c7009d630 0x7f8c700a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53892/0 (socket says 172.21.15.3:53892) 2023-10-03T10:13:25.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.873+0000 7f8c6f7fe700 1 -- 172.21.15.3:0/201141090 learned_addr learned my addr 172.21.15.3:0/201141090 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:25.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.873+0000 7f8c6f7fe700 1 -- 172.21.15.3:0/201141090 >> v2:172.21.15.3:3300/0 conn(0x7f8c700a0a30 msgr2=0x7f8c700a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:25.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.873+0000 7f8c6f7fe700 1 --2- 172.21.15.3:0/201141090 >> v2:172.21.15.3:3300/0 conn(0x7f8c700a0a30 0x7f8c700a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:25.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.873+0000 7f8c6f7fe700 1 -- 172.21.15.3:0/201141090 >> v1:172.21.15.3:6789/0 conn(0x7f8c70133930 legacy=0x7f8c70133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:25.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.873+0000 7f8c6effd700 1 -- 172.21.15.3:0/201141090 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1130766826 0 0) 0x7f8c700a2a70 con 0x7f8c70133930 2023-10-03T10:13:25.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.873+0000 7f8c6f7fe700 1 -- 172.21.15.3:0/201141090 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7009dc50 con 0x7f8c7009d630 2023-10-03T10:13:25.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.873+0000 7f8c6ffff700 1 --2- 172.21.15.3:0/201141090 >> v2:172.21.15.3:3300/0 conn(0x7f8c700a0a30 0x7f8c700a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:25.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.873+0000 7f8c6f7fe700 1 --2- 172.21.15.3:0/201141090 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c7009d630 0x7f8c700a1340 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8c640098f0 tx=0x7f8c64009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0a21e94848596cc server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:25.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.874+0000 7f8c6effd700 1 -- 172.21.15.3:0/201141090 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c64011830 con 0x7f8c7009d630 2023-10-03T10:13:25.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.874+0000 7f8c6effd700 1 -- 172.21.15.3:0/201141090 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c64011990 con 0x7f8c7009d630 2023-10-03T10:13:25.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.874+0000 7f8c6effd700 1 -- 172.21.15.3:0/201141090 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c64011d20 con 0x7f8c7009d630 2023-10-03T10:13:25.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.874+0000 7f8c762f3700 1 -- 172.21.15.3:0/201141090 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c7009d630 msgr2=0x7f8c700a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:25.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.874+0000 7f8c762f3700 1 --2- 172.21.15.3:0/201141090 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c7009d630 0x7f8c700a1340 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8c640098f0 tx=0x7f8c64009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:25.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.874+0000 7f8c762f3700 1 -- 172.21.15.3:0/201141090 shutdown_connections 2023-10-03T10:13:25.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.874+0000 7f8c762f3700 1 --2- 172.21.15.3:0/201141090 >> v2:172.21.15.3:3300/0 conn(0x7f8c700a0a30 0x7f8c700a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:25.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.874+0000 7f8c762f3700 1 --2- 172.21.15.3:0/201141090 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c7009d630 0x7f8c700a1340 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:25.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.874+0000 7f8c762f3700 1 -- 172.21.15.3:0/201141090 >> 172.21.15.3:0/201141090 conn(0x7f8c700a3050 msgr2=0x7f8c700a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:25.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.874+0000 7f8c762f3700 1 -- 172.21.15.3:0/201141090 shutdown_connections 2023-10-03T10:13:25.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.874+0000 7f8c762f3700 1 -- 172.21.15.3:0/201141090 wait complete. 2023-10-03T10:13:25.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.876+0000 7f8c762f3700 1 Processor -- start 2023-10-03T10:13:25.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.876+0000 7f8c762f3700 1 -- start start 2023-10-03T10:13:25.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.877+0000 7f8c762f3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c7009d630 0x7f8c701c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:25.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.877+0000 7f8c762f3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c700a0a30 0x7f8c701c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:25.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.877+0000 7f8c762f3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c70133930 0x7f8c701c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:25.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.877+0000 7f8c762f3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c70120700 con 0x7f8c7009d630 2023-10-03T10:13:25.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.877+0000 7f8c762f3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c70120510 con 0x7f8c700a0a30 2023-10-03T10:13:25.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.877+0000 7f8c762f3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c70122420 con 0x7f8c70133930 2023-10-03T10:13:25.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.877+0000 7f8c74890700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c70133930 0x7f8c701c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:25.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.877+0000 7f8c74890700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c70133930 0x7f8c701c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53898/0 (socket says 172.21.15.3:53898) 2023-10-03T10:13:25.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.877+0000 7f8c74890700 1 -- 172.21.15.3:0/116967778 learned_addr learned my addr 172.21.15.3:0/116967778 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:25.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.877+0000 7f8c6ffff700 1 --2- 172.21.15.3:0/116967778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c7009d630 0x7f8c701c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:25.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.877+0000 7f8c6f7fe700 1 --2- 172.21.15.3:0/116967778 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c700a0a30 0x7f8c701c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:25.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.877+0000 7f8c74890700 1 -- 172.21.15.3:0/116967778 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c700a0a30 msgr2=0x7f8c701c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:25.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.877+0000 7f8c74890700 1 --2- 172.21.15.3:0/116967778 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c700a0a30 0x7f8c701c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:25.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.877+0000 7f8c74890700 1 -- 172.21.15.3:0/116967778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c7009d630 msgr2=0x7f8c701c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:25.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.877+0000 7f8c74890700 1 --2- 172.21.15.3:0/116967778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c7009d630 0x7f8c701c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:25.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.877+0000 7f8c74890700 1 -- 172.21.15.3:0/116967778 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c600094a0 con 0x7f8c70133930 2023-10-03T10:13:25.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.877+0000 7f8c6ffff700 1 --2- 172.21.15.3:0/116967778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c7009d630 0x7f8c701c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:25.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.878+0000 7f8c74890700 1 --2- 172.21.15.3:0/116967778 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c70133930 0x7f8c701c8a10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800c810 tx=0x7f8c6800cb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:25.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.878+0000 7f8c6cff9700 1 -- 172.21.15.3:0/116967778 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c68003e30 con 0x7f8c70133930 2023-10-03T10:13:25.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.878+0000 7f8c6cff9700 1 -- 172.21.15.3:0/116967778 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c68005070 con 0x7f8c70133930 2023-10-03T10:13:25.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.878+0000 7f8c6cff9700 1 -- 172.21.15.3:0/116967778 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c68005420 con 0x7f8c70133930 2023-10-03T10:13:25.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.878+0000 7f8c762f3700 1 -- 172.21.15.3:0/116967778 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c64009550 con 0x7f8c70133930 2023-10-03T10:13:25.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.878+0000 7f8c762f3700 1 -- 172.21.15.3:0/116967778 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c70122140 con 0x7f8c70133930 2023-10-03T10:13:25.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.879+0000 7f8c6cff9700 1 -- 172.21.15.3:0/116967778 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8c68005580 con 0x7f8c70133930 2023-10-03T10:13:25.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.879+0000 7f8c6cff9700 1 -- 172.21.15.3:0/116967778 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8c6803f340 con 0x7f8c70133930 2023-10-03T10:13:25.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.880+0000 7f8c762f3700 1 -- 172.21.15.3:0/116967778 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c70121de0 con 0x7f8c70133930 2023-10-03T10:13:25.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.890+0000 7f8c6cff9700 1 -- 172.21.15.3:0/116967778 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8c70121de0 con 0x7f8c70133930 2023-10-03T10:13:25.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.933+0000 7fe3d10d9700 1 -- 172.21.15.3:0/883507703 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe3cc074ea0 con 0x7fe3cc09de50 2023-10-03T10:13:25.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.933+0000 7fe3b37fe700 1 -- 172.21.15.3:0/883507703 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe3bc021030 con 0x7fe3cc09de50 2023-10-03T10:13:25.933 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:25.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.937+0000 7fe3d10d9700 1 -- 172.21.15.3:0/883507703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3cc09de50 msgr2=0x7fe3cc11a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:25.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.937+0000 7fe3d10d9700 1 --2- 172.21.15.3:0/883507703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3cc09de50 0x7fe3cc11a5a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00ebe0 tx=0x7fe3bc00eef0 comp rx=0 tx=0).stop 2023-10-03T10:13:25.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.937+0000 7fe3d10d9700 1 -- 172.21.15.3:0/883507703 shutdown_connections 2023-10-03T10:13:25.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.937+0000 7fe3d10d9700 1 --2- 172.21.15.3:0/883507703 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3cc1338a0 0x7fe3cc11b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:25.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.937+0000 7fe3d10d9700 1 --2- 172.21.15.3:0/883507703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3cc09de50 0x7fe3cc11a5a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:25.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.937+0000 7fe3d10d9700 1 --2- 172.21.15.3:0/883507703 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3cc09e760 0x7fe3cc11aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:25.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.937+0000 7fe3d10d9700 1 -- 172.21.15.3:0/883507703 >> 172.21.15.3:0/883507703 conn(0x7fe3cc0a2aa0 msgr2=0x7fe3cc130730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:25.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.937+0000 7fe3d10d9700 1 -- 172.21.15.3:0/883507703 shutdown_connections 2023-10-03T10:13:25.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:25.937+0000 7fe3d10d9700 1 -- 172.21.15.3:0/883507703 wait complete. 2023-10-03T10:13:25.957 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:26.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.034+0000 7f0e0d4bf700 1 -- 172.21.15.3:0/2305019718 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0e0811c990 con 0x7f0e0809d6f0 2023-10-03T10:13:26.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.035+0000 7f0def7fe700 1 -- 172.21.15.3:0/2305019718 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0df8017070 con 0x7f0e0809d6f0 2023-10-03T10:13:26.035 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:26.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.037+0000 7f0e0d4bf700 1 -- 172.21.15.3:0/2305019718 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0e0809d6f0 msgr2=0x7f0e081cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:26.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.037+0000 7f0e0d4bf700 1 --2- 172.21.15.3:0/2305019718 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0e0809d6f0 0x7f0e081cccc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0df800e910 tx=0x7f0df800ec20 comp rx=0 tx=0).stop 2023-10-03T10:13:26.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.037+0000 7f0e0d4bf700 1 -- 172.21.15.3:0/2305019718 shutdown_connections 2023-10-03T10:13:26.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.037+0000 7f0e0d4bf700 1 --2- 172.21.15.3:0/2305019718 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e0809e410 0x7f0e081194b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:26.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.037+0000 7f0e0d4bf700 1 --2- 172.21.15.3:0/2305019718 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0e0809d6f0 0x7f0e081cccc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:26.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.037+0000 7f0e0d4bf700 1 --2- 172.21.15.3:0/2305019718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e0809e000 0x7f0e081cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:26.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.037+0000 7f0e0d4bf700 1 -- 172.21.15.3:0/2305019718 >> 172.21.15.3:0/2305019718 conn(0x7f0e08066f50 msgr2=0x7f0e081bafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:26.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.038+0000 7f0e0d4bf700 1 -- 172.21.15.3:0/2305019718 shutdown_connections 2023-10-03T10:13:26.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.038+0000 7f0e0d4bf700 1 -- 172.21.15.3:0/2305019718 wait complete. 2023-10-03T10:13:26.052 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:26.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.052+0000 7f9af0c2a700 1 -- 172.21.15.3:0/2283730287 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9ad0005860 con 0x7f9aec06c0e0 2023-10-03T10:13:26.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.053+0000 7f9adaffd700 1 -- 172.21.15.3:0/2283730287 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9adc028020 con 0x7f9aec06c0e0 2023-10-03T10:13:26.053 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:26.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.055+0000 7f9af0c2a700 1 -- 172.21.15.3:0/2283730287 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9aec06c0e0 msgr2=0x7f9aec130330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:26.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.055+0000 7f9af0c2a700 1 --2- 172.21.15.3:0/2283730287 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9aec06c0e0 0x7f9aec130330 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00b470 tx=0x7f9adc00b780 comp rx=0 tx=0).stop 2023-10-03T10:13:26.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.055+0000 7f9af0c2a700 1 -- 172.21.15.3:0/2283730287 shutdown_connections 2023-10-03T10:13:26.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.055+0000 7f9af0c2a700 1 --2- 172.21.15.3:0/2283730287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9aec13d240 0x7f9aec130db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:26.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.055+0000 7f9af0c2a700 1 --2- 172.21.15.3:0/2283730287 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9aec06c0e0 0x7f9aec130330 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:26.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.055+0000 7f9af0c2a700 1 --2- 172.21.15.3:0/2283730287 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9aec09ac90 0x7f9aec130870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:26.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.055+0000 7f9af0c2a700 1 -- 172.21.15.3:0/2283730287 >> 172.21.15.3:0/2283730287 conn(0x7f9aec0a4210 msgr2=0x7f9aec133560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:26.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.055+0000 7f9af0c2a700 1 -- 172.21.15.3:0/2283730287 shutdown_connections 2023-10-03T10:13:26.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.055+0000 7f9af0c2a700 1 -- 172.21.15.3:0/2283730287 wait complete. 2023-10-03T10:13:26.069 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:26.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.100+0000 7f8c762f3700 1 -- 172.21.15.3:0/116967778 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8c70121de0 con 0x7f8c70133930 2023-10-03T10:13:26.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.101+0000 7f8c6cff9700 1 -- 172.21.15.3:0/116967778 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8c70121de0 con 0x7f8c70133930 2023-10-03T10:13:26.101 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:26.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.103+0000 7f8c762f3700 1 -- 172.21.15.3:0/116967778 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c70133930 msgr2=0x7f8c701c8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:26.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.103+0000 7f8c762f3700 1 --2- 172.21.15.3:0/116967778 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c70133930 0x7f8c701c8a10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800c810 tx=0x7f8c6800cb20 comp rx=0 tx=0).stop 2023-10-03T10:13:26.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.103+0000 7f8c762f3700 1 -- 172.21.15.3:0/116967778 shutdown_connections 2023-10-03T10:13:26.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.103+0000 7f8c762f3700 1 --2- 172.21.15.3:0/116967778 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c70133930 0x7f8c701c8a10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:26.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.103+0000 7f8c762f3700 1 --2- 172.21.15.3:0/116967778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c7009d630 0x7f8c701c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:26.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.103+0000 7f8c762f3700 1 --2- 172.21.15.3:0/116967778 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c700a0a30 0x7f8c701c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:26.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.103+0000 7f8c762f3700 1 -- 172.21.15.3:0/116967778 >> 172.21.15.3:0/116967778 conn(0x7f8c700a3050 msgr2=0x7f8c701bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:26.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.103+0000 7f8c762f3700 1 -- 172.21.15.3:0/116967778 shutdown_connections 2023-10-03T10:13:26.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:26.103+0000 7f8c762f3700 1 -- 172.21.15.3:0/116967778 wait complete. 2023-10-03T10:13:26.117 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:26.958 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:27.053 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:27.070 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:27.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.107+0000 7f79f8071700 1 Processor -- start 2023-10-03T10:13:27.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.107+0000 7f79f8071700 1 -- start start 2023-10-03T10:13:27.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.108+0000 7f79f8071700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f79f0133a80 0x7f79f0133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:27.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.108+0000 7f79f8071700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79f0130a70 0x7f79f013b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:27.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.108+0000 7f79f8071700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f006bc00 con 0x7f79f012ff20 2023-10-03T10:13:27.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.108+0000 7f79f8071700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f006c570 con 0x7f79f0133a80 2023-10-03T10:13:27.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.108+0000 7f79f8071700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79f00a0040 con 0x7f79f0130a70 2023-10-03T10:13:27.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.108+0000 7f79f5e0d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f79f0133a80 0x7f79f0133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:27.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.108+0000 7f79f660e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79f0130a70 0x7f79f013b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:27.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.108+0000 7f79f5e0d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f79f0133a80 0x7f79f0133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53658/0 (socket says 172.21.15.3:53658) 2023-10-03T10:13:27.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.108+0000 7f79f5e0d700 1 -- 172.21.15.3:0/410462266 learned_addr learned my addr 172.21.15.3:0/410462266 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:27.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.108+0000 7f79f660e700 1 --2- 172.21.15.3:0/410462266 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79f0130a70 0x7f79f013b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53900/0 (socket says 172.21.15.3:53900) 2023-10-03T10:13:27.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.109+0000 7f79f5e0d700 1 -- 172.21.15.3:0/410462266 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79f0130a70 msgr2=0x7f79f013b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:27.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.109+0000 7f79f5e0d700 1 --2- 172.21.15.3:0/410462266 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79f0130a70 0x7f79f013b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.109+0000 7f79f5e0d700 1 -- 172.21.15.3:0/410462266 >> v1:172.21.15.3:6789/0 conn(0x7f79f012ff20 legacy=0x7f79f0130360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:27.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.109+0000 7f79f5e0d700 1 -- 172.21.15.3:0/410462266 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f0133860 con 0x7f79f0133a80 2023-10-03T10:13:27.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.109+0000 7f79f660e700 1 --2- 172.21.15.3:0/410462266 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79f0130a70 0x7f79f013b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:27.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.109+0000 7f79f5e0d700 1 --2- 172.21.15.3:0/410462266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79f0133a80 0x7f79f0133e50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0098f0 tx=0x7f79ec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad9ef7ee88ff427 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:27.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.109+0000 7f79f4e0b700 1 -- 172.21.15.3:0/410462266 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec011830 con 0x7f79f0133a80 2023-10-03T10:13:27.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.109+0000 7f79f4e0b700 1 -- 172.21.15.3:0/410462266 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ec011990 con 0x7f79f0133a80 2023-10-03T10:13:27.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.109+0000 7f79f4e0b700 1 -- 172.21.15.3:0/410462266 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec011d20 con 0x7f79f0133a80 2023-10-03T10:13:27.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.110+0000 7f79f8071700 1 -- 172.21.15.3:0/410462266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79f0133a80 msgr2=0x7f79f0133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:27.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.110+0000 7f79f8071700 1 --2- 172.21.15.3:0/410462266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79f0133a80 0x7f79f0133e50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0098f0 tx=0x7f79ec009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:27.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.110+0000 7f79f8071700 1 -- 172.21.15.3:0/410462266 shutdown_connections 2023-10-03T10:13:27.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.110+0000 7f79f8071700 1 --2- 172.21.15.3:0/410462266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79f0133a80 0x7f79f0133e50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.110+0000 7f79f8071700 1 --2- 172.21.15.3:0/410462266 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79f0130a70 0x7f79f013b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.110+0000 7f79f8071700 1 -- 172.21.15.3:0/410462266 >> 172.21.15.3:0/410462266 conn(0x7f79f00a4210 msgr2=0x7f79f00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:27.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.110+0000 7f79f8071700 1 -- 172.21.15.3:0/410462266 shutdown_connections 2023-10-03T10:13:27.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.110+0000 7f79f8071700 1 -- 172.21.15.3:0/410462266 wait complete. 2023-10-03T10:13:27.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.111+0000 7f79f8071700 1 Processor -- start 2023-10-03T10:13:27.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.111+0000 7f79f8071700 1 -- start start 2023-10-03T10:13:27.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.112+0000 7f79f8071700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79f012ff20 0x7f79f009c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:27.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.112+0000 7f79f8071700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79f0130a70 0x7f79f009cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:27.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.112+0000 7f79f8071700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79f0133a80 0x7f79f009d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:27.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.112+0000 7f79f8071700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f0120ab0 con 0x7f79f012ff20 2023-10-03T10:13:27.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.112+0000 7f79f8071700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f011ffe0 con 0x7f79f0133a80 2023-10-03T10:13:27.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.112+0000 7f79f8071700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79f01207c0 con 0x7f79f0130a70 2023-10-03T10:13:27.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.112+0000 7f79f560c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79f0130a70 0x7f79f009cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:27.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.112+0000 7f79f560c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79f0130a70 0x7f79f009cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53902/0 (socket says 172.21.15.3:53902) 2023-10-03T10:13:27.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.112+0000 7f79f560c700 1 -- 172.21.15.3:0/416139847 learned_addr learned my addr 172.21.15.3:0/416139847 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:27.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.112+0000 7f79f660e700 1 --2- 172.21.15.3:0/416139847 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79f0133a80 0x7f79f009d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:27.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.112+0000 7f79f5e0d700 1 --2- 172.21.15.3:0/416139847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79f012ff20 0x7f79f009c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:27.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.112+0000 7f79f560c700 1 -- 172.21.15.3:0/416139847 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79f0133a80 msgr2=0x7f79f009d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:27.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.112+0000 7f79f560c700 1 --2- 172.21.15.3:0/416139847 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79f0133a80 0x7f79f009d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.112+0000 7f79f560c700 1 -- 172.21.15.3:0/416139847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79f012ff20 msgr2=0x7f79f009c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:27.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.112+0000 7f79f560c700 1 --2- 172.21.15.3:0/416139847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79f012ff20 0x7f79f009c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.113+0000 7f79f560c700 1 -- 172.21.15.3:0/416139847 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e8009a70 con 0x7f79f0130a70 2023-10-03T10:13:27.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.113+0000 7f79f660e700 1 --2- 172.21.15.3:0/416139847 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79f0133a80 0x7f79f009d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:27.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.113+0000 7f79f560c700 1 --2- 172.21.15.3:0/416139847 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79f0130a70 0x7f79f009cc20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f79e000c4e0 tx=0x7f79e000c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:27.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.113+0000 7f79e67fc700 1 -- 172.21.15.3:0/416139847 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e00052d0 con 0x7f79f0130a70 2023-10-03T10:13:27.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.113+0000 7f79e67fc700 1 -- 172.21.15.3:0/416139847 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e0005430 con 0x7f79f0130a70 2023-10-03T10:13:27.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.113+0000 7f79f8071700 1 -- 172.21.15.3:0/416139847 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79ec009550 con 0x7f79f0130a70 2023-10-03T10:13:27.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.113+0000 7f79e67fc700 1 -- 172.21.15.3:0/416139847 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e00038c0 con 0x7f79f0130a70 2023-10-03T10:13:27.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.113+0000 7f79f8071700 1 -- 172.21.15.3:0/416139847 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f01202c0 con 0x7f79f0130a70 2023-10-03T10:13:27.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.114+0000 7f79e67fc700 1 -- 172.21.15.3:0/416139847 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f79e0003a20 con 0x7f79f0130a70 2023-10-03T10:13:27.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.114+0000 7f79e67fc700 1 -- 172.21.15.3:0/416139847 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f79e003eaa0 con 0x7f79f0130a70 2023-10-03T10:13:27.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.114+0000 7f79f8071700 1 -- 172.21.15.3:0/416139847 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f0074ea0 con 0x7f79f0130a70 2023-10-03T10:13:27.117 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:27.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.120+0000 7f79e67fc700 1 -- 172.21.15.3:0/416139847 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f79e0018260 con 0x7f79f0130a70 2023-10-03T10:13:27.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.217+0000 7f64deb30700 1 Processor -- start 2023-10-03T10:13:27.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.218+0000 7f64deb30700 1 -- start start 2023-10-03T10:13:27.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.218+0000 7f64deb30700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f64d8133510 0x7f64d81358f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:27.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.218+0000 7f64deb30700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64d8135e30 0x7f64d8138210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:27.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.218+0000 7f64deb30700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d8130260 con 0x7f64d8130670 2023-10-03T10:13:27.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.218+0000 7f64deb30700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d809a350 con 0x7f64d8133510 2023-10-03T10:13:27.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.218+0000 7f64deb30700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d8071ff0 con 0x7f64d8135e30 2023-10-03T10:13:27.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.219+0000 7f64dd0cd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64d8135e30 0x7f64d8138210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:27.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.219+0000 7f64dd0cd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64d8135e30 0x7f64d8138210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56820/0 (socket says 172.21.15.3:56820) 2023-10-03T10:13:27.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.219+0000 7f64dd0cd700 1 -- 172.21.15.3:0/2984289501 learned_addr learned my addr 172.21.15.3:0/2984289501 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:27.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.219+0000 7f64d7fff700 1 --2- 172.21.15.3:0/2984289501 >> v2:172.21.15.3:3300/0 conn(0x7f64d8133510 0x7f64d81358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:27.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.219+0000 7f64dd0cd700 1 -- 172.21.15.3:0/2984289501 >> v2:172.21.15.3:3300/0 conn(0x7f64d8133510 msgr2=0x7f64d81358f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:27.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.219+0000 7f64dd0cd700 1 --2- 172.21.15.3:0/2984289501 >> v2:172.21.15.3:3300/0 conn(0x7f64d8133510 0x7f64d81358f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.219+0000 7f64dd0cd700 1 -- 172.21.15.3:0/2984289501 >> v1:172.21.15.3:6789/0 conn(0x7f64d8130670 legacy=0x7f64d8130a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:27.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.219+0000 7f64dd0cd700 1 -- 172.21.15.3:0/2984289501 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d809ecf0 con 0x7f64d8135e30 2023-10-03T10:13:27.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.219+0000 7f64d7fff700 1 --2- 172.21.15.3:0/2984289501 >> v2:172.21.15.3:3300/0 conn(0x7f64d8133510 0x7f64d81358f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:27.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.220+0000 7f64dd0cd700 1 --2- 172.21.15.3:0/2984289501 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64d8135e30 0x7f64d8138210 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f64d000d150 tx=0x7f64d000d460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bb7a1ab7ad1e958 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:27.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.220+0000 7f64d77fe700 1 -- 172.21.15.3:0/2984289501 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d0010070 con 0x7f64d8135e30 2023-10-03T10:13:27.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.220+0000 7f64d77fe700 1 -- 172.21.15.3:0/2984289501 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d000dd10 con 0x7f64d8135e30 2023-10-03T10:13:27.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.220+0000 7f64d77fe700 1 -- 172.21.15.3:0/2984289501 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d000b310 con 0x7f64d8135e30 2023-10-03T10:13:27.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.221+0000 7f64deb30700 1 -- 172.21.15.3:0/2984289501 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64d8135e30 msgr2=0x7f64d8138210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:27.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.221+0000 7f64deb30700 1 --2- 172.21.15.3:0/2984289501 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64d8135e30 0x7f64d8138210 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f64d000d150 tx=0x7f64d000d460 comp rx=0 tx=0).stop 2023-10-03T10:13:27.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.221+0000 7f64deb30700 1 -- 172.21.15.3:0/2984289501 shutdown_connections 2023-10-03T10:13:27.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.221+0000 7f64deb30700 1 --2- 172.21.15.3:0/2984289501 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64d8135e30 0x7f64d8138210 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.221+0000 7f64deb30700 1 --2- 172.21.15.3:0/2984289501 >> v2:172.21.15.3:3300/0 conn(0x7f64d8133510 0x7f64d81358f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.221+0000 7f64deb30700 1 -- 172.21.15.3:0/2984289501 >> 172.21.15.3:0/2984289501 conn(0x7f64d80a2aa0 msgr2=0x7f64d80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:27.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.221+0000 7f64deb30700 1 -- 172.21.15.3:0/2984289501 shutdown_connections 2023-10-03T10:13:27.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.221+0000 7f64deb30700 1 -- 172.21.15.3:0/2984289501 wait complete. 2023-10-03T10:13:27.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.222+0000 7f64deb30700 1 Processor -- start 2023-10-03T10:13:27.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.222+0000 7f64deb30700 1 -- start start 2023-10-03T10:13:27.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.223+0000 7f64deb30700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64d8130670 0x7f64d809d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:27.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.223+0000 7f64deb30700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64d8133510 0x7f64d809d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:27.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.223+0000 7f64deb30700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64d8135e30 0x7f64d81c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:27.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.223+0000 7f64deb30700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d811ea00 con 0x7f64d8135e30 2023-10-03T10:13:27.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.223+0000 7f64deb30700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d811ef20 con 0x7f64d8133510 2023-10-03T10:13:27.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.223+0000 7f64deb30700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f64d812db30 con 0x7f64d8130670 2023-10-03T10:13:27.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.223+0000 7f64d7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64d8133510 0x7f64d809d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:27.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.223+0000 7f64dc8cc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64d8130670 0x7f64d809d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:27.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.223+0000 7f64dc8cc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64d8130670 0x7f64d809d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53918/0 (socket says 172.21.15.3:53918) 2023-10-03T10:13:27.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.223+0000 7f64d7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64d8133510 0x7f64d809d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56826/0 (socket says 172.21.15.3:56826) 2023-10-03T10:13:27.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.223+0000 7f64dc8cc700 1 -- 172.21.15.3:0/1346836666 learned_addr learned my addr 172.21.15.3:0/1346836666 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:27.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.223+0000 7f64dd0cd700 1 --2- 172.21.15.3:0/1346836666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64d8135e30 0x7f64d81c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:27.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.224+0000 7f64dc8cc700 1 -- 172.21.15.3:0/1346836666 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64d8133510 msgr2=0x7f64d809d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:27.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.224+0000 7f64dc8cc700 1 --2- 172.21.15.3:0/1346836666 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64d8133510 0x7f64d809d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.224+0000 7f64dc8cc700 1 -- 172.21.15.3:0/1346836666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64d8135e30 msgr2=0x7f64d81c81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:27.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.224+0000 7f64dc8cc700 1 --2- 172.21.15.3:0/1346836666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64d8135e30 0x7f64d81c81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.224+0000 7f64dc8cc700 1 -- 172.21.15.3:0/1346836666 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c40094a0 con 0x7f64d8130670 2023-10-03T10:13:27.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.224+0000 7f64dc8cc700 1 --2- 172.21.15.3:0/1346836666 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64d8130670 0x7f64d809d040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f64cc00c4e0 tx=0x7f64cc00c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:27.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.224+0000 7f64d57fa700 1 -- 172.21.15.3:0/1346836666 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc0052d0 con 0x7f64d8130670 2023-10-03T10:13:27.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.224+0000 7f64d57fa700 1 -- 172.21.15.3:0/1346836666 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64cc00ce90 con 0x7f64d8130670 2023-10-03T10:13:27.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.224+0000 7f64deb30700 1 -- 172.21.15.3:0/1346836666 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d000bd40 con 0x7f64d8130670 2023-10-03T10:13:27.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.224+0000 7f64d57fa700 1 -- 172.21.15.3:0/1346836666 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc003680 con 0x7f64d8130670 2023-10-03T10:13:27.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.224+0000 7f64deb30700 1 -- 172.21.15.3:0/1346836666 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d811d5c0 con 0x7f64d8130670 2023-10-03T10:13:27.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.225+0000 7f64d57fa700 1 -- 172.21.15.3:0/1346836666 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f64cc0037e0 con 0x7f64d8130670 2023-10-03T10:13:27.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.225+0000 7f64deb30700 1 -- 172.21.15.3:0/1346836666 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d8074ea0 con 0x7f64d8130670 2023-10-03T10:13:27.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.226+0000 7f64d57fa700 1 -- 172.21.15.3:0/1346836666 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f64cc03ebe0 con 0x7f64d8130670 2023-10-03T10:13:27.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.235+0000 7f64d57fa700 1 -- 172.21.15.3:0/1346836666 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f64cc011590 con 0x7f64d8130670 2023-10-03T10:13:27.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.262+0000 7fd5b0fd5700 1 Processor -- start 2023-10-03T10:13:27.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.262+0000 7fd5b0fd5700 1 -- start start 2023-10-03T10:13:27.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.263+0000 7f5fae95b700 1 Processor -- start 2023-10-03T10:13:27.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.263+0000 7fd5b0fd5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd5ac09d6f0 0x7fd5ac09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:27.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.263+0000 7fd5b0fd5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd5ac09e000 0x7fd5ac1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:27.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.263+0000 7fd5b0fd5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5ac09b120 con 0x7fd5ac09e410 2023-10-03T10:13:27.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.263+0000 7fd5b0fd5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac06b460 con 0x7fd5ac09d6f0 2023-10-03T10:13:27.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.263+0000 7fd5b0fd5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac09a7e0 con 0x7fd5ac09e000 2023-10-03T10:13:27.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.263+0000 7fd5aa59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd5ac09d6f0 0x7fd5ac09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:27.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.263+0000 7fd5aa59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd5ac09d6f0 0x7fd5ac09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56840/0 (socket says 172.21.15.3:56840) 2023-10-03T10:13:27.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.263+0000 7fd5aa59c700 1 -- 172.21.15.3:0/2335751493 learned_addr learned my addr 172.21.15.3:0/2335751493 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:27.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.263+0000 7f5fae95b700 1 -- start start 2023-10-03T10:13:27.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.264+0000 7fd5a9d9b700 1 --2- 172.21.15.3:0/2335751493 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd5ac09e000 0x7fd5ac1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:27.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.264+0000 7f5fae95b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5fa806f5d0 0x7f5fa812ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:27.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.264+0000 7f5fae95b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fa8135c20 0x7f5fa8138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:27.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.264+0000 7f5fae95b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa806fa40 con 0x7f5fa812cf60 2023-10-03T10:13:27.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.264+0000 7f5fae95b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fa812cd90 con 0x7f5fa806f5d0 2023-10-03T10:13:27.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.264+0000 7f5fae95b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa8071780 con 0x7f5fa8135c20 2023-10-03T10:13:27.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.264+0000 7fd5aa59c700 1 -- 172.21.15.3:0/2335751493 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd5ac09e000 msgr2=0x7fd5ac1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:27.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.264+0000 7fd5aa59c700 1 --2- 172.21.15.3:0/2335751493 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd5ac09e000 0x7fd5ac1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.264+0000 7fd5aa59c700 1 -- 172.21.15.3:0/2335751493 >> v1:172.21.15.3:6789/0 conn(0x7fd5ac09e410 legacy=0x7fd5ac133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:27.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.264+0000 7fd5aa59c700 1 -- 172.21.15.3:0/2335751493 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ac09a500 con 0x7fd5ac09d6f0 2023-10-03T10:13:27.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.264+0000 7f5fa77fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5fa806f5d0 0x7f5fa812ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:27.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.264+0000 7f5fa7fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5fa812cf60 0x7f5fa812f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48778/0 (socket says 172.21.15.3:48778) 2023-10-03T10:13:27.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.264+0000 7f5fa7fff700 1 -- 172.21.15.3:0/2396562404 learned_addr learned my addr 172.21.15.3:0/2396562404 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:27.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.264+0000 7f5fa6ffd700 1 -- 172.21.15.3:0/2396562404 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3952330988 0 0) 0x7f5fa806fa40 con 0x7f5fa812cf60 2023-10-03T10:13:27.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.265+0000 7f5fa6ffd700 1 -- 172.21.15.3:0/2396562404 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f900035b0 con 0x7f5fa812cf60 2023-10-03T10:13:27.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.265+0000 7f5facef8700 1 --2- 172.21.15.3:0/2396562404 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fa8135c20 0x7f5fa8138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:27.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.265+0000 7fd5aa59c700 1 --2- 172.21.15.3:0/2335751493 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd5ac09d6f0 0x7fd5ac09dac0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd59c009920 tx=0x7fd59c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c282bc032018c46 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:27.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.265+0000 7f5fa77fe700 1 -- 172.21.15.3:0/2396562404 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fa8135c20 msgr2=0x7f5fa8138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:27.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.265+0000 7f5fa77fe700 1 --2- 172.21.15.3:0/2396562404 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fa8135c20 0x7f5fa8138000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.265+0000 7f5fa77fe700 1 -- 172.21.15.3:0/2396562404 >> v1:172.21.15.3:6789/0 conn(0x7f5fa812cf60 legacy=0x7f5fa812f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:27.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.265+0000 7f5fa77fe700 1 -- 172.21.15.3:0/2396562404 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa8071a40 con 0x7f5fa806f5d0 2023-10-03T10:13:27.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.265+0000 7f5fa77fe700 1 --2- 172.21.15.3:0/2396562404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fa806f5d0 0x7f5fa812ffc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00bae0 tx=0x7f5f9c00bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261ff748e0c83edc server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:27.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.265+0000 7fd5a959a700 1 -- 172.21.15.3:0/2335751493 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c011830 con 0x7fd5ac09d6f0 2023-10-03T10:13:27.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.265+0000 7f5fa6ffd700 1 -- 172.21.15.3:0/2396562404 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c00dc70 con 0x7f5fa806f5d0 2023-10-03T10:13:27.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.265+0000 7fd5a959a700 1 -- 172.21.15.3:0/2335751493 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59c011990 con 0x7fd5ac09d6f0 2023-10-03T10:13:27.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.265+0000 7fd5a959a700 1 -- 172.21.15.3:0/2335751493 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c011d20 con 0x7fd5ac09d6f0 2023-10-03T10:13:27.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.265+0000 7f5fa6ffd700 1 -- 172.21.15.3:0/2396562404 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9c00ddd0 con 0x7f5fa806f5d0 2023-10-03T10:13:27.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.265+0000 7f5fa6ffd700 1 -- 172.21.15.3:0/2396562404 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c010670 con 0x7f5fa806f5d0 2023-10-03T10:13:27.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.266+0000 7fd5b0fd5700 1 -- 172.21.15.3:0/2335751493 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd5ac09d6f0 msgr2=0x7fd5ac09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:27.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.266+0000 7fd5b0fd5700 1 --2- 172.21.15.3:0/2335751493 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd5ac09d6f0 0x7fd5ac09dac0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd59c009920 tx=0x7fd59c009c30 comp rx=0 tx=0).stop 2023-10-03T10:13:27.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.266+0000 7fd5b0fd5700 1 -- 172.21.15.3:0/2335751493 shutdown_connections 2023-10-03T10:13:27.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.266+0000 7fd5b0fd5700 1 --2- 172.21.15.3:0/2335751493 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd5ac09d6f0 0x7fd5ac09dac0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.266+0000 7fd5b0fd5700 1 --2- 172.21.15.3:0/2335751493 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd5ac09e000 0x7fd5ac1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.266+0000 7fd5b0fd5700 1 -- 172.21.15.3:0/2335751493 >> 172.21.15.3:0/2335751493 conn(0x7fd5ac066f50 msgr2=0x7fd5ac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:27.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.266+0000 7fd5b0fd5700 1 -- 172.21.15.3:0/2335751493 shutdown_connections 2023-10-03T10:13:27.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.266+0000 7fd5b0fd5700 1 -- 172.21.15.3:0/2335751493 wait complete. 2023-10-03T10:13:27.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.267+0000 7f5fae95b700 1 -- 172.21.15.3:0/2396562404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fa806f5d0 msgr2=0x7f5fa812ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:27.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.267+0000 7f5fae95b700 1 --2- 172.21.15.3:0/2396562404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fa806f5d0 0x7f5fa812ffc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00bae0 tx=0x7f5f9c00bdf0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.268+0000 7fd5b0fd5700 1 Processor -- start 2023-10-03T10:13:27.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.268+0000 7fd5b0fd5700 1 -- start start 2023-10-03T10:13:27.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.268+0000 7fd5b0fd5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5ac09d6f0 0x7fd5ac122600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:27.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.268+0000 7fd5b0fd5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd5ac09e000 0x7fd5ac122b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:27.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.268+0000 7fd5b0fd5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd5ac09e410 0x7fd5ac123080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:27.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.268+0000 7fd5b0fd5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac0a1e20 con 0x7fd5ac09d6f0 2023-10-03T10:13:27.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.268+0000 7fd5b0fd5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac09c710 con 0x7fd5ac09e000 2023-10-03T10:13:27.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.268+0000 7fd5b0fd5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac124f90 con 0x7fd5ac09e410 2023-10-03T10:13:27.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.268+0000 7fd5aa59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5ac09d6f0 0x7fd5ac122600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:27.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.268+0000 7fd5aa59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5ac09d6f0 0x7fd5ac122600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53708/0 (socket says 172.21.15.3:53708) 2023-10-03T10:13:27.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.268+0000 7fd5aa59c700 1 -- 172.21.15.3:0/2644165838 learned_addr learned my addr 172.21.15.3:0/2644165838 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:27.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.268+0000 7fd5a9d9b700 1 --2- 172.21.15.3:0/2644165838 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd5ac09e000 0x7fd5ac122b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:27.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.269+0000 7fd5a9d9b700 1 -- 172.21.15.3:0/2644165838 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd5ac09e410 msgr2=0x7fd5ac123080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:27.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.269+0000 7fd5aad9d700 1 --2- 172.21.15.3:0/2644165838 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd5ac09e410 0x7fd5ac123080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:27.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.269+0000 7fd5a9d9b700 1 --2- 172.21.15.3:0/2644165838 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd5ac09e410 0x7fd5ac123080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.269+0000 7fd5a9d9b700 1 -- 172.21.15.3:0/2644165838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5ac09d6f0 msgr2=0x7fd5ac122600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:27.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.269+0000 7fd5a9d9b700 1 --2- 172.21.15.3:0/2644165838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5ac09d6f0 0x7fd5ac122600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.269+0000 7fd5a9d9b700 1 -- 172.21.15.3:0/2644165838 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd59c009550 con 0x7fd5ac09e000 2023-10-03T10:13:27.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.269+0000 7fd5aa59c700 1 --2- 172.21.15.3:0/2644165838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5ac09d6f0 0x7fd5ac122600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:27.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.269+0000 7fd5aad9d700 1 --2- 172.21.15.3:0/2644165838 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd5ac09e410 0x7fd5ac123080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:13:27.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.269+0000 7fd5a9d9b700 1 --2- 172.21.15.3:0/2644165838 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd5ac09e000 0x7fd5ac122b40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000e8d0 tx=0x7fd5a000ebe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:27.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.269+0000 7fd59affd700 1 -- 172.21.15.3:0/2644165838 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a000c690 con 0x7fd5ac09e000 2023-10-03T10:13:27.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.269+0000 7fd59affd700 1 -- 172.21.15.3:0/2644165838 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a000c7f0 con 0x7fd5ac09e000 2023-10-03T10:13:27.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.269+0000 7fd5b0fd5700 1 -- 172.21.15.3:0/2644165838 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5ac125810 con 0x7fd5ac09e000 2023-10-03T10:13:27.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.269+0000 7fd59affd700 1 -- 172.21.15.3:0/2644165838 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a000cb60 con 0x7fd5ac09e000 2023-10-03T10:13:27.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.269+0000 7fd5b0fd5700 1 -- 172.21.15.3:0/2644165838 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5ac126810 con 0x7fd5ac09e000 2023-10-03T10:13:27.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.270+0000 7fd59affd700 1 -- 172.21.15.3:0/2644165838 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd5a000ccc0 con 0x7fd5ac09e000 2023-10-03T10:13:27.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.270+0000 7fd59affd700 1 -- 172.21.15.3:0/2644165838 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd5a003f790 con 0x7fd5ac09e000 2023-10-03T10:13:27.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.270+0000 7fd5b0fd5700 1 -- 172.21.15.3:0/2644165838 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5ac11f030 con 0x7fd5ac09e000 2023-10-03T10:13:27.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.271+0000 7f5fae95b700 1 -- 172.21.15.3:0/2396562404 shutdown_connections 2023-10-03T10:13:27.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.271+0000 7f5fae95b700 1 --2- 172.21.15.3:0/2396562404 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fa8135c20 0x7f5fa8138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.271+0000 7f5fae95b700 1 --2- 172.21.15.3:0/2396562404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fa806f5d0 0x7f5fa812ffc0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.271+0000 7f5fae95b700 1 -- 172.21.15.3:0/2396562404 >> 172.21.15.3:0/2396562404 conn(0x7f5fa80a2aa0 msgr2=0x7f5fa80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:27.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.274+0000 7f5fae95b700 1 -- 172.21.15.3:0/2396562404 shutdown_connections 2023-10-03T10:13:27.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.274+0000 7f5fae95b700 1 -- 172.21.15.3:0/2396562404 wait complete. 2023-10-03T10:13:27.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.275+0000 7f5fae95b700 1 Processor -- start 2023-10-03T10:13:27.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.275+0000 7fd59affd700 1 -- 172.21.15.3:0/2644165838 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd5a0021070 con 0x7fd5ac09e000 2023-10-03T10:13:27.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.275+0000 7f5fae95b700 1 -- start start 2023-10-03T10:13:27.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.276+0000 7f5fae95b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fa806f5d0 0x7f5fa809c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:27.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.276+0000 7f5fae95b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fa812cf60 0x7f5fa809c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:27.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.276+0000 7f5fae95b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fa8135c20 0x7f5fa809cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:27.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.276+0000 7f5fae95b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa8129fa0 con 0x7f5fa812cf60 2023-10-03T10:13:27.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.276+0000 7f5fae95b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa809dc70 con 0x7f5fa806f5d0 2023-10-03T10:13:27.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.276+0000 7f5fae95b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa8131210 con 0x7f5fa8135c20 2023-10-03T10:13:27.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.276+0000 7f5fa7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fa806f5d0 0x7f5fa809c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:27.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.276+0000 7f5fa7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fa806f5d0 0x7f5fa809c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56858/0 (socket says 172.21.15.3:56858) 2023-10-03T10:13:27.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.276+0000 7f5fa7fff700 1 -- 172.21.15.3:0/361007940 learned_addr learned my addr 172.21.15.3:0/361007940 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:27.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.276+0000 7f5facef8700 1 --2- 172.21.15.3:0/361007940 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fa8135c20 0x7f5fa809cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:27.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.276+0000 7f5fa77fe700 1 --2- 172.21.15.3:0/361007940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fa812cf60 0x7f5fa809c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:27.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.277+0000 7f5fa7fff700 1 -- 172.21.15.3:0/361007940 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fa8135c20 msgr2=0x7f5fa809cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:27.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.277+0000 7f5fa7fff700 1 --2- 172.21.15.3:0/361007940 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fa8135c20 0x7f5fa809cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.277+0000 7f5fa7fff700 1 -- 172.21.15.3:0/361007940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fa812cf60 msgr2=0x7f5fa809c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:27.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.277+0000 7f5fa7fff700 1 --2- 172.21.15.3:0/361007940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fa812cf60 0x7f5fa809c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.277+0000 7f5fa7fff700 1 -- 172.21.15.3:0/361007940 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9c00b740 con 0x7f5fa806f5d0 2023-10-03T10:13:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.277+0000 7f5facef8700 1 --2- 172.21.15.3:0/361007940 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fa8135c20 0x7f5fa809cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.277+0000 7f5fa7fff700 1 --2- 172.21.15.3:0/361007940 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fa806f5d0 0x7f5fa809c2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800c6b0 tx=0x7f5f9800c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.277+0000 7f5fa4ff9700 1 -- 172.21.15.3:0/361007940 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f98011790 con 0x7f5fa806f5d0 2023-10-03T10:13:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.277+0000 7f5fa4ff9700 1 -- 172.21.15.3:0/361007940 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f980118f0 con 0x7f5fa806f5d0 2023-10-03T10:13:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.277+0000 7f5fa4ff9700 1 -- 172.21.15.3:0/361007940 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f98011c60 con 0x7f5fa806f5d0 2023-10-03T10:13:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.277+0000 7f5fae95b700 1 -- 172.21.15.3:0/361007940 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa81314d0 con 0x7f5fa806f5d0 2023-10-03T10:13:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.277+0000 7f5fae95b700 1 -- 172.21.15.3:0/361007940 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa8127af0 con 0x7f5fa806f5d0 2023-10-03T10:13:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.278+0000 7f5fae95b700 1 -- 172.21.15.3:0/361007940 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fa80a0a30 con 0x7f5fa806f5d0 2023-10-03T10:13:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.279+0000 7f5fa4ff9700 1 -- 172.21.15.3:0/361007940 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5f9801c460 con 0x7f5fa806f5d0 2023-10-03T10:13:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.279+0000 7f5fa4ff9700 1 -- 172.21.15.3:0/361007940 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5f9803f550 con 0x7f5fa806f5d0 2023-10-03T10:13:27.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.284+0000 7f5fa4ff9700 1 -- 172.21.15.3:0/361007940 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5f98011dc0 con 0x7f5fa806f5d0 2023-10-03T10:13:27.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.348+0000 7f79f8071700 1 -- 172.21.15.3:0/416139847 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f79f012db70 con 0x7f79f0130a70 2023-10-03T10:13:27.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.349+0000 7f79e67fc700 1 -- 172.21.15.3:0/416139847 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f79e0017020 con 0x7f79f0130a70 2023-10-03T10:13:27.349 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:27.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.351+0000 7f79f8071700 1 -- 172.21.15.3:0/416139847 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79f0130a70 msgr2=0x7f79f009cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:27.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.351+0000 7f79f8071700 1 --2- 172.21.15.3:0/416139847 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79f0130a70 0x7f79f009cc20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f79e000c4e0 tx=0x7f79e000c7f0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.351+0000 7f79f8071700 1 -- 172.21.15.3:0/416139847 shutdown_connections 2023-10-03T10:13:27.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.351+0000 7f79f8071700 1 --2- 172.21.15.3:0/416139847 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79f0133a80 0x7f79f009d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.351+0000 7f79f8071700 1 --2- 172.21.15.3:0/416139847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79f012ff20 0x7f79f009c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.351+0000 7f79f8071700 1 --2- 172.21.15.3:0/416139847 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79f0130a70 0x7f79f009cc20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.351+0000 7f79f8071700 1 -- 172.21.15.3:0/416139847 >> 172.21.15.3:0/416139847 conn(0x7f79f00a4210 msgr2=0x7f79f009e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:27.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.352+0000 7f79f8071700 1 -- 172.21.15.3:0/416139847 shutdown_connections 2023-10-03T10:13:27.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.352+0000 7f79f8071700 1 -- 172.21.15.3:0/416139847 wait complete. 2023-10-03T10:13:27.366 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:27.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.486+0000 7f64deb30700 1 -- 172.21.15.3:0/1346836666 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f64d8072210 con 0x7f64d8130670 2023-10-03T10:13:27.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.486+0000 7f64d57fa700 1 -- 172.21.15.3:0/1346836666 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f64cc031b20 con 0x7f64d8130670 2023-10-03T10:13:27.487 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:27.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.490+0000 7f64deb30700 1 -- 172.21.15.3:0/1346836666 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64d8130670 msgr2=0x7f64d809d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:27.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.490+0000 7f64deb30700 1 --2- 172.21.15.3:0/1346836666 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64d8130670 0x7f64d809d040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f64cc00c4e0 tx=0x7f64cc00c7f0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.490+0000 7f64deb30700 1 -- 172.21.15.3:0/1346836666 shutdown_connections 2023-10-03T10:13:27.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.490+0000 7f64deb30700 1 --2- 172.21.15.3:0/1346836666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64d8135e30 0x7f64d81c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.490+0000 7f64deb30700 1 --2- 172.21.15.3:0/1346836666 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64d8130670 0x7f64d809d040 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.490+0000 7f64deb30700 1 --2- 172.21.15.3:0/1346836666 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64d8133510 0x7f64d809d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.490+0000 7f64deb30700 1 -- 172.21.15.3:0/1346836666 >> 172.21.15.3:0/1346836666 conn(0x7f64d80a2aa0 msgr2=0x7f64d81312f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:27.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.490+0000 7f64deb30700 1 -- 172.21.15.3:0/1346836666 shutdown_connections 2023-10-03T10:13:27.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.490+0000 7f64deb30700 1 -- 172.21.15.3:0/1346836666 wait complete. 2023-10-03T10:13:27.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.493+0000 7f5fae95b700 1 -- 172.21.15.3:0/361007940 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5fa8074ea0 con 0x7f5fa806f5d0 2023-10-03T10:13:27.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.493+0000 7f5fa4ff9700 1 -- 172.21.15.3:0/361007940 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5f9801a360 con 0x7f5fa806f5d0 2023-10-03T10:13:27.493 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:27.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.495+0000 7f5fae95b700 1 -- 172.21.15.3:0/361007940 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fa806f5d0 msgr2=0x7f5fa809c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:27.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.495+0000 7f5fae95b700 1 --2- 172.21.15.3:0/361007940 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fa806f5d0 0x7f5fa809c2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800c6b0 tx=0x7f5f9800c9c0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.495+0000 7f5fae95b700 1 -- 172.21.15.3:0/361007940 shutdown_connections 2023-10-03T10:13:27.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.495+0000 7f5fae95b700 1 --2- 172.21.15.3:0/361007940 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fa8135c20 0x7f5fa809cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.495+0000 7f5fae95b700 1 --2- 172.21.15.3:0/361007940 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fa806f5d0 0x7f5fa809c2f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.495+0000 7f5fae95b700 1 --2- 172.21.15.3:0/361007940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fa812cf60 0x7f5fa809c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.495+0000 7f5fae95b700 1 -- 172.21.15.3:0/361007940 >> 172.21.15.3:0/361007940 conn(0x7f5fa80a2aa0 msgr2=0x7f5fa806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:27.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.495+0000 7f5fae95b700 1 -- 172.21.15.3:0/361007940 shutdown_connections 2023-10-03T10:13:27.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.495+0000 7f5fae95b700 1 -- 172.21.15.3:0/361007940 wait complete. 2023-10-03T10:13:27.506 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:27.510 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:27.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.524+0000 7fd5b0fd5700 1 -- 172.21.15.3:0/2644165838 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd5ac074ea0 con 0x7fd5ac09e000 2023-10-03T10:13:27.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.524+0000 7fd59affd700 1 -- 172.21.15.3:0/2644165838 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd5a00232f0 con 0x7fd5ac09e000 2023-10-03T10:13:27.525 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:27.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.526+0000 7fd5b0fd5700 1 -- 172.21.15.3:0/2644165838 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd5ac09e000 msgr2=0x7fd5ac122b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:27.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.526+0000 7fd5b0fd5700 1 --2- 172.21.15.3:0/2644165838 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd5ac09e000 0x7fd5ac122b40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000e8d0 tx=0x7fd5a000ebe0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.526+0000 7fd5b0fd5700 1 -- 172.21.15.3:0/2644165838 shutdown_connections 2023-10-03T10:13:27.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.526+0000 7fd5b0fd5700 1 --2- 172.21.15.3:0/2644165838 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd5ac09e410 0x7fd5ac123080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.526+0000 7fd5b0fd5700 1 --2- 172.21.15.3:0/2644165838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5ac09d6f0 0x7fd5ac122600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.526+0000 7fd5b0fd5700 1 --2- 172.21.15.3:0/2644165838 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd5ac09e000 0x7fd5ac122b40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:27.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.526+0000 7fd5b0fd5700 1 -- 172.21.15.3:0/2644165838 >> 172.21.15.3:0/2644165838 conn(0x7fd5ac066f50 msgr2=0x7fd5ac0a2580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:27.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.526+0000 7fd5b0fd5700 1 -- 172.21.15.3:0/2644165838 shutdown_connections 2023-10-03T10:13:27.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:27.526+0000 7fd5b0fd5700 1 -- 172.21.15.3:0/2644165838 wait complete. 2023-10-03T10:13:27.540 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:28.367 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:28.507 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:28.510 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.512+0000 7fe1cc5cc700 1 Processor -- start 2023-10-03T10:13:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.512+0000 7fe1cc5cc700 1 -- start start 2023-10-03T10:13:28.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.513+0000 7fe1cc5cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1c409f410 0x7fe1c4136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:28.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.513+0000 7fe1cc5cc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1c4136fe0 0x7fe1c41393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:28.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.513+0000 7fe1cc5cc700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c4070940 con 0x7fe1c409e930 2023-10-03T10:13:28.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.513+0000 7fe1cc5cc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c406fd60 con 0x7fe1c409f410 2023-10-03T10:13:28.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.513+0000 7fe1cc5cc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c406fa80 con 0x7fe1c4136fe0 2023-10-03T10:13:28.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.513+0000 7fe1cab69700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1c4136fe0 0x7fe1c41393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:28.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.513+0000 7fe1ca368700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe1c409e930 0x7fe1c409ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48792/0 (socket says 172.21.15.3:48792) 2023-10-03T10:13:28.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.513+0000 7fe1ca368700 1 -- 172.21.15.3:0/389693418 learned_addr learned my addr 172.21.15.3:0/389693418 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:28.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.513+0000 7fe1c9b67700 1 --2- 172.21.15.3:0/389693418 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1c409f410 0x7fe1c4136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.514+0000 7fe1c9b67700 1 -- 172.21.15.3:0/389693418 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1c4136fe0 msgr2=0x7fe1c41393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.514+0000 7fe1c9b67700 1 --2- 172.21.15.3:0/389693418 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1c4136fe0 0x7fe1c41393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.514+0000 7fe1c9b67700 1 -- 172.21.15.3:0/389693418 >> v1:172.21.15.3:6789/0 conn(0x7fe1c409e930 legacy=0x7fe1c409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.514+0000 7fe1c9366700 1 -- 172.21.15.3:0/389693418 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2201884662 0 0) 0x7fe1c4070940 con 0x7fe1c409e930 2023-10-03T10:13:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.514+0000 7fe1c9b67700 1 -- 172.21.15.3:0/389693418 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c409fb60 con 0x7fe1c409f410 2023-10-03T10:13:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.514+0000 7fe1cab69700 1 --2- 172.21.15.3:0/389693418 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1c4136fe0 0x7fe1c41393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.514+0000 7fe1c9b67700 1 --2- 172.21.15.3:0/389693418 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1c409f410 0x7fe1c4136aa0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4009870 tx=0x7fe1b4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f41112d349b5c7e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.514+0000 7fe1c9366700 1 -- 172.21.15.3:0/389693418 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4011830 con 0x7fe1c409f410 2023-10-03T10:13:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.514+0000 7fe1c9366700 1 -- 172.21.15.3:0/389693418 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b4011990 con 0x7fe1c409f410 2023-10-03T10:13:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.514+0000 7fe1c9366700 1 -- 172.21.15.3:0/389693418 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4011d20 con 0x7fe1c409f410 2023-10-03T10:13:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.515+0000 7fe1cc5cc700 1 -- 172.21.15.3:0/389693418 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1c409f410 msgr2=0x7fe1c4136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.515+0000 7fe1cc5cc700 1 --2- 172.21.15.3:0/389693418 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1c409f410 0x7fe1c4136aa0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4009870 tx=0x7fe1b4009b80 comp rx=0 tx=0).stop 2023-10-03T10:13:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.515+0000 7fe1cc5cc700 1 -- 172.21.15.3:0/389693418 shutdown_connections 2023-10-03T10:13:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.515+0000 7fe1cc5cc700 1 --2- 172.21.15.3:0/389693418 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1c4136fe0 0x7fe1c41393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.515+0000 7fe1cc5cc700 1 --2- 172.21.15.3:0/389693418 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1c409f410 0x7fe1c4136aa0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.515+0000 7fe1cc5cc700 1 -- 172.21.15.3:0/389693418 >> 172.21.15.3:0/389693418 conn(0x7fe1c40a4210 msgr2=0x7fe1c40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.515+0000 7fe1cc5cc700 1 -- 172.21.15.3:0/389693418 shutdown_connections 2023-10-03T10:13:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.515+0000 7fe1cc5cc700 1 -- 172.21.15.3:0/389693418 wait complete. 2023-10-03T10:13:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.516+0000 7fe1cc5cc700 1 Processor -- start 2023-10-03T10:13:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.516+0000 7fe1cc5cc700 1 -- start start 2023-10-03T10:13:28.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.517+0000 7fe1cc5cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1c409e930 0x7fe1c406c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:28.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.517+0000 7fe1cc5cc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1c409f410 0x7fe1c406c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:28.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.517+0000 7fe1cc5cc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1c4136fe0 0x7fe1c41c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:28.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.517+0000 7fe1cc5cc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c4121160 con 0x7fe1c409f410 2023-10-03T10:13:28.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.517+0000 7fe1cc5cc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c406d5c0 con 0x7fe1c409e930 2023-10-03T10:13:28.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.517+0000 7fe1cc5cc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c406cc40 con 0x7fe1c4136fe0 2023-10-03T10:13:28.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.517+0000 7fe1ca368700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1c409e930 0x7fe1c406c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:28.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.517+0000 7fe1cab69700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1c4136fe0 0x7fe1c41c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:28.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.517+0000 7fe1ca368700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1c409e930 0x7fe1c406c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56882/0 (socket says 172.21.15.3:56882) 2023-10-03T10:13:28.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.517+0000 7fe1cab69700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1c4136fe0 0x7fe1c41c8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53970/0 (socket says 172.21.15.3:53970) 2023-10-03T10:13:28.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.517+0000 7fe1ca368700 1 -- 172.21.15.3:0/2429967574 learned_addr learned my addr 172.21.15.3:0/2429967574 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:28.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.517+0000 7fe1c9b67700 1 --2- 172.21.15.3:0/2429967574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1c409f410 0x7fe1c406c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:28.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.517+0000 7fe1ca368700 1 -- 172.21.15.3:0/2429967574 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1c4136fe0 msgr2=0x7fe1c41c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:28.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.517+0000 7fe1ca368700 1 --2- 172.21.15.3:0/2429967574 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1c4136fe0 0x7fe1c41c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.517+0000 7fe1ca368700 1 -- 172.21.15.3:0/2429967574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1c409f410 msgr2=0x7fe1c406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:28.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.517+0000 7fe1ca368700 1 --2- 172.21.15.3:0/2429967574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1c409f410 0x7fe1c406c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.517+0000 7fe1ca368700 1 -- 172.21.15.3:0/2429967574 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1bc0094a0 con 0x7fe1c409e930 2023-10-03T10:13:28.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.518+0000 7fe1cab69700 1 --2- 172.21.15.3:0/2429967574 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1c4136fe0 0x7fe1c41c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:28.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.518+0000 7fe1ca368700 1 --2- 172.21.15.3:0/2429967574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1c409e930 0x7fe1c406c1c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000cc80 tx=0x7fe1c000cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:28.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.518+0000 7fe1baffd700 1 -- 172.21.15.3:0/2429967574 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c000ab00 con 0x7fe1c409e930 2023-10-03T10:13:28.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.518+0000 7fe1baffd700 1 -- 172.21.15.3:0/2429967574 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c000ac60 con 0x7fe1c409e930 2023-10-03T10:13:28.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.518+0000 7fe1baffd700 1 -- 172.21.15.3:0/2429967574 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c00070d0 con 0x7fe1c409e930 2023-10-03T10:13:28.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.518+0000 7fe1cc5cc700 1 -- 172.21.15.3:0/2429967574 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b40094a0 con 0x7fe1c409e930 2023-10-03T10:13:28.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.518+0000 7fe1cc5cc700 1 -- 172.21.15.3:0/2429967574 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c4121590 con 0x7fe1c409e930 2023-10-03T10:13:28.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.519+0000 7fe1baffd700 1 -- 172.21.15.3:0/2429967574 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe1c0007230 con 0x7fe1c409e930 2023-10-03T10:13:28.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.519+0000 7fe1baffd700 1 -- 172.21.15.3:0/2429967574 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe1c003f250 con 0x7fe1c409e930 2023-10-03T10:13:28.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.519+0000 7fe1cc5cc700 1 -- 172.21.15.3:0/2429967574 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c409fdb0 con 0x7fe1c409e930 2023-10-03T10:13:28.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.524+0000 7fe1baffd700 1 -- 172.21.15.3:0/2429967574 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe1c0018080 con 0x7fe1c409e930 2023-10-03T10:13:28.541 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:28.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.652+0000 7f532347f700 1 Processor -- start 2023-10-03T10:13:28.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.653+0000 7f532347f700 1 -- start start 2023-10-03T10:13:28.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.653+0000 7f532347f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f531c1337f0 0x7f531c138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:28.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.653+0000 7f532347f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f531c138dc0 0x7f531c13b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:28.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.653+0000 7f532347f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f531c06b990 con 0x7f531c133420 2023-10-03T10:13:28.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.653+0000 7f532347f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f531c09b0e0 con 0x7f531c1337f0 2023-10-03T10:13:28.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.653+0000 7f532347f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f531c133c20 con 0x7f531c138dc0 2023-10-03T10:13:28.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.653+0000 7f532121b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f531c133420 0x7f531c133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48798/0 (socket says 172.21.15.3:48798) 2023-10-03T10:13:28.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.653+0000 7f532121b700 1 -- 172.21.15.3:0/544845482 learned_addr learned my addr 172.21.15.3:0/544845482 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:28.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.653+0000 7f5321a1c700 1 --2- 172.21.15.3:0/544845482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f531c138dc0 0x7f531c13b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:28.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.653+0000 7f5320a1a700 1 --2- 172.21.15.3:0/544845482 >> v2:172.21.15.3:3300/0 conn(0x7f531c1337f0 0x7f531c138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:28.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.654+0000 7f5321a1c700 1 -- 172.21.15.3:0/544845482 >> v2:172.21.15.3:3300/0 conn(0x7f531c1337f0 msgr2=0x7f531c138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:28.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.654+0000 7f5321a1c700 1 --2- 172.21.15.3:0/544845482 >> v2:172.21.15.3:3300/0 conn(0x7f531c1337f0 0x7f531c138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.654+0000 7f5321a1c700 1 -- 172.21.15.3:0/544845482 >> v1:172.21.15.3:6789/0 conn(0x7f531c133420 legacy=0x7f531c133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:28.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.654+0000 7f5313fff700 1 -- 172.21.15.3:0/544845482 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3006361826 0 0) 0x7f531c06b990 con 0x7f531c133420 2023-10-03T10:13:28.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.654+0000 7f5321a1c700 1 -- 172.21.15.3:0/544845482 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531c0a0590 con 0x7f531c138dc0 2023-10-03T10:13:28.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.654+0000 7f5321a1c700 1 --2- 172.21.15.3:0/544845482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f531c138dc0 0x7f531c13b1a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5314009d90 tx=0x7f531400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e06d8dbeef372ce8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:28.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.654+0000 7f5313fff700 1 -- 172.21.15.3:0/544845482 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531400e070 con 0x7f531c138dc0 2023-10-03T10:13:28.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.654+0000 7f5313fff700 1 -- 172.21.15.3:0/544845482 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531400bb20 con 0x7f531c138dc0 2023-10-03T10:13:28.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.654+0000 7f5313fff700 1 -- 172.21.15.3:0/544845482 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5314012430 con 0x7f531c138dc0 2023-10-03T10:13:28.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.655+0000 7f532347f700 1 -- 172.21.15.3:0/544845482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f531c138dc0 msgr2=0x7f531c13b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:28.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.655+0000 7f532347f700 1 --2- 172.21.15.3:0/544845482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f531c138dc0 0x7f531c13b1a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5314009d90 tx=0x7f531400b330 comp rx=0 tx=0).stop 2023-10-03T10:13:28.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.655+0000 7f532347f700 1 -- 172.21.15.3:0/544845482 shutdown_connections 2023-10-03T10:13:28.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.655+0000 7f532347f700 1 --2- 172.21.15.3:0/544845482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f531c138dc0 0x7f531c13b1a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.655+0000 7f532347f700 1 --2- 172.21.15.3:0/544845482 >> v2:172.21.15.3:3300/0 conn(0x7f531c1337f0 0x7f531c138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.655+0000 7f532347f700 1 -- 172.21.15.3:0/544845482 >> 172.21.15.3:0/544845482 conn(0x7f531c0a4210 msgr2=0x7f531c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:28.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.655+0000 7f532347f700 1 -- 172.21.15.3:0/544845482 shutdown_connections 2023-10-03T10:13:28.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.655+0000 7f532347f700 1 -- 172.21.15.3:0/544845482 wait complete. 2023-10-03T10:13:28.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.656+0000 7f532347f700 1 Processor -- start 2023-10-03T10:13:28.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.656+0000 7f532347f700 1 -- start start 2023-10-03T10:13:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.657+0000 7f532347f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f531c133420 0x7f531c12ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.657+0000 7f532347f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f531c1337f0 0x7f531c12f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.657+0000 7f532347f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f531c138dc0 0x7f531c12f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.657+0000 7f532347f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c130740 con 0x7f531c1337f0 2023-10-03T10:13:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.657+0000 7f532347f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f531c130930 con 0x7f531c133420 2023-10-03T10:13:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.657+0000 7f532347f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f531c1302b0 con 0x7f531c138dc0 2023-10-03T10:13:28.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.657+0000 7f16bed9d700 1 Processor -- start 2023-10-03T10:13:28.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.657+0000 7f5321a1c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f531c138dc0 0x7f531c12f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:28.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.657+0000 7f5321a1c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f531c138dc0 0x7f531c12f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53992/0 (socket says 172.21.15.3:53992) 2023-10-03T10:13:28.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.657+0000 7f5321a1c700 1 -- 172.21.15.3:0/847490829 learned_addr learned my addr 172.21.15.3:0/847490829 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:28.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.657+0000 7f532121b700 1 --2- 172.21.15.3:0/847490829 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f531c133420 0x7f531c12ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:28.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.657+0000 7f5320a1a700 1 --2- 172.21.15.3:0/847490829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f531c1337f0 0x7f531c12f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:28.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.657+0000 7f5321a1c700 1 -- 172.21.15.3:0/847490829 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f531c133420 msgr2=0x7f531c12ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:28.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.657+0000 7f5321a1c700 1 --2- 172.21.15.3:0/847490829 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f531c133420 0x7f531c12ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.657+0000 7f5321a1c700 1 -- 172.21.15.3:0/847490829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f531c1337f0 msgr2=0x7f531c12f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:28.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.657+0000 7f5321a1c700 1 --2- 172.21.15.3:0/847490829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f531c1337f0 0x7f531c12f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.657+0000 7f5321a1c700 1 -- 172.21.15.3:0/847490829 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5314009a70 con 0x7f531c138dc0 2023-10-03T10:13:28.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.657+0000 7f5320a1a700 1 --2- 172.21.15.3:0/847490829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f531c1337f0 0x7f531c12f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:28.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.657+0000 7f16bed9d700 1 -- start start 2023-10-03T10:13:28.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.658+0000 7f5321a1c700 1 --2- 172.21.15.3:0/847490829 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f531c138dc0 0x7f531c12f8d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f531c133d60 tx=0x7f5314004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:28.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.658+0000 7f5311ffb700 1 -- 172.21.15.3:0/847490829 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5314012430 con 0x7f531c138dc0 2023-10-03T10:13:28.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.658+0000 7f5311ffb700 1 -- 172.21.15.3:0/847490829 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5314020070 con 0x7f531c138dc0 2023-10-03T10:13:28.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.658+0000 7f5311ffb700 1 -- 172.21.15.3:0/847490829 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5314004280 con 0x7f531c138dc0 2023-10-03T10:13:28.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.658+0000 7f16bed9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f16b809e1b0 0x7f16b813cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:28.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.658+0000 7f16bed9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f16b813d300 0x7f16b813f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:28.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.658+0000 7f16bed9d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16b809f0d0 con 0x7f16b809d6d0 2023-10-03T10:13:28.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.658+0000 7f16bed9d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b809e5a0 con 0x7f16b809e1b0 2023-10-03T10:13:28.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.658+0000 7f16bed9d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f16b809a2d0 con 0x7f16b813d300 2023-10-03T10:13:28.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.658+0000 7f16bcb39700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f16b809d6d0 0x7f16b809daa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48814/0 (socket says 172.21.15.3:48814) 2023-10-03T10:13:28.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.658+0000 7f16bcb39700 1 -- 172.21.15.3:0/854293457 learned_addr learned my addr 172.21.15.3:0/854293457 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:28.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.658+0000 7f16b7fff700 1 --2- 172.21.15.3:0/854293457 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f16b809e1b0 0x7f16b813cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:28.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.658+0000 7f532347f700 1 -- 172.21.15.3:0/847490829 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f531c09bc70 con 0x7f531c138dc0 2023-10-03T10:13:28.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.659+0000 7f532347f700 1 -- 172.21.15.3:0/847490829 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f531c09d840 con 0x7f531c138dc0 2023-10-03T10:13:28.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.659+0000 7f16b7fff700 1 -- 172.21.15.3:0/854293457 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f16b813d300 msgr2=0x7f16b813f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:28.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.659+0000 7f16b7fff700 1 --2- 172.21.15.3:0/854293457 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f16b813d300 0x7f16b813f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.658+0000 7f16b77fe700 1 -- 172.21.15.3:0/854293457 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2611076721 0 0) 0x7f16b809f0d0 con 0x7f16b809d6d0 2023-10-03T10:13:28.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.659+0000 7f16b7fff700 1 -- 172.21.15.3:0/854293457 >> v1:172.21.15.3:6789/0 conn(0x7f16b809d6d0 legacy=0x7f16b809daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:28.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.659+0000 7f16b7fff700 1 -- 172.21.15.3:0/854293457 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b809a570 con 0x7f16b809e1b0 2023-10-03T10:13:28.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.659+0000 7f16b7fff700 1 --2- 172.21.15.3:0/854293457 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f16b809e1b0 0x7f16b813cdc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009870 tx=0x7f16a4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26505931b0dec768 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:28.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.659+0000 7f5311ffb700 1 -- 172.21.15.3:0/847490829 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f53140043e0 con 0x7f531c138dc0 2023-10-03T10:13:28.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.660+0000 7f16b77fe700 1 -- 172.21.15.3:0/854293457 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a4011830 con 0x7f16b809e1b0 2023-10-03T10:13:28.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.660+0000 7f16b77fe700 1 -- 172.21.15.3:0/854293457 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a4011990 con 0x7f16b809e1b0 2023-10-03T10:13:28.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.660+0000 7f16b77fe700 1 -- 172.21.15.3:0/854293457 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a4011d20 con 0x7f16b809e1b0 2023-10-03T10:13:28.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.660+0000 7f5311ffb700 1 -- 172.21.15.3:0/847490829 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5314022070 con 0x7f531c138dc0 2023-10-03T10:13:28.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.660+0000 7f532347f700 1 -- 172.21.15.3:0/847490829 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f531c09d7e0 con 0x7f531c138dc0 2023-10-03T10:13:28.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.660+0000 7f16bed9d700 1 -- 172.21.15.3:0/854293457 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f16b809e1b0 msgr2=0x7f16b813cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:28.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.660+0000 7f16bed9d700 1 --2- 172.21.15.3:0/854293457 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f16b809e1b0 0x7f16b813cdc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009870 tx=0x7f16a4009b80 comp rx=0 tx=0).stop 2023-10-03T10:13:28.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.664+0000 7f16bed9d700 1 -- 172.21.15.3:0/854293457 shutdown_connections 2023-10-03T10:13:28.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.664+0000 7f16bed9d700 1 --2- 172.21.15.3:0/854293457 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f16b813d300 0x7f16b813f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.664+0000 7f16bed9d700 1 --2- 172.21.15.3:0/854293457 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f16b809e1b0 0x7f16b813cdc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.664+0000 7f16bed9d700 1 -- 172.21.15.3:0/854293457 >> 172.21.15.3:0/854293457 conn(0x7f16b8099c20 msgr2=0x7f16b80a2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:28.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.664+0000 7f16bed9d700 1 -- 172.21.15.3:0/854293457 shutdown_connections 2023-10-03T10:13:28.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.664+0000 7f16bed9d700 1 -- 172.21.15.3:0/854293457 wait complete. 2023-10-03T10:13:28.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.665+0000 7f16bed9d700 1 Processor -- start 2023-10-03T10:13:28.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.666+0000 7f5311ffb700 1 -- 172.21.15.3:0/847490829 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5314004690 con 0x7f531c138dc0 2023-10-03T10:13:28.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.665+0000 7f16bed9d700 1 -- start start 2023-10-03T10:13:28.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.666+0000 7f16bed9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f16b809d6d0 0x7f16b8124db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:28.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.666+0000 7f16bed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16b809e1b0 0x7f16b81252f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:28.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.666+0000 7f16bed9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f16b813d300 0x7f16b8125830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:28.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.666+0000 7f16bed9d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b8128600 con 0x7f16b809e1b0 2023-10-03T10:13:28.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.666+0000 7f16bed9d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b8128800 con 0x7f16b813d300 2023-10-03T10:13:28.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.666+0000 7f16bed9d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f16b813abd0 con 0x7f16b809d6d0 2023-10-03T10:13:28.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.666+0000 7f16bcb39700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f16b809d6d0 0x7f16b8124db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:28.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.666+0000 7f16bcb39700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f16b809d6d0 0x7f16b8124db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54012/0 (socket says 172.21.15.3:54012) 2023-10-03T10:13:28.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.666+0000 7f16bcb39700 1 -- 172.21.15.3:0/150299047 learned_addr learned my addr 172.21.15.3:0/150299047 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:28.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.666+0000 7f16b7fff700 1 --2- 172.21.15.3:0/150299047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16b809e1b0 0x7f16b81252f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:28.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.667+0000 7f16bd33a700 1 --2- 172.21.15.3:0/150299047 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f16b813d300 0x7f16b8125830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:28.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.667+0000 7f16bcb39700 1 -- 172.21.15.3:0/150299047 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f16b813d300 msgr2=0x7f16b8125830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:28.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.667+0000 7f16bcb39700 1 --2- 172.21.15.3:0/150299047 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f16b813d300 0x7f16b8125830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.667+0000 7f16bcb39700 1 -- 172.21.15.3:0/150299047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16b809e1b0 msgr2=0x7f16b81252f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:28.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.667+0000 7f16bcb39700 1 --2- 172.21.15.3:0/150299047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16b809e1b0 0x7f16b81252f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.667+0000 7f16bcb39700 1 -- 172.21.15.3:0/150299047 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a40094a0 con 0x7f16b809d6d0 2023-10-03T10:13:28.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.667+0000 7f16bcb39700 1 --2- 172.21.15.3:0/150299047 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f16b809d6d0 0x7f16b8124db0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f16ac0011e0 tx=0x7f16ac00ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:28.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.667+0000 7f16b57fa700 1 -- 172.21.15.3:0/150299047 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac004c80 con 0x7f16b809d6d0 2023-10-03T10:13:28.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.667+0000 7f16b57fa700 1 -- 172.21.15.3:0/150299047 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16ac004de0 con 0x7f16b809d6d0 2023-10-03T10:13:28.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.667+0000 7f16bed9d700 1 -- 172.21.15.3:0/150299047 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b809c6f0 con 0x7f16b809d6d0 2023-10-03T10:13:28.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.667+0000 7f16b57fa700 1 -- 172.21.15.3:0/150299047 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac007310 con 0x7f16b809d6d0 2023-10-03T10:13:28.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.667+0000 7f16bed9d700 1 -- 172.21.15.3:0/150299047 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b8128f90 con 0x7f16b809d6d0 2023-10-03T10:13:28.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.669+0000 7f16bed9d700 1 -- 172.21.15.3:0/150299047 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16b804f2a0 con 0x7f16b809d6d0 2023-10-03T10:13:28.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.670+0000 7f16b57fa700 1 -- 172.21.15.3:0/150299047 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f16ac007470 con 0x7f16b809d6d0 2023-10-03T10:13:28.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.670+0000 7f16b57fa700 1 -- 172.21.15.3:0/150299047 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f16ac03fa20 con 0x7f16b809d6d0 2023-10-03T10:13:28.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.678+0000 7f16b57fa700 1 -- 172.21.15.3:0/150299047 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f16ac01bd10 con 0x7f16b809d6d0 2023-10-03T10:13:28.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.694+0000 7f95d187e700 1 Processor -- start 2023-10-03T10:13:28.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.694+0000 7f95d187e700 1 -- start start 2023-10-03T10:13:28.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.695+0000 7f95d187e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f95cc09f410 0x7f95cc136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:28.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.695+0000 7f95d187e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f95cc136fe0 0x7f95cc1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:28.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.695+0000 7f95d187e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95cc070940 con 0x7f95cc09e930 2023-10-03T10:13:28.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.695+0000 7f95d187e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95cc06fd60 con 0x7f95cc09f410 2023-10-03T10:13:28.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.695+0000 7f95d187e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc06fa80 con 0x7f95cc136fe0 2023-10-03T10:13:28.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.695+0000 7f95cb7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f95cc136fe0 0x7f95cc1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:28.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.695+0000 7f95ca7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f95cc09f410 0x7f95cc136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:28.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.695+0000 7f95ca7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f95cc09f410 0x7f95cc136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53774/0 (socket says 172.21.15.3:53774) 2023-10-03T10:13:28.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.695+0000 7f95cb7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f95cc136fe0 0x7f95cc1393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56908/0 (socket says 172.21.15.3:56908) 2023-10-03T10:13:28.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.695+0000 7f95ca7fc700 1 -- 172.21.15.3:0/2954745355 learned_addr learned my addr 172.21.15.3:0/2954745355 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.696+0000 7f95ca7fc700 1 -- 172.21.15.3:0/2954745355 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f95cc136fe0 msgr2=0x7f95cc1393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.696+0000 7f95ca7fc700 1 --2- 172.21.15.3:0/2954745355 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f95cc136fe0 0x7f95cc1393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.696+0000 7f95ca7fc700 1 -- 172.21.15.3:0/2954745355 >> v1:172.21.15.3:6789/0 conn(0x7f95cc09e930 legacy=0x7f95cc09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.696+0000 7f95ca7fc700 1 -- 172.21.15.3:0/2954745355 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95cc09fb60 con 0x7f95cc09f410 2023-10-03T10:13:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.696+0000 7f95cb7fe700 1 --2- 172.21.15.3:0/2954745355 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f95cc136fe0 0x7f95cc1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.696+0000 7f95ca7fc700 1 --2- 172.21.15.3:0/2954745355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95cc09f410 0x7f95cc136aa0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f95c00098f0 tx=0x7f95c0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbdc4bd5887efa1a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.696+0000 7f95c9ffb700 1 -- 172.21.15.3:0/2954745355 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c0004230 con 0x7f95cc09f410 2023-10-03T10:13:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.696+0000 7f95c9ffb700 1 -- 172.21.15.3:0/2954745355 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c0004390 con 0x7f95cc09f410 2023-10-03T10:13:28.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.696+0000 7f95c9ffb700 1 -- 172.21.15.3:0/2954745355 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c00036a0 con 0x7f95cc09f410 2023-10-03T10:13:28.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.697+0000 7f95d187e700 1 -- 172.21.15.3:0/2954745355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95cc09f410 msgr2=0x7f95cc136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:28.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.697+0000 7f95d187e700 1 --2- 172.21.15.3:0/2954745355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95cc09f410 0x7f95cc136aa0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f95c00098f0 tx=0x7f95c0009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:28.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.697+0000 7f95d187e700 1 -- 172.21.15.3:0/2954745355 shutdown_connections 2023-10-03T10:13:28.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.697+0000 7f95d187e700 1 --2- 172.21.15.3:0/2954745355 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f95cc136fe0 0x7f95cc1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.697+0000 7f95d187e700 1 --2- 172.21.15.3:0/2954745355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95cc09f410 0x7f95cc136aa0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.697+0000 7f95d187e700 1 -- 172.21.15.3:0/2954745355 >> 172.21.15.3:0/2954745355 conn(0x7f95cc0a4210 msgr2=0x7f95cc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:28.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.697+0000 7f95d187e700 1 -- 172.21.15.3:0/2954745355 shutdown_connections 2023-10-03T10:13:28.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.697+0000 7f95d187e700 1 -- 172.21.15.3:0/2954745355 wait complete. 2023-10-03T10:13:28.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.698+0000 7f95d187e700 1 Processor -- start 2023-10-03T10:13:28.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.699+0000 7f95d187e700 1 -- start start 2023-10-03T10:13:28.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.699+0000 7f95d187e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f95cc09e930 0x7f95cc11c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:28.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.699+0000 7f95d187e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95cc09f410 0x7f95cc11cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:28.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.699+0000 7f95caffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f95cc09e930 0x7f95cc11c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.699+0000 7f95caffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f95cc09e930 0x7f95cc11c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56922/0 (socket says 172.21.15.3:56922) 2023-10-03T10:13:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.699+0000 7f95d187e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f95cc136fe0 0x7f95cc11d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.699+0000 7f95caffd700 1 -- 172.21.15.3:0/4035911356 learned_addr learned my addr 172.21.15.3:0/4035911356 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.699+0000 7f95ca7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95cc09f410 0x7f95cc11cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.699+0000 7f95d187e700 1 -- 172.21.15.3:0/4035911356 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc11fc90 con 0x7f95cc09f410 2023-10-03T10:13:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.699+0000 7f95d187e700 1 -- 172.21.15.3:0/4035911356 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc11e350 con 0x7f95cc09e930 2023-10-03T10:13:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.699+0000 7f95d187e700 1 -- 172.21.15.3:0/4035911356 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc0a2f40 con 0x7f95cc136fe0 2023-10-03T10:13:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.700+0000 7f95cb7fe700 1 --2- 172.21.15.3:0/4035911356 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f95cc136fe0 0x7f95cc11d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.700+0000 7f95caffd700 1 -- 172.21.15.3:0/4035911356 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f95cc136fe0 msgr2=0x7f95cc11d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.700+0000 7f95caffd700 1 --2- 172.21.15.3:0/4035911356 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f95cc136fe0 0x7f95cc11d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.700+0000 7f95caffd700 1 -- 172.21.15.3:0/4035911356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95cc09f410 msgr2=0x7f95cc11cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.700+0000 7f95caffd700 1 --2- 172.21.15.3:0/4035911356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95cc09f410 0x7f95cc11cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.700+0000 7f95caffd700 1 -- 172.21.15.3:0/4035911356 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c4009a70 con 0x7f95cc09e930 2023-10-03T10:13:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.700+0000 7f95ca7fc700 1 --2- 172.21.15.3:0/4035911356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95cc09f410 0x7f95cc11cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.700+0000 7f95caffd700 1 --2- 172.21.15.3:0/4035911356 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f95cc09e930 0x7f95cc11c680 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00c700 tx=0x7f95bc00ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.700+0000 7f95b3fff700 1 -- 172.21.15.3:0/4035911356 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc005480 con 0x7f95cc09e930 2023-10-03T10:13:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.700+0000 7f95d187e700 1 -- 172.21.15.3:0/4035911356 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c0009550 con 0x7f95cc09e930 2023-10-03T10:13:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.700+0000 7f95d187e700 1 -- 172.21.15.3:0/4035911356 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95cc11fe70 con 0x7f95cc09e930 2023-10-03T10:13:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.700+0000 7f95b3fff700 1 -- 172.21.15.3:0/4035911356 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95bc003680 con 0x7f95cc09e930 2023-10-03T10:13:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.700+0000 7f95b3fff700 1 -- 172.21.15.3:0/4035911356 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc003a30 con 0x7f95cc09e930 2023-10-03T10:13:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.701+0000 7f95b3fff700 1 -- 172.21.15.3:0/4035911356 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f95bc017070 con 0x7f95cc09e930 2023-10-03T10:13:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.701+0000 7f95b3fff700 1 -- 172.21.15.3:0/4035911356 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f95bc042fc0 con 0x7f95cc09e930 2023-10-03T10:13:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.702+0000 7f95d187e700 1 -- 172.21.15.3:0/4035911356 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95cc11fac0 con 0x7f95cc09e930 2023-10-03T10:13:28.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.711+0000 7f95b3fff700 1 -- 172.21.15.3:0/4035911356 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f95bc017b10 con 0x7f95cc09e930 2023-10-03T10:13:28.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.748+0000 7fe1cc5cc700 1 -- 172.21.15.3:0/2429967574 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe1c40a3d20 con 0x7fe1c409e930 2023-10-03T10:13:28.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.748+0000 7fe1baffd700 1 -- 172.21.15.3:0/2429967574 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe1c001b360 con 0x7fe1c409e930 2023-10-03T10:13:28.749 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:28.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.751+0000 7fe1cc5cc700 1 -- 172.21.15.3:0/2429967574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1c409e930 msgr2=0x7fe1c406c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:28.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.751+0000 7fe1cc5cc700 1 --2- 172.21.15.3:0/2429967574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1c409e930 0x7fe1c406c1c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000cc80 tx=0x7fe1c000cf90 comp rx=0 tx=0).stop 2023-10-03T10:13:28.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.751+0000 7fe1cc5cc700 1 -- 172.21.15.3:0/2429967574 shutdown_connections 2023-10-03T10:13:28.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.751+0000 7fe1cc5cc700 1 --2- 172.21.15.3:0/2429967574 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1c4136fe0 0x7fe1c41c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.751+0000 7fe1cc5cc700 1 --2- 172.21.15.3:0/2429967574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1c409e930 0x7fe1c406c1c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.751+0000 7fe1cc5cc700 1 --2- 172.21.15.3:0/2429967574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1c409f410 0x7fe1c406c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.751+0000 7fe1cc5cc700 1 -- 172.21.15.3:0/2429967574 >> 172.21.15.3:0/2429967574 conn(0x7fe1c40a4210 msgr2=0x7fe1c4135c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:28.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.751+0000 7fe1cc5cc700 1 -- 172.21.15.3:0/2429967574 shutdown_connections 2023-10-03T10:13:28.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.751+0000 7fe1cc5cc700 1 -- 172.21.15.3:0/2429967574 wait complete. 2023-10-03T10:13:28.766 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:28.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.882+0000 7f532347f700 1 -- 172.21.15.3:0/847490829 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f531c074ea0 con 0x7f531c138dc0 2023-10-03T10:13:28.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.883+0000 7f5311ffb700 1 -- 172.21.15.3:0/847490829 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f531401a380 con 0x7f531c138dc0 2023-10-03T10:13:28.883 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:28.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.886+0000 7f532347f700 1 -- 172.21.15.3:0/847490829 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f531c138dc0 msgr2=0x7f531c12f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:28.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.886+0000 7f532347f700 1 --2- 172.21.15.3:0/847490829 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f531c138dc0 0x7f531c12f8d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f531c133d60 tx=0x7f5314004ad0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.886+0000 7f532347f700 1 -- 172.21.15.3:0/847490829 shutdown_connections 2023-10-03T10:13:28.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.886+0000 7f532347f700 1 --2- 172.21.15.3:0/847490829 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f531c138dc0 0x7f531c12f8d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.886+0000 7f532347f700 1 --2- 172.21.15.3:0/847490829 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f531c133420 0x7f531c12ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.886+0000 7f532347f700 1 --2- 172.21.15.3:0/847490829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f531c1337f0 0x7f531c12f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.886+0000 7f532347f700 1 -- 172.21.15.3:0/847490829 >> 172.21.15.3:0/847490829 conn(0x7f531c0a4210 msgr2=0x7f531c0a0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:28.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.886+0000 7f532347f700 1 -- 172.21.15.3:0/847490829 shutdown_connections 2023-10-03T10:13:28.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.886+0000 7f532347f700 1 -- 172.21.15.3:0/847490829 wait complete. 2023-10-03T10:13:28.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.895+0000 7f16bed9d700 1 -- 172.21.15.3:0/150299047 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f16b806c230 con 0x7f16b809d6d0 2023-10-03T10:13:28.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.895+0000 7f16b57fa700 1 -- 172.21.15.3:0/150299047 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f16ac011360 con 0x7f16b809d6d0 2023-10-03T10:13:28.896 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:28.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.898+0000 7f16bed9d700 1 -- 172.21.15.3:0/150299047 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f16b809d6d0 msgr2=0x7f16b8124db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:28.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.898+0000 7f16bed9d700 1 --2- 172.21.15.3:0/150299047 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f16b809d6d0 0x7f16b8124db0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f16ac0011e0 tx=0x7f16ac00ca30 comp rx=0 tx=0).stop 2023-10-03T10:13:28.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.898+0000 7f16bed9d700 1 -- 172.21.15.3:0/150299047 shutdown_connections 2023-10-03T10:13:28.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.898+0000 7f16bed9d700 1 --2- 172.21.15.3:0/150299047 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f16b813d300 0x7f16b8125830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.898+0000 7f16bed9d700 1 --2- 172.21.15.3:0/150299047 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f16b809d6d0 0x7f16b8124db0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.898+0000 7f16bed9d700 1 --2- 172.21.15.3:0/150299047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16b809e1b0 0x7f16b81252f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.898+0000 7f16bed9d700 1 -- 172.21.15.3:0/150299047 >> 172.21.15.3:0/150299047 conn(0x7f16b8099c20 msgr2=0x7f16b813ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:28.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.898+0000 7f16bed9d700 1 -- 172.21.15.3:0/150299047 shutdown_connections 2023-10-03T10:13:28.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.898+0000 7f16bed9d700 1 -- 172.21.15.3:0/150299047 wait complete. 2023-10-03T10:13:28.906 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:28.912 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:28.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.933+0000 7f95d187e700 1 -- 172.21.15.3:0/4035911356 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f95cc074ea0 con 0x7f95cc09e930 2023-10-03T10:13:28.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.934+0000 7f95b3fff700 1 -- 172.21.15.3:0/4035911356 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f95bc0193d0 con 0x7f95cc09e930 2023-10-03T10:13:28.934 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:28.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.936+0000 7f95d187e700 1 -- 172.21.15.3:0/4035911356 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f95cc09e930 msgr2=0x7f95cc11c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:28.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.936+0000 7f95d187e700 1 --2- 172.21.15.3:0/4035911356 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f95cc09e930 0x7f95cc11c680 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00c700 tx=0x7f95bc00ca10 comp rx=0 tx=0).stop 2023-10-03T10:13:28.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.936+0000 7f95d187e700 1 -- 172.21.15.3:0/4035911356 shutdown_connections 2023-10-03T10:13:28.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.936+0000 7f95d187e700 1 --2- 172.21.15.3:0/4035911356 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f95cc136fe0 0x7f95cc11d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.936+0000 7f95d187e700 1 --2- 172.21.15.3:0/4035911356 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f95cc09e930 0x7f95cc11c680 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.936+0000 7f95d187e700 1 --2- 172.21.15.3:0/4035911356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95cc09f410 0x7f95cc11cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:28.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.936+0000 7f95d187e700 1 -- 172.21.15.3:0/4035911356 >> 172.21.15.3:0/4035911356 conn(0x7f95cc0a4210 msgr2=0x7f95cc0a27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:28.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.936+0000 7f95d187e700 1 -- 172.21.15.3:0/4035911356 shutdown_connections 2023-10-03T10:13:28.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:28.936+0000 7f95d187e700 1 -- 172.21.15.3:0/4035911356 wait complete. 2023-10-03T10:13:28.950 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:29.767 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:29.907 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:29.913 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:29.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.913+0000 7f57cc130700 1 Processor -- start 2023-10-03T10:13:29.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.913+0000 7f57cc130700 1 -- start start 2023-10-03T10:13:29.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.914+0000 7f57cc130700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57c406a770 0x7f57c406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:29.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.914+0000 7f57cc130700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f57c406b080 0x7f57c4131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:29.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.914+0000 7f57cc130700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c409ece0 con 0x7f57c4131b40 2023-10-03T10:13:29.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.914+0000 7f57cc130700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c409e760 con 0x7f57c406b080 2023-10-03T10:13:29.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.914+0000 7f57cc130700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f57c409e040 con 0x7f57c406a770 2023-10-03T10:13:29.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.914+0000 7f57c9ecc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57c406a770 0x7f57c406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:29.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.914+0000 7f57c9ecc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57c406a770 0x7f57c406ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54024/0 (socket says 172.21.15.3:54024) 2023-10-03T10:13:29.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.914+0000 7f57c9ecc700 1 -- 172.21.15.3:0/1809719975 learned_addr learned my addr 172.21.15.3:0/1809719975 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:29.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.914+0000 7f57c96cb700 1 --2- 172.21.15.3:0/1809719975 >> v2:172.21.15.3:3300/0 conn(0x7f57c406b080 0x7f57c4131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:29.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.915+0000 7f57c9ecc700 1 -- 172.21.15.3:0/1809719975 >> v2:172.21.15.3:3300/0 conn(0x7f57c406b080 msgr2=0x7f57c4131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:29.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.915+0000 7f57c9ecc700 1 --2- 172.21.15.3:0/1809719975 >> v2:172.21.15.3:3300/0 conn(0x7f57c406b080 0x7f57c4131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:29.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.915+0000 7f57c9ecc700 1 -- 172.21.15.3:0/1809719975 >> v1:172.21.15.3:6789/0 conn(0x7f57c4131b40 legacy=0x7f57c4138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:29.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.915+0000 7f57c9ecc700 1 -- 172.21.15.3:0/1809719975 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c409f630 con 0x7f57c406a770 2023-10-03T10:13:29.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.915+0000 7f57c8eca700 1 -- 172.21.15.3:0/1809719975 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2951722357 0 0) 0x7f57c409ece0 con 0x7f57c4131b40 2023-10-03T10:13:29.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.915+0000 7f57c9ecc700 1 --2- 172.21.15.3:0/1809719975 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57c406a770 0x7f57c406ab40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f57c00098f0 tx=0x7f57c0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eee21542be3ef3f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:29.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.915+0000 7f57c8eca700 1 -- 172.21.15.3:0/1809719975 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c0011830 con 0x7f57c406a770 2023-10-03T10:13:29.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.915+0000 7f57c8eca700 1 -- 172.21.15.3:0/1809719975 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c0011990 con 0x7f57c406a770 2023-10-03T10:13:29.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.915+0000 7f57c8eca700 1 -- 172.21.15.3:0/1809719975 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c0011d20 con 0x7f57c406a770 2023-10-03T10:13:29.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.916+0000 7f57cc130700 1 -- 172.21.15.3:0/1809719975 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57c406a770 msgr2=0x7f57c406ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:29.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.916+0000 7f57cc130700 1 --2- 172.21.15.3:0/1809719975 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57c406a770 0x7f57c406ab40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f57c00098f0 tx=0x7f57c0009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:29.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.917+0000 7f57cc130700 1 -- 172.21.15.3:0/1809719975 shutdown_connections 2023-10-03T10:13:29.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.917+0000 7f57cc130700 1 --2- 172.21.15.3:0/1809719975 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57c406a770 0x7f57c406ab40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:29.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.917+0000 7f57cc130700 1 --2- 172.21.15.3:0/1809719975 >> v2:172.21.15.3:3300/0 conn(0x7f57c406b080 0x7f57c4131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:29.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.917+0000 7f57cc130700 1 -- 172.21.15.3:0/1809719975 >> 172.21.15.3:0/1809719975 conn(0x7f57c4066f50 msgr2=0x7f57c409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:29.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.917+0000 7f57cc130700 1 -- 172.21.15.3:0/1809719975 shutdown_connections 2023-10-03T10:13:29.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.917+0000 7f57cc130700 1 -- 172.21.15.3:0/1809719975 wait complete. 2023-10-03T10:13:29.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.918+0000 7f57cc130700 1 Processor -- start 2023-10-03T10:13:29.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.918+0000 7f57cc130700 1 -- start start 2023-10-03T10:13:29.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.919+0000 7f57cc130700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57c406a770 0x7f57c41c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:29.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.919+0000 7f57cc130700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57c406b080 0x7f57c41c6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:29.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.919+0000 7f57cc130700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57c4131b40 0x7f57c41c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:29.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.919+0000 7f57cc130700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c41316c0 con 0x7f57c406a770 2023-10-03T10:13:29.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.919+0000 7f57cc130700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c4122ef0 con 0x7f57c4131b40 2023-10-03T10:13:29.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.919+0000 7f57cc130700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f57c4122280 con 0x7f57c406b080 2023-10-03T10:13:29.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.919+0000 7f57c9ecc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57c406a770 0x7f57c41c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:29.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.919+0000 7f57c9ecc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57c406a770 0x7f57c41c5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53806/0 (socket says 172.21.15.3:53806) 2023-10-03T10:13:29.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.919+0000 7f57c9ecc700 1 -- 172.21.15.3:0/508114511 learned_addr learned my addr 172.21.15.3:0/508114511 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:29.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.919+0000 7f57ca6cd700 1 --2- 172.21.15.3:0/508114511 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57c4131b40 0x7f57c41c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:29.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.919+0000 7f57c96cb700 1 --2- 172.21.15.3:0/508114511 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57c406b080 0x7f57c41c6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:29.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.919+0000 7f57c9ecc700 1 -- 172.21.15.3:0/508114511 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57c406b080 msgr2=0x7f57c41c6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:29.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.919+0000 7f57c9ecc700 1 --2- 172.21.15.3:0/508114511 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57c406b080 0x7f57c41c6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:29.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.919+0000 7f57c9ecc700 1 -- 172.21.15.3:0/508114511 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57c4131b40 msgr2=0x7f57c41c68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:29.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.919+0000 7f57c9ecc700 1 --2- 172.21.15.3:0/508114511 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57c4131b40 0x7f57c41c68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:29.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.919+0000 7f57c9ecc700 1 -- 172.21.15.3:0/508114511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c0009550 con 0x7f57c406a770 2023-10-03T10:13:29.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.920+0000 7f57c9ecc700 1 --2- 172.21.15.3:0/508114511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57c406a770 0x7f57c41c5e50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f57c0009c40 tx=0x7f57c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:29.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.920+0000 7f57ba7fc700 1 -- 172.21.15.3:0/508114511 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c00043c0 con 0x7f57c406a770 2023-10-03T10:13:29.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.920+0000 7f57ba7fc700 1 -- 172.21.15.3:0/508114511 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c0023070 con 0x7f57c406a770 2023-10-03T10:13:29.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.920+0000 7f57ba7fc700 1 -- 172.21.15.3:0/508114511 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c001f840 con 0x7f57c406a770 2023-10-03T10:13:29.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.920+0000 7f57cc130700 1 -- 172.21.15.3:0/508114511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c4071e60 con 0x7f57c406a770 2023-10-03T10:13:29.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.920+0000 7f57cc130700 1 -- 172.21.15.3:0/508114511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c4123eb0 con 0x7f57c406a770 2023-10-03T10:13:29.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.921+0000 7f57ba7fc700 1 -- 172.21.15.3:0/508114511 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f57c0025030 con 0x7f57c406a770 2023-10-03T10:13:29.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.921+0000 7f57ba7fc700 1 -- 172.21.15.3:0/508114511 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f57c0045020 con 0x7f57c406a770 2023-10-03T10:13:29.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.921+0000 7f57cc130700 1 -- 172.21.15.3:0/508114511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c4074ea0 con 0x7f57c406a770 2023-10-03T10:13:29.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:29.927+0000 7f57ba7fc700 1 -- 172.21.15.3:0/508114511 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f57c0015080 con 0x7f57c406a770 2023-10-03T10:13:29.951 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:30.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.057+0000 7f8c4fe3f700 1 Processor -- start 2023-10-03T10:13:30.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.057+0000 7f8c4fe3f700 1 -- start start 2023-10-03T10:13:30.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.058+0000 7fc6ad3de700 1 Processor -- start 2023-10-03T10:13:30.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.058+0000 7f8c4fe3f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8c48136830 0x7f8c48138c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:30.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.058+0000 7f8c4fe3f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c48139150 0x7f8c4813b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:30.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.058+0000 7fc6ad3de700 1 -- start start 2023-10-03T10:13:30.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.058+0000 7f8c4fe3f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c4806f720 con 0x7f8c481338c0 2023-10-03T10:13:30.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.058+0000 7f8c4fe3f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c48070c80 con 0x7f8c48136830 2023-10-03T10:13:30.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.058+0000 7f8c4fe3f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4806fce0 con 0x7f8c48139150 2023-10-03T10:13:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.058+0000 7f8c4d3da700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8c48136830 0x7f8c48138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.058+0000 7f8c4d3da700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8c48136830 0x7f8c48138c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53822/0 (socket says 172.21.15.3:53822) 2023-10-03T10:13:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.058+0000 7f8c4d3da700 1 -- 172.21.15.3:0/3792143974 learned_addr learned my addr 172.21.15.3:0/3792143974 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.058+0000 7f8c4e3dc700 1 --2- 172.21.15.3:0/3792143974 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c48139150 0x7f8c4813b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.059+0000 7fc6ad3de700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6a812cf60 0x7fc6a806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.059+0000 7f8c4d3da700 1 -- 172.21.15.3:0/3792143974 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c48139150 msgr2=0x7f8c4813b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.059+0000 7f8c4d3da700 1 --2- 172.21.15.3:0/3792143974 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c48139150 0x7f8c4813b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.059+0000 7f8c4d3da700 1 -- 172.21.15.3:0/3792143974 >> v1:172.21.15.3:6789/0 conn(0x7f8c481338c0 legacy=0x7f8c48133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:30.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.059+0000 7f8c4d3da700 1 -- 172.21.15.3:0/3792143974 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c481336a0 con 0x7f8c48136830 2023-10-03T10:13:30.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.059+0000 7fc6ad3de700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6a812f8b0 0x7fc6a812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:30.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.059+0000 7fc6ad3de700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6a812cd90 con 0x7fc6a8130260 2023-10-03T10:13:30.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.059+0000 7fc6ad3de700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a812d330 con 0x7fc6a812cf60 2023-10-03T10:13:30.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.059+0000 7fc6ad3de700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a81306d0 con 0x7fc6a812f8b0 2023-10-03T10:13:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.059+0000 7fc6a6ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6a812cf60 0x7fc6a806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.059+0000 7fc6a6ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6a812cf60 0x7fc6a806f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56940/0 (socket says 172.21.15.3:56940) 2023-10-03T10:13:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.059+0000 7fc6a6ffd700 1 -- 172.21.15.3:0/1759496193 learned_addr learned my addr 172.21.15.3:0/1759496193 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.059+0000 7fc6a67fc700 1 --2- 172.21.15.3:0/1759496193 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6a812f8b0 0x7fc6a812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.060+0000 7f8c4d3da700 1 --2- 172.21.15.3:0/3792143974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c48136830 0x7f8c48138c10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8c380098f0 tx=0x7f8c38009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9351b37b683c2a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.060+0000 7fc6a6ffd700 1 -- 172.21.15.3:0/1759496193 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6a812f8b0 msgr2=0x7fc6a812fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:30.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.060+0000 7fc6a6ffd700 1 --2- 172.21.15.3:0/1759496193 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6a812f8b0 0x7fc6a812fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.060+0000 7fc6a6ffd700 1 -- 172.21.15.3:0/1759496193 >> v1:172.21.15.3:6789/0 conn(0x7fc6a8130260 legacy=0x7fc6a8136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:30.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.060+0000 7fc6a6ffd700 1 -- 172.21.15.3:0/1759496193 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a8071780 con 0x7fc6a812cf60 2023-10-03T10:13:30.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.060+0000 7fc6a6ffd700 1 --2- 172.21.15.3:0/1759496193 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6a812cf60 0x7fc6a806f5d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc698013210 tx=0x7fc698013520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af95148c8a9e9768 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:30.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.060+0000 7f8c4cbd9700 1 -- 172.21.15.3:0/3792143974 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38011830 con 0x7f8c48136830 2023-10-03T10:13:30.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.060+0000 7fc6a5ffb700 1 -- 172.21.15.3:0/1759496193 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc698016070 con 0x7fc6a812cf60 2023-10-03T10:13:30.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.060+0000 7fc6a5ffb700 1 -- 172.21.15.3:0/1759496193 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc698013db0 con 0x7fc6a812cf60 2023-10-03T10:13:30.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.060+0000 7fc6a5ffb700 1 -- 172.21.15.3:0/1759496193 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6980112f0 con 0x7fc6a812cf60 2023-10-03T10:13:30.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.060+0000 7f8c4cbd9700 1 -- 172.21.15.3:0/3792143974 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c38011990 con 0x7f8c48136830 2023-10-03T10:13:30.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.060+0000 7f8c4cbd9700 1 -- 172.21.15.3:0/3792143974 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38011d20 con 0x7f8c48136830 2023-10-03T10:13:30.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.061+0000 7f8c4fe3f700 1 -- 172.21.15.3:0/3792143974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c48136830 msgr2=0x7f8c48138c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:30.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.061+0000 7f8c4fe3f700 1 --2- 172.21.15.3:0/3792143974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c48136830 0x7f8c48138c10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8c380098f0 tx=0x7f8c38009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:30.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.061+0000 7fc6ad3de700 1 -- 172.21.15.3:0/1759496193 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6a812cf60 msgr2=0x7fc6a806f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:30.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.061+0000 7fc6ad3de700 1 --2- 172.21.15.3:0/1759496193 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6a812cf60 0x7fc6a806f5d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc698013210 tx=0x7fc698013520 comp rx=0 tx=0).stop 2023-10-03T10:13:30.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.061+0000 7fc6ad3de700 1 -- 172.21.15.3:0/1759496193 shutdown_connections 2023-10-03T10:13:30.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.061+0000 7fc6ad3de700 1 --2- 172.21.15.3:0/1759496193 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6a812cf60 0x7fc6a806f5d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.061+0000 7fc6ad3de700 1 --2- 172.21.15.3:0/1759496193 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6a812f8b0 0x7fc6a812fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.061+0000 7f8c4fe3f700 1 -- 172.21.15.3:0/3792143974 shutdown_connections 2023-10-03T10:13:30.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.061+0000 7f8c4fe3f700 1 --2- 172.21.15.3:0/3792143974 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c48139150 0x7f8c4813b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.061+0000 7f8c4fe3f700 1 --2- 172.21.15.3:0/3792143974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c48136830 0x7f8c48138c10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.061+0000 7f8c4fe3f700 1 -- 172.21.15.3:0/3792143974 >> 172.21.15.3:0/3792143974 conn(0x7f8c480a2aa0 msgr2=0x7f8c480a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:30.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.061+0000 7fc6ad3de700 1 -- 172.21.15.3:0/1759496193 >> 172.21.15.3:0/1759496193 conn(0x7fc6a80a2aa0 msgr2=0x7fc6a80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:30.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.061+0000 7fc6ad3de700 1 -- 172.21.15.3:0/1759496193 shutdown_connections 2023-10-03T10:13:30.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.061+0000 7fc6ad3de700 1 -- 172.21.15.3:0/1759496193 wait complete. 2023-10-03T10:13:30.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.061+0000 7f8c4fe3f700 1 -- 172.21.15.3:0/3792143974 shutdown_connections 2023-10-03T10:13:30.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.061+0000 7f8c4fe3f700 1 -- 172.21.15.3:0/3792143974 wait complete. 2023-10-03T10:13:30.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.062+0000 7f8c4fe3f700 1 Processor -- start 2023-10-03T10:13:30.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.062+0000 7f8c4fe3f700 1 -- start start 2023-10-03T10:13:30.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.063+0000 7fc6ad3de700 1 Processor -- start 2023-10-03T10:13:30.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.063+0000 7f8c4fe3f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c481338c0 0x7f8c481c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:30.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.063+0000 7f8c4fe3f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c48136830 0x7f8c481c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:30.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.063+0000 7f8c4fe3f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c48139150 0x7f8c48119ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:30.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.063+0000 7f8c4fe3f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4811d3e0 con 0x7f8c48139150 2023-10-03T10:13:30.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.063+0000 7f8c4fe3f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4811dbc0 con 0x7f8c481338c0 2023-10-03T10:13:30.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.063+0000 7f8c4fe3f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4812fd30 con 0x7f8c48136830 2023-10-03T10:13:30.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.063+0000 7f8c4dbdb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c481338c0 0x7f8c481c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:30.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.063+0000 7f8c4dbdb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c481338c0 0x7f8c481c85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56948/0 (socket says 172.21.15.3:56948) 2023-10-03T10:13:30.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.063+0000 7f8c4dbdb700 1 -- 172.21.15.3:0/3711217868 learned_addr learned my addr 172.21.15.3:0/3711217868 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:30.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.063+0000 7f8c4e3dc700 1 --2- 172.21.15.3:0/3711217868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c48139150 0x7f8c48119ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:30.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.063+0000 7f8c4dbdb700 1 -- 172.21.15.3:0/3711217868 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c48136830 msgr2=0x7f8c481c8b30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:13:30.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.063+0000 7f8c4dbdb700 1 --2- 172.21.15.3:0/3711217868 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c48136830 0x7f8c481c8b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.063+0000 7f8c4dbdb700 1 -- 172.21.15.3:0/3711217868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c48139150 msgr2=0x7f8c48119ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:30.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.063+0000 7f8c4dbdb700 1 --2- 172.21.15.3:0/3711217868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c48139150 0x7f8c48119ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.063+0000 7f8c4dbdb700 1 -- 172.21.15.3:0/3711217868 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c38009550 con 0x7f8c481338c0 2023-10-03T10:13:30.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.064+0000 7f8c4dbdb700 1 --2- 172.21.15.3:0/3711217868 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c481338c0 0x7f8c481c85f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8c4400c4e0 tx=0x7f8c4400c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:30.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.063+0000 7fc6ad3de700 1 -- start start 2023-10-03T10:13:30.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.063+0000 7fc6ad3de700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6a812cf60 0x7fc6a809bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:30.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.064+0000 7fc6ad3de700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6a812f8b0 0x7fc6a809c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:30.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.064+0000 7fc6ad3de700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a8130260 0x7fc6a81c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:30.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.064+0000 7fc6ad3de700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a809ebe0 con 0x7fc6a8130260 2023-10-03T10:13:30.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.064+0000 7fc6ad3de700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a809b570 con 0x7fc6a812f8b0 2023-10-03T10:13:30.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.064+0000 7fc6ad3de700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a809e640 con 0x7fc6a812cf60 2023-10-03T10:13:30.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.064+0000 7fc6a77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a8130260 0x7fc6a81c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:30.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.064+0000 7fc6a77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a8130260 0x7fc6a81c3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53840/0 (socket says 172.21.15.3:53840) 2023-10-03T10:13:30.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.064+0000 7fc6a77fe700 1 -- 172.21.15.3:0/3270827178 learned_addr learned my addr 172.21.15.3:0/3270827178 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:30.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.064+0000 7fc6a67fc700 1 --2- 172.21.15.3:0/3270827178 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6a812f8b0 0x7fc6a809c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:30.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.064+0000 7f8c3e7fc700 1 -- 172.21.15.3:0/3711217868 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c440052d0 con 0x7f8c481338c0 2023-10-03T10:13:30.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.064+0000 7f8c3e7fc700 1 -- 172.21.15.3:0/3711217868 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c44005430 con 0x7f8c481338c0 2023-10-03T10:13:30.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.064+0000 7f8c3e7fc700 1 -- 172.21.15.3:0/3711217868 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c440038c0 con 0x7f8c481338c0 2023-10-03T10:13:30.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.064+0000 7fc6a77fe700 1 -- 172.21.15.3:0/3270827178 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6a812cf60 msgr2=0x7fc6a809bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:30.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.064+0000 7fc6a77fe700 1 --2- 172.21.15.3:0/3270827178 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6a812cf60 0x7fc6a809bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.064+0000 7fc6a77fe700 1 -- 172.21.15.3:0/3270827178 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6a812f8b0 msgr2=0x7fc6a809c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:30.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.064+0000 7fc6a77fe700 1 --2- 172.21.15.3:0/3270827178 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6a812f8b0 0x7fc6a809c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.064+0000 7fc6a77fe700 1 -- 172.21.15.3:0/3270827178 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc698011df0 con 0x7fc6a8130260 2023-10-03T10:13:30.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.064+0000 7fc6a67fc700 1 --2- 172.21.15.3:0/3270827178 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6a812f8b0 0x7fc6a809c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:30.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.064+0000 7f8c4fe3f700 1 -- 172.21.15.3:0/3711217868 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c4806f960 con 0x7f8c481338c0 2023-10-03T10:13:30.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.064+0000 7f8c4fe3f700 1 -- 172.21.15.3:0/3711217868 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4811d6a0 con 0x7f8c481338c0 2023-10-03T10:13:30.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.066+0000 7f8c3e7fc700 1 -- 172.21.15.3:0/3711217868 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8c44003a20 con 0x7f8c481338c0 2023-10-03T10:13:30.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.066+0000 7f8c4fe3f700 1 -- 172.21.15.3:0/3711217868 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c48074ea0 con 0x7f8c481338c0 2023-10-03T10:13:30.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.066+0000 7f8c3e7fc700 1 -- 172.21.15.3:0/3711217868 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8c4403ec70 con 0x7f8c481338c0 2023-10-03T10:13:30.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.064+0000 7fc6a77fe700 1 --2- 172.21.15.3:0/3270827178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a8130260 0x7fc6a81c3f20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000cbc0 tx=0x7fc6a000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:30.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.065+0000 7fc68f7fe700 1 -- 172.21.15.3:0/3270827178 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0003c00 con 0x7fc6a8130260 2023-10-03T10:13:30.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.065+0000 7fc68f7fe700 1 -- 172.21.15.3:0/3270827178 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a0003d60 con 0x7fc6a8130260 2023-10-03T10:13:30.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.065+0000 7fc68f7fe700 1 -- 172.21.15.3:0/3270827178 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a000abf0 con 0x7fc6a8130260 2023-10-03T10:13:30.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.066+0000 7fc6ad3de700 1 -- 172.21.15.3:0/3270827178 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a809e900 con 0x7fc6a8130260 2023-10-03T10:13:30.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.066+0000 7fc6ad3de700 1 -- 172.21.15.3:0/3270827178 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a8119180 con 0x7fc6a8130260 2023-10-03T10:13:30.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.067+0000 7fc68f7fe700 1 -- 172.21.15.3:0/3270827178 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc6a000ad50 con 0x7fc6a8130260 2023-10-03T10:13:30.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.067+0000 7fc6ad3de700 1 -- 172.21.15.3:0/3270827178 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6a8074ea0 con 0x7fc6a8130260 2023-10-03T10:13:30.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.068+0000 7fc68f7fe700 1 -- 172.21.15.3:0/3270827178 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc6a0012420 con 0x7fc6a8130260 2023-10-03T10:13:30.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.071+0000 7f8c3e7fc700 1 -- 172.21.15.3:0/3711217868 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8c44017080 con 0x7f8c481338c0 2023-10-03T10:13:30.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.077+0000 7fc68f7fe700 1 -- 172.21.15.3:0/3270827178 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc6a0044060 con 0x7fc6a8130260 2023-10-03T10:13:30.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.099+0000 7f2ddb930700 1 Processor -- start 2023-10-03T10:13:30.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.099+0000 7f2ddb930700 1 -- start start 2023-10-03T10:13:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.100+0000 7f2ddb930700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2dd409d6f0 0x7f2dd409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.100+0000 7f2ddb930700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2dd409e000 0x7f2dd4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.100+0000 7f2ddb930700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2dd409e410 0x7f2dd4130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.100+0000 7f2ddb930700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dd409f520 con 0x7f2dd409e410 2023-10-03T10:13:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.100+0000 7f2ddb930700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd409f130 con 0x7f2dd409d6f0 2023-10-03T10:13:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.100+0000 7f2ddb930700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd409e930 con 0x7f2dd409e000 2023-10-03T10:13:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.100+0000 7f2dd96cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2dd409d6f0 0x7f2dd409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.100+0000 7f2dd8ecb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2dd409e000 0x7f2dd4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.100+0000 7f2dd96cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2dd409d6f0 0x7f2dd409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56966/0 (socket says 172.21.15.3:56966) 2023-10-03T10:13:30.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.100+0000 7f2dd96cc700 1 -- 172.21.15.3:0/1841855456 learned_addr learned my addr 172.21.15.3:0/1841855456 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:30.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.100+0000 7f2dd9ecd700 1 --2- 172.21.15.3:0/1841855456 >> v2:172.21.15.3:3300/0 conn(0x7f2dd409e410 0x7f2dd4130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:30.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.101+0000 7f2dd8ecb700 1 -- 172.21.15.3:0/1841855456 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2dd409d6f0 msgr2=0x7f2dd409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:30.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.101+0000 7f2dd8ecb700 1 --2- 172.21.15.3:0/1841855456 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2dd409d6f0 0x7f2dd409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.101+0000 7f2dd8ecb700 1 -- 172.21.15.3:0/1841855456 >> v2:172.21.15.3:3300/0 conn(0x7f2dd409e410 msgr2=0x7f2dd4130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:30.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.101+0000 7f2dd8ecb700 1 --2- 172.21.15.3:0/1841855456 >> v2:172.21.15.3:3300/0 conn(0x7f2dd409e410 0x7f2dd4130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.101+0000 7f2dd8ecb700 1 -- 172.21.15.3:0/1841855456 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd409b2d0 con 0x7f2dd409e000 2023-10-03T10:13:30.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.101+0000 7f2dd8ecb700 1 --2- 172.21.15.3:0/1841855456 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2dd409e000 0x7f2dd4130660 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4009870 tx=0x7f2dc4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a610f1b0ac9edfc server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:30.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.101+0000 7f2dcbfff700 1 -- 172.21.15.3:0/1841855456 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4010a90 con 0x7f2dd409e000 2023-10-03T10:13:30.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.101+0000 7f2dcbfff700 1 -- 172.21.15.3:0/1841855456 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc4010bf0 con 0x7f2dd409e000 2023-10-03T10:13:30.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.101+0000 7f2dcbfff700 1 -- 172.21.15.3:0/1841855456 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4018440 con 0x7f2dd409e000 2023-10-03T10:13:30.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.102+0000 7f2ddb930700 1 -- 172.21.15.3:0/1841855456 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2dd409e000 msgr2=0x7f2dd4130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:30.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.102+0000 7f2ddb930700 1 --2- 172.21.15.3:0/1841855456 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2dd409e000 0x7f2dd4130660 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4009870 tx=0x7f2dc4009b80 comp rx=0 tx=0).stop 2023-10-03T10:13:30.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.102+0000 7f2ddb930700 1 -- 172.21.15.3:0/1841855456 shutdown_connections 2023-10-03T10:13:30.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.102+0000 7f2ddb930700 1 --2- 172.21.15.3:0/1841855456 >> v2:172.21.15.3:3300/0 conn(0x7f2dd409e410 0x7f2dd4130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.102+0000 7f2ddb930700 1 --2- 172.21.15.3:0/1841855456 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2dd409d6f0 0x7f2dd409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.102+0000 7f2ddb930700 1 --2- 172.21.15.3:0/1841855456 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2dd409e000 0x7f2dd4130660 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.102+0000 7f2ddb930700 1 -- 172.21.15.3:0/1841855456 >> 172.21.15.3:0/1841855456 conn(0x7f2dd4066f50 msgr2=0x7f2dd409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:30.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.102+0000 7f2ddb930700 1 -- 172.21.15.3:0/1841855456 shutdown_connections 2023-10-03T10:13:30.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.102+0000 7f2ddb930700 1 -- 172.21.15.3:0/1841855456 wait complete. 2023-10-03T10:13:30.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.103+0000 7f2ddb930700 1 Processor -- start 2023-10-03T10:13:30.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.103+0000 7f2ddb930700 1 -- start start 2023-10-03T10:13:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.104+0000 7f2ddb930700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2dd409d6f0 0x7f2dd411e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.104+0000 7f2ddb930700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2dd409e000 0x7f2dd411e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.104+0000 7f2ddb930700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dd409e410 0x7f2dd411eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.104+0000 7f2ddb930700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd4121bd0 con 0x7f2dd409e410 2023-10-03T10:13:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.104+0000 7f2ddb930700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd40a1d00 con 0x7f2dd409d6f0 2023-10-03T10:13:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.104+0000 7f2ddb930700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd409c700 con 0x7f2dd409e000 2023-10-03T10:13:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.104+0000 7f2dd96cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2dd409d6f0 0x7f2dd411e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:30.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.104+0000 7f2dd96cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2dd409d6f0 0x7f2dd411e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56976/0 (socket says 172.21.15.3:56976) 2023-10-03T10:13:30.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.104+0000 7f2dd96cc700 1 -- 172.21.15.3:0/3723581852 learned_addr learned my addr 172.21.15.3:0/3723581852 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:30.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.104+0000 7f2dd96cc700 1 -- 172.21.15.3:0/3723581852 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2dd409e000 msgr2=0x7f2dd411e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:30.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.104+0000 7f2dd96cc700 1 --2- 172.21.15.3:0/3723581852 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2dd409e000 0x7f2dd411e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.104+0000 7f2dd96cc700 1 -- 172.21.15.3:0/3723581852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dd409e410 msgr2=0x7f2dd411eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:30.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.104+0000 7f2dd9ecd700 1 --2- 172.21.15.3:0/3723581852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dd409e410 0x7f2dd411eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:30.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.104+0000 7f2dd96cc700 1 --2- 172.21.15.3:0/3723581852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dd409e410 0x7f2dd411eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.104+0000 7f2dd96cc700 1 -- 172.21.15.3:0/3723581852 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc40094a0 con 0x7f2dd409d6f0 2023-10-03T10:13:30.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.104+0000 7f2dd9ecd700 1 --2- 172.21.15.3:0/3723581852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dd409e410 0x7f2dd411eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:13:30.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.105+0000 7f2dd96cc700 1 --2- 172.21.15.3:0/3723581852 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2dd409d6f0 0x7f2dd411e0b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2dd000ed40 tx=0x7f2dd000d630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:30.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.105+0000 7f2dc9ffb700 1 -- 172.21.15.3:0/3723581852 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd000de40 con 0x7f2dd409d6f0 2023-10-03T10:13:30.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.105+0000 7f2dc9ffb700 1 -- 172.21.15.3:0/3723581852 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd0010440 con 0x7f2dd409d6f0 2023-10-03T10:13:30.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.105+0000 7f2ddb930700 1 -- 172.21.15.3:0/3723581852 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd4121630 con 0x7f2dd409d6f0 2023-10-03T10:13:30.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.105+0000 7f2dc9ffb700 1 -- 172.21.15.3:0/3723581852 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd00107b0 con 0x7f2dd409d6f0 2023-10-03T10:13:30.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.105+0000 7f2ddb930700 1 -- 172.21.15.3:0/3723581852 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd4120210 con 0x7f2dd409d6f0 2023-10-03T10:13:30.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.106+0000 7f2dc9ffb700 1 -- 172.21.15.3:0/3723581852 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2dd0010b20 con 0x7f2dd409d6f0 2023-10-03T10:13:30.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.106+0000 7f2dc9ffb700 1 -- 172.21.15.3:0/3723581852 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2dd0040020 con 0x7f2dd409d6f0 2023-10-03T10:13:30.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.106+0000 7f2ddb930700 1 -- 172.21.15.3:0/3723581852 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dd4074ea0 con 0x7f2dd409d6f0 2023-10-03T10:13:30.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.111+0000 7f2dc9ffb700 1 -- 172.21.15.3:0/3723581852 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2dd001d020 con 0x7f2dd409d6f0 2023-10-03T10:13:30.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.155+0000 7f57cc130700 1 -- 172.21.15.3:0/508114511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f57c406c360 con 0x7f57c406a770 2023-10-03T10:13:30.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.156+0000 7f57ba7fc700 1 -- 172.21.15.3:0/508114511 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f57c001f9a0 con 0x7f57c406a770 2023-10-03T10:13:30.156 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:30.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.158+0000 7f57cc130700 1 -- 172.21.15.3:0/508114511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57c406a770 msgr2=0x7f57c41c5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:30.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.158+0000 7f57cc130700 1 --2- 172.21.15.3:0/508114511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57c406a770 0x7f57c41c5e50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f57c0009c40 tx=0x7f57c0004750 comp rx=0 tx=0).stop 2023-10-03T10:13:30.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.158+0000 7f57cc130700 1 -- 172.21.15.3:0/508114511 shutdown_connections 2023-10-03T10:13:30.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.158+0000 7f57cc130700 1 --2- 172.21.15.3:0/508114511 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57c4131b40 0x7f57c41c68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.158+0000 7f57cc130700 1 --2- 172.21.15.3:0/508114511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57c406a770 0x7f57c41c5e50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.158+0000 7f57cc130700 1 --2- 172.21.15.3:0/508114511 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57c406b080 0x7f57c41c6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.158+0000 7f57cc130700 1 -- 172.21.15.3:0/508114511 >> 172.21.15.3:0/508114511 conn(0x7f57c4066f50 msgr2=0x7f57c406ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:30.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.158+0000 7f57cc130700 1 -- 172.21.15.3:0/508114511 shutdown_connections 2023-10-03T10:13:30.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.158+0000 7f57cc130700 1 -- 172.21.15.3:0/508114511 wait complete. 2023-10-03T10:13:30.173 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:30.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.320+0000 7f2ddb930700 1 -- 172.21.15.3:0/3723581852 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2dd406c360 con 0x7f2dd409d6f0 2023-10-03T10:13:30.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.320+0000 7f2dc9ffb700 1 -- 172.21.15.3:0/3723581852 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2dd0010dd0 con 0x7f2dd409d6f0 2023-10-03T10:13:30.321 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:30.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.322+0000 7f2ddb930700 1 -- 172.21.15.3:0/3723581852 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2dd409d6f0 msgr2=0x7f2dd411e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:30.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.322+0000 7f2ddb930700 1 --2- 172.21.15.3:0/3723581852 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2dd409d6f0 0x7f2dd411e0b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2dd000ed40 tx=0x7f2dd000d630 comp rx=0 tx=0).stop 2023-10-03T10:13:30.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.322+0000 7f2ddb930700 1 -- 172.21.15.3:0/3723581852 shutdown_connections 2023-10-03T10:13:30.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.322+0000 7f2ddb930700 1 --2- 172.21.15.3:0/3723581852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dd409e410 0x7f2dd411eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.322+0000 7f2ddb930700 1 --2- 172.21.15.3:0/3723581852 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2dd409d6f0 0x7f2dd411e0b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.322+0000 7f2ddb930700 1 --2- 172.21.15.3:0/3723581852 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2dd409e000 0x7f2dd411e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.322+0000 7f2ddb930700 1 -- 172.21.15.3:0/3723581852 >> 172.21.15.3:0/3723581852 conn(0x7f2dd4066f50 msgr2=0x7f2dd40a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:30.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.322+0000 7f2ddb930700 1 -- 172.21.15.3:0/3723581852 shutdown_connections 2023-10-03T10:13:30.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.322+0000 7f2ddb930700 1 -- 172.21.15.3:0/3723581852 wait complete. 2023-10-03T10:13:30.336 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:30.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.365+0000 7f8c4fe3f700 1 -- 172.21.15.3:0/3711217868 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8c4806b9e0 con 0x7f8c481338c0 2023-10-03T10:13:30.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.365+0000 7f8c3e7fc700 1 -- 172.21.15.3:0/3711217868 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8c44018260 con 0x7f8c481338c0 2023-10-03T10:13:30.366 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:30.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.368+0000 7f8c4fe3f700 1 -- 172.21.15.3:0/3711217868 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c481338c0 msgr2=0x7f8c481c85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:30.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.368+0000 7f8c4fe3f700 1 --2- 172.21.15.3:0/3711217868 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c481338c0 0x7f8c481c85f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8c4400c4e0 tx=0x7f8c4400c7f0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.368+0000 7f8c4fe3f700 1 -- 172.21.15.3:0/3711217868 shutdown_connections 2023-10-03T10:13:30.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.368+0000 7f8c4fe3f700 1 --2- 172.21.15.3:0/3711217868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c48139150 0x7f8c48119ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.368+0000 7f8c4fe3f700 1 --2- 172.21.15.3:0/3711217868 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c481338c0 0x7f8c481c85f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.368+0000 7f8c4fe3f700 1 --2- 172.21.15.3:0/3711217868 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c48136830 0x7f8c481c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.368+0000 7f8c4fe3f700 1 -- 172.21.15.3:0/3711217868 >> 172.21.15.3:0/3711217868 conn(0x7f8c480a2aa0 msgr2=0x7f8c4812ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:30.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.368+0000 7f8c4fe3f700 1 -- 172.21.15.3:0/3711217868 shutdown_connections 2023-10-03T10:13:30.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.368+0000 7f8c4fe3f700 1 -- 172.21.15.3:0/3711217868 wait complete. 2023-10-03T10:13:30.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.370+0000 7fc6ad3de700 1 -- 172.21.15.3:0/3270827178 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc6a8129c30 con 0x7fc6a8130260 2023-10-03T10:13:30.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.370+0000 7fc68f7fe700 1 -- 172.21.15.3:0/3270827178 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc6a0032090 con 0x7fc6a8130260 2023-10-03T10:13:30.371 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:30.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.373+0000 7fc6ad3de700 1 -- 172.21.15.3:0/3270827178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a8130260 msgr2=0x7fc6a81c3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:30.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.373+0000 7fc6ad3de700 1 --2- 172.21.15.3:0/3270827178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a8130260 0x7fc6a81c3f20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000cbc0 tx=0x7fc6a000cf80 comp rx=0 tx=0).stop 2023-10-03T10:13:30.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.373+0000 7fc6ad3de700 1 -- 172.21.15.3:0/3270827178 shutdown_connections 2023-10-03T10:13:30.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.373+0000 7fc6ad3de700 1 --2- 172.21.15.3:0/3270827178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a8130260 0x7fc6a81c3f20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.373+0000 7fc6ad3de700 1 --2- 172.21.15.3:0/3270827178 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6a812cf60 0x7fc6a809bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.373+0000 7fc6ad3de700 1 --2- 172.21.15.3:0/3270827178 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6a812f8b0 0x7fc6a809c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:30.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.373+0000 7fc6ad3de700 1 -- 172.21.15.3:0/3270827178 >> 172.21.15.3:0/3270827178 conn(0x7fc6a80a2aa0 msgr2=0x7fc6a806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:30.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.373+0000 7fc6ad3de700 1 -- 172.21.15.3:0/3270827178 shutdown_connections 2023-10-03T10:13:30.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:30.373+0000 7fc6ad3de700 1 -- 172.21.15.3:0/3270827178 wait complete. 2023-10-03T10:13:30.383 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:30.387 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:31.174 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:31.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.319+0000 7f4af4f74700 1 Processor -- start 2023-10-03T10:13:31.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.319+0000 7f4af4f74700 1 -- start start 2023-10-03T10:13:31.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.320+0000 7f4af4f74700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4af0133420 0x7f4af01337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:31.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.320+0000 7f4af4f74700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4af0134260 0x7f4af01306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:31.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.320+0000 7f4af4f74700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4af006b990 con 0x7f4af0133e30 2023-10-03T10:13:31.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.320+0000 7f4af4f74700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af00a0590 con 0x7f4af0133420 2023-10-03T10:13:31.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.320+0000 7f4af4f74700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4af009fb80 con 0x7f4af0134260 2023-10-03T10:13:31.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.320+0000 7f4aedd9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4af0133e30 0x7f4af012ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48844/0 (socket says 172.21.15.3:48844) 2023-10-03T10:13:31.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.320+0000 7f4aeed9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4af0134260 0x7f4af01306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:31.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.320+0000 7f4aedd9b700 1 -- 172.21.15.3:0/1676145774 learned_addr learned my addr 172.21.15.3:0/1676145774 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:31.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.320+0000 7f4aee59c700 1 --2- 172.21.15.3:0/1676145774 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4af0133420 0x7f4af01337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:31.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.320+0000 7f4aed59a700 1 -- 172.21.15.3:0/1676145774 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2529095217 0 0) 0x7f4af006b990 con 0x7f4af0133e30 2023-10-03T10:13:31.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.320+0000 7f4aed59a700 1 -- 172.21.15.3:0/1676145774 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4adc0035b0 con 0x7f4af0133e30 2023-10-03T10:13:31.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.320+0000 7f4aee59c700 1 -- 172.21.15.3:0/1676145774 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4af0134260 msgr2=0x7f4af01306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:31.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.320+0000 7f4aee59c700 1 --2- 172.21.15.3:0/1676145774 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4af0134260 0x7f4af01306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.320+0000 7f4aee59c700 1 -- 172.21.15.3:0/1676145774 >> v1:172.21.15.3:6789/0 conn(0x7f4af0133e30 legacy=0x7f4af012ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:31.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.321+0000 7f4aee59c700 1 -- 172.21.15.3:0/1676145774 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af00a0030 con 0x7f4af0133420 2023-10-03T10:13:31.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.321+0000 7f4aeed9d700 1 --2- 172.21.15.3:0/1676145774 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4af0134260 0x7f4af01306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:31.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.321+0000 7f4aee59c700 1 --2- 172.21.15.3:0/1676145774 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4af0133420 0x7f4af01337f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009870 tx=0x7f4ae0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa59a1dc9bbf5b2a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:31.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.321+0000 7f4aed59a700 1 -- 172.21.15.3:0/1676145774 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0011830 con 0x7f4af0133420 2023-10-03T10:13:31.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.321+0000 7f4aed59a700 1 -- 172.21.15.3:0/1676145774 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae0011990 con 0x7f4af0133420 2023-10-03T10:13:31.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.321+0000 7f4aed59a700 1 -- 172.21.15.3:0/1676145774 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0011d20 con 0x7f4af0133420 2023-10-03T10:13:31.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.321+0000 7f4af4f74700 1 -- 172.21.15.3:0/1676145774 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4af0133420 msgr2=0x7f4af01337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:31.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.321+0000 7f4af4f74700 1 --2- 172.21.15.3:0/1676145774 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4af0133420 0x7f4af01337f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009870 tx=0x7f4ae0009b80 comp rx=0 tx=0).stop 2023-10-03T10:13:31.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.322+0000 7f4af4f74700 1 -- 172.21.15.3:0/1676145774 shutdown_connections 2023-10-03T10:13:31.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.322+0000 7f4af4f74700 1 --2- 172.21.15.3:0/1676145774 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4af0133420 0x7f4af01337f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.322+0000 7f4af4f74700 1 --2- 172.21.15.3:0/1676145774 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4af0134260 0x7f4af01306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.322+0000 7f4af4f74700 1 -- 172.21.15.3:0/1676145774 >> 172.21.15.3:0/1676145774 conn(0x7f4af00a4210 msgr2=0x7f4af00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:31.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.322+0000 7f4af4f74700 1 -- 172.21.15.3:0/1676145774 shutdown_connections 2023-10-03T10:13:31.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.322+0000 7f4af4f74700 1 -- 172.21.15.3:0/1676145774 wait complete. 2023-10-03T10:13:31.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.323+0000 7f4af4f74700 1 Processor -- start 2023-10-03T10:13:31.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.323+0000 7f4af4f74700 1 -- start start 2023-10-03T10:13:31.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.323+0000 7f4af4f74700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4af0133420 0x7f4af009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:31.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.323+0000 7f4af4f74700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af0133e30 0x7f4af009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:31.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.323+0000 7f4af4f74700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4af0134260 0x7f4af009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:31.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.323+0000 7f4af4f74700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af011fc40 con 0x7f4af0133e30 2023-10-03T10:13:31.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.323+0000 7f4af4f74700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af0120600 con 0x7f4af0134260 2023-10-03T10:13:31.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.323+0000 7f4af4f74700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4af011fe20 con 0x7f4af0133420 2023-10-03T10:13:31.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.323+0000 7f4aedd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af0133e30 0x7f4af009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:31.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.323+0000 7f4aeed9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4af0134260 0x7f4af009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:31.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.323+0000 7f4aee59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4af0133420 0x7f4af009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:31.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.323+0000 7f4aeed9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4af0134260 0x7f4af009cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56996/0 (socket says 172.21.15.3:56996) 2023-10-03T10:13:31.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.323+0000 7f4aeed9d700 1 -- 172.21.15.3:0/2213280050 learned_addr learned my addr 172.21.15.3:0/2213280050 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:31.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.323+0000 7f4aedd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af0133e30 0x7f4af009c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53866/0 (socket says 172.21.15.3:53866) 2023-10-03T10:13:31.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.324+0000 7f4aeed9d700 1 -- 172.21.15.3:0/2213280050 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4af0133420 msgr2=0x7f4af009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:31.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.324+0000 7f4aeed9d700 1 --2- 172.21.15.3:0/2213280050 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4af0133420 0x7f4af009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.324+0000 7f4aeed9d700 1 -- 172.21.15.3:0/2213280050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af0133e30 msgr2=0x7f4af009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:31.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.324+0000 7f4aeed9d700 1 --2- 172.21.15.3:0/2213280050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af0133e30 0x7f4af009c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.324+0000 7f4aeed9d700 1 -- 172.21.15.3:0/2213280050 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae8009a70 con 0x7f4af0134260 2023-10-03T10:13:31.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.324+0000 7f4aeed9d700 1 --2- 172.21.15.3:0/2213280050 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4af0134260 0x7f4af009cf00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009eb0 tx=0x7f4ae800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:31.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.324+0000 7f4ae6ffd700 1 -- 172.21.15.3:0/2213280050 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8009d50 con 0x7f4af0134260 2023-10-03T10:13:31.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.324+0000 7f4ae6ffd700 1 -- 172.21.15.3:0/2213280050 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae800f040 con 0x7f4af0134260 2023-10-03T10:13:31.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.324+0000 7f4af4f74700 1 -- 172.21.15.3:0/2213280050 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae00094a0 con 0x7f4af0134260 2023-10-03T10:13:31.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.324+0000 7f4ae6ffd700 1 -- 172.21.15.3:0/2213280050 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae800ceb0 con 0x7f4af0134260 2023-10-03T10:13:31.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.324+0000 7f4af4f74700 1 -- 172.21.15.3:0/2213280050 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af009df60 con 0x7f4af0134260 2023-10-03T10:13:31.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.325+0000 7f4ae6ffd700 1 -- 172.21.15.3:0/2213280050 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4ae8014480 con 0x7f4af0134260 2023-10-03T10:13:31.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.325+0000 7f4ae6ffd700 1 -- 172.21.15.3:0/2213280050 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4ae80403b0 con 0x7f4af0134260 2023-10-03T10:13:31.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.325+0000 7f4af4f74700 1 -- 172.21.15.3:0/2213280050 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af0074ea0 con 0x7f4af0134260 2023-10-03T10:13:31.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.330+0000 7f4ae6ffd700 1 -- 172.21.15.3:0/2213280050 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4ae8007920 con 0x7f4af0134260 2023-10-03T10:13:31.337 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:31.384 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:31.389 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:31.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.484+0000 7fe5d2c57700 1 Processor -- start 2023-10-03T10:13:31.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.484+0000 7fe5d2c57700 1 -- start start 2023-10-03T10:13:31.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.484+0000 7fe5d2c57700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe5cc0a0a30 0x7fe5cc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:31.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.484+0000 7fe5d2c57700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5cc134170 0x7fe5cc13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:31.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.484+0000 7fe5d2c57700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5cc09edd0 con 0x7fe5cc0a1340 2023-10-03T10:13:31.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.484+0000 7fe5d2c57700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5cc09e1f0 con 0x7fe5cc0a0a30 2023-10-03T10:13:31.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.484+0000 7fe5d2c57700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc09dc50 con 0x7fe5cc134170 2023-10-03T10:13:31.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.485+0000 7fe5d09f3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe5cc0a0a30 0x7fe5cc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:31.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.485+0000 7fe5d11f4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5cc134170 0x7fe5cc13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:31.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.485+0000 7fe5d09f3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe5cc0a0a30 0x7fe5cc0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53880/0 (socket says 172.21.15.3:53880) 2023-10-03T10:13:31.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.485+0000 7fe5d09f3700 1 -- 172.21.15.3:0/2385463051 learned_addr learned my addr 172.21.15.3:0/2385463051 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:31.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.485+0000 7fe5cbfff700 1 --1- 172.21.15.3:0/2385463051 >> v1:172.21.15.3:6789/0 conn(0x7fe5cc0a1340 0x7fe5cc133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48850/0 (socket says 172.21.15.3:48850) 2023-10-03T10:13:31.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.485+0000 7fe5d11f4700 1 -- 172.21.15.3:0/2385463051 >> v2:172.21.15.3:3300/0 conn(0x7fe5cc0a0a30 msgr2=0x7fe5cc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:31.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.485+0000 7fe5d11f4700 1 --2- 172.21.15.3:0/2385463051 >> v2:172.21.15.3:3300/0 conn(0x7fe5cc0a0a30 0x7fe5cc0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.485+0000 7fe5d11f4700 1 -- 172.21.15.3:0/2385463051 >> v1:172.21.15.3:6789/0 conn(0x7fe5cc0a1340 legacy=0x7fe5cc133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:31.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.485+0000 7fe5d11f4700 1 -- 172.21.15.3:0/2385463051 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5cc0a1fe0 con 0x7fe5cc134170 2023-10-03T10:13:31.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.485+0000 7fe5d09f3700 1 --2- 172.21.15.3:0/2385463051 >> v2:172.21.15.3:3300/0 conn(0x7fe5cc0a0a30 0x7fe5cc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:31.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.485+0000 7fe5d11f4700 1 --2- 172.21.15.3:0/2385463051 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5cc134170 0x7fe5cc13bd30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4009d90 tx=0x7fe5c400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed3300c0aa247e96 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:31.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.485+0000 7fe5cb7fe700 1 -- 172.21.15.3:0/2385463051 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c400e070 con 0x7fe5cc134170 2023-10-03T10:13:31.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.486+0000 7fe5cb7fe700 1 -- 172.21.15.3:0/2385463051 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c400bb20 con 0x7fe5cc134170 2023-10-03T10:13:31.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.486+0000 7fe5cb7fe700 1 -- 172.21.15.3:0/2385463051 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4012430 con 0x7fe5cc134170 2023-10-03T10:13:31.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.487+0000 7fe5d2c57700 1 -- 172.21.15.3:0/2385463051 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5cc134170 msgr2=0x7fe5cc13bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:31.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.487+0000 7fe5d2c57700 1 --2- 172.21.15.3:0/2385463051 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5cc134170 0x7fe5cc13bd30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4009d90 tx=0x7fe5c400b330 comp rx=0 tx=0).stop 2023-10-03T10:13:31.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.487+0000 7fe5d2c57700 1 -- 172.21.15.3:0/2385463051 shutdown_connections 2023-10-03T10:13:31.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.487+0000 7fe5d2c57700 1 --2- 172.21.15.3:0/2385463051 >> v2:172.21.15.3:3300/0 conn(0x7fe5cc0a0a30 0x7fe5cc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.487+0000 7fe5d2c57700 1 --2- 172.21.15.3:0/2385463051 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5cc134170 0x7fe5cc13bd30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.487+0000 7fe5d2c57700 1 -- 172.21.15.3:0/2385463051 >> 172.21.15.3:0/2385463051 conn(0x7fe5cc0a6550 msgr2=0x7fe5cc0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:31.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.487+0000 7fe5d2c57700 1 -- 172.21.15.3:0/2385463051 shutdown_connections 2023-10-03T10:13:31.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.487+0000 7fe5d2c57700 1 -- 172.21.15.3:0/2385463051 wait complete. 2023-10-03T10:13:31.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.488+0000 7fe5d2c57700 1 Processor -- start 2023-10-03T10:13:31.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.488+0000 7fe5d2c57700 1 -- start start 2023-10-03T10:13:31.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.489+0000 7fe5d2c57700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5cc0a0a30 0x7fe5cc11e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:31.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.489+0000 7fe5d2c57700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5cc0a1340 0x7fe5cc11ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:31.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.489+0000 7fe5d2c57700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5cc134170 0x7fe5cc11f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:31.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.489+0000 7fe5d2c57700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc120220 con 0x7fe5cc134170 2023-10-03T10:13:31.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.489+0000 7fe5d2c57700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc1335a0 con 0x7fe5cc0a0a30 2023-10-03T10:13:31.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.489+0000 7fe5d2c57700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc122130 con 0x7fe5cc0a1340 2023-10-03T10:13:31.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.489+0000 7fe5d09f3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5cc0a0a30 0x7fe5cc11e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:31.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.489+0000 7fe5d09f3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5cc0a0a30 0x7fe5cc11e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:57000/0 (socket says 172.21.15.3:57000) 2023-10-03T10:13:31.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.489+0000 7fe5d09f3700 1 -- 172.21.15.3:0/1935145083 learned_addr learned my addr 172.21.15.3:0/1935145083 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:31.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.489+0000 7fe5cbfff700 1 --2- 172.21.15.3:0/1935145083 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5cc0a1340 0x7fe5cc11ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:31.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.489+0000 7fe5d11f4700 1 --2- 172.21.15.3:0/1935145083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5cc134170 0x7fe5cc11f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:31.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.489+0000 7fe5d09f3700 1 -- 172.21.15.3:0/1935145083 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5cc0a1340 msgr2=0x7fe5cc11ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:31.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.489+0000 7fe5d09f3700 1 --2- 172.21.15.3:0/1935145083 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5cc0a1340 0x7fe5cc11ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.489+0000 7fe5d09f3700 1 -- 172.21.15.3:0/1935145083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5cc134170 msgr2=0x7fe5cc11f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:31.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.489+0000 7fe5d09f3700 1 --2- 172.21.15.3:0/1935145083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5cc134170 0x7fe5cc11f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.489+0000 7fe5d09f3700 1 -- 172.21.15.3:0/1935145083 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c0009550 con 0x7fe5cc0a0a30 2023-10-03T10:13:31.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.490+0000 7fe5d09f3700 1 --2- 172.21.15.3:0/1935145083 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5cc0a0a30 0x7fe5cc11e6a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe5c000ebe0 tx=0x7fe5c000eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:31.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.490+0000 7fe5c97fa700 1 -- 172.21.15.3:0/1935145083 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c000c850 con 0x7fe5cc0a0a30 2023-10-03T10:13:31.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.490+0000 7fe5c97fa700 1 -- 172.21.15.3:0/1935145083 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c000c9b0 con 0x7fe5cc0a0a30 2023-10-03T10:13:31.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.490+0000 7fe5c97fa700 1 -- 172.21.15.3:0/1935145083 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c000cd40 con 0x7fe5cc0a0a30 2023-10-03T10:13:31.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.490+0000 7fe5d2c57700 1 -- 172.21.15.3:0/1935145083 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c4009a70 con 0x7fe5cc0a0a30 2023-10-03T10:13:31.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.490+0000 7fe5d2c57700 1 -- 172.21.15.3:0/1935145083 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5cc0a25e0 con 0x7fe5cc0a0a30 2023-10-03T10:13:31.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.491+0000 7fe5c97fa700 1 -- 172.21.15.3:0/1935145083 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe5c0013470 con 0x7fe5cc0a0a30 2023-10-03T10:13:31.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.491+0000 7fe5c97fa700 1 -- 172.21.15.3:0/1935145083 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe5c003f890 con 0x7fe5cc0a0a30 2023-10-03T10:13:31.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.492+0000 7fe5d2c57700 1 -- 172.21.15.3:0/1935145083 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5cc074ea0 con 0x7fe5cc0a0a30 2023-10-03T10:13:31.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.497+0000 7fe5c97fa700 1 -- 172.21.15.3:0/1935145083 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe5c00232f0 con 0x7fe5cc0a0a30 2023-10-03T10:13:31.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.531+0000 7f21cf129700 1 Processor -- start 2023-10-03T10:13:31.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.531+0000 7f21cf129700 1 -- start start 2023-10-03T10:13:31.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.532+0000 7f21cf129700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21c809eca0 0x7f21c813cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:31.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.532+0000 7f21cf129700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f21c806f050 0x7f21c813f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:31.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.532+0000 7f21cf129700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21c8070ee0 con 0x7f21c806e570 2023-10-03T10:13:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.532+0000 7f21cf129700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21c8071980 con 0x7f21c806f050 2023-10-03T10:13:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.532+0000 7f21cf129700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f21c8071480 con 0x7f21c809eca0 2023-10-03T10:13:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.532+0000 7f21c7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21c809eca0 0x7f21c813cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.532+0000 7f21cd6c6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f21c806f050 0x7f21c813f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.532+0000 7f21c7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21c809eca0 0x7f21c813cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54146/0 (socket says 172.21.15.3:54146) 2023-10-03T10:13:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.532+0000 7f21c7fff700 1 -- 172.21.15.3:0/3331396680 learned_addr learned my addr 172.21.15.3:0/3331396680 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:31.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.532+0000 7f21c7fff700 1 -- 172.21.15.3:0/3331396680 >> v2:172.21.15.3:3300/0 conn(0x7f21c806f050 msgr2=0x7f21c813f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:31.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.532+0000 7f21c7fff700 1 --2- 172.21.15.3:0/3331396680 >> v2:172.21.15.3:3300/0 conn(0x7f21c806f050 0x7f21c813f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.532+0000 7f21c7fff700 1 -- 172.21.15.3:0/3331396680 >> v1:172.21.15.3:6789/0 conn(0x7f21c806e570 legacy=0x7f21c806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:31.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.532+0000 7f21c7fff700 1 -- 172.21.15.3:0/3331396680 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c8071b90 con 0x7f21c809eca0 2023-10-03T10:13:31.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.532+0000 7f21cd6c6700 1 --2- 172.21.15.3:0/3331396680 >> v2:172.21.15.3:3300/0 conn(0x7f21c806f050 0x7f21c813f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:31.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.533+0000 7f21c7fff700 1 --2- 172.21.15.3:0/3331396680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21c809eca0 0x7f21c813cea0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f21b40098f0 tx=0x7f21b4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2fd88f33a6199cd server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:31.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.533+0000 7f21c77fe700 1 -- 172.21.15.3:0/3331396680 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b4011830 con 0x7f21c809eca0 2023-10-03T10:13:31.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.533+0000 7f21c77fe700 1 -- 172.21.15.3:0/3331396680 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b4011990 con 0x7f21c809eca0 2023-10-03T10:13:31.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.533+0000 7f21c77fe700 1 -- 172.21.15.3:0/3331396680 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b4011d20 con 0x7f21c809eca0 2023-10-03T10:13:31.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.534+0000 7f21cf129700 1 -- 172.21.15.3:0/3331396680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21c809eca0 msgr2=0x7f21c813cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:31.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.534+0000 7f21cf129700 1 --2- 172.21.15.3:0/3331396680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21c809eca0 0x7f21c813cea0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f21b40098f0 tx=0x7f21b4009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:31.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.534+0000 7f21cf129700 1 -- 172.21.15.3:0/3331396680 shutdown_connections 2023-10-03T10:13:31.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.534+0000 7f21cf129700 1 --2- 172.21.15.3:0/3331396680 >> v2:172.21.15.3:3300/0 conn(0x7f21c806f050 0x7f21c813f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.534+0000 7f21cf129700 1 --2- 172.21.15.3:0/3331396680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21c809eca0 0x7f21c813cea0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.534+0000 7f21cf129700 1 -- 172.21.15.3:0/3331396680 >> 172.21.15.3:0/3331396680 conn(0x7f21c80a2aa0 msgr2=0x7f21c80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:31.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.534+0000 7f21cf129700 1 -- 172.21.15.3:0/3331396680 shutdown_connections 2023-10-03T10:13:31.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.534+0000 7f21cf129700 1 -- 172.21.15.3:0/3331396680 wait complete. 2023-10-03T10:13:31.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.535+0000 7f21cf129700 1 Processor -- start 2023-10-03T10:13:31.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.535+0000 7f21cf129700 1 -- start start 2023-10-03T10:13:31.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.536+0000 7f21cf129700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21c806e570 0x7f21c813cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:31.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.536+0000 7f21cf129700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21c806f050 0x7f21c813d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:31.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.536+0000 7f21cf129700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21c812e180 0x7f21c813d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:31.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.536+0000 7f21cf129700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c812dd20 con 0x7f21c806f050 2023-10-03T10:13:31.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.536+0000 7f21cf129700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c812d6e0 con 0x7f21c806e570 2023-10-03T10:13:31.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.536+0000 7f21cf129700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f21c812c5d0 con 0x7f21c812e180 2023-10-03T10:13:31.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.536+0000 7f21c7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21c806f050 0x7f21c813d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:31.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.536+0000 7f21c7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21c806f050 0x7f21c813d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53910/0 (socket says 172.21.15.3:53910) 2023-10-03T10:13:31.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.536+0000 7f21c7fff700 1 -- 172.21.15.3:0/1429224320 learned_addr learned my addr 172.21.15.3:0/1429224320 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:31.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.536+0000 7f21cd6c6700 1 --2- 172.21.15.3:0/1429224320 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21c812e180 0x7f21c813d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:31.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.536+0000 7f21c7fff700 1 -- 172.21.15.3:0/1429224320 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21c812e180 msgr2=0x7f21c813d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:31.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.536+0000 7f21c7fff700 1 --2- 172.21.15.3:0/1429224320 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21c812e180 0x7f21c813d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.536+0000 7f21c7fff700 1 -- 172.21.15.3:0/1429224320 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21c806e570 msgr2=0x7f21c813cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:31.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.536+0000 7f21c7fff700 1 --2- 172.21.15.3:0/1429224320 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21c806e570 0x7f21c813cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.536+0000 7f21c7fff700 1 -- 172.21.15.3:0/1429224320 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c0009a70 con 0x7f21c806f050 2023-10-03T10:13:31.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.536+0000 7f21cd6c6700 1 --2- 172.21.15.3:0/1429224320 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21c812e180 0x7f21c813d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:31.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.537+0000 7f21c7fff700 1 --2- 172.21.15.3:0/1429224320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21c806f050 0x7f21c813d120 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f21b4009c40 tx=0x7f21b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:31.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.537+0000 7f21c57fa700 1 -- 172.21.15.3:0/1429224320 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b40045d0 con 0x7f21c806f050 2023-10-03T10:13:31.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.537+0000 7f21c57fa700 1 -- 172.21.15.3:0/1429224320 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b40036a0 con 0x7f21c806f050 2023-10-03T10:13:31.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.537+0000 7f21cf129700 1 -- 172.21.15.3:0/1429224320 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21b4009550 con 0x7f21c806f050 2023-10-03T10:13:31.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.537+0000 7f21c57fa700 1 -- 172.21.15.3:0/1429224320 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b4003a30 con 0x7f21c806f050 2023-10-03T10:13:31.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.537+0000 7f21cf129700 1 -- 172.21.15.3:0/1429224320 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c8070610 con 0x7f21c806f050 2023-10-03T10:13:31.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.538+0000 7f21c57fa700 1 -- 172.21.15.3:0/1429224320 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f21b4003b90 con 0x7f21c806f050 2023-10-03T10:13:31.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.538+0000 7f21c57fa700 1 -- 172.21.15.3:0/1429224320 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f21b4007e30 con 0x7f21c806f050 2023-10-03T10:13:31.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.538+0000 7f21cf129700 1 -- 172.21.15.3:0/1429224320 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c8074ea0 con 0x7f21c806f050 2023-10-03T10:13:31.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.544+0000 7f21c57fa700 1 -- 172.21.15.3:0/1429224320 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f21b4028020 con 0x7f21c806f050 2023-10-03T10:13:31.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.549+0000 7f4af4f74700 1 -- 172.21.15.3:0/2213280050 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4af006b250 con 0x7f4af0134260 2023-10-03T10:13:31.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.549+0000 7f4ae6ffd700 1 -- 172.21.15.3:0/2213280050 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4ae802c090 con 0x7f4af0134260 2023-10-03T10:13:31.551 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:31.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.552+0000 7f4af4f74700 1 -- 172.21.15.3:0/2213280050 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4af0134260 msgr2=0x7f4af009cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:31.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.552+0000 7f4af4f74700 1 --2- 172.21.15.3:0/2213280050 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4af0134260 0x7f4af009cf00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009eb0 tx=0x7f4ae800c3a0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.552+0000 7f4af4f74700 1 -- 172.21.15.3:0/2213280050 shutdown_connections 2023-10-03T10:13:31.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.552+0000 7f4af4f74700 1 --2- 172.21.15.3:0/2213280050 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4af0134260 0x7f4af009cf00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.552+0000 7f4af4f74700 1 --2- 172.21.15.3:0/2213280050 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4af0133420 0x7f4af009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.552+0000 7f4af4f74700 1 --2- 172.21.15.3:0/2213280050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af0133e30 0x7f4af009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.552+0000 7f4af4f74700 1 -- 172.21.15.3:0/2213280050 >> 172.21.15.3:0/2213280050 conn(0x7f4af00a4210 msgr2=0x7f4af00a2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:31.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.552+0000 7f4af4f74700 1 -- 172.21.15.3:0/2213280050 shutdown_connections 2023-10-03T10:13:31.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.552+0000 7f4af4f74700 1 -- 172.21.15.3:0/2213280050 wait complete. 2023-10-03T10:13:31.569 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:31.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.581+0000 7f240d0a4700 1 Processor -- start 2023-10-03T10:13:31.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.581+0000 7f240d0a4700 1 -- start start 2023-10-03T10:13:31.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.582+0000 7f240d0a4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f240809e930 0x7f240809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:31.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.582+0000 7f240d0a4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f240809f240 0x7f24081317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:31.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.582+0000 7f240d0a4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2408070940 con 0x7f240809f650 2023-10-03T10:13:31.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.582+0000 7f240d0a4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f240806fd60 con 0x7f240809e930 2023-10-03T10:13:31.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.582+0000 7f240d0a4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f240806fa80 con 0x7f240809f240 2023-10-03T10:13:31.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.582+0000 7f240659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f240809f240 0x7f24081317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:31.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.582+0000 7f2406d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f240809e930 0x7f240809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:31.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.582+0000 7f2406d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f240809e930 0x7f240809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53922/0 (socket says 172.21.15.3:53922) 2023-10-03T10:13:31.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.582+0000 7f240659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f240809f240 0x7f24081317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:57006/0 (socket says 172.21.15.3:57006) 2023-10-03T10:13:31.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.582+0000 7f2406d9d700 1 -- 172.21.15.3:0/3608820497 learned_addr learned my addr 172.21.15.3:0/3608820497 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:31.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.582+0000 7f240759e700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f240809f650 0x7f2408131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48870/0 (socket says 172.21.15.3:48870) 2023-10-03T10:13:31.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.582+0000 7f2405d9b700 1 -- 172.21.15.3:0/3608820497 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784051767 0 0) 0x7f2408070940 con 0x7f240809f650 2023-10-03T10:13:31.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.582+0000 7f240659c700 1 -- 172.21.15.3:0/3608820497 >> v2:172.21.15.3:3300/0 conn(0x7f240809e930 msgr2=0x7f240809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:31.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.582+0000 7f240659c700 1 --2- 172.21.15.3:0/3608820497 >> v2:172.21.15.3:3300/0 conn(0x7f240809e930 0x7f240809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.582+0000 7f240659c700 1 -- 172.21.15.3:0/3608820497 >> v1:172.21.15.3:6789/0 conn(0x7f240809f650 legacy=0x7f2408131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:31.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.582+0000 7f240659c700 1 -- 172.21.15.3:0/3608820497 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f240809fb60 con 0x7f240809f240 2023-10-03T10:13:31.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.582+0000 7f2406d9d700 1 --2- 172.21.15.3:0/3608820497 >> v2:172.21.15.3:3300/0 conn(0x7f240809e930 0x7f240809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:31.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.583+0000 7f240659c700 1 --2- 172.21.15.3:0/3608820497 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f240809f240 0x7f24081317f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00dd90 tx=0x7f23fc00f330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81cefc9b711aa67f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:31.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.583+0000 7f2405d9b700 1 -- 172.21.15.3:0/3608820497 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc012070 con 0x7f240809f240 2023-10-03T10:13:31.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.583+0000 7f2405d9b700 1 -- 172.21.15.3:0/3608820497 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23fc00fb20 con 0x7f240809f240 2023-10-03T10:13:31.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.583+0000 7f2405d9b700 1 -- 172.21.15.3:0/3608820497 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc016430 con 0x7f240809f240 2023-10-03T10:13:31.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.583+0000 7f240d0a4700 1 -- 172.21.15.3:0/3608820497 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f240809f240 msgr2=0x7f24081317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:31.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.583+0000 7f240d0a4700 1 --2- 172.21.15.3:0/3608820497 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f240809f240 0x7f24081317f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00dd90 tx=0x7f23fc00f330 comp rx=0 tx=0).stop 2023-10-03T10:13:31.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.584+0000 7f240d0a4700 1 -- 172.21.15.3:0/3608820497 shutdown_connections 2023-10-03T10:13:31.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.584+0000 7f240d0a4700 1 --2- 172.21.15.3:0/3608820497 >> v2:172.21.15.3:3300/0 conn(0x7f240809e930 0x7f240809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.584+0000 7f240d0a4700 1 --2- 172.21.15.3:0/3608820497 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f240809f240 0x7f24081317f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.584+0000 7f240d0a4700 1 -- 172.21.15.3:0/3608820497 >> 172.21.15.3:0/3608820497 conn(0x7f24080a4210 msgr2=0x7f24080a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:31.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.584+0000 7f240d0a4700 1 -- 172.21.15.3:0/3608820497 shutdown_connections 2023-10-03T10:13:31.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.584+0000 7f240d0a4700 1 -- 172.21.15.3:0/3608820497 wait complete. 2023-10-03T10:13:31.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.585+0000 7f240d0a4700 1 Processor -- start 2023-10-03T10:13:31.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.585+0000 7f240d0a4700 1 -- start start 2023-10-03T10:13:31.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.586+0000 7f240d0a4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f240809e930 0x7f24081b4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:31.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.586+0000 7f240d0a4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f240809f240 0x7f24081b4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:31.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.586+0000 7f240d0a4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f240809f650 0x7f24081a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:31.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.586+0000 7f240d0a4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24081a2040 con 0x7f240809f240 2023-10-03T10:13:31.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.586+0000 7f240d0a4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24080a30d0 con 0x7f240809f650 2023-10-03T10:13:31.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.586+0000 7f240d0a4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24081a4170 con 0x7f240809e930 2023-10-03T10:13:31.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.586+0000 7f2406d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f240809e930 0x7f24081b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:31.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.586+0000 7f240659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f240809f240 0x7f24081b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:31.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.586+0000 7f240659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f240809f240 0x7f24081b4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53928/0 (socket says 172.21.15.3:53928) 2023-10-03T10:13:31.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.586+0000 7f2406d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f240809e930 0x7f24081b4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54166/0 (socket says 172.21.15.3:54166) 2023-10-03T10:13:31.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.586+0000 7f240659c700 1 -- 172.21.15.3:0/242470188 learned_addr learned my addr 172.21.15.3:0/242470188 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:31.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.586+0000 7f240759e700 1 --2- 172.21.15.3:0/242470188 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f240809f650 0x7f24081a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:31.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.586+0000 7f240659c700 1 -- 172.21.15.3:0/242470188 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f240809e930 msgr2=0x7f24081b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:31.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.586+0000 7f240659c700 1 --2- 172.21.15.3:0/242470188 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f240809e930 0x7f24081b4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.586+0000 7f240659c700 1 -- 172.21.15.3:0/242470188 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f240809f650 msgr2=0x7f24081a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:31.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.586+0000 7f240659c700 1 --2- 172.21.15.3:0/242470188 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f240809f650 0x7f24081a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.586+0000 7f240659c700 1 -- 172.21.15.3:0/242470188 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f8009550 con 0x7f240809f240 2023-10-03T10:13:31.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.586+0000 7f2406d9d700 1 --2- 172.21.15.3:0/242470188 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f240809e930 0x7f24081b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:31.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.586+0000 7f240759e700 1 --2- 172.21.15.3:0/242470188 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f240809f650 0x7f24081a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:31.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.587+0000 7f240659c700 1 --2- 172.21.15.3:0/242470188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f240809f240 0x7f24081b4770 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f23fc018040 tx=0x7f23fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:31.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.587+0000 7f23ef7fe700 1 -- 172.21.15.3:0/242470188 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc016430 con 0x7f240809f240 2023-10-03T10:13:31.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.587+0000 7f23ef7fe700 1 -- 172.21.15.3:0/242470188 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23fc024070 con 0x7f240809f240 2023-10-03T10:13:31.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.587+0000 7f23ef7fe700 1 -- 172.21.15.3:0/242470188 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc003d10 con 0x7f240809f240 2023-10-03T10:13:31.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.587+0000 7f240d0a4700 1 -- 172.21.15.3:0/242470188 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23fc00da70 con 0x7f240809f240 2023-10-03T10:13:31.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.587+0000 7f240d0a4700 1 -- 172.21.15.3:0/242470188 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24080a0190 con 0x7f240809f240 2023-10-03T10:13:31.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.588+0000 7f23ef7fe700 1 -- 172.21.15.3:0/242470188 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f23fc016680 con 0x7f240809f240 2023-10-03T10:13:31.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.588+0000 7f240d0a4700 1 -- 172.21.15.3:0/242470188 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2408074ea0 con 0x7f240809f240 2023-10-03T10:13:31.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.588+0000 7f23ef7fe700 1 -- 172.21.15.3:0/242470188 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f23fc026070 con 0x7f240809f240 2023-10-03T10:13:31.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.597+0000 7f23ef7fe700 1 -- 172.21.15.3:0/242470188 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f23fc030020 con 0x7f240809f240 2023-10-03T10:13:31.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.711+0000 7fe5d2c57700 1 -- 172.21.15.3:0/1935145083 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe5cc0a28e0 con 0x7fe5cc0a0a30 2023-10-03T10:13:31.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.712+0000 7fe5c97fa700 1 -- 172.21.15.3:0/1935145083 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe5c0017020 con 0x7fe5cc0a0a30 2023-10-03T10:13:31.712 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:31.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.714+0000 7fe5d2c57700 1 -- 172.21.15.3:0/1935145083 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5cc0a0a30 msgr2=0x7fe5cc11e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:31.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.714+0000 7fe5d2c57700 1 --2- 172.21.15.3:0/1935145083 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5cc0a0a30 0x7fe5cc11e6a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe5c000ebe0 tx=0x7fe5c000eef0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.714+0000 7fe5d2c57700 1 -- 172.21.15.3:0/1935145083 shutdown_connections 2023-10-03T10:13:31.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.714+0000 7fe5d2c57700 1 --2- 172.21.15.3:0/1935145083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5cc134170 0x7fe5cc11f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.714+0000 7fe5d2c57700 1 --2- 172.21.15.3:0/1935145083 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5cc0a0a30 0x7fe5cc11e6a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.714+0000 7fe5d2c57700 1 --2- 172.21.15.3:0/1935145083 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5cc0a1340 0x7fe5cc11ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.714+0000 7fe5d2c57700 1 -- 172.21.15.3:0/1935145083 >> 172.21.15.3:0/1935145083 conn(0x7fe5cc0a6550 msgr2=0x7fe5cc09ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:31.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.714+0000 7fe5d2c57700 1 -- 172.21.15.3:0/1935145083 shutdown_connections 2023-10-03T10:13:31.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.714+0000 7fe5d2c57700 1 -- 172.21.15.3:0/1935145083 wait complete. 2023-10-03T10:13:31.728 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:31.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.757+0000 7f21cf129700 1 -- 172.21.15.3:0/1429224320 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f21c806b9e0 con 0x7f21c806f050 2023-10-03T10:13:31.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.758+0000 7f21c57fa700 1 -- 172.21.15.3:0/1429224320 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f21b4015070 con 0x7f21c806f050 2023-10-03T10:13:31.760 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:31.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.760+0000 7f21cf129700 1 -- 172.21.15.3:0/1429224320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21c806f050 msgr2=0x7f21c813d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:31.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.760+0000 7f21cf129700 1 --2- 172.21.15.3:0/1429224320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21c806f050 0x7f21c813d120 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f21b4009c40 tx=0x7f21b4004750 comp rx=0 tx=0).stop 2023-10-03T10:13:31.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.760+0000 7f21cf129700 1 -- 172.21.15.3:0/1429224320 shutdown_connections 2023-10-03T10:13:31.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.760+0000 7f21cf129700 1 --2- 172.21.15.3:0/1429224320 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21c812e180 0x7f21c813d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.760+0000 7f21cf129700 1 --2- 172.21.15.3:0/1429224320 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21c806e570 0x7f21c813cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.760+0000 7f21cf129700 1 --2- 172.21.15.3:0/1429224320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21c806f050 0x7f21c813d120 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.760+0000 7f21cf129700 1 -- 172.21.15.3:0/1429224320 >> 172.21.15.3:0/1429224320 conn(0x7f21c80a2aa0 msgr2=0x7f21c809cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:31.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.760+0000 7f21cf129700 1 -- 172.21.15.3:0/1429224320 shutdown_connections 2023-10-03T10:13:31.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.760+0000 7f21cf129700 1 -- 172.21.15.3:0/1429224320 wait complete. 2023-10-03T10:13:31.774 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:31.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.804+0000 7f240d0a4700 1 -- 172.21.15.3:0/242470188 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f24081a2830 con 0x7f240809f240 2023-10-03T10:13:31.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.804+0000 7f23ef7fe700 1 -- 172.21.15.3:0/242470188 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f23fc039090 con 0x7f240809f240 2023-10-03T10:13:31.805 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.806+0000 7f240d0a4700 1 -- 172.21.15.3:0/242470188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f240809f240 msgr2=0x7f24081b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:31.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.807+0000 7f240d0a4700 1 --2- 172.21.15.3:0/242470188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f240809f240 0x7f24081b4770 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f23fc018040 tx=0x7f23fc004750 comp rx=0 tx=0).stop 2023-10-03T10:13:31.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.807+0000 7f240d0a4700 1 -- 172.21.15.3:0/242470188 shutdown_connections 2023-10-03T10:13:31.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.807+0000 7f240d0a4700 1 --2- 172.21.15.3:0/242470188 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f240809f650 0x7f24081a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.807+0000 7f240d0a4700 1 --2- 172.21.15.3:0/242470188 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f240809e930 0x7f24081b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.807+0000 7f240d0a4700 1 --2- 172.21.15.3:0/242470188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f240809f240 0x7f24081b4770 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:31.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.807+0000 7f240d0a4700 1 -- 172.21.15.3:0/242470188 >> 172.21.15.3:0/242470188 conn(0x7f24080a4210 msgr2=0x7f24081b7890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:31.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.807+0000 7f240d0a4700 1 -- 172.21.15.3:0/242470188 shutdown_connections 2023-10-03T10:13:31.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:31.807+0000 7f240d0a4700 1 -- 172.21.15.3:0/242470188 wait complete. 2023-10-03T10:13:31.821 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:32.570 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:32.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.717+0000 7f7349bf0700 1 Processor -- start 2023-10-03T10:13:32.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.717+0000 7f7349bf0700 1 -- start start 2023-10-03T10:13:32.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.718+0000 7f7349bf0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f73441338a0 0x7f7344133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:32.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.718+0000 7f7349bf0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f734409e930 0x7f734413b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:32.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.718+0000 7f7349bf0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f734406ef30 con 0x7f734409de50 2023-10-03T10:13:32.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.718+0000 7f7349bf0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f734406fef0 con 0x7f73441338a0 2023-10-03T10:13:32.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.718+0000 7f7349bf0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73441334b0 con 0x7f734409e930 2023-10-03T10:13:32.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.718+0000 7f7343fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f734409e930 0x7f734413b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:32.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.718+0000 7f7343fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f734409e930 0x7f734413b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:57026/0 (socket says 172.21.15.3:57026) 2023-10-03T10:13:32.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.718+0000 7f7343fff700 1 -- 172.21.15.3:0/2540476844 learned_addr learned my addr 172.21.15.3:0/2540476844 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:32.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.718+0000 7f73437fe700 1 --2- 172.21.15.3:0/2540476844 >> v2:172.21.15.3:3300/0 conn(0x7f73441338a0 0x7f7344133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:32.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.718+0000 7f7343fff700 1 -- 172.21.15.3:0/2540476844 >> v2:172.21.15.3:3300/0 conn(0x7f73441338a0 msgr2=0x7f7344133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:32.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.718+0000 7f7343fff700 1 --2- 172.21.15.3:0/2540476844 >> v2:172.21.15.3:3300/0 conn(0x7f73441338a0 0x7f7344133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:32.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.718+0000 7f7343fff700 1 -- 172.21.15.3:0/2540476844 >> v1:172.21.15.3:6789/0 conn(0x7f734409de50 legacy=0x7f734409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:32.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.718+0000 7f7343fff700 1 -- 172.21.15.3:0/2540476844 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f734406fc30 con 0x7f734409e930 2023-10-03T10:13:32.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.718+0000 7f73437fe700 1 --2- 172.21.15.3:0/2540476844 >> v2:172.21.15.3:3300/0 conn(0x7f73441338a0 0x7f7344133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:32.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.719+0000 7f7343fff700 1 --2- 172.21.15.3:0/2540476844 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f734409e930 0x7f734413b740 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f733c009e50 tx=0x7f733c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85bd670d147dc57c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:32.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.719+0000 7f73427fc700 1 -- 172.21.15.3:0/2540476844 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733c00e070 con 0x7f734409e930 2023-10-03T10:13:32.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.719+0000 7f73427fc700 1 -- 172.21.15.3:0/2540476844 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733c00b910 con 0x7f734409e930 2023-10-03T10:13:32.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.719+0000 7f73427fc700 1 -- 172.21.15.3:0/2540476844 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733c00bcc0 con 0x7f734409e930 2023-10-03T10:13:32.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.719+0000 7f7349bf0700 1 -- 172.21.15.3:0/2540476844 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f734409e930 msgr2=0x7f734413b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:32.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.719+0000 7f7349bf0700 1 --2- 172.21.15.3:0/2540476844 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f734409e930 0x7f734413b740 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f733c009e50 tx=0x7f733c00b330 comp rx=0 tx=0).stop 2023-10-03T10:13:32.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.719+0000 7f7349bf0700 1 -- 172.21.15.3:0/2540476844 shutdown_connections 2023-10-03T10:13:32.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.719+0000 7f7349bf0700 1 --2- 172.21.15.3:0/2540476844 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f734409e930 0x7f734413b740 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:32.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.719+0000 7f7349bf0700 1 --2- 172.21.15.3:0/2540476844 >> v2:172.21.15.3:3300/0 conn(0x7f73441338a0 0x7f7344133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:32.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.719+0000 7f7349bf0700 1 -- 172.21.15.3:0/2540476844 >> 172.21.15.3:0/2540476844 conn(0x7f73440a2aa0 msgr2=0x7f73440a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:32.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.719+0000 7f7349bf0700 1 -- 172.21.15.3:0/2540476844 shutdown_connections 2023-10-03T10:13:32.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.719+0000 7f7349bf0700 1 -- 172.21.15.3:0/2540476844 wait complete. 2023-10-03T10:13:32.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.720+0000 7f7349bf0700 1 Processor -- start 2023-10-03T10:13:32.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.721+0000 7f7349bf0700 1 -- start start 2023-10-03T10:13:32.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.721+0000 7f7349bf0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f734409de50 0x7f734411d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:32.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.721+0000 7f7349bf0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f734409e930 0x7f734411dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:32.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.721+0000 7f7349bf0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73441338a0 0x7f734411e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:32.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.721+0000 7f7349bf0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7344121260 con 0x7f734409e930 2023-10-03T10:13:32.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.721+0000 7f7349bf0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734411f930 con 0x7f734409de50 2023-10-03T10:13:32.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.721+0000 7f7349bf0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73441209f0 con 0x7f73441338a0 2023-10-03T10:13:32.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.721+0000 7f73437fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f734409de50 0x7f734411d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:32.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.721+0000 7f73437fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f734409de50 0x7f734411d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:57036/0 (socket says 172.21.15.3:57036) 2023-10-03T10:13:32.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.721+0000 7f73437fe700 1 -- 172.21.15.3:0/4236609865 learned_addr learned my addr 172.21.15.3:0/4236609865 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:32.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.721+0000 7f7342ffd700 1 --2- 172.21.15.3:0/4236609865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f734409e930 0x7f734411dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:32.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.721+0000 7f7343fff700 1 --2- 172.21.15.3:0/4236609865 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73441338a0 0x7f734411e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:32.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.722+0000 7f7342ffd700 1 -- 172.21.15.3:0/4236609865 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73441338a0 msgr2=0x7f734411e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:32.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.722+0000 7f7342ffd700 1 --2- 172.21.15.3:0/4236609865 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73441338a0 0x7f734411e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:32.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.722+0000 7f7342ffd700 1 -- 172.21.15.3:0/4236609865 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f734409de50 msgr2=0x7f734411d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:32.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.722+0000 7f7342ffd700 1 --2- 172.21.15.3:0/4236609865 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f734409de50 0x7f734411d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:32.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.722+0000 7f7342ffd700 1 -- 172.21.15.3:0/4236609865 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73380094a0 con 0x7f734409e930 2023-10-03T10:13:32.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.722+0000 7f73437fe700 1 --2- 172.21.15.3:0/4236609865 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f734409de50 0x7f734411d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:32.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.722+0000 7f7342ffd700 1 --2- 172.21.15.3:0/4236609865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f734409e930 0x7f734411dcd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f733400c400 tx=0x7f733400c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:32.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.722+0000 7f732bfff700 1 -- 172.21.15.3:0/4236609865 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73340052d0 con 0x7f734409e930 2023-10-03T10:13:32.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.722+0000 7f732bfff700 1 -- 172.21.15.3:0/4236609865 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7334005430 con 0x7f734409e930 2023-10-03T10:13:32.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.722+0000 7f7349bf0700 1 -- 172.21.15.3:0/4236609865 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f733c009b30 con 0x7f734409e930 2023-10-03T10:13:32.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.722+0000 7f732bfff700 1 -- 172.21.15.3:0/4236609865 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73340038a0 con 0x7f734409e930 2023-10-03T10:13:32.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.722+0000 7f7349bf0700 1 -- 172.21.15.3:0/4236609865 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f734406f690 con 0x7f734409e930 2023-10-03T10:13:32.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.723+0000 7f732bfff700 1 -- 172.21.15.3:0/4236609865 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7334003a00 con 0x7f734409e930 2023-10-03T10:13:32.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.723+0000 7f7349bf0700 1 -- 172.21.15.3:0/4236609865 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7344120de0 con 0x7f734409e930 2023-10-03T10:13:32.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.723+0000 7f732bfff700 1 -- 172.21.15.3:0/4236609865 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f733403e910 con 0x7f734409e930 2023-10-03T10:13:32.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.728+0000 7f732bfff700 1 -- 172.21.15.3:0/4236609865 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7344120de0 con 0x7f734409e930 2023-10-03T10:13:32.730 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:32.775 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:32.822 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:32.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.883+0000 7fcb1356a700 1 Processor -- start 2023-10-03T10:13:32.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.883+0000 7fcb1356a700 1 -- start start 2023-10-03T10:13:32.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.884+0000 7fcb1356a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb0c12cf60 0x7fcb0c06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:32.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.884+0000 7fcb1356a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb0c12f8b0 0x7fcb0c12fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:32.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.884+0000 7fcb1356a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcb0c130260 0x7fcb0c136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:32.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.884+0000 7fcb1356a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb0c12d330 con 0x7fcb0c130260 2023-10-03T10:13:32.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.884+0000 7fcb1356a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c12cd90 con 0x7fcb0c12f8b0 2023-10-03T10:13:32.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.884+0000 7fcb1356a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c1306d0 con 0x7fcb0c12cf60 2023-10-03T10:13:32.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.884+0000 7fcb10b05700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb0c12f8b0 0x7fcb0c12fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:32.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.884+0000 7fcb11306700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb0c12cf60 0x7fcb0c06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:32.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.884+0000 7fcb11b07700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcb0c130260 0x7fcb0c136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:32.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.884+0000 7fcb10b05700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb0c12f8b0 0x7fcb0c12fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:57052/0 (socket says 172.21.15.3:57052) 2023-10-03T10:13:32.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.884+0000 7fcb10b05700 1 -- 172.21.15.3:0/2239781595 learned_addr learned my addr 172.21.15.3:0/2239781595 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:32.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.885+0000 7fcb11306700 1 -- 172.21.15.3:0/2239781595 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb0c12f8b0 msgr2=0x7fcb0c12fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:32.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.885+0000 7fcb11306700 1 --2- 172.21.15.3:0/2239781595 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb0c12f8b0 0x7fcb0c12fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:32.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.885+0000 7fcb11306700 1 -- 172.21.15.3:0/2239781595 >> v2:172.21.15.3:3300/0 conn(0x7fcb0c130260 msgr2=0x7fcb0c136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:32.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.885+0000 7fcb11306700 1 --2- 172.21.15.3:0/2239781595 >> v2:172.21.15.3:3300/0 conn(0x7fcb0c130260 0x7fcb0c136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:32.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.885+0000 7fcb11306700 1 -- 172.21.15.3:0/2239781595 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0c071780 con 0x7fcb0c12cf60 2023-10-03T10:13:32.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.885+0000 7fcb10b05700 1 --2- 172.21.15.3:0/2239781595 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb0c12f8b0 0x7fcb0c12fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:32.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.885+0000 7fcb11306700 1 --2- 172.21.15.3:0/2239781595 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb0c12cf60 0x7fcb0c06f5d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009840 tx=0x7fcb08009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36b8c9a260992be7 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:32.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.885+0000 7fcb03fff700 1 -- 172.21.15.3:0/2239781595 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb08011830 con 0x7fcb0c12cf60 2023-10-03T10:13:32.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.885+0000 7fcb03fff700 1 -- 172.21.15.3:0/2239781595 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb08011990 con 0x7fcb0c12cf60 2023-10-03T10:13:32.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.885+0000 7fcb03fff700 1 -- 172.21.15.3:0/2239781595 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb08011d20 con 0x7fcb0c12cf60 2023-10-03T10:13:32.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.886+0000 7fcb1356a700 1 -- 172.21.15.3:0/2239781595 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb0c12cf60 msgr2=0x7fcb0c06f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:32.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.886+0000 7fcb1356a700 1 --2- 172.21.15.3:0/2239781595 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb0c12cf60 0x7fcb0c06f5d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009840 tx=0x7fcb08009b50 comp rx=0 tx=0).stop 2023-10-03T10:13:32.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.886+0000 7fcb1356a700 1 -- 172.21.15.3:0/2239781595 shutdown_connections 2023-10-03T10:13:32.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.886+0000 7fcb1356a700 1 --2- 172.21.15.3:0/2239781595 >> v2:172.21.15.3:3300/0 conn(0x7fcb0c130260 0x7fcb0c136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:32.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.886+0000 7fcb1356a700 1 --2- 172.21.15.3:0/2239781595 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb0c12cf60 0x7fcb0c06f5d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:32.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.886+0000 7fcb1356a700 1 --2- 172.21.15.3:0/2239781595 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb0c12f8b0 0x7fcb0c12fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:32.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.886+0000 7fcb1356a700 1 -- 172.21.15.3:0/2239781595 >> 172.21.15.3:0/2239781595 conn(0x7fcb0c0a2aa0 msgr2=0x7fcb0c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:32.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.887+0000 7fcb1356a700 1 -- 172.21.15.3:0/2239781595 shutdown_connections 2023-10-03T10:13:32.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.887+0000 7fcb1356a700 1 -- 172.21.15.3:0/2239781595 wait complete. 2023-10-03T10:13:32.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.888+0000 7fcb1356a700 1 Processor -- start 2023-10-03T10:13:32.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.888+0000 7fcb1356a700 1 -- start start 2023-10-03T10:13:32.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.888+0000 7fcb1356a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb0c12cf60 0x7fcb0c09bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:32.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.888+0000 7fcb1356a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb0c12f8b0 0x7fcb0c09c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:32.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.888+0000 7fcb1356a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb0c130260 0x7fcb0c1c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:32.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.888+0000 7fcb1356a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c09b7b0 con 0x7fcb0c12cf60 2023-10-03T10:13:32.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.888+0000 7fcb1356a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c12d9b0 con 0x7fcb0c130260 2023-10-03T10:13:32.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.888+0000 7fcb1356a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c129fa0 con 0x7fcb0c12f8b0 2023-10-03T10:13:32.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.888+0000 7fcb10b05700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb0c12f8b0 0x7fcb0c09c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:32.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.888+0000 7fcb11b07700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb0c130260 0x7fcb0c1c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:32.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.888+0000 7fcb10b05700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb0c12f8b0 0x7fcb0c09c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54174/0 (socket says 172.21.15.3:54174) 2023-10-03T10:13:32.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.888+0000 7fcb10b05700 1 -- 172.21.15.3:0/864148700 learned_addr learned my addr 172.21.15.3:0/864148700 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:32.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.888+0000 7fcb11b07700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb0c130260 0x7fcb0c1c40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:57064/0 (socket says 172.21.15.3:57064) 2023-10-03T10:13:32.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.889+0000 7fcb11306700 1 --2- 172.21.15.3:0/864148700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb0c12cf60 0x7fcb0c09bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:32.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.889+0000 7fcb10b05700 1 -- 172.21.15.3:0/864148700 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb0c130260 msgr2=0x7fcb0c1c40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:32.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.889+0000 7fcb10b05700 1 --2- 172.21.15.3:0/864148700 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb0c130260 0x7fcb0c1c40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:32.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.889+0000 7fcb10b05700 1 -- 172.21.15.3:0/864148700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb0c12cf60 msgr2=0x7fcb0c09bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:32.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.889+0000 7fcb10b05700 1 --2- 172.21.15.3:0/864148700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb0c12cf60 0x7fcb0c09bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:32.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.889+0000 7fcb10b05700 1 -- 172.21.15.3:0/864148700 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcafc00a6f0 con 0x7fcb0c12f8b0 2023-10-03T10:13:32.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.889+0000 7fcb10b05700 1 --2- 172.21.15.3:0/864148700 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb0c12f8b0 0x7fcb0c09c170 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcafc00e930 tx=0x7fcafc00ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:32.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.889+0000 7fcb01ffb700 1 -- 172.21.15.3:0/864148700 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc004440 con 0x7fcb0c12f8b0 2023-10-03T10:13:32.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.889+0000 7fcb01ffb700 1 -- 172.21.15.3:0/864148700 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcafc0045a0 con 0x7fcb0c12f8b0 2023-10-03T10:13:32.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.889+0000 7fcb01ffb700 1 -- 172.21.15.3:0/864148700 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc0038a0 con 0x7fcb0c12f8b0 2023-10-03T10:13:32.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.890+0000 7fcb1356a700 1 -- 172.21.15.3:0/864148700 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb080094a0 con 0x7fcb0c12f8b0 2023-10-03T10:13:32.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.890+0000 7fcb1356a700 1 -- 172.21.15.3:0/864148700 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb0c09c6b0 con 0x7fcb0c12f8b0 2023-10-03T10:13:32.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.891+0000 7fcb01ffb700 1 -- 172.21.15.3:0/864148700 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcafc003a00 con 0x7fcb0c12f8b0 2023-10-03T10:13:32.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.892+0000 7fcb1356a700 1 -- 172.21.15.3:0/864148700 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb0c06fbd0 con 0x7fcb0c12f8b0 2023-10-03T10:13:32.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.892+0000 7fcb01ffb700 1 -- 172.21.15.3:0/864148700 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fcafc03f5a0 con 0x7fcb0c12f8b0 2023-10-03T10:13:32.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.897+0000 7fcb01ffb700 1 -- 172.21.15.3:0/864148700 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcafc010b80 con 0x7fcb0c12f8b0 2023-10-03T10:13:32.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.922+0000 7fb18c504700 1 Processor -- start 2023-10-03T10:13:32.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.922+0000 7fb18c504700 1 -- start start 2023-10-03T10:13:32.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.923+0000 7fb18c504700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb18409e1d0 0x7fb184135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:32.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.923+0000 7fb18c504700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb184135e50 0x7fb184138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:32.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.923+0000 7fb18c504700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb18409f130 con 0x7fb18409d6f0 2023-10-03T10:13:32.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.923+0000 7fb18c504700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18409f520 con 0x7fb184135e50 2023-10-03T10:13:32.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.923+0000 7fb18c504700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb18409e930 con 0x7fb18409e1d0 2023-10-03T10:13:32.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.924+0000 7fb18a2a0700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb18409d6f0 0x7fb18409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48882/0 (socket says 172.21.15.3:48882) 2023-10-03T10:13:32.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.924+0000 7fb18a2a0700 1 -- 172.21.15.3:0/2129505672 learned_addr learned my addr 172.21.15.3:0/2129505672 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:32.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.924+0000 7fb18aaa1700 1 --2- 172.21.15.3:0/2129505672 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb184135e50 0x7fb184138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:32.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.924+0000 7fb189a9f700 1 --2- 172.21.15.3:0/2129505672 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb18409e1d0 0x7fb184135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:32.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.924+0000 7fb18929e700 1 -- 172.21.15.3:0/2129505672 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1523373486 0 0) 0x7fb18409f130 con 0x7fb18409d6f0 2023-10-03T10:13:32.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.924+0000 7fb18929e700 1 -- 172.21.15.3:0/2129505672 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1700035b0 con 0x7fb18409d6f0 2023-10-03T10:13:32.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.924+0000 7fb189a9f700 1 -- 172.21.15.3:0/2129505672 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb184135e50 msgr2=0x7fb184138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:32.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.924+0000 7fb189a9f700 1 --2- 172.21.15.3:0/2129505672 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb184135e50 0x7fb184138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:32.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.924+0000 7fb189a9f700 1 -- 172.21.15.3:0/2129505672 >> v1:172.21.15.3:6789/0 conn(0x7fb18409d6f0 legacy=0x7fb18409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:32.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.924+0000 7fb189a9f700 1 -- 172.21.15.3:0/2129505672 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18409b2d0 con 0x7fb18409e1d0 2023-10-03T10:13:32.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.924+0000 7fb18aaa1700 1 --2- 172.21.15.3:0/2129505672 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb184135e50 0x7fb184138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:32.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.924+0000 7fb189a9f700 1 --2- 172.21.15.3:0/2129505672 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb18409e1d0 0x7fb184135910 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb174009870 tx=0x7fb174009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef0fffa94a776ada server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:32.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.925+0000 7fb18929e700 1 -- 172.21.15.3:0/2129505672 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb174011830 con 0x7fb18409e1d0 2023-10-03T10:13:32.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.925+0000 7fb18929e700 1 -- 172.21.15.3:0/2129505672 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb174011990 con 0x7fb18409e1d0 2023-10-03T10:13:32.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.925+0000 7fb18929e700 1 -- 172.21.15.3:0/2129505672 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb174011d00 con 0x7fb18409e1d0 2023-10-03T10:13:32.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.926+0000 7fb18c504700 1 -- 172.21.15.3:0/2129505672 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb18409e1d0 msgr2=0x7fb184135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:32.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.926+0000 7fb18c504700 1 --2- 172.21.15.3:0/2129505672 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb18409e1d0 0x7fb184135910 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb174009870 tx=0x7fb174009b80 comp rx=0 tx=0).stop 2023-10-03T10:13:32.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.926+0000 7fb18c504700 1 -- 172.21.15.3:0/2129505672 shutdown_connections 2023-10-03T10:13:32.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.926+0000 7fb18c504700 1 --2- 172.21.15.3:0/2129505672 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb184135e50 0x7fb184138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:32.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.926+0000 7fb18c504700 1 --2- 172.21.15.3:0/2129505672 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb18409e1d0 0x7fb184135910 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:32.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.926+0000 7fb18c504700 1 -- 172.21.15.3:0/2129505672 >> 172.21.15.3:0/2129505672 conn(0x7fb184066f50 msgr2=0x7fb18409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:32.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.926+0000 7fb18c504700 1 -- 172.21.15.3:0/2129505672 shutdown_connections 2023-10-03T10:13:32.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.926+0000 7fb18c504700 1 -- 172.21.15.3:0/2129505672 wait complete. 2023-10-03T10:13:32.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.927+0000 7fb18c504700 1 Processor -- start 2023-10-03T10:13:32.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.928+0000 7fb18c504700 1 -- start start 2023-10-03T10:13:32.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.928+0000 7fb18c504700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb18409d6f0 0x7fb18411b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:32.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.928+0000 7fb18c504700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb18409e1d0 0x7fb18411bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:32.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.928+0000 7fb18c504700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb184135e50 0x7fb18411c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:32.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.928+0000 7fb18c504700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb184135ad0 con 0x7fb18409d6f0 2023-10-03T10:13:32.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.928+0000 7fb18c504700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18411e4d0 con 0x7fb18409e1d0 2023-10-03T10:13:32.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.928+0000 7fb18c504700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb18411dae0 con 0x7fb184135e50 2023-10-03T10:13:32.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.928+0000 7fb18a2a0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb18409d6f0 0x7fb18411b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:32.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.928+0000 7fb18a2a0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb18409d6f0 0x7fb18411b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53972/0 (socket says 172.21.15.3:53972) 2023-10-03T10:13:32.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.928+0000 7fb18a2a0700 1 -- 172.21.15.3:0/3993139514 learned_addr learned my addr 172.21.15.3:0/3993139514 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:32.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.929+0000 7fb18aaa1700 1 --2- 172.21.15.3:0/3993139514 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb184135e50 0x7fb18411c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:32.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.929+0000 7fb189a9f700 1 --2- 172.21.15.3:0/3993139514 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb18409e1d0 0x7fb18411bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:32.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.929+0000 7fb18a2a0700 1 -- 172.21.15.3:0/3993139514 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb184135e50 msgr2=0x7fb18411c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:32.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.929+0000 7fb18a2a0700 1 --2- 172.21.15.3:0/3993139514 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb184135e50 0x7fb18411c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:32.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.929+0000 7fb18a2a0700 1 -- 172.21.15.3:0/3993139514 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb18409e1d0 msgr2=0x7fb18411bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:32.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.929+0000 7fb18a2a0700 1 --2- 172.21.15.3:0/3993139514 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb18409e1d0 0x7fb18411bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:32.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.929+0000 7fb18a2a0700 1 -- 172.21.15.3:0/3993139514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17c009a70 con 0x7fb18409d6f0 2023-10-03T10:13:32.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.929+0000 7fb18aaa1700 1 --2- 172.21.15.3:0/3993139514 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb184135e50 0x7fb18411c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:32.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.929+0000 7fb18a2a0700 1 --2- 172.21.15.3:0/3993139514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb18409d6f0 0x7fb18411b670 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb18000c730 tx=0x7fb18000ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:32.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.929+0000 7fb17affd700 1 -- 172.21.15.3:0/3993139514 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb180005d70 con 0x7fb18409d6f0 2023-10-03T10:13:32.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.929+0000 7fb17affd700 1 -- 172.21.15.3:0/3993139514 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1800070d0 con 0x7fb18409d6f0 2023-10-03T10:13:32.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.930+0000 7fb17affd700 1 -- 172.21.15.3:0/3993139514 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb180007460 con 0x7fb18409d6f0 2023-10-03T10:13:32.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.930+0000 7fb18c504700 1 -- 172.21.15.3:0/3993139514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1740094a0 con 0x7fb18409d6f0 2023-10-03T10:13:32.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.930+0000 7fb18c504700 1 -- 172.21.15.3:0/3993139514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1840a2090 con 0x7fb18409d6f0 2023-10-03T10:13:32.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.931+0000 7fb17affd700 1 -- 172.21.15.3:0/3993139514 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb1800075c0 con 0x7fb18409d6f0 2023-10-03T10:13:32.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.931+0000 7fb18c504700 1 -- 172.21.15.3:0/3993139514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb184071700 con 0x7fb18409d6f0 2023-10-03T10:13:32.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.931+0000 7fb17affd700 1 -- 172.21.15.3:0/3993139514 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb18003eb70 con 0x7fb18409d6f0 2023-10-03T10:13:32.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.940+0000 7fb17affd700 1 -- 172.21.15.3:0/3993139514 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb18000f260 con 0x7fb18409d6f0 2023-10-03T10:13:32.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.951+0000 7f7349bf0700 1 -- 172.21.15.3:0/4236609865 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7344120de0 con 0x7f734409e930 2023-10-03T10:13:32.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.951+0000 7f732bfff700 1 -- 172.21.15.3:0/4236609865 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7344120de0 con 0x7f734409e930 2023-10-03T10:13:32.952 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:32.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.954+0000 7f7349bf0700 1 -- 172.21.15.3:0/4236609865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f734409e930 msgr2=0x7f734411dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:32.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.954+0000 7f7349bf0700 1 --2- 172.21.15.3:0/4236609865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f734409e930 0x7f734411dcd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f733400c400 tx=0x7f733400c710 comp rx=0 tx=0).stop 2023-10-03T10:13:32.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.954+0000 7f7349bf0700 1 -- 172.21.15.3:0/4236609865 shutdown_connections 2023-10-03T10:13:32.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.954+0000 7f7349bf0700 1 --2- 172.21.15.3:0/4236609865 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73441338a0 0x7f734411e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:32.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.954+0000 7f7349bf0700 1 --2- 172.21.15.3:0/4236609865 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f734409de50 0x7f734411d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:32.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.954+0000 7f7349bf0700 1 --2- 172.21.15.3:0/4236609865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f734409e930 0x7f734411dcd0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:32.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.954+0000 7f7349bf0700 1 -- 172.21.15.3:0/4236609865 >> 172.21.15.3:0/4236609865 conn(0x7f73440a2aa0 msgr2=0x7f73441b9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:32.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.955+0000 7f7349bf0700 1 -- 172.21.15.3:0/4236609865 shutdown_connections 2023-10-03T10:13:32.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.955+0000 7f7349bf0700 1 -- 172.21.15.3:0/4236609865 wait complete. 2023-10-03T10:13:32.969 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:32.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.976+0000 7fd6ac3cb700 1 Processor -- start 2023-10-03T10:13:32.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.977+0000 7fd6ac3cb700 1 -- start start 2023-10-03T10:13:32.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.978+0000 7fd6ac3cb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6a412cf60 0x7fd6a406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:32.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.978+0000 7fd6ac3cb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6a412f8b0 0x7fd6a412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:32.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.978+0000 7fd6ac3cb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd6a4130260 0x7fd6a4136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:32.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.978+0000 7fd6ac3cb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6a412d330 con 0x7fd6a4130260 2023-10-03T10:13:32.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.978+0000 7fd6ac3cb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a412cd90 con 0x7fd6a412cf60 2023-10-03T10:13:32.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.978+0000 7fd6ac3cb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a41306d0 con 0x7fd6a412f8b0 2023-10-03T10:13:32.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.978+0000 7fd6aa167700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6a412cf60 0x7fd6a406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:32.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.978+0000 7fd6aa167700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6a412cf60 0x7fd6a406f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:57092/0 (socket says 172.21.15.3:57092) 2023-10-03T10:13:32.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.978+0000 7fd6aa167700 1 -- 172.21.15.3:0/427307343 learned_addr learned my addr 172.21.15.3:0/427307343 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:32.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.978+0000 7fd6a9966700 1 --2- 172.21.15.3:0/427307343 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6a412f8b0 0x7fd6a412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:32.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.978+0000 7fd6aa968700 1 --2- 172.21.15.3:0/427307343 >> v2:172.21.15.3:3300/0 conn(0x7fd6a4130260 0x7fd6a4136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:32.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.978+0000 7fd6aa167700 1 -- 172.21.15.3:0/427307343 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6a412f8b0 msgr2=0x7fd6a412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:32.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.978+0000 7fd6aa167700 1 --2- 172.21.15.3:0/427307343 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6a412f8b0 0x7fd6a412fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:32.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.978+0000 7fd6aa167700 1 -- 172.21.15.3:0/427307343 >> v2:172.21.15.3:3300/0 conn(0x7fd6a4130260 msgr2=0x7fd6a4136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:32.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.978+0000 7fd6aa167700 1 --2- 172.21.15.3:0/427307343 >> v2:172.21.15.3:3300/0 conn(0x7fd6a4130260 0x7fd6a4136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:32.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.978+0000 7fd6aa167700 1 -- 172.21.15.3:0/427307343 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a4071780 con 0x7fd6a412cf60 2023-10-03T10:13:32.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.978+0000 7fd6a9966700 1 --2- 172.21.15.3:0/427307343 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6a412f8b0 0x7fd6a412fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:32.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.978+0000 7fd6aa167700 1 --2- 172.21.15.3:0/427307343 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6a412cf60 0x7fd6a406f5d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00098f0 tx=0x7fd6a0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b72ef3edfa81bac server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:32.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.979+0000 7fd6a9165700 1 -- 172.21.15.3:0/427307343 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0011830 con 0x7fd6a412cf60 2023-10-03T10:13:32.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.979+0000 7fd6a9165700 1 -- 172.21.15.3:0/427307343 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a0011990 con 0x7fd6a412cf60 2023-10-03T10:13:32.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.979+0000 7fd6a9165700 1 -- 172.21.15.3:0/427307343 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0011d20 con 0x7fd6a412cf60 2023-10-03T10:13:32.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.979+0000 7fd6ac3cb700 1 -- 172.21.15.3:0/427307343 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6a412cf60 msgr2=0x7fd6a406f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:32.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.979+0000 7fd6ac3cb700 1 --2- 172.21.15.3:0/427307343 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6a412cf60 0x7fd6a406f5d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00098f0 tx=0x7fd6a0009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:32.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.980+0000 7fd6ac3cb700 1 -- 172.21.15.3:0/427307343 shutdown_connections 2023-10-03T10:13:32.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.980+0000 7fd6ac3cb700 1 --2- 172.21.15.3:0/427307343 >> v2:172.21.15.3:3300/0 conn(0x7fd6a4130260 0x7fd6a4136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:32.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.980+0000 7fd6ac3cb700 1 --2- 172.21.15.3:0/427307343 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6a412cf60 0x7fd6a406f5d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:32.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.980+0000 7fd6ac3cb700 1 --2- 172.21.15.3:0/427307343 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6a412f8b0 0x7fd6a412fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:32.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.980+0000 7fd6ac3cb700 1 -- 172.21.15.3:0/427307343 >> 172.21.15.3:0/427307343 conn(0x7fd6a40a2aa0 msgr2=0x7fd6a40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:32.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.980+0000 7fd6ac3cb700 1 -- 172.21.15.3:0/427307343 shutdown_connections 2023-10-03T10:13:32.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.980+0000 7fd6ac3cb700 1 -- 172.21.15.3:0/427307343 wait complete. 2023-10-03T10:13:32.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.981+0000 7fd6ac3cb700 1 Processor -- start 2023-10-03T10:13:32.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.981+0000 7fd6ac3cb700 1 -- start start 2023-10-03T10:13:32.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.981+0000 7fd6ac3cb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6a412cf60 0x7fd6a41309e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:32.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.981+0000 7fd6ac3cb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6a412f8b0 0x7fd6a4130f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:32.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.981+0000 7fd6ac3cb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6a4130260 0x7fd6a4131460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:32.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.981+0000 7fd6ac3cb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a4129fa0 con 0x7fd6a412cf60 2023-10-03T10:13:32.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.981+0000 7fd6ac3cb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a4133400 con 0x7fd6a412f8b0 2023-10-03T10:13:32.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.981+0000 7fd6ac3cb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a409ea50 con 0x7fd6a4130260 2023-10-03T10:13:32.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.981+0000 7fd6aa167700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6a412cf60 0x7fd6a41309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:32.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.981+0000 7fd6aa167700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6a412cf60 0x7fd6a41309e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53980/0 (socket says 172.21.15.3:53980) 2023-10-03T10:13:32.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.981+0000 7fd6aa167700 1 -- 172.21.15.3:0/2412218970 learned_addr learned my addr 172.21.15.3:0/2412218970 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:32.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.981+0000 7fd6aa968700 1 --2- 172.21.15.3:0/2412218970 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6a4130260 0x7fd6a4131460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:32.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.981+0000 7fd6a9966700 1 --2- 172.21.15.3:0/2412218970 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6a412f8b0 0x7fd6a4130f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:32.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.982+0000 7fd6aa167700 1 -- 172.21.15.3:0/2412218970 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6a4130260 msgr2=0x7fd6a4131460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:32.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.982+0000 7fd6aa167700 1 --2- 172.21.15.3:0/2412218970 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6a4130260 0x7fd6a4131460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:32.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.982+0000 7fd6aa167700 1 -- 172.21.15.3:0/2412218970 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6a412f8b0 msgr2=0x7fd6a4130f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:32.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.982+0000 7fd6aa167700 1 --2- 172.21.15.3:0/2412218970 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6a412f8b0 0x7fd6a4130f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:32.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.982+0000 7fd6aa167700 1 -- 172.21.15.3:0/2412218970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6940094a0 con 0x7fd6a412cf60 2023-10-03T10:13:32.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.982+0000 7fd6aa167700 1 --2- 172.21.15.3:0/2412218970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6a412cf60 0x7fd6a41309e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0000c00 tx=0x7fd6a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:32.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.982+0000 7fd69affd700 1 -- 172.21.15.3:0/2412218970 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0004240 con 0x7fd6a412cf60 2023-10-03T10:13:32.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.982+0000 7fd69affd700 1 -- 172.21.15.3:0/2412218970 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a001f070 con 0x7fd6a412cf60 2023-10-03T10:13:32.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.982+0000 7fd6ac3cb700 1 -- 172.21.15.3:0/2412218970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6a0009550 con 0x7fd6a412cf60 2023-10-03T10:13:32.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.982+0000 7fd69affd700 1 -- 172.21.15.3:0/2412218970 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a00045e0 con 0x7fd6a412cf60 2023-10-03T10:13:32.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.982+0000 7fd6ac3cb700 1 -- 172.21.15.3:0/2412218970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6a4131ce0 con 0x7fd6a412cf60 2023-10-03T10:13:32.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.983+0000 7fd69affd700 1 -- 172.21.15.3:0/2412218970 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd6a0021030 con 0x7fd6a412cf60 2023-10-03T10:13:32.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.983+0000 7fd69affd700 1 -- 172.21.15.3:0/2412218970 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd6a00408a0 con 0x7fd6a412cf60 2023-10-03T10:13:32.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.983+0000 7fd6ac3cb700 1 -- 172.21.15.3:0/2412218970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6a4074ea0 con 0x7fd6a412cf60 2023-10-03T10:13:32.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:32.988+0000 7fd69affd700 1 -- 172.21.15.3:0/2412218970 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd6a0015030 con 0x7fd6a412cf60 2023-10-03T10:13:33.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.122+0000 7fcb1356a700 1 -- 172.21.15.3:0/864148700 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcb0c074ea0 con 0x7fcb0c12f8b0 2023-10-03T10:13:33.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.123+0000 7fcb01ffb700 1 -- 172.21.15.3:0/864148700 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcafc011cd0 con 0x7fcb0c12f8b0 2023-10-03T10:13:33.123 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:33.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.125+0000 7fcb1356a700 1 -- 172.21.15.3:0/864148700 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb0c12f8b0 msgr2=0x7fcb0c09c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:33.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.125+0000 7fcb1356a700 1 --2- 172.21.15.3:0/864148700 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb0c12f8b0 0x7fcb0c09c170 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcafc00e930 tx=0x7fcafc00ec40 comp rx=0 tx=0).stop 2023-10-03T10:13:33.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.125+0000 7fcb1356a700 1 -- 172.21.15.3:0/864148700 shutdown_connections 2023-10-03T10:13:33.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.125+0000 7fcb1356a700 1 --2- 172.21.15.3:0/864148700 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb0c130260 0x7fcb0c1c40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:33.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.125+0000 7fcb1356a700 1 --2- 172.21.15.3:0/864148700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb0c12cf60 0x7fcb0c09bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:33.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.125+0000 7fcb1356a700 1 --2- 172.21.15.3:0/864148700 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb0c12f8b0 0x7fcb0c09c170 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:33.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.125+0000 7fcb1356a700 1 -- 172.21.15.3:0/864148700 >> 172.21.15.3:0/864148700 conn(0x7fcb0c0a2aa0 msgr2=0x7fcb0c1b7990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:33.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.125+0000 7fcb1356a700 1 -- 172.21.15.3:0/864148700 shutdown_connections 2023-10-03T10:13:33.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.125+0000 7fcb1356a700 1 -- 172.21.15.3:0/864148700 wait complete. 2023-10-03T10:13:33.140 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:33.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.149+0000 7fb18c504700 1 -- 172.21.15.3:0/3993139514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb18409c700 con 0x7fb18409d6f0 2023-10-03T10:13:33.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.149+0000 7fb17affd700 1 -- 172.21.15.3:0/3993139514 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb180017070 con 0x7fb18409d6f0 2023-10-03T10:13:33.149 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:33.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.151+0000 7fb18c504700 1 -- 172.21.15.3:0/3993139514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb18409d6f0 msgr2=0x7fb18411b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:33.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.151+0000 7fb18c504700 1 --2- 172.21.15.3:0/3993139514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb18409d6f0 0x7fb18411b670 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb18000c730 tx=0x7fb18000ca40 comp rx=0 tx=0).stop 2023-10-03T10:13:33.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.151+0000 7fb18c504700 1 -- 172.21.15.3:0/3993139514 shutdown_connections 2023-10-03T10:13:33.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.151+0000 7fb18c504700 1 --2- 172.21.15.3:0/3993139514 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb184135e50 0x7fb18411c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:33.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.151+0000 7fb18c504700 1 --2- 172.21.15.3:0/3993139514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb18409d6f0 0x7fb18411b670 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:33.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.151+0000 7fb18c504700 1 --2- 172.21.15.3:0/3993139514 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb18409e1d0 0x7fb18411bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:33.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.151+0000 7fb18c504700 1 -- 172.21.15.3:0/3993139514 >> 172.21.15.3:0/3993139514 conn(0x7fb184066f50 msgr2=0x7fb18406e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:33.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.151+0000 7fb18c504700 1 -- 172.21.15.3:0/3993139514 shutdown_connections 2023-10-03T10:13:33.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.151+0000 7fb18c504700 1 -- 172.21.15.3:0/3993139514 wait complete. 2023-10-03T10:13:33.165 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:33.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.196+0000 7fd6ac3cb700 1 -- 172.21.15.3:0/2412218970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd6a4132dc0 con 0x7fd6a412cf60 2023-10-03T10:13:33.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.197+0000 7fd69affd700 1 -- 172.21.15.3:0/2412218970 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd6a001a450 con 0x7fd6a412cf60 2023-10-03T10:13:33.197 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:33.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.199+0000 7fd6ac3cb700 1 -- 172.21.15.3:0/2412218970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6a412cf60 msgr2=0x7fd6a41309e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:33.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.199+0000 7fd6ac3cb700 1 --2- 172.21.15.3:0/2412218970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6a412cf60 0x7fd6a41309e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0000c00 tx=0x7fd6a0004750 comp rx=0 tx=0).stop 2023-10-03T10:13:33.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.199+0000 7fd6ac3cb700 1 -- 172.21.15.3:0/2412218970 shutdown_connections 2023-10-03T10:13:33.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.199+0000 7fd6ac3cb700 1 --2- 172.21.15.3:0/2412218970 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6a4130260 0x7fd6a4131460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:33.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.199+0000 7fd6ac3cb700 1 --2- 172.21.15.3:0/2412218970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6a412cf60 0x7fd6a41309e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:33.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.199+0000 7fd6ac3cb700 1 --2- 172.21.15.3:0/2412218970 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6a412f8b0 0x7fd6a4130f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:33.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.199+0000 7fd6ac3cb700 1 -- 172.21.15.3:0/2412218970 >> 172.21.15.3:0/2412218970 conn(0x7fd6a40a2aa0 msgr2=0x7fd6a406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:33.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.199+0000 7fd6ac3cb700 1 -- 172.21.15.3:0/2412218970 shutdown_connections 2023-10-03T10:13:33.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:33.199+0000 7fd6ac3cb700 1 -- 172.21.15.3:0/2412218970 wait complete. 2023-10-03T10:13:33.213 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:33.971 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:34.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.117+0000 7f9a0b051700 1 Processor -- start 2023-10-03T10:13:34.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.117+0000 7f9a0b051700 1 -- start start 2023-10-03T10:13:34.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.117+0000 7f9a0b051700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a0409f410 0x7f9a04136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:34.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.117+0000 7f9a0b051700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9a04136fe0 0x7f9a041393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:34.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.117+0000 7f9a0b051700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a04070940 con 0x7f9a0409e930 2023-10-03T10:13:34.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.117+0000 7f9a0b051700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0406fd60 con 0x7f9a04136fe0 2023-10-03T10:13:34.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.117+0000 7f9a0b051700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0406fa80 con 0x7f9a0409f410 2023-10-03T10:13:34.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.117+0000 7f9a03fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a0409f410 0x7f9a04136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:34.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.117+0000 7f9a095ee700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9a04136fe0 0x7f9a041393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:34.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.117+0000 7f9a095ee700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9a04136fe0 0x7f9a041393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45020/0 (socket says 172.21.15.3:45020) 2023-10-03T10:13:34.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.117+0000 7f9a03fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a0409f410 0x7f9a04136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44418/0 (socket says 172.21.15.3:44418) 2023-10-03T10:13:34.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.117+0000 7f9a095ee700 1 -- 172.21.15.3:0/3779719255 learned_addr learned my addr 172.21.15.3:0/3779719255 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:34.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.118+0000 7f9a095ee700 1 -- 172.21.15.3:0/3779719255 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a0409f410 msgr2=0x7f9a04136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:34.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.118+0000 7f9a095ee700 1 --2- 172.21.15.3:0/3779719255 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a0409f410 0x7f9a04136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.118+0000 7f9a095ee700 1 -- 172.21.15.3:0/3779719255 >> v1:172.21.15.3:6789/0 conn(0x7f9a0409e930 legacy=0x7f9a0409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:34.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.118+0000 7f9a095ee700 1 -- 172.21.15.3:0/3779719255 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0409fb60 con 0x7f9a04136fe0 2023-10-03T10:13:34.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.118+0000 7f9a03fff700 1 --2- 172.21.15.3:0/3779719255 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a0409f410 0x7f9a04136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:34.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.118+0000 7f9a095ee700 1 --2- 172.21.15.3:0/3779719255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a04136fe0 0x7f9a041393c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009e50 tx=0x7f99fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8488af55413d810f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:34.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.118+0000 7f9a037fe700 1 -- 172.21.15.3:0/3779719255 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc00e070 con 0x7f9a04136fe0 2023-10-03T10:13:34.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.118+0000 7f9a037fe700 1 -- 172.21.15.3:0/3779719255 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99fc00b910 con 0x7f9a04136fe0 2023-10-03T10:13:34.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.118+0000 7f9a037fe700 1 -- 172.21.15.3:0/3779719255 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc00bcc0 con 0x7f9a04136fe0 2023-10-03T10:13:34.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.119+0000 7f9a0b051700 1 -- 172.21.15.3:0/3779719255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a04136fe0 msgr2=0x7f9a041393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:34.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.119+0000 7f9a0b051700 1 --2- 172.21.15.3:0/3779719255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a04136fe0 0x7f9a041393c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009e50 tx=0x7f99fc00b330 comp rx=0 tx=0).stop 2023-10-03T10:13:34.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.119+0000 7f9a0b051700 1 -- 172.21.15.3:0/3779719255 shutdown_connections 2023-10-03T10:13:34.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.119+0000 7f9a0b051700 1 --2- 172.21.15.3:0/3779719255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a04136fe0 0x7f9a041393c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.119+0000 7f9a0b051700 1 --2- 172.21.15.3:0/3779719255 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a0409f410 0x7f9a04136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.119+0000 7f9a0b051700 1 -- 172.21.15.3:0/3779719255 >> 172.21.15.3:0/3779719255 conn(0x7f9a040a4210 msgr2=0x7f9a040a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:34.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.119+0000 7f9a0b051700 1 -- 172.21.15.3:0/3779719255 shutdown_connections 2023-10-03T10:13:34.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.119+0000 7f9a0b051700 1 -- 172.21.15.3:0/3779719255 wait complete. 2023-10-03T10:13:34.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.120+0000 7f9a0b051700 1 Processor -- start 2023-10-03T10:13:34.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.120+0000 7f9a0b051700 1 -- start start 2023-10-03T10:13:34.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.121+0000 7f9a0b051700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a0409e930 0x7f9a0411e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:34.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.121+0000 7f9a0b051700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a0409f410 0x7f9a0411eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:34.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.121+0000 7f9a0b051700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a04136fe0 0x7f9a0411f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:34.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.121+0000 7f9a0b051700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a04133ef0 con 0x7f9a0409f410 2023-10-03T10:13:34.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.121+0000 7f9a0b051700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a041201d0 con 0x7f9a0409e930 2023-10-03T10:13:34.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.121+0000 7f9a0b051700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a041344b0 con 0x7f9a04136fe0 2023-10-03T10:13:34.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.121+0000 7f9a08ded700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a0409e930 0x7f9a0411e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:34.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.121+0000 7f9a03fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a0409f410 0x7f9a0411eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:34.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.121+0000 7f9a08ded700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a0409e930 0x7f9a0411e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44424/0 (socket says 172.21.15.3:44424) 2023-10-03T10:13:34.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.121+0000 7f9a03fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a0409f410 0x7f9a0411eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45026/0 (socket says 172.21.15.3:45026) 2023-10-03T10:13:34.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.121+0000 7f9a08ded700 1 -- 172.21.15.3:0/840440929 learned_addr learned my addr 172.21.15.3:0/840440929 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:34.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.121+0000 7f9a095ee700 1 --2- 172.21.15.3:0/840440929 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a04136fe0 0x7f9a0411f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:34.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.122+0000 7f9a095ee700 1 -- 172.21.15.3:0/840440929 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a0409e930 msgr2=0x7f9a0411e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:34.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.122+0000 7f9a095ee700 1 --2- 172.21.15.3:0/840440929 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a0409e930 0x7f9a0411e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.122+0000 7f9a095ee700 1 -- 172.21.15.3:0/840440929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a0409f410 msgr2=0x7f9a0411eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:34.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.122+0000 7f9a095ee700 1 --2- 172.21.15.3:0/840440929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a0409f410 0x7f9a0411eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.122+0000 7f9a095ee700 1 -- 172.21.15.3:0/840440929 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f00094a0 con 0x7f9a04136fe0 2023-10-03T10:13:34.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.122+0000 7f9a03fff700 1 --2- 172.21.15.3:0/840440929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a0409f410 0x7f9a0411eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:34.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.122+0000 7f9a095ee700 1 --2- 172.21.15.3:0/840440929 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a04136fe0 0x7f9a0411f040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f99fc015000 tx=0x7f99fc004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:34.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.122+0000 7f9a017fa700 1 -- 172.21.15.3:0/840440929 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc004780 con 0x7f9a04136fe0 2023-10-03T10:13:34.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.122+0000 7f9a017fa700 1 -- 172.21.15.3:0/840440929 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99fc0092e0 con 0x7f9a04136fe0 2023-10-03T10:13:34.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.122+0000 7f9a0b051700 1 -- 172.21.15.3:0/840440929 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99fc009b30 con 0x7f9a04136fe0 2023-10-03T10:13:34.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.122+0000 7f9a017fa700 1 -- 172.21.15.3:0/840440929 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc01c680 con 0x7f9a04136fe0 2023-10-03T10:13:34.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.122+0000 7f9a0b051700 1 -- 172.21.15.3:0/840440929 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a04120a30 con 0x7f9a04136fe0 2023-10-03T10:13:34.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.123+0000 7f9a017fa700 1 -- 172.21.15.3:0/840440929 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f99fc01c7e0 con 0x7f9a04136fe0 2023-10-03T10:13:34.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.123+0000 7f9a017fa700 1 -- 172.21.15.3:0/840440929 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f99fc040e70 con 0x7f9a04136fe0 2023-10-03T10:13:34.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.123+0000 7f9a0b051700 1 -- 172.21.15.3:0/840440929 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a04121870 con 0x7f9a04136fe0 2023-10-03T10:13:34.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.128+0000 7f9a017fa700 1 -- 172.21.15.3:0/840440929 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f99fc0249a0 con 0x7f9a04136fe0 2023-10-03T10:13:34.141 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:34.166 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:34.213 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:34.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.286+0000 7fe67b67b700 1 Processor -- start 2023-10-03T10:13:34.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.286+0000 7fe67b67b700 1 -- start start 2023-10-03T10:13:34.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.287+0000 7fe67b67b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe67409d6f0 0x7fe67409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:34.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.287+0000 7fe67b67b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe67409e000 0x7fe674130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:34.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.287+0000 7fe67b67b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe67409f130 con 0x7fe67409e410 2023-10-03T10:13:34.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.287+0000 7fe67b67b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67409f520 con 0x7fe67409e000 2023-10-03T10:13:34.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.287+0000 7fe67b67b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe67409e930 con 0x7fe67409d6f0 2023-10-03T10:13:34.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.287+0000 7fe678c16700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe67409e000 0x7fe674130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:34.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.287+0000 7fe678c16700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe67409e000 0x7fe674130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44432/0 (socket says 172.21.15.3:44432) 2023-10-03T10:13:34.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.287+0000 7fe679c18700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe67409e410 0x7fe674130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35312/0 (socket says 172.21.15.3:35312) 2023-10-03T10:13:34.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.287+0000 7fe678c16700 1 -- 172.21.15.3:0/2232277501 learned_addr learned my addr 172.21.15.3:0/2232277501 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:34.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.287+0000 7fe679417700 1 --2- 172.21.15.3:0/2232277501 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe67409d6f0 0x7fe67409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:34.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.287+0000 7fe678c16700 1 -- 172.21.15.3:0/2232277501 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe67409d6f0 msgr2=0x7fe67409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:34.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.288+0000 7fe678c16700 1 --2- 172.21.15.3:0/2232277501 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe67409d6f0 0x7fe67409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.288+0000 7fe678c16700 1 -- 172.21.15.3:0/2232277501 >> v1:172.21.15.3:6789/0 conn(0x7fe67409e410 legacy=0x7fe674130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:34.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.288+0000 7fe66bfff700 1 -- 172.21.15.3:0/2232277501 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1942295775 0 0) 0x7fe67409f130 con 0x7fe67409e410 2023-10-03T10:13:34.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.288+0000 7fe678c16700 1 -- 172.21.15.3:0/2232277501 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67409b2d0 con 0x7fe67409e000 2023-10-03T10:13:34.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.288+0000 7fe678c16700 1 --2- 172.21.15.3:0/2232277501 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe67409e000 0x7fe674130660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe664009920 tx=0x7fe664009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea7e4a226874d824 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:34.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.288+0000 7fe66bfff700 1 -- 172.21.15.3:0/2232277501 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe664011830 con 0x7fe67409e000 2023-10-03T10:13:34.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.288+0000 7fe66bfff700 1 -- 172.21.15.3:0/2232277501 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe664011990 con 0x7fe67409e000 2023-10-03T10:13:34.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.288+0000 7fe66bfff700 1 -- 172.21.15.3:0/2232277501 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe664011d20 con 0x7fe67409e000 2023-10-03T10:13:34.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.289+0000 7fe67b67b700 1 -- 172.21.15.3:0/2232277501 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe67409e000 msgr2=0x7fe674130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:34.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.289+0000 7fe67b67b700 1 --2- 172.21.15.3:0/2232277501 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe67409e000 0x7fe674130660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe664009920 tx=0x7fe664009c30 comp rx=0 tx=0).stop 2023-10-03T10:13:34.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.289+0000 7fe67b67b700 1 -- 172.21.15.3:0/2232277501 shutdown_connections 2023-10-03T10:13:34.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.289+0000 7fe67b67b700 1 --2- 172.21.15.3:0/2232277501 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe67409d6f0 0x7fe67409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.289+0000 7fe67b67b700 1 --2- 172.21.15.3:0/2232277501 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe67409e000 0x7fe674130660 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.289+0000 7fe67b67b700 1 -- 172.21.15.3:0/2232277501 >> 172.21.15.3:0/2232277501 conn(0x7fe674066f50 msgr2=0x7fe67409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:34.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.289+0000 7fe67b67b700 1 -- 172.21.15.3:0/2232277501 shutdown_connections 2023-10-03T10:13:34.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.289+0000 7fe67b67b700 1 -- 172.21.15.3:0/2232277501 wait complete. 2023-10-03T10:13:34.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.290+0000 7fe67b67b700 1 Processor -- start 2023-10-03T10:13:34.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.290+0000 7fe67b67b700 1 -- start start 2023-10-03T10:13:34.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.291+0000 7fe67b67b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe67409d6f0 0x7fe674131670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:34.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.291+0000 7fe67b67b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe67409e000 0x7fe674131bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:34.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.291+0000 7fe67b67b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe67409e410 0x7fe6741320f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:34.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.291+0000 7fe67b67b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe674132b70 con 0x7fe67409e000 2023-10-03T10:13:34.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.291+0000 7fe67b67b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe674133dc0 con 0x7fe67409d6f0 2023-10-03T10:13:34.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.291+0000 7fe67b67b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe6741353f0 con 0x7fe67409e410 2023-10-03T10:13:34.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.291+0000 7fe678c16700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe67409e000 0x7fe674131bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:34.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.291+0000 7fe678c16700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe67409e000 0x7fe674131bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45038/0 (socket says 172.21.15.3:45038) 2023-10-03T10:13:34.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.291+0000 7fe678c16700 1 -- 172.21.15.3:0/177363590 learned_addr learned my addr 172.21.15.3:0/177363590 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:34.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.291+0000 7fe679c18700 1 --2- 172.21.15.3:0/177363590 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe67409e410 0x7fe6741320f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:34.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.291+0000 7fe679417700 1 --2- 172.21.15.3:0/177363590 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe67409d6f0 0x7fe674131670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:34.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.291+0000 7fe678c16700 1 -- 172.21.15.3:0/177363590 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe67409e410 msgr2=0x7fe6741320f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:34.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.291+0000 7fe678c16700 1 --2- 172.21.15.3:0/177363590 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe67409e410 0x7fe6741320f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.291+0000 7fe678c16700 1 -- 172.21.15.3:0/177363590 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe67409d6f0 msgr2=0x7fe674131670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:34.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.291+0000 7fe678c16700 1 --2- 172.21.15.3:0/177363590 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe67409d6f0 0x7fe674131670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.291+0000 7fe678c16700 1 -- 172.21.15.3:0/177363590 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe664009550 con 0x7fe67409e000 2023-10-03T10:13:34.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.291+0000 7fe678c16700 1 --2- 172.21.15.3:0/177363590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe67409e000 0x7fe674131bb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe664013000 tx=0x7fe664004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:34.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.292+0000 7fe669ffb700 1 -- 172.21.15.3:0/177363590 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe664004240 con 0x7fe67409e000 2023-10-03T10:13:34.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.292+0000 7fe669ffb700 1 -- 172.21.15.3:0/177363590 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66401f070 con 0x7fe67409e000 2023-10-03T10:13:34.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.292+0000 7fe67b67b700 1 -- 172.21.15.3:0/177363590 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe674133390 con 0x7fe67409e000 2023-10-03T10:13:34.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.292+0000 7fe669ffb700 1 -- 172.21.15.3:0/177363590 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6640045a0 con 0x7fe67409e000 2023-10-03T10:13:34.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.292+0000 7fe67b67b700 1 -- 172.21.15.3:0/177363590 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe674134210 con 0x7fe67409e000 2023-10-03T10:13:34.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.293+0000 7fe669ffb700 1 -- 172.21.15.3:0/177363590 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe664021030 con 0x7fe67409e000 2023-10-03T10:13:34.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.293+0000 7fe669ffb700 1 -- 172.21.15.3:0/177363590 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe6640408e0 con 0x7fe67409e000 2023-10-03T10:13:34.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.293+0000 7fe67b67b700 1 -- 172.21.15.3:0/177363590 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe674074ea0 con 0x7fe67409e000 2023-10-03T10:13:34.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.303+0000 7fe669ffb700 1 -- 172.21.15.3:0/177363590 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe6640036a0 con 0x7fe67409e000 2023-10-03T10:13:34.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.342+0000 7f458eac7700 1 Processor -- start 2023-10-03T10:13:34.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.342+0000 7f458eac7700 1 -- start start 2023-10-03T10:13:34.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.343+0000 7f458eac7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4588131810 0x7f4588131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:34.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.343+0000 7f458eac7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4588099a40 0x7f4588099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:34.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.343+0000 7f458eac7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45881313c0 con 0x7f458809a3b0 2023-10-03T10:13:34.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.343+0000 7f458eac7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458809b580 con 0x7f4588099a40 2023-10-03T10:13:34.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.343+0000 7f458eac7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f458806e8c0 con 0x7f4588131810 2023-10-03T10:13:34.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.343+0000 7f4587fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4588099a40 0x7f4588099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:34.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.343+0000 7f4587fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4588099a40 0x7f4588099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44456/0 (socket says 172.21.15.3:44456) 2023-10-03T10:13:34.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.343+0000 7f4587fff700 1 -- 172.21.15.3:0/1753170311 learned_addr learned my addr 172.21.15.3:0/1753170311 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:34.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.343+0000 7f458c863700 1 --2- 172.21.15.3:0/1753170311 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4588131810 0x7f4588131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:34.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.343+0000 7f4587fff700 1 -- 172.21.15.3:0/1753170311 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4588131810 msgr2=0x7f4588131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:34.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.343+0000 7f4587fff700 1 --2- 172.21.15.3:0/1753170311 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4588131810 0x7f4588131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.343+0000 7f4587fff700 1 -- 172.21.15.3:0/1753170311 >> v1:172.21.15.3:6789/0 conn(0x7f458809a3b0 legacy=0x7f4588138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:34.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.343+0000 7f45877fe700 1 -- 172.21.15.3:0/1753170311 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1976933068 0 0) 0x7f45881313c0 con 0x7f458809a3b0 2023-10-03T10:13:34.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.343+0000 7f4587fff700 1 -- 172.21.15.3:0/1753170311 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458809a820 con 0x7f4588099a40 2023-10-03T10:13:34.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.344+0000 7f458c863700 1 --2- 172.21.15.3:0/1753170311 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4588131810 0x7f4588131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:13:34.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.344+0000 7f4587fff700 1 --2- 172.21.15.3:0/1753170311 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4588099a40 0x7f4588099e70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f457c009920 tx=0x7f457c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a91226ad290c6f9f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:34.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.344+0000 7f45877fe700 1 -- 172.21.15.3:0/1753170311 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c00ba60 con 0x7f4588099a40 2023-10-03T10:13:34.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.344+0000 7f45877fe700 1 -- 172.21.15.3:0/1753170311 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f457c00bbc0 con 0x7f4588099a40 2023-10-03T10:13:34.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.344+0000 7f45877fe700 1 -- 172.21.15.3:0/1753170311 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c009e70 con 0x7f4588099a40 2023-10-03T10:13:34.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.345+0000 7f458eac7700 1 -- 172.21.15.3:0/1753170311 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4588099a40 msgr2=0x7f4588099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:34.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.345+0000 7f458eac7700 1 --2- 172.21.15.3:0/1753170311 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4588099a40 0x7f4588099e70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f457c009920 tx=0x7f457c009c30 comp rx=0 tx=0).stop 2023-10-03T10:13:34.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.345+0000 7f458eac7700 1 -- 172.21.15.3:0/1753170311 shutdown_connections 2023-10-03T10:13:34.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.345+0000 7f458eac7700 1 --2- 172.21.15.3:0/1753170311 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4588131810 0x7f4588131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.345+0000 7f458eac7700 1 --2- 172.21.15.3:0/1753170311 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4588099a40 0x7f4588099e70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.345+0000 7f458eac7700 1 -- 172.21.15.3:0/1753170311 >> 172.21.15.3:0/1753170311 conn(0x7f45880a2aa0 msgr2=0x7f45880a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:34.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.345+0000 7f458eac7700 1 -- 172.21.15.3:0/1753170311 shutdown_connections 2023-10-03T10:13:34.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.345+0000 7f458eac7700 1 -- 172.21.15.3:0/1753170311 wait complete. 2023-10-03T10:13:34.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.346+0000 7f458eac7700 1 Processor -- start 2023-10-03T10:13:34.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.346+0000 7f458eac7700 1 -- start start 2023-10-03T10:13:34.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.347+0000 7f458eac7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4588099a40 0x7f458806f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:34.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.347+0000 7f458eac7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458809a3b0 0x7f458806fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:34.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.347+0000 7f458eac7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4588131810 0x7f4588070060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:34.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.347+0000 7f458eac7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458811e0c0 con 0x7f4588099a40 2023-10-03T10:13:34.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.347+0000 7f458eac7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458811f160 con 0x7f458809a3b0 2023-10-03T10:13:34.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.347+0000 7f458eac7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f458811f3d0 con 0x7f4588131810 2023-10-03T10:13:34.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.347+0000 7f4587fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458809a3b0 0x7f458806fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:34.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.347+0000 7f4587fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458809a3b0 0x7f458806fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44472/0 (socket says 172.21.15.3:44472) 2023-10-03T10:13:34.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.347+0000 7f4587fff700 1 -- 172.21.15.3:0/2819635265 learned_addr learned my addr 172.21.15.3:0/2819635265 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:34.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.347+0000 7f458d064700 1 --2- 172.21.15.3:0/2819635265 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4588131810 0x7f4588070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:34.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.347+0000 7f458c863700 1 --2- 172.21.15.3:0/2819635265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4588099a40 0x7f458806f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:34.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.347+0000 7f4587fff700 1 -- 172.21.15.3:0/2819635265 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4588131810 msgr2=0x7f4588070060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:34.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.347+0000 7f4587fff700 1 --2- 172.21.15.3:0/2819635265 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4588131810 0x7f4588070060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.347+0000 7f4587fff700 1 -- 172.21.15.3:0/2819635265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4588099a40 msgr2=0x7f458806f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:34.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.347+0000 7f4587fff700 1 --2- 172.21.15.3:0/2819635265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4588099a40 0x7f458806f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.347+0000 7f4587fff700 1 -- 172.21.15.3:0/2819635265 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457c009550 con 0x7f458809a3b0 2023-10-03T10:13:34.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.347+0000 7f458d064700 1 --2- 172.21.15.3:0/2819635265 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4588131810 0x7f4588070060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:34.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.347+0000 7f458c863700 1 --2- 172.21.15.3:0/2819635265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4588099a40 0x7f458806f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:34.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.347+0000 7f4587fff700 1 --2- 172.21.15.3:0/2819635265 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458809a3b0 0x7f458806fb20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f457c009c70 tx=0x7f457c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:34.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.348+0000 7f45857fa700 1 -- 172.21.15.3:0/2819635265 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c0036a0 con 0x7f458809a3b0 2023-10-03T10:13:34.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.348+0000 7f45857fa700 1 -- 172.21.15.3:0/2819635265 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f457c003800 con 0x7f458809a3b0 2023-10-03T10:13:34.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.348+0000 7f458eac7700 1 -- 172.21.15.3:0/2819635265 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458811dc00 con 0x7f458809a3b0 2023-10-03T10:13:34.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.348+0000 7f458eac7700 1 -- 172.21.15.3:0/2819635265 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458811ee30 con 0x7f458809a3b0 2023-10-03T10:13:34.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.348+0000 7f45857fa700 1 -- 172.21.15.3:0/2819635265 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c003bb0 con 0x7f458809a3b0 2023-10-03T10:13:34.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.349+0000 7f45857fa700 1 -- 172.21.15.3:0/2819635265 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f457c003d10 con 0x7f458809a3b0 2023-10-03T10:13:34.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.349+0000 7f45857fa700 1 -- 172.21.15.3:0/2819635265 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f457c021030 con 0x7f458809a3b0 2023-10-03T10:13:34.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.349+0000 7f458eac7700 1 -- 172.21.15.3:0/2819635265 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45680052e0 con 0x7f458809a3b0 2023-10-03T10:13:34.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.354+0000 7f45857fa700 1 -- 172.21.15.3:0/2819635265 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f457c02b080 con 0x7f458809a3b0 2023-10-03T10:13:34.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.380+0000 7f9a0b051700 1 -- 172.21.15.3:0/840440929 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9a040a2f40 con 0x7f9a04136fe0 2023-10-03T10:13:34.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.380+0000 7f9a017fa700 1 -- 172.21.15.3:0/840440929 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f99fc034090 con 0x7f9a04136fe0 2023-10-03T10:13:34.380 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.383+0000 7f9a0b051700 1 -- 172.21.15.3:0/840440929 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a04136fe0 msgr2=0x7f9a0411f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.383+0000 7f9a0b051700 1 --2- 172.21.15.3:0/840440929 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a04136fe0 0x7f9a0411f040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f99fc015000 tx=0x7f99fc004ad0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.383+0000 7f9a0b051700 1 -- 172.21.15.3:0/840440929 shutdown_connections 2023-10-03T10:13:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.383+0000 7f9a0b051700 1 --2- 172.21.15.3:0/840440929 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a04136fe0 0x7f9a0411f040 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.383+0000 7f9a0b051700 1 --2- 172.21.15.3:0/840440929 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a0409e930 0x7f9a0411e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.383+0000 7f9a0b051700 1 --2- 172.21.15.3:0/840440929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a0409f410 0x7f9a0411eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.383+0000 7f9a0b051700 1 -- 172.21.15.3:0/840440929 >> 172.21.15.3:0/840440929 conn(0x7f9a040a4210 msgr2=0x7f9a040a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.384+0000 7f9a0b051700 1 -- 172.21.15.3:0/840440929 shutdown_connections 2023-10-03T10:13:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.384+0000 7f9a0b051700 1 -- 172.21.15.3:0/840440929 wait complete. 2023-10-03T10:13:34.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.394+0000 7fdeb9fdb700 1 Processor -- start 2023-10-03T10:13:34.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.394+0000 7fdeb9fdb700 1 -- start start 2023-10-03T10:13:34.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.395+0000 7fdeb9fdb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdeb409d6f0 0x7fdeb409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:34.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.395+0000 7fdeb9fdb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdeb409e000 0x7fdeb4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:34.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.395+0000 7fdeb9fdb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdeb409e410 0x7fdeb4130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:34.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.395+0000 7fdeb9fdb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeb409f520 con 0x7fdeb409e410 2023-10-03T10:13:34.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.395+0000 7fdeb9fdb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb409f130 con 0x7fdeb409e000 2023-10-03T10:13:34.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.395+0000 7fdeb9fdb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb409e930 con 0x7fdeb409d6f0 2023-10-03T10:13:34.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.395+0000 7fdeb3fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdeb409e410 0x7fdeb4130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:34.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.395+0000 7fdeb37fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdeb409d6f0 0x7fdeb409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:34.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.395+0000 7fdeb3fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdeb409e410 0x7fdeb4130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45068/0 (socket says 172.21.15.3:45068) 2023-10-03T10:13:34.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.395+0000 7fdeb3fff700 1 -- 172.21.15.3:0/1312286480 learned_addr learned my addr 172.21.15.3:0/1312286480 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:34.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.396+0000 7fdeb2ffd700 1 --2- 172.21.15.3:0/1312286480 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdeb409e000 0x7fdeb4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:34.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.396+0000 7fdeb37fe700 1 -- 172.21.15.3:0/1312286480 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdeb409e000 msgr2=0x7fdeb4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:34.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.396+0000 7fdeb37fe700 1 --2- 172.21.15.3:0/1312286480 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdeb409e000 0x7fdeb4130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.396+0000 7fdeb37fe700 1 -- 172.21.15.3:0/1312286480 >> v2:172.21.15.3:3300/0 conn(0x7fdeb409e410 msgr2=0x7fdeb4130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:34.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.396+0000 7fdeb37fe700 1 --2- 172.21.15.3:0/1312286480 >> v2:172.21.15.3:3300/0 conn(0x7fdeb409e410 0x7fdeb4130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.396+0000 7fdeb37fe700 1 -- 172.21.15.3:0/1312286480 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb409b2d0 con 0x7fdeb409d6f0 2023-10-03T10:13:34.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.396+0000 7fdeb2ffd700 1 --2- 172.21.15.3:0/1312286480 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdeb409e000 0x7fdeb4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:34.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.396+0000 7fdeb37fe700 1 --2- 172.21.15.3:0/1312286480 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdeb409d6f0 0x7fdeb409dac0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009870 tx=0x7fdea4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b45fdbf1b145302a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:34.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.396+0000 7fdeb27fc700 1 -- 172.21.15.3:0/1312286480 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4011830 con 0x7fdeb409d6f0 2023-10-03T10:13:34.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.396+0000 7fdeb27fc700 1 -- 172.21.15.3:0/1312286480 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea4011990 con 0x7fdeb409d6f0 2023-10-03T10:13:34.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.396+0000 7fdeb27fc700 1 -- 172.21.15.3:0/1312286480 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4011d20 con 0x7fdeb409d6f0 2023-10-03T10:13:34.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.397+0000 7fdeb9fdb700 1 -- 172.21.15.3:0/1312286480 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdeb409d6f0 msgr2=0x7fdeb409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:34.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.397+0000 7fdeb9fdb700 1 --2- 172.21.15.3:0/1312286480 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdeb409d6f0 0x7fdeb409dac0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009870 tx=0x7fdea4009b80 comp rx=0 tx=0).stop 2023-10-03T10:13:34.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.400+0000 7fdeb9fdb700 1 -- 172.21.15.3:0/1312286480 shutdown_connections 2023-10-03T10:13:34.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.400+0000 7fdeb9fdb700 1 --2- 172.21.15.3:0/1312286480 >> v2:172.21.15.3:3300/0 conn(0x7fdeb409e410 0x7fdeb4130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.400+0000 7fdeb9fdb700 1 --2- 172.21.15.3:0/1312286480 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdeb409d6f0 0x7fdeb409dac0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.400+0000 7fdeb9fdb700 1 --2- 172.21.15.3:0/1312286480 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdeb409e000 0x7fdeb4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.400+0000 7fdeb9fdb700 1 -- 172.21.15.3:0/1312286480 >> 172.21.15.3:0/1312286480 conn(0x7fdeb4066f50 msgr2=0x7fdeb409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:34.401 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:34.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.400+0000 7fdeb9fdb700 1 -- 172.21.15.3:0/1312286480 shutdown_connections 2023-10-03T10:13:34.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.400+0000 7fdeb9fdb700 1 -- 172.21.15.3:0/1312286480 wait complete. 2023-10-03T10:13:34.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.401+0000 7fdeb9fdb700 1 Processor -- start 2023-10-03T10:13:34.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.401+0000 7fdeb9fdb700 1 -- start start 2023-10-03T10:13:34.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.402+0000 7fdeb9fdb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdeb409d6f0 0x7fdeb41370d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:34.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.402+0000 7fdeb9fdb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdeb409e000 0x7fdeb4137610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:34.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.402+0000 7fdeb9fdb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdeb409e410 0x7fdeb41c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:34.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.402+0000 7fdeb9fdb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb4119fd0 con 0x7fdeb409d6f0 2023-10-03T10:13:34.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.402+0000 7fdeb9fdb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb4136ce0 con 0x7fdeb409e410 2023-10-03T10:13:34.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.402+0000 7fdeb9fdb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb4072380 con 0x7fdeb409e000 2023-10-03T10:13:34.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.402+0000 7fdeb2ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdeb409e000 0x7fdeb4137610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:34.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.402+0000 7fdeb37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdeb409d6f0 0x7fdeb41370d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:34.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.402+0000 7fdeb3fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdeb409e410 0x7fdeb41c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:34.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.402+0000 7fdeb37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdeb409d6f0 0x7fdeb41370d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45078/0 (socket says 172.21.15.3:45078) 2023-10-03T10:13:34.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.402+0000 7fdeb2ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdeb409e000 0x7fdeb4137610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56762/0 (socket says 172.21.15.3:56762) 2023-10-03T10:13:34.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.402+0000 7fdeb37fe700 1 -- 172.21.15.3:0/570736869 learned_addr learned my addr 172.21.15.3:0/570736869 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:34.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.403+0000 7fdeb3fff700 1 -- 172.21.15.3:0/570736869 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdeb409e000 msgr2=0x7fdeb4137610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:34.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.403+0000 7fdeb3fff700 1 --2- 172.21.15.3:0/570736869 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdeb409e000 0x7fdeb4137610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.403+0000 7fdeb3fff700 1 -- 172.21.15.3:0/570736869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdeb409d6f0 msgr2=0x7fdeb41370d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:34.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.403+0000 7fdeb3fff700 1 --2- 172.21.15.3:0/570736869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdeb409d6f0 0x7fdeb41370d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.403+0000 7fdeb3fff700 1 -- 172.21.15.3:0/570736869 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea40094a0 con 0x7fdeb409e410 2023-10-03T10:13:34.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.403+0000 7fdeb37fe700 1 --2- 172.21.15.3:0/570736869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdeb409d6f0 0x7fdeb41370d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:34.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.403+0000 7fdeb3fff700 1 --2- 172.21.15.3:0/570736869 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdeb409e410 0x7fdeb41c8100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009d90 tx=0x7fdeac00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:34.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.403+0000 7fde9bfff700 1 -- 172.21.15.3:0/570736869 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac003c20 con 0x7fdeb409e410 2023-10-03T10:13:34.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.403+0000 7fde9bfff700 1 -- 172.21.15.3:0/570736869 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeac00f040 con 0x7fdeb409e410 2023-10-03T10:13:34.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.403+0000 7fdeb9fdb700 1 -- 172.21.15.3:0/570736869 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeb409c700 con 0x7fdeb409e410 2023-10-03T10:13:34.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.403+0000 7fde9bfff700 1 -- 172.21.15.3:0/570736869 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac00c880 con 0x7fdeb409e410 2023-10-03T10:13:34.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.403+0000 7fdeb9fdb700 1 -- 172.21.15.3:0/570736869 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb4138370 con 0x7fdeb409e410 2023-10-03T10:13:34.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.404+0000 7fde9bfff700 1 -- 172.21.15.3:0/570736869 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdeac00c9e0 con 0x7fdeb409e410 2023-10-03T10:13:34.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.405+0000 7fde9bfff700 1 -- 172.21.15.3:0/570736869 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fdeac03f930 con 0x7fdeb409e410 2023-10-03T10:13:34.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.406+0000 7fdeb9fdb700 1 -- 172.21.15.3:0/570736869 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeb411a110 con 0x7fdeb409e410 2023-10-03T10:13:34.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.411+0000 7fde9bfff700 1 -- 172.21.15.3:0/570736869 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdeb411a110 con 0x7fdeb409e410 2023-10-03T10:13:34.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.557+0000 7fe67b67b700 1 -- 172.21.15.3:0/177363590 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe6741396f0 con 0x7fe67409e000 2023-10-03T10:13:34.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.557+0000 7fe669ffb700 1 -- 172.21.15.3:0/177363590 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe664015030 con 0x7fe67409e000 2023-10-03T10:13:34.557 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:34.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.559+0000 7fe67b67b700 1 -- 172.21.15.3:0/177363590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe67409e000 msgr2=0x7fe674131bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:34.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.559+0000 7fe67b67b700 1 --2- 172.21.15.3:0/177363590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe67409e000 0x7fe674131bb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe664013000 tx=0x7fe664004750 comp rx=0 tx=0).stop 2023-10-03T10:13:34.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.559+0000 7fe67b67b700 1 -- 172.21.15.3:0/177363590 shutdown_connections 2023-10-03T10:13:34.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.559+0000 7fe67b67b700 1 --2- 172.21.15.3:0/177363590 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe67409e410 0x7fe6741320f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.559+0000 7fe67b67b700 1 --2- 172.21.15.3:0/177363590 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe67409d6f0 0x7fe674131670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.559+0000 7fe67b67b700 1 --2- 172.21.15.3:0/177363590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe67409e000 0x7fe674131bb0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.560+0000 7fe67b67b700 1 -- 172.21.15.3:0/177363590 >> 172.21.15.3:0/177363590 conn(0x7fe674066f50 msgr2=0x7fe674070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:34.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.560+0000 7fe67b67b700 1 -- 172.21.15.3:0/177363590 shutdown_connections 2023-10-03T10:13:34.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.560+0000 7fe67b67b700 1 -- 172.21.15.3:0/177363590 wait complete. 2023-10-03T10:13:34.573 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:34.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.608+0000 7f458eac7700 1 -- 172.21.15.3:0/2819635265 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4568005150 con 0x7f458809a3b0 2023-10-03T10:13:34.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.608+0000 7f45857fa700 1 -- 172.21.15.3:0/2819635265 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f457c015020 con 0x7f458809a3b0 2023-10-03T10:13:34.608 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:34.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.610+0000 7f458eac7700 1 -- 172.21.15.3:0/2819635265 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458809a3b0 msgr2=0x7f458806fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:34.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.610+0000 7f458eac7700 1 --2- 172.21.15.3:0/2819635265 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458809a3b0 0x7f458806fb20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f457c009c70 tx=0x7f457c004750 comp rx=0 tx=0).stop 2023-10-03T10:13:34.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.611+0000 7f458eac7700 1 -- 172.21.15.3:0/2819635265 shutdown_connections 2023-10-03T10:13:34.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.611+0000 7f458eac7700 1 --2- 172.21.15.3:0/2819635265 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4588131810 0x7f4588070060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.611+0000 7f458eac7700 1 --2- 172.21.15.3:0/2819635265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4588099a40 0x7f458806f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.611+0000 7f458eac7700 1 --2- 172.21.15.3:0/2819635265 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458809a3b0 0x7f458806fb20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.611+0000 7f458eac7700 1 -- 172.21.15.3:0/2819635265 >> 172.21.15.3:0/2819635265 conn(0x7f45880a2aa0 msgr2=0x7f458812e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:34.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.611+0000 7f458eac7700 1 -- 172.21.15.3:0/2819635265 shutdown_connections 2023-10-03T10:13:34.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.611+0000 7f458eac7700 1 -- 172.21.15.3:0/2819635265 wait complete. 2023-10-03T10:13:34.624 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:34.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.645+0000 7fdeb9fdb700 1 -- 172.21.15.3:0/570736869 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdeb411a110 con 0x7fdeb409e410 2023-10-03T10:13:34.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.646+0000 7fde9bfff700 1 -- 172.21.15.3:0/570736869 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdeb411a110 con 0x7fdeb409e410 2023-10-03T10:13:34.646 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:34.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.648+0000 7fdeb9fdb700 1 -- 172.21.15.3:0/570736869 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdeb409e410 msgr2=0x7fdeb41c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:34.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.648+0000 7fdeb9fdb700 1 --2- 172.21.15.3:0/570736869 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdeb409e410 0x7fdeb41c8100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009d90 tx=0x7fdeac00c3a0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.648+0000 7fdeb9fdb700 1 -- 172.21.15.3:0/570736869 shutdown_connections 2023-10-03T10:13:34.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.648+0000 7fdeb9fdb700 1 --2- 172.21.15.3:0/570736869 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdeb409e410 0x7fdeb41c8100 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.648+0000 7fdeb9fdb700 1 --2- 172.21.15.3:0/570736869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdeb409d6f0 0x7fdeb41370d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.648+0000 7fdeb9fdb700 1 --2- 172.21.15.3:0/570736869 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdeb409e000 0x7fdeb4137610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:34.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.648+0000 7fdeb9fdb700 1 -- 172.21.15.3:0/570736869 >> 172.21.15.3:0/570736869 conn(0x7fdeb4066f50 msgr2=0x7fdeb40a18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:34.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.648+0000 7fdeb9fdb700 1 -- 172.21.15.3:0/570736869 shutdown_connections 2023-10-03T10:13:34.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:34.648+0000 7fdeb9fdb700 1 -- 172.21.15.3:0/570736869 wait complete. 2023-10-03T10:13:34.662 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:35.402 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:35.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.548+0000 7fd47daa3700 1 Processor -- start 2023-10-03T10:13:35.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.548+0000 7fd47daa3700 1 -- start start 2023-10-03T10:13:35.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.549+0000 7fd47daa3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd478136820 0x7fd478138c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:35.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.549+0000 7fd47daa3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd478139140 0x7fd47813b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:35.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.549+0000 7fd47daa3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd47806bc00 con 0x7fd478133a80 2023-10-03T10:13:35.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.549+0000 7fd47daa3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd47806c570 con 0x7fd478139140 2023-10-03T10:13:35.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.549+0000 7fd47daa3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd4780a0040 con 0x7fd478136820 2023-10-03T10:13:35.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.549+0000 7fd4777fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd478133a80 0x7fd478133e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35326/0 (socket says 172.21.15.3:35326) 2023-10-03T10:13:35.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.549+0000 7fd4777fe700 1 -- 172.21.15.3:0/539237967 learned_addr learned my addr 172.21.15.3:0/539237967 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:35.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.549+0000 7fd476ffd700 1 --2- 172.21.15.3:0/539237967 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd478136820 0x7fd478138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:35.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.549+0000 7fd477fff700 1 --2- 172.21.15.3:0/539237967 >> v2:172.21.15.3:3300/0 conn(0x7fd478139140 0x7fd47813b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:35.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.549+0000 7fd476ffd700 1 -- 172.21.15.3:0/539237967 >> v2:172.21.15.3:3300/0 conn(0x7fd478139140 msgr2=0x7fd47813b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:35.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.549+0000 7fd476ffd700 1 --2- 172.21.15.3:0/539237967 >> v2:172.21.15.3:3300/0 conn(0x7fd478139140 0x7fd47813b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:35.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.549+0000 7fd476ffd700 1 -- 172.21.15.3:0/539237967 >> v1:172.21.15.3:6789/0 conn(0x7fd478133a80 legacy=0x7fd478133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:35.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.549+0000 7fd476ffd700 1 -- 172.21.15.3:0/539237967 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd478133860 con 0x7fd478136820 2023-10-03T10:13:35.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.549+0000 7fd4767fc700 1 -- 172.21.15.3:0/539237967 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2605310222 0 0) 0x7fd47806bc00 con 0x7fd478133a80 2023-10-03T10:13:35.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.549+0000 7fd477fff700 1 --2- 172.21.15.3:0/539237967 >> v2:172.21.15.3:3300/0 conn(0x7fd478139140 0x7fd47813b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:35.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.549+0000 7fd476ffd700 1 --2- 172.21.15.3:0/539237967 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd478136820 0x7fd478138c00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd460009840 tx=0x7fd460009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1831dbbe8208f9e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:35.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.550+0000 7fd4767fc700 1 -- 172.21.15.3:0/539237967 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd460011830 con 0x7fd478136820 2023-10-03T10:13:35.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.550+0000 7fd4767fc700 1 -- 172.21.15.3:0/539237967 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd460011990 con 0x7fd478136820 2023-10-03T10:13:35.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.550+0000 7fd4767fc700 1 -- 172.21.15.3:0/539237967 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd460011d20 con 0x7fd478136820 2023-10-03T10:13:35.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.550+0000 7fd47daa3700 1 -- 172.21.15.3:0/539237967 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd478136820 msgr2=0x7fd478138c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:35.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.550+0000 7fd47daa3700 1 --2- 172.21.15.3:0/539237967 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd478136820 0x7fd478138c00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd460009840 tx=0x7fd460009b50 comp rx=0 tx=0).stop 2023-10-03T10:13:35.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.550+0000 7fd47daa3700 1 -- 172.21.15.3:0/539237967 shutdown_connections 2023-10-03T10:13:35.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.550+0000 7fd47daa3700 1 --2- 172.21.15.3:0/539237967 >> v2:172.21.15.3:3300/0 conn(0x7fd478139140 0x7fd47813b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:35.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.550+0000 7fd47daa3700 1 --2- 172.21.15.3:0/539237967 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd478136820 0x7fd478138c00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:35.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.550+0000 7fd47daa3700 1 -- 172.21.15.3:0/539237967 >> 172.21.15.3:0/539237967 conn(0x7fd4780a4210 msgr2=0x7fd4780a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:35.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.550+0000 7fd47daa3700 1 -- 172.21.15.3:0/539237967 shutdown_connections 2023-10-03T10:13:35.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.550+0000 7fd47daa3700 1 -- 172.21.15.3:0/539237967 wait complete. 2023-10-03T10:13:35.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.551+0000 7fd47daa3700 1 Processor -- start 2023-10-03T10:13:35.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.552+0000 7fd47daa3700 1 -- start start 2023-10-03T10:13:35.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.552+0000 7fd47daa3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd478133a80 0x7fd47812ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:35.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.552+0000 7fd47daa3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd478136820 0x7fd47809d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:35.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.552+0000 7fd47daa3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd478139140 0x7fd47809dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:35.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.552+0000 7fd47daa3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47809d0d0 con 0x7fd478136820 2023-10-03T10:13:35.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.552+0000 7fd47daa3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd478130460 con 0x7fd478133a80 2023-10-03T10:13:35.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.552+0000 7fd47daa3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd4780726b0 con 0x7fd478139140 2023-10-03T10:13:35.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.552+0000 7fd4777fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd478133a80 0x7fd47812ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:35.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.552+0000 7fd477fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd478139140 0x7fd47809dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:35.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.552+0000 7fd4777fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd478133a80 0x7fd47812ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44500/0 (socket says 172.21.15.3:44500) 2023-10-03T10:13:35.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.552+0000 7fd4777fe700 1 -- 172.21.15.3:0/2660252228 learned_addr learned my addr 172.21.15.3:0/2660252228 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:35.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.552+0000 7fd476ffd700 1 --2- 172.21.15.3:0/2660252228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd478136820 0x7fd47809d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:35.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.552+0000 7fd476ffd700 1 -- 172.21.15.3:0/2660252228 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd478139140 msgr2=0x7fd47809dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:35.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.552+0000 7fd476ffd700 1 --2- 172.21.15.3:0/2660252228 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd478139140 0x7fd47809dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:35.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.552+0000 7fd476ffd700 1 -- 172.21.15.3:0/2660252228 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd478133a80 msgr2=0x7fd47812ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:35.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.552+0000 7fd476ffd700 1 --2- 172.21.15.3:0/2660252228 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd478133a80 0x7fd47812ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:35.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.552+0000 7fd476ffd700 1 -- 172.21.15.3:0/2660252228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd470009a70 con 0x7fd478136820 2023-10-03T10:13:35.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.553+0000 7fd477fff700 1 --2- 172.21.15.3:0/2660252228 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd478139140 0x7fd47809dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.553+0000 7fd476ffd700 1 --2- 172.21.15.3:0/2660252228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd478136820 0x7fd47809d690 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd460016000 tx=0x7fd460004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.553+0000 7fd46ffff700 1 -- 172.21.15.3:0/2660252228 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4600045d0 con 0x7fd478136820 2023-10-03T10:13:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.553+0000 7fd46ffff700 1 -- 172.21.15.3:0/2660252228 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46001f070 con 0x7fd478136820 2023-10-03T10:13:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.553+0000 7fd46ffff700 1 -- 172.21.15.3:0/2660252228 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4600038e0 con 0x7fd478136820 2023-10-03T10:13:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.553+0000 7fd47daa3700 1 -- 172.21.15.3:0/2660252228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4600094a0 con 0x7fd478136820 2023-10-03T10:13:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.553+0000 7fd47daa3700 1 -- 172.21.15.3:0/2660252228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd478071f80 con 0x7fd478136820 2023-10-03T10:13:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.554+0000 7fd46ffff700 1 -- 172.21.15.3:0/2660252228 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd460003a40 con 0x7fd478136820 2023-10-03T10:13:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.554+0000 7fd46ffff700 1 -- 172.21.15.3:0/2660252228 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd460021070 con 0x7fd478136820 2023-10-03T10:13:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.554+0000 7fd47daa3700 1 -- 172.21.15.3:0/2660252228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd478074ea0 con 0x7fd478136820 2023-10-03T10:13:35.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.559+0000 7fd46ffff700 1 -- 172.21.15.3:0/2660252228 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd46002b020 con 0x7fd478136820 2023-10-03T10:13:35.574 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:35.625 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:35.663 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:35.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.716+0000 7f0c1e8b6700 1 Processor -- start 2023-10-03T10:13:35.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.716+0000 7f0c1e8b6700 1 -- start start 2023-10-03T10:13:35.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.717+0000 7f0c1e8b6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0c1806a770 0x7f0c1809d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:35.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.717+0000 7f0c1e8b6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c1809dc30 0x7f0c1809e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:35.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.717+0000 7f0c1e8b6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c1809a810 con 0x7f0c1813c1e0 2023-10-03T10:13:35.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.717+0000 7f0c1e8b6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c1806ab40 con 0x7f0c1806a770 2023-10-03T10:13:35.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.717+0000 7f0c1e8b6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1809ebc0 con 0x7f0c1809dc30 2023-10-03T10:13:35.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.717+0000 7f0c17fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0c1806a770 0x7f0c1809d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:35.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.717+0000 7f0c17fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0c1806a770 0x7f0c1809d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45098/0 (socket says 172.21.15.3:45098) 2023-10-03T10:13:35.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.717+0000 7f0c17fff700 1 -- 172.21.15.3:0/3177502623 learned_addr learned my addr 172.21.15.3:0/3177502623 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:35.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.717+0000 7f0c177fe700 1 --2- 172.21.15.3:0/3177502623 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c1809dc30 0x7f0c1809e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:35.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.717+0000 7f0c177fe700 1 -- 172.21.15.3:0/3177502623 >> v2:172.21.15.3:3300/0 conn(0x7f0c1806a770 msgr2=0x7f0c1809d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:35.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.717+0000 7f0c177fe700 1 --2- 172.21.15.3:0/3177502623 >> v2:172.21.15.3:3300/0 conn(0x7f0c1806a770 0x7f0c1809d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:35.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.717+0000 7f0c177fe700 1 -- 172.21.15.3:0/3177502623 >> v1:172.21.15.3:6789/0 conn(0x7f0c1813c1e0 legacy=0x7f0c1813e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:35.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.717+0000 7f0c177fe700 1 -- 172.21.15.3:0/3177502623 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1809e920 con 0x7f0c1809dc30 2023-10-03T10:13:35.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.718+0000 7f0c177fe700 1 --2- 172.21.15.3:0/3177502623 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c1809dc30 0x7f0c1809e080 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c009840 tx=0x7f0c0c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a905840eae418df server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:35.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.718+0000 7f0c16ffd700 1 -- 172.21.15.3:0/3177502623 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c011830 con 0x7f0c1809dc30 2023-10-03T10:13:35.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.718+0000 7f0c16ffd700 1 -- 172.21.15.3:0/3177502623 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0c011990 con 0x7f0c1809dc30 2023-10-03T10:13:35.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.718+0000 7f0c16ffd700 1 -- 172.21.15.3:0/3177502623 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c011d20 con 0x7f0c1809dc30 2023-10-03T10:13:35.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.719+0000 7f0c1e8b6700 1 -- 172.21.15.3:0/3177502623 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c1809dc30 msgr2=0x7f0c1809e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:35.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.719+0000 7f0c1e8b6700 1 --2- 172.21.15.3:0/3177502623 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c1809dc30 0x7f0c1809e080 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c009840 tx=0x7f0c0c009b50 comp rx=0 tx=0).stop 2023-10-03T10:13:35.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.719+0000 7f0c1e8b6700 1 -- 172.21.15.3:0/3177502623 shutdown_connections 2023-10-03T10:13:35.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.719+0000 7f0c1e8b6700 1 --2- 172.21.15.3:0/3177502623 >> v2:172.21.15.3:3300/0 conn(0x7f0c1806a770 0x7f0c1809d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:35.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.719+0000 7f0c1e8b6700 1 --2- 172.21.15.3:0/3177502623 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c1809dc30 0x7f0c1809e080 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:35.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.719+0000 7f0c1e8b6700 1 -- 172.21.15.3:0/3177502623 >> 172.21.15.3:0/3177502623 conn(0x7f0c18066f50 msgr2=0x7f0c1809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:35.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.719+0000 7f0c1e8b6700 1 -- 172.21.15.3:0/3177502623 shutdown_connections 2023-10-03T10:13:35.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.719+0000 7f0c1e8b6700 1 -- 172.21.15.3:0/3177502623 wait complete. 2023-10-03T10:13:35.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.720+0000 7f0c1e8b6700 1 Processor -- start 2023-10-03T10:13:35.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.720+0000 7f0c1e8b6700 1 -- start start 2023-10-03T10:13:35.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.721+0000 7f0c1e8b6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c1806a770 0x7f0c1811b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:35.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.721+0000 7f0c1e8b6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c1809dc30 0x7f0c1813a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:35.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.721+0000 7f0c1e8b6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c1813c1e0 0x7f0c1813acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:35.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.721+0000 7f0c1e8b6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1811d500 con 0x7f0c1813c1e0 2023-10-03T10:13:35.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.721+0000 7f0c1e8b6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1811c0b0 con 0x7f0c1809dc30 2023-10-03T10:13:35.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.721+0000 7f0c1e8b6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1811dda0 con 0x7f0c1806a770 2023-10-03T10:13:35.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.721+0000 7f0c177fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c1809dc30 0x7f0c1813a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:35.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.721+0000 7f0c1ce53700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c1813c1e0 0x7f0c1813acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:35.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.721+0000 7f0c177fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c1809dc30 0x7f0c1813a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44532/0 (socket says 172.21.15.3:44532) 2023-10-03T10:13:35.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.721+0000 7f0c177fe700 1 -- 172.21.15.3:0/1966707731 learned_addr learned my addr 172.21.15.3:0/1966707731 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:35.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.722+0000 7f0c17fff700 1 --2- 172.21.15.3:0/1966707731 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c1806a770 0x7f0c1811b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.722+0000 7f0c177fe700 1 -- 172.21.15.3:0/1966707731 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c1806a770 msgr2=0x7f0c1811b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.722+0000 7f0c177fe700 1 --2- 172.21.15.3:0/1966707731 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c1806a770 0x7f0c1811b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.722+0000 7f0c177fe700 1 -- 172.21.15.3:0/1966707731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c1813c1e0 msgr2=0x7f0c1813acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.722+0000 7f0c177fe700 1 --2- 172.21.15.3:0/1966707731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c1813c1e0 0x7f0c1813acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.722+0000 7f0c177fe700 1 -- 172.21.15.3:0/1966707731 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0c0094a0 con 0x7f0c1809dc30 2023-10-03T10:13:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.722+0000 7f0c1ce53700 1 --2- 172.21.15.3:0/1966707731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c1813c1e0 0x7f0c1813acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.722+0000 7f0c177fe700 1 --2- 172.21.15.3:0/1966707731 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c1809dc30 0x7f0c1813a780 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0c1809a9a0 tx=0x7f0c0c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.722+0000 7f0c14ff9700 1 -- 172.21.15.3:0/1966707731 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c0036a0 con 0x7f0c1809dc30 2023-10-03T10:13:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.722+0000 7f0c14ff9700 1 -- 172.21.15.3:0/1966707731 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0c01f070 con 0x7f0c1809dc30 2023-10-03T10:13:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.722+0000 7f0c1e8b6700 1 -- 172.21.15.3:0/1966707731 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c18071ee0 con 0x7f0c1809dc30 2023-10-03T10:13:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.722+0000 7f0c14ff9700 1 -- 172.21.15.3:0/1966707731 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c003a40 con 0x7f0c1809dc30 2023-10-03T10:13:35.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.722+0000 7f0c1e8b6700 1 -- 172.21.15.3:0/1966707731 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c1811e810 con 0x7f0c1809dc30 2023-10-03T10:13:35.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.723+0000 7f0c14ff9700 1 -- 172.21.15.3:0/1966707731 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0c0c003ba0 con 0x7f0c1809dc30 2023-10-03T10:13:35.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.724+0000 7f0c14ff9700 1 -- 172.21.15.3:0/1966707731 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0c0c021070 con 0x7f0c1809dc30 2023-10-03T10:13:35.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.724+0000 7f0c1e8b6700 1 -- 172.21.15.3:0/1966707731 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf80052e0 con 0x7f0c1809dc30 2023-10-03T10:13:35.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.733+0000 7f0c14ff9700 1 -- 172.21.15.3:0/1966707731 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0c0c06c750 con 0x7f0c1809dc30 2023-10-03T10:13:35.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.771+0000 7f6a7bc7f700 1 Processor -- start 2023-10-03T10:13:35.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.772+0000 7f6a7bc7f700 1 -- start start 2023-10-03T10:13:35.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.772+0000 7f6a7bc7f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6a740a0a30 0x7f6a740a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:35.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.772+0000 7f6a7bc7f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6a7409d630 0x7f6a740a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:35.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.772+0000 7f6a7bc7f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a740a2a70 con 0x7f6a74133930 2023-10-03T10:13:35.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.772+0000 7f6a7bc7f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a740a01d0 con 0x7f6a740a0a30 2023-10-03T10:13:35.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.772+0000 7f6a7bc7f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7409e2b0 con 0x7f6a7409d630 2023-10-03T10:13:35.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.772+0000 7f6a79a1b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6a740a0a30 0x7f6a740a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:35.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.772+0000 7f6a79a1b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6a740a0a30 0x7f6a740a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45120/0 (socket says 172.21.15.3:45120) 2023-10-03T10:13:35.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.772+0000 7f6a79a1b700 1 -- 172.21.15.3:0/756913422 learned_addr learned my addr 172.21.15.3:0/756913422 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:35.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.773+0000 7f6a79a1b700 1 -- 172.21.15.3:0/756913422 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6a7409d630 msgr2=0x7f6a740a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:35.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.773+0000 7f6a79a1b700 1 --2- 172.21.15.3:0/756913422 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6a7409d630 0x7f6a740a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:35.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.773+0000 7f6a79a1b700 1 -- 172.21.15.3:0/756913422 >> v1:172.21.15.3:6789/0 conn(0x7f6a74133930 legacy=0x7f6a74133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:35.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.773+0000 7f6a79a1b700 1 -- 172.21.15.3:0/756913422 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7409dc50 con 0x7f6a740a0a30 2023-10-03T10:13:35.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.773+0000 7f6a79a1b700 1 --2- 172.21.15.3:0/756913422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a740a0a30 0x7f6a740a0e00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6a700098f0 tx=0x7f6a70009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a0d10f6b79b198 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:35.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.773+0000 7f6a78a19700 1 -- 172.21.15.3:0/756913422 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7000b8a0 con 0x7f6a740a0a30 2023-10-03T10:13:35.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.773+0000 7f6a78a19700 1 -- 172.21.15.3:0/756913422 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7000ba00 con 0x7f6a740a0a30 2023-10-03T10:13:35.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.773+0000 7f6a78a19700 1 -- 172.21.15.3:0/756913422 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7000bd90 con 0x7f6a740a0a30 2023-10-03T10:13:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.774+0000 7f6a7bc7f700 1 -- 172.21.15.3:0/756913422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a740a0a30 msgr2=0x7f6a740a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.774+0000 7f6a7bc7f700 1 --2- 172.21.15.3:0/756913422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a740a0a30 0x7f6a740a0e00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6a700098f0 tx=0x7f6a70009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.774+0000 7f6a7bc7f700 1 -- 172.21.15.3:0/756913422 shutdown_connections 2023-10-03T10:13:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.774+0000 7f6a7bc7f700 1 --2- 172.21.15.3:0/756913422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a740a0a30 0x7f6a740a0e00 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.774+0000 7f6a7bc7f700 1 --2- 172.21.15.3:0/756913422 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6a7409d630 0x7f6a740a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.774+0000 7f6a7bc7f700 1 -- 172.21.15.3:0/756913422 >> 172.21.15.3:0/756913422 conn(0x7f6a740a3050 msgr2=0x7f6a740a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.774+0000 7f6a7bc7f700 1 -- 172.21.15.3:0/756913422 shutdown_connections 2023-10-03T10:13:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.774+0000 7f6a7bc7f700 1 -- 172.21.15.3:0/756913422 wait complete. 2023-10-03T10:13:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.775+0000 7f6a7bc7f700 1 Processor -- start 2023-10-03T10:13:35.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.775+0000 7f6a7bc7f700 1 -- start start 2023-10-03T10:13:35.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.776+0000 7f6a7bc7f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a7409d630 0x7f6a74132450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:35.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.776+0000 7f6a7bc7f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6a740a0a30 0x7f6a741210f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:35.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.776+0000 7f6a7bc7f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a74133930 0x7f6a74121630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:35.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.776+0000 7f6a7bc7f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a74123a70 con 0x7f6a7409d630 2023-10-03T10:13:35.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.776+0000 7f6a7bc7f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a74125080 con 0x7f6a74133930 2023-10-03T10:13:35.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.776+0000 7f6a7bc7f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6a741245e0 con 0x7f6a740a0a30 2023-10-03T10:13:35.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.776+0000 7f6a7921a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6a740a0a30 0x7f6a741210f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:35.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.776+0000 7f6a7921a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6a740a0a30 0x7f6a741210f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56808/0 (socket says 172.21.15.3:56808) 2023-10-03T10:13:35.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.776+0000 7f6a7921a700 1 -- 172.21.15.3:0/1293919046 learned_addr learned my addr 172.21.15.3:0/1293919046 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:35.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.776+0000 7f6a7a21c700 1 --2- 172.21.15.3:0/1293919046 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a74133930 0x7f6a74121630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:35.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.776+0000 7f6a79a1b700 1 --2- 172.21.15.3:0/1293919046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a7409d630 0x7f6a74132450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:35.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.776+0000 7f6a79a1b700 1 -- 172.21.15.3:0/1293919046 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6a740a0a30 msgr2=0x7f6a741210f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:35.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.776+0000 7f6a79a1b700 1 --2- 172.21.15.3:0/1293919046 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6a740a0a30 0x7f6a741210f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:35.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.776+0000 7f6a79a1b700 1 -- 172.21.15.3:0/1293919046 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a74133930 msgr2=0x7f6a74121630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:35.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.776+0000 7f6a79a1b700 1 --2- 172.21.15.3:0/1293919046 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a74133930 0x7f6a74121630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:35.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.776+0000 7f6a79a1b700 1 -- 172.21.15.3:0/1293919046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a70009550 con 0x7f6a7409d630 2023-10-03T10:13:35.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.776+0000 7f6a7921a700 1 --2- 172.21.15.3:0/1293919046 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6a740a0a30 0x7f6a741210f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:35.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.776+0000 7f6a79a1b700 1 --2- 172.21.15.3:0/1293919046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a7409d630 0x7f6a74132450 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6a70016000 tx=0x7f6a70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:35.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.777+0000 7f6a6a7fc700 1 -- 172.21.15.3:0/1293919046 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70004510 con 0x7f6a7409d630 2023-10-03T10:13:35.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.777+0000 7f6a6a7fc700 1 -- 172.21.15.3:0/1293919046 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a700036a0 con 0x7f6a7409d630 2023-10-03T10:13:35.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.777+0000 7f6a7bc7f700 1 -- 172.21.15.3:0/1293919046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a740a0410 con 0x7f6a7409d630 2023-10-03T10:13:35.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.777+0000 7fd47daa3700 1 -- 172.21.15.3:0/2660252228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd47806b250 con 0x7fd478136820 2023-10-03T10:13:35.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.777+0000 7f6a6a7fc700 1 -- 172.21.15.3:0/1293919046 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70003a10 con 0x7f6a7409d630 2023-10-03T10:13:35.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.777+0000 7f6a7bc7f700 1 -- 172.21.15.3:0/1293919046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a74122c70 con 0x7f6a7409d630 2023-10-03T10:13:35.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.778+0000 7fd46ffff700 1 -- 172.21.15.3:0/2660252228 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd460015070 con 0x7fd478136820 2023-10-03T10:13:35.780 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.778+0000 7f6a6a7fc700 1 -- 172.21.15.3:0/1293919046 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6a70021070 con 0x7f6a7409d630 2023-10-03T10:13:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.778+0000 7f6a7bc7f700 1 -- 172.21.15.3:0/1293919046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a7411df60 con 0x7f6a7409d630 2023-10-03T10:13:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.778+0000 7f6a6a7fc700 1 -- 172.21.15.3:0/1293919046 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6a70040c00 con 0x7f6a7409d630 2023-10-03T10:13:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.780+0000 7fd47daa3700 1 -- 172.21.15.3:0/2660252228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd478136820 msgr2=0x7fd47809d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.780+0000 7fd47daa3700 1 --2- 172.21.15.3:0/2660252228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd478136820 0x7fd47809d690 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd460016000 tx=0x7fd460004750 comp rx=0 tx=0).stop 2023-10-03T10:13:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.780+0000 7fd47daa3700 1 -- 172.21.15.3:0/2660252228 shutdown_connections 2023-10-03T10:13:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.780+0000 7fd47daa3700 1 --2- 172.21.15.3:0/2660252228 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd478139140 0x7fd47809dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.780+0000 7fd47daa3700 1 --2- 172.21.15.3:0/2660252228 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd478133a80 0x7fd47812ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.780+0000 7fd47daa3700 1 --2- 172.21.15.3:0/2660252228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd478136820 0x7fd47809d690 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.780+0000 7fd47daa3700 1 -- 172.21.15.3:0/2660252228 >> 172.21.15.3:0/2660252228 conn(0x7fd4780a4210 msgr2=0x7fd478134600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.780+0000 7fd47daa3700 1 -- 172.21.15.3:0/2660252228 shutdown_connections 2023-10-03T10:13:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.780+0000 7fd47daa3700 1 -- 172.21.15.3:0/2660252228 wait complete. 2023-10-03T10:13:35.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.785+0000 7f6a6a7fc700 1 -- 172.21.15.3:0/1293919046 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6a7001a450 con 0x7f6a7409d630 2023-10-03T10:13:35.802 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:35.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.831+0000 7f32595fd700 1 Processor -- start 2023-10-03T10:13:35.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.832+0000 7f32595fd700 1 -- start start 2023-10-03T10:13:35.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.832+0000 7f32595fd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f325406a770 0x7f325406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:35.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.832+0000 7f32595fd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f325406b080 0x7f3254131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:35.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.832+0000 7f32595fd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3254131b40 0x7f3254131fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:35.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.832+0000 7f32595fd700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f325409ece0 con 0x7f325406a770 2023-10-03T10:13:35.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.832+0000 7f32595fd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f325409e760 con 0x7f325406b080 2023-10-03T10:13:35.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.832+0000 7f32595fd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f325409e040 con 0x7f3254131b40 2023-10-03T10:13:35.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.832+0000 7f32527fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f325406b080 0x7f3254131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:35.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.832+0000 7f3252ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f325406a770 0x7f325406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:35.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.832+0000 7f32527fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f325406b080 0x7f3254131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44542/0 (socket says 172.21.15.3:44542) 2023-10-03T10:13:35.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.832+0000 7f32527fc700 1 -- 172.21.15.3:0/1081944469 learned_addr learned my addr 172.21.15.3:0/1081944469 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:35.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.832+0000 7f32537fe700 1 --2- 172.21.15.3:0/1081944469 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3254131b40 0x7f3254131fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:35.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.833+0000 7f3252ffd700 1 -- 172.21.15.3:0/1081944469 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3254131b40 msgr2=0x7f3254131fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:35.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.833+0000 7f3252ffd700 1 --2- 172.21.15.3:0/1081944469 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3254131b40 0x7f3254131fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:35.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.833+0000 7f3252ffd700 1 -- 172.21.15.3:0/1081944469 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f325406b080 msgr2=0x7f3254131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:35.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.833+0000 7f3252ffd700 1 --2- 172.21.15.3:0/1081944469 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f325406b080 0x7f3254131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:35.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.833+0000 7f3252ffd700 1 -- 172.21.15.3:0/1081944469 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f325409f630 con 0x7f325406a770 2023-10-03T10:13:35.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.833+0000 7f32527fc700 1 --2- 172.21.15.3:0/1081944469 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f325406b080 0x7f3254131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:35.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.833+0000 7f3252ffd700 1 --2- 172.21.15.3:0/1081944469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f325406a770 0x7f325406ab40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3244009840 tx=0x7f3244009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730238cd61b7521e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:35.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.833+0000 7f3251ffb700 1 -- 172.21.15.3:0/1081944469 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3244011830 con 0x7f325406a770 2023-10-03T10:13:35.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.833+0000 7f3251ffb700 1 -- 172.21.15.3:0/1081944469 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3244011990 con 0x7f325406a770 2023-10-03T10:13:35.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.833+0000 7f3251ffb700 1 -- 172.21.15.3:0/1081944469 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3244011d20 con 0x7f325406a770 2023-10-03T10:13:35.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.834+0000 7f32595fd700 1 -- 172.21.15.3:0/1081944469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f325406a770 msgr2=0x7f325406ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:35.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.834+0000 7f32595fd700 1 --2- 172.21.15.3:0/1081944469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f325406a770 0x7f325406ab40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3244009840 tx=0x7f3244009b50 comp rx=0 tx=0).stop 2023-10-03T10:13:35.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.834+0000 7f32595fd700 1 -- 172.21.15.3:0/1081944469 shutdown_connections 2023-10-03T10:13:35.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.834+0000 7f32595fd700 1 --2- 172.21.15.3:0/1081944469 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3254131b40 0x7f3254131fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:35.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.834+0000 7f32595fd700 1 --2- 172.21.15.3:0/1081944469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f325406a770 0x7f325406ab40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:35.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.834+0000 7f32595fd700 1 --2- 172.21.15.3:0/1081944469 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f325406b080 0x7f3254131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:35.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.834+0000 7f32595fd700 1 -- 172.21.15.3:0/1081944469 >> 172.21.15.3:0/1081944469 conn(0x7f3254066f50 msgr2=0x7f325409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:35.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.834+0000 7f32595fd700 1 -- 172.21.15.3:0/1081944469 shutdown_connections 2023-10-03T10:13:35.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.834+0000 7f32595fd700 1 -- 172.21.15.3:0/1081944469 wait complete. 2023-10-03T10:13:35.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.836+0000 7f32595fd700 1 Processor -- start 2023-10-03T10:13:35.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.836+0000 7f32595fd700 1 -- start start 2023-10-03T10:13:35.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.836+0000 7f32595fd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f325406a770 0x7f3254130000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:35.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.836+0000 7f32595fd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f325406b080 0x7f3254130540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:35.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.836+0000 7f32595fd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3254131b40 0x7f3254130a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:35.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.836+0000 7f32595fd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32541315f0 con 0x7f325406b080 2023-10-03T10:13:35.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.836+0000 7f32595fd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f325409c190 con 0x7f3254131b40 2023-10-03T10:13:35.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.836+0000 7f32595fd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3254071e60 con 0x7f325406a770 2023-10-03T10:13:35.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.837+0000 7f32527fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f325406b080 0x7f3254130540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:35.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.837+0000 7f3252ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f325406a770 0x7f3254130000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:35.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.837+0000 7f32527fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f325406b080 0x7f3254130540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45140/0 (socket says 172.21.15.3:45140) 2023-10-03T10:13:35.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.837+0000 7f3252ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f325406a770 0x7f3254130000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56830/0 (socket says 172.21.15.3:56830) 2023-10-03T10:13:35.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.837+0000 7f32527fc700 1 -- 172.21.15.3:0/1732250262 learned_addr learned my addr 172.21.15.3:0/1732250262 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:35.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.837+0000 7f32537fe700 1 --2- 172.21.15.3:0/1732250262 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3254131b40 0x7f3254130a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:35.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.837+0000 7f3252ffd700 1 -- 172.21.15.3:0/1732250262 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3254131b40 msgr2=0x7f3254130a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:35.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.837+0000 7f3252ffd700 1 --2- 172.21.15.3:0/1732250262 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3254131b40 0x7f3254130a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:35.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.837+0000 7f3252ffd700 1 -- 172.21.15.3:0/1732250262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f325406b080 msgr2=0x7f3254130540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:35.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.837+0000 7f3252ffd700 1 --2- 172.21.15.3:0/1732250262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f325406b080 0x7f3254130540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:35.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.837+0000 7f3252ffd700 1 -- 172.21.15.3:0/1732250262 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3248009550 con 0x7f325406a770 2023-10-03T10:13:35.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.837+0000 7f32527fc700 1 --2- 172.21.15.3:0/1732250262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f325406b080 0x7f3254130540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:35.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.837+0000 7f3252ffd700 1 --2- 172.21.15.3:0/1732250262 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f325406a770 0x7f3254130000 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3244016000 tx=0x7f3244004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:35.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.837+0000 7f323b7fe700 1 -- 172.21.15.3:0/1732250262 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3244004450 con 0x7f325406a770 2023-10-03T10:13:35.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.837+0000 7f323b7fe700 1 -- 172.21.15.3:0/1732250262 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f324401f070 con 0x7f325406a770 2023-10-03T10:13:35.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.837+0000 7f32595fd700 1 -- 172.21.15.3:0/1732250262 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32440094a0 con 0x7f325406a770 2023-10-03T10:13:35.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.837+0000 7f323b7fe700 1 -- 172.21.15.3:0/1732250262 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32440038e0 con 0x7f325406a770 2023-10-03T10:13:35.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.838+0000 7f32595fd700 1 -- 172.21.15.3:0/1732250262 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3254121b70 con 0x7f325406a770 2023-10-03T10:13:35.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.839+0000 7f323b7fe700 1 -- 172.21.15.3:0/1732250262 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3244003a40 con 0x7f325406a770 2023-10-03T10:13:35.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.839+0000 7f32595fd700 1 -- 172.21.15.3:0/1732250262 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3254131120 con 0x7f325406a770 2023-10-03T10:13:35.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.839+0000 7f323b7fe700 1 -- 172.21.15.3:0/1732250262 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3244021070 con 0x7f325406a770 2023-10-03T10:13:35.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.844+0000 7f323b7fe700 1 -- 172.21.15.3:0/1732250262 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f324402f3a0 con 0x7f325406a770 2023-10-03T10:13:35.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.990+0000 7f0c1e8b6700 1 -- 172.21.15.3:0/1966707731 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0bf8005590 con 0x7f0c1809dc30 2023-10-03T10:13:35.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.991+0000 7f0c14ff9700 1 -- 172.21.15.3:0/1966707731 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0c0c02b080 con 0x7f0c1809dc30 2023-10-03T10:13:35.991 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:35.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.993+0000 7f0c1e8b6700 1 -- 172.21.15.3:0/1966707731 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c1809dc30 msgr2=0x7f0c1813a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:35.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.993+0000 7f0c1e8b6700 1 --2- 172.21.15.3:0/1966707731 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c1809dc30 0x7f0c1813a780 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0c1809a9a0 tx=0x7f0c0c004750 comp rx=0 tx=0).stop 2023-10-03T10:13:35.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.993+0000 7f0c1e8b6700 1 -- 172.21.15.3:0/1966707731 shutdown_connections 2023-10-03T10:13:35.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.993+0000 7f0c1e8b6700 1 --2- 172.21.15.3:0/1966707731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c1813c1e0 0x7f0c1813acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:35.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.993+0000 7f0c1e8b6700 1 --2- 172.21.15.3:0/1966707731 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c1806a770 0x7f0c1811b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:35.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.993+0000 7f0c1e8b6700 1 --2- 172.21.15.3:0/1966707731 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c1809dc30 0x7f0c1813a780 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:35.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.993+0000 7f0c1e8b6700 1 -- 172.21.15.3:0/1966707731 >> 172.21.15.3:0/1966707731 conn(0x7f0c18066f50 msgr2=0x7f0c180a1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:35.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.993+0000 7f0c1e8b6700 1 -- 172.21.15.3:0/1966707731 shutdown_connections 2023-10-03T10:13:35.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:35.993+0000 7f0c1e8b6700 1 -- 172.21.15.3:0/1966707731 wait complete. 2023-10-03T10:13:36.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.001+0000 7f6a7bc7f700 1 -- 172.21.15.3:0/1293919046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6a740747f0 con 0x7f6a7409d630 2023-10-03T10:13:36.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.001+0000 7f6a6a7fc700 1 -- 172.21.15.3:0/1293919046 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6a70019ba0 con 0x7f6a7409d630 2023-10-03T10:13:36.002 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:36.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.003+0000 7f6a7bc7f700 1 -- 172.21.15.3:0/1293919046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a7409d630 msgr2=0x7f6a74132450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:36.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.003+0000 7f6a7bc7f700 1 --2- 172.21.15.3:0/1293919046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a7409d630 0x7f6a74132450 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6a70016000 tx=0x7f6a70004750 comp rx=0 tx=0).stop 2023-10-03T10:13:36.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.003+0000 7f6a7bc7f700 1 -- 172.21.15.3:0/1293919046 shutdown_connections 2023-10-03T10:13:36.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.003+0000 7f6a7bc7f700 1 --2- 172.21.15.3:0/1293919046 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a74133930 0x7f6a74121630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:36.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.003+0000 7f6a7bc7f700 1 --2- 172.21.15.3:0/1293919046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a7409d630 0x7f6a74132450 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:36.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.003+0000 7f6a7bc7f700 1 --2- 172.21.15.3:0/1293919046 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6a740a0a30 0x7f6a741210f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:36.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.004+0000 7f6a7bc7f700 1 -- 172.21.15.3:0/1293919046 >> 172.21.15.3:0/1293919046 conn(0x7f6a740a3050 msgr2=0x7f6a74133210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:36.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.005+0000 7f6a7bc7f700 1 -- 172.21.15.3:0/1293919046 shutdown_connections 2023-10-03T10:13:36.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.005+0000 7f6a7bc7f700 1 -- 172.21.15.3:0/1293919046 wait complete. 2023-10-03T10:13:36.008 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:36.019 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:36.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.071+0000 7f32595fd700 1 -- 172.21.15.3:0/1732250262 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3254074ea0 con 0x7f325406a770 2023-10-03T10:13:36.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.071+0000 7f323b7fe700 1 -- 172.21.15.3:0/1732250262 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3244015020 con 0x7f325406a770 2023-10-03T10:13:36.072 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:36.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.073+0000 7f32595fd700 1 -- 172.21.15.3:0/1732250262 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f325406a770 msgr2=0x7f3254130000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:36.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.073+0000 7f32595fd700 1 --2- 172.21.15.3:0/1732250262 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f325406a770 0x7f3254130000 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3244016000 tx=0x7f3244004750 comp rx=0 tx=0).stop 2023-10-03T10:13:36.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.073+0000 7f32595fd700 1 -- 172.21.15.3:0/1732250262 shutdown_connections 2023-10-03T10:13:36.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.073+0000 7f32595fd700 1 --2- 172.21.15.3:0/1732250262 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3254131b40 0x7f3254130a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:36.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.073+0000 7f32595fd700 1 --2- 172.21.15.3:0/1732250262 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f325406a770 0x7f3254130000 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:36.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.073+0000 7f32595fd700 1 --2- 172.21.15.3:0/1732250262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f325406b080 0x7f3254130540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:36.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.073+0000 7f32595fd700 1 -- 172.21.15.3:0/1732250262 >> 172.21.15.3:0/1732250262 conn(0x7f3254066f50 msgr2=0x7f32540700b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:36.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.073+0000 7f32595fd700 1 -- 172.21.15.3:0/1732250262 shutdown_connections 2023-10-03T10:13:36.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.073+0000 7f32595fd700 1 -- 172.21.15.3:0/1732250262 wait complete. 2023-10-03T10:13:36.088 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:36.802 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:36.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.949+0000 7f2c26294700 1 Processor -- start 2023-10-03T10:13:36.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.949+0000 7f2c26294700 1 -- start start 2023-10-03T10:13:36.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.949+0000 7f2c26294700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2c2009e930 0x7f2c2009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:36.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.949+0000 7f2c26294700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2c2009f240 0x7f2c201317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:36.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.949+0000 7f2c26294700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c2009f650 0x7f2c20131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:36.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.949+0000 7f2c26294700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c20070940 con 0x7f2c2009f240 2023-10-03T10:13:36.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.949+0000 7f2c26294700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2006fd60 con 0x7f2c2009e930 2023-10-03T10:13:36.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.949+0000 7f2c26294700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2006fa80 con 0x7f2c2009f650 2023-10-03T10:13:36.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.949+0000 7f2c1f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2c2009f240 0x7f2c201317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:36.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.949+0000 7f2c1ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2c2009e930 0x7f2c2009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:36.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.949+0000 7f2c24831700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c2009f650 0x7f2c20131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:36.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.950+0000 7f2c24831700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c2009f650 0x7f2c20131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56840/0 (socket says 172.21.15.3:56840) 2023-10-03T10:13:36.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.950+0000 7f2c1ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2c2009e930 0x7f2c2009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44564/0 (socket says 172.21.15.3:44564) 2023-10-03T10:13:36.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.950+0000 7f2c1f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2c2009f240 0x7f2c201317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45156/0 (socket says 172.21.15.3:45156) 2023-10-03T10:13:36.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.950+0000 7f2c24831700 1 -- 172.21.15.3:0/2696203922 learned_addr learned my addr 172.21.15.3:0/2696203922 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:36.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.950+0000 7f2c1ffff700 1 -- 172.21.15.3:0/2696203922 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c2009f650 msgr2=0x7f2c20131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:36.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.950+0000 7f2c1ffff700 1 --2- 172.21.15.3:0/2696203922 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c2009f650 0x7f2c20131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:36.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.950+0000 7f2c1ffff700 1 -- 172.21.15.3:0/2696203922 >> v2:172.21.15.3:3300/0 conn(0x7f2c2009f240 msgr2=0x7f2c201317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:36.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.950+0000 7f2c1ffff700 1 --2- 172.21.15.3:0/2696203922 >> v2:172.21.15.3:3300/0 conn(0x7f2c2009f240 0x7f2c201317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:36.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.950+0000 7f2c1ffff700 1 -- 172.21.15.3:0/2696203922 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2009fb60 con 0x7f2c2009e930 2023-10-03T10:13:36.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.950+0000 7f2c24831700 1 --2- 172.21.15.3:0/2696203922 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c2009f650 0x7f2c20131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:36.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.950+0000 7f2c1f7fe700 1 --2- 172.21.15.3:0/2696203922 >> v2:172.21.15.3:3300/0 conn(0x7f2c2009f240 0x7f2c201317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:36.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.950+0000 7f2c1ffff700 1 --2- 172.21.15.3:0/2696203922 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2c2009e930 0x7f2c2009ed00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009840 tx=0x7f2c10009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bd9bd47f8cdf3a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:36.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.950+0000 7f2c1effd700 1 -- 172.21.15.3:0/2696203922 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c10011830 con 0x7f2c2009e930 2023-10-03T10:13:36.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.950+0000 7f2c1effd700 1 -- 172.21.15.3:0/2696203922 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c10011990 con 0x7f2c2009e930 2023-10-03T10:13:36.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.950+0000 7f2c1effd700 1 -- 172.21.15.3:0/2696203922 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c10011d20 con 0x7f2c2009e930 2023-10-03T10:13:36.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.951+0000 7f2c26294700 1 -- 172.21.15.3:0/2696203922 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2c2009e930 msgr2=0x7f2c2009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:36.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.951+0000 7f2c26294700 1 --2- 172.21.15.3:0/2696203922 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2c2009e930 0x7f2c2009ed00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009840 tx=0x7f2c10009b50 comp rx=0 tx=0).stop 2023-10-03T10:13:36.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.951+0000 7f2c26294700 1 -- 172.21.15.3:0/2696203922 shutdown_connections 2023-10-03T10:13:36.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.951+0000 7f2c26294700 1 --2- 172.21.15.3:0/2696203922 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c2009f650 0x7f2c20131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:36.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.951+0000 7f2c26294700 1 --2- 172.21.15.3:0/2696203922 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2c2009e930 0x7f2c2009ed00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:36.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.951+0000 7f2c26294700 1 --2- 172.21.15.3:0/2696203922 >> v2:172.21.15.3:3300/0 conn(0x7f2c2009f240 0x7f2c201317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:36.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.951+0000 7f2c26294700 1 -- 172.21.15.3:0/2696203922 >> 172.21.15.3:0/2696203922 conn(0x7f2c200a4210 msgr2=0x7f2c200a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:36.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.951+0000 7f2c26294700 1 -- 172.21.15.3:0/2696203922 shutdown_connections 2023-10-03T10:13:36.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.951+0000 7f2c26294700 1 -- 172.21.15.3:0/2696203922 wait complete. 2023-10-03T10:13:36.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.952+0000 7f2c26294700 1 Processor -- start 2023-10-03T10:13:36.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.952+0000 7f2c26294700 1 -- start start 2023-10-03T10:13:36.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.953+0000 7f2c26294700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c2009e930 0x7f2c201a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:36.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.953+0000 7f2c26294700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2c2009f240 0x7f2c201a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:36.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.953+0000 7f2c26294700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c2009f650 0x7f2c201a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:36.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.953+0000 7f2c26294700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c201a7760 con 0x7f2c2009e930 2023-10-03T10:13:36.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.953+0000 7f2c26294700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c201a5e70 con 0x7f2c2009f240 2023-10-03T10:13:36.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.953+0000 7f2c26294700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c201a6b30 con 0x7f2c2009f650 2023-10-03T10:13:36.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.953+0000 7f2c1ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c2009e930 0x7f2c201a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:36.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.953+0000 7f2c1f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2c2009f240 0x7f2c201a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:36.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.953+0000 7f2c1ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c2009e930 0x7f2c201a3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45170/0 (socket says 172.21.15.3:45170) 2023-10-03T10:13:36.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.953+0000 7f2c1ffff700 1 -- 172.21.15.3:0/1926475328 learned_addr learned my addr 172.21.15.3:0/1926475328 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:36.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.953+0000 7f2c24831700 1 --2- 172.21.15.3:0/1926475328 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c2009f650 0x7f2c201a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:36.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.953+0000 7f2c1ffff700 1 -- 172.21.15.3:0/1926475328 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c2009f650 msgr2=0x7f2c201a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:36.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.953+0000 7f2c1ffff700 1 --2- 172.21.15.3:0/1926475328 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c2009f650 0x7f2c201a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:36.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.953+0000 7f2c1ffff700 1 -- 172.21.15.3:0/1926475328 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2c2009f240 msgr2=0x7f2c201a4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:36.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.953+0000 7f2c1ffff700 1 --2- 172.21.15.3:0/1926475328 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2c2009f240 0x7f2c201a4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:36.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.953+0000 7f2c1ffff700 1 -- 172.21.15.3:0/1926475328 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c080094a0 con 0x7f2c2009e930 2023-10-03T10:13:36.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.953+0000 7f2c1f7fe700 1 --2- 172.21.15.3:0/1926475328 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2c2009f240 0x7f2c201a4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:36.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.953+0000 7f2c24831700 1 --2- 172.21.15.3:0/1926475328 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c2009f650 0x7f2c201a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.953+0000 7f2c1ffff700 1 --2- 172.21.15.3:0/1926475328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c2009e930 0x7f2c201a3c40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2c10000c00 tx=0x7f2c10004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.953+0000 7f2c1cff9700 1 -- 172.21.15.3:0/1926475328 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c10004450 con 0x7f2c2009e930 2023-10-03T10:13:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.954+0000 7f2c1cff9700 1 -- 172.21.15.3:0/1926475328 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c1001f070 con 0x7f2c2009e930 2023-10-03T10:13:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.954+0000 7f2c26294700 1 -- 172.21.15.3:0/1926475328 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c18009b30 con 0x7f2c2009e930 2023-10-03T10:13:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.954+0000 7f2c1cff9700 1 -- 172.21.15.3:0/1926475328 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c100038e0 con 0x7f2c2009e930 2023-10-03T10:13:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.954+0000 7f2c26294700 1 -- 172.21.15.3:0/1926475328 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c100094a0 con 0x7f2c2009e930 2023-10-03T10:13:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.954+0000 7f2c1cff9700 1 -- 172.21.15.3:0/1926475328 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2c10003a40 con 0x7f2c2009e930 2023-10-03T10:13:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.955+0000 7f2c1cff9700 1 -- 172.21.15.3:0/1926475328 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2c10021070 con 0x7f2c2009e930 2023-10-03T10:13:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.955+0000 7f2c26294700 1 -- 172.21.15.3:0/1926475328 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c20071d80 con 0x7f2c2009e930 2023-10-03T10:13:36.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:36.960+0000 7f2c1cff9700 1 -- 172.21.15.3:0/1926475328 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2c1001a7b0 con 0x7f2c2009e930 2023-10-03T10:13:37.009 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:37.020 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:37.089 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:37.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.156+0000 7f07b4845700 1 Processor -- start 2023-10-03T10:13:37.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.156+0000 7f07b4845700 1 -- start start 2023-10-03T10:13:37.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.157+0000 7f07b4845700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f07b009e1d0 0x7f07b0138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:37.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.157+0000 7f07b4845700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07b0139170 0x7f07b013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:37.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.157+0000 7f07b4845700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07b009b120 con 0x7f07b009d6f0 2023-10-03T10:13:37.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.157+0000 7f07b4845700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07b006b460 con 0x7f07b009e1d0 2023-10-03T10:13:37.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.157+0000 7f07b4845700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f07b009a7e0 con 0x7f07b0139170 2023-10-03T10:13:37.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.157+0000 7f07aed9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07b0139170 0x7f07b013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:37.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.157+0000 7f07aed9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07b0139170 0x7f07b013b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56858/0 (socket says 172.21.15.3:56858) 2023-10-03T10:13:37.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.157+0000 7f07aed9d700 1 -- 172.21.15.3:0/525125899 learned_addr learned my addr 172.21.15.3:0/525125899 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:37.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.157+0000 7f07add9b700 1 --2- 172.21.15.3:0/525125899 >> v2:172.21.15.3:3300/0 conn(0x7f07b009e1d0 0x7f07b0138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:37.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.157+0000 7f07ad59a700 1 -- 172.21.15.3:0/525125899 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 131863701 0 0) 0x7f07b009b120 con 0x7f07b009d6f0 2023-10-03T10:13:37.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.157+0000 7f07ad59a700 1 -- 172.21.15.3:0/525125899 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07980035b0 con 0x7f07b009d6f0 2023-10-03T10:13:37.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.157+0000 7f07aed9d700 1 -- 172.21.15.3:0/525125899 >> v2:172.21.15.3:3300/0 conn(0x7f07b009e1d0 msgr2=0x7f07b0138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:37.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.157+0000 7f07aed9d700 1 --2- 172.21.15.3:0/525125899 >> v2:172.21.15.3:3300/0 conn(0x7f07b009e1d0 0x7f07b0138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.157+0000 7f07aed9d700 1 -- 172.21.15.3:0/525125899 >> v1:172.21.15.3:6789/0 conn(0x7f07b009d6f0 legacy=0x7f07b009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:37.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.157+0000 7f07aed9d700 1 -- 172.21.15.3:0/525125899 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b009a500 con 0x7f07b0139170 2023-10-03T10:13:37.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.158+0000 7f07aed9d700 1 --2- 172.21.15.3:0/525125899 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07b0139170 0x7f07b013b550 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009e80 tx=0x7f07a800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93f005f916728844 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:37.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.158+0000 7f07ad59a700 1 -- 172.21.15.3:0/525125899 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a800e070 con 0x7f07b0139170 2023-10-03T10:13:37.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.158+0000 7f07ad59a700 1 -- 172.21.15.3:0/525125899 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a8003a20 con 0x7f07b0139170 2023-10-03T10:13:37.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.158+0000 7f07ad59a700 1 -- 172.21.15.3:0/525125899 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a8003dd0 con 0x7f07b0139170 2023-10-03T10:13:37.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.159+0000 7f07b4845700 1 -- 172.21.15.3:0/525125899 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07b0139170 msgr2=0x7f07b013b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:37.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.159+0000 7f07b4845700 1 --2- 172.21.15.3:0/525125899 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07b0139170 0x7f07b013b550 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009e80 tx=0x7f07a800b330 comp rx=0 tx=0).stop 2023-10-03T10:13:37.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.159+0000 7f07b4845700 1 -- 172.21.15.3:0/525125899 shutdown_connections 2023-10-03T10:13:37.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.159+0000 7f07b4845700 1 --2- 172.21.15.3:0/525125899 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07b0139170 0x7f07b013b550 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.159+0000 7f07b4845700 1 --2- 172.21.15.3:0/525125899 >> v2:172.21.15.3:3300/0 conn(0x7f07b009e1d0 0x7f07b0138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.159+0000 7f07b4845700 1 -- 172.21.15.3:0/525125899 >> 172.21.15.3:0/525125899 conn(0x7f07b0066f50 msgr2=0x7f07b009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:37.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.159+0000 7f07b4845700 1 -- 172.21.15.3:0/525125899 shutdown_connections 2023-10-03T10:13:37.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.159+0000 7f07b4845700 1 -- 172.21.15.3:0/525125899 wait complete. 2023-10-03T10:13:37.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.161+0000 7f07b4845700 1 Processor -- start 2023-10-03T10:13:37.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.161+0000 7f07b4845700 1 -- start start 2023-10-03T10:13:37.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.161+0000 7f07b4845700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07b009d6f0 0x7f07b01207d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:37.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.161+0000 7f07b4845700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07b009e1d0 0x7f07b0120d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:37.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.161+0000 7f07b4845700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07b0139170 0x7f07b0121250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:37.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.161+0000 7f07b4845700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b0136680 con 0x7f07b0139170 2023-10-03T10:13:37.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.162+0000 7f07b4845700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b009c710 con 0x7f07b009d6f0 2023-10-03T10:13:37.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.162+0000 7f07b4845700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f07b0122c00 con 0x7f07b009e1d0 2023-10-03T10:13:37.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.162+0000 7f07ae59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07b009d6f0 0x7f07b01207d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:37.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.162+0000 7f07add9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07b009e1d0 0x7f07b0120d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:37.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.162+0000 7f07add9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07b009e1d0 0x7f07b0120d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56870/0 (socket says 172.21.15.3:56870) 2023-10-03T10:13:37.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.162+0000 7f07add9b700 1 -- 172.21.15.3:0/2464278129 learned_addr learned my addr 172.21.15.3:0/2464278129 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:37.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.162+0000 7f07aed9d700 1 --2- 172.21.15.3:0/2464278129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07b0139170 0x7f07b0121250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:37.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.162+0000 7f07add9b700 1 -- 172.21.15.3:0/2464278129 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07b009d6f0 msgr2=0x7f07b01207d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:37.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.162+0000 7f07add9b700 1 --2- 172.21.15.3:0/2464278129 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07b009d6f0 0x7f07b01207d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.162+0000 7f07add9b700 1 -- 172.21.15.3:0/2464278129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07b0139170 msgr2=0x7f07b0121250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:37.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.162+0000 7f07add9b700 1 --2- 172.21.15.3:0/2464278129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07b0139170 0x7f07b0121250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.162+0000 7f07add9b700 1 -- 172.21.15.3:0/2464278129 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a8009b30 con 0x7f07b009e1d0 2023-10-03T10:13:37.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.162+0000 7f07add9b700 1 --2- 172.21.15.3:0/2464278129 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07b009e1d0 0x7f07b0120d10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f07a400e9b0 tx=0x7f07a400ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:37.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.163+0000 7f079effd700 1 -- 172.21.15.3:0/2464278129 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a4009710 con 0x7f07b009e1d0 2023-10-03T10:13:37.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.163+0000 7f079effd700 1 -- 172.21.15.3:0/2464278129 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a400c850 con 0x7f07b009e1d0 2023-10-03T10:13:37.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.163+0000 7f07b4845700 1 -- 172.21.15.3:0/2464278129 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07b0071f60 con 0x7f07b009e1d0 2023-10-03T10:13:37.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.163+0000 7f079effd700 1 -- 172.21.15.3:0/2464278129 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a400cbc0 con 0x7f07b009e1d0 2023-10-03T10:13:37.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.163+0000 7f07b4845700 1 -- 172.21.15.3:0/2464278129 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b0124a40 con 0x7f07b009e1d0 2023-10-03T10:13:37.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.164+0000 7f079effd700 1 -- 172.21.15.3:0/2464278129 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f07a400cd20 con 0x7f07b009e1d0 2023-10-03T10:13:37.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.164+0000 7f079effd700 1 -- 172.21.15.3:0/2464278129 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f07a403f7a0 con 0x7f07b009e1d0 2023-10-03T10:13:37.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.164+0000 7f07b4845700 1 -- 172.21.15.3:0/2464278129 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07b0074ea0 con 0x7f07b009e1d0 2023-10-03T10:13:37.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.174+0000 7f079effd700 1 -- 172.21.15.3:0/2464278129 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f07a401b820 con 0x7f07b009e1d0 2023-10-03T10:13:37.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.182+0000 7f2c26294700 1 -- 172.21.15.3:0/1926475328 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2c20074ea0 con 0x7f2c2009e930 2023-10-03T10:13:37.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.182+0000 7f2c1cff9700 1 -- 172.21.15.3:0/1926475328 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2c10015020 con 0x7f2c2009e930 2023-10-03T10:13:37.182 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:37.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.184+0000 7f2c26294700 1 -- 172.21.15.3:0/1926475328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c2009e930 msgr2=0x7f2c201a3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:37.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.184+0000 7f2c26294700 1 --2- 172.21.15.3:0/1926475328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c2009e930 0x7f2c201a3c40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2c10000c00 tx=0x7f2c10004750 comp rx=0 tx=0).stop 2023-10-03T10:13:37.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.185+0000 7f2c26294700 1 -- 172.21.15.3:0/1926475328 shutdown_connections 2023-10-03T10:13:37.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.185+0000 7f2c26294700 1 --2- 172.21.15.3:0/1926475328 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c2009f650 0x7f2c201a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.185+0000 7f2c26294700 1 --2- 172.21.15.3:0/1926475328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c2009e930 0x7f2c201a3c40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.185+0000 7f2c26294700 1 --2- 172.21.15.3:0/1926475328 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2c2009f240 0x7f2c201a4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.185+0000 7f2c26294700 1 -- 172.21.15.3:0/1926475328 >> 172.21.15.3:0/1926475328 conn(0x7f2c200a4210 msgr2=0x7f2c201b8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:37.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.186+0000 7f2c26294700 1 -- 172.21.15.3:0/1926475328 shutdown_connections 2023-10-03T10:13:37.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.186+0000 7f2c26294700 1 -- 172.21.15.3:0/1926475328 wait complete. 2023-10-03T10:13:37.202 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:37.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.209+0000 7f6f1364f700 1 Processor -- start 2023-10-03T10:13:37.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.209+0000 7f6f1364f700 1 -- start start 2023-10-03T10:13:37.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.210+0000 7f6f1364f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f0c09d6f0 0x7f6f0c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:37.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.210+0000 7f6f1364f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6f0c09e000 0x7f6f0c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:37.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.210+0000 7f6f1364f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f0c09e410 0x7f6f0c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:37.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.210+0000 7f6f1364f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f0c06b460 con 0x7f6f0c09e000 2023-10-03T10:13:37.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.210+0000 7f6f1364f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c09b120 con 0x7f6f0c09d6f0 2023-10-03T10:13:37.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.210+0000 7f6f1364f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c09a7e0 con 0x7f6f0c09e410 2023-10-03T10:13:37.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.210+0000 7f6f10bea700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6f0c09e000 0x7f6f0c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:37.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.210+0000 7f6f10bea700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6f0c09e000 0x7f6f0c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45196/0 (socket says 172.21.15.3:45196) 2023-10-03T10:13:37.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.210+0000 7f6f10bea700 1 -- 172.21.15.3:0/1314786991 learned_addr learned my addr 172.21.15.3:0/1314786991 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:37.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.210+0000 7f6f11bec700 1 --2- 172.21.15.3:0/1314786991 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f0c09e410 0x7f6f0c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:37.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.210+0000 7f6f113eb700 1 --2- 172.21.15.3:0/1314786991 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f0c09d6f0 0x7f6f0c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:37.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.210+0000 7f6f10bea700 1 -- 172.21.15.3:0/1314786991 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f0c09e410 msgr2=0x7f6f0c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:37.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.210+0000 7f6f10bea700 1 --2- 172.21.15.3:0/1314786991 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f0c09e410 0x7f6f0c133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.210+0000 7f6f10bea700 1 -- 172.21.15.3:0/1314786991 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f0c09d6f0 msgr2=0x7f6f0c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:37.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.210+0000 7f6f10bea700 1 --2- 172.21.15.3:0/1314786991 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f0c09d6f0 0x7f6f0c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.210+0000 7f6f10bea700 1 -- 172.21.15.3:0/1314786991 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0c09a500 con 0x7f6f0c09e000 2023-10-03T10:13:37.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.211+0000 7f6f10bea700 1 --2- 172.21.15.3:0/1314786991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f0c09e000 0x7f6f0c1339f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0098f0 tx=0x7f6efc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2f035f72522fed server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:37.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.211+0000 7f6f03fff700 1 -- 172.21.15.3:0/1314786991 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6efc011830 con 0x7f6f0c09e000 2023-10-03T10:13:37.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.211+0000 7f6f03fff700 1 -- 172.21.15.3:0/1314786991 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6efc011990 con 0x7f6f0c09e000 2023-10-03T10:13:37.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.211+0000 7f6f03fff700 1 -- 172.21.15.3:0/1314786991 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6efc011d20 con 0x7f6f0c09e000 2023-10-03T10:13:37.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.212+0000 7f6f1364f700 1 -- 172.21.15.3:0/1314786991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f0c09e000 msgr2=0x7f6f0c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:37.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.212+0000 7f6f1364f700 1 --2- 172.21.15.3:0/1314786991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f0c09e000 0x7f6f0c1339f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0098f0 tx=0x7f6efc009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:37.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.212+0000 7f6f1364f700 1 -- 172.21.15.3:0/1314786991 shutdown_connections 2023-10-03T10:13:37.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.212+0000 7f6f1364f700 1 --2- 172.21.15.3:0/1314786991 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f0c09e410 0x7f6f0c133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.212+0000 7f6f1364f700 1 --2- 172.21.15.3:0/1314786991 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f0c09d6f0 0x7f6f0c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.212+0000 7f6f1364f700 1 --2- 172.21.15.3:0/1314786991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f0c09e000 0x7f6f0c1339f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.212+0000 7f6f1364f700 1 -- 172.21.15.3:0/1314786991 >> 172.21.15.3:0/1314786991 conn(0x7f6f0c066f50 msgr2=0x7f6f0c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:37.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.212+0000 7f6f1364f700 1 -- 172.21.15.3:0/1314786991 shutdown_connections 2023-10-03T10:13:37.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.212+0000 7f6f1364f700 1 -- 172.21.15.3:0/1314786991 wait complete. 2023-10-03T10:13:37.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.213+0000 7f6f1364f700 1 Processor -- start 2023-10-03T10:13:37.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.213+0000 7f6f1364f700 1 -- start start 2023-10-03T10:13:37.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.213+0000 7f6f1364f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f0c09d6f0 0x7f6f0c1c5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:37.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.213+0000 7f6f1364f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f0c09e000 0x7f6f0c1c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:37.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.213+0000 7f6f1364f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f0c09e410 0x7f6f0c1c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:37.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.213+0000 7f6f1364f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c1a7200 con 0x7f6f0c09e000 2023-10-03T10:13:37.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.213+0000 7f6f1364f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c1a8760 con 0x7f6f0c09e410 2023-10-03T10:13:37.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.213+0000 7f6f1364f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c1a8b20 con 0x7f6f0c09d6f0 2023-10-03T10:13:37.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.213+0000 7f6f10bea700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f0c09e000 0x7f6f0c1c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:37.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.213+0000 7f6f10bea700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f0c09e000 0x7f6f0c1c7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45208/0 (socket says 172.21.15.3:45208) 2023-10-03T10:13:37.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.213+0000 7f6f10bea700 1 -- 172.21.15.3:0/3523450970 learned_addr learned my addr 172.21.15.3:0/3523450970 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:37.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.214+0000 7f6f11bec700 1 --2- 172.21.15.3:0/3523450970 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f0c09e410 0x7f6f0c1c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:37.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.214+0000 7f6f10bea700 1 -- 172.21.15.3:0/3523450970 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f0c09d6f0 msgr2=0x7f6f0c1c5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:37.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.214+0000 7f6f10bea700 1 --2- 172.21.15.3:0/3523450970 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f0c09d6f0 0x7f6f0c1c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.214+0000 7f6f10bea700 1 -- 172.21.15.3:0/3523450970 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f0c09e410 msgr2=0x7f6f0c1c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:37.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.214+0000 7f6f10bea700 1 --2- 172.21.15.3:0/3523450970 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f0c09e410 0x7f6f0c1c84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.214+0000 7f6f10bea700 1 -- 172.21.15.3:0/3523450970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6efc009550 con 0x7f6f0c09e000 2023-10-03T10:13:37.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.214+0000 7f6f10bea700 1 --2- 172.21.15.3:0/3523450970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f0c09e000 0x7f6f0c1c7fb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6efc013000 tx=0x7f6efc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:37.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.214+0000 7f6f01ffb700 1 -- 172.21.15.3:0/3523450970 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6efc004450 con 0x7f6f0c09e000 2023-10-03T10:13:37.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.214+0000 7f6f01ffb700 1 -- 172.21.15.3:0/3523450970 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6efc0045b0 con 0x7f6f0c09e000 2023-10-03T10:13:37.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.214+0000 7f6f1364f700 1 -- 172.21.15.3:0/3523450970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f0c1a9520 con 0x7f6f0c09e000 2023-10-03T10:13:37.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.214+0000 7f6f01ffb700 1 -- 172.21.15.3:0/3523450970 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6efc0038c0 con 0x7f6f0c09e000 2023-10-03T10:13:37.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.215+0000 7f6f1364f700 1 -- 172.21.15.3:0/3523450970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f0c1a92c0 con 0x7f6f0c09e000 2023-10-03T10:13:37.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.216+0000 7f6f01ffb700 1 -- 172.21.15.3:0/3523450970 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6efc003a20 con 0x7f6f0c09e000 2023-10-03T10:13:37.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.216+0000 7f6f01ffb700 1 -- 172.21.15.3:0/3523450970 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6efc021030 con 0x7f6f0c09e000 2023-10-03T10:13:37.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.216+0000 7f6f1364f700 1 -- 172.21.15.3:0/3523450970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f0c074ea0 con 0x7f6f0c09e000 2023-10-03T10:13:37.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.221+0000 7f6f01ffb700 1 -- 172.21.15.3:0/3523450970 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6efc02b020 con 0x7f6f0c09e000 2023-10-03T10:13:37.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.249+0000 7f69c2b2c700 1 Processor -- start 2023-10-03T10:13:37.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.250+0000 7f69c2b2c700 1 -- start start 2023-10-03T10:13:37.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.250+0000 7f69c2b2c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f69bc09d6f0 0x7f69bc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:37.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.250+0000 7f69c2b2c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f69bc09e000 0x7f69bc1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:37.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.250+0000 7f69c2b2c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69bc09b120 con 0x7f69bc09e410 2023-10-03T10:13:37.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.250+0000 7f69c2b2c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69bc06b460 con 0x7f69bc09e000 2023-10-03T10:13:37.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.250+0000 7f69c2b2c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f69bc09a7e0 con 0x7f69bc09d6f0 2023-10-03T10:13:37.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.250+0000 7f69c08c8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f69bc09d6f0 0x7f69bc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:37.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.250+0000 7f69c08c8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f69bc09d6f0 0x7f69bc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56908/0 (socket says 172.21.15.3:56908) 2023-10-03T10:13:37.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.250+0000 7f69c08c8700 1 -- 172.21.15.3:0/3663038729 learned_addr learned my addr 172.21.15.3:0/3663038729 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:37.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.250+0000 7f69bbfff700 1 --2- 172.21.15.3:0/3663038729 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f69bc09e000 0x7f69bc1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:37.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.251+0000 7f69bb7fe700 1 -- 172.21.15.3:0/3663038729 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 357410954 0 0) 0x7f69bc09b120 con 0x7f69bc09e410 2023-10-03T10:13:37.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.251+0000 7f69bb7fe700 1 -- 172.21.15.3:0/3663038729 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69a40035b0 con 0x7f69bc09e410 2023-10-03T10:13:37.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.251+0000 7f69bbfff700 1 -- 172.21.15.3:0/3663038729 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f69bc09d6f0 msgr2=0x7f69bc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:37.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.251+0000 7f69bbfff700 1 --2- 172.21.15.3:0/3663038729 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f69bc09d6f0 0x7f69bc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.251+0000 7f69bbfff700 1 -- 172.21.15.3:0/3663038729 >> v1:172.21.15.3:6789/0 conn(0x7f69bc09e410 legacy=0x7f69bc133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:37.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.251+0000 7f69bbfff700 1 -- 172.21.15.3:0/3663038729 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69bc09a500 con 0x7f69bc09e000 2023-10-03T10:13:37.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.251+0000 7f69c08c8700 1 --2- 172.21.15.3:0/3663038729 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f69bc09d6f0 0x7f69bc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:37.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.251+0000 7f69bbfff700 1 --2- 172.21.15.3:0/3663038729 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f69bc09e000 0x7f69bc1339f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f69b0009870 tx=0x7f69b0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ed7d7c6bdd150f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:37.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.251+0000 7f69bb7fe700 1 -- 172.21.15.3:0/3663038729 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b0011830 con 0x7f69bc09e000 2023-10-03T10:13:37.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.251+0000 7f69bb7fe700 1 -- 172.21.15.3:0/3663038729 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b0011990 con 0x7f69bc09e000 2023-10-03T10:13:37.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.251+0000 7f69bb7fe700 1 -- 172.21.15.3:0/3663038729 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b0011d20 con 0x7f69bc09e000 2023-10-03T10:13:37.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.252+0000 7f69c2b2c700 1 -- 172.21.15.3:0/3663038729 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f69bc09e000 msgr2=0x7f69bc1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:37.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.252+0000 7f69c2b2c700 1 --2- 172.21.15.3:0/3663038729 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f69bc09e000 0x7f69bc1339f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f69b0009870 tx=0x7f69b0009b80 comp rx=0 tx=0).stop 2023-10-03T10:13:37.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.252+0000 7f69c2b2c700 1 -- 172.21.15.3:0/3663038729 shutdown_connections 2023-10-03T10:13:37.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.252+0000 7f69c2b2c700 1 --2- 172.21.15.3:0/3663038729 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f69bc09d6f0 0x7f69bc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.252+0000 7f69c2b2c700 1 --2- 172.21.15.3:0/3663038729 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f69bc09e000 0x7f69bc1339f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.252+0000 7f69c2b2c700 1 -- 172.21.15.3:0/3663038729 >> 172.21.15.3:0/3663038729 conn(0x7f69bc066f50 msgr2=0x7f69bc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:37.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.252+0000 7f69c2b2c700 1 -- 172.21.15.3:0/3663038729 shutdown_connections 2023-10-03T10:13:37.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.252+0000 7f69c2b2c700 1 -- 172.21.15.3:0/3663038729 wait complete. 2023-10-03T10:13:37.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.254+0000 7f69c2b2c700 1 Processor -- start 2023-10-03T10:13:37.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.254+0000 7f69c2b2c700 1 -- start start 2023-10-03T10:13:37.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.255+0000 7f69c2b2c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f69bc09d6f0 0x7f69bc122300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:37.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.255+0000 7f69c2b2c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69bc09e000 0x7f69bc122840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:37.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.255+0000 7f69c2b2c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f69bc09e410 0x7f69bc122d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:37.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.255+0000 7f69c2b2c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69bc1237d0 con 0x7f69bc09e000 2023-10-03T10:13:37.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.255+0000 7f69c2b2c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69bc0720b0 con 0x7f69bc09e410 2023-10-03T10:13:37.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.255+0000 7f69c2b2c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f69bc09c710 con 0x7f69bc09d6f0 2023-10-03T10:13:37.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.255+0000 7f69c08c8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f69bc09d6f0 0x7f69bc122300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:37.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.255+0000 7f69bbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69bc09e000 0x7f69bc122840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:37.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.255+0000 7f69bbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69bc09e000 0x7f69bc122840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45224/0 (socket says 172.21.15.3:45224) 2023-10-03T10:13:37.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.255+0000 7f69bbfff700 1 -- 172.21.15.3:0/3984208673 learned_addr learned my addr 172.21.15.3:0/3984208673 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:37.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.255+0000 7f69c08c8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f69bc09d6f0 0x7f69bc122300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56912/0 (socket says 172.21.15.3:56912) 2023-10-03T10:13:37.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.255+0000 7f69c10c9700 1 --2- 172.21.15.3:0/3984208673 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f69bc09e410 0x7f69bc122d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:37.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.256+0000 7f69c08c8700 1 -- 172.21.15.3:0/3984208673 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f69bc09e410 msgr2=0x7f69bc122d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:37.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.256+0000 7f69c08c8700 1 --2- 172.21.15.3:0/3984208673 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f69bc09e410 0x7f69bc122d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.256+0000 7f69c08c8700 1 -- 172.21.15.3:0/3984208673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69bc09e000 msgr2=0x7f69bc122840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:37.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.256+0000 7f69c08c8700 1 --2- 172.21.15.3:0/3984208673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69bc09e000 0x7f69bc122840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.256+0000 7f69c08c8700 1 -- 172.21.15.3:0/3984208673 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ac009b30 con 0x7f69bc09d6f0 2023-10-03T10:13:37.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.256+0000 7f69bbfff700 1 --2- 172.21.15.3:0/3984208673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69bc09e000 0x7f69bc122840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:37.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.256+0000 7f69c08c8700 1 --2- 172.21.15.3:0/3984208673 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f69bc09d6f0 0x7f69bc122300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009e40 tx=0x7f69ac00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:37.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.256+0000 7f69b97fa700 1 -- 172.21.15.3:0/3984208673 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac003c20 con 0x7f69bc09d6f0 2023-10-03T10:13:37.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.256+0000 7f69b97fa700 1 -- 172.21.15.3:0/3984208673 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69ac00f040 con 0x7f69bc09d6f0 2023-10-03T10:13:37.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.256+0000 7f69b97fa700 1 -- 172.21.15.3:0/3984208673 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac00c900 con 0x7f69bc09d6f0 2023-10-03T10:13:37.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.256+0000 7f69c2b2c700 1 -- 172.21.15.3:0/3984208673 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b00094a0 con 0x7f69bc09d6f0 2023-10-03T10:13:37.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.256+0000 7f69c2b2c700 1 -- 172.21.15.3:0/3984208673 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69bc125540 con 0x7f69bc09d6f0 2023-10-03T10:13:37.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.257+0000 7f69b97fa700 1 -- 172.21.15.3:0/3984208673 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f69ac00ca60 con 0x7f69bc09d6f0 2023-10-03T10:13:37.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.257+0000 7f69b97fa700 1 -- 172.21.15.3:0/3984208673 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f69ac03fa30 con 0x7f69bc09d6f0 2023-10-03T10:13:37.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.257+0000 7f69c2b2c700 1 -- 172.21.15.3:0/3984208673 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69bc074ea0 con 0x7f69bc09d6f0 2023-10-03T10:13:37.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.262+0000 7f69b97fa700 1 -- 172.21.15.3:0/3984208673 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f69ac014480 con 0x7f69bc09d6f0 2023-10-03T10:13:37.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.395+0000 7f07b4845700 1 -- 172.21.15.3:0/2464278129 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f07b0123c70 con 0x7f07b009e1d0 2023-10-03T10:13:37.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.395+0000 7f079effd700 1 -- 172.21.15.3:0/2464278129 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f07a4021030 con 0x7f07b009e1d0 2023-10-03T10:13:37.396 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:37.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.398+0000 7f07b4845700 1 -- 172.21.15.3:0/2464278129 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07b009e1d0 msgr2=0x7f07b0120d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:37.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.398+0000 7f07b4845700 1 --2- 172.21.15.3:0/2464278129 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07b009e1d0 0x7f07b0120d10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f07a400e9b0 tx=0x7f07a400ecc0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.398+0000 7f07b4845700 1 -- 172.21.15.3:0/2464278129 shutdown_connections 2023-10-03T10:13:37.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.398+0000 7f07b4845700 1 --2- 172.21.15.3:0/2464278129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07b0139170 0x7f07b0121250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.398+0000 7f07b4845700 1 --2- 172.21.15.3:0/2464278129 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07b009d6f0 0x7f07b01207d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.398+0000 7f07b4845700 1 --2- 172.21.15.3:0/2464278129 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07b009e1d0 0x7f07b0120d10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.398+0000 7f07b4845700 1 -- 172.21.15.3:0/2464278129 >> 172.21.15.3:0/2464278129 conn(0x7f07b0066f50 msgr2=0x7f07b0136b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:37.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.398+0000 7f07b4845700 1 -- 172.21.15.3:0/2464278129 shutdown_connections 2023-10-03T10:13:37.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.398+0000 7f07b4845700 1 -- 172.21.15.3:0/2464278129 wait complete. 2023-10-03T10:13:37.413 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:37.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.439+0000 7f6f1364f700 1 -- 172.21.15.3:0/3523450970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6f0c06c360 con 0x7f6f0c09e000 2023-10-03T10:13:37.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.439+0000 7f6f01ffb700 1 -- 172.21.15.3:0/3523450970 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6efc015070 con 0x7f6f0c09e000 2023-10-03T10:13:37.440 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:37.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.441+0000 7f6f1364f700 1 -- 172.21.15.3:0/3523450970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f0c09e000 msgr2=0x7f6f0c1c7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:37.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.441+0000 7f6f1364f700 1 --2- 172.21.15.3:0/3523450970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f0c09e000 0x7f6f0c1c7fb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6efc013000 tx=0x7f6efc004750 comp rx=0 tx=0).stop 2023-10-03T10:13:37.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.441+0000 7f6f1364f700 1 -- 172.21.15.3:0/3523450970 shutdown_connections 2023-10-03T10:13:37.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.441+0000 7f6f1364f700 1 --2- 172.21.15.3:0/3523450970 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f0c09e410 0x7f6f0c1c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.441+0000 7f6f1364f700 1 --2- 172.21.15.3:0/3523450970 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f0c09d6f0 0x7f6f0c1c5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.441+0000 7f6f1364f700 1 --2- 172.21.15.3:0/3523450970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f0c09e000 0x7f6f0c1c7fb0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.441+0000 7f6f1364f700 1 -- 172.21.15.3:0/3523450970 >> 172.21.15.3:0/3523450970 conn(0x7f6f0c066f50 msgr2=0x7f6f0c1bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:37.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.442+0000 7f6f1364f700 1 -- 172.21.15.3:0/3523450970 shutdown_connections 2023-10-03T10:13:37.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.442+0000 7f6f1364f700 1 -- 172.21.15.3:0/3523450970 wait complete. 2023-10-03T10:13:37.456 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:37.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.473+0000 7f69c2b2c700 1 -- 172.21.15.3:0/3984208673 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f69bc06c360 con 0x7f69bc09d6f0 2023-10-03T10:13:37.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.474+0000 7f69b97fa700 1 -- 172.21.15.3:0/3984208673 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f69ac0133e0 con 0x7f69bc09d6f0 2023-10-03T10:13:37.474 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:37.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.476+0000 7f69c2b2c700 1 -- 172.21.15.3:0/3984208673 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f69bc09d6f0 msgr2=0x7f69bc122300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:37.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.476+0000 7f69c2b2c700 1 --2- 172.21.15.3:0/3984208673 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f69bc09d6f0 0x7f69bc122300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009e40 tx=0x7f69ac00c3a0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.476+0000 7f69c2b2c700 1 -- 172.21.15.3:0/3984208673 shutdown_connections 2023-10-03T10:13:37.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.476+0000 7f69c2b2c700 1 --2- 172.21.15.3:0/3984208673 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f69bc09e410 0x7f69bc122d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.476+0000 7f69c2b2c700 1 --2- 172.21.15.3:0/3984208673 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f69bc09d6f0 0x7f69bc122300 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.476+0000 7f69c2b2c700 1 --2- 172.21.15.3:0/3984208673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69bc09e000 0x7f69bc122840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:37.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.476+0000 7f69c2b2c700 1 -- 172.21.15.3:0/3984208673 >> 172.21.15.3:0/3984208673 conn(0x7f69bc066f50 msgr2=0x7f69bc133e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:37.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.476+0000 7f69c2b2c700 1 -- 172.21.15.3:0/3984208673 shutdown_connections 2023-10-03T10:13:37.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:37.476+0000 7f69c2b2c700 1 -- 172.21.15.3:0/3984208673 wait complete. 2023-10-03T10:13:37.490 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:38.203 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:38.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.348+0000 7f54356ce700 1 Processor -- start 2023-10-03T10:13:38.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.348+0000 7f54356ce700 1 -- start start 2023-10-03T10:13:38.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.349+0000 7f54356ce700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f54300a0a30 0x7f54300a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.349+0000 7f54356ce700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5430138d80 0x7f543013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.349+0000 7f54356ce700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54300a2a70 con 0x7f543009d630 2023-10-03T10:13:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.349+0000 7f54356ce700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54300a01d0 con 0x7f54300a0a30 2023-10-03T10:13:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.349+0000 7f54356ce700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f543009e2b0 con 0x7f5430138d80 2023-10-03T10:13:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.349+0000 7f542f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5430138d80 0x7f543013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.349+0000 7f542effd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f54300a0a30 0x7f54300a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.349+0000 7f542f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5430138d80 0x7f543013b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56928/0 (socket says 172.21.15.3:56928) 2023-10-03T10:13:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.349+0000 7f542effd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f54300a0a30 0x7f54300a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45228/0 (socket says 172.21.15.3:45228) 2023-10-03T10:13:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.349+0000 7f542f7fe700 1 -- 172.21.15.3:0/3544507073 learned_addr learned my addr 172.21.15.3:0/3544507073 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.349+0000 7f542f7fe700 1 -- 172.21.15.3:0/3544507073 >> v2:172.21.15.3:3300/0 conn(0x7f54300a0a30 msgr2=0x7f54300a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.349+0000 7f542f7fe700 1 --2- 172.21.15.3:0/3544507073 >> v2:172.21.15.3:3300/0 conn(0x7f54300a0a30 0x7f54300a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.349+0000 7f542f7fe700 1 -- 172.21.15.3:0/3544507073 >> v1:172.21.15.3:6789/0 conn(0x7f543009d630 legacy=0x7f54300a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:38.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.349+0000 7f542f7fe700 1 -- 172.21.15.3:0/3544507073 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543009dc50 con 0x7f5430138d80 2023-10-03T10:13:38.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.349+0000 7f542effd700 1 --2- 172.21.15.3:0/3544507073 >> v2:172.21.15.3:3300/0 conn(0x7f54300a0a30 0x7f54300a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:38.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.349+0000 7f542f7fe700 1 --2- 172.21.15.3:0/3544507073 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5430138d80 0x7f543013b160 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5428009e50 tx=0x7f542800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de23bf160120f2b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:38.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.350+0000 7f542dffb700 1 -- 172.21.15.3:0/3544507073 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542800e070 con 0x7f5430138d80 2023-10-03T10:13:38.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.350+0000 7f542dffb700 1 -- 172.21.15.3:0/3544507073 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542800b910 con 0x7f5430138d80 2023-10-03T10:13:38.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.350+0000 7f542dffb700 1 -- 172.21.15.3:0/3544507073 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542800bcc0 con 0x7f5430138d80 2023-10-03T10:13:38.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.350+0000 7f54356ce700 1 -- 172.21.15.3:0/3544507073 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5430138d80 msgr2=0x7f543013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:38.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.350+0000 7f54356ce700 1 --2- 172.21.15.3:0/3544507073 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5430138d80 0x7f543013b160 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5428009e50 tx=0x7f542800b330 comp rx=0 tx=0).stop 2023-10-03T10:13:38.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.350+0000 7f54356ce700 1 -- 172.21.15.3:0/3544507073 shutdown_connections 2023-10-03T10:13:38.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.350+0000 7f54356ce700 1 --2- 172.21.15.3:0/3544507073 >> v2:172.21.15.3:3300/0 conn(0x7f54300a0a30 0x7f54300a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.350+0000 7f54356ce700 1 --2- 172.21.15.3:0/3544507073 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5430138d80 0x7f543013b160 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.350+0000 7f54356ce700 1 -- 172.21.15.3:0/3544507073 >> 172.21.15.3:0/3544507073 conn(0x7f54300a3050 msgr2=0x7f54300a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:38.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.350+0000 7f54356ce700 1 -- 172.21.15.3:0/3544507073 shutdown_connections 2023-10-03T10:13:38.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.350+0000 7f54356ce700 1 -- 172.21.15.3:0/3544507073 wait complete. 2023-10-03T10:13:38.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.351+0000 7f54356ce700 1 Processor -- start 2023-10-03T10:13:38.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.352+0000 7f54356ce700 1 -- start start 2023-10-03T10:13:38.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.352+0000 7f54356ce700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f543009d630 0x7f54301339c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:38.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.352+0000 7f54356ce700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f54300a0a30 0x7f5430133f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:38.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.352+0000 7f54356ce700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5430138d80 0x7f5430134440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:38.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.352+0000 7f54356ce700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5430136f40 con 0x7f543009d630 2023-10-03T10:13:38.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.352+0000 7f54356ce700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543013afd0 con 0x7f54300a0a30 2023-10-03T10:13:38.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.352+0000 7f54356ce700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f54301381d0 con 0x7f5430138d80 2023-10-03T10:13:38.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.352+0000 7f542effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f543009d630 0x7f54301339c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:38.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.352+0000 7f542effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f543009d630 0x7f54301339c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45240/0 (socket says 172.21.15.3:45240) 2023-10-03T10:13:38.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.352+0000 7f542f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5430138d80 0x7f5430134440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:38.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.352+0000 7f542effd700 1 -- 172.21.15.3:0/2970200157 learned_addr learned my addr 172.21.15.3:0/2970200157 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:38.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.352+0000 7f542e7fc700 1 --2- 172.21.15.3:0/2970200157 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f54300a0a30 0x7f5430133f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:38.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.352+0000 7f542effd700 1 -- 172.21.15.3:0/2970200157 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5430138d80 msgr2=0x7f5430134440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:38.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.352+0000 7f542effd700 1 --2- 172.21.15.3:0/2970200157 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5430138d80 0x7f5430134440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.352+0000 7f542effd700 1 -- 172.21.15.3:0/2970200157 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f54300a0a30 msgr2=0x7f5430133f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:38.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.352+0000 7f542effd700 1 --2- 172.21.15.3:0/2970200157 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f54300a0a30 0x7f5430133f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.352+0000 7f542effd700 1 -- 172.21.15.3:0/2970200157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54200094a0 con 0x7f543009d630 2023-10-03T10:13:38.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.352+0000 7f542e7fc700 1 --2- 172.21.15.3:0/2970200157 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f54300a0a30 0x7f5430133f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:38.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.352+0000 7f542f7fe700 1 --2- 172.21.15.3:0/2970200157 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5430138d80 0x7f5430134440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:38.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.353+0000 7f542effd700 1 --2- 172.21.15.3:0/2970200157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f543009d630 0x7f54301339c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f542000ebe0 tx=0x7f542000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:38.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.353+0000 7f54177fe700 1 -- 172.21.15.3:0/2970200157 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542000c850 con 0x7f543009d630 2023-10-03T10:13:38.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.353+0000 7f54177fe700 1 -- 172.21.15.3:0/2970200157 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542000c9b0 con 0x7f543009d630 2023-10-03T10:13:38.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.353+0000 7f54356ce700 1 -- 172.21.15.3:0/2970200157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5428009b30 con 0x7f543009d630 2023-10-03T10:13:38.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.353+0000 7f54177fe700 1 -- 172.21.15.3:0/2970200157 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542000cd40 con 0x7f543009d630 2023-10-03T10:13:38.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.353+0000 7f54356ce700 1 -- 172.21.15.3:0/2970200157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5430136c30 con 0x7f543009d630 2023-10-03T10:13:38.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.354+0000 7f54177fe700 1 -- 172.21.15.3:0/2970200157 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5420013470 con 0x7f543009d630 2023-10-03T10:13:38.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.354+0000 7f54177fe700 1 -- 172.21.15.3:0/2970200157 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f542003f820 con 0x7f543009d630 2023-10-03T10:13:38.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.354+0000 7f54356ce700 1 -- 172.21.15.3:0/2970200157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54301370d0 con 0x7f543009d630 2023-10-03T10:13:38.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.359+0000 7f54177fe700 1 -- 172.21.15.3:0/2970200157 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f54301370d0 con 0x7f543009d630 2023-10-03T10:13:38.413 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:38.456 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:38.491 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:38.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.555+0000 7f32e1bd3700 1 Processor -- start 2023-10-03T10:13:38.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.555+0000 7f32e1bd3700 1 -- start start 2023-10-03T10:13:38.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.556+0000 7f32e1bd3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f32dc1338a0 0x7f32dc133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:38.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.556+0000 7f32e1bd3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f32dc09de50 0x7f32dc09e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:38.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.556+0000 7f32e1bd3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32dc06ef30 con 0x7f32dc09e760 2023-10-03T10:13:38.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.556+0000 7f32e1bd3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32dc06fef0 con 0x7f32dc1338a0 2023-10-03T10:13:38.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.556+0000 7f32e1bd3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc1334b0 con 0x7f32dc09de50 2023-10-03T10:13:38.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.556+0000 7f32daffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f32dc09de50 0x7f32dc09e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:38.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.556+0000 7f32daffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f32dc09de50 0x7f32dc09e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44648/0 (socket says 172.21.15.3:44648) 2023-10-03T10:13:38.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.556+0000 7f32daffd700 1 -- 172.21.15.3:0/1811790325 learned_addr learned my addr 172.21.15.3:0/1811790325 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:38.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.556+0000 7f32db7fe700 1 --2- 172.21.15.3:0/1811790325 >> v2:172.21.15.3:3300/0 conn(0x7f32dc1338a0 0x7f32dc133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:38.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.557+0000 7f32daffd700 1 -- 172.21.15.3:0/1811790325 >> v2:172.21.15.3:3300/0 conn(0x7f32dc1338a0 msgr2=0x7f32dc133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:38.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.557+0000 7f32daffd700 1 --2- 172.21.15.3:0/1811790325 >> v2:172.21.15.3:3300/0 conn(0x7f32dc1338a0 0x7f32dc133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.557+0000 7f32daffd700 1 -- 172.21.15.3:0/1811790325 >> v1:172.21.15.3:6789/0 conn(0x7f32dc09e760 legacy=0x7f32dc13a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:38.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.557+0000 7f32daffd700 1 -- 172.21.15.3:0/1811790325 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32dc06fc30 con 0x7f32dc09de50 2023-10-03T10:13:38.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.557+0000 7f32daffd700 1 --2- 172.21.15.3:0/1811790325 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f32dc09de50 0x7f32dc09e220 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f32d00098f0 tx=0x7f32d0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3867c5ab6826a33b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:38.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.557+0000 7f32da7fc700 1 -- 172.21.15.3:0/1811790325 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d0011830 con 0x7f32dc09de50 2023-10-03T10:13:38.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.557+0000 7f32da7fc700 1 -- 172.21.15.3:0/1811790325 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d0011990 con 0x7f32dc09de50 2023-10-03T10:13:38.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.557+0000 7f32da7fc700 1 -- 172.21.15.3:0/1811790325 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d0011d20 con 0x7f32dc09de50 2023-10-03T10:13:38.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.558+0000 7f32e1bd3700 1 -- 172.21.15.3:0/1811790325 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f32dc09de50 msgr2=0x7f32dc09e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:38.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.558+0000 7f32e1bd3700 1 --2- 172.21.15.3:0/1811790325 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f32dc09de50 0x7f32dc09e220 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f32d00098f0 tx=0x7f32d0009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:38.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.558+0000 7f32e1bd3700 1 -- 172.21.15.3:0/1811790325 shutdown_connections 2023-10-03T10:13:38.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.558+0000 7f32e1bd3700 1 --2- 172.21.15.3:0/1811790325 >> v2:172.21.15.3:3300/0 conn(0x7f32dc1338a0 0x7f32dc133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.558+0000 7f32e1bd3700 1 --2- 172.21.15.3:0/1811790325 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f32dc09de50 0x7f32dc09e220 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.558+0000 7f32e1bd3700 1 -- 172.21.15.3:0/1811790325 >> 172.21.15.3:0/1811790325 conn(0x7f32dc0a2aa0 msgr2=0x7f32dc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:38.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.558+0000 7f32e1bd3700 1 -- 172.21.15.3:0/1811790325 shutdown_connections 2023-10-03T10:13:38.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.558+0000 7f32e1bd3700 1 -- 172.21.15.3:0/1811790325 wait complete. 2023-10-03T10:13:38.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.559+0000 7f32e1bd3700 1 Processor -- start 2023-10-03T10:13:38.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.559+0000 7f32e1bd3700 1 -- start start 2023-10-03T10:13:38.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.560+0000 7f32e1bd3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32dc09de50 0x7f32dc11ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:38.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.560+0000 7f32e1bd3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f32dc09e760 0x7f32dc11efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:38.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.560+0000 7f32e1bd3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f32dc1338a0 0x7f32dc11f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:38.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.560+0000 7f32e1bd3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc06f640 con 0x7f32dc09de50 2023-10-03T10:13:38.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.560+0000 7f32e1bd3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc1222b0 con 0x7f32dc1338a0 2023-10-03T10:13:38.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.560+0000 7f32e1bd3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc120790 con 0x7f32dc09e760 2023-10-03T10:13:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.560+0000 7f32db7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32dc09de50 0x7f32dc11ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.560+0000 7f32db7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32dc09de50 0x7f32dc11ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45266/0 (socket says 172.21.15.3:45266) 2023-10-03T10:13:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.560+0000 7f32db7fe700 1 -- 172.21.15.3:0/2109326542 learned_addr learned my addr 172.21.15.3:0/2109326542 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.560+0000 7f32dbfff700 1 --2- 172.21.15.3:0/2109326542 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f32dc1338a0 0x7f32dc11f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.560+0000 7f32daffd700 1 --2- 172.21.15.3:0/2109326542 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f32dc09e760 0x7f32dc11efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.561+0000 7f32db7fe700 1 -- 172.21.15.3:0/2109326542 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f32dc09e760 msgr2=0x7f32dc11efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.561+0000 7f32db7fe700 1 --2- 172.21.15.3:0/2109326542 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f32dc09e760 0x7f32dc11efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.561+0000 7f32db7fe700 1 -- 172.21.15.3:0/2109326542 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f32dc1338a0 msgr2=0x7f32dc11f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.561+0000 7f32db7fe700 1 --2- 172.21.15.3:0/2109326542 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f32dc1338a0 0x7f32dc11f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.561+0000 7f32db7fe700 1 -- 172.21.15.3:0/2109326542 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d0009550 con 0x7f32dc09de50 2023-10-03T10:13:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.561+0000 7f32daffd700 1 --2- 172.21.15.3:0/2109326542 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f32dc09e760 0x7f32dc11efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.561+0000 7f32db7fe700 1 --2- 172.21.15.3:0/2109326542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32dc09de50 0x7f32dc11ea70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f32cc00e9b0 tx=0x7f32cc00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:38.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.561+0000 7f32c3fff700 1 -- 172.21.15.3:0/2109326542 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc009710 con 0x7f32dc09de50 2023-10-03T10:13:38.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.561+0000 7f32c3fff700 1 -- 172.21.15.3:0/2109326542 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32cc00ee50 con 0x7f32dc09de50 2023-10-03T10:13:38.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.561+0000 7f32c3fff700 1 -- 172.21.15.3:0/2109326542 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc00ca50 con 0x7f32dc09de50 2023-10-03T10:13:38.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.561+0000 7f32e1bd3700 1 -- 172.21.15.3:0/2109326542 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32dc0a1000 con 0x7f32dc09de50 2023-10-03T10:13:38.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.562+0000 7f32e1bd3700 1 -- 172.21.15.3:0/2109326542 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32dc122d20 con 0x7f32dc09de50 2023-10-03T10:13:38.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.562+0000 7f32c3fff700 1 -- 172.21.15.3:0/2109326542 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f32cc00cbb0 con 0x7f32dc09de50 2023-10-03T10:13:38.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.563+0000 7f32e1bd3700 1 -- 172.21.15.3:0/2109326542 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32dc074ea0 con 0x7f32dc09de50 2023-10-03T10:13:38.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.563+0000 7f32c3fff700 1 -- 172.21.15.3:0/2109326542 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f32cc03faa0 con 0x7f32dc09de50 2023-10-03T10:13:38.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.572+0000 7f32c3fff700 1 -- 172.21.15.3:0/2109326542 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f32cc021070 con 0x7f32dc09de50 2023-10-03T10:13:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.579+0000 7f54356ce700 1 -- 172.21.15.3:0/2970200157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f54300747f0 con 0x7f543009d630 2023-10-03T10:13:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.580+0000 7f54177fe700 1 -- 172.21.15.3:0/2970200157 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f542000cea0 con 0x7f543009d630 2023-10-03T10:13:38.580 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.582+0000 7f54356ce700 1 -- 172.21.15.3:0/2970200157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f543009d630 msgr2=0x7f54301339c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.582+0000 7f54356ce700 1 --2- 172.21.15.3:0/2970200157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f543009d630 0x7f54301339c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f542000ebe0 tx=0x7f542000eef0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.582+0000 7f54356ce700 1 -- 172.21.15.3:0/2970200157 shutdown_connections 2023-10-03T10:13:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.582+0000 7f54356ce700 1 --2- 172.21.15.3:0/2970200157 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5430138d80 0x7f5430134440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.582+0000 7f54356ce700 1 --2- 172.21.15.3:0/2970200157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f543009d630 0x7f54301339c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.582+0000 7f54356ce700 1 --2- 172.21.15.3:0/2970200157 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f54300a0a30 0x7f5430133f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.582+0000 7f54356ce700 1 -- 172.21.15.3:0/2970200157 >> 172.21.15.3:0/2970200157 conn(0x7f54300a3050 msgr2=0x7f543009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:38.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.583+0000 7f54356ce700 1 -- 172.21.15.3:0/2970200157 shutdown_connections 2023-10-03T10:13:38.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.583+0000 7f54356ce700 1 -- 172.21.15.3:0/2970200157 wait complete. 2023-10-03T10:13:38.598 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:38.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.601+0000 7fd63af8c700 1 Processor -- start 2023-10-03T10:13:38.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.601+0000 7fd63af8c700 1 -- start start 2023-10-03T10:13:38.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.602+0000 7fd63af8c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6341338c0 0x7fd634133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:38.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.602+0000 7fd63af8c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd63409e560 0x7fd63413b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:38.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.602+0000 7fd63af8c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63406f720 con 0x7fd6341341d0 2023-10-03T10:13:38.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.602+0000 7fd63af8c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd634070c80 con 0x7fd6341338c0 2023-10-03T10:13:38.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.602+0000 7fd63af8c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd63406fce0 con 0x7fd63409e560 2023-10-03T10:13:38.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.602+0000 7fd638d28700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6341338c0 0x7fd634133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:38.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.602+0000 7fd633fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd6341341d0 0x7fd63409de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35390/0 (socket says 172.21.15.3:35390) 2023-10-03T10:13:38.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.602+0000 7fd633fff700 1 -- 172.21.15.3:0/394062981 learned_addr learned my addr 172.21.15.3:0/394062981 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:38.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.602+0000 7fd639529700 1 --2- 172.21.15.3:0/394062981 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd63409e560 0x7fd63413b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:38.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.602+0000 7fd6337fe700 1 -- 172.21.15.3:0/394062981 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2772511071 0 0) 0x7fd63406f720 con 0x7fd6341341d0 2023-10-03T10:13:38.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.602+0000 7fd6337fe700 1 -- 172.21.15.3:0/394062981 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6240035b0 con 0x7fd6341341d0 2023-10-03T10:13:38.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.602+0000 7fd638d28700 1 -- 172.21.15.3:0/394062981 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd63409e560 msgr2=0x7fd63413b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:38.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.602+0000 7fd638d28700 1 --2- 172.21.15.3:0/394062981 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd63409e560 0x7fd63413b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.602+0000 7fd638d28700 1 -- 172.21.15.3:0/394062981 >> v1:172.21.15.3:6789/0 conn(0x7fd6341341d0 legacy=0x7fd63409de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:38.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.602+0000 7fd638d28700 1 -- 172.21.15.3:0/394062981 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6341336a0 con 0x7fd6341338c0 2023-10-03T10:13:38.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.602+0000 7fd639529700 1 --2- 172.21.15.3:0/394062981 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd63409e560 0x7fd63413b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:38.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.603+0000 7fd638d28700 1 --2- 172.21.15.3:0/394062981 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6341338c0 0x7fd634133c90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd628009870 tx=0x7fd628009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b40fa988904aa0d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:38.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.603+0000 7fd6337fe700 1 -- 172.21.15.3:0/394062981 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd628011830 con 0x7fd6341338c0 2023-10-03T10:13:38.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.603+0000 7fd6337fe700 1 -- 172.21.15.3:0/394062981 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd628011990 con 0x7fd6341338c0 2023-10-03T10:13:38.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.603+0000 7fd6337fe700 1 -- 172.21.15.3:0/394062981 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd628011d20 con 0x7fd6341338c0 2023-10-03T10:13:38.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.603+0000 7fd63af8c700 1 -- 172.21.15.3:0/394062981 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6341338c0 msgr2=0x7fd634133c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:38.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.603+0000 7fd63af8c700 1 --2- 172.21.15.3:0/394062981 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6341338c0 0x7fd634133c90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd628009870 tx=0x7fd628009b80 comp rx=0 tx=0).stop 2023-10-03T10:13:38.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.603+0000 7fd63af8c700 1 -- 172.21.15.3:0/394062981 shutdown_connections 2023-10-03T10:13:38.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.603+0000 7fd63af8c700 1 --2- 172.21.15.3:0/394062981 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6341338c0 0x7fd634133c90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.603+0000 7fd63af8c700 1 --2- 172.21.15.3:0/394062981 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd63409e560 0x7fd63413b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.603+0000 7fd63af8c700 1 -- 172.21.15.3:0/394062981 >> 172.21.15.3:0/394062981 conn(0x7fd6340a2aa0 msgr2=0x7fd6340a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:38.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.604+0000 7fd63af8c700 1 -- 172.21.15.3:0/394062981 shutdown_connections 2023-10-03T10:13:38.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.604+0000 7fd63af8c700 1 -- 172.21.15.3:0/394062981 wait complete. 2023-10-03T10:13:38.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.605+0000 7fd63af8c700 1 Processor -- start 2023-10-03T10:13:38.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.605+0000 7fd63af8c700 1 -- start start 2023-10-03T10:13:38.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.605+0000 7fd63af8c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd63409e560 0x7fd6341c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:38.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.605+0000 7fd63af8c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6341338c0 0x7fd6341c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:38.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.605+0000 7fd63af8c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6341341d0 0x7fd6341c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:38.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.605+0000 7fd63af8c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd634130a70 con 0x7fd63409e560 2023-10-03T10:13:38.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.605+0000 7fd63af8c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63411dc50 con 0x7fd6341338c0 2023-10-03T10:13:38.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.605+0000 7fd63af8c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd63411f440 con 0x7fd6341341d0 2023-10-03T10:13:38.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.605+0000 7fd638d28700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd63409e560 0x7fd6341c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:38.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.605+0000 7fd638d28700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd63409e560 0x7fd6341c7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45278/0 (socket says 172.21.15.3:45278) 2023-10-03T10:13:38.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.605+0000 7fd638d28700 1 -- 172.21.15.3:0/2078839206 learned_addr learned my addr 172.21.15.3:0/2078839206 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:38.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.606+0000 7fd639529700 1 --2- 172.21.15.3:0/2078839206 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6341341d0 0x7fd6341c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:38.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.606+0000 7fd633fff700 1 --2- 172.21.15.3:0/2078839206 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6341338c0 0x7fd6341c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:38.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.606+0000 7fd638d28700 1 -- 172.21.15.3:0/2078839206 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6341341d0 msgr2=0x7fd6341c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:38.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.606+0000 7fd638d28700 1 --2- 172.21.15.3:0/2078839206 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6341341d0 0x7fd6341c8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.606+0000 7fd638d28700 1 -- 172.21.15.3:0/2078839206 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6341338c0 msgr2=0x7fd6341c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:38.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.606+0000 7fd638d28700 1 --2- 172.21.15.3:0/2078839206 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6341338c0 0x7fd6341c8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.606+0000 7fd638d28700 1 -- 172.21.15.3:0/2078839206 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62c009a70 con 0x7fd63409e560 2023-10-03T10:13:38.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.606+0000 7fd638d28700 1 --2- 172.21.15.3:0/2078839206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd63409e560 0x7fd6341c7ed0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd6280090b0 tx=0x7fd628004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:38.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.606+0000 7fd6317fa700 1 -- 172.21.15.3:0/2078839206 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd628004450 con 0x7fd63409e560 2023-10-03T10:13:38.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.606+0000 7fd6317fa700 1 -- 172.21.15.3:0/2078839206 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62801f070 con 0x7fd63409e560 2023-10-03T10:13:38.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.606+0000 7fd63af8c700 1 -- 172.21.15.3:0/2078839206 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6280094a0 con 0x7fd63409e560 2023-10-03T10:13:38.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.606+0000 7fd6317fa700 1 -- 172.21.15.3:0/2078839206 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6280038e0 con 0x7fd63409e560 2023-10-03T10:13:38.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.606+0000 7fd63af8c700 1 -- 172.21.15.3:0/2078839206 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd634130e80 con 0x7fd63409e560 2023-10-03T10:13:38.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.607+0000 7fd6317fa700 1 -- 172.21.15.3:0/2078839206 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd628003a40 con 0x7fd63409e560 2023-10-03T10:13:38.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.608+0000 7fd6317fa700 1 -- 172.21.15.3:0/2078839206 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd628021070 con 0x7fd63409e560 2023-10-03T10:13:38.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.608+0000 7fd63af8c700 1 -- 172.21.15.3:0/2078839206 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd634120230 con 0x7fd63409e560 2023-10-03T10:13:38.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.617+0000 7fd6317fa700 1 -- 172.21.15.3:0/2078839206 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd62801a7b0 con 0x7fd63409e560 2023-10-03T10:13:38.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.647+0000 7f4386aaf700 1 Processor -- start 2023-10-03T10:13:38.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.648+0000 7f4386aaf700 1 -- start start 2023-10-03T10:13:38.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.648+0000 7f4386aaf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f43800a1510 0x7f4380138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:38.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.648+0000 7f4386aaf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4380139250 0x7f438013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:38.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.648+0000 7f4386aaf700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f438009edd0 con 0x7f43800a0a30 2023-10-03T10:13:38.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.648+0000 7f4386aaf700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43800a18e0 con 0x7f4380139250 2023-10-03T10:13:38.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.648+0000 7f4386aaf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f438009e1f0 con 0x7f43800a1510 2023-10-03T10:13:38.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.648+0000 7f437ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f43800a1510 0x7f4380138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:38.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.648+0000 7f438484b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f43800a0a30 0x7f43800a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35402/0 (socket says 172.21.15.3:35402) 2023-10-03T10:13:38.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.648+0000 7f438484b700 1 -- 172.21.15.3:0/182567325 learned_addr learned my addr 172.21.15.3:0/182567325 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:38.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.648+0000 7f438504c700 1 --2- 172.21.15.3:0/182567325 >> v2:172.21.15.3:3300/0 conn(0x7f4380139250 0x7f438013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:38.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.649+0000 7f437f7fe700 1 -- 172.21.15.3:0/182567325 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3891484344 0 0) 0x7f438009edd0 con 0x7f43800a0a30 2023-10-03T10:13:38.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.649+0000 7f437ffff700 1 -- 172.21.15.3:0/182567325 >> v2:172.21.15.3:3300/0 conn(0x7f4380139250 msgr2=0x7f438013b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:38.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.649+0000 7f437ffff700 1 --2- 172.21.15.3:0/182567325 >> v2:172.21.15.3:3300/0 conn(0x7f4380139250 0x7f438013b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.649+0000 7f437ffff700 1 -- 172.21.15.3:0/182567325 >> v1:172.21.15.3:6789/0 conn(0x7f43800a0a30 legacy=0x7f43800a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:38.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.649+0000 7f437ffff700 1 -- 172.21.15.3:0/182567325 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f438009dc50 con 0x7f43800a1510 2023-10-03T10:13:38.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.649+0000 7f437ffff700 1 --2- 172.21.15.3:0/182567325 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f43800a1510 0x7f4380138d10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f436c009870 tx=0x7f436c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5077ed52a15760b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:38.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.649+0000 7f437f7fe700 1 -- 172.21.15.3:0/182567325 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436c011830 con 0x7f43800a1510 2023-10-03T10:13:38.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.649+0000 7f437f7fe700 1 -- 172.21.15.3:0/182567325 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436c011990 con 0x7f43800a1510 2023-10-03T10:13:38.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.649+0000 7f437f7fe700 1 -- 172.21.15.3:0/182567325 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436c011d20 con 0x7f43800a1510 2023-10-03T10:13:38.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.650+0000 7f4386aaf700 1 -- 172.21.15.3:0/182567325 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f43800a1510 msgr2=0x7f4380138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:38.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.650+0000 7f4386aaf700 1 --2- 172.21.15.3:0/182567325 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f43800a1510 0x7f4380138d10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f436c009870 tx=0x7f436c009b80 comp rx=0 tx=0).stop 2023-10-03T10:13:38.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.650+0000 7f4386aaf700 1 -- 172.21.15.3:0/182567325 shutdown_connections 2023-10-03T10:13:38.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.650+0000 7f4386aaf700 1 --2- 172.21.15.3:0/182567325 >> v2:172.21.15.3:3300/0 conn(0x7f4380139250 0x7f438013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.650+0000 7f4386aaf700 1 --2- 172.21.15.3:0/182567325 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f43800a1510 0x7f4380138d10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.650+0000 7f4386aaf700 1 -- 172.21.15.3:0/182567325 >> 172.21.15.3:0/182567325 conn(0x7f43800a6550 msgr2=0x7f43800a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:38.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.650+0000 7f4386aaf700 1 -- 172.21.15.3:0/182567325 shutdown_connections 2023-10-03T10:13:38.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.650+0000 7f4386aaf700 1 -- 172.21.15.3:0/182567325 wait complete. 2023-10-03T10:13:38.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.651+0000 7f4386aaf700 1 Processor -- start 2023-10-03T10:13:38.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.651+0000 7f4386aaf700 1 -- start start 2023-10-03T10:13:38.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.652+0000 7f4386aaf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f43800a0a30 0x7f43801c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:38.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.652+0000 7f4386aaf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43800a1510 0x7f43801c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:38.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.652+0000 7f4386aaf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4380139250 0x7f4380122770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:38.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.652+0000 7f4386aaf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4380124530 con 0x7f43800a1510 2023-10-03T10:13:38.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.652+0000 7f4386aaf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4380124340 con 0x7f4380139250 2023-10-03T10:13:38.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.652+0000 7f4386aaf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f438009ca50 con 0x7f43800a0a30 2023-10-03T10:13:38.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.652+0000 7f437ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43800a1510 0x7f43801c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:38.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.652+0000 7f437ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43800a1510 0x7f43801c8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45284/0 (socket says 172.21.15.3:45284) 2023-10-03T10:13:38.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.652+0000 7f437ffff700 1 -- 172.21.15.3:0/1826297053 learned_addr learned my addr 172.21.15.3:0/1826297053 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:38.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.652+0000 7f438484b700 1 --2- 172.21.15.3:0/1826297053 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f43800a0a30 0x7f43801c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:38.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.652+0000 7f438504c700 1 --2- 172.21.15.3:0/1826297053 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4380139250 0x7f4380122770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:38.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.652+0000 7f437ffff700 1 -- 172.21.15.3:0/1826297053 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f43800a0a30 msgr2=0x7f43801c85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:38.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.652+0000 7f437ffff700 1 --2- 172.21.15.3:0/1826297053 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f43800a0a30 0x7f43801c85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.652+0000 7f437ffff700 1 -- 172.21.15.3:0/1826297053 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4380139250 msgr2=0x7f4380122770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:38.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.652+0000 7f437ffff700 1 --2- 172.21.15.3:0/1826297053 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4380139250 0x7f4380122770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.652+0000 7f437ffff700 1 -- 172.21.15.3:0/1826297053 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f436c0094a0 con 0x7f43800a1510 2023-10-03T10:13:38.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.652+0000 7f438504c700 1 --2- 172.21.15.3:0/1826297053 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4380139250 0x7f4380122770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:13:38.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.653+0000 7f437ffff700 1 --2- 172.21.15.3:0/1826297053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43800a1510 0x7f43801c8af0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f436c000c00 tx=0x7f436c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:38.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.653+0000 7f437d7fa700 1 -- 172.21.15.3:0/1826297053 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436c0036a0 con 0x7f43800a1510 2023-10-03T10:13:38.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.653+0000 7f437d7fa700 1 -- 172.21.15.3:0/1826297053 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436c01f070 con 0x7f43800a1510 2023-10-03T10:13:38.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.653+0000 7f4386aaf700 1 -- 172.21.15.3:0/1826297053 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43800a0220 con 0x7f43800a1510 2023-10-03T10:13:38.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.653+0000 7f437d7fa700 1 -- 172.21.15.3:0/1826297053 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436c003a00 con 0x7f43800a1510 2023-10-03T10:13:38.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.653+0000 7f4386aaf700 1 -- 172.21.15.3:0/1826297053 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43801269e0 con 0x7f43800a1510 2023-10-03T10:13:38.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.654+0000 7f4386aaf700 1 -- 172.21.15.3:0/1826297053 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4380125c10 con 0x7f43800a1510 2023-10-03T10:13:38.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.654+0000 7f437d7fa700 1 -- 172.21.15.3:0/1826297053 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f436c003b60 con 0x7f43800a1510 2023-10-03T10:13:38.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.654+0000 7f437d7fa700 1 -- 172.21.15.3:0/1826297053 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f436c021070 con 0x7f43800a1510 2023-10-03T10:13:38.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.663+0000 7f437d7fa700 1 -- 172.21.15.3:0/1826297053 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4380125c10 con 0x7f43800a1510 2023-10-03T10:13:38.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.793+0000 7f32e1bd3700 1 -- 172.21.15.3:0/2109326542 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f32dc06b9e0 con 0x7f32dc09de50 2023-10-03T10:13:38.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.794+0000 7f32c3fff700 1 -- 172.21.15.3:0/2109326542 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f32cc02a090 con 0x7f32dc09de50 2023-10-03T10:13:38.794 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:38.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.796+0000 7f32e1bd3700 1 -- 172.21.15.3:0/2109326542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32dc09de50 msgr2=0x7f32dc11ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:38.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.796+0000 7f32e1bd3700 1 --2- 172.21.15.3:0/2109326542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32dc09de50 0x7f32dc11ea70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f32cc00e9b0 tx=0x7f32cc00ecc0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.796+0000 7f32e1bd3700 1 -- 172.21.15.3:0/2109326542 shutdown_connections 2023-10-03T10:13:38.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.796+0000 7f32e1bd3700 1 --2- 172.21.15.3:0/2109326542 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f32dc1338a0 0x7f32dc11f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.796+0000 7f32e1bd3700 1 --2- 172.21.15.3:0/2109326542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32dc09de50 0x7f32dc11ea70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.796+0000 7f32e1bd3700 1 --2- 172.21.15.3:0/2109326542 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f32dc09e760 0x7f32dc11efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.796+0000 7f32e1bd3700 1 -- 172.21.15.3:0/2109326542 >> 172.21.15.3:0/2109326542 conn(0x7f32dc0a2aa0 msgr2=0x7f32dc09ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:38.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.796+0000 7f32e1bd3700 1 -- 172.21.15.3:0/2109326542 shutdown_connections 2023-10-03T10:13:38.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.796+0000 7f32e1bd3700 1 -- 172.21.15.3:0/2109326542 wait complete. 2023-10-03T10:13:38.810 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:38.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.826+0000 7fd63af8c700 1 -- 172.21.15.3:0/2078839206 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd634074ea0 con 0x7fd63409e560 2023-10-03T10:13:38.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.826+0000 7fd6317fa700 1 -- 172.21.15.3:0/2078839206 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd62802b030 con 0x7fd63409e560 2023-10-03T10:13:38.826 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:38.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.828+0000 7fd63af8c700 1 -- 172.21.15.3:0/2078839206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd63409e560 msgr2=0x7fd6341c7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:38.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.828+0000 7fd63af8c700 1 --2- 172.21.15.3:0/2078839206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd63409e560 0x7fd6341c7ed0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd6280090b0 tx=0x7fd628004750 comp rx=0 tx=0).stop 2023-10-03T10:13:38.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.828+0000 7fd63af8c700 1 -- 172.21.15.3:0/2078839206 shutdown_connections 2023-10-03T10:13:38.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.828+0000 7fd63af8c700 1 --2- 172.21.15.3:0/2078839206 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6341341d0 0x7fd6341c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.828+0000 7fd63af8c700 1 --2- 172.21.15.3:0/2078839206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd63409e560 0x7fd6341c7ed0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.828+0000 7fd63af8c700 1 --2- 172.21.15.3:0/2078839206 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6341338c0 0x7fd6341c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.828+0000 7fd63af8c700 1 -- 172.21.15.3:0/2078839206 >> 172.21.15.3:0/2078839206 conn(0x7fd6340a2aa0 msgr2=0x7fd63409f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:38.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.828+0000 7fd63af8c700 1 -- 172.21.15.3:0/2078839206 shutdown_connections 2023-10-03T10:13:38.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.828+0000 7fd63af8c700 1 -- 172.21.15.3:0/2078839206 wait complete. 2023-10-03T10:13:38.842 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:38.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.872+0000 7f4386aaf700 1 -- 172.21.15.3:0/1826297053 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4380125c10 con 0x7f43800a1510 2023-10-03T10:13:38.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.872+0000 7f437d7fa700 1 -- 172.21.15.3:0/1826297053 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4380125c10 con 0x7f43800a1510 2023-10-03T10:13:38.873 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:38.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.875+0000 7f4386aaf700 1 -- 172.21.15.3:0/1826297053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43800a1510 msgr2=0x7f43801c8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:38.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.875+0000 7f4386aaf700 1 --2- 172.21.15.3:0/1826297053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43800a1510 0x7f43801c8af0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f436c000c00 tx=0x7f436c004750 comp rx=0 tx=0).stop 2023-10-03T10:13:38.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.875+0000 7f4386aaf700 1 -- 172.21.15.3:0/1826297053 shutdown_connections 2023-10-03T10:13:38.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.875+0000 7f4386aaf700 1 --2- 172.21.15.3:0/1826297053 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4380139250 0x7f4380122770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.875+0000 7f4386aaf700 1 --2- 172.21.15.3:0/1826297053 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f43800a0a30 0x7f43801c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.875+0000 7f4386aaf700 1 --2- 172.21.15.3:0/1826297053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43800a1510 0x7f43801c8af0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:38.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.875+0000 7f4386aaf700 1 -- 172.21.15.3:0/1826297053 >> 172.21.15.3:0/1826297053 conn(0x7f43800a6550 msgr2=0x7f43800a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:38.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.875+0000 7f4386aaf700 1 -- 172.21.15.3:0/1826297053 shutdown_connections 2023-10-03T10:13:38.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:38.875+0000 7f4386aaf700 1 -- 172.21.15.3:0/1826297053 wait complete. 2023-10-03T10:13:38.888 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:39.599 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:39.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.744+0000 7f1ff6b05700 1 Processor -- start 2023-10-03T10:13:39.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.744+0000 7f1ff6b05700 1 -- start start 2023-10-03T10:13:39.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.744+0000 7f1ff6b05700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ff0133420 0x7f1ff01337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:39.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.744+0000 7f1ff6b05700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ff0134260 0x7f1ff01306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:39.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.745+0000 7f1ff6b05700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ff006b990 con 0x7f1ff0133e30 2023-10-03T10:13:39.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.745+0000 7f1ff6b05700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff00a0590 con 0x7f1ff0134260 2023-10-03T10:13:39.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.745+0000 7f1ff6b05700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff009fb80 con 0x7f1ff0133420 2023-10-03T10:13:39.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.745+0000 7f1ff50a2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ff0134260 0x7f1ff01306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:39.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.745+0000 7f1ff48a1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ff0133420 0x7f1ff01337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:39.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.745+0000 7f1ff50a2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ff0134260 0x7f1ff01306a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44686/0 (socket says 172.21.15.3:44686) 2023-10-03T10:13:39.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.745+0000 7f1ff48a1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ff0133420 0x7f1ff01337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56990/0 (socket says 172.21.15.3:56990) 2023-10-03T10:13:39.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.745+0000 7f1ff50a2700 1 -- 172.21.15.3:0/226970173 learned_addr learned my addr 172.21.15.3:0/226970173 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:39.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.745+0000 7f1feffff700 1 --1- 172.21.15.3:0/226970173 >> v1:172.21.15.3:6789/0 conn(0x7f1ff0133e30 0x7f1ff012ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35408/0 (socket says 172.21.15.3:35408) 2023-10-03T10:13:39.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.745+0000 7f1ff50a2700 1 -- 172.21.15.3:0/226970173 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ff0133420 msgr2=0x7f1ff01337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:39.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.745+0000 7f1ff50a2700 1 --2- 172.21.15.3:0/226970173 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ff0133420 0x7f1ff01337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:39.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.745+0000 7f1ff50a2700 1 -- 172.21.15.3:0/226970173 >> v1:172.21.15.3:6789/0 conn(0x7f1ff0133e30 legacy=0x7f1ff012ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:39.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.745+0000 7f1fef7fe700 1 -- 172.21.15.3:0/226970173 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1157671940 0 0) 0x7f1ff006b990 con 0x7f1ff0133e30 2023-10-03T10:13:39.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.745+0000 7f1ff50a2700 1 -- 172.21.15.3:0/226970173 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff00a0030 con 0x7f1ff0134260 2023-10-03T10:13:39.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.745+0000 7f1ff48a1700 1 --2- 172.21.15.3:0/226970173 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ff0133420 0x7f1ff01337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:39.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.745+0000 7f1ff50a2700 1 --2- 172.21.15.3:0/226970173 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ff0134260 0x7f1ff01306a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009e80 tx=0x7f1fe800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0305be24d028af4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:39.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.745+0000 7f1fef7fe700 1 -- 172.21.15.3:0/226970173 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe800e070 con 0x7f1ff0134260 2023-10-03T10:13:39.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.746+0000 7f1fef7fe700 1 -- 172.21.15.3:0/226970173 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe800b910 con 0x7f1ff0134260 2023-10-03T10:13:39.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.746+0000 7f1fef7fe700 1 -- 172.21.15.3:0/226970173 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe800bcc0 con 0x7f1ff0134260 2023-10-03T10:13:39.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.746+0000 7f1ff6b05700 1 -- 172.21.15.3:0/226970173 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ff0134260 msgr2=0x7f1ff01306a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:39.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.746+0000 7f1ff6b05700 1 --2- 172.21.15.3:0/226970173 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ff0134260 0x7f1ff01306a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009e80 tx=0x7f1fe800b330 comp rx=0 tx=0).stop 2023-10-03T10:13:39.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.746+0000 7f1ff6b05700 1 -- 172.21.15.3:0/226970173 shutdown_connections 2023-10-03T10:13:39.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.746+0000 7f1ff6b05700 1 --2- 172.21.15.3:0/226970173 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ff0134260 0x7f1ff01306a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:39.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.746+0000 7f1ff6b05700 1 --2- 172.21.15.3:0/226970173 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ff0133420 0x7f1ff01337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:39.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.746+0000 7f1ff6b05700 1 -- 172.21.15.3:0/226970173 >> 172.21.15.3:0/226970173 conn(0x7f1ff00a4210 msgr2=0x7f1ff00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:39.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.746+0000 7f1ff6b05700 1 -- 172.21.15.3:0/226970173 shutdown_connections 2023-10-03T10:13:39.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.746+0000 7f1ff6b05700 1 -- 172.21.15.3:0/226970173 wait complete. 2023-10-03T10:13:39.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.747+0000 7f1ff6b05700 1 Processor -- start 2023-10-03T10:13:39.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.747+0000 7f1ff6b05700 1 -- start start 2023-10-03T10:13:39.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.748+0000 7f1ff6b05700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ff0133420 0x7f1ff009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:39.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.748+0000 7f1ff6b05700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ff0133e30 0x7f1ff009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:39.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.748+0000 7f1ff6b05700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ff0134260 0x7f1ff009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:39.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.748+0000 7f1ff6b05700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff012d9b0 con 0x7f1ff0133e30 2023-10-03T10:13:39.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.748+0000 7f1ff6b05700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff009d850 con 0x7f1ff0134260 2023-10-03T10:13:39.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.748+0000 7f1ff6b05700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff011feb0 con 0x7f1ff0133420 2023-10-03T10:13:39.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.748+0000 7f1ff48a1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ff0133420 0x7f1ff009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:39.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.748+0000 7f1feffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ff0133e30 0x7f1ff009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:39.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.748+0000 7f1ff50a2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ff0134260 0x7f1ff009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:39.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.748+0000 7f1ff48a1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ff0133420 0x7f1ff009c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57004/0 (socket says 172.21.15.3:57004) 2023-10-03T10:13:39.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.748+0000 7f1feffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ff0133e30 0x7f1ff009c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45300/0 (socket says 172.21.15.3:45300) 2023-10-03T10:13:39.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.748+0000 7f1ff48a1700 1 -- 172.21.15.3:0/2501164547 learned_addr learned my addr 172.21.15.3:0/2501164547 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:39.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.748+0000 7f1ff50a2700 1 -- 172.21.15.3:0/2501164547 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ff0133420 msgr2=0x7f1ff009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:39.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.748+0000 7f1ff50a2700 1 --2- 172.21.15.3:0/2501164547 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ff0133420 0x7f1ff009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:39.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.748+0000 7f1ff50a2700 1 -- 172.21.15.3:0/2501164547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ff0133e30 msgr2=0x7f1ff009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:39.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.748+0000 7f1ff50a2700 1 --2- 172.21.15.3:0/2501164547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ff0133e30 0x7f1ff009c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:39.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.748+0000 7f1ff50a2700 1 -- 172.21.15.3:0/2501164547 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe40094a0 con 0x7f1ff0134260 2023-10-03T10:13:39.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.749+0000 7f1ff50a2700 1 --2- 172.21.15.3:0/2501164547 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ff0134260 0x7f1ff009cf00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8015000 tx=0x7f1fe8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:39.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.749+0000 7f1fed7fa700 1 -- 172.21.15.3:0/2501164547 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8004670 con 0x7f1ff0134260 2023-10-03T10:13:39.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.749+0000 7f1fed7fa700 1 -- 172.21.15.3:0/2501164547 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe80092e0 con 0x7f1ff0134260 2023-10-03T10:13:39.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.749+0000 7f1ff6b05700 1 -- 172.21.15.3:0/2501164547 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe8009b30 con 0x7f1ff0134260 2023-10-03T10:13:39.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.749+0000 7f1fed7fa700 1 -- 172.21.15.3:0/2501164547 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe801c410 con 0x7f1ff0134260 2023-10-03T10:13:39.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.749+0000 7f1ff6b05700 1 -- 172.21.15.3:0/2501164547 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff0131590 con 0x7f1ff0134260 2023-10-03T10:13:39.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.750+0000 7f1fed7fa700 1 -- 172.21.15.3:0/2501164547 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1fe801c570 con 0x7f1ff0134260 2023-10-03T10:13:39.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.750+0000 7f1fed7fa700 1 -- 172.21.15.3:0/2501164547 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1fe8041490 con 0x7f1ff0134260 2023-10-03T10:13:39.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.750+0000 7f1ff6b05700 1 -- 172.21.15.3:0/2501164547 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff0074ea0 con 0x7f1ff0134260 2023-10-03T10:13:39.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.755+0000 7f1fed7fa700 1 -- 172.21.15.3:0/2501164547 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1fe801ca00 con 0x7f1ff0134260 2023-10-03T10:13:39.811 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:39.843 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:39.890 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:39.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.962+0000 7ff7c1d30700 1 Processor -- start 2023-10-03T10:13:39.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.962+0000 7ff7c1d30700 1 -- start start 2023-10-03T10:13:39.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.963+0000 7ff7c1d30700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7bc134630 0x7ff7bc136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:39.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.963+0000 7ff7c1d30700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff7bc136f50 0x7ff7bc139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:39.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.963+0000 7ff7c1d30700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7bc09b4f0 con 0x7ff7bc131810 2023-10-03T10:13:39.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.963+0000 7ff7c1d30700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7bc1313c0 con 0x7ff7bc136f50 2023-10-03T10:13:39.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.963+0000 7ff7c1d30700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc06e8c0 con 0x7ff7bc134630 2023-10-03T10:13:39.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.963+0000 7ff7bb7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff7bc131810 0x7ff7bc131be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35410/0 (socket says 172.21.15.3:35410) 2023-10-03T10:13:39.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.963+0000 7ff7bb7fe700 1 -- 172.21.15.3:0/250533892 learned_addr learned my addr 172.21.15.3:0/250533892 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:39.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.963+0000 7ff7bbfff700 1 --2- 172.21.15.3:0/250533892 >> v2:172.21.15.3:3300/0 conn(0x7ff7bc136f50 0x7ff7bc139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:39.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.964+0000 7ff7b2ffd700 1 --2- 172.21.15.3:0/250533892 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7bc134630 0x7ff7bc136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:39.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.964+0000 7ff7baffd700 1 -- 172.21.15.3:0/250533892 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2223849405 0 0) 0x7ff7bc09b4f0 con 0x7ff7bc131810 2023-10-03T10:13:39.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.964+0000 7ff7baffd700 1 -- 172.21.15.3:0/250533892 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7a80035b0 con 0x7ff7bc131810 2023-10-03T10:13:39.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.964+0000 7ff7bbfff700 1 -- 172.21.15.3:0/250533892 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7bc134630 msgr2=0x7ff7bc136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:39.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.964+0000 7ff7bbfff700 1 --2- 172.21.15.3:0/250533892 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7bc134630 0x7ff7bc136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:39.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.964+0000 7ff7bbfff700 1 -- 172.21.15.3:0/250533892 >> v1:172.21.15.3:6789/0 conn(0x7ff7bc131810 legacy=0x7ff7bc131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:39.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.964+0000 7ff7bbfff700 1 -- 172.21.15.3:0/250533892 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7bc071d20 con 0x7ff7bc136f50 2023-10-03T10:13:39.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.964+0000 7ff7bbfff700 1 --2- 172.21.15.3:0/250533892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7bc136f50 0x7ff7bc139330 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009840 tx=0x7ff7b4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a4cbf8040633c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:39.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.964+0000 7ff7baffd700 1 -- 172.21.15.3:0/250533892 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4011830 con 0x7ff7bc136f50 2023-10-03T10:13:39.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.964+0000 7ff7baffd700 1 -- 172.21.15.3:0/250533892 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b4011990 con 0x7ff7bc136f50 2023-10-03T10:13:39.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.964+0000 7ff7baffd700 1 -- 172.21.15.3:0/250533892 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4011d20 con 0x7ff7bc136f50 2023-10-03T10:13:39.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.965+0000 7ff7c1d30700 1 -- 172.21.15.3:0/250533892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7bc136f50 msgr2=0x7ff7bc139330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:39.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.965+0000 7ff7c1d30700 1 --2- 172.21.15.3:0/250533892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7bc136f50 0x7ff7bc139330 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009840 tx=0x7ff7b4009b50 comp rx=0 tx=0).stop 2023-10-03T10:13:39.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.965+0000 7ff7c1d30700 1 -- 172.21.15.3:0/250533892 shutdown_connections 2023-10-03T10:13:39.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.965+0000 7ff7c1d30700 1 --2- 172.21.15.3:0/250533892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7bc136f50 0x7ff7bc139330 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:39.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.965+0000 7ff7c1d30700 1 --2- 172.21.15.3:0/250533892 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7bc134630 0x7ff7bc136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:39.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.965+0000 7ff7c1d30700 1 -- 172.21.15.3:0/250533892 >> 172.21.15.3:0/250533892 conn(0x7ff7bc0a2aa0 msgr2=0x7ff7bc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:39.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.965+0000 7ff7c1d30700 1 -- 172.21.15.3:0/250533892 shutdown_connections 2023-10-03T10:13:39.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.966+0000 7ff7c1d30700 1 -- 172.21.15.3:0/250533892 wait complete. 2023-10-03T10:13:39.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.967+0000 7ff7c1d30700 1 Processor -- start 2023-10-03T10:13:39.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.967+0000 7ff7c1d30700 1 -- start start 2023-10-03T10:13:39.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.967+0000 7ff7c1d30700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7bc131810 0x7ff7bc06f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:39.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.967+0000 7ff7c1d30700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff7bc134630 0x7ff7bc06fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:39.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.967+0000 7ff7c1d30700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7bc136f50 0x7ff7bc070060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:39.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.967+0000 7ff7c1d30700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc11e520 con 0x7ff7bc136f50 2023-10-03T10:13:39.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.967+0000 7ff7c1d30700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc071110 con 0x7ff7bc131810 2023-10-03T10:13:39.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.967+0000 7ff7c1d30700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc0715b0 con 0x7ff7bc134630 2023-10-03T10:13:39.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.967+0000 7ff7bbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7bc136f50 0x7ff7bc070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:39.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.967+0000 7ff7bbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7bc136f50 0x7ff7bc070060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45316/0 (socket says 172.21.15.3:45316) 2023-10-03T10:13:39.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.967+0000 7ff7bbfff700 1 -- 172.21.15.3:0/874904270 learned_addr learned my addr 172.21.15.3:0/874904270 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:39.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.968+0000 7ff7bb7fe700 1 --2- 172.21.15.3:0/874904270 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7bc131810 0x7ff7bc06f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:39.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.968+0000 7ff7bbfff700 1 -- 172.21.15.3:0/874904270 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff7bc134630 msgr2=0x7ff7bc06fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:39.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.968+0000 7ff7bbfff700 1 --2- 172.21.15.3:0/874904270 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff7bc134630 0x7ff7bc06fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:39.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.968+0000 7ff7bbfff700 1 -- 172.21.15.3:0/874904270 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7bc131810 msgr2=0x7ff7bc06f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:39.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.968+0000 7ff7bbfff700 1 --2- 172.21.15.3:0/874904270 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7bc131810 0x7ff7bc06f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:39.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.968+0000 7ff7bbfff700 1 -- 172.21.15.3:0/874904270 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b40094a0 con 0x7ff7bc136f50 2023-10-03T10:13:39.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.968+0000 7ff7bbfff700 1 --2- 172.21.15.3:0/874904270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7bc136f50 0x7ff7bc070060 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009b90 tx=0x7ff7b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:39.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.968+0000 7ff7b8ff9700 1 -- 172.21.15.3:0/874904270 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b40044f0 con 0x7ff7bc136f50 2023-10-03T10:13:39.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.968+0000 7ff7b8ff9700 1 -- 172.21.15.3:0/874904270 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b401f070 con 0x7ff7bc136f50 2023-10-03T10:13:39.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.968+0000 7ff7c1d30700 1 -- 172.21.15.3:0/874904270 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7bc0a1d60 con 0x7ff7bc136f50 2023-10-03T10:13:39.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.968+0000 7ff7b8ff9700 1 -- 172.21.15.3:0/874904270 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b40038e0 con 0x7ff7bc136f50 2023-10-03T10:13:39.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.968+0000 7ff7c1d30700 1 -- 172.21.15.3:0/874904270 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7bc11ef90 con 0x7ff7bc136f50 2023-10-03T10:13:39.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.969+0000 7ff7b8ff9700 1 -- 172.21.15.3:0/874904270 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff7b4021030 con 0x7ff7bc136f50 2023-10-03T10:13:39.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.970+0000 7ff7b8ff9700 1 -- 172.21.15.3:0/874904270 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff7b4040b80 con 0x7ff7bc136f50 2023-10-03T10:13:39.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.970+0000 7ff7c1d30700 1 -- 172.21.15.3:0/874904270 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7bc12ecf0 con 0x7ff7bc136f50 2023-10-03T10:13:39.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.972+0000 7f1ff6b05700 1 -- 172.21.15.3:0/2501164547 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1ff006b250 con 0x7f1ff0134260 2023-10-03T10:13:39.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.972+0000 7f1fed7fa700 1 -- 172.21.15.3:0/2501164547 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1fe8033090 con 0x7f1ff0134260 2023-10-03T10:13:39.972 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:39.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.974+0000 7f1ff6b05700 1 -- 172.21.15.3:0/2501164547 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ff0134260 msgr2=0x7f1ff009cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:39.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.974+0000 7f1ff6b05700 1 --2- 172.21.15.3:0/2501164547 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ff0134260 0x7f1ff009cf00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8015000 tx=0x7f1fe8004ad0 comp rx=0 tx=0).stop 2023-10-03T10:13:39.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.975+0000 7f1ff6b05700 1 -- 172.21.15.3:0/2501164547 shutdown_connections 2023-10-03T10:13:39.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.975+0000 7f1ff6b05700 1 --2- 172.21.15.3:0/2501164547 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ff0134260 0x7f1ff009cf00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:39.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.975+0000 7f1ff6b05700 1 --2- 172.21.15.3:0/2501164547 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ff0133420 0x7f1ff009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:39.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.975+0000 7f1ff6b05700 1 --2- 172.21.15.3:0/2501164547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ff0133e30 0x7f1ff009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:39.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.975+0000 7f1ff6b05700 1 -- 172.21.15.3:0/2501164547 >> 172.21.15.3:0/2501164547 conn(0x7f1ff00a4210 msgr2=0x7f1ff00a3710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:39.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.975+0000 7f1ff6b05700 1 -- 172.21.15.3:0/2501164547 shutdown_connections 2023-10-03T10:13:39.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.975+0000 7f1ff6b05700 1 -- 172.21.15.3:0/2501164547 wait complete. 2023-10-03T10:13:39.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:39.979+0000 7ff7b8ff9700 1 -- 172.21.15.3:0/874904270 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff7b4040e60 con 0x7ff7bc136f50 2023-10-03T10:13:39.995 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:40.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.001+0000 7fcc2774f700 1 Processor -- start 2023-10-03T10:13:40.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.001+0000 7fcc2774f700 1 -- start start 2023-10-03T10:13:40.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.002+0000 7fcc2774f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcc200954c0 0x7fcc20095890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:40.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.002+0000 7fcc2774f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcc20099680 0x7fcc20099ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:40.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.002+0000 7fcc2774f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc2009c280 con 0x7fcc20099ff0 2023-10-03T10:13:40.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.002+0000 7fcc2774f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc2009aaf0 con 0x7fcc20099680 2023-10-03T10:13:40.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.002+0000 7fcc2774f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2009b960 con 0x7fcc200954c0 2023-10-03T10:13:40.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.002+0000 7fcc24cea700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcc20099680 0x7fcc20099ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:40.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.002+0000 7fcc25cec700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcc20099ff0 0x7fcc20137590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35418/0 (socket says 172.21.15.3:35418) 2023-10-03T10:13:40.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.002+0000 7fcc254eb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcc200954c0 0x7fcc20095890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:40.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.002+0000 7fcc25cec700 1 -- 172.21.15.3:0/2576424072 learned_addr learned my addr 172.21.15.3:0/2576424072 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:40.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.003+0000 7fcc254eb700 1 -- 172.21.15.3:0/2576424072 >> v2:172.21.15.3:3300/0 conn(0x7fcc20099680 msgr2=0x7fcc20099ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:40.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.003+0000 7fcc254eb700 1 --2- 172.21.15.3:0/2576424072 >> v2:172.21.15.3:3300/0 conn(0x7fcc20099680 0x7fcc20099ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.003+0000 7fcc254eb700 1 -- 172.21.15.3:0/2576424072 >> v1:172.21.15.3:6789/0 conn(0x7fcc20099ff0 legacy=0x7fcc20137590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.003+0000 7fcc254eb700 1 -- 172.21.15.3:0/2576424072 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2009a7b0 con 0x7fcc200954c0 2023-10-03T10:13:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.003+0000 7fcc24cea700 1 --2- 172.21.15.3:0/2576424072 >> v2:172.21.15.3:3300/0 conn(0x7fcc20099680 0x7fcc20099ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.003+0000 7fcc254eb700 1 --2- 172.21.15.3:0/2576424072 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcc200954c0 0x7fcc20095890 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00dc60 tx=0x7fcc1c00df70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3586e7b1018f465e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.003+0000 7fcc17fff700 1 -- 172.21.15.3:0/2576424072 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c012070 con 0x7fcc200954c0 2023-10-03T10:13:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.003+0000 7fcc17fff700 1 -- 172.21.15.3:0/2576424072 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c00d2e0 con 0x7fcc200954c0 2023-10-03T10:13:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.003+0000 7fcc17fff700 1 -- 172.21.15.3:0/2576424072 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c00f900 con 0x7fcc200954c0 2023-10-03T10:13:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.004+0000 7fcc2774f700 1 -- 172.21.15.3:0/2576424072 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcc200954c0 msgr2=0x7fcc20095890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.004+0000 7fcc2774f700 1 --2- 172.21.15.3:0/2576424072 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcc200954c0 0x7fcc20095890 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00dc60 tx=0x7fcc1c00df70 comp rx=0 tx=0).stop 2023-10-03T10:13:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.004+0000 7fcc2774f700 1 -- 172.21.15.3:0/2576424072 shutdown_connections 2023-10-03T10:13:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.004+0000 7fcc2774f700 1 --2- 172.21.15.3:0/2576424072 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcc200954c0 0x7fcc20095890 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.004+0000 7fcc2774f700 1 --2- 172.21.15.3:0/2576424072 >> v2:172.21.15.3:3300/0 conn(0x7fcc20099680 0x7fcc20099ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.004+0000 7fcc2774f700 1 -- 172.21.15.3:0/2576424072 >> 172.21.15.3:0/2576424072 conn(0x7fcc200a2360 msgr2=0x7fcc200a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:40.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.004+0000 7fcc2774f700 1 -- 172.21.15.3:0/2576424072 shutdown_connections 2023-10-03T10:13:40.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.004+0000 7fcc2774f700 1 -- 172.21.15.3:0/2576424072 wait complete. 2023-10-03T10:13:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.005+0000 7fcc2774f700 1 Processor -- start 2023-10-03T10:13:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.005+0000 7fcc2774f700 1 -- start start 2023-10-03T10:13:40.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.006+0000 7fcc2774f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcc200954c0 0x7fcc2008c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:40.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.006+0000 7fcc2774f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcc20099680 0x7fcc2008c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:40.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.006+0000 7fcc2774f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc20099ff0 0x7fcc2008caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:40.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.006+0000 7fcc2774f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2008d5d0 con 0x7fcc20099ff0 2023-10-03T10:13:40.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.006+0000 7fcc2774f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2008de50 con 0x7fcc20099680 2023-10-03T10:13:40.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.006+0000 7fcc2774f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc200a0a70 con 0x7fcc200954c0 2023-10-03T10:13:40.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.006+0000 7fcc254eb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcc200954c0 0x7fcc2008c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:40.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.006+0000 7fcc25cec700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc20099ff0 0x7fcc2008caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:40.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.006+0000 7fcc254eb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcc200954c0 0x7fcc2008c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57018/0 (socket says 172.21.15.3:57018) 2023-10-03T10:13:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.006+0000 7fcc254eb700 1 -- 172.21.15.3:0/979520930 learned_addr learned my addr 172.21.15.3:0/979520930 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.006+0000 7fcc24cea700 1 --2- 172.21.15.3:0/979520930 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcc20099680 0x7fcc2008c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.007+0000 7fcc254eb700 1 -- 172.21.15.3:0/979520930 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcc20099680 msgr2=0x7fcc2008c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.007+0000 7fcc254eb700 1 --2- 172.21.15.3:0/979520930 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcc20099680 0x7fcc2008c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.007+0000 7fcc254eb700 1 -- 172.21.15.3:0/979520930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc20099ff0 msgr2=0x7fcc2008caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.007+0000 7fcc254eb700 1 --2- 172.21.15.3:0/979520930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc20099ff0 0x7fcc2008caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.007+0000 7fcc254eb700 1 -- 172.21.15.3:0/979520930 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1000b550 con 0x7fcc200954c0 2023-10-03T10:13:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.007+0000 7fcc254eb700 1 --2- 172.21.15.3:0/979520930 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcc200954c0 0x7fcc2008c020 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcc2009baa0 tx=0x7fcc1c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.007+0000 7fcc15ffb700 1 -- 172.21.15.3:0/979520930 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c0036a0 con 0x7fcc200954c0 2023-10-03T10:13:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.007+0000 7fcc15ffb700 1 -- 172.21.15.3:0/979520930 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c003820 con 0x7fcc200954c0 2023-10-03T10:13:40.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.007+0000 7fcc15ffb700 1 -- 172.21.15.3:0/979520930 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c003bd0 con 0x7fcc200954c0 2023-10-03T10:13:40.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.007+0000 7fcc2774f700 1 -- 172.21.15.3:0/979520930 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1c00d940 con 0x7fcc200954c0 2023-10-03T10:13:40.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.008+0000 7fcc2774f700 1 -- 172.21.15.3:0/979520930 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc2008efb0 con 0x7fcc200954c0 2023-10-03T10:13:40.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.008+0000 7fcc15ffb700 1 -- 172.21.15.3:0/979520930 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcc1c003d30 con 0x7fcc200954c0 2023-10-03T10:13:40.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.009+0000 7fcc2774f700 1 -- 172.21.15.3:0/979520930 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc2008f1a0 con 0x7fcc200954c0 2023-10-03T10:13:40.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.009+0000 7fcc15ffb700 1 -- 172.21.15.3:0/979520930 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fcc1c044910 con 0x7fcc200954c0 2023-10-03T10:13:40.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.014+0000 7fcc15ffb700 1 -- 172.21.15.3:0/979520930 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcc1c006cc0 con 0x7fcc200954c0 2023-10-03T10:13:40.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.049+0000 7f6ea70f8700 1 Processor -- start 2023-10-03T10:13:40.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.049+0000 7f6ea70f8700 1 -- start start 2023-10-03T10:13:40.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.050+0000 7f6ea70f8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6ea012cf60 0x7f6ea006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:40.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.050+0000 7f6ea70f8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ea012f8b0 0x7f6ea012fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:40.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.050+0000 7f6ea70f8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ea0130260 0x7f6ea0136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:40.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.050+0000 7f6ea70f8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ea012d330 con 0x7f6ea012cf60 2023-10-03T10:13:40.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.050+0000 7f6ea70f8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea012cd90 con 0x7f6ea0130260 2023-10-03T10:13:40.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.050+0000 7f6ea70f8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea01306d0 con 0x7f6ea012f8b0 2023-10-03T10:13:40.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.050+0000 7f6e9ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ea012f8b0 0x7f6ea012fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:40.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.050+0000 7f6ea5695700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ea0130260 0x7f6ea0136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:40.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.050+0000 7f6e9ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ea012f8b0 0x7f6ea012fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57020/0 (socket says 172.21.15.3:57020) 2023-10-03T10:13:40.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.050+0000 7f6e9ffff700 1 -- 172.21.15.3:0/2376462968 learned_addr learned my addr 172.21.15.3:0/2376462968 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:40.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.050+0000 7f6ea4e94700 1 --2- 172.21.15.3:0/2376462968 >> v2:172.21.15.3:3300/0 conn(0x7f6ea012cf60 0x7f6ea006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:40.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.050+0000 7f6ea5695700 1 -- 172.21.15.3:0/2376462968 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ea012f8b0 msgr2=0x7f6ea012fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:40.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.050+0000 7f6ea5695700 1 --2- 172.21.15.3:0/2376462968 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ea012f8b0 0x7f6ea012fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:40.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.050+0000 7f6ea5695700 1 -- 172.21.15.3:0/2376462968 >> v2:172.21.15.3:3300/0 conn(0x7f6ea012cf60 msgr2=0x7f6ea006f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:40.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.050+0000 7f6ea5695700 1 --2- 172.21.15.3:0/2376462968 >> v2:172.21.15.3:3300/0 conn(0x7f6ea012cf60 0x7f6ea006f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:40.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.051+0000 7f6ea5695700 1 -- 172.21.15.3:0/2376462968 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea0071780 con 0x7f6ea0130260 2023-10-03T10:13:40.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.051+0000 7f6e9ffff700 1 --2- 172.21.15.3:0/2376462968 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ea012f8b0 0x7f6ea012fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:40.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.051+0000 7f6ea4e94700 1 --2- 172.21.15.3:0/2376462968 >> v2:172.21.15.3:3300/0 conn(0x7f6ea012cf60 0x7f6ea006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:40.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.051+0000 7f6ea5695700 1 --2- 172.21.15.3:0/2376462968 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ea0130260 0x7f6ea0136c20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6e98009d90 tx=0x7f6e9800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a462983af868e7dd server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:40.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.051+0000 7f6e9f7fe700 1 -- 172.21.15.3:0/2376462968 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9800e070 con 0x7f6ea0130260 2023-10-03T10:13:40.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.051+0000 7f6e9f7fe700 1 -- 172.21.15.3:0/2376462968 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e98009e20 con 0x7f6ea0130260 2023-10-03T10:13:40.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.051+0000 7f6e9f7fe700 1 -- 172.21.15.3:0/2376462968 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9800bd60 con 0x7f6ea0130260 2023-10-03T10:13:40.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.052+0000 7f6ea70f8700 1 -- 172.21.15.3:0/2376462968 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ea0130260 msgr2=0x7f6ea0136c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:40.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.052+0000 7f6ea70f8700 1 --2- 172.21.15.3:0/2376462968 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ea0130260 0x7f6ea0136c20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6e98009d90 tx=0x7f6e9800b330 comp rx=0 tx=0).stop 2023-10-03T10:13:40.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.052+0000 7f6ea70f8700 1 -- 172.21.15.3:0/2376462968 shutdown_connections 2023-10-03T10:13:40.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.052+0000 7f6ea70f8700 1 --2- 172.21.15.3:0/2376462968 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ea0130260 0x7f6ea0136c20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:40.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.052+0000 7f6ea70f8700 1 --2- 172.21.15.3:0/2376462968 >> v2:172.21.15.3:3300/0 conn(0x7f6ea012cf60 0x7f6ea006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:40.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.052+0000 7f6ea70f8700 1 --2- 172.21.15.3:0/2376462968 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ea012f8b0 0x7f6ea012fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:40.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.052+0000 7f6ea70f8700 1 -- 172.21.15.3:0/2376462968 >> 172.21.15.3:0/2376462968 conn(0x7f6ea00a2aa0 msgr2=0x7f6ea00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:40.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.052+0000 7f6ea70f8700 1 -- 172.21.15.3:0/2376462968 shutdown_connections 2023-10-03T10:13:40.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.052+0000 7f6ea70f8700 1 -- 172.21.15.3:0/2376462968 wait complete. 2023-10-03T10:13:40.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.053+0000 7f6ea70f8700 1 Processor -- start 2023-10-03T10:13:40.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.053+0000 7f6ea70f8700 1 -- start start 2023-10-03T10:13:40.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.054+0000 7f6ea70f8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ea012cf60 0x7f6ea009b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:40.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.054+0000 7f6ea70f8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ea012f8b0 0x7f6ea009bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:40.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.054+0000 7f6ea70f8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ea0130260 0x7f6ea01c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:40.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.054+0000 7f6ea70f8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea009e870 con 0x7f6ea012cf60 2023-10-03T10:13:40.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.054+0000 7f6ea70f8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea0129fa0 con 0x7f6ea012f8b0 2023-10-03T10:13:40.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.054+0000 7f6ea70f8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea009d1f0 con 0x7f6ea0130260 2023-10-03T10:13:40.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.054+0000 7f6ea4e94700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ea012cf60 0x7f6ea009b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:40.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.054+0000 7f6e9ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ea012f8b0 0x7f6ea009bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:40.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.054+0000 7f6ea4e94700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ea012cf60 0x7f6ea009b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45356/0 (socket says 172.21.15.3:45356) 2023-10-03T10:13:40.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.054+0000 7f6ea4e94700 1 -- 172.21.15.3:0/758897051 learned_addr learned my addr 172.21.15.3:0/758897051 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:40.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.055+0000 7f6ea5695700 1 --2- 172.21.15.3:0/758897051 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ea0130260 0x7f6ea01c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:40.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.055+0000 7f6e9ffff700 1 -- 172.21.15.3:0/758897051 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ea0130260 msgr2=0x7f6ea01c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:40.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.055+0000 7f6e9ffff700 1 --2- 172.21.15.3:0/758897051 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ea0130260 0x7f6ea01c81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:40.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.055+0000 7f6e9ffff700 1 -- 172.21.15.3:0/758897051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ea012cf60 msgr2=0x7f6ea009b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:40.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.055+0000 7f6e9ffff700 1 --2- 172.21.15.3:0/758897051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ea012cf60 0x7f6ea009b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:40.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.055+0000 7f6e9ffff700 1 -- 172.21.15.3:0/758897051 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9400b740 con 0x7f6ea012f8b0 2023-10-03T10:13:40.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.055+0000 7f6ea4e94700 1 --2- 172.21.15.3:0/758897051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ea012cf60 0x7f6ea009b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:40.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.055+0000 7f6ea5695700 1 --2- 172.21.15.3:0/758897051 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ea0130260 0x7f6ea01c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:40.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.055+0000 7f6e9ffff700 1 --2- 172.21.15.3:0/758897051 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ea012f8b0 0x7f6ea009bdc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c00e9d0 tx=0x7f6e8c00ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:40.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.055+0000 7f6e9d7fa700 1 -- 172.21.15.3:0/758897051 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c004030 con 0x7f6ea012f8b0 2023-10-03T10:13:40.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.055+0000 7f6e9d7fa700 1 -- 172.21.15.3:0/758897051 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8c004190 con 0x7f6ea012f8b0 2023-10-03T10:13:40.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.055+0000 7f6ea70f8700 1 -- 172.21.15.3:0/758897051 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e8c009550 con 0x7f6ea012f8b0 2023-10-03T10:13:40.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.055+0000 7f6e9d7fa700 1 -- 172.21.15.3:0/758897051 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c004520 con 0x7f6ea012f8b0 2023-10-03T10:13:40.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.055+0000 7f6ea70f8700 1 -- 172.21.15.3:0/758897051 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e98009a70 con 0x7f6ea012f8b0 2023-10-03T10:13:40.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.056+0000 7f6e9d7fa700 1 -- 172.21.15.3:0/758897051 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6e8c00c850 con 0x7f6ea012f8b0 2023-10-03T10:13:40.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.056+0000 7f6e9d7fa700 1 -- 172.21.15.3:0/758897051 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6e8c03f580 con 0x7f6ea012f8b0 2023-10-03T10:13:40.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.057+0000 7f6ea70f8700 1 -- 172.21.15.3:0/758897051 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea009e9b0 con 0x7f6ea012f8b0 2023-10-03T10:13:40.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.061+0000 7f6e9d7fa700 1 -- 172.21.15.3:0/758897051 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6ea009e9b0 con 0x7f6ea012f8b0 2023-10-03T10:13:40.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.196+0000 7ff7c1d30700 1 -- 172.21.15.3:0/874904270 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff7bc074ea0 con 0x7ff7bc136f50 2023-10-03T10:13:40.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.196+0000 7ff7b8ff9700 1 -- 172.21.15.3:0/874904270 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff7b4015020 con 0x7ff7bc136f50 2023-10-03T10:13:40.197 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:40.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.200+0000 7ff7c1d30700 1 -- 172.21.15.3:0/874904270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7bc136f50 msgr2=0x7ff7bc070060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:40.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.200+0000 7ff7c1d30700 1 --2- 172.21.15.3:0/874904270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7bc136f50 0x7ff7bc070060 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009b90 tx=0x7ff7b4004750 comp rx=0 tx=0).stop 2023-10-03T10:13:40.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.200+0000 7ff7c1d30700 1 -- 172.21.15.3:0/874904270 shutdown_connections 2023-10-03T10:13:40.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.200+0000 7ff7c1d30700 1 --2- 172.21.15.3:0/874904270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7bc136f50 0x7ff7bc070060 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:40.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.200+0000 7ff7c1d30700 1 --2- 172.21.15.3:0/874904270 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7bc131810 0x7ff7bc06f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:40.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.200+0000 7ff7c1d30700 1 --2- 172.21.15.3:0/874904270 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff7bc134630 0x7ff7bc06fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:40.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.200+0000 7ff7c1d30700 1 -- 172.21.15.3:0/874904270 >> 172.21.15.3:0/874904270 conn(0x7ff7bc0a2aa0 msgr2=0x7ff7bc0a09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:40.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.200+0000 7ff7c1d30700 1 -- 172.21.15.3:0/874904270 shutdown_connections 2023-10-03T10:13:40.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.200+0000 7ff7c1d30700 1 -- 172.21.15.3:0/874904270 wait complete. 2023-10-03T10:13:40.221 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:40.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.226+0000 7fcc2774f700 1 -- 172.21.15.3:0/979520930 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcc20074ea0 con 0x7fcc200954c0 2023-10-03T10:13:40.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.226+0000 7fcc15ffb700 1 -- 172.21.15.3:0/979520930 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcc1c01fb90 con 0x7fcc200954c0 2023-10-03T10:13:40.226 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:40.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.230+0000 7fcc2774f700 1 -- 172.21.15.3:0/979520930 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcc200954c0 msgr2=0x7fcc2008c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:40.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.230+0000 7fcc2774f700 1 --2- 172.21.15.3:0/979520930 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcc200954c0 0x7fcc2008c020 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcc2009baa0 tx=0x7fcc1c004750 comp rx=0 tx=0).stop 2023-10-03T10:13:40.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.230+0000 7fcc2774f700 1 -- 172.21.15.3:0/979520930 shutdown_connections 2023-10-03T10:13:40.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.230+0000 7fcc2774f700 1 --2- 172.21.15.3:0/979520930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc20099ff0 0x7fcc2008caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:40.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.230+0000 7fcc2774f700 1 --2- 172.21.15.3:0/979520930 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcc200954c0 0x7fcc2008c020 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:40.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.230+0000 7fcc2774f700 1 --2- 172.21.15.3:0/979520930 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcc20099680 0x7fcc2008c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:40.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.230+0000 7fcc2774f700 1 -- 172.21.15.3:0/979520930 >> 172.21.15.3:0/979520930 conn(0x7fcc200a2360 msgr2=0x7fcc20098570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:40.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.230+0000 7fcc2774f700 1 -- 172.21.15.3:0/979520930 shutdown_connections 2023-10-03T10:13:40.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.230+0000 7fcc2774f700 1 -- 172.21.15.3:0/979520930 wait complete. 2023-10-03T10:13:40.244 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:40.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.281+0000 7f6ea70f8700 1 -- 172.21.15.3:0/758897051 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6ea009e9b0 con 0x7f6ea012f8b0 2023-10-03T10:13:40.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.282+0000 7f6e9d7fa700 1 -- 172.21.15.3:0/758897051 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6ea009e9b0 con 0x7f6ea012f8b0 2023-10-03T10:13:40.282 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:40.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.284+0000 7f6ea70f8700 1 -- 172.21.15.3:0/758897051 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ea012f8b0 msgr2=0x7f6ea009bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:40.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.284+0000 7f6ea70f8700 1 --2- 172.21.15.3:0/758897051 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ea012f8b0 0x7f6ea009bdc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c00e9d0 tx=0x7f6e8c00ece0 comp rx=0 tx=0).stop 2023-10-03T10:13:40.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.284+0000 7f6ea70f8700 1 -- 172.21.15.3:0/758897051 shutdown_connections 2023-10-03T10:13:40.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.284+0000 7f6ea70f8700 1 --2- 172.21.15.3:0/758897051 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ea0130260 0x7f6ea01c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:40.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.284+0000 7f6ea70f8700 1 --2- 172.21.15.3:0/758897051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ea012cf60 0x7f6ea009b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:40.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.284+0000 7f6ea70f8700 1 --2- 172.21.15.3:0/758897051 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ea012f8b0 0x7f6ea009bdc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:40.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.284+0000 7f6ea70f8700 1 -- 172.21.15.3:0/758897051 >> 172.21.15.3:0/758897051 conn(0x7f6ea00a2aa0 msgr2=0x7f6ea01b7990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:40.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.284+0000 7f6ea70f8700 1 -- 172.21.15.3:0/758897051 shutdown_connections 2023-10-03T10:13:40.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:40.284+0000 7f6ea70f8700 1 -- 172.21.15.3:0/758897051 wait complete. 2023-10-03T10:13:40.298 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:40.996 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:41.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.143+0000 7f45a77e5700 1 Processor -- start 2023-10-03T10:13:41.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.143+0000 7f45a77e5700 1 -- start start 2023-10-03T10:13:41.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.143+0000 7f45a77e5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f45a009d6f0 0x7f45a009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:41.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.143+0000 7f45a77e5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f45a009e410 0x7f45a0130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:41.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.143+0000 7f45a77e5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45a009f130 con 0x7f45a009e000 2023-10-03T10:13:41.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.143+0000 7f45a77e5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45a009f520 con 0x7f45a009e410 2023-10-03T10:13:41.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.144+0000 7f45a77e5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f45a009e930 con 0x7f45a009d6f0 2023-10-03T10:13:41.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.144+0000 7f45a5d82700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f45a009e410 0x7f45a0130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:41.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.144+0000 7f45a5581700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f45a009d6f0 0x7f45a009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:41.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.144+0000 7f45a5d82700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f45a009e410 0x7f45a0130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45366/0 (socket says 172.21.15.3:45366) 2023-10-03T10:13:41.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.144+0000 7f45a5581700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f45a009d6f0 0x7f45a009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57030/0 (socket says 172.21.15.3:57030) 2023-10-03T10:13:41.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.144+0000 7f45a5d82700 1 -- 172.21.15.3:0/3019468420 learned_addr learned my addr 172.21.15.3:0/3019468420 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:41.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.144+0000 7f45a5581700 1 -- 172.21.15.3:0/3019468420 >> v2:172.21.15.3:3300/0 conn(0x7f45a009e410 msgr2=0x7f45a0130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:41.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.144+0000 7f45a5581700 1 --2- 172.21.15.3:0/3019468420 >> v2:172.21.15.3:3300/0 conn(0x7f45a009e410 0x7f45a0130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.144+0000 7f45a5581700 1 -- 172.21.15.3:0/3019468420 >> v1:172.21.15.3:6789/0 conn(0x7f45a009e000 legacy=0x7f45a0130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:41.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.144+0000 7f45a5581700 1 -- 172.21.15.3:0/3019468420 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a009b2d0 con 0x7f45a009d6f0 2023-10-03T10:13:41.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.144+0000 7f45a5d82700 1 --2- 172.21.15.3:0/3019468420 >> v2:172.21.15.3:3300/0 conn(0x7f45a009e410 0x7f45a0130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:41.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.144+0000 7f45a5581700 1 --2- 172.21.15.3:0/3019468420 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f45a009d6f0 0x7f45a009dac0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f459400bae0 tx=0x7f459400bdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a6b946b900e431a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:41.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.144+0000 7f4593fff700 1 -- 172.21.15.3:0/3019468420 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459400dc70 con 0x7f45a009d6f0 2023-10-03T10:13:41.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.144+0000 7f4593fff700 1 -- 172.21.15.3:0/3019468420 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459400ddd0 con 0x7f45a009d6f0 2023-10-03T10:13:41.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.144+0000 7f4593fff700 1 -- 172.21.15.3:0/3019468420 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4594010690 con 0x7f45a009d6f0 2023-10-03T10:13:41.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.145+0000 7f45a77e5700 1 -- 172.21.15.3:0/3019468420 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f45a009d6f0 msgr2=0x7f45a009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:41.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.145+0000 7f45a77e5700 1 --2- 172.21.15.3:0/3019468420 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f45a009d6f0 0x7f45a009dac0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f459400bae0 tx=0x7f459400bdf0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.145+0000 7f45a77e5700 1 -- 172.21.15.3:0/3019468420 shutdown_connections 2023-10-03T10:13:41.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.145+0000 7f45a77e5700 1 --2- 172.21.15.3:0/3019468420 >> v2:172.21.15.3:3300/0 conn(0x7f45a009e410 0x7f45a0130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.145+0000 7f45a77e5700 1 --2- 172.21.15.3:0/3019468420 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f45a009d6f0 0x7f45a009dac0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.145+0000 7f45a77e5700 1 -- 172.21.15.3:0/3019468420 >> 172.21.15.3:0/3019468420 conn(0x7f45a0066f50 msgr2=0x7f45a009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:41.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.145+0000 7f45a77e5700 1 -- 172.21.15.3:0/3019468420 shutdown_connections 2023-10-03T10:13:41.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.145+0000 7f45a77e5700 1 -- 172.21.15.3:0/3019468420 wait complete. 2023-10-03T10:13:41.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.146+0000 7f45a77e5700 1 Processor -- start 2023-10-03T10:13:41.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.146+0000 7f45a77e5700 1 -- start start 2023-10-03T10:13:41.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.146+0000 7f45a77e5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f45a009d6f0 0x7f45a011e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:41.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.146+0000 7f45a77e5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45a009e000 0x7f45a011e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:41.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.146+0000 7f45a77e5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f45a009e410 0x7f45a011eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:41.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.146+0000 7f45a77e5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a0121b40 con 0x7f45a009e000 2023-10-03T10:13:41.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.147+0000 7f45a77e5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a009c770 con 0x7f45a009e410 2023-10-03T10:13:41.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.147+0000 7f45a77e5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f45a00a1e90 con 0x7f45a009d6f0 2023-10-03T10:13:41.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.147+0000 7f45a5581700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f45a009d6f0 0x7f45a011e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:41.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.147+0000 7f45a4d80700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45a009e000 0x7f45a011e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:41.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.147+0000 7f45a5d82700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f45a009e410 0x7f45a011eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:41.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.147+0000 7f45a5581700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f45a009d6f0 0x7f45a011e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57032/0 (socket says 172.21.15.3:57032) 2023-10-03T10:13:41.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.147+0000 7f45a5581700 1 -- 172.21.15.3:0/1805211843 learned_addr learned my addr 172.21.15.3:0/1805211843 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:41.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.147+0000 7f45a4d80700 1 -- 172.21.15.3:0/1805211843 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f45a009d6f0 msgr2=0x7f45a011e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:41.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.147+0000 7f45a4d80700 1 --2- 172.21.15.3:0/1805211843 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f45a009d6f0 0x7f45a011e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.147+0000 7f45a4d80700 1 -- 172.21.15.3:0/1805211843 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f45a009e410 msgr2=0x7f45a011eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:41.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.147+0000 7f45a4d80700 1 --2- 172.21.15.3:0/1805211843 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f45a009e410 0x7f45a011eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.147+0000 7f45a4d80700 1 -- 172.21.15.3:0/1805211843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4598009b30 con 0x7f45a009e000 2023-10-03T10:13:41.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.147+0000 7f45a5d82700 1 --2- 172.21.15.3:0/1805211843 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f45a009e410 0x7f45a011eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:41.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.147+0000 7f45a4d80700 1 --2- 172.21.15.3:0/1805211843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45a009e000 0x7f45a011e5f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f459c000f60 tx=0x7f459c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:41.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.147+0000 7f4591ffb700 1 -- 172.21.15.3:0/1805211843 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c013070 con 0x7f45a009e000 2023-10-03T10:13:41.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.147+0000 7f4591ffb700 1 -- 172.21.15.3:0/1805211843 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459c00e2e0 con 0x7f45a009e000 2023-10-03T10:13:41.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.147+0000 7f45a77e5700 1 -- 172.21.15.3:0/1805211843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f459400b740 con 0x7f45a009e000 2023-10-03T10:13:41.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.147+0000 7f4591ffb700 1 -- 172.21.15.3:0/1805211843 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c0052f0 con 0x7f45a009e000 2023-10-03T10:13:41.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.148+0000 7f45a77e5700 1 -- 172.21.15.3:0/1805211843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a01215a0 con 0x7f45a009e000 2023-10-03T10:13:41.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.148+0000 7f4591ffb700 1 -- 172.21.15.3:0/1805211843 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f459c003680 con 0x7f45a009e000 2023-10-03T10:13:41.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.148+0000 7f4591ffb700 1 -- 172.21.15.3:0/1805211843 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f459c042c10 con 0x7f45a009e000 2023-10-03T10:13:41.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.149+0000 7f45a77e5700 1 -- 172.21.15.3:0/1805211843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a0074ea0 con 0x7f45a009e000 2023-10-03T10:13:41.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.154+0000 7f4591ffb700 1 -- 172.21.15.3:0/1805211843 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f459c003930 con 0x7f45a009e000 2023-10-03T10:13:41.221 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:41.245 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:41.299 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:41.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.363+0000 7f65ac9b2700 1 Processor -- start 2023-10-03T10:13:41.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.363+0000 7f65ac9b2700 1 -- start start 2023-10-03T10:13:41.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.364+0000 7f65ac9b2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65a81338a0 0x7f65a8133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:41.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.364+0000 7f65ac9b2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65a809de50 0x7f65a809e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:41.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.364+0000 7f65ac9b2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f65a809e760 0x7f65a813a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:41.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.364+0000 7f65ac9b2700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a806fef0 con 0x7f65a809e760 2023-10-03T10:13:41.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.364+0000 7f65ac9b2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a806ef30 con 0x7f65a809de50 2023-10-03T10:13:41.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.364+0000 7f65ac9b2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f65a81334b0 con 0x7f65a81338a0 2023-10-03T10:13:41.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.364+0000 7f65a5d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65a809de50 0x7f65a809e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:41.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.364+0000 7f65a5d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65a809de50 0x7f65a809e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44756/0 (socket says 172.21.15.3:44756) 2023-10-03T10:13:41.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.364+0000 7f65a5d9b700 1 -- 172.21.15.3:0/2765039463 learned_addr learned my addr 172.21.15.3:0/2765039463 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:41.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.364+0000 7f65a6d9d700 1 --2- 172.21.15.3:0/2765039463 >> v2:172.21.15.3:3300/0 conn(0x7f65a809e760 0x7f65a813a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:41.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.364+0000 7f65a659c700 1 --2- 172.21.15.3:0/2765039463 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65a81338a0 0x7f65a8133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:41.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.364+0000 7f65a5d9b700 1 -- 172.21.15.3:0/2765039463 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65a81338a0 msgr2=0x7f65a8133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:41.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.364+0000 7f65a5d9b700 1 --2- 172.21.15.3:0/2765039463 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65a81338a0 0x7f65a8133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.364+0000 7f65a5d9b700 1 -- 172.21.15.3:0/2765039463 >> v2:172.21.15.3:3300/0 conn(0x7f65a809e760 msgr2=0x7f65a813a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:41.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.364+0000 7f65a5d9b700 1 --2- 172.21.15.3:0/2765039463 >> v2:172.21.15.3:3300/0 conn(0x7f65a809e760 0x7f65a813a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.365+0000 7f65a5d9b700 1 -- 172.21.15.3:0/2765039463 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a806fc30 con 0x7f65a809de50 2023-10-03T10:13:41.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.365+0000 7f65a5d9b700 1 --2- 172.21.15.3:0/2765039463 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65a809de50 0x7f65a809e220 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f659c009920 tx=0x7f659c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d812cbd813e43f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:41.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.365+0000 7f65a559a700 1 -- 172.21.15.3:0/2765039463 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c011830 con 0x7f65a809de50 2023-10-03T10:13:41.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.365+0000 7f65a559a700 1 -- 172.21.15.3:0/2765039463 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c011990 con 0x7f65a809de50 2023-10-03T10:13:41.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.365+0000 7f65a559a700 1 -- 172.21.15.3:0/2765039463 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c011d20 con 0x7f65a809de50 2023-10-03T10:13:41.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.366+0000 7f65ac9b2700 1 -- 172.21.15.3:0/2765039463 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65a809de50 msgr2=0x7f65a809e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:41.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.366+0000 7f65ac9b2700 1 --2- 172.21.15.3:0/2765039463 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65a809de50 0x7f65a809e220 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f659c009920 tx=0x7f659c009c30 comp rx=0 tx=0).stop 2023-10-03T10:13:41.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.366+0000 7f65ac9b2700 1 -- 172.21.15.3:0/2765039463 shutdown_connections 2023-10-03T10:13:41.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.366+0000 7f65ac9b2700 1 --2- 172.21.15.3:0/2765039463 >> v2:172.21.15.3:3300/0 conn(0x7f65a809e760 0x7f65a813a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.366+0000 7f65ac9b2700 1 --2- 172.21.15.3:0/2765039463 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65a81338a0 0x7f65a8133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.366+0000 7f65ac9b2700 1 --2- 172.21.15.3:0/2765039463 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65a809de50 0x7f65a809e220 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.366+0000 7f65ac9b2700 1 -- 172.21.15.3:0/2765039463 >> 172.21.15.3:0/2765039463 conn(0x7f65a80a2aa0 msgr2=0x7f65a80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:41.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.366+0000 7f65ac9b2700 1 -- 172.21.15.3:0/2765039463 shutdown_connections 2023-10-03T10:13:41.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.366+0000 7f65ac9b2700 1 -- 172.21.15.3:0/2765039463 wait complete. 2023-10-03T10:13:41.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.367+0000 7f65ac9b2700 1 Processor -- start 2023-10-03T10:13:41.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.367+0000 7f65ac9b2700 1 -- start start 2023-10-03T10:13:41.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.368+0000 7f65ac9b2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65a809de50 0x7f65a81c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:41.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.368+0000 7f65ac9b2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65a809e760 0x7f65a81c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:41.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.368+0000 7f65ac9b2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65a81338a0 0x7f65a81c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:41.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.368+0000 7f65ac9b2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a8120a40 con 0x7f65a81338a0 2023-10-03T10:13:41.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.368+0000 7f65ac9b2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a8121740 con 0x7f65a809de50 2023-10-03T10:13:41.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.368+0000 7f65ac9b2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f65a8122950 con 0x7f65a809e760 2023-10-03T10:13:41.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.368+0000 7f65a659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65a809de50 0x7f65a81c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:41.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.368+0000 7f65a659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65a809de50 0x7f65a81c8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44770/0 (socket says 172.21.15.3:44770) 2023-10-03T10:13:41.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.368+0000 7f65a659c700 1 -- 172.21.15.3:0/2121749977 learned_addr learned my addr 172.21.15.3:0/2121749977 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:41.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.368+0000 7f65a6d9d700 1 --2- 172.21.15.3:0/2121749977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65a81338a0 0x7f65a81c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:41.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.368+0000 7f65a5d9b700 1 --2- 172.21.15.3:0/2121749977 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65a809e760 0x7f65a81c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:41.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.368+0000 7f65a659c700 1 -- 172.21.15.3:0/2121749977 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65a809e760 msgr2=0x7f65a81c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:41.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.368+0000 7f65a659c700 1 --2- 172.21.15.3:0/2121749977 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65a809e760 0x7f65a81c8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.368+0000 7f65a659c700 1 -- 172.21.15.3:0/2121749977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65a81338a0 msgr2=0x7f65a81c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:41.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.368+0000 7f65a659c700 1 --2- 172.21.15.3:0/2121749977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65a81338a0 0x7f65a81c8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.368+0000 7f65a659c700 1 -- 172.21.15.3:0/2121749977 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659c009550 con 0x7f65a809de50 2023-10-03T10:13:41.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.368+0000 7f65a6d9d700 1 --2- 172.21.15.3:0/2121749977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65a81338a0 0x7f65a81c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:41.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.369+0000 7f65a659c700 1 --2- 172.21.15.3:0/2121749977 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65a809de50 0x7f65a81c8570 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f659800e9b0 tx=0x7f659800ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:41.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.369+0000 7f6596ffd700 1 -- 172.21.15.3:0/2121749977 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6598009710 con 0x7f65a809de50 2023-10-03T10:13:41.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.369+0000 7f6596ffd700 1 -- 172.21.15.3:0/2121749977 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659800c850 con 0x7f65a809de50 2023-10-03T10:13:41.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.369+0000 7f6596ffd700 1 -- 172.21.15.3:0/2121749977 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659800cbc0 con 0x7f65a809de50 2023-10-03T10:13:41.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.369+0000 7f65ac9b2700 1 -- 172.21.15.3:0/2121749977 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a8120ff0 con 0x7f65a809de50 2023-10-03T10:13:41.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.369+0000 7f65ac9b2700 1 -- 172.21.15.3:0/2121749977 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a8122670 con 0x7f65a809de50 2023-10-03T10:13:41.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.370+0000 7f6596ffd700 1 -- 172.21.15.3:0/2121749977 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f659800cd20 con 0x7f65a809de50 2023-10-03T10:13:41.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.370+0000 7f65ac9b2700 1 -- 172.21.15.3:0/2121749977 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a8070790 con 0x7f65a809de50 2023-10-03T10:13:41.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.370+0000 7f6596ffd700 1 -- 172.21.15.3:0/2121749977 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f659803faf0 con 0x7f65a809de50 2023-10-03T10:13:41.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.376+0000 7f45a77e5700 1 -- 172.21.15.3:0/1805211843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f45a006c360 con 0x7f45a009e000 2023-10-03T10:13:41.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.376+0000 7f4591ffb700 1 -- 172.21.15.3:0/1805211843 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f459c0173e0 con 0x7f45a009e000 2023-10-03T10:13:41.376 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:41.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.378+0000 7f45a77e5700 1 -- 172.21.15.3:0/1805211843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45a009e000 msgr2=0x7f45a011e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:41.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.378+0000 7f45a77e5700 1 --2- 172.21.15.3:0/1805211843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45a009e000 0x7f45a011e5f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f459c000f60 tx=0x7f459c001a40 comp rx=0 tx=0).stop 2023-10-03T10:13:41.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.378+0000 7f45a77e5700 1 -- 172.21.15.3:0/1805211843 shutdown_connections 2023-10-03T10:13:41.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.378+0000 7f45a77e5700 1 --2- 172.21.15.3:0/1805211843 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f45a009e410 0x7f45a011eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.378+0000 7f45a77e5700 1 --2- 172.21.15.3:0/1805211843 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f45a009d6f0 0x7f45a011e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.378+0000 7f45a77e5700 1 --2- 172.21.15.3:0/1805211843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45a009e000 0x7f45a011e5f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.378+0000 7f45a77e5700 1 -- 172.21.15.3:0/1805211843 >> 172.21.15.3:0/1805211843 conn(0x7f45a0066f50 msgr2=0x7f45a00a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:41.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.379+0000 7f6596ffd700 1 -- 172.21.15.3:0/2121749977 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f659801adc0 con 0x7f65a809de50 2023-10-03T10:13:41.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.380+0000 7f45a77e5700 1 -- 172.21.15.3:0/1805211843 shutdown_connections 2023-10-03T10:13:41.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.380+0000 7f45a77e5700 1 -- 172.21.15.3:0/1805211843 wait complete. 2023-10-03T10:13:41.401 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:41.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.416+0000 7fed4442b700 1 Processor -- start 2023-10-03T10:13:41.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.416+0000 7fed4442b700 1 -- start start 2023-10-03T10:13:41.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.417+0000 7fed4442b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fed3c133420 0x7fed3c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:41.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.417+0000 7fed4442b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fed3c133e30 0x7fed3c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:41.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.417+0000 7fed4442b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fed3c134260 0x7fed3c1304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:41.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.417+0000 7fed4442b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed3c06b990 con 0x7fed3c133e30 2023-10-03T10:13:41.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.417+0000 7fed4442b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c0a0590 con 0x7fed3c133420 2023-10-03T10:13:41.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.417+0000 7fed4442b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c09fb80 con 0x7fed3c134260 2023-10-03T10:13:41.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.417+0000 7fed419c6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fed3c133e30 0x7fed3c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:41.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.417+0000 7fed419c6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fed3c133e30 0x7fed3c12ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45392/0 (socket says 172.21.15.3:45392) 2023-10-03T10:13:41.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.417+0000 7fed419c6700 1 -- 172.21.15.3:0/734499770 learned_addr learned my addr 172.21.15.3:0/734499770 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:41.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.417+0000 7fed421c7700 1 --2- 172.21.15.3:0/734499770 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fed3c133420 0x7fed3c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:41.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.417+0000 7fed429c8700 1 --2- 172.21.15.3:0/734499770 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fed3c134260 0x7fed3c1304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:41.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.417+0000 7fed421c7700 1 -- 172.21.15.3:0/734499770 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fed3c134260 msgr2=0x7fed3c1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:41.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.417+0000 7fed421c7700 1 --2- 172.21.15.3:0/734499770 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fed3c134260 0x7fed3c1304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.417+0000 7fed421c7700 1 -- 172.21.15.3:0/734499770 >> v2:172.21.15.3:3300/0 conn(0x7fed3c133e30 msgr2=0x7fed3c12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:41.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.417+0000 7fed421c7700 1 --2- 172.21.15.3:0/734499770 >> v2:172.21.15.3:3300/0 conn(0x7fed3c133e30 0x7fed3c12ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.417+0000 7fed421c7700 1 -- 172.21.15.3:0/734499770 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3c0a0030 con 0x7fed3c133420 2023-10-03T10:13:41.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.417+0000 7fed419c6700 1 --2- 172.21.15.3:0/734499770 >> v2:172.21.15.3:3300/0 conn(0x7fed3c133e30 0x7fed3c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:41.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.418+0000 7fed421c7700 1 --2- 172.21.15.3:0/734499770 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fed3c133420 0x7fed3c1337f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fed38009840 tx=0x7fed38009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c57b4883abc19b91 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:41.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.418+0000 7fed411c5700 1 -- 172.21.15.3:0/734499770 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed38011830 con 0x7fed3c133420 2023-10-03T10:13:41.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.418+0000 7fed411c5700 1 -- 172.21.15.3:0/734499770 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed38011990 con 0x7fed3c133420 2023-10-03T10:13:41.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.418+0000 7fed411c5700 1 -- 172.21.15.3:0/734499770 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed38011d20 con 0x7fed3c133420 2023-10-03T10:13:41.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.419+0000 7fed4442b700 1 -- 172.21.15.3:0/734499770 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fed3c133420 msgr2=0x7fed3c1337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:41.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.419+0000 7fed4442b700 1 --2- 172.21.15.3:0/734499770 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fed3c133420 0x7fed3c1337f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fed38009840 tx=0x7fed38009b50 comp rx=0 tx=0).stop 2023-10-03T10:13:41.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.419+0000 7fed4442b700 1 -- 172.21.15.3:0/734499770 shutdown_connections 2023-10-03T10:13:41.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.419+0000 7fed4442b700 1 --2- 172.21.15.3:0/734499770 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fed3c134260 0x7fed3c1304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.419+0000 7fed4442b700 1 --2- 172.21.15.3:0/734499770 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fed3c133420 0x7fed3c1337f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.419+0000 7fed4442b700 1 --2- 172.21.15.3:0/734499770 >> v2:172.21.15.3:3300/0 conn(0x7fed3c133e30 0x7fed3c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.419+0000 7fed4442b700 1 -- 172.21.15.3:0/734499770 >> 172.21.15.3:0/734499770 conn(0x7fed3c0a4210 msgr2=0x7fed3c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:41.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.419+0000 7fed4442b700 1 -- 172.21.15.3:0/734499770 shutdown_connections 2023-10-03T10:13:41.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.419+0000 7fed4442b700 1 -- 172.21.15.3:0/734499770 wait complete. 2023-10-03T10:13:41.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.420+0000 7fed4442b700 1 Processor -- start 2023-10-03T10:13:41.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.420+0000 7fed4442b700 1 -- start start 2023-10-03T10:13:41.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.420+0000 7fed4442b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fed3c133420 0x7fed3c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:41.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.420+0000 7fed4442b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fed3c133e30 0x7fed3c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:41.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.420+0000 7fed4442b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed3c134260 0x7fed3c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:41.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.420+0000 7fed4442b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c11ffb0 con 0x7fed3c134260 2023-10-03T10:13:41.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.420+0000 7fed4442b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c120190 con 0x7fed3c133e30 2023-10-03T10:13:41.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.420+0000 7fed4442b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c12da40 con 0x7fed3c133420 2023-10-03T10:13:41.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.421+0000 7fed421c7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fed3c133420 0x7fed3c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:41.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.421+0000 7fed419c6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fed3c133e30 0x7fed3c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:41.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.421+0000 7fed421c7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fed3c133420 0x7fed3c09c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57070/0 (socket says 172.21.15.3:57070) 2023-10-03T10:13:41.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.421+0000 7fed419c6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fed3c133e30 0x7fed3c09c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44792/0 (socket says 172.21.15.3:44792) 2023-10-03T10:13:41.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.421+0000 7fed421c7700 1 -- 172.21.15.3:0/3403764141 learned_addr learned my addr 172.21.15.3:0/3403764141 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:41.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.421+0000 7fed429c8700 1 --2- 172.21.15.3:0/3403764141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed3c134260 0x7fed3c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:41.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.421+0000 7fed419c6700 1 -- 172.21.15.3:0/3403764141 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fed3c133420 msgr2=0x7fed3c09c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:41.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.421+0000 7fed419c6700 1 --2- 172.21.15.3:0/3403764141 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fed3c133420 0x7fed3c09c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.421+0000 7fed419c6700 1 -- 172.21.15.3:0/3403764141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed3c134260 msgr2=0x7fed3c09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:41.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.421+0000 7fed419c6700 1 --2- 172.21.15.3:0/3403764141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed3c134260 0x7fed3c09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.421+0000 7fed419c6700 1 -- 172.21.15.3:0/3403764141 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2c009550 con 0x7fed3c133e30 2023-10-03T10:13:41.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.421+0000 7fed429c8700 1 --2- 172.21.15.3:0/3403764141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed3c134260 0x7fed3c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:41.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.421+0000 7fed421c7700 1 --2- 172.21.15.3:0/3403764141 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fed3c133420 0x7fed3c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:41.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.421+0000 7fed419c6700 1 --2- 172.21.15.3:0/3403764141 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fed3c133e30 0x7fed3c09c9c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00e9a0 tx=0x7fed2c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:41.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.421+0000 7fed32ffd700 1 -- 172.21.15.3:0/3403764141 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2c004240 con 0x7fed3c133e30 2023-10-03T10:13:41.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.422+0000 7fed32ffd700 1 -- 172.21.15.3:0/3403764141 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed2c0043a0 con 0x7fed3c133e30 2023-10-03T10:13:41.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.422+0000 7fed32ffd700 1 -- 172.21.15.3:0/3403764141 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2c0036a0 con 0x7fed3c133e30 2023-10-03T10:13:41.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.422+0000 7fed4442b700 1 -- 172.21.15.3:0/3403764141 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed380094a0 con 0x7fed3c133e30 2023-10-03T10:13:41.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.422+0000 7fed4442b700 1 -- 172.21.15.3:0/3403764141 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3c09de80 con 0x7fed3c133e30 2023-10-03T10:13:41.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.423+0000 7fed32ffd700 1 -- 172.21.15.3:0/3403764141 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fed2c003800 con 0x7fed3c133e30 2023-10-03T10:13:41.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.423+0000 7fed32ffd700 1 -- 172.21.15.3:0/3403764141 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fed2c03f7e0 con 0x7fed3c133e30 2023-10-03T10:13:41.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.423+0000 7fed4442b700 1 -- 172.21.15.3:0/3403764141 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed3c074ea0 con 0x7fed3c133e30 2023-10-03T10:13:41.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.428+0000 7fed32ffd700 1 -- 172.21.15.3:0/3403764141 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fed2c013470 con 0x7fed3c133e30 2023-10-03T10:13:41.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.461+0000 7fab86399700 1 Processor -- start 2023-10-03T10:13:41.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.461+0000 7fab86399700 1 -- start start 2023-10-03T10:13:41.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.461+0000 7fab86399700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab80133a80 0x7fab80133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:41.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.461+0000 7fab86399700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fab80130a40 0x7fab8013b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:41.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.461+0000 7fab86399700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab8006bc00 con 0x7fab8012ff20 2023-10-03T10:13:41.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.461+0000 7fab86399700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab8006c570 con 0x7fab80130a40 2023-10-03T10:13:41.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.461+0000 7fab86399700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab800a0040 con 0x7fab80133a80 2023-10-03T10:13:41.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.462+0000 7fab7ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab80133a80 0x7fab80133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:41.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.462+0000 7fab7ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab80133a80 0x7fab80133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44798/0 (socket says 172.21.15.3:44798) 2023-10-03T10:13:41.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.462+0000 7fab7ffff700 1 -- 172.21.15.3:0/22643485 learned_addr learned my addr 172.21.15.3:0/22643485 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:41.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.462+0000 7fab84936700 1 --2- 172.21.15.3:0/22643485 >> v2:172.21.15.3:3300/0 conn(0x7fab80130a40 0x7fab8013b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:41.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.462+0000 7fab7ffff700 1 -- 172.21.15.3:0/22643485 >> v2:172.21.15.3:3300/0 conn(0x7fab80130a40 msgr2=0x7fab8013b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:41.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.462+0000 7fab7ffff700 1 --2- 172.21.15.3:0/22643485 >> v2:172.21.15.3:3300/0 conn(0x7fab80130a40 0x7fab8013b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.462+0000 7fab7ffff700 1 -- 172.21.15.3:0/22643485 >> v1:172.21.15.3:6789/0 conn(0x7fab8012ff20 legacy=0x7fab80130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:41.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.462+0000 7fab7ffff700 1 -- 172.21.15.3:0/22643485 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab80133860 con 0x7fab80133a80 2023-10-03T10:13:41.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.462+0000 7fab7ffff700 1 --2- 172.21.15.3:0/22643485 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab80133a80 0x7fab80133e50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fab700098f0 tx=0x7fab70009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f9935ec08e30486 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:41.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.462+0000 7fab7effd700 1 -- 172.21.15.3:0/22643485 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab70011830 con 0x7fab80133a80 2023-10-03T10:13:41.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.462+0000 7fab7effd700 1 -- 172.21.15.3:0/22643485 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab70011990 con 0x7fab80133a80 2023-10-03T10:13:41.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.463+0000 7fab7effd700 1 -- 172.21.15.3:0/22643485 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab70011d20 con 0x7fab80133a80 2023-10-03T10:13:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.463+0000 7fab86399700 1 -- 172.21.15.3:0/22643485 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab80133a80 msgr2=0x7fab80133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.463+0000 7fab86399700 1 --2- 172.21.15.3:0/22643485 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab80133a80 0x7fab80133e50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fab700098f0 tx=0x7fab70009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.463+0000 7fab86399700 1 -- 172.21.15.3:0/22643485 shutdown_connections 2023-10-03T10:13:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.463+0000 7fab86399700 1 --2- 172.21.15.3:0/22643485 >> v2:172.21.15.3:3300/0 conn(0x7fab80130a40 0x7fab8013b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.463+0000 7fab86399700 1 --2- 172.21.15.3:0/22643485 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab80133a80 0x7fab80133e50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.463+0000 7fab86399700 1 -- 172.21.15.3:0/22643485 >> 172.21.15.3:0/22643485 conn(0x7fab800a4210 msgr2=0x7fab800a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.464+0000 7fab86399700 1 -- 172.21.15.3:0/22643485 shutdown_connections 2023-10-03T10:13:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.464+0000 7fab86399700 1 -- 172.21.15.3:0/22643485 wait complete. 2023-10-03T10:13:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.464+0000 7fab86399700 1 Processor -- start 2023-10-03T10:13:41.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.465+0000 7fab86399700 1 -- start start 2023-10-03T10:13:41.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.465+0000 7fab86399700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab8012ff20 0x7fab8009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:41.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.465+0000 7fab86399700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab80130a40 0x7fab8009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:41.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.465+0000 7fab86399700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab80133a80 0x7fab8009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.465+0000 7fab86399700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8011fd10 con 0x7fab80133a80 2023-10-03T10:13:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.465+0000 7fab86399700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab8012d770 con 0x7fab80130a40 2023-10-03T10:13:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.465+0000 7fab86399700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fab80120390 con 0x7fab8012ff20 2023-10-03T10:13:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.465+0000 7fab7f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab80130a40 0x7fab8009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:41.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.465+0000 7fab7f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab80130a40 0x7fab8009c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44800/0 (socket says 172.21.15.3:44800) 2023-10-03T10:13:41.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.465+0000 7fab7f7fe700 1 -- 172.21.15.3:0/2149667199 learned_addr learned my addr 172.21.15.3:0/2149667199 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:41.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.465+0000 7fab7ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab8012ff20 0x7fab8009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:41.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.466+0000 7fab84936700 1 --2- 172.21.15.3:0/2149667199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab80133a80 0x7fab8009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:41.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.466+0000 7fab7f7fe700 1 -- 172.21.15.3:0/2149667199 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab8012ff20 msgr2=0x7fab8009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:41.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.466+0000 7fab7f7fe700 1 --2- 172.21.15.3:0/2149667199 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab8012ff20 0x7fab8009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.466+0000 7fab7f7fe700 1 -- 172.21.15.3:0/2149667199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab80133a80 msgr2=0x7fab8009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:41.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.466+0000 7fab7f7fe700 1 --2- 172.21.15.3:0/2149667199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab80133a80 0x7fab8009cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.466+0000 7fab7f7fe700 1 -- 172.21.15.3:0/2149667199 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab70009550 con 0x7fab80130a40 2023-10-03T10:13:41.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.466+0000 7fab7f7fe700 1 --2- 172.21.15.3:0/2149667199 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab80130a40 0x7fab8009c9c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fab7400c4b0 tx=0x7fab7400c7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:41.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.466+0000 7fab7cff9700 1 -- 172.21.15.3:0/2149667199 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab740052d0 con 0x7fab80130a40 2023-10-03T10:13:41.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.466+0000 7fab7cff9700 1 -- 172.21.15.3:0/2149667199 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab74005430 con 0x7fab80130a40 2023-10-03T10:13:41.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.466+0000 7fab7cff9700 1 -- 172.21.15.3:0/2149667199 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab740038c0 con 0x7fab80130a40 2023-10-03T10:13:41.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.466+0000 7fab86399700 1 -- 172.21.15.3:0/2149667199 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab800a20e0 con 0x7fab80130a40 2023-10-03T10:13:41.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.466+0000 7fab86399700 1 -- 172.21.15.3:0/2149667199 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab8011f990 con 0x7fab80130a40 2023-10-03T10:13:41.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.468+0000 7fab86399700 1 -- 172.21.15.3:0/2149667199 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab8009d870 con 0x7fab80130a40 2023-10-03T10:13:41.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.468+0000 7fab7cff9700 1 -- 172.21.15.3:0/2149667199 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fab74003a20 con 0x7fab80130a40 2023-10-03T10:13:41.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.469+0000 7fab7cff9700 1 -- 172.21.15.3:0/2149667199 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fab7403eed0 con 0x7fab80130a40 2023-10-03T10:13:41.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.472+0000 7fab7cff9700 1 -- 172.21.15.3:0/2149667199 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fab74003cd0 con 0x7fab80130a40 2023-10-03T10:13:41.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.591+0000 7f65ac9b2700 1 -- 172.21.15.3:0/2121749977 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f65a80a1e80 con 0x7f65a809de50 2023-10-03T10:13:41.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.592+0000 7f6596ffd700 1 -- 172.21.15.3:0/2121749977 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f659803fe60 con 0x7f65a809de50 2023-10-03T10:13:41.592 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:41.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.594+0000 7f65ac9b2700 1 -- 172.21.15.3:0/2121749977 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65a809de50 msgr2=0x7f65a81c8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:41.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.594+0000 7f65ac9b2700 1 --2- 172.21.15.3:0/2121749977 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65a809de50 0x7f65a81c8570 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f659800e9b0 tx=0x7f659800ecc0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.594+0000 7f65ac9b2700 1 -- 172.21.15.3:0/2121749977 shutdown_connections 2023-10-03T10:13:41.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.594+0000 7f65ac9b2700 1 --2- 172.21.15.3:0/2121749977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65a81338a0 0x7f65a81c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.594+0000 7f65ac9b2700 1 --2- 172.21.15.3:0/2121749977 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65a809de50 0x7f65a81c8570 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.594+0000 7f65ac9b2700 1 --2- 172.21.15.3:0/2121749977 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65a809e760 0x7f65a81c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.594+0000 7f65ac9b2700 1 -- 172.21.15.3:0/2121749977 >> 172.21.15.3:0/2121749977 conn(0x7f65a80a2aa0 msgr2=0x7f65a809ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:41.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.594+0000 7f65ac9b2700 1 -- 172.21.15.3:0/2121749977 shutdown_connections 2023-10-03T10:13:41.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.594+0000 7f65ac9b2700 1 -- 172.21.15.3:0/2121749977 wait complete. 2023-10-03T10:13:41.608 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:41.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.638+0000 7fed4442b700 1 -- 172.21.15.3:0/3403764141 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fed3c11fd10 con 0x7fed3c133e30 2023-10-03T10:13:41.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.638+0000 7fed32ffd700 1 -- 172.21.15.3:0/3403764141 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fed2c02b090 con 0x7fed3c133e30 2023-10-03T10:13:41.639 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:41.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.640+0000 7fed4442b700 1 -- 172.21.15.3:0/3403764141 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fed3c133e30 msgr2=0x7fed3c09c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:41.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.640+0000 7fed4442b700 1 --2- 172.21.15.3:0/3403764141 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fed3c133e30 0x7fed3c09c9c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00e9a0 tx=0x7fed2c00ecb0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.641+0000 7fed4442b700 1 -- 172.21.15.3:0/3403764141 shutdown_connections 2023-10-03T10:13:41.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.641+0000 7fed4442b700 1 --2- 172.21.15.3:0/3403764141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed3c134260 0x7fed3c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.641+0000 7fed4442b700 1 --2- 172.21.15.3:0/3403764141 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fed3c133420 0x7fed3c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.641+0000 7fed4442b700 1 --2- 172.21.15.3:0/3403764141 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fed3c133e30 0x7fed3c09c9c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.641+0000 7fed4442b700 1 -- 172.21.15.3:0/3403764141 >> 172.21.15.3:0/3403764141 conn(0x7fed3c0a4210 msgr2=0x7fed3c0a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:41.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.641+0000 7fed4442b700 1 -- 172.21.15.3:0/3403764141 shutdown_connections 2023-10-03T10:13:41.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.641+0000 7fed4442b700 1 -- 172.21.15.3:0/3403764141 wait complete. 2023-10-03T10:13:41.654 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:41.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.681+0000 7fab86399700 1 -- 172.21.15.3:0/2149667199 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fab801201f0 con 0x7fab80130a40 2023-10-03T10:13:41.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.682+0000 7fab7cff9700 1 -- 172.21.15.3:0/2149667199 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fab801201f0 con 0x7fab80130a40 2023-10-03T10:13:41.682 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:41.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.684+0000 7fab86399700 1 -- 172.21.15.3:0/2149667199 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab80130a40 msgr2=0x7fab8009c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:41.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.684+0000 7fab86399700 1 --2- 172.21.15.3:0/2149667199 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab80130a40 0x7fab8009c9c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fab7400c4b0 tx=0x7fab7400c7c0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.684+0000 7fab86399700 1 -- 172.21.15.3:0/2149667199 shutdown_connections 2023-10-03T10:13:41.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.684+0000 7fab86399700 1 --2- 172.21.15.3:0/2149667199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab80133a80 0x7fab8009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.684+0000 7fab86399700 1 --2- 172.21.15.3:0/2149667199 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab8012ff20 0x7fab8009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.684+0000 7fab86399700 1 --2- 172.21.15.3:0/2149667199 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab80130a40 0x7fab8009c9c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:41.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.684+0000 7fab86399700 1 -- 172.21.15.3:0/2149667199 >> 172.21.15.3:0/2149667199 conn(0x7fab800a4210 msgr2=0x7fab8009e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:41.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.684+0000 7fab86399700 1 -- 172.21.15.3:0/2149667199 shutdown_connections 2023-10-03T10:13:41.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:41.684+0000 7fab86399700 1 -- 172.21.15.3:0/2149667199 wait complete. 2023-10-03T10:13:41.698 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:42.401 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:42.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.548+0000 7fd71ccd9700 1 Processor -- start 2023-10-03T10:13:42.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.548+0000 7fd71ccd9700 1 -- start start 2023-10-03T10:13:42.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.549+0000 7fd71ccd9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd71809d6f0 0x7fd71809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:42.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.549+0000 7fd71ccd9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd71809e000 0x7fd7181339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:42.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.549+0000 7fd71ccd9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd71809e410 0x7fd718133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:42.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.549+0000 7fd71ccd9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd71806b460 con 0x7fd71809e410 2023-10-03T10:13:42.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.549+0000 7fd71ccd9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71809b120 con 0x7fd71809d6f0 2023-10-03T10:13:42.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.549+0000 7fd71ccd9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd71809a7e0 con 0x7fd71809e000 2023-10-03T10:13:42.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.549+0000 7fd71659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd71809d6f0 0x7fd71809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:42.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.549+0000 7fd716d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd71809e410 0x7fd718133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:42.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.549+0000 7fd715d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd71809e000 0x7fd7181339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:42.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.549+0000 7fd716d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd71809e410 0x7fd718133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45436/0 (socket says 172.21.15.3:45436) 2023-10-03T10:13:42.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.549+0000 7fd71659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd71809d6f0 0x7fd71809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44808/0 (socket says 172.21.15.3:44808) 2023-10-03T10:13:42.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.549+0000 7fd716d9d700 1 -- 172.21.15.3:0/210892161 learned_addr learned my addr 172.21.15.3:0/210892161 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:42.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.549+0000 7fd715d9b700 1 -- 172.21.15.3:0/210892161 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd71809d6f0 msgr2=0x7fd71809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:42.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.549+0000 7fd715d9b700 1 --2- 172.21.15.3:0/210892161 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd71809d6f0 0x7fd71809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:42.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.549+0000 7fd715d9b700 1 -- 172.21.15.3:0/210892161 >> v2:172.21.15.3:3300/0 conn(0x7fd71809e410 msgr2=0x7fd718133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:42.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.549+0000 7fd715d9b700 1 --2- 172.21.15.3:0/210892161 >> v2:172.21.15.3:3300/0 conn(0x7fd71809e410 0x7fd718133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:42.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.549+0000 7fd715d9b700 1 -- 172.21.15.3:0/210892161 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71809a500 con 0x7fd71809e000 2023-10-03T10:13:42.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.549+0000 7fd71659c700 1 --2- 172.21.15.3:0/210892161 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd71809d6f0 0x7fd71809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:42.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.549+0000 7fd716d9d700 1 --2- 172.21.15.3:0/210892161 >> v2:172.21.15.3:3300/0 conn(0x7fd71809e410 0x7fd718133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:42.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.549+0000 7fd715d9b700 1 --2- 172.21.15.3:0/210892161 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd71809e000 0x7fd7181339f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd700009870 tx=0x7fd700009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70e03f43e133d1af server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:42.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.550+0000 7fd71559a700 1 -- 172.21.15.3:0/210892161 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd700011830 con 0x7fd71809e000 2023-10-03T10:13:42.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.550+0000 7fd71559a700 1 -- 172.21.15.3:0/210892161 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd700011990 con 0x7fd71809e000 2023-10-03T10:13:42.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.550+0000 7fd71559a700 1 -- 172.21.15.3:0/210892161 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd700011d20 con 0x7fd71809e000 2023-10-03T10:13:42.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.550+0000 7fd71ccd9700 1 -- 172.21.15.3:0/210892161 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd71809e000 msgr2=0x7fd7181339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:42.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.550+0000 7fd71ccd9700 1 --2- 172.21.15.3:0/210892161 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd71809e000 0x7fd7181339f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd700009870 tx=0x7fd700009b80 comp rx=0 tx=0).stop 2023-10-03T10:13:42.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.550+0000 7fd71ccd9700 1 -- 172.21.15.3:0/210892161 shutdown_connections 2023-10-03T10:13:42.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.550+0000 7fd71ccd9700 1 --2- 172.21.15.3:0/210892161 >> v2:172.21.15.3:3300/0 conn(0x7fd71809e410 0x7fd718133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:42.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.550+0000 7fd71ccd9700 1 --2- 172.21.15.3:0/210892161 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd71809d6f0 0x7fd71809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:42.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.550+0000 7fd71ccd9700 1 --2- 172.21.15.3:0/210892161 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd71809e000 0x7fd7181339f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:42.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.550+0000 7fd71ccd9700 1 -- 172.21.15.3:0/210892161 >> 172.21.15.3:0/210892161 conn(0x7fd718066f50 msgr2=0x7fd71809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:42.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.550+0000 7fd71ccd9700 1 -- 172.21.15.3:0/210892161 shutdown_connections 2023-10-03T10:13:42.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.550+0000 7fd71ccd9700 1 -- 172.21.15.3:0/210892161 wait complete. 2023-10-03T10:13:42.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.552+0000 7fd71ccd9700 1 Processor -- start 2023-10-03T10:13:42.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.552+0000 7fd71ccd9700 1 -- start start 2023-10-03T10:13:42.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.552+0000 7fd71ccd9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd71809d6f0 0x7fd7181a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:42.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.552+0000 7fd71ccd9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd71809e000 0x7fd7181a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:42.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.552+0000 7fd71ccd9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd71809e410 0x7fd7181c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:42.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.552+0000 7fd71ccd9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7181a7010 con 0x7fd71809d6f0 2023-10-03T10:13:42.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.552+0000 7fd71ccd9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7181a3c80 con 0x7fd71809e410 2023-10-03T10:13:42.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.552+0000 7fd71ccd9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd71809c710 con 0x7fd71809e000 2023-10-03T10:13:42.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.552+0000 7fd71659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd71809d6f0 0x7fd7181a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:42.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.552+0000 7fd715d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd71809e000 0x7fd7181a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:42.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.552+0000 7fd71659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd71809d6f0 0x7fd7181a4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45452/0 (socket says 172.21.15.3:45452) 2023-10-03T10:13:42.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.552+0000 7fd71659c700 1 -- 172.21.15.3:0/1036773349 learned_addr learned my addr 172.21.15.3:0/1036773349 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:42.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.552+0000 7fd716d9d700 1 --2- 172.21.15.3:0/1036773349 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd71809e410 0x7fd7181c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:42.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.553+0000 7fd716d9d700 1 -- 172.21.15.3:0/1036773349 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd71809e000 msgr2=0x7fd7181a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:42.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.553+0000 7fd716d9d700 1 --2- 172.21.15.3:0/1036773349 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd71809e000 0x7fd7181a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:42.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.553+0000 7fd716d9d700 1 -- 172.21.15.3:0/1036773349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd71809d6f0 msgr2=0x7fd7181a4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:42.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.553+0000 7fd716d9d700 1 --2- 172.21.15.3:0/1036773349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd71809d6f0 0x7fd7181a4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:42.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.553+0000 7fd716d9d700 1 -- 172.21.15.3:0/1036773349 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7080094a0 con 0x7fd71809e410 2023-10-03T10:13:42.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.553+0000 7fd71659c700 1 --2- 172.21.15.3:0/1036773349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd71809d6f0 0x7fd7181a4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:42.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.553+0000 7fd716d9d700 1 --2- 172.21.15.3:0/1036773349 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd71809e410 0x7fd7181c8100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd71000efc0 tx=0x7fd71000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:42.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.553+0000 7fd70effd700 1 -- 172.21.15.3:0/1036773349 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71000ca80 con 0x7fd71809e410 2023-10-03T10:13:42.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.553+0000 7fd70effd700 1 -- 172.21.15.3:0/1036773349 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71000f040 con 0x7fd71809e410 2023-10-03T10:13:42.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.553+0000 7fd70effd700 1 -- 172.21.15.3:0/1036773349 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71000ce20 con 0x7fd71809e410 2023-10-03T10:13:42.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.553+0000 7fd71ccd9700 1 -- 172.21.15.3:0/1036773349 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd710009b30 con 0x7fd71809e410 2023-10-03T10:13:42.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.553+0000 7fd71ccd9700 1 -- 172.21.15.3:0/1036773349 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7000094a0 con 0x7fd71809e410 2023-10-03T10:13:42.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.554+0000 7fd70effd700 1 -- 172.21.15.3:0/1036773349 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd710014480 con 0x7fd71809e410 2023-10-03T10:13:42.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.554+0000 7fd70effd700 1 -- 172.21.15.3:0/1036773349 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd71003fb50 con 0x7fd71809e410 2023-10-03T10:13:42.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.554+0000 7fd71ccd9700 1 -- 172.21.15.3:0/1036773349 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd718074ea0 con 0x7fd71809e410 2023-10-03T10:13:42.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.559+0000 7fd70effd700 1 -- 172.21.15.3:0/1036773349 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd710012930 con 0x7fd71809e410 2023-10-03T10:13:42.609 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:42.655 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:42.699 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:42.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.760+0000 7f770a72c700 1 Processor -- start 2023-10-03T10:13:42.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.760+0000 7f770a72c700 1 -- start start 2023-10-03T10:13:42.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.761+0000 7f770a72c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77040a0a30 0x7f77040a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:42.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.761+0000 7f770a72c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f77040a1340 0x7f7704133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:42.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.761+0000 7f770a72c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f770409edd0 con 0x7f7704133fa0 2023-10-03T10:13:42.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.761+0000 7f770a72c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f770409e1f0 con 0x7f77040a1340 2023-10-03T10:13:42.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.761+0000 7f770a72c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f770409dc50 con 0x7f77040a0a30 2023-10-03T10:13:42.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.761+0000 7f77037fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f77040a1340 0x7f7704133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:42.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.761+0000 7f77037fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f77040a1340 0x7f7704133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45468/0 (socket says 172.21.15.3:45468) 2023-10-03T10:13:42.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.761+0000 7f7703fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77040a0a30 0x7f77040a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:42.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.761+0000 7f77037fe700 1 -- 172.21.15.3:0/3534956895 learned_addr learned my addr 172.21.15.3:0/3534956895 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:42.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.761+0000 7f77037fe700 1 -- 172.21.15.3:0/3534956895 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77040a0a30 msgr2=0x7f77040a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:42.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.761+0000 7f77037fe700 1 --2- 172.21.15.3:0/3534956895 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77040a0a30 0x7f77040a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:42.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.761+0000 7f77037fe700 1 -- 172.21.15.3:0/3534956895 >> v1:172.21.15.3:6789/0 conn(0x7f7704133fa0 legacy=0x7f770413ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:42.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.761+0000 7f77037fe700 1 -- 172.21.15.3:0/3534956895 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77040a1fe0 con 0x7f77040a1340 2023-10-03T10:13:42.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.761+0000 7f7702ffd700 1 -- 172.21.15.3:0/3534956895 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4051206037 0 0) 0x7f770409edd0 con 0x7f7704133fa0 2023-10-03T10:13:42.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.762+0000 7f77037fe700 1 --2- 172.21.15.3:0/3534956895 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77040a1340 0x7f7704133a60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f76f80098f0 tx=0x7f76f8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ec28d23129948a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:42.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.762+0000 7f7702ffd700 1 -- 172.21.15.3:0/3534956895 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8011830 con 0x7f77040a1340 2023-10-03T10:13:42.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.762+0000 7f7702ffd700 1 -- 172.21.15.3:0/3534956895 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f8011990 con 0x7f77040a1340 2023-10-03T10:13:42.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.762+0000 7f7702ffd700 1 -- 172.21.15.3:0/3534956895 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8011d20 con 0x7f77040a1340 2023-10-03T10:13:42.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.763+0000 7f770a72c700 1 -- 172.21.15.3:0/3534956895 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77040a1340 msgr2=0x7f7704133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:42.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.763+0000 7f770a72c700 1 --2- 172.21.15.3:0/3534956895 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77040a1340 0x7f7704133a60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f76f80098f0 tx=0x7f76f8009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:42.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.763+0000 7f770a72c700 1 -- 172.21.15.3:0/3534956895 shutdown_connections 2023-10-03T10:13:42.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.763+0000 7f770a72c700 1 --2- 172.21.15.3:0/3534956895 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77040a0a30 0x7f77040a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:42.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.763+0000 7f770a72c700 1 --2- 172.21.15.3:0/3534956895 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77040a1340 0x7f7704133a60 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:42.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.763+0000 7f770a72c700 1 -- 172.21.15.3:0/3534956895 >> 172.21.15.3:0/3534956895 conn(0x7f77040a6550 msgr2=0x7f77040a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:42.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.763+0000 7f770a72c700 1 -- 172.21.15.3:0/3534956895 shutdown_connections 2023-10-03T10:13:42.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.763+0000 7f770a72c700 1 -- 172.21.15.3:0/3534956895 wait complete. 2023-10-03T10:13:42.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.765+0000 7f770a72c700 1 Processor -- start 2023-10-03T10:13:42.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.765+0000 7f770a72c700 1 -- start start 2023-10-03T10:13:42.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.766+0000 7f770a72c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77040a0a30 0x7f77041233d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:42.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.766+0000 7f770a72c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77040a1340 0x7f7704123910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:42.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.766+0000 7f770a72c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7704133fa0 0x7f77041c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:42.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.766+0000 7f770a72c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7704122ff0 con 0x7f77040a0a30 2023-10-03T10:13:42.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.766+0000 7f770a72c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7704125fc0 con 0x7f7704133fa0 2023-10-03T10:13:42.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.766+0000 7f770a72c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7704123e50 con 0x7f77040a1340 2023-10-03T10:13:42.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.766+0000 7f7703fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77040a0a30 0x7f77041233d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:42.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.766+0000 7f7703fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77040a0a30 0x7f77041233d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45480/0 (socket says 172.21.15.3:45480) 2023-10-03T10:13:42.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.766+0000 7f7703fff700 1 -- 172.21.15.3:0/650240844 learned_addr learned my addr 172.21.15.3:0/650240844 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:42.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.766+0000 7f77037fe700 1 --2- 172.21.15.3:0/650240844 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77040a1340 0x7f7704123910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:42.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.766+0000 7f7708cc9700 1 --2- 172.21.15.3:0/650240844 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7704133fa0 0x7f77041c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:42.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.766+0000 7f77037fe700 1 -- 172.21.15.3:0/650240844 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7704133fa0 msgr2=0x7f77041c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:42.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.767+0000 7f77037fe700 1 --2- 172.21.15.3:0/650240844 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7704133fa0 0x7f77041c8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:42.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.767+0000 7f77037fe700 1 -- 172.21.15.3:0/650240844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77040a0a30 msgr2=0x7f77041233d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:42.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.767+0000 7f77037fe700 1 --2- 172.21.15.3:0/650240844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77040a0a30 0x7f77041233d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:42.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.767+0000 7f77037fe700 1 -- 172.21.15.3:0/650240844 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f8009550 con 0x7f77040a1340 2023-10-03T10:13:42.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.767+0000 7f7708cc9700 1 --2- 172.21.15.3:0/650240844 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7704133fa0 0x7f77041c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:42.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.767+0000 7f77037fe700 1 --2- 172.21.15.3:0/650240844 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77040a1340 0x7f7704123910 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f76f8016000 tx=0x7f76f8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:42.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.767+0000 7f7700ff9700 1 -- 172.21.15.3:0/650240844 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8004510 con 0x7f77040a1340 2023-10-03T10:13:42.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.767+0000 7f7700ff9700 1 -- 172.21.15.3:0/650240844 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f80036a0 con 0x7f77040a1340 2023-10-03T10:13:42.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.767+0000 7f7700ff9700 1 -- 172.21.15.3:0/650240844 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8003a30 con 0x7f77040a1340 2023-10-03T10:13:42.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.767+0000 7f770a72c700 1 -- 172.21.15.3:0/650240844 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77040a0220 con 0x7f77040a1340 2023-10-03T10:13:42.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.767+0000 7f770a72c700 1 -- 172.21.15.3:0/650240844 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7704133760 con 0x7f77040a1340 2023-10-03T10:13:42.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.768+0000 7f7700ff9700 1 -- 172.21.15.3:0/650240844 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f76f8021070 con 0x7f77040a1340 2023-10-03T10:13:42.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.768+0000 7f770a72c700 1 -- 172.21.15.3:0/650240844 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7704099a40 con 0x7f77040a1340 2023-10-03T10:13:42.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.769+0000 7f7700ff9700 1 -- 172.21.15.3:0/650240844 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f76f8040a70 con 0x7f77040a1340 2023-10-03T10:13:42.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.778+0000 7f7700ff9700 1 -- 172.21.15.3:0/650240844 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f76f8019770 con 0x7f77040a1340 2023-10-03T10:13:42.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.784+0000 7fd71ccd9700 1 -- 172.21.15.3:0/1036773349 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd7180008d0 con 0x7fd71809e410 2023-10-03T10:13:42.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.784+0000 7fd70effd700 1 -- 172.21.15.3:0/1036773349 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd710012340 con 0x7fd71809e410 2023-10-03T10:13:42.785 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:42.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.787+0000 7fd71ccd9700 1 -- 172.21.15.3:0/1036773349 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd71809e410 msgr2=0x7fd7181c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:42.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.787+0000 7fd71ccd9700 1 --2- 172.21.15.3:0/1036773349 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd71809e410 0x7fd7181c8100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd71000efc0 tx=0x7fd71000c3a0 comp rx=0 tx=0).stop 2023-10-03T10:13:42.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.788+0000 7fd71ccd9700 1 -- 172.21.15.3:0/1036773349 shutdown_connections 2023-10-03T10:13:42.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.788+0000 7fd71ccd9700 1 --2- 172.21.15.3:0/1036773349 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd71809e410 0x7fd7181c8100 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:42.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.788+0000 7fd71ccd9700 1 --2- 172.21.15.3:0/1036773349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd71809d6f0 0x7fd7181a4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:42.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.788+0000 7fd71ccd9700 1 --2- 172.21.15.3:0/1036773349 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd71809e000 0x7fd7181a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:42.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.788+0000 7fd71ccd9700 1 -- 172.21.15.3:0/1036773349 >> 172.21.15.3:0/1036773349 conn(0x7fd718066f50 msgr2=0x7fd7181bbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:42.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.788+0000 7fd71ccd9700 1 -- 172.21.15.3:0/1036773349 shutdown_connections 2023-10-03T10:13:42.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.788+0000 7fd71ccd9700 1 -- 172.21.15.3:0/1036773349 wait complete. 2023-10-03T10:13:42.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.801+0000 7f1c2822d700 1 Processor -- start 2023-10-03T10:13:42.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.801+0000 7f1c2822d700 1 -- start start 2023-10-03T10:13:42.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.802+0000 7f1c2822d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1c2009d6f0 0x7f1c2009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:42.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.802+0000 7f1c2822d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c2009e440 0x7f1c20134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:42.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.802+0000 7f1c2822d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c2009b120 con 0x7f1c2009e000 2023-10-03T10:13:42.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.802+0000 7f1c2822d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c2006b460 con 0x7f1c2009d6f0 2023-10-03T10:13:42.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.802+0000 7f1c2822d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2009a7e0 con 0x7f1c2009e440 2023-10-03T10:13:42.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.802+0000 7f1c267ca700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c2009e440 0x7f1c20134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:42.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.802+0000 7f1c25fc9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1c2009d6f0 0x7f1c2009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:42.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.802+0000 7f1c267ca700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c2009e440 0x7f1c20134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44838/0 (socket says 172.21.15.3:44838) 2023-10-03T10:13:42.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.802+0000 7f1c267ca700 1 -- 172.21.15.3:0/3618758775 learned_addr learned my addr 172.21.15.3:0/3618758775 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:42.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.802+0000 7f1c25fc9700 1 --2- 172.21.15.3:0/3618758775 >> v2:172.21.15.3:3300/0 conn(0x7f1c2009d6f0 0x7f1c2009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45492/0 (socket says 172.21.15.3:45492) 2023-10-03T10:13:42.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.803+0000 7f1c267ca700 1 -- 172.21.15.3:0/3618758775 >> v2:172.21.15.3:3300/0 conn(0x7f1c2009d6f0 msgr2=0x7f1c2009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:42.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.803+0000 7f1c267ca700 1 --2- 172.21.15.3:0/3618758775 >> v2:172.21.15.3:3300/0 conn(0x7f1c2009d6f0 0x7f1c2009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:42.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.803+0000 7f1c267ca700 1 -- 172.21.15.3:0/3618758775 >> v1:172.21.15.3:6789/0 conn(0x7f1c2009e000 legacy=0x7f1c201339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:42.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.803+0000 7f1c267ca700 1 -- 172.21.15.3:0/3618758775 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2009a500 con 0x7f1c2009e440 2023-10-03T10:13:42.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.803+0000 7f1c267ca700 1 --2- 172.21.15.3:0/3618758775 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c2009e440 0x7f1c20134100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009e80 tx=0x7f1c1800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=912fd85273891fd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:42.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.803+0000 7f1c24fc7700 1 -- 172.21.15.3:0/3618758775 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1800e070 con 0x7f1c2009e440 2023-10-03T10:13:42.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.803+0000 7f1c24fc7700 1 -- 172.21.15.3:0/3618758775 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1800b8d0 con 0x7f1c2009e440 2023-10-03T10:13:42.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.803+0000 7f1c24fc7700 1 -- 172.21.15.3:0/3618758775 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1800bc80 con 0x7f1c2009e440 2023-10-03T10:13:42.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.804+0000 7f1c2822d700 1 -- 172.21.15.3:0/3618758775 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c2009e440 msgr2=0x7f1c20134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:42.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.804+0000 7f1c2822d700 1 --2- 172.21.15.3:0/3618758775 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c2009e440 0x7f1c20134100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009e80 tx=0x7f1c1800b330 comp rx=0 tx=0).stop 2023-10-03T10:13:42.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.804+0000 7f1c2822d700 1 -- 172.21.15.3:0/3618758775 shutdown_connections 2023-10-03T10:13:42.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.804+0000 7f1c2822d700 1 --2- 172.21.15.3:0/3618758775 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c2009e440 0x7f1c20134100 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:42.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.804+0000 7f1c2822d700 1 --2- 172.21.15.3:0/3618758775 >> v2:172.21.15.3:3300/0 conn(0x7f1c2009d6f0 0x7f1c2009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:42.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.804+0000 7f1c2822d700 1 -- 172.21.15.3:0/3618758775 >> 172.21.15.3:0/3618758775 conn(0x7f1c20066f50 msgr2=0x7f1c2009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:42.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.804+0000 7f1c2822d700 1 -- 172.21.15.3:0/3618758775 shutdown_connections 2023-10-03T10:13:42.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.804+0000 7f1c2822d700 1 -- 172.21.15.3:0/3618758775 wait complete. 2023-10-03T10:13:42.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.806+0000 7f1c2822d700 1 Processor -- start 2023-10-03T10:13:42.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.806+0000 7f1c2822d700 1 -- start start 2023-10-03T10:13:42.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.806+0000 7f1c2822d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c2009d6f0 0x7f1c201c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:42.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.806+0000 7f1c2822d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c2009e000 0x7f1c201c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:42.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.806+0000 7f1c2822d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c2009e440 0x7f1c201c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:42.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.806+0000 7f1c2822d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c201a80b0 con 0x7f1c2009e440 2023-10-03T10:13:42.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.806+0000 7f1c2822d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200a1f90 con 0x7f1c2009d6f0 2023-10-03T10:13:42.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.806+0000 7f1c2822d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c201b9550 con 0x7f1c2009e000 2023-10-03T10:13:42.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.807+0000 7f1c25fc9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c2009d6f0 0x7f1c201c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:42.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.807+0000 7f1c25fc9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c2009d6f0 0x7f1c201c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44844/0 (socket says 172.21.15.3:44844) 2023-10-03T10:13:42.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.807+0000 7f1c25fc9700 1 -- 172.21.15.3:0/3907333504 learned_addr learned my addr 172.21.15.3:0/3907333504 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:42.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.807+0000 7f1c267ca700 1 --2- 172.21.15.3:0/3907333504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c2009e440 0x7f1c201c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:42.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.807+0000 7f1c25fc9700 1 -- 172.21.15.3:0/3907333504 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c2009e000 msgr2=0x7f1c201c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:42.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.807+0000 7f1c25fc9700 1 --2- 172.21.15.3:0/3907333504 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c2009e000 0x7f1c201c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:42.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.807+0000 7f1c25fc9700 1 -- 172.21.15.3:0/3907333504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c2009e440 msgr2=0x7f1c201c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:42.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.807+0000 7f1c25fc9700 1 --2- 172.21.15.3:0/3907333504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c2009e440 0x7f1c201c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:42.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.807+0000 7f1c25fc9700 1 -- 172.21.15.3:0/3907333504 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c18009b30 con 0x7f1c2009d6f0 2023-10-03T10:13:42.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.807+0000 7f1c25fc9700 1 --2- 172.21.15.3:0/3907333504 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c2009d6f0 0x7f1c201c8560 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c00e9b0 tx=0x7f1c1c00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:42.810 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:42.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.808+0000 7f1c167fc700 1 -- 172.21.15.3:0/3907333504 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c00c850 con 0x7f1c2009d6f0 2023-10-03T10:13:42.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.809+0000 7f1c167fc700 1 -- 172.21.15.3:0/3907333504 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1c00c9b0 con 0x7f1c2009d6f0 2023-10-03T10:13:42.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.809+0000 7f1c2822d700 1 -- 172.21.15.3:0/3907333504 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c201a8980 con 0x7f1c2009d6f0 2023-10-03T10:13:42.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.809+0000 7f1c167fc700 1 -- 172.21.15.3:0/3907333504 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c00cd20 con 0x7f1c2009d6f0 2023-10-03T10:13:42.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.809+0000 7f1c2822d700 1 -- 172.21.15.3:0/3907333504 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c201a79c0 con 0x7f1c2009d6f0 2023-10-03T10:13:42.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.810+0000 7f1c167fc700 1 -- 172.21.15.3:0/3907333504 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1c1c013470 con 0x7f1c2009d6f0 2023-10-03T10:13:42.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.810+0000 7f1c2822d700 1 -- 172.21.15.3:0/3907333504 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c20074ea0 con 0x7f1c2009d6f0 2023-10-03T10:13:42.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.810+0000 7f1c167fc700 1 -- 172.21.15.3:0/3907333504 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1c1c03f680 con 0x7f1c2009d6f0 2023-10-03T10:13:42.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.815+0000 7f1c167fc700 1 -- 172.21.15.3:0/3907333504 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1c1c013720 con 0x7f1c2009d6f0 2023-10-03T10:13:42.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.850+0000 7f2e36fa7700 1 Processor -- start 2023-10-03T10:13:42.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.850+0000 7f2e36fa7700 1 -- start start 2023-10-03T10:13:42.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.851+0000 7f2e36fa7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e3009d6f0 0x7f2e3009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:42.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.851+0000 7f2e36fa7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2e3009e000 0x7f2e301339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:42.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.851+0000 7f2e36fa7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e3009b120 con 0x7f2e3009e410 2023-10-03T10:13:42.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.851+0000 7f2e36fa7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e3006b460 con 0x7f2e3009e000 2023-10-03T10:13:42.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.851+0000 7f2e36fa7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3009a7e0 con 0x7f2e3009d6f0 2023-10-03T10:13:42.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.851+0000 7f2e2ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2e3009e000 0x7f2e301339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:42.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.851+0000 7f2e34d43700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e3009d6f0 0x7f2e3009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:42.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.851+0000 7f2e2ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2e3009e000 0x7f2e301339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45520/0 (socket says 172.21.15.3:45520) 2023-10-03T10:13:42.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.851+0000 7f2e34d43700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e3009d6f0 0x7f2e3009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44856/0 (socket says 172.21.15.3:44856) 2023-10-03T10:13:42.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.851+0000 7f2e2ffff700 1 -- 172.21.15.3:0/829674045 learned_addr learned my addr 172.21.15.3:0/829674045 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:42.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.852+0000 7f2e34d43700 1 -- 172.21.15.3:0/829674045 >> v2:172.21.15.3:3300/0 conn(0x7f2e3009e000 msgr2=0x7f2e301339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:42.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.852+0000 7f2e34d43700 1 --2- 172.21.15.3:0/829674045 >> v2:172.21.15.3:3300/0 conn(0x7f2e3009e000 0x7f2e301339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:42.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.852+0000 7f2e34d43700 1 -- 172.21.15.3:0/829674045 >> v1:172.21.15.3:6789/0 conn(0x7f2e3009e410 legacy=0x7f2e30133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:42.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.852+0000 7f2e34d43700 1 -- 172.21.15.3:0/829674045 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e3009a500 con 0x7f2e3009d6f0 2023-10-03T10:13:42.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.852+0000 7f2e2ffff700 1 --2- 172.21.15.3:0/829674045 >> v2:172.21.15.3:3300/0 conn(0x7f2e3009e000 0x7f2e301339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:42.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.852+0000 7f2e34d43700 1 --2- 172.21.15.3:0/829674045 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e3009d6f0 0x7f2e3009dac0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2e24009870 tx=0x7f2e24009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1a6049b6e5bfced server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:42.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.852+0000 7f2e2f7fe700 1 -- 172.21.15.3:0/829674045 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e24011830 con 0x7f2e3009d6f0 2023-10-03T10:13:42.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.852+0000 7f2e2f7fe700 1 -- 172.21.15.3:0/829674045 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e24011990 con 0x7f2e3009d6f0 2023-10-03T10:13:42.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.852+0000 7f2e2f7fe700 1 -- 172.21.15.3:0/829674045 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e24011d20 con 0x7f2e3009d6f0 2023-10-03T10:13:42.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.853+0000 7f2e36fa7700 1 -- 172.21.15.3:0/829674045 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e3009d6f0 msgr2=0x7f2e3009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:42.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.853+0000 7f2e36fa7700 1 --2- 172.21.15.3:0/829674045 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e3009d6f0 0x7f2e3009dac0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2e24009870 tx=0x7f2e24009b80 comp rx=0 tx=0).stop 2023-10-03T10:13:42.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.853+0000 7f2e36fa7700 1 -- 172.21.15.3:0/829674045 shutdown_connections 2023-10-03T10:13:42.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.853+0000 7f2e36fa7700 1 --2- 172.21.15.3:0/829674045 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e3009d6f0 0x7f2e3009dac0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:42.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.853+0000 7f2e36fa7700 1 --2- 172.21.15.3:0/829674045 >> v2:172.21.15.3:3300/0 conn(0x7f2e3009e000 0x7f2e301339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:42.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.853+0000 7f2e36fa7700 1 -- 172.21.15.3:0/829674045 >> 172.21.15.3:0/829674045 conn(0x7f2e30066f50 msgr2=0x7f2e3009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:42.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.853+0000 7f2e36fa7700 1 -- 172.21.15.3:0/829674045 shutdown_connections 2023-10-03T10:13:42.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.853+0000 7f2e36fa7700 1 -- 172.21.15.3:0/829674045 wait complete. 2023-10-03T10:13:42.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.855+0000 7f2e36fa7700 1 Processor -- start 2023-10-03T10:13:42.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.855+0000 7f2e36fa7700 1 -- start start 2023-10-03T10:13:42.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.856+0000 7f2e36fa7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e3009d6f0 0x7f2e301a4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:42.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.856+0000 7f2e36fa7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e3009e000 0x7f2e301a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:42.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.856+0000 7f2e36fa7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e3009e410 0x7f2e301a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:42.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.856+0000 7f2e36fa7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e301a6ae0 con 0x7f2e3009e000 2023-10-03T10:13:42.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.856+0000 7f2e36fa7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e301a7d30 con 0x7f2e3009d6f0 2023-10-03T10:13:42.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.856+0000 7f2e36fa7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e301a70d0 con 0x7f2e3009e410 2023-10-03T10:13:42.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.856+0000 7f2e34d43700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e3009d6f0 0x7f2e301a4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:42.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.856+0000 7f2e34d43700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e3009d6f0 0x7f2e301a4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44858/0 (socket says 172.21.15.3:44858) 2023-10-03T10:13:42.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.856+0000 7f2e34d43700 1 -- 172.21.15.3:0/3816986730 learned_addr learned my addr 172.21.15.3:0/3816986730 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:42.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.856+0000 7f2e2ffff700 1 --2- 172.21.15.3:0/3816986730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e3009e000 0x7f2e301a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:42.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.856+0000 7f2e35544700 1 --2- 172.21.15.3:0/3816986730 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e3009e410 0x7f2e301a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:42.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.857+0000 7f2e34d43700 1 -- 172.21.15.3:0/3816986730 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e3009e410 msgr2=0x7f2e301a59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:42.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.857+0000 7f2e34d43700 1 --2- 172.21.15.3:0/3816986730 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e3009e410 0x7f2e301a59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:42.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.857+0000 7f2e34d43700 1 -- 172.21.15.3:0/3816986730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e3009e000 msgr2=0x7f2e301a54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:42.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.857+0000 7f2e34d43700 1 --2- 172.21.15.3:0/3816986730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e3009e000 0x7f2e301a54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:42.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.857+0000 7f2e34d43700 1 -- 172.21.15.3:0/3816986730 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e1c009550 con 0x7f2e3009d6f0 2023-10-03T10:13:42.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.857+0000 7f2e2ffff700 1 --2- 172.21.15.3:0/3816986730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e3009e000 0x7f2e301a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:42.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.857+0000 7f2e34d43700 1 --2- 172.21.15.3:0/3816986730 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e3009d6f0 0x7f2e301a4f60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2e24000c00 tx=0x7f2e24004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:42.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.857+0000 7f2e2d7fa700 1 -- 172.21.15.3:0/3816986730 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e240036a0 con 0x7f2e3009d6f0 2023-10-03T10:13:42.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.857+0000 7f2e2d7fa700 1 -- 172.21.15.3:0/3816986730 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2401f070 con 0x7f2e3009d6f0 2023-10-03T10:13:42.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.857+0000 7f2e36fa7700 1 -- 172.21.15.3:0/3816986730 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e240094a0 con 0x7f2e3009d6f0 2023-10-03T10:13:42.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.857+0000 7f2e2d7fa700 1 -- 172.21.15.3:0/3816986730 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e24003a60 con 0x7f2e3009d6f0 2023-10-03T10:13:42.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.857+0000 7f2e36fa7700 1 -- 172.21.15.3:0/3816986730 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3009c710 con 0x7f2e3009d6f0 2023-10-03T10:13:42.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.858+0000 7f2e2d7fa700 1 -- 172.21.15.3:0/3816986730 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2e24003bc0 con 0x7f2e3009d6f0 2023-10-03T10:13:42.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.858+0000 7f2e2d7fa700 1 -- 172.21.15.3:0/3816986730 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2e24021070 con 0x7f2e3009d6f0 2023-10-03T10:13:42.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.859+0000 7f2e36fa7700 1 -- 172.21.15.3:0/3816986730 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e30074ea0 con 0x7f2e3009d6f0 2023-10-03T10:13:42.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:42.863+0000 7f2e2d7fa700 1 -- 172.21.15.3:0/3816986730 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2e2402b080 con 0x7f2e3009d6f0 2023-10-03T10:13:43.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.000+0000 7f770a72c700 1 -- 172.21.15.3:0/650240844 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f77040a1d40 con 0x7f77040a1340 2023-10-03T10:13:43.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.001+0000 7f7700ff9700 1 -- 172.21.15.3:0/650240844 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f76f8040e60 con 0x7f77040a1340 2023-10-03T10:13:43.001 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.003+0000 7f770a72c700 1 -- 172.21.15.3:0/650240844 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77040a1340 msgr2=0x7f7704123910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.003+0000 7f770a72c700 1 --2- 172.21.15.3:0/650240844 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77040a1340 0x7f7704123910 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f76f8016000 tx=0x7f76f8004750 comp rx=0 tx=0).stop 2023-10-03T10:13:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.003+0000 7f770a72c700 1 -- 172.21.15.3:0/650240844 shutdown_connections 2023-10-03T10:13:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.003+0000 7f770a72c700 1 --2- 172.21.15.3:0/650240844 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7704133fa0 0x7f77041c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.003+0000 7f770a72c700 1 --2- 172.21.15.3:0/650240844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77040a0a30 0x7f77041233d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.003+0000 7f770a72c700 1 --2- 172.21.15.3:0/650240844 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77040a1340 0x7f7704123910 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.003+0000 7f770a72c700 1 -- 172.21.15.3:0/650240844 >> 172.21.15.3:0/650240844 conn(0x7f77040a6550 msgr2=0x7f77040a52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.003+0000 7f770a72c700 1 -- 172.21.15.3:0/650240844 shutdown_connections 2023-10-03T10:13:43.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.003+0000 7f770a72c700 1 -- 172.21.15.3:0/650240844 wait complete. 2023-10-03T10:13:43.018 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:43.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.044+0000 7f1c2822d700 1 -- 172.21.15.3:0/3907333504 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1c201a79c0 con 0x7f1c2009d6f0 2023-10-03T10:13:43.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.045+0000 7f1c167fc700 1 -- 172.21.15.3:0/3907333504 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1c201a79c0 con 0x7f1c2009d6f0 2023-10-03T10:13:43.045 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:43.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.047+0000 7f1c2822d700 1 -- 172.21.15.3:0/3907333504 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c2009d6f0 msgr2=0x7f1c201c8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:43.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.047+0000 7f1c2822d700 1 --2- 172.21.15.3:0/3907333504 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c2009d6f0 0x7f1c201c8560 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c00e9b0 tx=0x7f1c1c00ecc0 comp rx=0 tx=0).stop 2023-10-03T10:13:43.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.047+0000 7f1c2822d700 1 -- 172.21.15.3:0/3907333504 shutdown_connections 2023-10-03T10:13:43.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.047+0000 7f1c2822d700 1 --2- 172.21.15.3:0/3907333504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c2009e440 0x7f1c201c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:43.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.047+0000 7f1c2822d700 1 --2- 172.21.15.3:0/3907333504 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c2009d6f0 0x7f1c201c8560 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:43.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.047+0000 7f1c2822d700 1 --2- 172.21.15.3:0/3907333504 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c2009e000 0x7f1c201c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:43.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.047+0000 7f1c2822d700 1 -- 172.21.15.3:0/3907333504 >> 172.21.15.3:0/3907333504 conn(0x7f1c20066f50 msgr2=0x7f1c201bad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:43.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.047+0000 7f1c2822d700 1 -- 172.21.15.3:0/3907333504 shutdown_connections 2023-10-03T10:13:43.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.047+0000 7f1c2822d700 1 -- 172.21.15.3:0/3907333504 wait complete. 2023-10-03T10:13:43.061 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:43.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.085+0000 7f2e36fa7700 1 -- 172.21.15.3:0/3816986730 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2e3006c360 con 0x7f2e3009d6f0 2023-10-03T10:13:43.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.086+0000 7f2e2d7fa700 1 -- 172.21.15.3:0/3816986730 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2e24015020 con 0x7f2e3009d6f0 2023-10-03T10:13:43.086 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:43.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.088+0000 7f2e36fa7700 1 -- 172.21.15.3:0/3816986730 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e3009d6f0 msgr2=0x7f2e301a4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:43.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.088+0000 7f2e36fa7700 1 --2- 172.21.15.3:0/3816986730 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e3009d6f0 0x7f2e301a4f60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2e24000c00 tx=0x7f2e24004750 comp rx=0 tx=0).stop 2023-10-03T10:13:43.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.088+0000 7f2e36fa7700 1 -- 172.21.15.3:0/3816986730 shutdown_connections 2023-10-03T10:13:43.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.088+0000 7f2e36fa7700 1 --2- 172.21.15.3:0/3816986730 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e3009e410 0x7f2e301a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:43.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.088+0000 7f2e36fa7700 1 --2- 172.21.15.3:0/3816986730 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e3009d6f0 0x7f2e301a4f60 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:43.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.088+0000 7f2e36fa7700 1 --2- 172.21.15.3:0/3816986730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e3009e000 0x7f2e301a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:43.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.088+0000 7f2e36fa7700 1 -- 172.21.15.3:0/3816986730 >> 172.21.15.3:0/3816986730 conn(0x7f2e30066f50 msgr2=0x7f2e301bb010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:43.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.088+0000 7f2e36fa7700 1 -- 172.21.15.3:0/3816986730 shutdown_connections 2023-10-03T10:13:43.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.088+0000 7f2e36fa7700 1 -- 172.21.15.3:0/3816986730 wait complete. 2023-10-03T10:13:43.102 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:43.811 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:43.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.957+0000 7fd6fd77a700 1 Processor -- start 2023-10-03T10:13:43.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.957+0000 7fd6fd77a700 1 -- start start 2023-10-03T10:13:43.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.957+0000 7fd6fd77a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd6f809e850 0x7fd6f809ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:43.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.957+0000 7fd6fd77a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6f809ffb0 0x7fd6f809f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:43.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.957+0000 7fd6fd77a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6f8133b80 0x7fd6f8133fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:43.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.957+0000 7fd6fd77a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f806b920 con 0x7fd6f809e850 2023-10-03T10:13:43.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.957+0000 7fd6fd77a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f80a06b0 con 0x7fd6f809ffb0 2023-10-03T10:13:43.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.957+0000 7fd6fd77a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f809aa30 con 0x7fd6f8133b80 2023-10-03T10:13:43.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.957+0000 7fd6f67fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6f809ffb0 0x7fd6f809f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:43.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.957+0000 7fd6f6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd6f809e850 0x7fd6f809ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:43.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.957+0000 7fd6f77fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6f8133b80 0x7fd6f8133fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:43.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.958+0000 7fd6f6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd6f809e850 0x7fd6f809ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46606/0 (socket says 172.21.15.3:46606) 2023-10-03T10:13:43.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.958+0000 7fd6f67fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6f809ffb0 0x7fd6f809f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46116/0 (socket says 172.21.15.3:46116) 2023-10-03T10:13:43.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.958+0000 7fd6f6ffd700 1 -- 172.21.15.3:0/781499812 learned_addr learned my addr 172.21.15.3:0/781499812 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:43.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.958+0000 7fd6f77fe700 1 -- 172.21.15.3:0/781499812 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6f809ffb0 msgr2=0x7fd6f809f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:43.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.958+0000 7fd6f77fe700 1 --2- 172.21.15.3:0/781499812 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6f809ffb0 0x7fd6f809f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:43.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.958+0000 7fd6f77fe700 1 -- 172.21.15.3:0/781499812 >> v2:172.21.15.3:3300/0 conn(0x7fd6f809e850 msgr2=0x7fd6f809ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:43.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.958+0000 7fd6f77fe700 1 --2- 172.21.15.3:0/781499812 >> v2:172.21.15.3:3300/0 conn(0x7fd6f809e850 0x7fd6f809ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:43.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.958+0000 7fd6f77fe700 1 -- 172.21.15.3:0/781499812 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f806caa0 con 0x7fd6f8133b80 2023-10-03T10:13:43.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.958+0000 7fd6f6ffd700 1 --2- 172.21.15.3:0/781499812 >> v2:172.21.15.3:3300/0 conn(0x7fd6f809e850 0x7fd6f809ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:43.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.958+0000 7fd6f77fe700 1 --2- 172.21.15.3:0/781499812 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6f8133b80 0x7fd6f8133fe0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0009d90 tx=0x7fd6f000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e28c4a3fc68faf5f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:43.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.958+0000 7fd6f5ffb700 1 -- 172.21.15.3:0/781499812 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f000e070 con 0x7fd6f8133b80 2023-10-03T10:13:43.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.958+0000 7fd6f5ffb700 1 -- 172.21.15.3:0/781499812 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f000bb20 con 0x7fd6f8133b80 2023-10-03T10:13:43.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.958+0000 7fd6f5ffb700 1 -- 172.21.15.3:0/781499812 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0012430 con 0x7fd6f8133b80 2023-10-03T10:13:43.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.959+0000 7fd6fd77a700 1 -- 172.21.15.3:0/781499812 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6f8133b80 msgr2=0x7fd6f8133fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:43.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.959+0000 7fd6fd77a700 1 --2- 172.21.15.3:0/781499812 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6f8133b80 0x7fd6f8133fe0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0009d90 tx=0x7fd6f000b330 comp rx=0 tx=0).stop 2023-10-03T10:13:43.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.959+0000 7fd6fd77a700 1 -- 172.21.15.3:0/781499812 shutdown_connections 2023-10-03T10:13:43.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.959+0000 7fd6fd77a700 1 --2- 172.21.15.3:0/781499812 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6f8133b80 0x7fd6f8133fe0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:43.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.959+0000 7fd6fd77a700 1 --2- 172.21.15.3:0/781499812 >> v2:172.21.15.3:3300/0 conn(0x7fd6f809e850 0x7fd6f809ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:43.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.959+0000 7fd6fd77a700 1 --2- 172.21.15.3:0/781499812 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6f809ffb0 0x7fd6f809f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:43.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.959+0000 7fd6fd77a700 1 -- 172.21.15.3:0/781499812 >> 172.21.15.3:0/781499812 conn(0x7fd6f80a3f70 msgr2=0x7fd6f80a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:43.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.959+0000 7fd6fd77a700 1 -- 172.21.15.3:0/781499812 shutdown_connections 2023-10-03T10:13:43.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.959+0000 7fd6fd77a700 1 -- 172.21.15.3:0/781499812 wait complete. 2023-10-03T10:13:43.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.960+0000 7fd6fd77a700 1 Processor -- start 2023-10-03T10:13:43.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.960+0000 7fd6fd77a700 1 -- start start 2023-10-03T10:13:43.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.961+0000 7fd6fd77a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6f809e850 0x7fd6f8120510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:43.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.961+0000 7fd6fd77a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6f809ffb0 0x7fd6f8120a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:43.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.961+0000 7fd6fd77a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6f8133b80 0x7fd6f8120f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:43.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.961+0000 7fd6fd77a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f8123d20 con 0x7fd6f809e850 2023-10-03T10:13:43.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.961+0000 7fd6fd77a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f80a3430 con 0x7fd6f809ffb0 2023-10-03T10:13:43.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.961+0000 7fd6fd77a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f8122090 con 0x7fd6f8133b80 2023-10-03T10:13:43.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.961+0000 7fd6f6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6f809e850 0x7fd6f8120510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:43.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.961+0000 7fd6f6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6f809e850 0x7fd6f8120510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46616/0 (socket says 172.21.15.3:46616) 2023-10-03T10:13:43.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.961+0000 7fd6f6ffd700 1 -- 172.21.15.3:0/4240854862 learned_addr learned my addr 172.21.15.3:0/4240854862 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:43.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.961+0000 7fd6f67fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6f809ffb0 0x7fd6f8120a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:43.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.961+0000 7fd6f77fe700 1 --2- 172.21.15.3:0/4240854862 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6f8133b80 0x7fd6f8120f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:43.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.961+0000 7fd6f6ffd700 1 -- 172.21.15.3:0/4240854862 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6f8133b80 msgr2=0x7fd6f8120f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:43.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.961+0000 7fd6f6ffd700 1 --2- 172.21.15.3:0/4240854862 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6f8133b80 0x7fd6f8120f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:43.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.961+0000 7fd6f6ffd700 1 -- 172.21.15.3:0/4240854862 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6f809ffb0 msgr2=0x7fd6f8120a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:43.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.961+0000 7fd6f6ffd700 1 --2- 172.21.15.3:0/4240854862 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6f809ffb0 0x7fd6f8120a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:43.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.961+0000 7fd6f6ffd700 1 -- 172.21.15.3:0/4240854862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e8009550 con 0x7fd6f809e850 2023-10-03T10:13:43.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.961+0000 7fd6f67fc700 1 --2- 172.21.15.3:0/4240854862 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6f809ffb0 0x7fd6f8120a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:43.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.961+0000 7fd6f6ffd700 1 --2- 172.21.15.3:0/4240854862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6f809e850 0x7fd6f8120510 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd6e800ec10 tx=0x7fd6e800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:43.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.961+0000 7fd6df7fe700 1 -- 172.21.15.3:0/4240854862 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e800c850 con 0x7fd6f809e850 2023-10-03T10:13:43.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.961+0000 7fd6df7fe700 1 -- 172.21.15.3:0/4240854862 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e800c9b0 con 0x7fd6f809e850 2023-10-03T10:13:43.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.961+0000 7fd6df7fe700 1 -- 172.21.15.3:0/4240854862 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e800cd40 con 0x7fd6f809e850 2023-10-03T10:13:43.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.962+0000 7fd6fd77a700 1 -- 172.21.15.3:0/4240854862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f0009a70 con 0x7fd6f809e850 2023-10-03T10:13:43.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.962+0000 7fd6fd77a700 1 -- 172.21.15.3:0/4240854862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f8122df0 con 0x7fd6f809e850 2023-10-03T10:13:43.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.962+0000 7fd6df7fe700 1 -- 172.21.15.3:0/4240854862 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd6e8013470 con 0x7fd6f809e850 2023-10-03T10:13:43.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.963+0000 7fd6df7fe700 1 -- 172.21.15.3:0/4240854862 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd6e803fc20 con 0x7fd6f809e850 2023-10-03T10:13:43.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.963+0000 7fd6fd77a700 1 -- 172.21.15.3:0/4240854862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f809cbf0 con 0x7fd6f809e850 2023-10-03T10:13:43.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:43.968+0000 7fd6df7fe700 1 -- 172.21.15.3:0/4240854862 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd6e8013720 con 0x7fd6f809e850 2023-10-03T10:13:44.018 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:44.062 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:44.103 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:44.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.169+0000 7feeba110700 1 Processor -- start 2023-10-03T10:13:44.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.169+0000 7feeba110700 1 -- start start 2023-10-03T10:13:44.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.170+0000 7feeba110700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feeb409d6f0 0x7feeb409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:44.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.170+0000 7feeba110700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feeb409e000 0x7feeb4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:44.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.170+0000 7feeba110700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feeb409e410 0x7feeb4130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:44.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.170+0000 7feeba110700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb409f520 con 0x7feeb409d6f0 2023-10-03T10:13:44.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.170+0000 7feeba110700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb409f130 con 0x7feeb409e410 2023-10-03T10:13:44.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.170+0000 7feeba110700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feeb409e930 con 0x7feeb409e000 2023-10-03T10:13:44.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.170+0000 7feeb3fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feeb409e410 0x7feeb4130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:44.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.170+0000 7feeb37fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feeb409d6f0 0x7feeb409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:44.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.170+0000 7feeb3fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feeb409e410 0x7feeb4130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46136/0 (socket says 172.21.15.3:46136) 2023-10-03T10:13:44.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.170+0000 7feeb3fff700 1 -- 172.21.15.3:0/2638811469 learned_addr learned my addr 172.21.15.3:0/2638811469 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:44.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.170+0000 7feeb2ffd700 1 --2- 172.21.15.3:0/2638811469 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feeb409e000 0x7feeb4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:44.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.170+0000 7feeb2ffd700 1 -- 172.21.15.3:0/2638811469 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feeb409e410 msgr2=0x7feeb4130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:44.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.170+0000 7feeb2ffd700 1 --2- 172.21.15.3:0/2638811469 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feeb409e410 0x7feeb4130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.170+0000 7feeb2ffd700 1 -- 172.21.15.3:0/2638811469 >> v2:172.21.15.3:3300/0 conn(0x7feeb409d6f0 msgr2=0x7feeb409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:44.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.170+0000 7feeb2ffd700 1 --2- 172.21.15.3:0/2638811469 >> v2:172.21.15.3:3300/0 conn(0x7feeb409d6f0 0x7feeb409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.170+0000 7feeb2ffd700 1 -- 172.21.15.3:0/2638811469 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb409b2d0 con 0x7feeb409e000 2023-10-03T10:13:44.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.170+0000 7feeb37fe700 1 --2- 172.21.15.3:0/2638811469 >> v2:172.21.15.3:3300/0 conn(0x7feeb409d6f0 0x7feeb409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:44.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.171+0000 7feeb2ffd700 1 --2- 172.21.15.3:0/2638811469 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feeb409e000 0x7feeb4130660 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7feea800bdc0 tx=0x7feea800d330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b65e0203523727c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:44.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.171+0000 7feeb27fc700 1 -- 172.21.15.3:0/2638811469 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea8010070 con 0x7feeb409e000 2023-10-03T10:13:44.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.171+0000 7feeb27fc700 1 -- 172.21.15.3:0/2638811469 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea800dae0 con 0x7feeb409e000 2023-10-03T10:13:44.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.171+0000 7feeb27fc700 1 -- 172.21.15.3:0/2638811469 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea800de90 con 0x7feeb409e000 2023-10-03T10:13:44.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.172+0000 7feeba110700 1 -- 172.21.15.3:0/2638811469 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feeb409e000 msgr2=0x7feeb4130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:44.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.172+0000 7feeba110700 1 --2- 172.21.15.3:0/2638811469 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feeb409e000 0x7feeb4130660 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7feea800bdc0 tx=0x7feea800d330 comp rx=0 tx=0).stop 2023-10-03T10:13:44.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.172+0000 7feeba110700 1 -- 172.21.15.3:0/2638811469 shutdown_connections 2023-10-03T10:13:44.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.172+0000 7feeba110700 1 --2- 172.21.15.3:0/2638811469 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feeb409e410 0x7feeb4130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.172+0000 7feeba110700 1 --2- 172.21.15.3:0/2638811469 >> v2:172.21.15.3:3300/0 conn(0x7feeb409d6f0 0x7feeb409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.172+0000 7feeba110700 1 --2- 172.21.15.3:0/2638811469 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feeb409e000 0x7feeb4130660 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.172+0000 7feeba110700 1 -- 172.21.15.3:0/2638811469 >> 172.21.15.3:0/2638811469 conn(0x7feeb4066f50 msgr2=0x7feeb409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:44.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.172+0000 7feeba110700 1 -- 172.21.15.3:0/2638811469 shutdown_connections 2023-10-03T10:13:44.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.172+0000 7feeba110700 1 -- 172.21.15.3:0/2638811469 wait complete. 2023-10-03T10:13:44.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.173+0000 7feeba110700 1 Processor -- start 2023-10-03T10:13:44.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.173+0000 7feeba110700 1 -- start start 2023-10-03T10:13:44.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.174+0000 7feeba110700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feeb409d6f0 0x7feeb4122350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:44.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.174+0000 7feeba110700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feeb409e000 0x7feeb4122890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:44.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.174+0000 7feeba110700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feeb409e410 0x7feeb4122dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:44.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.174+0000 7feeba110700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb4125b90 con 0x7feeb409d6f0 2023-10-03T10:13:44.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.174+0000 7feeba110700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb4124070 con 0x7feeb409e410 2023-10-03T10:13:44.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.174+0000 7feeba110700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feeb409c700 con 0x7feeb409e000 2023-10-03T10:13:44.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.174+0000 7feeb37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feeb409d6f0 0x7feeb4122350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:44.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.174+0000 7feeb37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feeb409d6f0 0x7feeb4122350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46636/0 (socket says 172.21.15.3:46636) 2023-10-03T10:13:44.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.174+0000 7feeb37fe700 1 -- 172.21.15.3:0/1480992159 learned_addr learned my addr 172.21.15.3:0/1480992159 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:44.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.174+0000 7feeb2ffd700 1 --2- 172.21.15.3:0/1480992159 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feeb409e000 0x7feeb4122890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:44.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.174+0000 7feeb3fff700 1 --2- 172.21.15.3:0/1480992159 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feeb409e410 0x7feeb4122dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:44.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.174+0000 7feeb37fe700 1 -- 172.21.15.3:0/1480992159 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feeb409e000 msgr2=0x7feeb4122890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:44.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.174+0000 7feeb37fe700 1 --2- 172.21.15.3:0/1480992159 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feeb409e000 0x7feeb4122890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.174+0000 7feeb37fe700 1 -- 172.21.15.3:0/1480992159 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feeb409e410 msgr2=0x7feeb4122dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:44.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.174+0000 7feeb37fe700 1 --2- 172.21.15.3:0/1480992159 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feeb409e410 0x7feeb4122dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.174+0000 7feeb37fe700 1 -- 172.21.15.3:0/1480992159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea40094a0 con 0x7feeb409d6f0 2023-10-03T10:13:44.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.175+0000 7feeb37fe700 1 --2- 172.21.15.3:0/1480992159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feeb409d6f0 0x7feeb4122350 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7feea400ebe0 tx=0x7feea400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:44.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.175+0000 7fee9bfff700 1 -- 172.21.15.3:0/1480992159 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea4009780 con 0x7feeb409d6f0 2023-10-03T10:13:44.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.175+0000 7fee9bfff700 1 -- 172.21.15.3:0/1480992159 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea400c850 con 0x7feeb409d6f0 2023-10-03T10:13:44.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.175+0000 7feeba110700 1 -- 172.21.15.3:0/1480992159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea800ba70 con 0x7feeb409d6f0 2023-10-03T10:13:44.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.175+0000 7fee9bfff700 1 -- 172.21.15.3:0/1480992159 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea400cbe0 con 0x7feeb409d6f0 2023-10-03T10:13:44.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.175+0000 7feeba110700 1 -- 172.21.15.3:0/1480992159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb40a1700 con 0x7feeb409d6f0 2023-10-03T10:13:44.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.176+0000 7feeba110700 1 -- 172.21.15.3:0/1480992159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb41266c0 con 0x7feeb409d6f0 2023-10-03T10:13:44.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.177+0000 7fee9bfff700 1 -- 172.21.15.3:0/1480992159 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feea400cd40 con 0x7feeb409d6f0 2023-10-03T10:13:44.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.177+0000 7fee9bfff700 1 -- 172.21.15.3:0/1480992159 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7feea403ff30 con 0x7feeb409d6f0 2023-10-03T10:13:44.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.181+0000 7fd6fd77a700 1 -- 172.21.15.3:0/4240854862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd6f8122840 con 0x7fd6f809e850 2023-10-03T10:13:44.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.184+0000 7fd6df7fe700 1 -- 172.21.15.3:0/4240854862 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd6e8011b40 con 0x7fd6f809e850 2023-10-03T10:13:44.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.184+0000 7fee9bfff700 1 -- 172.21.15.3:0/1480992159 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feea4024350 con 0x7feeb409d6f0 2023-10-03T10:13:44.185 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:44.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.186+0000 7fd6fd77a700 1 -- 172.21.15.3:0/4240854862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6f809e850 msgr2=0x7fd6f8120510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:44.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.186+0000 7fd6fd77a700 1 --2- 172.21.15.3:0/4240854862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6f809e850 0x7fd6f8120510 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd6e800ec10 tx=0x7fd6e800ef20 comp rx=0 tx=0).stop 2023-10-03T10:13:44.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.187+0000 7fd6fd77a700 1 -- 172.21.15.3:0/4240854862 shutdown_connections 2023-10-03T10:13:44.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.187+0000 7fd6fd77a700 1 --2- 172.21.15.3:0/4240854862 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6f8133b80 0x7fd6f8120f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.187+0000 7fd6fd77a700 1 --2- 172.21.15.3:0/4240854862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6f809e850 0x7fd6f8120510 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.187+0000 7fd6fd77a700 1 --2- 172.21.15.3:0/4240854862 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6f809ffb0 0x7fd6f8120a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.187+0000 7fd6fd77a700 1 -- 172.21.15.3:0/4240854862 >> 172.21.15.3:0/4240854862 conn(0x7fd6f80a3f70 msgr2=0x7fd6f81bafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:44.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.187+0000 7fd6fd77a700 1 -- 172.21.15.3:0/4240854862 shutdown_connections 2023-10-03T10:13:44.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.187+0000 7fd6fd77a700 1 -- 172.21.15.3:0/4240854862 wait complete. 2023-10-03T10:13:44.202 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:44.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.218+0000 7f7e3d77e700 1 Processor -- start 2023-10-03T10:13:44.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.218+0000 7f7e3d77e700 1 -- start start 2023-10-03T10:13:44.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.219+0000 7f7e3d77e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e3809d630 0x7f7e380a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:44.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.219+0000 7f7e3d77e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e38138d80 0x7f7e3813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:44.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.219+0000 7f7e3d77e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e380a2a70 con 0x7f7e380a0a30 2023-10-03T10:13:44.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.219+0000 7f7e3d77e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e380a01d0 con 0x7f7e38138d80 2023-10-03T10:13:44.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.219+0000 7f7e3d77e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3809e2b0 con 0x7f7e3809d630 2023-10-03T10:13:44.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.219+0000 7f7e367fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e3809d630 0x7f7e380a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:44.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.220+0000 7f7e36ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7e380a0a30 0x7f7e380a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45608/0 (socket says 172.21.15.3:45608) 2023-10-03T10:13:44.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.220+0000 7f7e367fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e3809d630 0x7f7e380a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55548/0 (socket says 172.21.15.3:55548) 2023-10-03T10:13:44.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.220+0000 7f7e36ffd700 1 -- 172.21.15.3:0/275090075 learned_addr learned my addr 172.21.15.3:0/275090075 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:44.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.220+0000 7f7e377fe700 1 --2- 172.21.15.3:0/275090075 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e38138d80 0x7f7e3813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:44.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.220+0000 7f7e367fc700 1 -- 172.21.15.3:0/275090075 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e38138d80 msgr2=0x7f7e3813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:44.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.220+0000 7f7e367fc700 1 --2- 172.21.15.3:0/275090075 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e38138d80 0x7f7e3813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.220+0000 7f7e367fc700 1 -- 172.21.15.3:0/275090075 >> v1:172.21.15.3:6789/0 conn(0x7f7e380a0a30 legacy=0x7f7e380a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:44.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.220+0000 7f7e367fc700 1 -- 172.21.15.3:0/275090075 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3809dc50 con 0x7f7e3809d630 2023-10-03T10:13:44.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.220+0000 7f7e35ffb700 1 -- 172.21.15.3:0/275090075 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2778551493 0 0) 0x7f7e380a2a70 con 0x7f7e380a0a30 2023-10-03T10:13:44.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.220+0000 7f7e367fc700 1 --2- 172.21.15.3:0/275090075 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e3809d630 0x7f7e380a1510 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c009870 tx=0x7f7e2c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e9865bcd41d80ab server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:44.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.220+0000 7f7e35ffb700 1 -- 172.21.15.3:0/275090075 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c011830 con 0x7f7e3809d630 2023-10-03T10:13:44.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.220+0000 7f7e35ffb700 1 -- 172.21.15.3:0/275090075 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2c011990 con 0x7f7e3809d630 2023-10-03T10:13:44.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.220+0000 7f7e35ffb700 1 -- 172.21.15.3:0/275090075 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c011d20 con 0x7f7e3809d630 2023-10-03T10:13:44.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.221+0000 7f7e3d77e700 1 -- 172.21.15.3:0/275090075 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e3809d630 msgr2=0x7f7e380a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:44.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.221+0000 7f7e3d77e700 1 --2- 172.21.15.3:0/275090075 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e3809d630 0x7f7e380a1510 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c009870 tx=0x7f7e2c009b80 comp rx=0 tx=0).stop 2023-10-03T10:13:44.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.222+0000 7f7e3d77e700 1 -- 172.21.15.3:0/275090075 shutdown_connections 2023-10-03T10:13:44.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.222+0000 7f7e3d77e700 1 --2- 172.21.15.3:0/275090075 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e38138d80 0x7f7e3813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.222+0000 7f7e3d77e700 1 --2- 172.21.15.3:0/275090075 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e3809d630 0x7f7e380a1510 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.222+0000 7f7e3d77e700 1 -- 172.21.15.3:0/275090075 >> 172.21.15.3:0/275090075 conn(0x7f7e380a3050 msgr2=0x7f7e380a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:44.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.222+0000 7f7e3d77e700 1 -- 172.21.15.3:0/275090075 shutdown_connections 2023-10-03T10:13:44.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.222+0000 7f7e3d77e700 1 -- 172.21.15.3:0/275090075 wait complete. 2023-10-03T10:13:44.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.223+0000 7f7e3d77e700 1 Processor -- start 2023-10-03T10:13:44.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.223+0000 7f7e3d77e700 1 -- start start 2023-10-03T10:13:44.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.223+0000 7f7e3d77e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e3809d630 0x7f7e38133950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:44.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.223+0000 7f7e3d77e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e380a0a30 0x7f7e38133e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:44.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.223+0000 7f7e3d77e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e38138d80 0x7f7e381343d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:44.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.223+0000 7f7e3d77e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e38135af0 con 0x7f7e38138d80 2023-10-03T10:13:44.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.223+0000 7f7e3d77e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e38134db0 con 0x7f7e3809d630 2023-10-03T10:13:44.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.223+0000 7f7e3d77e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e380a0410 con 0x7f7e380a0a30 2023-10-03T10:13:44.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.223+0000 7f7e36ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e3809d630 0x7f7e38133950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:44.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.223+0000 7f7e377fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e38138d80 0x7f7e381343d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:44.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.223+0000 7f7e36ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e3809d630 0x7f7e38133950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46162/0 (socket says 172.21.15.3:46162) 2023-10-03T10:13:44.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.223+0000 7f7e36ffd700 1 -- 172.21.15.3:0/1800697507 learned_addr learned my addr 172.21.15.3:0/1800697507 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:44.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.224+0000 7f7e367fc700 1 --2- 172.21.15.3:0/1800697507 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e380a0a30 0x7f7e38133e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:44.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.224+0000 7f7e36ffd700 1 -- 172.21.15.3:0/1800697507 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e380a0a30 msgr2=0x7f7e38133e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:44.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.224+0000 7f7e36ffd700 1 --2- 172.21.15.3:0/1800697507 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e380a0a30 0x7f7e38133e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.224+0000 7f7e36ffd700 1 -- 172.21.15.3:0/1800697507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e38138d80 msgr2=0x7f7e381343d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:44.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.224+0000 7f7e36ffd700 1 --2- 172.21.15.3:0/1800697507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e38138d80 0x7f7e381343d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.224+0000 7f7e36ffd700 1 -- 172.21.15.3:0/1800697507 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2c0094a0 con 0x7f7e3809d630 2023-10-03T10:13:44.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.224+0000 7f7e377fe700 1 --2- 172.21.15.3:0/1800697507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e38138d80 0x7f7e381343d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:44.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.224+0000 7f7e36ffd700 1 --2- 172.21.15.3:0/1800697507 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e3809d630 0x7f7e38133950 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800c550 tx=0x7f7e2800c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:44.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.224+0000 7f7e1f7fe700 1 -- 172.21.15.3:0/1800697507 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e28004c80 con 0x7f7e3809d630 2023-10-03T10:13:44.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.224+0000 7f7e1f7fe700 1 -- 172.21.15.3:0/1800697507 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e28004e00 con 0x7f7e3809d630 2023-10-03T10:13:44.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.224+0000 7f7e3d77e700 1 -- 172.21.15.3:0/1800697507 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e380a4bd0 con 0x7f7e3809d630 2023-10-03T10:13:44.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.224+0000 7f7e1f7fe700 1 -- 172.21.15.3:0/1800697507 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e28007310 con 0x7f7e3809d630 2023-10-03T10:13:44.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.224+0000 7f7e3d77e700 1 -- 172.21.15.3:0/1800697507 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e38135cc0 con 0x7f7e3809d630 2023-10-03T10:13:44.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.225+0000 7f7e1f7fe700 1 -- 172.21.15.3:0/1800697507 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7e28007470 con 0x7f7e3809d630 2023-10-03T10:13:44.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.225+0000 7f7e1f7fe700 1 -- 172.21.15.3:0/1800697507 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7e2803f3b0 con 0x7f7e3809d630 2023-10-03T10:13:44.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.226+0000 7f7e3d77e700 1 -- 172.21.15.3:0/1800697507 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e380747f0 con 0x7f7e3809d630 2023-10-03T10:13:44.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.231+0000 7f7e1f7fe700 1 -- 172.21.15.3:0/1800697507 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7e280123b0 con 0x7f7e3809d630 2023-10-03T10:13:44.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.261+0000 7f4d6183b700 1 Processor -- start 2023-10-03T10:13:44.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.261+0000 7f4d6183b700 1 -- start start 2023-10-03T10:13:44.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.261+0000 7f4d6183b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d5c09d630 0x7f4d5c0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:44.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.261+0000 7f4d6183b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d5c138d80 0x7f4d5c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:44.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.261+0000 7f4d6183b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d5c0a2a70 con 0x7f4d5c0a0a30 2023-10-03T10:13:44.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.261+0000 7f4d6183b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d5c0a01d0 con 0x7f4d5c138d80 2023-10-03T10:13:44.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.261+0000 7f4d6183b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c09e2b0 con 0x7f4d5c09d630 2023-10-03T10:13:44.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.262+0000 7f4d5a7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d5c09d630 0x7f4d5c0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:44.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.262+0000 7f4d5b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d5c138d80 0x7f4d5c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:44.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.262+0000 7f4d5a7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d5c09d630 0x7f4d5c0a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46174/0 (socket says 172.21.15.3:46174) 2023-10-03T10:13:44.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.262+0000 7f4d5a7fc700 1 -- 172.21.15.3:0/4100853363 learned_addr learned my addr 172.21.15.3:0/4100853363 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:44.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.262+0000 7f4d5a7fc700 1 -- 172.21.15.3:0/4100853363 >> v2:172.21.15.3:3300/0 conn(0x7f4d5c138d80 msgr2=0x7f4d5c13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:44.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.262+0000 7f4d5a7fc700 1 --2- 172.21.15.3:0/4100853363 >> v2:172.21.15.3:3300/0 conn(0x7f4d5c138d80 0x7f4d5c13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.262+0000 7f4d5a7fc700 1 -- 172.21.15.3:0/4100853363 >> v1:172.21.15.3:6789/0 conn(0x7f4d5c0a0a30 legacy=0x7f4d5c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:44.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.262+0000 7f4d5a7fc700 1 -- 172.21.15.3:0/4100853363 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5c09dc50 con 0x7f4d5c09d630 2023-10-03T10:13:44.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.262+0000 7f4d59ffb700 1 -- 172.21.15.3:0/4100853363 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1348257898 0 0) 0x7f4d5c0a2a70 con 0x7f4d5c0a0a30 2023-10-03T10:13:44.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.262+0000 7f4d5b7fe700 1 --2- 172.21.15.3:0/4100853363 >> v2:172.21.15.3:3300/0 conn(0x7f4d5c138d80 0x7f4d5c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:44.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.262+0000 7f4d5a7fc700 1 --2- 172.21.15.3:0/4100853363 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d5c09d630 0x7f4d5c0a1510 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4d500098f0 tx=0x7f4d50009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a590d8ce7ad30e36 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:44.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.262+0000 7f4d59ffb700 1 -- 172.21.15.3:0/4100853363 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d50011830 con 0x7f4d5c09d630 2023-10-03T10:13:44.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.262+0000 7f4d59ffb700 1 -- 172.21.15.3:0/4100853363 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d50011990 con 0x7f4d5c09d630 2023-10-03T10:13:44.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.262+0000 7f4d59ffb700 1 -- 172.21.15.3:0/4100853363 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d50011d20 con 0x7f4d5c09d630 2023-10-03T10:13:44.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.263+0000 7f4d6183b700 1 -- 172.21.15.3:0/4100853363 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d5c09d630 msgr2=0x7f4d5c0a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:44.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.263+0000 7f4d6183b700 1 --2- 172.21.15.3:0/4100853363 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d5c09d630 0x7f4d5c0a1510 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4d500098f0 tx=0x7f4d50009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:44.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.263+0000 7f4d6183b700 1 -- 172.21.15.3:0/4100853363 shutdown_connections 2023-10-03T10:13:44.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.263+0000 7f4d6183b700 1 --2- 172.21.15.3:0/4100853363 >> v2:172.21.15.3:3300/0 conn(0x7f4d5c138d80 0x7f4d5c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.263+0000 7f4d6183b700 1 --2- 172.21.15.3:0/4100853363 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d5c09d630 0x7f4d5c0a1510 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.263+0000 7f4d6183b700 1 -- 172.21.15.3:0/4100853363 >> 172.21.15.3:0/4100853363 conn(0x7f4d5c0a3050 msgr2=0x7f4d5c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:44.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.263+0000 7f4d6183b700 1 -- 172.21.15.3:0/4100853363 shutdown_connections 2023-10-03T10:13:44.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.263+0000 7f4d6183b700 1 -- 172.21.15.3:0/4100853363 wait complete. 2023-10-03T10:13:44.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.265+0000 7f4d6183b700 1 Processor -- start 2023-10-03T10:13:44.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.265+0000 7f4d6183b700 1 -- start start 2023-10-03T10:13:44.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.266+0000 7f4d6183b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d5c09d630 0x7f4d5c133790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:44.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.266+0000 7f4d6183b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d5c0a0a30 0x7f4d5c133cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:44.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.266+0000 7f4d6183b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d5c138d80 0x7f4d5c134210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:44.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.266+0000 7f4d6183b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c134c60 con 0x7f4d5c138d80 2023-10-03T10:13:44.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.266+0000 7f4d6183b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c1365d0 con 0x7f4d5c09d630 2023-10-03T10:13:44.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.266+0000 7f4d6183b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c137220 con 0x7f4d5c0a0a30 2023-10-03T10:13:44.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.266+0000 7f4d5affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d5c09d630 0x7f4d5c133790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:44.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.266+0000 7f4d5affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d5c09d630 0x7f4d5c133790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46182/0 (socket says 172.21.15.3:46182) 2023-10-03T10:13:44.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.266+0000 7f4d5affd700 1 -- 172.21.15.3:0/229447074 learned_addr learned my addr 172.21.15.3:0/229447074 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:44.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.266+0000 7f4d5a7fc700 1 --2- 172.21.15.3:0/229447074 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d5c0a0a30 0x7f4d5c133cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:44.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.266+0000 7f4d5b7fe700 1 --2- 172.21.15.3:0/229447074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d5c138d80 0x7f4d5c134210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:44.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.266+0000 7f4d5affd700 1 -- 172.21.15.3:0/229447074 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d5c0a0a30 msgr2=0x7f4d5c133cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:44.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.266+0000 7f4d5affd700 1 --2- 172.21.15.3:0/229447074 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d5c0a0a30 0x7f4d5c133cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.266+0000 7f4d5affd700 1 -- 172.21.15.3:0/229447074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d5c138d80 msgr2=0x7f4d5c134210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:44.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.266+0000 7f4d5affd700 1 --2- 172.21.15.3:0/229447074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d5c138d80 0x7f4d5c134210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.266+0000 7f4d5affd700 1 -- 172.21.15.3:0/229447074 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d54009a70 con 0x7f4d5c09d630 2023-10-03T10:13:44.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.266+0000 7f4d5a7fc700 1 --2- 172.21.15.3:0/229447074 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d5c0a0a30 0x7f4d5c133cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:44.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.266+0000 7f4d5b7fe700 1 --2- 172.21.15.3:0/229447074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d5c138d80 0x7f4d5c134210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:44.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.267+0000 7f4d5affd700 1 --2- 172.21.15.3:0/229447074 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d5c09d630 0x7f4d5c133790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c001eb0 tx=0x7f4d4c00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:44.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.267+0000 7f4d43fff700 1 -- 172.21.15.3:0/229447074 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c0054e0 con 0x7f4d5c09d630 2023-10-03T10:13:44.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.267+0000 7f4d43fff700 1 -- 172.21.15.3:0/229447074 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d4c003680 con 0x7f4d5c09d630 2023-10-03T10:13:44.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.267+0000 7f4d6183b700 1 -- 172.21.15.3:0/229447074 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d50009550 con 0x7f4d5c09d630 2023-10-03T10:13:44.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.267+0000 7f4d43fff700 1 -- 172.21.15.3:0/229447074 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c0039f0 con 0x7f4d5c09d630 2023-10-03T10:13:44.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.267+0000 7f4d6183b700 1 -- 172.21.15.3:0/229447074 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d5c0a0470 con 0x7f4d5c09d630 2023-10-03T10:13:44.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.268+0000 7f4d6183b700 1 -- 172.21.15.3:0/229447074 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d5c137e00 con 0x7f4d5c09d630 2023-10-03T10:13:44.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.268+0000 7f4d43fff700 1 -- 172.21.15.3:0/229447074 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4d4c003b50 con 0x7f4d5c09d630 2023-10-03T10:13:44.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.268+0000 7f4d43fff700 1 -- 172.21.15.3:0/229447074 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4d4c040a70 con 0x7f4d5c09d630 2023-10-03T10:13:44.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.273+0000 7f4d43fff700 1 -- 172.21.15.3:0/229447074 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4d4c044060 con 0x7f4d5c09d630 2023-10-03T10:13:44.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.437+0000 7feeba110700 1 -- 172.21.15.3:0/1480992159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7feeb4074ea0 con 0x7feeb409d6f0 2023-10-03T10:13:44.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.437+0000 7fee9bfff700 1 -- 172.21.15.3:0/1480992159 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feea40123d0 con 0x7feeb409d6f0 2023-10-03T10:13:44.438 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:44.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.440+0000 7feeba110700 1 -- 172.21.15.3:0/1480992159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feeb409d6f0 msgr2=0x7feeb4122350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:44.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.440+0000 7feeba110700 1 --2- 172.21.15.3:0/1480992159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feeb409d6f0 0x7feeb4122350 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7feea400ebe0 tx=0x7feea400eef0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.440+0000 7feeba110700 1 -- 172.21.15.3:0/1480992159 shutdown_connections 2023-10-03T10:13:44.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.440+0000 7feeba110700 1 --2- 172.21.15.3:0/1480992159 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feeb409e410 0x7feeb4122dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.440+0000 7feeba110700 1 --2- 172.21.15.3:0/1480992159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feeb409d6f0 0x7feeb4122350 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.440+0000 7feeba110700 1 --2- 172.21.15.3:0/1480992159 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feeb409e000 0x7feeb4122890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.440+0000 7feeba110700 1 -- 172.21.15.3:0/1480992159 >> 172.21.15.3:0/1480992159 conn(0x7feeb4066f50 msgr2=0x7feeb4130920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:44.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.440+0000 7feeba110700 1 -- 172.21.15.3:0/1480992159 shutdown_connections 2023-10-03T10:13:44.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.440+0000 7feeba110700 1 -- 172.21.15.3:0/1480992159 wait complete. 2023-10-03T10:13:44.454 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:44.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.473+0000 7f7e3d77e700 1 -- 172.21.15.3:0/1800697507 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7e38072980 con 0x7f7e3809d630 2023-10-03T10:13:44.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.473+0000 7f7e1f7fe700 1 -- 172.21.15.3:0/1800697507 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7e28011360 con 0x7f7e3809d630 2023-10-03T10:13:44.473 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:44.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.475+0000 7f7e3d77e700 1 -- 172.21.15.3:0/1800697507 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e3809d630 msgr2=0x7f7e38133950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:44.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.475+0000 7f7e3d77e700 1 --2- 172.21.15.3:0/1800697507 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e3809d630 0x7f7e38133950 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800c550 tx=0x7f7e2800c860 comp rx=0 tx=0).stop 2023-10-03T10:13:44.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.476+0000 7f7e3d77e700 1 -- 172.21.15.3:0/1800697507 shutdown_connections 2023-10-03T10:13:44.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.476+0000 7f7e3d77e700 1 --2- 172.21.15.3:0/1800697507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e38138d80 0x7f7e381343d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.476+0000 7f7e3d77e700 1 --2- 172.21.15.3:0/1800697507 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e3809d630 0x7f7e38133950 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.476+0000 7f7e3d77e700 1 --2- 172.21.15.3:0/1800697507 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e380a0a30 0x7f7e38133e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.476+0000 7f7e3d77e700 1 -- 172.21.15.3:0/1800697507 >> 172.21.15.3:0/1800697507 conn(0x7f7e380a3050 msgr2=0x7f7e3809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:44.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.476+0000 7f7e3d77e700 1 -- 172.21.15.3:0/1800697507 shutdown_connections 2023-10-03T10:13:44.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.476+0000 7f7e3d77e700 1 -- 172.21.15.3:0/1800697507 wait complete. 2023-10-03T10:13:44.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.485+0000 7f4d6183b700 1 -- 172.21.15.3:0/229447074 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4d5c136ad0 con 0x7f4d5c09d630 2023-10-03T10:13:44.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.485+0000 7f4d43fff700 1 -- 172.21.15.3:0/229447074 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4d4c017070 con 0x7f4d5c09d630 2023-10-03T10:13:44.485 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.487+0000 7f4d6183b700 1 -- 172.21.15.3:0/229447074 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d5c09d630 msgr2=0x7f4d5c133790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.487+0000 7f4d6183b700 1 --2- 172.21.15.3:0/229447074 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d5c09d630 0x7f4d5c133790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c001eb0 tx=0x7f4d4c00c800 comp rx=0 tx=0).stop 2023-10-03T10:13:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.488+0000 7f4d6183b700 1 -- 172.21.15.3:0/229447074 shutdown_connections 2023-10-03T10:13:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.488+0000 7f4d6183b700 1 --2- 172.21.15.3:0/229447074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d5c138d80 0x7f4d5c134210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.488+0000 7f4d6183b700 1 --2- 172.21.15.3:0/229447074 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d5c09d630 0x7f4d5c133790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.488+0000 7f4d6183b700 1 --2- 172.21.15.3:0/229447074 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d5c0a0a30 0x7f4d5c133cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.488+0000 7f4d6183b700 1 -- 172.21.15.3:0/229447074 >> 172.21.15.3:0/229447074 conn(0x7f4d5c0a3050 msgr2=0x7f4d5c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.488+0000 7f4d6183b700 1 -- 172.21.15.3:0/229447074 shutdown_connections 2023-10-03T10:13:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:44.488+0000 7f4d6183b700 1 -- 172.21.15.3:0/229447074 wait complete. 2023-10-03T10:13:44.490 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:44.502 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:45.203 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:45.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.352+0000 7f0e07db5700 1 Processor -- start 2023-10-03T10:13:45.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.352+0000 7f0e07db5700 1 -- start start 2023-10-03T10:13:45.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.353+0000 7f0e07db5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e0006a770 0x7f0e0009d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:45.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.353+0000 7f0e07db5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0e0009dc30 0x7f0e0009e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:45.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.353+0000 7f0e07db5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e0009a810 con 0x7f0e0013c1e0 2023-10-03T10:13:45.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.353+0000 7f0e07db5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e0006ab40 con 0x7f0e0009dc30 2023-10-03T10:13:45.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.353+0000 7f0e07db5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0009ebc0 con 0x7f0e0006a770 2023-10-03T10:13:45.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.353+0000 7f0e05350700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0e0009dc30 0x7f0e0009e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.353+0000 7f0e05b51700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e0006a770 0x7f0e0009d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.353+0000 7f0e05b51700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e0006a770 0x7f0e0009d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55572/0 (socket says 172.21.15.3:55572) 2023-10-03T10:13:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.353+0000 7f0e05350700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0e0009dc30 0x7f0e0009e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46684/0 (socket says 172.21.15.3:46684) 2023-10-03T10:13:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.353+0000 7f0e05b51700 1 -- 172.21.15.3:0/1322848114 learned_addr learned my addr 172.21.15.3:0/1322848114 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.353+0000 7f0e05b51700 1 -- 172.21.15.3:0/1322848114 >> v2:172.21.15.3:3300/0 conn(0x7f0e0009dc30 msgr2=0x7f0e0009e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.353+0000 7f0e05b51700 1 --2- 172.21.15.3:0/1322848114 >> v2:172.21.15.3:3300/0 conn(0x7f0e0009dc30 0x7f0e0009e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.353+0000 7f0e05b51700 1 -- 172.21.15.3:0/1322848114 >> v1:172.21.15.3:6789/0 conn(0x7f0e0013c1e0 legacy=0x7f0e0013e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.353+0000 7f0e05b51700 1 -- 172.21.15.3:0/1322848114 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0009e920 con 0x7f0e0006a770 2023-10-03T10:13:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.353+0000 7f0e05350700 1 --2- 172.21.15.3:0/1322848114 >> v2:172.21.15.3:3300/0 conn(0x7f0e0009dc30 0x7f0e0009e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.354+0000 7f0e05b51700 1 --2- 172.21.15.3:0/1322848114 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e0006a770 0x7f0e0009d6f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc009920 tx=0x7f0dfc009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98877884e9d74d72 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.354+0000 7f0e04b4f700 1 -- 172.21.15.3:0/1322848114 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc011830 con 0x7f0e0006a770 2023-10-03T10:13:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.354+0000 7f0e04b4f700 1 -- 172.21.15.3:0/1322848114 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dfc011990 con 0x7f0e0006a770 2023-10-03T10:13:45.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.354+0000 7f0e04b4f700 1 -- 172.21.15.3:0/1322848114 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc011d20 con 0x7f0e0006a770 2023-10-03T10:13:45.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.354+0000 7f0e07db5700 1 -- 172.21.15.3:0/1322848114 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e0006a770 msgr2=0x7f0e0009d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:45.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.354+0000 7f0e07db5700 1 --2- 172.21.15.3:0/1322848114 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e0006a770 0x7f0e0009d6f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc009920 tx=0x7f0dfc009c30 comp rx=0 tx=0).stop 2023-10-03T10:13:45.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.354+0000 7f0e07db5700 1 -- 172.21.15.3:0/1322848114 shutdown_connections 2023-10-03T10:13:45.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.354+0000 7f0e07db5700 1 --2- 172.21.15.3:0/1322848114 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e0006a770 0x7f0e0009d6f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.354+0000 7f0e07db5700 1 --2- 172.21.15.3:0/1322848114 >> v2:172.21.15.3:3300/0 conn(0x7f0e0009dc30 0x7f0e0009e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.354+0000 7f0e07db5700 1 -- 172.21.15.3:0/1322848114 >> 172.21.15.3:0/1322848114 conn(0x7f0e00066f50 msgr2=0x7f0e0009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:45.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.355+0000 7f0e07db5700 1 -- 172.21.15.3:0/1322848114 shutdown_connections 2023-10-03T10:13:45.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.355+0000 7f0e07db5700 1 -- 172.21.15.3:0/1322848114 wait complete. 2023-10-03T10:13:45.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.355+0000 7f0e07db5700 1 Processor -- start 2023-10-03T10:13:45.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.356+0000 7f0e07db5700 1 -- start start 2023-10-03T10:13:45.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.356+0000 7f0e07db5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e0006a770 0x7f0e00127d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:45.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.356+0000 7f0e07db5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0e0009dc30 0x7f0e00128290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:45.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.356+0000 7f0e07db5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e0013c1e0 0x7f0e001cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:45.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.356+0000 7f0e07db5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e00129b60 con 0x7f0e0006a770 2023-10-03T10:13:45.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.356+0000 7f0e07db5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e00127800 con 0x7f0e0009dc30 2023-10-03T10:13:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.356+0000 7f0e07db5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0012a760 con 0x7f0e0013c1e0 2023-10-03T10:13:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.356+0000 7f0e05b51700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e0006a770 0x7f0e00127d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.356+0000 7f0e05350700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0e0009dc30 0x7f0e00128290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.356+0000 7f0e05b51700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e0006a770 0x7f0e00127d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46696/0 (socket says 172.21.15.3:46696) 2023-10-03T10:13:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.356+0000 7f0e05b51700 1 -- 172.21.15.3:0/1322926030 learned_addr learned my addr 172.21.15.3:0/1322926030 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.356+0000 7f0e06352700 1 --2- 172.21.15.3:0/1322926030 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e0013c1e0 0x7f0e001cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.356+0000 7f0e05b51700 1 -- 172.21.15.3:0/1322926030 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e0013c1e0 msgr2=0x7f0e001cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.356+0000 7f0e05b51700 1 --2- 172.21.15.3:0/1322926030 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e0013c1e0 0x7f0e001cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.356+0000 7f0e05b51700 1 -- 172.21.15.3:0/1322926030 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0e0009dc30 msgr2=0x7f0e00128290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.356+0000 7f0e05b51700 1 --2- 172.21.15.3:0/1322926030 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0e0009dc30 0x7f0e00128290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.356+0000 7f0e05b51700 1 -- 172.21.15.3:0/1322926030 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df00094a0 con 0x7f0e0006a770 2023-10-03T10:13:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.357+0000 7f0e05b51700 1 --2- 172.21.15.3:0/1322926030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e0006a770 0x7f0e00127d50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc000c00 tx=0x7f0dfc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.357+0000 7f0df67fc700 1 -- 172.21.15.3:0/1322926030 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc004300 con 0x7f0e0006a770 2023-10-03T10:13:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.357+0000 7f0df67fc700 1 -- 172.21.15.3:0/1322926030 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dfc01f070 con 0x7f0e0006a770 2023-10-03T10:13:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.357+0000 7f0e07db5700 1 -- 172.21.15.3:0/1322926030 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dfc009550 con 0x7f0e0006a770 2023-10-03T10:13:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.357+0000 7f0df67fc700 1 -- 172.21.15.3:0/1322926030 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc0036a0 con 0x7f0e0006a770 2023-10-03T10:13:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.357+0000 7f0e07db5700 1 -- 172.21.15.3:0/1322926030 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e0012b490 con 0x7f0e0006a770 2023-10-03T10:13:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.358+0000 7f0df67fc700 1 -- 172.21.15.3:0/1322926030 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0dfc021030 con 0x7f0e0006a770 2023-10-03T10:13:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.358+0000 7f0df67fc700 1 -- 172.21.15.3:0/1322926030 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0dfc040860 con 0x7f0e0006a770 2023-10-03T10:13:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.358+0000 7f0e07db5700 1 -- 172.21.15.3:0/1322926030 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e00074ea0 con 0x7f0e0006a770 2023-10-03T10:13:45.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.363+0000 7f0df67fc700 1 -- 172.21.15.3:0/1322926030 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0dfc015080 con 0x7f0e0006a770 2023-10-03T10:13:45.455 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:45.490 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:45.502 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:45.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.574+0000 7f0e07db5700 1 -- 172.21.15.3:0/1322926030 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0e0006c360 con 0x7f0e0006a770 2023-10-03T10:13:45.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.574+0000 7f0df67fc700 1 -- 172.21.15.3:0/1322926030 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0dfc003800 con 0x7f0e0006a770 2023-10-03T10:13:45.575 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:45.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.578+0000 7f0e07db5700 1 -- 172.21.15.3:0/1322926030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e0006a770 msgr2=0x7f0e00127d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:45.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.578+0000 7f0e07db5700 1 --2- 172.21.15.3:0/1322926030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e0006a770 0x7f0e00127d50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc000c00 tx=0x7f0dfc004750 comp rx=0 tx=0).stop 2023-10-03T10:13:45.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.578+0000 7f0e07db5700 1 -- 172.21.15.3:0/1322926030 shutdown_connections 2023-10-03T10:13:45.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.578+0000 7f0e07db5700 1 --2- 172.21.15.3:0/1322926030 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e0013c1e0 0x7f0e001cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.578+0000 7f0e07db5700 1 --2- 172.21.15.3:0/1322926030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e0006a770 0x7f0e00127d50 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.578+0000 7f0e07db5700 1 --2- 172.21.15.3:0/1322926030 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0e0009dc30 0x7f0e00128290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.578+0000 7f0e07db5700 1 -- 172.21.15.3:0/1322926030 >> 172.21.15.3:0/1322926030 conn(0x7f0e00066f50 msgr2=0x7f0e00070c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:45.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.578+0000 7f0e07db5700 1 -- 172.21.15.3:0/1322926030 shutdown_connections 2023-10-03T10:13:45.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.578+0000 7f0e07db5700 1 -- 172.21.15.3:0/1322926030 wait complete. 2023-10-03T10:13:45.597 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:45.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.600+0000 7fdf2caf5700 1 Processor -- start 2023-10-03T10:13:45.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.600+0000 7fdf2caf5700 1 -- start start 2023-10-03T10:13:45.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.601+0000 7fdf2caf5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf2809e1d0 0x7fdf28138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:45.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.601+0000 7fdf2caf5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf28139170 0x7fdf2813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:45.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.601+0000 7fdf2caf5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf2809b120 con 0x7fdf2809d6f0 2023-10-03T10:13:45.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.601+0000 7fdf2caf5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2806b460 con 0x7fdf2809e1d0 2023-10-03T10:13:45.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.601+0000 7fdf2caf5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2809a7e0 con 0x7fdf28139170 2023-10-03T10:13:45.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.601+0000 7fdf25d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf2809e1d0 0x7fdf28138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:45.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.601+0000 7fdf25d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf2809e1d0 0x7fdf28138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46196/0 (socket says 172.21.15.3:46196) 2023-10-03T10:13:45.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.601+0000 7fdf25d9b700 1 -- 172.21.15.3:0/3721166050 learned_addr learned my addr 172.21.15.3:0/3721166050 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:45.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.601+0000 7fdf26d9d700 1 --2- 172.21.15.3:0/3721166050 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf28139170 0x7fdf2813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:45.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.601+0000 7fdf25d9b700 1 -- 172.21.15.3:0/3721166050 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf28139170 msgr2=0x7fdf2813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:45.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.601+0000 7fdf25d9b700 1 --2- 172.21.15.3:0/3721166050 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf28139170 0x7fdf2813b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.601+0000 7fdf25d9b700 1 -- 172.21.15.3:0/3721166050 >> v1:172.21.15.3:6789/0 conn(0x7fdf2809d6f0 legacy=0x7fdf2809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:45.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.601+0000 7fdf25d9b700 1 -- 172.21.15.3:0/3721166050 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2809a500 con 0x7fdf2809e1d0 2023-10-03T10:13:45.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.601+0000 7fdf2559a700 1 -- 172.21.15.3:0/3721166050 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 982047727 0 0) 0x7fdf2809b120 con 0x7fdf2809d6f0 2023-10-03T10:13:45.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.601+0000 7fdf26d9d700 1 --2- 172.21.15.3:0/3721166050 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf28139170 0x7fdf2813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:45.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.602+0000 7fdf25d9b700 1 --2- 172.21.15.3:0/3721166050 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf2809e1d0 0x7fdf28138c30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009920 tx=0x7fdf1c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b508c177cf8d28d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:45.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.602+0000 7fdf2559a700 1 -- 172.21.15.3:0/3721166050 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c011830 con 0x7fdf2809e1d0 2023-10-03T10:13:45.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.602+0000 7fdf2559a700 1 -- 172.21.15.3:0/3721166050 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1c011990 con 0x7fdf2809e1d0 2023-10-03T10:13:45.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.602+0000 7fdf2559a700 1 -- 172.21.15.3:0/3721166050 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c011d20 con 0x7fdf2809e1d0 2023-10-03T10:13:45.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.603+0000 7fdf2caf5700 1 -- 172.21.15.3:0/3721166050 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf2809e1d0 msgr2=0x7fdf28138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:45.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.603+0000 7fdf2caf5700 1 --2- 172.21.15.3:0/3721166050 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf2809e1d0 0x7fdf28138c30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009920 tx=0x7fdf1c009c30 comp rx=0 tx=0).stop 2023-10-03T10:13:45.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.603+0000 7fdf2caf5700 1 -- 172.21.15.3:0/3721166050 shutdown_connections 2023-10-03T10:13:45.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.603+0000 7fdf2caf5700 1 --2- 172.21.15.3:0/3721166050 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf28139170 0x7fdf2813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.603+0000 7fdf2caf5700 1 --2- 172.21.15.3:0/3721166050 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf2809e1d0 0x7fdf28138c30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.603+0000 7fdf2caf5700 1 -- 172.21.15.3:0/3721166050 >> 172.21.15.3:0/3721166050 conn(0x7fdf28066f50 msgr2=0x7fdf2809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:45.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.603+0000 7fdf2caf5700 1 -- 172.21.15.3:0/3721166050 shutdown_connections 2023-10-03T10:13:45.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.603+0000 7fdf2caf5700 1 -- 172.21.15.3:0/3721166050 wait complete. 2023-10-03T10:13:45.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.604+0000 7fdf2caf5700 1 Processor -- start 2023-10-03T10:13:45.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.604+0000 7fdf2caf5700 1 -- start start 2023-10-03T10:13:45.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.605+0000 7fdf2caf5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf2809d6f0 0x7fdf2811e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:45.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.605+0000 7fdf2caf5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf2809e1d0 0x7fdf2811ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:45.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.605+0000 7fdf2caf5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf28139170 0x7fdf2811f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:45.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.605+0000 7fdf2caf5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf281222b0 con 0x7fdf28139170 2023-10-03T10:13:45.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.605+0000 7fdf2caf5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2809c710 con 0x7fdf2809e1d0 2023-10-03T10:13:45.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.605+0000 7fdf2caf5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf280a2080 con 0x7fdf2809d6f0 2023-10-03T10:13:45.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.605+0000 7fdf25d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf2809e1d0 0x7fdf2811ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:45.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.605+0000 7fdf2659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf2809d6f0 0x7fdf2811e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:45.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.605+0000 7fdf25d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf2809e1d0 0x7fdf2811ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46202/0 (socket says 172.21.15.3:46202) 2023-10-03T10:13:45.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.605+0000 7fdf25d9b700 1 -- 172.21.15.3:0/2280163715 learned_addr learned my addr 172.21.15.3:0/2280163715 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:45.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.605+0000 7fdf26d9d700 1 --2- 172.21.15.3:0/2280163715 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf28139170 0x7fdf2811f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:45.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.605+0000 7fdf25d9b700 1 -- 172.21.15.3:0/2280163715 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf2809d6f0 msgr2=0x7fdf2811e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:45.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.605+0000 7fdf25d9b700 1 --2- 172.21.15.3:0/2280163715 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf2809d6f0 0x7fdf2811e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.605+0000 7fdf25d9b700 1 -- 172.21.15.3:0/2280163715 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf28139170 msgr2=0x7fdf2811f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:45.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.605+0000 7fdf25d9b700 1 --2- 172.21.15.3:0/2280163715 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf28139170 0x7fdf2811f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.605+0000 7fdf25d9b700 1 -- 172.21.15.3:0/2280163715 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf20009a70 con 0x7fdf2809e1d0 2023-10-03T10:13:45.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.605+0000 7fdf2659c700 1 --2- 172.21.15.3:0/2280163715 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf2809d6f0 0x7fdf2811e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:45.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.605+0000 7fdf26d9d700 1 --2- 172.21.15.3:0/2280163715 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf28139170 0x7fdf2811f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:45.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.606+0000 7fdf25d9b700 1 --2- 172.21.15.3:0/2280163715 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf2809e1d0 0x7fdf2811ed60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009c70 tx=0x7fdf1c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:45.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.606+0000 7fdf16ffd700 1 -- 172.21.15.3:0/2280163715 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c004300 con 0x7fdf2809e1d0 2023-10-03T10:13:45.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.606+0000 7fdf16ffd700 1 -- 172.21.15.3:0/2280163715 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1c01f070 con 0x7fdf2809e1d0 2023-10-03T10:13:45.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.606+0000 7fdf16ffd700 1 -- 172.21.15.3:0/2280163715 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c0036a0 con 0x7fdf2809e1d0 2023-10-03T10:13:45.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.606+0000 7fdf2caf5700 1 -- 172.21.15.3:0/2280163715 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf1c009550 con 0x7fdf2809e1d0 2023-10-03T10:13:45.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.606+0000 7fdf2caf5700 1 -- 172.21.15.3:0/2280163715 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf28121d10 con 0x7fdf2809e1d0 2023-10-03T10:13:45.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.607+0000 7fdf16ffd700 1 -- 172.21.15.3:0/2280163715 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdf1c021030 con 0x7fdf2809e1d0 2023-10-03T10:13:45.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.607+0000 7fdf16ffd700 1 -- 172.21.15.3:0/2280163715 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fdf1c0408d0 con 0x7fdf2809e1d0 2023-10-03T10:13:45.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.607+0000 7fdf2caf5700 1 -- 172.21.15.3:0/2280163715 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf28074ea0 con 0x7fdf2809e1d0 2023-10-03T10:13:45.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.612+0000 7fdf16ffd700 1 -- 172.21.15.3:0/2280163715 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdf1c0199a0 con 0x7fdf2809e1d0 2023-10-03T10:13:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.653+0000 7f4a4c80e700 1 Processor -- start 2023-10-03T10:13:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.653+0000 7f4a4c80e700 1 -- start start 2023-10-03T10:13:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.654+0000 7f4a4c80e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4a4809d630 0x7f4a480a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.654+0000 7f4a4c80e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a48138d80 0x7f4a4813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.654+0000 7f4a4c80e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a480a2a70 con 0x7f4a480a0a30 2023-10-03T10:13:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.654+0000 7f4a4c80e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a480a01d0 con 0x7f4a4809d630 2023-10-03T10:13:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.654+0000 7f4a4c80e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4809e2b0 con 0x7f4a48138d80 2023-10-03T10:13:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.654+0000 7f4a3dd9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4a4809d630 0x7f4a480a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.654+0000 7f4a3dd9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4a4809d630 0x7f4a480a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46706/0 (socket says 172.21.15.3:46706) 2023-10-03T10:13:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.654+0000 7f4a46d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a48138d80 0x7f4a4813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.654+0000 7f4a3dd9b700 1 -- 172.21.15.3:0/2170472944 learned_addr learned my addr 172.21.15.3:0/2170472944 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.655+0000 7f4a46d9d700 1 -- 172.21.15.3:0/2170472944 >> v2:172.21.15.3:3300/0 conn(0x7f4a4809d630 msgr2=0x7f4a480a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.655+0000 7f4a46d9d700 1 --2- 172.21.15.3:0/2170472944 >> v2:172.21.15.3:3300/0 conn(0x7f4a4809d630 0x7f4a480a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.655+0000 7f4a46d9d700 1 -- 172.21.15.3:0/2170472944 >> v1:172.21.15.3:6789/0 conn(0x7f4a480a0a30 legacy=0x7f4a480a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.655+0000 7f4a46d9d700 1 -- 172.21.15.3:0/2170472944 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4809dc50 con 0x7f4a48138d80 2023-10-03T10:13:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.655+0000 7f4a46d9d700 1 --2- 172.21.15.3:0/2170472944 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a48138d80 0x7f4a4813b160 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4a40009d90 tx=0x7f4a4000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1a645ceefcbde40 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.655+0000 7f4a45d9b700 1 -- 172.21.15.3:0/2170472944 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4000e070 con 0x7f4a48138d80 2023-10-03T10:13:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.655+0000 7f4a45d9b700 1 -- 172.21.15.3:0/2170472944 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a4000bb20 con 0x7f4a48138d80 2023-10-03T10:13:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.655+0000 7f4a45d9b700 1 -- 172.21.15.3:0/2170472944 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40012430 con 0x7f4a48138d80 2023-10-03T10:13:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.656+0000 7f4a4c80e700 1 -- 172.21.15.3:0/2170472944 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a48138d80 msgr2=0x7f4a4813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:45.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.656+0000 7f4a4c80e700 1 --2- 172.21.15.3:0/2170472944 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a48138d80 0x7f4a4813b160 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4a40009d90 tx=0x7f4a4000b330 comp rx=0 tx=0).stop 2023-10-03T10:13:45.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.656+0000 7f4a4c80e700 1 -- 172.21.15.3:0/2170472944 shutdown_connections 2023-10-03T10:13:45.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.656+0000 7f4a4c80e700 1 --2- 172.21.15.3:0/2170472944 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a48138d80 0x7f4a4813b160 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.656+0000 7f4a4c80e700 1 --2- 172.21.15.3:0/2170472944 >> v2:172.21.15.3:3300/0 conn(0x7f4a4809d630 0x7f4a480a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.656+0000 7f4a4c80e700 1 -- 172.21.15.3:0/2170472944 >> 172.21.15.3:0/2170472944 conn(0x7f4a480a3050 msgr2=0x7f4a480a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:45.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.656+0000 7f4a4c80e700 1 -- 172.21.15.3:0/2170472944 shutdown_connections 2023-10-03T10:13:45.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.656+0000 7f4a4c80e700 1 -- 172.21.15.3:0/2170472944 wait complete. 2023-10-03T10:13:45.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.658+0000 7f4a4c80e700 1 Processor -- start 2023-10-03T10:13:45.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.658+0000 7f4a4c80e700 1 -- start start 2023-10-03T10:13:45.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.658+0000 7f4a4c80e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a4809d630 0x7f4a48120620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:45.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.658+0000 7f4a4c80e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a480a0a30 0x7f4a48120b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:45.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.659+0000 7f4a4c80e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a48138d80 0x7f4a481210a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:45.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.659+0000 7f4a4c80e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a48124140 con 0x7f4a4809d630 2023-10-03T10:13:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.659+0000 7f4a4c80e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a481234f0 con 0x7f4a480a0a30 2023-10-03T10:13:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.659+0000 7f4a4c80e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a48138b20 con 0x7f4a48138d80 2023-10-03T10:13:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.659+0000 7f4a46d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a48138d80 0x7f4a481210a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.659+0000 7f4a46d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a48138d80 0x7f4a481210a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55614/0 (socket says 172.21.15.3:55614) 2023-10-03T10:13:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.659+0000 7f4a46d9d700 1 -- 172.21.15.3:0/1836235458 learned_addr learned my addr 172.21.15.3:0/1836235458 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.659+0000 7f4a4659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a4809d630 0x7f4a48120620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.659+0000 7f4a4659c700 1 -- 172.21.15.3:0/1836235458 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a48138d80 msgr2=0x7f4a481210a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.659+0000 7f4a4659c700 1 --2- 172.21.15.3:0/1836235458 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a48138d80 0x7f4a481210a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.659+0000 7f4a4659c700 1 -- 172.21.15.3:0/1836235458 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a480a0a30 msgr2=0x7f4a48120b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.659+0000 7f4a4659c700 1 --2- 172.21.15.3:0/1836235458 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a480a0a30 0x7f4a48120b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.659+0000 7f4a4659c700 1 -- 172.21.15.3:0/1836235458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a40009a70 con 0x7f4a4809d630 2023-10-03T10:13:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.659+0000 7f4a46d9d700 1 --2- 172.21.15.3:0/1836235458 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a48138d80 0x7f4a481210a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.659+0000 7f4a4659c700 1 --2- 172.21.15.3:0/1836235458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a4809d630 0x7f4a48120620 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4a38000f60 tx=0x7f4a3800c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:45.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.660+0000 7f4a3f7fe700 1 -- 172.21.15.3:0/1836235458 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a380052d0 con 0x7f4a4809d630 2023-10-03T10:13:45.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.660+0000 7f4a3f7fe700 1 -- 172.21.15.3:0/1836235458 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a38005430 con 0x7f4a4809d630 2023-10-03T10:13:45.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.660+0000 7f4a4c80e700 1 -- 172.21.15.3:0/1836235458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a4813afc0 con 0x7f4a4809d630 2023-10-03T10:13:45.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.660+0000 7f4a3f7fe700 1 -- 172.21.15.3:0/1836235458 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a380038c0 con 0x7f4a4809d630 2023-10-03T10:13:45.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.660+0000 7f4a4c80e700 1 -- 172.21.15.3:0/1836235458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a481248d0 con 0x7f4a4809d630 2023-10-03T10:13:45.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.660+0000 7f4a3f7fe700 1 -- 172.21.15.3:0/1836235458 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4a38003a20 con 0x7f4a4809d630 2023-10-03T10:13:45.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.661+0000 7f4a3f7fe700 1 -- 172.21.15.3:0/1836235458 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4a3803ebb0 con 0x7f4a4809d630 2023-10-03T10:13:45.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.661+0000 7f4a4c80e700 1 -- 172.21.15.3:0/1836235458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a480747f0 con 0x7f4a4809d630 2023-10-03T10:13:45.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.667+0000 7f4a3f7fe700 1 -- 172.21.15.3:0/1836235458 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4a38017020 con 0x7f4a4809d630 2023-10-03T10:13:45.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.696+0000 7fa034b0e700 1 Processor -- start 2023-10-03T10:13:45.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.696+0000 7fa034b0e700 1 -- start start 2023-10-03T10:13:45.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.697+0000 7fa034b0e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa03009d760 0x7fa03009db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:45.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.697+0000 7fa034b0e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa03009e070 0x7fa030130800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:45.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.697+0000 7fa034b0e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa030130d40 0x7fa030131180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:45.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.697+0000 7fa034b0e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0301316c0 con 0x7fa03009e070 2023-10-03T10:13:45.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.697+0000 7fa034b0e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa03009c520 con 0x7fa030130d40 2023-10-03T10:13:45.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.697+0000 7fa034b0e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa03009f1a0 con 0x7fa03009d760 2023-10-03T10:13:45.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.697+0000 7fa02dd9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa03009e070 0x7fa030130800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:45.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.697+0000 7fa02dd9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa03009e070 0x7fa030130800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46720/0 (socket says 172.21.15.3:46720) 2023-10-03T10:13:45.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.697+0000 7fa02dd9b700 1 -- 172.21.15.3:0/2844497716 learned_addr learned my addr 172.21.15.3:0/2844497716 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:45.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.697+0000 7fa02ed9d700 1 --2- 172.21.15.3:0/2844497716 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa030130d40 0x7fa030131180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:45.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.697+0000 7fa02e59c700 1 --2- 172.21.15.3:0/2844497716 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa03009d760 0x7fa03009db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:45.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.697+0000 7fa02ed9d700 1 -- 172.21.15.3:0/2844497716 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa03009d760 msgr2=0x7fa03009db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:45.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.698+0000 7fa02ed9d700 1 --2- 172.21.15.3:0/2844497716 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa03009d760 0x7fa03009db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.698+0000 7fa02ed9d700 1 -- 172.21.15.3:0/2844497716 >> v2:172.21.15.3:3300/0 conn(0x7fa03009e070 msgr2=0x7fa030130800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:45.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.698+0000 7fa02ed9d700 1 --2- 172.21.15.3:0/2844497716 >> v2:172.21.15.3:3300/0 conn(0x7fa03009e070 0x7fa030130800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.698+0000 7fa02ed9d700 1 -- 172.21.15.3:0/2844497716 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa03009a940 con 0x7fa030130d40 2023-10-03T10:13:45.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.698+0000 7fa02dd9b700 1 --2- 172.21.15.3:0/2844497716 >> v2:172.21.15.3:3300/0 conn(0x7fa03009e070 0x7fa030130800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:45.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.698+0000 7fa02ed9d700 1 --2- 172.21.15.3:0/2844497716 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa030130d40 0x7fa030131180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa028009d90 tx=0x7fa02800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a07415fbc8e68b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:45.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.698+0000 7fa02d59a700 1 -- 172.21.15.3:0/2844497716 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02800e070 con 0x7fa030130d40 2023-10-03T10:13:45.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.698+0000 7fa02d59a700 1 -- 172.21.15.3:0/2844497716 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa02800bb20 con 0x7fa030130d40 2023-10-03T10:13:45.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.698+0000 7fa02d59a700 1 -- 172.21.15.3:0/2844497716 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa028012430 con 0x7fa030130d40 2023-10-03T10:13:45.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.699+0000 7fa034b0e700 1 -- 172.21.15.3:0/2844497716 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa030130d40 msgr2=0x7fa030131180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:45.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.699+0000 7fa034b0e700 1 --2- 172.21.15.3:0/2844497716 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa030130d40 0x7fa030131180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa028009d90 tx=0x7fa02800b330 comp rx=0 tx=0).stop 2023-10-03T10:13:45.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.699+0000 7fa034b0e700 1 -- 172.21.15.3:0/2844497716 shutdown_connections 2023-10-03T10:13:45.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.699+0000 7fa034b0e700 1 --2- 172.21.15.3:0/2844497716 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa030130d40 0x7fa030131180 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.699+0000 7fa034b0e700 1 --2- 172.21.15.3:0/2844497716 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa03009d760 0x7fa03009db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.699+0000 7fa034b0e700 1 --2- 172.21.15.3:0/2844497716 >> v2:172.21.15.3:3300/0 conn(0x7fa03009e070 0x7fa030130800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.699+0000 7fa034b0e700 1 -- 172.21.15.3:0/2844497716 >> 172.21.15.3:0/2844497716 conn(0x7fa030066f50 msgr2=0x7fa0300a2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:45.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.699+0000 7fa034b0e700 1 -- 172.21.15.3:0/2844497716 shutdown_connections 2023-10-03T10:13:45.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.699+0000 7fa034b0e700 1 -- 172.21.15.3:0/2844497716 wait complete. 2023-10-03T10:13:45.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.700+0000 7fa034b0e700 1 Processor -- start 2023-10-03T10:13:45.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.700+0000 7fa034b0e700 1 -- start start 2023-10-03T10:13:45.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.701+0000 7fa034b0e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa03009d760 0x7fa03011d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:45.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.701+0000 7fa034b0e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa03009e070 0x7fa03011de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:45.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.701+0000 7fa034b0e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa030130d40 0x7fa03011e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:45.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.701+0000 7fa034b0e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0301207b0 con 0x7fa03009d760 2023-10-03T10:13:45.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.701+0000 7fa034b0e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa03012c280 con 0x7fa030130d40 2023-10-03T10:13:45.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.701+0000 7fa034b0e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa030120450 con 0x7fa03009e070 2023-10-03T10:13:45.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.701+0000 7fa02e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa03009d760 0x7fa03011d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:45.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.701+0000 7fa02e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa03009d760 0x7fa03011d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46722/0 (socket says 172.21.15.3:46722) 2023-10-03T10:13:45.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.701+0000 7fa02e59c700 1 -- 172.21.15.3:0/3538160117 learned_addr learned my addr 172.21.15.3:0/3538160117 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:45.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.701+0000 7fa02dd9b700 1 --2- 172.21.15.3:0/3538160117 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa03009e070 0x7fa03011de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:45.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.701+0000 7fa02ed9d700 1 --2- 172.21.15.3:0/3538160117 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa030130d40 0x7fa03011e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:45.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.701+0000 7fa02e59c700 1 -- 172.21.15.3:0/3538160117 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa03009e070 msgr2=0x7fa03011de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:45.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.701+0000 7fa02e59c700 1 --2- 172.21.15.3:0/3538160117 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa03009e070 0x7fa03011de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.701+0000 7fa02e59c700 1 -- 172.21.15.3:0/3538160117 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa030130d40 msgr2=0x7fa03011e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:45.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.701+0000 7fa02e59c700 1 --2- 172.21.15.3:0/3538160117 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa030130d40 0x7fa03011e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.701+0000 7fa02e59c700 1 -- 172.21.15.3:0/3538160117 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa02400b7a0 con 0x7fa03009d760 2023-10-03T10:13:45.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.701+0000 7fa02e59c700 1 --2- 172.21.15.3:0/3538160117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa03009d760 0x7fa03011d8c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa02000e8a0 tx=0x7fa02000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:45.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.702+0000 7fa01effd700 1 -- 172.21.15.3:0/3538160117 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa020009590 con 0x7fa03009d760 2023-10-03T10:13:45.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.702+0000 7fa01effd700 1 -- 172.21.15.3:0/3538160117 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa02000eea0 con 0x7fa03009d760 2023-10-03T10:13:45.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.702+0000 7fa034b0e700 1 -- 172.21.15.3:0/3538160117 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa028009a70 con 0x7fa03009d760 2023-10-03T10:13:45.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.702+0000 7fa01effd700 1 -- 172.21.15.3:0/3538160117 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02000ca50 con 0x7fa03009d760 2023-10-03T10:13:45.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.702+0000 7fa034b0e700 1 -- 172.21.15.3:0/3538160117 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0300a00b0 con 0x7fa03009d760 2023-10-03T10:13:45.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.703+0000 7fa01effd700 1 -- 172.21.15.3:0/3538160117 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa02000cbb0 con 0x7fa03009d760 2023-10-03T10:13:45.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.703+0000 7fa034b0e700 1 -- 172.21.15.3:0/3538160117 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0300747f0 con 0x7fa03009d760 2023-10-03T10:13:45.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.703+0000 7fa01effd700 1 -- 172.21.15.3:0/3538160117 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa02003f720 con 0x7fa03009d760 2023-10-03T10:13:45.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.708+0000 7fa01effd700 1 -- 172.21.15.3:0/3538160117 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa0200232f0 con 0x7fa03009d760 2023-10-03T10:13:45.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.845+0000 7fdf2caf5700 1 -- 172.21.15.3:0/2280163715 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdf28134680 con 0x7fdf2809e1d0 2023-10-03T10:13:45.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.846+0000 7fdf16ffd700 1 -- 172.21.15.3:0/2280163715 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdf1c015020 con 0x7fdf2809e1d0 2023-10-03T10:13:45.846 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:45.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.848+0000 7fdf2caf5700 1 -- 172.21.15.3:0/2280163715 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf2809e1d0 msgr2=0x7fdf2811ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:45.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.848+0000 7fdf2caf5700 1 --2- 172.21.15.3:0/2280163715 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf2809e1d0 0x7fdf2811ed60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009c70 tx=0x7fdf1c004750 comp rx=0 tx=0).stop 2023-10-03T10:13:45.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.848+0000 7fdf2caf5700 1 -- 172.21.15.3:0/2280163715 shutdown_connections 2023-10-03T10:13:45.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.848+0000 7fdf2caf5700 1 --2- 172.21.15.3:0/2280163715 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf28139170 0x7fdf2811f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.848+0000 7fdf2caf5700 1 --2- 172.21.15.3:0/2280163715 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf2809d6f0 0x7fdf2811e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.848+0000 7fdf2caf5700 1 --2- 172.21.15.3:0/2280163715 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf2809e1d0 0x7fdf2811ed60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.848+0000 7fdf2caf5700 1 -- 172.21.15.3:0/2280163715 >> 172.21.15.3:0/2280163715 conn(0x7fdf28066f50 msgr2=0x7fdf28136b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:45.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.848+0000 7fdf2caf5700 1 -- 172.21.15.3:0/2280163715 shutdown_connections 2023-10-03T10:13:45.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.848+0000 7fdf2caf5700 1 -- 172.21.15.3:0/2280163715 wait complete. 2023-10-03T10:13:45.862 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:45.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.875+0000 7f4a4c80e700 1 -- 172.21.15.3:0/1836235458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4a48123da0 con 0x7f4a4809d630 2023-10-03T10:13:45.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.875+0000 7f4a3f7fe700 1 -- 172.21.15.3:0/1836235458 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4a38019e60 con 0x7f4a4809d630 2023-10-03T10:13:45.876 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:45.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.878+0000 7f4a4c80e700 1 -- 172.21.15.3:0/1836235458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a4809d630 msgr2=0x7f4a48120620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:45.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.878+0000 7f4a4c80e700 1 --2- 172.21.15.3:0/1836235458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a4809d630 0x7f4a48120620 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4a38000f60 tx=0x7f4a3800c730 comp rx=0 tx=0).stop 2023-10-03T10:13:45.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.878+0000 7f4a4c80e700 1 -- 172.21.15.3:0/1836235458 shutdown_connections 2023-10-03T10:13:45.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.878+0000 7f4a4c80e700 1 --2- 172.21.15.3:0/1836235458 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a48138d80 0x7f4a481210a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.878+0000 7f4a4c80e700 1 --2- 172.21.15.3:0/1836235458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a4809d630 0x7f4a48120620 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.878+0000 7f4a4c80e700 1 --2- 172.21.15.3:0/1836235458 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a480a0a30 0x7f4a48120b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.878+0000 7f4a4c80e700 1 -- 172.21.15.3:0/1836235458 >> 172.21.15.3:0/1836235458 conn(0x7f4a480a3050 msgr2=0x7f4a48139150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:45.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.878+0000 7f4a4c80e700 1 -- 172.21.15.3:0/1836235458 shutdown_connections 2023-10-03T10:13:45.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.878+0000 7f4a4c80e700 1 -- 172.21.15.3:0/1836235458 wait complete. 2023-10-03T10:13:45.892 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:45.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.914+0000 7fa034b0e700 1 -- 172.21.15.3:0/3538160117 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa03006c360 con 0x7fa03009d760 2023-10-03T10:13:45.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.915+0000 7fa01effd700 1 -- 172.21.15.3:0/3538160117 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa02001ae80 con 0x7fa03009d760 2023-10-03T10:13:45.915 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:45.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.917+0000 7fa034b0e700 1 -- 172.21.15.3:0/3538160117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa03009d760 msgr2=0x7fa03011d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:45.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.917+0000 7fa034b0e700 1 --2- 172.21.15.3:0/3538160117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa03009d760 0x7fa03011d8c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa02000e8a0 tx=0x7fa02000ec60 comp rx=0 tx=0).stop 2023-10-03T10:13:45.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.917+0000 7fa034b0e700 1 -- 172.21.15.3:0/3538160117 shutdown_connections 2023-10-03T10:13:45.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.917+0000 7fa034b0e700 1 --2- 172.21.15.3:0/3538160117 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa030130d40 0x7fa03011e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.917+0000 7fa034b0e700 1 --2- 172.21.15.3:0/3538160117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa03009d760 0x7fa03011d8c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.917+0000 7fa034b0e700 1 --2- 172.21.15.3:0/3538160117 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa03009e070 0x7fa03011de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:45.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.917+0000 7fa034b0e700 1 -- 172.21.15.3:0/3538160117 >> 172.21.15.3:0/3538160117 conn(0x7fa030066f50 msgr2=0x7fa03012da90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:45.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.917+0000 7fa034b0e700 1 -- 172.21.15.3:0/3538160117 shutdown_connections 2023-10-03T10:13:45.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:45.917+0000 7fa034b0e700 1 -- 172.21.15.3:0/3538160117 wait complete. 2023-10-03T10:13:45.931 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:46.599 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:46.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.748+0000 7fee7221a700 1 Processor -- start 2023-10-03T10:13:46.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.748+0000 7fee7221a700 1 -- start start 2023-10-03T10:13:46.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.749+0000 7fee7221a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fee6c09e930 0x7fee6c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:46.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.749+0000 7fee7221a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee6c09f650 0x7fee6c131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:46.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.749+0000 7fee7221a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee6c070940 con 0x7fee6c09f240 2023-10-03T10:13:46.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.749+0000 7fee7221a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee6c06fd60 con 0x7fee6c09e930 2023-10-03T10:13:46.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.749+0000 7fee7221a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c06fa80 con 0x7fee6c09f650 2023-10-03T10:13:46.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.749+0000 7fee6b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fee6c09e930 0x7fee6c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:46.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.749+0000 7fee6bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee6c09f650 0x7fee6c131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:46.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.749+0000 7fee6b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fee6c09e930 0x7fee6c09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46726/0 (socket says 172.21.15.3:46726) 2023-10-03T10:13:46.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.749+0000 7fee6bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee6c09f650 0x7fee6c131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55638/0 (socket says 172.21.15.3:55638) 2023-10-03T10:13:46.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.749+0000 7fee6b7fe700 1 -- 172.21.15.3:0/3032498619 learned_addr learned my addr 172.21.15.3:0/3032498619 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:46.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.749+0000 7fee6b7fe700 1 -- 172.21.15.3:0/3032498619 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee6c09f650 msgr2=0x7fee6c131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:46.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.749+0000 7fee6b7fe700 1 --2- 172.21.15.3:0/3032498619 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee6c09f650 0x7fee6c131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:46.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.749+0000 7fee6b7fe700 1 -- 172.21.15.3:0/3032498619 >> v1:172.21.15.3:6789/0 conn(0x7fee6c09f240 legacy=0x7fee6c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:46.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.749+0000 7fee6b7fe700 1 -- 172.21.15.3:0/3032498619 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6c09fb60 con 0x7fee6c09e930 2023-10-03T10:13:46.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.749+0000 7fee6bfff700 1 --2- 172.21.15.3:0/3032498619 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee6c09f650 0x7fee6c131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:46.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.750+0000 7fee6b7fe700 1 --2- 172.21.15.3:0/3032498619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee6c09e930 0x7fee6c09ed00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fee5c0098f0 tx=0x7fee5c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549cb24a2488c4c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:46.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.750+0000 7fee6a7fc700 1 -- 172.21.15.3:0/3032498619 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c011830 con 0x7fee6c09e930 2023-10-03T10:13:46.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.750+0000 7fee6a7fc700 1 -- 172.21.15.3:0/3032498619 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5c011990 con 0x7fee6c09e930 2023-10-03T10:13:46.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.750+0000 7fee6a7fc700 1 -- 172.21.15.3:0/3032498619 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c011d20 con 0x7fee6c09e930 2023-10-03T10:13:46.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.750+0000 7fee7221a700 1 -- 172.21.15.3:0/3032498619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee6c09e930 msgr2=0x7fee6c09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:46.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.750+0000 7fee7221a700 1 --2- 172.21.15.3:0/3032498619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee6c09e930 0x7fee6c09ed00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fee5c0098f0 tx=0x7fee5c009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:46.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.750+0000 7fee7221a700 1 -- 172.21.15.3:0/3032498619 shutdown_connections 2023-10-03T10:13:46.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.750+0000 7fee7221a700 1 --2- 172.21.15.3:0/3032498619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee6c09e930 0x7fee6c09ed00 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:46.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.750+0000 7fee7221a700 1 --2- 172.21.15.3:0/3032498619 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee6c09f650 0x7fee6c131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:46.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.750+0000 7fee7221a700 1 -- 172.21.15.3:0/3032498619 >> 172.21.15.3:0/3032498619 conn(0x7fee6c0a4210 msgr2=0x7fee6c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:46.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.751+0000 7fee7221a700 1 -- 172.21.15.3:0/3032498619 shutdown_connections 2023-10-03T10:13:46.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.751+0000 7fee7221a700 1 -- 172.21.15.3:0/3032498619 wait complete. 2023-10-03T10:13:46.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.752+0000 7fee7221a700 1 Processor -- start 2023-10-03T10:13:46.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.752+0000 7fee7221a700 1 -- start start 2023-10-03T10:13:46.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.752+0000 7fee7221a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee6c09e930 0x7fee6c1a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:46.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.752+0000 7fee7221a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee6c09f240 0x7fee6c1a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:46.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.752+0000 7fee7221a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee6c09f650 0x7fee6c1a2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:46.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.752+0000 7fee7221a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c1a3160 con 0x7fee6c09e930 2023-10-03T10:13:46.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.752+0000 7fee7221a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c1a38c0 con 0x7fee6c09f650 2023-10-03T10:13:46.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.752+0000 7fee7221a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c1a4b10 con 0x7fee6c09f240 2023-10-03T10:13:46.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.752+0000 7fee6b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee6c09e930 0x7fee6c1a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:46.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.752+0000 7fee6affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee6c09f240 0x7fee6c1a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:46.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.753+0000 7fee6b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee6c09e930 0x7fee6c1a1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46740/0 (socket says 172.21.15.3:46740) 2023-10-03T10:13:46.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.753+0000 7fee6b7fe700 1 -- 172.21.15.3:0/1094003536 learned_addr learned my addr 172.21.15.3:0/1094003536 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:46.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.753+0000 7fee6bfff700 1 --2- 172.21.15.3:0/1094003536 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee6c09f650 0x7fee6c1a2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:46.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.753+0000 7fee6b7fe700 1 -- 172.21.15.3:0/1094003536 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee6c09f240 msgr2=0x7fee6c1a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:46.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.753+0000 7fee6b7fe700 1 --2- 172.21.15.3:0/1094003536 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee6c09f240 0x7fee6c1a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:46.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.753+0000 7fee6b7fe700 1 -- 172.21.15.3:0/1094003536 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee6c09f650 msgr2=0x7fee6c1a2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:46.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.753+0000 7fee6b7fe700 1 --2- 172.21.15.3:0/1094003536 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee6c09f650 0x7fee6c1a2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:46.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.753+0000 7fee6b7fe700 1 -- 172.21.15.3:0/1094003536 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6400f050 con 0x7fee6c09e930 2023-10-03T10:13:46.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.753+0000 7fee6b7fe700 1 --2- 172.21.15.3:0/1094003536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee6c09e930 0x7fee6c1a1cb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009c40 tx=0x7fee5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:46.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.753+0000 7fee63fff700 1 -- 172.21.15.3:0/1094003536 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c004480 con 0x7fee6c09e930 2023-10-03T10:13:46.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.753+0000 7fee63fff700 1 -- 172.21.15.3:0/1094003536 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5c01f070 con 0x7fee6c09e930 2023-10-03T10:13:46.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.753+0000 7fee7221a700 1 -- 172.21.15.3:0/1094003536 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee5c009550 con 0x7fee6c09e930 2023-10-03T10:13:46.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.753+0000 7fee63fff700 1 -- 172.21.15.3:0/1094003536 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c0038c0 con 0x7fee6c09e930 2023-10-03T10:13:46.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.753+0000 7fee7221a700 1 -- 172.21.15.3:0/1094003536 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee6c0a0190 con 0x7fee6c09e930 2023-10-03T10:13:46.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.754+0000 7fee63fff700 1 -- 172.21.15.3:0/1094003536 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fee5c021030 con 0x7fee6c09e930 2023-10-03T10:13:46.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.754+0000 7fee63fff700 1 -- 172.21.15.3:0/1094003536 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fee5c040be0 con 0x7fee6c09e930 2023-10-03T10:13:46.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.754+0000 7fee61ffb700 1 -- 172.21.15.3:0/1094003536 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee440052e0 con 0x7fee6c09e930 2023-10-03T10:13:46.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.759+0000 7fee63fff700 1 -- 172.21.15.3:0/1094003536 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fee5c015020 con 0x7fee6c09e930 2023-10-03T10:13:46.863 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:46.893 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:46.931 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:46.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.967+0000 7fee61ffb700 1 -- 172.21.15.3:0/1094003536 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fee44005590 con 0x7fee6c09e930 2023-10-03T10:13:46.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.968+0000 7fee63fff700 1 -- 172.21.15.3:0/1094003536 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fee5c003a20 con 0x7fee6c09e930 2023-10-03T10:13:46.968 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.970+0000 7fee61ffb700 1 -- 172.21.15.3:0/1094003536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee6c09e930 msgr2=0x7fee6c1a1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.970+0000 7fee61ffb700 1 --2- 172.21.15.3:0/1094003536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee6c09e930 0x7fee6c1a1cb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009c40 tx=0x7fee5c004750 comp rx=0 tx=0).stop 2023-10-03T10:13:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.970+0000 7fee61ffb700 1 -- 172.21.15.3:0/1094003536 shutdown_connections 2023-10-03T10:13:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.970+0000 7fee61ffb700 1 --2- 172.21.15.3:0/1094003536 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee6c09f650 0x7fee6c1a2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.970+0000 7fee61ffb700 1 --2- 172.21.15.3:0/1094003536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee6c09e930 0x7fee6c1a1cb0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.970+0000 7fee61ffb700 1 --2- 172.21.15.3:0/1094003536 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee6c09f240 0x7fee6c1a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.970+0000 7fee61ffb700 1 -- 172.21.15.3:0/1094003536 >> 172.21.15.3:0/1094003536 conn(0x7fee6c0a4210 msgr2=0x7fee6c1b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.971+0000 7fee61ffb700 1 -- 172.21.15.3:0/1094003536 shutdown_connections 2023-10-03T10:13:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:46.971+0000 7fee61ffb700 1 -- 172.21.15.3:0/1094003536 wait complete. 2023-10-03T10:13:46.987 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:47.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.006+0000 7f94ed822700 1 Processor -- start 2023-10-03T10:13:47.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.006+0000 7f94ed822700 1 -- start start 2023-10-03T10:13:47.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.007+0000 7f94ed822700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94e809d630 0x7f94e80a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:47.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.007+0000 7f94ed822700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94e8138d80 0x7f94e813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:47.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.007+0000 7f94ed822700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94e80a2a70 con 0x7f94e80a0a30 2023-10-03T10:13:47.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.007+0000 7f94ed822700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e80a01d0 con 0x7f94e8138d80 2023-10-03T10:13:47.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.007+0000 7f94ed822700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94e809e2b0 con 0x7f94e809d630 2023-10-03T10:13:47.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.007+0000 7f94e6ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f94e80a0a30 0x7f94e80a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45660/0 (socket says 172.21.15.3:45660) 2023-10-03T10:13:47.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.007+0000 7f94e6ffd700 1 -- 172.21.15.3:0/2621401089 learned_addr learned my addr 172.21.15.3:0/2621401089 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:47.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.007+0000 7f94e77fe700 1 --2- 172.21.15.3:0/2621401089 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94e8138d80 0x7f94e813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:47.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.007+0000 7f94e67fc700 1 -- 172.21.15.3:0/2621401089 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2282018157 0 0) 0x7f94e80a2a70 con 0x7f94e80a0a30 2023-10-03T10:13:47.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.007+0000 7f94e77fe700 1 -- 172.21.15.3:0/2621401089 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94e809d630 msgr2=0x7f94e80a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:47.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.007+0000 7f94e77fe700 1 --2- 172.21.15.3:0/2621401089 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94e809d630 0x7f94e80a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:47.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.007+0000 7f94e77fe700 1 -- 172.21.15.3:0/2621401089 >> v1:172.21.15.3:6789/0 conn(0x7f94e80a0a30 legacy=0x7f94e80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:47.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.007+0000 7f94e77fe700 1 -- 172.21.15.3:0/2621401089 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e809dc50 con 0x7f94e8138d80 2023-10-03T10:13:47.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.007+0000 7f94e77fe700 1 --2- 172.21.15.3:0/2621401089 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94e8138d80 0x7f94e813b160 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f94e0009dc0 tx=0x7f94e000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b591dabd9f1afab6 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:47.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.008+0000 7f94e67fc700 1 -- 172.21.15.3:0/2621401089 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e000e070 con 0x7f94e8138d80 2023-10-03T10:13:47.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.008+0000 7f94e67fc700 1 -- 172.21.15.3:0/2621401089 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e000bbd0 con 0x7f94e8138d80 2023-10-03T10:13:47.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.008+0000 7f94e67fc700 1 -- 172.21.15.3:0/2621401089 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e0012430 con 0x7f94e8138d80 2023-10-03T10:13:47.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.008+0000 7f94ed822700 1 -- 172.21.15.3:0/2621401089 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94e8138d80 msgr2=0x7f94e813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:47.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.008+0000 7f94ed822700 1 --2- 172.21.15.3:0/2621401089 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94e8138d80 0x7f94e813b160 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f94e0009dc0 tx=0x7f94e000b330 comp rx=0 tx=0).stop 2023-10-03T10:13:47.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.009+0000 7f94ed822700 1 -- 172.21.15.3:0/2621401089 shutdown_connections 2023-10-03T10:13:47.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.009+0000 7f94ed822700 1 --2- 172.21.15.3:0/2621401089 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94e8138d80 0x7f94e813b160 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:47.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.009+0000 7f94ed822700 1 --2- 172.21.15.3:0/2621401089 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94e809d630 0x7f94e80a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:47.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.009+0000 7f94ed822700 1 -- 172.21.15.3:0/2621401089 >> 172.21.15.3:0/2621401089 conn(0x7f94e80a3050 msgr2=0x7f94e80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:47.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.009+0000 7f94ed822700 1 -- 172.21.15.3:0/2621401089 shutdown_connections 2023-10-03T10:13:47.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.009+0000 7f94ed822700 1 -- 172.21.15.3:0/2621401089 wait complete. 2023-10-03T10:13:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.010+0000 7f94ed822700 1 Processor -- start 2023-10-03T10:13:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.010+0000 7f94ed822700 1 -- start start 2023-10-03T10:13:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.011+0000 7f94ed822700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94e809d630 0x7f94e812c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.011+0000 7f94ed822700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94e80a0a30 0x7f94e812cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.011+0000 7f94ed822700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94e8138d80 0x7f94e812cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.011+0000 7f94ed822700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e80a0410 con 0x7f94e809d630 2023-10-03T10:13:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.011+0000 7f94ed822700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e8130200 con 0x7f94e80a0a30 2023-10-03T10:13:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.011+0000 7f94ed822700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94e812edd0 con 0x7f94e8138d80 2023-10-03T10:13:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.011+0000 7f94e6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94e809d630 0x7f94e812c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.011+0000 7f94e6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94e809d630 0x7f94e812c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46746/0 (socket says 172.21.15.3:46746) 2023-10-03T10:13:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.011+0000 7f94e6ffd700 1 -- 172.21.15.3:0/406819319 learned_addr learned my addr 172.21.15.3:0/406819319 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.011+0000 7f94e77fe700 1 --2- 172.21.15.3:0/406819319 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94e8138d80 0x7f94e812cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.011+0000 7f94de7fc700 1 --2- 172.21.15.3:0/406819319 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94e80a0a30 0x7f94e812cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:47.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.012+0000 7f94e6ffd700 1 -- 172.21.15.3:0/406819319 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94e8138d80 msgr2=0x7f94e812cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:47.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.012+0000 7f94e6ffd700 1 --2- 172.21.15.3:0/406819319 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94e8138d80 0x7f94e812cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:47.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.012+0000 7f94e6ffd700 1 -- 172.21.15.3:0/406819319 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94e80a0a30 msgr2=0x7f94e812cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:47.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.012+0000 7f94e6ffd700 1 --2- 172.21.15.3:0/406819319 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94e80a0a30 0x7f94e812cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:47.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.012+0000 7f94e6ffd700 1 -- 172.21.15.3:0/406819319 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e0009a70 con 0x7f94e809d630 2023-10-03T10:13:47.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.012+0000 7f94e77fe700 1 --2- 172.21.15.3:0/406819319 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94e8138d80 0x7f94e812cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:47.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.012+0000 7f94e6ffd700 1 --2- 172.21.15.3:0/406819319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94e809d630 0x7f94e812c570 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f94d800c520 tx=0x7f94d800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:47.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.012+0000 7f94e4ff9700 1 -- 172.21.15.3:0/406819319 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d8004c80 con 0x7f94e809d630 2023-10-03T10:13:47.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.012+0000 7f94e4ff9700 1 -- 172.21.15.3:0/406819319 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d8004e00 con 0x7f94e809d630 2023-10-03T10:13:47.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.012+0000 7f94ed822700 1 -- 172.21.15.3:0/406819319 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e813abb0 con 0x7f94e809d630 2023-10-03T10:13:47.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.012+0000 7f94e4ff9700 1 -- 172.21.15.3:0/406819319 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d8007310 con 0x7f94e809d630 2023-10-03T10:13:47.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.012+0000 7f94ed822700 1 -- 172.21.15.3:0/406819319 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e8130890 con 0x7f94e809d630 2023-10-03T10:13:47.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.013+0000 7f94e4ff9700 1 -- 172.21.15.3:0/406819319 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f94d8007470 con 0x7f94e809d630 2023-10-03T10:13:47.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.013+0000 7f94e4ff9700 1 -- 172.21.15.3:0/406819319 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f94d803f340 con 0x7f94e809d630 2023-10-03T10:13:47.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.013+0000 7f94ed822700 1 -- 172.21.15.3:0/406819319 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94e80747f0 con 0x7f94e809d630 2023-10-03T10:13:47.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.019+0000 7f94e4ff9700 1 -- 172.21.15.3:0/406819319 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f94d80123b0 con 0x7f94e809d630 2023-10-03T10:13:47.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.053+0000 7f5c5f5c3700 1 Processor -- start 2023-10-03T10:13:47.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.053+0000 7f5c5f5c3700 1 -- start start 2023-10-03T10:13:47.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.053+0000 7f5c5f5c3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c580a0a30 0x7f5c580a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:47.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.054+0000 7f5c5f5c3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c5809d630 0x7f5c580a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:47.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.054+0000 7f5c5f5c3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c580a2a70 con 0x7f5c58133930 2023-10-03T10:13:47.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.054+0000 7f5c5f5c3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c580a01d0 con 0x7f5c580a0a30 2023-10-03T10:13:47.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.054+0000 7f5c5f5c3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5809e2b0 con 0x7f5c5809d630 2023-10-03T10:13:47.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.054+0000 7f5c5cb5e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c5809d630 0x7f5c580a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:47.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.054+0000 7f5c5cb5e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c5809d630 0x7f5c580a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55664/0 (socket says 172.21.15.3:55664) 2023-10-03T10:13:47.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.054+0000 7f5c5cb5e700 1 -- 172.21.15.3:0/695512961 learned_addr learned my addr 172.21.15.3:0/695512961 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:47.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.054+0000 7f5c5d35f700 1 --2- 172.21.15.3:0/695512961 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c580a0a30 0x7f5c580a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:47.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.054+0000 7f5c5cb5e700 1 -- 172.21.15.3:0/695512961 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c580a0a30 msgr2=0x7f5c580a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:47.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.054+0000 7f5c5cb5e700 1 --2- 172.21.15.3:0/695512961 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c580a0a30 0x7f5c580a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:47.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.054+0000 7f5c5cb5e700 1 -- 172.21.15.3:0/695512961 >> v1:172.21.15.3:6789/0 conn(0x7f5c58133930 legacy=0x7f5c58133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:47.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.054+0000 7f5c5cb5e700 1 -- 172.21.15.3:0/695512961 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5809dc50 con 0x7f5c5809d630 2023-10-03T10:13:47.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.054+0000 7f5c4ffff700 1 -- 172.21.15.3:0/695512961 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3551655850 0 0) 0x7f5c580a2a70 con 0x7f5c58133930 2023-10-03T10:13:47.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.054+0000 7f5c5cb5e700 1 --2- 172.21.15.3:0/695512961 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c5809d630 0x7f5c580a1340 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009920 tx=0x7f5c48009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=602628bc51e1a536 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:47.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.055+0000 7f5c4ffff700 1 -- 172.21.15.3:0/695512961 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48011830 con 0x7f5c5809d630 2023-10-03T10:13:47.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.055+0000 7f5c4ffff700 1 -- 172.21.15.3:0/695512961 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c48011990 con 0x7f5c5809d630 2023-10-03T10:13:47.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.055+0000 7f5c4ffff700 1 -- 172.21.15.3:0/695512961 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48011d20 con 0x7f5c5809d630 2023-10-03T10:13:47.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.055+0000 7f5c5f5c3700 1 -- 172.21.15.3:0/695512961 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c5809d630 msgr2=0x7f5c580a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:47.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.055+0000 7f5c5f5c3700 1 --2- 172.21.15.3:0/695512961 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c5809d630 0x7f5c580a1340 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009920 tx=0x7f5c48009c30 comp rx=0 tx=0).stop 2023-10-03T10:13:47.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.055+0000 7f5c5f5c3700 1 -- 172.21.15.3:0/695512961 shutdown_connections 2023-10-03T10:13:47.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.055+0000 7f5c5f5c3700 1 --2- 172.21.15.3:0/695512961 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c580a0a30 0x7f5c580a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:47.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.055+0000 7f5c5f5c3700 1 --2- 172.21.15.3:0/695512961 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c5809d630 0x7f5c580a1340 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:47.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.055+0000 7f5c5f5c3700 1 -- 172.21.15.3:0/695512961 >> 172.21.15.3:0/695512961 conn(0x7f5c580a3050 msgr2=0x7f5c580a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:47.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.056+0000 7f5c5f5c3700 1 -- 172.21.15.3:0/695512961 shutdown_connections 2023-10-03T10:13:47.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.056+0000 7f5c5f5c3700 1 -- 172.21.15.3:0/695512961 wait complete. 2023-10-03T10:13:47.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.057+0000 7f5c5f5c3700 1 Processor -- start 2023-10-03T10:13:47.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.057+0000 7f5c5f5c3700 1 -- start start 2023-10-03T10:13:47.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.057+0000 7f5c5f5c3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c5809d630 0x7f5c5811f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:47.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.057+0000 7f5c5f5c3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c580a0a30 0x7f5c5811f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:47.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.057+0000 7f5c5f5c3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c58133930 0x7f5c581c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:47.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.057+0000 7f5c5f5c3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c581211d0 con 0x7f5c580a0a30 2023-10-03T10:13:47.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.057+0000 7f5c5f5c3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c580a0410 con 0x7f5c5809d630 2023-10-03T10:13:47.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.057+0000 7f5c5f5c3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5811ed80 con 0x7f5c58133930 2023-10-03T10:13:47.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.057+0000 7f5c5d35f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c5809d630 0x7f5c5811f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:47.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.057+0000 7f5c5d35f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c5809d630 0x7f5c5811f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46276/0 (socket says 172.21.15.3:46276) 2023-10-03T10:13:47.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.057+0000 7f5c5d35f700 1 -- 172.21.15.3:0/3646404318 learned_addr learned my addr 172.21.15.3:0/3646404318 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:47.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.057+0000 7f5c5cb5e700 1 --2- 172.21.15.3:0/3646404318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c580a0a30 0x7f5c5811f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:47.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.058+0000 7f5c5db60700 1 --2- 172.21.15.3:0/3646404318 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c58133930 0x7f5c581c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:47.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.058+0000 7f5c5d35f700 1 -- 172.21.15.3:0/3646404318 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c58133930 msgr2=0x7f5c581c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:47.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.058+0000 7f5c5d35f700 1 --2- 172.21.15.3:0/3646404318 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c58133930 0x7f5c581c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:47.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.058+0000 7f5c5d35f700 1 -- 172.21.15.3:0/3646404318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c580a0a30 msgr2=0x7f5c5811f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:47.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.058+0000 7f5c5d35f700 1 --2- 172.21.15.3:0/3646404318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c580a0a30 0x7f5c5811f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:47.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.058+0000 7f5c5d35f700 1 -- 172.21.15.3:0/3646404318 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c48009550 con 0x7f5c5809d630 2023-10-03T10:13:47.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.058+0000 7f5c5db60700 1 --2- 172.21.15.3:0/3646404318 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c58133930 0x7f5c581c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:13:47.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.058+0000 7f5c5d35f700 1 --2- 172.21.15.3:0/3646404318 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c5809d630 0x7f5c5811f200 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400e9b0 tx=0x7f5c5400ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:47.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.058+0000 7f5c4dffb700 1 -- 172.21.15.3:0/3646404318 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c54009710 con 0x7f5c5809d630 2023-10-03T10:13:47.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.058+0000 7f5c4dffb700 1 -- 172.21.15.3:0/3646404318 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c5400c850 con 0x7f5c5809d630 2023-10-03T10:13:47.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.058+0000 7f5c4dffb700 1 -- 172.21.15.3:0/3646404318 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5400cbc0 con 0x7f5c5809d630 2023-10-03T10:13:47.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.058+0000 7f5c5f5c3700 1 -- 172.21.15.3:0/3646404318 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c580a5e20 con 0x7f5c5809d630 2023-10-03T10:13:47.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.058+0000 7f5c5f5c3700 1 -- 172.21.15.3:0/3646404318 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c581204b0 con 0x7f5c5809d630 2023-10-03T10:13:47.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.059+0000 7f5c4dffb700 1 -- 172.21.15.3:0/3646404318 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5c5400cd20 con 0x7f5c5809d630 2023-10-03T10:13:47.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.059+0000 7f5c4dffb700 1 -- 172.21.15.3:0/3646404318 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5c5403f770 con 0x7f5c5809d630 2023-10-03T10:13:47.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.060+0000 7f5c5f5c3700 1 -- 172.21.15.3:0/3646404318 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c5811fd30 con 0x7f5c5809d630 2023-10-03T10:13:47.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.064+0000 7f5c4dffb700 1 -- 172.21.15.3:0/3646404318 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5c540232f0 con 0x7f5c5809d630 2023-10-03T10:13:47.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.096+0000 7fe9d024d700 1 Processor -- start 2023-10-03T10:13:47.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.096+0000 7fe9d024d700 1 -- start start 2023-10-03T10:13:47.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.097+0000 7fe9d024d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9c806b250 0x7fe9c8136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:47.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.097+0000 7fe9d024d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9c8136d80 0x7fe9c8139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:47.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.097+0000 7fe9d024d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c809ece0 con 0x7fe9c806a770 2023-10-03T10:13:47.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.097+0000 7fe9d024d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c806b640 con 0x7fe9c806b250 2023-10-03T10:13:47.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.097+0000 7fe9d024d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c809e760 con 0x7fe9c8136d80 2023-10-03T10:13:47.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.097+0000 7fe9cd7e8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9c806b250 0x7fe9c8136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:47.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.097+0000 7fe9cd7e8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9c806b250 0x7fe9c8136840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46288/0 (socket says 172.21.15.3:46288) 2023-10-03T10:13:47.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.097+0000 7fe9cd7e8700 1 -- 172.21.15.3:0/3097108224 learned_addr learned my addr 172.21.15.3:0/3097108224 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:47.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.097+0000 7fe9ce7ea700 1 --2- 172.21.15.3:0/3097108224 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9c8136d80 0x7fe9c8139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:47.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.098+0000 7fe9cd7e8700 1 -- 172.21.15.3:0/3097108224 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9c8136d80 msgr2=0x7fe9c8139160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:47.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.098+0000 7fe9cd7e8700 1 --2- 172.21.15.3:0/3097108224 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9c8136d80 0x7fe9c8139160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:47.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.098+0000 7fe9cd7e8700 1 -- 172.21.15.3:0/3097108224 >> v1:172.21.15.3:6789/0 conn(0x7fe9c806a770 legacy=0x7fe9c806ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:47.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.098+0000 7fe9cd7e8700 1 -- 172.21.15.3:0/3097108224 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c809e040 con 0x7fe9c806b250 2023-10-03T10:13:47.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.098+0000 7fe9ccfe7700 1 -- 172.21.15.3:0/3097108224 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778641701 0 0) 0x7fe9c809ece0 con 0x7fe9c806a770 2023-10-03T10:13:47.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.098+0000 7fe9cd7e8700 1 --2- 172.21.15.3:0/3097108224 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9c806b250 0x7fe9c8136840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009920 tx=0x7fe9b8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=936677b60ae31f59 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:47.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.098+0000 7fe9ccfe7700 1 -- 172.21.15.3:0/3097108224 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8011830 con 0x7fe9c806b250 2023-10-03T10:13:47.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.098+0000 7fe9ccfe7700 1 -- 172.21.15.3:0/3097108224 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b8011990 con 0x7fe9c806b250 2023-10-03T10:13:47.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.098+0000 7fe9ccfe7700 1 -- 172.21.15.3:0/3097108224 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8011d20 con 0x7fe9c806b250 2023-10-03T10:13:47.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.099+0000 7fe9d024d700 1 -- 172.21.15.3:0/3097108224 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9c806b250 msgr2=0x7fe9c8136840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:47.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.099+0000 7fe9d024d700 1 --2- 172.21.15.3:0/3097108224 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9c806b250 0x7fe9c8136840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009920 tx=0x7fe9b8009c30 comp rx=0 tx=0).stop 2023-10-03T10:13:47.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.099+0000 7fe9d024d700 1 -- 172.21.15.3:0/3097108224 shutdown_connections 2023-10-03T10:13:47.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.099+0000 7fe9d024d700 1 --2- 172.21.15.3:0/3097108224 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9c8136d80 0x7fe9c8139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:47.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.099+0000 7fe9d024d700 1 --2- 172.21.15.3:0/3097108224 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9c806b250 0x7fe9c8136840 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:47.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.099+0000 7fe9d024d700 1 -- 172.21.15.3:0/3097108224 >> 172.21.15.3:0/3097108224 conn(0x7fe9c8066f50 msgr2=0x7fe9c809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:47.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.099+0000 7fe9d024d700 1 -- 172.21.15.3:0/3097108224 shutdown_connections 2023-10-03T10:13:47.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.099+0000 7fe9d024d700 1 -- 172.21.15.3:0/3097108224 wait complete. 2023-10-03T10:13:47.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.100+0000 7fe9d024d700 1 Processor -- start 2023-10-03T10:13:47.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.100+0000 7fe9d024d700 1 -- start start 2023-10-03T10:13:47.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.101+0000 7fe9d024d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9c806a770 0x7fe9c81223f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:47.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.101+0000 7fe9d024d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c806b250 0x7fe9c8122930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:47.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.101+0000 7fe9d024d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9c8136d80 0x7fe9c81c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:47.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.101+0000 7fe9d024d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c81252d0 con 0x7fe9c806b250 2023-10-03T10:13:47.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.101+0000 7fe9d024d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c81242a0 con 0x7fe9c806a770 2023-10-03T10:13:47.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.101+0000 7fe9d024d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c809c2b0 con 0x7fe9c8136d80 2023-10-03T10:13:47.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.101+0000 7fe9cd7e8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c806b250 0x7fe9c8122930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:47.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.101+0000 7fe9cd7e8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c806b250 0x7fe9c8122930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46762/0 (socket says 172.21.15.3:46762) 2023-10-03T10:13:47.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.101+0000 7fe9cd7e8700 1 -- 172.21.15.3:0/17713419 learned_addr learned my addr 172.21.15.3:0/17713419 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:47.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.101+0000 7fe9cdfe9700 1 --2- 172.21.15.3:0/17713419 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9c806a770 0x7fe9c81223f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:47.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.101+0000 7fe9ce7ea700 1 --2- 172.21.15.3:0/17713419 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9c8136d80 0x7fe9c81c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:47.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.101+0000 7fe9cd7e8700 1 -- 172.21.15.3:0/17713419 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9c8136d80 msgr2=0x7fe9c81c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:47.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.101+0000 7fe9cd7e8700 1 --2- 172.21.15.3:0/17713419 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9c8136d80 0x7fe9c81c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:47.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.101+0000 7fe9cd7e8700 1 -- 172.21.15.3:0/17713419 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9c806a770 msgr2=0x7fe9c81223f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:47.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.101+0000 7fe9cd7e8700 1 --2- 172.21.15.3:0/17713419 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9c806a770 0x7fe9c81223f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:47.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.101+0000 7fe9cd7e8700 1 -- 172.21.15.3:0/17713419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b8009550 con 0x7fe9c806b250 2023-10-03T10:13:47.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.101+0000 7fe9cd7e8700 1 --2- 172.21.15.3:0/17713419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c806b250 0x7fe9c8122930 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8000c00 tx=0x7fe9b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:47.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.102+0000 7fe9be7fc700 1 -- 172.21.15.3:0/17713419 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8004450 con 0x7fe9c806b250 2023-10-03T10:13:47.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.102+0000 7fe9be7fc700 1 -- 172.21.15.3:0/17713419 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b80045b0 con 0x7fe9c806b250 2023-10-03T10:13:47.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.102+0000 7fe9d024d700 1 -- 172.21.15.3:0/17713419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c8071e60 con 0x7fe9c806b250 2023-10-03T10:13:47.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.102+0000 7fe9be7fc700 1 -- 172.21.15.3:0/17713419 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b80038c0 con 0x7fe9c806b250 2023-10-03T10:13:47.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.102+0000 7fe9d024d700 1 -- 172.21.15.3:0/17713419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c81235a0 con 0x7fe9c806b250 2023-10-03T10:13:47.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.103+0000 7fe9be7fc700 1 -- 172.21.15.3:0/17713419 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe9b8003a20 con 0x7fe9c806b250 2023-10-03T10:13:47.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.103+0000 7fe9be7fc700 1 -- 172.21.15.3:0/17713419 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe9b8021030 con 0x7fe9c806b250 2023-10-03T10:13:47.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.105+0000 7fe9d024d700 1 -- 172.21.15.3:0/17713419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c8121e50 con 0x7fe9c806b250 2023-10-03T10:13:47.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.110+0000 7fe9be7fc700 1 -- 172.21.15.3:0/17713419 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe9b802b080 con 0x7fe9c806b250 2023-10-03T10:13:47.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.235+0000 7f94ed822700 1 -- 172.21.15.3:0/406819319 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f94e8072980 con 0x7f94e809d630 2023-10-03T10:13:47.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.235+0000 7f94e4ff9700 1 -- 172.21.15.3:0/406819319 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f94d8011360 con 0x7f94e809d630 2023-10-03T10:13:47.235 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:47.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.237+0000 7f94ed822700 1 -- 172.21.15.3:0/406819319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94e809d630 msgr2=0x7f94e812c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:47.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.237+0000 7f94ed822700 1 --2- 172.21.15.3:0/406819319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94e809d630 0x7f94e812c570 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f94d800c520 tx=0x7f94d800c830 comp rx=0 tx=0).stop 2023-10-03T10:13:47.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.237+0000 7f94ed822700 1 -- 172.21.15.3:0/406819319 shutdown_connections 2023-10-03T10:13:47.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.237+0000 7f94ed822700 1 --2- 172.21.15.3:0/406819319 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94e8138d80 0x7f94e812cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:47.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.237+0000 7f94ed822700 1 --2- 172.21.15.3:0/406819319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94e809d630 0x7f94e812c570 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:47.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.237+0000 7f94ed822700 1 --2- 172.21.15.3:0/406819319 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94e80a0a30 0x7f94e812cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:47.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.237+0000 7f94ed822700 1 -- 172.21.15.3:0/406819319 >> 172.21.15.3:0/406819319 conn(0x7f94e80a3050 msgr2=0x7f94e80a4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:47.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.238+0000 7f94ed822700 1 -- 172.21.15.3:0/406819319 shutdown_connections 2023-10-03T10:13:47.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.239+0000 7f94ed822700 1 -- 172.21.15.3:0/406819319 wait complete. 2023-10-03T10:13:47.253 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:47.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.297+0000 7f5c5f5c3700 1 -- 172.21.15.3:0/3646404318 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5c580008d0 con 0x7f5c5809d630 2023-10-03T10:13:47.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.298+0000 7f5c4dffb700 1 -- 172.21.15.3:0/3646404318 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5c54021030 con 0x7f5c5809d630 2023-10-03T10:13:47.298 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:47.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.300+0000 7f5c5f5c3700 1 -- 172.21.15.3:0/3646404318 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c5809d630 msgr2=0x7f5c5811f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:47.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.300+0000 7f5c5f5c3700 1 --2- 172.21.15.3:0/3646404318 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c5809d630 0x7f5c5811f200 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400e9b0 tx=0x7f5c5400ecc0 comp rx=0 tx=0).stop 2023-10-03T10:13:47.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.300+0000 7f5c5f5c3700 1 -- 172.21.15.3:0/3646404318 shutdown_connections 2023-10-03T10:13:47.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.300+0000 7f5c5f5c3700 1 --2- 172.21.15.3:0/3646404318 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c58133930 0x7f5c581c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:47.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.300+0000 7f5c5f5c3700 1 --2- 172.21.15.3:0/3646404318 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c5809d630 0x7f5c5811f200 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:47.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.300+0000 7f5c5f5c3700 1 --2- 172.21.15.3:0/3646404318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c580a0a30 0x7f5c5811f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:47.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.300+0000 7f5c5f5c3700 1 -- 172.21.15.3:0/3646404318 >> 172.21.15.3:0/3646404318 conn(0x7f5c580a3050 msgr2=0x7f5c5809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:47.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.300+0000 7f5c5f5c3700 1 -- 172.21.15.3:0/3646404318 shutdown_connections 2023-10-03T10:13:47.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.300+0000 7f5c5f5c3700 1 -- 172.21.15.3:0/3646404318 wait complete. 2023-10-03T10:13:47.314 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:47.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.338+0000 7fe9d024d700 1 -- 172.21.15.3:0/17713419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe9c809f6b0 con 0x7fe9c806b250 2023-10-03T10:13:47.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.339+0000 7fe9be7fc700 1 -- 172.21.15.3:0/17713419 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe9b8015020 con 0x7fe9c806b250 2023-10-03T10:13:47.339 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:47.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.341+0000 7fe9d024d700 1 -- 172.21.15.3:0/17713419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c806b250 msgr2=0x7fe9c8122930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:47.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.341+0000 7fe9d024d700 1 --2- 172.21.15.3:0/17713419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c806b250 0x7fe9c8122930 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8000c00 tx=0x7fe9b8004750 comp rx=0 tx=0).stop 2023-10-03T10:13:47.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.341+0000 7fe9d024d700 1 -- 172.21.15.3:0/17713419 shutdown_connections 2023-10-03T10:13:47.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.341+0000 7fe9d024d700 1 --2- 172.21.15.3:0/17713419 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9c8136d80 0x7fe9c81c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:47.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.341+0000 7fe9d024d700 1 --2- 172.21.15.3:0/17713419 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9c806a770 0x7fe9c81223f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:47.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.341+0000 7fe9d024d700 1 --2- 172.21.15.3:0/17713419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c806b250 0x7fe9c8122930 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:47.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.341+0000 7fe9d024d700 1 -- 172.21.15.3:0/17713419 >> 172.21.15.3:0/17713419 conn(0x7fe9c8066f50 msgr2=0x7fe9c80a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:47.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.341+0000 7fe9d024d700 1 -- 172.21.15.3:0/17713419 shutdown_connections 2023-10-03T10:13:47.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:47.341+0000 7fe9d024d700 1 -- 172.21.15.3:0/17713419 wait complete. 2023-10-03T10:13:47.355 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:47.988 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:48.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.133+0000 7f5885ab8700 1 Processor -- start 2023-10-03T10:13:48.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.133+0000 7f5885ab8700 1 -- start start 2023-10-03T10:13:48.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.133+0000 7f5885ab8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f588009d6f0 0x7f588009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:48.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.133+0000 7f5885ab8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f588009e000 0x7f58801339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:48.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.133+0000 7f5885ab8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f588009e410 0x7f5880133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:48.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.133+0000 7f5885ab8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f588006b460 con 0x7f588009e410 2023-10-03T10:13:48.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.133+0000 7f5885ab8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588009b120 con 0x7f588009d6f0 2023-10-03T10:13:48.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.133+0000 7f5885ab8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f588009a7e0 con 0x7f588009e000 2023-10-03T10:13:48.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.134+0000 7f587f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f588009d6f0 0x7f588009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:48.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.134+0000 7f587effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f588009e000 0x7f58801339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:48.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.134+0000 7f587ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f588009e410 0x7f5880133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:48.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.134+0000 7f587f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f588009d6f0 0x7f588009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46300/0 (socket says 172.21.15.3:46300) 2023-10-03T10:13:48.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.134+0000 7f587f7fe700 1 -- 172.21.15.3:0/3782893451 learned_addr learned my addr 172.21.15.3:0/3782893451 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:48.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.134+0000 7f587effd700 1 -- 172.21.15.3:0/3782893451 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f588009d6f0 msgr2=0x7f588009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:48.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.134+0000 7f587effd700 1 --2- 172.21.15.3:0/3782893451 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f588009d6f0 0x7f588009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.134+0000 7f587effd700 1 -- 172.21.15.3:0/3782893451 >> v2:172.21.15.3:3300/0 conn(0x7f588009e410 msgr2=0x7f5880133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:48.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.134+0000 7f587effd700 1 --2- 172.21.15.3:0/3782893451 >> v2:172.21.15.3:3300/0 conn(0x7f588009e410 0x7f5880133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.134+0000 7f587effd700 1 -- 172.21.15.3:0/3782893451 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588009a500 con 0x7f588009e000 2023-10-03T10:13:48.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.134+0000 7f587f7fe700 1 --2- 172.21.15.3:0/3782893451 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f588009d6f0 0x7f588009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:48.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.134+0000 7f587ffff700 1 --2- 172.21.15.3:0/3782893451 >> v2:172.21.15.3:3300/0 conn(0x7f588009e410 0x7f5880133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:48.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.134+0000 7f587effd700 1 --2- 172.21.15.3:0/3782893451 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f588009e000 0x7f58801339f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5874009870 tx=0x7f5874009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9c269ea4e68bb15 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:48.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.134+0000 7f587e7fc700 1 -- 172.21.15.3:0/3782893451 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5874011830 con 0x7f588009e000 2023-10-03T10:13:48.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.134+0000 7f587e7fc700 1 -- 172.21.15.3:0/3782893451 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5874011990 con 0x7f588009e000 2023-10-03T10:13:48.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.134+0000 7f587e7fc700 1 -- 172.21.15.3:0/3782893451 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5874011d20 con 0x7f588009e000 2023-10-03T10:13:48.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.135+0000 7f5885ab8700 1 -- 172.21.15.3:0/3782893451 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f588009e000 msgr2=0x7f58801339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:48.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.135+0000 7f5885ab8700 1 --2- 172.21.15.3:0/3782893451 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f588009e000 0x7f58801339f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5874009870 tx=0x7f5874009b80 comp rx=0 tx=0).stop 2023-10-03T10:13:48.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.135+0000 7f5885ab8700 1 -- 172.21.15.3:0/3782893451 shutdown_connections 2023-10-03T10:13:48.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.135+0000 7f5885ab8700 1 --2- 172.21.15.3:0/3782893451 >> v2:172.21.15.3:3300/0 conn(0x7f588009e410 0x7f5880133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.135+0000 7f5885ab8700 1 --2- 172.21.15.3:0/3782893451 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f588009d6f0 0x7f588009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.135+0000 7f5885ab8700 1 --2- 172.21.15.3:0/3782893451 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f588009e000 0x7f58801339f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.135+0000 7f5885ab8700 1 -- 172.21.15.3:0/3782893451 >> 172.21.15.3:0/3782893451 conn(0x7f5880066f50 msgr2=0x7f588009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:48.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.135+0000 7f5885ab8700 1 -- 172.21.15.3:0/3782893451 shutdown_connections 2023-10-03T10:13:48.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.135+0000 7f5885ab8700 1 -- 172.21.15.3:0/3782893451 wait complete. 2023-10-03T10:13:48.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.136+0000 7f5885ab8700 1 Processor -- start 2023-10-03T10:13:48.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.137+0000 7f5885ab8700 1 -- start start 2023-10-03T10:13:48.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.137+0000 7f5885ab8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f588009d6f0 0x7f58801a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:48.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.137+0000 7f5885ab8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f588009e000 0x7f58801a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:48.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.137+0000 7f5885ab8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f588009e410 0x7f58801a6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:48.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.137+0000 7f5885ab8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58801a98e0 con 0x7f588009e410 2023-10-03T10:13:48.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.137+0000 7f5885ab8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588009c710 con 0x7f588009d6f0 2023-10-03T10:13:48.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.137+0000 7f5885ab8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f58800a2ff0 con 0x7f588009e000 2023-10-03T10:13:48.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.138+0000 7f587f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f588009d6f0 0x7f58801a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:48.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.138+0000 7f587f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f588009d6f0 0x7f58801a5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46312/0 (socket says 172.21.15.3:46312) 2023-10-03T10:13:48.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.138+0000 7f587f7fe700 1 -- 172.21.15.3:0/3351529627 learned_addr learned my addr 172.21.15.3:0/3351529627 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:48.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.138+0000 7f587ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f588009e410 0x7f58801a6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:48.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.138+0000 7f587effd700 1 --2- 172.21.15.3:0/3351529627 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f588009e000 0x7f58801a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:48.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.138+0000 7f587f7fe700 1 -- 172.21.15.3:0/3351529627 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f588009e000 msgr2=0x7f58801a6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:48.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.138+0000 7f587f7fe700 1 --2- 172.21.15.3:0/3351529627 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f588009e000 0x7f58801a6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.138+0000 7f587f7fe700 1 -- 172.21.15.3:0/3351529627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f588009e410 msgr2=0x7f58801a6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:48.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.138+0000 7f587f7fe700 1 --2- 172.21.15.3:0/3351529627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f588009e410 0x7f58801a6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.138+0000 7f587f7fe700 1 -- 172.21.15.3:0/3351529627 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5870009550 con 0x7f588009d6f0 2023-10-03T10:13:48.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.138+0000 7f587ffff700 1 --2- 172.21.15.3:0/3351529627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f588009e410 0x7f58801a6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:48.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.138+0000 7f587f7fe700 1 --2- 172.21.15.3:0/3351529627 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f588009d6f0 0x7f58801a5dc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f587000ebe0 tx=0x7f587000eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:48.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.138+0000 7f5867fff700 1 -- 172.21.15.3:0/3351529627 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587000c850 con 0x7f588009d6f0 2023-10-03T10:13:48.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.138+0000 7f5867fff700 1 -- 172.21.15.3:0/3351529627 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587000c9b0 con 0x7f588009d6f0 2023-10-03T10:13:48.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.138+0000 7f5885ab8700 1 -- 172.21.15.3:0/3351529627 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5878009a70 con 0x7f588009d6f0 2023-10-03T10:13:48.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.138+0000 7f5867fff700 1 -- 172.21.15.3:0/3351529627 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587000cd40 con 0x7f588009d6f0 2023-10-03T10:13:48.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.138+0000 7f5885ab8700 1 -- 172.21.15.3:0/3351529627 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58740094a0 con 0x7f588009d6f0 2023-10-03T10:13:48.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.139+0000 7f5867fff700 1 -- 172.21.15.3:0/3351529627 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5870013470 con 0x7f588009d6f0 2023-10-03T10:13:48.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.139+0000 7f5867fff700 1 -- 172.21.15.3:0/3351529627 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f587003f950 con 0x7f588009d6f0 2023-10-03T10:13:48.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.139+0000 7f5885ab8700 1 -- 172.21.15.3:0/3351529627 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5880074ea0 con 0x7f588009d6f0 2023-10-03T10:13:48.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.144+0000 7f5867fff700 1 -- 172.21.15.3:0/3351529627 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5870017020 con 0x7f588009d6f0 2023-10-03T10:13:48.253 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:48.315 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:48.356 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:48.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.360+0000 7f5885ab8700 1 -- 172.21.15.3:0/3351529627 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f58801a9460 con 0x7f588009d6f0 2023-10-03T10:13:48.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.361+0000 7f5867fff700 1 -- 172.21.15.3:0/3351529627 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f58801a9460 con 0x7f588009d6f0 2023-10-03T10:13:48.361 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:48.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.364+0000 7f5885ab8700 1 -- 172.21.15.3:0/3351529627 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f588009d6f0 msgr2=0x7f58801a5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:48.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.364+0000 7f5885ab8700 1 --2- 172.21.15.3:0/3351529627 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f588009d6f0 0x7f58801a5dc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f587000ebe0 tx=0x7f587000eef0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.364+0000 7f5885ab8700 1 -- 172.21.15.3:0/3351529627 shutdown_connections 2023-10-03T10:13:48.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.364+0000 7f5885ab8700 1 --2- 172.21.15.3:0/3351529627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f588009e410 0x7f58801a6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.364+0000 7f5885ab8700 1 --2- 172.21.15.3:0/3351529627 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f588009d6f0 0x7f58801a5dc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.364+0000 7f5885ab8700 1 --2- 172.21.15.3:0/3351529627 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f588009e000 0x7f58801a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.364+0000 7f5885ab8700 1 -- 172.21.15.3:0/3351529627 >> 172.21.15.3:0/3351529627 conn(0x7f5880066f50 msgr2=0x7f58801bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:48.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.364+0000 7f5885ab8700 1 -- 172.21.15.3:0/3351529627 shutdown_connections 2023-10-03T10:13:48.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.364+0000 7f5885ab8700 1 -- 172.21.15.3:0/3351529627 wait complete. 2023-10-03T10:13:48.382 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.398+0000 7f068984b700 1 Processor -- start 2023-10-03T10:13:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.398+0000 7f068984b700 1 -- start start 2023-10-03T10:13:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.399+0000 7f068984b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f068409d6f0 0x7f068409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.399+0000 7f068984b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f068409e410 0x7f0684134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.399+0000 7f068984b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f068409b120 con 0x7f068409e000 2023-10-03T10:13:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.399+0000 7f068984b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f068406b460 con 0x7f068409e410 2023-10-03T10:13:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.399+0000 7f068984b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f068409a7e0 con 0x7f068409d6f0 2023-10-03T10:13:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.399+0000 7f06827fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f068409e000 0x7f06841339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45690/0 (socket says 172.21.15.3:45690) 2023-10-03T10:13:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.399+0000 7f06827fc700 1 -- 172.21.15.3:0/2677970894 learned_addr learned my addr 172.21.15.3:0/2677970894 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.399+0000 7f06837fe700 1 --2- 172.21.15.3:0/2677970894 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f068409e410 0x7f0684134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.399+0000 7f0682ffd700 1 --2- 172.21.15.3:0/2677970894 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f068409d6f0 0x7f068409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:48.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.400+0000 7f06837fe700 1 -- 172.21.15.3:0/2677970894 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f068409d6f0 msgr2=0x7f068409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:48.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.400+0000 7f06837fe700 1 --2- 172.21.15.3:0/2677970894 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f068409d6f0 0x7f068409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.400+0000 7f06837fe700 1 -- 172.21.15.3:0/2677970894 >> v1:172.21.15.3:6789/0 conn(0x7f068409e000 legacy=0x7f06841339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.400+0000 7f06837fe700 1 -- 172.21.15.3:0/2677970894 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f068409a500 con 0x7f068409e410 2023-10-03T10:13:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.400+0000 7f0681ffb700 1 -- 172.21.15.3:0/2677970894 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 926880109 0 0) 0x7f068409b120 con 0x7f068409e000 2023-10-03T10:13:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.400+0000 7f06837fe700 1 --2- 172.21.15.3:0/2677970894 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f068409e410 0x7f0684134100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f067c009dc0 tx=0x7f067c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=217aaac372d7cb01 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.400+0000 7f0681ffb700 1 -- 172.21.15.3:0/2677970894 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c00e070 con 0x7f068409e410 2023-10-03T10:13:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.400+0000 7f0681ffb700 1 -- 172.21.15.3:0/2677970894 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067c00bb20 con 0x7f068409e410 2023-10-03T10:13:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.400+0000 7f0681ffb700 1 -- 172.21.15.3:0/2677970894 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c012430 con 0x7f068409e410 2023-10-03T10:13:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.401+0000 7f068984b700 1 -- 172.21.15.3:0/2677970894 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f068409e410 msgr2=0x7f0684134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.401+0000 7f068984b700 1 --2- 172.21.15.3:0/2677970894 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f068409e410 0x7f0684134100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f067c009dc0 tx=0x7f067c00b330 comp rx=0 tx=0).stop 2023-10-03T10:13:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.401+0000 7f068984b700 1 -- 172.21.15.3:0/2677970894 shutdown_connections 2023-10-03T10:13:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.401+0000 7f068984b700 1 --2- 172.21.15.3:0/2677970894 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f068409e410 0x7f0684134100 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.401+0000 7f068984b700 1 --2- 172.21.15.3:0/2677970894 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f068409d6f0 0x7f068409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.401+0000 7f068984b700 1 -- 172.21.15.3:0/2677970894 >> 172.21.15.3:0/2677970894 conn(0x7f0684066f50 msgr2=0x7f068409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:48.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.401+0000 7f068984b700 1 -- 172.21.15.3:0/2677970894 shutdown_connections 2023-10-03T10:13:48.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.401+0000 7f068984b700 1 -- 172.21.15.3:0/2677970894 wait complete. 2023-10-03T10:13:48.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.402+0000 7f068984b700 1 Processor -- start 2023-10-03T10:13:48.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.403+0000 7f068984b700 1 -- start start 2023-10-03T10:13:48.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.403+0000 7f068984b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f068409d6f0 0x7f06841c50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:48.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.403+0000 7f068984b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f068409e000 0x7f06841c5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:48.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.403+0000 7f068984b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f068409e410 0x7f06841c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:48.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.403+0000 7f068984b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06841c48d0 con 0x7f068409d6f0 2023-10-03T10:13:48.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.404+0000 7f068984b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06840a1d90 con 0x7f068409e000 2023-10-03T10:13:48.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.404+0000 7f068984b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f068409c710 con 0x7f068409e410 2023-10-03T10:13:48.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.404+0000 7f06837fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f068409e410 0x7f06841c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:48.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.404+0000 7f06837fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f068409e410 0x7f06841c7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55724/0 (socket says 172.21.15.3:55724) 2023-10-03T10:13:48.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.404+0000 7f0682ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f068409d6f0 0x7f06841c50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:48.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.404+0000 7f06837fe700 1 -- 172.21.15.3:0/1933924815 learned_addr learned my addr 172.21.15.3:0/1933924815 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:48.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.404+0000 7f06827fc700 1 --2- 172.21.15.3:0/1933924815 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f068409e000 0x7f06841c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:48.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.404+0000 7f06837fe700 1 -- 172.21.15.3:0/1933924815 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f068409e000 msgr2=0x7f06841c5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:48.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.404+0000 7f06837fe700 1 --2- 172.21.15.3:0/1933924815 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f068409e000 0x7f06841c5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.404+0000 7f06837fe700 1 -- 172.21.15.3:0/1933924815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f068409d6f0 msgr2=0x7f06841c50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:48.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.404+0000 7f06837fe700 1 --2- 172.21.15.3:0/1933924815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f068409d6f0 0x7f06841c50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.404+0000 7f06837fe700 1 -- 172.21.15.3:0/1933924815 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067c009a70 con 0x7f068409e410 2023-10-03T10:13:48.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.404+0000 7f06837fe700 1 --2- 172.21.15.3:0/1933924815 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f068409e410 0x7f06841c7f80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f067c00b680 tx=0x7f067c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:48.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.405+0000 7f066bfff700 1 -- 172.21.15.3:0/1933924815 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c012430 con 0x7f068409e410 2023-10-03T10:13:48.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.405+0000 7f066bfff700 1 -- 172.21.15.3:0/1933924815 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067c020070 con 0x7f068409e410 2023-10-03T10:13:48.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.405+0000 7f066bfff700 1 -- 172.21.15.3:0/1933924815 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c004280 con 0x7f068409e410 2023-10-03T10:13:48.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.405+0000 7f068984b700 1 -- 172.21.15.3:0/1933924815 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06841abae0 con 0x7f068409e410 2023-10-03T10:13:48.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.405+0000 7f068984b700 1 -- 172.21.15.3:0/1933924815 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06841a9cd0 con 0x7f068409e410 2023-10-03T10:13:48.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.406+0000 7f066bfff700 1 -- 172.21.15.3:0/1933924815 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f067c0043e0 con 0x7f068409e410 2023-10-03T10:13:48.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.406+0000 7f068984b700 1 -- 172.21.15.3:0/1933924815 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0684074ea0 con 0x7f068409e410 2023-10-03T10:13:48.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.406+0000 7f066bfff700 1 -- 172.21.15.3:0/1933924815 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f067c022070 con 0x7f068409e410 2023-10-03T10:13:48.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.411+0000 7f066bfff700 1 -- 172.21.15.3:0/1933924815 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f067c01a740 con 0x7f068409e410 2023-10-03T10:13:48.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.462+0000 7fda833b0700 1 Processor -- start 2023-10-03T10:13:48.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.462+0000 7fda833b0700 1 -- start start 2023-10-03T10:13:48.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.463+0000 7fda833b0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fda7c0a0a30 0x7fda7c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:48.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.463+0000 7fda833b0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda7c138d80 0x7fda7c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:48.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.463+0000 7fda833b0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda7c0a2a70 con 0x7fda7c09d630 2023-10-03T10:13:48.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.463+0000 7fda833b0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c0a01d0 con 0x7fda7c138d80 2023-10-03T10:13:48.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.463+0000 7fda833b0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c09e2b0 con 0x7fda7c0a0a30 2023-10-03T10:13:48.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.463+0000 7fda8094b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fda7c09d630 0x7fda7c0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45698/0 (socket says 172.21.15.3:45698) 2023-10-03T10:13:48.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.463+0000 7fda8094b700 1 -- 172.21.15.3:0/4077428570 learned_addr learned my addr 172.21.15.3:0/4077428570 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:48.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.463+0000 7fda8114c700 1 --2- 172.21.15.3:0/4077428570 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fda7c0a0a30 0x7fda7c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:48.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.463+0000 7fda8194d700 1 --2- 172.21.15.3:0/4077428570 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda7c138d80 0x7fda7c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:48.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.463+0000 7fda73fff700 1 -- 172.21.15.3:0/4077428570 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3194351090 0 0) 0x7fda7c0a2a70 con 0x7fda7c09d630 2023-10-03T10:13:48.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.463+0000 7fda8114c700 1 -- 172.21.15.3:0/4077428570 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda7c138d80 msgr2=0x7fda7c13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:48.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.463+0000 7fda8114c700 1 --2- 172.21.15.3:0/4077428570 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda7c138d80 0x7fda7c13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.463+0000 7fda8114c700 1 -- 172.21.15.3:0/4077428570 >> v1:172.21.15.3:6789/0 conn(0x7fda7c09d630 legacy=0x7fda7c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:48.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.463+0000 7fda8114c700 1 -- 172.21.15.3:0/4077428570 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda7c09dc50 con 0x7fda7c0a0a30 2023-10-03T10:13:48.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.464+0000 7fda8114c700 1 --2- 172.21.15.3:0/4077428570 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fda7c0a0a30 0x7fda7c0a0e00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fda78009870 tx=0x7fda78009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cdd9fcba0d7f11 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:48.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.464+0000 7fda73fff700 1 -- 172.21.15.3:0/4077428570 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda78011830 con 0x7fda7c0a0a30 2023-10-03T10:13:48.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.464+0000 7fda73fff700 1 -- 172.21.15.3:0/4077428570 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda78011990 con 0x7fda7c0a0a30 2023-10-03T10:13:48.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.464+0000 7fda73fff700 1 -- 172.21.15.3:0/4077428570 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda78011d20 con 0x7fda7c0a0a30 2023-10-03T10:13:48.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.465+0000 7fda833b0700 1 -- 172.21.15.3:0/4077428570 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fda7c0a0a30 msgr2=0x7fda7c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:48.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.465+0000 7fda833b0700 1 --2- 172.21.15.3:0/4077428570 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fda7c0a0a30 0x7fda7c0a0e00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fda78009870 tx=0x7fda78009b80 comp rx=0 tx=0).stop 2023-10-03T10:13:48.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.465+0000 7fda833b0700 1 -- 172.21.15.3:0/4077428570 shutdown_connections 2023-10-03T10:13:48.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.465+0000 7fda833b0700 1 --2- 172.21.15.3:0/4077428570 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda7c138d80 0x7fda7c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.465+0000 7fda833b0700 1 --2- 172.21.15.3:0/4077428570 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fda7c0a0a30 0x7fda7c0a0e00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.465+0000 7fda833b0700 1 -- 172.21.15.3:0/4077428570 >> 172.21.15.3:0/4077428570 conn(0x7fda7c0a3050 msgr2=0x7fda7c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:48.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.465+0000 7fda833b0700 1 -- 172.21.15.3:0/4077428570 shutdown_connections 2023-10-03T10:13:48.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.465+0000 7fda833b0700 1 -- 172.21.15.3:0/4077428570 wait complete. 2023-10-03T10:13:48.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.466+0000 7fda833b0700 1 Processor -- start 2023-10-03T10:13:48.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.466+0000 7fda833b0700 1 -- start start 2023-10-03T10:13:48.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.467+0000 7fda833b0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda7c09d630 0x7fda7c133790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:48.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.467+0000 7fda833b0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda7c0a0a30 0x7fda7c133cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:48.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.467+0000 7fda833b0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fda7c138d80 0x7fda7c134210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:48.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.467+0000 7fda833b0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c0a0410 con 0x7fda7c0a0a30 2023-10-03T10:13:48.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.467+0000 7fda833b0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c136ec0 con 0x7fda7c09d630 2023-10-03T10:13:48.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.467+0000 7fda833b0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c13ac70 con 0x7fda7c138d80 2023-10-03T10:13:48.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.467+0000 7fda8114c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda7c09d630 0x7fda7c133790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:48.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.467+0000 7fda8114c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda7c09d630 0x7fda7c133790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46338/0 (socket says 172.21.15.3:46338) 2023-10-03T10:13:48.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.467+0000 7fda8114c700 1 -- 172.21.15.3:0/633120547 learned_addr learned my addr 172.21.15.3:0/633120547 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:48.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.467+0000 7fda8194d700 1 --2- 172.21.15.3:0/633120547 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fda7c138d80 0x7fda7c134210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:48.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.467+0000 7fda8094b700 1 --2- 172.21.15.3:0/633120547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda7c0a0a30 0x7fda7c133cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:48.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.468+0000 7fda8114c700 1 -- 172.21.15.3:0/633120547 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fda7c138d80 msgr2=0x7fda7c134210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:48.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.468+0000 7fda8114c700 1 --2- 172.21.15.3:0/633120547 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fda7c138d80 0x7fda7c134210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.468+0000 7fda8114c700 1 -- 172.21.15.3:0/633120547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda7c0a0a30 msgr2=0x7fda7c133cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:48.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.468+0000 7fda8114c700 1 --2- 172.21.15.3:0/633120547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda7c0a0a30 0x7fda7c133cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.468+0000 7fda8114c700 1 -- 172.21.15.3:0/633120547 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda780094a0 con 0x7fda7c09d630 2023-10-03T10:13:48.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.468+0000 7fda8114c700 1 --2- 172.21.15.3:0/633120547 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda7c09d630 0x7fda7c133790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fda78009840 tx=0x7fda78004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:48.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.468+0000 7fda71ffb700 1 -- 172.21.15.3:0/633120547 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda78004300 con 0x7fda7c09d630 2023-10-03T10:13:48.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.468+0000 7fda71ffb700 1 -- 172.21.15.3:0/633120547 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7801f070 con 0x7fda7c09d630 2023-10-03T10:13:48.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.468+0000 7fda833b0700 1 -- 172.21.15.3:0/633120547 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda7c137930 con 0x7fda7c09d630 2023-10-03T10:13:48.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.468+0000 7fda71ffb700 1 -- 172.21.15.3:0/633120547 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda780036a0 con 0x7fda7c09d630 2023-10-03T10:13:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.468+0000 7fda833b0700 1 -- 172.21.15.3:0/633120547 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda7c1358a0 con 0x7fda7c09d630 2023-10-03T10:13:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.469+0000 7fda71ffb700 1 -- 172.21.15.3:0/633120547 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fda78021030 con 0x7fda7c09d630 2023-10-03T10:13:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.469+0000 7fda833b0700 1 -- 172.21.15.3:0/633120547 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda5c0052e0 con 0x7fda7c09d630 2023-10-03T10:13:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.470+0000 7fda71ffb700 1 -- 172.21.15.3:0/633120547 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fda780406f0 con 0x7fda7c09d630 2023-10-03T10:13:48.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.479+0000 7fda71ffb700 1 -- 172.21.15.3:0/633120547 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fda780199a0 con 0x7fda7c09d630 2023-10-03T10:13:48.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.504+0000 7f0142cdf700 1 Processor -- start 2023-10-03T10:13:48.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.504+0000 7f0142cdf700 1 -- start start 2023-10-03T10:13:48.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.505+0000 7f0142cdf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f013c0a0a30 0x7f013c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:48.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.505+0000 7f0142cdf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f013c0a1340 0x7f013c133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:48.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.505+0000 7f0142cdf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f013c133fa0 0x7f013c134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:48.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.505+0000 7f0142cdf700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f013c09edd0 con 0x7f013c0a1340 2023-10-03T10:13:48.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.505+0000 7f0142cdf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f013c09e1f0 con 0x7f013c0a0a30 2023-10-03T10:13:48.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.505+0000 7f0142cdf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f013c09dc50 con 0x7f013c133fa0 2023-10-03T10:13:48.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.505+0000 7f0140a7b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f013c0a0a30 0x7f013c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:48.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.505+0000 7f013bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f013c0a1340 0x7f013c133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:48.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.505+0000 7f013bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f013c0a1340 0x7f013c133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46808/0 (socket says 172.21.15.3:46808) 2023-10-03T10:13:48.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.505+0000 7f0140a7b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f013c0a0a30 0x7f013c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46346/0 (socket says 172.21.15.3:46346) 2023-10-03T10:13:48.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.505+0000 7f013bfff700 1 -- 172.21.15.3:0/2239082882 learned_addr learned my addr 172.21.15.3:0/2239082882 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:48.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.505+0000 7f014127c700 1 --2- 172.21.15.3:0/2239082882 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f013c133fa0 0x7f013c134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:48.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.506+0000 7f013bfff700 1 -- 172.21.15.3:0/2239082882 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f013c133fa0 msgr2=0x7f013c134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:48.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.506+0000 7f013bfff700 1 --2- 172.21.15.3:0/2239082882 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f013c133fa0 0x7f013c134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.506+0000 7f013bfff700 1 -- 172.21.15.3:0/2239082882 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f013c0a0a30 msgr2=0x7f013c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:48.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.506+0000 7f013bfff700 1 --2- 172.21.15.3:0/2239082882 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f013c0a0a30 0x7f013c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.506+0000 7f013bfff700 1 -- 172.21.15.3:0/2239082882 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f013c0a1fe0 con 0x7f013c0a1340 2023-10-03T10:13:48.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.506+0000 7f0140a7b700 1 --2- 172.21.15.3:0/2239082882 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f013c0a0a30 0x7f013c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:48.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.506+0000 7f013bfff700 1 --2- 172.21.15.3:0/2239082882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f013c0a1340 0x7f013c133a60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f01280098f0 tx=0x7f0128009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21b46ebf1310eb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:48.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.506+0000 7f013b7fe700 1 -- 172.21.15.3:0/2239082882 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0128011830 con 0x7f013c0a1340 2023-10-03T10:13:48.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.506+0000 7f013b7fe700 1 -- 172.21.15.3:0/2239082882 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0128011990 con 0x7f013c0a1340 2023-10-03T10:13:48.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.506+0000 7f013b7fe700 1 -- 172.21.15.3:0/2239082882 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0128011d20 con 0x7f013c0a1340 2023-10-03T10:13:48.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.507+0000 7f0142cdf700 1 -- 172.21.15.3:0/2239082882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f013c0a1340 msgr2=0x7f013c133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:48.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.507+0000 7f0142cdf700 1 --2- 172.21.15.3:0/2239082882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f013c0a1340 0x7f013c133a60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f01280098f0 tx=0x7f0128009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:48.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.507+0000 7f0142cdf700 1 -- 172.21.15.3:0/2239082882 shutdown_connections 2023-10-03T10:13:48.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.507+0000 7f0142cdf700 1 --2- 172.21.15.3:0/2239082882 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f013c133fa0 0x7f013c134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.507+0000 7f0142cdf700 1 --2- 172.21.15.3:0/2239082882 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f013c0a0a30 0x7f013c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.507+0000 7f0142cdf700 1 --2- 172.21.15.3:0/2239082882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f013c0a1340 0x7f013c133a60 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.507+0000 7f0142cdf700 1 -- 172.21.15.3:0/2239082882 >> 172.21.15.3:0/2239082882 conn(0x7f013c0a6550 msgr2=0x7f013c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:48.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.507+0000 7f0142cdf700 1 -- 172.21.15.3:0/2239082882 shutdown_connections 2023-10-03T10:13:48.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.507+0000 7f0142cdf700 1 -- 172.21.15.3:0/2239082882 wait complete. 2023-10-03T10:13:48.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.508+0000 7f0142cdf700 1 Processor -- start 2023-10-03T10:13:48.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.508+0000 7f0142cdf700 1 -- start start 2023-10-03T10:13:48.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.509+0000 7f0142cdf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f013c0a0a30 0x7f013c1214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:48.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.509+0000 7f0142cdf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f013c0a1340 0x7f013c121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:48.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.509+0000 7f0142cdf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f013c133fa0 0x7f013c121f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:48.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.509+0000 7f0142cdf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013c124d20 con 0x7f013c133fa0 2023-10-03T10:13:48.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.509+0000 7f0142cdf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f013c0a37e0 con 0x7f013c0a0a30 2023-10-03T10:13:48.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.509+0000 7f0142cdf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f013c123200 con 0x7f013c0a1340 2023-10-03T10:13:48.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.509+0000 7f0140a7b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f013c0a0a30 0x7f013c1214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:48.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.509+0000 7f0140a7b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f013c0a0a30 0x7f013c1214e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46350/0 (socket says 172.21.15.3:46350) 2023-10-03T10:13:48.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.509+0000 7f0140a7b700 1 -- 172.21.15.3:0/2069708063 learned_addr learned my addr 172.21.15.3:0/2069708063 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:48.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.509+0000 7f014127c700 1 --2- 172.21.15.3:0/2069708063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f013c133fa0 0x7f013c121f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:48.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.509+0000 7f013bfff700 1 --2- 172.21.15.3:0/2069708063 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f013c0a1340 0x7f013c121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:48.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.509+0000 7f014127c700 1 -- 172.21.15.3:0/2069708063 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f013c0a1340 msgr2=0x7f013c121a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:48.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.509+0000 7f014127c700 1 --2- 172.21.15.3:0/2069708063 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f013c0a1340 0x7f013c121a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.509+0000 7f014127c700 1 -- 172.21.15.3:0/2069708063 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f013c0a0a30 msgr2=0x7f013c1214e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:48.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.509+0000 7f014127c700 1 --2- 172.21.15.3:0/2069708063 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f013c0a0a30 0x7f013c1214e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.509+0000 7f014127c700 1 -- 172.21.15.3:0/2069708063 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01300094a0 con 0x7f013c133fa0 2023-10-03T10:13:48.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.509+0000 7f013bfff700 1 --2- 172.21.15.3:0/2069708063 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f013c0a1340 0x7f013c121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:48.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.509+0000 7f014127c700 1 --2- 172.21.15.3:0/2069708063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f013c133fa0 0x7f013c121f60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f013400eba0 tx=0x7f013400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:48.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.510+0000 7f01397fa700 1 -- 172.21.15.3:0/2069708063 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134009a80 con 0x7f013c133fa0 2023-10-03T10:13:48.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.510+0000 7f01397fa700 1 -- 172.21.15.3:0/2069708063 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013400f040 con 0x7f013c133fa0 2023-10-03T10:13:48.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.510+0000 7f0142cdf700 1 -- 172.21.15.3:0/2069708063 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0128009550 con 0x7f013c133fa0 2023-10-03T10:13:48.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.510+0000 7f01397fa700 1 -- 172.21.15.3:0/2069708063 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013400cc40 con 0x7f013c133fa0 2023-10-03T10:13:48.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.510+0000 7f0142cdf700 1 -- 172.21.15.3:0/2069708063 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f013c132000 con 0x7f013c133fa0 2023-10-03T10:13:48.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.511+0000 7f01397fa700 1 -- 172.21.15.3:0/2069708063 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0134014480 con 0x7f013c133fa0 2023-10-03T10:13:48.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.511+0000 7f0142cdf700 1 -- 172.21.15.3:0/2069708063 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f013c074ea0 con 0x7f013c133fa0 2023-10-03T10:13:48.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.511+0000 7f01397fa700 1 -- 172.21.15.3:0/2069708063 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0134040030 con 0x7f013c133fa0 2023-10-03T10:13:48.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.520+0000 7f01397fa700 1 -- 172.21.15.3:0/2069708063 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0134007720 con 0x7f013c133fa0 2023-10-03T10:13:48.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.632+0000 7f068984b700 1 -- 172.21.15.3:0/1933924815 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f06841a9cd0 con 0x7f068409e410 2023-10-03T10:13:48.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.632+0000 7f066bfff700 1 -- 172.21.15.3:0/1933924815 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f06841a9cd0 con 0x7f068409e410 2023-10-03T10:13:48.633 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:48.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.635+0000 7f068984b700 1 -- 172.21.15.3:0/1933924815 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f068409e410 msgr2=0x7f06841c7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:48.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.635+0000 7f068984b700 1 --2- 172.21.15.3:0/1933924815 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f068409e410 0x7f06841c7f80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f067c00b680 tx=0x7f067c004ad0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.635+0000 7f068984b700 1 -- 172.21.15.3:0/1933924815 shutdown_connections 2023-10-03T10:13:48.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.635+0000 7f068984b700 1 --2- 172.21.15.3:0/1933924815 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f068409e410 0x7f06841c7f80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.635+0000 7f068984b700 1 --2- 172.21.15.3:0/1933924815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f068409d6f0 0x7f06841c50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.635+0000 7f068984b700 1 --2- 172.21.15.3:0/1933924815 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f068409e000 0x7f06841c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.635+0000 7f068984b700 1 -- 172.21.15.3:0/1933924815 >> 172.21.15.3:0/1933924815 conn(0x7f0684066f50 msgr2=0x7f068406f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:48.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.635+0000 7f068984b700 1 -- 172.21.15.3:0/1933924815 shutdown_connections 2023-10-03T10:13:48.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.635+0000 7f068984b700 1 -- 172.21.15.3:0/1933924815 wait complete. 2023-10-03T10:13:48.649 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:48.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.689+0000 7fda833b0700 1 -- 172.21.15.3:0/633120547 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fda5c005590 con 0x7fda7c09d630 2023-10-03T10:13:48.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.689+0000 7fda71ffb700 1 -- 172.21.15.3:0/633120547 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fda7801b430 con 0x7fda7c09d630 2023-10-03T10:13:48.689 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:48.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.691+0000 7fda833b0700 1 -- 172.21.15.3:0/633120547 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda7c09d630 msgr2=0x7fda7c133790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:48.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.691+0000 7fda833b0700 1 --2- 172.21.15.3:0/633120547 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda7c09d630 0x7fda7c133790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fda78009840 tx=0x7fda78004750 comp rx=0 tx=0).stop 2023-10-03T10:13:48.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.691+0000 7fda833b0700 1 -- 172.21.15.3:0/633120547 shutdown_connections 2023-10-03T10:13:48.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.691+0000 7fda833b0700 1 --2- 172.21.15.3:0/633120547 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fda7c138d80 0x7fda7c134210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.691+0000 7fda833b0700 1 --2- 172.21.15.3:0/633120547 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda7c09d630 0x7fda7c133790 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.691+0000 7fda833b0700 1 --2- 172.21.15.3:0/633120547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda7c0a0a30 0x7fda7c133cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.691+0000 7fda833b0700 1 -- 172.21.15.3:0/633120547 >> 172.21.15.3:0/633120547 conn(0x7fda7c0a3050 msgr2=0x7fda7c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:48.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.691+0000 7fda833b0700 1 -- 172.21.15.3:0/633120547 shutdown_connections 2023-10-03T10:13:48.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.691+0000 7fda833b0700 1 -- 172.21.15.3:0/633120547 wait complete. 2023-10-03T10:13:48.705 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:48.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.730+0000 7f0142cdf700 1 -- 172.21.15.3:0/2069708063 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f013c0a28e0 con 0x7f013c133fa0 2023-10-03T10:13:48.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.730+0000 7f01397fa700 1 -- 172.21.15.3:0/2069708063 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0134007330 con 0x7f013c133fa0 2023-10-03T10:13:48.730 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:48.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.732+0000 7f0142cdf700 1 -- 172.21.15.3:0/2069708063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f013c133fa0 msgr2=0x7f013c121f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:48.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.732+0000 7f0142cdf700 1 --2- 172.21.15.3:0/2069708063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f013c133fa0 0x7f013c121f60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f013400eba0 tx=0x7f013400eeb0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.732+0000 7f0142cdf700 1 -- 172.21.15.3:0/2069708063 shutdown_connections 2023-10-03T10:13:48.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.732+0000 7f0142cdf700 1 --2- 172.21.15.3:0/2069708063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f013c133fa0 0x7f013c121f60 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.732+0000 7f0142cdf700 1 --2- 172.21.15.3:0/2069708063 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f013c0a0a30 0x7f013c1214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.732+0000 7f0142cdf700 1 --2- 172.21.15.3:0/2069708063 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f013c0a1340 0x7f013c121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:48.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.732+0000 7f0142cdf700 1 -- 172.21.15.3:0/2069708063 >> 172.21.15.3:0/2069708063 conn(0x7f013c0a6550 msgr2=0x7f013c133630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:48.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.732+0000 7f0142cdf700 1 -- 172.21.15.3:0/2069708063 shutdown_connections 2023-10-03T10:13:48.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:48.732+0000 7f0142cdf700 1 -- 172.21.15.3:0/2069708063 wait complete. 2023-10-03T10:13:48.746 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:49.383 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:49.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.531+0000 7f9de10d2700 1 Processor -- start 2023-10-03T10:13:49.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.531+0000 7f9de10d2700 1 -- start start 2023-10-03T10:13:49.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.531+0000 7f9de10d2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ddc09e930 0x7f9ddc09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:49.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.531+0000 7f9de10d2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9ddc09f240 0x7f9ddc1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:49.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.531+0000 7f9de10d2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ddc070940 con 0x7f9ddc09f650 2023-10-03T10:13:49.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.531+0000 7f9de10d2700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ddc06fd60 con 0x7f9ddc09f240 2023-10-03T10:13:49.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.531+0000 7f9de10d2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc06fa80 con 0x7f9ddc09e930 2023-10-03T10:13:49.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.531+0000 7f9dda59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9ddc09f240 0x7f9ddc1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:49.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.531+0000 7f9ddad9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ddc09e930 0x7f9ddc09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:49.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.531+0000 7f9dda59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9ddc09f240 0x7f9ddc1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46830/0 (socket says 172.21.15.3:46830) 2023-10-03T10:13:49.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.531+0000 7f9ddad9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ddc09e930 0x7f9ddc09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46360/0 (socket says 172.21.15.3:46360) 2023-10-03T10:13:49.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.531+0000 7f9dda59c700 1 -- 172.21.15.3:0/192657334 learned_addr learned my addr 172.21.15.3:0/192657334 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:49.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.532+0000 7f9ddad9d700 1 -- 172.21.15.3:0/192657334 >> v2:172.21.15.3:3300/0 conn(0x7f9ddc09f240 msgr2=0x7f9ddc1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:49.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.532+0000 7f9ddad9d700 1 --2- 172.21.15.3:0/192657334 >> v2:172.21.15.3:3300/0 conn(0x7f9ddc09f240 0x7f9ddc1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:49.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.532+0000 7f9ddad9d700 1 -- 172.21.15.3:0/192657334 >> v1:172.21.15.3:6789/0 conn(0x7f9ddc09f650 legacy=0x7f9ddc131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:49.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.532+0000 7f9ddad9d700 1 -- 172.21.15.3:0/192657334 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ddc09fb60 con 0x7f9ddc09e930 2023-10-03T10:13:49.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.532+0000 7f9dda59c700 1 --2- 172.21.15.3:0/192657334 >> v2:172.21.15.3:3300/0 conn(0x7f9ddc09f240 0x7f9ddc1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:49.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.532+0000 7f9dd9d9b700 1 -- 172.21.15.3:0/192657334 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4437638 0 0) 0x7f9ddc070940 con 0x7f9ddc09f650 2023-10-03T10:13:49.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.532+0000 7f9ddad9d700 1 --2- 172.21.15.3:0/192657334 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ddc09e930 0x7f9ddc09ed00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc009840 tx=0x7f9dcc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92fa971037bf7e94 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:49.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.532+0000 7f9dd9d9b700 1 -- 172.21.15.3:0/192657334 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc011830 con 0x7f9ddc09e930 2023-10-03T10:13:49.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.532+0000 7f9dd9d9b700 1 -- 172.21.15.3:0/192657334 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dcc011990 con 0x7f9ddc09e930 2023-10-03T10:13:49.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.532+0000 7f9dd9d9b700 1 -- 172.21.15.3:0/192657334 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc011d20 con 0x7f9ddc09e930 2023-10-03T10:13:49.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.533+0000 7f9de10d2700 1 -- 172.21.15.3:0/192657334 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ddc09e930 msgr2=0x7f9ddc09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:49.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.533+0000 7f9de10d2700 1 --2- 172.21.15.3:0/192657334 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ddc09e930 0x7f9ddc09ed00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc009840 tx=0x7f9dcc009b50 comp rx=0 tx=0).stop 2023-10-03T10:13:49.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.533+0000 7f9de10d2700 1 -- 172.21.15.3:0/192657334 shutdown_connections 2023-10-03T10:13:49.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.533+0000 7f9de10d2700 1 --2- 172.21.15.3:0/192657334 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ddc09e930 0x7f9ddc09ed00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:49.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.533+0000 7f9de10d2700 1 --2- 172.21.15.3:0/192657334 >> v2:172.21.15.3:3300/0 conn(0x7f9ddc09f240 0x7f9ddc1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:49.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.533+0000 7f9de10d2700 1 -- 172.21.15.3:0/192657334 >> 172.21.15.3:0/192657334 conn(0x7f9ddc0a4210 msgr2=0x7f9ddc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:49.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.533+0000 7f9de10d2700 1 -- 172.21.15.3:0/192657334 shutdown_connections 2023-10-03T10:13:49.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.533+0000 7f9de10d2700 1 -- 172.21.15.3:0/192657334 wait complete. 2023-10-03T10:13:49.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.534+0000 7f9de10d2700 1 Processor -- start 2023-10-03T10:13:49.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.534+0000 7f9de10d2700 1 -- start start 2023-10-03T10:13:49.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.535+0000 7f9de10d2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ddc09e930 0x7f9ddc1c2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:49.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.535+0000 7f9de10d2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ddc09f240 0x7f9ddc1c3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:49.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.535+0000 7f9de10d2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ddc09f650 0x7f9ddc1c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:49.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.535+0000 7f9de10d2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc1a7400 con 0x7f9ddc09f240 2023-10-03T10:13:49.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.535+0000 7f9de10d2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc1a75e0 con 0x7f9ddc09e930 2023-10-03T10:13:49.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.535+0000 7f9de10d2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc1c38a0 con 0x7f9ddc09f650 2023-10-03T10:13:49.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.535+0000 7f9ddad9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ddc09e930 0x7f9ddc1c2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:49.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.535+0000 7f9dda59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ddc09f240 0x7f9ddc1c3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.535+0000 7f9ddad9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ddc09e930 0x7f9ddc1c2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46372/0 (socket says 172.21.15.3:46372) 2023-10-03T10:13:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.535+0000 7f9ddad9d700 1 -- 172.21.15.3:0/803702008 learned_addr learned my addr 172.21.15.3:0/803702008 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.535+0000 7f9ddb59e700 1 --2- 172.21.15.3:0/803702008 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ddc09f650 0x7f9ddc1c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.535+0000 7f9ddad9d700 1 -- 172.21.15.3:0/803702008 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ddc09f650 msgr2=0x7f9ddc1c5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.535+0000 7f9ddad9d700 1 --2- 172.21.15.3:0/803702008 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ddc09f650 0x7f9ddc1c5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.535+0000 7f9ddad9d700 1 -- 172.21.15.3:0/803702008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ddc09f240 msgr2=0x7f9ddc1c3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.535+0000 7f9ddad9d700 1 --2- 172.21.15.3:0/803702008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ddc09f240 0x7f9ddc1c3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.535+0000 7f9ddad9d700 1 -- 172.21.15.3:0/803702008 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd0009550 con 0x7f9ddc09e930 2023-10-03T10:13:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.535+0000 7f9ddb59e700 1 --2- 172.21.15.3:0/803702008 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ddc09f650 0x7f9ddc1c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.535+0000 7f9dda59c700 1 --2- 172.21.15.3:0/803702008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ddc09f240 0x7f9ddc1c3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.535+0000 7f9ddad9d700 1 --2- 172.21.15.3:0/803702008 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ddc09e930 0x7f9ddc1c2e20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0090b0 tx=0x7f9dcc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.536+0000 7f9dc37fe700 1 -- 172.21.15.3:0/803702008 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc0036a0 con 0x7f9ddc09e930 2023-10-03T10:13:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.536+0000 7f9dc37fe700 1 -- 172.21.15.3:0/803702008 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dcc01f070 con 0x7f9ddc09e930 2023-10-03T10:13:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.536+0000 7f9de10d2700 1 -- 172.21.15.3:0/803702008 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dcc0094a0 con 0x7f9ddc09e930 2023-10-03T10:13:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.536+0000 7f9dc37fe700 1 -- 172.21.15.3:0/803702008 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc003a60 con 0x7f9ddc09e930 2023-10-03T10:13:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.536+0000 7f9de10d2700 1 -- 172.21.15.3:0/803702008 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ddc0a0190 con 0x7f9ddc09e930 2023-10-03T10:13:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.536+0000 7f9dc37fe700 1 -- 172.21.15.3:0/803702008 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9dcc003bc0 con 0x7f9ddc09e930 2023-10-03T10:13:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.537+0000 7f9de10d2700 1 -- 172.21.15.3:0/803702008 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ddc074ea0 con 0x7f9ddc09e930 2023-10-03T10:13:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.537+0000 7f9dc37fe700 1 -- 172.21.15.3:0/803702008 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9dcc021070 con 0x7f9ddc09e930 2023-10-03T10:13:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.541+0000 7f9dc37fe700 1 -- 172.21.15.3:0/803702008 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9dcc010790 con 0x7f9ddc09e930 2023-10-03T10:13:49.650 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:49.706 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:49.747 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:49.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.749+0000 7f9de10d2700 1 -- 172.21.15.3:0/803702008 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9ddc1a56d0 con 0x7f9ddc09e930 2023-10-03T10:13:49.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.750+0000 7f9dc37fe700 1 -- 172.21.15.3:0/803702008 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9dcc015030 con 0x7f9ddc09e930 2023-10-03T10:13:49.750 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:49.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.752+0000 7f9de10d2700 1 -- 172.21.15.3:0/803702008 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ddc09e930 msgr2=0x7f9ddc1c2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:49.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.752+0000 7f9de10d2700 1 --2- 172.21.15.3:0/803702008 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ddc09e930 0x7f9ddc1c2e20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0090b0 tx=0x7f9dcc004750 comp rx=0 tx=0).stop 2023-10-03T10:13:49.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.752+0000 7f9de10d2700 1 -- 172.21.15.3:0/803702008 shutdown_connections 2023-10-03T10:13:49.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.752+0000 7f9de10d2700 1 --2- 172.21.15.3:0/803702008 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ddc09f650 0x7f9ddc1c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:49.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.752+0000 7f9de10d2700 1 --2- 172.21.15.3:0/803702008 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ddc09e930 0x7f9ddc1c2e20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:49.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.752+0000 7f9de10d2700 1 --2- 172.21.15.3:0/803702008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ddc09f240 0x7f9ddc1c3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:49.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.752+0000 7f9de10d2700 1 -- 172.21.15.3:0/803702008 >> 172.21.15.3:0/803702008 conn(0x7f9ddc0a4210 msgr2=0x7f9ddc1b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:49.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.753+0000 7f9de10d2700 1 -- 172.21.15.3:0/803702008 shutdown_connections 2023-10-03T10:13:49.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.753+0000 7f9de10d2700 1 -- 172.21.15.3:0/803702008 wait complete. 2023-10-03T10:13:49.771 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:49.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.794+0000 7ff72a1f9700 1 Processor -- start 2023-10-03T10:13:49.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.794+0000 7ff72a1f9700 1 -- start start 2023-10-03T10:13:49.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.795+0000 7ff72a1f9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff7241305f0 0x7ff7241309c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:49.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.795+0000 7ff72a1f9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff724070160 0x7ff724138540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:49.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.795+0000 7ff72a1f9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7241301c0 con 0x7ff72406f660 2023-10-03T10:13:49.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.795+0000 7ff72a1f9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff72409a2e0 con 0x7ff724070160 2023-10-03T10:13:49.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.795+0000 7ff72a1f9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff724071ff0 con 0x7ff7241305f0 2023-10-03T10:13:49.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.795+0000 7ff7237fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff7241305f0 0x7ff7241309c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:49.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.796+0000 7ff7237fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff7241305f0 0x7ff7241309c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55770/0 (socket says 172.21.15.3:55770) 2023-10-03T10:13:49.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.796+0000 7ff7237fe700 1 -- 172.21.15.3:0/1833410693 learned_addr learned my addr 172.21.15.3:0/1833410693 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:49.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.796+0000 7ff723fff700 1 --2- 172.21.15.3:0/1833410693 >> v2:172.21.15.3:3300/0 conn(0x7ff724070160 0x7ff724138540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:49.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.796+0000 7ff7237fe700 1 -- 172.21.15.3:0/1833410693 >> v2:172.21.15.3:3300/0 conn(0x7ff724070160 msgr2=0x7ff724138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:49.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.796+0000 7ff7237fe700 1 --2- 172.21.15.3:0/1833410693 >> v2:172.21.15.3:3300/0 conn(0x7ff724070160 0x7ff724138540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:49.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.796+0000 7ff7237fe700 1 -- 172.21.15.3:0/1833410693 >> v1:172.21.15.3:6789/0 conn(0x7ff72406f660 legacy=0x7ff72406fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:49.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.796+0000 7ff7237fe700 1 -- 172.21.15.3:0/1833410693 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff724070e70 con 0x7ff7241305f0 2023-10-03T10:13:49.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.796+0000 7ff723fff700 1 --2- 172.21.15.3:0/1833410693 >> v2:172.21.15.3:3300/0 conn(0x7ff724070160 0x7ff724138540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:49.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.796+0000 7ff7237fe700 1 --2- 172.21.15.3:0/1833410693 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff7241305f0 0x7ff7241309c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff714009920 tx=0x7ff714009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3e69c3d62a01c89 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:49.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.796+0000 7ff7227fc700 1 -- 172.21.15.3:0/1833410693 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff714011830 con 0x7ff7241305f0 2023-10-03T10:13:49.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.796+0000 7ff7227fc700 1 -- 172.21.15.3:0/1833410693 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff714011990 con 0x7ff7241305f0 2023-10-03T10:13:49.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.796+0000 7ff7227fc700 1 -- 172.21.15.3:0/1833410693 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff714011d20 con 0x7ff7241305f0 2023-10-03T10:13:49.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.797+0000 7ff72a1f9700 1 -- 172.21.15.3:0/1833410693 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff7241305f0 msgr2=0x7ff7241309c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:49.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.797+0000 7ff72a1f9700 1 --2- 172.21.15.3:0/1833410693 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff7241305f0 0x7ff7241309c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff714009920 tx=0x7ff714009c30 comp rx=0 tx=0).stop 2023-10-03T10:13:49.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.797+0000 7ff72a1f9700 1 -- 172.21.15.3:0/1833410693 shutdown_connections 2023-10-03T10:13:49.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.797+0000 7ff72a1f9700 1 --2- 172.21.15.3:0/1833410693 >> v2:172.21.15.3:3300/0 conn(0x7ff724070160 0x7ff724138540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:49.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.797+0000 7ff72a1f9700 1 --2- 172.21.15.3:0/1833410693 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff7241305f0 0x7ff7241309c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:49.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.797+0000 7ff72a1f9700 1 -- 172.21.15.3:0/1833410693 >> 172.21.15.3:0/1833410693 conn(0x7ff7240a2aa0 msgr2=0x7ff7240a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:49.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.797+0000 7ff72a1f9700 1 -- 172.21.15.3:0/1833410693 shutdown_connections 2023-10-03T10:13:49.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.797+0000 7ff72a1f9700 1 -- 172.21.15.3:0/1833410693 wait complete. 2023-10-03T10:13:49.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.799+0000 7ff72a1f9700 1 Processor -- start 2023-10-03T10:13:49.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.799+0000 7ff72a1f9700 1 -- start start 2023-10-03T10:13:49.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.800+0000 7ff72a1f9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff72406f660 0x7ff724139a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:49.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.800+0000 7ff72a1f9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff724070160 0x7ff724139fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:49.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.800+0000 7ff72a1f9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7241305f0 0x7ff72413a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:49.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.800+0000 7ff72a1f9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72413b9c0 con 0x7ff72406f660 2023-10-03T10:13:49.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.800+0000 7ff72a1f9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff724071890 con 0x7ff7241305f0 2023-10-03T10:13:49.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.800+0000 7ff72a1f9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff724118cd0 con 0x7ff724070160 2023-10-03T10:13:49.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.800+0000 7ff723fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7241305f0 0x7ff72413a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:49.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.800+0000 7ff723fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7241305f0 0x7ff72413a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46374/0 (socket says 172.21.15.3:46374) 2023-10-03T10:13:49.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.800+0000 7ff723fff700 1 -- 172.21.15.3:0/2337376192 learned_addr learned my addr 172.21.15.3:0/2337376192 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:49.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.800+0000 7ff7237fe700 1 --2- 172.21.15.3:0/2337376192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff72406f660 0x7ff724139a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:49.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.800+0000 7ff722ffd700 1 --2- 172.21.15.3:0/2337376192 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff724070160 0x7ff724139fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:49.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.800+0000 7ff723fff700 1 -- 172.21.15.3:0/2337376192 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff724070160 msgr2=0x7ff724139fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:49.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.800+0000 7ff723fff700 1 --2- 172.21.15.3:0/2337376192 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff724070160 0x7ff724139fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:49.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.800+0000 7ff723fff700 1 -- 172.21.15.3:0/2337376192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff72406f660 msgr2=0x7ff724139a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:49.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.800+0000 7ff723fff700 1 --2- 172.21.15.3:0/2337376192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff72406f660 0x7ff724139a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:49.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.800+0000 7ff723fff700 1 -- 172.21.15.3:0/2337376192 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff71c009a70 con 0x7ff7241305f0 2023-10-03T10:13:49.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.800+0000 7ff722ffd700 1 --2- 172.21.15.3:0/2337376192 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff724070160 0x7ff724139fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:49.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.800+0000 7ff7237fe700 1 --2- 172.21.15.3:0/2337376192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff72406f660 0x7ff724139a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:49.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.800+0000 7ff723fff700 1 --2- 172.21.15.3:0/2337376192 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7241305f0 0x7ff72413a510 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff71c00efc0 tx=0x7ff71c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:49.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.801+0000 7ff70bfff700 1 -- 172.21.15.3:0/2337376192 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71c00ca90 con 0x7ff7241305f0 2023-10-03T10:13:49.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.801+0000 7ff70bfff700 1 -- 172.21.15.3:0/2337376192 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff71c00f040 con 0x7ff7241305f0 2023-10-03T10:13:49.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.801+0000 7ff72a1f9700 1 -- 172.21.15.3:0/2337376192 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff714009550 con 0x7ff7241305f0 2023-10-03T10:13:49.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.801+0000 7ff70bfff700 1 -- 172.21.15.3:0/2337376192 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71c00ce10 con 0x7ff7241305f0 2023-10-03T10:13:49.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.801+0000 7ff72a1f9700 1 -- 172.21.15.3:0/2337376192 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff724119b40 con 0x7ff7241305f0 2023-10-03T10:13:49.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.802+0000 7ff70bfff700 1 -- 172.21.15.3:0/2337376192 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff71c014480 con 0x7ff7241305f0 2023-10-03T10:13:49.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.802+0000 7ff70bfff700 1 -- 172.21.15.3:0/2337376192 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff71c03fe10 con 0x7ff7241305f0 2023-10-03T10:13:49.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.802+0000 7ff72a1f9700 1 -- 172.21.15.3:0/2337376192 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff72409dd00 con 0x7ff7241305f0 2023-10-03T10:13:49.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.807+0000 7ff70bfff700 1 -- 172.21.15.3:0/2337376192 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff71c012930 con 0x7ff7241305f0 2023-10-03T10:13:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.853+0000 7f4af8e7d700 1 Processor -- start 2023-10-03T10:13:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.853+0000 7f4af8e7d700 1 -- start start 2023-10-03T10:13:49.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.854+0000 7f4af8e7d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4af41316c0 0x7f4af4131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.854+0000 7f4af8e7d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4af409ed20 0x7f4af406eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.854+0000 7f4af8e7d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4af409b420 con 0x7f4af4131fd0 2023-10-03T10:13:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.854+0000 7f4af8e7d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4af41314a0 con 0x7f4af41316c0 2023-10-03T10:13:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.854+0000 7f4af8e7d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af4099ce0 con 0x7f4af409ed20 2023-10-03T10:13:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.854+0000 7f4af2d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4af409ed20 0x7f4af406eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.854+0000 7f4af259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4af41316c0 0x7f4af4131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.854+0000 7f4af2d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4af409ed20 0x7f4af406eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46382/0 (socket says 172.21.15.3:46382) 2023-10-03T10:13:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.854+0000 7f4af2d9d700 1 -- 172.21.15.3:0/2550480578 learned_addr learned my addr 172.21.15.3:0/2550480578 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.854+0000 7f4af259c700 1 -- 172.21.15.3:0/2550480578 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4af409ed20 msgr2=0x7f4af406eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:49.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.854+0000 7f4af259c700 1 --2- 172.21.15.3:0/2550480578 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4af409ed20 0x7f4af406eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:49.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.854+0000 7f4af259c700 1 -- 172.21.15.3:0/2550480578 >> v1:172.21.15.3:6789/0 conn(0x7f4af4131fd0 legacy=0x7f4af406e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:49.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.854+0000 7f4af259c700 1 -- 172.21.15.3:0/2550480578 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af409a510 con 0x7f4af41316c0 2023-10-03T10:13:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.855+0000 7f4af259c700 1 --2- 172.21.15.3:0/2550480578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af41316c0 0x7f4af4131a90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4009840 tx=0x7f4ae4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0c0ebefd096433 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.855+0000 7f4af159a700 1 -- 172.21.15.3:0/2550480578 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4011830 con 0x7f4af41316c0 2023-10-03T10:13:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.855+0000 7f4af159a700 1 -- 172.21.15.3:0/2550480578 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae4011990 con 0x7f4af41316c0 2023-10-03T10:13:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.855+0000 7f4af159a700 1 -- 172.21.15.3:0/2550480578 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4011d20 con 0x7f4af41316c0 2023-10-03T10:13:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.856+0000 7f4af8e7d700 1 -- 172.21.15.3:0/2550480578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af41316c0 msgr2=0x7f4af4131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.856+0000 7f4af8e7d700 1 --2- 172.21.15.3:0/2550480578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af41316c0 0x7f4af4131a90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4009840 tx=0x7f4ae4009b50 comp rx=0 tx=0).stop 2023-10-03T10:13:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.856+0000 7f4af8e7d700 1 -- 172.21.15.3:0/2550480578 shutdown_connections 2023-10-03T10:13:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.856+0000 7f4af8e7d700 1 --2- 172.21.15.3:0/2550480578 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4af409ed20 0x7f4af406eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.856+0000 7f4af8e7d700 1 --2- 172.21.15.3:0/2550480578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af41316c0 0x7f4af4131a90 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.856+0000 7f4af8e7d700 1 -- 172.21.15.3:0/2550480578 >> 172.21.15.3:0/2550480578 conn(0x7f4af40a2aa0 msgr2=0x7f4af40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.856+0000 7f4af8e7d700 1 -- 172.21.15.3:0/2550480578 shutdown_connections 2023-10-03T10:13:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.856+0000 7f4af8e7d700 1 -- 172.21.15.3:0/2550480578 wait complete. 2023-10-03T10:13:49.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.857+0000 7f4af8e7d700 1 Processor -- start 2023-10-03T10:13:49.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.857+0000 7f4af8e7d700 1 -- start start 2023-10-03T10:13:49.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.858+0000 7f4af8e7d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4af409ed20 0x7f4af406f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:49.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.858+0000 7f4af8e7d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4af41316c0 0x7f4af406fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:49.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.858+0000 7f4af8e7d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af4131fd0 0x7f4af40700d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:49.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.858+0000 7f4af8e7d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af411f100 con 0x7f4af4131fd0 2023-10-03T10:13:49.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.858+0000 7f4af8e7d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af411e2e0 con 0x7f4af409ed20 2023-10-03T10:13:49.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.858+0000 7f4af8e7d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4af411f2e0 con 0x7f4af41316c0 2023-10-03T10:13:49.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.858+0000 7f4af259c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4af409ed20 0x7f4af406f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:49.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.858+0000 7f4af259c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4af409ed20 0x7f4af406f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46390/0 (socket says 172.21.15.3:46390) 2023-10-03T10:13:49.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.858+0000 7f4af259c700 1 -- 172.21.15.3:0/1245982967 learned_addr learned my addr 172.21.15.3:0/1245982967 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:49.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.858+0000 7f4af2d9d700 1 --2- 172.21.15.3:0/1245982967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af4131fd0 0x7f4af40700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:49.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.858+0000 7f4af259c700 1 -- 172.21.15.3:0/1245982967 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4af41316c0 msgr2=0x7f4af406fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:49.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.858+0000 7f4af259c700 1 --2- 172.21.15.3:0/1245982967 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4af41316c0 0x7f4af406fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:49.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.858+0000 7f4af259c700 1 -- 172.21.15.3:0/1245982967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af4131fd0 msgr2=0x7f4af40700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:49.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.858+0000 7f4af259c700 1 --2- 172.21.15.3:0/1245982967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af4131fd0 0x7f4af40700d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:49.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.858+0000 7f4af259c700 1 -- 172.21.15.3:0/1245982967 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae40094a0 con 0x7f4af409ed20 2023-10-03T10:13:49.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.858+0000 7f4af259c700 1 --2- 172.21.15.3:0/1245982967 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4af409ed20 0x7f4af406f650 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4009fd0 tx=0x7f4ae4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:49.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.859+0000 7f4ae2ffd700 1 -- 172.21.15.3:0/1245982967 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae40036a0 con 0x7f4af409ed20 2023-10-03T10:13:49.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.859+0000 7f4ae2ffd700 1 -- 172.21.15.3:0/1245982967 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae401f070 con 0x7f4af409ed20 2023-10-03T10:13:49.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.859+0000 7f4af8e7d700 1 -- 172.21.15.3:0/1245982967 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af4070ab0 con 0x7f4af409ed20 2023-10-03T10:13:49.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.859+0000 7f4ae2ffd700 1 -- 172.21.15.3:0/1245982967 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4003a60 con 0x7f4af409ed20 2023-10-03T10:13:49.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.859+0000 7f4af8e7d700 1 -- 172.21.15.3:0/1245982967 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af4099f20 con 0x7f4af409ed20 2023-10-03T10:13:49.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.860+0000 7f4af8e7d700 1 -- 172.21.15.3:0/1245982967 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af4074ea0 con 0x7f4af409ed20 2023-10-03T10:13:49.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.860+0000 7f4ae2ffd700 1 -- 172.21.15.3:0/1245982967 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4ae4003bc0 con 0x7f4af409ed20 2023-10-03T10:13:49.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.861+0000 7f4ae2ffd700 1 -- 172.21.15.3:0/1245982967 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4ae4021070 con 0x7f4af409ed20 2023-10-03T10:13:49.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.865+0000 7f4ae2ffd700 1 -- 172.21.15.3:0/1245982967 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4ae40253e0 con 0x7f4af409ed20 2023-10-03T10:13:49.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.896+0000 7f82b3911700 1 Processor -- start 2023-10-03T10:13:49.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.896+0000 7f82b3911700 1 -- start start 2023-10-03T10:13:49.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.896+0000 7f82b3911700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f82ac09d630 0x7f82ac0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:49.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.896+0000 7f82b3911700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82ac138d80 0x7f82ac13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:49.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.896+0000 7f82b3911700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82ac0a2a70 con 0x7f82ac0a0a30 2023-10-03T10:13:49.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.897+0000 7f82b3911700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82ac0a01d0 con 0x7f82ac09d630 2023-10-03T10:13:49.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.897+0000 7f82b3911700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac09e2b0 con 0x7f82ac138d80 2023-10-03T10:13:49.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.897+0000 7f82b0eac700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f82ac09d630 0x7f82ac0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:49.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.897+0000 7f82b1eae700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82ac138d80 0x7f82ac13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.897+0000 7f82b0eac700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f82ac09d630 0x7f82ac0a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46880/0 (socket says 172.21.15.3:46880) 2023-10-03T10:13:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.897+0000 7f82b1eae700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82ac138d80 0x7f82ac13b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55806/0 (socket says 172.21.15.3:55806) 2023-10-03T10:13:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.897+0000 7f82b0eac700 1 -- 172.21.15.3:0/2717996817 learned_addr learned my addr 172.21.15.3:0/2717996817 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:49.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.897+0000 7f82b1eae700 1 -- 172.21.15.3:0/2717996817 >> v2:172.21.15.3:3300/0 conn(0x7f82ac09d630 msgr2=0x7f82ac0a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:49.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.897+0000 7f82b1eae700 1 --2- 172.21.15.3:0/2717996817 >> v2:172.21.15.3:3300/0 conn(0x7f82ac09d630 0x7f82ac0a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:49.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.897+0000 7f82b1eae700 1 -- 172.21.15.3:0/2717996817 >> v1:172.21.15.3:6789/0 conn(0x7f82ac0a0a30 legacy=0x7f82ac0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:49.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.897+0000 7f82b1eae700 1 -- 172.21.15.3:0/2717996817 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ac09dc50 con 0x7f82ac138d80 2023-10-03T10:13:49.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.897+0000 7f82b0eac700 1 --2- 172.21.15.3:0/2717996817 >> v2:172.21.15.3:3300/0 conn(0x7f82ac09d630 0x7f82ac0a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:49.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.897+0000 7f82b1eae700 1 --2- 172.21.15.3:0/2717996817 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82ac138d80 0x7f82ac13b160 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f82a4009d90 tx=0x7f82a400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40e2915a31e05b2b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:49.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.897+0000 7f82a3fff700 1 -- 172.21.15.3:0/2717996817 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a400e070 con 0x7f82ac138d80 2023-10-03T10:13:49.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.898+0000 7f82a3fff700 1 -- 172.21.15.3:0/2717996817 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a400bb20 con 0x7f82ac138d80 2023-10-03T10:13:49.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.898+0000 7f82a3fff700 1 -- 172.21.15.3:0/2717996817 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a4012430 con 0x7f82ac138d80 2023-10-03T10:13:49.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.898+0000 7f82b3911700 1 -- 172.21.15.3:0/2717996817 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82ac138d80 msgr2=0x7f82ac13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:49.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.898+0000 7f82b3911700 1 --2- 172.21.15.3:0/2717996817 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82ac138d80 0x7f82ac13b160 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f82a4009d90 tx=0x7f82a400b330 comp rx=0 tx=0).stop 2023-10-03T10:13:49.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.898+0000 7f82b3911700 1 -- 172.21.15.3:0/2717996817 shutdown_connections 2023-10-03T10:13:49.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.898+0000 7f82b3911700 1 --2- 172.21.15.3:0/2717996817 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82ac138d80 0x7f82ac13b160 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:49.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.898+0000 7f82b3911700 1 --2- 172.21.15.3:0/2717996817 >> v2:172.21.15.3:3300/0 conn(0x7f82ac09d630 0x7f82ac0a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:49.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.898+0000 7f82b3911700 1 -- 172.21.15.3:0/2717996817 >> 172.21.15.3:0/2717996817 conn(0x7f82ac0a3050 msgr2=0x7f82ac0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:49.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.898+0000 7f82b3911700 1 -- 172.21.15.3:0/2717996817 shutdown_connections 2023-10-03T10:13:49.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.898+0000 7f82b3911700 1 -- 172.21.15.3:0/2717996817 wait complete. 2023-10-03T10:13:49.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.900+0000 7f82b3911700 1 Processor -- start 2023-10-03T10:13:49.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.900+0000 7f82b3911700 1 -- start start 2023-10-03T10:13:49.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.900+0000 7f82b3911700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f82ac09d630 0x7f82ac120670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:49.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.900+0000 7f82b3911700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82ac0a0a30 0x7f82ac120bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:49.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.901+0000 7f82b3911700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82ac138d80 0x7f82ac1210f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:49.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.901+0000 7f82b3911700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac124190 con 0x7f82ac138d80 2023-10-03T10:13:49.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.901+0000 7f82b3911700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac121ad0 con 0x7f82ac09d630 2023-10-03T10:13:49.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.901+0000 7f82b3911700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac123120 con 0x7f82ac0a0a30 2023-10-03T10:13:49.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.901+0000 7f82b0eac700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82ac0a0a30 0x7f82ac120bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:49.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.901+0000 7f82b0eac700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82ac0a0a30 0x7f82ac120bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55816/0 (socket says 172.21.15.3:55816) 2023-10-03T10:13:49.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.901+0000 7f82b0eac700 1 -- 172.21.15.3:0/907579822 learned_addr learned my addr 172.21.15.3:0/907579822 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:49.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.901+0000 7f82b1eae700 1 --2- 172.21.15.3:0/907579822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82ac138d80 0x7f82ac1210f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:49.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.901+0000 7f82b0eac700 1 -- 172.21.15.3:0/907579822 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f82ac09d630 msgr2=0x7f82ac120670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:49.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.901+0000 7f82b0eac700 1 --2- 172.21.15.3:0/907579822 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f82ac09d630 0x7f82ac120670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:49.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.901+0000 7f82b0eac700 1 -- 172.21.15.3:0/907579822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82ac138d80 msgr2=0x7f82ac1210f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:49.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.901+0000 7f82b0eac700 1 --2- 172.21.15.3:0/907579822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82ac138d80 0x7f82ac1210f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:49.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.901+0000 7f82b0eac700 1 -- 172.21.15.3:0/907579822 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f829c009550 con 0x7f82ac0a0a30 2023-10-03T10:13:49.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.901+0000 7f82b0eac700 1 --2- 172.21.15.3:0/907579822 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82ac0a0a30 0x7f82ac120bb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f829c00ebe0 tx=0x7f829c00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:49.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.901+0000 7f82a1ffb700 1 -- 172.21.15.3:0/907579822 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c00c850 con 0x7f82ac0a0a30 2023-10-03T10:13:49.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.901+0000 7f82a1ffb700 1 -- 172.21.15.3:0/907579822 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829c00c9b0 con 0x7f82ac0a0a30 2023-10-03T10:13:49.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.901+0000 7f82b3911700 1 -- 172.21.15.3:0/907579822 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a4009a70 con 0x7f82ac0a0a30 2023-10-03T10:13:49.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.901+0000 7f82a1ffb700 1 -- 172.21.15.3:0/907579822 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c00cd20 con 0x7f82ac0a0a30 2023-10-03T10:13:49.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.902+0000 7f82b3911700 1 -- 172.21.15.3:0/907579822 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82ac13afb0 con 0x7f82ac0a0a30 2023-10-03T10:13:49.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.903+0000 7f82b3911700 1 -- 172.21.15.3:0/907579822 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82ac0747f0 con 0x7f82ac0a0a30 2023-10-03T10:13:49.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.904+0000 7f82a1ffb700 1 -- 172.21.15.3:0/907579822 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f829c004020 con 0x7f82ac0a0a30 2023-10-03T10:13:49.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.904+0000 7f82a1ffb700 1 -- 172.21.15.3:0/907579822 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f829c040050 con 0x7f82ac0a0a30 2023-10-03T10:13:49.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:49.908+0000 7f82a1ffb700 1 -- 172.21.15.3:0/907579822 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f829c024350 con 0x7f82ac0a0a30 2023-10-03T10:13:50.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.017+0000 7ff72a1f9700 1 -- 172.21.15.3:0/2337376192 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff724074ea0 con 0x7ff7241305f0 2023-10-03T10:13:50.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.017+0000 7ff70bfff700 1 -- 172.21.15.3:0/2337376192 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff71c012340 con 0x7ff7241305f0 2023-10-03T10:13:50.018 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:50.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.019+0000 7ff72a1f9700 1 -- 172.21.15.3:0/2337376192 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7241305f0 msgr2=0x7ff72413a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:50.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.019+0000 7ff72a1f9700 1 --2- 172.21.15.3:0/2337376192 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7241305f0 0x7ff72413a510 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff71c00efc0 tx=0x7ff71c00c3a0 comp rx=0 tx=0).stop 2023-10-03T10:13:50.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.019+0000 7ff72a1f9700 1 -- 172.21.15.3:0/2337376192 shutdown_connections 2023-10-03T10:13:50.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.019+0000 7ff72a1f9700 1 --2- 172.21.15.3:0/2337376192 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7241305f0 0x7ff72413a510 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:50.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.019+0000 7ff72a1f9700 1 --2- 172.21.15.3:0/2337376192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff72406f660 0x7ff724139a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:50.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.019+0000 7ff72a1f9700 1 --2- 172.21.15.3:0/2337376192 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff724070160 0x7ff724139fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:50.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.019+0000 7ff72a1f9700 1 -- 172.21.15.3:0/2337376192 >> 172.21.15.3:0/2337376192 conn(0x7ff7240a2aa0 msgr2=0x7ff7240a1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:50.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.020+0000 7ff72a1f9700 1 -- 172.21.15.3:0/2337376192 shutdown_connections 2023-10-03T10:13:50.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.020+0000 7ff72a1f9700 1 -- 172.21.15.3:0/2337376192 wait complete. 2023-10-03T10:13:50.034 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:50.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.073+0000 7f4af8e7d700 1 -- 172.21.15.3:0/1245982967 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4af409a730 con 0x7f4af409ed20 2023-10-03T10:13:50.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.074+0000 7f4ae2ffd700 1 -- 172.21.15.3:0/1245982967 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4ae4034090 con 0x7f4af409ed20 2023-10-03T10:13:50.074 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:50.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.076+0000 7f4af8e7d700 1 -- 172.21.15.3:0/1245982967 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4af409ed20 msgr2=0x7f4af406f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:50.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.076+0000 7f4af8e7d700 1 --2- 172.21.15.3:0/1245982967 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4af409ed20 0x7f4af406f650 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4009fd0 tx=0x7f4ae4004750 comp rx=0 tx=0).stop 2023-10-03T10:13:50.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.076+0000 7f4af8e7d700 1 -- 172.21.15.3:0/1245982967 shutdown_connections 2023-10-03T10:13:50.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.076+0000 7f4af8e7d700 1 --2- 172.21.15.3:0/1245982967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af4131fd0 0x7f4af40700d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:50.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.076+0000 7f4af8e7d700 1 --2- 172.21.15.3:0/1245982967 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4af409ed20 0x7f4af406f650 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:50.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.076+0000 7f4af8e7d700 1 --2- 172.21.15.3:0/1245982967 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4af41316c0 0x7f4af406fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:50.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.076+0000 7f4af8e7d700 1 -- 172.21.15.3:0/1245982967 >> 172.21.15.3:0/1245982967 conn(0x7f4af40a2aa0 msgr2=0x7f4af412e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:50.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.076+0000 7f4af8e7d700 1 -- 172.21.15.3:0/1245982967 shutdown_connections 2023-10-03T10:13:50.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.076+0000 7f4af8e7d700 1 -- 172.21.15.3:0/1245982967 wait complete. 2023-10-03T10:13:50.090 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:50.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.115+0000 7f82b3911700 1 -- 172.21.15.3:0/907579822 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f82ac072980 con 0x7f82ac0a0a30 2023-10-03T10:13:50.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.116+0000 7f82a1ffb700 1 -- 172.21.15.3:0/907579822 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f829c01ab50 con 0x7f82ac0a0a30 2023-10-03T10:13:50.116 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.118+0000 7f82b3911700 1 -- 172.21.15.3:0/907579822 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82ac0a0a30 msgr2=0x7f82ac120bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.118+0000 7f82b3911700 1 --2- 172.21.15.3:0/907579822 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82ac0a0a30 0x7f82ac120bb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f829c00ebe0 tx=0x7f829c00eef0 comp rx=0 tx=0).stop 2023-10-03T10:13:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.118+0000 7f82b3911700 1 -- 172.21.15.3:0/907579822 shutdown_connections 2023-10-03T10:13:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.118+0000 7f82b3911700 1 --2- 172.21.15.3:0/907579822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82ac138d80 0x7f82ac1210f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.118+0000 7f82b3911700 1 --2- 172.21.15.3:0/907579822 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f82ac09d630 0x7f82ac120670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:50.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.118+0000 7f82b3911700 1 --2- 172.21.15.3:0/907579822 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82ac0a0a30 0x7f82ac120bb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:50.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.118+0000 7f82b3911700 1 -- 172.21.15.3:0/907579822 >> 172.21.15.3:0/907579822 conn(0x7f82ac0a3050 msgr2=0x7f82ac09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:50.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.118+0000 7f82b3911700 1 -- 172.21.15.3:0/907579822 shutdown_connections 2023-10-03T10:13:50.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.118+0000 7f82b3911700 1 -- 172.21.15.3:0/907579822 wait complete. 2023-10-03T10:13:50.132 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:50.772 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:50.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.918+0000 7f86199b1700 1 Processor -- start 2023-10-03T10:13:50.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.918+0000 7f86199b1700 1 -- start start 2023-10-03T10:13:50.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.918+0000 7f86199b1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f861409e850 0x7f861409ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:50.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.918+0000 7f86199b1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f861409f160 0x7f8614133b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:50.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.918+0000 7f86199b1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f861406bec0 con 0x7f861409f550 2023-10-03T10:13:50.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.919+0000 7f86199b1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f861406bce0 con 0x7f861409e850 2023-10-03T10:13:50.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.919+0000 7f86199b1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f861406caa0 con 0x7f861409f160 2023-10-03T10:13:50.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.919+0000 7f86127fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f861409f160 0x7f8614133b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:50.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.919+0000 7f8612ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f861409e850 0x7f861409ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:50.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.919+0000 7f86127fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f861409f160 0x7f8614133b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55824/0 (socket says 172.21.15.3:55824) 2023-10-03T10:13:50.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.919+0000 7f86137fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f861409f550 0x7f86141340c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45740/0 (socket says 172.21.15.3:45740) 2023-10-03T10:13:50.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.919+0000 7f8612ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f861409e850 0x7f861409ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46394/0 (socket says 172.21.15.3:46394) 2023-10-03T10:13:50.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.919+0000 7f86127fc700 1 -- 172.21.15.3:0/1506535452 learned_addr learned my addr 172.21.15.3:0/1506535452 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:50.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.919+0000 7f8612ffd700 1 -- 172.21.15.3:0/1506535452 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f861409f160 msgr2=0x7f8614133b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:50.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.919+0000 7f8612ffd700 1 --2- 172.21.15.3:0/1506535452 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f861409f160 0x7f8614133b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:50.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.919+0000 7f8611ffb700 1 -- 172.21.15.3:0/1506535452 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3539242376 0 0) 0x7f861406bec0 con 0x7f861409f550 2023-10-03T10:13:50.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.919+0000 7f8612ffd700 1 -- 172.21.15.3:0/1506535452 >> v1:172.21.15.3:6789/0 conn(0x7f861409f550 legacy=0x7f86141340c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:50.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.919+0000 7f8612ffd700 1 -- 172.21.15.3:0/1506535452 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86140a00a0 con 0x7f861409e850 2023-10-03T10:13:50.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.919+0000 7f86127fc700 1 --2- 172.21.15.3:0/1506535452 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f861409f160 0x7f8614133b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:50.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.919+0000 7f8612ffd700 1 --2- 172.21.15.3:0/1506535452 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f861409e850 0x7f861409ec20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f86040098a0 tx=0x7f8604009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdfb9852a9af510f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:50.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.919+0000 7f8611ffb700 1 -- 172.21.15.3:0/1506535452 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8604011830 con 0x7f861409e850 2023-10-03T10:13:50.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.919+0000 7f8611ffb700 1 -- 172.21.15.3:0/1506535452 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8604011990 con 0x7f861409e850 2023-10-03T10:13:50.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.919+0000 7f8611ffb700 1 -- 172.21.15.3:0/1506535452 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8604011d20 con 0x7f861409e850 2023-10-03T10:13:50.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.920+0000 7f86199b1700 1 -- 172.21.15.3:0/1506535452 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f861409e850 msgr2=0x7f861409ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:50.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.920+0000 7f86199b1700 1 --2- 172.21.15.3:0/1506535452 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f861409e850 0x7f861409ec20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f86040098a0 tx=0x7f8604009bb0 comp rx=0 tx=0).stop 2023-10-03T10:13:50.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.920+0000 7f86199b1700 1 -- 172.21.15.3:0/1506535452 shutdown_connections 2023-10-03T10:13:50.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.920+0000 7f86199b1700 1 --2- 172.21.15.3:0/1506535452 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f861409e850 0x7f861409ec20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:50.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.920+0000 7f86199b1700 1 --2- 172.21.15.3:0/1506535452 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f861409f160 0x7f8614133b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:50.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.920+0000 7f86199b1700 1 -- 172.21.15.3:0/1506535452 >> 172.21.15.3:0/1506535452 conn(0x7f86140a3f70 msgr2=0x7f86140a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:50.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.920+0000 7f86199b1700 1 -- 172.21.15.3:0/1506535452 shutdown_connections 2023-10-03T10:13:50.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.920+0000 7f86199b1700 1 -- 172.21.15.3:0/1506535452 wait complete. 2023-10-03T10:13:50.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.921+0000 7f86199b1700 1 Processor -- start 2023-10-03T10:13:50.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.921+0000 7f86199b1700 1 -- start start 2023-10-03T10:13:50.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.922+0000 7f86199b1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f861409e850 0x7f8614122600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:50.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.922+0000 7f86199b1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f861409f160 0x7f8614122b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:50.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.922+0000 7f86199b1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f861409f550 0x7f8614123080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:50.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.922+0000 7f86199b1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8614125ef0 con 0x7f861409e850 2023-10-03T10:13:50.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.922+0000 7f86199b1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8614125b30 con 0x7f861409f550 2023-10-03T10:13:50.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.922+0000 7f86199b1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f86141260d0 con 0x7f861409f160 2023-10-03T10:13:50.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.922+0000 7f8612ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f861409e850 0x7f8614122600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:50.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.922+0000 7f86127fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f861409f160 0x7f8614122b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:50.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.922+0000 7f8612ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f861409e850 0x7f8614122600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46908/0 (socket says 172.21.15.3:46908) 2023-10-03T10:13:50.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.922+0000 7f8612ffd700 1 -- 172.21.15.3:0/2352545523 learned_addr learned my addr 172.21.15.3:0/2352545523 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:50.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.922+0000 7f86127fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f861409f160 0x7f8614122b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55838/0 (socket says 172.21.15.3:55838) 2023-10-03T10:13:50.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.922+0000 7f86137fe700 1 --2- 172.21.15.3:0/2352545523 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f861409f550 0x7f8614123080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:50.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.922+0000 7f86127fc700 1 -- 172.21.15.3:0/2352545523 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f861409f550 msgr2=0x7f8614123080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:50.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.922+0000 7f86127fc700 1 --2- 172.21.15.3:0/2352545523 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f861409f550 0x7f8614123080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:50.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.922+0000 7f86127fc700 1 -- 172.21.15.3:0/2352545523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f861409e850 msgr2=0x7f8614122600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:50.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.922+0000 7f86127fc700 1 --2- 172.21.15.3:0/2352545523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f861409e850 0x7f8614122600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:50.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.922+0000 7f86127fc700 1 -- 172.21.15.3:0/2352545523 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85fc009550 con 0x7f861409f160 2023-10-03T10:13:50.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.922+0000 7f8612ffd700 1 --2- 172.21.15.3:0/2352545523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f861409e850 0x7f8614122600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:50.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.922+0000 7f86127fc700 1 --2- 172.21.15.3:0/2352545523 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f861409f160 0x7f8614122b40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f86140a4fa0 tx=0x7f85fc00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:50.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.923+0000 7f860bfff700 1 -- 172.21.15.3:0/2352545523 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc004240 con 0x7f861409f160 2023-10-03T10:13:50.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.923+0000 7f860bfff700 1 -- 172.21.15.3:0/2352545523 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85fc0043a0 con 0x7f861409f160 2023-10-03T10:13:50.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.923+0000 7f86199b1700 1 -- 172.21.15.3:0/2352545523 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86040094a0 con 0x7f861409f160 2023-10-03T10:13:50.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.923+0000 7f860bfff700 1 -- 172.21.15.3:0/2352545523 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc0036a0 con 0x7f861409f160 2023-10-03T10:13:50.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.923+0000 7f86199b1700 1 -- 172.21.15.3:0/2352545523 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f861406ccc0 con 0x7f861409f160 2023-10-03T10:13:50.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.923+0000 7f860bfff700 1 -- 172.21.15.3:0/2352545523 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f85fc003800 con 0x7f861409f160 2023-10-03T10:13:50.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.924+0000 7f860bfff700 1 -- 172.21.15.3:0/2352545523 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f85fc03f7e0 con 0x7f861409f160 2023-10-03T10:13:50.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.924+0000 7f86199b1700 1 -- 172.21.15.3:0/2352545523 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86140747f0 con 0x7f861409f160 2023-10-03T10:13:50.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:50.929+0000 7f860bfff700 1 -- 172.21.15.3:0/2352545523 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f85fc013470 con 0x7f861409f160 2023-10-03T10:13:51.034 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:51.092 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:51.133 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:51.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.141+0000 7f86199b1700 1 -- 172.21.15.3:0/2352545523 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f86140727d0 con 0x7f861409f160 2023-10-03T10:13:51.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.141+0000 7f860bfff700 1 -- 172.21.15.3:0/2352545523 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f85fc02a090 con 0x7f861409f160 2023-10-03T10:13:51.142 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:51.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.145+0000 7f86199b1700 1 -- 172.21.15.3:0/2352545523 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f861409f160 msgr2=0x7f8614122b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:51.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.145+0000 7f86199b1700 1 --2- 172.21.15.3:0/2352545523 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f861409f160 0x7f8614122b40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f86140a4fa0 tx=0x7f85fc00ec80 comp rx=0 tx=0).stop 2023-10-03T10:13:51.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.146+0000 7f86199b1700 1 -- 172.21.15.3:0/2352545523 shutdown_connections 2023-10-03T10:13:51.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.146+0000 7f86199b1700 1 --2- 172.21.15.3:0/2352545523 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f861409f550 0x7f8614123080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:51.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.146+0000 7f86199b1700 1 --2- 172.21.15.3:0/2352545523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f861409e850 0x7f8614122600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:51.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.146+0000 7f86199b1700 1 --2- 172.21.15.3:0/2352545523 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f861409f160 0x7f8614122b40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:51.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.146+0000 7f86199b1700 1 -- 172.21.15.3:0/2352545523 >> 172.21.15.3:0/2352545523 conn(0x7f86140a3f70 msgr2=0x7f8614133f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:51.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.146+0000 7f86199b1700 1 -- 172.21.15.3:0/2352545523 shutdown_connections 2023-10-03T10:13:51.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.146+0000 7f86199b1700 1 -- 172.21.15.3:0/2352545523 wait complete. 2023-10-03T10:13:51.163 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:51.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.188+0000 7f6802829700 1 Processor -- start 2023-10-03T10:13:51.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.188+0000 7f6802829700 1 -- start start 2023-10-03T10:13:51.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.188+0000 7f6802829700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67fc06b250 0x7f67fc136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:51.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.188+0000 7f6802829700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f67fc136d80 0x7f67fc139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:51.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.188+0000 7f6802829700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67fc09ece0 con 0x7f67fc06a770 2023-10-03T10:13:51.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.188+0000 7f6802829700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67fc06b640 con 0x7f67fc136d80 2023-10-03T10:13:51.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.188+0000 7f6802829700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc09e760 con 0x7f67fc06b250 2023-10-03T10:13:51.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.188+0000 7f6800dc6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f67fc136d80 0x7f67fc139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:51.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.188+0000 7f67fb7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67fc06b250 0x7f67fc136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:51.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.188+0000 7f6800dc6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f67fc136d80 0x7f67fc139160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46920/0 (socket says 172.21.15.3:46920) 2023-10-03T10:13:51.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.188+0000 7f67fb7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67fc06b250 0x7f67fc136840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55854/0 (socket says 172.21.15.3:55854) 2023-10-03T10:13:51.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.188+0000 7f6800dc6700 1 -- 172.21.15.3:0/2691135583 learned_addr learned my addr 172.21.15.3:0/2691135583 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:51.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.189+0000 7f67fb7fe700 1 -- 172.21.15.3:0/2691135583 >> v2:172.21.15.3:3300/0 conn(0x7f67fc136d80 msgr2=0x7f67fc139160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:51.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.189+0000 7f67fb7fe700 1 --2- 172.21.15.3:0/2691135583 >> v2:172.21.15.3:3300/0 conn(0x7f67fc136d80 0x7f67fc139160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:51.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.189+0000 7f67fb7fe700 1 -- 172.21.15.3:0/2691135583 >> v1:172.21.15.3:6789/0 conn(0x7f67fc06a770 legacy=0x7f67fc06ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:51.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.189+0000 7f67fb7fe700 1 -- 172.21.15.3:0/2691135583 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67fc09e040 con 0x7f67fc06b250 2023-10-03T10:13:51.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.189+0000 7f6800dc6700 1 --2- 172.21.15.3:0/2691135583 >> v2:172.21.15.3:3300/0 conn(0x7f67fc136d80 0x7f67fc139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:51.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.189+0000 7f67fb7fe700 1 --2- 172.21.15.3:0/2691135583 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67fc06b250 0x7f67fc136840 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f67f0009840 tx=0x7f67f0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fc634401c087e64 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:51.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.189+0000 7f67faffd700 1 -- 172.21.15.3:0/2691135583 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f0011830 con 0x7f67fc06b250 2023-10-03T10:13:51.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.189+0000 7f67faffd700 1 -- 172.21.15.3:0/2691135583 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f0011990 con 0x7f67fc06b250 2023-10-03T10:13:51.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.189+0000 7f67faffd700 1 -- 172.21.15.3:0/2691135583 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f0011d20 con 0x7f67fc06b250 2023-10-03T10:13:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.190+0000 7f6802829700 1 -- 172.21.15.3:0/2691135583 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67fc06b250 msgr2=0x7f67fc136840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.190+0000 7f6802829700 1 --2- 172.21.15.3:0/2691135583 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67fc06b250 0x7f67fc136840 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f67f0009840 tx=0x7f67f0009b50 comp rx=0 tx=0).stop 2023-10-03T10:13:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.190+0000 7f6802829700 1 -- 172.21.15.3:0/2691135583 shutdown_connections 2023-10-03T10:13:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.190+0000 7f6802829700 1 --2- 172.21.15.3:0/2691135583 >> v2:172.21.15.3:3300/0 conn(0x7f67fc136d80 0x7f67fc139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.190+0000 7f6802829700 1 --2- 172.21.15.3:0/2691135583 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67fc06b250 0x7f67fc136840 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.190+0000 7f6802829700 1 -- 172.21.15.3:0/2691135583 >> 172.21.15.3:0/2691135583 conn(0x7f67fc066f50 msgr2=0x7f67fc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.190+0000 7f6802829700 1 -- 172.21.15.3:0/2691135583 shutdown_connections 2023-10-03T10:13:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.190+0000 7f6802829700 1 -- 172.21.15.3:0/2691135583 wait complete. 2023-10-03T10:13:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.191+0000 7f6802829700 1 Processor -- start 2023-10-03T10:13:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.192+0000 7f6802829700 1 -- start start 2023-10-03T10:13:51.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.193+0000 7f6802829700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67fc06a770 0x7f67fc09ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:51.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.193+0000 7f6802829700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67fc06b250 0x7f67fc09b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:51.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.193+0000 7f6802829700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67fc136d80 0x7f67fc1c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:51.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.193+0000 7f6802829700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc09c1a0 con 0x7f67fc06b250 2023-10-03T10:13:51.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.193+0000 7f6802829700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc09bf70 con 0x7f67fc06a770 2023-10-03T10:13:51.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.193+0000 7f6802829700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc122090 con 0x7f67fc136d80 2023-10-03T10:13:51.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.193+0000 7f67fb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67fc06b250 0x7f67fc09b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:51.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.193+0000 7f67fb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67fc06b250 0x7f67fc09b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46922/0 (socket says 172.21.15.3:46922) 2023-10-03T10:13:51.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.193+0000 7f67fb7fe700 1 -- 172.21.15.3:0/3712623545 learned_addr learned my addr 172.21.15.3:0/3712623545 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:51.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.193+0000 7f6800dc6700 1 --2- 172.21.15.3:0/3712623545 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67fc136d80 0x7f67fc1c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:51.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.193+0000 7f67fb7fe700 1 -- 172.21.15.3:0/3712623545 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67fc136d80 msgr2=0x7f67fc1c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:51.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.193+0000 7f67fb7fe700 1 --2- 172.21.15.3:0/3712623545 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67fc136d80 0x7f67fc1c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:51.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.193+0000 7f67fb7fe700 1 -- 172.21.15.3:0/3712623545 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67fc06a770 msgr2=0x7f67fc09ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:51.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.193+0000 7f67fb7fe700 1 --2- 172.21.15.3:0/3712623545 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67fc06a770 0x7f67fc09ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:51.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.193+0000 7f67fb7fe700 1 -- 172.21.15.3:0/3712623545 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f4009b30 con 0x7f67fc06b250 2023-10-03T10:13:51.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.193+0000 7f67fb7fe700 1 --2- 172.21.15.3:0/3712623545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67fc06b250 0x7f67fc09b350 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f67f0009fd0 tx=0x7f67f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:51.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.193+0000 7f67f8ff9700 1 -- 172.21.15.3:0/3712623545 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f00036a0 con 0x7f67fc06b250 2023-10-03T10:13:51.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.194+0000 7f67f8ff9700 1 -- 172.21.15.3:0/3712623545 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f001f070 con 0x7f67fc06b250 2023-10-03T10:13:51.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.194+0000 7f6802829700 1 -- 172.21.15.3:0/3712623545 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f00094a0 con 0x7f67fc06b250 2023-10-03T10:13:51.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.194+0000 7f6802829700 1 -- 172.21.15.3:0/3712623545 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67fc06e950 con 0x7f67fc06b250 2023-10-03T10:13:51.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.194+0000 7f67f8ff9700 1 -- 172.21.15.3:0/3712623545 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f0003a60 con 0x7f67fc06b250 2023-10-03T10:13:51.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.195+0000 7f67f8ff9700 1 -- 172.21.15.3:0/3712623545 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f67f0003bc0 con 0x7f67fc06b250 2023-10-03T10:13:51.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.195+0000 7f67f8ff9700 1 -- 172.21.15.3:0/3712623545 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f67f0021070 con 0x7f67fc06b250 2023-10-03T10:13:51.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.195+0000 7f6802829700 1 -- 172.21.15.3:0/3712623545 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67dc0052e0 con 0x7f67fc06b250 2023-10-03T10:13:51.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.200+0000 7f67f8ff9700 1 -- 172.21.15.3:0/3712623545 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f67f001a950 con 0x7f67fc06b250 2023-10-03T10:13:51.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.262+0000 7f1ab4566700 1 Processor -- start 2023-10-03T10:13:51.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.262+0000 7f1ab4566700 1 -- start start 2023-10-03T10:13:51.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.263+0000 7f1ab4566700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1aac134630 0x7f1aac136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:51.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.263+0000 7f1ab4566700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1aac136f50 0x7f1aac139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:51.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.263+0000 7f1ab4566700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aac09b4f0 con 0x7f1aac131810 2023-10-03T10:13:51.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.263+0000 7f1ab4566700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac1313c0 con 0x7f1aac136f50 2023-10-03T10:13:51.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.263+0000 7f1ab4566700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac06e8c0 con 0x7f1aac134630 2023-10-03T10:13:51.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.263+0000 7f1ab1b01700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1aac134630 0x7f1aac136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:51.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.263+0000 7f1ab2302700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1aac131810 0x7f1aac131be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45758/0 (socket says 172.21.15.3:45758) 2023-10-03T10:13:51.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.263+0000 7f1ab2302700 1 -- 172.21.15.3:0/4265114186 learned_addr learned my addr 172.21.15.3:0/4265114186 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:51.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.263+0000 7f1ab2b03700 1 --2- 172.21.15.3:0/4265114186 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1aac136f50 0x7f1aac139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:51.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.263+0000 7f1ab1300700 1 -- 172.21.15.3:0/4265114186 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3141595418 0 0) 0x7f1aac09b4f0 con 0x7f1aac131810 2023-10-03T10:13:51.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.263+0000 7f1ab1300700 1 -- 172.21.15.3:0/4265114186 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a980035b0 con 0x7f1aac131810 2023-10-03T10:13:51.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.263+0000 7f1ab2b03700 1 -- 172.21.15.3:0/4265114186 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1aac134630 msgr2=0x7f1aac136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:51.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.263+0000 7f1ab2b03700 1 --2- 172.21.15.3:0/4265114186 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1aac134630 0x7f1aac136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:51.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.263+0000 7f1ab2b03700 1 -- 172.21.15.3:0/4265114186 >> v1:172.21.15.3:6789/0 conn(0x7f1aac131810 legacy=0x7f1aac131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:51.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.263+0000 7f1ab2b03700 1 -- 172.21.15.3:0/4265114186 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aac071d20 con 0x7f1aac136f50 2023-10-03T10:13:51.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.264+0000 7f1ab2b03700 1 --2- 172.21.15.3:0/4265114186 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1aac136f50 0x7f1aac139330 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009dc0 tx=0x7f1aa400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c59c3f64139dda18 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:51.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.264+0000 7f1ab1300700 1 -- 172.21.15.3:0/4265114186 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa400e070 con 0x7f1aac136f50 2023-10-03T10:13:51.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.264+0000 7f1ab1300700 1 -- 172.21.15.3:0/4265114186 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa400bb20 con 0x7f1aac136f50 2023-10-03T10:13:51.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.264+0000 7f1ab1300700 1 -- 172.21.15.3:0/4265114186 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4012430 con 0x7f1aac136f50 2023-10-03T10:13:51.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.265+0000 7f1ab4566700 1 -- 172.21.15.3:0/4265114186 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1aac136f50 msgr2=0x7f1aac139330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:51.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.265+0000 7f1ab4566700 1 --2- 172.21.15.3:0/4265114186 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1aac136f50 0x7f1aac139330 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009dc0 tx=0x7f1aa400b330 comp rx=0 tx=0).stop 2023-10-03T10:13:51.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.265+0000 7f1ab4566700 1 -- 172.21.15.3:0/4265114186 shutdown_connections 2023-10-03T10:13:51.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.265+0000 7f1ab4566700 1 --2- 172.21.15.3:0/4265114186 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1aac136f50 0x7f1aac139330 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:51.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.265+0000 7f1ab4566700 1 --2- 172.21.15.3:0/4265114186 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1aac134630 0x7f1aac136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:51.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.265+0000 7f1ab4566700 1 -- 172.21.15.3:0/4265114186 >> 172.21.15.3:0/4265114186 conn(0x7f1aac0a2aa0 msgr2=0x7f1aac0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:51.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.265+0000 7f1ab4566700 1 -- 172.21.15.3:0/4265114186 shutdown_connections 2023-10-03T10:13:51.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.265+0000 7f1ab4566700 1 -- 172.21.15.3:0/4265114186 wait complete. 2023-10-03T10:13:51.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.266+0000 7f1ab4566700 1 Processor -- start 2023-10-03T10:13:51.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.266+0000 7f1ab4566700 1 -- start start 2023-10-03T10:13:51.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.267+0000 7f1ab4566700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1aac131810 0x7f1aac1c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:51.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.267+0000 7f1ab4566700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1aac134630 0x7f1aac1c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:51.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.267+0000 7f1ab4566700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1aac136f50 0x7f1aac06f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:51.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.267+0000 7f1ab4566700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac1bc0d0 con 0x7f1aac134630 2023-10-03T10:13:51.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.267+0000 7f1ab4566700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac130b00 con 0x7f1aac131810 2023-10-03T10:13:51.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.267+0000 7f1ab4566700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac06eb00 con 0x7f1aac136f50 2023-10-03T10:13:51.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.267+0000 7f1ab2302700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1aac131810 0x7f1aac1c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:51.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.267+0000 7f1ab2302700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1aac131810 0x7f1aac1c8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46434/0 (socket says 172.21.15.3:46434) 2023-10-03T10:13:51.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.267+0000 7f1ab2302700 1 -- 172.21.15.3:0/2129453209 learned_addr learned my addr 172.21.15.3:0/2129453209 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:51.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.267+0000 7f1ab1b01700 1 --2- 172.21.15.3:0/2129453209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1aac134630 0x7f1aac1c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:51.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.267+0000 7f1ab2b03700 1 --2- 172.21.15.3:0/2129453209 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1aac136f50 0x7f1aac06f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:51.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.267+0000 7f1ab2302700 1 -- 172.21.15.3:0/2129453209 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1aac136f50 msgr2=0x7f1aac06f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:51.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.267+0000 7f1ab2302700 1 --2- 172.21.15.3:0/2129453209 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1aac136f50 0x7f1aac06f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:51.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.267+0000 7f1ab2302700 1 -- 172.21.15.3:0/2129453209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1aac134630 msgr2=0x7f1aac1c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:51.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.267+0000 7f1ab2302700 1 --2- 172.21.15.3:0/2129453209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1aac134630 0x7f1aac1c89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:51.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.267+0000 7f1ab2302700 1 -- 172.21.15.3:0/2129453209 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa4009a70 con 0x7f1aac131810 2023-10-03T10:13:51.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.267+0000 7f1ab1b01700 1 --2- 172.21.15.3:0/2129453209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1aac134630 0x7f1aac1c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:51.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.268+0000 7f1ab2302700 1 --2- 172.21.15.3:0/2129453209 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1aac131810 0x7f1aac1c8470 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800c700 tx=0x7f1aa800ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:51.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.268+0000 7f1aa2ffd700 1 -- 172.21.15.3:0/2129453209 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8005d70 con 0x7f1aac131810 2023-10-03T10:13:51.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.268+0000 7f1aa2ffd700 1 -- 172.21.15.3:0/2129453209 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa80070d0 con 0x7f1aac131810 2023-10-03T10:13:51.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.268+0000 7f1ab4566700 1 -- 172.21.15.3:0/2129453209 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aac11e520 con 0x7f1aac131810 2023-10-03T10:13:51.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.268+0000 7f1aa2ffd700 1 -- 172.21.15.3:0/2129453209 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8007460 con 0x7f1aac131810 2023-10-03T10:13:51.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.268+0000 7f1ab4566700 1 -- 172.21.15.3:0/2129453209 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aac071100 con 0x7f1aac131810 2023-10-03T10:13:51.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.269+0000 7f1aa2ffd700 1 -- 172.21.15.3:0/2129453209 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1aa80075c0 con 0x7f1aac131810 2023-10-03T10:13:51.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.269+0000 7f1aa2ffd700 1 -- 172.21.15.3:0/2129453209 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1aa803eb30 con 0x7f1aac131810 2023-10-03T10:13:51.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.269+0000 7f1ab4566700 1 -- 172.21.15.3:0/2129453209 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aac074ea0 con 0x7f1aac131810 2023-10-03T10:13:51.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.274+0000 7f1aa2ffd700 1 -- 172.21.15.3:0/2129453209 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1aa8018260 con 0x7f1aac131810 2023-10-03T10:13:51.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.283+0000 7f941fd28700 1 Processor -- start 2023-10-03T10:13:51.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.284+0000 7f941fd28700 1 -- start start 2023-10-03T10:13:51.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.284+0000 7f941fd28700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f941812cf60 0x7f941806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:51.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.284+0000 7f941fd28700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9418130430 0x7f9418137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:51.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.284+0000 7f941fd28700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f941812cd90 con 0x7f941812f8b0 2023-10-03T10:13:51.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.284+0000 7f941fd28700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f941812d330 con 0x7f941812cf60 2023-10-03T10:13:51.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.284+0000 7f941fd28700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9418071780 con 0x7f9418130430 2023-10-03T10:13:51.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.284+0000 7f941dac4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f941812cf60 0x7f941806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:51.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.284+0000 7f941dac4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f941812cf60 0x7f941806f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46948/0 (socket says 172.21.15.3:46948) 2023-10-03T10:13:51.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.284+0000 7f941e2c5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9418130430 0x7f9418137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:51.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.284+0000 7f941dac4700 1 -- 172.21.15.3:0/3925753357 learned_addr learned my addr 172.21.15.3:0/3925753357 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:51.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.284+0000 7f941d2c3700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f941812f8b0 0x7f941812fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45772/0 (socket says 172.21.15.3:45772) 2023-10-03T10:13:51.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.285+0000 7f941dac4700 1 -- 172.21.15.3:0/3925753357 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9418130430 msgr2=0x7f9418137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:51.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.285+0000 7f941dac4700 1 --2- 172.21.15.3:0/3925753357 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9418130430 0x7f9418137c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:51.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.285+0000 7f941dac4700 1 -- 172.21.15.3:0/3925753357 >> v1:172.21.15.3:6789/0 conn(0x7f941812f8b0 legacy=0x7f941812fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:51.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.285+0000 7f941dac4700 1 -- 172.21.15.3:0/3925753357 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9418071a40 con 0x7f941812cf60 2023-10-03T10:13:51.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.285+0000 7f941cac2700 1 -- 172.21.15.3:0/3925753357 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1749282354 0 0) 0x7f941812cd90 con 0x7f941812f8b0 2023-10-03T10:13:51.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.285+0000 7f941e2c5700 1 --2- 172.21.15.3:0/3925753357 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9418130430 0x7f9418137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:51.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.285+0000 7f941dac4700 1 --2- 172.21.15.3:0/3925753357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f941812cf60 0x7f941806f5d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f941400aa90 tx=0x7f941400ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48df1e513d385b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:51.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.285+0000 7f941cac2700 1 -- 172.21.15.3:0/3925753357 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941400cb70 con 0x7f941812cf60 2023-10-03T10:13:51.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.285+0000 7f941cac2700 1 -- 172.21.15.3:0/3925753357 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941400ccd0 con 0x7f941812cf60 2023-10-03T10:13:51.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.285+0000 7f941cac2700 1 -- 172.21.15.3:0/3925753357 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9414010640 con 0x7f941812cf60 2023-10-03T10:13:51.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.286+0000 7f941fd28700 1 -- 172.21.15.3:0/3925753357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f941812cf60 msgr2=0x7f941806f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:51.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.286+0000 7f941fd28700 1 --2- 172.21.15.3:0/3925753357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f941812cf60 0x7f941806f5d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f941400aa90 tx=0x7f941400ada0 comp rx=0 tx=0).stop 2023-10-03T10:13:51.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.286+0000 7f941fd28700 1 -- 172.21.15.3:0/3925753357 shutdown_connections 2023-10-03T10:13:51.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.286+0000 7f941fd28700 1 --2- 172.21.15.3:0/3925753357 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9418130430 0x7f9418137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:51.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.286+0000 7f941fd28700 1 --2- 172.21.15.3:0/3925753357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f941812cf60 0x7f941806f5d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:51.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.286+0000 7f941fd28700 1 -- 172.21.15.3:0/3925753357 >> 172.21.15.3:0/3925753357 conn(0x7f94180a2aa0 msgr2=0x7f94180a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:51.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.286+0000 7f941fd28700 1 -- 172.21.15.3:0/3925753357 shutdown_connections 2023-10-03T10:13:51.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.286+0000 7f941fd28700 1 -- 172.21.15.3:0/3925753357 wait complete. 2023-10-03T10:13:51.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.287+0000 7f941fd28700 1 Processor -- start 2023-10-03T10:13:51.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.288+0000 7f941fd28700 1 -- start start 2023-10-03T10:13:51.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.288+0000 7f941fd28700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f941812cf60 0x7f94181c4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:51.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.288+0000 7f941fd28700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f941812f8b0 0x7f94181c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:51.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.288+0000 7f941fd28700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9418130430 0x7f94181c0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:51.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.288+0000 7f941fd28700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941809e390 con 0x7f941812f8b0 2023-10-03T10:13:51.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.288+0000 7f941fd28700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9418067cb0 con 0x7f941812cf60 2023-10-03T10:13:51.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.288+0000 7f941fd28700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f941809ea80 con 0x7f9418130430 2023-10-03T10:13:51.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.288+0000 7f941dac4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f941812cf60 0x7f94181c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:51.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.288+0000 7f941d2c3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f941812f8b0 0x7f94181c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:51.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.288+0000 7f941dac4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f941812cf60 0x7f94181c4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46458/0 (socket says 172.21.15.3:46458) 2023-10-03T10:13:51.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.288+0000 7f941d2c3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f941812f8b0 0x7f94181c4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46956/0 (socket says 172.21.15.3:46956) 2023-10-03T10:13:51.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.288+0000 7f941dac4700 1 -- 172.21.15.3:0/1852332977 learned_addr learned my addr 172.21.15.3:0/1852332977 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:51.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.289+0000 7f941e2c5700 1 --2- 172.21.15.3:0/1852332977 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9418130430 0x7f94181c0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:51.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.289+0000 7f941dac4700 1 -- 172.21.15.3:0/1852332977 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9418130430 msgr2=0x7f94181c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:51.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.289+0000 7f941dac4700 1 --2- 172.21.15.3:0/1852332977 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9418130430 0x7f94181c0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:51.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.289+0000 7f941dac4700 1 -- 172.21.15.3:0/1852332977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f941812f8b0 msgr2=0x7f94181c4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:51.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.289+0000 7f941dac4700 1 --2- 172.21.15.3:0/1852332977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f941812f8b0 0x7f94181c4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:51.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.289+0000 7f941dac4700 1 -- 172.21.15.3:0/1852332977 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9410009a70 con 0x7f941812cf60 2023-10-03T10:13:51.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.289+0000 7f941d2c3700 1 --2- 172.21.15.3:0/1852332977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f941812f8b0 0x7f94181c4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:51.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.289+0000 7f941dac4700 1 --2- 172.21.15.3:0/1852332977 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f941812cf60 0x7f94181c4700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f941812d470 tx=0x7f9414004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:51.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.289+0000 7f940e7fc700 1 -- 172.21.15.3:0/1852332977 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9414022070 con 0x7f941812cf60 2023-10-03T10:13:51.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.289+0000 7f940e7fc700 1 -- 172.21.15.3:0/1852332977 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941400ce40 con 0x7f941812cf60 2023-10-03T10:13:51.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.289+0000 7f941fd28700 1 -- 172.21.15.3:0/1852332977 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f941400a6f0 con 0x7f941812cf60 2023-10-03T10:13:51.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.290+0000 7f940e7fc700 1 -- 172.21.15.3:0/1852332977 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9414004260 con 0x7f941812cf60 2023-10-03T10:13:51.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.290+0000 7f941fd28700 1 -- 172.21.15.3:0/1852332977 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f941809d110 con 0x7f941812cf60 2023-10-03T10:13:51.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.291+0000 7f941fd28700 1 -- 172.21.15.3:0/1852332977 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f941809d2a0 con 0x7f941812cf60 2023-10-03T10:13:51.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.291+0000 7f940e7fc700 1 -- 172.21.15.3:0/1852332977 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9414026070 con 0x7f941812cf60 2023-10-03T10:13:51.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.291+0000 7f940e7fc700 1 -- 172.21.15.3:0/1852332977 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9414044b70 con 0x7f941812cf60 2023-10-03T10:13:51.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.296+0000 7f940e7fc700 1 -- 172.21.15.3:0/1852332977 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f94140043c0 con 0x7f941812cf60 2023-10-03T10:13:51.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.445+0000 7f6802829700 1 -- 172.21.15.3:0/3712623545 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f67dc005590 con 0x7f67fc06b250 2023-10-03T10:13:51.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.446+0000 7f67f8ff9700 1 -- 172.21.15.3:0/3712623545 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f67f0010b70 con 0x7f67fc06b250 2023-10-03T10:13:51.446 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:51.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.448+0000 7f6802829700 1 -- 172.21.15.3:0/3712623545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67fc06b250 msgr2=0x7f67fc09b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:51.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.448+0000 7f6802829700 1 --2- 172.21.15.3:0/3712623545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67fc06b250 0x7f67fc09b350 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f67f0009fd0 tx=0x7f67f0004750 comp rx=0 tx=0).stop 2023-10-03T10:13:51.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.448+0000 7f6802829700 1 -- 172.21.15.3:0/3712623545 shutdown_connections 2023-10-03T10:13:51.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.448+0000 7f6802829700 1 --2- 172.21.15.3:0/3712623545 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67fc136d80 0x7f67fc1c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:51.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.448+0000 7f6802829700 1 --2- 172.21.15.3:0/3712623545 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67fc06a770 0x7f67fc09ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:51.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.448+0000 7f6802829700 1 --2- 172.21.15.3:0/3712623545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67fc06b250 0x7f67fc09b350 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:51.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.448+0000 7f6802829700 1 -- 172.21.15.3:0/3712623545 >> 172.21.15.3:0/3712623545 conn(0x7f67fc066f50 msgr2=0x7f67fc135d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:51.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.448+0000 7f6802829700 1 -- 172.21.15.3:0/3712623545 shutdown_connections 2023-10-03T10:13:51.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.448+0000 7f6802829700 1 -- 172.21.15.3:0/3712623545 wait complete. 2023-10-03T10:13:51.463 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:51.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.486+0000 7f1ab4566700 1 -- 172.21.15.3:0/2129453209 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1aac1c8ef0 con 0x7f1aac131810 2023-10-03T10:13:51.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.487+0000 7f1aa2ffd700 1 -- 172.21.15.3:0/2129453209 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1aa8017030 con 0x7f1aac131810 2023-10-03T10:13:51.487 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:51.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.489+0000 7f1ab4566700 1 -- 172.21.15.3:0/2129453209 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1aac131810 msgr2=0x7f1aac1c8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:51.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.489+0000 7f1ab4566700 1 --2- 172.21.15.3:0/2129453209 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1aac131810 0x7f1aac1c8470 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800c700 tx=0x7f1aa800ca10 comp rx=0 tx=0).stop 2023-10-03T10:13:51.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.489+0000 7f1ab4566700 1 -- 172.21.15.3:0/2129453209 shutdown_connections 2023-10-03T10:13:51.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.489+0000 7f1ab4566700 1 --2- 172.21.15.3:0/2129453209 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1aac136f50 0x7f1aac06f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:51.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.489+0000 7f1ab4566700 1 --2- 172.21.15.3:0/2129453209 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1aac131810 0x7f1aac1c8470 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:51.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.489+0000 7f1ab4566700 1 --2- 172.21.15.3:0/2129453209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1aac134630 0x7f1aac1c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:51.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.489+0000 7f1ab4566700 1 -- 172.21.15.3:0/2129453209 >> 172.21.15.3:0/2129453209 conn(0x7f1aac0a2aa0 msgr2=0x7f1aac09f920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:51.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.489+0000 7f1ab4566700 1 -- 172.21.15.3:0/2129453209 shutdown_connections 2023-10-03T10:13:51.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.489+0000 7f1ab4566700 1 -- 172.21.15.3:0/2129453209 wait complete. 2023-10-03T10:13:51.503 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:51.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.523+0000 7f941fd28700 1 -- 172.21.15.3:0/1852332977 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f941809d6c0 con 0x7f941812cf60 2023-10-03T10:13:51.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.523+0000 7f940e7fc700 1 -- 172.21.15.3:0/1852332977 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9414014070 con 0x7f941812cf60 2023-10-03T10:13:51.524 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:51.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.526+0000 7f941fd28700 1 -- 172.21.15.3:0/1852332977 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f941812cf60 msgr2=0x7f94181c4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:51.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.526+0000 7f941fd28700 1 --2- 172.21.15.3:0/1852332977 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f941812cf60 0x7f94181c4700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f941812d470 tx=0x7f9414004750 comp rx=0 tx=0).stop 2023-10-03T10:13:51.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.526+0000 7f941fd28700 1 -- 172.21.15.3:0/1852332977 shutdown_connections 2023-10-03T10:13:51.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.526+0000 7f941fd28700 1 --2- 172.21.15.3:0/1852332977 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9418130430 0x7f94181c0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:51.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.526+0000 7f941fd28700 1 --2- 172.21.15.3:0/1852332977 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f941812cf60 0x7f94181c4700 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:51.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.526+0000 7f941fd28700 1 --2- 172.21.15.3:0/1852332977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f941812f8b0 0x7f94181c4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:51.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.526+0000 7f941fd28700 1 -- 172.21.15.3:0/1852332977 >> 172.21.15.3:0/1852332977 conn(0x7f94180a2aa0 msgr2=0x7f941812a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:51.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.526+0000 7f941fd28700 1 -- 172.21.15.3:0/1852332977 shutdown_connections 2023-10-03T10:13:51.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:51.526+0000 7f941fd28700 1 -- 172.21.15.3:0/1852332977 wait complete. 2023-10-03T10:13:51.539 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:52.164 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:52.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.314+0000 7f4c040fd700 1 Processor -- start 2023-10-03T10:13:52.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.314+0000 7f4c040fd700 1 -- start start 2023-10-03T10:13:52.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.314+0000 7f4c040fd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bfc09e930 0x7f4bfc09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:52.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.314+0000 7f4c040fd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4bfc09f650 0x7f4bfc131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:52.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.314+0000 7f4c040fd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bfc070940 con 0x7f4bfc09f240 2023-10-03T10:13:52.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.314+0000 7f4c040fd700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bfc06fd60 con 0x7f4bfc09f650 2023-10-03T10:13:52.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.315+0000 7f4c040fd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc06fa80 con 0x7f4bfc09e930 2023-10-03T10:13:52.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.315+0000 7f4c0269a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4bfc09f650 0x7f4bfc131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:52.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.315+0000 7f4c01e99700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bfc09e930 0x7f4bfc09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:52.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.315+0000 7f4c0269a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4bfc09f650 0x7f4bfc131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46972/0 (socket says 172.21.15.3:46972) 2023-10-03T10:13:52.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.315+0000 7f4c01e99700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bfc09e930 0x7f4bfc09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46464/0 (socket says 172.21.15.3:46464) 2023-10-03T10:13:52.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.315+0000 7f4c0269a700 1 -- 172.21.15.3:0/3008436682 learned_addr learned my addr 172.21.15.3:0/3008436682 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:52.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.315+0000 7f4c01e99700 1 -- 172.21.15.3:0/3008436682 >> v2:172.21.15.3:3300/0 conn(0x7f4bfc09f650 msgr2=0x7f4bfc131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:52.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.315+0000 7f4c01e99700 1 --2- 172.21.15.3:0/3008436682 >> v2:172.21.15.3:3300/0 conn(0x7f4bfc09f650 0x7f4bfc131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.315+0000 7f4c01e99700 1 -- 172.21.15.3:0/3008436682 >> v1:172.21.15.3:6789/0 conn(0x7f4bfc09f240 legacy=0x7f4bfc1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:52.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.315+0000 7f4c01e99700 1 -- 172.21.15.3:0/3008436682 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bfc09fb60 con 0x7f4bfc09e930 2023-10-03T10:13:52.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.315+0000 7f4c0269a700 1 --2- 172.21.15.3:0/3008436682 >> v2:172.21.15.3:3300/0 conn(0x7f4bfc09f650 0x7f4bfc131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:52.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.315+0000 7f4c01e99700 1 --2- 172.21.15.3:0/3008436682 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bfc09e930 0x7f4bfc09ed00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8009840 tx=0x7f4bf8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=214594f7395819d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:52.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.315+0000 7f4c00e97700 1 -- 172.21.15.3:0/3008436682 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8011830 con 0x7f4bfc09e930 2023-10-03T10:13:52.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.315+0000 7f4c00e97700 1 -- 172.21.15.3:0/3008436682 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf8011990 con 0x7f4bfc09e930 2023-10-03T10:13:52.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.315+0000 7f4c00e97700 1 -- 172.21.15.3:0/3008436682 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8011d20 con 0x7f4bfc09e930 2023-10-03T10:13:52.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.316+0000 7f4c040fd700 1 -- 172.21.15.3:0/3008436682 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bfc09e930 msgr2=0x7f4bfc09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:52.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.316+0000 7f4c040fd700 1 --2- 172.21.15.3:0/3008436682 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bfc09e930 0x7f4bfc09ed00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8009840 tx=0x7f4bf8009b50 comp rx=0 tx=0).stop 2023-10-03T10:13:52.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.316+0000 7f4c040fd700 1 -- 172.21.15.3:0/3008436682 shutdown_connections 2023-10-03T10:13:52.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.316+0000 7f4c040fd700 1 --2- 172.21.15.3:0/3008436682 >> v2:172.21.15.3:3300/0 conn(0x7f4bfc09f650 0x7f4bfc131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.316+0000 7f4c040fd700 1 --2- 172.21.15.3:0/3008436682 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bfc09e930 0x7f4bfc09ed00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.316+0000 7f4c040fd700 1 -- 172.21.15.3:0/3008436682 >> 172.21.15.3:0/3008436682 conn(0x7f4bfc0a4210 msgr2=0x7f4bfc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:52.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.316+0000 7f4c040fd700 1 -- 172.21.15.3:0/3008436682 shutdown_connections 2023-10-03T10:13:52.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.316+0000 7f4c040fd700 1 -- 172.21.15.3:0/3008436682 wait complete. 2023-10-03T10:13:52.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.317+0000 7f4c040fd700 1 Processor -- start 2023-10-03T10:13:52.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.317+0000 7f4c040fd700 1 -- start start 2023-10-03T10:13:52.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.318+0000 7f4c040fd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4bfc09e930 0x7f4bfc06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:52.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.318+0000 7f4c040fd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bfc09f240 0x7f4bfc06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:52.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.318+0000 7f4c040fd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bfc09f650 0x7f4bfc06c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:52.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.318+0000 7f4c040fd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc1b7240 con 0x7f4bfc09f650 2023-10-03T10:13:52.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.318+0000 7f4c040fd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc1a93f0 con 0x7f4bfc09f240 2023-10-03T10:13:52.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.318+0000 7f4c040fd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc1a8910 con 0x7f4bfc09e930 2023-10-03T10:13:52.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.318+0000 7f4c01e99700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4bfc09e930 0x7f4bfc06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:52.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.318+0000 7f4c0269a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bfc09f650 0x7f4bfc06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:52.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.318+0000 7f4c01698700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bfc09f240 0x7f4bfc06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:52.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.318+0000 7f4c01e99700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4bfc09e930 0x7f4bfc06b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55914/0 (socket says 172.21.15.3:55914) 2023-10-03T10:13:52.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.318+0000 7f4c01e99700 1 -- 172.21.15.3:0/2398158272 learned_addr learned my addr 172.21.15.3:0/2398158272 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:52.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.318+0000 7f4c01e99700 1 -- 172.21.15.3:0/2398158272 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bfc09f240 msgr2=0x7f4bfc06bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:52.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.318+0000 7f4c01e99700 1 --2- 172.21.15.3:0/2398158272 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bfc09f240 0x7f4bfc06bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.318+0000 7f4c01e99700 1 -- 172.21.15.3:0/2398158272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bfc09f650 msgr2=0x7f4bfc06c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:52.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.318+0000 7f4c01e99700 1 --2- 172.21.15.3:0/2398158272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bfc09f650 0x7f4bfc06c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.318+0000 7f4c01e99700 1 -- 172.21.15.3:0/2398158272 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf4009b30 con 0x7f4bfc09e930 2023-10-03T10:13:52.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.318+0000 7f4c01698700 1 --2- 172.21.15.3:0/2398158272 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bfc09f240 0x7f4bfc06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:52.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.318+0000 7f4c01e99700 1 --2- 172.21.15.3:0/2398158272 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4bfc09e930 0x7f4bfc06b8b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8009fd0 tx=0x7f4bf8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:52.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.319+0000 7f4bf27fc700 1 -- 172.21.15.3:0/2398158272 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf80036a0 con 0x7f4bfc09e930 2023-10-03T10:13:52.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.319+0000 7f4bf27fc700 1 -- 172.21.15.3:0/2398158272 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf801f070 con 0x7f4bfc09e930 2023-10-03T10:13:52.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.319+0000 7f4c040fd700 1 -- 172.21.15.3:0/2398158272 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf80094a0 con 0x7f4bfc09e930 2023-10-03T10:13:52.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.319+0000 7f4bf27fc700 1 -- 172.21.15.3:0/2398158272 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8003a60 con 0x7f4bfc09e930 2023-10-03T10:13:52.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.319+0000 7f4c040fd700 1 -- 172.21.15.3:0/2398158272 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bfc0a0190 con 0x7f4bfc09e930 2023-10-03T10:13:52.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.319+0000 7f4bf27fc700 1 -- 172.21.15.3:0/2398158272 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4bf8003bc0 con 0x7f4bfc09e930 2023-10-03T10:13:52.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.320+0000 7f4bf27fc700 1 -- 172.21.15.3:0/2398158272 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4bf8021070 con 0x7f4bfc09e930 2023-10-03T10:13:52.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.320+0000 7f4c040fd700 1 -- 172.21.15.3:0/2398158272 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bfc1a9180 con 0x7f4bfc09e930 2023-10-03T10:13:52.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.325+0000 7f4bf27fc700 1 -- 172.21.15.3:0/2398158272 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4bfc1a9180 con 0x7f4bfc09e930 2023-10-03T10:13:52.464 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:52.504 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:52.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.529+0000 7f4c040fd700 1 -- 172.21.15.3:0/2398158272 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4bfc1a9180 con 0x7f4bfc09e930 2023-10-03T10:13:52.530 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:52.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.530+0000 7f4bf27fc700 1 -- 172.21.15.3:0/2398158272 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4bfc1a9180 con 0x7f4bfc09e930 2023-10-03T10:13:52.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.532+0000 7f4c040fd700 1 -- 172.21.15.3:0/2398158272 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4bfc09e930 msgr2=0x7f4bfc06b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:52.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.532+0000 7f4c040fd700 1 --2- 172.21.15.3:0/2398158272 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4bfc09e930 0x7f4bfc06b8b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8009fd0 tx=0x7f4bf8004750 comp rx=0 tx=0).stop 2023-10-03T10:13:52.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.532+0000 7f4c040fd700 1 -- 172.21.15.3:0/2398158272 shutdown_connections 2023-10-03T10:13:52.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.532+0000 7f4c040fd700 1 --2- 172.21.15.3:0/2398158272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bfc09f650 0x7f4bfc06c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.532+0000 7f4c040fd700 1 --2- 172.21.15.3:0/2398158272 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4bfc09e930 0x7f4bfc06b8b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.532+0000 7f4c040fd700 1 --2- 172.21.15.3:0/2398158272 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bfc09f240 0x7f4bfc06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.532+0000 7f4c040fd700 1 -- 172.21.15.3:0/2398158272 >> 172.21.15.3:0/2398158272 conn(0x7f4bfc0a4210 msgr2=0x7f4bfc1b8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:52.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.532+0000 7f4c040fd700 1 -- 172.21.15.3:0/2398158272 shutdown_connections 2023-10-03T10:13:52.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.532+0000 7f4c040fd700 1 -- 172.21.15.3:0/2398158272 wait complete. 2023-10-03T10:13:52.540 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:52.548 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:52.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.609+0000 7f82f67e5700 1 Processor -- start 2023-10-03T10:13:52.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.609+0000 7f82f67e5700 1 -- start start 2023-10-03T10:13:52.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.609+0000 7f82f67e5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f82f009e930 0x7f82f009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:52.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.609+0000 7f82f67e5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f82f009f240 0x7f82f01317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:52.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.610+0000 7f82f67e5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f0070940 con 0x7f82f009f650 2023-10-03T10:13:52.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.610+0000 7f82f67e5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f006fd60 con 0x7f82f009e930 2023-10-03T10:13:52.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.610+0000 7f82f67e5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f006fa80 con 0x7f82f009f240 2023-10-03T10:13:52.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.610+0000 7f82ef7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f82f009f240 0x7f82f01317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:52.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.610+0000 7f82effff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f82f009e930 0x7f82f009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:52.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.610+0000 7f82effff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f82f009e930 0x7f82f009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46980/0 (socket says 172.21.15.3:46980) 2023-10-03T10:13:52.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.610+0000 7f82ef7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f82f009f240 0x7f82f01317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46480/0 (socket says 172.21.15.3:46480) 2023-10-03T10:13:52.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.610+0000 7f82effff700 1 -- 172.21.15.3:0/345004671 learned_addr learned my addr 172.21.15.3:0/345004671 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:52.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.610+0000 7f82f4d82700 1 --1- 172.21.15.3:0/345004671 >> v1:172.21.15.3:6789/0 conn(0x7f82f009f650 0x7f82f0131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45780/0 (socket says 172.21.15.3:45780) 2023-10-03T10:13:52.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.610+0000 7f82effff700 1 -- 172.21.15.3:0/345004671 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f82f009f240 msgr2=0x7f82f01317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:52.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.610+0000 7f82effff700 1 --2- 172.21.15.3:0/345004671 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f82f009f240 0x7f82f01317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.610+0000 7f82effff700 1 -- 172.21.15.3:0/345004671 >> v1:172.21.15.3:6789/0 conn(0x7f82f009f650 legacy=0x7f82f0131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:52.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.610+0000 7f82effff700 1 -- 172.21.15.3:0/345004671 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f009fb60 con 0x7f82f009e930 2023-10-03T10:13:52.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.610+0000 7f82ef7fe700 1 --2- 172.21.15.3:0/345004671 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f82f009f240 0x7f82f01317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:52.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.610+0000 7f82effff700 1 --2- 172.21.15.3:0/345004671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f009e930 0x7f82f009ed00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f82e00098f0 tx=0x7f82e0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7f18a3fa59be66 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:52.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.610+0000 7f82eeffd700 1 -- 172.21.15.3:0/345004671 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e0011830 con 0x7f82f009e930 2023-10-03T10:13:52.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.611+0000 7f82eeffd700 1 -- 172.21.15.3:0/345004671 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e0011990 con 0x7f82f009e930 2023-10-03T10:13:52.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.611+0000 7f82eeffd700 1 -- 172.21.15.3:0/345004671 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e0011d20 con 0x7f82f009e930 2023-10-03T10:13:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.611+0000 7f82f67e5700 1 -- 172.21.15.3:0/345004671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f009e930 msgr2=0x7f82f009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.611+0000 7f82f67e5700 1 --2- 172.21.15.3:0/345004671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f009e930 0x7f82f009ed00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f82e00098f0 tx=0x7f82e0009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.611+0000 7f82f67e5700 1 -- 172.21.15.3:0/345004671 shutdown_connections 2023-10-03T10:13:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.611+0000 7f82f67e5700 1 --2- 172.21.15.3:0/345004671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f009e930 0x7f82f009ed00 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.611+0000 7f82f67e5700 1 --2- 172.21.15.3:0/345004671 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f82f009f240 0x7f82f01317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.611+0000 7f82f67e5700 1 -- 172.21.15.3:0/345004671 >> 172.21.15.3:0/345004671 conn(0x7f82f00a4210 msgr2=0x7f82f00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.611+0000 7f82f67e5700 1 -- 172.21.15.3:0/345004671 shutdown_connections 2023-10-03T10:13:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.611+0000 7f82f67e5700 1 -- 172.21.15.3:0/345004671 wait complete. 2023-10-03T10:13:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.613+0000 7f82f67e5700 1 Processor -- start 2023-10-03T10:13:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.613+0000 7f82f67e5700 1 -- start start 2023-10-03T10:13:52.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.614+0000 7f82f67e5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82f009e930 0x7f82f01a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:52.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.614+0000 7f82f67e5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f82f009f240 0x7f82f01a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:52.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.614+0000 7f82f67e5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f009f650 0x7f82f01a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:52.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.614+0000 7f82f67e5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f01a8260 con 0x7f82f009f650 2023-10-03T10:13:52.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.614+0000 7f82f67e5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f01a78a0 con 0x7f82f009f240 2023-10-03T10:13:52.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.614+0000 7f82f67e5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f82f01a6be0 con 0x7f82f009e930 2023-10-03T10:13:52.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.614+0000 7f82ef7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f82f009f240 0x7f82f01a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:52.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.614+0000 7f82f4d82700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f009f650 0x7f82f01a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:52.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.614+0000 7f82ef7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f82f009f240 0x7f82f01a42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46492/0 (socket says 172.21.15.3:46492) 2023-10-03T10:13:52.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.614+0000 7f82ef7fe700 1 -- 172.21.15.3:0/168654335 learned_addr learned my addr 172.21.15.3:0/168654335 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:52.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.615+0000 7f82f4d82700 1 -- 172.21.15.3:0/168654335 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82f009e930 msgr2=0x7f82f01a3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:52.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.615+0000 7f82f4d82700 1 --2- 172.21.15.3:0/168654335 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82f009e930 0x7f82f01a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.615+0000 7f82f4d82700 1 -- 172.21.15.3:0/168654335 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f82f009f240 msgr2=0x7f82f01a42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:52.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.615+0000 7f82f4d82700 1 --2- 172.21.15.3:0/168654335 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f82f009f240 0x7f82f01a42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.615+0000 7f82f4d82700 1 -- 172.21.15.3:0/168654335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e40094a0 con 0x7f82f009f650 2023-10-03T10:13:52.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.615+0000 7f82ef7fe700 1 --2- 172.21.15.3:0/168654335 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f82f009f240 0x7f82f01a42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:52.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.615+0000 7f82f4d82700 1 --2- 172.21.15.3:0/168654335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f009f650 0x7f82f01a4800 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f82e800eeb0 tx=0x7f82e8010330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:52.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.615+0000 7f82ecff9700 1 -- 172.21.15.3:0/168654335 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8013070 con 0x7f82f009f650 2023-10-03T10:13:52.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.615+0000 7f82ecff9700 1 -- 172.21.15.3:0/168654335 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e800e2e0 con 0x7f82f009f650 2023-10-03T10:13:52.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.615+0000 7f82f67e5700 1 -- 172.21.15.3:0/168654335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e0009550 con 0x7f82f009f650 2023-10-03T10:13:52.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.615+0000 7f82ecff9700 1 -- 172.21.15.3:0/168654335 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8010bc0 con 0x7f82f009f650 2023-10-03T10:13:52.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.615+0000 7f82f67e5700 1 -- 172.21.15.3:0/168654335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f00a0190 con 0x7f82f009f650 2023-10-03T10:13:52.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.616+0000 7f82ecff9700 1 -- 172.21.15.3:0/168654335 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f82e8010d20 con 0x7f82f009f650 2023-10-03T10:13:52.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.616+0000 7f82ecff9700 1 -- 172.21.15.3:0/168654335 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f82e8043760 con 0x7f82f009f650 2023-10-03T10:13:52.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.616+0000 7f82f67e5700 1 -- 172.21.15.3:0/168654335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f01a59f0 con 0x7f82f009f650 2023-10-03T10:13:52.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.622+0000 7f82ecff9700 1 -- 172.21.15.3:0/168654335 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f82e8021440 con 0x7f82f009f650 2023-10-03T10:13:52.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.666+0000 7f2e14ffa700 1 Processor -- start 2023-10-03T10:13:52.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.666+0000 7f2e14ffa700 1 -- start start 2023-10-03T10:13:52.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.666+0000 7f2e14ffa700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e1009e930 0x7f2e1009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:52.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.666+0000 7f2e14ffa700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2e1009f240 0x7f2e101317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:52.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.666+0000 7f2e14ffa700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e1009f650 0x7f2e10131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:52.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.666+0000 7f2e14ffa700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e10070940 con 0x7f2e1009f240 2023-10-03T10:13:52.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.666+0000 7f2e14ffa700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1006fd60 con 0x7f2e1009e930 2023-10-03T10:13:52.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.666+0000 7f2e14ffa700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1006fa80 con 0x7f2e1009f650 2023-10-03T10:13:52.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.667+0000 7f2e0e59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e1009e930 0x7f2e1009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:52.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.667+0000 7f2e0e59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e1009e930 0x7f2e1009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46504/0 (socket says 172.21.15.3:46504) 2023-10-03T10:13:52.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.667+0000 7f2e0ed9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e1009f650 0x7f2e10131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:52.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.667+0000 7f2e0e59c700 1 -- 172.21.15.3:0/238912437 learned_addr learned my addr 172.21.15.3:0/238912437 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:52.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.667+0000 7f2e07fff700 1 --2- 172.21.15.3:0/238912437 >> v2:172.21.15.3:3300/0 conn(0x7f2e1009f240 0x7f2e101317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:52.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.667+0000 7f2e0e59c700 1 -- 172.21.15.3:0/238912437 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e1009f650 msgr2=0x7f2e10131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:52.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.667+0000 7f2e0e59c700 1 --2- 172.21.15.3:0/238912437 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e1009f650 0x7f2e10131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.667+0000 7f2e0e59c700 1 -- 172.21.15.3:0/238912437 >> v2:172.21.15.3:3300/0 conn(0x7f2e1009f240 msgr2=0x7f2e101317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:52.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.667+0000 7f2e0e59c700 1 --2- 172.21.15.3:0/238912437 >> v2:172.21.15.3:3300/0 conn(0x7f2e1009f240 0x7f2e101317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.667+0000 7f2e0e59c700 1 -- 172.21.15.3:0/238912437 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e1009fb60 con 0x7f2e1009e930 2023-10-03T10:13:52.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.667+0000 7f2e0ed9d700 1 --2- 172.21.15.3:0/238912437 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e1009f650 0x7f2e10131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:52.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.667+0000 7f2e0e59c700 1 --2- 172.21.15.3:0/238912437 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e1009e930 0x7f2e1009ed00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2e000098f0 tx=0x7f2e00009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35cc47e015d5213b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:52.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.667+0000 7f2e0dd9b700 1 -- 172.21.15.3:0/238912437 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e00011830 con 0x7f2e1009e930 2023-10-03T10:13:52.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.668+0000 7f2e0dd9b700 1 -- 172.21.15.3:0/238912437 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e00011990 con 0x7f2e1009e930 2023-10-03T10:13:52.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.668+0000 7f2e0dd9b700 1 -- 172.21.15.3:0/238912437 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e00011d20 con 0x7f2e1009e930 2023-10-03T10:13:52.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.668+0000 7f2e14ffa700 1 -- 172.21.15.3:0/238912437 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e1009e930 msgr2=0x7f2e1009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:52.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.668+0000 7f2e14ffa700 1 --2- 172.21.15.3:0/238912437 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e1009e930 0x7f2e1009ed00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2e000098f0 tx=0x7f2e00009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:52.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.668+0000 7f2e14ffa700 1 -- 172.21.15.3:0/238912437 shutdown_connections 2023-10-03T10:13:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.668+0000 7f2e14ffa700 1 --2- 172.21.15.3:0/238912437 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e1009f650 0x7f2e10131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.668+0000 7f2e14ffa700 1 --2- 172.21.15.3:0/238912437 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e1009e930 0x7f2e1009ed00 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.668+0000 7f2e14ffa700 1 --2- 172.21.15.3:0/238912437 >> v2:172.21.15.3:3300/0 conn(0x7f2e1009f240 0x7f2e101317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.668+0000 7f2e14ffa700 1 -- 172.21.15.3:0/238912437 >> 172.21.15.3:0/238912437 conn(0x7f2e100a4210 msgr2=0x7f2e100a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.668+0000 7f2e14ffa700 1 -- 172.21.15.3:0/238912437 shutdown_connections 2023-10-03T10:13:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.668+0000 7f2e14ffa700 1 -- 172.21.15.3:0/238912437 wait complete. 2023-10-03T10:13:52.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.670+0000 7f2e14ffa700 1 Processor -- start 2023-10-03T10:13:52.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.670+0000 7f2e14ffa700 1 -- start start 2023-10-03T10:13:52.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.671+0000 7f2e14ffa700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e1009e930 0x7f2e1006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:52.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.671+0000 7f2e14ffa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e1009f240 0x7f2e1006bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:52.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.671+0000 7f2e14ffa700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e1009f650 0x7f2e1006c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:52.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.671+0000 7f2e14ffa700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1006cd80 con 0x7f2e1009f240 2023-10-03T10:13:52.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.671+0000 7f2e14ffa700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e101a8920 con 0x7f2e1009f650 2023-10-03T10:13:52.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.671+0000 7f2e14ffa700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e101b7210 con 0x7f2e1009e930 2023-10-03T10:13:52.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.671+0000 7f2e07fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e1009f240 0x7f2e1006bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:52.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.671+0000 7f2e0ed9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e1009f650 0x7f2e1006c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:52.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.671+0000 7f2e07fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e1009f240 0x7f2e1006bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47006/0 (socket says 172.21.15.3:47006) 2023-10-03T10:13:52.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.671+0000 7f2e07fff700 1 -- 172.21.15.3:0/863618388 learned_addr learned my addr 172.21.15.3:0/863618388 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:52.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.671+0000 7f2e0e59c700 1 --2- 172.21.15.3:0/863618388 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e1009e930 0x7f2e1006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:52.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.671+0000 7f2e0e59c700 1 -- 172.21.15.3:0/863618388 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e1009f650 msgr2=0x7f2e1006c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:52.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.671+0000 7f2e0e59c700 1 --2- 172.21.15.3:0/863618388 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e1009f650 0x7f2e1006c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.671+0000 7f2e0e59c700 1 -- 172.21.15.3:0/863618388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e1009f240 msgr2=0x7f2e1006bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:52.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.671+0000 7f2e0e59c700 1 --2- 172.21.15.3:0/863618388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e1009f240 0x7f2e1006bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.671+0000 7f2e0e59c700 1 -- 172.21.15.3:0/863618388 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e08009a70 con 0x7f2e1009e930 2023-10-03T10:13:52.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.671+0000 7f2e07fff700 1 --2- 172.21.15.3:0/863618388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e1009f240 0x7f2e1006bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:52.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.672+0000 7f2e0ed9d700 1 --2- 172.21.15.3:0/863618388 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e1009f650 0x7f2e1006c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:52.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.672+0000 7f2e0e59c700 1 --2- 172.21.15.3:0/863618388 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e1009e930 0x7f2e1006b8b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2e00016000 tx=0x7f2e00004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:52.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.672+0000 7f2e06ffd700 1 -- 172.21.15.3:0/863618388 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e00004300 con 0x7f2e1009e930 2023-10-03T10:13:52.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.672+0000 7f2e06ffd700 1 -- 172.21.15.3:0/863618388 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0001f070 con 0x7f2e1009e930 2023-10-03T10:13:52.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.672+0000 7f2e14ffa700 1 -- 172.21.15.3:0/863618388 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e00009550 con 0x7f2e1009e930 2023-10-03T10:13:52.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.672+0000 7f2e06ffd700 1 -- 172.21.15.3:0/863618388 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e000036a0 con 0x7f2e1009e930 2023-10-03T10:13:52.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.672+0000 7f2e14ffa700 1 -- 172.21.15.3:0/863618388 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e100a0190 con 0x7f2e1009e930 2023-10-03T10:13:52.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.673+0000 7f2e06ffd700 1 -- 172.21.15.3:0/863618388 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2e00021030 con 0x7f2e1009e930 2023-10-03T10:13:52.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.673+0000 7f2e06ffd700 1 -- 172.21.15.3:0/863618388 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2e00040750 con 0x7f2e1009e930 2023-10-03T10:13:52.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.674+0000 7f2e14ffa700 1 -- 172.21.15.3:0/863618388 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e101a9320 con 0x7f2e1009e930 2023-10-03T10:13:52.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.679+0000 7f2e06ffd700 1 -- 172.21.15.3:0/863618388 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2e000199a0 con 0x7f2e1009e930 2023-10-03T10:13:52.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.686+0000 7ff1e29e3700 1 Processor -- start 2023-10-03T10:13:52.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.687+0000 7ff1e29e3700 1 -- start start 2023-10-03T10:13:52.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.687+0000 7ff1e29e3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1dc09d6f0 0x7ff1dc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:52.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.687+0000 7ff1e29e3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1dc09e410 0x7ff1dc134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:52.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.687+0000 7ff1e29e3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1dc09b120 con 0x7ff1dc09e000 2023-10-03T10:13:52.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.687+0000 7ff1e29e3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc06b460 con 0x7ff1dc09e410 2023-10-03T10:13:52.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.687+0000 7ff1e29e3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc09a7e0 con 0x7ff1dc09d6f0 2023-10-03T10:13:52.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.687+0000 7ff1db7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff1dc09e000 0x7ff1dc1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45784/0 (socket says 172.21.15.3:45784) 2023-10-03T10:13:52.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.687+0000 7ff1db7fe700 1 -- 172.21.15.3:0/1308258835 learned_addr learned my addr 172.21.15.3:0/1308258835 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:52.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.687+0000 7ff1dbfff700 1 --2- 172.21.15.3:0/1308258835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1dc09d6f0 0x7ff1dc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:52.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.687+0000 7ff1e0f80700 1 --2- 172.21.15.3:0/1308258835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1dc09e410 0x7ff1dc134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:52.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.688+0000 7ff1daffd700 1 -- 172.21.15.3:0/1308258835 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3118357699 0 0) 0x7ff1dc09b120 con 0x7ff1dc09e000 2023-10-03T10:13:52.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.688+0000 7ff1daffd700 1 -- 172.21.15.3:0/1308258835 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1c40035b0 con 0x7ff1dc09e000 2023-10-03T10:13:52.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.688+0000 7ff1dbfff700 1 -- 172.21.15.3:0/1308258835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1dc09e410 msgr2=0x7ff1dc134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:52.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.688+0000 7ff1dbfff700 1 --2- 172.21.15.3:0/1308258835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1dc09e410 0x7ff1dc134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.688+0000 7ff1dbfff700 1 -- 172.21.15.3:0/1308258835 >> v1:172.21.15.3:6789/0 conn(0x7ff1dc09e000 legacy=0x7ff1dc1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:52.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.688+0000 7ff1dbfff700 1 -- 172.21.15.3:0/1308258835 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1dc09a500 con 0x7ff1dc09d6f0 2023-10-03T10:13:52.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.688+0000 7ff1e0f80700 1 --2- 172.21.15.3:0/1308258835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1dc09e410 0x7ff1dc134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:52.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.688+0000 7ff1dbfff700 1 --2- 172.21.15.3:0/1308258835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1dc09d6f0 0x7ff1dc09dac0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0098a0 tx=0x7ff1cc009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0d93560251e8ed server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:52.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.688+0000 7ff1daffd700 1 -- 172.21.15.3:0/1308258835 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc011830 con 0x7ff1dc09d6f0 2023-10-03T10:13:52.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.688+0000 7ff1daffd700 1 -- 172.21.15.3:0/1308258835 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1cc011990 con 0x7ff1dc09d6f0 2023-10-03T10:13:52.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.688+0000 7ff1daffd700 1 -- 172.21.15.3:0/1308258835 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc011d00 con 0x7ff1dc09d6f0 2023-10-03T10:13:52.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.689+0000 7ff1e29e3700 1 -- 172.21.15.3:0/1308258835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1dc09d6f0 msgr2=0x7ff1dc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:52.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.689+0000 7ff1e29e3700 1 --2- 172.21.15.3:0/1308258835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1dc09d6f0 0x7ff1dc09dac0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0098a0 tx=0x7ff1cc009bb0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.689+0000 7ff1e29e3700 1 -- 172.21.15.3:0/1308258835 shutdown_connections 2023-10-03T10:13:52.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.689+0000 7ff1e29e3700 1 --2- 172.21.15.3:0/1308258835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1dc09e410 0x7ff1dc134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.689+0000 7ff1e29e3700 1 --2- 172.21.15.3:0/1308258835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1dc09d6f0 0x7ff1dc09dac0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.689+0000 7ff1e29e3700 1 -- 172.21.15.3:0/1308258835 >> 172.21.15.3:0/1308258835 conn(0x7ff1dc066f50 msgr2=0x7ff1dc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:52.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.689+0000 7ff1e29e3700 1 -- 172.21.15.3:0/1308258835 shutdown_connections 2023-10-03T10:13:52.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.689+0000 7ff1e29e3700 1 -- 172.21.15.3:0/1308258835 wait complete. 2023-10-03T10:13:52.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.690+0000 7ff1e29e3700 1 Processor -- start 2023-10-03T10:13:52.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.690+0000 7ff1e29e3700 1 -- start start 2023-10-03T10:13:52.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.691+0000 7ff1e29e3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1dc09d6f0 0x7ff1dc122240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:52.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.691+0000 7ff1e29e3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1dc09e000 0x7ff1dc122780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:52.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.691+0000 7ff1e29e3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1dc09e410 0x7ff1dc122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:52.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.691+0000 7ff1e29e3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc125b60 con 0x7ff1dc09e000 2023-10-03T10:13:52.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.691+0000 7ff1e29e3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc1257c0 con 0x7ff1dc09d6f0 2023-10-03T10:13:52.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.691+0000 7ff1e29e3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc124c50 con 0x7ff1dc09e410 2023-10-03T10:13:52.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.691+0000 7ff1dbfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1dc09d6f0 0x7ff1dc122240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:52.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.691+0000 7ff1dbfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1dc09d6f0 0x7ff1dc122240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46532/0 (socket says 172.21.15.3:46532) 2023-10-03T10:13:52.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.691+0000 7ff1dbfff700 1 -- 172.21.15.3:0/575220358 learned_addr learned my addr 172.21.15.3:0/575220358 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:52.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.691+0000 7ff1db7fe700 1 --2- 172.21.15.3:0/575220358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1dc09e000 0x7ff1dc122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:52.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.691+0000 7ff1e0f80700 1 --2- 172.21.15.3:0/575220358 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1dc09e410 0x7ff1dc122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:52.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.691+0000 7ff1dbfff700 1 -- 172.21.15.3:0/575220358 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1dc09e410 msgr2=0x7ff1dc122cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:52.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.691+0000 7ff1dbfff700 1 --2- 172.21.15.3:0/575220358 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1dc09e410 0x7ff1dc122cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.691+0000 7ff1dbfff700 1 -- 172.21.15.3:0/575220358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1dc09e000 msgr2=0x7ff1dc122780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:52.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.691+0000 7ff1dbfff700 1 --2- 172.21.15.3:0/575220358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1dc09e000 0x7ff1dc122780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.691+0000 7ff1dbfff700 1 -- 172.21.15.3:0/575220358 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d4009a70 con 0x7ff1dc09d6f0 2023-10-03T10:13:52.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.691+0000 7ff1db7fe700 1 --2- 172.21.15.3:0/575220358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1dc09e000 0x7ff1dc122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:52.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.692+0000 7ff1dbfff700 1 --2- 172.21.15.3:0/575220358 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1dc09d6f0 0x7ff1dc122240 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0090b0 tx=0x7ff1cc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:52.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.692+0000 7ff1d8ff9700 1 -- 172.21.15.3:0/575220358 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0044d0 con 0x7ff1dc09d6f0 2023-10-03T10:13:52.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.692+0000 7ff1d8ff9700 1 -- 172.21.15.3:0/575220358 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1cc01f070 con 0x7ff1dc09d6f0 2023-10-03T10:13:52.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.692+0000 7ff1d8ff9700 1 -- 172.21.15.3:0/575220358 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0038e0 con 0x7ff1dc09d6f0 2023-10-03T10:13:52.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.692+0000 7ff1e29e3700 1 -- 172.21.15.3:0/575220358 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1cc0094a0 con 0x7ff1dc09d6f0 2023-10-03T10:13:52.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.692+0000 7ff1e29e3700 1 -- 172.21.15.3:0/575220358 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1dc0a1aa0 con 0x7ff1dc09d6f0 2023-10-03T10:13:52.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.693+0000 7ff1d8ff9700 1 -- 172.21.15.3:0/575220358 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff1cc003a40 con 0x7ff1dc09d6f0 2023-10-03T10:13:52.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.693+0000 7ff1e29e3700 1 -- 172.21.15.3:0/575220358 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1dc1259c0 con 0x7ff1dc09d6f0 2023-10-03T10:13:52.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.693+0000 7ff1d8ff9700 1 -- 172.21.15.3:0/575220358 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff1cc021070 con 0x7ff1dc09d6f0 2023-10-03T10:13:52.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.698+0000 7ff1d8ff9700 1 -- 172.21.15.3:0/575220358 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff1dc1259c0 con 0x7ff1dc09d6f0 2023-10-03T10:13:52.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.861+0000 7f82f67e5700 1 -- 172.21.15.3:0/168654335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f82f01a7440 con 0x7f82f009f650 2023-10-03T10:13:52.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.861+0000 7f82ecff9700 1 -- 172.21.15.3:0/168654335 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f82e80173c0 con 0x7f82f009f650 2023-10-03T10:13:52.861 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:52.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.863+0000 7f82f67e5700 1 -- 172.21.15.3:0/168654335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f009f650 msgr2=0x7f82f01a4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:52.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.863+0000 7f82f67e5700 1 --2- 172.21.15.3:0/168654335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f009f650 0x7f82f01a4800 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f82e800eeb0 tx=0x7f82e8010330 comp rx=0 tx=0).stop 2023-10-03T10:13:52.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.863+0000 7f82f67e5700 1 -- 172.21.15.3:0/168654335 shutdown_connections 2023-10-03T10:13:52.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.863+0000 7f82f67e5700 1 --2- 172.21.15.3:0/168654335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82f009f650 0x7f82f01a4800 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.863+0000 7f82f67e5700 1 --2- 172.21.15.3:0/168654335 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f82f009e930 0x7f82f01a3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.863+0000 7f82f67e5700 1 --2- 172.21.15.3:0/168654335 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f82f009f240 0x7f82f01a42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.863+0000 7f82f67e5700 1 -- 172.21.15.3:0/168654335 >> 172.21.15.3:0/168654335 conn(0x7f82f00a4210 msgr2=0x7f82f01b8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:52.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.864+0000 7f82f67e5700 1 -- 172.21.15.3:0/168654335 shutdown_connections 2023-10-03T10:13:52.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.864+0000 7f82f67e5700 1 -- 172.21.15.3:0/168654335 wait complete. 2023-10-03T10:13:52.878 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:52.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.899+0000 7f2e14ffa700 1 -- 172.21.15.3:0/863618388 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2e10071d20 con 0x7f2e1009e930 2023-10-03T10:13:52.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.899+0000 7f2e06ffd700 1 -- 172.21.15.3:0/863618388 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2e00015020 con 0x7f2e1009e930 2023-10-03T10:13:52.900 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:52.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.901+0000 7f2e14ffa700 1 -- 172.21.15.3:0/863618388 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e1009e930 msgr2=0x7f2e1006b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:52.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.901+0000 7f2e14ffa700 1 --2- 172.21.15.3:0/863618388 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e1009e930 0x7f2e1006b8b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2e00016000 tx=0x7f2e00004750 comp rx=0 tx=0).stop 2023-10-03T10:13:52.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.901+0000 7f2e14ffa700 1 -- 172.21.15.3:0/863618388 shutdown_connections 2023-10-03T10:13:52.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.901+0000 7f2e14ffa700 1 --2- 172.21.15.3:0/863618388 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e1009f650 0x7f2e1006c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.901+0000 7f2e14ffa700 1 --2- 172.21.15.3:0/863618388 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e1009e930 0x7f2e1006b8b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.901+0000 7f2e14ffa700 1 --2- 172.21.15.3:0/863618388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e1009f240 0x7f2e1006bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.901+0000 7f2e14ffa700 1 -- 172.21.15.3:0/863618388 >> 172.21.15.3:0/863618388 conn(0x7f2e100a4210 msgr2=0x7f2e101b8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:52.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.901+0000 7f2e14ffa700 1 -- 172.21.15.3:0/863618388 shutdown_connections 2023-10-03T10:13:52.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.901+0000 7f2e14ffa700 1 -- 172.21.15.3:0/863618388 wait complete. 2023-10-03T10:13:52.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.905+0000 7ff1e29e3700 1 -- 172.21.15.3:0/575220358 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff1dc1259c0 con 0x7ff1dc09d6f0 2023-10-03T10:13:52.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.905+0000 7ff1d8ff9700 1 -- 172.21.15.3:0/575220358 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff1dc1259c0 con 0x7ff1dc09d6f0 2023-10-03T10:13:52.906 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:52.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.907+0000 7ff1e29e3700 1 -- 172.21.15.3:0/575220358 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1dc09d6f0 msgr2=0x7ff1dc122240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:52.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.907+0000 7ff1e29e3700 1 --2- 172.21.15.3:0/575220358 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1dc09d6f0 0x7ff1dc122240 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0090b0 tx=0x7ff1cc004750 comp rx=0 tx=0).stop 2023-10-03T10:13:52.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.908+0000 7ff1e29e3700 1 -- 172.21.15.3:0/575220358 shutdown_connections 2023-10-03T10:13:52.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.908+0000 7ff1e29e3700 1 --2- 172.21.15.3:0/575220358 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1dc09e410 0x7ff1dc122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.908+0000 7ff1e29e3700 1 --2- 172.21.15.3:0/575220358 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1dc09d6f0 0x7ff1dc122240 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.908+0000 7ff1e29e3700 1 --2- 172.21.15.3:0/575220358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1dc09e000 0x7ff1dc122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:52.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.908+0000 7ff1e29e3700 1 -- 172.21.15.3:0/575220358 >> 172.21.15.3:0/575220358 conn(0x7ff1dc066f50 msgr2=0x7ff1dc133e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:52.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.908+0000 7ff1e29e3700 1 -- 172.21.15.3:0/575220358 shutdown_connections 2023-10-03T10:13:52.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:52.908+0000 7ff1e29e3700 1 -- 172.21.15.3:0/575220358 wait complete. 2023-10-03T10:13:52.917 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:52.922 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:53.549 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:53.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.695+0000 7fbd67c24700 1 Processor -- start 2023-10-03T10:13:53.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.695+0000 7fbd67c24700 1 -- start start 2023-10-03T10:13:53.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.696+0000 7fbd67c24700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd6009d6f0 0x7fbd6009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:53.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.696+0000 7fbd67c24700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd6009e000 0x7fbd60130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:53.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.696+0000 7fbd67c24700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbd6009e410 0x7fbd60130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:53.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.696+0000 7fbd67c24700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd6009f520 con 0x7fbd6009e410 2023-10-03T10:13:53.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.696+0000 7fbd67c24700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6009f130 con 0x7fbd6009e000 2023-10-03T10:13:53.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.696+0000 7fbd67c24700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6009e930 con 0x7fbd6009d6f0 2023-10-03T10:13:53.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.696+0000 7fbd651bf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd6009e000 0x7fbd60130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:53.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.696+0000 7fbd659c0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd6009d6f0 0x7fbd6009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:53.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.696+0000 7fbd659c0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd6009d6f0 0x7fbd6009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43616/0 (socket says 172.21.15.3:43616) 2023-10-03T10:13:53.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.696+0000 7fbd651bf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd6009e000 0x7fbd60130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33136/0 (socket says 172.21.15.3:33136) 2023-10-03T10:13:53.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.696+0000 7fbd659c0700 1 -- 172.21.15.3:0/2472306292 learned_addr learned my addr 172.21.15.3:0/2472306292 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:53.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.696+0000 7fbd661c1700 1 --2- 172.21.15.3:0/2472306292 >> v2:172.21.15.3:3300/0 conn(0x7fbd6009e410 0x7fbd60130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:53.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.696+0000 7fbd659c0700 1 -- 172.21.15.3:0/2472306292 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd6009e000 msgr2=0x7fbd60130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:53.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.696+0000 7fbd659c0700 1 --2- 172.21.15.3:0/2472306292 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd6009e000 0x7fbd60130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:53.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.696+0000 7fbd659c0700 1 -- 172.21.15.3:0/2472306292 >> v2:172.21.15.3:3300/0 conn(0x7fbd6009e410 msgr2=0x7fbd60130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:53.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.696+0000 7fbd659c0700 1 --2- 172.21.15.3:0/2472306292 >> v2:172.21.15.3:3300/0 conn(0x7fbd6009e410 0x7fbd60130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:53.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.696+0000 7fbd659c0700 1 -- 172.21.15.3:0/2472306292 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6009b2d0 con 0x7fbd6009d6f0 2023-10-03T10:13:53.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.696+0000 7fbd651bf700 1 --2- 172.21.15.3:0/2472306292 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd6009e000 0x7fbd60130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:53.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.696+0000 7fbd661c1700 1 --2- 172.21.15.3:0/2472306292 >> v2:172.21.15.3:3300/0 conn(0x7fbd6009e410 0x7fbd60130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:53.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.696+0000 7fbd659c0700 1 --2- 172.21.15.3:0/2472306292 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd6009d6f0 0x7fbd6009dac0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c00bbc0 tx=0x7fbd5c00bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f61ca3b2cc0fb954 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:53.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.697+0000 7fbd649be700 1 -- 172.21.15.3:0/2472306292 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c00dcf0 con 0x7fbd6009d6f0 2023-10-03T10:13:53.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.697+0000 7fbd649be700 1 -- 172.21.15.3:0/2472306292 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5c00de50 con 0x7fbd6009d6f0 2023-10-03T10:13:53.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.697+0000 7fbd649be700 1 -- 172.21.15.3:0/2472306292 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c010690 con 0x7fbd6009d6f0 2023-10-03T10:13:53.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.697+0000 7fbd67c24700 1 -- 172.21.15.3:0/2472306292 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd6009d6f0 msgr2=0x7fbd6009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:53.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.697+0000 7fbd67c24700 1 --2- 172.21.15.3:0/2472306292 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd6009d6f0 0x7fbd6009dac0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c00bbc0 tx=0x7fbd5c00bed0 comp rx=0 tx=0).stop 2023-10-03T10:13:53.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.697+0000 7fbd67c24700 1 -- 172.21.15.3:0/2472306292 shutdown_connections 2023-10-03T10:13:53.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.697+0000 7fbd67c24700 1 --2- 172.21.15.3:0/2472306292 >> v2:172.21.15.3:3300/0 conn(0x7fbd6009e410 0x7fbd60130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:53.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.697+0000 7fbd67c24700 1 --2- 172.21.15.3:0/2472306292 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd6009d6f0 0x7fbd6009dac0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:53.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.697+0000 7fbd67c24700 1 --2- 172.21.15.3:0/2472306292 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd6009e000 0x7fbd60130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:53.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.697+0000 7fbd67c24700 1 -- 172.21.15.3:0/2472306292 >> 172.21.15.3:0/2472306292 conn(0x7fbd60066f50 msgr2=0x7fbd6009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:53.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.697+0000 7fbd67c24700 1 -- 172.21.15.3:0/2472306292 shutdown_connections 2023-10-03T10:13:53.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.697+0000 7fbd67c24700 1 -- 172.21.15.3:0/2472306292 wait complete. 2023-10-03T10:13:53.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.698+0000 7fbd67c24700 1 Processor -- start 2023-10-03T10:13:53.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.698+0000 7fbd67c24700 1 -- start start 2023-10-03T10:13:53.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.699+0000 7fbd67c24700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd6009d6f0 0x7fbd60122930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:53.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.699+0000 7fbd67c24700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd6009e000 0x7fbd60122e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:53.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.699+0000 7fbd67c24700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd6009e410 0x7fbd601233b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:53.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.699+0000 7fbd67c24700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd60123e00 con 0x7fbd6009e000 2023-10-03T10:13:53.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.699+0000 7fbd67c24700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd60124d30 con 0x7fbd6009d6f0 2023-10-03T10:13:53.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.699+0000 7fbd67c24700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd60124650 con 0x7fbd6009e410 2023-10-03T10:13:53.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.699+0000 7fbd659c0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd6009d6f0 0x7fbd60122930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:53.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.699+0000 7fbd661c1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd6009e410 0x7fbd601233b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:53.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.699+0000 7fbd659c0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd6009d6f0 0x7fbd60122930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33140/0 (socket says 172.21.15.3:33140) 2023-10-03T10:13:53.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.699+0000 7fbd659c0700 1 -- 172.21.15.3:0/2238173266 learned_addr learned my addr 172.21.15.3:0/2238173266 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:53.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.699+0000 7fbd661c1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd6009e410 0x7fbd601233b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43626/0 (socket says 172.21.15.3:43626) 2023-10-03T10:13:53.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.699+0000 7fbd651bf700 1 --2- 172.21.15.3:0/2238173266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd6009e000 0x7fbd60122e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:53.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.699+0000 7fbd659c0700 1 -- 172.21.15.3:0/2238173266 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd6009e410 msgr2=0x7fbd601233b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:53.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.699+0000 7fbd659c0700 1 --2- 172.21.15.3:0/2238173266 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd6009e410 0x7fbd601233b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:53.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.699+0000 7fbd659c0700 1 -- 172.21.15.3:0/2238173266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd6009e000 msgr2=0x7fbd60122e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:53.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.699+0000 7fbd659c0700 1 --2- 172.21.15.3:0/2238173266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd6009e000 0x7fbd60122e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:53.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.699+0000 7fbd659c0700 1 -- 172.21.15.3:0/2238173266 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd500094a0 con 0x7fbd6009d6f0 2023-10-03T10:13:53.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.699+0000 7fbd651bf700 1 --2- 172.21.15.3:0/2238173266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd6009e000 0x7fbd60122e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:53.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.699+0000 7fbd661c1700 1 --2- 172.21.15.3:0/2238173266 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd6009e410 0x7fbd601233b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:53.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.699+0000 7fbd659c0700 1 --2- 172.21.15.3:0/2238173266 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd6009d6f0 0x7fbd60122930 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c00d5d0 tx=0x7fbd5c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:53.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.700+0000 7fbd567fc700 1 -- 172.21.15.3:0/2238173266 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c003ab0 con 0x7fbd6009d6f0 2023-10-03T10:13:53.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.700+0000 7fbd567fc700 1 -- 172.21.15.3:0/2238173266 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5c003c10 con 0x7fbd6009d6f0 2023-10-03T10:13:53.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.700+0000 7fbd67c24700 1 -- 172.21.15.3:0/2238173266 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd58009a70 con 0x7fbd6009d6f0 2023-10-03T10:13:53.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.700+0000 7fbd567fc700 1 -- 172.21.15.3:0/2238173266 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c010470 con 0x7fbd6009d6f0 2023-10-03T10:13:53.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.700+0000 7fbd67c24700 1 -- 172.21.15.3:0/2238173266 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd5c00b7f0 con 0x7fbd6009d6f0 2023-10-03T10:13:53.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.700+0000 7fbd567fc700 1 -- 172.21.15.3:0/2238173266 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbd5c019400 con 0x7fbd6009d6f0 2023-10-03T10:13:53.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.701+0000 7fbd567fc700 1 -- 172.21.15.3:0/2238173266 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fbd5c0407e0 con 0x7fbd6009d6f0 2023-10-03T10:13:53.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.701+0000 7fbd67c24700 1 -- 172.21.15.3:0/2238173266 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd60074ea0 con 0x7fbd6009d6f0 2023-10-03T10:13:53.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.705+0000 7fbd567fc700 1 -- 172.21.15.3:0/2238173266 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbd5c0178f0 con 0x7fbd6009d6f0 2023-10-03T10:13:53.879 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:53.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.912+0000 7fbd67c24700 1 -- 172.21.15.3:0/2238173266 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbd60124a90 con 0x7fbd6009d6f0 2023-10-03T10:13:53.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.912+0000 7fbd567fc700 1 -- 172.21.15.3:0/2238173266 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbd5c017300 con 0x7fbd6009d6f0 2023-10-03T10:13:53.913 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:53.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.915+0000 7fbd67c24700 1 -- 172.21.15.3:0/2238173266 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd6009d6f0 msgr2=0x7fbd60122930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:53.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.915+0000 7fbd67c24700 1 --2- 172.21.15.3:0/2238173266 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd6009d6f0 0x7fbd60122930 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c00d5d0 tx=0x7fbd5c004750 comp rx=0 tx=0).stop 2023-10-03T10:13:53.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.915+0000 7fbd67c24700 1 -- 172.21.15.3:0/2238173266 shutdown_connections 2023-10-03T10:13:53.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.915+0000 7fbd67c24700 1 --2- 172.21.15.3:0/2238173266 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd6009e410 0x7fbd601233b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:53.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.915+0000 7fbd67c24700 1 --2- 172.21.15.3:0/2238173266 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd6009d6f0 0x7fbd60122930 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:53.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.915+0000 7fbd67c24700 1 --2- 172.21.15.3:0/2238173266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd6009e000 0x7fbd60122e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:53.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.915+0000 7fbd67c24700 1 -- 172.21.15.3:0/2238173266 >> 172.21.15.3:0/2238173266 conn(0x7fbd60066f50 msgr2=0x7fbd600a1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:53.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.915+0000 7fbd67c24700 1 -- 172.21.15.3:0/2238173266 shutdown_connections 2023-10-03T10:13:53.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:53.915+0000 7fbd67c24700 1 -- 172.21.15.3:0/2238173266 wait complete. 2023-10-03T10:13:53.918 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:53.922 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:53.930 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:54.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.020+0000 7ff90417d700 1 Processor -- start 2023-10-03T10:13:54.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.020+0000 7ff90417d700 1 -- start start 2023-10-03T10:13:54.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.021+0000 7ff90417d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff8fc130540 0x7ff8fc130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:54.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.021+0000 7ff90417d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8fc130e50 0x7ff8fc06f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:54.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.021+0000 7ff90417d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8fc0716a0 con 0x7ff8fc06fb20 2023-10-03T10:13:54.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.021+0000 7ff90417d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8fc09a390 con 0x7ff8fc130540 2023-10-03T10:13:54.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.021+0000 7ff90417d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc071290 con 0x7ff8fc130e50 2023-10-03T10:13:54.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.021+0000 7ff901f19700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff8fc130540 0x7ff8fc130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:54.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.021+0000 7ff901718700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8fc130e50 0x7ff8fc06f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:54.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.021+0000 7ff901f19700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff8fc130540 0x7ff8fc130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51278/0 (socket says 172.21.15.3:51278) 2023-10-03T10:13:54.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.021+0000 7ff901718700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8fc130e50 0x7ff8fc06f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43628/0 (socket says 172.21.15.3:43628) 2023-10-03T10:13:54.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.021+0000 7ff901f19700 1 -- 172.21.15.3:0/1429755438 learned_addr learned my addr 172.21.15.3:0/1429755438 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:54.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.021+0000 7ff901718700 1 -- 172.21.15.3:0/1429755438 >> v2:172.21.15.3:3300/0 conn(0x7ff8fc130540 msgr2=0x7ff8fc130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:54.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.021+0000 7ff901718700 1 --2- 172.21.15.3:0/1429755438 >> v2:172.21.15.3:3300/0 conn(0x7ff8fc130540 0x7ff8fc130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:54.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.021+0000 7ff901718700 1 -- 172.21.15.3:0/1429755438 >> v1:172.21.15.3:6789/0 conn(0x7ff8fc06fb20 legacy=0x7ff8fc137560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:54.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.021+0000 7ff901718700 1 -- 172.21.15.3:0/1429755438 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8fc130340 con 0x7ff8fc130e50 2023-10-03T10:13:54.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.022+0000 7ff901718700 1 --2- 172.21.15.3:0/1429755438 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8fc130e50 0x7ff8fc06f5e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009870 tx=0x7ff8ec009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=856b78970fc8750c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:54.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.022+0000 7ff900f17700 1 -- 172.21.15.3:0/1429755438 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec011830 con 0x7ff8fc130e50 2023-10-03T10:13:54.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.022+0000 7ff900f17700 1 -- 172.21.15.3:0/1429755438 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ec011990 con 0x7ff8fc130e50 2023-10-03T10:13:54.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.022+0000 7ff900f17700 1 -- 172.21.15.3:0/1429755438 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec011d20 con 0x7ff8fc130e50 2023-10-03T10:13:54.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.023+0000 7ff90417d700 1 -- 172.21.15.3:0/1429755438 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8fc130e50 msgr2=0x7ff8fc06f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:54.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.023+0000 7ff90417d700 1 --2- 172.21.15.3:0/1429755438 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8fc130e50 0x7ff8fc06f5e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009870 tx=0x7ff8ec009b80 comp rx=0 tx=0).stop 2023-10-03T10:13:54.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.023+0000 7ff90417d700 1 -- 172.21.15.3:0/1429755438 shutdown_connections 2023-10-03T10:13:54.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.023+0000 7ff90417d700 1 --2- 172.21.15.3:0/1429755438 >> v2:172.21.15.3:3300/0 conn(0x7ff8fc130540 0x7ff8fc130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:54.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.023+0000 7ff90417d700 1 --2- 172.21.15.3:0/1429755438 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8fc130e50 0x7ff8fc06f5e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:54.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.023+0000 7ff90417d700 1 -- 172.21.15.3:0/1429755438 >> 172.21.15.3:0/1429755438 conn(0x7ff8fc0a2aa0 msgr2=0x7ff8fc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:54.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.023+0000 7ff90417d700 1 -- 172.21.15.3:0/1429755438 shutdown_connections 2023-10-03T10:13:54.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.023+0000 7ff90417d700 1 -- 172.21.15.3:0/1429755438 wait complete. 2023-10-03T10:13:54.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.024+0000 7ff90417d700 1 Processor -- start 2023-10-03T10:13:54.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.024+0000 7ff90417d700 1 -- start start 2023-10-03T10:13:54.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.024+0000 7ff90417d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8fc06fb20 0x7ff8fc1c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:54.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.024+0000 7ff90417d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8fc130540 0x7ff8fc1c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:54.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.024+0000 7ff90417d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8fc130e50 0x7ff8fc1c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:54.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.024+0000 7ff90417d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc0718e0 con 0x7ff8fc130e50 2023-10-03T10:13:54.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.024+0000 7ff90417d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc0a0630 con 0x7ff8fc130540 2023-10-03T10:13:54.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.024+0000 7ff90417d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc129400 con 0x7ff8fc06fb20 2023-10-03T10:13:54.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.024+0000 7ff901f19700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8fc06fb20 0x7ff8fc1c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:54.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.024+0000 7ff901f19700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8fc06fb20 0x7ff8fc1c5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43634/0 (socket says 172.21.15.3:43634) 2023-10-03T10:13:54.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.024+0000 7ff901f19700 1 -- 172.21.15.3:0/1243429145 learned_addr learned my addr 172.21.15.3:0/1243429145 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:54.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.024+0000 7ff90271a700 1 --2- 172.21.15.3:0/1243429145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8fc130e50 0x7ff8fc1c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:54.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.025+0000 7ff901718700 1 --2- 172.21.15.3:0/1243429145 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8fc130540 0x7ff8fc1c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:54.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.025+0000 7ff901f19700 1 -- 172.21.15.3:0/1243429145 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8fc130540 msgr2=0x7ff8fc1c6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:54.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.025+0000 7ff901f19700 1 --2- 172.21.15.3:0/1243429145 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8fc130540 0x7ff8fc1c6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:54.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.025+0000 7ff901f19700 1 -- 172.21.15.3:0/1243429145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8fc130e50 msgr2=0x7ff8fc1c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:54.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.025+0000 7ff901f19700 1 --2- 172.21.15.3:0/1243429145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8fc130e50 0x7ff8fc1c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:54.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.025+0000 7ff901f19700 1 -- 172.21.15.3:0/1243429145 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ec0094a0 con 0x7ff8fc06fb20 2023-10-03T10:13:54.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.025+0000 7ff90271a700 1 --2- 172.21.15.3:0/1243429145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8fc130e50 0x7ff8fc1c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:54.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.025+0000 7ff901f19700 1 --2- 172.21.15.3:0/1243429145 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8fc06fb20 0x7ff8fc1c5de0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff8f800e910 tx=0x7ff8f800ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:54.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.025+0000 7ff8f27fc700 1 -- 172.21.15.3:0/1243429145 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f80096a0 con 0x7ff8fc06fb20 2023-10-03T10:13:54.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.025+0000 7ff90417d700 1 -- 172.21.15.3:0/1243429145 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8fc1296a0 con 0x7ff8fc06fb20 2023-10-03T10:13:54.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.025+0000 7ff8f27fc700 1 -- 172.21.15.3:0/1243429145 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f8013680 con 0x7ff8fc06fb20 2023-10-03T10:13:54.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.025+0000 7ff8f27fc700 1 -- 172.21.15.3:0/1243429145 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f8013a30 con 0x7ff8fc06fb20 2023-10-03T10:13:54.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.025+0000 7ff90417d700 1 -- 172.21.15.3:0/1243429145 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8fc12afd0 con 0x7ff8fc06fb20 2023-10-03T10:13:54.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.026+0000 7ff90417d700 1 -- 172.21.15.3:0/1243429145 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8fc074ea0 con 0x7ff8fc06fb20 2023-10-03T10:13:54.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.027+0000 7ff8f27fc700 1 -- 172.21.15.3:0/1243429145 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff8f8013b90 con 0x7ff8fc06fb20 2023-10-03T10:13:54.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.027+0000 7ff8f27fc700 1 -- 172.21.15.3:0/1243429145 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff8f803f710 con 0x7ff8fc06fb20 2023-10-03T10:13:54.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.036+0000 7ff8f27fc700 1 -- 172.21.15.3:0/1243429145 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff8f801bdf0 con 0x7ff8fc06fb20 2023-10-03T10:13:54.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.071+0000 7fe9f7db2700 1 Processor -- start 2023-10-03T10:13:54.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.071+0000 7fe9f7db2700 1 -- start start 2023-10-03T10:13:54.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.071+0000 7fe9f7db2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9f009e930 0x7fe9f009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:54.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.071+0000 7fe9f7db2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe9f009f240 0x7fe9f01317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:54.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.071+0000 7fe9f7db2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9f009f650 0x7fe9f0131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:54.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.071+0000 7fe9f7db2700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9f0070940 con 0x7fe9f009f240 2023-10-03T10:13:54.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.071+0000 7fe9f7db2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f006fd60 con 0x7fe9f009f650 2023-10-03T10:13:54.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.072+0000 7fe9f7db2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f006fa80 con 0x7fe9f009e930 2023-10-03T10:13:54.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.072+0000 7fe9f5b4e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9f009e930 0x7fe9f009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:54.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.072+0000 7fe9f534d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe9f009f240 0x7fe9f01317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:54.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.072+0000 7fe9f5b4e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9f009e930 0x7fe9f009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43646/0 (socket says 172.21.15.3:43646) 2023-10-03T10:13:54.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.072+0000 7fe9f534d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe9f009f240 0x7fe9f01317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51300/0 (socket says 172.21.15.3:51300) 2023-10-03T10:13:54.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.072+0000 7fe9f5b4e700 1 -- 172.21.15.3:0/1778733039 learned_addr learned my addr 172.21.15.3:0/1778733039 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:54.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.072+0000 7fe9f634f700 1 --2- 172.21.15.3:0/1778733039 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9f009f650 0x7fe9f0131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:54.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.072+0000 7fe9f5b4e700 1 -- 172.21.15.3:0/1778733039 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9f009f650 msgr2=0x7fe9f0131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:54.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.072+0000 7fe9f5b4e700 1 --2- 172.21.15.3:0/1778733039 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9f009f650 0x7fe9f0131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:54.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.072+0000 7fe9f5b4e700 1 -- 172.21.15.3:0/1778733039 >> v2:172.21.15.3:3300/0 conn(0x7fe9f009f240 msgr2=0x7fe9f01317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:54.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.072+0000 7fe9f5b4e700 1 --2- 172.21.15.3:0/1778733039 >> v2:172.21.15.3:3300/0 conn(0x7fe9f009f240 0x7fe9f01317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:54.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.072+0000 7fe9f5b4e700 1 -- 172.21.15.3:0/1778733039 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f009fb60 con 0x7fe9f009e930 2023-10-03T10:13:54.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.072+0000 7fe9f634f700 1 --2- 172.21.15.3:0/1778733039 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9f009f650 0x7fe9f0131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:54.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.072+0000 7fe9f534d700 1 --2- 172.21.15.3:0/1778733039 >> v2:172.21.15.3:3300/0 conn(0x7fe9f009f240 0x7fe9f01317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:54.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.072+0000 7fe9f5b4e700 1 --2- 172.21.15.3:0/1778733039 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9f009e930 0x7fe9f009ed00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec0098f0 tx=0x7fe9ec009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=368bf7cde444c6cc server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:54.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.073+0000 7fe9f4b4c700 1 -- 172.21.15.3:0/1778733039 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec011830 con 0x7fe9f009e930 2023-10-03T10:13:54.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.073+0000 7fe9f4b4c700 1 -- 172.21.15.3:0/1778733039 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9ec011990 con 0x7fe9f009e930 2023-10-03T10:13:54.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.073+0000 7fe9f4b4c700 1 -- 172.21.15.3:0/1778733039 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec011d20 con 0x7fe9f009e930 2023-10-03T10:13:54.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.073+0000 7fe9f7db2700 1 -- 172.21.15.3:0/1778733039 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9f009e930 msgr2=0x7fe9f009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:54.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.073+0000 7fe9f7db2700 1 --2- 172.21.15.3:0/1778733039 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9f009e930 0x7fe9f009ed00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec0098f0 tx=0x7fe9ec009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:54.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.073+0000 7fe9f7db2700 1 -- 172.21.15.3:0/1778733039 shutdown_connections 2023-10-03T10:13:54.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.073+0000 7fe9f7db2700 1 --2- 172.21.15.3:0/1778733039 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9f009f650 0x7fe9f0131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:54.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.073+0000 7fe9f7db2700 1 --2- 172.21.15.3:0/1778733039 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9f009e930 0x7fe9f009ed00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.073+0000 7fe9f7db2700 1 --2- 172.21.15.3:0/1778733039 >> v2:172.21.15.3:3300/0 conn(0x7fe9f009f240 0x7fe9f01317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.073+0000 7fe9f7db2700 1 -- 172.21.15.3:0/1778733039 >> 172.21.15.3:0/1778733039 conn(0x7fe9f00a4210 msgr2=0x7fe9f00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.073+0000 7fe9f7db2700 1 -- 172.21.15.3:0/1778733039 shutdown_connections 2023-10-03T10:13:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.074+0000 7fe9f7db2700 1 -- 172.21.15.3:0/1778733039 wait complete. 2023-10-03T10:13:54.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.075+0000 7fe9f7db2700 1 Processor -- start 2023-10-03T10:13:54.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.075+0000 7fe9f7db2700 1 -- start start 2023-10-03T10:13:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.076+0000 7fe9f7db2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9f009e930 0x7fe9f006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.076+0000 7fe9f7db2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9f009f240 0x7fe9f006bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.076+0000 7fe9f7db2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f009f650 0x7fe9f006c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.076+0000 7fe9f7db2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f01a9790 con 0x7fe9f009f650 2023-10-03T10:13:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.076+0000 7fe9f7db2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f01a9a70 con 0x7fe9f009f240 2023-10-03T10:13:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.076+0000 7fe9f7db2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f00a3630 con 0x7fe9f009e930 2023-10-03T10:13:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.076+0000 7fe9f534d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9f009f240 0x7fe9f006bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.076+0000 7fe9f534d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9f009f240 0x7fe9f006bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33176/0 (socket says 172.21.15.3:33176) 2023-10-03T10:13:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.076+0000 7fe9f534d700 1 -- 172.21.15.3:0/764092221 learned_addr learned my addr 172.21.15.3:0/764092221 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.076+0000 7fe9f5b4e700 1 --2- 172.21.15.3:0/764092221 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9f009e930 0x7fe9f006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:54.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.076+0000 7fe9f634f700 1 --2- 172.21.15.3:0/764092221 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f009f650 0x7fe9f006c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:54.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.076+0000 7fe9f534d700 1 -- 172.21.15.3:0/764092221 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9f009e930 msgr2=0x7fe9f006b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:54.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.076+0000 7fe9f534d700 1 --2- 172.21.15.3:0/764092221 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9f009e930 0x7fe9f006b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:54.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.076+0000 7fe9f534d700 1 -- 172.21.15.3:0/764092221 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f009f650 msgr2=0x7fe9f006c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:54.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.076+0000 7fe9f534d700 1 --2- 172.21.15.3:0/764092221 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f009f650 0x7fe9f006c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:54.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.076+0000 7fe9f534d700 1 -- 172.21.15.3:0/764092221 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e00094a0 con 0x7fe9f009f240 2023-10-03T10:13:54.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.076+0000 7fe9f634f700 1 --2- 172.21.15.3:0/764092221 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f009f650 0x7fe9f006c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:54.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.076+0000 7fe9f5b4e700 1 --2- 172.21.15.3:0/764092221 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9f009e930 0x7fe9f006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:54.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.076+0000 7fe9f534d700 1 --2- 172.21.15.3:0/764092221 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9f009f240 0x7fe9f006bdf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000ebb0 tx=0x7fe9e000eec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:54.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.077+0000 7fe9e67fc700 1 -- 172.21.15.3:0/764092221 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e000c850 con 0x7fe9f009f240 2023-10-03T10:13:54.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.077+0000 7fe9e67fc700 1 -- 172.21.15.3:0/764092221 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e000c9b0 con 0x7fe9f009f240 2023-10-03T10:13:54.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.077+0000 7fe9f7db2700 1 -- 172.21.15.3:0/764092221 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e8009a70 con 0x7fe9f009f240 2023-10-03T10:13:54.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.077+0000 7fe9e67fc700 1 -- 172.21.15.3:0/764092221 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e000cd40 con 0x7fe9f009f240 2023-10-03T10:13:54.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.077+0000 7fe9f7db2700 1 -- 172.21.15.3:0/764092221 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9ec009550 con 0x7fe9f009f240 2023-10-03T10:13:54.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.078+0000 7fe9e67fc700 1 -- 172.21.15.3:0/764092221 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe9e0013470 con 0x7fe9f009f240 2023-10-03T10:13:54.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.078+0000 7fe9e67fc700 1 -- 172.21.15.3:0/764092221 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe9e003fc20 con 0x7fe9f009f240 2023-10-03T10:13:54.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.078+0000 7fe9f7db2700 1 -- 172.21.15.3:0/764092221 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f00a3e20 con 0x7fe9f009f240 2023-10-03T10:13:54.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.083+0000 7fe9e67fc700 1 -- 172.21.15.3:0/764092221 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe9e001b3b0 con 0x7fe9f009f240 2023-10-03T10:13:54.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.130+0000 7f0d5e0d3700 1 Processor -- start 2023-10-03T10:13:54.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.130+0000 7f0d5e0d3700 1 -- start start 2023-10-03T10:13:54.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.130+0000 7f0d5e0d3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0d580a0a30 0x7f0d580a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:54.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.130+0000 7f0d5e0d3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0d5809d630 0x7f0d580a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:54.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.130+0000 7f0d5e0d3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d580a2a70 con 0x7f0d58133930 2023-10-03T10:13:54.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.130+0000 7f0d5e0d3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d580a01d0 con 0x7f0d580a0a30 2023-10-03T10:13:54.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.130+0000 7f0d5e0d3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5809e2b0 con 0x7f0d5809d630 2023-10-03T10:13:54.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.131+0000 7f0d56ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0d5809d630 0x7f0d580a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:54.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.131+0000 7f0d577fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0d580a0a30 0x7f0d580a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:54.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.131+0000 7f0d56ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0d5809d630 0x7f0d580a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43660/0 (socket says 172.21.15.3:43660) 2023-10-03T10:13:54.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.131+0000 7f0d56ffd700 1 -- 172.21.15.3:0/2337539961 learned_addr learned my addr 172.21.15.3:0/2337539961 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:54.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.131+0000 7f0d577fe700 1 -- 172.21.15.3:0/2337539961 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0d5809d630 msgr2=0x7f0d580a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:54.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.131+0000 7f0d577fe700 1 --2- 172.21.15.3:0/2337539961 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0d5809d630 0x7f0d580a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:54.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.131+0000 7f0d577fe700 1 -- 172.21.15.3:0/2337539961 >> v1:172.21.15.3:6789/0 conn(0x7f0d58133930 legacy=0x7f0d58133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:54.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.131+0000 7f0d577fe700 1 -- 172.21.15.3:0/2337539961 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d5809dc50 con 0x7f0d580a0a30 2023-10-03T10:13:54.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.131+0000 7f0d577fe700 1 --2- 172.21.15.3:0/2337539961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d580a0a30 0x7f0d580a0e00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0d48009840 tx=0x7f0d48009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3db68a944366f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:54.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.131+0000 7f0d567fc700 1 -- 172.21.15.3:0/2337539961 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d48011830 con 0x7f0d580a0a30 2023-10-03T10:13:54.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.131+0000 7f0d567fc700 1 -- 172.21.15.3:0/2337539961 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d48011990 con 0x7f0d580a0a30 2023-10-03T10:13:54.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.131+0000 7f0d567fc700 1 -- 172.21.15.3:0/2337539961 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d48011d20 con 0x7f0d580a0a30 2023-10-03T10:13:54.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.132+0000 7f0d5e0d3700 1 -- 172.21.15.3:0/2337539961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d580a0a30 msgr2=0x7f0d580a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:54.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.132+0000 7f0d5e0d3700 1 --2- 172.21.15.3:0/2337539961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d580a0a30 0x7f0d580a0e00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0d48009840 tx=0x7f0d48009b50 comp rx=0 tx=0).stop 2023-10-03T10:13:54.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.132+0000 7f0d5e0d3700 1 -- 172.21.15.3:0/2337539961 shutdown_connections 2023-10-03T10:13:54.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.132+0000 7f0d5e0d3700 1 --2- 172.21.15.3:0/2337539961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d580a0a30 0x7f0d580a0e00 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:54.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.132+0000 7f0d5e0d3700 1 --2- 172.21.15.3:0/2337539961 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0d5809d630 0x7f0d580a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:54.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.132+0000 7f0d5e0d3700 1 -- 172.21.15.3:0/2337539961 >> 172.21.15.3:0/2337539961 conn(0x7f0d580a3050 msgr2=0x7f0d580a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:54.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.132+0000 7f0d5e0d3700 1 -- 172.21.15.3:0/2337539961 shutdown_connections 2023-10-03T10:13:54.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.132+0000 7f0d5e0d3700 1 -- 172.21.15.3:0/2337539961 wait complete. 2023-10-03T10:13:54.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.133+0000 7f0d5e0d3700 1 Processor -- start 2023-10-03T10:13:54.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.133+0000 7f0d5e0d3700 1 -- start start 2023-10-03T10:13:54.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.134+0000 7f0d5e0d3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0d5809d630 0x7f0d58132840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:54.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.134+0000 7f0d5e0d3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d580a0a30 0x7f0d58132d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:54.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.134+0000 7f0d5e0d3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0d58133930 0x7f0d581c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:54.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.134+0000 7f0d5e0d3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d58120670 con 0x7f0d580a0a30 2023-10-03T10:13:54.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.134+0000 7f0d5e0d3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d58121fe0 con 0x7f0d58133930 2023-10-03T10:13:54.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.134+0000 7f0d5e0d3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d58122660 con 0x7f0d5809d630 2023-10-03T10:13:54.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.134+0000 7f0d56ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d580a0a30 0x7f0d58132d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:54.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.134+0000 7f0d56ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d580a0a30 0x7f0d58132d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51328/0 (socket says 172.21.15.3:51328) 2023-10-03T10:13:54.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.134+0000 7f0d56ffd700 1 -- 172.21.15.3:0/3906524416 learned_addr learned my addr 172.21.15.3:0/3906524416 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:54.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.134+0000 7f0d577fe700 1 --2- 172.21.15.3:0/3906524416 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0d5809d630 0x7f0d58132840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:54.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.134+0000 7f0d57fff700 1 --2- 172.21.15.3:0/3906524416 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0d58133930 0x7f0d581c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:54.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.134+0000 7f0d56ffd700 1 -- 172.21.15.3:0/3906524416 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0d5809d630 msgr2=0x7f0d58132840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:54.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.134+0000 7f0d56ffd700 1 --2- 172.21.15.3:0/3906524416 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0d5809d630 0x7f0d58132840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:54.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.134+0000 7f0d56ffd700 1 -- 172.21.15.3:0/3906524416 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0d58133930 msgr2=0x7f0d581c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:54.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.134+0000 7f0d56ffd700 1 --2- 172.21.15.3:0/3906524416 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0d58133930 0x7f0d581c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:54.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.134+0000 7f0d56ffd700 1 -- 172.21.15.3:0/3906524416 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d480094a0 con 0x7f0d580a0a30 2023-10-03T10:13:54.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.135+0000 7f0d56ffd700 1 --2- 172.21.15.3:0/3906524416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d580a0a30 0x7f0d58132d80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00e7e0 tx=0x7f0d4c00eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.135+0000 7f0d3ffff700 1 -- 172.21.15.3:0/3906524416 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c00c070 con 0x7f0d580a0a30 2023-10-03T10:13:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.135+0000 7f0d3ffff700 1 -- 172.21.15.3:0/3906524416 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d4c013610 con 0x7f0d580a0a30 2023-10-03T10:13:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.135+0000 7f0d5e0d3700 1 -- 172.21.15.3:0/3906524416 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d580a40c0 con 0x7f0d580a0a30 2023-10-03T10:13:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.135+0000 7f0d3ffff700 1 -- 172.21.15.3:0/3906524416 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c0139c0 con 0x7f0d580a0a30 2023-10-03T10:13:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.135+0000 7f0d5e0d3700 1 -- 172.21.15.3:0/3906524416 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d581332c0 con 0x7f0d580a0a30 2023-10-03T10:13:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.136+0000 7f0d3ffff700 1 -- 172.21.15.3:0/3906524416 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0d4c013b20 con 0x7f0d580a0a30 2023-10-03T10:13:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.136+0000 7f0d5e0d3700 1 -- 172.21.15.3:0/3906524416 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d580747f0 con 0x7f0d580a0a30 2023-10-03T10:13:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.136+0000 7f0d3ffff700 1 -- 172.21.15.3:0/3906524416 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0d4c03f4b0 con 0x7f0d580a0a30 2023-10-03T10:13:54.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.141+0000 7f0d3ffff700 1 -- 172.21.15.3:0/3906524416 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0d4c01c780 con 0x7f0d580a0a30 2023-10-03T10:13:54.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.250+0000 7ff90417d700 1 -- 172.21.15.3:0/1243429145 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff8fc06b9e0 con 0x7ff8fc06fb20 2023-10-03T10:13:54.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.251+0000 7ff8f27fc700 1 -- 172.21.15.3:0/1243429145 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff8f801b3b0 con 0x7ff8fc06fb20 2023-10-03T10:13:54.251 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:54.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.254+0000 7ff90417d700 1 -- 172.21.15.3:0/1243429145 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8fc06fb20 msgr2=0x7ff8fc1c5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:54.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.254+0000 7ff90417d700 1 --2- 172.21.15.3:0/1243429145 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8fc06fb20 0x7ff8fc1c5de0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff8f800e910 tx=0x7ff8f800ec20 comp rx=0 tx=0).stop 2023-10-03T10:13:54.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.254+0000 7ff90417d700 1 -- 172.21.15.3:0/1243429145 shutdown_connections 2023-10-03T10:13:54.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.254+0000 7ff90417d700 1 --2- 172.21.15.3:0/1243429145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8fc130e50 0x7ff8fc1c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:54.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.254+0000 7ff90417d700 1 --2- 172.21.15.3:0/1243429145 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8fc06fb20 0x7ff8fc1c5de0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:54.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.254+0000 7ff90417d700 1 --2- 172.21.15.3:0/1243429145 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8fc130540 0x7ff8fc1c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:54.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.254+0000 7ff90417d700 1 -- 172.21.15.3:0/1243429145 >> 172.21.15.3:0/1243429145 conn(0x7ff8fc0a2aa0 msgr2=0x7ff8fc06e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:54.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.254+0000 7ff90417d700 1 -- 172.21.15.3:0/1243429145 shutdown_connections 2023-10-03T10:13:54.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.254+0000 7ff90417d700 1 -- 172.21.15.3:0/1243429145 wait complete. 2023-10-03T10:13:54.274 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:54.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.304+0000 7fe9f7db2700 1 -- 172.21.15.3:0/764092221 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe9f0074ea0 con 0x7fe9f009f240 2023-10-03T10:13:54.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.305+0000 7fe9e67fc700 1 -- 172.21.15.3:0/764092221 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe9e0017020 con 0x7fe9f009f240 2023-10-03T10:13:54.305 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:54.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.307+0000 7fe9f7db2700 1 -- 172.21.15.3:0/764092221 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9f009f240 msgr2=0x7fe9f006bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:54.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.307+0000 7fe9f7db2700 1 --2- 172.21.15.3:0/764092221 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9f009f240 0x7fe9f006bdf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000ebb0 tx=0x7fe9e000eec0 comp rx=0 tx=0).stop 2023-10-03T10:13:54.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.307+0000 7fe9f7db2700 1 -- 172.21.15.3:0/764092221 shutdown_connections 2023-10-03T10:13:54.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.307+0000 7fe9f7db2700 1 --2- 172.21.15.3:0/764092221 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f009f650 0x7fe9f006c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:54.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.307+0000 7fe9f7db2700 1 --2- 172.21.15.3:0/764092221 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9f009e930 0x7fe9f006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:54.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.307+0000 7fe9f7db2700 1 --2- 172.21.15.3:0/764092221 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9f009f240 0x7fe9f006bdf0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:54.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.307+0000 7fe9f7db2700 1 -- 172.21.15.3:0/764092221 >> 172.21.15.3:0/764092221 conn(0x7fe9f00a4210 msgr2=0x7fe9f01b8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:54.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.307+0000 7fe9f7db2700 1 -- 172.21.15.3:0/764092221 shutdown_connections 2023-10-03T10:13:54.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.307+0000 7fe9f7db2700 1 -- 172.21.15.3:0/764092221 wait complete. 2023-10-03T10:13:54.321 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:54.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.362+0000 7f0d5e0d3700 1 -- 172.21.15.3:0/3906524416 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0d58072980 con 0x7f0d580a0a30 2023-10-03T10:13:54.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.362+0000 7f0d3ffff700 1 -- 172.21.15.3:0/3906524416 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0d4c01c400 con 0x7f0d580a0a30 2023-10-03T10:13:54.363 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:54.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.365+0000 7f0d5e0d3700 1 -- 172.21.15.3:0/3906524416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d580a0a30 msgr2=0x7f0d58132d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:54.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.365+0000 7f0d5e0d3700 1 --2- 172.21.15.3:0/3906524416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d580a0a30 0x7f0d58132d80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00e7e0 tx=0x7f0d4c00eaf0 comp rx=0 tx=0).stop 2023-10-03T10:13:54.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.365+0000 7f0d5e0d3700 1 -- 172.21.15.3:0/3906524416 shutdown_connections 2023-10-03T10:13:54.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.365+0000 7f0d5e0d3700 1 --2- 172.21.15.3:0/3906524416 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0d58133930 0x7f0d581c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:54.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.365+0000 7f0d5e0d3700 1 --2- 172.21.15.3:0/3906524416 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0d5809d630 0x7f0d58132840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:54.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.365+0000 7f0d5e0d3700 1 --2- 172.21.15.3:0/3906524416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d580a0a30 0x7f0d58132d80 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:54.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.365+0000 7f0d5e0d3700 1 -- 172.21.15.3:0/3906524416 >> 172.21.15.3:0/3906524416 conn(0x7f0d580a3050 msgr2=0x7f0d581bbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:54.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.365+0000 7f0d5e0d3700 1 -- 172.21.15.3:0/3906524416 shutdown_connections 2023-10-03T10:13:54.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:54.365+0000 7f0d5e0d3700 1 -- 172.21.15.3:0/3906524416 wait complete. 2023-10-03T10:13:54.379 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:54.930 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:55.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.077+0000 7f9d916bc700 1 Processor -- start 2023-10-03T10:13:55.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.077+0000 7f9d916bc700 1 -- start start 2023-10-03T10:13:55.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.077+0000 7f9d916bc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d8c09f330 0x7f9d8c138f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:55.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.077+0000 7f9d916bc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9d8c139450 0x7f9d8c13b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:55.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.077+0000 7f9d916bc700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d8c06bec0 con 0x7f9d8c09e850 2023-10-03T10:13:55.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.077+0000 7f9d916bc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d8c0a0030 con 0x7f9d8c139450 2023-10-03T10:13:55.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.077+0000 7f9d916bc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c0a0830 con 0x7f9d8c09f330 2023-10-03T10:13:55.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.077+0000 7f9d8b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9d8c139450 0x7f9d8c13b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:55.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.077+0000 7f9d8a7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d8c09f330 0x7f9d8c138f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:55.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.077+0000 7f9d8b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9d8c139450 0x7f9d8c13b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51336/0 (socket says 172.21.15.3:51336) 2023-10-03T10:13:55.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.077+0000 7f9d8a7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d8c09f330 0x7f9d8c138f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43678/0 (socket says 172.21.15.3:43678) 2023-10-03T10:13:55.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.077+0000 7f9d8b7fe700 1 -- 172.21.15.3:0/3625825368 learned_addr learned my addr 172.21.15.3:0/3625825368 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:55.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.078+0000 7f9d8b7fe700 1 -- 172.21.15.3:0/3625825368 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d8c09f330 msgr2=0x7f9d8c138f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:55.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.078+0000 7f9d8b7fe700 1 --2- 172.21.15.3:0/3625825368 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d8c09f330 0x7f9d8c138f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.078+0000 7f9d8b7fe700 1 -- 172.21.15.3:0/3625825368 >> v1:172.21.15.3:6789/0 conn(0x7f9d8c09e850 legacy=0x7f9d8c09ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:55.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.078+0000 7f9d8b7fe700 1 -- 172.21.15.3:0/3625825368 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8c06caa0 con 0x7f9d8c139450 2023-10-03T10:13:55.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.078+0000 7f9d8a7fc700 1 --2- 172.21.15.3:0/3625825368 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d8c09f330 0x7f9d8c138f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:55.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.078+0000 7f9d8b7fe700 1 --2- 172.21.15.3:0/3625825368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d8c139450 0x7f9d8c13b830 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9d84009e50 tx=0x7f9d8400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bd3c46508b72473 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:55.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.078+0000 7f9d89ffb700 1 -- 172.21.15.3:0/3625825368 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8400e070 con 0x7f9d8c139450 2023-10-03T10:13:55.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.078+0000 7f9d89ffb700 1 -- 172.21.15.3:0/3625825368 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8400b910 con 0x7f9d8c139450 2023-10-03T10:13:55.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.078+0000 7f9d89ffb700 1 -- 172.21.15.3:0/3625825368 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8400bcc0 con 0x7f9d8c139450 2023-10-03T10:13:55.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.079+0000 7f9d916bc700 1 -- 172.21.15.3:0/3625825368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d8c139450 msgr2=0x7f9d8c13b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:55.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.079+0000 7f9d916bc700 1 --2- 172.21.15.3:0/3625825368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d8c139450 0x7f9d8c13b830 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9d84009e50 tx=0x7f9d8400b330 comp rx=0 tx=0).stop 2023-10-03T10:13:55.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.079+0000 7f9d916bc700 1 -- 172.21.15.3:0/3625825368 shutdown_connections 2023-10-03T10:13:55.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.079+0000 7f9d916bc700 1 --2- 172.21.15.3:0/3625825368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d8c139450 0x7f9d8c13b830 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.079+0000 7f9d916bc700 1 --2- 172.21.15.3:0/3625825368 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d8c09f330 0x7f9d8c138f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.079+0000 7f9d916bc700 1 -- 172.21.15.3:0/3625825368 >> 172.21.15.3:0/3625825368 conn(0x7f9d8c0a3f70 msgr2=0x7f9d8c0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:55.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.079+0000 7f9d916bc700 1 -- 172.21.15.3:0/3625825368 shutdown_connections 2023-10-03T10:13:55.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.079+0000 7f9d916bc700 1 -- 172.21.15.3:0/3625825368 wait complete. 2023-10-03T10:13:55.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.080+0000 7f9d916bc700 1 Processor -- start 2023-10-03T10:13:55.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.080+0000 7f9d916bc700 1 -- start start 2023-10-03T10:13:55.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.080+0000 7f9d916bc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d8c09e850 0x7f9d8c1c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:55.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.080+0000 7f9d916bc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d8c09f330 0x7f9d8c1c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:55.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.080+0000 7f9d916bc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d8c139450 0x7f9d8c11ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:55.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.080+0000 7f9d916bc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c120d60 con 0x7f9d8c09e850 2023-10-03T10:13:55.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.080+0000 7f9d916bc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c1be5d0 con 0x7f9d8c09f330 2023-10-03T10:13:55.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.080+0000 7f9d916bc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c121e00 con 0x7f9d8c139450 2023-10-03T10:13:55.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.081+0000 7f9d8b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d8c139450 0x7f9d8c11ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:55.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.081+0000 7f9d8affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d8c09e850 0x7f9d8c1c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:55.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.081+0000 7f9d8a7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d8c09f330 0x7f9d8c1c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:55.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.081+0000 7f9d8b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d8c139450 0x7f9d8c11ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43692/0 (socket says 172.21.15.3:43692) 2023-10-03T10:13:55.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.081+0000 7f9d8affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d8c09e850 0x7f9d8c1c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51346/0 (socket says 172.21.15.3:51346) 2023-10-03T10:13:55.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.081+0000 7f9d8a7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d8c09f330 0x7f9d8c1c9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33204/0 (socket says 172.21.15.3:33204) 2023-10-03T10:13:55.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.081+0000 7f9d8b7fe700 1 -- 172.21.15.3:0/737233399 learned_addr learned my addr 172.21.15.3:0/737233399 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:55.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.081+0000 7f9d8b7fe700 1 -- 172.21.15.3:0/737233399 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d8c09f330 msgr2=0x7f9d8c1c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:55.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.081+0000 7f9d8b7fe700 1 --2- 172.21.15.3:0/737233399 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d8c09f330 0x7f9d8c1c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.081+0000 7f9d8b7fe700 1 -- 172.21.15.3:0/737233399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d8c09e850 msgr2=0x7f9d8c1c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:55.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.081+0000 7f9d8b7fe700 1 --2- 172.21.15.3:0/737233399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d8c09e850 0x7f9d8c1c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.081+0000 7f9d8b7fe700 1 -- 172.21.15.3:0/737233399 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d800094a0 con 0x7f9d8c139450 2023-10-03T10:13:55.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.081+0000 7f9d8b7fe700 1 --2- 172.21.15.3:0/737233399 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d8c139450 0x7f9d8c11ea30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400b680 tx=0x7f9d84004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:55.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.081+0000 7f9d737fe700 1 -- 172.21.15.3:0/737233399 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d84004830 con 0x7f9d8c139450 2023-10-03T10:13:55.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.081+0000 7f9d737fe700 1 -- 172.21.15.3:0/737233399 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d840099b0 con 0x7f9d8c139450 2023-10-03T10:13:55.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.081+0000 7f9d916bc700 1 -- 172.21.15.3:0/737233399 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d84009b30 con 0x7f9d8c139450 2023-10-03T10:13:55.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.081+0000 7f9d737fe700 1 -- 172.21.15.3:0/737233399 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8401c680 con 0x7f9d8c139450 2023-10-03T10:13:55.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.081+0000 7f9d916bc700 1 -- 172.21.15.3:0/737233399 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d8c122b30 con 0x7f9d8c139450 2023-10-03T10:13:55.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.082+0000 7f9d737fe700 1 -- 172.21.15.3:0/737233399 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9d8401c7e0 con 0x7f9d8c139450 2023-10-03T10:13:55.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.082+0000 7f9d737fe700 1 -- 172.21.15.3:0/737233399 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9d840411d0 con 0x7f9d8c139450 2023-10-03T10:13:55.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.082+0000 7f9d916bc700 1 -- 172.21.15.3:0/737233399 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d8c0747f0 con 0x7f9d8c139450 2023-10-03T10:13:55.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.087+0000 7f9d737fe700 1 -- 172.21.15.3:0/737233399 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9d840257d0 con 0x7f9d8c139450 2023-10-03T10:13:55.275 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:55.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.292+0000 7f9d916bc700 1 -- 172.21.15.3:0/737233399 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9d8c134970 con 0x7f9d8c139450 2023-10-03T10:13:55.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.292+0000 7f9d737fe700 1 -- 172.21.15.3:0/737233399 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9d8402d960 con 0x7f9d8c139450 2023-10-03T10:13:55.293 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:55.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.294+0000 7f9d916bc700 1 -- 172.21.15.3:0/737233399 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d8c139450 msgr2=0x7f9d8c11ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:55.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.294+0000 7f9d916bc700 1 --2- 172.21.15.3:0/737233399 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d8c139450 0x7f9d8c11ea30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400b680 tx=0x7f9d84004ad0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.295+0000 7f9d916bc700 1 -- 172.21.15.3:0/737233399 shutdown_connections 2023-10-03T10:13:55.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.295+0000 7f9d916bc700 1 --2- 172.21.15.3:0/737233399 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d8c139450 0x7f9d8c11ea30 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.295+0000 7f9d916bc700 1 --2- 172.21.15.3:0/737233399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d8c09e850 0x7f9d8c1c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.295+0000 7f9d916bc700 1 --2- 172.21.15.3:0/737233399 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d8c09f330 0x7f9d8c1c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.295+0000 7f9d916bc700 1 -- 172.21.15.3:0/737233399 >> 172.21.15.3:0/737233399 conn(0x7f9d8c0a3f70 msgr2=0x7f9d8c136f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:55.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.295+0000 7f9d916bc700 1 -- 172.21.15.3:0/737233399 shutdown_connections 2023-10-03T10:13:55.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.295+0000 7f9d916bc700 1 -- 172.21.15.3:0/737233399 wait complete. 2023-10-03T10:13:55.310 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:55.322 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:55.379 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:55.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.415+0000 7f38e97ed700 1 Processor -- start 2023-10-03T10:13:55.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.415+0000 7f38e97ed700 1 -- start start 2023-10-03T10:13:55.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.416+0000 7f38e97ed700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38e409d6f0 0x7f38e409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:55.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.416+0000 7f38e97ed700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38e409e000 0x7f38e41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:55.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.416+0000 7f38e97ed700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e409b120 con 0x7f38e409e410 2023-10-03T10:13:55.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.416+0000 7f38e97ed700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e406b460 con 0x7f38e409d6f0 2023-10-03T10:13:55.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.416+0000 7f38e97ed700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38e409a7e0 con 0x7f38e409e000 2023-10-03T10:13:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.416+0000 7f38e2ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38e409d6f0 0x7f38e409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.416+0000 7f38e27fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38e409e000 0x7f38e41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.416+0000 7f38e2ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38e409d6f0 0x7f38e409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33206/0 (socket says 172.21.15.3:33206) 2023-10-03T10:13:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.416+0000 7f38e2ffd700 1 -- 172.21.15.3:0/592768357 learned_addr learned my addr 172.21.15.3:0/592768357 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.416+0000 7f38e1ffb700 1 -- 172.21.15.3:0/592768357 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 825493345 0 0) 0x7f38e409b120 con 0x7f38e409e410 2023-10-03T10:13:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.416+0000 7f38e27fc700 1 -- 172.21.15.3:0/592768357 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38e409d6f0 msgr2=0x7f38e409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.416+0000 7f38e27fc700 1 --2- 172.21.15.3:0/592768357 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38e409d6f0 0x7f38e409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.416+0000 7f38e27fc700 1 -- 172.21.15.3:0/592768357 >> v1:172.21.15.3:6789/0 conn(0x7f38e409e410 legacy=0x7f38e4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.416+0000 7f38e27fc700 1 -- 172.21.15.3:0/592768357 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e409a500 con 0x7f38e409e000 2023-10-03T10:13:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.416+0000 7f38e2ffd700 1 --2- 172.21.15.3:0/592768357 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38e409d6f0 0x7f38e409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.417+0000 7f38e27fc700 1 --2- 172.21.15.3:0/592768357 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38e409e000 0x7f38e41339f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f38d80098a0 tx=0x7f38d8009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d75611c3c9fbdc07 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.417+0000 7f38e1ffb700 1 -- 172.21.15.3:0/592768357 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8011830 con 0x7f38e409e000 2023-10-03T10:13:55.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.417+0000 7f38e1ffb700 1 -- 172.21.15.3:0/592768357 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d8011990 con 0x7f38e409e000 2023-10-03T10:13:55.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.417+0000 7f38e1ffb700 1 -- 172.21.15.3:0/592768357 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8011d20 con 0x7f38e409e000 2023-10-03T10:13:55.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.418+0000 7f38e97ed700 1 -- 172.21.15.3:0/592768357 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38e409e000 msgr2=0x7f38e41339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:55.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.418+0000 7f38e97ed700 1 --2- 172.21.15.3:0/592768357 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38e409e000 0x7f38e41339f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f38d80098a0 tx=0x7f38d8009bb0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.418+0000 7f38e97ed700 1 -- 172.21.15.3:0/592768357 shutdown_connections 2023-10-03T10:13:55.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.418+0000 7f38e97ed700 1 --2- 172.21.15.3:0/592768357 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38e409d6f0 0x7f38e409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.418+0000 7f38e97ed700 1 --2- 172.21.15.3:0/592768357 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38e409e000 0x7f38e41339f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.418+0000 7f38e97ed700 1 -- 172.21.15.3:0/592768357 >> 172.21.15.3:0/592768357 conn(0x7f38e4066f50 msgr2=0x7f38e409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:55.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.418+0000 7f38e97ed700 1 -- 172.21.15.3:0/592768357 shutdown_connections 2023-10-03T10:13:55.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.418+0000 7f38e97ed700 1 -- 172.21.15.3:0/592768357 wait complete. 2023-10-03T10:13:55.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.420+0000 7f38e97ed700 1 Processor -- start 2023-10-03T10:13:55.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.420+0000 7f38e97ed700 1 -- start start 2023-10-03T10:13:55.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.420+0000 7f38e97ed700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38e409d6f0 0x7f38e41a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:55.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.420+0000 7f38e97ed700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38e409e000 0x7f38e41a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:55.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.420+0000 7f38e97ed700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38e409e410 0x7f38e41c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:55.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.420+0000 7f38e97ed700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e41b91f0 con 0x7f38e409e000 2023-10-03T10:13:55.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.420+0000 7f38e97ed700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e41a4d40 con 0x7f38e409e410 2023-10-03T10:13:55.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.420+0000 7f38e97ed700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38e41a6470 con 0x7f38e409d6f0 2023-10-03T10:13:55.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.421+0000 7f38e2ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38e409d6f0 0x7f38e41a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:55.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.421+0000 7f38e2ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38e409d6f0 0x7f38e41a5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43700/0 (socket says 172.21.15.3:43700) 2023-10-03T10:13:55.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.421+0000 7f38e2ffd700 1 -- 172.21.15.3:0/1288444811 learned_addr learned my addr 172.21.15.3:0/1288444811 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:55.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.421+0000 7f38e37fe700 1 --2- 172.21.15.3:0/1288444811 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38e409e410 0x7f38e41c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:55.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.421+0000 7f38e27fc700 1 --2- 172.21.15.3:0/1288444811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38e409e000 0x7f38e41a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:55.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.421+0000 7f38e2ffd700 1 -- 172.21.15.3:0/1288444811 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38e409e410 msgr2=0x7f38e41c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:55.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.421+0000 7f38e2ffd700 1 --2- 172.21.15.3:0/1288444811 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38e409e410 0x7f38e41c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.421+0000 7f38e2ffd700 1 -- 172.21.15.3:0/1288444811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38e409e000 msgr2=0x7f38e41a5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:55.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.421+0000 7f38e2ffd700 1 --2- 172.21.15.3:0/1288444811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38e409e000 0x7f38e41a5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.421+0000 7f38e2ffd700 1 -- 172.21.15.3:0/1288444811 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d4009550 con 0x7f38e409d6f0 2023-10-03T10:13:55.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.421+0000 7f38e27fc700 1 --2- 172.21.15.3:0/1288444811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38e409e000 0x7f38e41a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:55.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.421+0000 7f38e37fe700 1 --2- 172.21.15.3:0/1288444811 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38e409e410 0x7f38e41c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:55.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.421+0000 7f38e2ffd700 1 --2- 172.21.15.3:0/1288444811 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38e409d6f0 0x7f38e41a5130 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f38d400ebe0 tx=0x7f38d400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:55.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.422+0000 7f38cb7fe700 1 -- 172.21.15.3:0/1288444811 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d400c850 con 0x7f38e409d6f0 2023-10-03T10:13:55.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.422+0000 7f38cb7fe700 1 -- 172.21.15.3:0/1288444811 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d400c9b0 con 0x7f38e409d6f0 2023-10-03T10:13:55.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.422+0000 7f38e97ed700 1 -- 172.21.15.3:0/1288444811 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d80094a0 con 0x7f38e409d6f0 2023-10-03T10:13:55.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.422+0000 7f38e97ed700 1 -- 172.21.15.3:0/1288444811 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e409c710 con 0x7f38e409d6f0 2023-10-03T10:13:55.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.422+0000 7f38cb7fe700 1 -- 172.21.15.3:0/1288444811 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d400cd40 con 0x7f38e409d6f0 2023-10-03T10:13:55.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.423+0000 7f38cb7fe700 1 -- 172.21.15.3:0/1288444811 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f38d4013470 con 0x7f38e409d6f0 2023-10-03T10:13:55.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.423+0000 7f38e97ed700 1 -- 172.21.15.3:0/1288444811 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38c40052e0 con 0x7f38e409d6f0 2023-10-03T10:13:55.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.423+0000 7f38cb7fe700 1 -- 172.21.15.3:0/1288444811 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f38d403f8c0 con 0x7f38e409d6f0 2023-10-03T10:13:55.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.428+0000 7f38cb7fe700 1 -- 172.21.15.3:0/1288444811 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f38d40232f0 con 0x7f38e409d6f0 2023-10-03T10:13:55.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.495+0000 7fc6f2c8f700 1 Processor -- start 2023-10-03T10:13:55.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.495+0000 7fc6f2c8f700 1 -- start start 2023-10-03T10:13:55.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.495+0000 7fc6f2c8f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc6ec09d6f0 0x7fc6ec09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:55.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.495+0000 7fc6f2c8f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6ec09e440 0x7fc6ec134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:55.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.495+0000 7fc6f2c8f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6ec09b120 con 0x7fc6ec09e000 2023-10-03T10:13:55.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.495+0000 7fc6f2c8f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6ec06b460 con 0x7fc6ec09d6f0 2023-10-03T10:13:55.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.495+0000 7fc6f2c8f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec09a7e0 con 0x7fc6ec09e440 2023-10-03T10:13:55.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.496+0000 7fc6f0a2b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc6ec09d6f0 0x7fc6ec09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:55.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.496+0000 7fc6f0a2b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc6ec09d6f0 0x7fc6ec09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51368/0 (socket says 172.21.15.3:51368) 2023-10-03T10:13:55.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.496+0000 7fc6f122c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6ec09e440 0x7fc6ec134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:55.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.496+0000 7fc6f0a2b700 1 -- 172.21.15.3:0/2608428664 learned_addr learned my addr 172.21.15.3:0/2608428664 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:55.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.496+0000 7fc6f0a2b700 1 -- 172.21.15.3:0/2608428664 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6ec09e440 msgr2=0x7fc6ec134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:55.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.496+0000 7fc6f0a2b700 1 --2- 172.21.15.3:0/2608428664 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6ec09e440 0x7fc6ec134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.496+0000 7fc6f0a2b700 1 -- 172.21.15.3:0/2608428664 >> v1:172.21.15.3:6789/0 conn(0x7fc6ec09e000 legacy=0x7fc6ec1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:55.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.496+0000 7fc6f0a2b700 1 -- 172.21.15.3:0/2608428664 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ec09a500 con 0x7fc6ec09d6f0 2023-10-03T10:13:55.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.496+0000 7fc6f0a2b700 1 --2- 172.21.15.3:0/2608428664 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6ec09d6f0 0x7fc6ec09dac0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009920 tx=0x7fc6e0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a325aed7715f43af server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:55.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.496+0000 7fc6eb7fe700 1 -- 172.21.15.3:0/2608428664 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0011830 con 0x7fc6ec09d6f0 2023-10-03T10:13:55.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.496+0000 7fc6eb7fe700 1 -- 172.21.15.3:0/2608428664 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e0011990 con 0x7fc6ec09d6f0 2023-10-03T10:13:55.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.496+0000 7fc6eb7fe700 1 -- 172.21.15.3:0/2608428664 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0011d20 con 0x7fc6ec09d6f0 2023-10-03T10:13:55.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.497+0000 7fc6f2c8f700 1 -- 172.21.15.3:0/2608428664 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6ec09d6f0 msgr2=0x7fc6ec09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:55.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.497+0000 7fc6f2c8f700 1 --2- 172.21.15.3:0/2608428664 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6ec09d6f0 0x7fc6ec09dac0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009920 tx=0x7fc6e0009c30 comp rx=0 tx=0).stop 2023-10-03T10:13:55.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.497+0000 7fc6f2c8f700 1 -- 172.21.15.3:0/2608428664 shutdown_connections 2023-10-03T10:13:55.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.497+0000 7fc6f2c8f700 1 --2- 172.21.15.3:0/2608428664 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6ec09d6f0 0x7fc6ec09dac0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.497+0000 7fc6f2c8f700 1 --2- 172.21.15.3:0/2608428664 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6ec09e440 0x7fc6ec134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.497+0000 7fc6f2c8f700 1 -- 172.21.15.3:0/2608428664 >> 172.21.15.3:0/2608428664 conn(0x7fc6ec066f50 msgr2=0x7fc6ec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:55.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.497+0000 7fc6f2c8f700 1 -- 172.21.15.3:0/2608428664 shutdown_connections 2023-10-03T10:13:55.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.497+0000 7fc6f2c8f700 1 -- 172.21.15.3:0/2608428664 wait complete. 2023-10-03T10:13:55.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.498+0000 7fc6f2c8f700 1 Processor -- start 2023-10-03T10:13:55.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.498+0000 7fc6f2c8f700 1 -- start start 2023-10-03T10:13:55.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.499+0000 7fc6f2c8f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6ec09d6f0 0x7fc6ec1a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:55.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.499+0000 7fc6f2c8f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6ec09e000 0x7fc6ec1a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:55.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.499+0000 7fc6f2c8f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6ec09e440 0x7fc6ec1a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:55.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.499+0000 7fc6f2c8f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec1a8d00 con 0x7fc6ec09e440 2023-10-03T10:13:55.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.499+0000 7fc6f2c8f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec1a72e0 con 0x7fc6ec09d6f0 2023-10-03T10:13:55.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.499+0000 7fc6f2c8f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec1a9750 con 0x7fc6ec09e000 2023-10-03T10:13:55.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.499+0000 7fc6ebfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6ec09e000 0x7fc6ec1a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:55.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.499+0000 7fc6ebfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6ec09e000 0x7fc6ec1a6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43712/0 (socket says 172.21.15.3:43712) 2023-10-03T10:13:55.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.499+0000 7fc6ebfff700 1 -- 172.21.15.3:0/1134788017 learned_addr learned my addr 172.21.15.3:0/1134788017 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:55.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.499+0000 7fc6f122c700 1 --2- 172.21.15.3:0/1134788017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6ec09e440 0x7fc6ec1a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:55.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.499+0000 7fc6f0a2b700 1 --2- 172.21.15.3:0/1134788017 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6ec09d6f0 0x7fc6ec1a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:55.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.500+0000 7fc6f0a2b700 1 -- 172.21.15.3:0/1134788017 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6ec09e000 msgr2=0x7fc6ec1a6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:55.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.500+0000 7fc6f0a2b700 1 --2- 172.21.15.3:0/1134788017 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6ec09e000 0x7fc6ec1a6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.500+0000 7fc6f0a2b700 1 -- 172.21.15.3:0/1134788017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6ec09e440 msgr2=0x7fc6ec1a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:55.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.500+0000 7fc6f0a2b700 1 --2- 172.21.15.3:0/1134788017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6ec09e440 0x7fc6ec1a68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.500+0000 7fc6f0a2b700 1 -- 172.21.15.3:0/1134788017 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e0009550 con 0x7fc6ec09d6f0 2023-10-03T10:13:55.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.500+0000 7fc6f122c700 1 --2- 172.21.15.3:0/1134788017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6ec09e440 0x7fc6ec1a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:55.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.500+0000 7fc6f0a2b700 1 --2- 172.21.15.3:0/1134788017 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6ec09d6f0 0x7fc6ec1a5e30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0016000 tx=0x7fc6e0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:55.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.500+0000 7fc6e97fa700 1 -- 172.21.15.3:0/1134788017 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0004300 con 0x7fc6ec09d6f0 2023-10-03T10:13:55.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.500+0000 7fc6e97fa700 1 -- 172.21.15.3:0/1134788017 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e001f070 con 0x7fc6ec09d6f0 2023-10-03T10:13:55.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.500+0000 7fc6e97fa700 1 -- 172.21.15.3:0/1134788017 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e00036a0 con 0x7fc6ec09d6f0 2023-10-03T10:13:55.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.500+0000 7fc6f2c8f700 1 -- 172.21.15.3:0/1134788017 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6ec1a8840 con 0x7fc6ec09d6f0 2023-10-03T10:13:55.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.500+0000 7fc6f2c8f700 1 -- 172.21.15.3:0/1134788017 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6ec0a2200 con 0x7fc6ec09d6f0 2023-10-03T10:13:55.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.501+0000 7fc6f2c8f700 1 -- 172.21.15.3:0/1134788017 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6ec09c770 con 0x7fc6ec09d6f0 2023-10-03T10:13:55.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.501+0000 7fc6e97fa700 1 -- 172.21.15.3:0/1134788017 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc6e0021030 con 0x7fc6ec09d6f0 2023-10-03T10:13:55.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.501+0000 7fc6e97fa700 1 -- 172.21.15.3:0/1134788017 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc6e00407b0 con 0x7fc6ec09d6f0 2023-10-03T10:13:55.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.506+0000 7fc6e97fa700 1 -- 172.21.15.3:0/1134788017 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc6e00239c0 con 0x7fc6ec09d6f0 2023-10-03T10:13:55.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.536+0000 7f8a86a70700 1 Processor -- start 2023-10-03T10:13:55.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.536+0000 7f8a86a70700 1 -- start start 2023-10-03T10:13:55.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.537+0000 7f8a86a70700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8a800a0a30 0x7f8a800a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:55.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.537+0000 7f8a86a70700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a80134170 0x7f8a8013bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:55.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.537+0000 7f8a86a70700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a8009edd0 con 0x7f8a800a1340 2023-10-03T10:13:55.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.537+0000 7f8a86a70700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a8009e1f0 con 0x7f8a800a0a30 2023-10-03T10:13:55.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.537+0000 7f8a86a70700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8009dc50 con 0x7f8a80134170 2023-10-03T10:13:55.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.537+0000 7f8a8500d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a80134170 0x7f8a8013bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:55.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.537+0000 7f8a7ffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8a800a1340 0x7f8a80133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45202/0 (socket says 172.21.15.3:45202) 2023-10-03T10:13:55.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.537+0000 7f8a8480c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8a800a0a30 0x7f8a800a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:55.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.537+0000 7f8a7ffff700 1 -- 172.21.15.3:0/3971508397 learned_addr learned my addr 172.21.15.3:0/3971508397 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:55.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.537+0000 7f8a8480c700 1 -- 172.21.15.3:0/3971508397 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a80134170 msgr2=0x7f8a8013bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:55.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.537+0000 7f8a8480c700 1 --2- 172.21.15.3:0/3971508397 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a80134170 0x7f8a8013bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.537+0000 7f8a8480c700 1 -- 172.21.15.3:0/3971508397 >> v1:172.21.15.3:6789/0 conn(0x7f8a800a1340 legacy=0x7f8a80133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:55.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.537+0000 7f8a8480c700 1 -- 172.21.15.3:0/3971508397 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a800a1fe0 con 0x7f8a800a0a30 2023-10-03T10:13:55.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.538+0000 7f8a8480c700 1 --2- 172.21.15.3:0/3971508397 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a800a0a30 0x7f8a800a0e00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009840 tx=0x7f8a74009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2974134ae16da280 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:55.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.538+0000 7f8a7f7fe700 1 -- 172.21.15.3:0/3971508397 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a74011830 con 0x7f8a800a0a30 2023-10-03T10:13:55.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.538+0000 7f8a7f7fe700 1 -- 172.21.15.3:0/3971508397 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a74011990 con 0x7f8a800a0a30 2023-10-03T10:13:55.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.538+0000 7f8a7f7fe700 1 -- 172.21.15.3:0/3971508397 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a74011d20 con 0x7f8a800a0a30 2023-10-03T10:13:55.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.538+0000 7f8a86a70700 1 -- 172.21.15.3:0/3971508397 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a800a0a30 msgr2=0x7f8a800a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:55.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.538+0000 7f8a86a70700 1 --2- 172.21.15.3:0/3971508397 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a800a0a30 0x7f8a800a0e00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009840 tx=0x7f8a74009b50 comp rx=0 tx=0).stop 2023-10-03T10:13:55.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.538+0000 7f8a86a70700 1 -- 172.21.15.3:0/3971508397 shutdown_connections 2023-10-03T10:13:55.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.538+0000 7f8a86a70700 1 --2- 172.21.15.3:0/3971508397 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a80134170 0x7f8a8013bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.538+0000 7f8a86a70700 1 --2- 172.21.15.3:0/3971508397 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a800a0a30 0x7f8a800a0e00 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.538+0000 7f8a86a70700 1 -- 172.21.15.3:0/3971508397 >> 172.21.15.3:0/3971508397 conn(0x7f8a800a6550 msgr2=0x7f8a800a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:55.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.539+0000 7f8a86a70700 1 -- 172.21.15.3:0/3971508397 shutdown_connections 2023-10-03T10:13:55.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.539+0000 7f8a86a70700 1 -- 172.21.15.3:0/3971508397 wait complete. 2023-10-03T10:13:55.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.540+0000 7f8a86a70700 1 Processor -- start 2023-10-03T10:13:55.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.540+0000 7f8a86a70700 1 -- start start 2023-10-03T10:13:55.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.541+0000 7f8a86a70700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a800a0a30 0x7f8a801c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:55.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.541+0000 7f8a86a70700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a800a1340 0x7f8a801c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:55.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.541+0000 7f8a86a70700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a80134170 0x7f8a801227d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:55.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.541+0000 7f8a86a70700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a801263f0 con 0x7f8a80134170 2023-10-03T10:13:55.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.541+0000 7f8a86a70700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a801243e0 con 0x7f8a800a1340 2023-10-03T10:13:55.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.541+0000 7f8a86a70700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a800a4ec0 con 0x7f8a800a0a30 2023-10-03T10:13:55.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.541+0000 7f8a7ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a800a1340 0x7f8a801c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:55.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.541+0000 7f8a8480c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a800a0a30 0x7f8a801c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:55.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.541+0000 7f8a7ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a800a1340 0x7f8a801c8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33252/0 (socket says 172.21.15.3:33252) 2023-10-03T10:13:55.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.541+0000 7f8a7ffff700 1 -- 172.21.15.3:0/3395998651 learned_addr learned my addr 172.21.15.3:0/3395998651 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:55.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.541+0000 7f8a8480c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a800a0a30 0x7f8a801c8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43716/0 (socket says 172.21.15.3:43716) 2023-10-03T10:13:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.541+0000 7f8a8500d700 1 --2- 172.21.15.3:0/3395998651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a80134170 0x7f8a801227d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.542+0000 7f8a7ffff700 1 -- 172.21.15.3:0/3395998651 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a800a0a30 msgr2=0x7f8a801c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.542+0000 7f8a7ffff700 1 --2- 172.21.15.3:0/3395998651 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a800a0a30 0x7f8a801c8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.542+0000 7f8a7ffff700 1 -- 172.21.15.3:0/3395998651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a80134170 msgr2=0x7f8a801227d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.542+0000 7f8a7ffff700 1 --2- 172.21.15.3:0/3395998651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a80134170 0x7f8a801227d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.542+0000 7f8a7ffff700 1 -- 172.21.15.3:0/3395998651 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a740094a0 con 0x7f8a800a1340 2023-10-03T10:13:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.542+0000 7f8a7ffff700 1 --2- 172.21.15.3:0/3395998651 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a800a1340 0x7f8a801c8d80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c00c520 tx=0x7f8a6c00c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.542+0000 7f8a7d7fa700 1 -- 172.21.15.3:0/3395998651 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c005d70 con 0x7f8a800a1340 2023-10-03T10:13:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.542+0000 7f8a86a70700 1 -- 172.21.15.3:0/3395998651 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a800a25e0 con 0x7f8a800a1340 2023-10-03T10:13:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.542+0000 7f8a7d7fa700 1 -- 172.21.15.3:0/3395998651 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6c0070d0 con 0x7f8a800a1340 2023-10-03T10:13:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.542+0000 7f8a7d7fa700 1 -- 172.21.15.3:0/3395998651 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c007440 con 0x7f8a800a1340 2023-10-03T10:13:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.542+0000 7f8a86a70700 1 -- 172.21.15.3:0/3395998651 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a80125d50 con 0x7f8a800a1340 2023-10-03T10:13:55.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.543+0000 7f8a7d7fa700 1 -- 172.21.15.3:0/3395998651 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8a6c0075a0 con 0x7f8a800a1340 2023-10-03T10:13:55.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.543+0000 7f8a7d7fa700 1 -- 172.21.15.3:0/3395998651 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8a6c03ea60 con 0x7f8a800a1340 2023-10-03T10:13:55.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.543+0000 7f8a86a70700 1 -- 172.21.15.3:0/3395998651 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a80126230 con 0x7f8a800a1340 2023-10-03T10:13:55.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.553+0000 7f8a7d7fa700 1 -- 172.21.15.3:0/3395998651 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8a80126230 con 0x7f8a800a1340 2023-10-03T10:13:55.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.638+0000 7f38e97ed700 1 -- 172.21.15.3:0/1288444811 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f38c4005590 con 0x7f38e409d6f0 2023-10-03T10:13:55.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.639+0000 7f38cb7fe700 1 -- 172.21.15.3:0/1288444811 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f38d4011350 con 0x7f38e409d6f0 2023-10-03T10:13:55.639 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:55.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.642+0000 7f38e97ed700 1 -- 172.21.15.3:0/1288444811 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38e409d6f0 msgr2=0x7f38e41a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:55.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.642+0000 7f38e97ed700 1 --2- 172.21.15.3:0/1288444811 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38e409d6f0 0x7f38e41a5130 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f38d400ebe0 tx=0x7f38d400eef0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.642+0000 7f38e97ed700 1 -- 172.21.15.3:0/1288444811 shutdown_connections 2023-10-03T10:13:55.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.642+0000 7f38e97ed700 1 --2- 172.21.15.3:0/1288444811 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38e409e410 0x7f38e41c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.642+0000 7f38e97ed700 1 --2- 172.21.15.3:0/1288444811 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38e409d6f0 0x7f38e41a5130 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.642+0000 7f38e97ed700 1 --2- 172.21.15.3:0/1288444811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38e409e000 0x7f38e41a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.642+0000 7f38e97ed700 1 -- 172.21.15.3:0/1288444811 >> 172.21.15.3:0/1288444811 conn(0x7f38e4066f50 msgr2=0x7f38e40a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:55.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.642+0000 7f38e97ed700 1 -- 172.21.15.3:0/1288444811 shutdown_connections 2023-10-03T10:13:55.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.642+0000 7f38e97ed700 1 -- 172.21.15.3:0/1288444811 wait complete. 2023-10-03T10:13:55.661 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:55.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.732+0000 7fc6f2c8f700 1 -- 172.21.15.3:0/1134788017 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc6ec074ea0 con 0x7fc6ec09d6f0 2023-10-03T10:13:55.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.732+0000 7fc6e97fa700 1 -- 172.21.15.3:0/1134788017 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc6e0015070 con 0x7fc6ec09d6f0 2023-10-03T10:13:55.732 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:55.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.734+0000 7fc6f2c8f700 1 -- 172.21.15.3:0/1134788017 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6ec09d6f0 msgr2=0x7fc6ec1a5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:55.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.734+0000 7fc6f2c8f700 1 --2- 172.21.15.3:0/1134788017 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6ec09d6f0 0x7fc6ec1a5e30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0016000 tx=0x7fc6e0004750 comp rx=0 tx=0).stop 2023-10-03T10:13:55.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.734+0000 7fc6f2c8f700 1 -- 172.21.15.3:0/1134788017 shutdown_connections 2023-10-03T10:13:55.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.734+0000 7fc6f2c8f700 1 --2- 172.21.15.3:0/1134788017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6ec09e440 0x7fc6ec1a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.734+0000 7fc6f2c8f700 1 --2- 172.21.15.3:0/1134788017 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6ec09d6f0 0x7fc6ec1a5e30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.734+0000 7fc6f2c8f700 1 --2- 172.21.15.3:0/1134788017 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6ec09e000 0x7fc6ec1a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.734+0000 7fc6f2c8f700 1 -- 172.21.15.3:0/1134788017 >> 172.21.15.3:0/1134788017 conn(0x7fc6ec066f50 msgr2=0x7fc6ec1baea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:55.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.735+0000 7fc6f2c8f700 1 -- 172.21.15.3:0/1134788017 shutdown_connections 2023-10-03T10:13:55.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.735+0000 7fc6f2c8f700 1 -- 172.21.15.3:0/1134788017 wait complete. 2023-10-03T10:13:55.748 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:55.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.759+0000 7f8a86a70700 1 -- 172.21.15.3:0/3395998651 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8a80126230 con 0x7f8a800a1340 2023-10-03T10:13:55.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.760+0000 7f8a7d7fa700 1 -- 172.21.15.3:0/3395998651 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8a80126230 con 0x7f8a800a1340 2023-10-03T10:13:55.760 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:55.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.762+0000 7f8a86a70700 1 -- 172.21.15.3:0/3395998651 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a800a1340 msgr2=0x7f8a801c8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:55.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.762+0000 7f8a86a70700 1 --2- 172.21.15.3:0/3395998651 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a800a1340 0x7f8a801c8d80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c00c520 tx=0x7f8a6c00c830 comp rx=0 tx=0).stop 2023-10-03T10:13:55.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.762+0000 7f8a86a70700 1 -- 172.21.15.3:0/3395998651 shutdown_connections 2023-10-03T10:13:55.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.762+0000 7f8a86a70700 1 --2- 172.21.15.3:0/3395998651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a80134170 0x7f8a801227d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.762+0000 7f8a86a70700 1 --2- 172.21.15.3:0/3395998651 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a800a0a30 0x7f8a801c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.762+0000 7f8a86a70700 1 --2- 172.21.15.3:0/3395998651 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a800a1340 0x7f8a801c8d80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:55.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.762+0000 7f8a86a70700 1 -- 172.21.15.3:0/3395998651 >> 172.21.15.3:0/3395998651 conn(0x7f8a800a6550 msgr2=0x7f8a8009ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:55.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.762+0000 7f8a86a70700 1 -- 172.21.15.3:0/3395998651 shutdown_connections 2023-10-03T10:13:55.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:55.762+0000 7f8a86a70700 1 -- 172.21.15.3:0/3395998651 wait complete. 2023-10-03T10:13:55.776 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:56.311 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:56.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.456+0000 7f86abb49700 1 Processor -- start 2023-10-03T10:13:56.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.456+0000 7f86abb49700 1 -- start start 2023-10-03T10:13:56.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.456+0000 7f86abb49700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86a409d6f0 0x7f86a409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:56.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.456+0000 7f86abb49700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f86a409e000 0x7f86a4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:56.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.456+0000 7f86abb49700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86a409e410 0x7f86a4130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:56.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.456+0000 7f86abb49700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86a409f520 con 0x7f86a409e000 2023-10-03T10:13:56.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.456+0000 7f86abb49700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a409f130 con 0x7f86a409d6f0 2023-10-03T10:13:56.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.456+0000 7f86abb49700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f86a409e930 con 0x7f86a409e410 2023-10-03T10:13:56.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.457+0000 7f86a90e4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f86a409e000 0x7f86a4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:56.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.457+0000 7f86aa0e6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86a409e410 0x7f86a4130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:56.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.457+0000 7f86a98e5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86a409d6f0 0x7f86a409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:56.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.457+0000 7f86a90e4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f86a409e000 0x7f86a4130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51396/0 (socket says 172.21.15.3:51396) 2023-10-03T10:13:56.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.457+0000 7f86a90e4700 1 -- 172.21.15.3:0/1486055084 learned_addr learned my addr 172.21.15.3:0/1486055084 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:56.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.457+0000 7f86a90e4700 1 -- 172.21.15.3:0/1486055084 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86a409e410 msgr2=0x7f86a4130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:56.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.457+0000 7f86a90e4700 1 --2- 172.21.15.3:0/1486055084 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86a409e410 0x7f86a4130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:56.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.457+0000 7f86a90e4700 1 -- 172.21.15.3:0/1486055084 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86a409d6f0 msgr2=0x7f86a409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:56.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.457+0000 7f86a90e4700 1 --2- 172.21.15.3:0/1486055084 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86a409d6f0 0x7f86a409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:56.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.457+0000 7f86a90e4700 1 -- 172.21.15.3:0/1486055084 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a409b2d0 con 0x7f86a409e000 2023-10-03T10:13:56.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.457+0000 7f86a98e5700 1 --2- 172.21.15.3:0/1486055084 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86a409d6f0 0x7f86a409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:56.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.457+0000 7f86a90e4700 1 --2- 172.21.15.3:0/1486055084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86a409e000 0x7f86a4130660 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8694009920 tx=0x7f8694009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ab2786dab48b5c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:56.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.457+0000 7f86a88e3700 1 -- 172.21.15.3:0/1486055084 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8694011830 con 0x7f86a409e000 2023-10-03T10:13:56.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.457+0000 7f86a88e3700 1 -- 172.21.15.3:0/1486055084 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8694011990 con 0x7f86a409e000 2023-10-03T10:13:56.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.457+0000 7f86a88e3700 1 -- 172.21.15.3:0/1486055084 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8694011d20 con 0x7f86a409e000 2023-10-03T10:13:56.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.458+0000 7f86abb49700 1 -- 172.21.15.3:0/1486055084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86a409e000 msgr2=0x7f86a4130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:56.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.458+0000 7f86abb49700 1 --2- 172.21.15.3:0/1486055084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86a409e000 0x7f86a4130660 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8694009920 tx=0x7f8694009c30 comp rx=0 tx=0).stop 2023-10-03T10:13:56.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.458+0000 7f86abb49700 1 -- 172.21.15.3:0/1486055084 shutdown_connections 2023-10-03T10:13:56.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.458+0000 7f86abb49700 1 --2- 172.21.15.3:0/1486055084 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86a409e410 0x7f86a4130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:56.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.458+0000 7f86abb49700 1 --2- 172.21.15.3:0/1486055084 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86a409d6f0 0x7f86a409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:56.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.458+0000 7f86abb49700 1 --2- 172.21.15.3:0/1486055084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86a409e000 0x7f86a4130660 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:56.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.458+0000 7f86abb49700 1 -- 172.21.15.3:0/1486055084 >> 172.21.15.3:0/1486055084 conn(0x7f86a4066f50 msgr2=0x7f86a409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:56.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.458+0000 7f86abb49700 1 -- 172.21.15.3:0/1486055084 shutdown_connections 2023-10-03T10:13:56.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.458+0000 7f86abb49700 1 -- 172.21.15.3:0/1486055084 wait complete. 2023-10-03T10:13:56.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.459+0000 7f86abb49700 1 Processor -- start 2023-10-03T10:13:56.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.459+0000 7f86abb49700 1 -- start start 2023-10-03T10:13:56.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.460+0000 7f86abb49700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86a409d6f0 0x7f86a4122360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:56.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.460+0000 7f86abb49700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86a409e000 0x7f86a41228a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:56.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.460+0000 7f86abb49700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86a409e410 0x7f86a4122de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:56.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.460+0000 7f86abb49700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a4125580 con 0x7f86a409e410 2023-10-03T10:13:56.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.460+0000 7f86abb49700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a4123f70 con 0x7f86a409d6f0 2023-10-03T10:13:56.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.460+0000 7f86abb49700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f86a4124840 con 0x7f86a409e000 2023-10-03T10:13:56.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.460+0000 7f86a98e5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86a409d6f0 0x7f86a4122360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:56.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.460+0000 7f86a90e4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86a409e000 0x7f86a41228a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:56.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.460+0000 7f86a98e5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86a409d6f0 0x7f86a4122360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33274/0 (socket says 172.21.15.3:33274) 2023-10-03T10:13:56.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.460+0000 7f86a98e5700 1 -- 172.21.15.3:0/614712372 learned_addr learned my addr 172.21.15.3:0/614712372 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:56.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.460+0000 7f86aa0e6700 1 --2- 172.21.15.3:0/614712372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86a409e410 0x7f86a4122de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:56.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.460+0000 7f86a90e4700 1 -- 172.21.15.3:0/614712372 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86a409d6f0 msgr2=0x7f86a4122360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:56.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.460+0000 7f86a90e4700 1 --2- 172.21.15.3:0/614712372 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86a409d6f0 0x7f86a4122360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:56.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.460+0000 7f86a90e4700 1 -- 172.21.15.3:0/614712372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86a409e410 msgr2=0x7f86a4122de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:56.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.460+0000 7f86a90e4700 1 --2- 172.21.15.3:0/614712372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86a409e410 0x7f86a4122de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:56.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.460+0000 7f86a90e4700 1 -- 172.21.15.3:0/614712372 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a00094a0 con 0x7f86a409e000 2023-10-03T10:13:56.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.460+0000 7f86a98e5700 1 --2- 172.21.15.3:0/614712372 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86a409d6f0 0x7f86a4122360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:56.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.460+0000 7f86a90e4700 1 --2- 172.21.15.3:0/614712372 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86a409e000 0x7f86a41228a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8694016000 tx=0x7f8694004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:56.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.460+0000 7f869a7fc700 1 -- 172.21.15.3:0/614712372 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8694004450 con 0x7f86a409e000 2023-10-03T10:13:56.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.460+0000 7f869a7fc700 1 -- 172.21.15.3:0/614712372 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869401f070 con 0x7f86a409e000 2023-10-03T10:13:56.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.460+0000 7f86abb49700 1 -- 172.21.15.3:0/614712372 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8694009550 con 0x7f86a409e000 2023-10-03T10:13:56.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.460+0000 7f869a7fc700 1 -- 172.21.15.3:0/614712372 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86940045f0 con 0x7f86a409e000 2023-10-03T10:13:56.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.461+0000 7f86abb49700 1 -- 172.21.15.3:0/614712372 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a41258e0 con 0x7f86a409e000 2023-10-03T10:13:56.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.461+0000 7f869a7fc700 1 -- 172.21.15.3:0/614712372 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f86940038e0 con 0x7f86a409e000 2023-10-03T10:13:56.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.461+0000 7f869a7fc700 1 -- 172.21.15.3:0/614712372 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8694021070 con 0x7f86a409e000 2023-10-03T10:13:56.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.461+0000 7f86abb49700 1 -- 172.21.15.3:0/614712372 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86a4074ea0 con 0x7f86a409e000 2023-10-03T10:13:56.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.467+0000 7f869a7fc700 1 -- 172.21.15.3:0/614712372 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8694015080 con 0x7f86a409e000 2023-10-03T10:13:56.662 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:56.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.672+0000 7f86abb49700 1 -- 172.21.15.3:0/614712372 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f86a406c360 con 0x7f86a409e000 2023-10-03T10:13:56.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.673+0000 7f869a7fc700 1 -- 172.21.15.3:0/614712372 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f869402a020 con 0x7f86a409e000 2023-10-03T10:13:56.674 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:56.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.675+0000 7f86abb49700 1 -- 172.21.15.3:0/614712372 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86a409e000 msgr2=0x7f86a41228a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:56.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.675+0000 7f86abb49700 1 --2- 172.21.15.3:0/614712372 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86a409e000 0x7f86a41228a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8694016000 tx=0x7f8694004750 comp rx=0 tx=0).stop 2023-10-03T10:13:56.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.675+0000 7f86abb49700 1 -- 172.21.15.3:0/614712372 shutdown_connections 2023-10-03T10:13:56.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.675+0000 7f86abb49700 1 --2- 172.21.15.3:0/614712372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86a409e410 0x7f86a4122de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:56.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.675+0000 7f86abb49700 1 --2- 172.21.15.3:0/614712372 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86a409d6f0 0x7f86a4122360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:56.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.675+0000 7f86abb49700 1 --2- 172.21.15.3:0/614712372 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86a409e000 0x7f86a41228a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:56.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.675+0000 7f86abb49700 1 -- 172.21.15.3:0/614712372 >> 172.21.15.3:0/614712372 conn(0x7f86a4066f50 msgr2=0x7f86a406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:56.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.675+0000 7f86abb49700 1 -- 172.21.15.3:0/614712372 shutdown_connections 2023-10-03T10:13:56.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.675+0000 7f86abb49700 1 -- 172.21.15.3:0/614712372 wait complete. 2023-10-03T10:13:56.691 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:56.749 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:56.776 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:56.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.802+0000 7f41040ff700 1 Processor -- start 2023-10-03T10:13:56.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.803+0000 7f41040ff700 1 -- start start 2023-10-03T10:13:56.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.803+0000 7f41040ff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40fc136710 0x7f40fc138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:56.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.803+0000 7f41040ff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40fc139030 0x7f40fc13b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:56.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.803+0000 7f41040ff700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40fc06ef30 con 0x7f40fc1338a0 2023-10-03T10:13:56.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.803+0000 7f41040ff700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc06fef0 con 0x7f40fc139030 2023-10-03T10:13:56.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.803+0000 7f41040ff700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc1334b0 con 0x7f40fc136710 2023-10-03T10:13:56.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.803+0000 7f410269c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40fc139030 0x7f40fc13b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:56.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.803+0000 7f410269c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40fc139030 0x7f40fc13b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33280/0 (socket says 172.21.15.3:33280) 2023-10-03T10:13:56.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.803+0000 7f410269c700 1 -- 172.21.15.3:0/2130403759 learned_addr learned my addr 172.21.15.3:0/2130403759 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:56.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.804+0000 7f410169a700 1 --2- 172.21.15.3:0/2130403759 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40fc136710 0x7f40fc138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:56.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.804+0000 7f4100e99700 1 -- 172.21.15.3:0/2130403759 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 828875388 0 0) 0x7f40fc06ef30 con 0x7f40fc1338a0 2023-10-03T10:13:56.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.804+0000 7f4100e99700 1 -- 172.21.15.3:0/2130403759 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40e80035b0 con 0x7f40fc1338a0 2023-10-03T10:13:56.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.804+0000 7f410269c700 1 -- 172.21.15.3:0/2130403759 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40fc136710 msgr2=0x7f40fc138af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:56.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.804+0000 7f410269c700 1 --2- 172.21.15.3:0/2130403759 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40fc136710 0x7f40fc138af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:56.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.804+0000 7f410269c700 1 -- 172.21.15.3:0/2130403759 >> v1:172.21.15.3:6789/0 conn(0x7f40fc1338a0 legacy=0x7f40fc133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:56.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.804+0000 7f410269c700 1 -- 172.21.15.3:0/2130403759 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40fc06fc30 con 0x7f40fc139030 2023-10-03T10:13:56.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.804+0000 7f410169a700 1 --2- 172.21.15.3:0/2130403759 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40fc136710 0x7f40fc138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:56.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.804+0000 7f410269c700 1 --2- 172.21.15.3:0/2130403759 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40fc139030 0x7f40fc13b410 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009e80 tx=0x7f40f400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=116866137469cfa5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:56.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.804+0000 7f4100e99700 1 -- 172.21.15.3:0/2130403759 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f400e070 con 0x7f40fc139030 2023-10-03T10:13:56.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.804+0000 7f4100e99700 1 -- 172.21.15.3:0/2130403759 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f400b910 con 0x7f40fc139030 2023-10-03T10:13:56.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.804+0000 7f4100e99700 1 -- 172.21.15.3:0/2130403759 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f400bcc0 con 0x7f40fc139030 2023-10-03T10:13:56.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.805+0000 7f41040ff700 1 -- 172.21.15.3:0/2130403759 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40fc139030 msgr2=0x7f40fc13b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:56.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.805+0000 7f41040ff700 1 --2- 172.21.15.3:0/2130403759 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40fc139030 0x7f40fc13b410 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009e80 tx=0x7f40f400b330 comp rx=0 tx=0).stop 2023-10-03T10:13:56.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.805+0000 7f41040ff700 1 -- 172.21.15.3:0/2130403759 shutdown_connections 2023-10-03T10:13:56.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.805+0000 7f41040ff700 1 --2- 172.21.15.3:0/2130403759 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40fc139030 0x7f40fc13b410 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:56.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.805+0000 7f41040ff700 1 --2- 172.21.15.3:0/2130403759 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40fc136710 0x7f40fc138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:56.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.805+0000 7f41040ff700 1 -- 172.21.15.3:0/2130403759 >> 172.21.15.3:0/2130403759 conn(0x7f40fc0a2aa0 msgr2=0x7f40fc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:56.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.806+0000 7f41040ff700 1 -- 172.21.15.3:0/2130403759 shutdown_connections 2023-10-03T10:13:56.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.806+0000 7f41040ff700 1 -- 172.21.15.3:0/2130403759 wait complete. 2023-10-03T10:13:56.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.807+0000 7f41040ff700 1 Processor -- start 2023-10-03T10:13:56.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.807+0000 7f41040ff700 1 -- start start 2023-10-03T10:13:56.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.807+0000 7f41040ff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40fc1338a0 0x7f40fc099ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:56.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.807+0000 7f41040ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40fc136710 0x7f40fc09a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:56.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.807+0000 7f41040ff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40fc139030 0x7f40fc1cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:56.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.807+0000 7f41040ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc138c20 con 0x7f40fc136710 2023-10-03T10:13:56.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.807+0000 7f41040ff700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc09d2b0 con 0x7f40fc1338a0 2023-10-03T10:13:56.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.807+0000 7f41040ff700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc09b680 con 0x7f40fc139030 2023-10-03T10:13:56.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.808+0000 7f410169a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40fc136710 0x7f40fc09a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:56.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.808+0000 7f410169a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40fc136710 0x7f40fc09a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51408/0 (socket says 172.21.15.3:51408) 2023-10-03T10:13:56.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.808+0000 7f410169a700 1 -- 172.21.15.3:0/935159695 learned_addr learned my addr 172.21.15.3:0/935159695 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:56.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.808+0000 7f4101e9b700 1 --2- 172.21.15.3:0/935159695 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40fc1338a0 0x7f40fc099ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:56.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.808+0000 7f410269c700 1 --2- 172.21.15.3:0/935159695 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40fc139030 0x7f40fc1cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:56.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.808+0000 7f4101e9b700 1 -- 172.21.15.3:0/935159695 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40fc139030 msgr2=0x7f40fc1cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:56.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.808+0000 7f4101e9b700 1 --2- 172.21.15.3:0/935159695 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40fc139030 0x7f40fc1cc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:56.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.808+0000 7f4101e9b700 1 -- 172.21.15.3:0/935159695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40fc136710 msgr2=0x7f40fc09a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:56.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.808+0000 7f4101e9b700 1 --2- 172.21.15.3:0/935159695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40fc136710 0x7f40fc09a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:56.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.808+0000 7f4101e9b700 1 -- 172.21.15.3:0/935159695 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ec0094a0 con 0x7f40fc1338a0 2023-10-03T10:13:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.808+0000 7f410169a700 1 --2- 172.21.15.3:0/935159695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40fc136710 0x7f40fc09a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.808+0000 7f410269c700 1 --2- 172.21.15.3:0/935159695 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40fc139030 0x7f40fc1cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.808+0000 7f4101e9b700 1 --2- 172.21.15.3:0/935159695 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40fc1338a0 0x7f40fc099ce0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f40f800c450 tx=0x7f40f800c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.809+0000 7f40f27fc700 1 -- 172.21.15.3:0/935159695 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f80052d0 con 0x7f40fc1338a0 2023-10-03T10:13:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.809+0000 7f40f27fc700 1 -- 172.21.15.3:0/935159695 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f8005430 con 0x7f40fc1338a0 2023-10-03T10:13:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.809+0000 7f41040ff700 1 -- 172.21.15.3:0/935159695 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f4009b30 con 0x7f40fc1338a0 2023-10-03T10:13:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.809+0000 7f40f27fc700 1 -- 172.21.15.3:0/935159695 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f80038c0 con 0x7f40fc1338a0 2023-10-03T10:13:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.809+0000 7f41040ff700 1 -- 172.21.15.3:0/935159695 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40fc09be70 con 0x7f40fc1338a0 2023-10-03T10:13:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.810+0000 7f40f27fc700 1 -- 172.21.15.3:0/935159695 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f40f8003a20 con 0x7f40fc1338a0 2023-10-03T10:13:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.810+0000 7f40f27fc700 1 -- 172.21.15.3:0/935159695 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f40f803ebe0 con 0x7f40fc1338a0 2023-10-03T10:13:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.810+0000 7f41040ff700 1 -- 172.21.15.3:0/935159695 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40fc09d0f0 con 0x7f40fc1338a0 2023-10-03T10:13:56.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.815+0000 7f40f27fc700 1 -- 172.21.15.3:0/935159695 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f40f800f260 con 0x7f40fc1338a0 2023-10-03T10:13:56.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.894+0000 7f109e0f9700 1 Processor -- start 2023-10-03T10:13:56.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.894+0000 7f109e0f9700 1 -- start start 2023-10-03T10:13:56.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.894+0000 7f109e0f9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f109809e850 0x7f109809ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:56.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.894+0000 7f109e0f9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1098134290 0x7f109813be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:56.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.894+0000 7f109e0f9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f109806bc50 con 0x7f109809f160 2023-10-03T10:13:56.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.894+0000 7f109e0f9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f109809ae60 con 0x7f1098134290 2023-10-03T10:13:56.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.894+0000 7f109e0f9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109806b990 con 0x7f109809e850 2023-10-03T10:13:56.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.894+0000 7f10977fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f109809e850 0x7f109809ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:56.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.894+0000 7f10977fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f109809e850 0x7f109809ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33304/0 (socket says 172.21.15.3:33304) 2023-10-03T10:13:56.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.895+0000 7f10977fe700 1 -- 172.21.15.3:0/514326749 learned_addr learned my addr 172.21.15.3:0/514326749 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:56.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.895+0000 7f1097fff700 1 --2- 172.21.15.3:0/514326749 >> v2:172.21.15.3:3300/0 conn(0x7f1098134290 0x7f109813be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:56.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.895+0000 7f10977fe700 1 -- 172.21.15.3:0/514326749 >> v2:172.21.15.3:3300/0 conn(0x7f1098134290 msgr2=0x7f109813be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:56.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.895+0000 7f10977fe700 1 --2- 172.21.15.3:0/514326749 >> v2:172.21.15.3:3300/0 conn(0x7f1098134290 0x7f109813be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:56.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.895+0000 7f10977fe700 1 -- 172.21.15.3:0/514326749 >> v1:172.21.15.3:6789/0 conn(0x7f109809f160 legacy=0x7f1098133b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:56.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.895+0000 7f10977fe700 1 -- 172.21.15.3:0/514326749 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109806cb10 con 0x7f109809e850 2023-10-03T10:13:56.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.895+0000 7f10967fc700 1 -- 172.21.15.3:0/514326749 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1128635082 0 0) 0x7f109806bc50 con 0x7f109809f160 2023-10-03T10:13:56.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.895+0000 7f10977fe700 1 --2- 172.21.15.3:0/514326749 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f109809e850 0x7f109809ec20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f10880098f0 tx=0x7f1088009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bbcb38ef46f689f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:56.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.895+0000 7f10967fc700 1 -- 172.21.15.3:0/514326749 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1088011830 con 0x7f109809e850 2023-10-03T10:13:56.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.895+0000 7f10967fc700 1 -- 172.21.15.3:0/514326749 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1088011990 con 0x7f109809e850 2023-10-03T10:13:56.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.895+0000 7f10967fc700 1 -- 172.21.15.3:0/514326749 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1088011d20 con 0x7f109809e850 2023-10-03T10:13:56.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.896+0000 7f109e0f9700 1 -- 172.21.15.3:0/514326749 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f109809e850 msgr2=0x7f109809ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:56.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.896+0000 7f109e0f9700 1 --2- 172.21.15.3:0/514326749 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f109809e850 0x7f109809ec20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f10880098f0 tx=0x7f1088009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:56.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.896+0000 7f109e0f9700 1 -- 172.21.15.3:0/514326749 shutdown_connections 2023-10-03T10:13:56.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.896+0000 7f109e0f9700 1 --2- 172.21.15.3:0/514326749 >> v2:172.21.15.3:3300/0 conn(0x7f1098134290 0x7f109813be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:56.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.896+0000 7f109e0f9700 1 --2- 172.21.15.3:0/514326749 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f109809e850 0x7f109809ec20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:56.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.896+0000 7f109e0f9700 1 -- 172.21.15.3:0/514326749 >> 172.21.15.3:0/514326749 conn(0x7f10980a3f70 msgr2=0x7f10980a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:56.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.896+0000 7f109e0f9700 1 -- 172.21.15.3:0/514326749 shutdown_connections 2023-10-03T10:13:56.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.896+0000 7f109e0f9700 1 -- 172.21.15.3:0/514326749 wait complete. 2023-10-03T10:13:56.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.898+0000 7f109e0f9700 1 Processor -- start 2023-10-03T10:13:56.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.898+0000 7f109e0f9700 1 -- start start 2023-10-03T10:13:56.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.898+0000 7f109e0f9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f109809e850 0x7f1098122a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:56.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.898+0000 7f109e0f9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f109809f160 0x7f1098122f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:56.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.898+0000 7f109e0f9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1098134290 0x7f10981234b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:56.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.898+0000 7f109e0f9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1098123f00 con 0x7f109809e850 2023-10-03T10:13:56.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.898+0000 7f109e0f9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1098133550 con 0x7f109809f160 2023-10-03T10:13:56.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.898+0000 7f109e0f9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1098124750 con 0x7f1098134290 2023-10-03T10:13:56.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.898+0000 7f1096ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f109809f160 0x7f1098122f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:56.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.899+0000 7f1096ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f109809f160 0x7f1098122f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33318/0 (socket says 172.21.15.3:33318) 2023-10-03T10:13:56.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.899+0000 7f1096ffd700 1 -- 172.21.15.3:0/1006106226 learned_addr learned my addr 172.21.15.3:0/1006106226 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:56.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.899+0000 7f10977fe700 1 --2- 172.21.15.3:0/1006106226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f109809e850 0x7f1098122a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:56.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.899+0000 7f1097fff700 1 --2- 172.21.15.3:0/1006106226 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1098134290 0x7f10981234b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:56.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.899+0000 7f1096ffd700 1 -- 172.21.15.3:0/1006106226 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1098134290 msgr2=0x7f10981234b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:56.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.899+0000 7f1096ffd700 1 --2- 172.21.15.3:0/1006106226 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1098134290 0x7f10981234b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:56.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.899+0000 7f1096ffd700 1 -- 172.21.15.3:0/1006106226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f109809e850 msgr2=0x7f1098122a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:56.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.899+0000 7f1096ffd700 1 --2- 172.21.15.3:0/1006106226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f109809e850 0x7f1098122a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:56.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.899+0000 7f1096ffd700 1 -- 172.21.15.3:0/1006106226 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1088009550 con 0x7f109809f160 2023-10-03T10:13:56.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.899+0000 7f1096ffd700 1 --2- 172.21.15.3:0/1006106226 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f109809f160 0x7f1098122f70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f108c001eb0 tx=0x7f108c00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:56.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.899+0000 7f107ffff700 1 -- 172.21.15.3:0/1006106226 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c0054e0 con 0x7f109809f160 2023-10-03T10:13:56.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.899+0000 7f107ffff700 1 -- 172.21.15.3:0/1006106226 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108c003680 con 0x7f109809f160 2023-10-03T10:13:56.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.899+0000 7f107ffff700 1 -- 172.21.15.3:0/1006106226 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c0039f0 con 0x7f109809f160 2023-10-03T10:13:56.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.900+0000 7f109e0f9700 1 -- 172.21.15.3:0/1006106226 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1098133980 con 0x7f109809f160 2023-10-03T10:13:56.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.900+0000 7f109e0f9700 1 -- 172.21.15.3:0/1006106226 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1098126270 con 0x7f109809f160 2023-10-03T10:13:56.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.901+0000 7f107ffff700 1 -- 172.21.15.3:0/1006106226 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f108c003b50 con 0x7f109809f160 2023-10-03T10:13:56.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.901+0000 7f107ffff700 1 -- 172.21.15.3:0/1006106226 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f108c03eaf0 con 0x7f109809f160 2023-10-03T10:13:56.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.901+0000 7f109e0f9700 1 -- 172.21.15.3:0/1006106226 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10980747f0 con 0x7f109809f160 2023-10-03T10:13:56.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.910+0000 7f107ffff700 1 -- 172.21.15.3:0/1006106226 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f108c01b3b0 con 0x7f109809f160 2023-10-03T10:13:56.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.936+0000 7f7f68618700 1 Processor -- start 2023-10-03T10:13:56.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.936+0000 7f7f68618700 1 -- start start 2023-10-03T10:13:56.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.937+0000 7f7f68618700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f6009e930 0x7f7f6009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:56.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.937+0000 7f7f68618700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f6009f240 0x7f7f601317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:56.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.937+0000 7f7f68618700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7f6009f650 0x7f7f60131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:56.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.937+0000 7f7f68618700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f60070940 con 0x7f7f6009f650 2023-10-03T10:13:56.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.937+0000 7f7f68618700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6006fd60 con 0x7f7f6009f240 2023-10-03T10:13:56.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.937+0000 7f7f68618700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6006fa80 con 0x7f7f6009e930 2023-10-03T10:13:56.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.937+0000 7f7f65bb3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f6009f240 0x7f7f601317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:56.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.937+0000 7f7f65bb3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f6009f240 0x7f7f601317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33320/0 (socket says 172.21.15.3:33320) 2023-10-03T10:13:56.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.937+0000 7f7f663b4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f6009e930 0x7f7f6009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:56.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.937+0000 7f7f65bb3700 1 -- 172.21.15.3:0/2511713256 learned_addr learned my addr 172.21.15.3:0/2511713256 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:56.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.937+0000 7f7f66bb5700 1 --2- 172.21.15.3:0/2511713256 >> v2:172.21.15.3:3300/0 conn(0x7f7f6009f650 0x7f7f60131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:56.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.937+0000 7f7f65bb3700 1 -- 172.21.15.3:0/2511713256 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f6009e930 msgr2=0x7f7f6009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:56.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.937+0000 7f7f65bb3700 1 --2- 172.21.15.3:0/2511713256 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f6009e930 0x7f7f6009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:56.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.938+0000 7f7f65bb3700 1 -- 172.21.15.3:0/2511713256 >> v2:172.21.15.3:3300/0 conn(0x7f7f6009f650 msgr2=0x7f7f60131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:56.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.938+0000 7f7f65bb3700 1 --2- 172.21.15.3:0/2511713256 >> v2:172.21.15.3:3300/0 conn(0x7f7f6009f650 0x7f7f60131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:56.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.938+0000 7f7f65bb3700 1 -- 172.21.15.3:0/2511713256 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6009fb60 con 0x7f7f6009f240 2023-10-03T10:13:56.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.938+0000 7f7f663b4700 1 --2- 172.21.15.3:0/2511713256 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f6009e930 0x7f7f6009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:56.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.938+0000 7f7f65bb3700 1 --2- 172.21.15.3:0/2511713256 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f6009f240 0x7f7f601317f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7f500098f0 tx=0x7f7f50009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=318ac3f45463784b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:56.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.938+0000 7f7f653b2700 1 -- 172.21.15.3:0/2511713256 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f50011830 con 0x7f7f6009f240 2023-10-03T10:13:56.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.938+0000 7f7f653b2700 1 -- 172.21.15.3:0/2511713256 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f50011990 con 0x7f7f6009f240 2023-10-03T10:13:56.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.938+0000 7f7f653b2700 1 -- 172.21.15.3:0/2511713256 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f50011d20 con 0x7f7f6009f240 2023-10-03T10:13:56.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.939+0000 7f7f68618700 1 -- 172.21.15.3:0/2511713256 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f6009f240 msgr2=0x7f7f601317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:56.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.939+0000 7f7f68618700 1 --2- 172.21.15.3:0/2511713256 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f6009f240 0x7f7f601317f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7f500098f0 tx=0x7f7f50009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:56.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.939+0000 7f7f68618700 1 -- 172.21.15.3:0/2511713256 shutdown_connections 2023-10-03T10:13:56.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.939+0000 7f7f68618700 1 --2- 172.21.15.3:0/2511713256 >> v2:172.21.15.3:3300/0 conn(0x7f7f6009f650 0x7f7f60131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:56.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.939+0000 7f7f68618700 1 --2- 172.21.15.3:0/2511713256 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f6009e930 0x7f7f6009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:56.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.939+0000 7f7f68618700 1 --2- 172.21.15.3:0/2511713256 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f6009f240 0x7f7f601317f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:56.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.939+0000 7f7f68618700 1 -- 172.21.15.3:0/2511713256 >> 172.21.15.3:0/2511713256 conn(0x7f7f600a4210 msgr2=0x7f7f600a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:56.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.939+0000 7f7f68618700 1 -- 172.21.15.3:0/2511713256 shutdown_connections 2023-10-03T10:13:56.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.939+0000 7f7f68618700 1 -- 172.21.15.3:0/2511713256 wait complete. 2023-10-03T10:13:56.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.941+0000 7f7f68618700 1 Processor -- start 2023-10-03T10:13:56.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.941+0000 7f7f68618700 1 -- start start 2023-10-03T10:13:56.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.941+0000 7f7f68618700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f6009e930 0x7f7f6006b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:56.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.941+0000 7f7f68618700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f6009f240 0x7f7f6006bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:56.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.941+0000 7f7f68618700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f6009f650 0x7f7f6006c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:56.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.941+0000 7f7f68618700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f601a7960 con 0x7f7f6009e930 2023-10-03T10:13:56.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.941+0000 7f7f68618700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6006d530 con 0x7f7f6009f240 2023-10-03T10:13:56.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.941+0000 7f7f68618700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f600a3580 con 0x7f7f6009f650 2023-10-03T10:13:56.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.942+0000 7f7f65bb3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f6009f240 0x7f7f6006bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:56.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.942+0000 7f7f65bb3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f6009f240 0x7f7f6006bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33328/0 (socket says 172.21.15.3:33328) 2023-10-03T10:13:56.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.942+0000 7f7f65bb3700 1 -- 172.21.15.3:0/2247059826 learned_addr learned my addr 172.21.15.3:0/2247059826 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:56.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.942+0000 7f7f663b4700 1 --2- 172.21.15.3:0/2247059826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f6009e930 0x7f7f6006b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:56.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.942+0000 7f7f66bb5700 1 --2- 172.21.15.3:0/2247059826 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f6009f650 0x7f7f6006c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:56.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.942+0000 7f7f65bb3700 1 -- 172.21.15.3:0/2247059826 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f6009f650 msgr2=0x7f7f6006c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:56.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.942+0000 7f7f65bb3700 1 --2- 172.21.15.3:0/2247059826 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f6009f650 0x7f7f6006c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:56.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.942+0000 7f7f65bb3700 1 -- 172.21.15.3:0/2247059826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f6009e930 msgr2=0x7f7f6006b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:56.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.942+0000 7f7f65bb3700 1 --2- 172.21.15.3:0/2247059826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f6009e930 0x7f7f6006b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:56.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.942+0000 7f7f65bb3700 1 -- 172.21.15.3:0/2247059826 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5c0094a0 con 0x7f7f6009f240 2023-10-03T10:13:56.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.942+0000 7f7f65bb3700 1 --2- 172.21.15.3:0/2247059826 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f6009f240 0x7f7f6006bdc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7f50000c00 tx=0x7f7f50004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:56.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.942+0000 7f7f56ffd700 1 -- 172.21.15.3:0/2247059826 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f50004240 con 0x7f7f6009f240 2023-10-03T10:13:56.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.942+0000 7f7f56ffd700 1 -- 172.21.15.3:0/2247059826 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5001f070 con 0x7f7f6009f240 2023-10-03T10:13:56.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.942+0000 7f7f68618700 1 -- 172.21.15.3:0/2247059826 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f50009550 con 0x7f7f6009f240 2023-10-03T10:13:56.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.942+0000 7f7f56ffd700 1 -- 172.21.15.3:0/2247059826 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f500045e0 con 0x7f7f6009f240 2023-10-03T10:13:56.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.943+0000 7f7f68618700 1 -- 172.21.15.3:0/2247059826 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f600a0190 con 0x7f7f6009f240 2023-10-03T10:13:56.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.943+0000 7f7f56ffd700 1 -- 172.21.15.3:0/2247059826 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7f50021030 con 0x7f7f6009f240 2023-10-03T10:13:56.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.944+0000 7f7f56ffd700 1 -- 172.21.15.3:0/2247059826 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7f50040920 con 0x7f7f6009f240 2023-10-03T10:13:56.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.943+0000 7f7f68618700 1 -- 172.21.15.3:0/2247059826 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f60074ea0 con 0x7f7f6009f240 2023-10-03T10:13:56.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:56.953+0000 7f7f56ffd700 1 -- 172.21.15.3:0/2247059826 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7f5001a450 con 0x7f7f6009f240 2023-10-03T10:13:57.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.030+0000 7f41040ff700 1 -- 172.21.15.3:0/935159695 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f40fc074ea0 con 0x7f40fc1338a0 2023-10-03T10:13:57.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.030+0000 7f40f27fc700 1 -- 172.21.15.3:0/935159695 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f40f8017030 con 0x7f40fc1338a0 2023-10-03T10:13:57.031 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:57.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.032+0000 7f41040ff700 1 -- 172.21.15.3:0/935159695 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40fc1338a0 msgr2=0x7f40fc099ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:57.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.032+0000 7f41040ff700 1 --2- 172.21.15.3:0/935159695 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40fc1338a0 0x7f40fc099ce0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f40f800c450 tx=0x7f40f800c760 comp rx=0 tx=0).stop 2023-10-03T10:13:57.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.032+0000 7f41040ff700 1 -- 172.21.15.3:0/935159695 shutdown_connections 2023-10-03T10:13:57.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.032+0000 7f41040ff700 1 --2- 172.21.15.3:0/935159695 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40fc139030 0x7f40fc1cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:57.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.032+0000 7f41040ff700 1 --2- 172.21.15.3:0/935159695 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40fc1338a0 0x7f40fc099ce0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:57.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.032+0000 7f41040ff700 1 --2- 172.21.15.3:0/935159695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40fc136710 0x7f40fc09a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:57.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.032+0000 7f41040ff700 1 -- 172.21.15.3:0/935159695 >> 172.21.15.3:0/935159695 conn(0x7f40fc0a2aa0 msgr2=0x7f40fc1344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:57.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.033+0000 7f41040ff700 1 -- 172.21.15.3:0/935159695 shutdown_connections 2023-10-03T10:13:57.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.033+0000 7f41040ff700 1 -- 172.21.15.3:0/935159695 wait complete. 2023-10-03T10:13:57.047 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:57.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.127+0000 7f109e0f9700 1 -- 172.21.15.3:0/1006106226 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1098072840 con 0x7f109809f160 2023-10-03T10:13:57.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.127+0000 7f107ffff700 1 -- 172.21.15.3:0/1006106226 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f108c017020 con 0x7f109809f160 2023-10-03T10:13:57.128 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:57.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.129+0000 7f109e0f9700 1 -- 172.21.15.3:0/1006106226 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f109809f160 msgr2=0x7f1098122f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:57.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.129+0000 7f109e0f9700 1 --2- 172.21.15.3:0/1006106226 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f109809f160 0x7f1098122f70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f108c001eb0 tx=0x7f108c00c800 comp rx=0 tx=0).stop 2023-10-03T10:13:57.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.130+0000 7f109e0f9700 1 -- 172.21.15.3:0/1006106226 shutdown_connections 2023-10-03T10:13:57.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.130+0000 7f109e0f9700 1 --2- 172.21.15.3:0/1006106226 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1098134290 0x7f10981234b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:57.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.130+0000 7f109e0f9700 1 --2- 172.21.15.3:0/1006106226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f109809e850 0x7f1098122a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:57.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.130+0000 7f109e0f9700 1 --2- 172.21.15.3:0/1006106226 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f109809f160 0x7f1098122f70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:57.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.130+0000 7f109e0f9700 1 -- 172.21.15.3:0/1006106226 >> 172.21.15.3:0/1006106226 conn(0x7f10980a3f70 msgr2=0x7f1098070f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:57.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.130+0000 7f109e0f9700 1 -- 172.21.15.3:0/1006106226 shutdown_connections 2023-10-03T10:13:57.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.130+0000 7f109e0f9700 1 -- 172.21.15.3:0/1006106226 wait complete. 2023-10-03T10:13:57.144 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:57.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.160+0000 7f7f68618700 1 -- 172.21.15.3:0/2247059826 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7f6006b250 con 0x7f7f6009f240 2023-10-03T10:13:57.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.160+0000 7f7f56ffd700 1 -- 172.21.15.3:0/2247059826 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7f500193b0 con 0x7f7f6009f240 2023-10-03T10:13:57.161 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:57.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.162+0000 7f7f68618700 1 -- 172.21.15.3:0/2247059826 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f6009f240 msgr2=0x7f7f6006bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:57.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.162+0000 7f7f68618700 1 --2- 172.21.15.3:0/2247059826 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f6009f240 0x7f7f6006bdc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7f50000c00 tx=0x7f7f50004750 comp rx=0 tx=0).stop 2023-10-03T10:13:57.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.162+0000 7f7f68618700 1 -- 172.21.15.3:0/2247059826 shutdown_connections 2023-10-03T10:13:57.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.162+0000 7f7f68618700 1 --2- 172.21.15.3:0/2247059826 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f6009f650 0x7f7f6006c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:57.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.162+0000 7f7f68618700 1 --2- 172.21.15.3:0/2247059826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f6009e930 0x7f7f6006b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:57.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.162+0000 7f7f68618700 1 --2- 172.21.15.3:0/2247059826 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f6009f240 0x7f7f6006bdc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:57.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.162+0000 7f7f68618700 1 -- 172.21.15.3:0/2247059826 >> 172.21.15.3:0/2247059826 conn(0x7f7f600a4210 msgr2=0x7f7f601b8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:57.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.162+0000 7f7f68618700 1 -- 172.21.15.3:0/2247059826 shutdown_connections 2023-10-03T10:13:57.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.162+0000 7f7f68618700 1 -- 172.21.15.3:0/2247059826 wait complete. 2023-10-03T10:13:57.176 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:57.692 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:57.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.837+0000 7fbbed705700 1 Processor -- start 2023-10-03T10:13:57.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.838+0000 7fbbed705700 1 -- start start 2023-10-03T10:13:57.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.838+0000 7fbbed705700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbbe80a0a30 0x7fbbe80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:57.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.838+0000 7fbbed705700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbbe8138d80 0x7fbbe813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:57.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.838+0000 7fbbed705700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe80a2a70 con 0x7fbbe809d630 2023-10-03T10:13:57.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.838+0000 7fbbed705700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe80a01d0 con 0x7fbbe80a0a30 2023-10-03T10:13:57.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.838+0000 7fbbed705700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe809e2b0 con 0x7fbbe8138d80 2023-10-03T10:13:57.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.838+0000 7fbbe77fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbbe8138d80 0x7fbbe813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:57.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.838+0000 7fbbe6ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbbe80a0a30 0x7fbbe80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:57.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.838+0000 7fbbe67fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbbe809d630 0x7fbbe80a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45238/0 (socket says 172.21.15.3:45238) 2023-10-03T10:13:57.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.838+0000 7fbbe67fc700 1 -- 172.21.15.3:0/1770044666 learned_addr learned my addr 172.21.15.3:0/1770044666 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:57.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.839+0000 7fbbe5ffb700 1 -- 172.21.15.3:0/1770044666 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2538248449 0 0) 0x7fbbe80a2a70 con 0x7fbbe809d630 2023-10-03T10:13:57.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.839+0000 7fbbe77fe700 1 -- 172.21.15.3:0/1770044666 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbbe80a0a30 msgr2=0x7fbbe80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:57.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.839+0000 7fbbe77fe700 1 --2- 172.21.15.3:0/1770044666 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbbe80a0a30 0x7fbbe80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:57.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.839+0000 7fbbe77fe700 1 -- 172.21.15.3:0/1770044666 >> v1:172.21.15.3:6789/0 conn(0x7fbbe809d630 legacy=0x7fbbe80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:57.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.839+0000 7fbbe77fe700 1 -- 172.21.15.3:0/1770044666 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe809dc50 con 0x7fbbe8138d80 2023-10-03T10:13:57.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.839+0000 7fbbe6ffd700 1 --2- 172.21.15.3:0/1770044666 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbbe80a0a30 0x7fbbe80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:57.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.839+0000 7fbbe77fe700 1 --2- 172.21.15.3:0/1770044666 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbbe8138d80 0x7fbbe813b160 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009dc0 tx=0x7fbbe000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44f14f35694a51a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:57.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.839+0000 7fbbe5ffb700 1 -- 172.21.15.3:0/1770044666 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe000e070 con 0x7fbbe8138d80 2023-10-03T10:13:57.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.839+0000 7fbbe5ffb700 1 -- 172.21.15.3:0/1770044666 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe000bb20 con 0x7fbbe8138d80 2023-10-03T10:13:57.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.839+0000 7fbbe5ffb700 1 -- 172.21.15.3:0/1770044666 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0012430 con 0x7fbbe8138d80 2023-10-03T10:13:57.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.840+0000 7fbbed705700 1 -- 172.21.15.3:0/1770044666 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbbe8138d80 msgr2=0x7fbbe813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:57.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.840+0000 7fbbed705700 1 --2- 172.21.15.3:0/1770044666 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbbe8138d80 0x7fbbe813b160 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009dc0 tx=0x7fbbe000b330 comp rx=0 tx=0).stop 2023-10-03T10:13:57.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.840+0000 7fbbed705700 1 -- 172.21.15.3:0/1770044666 shutdown_connections 2023-10-03T10:13:57.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.840+0000 7fbbed705700 1 --2- 172.21.15.3:0/1770044666 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbbe80a0a30 0x7fbbe80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:57.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.840+0000 7fbbed705700 1 --2- 172.21.15.3:0/1770044666 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbbe8138d80 0x7fbbe813b160 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:57.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.840+0000 7fbbed705700 1 -- 172.21.15.3:0/1770044666 >> 172.21.15.3:0/1770044666 conn(0x7fbbe80a3050 msgr2=0x7fbbe80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:57.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.840+0000 7fbbed705700 1 -- 172.21.15.3:0/1770044666 shutdown_connections 2023-10-03T10:13:57.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.840+0000 7fbbed705700 1 -- 172.21.15.3:0/1770044666 wait complete. 2023-10-03T10:13:57.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.841+0000 7fbbed705700 1 Processor -- start 2023-10-03T10:13:57.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.841+0000 7fbbed705700 1 -- start start 2023-10-03T10:13:57.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.841+0000 7fbbed705700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbe809d630 0x7fbbe812c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:57.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.841+0000 7fbbed705700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbbe80a0a30 0x7fbbe812ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:57.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.841+0000 7fbbed705700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbbe8138d80 0x7fbbe812cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:57.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.841+0000 7fbbed705700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe80a49e0 con 0x7fbbe809d630 2023-10-03T10:13:57.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.841+0000 7fbbed705700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe812fb00 con 0x7fbbe8138d80 2023-10-03T10:13:57.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.841+0000 7fbbed705700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe812e090 con 0x7fbbe80a0a30 2023-10-03T10:13:57.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.842+0000 7fbbe6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbe809d630 0x7fbbe812c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:57.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.842+0000 7fbbe77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbbe8138d80 0x7fbbe812cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:57.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.842+0000 7fbbe6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbe809d630 0x7fbbe812c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51460/0 (socket says 172.21.15.3:51460) 2023-10-03T10:13:57.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.842+0000 7fbbe6ffd700 1 -- 172.21.15.3:0/1016389834 learned_addr learned my addr 172.21.15.3:0/1016389834 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:57.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.842+0000 7fbbe77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbbe8138d80 0x7fbbe812cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33342/0 (socket says 172.21.15.3:33342) 2023-10-03T10:13:57.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.842+0000 7fbbe67fc700 1 --2- 172.21.15.3:0/1016389834 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbbe80a0a30 0x7fbbe812ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:57.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.842+0000 7fbbe77fe700 1 -- 172.21.15.3:0/1016389834 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbbe80a0a30 msgr2=0x7fbbe812ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:57.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.842+0000 7fbbe77fe700 1 --2- 172.21.15.3:0/1016389834 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbbe80a0a30 0x7fbbe812ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:57.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.842+0000 7fbbe77fe700 1 -- 172.21.15.3:0/1016389834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbe809d630 msgr2=0x7fbbe812c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:57.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.842+0000 7fbbe77fe700 1 --2- 172.21.15.3:0/1016389834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbe809d630 0x7fbbe812c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:57.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.842+0000 7fbbe77fe700 1 -- 172.21.15.3:0/1016389834 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd80094a0 con 0x7fbbe8138d80 2023-10-03T10:13:57.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.842+0000 7fbbe67fc700 1 --2- 172.21.15.3:0/1016389834 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbbe80a0a30 0x7fbbe812ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:57.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.842+0000 7fbbe6ffd700 1 --2- 172.21.15.3:0/1016389834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbe809d630 0x7fbbe812c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:57.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.842+0000 7fbbe77fe700 1 --2- 172.21.15.3:0/1016389834 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbbe8138d80 0x7fbbe812cf40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80672b0 tx=0x7fbbe0004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:57.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.842+0000 7fbbcf7fe700 1 -- 172.21.15.3:0/1016389834 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0012430 con 0x7fbbe8138d80 2023-10-03T10:13:57.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.842+0000 7fbbcf7fe700 1 -- 172.21.15.3:0/1016389834 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe0020070 con 0x7fbbe8138d80 2023-10-03T10:13:57.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.842+0000 7fbbcf7fe700 1 -- 172.21.15.3:0/1016389834 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0004280 con 0x7fbbe8138d80 2023-10-03T10:13:57.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.842+0000 7fbbed705700 1 -- 172.21.15.3:0/1016389834 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe0009a70 con 0x7fbbe8138d80 2023-10-03T10:13:57.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.843+0000 7fbbed705700 1 -- 172.21.15.3:0/1016389834 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe80a0460 con 0x7fbbe8138d80 2023-10-03T10:13:57.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.844+0000 7fbbcf7fe700 1 -- 172.21.15.3:0/1016389834 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbbe00043e0 con 0x7fbbe8138d80 2023-10-03T10:13:57.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.844+0000 7fbbcf7fe700 1 -- 172.21.15.3:0/1016389834 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fbbe0022070 con 0x7fbbe8138d80 2023-10-03T10:13:57.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.844+0000 7fbbed705700 1 -- 172.21.15.3:0/1016389834 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe812ede0 con 0x7fbbe8138d80 2023-10-03T10:13:57.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:57.849+0000 7fbbcf7fe700 1 -- 172.21.15.3:0/1016389834 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbbe002c080 con 0x7fbbe8138d80 2023-10-03T10:13:58.048 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:58.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.051+0000 7fbbed705700 1 -- 172.21.15.3:0/1016389834 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbbe80747f0 con 0x7fbbe8138d80 2023-10-03T10:13:58.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.052+0000 7fbbcf7fe700 1 -- 172.21.15.3:0/1016389834 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbbe001a970 con 0x7fbbe8138d80 2023-10-03T10:13:58.052 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:58.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.054+0000 7fbbed705700 1 -- 172.21.15.3:0/1016389834 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbbe8138d80 msgr2=0x7fbbe812cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:58.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.054+0000 7fbbed705700 1 --2- 172.21.15.3:0/1016389834 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbbe8138d80 0x7fbbe812cf40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80672b0 tx=0x7fbbe0004ad0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.054+0000 7fbbed705700 1 -- 172.21.15.3:0/1016389834 shutdown_connections 2023-10-03T10:13:58.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.054+0000 7fbbed705700 1 --2- 172.21.15.3:0/1016389834 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbbe8138d80 0x7fbbe812cf40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.054+0000 7fbbed705700 1 --2- 172.21.15.3:0/1016389834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbe809d630 0x7fbbe812c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.054+0000 7fbbed705700 1 --2- 172.21.15.3:0/1016389834 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbbe80a0a30 0x7fbbe812ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.054+0000 7fbbed705700 1 -- 172.21.15.3:0/1016389834 >> 172.21.15.3:0/1016389834 conn(0x7fbbe80a3050 msgr2=0x7fbbe809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:58.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.055+0000 7fbbed705700 1 -- 172.21.15.3:0/1016389834 shutdown_connections 2023-10-03T10:13:58.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.055+0000 7fbbed705700 1 -- 172.21.15.3:0/1016389834 wait complete. 2023-10-03T10:13:58.071 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:58.144 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:58.177 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:58.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.189+0000 7f07ddae6700 1 Processor -- start 2023-10-03T10:13:58.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.189+0000 7f07ddae6700 1 -- start start 2023-10-03T10:13:58.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.190+0000 7f07ddae6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07d8130540 0x7f07d8130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:58.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.190+0000 7f07ddae6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07d8130e50 0x7f07d806f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:58.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.190+0000 7f07ddae6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f07d806fb20 0x7f07d806ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:58.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.190+0000 7f07ddae6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07d809a390 con 0x7f07d806fb20 2023-10-03T10:13:58.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.190+0000 7f07ddae6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d80716a0 con 0x7f07d8130540 2023-10-03T10:13:58.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.190+0000 7f07ddae6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f07d8071290 con 0x7f07d8130e50 2023-10-03T10:13:58.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.190+0000 7f07d6ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07d8130e50 0x7f07d806f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:58.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.190+0000 7f07d7fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f07d806fb20 0x7f07d806ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:58.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.190+0000 7f07d7fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f07d806fb20 0x7f07d806ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51472/0 (socket says 172.21.15.3:51472) 2023-10-03T10:13:58.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.190+0000 7f07d6ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07d8130e50 0x7f07d806f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43802/0 (socket says 172.21.15.3:43802) 2023-10-03T10:13:58.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.190+0000 7f07d7fff700 1 -- 172.21.15.3:0/4137025457 learned_addr learned my addr 172.21.15.3:0/4137025457 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:58.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.190+0000 7f07d77fe700 1 --2- 172.21.15.3:0/4137025457 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07d8130540 0x7f07d8130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:58.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.191+0000 7f07d7fff700 1 -- 172.21.15.3:0/4137025457 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07d8130e50 msgr2=0x7f07d806f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:58.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.191+0000 7f07d7fff700 1 --2- 172.21.15.3:0/4137025457 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07d8130e50 0x7f07d806f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.191+0000 7f07d7fff700 1 -- 172.21.15.3:0/4137025457 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07d8130540 msgr2=0x7f07d8130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:58.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.191+0000 7f07d7fff700 1 --2- 172.21.15.3:0/4137025457 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07d8130540 0x7f07d8130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.191+0000 7f07d7fff700 1 -- 172.21.15.3:0/4137025457 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d8130340 con 0x7f07d806fb20 2023-10-03T10:13:58.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.191+0000 7f07d6ffd700 1 --2- 172.21.15.3:0/4137025457 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07d8130e50 0x7f07d806f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:58.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.191+0000 7f07d77fe700 1 --2- 172.21.15.3:0/4137025457 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07d8130540 0x7f07d8130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:58.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.191+0000 7f07d7fff700 1 --2- 172.21.15.3:0/4137025457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07d806fb20 0x7f07d806ffc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f07d000d180 tx=0x7f07d000d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a52d85080a6229c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:58.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.191+0000 7f07d67fc700 1 -- 172.21.15.3:0/4137025457 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d0010070 con 0x7f07d806fb20 2023-10-03T10:13:58.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.191+0000 7f07d67fc700 1 -- 172.21.15.3:0/4137025457 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d000dd00 con 0x7f07d806fb20 2023-10-03T10:13:58.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.191+0000 7f07d67fc700 1 -- 172.21.15.3:0/4137025457 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d000b310 con 0x7f07d806fb20 2023-10-03T10:13:58.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.192+0000 7f07ddae6700 1 -- 172.21.15.3:0/4137025457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07d806fb20 msgr2=0x7f07d806ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:58.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.192+0000 7f07ddae6700 1 --2- 172.21.15.3:0/4137025457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07d806fb20 0x7f07d806ffc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f07d000d180 tx=0x7f07d000d490 comp rx=0 tx=0).stop 2023-10-03T10:13:58.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.192+0000 7f07ddae6700 1 -- 172.21.15.3:0/4137025457 shutdown_connections 2023-10-03T10:13:58.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.192+0000 7f07ddae6700 1 --2- 172.21.15.3:0/4137025457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07d806fb20 0x7f07d806ffc0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.192+0000 7f07ddae6700 1 --2- 172.21.15.3:0/4137025457 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07d8130540 0x7f07d8130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.192+0000 7f07ddae6700 1 --2- 172.21.15.3:0/4137025457 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07d8130e50 0x7f07d806f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.192+0000 7f07ddae6700 1 -- 172.21.15.3:0/4137025457 >> 172.21.15.3:0/4137025457 conn(0x7f07d80a2aa0 msgr2=0x7f07d80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:58.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.192+0000 7f07ddae6700 1 -- 172.21.15.3:0/4137025457 shutdown_connections 2023-10-03T10:13:58.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.192+0000 7f07ddae6700 1 -- 172.21.15.3:0/4137025457 wait complete. 2023-10-03T10:13:58.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.193+0000 7f07ddae6700 1 Processor -- start 2023-10-03T10:13:58.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.193+0000 7f07ddae6700 1 -- start start 2023-10-03T10:13:58.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.194+0000 7f07ddae6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07d806fb20 0x7f07d811a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:58.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.194+0000 7f07ddae6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07d8130540 0x7f07d811ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:58.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.194+0000 7f07ddae6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07d8130e50 0x7f07d811b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:58.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.194+0000 7f07ddae6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d811e1a0 con 0x7f07d8130e50 2023-10-03T10:13:58.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.194+0000 7f07ddae6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d80718e0 con 0x7f07d8130540 2023-10-03T10:13:58.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.194+0000 7f07ddae6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f07d811e3a0 con 0x7f07d806fb20 2023-10-03T10:13:58.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.194+0000 7f07d6ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07d8130540 0x7f07d811ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:58.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.194+0000 7f07d7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07d8130e50 0x7f07d811b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:58.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.194+0000 7f07d6ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07d8130540 0x7f07d811ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33348/0 (socket says 172.21.15.3:33348) 2023-10-03T10:13:58.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.194+0000 7f07d6ffd700 1 -- 172.21.15.3:0/937376657 learned_addr learned my addr 172.21.15.3:0/937376657 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:58.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.194+0000 7f07d77fe700 1 --2- 172.21.15.3:0/937376657 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07d806fb20 0x7f07d811a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:58.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.194+0000 7f07d6ffd700 1 -- 172.21.15.3:0/937376657 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07d806fb20 msgr2=0x7f07d811a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:58.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.194+0000 7f07d6ffd700 1 --2- 172.21.15.3:0/937376657 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07d806fb20 0x7f07d811a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.194+0000 7f07d6ffd700 1 -- 172.21.15.3:0/937376657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07d8130e50 msgr2=0x7f07d811b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:58.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.194+0000 7f07d6ffd700 1 --2- 172.21.15.3:0/937376657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07d8130e50 0x7f07d811b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.194+0000 7f07d6ffd700 1 -- 172.21.15.3:0/937376657 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c80094a0 con 0x7f07d8130540 2023-10-03T10:13:58.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.194+0000 7f07d7fff700 1 --2- 172.21.15.3:0/937376657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07d8130e50 0x7f07d811b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:13:58.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.194+0000 7f07d77fe700 1 --2- 172.21.15.3:0/937376657 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07d806fb20 0x7f07d811a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:58.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.194+0000 7f07d6ffd700 1 --2- 172.21.15.3:0/937376657 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07d8130540 0x7f07d811ae90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f07cc00db70 tx=0x7f07cc00de80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:58.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.195+0000 7f07bffff700 1 -- 172.21.15.3:0/937376657 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc00b7e0 con 0x7f07d8130540 2023-10-03T10:13:58.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.195+0000 7f07bffff700 1 -- 172.21.15.3:0/937376657 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07cc00b940 con 0x7f07d8130540 2023-10-03T10:13:58.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.195+0000 7f07ddae6700 1 -- 172.21.15.3:0/937376657 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07cc0094a0 con 0x7f07d8130540 2023-10-03T10:13:58.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.195+0000 7f07bffff700 1 -- 172.21.15.3:0/937376657 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc00bcd0 con 0x7f07d8130540 2023-10-03T10:13:58.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.195+0000 7f07ddae6700 1 -- 172.21.15.3:0/937376657 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d000bd40 con 0x7f07d8130540 2023-10-03T10:13:58.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.196+0000 7f07bffff700 1 -- 172.21.15.3:0/937376657 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f07cc012400 con 0x7f07d8130540 2023-10-03T10:13:58.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.196+0000 7f07bffff700 1 -- 172.21.15.3:0/937376657 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f07cc03f7b0 con 0x7f07d8130540 2023-10-03T10:13:58.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.196+0000 7f07ddae6700 1 -- 172.21.15.3:0/937376657 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07d80a0060 con 0x7f07d8130540 2023-10-03T10:13:58.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.201+0000 7f07bffff700 1 -- 172.21.15.3:0/937376657 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f07cc020080 con 0x7f07d8130540 2023-10-03T10:13:58.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.290+0000 7f7385244700 1 Processor -- start 2023-10-03T10:13:58.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.290+0000 7f7385244700 1 -- start start 2023-10-03T10:13:58.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.291+0000 7f7385244700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73800a0a30 0x7f73800a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:58.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.291+0000 7f7385244700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7380138d80 0x7f738013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:58.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.291+0000 7f7385244700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73800a2a70 con 0x7f738009d630 2023-10-03T10:13:58.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.291+0000 7f7385244700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73800a01d0 con 0x7f7380138d80 2023-10-03T10:13:58.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.291+0000 7f7385244700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f738009e2b0 con 0x7f73800a0a30 2023-10-03T10:13:58.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.291+0000 7f737ed9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73800a0a30 0x7f73800a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:58.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.291+0000 7f737659c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f738009d630 0x7f73800a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45252/0 (socket says 172.21.15.3:45252) 2023-10-03T10:13:58.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.291+0000 7f737659c700 1 -- 172.21.15.3:0/927402833 learned_addr learned my addr 172.21.15.3:0/927402833 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:13:58.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.291+0000 7f737f59e700 1 --2- 172.21.15.3:0/927402833 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7380138d80 0x7f738013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:58.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.291+0000 7f737ed9d700 1 -- 172.21.15.3:0/927402833 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7380138d80 msgr2=0x7f738013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:58.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.291+0000 7f737ed9d700 1 --2- 172.21.15.3:0/927402833 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7380138d80 0x7f738013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.291+0000 7f737ed9d700 1 -- 172.21.15.3:0/927402833 >> v1:172.21.15.3:6789/0 conn(0x7f738009d630 legacy=0x7f73800a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:58.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.291+0000 7f737ed9d700 1 -- 172.21.15.3:0/927402833 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f738009dc50 con 0x7f73800a0a30 2023-10-03T10:13:58.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.291+0000 7f737e59c700 1 -- 172.21.15.3:0/927402833 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3332038115 0 0) 0x7f73800a2a70 con 0x7f738009d630 2023-10-03T10:13:58.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.291+0000 7f737ed9d700 1 --2- 172.21.15.3:0/927402833 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73800a0a30 0x7f73800a0e00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7370009870 tx=0x7f7370009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=749a000c828c28f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:58.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.292+0000 7f737e59c700 1 -- 172.21.15.3:0/927402833 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7370011830 con 0x7f73800a0a30 2023-10-03T10:13:58.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.292+0000 7f737e59c700 1 -- 172.21.15.3:0/927402833 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7370011990 con 0x7f73800a0a30 2023-10-03T10:13:58.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.292+0000 7f737e59c700 1 -- 172.21.15.3:0/927402833 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7370011d20 con 0x7f73800a0a30 2023-10-03T10:13:58.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.292+0000 7f7385244700 1 -- 172.21.15.3:0/927402833 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73800a0a30 msgr2=0x7f73800a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:58.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.292+0000 7f7385244700 1 --2- 172.21.15.3:0/927402833 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73800a0a30 0x7f73800a0e00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7370009870 tx=0x7f7370009b80 comp rx=0 tx=0).stop 2023-10-03T10:13:58.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.292+0000 7f7385244700 1 -- 172.21.15.3:0/927402833 shutdown_connections 2023-10-03T10:13:58.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.292+0000 7f7385244700 1 --2- 172.21.15.3:0/927402833 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7380138d80 0x7f738013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.292+0000 7f7385244700 1 --2- 172.21.15.3:0/927402833 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73800a0a30 0x7f73800a0e00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.292+0000 7f7385244700 1 -- 172.21.15.3:0/927402833 >> 172.21.15.3:0/927402833 conn(0x7f73800a3050 msgr2=0x7f73800a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:58.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.292+0000 7f7385244700 1 -- 172.21.15.3:0/927402833 shutdown_connections 2023-10-03T10:13:58.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.292+0000 7f7385244700 1 -- 172.21.15.3:0/927402833 wait complete. 2023-10-03T10:13:58.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.294+0000 7f7385244700 1 Processor -- start 2023-10-03T10:13:58.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.294+0000 7f7385244700 1 -- start start 2023-10-03T10:13:58.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.294+0000 7f7385244700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f738009d630 0x7f7380133760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:58.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.294+0000 7f7385244700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73800a0a30 0x7f7380133ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:58.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.294+0000 7f7385244700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7380138d80 0x7f73801341e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:58.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.294+0000 7f7385244700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7380137280 con 0x7f738009d630 2023-10-03T10:13:58.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.294+0000 7f7385244700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7380134bc0 con 0x7f7380138d80 2023-10-03T10:13:58.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.294+0000 7f7385244700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7380138a90 con 0x7f73800a0a30 2023-10-03T10:13:58.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.295+0000 7f737659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73800a0a30 0x7f7380133ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:58.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.295+0000 7f737659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73800a0a30 0x7f7380133ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43828/0 (socket says 172.21.15.3:43828) 2023-10-03T10:13:58.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.295+0000 7f737659c700 1 -- 172.21.15.3:0/1932562764 learned_addr learned my addr 172.21.15.3:0/1932562764 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:58.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.295+0000 7f737ed9d700 1 --2- 172.21.15.3:0/1932562764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f738009d630 0x7f7380133760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:58.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.295+0000 7f737f59e700 1 --2- 172.21.15.3:0/1932562764 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7380138d80 0x7f73801341e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:58.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.295+0000 7f737ed9d700 1 -- 172.21.15.3:0/1932562764 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73800a0a30 msgr2=0x7f7380133ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:58.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.295+0000 7f737ed9d700 1 --2- 172.21.15.3:0/1932562764 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73800a0a30 0x7f7380133ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.295+0000 7f737ed9d700 1 -- 172.21.15.3:0/1932562764 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7380138d80 msgr2=0x7f73801341e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:58.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.295+0000 7f737ed9d700 1 --2- 172.21.15.3:0/1932562764 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7380138d80 0x7f73801341e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.295+0000 7f737ed9d700 1 -- 172.21.15.3:0/1932562764 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73700094a0 con 0x7f738009d630 2023-10-03T10:13:58.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.295+0000 7f737659c700 1 --2- 172.21.15.3:0/1932562764 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73800a0a30 0x7f7380133ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:58.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.295+0000 7f737ed9d700 1 --2- 172.21.15.3:0/1932562764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f738009d630 0x7f7380133760 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7370016000 tx=0x7f7370004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:58.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.295+0000 7f7377fff700 1 -- 172.21.15.3:0/1932562764 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7370004450 con 0x7f738009d630 2023-10-03T10:13:58.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.296+0000 7f7377fff700 1 -- 172.21.15.3:0/1932562764 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737001f070 con 0x7f738009d630 2023-10-03T10:13:58.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.296+0000 7f7385244700 1 -- 172.21.15.3:0/1932562764 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7380136630 con 0x7f738009d630 2023-10-03T10:13:58.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.296+0000 7f7377fff700 1 -- 172.21.15.3:0/1932562764 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73700038e0 con 0x7f738009d630 2023-10-03T10:13:58.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.296+0000 7f7385244700 1 -- 172.21.15.3:0/1932562764 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7380136ce0 con 0x7f738009d630 2023-10-03T10:13:58.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.297+0000 7f7377fff700 1 -- 172.21.15.3:0/1932562764 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7370003a40 con 0x7f738009d630 2023-10-03T10:13:58.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.297+0000 7f7385244700 1 -- 172.21.15.3:0/1932562764 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f738013b620 con 0x7f738009d630 2023-10-03T10:13:58.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.297+0000 7f7377fff700 1 -- 172.21.15.3:0/1932562764 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7370021070 con 0x7f738009d630 2023-10-03T10:13:58.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.302+0000 7f7377fff700 1 -- 172.21.15.3:0/1932562764 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f73700045b0 con 0x7f738009d630 2023-10-03T10:13:58.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.335+0000 7f0c6bec0700 1 Processor -- start 2023-10-03T10:13:58.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.336+0000 7f0c6bec0700 1 -- start start 2023-10-03T10:13:58.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.336+0000 7f0c6bec0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c640a0a30 0x7f0c640a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:58.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.336+0000 7f0c6bec0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c6409d630 0x7f0c640a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:58.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.336+0000 7f0c6bec0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0c64133930 0x7f0c64133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:58.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.336+0000 7f0c6bec0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c640a2a70 con 0x7f0c64133930 2023-10-03T10:13:58.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.336+0000 7f0c6bec0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c640a01d0 con 0x7f0c640a0a30 2023-10-03T10:13:58.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.336+0000 7f0c6bec0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6409e2b0 con 0x7f0c6409d630 2023-10-03T10:13:58.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.336+0000 7f0c6945b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c6409d630 0x7f0c640a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:58.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.336+0000 7f0c6a45d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0c64133930 0x7f0c64133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:58.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.336+0000 7f0c69c5c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c640a0a30 0x7f0c640a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:58.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.336+0000 7f0c6945b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c6409d630 0x7f0c640a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43830/0 (socket says 172.21.15.3:43830) 2023-10-03T10:13:58.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.336+0000 7f0c6945b700 1 -- 172.21.15.3:0/1168571137 learned_addr learned my addr 172.21.15.3:0/1168571137 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:58.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.337+0000 7f0c6945b700 1 -- 172.21.15.3:0/1168571137 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c640a0a30 msgr2=0x7f0c640a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:58.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.337+0000 7f0c6945b700 1 --2- 172.21.15.3:0/1168571137 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c640a0a30 0x7f0c640a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.337+0000 7f0c6945b700 1 -- 172.21.15.3:0/1168571137 >> v2:172.21.15.3:3300/0 conn(0x7f0c64133930 msgr2=0x7f0c64133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:58.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.337+0000 7f0c6945b700 1 --2- 172.21.15.3:0/1168571137 >> v2:172.21.15.3:3300/0 conn(0x7f0c64133930 0x7f0c64133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.337+0000 7f0c6945b700 1 -- 172.21.15.3:0/1168571137 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6409dc50 con 0x7f0c6409d630 2023-10-03T10:13:58.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.337+0000 7f0c6a45d700 1 --2- 172.21.15.3:0/1168571137 >> v2:172.21.15.3:3300/0 conn(0x7f0c64133930 0x7f0c64133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:58.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.337+0000 7f0c6945b700 1 --2- 172.21.15.3:0/1168571137 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c6409d630 0x7f0c640a1340 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0c540098f0 tx=0x7f0c54009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22ca3071853cbc4b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:58.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.337+0000 7f0c68c5a700 1 -- 172.21.15.3:0/1168571137 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c54011830 con 0x7f0c6409d630 2023-10-03T10:13:58.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.337+0000 7f0c68c5a700 1 -- 172.21.15.3:0/1168571137 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c54011990 con 0x7f0c6409d630 2023-10-03T10:13:58.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.337+0000 7f0c68c5a700 1 -- 172.21.15.3:0/1168571137 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c54011d20 con 0x7f0c6409d630 2023-10-03T10:13:58.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.338+0000 7f0c6bec0700 1 -- 172.21.15.3:0/1168571137 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c6409d630 msgr2=0x7f0c640a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:58.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.338+0000 7f0c6bec0700 1 --2- 172.21.15.3:0/1168571137 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c6409d630 0x7f0c640a1340 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0c540098f0 tx=0x7f0c54009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:58.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.338+0000 7f0c6bec0700 1 -- 172.21.15.3:0/1168571137 shutdown_connections 2023-10-03T10:13:58.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.338+0000 7f0c6bec0700 1 --2- 172.21.15.3:0/1168571137 >> v2:172.21.15.3:3300/0 conn(0x7f0c64133930 0x7f0c64133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.338+0000 7f0c6bec0700 1 --2- 172.21.15.3:0/1168571137 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c640a0a30 0x7f0c640a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.338+0000 7f0c6bec0700 1 --2- 172.21.15.3:0/1168571137 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c6409d630 0x7f0c640a1340 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.338+0000 7f0c6bec0700 1 -- 172.21.15.3:0/1168571137 >> 172.21.15.3:0/1168571137 conn(0x7f0c640a3050 msgr2=0x7f0c640a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:58.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.338+0000 7f0c6bec0700 1 -- 172.21.15.3:0/1168571137 shutdown_connections 2023-10-03T10:13:58.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.338+0000 7f0c6bec0700 1 -- 172.21.15.3:0/1168571137 wait complete. 2023-10-03T10:13:58.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.340+0000 7f0c6bec0700 1 Processor -- start 2023-10-03T10:13:58.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.340+0000 7f0c6bec0700 1 -- start start 2023-10-03T10:13:58.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.340+0000 7f0c6bec0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c6409d630 0x7f0c6411f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:58.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.340+0000 7f0c6bec0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c640a0a30 0x7f0c6411f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:58.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.340+0000 7f0c6bec0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c64133930 0x7f0c641c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:58.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.340+0000 7f0c6bec0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c641204d0 con 0x7f0c64133930 2023-10-03T10:13:58.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.340+0000 7f0c6bec0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c64133490 con 0x7f0c6409d630 2023-10-03T10:13:58.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.340+0000 7f0c6bec0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c640a40a0 con 0x7f0c640a0a30 2023-10-03T10:13:58.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.341+0000 7f0c69c5c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c6409d630 0x7f0c6411f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:58.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.341+0000 7f0c6945b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c640a0a30 0x7f0c6411f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:58.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.341+0000 7f0c6945b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c640a0a30 0x7f0c6411f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43846/0 (socket says 172.21.15.3:43846) 2023-10-03T10:13:58.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.341+0000 7f0c69c5c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c6409d630 0x7f0c6411f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33392/0 (socket says 172.21.15.3:33392) 2023-10-03T10:13:58.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.341+0000 7f0c6945b700 1 -- 172.21.15.3:0/2257896846 learned_addr learned my addr 172.21.15.3:0/2257896846 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:58.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.341+0000 7f0c6a45d700 1 --2- 172.21.15.3:0/2257896846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c64133930 0x7f0c641c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:58.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.341+0000 7f0c6945b700 1 -- 172.21.15.3:0/2257896846 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c6409d630 msgr2=0x7f0c6411f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:58.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.341+0000 7f0c6945b700 1 --2- 172.21.15.3:0/2257896846 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c6409d630 0x7f0c6411f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.341+0000 7f0c6945b700 1 -- 172.21.15.3:0/2257896846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c64133930 msgr2=0x7f0c641c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:58.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.341+0000 7f0c6945b700 1 --2- 172.21.15.3:0/2257896846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c64133930 0x7f0c641c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.341+0000 7f0c6945b700 1 -- 172.21.15.3:0/2257896846 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5c009a70 con 0x7f0c640a0a30 2023-10-03T10:13:58.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.341+0000 7f0c6a45d700 1 --2- 172.21.15.3:0/2257896846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c64133930 0x7f0c641c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:58.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.341+0000 7f0c69c5c700 1 --2- 172.21.15.3:0/2257896846 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c6409d630 0x7f0c6411f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:58.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.341+0000 7f0c6945b700 1 --2- 172.21.15.3:0/2257896846 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c640a0a30 0x7f0c6411f6f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0c54000c00 tx=0x7f0c54004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:58.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.341+0000 7f0c5a7fc700 1 -- 172.21.15.3:0/2257896846 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c54004240 con 0x7f0c640a0a30 2023-10-03T10:13:58.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.341+0000 7f0c5a7fc700 1 -- 172.21.15.3:0/2257896846 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c5401f070 con 0x7f0c640a0a30 2023-10-03T10:13:58.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.342+0000 7f0c6bec0700 1 -- 172.21.15.3:0/2257896846 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c54009550 con 0x7f0c640a0a30 2023-10-03T10:13:58.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.342+0000 7f0c5a7fc700 1 -- 172.21.15.3:0/2257896846 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c540045c0 con 0x7f0c640a0a30 2023-10-03T10:13:58.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.342+0000 7f0c6bec0700 1 -- 172.21.15.3:0/2257896846 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c64121e40 con 0x7f0c640a0a30 2023-10-03T10:13:58.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.342+0000 7f0c6bec0700 1 -- 172.21.15.3:0/2257896846 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c64120a40 con 0x7f0c640a0a30 2023-10-03T10:13:58.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.343+0000 7f0c5a7fc700 1 -- 172.21.15.3:0/2257896846 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0c54021030 con 0x7f0c640a0a30 2023-10-03T10:13:58.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.343+0000 7f0c5a7fc700 1 -- 172.21.15.3:0/2257896846 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0c54040d90 con 0x7f0c640a0a30 2023-10-03T10:13:58.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.352+0000 7f0c5a7fc700 1 -- 172.21.15.3:0/2257896846 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0c540237e0 con 0x7f0c640a0a30 2023-10-03T10:13:58.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.421+0000 7f07ddae6700 1 -- 172.21.15.3:0/937376657 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f07d8074ea0 con 0x7f07d8130540 2023-10-03T10:13:58.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.421+0000 7f07bffff700 1 -- 172.21.15.3:0/937376657 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f07cc016020 con 0x7f07d8130540 2023-10-03T10:13:58.422 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:58.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.423+0000 7f07ddae6700 1 -- 172.21.15.3:0/937376657 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07d8130540 msgr2=0x7f07d811ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:58.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.423+0000 7f07ddae6700 1 --2- 172.21.15.3:0/937376657 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07d8130540 0x7f07d811ae90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f07cc00db70 tx=0x7f07cc00de80 comp rx=0 tx=0).stop 2023-10-03T10:13:58.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.423+0000 7f07ddae6700 1 -- 172.21.15.3:0/937376657 shutdown_connections 2023-10-03T10:13:58.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.423+0000 7f07ddae6700 1 --2- 172.21.15.3:0/937376657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07d8130e50 0x7f07d811b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.423+0000 7f07ddae6700 1 --2- 172.21.15.3:0/937376657 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07d806fb20 0x7f07d811a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.423+0000 7f07ddae6700 1 --2- 172.21.15.3:0/937376657 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07d8130540 0x7f07d811ae90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.423+0000 7f07ddae6700 1 -- 172.21.15.3:0/937376657 >> 172.21.15.3:0/937376657 conn(0x7f07d80a2aa0 msgr2=0x7f07d812d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:58.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.424+0000 7f07ddae6700 1 -- 172.21.15.3:0/937376657 shutdown_connections 2023-10-03T10:13:58.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.424+0000 7f07ddae6700 1 -- 172.21.15.3:0/937376657 wait complete. 2023-10-03T10:13:58.438 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:58.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.514+0000 7f7385244700 1 -- 172.21.15.3:0/1932562764 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f73800747f0 con 0x7f738009d630 2023-10-03T10:13:58.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.515+0000 7f7377fff700 1 -- 172.21.15.3:0/1932562764 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7370015020 con 0x7f738009d630 2023-10-03T10:13:58.515 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:58.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.517+0000 7f7385244700 1 -- 172.21.15.3:0/1932562764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f738009d630 msgr2=0x7f7380133760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:58.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.517+0000 7f7385244700 1 --2- 172.21.15.3:0/1932562764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f738009d630 0x7f7380133760 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7370016000 tx=0x7f7370004750 comp rx=0 tx=0).stop 2023-10-03T10:13:58.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.517+0000 7f7385244700 1 -- 172.21.15.3:0/1932562764 shutdown_connections 2023-10-03T10:13:58.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.517+0000 7f7385244700 1 --2- 172.21.15.3:0/1932562764 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7380138d80 0x7f73801341e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.517+0000 7f7385244700 1 --2- 172.21.15.3:0/1932562764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f738009d630 0x7f7380133760 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.517+0000 7f7385244700 1 --2- 172.21.15.3:0/1932562764 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73800a0a30 0x7f7380133ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.517+0000 7f7385244700 1 -- 172.21.15.3:0/1932562764 >> 172.21.15.3:0/1932562764 conn(0x7f73800a3050 msgr2=0x7f73800a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:58.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.517+0000 7f7385244700 1 -- 172.21.15.3:0/1932562764 shutdown_connections 2023-10-03T10:13:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.517+0000 7f7385244700 1 -- 172.21.15.3:0/1932562764 wait complete. 2023-10-03T10:13:58.531 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:58.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.560+0000 7f0c6bec0700 1 -- 172.21.15.3:0/2257896846 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0c64120cc0 con 0x7f0c640a0a30 2023-10-03T10:13:58.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.560+0000 7f0c5a7fc700 1 -- 172.21.15.3:0/2257896846 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0c54015030 con 0x7f0c640a0a30 2023-10-03T10:13:58.560 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:58.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.562+0000 7f0c6bec0700 1 -- 172.21.15.3:0/2257896846 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c640a0a30 msgr2=0x7f0c6411f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:58.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.562+0000 7f0c6bec0700 1 --2- 172.21.15.3:0/2257896846 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c640a0a30 0x7f0c6411f6f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0c54000c00 tx=0x7f0c54004750 comp rx=0 tx=0).stop 2023-10-03T10:13:58.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.562+0000 7f0c6bec0700 1 -- 172.21.15.3:0/2257896846 shutdown_connections 2023-10-03T10:13:58.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.562+0000 7f0c6bec0700 1 --2- 172.21.15.3:0/2257896846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c64133930 0x7f0c641c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.562+0000 7f0c6bec0700 1 --2- 172.21.15.3:0/2257896846 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c6409d630 0x7f0c6411f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.562+0000 7f0c6bec0700 1 --2- 172.21.15.3:0/2257896846 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c640a0a30 0x7f0c6411f6f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:58.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.562+0000 7f0c6bec0700 1 -- 172.21.15.3:0/2257896846 >> 172.21.15.3:0/2257896846 conn(0x7f0c640a3050 msgr2=0x7f0c6409ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:58.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.562+0000 7f0c6bec0700 1 -- 172.21.15.3:0/2257896846 shutdown_connections 2023-10-03T10:13:58.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:58.562+0000 7f0c6bec0700 1 -- 172.21.15.3:0/2257896846 wait complete. 2023-10-03T10:13:58.576 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:13:59.072 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:13:59.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.218+0000 7fa88623f700 1 Processor -- start 2023-10-03T10:13:59.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.218+0000 7fa88623f700 1 -- start start 2023-10-03T10:13:59.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.218+0000 7fa88623f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa88009d6f0 0x7fa88009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:59.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.218+0000 7fa88623f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa88009e000 0x7fa880130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:59.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.218+0000 7fa88623f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa88009f130 con 0x7fa88009e410 2023-10-03T10:13:59.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.218+0000 7fa88623f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa88009f520 con 0x7fa88009d6f0 2023-10-03T10:13:59.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.218+0000 7fa88623f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa88009e930 con 0x7fa88009e000 2023-10-03T10:13:59.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.219+0000 7fa87effd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa88009e000 0x7fa880130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:59.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.219+0000 7fa87f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa88009d6f0 0x7fa88009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:59.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.219+0000 7fa87effd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa88009e000 0x7fa880130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33406/0 (socket says 172.21.15.3:33406) 2023-10-03T10:13:59.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.219+0000 7fa87effd700 1 -- 172.21.15.3:0/3166276887 learned_addr learned my addr 172.21.15.3:0/3166276887 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:59.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.219+0000 7fa87effd700 1 -- 172.21.15.3:0/3166276887 >> v2:172.21.15.3:3300/0 conn(0x7fa88009d6f0 msgr2=0x7fa88009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:59.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.219+0000 7fa87effd700 1 --2- 172.21.15.3:0/3166276887 >> v2:172.21.15.3:3300/0 conn(0x7fa88009d6f0 0x7fa88009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.219+0000 7fa87effd700 1 -- 172.21.15.3:0/3166276887 >> v1:172.21.15.3:6789/0 conn(0x7fa88009e410 legacy=0x7fa880130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:59.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.219+0000 7fa87effd700 1 -- 172.21.15.3:0/3166276887 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa88009b2d0 con 0x7fa88009e000 2023-10-03T10:13:59.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.219+0000 7fa87f7fe700 1 --2- 172.21.15.3:0/3166276887 >> v2:172.21.15.3:3300/0 conn(0x7fa88009d6f0 0x7fa88009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:59.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.219+0000 7fa87effd700 1 --2- 172.21.15.3:0/3166276887 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa88009e000 0x7fa880130660 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa87400bb90 tx=0x7fa87400bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c929eb431984a933 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:59.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.219+0000 7fa87e7fc700 1 -- 172.21.15.3:0/3166276887 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87400dd30 con 0x7fa88009e000 2023-10-03T10:13:59.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.219+0000 7fa87e7fc700 1 -- 172.21.15.3:0/3166276887 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87400de90 con 0x7fa88009e000 2023-10-03T10:13:59.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.219+0000 7fa87e7fc700 1 -- 172.21.15.3:0/3166276887 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa874010690 con 0x7fa88009e000 2023-10-03T10:13:59.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.220+0000 7fa88623f700 1 -- 172.21.15.3:0/3166276887 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa88009e000 msgr2=0x7fa880130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:59.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.220+0000 7fa88623f700 1 --2- 172.21.15.3:0/3166276887 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa88009e000 0x7fa880130660 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa87400bb90 tx=0x7fa87400bea0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.220+0000 7fa88623f700 1 -- 172.21.15.3:0/3166276887 shutdown_connections 2023-10-03T10:13:59.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.220+0000 7fa88623f700 1 --2- 172.21.15.3:0/3166276887 >> v2:172.21.15.3:3300/0 conn(0x7fa88009d6f0 0x7fa88009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.220+0000 7fa88623f700 1 --2- 172.21.15.3:0/3166276887 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa88009e000 0x7fa880130660 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.220+0000 7fa88623f700 1 -- 172.21.15.3:0/3166276887 >> 172.21.15.3:0/3166276887 conn(0x7fa880066f50 msgr2=0x7fa88009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:59.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.220+0000 7fa88623f700 1 -- 172.21.15.3:0/3166276887 shutdown_connections 2023-10-03T10:13:59.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.220+0000 7fa88623f700 1 -- 172.21.15.3:0/3166276887 wait complete. 2023-10-03T10:13:59.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.221+0000 7fa88623f700 1 Processor -- start 2023-10-03T10:13:59.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.221+0000 7fa88623f700 1 -- start start 2023-10-03T10:13:59.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.222+0000 7fa88623f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa88009d6f0 0x7fa88011e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:59.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.222+0000 7fa88623f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa88009e000 0x7fa88011e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:59.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.222+0000 7fa88623f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa88009e410 0x7fa88011eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:59.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.222+0000 7fa88623f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa880121930 con 0x7fa88009e410 2023-10-03T10:13:59.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.222+0000 7fa88623f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa88009c700 con 0x7fa88009e000 2023-10-03T10:13:59.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.222+0000 7fa88623f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa880130390 con 0x7fa88009d6f0 2023-10-03T10:13:59.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.222+0000 7fa87effd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa88009e000 0x7fa88011e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:59.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.222+0000 7fa87f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa88009d6f0 0x7fa88011e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:59.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.222+0000 7fa87effd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa88009e000 0x7fa88011e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33414/0 (socket says 172.21.15.3:33414) 2023-10-03T10:13:59.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.222+0000 7fa87effd700 1 -- 172.21.15.3:0/1679004307 learned_addr learned my addr 172.21.15.3:0/1679004307 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:59.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.222+0000 7fa87ffff700 1 --2- 172.21.15.3:0/1679004307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa88009e410 0x7fa88011eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:59.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.222+0000 7fa87effd700 1 -- 172.21.15.3:0/1679004307 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa88009d6f0 msgr2=0x7fa88011e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:59.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.222+0000 7fa87effd700 1 --2- 172.21.15.3:0/1679004307 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa88009d6f0 0x7fa88011e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.222+0000 7fa87effd700 1 -- 172.21.15.3:0/1679004307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa88009e410 msgr2=0x7fa88011eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:59.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.222+0000 7fa87effd700 1 --2- 172.21.15.3:0/1679004307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa88009e410 0x7fa88011eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.222+0000 7fa87effd700 1 -- 172.21.15.3:0/1679004307 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8700094a0 con 0x7fa88009e000 2023-10-03T10:13:59.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.222+0000 7fa87f7fe700 1 --2- 172.21.15.3:0/1679004307 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa88009d6f0 0x7fa88011e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:59.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.222+0000 7fa87effd700 1 --2- 172.21.15.3:0/1679004307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa88009e000 0x7fa88011e5d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa874000c00 tx=0x7fa874004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:59.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.223+0000 7fa867fff700 1 -- 172.21.15.3:0/1679004307 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa874003b70 con 0x7fa88009e000 2023-10-03T10:13:59.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.223+0000 7fa867fff700 1 -- 172.21.15.3:0/1679004307 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa874003cd0 con 0x7fa88009e000 2023-10-03T10:13:59.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.223+0000 7fa88623f700 1 -- 172.21.15.3:0/1679004307 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa87400b7f0 con 0x7fa88009e000 2023-10-03T10:13:59.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.223+0000 7fa867fff700 1 -- 172.21.15.3:0/1679004307 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8740106b0 con 0x7fa88009e000 2023-10-03T10:13:59.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.223+0000 7fa88623f700 1 -- 172.21.15.3:0/1679004307 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa880120960 con 0x7fa88009e000 2023-10-03T10:13:59.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.224+0000 7fa88623f700 1 -- 172.21.15.3:0/1679004307 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa880074ea0 con 0x7fa88009e000 2023-10-03T10:13:59.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.224+0000 7fa867fff700 1 -- 172.21.15.3:0/1679004307 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa874019400 con 0x7fa88009e000 2023-10-03T10:13:59.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.224+0000 7fa867fff700 1 -- 172.21.15.3:0/1679004307 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa8740410e0 con 0x7fa88009e000 2023-10-03T10:13:59.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.229+0000 7fa867fff700 1 -- 172.21.15.3:0/1679004307 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa874019e60 con 0x7fa88009e000 2023-10-03T10:13:59.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.430+0000 7fa88623f700 1 -- 172.21.15.3:0/1679004307 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa88006c360 con 0x7fa88009e000 2023-10-03T10:13:59.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.431+0000 7fa867fff700 1 -- 172.21.15.3:0/1679004307 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa87400e360 con 0x7fa88009e000 2023-10-03T10:13:59.431 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:13:59.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.433+0000 7fa88623f700 1 -- 172.21.15.3:0/1679004307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa88009e000 msgr2=0x7fa88011e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:59.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.433+0000 7fa88623f700 1 --2- 172.21.15.3:0/1679004307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa88009e000 0x7fa88011e5d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa874000c00 tx=0x7fa874004750 comp rx=0 tx=0).stop 2023-10-03T10:13:59.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.433+0000 7fa88623f700 1 -- 172.21.15.3:0/1679004307 shutdown_connections 2023-10-03T10:13:59.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.433+0000 7fa88623f700 1 --2- 172.21.15.3:0/1679004307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa88009e410 0x7fa88011eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.433+0000 7fa88623f700 1 --2- 172.21.15.3:0/1679004307 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa88009d6f0 0x7fa88011e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.433+0000 7fa88623f700 1 --2- 172.21.15.3:0/1679004307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa88009e000 0x7fa88011e5d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.433+0000 7fa88623f700 1 -- 172.21.15.3:0/1679004307 >> 172.21.15.3:0/1679004307 conn(0x7fa880066f50 msgr2=0x7fa88006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:59.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.433+0000 7fa88623f700 1 -- 172.21.15.3:0/1679004307 shutdown_connections 2023-10-03T10:13:59.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.433+0000 7fa88623f700 1 -- 172.21.15.3:0/1679004307 wait complete. 2023-10-03T10:13:59.439 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:13:59.447 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:13:59.532 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:13:59.576 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:13:59.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.576+0000 7f694121d700 1 Processor -- start 2023-10-03T10:13:59.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.577+0000 7f694121d700 1 -- start start 2023-10-03T10:13:59.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.577+0000 7f694121d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f693c0a0a30 0x7f693c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:59.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.577+0000 7f694121d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f693c0a1340 0x7f693c133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:59.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.577+0000 7f694121d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f693c09edd0 con 0x7f693c133fa0 2023-10-03T10:13:59.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.577+0000 7f694121d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f693c09e1f0 con 0x7f693c0a1340 2023-10-03T10:13:59.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.577+0000 7f694121d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f693c09dc50 con 0x7f693c0a0a30 2023-10-03T10:13:59.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.577+0000 7f693ad9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f693c0a0a30 0x7f693c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:59.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.577+0000 7f693ad9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f693c0a0a30 0x7f693c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43864/0 (socket says 172.21.15.3:43864) 2023-10-03T10:13:59.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.577+0000 7f693ad9d700 1 -- 172.21.15.3:0/3649452532 learned_addr learned my addr 172.21.15.3:0/3649452532 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:59.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.577+0000 7f693a59c700 1 --2- 172.21.15.3:0/3649452532 >> v2:172.21.15.3:3300/0 conn(0x7f693c0a1340 0x7f693c133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:59.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.578+0000 7f693ad9d700 1 -- 172.21.15.3:0/3649452532 >> v2:172.21.15.3:3300/0 conn(0x7f693c0a1340 msgr2=0x7f693c133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:59.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.578+0000 7f693ad9d700 1 --2- 172.21.15.3:0/3649452532 >> v2:172.21.15.3:3300/0 conn(0x7f693c0a1340 0x7f693c133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.578+0000 7f693ad9d700 1 -- 172.21.15.3:0/3649452532 >> v1:172.21.15.3:6789/0 conn(0x7f693c133fa0 legacy=0x7f693c13ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:59.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.578+0000 7f693ad9d700 1 -- 172.21.15.3:0/3649452532 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f693c0a1fe0 con 0x7f693c0a0a30 2023-10-03T10:13:59.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.578+0000 7f693a59c700 1 --2- 172.21.15.3:0/3649452532 >> v2:172.21.15.3:3300/0 conn(0x7f693c0a1340 0x7f693c133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:59.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.578+0000 7f693ad9d700 1 --2- 172.21.15.3:0/3649452532 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f693c0a0a30 0x7f693c0a0e00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f692c009e50 tx=0x7f692c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0ac826dd89ec036 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:59.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.578+0000 7f6939d9b700 1 -- 172.21.15.3:0/3649452532 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692c00e070 con 0x7f693c0a0a30 2023-10-03T10:13:59.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.578+0000 7f6939d9b700 1 -- 172.21.15.3:0/3649452532 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f692c00b910 con 0x7f693c0a0a30 2023-10-03T10:13:59.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.578+0000 7f6939d9b700 1 -- 172.21.15.3:0/3649452532 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692c00bcc0 con 0x7f693c0a0a30 2023-10-03T10:13:59.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.579+0000 7f694121d700 1 -- 172.21.15.3:0/3649452532 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f693c0a0a30 msgr2=0x7f693c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.579+0000 7f694121d700 1 --2- 172.21.15.3:0/3649452532 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f693c0a0a30 0x7f693c0a0e00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f692c009e50 tx=0x7f692c00b330 comp rx=0 tx=0).stop 2023-10-03T10:13:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.579+0000 7f694121d700 1 -- 172.21.15.3:0/3649452532 shutdown_connections 2023-10-03T10:13:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.579+0000 7f694121d700 1 --2- 172.21.15.3:0/3649452532 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f693c0a0a30 0x7f693c0a0e00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.579+0000 7f694121d700 1 --2- 172.21.15.3:0/3649452532 >> v2:172.21.15.3:3300/0 conn(0x7f693c0a1340 0x7f693c133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.579+0000 7f694121d700 1 -- 172.21.15.3:0/3649452532 >> 172.21.15.3:0/3649452532 conn(0x7f693c0a6550 msgr2=0x7f693c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.579+0000 7f694121d700 1 -- 172.21.15.3:0/3649452532 shutdown_connections 2023-10-03T10:13:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.579+0000 7f694121d700 1 -- 172.21.15.3:0/3649452532 wait complete. 2023-10-03T10:13:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.580+0000 7f694121d700 1 Processor -- start 2023-10-03T10:13:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.580+0000 7f694121d700 1 -- start start 2023-10-03T10:13:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.581+0000 7f694121d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f693c0a0a30 0x7f693c11e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.581+0000 7f694121d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f693c0a1340 0x7f693c11ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.581+0000 7f694121d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f693c133fa0 0x7f693c11f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.581+0000 7f694121d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693c121f50 con 0x7f693c0a0a30 2023-10-03T10:13:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.581+0000 7f694121d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f693c120430 con 0x7f693c0a1340 2023-10-03T10:13:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.581+0000 7f694121d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f693c0a37e0 con 0x7f693c133fa0 2023-10-03T10:13:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.581+0000 7f693ad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f693c0a0a30 0x7f693c11e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.581+0000 7f693a59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f693c0a1340 0x7f693c11ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.581+0000 7f693ad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f693c0a0a30 0x7f693c11e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51542/0 (socket says 172.21.15.3:51542) 2023-10-03T10:13:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.581+0000 7f693ad9d700 1 -- 172.21.15.3:0/2670253911 learned_addr learned my addr 172.21.15.3:0/2670253911 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.581+0000 7f693b59e700 1 --2- 172.21.15.3:0/2670253911 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f693c133fa0 0x7f693c11f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.581+0000 7f693ad9d700 1 -- 172.21.15.3:0/2670253911 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f693c133fa0 msgr2=0x7f693c11f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.581+0000 7f693ad9d700 1 --2- 172.21.15.3:0/2670253911 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f693c133fa0 0x7f693c11f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.582+0000 7f693ad9d700 1 -- 172.21.15.3:0/2670253911 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f693c0a1340 msgr2=0x7f693c11ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:59.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.582+0000 7f693ad9d700 1 --2- 172.21.15.3:0/2670253911 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f693c0a1340 0x7f693c11ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.582+0000 7f693ad9d700 1 -- 172.21.15.3:0/2670253911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69240094a0 con 0x7f693c0a0a30 2023-10-03T10:13:59.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.582+0000 7f693a59c700 1 --2- 172.21.15.3:0/2670253911 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f693c0a1340 0x7f693c11ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:59.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.582+0000 7f693ad9d700 1 --2- 172.21.15.3:0/2670253911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f693c0a0a30 0x7f693c11e7a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f692c00b370 tx=0x7f692c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:59.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.582+0000 7f69337fe700 1 -- 172.21.15.3:0/2670253911 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692c004800 con 0x7f693c0a0a30 2023-10-03T10:13:59.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.582+0000 7f69337fe700 1 -- 172.21.15.3:0/2670253911 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f692c0092e0 con 0x7f693c0a0a30 2023-10-03T10:13:59.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.582+0000 7f694121d700 1 -- 172.21.15.3:0/2670253911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f692c009b30 con 0x7f693c0a0a30 2023-10-03T10:13:59.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.582+0000 7f69337fe700 1 -- 172.21.15.3:0/2670253911 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692c01c680 con 0x7f693c0a0a30 2023-10-03T10:13:59.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.582+0000 7f694121d700 1 -- 172.21.15.3:0/2670253911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f693c120ad0 con 0x7f693c0a0a30 2023-10-03T10:13:59.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.583+0000 7f69337fe700 1 -- 172.21.15.3:0/2670253911 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f692c01c7e0 con 0x7f693c0a0a30 2023-10-03T10:13:59.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.583+0000 7f694121d700 1 -- 172.21.15.3:0/2670253911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f693c074ea0 con 0x7f693c0a0a30 2023-10-03T10:13:59.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.583+0000 7f69337fe700 1 -- 172.21.15.3:0/2670253911 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f692c041260 con 0x7f693c0a0a30 2023-10-03T10:13:59.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.588+0000 7f69337fe700 1 -- 172.21.15.3:0/2670253911 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f692c025450 con 0x7f693c0a0a30 2023-10-03T10:13:59.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.686+0000 7f6bb2fce700 1 Processor -- start 2023-10-03T10:13:59.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.686+0000 7f6bb2fce700 1 -- start start 2023-10-03T10:13:59.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.686+0000 7f6bb2fce700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6bac09e850 0x7f6bac09ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:59.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.686+0000 7f6bb2fce700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6bac09f160 0x7f6bac133bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:59.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.686+0000 7f6bb2fce700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6bac09f5b0 0x7f6bac134130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:59.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.687+0000 7f6bb2fce700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bac06bf30 con 0x7f6bac09e850 2023-10-03T10:13:59.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.687+0000 7f6bb2fce700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac09ffb0 con 0x7f6bac09f160 2023-10-03T10:13:59.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.687+0000 7f6bb2fce700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac06cb10 con 0x7f6bac09f5b0 2023-10-03T10:13:59.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.687+0000 7f6bb0d6a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6bac09e850 0x7f6bac09ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:59.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.687+0000 7f6babfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6bac09f160 0x7f6bac133bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:59.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.687+0000 7f6bb0d6a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6bac09e850 0x7f6bac09ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51556/0 (socket says 172.21.15.3:51556) 2023-10-03T10:13:59.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.687+0000 7f6babfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6bac09f160 0x7f6bac133bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33432/0 (socket says 172.21.15.3:33432) 2023-10-03T10:13:59.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.687+0000 7f6bb0d6a700 1 -- 172.21.15.3:0/3230078678 learned_addr learned my addr 172.21.15.3:0/3230078678 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:59.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.687+0000 7f6bb156b700 1 --2- 172.21.15.3:0/3230078678 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6bac09f5b0 0x7f6bac134130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:59.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.687+0000 7f6bb0d6a700 1 -- 172.21.15.3:0/3230078678 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6bac09f5b0 msgr2=0x7f6bac134130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:59.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.687+0000 7f6bb0d6a700 1 --2- 172.21.15.3:0/3230078678 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6bac09f5b0 0x7f6bac134130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.687+0000 7f6bb0d6a700 1 -- 172.21.15.3:0/3230078678 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6bac09f160 msgr2=0x7f6bac133bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:59.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.687+0000 7f6bb0d6a700 1 --2- 172.21.15.3:0/3230078678 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6bac09f160 0x7f6bac133bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.687+0000 7f6bb0d6a700 1 -- 172.21.15.3:0/3230078678 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac0a0270 con 0x7f6bac09e850 2023-10-03T10:13:59.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.687+0000 7f6bb0d6a700 1 --2- 172.21.15.3:0/3230078678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bac09e850 0x7f6bac09ec20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00098f0 tx=0x7f6ba0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b0822f79a5ae1e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:59.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.688+0000 7f6bab7fe700 1 -- 172.21.15.3:0/3230078678 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0011830 con 0x7f6bac09e850 2023-10-03T10:13:59.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.688+0000 7f6bab7fe700 1 -- 172.21.15.3:0/3230078678 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba0011990 con 0x7f6bac09e850 2023-10-03T10:13:59.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.688+0000 7f6bab7fe700 1 -- 172.21.15.3:0/3230078678 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0011d20 con 0x7f6bac09e850 2023-10-03T10:13:59.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.688+0000 7f6bb2fce700 1 -- 172.21.15.3:0/3230078678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bac09e850 msgr2=0x7f6bac09ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:59.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.688+0000 7f6bb2fce700 1 --2- 172.21.15.3:0/3230078678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bac09e850 0x7f6bac09ec20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00098f0 tx=0x7f6ba0009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:59.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.688+0000 7f6bb2fce700 1 -- 172.21.15.3:0/3230078678 shutdown_connections 2023-10-03T10:13:59.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.688+0000 7f6bb2fce700 1 --2- 172.21.15.3:0/3230078678 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6bac09f5b0 0x7f6bac134130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.688+0000 7f6bb2fce700 1 --2- 172.21.15.3:0/3230078678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bac09e850 0x7f6bac09ec20 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.688+0000 7f6bb2fce700 1 --2- 172.21.15.3:0/3230078678 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6bac09f160 0x7f6bac133bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.688+0000 7f6bb2fce700 1 -- 172.21.15.3:0/3230078678 >> 172.21.15.3:0/3230078678 conn(0x7f6bac0a3f70 msgr2=0x7f6bac0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:59.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.688+0000 7f6bb2fce700 1 -- 172.21.15.3:0/3230078678 shutdown_connections 2023-10-03T10:13:59.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.688+0000 7f6bb2fce700 1 -- 172.21.15.3:0/3230078678 wait complete. 2023-10-03T10:13:59.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.689+0000 7f6bb2fce700 1 Processor -- start 2023-10-03T10:13:59.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.689+0000 7f6bb2fce700 1 -- start start 2023-10-03T10:13:59.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.690+0000 7f6bb2fce700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6bac09e850 0x7f6bac1c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:59.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.690+0000 7f6bb2fce700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bac09f160 0x7f6bac1c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:59.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.690+0000 7f6bb2fce700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6bac09f5b0 0x7f6bac1c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:59.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.690+0000 7f6bb2fce700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac124970 con 0x7f6bac09f160 2023-10-03T10:13:59.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.690+0000 7f6bb2fce700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac125240 con 0x7f6bac09f5b0 2023-10-03T10:13:59.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.690+0000 7f6bb2fce700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac126880 con 0x7f6bac09e850 2023-10-03T10:13:59.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.690+0000 7f6babfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bac09f160 0x7f6bac1c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:59.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.690+0000 7f6bb156b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6bac09f5b0 0x7f6bac1c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:59.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.690+0000 7f6babfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bac09f160 0x7f6bac1c8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51558/0 (socket says 172.21.15.3:51558) 2023-10-03T10:13:59.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.690+0000 7f6bb156b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6bac09f5b0 0x7f6bac1c9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33448/0 (socket says 172.21.15.3:33448) 2023-10-03T10:13:59.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.690+0000 7f6babfff700 1 -- 172.21.15.3:0/1394669863 learned_addr learned my addr 172.21.15.3:0/1394669863 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:59.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.690+0000 7f6bb0d6a700 1 --2- 172.21.15.3:0/1394669863 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6bac09e850 0x7f6bac1c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:59.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.691+0000 7f6bb156b700 1 -- 172.21.15.3:0/1394669863 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6bac09e850 msgr2=0x7f6bac1c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:59.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.691+0000 7f6bb156b700 1 --2- 172.21.15.3:0/1394669863 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6bac09e850 0x7f6bac1c8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.691+0000 7f6bb156b700 1 -- 172.21.15.3:0/1394669863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bac09f160 msgr2=0x7f6bac1c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:59.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.691+0000 7f6bb156b700 1 --2- 172.21.15.3:0/1394669863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bac09f160 0x7f6bac1c8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.691+0000 7f6bb156b700 1 -- 172.21.15.3:0/1394669863 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba0009550 con 0x7f6bac09f5b0 2023-10-03T10:13:59.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.691+0000 7f6bb0d6a700 1 --2- 172.21.15.3:0/1394669863 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6bac09e850 0x7f6bac1c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:59.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.691+0000 7f6babfff700 1 --2- 172.21.15.3:0/1394669863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bac09f160 0x7f6bac1c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:13:59.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.691+0000 7f6bb156b700 1 --2- 172.21.15.3:0/1394669863 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6bac09f5b0 0x7f6bac1c9390 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400efc0 tx=0x7f6ba400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:59.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.691+0000 7f6ba97fa700 1 -- 172.21.15.3:0/1394669863 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba400cc90 con 0x7f6bac09f5b0 2023-10-03T10:13:59.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.691+0000 7f6ba97fa700 1 -- 172.21.15.3:0/1394669863 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba400f040 con 0x7f6bac09f5b0 2023-10-03T10:13:59.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.691+0000 7f6bb2fce700 1 -- 172.21.15.3:0/1394669863 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bac06f8a0 con 0x7f6bac09f5b0 2023-10-03T10:13:59.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.691+0000 7f6ba97fa700 1 -- 172.21.15.3:0/1394669863 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4009e60 con 0x7f6bac09f5b0 2023-10-03T10:13:59.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.691+0000 7f6bb2fce700 1 -- 172.21.15.3:0/1394669863 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bac124ee0 con 0x7f6bac09f5b0 2023-10-03T10:13:59.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.692+0000 7f6ba97fa700 1 -- 172.21.15.3:0/1394669863 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6ba4014480 con 0x7f6bac09f5b0 2023-10-03T10:13:59.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.692+0000 7f6ba97fa700 1 -- 172.21.15.3:0/1394669863 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6ba40401a0 con 0x7f6bac09f5b0 2023-10-03T10:13:59.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.693+0000 7f6bb2fce700 1 -- 172.21.15.3:0/1394669863 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bac0747f0 con 0x7f6bac09f5b0 2023-10-03T10:13:59.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.697+0000 7f6ba97fa700 1 -- 172.21.15.3:0/1394669863 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6ba4018020 con 0x7f6bac09f5b0 2023-10-03T10:13:59.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.717+0000 7f35e108b700 1 Processor -- start 2023-10-03T10:13:59.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.717+0000 7f35e108b700 1 -- start start 2023-10-03T10:13:59.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.718+0000 7f35e108b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f35dc09e850 0x7f35dc09ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:59.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.718+0000 7f35e108b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35dc09f160 0x7f35dc133c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:59.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.718+0000 7f35e108b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35dc06bec0 con 0x7f35dc09f5b0 2023-10-03T10:13:59.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.718+0000 7f35e108b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35dc0a0030 con 0x7f35dc09e850 2023-10-03T10:13:59.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.718+0000 7f35e108b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc0a0830 con 0x7f35dc09f160 2023-10-03T10:13:59.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.718+0000 7f35da59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35dc09f160 0x7f35dc133c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:59.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.718+0000 7f35da59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35dc09f160 0x7f35dc133c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43902/0 (socket says 172.21.15.3:43902) 2023-10-03T10:13:59.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.718+0000 7f35da59c700 1 -- 172.21.15.3:0/2897674091 learned_addr learned my addr 172.21.15.3:0/2897674091 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:59.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.718+0000 7f35dad9d700 1 --2- 172.21.15.3:0/2897674091 >> v2:172.21.15.3:3300/0 conn(0x7f35dc09e850 0x7f35dc09ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:59.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.718+0000 7f35da59c700 1 -- 172.21.15.3:0/2897674091 >> v2:172.21.15.3:3300/0 conn(0x7f35dc09e850 msgr2=0x7f35dc09ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:13:59.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.718+0000 7f35da59c700 1 --2- 172.21.15.3:0/2897674091 >> v2:172.21.15.3:3300/0 conn(0x7f35dc09e850 0x7f35dc09ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.718+0000 7f35da59c700 1 -- 172.21.15.3:0/2897674091 >> v1:172.21.15.3:6789/0 conn(0x7f35dc09f5b0 legacy=0x7f35dc1341a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:59.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.718+0000 7f35da59c700 1 -- 172.21.15.3:0/2897674091 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35dc06caa0 con 0x7f35dc09f160 2023-10-03T10:13:59.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.718+0000 7f35d9d9b700 1 -- 172.21.15.3:0/2897674091 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592024275 0 0) 0x7f35dc06bec0 con 0x7f35dc09f5b0 2023-10-03T10:13:59.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.719+0000 7f35da59c700 1 --2- 172.21.15.3:0/2897674091 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35dc09f160 0x7f35dc133c60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f35c40098f0 tx=0x7f35c4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ea3efd77f2064f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:59.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.719+0000 7f35d9d9b700 1 -- 172.21.15.3:0/2897674091 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c4011830 con 0x7f35dc09f160 2023-10-03T10:13:59.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.719+0000 7f35d9d9b700 1 -- 172.21.15.3:0/2897674091 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c4011990 con 0x7f35dc09f160 2023-10-03T10:13:59.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.719+0000 7f35d9d9b700 1 -- 172.21.15.3:0/2897674091 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c4011d20 con 0x7f35dc09f160 2023-10-03T10:13:59.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.719+0000 7f35e108b700 1 -- 172.21.15.3:0/2897674091 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35dc09f160 msgr2=0x7f35dc133c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:59.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.719+0000 7f35e108b700 1 --2- 172.21.15.3:0/2897674091 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35dc09f160 0x7f35dc133c60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f35c40098f0 tx=0x7f35c4009c00 comp rx=0 tx=0).stop 2023-10-03T10:13:59.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.719+0000 7f35e108b700 1 -- 172.21.15.3:0/2897674091 shutdown_connections 2023-10-03T10:13:59.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.719+0000 7f35e108b700 1 --2- 172.21.15.3:0/2897674091 >> v2:172.21.15.3:3300/0 conn(0x7f35dc09e850 0x7f35dc09ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.719+0000 7f35e108b700 1 --2- 172.21.15.3:0/2897674091 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35dc09f160 0x7f35dc133c60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.719+0000 7f35e108b700 1 -- 172.21.15.3:0/2897674091 >> 172.21.15.3:0/2897674091 conn(0x7f35dc0a3f70 msgr2=0x7f35dc0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:59.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.719+0000 7f35e108b700 1 -- 172.21.15.3:0/2897674091 shutdown_connections 2023-10-03T10:13:59.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.719+0000 7f35e108b700 1 -- 172.21.15.3:0/2897674091 wait complete. 2023-10-03T10:13:59.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.721+0000 7f35e108b700 1 Processor -- start 2023-10-03T10:13:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.721+0000 7f35e108b700 1 -- start start 2023-10-03T10:13:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.721+0000 7f35e108b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35dc09e850 0x7f35dc1203e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.721+0000 7f35e108b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35dc09f160 0x7f35dc120920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.721+0000 7f35e108b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35dc09f5b0 0x7f35dc120e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:13:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.721+0000 7f35e108b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc123600 con 0x7f35dc09e850 2023-10-03T10:13:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.721+0000 7f35e108b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc06ccc0 con 0x7f35dc09f5b0 2023-10-03T10:13:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.721+0000 7f35e108b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc0a3350 con 0x7f35dc09f160 2023-10-03T10:13:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.721+0000 7f35dad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35dc09e850 0x7f35dc1203e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.722+0000 7f35da59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35dc09f160 0x7f35dc120920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.722+0000 7f35dad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35dc09e850 0x7f35dc1203e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51568/0 (socket says 172.21.15.3:51568) 2023-10-03T10:13:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.722+0000 7f35dad9d700 1 -- 172.21.15.3:0/217781591 learned_addr learned my addr 172.21.15.3:0/217781591 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:13:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.722+0000 7f35db59e700 1 --2- 172.21.15.3:0/217781591 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35dc09f5b0 0x7f35dc120e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:13:59.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.722+0000 7f35dad9d700 1 -- 172.21.15.3:0/217781591 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35dc09f160 msgr2=0x7f35dc120920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:59.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.722+0000 7f35dad9d700 1 --2- 172.21.15.3:0/217781591 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35dc09f160 0x7f35dc120920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.722+0000 7f35dad9d700 1 -- 172.21.15.3:0/217781591 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35dc09f5b0 msgr2=0x7f35dc120e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:59.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.722+0000 7f35dad9d700 1 --2- 172.21.15.3:0/217781591 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35dc09f5b0 0x7f35dc120e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.722+0000 7f35dad9d700 1 -- 172.21.15.3:0/217781591 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c4009550 con 0x7f35dc09e850 2023-10-03T10:13:59.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.722+0000 7f35dad9d700 1 --2- 172.21.15.3:0/217781591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35dc09e850 0x7f35dc1203e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00efc0 tx=0x7f35cc00c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:13:59.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.722+0000 7f35d37fe700 1 -- 172.21.15.3:0/217781591 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc00cb50 con 0x7f35dc09e850 2023-10-03T10:13:59.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.722+0000 7f35d37fe700 1 -- 172.21.15.3:0/217781591 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35cc00f040 con 0x7f35dc09e850 2023-10-03T10:13:59.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.722+0000 7f35d37fe700 1 -- 172.21.15.3:0/217781591 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc014480 con 0x7f35dc09e850 2023-10-03T10:13:59.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.722+0000 7f35e108b700 1 -- 172.21.15.3:0/217781591 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35dc121f60 con 0x7f35dc09e850 2023-10-03T10:13:59.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.722+0000 7f35e108b700 1 -- 172.21.15.3:0/217781591 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35dc124370 con 0x7f35dc09e850 2023-10-03T10:13:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.723+0000 7f35d37fe700 1 -- 172.21.15.3:0/217781591 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f35cc0145e0 con 0x7f35dc09e850 2023-10-03T10:13:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.723+0000 7f35d37fe700 1 -- 172.21.15.3:0/217781591 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f35cc03fec0 con 0x7f35dc09e850 2023-10-03T10:13:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.724+0000 7f35e108b700 1 -- 172.21.15.3:0/217781591 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35dc09caf0 con 0x7f35dc09e850 2023-10-03T10:13:59.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.729+0000 7f35d37fe700 1 -- 172.21.15.3:0/217781591 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f35cc014890 con 0x7f35dc09e850 2023-10-03T10:13:59.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.842+0000 7f694121d700 1 -- 172.21.15.3:0/2670253911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f693c0a28e0 con 0x7f693c0a0a30 2023-10-03T10:13:59.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.843+0000 7f69337fe700 1 -- 172.21.15.3:0/2670253911 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f692c02d770 con 0x7f693c0a0a30 2023-10-03T10:13:59.843 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:13:59.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.845+0000 7f694121d700 1 -- 172.21.15.3:0/2670253911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f693c0a0a30 msgr2=0x7f693c11e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:59.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.845+0000 7f694121d700 1 --2- 172.21.15.3:0/2670253911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f693c0a0a30 0x7f693c11e7a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f692c00b370 tx=0x7f692c004ad0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.845+0000 7f694121d700 1 -- 172.21.15.3:0/2670253911 shutdown_connections 2023-10-03T10:13:59.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.845+0000 7f694121d700 1 --2- 172.21.15.3:0/2670253911 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f693c133fa0 0x7f693c11f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.845+0000 7f694121d700 1 --2- 172.21.15.3:0/2670253911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f693c0a0a30 0x7f693c11e7a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.845+0000 7f694121d700 1 --2- 172.21.15.3:0/2670253911 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f693c0a1340 0x7f693c11ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.845+0000 7f694121d700 1 -- 172.21.15.3:0/2670253911 >> 172.21.15.3:0/2670253911 conn(0x7f693c0a6550 msgr2=0x7f693c1b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:59.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.845+0000 7f694121d700 1 -- 172.21.15.3:0/2670253911 shutdown_connections 2023-10-03T10:13:59.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.845+0000 7f694121d700 1 -- 172.21.15.3:0/2670253911 wait complete. 2023-10-03T10:13:59.859 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:13:59.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.928+0000 7f6bb2fce700 1 -- 172.21.15.3:0/1394669863 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6bac06bbb0 con 0x7f6bac09f5b0 2023-10-03T10:13:59.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.929+0000 7f6ba97fa700 1 -- 172.21.15.3:0/1394669863 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6ba402b090 con 0x7f6bac09f5b0 2023-10-03T10:13:59.929 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:13:59.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.931+0000 7f6bb2fce700 1 -- 172.21.15.3:0/1394669863 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6bac09f5b0 msgr2=0x7f6bac1c9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:59.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.931+0000 7f6bb2fce700 1 --2- 172.21.15.3:0/1394669863 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6bac09f5b0 0x7f6bac1c9390 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400efc0 tx=0x7f6ba400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.931+0000 7f6bb2fce700 1 -- 172.21.15.3:0/1394669863 shutdown_connections 2023-10-03T10:13:59.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.931+0000 7f6bb2fce700 1 --2- 172.21.15.3:0/1394669863 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6bac09f5b0 0x7f6bac1c9390 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.931+0000 7f6bb2fce700 1 --2- 172.21.15.3:0/1394669863 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6bac09e850 0x7f6bac1c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.931+0000 7f6bb2fce700 1 --2- 172.21.15.3:0/1394669863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bac09f160 0x7f6bac1c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.931+0000 7f6bb2fce700 1 -- 172.21.15.3:0/1394669863 >> 172.21.15.3:0/1394669863 conn(0x7f6bac0a3f70 msgr2=0x7f6bac071130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:59.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.931+0000 7f6bb2fce700 1 -- 172.21.15.3:0/1394669863 shutdown_connections 2023-10-03T10:13:59.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.931+0000 7f6bb2fce700 1 -- 172.21.15.3:0/1394669863 wait complete. 2023-10-03T10:13:59.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.937+0000 7f35e108b700 1 -- 172.21.15.3:0/217781591 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f35dc123830 con 0x7f35dc09e850 2023-10-03T10:13:59.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.937+0000 7f35d37fe700 1 -- 172.21.15.3:0/217781591 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f35cc02b090 con 0x7f35dc09e850 2023-10-03T10:13:59.938 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:13:59.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.939+0000 7f35e108b700 1 -- 172.21.15.3:0/217781591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35dc09e850 msgr2=0x7f35dc1203e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:13:59.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.939+0000 7f35e108b700 1 --2- 172.21.15.3:0/217781591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35dc09e850 0x7f35dc1203e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00efc0 tx=0x7f35cc00c0c0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.939+0000 7f35e108b700 1 -- 172.21.15.3:0/217781591 shutdown_connections 2023-10-03T10:13:59.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.939+0000 7f35e108b700 1 --2- 172.21.15.3:0/217781591 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35dc09f5b0 0x7f35dc120e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.939+0000 7f35e108b700 1 --2- 172.21.15.3:0/217781591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35dc09e850 0x7f35dc1203e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.939+0000 7f35e108b700 1 --2- 172.21.15.3:0/217781591 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35dc09f160 0x7f35dc120920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:13:59.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.939+0000 7f35e108b700 1 -- 172.21.15.3:0/217781591 >> 172.21.15.3:0/217781591 conn(0x7f35dc0a3f70 msgr2=0x7f35dc1bb220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:13:59.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.940+0000 7f35e108b700 1 -- 172.21.15.3:0/217781591 shutdown_connections 2023-10-03T10:13:59.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:13:59.940+0000 7f35e108b700 1 -- 172.21.15.3:0/217781591 wait complete. 2023-10-03T10:13:59.947 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:13:59.954 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:00.448 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:00.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.596+0000 7f5c5b481700 1 Processor -- start 2023-10-03T10:14:00.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.597+0000 7f5c5b481700 1 -- start start 2023-10-03T10:14:00.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.597+0000 7f5c5b481700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c5409e930 0x7f5c5409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:00.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.597+0000 7f5c5b481700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c5409f650 0x7f5c54131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:00.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.597+0000 7f5c5b481700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c54070940 con 0x7f5c5409f240 2023-10-03T10:14:00.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.597+0000 7f5c5b481700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c5406fd60 con 0x7f5c5409e930 2023-10-03T10:14:00.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.597+0000 7f5c5b481700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5406fa80 con 0x7f5c5409f650 2023-10-03T10:14:00.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.597+0000 7f5c5921d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c5409e930 0x7f5c5409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:00.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.597+0000 7f5c59a1e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c5409f650 0x7f5c54131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:00.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.597+0000 7f5c59a1e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c5409f650 0x7f5c54131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43920/0 (socket says 172.21.15.3:43920) 2023-10-03T10:14:00.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.597+0000 7f5c5921d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c5409e930 0x7f5c5409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51584/0 (socket says 172.21.15.3:51584) 2023-10-03T10:14:00.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.597+0000 7f5c59a1e700 1 -- 172.21.15.3:0/3237045835 learned_addr learned my addr 172.21.15.3:0/3237045835 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:00.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.598+0000 7f5c59a1e700 1 -- 172.21.15.3:0/3237045835 >> v2:172.21.15.3:3300/0 conn(0x7f5c5409e930 msgr2=0x7f5c5409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:00.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.598+0000 7f5c59a1e700 1 --2- 172.21.15.3:0/3237045835 >> v2:172.21.15.3:3300/0 conn(0x7f5c5409e930 0x7f5c5409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:00.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.598+0000 7f5c59a1e700 1 -- 172.21.15.3:0/3237045835 >> v1:172.21.15.3:6789/0 conn(0x7f5c5409f240 legacy=0x7f5c541317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:00.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.598+0000 7f5c59a1e700 1 -- 172.21.15.3:0/3237045835 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5409fb60 con 0x7f5c5409f650 2023-10-03T10:14:00.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.598+0000 7f5c47fff700 1 -- 172.21.15.3:0/3237045835 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3870586867 0 0) 0x7f5c54070940 con 0x7f5c5409f240 2023-10-03T10:14:00.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.598+0000 7f5c5921d700 1 --2- 172.21.15.3:0/3237045835 >> v2:172.21.15.3:3300/0 conn(0x7f5c5409e930 0x7f5c5409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:00.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.598+0000 7f5c59a1e700 1 --2- 172.21.15.3:0/3237045835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c5409f650 0x7f5c54131f00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009e50 tx=0x7f5c4c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71713c6b3ca8362f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:00.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.598+0000 7f5c47fff700 1 -- 172.21.15.3:0/3237045835 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c00e070 con 0x7f5c5409f650 2023-10-03T10:14:00.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.598+0000 7f5c47fff700 1 -- 172.21.15.3:0/3237045835 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c4c00b910 con 0x7f5c5409f650 2023-10-03T10:14:00.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.598+0000 7f5c47fff700 1 -- 172.21.15.3:0/3237045835 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c00bcc0 con 0x7f5c5409f650 2023-10-03T10:14:00.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.599+0000 7f5c5b481700 1 -- 172.21.15.3:0/3237045835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c5409f650 msgr2=0x7f5c54131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:00.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.599+0000 7f5c5b481700 1 --2- 172.21.15.3:0/3237045835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c5409f650 0x7f5c54131f00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009e50 tx=0x7f5c4c00b330 comp rx=0 tx=0).stop 2023-10-03T10:14:00.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.599+0000 7f5c5b481700 1 -- 172.21.15.3:0/3237045835 shutdown_connections 2023-10-03T10:14:00.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.599+0000 7f5c5b481700 1 --2- 172.21.15.3:0/3237045835 >> v2:172.21.15.3:3300/0 conn(0x7f5c5409e930 0x7f5c5409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:00.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.599+0000 7f5c5b481700 1 --2- 172.21.15.3:0/3237045835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c5409f650 0x7f5c54131f00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:00.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.599+0000 7f5c5b481700 1 -- 172.21.15.3:0/3237045835 >> 172.21.15.3:0/3237045835 conn(0x7f5c540a4210 msgr2=0x7f5c540a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:00.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.599+0000 7f5c5b481700 1 -- 172.21.15.3:0/3237045835 shutdown_connections 2023-10-03T10:14:00.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.599+0000 7f5c5b481700 1 -- 172.21.15.3:0/3237045835 wait complete. 2023-10-03T10:14:00.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.600+0000 7f5c5b481700 1 Processor -- start 2023-10-03T10:14:00.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.600+0000 7f5c5b481700 1 -- start start 2023-10-03T10:14:00.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.600+0000 7f5c5b481700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c5409e930 0x7f5c5406b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:00.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.600+0000 7f5c5b481700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c5409f240 0x7f5c5406bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:00.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.600+0000 7f5c5b481700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c5409f650 0x7f5c5406c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:00.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.600+0000 7f5c5b481700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5406d4c0 con 0x7f5c5409f240 2023-10-03T10:14:00.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.600+0000 7f5c5b481700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c541a9460 con 0x7f5c5409e930 2023-10-03T10:14:00.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.600+0000 7f5c5b481700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c541a91a0 con 0x7f5c5409f650 2023-10-03T10:14:00.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.600+0000 7f5c5921d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c5409e930 0x7f5c5406b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:00.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.600+0000 7f5c58a1c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c5409f240 0x7f5c5406bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:00.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.600+0000 7f5c59a1e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c5409f650 0x7f5c5406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:00.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.600+0000 7f5c5921d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c5409e930 0x7f5c5406b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33464/0 (socket says 172.21.15.3:33464) 2023-10-03T10:14:00.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.600+0000 7f5c5921d700 1 -- 172.21.15.3:0/3300233245 learned_addr learned my addr 172.21.15.3:0/3300233245 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:00.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.601+0000 7f5c5921d700 1 -- 172.21.15.3:0/3300233245 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c5409f650 msgr2=0x7f5c5406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:00.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.601+0000 7f5c5921d700 1 --2- 172.21.15.3:0/3300233245 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c5409f650 0x7f5c5406c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:00.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.601+0000 7f5c5921d700 1 -- 172.21.15.3:0/3300233245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c5409f240 msgr2=0x7f5c5406bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:00.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.601+0000 7f5c5921d700 1 --2- 172.21.15.3:0/3300233245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c5409f240 0x7f5c5406bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:00.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.601+0000 7f5c5921d700 1 -- 172.21.15.3:0/3300233245 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c500094a0 con 0x7f5c5409e930 2023-10-03T10:14:00.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.601+0000 7f5c5921d700 1 --2- 172.21.15.3:0/3300233245 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c5409e930 0x7f5c5406b8b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000ebe0 tx=0x7f5c5000eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:00.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.601+0000 7f5c45ffb700 1 -- 172.21.15.3:0/3300233245 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5000c850 con 0x7f5c5409e930 2023-10-03T10:14:00.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.601+0000 7f5c45ffb700 1 -- 172.21.15.3:0/3300233245 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c5000c9b0 con 0x7f5c5409e930 2023-10-03T10:14:00.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.601+0000 7f5c5b481700 1 -- 172.21.15.3:0/3300233245 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4c009b30 con 0x7f5c5409e930 2023-10-03T10:14:00.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.601+0000 7f5c45ffb700 1 -- 172.21.15.3:0/3300233245 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5000cd40 con 0x7f5c5409e930 2023-10-03T10:14:00.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.601+0000 7f5c5b481700 1 -- 172.21.15.3:0/3300233245 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c540a0190 con 0x7f5c5409e930 2023-10-03T10:14:00.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.602+0000 7f5c45ffb700 1 -- 172.21.15.3:0/3300233245 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5c50013470 con 0x7f5c5409e930 2023-10-03T10:14:00.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.602+0000 7f5c45ffb700 1 -- 172.21.15.3:0/3300233245 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5c5003f7b0 con 0x7f5c5409e930 2023-10-03T10:14:00.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.602+0000 7f5c5b481700 1 -- 172.21.15.3:0/3300233245 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c541a0b30 con 0x7f5c5409e930 2023-10-03T10:14:00.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.607+0000 7f5c45ffb700 1 -- 172.21.15.3:0/3300233245 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5c50021020 con 0x7f5c5409e930 2023-10-03T10:14:00.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.813+0000 7f5c5b481700 1 -- 172.21.15.3:0/3300233245 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5c54071d20 con 0x7f5c5409e930 2023-10-03T10:14:00.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.813+0000 7f5c45ffb700 1 -- 172.21.15.3:0/3300233245 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5c5000cea0 con 0x7f5c5409e930 2023-10-03T10:14:00.814 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:00.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.815+0000 7f5c5b481700 1 -- 172.21.15.3:0/3300233245 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c5409e930 msgr2=0x7f5c5406b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:00.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.815+0000 7f5c5b481700 1 --2- 172.21.15.3:0/3300233245 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c5409e930 0x7f5c5406b8b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000ebe0 tx=0x7f5c5000eef0 comp rx=0 tx=0).stop 2023-10-03T10:14:00.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.816+0000 7f5c5b481700 1 -- 172.21.15.3:0/3300233245 shutdown_connections 2023-10-03T10:14:00.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.816+0000 7f5c5b481700 1 --2- 172.21.15.3:0/3300233245 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c5409f650 0x7f5c5406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:00.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.816+0000 7f5c5b481700 1 --2- 172.21.15.3:0/3300233245 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c5409e930 0x7f5c5406b8b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:00.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.816+0000 7f5c5b481700 1 --2- 172.21.15.3:0/3300233245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c5409f240 0x7f5c5406bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:00.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.816+0000 7f5c5b481700 1 -- 172.21.15.3:0/3300233245 >> 172.21.15.3:0/3300233245 conn(0x7f5c540a4210 msgr2=0x7f5c541b8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:00.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.816+0000 7f5c5b481700 1 -- 172.21.15.3:0/3300233245 shutdown_connections 2023-10-03T10:14:00.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:00.816+0000 7f5c5b481700 1 -- 172.21.15.3:0/3300233245 wait complete. 2023-10-03T10:14:00.829 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:00.860 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:00.948 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:00.954 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:01.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.001+0000 7f21aa7b9700 1 Processor -- start 2023-10-03T10:14:01.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.002+0000 7f21aa7b9700 1 -- start start 2023-10-03T10:14:01.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.002+0000 7f21aa7b9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21a4133510 0x7f21a41358f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:01.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.002+0000 7f21aa7b9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21a4135e30 0x7f21a4138210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:01.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.002+0000 7f21aa7b9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21a41310d0 con 0x7f21a41305f0 2023-10-03T10:14:01.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.002+0000 7f21aa7b9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a41301c0 con 0x7f21a4133510 2023-10-03T10:14:01.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.002+0000 7f21aa7b9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f21a4071ff0 con 0x7f21a4135e30 2023-10-03T10:14:01.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.003+0000 7f21a37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21a4133510 0x7f21a41358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:01.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.003+0000 7f21a8d56700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21a4135e30 0x7f21a4138210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:01.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.003+0000 7f21a37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21a4133510 0x7f21a41358f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33472/0 (socket says 172.21.15.3:33472) 2023-10-03T10:14:01.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.003+0000 7f21a3fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f21a41305f0 0x7f21a41309c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45298/0 (socket says 172.21.15.3:45298) 2023-10-03T10:14:01.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.003+0000 7f21a37fe700 1 -- 172.21.15.3:0/348705604 learned_addr learned my addr 172.21.15.3:0/348705604 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:01.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.003+0000 7f21a37fe700 1 -- 172.21.15.3:0/348705604 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21a4135e30 msgr2=0x7f21a4138210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:01.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.003+0000 7f21a37fe700 1 --2- 172.21.15.3:0/348705604 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21a4135e30 0x7f21a4138210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.003+0000 7f21a37fe700 1 -- 172.21.15.3:0/348705604 >> v1:172.21.15.3:6789/0 conn(0x7f21a41305f0 legacy=0x7f21a41309c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:01.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.003+0000 7f21a2ffd700 1 -- 172.21.15.3:0/348705604 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1237860728 0 0) 0x7f21a41310d0 con 0x7f21a41305f0 2023-10-03T10:14:01.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.003+0000 7f21a37fe700 1 -- 172.21.15.3:0/348705604 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a4070e70 con 0x7f21a4133510 2023-10-03T10:14:01.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.003+0000 7f21a8d56700 1 --2- 172.21.15.3:0/348705604 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21a4135e30 0x7f21a4138210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:01.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.003+0000 7f21a37fe700 1 --2- 172.21.15.3:0/348705604 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21a4133510 0x7f21a41358f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f219800b9d0 tx=0x7f219800bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48f34475925b3be4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:01.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.003+0000 7f21a2ffd700 1 -- 172.21.15.3:0/348705604 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219800cc60 con 0x7f21a4133510 2023-10-03T10:14:01.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.004+0000 7f21a2ffd700 1 -- 172.21.15.3:0/348705604 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219800cdc0 con 0x7f21a4133510 2023-10-03T10:14:01.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.004+0000 7f21a2ffd700 1 -- 172.21.15.3:0/348705604 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219800f680 con 0x7f21a4133510 2023-10-03T10:14:01.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.004+0000 7f21aa7b9700 1 -- 172.21.15.3:0/348705604 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21a4133510 msgr2=0x7f21a41358f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:01.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.004+0000 7f21aa7b9700 1 --2- 172.21.15.3:0/348705604 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21a4133510 0x7f21a41358f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f219800b9d0 tx=0x7f219800bce0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.005+0000 7f21aa7b9700 1 -- 172.21.15.3:0/348705604 shutdown_connections 2023-10-03T10:14:01.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.005+0000 7f21aa7b9700 1 --2- 172.21.15.3:0/348705604 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21a4135e30 0x7f21a4138210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.005+0000 7f21aa7b9700 1 --2- 172.21.15.3:0/348705604 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21a4133510 0x7f21a41358f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.005+0000 7f21aa7b9700 1 -- 172.21.15.3:0/348705604 >> 172.21.15.3:0/348705604 conn(0x7f21a40a2aa0 msgr2=0x7f21a40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:01.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.005+0000 7f21aa7b9700 1 -- 172.21.15.3:0/348705604 shutdown_connections 2023-10-03T10:14:01.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.005+0000 7f21aa7b9700 1 -- 172.21.15.3:0/348705604 wait complete. 2023-10-03T10:14:01.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.006+0000 7f21aa7b9700 1 Processor -- start 2023-10-03T10:14:01.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.006+0000 7f21aa7b9700 1 -- start start 2023-10-03T10:14:01.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.006+0000 7f21aa7b9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21a41305f0 0x7f21a409c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:01.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.006+0000 7f21aa7b9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21a4133510 0x7f21a409cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:01.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.006+0000 7f21aa7b9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21a4135e30 0x7f21a409d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:01.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.006+0000 7f21aa7b9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a4118e10 con 0x7f21a4133510 2023-10-03T10:14:01.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.006+0000 7f21aa7b9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a411a750 con 0x7f21a41305f0 2023-10-03T10:14:01.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.006+0000 7f21aa7b9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f21a4132e00 con 0x7f21a4135e30 2023-10-03T10:14:01.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.007+0000 7f21a37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21a4133510 0x7f21a409cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:01.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.007+0000 7f21a3fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21a41305f0 0x7f21a409c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:01.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.007+0000 7f21a37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21a4133510 0x7f21a409cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51602/0 (socket says 172.21.15.3:51602) 2023-10-03T10:14:01.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.007+0000 7f21a37fe700 1 -- 172.21.15.3:0/3710855469 learned_addr learned my addr 172.21.15.3:0/3710855469 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:01.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.007+0000 7f21a8d56700 1 --2- 172.21.15.3:0/3710855469 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21a4135e30 0x7f21a409d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:01.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.007+0000 7f21a37fe700 1 -- 172.21.15.3:0/3710855469 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21a4135e30 msgr2=0x7f21a409d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:01.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.007+0000 7f21a37fe700 1 --2- 172.21.15.3:0/3710855469 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21a4135e30 0x7f21a409d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.007+0000 7f21a37fe700 1 -- 172.21.15.3:0/3710855469 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21a41305f0 msgr2=0x7f21a409c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:01.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.007+0000 7f21a37fe700 1 --2- 172.21.15.3:0/3710855469 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21a41305f0 0x7f21a409c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.007+0000 7f21a37fe700 1 -- 172.21.15.3:0/3710855469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219c00f050 con 0x7f21a4133510 2023-10-03T10:14:01.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.007+0000 7f21a3fff700 1 --2- 172.21.15.3:0/3710855469 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21a41305f0 0x7f21a409c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:01.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.007+0000 7f21a37fe700 1 --2- 172.21.15.3:0/3710855469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21a4133510 0x7f21a409cbd0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2198000c00 tx=0x7f2198004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:01.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.007+0000 7f21a0ff9700 1 -- 172.21.15.3:0/3710855469 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219801b040 con 0x7f21a4133510 2023-10-03T10:14:01.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.008+0000 7f21a0ff9700 1 -- 172.21.15.3:0/3710855469 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2198021610 con 0x7f21a4133510 2023-10-03T10:14:01.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.008+0000 7f21a0ff9700 1 -- 172.21.15.3:0/3710855469 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21980219c0 con 0x7f21a4133510 2023-10-03T10:14:01.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.008+0000 7f21aa7b9700 1 -- 172.21.15.3:0/3710855469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f219800b600 con 0x7f21a4133510 2023-10-03T10:14:01.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.008+0000 7f21aa7b9700 1 -- 172.21.15.3:0/3710855469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a4119ee0 con 0x7f21a4133510 2023-10-03T10:14:01.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.009+0000 7f21aa7b9700 1 -- 172.21.15.3:0/3710855469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21a4074ea0 con 0x7f21a4133510 2023-10-03T10:14:01.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.009+0000 7f21a0ff9700 1 -- 172.21.15.3:0/3710855469 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2198021b20 con 0x7f21a4133510 2023-10-03T10:14:01.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.009+0000 7f21a0ff9700 1 -- 172.21.15.3:0/3710855469 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2198041bb0 con 0x7f21a4133510 2023-10-03T10:14:01.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.014+0000 7f21a0ff9700 1 -- 172.21.15.3:0/3710855469 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f219801e340 con 0x7f21a4133510 2023-10-03T10:14:01.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.091+0000 7f693c12a700 1 Processor -- start 2023-10-03T10:14:01.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.092+0000 7f693c12a700 1 -- start start 2023-10-03T10:14:01.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.092+0000 7f693c12a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f69340a0a30 0x7f69340a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:01.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.092+0000 7f693c12a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f693409d630 0x7f69340a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:01.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.092+0000 7f693c12a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69340a2a70 con 0x7f6934133930 2023-10-03T10:14:01.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.092+0000 7f693c12a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69340a01d0 con 0x7f693409d630 2023-10-03T10:14:01.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.092+0000 7f693c12a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f693409e2b0 con 0x7f69340a0a30 2023-10-03T10:14:01.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.092+0000 7f6939ec6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f69340a0a30 0x7f69340a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:01.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.093+0000 7f69396c5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f693409d630 0x7f69340a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:01.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.093+0000 7f6939ec6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f69340a0a30 0x7f69340a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43964/0 (socket says 172.21.15.3:43964) 2023-10-03T10:14:01.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.093+0000 7f69396c5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f693409d630 0x7f69340a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51604/0 (socket says 172.21.15.3:51604) 2023-10-03T10:14:01.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.093+0000 7f693a6c7700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6934133930 0x7f6934133db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45306/0 (socket says 172.21.15.3:45306) 2023-10-03T10:14:01.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.093+0000 7f6939ec6700 1 -- 172.21.15.3:0/2888225082 learned_addr learned my addr 172.21.15.3:0/2888225082 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.093+0000 7f69396c5700 1 -- 172.21.15.3:0/2888225082 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f69340a0a30 msgr2=0x7f69340a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.093+0000 7f69396c5700 1 --2- 172.21.15.3:0/2888225082 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f69340a0a30 0x7f69340a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.093+0000 7f69396c5700 1 -- 172.21.15.3:0/2888225082 >> v1:172.21.15.3:6789/0 conn(0x7f6934133930 legacy=0x7f6934133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.093+0000 7f69396c5700 1 -- 172.21.15.3:0/2888225082 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f693409dc50 con 0x7f693409d630 2023-10-03T10:14:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.093+0000 7f6938ec4700 1 -- 172.21.15.3:0/2888225082 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3298125078 0 0) 0x7f69340a2a70 con 0x7f6934133930 2023-10-03T10:14:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.093+0000 7f69396c5700 1 --2- 172.21.15.3:0/2888225082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f693409d630 0x7f69340a1340 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6924009840 tx=0x7f6924009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e614dfcd309c56b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.093+0000 7f6938ec4700 1 -- 172.21.15.3:0/2888225082 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6924011830 con 0x7f693409d630 2023-10-03T10:14:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.093+0000 7f6938ec4700 1 -- 172.21.15.3:0/2888225082 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6924011990 con 0x7f693409d630 2023-10-03T10:14:01.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.093+0000 7f6938ec4700 1 -- 172.21.15.3:0/2888225082 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6924011d20 con 0x7f693409d630 2023-10-03T10:14:01.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.094+0000 7f693c12a700 1 -- 172.21.15.3:0/2888225082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f693409d630 msgr2=0x7f69340a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:01.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.094+0000 7f693c12a700 1 --2- 172.21.15.3:0/2888225082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f693409d630 0x7f69340a1340 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6924009840 tx=0x7f6924009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:01.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.094+0000 7f693c12a700 1 -- 172.21.15.3:0/2888225082 shutdown_connections 2023-10-03T10:14:01.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.094+0000 7f693c12a700 1 --2- 172.21.15.3:0/2888225082 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f69340a0a30 0x7f69340a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.094+0000 7f693c12a700 1 --2- 172.21.15.3:0/2888225082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f693409d630 0x7f69340a1340 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.095+0000 7f693c12a700 1 -- 172.21.15.3:0/2888225082 >> 172.21.15.3:0/2888225082 conn(0x7f69340a3050 msgr2=0x7f69340a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:01.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.095+0000 7f693c12a700 1 -- 172.21.15.3:0/2888225082 shutdown_connections 2023-10-03T10:14:01.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.095+0000 7f693c12a700 1 -- 172.21.15.3:0/2888225082 wait complete. 2023-10-03T10:14:01.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.096+0000 7f693c12a700 1 Processor -- start 2023-10-03T10:14:01.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.096+0000 7f693c12a700 1 -- start start 2023-10-03T10:14:01.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.096+0000 7f693c12a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f693409d630 0x7f69341c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:01.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.096+0000 7f693c12a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69340a0a30 0x7f69341c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:01.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.096+0000 7f693c12a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6934133930 0x7f69341c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:01.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.096+0000 7f693c12a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69341206f0 con 0x7f69340a0a30 2023-10-03T10:14:01.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.096+0000 7f693c12a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6934120d40 con 0x7f693409d630 2023-10-03T10:14:01.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.096+0000 7f693c12a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6934122600 con 0x7f6934133930 2023-10-03T10:14:01.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.096+0000 7f69396c5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69340a0a30 0x7f69341c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:01.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.096+0000 7f69396c5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69340a0a30 0x7f69341c8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51614/0 (socket says 172.21.15.3:51614) 2023-10-03T10:14:01.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.096+0000 7f69396c5700 1 -- 172.21.15.3:0/3649730099 learned_addr learned my addr 172.21.15.3:0/3649730099 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:01.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.096+0000 7f6939ec6700 1 --2- 172.21.15.3:0/3649730099 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f693409d630 0x7f69341c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:01.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.096+0000 7f693a6c7700 1 --2- 172.21.15.3:0/3649730099 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6934133930 0x7f69341c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:01.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.097+0000 7f69396c5700 1 -- 172.21.15.3:0/3649730099 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6934133930 msgr2=0x7f69341c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.097+0000 7f69396c5700 1 --2- 172.21.15.3:0/3649730099 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6934133930 0x7f69341c8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.097+0000 7f69396c5700 1 -- 172.21.15.3:0/3649730099 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f693409d630 msgr2=0x7f69341c81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.097+0000 7f69396c5700 1 --2- 172.21.15.3:0/3649730099 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f693409d630 0x7f69341c81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.097+0000 7f69396c5700 1 -- 172.21.15.3:0/3649730099 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69240094a0 con 0x7f69340a0a30 2023-10-03T10:14:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.097+0000 7f693a6c7700 1 --2- 172.21.15.3:0/3649730099 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6934133930 0x7f69341c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:14:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.097+0000 7f69396c5700 1 --2- 172.21.15.3:0/3649730099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69340a0a30 0x7f69341c8700 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f69240090b0 tx=0x7f6924004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.097+0000 7f692a7fc700 1 -- 172.21.15.3:0/3649730099 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69240045d0 con 0x7f69340a0a30 2023-10-03T10:14:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.097+0000 7f692a7fc700 1 -- 172.21.15.3:0/3649730099 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f692401f070 con 0x7f69340a0a30 2023-10-03T10:14:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.097+0000 7f692a7fc700 1 -- 172.21.15.3:0/3649730099 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69240038e0 con 0x7f69340a0a30 2023-10-03T10:14:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.097+0000 7f693c12a700 1 -- 172.21.15.3:0/3649730099 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69340a4350 con 0x7f69340a0a30 2023-10-03T10:14:01.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.097+0000 7f693c12a700 1 -- 172.21.15.3:0/3649730099 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6934120ee0 con 0x7f69340a0a30 2023-10-03T10:14:01.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.098+0000 7f692a7fc700 1 -- 172.21.15.3:0/3649730099 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6924003a40 con 0x7f69340a0a30 2023-10-03T10:14:01.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.099+0000 7f692a7fc700 1 -- 172.21.15.3:0/3649730099 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6924021070 con 0x7f69340a0a30 2023-10-03T10:14:01.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.099+0000 7f6917fff700 1 -- 172.21.15.3:0/3649730099 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69100052e0 con 0x7f69340a0a30 2023-10-03T10:14:01.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.104+0000 7f692a7fc700 1 -- 172.21.15.3:0/3649730099 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f692402b080 con 0x7f69340a0a30 2023-10-03T10:14:01.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.135+0000 7f8850d23700 1 Processor -- start 2023-10-03T10:14:01.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.135+0000 7f8850d23700 1 -- start start 2023-10-03T10:14:01.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.136+0000 7f8850d23700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f884c0a0a30 0x7f884c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:01.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.136+0000 7f8850d23700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f884c09d630 0x7f884c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:01.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.136+0000 7f8850d23700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f884c133930 0x7f884c133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:01.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.136+0000 7f8850d23700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f884c0a2a70 con 0x7f884c0a0a30 2023-10-03T10:14:01.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.136+0000 7f8850d23700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f884c0a01d0 con 0x7f884c09d630 2023-10-03T10:14:01.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.136+0000 7f8850d23700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f884c09e2b0 con 0x7f884c133930 2023-10-03T10:14:01.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.136+0000 7f8849d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f884c09d630 0x7f884c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:01.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.136+0000 7f8849d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f884c09d630 0x7f884c0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33486/0 (socket says 172.21.15.3:33486) 2023-10-03T10:14:01.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.136+0000 7f8849d9b700 1 -- 172.21.15.3:0/777795621 learned_addr learned my addr 172.21.15.3:0/777795621 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:01.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.136+0000 7f884ad9d700 1 --2- 172.21.15.3:0/777795621 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f884c133930 0x7f884c133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:01.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.136+0000 7f884a59c700 1 --2- 172.21.15.3:0/777795621 >> v2:172.21.15.3:3300/0 conn(0x7f884c0a0a30 0x7f884c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:01.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.136+0000 7f8849d9b700 1 -- 172.21.15.3:0/777795621 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f884c133930 msgr2=0x7f884c133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:01.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.136+0000 7f8849d9b700 1 --2- 172.21.15.3:0/777795621 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f884c133930 0x7f884c133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.136+0000 7f8849d9b700 1 -- 172.21.15.3:0/777795621 >> v2:172.21.15.3:3300/0 conn(0x7f884c0a0a30 msgr2=0x7f884c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:01.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.136+0000 7f8849d9b700 1 --2- 172.21.15.3:0/777795621 >> v2:172.21.15.3:3300/0 conn(0x7f884c0a0a30 0x7f884c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.136+0000 7f8849d9b700 1 -- 172.21.15.3:0/777795621 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c09dc50 con 0x7f884c09d630 2023-10-03T10:14:01.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.137+0000 7f884a59c700 1 --2- 172.21.15.3:0/777795621 >> v2:172.21.15.3:3300/0 conn(0x7f884c0a0a30 0x7f884c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:14:01.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.137+0000 7f8849d9b700 1 --2- 172.21.15.3:0/777795621 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f884c09d630 0x7f884c0a1340 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f88400098f0 tx=0x7f8840009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2414eea9242e2029 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:01.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.137+0000 7f884959a700 1 -- 172.21.15.3:0/777795621 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8840011830 con 0x7f884c09d630 2023-10-03T10:14:01.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.137+0000 7f884959a700 1 -- 172.21.15.3:0/777795621 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8840011990 con 0x7f884c09d630 2023-10-03T10:14:01.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.137+0000 7f884959a700 1 -- 172.21.15.3:0/777795621 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8840011d20 con 0x7f884c09d630 2023-10-03T10:14:01.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.137+0000 7f8850d23700 1 -- 172.21.15.3:0/777795621 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f884c09d630 msgr2=0x7f884c0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:01.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.137+0000 7f8850d23700 1 --2- 172.21.15.3:0/777795621 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f884c09d630 0x7f884c0a1340 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f88400098f0 tx=0x7f8840009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:01.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.138+0000 7f8850d23700 1 -- 172.21.15.3:0/777795621 shutdown_connections 2023-10-03T10:14:01.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.138+0000 7f8850d23700 1 --2- 172.21.15.3:0/777795621 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f884c133930 0x7f884c133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.138+0000 7f8850d23700 1 --2- 172.21.15.3:0/777795621 >> v2:172.21.15.3:3300/0 conn(0x7f884c0a0a30 0x7f884c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.138+0000 7f8850d23700 1 --2- 172.21.15.3:0/777795621 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f884c09d630 0x7f884c0a1340 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.138+0000 7f8850d23700 1 -- 172.21.15.3:0/777795621 >> 172.21.15.3:0/777795621 conn(0x7f884c0a3050 msgr2=0x7f884c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:01.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.138+0000 7f8850d23700 1 -- 172.21.15.3:0/777795621 shutdown_connections 2023-10-03T10:14:01.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.138+0000 7f8850d23700 1 -- 172.21.15.3:0/777795621 wait complete. 2023-10-03T10:14:01.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.139+0000 7f8850d23700 1 Processor -- start 2023-10-03T10:14:01.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.139+0000 7f8850d23700 1 -- start start 2023-10-03T10:14:01.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.139+0000 7f8850d23700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f884c09d630 0x7f884c11f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:01.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.139+0000 7f8850d23700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f884c0a0a30 0x7f884c11fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:01.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.139+0000 7f8850d23700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f884c133930 0x7f884c1c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:01.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.139+0000 7f8850d23700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c11f250 con 0x7f884c133930 2023-10-03T10:14:01.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.139+0000 7f8850d23700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f884c122070 con 0x7f884c0a0a30 2023-10-03T10:14:01.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.139+0000 7f8850d23700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f884c122330 con 0x7f884c09d630 2023-10-03T10:14:01.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.140+0000 7f884a59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f884c09d630 0x7f884c11f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:01.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.140+0000 7f8849d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f884c0a0a30 0x7f884c11fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:01.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.140+0000 7f884a59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f884c09d630 0x7f884c11f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:44000/0 (socket says 172.21.15.3:44000) 2023-10-03T10:14:01.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.140+0000 7f884a59c700 1 -- 172.21.15.3:0/2308221017 learned_addr learned my addr 172.21.15.3:0/2308221017 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:01.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.140+0000 7f884ad9d700 1 --2- 172.21.15.3:0/2308221017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f884c133930 0x7f884c1c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:01.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.140+0000 7f8849d9b700 1 -- 172.21.15.3:0/2308221017 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f884c09d630 msgr2=0x7f884c11f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:01.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.140+0000 7f8849d9b700 1 --2- 172.21.15.3:0/2308221017 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f884c09d630 0x7f884c11f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.140+0000 7f8849d9b700 1 -- 172.21.15.3:0/2308221017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f884c133930 msgr2=0x7f884c1c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:01.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.140+0000 7f8849d9b700 1 --2- 172.21.15.3:0/2308221017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f884c133930 0x7f884c1c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.140+0000 7f8849d9b700 1 -- 172.21.15.3:0/2308221017 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8840009550 con 0x7f884c0a0a30 2023-10-03T10:14:01.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.140+0000 7f884a59c700 1 --2- 172.21.15.3:0/2308221017 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f884c09d630 0x7f884c11f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:01.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.140+0000 7f884ad9d700 1 --2- 172.21.15.3:0/2308221017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f884c133930 0x7f884c1c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:01.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.140+0000 7f8849d9b700 1 --2- 172.21.15.3:0/2308221017 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f884c0a0a30 0x7f884c11fef0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8840016000 tx=0x7f8840004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:01.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.140+0000 7f883affd700 1 -- 172.21.15.3:0/2308221017 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8840004510 con 0x7f884c0a0a30 2023-10-03T10:14:01.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.141+0000 7f883affd700 1 -- 172.21.15.3:0/2308221017 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88400036a0 con 0x7f884c0a0a30 2023-10-03T10:14:01.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.141+0000 7f883affd700 1 -- 172.21.15.3:0/2308221017 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8840003a30 con 0x7f884c0a0a30 2023-10-03T10:14:01.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.141+0000 7f8850d23700 1 -- 172.21.15.3:0/2308221017 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f884c122da0 con 0x7f884c0a0a30 2023-10-03T10:14:01.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.141+0000 7f8850d23700 1 -- 172.21.15.3:0/2308221017 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f884c120fd0 con 0x7f884c0a0a30 2023-10-03T10:14:01.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.142+0000 7f8850d23700 1 -- 172.21.15.3:0/2308221017 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f884c0747f0 con 0x7f884c0a0a30 2023-10-03T10:14:01.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.143+0000 7f883affd700 1 -- 172.21.15.3:0/2308221017 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8840021070 con 0x7f884c0a0a30 2023-10-03T10:14:01.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.143+0000 7f883affd700 1 -- 172.21.15.3:0/2308221017 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8840040c50 con 0x7f884c0a0a30 2023-10-03T10:14:01.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.147+0000 7f883affd700 1 -- 172.21.15.3:0/2308221017 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8840019770 con 0x7f884c0a0a30 2023-10-03T10:14:01.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.228+0000 7f21aa7b9700 1 -- 172.21.15.3:0/3710855469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f21a4072340 con 0x7f21a4133510 2023-10-03T10:14:01.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.228+0000 7f21a0ff9700 1 -- 172.21.15.3:0/3710855469 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2198020ad0 con 0x7f21a4133510 2023-10-03T10:14:01.229 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:01.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.232+0000 7f21aa7b9700 1 -- 172.21.15.3:0/3710855469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21a4133510 msgr2=0x7f21a409cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:01.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.232+0000 7f21aa7b9700 1 --2- 172.21.15.3:0/3710855469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21a4133510 0x7f21a409cbd0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2198000c00 tx=0x7f2198004750 comp rx=0 tx=0).stop 2023-10-03T10:14:01.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.232+0000 7f21aa7b9700 1 -- 172.21.15.3:0/3710855469 shutdown_connections 2023-10-03T10:14:01.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.232+0000 7f21aa7b9700 1 --2- 172.21.15.3:0/3710855469 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21a4135e30 0x7f21a409d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.232+0000 7f21aa7b9700 1 --2- 172.21.15.3:0/3710855469 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21a41305f0 0x7f21a409c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.232+0000 7f21aa7b9700 1 --2- 172.21.15.3:0/3710855469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21a4133510 0x7f21a409cbd0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.232+0000 7f21aa7b9700 1 -- 172.21.15.3:0/3710855469 >> 172.21.15.3:0/3710855469 conn(0x7f21a40a2aa0 msgr2=0x7f21a40678b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:01.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.232+0000 7f21aa7b9700 1 -- 172.21.15.3:0/3710855469 shutdown_connections 2023-10-03T10:14:01.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.232+0000 7f21aa7b9700 1 -- 172.21.15.3:0/3710855469 wait complete. 2023-10-03T10:14:01.252 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:01.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.329+0000 7f6917fff700 1 -- 172.21.15.3:0/3649730099 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6910005590 con 0x7f69340a0a30 2023-10-03T10:14:01.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.329+0000 7f692a7fc700 1 -- 172.21.15.3:0/3649730099 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6924015020 con 0x7f69340a0a30 2023-10-03T10:14:01.329 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:01.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.331+0000 7f6917fff700 1 -- 172.21.15.3:0/3649730099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69340a0a30 msgr2=0x7f69341c8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:01.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.331+0000 7f6917fff700 1 --2- 172.21.15.3:0/3649730099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69340a0a30 0x7f69341c8700 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f69240090b0 tx=0x7f6924004750 comp rx=0 tx=0).stop 2023-10-03T10:14:01.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.331+0000 7f6917fff700 1 -- 172.21.15.3:0/3649730099 shutdown_connections 2023-10-03T10:14:01.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.331+0000 7f6917fff700 1 --2- 172.21.15.3:0/3649730099 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6934133930 0x7f69341c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.331+0000 7f6917fff700 1 --2- 172.21.15.3:0/3649730099 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f693409d630 0x7f69341c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.331+0000 7f6917fff700 1 --2- 172.21.15.3:0/3649730099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69340a0a30 0x7f69341c8700 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.331+0000 7f6917fff700 1 -- 172.21.15.3:0/3649730099 >> 172.21.15.3:0/3649730099 conn(0x7f69340a3050 msgr2=0x7f693409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:01.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.331+0000 7f6917fff700 1 -- 172.21.15.3:0/3649730099 shutdown_connections 2023-10-03T10:14:01.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.331+0000 7f6917fff700 1 -- 172.21.15.3:0/3649730099 wait complete. 2023-10-03T10:14:01.350 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:01.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.369+0000 7f8850d23700 1 -- 172.21.15.3:0/2308221017 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f884c0a5a10 con 0x7f884c0a0a30 2023-10-03T10:14:01.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.369+0000 7f883affd700 1 -- 172.21.15.3:0/2308221017 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8840015030 con 0x7f884c0a0a30 2023-10-03T10:14:01.370 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:01.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.372+0000 7f8850d23700 1 -- 172.21.15.3:0/2308221017 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f884c0a0a30 msgr2=0x7f884c11fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:01.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.372+0000 7f8850d23700 1 --2- 172.21.15.3:0/2308221017 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f884c0a0a30 0x7f884c11fef0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8840016000 tx=0x7f8840004750 comp rx=0 tx=0).stop 2023-10-03T10:14:01.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.372+0000 7f8850d23700 1 -- 172.21.15.3:0/2308221017 shutdown_connections 2023-10-03T10:14:01.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.372+0000 7f8850d23700 1 --2- 172.21.15.3:0/2308221017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f884c133930 0x7f884c1c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.372+0000 7f8850d23700 1 --2- 172.21.15.3:0/2308221017 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f884c09d630 0x7f884c11f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.372+0000 7f8850d23700 1 --2- 172.21.15.3:0/2308221017 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f884c0a0a30 0x7f884c11fef0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.372+0000 7f8850d23700 1 -- 172.21.15.3:0/2308221017 >> 172.21.15.3:0/2308221017 conn(0x7f884c0a3050 msgr2=0x7f884c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:01.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.372+0000 7f8850d23700 1 -- 172.21.15.3:0/2308221017 shutdown_connections 2023-10-03T10:14:01.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.372+0000 7f8850d23700 1 -- 172.21.15.3:0/2308221017 wait complete. 2023-10-03T10:14:01.386 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:01.831 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:01.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.977+0000 7f29585d4700 1 Processor -- start 2023-10-03T10:14:01.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.977+0000 7f29585d4700 1 -- start start 2023-10-03T10:14:01.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.977+0000 7f29585d4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2950133a80 0x7f2950133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.977+0000 7f29585d4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f295012ff20 0x7f2950130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.977+0000 7f29585d4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2950130870 0x7f295013a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.977+0000 7f29585d4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f295006c570 con 0x7f2950133a80 2023-10-03T10:14:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.977+0000 7f29585d4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295006bc00 con 0x7f2950130870 2023-10-03T10:14:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.977+0000 7f29585d4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29500a0040 con 0x7f295012ff20 2023-10-03T10:14:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.977+0000 7f2956b71700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2950130870 0x7f295013a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:01.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.977+0000 7f2956370700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2950133a80 0x7f2950133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.978+0000 7f2956b71700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2950130870 0x7f295013a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33502/0 (socket says 172.21.15.3:33502) 2023-10-03T10:14:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.978+0000 7f2956370700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2950133a80 0x7f2950133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51650/0 (socket says 172.21.15.3:51650) 2023-10-03T10:14:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.978+0000 7f2956b71700 1 -- 172.21.15.3:0/2407907945 learned_addr learned my addr 172.21.15.3:0/2407907945 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.978+0000 7f2955b6f700 1 --2- 172.21.15.3:0/2407907945 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f295012ff20 0x7f2950130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.978+0000 7f2956370700 1 -- 172.21.15.3:0/2407907945 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f295012ff20 msgr2=0x7f2950130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.978+0000 7f2956370700 1 --2- 172.21.15.3:0/2407907945 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f295012ff20 0x7f2950130330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.978+0000 7f2956370700 1 -- 172.21.15.3:0/2407907945 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2950130870 msgr2=0x7f295013a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.978+0000 7f2956370700 1 --2- 172.21.15.3:0/2407907945 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2950130870 0x7f295013a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.978+0000 7f2956370700 1 -- 172.21.15.3:0/2407907945 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2950133860 con 0x7f2950133a80 2023-10-03T10:14:01.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.978+0000 7f2956b71700 1 --2- 172.21.15.3:0/2407907945 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2950130870 0x7f295013a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:01.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.978+0000 7f2956370700 1 --2- 172.21.15.3:0/2407907945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2950133a80 0x7f2950133e50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f294c009840 tx=0x7f294c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2420ddaac2eefb24 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:01.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.978+0000 7f295536e700 1 -- 172.21.15.3:0/2407907945 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c011830 con 0x7f2950133a80 2023-10-03T10:14:01.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.978+0000 7f295536e700 1 -- 172.21.15.3:0/2407907945 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294c011990 con 0x7f2950133a80 2023-10-03T10:14:01.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.978+0000 7f295536e700 1 -- 172.21.15.3:0/2407907945 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c011d20 con 0x7f2950133a80 2023-10-03T10:14:01.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.979+0000 7f29585d4700 1 -- 172.21.15.3:0/2407907945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2950133a80 msgr2=0x7f2950133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:01.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.979+0000 7f29585d4700 1 --2- 172.21.15.3:0/2407907945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2950133a80 0x7f2950133e50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f294c009840 tx=0x7f294c009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:01.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.979+0000 7f29585d4700 1 -- 172.21.15.3:0/2407907945 shutdown_connections 2023-10-03T10:14:01.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.979+0000 7f29585d4700 1 --2- 172.21.15.3:0/2407907945 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2950130870 0x7f295013a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.979+0000 7f29585d4700 1 --2- 172.21.15.3:0/2407907945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2950133a80 0x7f2950133e50 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.979+0000 7f29585d4700 1 --2- 172.21.15.3:0/2407907945 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f295012ff20 0x7f2950130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.979+0000 7f29585d4700 1 -- 172.21.15.3:0/2407907945 >> 172.21.15.3:0/2407907945 conn(0x7f29500a4210 msgr2=0x7f29500a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:01.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.979+0000 7f29585d4700 1 -- 172.21.15.3:0/2407907945 shutdown_connections 2023-10-03T10:14:01.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.979+0000 7f29585d4700 1 -- 172.21.15.3:0/2407907945 wait complete. 2023-10-03T10:14:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.980+0000 7f29585d4700 1 Processor -- start 2023-10-03T10:14:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.980+0000 7f29585d4700 1 -- start start 2023-10-03T10:14:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.980+0000 7f29585d4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f295012ff20 0x7f295009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.980+0000 7f29585d4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2950130870 0x7f295009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.981+0000 7f29585d4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2950133a80 0x7f295009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.981+0000 7f29585d4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295006d280 con 0x7f2950130870 2023-10-03T10:14:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.981+0000 7f29585d4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295011fd80 con 0x7f295012ff20 2023-10-03T10:14:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.981+0000 7f29585d4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f295012da90 con 0x7f2950133a80 2023-10-03T10:14:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.981+0000 7f2956370700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f295012ff20 0x7f295009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.981+0000 7f2956370700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f295012ff20 0x7f295009c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33516/0 (socket says 172.21.15.3:33516) 2023-10-03T10:14:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.981+0000 7f2956370700 1 -- 172.21.15.3:0/2858063185 learned_addr learned my addr 172.21.15.3:0/2858063185 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.981+0000 7f2955b6f700 1 --2- 172.21.15.3:0/2858063185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2950130870 0x7f295009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.981+0000 7f2956b71700 1 --2- 172.21.15.3:0/2858063185 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2950133a80 0x7f295009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.981+0000 7f2956370700 1 -- 172.21.15.3:0/2858063185 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2950133a80 msgr2=0x7f295009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.981+0000 7f2956370700 1 --2- 172.21.15.3:0/2858063185 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2950133a80 0x7f295009cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.981+0000 7f2956370700 1 -- 172.21.15.3:0/2858063185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2950130870 msgr2=0x7f295009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.981+0000 7f2956370700 1 --2- 172.21.15.3:0/2858063185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2950130870 0x7f295009c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.981+0000 7f2956370700 1 -- 172.21.15.3:0/2858063185 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2948009b30 con 0x7f295012ff20 2023-10-03T10:14:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.981+0000 7f2956b71700 1 --2- 172.21.15.3:0/2858063185 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2950133a80 0x7f295009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.981+0000 7f2955b6f700 1 --2- 172.21.15.3:0/2858063185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2950130870 0x7f295009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.981+0000 7f2956370700 1 --2- 172.21.15.3:0/2858063185 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f295012ff20 0x7f295009c480 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f294c0090b0 tx=0x7f294c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.981+0000 7f2946ffd700 1 -- 172.21.15.3:0/2858063185 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c004450 con 0x7f295012ff20 2023-10-03T10:14:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.981+0000 7f2946ffd700 1 -- 172.21.15.3:0/2858063185 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294c01f070 con 0x7f295012ff20 2023-10-03T10:14:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.981+0000 7f29585d4700 1 -- 172.21.15.3:0/2858063185 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294c0094a0 con 0x7f295012ff20 2023-10-03T10:14:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.981+0000 7f2946ffd700 1 -- 172.21.15.3:0/2858063185 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c0038e0 con 0x7f295012ff20 2023-10-03T10:14:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.981+0000 7f29585d4700 1 -- 172.21.15.3:0/2858063185 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f295009dee0 con 0x7f295012ff20 2023-10-03T10:14:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.982+0000 7f2946ffd700 1 -- 172.21.15.3:0/2858063185 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f294c003a40 con 0x7f295012ff20 2023-10-03T10:14:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.982+0000 7f2946ffd700 1 -- 172.21.15.3:0/2858063185 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f294c021070 con 0x7f295012ff20 2023-10-03T10:14:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.983+0000 7f29585d4700 1 -- 172.21.15.3:0/2858063185 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2950074ea0 con 0x7f295012ff20 2023-10-03T10:14:01.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:01.988+0000 7f2946ffd700 1 -- 172.21.15.3:0/2858063185 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f294c015080 con 0x7f295012ff20 2023-10-03T10:14:02.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.193+0000 7f29585d4700 1 -- 172.21.15.3:0/2858063185 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2950120260 con 0x7f295012ff20 2023-10-03T10:14:02.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.194+0000 7f2946ffd700 1 -- 172.21.15.3:0/2858063185 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f294c010380 con 0x7f295012ff20 2023-10-03T10:14:02.194 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:02.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.196+0000 7f29585d4700 1 -- 172.21.15.3:0/2858063185 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f295012ff20 msgr2=0x7f295009c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:02.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.197+0000 7f29585d4700 1 --2- 172.21.15.3:0/2858063185 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f295012ff20 0x7f295009c480 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f294c0090b0 tx=0x7f294c004750 comp rx=0 tx=0).stop 2023-10-03T10:14:02.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.197+0000 7f29585d4700 1 -- 172.21.15.3:0/2858063185 shutdown_connections 2023-10-03T10:14:02.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.197+0000 7f29585d4700 1 --2- 172.21.15.3:0/2858063185 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2950133a80 0x7f295009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.197+0000 7f29585d4700 1 --2- 172.21.15.3:0/2858063185 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f295012ff20 0x7f295009c480 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.197+0000 7f29585d4700 1 --2- 172.21.15.3:0/2858063185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2950130870 0x7f295009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.197+0000 7f29585d4700 1 -- 172.21.15.3:0/2858063185 >> 172.21.15.3:0/2858063185 conn(0x7f29500a4210 msgr2=0x7f295009e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:02.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.197+0000 7f29585d4700 1 -- 172.21.15.3:0/2858063185 shutdown_connections 2023-10-03T10:14:02.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.197+0000 7f29585d4700 1 -- 172.21.15.3:0/2858063185 wait complete. 2023-10-03T10:14:02.210 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:02.253 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:02.351 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:02.387 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:02.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.393+0000 7f175e3c5700 1 Processor -- start 2023-10-03T10:14:02.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.393+0000 7f175e3c5700 1 -- start start 2023-10-03T10:14:02.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.394+0000 7f175e3c5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f175809e930 0x7f175809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:02.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.394+0000 7f175e3c5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f175809f240 0x7f17581317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:02.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.394+0000 7f175e3c5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1758070940 con 0x7f175809f650 2023-10-03T10:14:02.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.394+0000 7f175e3c5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f175806fd60 con 0x7f175809f240 2023-10-03T10:14:02.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.394+0000 7f175e3c5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f175806fa80 con 0x7f175809e930 2023-10-03T10:14:02.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.394+0000 7f17577fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f175809f240 0x7f17581317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:02.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.394+0000 7f17577fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f175809f240 0x7f17581317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51676/0 (socket says 172.21.15.3:51676) 2023-10-03T10:14:02.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.394+0000 7f1757fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f175809e930 0x7f175809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:02.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.394+0000 7f17577fe700 1 -- 172.21.15.3:0/468672279 learned_addr learned my addr 172.21.15.3:0/468672279 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:02.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.394+0000 7f17577fe700 1 -- 172.21.15.3:0/468672279 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f175809e930 msgr2=0x7f175809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:02.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.394+0000 7f17577fe700 1 --2- 172.21.15.3:0/468672279 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f175809e930 0x7f175809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.394+0000 7f17577fe700 1 -- 172.21.15.3:0/468672279 >> v1:172.21.15.3:6789/0 conn(0x7f175809f650 legacy=0x7f1758131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:02.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.394+0000 7f17577fe700 1 -- 172.21.15.3:0/468672279 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175809fb60 con 0x7f175809f240 2023-10-03T10:14:02.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.394+0000 7f1757fff700 1 --2- 172.21.15.3:0/468672279 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f175809e930 0x7f175809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:02.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.395+0000 7f17577fe700 1 --2- 172.21.15.3:0/468672279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f175809f240 0x7f17581317f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f174c0098f0 tx=0x7f174c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471c1e7a71bace73 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:02.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.395+0000 7f1756ffd700 1 -- 172.21.15.3:0/468672279 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c011830 con 0x7f175809f240 2023-10-03T10:14:02.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.395+0000 7f1756ffd700 1 -- 172.21.15.3:0/468672279 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174c011990 con 0x7f175809f240 2023-10-03T10:14:02.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.395+0000 7f1756ffd700 1 -- 172.21.15.3:0/468672279 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c011d20 con 0x7f175809f240 2023-10-03T10:14:02.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.396+0000 7f175e3c5700 1 -- 172.21.15.3:0/468672279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f175809f240 msgr2=0x7f17581317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:02.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.396+0000 7f175e3c5700 1 --2- 172.21.15.3:0/468672279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f175809f240 0x7f17581317f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f174c0098f0 tx=0x7f174c009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:02.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.396+0000 7f175e3c5700 1 -- 172.21.15.3:0/468672279 shutdown_connections 2023-10-03T10:14:02.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.396+0000 7f175e3c5700 1 --2- 172.21.15.3:0/468672279 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f175809e930 0x7f175809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.396+0000 7f175e3c5700 1 --2- 172.21.15.3:0/468672279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f175809f240 0x7f17581317f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.396+0000 7f175e3c5700 1 -- 172.21.15.3:0/468672279 >> 172.21.15.3:0/468672279 conn(0x7f17580a4210 msgr2=0x7f17580a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:02.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.396+0000 7f175e3c5700 1 -- 172.21.15.3:0/468672279 shutdown_connections 2023-10-03T10:14:02.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.396+0000 7f175e3c5700 1 -- 172.21.15.3:0/468672279 wait complete. 2023-10-03T10:14:02.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.397+0000 7f175e3c5700 1 Processor -- start 2023-10-03T10:14:02.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.397+0000 7f175e3c5700 1 -- start start 2023-10-03T10:14:02.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.397+0000 7f175e3c5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f175809e930 0x7f17581a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:02.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.397+0000 7f175e3c5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f175809f240 0x7f17581a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:02.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.398+0000 7f175e3c5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f175809f650 0x7f17581c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:02.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.398+0000 7f175e3c5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17581a82c0 con 0x7f175809e930 2023-10-03T10:14:02.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.398+0000 7f175e3c5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17581a84a0 con 0x7f175809f240 2023-10-03T10:14:02.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.398+0000 7f175e3c5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f17581b9310 con 0x7f175809f650 2023-10-03T10:14:02.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.398+0000 7f1757fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f175809e930 0x7f17581a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:02.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.398+0000 7f1757fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f175809e930 0x7f17581a53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51682/0 (socket says 172.21.15.3:51682) 2023-10-03T10:14:02.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.398+0000 7f1757fff700 1 -- 172.21.15.3:0/2725072861 learned_addr learned my addr 172.21.15.3:0/2725072861 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:02.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.398+0000 7f175c962700 1 --2- 172.21.15.3:0/2725072861 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f175809f650 0x7f17581c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:02.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.398+0000 7f17577fe700 1 --2- 172.21.15.3:0/2725072861 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f175809f240 0x7f17581a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:02.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.398+0000 7f1757fff700 1 -- 172.21.15.3:0/2725072861 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f175809f650 msgr2=0x7f17581c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:02.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.398+0000 7f1757fff700 1 --2- 172.21.15.3:0/2725072861 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f175809f650 0x7f17581c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.398+0000 7f1757fff700 1 -- 172.21.15.3:0/2725072861 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f175809f240 msgr2=0x7f17581a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:02.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.398+0000 7f1757fff700 1 --2- 172.21.15.3:0/2725072861 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f175809f240 0x7f17581a5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.398+0000 7f1757fff700 1 -- 172.21.15.3:0/2725072861 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17480094a0 con 0x7f175809e930 2023-10-03T10:14:02.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.398+0000 7f1757fff700 1 --2- 172.21.15.3:0/2725072861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f175809e930 0x7f17581a53e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f174800ec10 tx=0x7f174800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:02.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.399+0000 7f1754ff9700 1 -- 172.21.15.3:0/2725072861 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174800c850 con 0x7f175809e930 2023-10-03T10:14:02.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.399+0000 7f1754ff9700 1 -- 172.21.15.3:0/2725072861 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174800c9b0 con 0x7f175809e930 2023-10-03T10:14:02.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.399+0000 7f175e3c5700 1 -- 172.21.15.3:0/2725072861 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f174c009550 con 0x7f175809e930 2023-10-03T10:14:02.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.399+0000 7f1754ff9700 1 -- 172.21.15.3:0/2725072861 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174800cd20 con 0x7f175809e930 2023-10-03T10:14:02.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.399+0000 7f175e3c5700 1 -- 172.21.15.3:0/2725072861 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17580a0190 con 0x7f175809e930 2023-10-03T10:14:02.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.400+0000 7f1754ff9700 1 -- 172.21.15.3:0/2725072861 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1748013470 con 0x7f175809e930 2023-10-03T10:14:02.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.400+0000 7f1754ff9700 1 -- 172.21.15.3:0/2725072861 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f174803fba0 con 0x7f175809e930 2023-10-03T10:14:02.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.401+0000 7f175e3c5700 1 -- 172.21.15.3:0/2725072861 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17581a6b80 con 0x7f175809e930 2023-10-03T10:14:02.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.406+0000 7f1754ff9700 1 -- 172.21.15.3:0/2725072861 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1748013720 con 0x7f175809e930 2023-10-03T10:14:02.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.496+0000 7f569ee82700 1 Processor -- start 2023-10-03T10:14:02.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.497+0000 7f569ee82700 1 -- start start 2023-10-03T10:14:02.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.497+0000 7f569ee82700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f569809d6f0 0x7f569809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:02.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.497+0000 7f569ee82700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f569809e410 0x7f5698130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:02.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.497+0000 7f569ee82700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f569809f130 con 0x7f569809e000 2023-10-03T10:14:02.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.497+0000 7f569ee82700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f569809f520 con 0x7f569809e410 2023-10-03T10:14:02.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.497+0000 7f569ee82700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f569809e930 con 0x7f569809d6f0 2023-10-03T10:14:02.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.497+0000 7f569d41f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f569809e410 0x7f5698130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:02.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.497+0000 7f569cc1e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f569809d6f0 0x7f569809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:02.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.497+0000 7f5697fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f569809e000 0x7f5698130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45316/0 (socket says 172.21.15.3:45316) 2023-10-03T10:14:02.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.497+0000 7f5697fff700 1 -- 172.21.15.3:0/1144275575 learned_addr learned my addr 172.21.15.3:0/1144275575 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:02.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.497+0000 7f569d41f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f569809e410 0x7f5698130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51688/0 (socket says 172.21.15.3:51688) 2023-10-03T10:14:02.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.498+0000 7f569d41f700 1 -- 172.21.15.3:0/1144275575 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f569809d6f0 msgr2=0x7f569809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:02.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.498+0000 7f569d41f700 1 --2- 172.21.15.3:0/1144275575 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f569809d6f0 0x7f569809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.498+0000 7f569d41f700 1 -- 172.21.15.3:0/1144275575 >> v1:172.21.15.3:6789/0 conn(0x7f569809e000 legacy=0x7f5698130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:02.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.498+0000 7f569d41f700 1 -- 172.21.15.3:0/1144275575 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569809b2d0 con 0x7f569809e410 2023-10-03T10:14:02.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.498+0000 7f56977fe700 1 -- 172.21.15.3:0/1144275575 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 985783110 0 0) 0x7f569809f130 con 0x7f569809e000 2023-10-03T10:14:02.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.498+0000 7f569cc1e700 1 --2- 172.21.15.3:0/1144275575 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f569809d6f0 0x7f569809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:02.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.498+0000 7f569d41f700 1 --2- 172.21.15.3:0/1144275575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f569809e410 0x7f5698130d70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f56900093f0 tx=0x7f569000f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8068105a5e765f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:02.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.498+0000 7f56977fe700 1 -- 172.21.15.3:0/1144275575 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5690012070 con 0x7f569809e410 2023-10-03T10:14:02.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.498+0000 7f56977fe700 1 -- 172.21.15.3:0/1144275575 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f569000fe40 con 0x7f569809e410 2023-10-03T10:14:02.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.498+0000 7f56977fe700 1 -- 172.21.15.3:0/1144275575 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5690016650 con 0x7f569809e410 2023-10-03T10:14:02.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.499+0000 7f569ee82700 1 -- 172.21.15.3:0/1144275575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f569809e410 msgr2=0x7f5698130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:02.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.499+0000 7f569ee82700 1 --2- 172.21.15.3:0/1144275575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f569809e410 0x7f5698130d70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f56900093f0 tx=0x7f569000f650 comp rx=0 tx=0).stop 2023-10-03T10:14:02.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.499+0000 7f569ee82700 1 -- 172.21.15.3:0/1144275575 shutdown_connections 2023-10-03T10:14:02.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.499+0000 7f569ee82700 1 --2- 172.21.15.3:0/1144275575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f569809e410 0x7f5698130d70 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.499+0000 7f569ee82700 1 --2- 172.21.15.3:0/1144275575 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f569809d6f0 0x7f569809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.499+0000 7f569ee82700 1 -- 172.21.15.3:0/1144275575 >> 172.21.15.3:0/1144275575 conn(0x7f5698066f50 msgr2=0x7f569809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:02.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.499+0000 7f569ee82700 1 -- 172.21.15.3:0/1144275575 shutdown_connections 2023-10-03T10:14:02.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.499+0000 7f569ee82700 1 -- 172.21.15.3:0/1144275575 wait complete. 2023-10-03T10:14:02.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.500+0000 7f569ee82700 1 Processor -- start 2023-10-03T10:14:02.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.500+0000 7f569ee82700 1 -- start start 2023-10-03T10:14:02.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.501+0000 7f569ee82700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f569809d6f0 0x7f569811e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:02.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.501+0000 7f569ee82700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f569809e000 0x7f569811e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:02.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.501+0000 7f569ee82700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f569809e410 0x7f569811ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:02.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.501+0000 7f569ee82700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569809c790 con 0x7f569809d6f0 2023-10-03T10:14:02.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.501+0000 7f569ee82700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569809a0c0 con 0x7f569809e000 2023-10-03T10:14:02.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.501+0000 7f569ee82700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5698120480 con 0x7f569809e410 2023-10-03T10:14:02.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.501+0000 7f5697fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f569809e000 0x7f569811e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:02.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.501+0000 7f569d41f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f569809e410 0x7f569811ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:02.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.501+0000 7f5697fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f569809e000 0x7f569811e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33540/0 (socket says 172.21.15.3:33540) 2023-10-03T10:14:02.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.501+0000 7f5697fff700 1 -- 172.21.15.3:0/4186105765 learned_addr learned my addr 172.21.15.3:0/4186105765 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:02.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.501+0000 7f569cc1e700 1 --2- 172.21.15.3:0/4186105765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f569809d6f0 0x7f569811e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:02.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.501+0000 7f5697fff700 1 -- 172.21.15.3:0/4186105765 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f569809e410 msgr2=0x7f569811ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:02.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.501+0000 7f5697fff700 1 --2- 172.21.15.3:0/4186105765 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f569809e410 0x7f569811ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.501+0000 7f5697fff700 1 -- 172.21.15.3:0/4186105765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f569809d6f0 msgr2=0x7f569811e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:02.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.501+0000 7f5697fff700 1 --2- 172.21.15.3:0/4186105765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f569809d6f0 0x7f569811e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.501+0000 7f5697fff700 1 -- 172.21.15.3:0/4186105765 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568c0094a0 con 0x7f569809e000 2023-10-03T10:14:02.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.501+0000 7f569d41f700 1 --2- 172.21.15.3:0/4186105765 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f569809e410 0x7f569811ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:02.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.502+0000 7f5697fff700 1 --2- 172.21.15.3:0/4186105765 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f569809e000 0x7f569811e550 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f568400c550 tx=0x7f568400c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:02.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.502+0000 7f56957fa700 1 -- 172.21.15.3:0/4186105765 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5684004c80 con 0x7f569809e000 2023-10-03T10:14:02.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.502+0000 7f56957fa700 1 -- 172.21.15.3:0/4186105765 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5684004de0 con 0x7f569809e000 2023-10-03T10:14:02.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.502+0000 7f56957fa700 1 -- 172.21.15.3:0/4186105765 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5684007310 con 0x7f569809e000 2023-10-03T10:14:02.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.502+0000 7f569ee82700 1 -- 172.21.15.3:0/4186105765 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f569000f050 con 0x7f569809e000 2023-10-03T10:14:02.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.502+0000 7f569ee82700 1 -- 172.21.15.3:0/4186105765 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5698121850 con 0x7f569809e000 2023-10-03T10:14:02.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.503+0000 7f569ee82700 1 -- 172.21.15.3:0/4186105765 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5698074ea0 con 0x7f569809e000 2023-10-03T10:14:02.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.504+0000 7f56957fa700 1 -- 172.21.15.3:0/4186105765 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5684007470 con 0x7f569809e000 2023-10-03T10:14:02.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.504+0000 7f56957fa700 1 -- 172.21.15.3:0/4186105765 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f568403f8f0 con 0x7f569809e000 2023-10-03T10:14:02.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.508+0000 7f56957fa700 1 -- 172.21.15.3:0/4186105765 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f56840133c0 con 0x7f569809e000 2023-10-03T10:14:02.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.537+0000 7f5a0e26c700 1 Processor -- start 2023-10-03T10:14:02.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.537+0000 7f5a0e26c700 1 -- start start 2023-10-03T10:14:02.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.538+0000 7f5a0e26c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5a081316c0 0x7f5a08131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:02.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.538+0000 7f5a0e26c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5a0809d9f0 0x7f5a0806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:02.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.538+0000 7f5a0e26c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5a0806acb0 0x7f5a0806b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:02.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.538+0000 7f5a0e26c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a0806b630 con 0x7f5a0809d9f0 2023-10-03T10:14:02.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.538+0000 7f5a0e26c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0809e130 con 0x7f5a081316c0 2023-10-03T10:14:02.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.538+0000 7f5a0e26c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a081312b0 con 0x7f5a0806acb0 2023-10-03T10:14:02.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.538+0000 7f5a077fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5a0809d9f0 0x7f5a0806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:02.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.539+0000 7f5a0c809700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5a0806acb0 0x7f5a0806b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:02.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.539+0000 7f5a077fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5a0809d9f0 0x7f5a0806a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51712/0 (socket says 172.21.15.3:51712) 2023-10-03T10:14:02.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.539+0000 7f5a077fe700 1 -- 172.21.15.3:0/155705943 learned_addr learned my addr 172.21.15.3:0/155705943 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:02.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.539+0000 7f5a07fff700 1 --2- 172.21.15.3:0/155705943 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5a081316c0 0x7f5a08131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:02.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.539+0000 7f5a0c809700 1 -- 172.21.15.3:0/155705943 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5a081316c0 msgr2=0x7f5a08131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:02.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.539+0000 7f5a0c809700 1 --2- 172.21.15.3:0/155705943 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5a081316c0 0x7f5a08131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.539+0000 7f5a0c809700 1 -- 172.21.15.3:0/155705943 >> v2:172.21.15.3:3300/0 conn(0x7f5a0809d9f0 msgr2=0x7f5a0806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:02.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.539+0000 7f5a0c809700 1 --2- 172.21.15.3:0/155705943 >> v2:172.21.15.3:3300/0 conn(0x7f5a0809d9f0 0x7f5a0806a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.539+0000 7f5a0c809700 1 -- 172.21.15.3:0/155705943 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0809f6e0 con 0x7f5a0806acb0 2023-10-03T10:14:02.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.539+0000 7f5a077fe700 1 --2- 172.21.15.3:0/155705943 >> v2:172.21.15.3:3300/0 conn(0x7f5a0809d9f0 0x7f5a0806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:02.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.539+0000 7f5a0c809700 1 --2- 172.21.15.3:0/155705943 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5a0806acb0 0x7f5a0806b0f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5a00009dc0 tx=0x7f5a0000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7413ef35412d4547 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:02.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.539+0000 7f5a06ffd700 1 -- 172.21.15.3:0/155705943 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0000e070 con 0x7f5a0806acb0 2023-10-03T10:14:02.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.539+0000 7f5a06ffd700 1 -- 172.21.15.3:0/155705943 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0000bae0 con 0x7f5a0806acb0 2023-10-03T10:14:02.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.539+0000 7f5a06ffd700 1 -- 172.21.15.3:0/155705943 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0000be90 con 0x7f5a0806acb0 2023-10-03T10:14:02.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.540+0000 7f5a0e26c700 1 -- 172.21.15.3:0/155705943 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5a0806acb0 msgr2=0x7f5a0806b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:02.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.540+0000 7f5a0e26c700 1 --2- 172.21.15.3:0/155705943 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5a0806acb0 0x7f5a0806b0f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5a00009dc0 tx=0x7f5a0000b330 comp rx=0 tx=0).stop 2023-10-03T10:14:02.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.540+0000 7f5a0e26c700 1 -- 172.21.15.3:0/155705943 shutdown_connections 2023-10-03T10:14:02.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.540+0000 7f5a0e26c700 1 --2- 172.21.15.3:0/155705943 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5a0806acb0 0x7f5a0806b0f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.540+0000 7f5a0e26c700 1 --2- 172.21.15.3:0/155705943 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5a081316c0 0x7f5a08131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.540+0000 7f5a0e26c700 1 --2- 172.21.15.3:0/155705943 >> v2:172.21.15.3:3300/0 conn(0x7f5a0809d9f0 0x7f5a0806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.540+0000 7f5a0e26c700 1 -- 172.21.15.3:0/155705943 >> 172.21.15.3:0/155705943 conn(0x7f5a08066f50 msgr2=0x7f5a0809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:02.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.540+0000 7f5a0e26c700 1 -- 172.21.15.3:0/155705943 shutdown_connections 2023-10-03T10:14:02.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.540+0000 7f5a0e26c700 1 -- 172.21.15.3:0/155705943 wait complete. 2023-10-03T10:14:02.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.541+0000 7f5a0e26c700 1 Processor -- start 2023-10-03T10:14:02.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.541+0000 7f5a0e26c700 1 -- start start 2023-10-03T10:14:02.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.542+0000 7f5a0e26c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5a0806acb0 0x7f5a081c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:02.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.542+0000 7f5a0e26c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a0809d9f0 0x7f5a081c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:02.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.542+0000 7f5a0e26c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5a081316c0 0x7f5a081c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:02.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.542+0000 7f5a0e26c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a08071e00 con 0x7f5a0809d9f0 2023-10-03T10:14:02.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.542+0000 7f5a0e26c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0812b900 con 0x7f5a0806acb0 2023-10-03T10:14:02.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.542+0000 7f5a0e26c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a08071420 con 0x7f5a081316c0 2023-10-03T10:14:02.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.542+0000 7f5a0c809700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5a081316c0 0x7f5a081c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:02.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.542+0000 7f5a077fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a0809d9f0 0x7f5a081c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:02.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.542+0000 7f5a0c809700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5a081316c0 0x7f5a081c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:44092/0 (socket says 172.21.15.3:44092) 2023-10-03T10:14:02.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.542+0000 7f5a0c809700 1 -- 172.21.15.3:0/918583084 learned_addr learned my addr 172.21.15.3:0/918583084 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:02.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.542+0000 7f5a07fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5a0806acb0 0x7f5a081c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:02.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.542+0000 7f5a077fe700 1 -- 172.21.15.3:0/918583084 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5a081316c0 msgr2=0x7f5a081c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:02.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.542+0000 7f5a077fe700 1 --2- 172.21.15.3:0/918583084 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5a081316c0 0x7f5a081c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.542+0000 7f5a077fe700 1 -- 172.21.15.3:0/918583084 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5a0806acb0 msgr2=0x7f5a081c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:02.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.542+0000 7f5a077fe700 1 --2- 172.21.15.3:0/918583084 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5a0806acb0 0x7f5a081c8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.542+0000 7f5a077fe700 1 -- 172.21.15.3:0/918583084 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59fc009550 con 0x7f5a0809d9f0 2023-10-03T10:14:02.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.542+0000 7f5a07fff700 1 --2- 172.21.15.3:0/918583084 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5a0806acb0 0x7f5a081c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:02.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.543+0000 7f5a077fe700 1 --2- 172.21.15.3:0/918583084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a0809d9f0 0x7f5a081c8870 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5a0809d810 tx=0x7f59fc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:02.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.543+0000 7f5a04ff9700 1 -- 172.21.15.3:0/918583084 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc004030 con 0x7f5a0809d9f0 2023-10-03T10:14:02.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.543+0000 7f5a04ff9700 1 -- 172.21.15.3:0/918583084 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59fc004190 con 0x7f5a0809d9f0 2023-10-03T10:14:02.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.543+0000 7f5a04ff9700 1 -- 172.21.15.3:0/918583084 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc004520 con 0x7f5a0809d9f0 2023-10-03T10:14:02.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.543+0000 7f5a0e26c700 1 -- 172.21.15.3:0/918583084 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a00009a70 con 0x7f5a0809d9f0 2023-10-03T10:14:02.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.543+0000 7f5a0e26c700 1 -- 172.21.15.3:0/918583084 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a08070e80 con 0x7f5a0809d9f0 2023-10-03T10:14:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.544+0000 7f5a04ff9700 1 -- 172.21.15.3:0/918583084 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f59fc0036a0 con 0x7f5a0809d9f0 2023-10-03T10:14:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.544+0000 7f5a04ff9700 1 -- 172.21.15.3:0/918583084 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f59fc03f4e0 con 0x7f5a0809d9f0 2023-10-03T10:14:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.544+0000 7f5a0e26c700 1 -- 172.21.15.3:0/918583084 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a08071070 con 0x7f5a0809d9f0 2023-10-03T10:14:02.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.549+0000 7f5a04ff9700 1 -- 172.21.15.3:0/918583084 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f59fc019d60 con 0x7f5a0809d9f0 2023-10-03T10:14:02.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.626+0000 7f175e3c5700 1 -- 172.21.15.3:0/2725072861 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f17581a8120 con 0x7f175809e930 2023-10-03T10:14:02.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.626+0000 7f1754ff9700 1 -- 172.21.15.3:0/2725072861 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f17581a8120 con 0x7f175809e930 2023-10-03T10:14:02.626 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:02.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.629+0000 7f175e3c5700 1 -- 172.21.15.3:0/2725072861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f175809e930 msgr2=0x7f17581a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:02.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.629+0000 7f175e3c5700 1 --2- 172.21.15.3:0/2725072861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f175809e930 0x7f17581a53e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f174800ec10 tx=0x7f174800ef20 comp rx=0 tx=0).stop 2023-10-03T10:14:02.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.629+0000 7f175e3c5700 1 -- 172.21.15.3:0/2725072861 shutdown_connections 2023-10-03T10:14:02.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.629+0000 7f175e3c5700 1 --2- 172.21.15.3:0/2725072861 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f175809f650 0x7f17581c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.629+0000 7f175e3c5700 1 --2- 172.21.15.3:0/2725072861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f175809e930 0x7f17581a53e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.629+0000 7f175e3c5700 1 --2- 172.21.15.3:0/2725072861 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f175809f240 0x7f17581a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.629+0000 7f175e3c5700 1 -- 172.21.15.3:0/2725072861 >> 172.21.15.3:0/2725072861 conn(0x7f17580a4210 msgr2=0x7f175806c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:02.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.630+0000 7f175e3c5700 1 -- 172.21.15.3:0/2725072861 shutdown_connections 2023-10-03T10:14:02.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.630+0000 7f175e3c5700 1 -- 172.21.15.3:0/2725072861 wait complete. 2023-10-03T10:14:02.648 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:02.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.737+0000 7f569ee82700 1 -- 172.21.15.3:0/4186105765 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f56980a21b0 con 0x7f569809e000 2023-10-03T10:14:02.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.738+0000 7f56957fa700 1 -- 172.21.15.3:0/4186105765 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5684011360 con 0x7f569809e000 2023-10-03T10:14:02.738 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:02.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.740+0000 7f569ee82700 1 -- 172.21.15.3:0/4186105765 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f569809e000 msgr2=0x7f569811e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:02.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.740+0000 7f569ee82700 1 --2- 172.21.15.3:0/4186105765 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f569809e000 0x7f569811e550 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f568400c550 tx=0x7f568400c860 comp rx=0 tx=0).stop 2023-10-03T10:14:02.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.740+0000 7f569ee82700 1 -- 172.21.15.3:0/4186105765 shutdown_connections 2023-10-03T10:14:02.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.740+0000 7f569ee82700 1 --2- 172.21.15.3:0/4186105765 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f569809e410 0x7f569811ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.740+0000 7f569ee82700 1 --2- 172.21.15.3:0/4186105765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f569809d6f0 0x7f569811e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.740+0000 7f569ee82700 1 --2- 172.21.15.3:0/4186105765 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f569809e000 0x7f569811e550 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.740+0000 7f569ee82700 1 -- 172.21.15.3:0/4186105765 >> 172.21.15.3:0/4186105765 conn(0x7f5698066f50 msgr2=0x7f5698070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:02.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.740+0000 7f569ee82700 1 -- 172.21.15.3:0/4186105765 shutdown_connections 2023-10-03T10:14:02.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.740+0000 7f569ee82700 1 -- 172.21.15.3:0/4186105765 wait complete. 2023-10-03T10:14:02.754 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:02.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.756+0000 7f5a0e26c700 1 -- 172.21.15.3:0/918583084 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5a08074ea0 con 0x7f5a0809d9f0 2023-10-03T10:14:02.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.757+0000 7f5a04ff9700 1 -- 172.21.15.3:0/918583084 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f59fc019300 con 0x7f5a0809d9f0 2023-10-03T10:14:02.757 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:02.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.759+0000 7f5a0e26c700 1 -- 172.21.15.3:0/918583084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a0809d9f0 msgr2=0x7f5a081c8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:02.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.759+0000 7f5a0e26c700 1 --2- 172.21.15.3:0/918583084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a0809d9f0 0x7f5a081c8870 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5a0809d810 tx=0x7f59fc00ec80 comp rx=0 tx=0).stop 2023-10-03T10:14:02.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.759+0000 7f5a0e26c700 1 -- 172.21.15.3:0/918583084 shutdown_connections 2023-10-03T10:14:02.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.759+0000 7f5a0e26c700 1 --2- 172.21.15.3:0/918583084 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5a081316c0 0x7f5a081c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.759+0000 7f5a0e26c700 1 --2- 172.21.15.3:0/918583084 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5a0806acb0 0x7f5a081c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.759+0000 7f5a0e26c700 1 --2- 172.21.15.3:0/918583084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a0809d9f0 0x7f5a081c8870 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:02.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.759+0000 7f5a0e26c700 1 -- 172.21.15.3:0/918583084 >> 172.21.15.3:0/918583084 conn(0x7f5a08066f50 msgr2=0x7f5a0812db20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:02.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.759+0000 7f5a0e26c700 1 -- 172.21.15.3:0/918583084 shutdown_connections 2023-10-03T10:14:02.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:02.759+0000 7f5a0e26c700 1 -- 172.21.15.3:0/918583084 wait complete. 2023-10-03T10:14:02.773 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:03.212 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:03.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.358+0000 7f807428a700 1 Processor -- start 2023-10-03T10:14:03.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.358+0000 7f807428a700 1 -- start start 2023-10-03T10:14:03.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.359+0000 7f807428a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f806c06a770 0x7f806c06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:03.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.359+0000 7f807428a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f806c06b080 0x7f806c131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:03.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.359+0000 7f807428a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f806c09ece0 con 0x7f806c131b40 2023-10-03T10:14:03.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.359+0000 7f807428a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f806c09e760 con 0x7f806c06b080 2023-10-03T10:14:03.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.359+0000 7f807428a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f806c09e040 con 0x7f806c06a770 2023-10-03T10:14:03.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.359+0000 7f8071825700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f806c06b080 0x7f806c131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:03.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.359+0000 7f8072026700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f806c06a770 0x7f806c06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:03.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.359+0000 7f8071825700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f806c06b080 0x7f806c131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57670/0 (socket says 172.21.15.3:57670) 2023-10-03T10:14:03.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.359+0000 7f8071825700 1 -- 172.21.15.3:0/2961360564 learned_addr learned my addr 172.21.15.3:0/2961360564 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:03.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.360+0000 7f8071825700 1 -- 172.21.15.3:0/2961360564 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f806c06a770 msgr2=0x7f806c06ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:03.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.360+0000 7f8071825700 1 --2- 172.21.15.3:0/2961360564 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f806c06a770 0x7f806c06ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:03.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.360+0000 7f8071825700 1 -- 172.21.15.3:0/2961360564 >> v1:172.21.15.3:6789/0 conn(0x7f806c131b40 legacy=0x7f806c138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:03.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.360+0000 7f8071825700 1 -- 172.21.15.3:0/2961360564 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f806c09f630 con 0x7f806c06b080 2023-10-03T10:14:03.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.360+0000 7f8072026700 1 --2- 172.21.15.3:0/2961360564 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f806c06a770 0x7f806c06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:03.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.360+0000 7f8071825700 1 --2- 172.21.15.3:0/2961360564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f806c06b080 0x7f806c131600 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f805c0098f0 tx=0x7f805c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d2907598cb630 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:03.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.360+0000 7f8071024700 1 -- 172.21.15.3:0/2961360564 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805c011830 con 0x7f806c06b080 2023-10-03T10:14:03.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.360+0000 7f8071024700 1 -- 172.21.15.3:0/2961360564 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f805c011990 con 0x7f806c06b080 2023-10-03T10:14:03.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.360+0000 7f8071024700 1 -- 172.21.15.3:0/2961360564 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805c011d20 con 0x7f806c06b080 2023-10-03T10:14:03.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.361+0000 7f807428a700 1 -- 172.21.15.3:0/2961360564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f806c06b080 msgr2=0x7f806c131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:03.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.361+0000 7f807428a700 1 --2- 172.21.15.3:0/2961360564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f806c06b080 0x7f806c131600 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f805c0098f0 tx=0x7f805c009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:03.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.361+0000 7f807428a700 1 -- 172.21.15.3:0/2961360564 shutdown_connections 2023-10-03T10:14:03.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.361+0000 7f807428a700 1 --2- 172.21.15.3:0/2961360564 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f806c06a770 0x7f806c06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:03.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.361+0000 7f807428a700 1 --2- 172.21.15.3:0/2961360564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f806c06b080 0x7f806c131600 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:03.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.361+0000 7f807428a700 1 -- 172.21.15.3:0/2961360564 >> 172.21.15.3:0/2961360564 conn(0x7f806c066f50 msgr2=0x7f806c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:03.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.361+0000 7f807428a700 1 -- 172.21.15.3:0/2961360564 shutdown_connections 2023-10-03T10:14:03.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.361+0000 7f807428a700 1 -- 172.21.15.3:0/2961360564 wait complete. 2023-10-03T10:14:03.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.362+0000 7f807428a700 1 Processor -- start 2023-10-03T10:14:03.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.362+0000 7f807428a700 1 -- start start 2023-10-03T10:14:03.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.362+0000 7f807428a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f806c06a770 0x7f806c1c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:03.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.362+0000 7f807428a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f806c06b080 0x7f806c1c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:03.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.362+0000 7f807428a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f806c131b40 0x7f806c1c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:03.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.362+0000 7f807428a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806c09c200 con 0x7f806c131b40 2023-10-03T10:14:03.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.362+0000 7f807428a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f806c09c010 con 0x7f806c06a770 2023-10-03T10:14:03.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.362+0000 7f807428a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f806c1313c0 con 0x7f806c06b080 2023-10-03T10:14:03.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.362+0000 7f8072026700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f806c06a770 0x7f806c1c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:03.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.362+0000 7f8072026700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f806c06a770 0x7f806c1c8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39774/0 (socket says 172.21.15.3:39774) 2023-10-03T10:14:03.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.362+0000 7f8072026700 1 -- 172.21.15.3:0/1161428091 learned_addr learned my addr 172.21.15.3:0/1161428091 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:03.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.362+0000 7f8071825700 1 --2- 172.21.15.3:0/1161428091 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f806c06b080 0x7f806c1c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:03.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.362+0000 7f8072827700 1 --2- 172.21.15.3:0/1161428091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f806c131b40 0x7f806c1c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:03.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.363+0000 7f8072026700 1 -- 172.21.15.3:0/1161428091 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f806c06b080 msgr2=0x7f806c1c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:03.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.363+0000 7f8072026700 1 --2- 172.21.15.3:0/1161428091 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f806c06b080 0x7f806c1c8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:03.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.363+0000 7f8072026700 1 -- 172.21.15.3:0/1161428091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f806c131b40 msgr2=0x7f806c1c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:03.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.363+0000 7f8072026700 1 --2- 172.21.15.3:0/1161428091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f806c131b40 0x7f806c1c8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:03.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.363+0000 7f8072026700 1 -- 172.21.15.3:0/1161428091 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80680094a0 con 0x7f806c06a770 2023-10-03T10:14:03.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.363+0000 7f8071825700 1 --2- 172.21.15.3:0/1161428091 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f806c06b080 0x7f806c1c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:03.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.363+0000 7f8072827700 1 --2- 172.21.15.3:0/1161428091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f806c131b40 0x7f806c1c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:03.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.363+0000 7f8072026700 1 --2- 172.21.15.3:0/1161428091 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f806c06a770 0x7f806c1c8220 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f806800ec10 tx=0x7f806800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:03.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.363+0000 7f8062ffd700 1 -- 172.21.15.3:0/1161428091 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806800c850 con 0x7f806c06a770 2023-10-03T10:14:03.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.363+0000 7f8062ffd700 1 -- 172.21.15.3:0/1161428091 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f806800c9b0 con 0x7f806c06a770 2023-10-03T10:14:03.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.363+0000 7f807428a700 1 -- 172.21.15.3:0/1161428091 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f805c009550 con 0x7f806c06a770 2023-10-03T10:14:03.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.363+0000 7f8062ffd700 1 -- 172.21.15.3:0/1161428091 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806800cd20 con 0x7f806c06a770 2023-10-03T10:14:03.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.363+0000 7f807428a700 1 -- 172.21.15.3:0/1161428091 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f806c121d70 con 0x7f806c06a770 2023-10-03T10:14:03.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.364+0000 7f8062ffd700 1 -- 172.21.15.3:0/1161428091 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8068013470 con 0x7f806c06a770 2023-10-03T10:14:03.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.364+0000 7f8062ffd700 1 -- 172.21.15.3:0/1161428091 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f806803fb50 con 0x7f806c06a770 2023-10-03T10:14:03.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.364+0000 7f807428a700 1 -- 172.21.15.3:0/1161428091 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f806c071e60 con 0x7f806c06a770 2023-10-03T10:14:03.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.369+0000 7f8062ffd700 1 -- 172.21.15.3:0/1161428091 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8068013720 con 0x7f806c06a770 2023-10-03T10:14:03.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.571+0000 7f807428a700 1 -- 172.21.15.3:0/1161428091 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f806c131630 con 0x7f806c06a770 2023-10-03T10:14:03.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.572+0000 7f8062ffd700 1 -- 172.21.15.3:0/1161428091 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8068011ab0 con 0x7f806c06a770 2023-10-03T10:14:03.572 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:03.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.574+0000 7f807428a700 1 -- 172.21.15.3:0/1161428091 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f806c06a770 msgr2=0x7f806c1c8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:03.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.574+0000 7f807428a700 1 --2- 172.21.15.3:0/1161428091 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f806c06a770 0x7f806c1c8220 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f806800ec10 tx=0x7f806800ef20 comp rx=0 tx=0).stop 2023-10-03T10:14:03.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.574+0000 7f807428a700 1 -- 172.21.15.3:0/1161428091 shutdown_connections 2023-10-03T10:14:03.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.574+0000 7f807428a700 1 --2- 172.21.15.3:0/1161428091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f806c131b40 0x7f806c1c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:03.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.574+0000 7f807428a700 1 --2- 172.21.15.3:0/1161428091 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f806c06a770 0x7f806c1c8220 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:03.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.574+0000 7f807428a700 1 --2- 172.21.15.3:0/1161428091 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f806c06b080 0x7f806c1c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:03.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.574+0000 7f807428a700 1 -- 172.21.15.3:0/1161428091 >> 172.21.15.3:0/1161428091 conn(0x7f806c066f50 msgr2=0x7f806c0a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:03.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.574+0000 7f807428a700 1 -- 172.21.15.3:0/1161428091 shutdown_connections 2023-10-03T10:14:03.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.574+0000 7f807428a700 1 -- 172.21.15.3:0/1161428091 wait complete. 2023-10-03T10:14:03.588 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:03.650 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:03.755 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:03.773 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:03.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.793+0000 7f42bba73700 1 Processor -- start 2023-10-03T10:14:03.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.793+0000 7f42bba73700 1 -- start start 2023-10-03T10:14:03.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.794+0000 7f42bba73700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f42b40a0a30 0x7f42b40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:03.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.794+0000 7f42bba73700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f42b40a1340 0x7f42b4133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:03.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.794+0000 7f42bba73700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f42b4133fa0 0x7f42b4134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:03.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.794+0000 7f42bba73700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42b409edd0 con 0x7f42b4133fa0 2023-10-03T10:14:03.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.794+0000 7f42bba73700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b409e1f0 con 0x7f42b40a0a30 2023-10-03T10:14:03.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.794+0000 7f42bba73700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f42b409dc50 con 0x7f42b40a1340 2023-10-03T10:14:03.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.794+0000 7f42b900e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f42b40a1340 0x7f42b4133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:03.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.794+0000 7f42b980f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f42b40a0a30 0x7f42b40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:03.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.794+0000 7f42b980f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f42b40a0a30 0x7f42b40a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39776/0 (socket says 172.21.15.3:39776) 2023-10-03T10:14:03.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.794+0000 7f42b900e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f42b40a1340 0x7f42b4133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:39974/0 (socket says 172.21.15.3:39974) 2023-10-03T10:14:03.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.794+0000 7f42b980f700 1 -- 172.21.15.3:0/232934906 learned_addr learned my addr 172.21.15.3:0/232934906 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:03.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.794+0000 7f42ba010700 1 --2- 172.21.15.3:0/232934906 >> v2:172.21.15.3:3300/0 conn(0x7f42b4133fa0 0x7f42b4134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:03.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.794+0000 7f42b980f700 1 -- 172.21.15.3:0/232934906 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f42b40a1340 msgr2=0x7f42b4133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:03.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.794+0000 7f42b980f700 1 --2- 172.21.15.3:0/232934906 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f42b40a1340 0x7f42b4133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:03.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.794+0000 7f42b980f700 1 -- 172.21.15.3:0/232934906 >> v2:172.21.15.3:3300/0 conn(0x7f42b4133fa0 msgr2=0x7f42b4134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:03.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.794+0000 7f42b980f700 1 --2- 172.21.15.3:0/232934906 >> v2:172.21.15.3:3300/0 conn(0x7f42b4133fa0 0x7f42b4134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:03.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.794+0000 7f42b980f700 1 -- 172.21.15.3:0/232934906 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b40a1fe0 con 0x7f42b40a0a30 2023-10-03T10:14:03.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.794+0000 7f42b900e700 1 --2- 172.21.15.3:0/232934906 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f42b40a1340 0x7f42b4133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:03.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.794+0000 7f42ba010700 1 --2- 172.21.15.3:0/232934906 >> v2:172.21.15.3:3300/0 conn(0x7f42b4133fa0 0x7f42b4134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:03.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.794+0000 7f42b980f700 1 --2- 172.21.15.3:0/232934906 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f42b40a0a30 0x7f42b40a0e00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f42b00098f0 tx=0x7f42b0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=968461754025b267 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:03.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.795+0000 7f42b880d700 1 -- 172.21.15.3:0/232934906 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b0011830 con 0x7f42b40a0a30 2023-10-03T10:14:03.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.795+0000 7f42b880d700 1 -- 172.21.15.3:0/232934906 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b0011990 con 0x7f42b40a0a30 2023-10-03T10:14:03.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.795+0000 7f42b880d700 1 -- 172.21.15.3:0/232934906 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b0011d20 con 0x7f42b40a0a30 2023-10-03T10:14:03.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.795+0000 7f42bba73700 1 -- 172.21.15.3:0/232934906 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f42b40a0a30 msgr2=0x7f42b40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:03.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.795+0000 7f42bba73700 1 --2- 172.21.15.3:0/232934906 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f42b40a0a30 0x7f42b40a0e00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f42b00098f0 tx=0x7f42b0009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:03.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.795+0000 7f42bba73700 1 -- 172.21.15.3:0/232934906 shutdown_connections 2023-10-03T10:14:03.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.795+0000 7f42bba73700 1 --2- 172.21.15.3:0/232934906 >> v2:172.21.15.3:3300/0 conn(0x7f42b4133fa0 0x7f42b4134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:03.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.795+0000 7f42bba73700 1 --2- 172.21.15.3:0/232934906 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f42b40a0a30 0x7f42b40a0e00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:03.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.795+0000 7f42bba73700 1 --2- 172.21.15.3:0/232934906 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f42b40a1340 0x7f42b4133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:03.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.795+0000 7f42bba73700 1 -- 172.21.15.3:0/232934906 >> 172.21.15.3:0/232934906 conn(0x7f42b40a6550 msgr2=0x7f42b40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:03.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.796+0000 7f42bba73700 1 -- 172.21.15.3:0/232934906 shutdown_connections 2023-10-03T10:14:03.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.796+0000 7f42bba73700 1 -- 172.21.15.3:0/232934906 wait complete. 2023-10-03T10:14:03.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.797+0000 7f42bba73700 1 Processor -- start 2023-10-03T10:14:03.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.797+0000 7f42bba73700 1 -- start start 2023-10-03T10:14:03.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.797+0000 7f42bba73700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f42b40a0a30 0x7f42b41200a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:03.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.797+0000 7f42bba73700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f42b40a1340 0x7f42b41205e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:03.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.797+0000 7f42bba73700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42b4133fa0 0x7f42b4120b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:03.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.797+0000 7f42bba73700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b4123bc0 con 0x7f42b4133fa0 2023-10-03T10:14:03.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.797+0000 7f42bba73700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b4122f90 con 0x7f42b40a1340 2023-10-03T10:14:03.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.797+0000 7f42bba73700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f42b40a34f0 con 0x7f42b40a0a30 2023-10-03T10:14:03.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.797+0000 7f42b900e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f42b40a1340 0x7f42b41205e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:03.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.797+0000 7f42b900e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f42b40a1340 0x7f42b41205e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39786/0 (socket says 172.21.15.3:39786) 2023-10-03T10:14:03.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.797+0000 7f42b900e700 1 -- 172.21.15.3:0/257967050 learned_addr learned my addr 172.21.15.3:0/257967050 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:03.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.797+0000 7f42b980f700 1 --2- 172.21.15.3:0/257967050 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f42b40a0a30 0x7f42b41200a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:03.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.797+0000 7f42ba010700 1 --2- 172.21.15.3:0/257967050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42b4133fa0 0x7f42b4120b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:03.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.798+0000 7f42b900e700 1 -- 172.21.15.3:0/257967050 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f42b40a0a30 msgr2=0x7f42b41200a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:03.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.798+0000 7f42b900e700 1 --2- 172.21.15.3:0/257967050 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f42b40a0a30 0x7f42b41200a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:03.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.798+0000 7f42b900e700 1 -- 172.21.15.3:0/257967050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42b4133fa0 msgr2=0x7f42b4120b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:03.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.798+0000 7f42b900e700 1 --2- 172.21.15.3:0/257967050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42b4133fa0 0x7f42b4120b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:03.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.798+0000 7f42b900e700 1 -- 172.21.15.3:0/257967050 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a40094a0 con 0x7f42b40a1340 2023-10-03T10:14:03.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.798+0000 7f42b900e700 1 --2- 172.21.15.3:0/257967050 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f42b40a1340 0x7f42b41205e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f42a400ebe0 tx=0x7f42a400eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:03.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.798+0000 7f42aa7fc700 1 -- 172.21.15.3:0/257967050 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a4009800 con 0x7f42b40a1340 2023-10-03T10:14:03.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.798+0000 7f42aa7fc700 1 -- 172.21.15.3:0/257967050 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a400c850 con 0x7f42b40a1340 2023-10-03T10:14:03.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.798+0000 7f42aa7fc700 1 -- 172.21.15.3:0/257967050 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a400cbc0 con 0x7f42b40a1340 2023-10-03T10:14:03.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.798+0000 7f42bba73700 1 -- 172.21.15.3:0/257967050 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42ac009a70 con 0x7f42b40a1340 2023-10-03T10:14:03.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.798+0000 7f42bba73700 1 -- 172.21.15.3:0/257967050 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42b0009550 con 0x7f42b40a1340 2023-10-03T10:14:03.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.799+0000 7f42aa7fc700 1 -- 172.21.15.3:0/257967050 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f42a400cd20 con 0x7f42b40a1340 2023-10-03T10:14:03.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.799+0000 7f42aa7fc700 1 -- 172.21.15.3:0/257967050 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f42a403fb80 con 0x7f42b40a1340 2023-10-03T10:14:03.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.800+0000 7f42bba73700 1 -- 172.21.15.3:0/257967050 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42b4121d10 con 0x7f42b40a1340 2023-10-03T10:14:03.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.805+0000 7f42aa7fc700 1 -- 172.21.15.3:0/257967050 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f42a401b3b0 con 0x7f42b40a1340 2023-10-03T10:14:03.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.898+0000 7fce41913700 1 Processor -- start 2023-10-03T10:14:03.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.898+0000 7fce41913700 1 -- start start 2023-10-03T10:14:03.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.899+0000 7fce41913700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fce3c09d630 0x7fce3c0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:03.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.899+0000 7fce41913700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce3c138d80 0x7fce3c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:03.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.899+0000 7fce41913700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce3c0a2a70 con 0x7fce3c0a0a30 2023-10-03T10:14:03.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.899+0000 7fce41913700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce3c0a01d0 con 0x7fce3c09d630 2023-10-03T10:14:03.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.899+0000 7fce41913700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c09e2b0 con 0x7fce3c138d80 2023-10-03T10:14:03.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.899+0000 7fce3b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce3c138d80 0x7fce3c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:03.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.899+0000 7fce3b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce3c138d80 0x7fce3c13b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39796/0 (socket says 172.21.15.3:39796) 2023-10-03T10:14:03.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.899+0000 7fce3b7fe700 1 -- 172.21.15.3:0/416601954 learned_addr learned my addr 172.21.15.3:0/416601954 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:03.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.899+0000 7fce33fff700 1 --2- 172.21.15.3:0/416601954 >> v2:172.21.15.3:3300/0 conn(0x7fce3c09d630 0x7fce3c0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:03.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.900+0000 7fce33fff700 1 -- 172.21.15.3:0/416601954 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce3c138d80 msgr2=0x7fce3c13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:03.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.900+0000 7fce33fff700 1 --2- 172.21.15.3:0/416601954 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce3c138d80 0x7fce3c13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:03.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.900+0000 7fce33fff700 1 -- 172.21.15.3:0/416601954 >> v1:172.21.15.3:6789/0 conn(0x7fce3c0a0a30 legacy=0x7fce3c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:03.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.900+0000 7fce33fff700 1 -- 172.21.15.3:0/416601954 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3c09dc50 con 0x7fce3c09d630 2023-10-03T10:14:03.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.900+0000 7fce3b7fe700 1 --2- 172.21.15.3:0/416601954 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce3c138d80 0x7fce3c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:03.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.900+0000 7fce33fff700 1 --2- 172.21.15.3:0/416601954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce3c09d630 0x7fce3c0a1510 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fce24009840 tx=0x7fce24009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d242421ab2ad01 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:03.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.900+0000 7fce3a7fc700 1 -- 172.21.15.3:0/416601954 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce24011830 con 0x7fce3c09d630 2023-10-03T10:14:03.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.900+0000 7fce3a7fc700 1 -- 172.21.15.3:0/416601954 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce24011990 con 0x7fce3c09d630 2023-10-03T10:14:03.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.900+0000 7fce3a7fc700 1 -- 172.21.15.3:0/416601954 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce24011d20 con 0x7fce3c09d630 2023-10-03T10:14:03.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.901+0000 7fce41913700 1 -- 172.21.15.3:0/416601954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce3c09d630 msgr2=0x7fce3c0a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:03.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.901+0000 7fce41913700 1 --2- 172.21.15.3:0/416601954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce3c09d630 0x7fce3c0a1510 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fce24009840 tx=0x7fce24009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:03.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.901+0000 7fce41913700 1 -- 172.21.15.3:0/416601954 shutdown_connections 2023-10-03T10:14:03.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.901+0000 7fce41913700 1 --2- 172.21.15.3:0/416601954 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce3c138d80 0x7fce3c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:03.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.901+0000 7fce41913700 1 --2- 172.21.15.3:0/416601954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce3c09d630 0x7fce3c0a1510 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:03.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.901+0000 7fce41913700 1 -- 172.21.15.3:0/416601954 >> 172.21.15.3:0/416601954 conn(0x7fce3c0a3050 msgr2=0x7fce3c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:03.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.901+0000 7fce41913700 1 -- 172.21.15.3:0/416601954 shutdown_connections 2023-10-03T10:14:03.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.901+0000 7fce41913700 1 -- 172.21.15.3:0/416601954 wait complete. 2023-10-03T10:14:03.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.902+0000 7fce41913700 1 Processor -- start 2023-10-03T10:14:03.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.902+0000 7fce41913700 1 -- start start 2023-10-03T10:14:03.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.902+0000 7fce41913700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce3c09d630 0x7fce3c120800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:03.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.902+0000 7fce41913700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce3c0a0a30 0x7fce3c120d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:03.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.902+0000 7fce41913700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce3c138d80 0x7fce3c121280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:03.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.902+0000 7fce41913700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c123fb0 con 0x7fce3c138d80 2023-10-03T10:14:03.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.902+0000 7fce41913700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c0a4e00 con 0x7fce3c09d630 2023-10-03T10:14:03.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.902+0000 7fce41913700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c0a0410 con 0x7fce3c0a0a30 2023-10-03T10:14:03.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.903+0000 7fce3affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce3c09d630 0x7fce3c120800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:03.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.903+0000 7fce3affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce3c09d630 0x7fce3c120800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39808/0 (socket says 172.21.15.3:39808) 2023-10-03T10:14:03.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.903+0000 7fce3affd700 1 -- 172.21.15.3:0/2460196959 learned_addr learned my addr 172.21.15.3:0/2460196959 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:03.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.903+0000 7fce3b7fe700 1 --2- 172.21.15.3:0/2460196959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce3c138d80 0x7fce3c121280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:03.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.903+0000 7fce33fff700 1 --2- 172.21.15.3:0/2460196959 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce3c0a0a30 0x7fce3c120d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:03.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.903+0000 7fce3b7fe700 1 -- 172.21.15.3:0/2460196959 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce3c0a0a30 msgr2=0x7fce3c120d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:03.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.903+0000 7fce3b7fe700 1 --2- 172.21.15.3:0/2460196959 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce3c0a0a30 0x7fce3c120d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:03.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.903+0000 7fce3b7fe700 1 -- 172.21.15.3:0/2460196959 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce3c09d630 msgr2=0x7fce3c120800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:03.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.903+0000 7fce3b7fe700 1 --2- 172.21.15.3:0/2460196959 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce3c09d630 0x7fce3c120800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:03.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.903+0000 7fce3b7fe700 1 -- 172.21.15.3:0/2460196959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce34009b30 con 0x7fce3c138d80 2023-10-03T10:14:03.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.903+0000 7fce3affd700 1 --2- 172.21.15.3:0/2460196959 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce3c09d630 0x7fce3c120800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:03.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.903+0000 7fce3b7fe700 1 --2- 172.21.15.3:0/2460196959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce3c138d80 0x7fce3c121280 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fce3400ef90 tx=0x7fce3400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:03.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.903+0000 7fce38ff9700 1 -- 172.21.15.3:0/2460196959 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3400cb30 con 0x7fce3c138d80 2023-10-03T10:14:03.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.903+0000 7fce38ff9700 1 -- 172.21.15.3:0/2460196959 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3400f040 con 0x7fce3c138d80 2023-10-03T10:14:03.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.903+0000 7fce41913700 1 -- 172.21.15.3:0/2460196959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce240094a0 con 0x7fce3c138d80 2023-10-03T10:14:03.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.903+0000 7fce38ff9700 1 -- 172.21.15.3:0/2460196959 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce34014480 con 0x7fce3c138d80 2023-10-03T10:14:03.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.903+0000 7fce41913700 1 -- 172.21.15.3:0/2460196959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce3c124a20 con 0x7fce3c138d80 2023-10-03T10:14:03.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.905+0000 7fce38ff9700 1 -- 172.21.15.3:0/2460196959 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fce340145e0 con 0x7fce3c138d80 2023-10-03T10:14:03.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.905+0000 7fce41913700 1 -- 172.21.15.3:0/2460196959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce3c0747f0 con 0x7fce3c138d80 2023-10-03T10:14:03.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.905+0000 7fce38ff9700 1 -- 172.21.15.3:0/2460196959 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fce3403fbd0 con 0x7fce3c138d80 2023-10-03T10:14:03.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.910+0000 7fce38ff9700 1 -- 172.21.15.3:0/2460196959 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fce34014d00 con 0x7fce3c138d80 2023-10-03T10:14:03.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.938+0000 7f73d0d50700 1 Processor -- start 2023-10-03T10:14:03.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.938+0000 7f73d0d50700 1 -- start start 2023-10-03T10:14:03.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.939+0000 7f73d0d50700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73cc09f410 0x7f73cc136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:03.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.939+0000 7f73d0d50700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73cc136fe0 0x7f73cc1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:03.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.939+0000 7f73d0d50700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73cc070940 con 0x7f73cc09e930 2023-10-03T10:14:03.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.939+0000 7f73d0d50700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc06fd60 con 0x7f73cc136fe0 2023-10-03T10:14:03.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.939+0000 7f73d0d50700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc06fa80 con 0x7f73cc09f410 2023-10-03T10:14:03.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.939+0000 7f73ca59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f73cc09e930 0x7f73cc09ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54254/0 (socket says 172.21.15.3:54254) 2023-10-03T10:14:03.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.939+0000 7f73ca59c700 1 -- 172.21.15.3:0/1532736115 learned_addr learned my addr 172.21.15.3:0/1532736115 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:03.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.939+0000 7f73c9d9b700 1 --2- 172.21.15.3:0/1532736115 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73cc09f410 0x7f73cc136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:03.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.939+0000 7f73cad9d700 1 --2- 172.21.15.3:0/1532736115 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73cc136fe0 0x7f73cc1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:03.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.939+0000 7f73c959a700 1 -- 172.21.15.3:0/1532736115 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1356406095 0 0) 0x7f73cc070940 con 0x7f73cc09e930 2023-10-03T10:14:03.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.940+0000 7f73c9d9b700 1 -- 172.21.15.3:0/1532736115 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73cc136fe0 msgr2=0x7f73cc1393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:03.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.940+0000 7f73c9d9b700 1 --2- 172.21.15.3:0/1532736115 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73cc136fe0 0x7f73cc1393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:03.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.940+0000 7f73c9d9b700 1 -- 172.21.15.3:0/1532736115 >> v1:172.21.15.3:6789/0 conn(0x7f73cc09e930 legacy=0x7f73cc09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:03.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.940+0000 7f73c9d9b700 1 -- 172.21.15.3:0/1532736115 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73cc09fb60 con 0x7f73cc09f410 2023-10-03T10:14:03.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.940+0000 7f73c9d9b700 1 --2- 172.21.15.3:0/1532736115 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73cc09f410 0x7f73cc136aa0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009870 tx=0x7f73c0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14ac5cf0ad628786 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:03.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.940+0000 7f73c959a700 1 -- 172.21.15.3:0/1532736115 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c0011830 con 0x7f73cc09f410 2023-10-03T10:14:03.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.940+0000 7f73c959a700 1 -- 172.21.15.3:0/1532736115 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c0011990 con 0x7f73cc09f410 2023-10-03T10:14:03.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.940+0000 7f73c959a700 1 -- 172.21.15.3:0/1532736115 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c0011d20 con 0x7f73cc09f410 2023-10-03T10:14:03.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.941+0000 7f73d0d50700 1 -- 172.21.15.3:0/1532736115 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73cc09f410 msgr2=0x7f73cc136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:03.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.941+0000 7f73d0d50700 1 --2- 172.21.15.3:0/1532736115 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73cc09f410 0x7f73cc136aa0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009870 tx=0x7f73c0009b80 comp rx=0 tx=0).stop 2023-10-03T10:14:03.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.941+0000 7f73d0d50700 1 -- 172.21.15.3:0/1532736115 shutdown_connections 2023-10-03T10:14:03.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.941+0000 7f73d0d50700 1 --2- 172.21.15.3:0/1532736115 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73cc136fe0 0x7f73cc1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:03.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.941+0000 7f73d0d50700 1 --2- 172.21.15.3:0/1532736115 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73cc09f410 0x7f73cc136aa0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:03.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.941+0000 7f73d0d50700 1 -- 172.21.15.3:0/1532736115 >> 172.21.15.3:0/1532736115 conn(0x7f73cc0a4210 msgr2=0x7f73cc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:03.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.941+0000 7f73d0d50700 1 -- 172.21.15.3:0/1532736115 shutdown_connections 2023-10-03T10:14:03.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.941+0000 7f73d0d50700 1 -- 172.21.15.3:0/1532736115 wait complete. 2023-10-03T10:14:03.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.942+0000 7f73d0d50700 1 Processor -- start 2023-10-03T10:14:03.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.942+0000 7f73d0d50700 1 -- start start 2023-10-03T10:14:03.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.943+0000 7f73d0d50700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73cc09e930 0x7f73cc11f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:03.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.943+0000 7f73d0d50700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73cc09f410 0x7f73cc11fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:03.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.943+0000 7f73d0d50700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73cc136fe0 0x7f73cc120050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:03.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.943+0000 7f73d0d50700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc134f00 con 0x7f73cc09e930 2023-10-03T10:14:03.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.943+0000 7f73d0d50700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc1354c0 con 0x7f73cc136fe0 2023-10-03T10:14:03.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.943+0000 7f73d0d50700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc09df30 con 0x7f73cc09f410 2023-10-03T10:14:03.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.943+0000 7f73ca59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73cc09e930 0x7f73cc11f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:03.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.943+0000 7f73ca59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73cc09e930 0x7f73cc11f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57730/0 (socket says 172.21.15.3:57730) 2023-10-03T10:14:03.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.943+0000 7f73ca59c700 1 -- 172.21.15.3:0/984589648 learned_addr learned my addr 172.21.15.3:0/984589648 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:03.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.943+0000 7f73cad9d700 1 --2- 172.21.15.3:0/984589648 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73cc136fe0 0x7f73cc120050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:03.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.943+0000 7f73c9d9b700 1 --2- 172.21.15.3:0/984589648 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73cc09f410 0x7f73cc11fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:03.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.944+0000 7f73ca59c700 1 -- 172.21.15.3:0/984589648 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73cc09f410 msgr2=0x7f73cc11fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:03.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.944+0000 7f73ca59c700 1 --2- 172.21.15.3:0/984589648 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73cc09f410 0x7f73cc11fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:03.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.944+0000 7f73ca59c700 1 -- 172.21.15.3:0/984589648 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73cc136fe0 msgr2=0x7f73cc120050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:03.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.944+0000 7f73ca59c700 1 --2- 172.21.15.3:0/984589648 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73cc136fe0 0x7f73cc120050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:03.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.944+0000 7f73ca59c700 1 -- 172.21.15.3:0/984589648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c00094a0 con 0x7f73cc09e930 2023-10-03T10:14:03.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.944+0000 7f73ca59c700 1 --2- 172.21.15.3:0/984589648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73cc09e930 0x7f73cc11f5d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00c520 tx=0x7f73bc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:03.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.944+0000 7f73baffd700 1 -- 172.21.15.3:0/984589648 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73bc004c80 con 0x7f73cc09e930 2023-10-03T10:14:03.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.944+0000 7f73d0d50700 1 -- 172.21.15.3:0/984589648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73cc071d20 con 0x7f73cc09e930 2023-10-03T10:14:03.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.944+0000 7f73baffd700 1 -- 172.21.15.3:0/984589648 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73bc004e00 con 0x7f73cc09e930 2023-10-03T10:14:03.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.944+0000 7f73baffd700 1 -- 172.21.15.3:0/984589648 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73bc007310 con 0x7f73cc09e930 2023-10-03T10:14:03.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.944+0000 7f73d0d50700 1 -- 172.21.15.3:0/984589648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73cc122e10 con 0x7f73cc09e930 2023-10-03T10:14:03.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.945+0000 7f73baffd700 1 -- 172.21.15.3:0/984589648 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f73bc007470 con 0x7f73cc09e930 2023-10-03T10:14:03.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.945+0000 7f73baffd700 1 -- 172.21.15.3:0/984589648 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f73bc03f490 con 0x7f73cc09e930 2023-10-03T10:14:03.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.945+0000 7f73d0d50700 1 -- 172.21.15.3:0/984589648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73cc123940 con 0x7f73cc09e930 2023-10-03T10:14:03.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:03.951+0000 7f73baffd700 1 -- 172.21.15.3:0/984589648 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f73bc0123b0 con 0x7f73cc09e930 2023-10-03T10:14:04.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.020+0000 7f42bba73700 1 -- 172.21.15.3:0/257967050 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f42b4074ea0 con 0x7f42b40a1340 2023-10-03T10:14:04.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.021+0000 7f42aa7fc700 1 -- 172.21.15.3:0/257967050 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f42a401ab10 con 0x7f42b40a1340 2023-10-03T10:14:04.021 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:04.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.023+0000 7f42bba73700 1 -- 172.21.15.3:0/257967050 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f42b40a1340 msgr2=0x7f42b41205e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:04.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.023+0000 7f42bba73700 1 --2- 172.21.15.3:0/257967050 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f42b40a1340 0x7f42b41205e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f42a400ebe0 tx=0x7f42a400eef0 comp rx=0 tx=0).stop 2023-10-03T10:14:04.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.023+0000 7f42bba73700 1 -- 172.21.15.3:0/257967050 shutdown_connections 2023-10-03T10:14:04.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.023+0000 7f42bba73700 1 --2- 172.21.15.3:0/257967050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42b4133fa0 0x7f42b4120b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:04.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.023+0000 7f42bba73700 1 --2- 172.21.15.3:0/257967050 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f42b40a0a30 0x7f42b41200a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:04.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.023+0000 7f42bba73700 1 --2- 172.21.15.3:0/257967050 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f42b40a1340 0x7f42b41205e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:04.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.023+0000 7f42bba73700 1 -- 172.21.15.3:0/257967050 >> 172.21.15.3:0/257967050 conn(0x7f42b40a6550 msgr2=0x7f42b41bae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:04.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.023+0000 7f42bba73700 1 -- 172.21.15.3:0/257967050 shutdown_connections 2023-10-03T10:14:04.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.023+0000 7f42bba73700 1 -- 172.21.15.3:0/257967050 wait complete. 2023-10-03T10:14:04.037 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:04.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.117+0000 7fce41913700 1 -- 172.21.15.3:0/2460196959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fce3c072980 con 0x7fce3c138d80 2023-10-03T10:14:04.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.117+0000 7fce38ff9700 1 -- 172.21.15.3:0/2460196959 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fce34012b30 con 0x7fce3c138d80 2023-10-03T10:14:04.118 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:04.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.119+0000 7fce41913700 1 -- 172.21.15.3:0/2460196959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce3c138d80 msgr2=0x7fce3c121280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:04.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.119+0000 7fce41913700 1 --2- 172.21.15.3:0/2460196959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce3c138d80 0x7fce3c121280 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fce3400ef90 tx=0x7fce3400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:14:04.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.119+0000 7fce41913700 1 -- 172.21.15.3:0/2460196959 shutdown_connections 2023-10-03T10:14:04.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.119+0000 7fce41913700 1 --2- 172.21.15.3:0/2460196959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce3c138d80 0x7fce3c121280 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:04.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.119+0000 7fce41913700 1 --2- 172.21.15.3:0/2460196959 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce3c09d630 0x7fce3c120800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:04.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.119+0000 7fce41913700 1 --2- 172.21.15.3:0/2460196959 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce3c0a0a30 0x7fce3c120d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:04.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.119+0000 7fce41913700 1 -- 172.21.15.3:0/2460196959 >> 172.21.15.3:0/2460196959 conn(0x7fce3c0a3050 msgr2=0x7fce3c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:04.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.119+0000 7fce41913700 1 -- 172.21.15.3:0/2460196959 shutdown_connections 2023-10-03T10:14:04.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.120+0000 7fce41913700 1 -- 172.21.15.3:0/2460196959 wait complete. 2023-10-03T10:14:04.133 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:04.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.159+0000 7f73d0d50700 1 -- 172.21.15.3:0/984589648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f73cc074ea0 con 0x7f73cc09e930 2023-10-03T10:14:04.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.159+0000 7f73baffd700 1 -- 172.21.15.3:0/984589648 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f73bc011360 con 0x7f73cc09e930 2023-10-03T10:14:04.159 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:04.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.162+0000 7f73d0d50700 1 -- 172.21.15.3:0/984589648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73cc09e930 msgr2=0x7f73cc11f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:04.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.162+0000 7f73d0d50700 1 --2- 172.21.15.3:0/984589648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73cc09e930 0x7f73cc11f5d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00c520 tx=0x7f73bc00c830 comp rx=0 tx=0).stop 2023-10-03T10:14:04.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.162+0000 7f73d0d50700 1 -- 172.21.15.3:0/984589648 shutdown_connections 2023-10-03T10:14:04.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.162+0000 7f73d0d50700 1 --2- 172.21.15.3:0/984589648 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73cc136fe0 0x7f73cc120050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:04.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.162+0000 7f73d0d50700 1 --2- 172.21.15.3:0/984589648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73cc09e930 0x7f73cc11f5d0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:04.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.162+0000 7f73d0d50700 1 --2- 172.21.15.3:0/984589648 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73cc09f410 0x7f73cc11fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:04.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.162+0000 7f73d0d50700 1 -- 172.21.15.3:0/984589648 >> 172.21.15.3:0/984589648 conn(0x7f73cc0a4210 msgr2=0x7f73cc135b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:04.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.162+0000 7f73d0d50700 1 -- 172.21.15.3:0/984589648 shutdown_connections 2023-10-03T10:14:04.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.162+0000 7f73d0d50700 1 -- 172.21.15.3:0/984589648 wait complete. 2023-10-03T10:14:04.175 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:04.589 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:04.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.736+0000 7fb7bf976700 1 Processor -- start 2023-10-03T10:14:04.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.736+0000 7fb7bf976700 1 -- start start 2023-10-03T10:14:04.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.737+0000 7fb7bf976700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7b809e1d0 0x7fb7b8138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:04.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.737+0000 7fb7bf976700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb7b8139170 0x7fb7b813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:04.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.737+0000 7fb7bf976700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7b809d2f0 con 0x7fb7b809d6f0 2023-10-03T10:14:04.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.737+0000 7fb7bf976700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7b809b0f0 con 0x7fb7b8139170 2023-10-03T10:14:04.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.737+0000 7fb7bf976700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b809a810 con 0x7fb7b809e1d0 2023-10-03T10:14:04.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.737+0000 7fb7bcf11700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7b809e1d0 0x7fb7b8138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:04.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.737+0000 7fb7bdf13700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb7b8139170 0x7fb7b813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:04.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.737+0000 7fb7bcf11700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7b809e1d0 0x7fb7b8138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:40008/0 (socket says 172.21.15.3:40008) 2023-10-03T10:14:04.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.737+0000 7fb7bcf11700 1 -- 172.21.15.3:0/1901849121 learned_addr learned my addr 172.21.15.3:0/1901849121 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:04.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.737+0000 7fb7bdf13700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb7b8139170 0x7fb7b813b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57732/0 (socket says 172.21.15.3:57732) 2023-10-03T10:14:04.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.737+0000 7fb7bdf13700 1 -- 172.21.15.3:0/1901849121 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7b809e1d0 msgr2=0x7fb7b8138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:04.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.737+0000 7fb7bdf13700 1 --2- 172.21.15.3:0/1901849121 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7b809e1d0 0x7fb7b8138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:04.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.737+0000 7fb7bdf13700 1 -- 172.21.15.3:0/1901849121 >> v1:172.21.15.3:6789/0 conn(0x7fb7b809d6f0 legacy=0x7fb7b809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:04.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.737+0000 7fb7bdf13700 1 -- 172.21.15.3:0/1901849121 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b809a470 con 0x7fb7b8139170 2023-10-03T10:14:04.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.737+0000 7fb7affff700 1 -- 172.21.15.3:0/1901849121 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1244664256 0 0) 0x7fb7b809d2f0 con 0x7fb7b809d6f0 2023-10-03T10:14:04.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.737+0000 7fb7bcf11700 1 --2- 172.21.15.3:0/1901849121 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7b809e1d0 0x7fb7b8138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:04.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.737+0000 7fb7bdf13700 1 --2- 172.21.15.3:0/1901849121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7b8139170 0x7fb7b813b550 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009d90 tx=0x7fb7b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f19803eca35abd server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:04.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.737+0000 7fb7affff700 1 -- 172.21.15.3:0/1901849121 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b000e070 con 0x7fb7b8139170 2023-10-03T10:14:04.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.737+0000 7fb7affff700 1 -- 172.21.15.3:0/1901849121 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b000bb20 con 0x7fb7b8139170 2023-10-03T10:14:04.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.737+0000 7fb7affff700 1 -- 172.21.15.3:0/1901849121 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0012430 con 0x7fb7b8139170 2023-10-03T10:14:04.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.738+0000 7fb7bf976700 1 -- 172.21.15.3:0/1901849121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7b8139170 msgr2=0x7fb7b813b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:04.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.738+0000 7fb7bf976700 1 --2- 172.21.15.3:0/1901849121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7b8139170 0x7fb7b813b550 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009d90 tx=0x7fb7b000b330 comp rx=0 tx=0).stop 2023-10-03T10:14:04.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.738+0000 7fb7bf976700 1 -- 172.21.15.3:0/1901849121 shutdown_connections 2023-10-03T10:14:04.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.738+0000 7fb7bf976700 1 --2- 172.21.15.3:0/1901849121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7b8139170 0x7fb7b813b550 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:04.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.738+0000 7fb7bf976700 1 --2- 172.21.15.3:0/1901849121 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7b809e1d0 0x7fb7b8138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:04.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.738+0000 7fb7bf976700 1 -- 172.21.15.3:0/1901849121 >> 172.21.15.3:0/1901849121 conn(0x7fb7b8066f50 msgr2=0x7fb7b809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:04.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.738+0000 7fb7bf976700 1 -- 172.21.15.3:0/1901849121 shutdown_connections 2023-10-03T10:14:04.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.738+0000 7fb7bf976700 1 -- 172.21.15.3:0/1901849121 wait complete. 2023-10-03T10:14:04.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.739+0000 7fb7bf976700 1 Processor -- start 2023-10-03T10:14:04.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.739+0000 7fb7bf976700 1 -- start start 2023-10-03T10:14:04.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.740+0000 7fb7bf976700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7b809d6f0 0x7fb7b8121660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:04.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.740+0000 7fb7bf976700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7b809e1d0 0x7fb7b8121ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:04.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.740+0000 7fb7bf976700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7b8139170 0x7fb7b81220e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:04.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.740+0000 7fb7bf976700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b8122a30 con 0x7fb7b809e1d0 2023-10-03T10:14:04.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.740+0000 7fb7bf976700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b8124370 con 0x7fb7b809d6f0 2023-10-03T10:14:04.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.740+0000 7fb7bf976700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b8124a20 con 0x7fb7b8139170 2023-10-03T10:14:04.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.740+0000 7fb7bcf11700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7b809e1d0 0x7fb7b8121ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:04.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.740+0000 7fb7bd712700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7b809d6f0 0x7fb7b8121660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:04.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.740+0000 7fb7bcf11700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7b809e1d0 0x7fb7b8121ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57740/0 (socket says 172.21.15.3:57740) 2023-10-03T10:14:04.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.740+0000 7fb7bd712700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7b809d6f0 0x7fb7b8121660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39818/0 (socket says 172.21.15.3:39818) 2023-10-03T10:14:04.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.740+0000 7fb7bcf11700 1 -- 172.21.15.3:0/2708129831 learned_addr learned my addr 172.21.15.3:0/2708129831 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:04.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.740+0000 7fb7bdf13700 1 --2- 172.21.15.3:0/2708129831 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7b8139170 0x7fb7b81220e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:04.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.740+0000 7fb7bcf11700 1 -- 172.21.15.3:0/2708129831 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7b8139170 msgr2=0x7fb7b81220e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:04.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.740+0000 7fb7bcf11700 1 --2- 172.21.15.3:0/2708129831 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7b8139170 0x7fb7b81220e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:04.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.740+0000 7fb7bcf11700 1 -- 172.21.15.3:0/2708129831 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7b809d6f0 msgr2=0x7fb7b8121660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:04.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.740+0000 7fb7bcf11700 1 --2- 172.21.15.3:0/2708129831 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7b809d6f0 0x7fb7b8121660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:04.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.740+0000 7fb7bcf11700 1 -- 172.21.15.3:0/2708129831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a8009550 con 0x7fb7b809e1d0 2023-10-03T10:14:04.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.740+0000 7fb7bcf11700 1 --2- 172.21.15.3:0/2708129831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7b809e1d0 0x7fb7b8121ba0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800ebe0 tx=0x7fb7a800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:04.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.741+0000 7fb7adffb700 1 -- 172.21.15.3:0/2708129831 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a800c850 con 0x7fb7b809e1d0 2023-10-03T10:14:04.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.741+0000 7fb7adffb700 1 -- 172.21.15.3:0/2708129831 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7a800c9b0 con 0x7fb7b809e1d0 2023-10-03T10:14:04.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.741+0000 7fb7bf976700 1 -- 172.21.15.3:0/2708129831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7b0009a70 con 0x7fb7b809e1d0 2023-10-03T10:14:04.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.741+0000 7fb7adffb700 1 -- 172.21.15.3:0/2708129831 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a800cd40 con 0x7fb7b809e1d0 2023-10-03T10:14:04.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.741+0000 7fb7bf976700 1 -- 172.21.15.3:0/2708129831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b8123170 con 0x7fb7b809e1d0 2023-10-03T10:14:04.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.742+0000 7fb7adffb700 1 -- 172.21.15.3:0/2708129831 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb7a8013470 con 0x7fb7b809e1d0 2023-10-03T10:14:04.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.742+0000 7fb7bf976700 1 -- 172.21.15.3:0/2708129831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b8074ea0 con 0x7fb7b809e1d0 2023-10-03T10:14:04.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.742+0000 7fb7adffb700 1 -- 172.21.15.3:0/2708129831 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb7a803f8d0 con 0x7fb7b809e1d0 2023-10-03T10:14:04.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.747+0000 7fb7adffb700 1 -- 172.21.15.3:0/2708129831 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb7a80232f0 con 0x7fb7b809e1d0 2023-10-03T10:14:04.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.949+0000 7fb7bf976700 1 -- 172.21.15.3:0/2708129831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb7b806c360 con 0x7fb7b809e1d0 2023-10-03T10:14:04.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.950+0000 7fb7adffb700 1 -- 172.21.15.3:0/2708129831 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb7a803be60 con 0x7fb7b809e1d0 2023-10-03T10:14:04.950 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:04.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.952+0000 7fb7bf976700 1 -- 172.21.15.3:0/2708129831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7b809e1d0 msgr2=0x7fb7b8121ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:04.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.952+0000 7fb7bf976700 1 --2- 172.21.15.3:0/2708129831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7b809e1d0 0x7fb7b8121ba0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800ebe0 tx=0x7fb7a800eef0 comp rx=0 tx=0).stop 2023-10-03T10:14:04.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.952+0000 7fb7bf976700 1 -- 172.21.15.3:0/2708129831 shutdown_connections 2023-10-03T10:14:04.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.952+0000 7fb7bf976700 1 --2- 172.21.15.3:0/2708129831 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7b8139170 0x7fb7b81220e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:04.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.952+0000 7fb7bf976700 1 --2- 172.21.15.3:0/2708129831 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7b809d6f0 0x7fb7b8121660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:04.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.952+0000 7fb7bf976700 1 --2- 172.21.15.3:0/2708129831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7b809e1d0 0x7fb7b8121ba0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:04.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.952+0000 7fb7bf976700 1 -- 172.21.15.3:0/2708129831 >> 172.21.15.3:0/2708129831 conn(0x7fb7b8066f50 msgr2=0x7fb7b8070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:04.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.952+0000 7fb7bf976700 1 -- 172.21.15.3:0/2708129831 shutdown_connections 2023-10-03T10:14:04.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:04.952+0000 7fb7bf976700 1 -- 172.21.15.3:0/2708129831 wait complete. 2023-10-03T10:14:04.966 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:05.038 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:05.134 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:05.176 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:05.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.181+0000 7f7bc6788700 1 Processor -- start 2023-10-03T10:14:05.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.181+0000 7f7bc6788700 1 -- start start 2023-10-03T10:14:05.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.182+0000 7f7bc6788700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7bc009e1d0 0x7f7bc0138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:05.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.182+0000 7f7bc6788700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7bc0139170 0x7f7bc013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:05.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.182+0000 7f7bc6788700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bc009b120 con 0x7f7bc009d6f0 2023-10-03T10:14:05.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.182+0000 7f7bc6788700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc006b460 con 0x7f7bc0139170 2023-10-03T10:14:05.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.182+0000 7f7bc6788700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc009a7e0 con 0x7f7bc009e1d0 2023-10-03T10:14:05.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.182+0000 7f7bbffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7bc009d6f0 0x7f7bc009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54272/0 (socket says 172.21.15.3:54272) 2023-10-03T10:14:05.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.182+0000 7f7bbffff700 1 -- 172.21.15.3:0/3585204968 learned_addr learned my addr 172.21.15.3:0/3585204968 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:05.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.182+0000 7f7bbf7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7bc009e1d0 0x7f7bc0138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:05.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.182+0000 7f7bc4d25700 1 --2- 172.21.15.3:0/3585204968 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7bc0139170 0x7f7bc013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:05.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.182+0000 7f7bbeffd700 1 -- 172.21.15.3:0/3585204968 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 628738949 0 0) 0x7f7bc009b120 con 0x7f7bc009d6f0 2023-10-03T10:14:05.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.182+0000 7f7bbeffd700 1 -- 172.21.15.3:0/3585204968 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ba80035b0 con 0x7f7bc009d6f0 2023-10-03T10:14:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.182+0000 7f7bc4d25700 1 -- 172.21.15.3:0/3585204968 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7bc009e1d0 msgr2=0x7f7bc0138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.182+0000 7f7bc4d25700 1 --2- 172.21.15.3:0/3585204968 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7bc009e1d0 0x7f7bc0138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.182+0000 7f7bc4d25700 1 -- 172.21.15.3:0/3585204968 >> v1:172.21.15.3:6789/0 conn(0x7f7bc009d6f0 legacy=0x7f7bc009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.182+0000 7f7bc4d25700 1 -- 172.21.15.3:0/3585204968 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc009a500 con 0x7f7bc0139170 2023-10-03T10:14:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.183+0000 7f7bc4d25700 1 --2- 172.21.15.3:0/3585204968 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7bc0139170 0x7f7bc013b550 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8009dc0 tx=0x7f7bb800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4b2cff43c1d00b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.183+0000 7f7bbeffd700 1 -- 172.21.15.3:0/3585204968 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb800e070 con 0x7f7bc0139170 2023-10-03T10:14:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.183+0000 7f7bbeffd700 1 -- 172.21.15.3:0/3585204968 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb800bb20 con 0x7f7bc0139170 2023-10-03T10:14:05.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.183+0000 7f7bbeffd700 1 -- 172.21.15.3:0/3585204968 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8012430 con 0x7f7bc0139170 2023-10-03T10:14:05.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.183+0000 7f7bc6788700 1 -- 172.21.15.3:0/3585204968 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7bc0139170 msgr2=0x7f7bc013b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:05.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.184+0000 7f7bc6788700 1 --2- 172.21.15.3:0/3585204968 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7bc0139170 0x7f7bc013b550 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8009dc0 tx=0x7f7bb800b330 comp rx=0 tx=0).stop 2023-10-03T10:14:05.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.184+0000 7f7bc6788700 1 -- 172.21.15.3:0/3585204968 shutdown_connections 2023-10-03T10:14:05.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.184+0000 7f7bc6788700 1 --2- 172.21.15.3:0/3585204968 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7bc0139170 0x7f7bc013b550 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:05.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.184+0000 7f7bc6788700 1 --2- 172.21.15.3:0/3585204968 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7bc009e1d0 0x7f7bc0138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:05.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.184+0000 7f7bc6788700 1 -- 172.21.15.3:0/3585204968 >> 172.21.15.3:0/3585204968 conn(0x7f7bc0066f50 msgr2=0x7f7bc009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:05.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.184+0000 7f7bc6788700 1 -- 172.21.15.3:0/3585204968 shutdown_connections 2023-10-03T10:14:05.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.184+0000 7f7bc6788700 1 -- 172.21.15.3:0/3585204968 wait complete. 2023-10-03T10:14:05.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.185+0000 7f7bc6788700 1 Processor -- start 2023-10-03T10:14:05.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.185+0000 7f7bc6788700 1 -- start start 2023-10-03T10:14:05.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.185+0000 7f7bc6788700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7bc009d6f0 0x7f7bc011e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:05.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.185+0000 7f7bc6788700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7bc009e1d0 0x7f7bc011ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:05.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.185+0000 7f7bc6788700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bc0139170 0x7f7bc011f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:05.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.185+0000 7f7bc6788700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc009c710 con 0x7f7bc0139170 2023-10-03T10:14:05.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.185+0000 7f7bc6788700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc0122050 con 0x7f7bc009e1d0 2023-10-03T10:14:05.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.185+0000 7f7bc6788700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc0121aa0 con 0x7f7bc009d6f0 2023-10-03T10:14:05.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.186+0000 7f7bc4d25700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bc0139170 0x7f7bc011f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:05.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.186+0000 7f7bbffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7bc009d6f0 0x7f7bc011e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:05.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.186+0000 7f7bc4d25700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bc0139170 0x7f7bc011f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57754/0 (socket says 172.21.15.3:57754) 2023-10-03T10:14:05.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.186+0000 7f7bc4d25700 1 -- 172.21.15.3:0/1961962375 learned_addr learned my addr 172.21.15.3:0/1961962375 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:05.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.186+0000 7f7bbf7fe700 1 --2- 172.21.15.3:0/1961962375 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7bc009e1d0 0x7f7bc011ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:05.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.186+0000 7f7bc4d25700 1 -- 172.21.15.3:0/1961962375 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7bc009d6f0 msgr2=0x7f7bc011e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:05.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.186+0000 7f7bc4d25700 1 --2- 172.21.15.3:0/1961962375 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7bc009d6f0 0x7f7bc011e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:05.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.186+0000 7f7bc4d25700 1 -- 172.21.15.3:0/1961962375 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7bc009e1d0 msgr2=0x7f7bc011ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:05.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.186+0000 7f7bc4d25700 1 --2- 172.21.15.3:0/1961962375 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7bc009e1d0 0x7f7bc011ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:05.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.186+0000 7f7bc4d25700 1 -- 172.21.15.3:0/1961962375 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb8009a70 con 0x7f7bc0139170 2023-10-03T10:14:05.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.186+0000 7f7bc4d25700 1 --2- 172.21.15.3:0/1961962375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bc0139170 0x7f7bc011f360 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8014040 tx=0x7f7bb8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:05.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.187+0000 7f7bbcff9700 1 -- 172.21.15.3:0/1961962375 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8004880 con 0x7f7bc0139170 2023-10-03T10:14:05.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.187+0000 7f7bbcff9700 1 -- 172.21.15.3:0/1961962375 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb8020070 con 0x7f7bc0139170 2023-10-03T10:14:05.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.187+0000 7f7bc6788700 1 -- 172.21.15.3:0/1961962375 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc0071eb0 con 0x7f7bc0139170 2023-10-03T10:14:05.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.187+0000 7f7bbcff9700 1 -- 172.21.15.3:0/1961962375 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb80276c0 con 0x7f7bc0139170 2023-10-03T10:14:05.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.187+0000 7f7bc6788700 1 -- 172.21.15.3:0/1961962375 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc0120c10 con 0x7f7bc0139170 2023-10-03T10:14:05.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.188+0000 7f7bbcff9700 1 -- 172.21.15.3:0/1961962375 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7bb8012430 con 0x7f7bc0139170 2023-10-03T10:14:05.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.188+0000 7f7bbcff9700 1 -- 172.21.15.3:0/1961962375 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7bb8022070 con 0x7f7bc0139170 2023-10-03T10:14:05.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.188+0000 7f7bc6788700 1 -- 172.21.15.3:0/1961962375 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bc0122190 con 0x7f7bc0139170 2023-10-03T10:14:05.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.194+0000 7f7bbcff9700 1 -- 172.21.15.3:0/1961962375 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7bc0122190 con 0x7f7bc0139170 2023-10-03T10:14:05.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.279+0000 7fd173ddf700 1 Processor -- start 2023-10-03T10:14:05.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.279+0000 7fd173ddf700 1 -- start start 2023-10-03T10:14:05.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.280+0000 7fd173ddf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd16c0a0a30 0x7fd16c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:05.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.280+0000 7fd173ddf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd16c0a1340 0x7fd16c133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:05.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.280+0000 7fd173ddf700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd16c09edd0 con 0x7fd16c133fa0 2023-10-03T10:14:05.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.280+0000 7fd173ddf700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd16c09e1f0 con 0x7fd16c0a0a30 2023-10-03T10:14:05.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.280+0000 7fd173ddf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c09dc50 con 0x7fd16c0a1340 2023-10-03T10:14:05.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.280+0000 7fd171b7b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd16c0a0a30 0x7fd16c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:05.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.280+0000 7fd17137a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd16c0a1340 0x7fd16c133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:05.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.280+0000 7fd17237c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd16c133fa0 0x7fd16c13ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54278/0 (socket says 172.21.15.3:54278) 2023-10-03T10:14:05.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.280+0000 7fd171b7b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd16c0a0a30 0x7fd16c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57766/0 (socket says 172.21.15.3:57766) 2023-10-03T10:14:05.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.280+0000 7fd17237c700 1 -- 172.21.15.3:0/1547938003 learned_addr learned my addr 172.21.15.3:0/1547938003 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:05.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.281+0000 7fd171b7b700 1 -- 172.21.15.3:0/1547938003 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd16c0a1340 msgr2=0x7fd16c133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:05.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.281+0000 7fd171b7b700 1 --2- 172.21.15.3:0/1547938003 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd16c0a1340 0x7fd16c133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:05.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.281+0000 7fd171b7b700 1 -- 172.21.15.3:0/1547938003 >> v1:172.21.15.3:6789/0 conn(0x7fd16c133fa0 legacy=0x7fd16c13ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:05.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.281+0000 7fd171b7b700 1 -- 172.21.15.3:0/1547938003 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16c0a1fe0 con 0x7fd16c0a0a30 2023-10-03T10:14:05.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.281+0000 7fd170b79700 1 -- 172.21.15.3:0/1547938003 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2072125291 0 0) 0x7fd16c09edd0 con 0x7fd16c133fa0 2023-10-03T10:14:05.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.281+0000 7fd171b7b700 1 --2- 172.21.15.3:0/1547938003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd16c0a0a30 0x7fd16c0a0e00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd168009840 tx=0x7fd168009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d132d2f37c66fc server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:05.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.281+0000 7fd170b79700 1 -- 172.21.15.3:0/1547938003 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168011830 con 0x7fd16c0a0a30 2023-10-03T10:14:05.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.281+0000 7fd170b79700 1 -- 172.21.15.3:0/1547938003 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd168011990 con 0x7fd16c0a0a30 2023-10-03T10:14:05.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.281+0000 7fd170b79700 1 -- 172.21.15.3:0/1547938003 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168011d20 con 0x7fd16c0a0a30 2023-10-03T10:14:05.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.282+0000 7fd173ddf700 1 -- 172.21.15.3:0/1547938003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd16c0a0a30 msgr2=0x7fd16c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:05.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.282+0000 7fd173ddf700 1 --2- 172.21.15.3:0/1547938003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd16c0a0a30 0x7fd16c0a0e00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd168009840 tx=0x7fd168009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:05.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.282+0000 7fd173ddf700 1 -- 172.21.15.3:0/1547938003 shutdown_connections 2023-10-03T10:14:05.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.282+0000 7fd173ddf700 1 --2- 172.21.15.3:0/1547938003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd16c0a0a30 0x7fd16c0a0e00 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:05.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.282+0000 7fd173ddf700 1 --2- 172.21.15.3:0/1547938003 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd16c0a1340 0x7fd16c133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:05.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.282+0000 7fd173ddf700 1 -- 172.21.15.3:0/1547938003 >> 172.21.15.3:0/1547938003 conn(0x7fd16c0a6550 msgr2=0x7fd16c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:05.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.282+0000 7fd173ddf700 1 -- 172.21.15.3:0/1547938003 shutdown_connections 2023-10-03T10:14:05.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.282+0000 7fd173ddf700 1 -- 172.21.15.3:0/1547938003 wait complete. 2023-10-03T10:14:05.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.283+0000 7fd173ddf700 1 Processor -- start 2023-10-03T10:14:05.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.283+0000 7fd173ddf700 1 -- start start 2023-10-03T10:14:05.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.283+0000 7fd173ddf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd16c0a0a30 0x7fd16c122930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:05.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.283+0000 7fd173ddf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd16c0a1340 0x7fd16c122e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:05.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.283+0000 7fd173ddf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd16c133fa0 0x7fd16c1233b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:05.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.283+0000 7fd173ddf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c1244b0 con 0x7fd16c133fa0 2023-10-03T10:14:05.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.284+0000 7fd173ddf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c124d80 con 0x7fd16c0a1340 2023-10-03T10:14:05.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.284+0000 7fd173ddf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c1263c0 con 0x7fd16c0a0a30 2023-10-03T10:14:05.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.284+0000 7fd17237c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd16c133fa0 0x7fd16c1233b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:05.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.284+0000 7fd17237c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd16c133fa0 0x7fd16c1233b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57774/0 (socket says 172.21.15.3:57774) 2023-10-03T10:14:05.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.284+0000 7fd17237c700 1 -- 172.21.15.3:0/393188734 learned_addr learned my addr 172.21.15.3:0/393188734 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:05.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.284+0000 7fd17137a700 1 --2- 172.21.15.3:0/393188734 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd16c0a1340 0x7fd16c122e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:05.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.284+0000 7fd17237c700 1 -- 172.21.15.3:0/393188734 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd16c0a0a30 msgr2=0x7fd16c122930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:05.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.284+0000 7fd17237c700 1 --2- 172.21.15.3:0/393188734 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd16c0a0a30 0x7fd16c122930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:05.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.284+0000 7fd17237c700 1 -- 172.21.15.3:0/393188734 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd16c0a1340 msgr2=0x7fd16c122e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:05.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.284+0000 7fd17237c700 1 --2- 172.21.15.3:0/393188734 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd16c0a1340 0x7fd16c122e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:05.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.284+0000 7fd17237c700 1 -- 172.21.15.3:0/393188734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1680094a0 con 0x7fd16c133fa0 2023-10-03T10:14:05.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.284+0000 7fd17137a700 1 --2- 172.21.15.3:0/393188734 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd16c0a1340 0x7fd16c122e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:05.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.284+0000 7fd17237c700 1 --2- 172.21.15.3:0/393188734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd16c133fa0 0x7fd16c1233b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd164002230 tx=0x7fd16400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:05.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.284+0000 7fd1627fc700 1 -- 172.21.15.3:0/393188734 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16400ab70 con 0x7fd16c133fa0 2023-10-03T10:14:05.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.285+0000 7fd1627fc700 1 -- 172.21.15.3:0/393188734 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16400acf0 con 0x7fd16c133fa0 2023-10-03T10:14:05.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.285+0000 7fd173ddf700 1 -- 172.21.15.3:0/393188734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd16c124a60 con 0x7fd16c133fa0 2023-10-03T10:14:05.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.285+0000 7fd1627fc700 1 -- 172.21.15.3:0/393188734 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd164007310 con 0x7fd16c133fa0 2023-10-03T10:14:05.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.285+0000 7fd173ddf700 1 -- 172.21.15.3:0/393188734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd16c1260e0 con 0x7fd16c133fa0 2023-10-03T10:14:05.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.285+0000 7fd1627fc700 1 -- 172.21.15.3:0/393188734 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd164007470 con 0x7fd16c133fa0 2023-10-03T10:14:05.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.286+0000 7fd1627fc700 1 -- 172.21.15.3:0/393188734 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd16403f340 con 0x7fd16c133fa0 2023-10-03T10:14:05.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.286+0000 7fd173ddf700 1 -- 172.21.15.3:0/393188734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd16c0a1d40 con 0x7fd16c133fa0 2023-10-03T10:14:05.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.296+0000 7fd1627fc700 1 -- 172.21.15.3:0/393188734 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd164018020 con 0x7fd16c133fa0 2023-10-03T10:14:05.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.322+0000 7fc931301700 1 Processor -- start 2023-10-03T10:14:05.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.322+0000 7fc931301700 1 -- start start 2023-10-03T10:14:05.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.322+0000 7fc931301700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc92c1367b0 0x7fc92c138b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:05.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.322+0000 7fc931301700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc92c1390d0 0x7fc92c13b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:05.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.322+0000 7fc931301700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc92c06f1c0 con 0x7fc92c130410 2023-10-03T10:14:05.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.322+0000 7fc931301700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc92c070700 con 0x7fc92c1390d0 2023-10-03T10:14:05.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.322+0000 7fc931301700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c071f20 con 0x7fc92c1367b0 2023-10-03T10:14:05.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.323+0000 7fc92a7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc92c1367b0 0x7fc92c138b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:05.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.323+0000 7fc92a7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc92c1367b0 0x7fc92c138b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:40046/0 (socket says 172.21.15.3:40046) 2023-10-03T10:14:05.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.323+0000 7fc92a7fc700 1 -- 172.21.15.3:0/2426482217 learned_addr learned my addr 172.21.15.3:0/2426482217 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:05.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.323+0000 7fc92affd700 1 --1- 172.21.15.3:0/2426482217 >> v1:172.21.15.3:6789/0 conn(0x7fc92c130410 0x7fc92c1307e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54280/0 (socket says 172.21.15.3:54280) 2023-10-03T10:14:05.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.323+0000 7fc92b7fe700 1 --2- 172.21.15.3:0/2426482217 >> v2:172.21.15.3:3300/0 conn(0x7fc92c1390d0 0x7fc92c13b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:05.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.323+0000 7fc92b7fe700 1 -- 172.21.15.3:0/2426482217 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc92c1367b0 msgr2=0x7fc92c138b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:05.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.323+0000 7fc92b7fe700 1 --2- 172.21.15.3:0/2426482217 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc92c1367b0 0x7fc92c138b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:05.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.323+0000 7fc92b7fe700 1 -- 172.21.15.3:0/2426482217 >> v1:172.21.15.3:6789/0 conn(0x7fc92c130410 legacy=0x7fc92c1307e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:05.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.323+0000 7fc92b7fe700 1 -- 172.21.15.3:0/2426482217 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92c12ffe0 con 0x7fc92c1390d0 2023-10-03T10:14:05.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.323+0000 7fc929ffb700 1 -- 172.21.15.3:0/2426482217 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1698522379 0 0) 0x7fc92c06f1c0 con 0x7fc92c130410 2023-10-03T10:14:05.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.323+0000 7fc92b7fe700 1 --2- 172.21.15.3:0/2426482217 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc92c1390d0 0x7fc92c13b4b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc924009840 tx=0x7fc924009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da5f03ac3e5bc5d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:05.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.323+0000 7fc929ffb700 1 -- 172.21.15.3:0/2426482217 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc924011830 con 0x7fc92c1390d0 2023-10-03T10:14:05.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.324+0000 7fc929ffb700 1 -- 172.21.15.3:0/2426482217 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc924011990 con 0x7fc92c1390d0 2023-10-03T10:14:05.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.324+0000 7fc929ffb700 1 -- 172.21.15.3:0/2426482217 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc924011d20 con 0x7fc92c1390d0 2023-10-03T10:14:05.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.324+0000 7fc931301700 1 -- 172.21.15.3:0/2426482217 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc92c1390d0 msgr2=0x7fc92c13b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:05.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.324+0000 7fc931301700 1 --2- 172.21.15.3:0/2426482217 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc92c1390d0 0x7fc92c13b4b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc924009840 tx=0x7fc924009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:05.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.324+0000 7fc931301700 1 -- 172.21.15.3:0/2426482217 shutdown_connections 2023-10-03T10:14:05.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.324+0000 7fc931301700 1 --2- 172.21.15.3:0/2426482217 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc92c1390d0 0x7fc92c13b4b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:05.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.324+0000 7fc931301700 1 --2- 172.21.15.3:0/2426482217 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc92c1367b0 0x7fc92c138b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:05.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.324+0000 7fc931301700 1 -- 172.21.15.3:0/2426482217 >> 172.21.15.3:0/2426482217 conn(0x7fc92c0a2610 msgr2=0x7fc92c0a2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:05.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.324+0000 7fc931301700 1 -- 172.21.15.3:0/2426482217 shutdown_connections 2023-10-03T10:14:05.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.324+0000 7fc931301700 1 -- 172.21.15.3:0/2426482217 wait complete. 2023-10-03T10:14:05.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.325+0000 7fc931301700 1 Processor -- start 2023-10-03T10:14:05.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.325+0000 7fc931301700 1 -- start start 2023-10-03T10:14:05.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.326+0000 7fc931301700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc92c130410 0x7fc92c119c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:05.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.326+0000 7fc931301700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc92c1367b0 0x7fc92c11a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:05.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.326+0000 7fc931301700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc92c1390d0 0x7fc92c11a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:05.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.326+0000 7fc931301700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c11d7b0 con 0x7fc92c1367b0 2023-10-03T10:14:05.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.326+0000 7fc931301700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c11cf60 con 0x7fc92c1390d0 2023-10-03T10:14:05.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.326+0000 7fc931301700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c138d30 con 0x7fc92c130410 2023-10-03T10:14:05.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.326+0000 7fc92b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc92c1390d0 0x7fc92c11a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:05.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.326+0000 7fc92affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc92c130410 0x7fc92c119c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:05.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.326+0000 7fc92b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc92c1390d0 0x7fc92c11a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39858/0 (socket says 172.21.15.3:39858) 2023-10-03T10:14:05.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.326+0000 7fc92a7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc92c1367b0 0x7fc92c11a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:05.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.326+0000 7fc92b7fe700 1 -- 172.21.15.3:0/4006330224 learned_addr learned my addr 172.21.15.3:0/4006330224 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:05.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.326+0000 7fc92b7fe700 1 -- 172.21.15.3:0/4006330224 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc92c130410 msgr2=0x7fc92c119c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:05.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.326+0000 7fc92b7fe700 1 --2- 172.21.15.3:0/4006330224 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc92c130410 0x7fc92c119c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:05.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.327+0000 7fc92b7fe700 1 -- 172.21.15.3:0/4006330224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc92c1367b0 msgr2=0x7fc92c11a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:05.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.327+0000 7fc92b7fe700 1 --2- 172.21.15.3:0/4006330224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc92c1367b0 0x7fc92c11a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:05.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.327+0000 7fc92b7fe700 1 -- 172.21.15.3:0/4006330224 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9240094a0 con 0x7fc92c1390d0 2023-10-03T10:14:05.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.327+0000 7fc92b7fe700 1 --2- 172.21.15.3:0/4006330224 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc92c1390d0 0x7fc92c11a710 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc924009fd0 tx=0x7fc924004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:05.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.327+0000 7fc9137fe700 1 -- 172.21.15.3:0/4006330224 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9240036a0 con 0x7fc92c1390d0 2023-10-03T10:14:05.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.327+0000 7fc9137fe700 1 -- 172.21.15.3:0/4006330224 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92401f070 con 0x7fc92c1390d0 2023-10-03T10:14:05.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.327+0000 7fc931301700 1 -- 172.21.15.3:0/4006330224 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc92c11c100 con 0x7fc92c1390d0 2023-10-03T10:14:05.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.327+0000 7fc9137fe700 1 -- 172.21.15.3:0/4006330224 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc924003a60 con 0x7fc92c1390d0 2023-10-03T10:14:05.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.327+0000 7fc931301700 1 -- 172.21.15.3:0/4006330224 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92c11b8f0 con 0x7fc92c1390d0 2023-10-03T10:14:05.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.328+0000 7fc9137fe700 1 -- 172.21.15.3:0/4006330224 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc924003bc0 con 0x7fc92c1390d0 2023-10-03T10:14:05.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.328+0000 7fc9137fe700 1 -- 172.21.15.3:0/4006330224 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc924021070 con 0x7fc92c1390d0 2023-10-03T10:14:05.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.328+0000 7fc931301700 1 -- 172.21.15.3:0/4006330224 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc92c0747f0 con 0x7fc92c1390d0 2023-10-03T10:14:05.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.333+0000 7fc9137fe700 1 -- 172.21.15.3:0/4006330224 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc9240257f0 con 0x7fc92c1390d0 2023-10-03T10:14:05.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.412+0000 7f7bc6788700 1 -- 172.21.15.3:0/1961962375 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7bc0122190 con 0x7f7bc0139170 2023-10-03T10:14:05.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.412+0000 7f7bbcff9700 1 -- 172.21.15.3:0/1961962375 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7bc0122190 con 0x7f7bc0139170 2023-10-03T10:14:05.413 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:05.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.415+0000 7f7bc6788700 1 -- 172.21.15.3:0/1961962375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bc0139170 msgr2=0x7f7bc011f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:05.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.415+0000 7f7bc6788700 1 --2- 172.21.15.3:0/1961962375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bc0139170 0x7f7bc011f360 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8014040 tx=0x7f7bb8004ad0 comp rx=0 tx=0).stop 2023-10-03T10:14:05.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.415+0000 7f7bc6788700 1 -- 172.21.15.3:0/1961962375 shutdown_connections 2023-10-03T10:14:05.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.415+0000 7f7bc6788700 1 --2- 172.21.15.3:0/1961962375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bc0139170 0x7f7bc011f360 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:05.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.415+0000 7f7bc6788700 1 --2- 172.21.15.3:0/1961962375 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7bc009d6f0 0x7f7bc011e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:05.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.415+0000 7f7bc6788700 1 --2- 172.21.15.3:0/1961962375 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7bc009e1d0 0x7f7bc011ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:05.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.415+0000 7f7bc6788700 1 -- 172.21.15.3:0/1961962375 >> 172.21.15.3:0/1961962375 conn(0x7f7bc0066f50 msgr2=0x7f7bc0136b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:05.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.415+0000 7f7bc6788700 1 -- 172.21.15.3:0/1961962375 shutdown_connections 2023-10-03T10:14:05.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.415+0000 7f7bc6788700 1 -- 172.21.15.3:0/1961962375 wait complete. 2023-10-03T10:14:05.430 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:05.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.508+0000 7fd173ddf700 1 -- 172.21.15.3:0/393188734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd16c074ea0 con 0x7fd16c133fa0 2023-10-03T10:14:05.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.508+0000 7fd1627fc700 1 -- 172.21.15.3:0/393188734 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd164019860 con 0x7fd16c133fa0 2023-10-03T10:14:05.509 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:05.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.510+0000 7fd173ddf700 1 -- 172.21.15.3:0/393188734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd16c133fa0 msgr2=0x7fd16c1233b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:05.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.510+0000 7fd173ddf700 1 --2- 172.21.15.3:0/393188734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd16c133fa0 0x7fd16c1233b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd164002230 tx=0x7fd16400cf50 comp rx=0 tx=0).stop 2023-10-03T10:14:05.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.510+0000 7fd173ddf700 1 -- 172.21.15.3:0/393188734 shutdown_connections 2023-10-03T10:14:05.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.510+0000 7fd173ddf700 1 --2- 172.21.15.3:0/393188734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd16c133fa0 0x7fd16c1233b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:05.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.510+0000 7fd173ddf700 1 --2- 172.21.15.3:0/393188734 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd16c0a0a30 0x7fd16c122930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:05.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.510+0000 7fd173ddf700 1 --2- 172.21.15.3:0/393188734 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd16c0a1340 0x7fd16c122e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:05.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.510+0000 7fd173ddf700 1 -- 172.21.15.3:0/393188734 >> 172.21.15.3:0/393188734 conn(0x7fd16c0a6550 msgr2=0x7fd16c11c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:05.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.511+0000 7fd173ddf700 1 -- 172.21.15.3:0/393188734 shutdown_connections 2023-10-03T10:14:05.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.511+0000 7fd173ddf700 1 -- 172.21.15.3:0/393188734 wait complete. 2023-10-03T10:14:05.529 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:05.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.549+0000 7fc931301700 1 -- 172.21.15.3:0/4006330224 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc92c06fa10 con 0x7fc92c1390d0 2023-10-03T10:14:05.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.550+0000 7fc9137fe700 1 -- 172.21.15.3:0/4006330224 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc9240193b0 con 0x7fc92c1390d0 2023-10-03T10:14:05.550 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:05.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.552+0000 7fc931301700 1 -- 172.21.15.3:0/4006330224 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc92c1390d0 msgr2=0x7fc92c11a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:05.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.552+0000 7fc931301700 1 --2- 172.21.15.3:0/4006330224 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc92c1390d0 0x7fc92c11a710 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc924009fd0 tx=0x7fc924004750 comp rx=0 tx=0).stop 2023-10-03T10:14:05.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.552+0000 7fc931301700 1 -- 172.21.15.3:0/4006330224 shutdown_connections 2023-10-03T10:14:05.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.552+0000 7fc931301700 1 --2- 172.21.15.3:0/4006330224 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc92c1390d0 0x7fc92c11a710 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:05.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.552+0000 7fc931301700 1 --2- 172.21.15.3:0/4006330224 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc92c130410 0x7fc92c119c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:05.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.552+0000 7fc931301700 1 --2- 172.21.15.3:0/4006330224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc92c1367b0 0x7fc92c11a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:05.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.552+0000 7fc931301700 1 -- 172.21.15.3:0/4006330224 >> 172.21.15.3:0/4006330224 conn(0x7fc92c0a2610 msgr2=0x7fc92c070970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:05.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.552+0000 7fc931301700 1 -- 172.21.15.3:0/4006330224 shutdown_connections 2023-10-03T10:14:05.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:05.552+0000 7fc931301700 1 -- 172.21.15.3:0/4006330224 wait complete. 2023-10-03T10:14:05.566 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:05.968 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:06.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.113+0000 7fcc662fa700 1 Processor -- start 2023-10-03T10:14:06.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.113+0000 7fcc662fa700 1 -- start start 2023-10-03T10:14:06.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.114+0000 7fcc662fa700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcc6009d6f0 0x7fcc6009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:06.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.114+0000 7fcc662fa700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcc6009e410 0x7fcc60134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:06.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.114+0000 7fcc662fa700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc6009b120 con 0x7fcc6009e000 2023-10-03T10:14:06.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.114+0000 7fcc662fa700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc6006b460 con 0x7fcc6009e410 2023-10-03T10:14:06.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.114+0000 7fcc662fa700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6009a7e0 con 0x7fcc6009d6f0 2023-10-03T10:14:06.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.114+0000 7fcc64897700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcc6009e410 0x7fcc60134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:06.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.114+0000 7fcc64897700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcc6009e410 0x7fcc60134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57802/0 (socket says 172.21.15.3:57802) 2023-10-03T10:14:06.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.114+0000 7fcc64897700 1 -- 172.21.15.3:0/4247498781 learned_addr learned my addr 172.21.15.3:0/4247498781 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:06.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.114+0000 7fcc5ffff700 1 --2- 172.21.15.3:0/4247498781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcc6009d6f0 0x7fcc6009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:06.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.114+0000 7fcc64897700 1 -- 172.21.15.3:0/4247498781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcc6009d6f0 msgr2=0x7fcc6009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:06.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.114+0000 7fcc64897700 1 --2- 172.21.15.3:0/4247498781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcc6009d6f0 0x7fcc6009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.115+0000 7fcc64897700 1 -- 172.21.15.3:0/4247498781 >> v1:172.21.15.3:6789/0 conn(0x7fcc6009e000 legacy=0x7fcc601339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:06.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.115+0000 7fcc64897700 1 -- 172.21.15.3:0/4247498781 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6009a500 con 0x7fcc6009e410 2023-10-03T10:14:06.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.115+0000 7fcc5ffff700 1 --2- 172.21.15.3:0/4247498781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcc6009d6f0 0x7fcc6009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:06.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.115+0000 7fcc64897700 1 --2- 172.21.15.3:0/4247498781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc6009e410 0x7fcc60134100 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fcc58009e50 tx=0x7fcc5800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb1d00b72df694e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:06.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.115+0000 7fcc5effd700 1 -- 172.21.15.3:0/4247498781 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5800e070 con 0x7fcc6009e410 2023-10-03T10:14:06.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.115+0000 7fcc5effd700 1 -- 172.21.15.3:0/4247498781 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5800b910 con 0x7fcc6009e410 2023-10-03T10:14:06.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.115+0000 7fcc5effd700 1 -- 172.21.15.3:0/4247498781 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5800bcc0 con 0x7fcc6009e410 2023-10-03T10:14:06.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.115+0000 7fcc662fa700 1 -- 172.21.15.3:0/4247498781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc6009e410 msgr2=0x7fcc60134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:06.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.115+0000 7fcc662fa700 1 --2- 172.21.15.3:0/4247498781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc6009e410 0x7fcc60134100 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fcc58009e50 tx=0x7fcc5800b330 comp rx=0 tx=0).stop 2023-10-03T10:14:06.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.116+0000 7fcc662fa700 1 -- 172.21.15.3:0/4247498781 shutdown_connections 2023-10-03T10:14:06.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.116+0000 7fcc662fa700 1 --2- 172.21.15.3:0/4247498781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc6009e410 0x7fcc60134100 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.116+0000 7fcc662fa700 1 --2- 172.21.15.3:0/4247498781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcc6009d6f0 0x7fcc6009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.116+0000 7fcc662fa700 1 -- 172.21.15.3:0/4247498781 >> 172.21.15.3:0/4247498781 conn(0x7fcc60066f50 msgr2=0x7fcc6009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:06.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.116+0000 7fcc662fa700 1 -- 172.21.15.3:0/4247498781 shutdown_connections 2023-10-03T10:14:06.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.116+0000 7fcc662fa700 1 -- 172.21.15.3:0/4247498781 wait complete. 2023-10-03T10:14:06.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.117+0000 7fcc662fa700 1 Processor -- start 2023-10-03T10:14:06.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.117+0000 7fcc662fa700 1 -- start start 2023-10-03T10:14:06.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.117+0000 7fcc662fa700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcc6009d6f0 0x7fcc601a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:06.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.117+0000 7fcc662fa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc6009e000 0x7fcc601a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:06.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.117+0000 7fcc662fa700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcc6009e410 0x7fcc601a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:06.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.117+0000 7fcc662fa700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc601a9870 con 0x7fcc6009e000 2023-10-03T10:14:06.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.117+0000 7fcc662fa700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc601a7b90 con 0x7fcc6009e410 2023-10-03T10:14:06.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.117+0000 7fcc662fa700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc601a9a70 con 0x7fcc6009d6f0 2023-10-03T10:14:06.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.117+0000 7fcc5f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc6009e000 0x7fcc601a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:06.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.117+0000 7fcc5f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc6009e000 0x7fcc601a65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57804/0 (socket says 172.21.15.3:57804) 2023-10-03T10:14:06.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.117+0000 7fcc5ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcc6009d6f0 0x7fcc601a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:06.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.117+0000 7fcc5f7fe700 1 -- 172.21.15.3:0/4070927414 learned_addr learned my addr 172.21.15.3:0/4070927414 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:06.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.117+0000 7fcc64897700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcc6009e410 0x7fcc601a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:06.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.118+0000 7fcc5f7fe700 1 -- 172.21.15.3:0/4070927414 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcc6009d6f0 msgr2=0x7fcc601a60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:06.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.118+0000 7fcc5f7fe700 1 --2- 172.21.15.3:0/4070927414 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcc6009d6f0 0x7fcc601a60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.118+0000 7fcc5f7fe700 1 -- 172.21.15.3:0/4070927414 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcc6009e410 msgr2=0x7fcc601a6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:06.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.118+0000 7fcc5f7fe700 1 --2- 172.21.15.3:0/4070927414 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcc6009e410 0x7fcc601a6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.118+0000 7fcc5f7fe700 1 -- 172.21.15.3:0/4070927414 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc500094a0 con 0x7fcc6009e000 2023-10-03T10:14:06.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.118+0000 7fcc5ffff700 1 --2- 172.21.15.3:0/4070927414 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcc6009d6f0 0x7fcc601a60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:06.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.118+0000 7fcc5f7fe700 1 --2- 172.21.15.3:0/4070927414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc6009e000 0x7fcc601a65e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fcc5400c4b0 tx=0x7fcc5400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:06.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.118+0000 7fcc5cff9700 1 -- 172.21.15.3:0/4070927414 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc540052d0 con 0x7fcc6009e000 2023-10-03T10:14:06.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.118+0000 7fcc5cff9700 1 -- 172.21.15.3:0/4070927414 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc54005430 con 0x7fcc6009e000 2023-10-03T10:14:06.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.118+0000 7fcc662fa700 1 -- 172.21.15.3:0/4070927414 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc58009b30 con 0x7fcc6009e000 2023-10-03T10:14:06.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.118+0000 7fcc5cff9700 1 -- 172.21.15.3:0/4070927414 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc540038c0 con 0x7fcc6009e000 2023-10-03T10:14:06.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.118+0000 7fcc662fa700 1 -- 172.21.15.3:0/4070927414 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc601a88f0 con 0x7fcc6009e000 2023-10-03T10:14:06.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.119+0000 7fcc5cff9700 1 -- 172.21.15.3:0/4070927414 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcc54003a20 con 0x7fcc6009e000 2023-10-03T10:14:06.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.119+0000 7fcc5cff9700 1 -- 172.21.15.3:0/4070927414 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fcc5403ea20 con 0x7fcc6009e000 2023-10-03T10:14:06.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.119+0000 7fcc662fa700 1 -- 172.21.15.3:0/4070927414 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc60074ea0 con 0x7fcc6009e000 2023-10-03T10:14:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.124+0000 7fcc5cff9700 1 -- 172.21.15.3:0/4070927414 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcc54018260 con 0x7fcc6009e000 2023-10-03T10:14:06.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.333+0000 7fcc662fa700 1 -- 172.21.15.3:0/4070927414 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcc601a82b0 con 0x7fcc6009e000 2023-10-03T10:14:06.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.333+0000 7fcc5cff9700 1 -- 172.21.15.3:0/4070927414 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcc54017020 con 0x7fcc6009e000 2023-10-03T10:14:06.333 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:06.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.335+0000 7fcc662fa700 1 -- 172.21.15.3:0/4070927414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc6009e000 msgr2=0x7fcc601a65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:06.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.335+0000 7fcc662fa700 1 --2- 172.21.15.3:0/4070927414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc6009e000 0x7fcc601a65e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fcc5400c4b0 tx=0x7fcc5400c7c0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.335+0000 7fcc662fa700 1 -- 172.21.15.3:0/4070927414 shutdown_connections 2023-10-03T10:14:06.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.335+0000 7fcc662fa700 1 --2- 172.21.15.3:0/4070927414 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcc6009e410 0x7fcc601a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.335+0000 7fcc662fa700 1 --2- 172.21.15.3:0/4070927414 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcc6009d6f0 0x7fcc601a60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.335+0000 7fcc662fa700 1 --2- 172.21.15.3:0/4070927414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc6009e000 0x7fcc601a65e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.335+0000 7fcc662fa700 1 -- 172.21.15.3:0/4070927414 >> 172.21.15.3:0/4070927414 conn(0x7fcc60066f50 msgr2=0x7fcc601baef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:06.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.335+0000 7fcc662fa700 1 -- 172.21.15.3:0/4070927414 shutdown_connections 2023-10-03T10:14:06.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.335+0000 7fcc662fa700 1 -- 172.21.15.3:0/4070927414 wait complete. 2023-10-03T10:14:06.349 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:06.431 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:06.530 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:06.567 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:06.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.575+0000 7fd2673cd700 1 Processor -- start 2023-10-03T10:14:06.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.575+0000 7fd2673cd700 1 -- start start 2023-10-03T10:14:06.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.576+0000 7fd2673cd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd260133a80 0x7fd260133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:06.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.576+0000 7fd2673cd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd260130a40 0x7fd26013b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:06.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.576+0000 7fd2673cd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd26006bc00 con 0x7fd26012ff20 2023-10-03T10:14:06.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.576+0000 7fd2673cd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd26006c570 con 0x7fd260133a80 2023-10-03T10:14:06.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.576+0000 7fd2673cd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd2600a0040 con 0x7fd260130a40 2023-10-03T10:14:06.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.576+0000 7fd265169700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd260133a80 0x7fd260133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:06.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.576+0000 7fd265169700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd260133a80 0x7fd260133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39872/0 (socket says 172.21.15.3:39872) 2023-10-03T10:14:06.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.576+0000 7fd26596a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd260130a40 0x7fd26013b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:06.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.576+0000 7fd264968700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd26012ff20 0x7fd260130330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54288/0 (socket says 172.21.15.3:54288) 2023-10-03T10:14:06.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.576+0000 7fd265169700 1 -- 172.21.15.3:0/381954323 learned_addr learned my addr 172.21.15.3:0/381954323 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:06.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.576+0000 7fd265169700 1 -- 172.21.15.3:0/381954323 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd260130a40 msgr2=0x7fd26013b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:06.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.576+0000 7fd265169700 1 --2- 172.21.15.3:0/381954323 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd260130a40 0x7fd26013b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.576+0000 7fd265169700 1 -- 172.21.15.3:0/381954323 >> v1:172.21.15.3:6789/0 conn(0x7fd26012ff20 legacy=0x7fd260130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:06.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.576+0000 7fd265169700 1 -- 172.21.15.3:0/381954323 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd260133860 con 0x7fd260133a80 2023-10-03T10:14:06.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.576+0000 7fd26596a700 1 --2- 172.21.15.3:0/381954323 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd260130a40 0x7fd26013b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:06.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.576+0000 7fd265169700 1 --2- 172.21.15.3:0/381954323 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd260133a80 0x7fd260133e50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009920 tx=0x7fd25c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bb7767c44ac0eb3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:06.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.577+0000 7fd257fff700 1 -- 172.21.15.3:0/381954323 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c011830 con 0x7fd260133a80 2023-10-03T10:14:06.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.577+0000 7fd257fff700 1 -- 172.21.15.3:0/381954323 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25c011990 con 0x7fd260133a80 2023-10-03T10:14:06.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.577+0000 7fd257fff700 1 -- 172.21.15.3:0/381954323 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c011d20 con 0x7fd260133a80 2023-10-03T10:14:06.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.577+0000 7fd2673cd700 1 -- 172.21.15.3:0/381954323 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd260133a80 msgr2=0x7fd260133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:06.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.577+0000 7fd2673cd700 1 --2- 172.21.15.3:0/381954323 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd260133a80 0x7fd260133e50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009920 tx=0x7fd25c009c30 comp rx=0 tx=0).stop 2023-10-03T10:14:06.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.578+0000 7fd2673cd700 1 -- 172.21.15.3:0/381954323 shutdown_connections 2023-10-03T10:14:06.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.578+0000 7fd2673cd700 1 --2- 172.21.15.3:0/381954323 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd260133a80 0x7fd260133e50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.578+0000 7fd2673cd700 1 --2- 172.21.15.3:0/381954323 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd260130a40 0x7fd26013b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.578+0000 7fd2673cd700 1 -- 172.21.15.3:0/381954323 >> 172.21.15.3:0/381954323 conn(0x7fd2600a4210 msgr2=0x7fd2600a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:06.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.578+0000 7fd2673cd700 1 -- 172.21.15.3:0/381954323 shutdown_connections 2023-10-03T10:14:06.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.578+0000 7fd2673cd700 1 -- 172.21.15.3:0/381954323 wait complete. 2023-10-03T10:14:06.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.579+0000 7fd2673cd700 1 Processor -- start 2023-10-03T10:14:06.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.579+0000 7fd2673cd700 1 -- start start 2023-10-03T10:14:06.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.579+0000 7fd2673cd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd26012ff20 0x7fd26009c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:06.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.579+0000 7fd2673cd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd260130a40 0x7fd26009ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:06.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.579+0000 7fd2673cd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd260133a80 0x7fd26009d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:06.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.579+0000 7fd2673cd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2601202a0 con 0x7fd260130a40 2023-10-03T10:14:06.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.579+0000 7fd2673cd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd26009e240 con 0x7fd26012ff20 2023-10-03T10:14:06.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.579+0000 7fd2673cd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd2600a20c0 con 0x7fd260133a80 2023-10-03T10:14:06.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.579+0000 7fd264968700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd260130a40 0x7fd26009ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:06.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.579+0000 7fd265169700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd26012ff20 0x7fd26009c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:06.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.579+0000 7fd264968700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd260130a40 0x7fd26009ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57816/0 (socket says 172.21.15.3:57816) 2023-10-03T10:14:06.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.579+0000 7fd264968700 1 -- 172.21.15.3:0/1012176908 learned_addr learned my addr 172.21.15.3:0/1012176908 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:06.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.579+0000 7fd26596a700 1 --2- 172.21.15.3:0/1012176908 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd260133a80 0x7fd26009d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:06.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.580+0000 7fd264968700 1 -- 172.21.15.3:0/1012176908 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd260133a80 msgr2=0x7fd26009d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:06.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.580+0000 7fd264968700 1 --2- 172.21.15.3:0/1012176908 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd260133a80 0x7fd26009d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.580+0000 7fd264968700 1 -- 172.21.15.3:0/1012176908 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd26012ff20 msgr2=0x7fd26009c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:06.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.580+0000 7fd264968700 1 --2- 172.21.15.3:0/1012176908 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd26012ff20 0x7fd26009c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.580+0000 7fd264968700 1 -- 172.21.15.3:0/1012176908 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd258009a70 con 0x7fd260130a40 2023-10-03T10:14:06.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.580+0000 7fd265169700 1 --2- 172.21.15.3:0/1012176908 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd26012ff20 0x7fd26009c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:06.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.580+0000 7fd26596a700 1 --2- 172.21.15.3:0/1012176908 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd260133a80 0x7fd26009d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:06.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.580+0000 7fd264968700 1 --2- 172.21.15.3:0/1012176908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd260130a40 0x7fd26009ce00 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd25000c4f0 tx=0x7fd25000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:06.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.580+0000 7fd255ffb700 1 -- 172.21.15.3:0/1012176908 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250005cb0 con 0x7fd260130a40 2023-10-03T10:14:06.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.580+0000 7fd255ffb700 1 -- 172.21.15.3:0/1012176908 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd250005e10 con 0x7fd260130a40 2023-10-03T10:14:06.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.580+0000 7fd2673cd700 1 -- 172.21.15.3:0/1012176908 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25c009550 con 0x7fd260130a40 2023-10-03T10:14:06.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.580+0000 7fd255ffb700 1 -- 172.21.15.3:0/1012176908 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2500072d0 con 0x7fd260130a40 2023-10-03T10:14:06.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.581+0000 7fd2673cd700 1 -- 172.21.15.3:0/1012176908 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd260120d10 con 0x7fd260130a40 2023-10-03T10:14:06.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.582+0000 7fd2673cd700 1 -- 172.21.15.3:0/1012176908 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2601204e0 con 0x7fd260130a40 2023-10-03T10:14:06.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.582+0000 7fd255ffb700 1 -- 172.21.15.3:0/1012176908 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd250007430 con 0x7fd260130a40 2023-10-03T10:14:06.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.582+0000 7fd255ffb700 1 -- 172.21.15.3:0/1012176908 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd25003ea70 con 0x7fd260130a40 2023-10-03T10:14:06.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.587+0000 7fd255ffb700 1 -- 172.21.15.3:0/1012176908 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd250040e60 con 0x7fd260130a40 2023-10-03T10:14:06.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.676+0000 7ff4f2ed3700 1 Processor -- start 2023-10-03T10:14:06.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.676+0000 7ff4f2ed3700 1 -- start start 2023-10-03T10:14:06.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.676+0000 7ff4f2ed3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff4ec0a0a30 0x7ff4ec0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:06.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.677+0000 7ff4f2ed3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff4ec138d80 0x7ff4ec13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:06.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.677+0000 7ff4f2ed3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4ec0a2a70 con 0x7ff4ec09d630 2023-10-03T10:14:06.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.677+0000 7ff4f2ed3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4ec0a01d0 con 0x7ff4ec0a0a30 2023-10-03T10:14:06.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.677+0000 7ff4f2ed3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec09e2b0 con 0x7ff4ec138d80 2023-10-03T10:14:06.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.677+0000 7ff4f0c6f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff4ec0a0a30 0x7ff4ec0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:06.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.677+0000 7ff4f0c6f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff4ec0a0a30 0x7ff4ec0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57818/0 (socket says 172.21.15.3:57818) 2023-10-03T10:14:06.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.677+0000 7ff4f0c6f700 1 -- 172.21.15.3:0/3283873563 learned_addr learned my addr 172.21.15.3:0/3283873563 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:06.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.677+0000 7ff4f1470700 1 --2- 172.21.15.3:0/3283873563 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff4ec138d80 0x7ff4ec13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:06.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.677+0000 7ff4f0c6f700 1 -- 172.21.15.3:0/3283873563 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff4ec138d80 msgr2=0x7ff4ec13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:06.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.677+0000 7ff4f0c6f700 1 --2- 172.21.15.3:0/3283873563 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff4ec138d80 0x7ff4ec13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.677+0000 7ff4f0c6f700 1 -- 172.21.15.3:0/3283873563 >> v1:172.21.15.3:6789/0 conn(0x7ff4ec09d630 legacy=0x7ff4ec0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:06.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.677+0000 7ff4f0c6f700 1 -- 172.21.15.3:0/3283873563 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ec09dc50 con 0x7ff4ec0a0a30 2023-10-03T10:14:06.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.677+0000 7ff4eb7fe700 1 -- 172.21.15.3:0/3283873563 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3656113757 0 0) 0x7ff4ec0a2a70 con 0x7ff4ec09d630 2023-10-03T10:14:06.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.677+0000 7ff4f0c6f700 1 --2- 172.21.15.3:0/3283873563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4ec0a0a30 0x7ff4ec0a0e00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff4e00098f0 tx=0x7ff4e0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c66bfef49f5ec2 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:06.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.678+0000 7ff4eb7fe700 1 -- 172.21.15.3:0/3283873563 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0011830 con 0x7ff4ec0a0a30 2023-10-03T10:14:06.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.678+0000 7ff4eb7fe700 1 -- 172.21.15.3:0/3283873563 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e0011990 con 0x7ff4ec0a0a30 2023-10-03T10:14:06.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.678+0000 7ff4eb7fe700 1 -- 172.21.15.3:0/3283873563 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0011d20 con 0x7ff4ec0a0a30 2023-10-03T10:14:06.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.679+0000 7ff4f2ed3700 1 -- 172.21.15.3:0/3283873563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4ec0a0a30 msgr2=0x7ff4ec0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:06.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.679+0000 7ff4f2ed3700 1 --2- 172.21.15.3:0/3283873563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4ec0a0a30 0x7ff4ec0a0e00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff4e00098f0 tx=0x7ff4e0009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:06.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.679+0000 7ff4f2ed3700 1 -- 172.21.15.3:0/3283873563 shutdown_connections 2023-10-03T10:14:06.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.679+0000 7ff4f2ed3700 1 --2- 172.21.15.3:0/3283873563 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff4ec138d80 0x7ff4ec13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.679+0000 7ff4f2ed3700 1 --2- 172.21.15.3:0/3283873563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4ec0a0a30 0x7ff4ec0a0e00 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.679+0000 7ff4f2ed3700 1 -- 172.21.15.3:0/3283873563 >> 172.21.15.3:0/3283873563 conn(0x7ff4ec0a3050 msgr2=0x7ff4ec0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:06.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.679+0000 7ff4f2ed3700 1 -- 172.21.15.3:0/3283873563 shutdown_connections 2023-10-03T10:14:06.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.679+0000 7ff4f2ed3700 1 -- 172.21.15.3:0/3283873563 wait complete. 2023-10-03T10:14:06.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.680+0000 7ff4f2ed3700 1 Processor -- start 2023-10-03T10:14:06.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.680+0000 7ff4f2ed3700 1 -- start start 2023-10-03T10:14:06.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.680+0000 7ff4f2ed3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4ec09d630 0x7ff4ec133710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:06.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.680+0000 7ff4f2ed3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff4ec0a0a30 0x7ff4ec133c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:06.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.680+0000 7ff4f2ed3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff4ec138d80 0x7ff4ec134190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:06.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.680+0000 7ff4f2ed3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec137d00 con 0x7ff4ec09d630 2023-10-03T10:14:06.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.680+0000 7ff4f2ed3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec0a4b00 con 0x7ff4ec0a0a30 2023-10-03T10:14:06.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.681+0000 7ff4f2ed3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec0a0410 con 0x7ff4ec138d80 2023-10-03T10:14:06.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.681+0000 7ff4f0c6f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4ec09d630 0x7ff4ec133710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:06.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.681+0000 7ff4f0c6f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4ec09d630 0x7ff4ec133710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57828/0 (socket says 172.21.15.3:57828) 2023-10-03T10:14:06.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.681+0000 7ff4f0c6f700 1 -- 172.21.15.3:0/2344784133 learned_addr learned my addr 172.21.15.3:0/2344784133 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:06.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.681+0000 7ff4f1470700 1 --2- 172.21.15.3:0/2344784133 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff4ec138d80 0x7ff4ec134190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:06.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.681+0000 7ff4ebfff700 1 --2- 172.21.15.3:0/2344784133 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff4ec0a0a30 0x7ff4ec133c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:06.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.681+0000 7ff4f0c6f700 1 -- 172.21.15.3:0/2344784133 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff4ec138d80 msgr2=0x7ff4ec134190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:06.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.681+0000 7ff4f0c6f700 1 --2- 172.21.15.3:0/2344784133 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff4ec138d80 0x7ff4ec134190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.681+0000 7ff4f0c6f700 1 -- 172.21.15.3:0/2344784133 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff4ec0a0a30 msgr2=0x7ff4ec133c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:06.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.681+0000 7ff4f0c6f700 1 --2- 172.21.15.3:0/2344784133 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff4ec0a0a30 0x7ff4ec133c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.681+0000 7ff4f0c6f700 1 -- 172.21.15.3:0/2344784133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e0009550 con 0x7ff4ec09d630 2023-10-03T10:14:06.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.681+0000 7ff4f0c6f700 1 --2- 172.21.15.3:0/2344784133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4ec09d630 0x7ff4ec133710 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0009c40 tx=0x7ff4e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:06.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.681+0000 7ff4e97fa700 1 -- 172.21.15.3:0/2344784133 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0004510 con 0x7ff4ec09d630 2023-10-03T10:14:06.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.681+0000 7ff4e97fa700 1 -- 172.21.15.3:0/2344784133 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e00036a0 con 0x7ff4ec09d630 2023-10-03T10:14:06.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.682+0000 7ff4f2ed3700 1 -- 172.21.15.3:0/2344784133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4ec1364d0 con 0x7ff4ec09d630 2023-10-03T10:14:06.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.682+0000 7ff4e97fa700 1 -- 172.21.15.3:0/2344784133 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0003a50 con 0x7ff4ec09d630 2023-10-03T10:14:06.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.682+0000 7ff4f2ed3700 1 -- 172.21.15.3:0/2344784133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4ec137120 con 0x7ff4ec09d630 2023-10-03T10:14:06.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.682+0000 7ff4e97fa700 1 -- 172.21.15.3:0/2344784133 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff4e0021070 con 0x7ff4ec09d630 2023-10-03T10:14:06.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.683+0000 7ff4e97fa700 1 -- 172.21.15.3:0/2344784133 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff4e0040b90 con 0x7ff4ec09d630 2023-10-03T10:14:06.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.683+0000 7ff4f2ed3700 1 -- 172.21.15.3:0/2344784133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4ec0747f0 con 0x7ff4ec09d630 2023-10-03T10:14:06.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.688+0000 7ff4e97fa700 1 -- 172.21.15.3:0/2344784133 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff4e0029070 con 0x7ff4ec09d630 2023-10-03T10:14:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.717+0000 7f12c1d9e700 1 Processor -- start 2023-10-03T10:14:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.717+0000 7f12c1d9e700 1 -- start start 2023-10-03T10:14:06.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.717+0000 7f12c1d9e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12bc0a0a30 0x7f12bc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:06.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.717+0000 7f12c1d9e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f12bc138d80 0x7f12bc13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:06.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.717+0000 7f12c1d9e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12bc0a2a70 con 0x7f12bc09d630 2023-10-03T10:14:06.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.717+0000 7f12c1d9e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12bc0a01d0 con 0x7f12bc138d80 2023-10-03T10:14:06.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.717+0000 7f12c1d9e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12bc09e2b0 con 0x7f12bc0a0a30 2023-10-03T10:14:06.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.718+0000 7f12bbfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f12bc138d80 0x7f12bc13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:06.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.718+0000 7f12bb7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12bc0a0a30 0x7f12bc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.718+0000 7f12bb7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12bc0a0a30 0x7f12bc0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:40098/0 (socket says 172.21.15.3:40098) 2023-10-03T10:14:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.718+0000 7f12bbfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f12bc138d80 0x7f12bc13b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57836/0 (socket says 172.21.15.3:57836) 2023-10-03T10:14:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.718+0000 7f12b2ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f12bc09d630 0x7f12bc0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54296/0 (socket says 172.21.15.3:54296) 2023-10-03T10:14:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.718+0000 7f12bb7fe700 1 -- 172.21.15.3:0/2566598046 learned_addr learned my addr 172.21.15.3:0/2566598046 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.718+0000 7f12bbfff700 1 -- 172.21.15.3:0/2566598046 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12bc0a0a30 msgr2=0x7f12bc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.718+0000 7f12bbfff700 1 --2- 172.21.15.3:0/2566598046 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12bc0a0a30 0x7f12bc0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.718+0000 7f12bbfff700 1 -- 172.21.15.3:0/2566598046 >> v1:172.21.15.3:6789/0 conn(0x7f12bc09d630 legacy=0x7f12bc0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.718+0000 7f12bbfff700 1 -- 172.21.15.3:0/2566598046 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12bc09dc50 con 0x7f12bc138d80 2023-10-03T10:14:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.718+0000 7f12bbfff700 1 --2- 172.21.15.3:0/2566598046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12bc138d80 0x7f12bc13b160 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009d90 tx=0x7f12b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa576b83196d813 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.718+0000 7f12baffd700 1 -- 172.21.15.3:0/2566598046 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b400e070 con 0x7f12bc138d80 2023-10-03T10:14:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.719+0000 7f12baffd700 1 -- 172.21.15.3:0/2566598046 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b400bb20 con 0x7f12bc138d80 2023-10-03T10:14:06.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.719+0000 7f12baffd700 1 -- 172.21.15.3:0/2566598046 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b4012430 con 0x7f12bc138d80 2023-10-03T10:14:06.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.719+0000 7f12c1d9e700 1 -- 172.21.15.3:0/2566598046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12bc138d80 msgr2=0x7f12bc13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:06.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.719+0000 7f12c1d9e700 1 --2- 172.21.15.3:0/2566598046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12bc138d80 0x7f12bc13b160 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009d90 tx=0x7f12b400b330 comp rx=0 tx=0).stop 2023-10-03T10:14:06.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.719+0000 7f12c1d9e700 1 -- 172.21.15.3:0/2566598046 shutdown_connections 2023-10-03T10:14:06.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.719+0000 7f12c1d9e700 1 --2- 172.21.15.3:0/2566598046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12bc138d80 0x7f12bc13b160 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.719+0000 7f12c1d9e700 1 --2- 172.21.15.3:0/2566598046 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12bc0a0a30 0x7f12bc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.719+0000 7f12c1d9e700 1 -- 172.21.15.3:0/2566598046 >> 172.21.15.3:0/2566598046 conn(0x7f12bc0a3050 msgr2=0x7f12bc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:06.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.719+0000 7f12c1d9e700 1 -- 172.21.15.3:0/2566598046 shutdown_connections 2023-10-03T10:14:06.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.719+0000 7f12c1d9e700 1 -- 172.21.15.3:0/2566598046 wait complete. 2023-10-03T10:14:06.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.720+0000 7f12c1d9e700 1 Processor -- start 2023-10-03T10:14:06.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.721+0000 7f12c1d9e700 1 -- start start 2023-10-03T10:14:06.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.721+0000 7f12c1d9e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12bc09d630 0x7f12bc133890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:06.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.721+0000 7f12c1d9e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12bc0a0a30 0x7f12bc133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:06.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.721+0000 7f12c1d9e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12bc138d80 0x7f12bc134310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:06.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.721+0000 7f12c1d9e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12bc1371b0 con 0x7f12bc0a0a30 2023-10-03T10:14:06.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.721+0000 7f12c1d9e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12bc138030 con 0x7f12bc09d630 2023-10-03T10:14:06.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.721+0000 7f12c1d9e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12bc1373b0 con 0x7f12bc138d80 2023-10-03T10:14:06.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.721+0000 7f12bbfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12bc138d80 0x7f12bc134310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:06.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.721+0000 7f12bbfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12bc138d80 0x7f12bc134310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:40106/0 (socket says 172.21.15.3:40106) 2023-10-03T10:14:06.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.721+0000 7f12bbfff700 1 -- 172.21.15.3:0/1010501496 learned_addr learned my addr 172.21.15.3:0/1010501496 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:06.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.721+0000 7f12bb7fe700 1 --2- 172.21.15.3:0/1010501496 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12bc09d630 0x7f12bc133890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:06.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.721+0000 7f12b2ffd700 1 --2- 172.21.15.3:0/1010501496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12bc0a0a30 0x7f12bc133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:06.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.722+0000 7f12bbfff700 1 -- 172.21.15.3:0/1010501496 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12bc09d630 msgr2=0x7f12bc133890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:06.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.722+0000 7f12bbfff700 1 --2- 172.21.15.3:0/1010501496 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12bc09d630 0x7f12bc133890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.722+0000 7f12bbfff700 1 -- 172.21.15.3:0/1010501496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12bc0a0a30 msgr2=0x7f12bc133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:06.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.722+0000 7f12bbfff700 1 --2- 172.21.15.3:0/1010501496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12bc0a0a30 0x7f12bc133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.722+0000 7f12bbfff700 1 -- 172.21.15.3:0/1010501496 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b4009a70 con 0x7f12bc138d80 2023-10-03T10:14:06.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.722+0000 7f12bbfff700 1 --2- 172.21.15.3:0/1010501496 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12bc138d80 0x7f12bc134310 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f12b4013000 tx=0x7f12b4004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:06.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.722+0000 7f12b8ff9700 1 -- 172.21.15.3:0/1010501496 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b4012430 con 0x7f12bc138d80 2023-10-03T10:14:06.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.722+0000 7f12b8ff9700 1 -- 172.21.15.3:0/1010501496 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b4020070 con 0x7f12bc138d80 2023-10-03T10:14:06.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.722+0000 7f12c1d9e700 1 -- 172.21.15.3:0/1010501496 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12bc1382d0 con 0x7f12bc138d80 2023-10-03T10:14:06.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.722+0000 7f12b8ff9700 1 -- 172.21.15.3:0/1010501496 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b4003ea0 con 0x7f12bc138d80 2023-10-03T10:14:06.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.722+0000 7f12c1d9e700 1 -- 172.21.15.3:0/1010501496 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12bc136b00 con 0x7f12bc138d80 2023-10-03T10:14:06.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.723+0000 7f12b8ff9700 1 -- 172.21.15.3:0/1010501496 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f12b4004280 con 0x7f12bc138d80 2023-10-03T10:14:06.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.723+0000 7f12b1ffb700 1 -- 172.21.15.3:0/1010501496 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12940052e0 con 0x7f12bc138d80 2023-10-03T10:14:06.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.723+0000 7f12b8ff9700 1 -- 172.21.15.3:0/1010501496 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f12b4022070 con 0x7f12bc138d80 2023-10-03T10:14:06.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.729+0000 7f12b8ff9700 1 -- 172.21.15.3:0/1010501496 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f12b4012e60 con 0x7f12bc138d80 2023-10-03T10:14:06.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.801+0000 7fd2673cd700 1 -- 172.21.15.3:0/1012176908 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd260074ea0 con 0x7fd260130a40 2023-10-03T10:14:06.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.801+0000 7fd255ffb700 1 -- 172.21.15.3:0/1012176908 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd250017030 con 0x7fd260130a40 2023-10-03T10:14:06.801 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:06.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.803+0000 7fd2673cd700 1 -- 172.21.15.3:0/1012176908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd260130a40 msgr2=0x7fd26009ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:06.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.803+0000 7fd2673cd700 1 --2- 172.21.15.3:0/1012176908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd260130a40 0x7fd26009ce00 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd25000c4f0 tx=0x7fd25000c800 comp rx=0 tx=0).stop 2023-10-03T10:14:06.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.803+0000 7fd2673cd700 1 -- 172.21.15.3:0/1012176908 shutdown_connections 2023-10-03T10:14:06.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.803+0000 7fd2673cd700 1 --2- 172.21.15.3:0/1012176908 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd260133a80 0x7fd26009d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.803+0000 7fd2673cd700 1 --2- 172.21.15.3:0/1012176908 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd26012ff20 0x7fd26009c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.803+0000 7fd2673cd700 1 --2- 172.21.15.3:0/1012176908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd260130a40 0x7fd26009ce00 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.803+0000 7fd2673cd700 1 -- 172.21.15.3:0/1012176908 >> 172.21.15.3:0/1012176908 conn(0x7fd2600a4210 msgr2=0x7fd26009e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:06.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.803+0000 7fd2673cd700 1 -- 172.21.15.3:0/1012176908 shutdown_connections 2023-10-03T10:14:06.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.803+0000 7fd2673cd700 1 -- 172.21.15.3:0/1012176908 wait complete. 2023-10-03T10:14:06.818 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:06.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.894+0000 7ff4f2ed3700 1 -- 172.21.15.3:0/2344784133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff4ec136ac0 con 0x7ff4ec09d630 2023-10-03T10:14:06.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.895+0000 7ff4e97fa700 1 -- 172.21.15.3:0/2344784133 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff4e0015030 con 0x7ff4ec09d630 2023-10-03T10:14:06.895 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:06.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.897+0000 7ff4f2ed3700 1 -- 172.21.15.3:0/2344784133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4ec09d630 msgr2=0x7ff4ec133710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:06.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.897+0000 7ff4f2ed3700 1 --2- 172.21.15.3:0/2344784133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4ec09d630 0x7ff4ec133710 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0009c40 tx=0x7ff4e0004750 comp rx=0 tx=0).stop 2023-10-03T10:14:06.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.898+0000 7ff4f2ed3700 1 -- 172.21.15.3:0/2344784133 shutdown_connections 2023-10-03T10:14:06.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.898+0000 7ff4f2ed3700 1 --2- 172.21.15.3:0/2344784133 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff4ec138d80 0x7ff4ec134190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.898+0000 7ff4f2ed3700 1 --2- 172.21.15.3:0/2344784133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4ec09d630 0x7ff4ec133710 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.898+0000 7ff4f2ed3700 1 --2- 172.21.15.3:0/2344784133 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff4ec0a0a30 0x7ff4ec133c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.898+0000 7ff4f2ed3700 1 -- 172.21.15.3:0/2344784133 >> 172.21.15.3:0/2344784133 conn(0x7ff4ec0a3050 msgr2=0x7ff4ec13ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:06.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.898+0000 7ff4f2ed3700 1 -- 172.21.15.3:0/2344784133 shutdown_connections 2023-10-03T10:14:06.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.898+0000 7ff4f2ed3700 1 -- 172.21.15.3:0/2344784133 wait complete. 2023-10-03T10:14:06.912 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:06.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.936+0000 7f12b1ffb700 1 -- 172.21.15.3:0/1010501496 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1294005590 con 0x7f12bc138d80 2023-10-03T10:14:06.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.937+0000 7f12b8ff9700 1 -- 172.21.15.3:0/1010501496 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f12b401a380 con 0x7f12bc138d80 2023-10-03T10:14:06.937 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:06.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.939+0000 7f12b1ffb700 1 -- 172.21.15.3:0/1010501496 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12bc138d80 msgr2=0x7f12bc134310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:06.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.939+0000 7f12b1ffb700 1 --2- 172.21.15.3:0/1010501496 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12bc138d80 0x7f12bc134310 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f12b4013000 tx=0x7f12b4004ad0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.939+0000 7f12b1ffb700 1 -- 172.21.15.3:0/1010501496 shutdown_connections 2023-10-03T10:14:06.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.939+0000 7f12b1ffb700 1 --2- 172.21.15.3:0/1010501496 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12bc138d80 0x7f12bc134310 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.939+0000 7f12b1ffb700 1 --2- 172.21.15.3:0/1010501496 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12bc09d630 0x7f12bc133890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.939+0000 7f12b1ffb700 1 --2- 172.21.15.3:0/1010501496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12bc0a0a30 0x7f12bc133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:06.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.939+0000 7f12b1ffb700 1 -- 172.21.15.3:0/1010501496 >> 172.21.15.3:0/1010501496 conn(0x7f12bc0a3050 msgr2=0x7f12bc09f620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:06.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.939+0000 7f12b1ffb700 1 -- 172.21.15.3:0/1010501496 shutdown_connections 2023-10-03T10:14:06.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:06.939+0000 7f12b1ffb700 1 -- 172.21.15.3:0/1010501496 wait complete. 2023-10-03T10:14:06.953 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:07.350 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:07.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.495+0000 7f0142b89700 1 Processor -- start 2023-10-03T10:14:07.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.495+0000 7f0142b89700 1 -- start start 2023-10-03T10:14:07.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.496+0000 7f0142b89700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f013c09e930 0x7f013c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:07.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.496+0000 7f0142b89700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f013c09f650 0x7f013c131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:07.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.496+0000 7f0142b89700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f013c070940 con 0x7f013c09f240 2023-10-03T10:14:07.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.496+0000 7f0142b89700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f013c06fd60 con 0x7f013c09f650 2023-10-03T10:14:07.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.496+0000 7f0142b89700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f013c06fa80 con 0x7f013c09e930 2023-10-03T10:14:07.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.496+0000 7f0141126700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f013c09f650 0x7f013c131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:07.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.496+0000 7f0140925700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f013c09e930 0x7f013c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:07.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.496+0000 7f013bfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f013c09f240 0x7f013c1317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54298/0 (socket says 172.21.15.3:54298) 2023-10-03T10:14:07.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.496+0000 7f013bfff700 1 -- 172.21.15.3:0/983170815 learned_addr learned my addr 172.21.15.3:0/983170815 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:07.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.496+0000 7f013b7fe700 1 -- 172.21.15.3:0/983170815 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3128738830 0 0) 0x7f013c070940 con 0x7f013c09f240 2023-10-03T10:14:07.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.496+0000 7f0141126700 1 -- 172.21.15.3:0/983170815 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f013c09e930 msgr2=0x7f013c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:07.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.496+0000 7f0141126700 1 --2- 172.21.15.3:0/983170815 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f013c09e930 0x7f013c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:07.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.496+0000 7f0141126700 1 -- 172.21.15.3:0/983170815 >> v1:172.21.15.3:6789/0 conn(0x7f013c09f240 legacy=0x7f013c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:07.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.496+0000 7f0141126700 1 -- 172.21.15.3:0/983170815 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f013c09fb60 con 0x7f013c09f650 2023-10-03T10:14:07.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.496+0000 7f0140925700 1 --2- 172.21.15.3:0/983170815 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f013c09e930 0x7f013c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:07.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.496+0000 7f0141126700 1 --2- 172.21.15.3:0/983170815 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f013c09f650 0x7f013c131f00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0134009dc0 tx=0x7f013400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6efaf3a87f3b624a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:07.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.496+0000 7f013b7fe700 1 -- 172.21.15.3:0/983170815 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013400e070 con 0x7f013c09f650 2023-10-03T10:14:07.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.497+0000 7f013b7fe700 1 -- 172.21.15.3:0/983170815 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013400bb20 con 0x7f013c09f650 2023-10-03T10:14:07.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.497+0000 7f013b7fe700 1 -- 172.21.15.3:0/983170815 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134012430 con 0x7f013c09f650 2023-10-03T10:14:07.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.497+0000 7f0142b89700 1 -- 172.21.15.3:0/983170815 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f013c09f650 msgr2=0x7f013c131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:07.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.497+0000 7f0142b89700 1 --2- 172.21.15.3:0/983170815 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f013c09f650 0x7f013c131f00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0134009dc0 tx=0x7f013400b330 comp rx=0 tx=0).stop 2023-10-03T10:14:07.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.497+0000 7f0142b89700 1 -- 172.21.15.3:0/983170815 shutdown_connections 2023-10-03T10:14:07.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.497+0000 7f0142b89700 1 --2- 172.21.15.3:0/983170815 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f013c09f650 0x7f013c131f00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:07.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.497+0000 7f0142b89700 1 --2- 172.21.15.3:0/983170815 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f013c09e930 0x7f013c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:07.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.497+0000 7f0142b89700 1 -- 172.21.15.3:0/983170815 >> 172.21.15.3:0/983170815 conn(0x7f013c0a4210 msgr2=0x7f013c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:07.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.497+0000 7f0142b89700 1 -- 172.21.15.3:0/983170815 shutdown_connections 2023-10-03T10:14:07.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.497+0000 7f0142b89700 1 -- 172.21.15.3:0/983170815 wait complete. 2023-10-03T10:14:07.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.498+0000 7f0142b89700 1 Processor -- start 2023-10-03T10:14:07.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.498+0000 7f0142b89700 1 -- start start 2023-10-03T10:14:07.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.499+0000 7f0142b89700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f013c09e930 0x7f013c1a3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:07.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.499+0000 7f0142b89700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f013c09f240 0x7f013c1c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:07.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.499+0000 7f0142b89700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f013c09f650 0x7f013c1c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:07.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.499+0000 7f0142b89700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013c1a4f60 con 0x7f013c09e930 2023-10-03T10:14:07.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.499+0000 7f0142b89700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f013c1b6f70 con 0x7f013c09f650 2023-10-03T10:14:07.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.499+0000 7f0142b89700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f013c071d20 con 0x7f013c09f240 2023-10-03T10:14:07.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.499+0000 7f0140925700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f013c09e930 0x7f013c1a3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:07.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.499+0000 7f013bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f013c09f240 0x7f013c1c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:07.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.499+0000 7f0141126700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f013c09f650 0x7f013c1c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:07.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.499+0000 7f0140925700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f013c09e930 0x7f013c1a3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57840/0 (socket says 172.21.15.3:57840) 2023-10-03T10:14:07.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.499+0000 7f0140925700 1 -- 172.21.15.3:0/2473023176 learned_addr learned my addr 172.21.15.3:0/2473023176 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:07.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.499+0000 7f0140925700 1 -- 172.21.15.3:0/2473023176 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f013c09f240 msgr2=0x7f013c1c5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:07.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.499+0000 7f0140925700 1 --2- 172.21.15.3:0/2473023176 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f013c09f240 0x7f013c1c5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:07.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.499+0000 7f0140925700 1 -- 172.21.15.3:0/2473023176 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f013c09f650 msgr2=0x7f013c1c6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:07.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.499+0000 7f0140925700 1 --2- 172.21.15.3:0/2473023176 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f013c09f650 0x7f013c1c6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:07.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.499+0000 7f0140925700 1 -- 172.21.15.3:0/2473023176 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01300094a0 con 0x7f013c09e930 2023-10-03T10:14:07.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.499+0000 7f013bfff700 1 --2- 172.21.15.3:0/2473023176 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f013c09f240 0x7f013c1c5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:07.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.500+0000 7f0140925700 1 --2- 172.21.15.3:0/2473023176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f013c09e930 0x7f013c1a3200 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f013000ebe0 tx=0x7f013000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:07.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.500+0000 7f01397fa700 1 -- 172.21.15.3:0/2473023176 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013000c850 con 0x7f013c09e930 2023-10-03T10:14:07.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.500+0000 7f01397fa700 1 -- 172.21.15.3:0/2473023176 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013000c9b0 con 0x7f013c09e930 2023-10-03T10:14:07.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.500+0000 7f0142b89700 1 -- 172.21.15.3:0/2473023176 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0134009a70 con 0x7f013c09e930 2023-10-03T10:14:07.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.500+0000 7f01397fa700 1 -- 172.21.15.3:0/2473023176 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013000cd40 con 0x7f013c09e930 2023-10-03T10:14:07.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.500+0000 7f0142b89700 1 -- 172.21.15.3:0/2473023176 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f013c0a0190 con 0x7f013c09e930 2023-10-03T10:14:07.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.500+0000 7f01397fa700 1 -- 172.21.15.3:0/2473023176 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0130013470 con 0x7f013c09e930 2023-10-03T10:14:07.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.501+0000 7f01397fa700 1 -- 172.21.15.3:0/2473023176 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f013003f730 con 0x7f013c09e930 2023-10-03T10:14:07.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.501+0000 7f0142b89700 1 -- 172.21.15.3:0/2473023176 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f013c074ea0 con 0x7f013c09e930 2023-10-03T10:14:07.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.506+0000 7f01397fa700 1 -- 172.21.15.3:0/2473023176 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0130013720 con 0x7f013c09e930 2023-10-03T10:14:07.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.711+0000 7f0142b89700 1 -- 172.21.15.3:0/2473023176 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f013c09ffe0 con 0x7f013c09e930 2023-10-03T10:14:07.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.711+0000 7f01397fa700 1 -- 172.21.15.3:0/2473023176 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0130021020 con 0x7f013c09e930 2023-10-03T10:14:07.712 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:07.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.714+0000 7f0142b89700 1 -- 172.21.15.3:0/2473023176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f013c09e930 msgr2=0x7f013c1a3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:07.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.714+0000 7f0142b89700 1 --2- 172.21.15.3:0/2473023176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f013c09e930 0x7f013c1a3200 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f013000ebe0 tx=0x7f013000eef0 comp rx=0 tx=0).stop 2023-10-03T10:14:07.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.714+0000 7f0142b89700 1 -- 172.21.15.3:0/2473023176 shutdown_connections 2023-10-03T10:14:07.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.714+0000 7f0142b89700 1 --2- 172.21.15.3:0/2473023176 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f013c09f650 0x7f013c1c6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:07.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.714+0000 7f0142b89700 1 --2- 172.21.15.3:0/2473023176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f013c09e930 0x7f013c1a3200 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:07.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.714+0000 7f0142b89700 1 --2- 172.21.15.3:0/2473023176 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f013c09f240 0x7f013c1c5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:07.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.714+0000 7f0142b89700 1 -- 172.21.15.3:0/2473023176 >> 172.21.15.3:0/2473023176 conn(0x7f013c0a4210 msgr2=0x7f013c0a27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:07.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.714+0000 7f0142b89700 1 -- 172.21.15.3:0/2473023176 shutdown_connections 2023-10-03T10:14:07.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.714+0000 7f0142b89700 1 -- 172.21.15.3:0/2473023176 wait complete. 2023-10-03T10:14:07.728 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:07.819 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:07.913 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:07.953 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:07.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.966+0000 7f2e7b6cc700 1 Processor -- start 2023-10-03T10:14:07.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.966+0000 7f2e7b6cc700 1 -- start start 2023-10-03T10:14:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.966+0000 7f2e7b6cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e7409d6f0 0x7f2e7409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.966+0000 7f2e7b6cc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2e7409e000 0x7f2e741339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.966+0000 7f2e7b6cc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e7409e410 0x7f2e74133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.966+0000 7f2e7b6cc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e7406b460 con 0x7f2e7409e000 2023-10-03T10:14:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.966+0000 7f2e7b6cc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7409b120 con 0x7f2e7409d6f0 2023-10-03T10:14:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.966+0000 7f2e7b6cc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7409a7e0 con 0x7f2e7409e410 2023-10-03T10:14:07.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.967+0000 7f2e78c67700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2e7409e000 0x7f2e741339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:07.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.967+0000 7f2e79c69700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e7409e410 0x7f2e74133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:07.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.967+0000 7f2e78c67700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2e7409e000 0x7f2e741339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57852/0 (socket says 172.21.15.3:57852) 2023-10-03T10:14:07.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.967+0000 7f2e78c67700 1 -- 172.21.15.3:0/2691519775 learned_addr learned my addr 172.21.15.3:0/2691519775 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:07.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.967+0000 7f2e79468700 1 --2- 172.21.15.3:0/2691519775 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e7409d6f0 0x7f2e7409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:07.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.967+0000 7f2e78c67700 1 -- 172.21.15.3:0/2691519775 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e7409e410 msgr2=0x7f2e74133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:07.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.967+0000 7f2e78c67700 1 --2- 172.21.15.3:0/2691519775 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e7409e410 0x7f2e74133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:07.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.967+0000 7f2e78c67700 1 -- 172.21.15.3:0/2691519775 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e7409d6f0 msgr2=0x7f2e7409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:07.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.967+0000 7f2e78c67700 1 --2- 172.21.15.3:0/2691519775 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e7409d6f0 0x7f2e7409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:07.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.967+0000 7f2e78c67700 1 -- 172.21.15.3:0/2691519775 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7409a500 con 0x7f2e7409e000 2023-10-03T10:14:07.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.967+0000 7f2e78c67700 1 --2- 172.21.15.3:0/2691519775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e7409e000 0x7f2e741339f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2e640098f0 tx=0x7f2e64009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd21e4be43f6f46 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:07.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.967+0000 7f2e6bfff700 1 -- 172.21.15.3:0/2691519775 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e64011830 con 0x7f2e7409e000 2023-10-03T10:14:07.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.968+0000 7f2e6bfff700 1 -- 172.21.15.3:0/2691519775 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e64011990 con 0x7f2e7409e000 2023-10-03T10:14:07.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.968+0000 7f2e6bfff700 1 -- 172.21.15.3:0/2691519775 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e64011d20 con 0x7f2e7409e000 2023-10-03T10:14:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.968+0000 7f2e7b6cc700 1 -- 172.21.15.3:0/2691519775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e7409e000 msgr2=0x7f2e741339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.968+0000 7f2e7b6cc700 1 --2- 172.21.15.3:0/2691519775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e7409e000 0x7f2e741339f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2e640098f0 tx=0x7f2e64009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.968+0000 7f2e7b6cc700 1 -- 172.21.15.3:0/2691519775 shutdown_connections 2023-10-03T10:14:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.968+0000 7f2e7b6cc700 1 --2- 172.21.15.3:0/2691519775 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e7409e410 0x7f2e74133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.968+0000 7f2e7b6cc700 1 --2- 172.21.15.3:0/2691519775 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e7409d6f0 0x7f2e7409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.968+0000 7f2e7b6cc700 1 --2- 172.21.15.3:0/2691519775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e7409e000 0x7f2e741339f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.968+0000 7f2e7b6cc700 1 -- 172.21.15.3:0/2691519775 >> 172.21.15.3:0/2691519775 conn(0x7f2e74066f50 msgr2=0x7f2e7409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.969+0000 7f2e7b6cc700 1 -- 172.21.15.3:0/2691519775 shutdown_connections 2023-10-03T10:14:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.969+0000 7f2e7b6cc700 1 -- 172.21.15.3:0/2691519775 wait complete. 2023-10-03T10:14:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.970+0000 7f2e7b6cc700 1 Processor -- start 2023-10-03T10:14:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.970+0000 7f2e7b6cc700 1 -- start start 2023-10-03T10:14:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.970+0000 7f2e7b6cc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e7409d6f0 0x7f2e741a5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.970+0000 7f2e7b6cc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e7409e000 0x7f2e741a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.970+0000 7f2e7b6cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e7409e410 0x7f2e741a6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.970+0000 7f2e7b6cc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e741a7bc0 con 0x7f2e7409d6f0 2023-10-03T10:14:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.970+0000 7f2e7b6cc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e741a81e0 con 0x7f2e7409e410 2023-10-03T10:14:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.970+0000 7f2e7b6cc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e741a8e10 con 0x7f2e7409e000 2023-10-03T10:14:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.970+0000 7f2e79468700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e7409d6f0 0x7f2e741a5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.971+0000 7f2e79468700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e7409d6f0 0x7f2e741a5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57862/0 (socket says 172.21.15.3:57862) 2023-10-03T10:14:07.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.971+0000 7f2e79468700 1 -- 172.21.15.3:0/2537264521 learned_addr learned my addr 172.21.15.3:0/2537264521 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:07.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.971+0000 7f2e78c67700 1 --2- 172.21.15.3:0/2537264521 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e7409e000 0x7f2e741a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:07.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.971+0000 7f2e79c69700 1 --2- 172.21.15.3:0/2537264521 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e7409e410 0x7f2e741a6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:07.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.971+0000 7f2e79468700 1 -- 172.21.15.3:0/2537264521 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e7409e000 msgr2=0x7f2e741a64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:07.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.971+0000 7f2e79468700 1 --2- 172.21.15.3:0/2537264521 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e7409e000 0x7f2e741a64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:07.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.971+0000 7f2e79468700 1 -- 172.21.15.3:0/2537264521 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e7409e410 msgr2=0x7f2e741a6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:07.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.971+0000 7f2e79468700 1 --2- 172.21.15.3:0/2537264521 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e7409e410 0x7f2e741a6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:07.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.971+0000 7f2e79468700 1 -- 172.21.15.3:0/2537264521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e64009550 con 0x7f2e7409d6f0 2023-10-03T10:14:07.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.971+0000 7f2e79468700 1 --2- 172.21.15.3:0/2537264521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e7409d6f0 0x7f2e741a5fb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2e7000e9e0 tx=0x7f2e7000ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:07.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.971+0000 7f2e69ffb700 1 -- 172.21.15.3:0/2537264521 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e70009710 con 0x7f2e7409d6f0 2023-10-03T10:14:07.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.971+0000 7f2e69ffb700 1 -- 172.21.15.3:0/2537264521 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e7000ee80 con 0x7f2e7409d6f0 2023-10-03T10:14:07.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.971+0000 7f2e7b6cc700 1 -- 172.21.15.3:0/2537264521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e741a97f0 con 0x7f2e7409d6f0 2023-10-03T10:14:07.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.971+0000 7f2e69ffb700 1 -- 172.21.15.3:0/2537264521 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7000ca50 con 0x7f2e7409d6f0 2023-10-03T10:14:07.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.972+0000 7f2e7b6cc700 1 -- 172.21.15.3:0/2537264521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e7409c710 con 0x7f2e7409d6f0 2023-10-03T10:14:07.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.973+0000 7f2e7b6cc700 1 -- 172.21.15.3:0/2537264521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e7409de20 con 0x7f2e7409d6f0 2023-10-03T10:14:07.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.974+0000 7f2e69ffb700 1 -- 172.21.15.3:0/2537264521 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2e7000cbb0 con 0x7f2e7409d6f0 2023-10-03T10:14:07.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.974+0000 7f2e69ffb700 1 -- 172.21.15.3:0/2537264521 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2e7003ff00 con 0x7f2e7409d6f0 2023-10-03T10:14:07.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:07.979+0000 7f2e69ffb700 1 -- 172.21.15.3:0/2537264521 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2e7001a360 con 0x7f2e7409d6f0 2023-10-03T10:14:08.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.060+0000 7fea17241700 1 Processor -- start 2023-10-03T10:14:08.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.060+0000 7fea17241700 1 -- start start 2023-10-03T10:14:08.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.061+0000 7fea17241700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fea10133420 0x7fea101337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:08.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.061+0000 7fea17241700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea10134260 0x7fea101306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:08.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.061+0000 7fea17241700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea1006b990 con 0x7fea10133e30 2023-10-03T10:14:08.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.061+0000 7fea17241700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea100a0590 con 0x7fea10133420 2023-10-03T10:14:08.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.061+0000 7fea17241700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1009fb80 con 0x7fea10134260 2023-10-03T10:14:08.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.061+0000 7fea14fdd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fea10133420 0x7fea101337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:08.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.061+0000 7fea0ffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fea10133e30 0x7fea1012ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54300/0 (socket says 172.21.15.3:54300) 2023-10-03T10:14:08.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.061+0000 7fea0ffff700 1 -- 172.21.15.3:0/3216808700 learned_addr learned my addr 172.21.15.3:0/3216808700 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:08.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.061+0000 7fea157de700 1 --2- 172.21.15.3:0/3216808700 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea10134260 0x7fea101306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:08.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.062+0000 7fea14fdd700 1 -- 172.21.15.3:0/3216808700 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea10134260 msgr2=0x7fea101306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:08.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.062+0000 7fea14fdd700 1 --2- 172.21.15.3:0/3216808700 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea10134260 0x7fea101306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:08.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.062+0000 7fea14fdd700 1 -- 172.21.15.3:0/3216808700 >> v1:172.21.15.3:6789/0 conn(0x7fea10133e30 legacy=0x7fea1012ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:08.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.062+0000 7fea14fdd700 1 -- 172.21.15.3:0/3216808700 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea100a0030 con 0x7fea10133420 2023-10-03T10:14:08.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.062+0000 7fea14fdd700 1 --2- 172.21.15.3:0/3216808700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea10133420 0x7fea101337f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fea04009840 tx=0x7fea04009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba30189c6e81bd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:08.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.062+0000 7fea0f7fe700 1 -- 172.21.15.3:0/3216808700 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea04011830 con 0x7fea10133420 2023-10-03T10:14:08.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.062+0000 7fea0f7fe700 1 -- 172.21.15.3:0/3216808700 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea04011990 con 0x7fea10133420 2023-10-03T10:14:08.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.062+0000 7fea0f7fe700 1 -- 172.21.15.3:0/3216808700 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea04011d20 con 0x7fea10133420 2023-10-03T10:14:08.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.063+0000 7fea17241700 1 -- 172.21.15.3:0/3216808700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea10133420 msgr2=0x7fea101337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:08.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.063+0000 7fea17241700 1 --2- 172.21.15.3:0/3216808700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea10133420 0x7fea101337f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fea04009840 tx=0x7fea04009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:08.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.063+0000 7fea17241700 1 -- 172.21.15.3:0/3216808700 shutdown_connections 2023-10-03T10:14:08.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.063+0000 7fea17241700 1 --2- 172.21.15.3:0/3216808700 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea10134260 0x7fea101306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:08.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.063+0000 7fea17241700 1 --2- 172.21.15.3:0/3216808700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea10133420 0x7fea101337f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:08.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.063+0000 7fea17241700 1 -- 172.21.15.3:0/3216808700 >> 172.21.15.3:0/3216808700 conn(0x7fea100a4210 msgr2=0x7fea100a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:08.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.063+0000 7fea17241700 1 -- 172.21.15.3:0/3216808700 shutdown_connections 2023-10-03T10:14:08.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.063+0000 7fea17241700 1 -- 172.21.15.3:0/3216808700 wait complete. 2023-10-03T10:14:08.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.064+0000 7fea17241700 1 Processor -- start 2023-10-03T10:14:08.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.064+0000 7fea17241700 1 -- start start 2023-10-03T10:14:08.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.065+0000 7fea17241700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea10133420 0x7fea1009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:08.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.065+0000 7fea17241700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea10133e30 0x7fea1009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:08.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.065+0000 7fea17241700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea10134260 0x7fea1009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:08.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.065+0000 7fea17241700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1009d9e0 con 0x7fea10133e30 2023-10-03T10:14:08.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.065+0000 7fea17241700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1011fce0 con 0x7fea10134260 2023-10-03T10:14:08.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.065+0000 7fea17241700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea1009dd80 con 0x7fea10133420 2023-10-03T10:14:08.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.065+0000 7fea0ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea10133e30 0x7fea1009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:08.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.065+0000 7fea0ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea10133e30 0x7fea1009c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57872/0 (socket says 172.21.15.3:57872) 2023-10-03T10:14:08.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.065+0000 7fea0ffff700 1 -- 172.21.15.3:0/243955919 learned_addr learned my addr 172.21.15.3:0/243955919 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:08.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.065+0000 7fea157de700 1 --2- 172.21.15.3:0/243955919 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea10134260 0x7fea1009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:08.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.065+0000 7fea14fdd700 1 --2- 172.21.15.3:0/243955919 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea10133420 0x7fea1009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:08.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.066+0000 7fea157de700 1 -- 172.21.15.3:0/243955919 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea10133420 msgr2=0x7fea1009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:08.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.066+0000 7fea157de700 1 --2- 172.21.15.3:0/243955919 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea10133420 0x7fea1009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:08.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.066+0000 7fea157de700 1 -- 172.21.15.3:0/243955919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea10133e30 msgr2=0x7fea1009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:08.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.066+0000 7fea157de700 1 --2- 172.21.15.3:0/243955919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea10133e30 0x7fea1009c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:08.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.066+0000 7fea157de700 1 -- 172.21.15.3:0/243955919 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea040094a0 con 0x7fea10134260 2023-10-03T10:14:08.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.066+0000 7fea157de700 1 --2- 172.21.15.3:0/243955919 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea10134260 0x7fea1009cf00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fea0800efc0 tx=0x7fea0800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:08.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.066+0000 7fea0d7fa700 1 -- 172.21.15.3:0/243955919 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0800cc90 con 0x7fea10134260 2023-10-03T10:14:08.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.066+0000 7fea0d7fa700 1 -- 172.21.15.3:0/243955919 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0800f040 con 0x7fea10134260 2023-10-03T10:14:08.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.066+0000 7fea17241700 1 -- 172.21.15.3:0/243955919 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea10120330 con 0x7fea10134260 2023-10-03T10:14:08.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.066+0000 7fea0d7fa700 1 -- 172.21.15.3:0/243955919 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea08009e60 con 0x7fea10134260 2023-10-03T10:14:08.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.067+0000 7fea17241700 1 -- 172.21.15.3:0/243955919 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea1011ff10 con 0x7fea10134260 2023-10-03T10:14:08.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.067+0000 7fea0d7fa700 1 -- 172.21.15.3:0/243955919 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fea08014480 con 0x7fea10134260 2023-10-03T10:14:08.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.067+0000 7fea0d7fa700 1 -- 172.21.15.3:0/243955919 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fea080434c0 con 0x7fea10134260 2023-10-03T10:14:08.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.068+0000 7fea17241700 1 -- 172.21.15.3:0/243955919 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea10074ea0 con 0x7fea10134260 2023-10-03T10:14:08.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.073+0000 7fea0d7fa700 1 -- 172.21.15.3:0/243955919 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fea08018020 con 0x7fea10134260 2023-10-03T10:14:08.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.099+0000 7f5e07bbd700 1 Processor -- start 2023-10-03T10:14:08.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.099+0000 7f5e07bbd700 1 -- start start 2023-10-03T10:14:08.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.099+0000 7f5e07bbd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e0009d630 0x7f5e000a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:08.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.099+0000 7f5e07bbd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5e00138d80 0x7f5e0013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:08.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.099+0000 7f5e07bbd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e000a2a70 con 0x7f5e000a0a30 2023-10-03T10:14:08.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.099+0000 7f5e07bbd700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e000a01d0 con 0x7f5e00138d80 2023-10-03T10:14:08.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.099+0000 7f5e07bbd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0009e2b0 con 0x7f5e0009d630 2023-10-03T10:14:08.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.100+0000 7f5e05959700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5e000a0a30 0x7f5e000a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54302/0 (socket says 172.21.15.3:54302) 2023-10-03T10:14:08.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.100+0000 7f5e05959700 1 -- 172.21.15.3:0/4142323482 learned_addr learned my addr 172.21.15.3:0/4142323482 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:08.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.100+0000 7f5e0615a700 1 --2- 172.21.15.3:0/4142323482 >> v2:172.21.15.3:3300/0 conn(0x7f5e00138d80 0x7f5e0013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:08.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.100+0000 7f5e05158700 1 --2- 172.21.15.3:0/4142323482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e0009d630 0x7f5e000a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:08.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.100+0000 7f5e04957700 1 -- 172.21.15.3:0/4142323482 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2693020234 0 0) 0x7f5e000a2a70 con 0x7f5e000a0a30 2023-10-03T10:14:08.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.100+0000 7f5e04957700 1 -- 172.21.15.3:0/4142323482 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dec0035b0 con 0x7f5e000a0a30 2023-10-03T10:14:08.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.100+0000 7f5e0615a700 1 -- 172.21.15.3:0/4142323482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e0009d630 msgr2=0x7f5e000a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:08.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.100+0000 7f5e0615a700 1 --2- 172.21.15.3:0/4142323482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e0009d630 0x7f5e000a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:08.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.100+0000 7f5e0615a700 1 -- 172.21.15.3:0/4142323482 >> v1:172.21.15.3:6789/0 conn(0x7f5e000a0a30 legacy=0x7f5e000a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:08.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.100+0000 7f5e0615a700 1 -- 172.21.15.3:0/4142323482 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e0009dc50 con 0x7f5e00138d80 2023-10-03T10:14:08.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.100+0000 7f5e0615a700 1 --2- 172.21.15.3:0/4142323482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e00138d80 0x7f5e0013b160 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009840 tx=0x7f5df8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4314f17f1bd7eac server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:08.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.100+0000 7f5e04957700 1 -- 172.21.15.3:0/4142323482 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df8011830 con 0x7f5e00138d80 2023-10-03T10:14:08.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.101+0000 7f5e04957700 1 -- 172.21.15.3:0/4142323482 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df8011990 con 0x7f5e00138d80 2023-10-03T10:14:08.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.101+0000 7f5e04957700 1 -- 172.21.15.3:0/4142323482 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df8011d20 con 0x7f5e00138d80 2023-10-03T10:14:08.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.101+0000 7f5e07bbd700 1 -- 172.21.15.3:0/4142323482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e00138d80 msgr2=0x7f5e0013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:08.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.101+0000 7f5e07bbd700 1 --2- 172.21.15.3:0/4142323482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e00138d80 0x7f5e0013b160 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009840 tx=0x7f5df8009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:08.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.101+0000 7f5e07bbd700 1 -- 172.21.15.3:0/4142323482 shutdown_connections 2023-10-03T10:14:08.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.101+0000 7f5e07bbd700 1 --2- 172.21.15.3:0/4142323482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e00138d80 0x7f5e0013b160 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:08.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.101+0000 7f5e07bbd700 1 --2- 172.21.15.3:0/4142323482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e0009d630 0x7f5e000a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:08.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.101+0000 7f5e07bbd700 1 -- 172.21.15.3:0/4142323482 >> 172.21.15.3:0/4142323482 conn(0x7f5e000a3050 msgr2=0x7f5e000a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:08.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.101+0000 7f5e07bbd700 1 -- 172.21.15.3:0/4142323482 shutdown_connections 2023-10-03T10:14:08.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.101+0000 7f5e07bbd700 1 -- 172.21.15.3:0/4142323482 wait complete. 2023-10-03T10:14:08.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.103+0000 7f5e07bbd700 1 Processor -- start 2023-10-03T10:14:08.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.103+0000 7f5e07bbd700 1 -- start start 2023-10-03T10:14:08.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.104+0000 7f5e07bbd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e0009d630 0x7f5e00133740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:08.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.104+0000 7f5e07bbd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e000a0a30 0x7f5e00133c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:08.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.104+0000 7f5e07bbd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e00138d80 0x7f5e001341c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:08.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.104+0000 7f5e07bbd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e00137260 con 0x7f5e0009d630 2023-10-03T10:14:08.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.104+0000 7f5e07bbd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e001365a0 con 0x7f5e000a0a30 2023-10-03T10:14:08.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.104+0000 7f5e07bbd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e000a4930 con 0x7f5e00138d80 2023-10-03T10:14:08.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.104+0000 7f5e05959700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e0009d630 0x7f5e00133740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:08.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.104+0000 7f5e05959700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e0009d630 0x7f5e00133740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57896/0 (socket says 172.21.15.3:57896) 2023-10-03T10:14:08.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.104+0000 7f5e05959700 1 -- 172.21.15.3:0/4264571728 learned_addr learned my addr 172.21.15.3:0/4264571728 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:08.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.104+0000 7f5e05158700 1 --2- 172.21.15.3:0/4264571728 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e000a0a30 0x7f5e00133c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:08.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.104+0000 7f5e0615a700 1 --2- 172.21.15.3:0/4264571728 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e00138d80 0x7f5e001341c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:08.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.104+0000 7f5e05158700 1 -- 172.21.15.3:0/4264571728 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e00138d80 msgr2=0x7f5e001341c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:08.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.104+0000 7f5e05158700 1 --2- 172.21.15.3:0/4264571728 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e00138d80 0x7f5e001341c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:08.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.104+0000 7f5e05158700 1 -- 172.21.15.3:0/4264571728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e0009d630 msgr2=0x7f5e00133740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:08.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.104+0000 7f5e05158700 1 --2- 172.21.15.3:0/4264571728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e0009d630 0x7f5e00133740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:08.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.104+0000 7f5e05158700 1 -- 172.21.15.3:0/4264571728 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df80094a0 con 0x7f5e000a0a30 2023-10-03T10:14:08.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.104+0000 7f5e05959700 1 --2- 172.21.15.3:0/4264571728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e0009d630 0x7f5e00133740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:08.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.104+0000 7f5e0615a700 1 --2- 172.21.15.3:0/4264571728 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e00138d80 0x7f5e001341c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:08.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.104+0000 7f5e05158700 1 --2- 172.21.15.3:0/4264571728 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e000a0a30 0x7f5e00133c80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5df000e900 tx=0x7f5df000ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:08.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.105+0000 7f5df67fc700 1 -- 172.21.15.3:0/4264571728 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df0013680 con 0x7f5e000a0a30 2023-10-03T10:14:08.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.105+0000 7f5df67fc700 1 -- 172.21.15.3:0/4264571728 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df00137e0 con 0x7f5e000a0a30 2023-10-03T10:14:08.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.105+0000 7f5e07bbd700 1 -- 172.21.15.3:0/4264571728 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e00135bb0 con 0x7f5e000a0a30 2023-10-03T10:14:08.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.105+0000 7f5df67fc700 1 -- 172.21.15.3:0/4264571728 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df0013b50 con 0x7f5e000a0a30 2023-10-03T10:14:08.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.105+0000 7f5e07bbd700 1 -- 172.21.15.3:0/4264571728 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e00137760 con 0x7f5e000a0a30 2023-10-03T10:14:08.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.106+0000 7f5df67fc700 1 -- 172.21.15.3:0/4264571728 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5df0013cb0 con 0x7f5e000a0a30 2023-10-03T10:14:08.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.106+0000 7f5df67fc700 1 -- 172.21.15.3:0/4264571728 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5df003fa80 con 0x7f5e000a0a30 2023-10-03T10:14:08.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.106+0000 7f5e07bbd700 1 -- 172.21.15.3:0/4264571728 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e000747f0 con 0x7f5e000a0a30 2023-10-03T10:14:08.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.115+0000 7f5df67fc700 1 -- 172.21.15.3:0/4264571728 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5df001adc0 con 0x7f5e000a0a30 2023-10-03T10:14:08.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.212+0000 7f2e7b6cc700 1 -- 172.21.15.3:0/2537264521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2e74074ea0 con 0x7f2e7409d6f0 2023-10-03T10:14:08.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.212+0000 7f2e69ffb700 1 -- 172.21.15.3:0/2537264521 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2e70021020 con 0x7f2e7409d6f0 2023-10-03T10:14:08.213 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:08.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.214+0000 7f2e7b6cc700 1 -- 172.21.15.3:0/2537264521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e7409d6f0 msgr2=0x7f2e741a5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:08.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.214+0000 7f2e7b6cc700 1 --2- 172.21.15.3:0/2537264521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e7409d6f0 0x7f2e741a5fb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2e7000e9e0 tx=0x7f2e7000ecf0 comp rx=0 tx=0).stop 2023-10-03T10:14:08.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.214+0000 7f2e7b6cc700 1 -- 172.21.15.3:0/2537264521 shutdown_connections 2023-10-03T10:14:08.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.214+0000 7f2e7b6cc700 1 --2- 172.21.15.3:0/2537264521 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e7409e410 0x7f2e741a6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:08.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.214+0000 7f2e7b6cc700 1 --2- 172.21.15.3:0/2537264521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e7409d6f0 0x7f2e741a5fb0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:08.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.214+0000 7f2e7b6cc700 1 --2- 172.21.15.3:0/2537264521 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e7409e000 0x7f2e741a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:08.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.214+0000 7f2e7b6cc700 1 -- 172.21.15.3:0/2537264521 >> 172.21.15.3:0/2537264521 conn(0x7f2e74066f50 msgr2=0x7f2e741baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:08.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.215+0000 7f2e7b6cc700 1 -- 172.21.15.3:0/2537264521 shutdown_connections 2023-10-03T10:14:08.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.215+0000 7f2e7b6cc700 1 -- 172.21.15.3:0/2537264521 wait complete. 2023-10-03T10:14:08.229 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:08.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.286+0000 7fea17241700 1 -- 172.21.15.3:0/243955919 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fea1011f860 con 0x7fea10134260 2023-10-03T10:14:08.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.287+0000 7fea0d7fa700 1 -- 172.21.15.3:0/243955919 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fea080133e0 con 0x7fea10134260 2023-10-03T10:14:08.287 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:08.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.289+0000 7fea17241700 1 -- 172.21.15.3:0/243955919 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea10134260 msgr2=0x7fea1009cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:08.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.289+0000 7fea17241700 1 --2- 172.21.15.3:0/243955919 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea10134260 0x7fea1009cf00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fea0800efc0 tx=0x7fea0800c3a0 comp rx=0 tx=0).stop 2023-10-03T10:14:08.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.289+0000 7fea17241700 1 -- 172.21.15.3:0/243955919 shutdown_connections 2023-10-03T10:14:08.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.289+0000 7fea17241700 1 --2- 172.21.15.3:0/243955919 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea10134260 0x7fea1009cf00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:08.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.289+0000 7fea17241700 1 --2- 172.21.15.3:0/243955919 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea10133420 0x7fea1009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:08.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.289+0000 7fea17241700 1 --2- 172.21.15.3:0/243955919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea10133e30 0x7fea1009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:08.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.289+0000 7fea17241700 1 -- 172.21.15.3:0/243955919 >> 172.21.15.3:0/243955919 conn(0x7fea100a4210 msgr2=0x7fea100a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:08.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.289+0000 7fea17241700 1 -- 172.21.15.3:0/243955919 shutdown_connections 2023-10-03T10:14:08.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.289+0000 7fea17241700 1 -- 172.21.15.3:0/243955919 wait complete. 2023-10-03T10:14:08.304 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:08.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.344+0000 7f5e07bbd700 1 -- 172.21.15.3:0/4264571728 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5e00137a40 con 0x7f5e000a0a30 2023-10-03T10:14:08.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.345+0000 7f5df67fc700 1 -- 172.21.15.3:0/4264571728 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5df001a360 con 0x7f5e000a0a30 2023-10-03T10:14:08.345 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:08.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.347+0000 7f5e07bbd700 1 -- 172.21.15.3:0/4264571728 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e000a0a30 msgr2=0x7f5e00133c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:08.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.347+0000 7f5e07bbd700 1 --2- 172.21.15.3:0/4264571728 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e000a0a30 0x7f5e00133c80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5df000e900 tx=0x7f5df000ec10 comp rx=0 tx=0).stop 2023-10-03T10:14:08.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.347+0000 7f5e07bbd700 1 -- 172.21.15.3:0/4264571728 shutdown_connections 2023-10-03T10:14:08.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.347+0000 7f5e07bbd700 1 --2- 172.21.15.3:0/4264571728 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e00138d80 0x7f5e001341c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:08.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.347+0000 7f5e07bbd700 1 --2- 172.21.15.3:0/4264571728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e0009d630 0x7f5e00133740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:08.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.347+0000 7f5e07bbd700 1 --2- 172.21.15.3:0/4264571728 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e000a0a30 0x7f5e00133c80 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:08.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.347+0000 7f5e07bbd700 1 -- 172.21.15.3:0/4264571728 >> 172.21.15.3:0/4264571728 conn(0x7f5e000a3050 msgr2=0x7f5e0009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:08.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.347+0000 7f5e07bbd700 1 -- 172.21.15.3:0/4264571728 shutdown_connections 2023-10-03T10:14:08.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.347+0000 7f5e07bbd700 1 -- 172.21.15.3:0/4264571728 wait complete. 2023-10-03T10:14:08.361 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:08.729 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:08.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.875+0000 7f5015c99700 1 Processor -- start 2023-10-03T10:14:08.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.875+0000 7f5015c99700 1 -- start start 2023-10-03T10:14:08.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.876+0000 7f5015c99700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f501009d6f0 0x7f501009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:08.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.876+0000 7f5015c99700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f501009e000 0x7f50101339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:08.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.876+0000 7f5015c99700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f501009e410 0x7f5010133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:08.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.876+0000 7f5015c99700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f501006b460 con 0x7f501009d6f0 2023-10-03T10:14:08.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.876+0000 7f5015c99700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f501009b120 con 0x7f501009e000 2023-10-03T10:14:08.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.876+0000 7f5015c99700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f501009a7e0 con 0x7f501009e410 2023-10-03T10:14:08.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.876+0000 7f500f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f501009d6f0 0x7f501009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:08.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.876+0000 7f500ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f501009e410 0x7f5010133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:08.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.876+0000 7f500effd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f501009e000 0x7f50101339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:08.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.876+0000 7f500f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f501009d6f0 0x7f501009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57900/0 (socket says 172.21.15.3:57900) 2023-10-03T10:14:08.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.876+0000 7f500f7fe700 1 -- 172.21.15.3:0/1672300326 learned_addr learned my addr 172.21.15.3:0/1672300326 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:08.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.876+0000 7f500f7fe700 1 -- 172.21.15.3:0/1672300326 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f501009e410 msgr2=0x7f5010133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:08.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.876+0000 7f500f7fe700 1 --2- 172.21.15.3:0/1672300326 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f501009e410 0x7f5010133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:08.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.876+0000 7f500f7fe700 1 -- 172.21.15.3:0/1672300326 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f501009e000 msgr2=0x7f50101339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:08.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.876+0000 7f500f7fe700 1 --2- 172.21.15.3:0/1672300326 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f501009e000 0x7f50101339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:08.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.876+0000 7f500f7fe700 1 -- 172.21.15.3:0/1672300326 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f501009a500 con 0x7f501009d6f0 2023-10-03T10:14:08.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.876+0000 7f500effd700 1 --2- 172.21.15.3:0/1672300326 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f501009e000 0x7f50101339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:08.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.876+0000 7f500ffff700 1 --2- 172.21.15.3:0/1672300326 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f501009e410 0x7f5010133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:08.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.876+0000 7f500f7fe700 1 --2- 172.21.15.3:0/1672300326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f501009d6f0 0x7f501009dac0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f50000098f0 tx=0x7f5000009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8406590569655bd server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:08.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.876+0000 7f500e7fc700 1 -- 172.21.15.3:0/1672300326 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5000011830 con 0x7f501009d6f0 2023-10-03T10:14:08.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.877+0000 7f500e7fc700 1 -- 172.21.15.3:0/1672300326 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5000011990 con 0x7f501009d6f0 2023-10-03T10:14:08.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.877+0000 7f500e7fc700 1 -- 172.21.15.3:0/1672300326 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5000011d20 con 0x7f501009d6f0 2023-10-03T10:14:08.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.877+0000 7f5015c99700 1 -- 172.21.15.3:0/1672300326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f501009d6f0 msgr2=0x7f501009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:08.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.877+0000 7f5015c99700 1 --2- 172.21.15.3:0/1672300326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f501009d6f0 0x7f501009dac0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f50000098f0 tx=0x7f5000009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:08.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.877+0000 7f5015c99700 1 -- 172.21.15.3:0/1672300326 shutdown_connections 2023-10-03T10:14:08.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.877+0000 7f5015c99700 1 --2- 172.21.15.3:0/1672300326 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f501009e410 0x7f5010133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:08.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.877+0000 7f5015c99700 1 --2- 172.21.15.3:0/1672300326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f501009d6f0 0x7f501009dac0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:08.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.877+0000 7f5015c99700 1 --2- 172.21.15.3:0/1672300326 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f501009e000 0x7f50101339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:08.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.877+0000 7f5015c99700 1 -- 172.21.15.3:0/1672300326 >> 172.21.15.3:0/1672300326 conn(0x7f5010066f50 msgr2=0x7f501009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:08.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.877+0000 7f5015c99700 1 -- 172.21.15.3:0/1672300326 shutdown_connections 2023-10-03T10:14:08.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.877+0000 7f5015c99700 1 -- 172.21.15.3:0/1672300326 wait complete. 2023-10-03T10:14:08.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.878+0000 7f5015c99700 1 Processor -- start 2023-10-03T10:14:08.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.878+0000 7f5015c99700 1 -- start start 2023-10-03T10:14:08.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.879+0000 7f5015c99700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f501009d6f0 0x7f50101c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:08.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.879+0000 7f5015c99700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f501009e000 0x7f50101c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:08.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.879+0000 7f5015c99700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f501009e410 0x7f50101c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:08.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.879+0000 7f5015c99700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50101a7190 con 0x7f501009d6f0 2023-10-03T10:14:08.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.879+0000 7f5015c99700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50101b9320 con 0x7f501009e410 2023-10-03T10:14:08.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.879+0000 7f5015c99700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f50101a6520 con 0x7f501009e000 2023-10-03T10:14:08.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.879+0000 7f500f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f501009d6f0 0x7f50101c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:08.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.879+0000 7f500ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f501009e410 0x7f50101c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:08.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.879+0000 7f500f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f501009d6f0 0x7f50101c80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57916/0 (socket says 172.21.15.3:57916) 2023-10-03T10:14:08.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.879+0000 7f500f7fe700 1 -- 172.21.15.3:0/2315064433 learned_addr learned my addr 172.21.15.3:0/2315064433 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:08.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.879+0000 7f500effd700 1 --2- 172.21.15.3:0/2315064433 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f501009e000 0x7f50101c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:08.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.879+0000 7f500f7fe700 1 -- 172.21.15.3:0/2315064433 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f501009e000 msgr2=0x7f50101c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:08.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.879+0000 7f500f7fe700 1 --2- 172.21.15.3:0/2315064433 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f501009e000 0x7f50101c85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:08.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.879+0000 7f500f7fe700 1 -- 172.21.15.3:0/2315064433 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f501009e410 msgr2=0x7f50101c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:08.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.879+0000 7f500f7fe700 1 --2- 172.21.15.3:0/2315064433 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f501009e410 0x7f50101c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:08.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.879+0000 7f500f7fe700 1 -- 172.21.15.3:0/2315064433 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff80094a0 con 0x7f501009d6f0 2023-10-03T10:14:08.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.879+0000 7f500ffff700 1 --2- 172.21.15.3:0/2315064433 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f501009e410 0x7f50101c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:08.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.880+0000 7f500f7fe700 1 --2- 172.21.15.3:0/2315064433 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f501009d6f0 0x7f50101c80b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5000011db0 tx=0x7f5000004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:08.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.880+0000 7f5007fff700 1 -- 172.21.15.3:0/2315064433 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5000004240 con 0x7f501009d6f0 2023-10-03T10:14:08.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.880+0000 7f5007fff700 1 -- 172.21.15.3:0/2315064433 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f500001f070 con 0x7f501009d6f0 2023-10-03T10:14:08.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.880+0000 7f5015c99700 1 -- 172.21.15.3:0/2315064433 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5008009a70 con 0x7f501009d6f0 2023-10-03T10:14:08.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.880+0000 7f5007fff700 1 -- 172.21.15.3:0/2315064433 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50000045a0 con 0x7f501009d6f0 2023-10-03T10:14:08.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.880+0000 7f5015c99700 1 -- 172.21.15.3:0/2315064433 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5000009550 con 0x7f501009d6f0 2023-10-03T10:14:08.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.880+0000 7f5007fff700 1 -- 172.21.15.3:0/2315064433 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5000021030 con 0x7f501009d6f0 2023-10-03T10:14:08.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.881+0000 7f5007fff700 1 -- 172.21.15.3:0/2315064433 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5000040720 con 0x7f501009d6f0 2023-10-03T10:14:08.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.881+0000 7f5015c99700 1 -- 172.21.15.3:0/2315064433 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5010074ea0 con 0x7f501009d6f0 2023-10-03T10:14:08.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:08.886+0000 7f5007fff700 1 -- 172.21.15.3:0/2315064433 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f500001a450 con 0x7f501009d6f0 2023-10-03T10:14:09.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.093+0000 7f5015c99700 1 -- 172.21.15.3:0/2315064433 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f501006c360 con 0x7f501009d6f0 2023-10-03T10:14:09.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.094+0000 7f5007fff700 1 -- 172.21.15.3:0/2315064433 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f500001b430 con 0x7f501009d6f0 2023-10-03T10:14:09.094 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:09.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.096+0000 7f5015c99700 1 -- 172.21.15.3:0/2315064433 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f501009d6f0 msgr2=0x7f50101c80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:09.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.096+0000 7f5015c99700 1 --2- 172.21.15.3:0/2315064433 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f501009d6f0 0x7f50101c80b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5000011db0 tx=0x7f5000004750 comp rx=0 tx=0).stop 2023-10-03T10:14:09.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.096+0000 7f5015c99700 1 -- 172.21.15.3:0/2315064433 shutdown_connections 2023-10-03T10:14:09.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.096+0000 7f5015c99700 1 --2- 172.21.15.3:0/2315064433 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f501009e410 0x7f50101c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.096+0000 7f5015c99700 1 --2- 172.21.15.3:0/2315064433 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f501009d6f0 0x7f50101c80b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.096+0000 7f5015c99700 1 --2- 172.21.15.3:0/2315064433 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f501009e000 0x7f50101c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.096+0000 7f5015c99700 1 -- 172.21.15.3:0/2315064433 >> 172.21.15.3:0/2315064433 conn(0x7f5010066f50 msgr2=0x7f50101bbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:09.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.096+0000 7f5015c99700 1 -- 172.21.15.3:0/2315064433 shutdown_connections 2023-10-03T10:14:09.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.096+0000 7f5015c99700 1 -- 172.21.15.3:0/2315064433 wait complete. 2023-10-03T10:14:09.110 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:09.230 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:09.305 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:09.361 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:09.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.376+0000 7f72494f4700 1 Processor -- start 2023-10-03T10:14:09.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.376+0000 7f72494f4700 1 -- start start 2023-10-03T10:14:09.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.377+0000 7f72494f4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f724409d6f0 0x7f724409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:09.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.377+0000 7f72494f4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f724409e410 0x7f7244134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:09.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.377+0000 7f72494f4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f724409b120 con 0x7f724409e000 2023-10-03T10:14:09.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.377+0000 7f72494f4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f724406b460 con 0x7f724409d6f0 2023-10-03T10:14:09.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.377+0000 7f72494f4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f724409a7e0 con 0x7f724409e410 2023-10-03T10:14:09.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.377+0000 7f7242ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f724409d6f0 0x7f724409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:09.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.377+0000 7f7242ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f724409d6f0 0x7f724409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40020/0 (socket says 172.21.15.3:40020) 2023-10-03T10:14:09.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.377+0000 7f72427fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f724409e000 0x7f72441339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54306/0 (socket says 172.21.15.3:54306) 2023-10-03T10:14:09.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.377+0000 7f7242ffd700 1 -- 172.21.15.3:0/3749723023 learned_addr learned my addr 172.21.15.3:0/3749723023 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:09.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.377+0000 7f72437fe700 1 --2- 172.21.15.3:0/3749723023 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f724409e410 0x7f7244134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:09.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.378+0000 7f7242ffd700 1 -- 172.21.15.3:0/3749723023 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f724409e410 msgr2=0x7f7244134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:09.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.378+0000 7f7242ffd700 1 --2- 172.21.15.3:0/3749723023 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f724409e410 0x7f7244134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.378+0000 7f7242ffd700 1 -- 172.21.15.3:0/3749723023 >> v1:172.21.15.3:6789/0 conn(0x7f724409e000 legacy=0x7f72441339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:09.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.378+0000 7f7242ffd700 1 -- 172.21.15.3:0/3749723023 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f724409a500 con 0x7f724409d6f0 2023-10-03T10:14:09.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.378+0000 7f72437fe700 1 --2- 172.21.15.3:0/3749723023 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f724409e410 0x7f7244134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:14:09.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.378+0000 7f7242ffd700 1 --2- 172.21.15.3:0/3749723023 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f724409d6f0 0x7f724409dac0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7234009e80 tx=0x7f723400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d6233d1bee9ddaf server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:09.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.378+0000 7f7241ffb700 1 -- 172.21.15.3:0/3749723023 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723400e070 con 0x7f724409d6f0 2023-10-03T10:14:09.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.378+0000 7f7241ffb700 1 -- 172.21.15.3:0/3749723023 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723400b9c0 con 0x7f724409d6f0 2023-10-03T10:14:09.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.378+0000 7f7241ffb700 1 -- 172.21.15.3:0/3749723023 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723400bd70 con 0x7f724409d6f0 2023-10-03T10:14:09.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.379+0000 7f72494f4700 1 -- 172.21.15.3:0/3749723023 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f724409d6f0 msgr2=0x7f724409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:09.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.379+0000 7f72494f4700 1 --2- 172.21.15.3:0/3749723023 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f724409d6f0 0x7f724409dac0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7234009e80 tx=0x7f723400b330 comp rx=0 tx=0).stop 2023-10-03T10:14:09.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.379+0000 7f72494f4700 1 -- 172.21.15.3:0/3749723023 shutdown_connections 2023-10-03T10:14:09.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.379+0000 7f72494f4700 1 --2- 172.21.15.3:0/3749723023 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f724409d6f0 0x7f724409dac0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.379+0000 7f72494f4700 1 --2- 172.21.15.3:0/3749723023 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f724409e410 0x7f7244134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.379+0000 7f72494f4700 1 -- 172.21.15.3:0/3749723023 >> 172.21.15.3:0/3749723023 conn(0x7f7244066f50 msgr2=0x7f724409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:09.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.379+0000 7f72494f4700 1 -- 172.21.15.3:0/3749723023 shutdown_connections 2023-10-03T10:14:09.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.379+0000 7f72494f4700 1 -- 172.21.15.3:0/3749723023 wait complete. 2023-10-03T10:14:09.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.380+0000 7f72494f4700 1 Processor -- start 2023-10-03T10:14:09.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.381+0000 7f72494f4700 1 -- start start 2023-10-03T10:14:09.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.381+0000 7f72494f4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f724409d6f0 0x7f7244122460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:09.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.381+0000 7f72494f4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f724409e000 0x7f72441229a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:09.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.381+0000 7f72494f4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f724409e410 0x7f7244122ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:09.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.381+0000 7f72494f4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724409c780 con 0x7f724409d6f0 2023-10-03T10:14:09.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.381+0000 7f72494f4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72440a1e20 con 0x7f724409e410 2023-10-03T10:14:09.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.381+0000 7f72494f4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7244124890 con 0x7f724409e000 2023-10-03T10:14:09.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.381+0000 7f7242ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f724409d6f0 0x7f7244122460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:09.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.381+0000 7f7242ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f724409d6f0 0x7f7244122460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57932/0 (socket says 172.21.15.3:57932) 2023-10-03T10:14:09.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.381+0000 7f7242ffd700 1 -- 172.21.15.3:0/1389017622 learned_addr learned my addr 172.21.15.3:0/1389017622 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:09.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.381+0000 7f72427fc700 1 --2- 172.21.15.3:0/1389017622 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f724409e000 0x7f72441229a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:09.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.381+0000 7f72437fe700 1 --2- 172.21.15.3:0/1389017622 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f724409e410 0x7f7244122ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:09.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.382+0000 7f7242ffd700 1 -- 172.21.15.3:0/1389017622 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f724409e000 msgr2=0x7f72441229a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:09.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.382+0000 7f7242ffd700 1 --2- 172.21.15.3:0/1389017622 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f724409e000 0x7f72441229a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.382+0000 7f7242ffd700 1 -- 172.21.15.3:0/1389017622 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f724409e410 msgr2=0x7f7244122ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:09.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.382+0000 7f7242ffd700 1 --2- 172.21.15.3:0/1389017622 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f724409e410 0x7f7244122ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.382+0000 7f7242ffd700 1 -- 172.21.15.3:0/1389017622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7234009b30 con 0x7f724409d6f0 2023-10-03T10:14:09.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.382+0000 7f7242ffd700 1 --2- 172.21.15.3:0/1389017622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f724409d6f0 0x7f7244122460 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7234003f50 tx=0x7f7234004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:09.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.382+0000 7f723b7fe700 1 -- 172.21.15.3:0/1389017622 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72340092e0 con 0x7f724409d6f0 2023-10-03T10:14:09.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.382+0000 7f723b7fe700 1 -- 172.21.15.3:0/1389017622 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7234004250 con 0x7f724409d6f0 2023-10-03T10:14:09.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.382+0000 7f72494f4700 1 -- 172.21.15.3:0/1389017622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72441252c0 con 0x7f724409d6f0 2023-10-03T10:14:09.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.382+0000 7f723b7fe700 1 -- 172.21.15.3:0/1389017622 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72340045e0 con 0x7f724409d6f0 2023-10-03T10:14:09.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.382+0000 7f72494f4700 1 -- 172.21.15.3:0/1389017622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7244125710 con 0x7f724409d6f0 2023-10-03T10:14:09.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.383+0000 7f723b7fe700 1 -- 172.21.15.3:0/1389017622 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7234004740 con 0x7f724409d6f0 2023-10-03T10:14:09.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.383+0000 7f723b7fe700 1 -- 172.21.15.3:0/1389017622 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7234040cd0 con 0x7f724409d6f0 2023-10-03T10:14:09.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.383+0000 7f72494f4700 1 -- 172.21.15.3:0/1389017622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7244074ea0 con 0x7f724409d6f0 2023-10-03T10:14:09.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.388+0000 7f723b7fe700 1 -- 172.21.15.3:0/1389017622 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7234019760 con 0x7f724409d6f0 2023-10-03T10:14:09.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.452+0000 7f633f409700 1 Processor -- start 2023-10-03T10:14:09.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.452+0000 7f633f409700 1 -- start start 2023-10-03T10:14:09.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.453+0000 7f633f409700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f63380a0a30 0x7f63380a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:09.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.453+0000 7f633f409700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f633809d630 0x7f63380a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:09.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.453+0000 7f633f409700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63380a2a70 con 0x7f6338133930 2023-10-03T10:14:09.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.453+0000 7f633f409700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63380a01d0 con 0x7f633809d630 2023-10-03T10:14:09.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.453+0000 7f633f409700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633809e2b0 con 0x7f63380a0a30 2023-10-03T10:14:09.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.453+0000 7f633c9a4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f633809d630 0x7f63380a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:09.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.453+0000 7f633d1a5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f63380a0a30 0x7f63380a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:09.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.453+0000 7f633d9a6700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6338133930 0x7f6338133db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54318/0 (socket says 172.21.15.3:54318) 2023-10-03T10:14:09.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.453+0000 7f633d9a6700 1 -- 172.21.15.3:0/2451404882 learned_addr learned my addr 172.21.15.3:0/2451404882 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:09.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.453+0000 7f633d1a5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f63380a0a30 0x7f63380a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40034/0 (socket says 172.21.15.3:40034) 2023-10-03T10:14:09.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.453+0000 7f633c9a4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f633809d630 0x7f63380a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57934/0 (socket says 172.21.15.3:57934) 2023-10-03T10:14:09.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.453+0000 7f633c9a4700 1 -- 172.21.15.3:0/2451404882 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f63380a0a30 msgr2=0x7f63380a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:09.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.453+0000 7f633c9a4700 1 --2- 172.21.15.3:0/2451404882 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f63380a0a30 0x7f63380a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.453+0000 7f633c9a4700 1 -- 172.21.15.3:0/2451404882 >> v1:172.21.15.3:6789/0 conn(0x7f6338133930 legacy=0x7f6338133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:09.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.453+0000 7f633c9a4700 1 -- 172.21.15.3:0/2451404882 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633809dc50 con 0x7f633809d630 2023-10-03T10:14:09.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.453+0000 7f633d1a5700 1 --2- 172.21.15.3:0/2451404882 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f63380a0a30 0x7f63380a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:09.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.454+0000 7f633c9a4700 1 --2- 172.21.15.3:0/2451404882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f633809d630 0x7f63380a1340 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6328009840 tx=0x7f6328009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a940d8b59ba63fc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:09.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.454+0000 7f632ffff700 1 -- 172.21.15.3:0/2451404882 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328011830 con 0x7f633809d630 2023-10-03T10:14:09.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.454+0000 7f632ffff700 1 -- 172.21.15.3:0/2451404882 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6328011990 con 0x7f633809d630 2023-10-03T10:14:09.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.454+0000 7f632ffff700 1 -- 172.21.15.3:0/2451404882 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328011d20 con 0x7f633809d630 2023-10-03T10:14:09.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.454+0000 7f633f409700 1 -- 172.21.15.3:0/2451404882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f633809d630 msgr2=0x7f63380a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:09.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.454+0000 7f633f409700 1 --2- 172.21.15.3:0/2451404882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f633809d630 0x7f63380a1340 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6328009840 tx=0x7f6328009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:09.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.454+0000 7f633f409700 1 -- 172.21.15.3:0/2451404882 shutdown_connections 2023-10-03T10:14:09.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.454+0000 7f633f409700 1 --2- 172.21.15.3:0/2451404882 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f63380a0a30 0x7f63380a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.455+0000 7f633f409700 1 --2- 172.21.15.3:0/2451404882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f633809d630 0x7f63380a1340 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.455+0000 7f633f409700 1 -- 172.21.15.3:0/2451404882 >> 172.21.15.3:0/2451404882 conn(0x7f63380a3050 msgr2=0x7f63380a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:09.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.455+0000 7f633f409700 1 -- 172.21.15.3:0/2451404882 shutdown_connections 2023-10-03T10:14:09.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.455+0000 7f633f409700 1 -- 172.21.15.3:0/2451404882 wait complete. 2023-10-03T10:14:09.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.456+0000 7f633f409700 1 Processor -- start 2023-10-03T10:14:09.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.456+0000 7f633f409700 1 -- start start 2023-10-03T10:14:09.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.456+0000 7f633f409700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f633809d630 0x7f63381c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:09.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.456+0000 7f633f409700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63380a0a30 0x7f63381c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:09.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.456+0000 7f633f409700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6338133930 0x7f63381c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:09.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.456+0000 7f633f409700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6338120a70 con 0x7f63380a0a30 2023-10-03T10:14:09.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.456+0000 7f633f409700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6338120880 con 0x7f633809d630 2023-10-03T10:14:09.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.456+0000 7f633f409700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6338121920 con 0x7f6338133930 2023-10-03T10:14:09.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.456+0000 7f633d1a5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f633809d630 0x7f63381c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:09.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.456+0000 7f633d1a5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f633809d630 0x7f63381c8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40036/0 (socket says 172.21.15.3:40036) 2023-10-03T10:14:09.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.456+0000 7f633d1a5700 1 -- 172.21.15.3:0/392724875 learned_addr learned my addr 172.21.15.3:0/392724875 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:09.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.456+0000 7f633c9a4700 1 --2- 172.21.15.3:0/392724875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63380a0a30 0x7f63381c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:09.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.457+0000 7f633d9a6700 1 --2- 172.21.15.3:0/392724875 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6338133930 0x7f63381c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:09.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.457+0000 7f633d1a5700 1 -- 172.21.15.3:0/392724875 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6338133930 msgr2=0x7f63381c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:09.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.457+0000 7f633d1a5700 1 --2- 172.21.15.3:0/392724875 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6338133930 0x7f63381c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.457+0000 7f633d1a5700 1 -- 172.21.15.3:0/392724875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63380a0a30 msgr2=0x7f63381c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:09.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.457+0000 7f633d1a5700 1 --2- 172.21.15.3:0/392724875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63380a0a30 0x7f63381c8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.457+0000 7f633d1a5700 1 -- 172.21.15.3:0/392724875 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63340094a0 con 0x7f633809d630 2023-10-03T10:14:09.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.457+0000 7f633c9a4700 1 --2- 172.21.15.3:0/392724875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63380a0a30 0x7f63381c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:09.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.457+0000 7f633d9a6700 1 --2- 172.21.15.3:0/392724875 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6338133930 0x7f63381c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:14:09.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.457+0000 7f633d1a5700 1 --2- 172.21.15.3:0/392724875 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f633809d630 0x7f63381c8310 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f633400ebe0 tx=0x7f633400eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:09.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.457+0000 7f632dffb700 1 -- 172.21.15.3:0/392724875 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6334009800 con 0x7f633809d630 2023-10-03T10:14:09.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.457+0000 7f632dffb700 1 -- 172.21.15.3:0/392724875 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633400c910 con 0x7f633809d630 2023-10-03T10:14:09.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.457+0000 7f633f409700 1 -- 172.21.15.3:0/392724875 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63280094a0 con 0x7f633809d630 2023-10-03T10:14:09.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.457+0000 7f632dffb700 1 -- 172.21.15.3:0/392724875 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633400cc80 con 0x7f633809d630 2023-10-03T10:14:09.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.457+0000 7f633f409700 1 -- 172.21.15.3:0/392724875 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63381224b0 con 0x7f633809d630 2023-10-03T10:14:09.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.458+0000 7f633f409700 1 -- 172.21.15.3:0/392724875 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6338121e90 con 0x7f633809d630 2023-10-03T10:14:09.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.459+0000 7f632dffb700 1 -- 172.21.15.3:0/392724875 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6334013470 con 0x7f633809d630 2023-10-03T10:14:09.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.459+0000 7f632dffb700 1 -- 172.21.15.3:0/392724875 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6334048390 con 0x7f633809d630 2023-10-03T10:14:09.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.464+0000 7f632dffb700 1 -- 172.21.15.3:0/392724875 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f63340102b0 con 0x7f633809d630 2023-10-03T10:14:09.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.507+0000 7f28d07f1700 1 Processor -- start 2023-10-03T10:14:09.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.507+0000 7f28d07f1700 1 -- start start 2023-10-03T10:14:09.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.508+0000 7f28d07f1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f28c809f410 0x7f28c8136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:09.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.508+0000 7f28d07f1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f28c8136fe0 0x7f28c81393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:09.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.508+0000 7f28d07f1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c8070940 con 0x7f28c809e930 2023-10-03T10:14:09.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.508+0000 7f28d07f1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c806fd60 con 0x7f28c809f410 2023-10-03T10:14:09.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.508+0000 7f28d07f1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c806fa80 con 0x7f28c8136fe0 2023-10-03T10:14:09.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.509+0000 7f28cdd8c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f28c809f410 0x7f28c8136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:09.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.509+0000 7f28ced8e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f28c8136fe0 0x7f28c81393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:09.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.509+0000 7f28cdd8c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f28c809f410 0x7f28c8136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57950/0 (socket says 172.21.15.3:57950) 2023-10-03T10:14:09.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.509+0000 7f28cdd8c700 1 -- 172.21.15.3:0/4151733691 learned_addr learned my addr 172.21.15.3:0/4151733691 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:09.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.509+0000 7f28cdd8c700 1 -- 172.21.15.3:0/4151733691 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f28c8136fe0 msgr2=0x7f28c81393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:09.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.509+0000 7f28cdd8c700 1 --2- 172.21.15.3:0/4151733691 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f28c8136fe0 0x7f28c81393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.509+0000 7f28cdd8c700 1 -- 172.21.15.3:0/4151733691 >> v1:172.21.15.3:6789/0 conn(0x7f28c809e930 legacy=0x7f28c809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:09.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.509+0000 7f28cdd8c700 1 -- 172.21.15.3:0/4151733691 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c809fb60 con 0x7f28c809f410 2023-10-03T10:14:09.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.509+0000 7f28ced8e700 1 --2- 172.21.15.3:0/4151733691 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f28c8136fe0 0x7f28c81393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:09.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.509+0000 7f28cd58b700 1 -- 172.21.15.3:0/4151733691 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1493949905 0 0) 0x7f28c8070940 con 0x7f28c809e930 2023-10-03T10:14:09.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.509+0000 7f28cdd8c700 1 --2- 172.21.15.3:0/4151733691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28c809f410 0x7f28c8136aa0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f28b80098f0 tx=0x7f28b8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fcc0fe7bd21ca56 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:09.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.509+0000 7f28cd58b700 1 -- 172.21.15.3:0/4151733691 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b8011830 con 0x7f28c809f410 2023-10-03T10:14:09.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.510+0000 7f28cd58b700 1 -- 172.21.15.3:0/4151733691 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b8011990 con 0x7f28c809f410 2023-10-03T10:14:09.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.510+0000 7f28cd58b700 1 -- 172.21.15.3:0/4151733691 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b8011d20 con 0x7f28c809f410 2023-10-03T10:14:09.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.510+0000 7f28d07f1700 1 -- 172.21.15.3:0/4151733691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28c809f410 msgr2=0x7f28c8136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:09.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.510+0000 7f28d07f1700 1 --2- 172.21.15.3:0/4151733691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28c809f410 0x7f28c8136aa0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f28b80098f0 tx=0x7f28b8009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:09.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.510+0000 7f28d07f1700 1 -- 172.21.15.3:0/4151733691 shutdown_connections 2023-10-03T10:14:09.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.510+0000 7f28d07f1700 1 --2- 172.21.15.3:0/4151733691 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f28c8136fe0 0x7f28c81393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.510+0000 7f28d07f1700 1 --2- 172.21.15.3:0/4151733691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28c809f410 0x7f28c8136aa0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.510+0000 7f28d07f1700 1 -- 172.21.15.3:0/4151733691 >> 172.21.15.3:0/4151733691 conn(0x7f28c80a4210 msgr2=0x7f28c80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:09.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.510+0000 7f28d07f1700 1 -- 172.21.15.3:0/4151733691 shutdown_connections 2023-10-03T10:14:09.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.510+0000 7f28d07f1700 1 -- 172.21.15.3:0/4151733691 wait complete. 2023-10-03T10:14:09.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.512+0000 7f28d07f1700 1 Processor -- start 2023-10-03T10:14:09.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.512+0000 7f28d07f1700 1 -- start start 2023-10-03T10:14:09.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.513+0000 7f28d07f1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f28c809e930 0x7f28c811c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:09.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.513+0000 7f28d07f1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28c809f410 0x7f28c811ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:09.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.513+0000 7f28d07f1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f28c8136fe0 0x7f28c811cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:09.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.513+0000 7f28d07f1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c811fd80 con 0x7f28c809f410 2023-10-03T10:14:09.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.513+0000 7f28d07f1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c8120b60 con 0x7f28c8136fe0 2023-10-03T10:14:09.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.513+0000 7f28d07f1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28c811ff80 con 0x7f28c809e930 2023-10-03T10:14:09.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.513+0000 7f28cdd8c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28c809f410 0x7f28c811ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:09.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.513+0000 7f28cdd8c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28c809f410 0x7f28c811ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57958/0 (socket says 172.21.15.3:57958) 2023-10-03T10:14:09.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.513+0000 7f28cdd8c700 1 -- 172.21.15.3:0/2076170589 learned_addr learned my addr 172.21.15.3:0/2076170589 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:09.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.513+0000 7f28ce58d700 1 --2- 172.21.15.3:0/2076170589 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f28c809e930 0x7f28c811c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:09.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.513+0000 7f28ced8e700 1 --2- 172.21.15.3:0/2076170589 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f28c8136fe0 0x7f28c811cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:09.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.513+0000 7f28cdd8c700 1 -- 172.21.15.3:0/2076170589 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f28c809e930 msgr2=0x7f28c811c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:09.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.513+0000 7f28cdd8c700 1 --2- 172.21.15.3:0/2076170589 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f28c809e930 0x7f28c811c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.513+0000 7f28cdd8c700 1 -- 172.21.15.3:0/2076170589 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f28c8136fe0 msgr2=0x7f28c811cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:09.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.513+0000 7f28cdd8c700 1 --2- 172.21.15.3:0/2076170589 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f28c8136fe0 0x7f28c811cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.513+0000 7f28cdd8c700 1 -- 172.21.15.3:0/2076170589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c0009a70 con 0x7f28c809f410 2023-10-03T10:14:09.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.513+0000 7f28cdd8c700 1 --2- 172.21.15.3:0/2076170589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28c809f410 0x7f28c811ca20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f28c806fea0 tx=0x7f28b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:09.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.514+0000 7f28beffd700 1 -- 172.21.15.3:0/2076170589 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b80042c0 con 0x7f28c809f410 2023-10-03T10:14:09.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.514+0000 7f28beffd700 1 -- 172.21.15.3:0/2076170589 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b801f070 con 0x7f28c809f410 2023-10-03T10:14:09.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.514+0000 7f28beffd700 1 -- 172.21.15.3:0/2076170589 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b80036a0 con 0x7f28c809f410 2023-10-03T10:14:09.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.514+0000 7f28d07f1700 1 -- 172.21.15.3:0/2076170589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b8009550 con 0x7f28c809f410 2023-10-03T10:14:09.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.514+0000 7f28d07f1700 1 -- 172.21.15.3:0/2076170589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c8071d20 con 0x7f28c809f410 2023-10-03T10:14:09.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.515+0000 7f28d07f1700 1 -- 172.21.15.3:0/2076170589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c81207d0 con 0x7f28c809f410 2023-10-03T10:14:09.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.515+0000 7f28beffd700 1 -- 172.21.15.3:0/2076170589 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f28b8021030 con 0x7f28c809f410 2023-10-03T10:14:09.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.515+0000 7f28beffd700 1 -- 172.21.15.3:0/2076170589 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f28b8040a00 con 0x7f28c809f410 2023-10-03T10:14:09.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.525+0000 7f28beffd700 1 -- 172.21.15.3:0/2076170589 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f28b80199a0 con 0x7f28c809f410 2023-10-03T10:14:09.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.598+0000 7f72494f4700 1 -- 172.21.15.3:0/1389017622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f724406c360 con 0x7f724409d6f0 2023-10-03T10:14:09.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.598+0000 7f723b7fe700 1 -- 172.21.15.3:0/1389017622 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7234013e20 con 0x7f724409d6f0 2023-10-03T10:14:09.599 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:09.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.602+0000 7f72494f4700 1 -- 172.21.15.3:0/1389017622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f724409d6f0 msgr2=0x7f7244122460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:09.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.602+0000 7f72494f4700 1 --2- 172.21.15.3:0/1389017622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f724409d6f0 0x7f7244122460 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7234003f50 tx=0x7f7234004ad0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.602+0000 7f72494f4700 1 -- 172.21.15.3:0/1389017622 shutdown_connections 2023-10-03T10:14:09.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.602+0000 7f72494f4700 1 --2- 172.21.15.3:0/1389017622 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f724409e410 0x7f7244122ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.602+0000 7f72494f4700 1 --2- 172.21.15.3:0/1389017622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f724409d6f0 0x7f7244122460 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.602+0000 7f72494f4700 1 --2- 172.21.15.3:0/1389017622 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f724409e000 0x7f72441229a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.602+0000 7f72494f4700 1 -- 172.21.15.3:0/1389017622 >> 172.21.15.3:0/1389017622 conn(0x7f7244066f50 msgr2=0x7f72440711b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:09.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.602+0000 7f72494f4700 1 -- 172.21.15.3:0/1389017622 shutdown_connections 2023-10-03T10:14:09.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.602+0000 7f72494f4700 1 -- 172.21.15.3:0/1389017622 wait complete. 2023-10-03T10:14:09.621 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:09.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.677+0000 7f633f409700 1 -- 172.21.15.3:0/392724875 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6338074770 con 0x7f633809d630 2023-10-03T10:14:09.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.677+0000 7f632dffb700 1 -- 172.21.15.3:0/392724875 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6334017070 con 0x7f633809d630 2023-10-03T10:14:09.678 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:09.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.679+0000 7f633f409700 1 -- 172.21.15.3:0/392724875 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f633809d630 msgr2=0x7f63381c8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:09.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.679+0000 7f633f409700 1 --2- 172.21.15.3:0/392724875 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f633809d630 0x7f63381c8310 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f633400ebe0 tx=0x7f633400eef0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.679+0000 7f633f409700 1 -- 172.21.15.3:0/392724875 shutdown_connections 2023-10-03T10:14:09.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.679+0000 7f633f409700 1 --2- 172.21.15.3:0/392724875 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6338133930 0x7f63381c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.679+0000 7f633f409700 1 --2- 172.21.15.3:0/392724875 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f633809d630 0x7f63381c8310 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.679+0000 7f633f409700 1 --2- 172.21.15.3:0/392724875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63380a0a30 0x7f63381c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.679+0000 7f633f409700 1 -- 172.21.15.3:0/392724875 >> 172.21.15.3:0/392724875 conn(0x7f63380a3050 msgr2=0x7f63381bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:09.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.679+0000 7f633f409700 1 -- 172.21.15.3:0/392724875 shutdown_connections 2023-10-03T10:14:09.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.680+0000 7f633f409700 1 -- 172.21.15.3:0/392724875 wait complete. 2023-10-03T10:14:09.694 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:09.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.742+0000 7f28d07f1700 1 -- 172.21.15.3:0/2076170589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f28c8074ea0 con 0x7f28c809f410 2023-10-03T10:14:09.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.742+0000 7f28beffd700 1 -- 172.21.15.3:0/2076170589 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f28b8015070 con 0x7f28c809f410 2023-10-03T10:14:09.743 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:09.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.745+0000 7f28d07f1700 1 -- 172.21.15.3:0/2076170589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28c809f410 msgr2=0x7f28c811ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:09.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.745+0000 7f28d07f1700 1 --2- 172.21.15.3:0/2076170589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28c809f410 0x7f28c811ca20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f28c806fea0 tx=0x7f28b8004750 comp rx=0 tx=0).stop 2023-10-03T10:14:09.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.745+0000 7f28d07f1700 1 -- 172.21.15.3:0/2076170589 shutdown_connections 2023-10-03T10:14:09.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.745+0000 7f28d07f1700 1 --2- 172.21.15.3:0/2076170589 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f28c8136fe0 0x7f28c811cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.745+0000 7f28d07f1700 1 --2- 172.21.15.3:0/2076170589 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f28c809e930 0x7f28c811c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.745+0000 7f28d07f1700 1 --2- 172.21.15.3:0/2076170589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28c809f410 0x7f28c811ca20 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:09.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.745+0000 7f28d07f1700 1 -- 172.21.15.3:0/2076170589 >> 172.21.15.3:0/2076170589 conn(0x7f28c80a4210 msgr2=0x7f28c8134b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:09.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.745+0000 7f28d07f1700 1 -- 172.21.15.3:0/2076170589 shutdown_connections 2023-10-03T10:14:09.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:09.745+0000 7f28d07f1700 1 -- 172.21.15.3:0/2076170589 wait complete. 2023-10-03T10:14:09.758 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:10.111 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:10.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.258+0000 7f58c4e2a700 1 Processor -- start 2023-10-03T10:14:10.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.258+0000 7f58c4e2a700 1 -- start start 2023-10-03T10:14:10.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.259+0000 7f58c4e2a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f58c009e930 0x7f58c009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:10.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.259+0000 7f58c4e2a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f58c009f650 0x7f58c0131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:10.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.260+0000 7f58c4e2a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c0070940 con 0x7f58c009f240 2023-10-03T10:14:10.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.260+0000 7f58c4e2a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c006fd60 con 0x7f58c009e930 2023-10-03T10:14:10.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.260+0000 7f58c4e2a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f58c006fa80 con 0x7f58c009f650 2023-10-03T10:14:10.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.260+0000 7f58be59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f58c009e930 0x7f58c009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:10.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.260+0000 7f58be59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f58c009e930 0x7f58c009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57968/0 (socket says 172.21.15.3:57968) 2023-10-03T10:14:10.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.260+0000 7f58be59c700 1 -- 172.21.15.3:0/3184129661 learned_addr learned my addr 172.21.15.3:0/3184129661 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:10.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.260+0000 7f58bed9d700 1 --2- 172.21.15.3:0/3184129661 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f58c009f650 0x7f58c0131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:10.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.260+0000 7f58be59c700 1 -- 172.21.15.3:0/3184129661 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f58c009f650 msgr2=0x7f58c0131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:10.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.260+0000 7f58be59c700 1 --2- 172.21.15.3:0/3184129661 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f58c009f650 0x7f58c0131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:10.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.260+0000 7f58be59c700 1 -- 172.21.15.3:0/3184129661 >> v1:172.21.15.3:6789/0 conn(0x7f58c009f240 legacy=0x7f58c01317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:10.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.260+0000 7f58be59c700 1 -- 172.21.15.3:0/3184129661 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c009fb60 con 0x7f58c009e930 2023-10-03T10:14:10.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.260+0000 7f58bd59a700 1 -- 172.21.15.3:0/3184129661 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3949828004 0 0) 0x7f58c0070940 con 0x7f58c009f240 2023-10-03T10:14:10.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.260+0000 7f58be59c700 1 --2- 172.21.15.3:0/3184129661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58c009e930 0x7f58c009ed00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f58b00098f0 tx=0x7f58b0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fccd184806cc6aa server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.260+0000 7f58bd59a700 1 -- 172.21.15.3:0/3184129661 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b0011830 con 0x7f58c009e930 2023-10-03T10:14:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.260+0000 7f58bd59a700 1 -- 172.21.15.3:0/3184129661 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b0011990 con 0x7f58c009e930 2023-10-03T10:14:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.260+0000 7f58bd59a700 1 -- 172.21.15.3:0/3184129661 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b0011d20 con 0x7f58c009e930 2023-10-03T10:14:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.261+0000 7f58c4e2a700 1 -- 172.21.15.3:0/3184129661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58c009e930 msgr2=0x7f58c009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.261+0000 7f58c4e2a700 1 --2- 172.21.15.3:0/3184129661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58c009e930 0x7f58c009ed00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f58b00098f0 tx=0x7f58b0009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.261+0000 7f58c4e2a700 1 -- 172.21.15.3:0/3184129661 shutdown_connections 2023-10-03T10:14:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.261+0000 7f58c4e2a700 1 --2- 172.21.15.3:0/3184129661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58c009e930 0x7f58c009ed00 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.261+0000 7f58c4e2a700 1 --2- 172.21.15.3:0/3184129661 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f58c009f650 0x7f58c0131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.261+0000 7f58c4e2a700 1 -- 172.21.15.3:0/3184129661 >> 172.21.15.3:0/3184129661 conn(0x7f58c00a4210 msgr2=0x7f58c00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.261+0000 7f58c4e2a700 1 -- 172.21.15.3:0/3184129661 shutdown_connections 2023-10-03T10:14:10.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.261+0000 7f58c4e2a700 1 -- 172.21.15.3:0/3184129661 wait complete. 2023-10-03T10:14:10.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.263+0000 7f58c4e2a700 1 Processor -- start 2023-10-03T10:14:10.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.263+0000 7f58c4e2a700 1 -- start start 2023-10-03T10:14:10.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.264+0000 7f58c4e2a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f58c009e930 0x7f58c01c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:10.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.264+0000 7f58c4e2a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58c009f240 0x7f58c01c3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:10.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.264+0000 7f58c4e2a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f58c009f650 0x7f58c01c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:10.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.264+0000 7f58c4e2a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c01a58a0 con 0x7f58c009f240 2023-10-03T10:14:10.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.264+0000 7f58c4e2a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c01a56b0 con 0x7f58c009f650 2023-10-03T10:14:10.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.264+0000 7f58c4e2a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f58c00a3310 con 0x7f58c009e930 2023-10-03T10:14:10.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.264+0000 7f58be59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f58c009e930 0x7f58c01c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:10.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.264+0000 7f58be59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f58c009e930 0x7f58c01c2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:40240/0 (socket says 172.21.15.3:40240) 2023-10-03T10:14:10.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.264+0000 7f58be59c700 1 -- 172.21.15.3:0/695239753 learned_addr learned my addr 172.21.15.3:0/695239753 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:10.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.264+0000 7f58bdd9b700 1 --2- 172.21.15.3:0/695239753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58c009f240 0x7f58c01c3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:10.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.264+0000 7f58bed9d700 1 --2- 172.21.15.3:0/695239753 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f58c009f650 0x7f58c01c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:10.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.264+0000 7f58be59c700 1 -- 172.21.15.3:0/695239753 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f58c009f650 msgr2=0x7f58c01c5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:10.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.264+0000 7f58be59c700 1 --2- 172.21.15.3:0/695239753 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f58c009f650 0x7f58c01c5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:10.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.264+0000 7f58be59c700 1 -- 172.21.15.3:0/695239753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58c009f240 msgr2=0x7f58c01c3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:10.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.264+0000 7f58be59c700 1 --2- 172.21.15.3:0/695239753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58c009f240 0x7f58c01c3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:10.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.264+0000 7f58be59c700 1 -- 172.21.15.3:0/695239753 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b0009550 con 0x7f58c009e930 2023-10-03T10:14:10.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.264+0000 7f58bed9d700 1 --2- 172.21.15.3:0/695239753 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f58c009f650 0x7f58c01c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:10.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.264+0000 7f58be59c700 1 --2- 172.21.15.3:0/695239753 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f58c009e930 0x7f58c01c2e50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f58c00a0b00 tx=0x7f58b0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:10.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.264+0000 7f58aeffd700 1 -- 172.21.15.3:0/695239753 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b0004300 con 0x7f58c009e930 2023-10-03T10:14:10.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.264+0000 7f58aeffd700 1 -- 172.21.15.3:0/695239753 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b001f070 con 0x7f58c009e930 2023-10-03T10:14:10.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.264+0000 7f58aeffd700 1 -- 172.21.15.3:0/695239753 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b00036a0 con 0x7f58c009e930 2023-10-03T10:14:10.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.265+0000 7f58c4e2a700 1 -- 172.21.15.3:0/695239753 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c00a0190 con 0x7f58c009e930 2023-10-03T10:14:10.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.265+0000 7f58c4e2a700 1 -- 172.21.15.3:0/695239753 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c01a7610 con 0x7f58c009e930 2023-10-03T10:14:10.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.265+0000 7f58aeffd700 1 -- 172.21.15.3:0/695239753 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f58b0021030 con 0x7f58c009e930 2023-10-03T10:14:10.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.265+0000 7f58c4e2a700 1 -- 172.21.15.3:0/695239753 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c01a6960 con 0x7f58c009e930 2023-10-03T10:14:10.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.266+0000 7f58aeffd700 1 -- 172.21.15.3:0/695239753 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f58b00408d0 con 0x7f58c009e930 2023-10-03T10:14:10.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.271+0000 7f58aeffd700 1 -- 172.21.15.3:0/695239753 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f58b001a450 con 0x7f58c009e930 2023-10-03T10:14:10.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.476+0000 7f58c4e2a700 1 -- 172.21.15.3:0/695239753 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f58c01a7610 con 0x7f58c009e930 2023-10-03T10:14:10.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.476+0000 7f58aeffd700 1 -- 172.21.15.3:0/695239753 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f58c01a7610 con 0x7f58c009e930 2023-10-03T10:14:10.477 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:10.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.479+0000 7f58c4e2a700 1 -- 172.21.15.3:0/695239753 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f58c009e930 msgr2=0x7f58c01c2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:10.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.479+0000 7f58c4e2a700 1 --2- 172.21.15.3:0/695239753 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f58c009e930 0x7f58c01c2e50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f58c00a0b00 tx=0x7f58b0004750 comp rx=0 tx=0).stop 2023-10-03T10:14:10.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.479+0000 7f58c4e2a700 1 -- 172.21.15.3:0/695239753 shutdown_connections 2023-10-03T10:14:10.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.479+0000 7f58c4e2a700 1 --2- 172.21.15.3:0/695239753 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f58c009f650 0x7f58c01c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:10.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.479+0000 7f58c4e2a700 1 --2- 172.21.15.3:0/695239753 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f58c009e930 0x7f58c01c2e50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:10.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.479+0000 7f58c4e2a700 1 --2- 172.21.15.3:0/695239753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58c009f240 0x7f58c01c3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:10.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.479+0000 7f58c4e2a700 1 -- 172.21.15.3:0/695239753 >> 172.21.15.3:0/695239753 conn(0x7f58c00a4210 msgr2=0x7f58c01b8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:10.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.479+0000 7f58c4e2a700 1 -- 172.21.15.3:0/695239753 shutdown_connections 2023-10-03T10:14:10.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.479+0000 7f58c4e2a700 1 -- 172.21.15.3:0/695239753 wait complete. 2023-10-03T10:14:10.493 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:10.622 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:10.694 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:10.759 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:10.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.764+0000 7f503ebc1700 1 Processor -- start 2023-10-03T10:14:10.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.764+0000 7f503ebc1700 1 -- start start 2023-10-03T10:14:10.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.765+0000 7f503ebc1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f50380a09c0 0x7f50380a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:10.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.765+0000 7f503ebc1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f503809d5c0 0x7f50380a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:10.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.765+0000 7f503ebc1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50380a1810 con 0x7f5038133920 2023-10-03T10:14:10.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.765+0000 7f503ebc1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f503809ea10 con 0x7f50380a09c0 2023-10-03T10:14:10.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.765+0000 7f503ebc1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f503809e210 con 0x7f503809d5c0 2023-10-03T10:14:10.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.765+0000 7f5037fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f503809d5c0 0x7f50380a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:10.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.765+0000 7f503c95d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f50380a09c0 0x7f50380a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:10.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.765+0000 7f503c95d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f50380a09c0 0x7f50380a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57992/0 (socket says 172.21.15.3:57992) 2023-10-03T10:14:10.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.765+0000 7f503c95d700 1 -- 172.21.15.3:0/2225110508 learned_addr learned my addr 172.21.15.3:0/2225110508 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:10.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.766+0000 7f503c95d700 1 -- 172.21.15.3:0/2225110508 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f503809d5c0 msgr2=0x7f50380a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:10.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.766+0000 7f503c95d700 1 --2- 172.21.15.3:0/2225110508 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f503809d5c0 0x7f50380a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:10.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.766+0000 7f503c95d700 1 -- 172.21.15.3:0/2225110508 >> v1:172.21.15.3:6789/0 conn(0x7f5038133920 legacy=0x7f5038133d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:10.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.766+0000 7f503c95d700 1 -- 172.21.15.3:0/2225110508 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50380a26f0 con 0x7f50380a09c0 2023-10-03T10:14:10.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.766+0000 7f5037fff700 1 --2- 172.21.15.3:0/2225110508 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f503809d5c0 0x7f50380a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:10.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.766+0000 7f503c95d700 1 --2- 172.21.15.3:0/2225110508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50380a09c0 0x7f50380a0d90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f502c0098f0 tx=0x7f502c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e82664eec49171 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:10.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.766+0000 7f50377fe700 1 -- 172.21.15.3:0/2225110508 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c011830 con 0x7f50380a09c0 2023-10-03T10:14:10.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.766+0000 7f50377fe700 1 -- 172.21.15.3:0/2225110508 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502c011990 con 0x7f50380a09c0 2023-10-03T10:14:10.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.766+0000 7f50377fe700 1 -- 172.21.15.3:0/2225110508 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c011d20 con 0x7f50380a09c0 2023-10-03T10:14:10.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.767+0000 7f503ebc1700 1 -- 172.21.15.3:0/2225110508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50380a09c0 msgr2=0x7f50380a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:10.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.767+0000 7f503ebc1700 1 --2- 172.21.15.3:0/2225110508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50380a09c0 0x7f50380a0d90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f502c0098f0 tx=0x7f502c009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:10.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.767+0000 7f503ebc1700 1 -- 172.21.15.3:0/2225110508 shutdown_connections 2023-10-03T10:14:10.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.767+0000 7f503ebc1700 1 --2- 172.21.15.3:0/2225110508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50380a09c0 0x7f50380a0d90 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:10.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.767+0000 7f503ebc1700 1 --2- 172.21.15.3:0/2225110508 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f503809d5c0 0x7f50380a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:10.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.767+0000 7f503ebc1700 1 -- 172.21.15.3:0/2225110508 >> 172.21.15.3:0/2225110508 conn(0x7f50380a2fe0 msgr2=0x7f50380a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:10.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.767+0000 7f503ebc1700 1 -- 172.21.15.3:0/2225110508 shutdown_connections 2023-10-03T10:14:10.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.767+0000 7f503ebc1700 1 -- 172.21.15.3:0/2225110508 wait complete. 2023-10-03T10:14:10.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.768+0000 7f503ebc1700 1 Processor -- start 2023-10-03T10:14:10.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.768+0000 7f503ebc1700 1 -- start start 2023-10-03T10:14:10.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.769+0000 7f503ebc1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f503809d5c0 0x7f503811f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:10.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.769+0000 7f503ebc1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50380a09c0 0x7f503811f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:10.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.769+0000 7f503ebc1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5038133920 0x7f503811fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:10.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.769+0000 7f503ebc1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5038120eb0 con 0x7f5038133920 2023-10-03T10:14:10.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.769+0000 7f503ebc1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5038120cc0 con 0x7f50380a09c0 2023-10-03T10:14:10.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.769+0000 7f503ebc1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5038121f80 con 0x7f503809d5c0 2023-10-03T10:14:10.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.769+0000 7f503c95d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f503809d5c0 0x7f503811f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:10.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.769+0000 7f503c95d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f503809d5c0 0x7f503811f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:40248/0 (socket says 172.21.15.3:40248) 2023-10-03T10:14:10.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.769+0000 7f503c95d700 1 -- 172.21.15.3:0/2110577485 learned_addr learned my addr 172.21.15.3:0/2110577485 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:10.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.769+0000 7f503d15e700 1 --2- 172.21.15.3:0/2110577485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5038133920 0x7f503811fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:10.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.769+0000 7f5037fff700 1 --2- 172.21.15.3:0/2110577485 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50380a09c0 0x7f503811f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:10.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.769+0000 7f5037fff700 1 -- 172.21.15.3:0/2110577485 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f503809d5c0 msgr2=0x7f503811f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:10.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.769+0000 7f5037fff700 1 --2- 172.21.15.3:0/2110577485 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f503809d5c0 0x7f503811f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:10.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.769+0000 7f5037fff700 1 -- 172.21.15.3:0/2110577485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5038133920 msgr2=0x7f503811fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:10.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.769+0000 7f5037fff700 1 --2- 172.21.15.3:0/2110577485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5038133920 0x7f503811fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:10.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.769+0000 7f5037fff700 1 -- 172.21.15.3:0/2110577485 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50240094a0 con 0x7f50380a09c0 2023-10-03T10:14:10.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.769+0000 7f503d15e700 1 --2- 172.21.15.3:0/2110577485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5038133920 0x7f503811fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:10.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.770+0000 7f5037fff700 1 --2- 172.21.15.3:0/2110577485 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50380a09c0 0x7f503811f5f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f502400e930 tx=0x7f502400ec40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:10.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.770+0000 7f50357fa700 1 -- 172.21.15.3:0/2110577485 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5024013680 con 0x7f50380a09c0 2023-10-03T10:14:10.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.770+0000 7f50357fa700 1 -- 172.21.15.3:0/2110577485 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50240137e0 con 0x7f50380a09c0 2023-10-03T10:14:10.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.770+0000 7f503ebc1700 1 -- 172.21.15.3:0/2110577485 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f502c009550 con 0x7f50380a09c0 2023-10-03T10:14:10.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.770+0000 7f50357fa700 1 -- 172.21.15.3:0/2110577485 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5024013b90 con 0x7f50380a09c0 2023-10-03T10:14:10.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.770+0000 7f503ebc1700 1 -- 172.21.15.3:0/2110577485 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5038122680 con 0x7f50380a09c0 2023-10-03T10:14:10.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.771+0000 7f50357fa700 1 -- 172.21.15.3:0/2110577485 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5024013cf0 con 0x7f50380a09c0 2023-10-03T10:14:10.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.771+0000 7f50357fa700 1 -- 172.21.15.3:0/2110577485 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f502403f610 con 0x7f50380a09c0 2023-10-03T10:14:10.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.771+0000 7f503ebc1700 1 -- 172.21.15.3:0/2110577485 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50180052e0 con 0x7f50380a09c0 2023-10-03T10:14:10.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.776+0000 7f50357fa700 1 -- 172.21.15.3:0/2110577485 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5024021080 con 0x7f50380a09c0 2023-10-03T10:14:10.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.845+0000 7f30a099a700 1 Processor -- start 2023-10-03T10:14:10.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.846+0000 7f30a099a700 1 -- start start 2023-10-03T10:14:10.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.846+0000 7f30a099a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f309c09e930 0x7f309c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:10.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.846+0000 7f30a099a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f309c09f650 0x7f309c131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:10.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.846+0000 7f30a099a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f309c070940 con 0x7f309c09f240 2023-10-03T10:14:10.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.846+0000 7f30a099a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f309c06fd60 con 0x7f309c09e930 2023-10-03T10:14:10.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.846+0000 7f30a099a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f309c06fa80 con 0x7f309c09f650 2023-10-03T10:14:10.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.846+0000 7f309a59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f309c09e930 0x7f309c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:10.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.847+0000 7f309a59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f309c09e930 0x7f309c09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58010/0 (socket says 172.21.15.3:58010) 2023-10-03T10:14:10.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.847+0000 7f309ad9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f309c09f650 0x7f309c131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:10.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.847+0000 7f309a59c700 1 -- 172.21.15.3:0/948397689 learned_addr learned my addr 172.21.15.3:0/948397689 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:10.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.847+0000 7f309ad9d700 1 -- 172.21.15.3:0/948397689 >> v2:172.21.15.3:3300/0 conn(0x7f309c09e930 msgr2=0x7f309c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:10.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.847+0000 7f309ad9d700 1 --2- 172.21.15.3:0/948397689 >> v2:172.21.15.3:3300/0 conn(0x7f309c09e930 0x7f309c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:10.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.847+0000 7f309ad9d700 1 -- 172.21.15.3:0/948397689 >> v1:172.21.15.3:6789/0 conn(0x7f309c09f240 legacy=0x7f309c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:10.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.847+0000 7f309ad9d700 1 -- 172.21.15.3:0/948397689 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309c09fb60 con 0x7f309c09f650 2023-10-03T10:14:10.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.847+0000 7f309a59c700 1 --2- 172.21.15.3:0/948397689 >> v2:172.21.15.3:3300/0 conn(0x7f309c09e930 0x7f309c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:10.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.847+0000 7f309ad9d700 1 --2- 172.21.15.3:0/948397689 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f309c09f650 0x7f309c131f00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3094009d90 tx=0x7f309400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a0fd890e4678bc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:10.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.847+0000 7f309959a700 1 -- 172.21.15.3:0/948397689 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309400e070 con 0x7f309c09f650 2023-10-03T10:14:10.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.847+0000 7f309959a700 1 -- 172.21.15.3:0/948397689 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f309400bb20 con 0x7f309c09f650 2023-10-03T10:14:10.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.847+0000 7f309959a700 1 -- 172.21.15.3:0/948397689 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3094012430 con 0x7f309c09f650 2023-10-03T10:14:10.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.848+0000 7f30a099a700 1 -- 172.21.15.3:0/948397689 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f309c09f650 msgr2=0x7f309c131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:10.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.848+0000 7f30a099a700 1 --2- 172.21.15.3:0/948397689 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f309c09f650 0x7f309c131f00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3094009d90 tx=0x7f309400b330 comp rx=0 tx=0).stop 2023-10-03T10:14:10.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.848+0000 7f30a099a700 1 -- 172.21.15.3:0/948397689 shutdown_connections 2023-10-03T10:14:10.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.848+0000 7f30a099a700 1 --2- 172.21.15.3:0/948397689 >> v2:172.21.15.3:3300/0 conn(0x7f309c09e930 0x7f309c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:10.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.848+0000 7f30a099a700 1 --2- 172.21.15.3:0/948397689 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f309c09f650 0x7f309c131f00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:10.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.848+0000 7f30a099a700 1 -- 172.21.15.3:0/948397689 >> 172.21.15.3:0/948397689 conn(0x7f309c0a4210 msgr2=0x7f309c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:10.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.848+0000 7f30a099a700 1 -- 172.21.15.3:0/948397689 shutdown_connections 2023-10-03T10:14:10.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.848+0000 7f30a099a700 1 -- 172.21.15.3:0/948397689 wait complete. 2023-10-03T10:14:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.849+0000 7f30a099a700 1 Processor -- start 2023-10-03T10:14:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.849+0000 7f30a099a700 1 -- start start 2023-10-03T10:14:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.850+0000 7f30a099a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f309c09e930 0x7f309c1a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.850+0000 7f30a099a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f309c09f240 0x7f309c1a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.850+0000 7f30a099a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f309c09f650 0x7f309c1a6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.850+0000 7f30a099a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c1a7920 con 0x7f309c09f650 2023-10-03T10:14:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.850+0000 7f30a099a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309c1a94b0 con 0x7f309c09f240 2023-10-03T10:14:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.850+0000 7f30a099a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f309c1aa0c0 con 0x7f309c09e930 2023-10-03T10:14:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.850+0000 7f309a59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f309c09e930 0x7f309c1a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.850+0000 7f3099d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f309c09f240 0x7f309c1a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.850+0000 7f309a59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f309c09e930 0x7f309c1a5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:40262/0 (socket says 172.21.15.3:40262) 2023-10-03T10:14:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.850+0000 7f309a59c700 1 -- 172.21.15.3:0/1787217472 learned_addr learned my addr 172.21.15.3:0/1787217472 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.850+0000 7f309ad9d700 1 --2- 172.21.15.3:0/1787217472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f309c09f650 0x7f309c1a6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.850+0000 7f309a59c700 1 -- 172.21.15.3:0/1787217472 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f309c09f240 msgr2=0x7f309c1a6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.850+0000 7f309a59c700 1 --2- 172.21.15.3:0/1787217472 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f309c09f240 0x7f309c1a6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.850+0000 7f309a59c700 1 -- 172.21.15.3:0/1787217472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f309c09f650 msgr2=0x7f309c1a6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.850+0000 7f309a59c700 1 --2- 172.21.15.3:0/1787217472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f309c09f650 0x7f309c1a6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.851+0000 7f309a59c700 1 -- 172.21.15.3:0/1787217472 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308c009550 con 0x7f309c09e930 2023-10-03T10:14:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.851+0000 7f3099d9b700 1 --2- 172.21.15.3:0/1787217472 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f309c09f240 0x7f309c1a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.851+0000 7f309a59c700 1 --2- 172.21.15.3:0/1787217472 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f309c09e930 0x7f309c1a5d00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f308c00ebe0 tx=0x7f308c00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.851+0000 7f308affd700 1 -- 172.21.15.3:0/1787217472 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c00c850 con 0x7f309c09e930 2023-10-03T10:14:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.851+0000 7f308affd700 1 -- 172.21.15.3:0/1787217472 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308c00c9b0 con 0x7f309c09e930 2023-10-03T10:14:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.851+0000 7f30a099a700 1 -- 172.21.15.3:0/1787217472 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3094009a70 con 0x7f309c09e930 2023-10-03T10:14:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.851+0000 7f308affd700 1 -- 172.21.15.3:0/1787217472 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c00cd20 con 0x7f309c09e930 2023-10-03T10:14:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.851+0000 7f30a099a700 1 -- 172.21.15.3:0/1787217472 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f309c0a0190 con 0x7f309c09e930 2023-10-03T10:14:10.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.852+0000 7f308affd700 1 -- 172.21.15.3:0/1787217472 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f308c013470 con 0x7f309c09e930 2023-10-03T10:14:10.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.852+0000 7f308affd700 1 -- 172.21.15.3:0/1787217472 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f308c03fc40 con 0x7f309c09e930 2023-10-03T10:14:10.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.852+0000 7f30a099a700 1 -- 172.21.15.3:0/1787217472 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f309c1a2b40 con 0x7f309c09e930 2023-10-03T10:14:10.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.857+0000 7f308affd700 1 -- 172.21.15.3:0/1787217472 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f308c01b3b0 con 0x7f309c09e930 2023-10-03T10:14:10.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.911+0000 7fdcd3154700 1 Processor -- start 2023-10-03T10:14:10.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.911+0000 7fdcd3154700 1 -- start start 2023-10-03T10:14:10.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.912+0000 7fdcd3154700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdccc131810 0x7fdccc131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:10.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.912+0000 7fdcd3154700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdccc09a580 0x7fdccc139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:10.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.912+0000 7fdcd3154700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdccc1313c0 con 0x7fdccc099a40 2023-10-03T10:14:10.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.912+0000 7fdcd3154700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc06e8c0 con 0x7fdccc131810 2023-10-03T10:14:10.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.912+0000 7fdcd3154700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc071d20 con 0x7fdccc09a580 2023-10-03T10:14:10.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.912+0000 7fdccbfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdccc099a40 0x7fdccc099e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54346/0 (socket says 172.21.15.3:54346) 2023-10-03T10:14:10.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.912+0000 7fdccbfff700 1 -- 172.21.15.3:0/3445621586 learned_addr learned my addr 172.21.15.3:0/3445621586 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:10.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.912+0000 7fdcd16f1700 1 --2- 172.21.15.3:0/3445621586 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdccc09a580 0x7fdccc139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:10.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.912+0000 7fdcd0ef0700 1 --2- 172.21.15.3:0/3445621586 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdccc131810 0x7fdccc131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:10.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.913+0000 7fdccb7fe700 1 -- 172.21.15.3:0/3445621586 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 331821916 0 0) 0x7fdccc1313c0 con 0x7fdccc099a40 2023-10-03T10:14:10.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.913+0000 7fdccb7fe700 1 -- 172.21.15.3:0/3445621586 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcbc0035b0 con 0x7fdccc099a40 2023-10-03T10:14:10.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.913+0000 7fdcd16f1700 1 -- 172.21.15.3:0/3445621586 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdccc131810 msgr2=0x7fdccc131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:10.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.913+0000 7fdcd16f1700 1 --2- 172.21.15.3:0/3445621586 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdccc131810 0x7fdccc131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:10.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.913+0000 7fdcd16f1700 1 -- 172.21.15.3:0/3445621586 >> v1:172.21.15.3:6789/0 conn(0x7fdccc099a40 legacy=0x7fdccc099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:10.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.913+0000 7fdcd16f1700 1 -- 172.21.15.3:0/3445621586 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdccc0721a0 con 0x7fdccc09a580 2023-10-03T10:14:10.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.913+0000 7fdcd0ef0700 1 --2- 172.21.15.3:0/3445621586 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdccc131810 0x7fdccc131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:10.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.913+0000 7fdcd16f1700 1 --2- 172.21.15.3:0/3445621586 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdccc09a580 0x7fdccc139660 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4009dc0 tx=0x7fdcc400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2de01af9f0966c9f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:10.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.913+0000 7fdccb7fe700 1 -- 172.21.15.3:0/3445621586 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc400e070 con 0x7fdccc09a580 2023-10-03T10:14:10.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.913+0000 7fdccb7fe700 1 -- 172.21.15.3:0/3445621586 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc400bb20 con 0x7fdccc09a580 2023-10-03T10:14:10.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.913+0000 7fdccb7fe700 1 -- 172.21.15.3:0/3445621586 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4012430 con 0x7fdccc09a580 2023-10-03T10:14:10.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.914+0000 7fdcd3154700 1 -- 172.21.15.3:0/3445621586 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdccc09a580 msgr2=0x7fdccc139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:10.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.914+0000 7fdcd3154700 1 --2- 172.21.15.3:0/3445621586 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdccc09a580 0x7fdccc139660 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4009dc0 tx=0x7fdcc400b330 comp rx=0 tx=0).stop 2023-10-03T10:14:10.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.914+0000 7fdcd3154700 1 -- 172.21.15.3:0/3445621586 shutdown_connections 2023-10-03T10:14:10.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.914+0000 7fdcd3154700 1 --2- 172.21.15.3:0/3445621586 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdccc131810 0x7fdccc131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:10.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.914+0000 7fdcd3154700 1 --2- 172.21.15.3:0/3445621586 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdccc09a580 0x7fdccc139660 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:10.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.914+0000 7fdcd3154700 1 -- 172.21.15.3:0/3445621586 >> 172.21.15.3:0/3445621586 conn(0x7fdccc0a2aa0 msgr2=0x7fdccc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:10.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.914+0000 7fdcd3154700 1 -- 172.21.15.3:0/3445621586 shutdown_connections 2023-10-03T10:14:10.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.914+0000 7fdcd3154700 1 -- 172.21.15.3:0/3445621586 wait complete. 2023-10-03T10:14:10.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.915+0000 7fdcd3154700 1 Processor -- start 2023-10-03T10:14:10.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.915+0000 7fdcd3154700 1 -- start start 2023-10-03T10:14:10.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.916+0000 7fdcd3154700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdccc099a40 0x7fdccc119730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:10.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.916+0000 7fdcd3154700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdccc09a580 0x7fdccc119c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:10.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.916+0000 7fdcd3154700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdccc131810 0x7fdccc11a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:10.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.916+0000 7fdcd3154700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc11abe0 con 0x7fdccc09a580 2023-10-03T10:14:10.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.916+0000 7fdcd3154700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc11b2b0 con 0x7fdccc131810 2023-10-03T10:14:10.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.916+0000 7fdcd3154700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc11c500 con 0x7fdccc099a40 2023-10-03T10:14:10.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.916+0000 7fdcd0ef0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdccc099a40 0x7fdccc119730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:10.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.916+0000 7fdcd0ef0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdccc099a40 0x7fdccc119730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:40272/0 (socket says 172.21.15.3:40272) 2023-10-03T10:14:10.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.916+0000 7fdcd16f1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdccc131810 0x7fdccc11a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:10.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.916+0000 7fdcd0ef0700 1 -- 172.21.15.3:0/3450681550 learned_addr learned my addr 172.21.15.3:0/3450681550 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:10.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.916+0000 7fdccbfff700 1 --2- 172.21.15.3:0/3450681550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdccc09a580 0x7fdccc119c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:10.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.917+0000 7fdcd16f1700 1 -- 172.21.15.3:0/3450681550 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdccc099a40 msgr2=0x7fdccc119730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:10.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.917+0000 7fdcd16f1700 1 --2- 172.21.15.3:0/3450681550 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdccc099a40 0x7fdccc119730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:10.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.917+0000 7fdcd16f1700 1 -- 172.21.15.3:0/3450681550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdccc09a580 msgr2=0x7fdccc119c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:10.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.917+0000 7fdcd16f1700 1 --2- 172.21.15.3:0/3450681550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdccc09a580 0x7fdccc119c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:10.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.917+0000 7fdcd16f1700 1 -- 172.21.15.3:0/3450681550 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc000d940 con 0x7fdccc131810 2023-10-03T10:14:10.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.917+0000 7fdcd0ef0700 1 --2- 172.21.15.3:0/3450681550 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdccc099a40 0x7fdccc119730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:10.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.917+0000 7fdcd16f1700 1 --2- 172.21.15.3:0/3450681550 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdccc131810 0x7fdccc11a1b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4016000 tx=0x7fdcc4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:10.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.917+0000 7fdcc97fa700 1 -- 172.21.15.3:0/3450681550 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4012df0 con 0x7fdccc131810 2023-10-03T10:14:10.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.917+0000 7fdcc97fa700 1 -- 172.21.15.3:0/3450681550 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc4020070 con 0x7fdccc131810 2023-10-03T10:14:10.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.917+0000 7fdcd3154700 1 -- 172.21.15.3:0/3450681550 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc4009a70 con 0x7fdccc131810 2023-10-03T10:14:10.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.917+0000 7fdcc97fa700 1 -- 172.21.15.3:0/3450681550 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4004280 con 0x7fdccc131810 2023-10-03T10:14:10.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.917+0000 7fdcd3154700 1 -- 172.21.15.3:0/3450681550 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdccc0a1a60 con 0x7fdccc131810 2023-10-03T10:14:10.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.918+0000 7fdcc97fa700 1 -- 172.21.15.3:0/3450681550 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdcc40043e0 con 0x7fdccc131810 2023-10-03T10:14:10.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.918+0000 7fdcc97fa700 1 -- 172.21.15.3:0/3450681550 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fdcc4022070 con 0x7fdccc131810 2023-10-03T10:14:10.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.918+0000 7fdcd3154700 1 -- 172.21.15.3:0/3450681550 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdccc06eb00 con 0x7fdccc131810 2023-10-03T10:14:10.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.923+0000 7fdcc97fa700 1 -- 172.21.15.3:0/3450681550 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdcc4004690 con 0x7fdccc131810 2023-10-03T10:14:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.990+0000 7f503ebc1700 1 -- 172.21.15.3:0/2110577485 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5018005590 con 0x7f50380a09c0 2023-10-03T10:14:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.990+0000 7f50357fa700 1 -- 172.21.15.3:0/2110577485 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5024017020 con 0x7f50380a09c0 2023-10-03T10:14:10.991 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.993+0000 7f503ebc1700 1 -- 172.21.15.3:0/2110577485 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50380a09c0 msgr2=0x7f503811f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.993+0000 7f503ebc1700 1 --2- 172.21.15.3:0/2110577485 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50380a09c0 0x7f503811f5f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f502400e930 tx=0x7f502400ec40 comp rx=0 tx=0).stop 2023-10-03T10:14:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.993+0000 7f503ebc1700 1 -- 172.21.15.3:0/2110577485 shutdown_connections 2023-10-03T10:14:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.993+0000 7f503ebc1700 1 --2- 172.21.15.3:0/2110577485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5038133920 0x7f503811fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.993+0000 7f503ebc1700 1 --2- 172.21.15.3:0/2110577485 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f503809d5c0 0x7f503811f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.993+0000 7f503ebc1700 1 --2- 172.21.15.3:0/2110577485 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50380a09c0 0x7f503811f5f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.993+0000 7f503ebc1700 1 -- 172.21.15.3:0/2110577485 >> 172.21.15.3:0/2110577485 conn(0x7f50380a2fe0 msgr2=0x7f5038132d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.993+0000 7f503ebc1700 1 -- 172.21.15.3:0/2110577485 shutdown_connections 2023-10-03T10:14:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:10.993+0000 7f503ebc1700 1 -- 172.21.15.3:0/2110577485 wait complete. 2023-10-03T10:14:11.007 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:11.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.066+0000 7f30a099a700 1 -- 172.21.15.3:0/1787217472 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f309c074ea0 con 0x7f309c09e930 2023-10-03T10:14:11.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.066+0000 7f308affd700 1 -- 172.21.15.3:0/1787217472 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f308c017020 con 0x7f309c09e930 2023-10-03T10:14:11.066 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.068+0000 7f30a099a700 1 -- 172.21.15.3:0/1787217472 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f309c09e930 msgr2=0x7f309c1a5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.068+0000 7f30a099a700 1 --2- 172.21.15.3:0/1787217472 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f309c09e930 0x7f309c1a5d00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f308c00ebe0 tx=0x7f308c00eef0 comp rx=0 tx=0).stop 2023-10-03T10:14:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.068+0000 7f30a099a700 1 -- 172.21.15.3:0/1787217472 shutdown_connections 2023-10-03T10:14:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.068+0000 7f30a099a700 1 --2- 172.21.15.3:0/1787217472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f309c09f650 0x7f309c1a6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.068+0000 7f30a099a700 1 --2- 172.21.15.3:0/1787217472 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f309c09e930 0x7f309c1a5d00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.068+0000 7f30a099a700 1 --2- 172.21.15.3:0/1787217472 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f309c09f240 0x7f309c1a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.068+0000 7f30a099a700 1 -- 172.21.15.3:0/1787217472 >> 172.21.15.3:0/1787217472 conn(0x7f309c0a4210 msgr2=0x7f309c09dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.068+0000 7f30a099a700 1 -- 172.21.15.3:0/1787217472 shutdown_connections 2023-10-03T10:14:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.068+0000 7f30a099a700 1 -- 172.21.15.3:0/1787217472 wait complete. 2023-10-03T10:14:11.087 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:11.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.139+0000 7fdcd3154700 1 -- 172.21.15.3:0/3450681550 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdccc074ea0 con 0x7fdccc131810 2023-10-03T10:14:11.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.139+0000 7fdcc97fa700 1 -- 172.21.15.3:0/3450681550 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdcc402c020 con 0x7fdccc131810 2023-10-03T10:14:11.140 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:11.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.141+0000 7fdcd3154700 1 -- 172.21.15.3:0/3450681550 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdccc131810 msgr2=0x7fdccc11a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:11.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.141+0000 7fdcd3154700 1 --2- 172.21.15.3:0/3450681550 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdccc131810 0x7fdccc11a1b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4016000 tx=0x7fdcc4004ad0 comp rx=0 tx=0).stop 2023-10-03T10:14:11.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.142+0000 7fdcd3154700 1 -- 172.21.15.3:0/3450681550 shutdown_connections 2023-10-03T10:14:11.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.142+0000 7fdcd3154700 1 --2- 172.21.15.3:0/3450681550 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdccc131810 0x7fdccc11a1b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:11.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.142+0000 7fdcd3154700 1 --2- 172.21.15.3:0/3450681550 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdccc099a40 0x7fdccc119730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:11.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.142+0000 7fdcd3154700 1 --2- 172.21.15.3:0/3450681550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdccc09a580 0x7fdccc119c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:11.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.142+0000 7fdcd3154700 1 -- 172.21.15.3:0/3450681550 >> 172.21.15.3:0/3450681550 conn(0x7fdccc0a2aa0 msgr2=0x7fdccc12f220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:11.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.142+0000 7fdcd3154700 1 -- 172.21.15.3:0/3450681550 shutdown_connections 2023-10-03T10:14:11.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.142+0000 7fdcd3154700 1 -- 172.21.15.3:0/3450681550 wait complete. 2023-10-03T10:14:11.155 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:11.494 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:11.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.640+0000 7f932982f700 1 Processor -- start 2023-10-03T10:14:11.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.640+0000 7f932982f700 1 -- start start 2023-10-03T10:14:11.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.640+0000 7f932982f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f93240a09c0 0x7f93240a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:11.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.640+0000 7f932982f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f932409d5c0 0x7f93240a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:11.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.640+0000 7f932982f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93240a1810 con 0x7f9324133920 2023-10-03T10:14:11.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.640+0000 7f932982f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f932409ea10 con 0x7f93240a09c0 2023-10-03T10:14:11.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.640+0000 7f932982f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f932409e210 con 0x7f932409d5c0 2023-10-03T10:14:11.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.641+0000 7f93227fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f932409d5c0 0x7f93240a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:11.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.641+0000 7f9322ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f93240a09c0 0x7f93240a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:11.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.641+0000 7f9322ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f93240a09c0 0x7f93240a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58040/0 (socket says 172.21.15.3:58040) 2023-10-03T10:14:11.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.641+0000 7f9322ffd700 1 -- 172.21.15.3:0/1259551183 learned_addr learned my addr 172.21.15.3:0/1259551183 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:11.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.641+0000 7f93227fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f932409d5c0 0x7f93240a12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:40280/0 (socket says 172.21.15.3:40280) 2023-10-03T10:14:11.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.641+0000 7f93227fc700 1 -- 172.21.15.3:0/1259551183 >> v2:172.21.15.3:3300/0 conn(0x7f93240a09c0 msgr2=0x7f93240a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:11.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.641+0000 7f93227fc700 1 --2- 172.21.15.3:0/1259551183 >> v2:172.21.15.3:3300/0 conn(0x7f93240a09c0 0x7f93240a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:11.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.641+0000 7f93227fc700 1 -- 172.21.15.3:0/1259551183 >> v1:172.21.15.3:6789/0 conn(0x7f9324133920 legacy=0x7f9324133d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:11.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.641+0000 7f93227fc700 1 -- 172.21.15.3:0/1259551183 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93240a26f0 con 0x7f932409d5c0 2023-10-03T10:14:11.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.641+0000 7f9322ffd700 1 --2- 172.21.15.3:0/1259551183 >> v2:172.21.15.3:3300/0 conn(0x7f93240a09c0 0x7f93240a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:11.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.641+0000 7f93227fc700 1 --2- 172.21.15.3:0/1259551183 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f932409d5c0 0x7f93240a12d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9318009840 tx=0x7f9318009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11f23ead2618f911 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:11.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.641+0000 7f9321ffb700 1 -- 172.21.15.3:0/1259551183 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318011830 con 0x7f932409d5c0 2023-10-03T10:14:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.641+0000 7f9321ffb700 1 -- 172.21.15.3:0/1259551183 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9318011990 con 0x7f932409d5c0 2023-10-03T10:14:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.641+0000 7f9321ffb700 1 -- 172.21.15.3:0/1259551183 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318011d20 con 0x7f932409d5c0 2023-10-03T10:14:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.642+0000 7f932982f700 1 -- 172.21.15.3:0/1259551183 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f932409d5c0 msgr2=0x7f93240a12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.642+0000 7f932982f700 1 --2- 172.21.15.3:0/1259551183 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f932409d5c0 0x7f93240a12d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9318009840 tx=0x7f9318009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.642+0000 7f932982f700 1 -- 172.21.15.3:0/1259551183 shutdown_connections 2023-10-03T10:14:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.642+0000 7f932982f700 1 --2- 172.21.15.3:0/1259551183 >> v2:172.21.15.3:3300/0 conn(0x7f93240a09c0 0x7f93240a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.642+0000 7f932982f700 1 --2- 172.21.15.3:0/1259551183 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f932409d5c0 0x7f93240a12d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.642+0000 7f932982f700 1 -- 172.21.15.3:0/1259551183 >> 172.21.15.3:0/1259551183 conn(0x7f93240a2fe0 msgr2=0x7f93240a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.642+0000 7f932982f700 1 -- 172.21.15.3:0/1259551183 shutdown_connections 2023-10-03T10:14:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.642+0000 7f932982f700 1 -- 172.21.15.3:0/1259551183 wait complete. 2023-10-03T10:14:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.643+0000 7f932982f700 1 Processor -- start 2023-10-03T10:14:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.643+0000 7f932982f700 1 -- start start 2023-10-03T10:14:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.644+0000 7f932982f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f932409d5c0 0x7f9324121180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.644+0000 7f932982f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f93240a09c0 0x7f93241216c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.644+0000 7f932982f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9324133920 0x7f9324121c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.644+0000 7f932982f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9324124a50 con 0x7f9324133920 2023-10-03T10:14:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.644+0000 7f932982f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93241246b0 con 0x7f932409d5c0 2023-10-03T10:14:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.644+0000 7f932982f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f93241225e0 con 0x7f93240a09c0 2023-10-03T10:14:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.644+0000 7f93227fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f93240a09c0 0x7f93241216c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.644+0000 7f93237fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9324133920 0x7f9324121c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.644+0000 7f93227fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f93240a09c0 0x7f93241216c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:40282/0 (socket says 172.21.15.3:40282) 2023-10-03T10:14:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.644+0000 7f93227fc700 1 -- 172.21.15.3:0/4023043797 learned_addr learned my addr 172.21.15.3:0/4023043797 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.644+0000 7f9322ffd700 1 --2- 172.21.15.3:0/4023043797 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f932409d5c0 0x7f9324121180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.644+0000 7f93227fc700 1 -- 172.21.15.3:0/4023043797 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f932409d5c0 msgr2=0x7f9324121180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:11.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.644+0000 7f93227fc700 1 --2- 172.21.15.3:0/4023043797 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f932409d5c0 0x7f9324121180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:11.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.644+0000 7f93227fc700 1 -- 172.21.15.3:0/4023043797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9324133920 msgr2=0x7f9324121c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:11.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.644+0000 7f93227fc700 1 --2- 172.21.15.3:0/4023043797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9324133920 0x7f9324121c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:11.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.644+0000 7f93227fc700 1 -- 172.21.15.3:0/4023043797 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9314009550 con 0x7f93240a09c0 2023-10-03T10:14:11.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.644+0000 7f93237fe700 1 --2- 172.21.15.3:0/4023043797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9324133920 0x7f9324121c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:11.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.644+0000 7f93227fc700 1 --2- 172.21.15.3:0/4023043797 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f93240a09c0 0x7f93241216c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9318009fd0 tx=0x7f9318004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:11.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.645+0000 7f930bfff700 1 -- 172.21.15.3:0/4023043797 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93180036a0 con 0x7f93240a09c0 2023-10-03T10:14:11.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.645+0000 7f930bfff700 1 -- 172.21.15.3:0/4023043797 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931801e070 con 0x7f93240a09c0 2023-10-03T10:14:11.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.645+0000 7f932982f700 1 -- 172.21.15.3:0/4023043797 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93180094a0 con 0x7f93240a09c0 2023-10-03T10:14:11.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.645+0000 7f930bfff700 1 -- 172.21.15.3:0/4023043797 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318003a60 con 0x7f93240a09c0 2023-10-03T10:14:11.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.645+0000 7f932982f700 1 -- 172.21.15.3:0/4023043797 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932409cec0 con 0x7f93240a09c0 2023-10-03T10:14:11.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.645+0000 7f930bfff700 1 -- 172.21.15.3:0/4023043797 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9318003bc0 con 0x7f93240a09c0 2023-10-03T10:14:11.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.646+0000 7f930bfff700 1 -- 172.21.15.3:0/4023043797 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9318020070 con 0x7f93240a09c0 2023-10-03T10:14:11.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.646+0000 7f932982f700 1 -- 172.21.15.3:0/4023043797 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f932411df50 con 0x7f93240a09c0 2023-10-03T10:14:11.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.651+0000 7f930bfff700 1 -- 172.21.15.3:0/4023043797 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9318015070 con 0x7f93240a09c0 2023-10-03T10:14:11.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.854+0000 7f932982f700 1 -- 172.21.15.3:0/4023043797 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f93240747f0 con 0x7f93240a09c0 2023-10-03T10:14:11.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.854+0000 7f930bfff700 1 -- 172.21.15.3:0/4023043797 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f93180243e0 con 0x7f93240a09c0 2023-10-03T10:14:11.855 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:11.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.856+0000 7f932982f700 1 -- 172.21.15.3:0/4023043797 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f93240a09c0 msgr2=0x7f93241216c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:11.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.856+0000 7f932982f700 1 --2- 172.21.15.3:0/4023043797 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f93240a09c0 0x7f93241216c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9318009fd0 tx=0x7f9318004750 comp rx=0 tx=0).stop 2023-10-03T10:14:11.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.856+0000 7f932982f700 1 -- 172.21.15.3:0/4023043797 shutdown_connections 2023-10-03T10:14:11.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.856+0000 7f932982f700 1 --2- 172.21.15.3:0/4023043797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9324133920 0x7f9324121c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:11.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.856+0000 7f932982f700 1 --2- 172.21.15.3:0/4023043797 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f932409d5c0 0x7f9324121180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:11.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.856+0000 7f932982f700 1 --2- 172.21.15.3:0/4023043797 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f93240a09c0 0x7f93241216c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:11.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.856+0000 7f932982f700 1 -- 172.21.15.3:0/4023043797 >> 172.21.15.3:0/4023043797 conn(0x7f93240a2fe0 msgr2=0x7f93241331c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:11.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.857+0000 7f932982f700 1 -- 172.21.15.3:0/4023043797 shutdown_connections 2023-10-03T10:14:11.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:11.857+0000 7f932982f700 1 -- 172.21.15.3:0/4023043797 wait complete. 2023-10-03T10:14:11.871 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:12.008 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:12.088 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:12.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.154+0000 7fe181048700 1 Processor -- start 2023-10-03T10:14:12.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.154+0000 7fe181048700 1 -- start start 2023-10-03T10:14:12.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.154+0000 7fe181048700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe17c1337f0 0x7fe17c138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:12.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.154+0000 7fe181048700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe17c138dc0 0x7fe17c13b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:12.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.154+0000 7fe181048700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe17c06b990 con 0x7fe17c133420 2023-10-03T10:14:12.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.154+0000 7fe181048700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c09b0e0 con 0x7fe17c1337f0 2023-10-03T10:14:12.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.155+0000 7fe181048700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c133c20 con 0x7fe17c138dc0 2023-10-03T10:14:12.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.155+0000 7fe17a59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe17c1337f0 0x7fe17c138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:12.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.155+0000 7fe17ad9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe17c133420 0x7fe17c133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54358/0 (socket says 172.21.15.3:54358) 2023-10-03T10:14:12.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.155+0000 7fe17ad9d700 1 -- 172.21.15.3:0/594451685 learned_addr learned my addr 172.21.15.3:0/594451685 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:12.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.155+0000 7fe17b59e700 1 --2- 172.21.15.3:0/594451685 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe17c138dc0 0x7fe17c13b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:12.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.155+0000 7fe17a59c700 1 -- 172.21.15.3:0/594451685 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe17c138dc0 msgr2=0x7fe17c13b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:12.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.155+0000 7fe179d9b700 1 -- 172.21.15.3:0/594451685 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3282413180 0 0) 0x7fe17c06b990 con 0x7fe17c133420 2023-10-03T10:14:12.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.155+0000 7fe17a59c700 1 --2- 172.21.15.3:0/594451685 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe17c138dc0 0x7fe17c13b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:12.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.155+0000 7fe17a59c700 1 -- 172.21.15.3:0/594451685 >> v1:172.21.15.3:6789/0 conn(0x7fe17c133420 legacy=0x7fe17c133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:12.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.155+0000 7fe17a59c700 1 -- 172.21.15.3:0/594451685 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17c0a0590 con 0x7fe17c1337f0 2023-10-03T10:14:12.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.155+0000 7fe17b59e700 1 --2- 172.21.15.3:0/594451685 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe17c138dc0 0x7fe17c13b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:12.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.155+0000 7fe17a59c700 1 --2- 172.21.15.3:0/594451685 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe17c1337f0 0x7fe17c138880 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe16c009870 tx=0x7fe16c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5385c39597f2e43 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:12.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.155+0000 7fe179d9b700 1 -- 172.21.15.3:0/594451685 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c011830 con 0x7fe17c1337f0 2023-10-03T10:14:12.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.156+0000 7fe179d9b700 1 -- 172.21.15.3:0/594451685 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16c011990 con 0x7fe17c1337f0 2023-10-03T10:14:12.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.156+0000 7fe179d9b700 1 -- 172.21.15.3:0/594451685 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c011d20 con 0x7fe17c1337f0 2023-10-03T10:14:12.157 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:12.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.156+0000 7fe181048700 1 -- 172.21.15.3:0/594451685 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe17c1337f0 msgr2=0x7fe17c138880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:12.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.156+0000 7fe181048700 1 --2- 172.21.15.3:0/594451685 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe17c1337f0 0x7fe17c138880 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe16c009870 tx=0x7fe16c009b80 comp rx=0 tx=0).stop 2023-10-03T10:14:12.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.156+0000 7fe181048700 1 -- 172.21.15.3:0/594451685 shutdown_connections 2023-10-03T10:14:12.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.156+0000 7fe181048700 1 --2- 172.21.15.3:0/594451685 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe17c138dc0 0x7fe17c13b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:12.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.156+0000 7fe181048700 1 --2- 172.21.15.3:0/594451685 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe17c1337f0 0x7fe17c138880 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:12.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.156+0000 7fe181048700 1 -- 172.21.15.3:0/594451685 >> 172.21.15.3:0/594451685 conn(0x7fe17c0a4210 msgr2=0x7fe17c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:12.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.156+0000 7fe181048700 1 -- 172.21.15.3:0/594451685 shutdown_connections 2023-10-03T10:14:12.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.156+0000 7fe181048700 1 -- 172.21.15.3:0/594451685 wait complete. 2023-10-03T10:14:12.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.158+0000 7fe181048700 1 Processor -- start 2023-10-03T10:14:12.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.158+0000 7fe181048700 1 -- start start 2023-10-03T10:14:12.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.158+0000 7fe181048700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe17c133420 0x7fe17c12ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:12.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.158+0000 7fe181048700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe17c1337f0 0x7fe17c12f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:12.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.158+0000 7fe181048700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe17c138dc0 0x7fe17c12f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:12.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.158+0000 7fe181048700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c1330a0 con 0x7fe17c1337f0 2023-10-03T10:14:12.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.158+0000 7fe181048700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c09da90 con 0x7fe17c133420 2023-10-03T10:14:12.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.158+0000 7fe181048700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c09c810 con 0x7fe17c138dc0 2023-10-03T10:14:12.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.158+0000 7fe17ad9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe17c133420 0x7fe17c12ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:12.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.158+0000 7fe17b59e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe17c138dc0 0x7fe17c12f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:12.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.158+0000 7fe17a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe17c1337f0 0x7fe17c12f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:12.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.158+0000 7fe17ad9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe17c133420 0x7fe17c12ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40106/0 (socket says 172.21.15.3:40106) 2023-10-03T10:14:12.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.158+0000 7fe17b59e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe17c138dc0 0x7fe17c12f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:40298/0 (socket says 172.21.15.3:40298) 2023-10-03T10:14:12.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.158+0000 7fe17ad9d700 1 -- 172.21.15.3:0/2471616696 learned_addr learned my addr 172.21.15.3:0/2471616696 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:12.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.159+0000 7fe17a59c700 1 -- 172.21.15.3:0/2471616696 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe17c138dc0 msgr2=0x7fe17c12f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:12.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.159+0000 7fe17a59c700 1 --2- 172.21.15.3:0/2471616696 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe17c138dc0 0x7fe17c12f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:12.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.159+0000 7fe17a59c700 1 -- 172.21.15.3:0/2471616696 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe17c133420 msgr2=0x7fe17c12ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:12.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.159+0000 7fe17a59c700 1 --2- 172.21.15.3:0/2471616696 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe17c133420 0x7fe17c12ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:12.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.159+0000 7fe17a59c700 1 -- 172.21.15.3:0/2471616696 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe174009a70 con 0x7fe17c1337f0 2023-10-03T10:14:12.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.159+0000 7fe17ad9d700 1 --2- 172.21.15.3:0/2471616696 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe17c133420 0x7fe17c12ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:12.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.159+0000 7fe17a59c700 1 --2- 172.21.15.3:0/2471616696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe17c1337f0 0x7fe17c12f390 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe16c016000 tx=0x7fe16c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:12.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.159+0000 7fe1737fe700 1 -- 172.21.15.3:0/2471616696 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c004550 con 0x7fe17c1337f0 2023-10-03T10:14:12.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.159+0000 7fe1737fe700 1 -- 172.21.15.3:0/2471616696 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16c01f070 con 0x7fe17c1337f0 2023-10-03T10:14:12.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.159+0000 7fe181048700 1 -- 172.21.15.3:0/2471616696 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe16c0094a0 con 0x7fe17c1337f0 2023-10-03T10:14:12.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.159+0000 7fe1737fe700 1 -- 172.21.15.3:0/2471616696 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c0038e0 con 0x7fe17c1337f0 2023-10-03T10:14:12.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.159+0000 7fe181048700 1 -- 172.21.15.3:0/2471616696 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe17c09cad0 con 0x7fe17c1337f0 2023-10-03T10:14:12.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.160+0000 7fe1737fe700 1 -- 172.21.15.3:0/2471616696 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe16c003a40 con 0x7fe17c1337f0 2023-10-03T10:14:12.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.160+0000 7fe1717fa700 1 -- 172.21.15.3:0/2471616696 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1540052e0 con 0x7fe17c1337f0 2023-10-03T10:14:12.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.161+0000 7fe1737fe700 1 -- 172.21.15.3:0/2471616696 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe16c021070 con 0x7fe17c1337f0 2023-10-03T10:14:12.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.166+0000 7fe1737fe700 1 -- 172.21.15.3:0/2471616696 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe16c010380 con 0x7fe17c1337f0 2023-10-03T10:14:12.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.236+0000 7f6ce991f700 1 Processor -- start 2023-10-03T10:14:12.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.236+0000 7f6ce991f700 1 -- start start 2023-10-03T10:14:12.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.237+0000 7f6ce991f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ce41316c0 0x7f6ce4131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:12.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.237+0000 7f6ce991f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ce406ae80 0x7f6ce4139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:12.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.237+0000 7f6ce991f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ce409e130 con 0x7f6ce409d9f0 2023-10-03T10:14:12.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.237+0000 7f6ce991f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce41312b0 con 0x7f6ce406ae80 2023-10-03T10:14:12.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.237+0000 7f6ce991f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce409f6e0 con 0x7f6ce41316c0 2023-10-03T10:14:12.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.237+0000 7f6ce37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ce406ae80 0x7f6ce4139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:12.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.237+0000 7f6ce2ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ce41316c0 0x7f6ce4131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:12.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.237+0000 7f6ce27fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6ce409d9f0 0x7f6ce406a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54366/0 (socket says 172.21.15.3:54366) 2023-10-03T10:14:12.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.237+0000 7f6ce37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ce406ae80 0x7f6ce4139550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40108/0 (socket says 172.21.15.3:40108) 2023-10-03T10:14:12.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.237+0000 7f6ce27fc700 1 -- 172.21.15.3:0/116361114 learned_addr learned my addr 172.21.15.3:0/116361114 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:12.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.237+0000 7f6ce37fe700 1 -- 172.21.15.3:0/116361114 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ce41316c0 msgr2=0x7f6ce4131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:12.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.237+0000 7f6ce37fe700 1 --2- 172.21.15.3:0/116361114 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ce41316c0 0x7f6ce4131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:12.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.237+0000 7f6ce37fe700 1 -- 172.21.15.3:0/116361114 >> v1:172.21.15.3:6789/0 conn(0x7f6ce409d9f0 legacy=0x7f6ce406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:12.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.237+0000 7f6ce37fe700 1 -- 172.21.15.3:0/116361114 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce409e760 con 0x7f6ce406ae80 2023-10-03T10:14:12.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.237+0000 7f6ce1ffb700 1 -- 172.21.15.3:0/116361114 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3947727333 0 0) 0x7f6cd8001110 con 0x7f6ce409d9f0 2023-10-03T10:14:12.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.237+0000 7f6ce2ffd700 1 --2- 172.21.15.3:0/116361114 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ce41316c0 0x7f6ce4131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:12.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.237+0000 7f6ce37fe700 1 --2- 172.21.15.3:0/116361114 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ce406ae80 0x7f6ce4139550 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009dc0 tx=0x7f6cdc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e26c3901d477f952 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:12.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.238+0000 7f6ce1ffb700 1 -- 172.21.15.3:0/116361114 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc00e070 con 0x7f6ce406ae80 2023-10-03T10:14:12.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.238+0000 7f6ce1ffb700 1 -- 172.21.15.3:0/116361114 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cdc00bb20 con 0x7f6ce406ae80 2023-10-03T10:14:12.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.238+0000 7f6ce1ffb700 1 -- 172.21.15.3:0/116361114 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc012430 con 0x7f6ce406ae80 2023-10-03T10:14:12.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.238+0000 7f6ce991f700 1 -- 172.21.15.3:0/116361114 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ce406ae80 msgr2=0x7f6ce4139550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:12.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.238+0000 7f6ce991f700 1 --2- 172.21.15.3:0/116361114 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ce406ae80 0x7f6ce4139550 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009dc0 tx=0x7f6cdc00b330 comp rx=0 tx=0).stop 2023-10-03T10:14:12.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.238+0000 7f6ce991f700 1 -- 172.21.15.3:0/116361114 shutdown_connections 2023-10-03T10:14:12.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.238+0000 7f6ce991f700 1 --2- 172.21.15.3:0/116361114 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ce406ae80 0x7f6ce4139550 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:12.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.238+0000 7f6ce991f700 1 --2- 172.21.15.3:0/116361114 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ce41316c0 0x7f6ce4131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:12.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.238+0000 7f6ce991f700 1 -- 172.21.15.3:0/116361114 >> 172.21.15.3:0/116361114 conn(0x7f6ce4066f50 msgr2=0x7f6ce409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:12.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.238+0000 7f6ce991f700 1 -- 172.21.15.3:0/116361114 shutdown_connections 2023-10-03T10:14:12.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.238+0000 7f6ce991f700 1 -- 172.21.15.3:0/116361114 wait complete. 2023-10-03T10:14:12.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.239+0000 7f6ce991f700 1 Processor -- start 2023-10-03T10:14:12.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.240+0000 7f6ce991f700 1 -- start start 2023-10-03T10:14:12.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.240+0000 7f6ce991f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ce406ae80 0x7f6ce40709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:12.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.240+0000 7f6ce991f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ce409d9f0 0x7f6ce4070f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:12.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.240+0000 7f6ce991f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ce41316c0 0x7f6ce4071450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:12.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.240+0000 7f6ce991f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce411b0f0 con 0x7f6ce409d9f0 2023-10-03T10:14:12.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.240+0000 7f6ce991f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce411a6c0 con 0x7f6ce41316c0 2023-10-03T10:14:12.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.240+0000 7f6ce991f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce40a36d0 con 0x7f6ce406ae80 2023-10-03T10:14:12.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.240+0000 7f6ce27fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ce409d9f0 0x7f6ce4070f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:12.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.240+0000 7f6ce37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ce41316c0 0x7f6ce4071450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:12.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.240+0000 7f6ce27fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ce409d9f0 0x7f6ce4070f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58068/0 (socket says 172.21.15.3:58068) 2023-10-03T10:14:12.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.240+0000 7f6ce27fc700 1 -- 172.21.15.3:0/1535913060 learned_addr learned my addr 172.21.15.3:0/1535913060 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:12.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.240+0000 7f6ce2ffd700 1 --2- 172.21.15.3:0/1535913060 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ce406ae80 0x7f6ce40709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:12.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.241+0000 7f6ce27fc700 1 -- 172.21.15.3:0/1535913060 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ce406ae80 msgr2=0x7f6ce40709d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:12.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.241+0000 7f6ce27fc700 1 --2- 172.21.15.3:0/1535913060 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ce406ae80 0x7f6ce40709d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:12.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.241+0000 7f6ce27fc700 1 -- 172.21.15.3:0/1535913060 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ce41316c0 msgr2=0x7f6ce4071450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:12.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.241+0000 7f6ce27fc700 1 --2- 172.21.15.3:0/1535913060 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ce41316c0 0x7f6ce4071450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:12.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.241+0000 7f6ce27fc700 1 -- 172.21.15.3:0/1535913060 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd40094a0 con 0x7f6ce409d9f0 2023-10-03T10:14:12.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.241+0000 7f6ce37fe700 1 --2- 172.21.15.3:0/1535913060 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ce41316c0 0x7f6ce4071450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:12.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.241+0000 7f6ce27fc700 1 --2- 172.21.15.3:0/1535913060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ce409d9f0 0x7f6ce4070f10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6cd800c750 tx=0x7f6cd800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:12.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.241+0000 7f6ccbfff700 1 -- 172.21.15.3:0/1535913060 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8004c80 con 0x7f6ce409d9f0 2023-10-03T10:14:12.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.241+0000 7f6ccbfff700 1 -- 172.21.15.3:0/1535913060 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd8004de0 con 0x7f6ce409d9f0 2023-10-03T10:14:12.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.241+0000 7f6ce991f700 1 -- 172.21.15.3:0/1535913060 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cdc009a70 con 0x7f6ce409d9f0 2023-10-03T10:14:12.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.241+0000 7f6ccbfff700 1 -- 172.21.15.3:0/1535913060 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8007310 con 0x7f6ce409d9f0 2023-10-03T10:14:12.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.241+0000 7f6ce991f700 1 -- 172.21.15.3:0/1535913060 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce412b900 con 0x7f6ce409d9f0 2023-10-03T10:14:12.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.242+0000 7f6ccbfff700 1 -- 172.21.15.3:0/1535913060 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6cd8007470 con 0x7f6ce409d9f0 2023-10-03T10:14:12.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.242+0000 7f6ccbfff700 1 -- 172.21.15.3:0/1535913060 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6cd803f230 con 0x7f6ce409d9f0 2023-10-03T10:14:12.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.242+0000 7f6ce991f700 1 -- 172.21.15.3:0/1535913060 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce409e130 con 0x7f6ce409d9f0 2023-10-03T10:14:12.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.248+0000 7f6ccbfff700 1 -- 172.21.15.3:0/1535913060 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6ce409e130 con 0x7f6ce409d9f0 2023-10-03T10:14:12.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.302+0000 7fa5ea844700 1 Processor -- start 2023-10-03T10:14:12.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.302+0000 7fa5ea844700 1 -- start start 2023-10-03T10:14:12.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.303+0000 7fa5ea844700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa5e406a770 0x7fa5e406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:12.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.303+0000 7fa5ea844700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5e4131d40 0x7fa5e4139aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:12.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.303+0000 7fa5ea844700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5e409f760 con 0x7fa5e406b080 2023-10-03T10:14:12.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.303+0000 7fa5ea844700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e409ee60 con 0x7fa5e4131d40 2023-10-03T10:14:12.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.303+0000 7fa5ea844700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e409e8c0 con 0x7fa5e406a770 2023-10-03T10:14:12.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.303+0000 7fa5e37fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa5e406b080 0x7fa5e4131630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54368/0 (socket says 172.21.15.3:54368) 2023-10-03T10:14:12.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.303+0000 7fa5e8de1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5e4131d40 0x7fa5e4139aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:12.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.303+0000 7fa5e37fe700 1 -- 172.21.15.3:0/2704866229 learned_addr learned my addr 172.21.15.3:0/2704866229 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:12.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.303+0000 7fa5e3fff700 1 --2- 172.21.15.3:0/2704866229 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa5e406a770 0x7fa5e406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:12.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.303+0000 7fa5e2ffd700 1 -- 172.21.15.3:0/2704866229 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3240622568 0 0) 0x7fa5e409f760 con 0x7fa5e406b080 2023-10-03T10:14:12.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.303+0000 7fa5e2ffd700 1 -- 172.21.15.3:0/2704866229 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5cc0035b0 con 0x7fa5e406b080 2023-10-03T10:14:12.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.303+0000 7fa5e8de1700 1 -- 172.21.15.3:0/2704866229 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa5e406a770 msgr2=0x7fa5e406ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:12.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.303+0000 7fa5e8de1700 1 --2- 172.21.15.3:0/2704866229 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa5e406a770 0x7fa5e406ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:12.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.303+0000 7fa5e8de1700 1 -- 172.21.15.3:0/2704866229 >> v1:172.21.15.3:6789/0 conn(0x7fa5e406b080 legacy=0x7fa5e4131630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:12.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.303+0000 7fa5e8de1700 1 -- 172.21.15.3:0/2704866229 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e409cf10 con 0x7fa5e4131d40 2023-10-03T10:14:12.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.303+0000 7fa5e3fff700 1 --2- 172.21.15.3:0/2704866229 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa5e406a770 0x7fa5e406ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:14:12.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.304+0000 7fa5e8de1700 1 --2- 172.21.15.3:0/2704866229 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5e4131d40 0x7fa5e4139aa0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc009dc0 tx=0x7fa5dc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95e42247917ee476 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:12.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.304+0000 7fa5e2ffd700 1 -- 172.21.15.3:0/2704866229 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc00e070 con 0x7fa5e4131d40 2023-10-03T10:14:12.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.304+0000 7fa5e2ffd700 1 -- 172.21.15.3:0/2704866229 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5dc00bb20 con 0x7fa5e4131d40 2023-10-03T10:14:12.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.304+0000 7fa5e2ffd700 1 -- 172.21.15.3:0/2704866229 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc012430 con 0x7fa5e4131d40 2023-10-03T10:14:12.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.304+0000 7fa5ea844700 1 -- 172.21.15.3:0/2704866229 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5e4131d40 msgr2=0x7fa5e4139aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:12.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.304+0000 7fa5ea844700 1 --2- 172.21.15.3:0/2704866229 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5e4131d40 0x7fa5e4139aa0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc009dc0 tx=0x7fa5dc00b330 comp rx=0 tx=0).stop 2023-10-03T10:14:12.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.305+0000 7fa5ea844700 1 -- 172.21.15.3:0/2704866229 shutdown_connections 2023-10-03T10:14:12.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.305+0000 7fa5ea844700 1 --2- 172.21.15.3:0/2704866229 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5e4131d40 0x7fa5e4139aa0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:12.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.305+0000 7fa5ea844700 1 --2- 172.21.15.3:0/2704866229 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa5e406a770 0x7fa5e406ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:12.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.305+0000 7fa5ea844700 1 -- 172.21.15.3:0/2704866229 >> 172.21.15.3:0/2704866229 conn(0x7fa5e4066f50 msgr2=0x7fa5e40a2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:12.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.305+0000 7fa5ea844700 1 -- 172.21.15.3:0/2704866229 shutdown_connections 2023-10-03T10:14:12.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.305+0000 7fa5ea844700 1 -- 172.21.15.3:0/2704866229 wait complete. 2023-10-03T10:14:12.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.306+0000 7fa5ea844700 1 Processor -- start 2023-10-03T10:14:12.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.306+0000 7fa5ea844700 1 -- start start 2023-10-03T10:14:12.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.306+0000 7fa5ea844700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5e406a770 0x7fa5e411ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:12.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.306+0000 7fa5ea844700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e406b080 0x7fa5e411f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:12.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.306+0000 7fa5ea844700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa5e4131d40 0x7fa5e411f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:12.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.306+0000 7fa5ea844700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e41229e0 con 0x7fa5e406b080 2023-10-03T10:14:12.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.306+0000 7fa5ea844700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e4121960 con 0x7fa5e406a770 2023-10-03T10:14:12.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.306+0000 7fa5ea844700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e4071ed0 con 0x7fa5e4131d40 2023-10-03T10:14:12.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.306+0000 7fa5e3fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5e406a770 0x7fa5e411ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:12.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.306+0000 7fa5e3fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5e406a770 0x7fa5e411ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40132/0 (socket says 172.21.15.3:40132) 2023-10-03T10:14:12.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.306+0000 7fa5e3fff700 1 -- 172.21.15.3:0/4036270728 learned_addr learned my addr 172.21.15.3:0/4036270728 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:12.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.307+0000 7fa5e37fe700 1 --2- 172.21.15.3:0/4036270728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e406b080 0x7fa5e411f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:12.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.307+0000 7fa5e8de1700 1 --2- 172.21.15.3:0/4036270728 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa5e4131d40 0x7fa5e411f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:12.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.307+0000 7fa5e3fff700 1 -- 172.21.15.3:0/4036270728 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa5e4131d40 msgr2=0x7fa5e411f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:12.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.307+0000 7fa5e3fff700 1 --2- 172.21.15.3:0/4036270728 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa5e4131d40 0x7fa5e411f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:12.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.307+0000 7fa5e3fff700 1 -- 172.21.15.3:0/4036270728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e406b080 msgr2=0x7fa5e411f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:12.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.307+0000 7fa5e3fff700 1 --2- 172.21.15.3:0/4036270728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e406b080 0x7fa5e411f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:12.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.307+0000 7fa5e3fff700 1 -- 172.21.15.3:0/4036270728 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5dc009a70 con 0x7fa5e406a770 2023-10-03T10:14:12.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.307+0000 7fa5e37fe700 1 --2- 172.21.15.3:0/4036270728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e406b080 0x7fa5e411f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:12.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.307+0000 7fa5e3fff700 1 --2- 172.21.15.3:0/4036270728 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5e406a770 0x7fa5e411ef50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa5d400e870 tx=0x7fa5d400eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:12.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.307+0000 7fa5e0ff9700 1 -- 172.21.15.3:0/4036270728 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4013610 con 0x7fa5e406a770 2023-10-03T10:14:12.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.307+0000 7fa5e0ff9700 1 -- 172.21.15.3:0/4036270728 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d4013770 con 0x7fa5e406a770 2023-10-03T10:14:12.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.307+0000 7fa5ea844700 1 -- 172.21.15.3:0/4036270728 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5e4122170 con 0x7fa5e406a770 2023-10-03T10:14:12.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.307+0000 7fa5e0ff9700 1 -- 172.21.15.3:0/4036270728 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4013b20 con 0x7fa5e406a770 2023-10-03T10:14:12.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.307+0000 7fa5ea844700 1 -- 172.21.15.3:0/4036270728 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e4122440 con 0x7fa5e406a770 2023-10-03T10:14:12.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.308+0000 7fa5ea844700 1 -- 172.21.15.3:0/4036270728 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5e411bc60 con 0x7fa5e406a770 2023-10-03T10:14:12.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.309+0000 7fa5e0ff9700 1 -- 172.21.15.3:0/4036270728 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa5d4013c80 con 0x7fa5e406a770 2023-10-03T10:14:12.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.309+0000 7fa5e0ff9700 1 -- 172.21.15.3:0/4036270728 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa5d403f360 con 0x7fa5e406a770 2023-10-03T10:14:12.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.314+0000 7fa5e0ff9700 1 -- 172.21.15.3:0/4036270728 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa5d4012370 con 0x7fa5e406a770 2023-10-03T10:14:12.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.380+0000 7fe1717fa700 1 -- 172.21.15.3:0/2471616696 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe154005590 con 0x7fe17c1337f0 2023-10-03T10:14:12.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.381+0000 7fe1737fe700 1 -- 172.21.15.3:0/2471616696 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe16c02b030 con 0x7fe17c1337f0 2023-10-03T10:14:12.381 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:12.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.383+0000 7fe1717fa700 1 -- 172.21.15.3:0/2471616696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe17c1337f0 msgr2=0x7fe17c12f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:12.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.383+0000 7fe1717fa700 1 --2- 172.21.15.3:0/2471616696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe17c1337f0 0x7fe17c12f390 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe16c016000 tx=0x7fe16c004750 comp rx=0 tx=0).stop 2023-10-03T10:14:12.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.383+0000 7fe1717fa700 1 -- 172.21.15.3:0/2471616696 shutdown_connections 2023-10-03T10:14:12.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.383+0000 7fe1717fa700 1 --2- 172.21.15.3:0/2471616696 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe17c138dc0 0x7fe17c12f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:12.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.383+0000 7fe1717fa700 1 --2- 172.21.15.3:0/2471616696 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe17c133420 0x7fe17c12ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:12.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.383+0000 7fe1717fa700 1 --2- 172.21.15.3:0/2471616696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe17c1337f0 0x7fe17c12f390 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:12.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.383+0000 7fe1717fa700 1 -- 172.21.15.3:0/2471616696 >> 172.21.15.3:0/2471616696 conn(0x7fe17c0a4210 msgr2=0x7fe17c0a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:12.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.383+0000 7fe1717fa700 1 -- 172.21.15.3:0/2471616696 shutdown_connections 2023-10-03T10:14:12.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.383+0000 7fe1717fa700 1 -- 172.21.15.3:0/2471616696 wait complete. 2023-10-03T10:14:12.397 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:12.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.460+0000 7f6ce991f700 1 -- 172.21.15.3:0/1535913060 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6ce409e130 con 0x7f6ce409d9f0 2023-10-03T10:14:12.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.460+0000 7f6ccbfff700 1 -- 172.21.15.3:0/1535913060 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6ce409e130 con 0x7f6ce409d9f0 2023-10-03T10:14:12.461 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:12.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.463+0000 7f6ce991f700 1 -- 172.21.15.3:0/1535913060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ce409d9f0 msgr2=0x7f6ce4070f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:12.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.463+0000 7f6ce991f700 1 --2- 172.21.15.3:0/1535913060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ce409d9f0 0x7f6ce4070f10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6cd800c750 tx=0x7f6cd800ca60 comp rx=0 tx=0).stop 2023-10-03T10:14:12.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.463+0000 7f6ce991f700 1 -- 172.21.15.3:0/1535913060 shutdown_connections 2023-10-03T10:14:12.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.463+0000 7f6ce991f700 1 --2- 172.21.15.3:0/1535913060 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ce41316c0 0x7f6ce4071450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:12.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.463+0000 7f6ce991f700 1 --2- 172.21.15.3:0/1535913060 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ce406ae80 0x7f6ce40709d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:12.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.463+0000 7f6ce991f700 1 --2- 172.21.15.3:0/1535913060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ce409d9f0 0x7f6ce4070f10 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:12.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.463+0000 7f6ce991f700 1 -- 172.21.15.3:0/1535913060 >> 172.21.15.3:0/1535913060 conn(0x7f6ce4066f50 msgr2=0x7f6ce40a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:12.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.463+0000 7f6ce991f700 1 -- 172.21.15.3:0/1535913060 shutdown_connections 2023-10-03T10:14:12.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.463+0000 7f6ce991f700 1 -- 172.21.15.3:0/1535913060 wait complete. 2023-10-03T10:14:12.477 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:12.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.519+0000 7fa5ea844700 1 -- 172.21.15.3:0/4036270728 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa5e40747f0 con 0x7fa5e406a770 2023-10-03T10:14:12.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.520+0000 7fa5e0ff9700 1 -- 172.21.15.3:0/4036270728 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa5d4010350 con 0x7fa5e406a770 2023-10-03T10:14:12.520 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:12.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.522+0000 7fa5ea844700 1 -- 172.21.15.3:0/4036270728 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5e406a770 msgr2=0x7fa5e411ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:12.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.522+0000 7fa5ea844700 1 --2- 172.21.15.3:0/4036270728 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5e406a770 0x7fa5e411ef50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa5d400e870 tx=0x7fa5d400eb80 comp rx=0 tx=0).stop 2023-10-03T10:14:12.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.522+0000 7fa5ea844700 1 -- 172.21.15.3:0/4036270728 shutdown_connections 2023-10-03T10:14:12.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.522+0000 7fa5ea844700 1 --2- 172.21.15.3:0/4036270728 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa5e4131d40 0x7fa5e411f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:12.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.522+0000 7fa5ea844700 1 --2- 172.21.15.3:0/4036270728 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5e406a770 0x7fa5e411ef50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:12.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.522+0000 7fa5ea844700 1 --2- 172.21.15.3:0/4036270728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e406b080 0x7fa5e411f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:12.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.522+0000 7fa5ea844700 1 -- 172.21.15.3:0/4036270728 >> 172.21.15.3:0/4036270728 conn(0x7fa5e4066f50 msgr2=0x7fa5e4130ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:12.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.522+0000 7fa5ea844700 1 -- 172.21.15.3:0/4036270728 shutdown_connections 2023-10-03T10:14:12.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:12.522+0000 7fa5ea844700 1 -- 172.21.15.3:0/4036270728 wait complete. 2023-10-03T10:14:12.536 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:12.871 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:13.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.017+0000 7f57a7c49700 1 Processor -- start 2023-10-03T10:14:13.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.018+0000 7f57a7c49700 1 -- start start 2023-10-03T10:14:13.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.018+0000 7f57a7c49700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57a009d6f0 0x7f57a009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:13.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.018+0000 7f57a7c49700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57a009e000 0x7f57a01339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:13.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.018+0000 7f57a7c49700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f57a009e410 0x7f57a0133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:13.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.018+0000 7f57a7c49700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a006b460 con 0x7f57a009e410 2023-10-03T10:14:13.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.018+0000 7f57a7c49700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a009b120 con 0x7f57a009e000 2023-10-03T10:14:13.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.018+0000 7f57a7c49700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f57a009a7e0 con 0x7f57a009d6f0 2023-10-03T10:14:13.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.018+0000 7f57a61e6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f57a009e410 0x7f57a0133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:13.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.018+0000 7f57a59e5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57a009d6f0 0x7f57a009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:13.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.018+0000 7f57a61e6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f57a009e410 0x7f57a0133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58088/0 (socket says 172.21.15.3:58088) 2023-10-03T10:14:13.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.018+0000 7f57a51e4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57a009e000 0x7f57a01339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:13.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.018+0000 7f57a61e6700 1 -- 172.21.15.3:0/1348056783 learned_addr learned my addr 172.21.15.3:0/1348056783 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:13.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.019+0000 7f57a61e6700 1 -- 172.21.15.3:0/1348056783 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57a009d6f0 msgr2=0x7f57a009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:13.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.019+0000 7f57a61e6700 1 --2- 172.21.15.3:0/1348056783 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57a009d6f0 0x7f57a009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.019+0000 7f57a61e6700 1 -- 172.21.15.3:0/1348056783 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57a009e000 msgr2=0x7f57a01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:13.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.019+0000 7f57a61e6700 1 --2- 172.21.15.3:0/1348056783 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57a009e000 0x7f57a01339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.019+0000 7f57a61e6700 1 -- 172.21.15.3:0/1348056783 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a009a500 con 0x7f57a009e410 2023-10-03T10:14:13.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.019+0000 7f57a51e4700 1 --2- 172.21.15.3:0/1348056783 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57a009e000 0x7f57a01339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:13.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.019+0000 7f57a61e6700 1 --2- 172.21.15.3:0/1348056783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a009e410 0x7f57a0133f30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5798009e50 tx=0x7f579800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fdb62021893158f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:13.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.019+0000 7f57a49e3700 1 -- 172.21.15.3:0/1348056783 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579800e070 con 0x7f57a009e410 2023-10-03T10:14:13.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.019+0000 7f57a49e3700 1 -- 172.21.15.3:0/1348056783 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579800b910 con 0x7f57a009e410 2023-10-03T10:14:13.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.019+0000 7f57a49e3700 1 -- 172.21.15.3:0/1348056783 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579800bcc0 con 0x7f57a009e410 2023-10-03T10:14:13.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.020+0000 7f57a7c49700 1 -- 172.21.15.3:0/1348056783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a009e410 msgr2=0x7f57a0133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:13.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.020+0000 7f57a7c49700 1 --2- 172.21.15.3:0/1348056783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a009e410 0x7f57a0133f30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5798009e50 tx=0x7f579800b330 comp rx=0 tx=0).stop 2023-10-03T10:14:13.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.020+0000 7f57a7c49700 1 -- 172.21.15.3:0/1348056783 shutdown_connections 2023-10-03T10:14:13.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.020+0000 7f57a7c49700 1 --2- 172.21.15.3:0/1348056783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a009e410 0x7f57a0133f30 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.020+0000 7f57a7c49700 1 --2- 172.21.15.3:0/1348056783 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57a009d6f0 0x7f57a009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.020+0000 7f57a7c49700 1 --2- 172.21.15.3:0/1348056783 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57a009e000 0x7f57a01339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.020+0000 7f57a7c49700 1 -- 172.21.15.3:0/1348056783 >> 172.21.15.3:0/1348056783 conn(0x7f57a0066f50 msgr2=0x7f57a009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:13.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.020+0000 7f57a7c49700 1 -- 172.21.15.3:0/1348056783 shutdown_connections 2023-10-03T10:14:13.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.020+0000 7f57a7c49700 1 -- 172.21.15.3:0/1348056783 wait complete. 2023-10-03T10:14:13.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.021+0000 7f57a7c49700 1 Processor -- start 2023-10-03T10:14:13.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.021+0000 7f57a7c49700 1 -- start start 2023-10-03T10:14:13.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.021+0000 7f57a7c49700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a009d6f0 0x7f57a01a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:13.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.021+0000 7f57a7c49700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57a009e000 0x7f57a01a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:13.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.021+0000 7f57a7c49700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57a009e410 0x7f57a01c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:13.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.021+0000 7f57a7c49700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a01a5c90 con 0x7f57a009d6f0 2023-10-03T10:14:13.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.021+0000 7f57a7c49700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a01b9140 con 0x7f57a009e410 2023-10-03T10:14:13.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.021+0000 7f57a7c49700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f57a01a4d20 con 0x7f57a009e000 2023-10-03T10:14:13.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.022+0000 7f57a59e5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a009d6f0 0x7f57a01a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:13.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.022+0000 7f57a59e5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a009d6f0 0x7f57a01a5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58092/0 (socket says 172.21.15.3:58092) 2023-10-03T10:14:13.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.022+0000 7f57a61e6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57a009e410 0x7f57a01c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:13.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.022+0000 7f57a59e5700 1 -- 172.21.15.3:0/3364414824 learned_addr learned my addr 172.21.15.3:0/3364414824 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:13.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.022+0000 7f57a51e4700 1 --2- 172.21.15.3:0/3364414824 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57a009e000 0x7f57a01a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:13.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.022+0000 7f57a59e5700 1 -- 172.21.15.3:0/3364414824 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57a009e000 msgr2=0x7f57a01a5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:13.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.022+0000 7f57a59e5700 1 --2- 172.21.15.3:0/3364414824 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57a009e000 0x7f57a01a5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.022+0000 7f57a59e5700 1 -- 172.21.15.3:0/3364414824 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57a009e410 msgr2=0x7f57a01c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:13.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.022+0000 7f57a59e5700 1 --2- 172.21.15.3:0/3364414824 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57a009e410 0x7f57a01c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.022+0000 7f57a59e5700 1 -- 172.21.15.3:0/3364414824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57900094a0 con 0x7f57a009d6f0 2023-10-03T10:14:13.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.022+0000 7f57a61e6700 1 --2- 172.21.15.3:0/3364414824 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57a009e410 0x7f57a01c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:13.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.022+0000 7f57a59e5700 1 --2- 172.21.15.3:0/3364414824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a009d6f0 0x7f57a01a5210 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f579c00e9b0 tx=0x7f579c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:13.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.022+0000 7f57967fc700 1 -- 172.21.15.3:0/3364414824 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c009710 con 0x7f57a009d6f0 2023-10-03T10:14:13.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.022+0000 7f57967fc700 1 -- 172.21.15.3:0/3364414824 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579c00ee50 con 0x7f57a009d6f0 2023-10-03T10:14:13.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.022+0000 7f57967fc700 1 -- 172.21.15.3:0/3364414824 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c00ca50 con 0x7f57a009d6f0 2023-10-03T10:14:13.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.022+0000 7f57a7c49700 1 -- 172.21.15.3:0/3364414824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5798009b30 con 0x7f57a009d6f0 2023-10-03T10:14:13.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.022+0000 7f57a7c49700 1 -- 172.21.15.3:0/3364414824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a01a8420 con 0x7f57a009d6f0 2023-10-03T10:14:13.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.023+0000 7f57967fc700 1 -- 172.21.15.3:0/3364414824 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f579c00cbb0 con 0x7f57a009d6f0 2023-10-03T10:14:13.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.023+0000 7f57967fc700 1 -- 172.21.15.3:0/3364414824 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f579c03faa0 con 0x7f57a009d6f0 2023-10-03T10:14:13.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.023+0000 7f57a7c49700 1 -- 172.21.15.3:0/3364414824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a01a7e50 con 0x7f57a009d6f0 2023-10-03T10:14:13.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.028+0000 7f57967fc700 1 -- 172.21.15.3:0/3364414824 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f57a01a7e50 con 0x7f57a009d6f0 2023-10-03T10:14:13.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.232+0000 7f57a7c49700 1 -- 172.21.15.3:0/3364414824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f57a01a7e50 con 0x7f57a009d6f0 2023-10-03T10:14:13.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.233+0000 7f57967fc700 1 -- 172.21.15.3:0/3364414824 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f57a01a7e50 con 0x7f57a009d6f0 2023-10-03T10:14:13.233 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:13.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.235+0000 7f57a7c49700 1 -- 172.21.15.3:0/3364414824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a009d6f0 msgr2=0x7f57a01a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:13.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.235+0000 7f57a7c49700 1 --2- 172.21.15.3:0/3364414824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a009d6f0 0x7f57a01a5210 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f579c00e9b0 tx=0x7f579c00ecc0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.235+0000 7f57a7c49700 1 -- 172.21.15.3:0/3364414824 shutdown_connections 2023-10-03T10:14:13.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.235+0000 7f57a7c49700 1 --2- 172.21.15.3:0/3364414824 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f57a009e410 0x7f57a01c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.235+0000 7f57a7c49700 1 --2- 172.21.15.3:0/3364414824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a009d6f0 0x7f57a01a5210 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.235+0000 7f57a7c49700 1 --2- 172.21.15.3:0/3364414824 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57a009e000 0x7f57a01a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.235+0000 7f57a7c49700 1 -- 172.21.15.3:0/3364414824 >> 172.21.15.3:0/3364414824 conn(0x7f57a0066f50 msgr2=0x7f57a01bbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:13.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.235+0000 7f57a7c49700 1 -- 172.21.15.3:0/3364414824 shutdown_connections 2023-10-03T10:14:13.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.235+0000 7f57a7c49700 1 -- 172.21.15.3:0/3364414824 wait complete. 2023-10-03T10:14:13.249 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:13.399 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:13.477 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:13.536 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:13.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.547+0000 7f4ae1ac6700 1 Processor -- start 2023-10-03T10:14:13.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.547+0000 7f4ae1ac6700 1 -- start start 2023-10-03T10:14:13.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.548+0000 7f4ae1ac6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4adc09d6f0 0x7f4adc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:13.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.548+0000 7f4ae1ac6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4adc09e440 0x7f4adc134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:13.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.548+0000 7f4ae1ac6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4adc09b120 con 0x7f4adc09e000 2023-10-03T10:14:13.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.548+0000 7f4ae1ac6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4adc06b460 con 0x7f4adc09d6f0 2023-10-03T10:14:13.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.548+0000 7f4ae1ac6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc09a7e0 con 0x7f4adc09e440 2023-10-03T10:14:13.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.548+0000 7f4adb7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4adc09d6f0 0x7f4adc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:13.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.548+0000 7f4adb7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4adc09d6f0 0x7f4adc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40766/0 (socket says 172.21.15.3:40766) 2023-10-03T10:14:13.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.548+0000 7f4adb7fe700 1 -- 172.21.15.3:0/3924361050 learned_addr learned my addr 172.21.15.3:0/3924361050 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:13.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.548+0000 7f4adbfff700 1 --2- 172.21.15.3:0/3924361050 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4adc09e440 0x7f4adc134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:13.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.548+0000 7f4adbfff700 1 -- 172.21.15.3:0/3924361050 >> v2:172.21.15.3:3300/0 conn(0x7f4adc09d6f0 msgr2=0x7f4adc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:13.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.548+0000 7f4adbfff700 1 --2- 172.21.15.3:0/3924361050 >> v2:172.21.15.3:3300/0 conn(0x7f4adc09d6f0 0x7f4adc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.548+0000 7f4adbfff700 1 -- 172.21.15.3:0/3924361050 >> v1:172.21.15.3:6789/0 conn(0x7f4adc09e000 legacy=0x7f4adc1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:13.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.548+0000 7f4adbfff700 1 -- 172.21.15.3:0/3924361050 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc09a500 con 0x7f4adc09e440 2023-10-03T10:14:13.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.549+0000 7f4adbfff700 1 --2- 172.21.15.3:0/3924361050 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4adc09e440 0x7f4adc134100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009dc0 tx=0x7f4ad400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c82da3bd382044 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:13.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.549+0000 7f4ada7fc700 1 -- 172.21.15.3:0/3924361050 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad400e070 con 0x7f4adc09e440 2023-10-03T10:14:13.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.549+0000 7f4ada7fc700 1 -- 172.21.15.3:0/3924361050 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad400baa0 con 0x7f4adc09e440 2023-10-03T10:14:13.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.549+0000 7f4ada7fc700 1 -- 172.21.15.3:0/3924361050 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad400be50 con 0x7f4adc09e440 2023-10-03T10:14:13.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.550+0000 7f4ae1ac6700 1 -- 172.21.15.3:0/3924361050 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4adc09e440 msgr2=0x7f4adc134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:13.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.550+0000 7f4ae1ac6700 1 --2- 172.21.15.3:0/3924361050 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4adc09e440 0x7f4adc134100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009dc0 tx=0x7f4ad400b330 comp rx=0 tx=0).stop 2023-10-03T10:14:13.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.550+0000 7f4ae1ac6700 1 -- 172.21.15.3:0/3924361050 shutdown_connections 2023-10-03T10:14:13.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.550+0000 7f4ae1ac6700 1 --2- 172.21.15.3:0/3924361050 >> v2:172.21.15.3:3300/0 conn(0x7f4adc09d6f0 0x7f4adc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.550+0000 7f4ae1ac6700 1 --2- 172.21.15.3:0/3924361050 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4adc09e440 0x7f4adc134100 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.550+0000 7f4ae1ac6700 1 -- 172.21.15.3:0/3924361050 >> 172.21.15.3:0/3924361050 conn(0x7f4adc066f50 msgr2=0x7f4adc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:13.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.550+0000 7f4ae1ac6700 1 -- 172.21.15.3:0/3924361050 shutdown_connections 2023-10-03T10:14:13.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.550+0000 7f4ae1ac6700 1 -- 172.21.15.3:0/3924361050 wait complete. 2023-10-03T10:14:13.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.551+0000 7f4ae1ac6700 1 Processor -- start 2023-10-03T10:14:13.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.551+0000 7f4ae1ac6700 1 -- start start 2023-10-03T10:14:13.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.552+0000 7f4ae1ac6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4adc09d6f0 0x7f4adc1b91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:13.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.552+0000 7f4ae1ac6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4adc09e000 0x7f4adc1a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:13.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.552+0000 7f4ae1ac6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4adc09e440 0x7f4adc1a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:13.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.552+0000 7f4ae1ac6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc1a96b0 con 0x7f4adc09e440 2023-10-03T10:14:13.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.552+0000 7f4ae1ac6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc1a9900 con 0x7f4adc09e000 2023-10-03T10:14:13.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.552+0000 7f4ae1ac6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc09c6d0 con 0x7f4adc09d6f0 2023-10-03T10:14:13.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.552+0000 7f4adaffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4adc09e000 0x7f4adc1a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:13.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.552+0000 7f4adbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4adc09e440 0x7f4adc1a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:13.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.552+0000 7f4adaffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4adc09e000 0x7f4adc1a5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:55950/0 (socket says 172.21.15.3:55950) 2023-10-03T10:14:13.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.552+0000 7f4adaffd700 1 -- 172.21.15.3:0/1045929815 learned_addr learned my addr 172.21.15.3:0/1045929815 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:13.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.552+0000 7f4adb7fe700 1 --2- 172.21.15.3:0/1045929815 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4adc09d6f0 0x7f4adc1b91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:13.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.553+0000 7f4adaffd700 1 -- 172.21.15.3:0/1045929815 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4adc09d6f0 msgr2=0x7f4adc1b91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:13.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.553+0000 7f4adaffd700 1 --2- 172.21.15.3:0/1045929815 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4adc09d6f0 0x7f4adc1b91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.553+0000 7f4adaffd700 1 -- 172.21.15.3:0/1045929815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4adc09e440 msgr2=0x7f4adc1a6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:13.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.553+0000 7f4adaffd700 1 --2- 172.21.15.3:0/1045929815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4adc09e440 0x7f4adc1a6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.553+0000 7f4adaffd700 1 -- 172.21.15.3:0/1045929815 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad4009a70 con 0x7f4adc09e000 2023-10-03T10:14:13.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.553+0000 7f4adbfff700 1 --2- 172.21.15.3:0/1045929815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4adc09e440 0x7f4adc1a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:13.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.553+0000 7f4adaffd700 1 --2- 172.21.15.3:0/1045929815 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4adc09e000 0x7f4adc1a5e30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000c4d0 tx=0x7f4ad000c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:13.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.553+0000 7f4ac3fff700 1 -- 172.21.15.3:0/1045929815 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0011790 con 0x7f4adc09e000 2023-10-03T10:14:13.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.553+0000 7f4ac3fff700 1 -- 172.21.15.3:0/1045929815 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad00118f0 con 0x7f4adc09e000 2023-10-03T10:14:13.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.553+0000 7f4ae1ac6700 1 -- 172.21.15.3:0/1045929815 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4adc0a1e90 con 0x7f4adc09e000 2023-10-03T10:14:13.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.553+0000 7f4ac3fff700 1 -- 172.21.15.3:0/1045929815 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0011ca0 con 0x7f4adc09e000 2023-10-03T10:14:13.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.553+0000 7f4ae1ac6700 1 -- 172.21.15.3:0/1045929815 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4adc1a8c40 con 0x7f4adc09e000 2023-10-03T10:14:13.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.554+0000 7f4ac3fff700 1 -- 172.21.15.3:0/1045929815 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4ad001c460 con 0x7f4adc09e000 2023-10-03T10:14:13.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.554+0000 7f4ae1ac6700 1 -- 172.21.15.3:0/1045929815 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4adc074ea0 con 0x7f4adc09e000 2023-10-03T10:14:13.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.555+0000 7f4ac3fff700 1 -- 172.21.15.3:0/1045929815 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4ad003f3a0 con 0x7f4adc09e000 2023-10-03T10:14:13.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.559+0000 7f4ac3fff700 1 -- 172.21.15.3:0/1045929815 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4ad00108d0 con 0x7f4adc09e000 2023-10-03T10:14:13.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.620+0000 7f95e4df4700 1 Processor -- start 2023-10-03T10:14:13.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.620+0000 7f95e4df4700 1 -- start start 2023-10-03T10:14:13.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.621+0000 7f95e4df4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f95e00a0a30 0x7f95e00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:13.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.621+0000 7f95e4df4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f95e009d630 0x7f95e00a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:13.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.622+0000 7f95e4df4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e00a2a70 con 0x7f95e0133930 2023-10-03T10:14:13.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.622+0000 7f95e4df4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e00a01d0 con 0x7f95e00a0a30 2023-10-03T10:14:13.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.622+0000 7f95e4df4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95e009e2b0 con 0x7f95e009d630 2023-10-03T10:14:13.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.622+0000 7f95de59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f95e00a0a30 0x7f95e00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:13.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.622+0000 7f95ddd9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f95e009d630 0x7f95e00a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:13.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.622+0000 7f95de59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f95e00a0a30 0x7f95e00a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40786/0 (socket says 172.21.15.3:40786) 2023-10-03T10:14:13.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.622+0000 7f95de59c700 1 -- 172.21.15.3:0/2347205818 learned_addr learned my addr 172.21.15.3:0/2347205818 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:13.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.622+0000 7f95ddd9b700 1 -- 172.21.15.3:0/2347205818 >> v2:172.21.15.3:3300/0 conn(0x7f95e00a0a30 msgr2=0x7f95e00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:13.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.622+0000 7f95ddd9b700 1 --2- 172.21.15.3:0/2347205818 >> v2:172.21.15.3:3300/0 conn(0x7f95e00a0a30 0x7f95e00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.622+0000 7f95ddd9b700 1 -- 172.21.15.3:0/2347205818 >> v1:172.21.15.3:6789/0 conn(0x7f95e0133930 legacy=0x7f95e0133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:13.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.622+0000 7f95ddd9b700 1 -- 172.21.15.3:0/2347205818 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e009dc50 con 0x7f95e009d630 2023-10-03T10:14:13.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.622+0000 7f95de59c700 1 --2- 172.21.15.3:0/2347205818 >> v2:172.21.15.3:3300/0 conn(0x7f95e00a0a30 0x7f95e00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:13.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.622+0000 7f95ddd9b700 1 --2- 172.21.15.3:0/2347205818 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f95e009d630 0x7f95e00a1340 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009840 tx=0x7f95d4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73271441c8ab5f71 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:13.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.622+0000 7f95dd59a700 1 -- 172.21.15.3:0/2347205818 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d4011830 con 0x7f95e009d630 2023-10-03T10:14:13.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.623+0000 7f95dd59a700 1 -- 172.21.15.3:0/2347205818 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d4011990 con 0x7f95e009d630 2023-10-03T10:14:13.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.623+0000 7f95dd59a700 1 -- 172.21.15.3:0/2347205818 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d4011d20 con 0x7f95e009d630 2023-10-03T10:14:13.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.623+0000 7f95e4df4700 1 -- 172.21.15.3:0/2347205818 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f95e009d630 msgr2=0x7f95e00a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:13.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.623+0000 7f95e4df4700 1 --2- 172.21.15.3:0/2347205818 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f95e009d630 0x7f95e00a1340 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009840 tx=0x7f95d4009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:13.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.623+0000 7f95e4df4700 1 -- 172.21.15.3:0/2347205818 shutdown_connections 2023-10-03T10:14:13.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.623+0000 7f95e4df4700 1 --2- 172.21.15.3:0/2347205818 >> v2:172.21.15.3:3300/0 conn(0x7f95e00a0a30 0x7f95e00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.623+0000 7f95e4df4700 1 --2- 172.21.15.3:0/2347205818 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f95e009d630 0x7f95e00a1340 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.623+0000 7f95e4df4700 1 -- 172.21.15.3:0/2347205818 >> 172.21.15.3:0/2347205818 conn(0x7f95e00a3050 msgr2=0x7f95e00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:13.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.624+0000 7f95e4df4700 1 -- 172.21.15.3:0/2347205818 shutdown_connections 2023-10-03T10:14:13.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.624+0000 7f95e4df4700 1 -- 172.21.15.3:0/2347205818 wait complete. 2023-10-03T10:14:13.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.625+0000 7f95e4df4700 1 Processor -- start 2023-10-03T10:14:13.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.625+0000 7f95e4df4700 1 -- start start 2023-10-03T10:14:13.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.625+0000 7f95e4df4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f95e009d630 0x7f95e0121100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:13.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.625+0000 7f95e4df4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f95e00a0a30 0x7f95e0121640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:13.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.625+0000 7f95e4df4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95e0133930 0x7f95e0121b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:13.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.625+0000 7f95e4df4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e01225b0 con 0x7f95e0133930 2023-10-03T10:14:13.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.625+0000 7f95e4df4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e0124bd0 con 0x7f95e009d630 2023-10-03T10:14:13.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.625+0000 7f95e4df4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95e0124630 con 0x7f95e00a0a30 2023-10-03T10:14:13.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.625+0000 7f95ddd9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f95e00a0a30 0x7f95e0121640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:13.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.626+0000 7f95ddd9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f95e00a0a30 0x7f95e0121640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52554/0 (socket says 172.21.15.3:52554) 2023-10-03T10:14:13.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.626+0000 7f95ddd9b700 1 -- 172.21.15.3:0/2316510515 learned_addr learned my addr 172.21.15.3:0/2316510515 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:13.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.626+0000 7f95de59c700 1 --2- 172.21.15.3:0/2316510515 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f95e009d630 0x7f95e0121100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:13.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.626+0000 7f95ded9d700 1 --2- 172.21.15.3:0/2316510515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95e0133930 0x7f95e0121b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:13.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.626+0000 7f95ddd9b700 1 -- 172.21.15.3:0/2316510515 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f95e009d630 msgr2=0x7f95e0121100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:13.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.626+0000 7f95ddd9b700 1 --2- 172.21.15.3:0/2316510515 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f95e009d630 0x7f95e0121100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.626+0000 7f95ddd9b700 1 -- 172.21.15.3:0/2316510515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95e0133930 msgr2=0x7f95e0121b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:13.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.626+0000 7f95ddd9b700 1 --2- 172.21.15.3:0/2316510515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95e0133930 0x7f95e0121b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.626+0000 7f95ddd9b700 1 -- 172.21.15.3:0/2316510515 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d0009550 con 0x7f95e00a0a30 2023-10-03T10:14:13.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.626+0000 7f95ddd9b700 1 --2- 172.21.15.3:0/2316510515 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f95e00a0a30 0x7f95e0121640 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009fd0 tx=0x7f95d4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:13.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.626+0000 7f95ceffd700 1 -- 172.21.15.3:0/2316510515 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d40036a0 con 0x7f95e00a0a30 2023-10-03T10:14:13.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.626+0000 7f95ceffd700 1 -- 172.21.15.3:0/2316510515 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d401f070 con 0x7f95e00a0a30 2023-10-03T10:14:13.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.626+0000 7f95e4df4700 1 -- 172.21.15.3:0/2316510515 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d40094a0 con 0x7f95e00a0a30 2023-10-03T10:14:13.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.626+0000 7f95ceffd700 1 -- 172.21.15.3:0/2316510515 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d4003a60 con 0x7f95e00a0a30 2023-10-03T10:14:13.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.627+0000 7f95e4df4700 1 -- 172.21.15.3:0/2316510515 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e00a0410 con 0x7f95e00a0a30 2023-10-03T10:14:13.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.627+0000 7f95e4df4700 1 -- 172.21.15.3:0/2316510515 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e0124850 con 0x7f95e00a0a30 2023-10-03T10:14:13.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.628+0000 7f95ceffd700 1 -- 172.21.15.3:0/2316510515 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f95d4003bc0 con 0x7f95e00a0a30 2023-10-03T10:14:13.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.629+0000 7f95ceffd700 1 -- 172.21.15.3:0/2316510515 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f95d4021070 con 0x7f95e00a0a30 2023-10-03T10:14:13.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.633+0000 7f95ceffd700 1 -- 172.21.15.3:0/2316510515 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f95d4025db0 con 0x7f95e00a0a30 2023-10-03T10:14:13.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.688+0000 7f5d1205e700 1 Processor -- start 2023-10-03T10:14:13.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.688+0000 7f5d1205e700 1 -- start start 2023-10-03T10:14:13.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.689+0000 7f5d1205e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d0c0a0a30 0x7f5d0c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:13.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.689+0000 7f5d1205e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5d0c0a1340 0x7f5d0c133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:13.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.689+0000 7f5d1205e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d0c09edd0 con 0x7f5d0c133fa0 2023-10-03T10:14:13.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.689+0000 7f5d1205e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d0c09e1f0 con 0x7f5d0c0a1340 2023-10-03T10:14:13.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.689+0000 7f5d1205e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c09dc50 con 0x7f5d0c0a0a30 2023-10-03T10:14:13.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.689+0000 7f5d0affd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5d0c0a1340 0x7f5d0c133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:13.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.689+0000 7f5d0b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d0c0a0a30 0x7f5d0c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:13.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.689+0000 7f5d0affd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5d0c0a1340 0x7f5d0c133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40792/0 (socket says 172.21.15.3:40792) 2023-10-03T10:14:13.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.689+0000 7f5d0affd700 1 -- 172.21.15.3:0/1124958926 learned_addr learned my addr 172.21.15.3:0/1124958926 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:13.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.689+0000 7f5d0affd700 1 -- 172.21.15.3:0/1124958926 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d0c0a0a30 msgr2=0x7f5d0c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:13.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.689+0000 7f5d0affd700 1 --2- 172.21.15.3:0/1124958926 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d0c0a0a30 0x7f5d0c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.689+0000 7f5d0affd700 1 -- 172.21.15.3:0/1124958926 >> v1:172.21.15.3:6789/0 conn(0x7f5d0c133fa0 legacy=0x7f5d0c13ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:13.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.689+0000 7f5d0affd700 1 -- 172.21.15.3:0/1124958926 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0c0a1fe0 con 0x7f5d0c0a1340 2023-10-03T10:14:13.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.690+0000 7f5d0b7fe700 1 --2- 172.21.15.3:0/1124958926 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d0c0a0a30 0x7f5d0c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:13.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.690+0000 7f5d0affd700 1 --2- 172.21.15.3:0/1124958926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d0c0a1340 0x7f5d0c133a60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5d000098f0 tx=0x7f5d00009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d9229c1b7b80ae server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:13.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.690+0000 7f5d0a7fc700 1 -- 172.21.15.3:0/1124958926 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d00011830 con 0x7f5d0c0a1340 2023-10-03T10:14:13.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.690+0000 7f5d0a7fc700 1 -- 172.21.15.3:0/1124958926 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d00011990 con 0x7f5d0c0a1340 2023-10-03T10:14:13.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.690+0000 7f5d0a7fc700 1 -- 172.21.15.3:0/1124958926 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d00011d20 con 0x7f5d0c0a1340 2023-10-03T10:14:13.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.691+0000 7f5d1205e700 1 -- 172.21.15.3:0/1124958926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d0c0a1340 msgr2=0x7f5d0c133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:13.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.691+0000 7f5d1205e700 1 --2- 172.21.15.3:0/1124958926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d0c0a1340 0x7f5d0c133a60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5d000098f0 tx=0x7f5d00009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:13.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.691+0000 7f5d1205e700 1 -- 172.21.15.3:0/1124958926 shutdown_connections 2023-10-03T10:14:13.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.691+0000 7f5d1205e700 1 --2- 172.21.15.3:0/1124958926 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d0c0a0a30 0x7f5d0c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.691+0000 7f5d1205e700 1 --2- 172.21.15.3:0/1124958926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d0c0a1340 0x7f5d0c133a60 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.691+0000 7f5d1205e700 1 -- 172.21.15.3:0/1124958926 >> 172.21.15.3:0/1124958926 conn(0x7f5d0c0a6550 msgr2=0x7f5d0c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:13.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.691+0000 7f5d1205e700 1 -- 172.21.15.3:0/1124958926 shutdown_connections 2023-10-03T10:14:13.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.691+0000 7f5d1205e700 1 -- 172.21.15.3:0/1124958926 wait complete. 2023-10-03T10:14:13.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.692+0000 7f5d1205e700 1 Processor -- start 2023-10-03T10:14:13.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.692+0000 7f5d1205e700 1 -- start start 2023-10-03T10:14:13.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.692+0000 7f5d1205e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d0c0a0a30 0x7f5d0c11f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:13.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.692+0000 7f5d1205e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d0c0a1340 0x7f5d0c11f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:13.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.692+0000 7f5d1205e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d0c133fa0 0x7f5d0c1c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:13.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.692+0000 7f5d1205e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c09c030 con 0x7f5d0c0a0a30 2023-10-03T10:14:13.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.692+0000 7f5d1205e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c0a0220 con 0x7f5d0c133fa0 2023-10-03T10:14:13.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.692+0000 7f5d1205e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c1206b0 con 0x7f5d0c0a1340 2023-10-03T10:14:13.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.692+0000 7f5d0b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d0c0a0a30 0x7f5d0c11f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:13.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.693+0000 7f5d0b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d0c0a0a30 0x7f5d0c11f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40794/0 (socket says 172.21.15.3:40794) 2023-10-03T10:14:13.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.693+0000 7f5d0b7fe700 1 -- 172.21.15.3:0/1008442379 learned_addr learned my addr 172.21.15.3:0/1008442379 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:13.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.693+0000 7f5d0affd700 1 --2- 172.21.15.3:0/1008442379 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d0c0a1340 0x7f5d0c11f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:13.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.693+0000 7f5d0bfff700 1 --2- 172.21.15.3:0/1008442379 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d0c133fa0 0x7f5d0c1c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:13.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.693+0000 7f5d0b7fe700 1 -- 172.21.15.3:0/1008442379 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d0c0a1340 msgr2=0x7f5d0c11f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:13.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.693+0000 7f5d0b7fe700 1 --2- 172.21.15.3:0/1008442379 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d0c0a1340 0x7f5d0c11f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.693+0000 7f5d0b7fe700 1 -- 172.21.15.3:0/1008442379 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d0c133fa0 msgr2=0x7f5d0c1c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:13.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.693+0000 7f5d0b7fe700 1 --2- 172.21.15.3:0/1008442379 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d0c133fa0 0x7f5d0c1c8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.693+0000 7f5d0b7fe700 1 -- 172.21.15.3:0/1008442379 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cfc0094a0 con 0x7f5d0c0a0a30 2023-10-03T10:14:13.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.693+0000 7f5d0bfff700 1 --2- 172.21.15.3:0/1008442379 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d0c133fa0 0x7f5d0c1c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:14:13.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.693+0000 7f5d0b7fe700 1 --2- 172.21.15.3:0/1008442379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d0c0a0a30 0x7f5d0c11f460 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc00ebe0 tx=0x7f5cfc00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:13.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.693+0000 7f5cf3fff700 1 -- 172.21.15.3:0/1008442379 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc009800 con 0x7f5d0c0a0a30 2023-10-03T10:14:13.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.693+0000 7f5cf3fff700 1 -- 172.21.15.3:0/1008442379 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cfc00c910 con 0x7f5d0c0a0a30 2023-10-03T10:14:13.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.693+0000 7f5d1205e700 1 -- 172.21.15.3:0/1008442379 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d00009550 con 0x7f5d0c0a0a30 2023-10-03T10:14:13.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.693+0000 7f5cf3fff700 1 -- 172.21.15.3:0/1008442379 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc00cc80 con 0x7f5d0c0a0a30 2023-10-03T10:14:13.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.693+0000 7f5d1205e700 1 -- 172.21.15.3:0/1008442379 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d0c120e70 con 0x7f5d0c0a0a30 2023-10-03T10:14:13.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.694+0000 7f5cf3fff700 1 -- 172.21.15.3:0/1008442379 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5cfc013470 con 0x7f5d0c0a0a30 2023-10-03T10:14:13.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.694+0000 7f5d1205e700 1 -- 172.21.15.3:0/1008442379 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0c04ea50 con 0x7f5d0c0a0a30 2023-10-03T10:14:13.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.695+0000 7f5cf3fff700 1 -- 172.21.15.3:0/1008442379 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5cfc03f880 con 0x7f5d0c0a0a30 2023-10-03T10:14:13.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.700+0000 7f5cf3fff700 1 -- 172.21.15.3:0/1008442379 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5cfc013720 con 0x7f5d0c0a0a30 2023-10-03T10:14:13.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.771+0000 7f4ae1ac6700 1 -- 172.21.15.3:0/1045929815 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4adc09f420 con 0x7f4adc09e000 2023-10-03T10:14:13.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.772+0000 7f4ac3fff700 1 -- 172.21.15.3:0/1045929815 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4ad00102e0 con 0x7f4adc09e000 2023-10-03T10:14:13.772 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:13.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.774+0000 7f4ae1ac6700 1 -- 172.21.15.3:0/1045929815 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4adc09e000 msgr2=0x7f4adc1a5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:13.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.774+0000 7f4ae1ac6700 1 --2- 172.21.15.3:0/1045929815 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4adc09e000 0x7f4adc1a5e30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000c4d0 tx=0x7f4ad000c7e0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.774+0000 7f4ae1ac6700 1 -- 172.21.15.3:0/1045929815 shutdown_connections 2023-10-03T10:14:13.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.774+0000 7f4ae1ac6700 1 --2- 172.21.15.3:0/1045929815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4adc09e440 0x7f4adc1a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.774+0000 7f4ae1ac6700 1 --2- 172.21.15.3:0/1045929815 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4adc09d6f0 0x7f4adc1b91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.774+0000 7f4ae1ac6700 1 --2- 172.21.15.3:0/1045929815 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4adc09e000 0x7f4adc1a5e30 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.774+0000 7f4ae1ac6700 1 -- 172.21.15.3:0/1045929815 >> 172.21.15.3:0/1045929815 conn(0x7f4adc066f50 msgr2=0x7f4adc1bae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:13.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.774+0000 7f4ae1ac6700 1 -- 172.21.15.3:0/1045929815 shutdown_connections 2023-10-03T10:14:13.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.774+0000 7f4ae1ac6700 1 -- 172.21.15.3:0/1045929815 wait complete. 2023-10-03T10:14:13.788 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:13.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.853+0000 7f95e4df4700 1 -- 172.21.15.3:0/2316510515 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f95e0124850 con 0x7f95e00a0a30 2023-10-03T10:14:13.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.853+0000 7f95ceffd700 1 -- 172.21.15.3:0/2316510515 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f95e0124850 con 0x7f95e00a0a30 2023-10-03T10:14:13.853 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:13.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.856+0000 7f95e4df4700 1 -- 172.21.15.3:0/2316510515 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f95e00a0a30 msgr2=0x7f95e0121640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:13.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.856+0000 7f95e4df4700 1 --2- 172.21.15.3:0/2316510515 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f95e00a0a30 0x7f95e0121640 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009fd0 tx=0x7f95d4004750 comp rx=0 tx=0).stop 2023-10-03T10:14:13.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.856+0000 7f95e4df4700 1 -- 172.21.15.3:0/2316510515 shutdown_connections 2023-10-03T10:14:13.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.856+0000 7f95e4df4700 1 --2- 172.21.15.3:0/2316510515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95e0133930 0x7f95e0121b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.856+0000 7f95e4df4700 1 --2- 172.21.15.3:0/2316510515 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f95e009d630 0x7f95e0121100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.856+0000 7f95e4df4700 1 --2- 172.21.15.3:0/2316510515 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f95e00a0a30 0x7f95e0121640 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.856+0000 7f95e4df4700 1 -- 172.21.15.3:0/2316510515 >> 172.21.15.3:0/2316510515 conn(0x7f95e00a3050 msgr2=0x7f95e0133210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:13.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.856+0000 7f95e4df4700 1 -- 172.21.15.3:0/2316510515 shutdown_connections 2023-10-03T10:14:13.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.857+0000 7f95e4df4700 1 -- 172.21.15.3:0/2316510515 wait complete. 2023-10-03T10:14:13.875 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:13.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.917+0000 7f5d1205e700 1 -- 172.21.15.3:0/1008442379 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5d0c11efb0 con 0x7f5d0c0a0a30 2023-10-03T10:14:13.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.917+0000 7f5cf3fff700 1 -- 172.21.15.3:0/1008442379 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5cfc021070 con 0x7f5d0c0a0a30 2023-10-03T10:14:13.917 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:13.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.920+0000 7f5d1205e700 1 -- 172.21.15.3:0/1008442379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d0c0a0a30 msgr2=0x7f5d0c11f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:13.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.920+0000 7f5d1205e700 1 --2- 172.21.15.3:0/1008442379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d0c0a0a30 0x7f5d0c11f460 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc00ebe0 tx=0x7f5cfc00eef0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.920+0000 7f5d1205e700 1 -- 172.21.15.3:0/1008442379 shutdown_connections 2023-10-03T10:14:13.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.920+0000 7f5d1205e700 1 --2- 172.21.15.3:0/1008442379 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d0c133fa0 0x7f5d0c1c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.920+0000 7f5d1205e700 1 --2- 172.21.15.3:0/1008442379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d0c0a0a30 0x7f5d0c11f460 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.920+0000 7f5d1205e700 1 --2- 172.21.15.3:0/1008442379 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d0c0a1340 0x7f5d0c11f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:13.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.920+0000 7f5d1205e700 1 -- 172.21.15.3:0/1008442379 >> 172.21.15.3:0/1008442379 conn(0x7f5d0c0a6550 msgr2=0x7f5d0c1b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:13.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.920+0000 7f5d1205e700 1 -- 172.21.15.3:0/1008442379 shutdown_connections 2023-10-03T10:14:13.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:13.920+0000 7f5d1205e700 1 -- 172.21.15.3:0/1008442379 wait complete. 2023-10-03T10:14:13.933 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:14.250 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:14.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.394+0000 7f3ae57b9700 1 Processor -- start 2023-10-03T10:14:14.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.394+0000 7f3ae57b9700 1 -- start start 2023-10-03T10:14:14.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.394+0000 7f3ae57b9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3ae0130410 0x7f3ae01307e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:14.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.394+0000 7f3ae57b9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ae009de50 0x7f3ae009e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:14.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.395+0000 7f3ae57b9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae006f1c0 con 0x7f3ae009e7c0 2023-10-03T10:14:14.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.395+0000 7f3ae57b9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ae0070700 con 0x7f3ae0130410 2023-10-03T10:14:14.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.395+0000 7f3ae57b9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae0071f20 con 0x7f3ae009de50 2023-10-03T10:14:14.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.395+0000 7f3adeffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3ae0130410 0x7f3ae01307e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:14.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.395+0000 7f3ade7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ae009de50 0x7f3ae009e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:14.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.395+0000 7f3ade7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ae009de50 0x7f3ae009e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:55968/0 (socket says 172.21.15.3:55968) 2023-10-03T10:14:14.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.395+0000 7f3adeffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3ae0130410 0x7f3ae01307e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40800/0 (socket says 172.21.15.3:40800) 2023-10-03T10:14:14.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.395+0000 7f3ade7fc700 1 -- 172.21.15.3:0/2471213273 learned_addr learned my addr 172.21.15.3:0/2471213273 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:14.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.395+0000 7f3adeffd700 1 -- 172.21.15.3:0/2471213273 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ae009de50 msgr2=0x7f3ae009e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:14.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.395+0000 7f3adeffd700 1 --2- 172.21.15.3:0/2471213273 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ae009de50 0x7f3ae009e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:14.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.395+0000 7f3adeffd700 1 -- 172.21.15.3:0/2471213273 >> v1:172.21.15.3:6789/0 conn(0x7f3ae009e7c0 legacy=0x7f3ae013a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:14.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.395+0000 7f3adeffd700 1 -- 172.21.15.3:0/2471213273 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae012ffe0 con 0x7f3ae0130410 2023-10-03T10:14:14.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.395+0000 7f3ade7fc700 1 --2- 172.21.15.3:0/2471213273 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ae009de50 0x7f3ae009e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:14.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.395+0000 7f3adeffd700 1 --2- 172.21.15.3:0/2471213273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ae0130410 0x7f3ae01307e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009840 tx=0x7f3ad0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d82c7e6343eef4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:14.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.395+0000 7f3addffb700 1 -- 172.21.15.3:0/2471213273 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0011830 con 0x7f3ae0130410 2023-10-03T10:14:14.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.395+0000 7f3addffb700 1 -- 172.21.15.3:0/2471213273 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad0011990 con 0x7f3ae0130410 2023-10-03T10:14:14.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.395+0000 7f3addffb700 1 -- 172.21.15.3:0/2471213273 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0011d20 con 0x7f3ae0130410 2023-10-03T10:14:14.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.396+0000 7f3ae57b9700 1 -- 172.21.15.3:0/2471213273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ae0130410 msgr2=0x7f3ae01307e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:14.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.396+0000 7f3ae57b9700 1 --2- 172.21.15.3:0/2471213273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ae0130410 0x7f3ae01307e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009840 tx=0x7f3ad0009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:14.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.396+0000 7f3ae57b9700 1 -- 172.21.15.3:0/2471213273 shutdown_connections 2023-10-03T10:14:14.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.396+0000 7f3ae57b9700 1 --2- 172.21.15.3:0/2471213273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ae0130410 0x7f3ae01307e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:14.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.396+0000 7f3ae57b9700 1 --2- 172.21.15.3:0/2471213273 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ae009de50 0x7f3ae009e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:14.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.396+0000 7f3ae57b9700 1 -- 172.21.15.3:0/2471213273 >> 172.21.15.3:0/2471213273 conn(0x7f3ae00a2610 msgr2=0x7f3ae00a2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:14.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.396+0000 7f3ae57b9700 1 -- 172.21.15.3:0/2471213273 shutdown_connections 2023-10-03T10:14:14.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.396+0000 7f3ae57b9700 1 -- 172.21.15.3:0/2471213273 wait complete. 2023-10-03T10:14:14.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.397+0000 7f3ae57b9700 1 Processor -- start 2023-10-03T10:14:14.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.397+0000 7f3ae57b9700 1 -- start start 2023-10-03T10:14:14.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.398+0000 7f3ae57b9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ae009de50 0x7f3ae011c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:14.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.398+0000 7f3ae57b9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ae009e7c0 0x7f3ae011cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:14.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.398+0000 7f3ae57b9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ae0130410 0x7f3ae011d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:14.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.398+0000 7f3ae57b9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae011ff70 con 0x7f3ae009e7c0 2023-10-03T10:14:14.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.398+0000 7f3ae57b9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae012fa80 con 0x7f3ae009de50 2023-10-03T10:14:14.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.398+0000 7f3ae57b9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae011f4f0 con 0x7f3ae0130410 2023-10-03T10:14:14.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.398+0000 7f3adeffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ae009de50 0x7f3ae011c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:14.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.398+0000 7f3ade7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ae009e7c0 0x7f3ae011cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:14.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.398+0000 7f3adeffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ae009de50 0x7f3ae011c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:55978/0 (socket says 172.21.15.3:55978) 2023-10-03T10:14:14.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.398+0000 7f3adeffd700 1 -- 172.21.15.3:0/1818053141 learned_addr learned my addr 172.21.15.3:0/1818053141 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:14.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.398+0000 7f3adf7fe700 1 --2- 172.21.15.3:0/1818053141 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ae0130410 0x7f3ae011d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:14.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.398+0000 7f3adeffd700 1 -- 172.21.15.3:0/1818053141 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ae0130410 msgr2=0x7f3ae011d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:14.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.398+0000 7f3adeffd700 1 --2- 172.21.15.3:0/1818053141 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ae0130410 0x7f3ae011d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:14.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.398+0000 7f3adeffd700 1 -- 172.21.15.3:0/1818053141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ae009e7c0 msgr2=0x7f3ae011cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:14.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.398+0000 7f3adeffd700 1 --2- 172.21.15.3:0/1818053141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ae009e7c0 0x7f3ae011cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:14.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.398+0000 7f3adeffd700 1 -- 172.21.15.3:0/1818053141 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad4009550 con 0x7f3ae009de50 2023-10-03T10:14:14.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.398+0000 7f3ade7fc700 1 --2- 172.21.15.3:0/1818053141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ae009e7c0 0x7f3ae011cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:14.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.398+0000 7f3adeffd700 1 --2- 172.21.15.3:0/1818053141 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ae009de50 0x7f3ae011c690 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009fd0 tx=0x7f3ad0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:14.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.399+0000 7f3ac77fe700 1 -- 172.21.15.3:0/1818053141 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad00036a0 con 0x7f3ae009de50 2023-10-03T10:14:14.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.399+0000 7f3ac77fe700 1 -- 172.21.15.3:0/1818053141 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad001f070 con 0x7f3ae009de50 2023-10-03T10:14:14.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.399+0000 7f3ae57b9700 1 -- 172.21.15.3:0/1818053141 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad00094a0 con 0x7f3ae009de50 2023-10-03T10:14:14.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.399+0000 7f3ac77fe700 1 -- 172.21.15.3:0/1818053141 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0003a60 con 0x7f3ae009de50 2023-10-03T10:14:14.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.399+0000 7f3ae57b9700 1 -- 172.21.15.3:0/1818053141 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae011f990 con 0x7f3ae009de50 2023-10-03T10:14:14.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.399+0000 7f3ac77fe700 1 -- 172.21.15.3:0/1818053141 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3ad0003bc0 con 0x7f3ae009de50 2023-10-03T10:14:14.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.400+0000 7f3ac77fe700 1 -- 172.21.15.3:0/1818053141 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3ad0021070 con 0x7f3ae009de50 2023-10-03T10:14:14.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.400+0000 7f3ae57b9700 1 -- 172.21.15.3:0/1818053141 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae011e880 con 0x7f3ae009de50 2023-10-03T10:14:14.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.405+0000 7f3ac77fe700 1 -- 172.21.15.3:0/1818053141 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3ad0015080 con 0x7f3ae009de50 2023-10-03T10:14:14.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.607+0000 7f3ae57b9700 1 -- 172.21.15.3:0/1818053141 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3ae00747f0 con 0x7f3ae009de50 2023-10-03T10:14:14.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.607+0000 7f3ac77fe700 1 -- 172.21.15.3:0/1818053141 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3ad0035090 con 0x7f3ae009de50 2023-10-03T10:14:14.608 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:14.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.609+0000 7f3ae57b9700 1 -- 172.21.15.3:0/1818053141 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ae009de50 msgr2=0x7f3ae011c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:14.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.609+0000 7f3ae57b9700 1 --2- 172.21.15.3:0/1818053141 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ae009de50 0x7f3ae011c690 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009fd0 tx=0x7f3ad0004750 comp rx=0 tx=0).stop 2023-10-03T10:14:14.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.609+0000 7f3ae57b9700 1 -- 172.21.15.3:0/1818053141 shutdown_connections 2023-10-03T10:14:14.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.609+0000 7f3ae57b9700 1 --2- 172.21.15.3:0/1818053141 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ae0130410 0x7f3ae011d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:14.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.609+0000 7f3ae57b9700 1 --2- 172.21.15.3:0/1818053141 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ae009de50 0x7f3ae011c690 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:14.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.609+0000 7f3ae57b9700 1 --2- 172.21.15.3:0/1818053141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ae009e7c0 0x7f3ae011cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:14.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.609+0000 7f3ae57b9700 1 -- 172.21.15.3:0/1818053141 >> 172.21.15.3:0/1818053141 conn(0x7f3ae00a2610 msgr2=0x7f3ae01bad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:14.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.609+0000 7f3ae57b9700 1 -- 172.21.15.3:0/1818053141 shutdown_connections 2023-10-03T10:14:14.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.609+0000 7f3ae57b9700 1 -- 172.21.15.3:0/1818053141 wait complete. 2023-10-03T10:14:14.623 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:14.789 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:14.877 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:14.934 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:14.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.937+0000 7f66bf62a700 1 Processor -- start 2023-10-03T10:14:14.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.938+0000 7f66bf62a700 1 -- start start 2023-10-03T10:14:14.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.938+0000 7f66bf62a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f66b809d6f0 0x7f66b809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:14.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.938+0000 7f66bf62a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66b809e000 0x7f66b8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:14.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.938+0000 7f66bf62a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66b809e410 0x7f66b8130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:14.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.938+0000 7f66bf62a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66b809f520 con 0x7f66b809d6f0 2023-10-03T10:14:14.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.938+0000 7f66bf62a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b809f130 con 0x7f66b809e000 2023-10-03T10:14:14.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.938+0000 7f66bf62a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66b809e930 con 0x7f66b809e410 2023-10-03T10:14:14.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.938+0000 7f66bcbc5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66b809e000 0x7f66b8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:14.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.938+0000 7f66bcbc5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66b809e000 0x7f66b8130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:55988/0 (socket says 172.21.15.3:55988) 2023-10-03T10:14:14.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.938+0000 7f66bcbc5700 1 -- 172.21.15.3:0/830241718 learned_addr learned my addr 172.21.15.3:0/830241718 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:14.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.938+0000 7f66bd3c6700 1 --2- 172.21.15.3:0/830241718 >> v2:172.21.15.3:3300/0 conn(0x7f66b809d6f0 0x7f66b809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:14.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.939+0000 7f66bdbc7700 1 --2- 172.21.15.3:0/830241718 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66b809e410 0x7f66b8130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:14.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.939+0000 7f66bd3c6700 1 -- 172.21.15.3:0/830241718 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66b809e410 msgr2=0x7f66b8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:14.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.939+0000 7f66bd3c6700 1 --2- 172.21.15.3:0/830241718 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66b809e410 0x7f66b8130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:14.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.939+0000 7f66bd3c6700 1 -- 172.21.15.3:0/830241718 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66b809e000 msgr2=0x7f66b8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:14.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.939+0000 7f66bd3c6700 1 --2- 172.21.15.3:0/830241718 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66b809e000 0x7f66b8130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:14.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.939+0000 7f66bd3c6700 1 -- 172.21.15.3:0/830241718 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b809b2d0 con 0x7f66b809d6f0 2023-10-03T10:14:14.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.939+0000 7f66bcbc5700 1 --2- 172.21.15.3:0/830241718 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66b809e000 0x7f66b8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:14.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.939+0000 7f66bd3c6700 1 --2- 172.21.15.3:0/830241718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66b809d6f0 0x7f66b809dac0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f66b4009870 tx=0x7f66b4009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e8d46219fd3b1b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:14.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.939+0000 7f66affff700 1 -- 172.21.15.3:0/830241718 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b4011830 con 0x7f66b809d6f0 2023-10-03T10:14:14.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.939+0000 7f66affff700 1 -- 172.21.15.3:0/830241718 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b4011990 con 0x7f66b809d6f0 2023-10-03T10:14:14.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.939+0000 7f66affff700 1 -- 172.21.15.3:0/830241718 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b4011d20 con 0x7f66b809d6f0 2023-10-03T10:14:14.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.940+0000 7f66bf62a700 1 -- 172.21.15.3:0/830241718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66b809d6f0 msgr2=0x7f66b809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.940+0000 7f66bf62a700 1 --2- 172.21.15.3:0/830241718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66b809d6f0 0x7f66b809dac0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f66b4009870 tx=0x7f66b4009b80 comp rx=0 tx=0).stop 2023-10-03T10:14:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.940+0000 7f66bf62a700 1 -- 172.21.15.3:0/830241718 shutdown_connections 2023-10-03T10:14:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.940+0000 7f66bf62a700 1 --2- 172.21.15.3:0/830241718 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66b809e410 0x7f66b8130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.940+0000 7f66bf62a700 1 --2- 172.21.15.3:0/830241718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66b809d6f0 0x7f66b809dac0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.940+0000 7f66bf62a700 1 --2- 172.21.15.3:0/830241718 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66b809e000 0x7f66b8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.940+0000 7f66bf62a700 1 -- 172.21.15.3:0/830241718 >> 172.21.15.3:0/830241718 conn(0x7f66b8066f50 msgr2=0x7f66b809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.940+0000 7f66bf62a700 1 -- 172.21.15.3:0/830241718 shutdown_connections 2023-10-03T10:14:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.940+0000 7f66bf62a700 1 -- 172.21.15.3:0/830241718 wait complete. 2023-10-03T10:14:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.942+0000 7f66bf62a700 1 Processor -- start 2023-10-03T10:14:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.942+0000 7f66bf62a700 1 -- start start 2023-10-03T10:14:14.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.943+0000 7f66bf62a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66b809d6f0 0x7f66b811e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:14.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.943+0000 7f66bf62a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66b809e000 0x7f66b811e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:14.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.943+0000 7f66bf62a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66b809e410 0x7f66b811eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:14.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.943+0000 7f66bf62a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b811fb30 con 0x7f66b809e410 2023-10-03T10:14:14.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.943+0000 7f66bf62a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b8120d30 con 0x7f66b809d6f0 2023-10-03T10:14:14.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.943+0000 7f66bf62a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66b81216a0 con 0x7f66b809e000 2023-10-03T10:14:14.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.943+0000 7f66bcbc5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66b809e000 0x7f66b811e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:14.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.943+0000 7f66bcbc5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66b809e000 0x7f66b811e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52616/0 (socket says 172.21.15.3:52616) 2023-10-03T10:14:14.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.943+0000 7f66bcbc5700 1 -- 172.21.15.3:0/2782099402 learned_addr learned my addr 172.21.15.3:0/2782099402 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:14.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.943+0000 7f66bdbc7700 1 --2- 172.21.15.3:0/2782099402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66b809e410 0x7f66b811eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:14.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.943+0000 7f66bd3c6700 1 --2- 172.21.15.3:0/2782099402 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66b809d6f0 0x7f66b811e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:14.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.943+0000 7f66bcbc5700 1 -- 172.21.15.3:0/2782099402 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66b809d6f0 msgr2=0x7f66b811e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:14.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.943+0000 7f66bcbc5700 1 --2- 172.21.15.3:0/2782099402 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66b809d6f0 0x7f66b811e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:14.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.943+0000 7f66bcbc5700 1 -- 172.21.15.3:0/2782099402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66b809e410 msgr2=0x7f66b811eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:14.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.943+0000 7f66bcbc5700 1 --2- 172.21.15.3:0/2782099402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66b809e410 0x7f66b811eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:14.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.943+0000 7f66bcbc5700 1 -- 172.21.15.3:0/2782099402 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a800b7f0 con 0x7f66b809e000 2023-10-03T10:14:14.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.944+0000 7f66bcbc5700 1 --2- 172.21.15.3:0/2782099402 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66b809e000 0x7f66b811e560 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f66a800bfa0 tx=0x7f66a800e640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:14.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.944+0000 7f66adffb700 1 -- 172.21.15.3:0/2782099402 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a8004030 con 0x7f66b809e000 2023-10-03T10:14:14.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.944+0000 7f66adffb700 1 -- 172.21.15.3:0/2782099402 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a8004190 con 0x7f66b809e000 2023-10-03T10:14:14.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.944+0000 7f66adffb700 1 -- 172.21.15.3:0/2782099402 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a8004540 con 0x7f66b809e000 2023-10-03T10:14:14.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.944+0000 7f66bf62a700 1 -- 172.21.15.3:0/2782099402 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b40094a0 con 0x7f66b809e000 2023-10-03T10:14:14.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.944+0000 7f66bf62a700 1 -- 172.21.15.3:0/2782099402 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b811fdb0 con 0x7f66b809e000 2023-10-03T10:14:14.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.945+0000 7f66adffb700 1 -- 172.21.15.3:0/2782099402 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f66a8015860 con 0x7f66b809e000 2023-10-03T10:14:14.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.945+0000 7f66adffb700 1 -- 172.21.15.3:0/2782099402 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f66a8043dc0 con 0x7f66b809e000 2023-10-03T10:14:14.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.946+0000 7f66bf62a700 1 -- 172.21.15.3:0/2782099402 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66b8121230 con 0x7f66b809e000 2023-10-03T10:14:14.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:14.951+0000 7f66adffb700 1 -- 172.21.15.3:0/2782099402 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f66a80036a0 con 0x7f66b809e000 2023-10-03T10:14:15.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.021+0000 7f477607b700 1 Processor -- start 2023-10-03T10:14:15.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.021+0000 7f477607b700 1 -- start start 2023-10-03T10:14:15.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.021+0000 7f477607b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4770136820 0x7f4770138c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:15.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.021+0000 7f477607b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4770139140 0x7f477013b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:15.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.021+0000 7f477607b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f477006bc00 con 0x7f4770133a80 2023-10-03T10:14:15.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.021+0000 7f477607b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f477006c570 con 0x7f4770139140 2023-10-03T10:14:15.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.021+0000 7f477607b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47700a0040 con 0x7f4770136820 2023-10-03T10:14:15.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.021+0000 7f476f7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4770133a80 0x7f4770133e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44014/0 (socket says 172.21.15.3:44014) 2023-10-03T10:14:15.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.021+0000 7f476f7fe700 1 -- 172.21.15.3:0/710886587 learned_addr learned my addr 172.21.15.3:0/710886587 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:15.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.021+0000 7f476ffff700 1 --2- 172.21.15.3:0/710886587 >> v2:172.21.15.3:3300/0 conn(0x7f4770139140 0x7f477013b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:15.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.021+0000 7f476effd700 1 --2- 172.21.15.3:0/710886587 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4770136820 0x7f4770138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:15.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.022+0000 7f476effd700 1 -- 172.21.15.3:0/710886587 >> v2:172.21.15.3:3300/0 conn(0x7f4770139140 msgr2=0x7f477013b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:15.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.022+0000 7f476effd700 1 --2- 172.21.15.3:0/710886587 >> v2:172.21.15.3:3300/0 conn(0x7f4770139140 0x7f477013b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:15.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.022+0000 7f476effd700 1 -- 172.21.15.3:0/710886587 >> v1:172.21.15.3:6789/0 conn(0x7f4770133a80 legacy=0x7f4770133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:15.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.022+0000 7f476effd700 1 -- 172.21.15.3:0/710886587 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4770133860 con 0x7f4770136820 2023-10-03T10:14:15.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.022+0000 7f476ffff700 1 --2- 172.21.15.3:0/710886587 >> v2:172.21.15.3:3300/0 conn(0x7f4770139140 0x7f477013b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:15.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.022+0000 7f476effd700 1 --2- 172.21.15.3:0/710886587 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4770136820 0x7f4770138c00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4764009840 tx=0x7f4764009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=897914de32affaba server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:15.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.022+0000 7f476e7fc700 1 -- 172.21.15.3:0/710886587 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764011830 con 0x7f4770136820 2023-10-03T10:14:15.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.022+0000 7f476e7fc700 1 -- 172.21.15.3:0/710886587 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4764011990 con 0x7f4770136820 2023-10-03T10:14:15.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.022+0000 7f476e7fc700 1 -- 172.21.15.3:0/710886587 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764011d20 con 0x7f4770136820 2023-10-03T10:14:15.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.023+0000 7f477607b700 1 -- 172.21.15.3:0/710886587 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4770136820 msgr2=0x7f4770138c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:15.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.023+0000 7f477607b700 1 --2- 172.21.15.3:0/710886587 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4770136820 0x7f4770138c00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4764009840 tx=0x7f4764009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:15.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.023+0000 7f477607b700 1 -- 172.21.15.3:0/710886587 shutdown_connections 2023-10-03T10:14:15.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.023+0000 7f477607b700 1 --2- 172.21.15.3:0/710886587 >> v2:172.21.15.3:3300/0 conn(0x7f4770139140 0x7f477013b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:15.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.023+0000 7f477607b700 1 --2- 172.21.15.3:0/710886587 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4770136820 0x7f4770138c00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:15.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.023+0000 7f477607b700 1 -- 172.21.15.3:0/710886587 >> 172.21.15.3:0/710886587 conn(0x7f47700a4210 msgr2=0x7f47700a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:15.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.023+0000 7f477607b700 1 -- 172.21.15.3:0/710886587 shutdown_connections 2023-10-03T10:14:15.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.023+0000 7f477607b700 1 -- 172.21.15.3:0/710886587 wait complete. 2023-10-03T10:14:15.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.025+0000 7f477607b700 1 Processor -- start 2023-10-03T10:14:15.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.025+0000 7f477607b700 1 -- start start 2023-10-03T10:14:15.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.026+0000 7f477607b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4770133a80 0x7f477006f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:15.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.026+0000 7f477607b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4770136820 0x7f477006fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:15.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.026+0000 7f477607b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4770139140 0x7f47700703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:15.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.026+0000 7f477607b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477009bcf0 con 0x7f4770139140 2023-10-03T10:14:15.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.026+0000 7f477607b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4770070b00 con 0x7f4770133a80 2023-10-03T10:14:15.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.026+0000 7f477607b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4770072620 con 0x7f4770136820 2023-10-03T10:14:15.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.026+0000 7f476effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4770136820 0x7f477006fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:15.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.026+0000 7f476f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4770133a80 0x7f477006f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:15.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.026+0000 7f476effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4770136820 0x7f477006fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52630/0 (socket says 172.21.15.3:52630) 2023-10-03T10:14:15.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.026+0000 7f476f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4770133a80 0x7f477006f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56010/0 (socket says 172.21.15.3:56010) 2023-10-03T10:14:15.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.026+0000 7f476effd700 1 -- 172.21.15.3:0/1699504242 learned_addr learned my addr 172.21.15.3:0/1699504242 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:15.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.026+0000 7f476ffff700 1 --2- 172.21.15.3:0/1699504242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4770139140 0x7f47700703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:15.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.026+0000 7f476f7fe700 1 -- 172.21.15.3:0/1699504242 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4770136820 msgr2=0x7f477006fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:15.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.026+0000 7f476f7fe700 1 --2- 172.21.15.3:0/1699504242 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4770136820 0x7f477006fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:15.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.026+0000 7f476f7fe700 1 -- 172.21.15.3:0/1699504242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4770139140 msgr2=0x7f47700703d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:15.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.026+0000 7f476f7fe700 1 --2- 172.21.15.3:0/1699504242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4770139140 0x7f47700703d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:15.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.026+0000 7f476f7fe700 1 -- 172.21.15.3:0/1699504242 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4768009a70 con 0x7f4770133a80 2023-10-03T10:14:15.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.026+0000 7f476f7fe700 1 --2- 172.21.15.3:0/1699504242 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4770133a80 0x7f477006f950 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f476000c500 tx=0x7f476000c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:15.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.027+0000 7f4757fff700 1 -- 172.21.15.3:0/1699504242 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476000cea0 con 0x7f4770133a80 2023-10-03T10:14:15.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.027+0000 7f4757fff700 1 -- 172.21.15.3:0/1699504242 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4760005d70 con 0x7f4770133a80 2023-10-03T10:14:15.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.027+0000 7f477607b700 1 -- 172.21.15.3:0/1699504242 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47640094a0 con 0x7f4770133a80 2023-10-03T10:14:15.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.027+0000 7f4757fff700 1 -- 172.21.15.3:0/1699504242 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47600072d0 con 0x7f4770133a80 2023-10-03T10:14:15.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.027+0000 7f477607b700 1 -- 172.21.15.3:0/1699504242 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47700711e0 con 0x7f4770133a80 2023-10-03T10:14:15.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.028+0000 7f4757fff700 1 -- 172.21.15.3:0/1699504242 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4760007430 con 0x7f4770133a80 2023-10-03T10:14:15.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.028+0000 7f477607b700 1 -- 172.21.15.3:0/1699504242 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4770074ea0 con 0x7f4770133a80 2023-10-03T10:14:15.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.028+0000 7f4757fff700 1 -- 172.21.15.3:0/1699504242 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f476003e9a0 con 0x7f4770133a80 2023-10-03T10:14:15.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.033+0000 7f4757fff700 1 -- 172.21.15.3:0/1699504242 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f476001b3b0 con 0x7f4770133a80 2023-10-03T10:14:15.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.077+0000 7f0d0e18b700 1 Processor -- start 2023-10-03T10:14:15.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.077+0000 7f0d0e18b700 1 -- start start 2023-10-03T10:14:15.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.078+0000 7f0d0e18b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0d0809f410 0x7f0d08136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:15.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.078+0000 7f0d0e18b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0d08136fe0 0x7f0d081393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:15.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.078+0000 7f0d0e18b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d08070940 con 0x7f0d0809e930 2023-10-03T10:14:15.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.078+0000 7f0d0e18b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d0806fd60 con 0x7f0d0809f410 2023-10-03T10:14:15.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.078+0000 7f0d0e18b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0806fa80 con 0x7f0d08136fe0 2023-10-03T10:14:15.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.078+0000 7f0d06ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0d0809f410 0x7f0d08136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:15.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.078+0000 7f0d06ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0d0809f410 0x7f0d08136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40850/0 (socket says 172.21.15.3:40850) 2023-10-03T10:14:15.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.078+0000 7f0d06ffd700 1 -- 172.21.15.3:0/3965009417 learned_addr learned my addr 172.21.15.3:0/3965009417 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:15.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.078+0000 7f0d07fff700 1 --2- 172.21.15.3:0/3965009417 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0d08136fe0 0x7f0d081393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:15.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.078+0000 7f0d06ffd700 1 -- 172.21.15.3:0/3965009417 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0d08136fe0 msgr2=0x7f0d081393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:15.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.078+0000 7f0d06ffd700 1 --2- 172.21.15.3:0/3965009417 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0d08136fe0 0x7f0d081393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:15.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.078+0000 7f0d06ffd700 1 -- 172.21.15.3:0/3965009417 >> v1:172.21.15.3:6789/0 conn(0x7f0d0809e930 legacy=0x7f0d0809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:15.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.078+0000 7f0d06ffd700 1 -- 172.21.15.3:0/3965009417 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0809fb60 con 0x7f0d0809f410 2023-10-03T10:14:15.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.079+0000 7f0d06ffd700 1 --2- 172.21.15.3:0/3965009417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d0809f410 0x7f0d08136aa0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0098f0 tx=0x7f0cfc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e2146abbceb70e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:15.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.079+0000 7f0d067fc700 1 -- 172.21.15.3:0/3965009417 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc011830 con 0x7f0d0809f410 2023-10-03T10:14:15.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.079+0000 7f0d067fc700 1 -- 172.21.15.3:0/3965009417 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cfc011990 con 0x7f0d0809f410 2023-10-03T10:14:15.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.079+0000 7f0d067fc700 1 -- 172.21.15.3:0/3965009417 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc011d20 con 0x7f0d0809f410 2023-10-03T10:14:15.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.079+0000 7f0d0e18b700 1 -- 172.21.15.3:0/3965009417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d0809f410 msgr2=0x7f0d08136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:15.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.079+0000 7f0d0e18b700 1 --2- 172.21.15.3:0/3965009417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d0809f410 0x7f0d08136aa0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0098f0 tx=0x7f0cfc009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:15.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.079+0000 7f0d0e18b700 1 -- 172.21.15.3:0/3965009417 shutdown_connections 2023-10-03T10:14:15.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.079+0000 7f0d0e18b700 1 --2- 172.21.15.3:0/3965009417 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0d08136fe0 0x7f0d081393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:15.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.079+0000 7f0d0e18b700 1 --2- 172.21.15.3:0/3965009417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d0809f410 0x7f0d08136aa0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:15.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.079+0000 7f0d0e18b700 1 -- 172.21.15.3:0/3965009417 >> 172.21.15.3:0/3965009417 conn(0x7f0d080a4210 msgr2=0x7f0d080a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:15.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.080+0000 7f0d0e18b700 1 -- 172.21.15.3:0/3965009417 shutdown_connections 2023-10-03T10:14:15.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.080+0000 7f0d0e18b700 1 -- 172.21.15.3:0/3965009417 wait complete. 2023-10-03T10:14:15.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.080+0000 7f0d0e18b700 1 Processor -- start 2023-10-03T10:14:15.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.081+0000 7f0d0e18b700 1 -- start start 2023-10-03T10:14:15.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.081+0000 7f0d0e18b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0d0809e930 0x7f0d081c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:15.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.081+0000 7f0d0e18b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0d0809f410 0x7f0d081c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:15.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.081+0000 7f0d0e18b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d08136fe0 0x7f0d081c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:15.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.081+0000 7f0d0e18b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d08122130 con 0x7f0d08136fe0 2023-10-03T10:14:15.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.081+0000 7f0d0e18b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d08071d20 con 0x7f0d0809e930 2023-10-03T10:14:15.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.081+0000 7f0d0e18b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0806d4c0 con 0x7f0d0809f410 2023-10-03T10:14:15.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.081+0000 7f0d07fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d08136fe0 0x7f0d081c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:15.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.081+0000 7f0d07fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d08136fe0 0x7f0d081c8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40866/0 (socket says 172.21.15.3:40866) 2023-10-03T10:14:15.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.081+0000 7f0d07fff700 1 -- 172.21.15.3:0/1513001065 learned_addr learned my addr 172.21.15.3:0/1513001065 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:15.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.081+0000 7f0d077fe700 1 --2- 172.21.15.3:0/1513001065 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0d0809e930 0x7f0d081c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:15.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.082+0000 7f0d06ffd700 1 --2- 172.21.15.3:0/1513001065 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0d0809f410 0x7f0d081c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:15.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.082+0000 7f0d07fff700 1 -- 172.21.15.3:0/1513001065 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0d0809f410 msgr2=0x7f0d081c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:15.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.082+0000 7f0d07fff700 1 --2- 172.21.15.3:0/1513001065 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0d0809f410 0x7f0d081c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:15.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.082+0000 7f0d07fff700 1 -- 172.21.15.3:0/1513001065 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0d0809e930 msgr2=0x7f0d081c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:15.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.082+0000 7f0d07fff700 1 --2- 172.21.15.3:0/1513001065 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0d0809e930 0x7f0d081c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:15.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.082+0000 7f0d07fff700 1 -- 172.21.15.3:0/1513001065 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc009550 con 0x7f0d08136fe0 2023-10-03T10:14:15.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.082+0000 7f0d07fff700 1 --2- 172.21.15.3:0/1513001065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d08136fe0 0x7f0d081c8ce0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000efc0 tx=0x7f0d0000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:15.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.082+0000 7f0ceffff700 1 -- 172.21.15.3:0/1513001065 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d00009ea0 con 0x7f0d08136fe0 2023-10-03T10:14:15.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.082+0000 7f0ceffff700 1 -- 172.21.15.3:0/1513001065 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0000f040 con 0x7f0d08136fe0 2023-10-03T10:14:15.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.082+0000 7f0d0e18b700 1 -- 172.21.15.3:0/1513001065 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d080a2f40 con 0x7f0d08136fe0 2023-10-03T10:14:15.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.082+0000 7f0ceffff700 1 -- 172.21.15.3:0/1513001065 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0000ced0 con 0x7f0d08136fe0 2023-10-03T10:14:15.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.082+0000 7f0d0e18b700 1 -- 172.21.15.3:0/1513001065 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d08121090 con 0x7f0d08136fe0 2023-10-03T10:14:15.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.083+0000 7f0ceffff700 1 -- 172.21.15.3:0/1513001065 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0d00014480 con 0x7f0d08136fe0 2023-10-03T10:14:15.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.083+0000 7f0ceffff700 1 -- 172.21.15.3:0/1513001065 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0d00040180 con 0x7f0d08136fe0 2023-10-03T10:14:15.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.084+0000 7f0d0e18b700 1 -- 172.21.15.3:0/1513001065 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d08074ea0 con 0x7f0d08136fe0 2023-10-03T10:14:15.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.089+0000 7f0ceffff700 1 -- 172.21.15.3:0/1513001065 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0d00018030 con 0x7f0d08136fe0 2023-10-03T10:14:15.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.163+0000 7f66bf62a700 1 -- 172.21.15.3:0/2782099402 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f66b8074ea0 con 0x7f66b809e000 2023-10-03T10:14:15.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.163+0000 7f66adffb700 1 -- 172.21.15.3:0/2782099402 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f66a80252f0 con 0x7f66b809e000 2023-10-03T10:14:15.164 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:15.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.166+0000 7f66bf62a700 1 -- 172.21.15.3:0/2782099402 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66b809e000 msgr2=0x7f66b811e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:15.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.166+0000 7f66bf62a700 1 --2- 172.21.15.3:0/2782099402 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66b809e000 0x7f66b811e560 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f66a800bfa0 tx=0x7f66a800e640 comp rx=0 tx=0).stop 2023-10-03T10:14:15.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.166+0000 7f66bf62a700 1 -- 172.21.15.3:0/2782099402 shutdown_connections 2023-10-03T10:14:15.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.166+0000 7f66bf62a700 1 --2- 172.21.15.3:0/2782099402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66b809e410 0x7f66b811eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:15.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.166+0000 7f66bf62a700 1 --2- 172.21.15.3:0/2782099402 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66b809d6f0 0x7f66b811e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:15.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.166+0000 7f66bf62a700 1 --2- 172.21.15.3:0/2782099402 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66b809e000 0x7f66b811e560 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:15.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.166+0000 7f66bf62a700 1 -- 172.21.15.3:0/2782099402 >> 172.21.15.3:0/2782099402 conn(0x7f66b8066f50 msgr2=0x7f66b806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:15.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.166+0000 7f66bf62a700 1 -- 172.21.15.3:0/2782099402 shutdown_connections 2023-10-03T10:14:15.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.166+0000 7f66bf62a700 1 -- 172.21.15.3:0/2782099402 wait complete. 2023-10-03T10:14:15.180 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:15.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.275+0000 7f477607b700 1 -- 172.21.15.3:0/1699504242 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f47700722c0 con 0x7f4770133a80 2023-10-03T10:14:15.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.275+0000 7f4757fff700 1 -- 172.21.15.3:0/1699504242 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4760017030 con 0x7f4770133a80 2023-10-03T10:14:15.275 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:15.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.277+0000 7f477607b700 1 -- 172.21.15.3:0/1699504242 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4770133a80 msgr2=0x7f477006f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:15.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.277+0000 7f477607b700 1 --2- 172.21.15.3:0/1699504242 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4770133a80 0x7f477006f950 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f476000c500 tx=0x7f476000c810 comp rx=0 tx=0).stop 2023-10-03T10:14:15.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.277+0000 7f477607b700 1 -- 172.21.15.3:0/1699504242 shutdown_connections 2023-10-03T10:14:15.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.277+0000 7f477607b700 1 --2- 172.21.15.3:0/1699504242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4770139140 0x7f47700703d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:15.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.277+0000 7f477607b700 1 --2- 172.21.15.3:0/1699504242 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4770133a80 0x7f477006f950 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:15.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.277+0000 7f477607b700 1 --2- 172.21.15.3:0/1699504242 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4770136820 0x7f477006fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:15.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.277+0000 7f477607b700 1 -- 172.21.15.3:0/1699504242 >> 172.21.15.3:0/1699504242 conn(0x7f47700a4210 msgr2=0x7f47700a0920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:15.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.278+0000 7f477607b700 1 -- 172.21.15.3:0/1699504242 shutdown_connections 2023-10-03T10:14:15.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.278+0000 7f477607b700 1 -- 172.21.15.3:0/1699504242 wait complete. 2023-10-03T10:14:15.291 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:15.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.329+0000 7f0d0e18b700 1 -- 172.21.15.3:0/1513001065 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0d0806b250 con 0x7f0d08136fe0 2023-10-03T10:14:15.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.330+0000 7f0ceffff700 1 -- 172.21.15.3:0/1513001065 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0d00007920 con 0x7f0d08136fe0 2023-10-03T10:14:15.330 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:15.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.332+0000 7f0d0e18b700 1 -- 172.21.15.3:0/1513001065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d08136fe0 msgr2=0x7f0d081c8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:15.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.332+0000 7f0d0e18b700 1 --2- 172.21.15.3:0/1513001065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d08136fe0 0x7f0d081c8ce0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000efc0 tx=0x7f0d0000c3a0 comp rx=0 tx=0).stop 2023-10-03T10:14:15.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.332+0000 7f0d0e18b700 1 -- 172.21.15.3:0/1513001065 shutdown_connections 2023-10-03T10:14:15.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.332+0000 7f0d0e18b700 1 --2- 172.21.15.3:0/1513001065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d08136fe0 0x7f0d081c8ce0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:15.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.332+0000 7f0d0e18b700 1 --2- 172.21.15.3:0/1513001065 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0d0809e930 0x7f0d081c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:15.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.332+0000 7f0d0e18b700 1 --2- 172.21.15.3:0/1513001065 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0d0809f410 0x7f0d081c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:15.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.332+0000 7f0d0e18b700 1 -- 172.21.15.3:0/1513001065 >> 172.21.15.3:0/1513001065 conn(0x7f0d080a4210 msgr2=0x7f0d08134a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:15.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.332+0000 7f0d0e18b700 1 -- 172.21.15.3:0/1513001065 shutdown_connections 2023-10-03T10:14:15.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.332+0000 7f0d0e18b700 1 -- 172.21.15.3:0/1513001065 wait complete. 2023-10-03T10:14:15.346 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:15.624 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:15.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.770+0000 7f5970c8d700 1 Processor -- start 2023-10-03T10:14:15.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.770+0000 7f5970c8d700 1 -- start start 2023-10-03T10:14:15.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.771+0000 7f5970c8d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f596c09d6f0 0x7f596c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:15.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.771+0000 7f5970c8d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f596c09e440 0x7f596c134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:15.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.771+0000 7f5970c8d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f596c09b120 con 0x7f596c09e000 2023-10-03T10:14:15.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.771+0000 7f5970c8d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f596c06b460 con 0x7f596c09d6f0 2023-10-03T10:14:15.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.771+0000 7f5970c8d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f596c09a7e0 con 0x7f596c09e440 2023-10-03T10:14:15.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.771+0000 7f596a59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f596c09d6f0 0x7f596c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:15.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.771+0000 7f596ad9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f596c09e440 0x7f596c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:15.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.771+0000 7f596a59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f596c09d6f0 0x7f596c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40872/0 (socket says 172.21.15.3:40872) 2023-10-03T10:14:15.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.771+0000 7f596a59c700 1 -- 172.21.15.3:0/2846314389 learned_addr learned my addr 172.21.15.3:0/2846314389 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:15.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.771+0000 7f596a59c700 1 -- 172.21.15.3:0/2846314389 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f596c09e440 msgr2=0x7f596c134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:15.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.771+0000 7f596a59c700 1 --2- 172.21.15.3:0/2846314389 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f596c09e440 0x7f596c134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:15.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.771+0000 7f596a59c700 1 -- 172.21.15.3:0/2846314389 >> v1:172.21.15.3:6789/0 conn(0x7f596c09e000 legacy=0x7f596c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:15.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.771+0000 7f596a59c700 1 -- 172.21.15.3:0/2846314389 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c09a500 con 0x7f596c09d6f0 2023-10-03T10:14:15.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.771+0000 7f596ad9d700 1 --2- 172.21.15.3:0/2846314389 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f596c09e440 0x7f596c134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:15.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.771+0000 7f596a59c700 1 --2- 172.21.15.3:0/2846314389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f596c09d6f0 0x7f596c09dac0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f595c009920 tx=0x7f595c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b7499929a754b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:15.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.772+0000 7f596959a700 1 -- 172.21.15.3:0/2846314389 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c011830 con 0x7f596c09d6f0 2023-10-03T10:14:15.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.772+0000 7f596959a700 1 -- 172.21.15.3:0/2846314389 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595c011990 con 0x7f596c09d6f0 2023-10-03T10:14:15.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.772+0000 7f596959a700 1 -- 172.21.15.3:0/2846314389 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c011d20 con 0x7f596c09d6f0 2023-10-03T10:14:15.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.772+0000 7f5970c8d700 1 -- 172.21.15.3:0/2846314389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f596c09d6f0 msgr2=0x7f596c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:15.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.772+0000 7f5970c8d700 1 --2- 172.21.15.3:0/2846314389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f596c09d6f0 0x7f596c09dac0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f595c009920 tx=0x7f595c009c30 comp rx=0 tx=0).stop 2023-10-03T10:14:15.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.772+0000 7f5970c8d700 1 -- 172.21.15.3:0/2846314389 shutdown_connections 2023-10-03T10:14:15.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.772+0000 7f5970c8d700 1 --2- 172.21.15.3:0/2846314389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f596c09d6f0 0x7f596c09dac0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:15.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.772+0000 7f5970c8d700 1 --2- 172.21.15.3:0/2846314389 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f596c09e440 0x7f596c134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:15.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.772+0000 7f5970c8d700 1 -- 172.21.15.3:0/2846314389 >> 172.21.15.3:0/2846314389 conn(0x7f596c066f50 msgr2=0x7f596c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:15.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.772+0000 7f5970c8d700 1 -- 172.21.15.3:0/2846314389 shutdown_connections 2023-10-03T10:14:15.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.772+0000 7f5970c8d700 1 -- 172.21.15.3:0/2846314389 wait complete. 2023-10-03T10:14:15.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.774+0000 7f5970c8d700 1 Processor -- start 2023-10-03T10:14:15.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.774+0000 7f5970c8d700 1 -- start start 2023-10-03T10:14:15.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.774+0000 7f5970c8d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f596c09d6f0 0x7f596c1c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:15.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.774+0000 7f5970c8d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f596c09e000 0x7f596c1c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:15.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.774+0000 7f5970c8d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f596c09e440 0x7f596c1c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:15.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.774+0000 7f5970c8d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c1a7170 con 0x7f596c09e000 2023-10-03T10:14:15.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.774+0000 7f5970c8d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596c1a8bd0 con 0x7f596c09e440 2023-10-03T10:14:15.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.774+0000 7f5970c8d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f596c1a9280 con 0x7f596c09d6f0 2023-10-03T10:14:15.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.774+0000 7f596a59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f596c09d6f0 0x7f596c1c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:15.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.774+0000 7f596a59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f596c09d6f0 0x7f596c1c85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52652/0 (socket says 172.21.15.3:52652) 2023-10-03T10:14:15.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.774+0000 7f596ad9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f596c09e440 0x7f596c1c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:15.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.774+0000 7f596a59c700 1 -- 172.21.15.3:0/1410994676 learned_addr learned my addr 172.21.15.3:0/1410994676 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:15.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.774+0000 7f5969d9b700 1 --2- 172.21.15.3:0/1410994676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f596c09e000 0x7f596c1c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:15.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.775+0000 7f596a59c700 1 -- 172.21.15.3:0/1410994676 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f596c09e440 msgr2=0x7f596c1c4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:15.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.775+0000 7f596a59c700 1 --2- 172.21.15.3:0/1410994676 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f596c09e440 0x7f596c1c4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:15.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.775+0000 7f596a59c700 1 -- 172.21.15.3:0/1410994676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f596c09e000 msgr2=0x7f596c1c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:15.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.775+0000 7f596a59c700 1 --2- 172.21.15.3:0/1410994676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f596c09e000 0x7f596c1c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:15.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.775+0000 7f596a59c700 1 -- 172.21.15.3:0/1410994676 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5964009a70 con 0x7f596c09d6f0 2023-10-03T10:14:15.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.775+0000 7f5969d9b700 1 --2- 172.21.15.3:0/1410994676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f596c09e000 0x7f596c1c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:15.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.775+0000 7f596ad9d700 1 --2- 172.21.15.3:0/1410994676 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f596c09e440 0x7f596c1c4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:15.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.775+0000 7f596a59c700 1 --2- 172.21.15.3:0/1410994676 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f596c09d6f0 0x7f596c1c85f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f595c000c00 tx=0x7f595c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:15.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.775+0000 7f5962ffd700 1 -- 172.21.15.3:0/1410994676 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c0042c0 con 0x7f596c09d6f0 2023-10-03T10:14:15.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.775+0000 7f5962ffd700 1 -- 172.21.15.3:0/1410994676 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595c01f070 con 0x7f596c09d6f0 2023-10-03T10:14:15.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.775+0000 7f5970c8d700 1 -- 172.21.15.3:0/1410994676 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f595c009550 con 0x7f596c09d6f0 2023-10-03T10:14:15.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.775+0000 7f5962ffd700 1 -- 172.21.15.3:0/1410994676 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c0036a0 con 0x7f596c09d6f0 2023-10-03T10:14:15.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.775+0000 7f5970c8d700 1 -- 172.21.15.3:0/1410994676 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f596c1a8710 con 0x7f596c09d6f0 2023-10-03T10:14:15.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.776+0000 7f5962ffd700 1 -- 172.21.15.3:0/1410994676 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f595c021030 con 0x7f596c09d6f0 2023-10-03T10:14:15.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.776+0000 7f5970c8d700 1 -- 172.21.15.3:0/1410994676 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f596c074ea0 con 0x7f596c09d6f0 2023-10-03T10:14:15.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.776+0000 7f5962ffd700 1 -- 172.21.15.3:0/1410994676 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f595c040880 con 0x7f596c09d6f0 2023-10-03T10:14:15.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.781+0000 7f5962ffd700 1 -- 172.21.15.3:0/1410994676 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f595c015070 con 0x7f596c09d6f0 2023-10-03T10:14:15.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.986+0000 7f5970c8d700 1 -- 172.21.15.3:0/1410994676 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f596c06c360 con 0x7f596c09d6f0 2023-10-03T10:14:15.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.987+0000 7f5962ffd700 1 -- 172.21.15.3:0/1410994676 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f595c0239c0 con 0x7f596c09d6f0 2023-10-03T10:14:15.987 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:15.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.989+0000 7f5970c8d700 1 -- 172.21.15.3:0/1410994676 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f596c09d6f0 msgr2=0x7f596c1c85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:15.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.989+0000 7f5970c8d700 1 --2- 172.21.15.3:0/1410994676 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f596c09d6f0 0x7f596c1c85f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f595c000c00 tx=0x7f595c004750 comp rx=0 tx=0).stop 2023-10-03T10:14:15.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.989+0000 7f5970c8d700 1 -- 172.21.15.3:0/1410994676 shutdown_connections 2023-10-03T10:14:15.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.989+0000 7f5970c8d700 1 --2- 172.21.15.3:0/1410994676 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f596c09e440 0x7f596c1c4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:15.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.989+0000 7f5970c8d700 1 --2- 172.21.15.3:0/1410994676 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f596c09d6f0 0x7f596c1c85f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:15.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.989+0000 7f5970c8d700 1 --2- 172.21.15.3:0/1410994676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f596c09e000 0x7f596c1c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:15.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.989+0000 7f5970c8d700 1 -- 172.21.15.3:0/1410994676 >> 172.21.15.3:0/1410994676 conn(0x7f596c066f50 msgr2=0x7f596c1bac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:15.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.989+0000 7f5970c8d700 1 -- 172.21.15.3:0/1410994676 shutdown_connections 2023-10-03T10:14:15.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:15.989+0000 7f5970c8d700 1 -- 172.21.15.3:0/1410994676 wait complete. 2023-10-03T10:14:16.003 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:16.181 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:16.292 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:16.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.332+0000 7f3ad3334700 1 Processor -- start 2023-10-03T10:14:16.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.332+0000 7f3ad3334700 1 -- start start 2023-10-03T10:14:16.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.333+0000 7f3ad3334700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3acc0a1510 0x7f3acc138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:16.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.333+0000 7f3ad3334700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3acc139250 0x7f3acc13b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:16.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.333+0000 7f3ad3334700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3acc09edd0 con 0x7f3acc0a0a30 2023-10-03T10:14:16.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.333+0000 7f3ad3334700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc0a18e0 con 0x7f3acc0a1510 2023-10-03T10:14:16.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.333+0000 7f3ad3334700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc09e1f0 con 0x7f3acc139250 2023-10-03T10:14:16.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.333+0000 7f3ad08cf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3acc0a1510 0x7f3acc138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:16.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.333+0000 7f3ad18d1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3acc139250 0x7f3acc13b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:16.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.333+0000 7f3ad10d0700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3acc0a0a30 0x7f3acc0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44036/0 (socket says 172.21.15.3:44036) 2023-10-03T10:14:16.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.333+0000 7f3ad10d0700 1 -- 172.21.15.3:0/3020831827 learned_addr learned my addr 172.21.15.3:0/3020831827 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:16.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.333+0000 7f3acbfff700 1 -- 172.21.15.3:0/3020831827 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1572497951 0 0) 0x7f3acc09edd0 con 0x7f3acc0a0a30 2023-10-03T10:14:16.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.333+0000 7f3ad18d1700 1 -- 172.21.15.3:0/3020831827 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3acc0a1510 msgr2=0x7f3acc138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:16.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.333+0000 7f3ad18d1700 1 --2- 172.21.15.3:0/3020831827 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3acc0a1510 0x7f3acc138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:16.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.333+0000 7f3ad18d1700 1 -- 172.21.15.3:0/3020831827 >> v1:172.21.15.3:6789/0 conn(0x7f3acc0a0a30 legacy=0x7f3acc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:16.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.333+0000 7f3ad18d1700 1 -- 172.21.15.3:0/3020831827 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3acc09dc50 con 0x7f3acc139250 2023-10-03T10:14:16.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.333+0000 7f3ad08cf700 1 --2- 172.21.15.3:0/3020831827 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3acc0a1510 0x7f3acc138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:16.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.334+0000 7f3ad18d1700 1 --2- 172.21.15.3:0/3020831827 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3acc139250 0x7f3acc13b630 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009dc0 tx=0x7f3abc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8813113897553af4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:16.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.334+0000 7f3acbfff700 1 -- 172.21.15.3:0/3020831827 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc00e070 con 0x7f3acc139250 2023-10-03T10:14:16.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.334+0000 7f3acbfff700 1 -- 172.21.15.3:0/3020831827 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3abc00bb20 con 0x7f3acc139250 2023-10-03T10:14:16.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.334+0000 7f3acbfff700 1 -- 172.21.15.3:0/3020831827 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc012430 con 0x7f3acc139250 2023-10-03T10:14:16.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.334+0000 7f3ad3334700 1 -- 172.21.15.3:0/3020831827 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3acc139250 msgr2=0x7f3acc13b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:16.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.334+0000 7f3ad3334700 1 --2- 172.21.15.3:0/3020831827 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3acc139250 0x7f3acc13b630 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009dc0 tx=0x7f3abc00b330 comp rx=0 tx=0).stop 2023-10-03T10:14:16.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.334+0000 7f3ad3334700 1 -- 172.21.15.3:0/3020831827 shutdown_connections 2023-10-03T10:14:16.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.334+0000 7f3ad3334700 1 --2- 172.21.15.3:0/3020831827 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3acc139250 0x7f3acc13b630 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:16.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.334+0000 7f3ad3334700 1 --2- 172.21.15.3:0/3020831827 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3acc0a1510 0x7f3acc138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:16.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.334+0000 7f3ad3334700 1 -- 172.21.15.3:0/3020831827 >> 172.21.15.3:0/3020831827 conn(0x7f3acc0a6550 msgr2=0x7f3acc0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:16.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.335+0000 7f3ad3334700 1 -- 172.21.15.3:0/3020831827 shutdown_connections 2023-10-03T10:14:16.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.335+0000 7f3ad3334700 1 -- 172.21.15.3:0/3020831827 wait complete. 2023-10-03T10:14:16.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.336+0000 7f3ad3334700 1 Processor -- start 2023-10-03T10:14:16.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.336+0000 7f3ad3334700 1 -- start start 2023-10-03T10:14:16.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.337+0000 7f3ad3334700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3acc0a0a30 0x7f3acc11e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:16.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.337+0000 7f3ad3334700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3acc0a1510 0x7f3acc11ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:16.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.337+0000 7f3ad3334700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3acc139250 0x7f3acc11f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:16.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.337+0000 7f3ad3334700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc120b00 con 0x7f3acc0a1510 2023-10-03T10:14:16.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.337+0000 7f3ad3334700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc122140 con 0x7f3acc139250 2023-10-03T10:14:16.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.337+0000 7f3ad3334700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc09c9e0 con 0x7f3acc0a0a30 2023-10-03T10:14:16.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.337+0000 7f3ad18d1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3acc139250 0x7f3acc11f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:16.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.337+0000 7f3ad18d1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3acc139250 0x7f3acc11f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56032/0 (socket says 172.21.15.3:56032) 2023-10-03T10:14:16.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.337+0000 7f3ad18d1700 1 -- 172.21.15.3:0/915118440 learned_addr learned my addr 172.21.15.3:0/915118440 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:16.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.337+0000 7f3ad08cf700 1 --2- 172.21.15.3:0/915118440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3acc0a1510 0x7f3acc11ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:16.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.337+0000 7f3ad10d0700 1 --2- 172.21.15.3:0/915118440 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3acc0a0a30 0x7f3acc11e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:16.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.337+0000 7f3ad18d1700 1 -- 172.21.15.3:0/915118440 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3acc0a0a30 msgr2=0x7f3acc11e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:16.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.337+0000 7f3ad18d1700 1 --2- 172.21.15.3:0/915118440 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3acc0a0a30 0x7f3acc11e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:16.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.337+0000 7f3ad18d1700 1 -- 172.21.15.3:0/915118440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3acc0a1510 msgr2=0x7f3acc11ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:16.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.337+0000 7f3ad18d1700 1 --2- 172.21.15.3:0/915118440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3acc0a1510 0x7f3acc11ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:16.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.337+0000 7f3ad18d1700 1 -- 172.21.15.3:0/915118440 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac00094a0 con 0x7f3acc139250 2023-10-03T10:14:16.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.337+0000 7f3ad18d1700 1 --2- 172.21.15.3:0/915118440 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3acc139250 0x7f3acc11f1c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3acc09e380 tx=0x7f3abc004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:16.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.338+0000 7f3ac9ffb700 1 -- 172.21.15.3:0/915118440 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc012430 con 0x7f3acc139250 2023-10-03T10:14:16.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.338+0000 7f3ac9ffb700 1 -- 172.21.15.3:0/915118440 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3abc020070 con 0x7f3acc139250 2023-10-03T10:14:16.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.338+0000 7f3ad3334700 1 -- 172.21.15.3:0/915118440 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3abc009a70 con 0x7f3acc139250 2023-10-03T10:14:16.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.338+0000 7f3ac9ffb700 1 -- 172.21.15.3:0/915118440 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc004280 con 0x7f3acc139250 2023-10-03T10:14:16.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.338+0000 7f3ad3334700 1 -- 172.21.15.3:0/915118440 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3acc120230 con 0x7f3acc139250 2023-10-03T10:14:16.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.339+0000 7f3ac9ffb700 1 -- 172.21.15.3:0/915118440 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3abc0043e0 con 0x7f3acc139250 2023-10-03T10:14:16.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.339+0000 7f3ac9ffb700 1 -- 172.21.15.3:0/915118440 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3abc022070 con 0x7f3acc139250 2023-10-03T10:14:16.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.339+0000 7f3ad3334700 1 -- 172.21.15.3:0/915118440 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3acc121ce0 con 0x7f3acc139250 2023-10-03T10:14:16.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.344+0000 7f3ac9ffb700 1 -- 172.21.15.3:0/915118440 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3abc02c080 con 0x7f3acc139250 2023-10-03T10:14:16.346 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:16.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.434+0000 7f7f00b7e700 1 Processor -- start 2023-10-03T10:14:16.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.434+0000 7f7f00b7e700 1 -- start start 2023-10-03T10:14:16.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.435+0000 7f7f00b7e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7efc09e1d0 0x7f7efc138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:16.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.435+0000 7f7f00b7e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7efc139170 0x7f7efc13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:16.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.435+0000 7f7f00b7e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7efc09b120 con 0x7f7efc09d6f0 2023-10-03T10:14:16.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.435+0000 7f7f00b7e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc06b460 con 0x7f7efc139170 2023-10-03T10:14:16.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.435+0000 7f7f00b7e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc09a7e0 con 0x7f7efc09e1d0 2023-10-03T10:14:16.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.435+0000 7f7ef9d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7efc09e1d0 0x7f7efc138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:16.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.435+0000 7f7efad9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7efc139170 0x7f7efc13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:16.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.435+0000 7f7ef9d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7efc09e1d0 0x7f7efc138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52678/0 (socket says 172.21.15.3:52678) 2023-10-03T10:14:16.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.435+0000 7f7efad9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7efc139170 0x7f7efc13b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56034/0 (socket says 172.21.15.3:56034) 2023-10-03T10:14:16.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.435+0000 7f7efa59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7efc09d6f0 0x7f7efc09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44044/0 (socket says 172.21.15.3:44044) 2023-10-03T10:14:16.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.435+0000 7f7ef9d9b700 1 -- 172.21.15.3:0/1550208124 learned_addr learned my addr 172.21.15.3:0/1550208124 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:16.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.435+0000 7f7efad9d700 1 -- 172.21.15.3:0/1550208124 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7efc09e1d0 msgr2=0x7f7efc138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:16.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.435+0000 7f7efad9d700 1 --2- 172.21.15.3:0/1550208124 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7efc09e1d0 0x7f7efc138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:16.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.435+0000 7f7efad9d700 1 -- 172.21.15.3:0/1550208124 >> v1:172.21.15.3:6789/0 conn(0x7f7efc09d6f0 legacy=0x7f7efc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:16.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.435+0000 7f7efad9d700 1 -- 172.21.15.3:0/1550208124 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7efc09a500 con 0x7f7efc139170 2023-10-03T10:14:16.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.435+0000 7f7ef959a700 1 -- 172.21.15.3:0/1550208124 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1719084919 0 0) 0x7f7efc09b120 con 0x7f7efc09d6f0 2023-10-03T10:14:16.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.435+0000 7f7ef9d9b700 1 --2- 172.21.15.3:0/1550208124 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7efc09e1d0 0x7f7efc138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:16.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.436+0000 7f7efad9d700 1 --2- 172.21.15.3:0/1550208124 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7efc139170 0x7f7efc13b550 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009dc0 tx=0x7f7ef400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c7cddc92198a4cf server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:16.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.436+0000 7f7ef959a700 1 -- 172.21.15.3:0/1550208124 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef400e070 con 0x7f7efc139170 2023-10-03T10:14:16.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.436+0000 7f7ef959a700 1 -- 172.21.15.3:0/1550208124 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef400bb20 con 0x7f7efc139170 2023-10-03T10:14:16.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.436+0000 7f7ef959a700 1 -- 172.21.15.3:0/1550208124 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4012430 con 0x7f7efc139170 2023-10-03T10:14:16.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.436+0000 7f7f00b7e700 1 -- 172.21.15.3:0/1550208124 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7efc139170 msgr2=0x7f7efc13b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:16.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.436+0000 7f7f00b7e700 1 --2- 172.21.15.3:0/1550208124 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7efc139170 0x7f7efc13b550 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009dc0 tx=0x7f7ef400b330 comp rx=0 tx=0).stop 2023-10-03T10:14:16.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.437+0000 7f7f00b7e700 1 -- 172.21.15.3:0/1550208124 shutdown_connections 2023-10-03T10:14:16.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.437+0000 7f7f00b7e700 1 --2- 172.21.15.3:0/1550208124 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7efc139170 0x7f7efc13b550 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:16.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.437+0000 7f7f00b7e700 1 --2- 172.21.15.3:0/1550208124 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7efc09e1d0 0x7f7efc138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:16.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.437+0000 7f7f00b7e700 1 -- 172.21.15.3:0/1550208124 >> 172.21.15.3:0/1550208124 conn(0x7f7efc066f50 msgr2=0x7f7efc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:16.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.437+0000 7f7f00b7e700 1 -- 172.21.15.3:0/1550208124 shutdown_connections 2023-10-03T10:14:16.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.437+0000 7f7f00b7e700 1 -- 172.21.15.3:0/1550208124 wait complete. 2023-10-03T10:14:16.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.438+0000 7f7f00b7e700 1 Processor -- start 2023-10-03T10:14:16.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.438+0000 7f7f00b7e700 1 -- start start 2023-10-03T10:14:16.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.438+0000 7f7f00b7e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7efc09d6f0 0x7f7efc1205f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:16.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.438+0000 7f7f00b7e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7efc09e1d0 0x7f7efc120b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:16.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.438+0000 7f7f00b7e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7efc139170 0x7f7efc121070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:16.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.438+0000 7f7f00b7e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc123f30 con 0x7f7efc139170 2023-10-03T10:14:16.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.438+0000 7f7f00b7e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc124110 con 0x7f7efc09d6f0 2023-10-03T10:14:16.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.438+0000 7f7f00b7e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc1230a0 con 0x7f7efc09e1d0 2023-10-03T10:14:16.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.439+0000 7f7efa59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7efc09d6f0 0x7f7efc1205f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:16.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.439+0000 7f7efa59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7efc09d6f0 0x7f7efc1205f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56048/0 (socket says 172.21.15.3:56048) 2023-10-03T10:14:16.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.439+0000 7f7efa59c700 1 -- 172.21.15.3:0/3062104483 learned_addr learned my addr 172.21.15.3:0/3062104483 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:16.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.439+0000 7f7efad9d700 1 --2- 172.21.15.3:0/3062104483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7efc139170 0x7f7efc121070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:16.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.439+0000 7f7ef9d9b700 1 --2- 172.21.15.3:0/3062104483 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7efc09e1d0 0x7f7efc120b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:16.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.439+0000 7f7efad9d700 1 -- 172.21.15.3:0/3062104483 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7efc09e1d0 msgr2=0x7f7efc120b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:16.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.439+0000 7f7efad9d700 1 --2- 172.21.15.3:0/3062104483 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7efc09e1d0 0x7f7efc120b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:16.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.439+0000 7f7efad9d700 1 -- 172.21.15.3:0/3062104483 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7efc09d6f0 msgr2=0x7f7efc1205f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:16.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.439+0000 7f7efad9d700 1 --2- 172.21.15.3:0/3062104483 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7efc09d6f0 0x7f7efc1205f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:16.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.439+0000 7f7efad9d700 1 -- 172.21.15.3:0/3062104483 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef0009550 con 0x7f7efc139170 2023-10-03T10:14:16.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.439+0000 7f7efa59c700 1 --2- 172.21.15.3:0/3062104483 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7efc09d6f0 0x7f7efc1205f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:16.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.439+0000 7f7efad9d700 1 --2- 172.21.15.3:0/3062104483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7efc139170 0x7f7efc121070 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4016000 tx=0x7f7ef4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:16.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.440+0000 7f7eeaffd700 1 -- 172.21.15.3:0/3062104483 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef401b810 con 0x7f7efc139170 2023-10-03T10:14:16.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.440+0000 7f7eeaffd700 1 -- 172.21.15.3:0/3062104483 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef4020070 con 0x7f7efc139170 2023-10-03T10:14:16.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.440+0000 7f7f00b7e700 1 -- 172.21.15.3:0/3062104483 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef4009a70 con 0x7f7efc139170 2023-10-03T10:14:16.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.440+0000 7f7eeaffd700 1 -- 172.21.15.3:0/3062104483 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef401bbd0 con 0x7f7efc139170 2023-10-03T10:14:16.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.440+0000 7f7f00b7e700 1 -- 172.21.15.3:0/3062104483 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7efc0721b0 con 0x7f7efc139170 2023-10-03T10:14:16.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.441+0000 7f7eeaffd700 1 -- 172.21.15.3:0/3062104483 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7ef401bd30 con 0x7f7efc139170 2023-10-03T10:14:16.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.441+0000 7f7eeaffd700 1 -- 172.21.15.3:0/3062104483 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7ef4022070 con 0x7f7efc139170 2023-10-03T10:14:16.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.441+0000 7f7f00b7e700 1 -- 172.21.15.3:0/3062104483 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7efc136880 con 0x7f7efc139170 2023-10-03T10:14:16.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.446+0000 7f7eeaffd700 1 -- 172.21.15.3:0/3062104483 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7ef4030390 con 0x7f7efc139170 2023-10-03T10:14:16.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.487+0000 7f9f3c2e7700 1 Processor -- start 2023-10-03T10:14:16.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.487+0000 7f9f3c2e7700 1 -- start start 2023-10-03T10:14:16.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.488+0000 7f9f3c2e7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9f3409d6f0 0x7f9f3409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:16.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.488+0000 7f9f3c2e7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9f3409e410 0x7f9f34130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:16.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.488+0000 7f9f3c2e7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f3409f130 con 0x7f9f3409e000 2023-10-03T10:14:16.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.488+0000 7f9f3c2e7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f3409f520 con 0x7f9f3409d6f0 2023-10-03T10:14:16.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.488+0000 7f9f3c2e7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3409e930 con 0x7f9f3409e410 2023-10-03T10:14:16.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.488+0000 7f9f3a083700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9f3409d6f0 0x7f9f3409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:16.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.488+0000 7f9f39882700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9f3409e000 0x7f9f34130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44048/0 (socket says 172.21.15.3:44048) 2023-10-03T10:14:16.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.488+0000 7f9f39882700 1 -- 172.21.15.3:0/615450412 learned_addr learned my addr 172.21.15.3:0/615450412 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:16.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.488+0000 7f9f3a884700 1 --2- 172.21.15.3:0/615450412 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9f3409e410 0x7f9f34130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:16.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.489+0000 7f9f3a083700 1 -- 172.21.15.3:0/615450412 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9f3409e410 msgr2=0x7f9f34130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:16.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.489+0000 7f9f3a083700 1 --2- 172.21.15.3:0/615450412 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9f3409e410 0x7f9f34130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:16.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.489+0000 7f9f3a083700 1 -- 172.21.15.3:0/615450412 >> v1:172.21.15.3:6789/0 conn(0x7f9f3409e000 legacy=0x7f9f34130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:16.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.489+0000 7f9f3a083700 1 -- 172.21.15.3:0/615450412 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f3409b2d0 con 0x7f9f3409d6f0 2023-10-03T10:14:16.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.489+0000 7f9f3a884700 1 --2- 172.21.15.3:0/615450412 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9f3409e410 0x7f9f34130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:16.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.489+0000 7f9f3a083700 1 --2- 172.21.15.3:0/615450412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f3409d6f0 0x7f9f3409dac0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000bae0 tx=0x7f9f3000bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e72a0907578d7b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:16.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.489+0000 7f9f39081700 1 -- 172.21.15.3:0/615450412 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3000dc70 con 0x7f9f3409d6f0 2023-10-03T10:14:16.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.489+0000 7f9f39081700 1 -- 172.21.15.3:0/615450412 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3000ddd0 con 0x7f9f3409d6f0 2023-10-03T10:14:16.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.489+0000 7f9f39081700 1 -- 172.21.15.3:0/615450412 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f30010690 con 0x7f9f3409d6f0 2023-10-03T10:14:16.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.490+0000 7f9f3c2e7700 1 -- 172.21.15.3:0/615450412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f3409d6f0 msgr2=0x7f9f3409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:16.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.490+0000 7f9f3c2e7700 1 --2- 172.21.15.3:0/615450412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f3409d6f0 0x7f9f3409dac0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000bae0 tx=0x7f9f3000bdf0 comp rx=0 tx=0).stop 2023-10-03T10:14:16.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.490+0000 7f9f3c2e7700 1 -- 172.21.15.3:0/615450412 shutdown_connections 2023-10-03T10:14:16.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.490+0000 7f9f3c2e7700 1 --2- 172.21.15.3:0/615450412 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9f3409e410 0x7f9f34130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:16.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.490+0000 7f9f3c2e7700 1 --2- 172.21.15.3:0/615450412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f3409d6f0 0x7f9f3409dac0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:16.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.490+0000 7f9f3c2e7700 1 -- 172.21.15.3:0/615450412 >> 172.21.15.3:0/615450412 conn(0x7f9f34066f50 msgr2=0x7f9f3409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:16.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.490+0000 7f9f3c2e7700 1 -- 172.21.15.3:0/615450412 shutdown_connections 2023-10-03T10:14:16.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.490+0000 7f9f3c2e7700 1 -- 172.21.15.3:0/615450412 wait complete. 2023-10-03T10:14:16.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.491+0000 7f9f3c2e7700 1 Processor -- start 2023-10-03T10:14:16.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.491+0000 7f9f3c2e7700 1 -- start start 2023-10-03T10:14:16.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.492+0000 7f9f3c2e7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f3409d6f0 0x7f9f3411e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:16.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.492+0000 7f9f3c2e7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9f3409e000 0x7f9f3411e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:16.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.492+0000 7f9f3c2e7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9f3409e410 0x7f9f3411ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:16.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.492+0000 7f9f3c2e7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3409c770 con 0x7f9f3409d6f0 2023-10-03T10:14:16.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.492+0000 7f9f3c2e7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f340a1d00 con 0x7f9f3409e410 2023-10-03T10:14:16.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.492+0000 7f9f3c2e7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f341204c0 con 0x7f9f3409e000 2023-10-03T10:14:16.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.492+0000 7f9f3a083700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f3409d6f0 0x7f9f3411e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:16.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.492+0000 7f9f3a083700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f3409d6f0 0x7f9f3411e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40918/0 (socket says 172.21.15.3:40918) 2023-10-03T10:14:16.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.492+0000 7f9f3a083700 1 -- 172.21.15.3:0/2836168801 learned_addr learned my addr 172.21.15.3:0/2836168801 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:16.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.492+0000 7f9f39882700 1 --2- 172.21.15.3:0/2836168801 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9f3409e000 0x7f9f3411e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:16.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.492+0000 7f9f3a884700 1 --2- 172.21.15.3:0/2836168801 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9f3409e410 0x7f9f3411ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:16.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.493+0000 7f9f3a083700 1 -- 172.21.15.3:0/2836168801 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9f3409e000 msgr2=0x7f9f3411e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:16.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.493+0000 7f9f3a083700 1 --2- 172.21.15.3:0/2836168801 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9f3409e000 0x7f9f3411e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:16.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.493+0000 7f9f3a083700 1 -- 172.21.15.3:0/2836168801 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9f3409e410 msgr2=0x7f9f3411ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:16.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.493+0000 7f9f3a083700 1 --2- 172.21.15.3:0/2836168801 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9f3409e410 0x7f9f3411ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:16.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.493+0000 7f9f3a083700 1 -- 172.21.15.3:0/2836168801 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2c009a70 con 0x7f9f3409d6f0 2023-10-03T10:14:16.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.493+0000 7f9f3a083700 1 --2- 172.21.15.3:0/2836168801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f3409d6f0 0x7f9f3411e050 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000b2c0 tx=0x7f9f30004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:16.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.493+0000 7f9f2affd700 1 -- 172.21.15.3:0/2836168801 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3001f070 con 0x7f9f3409d6f0 2023-10-03T10:14:16.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.493+0000 7f9f2affd700 1 -- 172.21.15.3:0/2836168801 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f300038b0 con 0x7f9f3409d6f0 2023-10-03T10:14:16.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.493+0000 7f9f3c2e7700 1 -- 172.21.15.3:0/2836168801 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f3000b740 con 0x7f9f3409d6f0 2023-10-03T10:14:16.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.493+0000 7f9f2affd700 1 -- 172.21.15.3:0/2836168801 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f30003c40 con 0x7f9f3409d6f0 2023-10-03T10:14:16.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.493+0000 7f9f3c2e7700 1 -- 172.21.15.3:0/2836168801 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f341215d0 con 0x7f9f3409d6f0 2023-10-03T10:14:16.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.494+0000 7f9f2affd700 1 -- 172.21.15.3:0/2836168801 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9f30019400 con 0x7f9f3409d6f0 2023-10-03T10:14:16.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.494+0000 7f9f2affd700 1 -- 172.21.15.3:0/2836168801 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9f30042af0 con 0x7f9f3409d6f0 2023-10-03T10:14:16.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.494+0000 7f9f3c2e7700 1 -- 172.21.15.3:0/2836168801 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f34074ea0 con 0x7f9f3409d6f0 2023-10-03T10:14:16.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.500+0000 7f9f2affd700 1 -- 172.21.15.3:0/2836168801 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9f30003da0 con 0x7f9f3409d6f0 2023-10-03T10:14:16.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.578+0000 7f3ad3334700 1 -- 172.21.15.3:0/915118440 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3acc121b00 con 0x7f3acc139250 2023-10-03T10:14:16.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.578+0000 7f3ac9ffb700 1 -- 172.21.15.3:0/915118440 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3acc121b00 con 0x7f3acc139250 2023-10-03T10:14:16.579 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:16.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.580+0000 7f3ad3334700 1 -- 172.21.15.3:0/915118440 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3acc139250 msgr2=0x7f3acc11f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:16.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.580+0000 7f3ad3334700 1 --2- 172.21.15.3:0/915118440 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3acc139250 0x7f3acc11f1c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3acc09e380 tx=0x7f3abc004ad0 comp rx=0 tx=0).stop 2023-10-03T10:14:16.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.580+0000 7f3ad3334700 1 -- 172.21.15.3:0/915118440 shutdown_connections 2023-10-03T10:14:16.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.580+0000 7f3ad3334700 1 --2- 172.21.15.3:0/915118440 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3acc139250 0x7f3acc11f1c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:16.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.580+0000 7f3ad3334700 1 --2- 172.21.15.3:0/915118440 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3acc0a0a30 0x7f3acc11e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:16.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.580+0000 7f3ad3334700 1 --2- 172.21.15.3:0/915118440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3acc0a1510 0x7f3acc11ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:16.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.580+0000 7f3ad3334700 1 -- 172.21.15.3:0/915118440 >> 172.21.15.3:0/915118440 conn(0x7f3acc0a6550 msgr2=0x7f3acc0a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:16.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.580+0000 7f3ad3334700 1 -- 172.21.15.3:0/915118440 shutdown_connections 2023-10-03T10:14:16.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.581+0000 7f3ad3334700 1 -- 172.21.15.3:0/915118440 wait complete. 2023-10-03T10:14:16.595 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:16.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.678+0000 7f7f00b7e700 1 -- 172.21.15.3:0/3062104483 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7efc074ea0 con 0x7f7efc139170 2023-10-03T10:14:16.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.679+0000 7f7eeaffd700 1 -- 172.21.15.3:0/3062104483 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7ef4036090 con 0x7f7efc139170 2023-10-03T10:14:16.679 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:16.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.681+0000 7f7f00b7e700 1 -- 172.21.15.3:0/3062104483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7efc139170 msgr2=0x7f7efc121070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:16.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.681+0000 7f7f00b7e700 1 --2- 172.21.15.3:0/3062104483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7efc139170 0x7f7efc121070 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4016000 tx=0x7f7ef4004ad0 comp rx=0 tx=0).stop 2023-10-03T10:14:16.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.681+0000 7f7f00b7e700 1 -- 172.21.15.3:0/3062104483 shutdown_connections 2023-10-03T10:14:16.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.681+0000 7f7f00b7e700 1 --2- 172.21.15.3:0/3062104483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7efc139170 0x7f7efc121070 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:16.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.681+0000 7f7f00b7e700 1 --2- 172.21.15.3:0/3062104483 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7efc09d6f0 0x7f7efc1205f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:16.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.681+0000 7f7f00b7e700 1 --2- 172.21.15.3:0/3062104483 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7efc09e1d0 0x7f7efc120b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:16.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.681+0000 7f7f00b7e700 1 -- 172.21.15.3:0/3062104483 >> 172.21.15.3:0/3062104483 conn(0x7f7efc066f50 msgr2=0x7f7efc070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:16.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.681+0000 7f7f00b7e700 1 -- 172.21.15.3:0/3062104483 shutdown_connections 2023-10-03T10:14:16.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.681+0000 7f7f00b7e700 1 -- 172.21.15.3:0/3062104483 wait complete. 2023-10-03T10:14:16.695 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:16.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.708+0000 7f9f3c2e7700 1 -- 172.21.15.3:0/2836168801 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9f3406c360 con 0x7f9f3409d6f0 2023-10-03T10:14:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.709+0000 7f9f2affd700 1 -- 172.21.15.3:0/2836168801 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9f30023bc0 con 0x7f9f3409d6f0 2023-10-03T10:14:16.709 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:16.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.711+0000 7f9f3c2e7700 1 -- 172.21.15.3:0/2836168801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f3409d6f0 msgr2=0x7f9f3411e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:16.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.711+0000 7f9f3c2e7700 1 --2- 172.21.15.3:0/2836168801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f3409d6f0 0x7f9f3411e050 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000b2c0 tx=0x7f9f30004750 comp rx=0 tx=0).stop 2023-10-03T10:14:16.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.711+0000 7f9f3c2e7700 1 -- 172.21.15.3:0/2836168801 shutdown_connections 2023-10-03T10:14:16.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.711+0000 7f9f3c2e7700 1 --2- 172.21.15.3:0/2836168801 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9f3409e410 0x7f9f3411ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:16.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.711+0000 7f9f3c2e7700 1 --2- 172.21.15.3:0/2836168801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f3409d6f0 0x7f9f3411e050 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:16.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.711+0000 7f9f3c2e7700 1 --2- 172.21.15.3:0/2836168801 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9f3409e000 0x7f9f3411e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:16.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.711+0000 7f9f3c2e7700 1 -- 172.21.15.3:0/2836168801 >> 172.21.15.3:0/2836168801 conn(0x7f9f34066f50 msgr2=0x7f9f340710c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:16.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.711+0000 7f9f3c2e7700 1 -- 172.21.15.3:0/2836168801 shutdown_connections 2023-10-03T10:14:16.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:16.711+0000 7f9f3c2e7700 1 -- 172.21.15.3:0/2836168801 wait complete. 2023-10-03T10:14:16.725 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:17.004 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:17.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.150+0000 7fae12f99700 1 Processor -- start 2023-10-03T10:14:17.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.150+0000 7fae12f99700 1 -- start start 2023-10-03T10:14:17.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.151+0000 7fae12f99700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae0c09d6f0 0x7fae0c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:17.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.151+0000 7fae12f99700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fae0c09e410 0x7fae0c130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:17.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.151+0000 7fae12f99700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae0c09f130 con 0x7fae0c09e000 2023-10-03T10:14:17.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.151+0000 7fae12f99700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae0c09f520 con 0x7fae0c09e410 2023-10-03T10:14:17.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.151+0000 7fae12f99700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c09e930 con 0x7fae0c09d6f0 2023-10-03T10:14:17.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.151+0000 7fae11536700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fae0c09e410 0x7fae0c130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:17.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.151+0000 7fae10d35700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae0c09d6f0 0x7fae0c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:17.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.151+0000 7fae10d35700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae0c09d6f0 0x7fae0c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52698/0 (socket says 172.21.15.3:52698) 2023-10-03T10:14:17.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.151+0000 7fae11536700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fae0c09e410 0x7fae0c130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40934/0 (socket says 172.21.15.3:40934) 2023-10-03T10:14:17.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.151+0000 7fae10d35700 1 -- 172.21.15.3:0/688160112 learned_addr learned my addr 172.21.15.3:0/688160112 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:17.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.151+0000 7fae11536700 1 -- 172.21.15.3:0/688160112 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae0c09d6f0 msgr2=0x7fae0c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:17.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.151+0000 7fae11536700 1 --2- 172.21.15.3:0/688160112 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae0c09d6f0 0x7fae0c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.151+0000 7fae11536700 1 -- 172.21.15.3:0/688160112 >> v1:172.21.15.3:6789/0 conn(0x7fae0c09e000 legacy=0x7fae0c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:17.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.151+0000 7fae0b7fe700 1 -- 172.21.15.3:0/688160112 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3514722274 0 0) 0x7fae0c09f130 con 0x7fae0c09e000 2023-10-03T10:14:17.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.151+0000 7fae11536700 1 -- 172.21.15.3:0/688160112 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0c09b2d0 con 0x7fae0c09e410 2023-10-03T10:14:17.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.151+0000 7fae10d35700 1 --2- 172.21.15.3:0/688160112 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae0c09d6f0 0x7fae0c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:17.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.151+0000 7fae11536700 1 --2- 172.21.15.3:0/688160112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae0c09e410 0x7fae0c130d70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fae04009d90 tx=0x7fae0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90030b66afb91e64 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:17.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.151+0000 7fae0b7fe700 1 -- 172.21.15.3:0/688160112 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0400e070 con 0x7fae0c09e410 2023-10-03T10:14:17.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.152+0000 7fae0b7fe700 1 -- 172.21.15.3:0/688160112 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0400bb20 con 0x7fae0c09e410 2023-10-03T10:14:17.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.152+0000 7fae0b7fe700 1 -- 172.21.15.3:0/688160112 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae04012430 con 0x7fae0c09e410 2023-10-03T10:14:17.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.152+0000 7fae12f99700 1 -- 172.21.15.3:0/688160112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae0c09e410 msgr2=0x7fae0c130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:17.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.152+0000 7fae12f99700 1 --2- 172.21.15.3:0/688160112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae0c09e410 0x7fae0c130d70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fae04009d90 tx=0x7fae0400b330 comp rx=0 tx=0).stop 2023-10-03T10:14:17.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.152+0000 7fae12f99700 1 -- 172.21.15.3:0/688160112 shutdown_connections 2023-10-03T10:14:17.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.152+0000 7fae12f99700 1 --2- 172.21.15.3:0/688160112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae0c09e410 0x7fae0c130d70 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.152+0000 7fae12f99700 1 --2- 172.21.15.3:0/688160112 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae0c09d6f0 0x7fae0c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.152+0000 7fae12f99700 1 -- 172.21.15.3:0/688160112 >> 172.21.15.3:0/688160112 conn(0x7fae0c066f50 msgr2=0x7fae0c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:17.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.152+0000 7fae12f99700 1 -- 172.21.15.3:0/688160112 shutdown_connections 2023-10-03T10:14:17.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.152+0000 7fae12f99700 1 -- 172.21.15.3:0/688160112 wait complete. 2023-10-03T10:14:17.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.153+0000 7fae12f99700 1 Processor -- start 2023-10-03T10:14:17.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.153+0000 7fae12f99700 1 -- start start 2023-10-03T10:14:17.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.154+0000 7fae12f99700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae0c09d6f0 0x7fae0c120100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:17.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.154+0000 7fae12f99700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fae0c09e000 0x7fae0c120640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:17.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.154+0000 7fae12f99700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae0c09e410 0x7fae0c120b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:17.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.154+0000 7fae12f99700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c130390 con 0x7fae0c09d6f0 2023-10-03T10:14:17.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.154+0000 7fae12f99700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c123550 con 0x7fae0c09e000 2023-10-03T10:14:17.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.154+0000 7fae12f99700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c123810 con 0x7fae0c09e410 2023-10-03T10:14:17.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.154+0000 7fae10d35700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae0c09d6f0 0x7fae0c120100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:17.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.154+0000 7fae10d35700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae0c09d6f0 0x7fae0c120100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40936/0 (socket says 172.21.15.3:40936) 2023-10-03T10:14:17.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.154+0000 7fae0bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fae0c09e000 0x7fae0c120640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:17.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.154+0000 7fae10d35700 1 -- 172.21.15.3:0/3373816300 learned_addr learned my addr 172.21.15.3:0/3373816300 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:17.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.154+0000 7fae11536700 1 --2- 172.21.15.3:0/3373816300 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae0c09e410 0x7fae0c120b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:17.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.154+0000 7fae10d35700 1 -- 172.21.15.3:0/3373816300 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae0c09e410 msgr2=0x7fae0c120b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:17.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.154+0000 7fae10d35700 1 --2- 172.21.15.3:0/3373816300 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae0c09e410 0x7fae0c120b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.154+0000 7fae10d35700 1 -- 172.21.15.3:0/3373816300 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fae0c09e000 msgr2=0x7fae0c120640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:17.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.154+0000 7fae10d35700 1 --2- 172.21.15.3:0/3373816300 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fae0c09e000 0x7fae0c120640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.154+0000 7fae10d35700 1 -- 172.21.15.3:0/3373816300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae00009550 con 0x7fae0c09d6f0 2023-10-03T10:14:17.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.154+0000 7fae0bfff700 1 --2- 172.21.15.3:0/3373816300 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fae0c09e000 0x7fae0c120640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:17.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.155+0000 7fae10d35700 1 --2- 172.21.15.3:0/3373816300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae0c09d6f0 0x7fae0c120100 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fae0000ebe0 tx=0x7fae0000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:17.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.155+0000 7fae097fa700 1 -- 172.21.15.3:0/3373816300 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0000c850 con 0x7fae0c09d6f0 2023-10-03T10:14:17.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.155+0000 7fae097fa700 1 -- 172.21.15.3:0/3373816300 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0000c9b0 con 0x7fae0c09d6f0 2023-10-03T10:14:17.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.155+0000 7fae12f99700 1 -- 172.21.15.3:0/3373816300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae04009a70 con 0x7fae0c09d6f0 2023-10-03T10:14:17.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.155+0000 7fae097fa700 1 -- 172.21.15.3:0/3373816300 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0000cd40 con 0x7fae0c09d6f0 2023-10-03T10:14:17.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.155+0000 7fae12f99700 1 -- 172.21.15.3:0/3373816300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae0c123280 con 0x7fae0c09d6f0 2023-10-03T10:14:17.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.156+0000 7fae12f99700 1 -- 172.21.15.3:0/3373816300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae0c074ea0 con 0x7fae0c09d6f0 2023-10-03T10:14:17.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.156+0000 7fae097fa700 1 -- 172.21.15.3:0/3373816300 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fae00026480 con 0x7fae0c09d6f0 2023-10-03T10:14:17.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.156+0000 7fae097fa700 1 -- 172.21.15.3:0/3373816300 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fae00040200 con 0x7fae0c09d6f0 2023-10-03T10:14:17.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.161+0000 7fae097fa700 1 -- 172.21.15.3:0/3373816300 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fae0001ab50 con 0x7fae0c09d6f0 2023-10-03T10:14:17.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.372+0000 7fae12f99700 1 -- 172.21.15.3:0/3373816300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fae0c1223e0 con 0x7fae0c09d6f0 2023-10-03T10:14:17.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.372+0000 7fae097fa700 1 -- 172.21.15.3:0/3373816300 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fae0001ab50 con 0x7fae0c09d6f0 2023-10-03T10:14:17.373 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:17.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.374+0000 7fae12f99700 1 -- 172.21.15.3:0/3373816300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae0c09d6f0 msgr2=0x7fae0c120100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:17.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.374+0000 7fae12f99700 1 --2- 172.21.15.3:0/3373816300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae0c09d6f0 0x7fae0c120100 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fae0000ebe0 tx=0x7fae0000eef0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.374+0000 7fae12f99700 1 -- 172.21.15.3:0/3373816300 shutdown_connections 2023-10-03T10:14:17.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.374+0000 7fae12f99700 1 --2- 172.21.15.3:0/3373816300 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae0c09e410 0x7fae0c120b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.374+0000 7fae12f99700 1 --2- 172.21.15.3:0/3373816300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae0c09d6f0 0x7fae0c120100 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.374+0000 7fae12f99700 1 --2- 172.21.15.3:0/3373816300 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fae0c09e000 0x7fae0c120640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.374+0000 7fae12f99700 1 -- 172.21.15.3:0/3373816300 >> 172.21.15.3:0/3373816300 conn(0x7fae0c066f50 msgr2=0x7fae0c071110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:17.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.374+0000 7fae12f99700 1 -- 172.21.15.3:0/3373816300 shutdown_connections 2023-10-03T10:14:17.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.374+0000 7fae12f99700 1 -- 172.21.15.3:0/3373816300 wait complete. 2023-10-03T10:14:17.388 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:17.596 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:17.696 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:17.726 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:17.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.746+0000 7f17b179e700 1 Processor -- start 2023-10-03T10:14:17.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.746+0000 7f17b179e700 1 -- start start 2023-10-03T10:14:17.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.746+0000 7f17b179e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17ac09d6f0 0x7f17ac09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:17.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.746+0000 7f17b179e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f17ac09e000 0x7f17ac1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:17.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.746+0000 7f17b179e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f17ac09e410 0x7f17ac133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:17.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.746+0000 7f17b179e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17ac06b460 con 0x7f17ac09e410 2023-10-03T10:14:17.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.746+0000 7f17b179e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac09b120 con 0x7f17ac09d6f0 2023-10-03T10:14:17.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.746+0000 7f17b179e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac09a7e0 con 0x7f17ac09e000 2023-10-03T10:14:17.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.747+0000 7f17aaffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17ac09d6f0 0x7f17ac09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:17.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.747+0000 7f17aaffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17ac09d6f0 0x7f17ac09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56076/0 (socket says 172.21.15.3:56076) 2023-10-03T10:14:17.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.747+0000 7f17aaffd700 1 -- 172.21.15.3:0/4155280360 learned_addr learned my addr 172.21.15.3:0/4155280360 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:17.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.747+0000 7f17aa7fc700 1 --2- 172.21.15.3:0/4155280360 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f17ac09e000 0x7f17ac1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:17.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.747+0000 7f17ab7fe700 1 --2- 172.21.15.3:0/4155280360 >> v2:172.21.15.3:3300/0 conn(0x7f17ac09e410 0x7f17ac133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:17.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.747+0000 7f17aa7fc700 1 -- 172.21.15.3:0/4155280360 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17ac09d6f0 msgr2=0x7f17ac09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:17.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.747+0000 7f17aa7fc700 1 --2- 172.21.15.3:0/4155280360 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17ac09d6f0 0x7f17ac09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.747+0000 7f17aa7fc700 1 -- 172.21.15.3:0/4155280360 >> v2:172.21.15.3:3300/0 conn(0x7f17ac09e410 msgr2=0x7f17ac133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:17.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.747+0000 7f17aa7fc700 1 --2- 172.21.15.3:0/4155280360 >> v2:172.21.15.3:3300/0 conn(0x7f17ac09e410 0x7f17ac133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.747+0000 7f17aa7fc700 1 -- 172.21.15.3:0/4155280360 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ac09a500 con 0x7f17ac09e000 2023-10-03T10:14:17.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.747+0000 7f17aaffd700 1 --2- 172.21.15.3:0/4155280360 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17ac09d6f0 0x7f17ac09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:17.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.747+0000 7f17aa7fc700 1 --2- 172.21.15.3:0/4155280360 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f17ac09e000 0x7f17ac1339f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009870 tx=0x7f17a0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c73f32e26f51f9d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:17.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.747+0000 7f17a9ffb700 1 -- 172.21.15.3:0/4155280360 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a0011830 con 0x7f17ac09e000 2023-10-03T10:14:17.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.748+0000 7f17a9ffb700 1 -- 172.21.15.3:0/4155280360 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a0011990 con 0x7f17ac09e000 2023-10-03T10:14:17.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.748+0000 7f17a9ffb700 1 -- 172.21.15.3:0/4155280360 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a0011d20 con 0x7f17ac09e000 2023-10-03T10:14:17.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.748+0000 7f17b179e700 1 -- 172.21.15.3:0/4155280360 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f17ac09e000 msgr2=0x7f17ac1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:17.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.748+0000 7f17b179e700 1 --2- 172.21.15.3:0/4155280360 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f17ac09e000 0x7f17ac1339f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009870 tx=0x7f17a0009b80 comp rx=0 tx=0).stop 2023-10-03T10:14:17.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.748+0000 7f17b179e700 1 -- 172.21.15.3:0/4155280360 shutdown_connections 2023-10-03T10:14:17.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.748+0000 7f17b179e700 1 --2- 172.21.15.3:0/4155280360 >> v2:172.21.15.3:3300/0 conn(0x7f17ac09e410 0x7f17ac133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.748+0000 7f17b179e700 1 --2- 172.21.15.3:0/4155280360 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17ac09d6f0 0x7f17ac09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.748+0000 7f17b179e700 1 --2- 172.21.15.3:0/4155280360 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f17ac09e000 0x7f17ac1339f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.748+0000 7f17b179e700 1 -- 172.21.15.3:0/4155280360 >> 172.21.15.3:0/4155280360 conn(0x7f17ac066f50 msgr2=0x7f17ac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:17.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.748+0000 7f17b179e700 1 -- 172.21.15.3:0/4155280360 shutdown_connections 2023-10-03T10:14:17.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.748+0000 7f17b179e700 1 -- 172.21.15.3:0/4155280360 wait complete. 2023-10-03T10:14:17.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.750+0000 7f17b179e700 1 Processor -- start 2023-10-03T10:14:17.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.750+0000 7f17b179e700 1 -- start start 2023-10-03T10:14:17.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.750+0000 7f17b179e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f17ac09d6f0 0x7f17ac1a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:17.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.750+0000 7f17b179e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17ac09e000 0x7f17ac1a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:17.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.750+0000 7f17b179e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17ac09e410 0x7f17ac1a6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:17.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.750+0000 7f17b179e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac1aa270 con 0x7f17ac09e000 2023-10-03T10:14:17.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.750+0000 7f17b179e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac1a79a0 con 0x7f17ac09e410 2023-10-03T10:14:17.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.750+0000 7f17b179e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac1a7fc0 con 0x7f17ac09d6f0 2023-10-03T10:14:17.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.751+0000 7f17aaffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f17ac09d6f0 0x7f17ac1a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:17.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.751+0000 7f17aa7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17ac09e000 0x7f17ac1a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:17.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.751+0000 7f17aa7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17ac09e000 0x7f17ac1a62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40946/0 (socket says 172.21.15.3:40946) 2023-10-03T10:14:17.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.751+0000 7f17aaffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f17ac09d6f0 0x7f17ac1a5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52718/0 (socket says 172.21.15.3:52718) 2023-10-03T10:14:17.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.751+0000 7f17aa7fc700 1 -- 172.21.15.3:0/4212670775 learned_addr learned my addr 172.21.15.3:0/4212670775 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:17.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.751+0000 7f17ab7fe700 1 --2- 172.21.15.3:0/4212670775 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17ac09e410 0x7f17ac1a6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:17.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.751+0000 7f17aa7fc700 1 -- 172.21.15.3:0/4212670775 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f17ac09d6f0 msgr2=0x7f17ac1a5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:17.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.751+0000 7f17aa7fc700 1 --2- 172.21.15.3:0/4212670775 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f17ac09d6f0 0x7f17ac1a5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.751+0000 7f17aa7fc700 1 -- 172.21.15.3:0/4212670775 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17ac09e410 msgr2=0x7f17ac1a6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:17.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.751+0000 7f17aa7fc700 1 --2- 172.21.15.3:0/4212670775 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17ac09e410 0x7f17ac1a6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.751+0000 7f17aa7fc700 1 -- 172.21.15.3:0/4212670775 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f179c009550 con 0x7f17ac09e000 2023-10-03T10:14:17.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.751+0000 7f17aa7fc700 1 --2- 172.21.15.3:0/4212670775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17ac09e000 0x7f17ac1a62d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f17a00090b0 tx=0x7f17a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:17.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.752+0000 7f17937fe700 1 -- 172.21.15.3:0/4212670775 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a0004450 con 0x7f17ac09e000 2023-10-03T10:14:17.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.752+0000 7f17937fe700 1 -- 172.21.15.3:0/4212670775 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a001f070 con 0x7f17ac09e000 2023-10-03T10:14:17.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.752+0000 7f17b179e700 1 -- 172.21.15.3:0/4212670775 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17a00094a0 con 0x7f17ac09e000 2023-10-03T10:14:17.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.752+0000 7f17b179e700 1 -- 172.21.15.3:0/4212670775 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17ac0a2f60 con 0x7f17ac09e000 2023-10-03T10:14:17.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.752+0000 7f17937fe700 1 -- 172.21.15.3:0/4212670775 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a00038a0 con 0x7f17ac09e000 2023-10-03T10:14:17.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.753+0000 7f17937fe700 1 -- 172.21.15.3:0/4212670775 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f17a0003a00 con 0x7f17ac09e000 2023-10-03T10:14:17.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.753+0000 7f17937fe700 1 -- 172.21.15.3:0/4212670775 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f17a0021070 con 0x7f17ac09e000 2023-10-03T10:14:17.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.755+0000 7f17b179e700 1 -- 172.21.15.3:0/4212670775 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f178c0052e0 con 0x7f17ac09e000 2023-10-03T10:14:17.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.761+0000 7f17937fe700 1 -- 172.21.15.3:0/4212670775 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f17a0015080 con 0x7f17ac09e000 2023-10-03T10:14:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.849+0000 7f210c3f8700 1 Processor -- start 2023-10-03T10:14:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.849+0000 7f210c3f8700 1 -- start start 2023-10-03T10:14:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.850+0000 7f210c3f8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210409c650 0x7f210409ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.850+0000 7f210c3f8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210409cf60 0x7f2104132080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:17.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.850+0000 7f210c3f8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f210409ba70 con 0x7f21040a0090 2023-10-03T10:14:17.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.850+0000 7f210c3f8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210409e8f0 con 0x7f210409cf60 2023-10-03T10:14:17.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.850+0000 7f210c3f8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f210409d400 con 0x7f210409c650 2023-10-03T10:14:17.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.850+0000 7f210a995700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210409cf60 0x7f2104132080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:17.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.850+0000 7f210a194700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210409c650 0x7f210409ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:17.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.850+0000 7f210a995700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210409cf60 0x7f2104132080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56094/0 (socket says 172.21.15.3:56094) 2023-10-03T10:14:17.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.850+0000 7f210a995700 1 -- 172.21.15.3:0/3374981435 learned_addr learned my addr 172.21.15.3:0/3374981435 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:17.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.850+0000 7f210a194700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210409c650 0x7f210409ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52732/0 (socket says 172.21.15.3:52732) 2023-10-03T10:14:17.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.851+0000 7f210a995700 1 -- 172.21.15.3:0/3374981435 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210409c650 msgr2=0x7f210409ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:17.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.851+0000 7f210a995700 1 --2- 172.21.15.3:0/3374981435 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210409c650 0x7f210409ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.851+0000 7f210a995700 1 -- 172.21.15.3:0/3374981435 >> v1:172.21.15.3:6789/0 conn(0x7f21040a0090 legacy=0x7f2104131970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.851+0000 7f2109192700 1 -- 172.21.15.3:0/3374981435 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2897393774 0 0) 0x7f210409ba70 con 0x7f21040a0090 2023-10-03T10:14:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.851+0000 7f210a995700 1 -- 172.21.15.3:0/3374981435 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21040a0760 con 0x7f210409cf60 2023-10-03T10:14:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.851+0000 7f210a995700 1 --2- 172.21.15.3:0/3374981435 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210409cf60 0x7f2104132080 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00f3a0 tx=0x7f20fc00f6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40e4a113c084d34c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.851+0000 7f2109192700 1 -- 172.21.15.3:0/3374981435 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc012070 con 0x7f210409cf60 2023-10-03T10:14:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.851+0000 7f2109192700 1 -- 172.21.15.3:0/3374981435 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20fc00d0d0 con 0x7f210409cf60 2023-10-03T10:14:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.851+0000 7f2109192700 1 -- 172.21.15.3:0/3374981435 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc00d480 con 0x7f210409cf60 2023-10-03T10:14:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.852+0000 7f210c3f8700 1 -- 172.21.15.3:0/3374981435 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210409cf60 msgr2=0x7f2104132080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.852+0000 7f210c3f8700 1 --2- 172.21.15.3:0/3374981435 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210409cf60 0x7f2104132080 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00f3a0 tx=0x7f20fc00f6b0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.852+0000 7f210c3f8700 1 -- 172.21.15.3:0/3374981435 shutdown_connections 2023-10-03T10:14:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.852+0000 7f210c3f8700 1 --2- 172.21.15.3:0/3374981435 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210409cf60 0x7f2104132080 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.852+0000 7f210c3f8700 1 --2- 172.21.15.3:0/3374981435 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210409c650 0x7f210409ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.852+0000 7f210c3f8700 1 -- 172.21.15.3:0/3374981435 >> 172.21.15.3:0/3374981435 conn(0x7f21040a3f70 msgr2=0x7f21040a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:17.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.852+0000 7f210c3f8700 1 -- 172.21.15.3:0/3374981435 shutdown_connections 2023-10-03T10:14:17.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.852+0000 7f210c3f8700 1 -- 172.21.15.3:0/3374981435 wait complete. 2023-10-03T10:14:17.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.853+0000 7f210c3f8700 1 Processor -- start 2023-10-03T10:14:17.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.853+0000 7f210c3f8700 1 -- start start 2023-10-03T10:14:17.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.854+0000 7f210c3f8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210409c650 0x7f210411a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:17.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.854+0000 7f210c3f8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210409cf60 0x7f210411ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:17.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.854+0000 7f210c3f8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21040a0090 0x7f210411b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:17.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.854+0000 7f210c3f8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210411d390 con 0x7f21040a0090 2023-10-03T10:14:17.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.854+0000 7f210c3f8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210412fb00 con 0x7f210409cf60 2023-10-03T10:14:17.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.854+0000 7f210c3f8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f210409d980 con 0x7f210409c650 2023-10-03T10:14:17.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.854+0000 7f210a194700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210409c650 0x7f210411a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:17.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.854+0000 7f210a194700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210409c650 0x7f210411a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52738/0 (socket says 172.21.15.3:52738) 2023-10-03T10:14:17.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.854+0000 7f210a194700 1 -- 172.21.15.3:0/2252928784 learned_addr learned my addr 172.21.15.3:0/2252928784 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:17.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.854+0000 7f210a995700 1 --2- 172.21.15.3:0/2252928784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21040a0090 0x7f210411b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:17.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.854+0000 7f2109993700 1 --2- 172.21.15.3:0/2252928784 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210409cf60 0x7f210411ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:17.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.854+0000 7f210a995700 1 -- 172.21.15.3:0/2252928784 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210409c650 msgr2=0x7f210411a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:17.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.854+0000 7f210a995700 1 --2- 172.21.15.3:0/2252928784 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210409c650 0x7f210411a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.854+0000 7f210a995700 1 -- 172.21.15.3:0/2252928784 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210409cf60 msgr2=0x7f210411ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:17.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.854+0000 7f210a995700 1 --2- 172.21.15.3:0/2252928784 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210409cf60 0x7f210411ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.854+0000 7f210a995700 1 -- 172.21.15.3:0/2252928784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20fc00f050 con 0x7f21040a0090 2023-10-03T10:14:17.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.854+0000 7f210a995700 1 --2- 172.21.15.3:0/2252928784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21040a0090 0x7f210411b0a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f20fc019000 tx=0x7f20fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:17.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.855+0000 7f20faffd700 1 -- 172.21.15.3:0/2252928784 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc00bc10 con 0x7f21040a0090 2023-10-03T10:14:17.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.855+0000 7f20faffd700 1 -- 172.21.15.3:0/2252928784 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20fc00bd70 con 0x7f21040a0090 2023-10-03T10:14:17.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.855+0000 7f20faffd700 1 -- 172.21.15.3:0/2252928784 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc004240 con 0x7f21040a0090 2023-10-03T10:14:17.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.855+0000 7f210c3f8700 1 -- 172.21.15.3:0/2252928784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f210411cf40 con 0x7f21040a0090 2023-10-03T10:14:17.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.855+0000 7f210c3f8700 1 -- 172.21.15.3:0/2252928784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f210411e390 con 0x7f21040a0090 2023-10-03T10:14:17.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.856+0000 7f20faffd700 1 -- 172.21.15.3:0/2252928784 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f20fc0043a0 con 0x7f21040a0090 2023-10-03T10:14:17.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.856+0000 7f210c3f8700 1 -- 172.21.15.3:0/2252928784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f210411d7a0 con 0x7f21040a0090 2023-10-03T10:14:17.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.856+0000 7f20faffd700 1 -- 172.21.15.3:0/2252928784 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f20fc045560 con 0x7f21040a0090 2023-10-03T10:14:17.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.861+0000 7f20faffd700 1 -- 172.21.15.3:0/2252928784 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f20fc020440 con 0x7f21040a0090 2023-10-03T10:14:17.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.880+0000 7ff2a99cc700 1 Processor -- start 2023-10-03T10:14:17.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.880+0000 7ff2a99cc700 1 -- start start 2023-10-03T10:14:17.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.881+0000 7ff2a99cc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2a409e1d0 0x7ff2a4138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:17.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.881+0000 7ff2a99cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2a4139170 0x7ff2a413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:17.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.881+0000 7ff2a99cc700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a409b120 con 0x7ff2a409d6f0 2023-10-03T10:14:17.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.881+0000 7ff2a99cc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a406b460 con 0x7ff2a4139170 2023-10-03T10:14:17.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.881+0000 7ff2a99cc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a409a7e0 con 0x7ff2a409e1d0 2023-10-03T10:14:17.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.881+0000 7ff2a37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2a4139170 0x7ff2a413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:17.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.881+0000 7ff2a37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2a4139170 0x7ff2a413b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56126/0 (socket says 172.21.15.3:56126) 2023-10-03T10:14:17.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.881+0000 7ff2a37fe700 1 -- 172.21.15.3:0/3231042305 learned_addr learned my addr 172.21.15.3:0/3231042305 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:17.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.881+0000 7ff2a27fc700 1 --2- 172.21.15.3:0/3231042305 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2a409e1d0 0x7ff2a4138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:17.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.881+0000 7ff2a37fe700 1 -- 172.21.15.3:0/3231042305 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2a409e1d0 msgr2=0x7ff2a4138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:17.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.881+0000 7ff2a37fe700 1 --2- 172.21.15.3:0/3231042305 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2a409e1d0 0x7ff2a4138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.881+0000 7ff2a37fe700 1 -- 172.21.15.3:0/3231042305 >> v1:172.21.15.3:6789/0 conn(0x7ff2a409d6f0 legacy=0x7ff2a409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:17.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.881+0000 7ff2a37fe700 1 -- 172.21.15.3:0/3231042305 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a409a500 con 0x7ff2a4139170 2023-10-03T10:14:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.881+0000 7ff2a27fc700 1 --2- 172.21.15.3:0/3231042305 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2a409e1d0 0x7ff2a4138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.882+0000 7ff2a37fe700 1 --2- 172.21.15.3:0/3231042305 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2a4139170 0x7ff2a413b550 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff29c009e80 tx=0x7ff29c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1810666fd92946b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.882+0000 7ff2a1ffb700 1 -- 172.21.15.3:0/3231042305 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c00e070 con 0x7ff2a4139170 2023-10-03T10:14:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.882+0000 7ff2a1ffb700 1 -- 172.21.15.3:0/3231042305 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c00b910 con 0x7ff2a4139170 2023-10-03T10:14:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.882+0000 7ff2a1ffb700 1 -- 172.21.15.3:0/3231042305 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c00bcc0 con 0x7ff2a4139170 2023-10-03T10:14:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.883+0000 7ff2a99cc700 1 -- 172.21.15.3:0/3231042305 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2a4139170 msgr2=0x7ff2a413b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.883+0000 7ff2a99cc700 1 --2- 172.21.15.3:0/3231042305 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2a4139170 0x7ff2a413b550 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff29c009e80 tx=0x7ff29c00b330 comp rx=0 tx=0).stop 2023-10-03T10:14:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.883+0000 7ff2a99cc700 1 -- 172.21.15.3:0/3231042305 shutdown_connections 2023-10-03T10:14:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.883+0000 7ff2a99cc700 1 --2- 172.21.15.3:0/3231042305 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2a4139170 0x7ff2a413b550 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.883+0000 7ff2a99cc700 1 --2- 172.21.15.3:0/3231042305 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2a409e1d0 0x7ff2a4138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.883+0000 7ff2a99cc700 1 -- 172.21.15.3:0/3231042305 >> 172.21.15.3:0/3231042305 conn(0x7ff2a4066f50 msgr2=0x7ff2a409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.883+0000 7ff2a99cc700 1 -- 172.21.15.3:0/3231042305 shutdown_connections 2023-10-03T10:14:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.883+0000 7ff2a99cc700 1 -- 172.21.15.3:0/3231042305 wait complete. 2023-10-03T10:14:17.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.884+0000 7ff2a99cc700 1 Processor -- start 2023-10-03T10:14:17.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.884+0000 7ff2a99cc700 1 -- start start 2023-10-03T10:14:17.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.884+0000 7ff2a99cc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2a409d6f0 0x7ff2a411e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:17.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.884+0000 7ff2a99cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2a409e1d0 0x7ff2a411ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.884+0000 7ff2a99cc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2a4139170 0x7ff2a411f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.884+0000 7ff2a99cc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a40a1990 con 0x7ff2a4139170 2023-10-03T10:14:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.884+0000 7ff2a99cc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a4120490 con 0x7ff2a409e1d0 2023-10-03T10:14:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.884+0000 7ff2a99cc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a4121fb0 con 0x7ff2a409d6f0 2023-10-03T10:14:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.885+0000 7ff2a27fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2a409e1d0 0x7ff2a411ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.885+0000 7ff2a27fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2a409e1d0 0x7ff2a411ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56134/0 (socket says 172.21.15.3:56134) 2023-10-03T10:14:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.885+0000 7ff2a27fc700 1 -- 172.21.15.3:0/1730061501 learned_addr learned my addr 172.21.15.3:0/1730061501 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.885+0000 7ff2a2ffd700 1 --2- 172.21.15.3:0/1730061501 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2a409d6f0 0x7ff2a411e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.885+0000 7ff2a37fe700 1 --2- 172.21.15.3:0/1730061501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2a4139170 0x7ff2a411f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.885+0000 7ff2a27fc700 1 -- 172.21.15.3:0/1730061501 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2a409d6f0 msgr2=0x7ff2a411e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.885+0000 7ff2a27fc700 1 --2- 172.21.15.3:0/1730061501 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2a409d6f0 0x7ff2a411e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.885+0000 7ff2a27fc700 1 -- 172.21.15.3:0/1730061501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2a4139170 msgr2=0x7ff2a411f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:17.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.885+0000 7ff2a27fc700 1 --2- 172.21.15.3:0/1730061501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2a4139170 0x7ff2a411f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.885+0000 7ff2a27fc700 1 -- 172.21.15.3:0/1730061501 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2980094a0 con 0x7ff2a409e1d0 2023-10-03T10:14:17.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.885+0000 7ff2a27fc700 1 --2- 172.21.15.3:0/1730061501 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2a409e1d0 0x7ff2a411ee20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff29800ebe0 tx=0x7ff29800eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:17.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.885+0000 7ff28bfff700 1 -- 172.21.15.3:0/1730061501 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29800c850 con 0x7ff2a409e1d0 2023-10-03T10:14:17.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.885+0000 7ff28bfff700 1 -- 172.21.15.3:0/1730061501 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29800c9b0 con 0x7ff2a409e1d0 2023-10-03T10:14:17.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.885+0000 7ff2a99cc700 1 -- 172.21.15.3:0/1730061501 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff29c009b30 con 0x7ff2a409e1d0 2023-10-03T10:14:17.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.885+0000 7ff28bfff700 1 -- 172.21.15.3:0/1730061501 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29800cd40 con 0x7ff2a409e1d0 2023-10-03T10:14:17.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.885+0000 7ff2a99cc700 1 -- 172.21.15.3:0/1730061501 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a4120b70 con 0x7ff2a409e1d0 2023-10-03T10:14:17.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.887+0000 7ff28bfff700 1 -- 172.21.15.3:0/1730061501 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff298013470 con 0x7ff2a409e1d0 2023-10-03T10:14:17.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.887+0000 7ff2a99cc700 1 -- 172.21.15.3:0/1730061501 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a4074ea0 con 0x7ff2a409e1d0 2023-10-03T10:14:17.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.887+0000 7ff28bfff700 1 -- 172.21.15.3:0/1730061501 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff29803f870 con 0x7ff2a409e1d0 2023-10-03T10:14:17.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.896+0000 7ff28bfff700 1 -- 172.21.15.3:0/1730061501 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff298013720 con 0x7ff2a409e1d0 2023-10-03T10:14:17.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.983+0000 7f17b179e700 1 -- 172.21.15.3:0/4212670775 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f178c005590 con 0x7f17ac09e000 2023-10-03T10:14:17.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.984+0000 7f17937fe700 1 -- 172.21.15.3:0/4212670775 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f17a002e990 con 0x7f17ac09e000 2023-10-03T10:14:17.984 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:17.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.986+0000 7f17b179e700 1 -- 172.21.15.3:0/4212670775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17ac09e000 msgr2=0x7f17ac1a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:17.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.986+0000 7f17b179e700 1 --2- 172.21.15.3:0/4212670775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17ac09e000 0x7f17ac1a62d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f17a00090b0 tx=0x7f17a0004750 comp rx=0 tx=0).stop 2023-10-03T10:14:17.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.986+0000 7f17b179e700 1 -- 172.21.15.3:0/4212670775 shutdown_connections 2023-10-03T10:14:17.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.986+0000 7f17b179e700 1 --2- 172.21.15.3:0/4212670775 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17ac09e410 0x7f17ac1a6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.986+0000 7f17b179e700 1 --2- 172.21.15.3:0/4212670775 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f17ac09d6f0 0x7f17ac1a5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.986+0000 7f17b179e700 1 --2- 172.21.15.3:0/4212670775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17ac09e000 0x7f17ac1a62d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:17.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.986+0000 7f17b179e700 1 -- 172.21.15.3:0/4212670775 >> 172.21.15.3:0/4212670775 conn(0x7f17ac066f50 msgr2=0x7f17ac1bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:17.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.986+0000 7f17b179e700 1 -- 172.21.15.3:0/4212670775 shutdown_connections 2023-10-03T10:14:17.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:17.986+0000 7f17b179e700 1 -- 172.21.15.3:0/4212670775 wait complete. 2023-10-03T10:14:18.001 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:18.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.071+0000 7f210c3f8700 1 -- 172.21.15.3:0/2252928784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f21040747f0 con 0x7f21040a0090 2023-10-03T10:14:18.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.072+0000 7f20faffd700 1 -- 172.21.15.3:0/2252928784 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f20fc021e60 con 0x7f21040a0090 2023-10-03T10:14:18.072 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:18.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.074+0000 7f210c3f8700 1 -- 172.21.15.3:0/2252928784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21040a0090 msgr2=0x7f210411b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:18.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.074+0000 7f210c3f8700 1 --2- 172.21.15.3:0/2252928784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21040a0090 0x7f210411b0a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f20fc019000 tx=0x7f20fc004ad0 comp rx=0 tx=0).stop 2023-10-03T10:14:18.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.074+0000 7f210c3f8700 1 -- 172.21.15.3:0/2252928784 shutdown_connections 2023-10-03T10:14:18.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.074+0000 7f210c3f8700 1 --2- 172.21.15.3:0/2252928784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21040a0090 0x7f210411b0a0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:18.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.074+0000 7f210c3f8700 1 --2- 172.21.15.3:0/2252928784 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210409c650 0x7f210411a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:18.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.074+0000 7f210c3f8700 1 --2- 172.21.15.3:0/2252928784 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210409cf60 0x7f210411ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:18.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.074+0000 7f210c3f8700 1 -- 172.21.15.3:0/2252928784 >> 172.21.15.3:0/2252928784 conn(0x7f21040a3f70 msgr2=0x7f210412ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:18.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.074+0000 7f210c3f8700 1 -- 172.21.15.3:0/2252928784 shutdown_connections 2023-10-03T10:14:18.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.074+0000 7f210c3f8700 1 -- 172.21.15.3:0/2252928784 wait complete. 2023-10-03T10:14:18.088 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:18.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.103+0000 7ff2a99cc700 1 -- 172.21.15.3:0/1730061501 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff2a406c360 con 0x7ff2a409e1d0 2023-10-03T10:14:18.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.103+0000 7ff28bfff700 1 -- 172.21.15.3:0/1730061501 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff298017070 con 0x7ff2a409e1d0 2023-10-03T10:14:18.103 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:18.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.105+0000 7ff2a99cc700 1 -- 172.21.15.3:0/1730061501 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2a409e1d0 msgr2=0x7ff2a411ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:18.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.105+0000 7ff2a99cc700 1 --2- 172.21.15.3:0/1730061501 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2a409e1d0 0x7ff2a411ee20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff29800ebe0 tx=0x7ff29800eef0 comp rx=0 tx=0).stop 2023-10-03T10:14:18.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.105+0000 7ff2a99cc700 1 -- 172.21.15.3:0/1730061501 shutdown_connections 2023-10-03T10:14:18.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.105+0000 7ff2a99cc700 1 --2- 172.21.15.3:0/1730061501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2a4139170 0x7ff2a411f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:18.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.105+0000 7ff2a99cc700 1 --2- 172.21.15.3:0/1730061501 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2a409d6f0 0x7ff2a411e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:18.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.105+0000 7ff2a99cc700 1 --2- 172.21.15.3:0/1730061501 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2a409e1d0 0x7ff2a411ee20 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:18.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.105+0000 7ff2a99cc700 1 -- 172.21.15.3:0/1730061501 >> 172.21.15.3:0/1730061501 conn(0x7ff2a4066f50 msgr2=0x7ff2a409b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:18.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.105+0000 7ff2a99cc700 1 -- 172.21.15.3:0/1730061501 shutdown_connections 2023-10-03T10:14:18.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.105+0000 7ff2a99cc700 1 -- 172.21.15.3:0/1730061501 wait complete. 2023-10-03T10:14:18.119 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:18.389 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:18.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.536+0000 7f8ebdb0c700 1 Processor -- start 2023-10-03T10:14:18.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.536+0000 7f8ebdb0c700 1 -- start start 2023-10-03T10:14:18.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.537+0000 7f8ebdb0c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8eb806a770 0x7f8eb806ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:18.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.537+0000 7f8ebdb0c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8eb806b080 0x7f8eb8131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:18.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.537+0000 7f8ebdb0c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb809ece0 con 0x7f8eb8131b40 2023-10-03T10:14:18.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.537+0000 7f8ebdb0c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb809e760 con 0x7f8eb806b080 2023-10-03T10:14:18.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.537+0000 7f8ebdb0c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb809e040 con 0x7f8eb806a770 2023-10-03T10:14:18.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.537+0000 7f8eb6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8eb806b080 0x7f8eb8131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:18.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.537+0000 7f8eb77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8eb806a770 0x7f8eb806ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:18.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.537+0000 7f8eb6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8eb806b080 0x7f8eb8131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40972/0 (socket says 172.21.15.3:40972) 2023-10-03T10:14:18.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.537+0000 7f8eb77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8eb806a770 0x7f8eb806ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56136/0 (socket says 172.21.15.3:56136) 2023-10-03T10:14:18.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.537+0000 7f8eb6ffd700 1 -- 172.21.15.3:0/1004487916 learned_addr learned my addr 172.21.15.3:0/1004487916 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:18.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.537+0000 7f8eb77fe700 1 -- 172.21.15.3:0/1004487916 >> v2:172.21.15.3:3300/0 conn(0x7f8eb806b080 msgr2=0x7f8eb8131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:18.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.537+0000 7f8eb77fe700 1 --2- 172.21.15.3:0/1004487916 >> v2:172.21.15.3:3300/0 conn(0x7f8eb806b080 0x7f8eb8131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:18.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.537+0000 7f8eb77fe700 1 -- 172.21.15.3:0/1004487916 >> v1:172.21.15.3:6789/0 conn(0x7f8eb8131b40 legacy=0x7f8eb8138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:18.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.537+0000 7f8eb77fe700 1 -- 172.21.15.3:0/1004487916 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb809f630 con 0x7f8eb806a770 2023-10-03T10:14:18.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.537+0000 7f8eb6ffd700 1 --2- 172.21.15.3:0/1004487916 >> v2:172.21.15.3:3300/0 conn(0x7f8eb806b080 0x7f8eb8131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:18.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.537+0000 7f8eb77fe700 1 --2- 172.21.15.3:0/1004487916 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8eb806a770 0x7f8eb806ab40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009840 tx=0x7f8ea8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2639e1ae2b37f0ce server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:18.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.537+0000 7f8eb67fc700 1 -- 172.21.15.3:0/1004487916 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8011830 con 0x7f8eb806a770 2023-10-03T10:14:18.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.538+0000 7f8eb67fc700 1 -- 172.21.15.3:0/1004487916 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea8011990 con 0x7f8eb806a770 2023-10-03T10:14:18.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.538+0000 7f8eb67fc700 1 -- 172.21.15.3:0/1004487916 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8011d20 con 0x7f8eb806a770 2023-10-03T10:14:18.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.538+0000 7f8ebdb0c700 1 -- 172.21.15.3:0/1004487916 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8eb806a770 msgr2=0x7f8eb806ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:18.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.538+0000 7f8ebdb0c700 1 --2- 172.21.15.3:0/1004487916 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8eb806a770 0x7f8eb806ab40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009840 tx=0x7f8ea8009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:18.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.538+0000 7f8ebdb0c700 1 -- 172.21.15.3:0/1004487916 shutdown_connections 2023-10-03T10:14:18.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.538+0000 7f8ebdb0c700 1 --2- 172.21.15.3:0/1004487916 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8eb806a770 0x7f8eb806ab40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:18.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.538+0000 7f8ebdb0c700 1 --2- 172.21.15.3:0/1004487916 >> v2:172.21.15.3:3300/0 conn(0x7f8eb806b080 0x7f8eb8131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:18.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.538+0000 7f8ebdb0c700 1 -- 172.21.15.3:0/1004487916 >> 172.21.15.3:0/1004487916 conn(0x7f8eb8066f50 msgr2=0x7f8eb809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:18.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.538+0000 7f8ebdb0c700 1 -- 172.21.15.3:0/1004487916 shutdown_connections 2023-10-03T10:14:18.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.538+0000 7f8ebdb0c700 1 -- 172.21.15.3:0/1004487916 wait complete. 2023-10-03T10:14:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.539+0000 7f8ebdb0c700 1 Processor -- start 2023-10-03T10:14:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.539+0000 7f8ebdb0c700 1 -- start start 2023-10-03T10:14:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.540+0000 7f8ebdb0c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8eb806a770 0x7f8eb809acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.540+0000 7f8ebdb0c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8eb806b080 0x7f8eb809b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.540+0000 7f8ebdb0c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8eb8131b40 0x7f8eb81c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.540+0000 7f8ebdb0c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb809bf10 con 0x7f8eb8131b40 2023-10-03T10:14:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.540+0000 7f8ebdb0c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb8121e40 con 0x7f8eb806a770 2023-10-03T10:14:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.540+0000 7f8ebdb0c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb8121450 con 0x7f8eb806b080 2023-10-03T10:14:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.540+0000 7f8eb77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8eb806a770 0x7f8eb809acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:18.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.540+0000 7f8eb77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8eb806a770 0x7f8eb809acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56148/0 (socket says 172.21.15.3:56148) 2023-10-03T10:14:18.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.540+0000 7f8eb77fe700 1 -- 172.21.15.3:0/1834260819 learned_addr learned my addr 172.21.15.3:0/1834260819 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:18.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.540+0000 7f8eb6ffd700 1 --2- 172.21.15.3:0/1834260819 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8eb806b080 0x7f8eb809b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:18.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.540+0000 7f8eb7fff700 1 --2- 172.21.15.3:0/1834260819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8eb8131b40 0x7f8eb81c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:18.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.540+0000 7f8eb77fe700 1 -- 172.21.15.3:0/1834260819 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8eb806b080 msgr2=0x7f8eb809b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:18.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.540+0000 7f8eb77fe700 1 --2- 172.21.15.3:0/1834260819 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8eb806b080 0x7f8eb809b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:18.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.540+0000 7f8eb77fe700 1 -- 172.21.15.3:0/1834260819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8eb8131b40 msgr2=0x7f8eb81c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:18.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.540+0000 7f8eb77fe700 1 --2- 172.21.15.3:0/1834260819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8eb8131b40 0x7f8eb81c8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:18.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.540+0000 7f8eb77fe700 1 -- 172.21.15.3:0/1834260819 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eac009550 con 0x7f8eb806a770 2023-10-03T10:14:18.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.541+0000 7f8eb77fe700 1 --2- 172.21.15.3:0/1834260819 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8eb806a770 0x7f8eb809acb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8ea80090b0 tx=0x7f8ea8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:18.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.541+0000 7f8e9ffff700 1 -- 172.21.15.3:0/1834260819 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea80036a0 con 0x7f8eb806a770 2023-10-03T10:14:18.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.541+0000 7f8e9ffff700 1 -- 172.21.15.3:0/1834260819 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea801f070 con 0x7f8eb806a770 2023-10-03T10:14:18.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.541+0000 7f8ebdb0c700 1 -- 172.21.15.3:0/1834260819 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea80094a0 con 0x7f8eb806a770 2023-10-03T10:14:18.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.541+0000 7f8e9ffff700 1 -- 172.21.15.3:0/1834260819 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8003a60 con 0x7f8eb806a770 2023-10-03T10:14:18.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.541+0000 7f8ebdb0c700 1 -- 172.21.15.3:0/1834260819 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb809b730 con 0x7f8eb806a770 2023-10-03T10:14:18.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.542+0000 7f8e9ffff700 1 -- 172.21.15.3:0/1834260819 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8ea8003bc0 con 0x7f8eb806a770 2023-10-03T10:14:18.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.542+0000 7f8e9ffff700 1 -- 172.21.15.3:0/1834260819 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8ea8021070 con 0x7f8eb806a770 2023-10-03T10:14:18.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.542+0000 7f8ebdb0c700 1 -- 172.21.15.3:0/1834260819 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb8074ea0 con 0x7f8eb806a770 2023-10-03T10:14:18.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.547+0000 7f8e9ffff700 1 -- 172.21.15.3:0/1834260819 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8ea801a950 con 0x7f8eb806a770 2023-10-03T10:14:18.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.751+0000 7f8ebdb0c700 1 -- 172.21.15.3:0/1834260819 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8eb809a0c0 con 0x7f8eb806a770 2023-10-03T10:14:18.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.751+0000 7f8e9ffff700 1 -- 172.21.15.3:0/1834260819 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8ea8015020 con 0x7f8eb806a770 2023-10-03T10:14:18.752 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:18.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.754+0000 7f8ebdb0c700 1 -- 172.21.15.3:0/1834260819 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8eb806a770 msgr2=0x7f8eb809acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:18.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.754+0000 7f8ebdb0c700 1 --2- 172.21.15.3:0/1834260819 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8eb806a770 0x7f8eb809acb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8ea80090b0 tx=0x7f8ea8004750 comp rx=0 tx=0).stop 2023-10-03T10:14:18.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.754+0000 7f8ebdb0c700 1 -- 172.21.15.3:0/1834260819 shutdown_connections 2023-10-03T10:14:18.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.754+0000 7f8ebdb0c700 1 --2- 172.21.15.3:0/1834260819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8eb8131b40 0x7f8eb81c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:18.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.754+0000 7f8ebdb0c700 1 --2- 172.21.15.3:0/1834260819 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8eb806a770 0x7f8eb809acb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:18.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.754+0000 7f8ebdb0c700 1 --2- 172.21.15.3:0/1834260819 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8eb806b080 0x7f8eb809b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:18.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.754+0000 7f8ebdb0c700 1 -- 172.21.15.3:0/1834260819 >> 172.21.15.3:0/1834260819 conn(0x7f8eb8066f50 msgr2=0x7f8eb8070160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:18.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.754+0000 7f8ebdb0c700 1 -- 172.21.15.3:0/1834260819 shutdown_connections 2023-10-03T10:14:18.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:18.754+0000 7f8ebdb0c700 1 -- 172.21.15.3:0/1834260819 wait complete. 2023-10-03T10:14:18.768 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:19.002 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:19.089 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:19.120 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:19.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.149+0000 7f60bfd63700 1 Processor -- start 2023-10-03T10:14:19.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.150+0000 7f60bfd63700 1 -- start start 2023-10-03T10:14:19.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.150+0000 7f60bfd63700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f60b809e1d0 0x7f60b8138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:19.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.150+0000 7f60bfd63700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f60b8139170 0x7f60b813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.150+0000 7f60bfd63700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60b809b120 con 0x7f60b809d6f0 2023-10-03T10:14:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.150+0000 7f60bfd63700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60b806b460 con 0x7f60b809e1d0 2023-10-03T10:14:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.150+0000 7f60bfd63700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60b809a7e0 con 0x7f60b8139170 2023-10-03T10:14:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.150+0000 7f60be300700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f60b8139170 0x7f60b813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.150+0000 7f60bd2fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f60b809e1d0 0x7f60b8138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.150+0000 7f60bd2fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f60b809e1d0 0x7f60b8138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41000/0 (socket says 172.21.15.3:41000) 2023-10-03T10:14:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.150+0000 7f60be300700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f60b8139170 0x7f60b813b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56156/0 (socket says 172.21.15.3:56156) 2023-10-03T10:14:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.150+0000 7f60bd2fe700 1 -- 172.21.15.3:0/2345276301 learned_addr learned my addr 172.21.15.3:0/2345276301 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:19.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.151+0000 7f60bd2fe700 1 -- 172.21.15.3:0/2345276301 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f60b8139170 msgr2=0x7f60b813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:19.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.151+0000 7f60bd2fe700 1 --2- 172.21.15.3:0/2345276301 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f60b8139170 0x7f60b813b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.151+0000 7f60bd2fe700 1 -- 172.21.15.3:0/2345276301 >> v1:172.21.15.3:6789/0 conn(0x7f60b809d6f0 legacy=0x7f60b809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:19.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.151+0000 7f60bcafd700 1 -- 172.21.15.3:0/2345276301 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1481097322 0 0) 0x7f60b809b120 con 0x7f60b809d6f0 2023-10-03T10:14:19.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.151+0000 7f60bd2fe700 1 -- 172.21.15.3:0/2345276301 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b809a500 con 0x7f60b809e1d0 2023-10-03T10:14:19.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.151+0000 7f60bd2fe700 1 --2- 172.21.15.3:0/2345276301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60b809e1d0 0x7f60b8138c30 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009920 tx=0x7f60a8009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700855f4a66734b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:19.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.151+0000 7f60bcafd700 1 -- 172.21.15.3:0/2345276301 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a8011830 con 0x7f60b809e1d0 2023-10-03T10:14:19.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.151+0000 7f60bcafd700 1 -- 172.21.15.3:0/2345276301 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60a8011990 con 0x7f60b809e1d0 2023-10-03T10:14:19.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.151+0000 7f60bcafd700 1 -- 172.21.15.3:0/2345276301 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a8011d20 con 0x7f60b809e1d0 2023-10-03T10:14:19.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.152+0000 7f60bfd63700 1 -- 172.21.15.3:0/2345276301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60b809e1d0 msgr2=0x7f60b8138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:19.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.152+0000 7f60bfd63700 1 --2- 172.21.15.3:0/2345276301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60b809e1d0 0x7f60b8138c30 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009920 tx=0x7f60a8009c30 comp rx=0 tx=0).stop 2023-10-03T10:14:19.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.152+0000 7f60bfd63700 1 -- 172.21.15.3:0/2345276301 shutdown_connections 2023-10-03T10:14:19.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.152+0000 7f60bfd63700 1 --2- 172.21.15.3:0/2345276301 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f60b8139170 0x7f60b813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.152+0000 7f60bfd63700 1 --2- 172.21.15.3:0/2345276301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60b809e1d0 0x7f60b8138c30 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.152+0000 7f60bfd63700 1 -- 172.21.15.3:0/2345276301 >> 172.21.15.3:0/2345276301 conn(0x7f60b8066f50 msgr2=0x7f60b809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:19.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.152+0000 7f60bfd63700 1 -- 172.21.15.3:0/2345276301 shutdown_connections 2023-10-03T10:14:19.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.152+0000 7f60bfd63700 1 -- 172.21.15.3:0/2345276301 wait complete. 2023-10-03T10:14:19.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.154+0000 7f60bfd63700 1 Processor -- start 2023-10-03T10:14:19.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.154+0000 7f60bfd63700 1 -- start start 2023-10-03T10:14:19.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.154+0000 7f60bfd63700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f60b809d6f0 0x7f60b811e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:19.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.155+0000 7f60bfd63700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60b809e1d0 0x7f60b811ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:19.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.155+0000 7f60bfd63700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f60b8139170 0x7f60b811f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:19.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.155+0000 7f60bfd63700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b81206a0 con 0x7f60b809e1d0 2023-10-03T10:14:19.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.155+0000 7f60bfd63700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60b809c710 con 0x7f60b809d6f0 2023-10-03T10:14:19.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.155+0000 7f60bfd63700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f60b809a0c0 con 0x7f60b8139170 2023-10-03T10:14:19.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.155+0000 7f60bd2fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60b809e1d0 0x7f60b811ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:19.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.155+0000 7f60bdaff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f60b809d6f0 0x7f60b811e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:19.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.155+0000 7f60bd2fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60b809e1d0 0x7f60b811ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41012/0 (socket says 172.21.15.3:41012) 2023-10-03T10:14:19.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.155+0000 7f60bdaff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f60b809d6f0 0x7f60b811e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56170/0 (socket says 172.21.15.3:56170) 2023-10-03T10:14:19.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.155+0000 7f60bd2fe700 1 -- 172.21.15.3:0/3692385307 learned_addr learned my addr 172.21.15.3:0/3692385307 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:19.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.155+0000 7f60bdaff700 1 -- 172.21.15.3:0/3692385307 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f60b8139170 msgr2=0x7f60b811f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:19.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.155+0000 7f60bdaff700 1 --2- 172.21.15.3:0/3692385307 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f60b8139170 0x7f60b811f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.155+0000 7f60bdaff700 1 -- 172.21.15.3:0/3692385307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60b809e1d0 msgr2=0x7f60b811ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:19.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.155+0000 7f60bdaff700 1 --2- 172.21.15.3:0/3692385307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60b809e1d0 0x7f60b811ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.155+0000 7f60bdaff700 1 -- 172.21.15.3:0/3692385307 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a8009550 con 0x7f60b809d6f0 2023-10-03T10:14:19.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.155+0000 7f60bd2fe700 1 --2- 172.21.15.3:0/3692385307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60b809e1d0 0x7f60b811ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:19.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.155+0000 7f60bdaff700 1 --2- 172.21.15.3:0/3692385307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f60b809d6f0 0x7f60b811e9e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f60b400c3b0 tx=0x7f60b400c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:19.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.156+0000 7f60ae7fc700 1 -- 172.21.15.3:0/3692385307 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b40054e0 con 0x7f60b809d6f0 2023-10-03T10:14:19.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.156+0000 7f60ae7fc700 1 -- 172.21.15.3:0/3692385307 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b400ce70 con 0x7f60b809d6f0 2023-10-03T10:14:19.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.156+0000 7f60ae7fc700 1 -- 172.21.15.3:0/3692385307 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b40038a0 con 0x7f60b809d6f0 2023-10-03T10:14:19.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.156+0000 7f60bfd63700 1 -- 172.21.15.3:0/3692385307 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b8121bc0 con 0x7f60b809d6f0 2023-10-03T10:14:19.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.156+0000 7f60bfd63700 1 -- 172.21.15.3:0/3692385307 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b8120cf0 con 0x7f60b809d6f0 2023-10-03T10:14:19.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.157+0000 7f60ae7fc700 1 -- 172.21.15.3:0/3692385307 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f60b4003a00 con 0x7f60b809d6f0 2023-10-03T10:14:19.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.157+0000 7f60ae7fc700 1 -- 172.21.15.3:0/3692385307 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f60b403ea70 con 0x7f60b809d6f0 2023-10-03T10:14:19.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.157+0000 7f60bfd63700 1 -- 172.21.15.3:0/3692385307 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60b8074ea0 con 0x7f60b809d6f0 2023-10-03T10:14:19.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.162+0000 7f60ae7fc700 1 -- 172.21.15.3:0/3692385307 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f60b4017080 con 0x7f60b809d6f0 2023-10-03T10:14:19.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.234+0000 7f88039e1700 1 Processor -- start 2023-10-03T10:14:19.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.234+0000 7f88039e1700 1 -- start start 2023-10-03T10:14:19.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.234+0000 7f88039e1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f87fc09e930 0x7f87fc09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:19.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.234+0000 7f88039e1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f87fc09f240 0x7f87fc1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:19.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.234+0000 7f88039e1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87fc070940 con 0x7f87fc09f650 2023-10-03T10:14:19.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.234+0000 7f88039e1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc06fd60 con 0x7f87fc09f240 2023-10-03T10:14:19.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.234+0000 7f88039e1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc06fa80 con 0x7f87fc09e930 2023-10-03T10:14:19.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.235+0000 7f880177d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f87fc09e930 0x7f87fc09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:19.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.235+0000 7f8800f7c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f87fc09f240 0x7f87fc1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:19.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.235+0000 7f880177d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f87fc09e930 0x7f87fc09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52768/0 (socket says 172.21.15.3:52768) 2023-10-03T10:14:19.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.235+0000 7f8801f7e700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f87fc09f650 0x7f87fc131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44084/0 (socket says 172.21.15.3:44084) 2023-10-03T10:14:19.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.235+0000 7f880177d700 1 -- 172.21.15.3:0/42884367 learned_addr learned my addr 172.21.15.3:0/42884367 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:19.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.235+0000 7f8800f7c700 1 -- 172.21.15.3:0/42884367 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f87fc09e930 msgr2=0x7f87fc09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:19.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.235+0000 7f8800f7c700 1 --2- 172.21.15.3:0/42884367 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f87fc09e930 0x7f87fc09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.235+0000 7f8800f7c700 1 -- 172.21.15.3:0/42884367 >> v1:172.21.15.3:6789/0 conn(0x7f87fc09f650 legacy=0x7f87fc131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:19.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.235+0000 7f87f3fff700 1 -- 172.21.15.3:0/42884367 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3264105980 0 0) 0x7f87fc070940 con 0x7f87fc09f650 2023-10-03T10:14:19.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.235+0000 7f8800f7c700 1 -- 172.21.15.3:0/42884367 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87fc09fb60 con 0x7f87fc09f240 2023-10-03T10:14:19.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.235+0000 7f8800f7c700 1 --2- 172.21.15.3:0/42884367 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f87fc09f240 0x7f87fc1317f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f87ec009870 tx=0x7f87ec009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44634957933c468a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:19.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.235+0000 7f87f3fff700 1 -- 172.21.15.3:0/42884367 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec011830 con 0x7f87fc09f240 2023-10-03T10:14:19.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.235+0000 7f87f3fff700 1 -- 172.21.15.3:0/42884367 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ec011990 con 0x7f87fc09f240 2023-10-03T10:14:19.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.235+0000 7f87f3fff700 1 -- 172.21.15.3:0/42884367 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec011d20 con 0x7f87fc09f240 2023-10-03T10:14:19.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.236+0000 7f88039e1700 1 -- 172.21.15.3:0/42884367 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f87fc09f240 msgr2=0x7f87fc1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:19.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.236+0000 7f88039e1700 1 --2- 172.21.15.3:0/42884367 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f87fc09f240 0x7f87fc1317f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f87ec009870 tx=0x7f87ec009b80 comp rx=0 tx=0).stop 2023-10-03T10:14:19.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.236+0000 7f88039e1700 1 -- 172.21.15.3:0/42884367 shutdown_connections 2023-10-03T10:14:19.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.236+0000 7f88039e1700 1 --2- 172.21.15.3:0/42884367 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f87fc09e930 0x7f87fc09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.236+0000 7f88039e1700 1 --2- 172.21.15.3:0/42884367 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f87fc09f240 0x7f87fc1317f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.236+0000 7f88039e1700 1 -- 172.21.15.3:0/42884367 >> 172.21.15.3:0/42884367 conn(0x7f87fc0a4210 msgr2=0x7f87fc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:19.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.236+0000 7f88039e1700 1 -- 172.21.15.3:0/42884367 shutdown_connections 2023-10-03T10:14:19.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.236+0000 7f88039e1700 1 -- 172.21.15.3:0/42884367 wait complete. 2023-10-03T10:14:19.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.238+0000 7f88039e1700 1 Processor -- start 2023-10-03T10:14:19.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.238+0000 7f88039e1700 1 -- start start 2023-10-03T10:14:19.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.239+0000 7f88039e1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f87fc09e930 0x7f87fc06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:19.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.239+0000 7f88039e1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87fc09f240 0x7f87fc06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:19.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.239+0000 7f88039e1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f87fc09f650 0x7f87fc06c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:19.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.239+0000 7f88039e1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc1a98b0 con 0x7f87fc09f240 2023-10-03T10:14:19.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.239+0000 7f88039e1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc1a9040 con 0x7f87fc09e930 2023-10-03T10:14:19.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.239+0000 7f88039e1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc1a9310 con 0x7f87fc09f650 2023-10-03T10:14:19.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.239+0000 7f880177d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f87fc09e930 0x7f87fc06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:19.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.239+0000 7f8800f7c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87fc09f240 0x7f87fc06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:19.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.239+0000 7f8801f7e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f87fc09f650 0x7f87fc06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:19.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.239+0000 7f8800f7c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87fc09f240 0x7f87fc06bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41018/0 (socket says 172.21.15.3:41018) 2023-10-03T10:14:19.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.239+0000 7f880177d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f87fc09e930 0x7f87fc06b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56192/0 (socket says 172.21.15.3:56192) 2023-10-03T10:14:19.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.239+0000 7f8800f7c700 1 -- 172.21.15.3:0/2552412591 learned_addr learned my addr 172.21.15.3:0/2552412591 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:19.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.239+0000 7f8800f7c700 1 -- 172.21.15.3:0/2552412591 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f87fc09f650 msgr2=0x7f87fc06c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:19.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.239+0000 7f8800f7c700 1 --2- 172.21.15.3:0/2552412591 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f87fc09f650 0x7f87fc06c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.239+0000 7f8800f7c700 1 -- 172.21.15.3:0/2552412591 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f87fc09e930 msgr2=0x7f87fc06b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:19.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.239+0000 7f8800f7c700 1 --2- 172.21.15.3:0/2552412591 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f87fc09e930 0x7f87fc06b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.239+0000 7f8800f7c700 1 -- 172.21.15.3:0/2552412591 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ec0094a0 con 0x7f87fc09f240 2023-10-03T10:14:19.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.239+0000 7f880177d700 1 --2- 172.21.15.3:0/2552412591 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f87fc09e930 0x7f87fc06b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:19.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.240+0000 7f8800f7c700 1 --2- 172.21.15.3:0/2552412591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87fc09f240 0x7f87fc06bdf0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0090b0 tx=0x7f87ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:19.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.240+0000 7f87f1ffb700 1 -- 172.21.15.3:0/2552412591 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec004510 con 0x7f87fc09f240 2023-10-03T10:14:19.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.240+0000 7f87f1ffb700 1 -- 172.21.15.3:0/2552412591 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ec01f070 con 0x7f87fc09f240 2023-10-03T10:14:19.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.240+0000 7f88039e1700 1 -- 172.21.15.3:0/2552412591 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87fc0a0190 con 0x7f87fc09f240 2023-10-03T10:14:19.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.240+0000 7f87f1ffb700 1 -- 172.21.15.3:0/2552412591 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec0038c0 con 0x7f87fc09f240 2023-10-03T10:14:19.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.240+0000 7f88039e1700 1 -- 172.21.15.3:0/2552412591 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87fc06d420 con 0x7f87fc09f240 2023-10-03T10:14:19.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.241+0000 7f87f1ffb700 1 -- 172.21.15.3:0/2552412591 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f87ec003a20 con 0x7f87fc09f240 2023-10-03T10:14:19.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.241+0000 7f88039e1700 1 -- 172.21.15.3:0/2552412591 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87fc1a0b30 con 0x7f87fc09f240 2023-10-03T10:14:19.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.241+0000 7f87f1ffb700 1 -- 172.21.15.3:0/2552412591 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f87ec021070 con 0x7f87fc09f240 2023-10-03T10:14:19.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.246+0000 7f87f1ffb700 1 -- 172.21.15.3:0/2552412591 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f87ec010b70 con 0x7f87fc09f240 2023-10-03T10:14:19.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.276+0000 7fe3eac60700 1 Processor -- start 2023-10-03T10:14:19.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.276+0000 7fe3eac60700 1 -- start start 2023-10-03T10:14:19.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.277+0000 7fe3eac60700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3e40a0a30 0x7fe3e40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:19.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.277+0000 7fe3eac60700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3e4138d80 0x7fe3e413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:19.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.277+0000 7fe3eac60700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e40a2a70 con 0x7fe3e409d630 2023-10-03T10:14:19.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.277+0000 7fe3eac60700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e40a01d0 con 0x7fe3e4138d80 2023-10-03T10:14:19.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.277+0000 7fe3eac60700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e409e2b0 con 0x7fe3e40a0a30 2023-10-03T10:14:19.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.277+0000 7fe3e3fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe3e409d630 0x7fe3e40a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44088/0 (socket says 172.21.15.3:44088) 2023-10-03T10:14:19.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.277+0000 7fe3e3fff700 1 -- 172.21.15.3:0/1924972226 learned_addr learned my addr 172.21.15.3:0/1924972226 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:19.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.277+0000 7fe3e89fc700 1 --2- 172.21.15.3:0/1924972226 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3e40a0a30 0x7fe3e40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:19.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.277+0000 7fe3e91fd700 1 --2- 172.21.15.3:0/1924972226 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3e4138d80 0x7fe3e413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:19.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.277+0000 7fe3e37fe700 1 -- 172.21.15.3:0/1924972226 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3057008962 0 0) 0x7fe3e40a2a70 con 0x7fe3e409d630 2023-10-03T10:14:19.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.277+0000 7fe3e89fc700 1 -- 172.21.15.3:0/1924972226 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3e4138d80 msgr2=0x7fe3e413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:19.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.277+0000 7fe3e89fc700 1 --2- 172.21.15.3:0/1924972226 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3e4138d80 0x7fe3e413b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.277+0000 7fe3e89fc700 1 -- 172.21.15.3:0/1924972226 >> v1:172.21.15.3:6789/0 conn(0x7fe3e409d630 legacy=0x7fe3e40a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:19.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.277+0000 7fe3e89fc700 1 -- 172.21.15.3:0/1924972226 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e409dc50 con 0x7fe3e40a0a30 2023-10-03T10:14:19.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.278+0000 7fe3e89fc700 1 --2- 172.21.15.3:0/1924972226 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3e40a0a30 0x7fe3e40a0e00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8009870 tx=0x7fe3d8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f039f3d6188ac668 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:19.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.278+0000 7fe3e37fe700 1 -- 172.21.15.3:0/1924972226 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8011830 con 0x7fe3e40a0a30 2023-10-03T10:14:19.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.278+0000 7fe3e37fe700 1 -- 172.21.15.3:0/1924972226 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d8011990 con 0x7fe3e40a0a30 2023-10-03T10:14:19.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.278+0000 7fe3e37fe700 1 -- 172.21.15.3:0/1924972226 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8011d20 con 0x7fe3e40a0a30 2023-10-03T10:14:19.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.278+0000 7fe3eac60700 1 -- 172.21.15.3:0/1924972226 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3e40a0a30 msgr2=0x7fe3e40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:19.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.278+0000 7fe3eac60700 1 --2- 172.21.15.3:0/1924972226 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3e40a0a30 0x7fe3e40a0e00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8009870 tx=0x7fe3d8009b80 comp rx=0 tx=0).stop 2023-10-03T10:14:19.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.279+0000 7fe3eac60700 1 -- 172.21.15.3:0/1924972226 shutdown_connections 2023-10-03T10:14:19.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.279+0000 7fe3eac60700 1 --2- 172.21.15.3:0/1924972226 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3e4138d80 0x7fe3e413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.279+0000 7fe3eac60700 1 --2- 172.21.15.3:0/1924972226 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3e40a0a30 0x7fe3e40a0e00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.279+0000 7fe3eac60700 1 -- 172.21.15.3:0/1924972226 >> 172.21.15.3:0/1924972226 conn(0x7fe3e40a3050 msgr2=0x7fe3e40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:19.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.279+0000 7fe3eac60700 1 -- 172.21.15.3:0/1924972226 shutdown_connections 2023-10-03T10:14:19.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.279+0000 7fe3eac60700 1 -- 172.21.15.3:0/1924972226 wait complete. 2023-10-03T10:14:19.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.280+0000 7fe3eac60700 1 Processor -- start 2023-10-03T10:14:19.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.280+0000 7fe3eac60700 1 -- start start 2023-10-03T10:14:19.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.280+0000 7fe3eac60700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3e409d630 0x7fe3e4133790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:19.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.280+0000 7fe3eac60700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3e40a0a30 0x7fe3e4133cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:19.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.280+0000 7fe3eac60700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3e4138d80 0x7fe3e4134210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:19.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.280+0000 7fe3eac60700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e40a0410 con 0x7fe3e409d630 2023-10-03T10:14:19.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.280+0000 7fe3eac60700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e4136ec0 con 0x7fe3e4138d80 2023-10-03T10:14:19.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.280+0000 7fe3eac60700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e413ac70 con 0x7fe3e40a0a30 2023-10-03T10:14:19.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.280+0000 7fe3e89fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3e409d630 0x7fe3e4133790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:19.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.280+0000 7fe3e89fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3e409d630 0x7fe3e4133790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41032/0 (socket says 172.21.15.3:41032) 2023-10-03T10:14:19.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.280+0000 7fe3e89fc700 1 -- 172.21.15.3:0/3746436494 learned_addr learned my addr 172.21.15.3:0/3746436494 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:19.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.281+0000 7fe3e3fff700 1 --2- 172.21.15.3:0/3746436494 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3e40a0a30 0x7fe3e4133cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:19.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.281+0000 7fe3e91fd700 1 --2- 172.21.15.3:0/3746436494 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3e4138d80 0x7fe3e4134210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:19.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.281+0000 7fe3e89fc700 1 -- 172.21.15.3:0/3746436494 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3e40a0a30 msgr2=0x7fe3e4133cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:19.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.281+0000 7fe3e89fc700 1 --2- 172.21.15.3:0/3746436494 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3e40a0a30 0x7fe3e4133cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.281+0000 7fe3e89fc700 1 -- 172.21.15.3:0/3746436494 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3e4138d80 msgr2=0x7fe3e4134210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:19.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.281+0000 7fe3e89fc700 1 --2- 172.21.15.3:0/3746436494 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3e4138d80 0x7fe3e4134210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.281+0000 7fe3e89fc700 1 -- 172.21.15.3:0/3746436494 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d80094a0 con 0x7fe3e409d630 2023-10-03T10:14:19.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.281+0000 7fe3e91fd700 1 --2- 172.21.15.3:0/3746436494 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3e4138d80 0x7fe3e4134210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:19.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.281+0000 7fe3e89fc700 1 --2- 172.21.15.3:0/3746436494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3e409d630 0x7fe3e4133790 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80090b0 tx=0x7fe3d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:19.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.281+0000 7fe3e17fa700 1 -- 172.21.15.3:0/3746436494 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8004300 con 0x7fe3e409d630 2023-10-03T10:14:19.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.281+0000 7fe3e17fa700 1 -- 172.21.15.3:0/3746436494 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d801f070 con 0x7fe3e409d630 2023-10-03T10:14:19.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.281+0000 7fe3eac60700 1 -- 172.21.15.3:0/3746436494 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e4137930 con 0x7fe3e409d630 2023-10-03T10:14:19.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.281+0000 7fe3e17fa700 1 -- 172.21.15.3:0/3746436494 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d80036a0 con 0x7fe3e409d630 2023-10-03T10:14:19.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.281+0000 7fe3eac60700 1 -- 172.21.15.3:0/3746436494 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e41358a0 con 0x7fe3e409d630 2023-10-03T10:14:19.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.282+0000 7fe3e17fa700 1 -- 172.21.15.3:0/3746436494 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe3d8021030 con 0x7fe3e409d630 2023-10-03T10:14:19.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.282+0000 7fe3ceffd700 1 -- 172.21.15.3:0/3746436494 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3bc0052e0 con 0x7fe3e409d630 2023-10-03T10:14:19.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.283+0000 7fe3e17fa700 1 -- 172.21.15.3:0/3746436494 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe3d8040680 con 0x7fe3e409d630 2023-10-03T10:14:19.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.288+0000 7fe3e17fa700 1 -- 172.21.15.3:0/3746436494 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe3d80199a0 con 0x7fe3e409d630 2023-10-03T10:14:19.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.395+0000 7f60bfd63700 1 -- 172.21.15.3:0/3692385307 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f60b809f420 con 0x7f60b809d6f0 2023-10-03T10:14:19.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.395+0000 7f60ae7fc700 1 -- 172.21.15.3:0/3692385307 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f60b4019b20 con 0x7f60b809d6f0 2023-10-03T10:14:19.395 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:19.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.398+0000 7f60bfd63700 1 -- 172.21.15.3:0/3692385307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f60b809d6f0 msgr2=0x7f60b811e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:19.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.398+0000 7f60bfd63700 1 --2- 172.21.15.3:0/3692385307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f60b809d6f0 0x7f60b811e9e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f60b400c3b0 tx=0x7f60b400c6c0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.398+0000 7f60bfd63700 1 -- 172.21.15.3:0/3692385307 shutdown_connections 2023-10-03T10:14:19.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.398+0000 7f60bfd63700 1 --2- 172.21.15.3:0/3692385307 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f60b8139170 0x7f60b811f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.398+0000 7f60bfd63700 1 --2- 172.21.15.3:0/3692385307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f60b809d6f0 0x7f60b811e9e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.398+0000 7f60bfd63700 1 --2- 172.21.15.3:0/3692385307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60b809e1d0 0x7f60b811ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.398+0000 7f60bfd63700 1 -- 172.21.15.3:0/3692385307 >> 172.21.15.3:0/3692385307 conn(0x7f60b8066f50 msgr2=0x7f60b8138070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:19.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.399+0000 7f60bfd63700 1 -- 172.21.15.3:0/3692385307 shutdown_connections 2023-10-03T10:14:19.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.399+0000 7f60bfd63700 1 -- 172.21.15.3:0/3692385307 wait complete. 2023-10-03T10:14:19.417 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.468+0000 7f88039e1700 1 -- 172.21.15.3:0/2552412591 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f87fc074ea0 con 0x7f87fc09f240 2023-10-03T10:14:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.469+0000 7f87f1ffb700 1 -- 172.21.15.3:0/2552412591 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f87ec015020 con 0x7f87fc09f240 2023-10-03T10:14:19.469 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.471+0000 7f88039e1700 1 -- 172.21.15.3:0/2552412591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87fc09f240 msgr2=0x7f87fc06bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.471+0000 7f88039e1700 1 --2- 172.21.15.3:0/2552412591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87fc09f240 0x7f87fc06bdf0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0090b0 tx=0x7f87ec004750 comp rx=0 tx=0).stop 2023-10-03T10:14:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.471+0000 7f88039e1700 1 -- 172.21.15.3:0/2552412591 shutdown_connections 2023-10-03T10:14:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.471+0000 7f88039e1700 1 --2- 172.21.15.3:0/2552412591 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f87fc09f650 0x7f87fc06c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.471+0000 7f88039e1700 1 --2- 172.21.15.3:0/2552412591 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f87fc09e930 0x7f87fc06b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.471+0000 7f88039e1700 1 --2- 172.21.15.3:0/2552412591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87fc09f240 0x7f87fc06bdf0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.471+0000 7f88039e1700 1 -- 172.21.15.3:0/2552412591 >> 172.21.15.3:0/2552412591 conn(0x7f87fc0a4210 msgr2=0x7f87fc1b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.471+0000 7f88039e1700 1 -- 172.21.15.3:0/2552412591 shutdown_connections 2023-10-03T10:14:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.471+0000 7f88039e1700 1 -- 172.21.15.3:0/2552412591 wait complete. 2023-10-03T10:14:19.485 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:19.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.512+0000 7fe3ceffd700 1 -- 172.21.15.3:0/3746436494 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe3bc005590 con 0x7fe3e409d630 2023-10-03T10:14:19.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.512+0000 7fe3e17fa700 1 -- 172.21.15.3:0/3746436494 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe3d8015070 con 0x7fe3e409d630 2023-10-03T10:14:19.513 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:19.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.515+0000 7fe3ceffd700 1 -- 172.21.15.3:0/3746436494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3e409d630 msgr2=0x7fe3e4133790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:19.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.515+0000 7fe3ceffd700 1 --2- 172.21.15.3:0/3746436494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3e409d630 0x7fe3e4133790 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80090b0 tx=0x7fe3d8004750 comp rx=0 tx=0).stop 2023-10-03T10:14:19.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.515+0000 7fe3ceffd700 1 -- 172.21.15.3:0/3746436494 shutdown_connections 2023-10-03T10:14:19.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.515+0000 7fe3ceffd700 1 --2- 172.21.15.3:0/3746436494 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3e4138d80 0x7fe3e4134210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.515+0000 7fe3ceffd700 1 --2- 172.21.15.3:0/3746436494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3e409d630 0x7fe3e4133790 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.515+0000 7fe3ceffd700 1 --2- 172.21.15.3:0/3746436494 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3e40a0a30 0x7fe3e4133cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.515+0000 7fe3ceffd700 1 -- 172.21.15.3:0/3746436494 >> 172.21.15.3:0/3746436494 conn(0x7fe3e40a3050 msgr2=0x7fe3e409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:19.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.515+0000 7fe3ceffd700 1 -- 172.21.15.3:0/3746436494 shutdown_connections 2023-10-03T10:14:19.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.515+0000 7fe3ceffd700 1 -- 172.21.15.3:0/3746436494 wait complete. 2023-10-03T10:14:19.528 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:19.769 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:19.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.916+0000 7f78ca725700 1 Processor -- start 2023-10-03T10:14:19.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.916+0000 7f78ca725700 1 -- start start 2023-10-03T10:14:19.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.917+0000 7f78ca725700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f78c409e930 0x7f78c409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:19.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.917+0000 7f78ca725700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f78c409f240 0x7f78c41317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:19.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.917+0000 7f78ca725700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78c409f650 0x7f78c4131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:19.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.917+0000 7f78ca725700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78c4070940 con 0x7f78c409f240 2023-10-03T10:14:19.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.917+0000 7f78ca725700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c406fd60 con 0x7f78c409f650 2023-10-03T10:14:19.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.917+0000 7f78ca725700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f78c406fa80 con 0x7f78c409e930 2023-10-03T10:14:19.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.917+0000 7f78c3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f78c409e930 0x7f78c409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:19.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.917+0000 7f78c37fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f78c409f240 0x7f78c41317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:19.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.917+0000 7f78c8cc2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78c409f650 0x7f78c4131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:19.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.917+0000 7f78c37fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f78c409f240 0x7f78c41317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41034/0 (socket says 172.21.15.3:41034) 2023-10-03T10:14:19.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.917+0000 7f78c37fe700 1 -- 172.21.15.3:0/2407807668 learned_addr learned my addr 172.21.15.3:0/2407807668 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:19.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.917+0000 7f78c3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f78c409e930 0x7f78c409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52792/0 (socket says 172.21.15.3:52792) 2023-10-03T10:14:19.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.918+0000 7f78c8cc2700 1 -- 172.21.15.3:0/2407807668 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f78c409e930 msgr2=0x7f78c409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:19.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.918+0000 7f78c8cc2700 1 --2- 172.21.15.3:0/2407807668 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f78c409e930 0x7f78c409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.918+0000 7f78c8cc2700 1 -- 172.21.15.3:0/2407807668 >> v2:172.21.15.3:3300/0 conn(0x7f78c409f240 msgr2=0x7f78c41317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:19.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.918+0000 7f78c8cc2700 1 --2- 172.21.15.3:0/2407807668 >> v2:172.21.15.3:3300/0 conn(0x7f78c409f240 0x7f78c41317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.918+0000 7f78c8cc2700 1 -- 172.21.15.3:0/2407807668 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c409fb60 con 0x7f78c409f650 2023-10-03T10:14:19.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.918+0000 7f78c37fe700 1 --2- 172.21.15.3:0/2407807668 >> v2:172.21.15.3:3300/0 conn(0x7f78c409f240 0x7f78c41317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:19.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.918+0000 7f78c3fff700 1 --2- 172.21.15.3:0/2407807668 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f78c409e930 0x7f78c409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:19.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.918+0000 7f78c8cc2700 1 --2- 172.21.15.3:0/2407807668 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78c409f650 0x7f78c4131d30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009d90 tx=0x7f78bc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c80f0c9187ed0c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:19.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.918+0000 7f78c2ffd700 1 -- 172.21.15.3:0/2407807668 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78bc00e070 con 0x7f78c409f650 2023-10-03T10:14:19.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.918+0000 7f78c2ffd700 1 -- 172.21.15.3:0/2407807668 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78bc00bb20 con 0x7f78c409f650 2023-10-03T10:14:19.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.918+0000 7f78c2ffd700 1 -- 172.21.15.3:0/2407807668 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78bc012430 con 0x7f78c409f650 2023-10-03T10:14:19.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.919+0000 7f78ca725700 1 -- 172.21.15.3:0/2407807668 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78c409f650 msgr2=0x7f78c4131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:19.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.919+0000 7f78ca725700 1 --2- 172.21.15.3:0/2407807668 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78c409f650 0x7f78c4131d30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009d90 tx=0x7f78bc00b330 comp rx=0 tx=0).stop 2023-10-03T10:14:19.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.919+0000 7f78ca725700 1 -- 172.21.15.3:0/2407807668 shutdown_connections 2023-10-03T10:14:19.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.919+0000 7f78ca725700 1 --2- 172.21.15.3:0/2407807668 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78c409f650 0x7f78c4131d30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.919+0000 7f78ca725700 1 --2- 172.21.15.3:0/2407807668 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f78c409e930 0x7f78c409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.919+0000 7f78ca725700 1 --2- 172.21.15.3:0/2407807668 >> v2:172.21.15.3:3300/0 conn(0x7f78c409f240 0x7f78c41317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.919+0000 7f78ca725700 1 -- 172.21.15.3:0/2407807668 >> 172.21.15.3:0/2407807668 conn(0x7f78c40a4210 msgr2=0x7f78c40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:19.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.919+0000 7f78ca725700 1 -- 172.21.15.3:0/2407807668 shutdown_connections 2023-10-03T10:14:19.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.919+0000 7f78ca725700 1 -- 172.21.15.3:0/2407807668 wait complete. 2023-10-03T10:14:19.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.920+0000 7f78ca725700 1 Processor -- start 2023-10-03T10:14:19.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.920+0000 7f78ca725700 1 -- start start 2023-10-03T10:14:19.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.921+0000 7f78ca725700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78c409e930 0x7f78c41a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:19.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.921+0000 7f78ca725700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78c409f240 0x7f78c41a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:19.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.921+0000 7f78ca725700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f78c409f650 0x7f78c41a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:19.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.921+0000 7f78ca725700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c41a8070 con 0x7f78c409e930 2023-10-03T10:14:19.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.921+0000 7f78ca725700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c4071d20 con 0x7f78c409f240 2023-10-03T10:14:19.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.921+0000 7f78ca725700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f78c41a5860 con 0x7f78c409f650 2023-10-03T10:14:19.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.921+0000 7f78c37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78c409f240 0x7f78c41a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:19.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.921+0000 7f78c37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78c409f240 0x7f78c41a40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56244/0 (socket says 172.21.15.3:56244) 2023-10-03T10:14:19.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.921+0000 7f78c37fe700 1 -- 172.21.15.3:0/3164476802 learned_addr learned my addr 172.21.15.3:0/3164476802 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:19.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.921+0000 7f78c8cc2700 1 --2- 172.21.15.3:0/3164476802 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f78c409f650 0x7f78c41a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:19.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.921+0000 7f78c3fff700 1 --2- 172.21.15.3:0/3164476802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78c409e930 0x7f78c41a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:19.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.921+0000 7f78c37fe700 1 -- 172.21.15.3:0/3164476802 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f78c409f650 msgr2=0x7f78c41a4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:19.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.921+0000 7f78c37fe700 1 --2- 172.21.15.3:0/3164476802 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f78c409f650 0x7f78c41a4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.921+0000 7f78c37fe700 1 -- 172.21.15.3:0/3164476802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78c409e930 msgr2=0x7f78c41a3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:19.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.921+0000 7f78c37fe700 1 --2- 172.21.15.3:0/3164476802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78c409e930 0x7f78c41a3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:19.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.921+0000 7f78c37fe700 1 -- 172.21.15.3:0/3164476802 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b40094a0 con 0x7f78c409f240 2023-10-03T10:14:19.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.922+0000 7f78c37fe700 1 --2- 172.21.15.3:0/3164476802 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78c409f240 0x7f78c41a40f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00ebe0 tx=0x7f78ac00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:19.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.922+0000 7f78c0ff9700 1 -- 172.21.15.3:0/3164476802 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ac00c850 con 0x7f78c409f240 2023-10-03T10:14:19.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.922+0000 7f78c0ff9700 1 -- 172.21.15.3:0/3164476802 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ac00c9b0 con 0x7f78c409f240 2023-10-03T10:14:19.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.922+0000 7f78ca725700 1 -- 172.21.15.3:0/3164476802 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78ac009550 con 0x7f78c409f240 2023-10-03T10:14:19.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.922+0000 7f78c0ff9700 1 -- 172.21.15.3:0/3164476802 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ac00cd60 con 0x7f78c409f240 2023-10-03T10:14:19.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.922+0000 7f78ca725700 1 -- 172.21.15.3:0/3164476802 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78bc009a70 con 0x7f78c409f240 2023-10-03T10:14:19.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.923+0000 7f78c0ff9700 1 -- 172.21.15.3:0/3164476802 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f78ac013470 con 0x7f78c409f240 2023-10-03T10:14:19.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.923+0000 7f78c0ff9700 1 -- 172.21.15.3:0/3164476802 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f78ac03f7c0 con 0x7f78c409f240 2023-10-03T10:14:19.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.923+0000 7f78ca725700 1 -- 172.21.15.3:0/3164476802 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78c4074ea0 con 0x7f78c409f240 2023-10-03T10:14:19.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:19.928+0000 7f78c0ff9700 1 -- 172.21.15.3:0/3164476802 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f78ac0232f0 con 0x7f78c409f240 2023-10-03T10:14:20.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.133+0000 7f78ca725700 1 -- 172.21.15.3:0/3164476802 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f78c41a6a10 con 0x7f78c409f240 2023-10-03T10:14:20.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.134+0000 7f78c0ff9700 1 -- 172.21.15.3:0/3164476802 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f78ac021020 con 0x7f78c409f240 2023-10-03T10:14:20.134 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:20.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.136+0000 7f78ca725700 1 -- 172.21.15.3:0/3164476802 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78c409f240 msgr2=0x7f78c41a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:20.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.136+0000 7f78ca725700 1 --2- 172.21.15.3:0/3164476802 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78c409f240 0x7f78c41a40f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00ebe0 tx=0x7f78ac00eef0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.136+0000 7f78ca725700 1 -- 172.21.15.3:0/3164476802 shutdown_connections 2023-10-03T10:14:20.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.136+0000 7f78ca725700 1 --2- 172.21.15.3:0/3164476802 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f78c409f650 0x7f78c41a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.136+0000 7f78ca725700 1 --2- 172.21.15.3:0/3164476802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78c409e930 0x7f78c41a3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.136+0000 7f78ca725700 1 --2- 172.21.15.3:0/3164476802 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78c409f240 0x7f78c41a40f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.136+0000 7f78ca725700 1 -- 172.21.15.3:0/3164476802 >> 172.21.15.3:0/3164476802 conn(0x7f78c40a4210 msgr2=0x7f78c41b8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:20.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.136+0000 7f78ca725700 1 -- 172.21.15.3:0/3164476802 shutdown_connections 2023-10-03T10:14:20.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.136+0000 7f78ca725700 1 -- 172.21.15.3:0/3164476802 wait complete. 2023-10-03T10:14:20.150 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:20.418 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:20.486 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:20.529 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:20.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.567+0000 7f0dc74df700 1 Processor -- start 2023-10-03T10:14:20.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.567+0000 7f0dc74df700 1 -- start start 2023-10-03T10:14:20.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.568+0000 7f0dc74df700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0dc0130670 0x7f0dc0130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:20.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.568+0000 7f0dc74df700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dc006f680 0x7f0dc006fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:20.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.568+0000 7f0dc74df700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dc006ffd0 0x7f0dc0137530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:20.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.568+0000 7f0dc74df700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dc009a350 con 0x7f0dc0130670 2023-10-03T10:14:20.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.568+0000 7f0dc74df700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc0130260 con 0x7f0dc006f680 2023-10-03T10:14:20.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.568+0000 7f0dc74df700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc0071ff0 con 0x7f0dc006ffd0 2023-10-03T10:14:20.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.568+0000 7f0dc527b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0dc0130670 0x7f0dc0130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:20.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.568+0000 7f0dc4a7a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dc006f680 0x7f0dc006fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:20.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.568+0000 7f0dc527b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0dc0130670 0x7f0dc0130a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41044/0 (socket says 172.21.15.3:41044) 2023-10-03T10:14:20.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.568+0000 7f0dc527b700 1 -- 172.21.15.3:0/931549075 learned_addr learned my addr 172.21.15.3:0/931549075 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:20.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.569+0000 7f0dc5a7c700 1 --2- 172.21.15.3:0/931549075 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dc006ffd0 0x7f0dc0137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:20.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.569+0000 7f0dc4a7a700 1 -- 172.21.15.3:0/931549075 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dc006ffd0 msgr2=0x7f0dc0137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:20.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.569+0000 7f0dc4a7a700 1 --2- 172.21.15.3:0/931549075 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dc006ffd0 0x7f0dc0137530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.569+0000 7f0dc4a7a700 1 -- 172.21.15.3:0/931549075 >> v2:172.21.15.3:3300/0 conn(0x7f0dc0130670 msgr2=0x7f0dc0130a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:20.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.569+0000 7f0dc4a7a700 1 --2- 172.21.15.3:0/931549075 >> v2:172.21.15.3:3300/0 conn(0x7f0dc0130670 0x7f0dc0130a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.569+0000 7f0dc4a7a700 1 -- 172.21.15.3:0/931549075 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc009ecf0 con 0x7f0dc006f680 2023-10-03T10:14:20.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.569+0000 7f0dc527b700 1 --2- 172.21.15.3:0/931549075 >> v2:172.21.15.3:3300/0 conn(0x7f0dc0130670 0x7f0dc0130a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:20.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.569+0000 7f0dc4a7a700 1 --2- 172.21.15.3:0/931549075 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dc006f680 0x7f0dc006fa90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009870 tx=0x7f0db0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97fc7a8321af1bf5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:20.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.569+0000 7f0db7fff700 1 -- 172.21.15.3:0/931549075 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db0011830 con 0x7f0dc006f680 2023-10-03T10:14:20.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.569+0000 7f0db7fff700 1 -- 172.21.15.3:0/931549075 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db0011990 con 0x7f0dc006f680 2023-10-03T10:14:20.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.569+0000 7f0db7fff700 1 -- 172.21.15.3:0/931549075 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db0011d20 con 0x7f0dc006f680 2023-10-03T10:14:20.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.570+0000 7f0dc74df700 1 -- 172.21.15.3:0/931549075 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dc006f680 msgr2=0x7f0dc006fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:20.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.570+0000 7f0dc74df700 1 --2- 172.21.15.3:0/931549075 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dc006f680 0x7f0dc006fa90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009870 tx=0x7f0db0009b80 comp rx=0 tx=0).stop 2023-10-03T10:14:20.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.570+0000 7f0dc74df700 1 -- 172.21.15.3:0/931549075 shutdown_connections 2023-10-03T10:14:20.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.570+0000 7f0dc74df700 1 --2- 172.21.15.3:0/931549075 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dc006ffd0 0x7f0dc0137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.570+0000 7f0dc74df700 1 --2- 172.21.15.3:0/931549075 >> v2:172.21.15.3:3300/0 conn(0x7f0dc0130670 0x7f0dc0130a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.570+0000 7f0dc74df700 1 --2- 172.21.15.3:0/931549075 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dc006f680 0x7f0dc006fa90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.570+0000 7f0dc74df700 1 -- 172.21.15.3:0/931549075 >> 172.21.15.3:0/931549075 conn(0x7f0dc00a2aa0 msgr2=0x7f0dc00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:20.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.571+0000 7f0dc74df700 1 -- 172.21.15.3:0/931549075 shutdown_connections 2023-10-03T10:14:20.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.571+0000 7f0dc74df700 1 -- 172.21.15.3:0/931549075 wait complete. 2023-10-03T10:14:20.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.572+0000 7f0dc74df700 1 Processor -- start 2023-10-03T10:14:20.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.572+0000 7f0dc74df700 1 -- start start 2023-10-03T10:14:20.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.573+0000 7f0dc74df700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dc006f680 0x7f0dc011d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:20.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.573+0000 7f0dc74df700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dc006ffd0 0x7f0dc011d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:20.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.573+0000 7f0dc74df700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dc0130670 0x7f0dc011db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:20.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.573+0000 7f0dc74df700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc012b410 con 0x7f0dc0130670 2023-10-03T10:14:20.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.573+0000 7f0dc74df700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc011e4b0 con 0x7f0dc006ffd0 2023-10-03T10:14:20.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.573+0000 7f0dc74df700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc011fe90 con 0x7f0dc006f680 2023-10-03T10:14:20.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.573+0000 7f0dc4a7a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dc006ffd0 0x7f0dc011d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:20.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.573+0000 7f0dc527b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dc006f680 0x7f0dc011d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:20.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.573+0000 7f0dc4a7a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dc006ffd0 0x7f0dc011d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56260/0 (socket says 172.21.15.3:56260) 2023-10-03T10:14:20.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.573+0000 7f0dc4a7a700 1 -- 172.21.15.3:0/1876770266 learned_addr learned my addr 172.21.15.3:0/1876770266 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:20.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.573+0000 7f0dc5a7c700 1 --2- 172.21.15.3:0/1876770266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dc0130670 0x7f0dc011db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:20.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.573+0000 7f0dc527b700 1 -- 172.21.15.3:0/1876770266 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dc006ffd0 msgr2=0x7f0dc011d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:20.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.573+0000 7f0dc527b700 1 --2- 172.21.15.3:0/1876770266 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dc006ffd0 0x7f0dc011d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.573+0000 7f0dc527b700 1 -- 172.21.15.3:0/1876770266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dc0130670 msgr2=0x7f0dc011db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:20.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.573+0000 7f0dc527b700 1 --2- 172.21.15.3:0/1876770266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dc0130670 0x7f0dc011db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.573+0000 7f0dc527b700 1 -- 172.21.15.3:0/1876770266 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dbc00b7f0 con 0x7f0dc006f680 2023-10-03T10:14:20.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.573+0000 7f0dc4a7a700 1 --2- 172.21.15.3:0/1876770266 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dc006ffd0 0x7f0dc011d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:20.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.574+0000 7f0dc527b700 1 --2- 172.21.15.3:0/1876770266 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dc006f680 0x7f0dc011d0e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00ed40 tx=0x7f0dbc00d630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:20.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.574+0000 7f0db5ffb700 1 -- 172.21.15.3:0/1876770266 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc00dd90 con 0x7f0dc006f680 2023-10-03T10:14:20.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.574+0000 7f0db5ffb700 1 -- 172.21.15.3:0/1876770266 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dbc004030 con 0x7f0dc006f680 2023-10-03T10:14:20.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.574+0000 7f0dc74df700 1 -- 172.21.15.3:0/1876770266 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db00094a0 con 0x7f0dc006f680 2023-10-03T10:14:20.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.574+0000 7f0db5ffb700 1 -- 172.21.15.3:0/1876770266 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc0043c0 con 0x7f0dc006f680 2023-10-03T10:14:20.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.574+0000 7f0dc74df700 1 -- 172.21.15.3:0/1876770266 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc011fa80 con 0x7f0dc006f680 2023-10-03T10:14:20.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.575+0000 7f0db5ffb700 1 -- 172.21.15.3:0/1876770266 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0dbc0036a0 con 0x7f0dc006f680 2023-10-03T10:14:20.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.575+0000 7f0db5ffb700 1 -- 172.21.15.3:0/1876770266 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0dbc03fad0 con 0x7f0dc006f680 2023-10-03T10:14:20.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.576+0000 7f0dc74df700 1 -- 172.21.15.3:0/1876770266 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dc0074ea0 con 0x7f0dc006f680 2023-10-03T10:14:20.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.581+0000 7f0db5ffb700 1 -- 172.21.15.3:0/1876770266 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0dbc01f8f0 con 0x7f0dc006f680 2023-10-03T10:14:20.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.655+0000 7fa3b79b4700 1 Processor -- start 2023-10-03T10:14:20.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.655+0000 7fa3b79b4700 1 -- start start 2023-10-03T10:14:20.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.656+0000 7fa3b79b4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3b009d6f0 0x7fa3b009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:20.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.656+0000 7fa3b79b4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3b009e000 0x7fa3b01339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:20.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.656+0000 7fa3b79b4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b009b120 con 0x7fa3b009e410 2023-10-03T10:14:20.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.656+0000 7fa3b79b4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b006b460 con 0x7fa3b009e000 2023-10-03T10:14:20.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.656+0000 7fa3b79b4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b009a7e0 con 0x7fa3b009d6f0 2023-10-03T10:14:20.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.656+0000 7fa3b5750700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3b009d6f0 0x7fa3b009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:20.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.656+0000 7fa3b4f4f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3b009e000 0x7fa3b01339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:20.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.656+0000 7fa3b5750700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3b009d6f0 0x7fa3b009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52832/0 (socket says 172.21.15.3:52832) 2023-10-03T10:14:20.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.656+0000 7fa3b4f4f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3b009e000 0x7fa3b01339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56270/0 (socket says 172.21.15.3:56270) 2023-10-03T10:14:20.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.656+0000 7fa3b5750700 1 -- 172.21.15.3:0/3040415502 learned_addr learned my addr 172.21.15.3:0/3040415502 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:20.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.656+0000 7fa3a7fff700 1 -- 172.21.15.3:0/3040415502 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2798529103 0 0) 0x7fa3b009b120 con 0x7fa3b009e410 2023-10-03T10:14:20.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.656+0000 7fa3b5750700 1 -- 172.21.15.3:0/3040415502 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3b009e000 msgr2=0x7fa3b01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:20.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.656+0000 7fa3b5750700 1 --2- 172.21.15.3:0/3040415502 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3b009e000 0x7fa3b01339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.656+0000 7fa3b5750700 1 -- 172.21.15.3:0/3040415502 >> v1:172.21.15.3:6789/0 conn(0x7fa3b009e410 legacy=0x7fa3b0133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:20.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.656+0000 7fa3b5750700 1 -- 172.21.15.3:0/3040415502 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b009a500 con 0x7fa3b009d6f0 2023-10-03T10:14:20.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.657+0000 7fa3b5750700 1 --2- 172.21.15.3:0/3040415502 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3b009d6f0 0x7fa3b009dac0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac009950 tx=0x7fa3ac009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=865a078167dfc54a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:20.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.657+0000 7fa3a7fff700 1 -- 172.21.15.3:0/3040415502 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac009ea0 con 0x7fa3b009d6f0 2023-10-03T10:14:20.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.657+0000 7fa3a7fff700 1 -- 172.21.15.3:0/3040415502 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ac00ba20 con 0x7fa3b009d6f0 2023-10-03T10:14:20.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.657+0000 7fa3a7fff700 1 -- 172.21.15.3:0/3040415502 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac00bdb0 con 0x7fa3b009d6f0 2023-10-03T10:14:20.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.658+0000 7fa3b79b4700 1 -- 172.21.15.3:0/3040415502 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3b009d6f0 msgr2=0x7fa3b009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:20.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.658+0000 7fa3b79b4700 1 --2- 172.21.15.3:0/3040415502 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3b009d6f0 0x7fa3b009dac0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac009950 tx=0x7fa3ac009c60 comp rx=0 tx=0).stop 2023-10-03T10:14:20.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.658+0000 7fa3b79b4700 1 -- 172.21.15.3:0/3040415502 shutdown_connections 2023-10-03T10:14:20.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.658+0000 7fa3b79b4700 1 --2- 172.21.15.3:0/3040415502 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3b009d6f0 0x7fa3b009dac0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.658+0000 7fa3b79b4700 1 --2- 172.21.15.3:0/3040415502 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3b009e000 0x7fa3b01339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.658+0000 7fa3b79b4700 1 -- 172.21.15.3:0/3040415502 >> 172.21.15.3:0/3040415502 conn(0x7fa3b0066f50 msgr2=0x7fa3b009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:20.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.658+0000 7fa3b79b4700 1 -- 172.21.15.3:0/3040415502 shutdown_connections 2023-10-03T10:14:20.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.658+0000 7fa3b79b4700 1 -- 172.21.15.3:0/3040415502 wait complete. 2023-10-03T10:14:20.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.659+0000 7fa3b79b4700 1 Processor -- start 2023-10-03T10:14:20.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.659+0000 7fa3b79b4700 1 -- start start 2023-10-03T10:14:20.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.659+0000 7fa3b79b4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3b009d6f0 0x7fa3b01c5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:20.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.660+0000 7fa3b79b4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3b009e000 0x7fa3b01c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:20.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.660+0000 7fa3b79b4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3b009e410 0x7fa3b01c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:20.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.660+0000 7fa3b79b4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b01a72e0 con 0x7fa3b009e410 2023-10-03T10:14:20.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.660+0000 7fa3b79b4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b01c5ad0 con 0x7fa3b009e000 2023-10-03T10:14:20.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.660+0000 7fa3b79b4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b01a9820 con 0x7fa3b009d6f0 2023-10-03T10:14:20.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.660+0000 7fa3b5750700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3b009d6f0 0x7fa3b01c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:20.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.660+0000 7fa3b5f51700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3b009e410 0x7fa3b01c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:20.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.660+0000 7fa3b5750700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3b009d6f0 0x7fa3b01c5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52834/0 (socket says 172.21.15.3:52834) 2023-10-03T10:14:20.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.660+0000 7fa3b5750700 1 -- 172.21.15.3:0/895520398 learned_addr learned my addr 172.21.15.3:0/895520398 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:20.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.660+0000 7fa3b4f4f700 1 --2- 172.21.15.3:0/895520398 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3b009e000 0x7fa3b01c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:20.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.660+0000 7fa3b5750700 1 -- 172.21.15.3:0/895520398 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3b009e000 msgr2=0x7fa3b01c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:20.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.660+0000 7fa3b5750700 1 --2- 172.21.15.3:0/895520398 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3b009e000 0x7fa3b01c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.660+0000 7fa3b5750700 1 -- 172.21.15.3:0/895520398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3b009e410 msgr2=0x7fa3b01c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:20.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.660+0000 7fa3b5750700 1 --2- 172.21.15.3:0/895520398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3b009e410 0x7fa3b01c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.660+0000 7fa3b5750700 1 -- 172.21.15.3:0/895520398 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac009550 con 0x7fa3b009d6f0 2023-10-03T10:14:20.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.660+0000 7fa3b5750700 1 --2- 172.21.15.3:0/895520398 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3b009d6f0 0x7fa3b01c5050 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac009870 tx=0x7fa3ac004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:20.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.660+0000 7fa3a5ffb700 1 -- 172.21.15.3:0/895520398 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac00be90 con 0x7fa3b009d6f0 2023-10-03T10:14:20.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.661+0000 7fa3a5ffb700 1 -- 172.21.15.3:0/895520398 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ac01f070 con 0x7fa3b009d6f0 2023-10-03T10:14:20.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.661+0000 7fa3b79b4700 1 -- 172.21.15.3:0/895520398 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b00a3040 con 0x7fa3b009d6f0 2023-10-03T10:14:20.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.661+0000 7fa3a5ffb700 1 -- 172.21.15.3:0/895520398 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac0036a0 con 0x7fa3b009d6f0 2023-10-03T10:14:20.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.661+0000 7fa3b79b4700 1 -- 172.21.15.3:0/895520398 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b01a7910 con 0x7fa3b009d6f0 2023-10-03T10:14:20.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.662+0000 7fa3b79b4700 1 -- 172.21.15.3:0/895520398 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b0074ea0 con 0x7fa3b009d6f0 2023-10-03T10:14:20.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.662+0000 7fa3a5ffb700 1 -- 172.21.15.3:0/895520398 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa3ac021030 con 0x7fa3b009d6f0 2023-10-03T10:14:20.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.663+0000 7fa3a5ffb700 1 -- 172.21.15.3:0/895520398 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa3ac041240 con 0x7fa3b009d6f0 2023-10-03T10:14:20.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.667+0000 7fa3a5ffb700 1 -- 172.21.15.3:0/895520398 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa3ac0199a0 con 0x7fa3b009d6f0 2023-10-03T10:14:20.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.675+0000 7fe70dd09700 1 Processor -- start 2023-10-03T10:14:20.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.675+0000 7fe70dd09700 1 -- start start 2023-10-03T10:14:20.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.675+0000 7fe70dd09700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7080a0090 0x7fe708136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:20.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.675+0000 7fe70dd09700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe70809d130 0x7fe708139170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:20.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.675+0000 7fe70dd09700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe70809ba70 con 0x7fe70809c650 2023-10-03T10:14:20.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.675+0000 7fe70dd09700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe70809e8f0 con 0x7fe70809d130 2023-10-03T10:14:20.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.675+0000 7fe70dd09700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7080a0760 con 0x7fe7080a0090 2023-10-03T10:14:20.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.675+0000 7fe707fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe70809d130 0x7fe708139170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:20.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.675+0000 7fe706ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7080a0090 0x7fe708136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:20.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.675+0000 7fe706ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7080a0090 0x7fe708136c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56292/0 (socket says 172.21.15.3:56292) 2023-10-03T10:14:20.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.675+0000 7fe707fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe70809d130 0x7fe708139170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41080/0 (socket says 172.21.15.3:41080) 2023-10-03T10:14:20.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.675+0000 7fe706ffd700 1 -- 172.21.15.3:0/2143640041 learned_addr learned my addr 172.21.15.3:0/2143640041 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:20.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.676+0000 7fe706ffd700 1 -- 172.21.15.3:0/2143640041 >> v2:172.21.15.3:3300/0 conn(0x7fe70809d130 msgr2=0x7fe708139170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:20.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.676+0000 7fe706ffd700 1 --2- 172.21.15.3:0/2143640041 >> v2:172.21.15.3:3300/0 conn(0x7fe70809d130 0x7fe708139170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.676+0000 7fe706ffd700 1 -- 172.21.15.3:0/2143640041 >> v1:172.21.15.3:6789/0 conn(0x7fe70809c650 legacy=0x7fe70809ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:20.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.676+0000 7fe706ffd700 1 -- 172.21.15.3:0/2143640041 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70809a080 con 0x7fe7080a0090 2023-10-03T10:14:20.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.676+0000 7fe707fff700 1 --2- 172.21.15.3:0/2143640041 >> v2:172.21.15.3:3300/0 conn(0x7fe70809d130 0x7fe708139170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:20.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.676+0000 7fe706ffd700 1 --2- 172.21.15.3:0/2143640041 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7080a0090 0x7fe708136c20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc0098f0 tx=0x7fe6fc009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e279fa79e289627 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:20.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.676+0000 7fe7067fc700 1 -- 172.21.15.3:0/2143640041 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc011830 con 0x7fe7080a0090 2023-10-03T10:14:20.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.676+0000 7fe7067fc700 1 -- 172.21.15.3:0/2143640041 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6fc011990 con 0x7fe7080a0090 2023-10-03T10:14:20.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.676+0000 7fe7067fc700 1 -- 172.21.15.3:0/2143640041 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc011d20 con 0x7fe7080a0090 2023-10-03T10:14:20.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.677+0000 7fe70dd09700 1 -- 172.21.15.3:0/2143640041 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7080a0090 msgr2=0x7fe708136c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:20.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.677+0000 7fe70dd09700 1 --2- 172.21.15.3:0/2143640041 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7080a0090 0x7fe708136c20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc0098f0 tx=0x7fe6fc009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:20.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.677+0000 7fe70dd09700 1 -- 172.21.15.3:0/2143640041 shutdown_connections 2023-10-03T10:14:20.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.677+0000 7fe70dd09700 1 --2- 172.21.15.3:0/2143640041 >> v2:172.21.15.3:3300/0 conn(0x7fe70809d130 0x7fe708139170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.677+0000 7fe70dd09700 1 --2- 172.21.15.3:0/2143640041 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7080a0090 0x7fe708136c20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.677+0000 7fe70dd09700 1 -- 172.21.15.3:0/2143640041 >> 172.21.15.3:0/2143640041 conn(0x7fe7080a3f70 msgr2=0x7fe7080a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:20.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.677+0000 7fe70dd09700 1 -- 172.21.15.3:0/2143640041 shutdown_connections 2023-10-03T10:14:20.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.677+0000 7fe70dd09700 1 -- 172.21.15.3:0/2143640041 wait complete. 2023-10-03T10:14:20.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.678+0000 7fe70dd09700 1 Processor -- start 2023-10-03T10:14:20.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.679+0000 7fe70dd09700 1 -- start start 2023-10-03T10:14:20.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.679+0000 7fe70dd09700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe70809c650 0x7fe708135360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:20.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.679+0000 7fe70dd09700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe70809d130 0x7fe7081358a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:20.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.679+0000 7fe70dd09700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7080a0090 0x7fe708135de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:20.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.679+0000 7fe70dd09700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70811aba0 con 0x7fe70809d130 2023-10-03T10:14:20.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.679+0000 7fe70dd09700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe70809d980 con 0x7fe7080a0090 2023-10-03T10:14:20.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.679+0000 7fe70dd09700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe708137550 con 0x7fe70809c650 2023-10-03T10:14:20.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.679+0000 7fe7077fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe70809c650 0x7fe708135360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:20.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.679+0000 7fe7077fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe70809c650 0x7fe708135360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52850/0 (socket says 172.21.15.3:52850) 2023-10-03T10:14:20.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.679+0000 7fe7077fe700 1 -- 172.21.15.3:0/110332235 learned_addr learned my addr 172.21.15.3:0/110332235 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:20.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.679+0000 7fe707fff700 1 --2- 172.21.15.3:0/110332235 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7080a0090 0x7fe708135de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:20.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.679+0000 7fe706ffd700 1 --2- 172.21.15.3:0/110332235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe70809d130 0x7fe7081358a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:20.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.680+0000 7fe7077fe700 1 -- 172.21.15.3:0/110332235 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7080a0090 msgr2=0x7fe708135de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:20.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.680+0000 7fe7077fe700 1 --2- 172.21.15.3:0/110332235 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7080a0090 0x7fe708135de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.680+0000 7fe7077fe700 1 -- 172.21.15.3:0/110332235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe70809d130 msgr2=0x7fe7081358a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:20.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.680+0000 7fe7077fe700 1 --2- 172.21.15.3:0/110332235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe70809d130 0x7fe7081358a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.680+0000 7fe7077fe700 1 -- 172.21.15.3:0/110332235 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe700009a70 con 0x7fe70809c650 2023-10-03T10:14:20.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.680+0000 7fe7077fe700 1 --2- 172.21.15.3:0/110332235 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe70809c650 0x7fe708135360 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800c4e0 tx=0x7fe6f800c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:20.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.680+0000 7fe6effff700 1 -- 172.21.15.3:0/110332235 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8011790 con 0x7fe70809c650 2023-10-03T10:14:20.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.680+0000 7fe6effff700 1 -- 172.21.15.3:0/110332235 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f80118f0 con 0x7fe70809c650 2023-10-03T10:14:20.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.680+0000 7fe70dd09700 1 -- 172.21.15.3:0/110332235 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6fc009550 con 0x7fe70809c650 2023-10-03T10:14:20.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.680+0000 7fe6effff700 1 -- 172.21.15.3:0/110332235 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8011c60 con 0x7fe70809c650 2023-10-03T10:14:20.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.680+0000 7fe70dd09700 1 -- 172.21.15.3:0/110332235 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe70812f4c0 con 0x7fe70809c650 2023-10-03T10:14:20.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.681+0000 7fe6effff700 1 -- 172.21.15.3:0/110332235 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe6f8005080 con 0x7fe70809c650 2023-10-03T10:14:20.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.681+0000 7fe6effff700 1 -- 172.21.15.3:0/110332235 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe6f803eef0 con 0x7fe70809c650 2023-10-03T10:14:20.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.681+0000 7fe70dd09700 1 -- 172.21.15.3:0/110332235 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7080747f0 con 0x7fe70809c650 2023-10-03T10:14:20.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.686+0000 7fe6effff700 1 -- 172.21.15.3:0/110332235 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe6f8019d50 con 0x7fe70809c650 2023-10-03T10:14:20.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.824+0000 7f0dc74df700 1 -- 172.21.15.3:0/1876770266 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0dc0072210 con 0x7f0dc006f680 2023-10-03T10:14:20.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.824+0000 7f0db5ffb700 1 -- 172.21.15.3:0/1876770266 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0dbc027090 con 0x7f0dc006f680 2023-10-03T10:14:20.825 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:20.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.827+0000 7f0dc74df700 1 -- 172.21.15.3:0/1876770266 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dc006f680 msgr2=0x7f0dc011d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:20.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.827+0000 7f0dc74df700 1 --2- 172.21.15.3:0/1876770266 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dc006f680 0x7f0dc011d0e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00ed40 tx=0x7f0dbc00d630 comp rx=0 tx=0).stop 2023-10-03T10:14:20.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.827+0000 7f0dc74df700 1 -- 172.21.15.3:0/1876770266 shutdown_connections 2023-10-03T10:14:20.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.827+0000 7f0dc74df700 1 --2- 172.21.15.3:0/1876770266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dc0130670 0x7f0dc011db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.827+0000 7f0dc74df700 1 --2- 172.21.15.3:0/1876770266 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dc006f680 0x7f0dc011d0e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.827+0000 7f0dc74df700 1 --2- 172.21.15.3:0/1876770266 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dc006ffd0 0x7f0dc011d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.827+0000 7f0dc74df700 1 -- 172.21.15.3:0/1876770266 >> 172.21.15.3:0/1876770266 conn(0x7f0dc00a2aa0 msgr2=0x7f0dc012d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:20.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.828+0000 7f0dc74df700 1 -- 172.21.15.3:0/1876770266 shutdown_connections 2023-10-03T10:14:20.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.828+0000 7f0dc74df700 1 -- 172.21.15.3:0/1876770266 wait complete. 2023-10-03T10:14:20.842 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:20.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.905+0000 7fa3b79b4700 1 -- 172.21.15.3:0/895520398 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa3b006c360 con 0x7fa3b009d6f0 2023-10-03T10:14:20.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.905+0000 7fa3a5ffb700 1 -- 172.21.15.3:0/895520398 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa3ac015070 con 0x7fa3b009d6f0 2023-10-03T10:14:20.906 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:20.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.908+0000 7fa3b79b4700 1 -- 172.21.15.3:0/895520398 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3b009d6f0 msgr2=0x7fa3b01c5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:20.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.908+0000 7fa3b79b4700 1 --2- 172.21.15.3:0/895520398 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3b009d6f0 0x7fa3b01c5050 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac009870 tx=0x7fa3ac004750 comp rx=0 tx=0).stop 2023-10-03T10:14:20.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.908+0000 7fa3b79b4700 1 -- 172.21.15.3:0/895520398 shutdown_connections 2023-10-03T10:14:20.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.908+0000 7fa3b79b4700 1 --2- 172.21.15.3:0/895520398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3b009e410 0x7fa3b01c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.908+0000 7fa3b79b4700 1 --2- 172.21.15.3:0/895520398 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3b009d6f0 0x7fa3b01c5050 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.908+0000 7fa3b79b4700 1 --2- 172.21.15.3:0/895520398 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3b009e000 0x7fa3b01c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.908+0000 7fa3b79b4700 1 -- 172.21.15.3:0/895520398 >> 172.21.15.3:0/895520398 conn(0x7fa3b0066f50 msgr2=0x7fa3b01bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:20.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.908+0000 7fa3b79b4700 1 -- 172.21.15.3:0/895520398 shutdown_connections 2023-10-03T10:14:20.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.908+0000 7fa3b79b4700 1 -- 172.21.15.3:0/895520398 wait complete. 2023-10-03T10:14:20.922 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:20.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.921+0000 7fe70dd09700 1 -- 172.21.15.3:0/110332235 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe708072840 con 0x7fe70809c650 2023-10-03T10:14:20.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.921+0000 7fe6effff700 1 -- 172.21.15.3:0/110332235 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe6f8019310 con 0x7fe70809c650 2023-10-03T10:14:20.922 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:20.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.924+0000 7fe70dd09700 1 -- 172.21.15.3:0/110332235 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe70809c650 msgr2=0x7fe708135360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:20.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.924+0000 7fe70dd09700 1 --2- 172.21.15.3:0/110332235 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe70809c650 0x7fe708135360 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800c4e0 tx=0x7fe6f800c7f0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.924+0000 7fe70dd09700 1 -- 172.21.15.3:0/110332235 shutdown_connections 2023-10-03T10:14:20.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.924+0000 7fe70dd09700 1 --2- 172.21.15.3:0/110332235 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7080a0090 0x7fe708135de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.924+0000 7fe70dd09700 1 --2- 172.21.15.3:0/110332235 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe70809c650 0x7fe708135360 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.924+0000 7fe70dd09700 1 --2- 172.21.15.3:0/110332235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe70809d130 0x7fe7081358a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:20.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.924+0000 7fe70dd09700 1 -- 172.21.15.3:0/110332235 >> 172.21.15.3:0/110332235 conn(0x7fe7080a3f70 msgr2=0x7fe708071820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:20.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.924+0000 7fe70dd09700 1 -- 172.21.15.3:0/110332235 shutdown_connections 2023-10-03T10:14:20.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:20.924+0000 7fe70dd09700 1 -- 172.21.15.3:0/110332235 wait complete. 2023-10-03T10:14:20.938 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:21.151 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:21.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.298+0000 7f1bd60a5700 1 Processor -- start 2023-10-03T10:14:21.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.298+0000 7f1bd60a5700 1 -- start start 2023-10-03T10:14:21.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.299+0000 7f1bd60a5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1bd01338a0 0x7f1bd0133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:21.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.299+0000 7f1bd60a5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1bd009de50 0x7f1bd009e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:21.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.299+0000 7f1bd60a5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1bd009e760 0x7f1bd013a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:21.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.299+0000 7f1bd60a5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bd006fef0 con 0x7f1bd01338a0 2023-10-03T10:14:21.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.299+0000 7f1bd60a5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd006ef30 con 0x7f1bd009de50 2023-10-03T10:14:21.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.299+0000 7f1bd60a5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd01334b0 con 0x7f1bd009e760 2023-10-03T10:14:21.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.299+0000 7f1bceffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1bd009de50 0x7f1bd009e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:21.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.299+0000 7f1bcffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1bd009e760 0x7f1bd013a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:21.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.299+0000 7f1bceffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1bd009de50 0x7f1bd009e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56320/0 (socket says 172.21.15.3:56320) 2023-10-03T10:14:21.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.299+0000 7f1bceffd700 1 -- 172.21.15.3:0/2480031968 learned_addr learned my addr 172.21.15.3:0/2480031968 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:21.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.299+0000 7f1bcf7fe700 1 --2- 172.21.15.3:0/2480031968 >> v2:172.21.15.3:3300/0 conn(0x7f1bd01338a0 0x7f1bd0133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:21.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.299+0000 7f1bcffff700 1 -- 172.21.15.3:0/2480031968 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1bd009de50 msgr2=0x7f1bd009e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:21.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.299+0000 7f1bcffff700 1 --2- 172.21.15.3:0/2480031968 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1bd009de50 0x7f1bd009e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:21.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.299+0000 7f1bcffff700 1 -- 172.21.15.3:0/2480031968 >> v2:172.21.15.3:3300/0 conn(0x7f1bd01338a0 msgr2=0x7f1bd0133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:21.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.299+0000 7f1bcffff700 1 --2- 172.21.15.3:0/2480031968 >> v2:172.21.15.3:3300/0 conn(0x7f1bd01338a0 0x7f1bd0133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:21.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.299+0000 7f1bcffff700 1 -- 172.21.15.3:0/2480031968 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd006fc30 con 0x7f1bd009e760 2023-10-03T10:14:21.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.299+0000 7f1bcf7fe700 1 --2- 172.21.15.3:0/2480031968 >> v2:172.21.15.3:3300/0 conn(0x7f1bd01338a0 0x7f1bd0133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:21.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.299+0000 7f1bceffd700 1 --2- 172.21.15.3:0/2480031968 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1bd009de50 0x7f1bd009e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:21.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.299+0000 7f1bcffff700 1 --2- 172.21.15.3:0/2480031968 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1bd009e760 0x7f1bd013a730 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8009d90 tx=0x7f1bc800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64e775da1c384ec3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:21.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.299+0000 7f1bce7fc700 1 -- 172.21.15.3:0/2480031968 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc800e070 con 0x7f1bd009e760 2023-10-03T10:14:21.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.300+0000 7f1bce7fc700 1 -- 172.21.15.3:0/2480031968 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc800bb20 con 0x7f1bd009e760 2023-10-03T10:14:21.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.300+0000 7f1bce7fc700 1 -- 172.21.15.3:0/2480031968 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8012430 con 0x7f1bd009e760 2023-10-03T10:14:21.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.300+0000 7f1bd60a5700 1 -- 172.21.15.3:0/2480031968 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1bd009e760 msgr2=0x7f1bd013a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:21.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.300+0000 7f1bd60a5700 1 --2- 172.21.15.3:0/2480031968 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1bd009e760 0x7f1bd013a730 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8009d90 tx=0x7f1bc800b330 comp rx=0 tx=0).stop 2023-10-03T10:14:21.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.300+0000 7f1bd60a5700 1 -- 172.21.15.3:0/2480031968 shutdown_connections 2023-10-03T10:14:21.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.300+0000 7f1bd60a5700 1 --2- 172.21.15.3:0/2480031968 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1bd009e760 0x7f1bd013a730 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:21.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.300+0000 7f1bd60a5700 1 --2- 172.21.15.3:0/2480031968 >> v2:172.21.15.3:3300/0 conn(0x7f1bd01338a0 0x7f1bd0133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:21.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.300+0000 7f1bd60a5700 1 --2- 172.21.15.3:0/2480031968 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1bd009de50 0x7f1bd009e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:21.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.300+0000 7f1bd60a5700 1 -- 172.21.15.3:0/2480031968 >> 172.21.15.3:0/2480031968 conn(0x7f1bd00a2aa0 msgr2=0x7f1bd00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:21.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.300+0000 7f1bd60a5700 1 -- 172.21.15.3:0/2480031968 shutdown_connections 2023-10-03T10:14:21.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.300+0000 7f1bd60a5700 1 -- 172.21.15.3:0/2480031968 wait complete. 2023-10-03T10:14:21.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.301+0000 7f1bd60a5700 1 Processor -- start 2023-10-03T10:14:21.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.301+0000 7f1bd60a5700 1 -- start start 2023-10-03T10:14:21.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.302+0000 7f1bd60a5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1bd009de50 0x7f1bd011a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:21.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.302+0000 7f1bd60a5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1bd009e760 0x7f1bd011aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:21.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.302+0000 7f1bd60a5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1bd01338a0 0x7f1bd011b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:21.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.302+0000 7f1bd60a5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd011e9f0 con 0x7f1bd009e760 2023-10-03T10:14:21.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.302+0000 7f1bd60a5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd011e030 con 0x7f1bd009de50 2023-10-03T10:14:21.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.302+0000 7f1bd60a5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd011d370 con 0x7f1bd01338a0 2023-10-03T10:14:21.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.302+0000 7f1bcf7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1bd009de50 0x7f1bd011a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:21.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.302+0000 7f1bcf7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1bd009de50 0x7f1bd011a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56326/0 (socket says 172.21.15.3:56326) 2023-10-03T10:14:21.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.302+0000 7f1bcf7fe700 1 -- 172.21.15.3:0/3781610182 learned_addr learned my addr 172.21.15.3:0/3781610182 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:21.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.302+0000 7f1bceffd700 1 --2- 172.21.15.3:0/3781610182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1bd009e760 0x7f1bd011aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:21.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.302+0000 7f1bcffff700 1 --2- 172.21.15.3:0/3781610182 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1bd01338a0 0x7f1bd011b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:21.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.302+0000 7f1bcf7fe700 1 -- 172.21.15.3:0/3781610182 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1bd01338a0 msgr2=0x7f1bd011b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:21.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.302+0000 7f1bcf7fe700 1 --2- 172.21.15.3:0/3781610182 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1bd01338a0 0x7f1bd011b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:21.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.302+0000 7f1bcf7fe700 1 -- 172.21.15.3:0/3781610182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1bd009e760 msgr2=0x7f1bd011aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:21.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.302+0000 7f1bcf7fe700 1 --2- 172.21.15.3:0/3781610182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1bd009e760 0x7f1bd011aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:21.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.302+0000 7f1bcf7fe700 1 -- 172.21.15.3:0/3781610182 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc4009550 con 0x7f1bd009de50 2023-10-03T10:14:21.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.302+0000 7f1bcf7fe700 1 --2- 172.21.15.3:0/3781610182 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1bd009de50 0x7f1bd011a5a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1bc000ec10 tx=0x7f1bc000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:21.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.303+0000 7f1bb7fff700 1 -- 172.21.15.3:0/3781610182 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc000c850 con 0x7f1bd009de50 2023-10-03T10:14:21.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.303+0000 7f1bb7fff700 1 -- 172.21.15.3:0/3781610182 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc000c9b0 con 0x7f1bd009de50 2023-10-03T10:14:21.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.303+0000 7f1bb7fff700 1 -- 172.21.15.3:0/3781610182 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc000cd20 con 0x7f1bd009de50 2023-10-03T10:14:21.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.303+0000 7f1bd60a5700 1 -- 172.21.15.3:0/3781610182 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc00094a0 con 0x7f1bd009de50 2023-10-03T10:14:21.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.303+0000 7f1bd60a5700 1 -- 172.21.15.3:0/3781610182 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc8009a70 con 0x7f1bd009de50 2023-10-03T10:14:21.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.303+0000 7f1bb7fff700 1 -- 172.21.15.3:0/3781610182 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1bc0013470 con 0x7f1bd009de50 2023-10-03T10:14:21.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.304+0000 7f1bb7fff700 1 -- 172.21.15.3:0/3781610182 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1bc003fae0 con 0x7f1bd009de50 2023-10-03T10:14:21.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.304+0000 7f1bd60a5700 1 -- 172.21.15.3:0/3781610182 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bd011dbd0 con 0x7f1bd009de50 2023-10-03T10:14:21.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.309+0000 7f1bb7fff700 1 -- 172.21.15.3:0/3781610182 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1bc0021020 con 0x7f1bd009de50 2023-10-03T10:14:21.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.514+0000 7f1bd60a5700 1 -- 172.21.15.3:0/3781610182 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1bd0074ea0 con 0x7f1bd009de50 2023-10-03T10:14:21.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.514+0000 7f1bb7fff700 1 -- 172.21.15.3:0/3781610182 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1bc0017070 con 0x7f1bd009de50 2023-10-03T10:14:21.515 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:21.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.516+0000 7f1bd60a5700 1 -- 172.21.15.3:0/3781610182 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1bd009de50 msgr2=0x7f1bd011a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:21.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.516+0000 7f1bd60a5700 1 --2- 172.21.15.3:0/3781610182 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1bd009de50 0x7f1bd011a5a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1bc000ec10 tx=0x7f1bc000ef20 comp rx=0 tx=0).stop 2023-10-03T10:14:21.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.516+0000 7f1bd60a5700 1 -- 172.21.15.3:0/3781610182 shutdown_connections 2023-10-03T10:14:21.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.516+0000 7f1bd60a5700 1 --2- 172.21.15.3:0/3781610182 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1bd01338a0 0x7f1bd011b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:21.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.516+0000 7f1bd60a5700 1 --2- 172.21.15.3:0/3781610182 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1bd009de50 0x7f1bd011a5a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:21.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.516+0000 7f1bd60a5700 1 --2- 172.21.15.3:0/3781610182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1bd009e760 0x7f1bd011aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:21.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.516+0000 7f1bd60a5700 1 -- 172.21.15.3:0/3781610182 >> 172.21.15.3:0/3781610182 conn(0x7f1bd00a2aa0 msgr2=0x7f1bd0130730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:21.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.516+0000 7f1bd60a5700 1 -- 172.21.15.3:0/3781610182 shutdown_connections 2023-10-03T10:14:21.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.516+0000 7f1bd60a5700 1 -- 172.21.15.3:0/3781610182 wait complete. 2023-10-03T10:14:21.530 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:21.843 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:21.923 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:21.938 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:21.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.994+0000 7f470604a700 1 Processor -- start 2023-10-03T10:14:21.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.994+0000 7f470604a700 1 -- start start 2023-10-03T10:14:21.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.995+0000 7f470604a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f47000a0a30 0x7f47000a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:21.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.995+0000 7f470604a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f470009d630 0x7f47000a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:21.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.995+0000 7f470604a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47000a2a70 con 0x7f4700133930 2023-10-03T10:14:21.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.995+0000 7f470604a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47000a01d0 con 0x7f47000a0a30 2023-10-03T10:14:21.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.995+0000 7f46feffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f470009d630 0x7f47000a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:21.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.995+0000 7f470604a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470009e2b0 con 0x7f470009d630 2023-10-03T10:14:21.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.995+0000 7f46ff7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f47000a0a30 0x7f47000a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:21.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.995+0000 7f46ff7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f47000a0a30 0x7f47000a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41120/0 (socket says 172.21.15.3:41120) 2023-10-03T10:14:21.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.995+0000 7f46ff7fe700 1 -- 172.21.15.3:0/2881326280 learned_addr learned my addr 172.21.15.3:0/2881326280 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:21.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.995+0000 7f46fffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4700133930 0x7f4700133d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44106/0 (socket says 172.21.15.3:44106) 2023-10-03T10:14:21.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.995+0000 7f46ff7fe700 1 -- 172.21.15.3:0/2881326280 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f470009d630 msgr2=0x7f47000a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:21.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.995+0000 7f46ff7fe700 1 --2- 172.21.15.3:0/2881326280 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f470009d630 0x7f47000a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:21.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.995+0000 7f46ff7fe700 1 -- 172.21.15.3:0/2881326280 >> v1:172.21.15.3:6789/0 conn(0x7f4700133930 legacy=0x7f4700133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:21.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.996+0000 7f46ff7fe700 1 -- 172.21.15.3:0/2881326280 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470009dc50 con 0x7f47000a0a30 2023-10-03T10:14:21.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.996+0000 7f46fe7fc700 1 -- 172.21.15.3:0/2881326280 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3144051324 0 0) 0x7f47000a2a70 con 0x7f4700133930 2023-10-03T10:14:21.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.996+0000 7f46feffd700 1 --2- 172.21.15.3:0/2881326280 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f470009d630 0x7f47000a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:21.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.996+0000 7f46ff7fe700 1 --2- 172.21.15.3:0/2881326280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47000a0a30 0x7f47000a0e00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f46f00098f0 tx=0x7f46f0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f430566fd23818fd server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:21.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.996+0000 7f46fe7fc700 1 -- 172.21.15.3:0/2881326280 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0011830 con 0x7f47000a0a30 2023-10-03T10:14:21.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.996+0000 7f46fe7fc700 1 -- 172.21.15.3:0/2881326280 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f0011990 con 0x7f47000a0a30 2023-10-03T10:14:21.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.996+0000 7f46fe7fc700 1 -- 172.21.15.3:0/2881326280 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0011d20 con 0x7f47000a0a30 2023-10-03T10:14:21.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.997+0000 7f470604a700 1 -- 172.21.15.3:0/2881326280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47000a0a30 msgr2=0x7f47000a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:21.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.997+0000 7f470604a700 1 --2- 172.21.15.3:0/2881326280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47000a0a30 0x7f47000a0e00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f46f00098f0 tx=0x7f46f0009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:21.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.997+0000 7f470604a700 1 -- 172.21.15.3:0/2881326280 shutdown_connections 2023-10-03T10:14:21.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.997+0000 7f470604a700 1 --2- 172.21.15.3:0/2881326280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47000a0a30 0x7f47000a0e00 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:21.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.997+0000 7f470604a700 1 --2- 172.21.15.3:0/2881326280 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f470009d630 0x7f47000a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:21.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.997+0000 7f470604a700 1 -- 172.21.15.3:0/2881326280 >> 172.21.15.3:0/2881326280 conn(0x7f47000a3050 msgr2=0x7f47000a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:21.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.997+0000 7f470604a700 1 -- 172.21.15.3:0/2881326280 shutdown_connections 2023-10-03T10:14:21.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.997+0000 7f470604a700 1 -- 172.21.15.3:0/2881326280 wait complete. 2023-10-03T10:14:22.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.998+0000 7f470604a700 1 Processor -- start 2023-10-03T10:14:22.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.998+0000 7f470604a700 1 -- start start 2023-10-03T10:14:22.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.999+0000 7f470604a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f470009d630 0x7f47001c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:22.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.999+0000 7f470604a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47000a0a30 0x7f47001c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:22.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.999+0000 7f470604a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4700133930 0x7f47001c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:22.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.999+0000 7f470604a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47001206a0 con 0x7f4700133930 2023-10-03T10:14:22.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.999+0000 7f470604a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47001204b0 con 0x7f470009d630 2023-10-03T10:14:22.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.999+0000 7f470604a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f47001223c0 con 0x7f47000a0a30 2023-10-03T10:14:22.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.999+0000 7f46ff7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f470009d630 0x7f47001c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:22.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.999+0000 7f46fffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4700133930 0x7f47001c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:22.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.999+0000 7f46ff7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f470009d630 0x7f47001c7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56348/0 (socket says 172.21.15.3:56348) 2023-10-03T10:14:22.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.999+0000 7f46fffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4700133930 0x7f47001c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41124/0 (socket says 172.21.15.3:41124) 2023-10-03T10:14:22.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.999+0000 7f46ff7fe700 1 -- 172.21.15.3:0/126124665 learned_addr learned my addr 172.21.15.3:0/126124665 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:22.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.999+0000 7f46feffd700 1 --2- 172.21.15.3:0/126124665 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47000a0a30 0x7f47001c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:22.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.999+0000 7f46fffff700 1 -- 172.21.15.3:0/126124665 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47000a0a30 msgr2=0x7f47001c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:22.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.999+0000 7f46fffff700 1 --2- 172.21.15.3:0/126124665 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47000a0a30 0x7f47001c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.999+0000 7f46fffff700 1 -- 172.21.15.3:0/126124665 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f470009d630 msgr2=0x7f47001c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:22.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.999+0000 7f46fffff700 1 --2- 172.21.15.3:0/126124665 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f470009d630 0x7f47001c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.999+0000 7f46fffff700 1 -- 172.21.15.3:0/126124665 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f40094a0 con 0x7f4700133930 2023-10-03T10:14:22.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.999+0000 7f46ff7fe700 1 --2- 172.21.15.3:0/126124665 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f470009d630 0x7f47001c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:22.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:21.999+0000 7f46fffff700 1 --2- 172.21.15.3:0/126124665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4700133930 0x7f47001c8a10 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f46f800cbe0 tx=0x7f46f800cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:22.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.000+0000 7f46e7fff700 1 -- 172.21.15.3:0/126124665 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f800aa40 con 0x7f4700133930 2023-10-03T10:14:22.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.000+0000 7f46e7fff700 1 -- 172.21.15.3:0/126124665 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f800abc0 con 0x7f4700133930 2023-10-03T10:14:22.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.000+0000 7f470604a700 1 -- 172.21.15.3:0/126124665 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f0009550 con 0x7f4700133930 2023-10-03T10:14:22.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.000+0000 7f46e7fff700 1 -- 172.21.15.3:0/126124665 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f80070d0 con 0x7f4700133930 2023-10-03T10:14:22.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.000+0000 7f470604a700 1 -- 172.21.15.3:0/126124665 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47001220e0 con 0x7f4700133930 2023-10-03T10:14:22.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.001+0000 7f46e7fff700 1 -- 172.21.15.3:0/126124665 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f46f8007230 con 0x7f4700133930 2023-10-03T10:14:22.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.001+0000 7f470604a700 1 -- 172.21.15.3:0/126124665 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47000747f0 con 0x7f4700133930 2023-10-03T10:14:22.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.001+0000 7f46e7fff700 1 -- 172.21.15.3:0/126124665 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f46f803f0f0 con 0x7f4700133930 2023-10-03T10:14:22.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.006+0000 7f46e7fff700 1 -- 172.21.15.3:0/126124665 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f46f8019860 con 0x7f4700133930 2023-10-03T10:14:22.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.067+0000 7fe5491d7700 1 Processor -- start 2023-10-03T10:14:22.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.067+0000 7fe5491d7700 1 -- start start 2023-10-03T10:14:22.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.068+0000 7fe5491d7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe54406a770 0x7fe54406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:22.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.068+0000 7fe5491d7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe544131d10 0x7fe544139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:22.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.068+0000 7fe5491d7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe54409ece0 con 0x7fe54406b080 2023-10-03T10:14:22.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.068+0000 7fe5491d7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe54409e760 con 0x7fe544131d10 2023-10-03T10:14:22.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.068+0000 7fe5491d7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe54409e040 con 0x7fe54406a770 2023-10-03T10:14:22.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.068+0000 7fe542d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe54406a770 0x7fe54406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:22.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.068+0000 7fe54259c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe54406b080 0x7fe544131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44118/0 (socket says 172.21.15.3:44118) 2023-10-03T10:14:22.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.068+0000 7fe54359e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe544131d10 0x7fe544139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:22.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.068+0000 7fe54259c700 1 -- 172.21.15.3:0/200407724 learned_addr learned my addr 172.21.15.3:0/200407724 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:22.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.068+0000 7fe541d9b700 1 -- 172.21.15.3:0/200407724 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2220839867 0 0) 0x7fe54409ece0 con 0x7fe54406b080 2023-10-03T10:14:22.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.068+0000 7fe541d9b700 1 -- 172.21.15.3:0/200407724 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe52c0035b0 con 0x7fe54406b080 2023-10-03T10:14:22.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.068+0000 7fe542d9d700 1 -- 172.21.15.3:0/200407724 >> v2:172.21.15.3:3300/0 conn(0x7fe544131d10 msgr2=0x7fe544139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:22.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.068+0000 7fe542d9d700 1 --2- 172.21.15.3:0/200407724 >> v2:172.21.15.3:3300/0 conn(0x7fe544131d10 0x7fe544139860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.068+0000 7fe542d9d700 1 -- 172.21.15.3:0/200407724 >> v1:172.21.15.3:6789/0 conn(0x7fe54406b080 legacy=0x7fe544131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:22.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.068+0000 7fe542d9d700 1 -- 172.21.15.3:0/200407724 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54409f630 con 0x7fe54406a770 2023-10-03T10:14:22.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.068+0000 7fe54359e700 1 --2- 172.21.15.3:0/200407724 >> v2:172.21.15.3:3300/0 conn(0x7fe544131d10 0x7fe544139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:22.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.069+0000 7fe542d9d700 1 --2- 172.21.15.3:0/200407724 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe54406a770 0x7fe54406ab40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe534009840 tx=0x7fe534009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a9fcaafd6ed4f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:22.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.069+0000 7fe541d9b700 1 -- 172.21.15.3:0/200407724 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe534011830 con 0x7fe54406a770 2023-10-03T10:14:22.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.069+0000 7fe541d9b700 1 -- 172.21.15.3:0/200407724 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe534011990 con 0x7fe54406a770 2023-10-03T10:14:22.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.069+0000 7fe541d9b700 1 -- 172.21.15.3:0/200407724 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe534011d20 con 0x7fe54406a770 2023-10-03T10:14:22.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.070+0000 7fe5491d7700 1 -- 172.21.15.3:0/200407724 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe54406a770 msgr2=0x7fe54406ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:22.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.070+0000 7fe5491d7700 1 --2- 172.21.15.3:0/200407724 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe54406a770 0x7fe54406ab40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe534009840 tx=0x7fe534009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:22.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.070+0000 7fe5491d7700 1 -- 172.21.15.3:0/200407724 shutdown_connections 2023-10-03T10:14:22.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.070+0000 7fe5491d7700 1 --2- 172.21.15.3:0/200407724 >> v2:172.21.15.3:3300/0 conn(0x7fe544131d10 0x7fe544139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.070+0000 7fe5491d7700 1 --2- 172.21.15.3:0/200407724 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe54406a770 0x7fe54406ab40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.070+0000 7fe5491d7700 1 -- 172.21.15.3:0/200407724 >> 172.21.15.3:0/200407724 conn(0x7fe544066f50 msgr2=0x7fe54409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:22.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.070+0000 7fe5491d7700 1 -- 172.21.15.3:0/200407724 shutdown_connections 2023-10-03T10:14:22.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.070+0000 7fe5491d7700 1 -- 172.21.15.3:0/200407724 wait complete. 2023-10-03T10:14:22.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.071+0000 7fe5491d7700 1 Processor -- start 2023-10-03T10:14:22.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.071+0000 7fe5491d7700 1 -- start start 2023-10-03T10:14:22.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.072+0000 7fe5491d7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe54406a770 0x7fe5441c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:22.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.072+0000 7fe5491d7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe54406b080 0x7fe5441c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:22.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.072+0000 7fe5491d7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe544131d10 0x7fe5441c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:22.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.072+0000 7fe5491d7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe544124800 con 0x7fe54406b080 2023-10-03T10:14:22.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.072+0000 7fe5491d7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5441245d0 con 0x7fe54406a770 2023-10-03T10:14:22.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.072+0000 7fe5491d7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe544122e10 con 0x7fe544131d10 2023-10-03T10:14:22.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.072+0000 7fe54259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe54406b080 0x7fe5441c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:22.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.072+0000 7fe542d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe54406a770 0x7fe5441c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:22.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.072+0000 7fe542d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe54406a770 0x7fe5441c7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56362/0 (socket says 172.21.15.3:56362) 2023-10-03T10:14:22.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.072+0000 7fe54259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe54406b080 0x7fe5441c8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41136/0 (socket says 172.21.15.3:41136) 2023-10-03T10:14:22.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.072+0000 7fe542d9d700 1 -- 172.21.15.3:0/135174212 learned_addr learned my addr 172.21.15.3:0/135174212 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:22.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.072+0000 7fe54359e700 1 --2- 172.21.15.3:0/135174212 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe544131d10 0x7fe5441c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:22.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.072+0000 7fe54259c700 1 -- 172.21.15.3:0/135174212 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe544131d10 msgr2=0x7fe5441c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:22.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.072+0000 7fe54259c700 1 --2- 172.21.15.3:0/135174212 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe544131d10 0x7fe5441c8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.072+0000 7fe54259c700 1 -- 172.21.15.3:0/135174212 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe54406a770 msgr2=0x7fe5441c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:22.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.072+0000 7fe54259c700 1 --2- 172.21.15.3:0/135174212 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe54406a770 0x7fe5441c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.072+0000 7fe54259c700 1 -- 172.21.15.3:0/135174212 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe53c009a70 con 0x7fe54406b080 2023-10-03T10:14:22.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.072+0000 7fe54359e700 1 --2- 172.21.15.3:0/135174212 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe544131d10 0x7fe5441c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:22.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.072+0000 7fe542d9d700 1 --2- 172.21.15.3:0/135174212 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe54406a770 0x7fe5441c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:22.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.073+0000 7fe54259c700 1 --2- 172.21.15.3:0/135174212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe54406b080 0x7fe5441c8510 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe53800c490 tx=0x7fe53800c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:22.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.073+0000 7fe52b7fe700 1 -- 172.21.15.3:0/135174212 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe538005d70 con 0x7fe54406b080 2023-10-03T10:14:22.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.073+0000 7fe52b7fe700 1 -- 172.21.15.3:0/135174212 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5380070d0 con 0x7fe54406b080 2023-10-03T10:14:22.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.073+0000 7fe52b7fe700 1 -- 172.21.15.3:0/135174212 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe538007440 con 0x7fe54406b080 2023-10-03T10:14:22.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.073+0000 7fe5491d7700 1 -- 172.21.15.3:0/135174212 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5340094a0 con 0x7fe54406b080 2023-10-03T10:14:22.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.073+0000 7fe5491d7700 1 -- 172.21.15.3:0/135174212 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe544071e60 con 0x7fe54406b080 2023-10-03T10:14:22.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.074+0000 7fe52b7fe700 1 -- 172.21.15.3:0/135174212 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe5380075a0 con 0x7fe54406b080 2023-10-03T10:14:22.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.074+0000 7fe5491d7700 1 -- 172.21.15.3:0/135174212 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe544131500 con 0x7fe54406b080 2023-10-03T10:14:22.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.074+0000 7fe52b7fe700 1 -- 172.21.15.3:0/135174212 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe53803e710 con 0x7fe54406b080 2023-10-03T10:14:22.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.079+0000 7fe52b7fe700 1 -- 172.21.15.3:0/135174212 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe538017020 con 0x7fe54406b080 2023-10-03T10:14:22.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.108+0000 7fd69b4e7700 1 Processor -- start 2023-10-03T10:14:22.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.108+0000 7fd69b4e7700 1 -- start start 2023-10-03T10:14:22.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.109+0000 7fd69b4e7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd694130750 0x7fd694130b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:22.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.109+0000 7fd69b4e7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd69409d6f0 0x7fd69409db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:22.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.109+0000 7fd69b4e7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd69409c190 con 0x7fd69409e080 2023-10-03T10:14:22.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.109+0000 7fd69b4e7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd69409b090 con 0x7fd694130750 2023-10-03T10:14:22.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.109+0000 7fd69b4e7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69409a6c0 con 0x7fd69409d6f0 2023-10-03T10:14:22.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.109+0000 7fd699283700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd694130750 0x7fd694130b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:22.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.109+0000 7fd699a84700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd69409e080 0x7fd6941374b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44132/0 (socket says 172.21.15.3:44132) 2023-10-03T10:14:22.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.109+0000 7fd698a82700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd69409d6f0 0x7fd69409db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:22.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.109+0000 7fd699a84700 1 -- 172.21.15.3:0/4280315988 learned_addr learned my addr 172.21.15.3:0/4280315988 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:22.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.109+0000 7fd68bfff700 1 -- 172.21.15.3:0/4280315988 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 895723875 0 0) 0x7fd69409c190 con 0x7fd69409e080 2023-10-03T10:14:22.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.110+0000 7fd698a82700 1 -- 172.21.15.3:0/4280315988 >> v2:172.21.15.3:3300/0 conn(0x7fd694130750 msgr2=0x7fd694130b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:22.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.110+0000 7fd698a82700 1 --2- 172.21.15.3:0/4280315988 >> v2:172.21.15.3:3300/0 conn(0x7fd694130750 0x7fd694130b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.110+0000 7fd698a82700 1 -- 172.21.15.3:0/4280315988 >> v1:172.21.15.3:6789/0 conn(0x7fd69409e080 legacy=0x7fd6941374b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:22.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.110+0000 7fd698a82700 1 -- 172.21.15.3:0/4280315988 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69409e500 con 0x7fd69409d6f0 2023-10-03T10:14:22.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.110+0000 7fd699283700 1 --2- 172.21.15.3:0/4280315988 >> v2:172.21.15.3:3300/0 conn(0x7fd694130750 0x7fd694130b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:22.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.110+0000 7fd698a82700 1 --2- 172.21.15.3:0/4280315988 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd69409d6f0 0x7fd69409db40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd684009840 tx=0x7fd684009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3789f495a641127 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:22.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.110+0000 7fd68bfff700 1 -- 172.21.15.3:0/4280315988 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd684011830 con 0x7fd69409d6f0 2023-10-03T10:14:22.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.110+0000 7fd68bfff700 1 -- 172.21.15.3:0/4280315988 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd684011990 con 0x7fd69409d6f0 2023-10-03T10:14:22.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.110+0000 7fd68bfff700 1 -- 172.21.15.3:0/4280315988 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd684011d20 con 0x7fd69409d6f0 2023-10-03T10:14:22.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.111+0000 7fd69b4e7700 1 -- 172.21.15.3:0/4280315988 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd69409d6f0 msgr2=0x7fd69409db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:22.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.111+0000 7fd69b4e7700 1 --2- 172.21.15.3:0/4280315988 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd69409d6f0 0x7fd69409db40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd684009840 tx=0x7fd684009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:22.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.111+0000 7fd69b4e7700 1 -- 172.21.15.3:0/4280315988 shutdown_connections 2023-10-03T10:14:22.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.111+0000 7fd69b4e7700 1 --2- 172.21.15.3:0/4280315988 >> v2:172.21.15.3:3300/0 conn(0x7fd694130750 0x7fd694130b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.111+0000 7fd69b4e7700 1 --2- 172.21.15.3:0/4280315988 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd69409d6f0 0x7fd69409db40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.111+0000 7fd69b4e7700 1 -- 172.21.15.3:0/4280315988 >> 172.21.15.3:0/4280315988 conn(0x7fd69409f470 msgr2=0x7fd6940a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:22.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.111+0000 7fd69b4e7700 1 -- 172.21.15.3:0/4280315988 shutdown_connections 2023-10-03T10:14:22.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.111+0000 7fd69b4e7700 1 -- 172.21.15.3:0/4280315988 wait complete. 2023-10-03T10:14:22.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.112+0000 7fd69b4e7700 1 Processor -- start 2023-10-03T10:14:22.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.112+0000 7fd69b4e7700 1 -- start start 2023-10-03T10:14:22.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.112+0000 7fd69b4e7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd69409d6f0 0x7fd69412dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:22.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.112+0000 7fd69b4e7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd69409e080 0x7fd69412e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:22.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.112+0000 7fd69b4e7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd694130750 0x7fd69412e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:22.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.112+0000 7fd69b4e7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69408bb20 con 0x7fd69409e080 2023-10-03T10:14:22.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.112+0000 7fd69b4e7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69409b2b0 con 0x7fd694130750 2023-10-03T10:14:22.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.112+0000 7fd69b4e7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd69408c3d0 con 0x7fd69409d6f0 2023-10-03T10:14:22.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.113+0000 7fd699a84700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd694130750 0x7fd69412e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:22.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.113+0000 7fd699283700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd69409d6f0 0x7fd69412dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:22.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.113+0000 7fd699a84700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd694130750 0x7fd69412e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56368/0 (socket says 172.21.15.3:56368) 2023-10-03T10:14:22.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.113+0000 7fd699283700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd69409d6f0 0x7fd69412dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52916/0 (socket says 172.21.15.3:52916) 2023-10-03T10:14:22.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.113+0000 7fd699a84700 1 -- 172.21.15.3:0/2548785403 learned_addr learned my addr 172.21.15.3:0/2548785403 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:22.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.113+0000 7fd698a82700 1 --2- 172.21.15.3:0/2548785403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd69409e080 0x7fd69412e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:22.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.113+0000 7fd699283700 1 -- 172.21.15.3:0/2548785403 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd694130750 msgr2=0x7fd69412e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:22.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.113+0000 7fd699283700 1 --2- 172.21.15.3:0/2548785403 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd694130750 0x7fd69412e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.113+0000 7fd699283700 1 -- 172.21.15.3:0/2548785403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd69409e080 msgr2=0x7fd69412e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:22.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.113+0000 7fd699283700 1 --2- 172.21.15.3:0/2548785403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd69409e080 0x7fd69412e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.113+0000 7fd699283700 1 -- 172.21.15.3:0/2548785403 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6900094a0 con 0x7fd69409d6f0 2023-10-03T10:14:22.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.113+0000 7fd698a82700 1 --2- 172.21.15.3:0/2548785403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd69409e080 0x7fd69412e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:22.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.113+0000 7fd699a84700 1 --2- 172.21.15.3:0/2548785403 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd694130750 0x7fd69412e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:22.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.113+0000 7fd699283700 1 --2- 172.21.15.3:0/2548785403 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd69409d6f0 0x7fd69412dce0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd69000e9e0 tx=0x7fd69000ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:22.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.113+0000 7fd689ffb700 1 -- 172.21.15.3:0/2548785403 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69000ee80 con 0x7fd69409d6f0 2023-10-03T10:14:22.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.113+0000 7fd689ffb700 1 -- 172.21.15.3:0/2548785403 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69000c850 con 0x7fd69409d6f0 2023-10-03T10:14:22.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.113+0000 7fd69b4e7700 1 -- 172.21.15.3:0/2548785403 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6840094a0 con 0x7fd69409d6f0 2023-10-03T10:14:22.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.114+0000 7fd689ffb700 1 -- 172.21.15.3:0/2548785403 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69000cc00 con 0x7fd69409d6f0 2023-10-03T10:14:22.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.114+0000 7fd69b4e7700 1 -- 172.21.15.3:0/2548785403 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd69408ce40 con 0x7fd69409d6f0 2023-10-03T10:14:22.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.115+0000 7fd69b4e7700 1 -- 172.21.15.3:0/2548785403 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6940747f0 con 0x7fd69409d6f0 2023-10-03T10:14:22.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.116+0000 7fd689ffb700 1 -- 172.21.15.3:0/2548785403 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd69000cd60 con 0x7fd69409d6f0 2023-10-03T10:14:22.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.116+0000 7fd689ffb700 1 -- 172.21.15.3:0/2548785403 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd69003fee0 con 0x7fd69409d6f0 2023-10-03T10:14:22.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.120+0000 7fd689ffb700 1 -- 172.21.15.3:0/2548785403 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd690011350 con 0x7fd69409d6f0 2023-10-03T10:14:22.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.218+0000 7f470604a700 1 -- 172.21.15.3:0/126124665 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4700120ca0 con 0x7f4700133930 2023-10-03T10:14:22.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.218+0000 7f46e7fff700 1 -- 172.21.15.3:0/126124665 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f46f8018020 con 0x7f4700133930 2023-10-03T10:14:22.219 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:22.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.220+0000 7f470604a700 1 -- 172.21.15.3:0/126124665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4700133930 msgr2=0x7f47001c8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:22.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.220+0000 7f470604a700 1 --2- 172.21.15.3:0/126124665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4700133930 0x7f47001c8a10 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f46f800cbe0 tx=0x7f46f800cef0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.220+0000 7f470604a700 1 -- 172.21.15.3:0/126124665 shutdown_connections 2023-10-03T10:14:22.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.220+0000 7f470604a700 1 --2- 172.21.15.3:0/126124665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4700133930 0x7f47001c8a10 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.220+0000 7f470604a700 1 --2- 172.21.15.3:0/126124665 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f470009d630 0x7f47001c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.220+0000 7f470604a700 1 --2- 172.21.15.3:0/126124665 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47000a0a30 0x7f47001c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.220+0000 7f470604a700 1 -- 172.21.15.3:0/126124665 >> 172.21.15.3:0/126124665 conn(0x7f47000a3050 msgr2=0x7f47001bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:22.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.220+0000 7f470604a700 1 -- 172.21.15.3:0/126124665 shutdown_connections 2023-10-03T10:14:22.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.220+0000 7f470604a700 1 -- 172.21.15.3:0/126124665 wait complete. 2023-10-03T10:14:22.235 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:22.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.289+0000 7fe5491d7700 1 -- 172.21.15.3:0/135174212 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe544074ea0 con 0x7fe54406b080 2023-10-03T10:14:22.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.289+0000 7fe52b7fe700 1 -- 172.21.15.3:0/135174212 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe538018af0 con 0x7fe54406b080 2023-10-03T10:14:22.289 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:22.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.291+0000 7fe5491d7700 1 -- 172.21.15.3:0/135174212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe54406b080 msgr2=0x7fe5441c8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:22.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.291+0000 7fe5491d7700 1 --2- 172.21.15.3:0/135174212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe54406b080 0x7fe5441c8510 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe53800c490 tx=0x7fe53800c7a0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.291+0000 7fe5491d7700 1 -- 172.21.15.3:0/135174212 shutdown_connections 2023-10-03T10:14:22.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.291+0000 7fe5491d7700 1 --2- 172.21.15.3:0/135174212 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe544131d10 0x7fe5441c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.291+0000 7fe5491d7700 1 --2- 172.21.15.3:0/135174212 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe54406a770 0x7fe5441c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.291+0000 7fe5491d7700 1 --2- 172.21.15.3:0/135174212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe54406b080 0x7fe5441c8510 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.291+0000 7fe5491d7700 1 -- 172.21.15.3:0/135174212 >> 172.21.15.3:0/135174212 conn(0x7fe544066f50 msgr2=0x7fe5440708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:22.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.291+0000 7fe5491d7700 1 -- 172.21.15.3:0/135174212 shutdown_connections 2023-10-03T10:14:22.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.292+0000 7fe5491d7700 1 -- 172.21.15.3:0/135174212 wait complete. 2023-10-03T10:14:22.305 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:22.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.330+0000 7fd69b4e7700 1 -- 172.21.15.3:0/2548785403 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd69406b250 con 0x7fd69409d6f0 2023-10-03T10:14:22.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.331+0000 7fd689ffb700 1 -- 172.21.15.3:0/2548785403 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd690021020 con 0x7fd69409d6f0 2023-10-03T10:14:22.331 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:22.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.333+0000 7fd69b4e7700 1 -- 172.21.15.3:0/2548785403 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd69409d6f0 msgr2=0x7fd69412dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:22.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.333+0000 7fd69b4e7700 1 --2- 172.21.15.3:0/2548785403 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd69409d6f0 0x7fd69412dce0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd69000e9e0 tx=0x7fd69000ecf0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.333+0000 7fd69b4e7700 1 -- 172.21.15.3:0/2548785403 shutdown_connections 2023-10-03T10:14:22.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.333+0000 7fd69b4e7700 1 --2- 172.21.15.3:0/2548785403 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd694130750 0x7fd69412e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.333+0000 7fd69b4e7700 1 --2- 172.21.15.3:0/2548785403 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd69409d6f0 0x7fd69412dce0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.333+0000 7fd69b4e7700 1 --2- 172.21.15.3:0/2548785403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd69409e080 0x7fd69412e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.333+0000 7fd69b4e7700 1 -- 172.21.15.3:0/2548785403 >> 172.21.15.3:0/2548785403 conn(0x7fd69409f470 msgr2=0x7fd694070100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:22.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.333+0000 7fd69b4e7700 1 -- 172.21.15.3:0/2548785403 shutdown_connections 2023-10-03T10:14:22.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.333+0000 7fd69b4e7700 1 -- 172.21.15.3:0/2548785403 wait complete. 2023-10-03T10:14:22.347 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:22.531 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:22.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.678+0000 7fdc5c4ff700 1 Processor -- start 2023-10-03T10:14:22.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.678+0000 7fdc5c4ff700 1 -- start start 2023-10-03T10:14:22.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.679+0000 7fdc5c4ff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc5409d6f0 0x7fdc5409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:22.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.679+0000 7fdc5c4ff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdc5409e000 0x7fdc541339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:22.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.679+0000 7fdc5c4ff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc5409e410 0x7fdc54133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:22.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.679+0000 7fdc5c4ff700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc5406b460 con 0x7fdc5409e000 2023-10-03T10:14:22.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.679+0000 7fdc5c4ff700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5409b120 con 0x7fdc5409d6f0 2023-10-03T10:14:22.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.679+0000 7fdc5c4ff700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5409a7e0 con 0x7fdc5409e410 2023-10-03T10:14:22.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.679+0000 7fdc5a29b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc5409d6f0 0x7fdc5409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:22.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.679+0000 7fdc59a9a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdc5409e000 0x7fdc541339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:22.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.679+0000 7fdc5aa9c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc5409e410 0x7fdc54133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:22.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.679+0000 7fdc5a29b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc5409d6f0 0x7fdc5409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:56384/0 (socket says 172.21.15.3:56384) 2023-10-03T10:14:22.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.679+0000 7fdc5a29b700 1 -- 172.21.15.3:0/2726455183 learned_addr learned my addr 172.21.15.3:0/2726455183 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:22.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.679+0000 7fdc5a29b700 1 -- 172.21.15.3:0/2726455183 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc5409e410 msgr2=0x7fdc54133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:22.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.679+0000 7fdc5a29b700 1 --2- 172.21.15.3:0/2726455183 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc5409e410 0x7fdc54133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.679+0000 7fdc5a29b700 1 -- 172.21.15.3:0/2726455183 >> v2:172.21.15.3:3300/0 conn(0x7fdc5409e000 msgr2=0x7fdc541339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:22.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.679+0000 7fdc5a29b700 1 --2- 172.21.15.3:0/2726455183 >> v2:172.21.15.3:3300/0 conn(0x7fdc5409e000 0x7fdc541339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.679+0000 7fdc5a29b700 1 -- 172.21.15.3:0/2726455183 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc5409a500 con 0x7fdc5409d6f0 2023-10-03T10:14:22.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.679+0000 7fdc59a9a700 1 --2- 172.21.15.3:0/2726455183 >> v2:172.21.15.3:3300/0 conn(0x7fdc5409e000 0x7fdc541339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:22.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.679+0000 7fdc5aa9c700 1 --2- 172.21.15.3:0/2726455183 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc5409e410 0x7fdc54133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:22.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.680+0000 7fdc5a29b700 1 --2- 172.21.15.3:0/2726455183 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc5409d6f0 0x7fdc5409dac0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009920 tx=0x7fdc50009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fda13293cd7e3114 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:22.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.680+0000 7fdc59299700 1 -- 172.21.15.3:0/2726455183 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc50011830 con 0x7fdc5409d6f0 2023-10-03T10:14:22.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.680+0000 7fdc59299700 1 -- 172.21.15.3:0/2726455183 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc50011990 con 0x7fdc5409d6f0 2023-10-03T10:14:22.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.680+0000 7fdc59299700 1 -- 172.21.15.3:0/2726455183 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc50011d20 con 0x7fdc5409d6f0 2023-10-03T10:14:22.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.680+0000 7fdc5c4ff700 1 -- 172.21.15.3:0/2726455183 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc5409d6f0 msgr2=0x7fdc5409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:22.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.680+0000 7fdc5c4ff700 1 --2- 172.21.15.3:0/2726455183 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc5409d6f0 0x7fdc5409dac0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009920 tx=0x7fdc50009c30 comp rx=0 tx=0).stop 2023-10-03T10:14:22.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.681+0000 7fdc5c4ff700 1 -- 172.21.15.3:0/2726455183 shutdown_connections 2023-10-03T10:14:22.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.681+0000 7fdc5c4ff700 1 --2- 172.21.15.3:0/2726455183 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc5409e410 0x7fdc54133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.681+0000 7fdc5c4ff700 1 --2- 172.21.15.3:0/2726455183 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc5409d6f0 0x7fdc5409dac0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.681+0000 7fdc5c4ff700 1 --2- 172.21.15.3:0/2726455183 >> v2:172.21.15.3:3300/0 conn(0x7fdc5409e000 0x7fdc541339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.681+0000 7fdc5c4ff700 1 -- 172.21.15.3:0/2726455183 >> 172.21.15.3:0/2726455183 conn(0x7fdc54066f50 msgr2=0x7fdc5409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:22.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.681+0000 7fdc5c4ff700 1 -- 172.21.15.3:0/2726455183 shutdown_connections 2023-10-03T10:14:22.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.681+0000 7fdc5c4ff700 1 -- 172.21.15.3:0/2726455183 wait complete. 2023-10-03T10:14:22.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.682+0000 7fdc5c4ff700 1 Processor -- start 2023-10-03T10:14:22.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.682+0000 7fdc5c4ff700 1 -- start start 2023-10-03T10:14:22.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.682+0000 7fdc5c4ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc5409d6f0 0x7fdc541a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:22.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.682+0000 7fdc5c4ff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc5409e000 0x7fdc541a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:22.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.682+0000 7fdc5c4ff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc5409e410 0x7fdc541a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:22.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.682+0000 7fdc5c4ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc541aa270 con 0x7fdc5409d6f0 2023-10-03T10:14:22.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.682+0000 7fdc5c4ff700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc541a79a0 con 0x7fdc5409e410 2023-10-03T10:14:22.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.682+0000 7fdc5c4ff700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc541a8c80 con 0x7fdc5409e000 2023-10-03T10:14:22.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.682+0000 7fdc5a29b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc5409d6f0 0x7fdc541a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:22.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.682+0000 7fdc5aa9c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc5409e410 0x7fdc541a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:22.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.682+0000 7fdc59a9a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc5409e000 0x7fdc541a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:22.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.682+0000 7fdc5a29b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc5409d6f0 0x7fdc541a5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41156/0 (socket says 172.21.15.3:41156) 2023-10-03T10:14:22.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.682+0000 7fdc5a29b700 1 -- 172.21.15.3:0/907438055 learned_addr learned my addr 172.21.15.3:0/907438055 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:22.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.683+0000 7fdc5a29b700 1 -- 172.21.15.3:0/907438055 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc5409e000 msgr2=0x7fdc541a6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:22.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.683+0000 7fdc5a29b700 1 --2- 172.21.15.3:0/907438055 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc5409e000 0x7fdc541a6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.683+0000 7fdc5a29b700 1 -- 172.21.15.3:0/907438055 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc5409e410 msgr2=0x7fdc541a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:22.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.683+0000 7fdc5a29b700 1 --2- 172.21.15.3:0/907438055 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc5409e410 0x7fdc541a68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.683+0000 7fdc5a29b700 1 -- 172.21.15.3:0/907438055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc440094a0 con 0x7fdc5409d6f0 2023-10-03T10:14:22.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.683+0000 7fdc5aa9c700 1 --2- 172.21.15.3:0/907438055 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc5409e410 0x7fdc541a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:22.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.683+0000 7fdc59a9a700 1 --2- 172.21.15.3:0/907438055 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc5409e000 0x7fdc541a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:22.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.683+0000 7fdc5a29b700 1 --2- 172.21.15.3:0/907438055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc5409d6f0 0x7fdc541a5e30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009c70 tx=0x7fdc50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:22.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.683+0000 7fdc4affd700 1 -- 172.21.15.3:0/907438055 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc50004240 con 0x7fdc5409d6f0 2023-10-03T10:14:22.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.683+0000 7fdc4affd700 1 -- 172.21.15.3:0/907438055 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5001f070 con 0x7fdc5409d6f0 2023-10-03T10:14:22.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.683+0000 7fdc5c4ff700 1 -- 172.21.15.3:0/907438055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc4c009a70 con 0x7fdc5409d6f0 2023-10-03T10:14:22.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.683+0000 7fdc4affd700 1 -- 172.21.15.3:0/907438055 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc500036a0 con 0x7fdc5409d6f0 2023-10-03T10:14:22.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.683+0000 7fdc5c4ff700 1 -- 172.21.15.3:0/907438055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc50009550 con 0x7fdc5409d6f0 2023-10-03T10:14:22.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.684+0000 7fdc4affd700 1 -- 172.21.15.3:0/907438055 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdc50021030 con 0x7fdc5409d6f0 2023-10-03T10:14:22.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.684+0000 7fdc4affd700 1 -- 172.21.15.3:0/907438055 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fdc50040820 con 0x7fdc5409d6f0 2023-10-03T10:14:22.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.684+0000 7fdc5c4ff700 1 -- 172.21.15.3:0/907438055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc5409c770 con 0x7fdc5409d6f0 2023-10-03T10:14:22.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.689+0000 7fdc4affd700 1 -- 172.21.15.3:0/907438055 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdc50015080 con 0x7fdc5409d6f0 2023-10-03T10:14:22.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.895+0000 7fdc5c4ff700 1 -- 172.21.15.3:0/907438055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdc5406c360 con 0x7fdc5409d6f0 2023-10-03T10:14:22.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.895+0000 7fdc4affd700 1 -- 172.21.15.3:0/907438055 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdc500199a0 con 0x7fdc5409d6f0 2023-10-03T10:14:22.896 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:22.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.898+0000 7fdc5c4ff700 1 -- 172.21.15.3:0/907438055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc5409d6f0 msgr2=0x7fdc541a5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:22.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.898+0000 7fdc5c4ff700 1 --2- 172.21.15.3:0/907438055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc5409d6f0 0x7fdc541a5e30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009c70 tx=0x7fdc50004750 comp rx=0 tx=0).stop 2023-10-03T10:14:22.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.898+0000 7fdc5c4ff700 1 -- 172.21.15.3:0/907438055 shutdown_connections 2023-10-03T10:14:22.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.898+0000 7fdc5c4ff700 1 --2- 172.21.15.3:0/907438055 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc5409e410 0x7fdc541a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.898+0000 7fdc5c4ff700 1 --2- 172.21.15.3:0/907438055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc5409d6f0 0x7fdc541a5e30 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.898+0000 7fdc5c4ff700 1 --2- 172.21.15.3:0/907438055 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc5409e000 0x7fdc541a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:22.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.898+0000 7fdc5c4ff700 1 -- 172.21.15.3:0/907438055 >> 172.21.15.3:0/907438055 conn(0x7fdc54066f50 msgr2=0x7fdc541badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:22.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.898+0000 7fdc5c4ff700 1 -- 172.21.15.3:0/907438055 shutdown_connections 2023-10-03T10:14:22.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:22.898+0000 7fdc5c4ff700 1 -- 172.21.15.3:0/907438055 wait complete. 2023-10-03T10:14:22.912 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:23.236 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:23.307 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:23.348 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:23.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.384+0000 7fbe2ea6c700 1 Processor -- start 2023-10-03T10:14:23.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.384+0000 7fbe2ea6c700 1 -- start start 2023-10-03T10:14:23.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.385+0000 7fbe2ea6c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe28130670 0x7fbe28130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:23.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.385+0000 7fbe2ea6c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe2806f680 0x7fbe2806fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:23.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.385+0000 7fbe2ea6c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbe2806ffd0 0x7fbe28137530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:23.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.385+0000 7fbe2ea6c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe2809a350 con 0x7fbe2806ffd0 2023-10-03T10:14:23.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.385+0000 7fbe2ea6c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe28130260 con 0x7fbe2806f680 2023-10-03T10:14:23.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.385+0000 7fbe2ea6c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbe28071ff0 con 0x7fbe28130670 2023-10-03T10:14:23.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.385+0000 7fbe27fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe2806f680 0x7fbe2806fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:23.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.385+0000 7fbe2c808700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe28130670 0x7fbe28130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:23.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.385+0000 7fbe2d009700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbe2806ffd0 0x7fbe28137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:23.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.385+0000 7fbe27fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe2806f680 0x7fbe2806fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44402/0 (socket says 172.21.15.3:44402) 2023-10-03T10:14:23.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.385+0000 7fbe2c808700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe28130670 0x7fbe28130a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42658/0 (socket says 172.21.15.3:42658) 2023-10-03T10:14:23.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.385+0000 7fbe27fff700 1 -- 172.21.15.3:0/3895937989 learned_addr learned my addr 172.21.15.3:0/3895937989 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:23.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.385+0000 7fbe2d009700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbe2806ffd0 0x7fbe28137530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32838/0 (socket says 172.21.15.3:32838) 2023-10-03T10:14:23.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.386+0000 7fbe2c808700 1 -- 172.21.15.3:0/3895937989 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe2806f680 msgr2=0x7fbe2806fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:23.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.386+0000 7fbe2c808700 1 --2- 172.21.15.3:0/3895937989 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe2806f680 0x7fbe2806fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:23.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.386+0000 7fbe2c808700 1 -- 172.21.15.3:0/3895937989 >> v2:172.21.15.3:3300/0 conn(0x7fbe2806ffd0 msgr2=0x7fbe28137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:23.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.386+0000 7fbe2c808700 1 --2- 172.21.15.3:0/3895937989 >> v2:172.21.15.3:3300/0 conn(0x7fbe2806ffd0 0x7fbe28137530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:23.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.386+0000 7fbe2c808700 1 -- 172.21.15.3:0/3895937989 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe28070470 con 0x7fbe28130670 2023-10-03T10:14:23.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.386+0000 7fbe27fff700 1 --2- 172.21.15.3:0/3895937989 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe2806f680 0x7fbe2806fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:23.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.386+0000 7fbe2d009700 1 --2- 172.21.15.3:0/3895937989 >> v2:172.21.15.3:3300/0 conn(0x7fbe2806ffd0 0x7fbe28137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:23.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.386+0000 7fbe2c808700 1 --2- 172.21.15.3:0/3895937989 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe28130670 0x7fbe28130a40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c009870 tx=0x7fbe1c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c2ee1dbe322a538 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:23.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.386+0000 7fbe277fe700 1 -- 172.21.15.3:0/3895937989 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c00f8a0 con 0x7fbe28130670 2023-10-03T10:14:23.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.386+0000 7fbe277fe700 1 -- 172.21.15.3:0/3895937989 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe1c00fa00 con 0x7fbe28130670 2023-10-03T10:14:23.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.386+0000 7fbe277fe700 1 -- 172.21.15.3:0/3895937989 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c00fd90 con 0x7fbe28130670 2023-10-03T10:14:23.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.387+0000 7fbe2ea6c700 1 -- 172.21.15.3:0/3895937989 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe28130670 msgr2=0x7fbe28130a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:23.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.387+0000 7fbe2ea6c700 1 --2- 172.21.15.3:0/3895937989 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe28130670 0x7fbe28130a40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c009870 tx=0x7fbe1c009b80 comp rx=0 tx=0).stop 2023-10-03T10:14:23.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.387+0000 7fbe2ea6c700 1 -- 172.21.15.3:0/3895937989 shutdown_connections 2023-10-03T10:14:23.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.387+0000 7fbe2ea6c700 1 --2- 172.21.15.3:0/3895937989 >> v2:172.21.15.3:3300/0 conn(0x7fbe2806ffd0 0x7fbe28137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:23.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.387+0000 7fbe2ea6c700 1 --2- 172.21.15.3:0/3895937989 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe28130670 0x7fbe28130a40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:23.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.387+0000 7fbe2ea6c700 1 --2- 172.21.15.3:0/3895937989 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe2806f680 0x7fbe2806fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:23.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.387+0000 7fbe2ea6c700 1 -- 172.21.15.3:0/3895937989 >> 172.21.15.3:0/3895937989 conn(0x7fbe280a2aa0 msgr2=0x7fbe280a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:23.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.387+0000 7fbe2ea6c700 1 -- 172.21.15.3:0/3895937989 shutdown_connections 2023-10-03T10:14:23.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.387+0000 7fbe2ea6c700 1 -- 172.21.15.3:0/3895937989 wait complete. 2023-10-03T10:14:23.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.389+0000 7fbe2ea6c700 1 Processor -- start 2023-10-03T10:14:23.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.389+0000 7fbe2ea6c700 1 -- start start 2023-10-03T10:14:23.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.390+0000 7fbe2ea6c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe2806f680 0x7fbe2811fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:23.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.390+0000 7fbe2ea6c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe2806ffd0 0x7fbe281201f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:23.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.390+0000 7fbe2ea6c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe28130670 0x7fbe28120730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:23.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.390+0000 7fbe2ea6c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe281218c0 con 0x7fbe2806f680 2023-10-03T10:14:23.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.390+0000 7fbe2ea6c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe28071890 con 0x7fbe28130670 2023-10-03T10:14:23.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.390+0000 7fbe2ea6c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbe28121ec0 con 0x7fbe2806ffd0 2023-10-03T10:14:23.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.390+0000 7fbe2c808700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe2806f680 0x7fbe2811fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:23.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.390+0000 7fbe27fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe2806ffd0 0x7fbe281201f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:23.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.390+0000 7fbe2d009700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe28130670 0x7fbe28120730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:23.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.390+0000 7fbe2c808700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe2806f680 0x7fbe2811fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32844/0 (socket says 172.21.15.3:32844) 2023-10-03T10:14:23.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.390+0000 7fbe2c808700 1 -- 172.21.15.3:0/2450230669 learned_addr learned my addr 172.21.15.3:0/2450230669 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:23.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.390+0000 7fbe27fff700 1 -- 172.21.15.3:0/2450230669 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe28130670 msgr2=0x7fbe28120730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:23.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.390+0000 7fbe27fff700 1 --2- 172.21.15.3:0/2450230669 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe28130670 0x7fbe28120730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:23.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.390+0000 7fbe27fff700 1 -- 172.21.15.3:0/2450230669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe2806f680 msgr2=0x7fbe2811fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:23.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.390+0000 7fbe27fff700 1 --2- 172.21.15.3:0/2450230669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe2806f680 0x7fbe2811fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:23.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.390+0000 7fbe27fff700 1 -- 172.21.15.3:0/2450230669 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe14009550 con 0x7fbe2806ffd0 2023-10-03T10:14:23.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.391+0000 7fbe2d009700 1 --2- 172.21.15.3:0/2450230669 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe28130670 0x7fbe28120730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:23.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.391+0000 7fbe2c808700 1 --2- 172.21.15.3:0/2450230669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe2806f680 0x7fbe2811fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:23.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.391+0000 7fbe27fff700 1 --2- 172.21.15.3:0/2450230669 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe2806ffd0 0x7fbe281201f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbe28066940 tx=0x7fbe1400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:23.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.391+0000 7fbe257fa700 1 -- 172.21.15.3:0/2450230669 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe14004030 con 0x7fbe2806ffd0 2023-10-03T10:14:23.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.391+0000 7fbe257fa700 1 -- 172.21.15.3:0/2450230669 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe14004190 con 0x7fbe2806ffd0 2023-10-03T10:14:23.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.391+0000 7fbe2ea6c700 1 -- 172.21.15.3:0/2450230669 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe20009a70 con 0x7fbe2806ffd0 2023-10-03T10:14:23.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.391+0000 7fbe257fa700 1 -- 172.21.15.3:0/2450230669 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe14004520 con 0x7fbe2806ffd0 2023-10-03T10:14:23.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.391+0000 7fbe2ea6c700 1 -- 172.21.15.3:0/2450230669 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe1c0094a0 con 0x7fbe2806ffd0 2023-10-03T10:14:23.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.392+0000 7fbe257fa700 1 -- 172.21.15.3:0/2450230669 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbe140036a0 con 0x7fbe2806ffd0 2023-10-03T10:14:23.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.392+0000 7fbe257fa700 1 -- 172.21.15.3:0/2450230669 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fbe1403fbe0 con 0x7fbe2806ffd0 2023-10-03T10:14:23.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.392+0000 7fbe2ea6c700 1 -- 172.21.15.3:0/2450230669 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe28074ea0 con 0x7fbe2806ffd0 2023-10-03T10:14:23.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.398+0000 7fbe257fa700 1 -- 172.21.15.3:0/2450230669 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbe1401bbe0 con 0x7fbe2806ffd0 2023-10-03T10:14:23.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.480+0000 7efe945cc700 1 Processor -- start 2023-10-03T10:14:23.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.480+0000 7efe945cc700 1 -- start start 2023-10-03T10:14:23.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.481+0000 7efe945cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efe8c09e230 0x7efe8c131c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:23.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.481+0000 7efe945cc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efe8c06ae80 0x7efe8c139590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:23.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.481+0000 7efe945cc700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe8c09dba0 con 0x7efe8c131830 2023-10-03T10:14:23.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.481+0000 7efe945cc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c132140 con 0x7efe8c09e230 2023-10-03T10:14:23.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.481+0000 7efe945cc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c1313e0 con 0x7efe8c06ae80 2023-10-03T10:14:23.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.481+0000 7efe91b67700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efe8c09e230 0x7efe8c131c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:23.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.481+0000 7efe92368700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7efe8c131830 0x7efe8c06a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50078/0 (socket says 172.21.15.3:50078) 2023-10-03T10:14:23.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.481+0000 7efe92368700 1 -- 172.21.15.3:0/137388442 learned_addr learned my addr 172.21.15.3:0/137388442 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:23.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.481+0000 7efe92b69700 1 --2- 172.21.15.3:0/137388442 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efe8c06ae80 0x7efe8c139590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:23.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.481+0000 7efe91366700 1 -- 172.21.15.3:0/137388442 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 82048128 0 0) 0x7efe8c09dba0 con 0x7efe8c131830 2023-10-03T10:14:23.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.481+0000 7efe91366700 1 -- 172.21.15.3:0/137388442 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe780035b0 con 0x7efe8c131830 2023-10-03T10:14:23.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.481+0000 7efe91b67700 1 -- 172.21.15.3:0/137388442 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efe8c06ae80 msgr2=0x7efe8c139590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:23.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.481+0000 7efe91b67700 1 --2- 172.21.15.3:0/137388442 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efe8c06ae80 0x7efe8c139590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:23.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.481+0000 7efe91b67700 1 -- 172.21.15.3:0/137388442 >> v1:172.21.15.3:6789/0 conn(0x7efe8c131830 legacy=0x7efe8c06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:23.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.481+0000 7efe91b67700 1 -- 172.21.15.3:0/137388442 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe8c09d6d0 con 0x7efe8c09e230 2023-10-03T10:14:23.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.481+0000 7efe92b69700 1 --2- 172.21.15.3:0/137388442 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efe8c06ae80 0x7efe8c139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:23.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.482+0000 7efe91b67700 1 --2- 172.21.15.3:0/137388442 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efe8c09e230 0x7efe8c131c00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7efe7c009870 tx=0x7efe7c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1681cd546d97f0aa server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:23.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.482+0000 7efe91366700 1 -- 172.21.15.3:0/137388442 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c011830 con 0x7efe8c09e230 2023-10-03T10:14:23.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.482+0000 7efe91366700 1 -- 172.21.15.3:0/137388442 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7c011990 con 0x7efe8c09e230 2023-10-03T10:14:23.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.482+0000 7efe91366700 1 -- 172.21.15.3:0/137388442 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c011d20 con 0x7efe8c09e230 2023-10-03T10:14:23.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.483+0000 7efe945cc700 1 -- 172.21.15.3:0/137388442 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efe8c09e230 msgr2=0x7efe8c131c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:23.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.483+0000 7efe945cc700 1 --2- 172.21.15.3:0/137388442 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efe8c09e230 0x7efe8c131c00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7efe7c009870 tx=0x7efe7c009b80 comp rx=0 tx=0).stop 2023-10-03T10:14:23.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.483+0000 7efe945cc700 1 -- 172.21.15.3:0/137388442 shutdown_connections 2023-10-03T10:14:23.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.483+0000 7efe945cc700 1 --2- 172.21.15.3:0/137388442 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efe8c06ae80 0x7efe8c139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:23.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.483+0000 7efe945cc700 1 --2- 172.21.15.3:0/137388442 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efe8c09e230 0x7efe8c131c00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:23.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.483+0000 7efe945cc700 1 -- 172.21.15.3:0/137388442 >> 172.21.15.3:0/137388442 conn(0x7efe8c099c20 msgr2=0x7efe8c0a2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:23.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.483+0000 7efe945cc700 1 -- 172.21.15.3:0/137388442 shutdown_connections 2023-10-03T10:14:23.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.483+0000 7efe945cc700 1 -- 172.21.15.3:0/137388442 wait complete. 2023-10-03T10:14:23.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.484+0000 7efe945cc700 1 Processor -- start 2023-10-03T10:14:23.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.484+0000 7efe945cc700 1 -- start start 2023-10-03T10:14:23.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.484+0000 7efe945cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efe8c06ae80 0x7efe8c06e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:23.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.484+0000 7efe945cc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efe8c09e230 0x7efe8c06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:23.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.484+0000 7efe945cc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe8c131830 0x7efe8c06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:23.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.484+0000 7efe945cc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c0a0140 con 0x7efe8c131830 2023-10-03T10:14:23.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.484+0000 7efe945cc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c12b8a0 con 0x7efe8c06ae80 2023-10-03T10:14:23.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.484+0000 7efe945cc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c12b0f0 con 0x7efe8c09e230 2023-10-03T10:14:23.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.484+0000 7efe92368700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efe8c06ae80 0x7efe8c06e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:23.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.484+0000 7efe92368700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efe8c06ae80 0x7efe8c06e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44444/0 (socket says 172.21.15.3:44444) 2023-10-03T10:14:23.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.485+0000 7efe92368700 1 -- 172.21.15.3:0/2588269762 learned_addr learned my addr 172.21.15.3:0/2588269762 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:23.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.485+0000 7efe92b69700 1 --2- 172.21.15.3:0/2588269762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe8c131830 0x7efe8c06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:23.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.485+0000 7efe91b67700 1 --2- 172.21.15.3:0/2588269762 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efe8c09e230 0x7efe8c06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:23.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.485+0000 7efe92368700 1 -- 172.21.15.3:0/2588269762 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efe8c09e230 msgr2=0x7efe8c06eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:23.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.485+0000 7efe92368700 1 --2- 172.21.15.3:0/2588269762 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efe8c09e230 0x7efe8c06eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:23.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.485+0000 7efe92368700 1 -- 172.21.15.3:0/2588269762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe8c131830 msgr2=0x7efe8c06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:23.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.485+0000 7efe92368700 1 --2- 172.21.15.3:0/2588269762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe8c131830 0x7efe8c06f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:23.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.485+0000 7efe92368700 1 -- 172.21.15.3:0/2588269762 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe84009a70 con 0x7efe8c06ae80 2023-10-03T10:14:23.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.485+0000 7efe92368700 1 --2- 172.21.15.3:0/2588269762 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efe8c06ae80 0x7efe8c06e970 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efe8800c730 tx=0x7efe8800caf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:23.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.485+0000 7efe82ffd700 1 -- 172.21.15.3:0/2588269762 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe88005d70 con 0x7efe8c06ae80 2023-10-03T10:14:23.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.485+0000 7efe82ffd700 1 -- 172.21.15.3:0/2588269762 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe880070d0 con 0x7efe8c06ae80 2023-10-03T10:14:23.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.485+0000 7efe945cc700 1 -- 172.21.15.3:0/2588269762 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe7c0094a0 con 0x7efe8c06ae80 2023-10-03T10:14:23.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.485+0000 7efe82ffd700 1 -- 172.21.15.3:0/2588269762 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe88007440 con 0x7efe8c06ae80 2023-10-03T10:14:23.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.485+0000 7efe945cc700 1 -- 172.21.15.3:0/2588269762 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe8c12a870 con 0x7efe8c06ae80 2023-10-03T10:14:23.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.487+0000 7efe945cc700 1 -- 172.21.15.3:0/2588269762 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe8c070e20 con 0x7efe8c06ae80 2023-10-03T10:14:23.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.487+0000 7efe82ffd700 1 -- 172.21.15.3:0/2588269762 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efe880075a0 con 0x7efe8c06ae80 2023-10-03T10:14:23.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.487+0000 7efe82ffd700 1 -- 172.21.15.3:0/2588269762 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7efe88040c40 con 0x7efe8c06ae80 2023-10-03T10:14:23.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.492+0000 7efe82ffd700 1 -- 172.21.15.3:0/2588269762 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efe88044060 con 0x7efe8c06ae80 2023-10-03T10:14:23.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.494+0000 7f34714ad700 1 Processor -- start 2023-10-03T10:14:23.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.494+0000 7f34714ad700 1 -- start start 2023-10-03T10:14:23.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.495+0000 7f34714ad700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f346c136830 0x7f346c138c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:23.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.495+0000 7f34714ad700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f346c139150 0x7f346c13b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:23.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.495+0000 7f34714ad700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f346c06f720 con 0x7f346c1338c0 2023-10-03T10:14:23.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.495+0000 7f34714ad700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346c070c80 con 0x7f346c139150 2023-10-03T10:14:23.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.495+0000 7f34714ad700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f346c06fce0 con 0x7f346c136830 2023-10-03T10:14:23.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.495+0000 7f346affd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f346c1338c0 0x7f346c133c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50086/0 (socket says 172.21.15.3:50086) 2023-10-03T10:14:23.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.495+0000 7f346affd700 1 -- 172.21.15.3:0/2651772030 learned_addr learned my addr 172.21.15.3:0/2651772030 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:23.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.495+0000 7f346a7fc700 1 --2- 172.21.15.3:0/2651772030 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f346c136830 0x7f346c138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:23.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.495+0000 7f346b7fe700 1 --2- 172.21.15.3:0/2651772030 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f346c139150 0x7f346c13b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:23.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.496+0000 7f3469ffb700 1 -- 172.21.15.3:0/2651772030 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2400657458 0 0) 0x7f346c06f720 con 0x7f346c1338c0 2023-10-03T10:14:23.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.496+0000 7f3469ffb700 1 -- 172.21.15.3:0/2651772030 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34540035b0 con 0x7f346c1338c0 2023-10-03T10:14:23.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.496+0000 7f346a7fc700 1 -- 172.21.15.3:0/2651772030 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f346c139150 msgr2=0x7f346c13b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:23.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.496+0000 7f346a7fc700 1 --2- 172.21.15.3:0/2651772030 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f346c139150 0x7f346c13b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:23.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.496+0000 7f346a7fc700 1 -- 172.21.15.3:0/2651772030 >> v1:172.21.15.3:6789/0 conn(0x7f346c1338c0 legacy=0x7f346c133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:23.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.496+0000 7f346a7fc700 1 -- 172.21.15.3:0/2651772030 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f346c1336a0 con 0x7f346c136830 2023-10-03T10:14:23.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.496+0000 7f346a7fc700 1 --2- 172.21.15.3:0/2651772030 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f346c136830 0x7f346c138c10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3460009870 tx=0x7f3460009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21c166beb97bd47d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:23.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.496+0000 7f3469ffb700 1 -- 172.21.15.3:0/2651772030 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3460011830 con 0x7f346c136830 2023-10-03T10:14:23.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.496+0000 7f3469ffb700 1 -- 172.21.15.3:0/2651772030 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3460011990 con 0x7f346c136830 2023-10-03T10:14:23.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.496+0000 7f3469ffb700 1 -- 172.21.15.3:0/2651772030 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3460011d20 con 0x7f346c136830 2023-10-03T10:14:23.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.497+0000 7f34714ad700 1 -- 172.21.15.3:0/2651772030 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f346c136830 msgr2=0x7f346c138c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:23.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.497+0000 7f34714ad700 1 --2- 172.21.15.3:0/2651772030 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f346c136830 0x7f346c138c10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3460009870 tx=0x7f3460009b80 comp rx=0 tx=0).stop 2023-10-03T10:14:23.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.497+0000 7f34714ad700 1 -- 172.21.15.3:0/2651772030 shutdown_connections 2023-10-03T10:14:23.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.497+0000 7f34714ad700 1 --2- 172.21.15.3:0/2651772030 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f346c139150 0x7f346c13b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:23.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.497+0000 7f34714ad700 1 --2- 172.21.15.3:0/2651772030 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f346c136830 0x7f346c138c10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:23.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.497+0000 7f34714ad700 1 -- 172.21.15.3:0/2651772030 >> 172.21.15.3:0/2651772030 conn(0x7f346c0a2aa0 msgr2=0x7f346c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:23.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.498+0000 7f34714ad700 1 -- 172.21.15.3:0/2651772030 shutdown_connections 2023-10-03T10:14:23.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.498+0000 7f34714ad700 1 -- 172.21.15.3:0/2651772030 wait complete. 2023-10-03T10:14:23.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.499+0000 7f34714ad700 1 Processor -- start 2023-10-03T10:14:23.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.499+0000 7f34714ad700 1 -- start start 2023-10-03T10:14:23.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.500+0000 7f34714ad700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f346c1338c0 0x7f346c1c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:23.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.500+0000 7f34714ad700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f346c136830 0x7f346c1c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:23.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.500+0000 7f34714ad700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f346c139150 0x7f346c119c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:23.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.500+0000 7f34714ad700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346c11b840 con 0x7f346c1338c0 2023-10-03T10:14:23.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.500+0000 7f34714ad700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346c06f960 con 0x7f346c136830 2023-10-03T10:14:23.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.500+0000 7f34714ad700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f346c11d4e0 con 0x7f346c139150 2023-10-03T10:14:23.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.500+0000 7f346a7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f346c136830 0x7f346c1c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:23.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.500+0000 7f346affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f346c1338c0 0x7f346c1c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:23.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.500+0000 7f346a7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f346c136830 0x7f346c1c8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44476/0 (socket says 172.21.15.3:44476) 2023-10-03T10:14:23.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.500+0000 7f346a7fc700 1 -- 172.21.15.3:0/2460337919 learned_addr learned my addr 172.21.15.3:0/2460337919 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:23.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.500+0000 7f346affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f346c1338c0 0x7f346c1c8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32860/0 (socket says 172.21.15.3:32860) 2023-10-03T10:14:23.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.500+0000 7f346b7fe700 1 --2- 172.21.15.3:0/2460337919 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f346c139150 0x7f346c119c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:23.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.501+0000 7f346a7fc700 1 -- 172.21.15.3:0/2460337919 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f346c139150 msgr2=0x7f346c119c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:23.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.501+0000 7f346a7fc700 1 --2- 172.21.15.3:0/2460337919 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f346c139150 0x7f346c119c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:23.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.501+0000 7f346a7fc700 1 -- 172.21.15.3:0/2460337919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f346c1338c0 msgr2=0x7f346c1c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:23.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.501+0000 7f346a7fc700 1 --2- 172.21.15.3:0/2460337919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f346c1338c0 0x7f346c1c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:23.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.501+0000 7f346a7fc700 1 -- 172.21.15.3:0/2460337919 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34600094a0 con 0x7f346c136830 2023-10-03T10:14:23.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.501+0000 7f346a7fc700 1 --2- 172.21.15.3:0/2460337919 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f346c136830 0x7f346c1c8a80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f34600090b0 tx=0x7f3460004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:23.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.501+0000 7f34537fe700 1 -- 172.21.15.3:0/2460337919 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3460004450 con 0x7f346c136830 2023-10-03T10:14:23.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.501+0000 7f34537fe700 1 -- 172.21.15.3:0/2460337919 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f346001f070 con 0x7f346c136830 2023-10-03T10:14:23.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.501+0000 7f34537fe700 1 -- 172.21.15.3:0/2460337919 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34600038e0 con 0x7f346c136830 2023-10-03T10:14:23.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.501+0000 7f34714ad700 1 -- 172.21.15.3:0/2460337919 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f346c11d750 con 0x7f346c136830 2023-10-03T10:14:23.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.501+0000 7f34714ad700 1 -- 172.21.15.3:0/2460337919 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f346c11d1b0 con 0x7f346c136830 2023-10-03T10:14:23.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.502+0000 7f34714ad700 1 -- 172.21.15.3:0/2460337919 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f346c074ea0 con 0x7f346c136830 2023-10-03T10:14:23.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.503+0000 7f34537fe700 1 -- 172.21.15.3:0/2460337919 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3460003a40 con 0x7f346c136830 2023-10-03T10:14:23.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.503+0000 7f34537fe700 1 -- 172.21.15.3:0/2460337919 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3460021070 con 0x7f346c136830 2023-10-03T10:14:23.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.508+0000 7f34537fe700 1 -- 172.21.15.3:0/2460337919 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f346001a430 con 0x7f346c136830 2023-10-03T10:14:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.642+0000 7fbe2ea6c700 1 -- 172.21.15.3:0/2450230669 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbe28072210 con 0x7fbe2806ffd0 2023-10-03T10:14:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.643+0000 7fbe257fa700 1 -- 172.21.15.3:0/2450230669 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbe1401b3b0 con 0x7fbe2806ffd0 2023-10-03T10:14:23.643 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:23.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.645+0000 7fbe2ea6c700 1 -- 172.21.15.3:0/2450230669 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe2806ffd0 msgr2=0x7fbe281201f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:23.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.645+0000 7fbe2ea6c700 1 --2- 172.21.15.3:0/2450230669 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe2806ffd0 0x7fbe281201f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbe28066940 tx=0x7fbe1400ec80 comp rx=0 tx=0).stop 2023-10-03T10:14:23.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.645+0000 7fbe2ea6c700 1 -- 172.21.15.3:0/2450230669 shutdown_connections 2023-10-03T10:14:23.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.645+0000 7fbe2ea6c700 1 --2- 172.21.15.3:0/2450230669 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe28130670 0x7fbe28120730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:23.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.645+0000 7fbe2ea6c700 1 --2- 172.21.15.3:0/2450230669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe2806f680 0x7fbe2811fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:23.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.645+0000 7fbe2ea6c700 1 --2- 172.21.15.3:0/2450230669 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe2806ffd0 0x7fbe281201f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:23.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.645+0000 7fbe2ea6c700 1 -- 172.21.15.3:0/2450230669 >> 172.21.15.3:0/2450230669 conn(0x7fbe280a2aa0 msgr2=0x7fbe2806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:23.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.645+0000 7fbe2ea6c700 1 -- 172.21.15.3:0/2450230669 shutdown_connections 2023-10-03T10:14:23.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.645+0000 7fbe2ea6c700 1 -- 172.21.15.3:0/2450230669 wait complete. 2023-10-03T10:14:23.659 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:23.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.719+0000 7efe945cc700 1 -- 172.21.15.3:0/2588269762 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7efe8c070b60 con 0x7efe8c06ae80 2023-10-03T10:14:23.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.720+0000 7efe82ffd700 1 -- 172.21.15.3:0/2588269762 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efe88044070 con 0x7efe8c06ae80 2023-10-03T10:14:23.720 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:23.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.722+0000 7efe945cc700 1 -- 172.21.15.3:0/2588269762 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efe8c06ae80 msgr2=0x7efe8c06e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:23.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.722+0000 7efe945cc700 1 --2- 172.21.15.3:0/2588269762 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efe8c06ae80 0x7efe8c06e970 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efe8800c730 tx=0x7efe8800caf0 comp rx=0 tx=0).stop 2023-10-03T10:14:23.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.722+0000 7efe945cc700 1 -- 172.21.15.3:0/2588269762 shutdown_connections 2023-10-03T10:14:23.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.722+0000 7efe945cc700 1 --2- 172.21.15.3:0/2588269762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe8c131830 0x7efe8c06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:23.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.722+0000 7efe945cc700 1 --2- 172.21.15.3:0/2588269762 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efe8c06ae80 0x7efe8c06e970 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:23.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.722+0000 7efe945cc700 1 --2- 172.21.15.3:0/2588269762 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efe8c09e230 0x7efe8c06eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:23.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.722+0000 7efe945cc700 1 -- 172.21.15.3:0/2588269762 >> 172.21.15.3:0/2588269762 conn(0x7efe8c099c20 msgr2=0x7efe8c1304d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:23.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.722+0000 7efe945cc700 1 -- 172.21.15.3:0/2588269762 shutdown_connections 2023-10-03T10:14:23.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.722+0000 7efe945cc700 1 -- 172.21.15.3:0/2588269762 wait complete. 2023-10-03T10:14:23.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.732+0000 7f34714ad700 1 -- 172.21.15.3:0/2460337919 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f346c06b9e0 con 0x7f346c136830 2023-10-03T10:14:23.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.732+0000 7f34537fe700 1 -- 172.21.15.3:0/2460337919 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3460015020 con 0x7f346c136830 2023-10-03T10:14:23.732 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:23.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.734+0000 7f34714ad700 1 -- 172.21.15.3:0/2460337919 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f346c136830 msgr2=0x7f346c1c8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:23.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.734+0000 7f34714ad700 1 --2- 172.21.15.3:0/2460337919 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f346c136830 0x7f346c1c8a80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f34600090b0 tx=0x7f3460004750 comp rx=0 tx=0).stop 2023-10-03T10:14:23.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.734+0000 7f34714ad700 1 -- 172.21.15.3:0/2460337919 shutdown_connections 2023-10-03T10:14:23.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.734+0000 7f34714ad700 1 --2- 172.21.15.3:0/2460337919 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f346c139150 0x7f346c119c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:23.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.734+0000 7f34714ad700 1 --2- 172.21.15.3:0/2460337919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f346c1338c0 0x7f346c1c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:23.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.734+0000 7f34714ad700 1 --2- 172.21.15.3:0/2460337919 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f346c136830 0x7f346c1c8a80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:23.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.734+0000 7f34714ad700 1 -- 172.21.15.3:0/2460337919 >> 172.21.15.3:0/2460337919 conn(0x7f346c0a2aa0 msgr2=0x7f346c134e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:23.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.734+0000 7f34714ad700 1 -- 172.21.15.3:0/2460337919 shutdown_connections 2023-10-03T10:14:23.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:23.734+0000 7f34714ad700 1 -- 172.21.15.3:0/2460337919 wait complete. 2023-10-03T10:14:23.736 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:23.748 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:23.913 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:24.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.056+0000 7f7159e3a700 1 Processor -- start 2023-10-03T10:14:24.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.056+0000 7f7159e3a700 1 -- start start 2023-10-03T10:14:24.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.057+0000 7f7159e3a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f715409e850 0x7f715409ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:24.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.057+0000 7f7159e3a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f715409f160 0x7f7154131790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:24.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.057+0000 7f7159e3a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f715409b560 con 0x7f7154131cd0 2023-10-03T10:14:24.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.057+0000 7f7159e3a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71540a0040 con 0x7f715409f160 2023-10-03T10:14:24.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.057+0000 7f7159e3a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71540a02e0 con 0x7f715409e850 2023-10-03T10:14:24.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.057+0000 7f71537fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f715409e850 0x7f715409ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:24.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.057+0000 7f7152ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f715409f160 0x7f7154131790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:24.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.057+0000 7f71537fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f715409e850 0x7f715409ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44482/0 (socket says 172.21.15.3:44482) 2023-10-03T10:14:24.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.057+0000 7f71537fe700 1 -- 172.21.15.3:0/1267095681 learned_addr learned my addr 172.21.15.3:0/1267095681 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:24.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.057+0000 7f7152ffd700 1 -- 172.21.15.3:0/1267095681 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f715409e850 msgr2=0x7f715409ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:24.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.057+0000 7f7152ffd700 1 --2- 172.21.15.3:0/1267095681 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f715409e850 0x7f715409ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:24.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.057+0000 7f7152ffd700 1 -- 172.21.15.3:0/1267095681 >> v1:172.21.15.3:6789/0 conn(0x7f7154131cd0 legacy=0x7f7154138bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:24.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.057+0000 7f7152ffd700 1 -- 172.21.15.3:0/1267095681 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715409fab0 con 0x7f715409f160 2023-10-03T10:14:24.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.057+0000 7f71537fe700 1 --2- 172.21.15.3:0/1267095681 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f715409e850 0x7f715409ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:24.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.058+0000 7f7152ffd700 1 --2- 172.21.15.3:0/1267095681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f715409f160 0x7f7154131790 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7148009840 tx=0x7f7148009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d246bd3c7b4c112b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:24.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.058+0000 7f71527fc700 1 -- 172.21.15.3:0/1267095681 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7148011830 con 0x7f715409f160 2023-10-03T10:14:24.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.058+0000 7f71527fc700 1 -- 172.21.15.3:0/1267095681 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7148011990 con 0x7f715409f160 2023-10-03T10:14:24.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.058+0000 7f71527fc700 1 -- 172.21.15.3:0/1267095681 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7148011d20 con 0x7f715409f160 2023-10-03T10:14:24.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.058+0000 7f7159e3a700 1 -- 172.21.15.3:0/1267095681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f715409f160 msgr2=0x7f7154131790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:24.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.058+0000 7f7159e3a700 1 --2- 172.21.15.3:0/1267095681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f715409f160 0x7f7154131790 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7148009840 tx=0x7f7148009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:24.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.058+0000 7f7159e3a700 1 -- 172.21.15.3:0/1267095681 shutdown_connections 2023-10-03T10:14:24.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.058+0000 7f7159e3a700 1 --2- 172.21.15.3:0/1267095681 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f715409e850 0x7f715409ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:24.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.058+0000 7f7159e3a700 1 --2- 172.21.15.3:0/1267095681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f715409f160 0x7f7154131790 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:24.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.058+0000 7f7159e3a700 1 -- 172.21.15.3:0/1267095681 >> 172.21.15.3:0/1267095681 conn(0x7f71540a3f70 msgr2=0x7f71540a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:24.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.059+0000 7f7159e3a700 1 -- 172.21.15.3:0/1267095681 shutdown_connections 2023-10-03T10:14:24.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.059+0000 7f7159e3a700 1 -- 172.21.15.3:0/1267095681 wait complete. 2023-10-03T10:14:24.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.060+0000 7f7159e3a700 1 Processor -- start 2023-10-03T10:14:24.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.060+0000 7f7159e3a700 1 -- start start 2023-10-03T10:14:24.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.060+0000 7f7159e3a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f715409e850 0x7f7154120250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:24.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.060+0000 7f7159e3a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f715409f160 0x7f7154120790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:24.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.060+0000 7f7159e3a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7154131cd0 0x7f7154120cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:24.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.060+0000 7f7159e3a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71541237d0 con 0x7f715409f160 2023-10-03T10:14:24.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.060+0000 7f7159e3a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71541226a0 con 0x7f7154131cd0 2023-10-03T10:14:24.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.060+0000 7f7159e3a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f715409d420 con 0x7f715409e850 2023-10-03T10:14:24.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.060+0000 7f7152ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f715409f160 0x7f7154120790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:24.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.060+0000 7f7153fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7154131cd0 0x7f7154120cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:24.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.060+0000 7f71537fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f715409e850 0x7f7154120250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:24.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.060+0000 7f7152ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f715409f160 0x7f7154120790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32866/0 (socket says 172.21.15.3:32866) 2023-10-03T10:14:24.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.060+0000 7f7152ffd700 1 -- 172.21.15.3:0/377074212 learned_addr learned my addr 172.21.15.3:0/377074212 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:24.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.061+0000 7f7153fff700 1 -- 172.21.15.3:0/377074212 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f715409e850 msgr2=0x7f7154120250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:24.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.061+0000 7f7153fff700 1 --2- 172.21.15.3:0/377074212 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f715409e850 0x7f7154120250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:24.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.061+0000 7f7153fff700 1 -- 172.21.15.3:0/377074212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f715409f160 msgr2=0x7f7154120790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:24.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.061+0000 7f7153fff700 1 --2- 172.21.15.3:0/377074212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f715409f160 0x7f7154120790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:24.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.061+0000 7f7153fff700 1 -- 172.21.15.3:0/377074212 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7144009550 con 0x7f7154131cd0 2023-10-03T10:14:24.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.061+0000 7f71537fe700 1 --2- 172.21.15.3:0/377074212 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f715409e850 0x7f7154120250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:24.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.061+0000 7f7152ffd700 1 --2- 172.21.15.3:0/377074212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f715409f160 0x7f7154120790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:24.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.061+0000 7f7153fff700 1 --2- 172.21.15.3:0/377074212 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7154131cd0 0x7f7154120cd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f714c00cb90 tx=0x7f714c00cea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:24.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.061+0000 7f713bfff700 1 -- 172.21.15.3:0/377074212 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c003c00 con 0x7f7154131cd0 2023-10-03T10:14:24.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.061+0000 7f713bfff700 1 -- 172.21.15.3:0/377074212 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714c003d60 con 0x7f7154131cd0 2023-10-03T10:14:24.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.061+0000 7f7159e3a700 1 -- 172.21.15.3:0/377074212 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71480094a0 con 0x7f7154131cd0 2023-10-03T10:14:24.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.061+0000 7f713bfff700 1 -- 172.21.15.3:0/377074212 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c00aa80 con 0x7f7154131cd0 2023-10-03T10:14:24.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.061+0000 7f7159e3a700 1 -- 172.21.15.3:0/377074212 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7154123a90 con 0x7f7154131cd0 2023-10-03T10:14:24.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.062+0000 7f713bfff700 1 -- 172.21.15.3:0/377074212 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f714c00abe0 con 0x7f7154131cd0 2023-10-03T10:14:24.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.062+0000 7f713bfff700 1 -- 172.21.15.3:0/377074212 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f714c03f5e0 con 0x7f7154131cd0 2023-10-03T10:14:24.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.062+0000 7f7159e3a700 1 -- 172.21.15.3:0/377074212 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71540747f0 con 0x7f7154131cd0 2023-10-03T10:14:24.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.067+0000 7f713bfff700 1 -- 172.21.15.3:0/377074212 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f714c0198f0 con 0x7f7154131cd0 2023-10-03T10:14:24.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.275+0000 7f7159e3a700 1 -- 172.21.15.3:0/377074212 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f715409b780 con 0x7f7154131cd0 2023-10-03T10:14:24.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.275+0000 7f713bfff700 1 -- 172.21.15.3:0/377074212 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f714c019300 con 0x7f7154131cd0 2023-10-03T10:14:24.276 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:24.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.278+0000 7f7159e3a700 1 -- 172.21.15.3:0/377074212 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7154131cd0 msgr2=0x7f7154120cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:24.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.278+0000 7f7159e3a700 1 --2- 172.21.15.3:0/377074212 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7154131cd0 0x7f7154120cd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f714c00cb90 tx=0x7f714c00cea0 comp rx=0 tx=0).stop 2023-10-03T10:14:24.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.278+0000 7f7159e3a700 1 -- 172.21.15.3:0/377074212 shutdown_connections 2023-10-03T10:14:24.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.278+0000 7f7159e3a700 1 --2- 172.21.15.3:0/377074212 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7154131cd0 0x7f7154120cd0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:24.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.278+0000 7f7159e3a700 1 --2- 172.21.15.3:0/377074212 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f715409e850 0x7f7154120250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:24.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.278+0000 7f7159e3a700 1 --2- 172.21.15.3:0/377074212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f715409f160 0x7f7154120790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:24.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.278+0000 7f7159e3a700 1 -- 172.21.15.3:0/377074212 >> 172.21.15.3:0/377074212 conn(0x7f71540a3f70 msgr2=0x7f715411a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:24.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.278+0000 7f7159e3a700 1 -- 172.21.15.3:0/377074212 shutdown_connections 2023-10-03T10:14:24.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.278+0000 7f7159e3a700 1 -- 172.21.15.3:0/377074212 wait complete. 2023-10-03T10:14:24.292 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:24.660 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:24.737 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:24.749 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:24.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.808+0000 7f1b4f61f700 1 Processor -- start 2023-10-03T10:14:24.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.809+0000 7f1b4f61f700 1 -- start start 2023-10-03T10:14:24.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.809+0000 7f1b4f61f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1b4809d6f0 0x7f1b4809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:24.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.809+0000 7f1b4f61f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b4809e000 0x7f1b48130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:24.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.809+0000 7f1b4f61f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b4809e410 0x7f1b48130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:24.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.809+0000 7f1b4f61f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b4809f520 con 0x7f1b4809d6f0 2023-10-03T10:14:24.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.809+0000 7f1b4f61f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4809f130 con 0x7f1b4809e410 2023-10-03T10:14:24.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.809+0000 7f1b4f61f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4809e930 con 0x7f1b4809e000 2023-10-03T10:14:24.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.810+0000 7f1b4dbbc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b4809e410 0x7f1b48130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:24.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.810+0000 7f1b4d3bb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1b4809d6f0 0x7f1b4809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:24.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.810+0000 7f1b4cbba700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b4809e000 0x7f1b48130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:24.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.810+0000 7f1b4dbbc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b4809e410 0x7f1b48130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44496/0 (socket says 172.21.15.3:44496) 2023-10-03T10:14:24.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.810+0000 7f1b4d3bb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1b4809d6f0 0x7f1b4809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32878/0 (socket says 172.21.15.3:32878) 2023-10-03T10:14:24.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.810+0000 7f1b4dbbc700 1 -- 172.21.15.3:0/1554516422 learned_addr learned my addr 172.21.15.3:0/1554516422 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:24.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.810+0000 7f1b4dbbc700 1 -- 172.21.15.3:0/1554516422 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b4809e000 msgr2=0x7f1b48130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:24.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.810+0000 7f1b4dbbc700 1 --2- 172.21.15.3:0/1554516422 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b4809e000 0x7f1b48130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:24.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.810+0000 7f1b4dbbc700 1 -- 172.21.15.3:0/1554516422 >> v2:172.21.15.3:3300/0 conn(0x7f1b4809d6f0 msgr2=0x7f1b4809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:24.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.810+0000 7f1b4dbbc700 1 --2- 172.21.15.3:0/1554516422 >> v2:172.21.15.3:3300/0 conn(0x7f1b4809d6f0 0x7f1b4809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:24.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.810+0000 7f1b4dbbc700 1 -- 172.21.15.3:0/1554516422 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4809b2d0 con 0x7f1b4809e410 2023-10-03T10:14:24.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.810+0000 7f1b4dbbc700 1 --2- 172.21.15.3:0/1554516422 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b4809e410 0x7f1b48130ba0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000bef0 tx=0x7f1b400093b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a867fbff6405504a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:24.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.810+0000 7f1b3ffff700 1 -- 172.21.15.3:0/1554516422 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4000b4e0 con 0x7f1b4809e410 2023-10-03T10:14:24.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.810+0000 7f1b3ffff700 1 -- 172.21.15.3:0/1554516422 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4000c950 con 0x7f1b4809e410 2023-10-03T10:14:24.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.810+0000 7f1b3ffff700 1 -- 172.21.15.3:0/1554516422 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4000cd00 con 0x7f1b4809e410 2023-10-03T10:14:24.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.811+0000 7f1b4f61f700 1 -- 172.21.15.3:0/1554516422 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b4809e410 msgr2=0x7f1b48130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:24.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.811+0000 7f1b4f61f700 1 --2- 172.21.15.3:0/1554516422 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b4809e410 0x7f1b48130ba0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000bef0 tx=0x7f1b400093b0 comp rx=0 tx=0).stop 2023-10-03T10:14:24.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.811+0000 7f1b4f61f700 1 -- 172.21.15.3:0/1554516422 shutdown_connections 2023-10-03T10:14:24.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.811+0000 7f1b4f61f700 1 --2- 172.21.15.3:0/1554516422 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b4809e410 0x7f1b48130ba0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:24.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.811+0000 7f1b4f61f700 1 --2- 172.21.15.3:0/1554516422 >> v2:172.21.15.3:3300/0 conn(0x7f1b4809d6f0 0x7f1b4809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:24.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.811+0000 7f1b4f61f700 1 --2- 172.21.15.3:0/1554516422 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b4809e000 0x7f1b48130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:24.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.811+0000 7f1b4f61f700 1 -- 172.21.15.3:0/1554516422 >> 172.21.15.3:0/1554516422 conn(0x7f1b48066f50 msgr2=0x7f1b4809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:24.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.811+0000 7f1b4f61f700 1 -- 172.21.15.3:0/1554516422 shutdown_connections 2023-10-03T10:14:24.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.811+0000 7f1b4f61f700 1 -- 172.21.15.3:0/1554516422 wait complete. 2023-10-03T10:14:24.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.812+0000 7f1b4f61f700 1 Processor -- start 2023-10-03T10:14:24.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.812+0000 7f1b4f61f700 1 -- start start 2023-10-03T10:14:24.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.813+0000 7f1b4f61f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b4809d6f0 0x7f1b4811e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:24.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.813+0000 7f1b4f61f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b4809e000 0x7f1b4811e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:24.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.813+0000 7f1b4f61f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b4809e410 0x7f1b4811ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:24.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.813+0000 7f1b4f61f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4811f4d0 con 0x7f1b4809e410 2023-10-03T10:14:24.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.813+0000 7f1b4f61f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b48121ad0 con 0x7f1b4809e000 2023-10-03T10:14:24.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.813+0000 7f1b4f61f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1b48120ea0 con 0x7f1b4809d6f0 2023-10-03T10:14:24.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.813+0000 7f1b4d3bb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b4809d6f0 0x7f1b4811e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:24.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.813+0000 7f1b4d3bb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b4809d6f0 0x7f1b4811e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42724/0 (socket says 172.21.15.3:42724) 2023-10-03T10:14:24.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.813+0000 7f1b4d3bb700 1 -- 172.21.15.3:0/3368758475 learned_addr learned my addr 172.21.15.3:0/3368758475 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:24.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.813+0000 7f1b4dbbc700 1 --2- 172.21.15.3:0/3368758475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b4809e410 0x7f1b4811ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:24.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.813+0000 7f1b4cbba700 1 --2- 172.21.15.3:0/3368758475 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b4809e000 0x7f1b4811e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:24.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.813+0000 7f1b4dbbc700 1 -- 172.21.15.3:0/3368758475 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b4809d6f0 msgr2=0x7f1b4811e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:24.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.814+0000 7f1b4dbbc700 1 --2- 172.21.15.3:0/3368758475 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b4809d6f0 0x7f1b4811e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:24.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.814+0000 7f1b4dbbc700 1 -- 172.21.15.3:0/3368758475 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b4809e000 msgr2=0x7f1b4811e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:24.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.814+0000 7f1b4dbbc700 1 --2- 172.21.15.3:0/3368758475 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b4809e000 0x7f1b4811e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:24.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.814+0000 7f1b4dbbc700 1 -- 172.21.15.3:0/3368758475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4000bb50 con 0x7f1b4809e410 2023-10-03T10:14:24.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.814+0000 7f1b4dbbc700 1 --2- 172.21.15.3:0/3368758475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b4809e410 0x7f1b4811ea80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1b40015000 tx=0x7f1b40004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:24.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.814+0000 7f1b3dffb700 1 -- 172.21.15.3:0/3368758475 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b40009e00 con 0x7f1b4809e410 2023-10-03T10:14:24.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.814+0000 7f1b3dffb700 1 -- 172.21.15.3:0/3368758475 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4000ceb0 con 0x7f1b4809e410 2023-10-03T10:14:24.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.814+0000 7f1b4f61f700 1 -- 172.21.15.3:0/3368758475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b48121260 con 0x7f1b4809e410 2023-10-03T10:14:24.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.814+0000 7f1b3dffb700 1 -- 172.21.15.3:0/3368758475 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4001d630 con 0x7f1b4809e410 2023-10-03T10:14:24.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.814+0000 7f1b4f61f700 1 -- 172.21.15.3:0/3368758475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b4809c700 con 0x7f1b4809e410 2023-10-03T10:14:24.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.815+0000 7f1b3dffb700 1 -- 172.21.15.3:0/3368758475 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1b4001a070 con 0x7f1b4809e410 2023-10-03T10:14:24.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.815+0000 7f1b3dffb700 1 -- 172.21.15.3:0/3368758475 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1b400410d0 con 0x7f1b4809e410 2023-10-03T10:14:24.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.816+0000 7f1b4f61f700 1 -- 172.21.15.3:0/3368758475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b48121490 con 0x7f1b4809e410 2023-10-03T10:14:24.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.821+0000 7f1b3dffb700 1 -- 172.21.15.3:0/3368758475 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1b48121490 con 0x7f1b4809e410 2023-10-03T10:14:24.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.881+0000 7f6fa3ad3700 1 Processor -- start 2023-10-03T10:14:24.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.881+0000 7f6fa3ad3700 1 -- start start 2023-10-03T10:14:24.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.881+0000 7f6fa3ad3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6f9c0a0a30 0x7f6f9c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:24.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.881+0000 7f6fa3ad3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f9c0a1340 0x7f6f9c13c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:24.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.881+0000 7f6fa3ad3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f9c09d920 con 0x7f6f9c13c640 2023-10-03T10:14:24.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.881+0000 7f6fa3ad3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f9c09d020 con 0x7f6f9c0a0a30 2023-10-03T10:14:24.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.881+0000 7f6fa3ad3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c0a2010 con 0x7f6f9c0a1340 2023-10-03T10:14:24.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.882+0000 7f6fa186f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6f9c0a0a30 0x7f6f9c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:24.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.882+0000 7f6fa106e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f9c0a1340 0x7f6f9c13c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:24.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.882+0000 7f6fa186f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6f9c0a0a30 0x7f6f9c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32892/0 (socket says 172.21.15.3:32892) 2023-10-03T10:14:24.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.882+0000 7f6fa106e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f9c0a1340 0x7f6f9c13c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44518/0 (socket says 172.21.15.3:44518) 2023-10-03T10:14:24.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.882+0000 7f6fa186f700 1 -- 172.21.15.3:0/1178110334 learned_addr learned my addr 172.21.15.3:0/1178110334 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:24.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.882+0000 7f6fa106e700 1 -- 172.21.15.3:0/1178110334 >> v2:172.21.15.3:3300/0 conn(0x7f6f9c0a0a30 msgr2=0x7f6f9c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:24.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.882+0000 7f6fa106e700 1 --2- 172.21.15.3:0/1178110334 >> v2:172.21.15.3:3300/0 conn(0x7f6f9c0a0a30 0x7f6f9c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:24.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.882+0000 7f6fa106e700 1 -- 172.21.15.3:0/1178110334 >> v1:172.21.15.3:6789/0 conn(0x7f6f9c13c640 legacy=0x7f6f9c13ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:24.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.882+0000 7f6fa106e700 1 -- 172.21.15.3:0/1178110334 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c09dbe0 con 0x7f6f9c0a1340 2023-10-03T10:14:24.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.882+0000 7f6fa106e700 1 --2- 172.21.15.3:0/1178110334 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f9c0a1340 0x7f6f9c13c100 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009840 tx=0x7f6f8c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9a18b5e73070421 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:24.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.883+0000 7f6fa086d700 1 -- 172.21.15.3:0/1178110334 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c011830 con 0x7f6f9c0a1340 2023-10-03T10:14:24.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.883+0000 7f6fa086d700 1 -- 172.21.15.3:0/1178110334 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8c011990 con 0x7f6f9c0a1340 2023-10-03T10:14:24.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.883+0000 7f6fa086d700 1 -- 172.21.15.3:0/1178110334 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c011d20 con 0x7f6f9c0a1340 2023-10-03T10:14:24.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.883+0000 7f6fa3ad3700 1 -- 172.21.15.3:0/1178110334 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f9c0a1340 msgr2=0x7f6f9c13c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:24.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.883+0000 7f6fa3ad3700 1 --2- 172.21.15.3:0/1178110334 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f9c0a1340 0x7f6f9c13c100 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009840 tx=0x7f6f8c009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:24.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.883+0000 7f6fa3ad3700 1 -- 172.21.15.3:0/1178110334 shutdown_connections 2023-10-03T10:14:24.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.883+0000 7f6fa3ad3700 1 --2- 172.21.15.3:0/1178110334 >> v2:172.21.15.3:3300/0 conn(0x7f6f9c0a0a30 0x7f6f9c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:24.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.883+0000 7f6fa3ad3700 1 --2- 172.21.15.3:0/1178110334 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f9c0a1340 0x7f6f9c13c100 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:24.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.883+0000 7f6fa3ad3700 1 -- 172.21.15.3:0/1178110334 >> 172.21.15.3:0/1178110334 conn(0x7f6f9c0a6550 msgr2=0x7f6f9c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:24.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.884+0000 7f6fa3ad3700 1 -- 172.21.15.3:0/1178110334 shutdown_connections 2023-10-03T10:14:24.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.884+0000 7f6fa3ad3700 1 -- 172.21.15.3:0/1178110334 wait complete. 2023-10-03T10:14:24.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.885+0000 7f6fa3ad3700 1 Processor -- start 2023-10-03T10:14:24.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.885+0000 7f6fa3ad3700 1 -- start start 2023-10-03T10:14:24.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.885+0000 7f6fa3ad3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f9c0a0a30 0x7f6f9c126fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:24.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.885+0000 7f6fa3ad3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f9c0a1340 0x7f6f9c127510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:24.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.886+0000 7f6fa3ad3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f9c13c640 0x7f6f9c127a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:24.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.886+0000 7f6fa3ad3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c12aaf0 con 0x7f6f9c0a1340 2023-10-03T10:14:24.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.886+0000 7f6fa3ad3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c13b960 con 0x7f6f9c13c640 2023-10-03T10:14:24.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.886+0000 7f6fa3ad3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c128be0 con 0x7f6f9c0a0a30 2023-10-03T10:14:24.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.886+0000 7f6fa106e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f9c0a1340 0x7f6f9c127510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:24.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.886+0000 7f6fa106e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f9c0a1340 0x7f6f9c127510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32894/0 (socket says 172.21.15.3:32894) 2023-10-03T10:14:24.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.886+0000 7f6fa106e700 1 -- 172.21.15.3:0/3616868419 learned_addr learned my addr 172.21.15.3:0/3616868419 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:24.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.886+0000 7f6fa186f700 1 --2- 172.21.15.3:0/3616868419 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f9c0a0a30 0x7f6f9c126fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:24.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.886+0000 7f6fa2070700 1 --2- 172.21.15.3:0/3616868419 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f9c13c640 0x7f6f9c127a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:24.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.886+0000 7f6fa106e700 1 -- 172.21.15.3:0/3616868419 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f9c0a0a30 msgr2=0x7f6f9c126fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:24.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.886+0000 7f6fa106e700 1 --2- 172.21.15.3:0/3616868419 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f9c0a0a30 0x7f6f9c126fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:24.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.886+0000 7f6fa106e700 1 -- 172.21.15.3:0/3616868419 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f9c13c640 msgr2=0x7f6f9c127a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:24.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.886+0000 7f6fa106e700 1 --2- 172.21.15.3:0/3616868419 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f9c13c640 0x7f6f9c127a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:24.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.886+0000 7f6fa106e700 1 -- 172.21.15.3:0/3616868419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8c0094a0 con 0x7f6f9c0a1340 2023-10-03T10:14:24.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.886+0000 7f6fa106e700 1 --2- 172.21.15.3:0/3616868419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f9c0a1340 0x7f6f9c127510 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009fd0 tx=0x7f6f8c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:24.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.886+0000 7f6f927fc700 1 -- 172.21.15.3:0/3616868419 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c0036a0 con 0x7f6f9c0a1340 2023-10-03T10:14:24.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.887+0000 7f6f927fc700 1 -- 172.21.15.3:0/3616868419 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8c01f070 con 0x7f6f9c0a1340 2023-10-03T10:14:24.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.887+0000 7f6fa3ad3700 1 -- 172.21.15.3:0/3616868419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f9c0a4a90 con 0x7f6f9c0a1340 2023-10-03T10:14:24.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.887+0000 7f6f927fc700 1 -- 172.21.15.3:0/3616868419 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c003a60 con 0x7f6f9c0a1340 2023-10-03T10:14:24.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.887+0000 7f6fa3ad3700 1 -- 172.21.15.3:0/3616868419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9c1293d0 con 0x7f6f9c0a1340 2023-10-03T10:14:24.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.888+0000 7f6fa3ad3700 1 -- 172.21.15.3:0/3616868419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f9c071e20 con 0x7f6f9c0a1340 2023-10-03T10:14:24.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.888+0000 7f6f927fc700 1 -- 172.21.15.3:0/3616868419 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6f8c003bc0 con 0x7f6f9c0a1340 2023-10-03T10:14:24.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.888+0000 7f6f927fc700 1 -- 172.21.15.3:0/3616868419 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6f8c021070 con 0x7f6f9c0a1340 2023-10-03T10:14:24.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.893+0000 7f6f927fc700 1 -- 172.21.15.3:0/3616868419 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6f8c0257f0 con 0x7f6f9c0a1340 2023-10-03T10:14:24.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.923+0000 7f4593207700 1 Processor -- start 2023-10-03T10:14:24.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.924+0000 7f4593207700 1 -- start start 2023-10-03T10:14:24.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.924+0000 7f4593207700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f458c09e1d0 0x7f458c135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:24.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.924+0000 7f4593207700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458c135e50 0x7f458c138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:24.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.924+0000 7f4593207700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f458c09f130 con 0x7f458c09d6f0 2023-10-03T10:14:24.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.924+0000 7f4593207700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f458c09f520 con 0x7f458c09e1d0 2023-10-03T10:14:24.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.924+0000 7f4593207700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458c09e930 con 0x7f458c135e50 2023-10-03T10:14:24.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.925+0000 7f458bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f458c09e1d0 0x7f458c135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:24.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.925+0000 7f458bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f458c09e1d0 0x7f458c135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32902/0 (socket says 172.21.15.3:32902) 2023-10-03T10:14:24.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.925+0000 7f4590fa3700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f458c09d6f0 0x7f458c09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50112/0 (socket says 172.21.15.3:50112) 2023-10-03T10:14:24.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.925+0000 7f458bfff700 1 -- 172.21.15.3:0/2018447476 learned_addr learned my addr 172.21.15.3:0/2018447476 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:24.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.925+0000 7f45917a4700 1 --2- 172.21.15.3:0/2018447476 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458c135e50 0x7f458c138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:24.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.925+0000 7f458bfff700 1 -- 172.21.15.3:0/2018447476 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458c135e50 msgr2=0x7f458c138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:24.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.925+0000 7f458bfff700 1 --2- 172.21.15.3:0/2018447476 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458c135e50 0x7f458c138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:24.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.925+0000 7f458bfff700 1 -- 172.21.15.3:0/2018447476 >> v1:172.21.15.3:6789/0 conn(0x7f458c09d6f0 legacy=0x7f458c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:24.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.925+0000 7f458bfff700 1 -- 172.21.15.3:0/2018447476 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458c09b2d0 con 0x7f458c09e1d0 2023-10-03T10:14:24.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.925+0000 7f458b7fe700 1 -- 172.21.15.3:0/2018447476 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1130468151 0 0) 0x7f458c09f130 con 0x7f458c09d6f0 2023-10-03T10:14:24.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.925+0000 7f458bfff700 1 --2- 172.21.15.3:0/2018447476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f458c09e1d0 0x7f458c135910 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f457800bb90 tx=0x7f457800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830370a8a1a71c23 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:24.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.925+0000 7f458b7fe700 1 -- 172.21.15.3:0/2018447476 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457800dd30 con 0x7f458c09e1d0 2023-10-03T10:14:24.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.926+0000 7f458b7fe700 1 -- 172.21.15.3:0/2018447476 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f457800de90 con 0x7f458c09e1d0 2023-10-03T10:14:24.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.926+0000 7f458b7fe700 1 -- 172.21.15.3:0/2018447476 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4578010690 con 0x7f458c09e1d0 2023-10-03T10:14:24.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.926+0000 7f4593207700 1 -- 172.21.15.3:0/2018447476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f458c09e1d0 msgr2=0x7f458c135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:24.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.926+0000 7f4593207700 1 --2- 172.21.15.3:0/2018447476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f458c09e1d0 0x7f458c135910 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f457800bb90 tx=0x7f457800bea0 comp rx=0 tx=0).stop 2023-10-03T10:14:24.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.926+0000 7f4593207700 1 -- 172.21.15.3:0/2018447476 shutdown_connections 2023-10-03T10:14:24.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.926+0000 7f4593207700 1 --2- 172.21.15.3:0/2018447476 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458c135e50 0x7f458c138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:24.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.926+0000 7f4593207700 1 --2- 172.21.15.3:0/2018447476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f458c09e1d0 0x7f458c135910 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:24.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.926+0000 7f4593207700 1 -- 172.21.15.3:0/2018447476 >> 172.21.15.3:0/2018447476 conn(0x7f458c066f50 msgr2=0x7f458c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:24.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.926+0000 7f4593207700 1 -- 172.21.15.3:0/2018447476 shutdown_connections 2023-10-03T10:14:24.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.926+0000 7f4593207700 1 -- 172.21.15.3:0/2018447476 wait complete. 2023-10-03T10:14:24.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.928+0000 7f4593207700 1 Processor -- start 2023-10-03T10:14:24.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.928+0000 7f4593207700 1 -- start start 2023-10-03T10:14:24.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.929+0000 7f4593207700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f458c09d6f0 0x7f458c11e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:24.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.929+0000 7f4593207700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458c09e1d0 0x7f458c11e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:24.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.929+0000 7f4593207700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f458c135e50 0x7f458c1c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:24.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.929+0000 7f4593207700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c11f540 con 0x7f458c09d6f0 2023-10-03T10:14:24.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.929+0000 7f4593207700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458c11f350 con 0x7f458c09e1d0 2023-10-03T10:14:24.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.929+0000 7f4593207700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f458c11fc20 con 0x7f458c135e50 2023-10-03T10:14:24.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.929+0000 7f45917a4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f458c135e50 0x7f458c1c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:24.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.929+0000 7f45917a4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f458c135e50 0x7f458c1c81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42738/0 (socket says 172.21.15.3:42738) 2023-10-03T10:14:24.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.929+0000 7f45917a4700 1 -- 172.21.15.3:0/2512374692 learned_addr learned my addr 172.21.15.3:0/2512374692 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:24.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.929+0000 7f4590fa3700 1 --2- 172.21.15.3:0/2512374692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f458c09d6f0 0x7f458c11e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:24.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.929+0000 7f458bfff700 1 --2- 172.21.15.3:0/2512374692 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458c09e1d0 0x7f458c11e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:24.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.929+0000 7f45917a4700 1 -- 172.21.15.3:0/2512374692 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458c09e1d0 msgr2=0x7f458c11e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:24.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.929+0000 7f45917a4700 1 --2- 172.21.15.3:0/2512374692 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458c09e1d0 0x7f458c11e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:24.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.929+0000 7f45917a4700 1 -- 172.21.15.3:0/2512374692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f458c09d6f0 msgr2=0x7f458c11e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:24.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.929+0000 7f45917a4700 1 --2- 172.21.15.3:0/2512374692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f458c09d6f0 0x7f458c11e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:24.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.929+0000 7f45917a4700 1 -- 172.21.15.3:0/2512374692 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457800b7f0 con 0x7f458c135e50 2023-10-03T10:14:24.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.929+0000 7f45917a4700 1 --2- 172.21.15.3:0/2512374692 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f458c135e50 0x7f458c1c81e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f458400efc0 tx=0x7f458400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:24.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.930+0000 7f45897fa700 1 -- 172.21.15.3:0/2512374692 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458400cc90 con 0x7f458c135e50 2023-10-03T10:14:24.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.930+0000 7f45897fa700 1 -- 172.21.15.3:0/2512374692 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458400f040 con 0x7f458c135e50 2023-10-03T10:14:24.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.930+0000 7f4593207700 1 -- 172.21.15.3:0/2512374692 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458c0a1d00 con 0x7f458c135e50 2023-10-03T10:14:24.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.930+0000 7f45897fa700 1 -- 172.21.15.3:0/2512374692 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45840146c0 con 0x7f458c135e50 2023-10-03T10:14:24.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.930+0000 7f4593207700 1 -- 172.21.15.3:0/2512374692 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458c11ebc0 con 0x7f458c135e50 2023-10-03T10:14:24.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.931+0000 7f45897fa700 1 -- 172.21.15.3:0/2512374692 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4584014820 con 0x7f458c135e50 2023-10-03T10:14:24.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.931+0000 7f45897fa700 1 -- 172.21.15.3:0/2512374692 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4584040380 con 0x7f458c135e50 2023-10-03T10:14:24.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.931+0000 7f4593207700 1 -- 172.21.15.3:0/2512374692 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f458c074ea0 con 0x7f458c135e50 2023-10-03T10:14:24.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:24.936+0000 7f45897fa700 1 -- 172.21.15.3:0/2512374692 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4584018070 con 0x7f458c135e50 2023-10-03T10:14:25.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.041+0000 7f1b4f61f700 1 -- 172.21.15.3:0/3368758475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1b48121490 con 0x7f1b4809e410 2023-10-03T10:14:25.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.041+0000 7f1b3dffb700 1 -- 172.21.15.3:0/3368758475 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1b48121490 con 0x7f1b4809e410 2023-10-03T10:14:25.041 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:25.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.044+0000 7f1b4f61f700 1 -- 172.21.15.3:0/3368758475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b4809e410 msgr2=0x7f1b4811ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:25.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.044+0000 7f1b4f61f700 1 --2- 172.21.15.3:0/3368758475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b4809e410 0x7f1b4811ea80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1b40015000 tx=0x7f1b40004ad0 comp rx=0 tx=0).stop 2023-10-03T10:14:25.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.045+0000 7f1b4f61f700 1 -- 172.21.15.3:0/3368758475 shutdown_connections 2023-10-03T10:14:25.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.045+0000 7f1b4f61f700 1 --2- 172.21.15.3:0/3368758475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b4809e410 0x7f1b4811ea80 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:25.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.045+0000 7f1b4f61f700 1 --2- 172.21.15.3:0/3368758475 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b4809d6f0 0x7f1b4811e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:25.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.045+0000 7f1b4f61f700 1 --2- 172.21.15.3:0/3368758475 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b4809e000 0x7f1b4811e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:25.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.045+0000 7f1b4f61f700 1 -- 172.21.15.3:0/3368758475 >> 172.21.15.3:0/3368758475 conn(0x7f1b48066f50 msgr2=0x7f1b48130970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:25.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.045+0000 7f1b4f61f700 1 -- 172.21.15.3:0/3368758475 shutdown_connections 2023-10-03T10:14:25.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.045+0000 7f1b4f61f700 1 -- 172.21.15.3:0/3368758475 wait complete. 2023-10-03T10:14:25.064 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:25.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.119+0000 7f6fa3ad3700 1 -- 172.21.15.3:0/3616868419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6f9c074ea0 con 0x7f6f9c0a1340 2023-10-03T10:14:25.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.120+0000 7f6f927fc700 1 -- 172.21.15.3:0/3616868419 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6f8c015020 con 0x7f6f9c0a1340 2023-10-03T10:14:25.120 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.122+0000 7f6fa3ad3700 1 -- 172.21.15.3:0/3616868419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f9c0a1340 msgr2=0x7f6f9c127510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.122+0000 7f6fa3ad3700 1 --2- 172.21.15.3:0/3616868419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f9c0a1340 0x7f6f9c127510 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009fd0 tx=0x7f6f8c004750 comp rx=0 tx=0).stop 2023-10-03T10:14:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.122+0000 7f6fa3ad3700 1 -- 172.21.15.3:0/3616868419 shutdown_connections 2023-10-03T10:14:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.122+0000 7f6fa3ad3700 1 --2- 172.21.15.3:0/3616868419 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f9c13c640 0x7f6f9c127a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.122+0000 7f6fa3ad3700 1 --2- 172.21.15.3:0/3616868419 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f9c0a0a30 0x7f6f9c126fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.122+0000 7f6fa3ad3700 1 --2- 172.21.15.3:0/3616868419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f9c0a1340 0x7f6f9c127510 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.122+0000 7f6fa3ad3700 1 -- 172.21.15.3:0/3616868419 >> 172.21.15.3:0/3616868419 conn(0x7f6f9c0a6550 msgr2=0x7f6f9c0a2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.122+0000 7f6fa3ad3700 1 -- 172.21.15.3:0/3616868419 shutdown_connections 2023-10-03T10:14:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.122+0000 7f6fa3ad3700 1 -- 172.21.15.3:0/3616868419 wait complete. 2023-10-03T10:14:25.136 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:25.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.142+0000 7f4593207700 1 -- 172.21.15.3:0/2512374692 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f458c11f8c0 con 0x7f458c135e50 2023-10-03T10:14:25.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.142+0000 7f45897fa700 1 -- 172.21.15.3:0/2512374692 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f458402c090 con 0x7f458c135e50 2023-10-03T10:14:25.143 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:25.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.145+0000 7f4593207700 1 -- 172.21.15.3:0/2512374692 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f458c135e50 msgr2=0x7f458c1c81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:25.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.145+0000 7f4593207700 1 --2- 172.21.15.3:0/2512374692 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f458c135e50 0x7f458c1c81e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f458400efc0 tx=0x7f458400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:14:25.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.145+0000 7f4593207700 1 -- 172.21.15.3:0/2512374692 shutdown_connections 2023-10-03T10:14:25.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.145+0000 7f4593207700 1 --2- 172.21.15.3:0/2512374692 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f458c135e50 0x7f458c1c81e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:25.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.145+0000 7f4593207700 1 --2- 172.21.15.3:0/2512374692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f458c09d6f0 0x7f458c11e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:25.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.145+0000 7f4593207700 1 --2- 172.21.15.3:0/2512374692 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458c09e1d0 0x7f458c11e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:25.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.145+0000 7f4593207700 1 -- 172.21.15.3:0/2512374692 >> 172.21.15.3:0/2512374692 conn(0x7f458c066f50 msgr2=0x7f458c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:25.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.145+0000 7f4593207700 1 -- 172.21.15.3:0/2512374692 shutdown_connections 2023-10-03T10:14:25.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.145+0000 7f4593207700 1 -- 172.21.15.3:0/2512374692 wait complete. 2023-10-03T10:14:25.158 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:25.293 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:25.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.438+0000 7f5c6eda3700 1 Processor -- start 2023-10-03T10:14:25.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.438+0000 7f5c6eda3700 1 -- start start 2023-10-03T10:14:25.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.438+0000 7f5c6eda3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c680a0a30 0x7f5c680a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:25.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.438+0000 7f5c6eda3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c680a1340 0x7f5c68133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:25.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.438+0000 7f5c6eda3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c68133fa0 0x7f5c68134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:25.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.438+0000 7f5c6eda3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c6809edd0 con 0x7f5c680a1340 2023-10-03T10:14:25.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.438+0000 7f5c6eda3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6809e1f0 con 0x7f5c68133fa0 2023-10-03T10:14:25.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.438+0000 7f5c6eda3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6809dc50 con 0x7f5c680a0a30 2023-10-03T10:14:25.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.439+0000 7f5c6d340700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c68133fa0 0x7f5c68134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:25.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.439+0000 7f5c67fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c680a1340 0x7f5c68133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:25.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.439+0000 7f5c6cb3f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c680a0a30 0x7f5c680a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:25.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.439+0000 7f5c67fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c680a1340 0x7f5c68133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32918/0 (socket says 172.21.15.3:32918) 2023-10-03T10:14:25.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.439+0000 7f5c6d340700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c68133fa0 0x7f5c68134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44568/0 (socket says 172.21.15.3:44568) 2023-10-03T10:14:25.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.439+0000 7f5c67fff700 1 -- 172.21.15.3:0/1469593728 learned_addr learned my addr 172.21.15.3:0/1469593728 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:25.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.439+0000 7f5c67fff700 1 -- 172.21.15.3:0/1469593728 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c680a0a30 msgr2=0x7f5c680a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:25.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.439+0000 7f5c67fff700 1 --2- 172.21.15.3:0/1469593728 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c680a0a30 0x7f5c680a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:25.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.439+0000 7f5c67fff700 1 -- 172.21.15.3:0/1469593728 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c68133fa0 msgr2=0x7f5c68134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:25.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.439+0000 7f5c67fff700 1 --2- 172.21.15.3:0/1469593728 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c68133fa0 0x7f5c68134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:25.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.439+0000 7f5c67fff700 1 -- 172.21.15.3:0/1469593728 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c680a1fe0 con 0x7f5c680a1340 2023-10-03T10:14:25.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.439+0000 7f5c6cb3f700 1 --2- 172.21.15.3:0/1469593728 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c680a0a30 0x7f5c680a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:25.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.439+0000 7f5c6d340700 1 --2- 172.21.15.3:0/1469593728 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c68133fa0 0x7f5c68134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:25.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.439+0000 7f5c67fff700 1 --2- 172.21.15.3:0/1469593728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c680a1340 0x7f5c68133a60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c0098f0 tx=0x7f5c5c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9351e9560016d784 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:25.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.439+0000 7f5c677fe700 1 -- 172.21.15.3:0/1469593728 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c011830 con 0x7f5c680a1340 2023-10-03T10:14:25.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.439+0000 7f5c677fe700 1 -- 172.21.15.3:0/1469593728 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c5c011990 con 0x7f5c680a1340 2023-10-03T10:14:25.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.439+0000 7f5c677fe700 1 -- 172.21.15.3:0/1469593728 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c011d20 con 0x7f5c680a1340 2023-10-03T10:14:25.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.440+0000 7f5c6eda3700 1 -- 172.21.15.3:0/1469593728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c680a1340 msgr2=0x7f5c68133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:25.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.440+0000 7f5c6eda3700 1 --2- 172.21.15.3:0/1469593728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c680a1340 0x7f5c68133a60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c0098f0 tx=0x7f5c5c009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:25.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.440+0000 7f5c6eda3700 1 -- 172.21.15.3:0/1469593728 shutdown_connections 2023-10-03T10:14:25.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.440+0000 7f5c6eda3700 1 --2- 172.21.15.3:0/1469593728 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c68133fa0 0x7f5c68134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:25.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.440+0000 7f5c6eda3700 1 --2- 172.21.15.3:0/1469593728 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c680a0a30 0x7f5c680a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:25.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.440+0000 7f5c6eda3700 1 --2- 172.21.15.3:0/1469593728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c680a1340 0x7f5c68133a60 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:25.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.440+0000 7f5c6eda3700 1 -- 172.21.15.3:0/1469593728 >> 172.21.15.3:0/1469593728 conn(0x7f5c680a6550 msgr2=0x7f5c680a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:25.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.440+0000 7f5c6eda3700 1 -- 172.21.15.3:0/1469593728 shutdown_connections 2023-10-03T10:14:25.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.440+0000 7f5c6eda3700 1 -- 172.21.15.3:0/1469593728 wait complete. 2023-10-03T10:14:25.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.441+0000 7f5c6eda3700 1 Processor -- start 2023-10-03T10:14:25.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.441+0000 7f5c6eda3700 1 -- start start 2023-10-03T10:14:25.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.442+0000 7f5c6eda3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c680a0a30 0x7f5c68120270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:25.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.442+0000 7f5c6eda3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c680a1340 0x7f5c681207b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:25.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.442+0000 7f5c6eda3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c68133fa0 0x7f5c68120cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:25.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.442+0000 7f5c6eda3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c68121740 con 0x7f5c680a1340 2023-10-03T10:14:25.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.442+0000 7f5c6eda3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c68133660 con 0x7f5c680a0a30 2023-10-03T10:14:25.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.442+0000 7f5c6eda3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c681230d0 con 0x7f5c68133fa0 2023-10-03T10:14:25.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.442+0000 7f5c67fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c680a1340 0x7f5c681207b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:25.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.442+0000 7f5c67fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c680a1340 0x7f5c681207b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32934/0 (socket says 172.21.15.3:32934) 2023-10-03T10:14:25.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.442+0000 7f5c67fff700 1 -- 172.21.15.3:0/2162860049 learned_addr learned my addr 172.21.15.3:0/2162860049 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:25.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.442+0000 7f5c6cb3f700 1 --2- 172.21.15.3:0/2162860049 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c680a0a30 0x7f5c68120270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:25.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.442+0000 7f5c67fff700 1 -- 172.21.15.3:0/2162860049 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c68133fa0 msgr2=0x7f5c68120cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:25.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.442+0000 7f5c67fff700 1 --2- 172.21.15.3:0/2162860049 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c68133fa0 0x7f5c68120cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:25.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.442+0000 7f5c67fff700 1 -- 172.21.15.3:0/2162860049 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c680a0a30 msgr2=0x7f5c68120270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:25.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.442+0000 7f5c67fff700 1 --2- 172.21.15.3:0/2162860049 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c680a0a30 0x7f5c68120270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:25.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.442+0000 7f5c67fff700 1 -- 172.21.15.3:0/2162860049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c580094a0 con 0x7f5c680a1340 2023-10-03T10:14:25.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.442+0000 7f5c67fff700 1 --2- 172.21.15.3:0/2162860049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c680a1340 0x7f5c681207b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c011db0 tx=0x7f5c5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:25.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.443+0000 7f5c657fa700 1 -- 172.21.15.3:0/2162860049 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c004240 con 0x7f5c680a1340 2023-10-03T10:14:25.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.443+0000 7f5c657fa700 1 -- 172.21.15.3:0/2162860049 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c5c01f070 con 0x7f5c680a1340 2023-10-03T10:14:25.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.443+0000 7f5c6eda3700 1 -- 172.21.15.3:0/2162860049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c60009a70 con 0x7f5c680a1340 2023-10-03T10:14:25.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.443+0000 7f5c657fa700 1 -- 172.21.15.3:0/2162860049 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c0045a0 con 0x7f5c680a1340 2023-10-03T10:14:25.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.443+0000 7f5c6eda3700 1 -- 172.21.15.3:0/2162860049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c5c009550 con 0x7f5c680a1340 2023-10-03T10:14:25.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.443+0000 7f5c657fa700 1 -- 172.21.15.3:0/2162860049 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5c5c021030 con 0x7f5c680a1340 2023-10-03T10:14:25.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.444+0000 7f5c657fa700 1 -- 172.21.15.3:0/2162860049 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5c5c040910 con 0x7f5c680a1340 2023-10-03T10:14:25.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.444+0000 7f5c6eda3700 1 -- 172.21.15.3:0/2162860049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c680a0220 con 0x7f5c680a1340 2023-10-03T10:14:25.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.449+0000 7f5c657fa700 1 -- 172.21.15.3:0/2162860049 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5c5c015020 con 0x7f5c680a1340 2023-10-03T10:14:25.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.653+0000 7f5c6eda3700 1 -- 172.21.15.3:0/2162860049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5c68074ea0 con 0x7f5c680a1340 2023-10-03T10:14:25.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.653+0000 7f5c657fa700 1 -- 172.21.15.3:0/2162860049 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5c5c0036a0 con 0x7f5c680a1340 2023-10-03T10:14:25.654 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:25.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.656+0000 7f5c6eda3700 1 -- 172.21.15.3:0/2162860049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c680a1340 msgr2=0x7f5c681207b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:25.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.656+0000 7f5c6eda3700 1 --2- 172.21.15.3:0/2162860049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c680a1340 0x7f5c681207b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c011db0 tx=0x7f5c5c004750 comp rx=0 tx=0).stop 2023-10-03T10:14:25.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.656+0000 7f5c6eda3700 1 -- 172.21.15.3:0/2162860049 shutdown_connections 2023-10-03T10:14:25.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.656+0000 7f5c6eda3700 1 --2- 172.21.15.3:0/2162860049 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c68133fa0 0x7f5c68120cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:25.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.656+0000 7f5c6eda3700 1 --2- 172.21.15.3:0/2162860049 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c680a0a30 0x7f5c68120270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:25.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.656+0000 7f5c6eda3700 1 --2- 172.21.15.3:0/2162860049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c680a1340 0x7f5c681207b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:25.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.656+0000 7f5c6eda3700 1 -- 172.21.15.3:0/2162860049 >> 172.21.15.3:0/2162860049 conn(0x7f5c680a6550 msgr2=0x7f5c681baf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:25.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.656+0000 7f5c6eda3700 1 -- 172.21.15.3:0/2162860049 shutdown_connections 2023-10-03T10:14:25.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:25.656+0000 7f5c6eda3700 1 -- 172.21.15.3:0/2162860049 wait complete. 2023-10-03T10:14:25.670 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:26.065 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:26.137 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:26.160 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:26.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.215+0000 7f574f673700 1 Processor -- start 2023-10-03T10:14:26.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.215+0000 7f574f673700 1 -- start start 2023-10-03T10:14:26.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.216+0000 7f574f673700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f574809d6f0 0x7f574809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:26.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.216+0000 7f574f673700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f574809e410 0x7f5748130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:26.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.216+0000 7f574f673700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574809f130 con 0x7f574809e000 2023-10-03T10:14:26.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.216+0000 7f574f673700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574809f520 con 0x7f574809d6f0 2023-10-03T10:14:26.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.216+0000 7f574f673700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f574809e930 con 0x7f574809e410 2023-10-03T10:14:26.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.216+0000 7f574dc10700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f574809e410 0x7f5748130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:26.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.216+0000 7f574d40f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f574809d6f0 0x7f574809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:26.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.216+0000 7f574d40f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f574809d6f0 0x7f574809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44580/0 (socket says 172.21.15.3:44580) 2023-10-03T10:14:26.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.216+0000 7f574dc10700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f574809e410 0x7f5748130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42756/0 (socket says 172.21.15.3:42756) 2023-10-03T10:14:26.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.216+0000 7f574d40f700 1 -- 172.21.15.3:0/2581501845 learned_addr learned my addr 172.21.15.3:0/2581501845 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:26.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.216+0000 7f574dc10700 1 -- 172.21.15.3:0/2581501845 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f574809d6f0 msgr2=0x7f574809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:26.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.216+0000 7f574dc10700 1 --2- 172.21.15.3:0/2581501845 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f574809d6f0 0x7f574809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.216+0000 7f574dc10700 1 -- 172.21.15.3:0/2581501845 >> v1:172.21.15.3:6789/0 conn(0x7f574809e000 legacy=0x7f5748130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:26.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.216+0000 7f574dc10700 1 -- 172.21.15.3:0/2581501845 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574809b2d0 con 0x7f574809e410 2023-10-03T10:14:26.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.216+0000 7f573ffff700 1 -- 172.21.15.3:0/2581501845 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3598057575 0 0) 0x7f574809f130 con 0x7f574809e000 2023-10-03T10:14:26.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.217+0000 7f574dc10700 1 --2- 172.21.15.3:0/2581501845 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f574809e410 0x7f5748130d70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5740009dc0 tx=0x7f574000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20b18a3fcd1119b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:26.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.217+0000 7f573ffff700 1 -- 172.21.15.3:0/2581501845 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574000e070 con 0x7f574809e410 2023-10-03T10:14:26.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.217+0000 7f573ffff700 1 -- 172.21.15.3:0/2581501845 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f574000bb20 con 0x7f574809e410 2023-10-03T10:14:26.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.217+0000 7f573ffff700 1 -- 172.21.15.3:0/2581501845 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5740012430 con 0x7f574809e410 2023-10-03T10:14:26.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.218+0000 7f574f673700 1 -- 172.21.15.3:0/2581501845 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f574809e410 msgr2=0x7f5748130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:26.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.218+0000 7f574f673700 1 --2- 172.21.15.3:0/2581501845 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f574809e410 0x7f5748130d70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5740009dc0 tx=0x7f574000b330 comp rx=0 tx=0).stop 2023-10-03T10:14:26.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.218+0000 7f574f673700 1 -- 172.21.15.3:0/2581501845 shutdown_connections 2023-10-03T10:14:26.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.218+0000 7f574f673700 1 --2- 172.21.15.3:0/2581501845 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f574809d6f0 0x7f574809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.218+0000 7f574f673700 1 --2- 172.21.15.3:0/2581501845 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f574809e410 0x7f5748130d70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.218+0000 7f574f673700 1 -- 172.21.15.3:0/2581501845 >> 172.21.15.3:0/2581501845 conn(0x7f5748066f50 msgr2=0x7f574809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:26.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.218+0000 7f574f673700 1 -- 172.21.15.3:0/2581501845 shutdown_connections 2023-10-03T10:14:26.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.218+0000 7f574f673700 1 -- 172.21.15.3:0/2581501845 wait complete. 2023-10-03T10:14:26.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.219+0000 7f574f673700 1 Processor -- start 2023-10-03T10:14:26.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.219+0000 7f574f673700 1 -- start start 2023-10-03T10:14:26.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.220+0000 7f574f673700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f574809d6f0 0x7f57481210c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:26.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.220+0000 7f574f673700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f574809e000 0x7f5748121600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:26.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.220+0000 7f574f673700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f574809e410 0x7f5748121b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:26.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.220+0000 7f574f673700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5748130360 con 0x7f574809d6f0 2023-10-03T10:14:26.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.220+0000 7f574f673700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5748123520 con 0x7f574809e000 2023-10-03T10:14:26.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.220+0000 7f574f673700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5748122410 con 0x7f574809e410 2023-10-03T10:14:26.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.220+0000 7f574dc10700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f574809e410 0x7f5748121b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:26.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.220+0000 7f574cc0e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f574809e000 0x7f5748121600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:26.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.220+0000 7f574dc10700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f574809e410 0x7f5748121b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42766/0 (socket says 172.21.15.3:42766) 2023-10-03T10:14:26.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.220+0000 7f574dc10700 1 -- 172.21.15.3:0/3827340934 learned_addr learned my addr 172.21.15.3:0/3827340934 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:26.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.220+0000 7f574cc0e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f574809e000 0x7f5748121600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44584/0 (socket says 172.21.15.3:44584) 2023-10-03T10:14:26.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.221+0000 7f574d40f700 1 --2- 172.21.15.3:0/3827340934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f574809d6f0 0x7f57481210c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:26.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.221+0000 7f574cc0e700 1 -- 172.21.15.3:0/3827340934 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f574809e410 msgr2=0x7f5748121b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:26.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.221+0000 7f574cc0e700 1 --2- 172.21.15.3:0/3827340934 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f574809e410 0x7f5748121b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.221+0000 7f574cc0e700 1 -- 172.21.15.3:0/3827340934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f574809d6f0 msgr2=0x7f57481210c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:26.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.221+0000 7f574cc0e700 1 --2- 172.21.15.3:0/3827340934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f574809d6f0 0x7f57481210c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.221+0000 7f574cc0e700 1 -- 172.21.15.3:0/3827340934 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5740009a70 con 0x7f574809e000 2023-10-03T10:14:26.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.221+0000 7f574d40f700 1 --2- 172.21.15.3:0/3827340934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f574809d6f0 0x7f57481210c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:26.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.221+0000 7f574dc10700 1 --2- 172.21.15.3:0/3827340934 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f574809e410 0x7f5748121b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:26.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.221+0000 7f574cc0e700 1 --2- 172.21.15.3:0/3827340934 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f574809e000 0x7f5748121600 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f573800c460 tx=0x7f573800c770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:26.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.221+0000 7f573dffb700 1 -- 172.21.15.3:0/3827340934 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57380052d0 con 0x7f574809e000 2023-10-03T10:14:26.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.221+0000 7f573dffb700 1 -- 172.21.15.3:0/3827340934 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5738005450 con 0x7f574809e000 2023-10-03T10:14:26.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.221+0000 7f573dffb700 1 -- 172.21.15.3:0/3827340934 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57380038c0 con 0x7f574809e000 2023-10-03T10:14:26.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.221+0000 7f574f673700 1 -- 172.21.15.3:0/3827340934 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57481237e0 con 0x7f574809e000 2023-10-03T10:14:26.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.222+0000 7f574f673700 1 -- 172.21.15.3:0/3827340934 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574806ebf0 con 0x7f574809e000 2023-10-03T10:14:26.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.222+0000 7f573dffb700 1 -- 172.21.15.3:0/3827340934 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5738003a20 con 0x7f574809e000 2023-10-03T10:14:26.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.223+0000 7f573dffb700 1 -- 172.21.15.3:0/3827340934 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5738040a10 con 0x7f574809e000 2023-10-03T10:14:26.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.223+0000 7f574f673700 1 -- 172.21.15.3:0/3827340934 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f574809c760 con 0x7f574809e000 2023-10-03T10:14:26.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.228+0000 7f573dffb700 1 -- 172.21.15.3:0/3827340934 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5738017020 con 0x7f574809e000 2023-10-03T10:14:26.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.281+0000 7f40e4ec1700 1 Processor -- start 2023-10-03T10:14:26.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.282+0000 7f40e4ec1700 1 -- start start 2023-10-03T10:14:26.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.283+0000 7f40e4ec1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40e009e930 0x7f40e009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:26.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.283+0000 7f40e4ec1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40e009f240 0x7f40e01317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:26.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.283+0000 7f40e4ec1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40e0070940 con 0x7f40e009f650 2023-10-03T10:14:26.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.283+0000 7f40e4ec1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e006fd60 con 0x7f40e009e930 2023-10-03T10:14:26.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.283+0000 7f40e4ec1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40e006fa80 con 0x7f40e009f240 2023-10-03T10:14:26.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.283+0000 7f40de59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40e009e930 0x7f40e009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:26.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.283+0000 7f40ded9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f40e009f650 0x7f40e0131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50142/0 (socket says 172.21.15.3:50142) 2023-10-03T10:14:26.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.283+0000 7f40ded9d700 1 -- 172.21.15.3:0/712289896 learned_addr learned my addr 172.21.15.3:0/712289896 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:26.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.283+0000 7f40ddd9b700 1 --2- 172.21.15.3:0/712289896 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40e009f240 0x7f40e01317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:26.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.283+0000 7f40dd59a700 1 -- 172.21.15.3:0/712289896 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 563319169 0 0) 0x7f40e0070940 con 0x7f40e009f650 2023-10-03T10:14:26.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.283+0000 7f40de59c700 1 -- 172.21.15.3:0/712289896 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40e009f240 msgr2=0x7f40e01317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:26.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.283+0000 7f40de59c700 1 --2- 172.21.15.3:0/712289896 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40e009f240 0x7f40e01317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.283+0000 7f40de59c700 1 -- 172.21.15.3:0/712289896 >> v1:172.21.15.3:6789/0 conn(0x7f40e009f650 legacy=0x7f40e0131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:26.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.283+0000 7f40de59c700 1 -- 172.21.15.3:0/712289896 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e009fb60 con 0x7f40e009e930 2023-10-03T10:14:26.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.283+0000 7f40ddd9b700 1 --2- 172.21.15.3:0/712289896 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40e009f240 0x7f40e01317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:26.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.283+0000 7f40de59c700 1 --2- 172.21.15.3:0/712289896 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40e009e930 0x7f40e009ed00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f40d0009870 tx=0x7f40d0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccfe0f27c334faad server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:26.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.284+0000 7f40dd59a700 1 -- 172.21.15.3:0/712289896 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d0011830 con 0x7f40e009e930 2023-10-03T10:14:26.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.284+0000 7f40dd59a700 1 -- 172.21.15.3:0/712289896 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d0011990 con 0x7f40e009e930 2023-10-03T10:14:26.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.284+0000 7f40dd59a700 1 -- 172.21.15.3:0/712289896 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d0011d20 con 0x7f40e009e930 2023-10-03T10:14:26.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.284+0000 7f40e4ec1700 1 -- 172.21.15.3:0/712289896 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40e009e930 msgr2=0x7f40e009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:26.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.284+0000 7f40e4ec1700 1 --2- 172.21.15.3:0/712289896 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40e009e930 0x7f40e009ed00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f40d0009870 tx=0x7f40d0009b80 comp rx=0 tx=0).stop 2023-10-03T10:14:26.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.285+0000 7f40e4ec1700 1 -- 172.21.15.3:0/712289896 shutdown_connections 2023-10-03T10:14:26.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.285+0000 7f40e4ec1700 1 --2- 172.21.15.3:0/712289896 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40e009e930 0x7f40e009ed00 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.285+0000 7f40e4ec1700 1 --2- 172.21.15.3:0/712289896 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40e009f240 0x7f40e01317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.285+0000 7f40e4ec1700 1 -- 172.21.15.3:0/712289896 >> 172.21.15.3:0/712289896 conn(0x7f40e00a4210 msgr2=0x7f40e00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:26.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.285+0000 7f40e4ec1700 1 -- 172.21.15.3:0/712289896 shutdown_connections 2023-10-03T10:14:26.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.285+0000 7f40e4ec1700 1 -- 172.21.15.3:0/712289896 wait complete. 2023-10-03T10:14:26.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.286+0000 7f40e4ec1700 1 Processor -- start 2023-10-03T10:14:26.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.286+0000 7f40e4ec1700 1 -- start start 2023-10-03T10:14:26.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.286+0000 7f40e4ec1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40e009e930 0x7f40e01a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:26.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.286+0000 7f40e4ec1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40e009f240 0x7f40e01a4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:26.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.286+0000 7f40e4ec1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e009f650 0x7f40e01a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:26.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.286+0000 7f40e4ec1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e01a7650 con 0x7f40e009f650 2023-10-03T10:14:26.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.286+0000 7f40e4ec1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e01a6a20 con 0x7f40e009f240 2023-10-03T10:14:26.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.286+0000 7f40e4ec1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40e00a3410 con 0x7f40e009e930 2023-10-03T10:14:26.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.287+0000 7f40de59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40e009e930 0x7f40e01a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:26.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.287+0000 7f40de59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40e009e930 0x7f40e01a3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42778/0 (socket says 172.21.15.3:42778) 2023-10-03T10:14:26.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.287+0000 7f40de59c700 1 -- 172.21.15.3:0/1160666652 learned_addr learned my addr 172.21.15.3:0/1160666652 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:26.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.287+0000 7f40ddd9b700 1 --2- 172.21.15.3:0/1160666652 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40e009f240 0x7f40e01a4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:26.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.287+0000 7f40ded9d700 1 --2- 172.21.15.3:0/1160666652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e009f650 0x7f40e01a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:26.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.287+0000 7f40ddd9b700 1 -- 172.21.15.3:0/1160666652 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40e009e930 msgr2=0x7f40e01a3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:26.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.287+0000 7f40ddd9b700 1 --2- 172.21.15.3:0/1160666652 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40e009e930 0x7f40e01a3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.287+0000 7f40ddd9b700 1 -- 172.21.15.3:0/1160666652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e009f650 msgr2=0x7f40e01a4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:26.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.287+0000 7f40ddd9b700 1 --2- 172.21.15.3:0/1160666652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e009f650 0x7f40e01a4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.287+0000 7f40ddd9b700 1 -- 172.21.15.3:0/1160666652 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d40094a0 con 0x7f40e009f240 2023-10-03T10:14:26.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.287+0000 7f40de59c700 1 --2- 172.21.15.3:0/1160666652 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40e009e930 0x7f40e01a3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:26.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.287+0000 7f40ddd9b700 1 --2- 172.21.15.3:0/1160666652 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40e009f240 0x7f40e01a4110 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f40d400e980 tx=0x7f40d400ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:26.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.287+0000 7f40ceffd700 1 -- 172.21.15.3:0/1160666652 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d400c690 con 0x7f40e009f240 2023-10-03T10:14:26.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.288+0000 7f40ceffd700 1 -- 172.21.15.3:0/1160666652 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d400c7f0 con 0x7f40e009f240 2023-10-03T10:14:26.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.288+0000 7f40e4ec1700 1 -- 172.21.15.3:0/1160666652 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d00094a0 con 0x7f40e009f240 2023-10-03T10:14:26.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.288+0000 7f40ceffd700 1 -- 172.21.15.3:0/1160666652 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d400cb80 con 0x7f40e009f240 2023-10-03T10:14:26.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.288+0000 7f40e4ec1700 1 -- 172.21.15.3:0/1160666652 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e00a0190 con 0x7f40e009f240 2023-10-03T10:14:26.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.289+0000 7f40ceffd700 1 -- 172.21.15.3:0/1160666652 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f40d400cce0 con 0x7f40e009f240 2023-10-03T10:14:26.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.289+0000 7f40ceffd700 1 -- 172.21.15.3:0/1160666652 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f40d403fab0 con 0x7f40e009f240 2023-10-03T10:14:26.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.289+0000 7f40e4ec1700 1 -- 172.21.15.3:0/1160666652 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40e01a0b30 con 0x7f40e009f240 2023-10-03T10:14:26.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.298+0000 7f40ceffd700 1 -- 172.21.15.3:0/1160666652 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f40d4021080 con 0x7f40e009f240 2023-10-03T10:14:26.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.333+0000 7f7cf0a48700 1 Processor -- start 2023-10-03T10:14:26.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.333+0000 7f7cf0a48700 1 -- start start 2023-10-03T10:14:26.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.333+0000 7f7cf0a48700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7cec0a0a30 0x7f7cec0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:26.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.333+0000 7f7cf0a48700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cec138d80 0x7f7cec13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:26.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.333+0000 7f7cf0a48700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cec0a2a70 con 0x7f7cec09d630 2023-10-03T10:14:26.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.333+0000 7f7cf0a48700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cec0a01d0 con 0x7f7cec0a0a30 2023-10-03T10:14:26.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.333+0000 7f7cf0a48700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7cec09e2b0 con 0x7f7cec138d80 2023-10-03T10:14:26.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.333+0000 7f7cea59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7cec0a0a30 0x7f7cec0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:26.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.333+0000 7f7cea59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7cec0a0a30 0x7f7cec0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32966/0 (socket says 172.21.15.3:32966) 2023-10-03T10:14:26.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.333+0000 7f7cea59c700 1 -- 172.21.15.3:0/1087825210 learned_addr learned my addr 172.21.15.3:0/1087825210 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:26.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.333+0000 7f7cead9d700 1 --2- 172.21.15.3:0/1087825210 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cec138d80 0x7f7cec13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:26.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.334+0000 7f7cea59c700 1 -- 172.21.15.3:0/1087825210 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cec138d80 msgr2=0x7f7cec13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:26.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.334+0000 7f7cea59c700 1 --2- 172.21.15.3:0/1087825210 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cec138d80 0x7f7cec13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.334+0000 7f7cea59c700 1 -- 172.21.15.3:0/1087825210 >> v1:172.21.15.3:6789/0 conn(0x7f7cec09d630 legacy=0x7f7cec0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:26.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.334+0000 7f7cea59c700 1 -- 172.21.15.3:0/1087825210 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cec09dc50 con 0x7f7cec0a0a30 2023-10-03T10:14:26.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.334+0000 7f7cead9d700 1 --2- 172.21.15.3:0/1087825210 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cec138d80 0x7f7cec13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:26.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.334+0000 7f7cea59c700 1 --2- 172.21.15.3:0/1087825210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cec0a0a30 0x7f7cec0a0e00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc0098f0 tx=0x7f7cdc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3834e8465227f4d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:26.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.334+0000 7f7ce9d9b700 1 -- 172.21.15.3:0/1087825210 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc011830 con 0x7f7cec0a0a30 2023-10-03T10:14:26.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.334+0000 7f7ce9d9b700 1 -- 172.21.15.3:0/1087825210 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cdc011990 con 0x7f7cec0a0a30 2023-10-03T10:14:26.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.334+0000 7f7ce9d9b700 1 -- 172.21.15.3:0/1087825210 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc011d20 con 0x7f7cec0a0a30 2023-10-03T10:14:26.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.335+0000 7f7cf0a48700 1 -- 172.21.15.3:0/1087825210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cec0a0a30 msgr2=0x7f7cec0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:26.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.335+0000 7f7cf0a48700 1 --2- 172.21.15.3:0/1087825210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cec0a0a30 0x7f7cec0a0e00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc0098f0 tx=0x7f7cdc009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:26.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.335+0000 7f7cf0a48700 1 -- 172.21.15.3:0/1087825210 shutdown_connections 2023-10-03T10:14:26.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.335+0000 7f7cf0a48700 1 --2- 172.21.15.3:0/1087825210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cec0a0a30 0x7f7cec0a0e00 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.335+0000 7f7cf0a48700 1 --2- 172.21.15.3:0/1087825210 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cec138d80 0x7f7cec13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.335+0000 7f7cf0a48700 1 -- 172.21.15.3:0/1087825210 >> 172.21.15.3:0/1087825210 conn(0x7f7cec0a3050 msgr2=0x7f7cec0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:26.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.335+0000 7f7cf0a48700 1 -- 172.21.15.3:0/1087825210 shutdown_connections 2023-10-03T10:14:26.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.335+0000 7f7cf0a48700 1 -- 172.21.15.3:0/1087825210 wait complete. 2023-10-03T10:14:26.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.336+0000 7f7cf0a48700 1 Processor -- start 2023-10-03T10:14:26.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.336+0000 7f7cf0a48700 1 -- start start 2023-10-03T10:14:26.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.337+0000 7f7cf0a48700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cec09d630 0x7f7cec120700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:26.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.337+0000 7f7cf0a48700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cec0a0a30 0x7f7cec120c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:26.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.337+0000 7f7cf0a48700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cec138d80 0x7f7cec121180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:26.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.337+0000 7f7cf0a48700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cec123110 con 0x7f7cec138d80 2023-10-03T10:14:26.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.337+0000 7f7cf0a48700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cec122be0 con 0x7f7cec0a0a30 2023-10-03T10:14:26.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.337+0000 7f7cf0a48700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7cec122310 con 0x7f7cec09d630 2023-10-03T10:14:26.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.337+0000 7f7cea59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cec09d630 0x7f7cec120700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:26.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.337+0000 7f7ce3fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cec0a0a30 0x7f7cec120c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:26.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.337+0000 7f7cea59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cec09d630 0x7f7cec120700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42800/0 (socket says 172.21.15.3:42800) 2023-10-03T10:14:26.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.337+0000 7f7cea59c700 1 -- 172.21.15.3:0/1107893422 learned_addr learned my addr 172.21.15.3:0/1107893422 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:26.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.337+0000 7f7cead9d700 1 --2- 172.21.15.3:0/1107893422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cec138d80 0x7f7cec121180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:26.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.337+0000 7f7cea59c700 1 -- 172.21.15.3:0/1107893422 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cec0a0a30 msgr2=0x7f7cec120c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:26.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.337+0000 7f7cea59c700 1 --2- 172.21.15.3:0/1107893422 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cec0a0a30 0x7f7cec120c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.337+0000 7f7cea59c700 1 -- 172.21.15.3:0/1107893422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cec138d80 msgr2=0x7f7cec121180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:26.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.337+0000 7f7cea59c700 1 --2- 172.21.15.3:0/1107893422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cec138d80 0x7f7cec121180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.337+0000 7f7cea59c700 1 -- 172.21.15.3:0/1107893422 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce4009a70 con 0x7f7cec09d630 2023-10-03T10:14:26.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.337+0000 7f7cea59c700 1 --2- 172.21.15.3:0/1107893422 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cec09d630 0x7f7cec120700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc000c00 tx=0x7f7cdc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:26.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.338+0000 7f7ce2ffd700 1 -- 172.21.15.3:0/1107893422 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc0036a0 con 0x7f7cec09d630 2023-10-03T10:14:26.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.338+0000 7f7ce2ffd700 1 -- 172.21.15.3:0/1107893422 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cdc003800 con 0x7f7cec09d630 2023-10-03T10:14:26.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.338+0000 7f7cf0a48700 1 -- 172.21.15.3:0/1107893422 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cdc009550 con 0x7f7cec09d630 2023-10-03T10:14:26.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.338+0000 7f7ce2ffd700 1 -- 172.21.15.3:0/1107893422 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc003bb0 con 0x7f7cec09d630 2023-10-03T10:14:26.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.338+0000 7f7cf0a48700 1 -- 172.21.15.3:0/1107893422 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cec0a4ec0 con 0x7f7cec09d630 2023-10-03T10:14:26.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.339+0000 7f7ce2ffd700 1 -- 172.21.15.3:0/1107893422 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7cdc021070 con 0x7f7cec09d630 2023-10-03T10:14:26.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.339+0000 7f7cf0a48700 1 -- 172.21.15.3:0/1107893422 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cec0a0470 con 0x7f7cec09d630 2023-10-03T10:14:26.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.339+0000 7f7ce2ffd700 1 -- 172.21.15.3:0/1107893422 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7cdc040cd0 con 0x7f7cec09d630 2023-10-03T10:14:26.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.344+0000 7f7ce2ffd700 1 -- 172.21.15.3:0/1107893422 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7cdc02d3a0 con 0x7f7cec09d630 2023-10-03T10:14:26.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.444+0000 7f574f673700 1 -- 172.21.15.3:0/3827340934 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f574811c340 con 0x7f574809e000 2023-10-03T10:14:26.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.444+0000 7f573dffb700 1 -- 172.21.15.3:0/3827340934 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5738044060 con 0x7f574809e000 2023-10-03T10:14:26.445 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:26.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.447+0000 7f574f673700 1 -- 172.21.15.3:0/3827340934 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f574809e000 msgr2=0x7f5748121600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:26.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.447+0000 7f574f673700 1 --2- 172.21.15.3:0/3827340934 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f574809e000 0x7f5748121600 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f573800c460 tx=0x7f573800c770 comp rx=0 tx=0).stop 2023-10-03T10:14:26.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.447+0000 7f574f673700 1 -- 172.21.15.3:0/3827340934 shutdown_connections 2023-10-03T10:14:26.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.447+0000 7f574f673700 1 --2- 172.21.15.3:0/3827340934 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f574809e410 0x7f5748121b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.447+0000 7f574f673700 1 --2- 172.21.15.3:0/3827340934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f574809d6f0 0x7f57481210c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.447+0000 7f574f673700 1 --2- 172.21.15.3:0/3827340934 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f574809e000 0x7f5748121600 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.447+0000 7f574f673700 1 -- 172.21.15.3:0/3827340934 >> 172.21.15.3:0/3827340934 conn(0x7f5748066f50 msgr2=0x7f5748119480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:26.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.447+0000 7f574f673700 1 -- 172.21.15.3:0/3827340934 shutdown_connections 2023-10-03T10:14:26.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.447+0000 7f574f673700 1 -- 172.21.15.3:0/3827340934 wait complete. 2023-10-03T10:14:26.462 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:26.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.518+0000 7f40e4ec1700 1 -- 172.21.15.3:0/1160666652 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f40e0071d20 con 0x7f40e009f240 2023-10-03T10:14:26.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.518+0000 7f40ceffd700 1 -- 172.21.15.3:0/1160666652 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f40d4017020 con 0x7f40e009f240 2023-10-03T10:14:26.518 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:26.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.520+0000 7f40e4ec1700 1 -- 172.21.15.3:0/1160666652 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40e009f240 msgr2=0x7f40e01a4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:26.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.520+0000 7f40e4ec1700 1 --2- 172.21.15.3:0/1160666652 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40e009f240 0x7f40e01a4110 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f40d400e980 tx=0x7f40d400ec90 comp rx=0 tx=0).stop 2023-10-03T10:14:26.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.520+0000 7f40e4ec1700 1 -- 172.21.15.3:0/1160666652 shutdown_connections 2023-10-03T10:14:26.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.520+0000 7f40e4ec1700 1 --2- 172.21.15.3:0/1160666652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e009f650 0x7f40e01a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.520+0000 7f40e4ec1700 1 --2- 172.21.15.3:0/1160666652 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40e009e930 0x7f40e01a3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.520+0000 7f40e4ec1700 1 --2- 172.21.15.3:0/1160666652 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40e009f240 0x7f40e01a4110 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.520+0000 7f40e4ec1700 1 -- 172.21.15.3:0/1160666652 >> 172.21.15.3:0/1160666652 conn(0x7f40e00a4210 msgr2=0x7f40e01b8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:26.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.520+0000 7f40e4ec1700 1 -- 172.21.15.3:0/1160666652 shutdown_connections 2023-10-03T10:14:26.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.520+0000 7f40e4ec1700 1 -- 172.21.15.3:0/1160666652 wait complete. 2023-10-03T10:14:26.535 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:26.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.553+0000 7f7cf0a48700 1 -- 172.21.15.3:0/1107893422 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7cec0747f0 con 0x7f7cec09d630 2023-10-03T10:14:26.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.553+0000 7f7ce2ffd700 1 -- 172.21.15.3:0/1107893422 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7cdc029030 con 0x7f7cec09d630 2023-10-03T10:14:26.554 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:26.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.556+0000 7f7cf0a48700 1 -- 172.21.15.3:0/1107893422 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cec09d630 msgr2=0x7f7cec120700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:26.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.556+0000 7f7cf0a48700 1 --2- 172.21.15.3:0/1107893422 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cec09d630 0x7f7cec120700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc000c00 tx=0x7f7cdc004750 comp rx=0 tx=0).stop 2023-10-03T10:14:26.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.556+0000 7f7cf0a48700 1 -- 172.21.15.3:0/1107893422 shutdown_connections 2023-10-03T10:14:26.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.556+0000 7f7cf0a48700 1 --2- 172.21.15.3:0/1107893422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cec138d80 0x7f7cec121180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.556+0000 7f7cf0a48700 1 --2- 172.21.15.3:0/1107893422 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cec09d630 0x7f7cec120700 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.556+0000 7f7cf0a48700 1 --2- 172.21.15.3:0/1107893422 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cec0a0a30 0x7f7cec120c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.556+0000 7f7cf0a48700 1 -- 172.21.15.3:0/1107893422 >> 172.21.15.3:0/1107893422 conn(0x7f7cec0a3050 msgr2=0x7f7cec09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:26.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.556+0000 7f7cf0a48700 1 -- 172.21.15.3:0/1107893422 shutdown_connections 2023-10-03T10:14:26.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.556+0000 7f7cf0a48700 1 -- 172.21.15.3:0/1107893422 wait complete. 2023-10-03T10:14:26.570 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:26.670 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:26.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.813+0000 7f4fbb578700 1 Processor -- start 2023-10-03T10:14:26.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.814+0000 7f4fbb578700 1 -- start start 2023-10-03T10:14:26.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.814+0000 7f4fbb578700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fb409d6f0 0x7f4fb409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:26.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.814+0000 7f4fbb578700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fb409e410 0x7f4fb4134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:26.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.814+0000 7f4fbb578700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb409b120 con 0x7f4fb409e000 2023-10-03T10:14:26.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.814+0000 7f4fbb578700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb406b460 con 0x7f4fb409d6f0 2023-10-03T10:14:26.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.814+0000 7f4fbb578700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb409a7e0 con 0x7f4fb409e410 2023-10-03T10:14:26.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.814+0000 7f4fb9b15700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fb409e410 0x7f4fb4134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:26.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.814+0000 7f4fb9314700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fb409d6f0 0x7f4fb409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:26.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.814+0000 7f4fb8b13700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4fb409e000 0x7f4fb41339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50162/0 (socket says 172.21.15.3:50162) 2023-10-03T10:14:26.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.814+0000 7f4fb8b13700 1 -- 172.21.15.3:0/2306798309 learned_addr learned my addr 172.21.15.3:0/2306798309 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:26.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.815+0000 7f4fabfff700 1 -- 172.21.15.3:0/2306798309 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1856193965 0 0) 0x7f4fb409b120 con 0x7f4fb409e000 2023-10-03T10:14:26.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.815+0000 7f4fb9b15700 1 -- 172.21.15.3:0/2306798309 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fb409d6f0 msgr2=0x7f4fb409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:26.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.815+0000 7f4fb9b15700 1 --2- 172.21.15.3:0/2306798309 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fb409d6f0 0x7f4fb409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.815+0000 7f4fb9b15700 1 -- 172.21.15.3:0/2306798309 >> v1:172.21.15.3:6789/0 conn(0x7f4fb409e000 legacy=0x7f4fb41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:26.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.815+0000 7f4fb9b15700 1 -- 172.21.15.3:0/2306798309 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb409a500 con 0x7f4fb409e410 2023-10-03T10:14:26.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.815+0000 7f4fb9314700 1 --2- 172.21.15.3:0/2306798309 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fb409d6f0 0x7f4fb409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:26.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.815+0000 7f4fb9b15700 1 --2- 172.21.15.3:0/2306798309 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fb409e410 0x7f4fb4134100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4fac009df0 tx=0x7f4fac00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=626e20894e7d2fcf server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:26.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.815+0000 7f4fabfff700 1 -- 172.21.15.3:0/2306798309 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fac00e070 con 0x7f4fb409e410 2023-10-03T10:14:26.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.815+0000 7f4fabfff700 1 -- 172.21.15.3:0/2306798309 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fac00bae0 con 0x7f4fb409e410 2023-10-03T10:14:26.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.815+0000 7f4fabfff700 1 -- 172.21.15.3:0/2306798309 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fac00be90 con 0x7f4fb409e410 2023-10-03T10:14:26.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.816+0000 7f4fbb578700 1 -- 172.21.15.3:0/2306798309 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fb409e410 msgr2=0x7f4fb4134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:26.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.816+0000 7f4fbb578700 1 --2- 172.21.15.3:0/2306798309 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fb409e410 0x7f4fb4134100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4fac009df0 tx=0x7f4fac00b330 comp rx=0 tx=0).stop 2023-10-03T10:14:26.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.816+0000 7f4fbb578700 1 -- 172.21.15.3:0/2306798309 shutdown_connections 2023-10-03T10:14:26.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.816+0000 7f4fbb578700 1 --2- 172.21.15.3:0/2306798309 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fb409d6f0 0x7f4fb409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.816+0000 7f4fbb578700 1 --2- 172.21.15.3:0/2306798309 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fb409e410 0x7f4fb4134100 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.816+0000 7f4fbb578700 1 -- 172.21.15.3:0/2306798309 >> 172.21.15.3:0/2306798309 conn(0x7f4fb4066f50 msgr2=0x7f4fb409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:26.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.816+0000 7f4fbb578700 1 -- 172.21.15.3:0/2306798309 shutdown_connections 2023-10-03T10:14:26.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.816+0000 7f4fbb578700 1 -- 172.21.15.3:0/2306798309 wait complete. 2023-10-03T10:14:26.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.817+0000 7f4fbb578700 1 Processor -- start 2023-10-03T10:14:26.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.817+0000 7f4fbb578700 1 -- start start 2023-10-03T10:14:26.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.817+0000 7f4fbb578700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fb409d6f0 0x7f4fb4122300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:26.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.817+0000 7f4fbb578700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fb409e000 0x7f4fb4122840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:26.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.817+0000 7f4fbb578700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fb409e410 0x7f4fb4122d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:26.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.817+0000 7f4fbb578700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb41237d0 con 0x7f4fb409e000 2023-10-03T10:14:26.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.817+0000 7f4fbb578700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb41336d0 con 0x7f4fb409e410 2023-10-03T10:14:26.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.817+0000 7f4fbb578700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb4124da0 con 0x7f4fb409d6f0 2023-10-03T10:14:26.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.817+0000 7f4fb9314700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fb409d6f0 0x7f4fb4122300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:26.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.817+0000 7f4fb9314700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fb409d6f0 0x7f4fb4122300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42820/0 (socket says 172.21.15.3:42820) 2023-10-03T10:14:26.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.817+0000 7f4fb8b13700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fb409e000 0x7f4fb4122840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:26.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.817+0000 7f4fb9314700 1 -- 172.21.15.3:0/427873830 learned_addr learned my addr 172.21.15.3:0/427873830 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:26.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.817+0000 7f4fb9b15700 1 --2- 172.21.15.3:0/427873830 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fb409e410 0x7f4fb4122d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:26.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.818+0000 7f4fb9314700 1 -- 172.21.15.3:0/427873830 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fb409e410 msgr2=0x7f4fb4122d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:26.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.818+0000 7f4fb9314700 1 --2- 172.21.15.3:0/427873830 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fb409e410 0x7f4fb4122d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.818+0000 7f4fb9314700 1 -- 172.21.15.3:0/427873830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fb409e000 msgr2=0x7f4fb4122840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:26.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.818+0000 7f4fb9314700 1 --2- 172.21.15.3:0/427873830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fb409e000 0x7f4fb4122840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:26.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.818+0000 7f4fb9314700 1 -- 172.21.15.3:0/427873830 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb00094a0 con 0x7f4fb409d6f0 2023-10-03T10:14:26.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.818+0000 7f4fb8b13700 1 --2- 172.21.15.3:0/427873830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fb409e000 0x7f4fb4122840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:26.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.818+0000 7f4fb9314700 1 --2- 172.21.15.3:0/427873830 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fb409d6f0 0x7f4fb4122300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000ec10 tx=0x7f4fb000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:26.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.818+0000 7f4fa9ffb700 1 -- 172.21.15.3:0/427873830 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb000c850 con 0x7f4fb409d6f0 2023-10-03T10:14:26.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.818+0000 7f4fa9ffb700 1 -- 172.21.15.3:0/427873830 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb000c9b0 con 0x7f4fb409d6f0 2023-10-03T10:14:26.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.818+0000 7f4fbb578700 1 -- 172.21.15.3:0/427873830 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fac009a70 con 0x7f4fb409d6f0 2023-10-03T10:14:26.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.818+0000 7f4fa9ffb700 1 -- 172.21.15.3:0/427873830 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb000cd20 con 0x7f4fb409d6f0 2023-10-03T10:14:26.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.818+0000 7f4fbb578700 1 -- 172.21.15.3:0/427873830 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb40a2020 con 0x7f4fb409d6f0 2023-10-03T10:14:26.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.819+0000 7f4fa9ffb700 1 -- 172.21.15.3:0/427873830 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4fb0013470 con 0x7f4fb409d6f0 2023-10-03T10:14:26.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.819+0000 7f4fa9ffb700 1 -- 172.21.15.3:0/427873830 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4fb003f950 con 0x7f4fb409d6f0 2023-10-03T10:14:26.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.820+0000 7f4fbb578700 1 -- 172.21.15.3:0/427873830 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb411f030 con 0x7f4fb409d6f0 2023-10-03T10:14:26.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:26.824+0000 7f4fa9ffb700 1 -- 172.21.15.3:0/427873830 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4fb0013720 con 0x7f4fb409d6f0 2023-10-03T10:14:27.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.027+0000 7f4fbb578700 1 -- 172.21.15.3:0/427873830 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4fb4074ea0 con 0x7f4fb409d6f0 2023-10-03T10:14:27.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.028+0000 7f4fa9ffb700 1 -- 172.21.15.3:0/427873830 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4fb002b090 con 0x7f4fb409d6f0 2023-10-03T10:14:27.028 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:27.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.030+0000 7f4fbb578700 1 -- 172.21.15.3:0/427873830 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fb409d6f0 msgr2=0x7f4fb4122300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:27.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.030+0000 7f4fbb578700 1 --2- 172.21.15.3:0/427873830 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fb409d6f0 0x7f4fb4122300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000ec10 tx=0x7f4fb000ef20 comp rx=0 tx=0).stop 2023-10-03T10:14:27.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.030+0000 7f4fbb578700 1 -- 172.21.15.3:0/427873830 shutdown_connections 2023-10-03T10:14:27.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.030+0000 7f4fbb578700 1 --2- 172.21.15.3:0/427873830 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fb409e410 0x7f4fb4122d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:27.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.030+0000 7f4fbb578700 1 --2- 172.21.15.3:0/427873830 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fb409d6f0 0x7f4fb4122300 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:27.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.030+0000 7f4fbb578700 1 --2- 172.21.15.3:0/427873830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fb409e000 0x7f4fb4122840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:27.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.030+0000 7f4fbb578700 1 -- 172.21.15.3:0/427873830 >> 172.21.15.3:0/427873830 conn(0x7f4fb4066f50 msgr2=0x7f4fb4133e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:27.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.030+0000 7f4fbb578700 1 -- 172.21.15.3:0/427873830 shutdown_connections 2023-10-03T10:14:27.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.030+0000 7f4fbb578700 1 -- 172.21.15.3:0/427873830 wait complete. 2023-10-03T10:14:27.044 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:27.463 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:27.536 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:27.570 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:27.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.611+0000 7ff8f45b7700 1 Processor -- start 2023-10-03T10:14:27.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.612+0000 7ff8f45b7700 1 -- start start 2023-10-03T10:14:27.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.612+0000 7ff8f45b7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff8ec09d6f0 0x7ff8ec09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:27.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.612+0000 7ff8f45b7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8ec09e440 0x7ff8ec134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:27.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.612+0000 7ff8f45b7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8ec09b120 con 0x7ff8ec09e000 2023-10-03T10:14:27.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.612+0000 7ff8f45b7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8ec06b460 con 0x7ff8ec09d6f0 2023-10-03T10:14:27.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.612+0000 7ff8f45b7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec09a7e0 con 0x7ff8ec09e440 2023-10-03T10:14:27.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.613+0000 7ff8f2353700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff8ec09d6f0 0x7ff8ec09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:27.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.613+0000 7ff8f2b54700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8ec09e440 0x7ff8ec134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:27.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.613+0000 7ff8f2353700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff8ec09d6f0 0x7ff8ec09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32996/0 (socket says 172.21.15.3:32996) 2023-10-03T10:14:27.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.613+0000 7ff8f2353700 1 -- 172.21.15.3:0/1222822690 learned_addr learned my addr 172.21.15.3:0/1222822690 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:27.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.613+0000 7ff8f2b54700 1 -- 172.21.15.3:0/1222822690 >> v2:172.21.15.3:3300/0 conn(0x7ff8ec09d6f0 msgr2=0x7ff8ec09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:27.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.613+0000 7ff8f2b54700 1 --2- 172.21.15.3:0/1222822690 >> v2:172.21.15.3:3300/0 conn(0x7ff8ec09d6f0 0x7ff8ec09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:27.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.613+0000 7ff8f2b54700 1 -- 172.21.15.3:0/1222822690 >> v1:172.21.15.3:6789/0 conn(0x7ff8ec09e000 legacy=0x7ff8ec1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:27.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.613+0000 7ff8f2b54700 1 -- 172.21.15.3:0/1222822690 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ec09a500 con 0x7ff8ec09e440 2023-10-03T10:14:27.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.613+0000 7ff8f2b54700 1 --2- 172.21.15.3:0/1222822690 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8ec09e440 0x7ff8ec134100 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009dc0 tx=0x7ff8e400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3facbbc5cc42663 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:27.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.613+0000 7ff8f1351700 1 -- 172.21.15.3:0/1222822690 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e400e070 con 0x7ff8ec09e440 2023-10-03T10:14:27.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.613+0000 7ff8f1351700 1 -- 172.21.15.3:0/1222822690 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e400bae0 con 0x7ff8ec09e440 2023-10-03T10:14:27.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.613+0000 7ff8f1351700 1 -- 172.21.15.3:0/1222822690 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e400be90 con 0x7ff8ec09e440 2023-10-03T10:14:27.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.614+0000 7ff8f45b7700 1 -- 172.21.15.3:0/1222822690 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8ec09e440 msgr2=0x7ff8ec134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:27.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.614+0000 7ff8f45b7700 1 --2- 172.21.15.3:0/1222822690 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8ec09e440 0x7ff8ec134100 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009dc0 tx=0x7ff8e400b330 comp rx=0 tx=0).stop 2023-10-03T10:14:27.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.615+0000 7ff8f45b7700 1 -- 172.21.15.3:0/1222822690 shutdown_connections 2023-10-03T10:14:27.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.615+0000 7ff8f45b7700 1 --2- 172.21.15.3:0/1222822690 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8ec09e440 0x7ff8ec134100 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:27.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.615+0000 7ff8f45b7700 1 --2- 172.21.15.3:0/1222822690 >> v2:172.21.15.3:3300/0 conn(0x7ff8ec09d6f0 0x7ff8ec09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:27.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.615+0000 7ff8f45b7700 1 -- 172.21.15.3:0/1222822690 >> 172.21.15.3:0/1222822690 conn(0x7ff8ec066f50 msgr2=0x7ff8ec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:27.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.615+0000 7ff8f45b7700 1 -- 172.21.15.3:0/1222822690 shutdown_connections 2023-10-03T10:14:27.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.615+0000 7ff8f45b7700 1 -- 172.21.15.3:0/1222822690 wait complete. 2023-10-03T10:14:27.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.616+0000 7ff8f45b7700 1 Processor -- start 2023-10-03T10:14:27.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.616+0000 7ff8f45b7700 1 -- start start 2023-10-03T10:14:27.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.617+0000 7ff8f45b7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8ec09d6f0 0x7ff8ec119770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:27.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.617+0000 7ff8f45b7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8ec09e000 0x7ff8ec119cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:27.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.617+0000 7ff8f45b7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8ec09e440 0x7ff8ec11a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:27.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.617+0000 7ff8f45b7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec11cd30 con 0x7ff8ec09e000 2023-10-03T10:14:27.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.617+0000 7ff8f45b7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec1b94b0 con 0x7ff8ec09e440 2023-10-03T10:14:27.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.617+0000 7ff8f45b7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec11cf80 con 0x7ff8ec09d6f0 2023-10-03T10:14:27.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.617+0000 7ff8f1b52700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8ec09e000 0x7ff8ec119cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:27.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.617+0000 7ff8f2b54700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8ec09e440 0x7ff8ec11a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:27.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.617+0000 7ff8f1b52700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8ec09e000 0x7ff8ec119cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33002/0 (socket says 172.21.15.3:33002) 2023-10-03T10:14:27.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.617+0000 7ff8f1b52700 1 -- 172.21.15.3:0/233633367 learned_addr learned my addr 172.21.15.3:0/233633367 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:27.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.617+0000 7ff8f2353700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8ec09d6f0 0x7ff8ec119770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:27.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.618+0000 7ff8f2353700 1 -- 172.21.15.3:0/233633367 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8ec09e440 msgr2=0x7ff8ec11a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:27.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.618+0000 7ff8f2353700 1 --2- 172.21.15.3:0/233633367 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8ec09e440 0x7ff8ec11a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:27.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.618+0000 7ff8f2353700 1 -- 172.21.15.3:0/233633367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8ec09e000 msgr2=0x7ff8ec119cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:27.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.618+0000 7ff8f2353700 1 --2- 172.21.15.3:0/233633367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8ec09e000 0x7ff8ec119cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:27.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.618+0000 7ff8f2353700 1 -- 172.21.15.3:0/233633367 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e4009a70 con 0x7ff8ec09d6f0 2023-10-03T10:14:27.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.618+0000 7ff8f2b54700 1 --2- 172.21.15.3:0/233633367 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8ec09e440 0x7ff8ec11a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:27.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.618+0000 7ff8f1b52700 1 --2- 172.21.15.3:0/233633367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8ec09e000 0x7ff8ec119cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:27.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.618+0000 7ff8f2353700 1 --2- 172.21.15.3:0/233633367 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8ec09d6f0 0x7ff8ec119770 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff8e800e910 tx=0x7ff8e800ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:27.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.618+0000 7ff8e2ffd700 1 -- 172.21.15.3:0/233633367 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8013680 con 0x7ff8ec09d6f0 2023-10-03T10:14:27.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.618+0000 7ff8e2ffd700 1 -- 172.21.15.3:0/233633367 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e80137e0 con 0x7ff8ec09d6f0 2023-10-03T10:14:27.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.618+0000 7ff8f45b7700 1 -- 172.21.15.3:0/233633367 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8ec11c350 con 0x7ff8ec09d6f0 2023-10-03T10:14:27.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.618+0000 7ff8e2ffd700 1 -- 172.21.15.3:0/233633367 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8013b70 con 0x7ff8ec09d6f0 2023-10-03T10:14:27.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.618+0000 7ff8f45b7700 1 -- 172.21.15.3:0/233633367 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8ec11bce0 con 0x7ff8ec09d6f0 2023-10-03T10:14:27.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.619+0000 7ff8e2ffd700 1 -- 172.21.15.3:0/233633367 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff8e8013cd0 con 0x7ff8ec09d6f0 2023-10-03T10:14:27.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.619+0000 7ff8e2ffd700 1 -- 172.21.15.3:0/233633367 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff8e803fad0 con 0x7ff8ec09d6f0 2023-10-03T10:14:27.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.620+0000 7ff8f45b7700 1 -- 172.21.15.3:0/233633367 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8ec074ea0 con 0x7ff8ec09d6f0 2023-10-03T10:14:27.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.625+0000 7ff8e2ffd700 1 -- 172.21.15.3:0/233633367 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff8e8011db0 con 0x7ff8ec09d6f0 2023-10-03T10:14:27.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.716+0000 7f7d118d1700 1 Processor -- start 2023-10-03T10:14:27.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.716+0000 7f7d118d1700 1 -- start start 2023-10-03T10:14:27.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.717+0000 7f7d118d1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7d0c0a0a30 0x7f7d0c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:27.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.717+0000 7f7d118d1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d0c138d80 0x7f7d0c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:27.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.717+0000 7f7d118d1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d0c0a2a70 con 0x7f7d0c09d630 2023-10-03T10:14:27.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.717+0000 7f7d118d1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d0c0a01d0 con 0x7f7d0c0a0a30 2023-10-03T10:14:27.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.717+0000 7f7d118d1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c09e2b0 con 0x7f7d0c138d80 2023-10-03T10:14:27.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.717+0000 7f7d0b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d0c138d80 0x7f7d0c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:27.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.717+0000 7f7d0affd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7d0c0a0a30 0x7f7d0c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:27.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.717+0000 7f7d0b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d0c138d80 0x7f7d0c13b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42834/0 (socket says 172.21.15.3:42834) 2023-10-03T10:14:27.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.717+0000 7f7d0b7fe700 1 -- 172.21.15.3:0/3667420647 learned_addr learned my addr 172.21.15.3:0/3667420647 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:27.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.717+0000 7f7d0b7fe700 1 -- 172.21.15.3:0/3667420647 >> v2:172.21.15.3:3300/0 conn(0x7f7d0c0a0a30 msgr2=0x7f7d0c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:27.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.717+0000 7f7d0b7fe700 1 --2- 172.21.15.3:0/3667420647 >> v2:172.21.15.3:3300/0 conn(0x7f7d0c0a0a30 0x7f7d0c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:27.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.717+0000 7f7d0b7fe700 1 -- 172.21.15.3:0/3667420647 >> v1:172.21.15.3:6789/0 conn(0x7f7d0c09d630 legacy=0x7f7d0c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:27.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.717+0000 7f7d0b7fe700 1 -- 172.21.15.3:0/3667420647 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d0c09dc50 con 0x7f7d0c138d80 2023-10-03T10:14:27.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.717+0000 7f7d0affd700 1 --2- 172.21.15.3:0/3667420647 >> v2:172.21.15.3:3300/0 conn(0x7f7d0c0a0a30 0x7f7d0c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:27.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.717+0000 7f7d0b7fe700 1 --2- 172.21.15.3:0/3667420647 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d0c138d80 0x7f7d0c13b160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009e50 tx=0x7f7d0400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2aa2d2aba51da7b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:27.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.718+0000 7f7d09ffb700 1 -- 172.21.15.3:0/3667420647 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0400e070 con 0x7f7d0c138d80 2023-10-03T10:14:27.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.718+0000 7f7d09ffb700 1 -- 172.21.15.3:0/3667420647 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0400b910 con 0x7f7d0c138d80 2023-10-03T10:14:27.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.718+0000 7f7d09ffb700 1 -- 172.21.15.3:0/3667420647 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0400bcc0 con 0x7f7d0c138d80 2023-10-03T10:14:27.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.718+0000 7f7d118d1700 1 -- 172.21.15.3:0/3667420647 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d0c138d80 msgr2=0x7f7d0c13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:27.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.718+0000 7f7d118d1700 1 --2- 172.21.15.3:0/3667420647 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d0c138d80 0x7f7d0c13b160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009e50 tx=0x7f7d0400b330 comp rx=0 tx=0).stop 2023-10-03T10:14:27.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.718+0000 7f7d118d1700 1 -- 172.21.15.3:0/3667420647 shutdown_connections 2023-10-03T10:14:27.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.718+0000 7f7d118d1700 1 --2- 172.21.15.3:0/3667420647 >> v2:172.21.15.3:3300/0 conn(0x7f7d0c0a0a30 0x7f7d0c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:27.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.718+0000 7f7d118d1700 1 --2- 172.21.15.3:0/3667420647 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d0c138d80 0x7f7d0c13b160 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:27.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.718+0000 7f7d118d1700 1 -- 172.21.15.3:0/3667420647 >> 172.21.15.3:0/3667420647 conn(0x7f7d0c0a3050 msgr2=0x7f7d0c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:27.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.719+0000 7f7d118d1700 1 -- 172.21.15.3:0/3667420647 shutdown_connections 2023-10-03T10:14:27.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.719+0000 7f7d118d1700 1 -- 172.21.15.3:0/3667420647 wait complete. 2023-10-03T10:14:27.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.720+0000 7f7d118d1700 1 Processor -- start 2023-10-03T10:14:27.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.720+0000 7f7d118d1700 1 -- start start 2023-10-03T10:14:27.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.721+0000 7f7d118d1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d0c09d630 0x7f7d0c133a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:27.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.721+0000 7f7d118d1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d0c0a0a30 0x7f7d0c133f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:27.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.721+0000 7f7d118d1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d0c138d80 0x7f7d0c1344b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:27.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.721+0000 7f7d118d1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c136b90 con 0x7f7d0c09d630 2023-10-03T10:14:27.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.721+0000 7f7d118d1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c138090 con 0x7f7d0c0a0a30 2023-10-03T10:14:27.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.721+0000 7f7d118d1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c1374b0 con 0x7f7d0c138d80 2023-10-03T10:14:27.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.721+0000 7f7d0affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d0c09d630 0x7f7d0c133a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:27.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.721+0000 7f7d0a7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d0c0a0a30 0x7f7d0c133f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:27.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.721+0000 7f7d0affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d0c09d630 0x7f7d0c133a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33010/0 (socket says 172.21.15.3:33010) 2023-10-03T10:14:27.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.721+0000 7f7d0affd700 1 -- 172.21.15.3:0/2817315932 learned_addr learned my addr 172.21.15.3:0/2817315932 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:27.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.721+0000 7f7d0b7fe700 1 --2- 172.21.15.3:0/2817315932 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d0c138d80 0x7f7d0c1344b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:27.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.721+0000 7f7d0a7fc700 1 -- 172.21.15.3:0/2817315932 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d0c138d80 msgr2=0x7f7d0c1344b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:27.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.721+0000 7f7d0a7fc700 1 --2- 172.21.15.3:0/2817315932 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d0c138d80 0x7f7d0c1344b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:27.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.721+0000 7f7d0a7fc700 1 -- 172.21.15.3:0/2817315932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d0c09d630 msgr2=0x7f7d0c133a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:27.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.721+0000 7f7d0a7fc700 1 --2- 172.21.15.3:0/2817315932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d0c09d630 0x7f7d0c133a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:27.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.721+0000 7f7d0a7fc700 1 -- 172.21.15.3:0/2817315932 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc0094a0 con 0x7f7d0c0a0a30 2023-10-03T10:14:27.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.722+0000 7f7d0affd700 1 --2- 172.21.15.3:0/2817315932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d0c09d630 0x7f7d0c133a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:27.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.722+0000 7f7d0a7fc700 1 --2- 172.21.15.3:0/2817315932 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d0c0a0a30 0x7f7d0c133f70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7d00000f60 tx=0x7f7d0000c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:27.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.722+0000 7f7cf3fff700 1 -- 172.21.15.3:0/2817315932 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d000052d0 con 0x7f7d0c0a0a30 2023-10-03T10:14:27.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.722+0000 7f7cf3fff700 1 -- 172.21.15.3:0/2817315932 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d00005430 con 0x7f7d0c0a0a30 2023-10-03T10:14:27.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.722+0000 7f7d118d1700 1 -- 172.21.15.3:0/2817315932 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d04009b30 con 0x7f7d0c0a0a30 2023-10-03T10:14:27.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.722+0000 7f7cf3fff700 1 -- 172.21.15.3:0/2817315932 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d000038c0 con 0x7f7d0c0a0a30 2023-10-03T10:14:27.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.722+0000 7f7d118d1700 1 -- 172.21.15.3:0/2817315932 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d0c1371d0 con 0x7f7d0c0a0a30 2023-10-03T10:14:27.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.723+0000 7f7cf3fff700 1 -- 172.21.15.3:0/2817315932 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7d00003a20 con 0x7f7d0c0a0a30 2023-10-03T10:14:27.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.723+0000 7f7cf3fff700 1 -- 172.21.15.3:0/2817315932 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7d0003e990 con 0x7f7d0c0a0a30 2023-10-03T10:14:27.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.723+0000 7f7d118d1700 1 -- 172.21.15.3:0/2817315932 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d0c137030 con 0x7f7d0c0a0a30 2023-10-03T10:14:27.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.728+0000 7f7cf3fff700 1 -- 172.21.15.3:0/2817315932 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7d0c137030 con 0x7f7d0c0a0a30 2023-10-03T10:14:27.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.731+0000 7f3047775700 1 Processor -- start 2023-10-03T10:14:27.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.731+0000 7f3047775700 1 -- start start 2023-10-03T10:14:27.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.732+0000 7f3047775700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3040133420 0x7f30401337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:27.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.732+0000 7f3047775700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3040134260 0x7f30401306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:27.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.732+0000 7f3047775700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f304006b990 con 0x7f3040133e30 2023-10-03T10:14:27.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.732+0000 7f3047775700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30400a0590 con 0x7f3040133420 2023-10-03T10:14:27.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.732+0000 7f3047775700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f304009fb80 con 0x7f3040134260 2023-10-03T10:14:27.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.732+0000 7f3045511700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3040133420 0x7f30401337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:27.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.732+0000 7f3045d12700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3040134260 0x7f30401306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:27.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.732+0000 7f3045511700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3040133420 0x7f30401337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33026/0 (socket says 172.21.15.3:33026) 2023-10-03T10:14:27.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.732+0000 7f3045d12700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3040134260 0x7f30401306a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42852/0 (socket says 172.21.15.3:42852) 2023-10-03T10:14:27.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.732+0000 7f3045511700 1 -- 172.21.15.3:0/1770255824 learned_addr learned my addr 172.21.15.3:0/1770255824 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:27.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.733+0000 7f3045d12700 1 -- 172.21.15.3:0/1770255824 >> v2:172.21.15.3:3300/0 conn(0x7f3040133420 msgr2=0x7f30401337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:27.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.733+0000 7f3045d12700 1 --2- 172.21.15.3:0/1770255824 >> v2:172.21.15.3:3300/0 conn(0x7f3040133420 0x7f30401337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:27.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.733+0000 7f3045d12700 1 -- 172.21.15.3:0/1770255824 >> v1:172.21.15.3:6789/0 conn(0x7f3040133e30 legacy=0x7f304012ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:27.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.733+0000 7f3037fff700 1 -- 172.21.15.3:0/1770255824 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2918905172 0 0) 0x7f304006b990 con 0x7f3040133e30 2023-10-03T10:14:27.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.733+0000 7f3045d12700 1 -- 172.21.15.3:0/1770255824 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30400a0030 con 0x7f3040134260 2023-10-03T10:14:27.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.733+0000 7f3045511700 1 --2- 172.21.15.3:0/1770255824 >> v2:172.21.15.3:3300/0 conn(0x7f3040133420 0x7f30401337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:27.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.733+0000 7f3045d12700 1 --2- 172.21.15.3:0/1770255824 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3040134260 0x7f30401306a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3038009d90 tx=0x7f303800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a8be43402375262 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:27.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.733+0000 7f3037fff700 1 -- 172.21.15.3:0/1770255824 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303800e070 con 0x7f3040134260 2023-10-03T10:14:27.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.733+0000 7f3037fff700 1 -- 172.21.15.3:0/1770255824 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f303800bb20 con 0x7f3040134260 2023-10-03T10:14:27.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.733+0000 7f3037fff700 1 -- 172.21.15.3:0/1770255824 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3038012430 con 0x7f3040134260 2023-10-03T10:14:27.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.734+0000 7f3047775700 1 -- 172.21.15.3:0/1770255824 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3040134260 msgr2=0x7f30401306a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:27.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.734+0000 7f3047775700 1 --2- 172.21.15.3:0/1770255824 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3040134260 0x7f30401306a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3038009d90 tx=0x7f303800b330 comp rx=0 tx=0).stop 2023-10-03T10:14:27.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.734+0000 7f3047775700 1 -- 172.21.15.3:0/1770255824 shutdown_connections 2023-10-03T10:14:27.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.734+0000 7f3047775700 1 --2- 172.21.15.3:0/1770255824 >> v2:172.21.15.3:3300/0 conn(0x7f3040133420 0x7f30401337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:27.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.734+0000 7f3047775700 1 --2- 172.21.15.3:0/1770255824 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3040134260 0x7f30401306a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:27.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.734+0000 7f3047775700 1 -- 172.21.15.3:0/1770255824 >> 172.21.15.3:0/1770255824 conn(0x7f30400a4210 msgr2=0x7f30400a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:27.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.734+0000 7f3047775700 1 -- 172.21.15.3:0/1770255824 shutdown_connections 2023-10-03T10:14:27.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.734+0000 7f3047775700 1 -- 172.21.15.3:0/1770255824 wait complete. 2023-10-03T10:14:27.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.735+0000 7f3047775700 1 Processor -- start 2023-10-03T10:14:27.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.735+0000 7f3047775700 1 -- start start 2023-10-03T10:14:27.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.735+0000 7f3047775700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3040133420 0x7f304009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:27.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.735+0000 7f3047775700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3040133e30 0x7f304009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:27.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.735+0000 7f3047775700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3040134260 0x7f304009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:27.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.736+0000 7f3047775700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3040120600 con 0x7f3040134260 2023-10-03T10:14:27.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.736+0000 7f3047775700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304011fb60 con 0x7f3040133420 2023-10-03T10:14:27.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.736+0000 7f3047775700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3040131590 con 0x7f3040133e30 2023-10-03T10:14:27.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.736+0000 7f3045511700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3040133420 0x7f304009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:27.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.736+0000 7f3044d10700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3040133e30 0x7f304009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:27.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.736+0000 7f3045511700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3040133420 0x7f304009c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44654/0 (socket says 172.21.15.3:44654) 2023-10-03T10:14:27.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.736+0000 7f3045511700 1 -- 172.21.15.3:0/3661676126 learned_addr learned my addr 172.21.15.3:0/3661676126 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:27.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.736+0000 7f3045d12700 1 --2- 172.21.15.3:0/3661676126 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3040134260 0x7f304009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:27.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.736+0000 7f3044d10700 1 -- 172.21.15.3:0/3661676126 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3040133420 msgr2=0x7f304009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:27.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.736+0000 7f3044d10700 1 --2- 172.21.15.3:0/3661676126 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3040133420 0x7f304009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:27.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.736+0000 7f3044d10700 1 -- 172.21.15.3:0/3661676126 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3040134260 msgr2=0x7f304009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:27.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.736+0000 7f3044d10700 1 --2- 172.21.15.3:0/3661676126 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3040134260 0x7f304009cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:27.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.736+0000 7f3044d10700 1 -- 172.21.15.3:0/3661676126 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f303c009550 con 0x7f3040133e30 2023-10-03T10:14:27.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.736+0000 7f3044d10700 1 --2- 172.21.15.3:0/3661676126 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3040133e30 0x7f304009c9c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f303000c430 tx=0x7f303000c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:27.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.736+0000 7f3035ffb700 1 -- 172.21.15.3:0/3661676126 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30300054e0 con 0x7f3040133e30 2023-10-03T10:14:27.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.736+0000 7f3035ffb700 1 -- 172.21.15.3:0/3661676126 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f303000ce40 con 0x7f3040133e30 2023-10-03T10:14:27.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.736+0000 7f3047775700 1 -- 172.21.15.3:0/3661676126 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3038009a70 con 0x7f3040133e30 2023-10-03T10:14:27.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.736+0000 7f3035ffb700 1 -- 172.21.15.3:0/3661676126 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30300038c0 con 0x7f3040133e30 2023-10-03T10:14:27.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.737+0000 7f3047775700 1 -- 172.21.15.3:0/3661676126 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3040120170 con 0x7f3040133e30 2023-10-03T10:14:27.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.737+0000 7f3035ffb700 1 -- 172.21.15.3:0/3661676126 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3030003a20 con 0x7f3040133e30 2023-10-03T10:14:27.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.738+0000 7f3035ffb700 1 -- 172.21.15.3:0/3661676126 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f303003e9e0 con 0x7f3040133e30 2023-10-03T10:14:27.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.738+0000 7f3047775700 1 -- 172.21.15.3:0/3661676126 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f304011ffb0 con 0x7f3040133e30 2023-10-03T10:14:27.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.743+0000 7f3035ffb700 1 -- 172.21.15.3:0/3661676126 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3030017080 con 0x7f3040133e30 2023-10-03T10:14:27.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.887+0000 7ff8f45b7700 1 -- 172.21.15.3:0/233633367 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff8ec09f420 con 0x7ff8ec09d6f0 2023-10-03T10:14:27.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.887+0000 7ff8e2ffd700 1 -- 172.21.15.3:0/233633367 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff8e8017020 con 0x7ff8ec09d6f0 2023-10-03T10:14:27.888 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:27.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.891+0000 7ff8f45b7700 1 -- 172.21.15.3:0/233633367 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8ec09d6f0 msgr2=0x7ff8ec119770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:27.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.891+0000 7ff8f45b7700 1 --2- 172.21.15.3:0/233633367 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8ec09d6f0 0x7ff8ec119770 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff8e800e910 tx=0x7ff8e800ec20 comp rx=0 tx=0).stop 2023-10-03T10:14:27.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.891+0000 7ff8f45b7700 1 -- 172.21.15.3:0/233633367 shutdown_connections 2023-10-03T10:14:27.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.891+0000 7ff8f45b7700 1 --2- 172.21.15.3:0/233633367 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8ec09e440 0x7ff8ec11a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:27.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.891+0000 7ff8f45b7700 1 --2- 172.21.15.3:0/233633367 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8ec09d6f0 0x7ff8ec119770 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:27.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.891+0000 7ff8f45b7700 1 --2- 172.21.15.3:0/233633367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8ec09e000 0x7ff8ec119cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:27.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.891+0000 7ff8f45b7700 1 -- 172.21.15.3:0/233633367 >> 172.21.15.3:0/233633367 conn(0x7ff8ec066f50 msgr2=0x7ff8ec1bad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:27.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.891+0000 7ff8f45b7700 1 -- 172.21.15.3:0/233633367 shutdown_connections 2023-10-03T10:14:27.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.891+0000 7ff8f45b7700 1 -- 172.21.15.3:0/233633367 wait complete. 2023-10-03T10:14:27.906 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:27.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.949+0000 7f3047775700 1 -- 172.21.15.3:0/3661676126 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f304009e120 con 0x7f3040133e30 2023-10-03T10:14:27.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.949+0000 7f3035ffb700 1 -- 172.21.15.3:0/3661676126 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3030003cd0 con 0x7f3040133e30 2023-10-03T10:14:27.950 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:27.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.949+0000 7f7d118d1700 1 -- 172.21.15.3:0/2817315932 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7d0c137030 con 0x7f7d0c0a0a30 2023-10-03T10:14:27.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.950+0000 7f7cf3fff700 1 -- 172.21.15.3:0/2817315932 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7d0c137030 con 0x7f7d0c0a0a30 2023-10-03T10:14:27.951 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:27.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.951+0000 7f3047775700 1 -- 172.21.15.3:0/3661676126 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3040133e30 msgr2=0x7f304009c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:27.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.951+0000 7f3047775700 1 --2- 172.21.15.3:0/3661676126 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3040133e30 0x7f304009c9c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f303000c430 tx=0x7f303000c740 comp rx=0 tx=0).stop 2023-10-03T10:14:27.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.952+0000 7f3047775700 1 -- 172.21.15.3:0/3661676126 shutdown_connections 2023-10-03T10:14:27.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.952+0000 7f3047775700 1 --2- 172.21.15.3:0/3661676126 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3040134260 0x7f304009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:27.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.952+0000 7f3047775700 1 --2- 172.21.15.3:0/3661676126 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3040133420 0x7f304009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:27.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.952+0000 7f3047775700 1 --2- 172.21.15.3:0/3661676126 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3040133e30 0x7f304009c9c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:27.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.952+0000 7f3047775700 1 -- 172.21.15.3:0/3661676126 >> 172.21.15.3:0/3661676126 conn(0x7f30400a4210 msgr2=0x7f3040130480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:27.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.952+0000 7f3047775700 1 -- 172.21.15.3:0/3661676126 shutdown_connections 2023-10-03T10:14:27.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.952+0000 7f3047775700 1 -- 172.21.15.3:0/3661676126 wait complete. 2023-10-03T10:14:27.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.952+0000 7f7d118d1700 1 -- 172.21.15.3:0/2817315932 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d0c0a0a30 msgr2=0x7f7d0c133f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:27.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.952+0000 7f7d118d1700 1 --2- 172.21.15.3:0/2817315932 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d0c0a0a30 0x7f7d0c133f70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7d00000f60 tx=0x7f7d0000c730 comp rx=0 tx=0).stop 2023-10-03T10:14:27.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.952+0000 7f7d118d1700 1 -- 172.21.15.3:0/2817315932 shutdown_connections 2023-10-03T10:14:27.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.952+0000 7f7d118d1700 1 --2- 172.21.15.3:0/2817315932 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d0c138d80 0x7f7d0c1344b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:27.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.952+0000 7f7d118d1700 1 --2- 172.21.15.3:0/2817315932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d0c09d630 0x7f7d0c133a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:27.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.952+0000 7f7d118d1700 1 --2- 172.21.15.3:0/2817315932 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d0c0a0a30 0x7f7d0c133f70 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:27.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.952+0000 7f7d118d1700 1 -- 172.21.15.3:0/2817315932 >> 172.21.15.3:0/2817315932 conn(0x7f7d0c0a3050 msgr2=0x7f7d0c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:27.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.952+0000 7f7d118d1700 1 -- 172.21.15.3:0/2817315932 shutdown_connections 2023-10-03T10:14:27.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:27.952+0000 7f7d118d1700 1 -- 172.21.15.3:0/2817315932 wait complete. 2023-10-03T10:14:27.967 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:27.968 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:28.045 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:28.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.188+0000 7f6ed82a7700 1 Processor -- start 2023-10-03T10:14:28.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.188+0000 7f6ed82a7700 1 -- start start 2023-10-03T10:14:28.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.189+0000 7f6ed82a7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ed009e1d0 0x7f6ed0135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:28.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.189+0000 7f6ed82a7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ed0135e50 0x7f6ed0138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:28.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.189+0000 7f6ed82a7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ed009f130 con 0x7f6ed009d6f0 2023-10-03T10:14:28.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.189+0000 7f6ed82a7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed009f520 con 0x7f6ed0135e50 2023-10-03T10:14:28.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.189+0000 7f6ed82a7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed009e930 con 0x7f6ed009e1d0 2023-10-03T10:14:28.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.189+0000 7f6ed5842700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ed009e1d0 0x7f6ed0135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:28.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.189+0000 7f6ed6844700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ed0135e50 0x7f6ed0138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:28.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.189+0000 7f6ed6043700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6ed009d6f0 0x7f6ed009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50210/0 (socket says 172.21.15.3:50210) 2023-10-03T10:14:28.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.189+0000 7f6ed6043700 1 -- 172.21.15.3:0/2456361222 learned_addr learned my addr 172.21.15.3:0/2456361222 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:28.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.190+0000 7f6ed5041700 1 -- 172.21.15.3:0/2456361222 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1236632589 0 0) 0x7f6ed009f130 con 0x7f6ed009d6f0 2023-10-03T10:14:28.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.190+0000 7f6ed5041700 1 -- 172.21.15.3:0/2456361222 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ebc0035b0 con 0x7f6ed009d6f0 2023-10-03T10:14:28.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.190+0000 7f6ed6844700 1 -- 172.21.15.3:0/2456361222 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ed009e1d0 msgr2=0x7f6ed0135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:28.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.190+0000 7f6ed6844700 1 --2- 172.21.15.3:0/2456361222 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ed009e1d0 0x7f6ed0135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:28.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.190+0000 7f6ed6844700 1 -- 172.21.15.3:0/2456361222 >> v1:172.21.15.3:6789/0 conn(0x7f6ed009d6f0 legacy=0x7f6ed009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:28.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.190+0000 7f6ed6844700 1 -- 172.21.15.3:0/2456361222 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed009b2d0 con 0x7f6ed0135e50 2023-10-03T10:14:28.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.190+0000 7f6ed5842700 1 --2- 172.21.15.3:0/2456361222 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ed009e1d0 0x7f6ed0135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:28.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.190+0000 7f6ed6844700 1 --2- 172.21.15.3:0/2456361222 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ed0135e50 0x7f6ed0138230 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8009dc0 tx=0x7f6ec800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aadbe50869c64a68 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:28.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.190+0000 7f6ed5041700 1 -- 172.21.15.3:0/2456361222 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec800e070 con 0x7f6ed0135e50 2023-10-03T10:14:28.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.190+0000 7f6ed5041700 1 -- 172.21.15.3:0/2456361222 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ec800bb20 con 0x7f6ed0135e50 2023-10-03T10:14:28.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.190+0000 7f6ed5041700 1 -- 172.21.15.3:0/2456361222 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec800be90 con 0x7f6ed0135e50 2023-10-03T10:14:28.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.191+0000 7f6ed82a7700 1 -- 172.21.15.3:0/2456361222 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ed0135e50 msgr2=0x7f6ed0138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:28.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.191+0000 7f6ed82a7700 1 --2- 172.21.15.3:0/2456361222 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ed0135e50 0x7f6ed0138230 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8009dc0 tx=0x7f6ec800b330 comp rx=0 tx=0).stop 2023-10-03T10:14:28.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.191+0000 7f6ed82a7700 1 -- 172.21.15.3:0/2456361222 shutdown_connections 2023-10-03T10:14:28.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.191+0000 7f6ed82a7700 1 --2- 172.21.15.3:0/2456361222 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ed0135e50 0x7f6ed0138230 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:28.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.191+0000 7f6ed82a7700 1 --2- 172.21.15.3:0/2456361222 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ed009e1d0 0x7f6ed0135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:28.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.191+0000 7f6ed82a7700 1 -- 172.21.15.3:0/2456361222 >> 172.21.15.3:0/2456361222 conn(0x7f6ed0066f50 msgr2=0x7f6ed009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:28.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.191+0000 7f6ed82a7700 1 -- 172.21.15.3:0/2456361222 shutdown_connections 2023-10-03T10:14:28.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.191+0000 7f6ed82a7700 1 -- 172.21.15.3:0/2456361222 wait complete. 2023-10-03T10:14:28.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.192+0000 7f6ed82a7700 1 Processor -- start 2023-10-03T10:14:28.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.192+0000 7f6ed82a7700 1 -- start start 2023-10-03T10:14:28.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.192+0000 7f6ed82a7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ed009d6f0 0x7f6ed01c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:28.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.192+0000 7f6ed82a7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ed009e1d0 0x7f6ed01c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:28.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.192+0000 7f6ed82a7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ed0135e50 0x7f6ed01c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:28.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.192+0000 7f6ed82a7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed01321c0 con 0x7f6ed009e1d0 2023-10-03T10:14:28.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.192+0000 7f6ed82a7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed0070620 con 0x7f6ed0135e50 2023-10-03T10:14:28.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.192+0000 7f6ed82a7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed0070430 con 0x7f6ed009d6f0 2023-10-03T10:14:28.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.192+0000 7f6ed5842700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ed009e1d0 0x7f6ed01c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:28.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.192+0000 7f6ed6844700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ed0135e50 0x7f6ed01c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:28.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.192+0000 7f6ed6844700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ed0135e50 0x7f6ed01c8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44666/0 (socket says 172.21.15.3:44666) 2023-10-03T10:14:28.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.192+0000 7f6ed5842700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ed009e1d0 0x7f6ed01c87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33054/0 (socket says 172.21.15.3:33054) 2023-10-03T10:14:28.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.192+0000 7f6ed6844700 1 -- 172.21.15.3:0/92478406 learned_addr learned my addr 172.21.15.3:0/92478406 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:28.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.193+0000 7f6ed6043700 1 --2- 172.21.15.3:0/92478406 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ed009d6f0 0x7f6ed01c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:28.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.193+0000 7f6ed5842700 1 -- 172.21.15.3:0/92478406 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ed009d6f0 msgr2=0x7f6ed01c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:28.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.193+0000 7f6ed5842700 1 --2- 172.21.15.3:0/92478406 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ed009d6f0 0x7f6ed01c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:28.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.193+0000 7f6ed5842700 1 -- 172.21.15.3:0/92478406 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ed0135e50 msgr2=0x7f6ed01c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:28.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.193+0000 7f6ed5842700 1 --2- 172.21.15.3:0/92478406 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ed0135e50 0x7f6ed01c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:28.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.193+0000 7f6ed5842700 1 -- 172.21.15.3:0/92478406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec00094a0 con 0x7f6ed009e1d0 2023-10-03T10:14:28.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.193+0000 7f6ed6844700 1 --2- 172.21.15.3:0/92478406 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ed0135e50 0x7f6ed01c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:28.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.193+0000 7f6ed5842700 1 --2- 172.21.15.3:0/92478406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ed009e1d0 0x7f6ed01c87c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000e9e0 tx=0x7f6ec000ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:28.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.193+0000 7f6ec6ffd700 1 -- 172.21.15.3:0/92478406 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec000ee80 con 0x7f6ed009e1d0 2023-10-03T10:14:28.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.193+0000 7f6ec6ffd700 1 -- 172.21.15.3:0/92478406 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ec000c850 con 0x7f6ed009e1d0 2023-10-03T10:14:28.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.193+0000 7f6ec6ffd700 1 -- 172.21.15.3:0/92478406 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec000cc00 con 0x7f6ed009e1d0 2023-10-03T10:14:28.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.193+0000 7f6ed82a7700 1 -- 172.21.15.3:0/92478406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec8009a70 con 0x7f6ed009e1d0 2023-10-03T10:14:28.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.193+0000 7f6ed82a7700 1 -- 172.21.15.3:0/92478406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ed01232c0 con 0x7f6ed009e1d0 2023-10-03T10:14:28.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.194+0000 7f6ec6ffd700 1 -- 172.21.15.3:0/92478406 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6ec000cd60 con 0x7f6ed009e1d0 2023-10-03T10:14:28.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.194+0000 7f6ed82a7700 1 -- 172.21.15.3:0/92478406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ed0074ea0 con 0x7f6ed009e1d0 2023-10-03T10:14:28.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.194+0000 7f6ec6ffd700 1 -- 172.21.15.3:0/92478406 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6ec003f8e0 con 0x7f6ed009e1d0 2023-10-03T10:14:28.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.200+0000 7f6ec6ffd700 1 -- 172.21.15.3:0/92478406 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6ec0017070 con 0x7f6ed009e1d0 2023-10-03T10:14:28.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.407+0000 7f6ed82a7700 1 -- 172.21.15.3:0/92478406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6ed01232c0 con 0x7f6ed009e1d0 2023-10-03T10:14:28.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.408+0000 7f6ec6ffd700 1 -- 172.21.15.3:0/92478406 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6ed01232c0 con 0x7f6ed009e1d0 2023-10-03T10:14:28.408 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:28.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.410+0000 7f6ed82a7700 1 -- 172.21.15.3:0/92478406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ed009e1d0 msgr2=0x7f6ed01c87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:28.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.410+0000 7f6ed82a7700 1 --2- 172.21.15.3:0/92478406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ed009e1d0 0x7f6ed01c87c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000e9e0 tx=0x7f6ec000ecf0 comp rx=0 tx=0).stop 2023-10-03T10:14:28.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.410+0000 7f6ed82a7700 1 -- 172.21.15.3:0/92478406 shutdown_connections 2023-10-03T10:14:28.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.410+0000 7f6ed82a7700 1 --2- 172.21.15.3:0/92478406 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ed0135e50 0x7f6ed01c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:28.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.410+0000 7f6ed82a7700 1 --2- 172.21.15.3:0/92478406 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ed009d6f0 0x7f6ed01c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:28.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.410+0000 7f6ed82a7700 1 --2- 172.21.15.3:0/92478406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ed009e1d0 0x7f6ed01c87c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:28.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.410+0000 7f6ed82a7700 1 -- 172.21.15.3:0/92478406 >> 172.21.15.3:0/92478406 conn(0x7f6ed0066f50 msgr2=0x7f6ed0134810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:28.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.410+0000 7f6ed82a7700 1 -- 172.21.15.3:0/92478406 shutdown_connections 2023-10-03T10:14:28.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:28.410+0000 7f6ed82a7700 1 -- 172.21.15.3:0/92478406 wait complete. 2023-10-03T10:14:28.424 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:28.907 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:28.968 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:28.969 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:29.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.059+0000 7f9c3c2d7700 1 Processor -- start 2023-10-03T10:14:29.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.059+0000 7f9c3c2d7700 1 -- start start 2023-10-03T10:14:29.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.060+0000 7f9c3c2d7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9c341337f0 0x7f9c34138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:29.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.060+0000 7f9c3c2d7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c34138dc0 0x7f9c3413b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:29.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.060+0000 7f9c3c2d7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c3406b990 con 0x7f9c34133420 2023-10-03T10:14:29.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.060+0000 7f9c3c2d7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c3409b0e0 con 0x7f9c341337f0 2023-10-03T10:14:29.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.060+0000 7f9c3c2d7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c34133c20 con 0x7f9c34138dc0 2023-10-03T10:14:29.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.060+0000 7f9c3a874700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c34138dc0 0x7f9c3413b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:29.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.060+0000 7f9c39872700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9c341337f0 0x7f9c34138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:29.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.060+0000 7f9c39872700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9c341337f0 0x7f9c34138880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33060/0 (socket says 172.21.15.3:33060) 2023-10-03T10:14:29.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.060+0000 7f9c39872700 1 -- 172.21.15.3:0/4220454063 learned_addr learned my addr 172.21.15.3:0/4220454063 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:29.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.060+0000 7f9c3a874700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c34138dc0 0x7f9c3413b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42902/0 (socket says 172.21.15.3:42902) 2023-10-03T10:14:29.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.060+0000 7f9c39872700 1 -- 172.21.15.3:0/4220454063 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c34138dc0 msgr2=0x7f9c3413b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:29.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.060+0000 7f9c39872700 1 --2- 172.21.15.3:0/4220454063 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c34138dc0 0x7f9c3413b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.060+0000 7f9c39872700 1 -- 172.21.15.3:0/4220454063 >> v1:172.21.15.3:6789/0 conn(0x7f9c34133420 legacy=0x7f9c34133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:29.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.060+0000 7f9c39872700 1 -- 172.21.15.3:0/4220454063 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c340a0590 con 0x7f9c341337f0 2023-10-03T10:14:29.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.060+0000 7f9c3a874700 1 --2- 172.21.15.3:0/4220454063 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c34138dc0 0x7f9c3413b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:29.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.060+0000 7f9c39872700 1 --2- 172.21.15.3:0/4220454063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c341337f0 0x7f9c34138880 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9c240098f0 tx=0x7f9c24009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=effe61b70b8cac35 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:29.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.061+0000 7f9c39071700 1 -- 172.21.15.3:0/4220454063 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c24011830 con 0x7f9c341337f0 2023-10-03T10:14:29.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.061+0000 7f9c39071700 1 -- 172.21.15.3:0/4220454063 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c24011990 con 0x7f9c341337f0 2023-10-03T10:14:29.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.061+0000 7f9c39071700 1 -- 172.21.15.3:0/4220454063 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c24011d20 con 0x7f9c341337f0 2023-10-03T10:14:29.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.061+0000 7f9c3c2d7700 1 -- 172.21.15.3:0/4220454063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c341337f0 msgr2=0x7f9c34138880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:29.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.061+0000 7f9c3c2d7700 1 --2- 172.21.15.3:0/4220454063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c341337f0 0x7f9c34138880 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9c240098f0 tx=0x7f9c24009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:29.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.061+0000 7f9c3c2d7700 1 -- 172.21.15.3:0/4220454063 shutdown_connections 2023-10-03T10:14:29.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.061+0000 7f9c3c2d7700 1 --2- 172.21.15.3:0/4220454063 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c34138dc0 0x7f9c3413b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.061+0000 7f9c3c2d7700 1 --2- 172.21.15.3:0/4220454063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c341337f0 0x7f9c34138880 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.061+0000 7f9c3c2d7700 1 -- 172.21.15.3:0/4220454063 >> 172.21.15.3:0/4220454063 conn(0x7f9c340a4210 msgr2=0x7f9c340a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:29.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.061+0000 7f9c3c2d7700 1 -- 172.21.15.3:0/4220454063 shutdown_connections 2023-10-03T10:14:29.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.061+0000 7f9c3c2d7700 1 -- 172.21.15.3:0/4220454063 wait complete. 2023-10-03T10:14:29.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.062+0000 7f9c3c2d7700 1 Processor -- start 2023-10-03T10:14:29.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.062+0000 7f9c3c2d7700 1 -- start start 2023-10-03T10:14:29.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.063+0000 7f9c3c2d7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c34133420 0x7f9c3409c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:29.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.063+0000 7f9c3c2d7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c341337f0 0x7f9c3409cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:29.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.063+0000 7f9c3c2d7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c34138dc0 0x7f9c3409d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:29.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.063+0000 7f9c3c2d7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3412df60 con 0x7f9c34138dc0 2023-10-03T10:14:29.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.063+0000 7f9c3c2d7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3412f950 con 0x7f9c341337f0 2023-10-03T10:14:29.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.063+0000 7f9c3c2d7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c34130780 con 0x7f9c34133420 2023-10-03T10:14:29.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.063+0000 7f9c39872700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c341337f0 0x7f9c3409cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:29.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.063+0000 7f9c39872700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c341337f0 0x7f9c3409cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44682/0 (socket says 172.21.15.3:44682) 2023-10-03T10:14:29.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.063+0000 7f9c39872700 1 -- 172.21.15.3:0/4279434361 learned_addr learned my addr 172.21.15.3:0/4279434361 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:29.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.063+0000 7f9c3a874700 1 --2- 172.21.15.3:0/4279434361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c34138dc0 0x7f9c3409d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:29.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.063+0000 7f9c39872700 1 -- 172.21.15.3:0/4279434361 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c34133420 msgr2=0x7f9c3409c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:29.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.063+0000 7f9c3a073700 1 --2- 172.21.15.3:0/4279434361 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c34133420 0x7f9c3409c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:29.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.064+0000 7f9c39872700 1 --2- 172.21.15.3:0/4279434361 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c34133420 0x7f9c3409c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.064+0000 7f9c39872700 1 -- 172.21.15.3:0/4279434361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c34138dc0 msgr2=0x7f9c3409d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:29.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.064+0000 7f9c39872700 1 --2- 172.21.15.3:0/4279434361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c34138dc0 0x7f9c3409d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.064+0000 7f9c39872700 1 -- 172.21.15.3:0/4279434361 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2c009a70 con 0x7f9c341337f0 2023-10-03T10:14:29.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.064+0000 7f9c3a874700 1 --2- 172.21.15.3:0/4279434361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c34138dc0 0x7f9c3409d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:29.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.064+0000 7f9c3a073700 1 --2- 172.21.15.3:0/4279434361 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c34133420 0x7f9c3409c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:14:29.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.064+0000 7f9c39872700 1 --2- 172.21.15.3:0/4279434361 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c341337f0 0x7f9c3409cf20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009c40 tx=0x7f9c24004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:29.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.064+0000 7f9c2affd700 1 -- 172.21.15.3:0/4279434361 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c240036a0 con 0x7f9c341337f0 2023-10-03T10:14:29.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.064+0000 7f9c2affd700 1 -- 172.21.15.3:0/4279434361 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c24003800 con 0x7f9c341337f0 2023-10-03T10:14:29.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.064+0000 7f9c3c2d7700 1 -- 172.21.15.3:0/4279434361 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c24009550 con 0x7f9c341337f0 2023-10-03T10:14:29.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.064+0000 7f9c2affd700 1 -- 172.21.15.3:0/4279434361 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c24003b90 con 0x7f9c341337f0 2023-10-03T10:14:29.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.064+0000 7f9c3c2d7700 1 -- 172.21.15.3:0/4279434361 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c34130d40 con 0x7f9c341337f0 2023-10-03T10:14:29.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.065+0000 7f9c3c2d7700 1 -- 172.21.15.3:0/4279434361 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c34074ea0 con 0x7f9c341337f0 2023-10-03T10:14:29.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.065+0000 7f9c2affd700 1 -- 172.21.15.3:0/4279434361 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9c24021070 con 0x7f9c341337f0 2023-10-03T10:14:29.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.066+0000 7f9c2affd700 1 -- 172.21.15.3:0/4279434361 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9c24040b70 con 0x7f9c341337f0 2023-10-03T10:14:29.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.074+0000 7f9c2affd700 1 -- 172.21.15.3:0/4279434361 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9c24010380 con 0x7f9c341337f0 2023-10-03T10:14:29.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.114+0000 7f162009f700 1 Processor -- start 2023-10-03T10:14:29.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.114+0000 7f162009f700 1 -- start start 2023-10-03T10:14:29.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.115+0000 7f162009f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f161809e850 0x7f161809ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:29.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.115+0000 7f162009f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f161809f160 0x7f1618133b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:29.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.115+0000 7f162009f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f16181340c0 0x7f16181344e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:29.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.115+0000 7f162009f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f161809ae60 con 0x7f161809f160 2023-10-03T10:14:29.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.115+0000 7f162009f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161806b990 con 0x7f161809e850 2023-10-03T10:14:29.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.115+0000 7f162009f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1618134a20 con 0x7f16181340c0 2023-10-03T10:14:29.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.115+0000 7f161d63a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f161809f160 0x7f1618133b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:29.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.115+0000 7f161e63c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f16181340c0 0x7f16181344e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:29.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.115+0000 7f161d63a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f161809f160 0x7f1618133b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33064/0 (socket says 172.21.15.3:33064) 2023-10-03T10:14:29.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.115+0000 7f161e63c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f16181340c0 0x7f16181344e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42914/0 (socket says 172.21.15.3:42914) 2023-10-03T10:14:29.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.115+0000 7f161d63a700 1 -- 172.21.15.3:0/3103161963 learned_addr learned my addr 172.21.15.3:0/3103161963 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:29.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.115+0000 7f161de3b700 1 --2- 172.21.15.3:0/3103161963 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f161809e850 0x7f161809ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:29.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.115+0000 7f161e63c700 1 -- 172.21.15.3:0/3103161963 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f161809e850 msgr2=0x7f161809ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:29.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.115+0000 7f161e63c700 1 --2- 172.21.15.3:0/3103161963 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f161809e850 0x7f161809ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.115+0000 7f161e63c700 1 -- 172.21.15.3:0/3103161963 >> v2:172.21.15.3:3300/0 conn(0x7f161809f160 msgr2=0x7f1618133b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:29.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.115+0000 7f161e63c700 1 --2- 172.21.15.3:0/3103161963 >> v2:172.21.15.3:3300/0 conn(0x7f161809f160 0x7f1618133b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.115+0000 7f161e63c700 1 -- 172.21.15.3:0/3103161963 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161806bc50 con 0x7f16181340c0 2023-10-03T10:14:29.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.116+0000 7f161e63c700 1 --2- 172.21.15.3:0/3103161963 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f16181340c0 0x7f16181344e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1610009d90 tx=0x7f161000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a9baab9795dfbb8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:29.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.116+0000 7fdf6e923700 1 Processor -- start 2023-10-03T10:14:29.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.116+0000 7f161ce39700 1 -- 172.21.15.3:0/3103161963 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161000e070 con 0x7f16181340c0 2023-10-03T10:14:29.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.116+0000 7fdf6e923700 1 -- start start 2023-10-03T10:14:29.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.116+0000 7f161ce39700 1 -- 172.21.15.3:0/3103161963 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161000bb20 con 0x7f16181340c0 2023-10-03T10:14:29.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.116+0000 7f161ce39700 1 -- 172.21.15.3:0/3103161963 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1610012430 con 0x7f16181340c0 2023-10-03T10:14:29.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.116+0000 7fdf6e923700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdf680a0a30 0x7fdf680a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:29.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.116+0000 7fdf6e923700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf680a1340 0x7fdf68133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:29.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.116+0000 7fdf6e923700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf68133fa0 0x7fdf68134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:29.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.116+0000 7fdf6e923700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf6809edd0 con 0x7fdf680a0a30 2023-10-03T10:14:29.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.116+0000 7fdf6e923700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6809e1f0 con 0x7fdf68133fa0 2023-10-03T10:14:29.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.116+0000 7fdf6e923700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6809dc50 con 0x7fdf680a1340 2023-10-03T10:14:29.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.117+0000 7fdf6cec0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf68133fa0 0x7fdf68134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:29.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.117+0000 7fdf677fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf680a1340 0x7fdf68133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:29.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.117+0000 7fdf6cec0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf68133fa0 0x7fdf68134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44696/0 (socket says 172.21.15.3:44696) 2023-10-03T10:14:29.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.117+0000 7fdf6cec0700 1 -- 172.21.15.3:0/2688852766 learned_addr learned my addr 172.21.15.3:0/2688852766 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:29.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.117+0000 7fdf677fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf680a1340 0x7fdf68133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42928/0 (socket says 172.21.15.3:42928) 2023-10-03T10:14:29.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.117+0000 7f162009f700 1 -- 172.21.15.3:0/3103161963 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f16181340c0 msgr2=0x7f16181344e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:29.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.117+0000 7f162009f700 1 --2- 172.21.15.3:0/3103161963 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f16181340c0 0x7f16181344e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1610009d90 tx=0x7f161000b330 comp rx=0 tx=0).stop 2023-10-03T10:14:29.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.117+0000 7f162009f700 1 -- 172.21.15.3:0/3103161963 shutdown_connections 2023-10-03T10:14:29.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.117+0000 7f162009f700 1 --2- 172.21.15.3:0/3103161963 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f16181340c0 0x7f16181344e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.117+0000 7f162009f700 1 --2- 172.21.15.3:0/3103161963 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f161809e850 0x7f161809ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.117+0000 7f162009f700 1 --2- 172.21.15.3:0/3103161963 >> v2:172.21.15.3:3300/0 conn(0x7f161809f160 0x7f1618133b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.117+0000 7f162009f700 1 -- 172.21.15.3:0/3103161963 >> 172.21.15.3:0/3103161963 conn(0x7f16180a3f70 msgr2=0x7f16180a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:29.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.117+0000 7fdf677fe700 1 -- 172.21.15.3:0/2688852766 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf68133fa0 msgr2=0x7fdf68134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:29.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.117+0000 7fdf677fe700 1 --2- 172.21.15.3:0/2688852766 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf68133fa0 0x7fdf68134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.117+0000 7fdf677fe700 1 -- 172.21.15.3:0/2688852766 >> v2:172.21.15.3:3300/0 conn(0x7fdf680a0a30 msgr2=0x7fdf680a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:29.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.117+0000 7fdf677fe700 1 --2- 172.21.15.3:0/2688852766 >> v2:172.21.15.3:3300/0 conn(0x7fdf680a0a30 0x7fdf680a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.117+0000 7fdf677fe700 1 -- 172.21.15.3:0/2688852766 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf680a1fe0 con 0x7fdf680a1340 2023-10-03T10:14:29.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.117+0000 7f162009f700 1 -- 172.21.15.3:0/3103161963 shutdown_connections 2023-10-03T10:14:29.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.117+0000 7f162009f700 1 -- 172.21.15.3:0/3103161963 wait complete. 2023-10-03T10:14:29.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.117+0000 7fdf677fe700 1 --2- 172.21.15.3:0/2688852766 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf680a1340 0x7fdf68133a60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c009870 tx=0x7fdf5c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=965735b315b2847f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:29.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.117+0000 7fdf66ffd700 1 -- 172.21.15.3:0/2688852766 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c011830 con 0x7fdf680a1340 2023-10-03T10:14:29.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.117+0000 7fdf66ffd700 1 -- 172.21.15.3:0/2688852766 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5c011990 con 0x7fdf680a1340 2023-10-03T10:14:29.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.118+0000 7fdf66ffd700 1 -- 172.21.15.3:0/2688852766 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c011d20 con 0x7fdf680a1340 2023-10-03T10:14:29.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.118+0000 7f162009f700 1 Processor -- start 2023-10-03T10:14:29.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.118+0000 7f162009f700 1 -- start start 2023-10-03T10:14:29.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.118+0000 7fdf6e923700 1 -- 172.21.15.3:0/2688852766 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf680a1340 msgr2=0x7fdf68133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:29.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.118+0000 7fdf6e923700 1 --2- 172.21.15.3:0/2688852766 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf680a1340 0x7fdf68133a60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c009870 tx=0x7fdf5c009b80 comp rx=0 tx=0).stop 2023-10-03T10:14:29.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.119+0000 7fdf6e923700 1 -- 172.21.15.3:0/2688852766 shutdown_connections 2023-10-03T10:14:29.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.119+0000 7fdf6e923700 1 --2- 172.21.15.3:0/2688852766 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf68133fa0 0x7fdf68134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.119+0000 7fdf6e923700 1 --2- 172.21.15.3:0/2688852766 >> v2:172.21.15.3:3300/0 conn(0x7fdf680a0a30 0x7fdf680a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.119+0000 7fdf6e923700 1 --2- 172.21.15.3:0/2688852766 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf680a1340 0x7fdf68133a60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.119+0000 7fdf6e923700 1 -- 172.21.15.3:0/2688852766 >> 172.21.15.3:0/2688852766 conn(0x7fdf680a6550 msgr2=0x7fdf680a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:29.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.119+0000 7fdf6e923700 1 -- 172.21.15.3:0/2688852766 shutdown_connections 2023-10-03T10:14:29.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.119+0000 7fdf6e923700 1 -- 172.21.15.3:0/2688852766 wait complete. 2023-10-03T10:14:29.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.119+0000 7f162009f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f161809e850 0x7f16181224f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:29.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.119+0000 7f162009f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f161809f160 0x7f1618122a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:29.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.119+0000 7f162009f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f16181340c0 0x7f1618122f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:29.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.119+0000 7f162009f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1618126010 con 0x7f161809e850 2023-10-03T10:14:29.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.119+0000 7f162009f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1618125b00 con 0x7f161809f160 2023-10-03T10:14:29.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.119+0000 7f162009f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1618124f90 con 0x7f16181340c0 2023-10-03T10:14:29.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.119+0000 7f161d63a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f161809f160 0x7f1618122a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:29.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.119+0000 7f161d63a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f161809f160 0x7f1618122a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44708/0 (socket says 172.21.15.3:44708) 2023-10-03T10:14:29.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.119+0000 7f161d63a700 1 -- 172.21.15.3:0/3884202883 learned_addr learned my addr 172.21.15.3:0/3884202883 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:29.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.119+0000 7f161e63c700 1 --2- 172.21.15.3:0/3884202883 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f16181340c0 0x7f1618122f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:29.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.120+0000 7f161de3b700 1 --2- 172.21.15.3:0/3884202883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f161809e850 0x7f16181224f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:29.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.120+0000 7f161e63c700 1 -- 172.21.15.3:0/3884202883 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f161809f160 msgr2=0x7f1618122a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:29.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.120+0000 7f161e63c700 1 --2- 172.21.15.3:0/3884202883 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f161809f160 0x7f1618122a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.120+0000 7f161e63c700 1 -- 172.21.15.3:0/3884202883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f161809e850 msgr2=0x7f16181224f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:29.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.120+0000 7f161e63c700 1 --2- 172.21.15.3:0/3884202883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f161809e850 0x7f16181224f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.120+0000 7f161e63c700 1 -- 172.21.15.3:0/3884202883 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1610009a70 con 0x7f16181340c0 2023-10-03T10:14:29.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.120+0000 7f161de3b700 1 --2- 172.21.15.3:0/3884202883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f161809e850 0x7f16181224f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:14:29.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.120+0000 7f161e63c700 1 --2- 172.21.15.3:0/3884202883 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f16181340c0 0x7f1618122f70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1610016000 tx=0x7f1610004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:29.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.120+0000 7fdf6e923700 1 Processor -- start 2023-10-03T10:14:29.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.120+0000 7f160e7fc700 1 -- 172.21.15.3:0/3884202883 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1610012430 con 0x7f16181340c0 2023-10-03T10:14:29.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.120+0000 7f160e7fc700 1 -- 172.21.15.3:0/3884202883 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1610020070 con 0x7f16181340c0 2023-10-03T10:14:29.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.120+0000 7f162009f700 1 -- 172.21.15.3:0/3884202883 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f161806cd30 con 0x7f16181340c0 2023-10-03T10:14:29.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.120+0000 7f160e7fc700 1 -- 172.21.15.3:0/3884202883 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1610004280 con 0x7f16181340c0 2023-10-03T10:14:29.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.120+0000 7f162009f700 1 -- 172.21.15.3:0/3884202883 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16180a1740 con 0x7f16181340c0 2023-10-03T10:14:29.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.120+0000 7fdf6e923700 1 -- start start 2023-10-03T10:14:29.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.121+0000 7f160e7fc700 1 -- 172.21.15.3:0/3884202883 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f16100043e0 con 0x7f16181340c0 2023-10-03T10:14:29.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.121+0000 7fdf6e923700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf680a0a30 0x7fdf681207f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:29.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.121+0000 7fdf6e923700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf680a1340 0x7fdf68120d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:29.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.121+0000 7fdf6e923700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf68133fa0 0x7fdf68121270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:29.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.121+0000 7fdf6e923700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf68121cc0 con 0x7fdf680a1340 2023-10-03T10:14:29.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.121+0000 7fdf6e923700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf68124090 con 0x7fdf680a0a30 2023-10-03T10:14:29.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.121+0000 7fdf6e923700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf681223b0 con 0x7fdf68133fa0 2023-10-03T10:14:29.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.121+0000 7fdf677fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf680a1340 0x7fdf68120d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:29.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.121+0000 7f160e7fc700 1 -- 172.21.15.3:0/3884202883 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1610022070 con 0x7f16181340c0 2023-10-03T10:14:29.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.121+0000 7fdf677fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf680a1340 0x7fdf68120d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33090/0 (socket says 172.21.15.3:33090) 2023-10-03T10:14:29.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.121+0000 7fdf677fe700 1 -- 172.21.15.3:0/546226322 learned_addr learned my addr 172.21.15.3:0/546226322 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:29.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.121+0000 7fdf6cec0700 1 --2- 172.21.15.3:0/546226322 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf68133fa0 0x7fdf68121270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:29.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.122+0000 7f162009f700 1 -- 172.21.15.3:0/3884202883 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16180747f0 con 0x7f16181340c0 2023-10-03T10:14:29.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.122+0000 7fdf67fff700 1 --2- 172.21.15.3:0/546226322 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf680a0a30 0x7fdf681207f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:29.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.122+0000 7fdf6cec0700 1 -- 172.21.15.3:0/546226322 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf680a0a30 msgr2=0x7fdf681207f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:29.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.122+0000 7fdf6cec0700 1 --2- 172.21.15.3:0/546226322 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf680a0a30 0x7fdf681207f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.122+0000 7fdf6cec0700 1 -- 172.21.15.3:0/546226322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf680a1340 msgr2=0x7fdf68120d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:29.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.122+0000 7fdf6cec0700 1 --2- 172.21.15.3:0/546226322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf680a1340 0x7fdf68120d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.122+0000 7fdf6cec0700 1 -- 172.21.15.3:0/546226322 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5c0094a0 con 0x7fdf68133fa0 2023-10-03T10:14:29.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.122+0000 7fdf67fff700 1 --2- 172.21.15.3:0/546226322 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf680a0a30 0x7fdf681207f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:14:29.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.122+0000 7fdf677fe700 1 --2- 172.21.15.3:0/546226322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf680a1340 0x7fdf68120d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:29.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.122+0000 7fdf6cec0700 1 --2- 172.21.15.3:0/546226322 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf68133fa0 0x7fdf68121270 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdf60009d90 tx=0x7fdf6000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:29.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.128+0000 7fdf64ff9700 1 -- 172.21.15.3:0/546226322 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf60003c20 con 0x7fdf68133fa0 2023-10-03T10:14:29.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.128+0000 7fdf64ff9700 1 -- 172.21.15.3:0/546226322 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6000f040 con 0x7fdf68133fa0 2023-10-03T10:14:29.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.128+0000 7fdf6e923700 1 -- 172.21.15.3:0/546226322 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf680a0220 con 0x7fdf68133fa0 2023-10-03T10:14:29.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.128+0000 7fdf64ff9700 1 -- 172.21.15.3:0/546226322 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6000c930 con 0x7fdf68133fa0 2023-10-03T10:14:29.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.128+0000 7fdf6e923700 1 -- 172.21.15.3:0/546226322 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf68123110 con 0x7fdf68133fa0 2023-10-03T10:14:29.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.128+0000 7f160e7fc700 1 -- 172.21.15.3:0/3884202883 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f161001c400 con 0x7f16181340c0 2023-10-03T10:14:29.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.128+0000 7fdf6e923700 1 -- 172.21.15.3:0/546226322 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf68074ea0 con 0x7fdf68133fa0 2023-10-03T10:14:29.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.128+0000 7fdf64ff9700 1 -- 172.21.15.3:0/546226322 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdf6000ca90 con 0x7fdf68133fa0 2023-10-03T10:14:29.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.129+0000 7fdf64ff9700 1 -- 172.21.15.3:0/546226322 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fdf6003fa50 con 0x7fdf68133fa0 2023-10-03T10:14:29.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.135+0000 7fdf64ff9700 1 -- 172.21.15.3:0/546226322 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdf6001c770 con 0x7fdf68133fa0 2023-10-03T10:14:29.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.285+0000 7f9c3c2d7700 1 -- 172.21.15.3:0/4279434361 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9c3406b250 con 0x7f9c341337f0 2023-10-03T10:14:29.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.285+0000 7f9c2affd700 1 -- 172.21.15.3:0/4279434361 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9c24015020 con 0x7f9c341337f0 2023-10-03T10:14:29.285 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:29.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.287+0000 7f9c3c2d7700 1 -- 172.21.15.3:0/4279434361 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c341337f0 msgr2=0x7f9c3409cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:29.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.287+0000 7f9c3c2d7700 1 --2- 172.21.15.3:0/4279434361 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c341337f0 0x7f9c3409cf20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009c40 tx=0x7f9c24004750 comp rx=0 tx=0).stop 2023-10-03T10:14:29.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.287+0000 7f9c3c2d7700 1 -- 172.21.15.3:0/4279434361 shutdown_connections 2023-10-03T10:14:29.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.287+0000 7f9c3c2d7700 1 --2- 172.21.15.3:0/4279434361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c34138dc0 0x7f9c3409d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.287+0000 7f9c3c2d7700 1 --2- 172.21.15.3:0/4279434361 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c34133420 0x7f9c3409c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.287+0000 7f9c3c2d7700 1 --2- 172.21.15.3:0/4279434361 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c341337f0 0x7f9c3409cf20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.287+0000 7f9c3c2d7700 1 -- 172.21.15.3:0/4279434361 >> 172.21.15.3:0/4279434361 conn(0x7f9c340a4210 msgr2=0x7f9c340a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:29.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.287+0000 7f9c3c2d7700 1 -- 172.21.15.3:0/4279434361 shutdown_connections 2023-10-03T10:14:29.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.287+0000 7f9c3c2d7700 1 -- 172.21.15.3:0/4279434361 wait complete. 2023-10-03T10:14:29.302 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:29.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.359+0000 7f162009f700 1 -- 172.21.15.3:0/3884202883 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1618072840 con 0x7f16181340c0 2023-10-03T10:14:29.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.359+0000 7f160e7fc700 1 -- 172.21.15.3:0/3884202883 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f16100263e0 con 0x7f16181340c0 2023-10-03T10:14:29.360 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:29.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.362+0000 7f162009f700 1 -- 172.21.15.3:0/3884202883 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f16181340c0 msgr2=0x7f1618122f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:29.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.362+0000 7f162009f700 1 --2- 172.21.15.3:0/3884202883 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f16181340c0 0x7f1618122f70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1610016000 tx=0x7f1610004ad0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.362+0000 7f162009f700 1 -- 172.21.15.3:0/3884202883 shutdown_connections 2023-10-03T10:14:29.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.362+0000 7f162009f700 1 --2- 172.21.15.3:0/3884202883 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f16181340c0 0x7f1618122f70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.363+0000 7f162009f700 1 --2- 172.21.15.3:0/3884202883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f161809e850 0x7f16181224f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.363+0000 7f162009f700 1 --2- 172.21.15.3:0/3884202883 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f161809f160 0x7f1618122a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.363+0000 7f162009f700 1 -- 172.21.15.3:0/3884202883 >> 172.21.15.3:0/3884202883 conn(0x7f16180a3f70 msgr2=0x7f1618070930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:29.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.363+0000 7f162009f700 1 -- 172.21.15.3:0/3884202883 shutdown_connections 2023-10-03T10:14:29.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.363+0000 7f162009f700 1 -- 172.21.15.3:0/3884202883 wait complete. 2023-10-03T10:14:29.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.366+0000 7fdf6e923700 1 -- 172.21.15.3:0/546226322 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fdf680a28e0 con 0x7fdf68133fa0 2023-10-03T10:14:29.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.366+0000 7fdf64ff9700 1 -- 172.21.15.3:0/546226322 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdf600073f0 con 0x7fdf68133fa0 2023-10-03T10:14:29.366 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:29.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.370+0000 7fdf6e923700 1 -- 172.21.15.3:0/546226322 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf68133fa0 msgr2=0x7fdf68121270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:29.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.370+0000 7fdf6e923700 1 --2- 172.21.15.3:0/546226322 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf68133fa0 0x7fdf68121270 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdf60009d90 tx=0x7fdf6000c3a0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.370+0000 7fdf6e923700 1 -- 172.21.15.3:0/546226322 shutdown_connections 2023-10-03T10:14:29.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.370+0000 7fdf6e923700 1 --2- 172.21.15.3:0/546226322 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf68133fa0 0x7fdf68121270 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.370+0000 7fdf6e923700 1 --2- 172.21.15.3:0/546226322 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf680a0a30 0x7fdf681207f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.370+0000 7fdf6e923700 1 --2- 172.21.15.3:0/546226322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf680a1340 0x7fdf68120d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.370+0000 7fdf6e923700 1 -- 172.21.15.3:0/546226322 >> 172.21.15.3:0/546226322 conn(0x7fdf680a6550 msgr2=0x7fdf681ba800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:29.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.371+0000 7fdf6e923700 1 -- 172.21.15.3:0/546226322 shutdown_connections 2023-10-03T10:14:29.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.371+0000 7fdf6e923700 1 -- 172.21.15.3:0/546226322 wait complete. 2023-10-03T10:14:29.379 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:29.385 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:29.425 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:29.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.565+0000 7f00cca20700 1 Processor -- start 2023-10-03T10:14:29.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.565+0000 7f00cca20700 1 -- start start 2023-10-03T10:14:29.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.566+0000 7f00cca20700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f00c809e930 0x7f00c809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:29.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.566+0000 7f00cca20700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f00c81342f0 0x7f00c813be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:29.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.566+0000 7f00cca20700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00c806c9f0 con 0x7f00c809f240 2023-10-03T10:14:29.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.566+0000 7f00cca20700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c806c670 con 0x7f00c809e930 2023-10-03T10:14:29.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.566+0000 7f00cca20700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f00c806bc90 con 0x7f00c81342f0 2023-10-03T10:14:29.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.566+0000 7f00c659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f00c809e930 0x7f00c809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:29.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.566+0000 7f00c5d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f00c809f240 0x7f00c8133be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50222/0 (socket says 172.21.15.3:50222) 2023-10-03T10:14:29.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.566+0000 7f00c6d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f00c81342f0 0x7f00c813be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:29.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.566+0000 7f00c659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f00c809e930 0x7f00c809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44728/0 (socket says 172.21.15.3:44728) 2023-10-03T10:14:29.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.566+0000 7f00c5d9b700 1 -- 172.21.15.3:0/4212259782 learned_addr learned my addr 172.21.15.3:0/4212259782 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:29.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.566+0000 7f00c559a700 1 -- 172.21.15.3:0/4212259782 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3453100710 0 0) 0x7f00b0001110 con 0x7f00c809f240 2023-10-03T10:14:29.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.566+0000 7f00c6d9d700 1 -- 172.21.15.3:0/4212259782 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f00c809e930 msgr2=0x7f00c809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:29.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.566+0000 7f00c6d9d700 1 --2- 172.21.15.3:0/4212259782 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f00c809e930 0x7f00c809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.566+0000 7f00c6d9d700 1 -- 172.21.15.3:0/4212259782 >> v1:172.21.15.3:6789/0 conn(0x7f00c809f240 legacy=0x7f00c8133be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:29.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.566+0000 7f00c6d9d700 1 -- 172.21.15.3:0/4212259782 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c806d860 con 0x7f00c81342f0 2023-10-03T10:14:29.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.566+0000 7f00c659c700 1 --2- 172.21.15.3:0/4212259782 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f00c809e930 0x7f00c809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:29.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.567+0000 7f00c6d9d700 1 --2- 172.21.15.3:0/4212259782 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f00c81342f0 0x7f00c813be40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f00c0009df0 tx=0x7f00c000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93d9163cd32407d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:29.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.567+0000 7f00c559a700 1 -- 172.21.15.3:0/4212259782 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c000e070 con 0x7f00c81342f0 2023-10-03T10:14:29.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.567+0000 7f00c559a700 1 -- 172.21.15.3:0/4212259782 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c000bae0 con 0x7f00c81342f0 2023-10-03T10:14:29.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.567+0000 7f00c559a700 1 -- 172.21.15.3:0/4212259782 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c000be90 con 0x7f00c81342f0 2023-10-03T10:14:29.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.567+0000 7f00cca20700 1 -- 172.21.15.3:0/4212259782 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f00c81342f0 msgr2=0x7f00c813be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:29.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.567+0000 7f00cca20700 1 --2- 172.21.15.3:0/4212259782 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f00c81342f0 0x7f00c813be40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f00c0009df0 tx=0x7f00c000b330 comp rx=0 tx=0).stop 2023-10-03T10:14:29.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.567+0000 7f00cca20700 1 -- 172.21.15.3:0/4212259782 shutdown_connections 2023-10-03T10:14:29.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.567+0000 7f00cca20700 1 --2- 172.21.15.3:0/4212259782 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f00c809e930 0x7f00c809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.567+0000 7f00cca20700 1 --2- 172.21.15.3:0/4212259782 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f00c81342f0 0x7f00c813be40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.567+0000 7f00cca20700 1 -- 172.21.15.3:0/4212259782 >> 172.21.15.3:0/4212259782 conn(0x7f00c80a4210 msgr2=0x7f00c80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:29.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.567+0000 7f00cca20700 1 -- 172.21.15.3:0/4212259782 shutdown_connections 2023-10-03T10:14:29.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.567+0000 7f00cca20700 1 -- 172.21.15.3:0/4212259782 wait complete. 2023-10-03T10:14:29.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.569+0000 7f00cca20700 1 Processor -- start 2023-10-03T10:14:29.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.569+0000 7f00cca20700 1 -- start start 2023-10-03T10:14:29.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.569+0000 7f00cca20700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f00c809e930 0x7f00c81229d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:29.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.569+0000 7f00cca20700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00c809f240 0x7f00c8122f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:29.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.569+0000 7f00cca20700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f00c81342f0 0x7f00c8123450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:29.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.569+0000 7f00cca20700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c8124590 con 0x7f00c809f240 2023-10-03T10:14:29.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.569+0000 7f00cca20700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c8133510 con 0x7f00c809e930 2023-10-03T10:14:29.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.569+0000 7f00cca20700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f00c8125f00 con 0x7f00c81342f0 2023-10-03T10:14:29.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.569+0000 7f00c659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f00c809e930 0x7f00c81229d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:29.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.569+0000 7f00c659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f00c809e930 0x7f00c81229d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44730/0 (socket says 172.21.15.3:44730) 2023-10-03T10:14:29.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.569+0000 7f00c5d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00c809f240 0x7f00c8122f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:29.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.569+0000 7f00c659c700 1 -- 172.21.15.3:0/3607829560 learned_addr learned my addr 172.21.15.3:0/3607829560 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:29.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.569+0000 7f00c6d9d700 1 --2- 172.21.15.3:0/3607829560 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f00c81342f0 0x7f00c8123450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:29.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.570+0000 7f00c659c700 1 -- 172.21.15.3:0/3607829560 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f00c81342f0 msgr2=0x7f00c8123450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:29.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.570+0000 7f00c659c700 1 --2- 172.21.15.3:0/3607829560 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f00c81342f0 0x7f00c8123450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.570+0000 7f00c659c700 1 -- 172.21.15.3:0/3607829560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00c809f240 msgr2=0x7f00c8122f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:29.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.570+0000 7f00c659c700 1 --2- 172.21.15.3:0/3607829560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00c809f240 0x7f00c8122f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.570+0000 7f00c659c700 1 -- 172.21.15.3:0/3607829560 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b80094a0 con 0x7f00c809e930 2023-10-03T10:14:29.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.570+0000 7f00c5d9b700 1 --2- 172.21.15.3:0/3607829560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00c809f240 0x7f00c8122f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:29.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.570+0000 7f00c659c700 1 --2- 172.21.15.3:0/3607829560 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f00c809e930 0x7f00c81229d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f00b800ec10 tx=0x7f00b800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:29.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.570+0000 7f00beffd700 1 -- 172.21.15.3:0/3607829560 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b800c850 con 0x7f00c809e930 2023-10-03T10:14:29.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.570+0000 7f00beffd700 1 -- 172.21.15.3:0/3607829560 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b800c9b0 con 0x7f00c809e930 2023-10-03T10:14:29.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.570+0000 7f00cca20700 1 -- 172.21.15.3:0/3607829560 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c0009a70 con 0x7f00c809e930 2023-10-03T10:14:29.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.570+0000 7f00beffd700 1 -- 172.21.15.3:0/3607829560 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b800cd20 con 0x7f00c809e930 2023-10-03T10:14:29.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.570+0000 7f00cca20700 1 -- 172.21.15.3:0/3607829560 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c806cbb0 con 0x7f00c809e930 2023-10-03T10:14:29.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.571+0000 7f00cca20700 1 -- 172.21.15.3:0/3607829560 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c8074ea0 con 0x7f00c809e930 2023-10-03T10:14:29.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.571+0000 7f00beffd700 1 -- 172.21.15.3:0/3607829560 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f00b8013470 con 0x7f00c809e930 2023-10-03T10:14:29.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.572+0000 7f00beffd700 1 -- 172.21.15.3:0/3607829560 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f00b803ff90 con 0x7f00c809e930 2023-10-03T10:14:29.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.577+0000 7f00beffd700 1 -- 172.21.15.3:0/3607829560 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f00b8013720 con 0x7f00c809e930 2023-10-03T10:14:29.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.780+0000 7f00cca20700 1 -- 172.21.15.3:0/3607829560 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f00c8126280 con 0x7f00c809e930 2023-10-03T10:14:29.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.780+0000 7f00beffd700 1 -- 172.21.15.3:0/3607829560 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f00b8021070 con 0x7f00c809e930 2023-10-03T10:14:29.780 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:29.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.783+0000 7f00cca20700 1 -- 172.21.15.3:0/3607829560 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f00c809e930 msgr2=0x7f00c81229d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:29.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.783+0000 7f00cca20700 1 --2- 172.21.15.3:0/3607829560 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f00c809e930 0x7f00c81229d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f00b800ec10 tx=0x7f00b800ef20 comp rx=0 tx=0).stop 2023-10-03T10:14:29.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.783+0000 7f00cca20700 1 -- 172.21.15.3:0/3607829560 shutdown_connections 2023-10-03T10:14:29.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.783+0000 7f00cca20700 1 --2- 172.21.15.3:0/3607829560 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f00c81342f0 0x7f00c8123450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.783+0000 7f00cca20700 1 --2- 172.21.15.3:0/3607829560 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f00c809e930 0x7f00c81229d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.783+0000 7f00cca20700 1 --2- 172.21.15.3:0/3607829560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00c809f240 0x7f00c8122f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:29.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.783+0000 7f00cca20700 1 -- 172.21.15.3:0/3607829560 >> 172.21.15.3:0/3607829560 conn(0x7f00c80a4210 msgr2=0x7f00c80710a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:29.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.783+0000 7f00cca20700 1 -- 172.21.15.3:0/3607829560 shutdown_connections 2023-10-03T10:14:29.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:29.783+0000 7f00cca20700 1 -- 172.21.15.3:0/3607829560 wait complete. 2023-10-03T10:14:29.796 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:30.303 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:30.380 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:30.386 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:30.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.452+0000 7f81f9cbe700 1 Processor -- start 2023-10-03T10:14:30.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.452+0000 7f81f9cbe700 1 -- start start 2023-10-03T10:14:30.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.453+0000 7f81f9cbe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f81f4133420 0x7f81f41337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:30.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.453+0000 7f81f9cbe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f81f4133e30 0x7f81f412ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:30.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.453+0000 7f81f9cbe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f81f4134260 0x7f81f41304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:30.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.453+0000 7f81f9cbe700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f406b990 con 0x7f81f4133e30 2023-10-03T10:14:30.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.453+0000 7f81f9cbe700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f40a0590 con 0x7f81f4133420 2023-10-03T10:14:30.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.453+0000 7f81f9cbe700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81f409fb80 con 0x7f81f4134260 2023-10-03T10:14:30.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.453+0000 7f81f37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f81f4133420 0x7f81f41337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:30.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.453+0000 7f81f37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f81f4133420 0x7f81f41337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44744/0 (socket says 172.21.15.3:44744) 2023-10-03T10:14:30.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.453+0000 7f81f37fe700 1 -- 172.21.15.3:0/4196129431 learned_addr learned my addr 172.21.15.3:0/4196129431 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:30.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.453+0000 7f81f2ffd700 1 --2- 172.21.15.3:0/4196129431 >> v2:172.21.15.3:3300/0 conn(0x7f81f4133e30 0x7f81f412ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:30.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.454+0000 7f81f37fe700 1 -- 172.21.15.3:0/4196129431 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f81f4134260 msgr2=0x7f81f41304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:30.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.454+0000 7f81f37fe700 1 --2- 172.21.15.3:0/4196129431 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f81f4134260 0x7f81f41304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.454+0000 7f81f37fe700 1 -- 172.21.15.3:0/4196129431 >> v2:172.21.15.3:3300/0 conn(0x7f81f4133e30 msgr2=0x7f81f412ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:30.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.454+0000 7f81f37fe700 1 --2- 172.21.15.3:0/4196129431 >> v2:172.21.15.3:3300/0 conn(0x7f81f4133e30 0x7f81f412ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.454+0000 7f81f37fe700 1 -- 172.21.15.3:0/4196129431 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f40a0030 con 0x7f81f4133420 2023-10-03T10:14:30.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.454+0000 7f81f2ffd700 1 --2- 172.21.15.3:0/4196129431 >> v2:172.21.15.3:3300/0 conn(0x7f81f4133e30 0x7f81f412ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:30.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.454+0000 7f81f37fe700 1 --2- 172.21.15.3:0/4196129431 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f81f4133420 0x7f81f41337f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f81e40098f0 tx=0x7f81e4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da94ce0dd4782abf server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:30.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.454+0000 7f81f27fc700 1 -- 172.21.15.3:0/4196129431 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e4011830 con 0x7f81f4133420 2023-10-03T10:14:30.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.454+0000 7f81f27fc700 1 -- 172.21.15.3:0/4196129431 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e4011990 con 0x7f81f4133420 2023-10-03T10:14:30.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.454+0000 7f81f27fc700 1 -- 172.21.15.3:0/4196129431 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e4011d20 con 0x7f81f4133420 2023-10-03T10:14:30.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.455+0000 7f81f9cbe700 1 -- 172.21.15.3:0/4196129431 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f81f4133420 msgr2=0x7f81f41337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:30.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.455+0000 7f81f9cbe700 1 --2- 172.21.15.3:0/4196129431 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f81f4133420 0x7f81f41337f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f81e40098f0 tx=0x7f81e4009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:30.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.455+0000 7f81f9cbe700 1 -- 172.21.15.3:0/4196129431 shutdown_connections 2023-10-03T10:14:30.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.455+0000 7f81f9cbe700 1 --2- 172.21.15.3:0/4196129431 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f81f4134260 0x7f81f41304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.455+0000 7f81f9cbe700 1 --2- 172.21.15.3:0/4196129431 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f81f4133420 0x7f81f41337f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.455+0000 7f81f9cbe700 1 --2- 172.21.15.3:0/4196129431 >> v2:172.21.15.3:3300/0 conn(0x7f81f4133e30 0x7f81f412ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.455+0000 7f81f9cbe700 1 -- 172.21.15.3:0/4196129431 >> 172.21.15.3:0/4196129431 conn(0x7f81f40a4210 msgr2=0x7f81f40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:30.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.455+0000 7f81f9cbe700 1 -- 172.21.15.3:0/4196129431 shutdown_connections 2023-10-03T10:14:30.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.455+0000 7f81f9cbe700 1 -- 172.21.15.3:0/4196129431 wait complete. 2023-10-03T10:14:30.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.457+0000 7f81f9cbe700 1 Processor -- start 2023-10-03T10:14:30.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.457+0000 7f81f9cbe700 1 -- start start 2023-10-03T10:14:30.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.457+0000 7f81f9cbe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81f4133420 0x7f81f409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:30.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.457+0000 7f81f9cbe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f81f4133e30 0x7f81f409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:30.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.457+0000 7f81f9cbe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f81f4134260 0x7f81f409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:30.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.458+0000 7f81f9cbe700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f4131570 con 0x7f81f4133420 2023-10-03T10:14:30.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.458+0000 7f81f9cbe700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f4120340 con 0x7f81f4133e30 2023-10-03T10:14:30.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.458+0000 7f81f9cbe700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81f409deb0 con 0x7f81f4134260 2023-10-03T10:14:30.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.458+0000 7f81f37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81f4133420 0x7f81f409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:30.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.458+0000 7f81f37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81f4133420 0x7f81f409c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33110/0 (socket says 172.21.15.3:33110) 2023-10-03T10:14:30.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.458+0000 7f81f37fe700 1 -- 172.21.15.3:0/2516004312 learned_addr learned my addr 172.21.15.3:0/2516004312 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:30.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.458+0000 7f81f2ffd700 1 --2- 172.21.15.3:0/2516004312 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f81f4133e30 0x7f81f409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:30.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.458+0000 7f81f3fff700 1 --2- 172.21.15.3:0/2516004312 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f81f4134260 0x7f81f409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:30.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.458+0000 7f81f3fff700 1 -- 172.21.15.3:0/2516004312 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f81f4133e30 msgr2=0x7f81f409c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:30.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.458+0000 7f81f3fff700 1 --2- 172.21.15.3:0/2516004312 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f81f4133e30 0x7f81f409c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.458+0000 7f81f3fff700 1 -- 172.21.15.3:0/2516004312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81f4133420 msgr2=0x7f81f409c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:30.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.458+0000 7f81f3fff700 1 --2- 172.21.15.3:0/2516004312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81f4133420 0x7f81f409c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.458+0000 7f81f3fff700 1 -- 172.21.15.3:0/2516004312 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e80094a0 con 0x7f81f4134260 2023-10-03T10:14:30.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.458+0000 7f81f37fe700 1 --2- 172.21.15.3:0/2516004312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81f4133420 0x7f81f409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:30.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.458+0000 7f81f2ffd700 1 --2- 172.21.15.3:0/2516004312 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f81f4133e30 0x7f81f409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:14:30.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.458+0000 7f81f3fff700 1 --2- 172.21.15.3:0/2516004312 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f81f4134260 0x7f81f409cf00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f81ec00be20 tx=0x7f81ec009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:30.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.459+0000 7f81dbfff700 1 -- 172.21.15.3:0/2516004312 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec00cc10 con 0x7f81f4134260 2023-10-03T10:14:30.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.459+0000 7f81f9cbe700 1 -- 172.21.15.3:0/2516004312 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e4009550 con 0x7f81f4134260 2023-10-03T10:14:30.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.459+0000 7f81dbfff700 1 -- 172.21.15.3:0/2516004312 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ec00cd90 con 0x7f81f4134260 2023-10-03T10:14:30.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.459+0000 7f81dbfff700 1 -- 172.21.15.3:0/2516004312 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec014680 con 0x7f81f4134260 2023-10-03T10:14:30.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.459+0000 7f81f9cbe700 1 -- 172.21.15.3:0/2516004312 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f4120060 con 0x7f81f4134260 2023-10-03T10:14:30.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.460+0000 7f81dbfff700 1 -- 172.21.15.3:0/2516004312 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f81ec01c360 con 0x7f81f4134260 2023-10-03T10:14:30.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.460+0000 7f81f9cbe700 1 -- 172.21.15.3:0/2516004312 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f4074ea0 con 0x7f81f4134260 2023-10-03T10:14:30.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.460+0000 7f81dbfff700 1 -- 172.21.15.3:0/2516004312 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f81ec006e40 con 0x7f81f4134260 2023-10-03T10:14:30.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.465+0000 7f81dbfff700 1 -- 172.21.15.3:0/2516004312 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f81ec01d3b0 con 0x7f81f4134260 2023-10-03T10:14:30.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.525+0000 7fb2256f3700 1 Processor -- start 2023-10-03T10:14:30.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.525+0000 7fb2256f3700 1 -- start start 2023-10-03T10:14:30.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.525+0000 7fb2256f3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb220133a80 0x7fb220133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:30.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.525+0000 7fb2256f3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb220130a70 0x7fb22013b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:30.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.525+0000 7fb2256f3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb22006bc00 con 0x7fb22012ff20 2023-10-03T10:14:30.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.525+0000 7fb2256f3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb22006c570 con 0x7fb220133a80 2023-10-03T10:14:30.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.525+0000 7fb2256f3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2200a0040 con 0x7fb220130a70 2023-10-03T10:14:30.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.526+0000 7fb21effd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb220133a80 0x7fb220133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:30.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.526+0000 7fb21f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb220130a70 0x7fb22013b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:30.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.526+0000 7fb21effd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb220133a80 0x7fb220133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33116/0 (socket says 172.21.15.3:33116) 2023-10-03T10:14:30.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.526+0000 7fb21effd700 1 -- 172.21.15.3:0/1748189068 learned_addr learned my addr 172.21.15.3:0/1748189068 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:30.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.526+0000 7fb21effd700 1 -- 172.21.15.3:0/1748189068 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb220130a70 msgr2=0x7fb22013b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:30.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.526+0000 7fb21effd700 1 --2- 172.21.15.3:0/1748189068 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb220130a70 0x7fb22013b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.526+0000 7fb21effd700 1 -- 172.21.15.3:0/1748189068 >> v1:172.21.15.3:6789/0 conn(0x7fb22012ff20 legacy=0x7fb220130360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:30.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.526+0000 7fb21effd700 1 -- 172.21.15.3:0/1748189068 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb220133860 con 0x7fb220133a80 2023-10-03T10:14:30.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.526+0000 7fb21f7fe700 1 --2- 172.21.15.3:0/1748189068 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb220130a70 0x7fb22013b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:30.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.526+0000 7fb21effd700 1 --2- 172.21.15.3:0/1748189068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb220133a80 0x7fb220133e50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb2100098f0 tx=0x7fb210009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2bcae4ab7ea67b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:30.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.526+0000 7fb21dffb700 1 -- 172.21.15.3:0/1748189068 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb210011830 con 0x7fb220133a80 2023-10-03T10:14:30.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.526+0000 7fb21dffb700 1 -- 172.21.15.3:0/1748189068 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb210011990 con 0x7fb220133a80 2023-10-03T10:14:30.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.526+0000 7fb21dffb700 1 -- 172.21.15.3:0/1748189068 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb210011d20 con 0x7fb220133a80 2023-10-03T10:14:30.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.527+0000 7fb2256f3700 1 -- 172.21.15.3:0/1748189068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb220133a80 msgr2=0x7fb220133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:30.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.527+0000 7fb2256f3700 1 --2- 172.21.15.3:0/1748189068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb220133a80 0x7fb220133e50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb2100098f0 tx=0x7fb210009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:30.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.527+0000 7fb2256f3700 1 -- 172.21.15.3:0/1748189068 shutdown_connections 2023-10-03T10:14:30.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.527+0000 7fb2256f3700 1 --2- 172.21.15.3:0/1748189068 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb220130a70 0x7fb22013b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.527+0000 7fb2256f3700 1 --2- 172.21.15.3:0/1748189068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb220133a80 0x7fb220133e50 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.527+0000 7fb2256f3700 1 -- 172.21.15.3:0/1748189068 >> 172.21.15.3:0/1748189068 conn(0x7fb2200a4210 msgr2=0x7fb2200a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:30.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.527+0000 7fb2256f3700 1 -- 172.21.15.3:0/1748189068 shutdown_connections 2023-10-03T10:14:30.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.527+0000 7fb2256f3700 1 -- 172.21.15.3:0/1748189068 wait complete. 2023-10-03T10:14:30.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.529+0000 7fb2256f3700 1 Processor -- start 2023-10-03T10:14:30.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.529+0000 7fb2256f3700 1 -- start start 2023-10-03T10:14:30.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.529+0000 7fb2256f3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb22012ff20 0x7fb22009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:30.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.529+0000 7fb2256f3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb220130a70 0x7fb22009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:30.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.529+0000 7fb2256f3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb220133a80 0x7fb22009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:30.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.529+0000 7fb2256f3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22011ed70 con 0x7fb22012ff20 2023-10-03T10:14:30.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.529+0000 7fb2256f3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb22011ea90 con 0x7fb220133a80 2023-10-03T10:14:30.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.529+0000 7fb2256f3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb220120090 con 0x7fb220130a70 2023-10-03T10:14:30.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.529+0000 7fb21effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb22012ff20 0x7fb22009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:30.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.529+0000 7fb21effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb22012ff20 0x7fb22009c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33126/0 (socket says 172.21.15.3:33126) 2023-10-03T10:14:30.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.529+0000 7fb21effd700 1 -- 172.21.15.3:0/870805358 learned_addr learned my addr 172.21.15.3:0/870805358 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:30.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.529+0000 7fb21e7fc700 1 --2- 172.21.15.3:0/870805358 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb220130a70 0x7fb22009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:30.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.529+0000 7fb21f7fe700 1 --2- 172.21.15.3:0/870805358 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb220133a80 0x7fb22009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:30.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.530+0000 7fb21effd700 1 -- 172.21.15.3:0/870805358 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb220130a70 msgr2=0x7fb22009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:30.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.530+0000 7fb21effd700 1 --2- 172.21.15.3:0/870805358 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb220130a70 0x7fb22009c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.530+0000 7fb21effd700 1 -- 172.21.15.3:0/870805358 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb220133a80 msgr2=0x7fb22009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:30.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.530+0000 7fb21effd700 1 --2- 172.21.15.3:0/870805358 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb220133a80 0x7fb22009cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.530+0000 7fb21effd700 1 -- 172.21.15.3:0/870805358 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb218009a70 con 0x7fb22012ff20 2023-10-03T10:14:30.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.530+0000 7fb21effd700 1 --2- 172.21.15.3:0/870805358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb22012ff20 0x7fb22009c480 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb210009c40 tx=0x7fb210004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:30.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.530+0000 7fb2077fe700 1 -- 172.21.15.3:0/870805358 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2100043c0 con 0x7fb22012ff20 2023-10-03T10:14:30.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.530+0000 7fb2077fe700 1 -- 172.21.15.3:0/870805358 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21001f070 con 0x7fb22012ff20 2023-10-03T10:14:30.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.530+0000 7fb2077fe700 1 -- 172.21.15.3:0/870805358 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2100036a0 con 0x7fb22012ff20 2023-10-03T10:14:30.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.530+0000 7fb2256f3700 1 -- 172.21.15.3:0/870805358 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb210009550 con 0x7fb22012ff20 2023-10-03T10:14:30.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.530+0000 7fb2256f3700 1 -- 172.21.15.3:0/870805358 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb22012d300 con 0x7fb22012ff20 2023-10-03T10:14:30.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.531+0000 7fb2256f3700 1 -- 172.21.15.3:0/870805358 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2200a20c0 con 0x7fb22012ff20 2023-10-03T10:14:30.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.531+0000 7fb2077fe700 1 -- 172.21.15.3:0/870805358 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb210021030 con 0x7fb22012ff20 2023-10-03T10:14:30.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.532+0000 7fb2077fe700 1 -- 172.21.15.3:0/870805358 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb210040a10 con 0x7fb22012ff20 2023-10-03T10:14:30.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.541+0000 7fb2077fe700 1 -- 172.21.15.3:0/870805358 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb2100199a0 con 0x7fb22012ff20 2023-10-03T10:14:30.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.570+0000 7f59a57a1700 1 Processor -- start 2023-10-03T10:14:30.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.570+0000 7f59a57a1700 1 -- start start 2023-10-03T10:14:30.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.570+0000 7f59a57a1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f59a009e930 0x7f59a009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:30.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.570+0000 7f59a57a1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59a009f650 0x7f59a0131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:30.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.570+0000 7f59a57a1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59a0070940 con 0x7f59a009f240 2023-10-03T10:14:30.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.570+0000 7f59a57a1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59a006fd60 con 0x7f59a009e930 2023-10-03T10:14:30.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.570+0000 7f59a57a1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f59a006fa80 con 0x7f59a009f650 2023-10-03T10:14:30.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.571+0000 7f599effd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f59a009e930 0x7f59a009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:30.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.571+0000 7f599f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59a009f650 0x7f59a0131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:30.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.571+0000 7f599effd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f59a009e930 0x7f59a009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33130/0 (socket says 172.21.15.3:33130) 2023-10-03T10:14:30.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.571+0000 7f599e7fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f59a009f240 0x7f59a01317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50232/0 (socket says 172.21.15.3:50232) 2023-10-03T10:14:30.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.571+0000 7f599effd700 1 -- 172.21.15.3:0/1569794490 learned_addr learned my addr 172.21.15.3:0/1569794490 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:30.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.571+0000 7f599effd700 1 -- 172.21.15.3:0/1569794490 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59a009f650 msgr2=0x7f59a0131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:30.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.571+0000 7f599effd700 1 --2- 172.21.15.3:0/1569794490 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59a009f650 0x7f59a0131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.571+0000 7f599effd700 1 -- 172.21.15.3:0/1569794490 >> v1:172.21.15.3:6789/0 conn(0x7f59a009f240 legacy=0x7f59a01317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:30.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.571+0000 7f599effd700 1 -- 172.21.15.3:0/1569794490 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a009fb60 con 0x7f59a009e930 2023-10-03T10:14:30.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.571+0000 7f599effd700 1 --2- 172.21.15.3:0/1569794490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59a009e930 0x7f59a009ed00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f59900098f0 tx=0x7f5990009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c266f87a62ed9929 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:30.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.571+0000 7f599dffb700 1 -- 172.21.15.3:0/1569794490 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5990011830 con 0x7f59a009e930 2023-10-03T10:14:30.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.571+0000 7f599dffb700 1 -- 172.21.15.3:0/1569794490 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5990011990 con 0x7f59a009e930 2023-10-03T10:14:30.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.571+0000 7f599dffb700 1 -- 172.21.15.3:0/1569794490 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5990011d20 con 0x7f59a009e930 2023-10-03T10:14:30.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.572+0000 7f59a57a1700 1 -- 172.21.15.3:0/1569794490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59a009e930 msgr2=0x7f59a009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:30.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.572+0000 7f59a57a1700 1 --2- 172.21.15.3:0/1569794490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59a009e930 0x7f59a009ed00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f59900098f0 tx=0x7f5990009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:30.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.572+0000 7f59a57a1700 1 -- 172.21.15.3:0/1569794490 shutdown_connections 2023-10-03T10:14:30.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.572+0000 7f59a57a1700 1 --2- 172.21.15.3:0/1569794490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59a009e930 0x7f59a009ed00 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.572+0000 7f59a57a1700 1 --2- 172.21.15.3:0/1569794490 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59a009f650 0x7f59a0131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.572+0000 7f59a57a1700 1 -- 172.21.15.3:0/1569794490 >> 172.21.15.3:0/1569794490 conn(0x7f59a00a4210 msgr2=0x7f59a00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:30.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.572+0000 7f59a57a1700 1 -- 172.21.15.3:0/1569794490 shutdown_connections 2023-10-03T10:14:30.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.572+0000 7f59a57a1700 1 -- 172.21.15.3:0/1569794490 wait complete. 2023-10-03T10:14:30.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.573+0000 7f59a57a1700 1 Processor -- start 2023-10-03T10:14:30.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.573+0000 7f59a57a1700 1 -- start start 2023-10-03T10:14:30.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.574+0000 7f59a57a1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59a009e930 0x7f59a006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:30.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.574+0000 7f59a57a1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59a009f240 0x7f59a006bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:30.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.574+0000 7f59a57a1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59a009f650 0x7f59a006c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:30.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.574+0000 7f59a57a1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a01b7340 con 0x7f59a009f650 2023-10-03T10:14:30.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.574+0000 7f59a57a1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59a01a94f0 con 0x7f59a009e930 2023-10-03T10:14:30.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.574+0000 7f59a57a1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f59a01a8a10 con 0x7f59a009f240 2023-10-03T10:14:30.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.574+0000 7f599e7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59a009f240 0x7f59a006bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:30.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.574+0000 7f599f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59a009f650 0x7f59a006c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:30.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.574+0000 7f599f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59a009f650 0x7f59a006c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33142/0 (socket says 172.21.15.3:33142) 2023-10-03T10:14:30.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.574+0000 7f599f7fe700 1 -- 172.21.15.3:0/1756562938 learned_addr learned my addr 172.21.15.3:0/1756562938 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:30.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.574+0000 7f599effd700 1 --2- 172.21.15.3:0/1756562938 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59a009e930 0x7f59a006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:30.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.574+0000 7f599e7fc700 1 -- 172.21.15.3:0/1756562938 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59a009e930 msgr2=0x7f59a006b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:30.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.574+0000 7f599e7fc700 1 --2- 172.21.15.3:0/1756562938 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59a009e930 0x7f59a006b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.574+0000 7f599e7fc700 1 -- 172.21.15.3:0/1756562938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59a009f650 msgr2=0x7f59a006c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:30.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.574+0000 7f599e7fc700 1 --2- 172.21.15.3:0/1756562938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59a009f650 0x7f59a006c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.575+0000 7f599e7fc700 1 -- 172.21.15.3:0/1756562938 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5990009550 con 0x7f59a009f240 2023-10-03T10:14:30.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.575+0000 7f599f7fe700 1 --2- 172.21.15.3:0/1756562938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59a009f650 0x7f59a006c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:30.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.575+0000 7f599e7fc700 1 --2- 172.21.15.3:0/1756562938 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59a009f240 0x7f59a006bdf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f599400c450 tx=0x7f599400c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:30.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.575+0000 7f59877fe700 1 -- 172.21.15.3:0/1756562938 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5994005cb0 con 0x7f59a009f240 2023-10-03T10:14:30.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.575+0000 7f59877fe700 1 -- 172.21.15.3:0/1756562938 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5994005e10 con 0x7f59a009f240 2023-10-03T10:14:30.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.575+0000 7f59877fe700 1 -- 172.21.15.3:0/1756562938 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59940072d0 con 0x7f59a009f240 2023-10-03T10:14:30.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.575+0000 7f59a57a1700 1 -- 172.21.15.3:0/1756562938 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59a00a0190 con 0x7f59a009f240 2023-10-03T10:14:30.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.575+0000 7f59a57a1700 1 -- 172.21.15.3:0/1756562938 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59a01a9800 con 0x7f59a009f240 2023-10-03T10:14:30.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.576+0000 7f59877fe700 1 -- 172.21.15.3:0/1756562938 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5994007430 con 0x7f59a009f240 2023-10-03T10:14:30.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.576+0000 7f59a57a1700 1 -- 172.21.15.3:0/1756562938 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59a0071d20 con 0x7f59a009f240 2023-10-03T10:14:30.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.576+0000 7f59877fe700 1 -- 172.21.15.3:0/1756562938 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f599403e840 con 0x7f59a009f240 2023-10-03T10:14:30.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.581+0000 7f59877fe700 1 -- 172.21.15.3:0/1756562938 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f599400ce60 con 0x7f59a009f240 2023-10-03T10:14:30.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.674+0000 7f81f9cbe700 1 -- 172.21.15.3:0/2516004312 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f81f4120b30 con 0x7f81f4134260 2023-10-03T10:14:30.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.675+0000 7f81dbfff700 1 -- 172.21.15.3:0/2516004312 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f81ec019030 con 0x7f81f4134260 2023-10-03T10:14:30.674 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:30.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.677+0000 7f81f9cbe700 1 -- 172.21.15.3:0/2516004312 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f81f4134260 msgr2=0x7f81f409cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:30.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.677+0000 7f81f9cbe700 1 --2- 172.21.15.3:0/2516004312 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f81f4134260 0x7f81f409cf00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f81ec00be20 tx=0x7f81ec009ed0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.677+0000 7f81f9cbe700 1 -- 172.21.15.3:0/2516004312 shutdown_connections 2023-10-03T10:14:30.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.677+0000 7f81f9cbe700 1 --2- 172.21.15.3:0/2516004312 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f81f4134260 0x7f81f409cf00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.677+0000 7f81f9cbe700 1 --2- 172.21.15.3:0/2516004312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81f4133420 0x7f81f409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.677+0000 7f81f9cbe700 1 --2- 172.21.15.3:0/2516004312 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f81f4133e30 0x7f81f409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.677+0000 7f81f9cbe700 1 -- 172.21.15.3:0/2516004312 >> 172.21.15.3:0/2516004312 conn(0x7f81f40a4210 msgr2=0x7f81f40a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:30.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.677+0000 7f81f9cbe700 1 -- 172.21.15.3:0/2516004312 shutdown_connections 2023-10-03T10:14:30.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.677+0000 7f81f9cbe700 1 -- 172.21.15.3:0/2516004312 wait complete. 2023-10-03T10:14:30.690 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:30.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.748+0000 7fb2256f3700 1 -- 172.21.15.3:0/870805358 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb220120360 con 0x7fb22012ff20 2023-10-03T10:14:30.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.749+0000 7fb2077fe700 1 -- 172.21.15.3:0/870805358 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb210032020 con 0x7fb22012ff20 2023-10-03T10:14:30.748 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:30.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.751+0000 7fb2256f3700 1 -- 172.21.15.3:0/870805358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb22012ff20 msgr2=0x7fb22009c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:30.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.751+0000 7fb2256f3700 1 --2- 172.21.15.3:0/870805358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb22012ff20 0x7fb22009c480 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb210009c40 tx=0x7fb210004750 comp rx=0 tx=0).stop 2023-10-03T10:14:30.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.751+0000 7fb2256f3700 1 -- 172.21.15.3:0/870805358 shutdown_connections 2023-10-03T10:14:30.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.751+0000 7fb2256f3700 1 --2- 172.21.15.3:0/870805358 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb220133a80 0x7fb22009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.751+0000 7fb2256f3700 1 --2- 172.21.15.3:0/870805358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb22012ff20 0x7fb22009c480 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.751+0000 7fb2256f3700 1 --2- 172.21.15.3:0/870805358 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb220130a70 0x7fb22009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.751+0000 7fb2256f3700 1 -- 172.21.15.3:0/870805358 >> 172.21.15.3:0/870805358 conn(0x7fb2200a4210 msgr2=0x7fb22009e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:30.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.751+0000 7fb2256f3700 1 -- 172.21.15.3:0/870805358 shutdown_connections 2023-10-03T10:14:30.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.751+0000 7fb2256f3700 1 -- 172.21.15.3:0/870805358 wait complete. 2023-10-03T10:14:30.764 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:30.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.786+0000 7f59a57a1700 1 -- 172.21.15.3:0/1756562938 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f59a01a9800 con 0x7f59a009f240 2023-10-03T10:14:30.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.787+0000 7f59877fe700 1 -- 172.21.15.3:0/1756562938 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f59a01a9800 con 0x7f59a009f240 2023-10-03T10:14:30.786 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:30.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.789+0000 7f59a57a1700 1 -- 172.21.15.3:0/1756562938 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59a009f240 msgr2=0x7f59a006bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:30.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.789+0000 7f59a57a1700 1 --2- 172.21.15.3:0/1756562938 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59a009f240 0x7f59a006bdf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f599400c450 tx=0x7f599400c760 comp rx=0 tx=0).stop 2023-10-03T10:14:30.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.789+0000 7f59a57a1700 1 -- 172.21.15.3:0/1756562938 shutdown_connections 2023-10-03T10:14:30.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.789+0000 7f59a57a1700 1 --2- 172.21.15.3:0/1756562938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59a009f650 0x7f59a006c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.789+0000 7f59a57a1700 1 --2- 172.21.15.3:0/1756562938 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59a009e930 0x7f59a006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.789+0000 7f59a57a1700 1 --2- 172.21.15.3:0/1756562938 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59a009f240 0x7f59a006bdf0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.789+0000 7f59a57a1700 1 -- 172.21.15.3:0/1756562938 >> 172.21.15.3:0/1756562938 conn(0x7f59a00a4210 msgr2=0x7f59a01b8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:30.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.789+0000 7f59a57a1700 1 -- 172.21.15.3:0/1756562938 shutdown_connections 2023-10-03T10:14:30.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.789+0000 7f59a57a1700 1 -- 172.21.15.3:0/1756562938 wait complete. 2023-10-03T10:14:30.797 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:30.802 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:30.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.934+0000 7f7f82563700 1 Processor -- start 2023-10-03T10:14:30.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.934+0000 7f7f82563700 1 -- start start 2023-10-03T10:14:30.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.934+0000 7f7f82563700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f7c0a0a30 0x7f7f7c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:30.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.934+0000 7f7f82563700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f7c0a1340 0x7f7f7c133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:30.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.934+0000 7f7f82563700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7f7c133fa0 0x7f7f7c134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:30.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.934+0000 7f7f82563700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f7c09edd0 con 0x7f7f7c133fa0 2023-10-03T10:14:30.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.934+0000 7f7f82563700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c09e1f0 con 0x7f7f7c0a0a30 2023-10-03T10:14:30.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.934+0000 7f7f82563700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c09dc50 con 0x7f7f7c0a1340 2023-10-03T10:14:30.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.935+0000 7f7f7b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f7c0a1340 0x7f7f7c133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:30.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.935+0000 7f7f80b00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7f7c133fa0 0x7f7f7c134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:30.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.935+0000 7f7f80b00700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7f7c133fa0 0x7f7f7c134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33146/0 (socket says 172.21.15.3:33146) 2023-10-03T10:14:30.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.935+0000 7f7f7b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f7c0a1340 0x7f7f7c133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42976/0 (socket says 172.21.15.3:42976) 2023-10-03T10:14:30.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.935+0000 7f7f7bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f7c0a0a30 0x7f7f7c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:30.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.935+0000 7f7f80b00700 1 -- 172.21.15.3:0/1298116759 learned_addr learned my addr 172.21.15.3:0/1298116759 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:30.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.935+0000 7f7f7bfff700 1 -- 172.21.15.3:0/1298116759 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f7c0a1340 msgr2=0x7f7f7c133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:30.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.935+0000 7f7f7bfff700 1 --2- 172.21.15.3:0/1298116759 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f7c0a1340 0x7f7f7c133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.935+0000 7f7f7bfff700 1 -- 172.21.15.3:0/1298116759 >> v2:172.21.15.3:3300/0 conn(0x7f7f7c133fa0 msgr2=0x7f7f7c134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:30.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.935+0000 7f7f7bfff700 1 --2- 172.21.15.3:0/1298116759 >> v2:172.21.15.3:3300/0 conn(0x7f7f7c133fa0 0x7f7f7c134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.935+0000 7f7f7bfff700 1 -- 172.21.15.3:0/1298116759 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7c0a1fe0 con 0x7f7f7c0a0a30 2023-10-03T10:14:30.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.935+0000 7f7f80b00700 1 --2- 172.21.15.3:0/1298116759 >> v2:172.21.15.3:3300/0 conn(0x7f7f7c133fa0 0x7f7f7c134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:30.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.935+0000 7f7f7bfff700 1 --2- 172.21.15.3:0/1298116759 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f7c0a0a30 0x7f7f7c0a0e00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009840 tx=0x7f7f6c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6a29f14826d488c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:30.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.935+0000 7f7f7affd700 1 -- 172.21.15.3:0/1298116759 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c011830 con 0x7f7f7c0a0a30 2023-10-03T10:14:30.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.935+0000 7f7f7affd700 1 -- 172.21.15.3:0/1298116759 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6c011990 con 0x7f7f7c0a0a30 2023-10-03T10:14:30.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.935+0000 7f7f7affd700 1 -- 172.21.15.3:0/1298116759 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c011d20 con 0x7f7f7c0a0a30 2023-10-03T10:14:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.936+0000 7f7f82563700 1 -- 172.21.15.3:0/1298116759 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f7c0a0a30 msgr2=0x7f7f7c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.936+0000 7f7f82563700 1 --2- 172.21.15.3:0/1298116759 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f7c0a0a30 0x7f7f7c0a0e00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009840 tx=0x7f7f6c009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.936+0000 7f7f82563700 1 -- 172.21.15.3:0/1298116759 shutdown_connections 2023-10-03T10:14:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.936+0000 7f7f82563700 1 --2- 172.21.15.3:0/1298116759 >> v2:172.21.15.3:3300/0 conn(0x7f7f7c133fa0 0x7f7f7c134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.936+0000 7f7f82563700 1 --2- 172.21.15.3:0/1298116759 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f7c0a0a30 0x7f7f7c0a0e00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.936+0000 7f7f82563700 1 --2- 172.21.15.3:0/1298116759 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f7c0a1340 0x7f7f7c133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.936+0000 7f7f82563700 1 -- 172.21.15.3:0/1298116759 >> 172.21.15.3:0/1298116759 conn(0x7f7f7c0a6550 msgr2=0x7f7f7c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.936+0000 7f7f82563700 1 -- 172.21.15.3:0/1298116759 shutdown_connections 2023-10-03T10:14:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.936+0000 7f7f82563700 1 -- 172.21.15.3:0/1298116759 wait complete. 2023-10-03T10:14:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.937+0000 7f7f82563700 1 Processor -- start 2023-10-03T10:14:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.937+0000 7f7f82563700 1 -- start start 2023-10-03T10:14:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.938+0000 7f7f82563700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f7c0a0a30 0x7f7f7c11df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.938+0000 7f7f82563700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f7c0a1340 0x7f7f7c11e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:30.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.938+0000 7f7f82563700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f7c133fa0 0x7f7f7c11e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:30.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.938+0000 7f7f82563700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c122430 con 0x7f7f7c133fa0 2023-10-03T10:14:30.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.938+0000 7f7f82563700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c11fb60 con 0x7f7f7c0a1340 2023-10-03T10:14:30.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.938+0000 7f7f82563700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c120db0 con 0x7f7f7c0a0a30 2023-10-03T10:14:30.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.938+0000 7f7f7bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f7c0a0a30 0x7f7f7c11df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:30.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.938+0000 7f7f7bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f7c0a0a30 0x7f7f7c11df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42988/0 (socket says 172.21.15.3:42988) 2023-10-03T10:14:30.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.938+0000 7f7f7bfff700 1 -- 172.21.15.3:0/2427771937 learned_addr learned my addr 172.21.15.3:0/2427771937 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:30.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.938+0000 7f7f7b7fe700 1 --2- 172.21.15.3:0/2427771937 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f7c0a1340 0x7f7f7c11e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:30.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.938+0000 7f7f80b00700 1 --2- 172.21.15.3:0/2427771937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f7c133fa0 0x7f7f7c11e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:30.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.938+0000 7f7f7bfff700 1 -- 172.21.15.3:0/2427771937 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f7c0a1340 msgr2=0x7f7f7c11e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:30.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.938+0000 7f7f7bfff700 1 --2- 172.21.15.3:0/2427771937 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f7c0a1340 0x7f7f7c11e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.938+0000 7f7f7bfff700 1 -- 172.21.15.3:0/2427771937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f7c133fa0 msgr2=0x7f7f7c11e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:30.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.938+0000 7f7f7bfff700 1 --2- 172.21.15.3:0/2427771937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f7c133fa0 0x7f7f7c11e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:30.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.938+0000 7f7f7bfff700 1 -- 172.21.15.3:0/2427771937 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f74009b30 con 0x7f7f7c0a0a30 2023-10-03T10:14:30.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.938+0000 7f7f7b7fe700 1 --2- 172.21.15.3:0/2427771937 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f7c0a1340 0x7f7f7c11e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:30.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.938+0000 7f7f7bfff700 1 --2- 172.21.15.3:0/2427771937 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f7c0a0a30 0x7f7f7c11df50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c09e380 tx=0x7f7f6c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:30.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.939+0000 7f7f78ff9700 1 -- 172.21.15.3:0/2427771937 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c004450 con 0x7f7f7c0a0a30 2023-10-03T10:14:30.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.939+0000 7f7f78ff9700 1 -- 172.21.15.3:0/2427771937 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6c01f070 con 0x7f7f7c0a0a30 2023-10-03T10:14:30.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.939+0000 7f7f78ff9700 1 -- 172.21.15.3:0/2427771937 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c0038e0 con 0x7f7f7c0a0a30 2023-10-03T10:14:30.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.939+0000 7f7f82563700 1 -- 172.21.15.3:0/2427771937 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f6c0094a0 con 0x7f7f7c0a0a30 2023-10-03T10:14:30.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.939+0000 7f7f82563700 1 -- 172.21.15.3:0/2427771937 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f7c0a3630 con 0x7f7f7c0a0a30 2023-10-03T10:14:30.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.939+0000 7f7f78ff9700 1 -- 172.21.15.3:0/2427771937 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7f6c003a40 con 0x7f7f7c0a0a30 2023-10-03T10:14:30.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.940+0000 7f7f78ff9700 1 -- 172.21.15.3:0/2427771937 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7f6c021070 con 0x7f7f7c0a0a30 2023-10-03T10:14:30.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.940+0000 7f7f82563700 1 -- 172.21.15.3:0/2427771937 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f7c0a0220 con 0x7f7f7c0a0a30 2023-10-03T10:14:30.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:30.945+0000 7f7f78ff9700 1 -- 172.21.15.3:0/2427771937 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7f6c015080 con 0x7f7f7c0a0a30 2023-10-03T10:14:31.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.152+0000 7f7f82563700 1 -- 172.21.15.3:0/2427771937 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7f7c074ea0 con 0x7f7f7c0a0a30 2023-10-03T10:14:31.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.152+0000 7f7f78ff9700 1 -- 172.21.15.3:0/2427771937 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7f6c01b7b0 con 0x7f7f7c0a0a30 2023-10-03T10:14:31.151 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:31.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.154+0000 7f7f82563700 1 -- 172.21.15.3:0/2427771937 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f7c0a0a30 msgr2=0x7f7f7c11df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:31.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.154+0000 7f7f82563700 1 --2- 172.21.15.3:0/2427771937 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f7c0a0a30 0x7f7f7c11df50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c09e380 tx=0x7f7f6c004750 comp rx=0 tx=0).stop 2023-10-03T10:14:31.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.154+0000 7f7f82563700 1 -- 172.21.15.3:0/2427771937 shutdown_connections 2023-10-03T10:14:31.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.154+0000 7f7f82563700 1 --2- 172.21.15.3:0/2427771937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f7c133fa0 0x7f7f7c11e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:31.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.154+0000 7f7f82563700 1 --2- 172.21.15.3:0/2427771937 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f7c0a0a30 0x7f7f7c11df50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:31.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.154+0000 7f7f82563700 1 --2- 172.21.15.3:0/2427771937 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f7c0a1340 0x7f7f7c11e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:31.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.154+0000 7f7f82563700 1 -- 172.21.15.3:0/2427771937 >> 172.21.15.3:0/2427771937 conn(0x7f7f7c0a6550 msgr2=0x7f7f7c133060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:31.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.154+0000 7f7f82563700 1 -- 172.21.15.3:0/2427771937 shutdown_connections 2023-10-03T10:14:31.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.154+0000 7f7f82563700 1 -- 172.21.15.3:0/2427771937 wait complete. 2023-10-03T10:14:31.167 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:31.691 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:31.765 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:31.803 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:31.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.839+0000 7fe40d2d0700 1 Processor -- start 2023-10-03T10:14:31.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.839+0000 7fe40d2d0700 1 -- start start 2023-10-03T10:14:31.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.839+0000 7fe40d2d0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe40809d6f0 0x7fe40809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:31.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.839+0000 7fe40d2d0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe40809e000 0x7fe408130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:31.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.839+0000 7fe40d2d0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe40809f130 con 0x7fe40809e410 2023-10-03T10:14:31.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.839+0000 7fe40d2d0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe40809f520 con 0x7fe40809d6f0 2023-10-03T10:14:31.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.839+0000 7fe40d2d0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40809e930 con 0x7fe40809e000 2023-10-03T10:14:31.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.840+0000 7fe4067fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe40809e000 0x7fe408130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:31.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.840+0000 7fe4067fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe40809e000 0x7fe408130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44828/0 (socket says 172.21.15.3:44828) 2023-10-03T10:14:31.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.840+0000 7fe4067fc700 1 -- 172.21.15.3:0/3923619135 learned_addr learned my addr 172.21.15.3:0/3923619135 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:31.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.840+0000 7fe406ffd700 1 --2- 172.21.15.3:0/3923619135 >> v2:172.21.15.3:3300/0 conn(0x7fe40809d6f0 0x7fe40809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:31.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.840+0000 7fe4067fc700 1 -- 172.21.15.3:0/3923619135 >> v2:172.21.15.3:3300/0 conn(0x7fe40809d6f0 msgr2=0x7fe40809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:31.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.840+0000 7fe4067fc700 1 --2- 172.21.15.3:0/3923619135 >> v2:172.21.15.3:3300/0 conn(0x7fe40809d6f0 0x7fe40809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:31.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.840+0000 7fe4067fc700 1 -- 172.21.15.3:0/3923619135 >> v1:172.21.15.3:6789/0 conn(0x7fe40809e410 legacy=0x7fe408130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:31.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.840+0000 7fe4067fc700 1 -- 172.21.15.3:0/3923619135 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40809b2d0 con 0x7fe40809e000 2023-10-03T10:14:31.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.840+0000 7fe406ffd700 1 --2- 172.21.15.3:0/3923619135 >> v2:172.21.15.3:3300/0 conn(0x7fe40809d6f0 0x7fe40809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:14:31.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.840+0000 7fe4067fc700 1 --2- 172.21.15.3:0/3923619135 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe40809e000 0x7fe408130660 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00bb90 tx=0x7fe3fc00bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ba2022f23f3b0b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:31.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.840+0000 7fe405ffb700 1 -- 172.21.15.3:0/3923619135 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc00dd10 con 0x7fe40809e000 2023-10-03T10:14:31.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.841+0000 7fe405ffb700 1 -- 172.21.15.3:0/3923619135 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3fc00de70 con 0x7fe40809e000 2023-10-03T10:14:31.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.841+0000 7fe405ffb700 1 -- 172.21.15.3:0/3923619135 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc004230 con 0x7fe40809e000 2023-10-03T10:14:31.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.841+0000 7fe40d2d0700 1 -- 172.21.15.3:0/3923619135 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe40809e000 msgr2=0x7fe408130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:31.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.841+0000 7fe40d2d0700 1 --2- 172.21.15.3:0/3923619135 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe40809e000 0x7fe408130660 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00bb90 tx=0x7fe3fc00bea0 comp rx=0 tx=0).stop 2023-10-03T10:14:31.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.841+0000 7fe40d2d0700 1 -- 172.21.15.3:0/3923619135 shutdown_connections 2023-10-03T10:14:31.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.841+0000 7fe40d2d0700 1 --2- 172.21.15.3:0/3923619135 >> v2:172.21.15.3:3300/0 conn(0x7fe40809d6f0 0x7fe40809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:31.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.841+0000 7fe40d2d0700 1 --2- 172.21.15.3:0/3923619135 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe40809e000 0x7fe408130660 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:31.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.841+0000 7fe40d2d0700 1 -- 172.21.15.3:0/3923619135 >> 172.21.15.3:0/3923619135 conn(0x7fe408066f50 msgr2=0x7fe40809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:31.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.841+0000 7fe40d2d0700 1 -- 172.21.15.3:0/3923619135 shutdown_connections 2023-10-03T10:14:31.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.841+0000 7fe40d2d0700 1 -- 172.21.15.3:0/3923619135 wait complete. 2023-10-03T10:14:31.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.843+0000 7fe40d2d0700 1 Processor -- start 2023-10-03T10:14:31.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.843+0000 7fe40d2d0700 1 -- start start 2023-10-03T10:14:31.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.843+0000 7fe40d2d0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe40809d6f0 0x7fe40811dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:31.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.843+0000 7fe40d2d0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe40809e000 0x7fe40811e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:31.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.843+0000 7fe40d2d0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe40809e410 0x7fe40811e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:31.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.843+0000 7fe40d2d0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe408121a00 con 0x7fe40809e000 2023-10-03T10:14:31.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.843+0000 7fe40d2d0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe408120110 con 0x7fe40809e410 2023-10-03T10:14:31.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.843+0000 7fe40d2d0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe408121190 con 0x7fe40809d6f0 2023-10-03T10:14:31.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.844+0000 7fe4077fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe40809e410 0x7fe40811e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:31.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.844+0000 7fe4067fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe40809e000 0x7fe40811e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:31.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.844+0000 7fe4077fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe40809e410 0x7fe40811e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44840/0 (socket says 172.21.15.3:44840) 2023-10-03T10:14:31.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.844+0000 7fe4067fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe40809e000 0x7fe40811e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33176/0 (socket says 172.21.15.3:33176) 2023-10-03T10:14:31.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.844+0000 7fe4077fe700 1 -- 172.21.15.3:0/2873747038 learned_addr learned my addr 172.21.15.3:0/2873747038 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:31.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.844+0000 7fe406ffd700 1 --2- 172.21.15.3:0/2873747038 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe40809d6f0 0x7fe40811dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:31.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.844+0000 7fe4077fe700 1 -- 172.21.15.3:0/2873747038 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe40809d6f0 msgr2=0x7fe40811dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:31.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.844+0000 7fe4077fe700 1 --2- 172.21.15.3:0/2873747038 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe40809d6f0 0x7fe40811dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:31.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.844+0000 7fe4077fe700 1 -- 172.21.15.3:0/2873747038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe40809e000 msgr2=0x7fe40811e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:31.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.844+0000 7fe4077fe700 1 --2- 172.21.15.3:0/2873747038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe40809e000 0x7fe40811e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:31.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.844+0000 7fe4077fe700 1 -- 172.21.15.3:0/2873747038 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3fc00b7f0 con 0x7fe40809e410 2023-10-03T10:14:31.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.844+0000 7fe4077fe700 1 --2- 172.21.15.3:0/2873747038 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe40809e410 0x7fe40811e960 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe40000cc10 tx=0x7fe40000cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:31.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.845+0000 7fe3ef7fe700 1 -- 172.21.15.3:0/2873747038 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40000aa60 con 0x7fe40809e410 2023-10-03T10:14:31.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.845+0000 7fe3ef7fe700 1 -- 172.21.15.3:0/2873747038 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40000abc0 con 0x7fe40809e410 2023-10-03T10:14:31.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.845+0000 7fe3ef7fe700 1 -- 172.21.15.3:0/2873747038 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe400012420 con 0x7fe40809e410 2023-10-03T10:14:31.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.845+0000 7fe40d2d0700 1 -- 172.21.15.3:0/2873747038 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe40809c700 con 0x7fe40809e410 2023-10-03T10:14:31.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.845+0000 7fe40d2d0700 1 -- 172.21.15.3:0/2873747038 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4081302c0 con 0x7fe40809e410 2023-10-03T10:14:31.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.846+0000 7fe3ef7fe700 1 -- 172.21.15.3:0/2873747038 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe400012580 con 0x7fe40809e410 2023-10-03T10:14:31.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.846+0000 7fe40d2d0700 1 -- 172.21.15.3:0/2873747038 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe40811ac20 con 0x7fe40809e410 2023-10-03T10:14:31.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.846+0000 7fe3ef7fe700 1 -- 172.21.15.3:0/2873747038 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe40003f950 con 0x7fe40809e410 2023-10-03T10:14:31.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.851+0000 7fe3ef7fe700 1 -- 172.21.15.3:0/2873747038 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe400019b90 con 0x7fe40809e410 2023-10-03T10:14:31.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.908+0000 7fdb8f279700 1 Processor -- start 2023-10-03T10:14:31.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.908+0000 7fdb8f279700 1 -- start start 2023-10-03T10:14:31.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.908+0000 7fdb8f279700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb880a0a30 0x7fdb880a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:31.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.908+0000 7fdb8f279700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdb8809d630 0x7fdb880a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:31.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.908+0000 7fdb8f279700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb880a2a70 con 0x7fdb88133930 2023-10-03T10:14:31.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.908+0000 7fdb8f279700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb880a01d0 con 0x7fdb8809d630 2023-10-03T10:14:31.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.908+0000 7fdb8f279700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8809e2b0 con 0x7fdb880a0a30 2023-10-03T10:14:31.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.909+0000 7fdb8c814700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdb8809d630 0x7fdb880a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:31.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.909+0000 7fdb8c814700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdb8809d630 0x7fdb880a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33182/0 (socket says 172.21.15.3:33182) 2023-10-03T10:14:31.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.909+0000 7fdb8c814700 1 -- 172.21.15.3:0/2172416064 learned_addr learned my addr 172.21.15.3:0/2172416064 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:31.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.909+0000 7fdb8d015700 1 --2- 172.21.15.3:0/2172416064 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb880a0a30 0x7fdb880a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:31.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.909+0000 7fdb8d015700 1 -- 172.21.15.3:0/2172416064 >> v2:172.21.15.3:3300/0 conn(0x7fdb8809d630 msgr2=0x7fdb880a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:31.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.909+0000 7fdb8d015700 1 --2- 172.21.15.3:0/2172416064 >> v2:172.21.15.3:3300/0 conn(0x7fdb8809d630 0x7fdb880a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:31.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.909+0000 7fdb8d015700 1 -- 172.21.15.3:0/2172416064 >> v1:172.21.15.3:6789/0 conn(0x7fdb88133930 legacy=0x7fdb88133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:31.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.909+0000 7fdb7ffff700 1 -- 172.21.15.3:0/2172416064 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1978247828 0 0) 0x7fdb880a2a70 con 0x7fdb88133930 2023-10-03T10:14:31.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.909+0000 7fdb8d015700 1 -- 172.21.15.3:0/2172416064 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8809dc50 con 0x7fdb880a0a30 2023-10-03T10:14:31.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.909+0000 7fdb8d015700 1 --2- 172.21.15.3:0/2172416064 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb880a0a30 0x7fdb880a0e00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdb84009840 tx=0x7fdb84009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=607a47eaf627747d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:31.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.910+0000 7fdb7ffff700 1 -- 172.21.15.3:0/2172416064 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb84011830 con 0x7fdb880a0a30 2023-10-03T10:14:31.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.910+0000 7fdb7ffff700 1 -- 172.21.15.3:0/2172416064 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb84011990 con 0x7fdb880a0a30 2023-10-03T10:14:31.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.910+0000 7fdb7ffff700 1 -- 172.21.15.3:0/2172416064 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb84011d20 con 0x7fdb880a0a30 2023-10-03T10:14:31.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.910+0000 7fdb8f279700 1 -- 172.21.15.3:0/2172416064 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb880a0a30 msgr2=0x7fdb880a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:31.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.910+0000 7fdb8f279700 1 --2- 172.21.15.3:0/2172416064 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb880a0a30 0x7fdb880a0e00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdb84009840 tx=0x7fdb84009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:31.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.910+0000 7fdb8f279700 1 -- 172.21.15.3:0/2172416064 shutdown_connections 2023-10-03T10:14:31.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.910+0000 7fdb8f279700 1 --2- 172.21.15.3:0/2172416064 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb880a0a30 0x7fdb880a0e00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:31.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.910+0000 7fdb8f279700 1 --2- 172.21.15.3:0/2172416064 >> v2:172.21.15.3:3300/0 conn(0x7fdb8809d630 0x7fdb880a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:31.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.910+0000 7fdb8f279700 1 -- 172.21.15.3:0/2172416064 >> 172.21.15.3:0/2172416064 conn(0x7fdb880a3050 msgr2=0x7fdb880a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:31.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.911+0000 7fdb8f279700 1 -- 172.21.15.3:0/2172416064 shutdown_connections 2023-10-03T10:14:31.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.911+0000 7fdb8f279700 1 -- 172.21.15.3:0/2172416064 wait complete. 2023-10-03T10:14:31.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.912+0000 7fdb8f279700 1 Processor -- start 2023-10-03T10:14:31.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.912+0000 7fdb8f279700 1 -- start start 2023-10-03T10:14:31.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.912+0000 7fdb8f279700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb8809d630 0x7fdb8811f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:31.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.912+0000 7fdb8f279700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb880a0a30 0x7fdb8811faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:31.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.912+0000 7fdb8f279700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb88133930 0x7fdb881c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:31.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.912+0000 7fdb8f279700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb880a0410 con 0x7fdb88133930 2023-10-03T10:14:31.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.912+0000 7fdb8f279700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8811f110 con 0x7fdb8809d630 2023-10-03T10:14:31.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.912+0000 7fdb8f279700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdb88121870 con 0x7fdb880a0a30 2023-10-03T10:14:31.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.912+0000 7fdb8c814700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb880a0a30 0x7fdb8811faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:31.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.912+0000 7fdb8c814700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb880a0a30 0x7fdb8811faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43022/0 (socket says 172.21.15.3:43022) 2023-10-03T10:14:31.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.912+0000 7fdb8c814700 1 -- 172.21.15.3:0/863098189 learned_addr learned my addr 172.21.15.3:0/863098189 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:31.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.912+0000 7fdb8d816700 1 --2- 172.21.15.3:0/863098189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb88133930 0x7fdb881c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:31.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.913+0000 7fdb8d015700 1 --2- 172.21.15.3:0/863098189 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb8809d630 0x7fdb8811f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:31.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.913+0000 7fdb8c814700 1 -- 172.21.15.3:0/863098189 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb8809d630 msgr2=0x7fdb8811f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:31.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.913+0000 7fdb8c814700 1 --2- 172.21.15.3:0/863098189 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb8809d630 0x7fdb8811f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:31.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.913+0000 7fdb8c814700 1 -- 172.21.15.3:0/863098189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb88133930 msgr2=0x7fdb881c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:31.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.913+0000 7fdb8c814700 1 --2- 172.21.15.3:0/863098189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb88133930 0x7fdb881c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:31.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.913+0000 7fdb8c814700 1 -- 172.21.15.3:0/863098189 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb840094a0 con 0x7fdb880a0a30 2023-10-03T10:14:31.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.913+0000 7fdb8d816700 1 --2- 172.21.15.3:0/863098189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb88133930 0x7fdb881c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:14:31.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.913+0000 7fdb8c814700 1 --2- 172.21.15.3:0/863098189 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb880a0a30 0x7fdb8811faf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800e8e0 tx=0x7fdb7800eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:31.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.913+0000 7fdb7dffb700 1 -- 172.21.15.3:0/863098189 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7800eee0 con 0x7fdb880a0a30 2023-10-03T10:14:31.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.913+0000 7fdb7dffb700 1 -- 172.21.15.3:0/863098189 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb7800c850 con 0x7fdb880a0a30 2023-10-03T10:14:31.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.913+0000 7fdb8f279700 1 -- 172.21.15.3:0/863098189 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb88120030 con 0x7fdb880a0a30 2023-10-03T10:14:31.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.913+0000 7fdb7dffb700 1 -- 172.21.15.3:0/863098189 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7800cbc0 con 0x7fdb880a0a30 2023-10-03T10:14:31.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.913+0000 7fdb8f279700 1 -- 172.21.15.3:0/863098189 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb88122f00 con 0x7fdb880a0a30 2023-10-03T10:14:31.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.914+0000 7fdb7dffb700 1 -- 172.21.15.3:0/863098189 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdb7800cd20 con 0x7fdb880a0a30 2023-10-03T10:14:31.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.914+0000 7fdb7dffb700 1 -- 172.21.15.3:0/863098189 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fdb7803f890 con 0x7fdb880a0a30 2023-10-03T10:14:31.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.914+0000 7fdb8f279700 1 -- 172.21.15.3:0/863098189 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb88074560 con 0x7fdb880a0a30 2023-10-03T10:14:31.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.920+0000 7fdb7dffb700 1 -- 172.21.15.3:0/863098189 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdb78021080 con 0x7fdb880a0a30 2023-10-03T10:14:31.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.950+0000 7f1c78ae2700 1 Processor -- start 2023-10-03T10:14:31.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.951+0000 7f1c78ae2700 1 -- start start 2023-10-03T10:14:31.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.951+0000 7f1c78ae2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c7409e930 0x7f1c7409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:31.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.951+0000 7f1c78ae2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c7409f240 0x7f1c741317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:31.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.951+0000 7f1c78ae2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c74070940 con 0x7f1c7409f650 2023-10-03T10:14:31.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.951+0000 7f1c78ae2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7406fd60 con 0x7f1c7409f240 2023-10-03T10:14:31.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.951+0000 7f1c78ae2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7406fa80 con 0x7f1c7409e930 2023-10-03T10:14:31.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.951+0000 7f1c7259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c7409e930 0x7f1c7409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:31.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.951+0000 7f1c71d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c7409f240 0x7f1c741317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:31.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.952+0000 7f1c71d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c7409f240 0x7f1c741317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44860/0 (socket says 172.21.15.3:44860) 2023-10-03T10:14:31.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.952+0000 7f1c7259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c7409e930 0x7f1c7409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43036/0 (socket says 172.21.15.3:43036) 2023-10-03T10:14:31.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.952+0000 7f1c71d9b700 1 -- 172.21.15.3:0/1861849199 learned_addr learned my addr 172.21.15.3:0/1861849199 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:31.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.952+0000 7f1c71d9b700 1 -- 172.21.15.3:0/1861849199 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c7409e930 msgr2=0x7f1c7409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:31.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.952+0000 7f1c71d9b700 1 --2- 172.21.15.3:0/1861849199 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c7409e930 0x7f1c7409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:31.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.952+0000 7f1c71d9b700 1 -- 172.21.15.3:0/1861849199 >> v1:172.21.15.3:6789/0 conn(0x7f1c7409f650 legacy=0x7f1c74131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:31.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.952+0000 7f1c71d9b700 1 -- 172.21.15.3:0/1861849199 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7409fb60 con 0x7f1c7409f240 2023-10-03T10:14:31.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.952+0000 7f1c7159a700 1 -- 172.21.15.3:0/1861849199 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4221120297 0 0) 0x7f1c74070940 con 0x7f1c7409f650 2023-10-03T10:14:31.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.952+0000 7f1c71d9b700 1 --2- 172.21.15.3:0/1861849199 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c7409f240 0x7f1c741317f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009920 tx=0x7f1c68009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f77e4bab2b3e7214 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:31.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.952+0000 7f1c7159a700 1 -- 172.21.15.3:0/1861849199 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c68011830 con 0x7f1c7409f240 2023-10-03T10:14:31.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.952+0000 7f1c7159a700 1 -- 172.21.15.3:0/1861849199 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c68011990 con 0x7f1c7409f240 2023-10-03T10:14:31.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.952+0000 7f1c7159a700 1 -- 172.21.15.3:0/1861849199 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c68011d20 con 0x7f1c7409f240 2023-10-03T10:14:31.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.953+0000 7f1c78ae2700 1 -- 172.21.15.3:0/1861849199 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c7409f240 msgr2=0x7f1c741317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:31.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.953+0000 7f1c78ae2700 1 --2- 172.21.15.3:0/1861849199 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c7409f240 0x7f1c741317f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009920 tx=0x7f1c68009c30 comp rx=0 tx=0).stop 2023-10-03T10:14:31.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.953+0000 7f1c78ae2700 1 -- 172.21.15.3:0/1861849199 shutdown_connections 2023-10-03T10:14:31.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.953+0000 7f1c78ae2700 1 --2- 172.21.15.3:0/1861849199 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c7409e930 0x7f1c7409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:31.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.953+0000 7f1c78ae2700 1 --2- 172.21.15.3:0/1861849199 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c7409f240 0x7f1c741317f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:31.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.953+0000 7f1c78ae2700 1 -- 172.21.15.3:0/1861849199 >> 172.21.15.3:0/1861849199 conn(0x7f1c740a4210 msgr2=0x7f1c740a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:31.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.953+0000 7f1c78ae2700 1 -- 172.21.15.3:0/1861849199 shutdown_connections 2023-10-03T10:14:31.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.953+0000 7f1c78ae2700 1 -- 172.21.15.3:0/1861849199 wait complete. 2023-10-03T10:14:31.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.955+0000 7f1c78ae2700 1 Processor -- start 2023-10-03T10:14:31.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.955+0000 7f1c78ae2700 1 -- start start 2023-10-03T10:14:31.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.956+0000 7f1c78ae2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c7409e930 0x7f1c741a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:31.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.956+0000 7f1c78ae2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c7409f240 0x7f1c741a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:31.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.956+0000 7f1c78ae2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c7409f650 0x7f1c741a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:31.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.956+0000 7f1c78ae2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c741a5f20 con 0x7f1c7409e930 2023-10-03T10:14:31.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.956+0000 7f1c78ae2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c741a81a0 con 0x7f1c7409f240 2023-10-03T10:14:31.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.956+0000 7f1c78ae2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c741a77e0 con 0x7f1c7409f650 2023-10-03T10:14:31.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.956+0000 7f1c71d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c7409f240 0x7f1c741a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:31.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.956+0000 7f1c71d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c7409f240 0x7f1c741a4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44864/0 (socket says 172.21.15.3:44864) 2023-10-03T10:14:31.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.956+0000 7f1c71d9b700 1 -- 172.21.15.3:0/2910543865 learned_addr learned my addr 172.21.15.3:0/2910543865 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:31.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.956+0000 7f1c7259c700 1 --2- 172.21.15.3:0/2910543865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c7409e930 0x7f1c741a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:31.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.956+0000 7f1c72d9d700 1 --2- 172.21.15.3:0/2910543865 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c7409f650 0x7f1c741a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:31.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.956+0000 7f1c71d9b700 1 -- 172.21.15.3:0/2910543865 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c7409f650 msgr2=0x7f1c741a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:31.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.956+0000 7f1c71d9b700 1 --2- 172.21.15.3:0/2910543865 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c7409f650 0x7f1c741a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:31.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.956+0000 7f1c71d9b700 1 -- 172.21.15.3:0/2910543865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c7409e930 msgr2=0x7f1c741a3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:31.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.956+0000 7f1c71d9b700 1 --2- 172.21.15.3:0/2910543865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c7409e930 0x7f1c741a3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:31.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.956+0000 7f1c71d9b700 1 -- 172.21.15.3:0/2910543865 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c68009550 con 0x7f1c7409f240 2023-10-03T10:14:31.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.956+0000 7f1c7259c700 1 --2- 172.21.15.3:0/2910543865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c7409e930 0x7f1c741a3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:14:31.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.956+0000 7f1c71d9b700 1 --2- 172.21.15.3:0/2910543865 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c7409f240 0x7f1c741a4200 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1c68013000 tx=0x7f1c68004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:31.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.957+0000 7f1c62ffd700 1 -- 172.21.15.3:0/2910543865 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c68004510 con 0x7f1c7409f240 2023-10-03T10:14:31.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.957+0000 7f1c62ffd700 1 -- 172.21.15.3:0/2910543865 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c680036a0 con 0x7f1c7409f240 2023-10-03T10:14:31.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.957+0000 7f1c78ae2700 1 -- 172.21.15.3:0/2910543865 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c740a0190 con 0x7f1c7409f240 2023-10-03T10:14:31.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.957+0000 7f1c62ffd700 1 -- 172.21.15.3:0/2910543865 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c68003a30 con 0x7f1c7409f240 2023-10-03T10:14:31.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.957+0000 7f1c78ae2700 1 -- 172.21.15.3:0/2910543865 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c74071d20 con 0x7f1c7409f240 2023-10-03T10:14:31.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.958+0000 7f1c62ffd700 1 -- 172.21.15.3:0/2910543865 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1c68003b90 con 0x7f1c7409f240 2023-10-03T10:14:31.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.958+0000 7f1c78ae2700 1 -- 172.21.15.3:0/2910543865 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c741a71a0 con 0x7f1c7409f240 2023-10-03T10:14:31.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.958+0000 7f1c62ffd700 1 -- 172.21.15.3:0/2910543865 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1c68021030 con 0x7f1c7409f240 2023-10-03T10:14:31.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:31.963+0000 7f1c62ffd700 1 -- 172.21.15.3:0/2910543865 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1c741a71a0 con 0x7f1c7409f240 2023-10-03T10:14:32.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.064+0000 7fe40d2d0700 1 -- 172.21.15.3:0/2873747038 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe408074ea0 con 0x7fe40809e410 2023-10-03T10:14:32.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.064+0000 7fe3ef7fe700 1 -- 172.21.15.3:0/2873747038 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe400019780 con 0x7fe40809e410 2023-10-03T10:14:32.065 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:32.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.068+0000 7fe40d2d0700 1 -- 172.21.15.3:0/2873747038 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe40809e410 msgr2=0x7fe40811e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:32.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.068+0000 7fe40d2d0700 1 --2- 172.21.15.3:0/2873747038 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe40809e410 0x7fe40811e960 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe40000cc10 tx=0x7fe40000cf20 comp rx=0 tx=0).stop 2023-10-03T10:14:32.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.068+0000 7fe40d2d0700 1 -- 172.21.15.3:0/2873747038 shutdown_connections 2023-10-03T10:14:32.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.068+0000 7fe40d2d0700 1 --2- 172.21.15.3:0/2873747038 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe40809e410 0x7fe40811e960 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:32.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.068+0000 7fe40d2d0700 1 --2- 172.21.15.3:0/2873747038 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe40809d6f0 0x7fe40811dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:32.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.068+0000 7fe40d2d0700 1 --2- 172.21.15.3:0/2873747038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe40809e000 0x7fe40811e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:32.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.068+0000 7fe40d2d0700 1 -- 172.21.15.3:0/2873747038 >> 172.21.15.3:0/2873747038 conn(0x7fe408066f50 msgr2=0x7fe408130b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:32.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.068+0000 7fe40d2d0700 1 -- 172.21.15.3:0/2873747038 shutdown_connections 2023-10-03T10:14:32.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.068+0000 7fe40d2d0700 1 -- 172.21.15.3:0/2873747038 wait complete. 2023-10-03T10:14:32.082 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:32.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.128+0000 7fdb8f279700 1 -- 172.21.15.3:0/863098189 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdb88133700 con 0x7fdb880a0a30 2023-10-03T10:14:32.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.128+0000 7fdb7dffb700 1 -- 172.21.15.3:0/863098189 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdb78017020 con 0x7fdb880a0a30 2023-10-03T10:14:32.129 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:32.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.130+0000 7fdb8f279700 1 -- 172.21.15.3:0/863098189 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb880a0a30 msgr2=0x7fdb8811faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:32.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.130+0000 7fdb8f279700 1 --2- 172.21.15.3:0/863098189 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb880a0a30 0x7fdb8811faf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800e8e0 tx=0x7fdb7800eca0 comp rx=0 tx=0).stop 2023-10-03T10:14:32.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.131+0000 7fdb8f279700 1 -- 172.21.15.3:0/863098189 shutdown_connections 2023-10-03T10:14:32.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.131+0000 7fdb8f279700 1 --2- 172.21.15.3:0/863098189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb88133930 0x7fdb881c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:32.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.131+0000 7fdb8f279700 1 --2- 172.21.15.3:0/863098189 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb8809d630 0x7fdb8811f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:32.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.131+0000 7fdb8f279700 1 --2- 172.21.15.3:0/863098189 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb880a0a30 0x7fdb8811faf0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:32.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.131+0000 7fdb8f279700 1 -- 172.21.15.3:0/863098189 >> 172.21.15.3:0/863098189 conn(0x7fdb880a3050 msgr2=0x7fdb880a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:32.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.131+0000 7fdb8f279700 1 -- 172.21.15.3:0/863098189 shutdown_connections 2023-10-03T10:14:32.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.131+0000 7fdb8f279700 1 -- 172.21.15.3:0/863098189 wait complete. 2023-10-03T10:14:32.145 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:32.168 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:32.171 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:32.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.170+0000 7f1c78ae2700 1 -- 172.21.15.3:0/2910543865 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1c741a71a0 con 0x7f1c7409f240 2023-10-03T10:14:32.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.170+0000 7f1c62ffd700 1 -- 172.21.15.3:0/2910543865 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1c741a71a0 con 0x7f1c7409f240 2023-10-03T10:14:32.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.173+0000 7f1c78ae2700 1 -- 172.21.15.3:0/2910543865 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c7409f240 msgr2=0x7f1c741a4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:32.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.173+0000 7f1c78ae2700 1 --2- 172.21.15.3:0/2910543865 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c7409f240 0x7f1c741a4200 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1c68013000 tx=0x7f1c68004750 comp rx=0 tx=0).stop 2023-10-03T10:14:32.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.173+0000 7f1c78ae2700 1 -- 172.21.15.3:0/2910543865 shutdown_connections 2023-10-03T10:14:32.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.173+0000 7f1c78ae2700 1 --2- 172.21.15.3:0/2910543865 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c7409f650 0x7f1c741a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:32.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.173+0000 7f1c78ae2700 1 --2- 172.21.15.3:0/2910543865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c7409e930 0x7f1c741a3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:32.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.173+0000 7f1c78ae2700 1 --2- 172.21.15.3:0/2910543865 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c7409f240 0x7f1c741a4200 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:32.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.173+0000 7f1c78ae2700 1 -- 172.21.15.3:0/2910543865 >> 172.21.15.3:0/2910543865 conn(0x7f1c740a4210 msgr2=0x7f1c741b8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:32.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.173+0000 7f1c78ae2700 1 -- 172.21.15.3:0/2910543865 shutdown_connections 2023-10-03T10:14:32.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.173+0000 7f1c78ae2700 1 -- 172.21.15.3:0/2910543865 wait complete. 2023-10-03T10:14:32.189 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.308+0000 7f282e7ec700 1 Processor -- start 2023-10-03T10:14:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.308+0000 7f282e7ec700 1 -- start start 2023-10-03T10:14:32.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.309+0000 7f282e7ec700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f282809d6f0 0x7f282809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:32.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.309+0000 7f282e7ec700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f282809e000 0x7f28281339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:32.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.309+0000 7f282e7ec700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f282809b120 con 0x7f282809e410 2023-10-03T10:14:32.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.309+0000 7f282e7ec700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f282806b460 con 0x7f282809e000 2023-10-03T10:14:32.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.309+0000 7f282e7ec700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f282809a7e0 con 0x7f282809d6f0 2023-10-03T10:14:32.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.309+0000 7f28277fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f282809e000 0x7f28281339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:32.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.309+0000 7f28277fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f282809e000 0x7f28281339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33206/0 (socket says 172.21.15.3:33206) 2023-10-03T10:14:32.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.309+0000 7f28277fe700 1 -- 172.21.15.3:0/1325506680 learned_addr learned my addr 172.21.15.3:0/1325506680 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:32.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.309+0000 7f2827fff700 1 --2- 172.21.15.3:0/1325506680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f282809d6f0 0x7f282809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:32.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.309+0000 7f2827fff700 1 -- 172.21.15.3:0/1325506680 >> v2:172.21.15.3:3300/0 conn(0x7f282809e000 msgr2=0x7f28281339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:32.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.309+0000 7f2827fff700 1 --2- 172.21.15.3:0/1325506680 >> v2:172.21.15.3:3300/0 conn(0x7f282809e000 0x7f28281339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:32.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.309+0000 7f2827fff700 1 -- 172.21.15.3:0/1325506680 >> v1:172.21.15.3:6789/0 conn(0x7f282809e410 legacy=0x7f2828133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:32.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.309+0000 7f2827fff700 1 -- 172.21.15.3:0/1325506680 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282809a500 con 0x7f282809d6f0 2023-10-03T10:14:32.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.309+0000 7f28277fe700 1 --2- 172.21.15.3:0/1325506680 >> v2:172.21.15.3:3300/0 conn(0x7f282809e000 0x7f28281339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:32.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.309+0000 7f2827fff700 1 --2- 172.21.15.3:0/1325506680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f282809d6f0 0x7f282809dac0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2818009870 tx=0x7f2818009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=566edb31b63751b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:32.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.310+0000 7f2826ffd700 1 -- 172.21.15.3:0/1325506680 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2818011830 con 0x7f282809d6f0 2023-10-03T10:14:32.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.310+0000 7f2826ffd700 1 -- 172.21.15.3:0/1325506680 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2818011990 con 0x7f282809d6f0 2023-10-03T10:14:32.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.310+0000 7f2826ffd700 1 -- 172.21.15.3:0/1325506680 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2818011d20 con 0x7f282809d6f0 2023-10-03T10:14:32.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.310+0000 7f282e7ec700 1 -- 172.21.15.3:0/1325506680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f282809d6f0 msgr2=0x7f282809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:32.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.310+0000 7f282e7ec700 1 --2- 172.21.15.3:0/1325506680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f282809d6f0 0x7f282809dac0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2818009870 tx=0x7f2818009b80 comp rx=0 tx=0).stop 2023-10-03T10:14:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.310+0000 7f282e7ec700 1 -- 172.21.15.3:0/1325506680 shutdown_connections 2023-10-03T10:14:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.310+0000 7f282e7ec700 1 --2- 172.21.15.3:0/1325506680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f282809d6f0 0x7f282809dac0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.310+0000 7f282e7ec700 1 --2- 172.21.15.3:0/1325506680 >> v2:172.21.15.3:3300/0 conn(0x7f282809e000 0x7f28281339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.310+0000 7f282e7ec700 1 -- 172.21.15.3:0/1325506680 >> 172.21.15.3:0/1325506680 conn(0x7f2828066f50 msgr2=0x7f282809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.310+0000 7f282e7ec700 1 -- 172.21.15.3:0/1325506680 shutdown_connections 2023-10-03T10:14:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.310+0000 7f282e7ec700 1 -- 172.21.15.3:0/1325506680 wait complete. 2023-10-03T10:14:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.312+0000 7f282e7ec700 1 Processor -- start 2023-10-03T10:14:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.312+0000 7f282e7ec700 1 -- start start 2023-10-03T10:14:32.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.313+0000 7f282e7ec700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f282809d6f0 0x7f28281a4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.313+0000 7f282e7ec700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f282809e000 0x7f28281a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.313+0000 7f282e7ec700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f282809e410 0x7f28281a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.313+0000 7f282e7ec700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28281a6ae0 con 0x7f282809e410 2023-10-03T10:14:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.313+0000 7f282e7ec700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28281a7d30 con 0x7f282809e000 2023-10-03T10:14:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.313+0000 7f282e7ec700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28281a70d0 con 0x7f282809d6f0 2023-10-03T10:14:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.313+0000 7f282cd89700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f282809e410 0x7f28281a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.313+0000 7f28277fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f282809e000 0x7f28281a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.313+0000 7f2827fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f282809d6f0 0x7f28281a4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.313+0000 7f282cd89700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f282809e410 0x7f28281a59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33208/0 (socket says 172.21.15.3:33208) 2023-10-03T10:14:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.313+0000 7f28277fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f282809e000 0x7f28281a54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44866/0 (socket says 172.21.15.3:44866) 2023-10-03T10:14:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.313+0000 7f282cd89700 1 -- 172.21.15.3:0/552898630 learned_addr learned my addr 172.21.15.3:0/552898630 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:32.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.313+0000 7f2827fff700 1 -- 172.21.15.3:0/552898630 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f282809e000 msgr2=0x7f28281a54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:32.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.313+0000 7f2827fff700 1 --2- 172.21.15.3:0/552898630 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f282809e000 0x7f28281a54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:32.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.313+0000 7f2827fff700 1 -- 172.21.15.3:0/552898630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f282809e410 msgr2=0x7f28281a59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:32.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.313+0000 7f2827fff700 1 --2- 172.21.15.3:0/552898630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f282809e410 0x7f28281a59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:32.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.313+0000 7f2827fff700 1 -- 172.21.15.3:0/552898630 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281c009550 con 0x7f282809d6f0 2023-10-03T10:14:32.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.313+0000 7f28277fe700 1 --2- 172.21.15.3:0/552898630 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f282809e000 0x7f28281a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:32.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.313+0000 7f282cd89700 1 --2- 172.21.15.3:0/552898630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f282809e410 0x7f28281a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:32.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.313+0000 7f2827fff700 1 --2- 172.21.15.3:0/552898630 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f282809d6f0 0x7f28281a4f60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2818016000 tx=0x7f2818004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:32.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.314+0000 7f2824ff9700 1 -- 172.21.15.3:0/552898630 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28180036a0 con 0x7f282809d6f0 2023-10-03T10:14:32.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.314+0000 7f2824ff9700 1 -- 172.21.15.3:0/552898630 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281801f070 con 0x7f282809d6f0 2023-10-03T10:14:32.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.314+0000 7f282e7ec700 1 -- 172.21.15.3:0/552898630 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28180094a0 con 0x7f282809d6f0 2023-10-03T10:14:32.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.314+0000 7f2824ff9700 1 -- 172.21.15.3:0/552898630 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2818003a20 con 0x7f282809d6f0 2023-10-03T10:14:32.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.314+0000 7f282e7ec700 1 -- 172.21.15.3:0/552898630 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f282809c710 con 0x7f282809d6f0 2023-10-03T10:14:32.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.314+0000 7f2824ff9700 1 -- 172.21.15.3:0/552898630 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2818003b80 con 0x7f282809d6f0 2023-10-03T10:14:32.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.315+0000 7f2824ff9700 1 -- 172.21.15.3:0/552898630 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2818021070 con 0x7f282809d6f0 2023-10-03T10:14:32.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.315+0000 7f282e7ec700 1 -- 172.21.15.3:0/552898630 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2828074ea0 con 0x7f282809d6f0 2023-10-03T10:14:32.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.320+0000 7f2824ff9700 1 -- 172.21.15.3:0/552898630 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2818015070 con 0x7f282809d6f0 2023-10-03T10:14:32.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.526+0000 7f282e7ec700 1 -- 172.21.15.3:0/552898630 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f282806c360 con 0x7f282809d6f0 2023-10-03T10:14:32.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.527+0000 7f2824ff9700 1 -- 172.21.15.3:0/552898630 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2818019ba0 con 0x7f282809d6f0 2023-10-03T10:14:32.528 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:32.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.529+0000 7f282e7ec700 1 -- 172.21.15.3:0/552898630 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f282809d6f0 msgr2=0x7f28281a4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:32.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.529+0000 7f282e7ec700 1 --2- 172.21.15.3:0/552898630 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f282809d6f0 0x7f28281a4f60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2818016000 tx=0x7f2818004750 comp rx=0 tx=0).stop 2023-10-03T10:14:32.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.529+0000 7f282e7ec700 1 -- 172.21.15.3:0/552898630 shutdown_connections 2023-10-03T10:14:32.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.529+0000 7f282e7ec700 1 --2- 172.21.15.3:0/552898630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f282809e410 0x7f28281a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:32.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.529+0000 7f282e7ec700 1 --2- 172.21.15.3:0/552898630 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f282809d6f0 0x7f28281a4f60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:32.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.529+0000 7f282e7ec700 1 --2- 172.21.15.3:0/552898630 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f282809e000 0x7f28281a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:32.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.529+0000 7f282e7ec700 1 -- 172.21.15.3:0/552898630 >> 172.21.15.3:0/552898630 conn(0x7f2828066f50 msgr2=0x7f28281bb010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:32.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.529+0000 7f282e7ec700 1 -- 172.21.15.3:0/552898630 shutdown_connections 2023-10-03T10:14:32.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:32.529+0000 7f282e7ec700 1 -- 172.21.15.3:0/552898630 wait complete. 2023-10-03T10:14:32.543 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:33.083 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:33.146 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:33.189 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:33.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.232+0000 7fb532145700 1 Processor -- start 2023-10-03T10:14:33.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.233+0000 7fb532145700 1 -- start start 2023-10-03T10:14:33.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.233+0000 7fb532145700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb52c133420 0x7fb52c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:33.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.233+0000 7fb532145700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb52c133e30 0x7fb52c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:33.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.233+0000 7fb532145700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb52c134260 0x7fb52c1304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:33.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.233+0000 7fb532145700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb52c06b990 con 0x7fb52c133420 2023-10-03T10:14:33.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.233+0000 7fb532145700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c0a0590 con 0x7fb52c134260 2023-10-03T10:14:33.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.233+0000 7fb532145700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c09fb80 con 0x7fb52c133e30 2023-10-03T10:14:33.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.233+0000 7fb52affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb52c133e30 0x7fb52c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:33.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.233+0000 7fb52affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb52c133e30 0x7fb52c12ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:60862/0 (socket says 172.21.15.3:60862) 2023-10-03T10:14:33.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.233+0000 7fb52affd700 1 -- 172.21.15.3:0/681603715 learned_addr learned my addr 172.21.15.3:0/681603715 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:33.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.233+0000 7fb52b7fe700 1 --2- 172.21.15.3:0/681603715 >> v2:172.21.15.3:3300/0 conn(0x7fb52c133420 0x7fb52c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:33.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.233+0000 7fb52bfff700 1 --2- 172.21.15.3:0/681603715 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb52c134260 0x7fb52c1304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:33.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.234+0000 7fb52bfff700 1 -- 172.21.15.3:0/681603715 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb52c133e30 msgr2=0x7fb52c12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:33.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.234+0000 7fb52bfff700 1 --2- 172.21.15.3:0/681603715 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb52c133e30 0x7fb52c12ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.234+0000 7fb52bfff700 1 -- 172.21.15.3:0/681603715 >> v2:172.21.15.3:3300/0 conn(0x7fb52c133420 msgr2=0x7fb52c1337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:33.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.234+0000 7fb52bfff700 1 --2- 172.21.15.3:0/681603715 >> v2:172.21.15.3:3300/0 conn(0x7fb52c133420 0x7fb52c1337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.234+0000 7fb52bfff700 1 -- 172.21.15.3:0/681603715 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52c0a0030 con 0x7fb52c134260 2023-10-03T10:14:33.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.234+0000 7fb52affd700 1 --2- 172.21.15.3:0/681603715 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb52c133e30 0x7fb52c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:33.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.234+0000 7fb52b7fe700 1 --2- 172.21.15.3:0/681603715 >> v2:172.21.15.3:3300/0 conn(0x7fb52c133420 0x7fb52c1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:33.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.234+0000 7fb52bfff700 1 --2- 172.21.15.3:0/681603715 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb52c134260 0x7fb52c1304d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb524009d90 tx=0x7fb52400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fe854429c480d58 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:33.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.234+0000 7fb52a7fc700 1 -- 172.21.15.3:0/681603715 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52400e070 con 0x7fb52c134260 2023-10-03T10:14:33.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.234+0000 7fb52a7fc700 1 -- 172.21.15.3:0/681603715 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52400bb20 con 0x7fb52c134260 2023-10-03T10:14:33.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.234+0000 7fb52a7fc700 1 -- 172.21.15.3:0/681603715 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb524012430 con 0x7fb52c134260 2023-10-03T10:14:33.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.235+0000 7fb532145700 1 -- 172.21.15.3:0/681603715 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb52c134260 msgr2=0x7fb52c1304d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:33.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.235+0000 7fb532145700 1 --2- 172.21.15.3:0/681603715 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb52c134260 0x7fb52c1304d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb524009d90 tx=0x7fb52400b330 comp rx=0 tx=0).stop 2023-10-03T10:14:33.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.235+0000 7fb532145700 1 -- 172.21.15.3:0/681603715 shutdown_connections 2023-10-03T10:14:33.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.235+0000 7fb532145700 1 --2- 172.21.15.3:0/681603715 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb52c134260 0x7fb52c1304d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.235+0000 7fb532145700 1 --2- 172.21.15.3:0/681603715 >> v2:172.21.15.3:3300/0 conn(0x7fb52c133420 0x7fb52c1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.235+0000 7fb532145700 1 --2- 172.21.15.3:0/681603715 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb52c133e30 0x7fb52c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.235+0000 7fb532145700 1 -- 172.21.15.3:0/681603715 >> 172.21.15.3:0/681603715 conn(0x7fb52c0a4210 msgr2=0x7fb52c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:33.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.235+0000 7fb532145700 1 -- 172.21.15.3:0/681603715 shutdown_connections 2023-10-03T10:14:33.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.235+0000 7fb532145700 1 -- 172.21.15.3:0/681603715 wait complete. 2023-10-03T10:14:33.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.236+0000 7fb532145700 1 Processor -- start 2023-10-03T10:14:33.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.236+0000 7fb532145700 1 -- start start 2023-10-03T10:14:33.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.237+0000 7fb532145700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb52c133420 0x7fb52c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:33.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.237+0000 7fb532145700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb52c133e30 0x7fb52c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:33.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.237+0000 7fb532145700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb52c134260 0x7fb52c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:33.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.237+0000 7fb532145700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c11f970 con 0x7fb52c133e30 2023-10-03T10:14:33.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.237+0000 7fb532145700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c11fc20 con 0x7fb52c134260 2023-10-03T10:14:33.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.237+0000 7fb532145700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c09de90 con 0x7fb52c133420 2023-10-03T10:14:33.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.237+0000 7fb52affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb52c133e30 0x7fb52c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:33.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.237+0000 7fb52affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb52c133e30 0x7fb52c09c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49818/0 (socket says 172.21.15.3:49818) 2023-10-03T10:14:33.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.237+0000 7fb52affd700 1 -- 172.21.15.3:0/891400367 learned_addr learned my addr 172.21.15.3:0/891400367 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:33.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.237+0000 7fb52b7fe700 1 --2- 172.21.15.3:0/891400367 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb52c133420 0x7fb52c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:33.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.237+0000 7fb52bfff700 1 --2- 172.21.15.3:0/891400367 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb52c134260 0x7fb52c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:33.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.237+0000 7fb52affd700 1 -- 172.21.15.3:0/891400367 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb52c133420 msgr2=0x7fb52c09c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:33.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.237+0000 7fb52affd700 1 --2- 172.21.15.3:0/891400367 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb52c133420 0x7fb52c09c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.237+0000 7fb52affd700 1 -- 172.21.15.3:0/891400367 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb52c134260 msgr2=0x7fb52c09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:33.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.237+0000 7fb52affd700 1 --2- 172.21.15.3:0/891400367 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb52c134260 0x7fb52c09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.237+0000 7fb52affd700 1 -- 172.21.15.3:0/891400367 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c0094a0 con 0x7fb52c133e30 2023-10-03T10:14:33.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.238+0000 7fb52affd700 1 --2- 172.21.15.3:0/891400367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb52c133e30 0x7fb52c09c9c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb52000ec10 tx=0x7fb52000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:33.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.238+0000 7fb513fff700 1 -- 172.21.15.3:0/891400367 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52000c850 con 0x7fb52c133e30 2023-10-03T10:14:33.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.238+0000 7fb513fff700 1 -- 172.21.15.3:0/891400367 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52000c9b0 con 0x7fb52c133e30 2023-10-03T10:14:33.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.238+0000 7fb532145700 1 -- 172.21.15.3:0/891400367 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb520009550 con 0x7fb52c133e30 2023-10-03T10:14:33.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.238+0000 7fb513fff700 1 -- 172.21.15.3:0/891400367 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52000cd40 con 0x7fb52c133e30 2023-10-03T10:14:33.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.238+0000 7fb532145700 1 -- 172.21.15.3:0/891400367 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb524009a70 con 0x7fb52c133e30 2023-10-03T10:14:33.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.239+0000 7fb513fff700 1 -- 172.21.15.3:0/891400367 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb520013470 con 0x7fb52c133e30 2023-10-03T10:14:33.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.239+0000 7fb513fff700 1 -- 172.21.15.3:0/891400367 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb52003fcc0 con 0x7fb52c133e30 2023-10-03T10:14:33.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.239+0000 7fb532145700 1 -- 172.21.15.3:0/891400367 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52c074ea0 con 0x7fb52c133e30 2023-10-03T10:14:33.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.244+0000 7fb513fff700 1 -- 172.21.15.3:0/891400367 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb520021080 con 0x7fb52c133e30 2023-10-03T10:14:33.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.292+0000 7f1f544c8700 1 Processor -- start 2023-10-03T10:14:33.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.292+0000 7f1f544c8700 1 -- start start 2023-10-03T10:14:33.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.293+0000 7f1f544c8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1f4c12cf60 0x7f1f4c06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:33.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.293+0000 7f1f544c8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1f4c12f8b0 0x7f1f4c12fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:33.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.293+0000 7f1f544c8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1f4c130260 0x7f1f4c136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:33.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.293+0000 7f1f544c8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f4c12d330 con 0x7f1f4c12cf60 2023-10-03T10:14:33.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.293+0000 7f1f544c8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c12cd90 con 0x7f1f4c130260 2023-10-03T10:14:33.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.293+0000 7f1f544c8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c1306d0 con 0x7f1f4c12f8b0 2023-10-03T10:14:33.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.293+0000 7f1f52264700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1f4c12cf60 0x7f1f4c06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:33.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.293+0000 7f1f52264700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1f4c12cf60 0x7f1f4c06f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49822/0 (socket says 172.21.15.3:49822) 2023-10-03T10:14:33.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.293+0000 7f1f52264700 1 -- 172.21.15.3:0/447079818 learned_addr learned my addr 172.21.15.3:0/447079818 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:33.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.293+0000 7f1f52a65700 1 --2- 172.21.15.3:0/447079818 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1f4c130260 0x7f1f4c136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:33.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.293+0000 7f1f52264700 1 -- 172.21.15.3:0/447079818 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1f4c12f8b0 msgr2=0x7f1f4c12fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:33.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.293+0000 7f1f52264700 1 --2- 172.21.15.3:0/447079818 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1f4c12f8b0 0x7f1f4c12fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.293+0000 7f1f52264700 1 -- 172.21.15.3:0/447079818 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1f4c130260 msgr2=0x7f1f4c136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:33.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.293+0000 7f1f52264700 1 --2- 172.21.15.3:0/447079818 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1f4c130260 0x7f1f4c136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.293+0000 7f1f52264700 1 -- 172.21.15.3:0/447079818 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c071780 con 0x7f1f4c12cf60 2023-10-03T10:14:33.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.293+0000 7f1f52a65700 1 --2- 172.21.15.3:0/447079818 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1f4c130260 0x7f1f4c136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:33.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.294+0000 7f1f52264700 1 --2- 172.21.15.3:0/447079818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f4c12cf60 0x7f1f4c06f5d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009920 tx=0x7f1f3c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c7aab1dc1069202 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:33.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.294+0000 7f1f51262700 1 -- 172.21.15.3:0/447079818 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c011830 con 0x7f1f4c12cf60 2023-10-03T10:14:33.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.294+0000 7f1f51262700 1 -- 172.21.15.3:0/447079818 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3c011990 con 0x7f1f4c12cf60 2023-10-03T10:14:33.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.294+0000 7f1f51262700 1 -- 172.21.15.3:0/447079818 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c011d20 con 0x7f1f4c12cf60 2023-10-03T10:14:33.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.295+0000 7f1f544c8700 1 -- 172.21.15.3:0/447079818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f4c12cf60 msgr2=0x7f1f4c06f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:33.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.295+0000 7f1f544c8700 1 --2- 172.21.15.3:0/447079818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f4c12cf60 0x7f1f4c06f5d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009920 tx=0x7f1f3c009c30 comp rx=0 tx=0).stop 2023-10-03T10:14:33.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.295+0000 7f1f544c8700 1 -- 172.21.15.3:0/447079818 shutdown_connections 2023-10-03T10:14:33.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.295+0000 7f1f544c8700 1 --2- 172.21.15.3:0/447079818 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1f4c130260 0x7f1f4c136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.295+0000 7f1f544c8700 1 --2- 172.21.15.3:0/447079818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f4c12cf60 0x7f1f4c06f5d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.295+0000 7f1f544c8700 1 --2- 172.21.15.3:0/447079818 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1f4c12f8b0 0x7f1f4c12fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.295+0000 7f1f544c8700 1 -- 172.21.15.3:0/447079818 >> 172.21.15.3:0/447079818 conn(0x7f1f4c0a2aa0 msgr2=0x7f1f4c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:33.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.295+0000 7f1f544c8700 1 -- 172.21.15.3:0/447079818 shutdown_connections 2023-10-03T10:14:33.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.295+0000 7f1f544c8700 1 -- 172.21.15.3:0/447079818 wait complete. 2023-10-03T10:14:33.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.296+0000 7f1f544c8700 1 Processor -- start 2023-10-03T10:14:33.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.297+0000 7f1f544c8700 1 -- start start 2023-10-03T10:14:33.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.297+0000 7f1f544c8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f4c12cf60 0x7f1f4c09ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:33.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.297+0000 7f1f544c8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1f4c12f8b0 0x7f1f4c09b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:33.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.297+0000 7f1f544c8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1f4c130260 0x7f1f4c09b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:33.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.297+0000 7f1f544c8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c129fa0 con 0x7f1f4c12cf60 2023-10-03T10:14:33.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.297+0000 7f1f544c8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c09ca20 con 0x7f1f4c12f8b0 2023-10-03T10:14:33.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.297+0000 7f1f544c8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c12f520 con 0x7f1f4c130260 2023-10-03T10:14:33.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.298+0000 7f1f51a63700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1f4c12f8b0 0x7f1f4c09b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:33.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.298+0000 7f1f51a63700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1f4c12f8b0 0x7f1f4c09b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50744/0 (socket says 172.21.15.3:50744) 2023-10-03T10:14:33.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.298+0000 7f1f51a63700 1 -- 172.21.15.3:0/2318968990 learned_addr learned my addr 172.21.15.3:0/2318968990 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:33.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.298+0000 7f1f52a65700 1 --2- 172.21.15.3:0/2318968990 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1f4c130260 0x7f1f4c09b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:33.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.298+0000 7f1f52264700 1 --2- 172.21.15.3:0/2318968990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f4c12cf60 0x7f1f4c09ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:33.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.298+0000 7f1f51a63700 1 -- 172.21.15.3:0/2318968990 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1f4c130260 msgr2=0x7f1f4c09b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:33.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.298+0000 7f1f51a63700 1 --2- 172.21.15.3:0/2318968990 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1f4c130260 0x7f1f4c09b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.298+0000 7f1f51a63700 1 -- 172.21.15.3:0/2318968990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f4c12cf60 msgr2=0x7f1f4c09ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:33.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.298+0000 7f1f51a63700 1 --2- 172.21.15.3:0/2318968990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f4c12cf60 0x7f1f4c09ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.298+0000 7f1f51a63700 1 -- 172.21.15.3:0/2318968990 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4400ab80 con 0x7f1f4c12f8b0 2023-10-03T10:14:33.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.298+0000 7f1f52264700 1 --2- 172.21.15.3:0/2318968990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f4c12cf60 0x7f1f4c09ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:14:33.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.299+0000 7f1f51a63700 1 --2- 172.21.15.3:0/2318968990 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1f4c12f8b0 0x7f1f4c09b2b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800b530 tx=0x7f1f4800b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:33.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.299+0000 7f1f42ffd700 1 -- 172.21.15.3:0/2318968990 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f48011830 con 0x7f1f4c12f8b0 2023-10-03T10:14:33.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.299+0000 7f1f42ffd700 1 -- 172.21.15.3:0/2318968990 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f48011990 con 0x7f1f4c12f8b0 2023-10-03T10:14:33.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.299+0000 7f1f42ffd700 1 -- 172.21.15.3:0/2318968990 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f48011d20 con 0x7f1f4c12f8b0 2023-10-03T10:14:33.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.299+0000 7f1f544c8700 1 -- 172.21.15.3:0/2318968990 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f3c009550 con 0x7f1f4c12f8b0 2023-10-03T10:14:33.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.299+0000 7f1f544c8700 1 -- 172.21.15.3:0/2318968990 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f4c129c00 con 0x7f1f4c12f8b0 2023-10-03T10:14:33.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.300+0000 7f1f42ffd700 1 -- 172.21.15.3:0/2318968990 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1f4801a400 con 0x7f1f4c12f8b0 2023-10-03T10:14:33.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.300+0000 7f1f42ffd700 1 -- 172.21.15.3:0/2318968990 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1f4803f790 con 0x7f1f4c12f8b0 2023-10-03T10:14:33.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.300+0000 7f1f544c8700 1 -- 172.21.15.3:0/2318968990 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f4c09e3f0 con 0x7f1f4c12f8b0 2023-10-03T10:14:33.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.307+0000 7f1f42ffd700 1 -- 172.21.15.3:0/2318968990 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1f48015080 con 0x7f1f4c12f8b0 2023-10-03T10:14:33.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.338+0000 7ff46e3de700 1 Processor -- start 2023-10-03T10:14:33.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.338+0000 7ff46e3de700 1 -- start start 2023-10-03T10:14:33.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.339+0000 7ff46e3de700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff4680a0a30 0x7ff4680a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:33.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.339+0000 7ff46e3de700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff468138d80 0x7ff46813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:33.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.339+0000 7ff46e3de700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4680a2a70 con 0x7ff46809d630 2023-10-03T10:14:33.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.339+0000 7ff46e3de700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4680a01d0 con 0x7ff468138d80 2023-10-03T10:14:33.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.339+0000 7ff46e3de700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff46809e2b0 con 0x7ff4680a0a30 2023-10-03T10:14:33.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.339+0000 7ff467fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff4680a0a30 0x7ff4680a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:33.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.339+0000 7ff46c97b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff468138d80 0x7ff46813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:33.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.339+0000 7ff467fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff4680a0a30 0x7ff4680a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50756/0 (socket says 172.21.15.3:50756) 2023-10-03T10:14:33.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.339+0000 7ff467fff700 1 -- 172.21.15.3:0/315393224 learned_addr learned my addr 172.21.15.3:0/315393224 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:33.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.339+0000 7ff46c97b700 1 -- 172.21.15.3:0/315393224 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff4680a0a30 msgr2=0x7ff4680a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:33.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.339+0000 7ff46c97b700 1 --2- 172.21.15.3:0/315393224 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff4680a0a30 0x7ff4680a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.339+0000 7ff46c97b700 1 -- 172.21.15.3:0/315393224 >> v1:172.21.15.3:6789/0 conn(0x7ff46809d630 legacy=0x7ff4680a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:33.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.339+0000 7ff466ffd700 1 -- 172.21.15.3:0/315393224 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2171012537 0 0) 0x7ff4680a2a70 con 0x7ff46809d630 2023-10-03T10:14:33.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.339+0000 7ff46c97b700 1 -- 172.21.15.3:0/315393224 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff46809dc50 con 0x7ff468138d80 2023-10-03T10:14:33.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.340+0000 7ff46c97b700 1 --2- 172.21.15.3:0/315393224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff468138d80 0x7ff46813b160 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff460009d90 tx=0x7ff46000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00e2988c35c3072 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:33.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.340+0000 7ff466ffd700 1 -- 172.21.15.3:0/315393224 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46000e070 con 0x7ff468138d80 2023-10-03T10:14:33.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.340+0000 7ff466ffd700 1 -- 172.21.15.3:0/315393224 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46000bb20 con 0x7ff468138d80 2023-10-03T10:14:33.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.340+0000 7ff466ffd700 1 -- 172.21.15.3:0/315393224 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff460012430 con 0x7ff468138d80 2023-10-03T10:14:33.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.340+0000 7ff46e3de700 1 -- 172.21.15.3:0/315393224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff468138d80 msgr2=0x7ff46813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:33.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.340+0000 7ff46e3de700 1 --2- 172.21.15.3:0/315393224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff468138d80 0x7ff46813b160 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff460009d90 tx=0x7ff46000b330 comp rx=0 tx=0).stop 2023-10-03T10:14:33.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.340+0000 7ff46e3de700 1 -- 172.21.15.3:0/315393224 shutdown_connections 2023-10-03T10:14:33.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.340+0000 7ff46e3de700 1 --2- 172.21.15.3:0/315393224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff468138d80 0x7ff46813b160 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.340+0000 7ff46e3de700 1 --2- 172.21.15.3:0/315393224 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff4680a0a30 0x7ff4680a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.340+0000 7ff46e3de700 1 -- 172.21.15.3:0/315393224 >> 172.21.15.3:0/315393224 conn(0x7ff4680a3050 msgr2=0x7ff4680a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:33.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.341+0000 7ff46e3de700 1 -- 172.21.15.3:0/315393224 shutdown_connections 2023-10-03T10:14:33.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.341+0000 7ff46e3de700 1 -- 172.21.15.3:0/315393224 wait complete. 2023-10-03T10:14:33.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.342+0000 7ff46e3de700 1 Processor -- start 2023-10-03T10:14:33.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.342+0000 7ff46e3de700 1 -- start start 2023-10-03T10:14:33.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.343+0000 7ff46e3de700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff46809d630 0x7ff4681336f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:33.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.343+0000 7ff46e3de700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff4680a0a30 0x7ff468133c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:33.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.343+0000 7ff46e3de700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff468138d80 0x7ff468134170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:33.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.343+0000 7ff46e3de700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff468136130 con 0x7ff46809d630 2023-10-03T10:14:33.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.343+0000 7ff46e3de700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff468136e50 con 0x7ff4680a0a30 2023-10-03T10:14:33.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.343+0000 7ff46e3de700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff4680a4b90 con 0x7ff468138d80 2023-10-03T10:14:33.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.343+0000 7ff4677fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff4680a0a30 0x7ff468133c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:33.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.343+0000 7ff4677fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff4680a0a30 0x7ff468133c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50764/0 (socket says 172.21.15.3:50764) 2023-10-03T10:14:33.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.343+0000 7ff4677fe700 1 -- 172.21.15.3:0/2373008858 learned_addr learned my addr 172.21.15.3:0/2373008858 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:33.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.343+0000 7ff467fff700 1 --2- 172.21.15.3:0/2373008858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff46809d630 0x7ff4681336f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:33.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.343+0000 7ff46c97b700 1 --2- 172.21.15.3:0/2373008858 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff468138d80 0x7ff468134170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:33.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.344+0000 7ff46c97b700 1 -- 172.21.15.3:0/2373008858 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff4680a0a30 msgr2=0x7ff468133c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:33.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.344+0000 7ff46c97b700 1 --2- 172.21.15.3:0/2373008858 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff4680a0a30 0x7ff468133c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.344+0000 7ff46c97b700 1 -- 172.21.15.3:0/2373008858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff46809d630 msgr2=0x7ff4681336f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:33.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.344+0000 7ff46c97b700 1 --2- 172.21.15.3:0/2373008858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff46809d630 0x7ff4681336f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.344+0000 7ff46c97b700 1 -- 172.21.15.3:0/2373008858 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff460009a70 con 0x7ff468138d80 2023-10-03T10:14:33.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.344+0000 7ff467fff700 1 --2- 172.21.15.3:0/2373008858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff46809d630 0x7ff4681336f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:33.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.344+0000 7ff46c97b700 1 --2- 172.21.15.3:0/2373008858 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff468138d80 0x7ff468134170 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff460016000 tx=0x7ff460004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:33.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.344+0000 7ff464ff9700 1 -- 172.21.15.3:0/2373008858 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff460012430 con 0x7ff468138d80 2023-10-03T10:14:33.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.344+0000 7ff464ff9700 1 -- 172.21.15.3:0/2373008858 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff460020070 con 0x7ff468138d80 2023-10-03T10:14:33.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.344+0000 7ff46e3de700 1 -- 172.21.15.3:0/2373008858 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4680a0410 con 0x7ff468138d80 2023-10-03T10:14:33.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.344+0000 7ff464ff9700 1 -- 172.21.15.3:0/2373008858 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff460004280 con 0x7ff468138d80 2023-10-03T10:14:33.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.344+0000 7ff46e3de700 1 -- 172.21.15.3:0/2373008858 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4681378c0 con 0x7ff468138d80 2023-10-03T10:14:33.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.345+0000 7ff464ff9700 1 -- 172.21.15.3:0/2373008858 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff4600043e0 con 0x7ff468138d80 2023-10-03T10:14:33.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.345+0000 7ff464ff9700 1 -- 172.21.15.3:0/2373008858 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff460022070 con 0x7ff468138d80 2023-10-03T10:14:33.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.345+0000 7ff46e3de700 1 -- 172.21.15.3:0/2373008858 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4680747f0 con 0x7ff468138d80 2023-10-03T10:14:33.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.350+0000 7ff464ff9700 1 -- 172.21.15.3:0/2373008858 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff460004690 con 0x7ff468138d80 2023-10-03T10:14:33.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.465+0000 7fb532145700 1 -- 172.21.15.3:0/891400367 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb52c11ee20 con 0x7fb52c133e30 2023-10-03T10:14:33.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.465+0000 7fb513fff700 1 -- 172.21.15.3:0/891400367 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb520011d90 con 0x7fb52c133e30 2023-10-03T10:14:33.466 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:33.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.468+0000 7fb532145700 1 -- 172.21.15.3:0/891400367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb52c133e30 msgr2=0x7fb52c09c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:33.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.468+0000 7fb532145700 1 --2- 172.21.15.3:0/891400367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb52c133e30 0x7fb52c09c9c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb52000ec10 tx=0x7fb52000ef20 comp rx=0 tx=0).stop 2023-10-03T10:14:33.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.468+0000 7fb532145700 1 -- 172.21.15.3:0/891400367 shutdown_connections 2023-10-03T10:14:33.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.468+0000 7fb532145700 1 --2- 172.21.15.3:0/891400367 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb52c134260 0x7fb52c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.468+0000 7fb532145700 1 --2- 172.21.15.3:0/891400367 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb52c133420 0x7fb52c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.468+0000 7fb532145700 1 --2- 172.21.15.3:0/891400367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb52c133e30 0x7fb52c09c9c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.468+0000 7fb532145700 1 -- 172.21.15.3:0/891400367 >> 172.21.15.3:0/891400367 conn(0x7fb52c0a4210 msgr2=0x7fb52c130250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:33.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.468+0000 7fb532145700 1 -- 172.21.15.3:0/891400367 shutdown_connections 2023-10-03T10:14:33.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.468+0000 7fb532145700 1 -- 172.21.15.3:0/891400367 wait complete. 2023-10-03T10:14:33.483 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:33.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.517+0000 7f1f544c8700 1 -- 172.21.15.3:0/2318968990 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1f4c074ea0 con 0x7f1f4c12f8b0 2023-10-03T10:14:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.518+0000 7f1f42ffd700 1 -- 172.21.15.3:0/2318968990 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1f4800f8d0 con 0x7f1f4c12f8b0 2023-10-03T10:14:33.518 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.520+0000 7f1f544c8700 1 -- 172.21.15.3:0/2318968990 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1f4c12f8b0 msgr2=0x7f1f4c09b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.520+0000 7f1f544c8700 1 --2- 172.21.15.3:0/2318968990 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1f4c12f8b0 0x7f1f4c09b2b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800b530 tx=0x7f1f4800b840 comp rx=0 tx=0).stop 2023-10-03T10:14:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.520+0000 7f1f544c8700 1 -- 172.21.15.3:0/2318968990 shutdown_connections 2023-10-03T10:14:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.520+0000 7f1f544c8700 1 --2- 172.21.15.3:0/2318968990 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1f4c130260 0x7f1f4c09b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.520+0000 7f1f544c8700 1 --2- 172.21.15.3:0/2318968990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f4c12cf60 0x7f1f4c09ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.520+0000 7f1f544c8700 1 --2- 172.21.15.3:0/2318968990 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1f4c12f8b0 0x7f1f4c09b2b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.520+0000 7f1f544c8700 1 -- 172.21.15.3:0/2318968990 >> 172.21.15.3:0/2318968990 conn(0x7f1f4c0a2aa0 msgr2=0x7f1f4c06e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.520+0000 7f1f544c8700 1 -- 172.21.15.3:0/2318968990 shutdown_connections 2023-10-03T10:14:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.520+0000 7f1f544c8700 1 -- 172.21.15.3:0/2318968990 wait complete. 2023-10-03T10:14:33.534 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:33.544 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:33.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.556+0000 7ff46e3de700 1 -- 172.21.15.3:0/2373008858 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff468072980 con 0x7ff468138d80 2023-10-03T10:14:33.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.556+0000 7ff464ff9700 1 -- 172.21.15.3:0/2373008858 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff46002fe60 con 0x7ff468138d80 2023-10-03T10:14:33.557 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:33.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.559+0000 7ff46e3de700 1 -- 172.21.15.3:0/2373008858 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff468138d80 msgr2=0x7ff468134170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:33.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.559+0000 7ff46e3de700 1 --2- 172.21.15.3:0/2373008858 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff468138d80 0x7ff468134170 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff460016000 tx=0x7ff460004ad0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.559+0000 7ff46e3de700 1 -- 172.21.15.3:0/2373008858 shutdown_connections 2023-10-03T10:14:33.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.559+0000 7ff46e3de700 1 --2- 172.21.15.3:0/2373008858 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff468138d80 0x7ff468134170 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.559+0000 7ff46e3de700 1 --2- 172.21.15.3:0/2373008858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff46809d630 0x7ff4681336f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.559+0000 7ff46e3de700 1 --2- 172.21.15.3:0/2373008858 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff4680a0a30 0x7ff468133c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.559+0000 7ff46e3de700 1 -- 172.21.15.3:0/2373008858 >> 172.21.15.3:0/2373008858 conn(0x7ff4680a3050 msgr2=0x7ff46809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:33.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.559+0000 7ff46e3de700 1 -- 172.21.15.3:0/2373008858 shutdown_connections 2023-10-03T10:14:33.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.559+0000 7ff46e3de700 1 -- 172.21.15.3:0/2373008858 wait complete. 2023-10-03T10:14:33.573 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:33.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.693+0000 7f4fbac09700 1 Processor -- start 2023-10-03T10:14:33.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.693+0000 7f4fbac09700 1 -- start start 2023-10-03T10:14:33.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.693+0000 7f4fbac09700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4fb4130750 0x7f4fb4130b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:33.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.693+0000 7f4fbac09700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fb409d6f0 0x7f4fb409db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:33.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.693+0000 7f4fbac09700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb409c190 con 0x7f4fb409e080 2023-10-03T10:14:33.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.693+0000 7f4fbac09700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fb409b090 con 0x7f4fb4130750 2023-10-03T10:14:33.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.693+0000 7f4fbac09700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb409a6c0 con 0x7f4fb409d6f0 2023-10-03T10:14:33.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.693+0000 7f4fb3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fb409d6f0 0x7f4fb409db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:33.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.693+0000 7f4fb3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fb409d6f0 0x7f4fb409db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:60888/0 (socket says 172.21.15.3:60888) 2023-10-03T10:14:33.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.693+0000 7f4fb3fff700 1 -- 172.21.15.3:0/1985564770 learned_addr learned my addr 172.21.15.3:0/1985564770 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:33.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.693+0000 7f4fb89a5700 1 --2- 172.21.15.3:0/1985564770 >> v2:172.21.15.3:3300/0 conn(0x7f4fb4130750 0x7f4fb4130b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.694+0000 7f4fb89a5700 1 -- 172.21.15.3:0/1985564770 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fb409d6f0 msgr2=0x7f4fb409db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.694+0000 7f4fb89a5700 1 --2- 172.21.15.3:0/1985564770 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fb409d6f0 0x7f4fb409db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.694+0000 7f4fb89a5700 1 -- 172.21.15.3:0/1985564770 >> v1:172.21.15.3:6789/0 conn(0x7f4fb409e080 legacy=0x7f4fb41374b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.694+0000 7f4fb89a5700 1 -- 172.21.15.3:0/1985564770 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb409e500 con 0x7f4fb4130750 2023-10-03T10:14:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.694+0000 7f4fb3fff700 1 --2- 172.21.15.3:0/1985564770 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fb409d6f0 0x7f4fb409db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.694+0000 7f4fb89a5700 1 --2- 172.21.15.3:0/1985564770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fb4130750 0x7f4fb4130b20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8009840 tx=0x7f4fa8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b5bed3bd31653e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.694+0000 7f4fb37fe700 1 -- 172.21.15.3:0/1985564770 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8011830 con 0x7f4fb4130750 2023-10-03T10:14:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.694+0000 7f4fb37fe700 1 -- 172.21.15.3:0/1985564770 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa8011990 con 0x7f4fb4130750 2023-10-03T10:14:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.694+0000 7f4fb37fe700 1 -- 172.21.15.3:0/1985564770 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8011d20 con 0x7f4fb4130750 2023-10-03T10:14:33.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.695+0000 7f4fbac09700 1 -- 172.21.15.3:0/1985564770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fb4130750 msgr2=0x7f4fb4130b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:33.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.695+0000 7f4fbac09700 1 --2- 172.21.15.3:0/1985564770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fb4130750 0x7f4fb4130b20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8009840 tx=0x7f4fa8009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:33.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.695+0000 7f4fbac09700 1 -- 172.21.15.3:0/1985564770 shutdown_connections 2023-10-03T10:14:33.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.695+0000 7f4fbac09700 1 --2- 172.21.15.3:0/1985564770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fb4130750 0x7f4fb4130b20 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.695+0000 7f4fbac09700 1 --2- 172.21.15.3:0/1985564770 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fb409d6f0 0x7f4fb409db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.695+0000 7f4fbac09700 1 -- 172.21.15.3:0/1985564770 >> 172.21.15.3:0/1985564770 conn(0x7f4fb409f470 msgr2=0x7f4fb40a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:33.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.695+0000 7f4fbac09700 1 -- 172.21.15.3:0/1985564770 shutdown_connections 2023-10-03T10:14:33.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.695+0000 7f4fbac09700 1 -- 172.21.15.3:0/1985564770 wait complete. 2023-10-03T10:14:33.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.696+0000 7f4fbac09700 1 Processor -- start 2023-10-03T10:14:33.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.696+0000 7f4fbac09700 1 -- start start 2023-10-03T10:14:33.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.697+0000 7f4fbac09700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fb409d6f0 0x7f4fb41c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:33.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.697+0000 7f4fbac09700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fb409e080 0x7f4fb41c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:33.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.697+0000 7f4fbac09700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fb4130750 0x7f4fb41c6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:33.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.697+0000 7f4fbac09700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb408c0d0 con 0x7f4fb409d6f0 2023-10-03T10:14:33.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.697+0000 7f4fbac09700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb408d240 con 0x7f4fb409e080 2023-10-03T10:14:33.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.697+0000 7f4fbac09700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb408c5d0 con 0x7f4fb4130750 2023-10-03T10:14:33.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.697+0000 7f4fb89a5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fb409d6f0 0x7f4fb41c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:33.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.697+0000 7f4fb89a5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fb409d6f0 0x7f4fb41c6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49872/0 (socket says 172.21.15.3:49872) 2023-10-03T10:14:33.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.697+0000 7f4fb89a5700 1 -- 172.21.15.3:0/1326130140 learned_addr learned my addr 172.21.15.3:0/1326130140 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:33.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.697+0000 7f4fb91a6700 1 --2- 172.21.15.3:0/1326130140 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fb4130750 0x7f4fb41c6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:33.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.697+0000 7f4fb3fff700 1 --2- 172.21.15.3:0/1326130140 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fb409e080 0x7f4fb41c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.697+0000 7f4fb91a6700 1 -- 172.21.15.3:0/1326130140 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fb409e080 msgr2=0x7f4fb41c67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.697+0000 7f4fb91a6700 1 --2- 172.21.15.3:0/1326130140 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fb409e080 0x7f4fb41c67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.697+0000 7f4fb91a6700 1 -- 172.21.15.3:0/1326130140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fb409d6f0 msgr2=0x7f4fb41c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.697+0000 7f4fb91a6700 1 --2- 172.21.15.3:0/1326130140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fb409d6f0 0x7f4fb41c6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.697+0000 7f4fb91a6700 1 -- 172.21.15.3:0/1326130140 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa0009550 con 0x7f4fb4130750 2023-10-03T10:14:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.697+0000 7f4fb89a5700 1 --2- 172.21.15.3:0/1326130140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fb409d6f0 0x7f4fb41c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.697+0000 7f4fb91a6700 1 --2- 172.21.15.3:0/1326130140 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fb4130750 0x7f4fb41c6d10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4fac00cbc0 tx=0x7f4fac00ced0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.697+0000 7f4fb17fa700 1 -- 172.21.15.3:0/1326130140 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fac003c00 con 0x7f4fb4130750 2023-10-03T10:14:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.697+0000 7f4fb17fa700 1 -- 172.21.15.3:0/1326130140 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fac003d60 con 0x7f4fb4130750 2023-10-03T10:14:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.697+0000 7f4fb17fa700 1 -- 172.21.15.3:0/1326130140 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fac00aa40 con 0x7f4fb4130750 2023-10-03T10:14:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.697+0000 7f4fbac09700 1 -- 172.21.15.3:0/1326130140 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fa80094a0 con 0x7f4fb4130750 2023-10-03T10:14:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.698+0000 7f4fbac09700 1 -- 172.21.15.3:0/1326130140 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb408c8b0 con 0x7f4fb4130750 2023-10-03T10:14:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.698+0000 7f4fb17fa700 1 -- 172.21.15.3:0/1326130140 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4fac00aba0 con 0x7f4fb4130750 2023-10-03T10:14:33.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.698+0000 7f4fbac09700 1 -- 172.21.15.3:0/1326130140 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb408bf10 con 0x7f4fb4130750 2023-10-03T10:14:33.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.699+0000 7f4fb17fa700 1 -- 172.21.15.3:0/1326130140 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4fac03f6c0 con 0x7f4fb4130750 2023-10-03T10:14:33.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.704+0000 7f4fb17fa700 1 -- 172.21.15.3:0/1326130140 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4fac012420 con 0x7f4fb4130750 2023-10-03T10:14:33.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.907+0000 7f4fbac09700 1 -- 172.21.15.3:0/1326130140 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4fb40747f0 con 0x7f4fb4130750 2023-10-03T10:14:33.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.908+0000 7f4fb17fa700 1 -- 172.21.15.3:0/1326130140 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4fac01a360 con 0x7f4fb4130750 2023-10-03T10:14:33.908 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:33.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.910+0000 7f4fbac09700 1 -- 172.21.15.3:0/1326130140 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fb4130750 msgr2=0x7f4fb41c6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:33.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.910+0000 7f4fbac09700 1 --2- 172.21.15.3:0/1326130140 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fb4130750 0x7f4fb41c6d10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4fac00cbc0 tx=0x7f4fac00ced0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.910+0000 7f4fbac09700 1 -- 172.21.15.3:0/1326130140 shutdown_connections 2023-10-03T10:14:33.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.910+0000 7f4fbac09700 1 --2- 172.21.15.3:0/1326130140 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fb4130750 0x7f4fb41c6d10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.910+0000 7f4fbac09700 1 --2- 172.21.15.3:0/1326130140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fb409d6f0 0x7f4fb41c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.910+0000 7f4fbac09700 1 --2- 172.21.15.3:0/1326130140 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fb409e080 0x7f4fb41c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:33.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.910+0000 7f4fbac09700 1 -- 172.21.15.3:0/1326130140 >> 172.21.15.3:0/1326130140 conn(0x7f4fb409f470 msgr2=0x7f4fb40992c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:33.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.910+0000 7f4fbac09700 1 -- 172.21.15.3:0/1326130140 shutdown_connections 2023-10-03T10:14:33.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:33.910+0000 7f4fbac09700 1 -- 172.21.15.3:0/1326130140 wait complete. 2023-10-03T10:14:33.924 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:34.484 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:34.535 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:34.574 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:34.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.632+0000 7f5c81fd5700 1 Processor -- start 2023-10-03T10:14:34.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.632+0000 7f5c81fd5700 1 -- start start 2023-10-03T10:14:34.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.633+0000 7f5c81fd5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c7c09f410 0x7f5c7c136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:34.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.633+0000 7f5c81fd5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c7c136fe0 0x7f5c7c1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:34.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.633+0000 7f5c81fd5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c7c070940 con 0x7f5c7c09e930 2023-10-03T10:14:34.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.633+0000 7f5c81fd5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c7c06fd60 con 0x7f5c7c09f410 2023-10-03T10:14:34.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.633+0000 7f5c81fd5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c06fa80 con 0x7f5c7c136fe0 2023-10-03T10:14:34.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.633+0000 7f5c7affd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c7c09f410 0x7f5c7c136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:34.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.633+0000 7f5c7bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c7c136fe0 0x7f5c7c1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:34.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.633+0000 7f5c7affd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c7c09f410 0x7f5c7c136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49874/0 (socket says 172.21.15.3:49874) 2023-10-03T10:14:34.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.633+0000 7f5c7affd700 1 -- 172.21.15.3:0/2833436553 learned_addr learned my addr 172.21.15.3:0/2833436553 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:34.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.633+0000 7f5c7affd700 1 -- 172.21.15.3:0/2833436553 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c7c136fe0 msgr2=0x7f5c7c1393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:34.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.633+0000 7f5c7affd700 1 --2- 172.21.15.3:0/2833436553 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c7c136fe0 0x7f5c7c1393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:34.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.633+0000 7f5c7affd700 1 -- 172.21.15.3:0/2833436553 >> v1:172.21.15.3:6789/0 conn(0x7f5c7c09e930 legacy=0x7f5c7c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:34.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.633+0000 7f5c7affd700 1 -- 172.21.15.3:0/2833436553 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7c09fb60 con 0x7f5c7c09f410 2023-10-03T10:14:34.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.633+0000 7f5c7bfff700 1 --2- 172.21.15.3:0/2833436553 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c7c136fe0 0x7f5c7c1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:34.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.634+0000 7f5c7affd700 1 --2- 172.21.15.3:0/2833436553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c7c09f410 0x7f5c7c136aa0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5c700098f0 tx=0x7f5c70009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7600597c4c41fe server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:34.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.634+0000 7f5c7a7fc700 1 -- 172.21.15.3:0/2833436553 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70011830 con 0x7f5c7c09f410 2023-10-03T10:14:34.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.634+0000 7f5c7a7fc700 1 -- 172.21.15.3:0/2833436553 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c70011990 con 0x7f5c7c09f410 2023-10-03T10:14:34.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.634+0000 7f5c7a7fc700 1 -- 172.21.15.3:0/2833436553 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70011d20 con 0x7f5c7c09f410 2023-10-03T10:14:34.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.634+0000 7f5c81fd5700 1 -- 172.21.15.3:0/2833436553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c7c09f410 msgr2=0x7f5c7c136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:34.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.635+0000 7f5c81fd5700 1 --2- 172.21.15.3:0/2833436553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c7c09f410 0x7f5c7c136aa0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5c700098f0 tx=0x7f5c70009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:34.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.635+0000 7f5c81fd5700 1 -- 172.21.15.3:0/2833436553 shutdown_connections 2023-10-03T10:14:34.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.635+0000 7f5c81fd5700 1 --2- 172.21.15.3:0/2833436553 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c7c136fe0 0x7f5c7c1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:34.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.635+0000 7f5c81fd5700 1 --2- 172.21.15.3:0/2833436553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c7c09f410 0x7f5c7c136aa0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:34.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.635+0000 7f5c81fd5700 1 -- 172.21.15.3:0/2833436553 >> 172.21.15.3:0/2833436553 conn(0x7f5c7c0a4210 msgr2=0x7f5c7c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:34.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.635+0000 7f5c81fd5700 1 -- 172.21.15.3:0/2833436553 shutdown_connections 2023-10-03T10:14:34.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.635+0000 7f5c81fd5700 1 -- 172.21.15.3:0/2833436553 wait complete. 2023-10-03T10:14:34.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.636+0000 7f5c81fd5700 1 Processor -- start 2023-10-03T10:14:34.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.636+0000 7f5c81fd5700 1 -- start start 2023-10-03T10:14:34.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.636+0000 7f5c81fd5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c7c09e930 0x7f5c7c1c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:34.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.636+0000 7f5c81fd5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c7c09f410 0x7f5c7c1c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:34.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.636+0000 7f5c81fd5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c7c136fe0 0x7f5c7c1c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:34.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.636+0000 7f5c81fd5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c071d20 con 0x7f5c7c136fe0 2023-10-03T10:14:34.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.636+0000 7f5c81fd5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c122180 con 0x7f5c7c09e930 2023-10-03T10:14:34.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.636+0000 7f5c81fd5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c06d4c0 con 0x7f5c7c09f410 2023-10-03T10:14:34.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.637+0000 7f5c7bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c7c136fe0 0x7f5c7c1c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:34.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.637+0000 7f5c7b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c7c09e930 0x7f5c7c1c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:34.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.637+0000 7f5c7b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c7c09e930 0x7f5c7c1c8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50786/0 (socket says 172.21.15.3:50786) 2023-10-03T10:14:34.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.637+0000 7f5c7bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c7c136fe0 0x7f5c7c1c8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49888/0 (socket says 172.21.15.3:49888) 2023-10-03T10:14:34.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.637+0000 7f5c7b7fe700 1 -- 172.21.15.3:0/605447313 learned_addr learned my addr 172.21.15.3:0/605447313 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:34.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.637+0000 7f5c7affd700 1 --2- 172.21.15.3:0/605447313 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c7c09f410 0x7f5c7c1c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:34.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.637+0000 7f5c7bfff700 1 -- 172.21.15.3:0/605447313 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c7c09f410 msgr2=0x7f5c7c1c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:34.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.637+0000 7f5c7bfff700 1 --2- 172.21.15.3:0/605447313 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c7c09f410 0x7f5c7c1c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:34.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.637+0000 7f5c7bfff700 1 -- 172.21.15.3:0/605447313 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c7c09e930 msgr2=0x7f5c7c1c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:34.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.637+0000 7f5c7bfff700 1 --2- 172.21.15.3:0/605447313 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c7c09e930 0x7f5c7c1c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:34.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.637+0000 7f5c7bfff700 1 -- 172.21.15.3:0/605447313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c74009a70 con 0x7f5c7c136fe0 2023-10-03T10:14:34.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.637+0000 7f5c7b7fe700 1 --2- 172.21.15.3:0/605447313 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c7c09e930 0x7f5c7c1c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:34.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.637+0000 7f5c7affd700 1 --2- 172.21.15.3:0/605447313 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c7c09f410 0x7f5c7c1c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:34.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.637+0000 7f5c7bfff700 1 --2- 172.21.15.3:0/605447313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c7c136fe0 0x7f5c7c1c8ce0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400efc0 tx=0x7f5c7400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:34.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.638+0000 7f5c63fff700 1 -- 172.21.15.3:0/605447313 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7400ca90 con 0x7f5c7c136fe0 2023-10-03T10:14:34.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.638+0000 7f5c63fff700 1 -- 172.21.15.3:0/605447313 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7400f040 con 0x7f5c7c136fe0 2023-10-03T10:14:34.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.638+0000 7f5c81fd5700 1 -- 172.21.15.3:0/605447313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c70009550 con 0x7f5c7c136fe0 2023-10-03T10:14:34.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.638+0000 7f5c63fff700 1 -- 172.21.15.3:0/605447313 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7400ce30 con 0x7f5c7c136fe0 2023-10-03T10:14:34.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.638+0000 7f5c81fd5700 1 -- 172.21.15.3:0/605447313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7c120c60 con 0x7f5c7c136fe0 2023-10-03T10:14:34.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.639+0000 7f5c63fff700 1 -- 172.21.15.3:0/605447313 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5c74014480 con 0x7f5c7c136fe0 2023-10-03T10:14:34.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.639+0000 7f5c63fff700 1 -- 172.21.15.3:0/605447313 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5c7403fd20 con 0x7f5c7c136fe0 2023-10-03T10:14:34.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.639+0000 7f5c81fd5700 1 -- 172.21.15.3:0/605447313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c7c121140 con 0x7f5c7c136fe0 2023-10-03T10:14:34.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.648+0000 7f5c63fff700 1 -- 172.21.15.3:0/605447313 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5c74012910 con 0x7f5c7c136fe0 2023-10-03T10:14:34.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.683+0000 7ff756005700 1 Processor -- start 2023-10-03T10:14:34.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.684+0000 7ff756005700 1 -- start start 2023-10-03T10:14:34.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.684+0000 7ff756005700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff75009e1d0 0x7ff750138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:34.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.684+0000 7ff756005700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff750139170 0x7ff75013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:34.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.684+0000 7ff756005700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75009b120 con 0x7ff75009d6f0 2023-10-03T10:14:34.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.684+0000 7ff756005700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff75006b460 con 0x7ff75009e1d0 2023-10-03T10:14:34.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.684+0000 7ff756005700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff75009a7e0 con 0x7ff750139170 2023-10-03T10:14:34.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.684+0000 7ff74f7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff75009d6f0 0x7ff75009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40402/0 (socket says 172.21.15.3:40402) 2023-10-03T10:14:34.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.684+0000 7ff74f7fe700 1 -- 172.21.15.3:0/3972851812 learned_addr learned my addr 172.21.15.3:0/3972851812 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:34.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.684+0000 7ff74ffff700 1 --2- 172.21.15.3:0/3972851812 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff750139170 0x7ff75013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:34.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.685+0000 7ff74effd700 1 --2- 172.21.15.3:0/3972851812 >> v2:172.21.15.3:3300/0 conn(0x7ff75009e1d0 0x7ff750138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:34.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.685+0000 7ff74e7fc700 1 -- 172.21.15.3:0/3972851812 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1806579692 0 0) 0x7ff75009b120 con 0x7ff75009d6f0 2023-10-03T10:14:34.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.685+0000 7ff74e7fc700 1 -- 172.21.15.3:0/3972851812 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7380035b0 con 0x7ff75009d6f0 2023-10-03T10:14:34.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.685+0000 7ff74effd700 1 -- 172.21.15.3:0/3972851812 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff750139170 msgr2=0x7ff75013b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:34.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.685+0000 7ff74effd700 1 --2- 172.21.15.3:0/3972851812 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff750139170 0x7ff75013b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:34.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.685+0000 7ff74effd700 1 -- 172.21.15.3:0/3972851812 >> v1:172.21.15.3:6789/0 conn(0x7ff75009d6f0 legacy=0x7ff75009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:34.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.685+0000 7ff74effd700 1 -- 172.21.15.3:0/3972851812 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75009a500 con 0x7ff75009e1d0 2023-10-03T10:14:34.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.685+0000 7ff74effd700 1 --2- 172.21.15.3:0/3972851812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff75009e1d0 0x7ff750138c30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff744009870 tx=0x7ff744009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9a04d300ffc81e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:34.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.685+0000 7ff74e7fc700 1 -- 172.21.15.3:0/3972851812 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff744011830 con 0x7ff75009e1d0 2023-10-03T10:14:34.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.685+0000 7ff74e7fc700 1 -- 172.21.15.3:0/3972851812 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff744011990 con 0x7ff75009e1d0 2023-10-03T10:14:34.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.685+0000 7ff74e7fc700 1 -- 172.21.15.3:0/3972851812 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff744011d00 con 0x7ff75009e1d0 2023-10-03T10:14:34.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.686+0000 7ff756005700 1 -- 172.21.15.3:0/3972851812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff75009e1d0 msgr2=0x7ff750138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:34.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.686+0000 7ff756005700 1 --2- 172.21.15.3:0/3972851812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff75009e1d0 0x7ff750138c30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff744009870 tx=0x7ff744009b80 comp rx=0 tx=0).stop 2023-10-03T10:14:34.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.686+0000 7ff756005700 1 -- 172.21.15.3:0/3972851812 shutdown_connections 2023-10-03T10:14:34.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.686+0000 7ff756005700 1 --2- 172.21.15.3:0/3972851812 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff750139170 0x7ff75013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:34.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.686+0000 7ff756005700 1 --2- 172.21.15.3:0/3972851812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff75009e1d0 0x7ff750138c30 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:34.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.686+0000 7ff756005700 1 -- 172.21.15.3:0/3972851812 >> 172.21.15.3:0/3972851812 conn(0x7ff750066f50 msgr2=0x7ff75009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:34.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.686+0000 7ff756005700 1 -- 172.21.15.3:0/3972851812 shutdown_connections 2023-10-03T10:14:34.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.686+0000 7ff756005700 1 -- 172.21.15.3:0/3972851812 wait complete. 2023-10-03T10:14:34.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.687+0000 7ff756005700 1 Processor -- start 2023-10-03T10:14:34.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.688+0000 7ff756005700 1 -- start start 2023-10-03T10:14:34.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.688+0000 7ff756005700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff75009d6f0 0x7ff750121760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:34.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.688+0000 7ff756005700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff75009e1d0 0x7ff750121ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:34.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.688+0000 7ff756005700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff750139170 0x7ff7501221e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:34.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.688+0000 7ff756005700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff750123ac0 con 0x7ff750139170 2023-10-03T10:14:34.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.688+0000 7ff756005700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7501233b0 con 0x7ff75009e1d0 2023-10-03T10:14:34.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.688+0000 7ff756005700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff750124c10 con 0x7ff75009d6f0 2023-10-03T10:14:34.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.688+0000 7ff74f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff75009d6f0 0x7ff750121760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:34.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.688+0000 7ff74effd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff75009e1d0 0x7ff750121ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:34.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.688+0000 7ff74f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff75009d6f0 0x7ff750121760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:60930/0 (socket says 172.21.15.3:60930) 2023-10-03T10:14:34.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.688+0000 7ff74effd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff75009e1d0 0x7ff750121ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50796/0 (socket says 172.21.15.3:50796) 2023-10-03T10:14:34.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.688+0000 7ff74f7fe700 1 -- 172.21.15.3:0/3042899047 learned_addr learned my addr 172.21.15.3:0/3042899047 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:34.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.688+0000 7ff74ffff700 1 --2- 172.21.15.3:0/3042899047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff750139170 0x7ff7501221e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:34.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.689+0000 7ff74f7fe700 1 -- 172.21.15.3:0/3042899047 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff75009e1d0 msgr2=0x7ff750121ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:34.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.689+0000 7ff74f7fe700 1 --2- 172.21.15.3:0/3042899047 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff75009e1d0 0x7ff750121ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:34.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.689+0000 7ff74f7fe700 1 -- 172.21.15.3:0/3042899047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff750139170 msgr2=0x7ff7501221e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:34.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.689+0000 7ff74f7fe700 1 --2- 172.21.15.3:0/3042899047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff750139170 0x7ff7501221e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:34.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.689+0000 7ff74f7fe700 1 -- 172.21.15.3:0/3042899047 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7440094a0 con 0x7ff75009d6f0 2023-10-03T10:14:34.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.689+0000 7ff74effd700 1 --2- 172.21.15.3:0/3042899047 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff75009e1d0 0x7ff750121ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:34.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.689+0000 7ff74f7fe700 1 --2- 172.21.15.3:0/3042899047 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff75009d6f0 0x7ff750121760 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff74000c700 tx=0x7ff74000ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:34.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.689+0000 7ff737fff700 1 -- 172.21.15.3:0/3042899047 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff740005d70 con 0x7ff75009d6f0 2023-10-03T10:14:34.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.689+0000 7ff737fff700 1 -- 172.21.15.3:0/3042899047 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7400070d0 con 0x7ff75009d6f0 2023-10-03T10:14:34.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.689+0000 7ff756005700 1 -- 172.21.15.3:0/3042899047 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7500a1790 con 0x7ff75009d6f0 2023-10-03T10:14:34.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.689+0000 7ff737fff700 1 -- 172.21.15.3:0/3042899047 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff740007460 con 0x7ff75009d6f0 2023-10-03T10:14:34.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.689+0000 7ff756005700 1 -- 172.21.15.3:0/3042899047 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff750125940 con 0x7ff75009d6f0 2023-10-03T10:14:34.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.690+0000 7ff737fff700 1 -- 172.21.15.3:0/3042899047 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff7400075c0 con 0x7ff75009d6f0 2023-10-03T10:14:34.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.690+0000 7ff737fff700 1 -- 172.21.15.3:0/3042899047 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff74003eb40 con 0x7ff75009d6f0 2023-10-03T10:14:34.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.690+0000 7ff756005700 1 -- 172.21.15.3:0/3042899047 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff750074ea0 con 0x7ff75009d6f0 2023-10-03T10:14:34.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.695+0000 7ff737fff700 1 -- 172.21.15.3:0/3042899047 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff740017080 con 0x7ff75009d6f0 2023-10-03T10:14:34.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.730+0000 7f952bfae700 1 Processor -- start 2023-10-03T10:14:34.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.730+0000 7f952bfae700 1 -- start start 2023-10-03T10:14:34.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.731+0000 7f952bfae700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f95240a0090 0x7f9524136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:34.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.731+0000 7f952bfae700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f952409d130 0x7f9524139170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:34.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.731+0000 7f952bfae700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f952409ba70 con 0x7f952409c650 2023-10-03T10:14:34.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.731+0000 7f952bfae700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952409e8f0 con 0x7f952409d130 2023-10-03T10:14:34.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.731+0000 7f952bfae700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95240a0760 con 0x7f95240a0090 2023-10-03T10:14:34.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.731+0000 7f9529549700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f95240a0090 0x7f9524136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:34.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.731+0000 7f9529549700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f95240a0090 0x7f9524136c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:60938/0 (socket says 172.21.15.3:60938) 2023-10-03T10:14:34.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.731+0000 7f9529549700 1 -- 172.21.15.3:0/298163552 learned_addr learned my addr 172.21.15.3:0/298163552 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:34.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.731+0000 7f9529d4a700 1 --1- 172.21.15.3:0/298163552 >> v1:172.21.15.3:6789/0 conn(0x7f952409c650 0x7f952409ca20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40414/0 (socket says 172.21.15.3:40414) 2023-10-03T10:14:34.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.731+0000 7f952a54b700 1 --2- 172.21.15.3:0/298163552 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f952409d130 0x7f9524139170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:34.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.732+0000 7f9529549700 1 -- 172.21.15.3:0/298163552 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f952409d130 msgr2=0x7f9524139170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:34.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.732+0000 7f9529549700 1 --2- 172.21.15.3:0/298163552 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f952409d130 0x7f9524139170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:34.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.732+0000 7f9529549700 1 -- 172.21.15.3:0/298163552 >> v1:172.21.15.3:6789/0 conn(0x7f952409c650 legacy=0x7f952409ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:34.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.732+0000 7f9529549700 1 -- 172.21.15.3:0/298163552 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952409a080 con 0x7f95240a0090 2023-10-03T10:14:34.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.732+0000 7f9529549700 1 --2- 172.21.15.3:0/298163552 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f95240a0090 0x7f9524136c20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9514009920 tx=0x7f9514009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be4bdd24d41aac15 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:34.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.732+0000 7f9528d48700 1 -- 172.21.15.3:0/298163552 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514011830 con 0x7f95240a0090 2023-10-03T10:14:34.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.732+0000 7f9528d48700 1 -- 172.21.15.3:0/298163552 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9514011990 con 0x7f95240a0090 2023-10-03T10:14:34.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.732+0000 7f9528d48700 1 -- 172.21.15.3:0/298163552 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514011d20 con 0x7f95240a0090 2023-10-03T10:14:34.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.733+0000 7f952bfae700 1 -- 172.21.15.3:0/298163552 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f95240a0090 msgr2=0x7f9524136c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:34.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.733+0000 7f952bfae700 1 --2- 172.21.15.3:0/298163552 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f95240a0090 0x7f9524136c20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9514009920 tx=0x7f9514009c30 comp rx=0 tx=0).stop 2023-10-03T10:14:34.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.733+0000 7f952bfae700 1 -- 172.21.15.3:0/298163552 shutdown_connections 2023-10-03T10:14:34.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.733+0000 7f952bfae700 1 --2- 172.21.15.3:0/298163552 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f952409d130 0x7f9524139170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:34.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.733+0000 7f952bfae700 1 --2- 172.21.15.3:0/298163552 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f95240a0090 0x7f9524136c20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:34.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.733+0000 7f952bfae700 1 -- 172.21.15.3:0/298163552 >> 172.21.15.3:0/298163552 conn(0x7f95240a3f70 msgr2=0x7f95240a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:34.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.733+0000 7f952bfae700 1 -- 172.21.15.3:0/298163552 shutdown_connections 2023-10-03T10:14:34.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.733+0000 7f952bfae700 1 -- 172.21.15.3:0/298163552 wait complete. 2023-10-03T10:14:34.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.735+0000 7f952bfae700 1 Processor -- start 2023-10-03T10:14:34.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.735+0000 7f952bfae700 1 -- start start 2023-10-03T10:14:34.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.736+0000 7f952bfae700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f952409c650 0x7f9524135340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:34.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.736+0000 7f952bfae700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f952409d130 0x7f9524135880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:34.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.736+0000 7f952bfae700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95240a0090 0x7f9524135dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:34.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.736+0000 7f952bfae700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9524136810 con 0x7f95240a0090 2023-10-03T10:14:34.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.736+0000 7f952bfae700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952411ab80 con 0x7f952409d130 2023-10-03T10:14:34.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.736+0000 7f952bfae700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f952409d9f0 con 0x7f952409c650 2023-10-03T10:14:34.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.736+0000 7f9529549700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f952409d130 0x7f9524135880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:34.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.736+0000 7f9529d4a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f952409c650 0x7f9524135340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:34.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.736+0000 7f9529549700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f952409d130 0x7f9524135880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50822/0 (socket says 172.21.15.3:50822) 2023-10-03T10:14:34.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.736+0000 7f9529549700 1 -- 172.21.15.3:0/2046350821 learned_addr learned my addr 172.21.15.3:0/2046350821 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:34.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.736+0000 7f952a54b700 1 --2- 172.21.15.3:0/2046350821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95240a0090 0x7f9524135dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:34.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.736+0000 7f9529549700 1 -- 172.21.15.3:0/2046350821 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f952409c650 msgr2=0x7f9524135340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:34.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.736+0000 7f9529549700 1 --2- 172.21.15.3:0/2046350821 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f952409c650 0x7f9524135340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:34.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.736+0000 7f9529549700 1 -- 172.21.15.3:0/2046350821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95240a0090 msgr2=0x7f9524135dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:34.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.736+0000 7f9529549700 1 --2- 172.21.15.3:0/2046350821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95240a0090 0x7f9524135dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:34.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.736+0000 7f9529549700 1 -- 172.21.15.3:0/2046350821 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9514009550 con 0x7f952409d130 2023-10-03T10:14:34.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.736+0000 7f9529d4a700 1 --2- 172.21.15.3:0/2046350821 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f952409c650 0x7f9524135340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:34.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.736+0000 7f952a54b700 1 --2- 172.21.15.3:0/2046350821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95240a0090 0x7f9524135dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:14:34.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.736+0000 7f9529549700 1 --2- 172.21.15.3:0/2046350821 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f952409d130 0x7f9524135880 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f951400a010 tx=0x7f9514004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:34.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.737+0000 7f951a7fc700 1 -- 172.21.15.3:0/2046350821 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514004510 con 0x7f952409d130 2023-10-03T10:14:34.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.737+0000 7f951a7fc700 1 -- 172.21.15.3:0/2046350821 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95140036a0 con 0x7f952409d130 2023-10-03T10:14:34.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.737+0000 7f951a7fc700 1 -- 172.21.15.3:0/2046350821 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514003a50 con 0x7f952409d130 2023-10-03T10:14:34.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.737+0000 7f952bfae700 1 -- 172.21.15.3:0/2046350821 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95241385f0 con 0x7f952409d130 2023-10-03T10:14:34.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.737+0000 7f952bfae700 1 -- 172.21.15.3:0/2046350821 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952412f4a0 con 0x7f952409d130 2023-10-03T10:14:34.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.737+0000 7f951a7fc700 1 -- 172.21.15.3:0/2046350821 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9514003bb0 con 0x7f952409d130 2023-10-03T10:14:34.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.738+0000 7f951a7fc700 1 -- 172.21.15.3:0/2046350821 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9514040860 con 0x7f952409d130 2023-10-03T10:14:34.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.738+0000 7f9507fff700 1 -- 172.21.15.3:0/2046350821 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95000052e0 con 0x7f952409d130 2023-10-03T10:14:34.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.743+0000 7f951a7fc700 1 -- 172.21.15.3:0/2046350821 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f951402c970 con 0x7f952409d130 2023-10-03T10:14:34.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.900+0000 7f5c81fd5700 1 -- 172.21.15.3:0/605447313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5c7c074ea0 con 0x7f5c7c136fe0 2023-10-03T10:14:34.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.901+0000 7f5c63fff700 1 -- 172.21.15.3:0/605447313 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5c74012340 con 0x7f5c7c136fe0 2023-10-03T10:14:34.901 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:34.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.903+0000 7f5c81fd5700 1 -- 172.21.15.3:0/605447313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c7c136fe0 msgr2=0x7f5c7c1c8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:34.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.903+0000 7f5c81fd5700 1 --2- 172.21.15.3:0/605447313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c7c136fe0 0x7f5c7c1c8ce0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400efc0 tx=0x7f5c7400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:14:34.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.903+0000 7f5c81fd5700 1 -- 172.21.15.3:0/605447313 shutdown_connections 2023-10-03T10:14:34.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.903+0000 7f5c81fd5700 1 --2- 172.21.15.3:0/605447313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c7c136fe0 0x7f5c7c1c8ce0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:34.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.903+0000 7f5c81fd5700 1 --2- 172.21.15.3:0/605447313 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c7c09e930 0x7f5c7c1c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:34.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.903+0000 7f5c81fd5700 1 --2- 172.21.15.3:0/605447313 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c7c09f410 0x7f5c7c1c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:34.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.903+0000 7f5c81fd5700 1 -- 172.21.15.3:0/605447313 >> 172.21.15.3:0/605447313 conn(0x7f5c7c0a4210 msgr2=0x7f5c7c134a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:34.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.904+0000 7f5c81fd5700 1 -- 172.21.15.3:0/605447313 shutdown_connections 2023-10-03T10:14:34.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.904+0000 7f5c81fd5700 1 -- 172.21.15.3:0/605447313 wait complete. 2023-10-03T10:14:34.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.905+0000 7ff756005700 1 -- 172.21.15.3:0/3042899047 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff75009f420 con 0x7ff75009d6f0 2023-10-03T10:14:34.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.905+0000 7ff737fff700 1 -- 172.21.15.3:0/3042899047 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff740018260 con 0x7ff75009d6f0 2023-10-03T10:14:34.905 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:34.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.908+0000 7ff756005700 1 -- 172.21.15.3:0/3042899047 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff75009d6f0 msgr2=0x7ff750121760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:34.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.908+0000 7ff756005700 1 --2- 172.21.15.3:0/3042899047 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff75009d6f0 0x7ff750121760 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff74000c700 tx=0x7ff74000ca10 comp rx=0 tx=0).stop 2023-10-03T10:14:34.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.908+0000 7ff756005700 1 -- 172.21.15.3:0/3042899047 shutdown_connections 2023-10-03T10:14:34.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.908+0000 7ff756005700 1 --2- 172.21.15.3:0/3042899047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff750139170 0x7ff7501221e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:34.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.908+0000 7ff756005700 1 --2- 172.21.15.3:0/3042899047 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff75009d6f0 0x7ff750121760 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:34.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.908+0000 7ff756005700 1 --2- 172.21.15.3:0/3042899047 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff75009e1d0 0x7ff750121ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:34.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.908+0000 7ff756005700 1 -- 172.21.15.3:0/3042899047 >> 172.21.15.3:0/3042899047 conn(0x7ff750066f50 msgr2=0x7ff75009b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:34.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.908+0000 7ff756005700 1 -- 172.21.15.3:0/3042899047 shutdown_connections 2023-10-03T10:14:34.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.908+0000 7ff756005700 1 -- 172.21.15.3:0/3042899047 wait complete. 2023-10-03T10:14:34.919 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:34.923 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:34.925 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:34.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.980+0000 7f9507fff700 1 -- 172.21.15.3:0/2046350821 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9500005590 con 0x7f952409d130 2023-10-03T10:14:34.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.980+0000 7f951a7fc700 1 -- 172.21.15.3:0/2046350821 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9514010b70 con 0x7f952409d130 2023-10-03T10:14:34.981 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:34.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.983+0000 7f9507fff700 1 -- 172.21.15.3:0/2046350821 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f952409d130 msgr2=0x7f9524135880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:34.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.983+0000 7f9507fff700 1 --2- 172.21.15.3:0/2046350821 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f952409d130 0x7f9524135880 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f951400a010 tx=0x7f9514004750 comp rx=0 tx=0).stop 2023-10-03T10:14:34.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.983+0000 7f9507fff700 1 -- 172.21.15.3:0/2046350821 shutdown_connections 2023-10-03T10:14:34.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.983+0000 7f9507fff700 1 --2- 172.21.15.3:0/2046350821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95240a0090 0x7f9524135dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:34.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.983+0000 7f9507fff700 1 --2- 172.21.15.3:0/2046350821 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f952409c650 0x7f9524135340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:34.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.983+0000 7f9507fff700 1 --2- 172.21.15.3:0/2046350821 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f952409d130 0x7f9524135880 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:34.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.983+0000 7f9507fff700 1 -- 172.21.15.3:0/2046350821 >> 172.21.15.3:0/2046350821 conn(0x7f95240a3f70 msgr2=0x7f952412f870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:34.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.983+0000 7f9507fff700 1 -- 172.21.15.3:0/2046350821 shutdown_connections 2023-10-03T10:14:34.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:34.983+0000 7f9507fff700 1 -- 172.21.15.3:0/2046350821 wait complete. 2023-10-03T10:14:34.998 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:35.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.102+0000 7f07f21dc700 1 Processor -- start 2023-10-03T10:14:35.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.102+0000 7f07f21dc700 1 -- start start 2023-10-03T10:14:35.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.102+0000 7f07f21dc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07ec09d6f0 0x7f07ec09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:35.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.102+0000 7f07f21dc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f07ec09e000 0x7f07ec1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:35.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.102+0000 7f07f21dc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07ec09e410 0x7f07ec133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:35.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.102+0000 7f07f21dc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07ec06b460 con 0x7f07ec09e000 2023-10-03T10:14:35.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.102+0000 7f07f21dc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec09b120 con 0x7f07ec09d6f0 2023-10-03T10:14:35.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.102+0000 7f07f21dc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec09a7e0 con 0x7f07ec09e410 2023-10-03T10:14:35.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.102+0000 7f07eb7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07ec09d6f0 0x7f07ec09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:35.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.102+0000 7f07eaffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f07ec09e000 0x7f07ec1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:35.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.102+0000 7f07ebfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07ec09e410 0x7f07ec133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:35.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.102+0000 7f07eb7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07ec09d6f0 0x7f07ec09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50832/0 (socket says 172.21.15.3:50832) 2023-10-03T10:14:35.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.103+0000 7f07eb7fe700 1 -- 172.21.15.3:0/2408831249 learned_addr learned my addr 172.21.15.3:0/2408831249 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:35.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.103+0000 7f07eb7fe700 1 -- 172.21.15.3:0/2408831249 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07ec09e410 msgr2=0x7f07ec133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:35.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.103+0000 7f07eb7fe700 1 --2- 172.21.15.3:0/2408831249 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07ec09e410 0x7f07ec133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:35.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.103+0000 7f07eb7fe700 1 -- 172.21.15.3:0/2408831249 >> v2:172.21.15.3:3300/0 conn(0x7f07ec09e000 msgr2=0x7f07ec1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:35.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.103+0000 7f07eb7fe700 1 --2- 172.21.15.3:0/2408831249 >> v2:172.21.15.3:3300/0 conn(0x7f07ec09e000 0x7f07ec1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:35.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.103+0000 7f07eb7fe700 1 -- 172.21.15.3:0/2408831249 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ec09a500 con 0x7f07ec09d6f0 2023-10-03T10:14:35.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.103+0000 7f07eaffd700 1 --2- 172.21.15.3:0/2408831249 >> v2:172.21.15.3:3300/0 conn(0x7f07ec09e000 0x7f07ec1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:35.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.103+0000 7f07ebfff700 1 --2- 172.21.15.3:0/2408831249 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07ec09e410 0x7f07ec133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:35.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.103+0000 7f07eb7fe700 1 --2- 172.21.15.3:0/2408831249 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07ec09d6f0 0x7f07ec09dac0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f07dc009920 tx=0x7f07dc009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f0841abf1ffca44 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:35.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.103+0000 7f07ea7fc700 1 -- 172.21.15.3:0/2408831249 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc011830 con 0x7f07ec09d6f0 2023-10-03T10:14:35.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.103+0000 7f07ea7fc700 1 -- 172.21.15.3:0/2408831249 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07dc011990 con 0x7f07ec09d6f0 2023-10-03T10:14:35.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.103+0000 7f07ea7fc700 1 -- 172.21.15.3:0/2408831249 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc011d20 con 0x7f07ec09d6f0 2023-10-03T10:14:35.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.104+0000 7f07f21dc700 1 -- 172.21.15.3:0/2408831249 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07ec09d6f0 msgr2=0x7f07ec09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:35.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.104+0000 7f07f21dc700 1 --2- 172.21.15.3:0/2408831249 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07ec09d6f0 0x7f07ec09dac0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f07dc009920 tx=0x7f07dc009c30 comp rx=0 tx=0).stop 2023-10-03T10:14:35.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.104+0000 7f07f21dc700 1 -- 172.21.15.3:0/2408831249 shutdown_connections 2023-10-03T10:14:35.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.104+0000 7f07f21dc700 1 --2- 172.21.15.3:0/2408831249 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07ec09e410 0x7f07ec133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:35.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.104+0000 7f07f21dc700 1 --2- 172.21.15.3:0/2408831249 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07ec09d6f0 0x7f07ec09dac0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:35.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.104+0000 7f07f21dc700 1 --2- 172.21.15.3:0/2408831249 >> v2:172.21.15.3:3300/0 conn(0x7f07ec09e000 0x7f07ec1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:35.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.104+0000 7f07f21dc700 1 -- 172.21.15.3:0/2408831249 >> 172.21.15.3:0/2408831249 conn(0x7f07ec066f50 msgr2=0x7f07ec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:35.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.104+0000 7f07f21dc700 1 -- 172.21.15.3:0/2408831249 shutdown_connections 2023-10-03T10:14:35.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.104+0000 7f07f21dc700 1 -- 172.21.15.3:0/2408831249 wait complete. 2023-10-03T10:14:35.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.105+0000 7f07f21dc700 1 Processor -- start 2023-10-03T10:14:35.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.105+0000 7f07f21dc700 1 -- start start 2023-10-03T10:14:35.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.106+0000 7f07f21dc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07ec09d6f0 0x7f07ec1a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:35.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.106+0000 7f07f21dc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07ec09e000 0x7f07ec1a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:35.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.106+0000 7f07f21dc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07ec09e410 0x7f07ec1a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:35.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.106+0000 7f07f21dc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec1aa270 con 0x7f07ec09d6f0 2023-10-03T10:14:35.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.106+0000 7f07f21dc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec1a79a0 con 0x7f07ec09e410 2023-10-03T10:14:35.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.106+0000 7f07f21dc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec1a8c80 con 0x7f07ec09e000 2023-10-03T10:14:35.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.106+0000 7f07eb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07ec09d6f0 0x7f07ec1a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:35.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.106+0000 7f07ebfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07ec09e410 0x7f07ec1a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:35.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.106+0000 7f07eb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07ec09d6f0 0x7f07ec1a5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49924/0 (socket says 172.21.15.3:49924) 2023-10-03T10:14:35.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.106+0000 7f07eb7fe700 1 -- 172.21.15.3:0/1460352193 learned_addr learned my addr 172.21.15.3:0/1460352193 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:35.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.106+0000 7f07eaffd700 1 --2- 172.21.15.3:0/1460352193 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07ec09e000 0x7f07ec1a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:35.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.106+0000 7f07ebfff700 1 -- 172.21.15.3:0/1460352193 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07ec09e000 msgr2=0x7f07ec1a6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:35.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.106+0000 7f07ebfff700 1 --2- 172.21.15.3:0/1460352193 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07ec09e000 0x7f07ec1a6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:35.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.106+0000 7f07ebfff700 1 -- 172.21.15.3:0/1460352193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07ec09d6f0 msgr2=0x7f07ec1a5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:35.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.106+0000 7f07ebfff700 1 --2- 172.21.15.3:0/1460352193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07ec09d6f0 0x7f07ec1a5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:35.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.106+0000 7f07ebfff700 1 -- 172.21.15.3:0/1460352193 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e00094a0 con 0x7f07ec09e410 2023-10-03T10:14:35.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.106+0000 7f07eb7fe700 1 --2- 172.21.15.3:0/1460352193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07ec09d6f0 0x7f07ec1a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:35.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.106+0000 7f07ebfff700 1 --2- 172.21.15.3:0/1460352193 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07ec09e410 0x7f07ec1a68b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f07e4009e50 tx=0x7f07e400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:35.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.106+0000 7f07d3fff700 1 -- 172.21.15.3:0/1460352193 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e400ca90 con 0x7f07ec09e410 2023-10-03T10:14:35.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.106+0000 7f07d3fff700 1 -- 172.21.15.3:0/1460352193 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e400f040 con 0x7f07ec09e410 2023-10-03T10:14:35.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.107+0000 7f07f21dc700 1 -- 172.21.15.3:0/1460352193 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e4009a70 con 0x7f07ec09e410 2023-10-03T10:14:35.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.107+0000 7f07d3fff700 1 -- 172.21.15.3:0/1460352193 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e400ce50 con 0x7f07ec09e410 2023-10-03T10:14:35.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.107+0000 7f07f21dc700 1 -- 172.21.15.3:0/1460352193 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07dc009550 con 0x7f07ec09e410 2023-10-03T10:14:35.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.107+0000 7f07d3fff700 1 -- 172.21.15.3:0/1460352193 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f07e4014480 con 0x7f07ec09e410 2023-10-03T10:14:35.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.107+0000 7f07d3fff700 1 -- 172.21.15.3:0/1460352193 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f07e403fba0 con 0x7f07ec09e410 2023-10-03T10:14:35.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.108+0000 7f07f21dc700 1 -- 172.21.15.3:0/1460352193 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07ec074ea0 con 0x7f07ec09e410 2023-10-03T10:14:35.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.112+0000 7f07d3fff700 1 -- 172.21.15.3:0/1460352193 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f07e4012910 con 0x7f07ec09e410 2023-10-03T10:14:35.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.318+0000 7f07f21dc700 1 -- 172.21.15.3:0/1460352193 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f07ec06c360 con 0x7f07ec09e410 2023-10-03T10:14:35.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.319+0000 7f07d3fff700 1 -- 172.21.15.3:0/1460352193 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f07e4012340 con 0x7f07ec09e410 2023-10-03T10:14:35.319 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:35.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.321+0000 7f07f21dc700 1 -- 172.21.15.3:0/1460352193 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07ec09e410 msgr2=0x7f07ec1a68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:35.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.321+0000 7f07f21dc700 1 --2- 172.21.15.3:0/1460352193 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07ec09e410 0x7f07ec1a68b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f07e4009e50 tx=0x7f07e400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:14:35.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.321+0000 7f07f21dc700 1 -- 172.21.15.3:0/1460352193 shutdown_connections 2023-10-03T10:14:35.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.321+0000 7f07f21dc700 1 --2- 172.21.15.3:0/1460352193 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07ec09e410 0x7f07ec1a68b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:35.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.321+0000 7f07f21dc700 1 --2- 172.21.15.3:0/1460352193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07ec09d6f0 0x7f07ec1a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:35.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.321+0000 7f07f21dc700 1 --2- 172.21.15.3:0/1460352193 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07ec09e000 0x7f07ec1a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:35.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.321+0000 7f07f21dc700 1 -- 172.21.15.3:0/1460352193 >> 172.21.15.3:0/1460352193 conn(0x7f07ec066f50 msgr2=0x7f07ec1badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:35.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.321+0000 7f07f21dc700 1 -- 172.21.15.3:0/1460352193 shutdown_connections 2023-10-03T10:14:35.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:35.321+0000 7f07f21dc700 1 -- 172.21.15.3:0/1460352193 wait complete. 2023-10-03T10:14:35.335 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:35.920 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:35.923 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:35.999 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:36.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.070+0000 7fa6627fe700 1 Processor -- start 2023-10-03T10:14:36.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.070+0000 7fa6627fe700 1 -- start start 2023-10-03T10:14:36.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.071+0000 7fa6627fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa65c0a0a30 0x7fa65c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:36.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.071+0000 7fa6627fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa65c134170 0x7fa65c13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:36.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.071+0000 7fa6627fe700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65c09edd0 con 0x7fa65c0a1340 2023-10-03T10:14:36.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.071+0000 7fa6627fe700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa65c09e1f0 con 0x7fa65c0a0a30 2023-10-03T10:14:36.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.071+0000 7fa6627fe700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c09dc50 con 0x7fa65c134170 2023-10-03T10:14:36.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.071+0000 7fa660d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa65c134170 0x7fa65c13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:36.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.071+0000 7fa65bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa65c0a0a30 0x7fa65c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:36.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.071+0000 7fa660d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa65c134170 0x7fa65c13bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50854/0 (socket says 172.21.15.3:50854) 2023-10-03T10:14:36.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.071+0000 7fa660d9b700 1 -- 172.21.15.3:0/1196873505 learned_addr learned my addr 172.21.15.3:0/1196873505 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:36.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.071+0000 7fa660d9b700 1 -- 172.21.15.3:0/1196873505 >> v2:172.21.15.3:3300/0 conn(0x7fa65c0a0a30 msgr2=0x7fa65c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:36.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.071+0000 7fa660d9b700 1 --2- 172.21.15.3:0/1196873505 >> v2:172.21.15.3:3300/0 conn(0x7fa65c0a0a30 0x7fa65c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.071+0000 7fa660d9b700 1 -- 172.21.15.3:0/1196873505 >> v1:172.21.15.3:6789/0 conn(0x7fa65c0a1340 legacy=0x7fa65c133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:36.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.071+0000 7fa660d9b700 1 -- 172.21.15.3:0/1196873505 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65c0a1fe0 con 0x7fa65c134170 2023-10-03T10:14:36.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.072+0000 7fa660d9b700 1 --2- 172.21.15.3:0/1196873505 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa65c134170 0x7fa65c13bd30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa654009e50 tx=0x7fa65400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b8c1f1bea6d1279 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:36.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.072+0000 7fa65affd700 1 -- 172.21.15.3:0/1196873505 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65400e070 con 0x7fa65c134170 2023-10-03T10:14:36.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.072+0000 7fa65affd700 1 -- 172.21.15.3:0/1196873505 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65400b910 con 0x7fa65c134170 2023-10-03T10:14:36.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.072+0000 7fa65affd700 1 -- 172.21.15.3:0/1196873505 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65400bcc0 con 0x7fa65c134170 2023-10-03T10:14:36.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.072+0000 7fa6627fe700 1 -- 172.21.15.3:0/1196873505 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa65c134170 msgr2=0x7fa65c13bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:36.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.072+0000 7fa6627fe700 1 --2- 172.21.15.3:0/1196873505 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa65c134170 0x7fa65c13bd30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa654009e50 tx=0x7fa65400b330 comp rx=0 tx=0).stop 2023-10-03T10:14:36.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.073+0000 7fa6627fe700 1 -- 172.21.15.3:0/1196873505 shutdown_connections 2023-10-03T10:14:36.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.073+0000 7fa6627fe700 1 --2- 172.21.15.3:0/1196873505 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa65c134170 0x7fa65c13bd30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.073+0000 7fa6627fe700 1 --2- 172.21.15.3:0/1196873505 >> v2:172.21.15.3:3300/0 conn(0x7fa65c0a0a30 0x7fa65c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.073+0000 7fa6627fe700 1 -- 172.21.15.3:0/1196873505 >> 172.21.15.3:0/1196873505 conn(0x7fa65c0a6550 msgr2=0x7fa65c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:36.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.073+0000 7fa6627fe700 1 -- 172.21.15.3:0/1196873505 shutdown_connections 2023-10-03T10:14:36.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.073+0000 7fa6627fe700 1 -- 172.21.15.3:0/1196873505 wait complete. 2023-10-03T10:14:36.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.074+0000 7fa6627fe700 1 Processor -- start 2023-10-03T10:14:36.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.074+0000 7fa6627fe700 1 -- start start 2023-10-03T10:14:36.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.074+0000 7fa6627fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa65c0a0a30 0x7fa65c1232e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:36.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.074+0000 7fa6627fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa65c0a1340 0x7fa65c123820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:36.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.074+0000 7fa6627fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa65c134170 0x7fa65c1c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:36.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.074+0000 7fa6627fe700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c122f60 con 0x7fa65c0a1340 2023-10-03T10:14:36.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.074+0000 7fa6627fe700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c0a4f40 con 0x7fa65c134170 2023-10-03T10:14:36.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.074+0000 7fa6627fe700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c126200 con 0x7fa65c0a0a30 2023-10-03T10:14:36.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.075+0000 7fa65bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa65c0a0a30 0x7fa65c1232e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:36.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.075+0000 7fa65bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa65c0a0a30 0x7fa65c1232e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:60962/0 (socket says 172.21.15.3:60962) 2023-10-03T10:14:36.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.075+0000 7fa65bfff700 1 -- 172.21.15.3:0/2373845064 learned_addr learned my addr 172.21.15.3:0/2373845064 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:36.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.075+0000 7fa660d9b700 1 --2- 172.21.15.3:0/2373845064 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa65c134170 0x7fa65c1c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:36.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.075+0000 7fa65b7fe700 1 --2- 172.21.15.3:0/2373845064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa65c0a1340 0x7fa65c123820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:36.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.075+0000 7fa660d9b700 1 -- 172.21.15.3:0/2373845064 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa65c0a0a30 msgr2=0x7fa65c1232e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:36.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.075+0000 7fa660d9b700 1 --2- 172.21.15.3:0/2373845064 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa65c0a0a30 0x7fa65c1232e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.075+0000 7fa660d9b700 1 -- 172.21.15.3:0/2373845064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa65c0a1340 msgr2=0x7fa65c123820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:36.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.075+0000 7fa660d9b700 1 --2- 172.21.15.3:0/2373845064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa65c0a1340 0x7fa65c123820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.075+0000 7fa660d9b700 1 -- 172.21.15.3:0/2373845064 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa654009b30 con 0x7fa65c134170 2023-10-03T10:14:36.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.075+0000 7fa65bfff700 1 --2- 172.21.15.3:0/2373845064 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa65c0a0a30 0x7fa65c1232e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:36.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.075+0000 7fa65b7fe700 1 --2- 172.21.15.3:0/2373845064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa65c0a1340 0x7fa65c123820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:14:36.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.075+0000 7fa660d9b700 1 --2- 172.21.15.3:0/2373845064 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa65c134170 0x7fa65c1c83e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa654015000 tx=0x7fa654004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:36.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.075+0000 7fa658ff9700 1 -- 172.21.15.3:0/2373845064 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6540048b0 con 0x7fa65c134170 2023-10-03T10:14:36.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.075+0000 7fa658ff9700 1 -- 172.21.15.3:0/2373845064 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6540099b0 con 0x7fa65c134170 2023-10-03T10:14:36.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.076+0000 7fa6627fe700 1 -- 172.21.15.3:0/2373845064 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa65c0a25e0 con 0x7fa65c134170 2023-10-03T10:14:36.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.076+0000 7fa658ff9700 1 -- 172.21.15.3:0/2373845064 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65401c650 con 0x7fa65c134170 2023-10-03T10:14:36.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.076+0000 7fa6627fe700 1 -- 172.21.15.3:0/2373845064 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa65c124dc0 con 0x7fa65c134170 2023-10-03T10:14:36.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.076+0000 7fa658ff9700 1 -- 172.21.15.3:0/2373845064 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa65401c7b0 con 0x7fa65c134170 2023-10-03T10:14:36.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.077+0000 7fa658ff9700 1 -- 172.21.15.3:0/2373845064 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa654041150 con 0x7fa65c134170 2023-10-03T10:14:36.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.077+0000 7fa6627fe700 1 -- 172.21.15.3:0/2373845064 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa65c074ea0 con 0x7fa65c134170 2023-10-03T10:14:36.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.082+0000 7fa658ff9700 1 -- 172.21.15.3:0/2373845064 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa65401a560 con 0x7fa65c134170 2023-10-03T10:14:36.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.110+0000 7f0728130700 1 Processor -- start 2023-10-03T10:14:36.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.110+0000 7f0728130700 1 -- start start 2023-10-03T10:14:36.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.111+0000 7f0728130700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f072009d6f0 0x7f072009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:36.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.111+0000 7f0728130700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f072009e000 0x7f07201339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:36.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.111+0000 7f0728130700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f072009b120 con 0x7f072009e410 2023-10-03T10:14:36.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.111+0000 7f0728130700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f072006b460 con 0x7f072009e000 2023-10-03T10:14:36.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.111+0000 7f0728130700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f072009a7e0 con 0x7f072009d6f0 2023-10-03T10:14:36.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.111+0000 7f0725ecc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f072009d6f0 0x7f072009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:36.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.111+0000 7f07266cd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f072009e410 0x7f0720133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40428/0 (socket says 172.21.15.3:40428) 2023-10-03T10:14:36.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.111+0000 7f07266cd700 1 -- 172.21.15.3:0/896806374 learned_addr learned my addr 172.21.15.3:0/896806374 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:36.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.111+0000 7f07256cb700 1 --2- 172.21.15.3:0/896806374 >> v2:172.21.15.3:3300/0 conn(0x7f072009e000 0x7f07201339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:36.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.112+0000 7f07256cb700 1 -- 172.21.15.3:0/896806374 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f072009d6f0 msgr2=0x7f072009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:36.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.112+0000 7f07256cb700 1 --2- 172.21.15.3:0/896806374 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f072009d6f0 0x7f072009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.112+0000 7f07256cb700 1 -- 172.21.15.3:0/896806374 >> v1:172.21.15.3:6789/0 conn(0x7f072009e410 legacy=0x7f0720133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:36.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.112+0000 7f07256cb700 1 -- 172.21.15.3:0/896806374 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072009a500 con 0x7f072009e000 2023-10-03T10:14:36.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.112+0000 7f0724eca700 1 -- 172.21.15.3:0/896806374 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4130995480 0 0) 0x7f072009b120 con 0x7f072009e410 2023-10-03T10:14:36.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.112+0000 7f0725ecc700 1 --2- 172.21.15.3:0/896806374 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f072009d6f0 0x7f072009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:36.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.112+0000 7f07256cb700 1 --2- 172.21.15.3:0/896806374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f072009e000 0x7f07201339f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0710009840 tx=0x7f0710009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f077bc85c818d988 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:36.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.112+0000 7f0724eca700 1 -- 172.21.15.3:0/896806374 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0710011830 con 0x7f072009e000 2023-10-03T10:14:36.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.112+0000 7f0724eca700 1 -- 172.21.15.3:0/896806374 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0710011990 con 0x7f072009e000 2023-10-03T10:14:36.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.112+0000 7f0724eca700 1 -- 172.21.15.3:0/896806374 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0710011d20 con 0x7f072009e000 2023-10-03T10:14:36.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.113+0000 7f0728130700 1 -- 172.21.15.3:0/896806374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f072009e000 msgr2=0x7f07201339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:36.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.113+0000 7f0728130700 1 --2- 172.21.15.3:0/896806374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f072009e000 0x7f07201339f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0710009840 tx=0x7f0710009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:36.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.113+0000 7f0728130700 1 -- 172.21.15.3:0/896806374 shutdown_connections 2023-10-03T10:14:36.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.113+0000 7f0728130700 1 --2- 172.21.15.3:0/896806374 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f072009d6f0 0x7f072009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.113+0000 7f0728130700 1 --2- 172.21.15.3:0/896806374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f072009e000 0x7f07201339f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.113+0000 7f0728130700 1 -- 172.21.15.3:0/896806374 >> 172.21.15.3:0/896806374 conn(0x7f0720066f50 msgr2=0x7f072009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:36.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.113+0000 7f0728130700 1 -- 172.21.15.3:0/896806374 shutdown_connections 2023-10-03T10:14:36.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.113+0000 7f0728130700 1 -- 172.21.15.3:0/896806374 wait complete. 2023-10-03T10:14:36.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.114+0000 7f0728130700 1 Processor -- start 2023-10-03T10:14:36.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.114+0000 7f0728130700 1 -- start start 2023-10-03T10:14:36.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.115+0000 7f0728130700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f072009d6f0 0x7f07201a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:36.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.115+0000 7f0728130700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f072009e000 0x7f07201a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:36.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.115+0000 7f0728130700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f072009e410 0x7f07201c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:36.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.115+0000 7f0728130700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07201b9180 con 0x7f072009d6f0 2023-10-03T10:14:36.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.115+0000 7f0728130700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07201a4cd0 con 0x7f072009e410 2023-10-03T10:14:36.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.115+0000 7f0728130700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f07201a6400 con 0x7f072009e000 2023-10-03T10:14:36.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.115+0000 7f07266cd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f072009e410 0x7f07201c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:36.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.115+0000 7f07266cd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f072009e410 0x7f07201c8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50882/0 (socket says 172.21.15.3:50882) 2023-10-03T10:14:36.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.115+0000 7f07266cd700 1 -- 172.21.15.3:0/2800469330 learned_addr learned my addr 172.21.15.3:0/2800469330 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:36.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.115+0000 7f0725ecc700 1 --2- 172.21.15.3:0/2800469330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f072009d6f0 0x7f07201a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:36.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.115+0000 7f07256cb700 1 --2- 172.21.15.3:0/2800469330 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f072009e000 0x7f07201a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:36.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.115+0000 7f07266cd700 1 -- 172.21.15.3:0/2800469330 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f072009e000 msgr2=0x7f07201a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:36.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.115+0000 7f07266cd700 1 --2- 172.21.15.3:0/2800469330 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f072009e000 0x7f07201a5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.116+0000 7f07266cd700 1 -- 172.21.15.3:0/2800469330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f072009d6f0 msgr2=0x7f07201a50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:36.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.116+0000 7f07266cd700 1 --2- 172.21.15.3:0/2800469330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f072009d6f0 0x7f07201a50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.116+0000 7f07266cd700 1 -- 172.21.15.3:0/2800469330 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071c0094a0 con 0x7f072009e410 2023-10-03T10:14:36.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.116+0000 7f0725ecc700 1 --2- 172.21.15.3:0/2800469330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f072009d6f0 0x7f07201a50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:36.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.116+0000 7f07266cd700 1 --2- 172.21.15.3:0/2800469330 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f072009e410 0x7f07201c8020 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0718002230 tx=0x7f071800cf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:36.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.116+0000 7f07167fc700 1 -- 172.21.15.3:0/2800469330 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071800aac0 con 0x7f072009e410 2023-10-03T10:14:36.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.116+0000 7f07167fc700 1 -- 172.21.15.3:0/2800469330 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071800ac40 con 0x7f072009e410 2023-10-03T10:14:36.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.116+0000 7f0728130700 1 -- 172.21.15.3:0/2800469330 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07100094a0 con 0x7f072009e410 2023-10-03T10:14:36.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.116+0000 7f07167fc700 1 -- 172.21.15.3:0/2800469330 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07180070d0 con 0x7f072009e410 2023-10-03T10:14:36.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.116+0000 7f0728130700 1 -- 172.21.15.3:0/2800469330 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f072009c710 con 0x7f072009e410 2023-10-03T10:14:36.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.117+0000 7f07167fc700 1 -- 172.21.15.3:0/2800469330 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0718007250 con 0x7f072009e410 2023-10-03T10:14:36.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.117+0000 7f07167fc700 1 -- 172.21.15.3:0/2800469330 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f071803f1a0 con 0x7f072009e410 2023-10-03T10:14:36.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.117+0000 7f0728130700 1 -- 172.21.15.3:0/2800469330 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0720074ea0 con 0x7f072009e410 2023-10-03T10:14:36.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.122+0000 7f07167fc700 1 -- 172.21.15.3:0/2800469330 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0718018080 con 0x7f072009e410 2023-10-03T10:14:36.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.151+0000 7f74a4003700 1 Processor -- start 2023-10-03T10:14:36.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.151+0000 7f74a4003700 1 -- start start 2023-10-03T10:14:36.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.152+0000 7f74a4003700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f749c0a0a30 0x7f749c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:36.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.152+0000 7f74a4003700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f749c09d630 0x7f749c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:36.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.152+0000 7f74a4003700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f749c0a2a70 con 0x7f749c133930 2023-10-03T10:14:36.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.152+0000 7f74a4003700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f749c0a01d0 con 0x7f749c09d630 2023-10-03T10:14:36.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.152+0000 7f74a4003700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f749c09e2b0 con 0x7f749c0a0a30 2023-10-03T10:14:36.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.152+0000 7f74a1d9f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f749c0a0a30 0x7f749c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:36.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.152+0000 7f74a159e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f749c09d630 0x7f749c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:36.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.152+0000 7f74a159e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f749c09d630 0x7f749c0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49968/0 (socket says 172.21.15.3:49968) 2023-10-03T10:14:36.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.152+0000 7f74a1d9f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f749c0a0a30 0x7f749c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:60990/0 (socket says 172.21.15.3:60990) 2023-10-03T10:14:36.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.152+0000 7f74a159e700 1 -- 172.21.15.3:0/1141767131 learned_addr learned my addr 172.21.15.3:0/1141767131 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:36.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.152+0000 7f74a159e700 1 -- 172.21.15.3:0/1141767131 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f749c0a0a30 msgr2=0x7f749c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:36.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.152+0000 7f74a159e700 1 --2- 172.21.15.3:0/1141767131 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f749c0a0a30 0x7f749c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.152+0000 7f74a159e700 1 -- 172.21.15.3:0/1141767131 >> v1:172.21.15.3:6789/0 conn(0x7f749c133930 legacy=0x7f749c133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:36.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.152+0000 7f74a159e700 1 -- 172.21.15.3:0/1141767131 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749c09dc50 con 0x7f749c09d630 2023-10-03T10:14:36.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.153+0000 7f74a159e700 1 --2- 172.21.15.3:0/1141767131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f749c09d630 0x7f749c0a1340 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f748c0098f0 tx=0x7f748c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3880c392394f8c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:36.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.153+0000 7f74a0d9d700 1 -- 172.21.15.3:0/1141767131 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c011830 con 0x7f749c09d630 2023-10-03T10:14:36.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.153+0000 7f74a0d9d700 1 -- 172.21.15.3:0/1141767131 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748c011990 con 0x7f749c09d630 2023-10-03T10:14:36.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.153+0000 7f74a0d9d700 1 -- 172.21.15.3:0/1141767131 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c011d20 con 0x7f749c09d630 2023-10-03T10:14:36.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.153+0000 7f74a4003700 1 -- 172.21.15.3:0/1141767131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f749c09d630 msgr2=0x7f749c0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:36.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.153+0000 7f74a4003700 1 --2- 172.21.15.3:0/1141767131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f749c09d630 0x7f749c0a1340 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f748c0098f0 tx=0x7f748c009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:36.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.153+0000 7f74a4003700 1 -- 172.21.15.3:0/1141767131 shutdown_connections 2023-10-03T10:14:36.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.153+0000 7f74a4003700 1 --2- 172.21.15.3:0/1141767131 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f749c0a0a30 0x7f749c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.153+0000 7f74a4003700 1 --2- 172.21.15.3:0/1141767131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f749c09d630 0x7f749c0a1340 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.154+0000 7f74a4003700 1 -- 172.21.15.3:0/1141767131 >> 172.21.15.3:0/1141767131 conn(0x7f749c0a3050 msgr2=0x7f749c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:36.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.154+0000 7f74a4003700 1 -- 172.21.15.3:0/1141767131 shutdown_connections 2023-10-03T10:14:36.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.154+0000 7f74a4003700 1 -- 172.21.15.3:0/1141767131 wait complete. 2023-10-03T10:14:36.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.155+0000 7f74a4003700 1 Processor -- start 2023-10-03T10:14:36.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.155+0000 7f74a4003700 1 -- start start 2023-10-03T10:14:36.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.156+0000 7f74a4003700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f749c09d630 0x7f749c1c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:36.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.156+0000 7f74a4003700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f749c0a0a30 0x7f749c1c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:36.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.156+0000 7f74a4003700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f749c133930 0x7f749c11e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:36.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.156+0000 7f74a4003700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749c1bcfe0 con 0x7f749c0a0a30 2023-10-03T10:14:36.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.156+0000 7f74a4003700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f749c121d80 con 0x7f749c09d630 2023-10-03T10:14:36.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.156+0000 7f74a4003700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f749c122420 con 0x7f749c133930 2023-10-03T10:14:36.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.156+0000 7f74a1d9f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f749c09d630 0x7f749c1c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:36.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.156+0000 7f74a159e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f749c0a0a30 0x7f749c1c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:36.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.156+0000 7f74a159e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f749c0a0a30 0x7f749c1c8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49978/0 (socket says 172.21.15.3:49978) 2023-10-03T10:14:36.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.156+0000 7f74a1d9f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f749c09d630 0x7f749c1c8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50886/0 (socket says 172.21.15.3:50886) 2023-10-03T10:14:36.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.156+0000 7f74a159e700 1 -- 172.21.15.3:0/1340325603 learned_addr learned my addr 172.21.15.3:0/1340325603 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:36.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.156+0000 7f74a25a0700 1 --2- 172.21.15.3:0/1340325603 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f749c133930 0x7f749c11e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:36.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.156+0000 7f74a159e700 1 -- 172.21.15.3:0/1340325603 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f749c133930 msgr2=0x7f749c11e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:36.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.156+0000 7f74a159e700 1 --2- 172.21.15.3:0/1340325603 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f749c133930 0x7f749c11e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.156+0000 7f74a159e700 1 -- 172.21.15.3:0/1340325603 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f749c09d630 msgr2=0x7f749c1c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:36.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.156+0000 7f74a159e700 1 --2- 172.21.15.3:0/1340325603 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f749c09d630 0x7f749c1c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.156+0000 7f74a159e700 1 -- 172.21.15.3:0/1340325603 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748c009550 con 0x7f749c0a0a30 2023-10-03T10:14:36.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.157+0000 7f74a159e700 1 --2- 172.21.15.3:0/1340325603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f749c0a0a30 0x7f749c1c8990 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f748c000c00 tx=0x7f748c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:36.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.157+0000 7f74927fc700 1 -- 172.21.15.3:0/1340325603 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c0036a0 con 0x7f749c0a0a30 2023-10-03T10:14:36.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.157+0000 7f74927fc700 1 -- 172.21.15.3:0/1340325603 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748c003800 con 0x7f749c0a0a30 2023-10-03T10:14:36.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.157+0000 7f74a4003700 1 -- 172.21.15.3:0/1340325603 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f749c0a4030 con 0x7f749c0a0a30 2023-10-03T10:14:36.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.157+0000 7f74a4003700 1 -- 172.21.15.3:0/1340325603 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f749c122ab0 con 0x7f749c0a0a30 2023-10-03T10:14:36.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.157+0000 7f74927fc700 1 -- 172.21.15.3:0/1340325603 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c003bb0 con 0x7f749c0a0a30 2023-10-03T10:14:36.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.158+0000 7f74a4003700 1 -- 172.21.15.3:0/1340325603 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f749c122280 con 0x7f749c0a0a30 2023-10-03T10:14:36.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.158+0000 7f74927fc700 1 -- 172.21.15.3:0/1340325603 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f748c021070 con 0x7f749c0a0a30 2023-10-03T10:14:36.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.158+0000 7f74927fc700 1 -- 172.21.15.3:0/1340325603 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f748c040aa0 con 0x7f749c0a0a30 2023-10-03T10:14:36.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.163+0000 7f74927fc700 1 -- 172.21.15.3:0/1340325603 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f749c122280 con 0x7f749c0a0a30 2023-10-03T10:14:36.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.295+0000 7fa6627fe700 1 -- 172.21.15.3:0/2373845064 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa65c0a28e0 con 0x7fa65c134170 2023-10-03T10:14:36.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.296+0000 7fa658ff9700 1 -- 172.21.15.3:0/2373845064 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa654029020 con 0x7fa65c134170 2023-10-03T10:14:36.297 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:36.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.298+0000 7fa6627fe700 1 -- 172.21.15.3:0/2373845064 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa65c134170 msgr2=0x7fa65c1c83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.298+0000 7fa6627fe700 1 --2- 172.21.15.3:0/2373845064 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa65c134170 0x7fa65c1c83e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa654015000 tx=0x7fa654004ad0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.298+0000 7fa6627fe700 1 -- 172.21.15.3:0/2373845064 shutdown_connections 2023-10-03T10:14:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.298+0000 7fa6627fe700 1 --2- 172.21.15.3:0/2373845064 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa65c134170 0x7fa65c1c83e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.298+0000 7fa6627fe700 1 --2- 172.21.15.3:0/2373845064 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa65c0a0a30 0x7fa65c1232e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.298+0000 7fa6627fe700 1 --2- 172.21.15.3:0/2373845064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa65c0a1340 0x7fa65c123820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.298+0000 7fa6627fe700 1 -- 172.21.15.3:0/2373845064 >> 172.21.15.3:0/2373845064 conn(0x7fa65c0a6550 msgr2=0x7fa65c09ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.298+0000 7fa6627fe700 1 -- 172.21.15.3:0/2373845064 shutdown_connections 2023-10-03T10:14:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.298+0000 7fa6627fe700 1 -- 172.21.15.3:0/2373845064 wait complete. 2023-10-03T10:14:36.312 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:36.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.331+0000 7f0728130700 1 -- 172.21.15.3:0/2800469330 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f07201a7070 con 0x7f072009e410 2023-10-03T10:14:36.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.331+0000 7f07167fc700 1 -- 172.21.15.3:0/2800469330 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0718007500 con 0x7f072009e410 2023-10-03T10:14:36.331 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:36.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.333+0000 7f0728130700 1 -- 172.21.15.3:0/2800469330 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f072009e410 msgr2=0x7f07201c8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:36.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.333+0000 7f0728130700 1 --2- 172.21.15.3:0/2800469330 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f072009e410 0x7f07201c8020 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0718002230 tx=0x7f071800cf50 comp rx=0 tx=0).stop 2023-10-03T10:14:36.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.333+0000 7f0728130700 1 -- 172.21.15.3:0/2800469330 shutdown_connections 2023-10-03T10:14:36.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.333+0000 7f0728130700 1 --2- 172.21.15.3:0/2800469330 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f072009e410 0x7f07201c8020 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.333+0000 7f0728130700 1 --2- 172.21.15.3:0/2800469330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f072009d6f0 0x7f07201a50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.333+0000 7f0728130700 1 --2- 172.21.15.3:0/2800469330 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f072009e000 0x7f07201a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.333+0000 7f0728130700 1 -- 172.21.15.3:0/2800469330 >> 172.21.15.3:0/2800469330 conn(0x7f0720066f50 msgr2=0x7f07201bbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:36.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.333+0000 7f0728130700 1 -- 172.21.15.3:0/2800469330 shutdown_connections 2023-10-03T10:14:36.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.333+0000 7f0728130700 1 -- 172.21.15.3:0/2800469330 wait complete. 2023-10-03T10:14:36.336 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:36.348 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:36.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.371+0000 7f74a4003700 1 -- 172.21.15.3:0/1340325603 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f749c122280 con 0x7f749c0a0a30 2023-10-03T10:14:36.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.371+0000 7f74927fc700 1 -- 172.21.15.3:0/1340325603 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f749c122280 con 0x7f749c0a0a30 2023-10-03T10:14:36.372 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:36.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.374+0000 7f74a4003700 1 -- 172.21.15.3:0/1340325603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f749c0a0a30 msgr2=0x7f749c1c8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:36.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.374+0000 7f74a4003700 1 --2- 172.21.15.3:0/1340325603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f749c0a0a30 0x7f749c1c8990 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f748c000c00 tx=0x7f748c004750 comp rx=0 tx=0).stop 2023-10-03T10:14:36.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.374+0000 7f74a4003700 1 -- 172.21.15.3:0/1340325603 shutdown_connections 2023-10-03T10:14:36.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.374+0000 7f74a4003700 1 --2- 172.21.15.3:0/1340325603 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f749c133930 0x7f749c11e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.374+0000 7f74a4003700 1 --2- 172.21.15.3:0/1340325603 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f749c09d630 0x7f749c1c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.374+0000 7f74a4003700 1 --2- 172.21.15.3:0/1340325603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f749c0a0a30 0x7f749c1c8990 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.374+0000 7f74a4003700 1 -- 172.21.15.3:0/1340325603 >> 172.21.15.3:0/1340325603 conn(0x7f749c0a3050 msgr2=0x7f749c1bbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:36.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.374+0000 7f74a4003700 1 -- 172.21.15.3:0/1340325603 shutdown_connections 2023-10-03T10:14:36.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.374+0000 7f74a4003700 1 -- 172.21.15.3:0/1340325603 wait complete. 2023-10-03T10:14:36.388 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:36.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.473+0000 7fc6c93e3700 1 Processor -- start 2023-10-03T10:14:36.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.474+0000 7fc6c93e3700 1 -- start start 2023-10-03T10:14:36.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.474+0000 7fc6c93e3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc6c409e850 0x7fc6c409ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:36.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.474+0000 7fc6c93e3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6c409ffb0 0x7fc6c409f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:36.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.474+0000 7fc6c93e3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6c4133b80 0x7fc6c4133fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:36.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.474+0000 7fc6c93e3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6c406b920 con 0x7fc6c409e850 2023-10-03T10:14:36.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.474+0000 7fc6c93e3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c40a06b0 con 0x7fc6c409ffb0 2023-10-03T10:14:36.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.474+0000 7fc6c93e3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c409aa30 con 0x7fc6c4133b80 2023-10-03T10:14:36.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.474+0000 7fc6c27fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6c409ffb0 0x7fc6c409f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:36.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.474+0000 7fc6c37fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6c4133b80 0x7fc6c4133fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:36.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.474+0000 7fc6c37fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6c4133b80 0x7fc6c4133fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:32772/0 (socket says 172.21.15.3:32772) 2023-10-03T10:14:36.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.474+0000 7fc6c27fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6c409ffb0 0x7fc6c409f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50902/0 (socket says 172.21.15.3:50902) 2023-10-03T10:14:36.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.474+0000 7fc6c2ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc6c409e850 0x7fc6c409ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:36.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.474+0000 7fc6c37fe700 1 -- 172.21.15.3:0/420064526 learned_addr learned my addr 172.21.15.3:0/420064526 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:36.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.475+0000 7fc6c37fe700 1 -- 172.21.15.3:0/420064526 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6c409ffb0 msgr2=0x7fc6c409f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:36.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.475+0000 7fc6c37fe700 1 --2- 172.21.15.3:0/420064526 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6c409ffb0 0x7fc6c409f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.475+0000 7fc6c37fe700 1 -- 172.21.15.3:0/420064526 >> v2:172.21.15.3:3300/0 conn(0x7fc6c409e850 msgr2=0x7fc6c409ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:36.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.475+0000 7fc6c37fe700 1 --2- 172.21.15.3:0/420064526 >> v2:172.21.15.3:3300/0 conn(0x7fc6c409e850 0x7fc6c409ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.475+0000 7fc6c37fe700 1 -- 172.21.15.3:0/420064526 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c406caa0 con 0x7fc6c4133b80 2023-10-03T10:14:36.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.475+0000 7fc6c27fc700 1 --2- 172.21.15.3:0/420064526 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6c409ffb0 0x7fc6c409f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:36.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.475+0000 7fc6c2ffd700 1 --2- 172.21.15.3:0/420064526 >> v2:172.21.15.3:3300/0 conn(0x7fc6c409e850 0x7fc6c409ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:36.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.475+0000 7fc6c37fe700 1 --2- 172.21.15.3:0/420064526 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6c4133b80 0x7fc6c4133fe0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009e50 tx=0x7fc6bc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be623174c7b3fb4c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:36.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.475+0000 7fc6c1ffb700 1 -- 172.21.15.3:0/420064526 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc00e070 con 0x7fc6c4133b80 2023-10-03T10:14:36.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.475+0000 7fc6c1ffb700 1 -- 172.21.15.3:0/420064526 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6bc00b910 con 0x7fc6c4133b80 2023-10-03T10:14:36.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.475+0000 7fc6c1ffb700 1 -- 172.21.15.3:0/420064526 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc00bcc0 con 0x7fc6c4133b80 2023-10-03T10:14:36.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.476+0000 7fc6c93e3700 1 -- 172.21.15.3:0/420064526 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6c4133b80 msgr2=0x7fc6c4133fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:36.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.476+0000 7fc6c93e3700 1 --2- 172.21.15.3:0/420064526 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6c4133b80 0x7fc6c4133fe0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009e50 tx=0x7fc6bc00b330 comp rx=0 tx=0).stop 2023-10-03T10:14:36.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.476+0000 7fc6c93e3700 1 -- 172.21.15.3:0/420064526 shutdown_connections 2023-10-03T10:14:36.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.476+0000 7fc6c93e3700 1 --2- 172.21.15.3:0/420064526 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6c4133b80 0x7fc6c4133fe0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.476+0000 7fc6c93e3700 1 --2- 172.21.15.3:0/420064526 >> v2:172.21.15.3:3300/0 conn(0x7fc6c409e850 0x7fc6c409ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.476+0000 7fc6c93e3700 1 --2- 172.21.15.3:0/420064526 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6c409ffb0 0x7fc6c409f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.476+0000 7fc6c93e3700 1 -- 172.21.15.3:0/420064526 >> 172.21.15.3:0/420064526 conn(0x7fc6c40a3f70 msgr2=0x7fc6c40a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:36.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.476+0000 7fc6c93e3700 1 -- 172.21.15.3:0/420064526 shutdown_connections 2023-10-03T10:14:36.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.476+0000 7fc6c93e3700 1 -- 172.21.15.3:0/420064526 wait complete. 2023-10-03T10:14:36.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.477+0000 7fc6c93e3700 1 Processor -- start 2023-10-03T10:14:36.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.477+0000 7fc6c93e3700 1 -- start start 2023-10-03T10:14:36.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.477+0000 7fc6c93e3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6c409e850 0x7fc6c4120510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:36.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.477+0000 7fc6c93e3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6c409ffb0 0x7fc6c4120a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:36.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.477+0000 7fc6c93e3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6c4133b80 0x7fc6c4120f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:36.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.477+0000 7fc6c93e3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c4123d20 con 0x7fc6c4133b80 2023-10-03T10:14:36.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.477+0000 7fc6c93e3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c40a3430 con 0x7fc6c409e850 2023-10-03T10:14:36.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.477+0000 7fc6c93e3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c4122090 con 0x7fc6c409ffb0 2023-10-03T10:14:36.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.478+0000 7fc6c2ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6c409e850 0x7fc6c4120510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:36.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.478+0000 7fc6c2ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6c409e850 0x7fc6c4120510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50904/0 (socket says 172.21.15.3:50904) 2023-10-03T10:14:36.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.478+0000 7fc6c2ffd700 1 -- 172.21.15.3:0/2050590677 learned_addr learned my addr 172.21.15.3:0/2050590677 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:36.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.478+0000 7fc6c27fc700 1 --2- 172.21.15.3:0/2050590677 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6c409ffb0 0x7fc6c4120a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:36.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.478+0000 7fc6c37fe700 1 --2- 172.21.15.3:0/2050590677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6c4133b80 0x7fc6c4120f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:36.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.478+0000 7fc6c27fc700 1 -- 172.21.15.3:0/2050590677 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6c409e850 msgr2=0x7fc6c4120510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:36.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.478+0000 7fc6c27fc700 1 --2- 172.21.15.3:0/2050590677 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6c409e850 0x7fc6c4120510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.478+0000 7fc6c27fc700 1 -- 172.21.15.3:0/2050590677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6c4133b80 msgr2=0x7fc6c4120f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:36.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.478+0000 7fc6c27fc700 1 --2- 172.21.15.3:0/2050590677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6c4133b80 0x7fc6c4120f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.478+0000 7fc6c27fc700 1 -- 172.21.15.3:0/2050590677 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b40094a0 con 0x7fc6c409ffb0 2023-10-03T10:14:36.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.478+0000 7fc6c2ffd700 1 --2- 172.21.15.3:0/2050590677 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6c409e850 0x7fc6c4120510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:36.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.478+0000 7fc6c37fe700 1 --2- 172.21.15.3:0/2050590677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6c4133b80 0x7fc6c4120f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:36.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.478+0000 7fc6c27fc700 1 --2- 172.21.15.3:0/2050590677 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6c409ffb0 0x7fc6c4120a50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc6b800e9b0 tx=0x7fc6b800ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:36.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.478+0000 7fc6ab7fe700 1 -- 172.21.15.3:0/2050590677 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b800ee50 con 0x7fc6c409ffb0 2023-10-03T10:14:36.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.478+0000 7fc6ab7fe700 1 -- 172.21.15.3:0/2050590677 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b800c850 con 0x7fc6c409ffb0 2023-10-03T10:14:36.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.478+0000 7fc6c93e3700 1 -- 172.21.15.3:0/2050590677 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b80094a0 con 0x7fc6c409ffb0 2023-10-03T10:14:36.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.478+0000 7fc6ab7fe700 1 -- 172.21.15.3:0/2050590677 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b800cc00 con 0x7fc6c409ffb0 2023-10-03T10:14:36.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.478+0000 7fc6c93e3700 1 -- 172.21.15.3:0/2050590677 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6bc009b30 con 0x7fc6c409ffb0 2023-10-03T10:14:36.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.479+0000 7fc6ab7fe700 1 -- 172.21.15.3:0/2050590677 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc6b8013470 con 0x7fc6c409ffb0 2023-10-03T10:14:36.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.479+0000 7fc6ab7fe700 1 -- 172.21.15.3:0/2050590677 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc6b803f9e0 con 0x7fc6c409ffb0 2023-10-03T10:14:36.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.480+0000 7fc6c93e3700 1 -- 172.21.15.3:0/2050590677 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c409cbf0 con 0x7fc6c409ffb0 2023-10-03T10:14:36.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.485+0000 7fc6ab7fe700 1 -- 172.21.15.3:0/2050590677 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc6b8013720 con 0x7fc6c409ffb0 2023-10-03T10:14:36.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.691+0000 7fc6c93e3700 1 -- 172.21.15.3:0/2050590677 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc6c40747f0 con 0x7fc6c409ffb0 2023-10-03T10:14:36.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.691+0000 7fc6ab7fe700 1 -- 172.21.15.3:0/2050590677 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc6b8017030 con 0x7fc6c409ffb0 2023-10-03T10:14:36.692 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:36.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.693+0000 7fc6c93e3700 1 -- 172.21.15.3:0/2050590677 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6c409ffb0 msgr2=0x7fc6c4120a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:36.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.693+0000 7fc6c93e3700 1 --2- 172.21.15.3:0/2050590677 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6c409ffb0 0x7fc6c4120a50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc6b800e9b0 tx=0x7fc6b800ecc0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.693+0000 7fc6c93e3700 1 -- 172.21.15.3:0/2050590677 shutdown_connections 2023-10-03T10:14:36.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.693+0000 7fc6c93e3700 1 --2- 172.21.15.3:0/2050590677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6c4133b80 0x7fc6c4120f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.693+0000 7fc6c93e3700 1 --2- 172.21.15.3:0/2050590677 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6c409e850 0x7fc6c4120510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.693+0000 7fc6c93e3700 1 --2- 172.21.15.3:0/2050590677 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6c409ffb0 0x7fc6c4120a50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:36.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.693+0000 7fc6c93e3700 1 -- 172.21.15.3:0/2050590677 >> 172.21.15.3:0/2050590677 conn(0x7fc6c40a3f70 msgr2=0x7fc6c41bafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:36.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.693+0000 7fc6c93e3700 1 -- 172.21.15.3:0/2050590677 shutdown_connections 2023-10-03T10:14:36.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:36.693+0000 7fc6c93e3700 1 -- 172.21.15.3:0/2050590677 wait complete. 2023-10-03T10:14:36.708 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:37.313 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:37.348 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:37.388 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:37.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.463+0000 7fb018a26700 1 Processor -- start 2023-10-03T10:14:37.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.463+0000 7fb018a26700 1 -- start start 2023-10-03T10:14:37.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.464+0000 7fb018a26700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb01409e1d0 0x7fb014138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:37.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.464+0000 7fb018a26700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb014139170 0x7fb01413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:37.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.464+0000 7fb018a26700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb01409b120 con 0x7fb01409d6f0 2023-10-03T10:14:37.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.464+0000 7fb018a26700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb01406b460 con 0x7fb01409e1d0 2023-10-03T10:14:37.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.464+0000 7fb018a26700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb01409a7e0 con 0x7fb014139170 2023-10-03T10:14:37.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.464+0000 7fb012d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb014139170 0x7fb01413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:37.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.464+0000 7fb011d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb01409e1d0 0x7fb014138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:37.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.464+0000 7fb01259c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb01409d6f0 0x7fb01409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40442/0 (socket says 172.21.15.3:40442) 2023-10-03T10:14:37.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.464+0000 7fb01259c700 1 -- 172.21.15.3:0/258579956 learned_addr learned my addr 172.21.15.3:0/258579956 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:37.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.464+0000 7fb012d9d700 1 --2- 172.21.15.3:0/258579956 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb014139170 0x7fb01413b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:32794/0 (socket says 172.21.15.3:32794) 2023-10-03T10:14:37.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.465+0000 7fb012d9d700 1 -- 172.21.15.3:0/258579956 >> v2:172.21.15.3:3300/0 conn(0x7fb01409e1d0 msgr2=0x7fb014138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:37.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.465+0000 7fb012d9d700 1 --2- 172.21.15.3:0/258579956 >> v2:172.21.15.3:3300/0 conn(0x7fb01409e1d0 0x7fb014138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.465+0000 7fb012d9d700 1 -- 172.21.15.3:0/258579956 >> v1:172.21.15.3:6789/0 conn(0x7fb01409d6f0 legacy=0x7fb01409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:37.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.465+0000 7fb012d9d700 1 -- 172.21.15.3:0/258579956 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01409a500 con 0x7fb014139170 2023-10-03T10:14:37.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.465+0000 7fb011d9b700 1 --2- 172.21.15.3:0/258579956 >> v2:172.21.15.3:3300/0 conn(0x7fb01409e1d0 0x7fb014138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:37.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.465+0000 7fb012d9d700 1 --2- 172.21.15.3:0/258579956 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb014139170 0x7fb01413b550 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009dc0 tx=0x7fb00c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8aa10c197bc0e59d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:37.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.465+0000 7fb01159a700 1 -- 172.21.15.3:0/258579956 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c00e070 con 0x7fb014139170 2023-10-03T10:14:37.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.465+0000 7fb01159a700 1 -- 172.21.15.3:0/258579956 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00c00bae0 con 0x7fb014139170 2023-10-03T10:14:37.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.465+0000 7fb01159a700 1 -- 172.21.15.3:0/258579956 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c00be90 con 0x7fb014139170 2023-10-03T10:14:37.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.466+0000 7fb018a26700 1 -- 172.21.15.3:0/258579956 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb014139170 msgr2=0x7fb01413b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:37.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.466+0000 7fb018a26700 1 --2- 172.21.15.3:0/258579956 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb014139170 0x7fb01413b550 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009dc0 tx=0x7fb00c00b330 comp rx=0 tx=0).stop 2023-10-03T10:14:37.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.466+0000 7fb018a26700 1 -- 172.21.15.3:0/258579956 shutdown_connections 2023-10-03T10:14:37.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.466+0000 7fb018a26700 1 --2- 172.21.15.3:0/258579956 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb014139170 0x7fb01413b550 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.466+0000 7fb018a26700 1 --2- 172.21.15.3:0/258579956 >> v2:172.21.15.3:3300/0 conn(0x7fb01409e1d0 0x7fb014138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.466+0000 7fb018a26700 1 -- 172.21.15.3:0/258579956 >> 172.21.15.3:0/258579956 conn(0x7fb014066f50 msgr2=0x7fb01409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:37.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.466+0000 7fb018a26700 1 -- 172.21.15.3:0/258579956 shutdown_connections 2023-10-03T10:14:37.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.466+0000 7fb018a26700 1 -- 172.21.15.3:0/258579956 wait complete. 2023-10-03T10:14:37.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.468+0000 7fb018a26700 1 Processor -- start 2023-10-03T10:14:37.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.468+0000 7fb018a26700 1 -- start start 2023-10-03T10:14:37.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.468+0000 7fb018a26700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb01409d6f0 0x7fb0141c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:37.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.468+0000 7fb018a26700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb01409e1d0 0x7fb0141c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:37.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.469+0000 7fb018a26700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb014139170 0x7fb01411e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:37.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.469+0000 7fb018a26700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb014121fe0 con 0x7fb01409e1d0 2023-10-03T10:14:37.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.469+0000 7fb018a26700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0141be250 con 0x7fb014139170 2023-10-03T10:14:37.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.469+0000 7fb018a26700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb014122230 con 0x7fb01409d6f0 2023-10-03T10:14:37.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.469+0000 7fb012d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb014139170 0x7fb01411e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:37.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.469+0000 7fb012d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb014139170 0x7fb01411e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50910/0 (socket says 172.21.15.3:50910) 2023-10-03T10:14:37.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.469+0000 7fb012d9d700 1 -- 172.21.15.3:0/3029034906 learned_addr learned my addr 172.21.15.3:0/3029034906 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:37.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.469+0000 7fb011d9b700 1 --2- 172.21.15.3:0/3029034906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb01409e1d0 0x7fb0141c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:37.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.469+0000 7fb01259c700 1 --2- 172.21.15.3:0/3029034906 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb01409d6f0 0x7fb0141c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:37.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.469+0000 7fb012d9d700 1 -- 172.21.15.3:0/3029034906 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb01409d6f0 msgr2=0x7fb0141c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:37.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.469+0000 7fb012d9d700 1 --2- 172.21.15.3:0/3029034906 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb01409d6f0 0x7fb0141c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.469+0000 7fb012d9d700 1 -- 172.21.15.3:0/3029034906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb01409e1d0 msgr2=0x7fb0141c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:37.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.469+0000 7fb012d9d700 1 --2- 172.21.15.3:0/3029034906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb01409e1d0 0x7fb0141c8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.469+0000 7fb012d9d700 1 -- 172.21.15.3:0/3029034906 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0080094a0 con 0x7fb014139170 2023-10-03T10:14:37.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.469+0000 7fb011d9b700 1 --2- 172.21.15.3:0/3029034906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb01409e1d0 0x7fb0141c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:37.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.469+0000 7fb012d9d700 1 --2- 172.21.15.3:0/3029034906 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb014139170 0x7fb01411e710 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00b370 tx=0x7fb00c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:37.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.470+0000 7fb002ffd700 1 -- 172.21.15.3:0/3029034906 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c0045d0 con 0x7fb014139170 2023-10-03T10:14:37.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.470+0000 7fb002ffd700 1 -- 172.21.15.3:0/3029034906 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00c020070 con 0x7fb014139170 2023-10-03T10:14:37.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.470+0000 7fb002ffd700 1 -- 172.21.15.3:0/3029034906 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c027480 con 0x7fb014139170 2023-10-03T10:14:37.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.470+0000 7fb018a26700 1 -- 172.21.15.3:0/3029034906 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb00c009a70 con 0x7fb014139170 2023-10-03T10:14:37.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.470+0000 7fb018a26700 1 -- 172.21.15.3:0/3029034906 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb01409c710 con 0x7fb014139170 2023-10-03T10:14:37.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.470+0000 7fb002ffd700 1 -- 172.21.15.3:0/3029034906 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb00c012430 con 0x7fb014139170 2023-10-03T10:14:37.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.471+0000 7fb002ffd700 1 -- 172.21.15.3:0/3029034906 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb00c022070 con 0x7fb014139170 2023-10-03T10:14:37.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.471+0000 7fb018a26700 1 -- 172.21.15.3:0/3029034906 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb014074ea0 con 0x7fb014139170 2023-10-03T10:14:37.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.476+0000 7fb002ffd700 1 -- 172.21.15.3:0/3029034906 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb00c01c400 con 0x7fb014139170 2023-10-03T10:14:37.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.507+0000 7fe1ee0ad700 1 Processor -- start 2023-10-03T10:14:37.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.507+0000 7fe1ee0ad700 1 -- start start 2023-10-03T10:14:37.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.508+0000 7fe1ee0ad700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1e809e930 0x7fe1e809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:37.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.508+0000 7fe1ee0ad700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe1e809f240 0x7fe1e81317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:37.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.508+0000 7fe1ee0ad700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1e8070940 con 0x7fe1e809f650 2023-10-03T10:14:37.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.508+0000 7fe1ee0ad700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1e806fd60 con 0x7fe1e809f240 2023-10-03T10:14:37.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.508+0000 7fe1ee0ad700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e806fa80 con 0x7fe1e809e930 2023-10-03T10:14:37.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.508+0000 7fe1e77fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1e809e930 0x7fe1e809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:37.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.508+0000 7fe1e77fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1e809e930 0x7fe1e809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:32816/0 (socket says 172.21.15.3:32816) 2023-10-03T10:14:37.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.508+0000 7fe1e77fe700 1 -- 172.21.15.3:0/3685197641 learned_addr learned my addr 172.21.15.3:0/3685197641 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:37.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.508+0000 7fe1e6ffd700 1 --2- 172.21.15.3:0/3685197641 >> v2:172.21.15.3:3300/0 conn(0x7fe1e809f240 0x7fe1e81317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:37.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.508+0000 7fe1e77fe700 1 -- 172.21.15.3:0/3685197641 >> v2:172.21.15.3:3300/0 conn(0x7fe1e809f240 msgr2=0x7fe1e81317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:37.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.508+0000 7fe1e77fe700 1 --2- 172.21.15.3:0/3685197641 >> v2:172.21.15.3:3300/0 conn(0x7fe1e809f240 0x7fe1e81317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.508+0000 7fe1e77fe700 1 -- 172.21.15.3:0/3685197641 >> v1:172.21.15.3:6789/0 conn(0x7fe1e809f650 legacy=0x7fe1e8131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:37.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.508+0000 7fe1e77fe700 1 -- 172.21.15.3:0/3685197641 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e809fb60 con 0x7fe1e809e930 2023-10-03T10:14:37.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.508+0000 7fe1e67fc700 1 -- 172.21.15.3:0/3685197641 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2028602013 0 0) 0x7fe1e8070940 con 0x7fe1e809f650 2023-10-03T10:14:37.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.508+0000 7fe1e6ffd700 1 --2- 172.21.15.3:0/3685197641 >> v2:172.21.15.3:3300/0 conn(0x7fe1e809f240 0x7fe1e81317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:37.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.509+0000 7fe1e77fe700 1 --2- 172.21.15.3:0/3685197641 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1e809e930 0x7fe1e809ed00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe1d80098f0 tx=0x7fe1d8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1677f25fdfb38873 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:37.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.509+0000 7fe1e67fc700 1 -- 172.21.15.3:0/3685197641 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8011830 con 0x7fe1e809e930 2023-10-03T10:14:37.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.509+0000 7fe1e67fc700 1 -- 172.21.15.3:0/3685197641 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d8011990 con 0x7fe1e809e930 2023-10-03T10:14:37.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.509+0000 7fe1e67fc700 1 -- 172.21.15.3:0/3685197641 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8011d20 con 0x7fe1e809e930 2023-10-03T10:14:37.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.510+0000 7fe1ee0ad700 1 -- 172.21.15.3:0/3685197641 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1e809e930 msgr2=0x7fe1e809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:37.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.510+0000 7fe1ee0ad700 1 --2- 172.21.15.3:0/3685197641 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1e809e930 0x7fe1e809ed00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe1d80098f0 tx=0x7fe1d8009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:37.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.510+0000 7fe1ee0ad700 1 -- 172.21.15.3:0/3685197641 shutdown_connections 2023-10-03T10:14:37.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.510+0000 7fe1ee0ad700 1 --2- 172.21.15.3:0/3685197641 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1e809e930 0x7fe1e809ed00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.510+0000 7fe1ee0ad700 1 --2- 172.21.15.3:0/3685197641 >> v2:172.21.15.3:3300/0 conn(0x7fe1e809f240 0x7fe1e81317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.510+0000 7fe1ee0ad700 1 -- 172.21.15.3:0/3685197641 >> 172.21.15.3:0/3685197641 conn(0x7fe1e80a4210 msgr2=0x7fe1e80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:37.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.510+0000 7fe1ee0ad700 1 -- 172.21.15.3:0/3685197641 shutdown_connections 2023-10-03T10:14:37.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.510+0000 7fe1ee0ad700 1 -- 172.21.15.3:0/3685197641 wait complete. 2023-10-03T10:14:37.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.511+0000 7fe1ee0ad700 1 Processor -- start 2023-10-03T10:14:37.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.511+0000 7fe1ee0ad700 1 -- start start 2023-10-03T10:14:37.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.512+0000 7fe1ee0ad700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1e809e930 0x7fe1e81a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:37.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.512+0000 7fe1ee0ad700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1e809f240 0x7fe1e81a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:37.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.512+0000 7fe1ee0ad700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1e809f650 0x7fe1e81c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:37.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.512+0000 7fe1ee0ad700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e80a2fb0 con 0x7fe1e809f240 2023-10-03T10:14:37.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.512+0000 7fe1ee0ad700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e81b93e0 con 0x7fe1e809f650 2023-10-03T10:14:37.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.512+0000 7fe1ee0ad700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e81a4f30 con 0x7fe1e809e930 2023-10-03T10:14:37.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.512+0000 7fe1e77fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1e809e930 0x7fe1e81a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:37.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.512+0000 7fe1e77fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1e809e930 0x7fe1e81a54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:32830/0 (socket says 172.21.15.3:32830) 2023-10-03T10:14:37.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.512+0000 7fe1e77fe700 1 -- 172.21.15.3:0/3831201660 learned_addr learned my addr 172.21.15.3:0/3831201660 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:37.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.512+0000 7fe1e7fff700 1 --2- 172.21.15.3:0/3831201660 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1e809f650 0x7fe1e81c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:37.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.512+0000 7fe1e6ffd700 1 --2- 172.21.15.3:0/3831201660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1e809f240 0x7fe1e81a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:37.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.512+0000 7fe1e6ffd700 1 -- 172.21.15.3:0/3831201660 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1e809e930 msgr2=0x7fe1e81a54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:37.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.512+0000 7fe1e6ffd700 1 --2- 172.21.15.3:0/3831201660 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1e809e930 0x7fe1e81a54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.512+0000 7fe1e6ffd700 1 -- 172.21.15.3:0/3831201660 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1e809f650 msgr2=0x7fe1e81c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:37.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.512+0000 7fe1e6ffd700 1 --2- 172.21.15.3:0/3831201660 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1e809f650 0x7fe1e81c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.512+0000 7fe1e6ffd700 1 -- 172.21.15.3:0/3831201660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1dc0094a0 con 0x7fe1e809f240 2023-10-03T10:14:37.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.512+0000 7fe1e7fff700 1 --2- 172.21.15.3:0/3831201660 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1e809f650 0x7fe1e81c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:37.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.512+0000 7fe1e77fe700 1 --2- 172.21.15.3:0/3831201660 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1e809e930 0x7fe1e81a54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:37.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.512+0000 7fe1e6ffd700 1 --2- 172.21.15.3:0/3831201660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1e809f240 0x7fe1e81a59f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00e980 tx=0x7fe1dc00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:37.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.513+0000 7fe1cffff700 1 -- 172.21.15.3:0/3831201660 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc00c690 con 0x7fe1e809f240 2023-10-03T10:14:37.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.513+0000 7fe1cffff700 1 -- 172.21.15.3:0/3831201660 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1dc00c7f0 con 0x7fe1e809f240 2023-10-03T10:14:37.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.513+0000 7fe1ee0ad700 1 -- 172.21.15.3:0/3831201660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d8009550 con 0x7fe1e809f240 2023-10-03T10:14:37.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.513+0000 7fe1cffff700 1 -- 172.21.15.3:0/3831201660 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc00cb80 con 0x7fe1e809f240 2023-10-03T10:14:37.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.513+0000 7fe1ee0ad700 1 -- 172.21.15.3:0/3831201660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e80a0190 con 0x7fe1e809f240 2023-10-03T10:14:37.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.514+0000 7fe1cffff700 1 -- 172.21.15.3:0/3831201660 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe1dc00cce0 con 0x7fe1e809f240 2023-10-03T10:14:37.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.514+0000 7fe1cffff700 1 -- 172.21.15.3:0/3831201660 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe1dc03f6a0 con 0x7fe1e809f240 2023-10-03T10:14:37.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.514+0000 7fe1ee0ad700 1 -- 172.21.15.3:0/3831201660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1e81a6c10 con 0x7fe1e809f240 2023-10-03T10:14:37.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.519+0000 7fe1cffff700 1 -- 172.21.15.3:0/3831201660 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe1dc01bb50 con 0x7fe1e809f240 2023-10-03T10:14:37.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.547+0000 7f103a51d700 1 Processor -- start 2023-10-03T10:14:37.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.547+0000 7f103a51d700 1 -- start start 2023-10-03T10:14:37.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.547+0000 7f103a51d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f10340a0a30 0x7f10340a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:37.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.547+0000 7f103a51d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1034134170 0x7f103413bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:37.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.547+0000 7f103a51d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f103409edd0 con 0x7f10340a1340 2023-10-03T10:14:37.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.547+0000 7f103a51d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f103409e1f0 con 0x7f1034134170 2023-10-03T10:14:37.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.547+0000 7f103a51d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f103409dc50 con 0x7f10340a0a30 2023-10-03T10:14:37.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.548+0000 7f10337fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f10340a1340 0x7f1034133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40458/0 (socket says 172.21.15.3:40458) 2023-10-03T10:14:37.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.548+0000 7f10337fe700 1 -- 172.21.15.3:0/2324594681 learned_addr learned my addr 172.21.15.3:0/2324594681 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:37.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.548+0000 7f1038aba700 1 --2- 172.21.15.3:0/2324594681 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1034134170 0x7f103413bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:37.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.548+0000 7f1033fff700 1 --2- 172.21.15.3:0/2324594681 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f10340a0a30 0x7f10340a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:37.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.548+0000 7f1032ffd700 1 -- 172.21.15.3:0/2324594681 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2563716555 0 0) 0x7f103409edd0 con 0x7f10340a1340 2023-10-03T10:14:37.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.548+0000 7f1032ffd700 1 -- 172.21.15.3:0/2324594681 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f101c0035b0 con 0x7f10340a1340 2023-10-03T10:14:37.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.548+0000 7f1038aba700 1 -- 172.21.15.3:0/2324594681 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f10340a0a30 msgr2=0x7f10340a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:37.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.548+0000 7f1038aba700 1 --2- 172.21.15.3:0/2324594681 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f10340a0a30 0x7f10340a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.548+0000 7f1038aba700 1 -- 172.21.15.3:0/2324594681 >> v1:172.21.15.3:6789/0 conn(0x7f10340a1340 legacy=0x7f1034133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:37.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.548+0000 7f1038aba700 1 -- 172.21.15.3:0/2324594681 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10340a1fe0 con 0x7f1034134170 2023-10-03T10:14:37.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.548+0000 7f1033fff700 1 --2- 172.21.15.3:0/2324594681 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f10340a0a30 0x7f10340a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:37.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.548+0000 7f1038aba700 1 --2- 172.21.15.3:0/2324594681 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1034134170 0x7f103413bd30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f102c009dc0 tx=0x7f102c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4603b45d225f084c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:37.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.549+0000 7f1032ffd700 1 -- 172.21.15.3:0/2324594681 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c00e070 con 0x7f1034134170 2023-10-03T10:14:37.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.549+0000 7f1032ffd700 1 -- 172.21.15.3:0/2324594681 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102c00bb20 con 0x7f1034134170 2023-10-03T10:14:37.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.549+0000 7f1032ffd700 1 -- 172.21.15.3:0/2324594681 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c012430 con 0x7f1034134170 2023-10-03T10:14:37.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.549+0000 7f103a51d700 1 -- 172.21.15.3:0/2324594681 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1034134170 msgr2=0x7f103413bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:37.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.549+0000 7f103a51d700 1 --2- 172.21.15.3:0/2324594681 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1034134170 0x7f103413bd30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f102c009dc0 tx=0x7f102c00b330 comp rx=0 tx=0).stop 2023-10-03T10:14:37.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.549+0000 7f103a51d700 1 -- 172.21.15.3:0/2324594681 shutdown_connections 2023-10-03T10:14:37.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.549+0000 7f103a51d700 1 --2- 172.21.15.3:0/2324594681 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1034134170 0x7f103413bd30 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.549+0000 7f103a51d700 1 --2- 172.21.15.3:0/2324594681 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f10340a0a30 0x7f10340a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.549+0000 7f103a51d700 1 -- 172.21.15.3:0/2324594681 >> 172.21.15.3:0/2324594681 conn(0x7f10340a6550 msgr2=0x7f10340a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:37.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.549+0000 7f103a51d700 1 -- 172.21.15.3:0/2324594681 shutdown_connections 2023-10-03T10:14:37.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.549+0000 7f103a51d700 1 -- 172.21.15.3:0/2324594681 wait complete. 2023-10-03T10:14:37.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.551+0000 7f103a51d700 1 Processor -- start 2023-10-03T10:14:37.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.551+0000 7f103a51d700 1 -- start start 2023-10-03T10:14:37.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.552+0000 7f103a51d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10340a0a30 0x7f1034122890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:37.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.552+0000 7f103a51d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f10340a1340 0x7f1034122dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:37.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.552+0000 7f103a51d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1034134170 0x7f1034123310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:37.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.552+0000 7f103a51d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1034124400 con 0x7f10340a0a30 2023-10-03T10:14:37.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.552+0000 7f103a51d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f103409bce0 con 0x7f10340a1340 2023-10-03T10:14:37.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.552+0000 7f103a51d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f10340a0220 con 0x7f1034134170 2023-10-03T10:14:37.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.552+0000 7f1033fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10340a0a30 0x7f1034122890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:37.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.552+0000 7f1033fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10340a0a30 0x7f1034122890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50032/0 (socket says 172.21.15.3:50032) 2023-10-03T10:14:37.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.552+0000 7f1033fff700 1 -- 172.21.15.3:0/759806053 learned_addr learned my addr 172.21.15.3:0/759806053 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:37.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.552+0000 7f10337fe700 1 --2- 172.21.15.3:0/759806053 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f10340a1340 0x7f1034122dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:37.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.552+0000 7f1038aba700 1 --2- 172.21.15.3:0/759806053 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1034134170 0x7f1034123310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:37.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.552+0000 7f1033fff700 1 -- 172.21.15.3:0/759806053 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1034134170 msgr2=0x7f1034123310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:37.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.552+0000 7f1033fff700 1 --2- 172.21.15.3:0/759806053 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1034134170 0x7f1034123310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.552+0000 7f1033fff700 1 -- 172.21.15.3:0/759806053 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f10340a1340 msgr2=0x7f1034122dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:37.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.552+0000 7f1033fff700 1 --2- 172.21.15.3:0/759806053 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f10340a1340 0x7f1034122dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.552+0000 7f1033fff700 1 -- 172.21.15.3:0/759806053 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10240094a0 con 0x7f10340a0a30 2023-10-03T10:14:37.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.553+0000 7f1033fff700 1 --2- 172.21.15.3:0/759806053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10340a0a30 0x7f1034122890 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f102400ebe0 tx=0x7f102400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:37.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.553+0000 7f1030ff9700 1 -- 172.21.15.3:0/759806053 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102400c850 con 0x7f10340a0a30 2023-10-03T10:14:37.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.553+0000 7f1030ff9700 1 -- 172.21.15.3:0/759806053 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102400c9b0 con 0x7f10340a0a30 2023-10-03T10:14:37.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.553+0000 7f103a51d700 1 -- 172.21.15.3:0/759806053 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f102c009a70 con 0x7f10340a0a30 2023-10-03T10:14:37.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.553+0000 7f1030ff9700 1 -- 172.21.15.3:0/759806053 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102400cd20 con 0x7f10340a0a30 2023-10-03T10:14:37.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.553+0000 7f103a51d700 1 -- 172.21.15.3:0/759806053 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10340a25e0 con 0x7f10340a0a30 2023-10-03T10:14:37.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.554+0000 7f1030ff9700 1 -- 172.21.15.3:0/759806053 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1024013470 con 0x7f10340a0a30 2023-10-03T10:14:37.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.554+0000 7f1030ff9700 1 -- 172.21.15.3:0/759806053 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f102403f7b0 con 0x7f10340a0a30 2023-10-03T10:14:37.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.555+0000 7f103a51d700 1 -- 172.21.15.3:0/759806053 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1034074ea0 con 0x7f10340a0a30 2023-10-03T10:14:37.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.560+0000 7f1030ff9700 1 -- 172.21.15.3:0/759806053 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1024013720 con 0x7f10340a0a30 2023-10-03T10:14:37.709 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:37.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.714+0000 7fb018a26700 1 -- 172.21.15.3:0/3029034906 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb01409f420 con 0x7fb014139170 2023-10-03T10:14:37.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.715+0000 7fb002ffd700 1 -- 172.21.15.3:0/3029034906 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb00c036090 con 0x7fb014139170 2023-10-03T10:14:37.715 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:37.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.717+0000 7fb018a26700 1 -- 172.21.15.3:0/3029034906 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb014139170 msgr2=0x7fb01411e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:37.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.717+0000 7fb018a26700 1 --2- 172.21.15.3:0/3029034906 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb014139170 0x7fb01411e710 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00b370 tx=0x7fb00c004ad0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.717+0000 7fb018a26700 1 -- 172.21.15.3:0/3029034906 shutdown_connections 2023-10-03T10:14:37.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.717+0000 7fb018a26700 1 --2- 172.21.15.3:0/3029034906 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb014139170 0x7fb01411e710 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.717+0000 7fb018a26700 1 --2- 172.21.15.3:0/3029034906 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb01409d6f0 0x7fb0141c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.717+0000 7fb018a26700 1 --2- 172.21.15.3:0/3029034906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb01409e1d0 0x7fb0141c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.717+0000 7fb018a26700 1 -- 172.21.15.3:0/3029034906 >> 172.21.15.3:0/3029034906 conn(0x7fb014066f50 msgr2=0x7fb0140a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:37.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.718+0000 7fb018a26700 1 -- 172.21.15.3:0/3029034906 shutdown_connections 2023-10-03T10:14:37.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.718+0000 7fb018a26700 1 -- 172.21.15.3:0/3029034906 wait complete. 2023-10-03T10:14:37.733 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:37.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.762+0000 7fe1ee0ad700 1 -- 172.21.15.3:0/3831201660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe1e8074ea0 con 0x7fe1e809f240 2023-10-03T10:14:37.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.762+0000 7fe1cffff700 1 -- 172.21.15.3:0/3831201660 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe1dc021020 con 0x7fe1e809f240 2023-10-03T10:14:37.763 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:37.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.765+0000 7fe1ee0ad700 1 -- 172.21.15.3:0/3831201660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1e809f240 msgr2=0x7fe1e81a59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:37.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.765+0000 7fe1ee0ad700 1 --2- 172.21.15.3:0/3831201660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1e809f240 0x7fe1e81a59f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00e980 tx=0x7fe1dc00ec90 comp rx=0 tx=0).stop 2023-10-03T10:14:37.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.765+0000 7fe1ee0ad700 1 -- 172.21.15.3:0/3831201660 shutdown_connections 2023-10-03T10:14:37.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.765+0000 7fe1ee0ad700 1 --2- 172.21.15.3:0/3831201660 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1e809f650 0x7fe1e81c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.765+0000 7fe1ee0ad700 1 --2- 172.21.15.3:0/3831201660 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1e809e930 0x7fe1e81a54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.765+0000 7fe1ee0ad700 1 --2- 172.21.15.3:0/3831201660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1e809f240 0x7fe1e81a59f0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.765+0000 7fe1ee0ad700 1 -- 172.21.15.3:0/3831201660 >> 172.21.15.3:0/3831201660 conn(0x7fe1e80a4210 msgr2=0x7fe1e80a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:37.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.766+0000 7fe1ee0ad700 1 -- 172.21.15.3:0/3831201660 shutdown_connections 2023-10-03T10:14:37.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.766+0000 7fe1ee0ad700 1 -- 172.21.15.3:0/3831201660 wait complete. 2023-10-03T10:14:37.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.768+0000 7f103a51d700 1 -- 172.21.15.3:0/759806053 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f10340a28e0 con 0x7f10340a0a30 2023-10-03T10:14:37.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.769+0000 7f1030ff9700 1 -- 172.21.15.3:0/759806053 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1024017030 con 0x7f10340a0a30 2023-10-03T10:14:37.769 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:37.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.771+0000 7f103a51d700 1 -- 172.21.15.3:0/759806053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10340a0a30 msgr2=0x7f1034122890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:37.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.771+0000 7f103a51d700 1 --2- 172.21.15.3:0/759806053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10340a0a30 0x7f1034122890 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f102400ebe0 tx=0x7f102400eef0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.771+0000 7f103a51d700 1 -- 172.21.15.3:0/759806053 shutdown_connections 2023-10-03T10:14:37.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.771+0000 7f103a51d700 1 --2- 172.21.15.3:0/759806053 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1034134170 0x7f1034123310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.771+0000 7f103a51d700 1 --2- 172.21.15.3:0/759806053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10340a0a30 0x7f1034122890 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.771+0000 7f103a51d700 1 --2- 172.21.15.3:0/759806053 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f10340a1340 0x7f1034122dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.771+0000 7f103a51d700 1 -- 172.21.15.3:0/759806053 >> 172.21.15.3:0/759806053 conn(0x7f10340a6550 msgr2=0x7f103409edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:37.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.771+0000 7f103a51d700 1 -- 172.21.15.3:0/759806053 shutdown_connections 2023-10-03T10:14:37.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.771+0000 7f103a51d700 1 -- 172.21.15.3:0/759806053 wait complete. 2023-10-03T10:14:37.786 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:37.788 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:37.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.857+0000 7f9e603c0700 1 Processor -- start 2023-10-03T10:14:37.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.857+0000 7f9e603c0700 1 -- start start 2023-10-03T10:14:37.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.858+0000 7f9e603c0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9e5809d6f0 0x7f9e5809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:37.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.858+0000 7f9e603c0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e5809e000 0x7f9e58130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.858+0000 7f9e603c0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e5809f130 con 0x7f9e5809e410 2023-10-03T10:14:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.858+0000 7f9e603c0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e5809f520 con 0x7f9e5809d6f0 2023-10-03T10:14:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.858+0000 7f9e603c0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5809e930 con 0x7f9e5809e000 2023-10-03T10:14:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.858+0000 7f9e5e15c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9e5809d6f0 0x7f9e5809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.858+0000 7f9e5d95b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e5809e000 0x7f9e58130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.858+0000 7f9e5e15c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9e5809d6f0 0x7f9e5809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50034/0 (socket says 172.21.15.3:50034) 2023-10-03T10:14:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.858+0000 7f9e5d95b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e5809e000 0x7f9e58130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50938/0 (socket says 172.21.15.3:50938) 2023-10-03T10:14:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.858+0000 7f9e5e15c700 1 -- 172.21.15.3:0/2765694553 learned_addr learned my addr 172.21.15.3:0/2765694553 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.858+0000 7f9e5d95b700 1 -- 172.21.15.3:0/2765694553 >> v2:172.21.15.3:3300/0 conn(0x7f9e5809d6f0 msgr2=0x7f9e5809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.858+0000 7f9e5d95b700 1 --2- 172.21.15.3:0/2765694553 >> v2:172.21.15.3:3300/0 conn(0x7f9e5809d6f0 0x7f9e5809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.858+0000 7f9e5d95b700 1 -- 172.21.15.3:0/2765694553 >> v1:172.21.15.3:6789/0 conn(0x7f9e5809e410 legacy=0x7f9e58130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.859+0000 7f9e5d95b700 1 -- 172.21.15.3:0/2765694553 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5809b2d0 con 0x7f9e5809e000 2023-10-03T10:14:37.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.859+0000 7f9e5d15a700 1 -- 172.21.15.3:0/2765694553 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3955343758 0 0) 0x7f9e5809f130 con 0x7f9e5809e410 2023-10-03T10:14:37.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.859+0000 7f9e5e15c700 1 --2- 172.21.15.3:0/2765694553 >> v2:172.21.15.3:3300/0 conn(0x7f9e5809d6f0 0x7f9e5809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:37.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.859+0000 7f9e5d95b700 1 --2- 172.21.15.3:0/2765694553 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e5809e000 0x7f9e58130660 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009840 tx=0x7f9e48009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=560d7628435c4013 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:37.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.859+0000 7f9e5d15a700 1 -- 172.21.15.3:0/2765694553 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e48011830 con 0x7f9e5809e000 2023-10-03T10:14:37.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.859+0000 7f9e5d15a700 1 -- 172.21.15.3:0/2765694553 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e48011990 con 0x7f9e5809e000 2023-10-03T10:14:37.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.859+0000 7f9e5d15a700 1 -- 172.21.15.3:0/2765694553 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e48011d20 con 0x7f9e5809e000 2023-10-03T10:14:37.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.859+0000 7f9e603c0700 1 -- 172.21.15.3:0/2765694553 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e5809e000 msgr2=0x7f9e58130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:37.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.859+0000 7f9e603c0700 1 --2- 172.21.15.3:0/2765694553 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e5809e000 0x7f9e58130660 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009840 tx=0x7f9e48009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:37.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.860+0000 7f9e603c0700 1 -- 172.21.15.3:0/2765694553 shutdown_connections 2023-10-03T10:14:37.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.860+0000 7f9e603c0700 1 --2- 172.21.15.3:0/2765694553 >> v2:172.21.15.3:3300/0 conn(0x7f9e5809d6f0 0x7f9e5809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.860+0000 7f9e603c0700 1 --2- 172.21.15.3:0/2765694553 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e5809e000 0x7f9e58130660 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.860+0000 7f9e603c0700 1 -- 172.21.15.3:0/2765694553 >> 172.21.15.3:0/2765694553 conn(0x7f9e58066f50 msgr2=0x7f9e5809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:37.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.860+0000 7f9e603c0700 1 -- 172.21.15.3:0/2765694553 shutdown_connections 2023-10-03T10:14:37.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.860+0000 7f9e603c0700 1 -- 172.21.15.3:0/2765694553 wait complete. 2023-10-03T10:14:37.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.861+0000 7f9e603c0700 1 Processor -- start 2023-10-03T10:14:37.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.861+0000 7f9e603c0700 1 -- start start 2023-10-03T10:14:37.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.861+0000 7f9e603c0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e5809d6f0 0x7f9e5811ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:37.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.861+0000 7f9e603c0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e5809e000 0x7f9e58120440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:37.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.861+0000 7f9e603c0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e5809e410 0x7f9e58120980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:37.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.861+0000 7f9e603c0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e581302c0 con 0x7f9e5809e410 2023-10-03T10:14:37.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.861+0000 7f9e603c0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e58123480 con 0x7f9e5809e000 2023-10-03T10:14:37.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.861+0000 7f9e603c0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e58123740 con 0x7f9e5809d6f0 2023-10-03T10:14:37.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.861+0000 7f9e5e15c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e5809d6f0 0x7f9e5811ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:37.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.861+0000 7f9e5d95b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e5809e000 0x7f9e58120440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:37.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.861+0000 7f9e5e95d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e5809e410 0x7f9e58120980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:37.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.861+0000 7f9e5e15c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e5809d6f0 0x7f9e5811ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:32856/0 (socket says 172.21.15.3:32856) 2023-10-03T10:14:37.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.861+0000 7f9e5e15c700 1 -- 172.21.15.3:0/1552476598 learned_addr learned my addr 172.21.15.3:0/1552476598 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:37.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.862+0000 7f9e5d95b700 1 -- 172.21.15.3:0/1552476598 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e5809d6f0 msgr2=0x7f9e5811ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:37.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.862+0000 7f9e5d95b700 1 --2- 172.21.15.3:0/1552476598 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e5809d6f0 0x7f9e5811ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.862+0000 7f9e5d95b700 1 -- 172.21.15.3:0/1552476598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e5809e410 msgr2=0x7f9e58120980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:37.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.862+0000 7f9e5d95b700 1 --2- 172.21.15.3:0/1552476598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e5809e410 0x7f9e58120980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:37.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.862+0000 7f9e5d95b700 1 -- 172.21.15.3:0/1552476598 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e54009550 con 0x7f9e5809e000 2023-10-03T10:14:37.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.862+0000 7f9e5e95d700 1 --2- 172.21.15.3:0/1552476598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e5809e410 0x7f9e58120980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:37.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.862+0000 7f9e5d95b700 1 --2- 172.21.15.3:0/1552476598 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e5809e000 0x7f9e58120440 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9e48016000 tx=0x7f9e48004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:37.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.862+0000 7f9e4effd700 1 -- 172.21.15.3:0/1552476598 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e480036a0 con 0x7f9e5809e000 2023-10-03T10:14:37.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.862+0000 7f9e4effd700 1 -- 172.21.15.3:0/1552476598 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4801f070 con 0x7f9e5809e000 2023-10-03T10:14:37.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.862+0000 7f9e603c0700 1 -- 172.21.15.3:0/1552476598 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e480094a0 con 0x7f9e5809e000 2023-10-03T10:14:37.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.862+0000 7f9e4effd700 1 -- 172.21.15.3:0/1552476598 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e48003a20 con 0x7f9e5809e000 2023-10-03T10:14:37.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.862+0000 7f9e603c0700 1 -- 172.21.15.3:0/1552476598 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e580a1b80 con 0x7f9e5809e000 2023-10-03T10:14:37.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.863+0000 7f9e4effd700 1 -- 172.21.15.3:0/1552476598 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9e48003b80 con 0x7f9e5809e000 2023-10-03T10:14:37.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.863+0000 7f9e4effd700 1 -- 172.21.15.3:0/1552476598 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9e48021070 con 0x7f9e5809e000 2023-10-03T10:14:37.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.863+0000 7f9e603c0700 1 -- 172.21.15.3:0/1552476598 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e58074ea0 con 0x7f9e5809e000 2023-10-03T10:14:37.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:37.868+0000 7f9e4effd700 1 -- 172.21.15.3:0/1552476598 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9e4802b070 con 0x7f9e5809e000 2023-10-03T10:14:38.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.070+0000 7f9e603c0700 1 -- 172.21.15.3:0/1552476598 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9e5806c360 con 0x7f9e5809e000 2023-10-03T10:14:38.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.071+0000 7f9e4effd700 1 -- 172.21.15.3:0/1552476598 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9e4802b030 con 0x7f9e5809e000 2023-10-03T10:14:38.071 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:38.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.073+0000 7f9e603c0700 1 -- 172.21.15.3:0/1552476598 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e5809e000 msgr2=0x7f9e58120440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:38.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.073+0000 7f9e603c0700 1 --2- 172.21.15.3:0/1552476598 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e5809e000 0x7f9e58120440 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9e48016000 tx=0x7f9e48004750 comp rx=0 tx=0).stop 2023-10-03T10:14:38.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.073+0000 7f9e603c0700 1 -- 172.21.15.3:0/1552476598 shutdown_connections 2023-10-03T10:14:38.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.073+0000 7f9e603c0700 1 --2- 172.21.15.3:0/1552476598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e5809e410 0x7f9e58120980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:38.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.073+0000 7f9e603c0700 1 --2- 172.21.15.3:0/1552476598 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e5809d6f0 0x7f9e5811ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:38.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.073+0000 7f9e603c0700 1 --2- 172.21.15.3:0/1552476598 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e5809e000 0x7f9e58120440 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:38.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.073+0000 7f9e603c0700 1 -- 172.21.15.3:0/1552476598 >> 172.21.15.3:0/1552476598 conn(0x7f9e58066f50 msgr2=0x7f9e58071070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:38.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.073+0000 7f9e603c0700 1 -- 172.21.15.3:0/1552476598 shutdown_connections 2023-10-03T10:14:38.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.073+0000 7f9e603c0700 1 -- 172.21.15.3:0/1552476598 wait complete. 2023-10-03T10:14:38.087 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:38.734 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:38.787 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:38.789 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:38.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.885+0000 7f1768fa2700 1 Processor -- start 2023-10-03T10:14:38.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.885+0000 7f1768fa2700 1 -- start start 2023-10-03T10:14:38.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.886+0000 7f1768fa2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1764136830 0x7f1764138c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:38.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.886+0000 7f1768fa2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1764139150 0x7f176413b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:38.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.886+0000 7f1768fa2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f176406f720 con 0x7f17641338c0 2023-10-03T10:14:38.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.886+0000 7f1768fa2700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1764070c80 con 0x7f1764136830 2023-10-03T10:14:38.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.886+0000 7f1768fa2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176406fce0 con 0x7f1764139150 2023-10-03T10:14:38.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.886+0000 7f1761d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1764136830 0x7f1764138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:38.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.886+0000 7f1762d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1764139150 0x7f176413b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:38.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.886+0000 7f1761d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1764136830 0x7f1764138c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50054/0 (socket says 172.21.15.3:50054) 2023-10-03T10:14:38.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.886+0000 7f1761d9b700 1 -- 172.21.15.3:0/499588745 learned_addr learned my addr 172.21.15.3:0/499588745 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:38.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.886+0000 7f1761d9b700 1 -- 172.21.15.3:0/499588745 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1764139150 msgr2=0x7f176413b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:38.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.886+0000 7f1761d9b700 1 --2- 172.21.15.3:0/499588745 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1764139150 0x7f176413b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:38.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.886+0000 7f1761d9b700 1 -- 172.21.15.3:0/499588745 >> v1:172.21.15.3:6789/0 conn(0x7f17641338c0 legacy=0x7f1764133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:38.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.886+0000 7f1761d9b700 1 -- 172.21.15.3:0/499588745 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17641336a0 con 0x7f1764136830 2023-10-03T10:14:38.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.886+0000 7f1762d9d700 1 --2- 172.21.15.3:0/499588745 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1764139150 0x7f176413b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:38.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.886+0000 7f1761d9b700 1 --2- 172.21.15.3:0/499588745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1764136830 0x7f1764138c10 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f17580098f0 tx=0x7f1758009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1e6e0433197fe4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:38.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.887+0000 7f176159a700 1 -- 172.21.15.3:0/499588745 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1758011830 con 0x7f1764136830 2023-10-03T10:14:38.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.887+0000 7f176159a700 1 -- 172.21.15.3:0/499588745 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1758011990 con 0x7f1764136830 2023-10-03T10:14:38.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.887+0000 7f176159a700 1 -- 172.21.15.3:0/499588745 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1758011d00 con 0x7f1764136830 2023-10-03T10:14:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.888+0000 7f1768fa2700 1 -- 172.21.15.3:0/499588745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1764136830 msgr2=0x7f1764138c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.888+0000 7f1768fa2700 1 --2- 172.21.15.3:0/499588745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1764136830 0x7f1764138c10 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f17580098f0 tx=0x7f1758009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.888+0000 7f1768fa2700 1 -- 172.21.15.3:0/499588745 shutdown_connections 2023-10-03T10:14:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.888+0000 7f1768fa2700 1 --2- 172.21.15.3:0/499588745 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1764139150 0x7f176413b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:38.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.888+0000 7f1768fa2700 1 --2- 172.21.15.3:0/499588745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1764136830 0x7f1764138c10 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:38.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.888+0000 7f1768fa2700 1 -- 172.21.15.3:0/499588745 >> 172.21.15.3:0/499588745 conn(0x7f17640a2aa0 msgr2=0x7f17640a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:38.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.888+0000 7f1768fa2700 1 -- 172.21.15.3:0/499588745 shutdown_connections 2023-10-03T10:14:38.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.888+0000 7f1768fa2700 1 -- 172.21.15.3:0/499588745 wait complete. 2023-10-03T10:14:38.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.889+0000 7f1768fa2700 1 Processor -- start 2023-10-03T10:14:38.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.889+0000 7f1768fa2700 1 -- start start 2023-10-03T10:14:38.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.890+0000 7f1768fa2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17641338c0 0x7f1764119c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:38.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.890+0000 7f1768fa2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1764136830 0x7f176411a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:38.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.890+0000 7f1768fa2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1764139150 0x7f17641c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:38.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.890+0000 7f1768fa2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176411c9a0 con 0x7f1764139150 2023-10-03T10:14:38.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.890+0000 7f1768fa2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1764119640 con 0x7f17641338c0 2023-10-03T10:14:38.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.890+0000 7f1768fa2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f176406f960 con 0x7f1764136830 2023-10-03T10:14:38.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.890+0000 7f1761d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1764136830 0x7f176411a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:38.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.890+0000 7f176259c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17641338c0 0x7f1764119c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:38.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.890+0000 7f1761d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1764136830 0x7f176411a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:32860/0 (socket says 172.21.15.3:32860) 2023-10-03T10:14:38.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.890+0000 7f1761d9b700 1 -- 172.21.15.3:0/2356015835 learned_addr learned my addr 172.21.15.3:0/2356015835 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:38.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.890+0000 7f1762d9d700 1 --2- 172.21.15.3:0/2356015835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1764139150 0x7f17641c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:38.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.890+0000 7f1761d9b700 1 -- 172.21.15.3:0/2356015835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17641338c0 msgr2=0x7f1764119c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:38.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.890+0000 7f1761d9b700 1 --2- 172.21.15.3:0/2356015835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17641338c0 0x7f1764119c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:38.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.890+0000 7f1761d9b700 1 -- 172.21.15.3:0/2356015835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1764139150 msgr2=0x7f17641c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:38.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.890+0000 7f1761d9b700 1 --2- 172.21.15.3:0/2356015835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1764139150 0x7f17641c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:38.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.890+0000 7f1761d9b700 1 -- 172.21.15.3:0/2356015835 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c009a70 con 0x7f1764136830 2023-10-03T10:14:38.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.890+0000 7f1761d9b700 1 --2- 172.21.15.3:0/2356015835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1764136830 0x7f176411a160 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1758013000 tx=0x7f1758004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:38.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.891+0000 7f1752ffd700 1 -- 172.21.15.3:0/2356015835 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1758004400 con 0x7f1764136830 2023-10-03T10:14:38.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.891+0000 7f1752ffd700 1 -- 172.21.15.3:0/2356015835 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175801f070 con 0x7f1764136830 2023-10-03T10:14:38.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.891+0000 7f1752ffd700 1 -- 172.21.15.3:0/2356015835 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17580038e0 con 0x7f1764136830 2023-10-03T10:14:38.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.891+0000 7f1768fa2700 1 -- 172.21.15.3:0/2356015835 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1758009550 con 0x7f1764136830 2023-10-03T10:14:38.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.891+0000 7f1768fa2700 1 -- 172.21.15.3:0/2356015835 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1764136110 con 0x7f1764136830 2023-10-03T10:14:38.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.892+0000 7f1752ffd700 1 -- 172.21.15.3:0/2356015835 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1758021030 con 0x7f1764136830 2023-10-03T10:14:38.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.892+0000 7f1752ffd700 1 -- 172.21.15.3:0/2356015835 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1758040a70 con 0x7f1764136830 2023-10-03T10:14:38.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.892+0000 7f1768fa2700 1 -- 172.21.15.3:0/2356015835 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1764074ea0 con 0x7f1764136830 2023-10-03T10:14:38.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.897+0000 7f1752ffd700 1 -- 172.21.15.3:0/2356015835 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1758004580 con 0x7f1764136830 2023-10-03T10:14:38.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.932+0000 7f65f6311700 1 Processor -- start 2023-10-03T10:14:38.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.933+0000 7f65f6311700 1 -- start start 2023-10-03T10:14:38.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.933+0000 7f65f6311700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65f009d630 0x7f65f00a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:38.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.933+0000 7f65f6311700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f65f0138d80 0x7f65f013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:38.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.934+0000 7f65f6311700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65f00a2a70 con 0x7f65f00a0a30 2023-10-03T10:14:38.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.934+0000 7f65f6311700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f00a01d0 con 0x7f65f0138d80 2023-10-03T10:14:38.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.934+0000 7f65f6311700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f009e2b0 con 0x7f65f009d630 2023-10-03T10:14:38.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.934+0000 7f65effff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f65f00a0a30 0x7f65f00a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40484/0 (socket says 172.21.15.3:40484) 2023-10-03T10:14:38.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.934+0000 7f65effff700 1 -- 172.21.15.3:0/3297416221 learned_addr learned my addr 172.21.15.3:0/3297416221 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:38.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.934+0000 7f65ef7fe700 1 --2- 172.21.15.3:0/3297416221 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65f009d630 0x7f65f00a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:38.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.934+0000 7f65f48ae700 1 --2- 172.21.15.3:0/3297416221 >> v2:172.21.15.3:3300/0 conn(0x7f65f0138d80 0x7f65f013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.934+0000 7f65ef7fe700 1 -- 172.21.15.3:0/3297416221 >> v2:172.21.15.3:3300/0 conn(0x7f65f0138d80 msgr2=0x7f65f013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.934+0000 7f65ef7fe700 1 --2- 172.21.15.3:0/3297416221 >> v2:172.21.15.3:3300/0 conn(0x7f65f0138d80 0x7f65f013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.934+0000 7f65ef7fe700 1 -- 172.21.15.3:0/3297416221 >> v1:172.21.15.3:6789/0 conn(0x7f65f00a0a30 legacy=0x7f65f00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.934+0000 7f65ef7fe700 1 -- 172.21.15.3:0/3297416221 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f009dc50 con 0x7f65f009d630 2023-10-03T10:14:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.934+0000 7f65f48ae700 1 --2- 172.21.15.3:0/3297416221 >> v2:172.21.15.3:3300/0 conn(0x7f65f0138d80 0x7f65f013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.934+0000 7f65ef7fe700 1 --2- 172.21.15.3:0/3297416221 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65f009d630 0x7f65f00a1510 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f65e4009840 tx=0x7f65e4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a56557132adc9b56 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.934+0000 7f65eeffd700 1 -- 172.21.15.3:0/3297416221 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e4011830 con 0x7f65f009d630 2023-10-03T10:14:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.935+0000 7f65eeffd700 1 -- 172.21.15.3:0/3297416221 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e4011990 con 0x7f65f009d630 2023-10-03T10:14:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.935+0000 7f65eeffd700 1 -- 172.21.15.3:0/3297416221 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e4011d20 con 0x7f65f009d630 2023-10-03T10:14:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.935+0000 7f65f6311700 1 -- 172.21.15.3:0/3297416221 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65f009d630 msgr2=0x7f65f00a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.935+0000 7f65f6311700 1 --2- 172.21.15.3:0/3297416221 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65f009d630 0x7f65f00a1510 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f65e4009840 tx=0x7f65e4009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.935+0000 7f65f6311700 1 -- 172.21.15.3:0/3297416221 shutdown_connections 2023-10-03T10:14:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.935+0000 7f65f6311700 1 --2- 172.21.15.3:0/3297416221 >> v2:172.21.15.3:3300/0 conn(0x7f65f0138d80 0x7f65f013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.935+0000 7f65f6311700 1 --2- 172.21.15.3:0/3297416221 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65f009d630 0x7f65f00a1510 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.935+0000 7f65f6311700 1 -- 172.21.15.3:0/3297416221 >> 172.21.15.3:0/3297416221 conn(0x7f65f00a3050 msgr2=0x7f65f00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.935+0000 7f65f6311700 1 -- 172.21.15.3:0/3297416221 shutdown_connections 2023-10-03T10:14:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.935+0000 7f65f6311700 1 -- 172.21.15.3:0/3297416221 wait complete. 2023-10-03T10:14:38.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.936+0000 7f65f6311700 1 Processor -- start 2023-10-03T10:14:38.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.937+0000 7f65f6311700 1 -- start start 2023-10-03T10:14:38.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.937+0000 7f65f6311700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65f009d630 0x7f65f0133b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:38.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.937+0000 7f65f6311700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65f00a0a30 0x7f65f0134090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:38.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.937+0000 7f65f6311700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65f0138d80 0x7f65f01345d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:38.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.937+0000 7f65f6311700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f0135010 con 0x7f65f0138d80 2023-10-03T10:14:38.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.937+0000 7f65f6311700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f013afc0 con 0x7f65f00a0a30 2023-10-03T10:14:38.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.937+0000 7f65f6311700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f65f0135c30 con 0x7f65f009d630 2023-10-03T10:14:38.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.937+0000 7f65ef7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65f00a0a30 0x7f65f0134090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:38.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.937+0000 7f65f48ae700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65f0138d80 0x7f65f01345d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:38.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.937+0000 7f65f48ae700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65f0138d80 0x7f65f01345d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50082/0 (socket says 172.21.15.3:50082) 2023-10-03T10:14:38.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.937+0000 7f65ef7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65f00a0a30 0x7f65f0134090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50986/0 (socket says 172.21.15.3:50986) 2023-10-03T10:14:38.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.937+0000 7f65f48ae700 1 -- 172.21.15.3:0/3893426352 learned_addr learned my addr 172.21.15.3:0/3893426352 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:38.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.938+0000 7f65ef7fe700 1 -- 172.21.15.3:0/3893426352 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65f009d630 msgr2=0x7f65f0133b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:38.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.938+0000 7f65effff700 1 --2- 172.21.15.3:0/3893426352 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65f009d630 0x7f65f0133b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:38.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.938+0000 7f65ef7fe700 1 --2- 172.21.15.3:0/3893426352 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65f009d630 0x7f65f0133b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:38.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.938+0000 7f65ef7fe700 1 -- 172.21.15.3:0/3893426352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65f0138d80 msgr2=0x7f65f01345d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:38.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.938+0000 7f65ef7fe700 1 --2- 172.21.15.3:0/3893426352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65f0138d80 0x7f65f01345d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:38.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.938+0000 7f65ef7fe700 1 -- 172.21.15.3:0/3893426352 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e8009a70 con 0x7f65f00a0a30 2023-10-03T10:14:38.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.938+0000 7f65f48ae700 1 --2- 172.21.15.3:0/3893426352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65f0138d80 0x7f65f01345d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:38.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.938+0000 7f65effff700 1 --2- 172.21.15.3:0/3893426352 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65f009d630 0x7f65f0133b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:14:38.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.938+0000 7f65ef7fe700 1 --2- 172.21.15.3:0/3893426352 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65f00a0a30 0x7f65f0134090 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f65e4016000 tx=0x7f65e4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:38.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.938+0000 7f65ecff9700 1 -- 172.21.15.3:0/3893426352 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e4004480 con 0x7f65f00a0a30 2023-10-03T10:14:38.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.938+0000 7f65ecff9700 1 -- 172.21.15.3:0/3893426352 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e401f070 con 0x7f65f00a0a30 2023-10-03T10:14:38.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.938+0000 7f65f6311700 1 -- 172.21.15.3:0/3893426352 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e40094a0 con 0x7f65f00a0a30 2023-10-03T10:14:38.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.938+0000 7f65ecff9700 1 -- 172.21.15.3:0/3893426352 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e40038e0 con 0x7f65f00a0a30 2023-10-03T10:14:38.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.938+0000 7f65f6311700 1 -- 172.21.15.3:0/3893426352 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f0137590 con 0x7f65f00a0a30 2023-10-03T10:14:38.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.940+0000 7f65ecff9700 1 -- 172.21.15.3:0/3893426352 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f65e4021030 con 0x7f65f00a0a30 2023-10-03T10:14:38.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.940+0000 7f65ecff9700 1 -- 172.21.15.3:0/3893426352 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f65e4040d00 con 0x7f65f00a0a30 2023-10-03T10:14:38.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.940+0000 7f65f6311700 1 -- 172.21.15.3:0/3893426352 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f00747f0 con 0x7f65f00a0a30 2023-10-03T10:14:38.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.942+0000 7f6824ceb700 1 Processor -- start 2023-10-03T10:14:38.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.942+0000 7f6824ceb700 1 -- start start 2023-10-03T10:14:38.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.943+0000 7f6824ceb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6820130750 0x7f6820130b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:38.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.943+0000 7f6824ceb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f682009e250 0x7f68201384c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:38.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.943+0000 7f6824ceb700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f682009c190 con 0x7f682009d6f0 2023-10-03T10:14:38.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.943+0000 7f6824ceb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f682009b090 con 0x7f6820130750 2023-10-03T10:14:38.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.943+0000 7f6824ceb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f682009a6c0 con 0x7f682009e250 2023-10-03T10:14:38.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.943+0000 7f681e59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6820130750 0x7f6820130b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:38.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.943+0000 7f681e59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6820130750 0x7f6820130b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50998/0 (socket says 172.21.15.3:50998) 2023-10-03T10:14:38.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.943+0000 7f681dd9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f682009d6f0 0x7f682009db40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40488/0 (socket says 172.21.15.3:40488) 2023-10-03T10:14:38.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.943+0000 7f681e59c700 1 -- 172.21.15.3:0/3416101980 learned_addr learned my addr 172.21.15.3:0/3416101980 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:38.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.944+0000 7f681ed9d700 1 --2- 172.21.15.3:0/3416101980 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f682009e250 0x7f68201384c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:38.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.944+0000 7f681e59c700 1 -- 172.21.15.3:0/3416101980 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f682009e250 msgr2=0x7f68201384c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:38.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.944+0000 7f681d59a700 1 -- 172.21.15.3:0/3416101980 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1987086831 0 0) 0x7f682009c190 con 0x7f682009d6f0 2023-10-03T10:14:38.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.944+0000 7f681e59c700 1 --2- 172.21.15.3:0/3416101980 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f682009e250 0x7f68201384c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:38.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.944+0000 7f681e59c700 1 -- 172.21.15.3:0/3416101980 >> v1:172.21.15.3:6789/0 conn(0x7f682009d6f0 legacy=0x7f682009db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:38.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.944+0000 7f681e59c700 1 -- 172.21.15.3:0/3416101980 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6820131060 con 0x7f6820130750 2023-10-03T10:14:38.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.944+0000 7f681e59c700 1 --2- 172.21.15.3:0/3416101980 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6820130750 0x7f6820130b20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f681000bbc0 tx=0x7f681000bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51177004ef1a4a39 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:38.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.946+0000 7f681d59a700 1 -- 172.21.15.3:0/3416101980 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681000dd30 con 0x7f6820130750 2023-10-03T10:14:38.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.946+0000 7f681d59a700 1 -- 172.21.15.3:0/3416101980 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681000de90 con 0x7f6820130750 2023-10-03T10:14:38.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.946+0000 7f681d59a700 1 -- 172.21.15.3:0/3416101980 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6810010690 con 0x7f6820130750 2023-10-03T10:14:38.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.946+0000 7f65ecff9700 1 -- 172.21.15.3:0/3893426352 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f65e4015070 con 0x7f65f00a0a30 2023-10-03T10:14:38.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.946+0000 7f6824ceb700 1 -- 172.21.15.3:0/3416101980 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6820130750 msgr2=0x7f6820130b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:38.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.946+0000 7f6824ceb700 1 --2- 172.21.15.3:0/3416101980 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6820130750 0x7f6820130b20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f681000bbc0 tx=0x7f681000bed0 comp rx=0 tx=0).stop 2023-10-03T10:14:38.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.946+0000 7f6824ceb700 1 -- 172.21.15.3:0/3416101980 shutdown_connections 2023-10-03T10:14:38.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.946+0000 7f6824ceb700 1 --2- 172.21.15.3:0/3416101980 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6820130750 0x7f6820130b20 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:38.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.946+0000 7f6824ceb700 1 --2- 172.21.15.3:0/3416101980 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f682009e250 0x7f68201384c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:38.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.946+0000 7f6824ceb700 1 -- 172.21.15.3:0/3416101980 >> 172.21.15.3:0/3416101980 conn(0x7f682009f470 msgr2=0x7f68200a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:38.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.947+0000 7f6824ceb700 1 -- 172.21.15.3:0/3416101980 shutdown_connections 2023-10-03T10:14:38.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.947+0000 7f6824ceb700 1 -- 172.21.15.3:0/3416101980 wait complete. 2023-10-03T10:14:38.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.948+0000 7f6824ceb700 1 Processor -- start 2023-10-03T10:14:38.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.948+0000 7f6824ceb700 1 -- start start 2023-10-03T10:14:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.948+0000 7f6824ceb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f682009d6f0 0x7f682012aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.948+0000 7f6824ceb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f682009e250 0x7f682012af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.948+0000 7f6824ceb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6820130750 0x7f682012b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.948+0000 7f6824ceb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68200a0240 con 0x7f6820130750 2023-10-03T10:14:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.948+0000 7f6824ceb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f682012d380 con 0x7f682009d6f0 2023-10-03T10:14:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.948+0000 7f6824ceb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f682009b2b0 con 0x7f682009e250 2023-10-03T10:14:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.949+0000 7f681dd9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f682009e250 0x7f682012af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.949+0000 7f681dd9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f682009e250 0x7f682012af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:32896/0 (socket says 172.21.15.3:32896) 2023-10-03T10:14:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.949+0000 7f681dd9b700 1 -- 172.21.15.3:0/2634038299 learned_addr learned my addr 172.21.15.3:0/2634038299 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.949+0000 7f681ed9d700 1 --2- 172.21.15.3:0/2634038299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6820130750 0x7f682012b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:38.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.949+0000 7f681e59c700 1 --2- 172.21.15.3:0/2634038299 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f682009d6f0 0x7f682012aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:38.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.949+0000 7f681dd9b700 1 -- 172.21.15.3:0/2634038299 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f682009d6f0 msgr2=0x7f682012aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:38.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.949+0000 7f681dd9b700 1 --2- 172.21.15.3:0/2634038299 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f682009d6f0 0x7f682012aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:38.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.949+0000 7f681dd9b700 1 -- 172.21.15.3:0/2634038299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6820130750 msgr2=0x7f682012b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:38.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.949+0000 7f681dd9b700 1 --2- 172.21.15.3:0/2634038299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6820130750 0x7f682012b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:38.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.949+0000 7f681dd9b700 1 -- 172.21.15.3:0/2634038299 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681000b7f0 con 0x7f682009e250 2023-10-03T10:14:38.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.949+0000 7f681dd9b700 1 --2- 172.21.15.3:0/2634038299 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f682009e250 0x7f682012af80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f681400c450 tx=0x7f681400c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:38.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.949+0000 7f680effd700 1 -- 172.21.15.3:0/2634038299 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814004c80 con 0x7f682009e250 2023-10-03T10:14:38.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.949+0000 7f680effd700 1 -- 172.21.15.3:0/2634038299 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6814004e00 con 0x7f682009e250 2023-10-03T10:14:38.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.949+0000 7f6824ceb700 1 -- 172.21.15.3:0/2634038299 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f682012d7a0 con 0x7f682009e250 2023-10-03T10:14:38.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.949+0000 7f680effd700 1 -- 172.21.15.3:0/2634038299 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814007310 con 0x7f682009e250 2023-10-03T10:14:38.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.950+0000 7f6824ceb700 1 -- 172.21.15.3:0/2634038299 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f682012eb80 con 0x7f682009e250 2023-10-03T10:14:38.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.950+0000 7f680effd700 1 -- 172.21.15.3:0/2634038299 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6814007470 con 0x7f682009e250 2023-10-03T10:14:38.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.950+0000 7f680effd700 1 -- 172.21.15.3:0/2634038299 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f681403f2d0 con 0x7f682009e250 2023-10-03T10:14:38.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.951+0000 7f6824ceb700 1 -- 172.21.15.3:0/2634038299 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68200747f0 con 0x7f682009e250 2023-10-03T10:14:38.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:38.956+0000 7f680effd700 1 -- 172.21.15.3:0/2634038299 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6814007720 con 0x7f682009e250 2023-10-03T10:14:39.088 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:39.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.131+0000 7f1768fa2700 1 -- 172.21.15.3:0/2356015835 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1764072120 con 0x7f1764136830 2023-10-03T10:14:39.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.131+0000 7f1752ffd700 1 -- 172.21.15.3:0/2356015835 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1758015020 con 0x7f1764136830 2023-10-03T10:14:39.132 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:39.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.135+0000 7f1768fa2700 1 -- 172.21.15.3:0/2356015835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1764136830 msgr2=0x7f176411a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:39.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.135+0000 7f1768fa2700 1 --2- 172.21.15.3:0/2356015835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1764136830 0x7f176411a160 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1758013000 tx=0x7f1758004750 comp rx=0 tx=0).stop 2023-10-03T10:14:39.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.135+0000 7f1768fa2700 1 -- 172.21.15.3:0/2356015835 shutdown_connections 2023-10-03T10:14:39.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.135+0000 7f1768fa2700 1 --2- 172.21.15.3:0/2356015835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1764139150 0x7f17641c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:39.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.135+0000 7f1768fa2700 1 --2- 172.21.15.3:0/2356015835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17641338c0 0x7f1764119c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:39.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.135+0000 7f1768fa2700 1 --2- 172.21.15.3:0/2356015835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1764136830 0x7f176411a160 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:39.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.135+0000 7f1768fa2700 1 -- 172.21.15.3:0/2356015835 >> 172.21.15.3:0/2356015835 conn(0x7f17640a2aa0 msgr2=0x7f176412f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:39.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.135+0000 7f1768fa2700 1 -- 172.21.15.3:0/2356015835 shutdown_connections 2023-10-03T10:14:39.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.135+0000 7f1768fa2700 1 -- 172.21.15.3:0/2356015835 wait complete. 2023-10-03T10:14:39.155 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:39.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.172+0000 7f65f6311700 1 -- 172.21.15.3:0/3893426352 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f65f0072980 con 0x7f65f00a0a30 2023-10-03T10:14:39.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.172+0000 7f65ecff9700 1 -- 172.21.15.3:0/3893426352 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f65e401a7d0 con 0x7f65f00a0a30 2023-10-03T10:14:39.172 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:39.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.175+0000 7f65f6311700 1 -- 172.21.15.3:0/3893426352 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65f00a0a30 msgr2=0x7f65f0134090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:39.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.175+0000 7f65f6311700 1 --2- 172.21.15.3:0/3893426352 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65f00a0a30 0x7f65f0134090 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f65e4016000 tx=0x7f65e4004750 comp rx=0 tx=0).stop 2023-10-03T10:14:39.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.175+0000 7f65f6311700 1 -- 172.21.15.3:0/3893426352 shutdown_connections 2023-10-03T10:14:39.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.175+0000 7f65f6311700 1 --2- 172.21.15.3:0/3893426352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65f0138d80 0x7f65f01345d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:39.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.175+0000 7f65f6311700 1 --2- 172.21.15.3:0/3893426352 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65f009d630 0x7f65f0133b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:39.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.175+0000 7f65f6311700 1 --2- 172.21.15.3:0/3893426352 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65f00a0a30 0x7f65f0134090 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:39.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.175+0000 7f65f6311700 1 -- 172.21.15.3:0/3893426352 >> 172.21.15.3:0/3893426352 conn(0x7f65f00a3050 msgr2=0x7f65f009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:39.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.175+0000 7f65f6311700 1 -- 172.21.15.3:0/3893426352 shutdown_connections 2023-10-03T10:14:39.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.175+0000 7f65f6311700 1 -- 172.21.15.3:0/3893426352 wait complete. 2023-10-03T10:14:39.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.184+0000 7f6824ceb700 1 -- 172.21.15.3:0/2634038299 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f682012efd0 con 0x7f682009e250 2023-10-03T10:14:39.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.184+0000 7f680effd700 1 -- 172.21.15.3:0/2634038299 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f681401b340 con 0x7f682009e250 2023-10-03T10:14:39.184 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:39.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.187+0000 7f6824ceb700 1 -- 172.21.15.3:0/2634038299 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f682009e250 msgr2=0x7f682012af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:39.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.187+0000 7f6824ceb700 1 --2- 172.21.15.3:0/2634038299 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f682009e250 0x7f682012af80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f681400c450 tx=0x7f681400c760 comp rx=0 tx=0).stop 2023-10-03T10:14:39.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.187+0000 7f6824ceb700 1 -- 172.21.15.3:0/2634038299 shutdown_connections 2023-10-03T10:14:39.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.187+0000 7f6824ceb700 1 --2- 172.21.15.3:0/2634038299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6820130750 0x7f682012b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:39.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.187+0000 7f6824ceb700 1 --2- 172.21.15.3:0/2634038299 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f682009d6f0 0x7f682012aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:39.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.187+0000 7f6824ceb700 1 --2- 172.21.15.3:0/2634038299 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f682009e250 0x7f682012af80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:39.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.187+0000 7f6824ceb700 1 -- 172.21.15.3:0/2634038299 >> 172.21.15.3:0/2634038299 conn(0x7f682009f470 msgr2=0x7f6820071840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:39.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.188+0000 7f6824ceb700 1 -- 172.21.15.3:0/2634038299 shutdown_connections 2023-10-03T10:14:39.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.188+0000 7f6824ceb700 1 -- 172.21.15.3:0/2634038299 wait complete. 2023-10-03T10:14:39.190 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:39.202 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:39.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.232+0000 7f6018f90700 1 Processor -- start 2023-10-03T10:14:39.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.232+0000 7f6018f90700 1 -- start start 2023-10-03T10:14:39.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.232+0000 7f6018f90700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f601409d6f0 0x7f601409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:39.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.232+0000 7f6018f90700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f601409e410 0x7f6014134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:39.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.232+0000 7f6018f90700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f601409b120 con 0x7f601409e000 2023-10-03T10:14:39.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.232+0000 7f6018f90700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f601406b460 con 0x7f601409e410 2023-10-03T10:14:39.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.232+0000 7f6018f90700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f601409a7e0 con 0x7f601409d6f0 2023-10-03T10:14:39.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.233+0000 7f601259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f601409d6f0 0x7f601409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:39.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.233+0000 7f6012d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f601409e410 0x7f6014134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:39.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.233+0000 7f6012d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f601409e410 0x7f6014134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50098/0 (socket says 172.21.15.3:50098) 2023-10-03T10:14:39.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.233+0000 7f601259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f601409d6f0 0x7f601409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:32912/0 (socket says 172.21.15.3:32912) 2023-10-03T10:14:39.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.233+0000 7f6012d9d700 1 -- 172.21.15.3:0/3365137067 learned_addr learned my addr 172.21.15.3:0/3365137067 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:39.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.233+0000 7f6012d9d700 1 -- 172.21.15.3:0/3365137067 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f601409d6f0 msgr2=0x7f601409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:39.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.233+0000 7f6012d9d700 1 --2- 172.21.15.3:0/3365137067 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f601409d6f0 0x7f601409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:39.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.233+0000 7f6012d9d700 1 -- 172.21.15.3:0/3365137067 >> v1:172.21.15.3:6789/0 conn(0x7f601409e000 legacy=0x7f60141339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:39.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.233+0000 7f6012d9d700 1 -- 172.21.15.3:0/3365137067 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601409a500 con 0x7f601409e410 2023-10-03T10:14:39.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.233+0000 7f601259c700 1 --2- 172.21.15.3:0/3365137067 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f601409d6f0 0x7f601409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:39.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.233+0000 7f6012d9d700 1 --2- 172.21.15.3:0/3365137067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f601409e410 0x7f6014134100 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f600c009e50 tx=0x7f600c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195bf54d631bec91 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:39.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.233+0000 7f601159a700 1 -- 172.21.15.3:0/3365137067 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c00e070 con 0x7f601409e410 2023-10-03T10:14:39.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.233+0000 7f601159a700 1 -- 172.21.15.3:0/3365137067 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600c00b910 con 0x7f601409e410 2023-10-03T10:14:39.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.233+0000 7f601159a700 1 -- 172.21.15.3:0/3365137067 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c00bcc0 con 0x7f601409e410 2023-10-03T10:14:39.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.234+0000 7f6018f90700 1 -- 172.21.15.3:0/3365137067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f601409e410 msgr2=0x7f6014134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:39.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.234+0000 7f6018f90700 1 --2- 172.21.15.3:0/3365137067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f601409e410 0x7f6014134100 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f600c009e50 tx=0x7f600c00b330 comp rx=0 tx=0).stop 2023-10-03T10:14:39.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.234+0000 7f6018f90700 1 -- 172.21.15.3:0/3365137067 shutdown_connections 2023-10-03T10:14:39.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.234+0000 7f6018f90700 1 --2- 172.21.15.3:0/3365137067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f601409e410 0x7f6014134100 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:39.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.234+0000 7f6018f90700 1 --2- 172.21.15.3:0/3365137067 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f601409d6f0 0x7f601409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:39.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.234+0000 7f6018f90700 1 -- 172.21.15.3:0/3365137067 >> 172.21.15.3:0/3365137067 conn(0x7f6014066f50 msgr2=0x7f601409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:39.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.234+0000 7f6018f90700 1 -- 172.21.15.3:0/3365137067 shutdown_connections 2023-10-03T10:14:39.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.234+0000 7f6018f90700 1 -- 172.21.15.3:0/3365137067 wait complete. 2023-10-03T10:14:39.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.235+0000 7f6018f90700 1 Processor -- start 2023-10-03T10:14:39.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.235+0000 7f6018f90700 1 -- start start 2023-10-03T10:14:39.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.236+0000 7f6018f90700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f601409d6f0 0x7f60141a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:39.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.236+0000 7f6018f90700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f601409e000 0x7f60141a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:39.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.236+0000 7f6018f90700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f601409e410 0x7f60141a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:39.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.236+0000 7f6018f90700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60141a9870 con 0x7f601409e000 2023-10-03T10:14:39.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.236+0000 7f6018f90700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60141a7b90 con 0x7f601409e410 2023-10-03T10:14:39.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.236+0000 7f6018f90700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f60141a9a70 con 0x7f601409d6f0 2023-10-03T10:14:39.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.236+0000 7f601259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f601409d6f0 0x7f60141a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:39.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.236+0000 7f6011d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f601409e000 0x7f60141a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:39.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.236+0000 7f601259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f601409d6f0 0x7f60141a60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:32924/0 (socket says 172.21.15.3:32924) 2023-10-03T10:14:39.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.236+0000 7f6012d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f601409e410 0x7f60141a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:39.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.236+0000 7f601259c700 1 -- 172.21.15.3:0/3594286881 learned_addr learned my addr 172.21.15.3:0/3594286881 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:39.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.236+0000 7f601259c700 1 -- 172.21.15.3:0/3594286881 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f601409e410 msgr2=0x7f60141a6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:39.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.236+0000 7f601259c700 1 --2- 172.21.15.3:0/3594286881 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f601409e410 0x7f60141a6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:39.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.236+0000 7f601259c700 1 -- 172.21.15.3:0/3594286881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f601409e000 msgr2=0x7f60141a65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:39.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.236+0000 7f601259c700 1 --2- 172.21.15.3:0/3594286881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f601409e000 0x7f60141a65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:39.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.236+0000 7f601259c700 1 -- 172.21.15.3:0/3594286881 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60040094a0 con 0x7f601409d6f0 2023-10-03T10:14:39.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.236+0000 7f6011d9b700 1 --2- 172.21.15.3:0/3594286881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f601409e000 0x7f60141a65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:39.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.236+0000 7f601259c700 1 --2- 172.21.15.3:0/3594286881 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f601409d6f0 0x7f60141a60a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f600400ec10 tx=0x7f600400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:39.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.237+0000 7f6002ffd700 1 -- 172.21.15.3:0/3594286881 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600400c850 con 0x7f601409d6f0 2023-10-03T10:14:39.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.237+0000 7f6002ffd700 1 -- 172.21.15.3:0/3594286881 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600400c9b0 con 0x7f601409d6f0 2023-10-03T10:14:39.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.237+0000 7f6018f90700 1 -- 172.21.15.3:0/3594286881 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600c009b30 con 0x7f601409d6f0 2023-10-03T10:14:39.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.237+0000 7f6002ffd700 1 -- 172.21.15.3:0/3594286881 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600400cd20 con 0x7f601409d6f0 2023-10-03T10:14:39.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.237+0000 7f6018f90700 1 -- 172.21.15.3:0/3594286881 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60141a88f0 con 0x7f601409d6f0 2023-10-03T10:14:39.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.238+0000 7f6002ffd700 1 -- 172.21.15.3:0/3594286881 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6004013470 con 0x7f601409d6f0 2023-10-03T10:14:39.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.238+0000 7f6002ffd700 1 -- 172.21.15.3:0/3594286881 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f600403f830 con 0x7f601409d6f0 2023-10-03T10:14:39.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.238+0000 7f6018f90700 1 -- 172.21.15.3:0/3594286881 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6014074ea0 con 0x7f601409d6f0 2023-10-03T10:14:39.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.243+0000 7f6002ffd700 1 -- 172.21.15.3:0/3594286881 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6004013720 con 0x7f601409d6f0 2023-10-03T10:14:39.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.449+0000 7f6018f90700 1 -- 172.21.15.3:0/3594286881 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f60141aa3a0 con 0x7f601409d6f0 2023-10-03T10:14:39.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.449+0000 7f6002ffd700 1 -- 172.21.15.3:0/3594286881 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f600401bb60 con 0x7f601409d6f0 2023-10-03T10:14:39.450 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:39.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.452+0000 7f6018f90700 1 -- 172.21.15.3:0/3594286881 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f601409d6f0 msgr2=0x7f60141a60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:39.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.452+0000 7f6018f90700 1 --2- 172.21.15.3:0/3594286881 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f601409d6f0 0x7f60141a60a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f600400ec10 tx=0x7f600400ef20 comp rx=0 tx=0).stop 2023-10-03T10:14:39.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.452+0000 7f6018f90700 1 -- 172.21.15.3:0/3594286881 shutdown_connections 2023-10-03T10:14:39.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.452+0000 7f6018f90700 1 --2- 172.21.15.3:0/3594286881 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f601409e410 0x7f60141a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:39.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.452+0000 7f6018f90700 1 --2- 172.21.15.3:0/3594286881 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f601409d6f0 0x7f60141a60a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:39.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.452+0000 7f6018f90700 1 --2- 172.21.15.3:0/3594286881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f601409e000 0x7f60141a65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:39.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.452+0000 7f6018f90700 1 -- 172.21.15.3:0/3594286881 >> 172.21.15.3:0/3594286881 conn(0x7f6014066f50 msgr2=0x7f60141baef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:39.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.452+0000 7f6018f90700 1 -- 172.21.15.3:0/3594286881 shutdown_connections 2023-10-03T10:14:39.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:39.452+0000 7f6018f90700 1 -- 172.21.15.3:0/3594286881 wait complete. 2023-10-03T10:14:39.466 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:40.156 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:40.190 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:40.202 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:40.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.309+0000 7f44bf6ba700 1 Processor -- start 2023-10-03T10:14:40.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.309+0000 7f44bf6ba700 1 -- start start 2023-10-03T10:14:40.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.310+0000 7f44bf6ba700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44b809e930 0x7f44b809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:40.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.310+0000 7f44bf6ba700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44b809f650 0x7f44b8131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:40.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.310+0000 7f44bf6ba700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44b8070940 con 0x7f44b809f240 2023-10-03T10:14:40.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.310+0000 7f44bf6ba700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44b806fd60 con 0x7f44b809e930 2023-10-03T10:14:40.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.310+0000 7f44bf6ba700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b806fa80 con 0x7f44b809f650 2023-10-03T10:14:40.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.310+0000 7f44bd456700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44b809e930 0x7f44b809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:40.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.310+0000 7f44bd456700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44b809e930 0x7f44b809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50120/0 (socket says 172.21.15.3:50120) 2023-10-03T10:14:40.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.310+0000 7f44bd456700 1 -- 172.21.15.3:0/3826907050 learned_addr learned my addr 172.21.15.3:0/3826907050 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:40.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.311+0000 7f44bdc57700 1 --2- 172.21.15.3:0/3826907050 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44b809f650 0x7f44b8131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:40.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.311+0000 7f44bd456700 1 -- 172.21.15.3:0/3826907050 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44b809f650 msgr2=0x7f44b8131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:40.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.311+0000 7f44bd456700 1 --2- 172.21.15.3:0/3826907050 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44b809f650 0x7f44b8131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.311+0000 7f44bd456700 1 -- 172.21.15.3:0/3826907050 >> v1:172.21.15.3:6789/0 conn(0x7f44b809f240 legacy=0x7f44b81317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:40.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.311+0000 7f44bd456700 1 -- 172.21.15.3:0/3826907050 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b809fb60 con 0x7f44b809e930 2023-10-03T10:14:40.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.311+0000 7f44bdc57700 1 --2- 172.21.15.3:0/3826907050 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44b809f650 0x7f44b8131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:14:40.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.311+0000 7f44bd456700 1 --2- 172.21.15.3:0/3826907050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44b809e930 0x7f44b809ed00 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f44b40098f0 tx=0x7f44b4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4e428364abe798 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:40.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.311+0000 7f44affff700 1 -- 172.21.15.3:0/3826907050 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4010a90 con 0x7f44b809e930 2023-10-03T10:14:40.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.311+0000 7f44affff700 1 -- 172.21.15.3:0/3826907050 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b4010bf0 con 0x7f44b809e930 2023-10-03T10:14:40.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.311+0000 7f44affff700 1 -- 172.21.15.3:0/3826907050 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4018440 con 0x7f44b809e930 2023-10-03T10:14:40.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.312+0000 7f44bf6ba700 1 -- 172.21.15.3:0/3826907050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44b809e930 msgr2=0x7f44b809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:40.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.312+0000 7f44bf6ba700 1 --2- 172.21.15.3:0/3826907050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44b809e930 0x7f44b809ed00 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f44b40098f0 tx=0x7f44b4009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:40.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.312+0000 7f44bf6ba700 1 -- 172.21.15.3:0/3826907050 shutdown_connections 2023-10-03T10:14:40.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.312+0000 7f44bf6ba700 1 --2- 172.21.15.3:0/3826907050 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44b809f650 0x7f44b8131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.312+0000 7f44bf6ba700 1 --2- 172.21.15.3:0/3826907050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44b809e930 0x7f44b809ed00 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.312+0000 7f44bf6ba700 1 -- 172.21.15.3:0/3826907050 >> 172.21.15.3:0/3826907050 conn(0x7f44b80a4210 msgr2=0x7f44b80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:40.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.312+0000 7f44bf6ba700 1 -- 172.21.15.3:0/3826907050 shutdown_connections 2023-10-03T10:14:40.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.312+0000 7f44bf6ba700 1 -- 172.21.15.3:0/3826907050 wait complete. 2023-10-03T10:14:40.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.314+0000 7f44bf6ba700 1 Processor -- start 2023-10-03T10:14:40.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.314+0000 7f44bf6ba700 1 -- start start 2023-10-03T10:14:40.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.314+0000 7f44bf6ba700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44b809e930 0x7f44b81a1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:40.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.314+0000 7f44bf6ba700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44b809f240 0x7f44b81a20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:40.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.314+0000 7f44bf6ba700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44b809f650 0x7f44b81a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:40.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.314+0000 7f44bf6ba700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b81a3060 con 0x7f44b809e930 2023-10-03T10:14:40.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.314+0000 7f44bf6ba700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b81a5fd0 con 0x7f44b809f650 2023-10-03T10:14:40.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.314+0000 7f44bf6ba700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f44b81a3700 con 0x7f44b809f240 2023-10-03T10:14:40.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.315+0000 7f44bd456700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44b809e930 0x7f44b81a1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:40.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.315+0000 7f44bd456700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44b809e930 0x7f44b81a1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50136/0 (socket says 172.21.15.3:50136) 2023-10-03T10:14:40.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.315+0000 7f44bd456700 1 -- 172.21.15.3:0/411488577 learned_addr learned my addr 172.21.15.3:0/411488577 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:40.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.315+0000 7f44bcc55700 1 --2- 172.21.15.3:0/411488577 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44b809f240 0x7f44b81a20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:40.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.315+0000 7f44bdc57700 1 --2- 172.21.15.3:0/411488577 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44b809f650 0x7f44b81a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:40.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.315+0000 7f44bd456700 1 -- 172.21.15.3:0/411488577 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44b809f240 msgr2=0x7f44b81a20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:40.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.315+0000 7f44bd456700 1 --2- 172.21.15.3:0/411488577 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44b809f240 0x7f44b81a20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.315+0000 7f44bd456700 1 -- 172.21.15.3:0/411488577 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44b809f650 msgr2=0x7f44b81a2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:40.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.315+0000 7f44bd456700 1 --2- 172.21.15.3:0/411488577 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44b809f650 0x7f44b81a2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.315+0000 7f44bd456700 1 -- 172.21.15.3:0/411488577 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b4009550 con 0x7f44b809e930 2023-10-03T10:14:40.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.315+0000 7f44bdc57700 1 --2- 172.21.15.3:0/411488577 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44b809f650 0x7f44b81a2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:14:40.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.315+0000 7f44bd456700 1 --2- 172.21.15.3:0/411488577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44b809e930 0x7f44b81a1b90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f44b80a0b00 tx=0x7f44b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:40.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.315+0000 7f44adffb700 1 -- 172.21.15.3:0/411488577 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4003ab0 con 0x7f44b809e930 2023-10-03T10:14:40.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.315+0000 7f44adffb700 1 -- 172.21.15.3:0/411488577 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b4003c10 con 0x7f44b809e930 2023-10-03T10:14:40.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.315+0000 7f44bf6ba700 1 -- 172.21.15.3:0/411488577 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b80a0190 con 0x7f44b809e930 2023-10-03T10:14:40.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.315+0000 7f44adffb700 1 -- 172.21.15.3:0/411488577 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4023470 con 0x7f44b809e930 2023-10-03T10:14:40.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.316+0000 7f44bf6ba700 1 -- 172.21.15.3:0/411488577 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b80a33c0 con 0x7f44b809e930 2023-10-03T10:14:40.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.317+0000 7f44bf6ba700 1 -- 172.21.15.3:0/411488577 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44b8071d20 con 0x7f44b809e930 2023-10-03T10:14:40.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.317+0000 7f44adffb700 1 -- 172.21.15.3:0/411488577 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f44b401f070 con 0x7f44b809e930 2023-10-03T10:14:40.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.318+0000 7f44adffb700 1 -- 172.21.15.3:0/411488577 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f44b4045070 con 0x7f44b809e930 2023-10-03T10:14:40.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.322+0000 7f44adffb700 1 -- 172.21.15.3:0/411488577 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f44b402c7d0 con 0x7f44b809e930 2023-10-03T10:14:40.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.347+0000 7f3034bb7700 1 Processor -- start 2023-10-03T10:14:40.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.347+0000 7f3034bb7700 1 -- start start 2023-10-03T10:14:40.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.347+0000 7f3034bb7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f30300a0a30 0x7f30300a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:40.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.347+0000 7f3034bb7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f30300a1340 0x7f3030133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:40.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.347+0000 7f3034bb7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f303009edd0 con 0x7f3030133fa0 2023-10-03T10:14:40.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.347+0000 7f3034bb7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f303009e1f0 con 0x7f30300a0a30 2023-10-03T10:14:40.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.347+0000 7f3034bb7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f303009dc50 con 0x7f30300a1340 2023-10-03T10:14:40.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.348+0000 7f302e59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f30300a0a30 0x7f30300a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:40.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.348+0000 7f302e59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f30300a0a30 0x7f30300a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50146/0 (socket says 172.21.15.3:50146) 2023-10-03T10:14:40.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.348+0000 7f302e59c700 1 -- 172.21.15.3:0/2892577063 learned_addr learned my addr 172.21.15.3:0/2892577063 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:40.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.348+0000 7f302dd9b700 1 --2- 172.21.15.3:0/2892577063 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f30300a1340 0x7f3030133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:40.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.348+0000 7f302d59a700 1 -- 172.21.15.3:0/2892577063 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2199899845 0 0) 0x7f303009edd0 con 0x7f3030133fa0 2023-10-03T10:14:40.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.348+0000 7f302e59c700 1 -- 172.21.15.3:0/2892577063 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f30300a1340 msgr2=0x7f3030133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:40.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.348+0000 7f302e59c700 1 --2- 172.21.15.3:0/2892577063 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f30300a1340 0x7f3030133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.348+0000 7f302e59c700 1 -- 172.21.15.3:0/2892577063 >> v1:172.21.15.3:6789/0 conn(0x7f3030133fa0 legacy=0x7f303013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:40.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.348+0000 7f302e59c700 1 -- 172.21.15.3:0/2892577063 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30300a1fe0 con 0x7f30300a0a30 2023-10-03T10:14:40.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.348+0000 7f302e59c700 1 --2- 172.21.15.3:0/2892577063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30300a0a30 0x7f30300a0e00 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f30200098f0 tx=0x7f3020009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff12126bfae10e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:40.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.349+0000 7f302d59a700 1 -- 172.21.15.3:0/2892577063 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3020011830 con 0x7f30300a0a30 2023-10-03T10:14:40.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.349+0000 7f302d59a700 1 -- 172.21.15.3:0/2892577063 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3020011990 con 0x7f30300a0a30 2023-10-03T10:14:40.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.349+0000 7f302d59a700 1 -- 172.21.15.3:0/2892577063 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3020011d20 con 0x7f30300a0a30 2023-10-03T10:14:40.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.349+0000 7f3034bb7700 1 -- 172.21.15.3:0/2892577063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30300a0a30 msgr2=0x7f30300a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:40.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.349+0000 7f3034bb7700 1 --2- 172.21.15.3:0/2892577063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30300a0a30 0x7f30300a0e00 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f30200098f0 tx=0x7f3020009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:40.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.349+0000 7f3034bb7700 1 -- 172.21.15.3:0/2892577063 shutdown_connections 2023-10-03T10:14:40.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.349+0000 7f3034bb7700 1 --2- 172.21.15.3:0/2892577063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30300a0a30 0x7f30300a0e00 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.349+0000 7f3034bb7700 1 --2- 172.21.15.3:0/2892577063 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f30300a1340 0x7f3030133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.349+0000 7f3034bb7700 1 -- 172.21.15.3:0/2892577063 >> 172.21.15.3:0/2892577063 conn(0x7f30300a6550 msgr2=0x7f30300a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:40.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.350+0000 7f3034bb7700 1 -- 172.21.15.3:0/2892577063 shutdown_connections 2023-10-03T10:14:40.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.350+0000 7f3034bb7700 1 -- 172.21.15.3:0/2892577063 wait complete. 2023-10-03T10:14:40.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.351+0000 7f3034bb7700 1 Processor -- start 2023-10-03T10:14:40.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.351+0000 7f3034bb7700 1 -- start start 2023-10-03T10:14:40.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.351+0000 7f3034bb7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30300a0a30 0x7f3030122840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:40.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.351+0000 7f3034bb7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f30300a1340 0x7f3030122d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:40.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.351+0000 7f3034bb7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3030133fa0 0x7f30301232c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:40.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.351+0000 7f3034bb7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3030133360 con 0x7f30300a0a30 2023-10-03T10:14:40.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.351+0000 7f3034bb7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3030123cf0 con 0x7f30300a1340 2023-10-03T10:14:40.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.351+0000 7f3034bb7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3030126080 con 0x7f3030133fa0 2023-10-03T10:14:40.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.351+0000 7f302dd9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f30300a1340 0x7f3030122d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:40.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.351+0000 7f302ed9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3030133fa0 0x7f30301232c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:40.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.351+0000 7f302dd9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f30300a1340 0x7f3030122d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51058/0 (socket says 172.21.15.3:51058) 2023-10-03T10:14:40.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.351+0000 7f302dd9b700 1 -- 172.21.15.3:0/2327020615 learned_addr learned my addr 172.21.15.3:0/2327020615 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:40.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.352+0000 7f302e59c700 1 --2- 172.21.15.3:0/2327020615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30300a0a30 0x7f3030122840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:40.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.352+0000 7f302ed9d700 1 -- 172.21.15.3:0/2327020615 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f30300a1340 msgr2=0x7f3030122d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:40.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.352+0000 7f302ed9d700 1 --2- 172.21.15.3:0/2327020615 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f30300a1340 0x7f3030122d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.352+0000 7f302ed9d700 1 -- 172.21.15.3:0/2327020615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30300a0a30 msgr2=0x7f3030122840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:40.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.352+0000 7f302ed9d700 1 --2- 172.21.15.3:0/2327020615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30300a0a30 0x7f3030122840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.352+0000 7f302ed9d700 1 -- 172.21.15.3:0/2327020615 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3020009550 con 0x7f3030133fa0 2023-10-03T10:14:40.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.352+0000 7f302ed9d700 1 --2- 172.21.15.3:0/2327020615 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3030133fa0 0x7f30301232c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f302800c820 tx=0x7f302800cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:40.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.352+0000 7f301effd700 1 -- 172.21.15.3:0/2327020615 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3028003e30 con 0x7f3030133fa0 2023-10-03T10:14:40.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.352+0000 7f301effd700 1 -- 172.21.15.3:0/2327020615 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3028005070 con 0x7f3030133fa0 2023-10-03T10:14:40.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.352+0000 7f3034bb7700 1 -- 172.21.15.3:0/2327020615 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3030124560 con 0x7f3030133fa0 2023-10-03T10:14:40.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.352+0000 7f301effd700 1 -- 172.21.15.3:0/2327020615 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3028005420 con 0x7f3030133fa0 2023-10-03T10:14:40.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.352+0000 7f3034bb7700 1 -- 172.21.15.3:0/2327020615 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3030124c40 con 0x7f3030133fa0 2023-10-03T10:14:40.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.353+0000 7f301effd700 1 -- 172.21.15.3:0/2327020615 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3028005580 con 0x7f3030133fa0 2023-10-03T10:14:40.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.353+0000 7f301effd700 1 -- 172.21.15.3:0/2327020615 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f302803ecc0 con 0x7f3030133fa0 2023-10-03T10:14:40.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.354+0000 7f3034bb7700 1 -- 172.21.15.3:0/2327020615 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30300a1d40 con 0x7f3030133fa0 2023-10-03T10:14:40.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.359+0000 7f301effd700 1 -- 172.21.15.3:0/2327020615 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3028017080 con 0x7f3030133fa0 2023-10-03T10:14:40.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.391+0000 7f7816948700 1 Processor -- start 2023-10-03T10:14:40.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.391+0000 7f7816948700 1 -- start start 2023-10-03T10:14:40.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.391+0000 7f7816948700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f781009d630 0x7f78100a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:40.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.391+0000 7f7816948700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7810138d80 0x7f781013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:40.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.391+0000 7f7816948700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78100a2a70 con 0x7f78100a0a30 2023-10-03T10:14:40.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.391+0000 7f7816948700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78100a01d0 con 0x7f781009d630 2023-10-03T10:14:40.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.391+0000 7f7816948700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f781009e2b0 con 0x7f7810138d80 2023-10-03T10:14:40.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.391+0000 7f780f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f781009d630 0x7f78100a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:40.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.392+0000 7f780f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f781009d630 0x7f78100a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51068/0 (socket says 172.21.15.3:51068) 2023-10-03T10:14:40.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.392+0000 7f780f7fe700 1 -- 172.21.15.3:0/1426479630 learned_addr learned my addr 172.21.15.3:0/1426479630 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:40.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.392+0000 7f780ffff700 1 --1- 172.21.15.3:0/1426479630 >> v1:172.21.15.3:6789/0 conn(0x7f78100a0a30 0x7f78100a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40530/0 (socket says 172.21.15.3:40530) 2023-10-03T10:14:40.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.392+0000 7f7814ee5700 1 --2- 172.21.15.3:0/1426479630 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7810138d80 0x7f781013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:40.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.392+0000 7f780f7fe700 1 -- 172.21.15.3:0/1426479630 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7810138d80 msgr2=0x7f781013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:40.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.392+0000 7f780f7fe700 1 --2- 172.21.15.3:0/1426479630 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7810138d80 0x7f781013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.392+0000 7f780f7fe700 1 -- 172.21.15.3:0/1426479630 >> v1:172.21.15.3:6789/0 conn(0x7f78100a0a30 legacy=0x7f78100a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:40.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.392+0000 7f780f7fe700 1 -- 172.21.15.3:0/1426479630 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781009dc50 con 0x7f781009d630 2023-10-03T10:14:40.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.392+0000 7f780f7fe700 1 --2- 172.21.15.3:0/1426479630 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f781009d630 0x7f78100a1510 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7804009920 tx=0x7f7804009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e534191469e83905 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:40.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.392+0000 7f780effd700 1 -- 172.21.15.3:0/1426479630 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7804011830 con 0x7f781009d630 2023-10-03T10:14:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.392+0000 7f780effd700 1 -- 172.21.15.3:0/1426479630 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7804011990 con 0x7f781009d630 2023-10-03T10:14:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.392+0000 7f780effd700 1 -- 172.21.15.3:0/1426479630 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7804011d20 con 0x7f781009d630 2023-10-03T10:14:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.393+0000 7f7816948700 1 -- 172.21.15.3:0/1426479630 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f781009d630 msgr2=0x7f78100a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.393+0000 7f7816948700 1 --2- 172.21.15.3:0/1426479630 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f781009d630 0x7f78100a1510 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7804009920 tx=0x7f7804009c30 comp rx=0 tx=0).stop 2023-10-03T10:14:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.393+0000 7f7816948700 1 -- 172.21.15.3:0/1426479630 shutdown_connections 2023-10-03T10:14:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.393+0000 7f7816948700 1 --2- 172.21.15.3:0/1426479630 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7810138d80 0x7f781013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.393+0000 7f7816948700 1 --2- 172.21.15.3:0/1426479630 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f781009d630 0x7f78100a1510 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.393+0000 7f7816948700 1 -- 172.21.15.3:0/1426479630 >> 172.21.15.3:0/1426479630 conn(0x7f78100a3050 msgr2=0x7f78100a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.393+0000 7f7816948700 1 -- 172.21.15.3:0/1426479630 shutdown_connections 2023-10-03T10:14:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.393+0000 7f7816948700 1 -- 172.21.15.3:0/1426479630 wait complete. 2023-10-03T10:14:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.395+0000 7f7816948700 1 Processor -- start 2023-10-03T10:14:40.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.395+0000 7f7816948700 1 -- start start 2023-10-03T10:14:40.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.396+0000 7f7816948700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f781009d630 0x7f781011d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:40.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.396+0000 7f7816948700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f78100a0a30 0x7f781011da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:40.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.396+0000 7f7816948700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7810138d80 0x7f781011df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:40.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.396+0000 7f7816948700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781011f080 con 0x7f7810138d80 2023-10-03T10:14:40.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.396+0000 7f7816948700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78101203d0 con 0x7f781009d630 2023-10-03T10:14:40.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.396+0000 7f7816948700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7810120f90 con 0x7f78100a0a30 2023-10-03T10:14:40.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.396+0000 7f780ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f781009d630 0x7f781011d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:40.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.396+0000 7f780ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f781009d630 0x7f781011d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51070/0 (socket says 172.21.15.3:51070) 2023-10-03T10:14:40.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.396+0000 7f780ffff700 1 -- 172.21.15.3:0/352156878 learned_addr learned my addr 172.21.15.3:0/352156878 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:40.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.396+0000 7f780f7fe700 1 --2- 172.21.15.3:0/352156878 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f78100a0a30 0x7f781011da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:40.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.396+0000 7f7814ee5700 1 --2- 172.21.15.3:0/352156878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7810138d80 0x7f781011df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:40.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.396+0000 7f780ffff700 1 -- 172.21.15.3:0/352156878 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f78100a0a30 msgr2=0x7f781011da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:40.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.396+0000 7f780ffff700 1 --2- 172.21.15.3:0/352156878 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f78100a0a30 0x7f781011da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.396+0000 7f780ffff700 1 -- 172.21.15.3:0/352156878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7810138d80 msgr2=0x7f781011df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:40.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.396+0000 7f780ffff700 1 --2- 172.21.15.3:0/352156878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7810138d80 0x7f781011df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.396+0000 7f780ffff700 1 -- 172.21.15.3:0/352156878 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7804009550 con 0x7f781009d630 2023-10-03T10:14:40.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.397+0000 7f780ffff700 1 --2- 172.21.15.3:0/352156878 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f781009d630 0x7f781011d500 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f780000c520 tx=0x7f780000c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:40.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.397+0000 7f780cff9700 1 -- 172.21.15.3:0/352156878 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7800005cb0 con 0x7f781009d630 2023-10-03T10:14:40.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.397+0000 7f780cff9700 1 -- 172.21.15.3:0/352156878 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7800005e10 con 0x7f781009d630 2023-10-03T10:14:40.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.397+0000 7f7816948700 1 -- 172.21.15.3:0/352156878 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f781011f8c0 con 0x7f781009d630 2023-10-03T10:14:40.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.397+0000 7f780cff9700 1 -- 172.21.15.3:0/352156878 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78000072d0 con 0x7f781009d630 2023-10-03T10:14:40.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.397+0000 7f7816948700 1 -- 172.21.15.3:0/352156878 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7810120cb0 con 0x7f781009d630 2023-10-03T10:14:40.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.398+0000 7f7816948700 1 -- 172.21.15.3:0/352156878 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7810120b10 con 0x7f781009d630 2023-10-03T10:14:40.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.399+0000 7f780cff9700 1 -- 172.21.15.3:0/352156878 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7800007430 con 0x7f781009d630 2023-10-03T10:14:40.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.399+0000 7f780cff9700 1 -- 172.21.15.3:0/352156878 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f780003ee80 con 0x7f781009d630 2023-10-03T10:14:40.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.403+0000 7f780cff9700 1 -- 172.21.15.3:0/352156878 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f78000076e0 con 0x7f781009d630 2023-10-03T10:14:40.467 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:40.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.551+0000 7f44bf6ba700 1 -- 172.21.15.3:0/411488577 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f44b8074ea0 con 0x7f44b809e930 2023-10-03T10:14:40.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.551+0000 7f44adffb700 1 -- 172.21.15.3:0/411488577 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f44b401d030 con 0x7f44b809e930 2023-10-03T10:14:40.551 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:40.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.553+0000 7f44bf6ba700 1 -- 172.21.15.3:0/411488577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44b809e930 msgr2=0x7f44b81a1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:40.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.553+0000 7f44bf6ba700 1 --2- 172.21.15.3:0/411488577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44b809e930 0x7f44b81a1b90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f44b80a0b00 tx=0x7f44b4004750 comp rx=0 tx=0).stop 2023-10-03T10:14:40.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.553+0000 7f44bf6ba700 1 -- 172.21.15.3:0/411488577 shutdown_connections 2023-10-03T10:14:40.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.553+0000 7f44bf6ba700 1 --2- 172.21.15.3:0/411488577 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44b809f650 0x7f44b81a2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.553+0000 7f44bf6ba700 1 --2- 172.21.15.3:0/411488577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44b809e930 0x7f44b81a1b90 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.553+0000 7f44bf6ba700 1 --2- 172.21.15.3:0/411488577 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44b809f240 0x7f44b81a20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.553+0000 7f44bf6ba700 1 -- 172.21.15.3:0/411488577 >> 172.21.15.3:0/411488577 conn(0x7f44b80a4210 msgr2=0x7f44b81b6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:40.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.554+0000 7f44bf6ba700 1 -- 172.21.15.3:0/411488577 shutdown_connections 2023-10-03T10:14:40.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.554+0000 7f44bf6ba700 1 -- 172.21.15.3:0/411488577 wait complete. 2023-10-03T10:14:40.568 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:40.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.580+0000 7f3034bb7700 1 -- 172.21.15.3:0/2327020615 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3030074ea0 con 0x7f3030133fa0 2023-10-03T10:14:40.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.580+0000 7f301effd700 1 -- 172.21.15.3:0/2327020615 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3028005830 con 0x7f3030133fa0 2023-10-03T10:14:40.581 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:40.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.582+0000 7f3034bb7700 1 -- 172.21.15.3:0/2327020615 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3030133fa0 msgr2=0x7f30301232c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:40.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.582+0000 7f3034bb7700 1 --2- 172.21.15.3:0/2327020615 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3030133fa0 0x7f30301232c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f302800c820 tx=0x7f302800cb30 comp rx=0 tx=0).stop 2023-10-03T10:14:40.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.582+0000 7f3034bb7700 1 -- 172.21.15.3:0/2327020615 shutdown_connections 2023-10-03T10:14:40.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.582+0000 7f3034bb7700 1 --2- 172.21.15.3:0/2327020615 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3030133fa0 0x7f30301232c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.582+0000 7f3034bb7700 1 --2- 172.21.15.3:0/2327020615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30300a0a30 0x7f3030122840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.582+0000 7f3034bb7700 1 --2- 172.21.15.3:0/2327020615 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f30300a1340 0x7f3030122d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.583+0000 7f3034bb7700 1 -- 172.21.15.3:0/2327020615 >> 172.21.15.3:0/2327020615 conn(0x7f30300a6550 msgr2=0x7f303011c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:40.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.583+0000 7f3034bb7700 1 -- 172.21.15.3:0/2327020615 shutdown_connections 2023-10-03T10:14:40.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.583+0000 7f3034bb7700 1 -- 172.21.15.3:0/2327020615 wait complete. 2023-10-03T10:14:40.597 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:40.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.607+0000 7fee4c1fc700 1 Processor -- start 2023-10-03T10:14:40.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.607+0000 7fee4c1fc700 1 -- start start 2023-10-03T10:14:40.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.607+0000 7fee4c1fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fee4406a770 0x7fee4406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:40.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.607+0000 7fee4c1fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee4406b080 0x7fee44131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:40.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.607+0000 7fee4c1fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee44131b40 0x7fee44131fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:40.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.607+0000 7fee4c1fc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee4409ece0 con 0x7fee4406a770 2023-10-03T10:14:40.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.607+0000 7fee4c1fc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4409e760 con 0x7fee4406b080 2023-10-03T10:14:40.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.607+0000 7fee4c1fc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee4409e040 con 0x7fee44131b40 2023-10-03T10:14:40.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.608+0000 7fee49f98700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fee4406a770 0x7fee4406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:40.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.608+0000 7fee4a799700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee44131b40 0x7fee44131fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:40.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.608+0000 7fee49797700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee4406b080 0x7fee44131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:40.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.608+0000 7fee49f98700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fee4406a770 0x7fee4406ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50184/0 (socket says 172.21.15.3:50184) 2023-10-03T10:14:40.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.608+0000 7fee49f98700 1 -- 172.21.15.3:0/3325800143 learned_addr learned my addr 172.21.15.3:0/3325800143 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:40.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.608+0000 7fee4a799700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee44131b40 0x7fee44131fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:32954/0 (socket says 172.21.15.3:32954) 2023-10-03T10:14:40.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.608+0000 7fee49797700 1 -- 172.21.15.3:0/3325800143 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee44131b40 msgr2=0x7fee44131fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:40.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.608+0000 7fee49797700 1 --2- 172.21.15.3:0/3325800143 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee44131b40 0x7fee44131fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.608+0000 7fee49797700 1 -- 172.21.15.3:0/3325800143 >> v2:172.21.15.3:3300/0 conn(0x7fee4406a770 msgr2=0x7fee4406ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:40.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.608+0000 7fee49797700 1 --2- 172.21.15.3:0/3325800143 >> v2:172.21.15.3:3300/0 conn(0x7fee4406a770 0x7fee4406ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.608+0000 7fee49797700 1 -- 172.21.15.3:0/3325800143 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4409f630 con 0x7fee4406b080 2023-10-03T10:14:40.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.608+0000 7fee4a799700 1 --2- 172.21.15.3:0/3325800143 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee44131b40 0x7fee44131fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:40.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.608+0000 7fee49f98700 1 --2- 172.21.15.3:0/3325800143 >> v2:172.21.15.3:3300/0 conn(0x7fee4406a770 0x7fee4406ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:40.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.608+0000 7fee49797700 1 --2- 172.21.15.3:0/3325800143 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee4406b080 0x7fee44131600 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fee34009840 tx=0x7fee34009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8562b68491f0e42a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:40.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.608+0000 7fee48f96700 1 -- 172.21.15.3:0/3325800143 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee34011830 con 0x7fee4406b080 2023-10-03T10:14:40.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.608+0000 7fee48f96700 1 -- 172.21.15.3:0/3325800143 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee34011990 con 0x7fee4406b080 2023-10-03T10:14:40.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.609+0000 7fee48f96700 1 -- 172.21.15.3:0/3325800143 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee34011d20 con 0x7fee4406b080 2023-10-03T10:14:40.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.609+0000 7fee4c1fc700 1 -- 172.21.15.3:0/3325800143 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee4406b080 msgr2=0x7fee44131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:40.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.609+0000 7fee4c1fc700 1 --2- 172.21.15.3:0/3325800143 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee4406b080 0x7fee44131600 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fee34009840 tx=0x7fee34009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:40.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.609+0000 7fee4c1fc700 1 -- 172.21.15.3:0/3325800143 shutdown_connections 2023-10-03T10:14:40.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.609+0000 7fee4c1fc700 1 --2- 172.21.15.3:0/3325800143 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee44131b40 0x7fee44131fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.609+0000 7fee4c1fc700 1 --2- 172.21.15.3:0/3325800143 >> v2:172.21.15.3:3300/0 conn(0x7fee4406a770 0x7fee4406ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.609+0000 7fee4c1fc700 1 --2- 172.21.15.3:0/3325800143 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee4406b080 0x7fee44131600 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.609+0000 7fee4c1fc700 1 -- 172.21.15.3:0/3325800143 >> 172.21.15.3:0/3325800143 conn(0x7fee44066f50 msgr2=0x7fee4409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:40.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.609+0000 7fee4c1fc700 1 -- 172.21.15.3:0/3325800143 shutdown_connections 2023-10-03T10:14:40.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.609+0000 7fee4c1fc700 1 -- 172.21.15.3:0/3325800143 wait complete. 2023-10-03T10:14:40.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.611+0000 7fee4c1fc700 1 Processor -- start 2023-10-03T10:14:40.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.611+0000 7fee4c1fc700 1 -- start start 2023-10-03T10:14:40.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.612+0000 7fee4c1fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee4406a770 0x7fee441212a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:40.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.612+0000 7fee4c1fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee4406b080 0x7fee441217e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:40.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.612+0000 7fee4c1fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee44131b40 0x7fee441c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:40.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.612+0000 7fee4c1fc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee44124210 con 0x7fee4406a770 2023-10-03T10:14:40.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.612+0000 7fee4c1fc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee44120ea0 con 0x7fee44131b40 2023-10-03T10:14:40.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.612+0000 7fee4c1fc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee441235f0 con 0x7fee4406b080 2023-10-03T10:14:40.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.612+0000 7fee4a799700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee44131b40 0x7fee441c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:40.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.612+0000 7fee49797700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee4406b080 0x7fee441217e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:40.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.612+0000 7fee4a799700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee44131b40 0x7fee441c6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51080/0 (socket says 172.21.15.3:51080) 2023-10-03T10:14:40.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.612+0000 7fee4a799700 1 -- 172.21.15.3:0/2831597476 learned_addr learned my addr 172.21.15.3:0/2831597476 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:40.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.612+0000 7fee49797700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee4406b080 0x7fee441217e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:32958/0 (socket says 172.21.15.3:32958) 2023-10-03T10:14:40.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.612+0000 7fee49f98700 1 --2- 172.21.15.3:0/2831597476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee4406a770 0x7fee441212a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:40.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.612+0000 7fee4a799700 1 -- 172.21.15.3:0/2831597476 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee4406b080 msgr2=0x7fee441217e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:40.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.612+0000 7fee4a799700 1 --2- 172.21.15.3:0/2831597476 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee4406b080 0x7fee441217e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.612+0000 7fee4a799700 1 -- 172.21.15.3:0/2831597476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee4406a770 msgr2=0x7fee441212a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:40.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.612+0000 7fee4a799700 1 --2- 172.21.15.3:0/2831597476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee4406a770 0x7fee441212a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.612+0000 7fee4a799700 1 -- 172.21.15.3:0/2831597476 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee40009550 con 0x7fee44131b40 2023-10-03T10:14:40.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.612+0000 7fee49797700 1 --2- 172.21.15.3:0/2831597476 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee4406b080 0x7fee441217e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:40.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.612+0000 7fee49f98700 1 --2- 172.21.15.3:0/2831597476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee4406a770 0x7fee441212a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:40.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.612+0000 7fee4a799700 1 --2- 172.21.15.3:0/2831597476 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee44131b40 0x7fee441c6190 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00efc0 tx=0x7fee3c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:40.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.612+0000 7fee3a7fc700 1 -- 172.21.15.3:0/2831597476 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c00ca90 con 0x7fee44131b40 2023-10-03T10:14:40.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.613+0000 7fee3a7fc700 1 -- 172.21.15.3:0/2831597476 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3c00f040 con 0x7fee44131b40 2023-10-03T10:14:40.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.613+0000 7f7816948700 1 -- 172.21.15.3:0/352156878 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7810120b10 con 0x7f781009d630 2023-10-03T10:14:40.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.613+0000 7fee4c1fc700 1 -- 172.21.15.3:0/2831597476 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee3c009a70 con 0x7fee44131b40 2023-10-03T10:14:40.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.613+0000 7fee3a7fc700 1 -- 172.21.15.3:0/2831597476 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c00ce10 con 0x7fee44131b40 2023-10-03T10:14:40.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.613+0000 7fee4c1fc700 1 -- 172.21.15.3:0/2831597476 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee340094a0 con 0x7fee44131b40 2023-10-03T10:14:40.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.613+0000 7f780cff9700 1 -- 172.21.15.3:0/352156878 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7810120b10 con 0x7f781009d630 2023-10-03T10:14:40.617 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:40.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.614+0000 7fee3a7fc700 1 -- 172.21.15.3:0/2831597476 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fee3c014480 con 0x7fee44131b40 2023-10-03T10:14:40.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.614+0000 7fee4c1fc700 1 -- 172.21.15.3:0/2831597476 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee44074ea0 con 0x7fee44131b40 2023-10-03T10:14:40.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.614+0000 7fee3a7fc700 1 -- 172.21.15.3:0/2831597476 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fee3c03fc20 con 0x7fee44131b40 2023-10-03T10:14:40.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.615+0000 7f7816948700 1 -- 172.21.15.3:0/352156878 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f781009d630 msgr2=0x7f781011d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:40.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.615+0000 7f7816948700 1 --2- 172.21.15.3:0/352156878 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f781009d630 0x7f781011d500 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f780000c520 tx=0x7f780000c830 comp rx=0 tx=0).stop 2023-10-03T10:14:40.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.616+0000 7f7816948700 1 -- 172.21.15.3:0/352156878 shutdown_connections 2023-10-03T10:14:40.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.616+0000 7f7816948700 1 --2- 172.21.15.3:0/352156878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7810138d80 0x7f781011df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.616+0000 7f7816948700 1 --2- 172.21.15.3:0/352156878 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f781009d630 0x7f781011d500 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.616+0000 7f7816948700 1 --2- 172.21.15.3:0/352156878 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f78100a0a30 0x7f781011da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.616+0000 7f7816948700 1 -- 172.21.15.3:0/352156878 >> 172.21.15.3:0/352156878 conn(0x7f78100a3050 msgr2=0x7f78100a5310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:40.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.616+0000 7f7816948700 1 -- 172.21.15.3:0/352156878 shutdown_connections 2023-10-03T10:14:40.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.616+0000 7f7816948700 1 -- 172.21.15.3:0/352156878 wait complete. 2023-10-03T10:14:40.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.619+0000 7fee3a7fc700 1 -- 172.21.15.3:0/2831597476 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fee3c012930 con 0x7fee44131b40 2023-10-03T10:14:40.630 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:40.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.824+0000 7fee4c1fc700 1 -- 172.21.15.3:0/2831597476 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fee4406e8e0 con 0x7fee44131b40 2023-10-03T10:14:40.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.825+0000 7fee3a7fc700 1 -- 172.21.15.3:0/2831597476 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fee3c009d70 con 0x7fee44131b40 2023-10-03T10:14:40.825 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:40.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.827+0000 7fee4c1fc700 1 -- 172.21.15.3:0/2831597476 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee44131b40 msgr2=0x7fee441c6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:40.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.827+0000 7fee4c1fc700 1 --2- 172.21.15.3:0/2831597476 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee44131b40 0x7fee441c6190 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00efc0 tx=0x7fee3c00c3a0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.827+0000 7fee4c1fc700 1 -- 172.21.15.3:0/2831597476 shutdown_connections 2023-10-03T10:14:40.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.827+0000 7fee4c1fc700 1 --2- 172.21.15.3:0/2831597476 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee44131b40 0x7fee441c6190 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.827+0000 7fee4c1fc700 1 --2- 172.21.15.3:0/2831597476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee4406a770 0x7fee441212a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.827+0000 7fee4c1fc700 1 --2- 172.21.15.3:0/2831597476 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee4406b080 0x7fee441217e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:40.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.827+0000 7fee4c1fc700 1 -- 172.21.15.3:0/2831597476 >> 172.21.15.3:0/2831597476 conn(0x7fee44066f50 msgr2=0x7fee440700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:40.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.827+0000 7fee4c1fc700 1 -- 172.21.15.3:0/2831597476 shutdown_connections 2023-10-03T10:14:40.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:40.827+0000 7fee4c1fc700 1 -- 172.21.15.3:0/2831597476 wait complete. 2023-10-03T10:14:40.840 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:41.569 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:41.597 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:41.630 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:41.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.719+0000 7f59e7474700 1 Processor -- start 2023-10-03T10:14:41.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.719+0000 7f59e7474700 1 -- start start 2023-10-03T10:14:41.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.720+0000 7f59e7474700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59e009d6f0 0x7f59e009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:41.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.720+0000 7f59e7474700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f59e009e410 0x7f59e0134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:41.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.720+0000 7f59e7474700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59e009b120 con 0x7f59e009e000 2023-10-03T10:14:41.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.720+0000 7f59e7474700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e006b460 con 0x7f59e009e410 2023-10-03T10:14:41.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.720+0000 7f59e7474700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e009a7e0 con 0x7f59e009d6f0 2023-10-03T10:14:41.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.720+0000 7f59e5210700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59e009d6f0 0x7f59e009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:41.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.720+0000 7f59e5210700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59e009d6f0 0x7f59e009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51092/0 (socket says 172.21.15.3:51092) 2023-10-03T10:14:41.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.720+0000 7f59e5210700 1 -- 172.21.15.3:0/3916308323 learned_addr learned my addr 172.21.15.3:0/3916308323 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:41.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.720+0000 7f59e5a11700 1 --2- 172.21.15.3:0/3916308323 >> v2:172.21.15.3:3300/0 conn(0x7f59e009e410 0x7f59e0134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:41.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.720+0000 7f59e5a11700 1 -- 172.21.15.3:0/3916308323 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59e009d6f0 msgr2=0x7f59e009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:41.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.720+0000 7f59e5a11700 1 --2- 172.21.15.3:0/3916308323 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59e009d6f0 0x7f59e009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:41.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.720+0000 7f59e5a11700 1 -- 172.21.15.3:0/3916308323 >> v1:172.21.15.3:6789/0 conn(0x7f59e009e000 legacy=0x7f59e01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:41.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.720+0000 7f59e5a11700 1 -- 172.21.15.3:0/3916308323 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e009a500 con 0x7f59e009e410 2023-10-03T10:14:41.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.721+0000 7f59e5a11700 1 --2- 172.21.15.3:0/3916308323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e009e410 0x7f59e0134100 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f59d8009d90 tx=0x7f59d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e228a30bab31bdf1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:41.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.721+0000 7f59d7fff700 1 -- 172.21.15.3:0/3916308323 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d800e070 con 0x7f59e009e410 2023-10-03T10:14:41.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.721+0000 7f59d7fff700 1 -- 172.21.15.3:0/3916308323 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d800bb20 con 0x7f59e009e410 2023-10-03T10:14:41.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.721+0000 7f59d7fff700 1 -- 172.21.15.3:0/3916308323 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d8012430 con 0x7f59e009e410 2023-10-03T10:14:41.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.721+0000 7f59e7474700 1 -- 172.21.15.3:0/3916308323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e009e410 msgr2=0x7f59e0134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:41.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.721+0000 7f59e7474700 1 --2- 172.21.15.3:0/3916308323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e009e410 0x7f59e0134100 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f59d8009d90 tx=0x7f59d800b330 comp rx=0 tx=0).stop 2023-10-03T10:14:41.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.722+0000 7f59e7474700 1 -- 172.21.15.3:0/3916308323 shutdown_connections 2023-10-03T10:14:41.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.722+0000 7f59e7474700 1 --2- 172.21.15.3:0/3916308323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e009e410 0x7f59e0134100 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:41.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.722+0000 7f59e7474700 1 --2- 172.21.15.3:0/3916308323 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59e009d6f0 0x7f59e009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:41.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.722+0000 7f59e7474700 1 -- 172.21.15.3:0/3916308323 >> 172.21.15.3:0/3916308323 conn(0x7f59e0066f50 msgr2=0x7f59e009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:41.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.722+0000 7f59e7474700 1 -- 172.21.15.3:0/3916308323 shutdown_connections 2023-10-03T10:14:41.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.722+0000 7f59e7474700 1 -- 172.21.15.3:0/3916308323 wait complete. 2023-10-03T10:14:41.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.723+0000 7f59e7474700 1 Processor -- start 2023-10-03T10:14:41.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.723+0000 7f59e7474700 1 -- start start 2023-10-03T10:14:41.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.724+0000 7f59e7474700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59e009d6f0 0x7f59e01a5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:41.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.724+0000 7f59e7474700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59e009e000 0x7f59e01a6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:41.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.724+0000 7f59e7474700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e009e410 0x7f59e01a6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:41.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.724+0000 7f59e7474700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e01a8940 con 0x7f59e009e410 2023-10-03T10:14:41.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.724+0000 7f59e7474700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e01a7470 con 0x7f59e009d6f0 2023-10-03T10:14:41.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.724+0000 7f59e7474700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f59e01a7b40 con 0x7f59e009e000 2023-10-03T10:14:41.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.724+0000 7f59e5210700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59e009d6f0 0x7f59e01a5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:41.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.724+0000 7f59e5a11700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e009e410 0x7f59e01a6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:41.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.724+0000 7f59e5210700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59e009d6f0 0x7f59e01a5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51096/0 (socket says 172.21.15.3:51096) 2023-10-03T10:14:41.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.724+0000 7f59e5210700 1 -- 172.21.15.3:0/3611795470 learned_addr learned my addr 172.21.15.3:0/3611795470 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:41.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.725+0000 7f59e4a0f700 1 --2- 172.21.15.3:0/3611795470 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59e009e000 0x7f59e01a6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:41.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.725+0000 7f59e5a11700 1 -- 172.21.15.3:0/3611795470 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59e009e000 msgr2=0x7f59e01a6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:41.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.725+0000 7f59e5a11700 1 --2- 172.21.15.3:0/3611795470 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59e009e000 0x7f59e01a6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:41.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.725+0000 7f59e5a11700 1 -- 172.21.15.3:0/3611795470 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59e009d6f0 msgr2=0x7f59e01a5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:41.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.725+0000 7f59e5a11700 1 --2- 172.21.15.3:0/3611795470 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59e009d6f0 0x7f59e01a5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:41.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.725+0000 7f59e5a11700 1 -- 172.21.15.3:0/3611795470 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d8009a70 con 0x7f59e009e410 2023-10-03T10:14:41.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.725+0000 7f59e5210700 1 --2- 172.21.15.3:0/3611795470 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59e009d6f0 0x7f59e01a5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:41.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.725+0000 7f59e5a11700 1 --2- 172.21.15.3:0/3611795470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e009e410 0x7f59e01a6a40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f59d8016000 tx=0x7f59d8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:41.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.725+0000 7f59d5ffb700 1 -- 172.21.15.3:0/3611795470 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d8012430 con 0x7f59e009e410 2023-10-03T10:14:41.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.725+0000 7f59d5ffb700 1 -- 172.21.15.3:0/3611795470 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d8020070 con 0x7f59e009e410 2023-10-03T10:14:41.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.725+0000 7f59d5ffb700 1 -- 172.21.15.3:0/3611795470 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d8004280 con 0x7f59e009e410 2023-10-03T10:14:41.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.725+0000 7f59e7474700 1 -- 172.21.15.3:0/3611795470 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e01a8e00 con 0x7f59e009e410 2023-10-03T10:14:41.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.726+0000 7f59e7474700 1 -- 172.21.15.3:0/3611795470 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e00a30e0 con 0x7f59e009e410 2023-10-03T10:14:41.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.726+0000 7f59d5ffb700 1 -- 172.21.15.3:0/3611795470 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f59d80043e0 con 0x7f59e009e410 2023-10-03T10:14:41.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.727+0000 7f59d5ffb700 1 -- 172.21.15.3:0/3611795470 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f59d8022070 con 0x7f59e009e410 2023-10-03T10:14:41.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.727+0000 7f59e7474700 1 -- 172.21.15.3:0/3611795470 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e0074ea0 con 0x7f59e009e410 2023-10-03T10:14:41.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.732+0000 7f59d5ffb700 1 -- 172.21.15.3:0/3611795470 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f59d801b400 con 0x7f59e009e410 2023-10-03T10:14:41.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.769+0000 7f14db130700 1 Processor -- start 2023-10-03T10:14:41.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.769+0000 7f14db130700 1 -- start start 2023-10-03T10:14:41.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.770+0000 7f14db130700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f14d409e930 0x7f14d409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:41.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.770+0000 7f14db130700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f14d409f650 0x7f14d4131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:41.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.770+0000 7f14db130700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d4070940 con 0x7f14d409f240 2023-10-03T10:14:41.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.770+0000 7f14db130700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14d406fd60 con 0x7f14d409f650 2023-10-03T10:14:41.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.770+0000 7f14db130700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d406fa80 con 0x7f14d409e930 2023-10-03T10:14:41.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.770+0000 7f14d3fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f14d409f240 0x7f14d41317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40552/0 (socket says 172.21.15.3:40552) 2023-10-03T10:14:41.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.770+0000 7f14d3fff700 1 -- 172.21.15.3:0/4130945423 learned_addr learned my addr 172.21.15.3:0/4130945423 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:41.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.770+0000 7f14d96cd700 1 --2- 172.21.15.3:0/4130945423 >> v2:172.21.15.3:3300/0 conn(0x7f14d409f650 0x7f14d4131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:41.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.771+0000 7f14d8ecc700 1 --2- 172.21.15.3:0/4130945423 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f14d409e930 0x7f14d409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:41.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.771+0000 7f14d96cd700 1 -- 172.21.15.3:0/4130945423 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f14d409e930 msgr2=0x7f14d409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:41.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.771+0000 7f14d96cd700 1 --2- 172.21.15.3:0/4130945423 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f14d409e930 0x7f14d409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:41.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.771+0000 7f14d96cd700 1 -- 172.21.15.3:0/4130945423 >> v1:172.21.15.3:6789/0 conn(0x7f14d409f240 legacy=0x7f14d41317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:41.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.771+0000 7f14d96cd700 1 -- 172.21.15.3:0/4130945423 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d409fb60 con 0x7f14d409f650 2023-10-03T10:14:41.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.771+0000 7f14d96cd700 1 --2- 172.21.15.3:0/4130945423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14d409f650 0x7f14d4131f00 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f14cc009d90 tx=0x7f14cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559983254c7d77be server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:41.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.771+0000 7f14d37fe700 1 -- 172.21.15.3:0/4130945423 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc00e070 con 0x7f14d409f650 2023-10-03T10:14:41.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.771+0000 7f14d37fe700 1 -- 172.21.15.3:0/4130945423 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14cc00bb20 con 0x7f14d409f650 2023-10-03T10:14:41.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.771+0000 7f14d37fe700 1 -- 172.21.15.3:0/4130945423 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc012430 con 0x7f14d409f650 2023-10-03T10:14:41.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.772+0000 7f14db130700 1 -- 172.21.15.3:0/4130945423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14d409f650 msgr2=0x7f14d4131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:41.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.772+0000 7f14db130700 1 --2- 172.21.15.3:0/4130945423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14d409f650 0x7f14d4131f00 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f14cc009d90 tx=0x7f14cc00b330 comp rx=0 tx=0).stop 2023-10-03T10:14:41.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.772+0000 7f14db130700 1 -- 172.21.15.3:0/4130945423 shutdown_connections 2023-10-03T10:14:41.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.772+0000 7f14db130700 1 --2- 172.21.15.3:0/4130945423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14d409f650 0x7f14d4131f00 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:41.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.772+0000 7f14db130700 1 --2- 172.21.15.3:0/4130945423 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f14d409e930 0x7f14d409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:41.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.772+0000 7f14db130700 1 -- 172.21.15.3:0/4130945423 >> 172.21.15.3:0/4130945423 conn(0x7f14d40a4210 msgr2=0x7f14d40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:41.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.772+0000 7f14db130700 1 -- 172.21.15.3:0/4130945423 shutdown_connections 2023-10-03T10:14:41.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.772+0000 7f14db130700 1 -- 172.21.15.3:0/4130945423 wait complete. 2023-10-03T10:14:41.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.774+0000 7f14db130700 1 Processor -- start 2023-10-03T10:14:41.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.774+0000 7f14db130700 1 -- start start 2023-10-03T10:14:41.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.775+0000 7f14db130700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f14d409e930 0x7f14d406b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:41.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.775+0000 7f14db130700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f14d409f240 0x7f14d406bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:41.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.775+0000 7f14db130700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14d409f650 0x7f14d406c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:41.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.775+0000 7f14db130700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d406cd80 con 0x7f14d409f650 2023-10-03T10:14:41.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.775+0000 7f14db130700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d41a88e0 con 0x7f14d409f240 2023-10-03T10:14:41.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.775+0000 7f14db130700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14d41b71d0 con 0x7f14d409e930 2023-10-03T10:14:41.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.775+0000 7f14d3fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f14d409f240 0x7f14d406bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:41.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.775+0000 7f14d3fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f14d409f240 0x7f14d406bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51118/0 (socket says 172.21.15.3:51118) 2023-10-03T10:14:41.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.775+0000 7f14d3fff700 1 -- 172.21.15.3:0/111057541 learned_addr learned my addr 172.21.15.3:0/111057541 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:41.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.775+0000 7f14d96cd700 1 --2- 172.21.15.3:0/111057541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14d409f650 0x7f14d406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:41.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.775+0000 7f14d8ecc700 1 --2- 172.21.15.3:0/111057541 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f14d409e930 0x7f14d406b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:41.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.776+0000 7f14d3fff700 1 -- 172.21.15.3:0/111057541 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f14d409e930 msgr2=0x7f14d406b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:41.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.776+0000 7f14d3fff700 1 --2- 172.21.15.3:0/111057541 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f14d409e930 0x7f14d406b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:41.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.776+0000 7f14d3fff700 1 -- 172.21.15.3:0/111057541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14d409f650 msgr2=0x7f14d406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:41.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.776+0000 7f14d3fff700 1 --2- 172.21.15.3:0/111057541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14d409f650 0x7f14d406c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:41.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.776+0000 7f14d3fff700 1 -- 172.21.15.3:0/111057541 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14cc009a70 con 0x7f14d409f240 2023-10-03T10:14:41.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.776+0000 7f14d3fff700 1 --2- 172.21.15.3:0/111057541 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f14d409f240 0x7f14d406bdf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f14c000c530 tx=0x7f14c000c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:41.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.776+0000 7f14d17fa700 1 -- 172.21.15.3:0/111057541 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0005d70 con 0x7f14d409f240 2023-10-03T10:14:41.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.776+0000 7f14d17fa700 1 -- 172.21.15.3:0/111057541 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c00070d0 con 0x7f14d409f240 2023-10-03T10:14:41.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.776+0000 7f14db130700 1 -- 172.21.15.3:0/111057541 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d40a0190 con 0x7f14d409f240 2023-10-03T10:14:41.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.776+0000 7f14d17fa700 1 -- 172.21.15.3:0/111057541 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0007440 con 0x7f14d409f240 2023-10-03T10:14:41.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.776+0000 7f14db130700 1 -- 172.21.15.3:0/111057541 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d40a3460 con 0x7f14d409f240 2023-10-03T10:14:41.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.777+0000 7f14d17fa700 1 -- 172.21.15.3:0/111057541 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f14c00075a0 con 0x7f14d409f240 2023-10-03T10:14:41.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.777+0000 7f14d17fa700 1 -- 172.21.15.3:0/111057541 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f14c003ea50 con 0x7f14d409f240 2023-10-03T10:14:41.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.778+0000 7f14db130700 1 -- 172.21.15.3:0/111057541 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d4071d80 con 0x7f14d409f240 2023-10-03T10:14:41.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.786+0000 7f14d17fa700 1 -- 172.21.15.3:0/111057541 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f14c0019b40 con 0x7f14d409f240 2023-10-03T10:14:41.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.804+0000 7f15faf8a700 1 Processor -- start 2023-10-03T10:14:41.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.804+0000 7f15faf8a700 1 -- start start 2023-10-03T10:14:41.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.805+0000 7f15faf8a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f15f40a0a30 0x7f15f40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:41.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.805+0000 7f15faf8a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f15f40a1340 0x7f15f4133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:41.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.805+0000 7f15faf8a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f15f4133fa0 0x7f15f4134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:41.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.805+0000 7f15faf8a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15f409edd0 con 0x7f15f4133fa0 2023-10-03T10:14:41.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.805+0000 7f15faf8a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f409e1f0 con 0x7f15f40a0a30 2023-10-03T10:14:41.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.805+0000 7f15faf8a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f15f409dc50 con 0x7f15f40a1340 2023-10-03T10:14:41.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.805+0000 7f15f8d26700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f15f40a0a30 0x7f15f40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:41.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.805+0000 7f15ebfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f15f40a1340 0x7f15f4133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:41.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.805+0000 7f15ebfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f15f40a1340 0x7f15f4133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:32974/0 (socket says 172.21.15.3:32974) 2023-10-03T10:14:41.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.805+0000 7f15f9527700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f15f4133fa0 0x7f15f4134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:41.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.805+0000 7f15ebfff700 1 -- 172.21.15.3:0/3883412861 learned_addr learned my addr 172.21.15.3:0/3883412861 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:41.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.805+0000 7f15f8d26700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f15f40a0a30 0x7f15f40a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51134/0 (socket says 172.21.15.3:51134) 2023-10-03T10:14:41.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.805+0000 7f15ebfff700 1 -- 172.21.15.3:0/3883412861 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f15f40a0a30 msgr2=0x7f15f40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:41.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.805+0000 7f15ebfff700 1 --2- 172.21.15.3:0/3883412861 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f15f40a0a30 0x7f15f40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:41.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.805+0000 7f15ebfff700 1 -- 172.21.15.3:0/3883412861 >> v2:172.21.15.3:3300/0 conn(0x7f15f4133fa0 msgr2=0x7f15f4134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:41.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.805+0000 7f15ebfff700 1 --2- 172.21.15.3:0/3883412861 >> v2:172.21.15.3:3300/0 conn(0x7f15f4133fa0 0x7f15f4134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:41.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.805+0000 7f15ebfff700 1 -- 172.21.15.3:0/3883412861 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f40a1fe0 con 0x7f15f40a1340 2023-10-03T10:14:41.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.805+0000 7f15f9527700 1 --2- 172.21.15.3:0/3883412861 >> v2:172.21.15.3:3300/0 conn(0x7f15f4133fa0 0x7f15f4134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:41.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.805+0000 7f15f8d26700 1 --2- 172.21.15.3:0/3883412861 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f15f40a0a30 0x7f15f40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:41.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.805+0000 7f15ebfff700 1 --2- 172.21.15.3:0/3883412861 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f15f40a1340 0x7f15f4133a60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f15e40098f0 tx=0x7f15e4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47d4fe3341a6e405 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:41.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.806+0000 7f15eb7fe700 1 -- 172.21.15.3:0/3883412861 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e4011830 con 0x7f15f40a1340 2023-10-03T10:14:41.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.806+0000 7f15eb7fe700 1 -- 172.21.15.3:0/3883412861 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e4011990 con 0x7f15f40a1340 2023-10-03T10:14:41.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.806+0000 7f15eb7fe700 1 -- 172.21.15.3:0/3883412861 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e4011d20 con 0x7f15f40a1340 2023-10-03T10:14:41.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.806+0000 7f15faf8a700 1 -- 172.21.15.3:0/3883412861 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f15f40a1340 msgr2=0x7f15f4133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:41.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.806+0000 7f15faf8a700 1 --2- 172.21.15.3:0/3883412861 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f15f40a1340 0x7f15f4133a60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f15e40098f0 tx=0x7f15e4009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:41.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.806+0000 7f15faf8a700 1 -- 172.21.15.3:0/3883412861 shutdown_connections 2023-10-03T10:14:41.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.806+0000 7f15faf8a700 1 --2- 172.21.15.3:0/3883412861 >> v2:172.21.15.3:3300/0 conn(0x7f15f4133fa0 0x7f15f4134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:41.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.806+0000 7f15faf8a700 1 --2- 172.21.15.3:0/3883412861 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f15f40a0a30 0x7f15f40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:41.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.806+0000 7f15faf8a700 1 --2- 172.21.15.3:0/3883412861 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f15f40a1340 0x7f15f4133a60 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:41.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.806+0000 7f15faf8a700 1 -- 172.21.15.3:0/3883412861 >> 172.21.15.3:0/3883412861 conn(0x7f15f40a6550 msgr2=0x7f15f40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:41.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.806+0000 7f15faf8a700 1 -- 172.21.15.3:0/3883412861 shutdown_connections 2023-10-03T10:14:41.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.806+0000 7f15faf8a700 1 -- 172.21.15.3:0/3883412861 wait complete. 2023-10-03T10:14:41.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.808+0000 7f15faf8a700 1 Processor -- start 2023-10-03T10:14:41.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.808+0000 7f15faf8a700 1 -- start start 2023-10-03T10:14:41.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.809+0000 7f15faf8a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f15f40a0a30 0x7f15f4120220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:41.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.809+0000 7f15faf8a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f15f40a1340 0x7f15f4120760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:41.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.809+0000 7f15faf8a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15f4133fa0 0x7f15f4120ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:41.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.809+0000 7f15faf8a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f4123430 con 0x7f15f4133fa0 2023-10-03T10:14:41.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.809+0000 7f15faf8a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f40a0220 con 0x7f15f40a1340 2023-10-03T10:14:41.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.809+0000 7f15faf8a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f15f40a3430 con 0x7f15f40a0a30 2023-10-03T10:14:41.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.809+0000 7f15f9527700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15f4133fa0 0x7f15f4120ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:41.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.809+0000 7f15f9527700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15f4133fa0 0x7f15f4120ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50230/0 (socket says 172.21.15.3:50230) 2023-10-03T10:14:41.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.809+0000 7f15f9527700 1 -- 172.21.15.3:0/3737871902 learned_addr learned my addr 172.21.15.3:0/3737871902 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:41.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.809+0000 7f15f8d26700 1 --2- 172.21.15.3:0/3737871902 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f15f40a0a30 0x7f15f4120220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:41.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.809+0000 7f15ebfff700 1 --2- 172.21.15.3:0/3737871902 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f15f40a1340 0x7f15f4120760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:41.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.809+0000 7f15f9527700 1 -- 172.21.15.3:0/3737871902 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f15f40a0a30 msgr2=0x7f15f4120220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:41.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.809+0000 7f15f9527700 1 --2- 172.21.15.3:0/3737871902 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f15f40a0a30 0x7f15f4120220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:41.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.809+0000 7f15f9527700 1 -- 172.21.15.3:0/3737871902 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f15f40a1340 msgr2=0x7f15f4120760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:41.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.809+0000 7f15f9527700 1 --2- 172.21.15.3:0/3737871902 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f15f40a1340 0x7f15f4120760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:41.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.809+0000 7f15f9527700 1 -- 172.21.15.3:0/3737871902 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f00094a0 con 0x7f15f4133fa0 2023-10-03T10:14:41.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.810+0000 7f15f9527700 1 --2- 172.21.15.3:0/3737871902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15f4133fa0 0x7f15f4120ca0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00efc0 tx=0x7f15ec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:41.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.810+0000 7f15e97fa700 1 -- 172.21.15.3:0/3737871902 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec00ca90 con 0x7f15f4133fa0 2023-10-03T10:14:41.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.810+0000 7f15e97fa700 1 -- 172.21.15.3:0/3737871902 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ec00f040 con 0x7f15f4133fa0 2023-10-03T10:14:41.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.810+0000 7f15e97fa700 1 -- 172.21.15.3:0/3737871902 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec00ce10 con 0x7f15f4133fa0 2023-10-03T10:14:41.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.810+0000 7f15faf8a700 1 -- 172.21.15.3:0/3737871902 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15ec009a70 con 0x7f15f4133fa0 2023-10-03T10:14:41.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.810+0000 7f15faf8a700 1 -- 172.21.15.3:0/3737871902 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e4009550 con 0x7f15f4133fa0 2023-10-03T10:14:41.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.811+0000 7f15e97fa700 1 -- 172.21.15.3:0/3737871902 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f15ec014480 con 0x7f15f4133fa0 2023-10-03T10:14:41.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.811+0000 7f15e97fa700 1 -- 172.21.15.3:0/3737871902 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f15ec03fc80 con 0x7f15f4133fa0 2023-10-03T10:14:41.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.812+0000 7f15faf8a700 1 -- 172.21.15.3:0/3737871902 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15f4123660 con 0x7f15f4133fa0 2023-10-03T10:14:41.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.817+0000 7f15e97fa700 1 -- 172.21.15.3:0/3737871902 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f15f4123660 con 0x7f15f4133fa0 2023-10-03T10:14:41.841 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:41.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.981+0000 7feab236b700 1 Processor -- start 2023-10-03T10:14:41.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.981+0000 7feab236b700 1 -- start start 2023-10-03T10:14:41.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.982+0000 7feab236b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaac09e1d0 0x7feaac138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:41.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.982+0000 7feab236b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feaac139170 0x7feaac13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:41.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.982+0000 7feab236b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaac09b120 con 0x7feaac09d6f0 2023-10-03T10:14:41.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.982+0000 7feab236b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaac06b460 con 0x7feaac139170 2023-10-03T10:14:41.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.982+0000 7feab236b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaac09a7e0 con 0x7feaac09e1d0 2023-10-03T10:14:41.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.982+0000 7feaab7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaac09e1d0 0x7feaac138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:41.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.982+0000 7feaabfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7feaac09d6f0 0x7feaac09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40566/0 (socket says 172.21.15.3:40566) 2023-10-03T10:14:41.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.982+0000 7feaabfff700 1 -- 172.21.15.3:0/1449295134 learned_addr learned my addr 172.21.15.3:0/1449295134 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:41.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.982+0000 7feab0908700 1 --2- 172.21.15.3:0/1449295134 >> v2:172.21.15.3:3300/0 conn(0x7feaac139170 0x7feaac13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:41.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.983+0000 7feaab7fe700 1 -- 172.21.15.3:0/1449295134 >> v2:172.21.15.3:3300/0 conn(0x7feaac139170 msgr2=0x7feaac13b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:41.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.983+0000 7feaab7fe700 1 --2- 172.21.15.3:0/1449295134 >> v2:172.21.15.3:3300/0 conn(0x7feaac139170 0x7feaac13b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:41.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.983+0000 7feaab7fe700 1 -- 172.21.15.3:0/1449295134 >> v1:172.21.15.3:6789/0 conn(0x7feaac09d6f0 legacy=0x7feaac09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:41.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.983+0000 7feaab7fe700 1 -- 172.21.15.3:0/1449295134 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaac09a500 con 0x7feaac09e1d0 2023-10-03T10:14:41.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.983+0000 7feaaaffd700 1 -- 172.21.15.3:0/1449295134 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1257057597 0 0) 0x7feaac09b120 con 0x7feaac09d6f0 2023-10-03T10:14:41.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.983+0000 7feab0908700 1 --2- 172.21.15.3:0/1449295134 >> v2:172.21.15.3:3300/0 conn(0x7feaac139170 0x7feaac13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:41.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.983+0000 7feaab7fe700 1 --2- 172.21.15.3:0/1449295134 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaac09e1d0 0x7feaac138c30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7feaa0009870 tx=0x7feaa0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=593eb1e3e1287a0c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:41.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.983+0000 7feaaaffd700 1 -- 172.21.15.3:0/1449295134 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa0011830 con 0x7feaac09e1d0 2023-10-03T10:14:41.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.983+0000 7feaaaffd700 1 -- 172.21.15.3:0/1449295134 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa0011990 con 0x7feaac09e1d0 2023-10-03T10:14:41.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.983+0000 7feaaaffd700 1 -- 172.21.15.3:0/1449295134 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa0011d20 con 0x7feaac09e1d0 2023-10-03T10:14:41.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.984+0000 7feab236b700 1 -- 172.21.15.3:0/1449295134 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaac09e1d0 msgr2=0x7feaac138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:41.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.984+0000 7feab236b700 1 --2- 172.21.15.3:0/1449295134 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaac09e1d0 0x7feaac138c30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7feaa0009870 tx=0x7feaa0009b80 comp rx=0 tx=0).stop 2023-10-03T10:14:41.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.984+0000 7feab236b700 1 -- 172.21.15.3:0/1449295134 shutdown_connections 2023-10-03T10:14:41.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.984+0000 7feab236b700 1 --2- 172.21.15.3:0/1449295134 >> v2:172.21.15.3:3300/0 conn(0x7feaac139170 0x7feaac13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:41.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.984+0000 7feab236b700 1 --2- 172.21.15.3:0/1449295134 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaac09e1d0 0x7feaac138c30 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:41.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.984+0000 7feab236b700 1 -- 172.21.15.3:0/1449295134 >> 172.21.15.3:0/1449295134 conn(0x7feaac066f50 msgr2=0x7feaac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:41.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.984+0000 7feab236b700 1 -- 172.21.15.3:0/1449295134 shutdown_connections 2023-10-03T10:14:41.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.984+0000 7feab236b700 1 -- 172.21.15.3:0/1449295134 wait complete. 2023-10-03T10:14:41.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.986+0000 7feab236b700 1 Processor -- start 2023-10-03T10:14:41.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.986+0000 7feab236b700 1 -- start start 2023-10-03T10:14:41.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.987+0000 7feab236b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaac09d6f0 0x7feaac1c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:41.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.987+0000 7feab236b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaac09e1d0 0x7feaac1c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:41.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.987+0000 7feab236b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaac139170 0x7feaac121870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:41.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.987+0000 7feab236b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaac124b60 con 0x7feaac09d6f0 2023-10-03T10:14:41.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.987+0000 7feab236b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaac1376f0 con 0x7feaac09e1d0 2023-10-03T10:14:41.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.987+0000 7feab236b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feaac1235b0 con 0x7feaac139170 2023-10-03T10:14:41.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.987+0000 7feaabfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaac09d6f0 0x7feaac1c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:41.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.987+0000 7feaabfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaac09d6f0 0x7feaac1c8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50246/0 (socket says 172.21.15.3:50246) 2023-10-03T10:14:41.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.987+0000 7feaabfff700 1 -- 172.21.15.3:0/2990428116 learned_addr learned my addr 172.21.15.3:0/2990428116 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:41.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.987+0000 7feaab7fe700 1 --2- 172.21.15.3:0/2990428116 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaac09e1d0 0x7feaac1c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:41.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.987+0000 7feab0908700 1 --2- 172.21.15.3:0/2990428116 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaac139170 0x7feaac121870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:41.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.987+0000 7feaabfff700 1 -- 172.21.15.3:0/2990428116 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaac139170 msgr2=0x7feaac121870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:41.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.987+0000 7feaabfff700 1 --2- 172.21.15.3:0/2990428116 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaac139170 0x7feaac121870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:41.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.987+0000 7feaabfff700 1 -- 172.21.15.3:0/2990428116 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaac09e1d0 msgr2=0x7feaac1c89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:41.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.987+0000 7feaabfff700 1 --2- 172.21.15.3:0/2990428116 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaac09e1d0 0x7feaac1c89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:41.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.988+0000 7feaabfff700 1 -- 172.21.15.3:0/2990428116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa4009a70 con 0x7feaac09d6f0 2023-10-03T10:14:41.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.988+0000 7feaabfff700 1 --2- 172.21.15.3:0/2990428116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaac09d6f0 0x7feaac1c8460 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00c550 tx=0x7fea9c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:41.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.988+0000 7feaa8ff9700 1 -- 172.21.15.3:0/2990428116 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9c004c80 con 0x7feaac09d6f0 2023-10-03T10:14:41.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.988+0000 7feaa8ff9700 1 -- 172.21.15.3:0/2990428116 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea9c004e20 con 0x7feaac09d6f0 2023-10-03T10:14:41.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.988+0000 7feaa8ff9700 1 -- 172.21.15.3:0/2990428116 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9c007310 con 0x7feaac09d6f0 2023-10-03T10:14:41.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.988+0000 7feab236b700 1 -- 172.21.15.3:0/2990428116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaa00094a0 con 0x7feaac09d6f0 2023-10-03T10:14:41.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.988+0000 7feab236b700 1 -- 172.21.15.3:0/2990428116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaac09c710 con 0x7feaac09d6f0 2023-10-03T10:14:41.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.989+0000 7feaa8ff9700 1 -- 172.21.15.3:0/2990428116 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fea9c007470 con 0x7feaac09d6f0 2023-10-03T10:14:41.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.989+0000 7feaa8ff9700 1 -- 172.21.15.3:0/2990428116 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fea9c03f340 con 0x7feaac09d6f0 2023-10-03T10:14:41.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.989+0000 7feab236b700 1 -- 172.21.15.3:0/2990428116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaac074ea0 con 0x7feaac09d6f0 2023-10-03T10:14:41.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.994+0000 7f59e7474700 1 -- 172.21.15.3:0/3611795470 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f59e01a9830 con 0x7f59e009e410 2023-10-03T10:14:41.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.994+0000 7f59d5ffb700 1 -- 172.21.15.3:0/3611795470 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f59d802c020 con 0x7f59e009e410 2023-10-03T10:14:41.995 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:41.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.995+0000 7feaa8ff9700 1 -- 172.21.15.3:0/2990428116 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fea9c0123b0 con 0x7feaac09d6f0 2023-10-03T10:14:41.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.997+0000 7f59e7474700 1 -- 172.21.15.3:0/3611795470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e009e410 msgr2=0x7f59e01a6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:41.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:41.997+0000 7f59e7474700 1 --2- 172.21.15.3:0/3611795470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e009e410 0x7f59e01a6a40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f59d8016000 tx=0x7f59d8004ad0 comp rx=0 tx=0).stop 2023-10-03T10:14:42.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.000+0000 7f59e7474700 1 -- 172.21.15.3:0/3611795470 shutdown_connections 2023-10-03T10:14:42.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.000+0000 7f59e7474700 1 --2- 172.21.15.3:0/3611795470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e009e410 0x7f59e01a6a40 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:42.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.000+0000 7f59e7474700 1 --2- 172.21.15.3:0/3611795470 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59e009d6f0 0x7f59e01a5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:42.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.000+0000 7f59e7474700 1 --2- 172.21.15.3:0/3611795470 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59e009e000 0x7f59e01a6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:42.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.000+0000 7f59e7474700 1 -- 172.21.15.3:0/3611795470 >> 172.21.15.3:0/3611795470 conn(0x7f59e0066f50 msgr2=0x7f59e01baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:42.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.000+0000 7f59e7474700 1 -- 172.21.15.3:0/3611795470 shutdown_connections 2023-10-03T10:14:42.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.000+0000 7f59e7474700 1 -- 172.21.15.3:0/3611795470 wait complete. 2023-10-03T10:14:42.014 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:42.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.029+0000 7f14db130700 1 -- 172.21.15.3:0/111057541 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f14d41a0b30 con 0x7f14d409f240 2023-10-03T10:14:42.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.029+0000 7f14d17fa700 1 -- 172.21.15.3:0/111057541 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f14c0017020 con 0x7f14d409f240 2023-10-03T10:14:42.030 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:42.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.030+0000 7f15faf8a700 1 -- 172.21.15.3:0/3737871902 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f15f4123660 con 0x7f15f4133fa0 2023-10-03T10:14:42.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.030+0000 7f15e97fa700 1 -- 172.21.15.3:0/3737871902 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f15f4123660 con 0x7f15f4133fa0 2023-10-03T10:14:42.031 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:42.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.032+0000 7f14db130700 1 -- 172.21.15.3:0/111057541 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f14d409f240 msgr2=0x7f14d406bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:42.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.032+0000 7f14db130700 1 --2- 172.21.15.3:0/111057541 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f14d409f240 0x7f14d406bdf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f14c000c530 tx=0x7f14c000c840 comp rx=0 tx=0).stop 2023-10-03T10:14:42.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.032+0000 7f14db130700 1 -- 172.21.15.3:0/111057541 shutdown_connections 2023-10-03T10:14:42.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.032+0000 7f14db130700 1 --2- 172.21.15.3:0/111057541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14d409f650 0x7f14d406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:42.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.032+0000 7f14db130700 1 --2- 172.21.15.3:0/111057541 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f14d409e930 0x7f14d406b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:42.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.032+0000 7f14db130700 1 --2- 172.21.15.3:0/111057541 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f14d409f240 0x7f14d406bdf0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:42.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.032+0000 7f14db130700 1 -- 172.21.15.3:0/111057541 >> 172.21.15.3:0/111057541 conn(0x7f14d40a4210 msgr2=0x7f14d41b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:42.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.032+0000 7f14db130700 1 -- 172.21.15.3:0/111057541 shutdown_connections 2023-10-03T10:14:42.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.032+0000 7f14db130700 1 -- 172.21.15.3:0/111057541 wait complete. 2023-10-03T10:14:42.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.033+0000 7f15faf8a700 1 -- 172.21.15.3:0/3737871902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15f4133fa0 msgr2=0x7f15f4120ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:42.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.033+0000 7f15faf8a700 1 --2- 172.21.15.3:0/3737871902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15f4133fa0 0x7f15f4120ca0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00efc0 tx=0x7f15ec00c3a0 comp rx=0 tx=0).stop 2023-10-03T10:14:42.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.033+0000 7f15faf8a700 1 -- 172.21.15.3:0/3737871902 shutdown_connections 2023-10-03T10:14:42.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.033+0000 7f15faf8a700 1 --2- 172.21.15.3:0/3737871902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15f4133fa0 0x7f15f4120ca0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:42.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.033+0000 7f15faf8a700 1 --2- 172.21.15.3:0/3737871902 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f15f40a0a30 0x7f15f4120220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:42.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.033+0000 7f15faf8a700 1 --2- 172.21.15.3:0/3737871902 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f15f40a1340 0x7f15f4120760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:42.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.033+0000 7f15faf8a700 1 -- 172.21.15.3:0/3737871902 >> 172.21.15.3:0/3737871902 conn(0x7f15f40a6550 msgr2=0x7f15f41baff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:42.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.033+0000 7f15faf8a700 1 -- 172.21.15.3:0/3737871902 shutdown_connections 2023-10-03T10:14:42.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.033+0000 7f15faf8a700 1 -- 172.21.15.3:0/3737871902 wait complete. 2023-10-03T10:14:42.049 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:42.049 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:42.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.209+0000 7feab236b700 1 -- 172.21.15.3:0/2990428116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7feaac09f420 con 0x7feaac09d6f0 2023-10-03T10:14:42.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.210+0000 7feaa8ff9700 1 -- 172.21.15.3:0/2990428116 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fea9c011360 con 0x7feaac09d6f0 2023-10-03T10:14:42.210 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:42.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.212+0000 7feab236b700 1 -- 172.21.15.3:0/2990428116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaac09d6f0 msgr2=0x7feaac1c8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:42.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.212+0000 7feab236b700 1 --2- 172.21.15.3:0/2990428116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaac09d6f0 0x7feaac1c8460 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00c550 tx=0x7fea9c00c860 comp rx=0 tx=0).stop 2023-10-03T10:14:42.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.212+0000 7feab236b700 1 -- 172.21.15.3:0/2990428116 shutdown_connections 2023-10-03T10:14:42.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.212+0000 7feab236b700 1 --2- 172.21.15.3:0/2990428116 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaac139170 0x7feaac121870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:42.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.212+0000 7feab236b700 1 --2- 172.21.15.3:0/2990428116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaac09d6f0 0x7feaac1c8460 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:42.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.212+0000 7feab236b700 1 --2- 172.21.15.3:0/2990428116 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaac09e1d0 0x7feaac1c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:42.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.212+0000 7feab236b700 1 -- 172.21.15.3:0/2990428116 >> 172.21.15.3:0/2990428116 conn(0x7feaac066f50 msgr2=0x7feaac070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:42.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.212+0000 7feab236b700 1 -- 172.21.15.3:0/2990428116 shutdown_connections 2023-10-03T10:14:42.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:42.212+0000 7feab236b700 1 -- 172.21.15.3:0/2990428116 wait complete. 2023-10-03T10:14:42.226 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:43.015 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:43.050 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:43.050 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:43.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.169+0000 7fc0011db700 1 Processor -- start 2023-10-03T10:14:43.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.169+0000 7fc0011db700 1 -- start start 2023-10-03T10:14:43.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.170+0000 7fc0011db700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbffc136710 0x7fbffc138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:43.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.170+0000 7fc0011db700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbffc139030 0x7fbffc13b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:43.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.170+0000 7fc0011db700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbffc06ef30 con 0x7fbffc1338a0 2023-10-03T10:14:43.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.170+0000 7fc0011db700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbffc06fef0 con 0x7fbffc139030 2023-10-03T10:14:43.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.170+0000 7fc0011db700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc1334b0 con 0x7fbffc136710 2023-10-03T10:14:43.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.170+0000 7fbffa59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbffc136710 0x7fbffc138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:43.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.170+0000 7fbffa59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbffc136710 0x7fbffc138af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57218/0 (socket says 172.21.15.3:57218) 2023-10-03T10:14:43.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.170+0000 7fbffa59c700 1 -- 172.21.15.3:0/2687242680 learned_addr learned my addr 172.21.15.3:0/2687242680 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:43.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.170+0000 7fbffb59e700 1 --2- 172.21.15.3:0/2687242680 >> v2:172.21.15.3:3300/0 conn(0x7fbffc139030 0x7fbffc13b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:43.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.170+0000 7fbffa59c700 1 -- 172.21.15.3:0/2687242680 >> v2:172.21.15.3:3300/0 conn(0x7fbffc139030 msgr2=0x7fbffc13b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:43.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.170+0000 7fbffa59c700 1 --2- 172.21.15.3:0/2687242680 >> v2:172.21.15.3:3300/0 conn(0x7fbffc139030 0x7fbffc13b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.170+0000 7fbffa59c700 1 -- 172.21.15.3:0/2687242680 >> v1:172.21.15.3:6789/0 conn(0x7fbffc1338a0 legacy=0x7fbffc133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:43.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.170+0000 7fbffa59c700 1 -- 172.21.15.3:0/2687242680 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc06fc30 con 0x7fbffc136710 2023-10-03T10:14:43.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.170+0000 7fbff9d9b700 1 -- 172.21.15.3:0/2687242680 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3664824271 0 0) 0x7fbffc06ef30 con 0x7fbffc1338a0 2023-10-03T10:14:43.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.170+0000 7fbffb59e700 1 --2- 172.21.15.3:0/2687242680 >> v2:172.21.15.3:3300/0 conn(0x7fbffc139030 0x7fbffc13b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:43.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.171+0000 7fbffa59c700 1 --2- 172.21.15.3:0/2687242680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbffc136710 0x7fbffc138af0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbff00098f0 tx=0x7fbff0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5303a5a1dcd12294 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:43.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.171+0000 7fbff9d9b700 1 -- 172.21.15.3:0/2687242680 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0011830 con 0x7fbffc136710 2023-10-03T10:14:43.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.171+0000 7fbff9d9b700 1 -- 172.21.15.3:0/2687242680 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff0011990 con 0x7fbffc136710 2023-10-03T10:14:43.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.171+0000 7fbff9d9b700 1 -- 172.21.15.3:0/2687242680 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0011d20 con 0x7fbffc136710 2023-10-03T10:14:43.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.171+0000 7fc0011db700 1 -- 172.21.15.3:0/2687242680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbffc136710 msgr2=0x7fbffc138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:43.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.171+0000 7fc0011db700 1 --2- 172.21.15.3:0/2687242680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbffc136710 0x7fbffc138af0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbff00098f0 tx=0x7fbff0009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:43.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.171+0000 7fc0011db700 1 -- 172.21.15.3:0/2687242680 shutdown_connections 2023-10-03T10:14:43.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.171+0000 7fc0011db700 1 --2- 172.21.15.3:0/2687242680 >> v2:172.21.15.3:3300/0 conn(0x7fbffc139030 0x7fbffc13b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.171+0000 7fc0011db700 1 --2- 172.21.15.3:0/2687242680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbffc136710 0x7fbffc138af0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.172+0000 7fc0011db700 1 -- 172.21.15.3:0/2687242680 >> 172.21.15.3:0/2687242680 conn(0x7fbffc0a2aa0 msgr2=0x7fbffc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:43.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.172+0000 7fc0011db700 1 -- 172.21.15.3:0/2687242680 shutdown_connections 2023-10-03T10:14:43.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.172+0000 7fc0011db700 1 -- 172.21.15.3:0/2687242680 wait complete. 2023-10-03T10:14:43.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.173+0000 7fc0011db700 1 Processor -- start 2023-10-03T10:14:43.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.173+0000 7fc0011db700 1 -- start start 2023-10-03T10:14:43.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.173+0000 7fc0011db700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbffc1338a0 0x7fbffc11ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:43.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.174+0000 7fc0011db700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbffc136710 0x7fbffc11b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:43.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.174+0000 7fc0011db700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbffc139030 0x7fbffc11b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:43.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.174+0000 7fc0011db700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc130590 con 0x7fbffc1338a0 2023-10-03T10:14:43.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.174+0000 7fc0011db700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc11cea0 con 0x7fbffc136710 2023-10-03T10:14:43.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.174+0000 7fc0011db700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc11c980 con 0x7fbffc139030 2023-10-03T10:14:43.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.174+0000 7fbffa59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbffc136710 0x7fbffc11b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:43.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.174+0000 7fbffa59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbffc136710 0x7fbffc11b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58916/0 (socket says 172.21.15.3:58916) 2023-10-03T10:14:43.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.174+0000 7fbffa59c700 1 -- 172.21.15.3:0/1559043129 learned_addr learned my addr 172.21.15.3:0/1559043129 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:43.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.174+0000 7fbffad9d700 1 --2- 172.21.15.3:0/1559043129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbffc1338a0 0x7fbffc11ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:43.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.174+0000 7fbffb59e700 1 --2- 172.21.15.3:0/1559043129 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbffc139030 0x7fbffc11b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:43.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.174+0000 7fbffa59c700 1 -- 172.21.15.3:0/1559043129 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbffc139030 msgr2=0x7fbffc11b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:43.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.174+0000 7fbffa59c700 1 --2- 172.21.15.3:0/1559043129 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbffc139030 0x7fbffc11b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.174+0000 7fbffa59c700 1 -- 172.21.15.3:0/1559043129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbffc1338a0 msgr2=0x7fbffc11ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:43.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.174+0000 7fbffa59c700 1 --2- 172.21.15.3:0/1559043129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbffc1338a0 0x7fbffc11ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.174+0000 7fbffa59c700 1 -- 172.21.15.3:0/1559043129 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff4009a70 con 0x7fbffc136710 2023-10-03T10:14:43.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.174+0000 7fbffa59c700 1 --2- 172.21.15.3:0/1559043129 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbffc136710 0x7fbffc11b1a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbff0009c40 tx=0x7fbff0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:43.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.175+0000 7fbfe37fe700 1 -- 172.21.15.3:0/1559043129 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0004400 con 0x7fbffc136710 2023-10-03T10:14:43.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.175+0000 7fbfe37fe700 1 -- 172.21.15.3:0/1559043129 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff001f070 con 0x7fbffc136710 2023-10-03T10:14:43.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.175+0000 7fc0011db700 1 -- 172.21.15.3:0/1559043129 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff0009550 con 0x7fbffc136710 2023-10-03T10:14:43.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.175+0000 7fbfe37fe700 1 -- 172.21.15.3:0/1559043129 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff00038a0 con 0x7fbffc136710 2023-10-03T10:14:43.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.175+0000 7fc0011db700 1 -- 172.21.15.3:0/1559043129 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbffc11d060 con 0x7fbffc136710 2023-10-03T10:14:43.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.176+0000 7fbfe37fe700 1 -- 172.21.15.3:0/1559043129 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbff0021030 con 0x7fbffc136710 2023-10-03T10:14:43.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.176+0000 7fbfe37fe700 1 -- 172.21.15.3:0/1559043129 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fbff0040aa0 con 0x7fbffc136710 2023-10-03T10:14:43.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.176+0000 7fc0011db700 1 -- 172.21.15.3:0/1559043129 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbffc11e140 con 0x7fbffc136710 2023-10-03T10:14:43.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.181+0000 7fbfe37fe700 1 -- 172.21.15.3:0/1559043129 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbffc11e140 con 0x7fbffc136710 2023-10-03T10:14:43.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.205+0000 7fac973c0700 1 Processor -- start 2023-10-03T10:14:43.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.206+0000 7fac973c0700 1 -- start start 2023-10-03T10:14:43.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.206+0000 7fac973c0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fac9009e1d0 0x7fac90138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:43.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.206+0000 7fac973c0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fac90139170 0x7fac9013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:43.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.206+0000 7fac973c0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac9009b120 con 0x7fac9009d6f0 2023-10-03T10:14:43.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.206+0000 7fac973c0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac9006b460 con 0x7fac9009e1d0 2023-10-03T10:14:43.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.206+0000 7fac973c0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fac9009a7e0 con 0x7fac90139170 2023-10-03T10:14:43.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.206+0000 7fac9595d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fac90139170 0x7fac9013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:43.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.206+0000 7fac9595d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fac90139170 0x7fac9013b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57242/0 (socket says 172.21.15.3:57242) 2023-10-03T10:14:43.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.206+0000 7fac9595d700 1 -- 172.21.15.3:0/115977475 learned_addr learned my addr 172.21.15.3:0/115977475 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:43.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.206+0000 7fac9495b700 1 --2- 172.21.15.3:0/115977475 >> v2:172.21.15.3:3300/0 conn(0x7fac9009e1d0 0x7fac90138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:43.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.207+0000 7fac9595d700 1 -- 172.21.15.3:0/115977475 >> v2:172.21.15.3:3300/0 conn(0x7fac9009e1d0 msgr2=0x7fac90138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:43.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.207+0000 7fac9595d700 1 --2- 172.21.15.3:0/115977475 >> v2:172.21.15.3:3300/0 conn(0x7fac9009e1d0 0x7fac90138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.207+0000 7fac87fff700 1 -- 172.21.15.3:0/115977475 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3803378286 0 0) 0x7fac9009b120 con 0x7fac9009d6f0 2023-10-03T10:14:43.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.207+0000 7fac9595d700 1 -- 172.21.15.3:0/115977475 >> v1:172.21.15.3:6789/0 conn(0x7fac9009d6f0 legacy=0x7fac9009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:43.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.207+0000 7fac9595d700 1 -- 172.21.15.3:0/115977475 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9009a500 con 0x7fac90139170 2023-10-03T10:14:43.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.207+0000 7fac9595d700 1 --2- 172.21.15.3:0/115977475 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fac90139170 0x7fac9013b550 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fac88009e80 tx=0x7fac8800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d535b1b79b79b0bb server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:43.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.207+0000 7fac87fff700 1 -- 172.21.15.3:0/115977475 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8800e070 con 0x7fac90139170 2023-10-03T10:14:43.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.207+0000 7fac87fff700 1 -- 172.21.15.3:0/115977475 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8800b8d0 con 0x7fac90139170 2023-10-03T10:14:43.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.207+0000 7fac87fff700 1 -- 172.21.15.3:0/115977475 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8800bc80 con 0x7fac90139170 2023-10-03T10:14:43.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.208+0000 7fac973c0700 1 -- 172.21.15.3:0/115977475 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fac90139170 msgr2=0x7fac9013b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:43.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.208+0000 7fac973c0700 1 --2- 172.21.15.3:0/115977475 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fac90139170 0x7fac9013b550 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fac88009e80 tx=0x7fac8800b330 comp rx=0 tx=0).stop 2023-10-03T10:14:43.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.208+0000 7fac973c0700 1 -- 172.21.15.3:0/115977475 shutdown_connections 2023-10-03T10:14:43.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.208+0000 7fac973c0700 1 --2- 172.21.15.3:0/115977475 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fac90139170 0x7fac9013b550 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.208+0000 7fac973c0700 1 --2- 172.21.15.3:0/115977475 >> v2:172.21.15.3:3300/0 conn(0x7fac9009e1d0 0x7fac90138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.208+0000 7fac973c0700 1 -- 172.21.15.3:0/115977475 >> 172.21.15.3:0/115977475 conn(0x7fac90066f50 msgr2=0x7fac9009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:43.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.208+0000 7fac973c0700 1 -- 172.21.15.3:0/115977475 shutdown_connections 2023-10-03T10:14:43.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.208+0000 7fac973c0700 1 -- 172.21.15.3:0/115977475 wait complete. 2023-10-03T10:14:43.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.209+0000 7fac973c0700 1 Processor -- start 2023-10-03T10:14:43.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.209+0000 7fac973c0700 1 -- start start 2023-10-03T10:14:43.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.210+0000 7fac973c0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fac9009d6f0 0x7fac90121740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:43.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.210+0000 7fac973c0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac9009e1d0 0x7fac90121c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:43.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.210+0000 7fac973c0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fac90139170 0x7fac901221c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:43.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.210+0000 7fac973c0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac90124260 con 0x7fac9009e1d0 2023-10-03T10:14:43.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.210+0000 7fac973c0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac90124f80 con 0x7fac9009d6f0 2023-10-03T10:14:43.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.210+0000 7fac973c0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fac900a3ab0 con 0x7fac90139170 2023-10-03T10:14:43.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.210+0000 7fac9495b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac9009e1d0 0x7fac90121c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:43.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.210+0000 7fac9515c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fac9009d6f0 0x7fac90121740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:43.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.210+0000 7fac9495b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac9009e1d0 0x7fac90121c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56046/0 (socket says 172.21.15.3:56046) 2023-10-03T10:14:43.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.210+0000 7fac9495b700 1 -- 172.21.15.3:0/3407444088 learned_addr learned my addr 172.21.15.3:0/3407444088 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:43.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.210+0000 7fac9595d700 1 --2- 172.21.15.3:0/3407444088 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fac90139170 0x7fac901221c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:43.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.210+0000 7fac9495b700 1 -- 172.21.15.3:0/3407444088 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fac90139170 msgr2=0x7fac901221c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:43.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.210+0000 7fac9495b700 1 --2- 172.21.15.3:0/3407444088 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fac90139170 0x7fac901221c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.210+0000 7fac9495b700 1 -- 172.21.15.3:0/3407444088 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fac9009d6f0 msgr2=0x7fac90121740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:43.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.210+0000 7fac9495b700 1 --2- 172.21.15.3:0/3407444088 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fac9009d6f0 0x7fac90121740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.210+0000 7fac9495b700 1 -- 172.21.15.3:0/3407444088 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac88009b30 con 0x7fac9009e1d0 2023-10-03T10:14:43.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.211+0000 7fac9495b700 1 --2- 172.21.15.3:0/3407444088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac9009e1d0 0x7fac90121c80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fac8000e9e0 tx=0x7fac8000ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:43.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.211+0000 7fac85ffb700 1 -- 172.21.15.3:0/3407444088 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac80009710 con 0x7fac9009e1d0 2023-10-03T10:14:43.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.211+0000 7fac85ffb700 1 -- 172.21.15.3:0/3407444088 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8000c850 con 0x7fac9009e1d0 2023-10-03T10:14:43.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.211+0000 7fac973c0700 1 -- 172.21.15.3:0/3407444088 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac9009c710 con 0x7fac9009e1d0 2023-10-03T10:14:43.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.211+0000 7fac85ffb700 1 -- 172.21.15.3:0/3407444088 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8000cbc0 con 0x7fac9009e1d0 2023-10-03T10:14:43.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.211+0000 7fac973c0700 1 -- 172.21.15.3:0/3407444088 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac90123bf0 con 0x7fac9009e1d0 2023-10-03T10:14:43.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.212+0000 7fac85ffb700 1 -- 172.21.15.3:0/3407444088 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fac8000cd20 con 0x7fac9009e1d0 2023-10-03T10:14:43.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.212+0000 7fac85ffb700 1 -- 172.21.15.3:0/3407444088 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fac8003fba0 con 0x7fac9009e1d0 2023-10-03T10:14:43.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.213+0000 7fac973c0700 1 -- 172.21.15.3:0/3407444088 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac901250c0 con 0x7fac9009e1d0 2023-10-03T10:14:43.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.216+0000 7f5f75b67700 1 Processor -- start 2023-10-03T10:14:43.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.216+0000 7f5f75b67700 1 -- start start 2023-10-03T10:14:43.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.217+0000 7f5f75b67700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5f7012cf60 0x7f5f7006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.217+0000 7f5f75b67700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f7012f8b0 0x7f5f7012fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.217+0000 7f5f75b67700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7012cd90 con 0x7f5f70130260 2023-10-03T10:14:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.217+0000 7f5f75b67700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f7012d330 con 0x7f5f7012cf60 2023-10-03T10:14:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.217+0000 7f5f75b67700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f701306d0 con 0x7f5f7012f8b0 2023-10-03T10:14:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.217+0000 7f5f6effd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f7012f8b0 0x7f5f7012fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.217+0000 7f5f6ffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5f70130260 0x7f5f70136c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54270/0 (socket says 172.21.15.3:54270) 2023-10-03T10:14:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.217+0000 7f5f6ffff700 1 -- 172.21.15.3:0/1744589047 learned_addr learned my addr 172.21.15.3:0/1744589047 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.217+0000 7f5f6f7fe700 1 --2- 172.21.15.3:0/1744589047 >> v2:172.21.15.3:3300/0 conn(0x7f5f7012cf60 0x7f5f7006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.217+0000 7f5f6effd700 1 -- 172.21.15.3:0/1744589047 >> v2:172.21.15.3:3300/0 conn(0x7f5f7012cf60 msgr2=0x7f5f7006f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.217+0000 7f5f6effd700 1 --2- 172.21.15.3:0/1744589047 >> v2:172.21.15.3:3300/0 conn(0x7f5f7012cf60 0x7f5f7006f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.217+0000 7f5f6effd700 1 -- 172.21.15.3:0/1744589047 >> v1:172.21.15.3:6789/0 conn(0x7f5f70130260 legacy=0x7f5f70136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.217+0000 7f5f6effd700 1 -- 172.21.15.3:0/1744589047 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f70071780 con 0x7f5f7012f8b0 2023-10-03T10:14:43.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.218+0000 7f5f6effd700 1 --2- 172.21.15.3:0/1744589047 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f7012f8b0 0x7f5f7012fd20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400a9e0 tx=0x7f5f6400acf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a060329f15b8b0a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:43.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.218+0000 7f5f6e7fc700 1 -- 172.21.15.3:0/1744589047 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6400cb70 con 0x7f5f7012f8b0 2023-10-03T10:14:43.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.218+0000 7f5f6e7fc700 1 -- 172.21.15.3:0/1744589047 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6400ccd0 con 0x7f5f7012f8b0 2023-10-03T10:14:43.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.218+0000 7f5f6e7fc700 1 -- 172.21.15.3:0/1744589047 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f64018620 con 0x7f5f7012f8b0 2023-10-03T10:14:43.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.218+0000 7fac85ffb700 1 -- 172.21.15.3:0/3407444088 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fac901250c0 con 0x7fac9009e1d0 2023-10-03T10:14:43.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.218+0000 7f5f75b67700 1 -- 172.21.15.3:0/1744589047 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f7012f8b0 msgr2=0x7f5f7012fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:43.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.219+0000 7f5f75b67700 1 --2- 172.21.15.3:0/1744589047 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f7012f8b0 0x7f5f7012fd20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400a9e0 tx=0x7f5f6400acf0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.219+0000 7f5f75b67700 1 -- 172.21.15.3:0/1744589047 shutdown_connections 2023-10-03T10:14:43.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.219+0000 7f5f75b67700 1 --2- 172.21.15.3:0/1744589047 >> v2:172.21.15.3:3300/0 conn(0x7f5f7012cf60 0x7f5f7006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.219+0000 7f5f75b67700 1 --2- 172.21.15.3:0/1744589047 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f7012f8b0 0x7f5f7012fd20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.219+0000 7f5f75b67700 1 -- 172.21.15.3:0/1744589047 >> 172.21.15.3:0/1744589047 conn(0x7f5f700a2aa0 msgr2=0x7f5f700a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:43.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.219+0000 7f5f75b67700 1 -- 172.21.15.3:0/1744589047 shutdown_connections 2023-10-03T10:14:43.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.219+0000 7f5f75b67700 1 -- 172.21.15.3:0/1744589047 wait complete. 2023-10-03T10:14:43.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.220+0000 7f5f75b67700 1 Processor -- start 2023-10-03T10:14:43.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.221+0000 7f5f75b67700 1 -- start start 2023-10-03T10:14:43.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.221+0000 7f5f75b67700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f7012cf60 0x7f5f7009bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:43.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.221+0000 7f5f75b67700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f7012f8b0 0x7f5f7009c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:43.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.221+0000 7f5f75b67700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f70130260 0x7f5f701c3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:43.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.221+0000 7f5f75b67700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7009edc0 con 0x7f5f7012cf60 2023-10-03T10:14:43.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.221+0000 7f5f75b67700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7009b6f0 con 0x7f5f7012f8b0 2023-10-03T10:14:43.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.221+0000 7f5f75b67700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7009b990 con 0x7f5f70130260 2023-10-03T10:14:43.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.221+0000 7f5f6f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f7012cf60 0x7f5f7009bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:43.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.221+0000 7f5f6ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f70130260 0x7f5f701c3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:43.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.221+0000 7f5f6f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f7012cf60 0x7f5f7009bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56064/0 (socket says 172.21.15.3:56064) 2023-10-03T10:14:43.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.221+0000 7f5f6f7fe700 1 -- 172.21.15.3:0/3483822208 learned_addr learned my addr 172.21.15.3:0/3483822208 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:43.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.222+0000 7f5f6f7fe700 1 -- 172.21.15.3:0/3483822208 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f70130260 msgr2=0x7f5f701c3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:43.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.222+0000 7f5f6f7fe700 1 --2- 172.21.15.3:0/3483822208 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f70130260 0x7f5f701c3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.222+0000 7f5f6f7fe700 1 -- 172.21.15.3:0/3483822208 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f7012f8b0 msgr2=0x7f5f7009c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:43.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.222+0000 7f5f6f7fe700 1 --2- 172.21.15.3:0/3483822208 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f7012f8b0 0x7f5f7009c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.222+0000 7f5f6f7fe700 1 -- 172.21.15.3:0/3483822208 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f6400a640 con 0x7f5f7012cf60 2023-10-03T10:14:43.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.222+0000 7f5f6f7fe700 1 --2- 172.21.15.3:0/3483822208 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f7012cf60 0x7f5f7009bc20 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000e9b0 tx=0x7f5f6000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:43.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.222+0000 7f5f57fff700 1 -- 172.21.15.3:0/3483822208 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60009710 con 0x7f5f7012cf60 2023-10-03T10:14:43.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.222+0000 7f5f57fff700 1 -- 172.21.15.3:0/3483822208 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6000c850 con 0x7f5f7012cf60 2023-10-03T10:14:43.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.222+0000 7f5f75b67700 1 -- 172.21.15.3:0/3483822208 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f70129fa0 con 0x7f5f7012cf60 2023-10-03T10:14:43.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.222+0000 7f5f57fff700 1 -- 172.21.15.3:0/3483822208 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6000cbc0 con 0x7f5f7012cf60 2023-10-03T10:14:43.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.222+0000 7f5f75b67700 1 -- 172.21.15.3:0/3483822208 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f7009cc80 con 0x7f5f7012cf60 2023-10-03T10:14:43.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.223+0000 7f5f57fff700 1 -- 172.21.15.3:0/3483822208 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5f6000cd20 con 0x7f5f7012cf60 2023-10-03T10:14:43.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.224+0000 7f5f75b67700 1 -- 172.21.15.3:0/3483822208 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f70074ea0 con 0x7f5f7012cf60 2023-10-03T10:14:43.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.224+0000 7f5f57fff700 1 -- 172.21.15.3:0/3483822208 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5f6003f880 con 0x7f5f7012cf60 2023-10-03T10:14:43.227 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:43.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.229+0000 7f5f57fff700 1 -- 172.21.15.3:0/3483822208 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5f6001be10 con 0x7f5f7012cf60 2023-10-03T10:14:43.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.378+0000 7f9e6047c700 1 Processor -- start 2023-10-03T10:14:43.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.378+0000 7f9e6047c700 1 -- start start 2023-10-03T10:14:43.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.379+0000 7f9e6047c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e5809e930 0x7f9e5809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:43.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.379+0000 7f9e6047c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e5809f650 0x7f9e58131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:43.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.379+0000 7f9e6047c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e58070940 con 0x7f9e5809f240 2023-10-03T10:14:43.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.379+0000 7f9e6047c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5806fd60 con 0x7f9e5809e930 2023-10-03T10:14:43.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.379+0000 7f9e6047c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5806fa80 con 0x7f9e5809f650 2023-10-03T10:14:43.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.379+0000 7f9e5da17700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9e5809f240 0x7f9e581317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54274/0 (socket says 172.21.15.3:54274) 2023-10-03T10:14:43.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.380+0000 7f9e5da17700 1 -- 172.21.15.3:0/3580315795 learned_addr learned my addr 172.21.15.3:0/3580315795 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:43.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.380+0000 7f9e5e218700 1 --2- 172.21.15.3:0/3580315795 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e5809e930 0x7f9e5809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:43.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.380+0000 7f9e5ea19700 1 --2- 172.21.15.3:0/3580315795 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e5809f650 0x7f9e58131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:43.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.380+0000 7f9e5d216700 1 -- 172.21.15.3:0/3580315795 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2570196350 0 0) 0x7f9e58070940 con 0x7f9e5809f240 2023-10-03T10:14:43.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.380+0000 7f9e5ea19700 1 -- 172.21.15.3:0/3580315795 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e5809e930 msgr2=0x7f9e5809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:43.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.380+0000 7f9e5ea19700 1 --2- 172.21.15.3:0/3580315795 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e5809e930 0x7f9e5809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.380+0000 7f9e5ea19700 1 -- 172.21.15.3:0/3580315795 >> v1:172.21.15.3:6789/0 conn(0x7f9e5809f240 legacy=0x7f9e581317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:43.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.380+0000 7f9e5ea19700 1 -- 172.21.15.3:0/3580315795 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5809fb60 con 0x7f9e5809f650 2023-10-03T10:14:43.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.380+0000 7f9e5ea19700 1 --2- 172.21.15.3:0/3580315795 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e5809f650 0x7f9e58131f00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009dc0 tx=0x7f9e5000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9ff771b7dcf5305 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:43.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.380+0000 7f9e5d216700 1 -- 172.21.15.3:0/3580315795 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5000e070 con 0x7f9e5809f650 2023-10-03T10:14:43.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.381+0000 7f9e5d216700 1 -- 172.21.15.3:0/3580315795 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5000bb20 con 0x7f9e5809f650 2023-10-03T10:14:43.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.381+0000 7f9e5d216700 1 -- 172.21.15.3:0/3580315795 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50012430 con 0x7f9e5809f650 2023-10-03T10:14:43.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.381+0000 7f9e6047c700 1 -- 172.21.15.3:0/3580315795 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e5809f650 msgr2=0x7f9e58131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:43.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.381+0000 7f9e6047c700 1 --2- 172.21.15.3:0/3580315795 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e5809f650 0x7f9e58131f00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009dc0 tx=0x7f9e5000b330 comp rx=0 tx=0).stop 2023-10-03T10:14:43.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.381+0000 7f9e6047c700 1 -- 172.21.15.3:0/3580315795 shutdown_connections 2023-10-03T10:14:43.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.381+0000 7f9e6047c700 1 --2- 172.21.15.3:0/3580315795 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e5809e930 0x7f9e5809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.381+0000 7f9e6047c700 1 --2- 172.21.15.3:0/3580315795 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e5809f650 0x7f9e58131f00 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.381+0000 7f9e6047c700 1 -- 172.21.15.3:0/3580315795 >> 172.21.15.3:0/3580315795 conn(0x7f9e580a4210 msgr2=0x7f9e580a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:43.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.381+0000 7f9e6047c700 1 -- 172.21.15.3:0/3580315795 shutdown_connections 2023-10-03T10:14:43.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.381+0000 7f9e6047c700 1 -- 172.21.15.3:0/3580315795 wait complete. 2023-10-03T10:14:43.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.382+0000 7f9e6047c700 1 Processor -- start 2023-10-03T10:14:43.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.383+0000 7f9e6047c700 1 -- start start 2023-10-03T10:14:43.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.383+0000 7f9e6047c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e5809e930 0x7f9e5806b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:43.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.383+0000 7f9e6047c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e5809f240 0x7f9e5806bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:43.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.383+0000 7f9e6047c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e5809f650 0x7f9e5806c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:43.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.383+0000 7f9e6047c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5806d4c0 con 0x7f9e5809f240 2023-10-03T10:14:43.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.383+0000 7f9e6047c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e581a92b0 con 0x7f9e5809e930 2023-10-03T10:14:43.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.383+0000 7f9e6047c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5806cd10 con 0x7f9e5809f650 2023-10-03T10:14:43.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.383+0000 7f9e5e218700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e5809e930 0x7f9e5806b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:43.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.383+0000 7f9e5e218700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e5809e930 0x7f9e5806b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58972/0 (socket says 172.21.15.3:58972) 2023-10-03T10:14:43.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.383+0000 7f9e5e218700 1 -- 172.21.15.3:0/1709609505 learned_addr learned my addr 172.21.15.3:0/1709609505 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:43.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.383+0000 7f9e5ea19700 1 --2- 172.21.15.3:0/1709609505 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e5809f650 0x7f9e5806c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:43.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.383+0000 7f9e5da17700 1 --2- 172.21.15.3:0/1709609505 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e5809f240 0x7f9e5806bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:43.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.384+0000 7f9e5e218700 1 -- 172.21.15.3:0/1709609505 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e5809f650 msgr2=0x7f9e5806c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:43.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.384+0000 7f9e5e218700 1 --2- 172.21.15.3:0/1709609505 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e5809f650 0x7f9e5806c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.384+0000 7f9e5e218700 1 -- 172.21.15.3:0/1709609505 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e5809f240 msgr2=0x7f9e5806bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:43.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.384+0000 7f9e5e218700 1 --2- 172.21.15.3:0/1709609505 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e5809f240 0x7f9e5806bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.384+0000 7f9e5e218700 1 -- 172.21.15.3:0/1709609505 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e50009a70 con 0x7f9e5809e930 2023-10-03T10:14:43.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.384+0000 7f9e5da17700 1 --2- 172.21.15.3:0/1709609505 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e5809f240 0x7f9e5806bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:43.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.384+0000 7f9e5e218700 1 --2- 172.21.15.3:0/1709609505 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e5809e930 0x7f9e5806b8b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9e5400e9e0 tx=0x7f9e5400ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:43.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.384+0000 7f9e4effd700 1 -- 172.21.15.3:0/1709609505 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e54009710 con 0x7f9e5809e930 2023-10-03T10:14:43.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.384+0000 7f9e4effd700 1 -- 172.21.15.3:0/1709609505 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5400ee80 con 0x7f9e5809e930 2023-10-03T10:14:43.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.384+0000 7f9e6047c700 1 -- 172.21.15.3:0/1709609505 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e580a0190 con 0x7f9e5809e930 2023-10-03T10:14:43.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.384+0000 7f9e6047c700 1 -- 172.21.15.3:0/1709609505 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e581a8fe0 con 0x7f9e5809e930 2023-10-03T10:14:43.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.384+0000 7f9e4effd700 1 -- 172.21.15.3:0/1709609505 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5400ca50 con 0x7f9e5809e930 2023-10-03T10:14:43.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.386+0000 7f9e6047c700 1 -- 172.21.15.3:0/1709609505 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e400052e0 con 0x7f9e5809e930 2023-10-03T10:14:43.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.386+0000 7f9e4effd700 1 -- 172.21.15.3:0/1709609505 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9e5400cbb0 con 0x7f9e5809e930 2023-10-03T10:14:43.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.386+0000 7f9e4effd700 1 -- 172.21.15.3:0/1709609505 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9e5403fa50 con 0x7f9e5809e930 2023-10-03T10:14:43.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.395+0000 7f9e4effd700 1 -- 172.21.15.3:0/1709609505 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9e5401b3b0 con 0x7f9e5809e930 2023-10-03T10:14:43.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.412+0000 7fc0011db700 1 -- 172.21.15.3:0/1559043129 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbffc11e140 con 0x7fbffc136710 2023-10-03T10:14:43.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.413+0000 7fbfe37fe700 1 -- 172.21.15.3:0/1559043129 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbffc11e140 con 0x7fbffc136710 2023-10-03T10:14:43.413 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:43.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.415+0000 7fc0011db700 1 -- 172.21.15.3:0/1559043129 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbffc136710 msgr2=0x7fbffc11b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:43.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.415+0000 7fc0011db700 1 --2- 172.21.15.3:0/1559043129 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbffc136710 0x7fbffc11b1a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbff0009c40 tx=0x7fbff0004750 comp rx=0 tx=0).stop 2023-10-03T10:14:43.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.415+0000 7fc0011db700 1 -- 172.21.15.3:0/1559043129 shutdown_connections 2023-10-03T10:14:43.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.415+0000 7fc0011db700 1 --2- 172.21.15.3:0/1559043129 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbffc139030 0x7fbffc11b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.415+0000 7fc0011db700 1 --2- 172.21.15.3:0/1559043129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbffc1338a0 0x7fbffc11ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.415+0000 7fc0011db700 1 --2- 172.21.15.3:0/1559043129 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbffc136710 0x7fbffc11b1a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.415+0000 7fc0011db700 1 -- 172.21.15.3:0/1559043129 >> 172.21.15.3:0/1559043129 conn(0x7fbffc0a2aa0 msgr2=0x7fbffc1344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:43.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.415+0000 7fc0011db700 1 -- 172.21.15.3:0/1559043129 shutdown_connections 2023-10-03T10:14:43.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.415+0000 7fc0011db700 1 -- 172.21.15.3:0/1559043129 wait complete. 2023-10-03T10:14:43.430 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:43.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.438+0000 7f5f75b67700 1 -- 172.21.15.3:0/3483822208 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5f700713a0 con 0x7f5f7012cf60 2023-10-03T10:14:43.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.438+0000 7f5f57fff700 1 -- 172.21.15.3:0/3483822208 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5f6002a090 con 0x7f5f7012cf60 2023-10-03T10:14:43.439 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.441+0000 7f5f75b67700 1 -- 172.21.15.3:0/3483822208 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f7012cf60 msgr2=0x7f5f7009bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.441+0000 7f5f75b67700 1 --2- 172.21.15.3:0/3483822208 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f7012cf60 0x7f5f7009bc20 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000e9b0 tx=0x7f5f6000ecc0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.441+0000 7f5f75b67700 1 -- 172.21.15.3:0/3483822208 shutdown_connections 2023-10-03T10:14:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.441+0000 7f5f75b67700 1 --2- 172.21.15.3:0/3483822208 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f70130260 0x7f5f701c3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.441+0000 7f5f75b67700 1 --2- 172.21.15.3:0/3483822208 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f7012cf60 0x7f5f7009bc20 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.441+0000 7f5f75b67700 1 --2- 172.21.15.3:0/3483822208 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f7012f8b0 0x7f5f7009c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.441+0000 7f5f75b67700 1 -- 172.21.15.3:0/3483822208 >> 172.21.15.3:0/3483822208 conn(0x7f5f700a2aa0 msgr2=0x7f5f7006e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:43.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.441+0000 7f5f75b67700 1 -- 172.21.15.3:0/3483822208 shutdown_connections 2023-10-03T10:14:43.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.441+0000 7f5f75b67700 1 -- 172.21.15.3:0/3483822208 wait complete. 2023-10-03T10:14:43.455 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:43.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.460+0000 7fac973c0700 1 -- 172.21.15.3:0/3407444088 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fac901250c0 con 0x7fac9009e1d0 2023-10-03T10:14:43.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.460+0000 7fac85ffb700 1 -- 172.21.15.3:0/3407444088 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fac901250c0 con 0x7fac9009e1d0 2023-10-03T10:14:43.461 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:43.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.463+0000 7fac973c0700 1 -- 172.21.15.3:0/3407444088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac9009e1d0 msgr2=0x7fac90121c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:43.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.463+0000 7fac973c0700 1 --2- 172.21.15.3:0/3407444088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac9009e1d0 0x7fac90121c80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fac8000e9e0 tx=0x7fac8000ecf0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.463+0000 7fac973c0700 1 -- 172.21.15.3:0/3407444088 shutdown_connections 2023-10-03T10:14:43.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.463+0000 7fac973c0700 1 --2- 172.21.15.3:0/3407444088 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fac90139170 0x7fac901221c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.463+0000 7fac973c0700 1 --2- 172.21.15.3:0/3407444088 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fac9009d6f0 0x7fac90121740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.463+0000 7fac973c0700 1 --2- 172.21.15.3:0/3407444088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac9009e1d0 0x7fac90121c80 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.463+0000 7fac973c0700 1 -- 172.21.15.3:0/3407444088 >> 172.21.15.3:0/3407444088 conn(0x7fac90066f50 msgr2=0x7fac90070c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:43.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.463+0000 7fac973c0700 1 -- 172.21.15.3:0/3407444088 shutdown_connections 2023-10-03T10:14:43.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.463+0000 7fac973c0700 1 -- 172.21.15.3:0/3407444088 wait complete. 2023-10-03T10:14:43.478 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:43.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.616+0000 7f9e6047c700 1 -- 172.21.15.3:0/1709609505 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9e40005150 con 0x7f9e5809e930 2023-10-03T10:14:43.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.616+0000 7f9e4effd700 1 -- 172.21.15.3:0/1709609505 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9e54021070 con 0x7f9e5809e930 2023-10-03T10:14:43.616 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:43.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.618+0000 7f9e6047c700 1 -- 172.21.15.3:0/1709609505 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e5809e930 msgr2=0x7f9e5806b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:43.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.618+0000 7f9e6047c700 1 --2- 172.21.15.3:0/1709609505 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e5809e930 0x7f9e5806b8b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9e5400e9e0 tx=0x7f9e5400ecf0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.618+0000 7f9e6047c700 1 -- 172.21.15.3:0/1709609505 shutdown_connections 2023-10-03T10:14:43.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.618+0000 7f9e6047c700 1 --2- 172.21.15.3:0/1709609505 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e5809f650 0x7f9e5806c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.618+0000 7f9e6047c700 1 --2- 172.21.15.3:0/1709609505 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e5809e930 0x7f9e5806b8b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.618+0000 7f9e6047c700 1 --2- 172.21.15.3:0/1709609505 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e5809f240 0x7f9e5806bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:43.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.618+0000 7f9e6047c700 1 -- 172.21.15.3:0/1709609505 >> 172.21.15.3:0/1709609505 conn(0x7f9e580a4210 msgr2=0x7f9e581b8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:43.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.618+0000 7f9e6047c700 1 -- 172.21.15.3:0/1709609505 shutdown_connections 2023-10-03T10:14:43.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:43.618+0000 7f9e6047c700 1 -- 172.21.15.3:0/1709609505 wait complete. 2023-10-03T10:14:43.632 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:44.431 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:44.457 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:44.479 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:44.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.581+0000 7fa44ea60700 1 Processor -- start 2023-10-03T10:14:44.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.581+0000 7fa44ea60700 1 -- start start 2023-10-03T10:14:44.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.581+0000 7fa44ea60700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa4480a0a30 0x7fa4480a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:44.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.581+0000 7fa44ea60700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa4480a1340 0x7fa448133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:44.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.581+0000 7fa44ea60700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa448133fa0 0x7fa448134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:44.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.581+0000 7fa44ea60700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa44809edd0 con 0x7fa4480a0a30 2023-10-03T10:14:44.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.581+0000 7fa44ea60700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa44809e1f0 con 0x7fa448133fa0 2023-10-03T10:14:44.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.581+0000 7fa44ea60700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa44809dc50 con 0x7fa4480a1340 2023-10-03T10:14:44.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.582+0000 7fa4477fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa4480a1340 0x7fa448133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:44.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.582+0000 7fa44cffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa448133fa0 0x7fa448134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:44.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.582+0000 7fa4477fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa4480a1340 0x7fa448133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57302/0 (socket says 172.21.15.3:57302) 2023-10-03T10:14:44.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.582+0000 7fa4477fe700 1 -- 172.21.15.3:0/3216053308 learned_addr learned my addr 172.21.15.3:0/3216053308 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:44.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.582+0000 7fa447fff700 1 --2- 172.21.15.3:0/3216053308 >> v2:172.21.15.3:3300/0 conn(0x7fa4480a0a30 0x7fa4480a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:44.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.582+0000 7fa44cffd700 1 -- 172.21.15.3:0/3216053308 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa4480a1340 msgr2=0x7fa448133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:44.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.582+0000 7fa44cffd700 1 --2- 172.21.15.3:0/3216053308 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa4480a1340 0x7fa448133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.582+0000 7fa44cffd700 1 -- 172.21.15.3:0/3216053308 >> v2:172.21.15.3:3300/0 conn(0x7fa4480a0a30 msgr2=0x7fa4480a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:44.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.582+0000 7fa44cffd700 1 --2- 172.21.15.3:0/3216053308 >> v2:172.21.15.3:3300/0 conn(0x7fa4480a0a30 0x7fa4480a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.582+0000 7fa44cffd700 1 -- 172.21.15.3:0/3216053308 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4480a1fe0 con 0x7fa448133fa0 2023-10-03T10:14:44.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.582+0000 7fa4477fe700 1 --2- 172.21.15.3:0/3216053308 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa4480a1340 0x7fa448133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:44.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.582+0000 7fa447fff700 1 --2- 172.21.15.3:0/3216053308 >> v2:172.21.15.3:3300/0 conn(0x7fa4480a0a30 0x7fa4480a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:44.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.582+0000 7fa44cffd700 1 --2- 172.21.15.3:0/3216053308 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa448133fa0 0x7fa448134410 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa440009dc0 tx=0x7fa44000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43bca93a9f23267d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:44.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.582+0000 7fa446ffd700 1 -- 172.21.15.3:0/3216053308 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44000e070 con 0x7fa448133fa0 2023-10-03T10:14:44.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.582+0000 7fa446ffd700 1 -- 172.21.15.3:0/3216053308 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44000bae0 con 0x7fa448133fa0 2023-10-03T10:14:44.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.582+0000 7fa446ffd700 1 -- 172.21.15.3:0/3216053308 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44000be90 con 0x7fa448133fa0 2023-10-03T10:14:44.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.583+0000 7fa44ea60700 1 -- 172.21.15.3:0/3216053308 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa448133fa0 msgr2=0x7fa448134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:44.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.583+0000 7fa44ea60700 1 --2- 172.21.15.3:0/3216053308 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa448133fa0 0x7fa448134410 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa440009dc0 tx=0x7fa44000b330 comp rx=0 tx=0).stop 2023-10-03T10:14:44.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.583+0000 7fa44ea60700 1 -- 172.21.15.3:0/3216053308 shutdown_connections 2023-10-03T10:14:44.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.583+0000 7fa44ea60700 1 --2- 172.21.15.3:0/3216053308 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa448133fa0 0x7fa448134410 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.583+0000 7fa44ea60700 1 --2- 172.21.15.3:0/3216053308 >> v2:172.21.15.3:3300/0 conn(0x7fa4480a0a30 0x7fa4480a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.583+0000 7fa44ea60700 1 --2- 172.21.15.3:0/3216053308 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa4480a1340 0x7fa448133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.583+0000 7fa44ea60700 1 -- 172.21.15.3:0/3216053308 >> 172.21.15.3:0/3216053308 conn(0x7fa4480a6550 msgr2=0x7fa4480a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:44.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.583+0000 7fa44ea60700 1 -- 172.21.15.3:0/3216053308 shutdown_connections 2023-10-03T10:14:44.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.583+0000 7fa44ea60700 1 -- 172.21.15.3:0/3216053308 wait complete. 2023-10-03T10:14:44.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.585+0000 7fa44ea60700 1 Processor -- start 2023-10-03T10:14:44.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.585+0000 7fa44ea60700 1 -- start start 2023-10-03T10:14:44.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.586+0000 7fa44ea60700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa4480a0a30 0x7fa4481200e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:44.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.586+0000 7fa44ea60700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4480a1340 0x7fa448120620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:44.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.586+0000 7fa44ea60700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa448133fa0 0x7fa448120b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:44.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.586+0000 7fa44ea60700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa448124530 con 0x7fa4480a1340 2023-10-03T10:14:44.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.586+0000 7fa44ea60700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa448121d00 con 0x7fa4480a0a30 2023-10-03T10:14:44.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.586+0000 7fa44ea60700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa448123890 con 0x7fa448133fa0 2023-10-03T10:14:44.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.586+0000 7fa447fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa4480a0a30 0x7fa4481200e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:44.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.586+0000 7fa447fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa4480a0a30 0x7fa4481200e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58982/0 (socket says 172.21.15.3:58982) 2023-10-03T10:14:44.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.586+0000 7fa447fff700 1 -- 172.21.15.3:0/878896181 learned_addr learned my addr 172.21.15.3:0/878896181 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:44.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.586+0000 7fa44cffd700 1 --2- 172.21.15.3:0/878896181 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa448133fa0 0x7fa448120b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:44.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.586+0000 7fa4477fe700 1 --2- 172.21.15.3:0/878896181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4480a1340 0x7fa448120620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:44.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.586+0000 7fa447fff700 1 -- 172.21.15.3:0/878896181 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa448133fa0 msgr2=0x7fa448120b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:44.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.586+0000 7fa447fff700 1 --2- 172.21.15.3:0/878896181 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa448133fa0 0x7fa448120b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.586+0000 7fa447fff700 1 -- 172.21.15.3:0/878896181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4480a1340 msgr2=0x7fa448120620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:44.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.586+0000 7fa447fff700 1 --2- 172.21.15.3:0/878896181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4480a1340 0x7fa448120620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.586+0000 7fa447fff700 1 -- 172.21.15.3:0/878896181 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4380094a0 con 0x7fa4480a0a30 2023-10-03T10:14:44.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.586+0000 7fa447fff700 1 --2- 172.21.15.3:0/878896181 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa4480a0a30 0x7fa4481200e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa43800ec10 tx=0x7fa43800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:44.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.587+0000 7fa444ff9700 1 -- 172.21.15.3:0/878896181 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43800c850 con 0x7fa4480a0a30 2023-10-03T10:14:44.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.587+0000 7fa444ff9700 1 -- 172.21.15.3:0/878896181 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa43800c9b0 con 0x7fa4480a0a30 2023-10-03T10:14:44.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.587+0000 7fa444ff9700 1 -- 172.21.15.3:0/878896181 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43800cd20 con 0x7fa4480a0a30 2023-10-03T10:14:44.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.587+0000 7fa44ea60700 1 -- 172.21.15.3:0/878896181 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa43c009550 con 0x7fa4480a0a30 2023-10-03T10:14:44.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.587+0000 7fa44ea60700 1 -- 172.21.15.3:0/878896181 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa440009a70 con 0x7fa4480a0a30 2023-10-03T10:14:44.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.588+0000 7fa444ff9700 1 -- 172.21.15.3:0/878896181 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa438013470 con 0x7fa4480a0a30 2023-10-03T10:14:44.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.588+0000 7fa444ff9700 1 -- 172.21.15.3:0/878896181 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa43803fb70 con 0x7fa4480a0a30 2023-10-03T10:14:44.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.588+0000 7fa44ea60700 1 -- 172.21.15.3:0/878896181 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4480a0220 con 0x7fa4480a0a30 2023-10-03T10:14:44.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.597+0000 7fa444ff9700 1 -- 172.21.15.3:0/878896181 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa438013720 con 0x7fa4480a0a30 2023-10-03T10:14:44.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.622+0000 7fdf2b0c3700 1 Processor -- start 2023-10-03T10:14:44.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.622+0000 7fdf2b0c3700 1 -- start start 2023-10-03T10:14:44.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.622+0000 7fdf2b0c3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf240a0a30 0x7fdf240a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:44.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.622+0000 7fdf2b0c3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf240a1340 0x7fdf24133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:44.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.622+0000 7fdf2b0c3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdf24133fa0 0x7fdf24134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:44.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.622+0000 7fdf2b0c3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf2409edd0 con 0x7fdf24133fa0 2023-10-03T10:14:44.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.622+0000 7fdf2b0c3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2409e1f0 con 0x7fdf240a0a30 2023-10-03T10:14:44.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.622+0000 7fdf2b0c3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2409dc50 con 0x7fdf240a1340 2023-10-03T10:14:44.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.623+0000 7fdf23fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf240a1340 0x7fdf24133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:44.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.623+0000 7fdf29660700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdf24133fa0 0x7fdf24134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:44.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.623+0000 7fdf29660700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdf24133fa0 0x7fdf24134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56104/0 (socket says 172.21.15.3:56104) 2023-10-03T10:14:44.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.623+0000 7fdf23fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf240a1340 0x7fdf24133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57322/0 (socket says 172.21.15.3:57322) 2023-10-03T10:14:44.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.623+0000 7fdf29660700 1 -- 172.21.15.3:0/2040324356 learned_addr learned my addr 172.21.15.3:0/2040324356 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:44.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.623+0000 7fdf28e5f700 1 --2- 172.21.15.3:0/2040324356 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf240a0a30 0x7fdf240a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:44.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.623+0000 7fdf29660700 1 -- 172.21.15.3:0/2040324356 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf240a1340 msgr2=0x7fdf24133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:44.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.623+0000 7fdf29660700 1 --2- 172.21.15.3:0/2040324356 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf240a1340 0x7fdf24133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.623+0000 7fdf29660700 1 -- 172.21.15.3:0/2040324356 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf240a0a30 msgr2=0x7fdf240a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:44.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.623+0000 7fdf29660700 1 --2- 172.21.15.3:0/2040324356 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf240a0a30 0x7fdf240a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.623+0000 7fdf29660700 1 -- 172.21.15.3:0/2040324356 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf240a1fe0 con 0x7fdf24133fa0 2023-10-03T10:14:44.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.623+0000 7fdf23fff700 1 --2- 172.21.15.3:0/2040324356 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf240a1340 0x7fdf24133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:44.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.623+0000 7fdf29660700 1 --2- 172.21.15.3:0/2040324356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf24133fa0 0x7fdf24134410 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009e50 tx=0x7fdf1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6450600c209ab91 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:44.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.623+0000 7fdf237fe700 1 -- 172.21.15.3:0/2040324356 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c00e070 con 0x7fdf24133fa0 2023-10-03T10:14:44.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.624+0000 7fdf237fe700 1 -- 172.21.15.3:0/2040324356 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1c00b910 con 0x7fdf24133fa0 2023-10-03T10:14:44.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.624+0000 7fdf237fe700 1 -- 172.21.15.3:0/2040324356 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c00bcc0 con 0x7fdf24133fa0 2023-10-03T10:14:44.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.624+0000 7fdf2b0c3700 1 -- 172.21.15.3:0/2040324356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf24133fa0 msgr2=0x7fdf24134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:44.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.624+0000 7fdf2b0c3700 1 --2- 172.21.15.3:0/2040324356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf24133fa0 0x7fdf24134410 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009e50 tx=0x7fdf1c00b330 comp rx=0 tx=0).stop 2023-10-03T10:14:44.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.624+0000 7fdf2b0c3700 1 -- 172.21.15.3:0/2040324356 shutdown_connections 2023-10-03T10:14:44.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.624+0000 7fdf2b0c3700 1 --2- 172.21.15.3:0/2040324356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf24133fa0 0x7fdf24134410 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.624+0000 7fdf2b0c3700 1 --2- 172.21.15.3:0/2040324356 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf240a0a30 0x7fdf240a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.624+0000 7fdf2b0c3700 1 --2- 172.21.15.3:0/2040324356 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf240a1340 0x7fdf24133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.624+0000 7fdf2b0c3700 1 -- 172.21.15.3:0/2040324356 >> 172.21.15.3:0/2040324356 conn(0x7fdf240a6550 msgr2=0x7fdf240a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:44.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.624+0000 7fdf2b0c3700 1 -- 172.21.15.3:0/2040324356 shutdown_connections 2023-10-03T10:14:44.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.624+0000 7fdf2b0c3700 1 -- 172.21.15.3:0/2040324356 wait complete. 2023-10-03T10:14:44.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.625+0000 7fdf2b0c3700 1 Processor -- start 2023-10-03T10:14:44.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.626+0000 7fdf2b0c3700 1 -- start start 2023-10-03T10:14:44.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.626+0000 7fdf2b0c3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf240a0a30 0x7fdf24120030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:44.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.626+0000 7fdf2b0c3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf240a1340 0x7fdf24120570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:44.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.626+0000 7fdf2b0c3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf24133fa0 0x7fdf24120ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:44.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.626+0000 7fdf2b0c3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf241235b0 con 0x7fdf240a1340 2023-10-03T10:14:44.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.626+0000 7fdf2b0c3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf24121d50 con 0x7fdf240a0a30 2023-10-03T10:14:44.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.626+0000 7fdf2b0c3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf240a0220 con 0x7fdf24133fa0 2023-10-03T10:14:44.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.626+0000 7fdf28e5f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf240a0a30 0x7fdf24120030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:44.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.626+0000 7fdf28e5f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf240a0a30 0x7fdf24120030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59000/0 (socket says 172.21.15.3:59000) 2023-10-03T10:14:44.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.626+0000 7fdf29660700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf24133fa0 0x7fdf24120ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:44.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.626+0000 7fdf28e5f700 1 -- 172.21.15.3:0/1855687317 learned_addr learned my addr 172.21.15.3:0/1855687317 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:44.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.627+0000 7fdf23fff700 1 --2- 172.21.15.3:0/1855687317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf240a1340 0x7fdf24120570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:44.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.627+0000 7fdf28e5f700 1 -- 172.21.15.3:0/1855687317 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf24133fa0 msgr2=0x7fdf24120ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:44.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.627+0000 7fdf28e5f700 1 --2- 172.21.15.3:0/1855687317 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf24133fa0 0x7fdf24120ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.627+0000 7fdf28e5f700 1 -- 172.21.15.3:0/1855687317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf240a1340 msgr2=0x7fdf24120570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:44.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.627+0000 7fdf28e5f700 1 --2- 172.21.15.3:0/1855687317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf240a1340 0x7fdf24120570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.627+0000 7fdf28e5f700 1 -- 172.21.15.3:0/1855687317 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf100094a0 con 0x7fdf240a0a30 2023-10-03T10:14:44.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.627+0000 7fdf28e5f700 1 --2- 172.21.15.3:0/1855687317 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf240a0a30 0x7fdf24120030 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800e9b0 tx=0x7fdf1800ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:44.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.627+0000 7fdf217fa700 1 -- 172.21.15.3:0/1855687317 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf18009710 con 0x7fdf240a0a30 2023-10-03T10:14:44.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.627+0000 7fdf217fa700 1 -- 172.21.15.3:0/1855687317 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1800c850 con 0x7fdf240a0a30 2023-10-03T10:14:44.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.627+0000 7fdf2b0c3700 1 -- 172.21.15.3:0/1855687317 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf1c009b30 con 0x7fdf240a0a30 2023-10-03T10:14:44.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.627+0000 7fdf217fa700 1 -- 172.21.15.3:0/1855687317 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1800cbc0 con 0x7fdf240a0a30 2023-10-03T10:14:44.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.627+0000 7fdf2b0c3700 1 -- 172.21.15.3:0/1855687317 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf24122a40 con 0x7fdf240a0a30 2023-10-03T10:14:44.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.628+0000 7fdf217fa700 1 -- 172.21.15.3:0/1855687317 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdf1800cd20 con 0x7fdf240a0a30 2023-10-03T10:14:44.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.629+0000 7fdf2b0c3700 1 -- 172.21.15.3:0/1855687317 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf24074ea0 con 0x7fdf240a0a30 2023-10-03T10:14:44.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.629+0000 7fdf217fa700 1 -- 172.21.15.3:0/1855687317 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fdf1803fba0 con 0x7fdf240a0a30 2023-10-03T10:14:44.633 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:44.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.634+0000 7fdf217fa700 1 -- 172.21.15.3:0/1855687317 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdf1801adc0 con 0x7fdf240a0a30 2023-10-03T10:14:44.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.676+0000 7ffaee25c700 1 Processor -- start 2023-10-03T10:14:44.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.677+0000 7ffaee25c700 1 -- start start 2023-10-03T10:14:44.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.677+0000 7ffaee25c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffae80a0a30 0x7ffae80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:44.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.677+0000 7ffaee25c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffae80a1340 0x7ffae8133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:44.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.677+0000 7ffaee25c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffae8133fa0 0x7ffae8134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:44.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.677+0000 7ffaee25c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffae809edd0 con 0x7ffae80a1340 2023-10-03T10:14:44.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.677+0000 7ffaee25c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae809e1f0 con 0x7ffae80a0a30 2023-10-03T10:14:44.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.677+0000 7ffaee25c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffae809dc50 con 0x7ffae8133fa0 2023-10-03T10:14:44.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.678+0000 7ffae7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffae8133fa0 0x7ffae8134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:44.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.678+0000 7ffae6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffae80a1340 0x7ffae8133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:44.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.678+0000 7ffae7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffae8133fa0 0x7ffae8134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57338/0 (socket says 172.21.15.3:57338) 2023-10-03T10:14:44.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.678+0000 7ffae6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffae80a1340 0x7ffae8133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56124/0 (socket says 172.21.15.3:56124) 2023-10-03T10:14:44.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.678+0000 7ffae77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffae80a0a30 0x7ffae80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:44.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.678+0000 7ffae7fff700 1 -- 172.21.15.3:0/557990560 learned_addr learned my addr 172.21.15.3:0/557990560 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:44.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.678+0000 7ffae7fff700 1 -- 172.21.15.3:0/557990560 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffae80a0a30 msgr2=0x7ffae80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:44.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.678+0000 7ffae7fff700 1 --2- 172.21.15.3:0/557990560 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffae80a0a30 0x7ffae80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.678+0000 7ffae7fff700 1 -- 172.21.15.3:0/557990560 >> v2:172.21.15.3:3300/0 conn(0x7ffae80a1340 msgr2=0x7ffae8133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:44.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.678+0000 7ffae7fff700 1 --2- 172.21.15.3:0/557990560 >> v2:172.21.15.3:3300/0 conn(0x7ffae80a1340 0x7ffae8133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.678+0000 7ffae7fff700 1 -- 172.21.15.3:0/557990560 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae80a1fe0 con 0x7ffae8133fa0 2023-10-03T10:14:44.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.678+0000 7ffae6ffd700 1 --2- 172.21.15.3:0/557990560 >> v2:172.21.15.3:3300/0 conn(0x7ffae80a1340 0x7ffae8133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:44.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.678+0000 7ffae7fff700 1 --2- 172.21.15.3:0/557990560 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffae8133fa0 0x7ffae8134410 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ffae0009e80 tx=0x7ffae000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71b343e899935794 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:44.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.679+0000 7ffae67fc700 1 -- 172.21.15.3:0/557990560 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae000e070 con 0x7ffae8133fa0 2023-10-03T10:14:44.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.679+0000 7ffae67fc700 1 -- 172.21.15.3:0/557990560 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae000b8d0 con 0x7ffae8133fa0 2023-10-03T10:14:44.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.679+0000 7ffae67fc700 1 -- 172.21.15.3:0/557990560 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae000bc80 con 0x7ffae8133fa0 2023-10-03T10:14:44.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.679+0000 7ffaee25c700 1 -- 172.21.15.3:0/557990560 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffae8133fa0 msgr2=0x7ffae8134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:44.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.679+0000 7ffaee25c700 1 --2- 172.21.15.3:0/557990560 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffae8133fa0 0x7ffae8134410 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ffae0009e80 tx=0x7ffae000b330 comp rx=0 tx=0).stop 2023-10-03T10:14:44.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.680+0000 7ffaee25c700 1 -- 172.21.15.3:0/557990560 shutdown_connections 2023-10-03T10:14:44.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.680+0000 7ffaee25c700 1 --2- 172.21.15.3:0/557990560 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffae8133fa0 0x7ffae8134410 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.680+0000 7ffaee25c700 1 --2- 172.21.15.3:0/557990560 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffae80a0a30 0x7ffae80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.680+0000 7ffaee25c700 1 --2- 172.21.15.3:0/557990560 >> v2:172.21.15.3:3300/0 conn(0x7ffae80a1340 0x7ffae8133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.680+0000 7ffaee25c700 1 -- 172.21.15.3:0/557990560 >> 172.21.15.3:0/557990560 conn(0x7ffae80a6550 msgr2=0x7ffae80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:44.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.680+0000 7ffaee25c700 1 -- 172.21.15.3:0/557990560 shutdown_connections 2023-10-03T10:14:44.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.680+0000 7ffaee25c700 1 -- 172.21.15.3:0/557990560 wait complete. 2023-10-03T10:14:44.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.681+0000 7ffaee25c700 1 Processor -- start 2023-10-03T10:14:44.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.681+0000 7ffaee25c700 1 -- start start 2023-10-03T10:14:44.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.681+0000 7ffaee25c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffae80a0a30 0x7ffae81200e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:44.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.681+0000 7ffaee25c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffae80a1340 0x7ffae8120620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:44.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.681+0000 7ffaee25c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffae8133fa0 0x7ffae8120b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:44.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.682+0000 7ffaee25c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae8121d90 con 0x7ffae8133fa0 2023-10-03T10:14:44.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.682+0000 7ffaee25c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae8123920 con 0x7ffae80a1340 2023-10-03T10:14:44.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.682+0000 7ffaee25c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffae80a6bb0 con 0x7ffae80a0a30 2023-10-03T10:14:44.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.682+0000 7ffae77fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffae80a0a30 0x7ffae81200e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:44.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.682+0000 7ffae77fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffae80a0a30 0x7ffae81200e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57354/0 (socket says 172.21.15.3:57354) 2023-10-03T10:14:44.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.682+0000 7ffae77fe700 1 -- 172.21.15.3:0/2394404976 learned_addr learned my addr 172.21.15.3:0/2394404976 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:44.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.682+0000 7ffae6ffd700 1 --2- 172.21.15.3:0/2394404976 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffae80a1340 0x7ffae8120620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:44.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.682+0000 7ffae7fff700 1 --2- 172.21.15.3:0/2394404976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffae8133fa0 0x7ffae8120b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:44.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.682+0000 7ffae7fff700 1 -- 172.21.15.3:0/2394404976 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffae80a0a30 msgr2=0x7ffae81200e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:44.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.682+0000 7ffae7fff700 1 --2- 172.21.15.3:0/2394404976 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffae80a0a30 0x7ffae81200e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.682+0000 7ffae7fff700 1 -- 172.21.15.3:0/2394404976 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffae80a1340 msgr2=0x7ffae8120620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:44.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.682+0000 7ffae7fff700 1 --2- 172.21.15.3:0/2394404976 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffae80a1340 0x7ffae8120620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.682+0000 7ffae7fff700 1 -- 172.21.15.3:0/2394404976 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffadc0094a0 con 0x7ffae8133fa0 2023-10-03T10:14:44.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.682+0000 7ffae6ffd700 1 --2- 172.21.15.3:0/2394404976 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffae80a1340 0x7ffae8120620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:44.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.683+0000 7ffae7fff700 1 --2- 172.21.15.3:0/2394404976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffae8133fa0 0x7ffae8120b60 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ffae0015000 tx=0x7ffae0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:44.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.683+0000 7ffacffff700 1 -- 172.21.15.3:0/2394404976 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae0004450 con 0x7ffae8133fa0 2023-10-03T10:14:44.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.683+0000 7ffacffff700 1 -- 172.21.15.3:0/2394404976 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae00045f0 con 0x7ffae8133fa0 2023-10-03T10:14:44.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.683+0000 7ffaee25c700 1 -- 172.21.15.3:0/2394404976 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae0009b30 con 0x7ffae8133fa0 2023-10-03T10:14:44.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.683+0000 7ffacffff700 1 -- 172.21.15.3:0/2394404976 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae00099b0 con 0x7ffae8133fa0 2023-10-03T10:14:44.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.683+0000 7ffaee25c700 1 -- 172.21.15.3:0/2394404976 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae8122f40 con 0x7ffae8133fa0 2023-10-03T10:14:44.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.684+0000 7ffacffff700 1 -- 172.21.15.3:0/2394404976 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffae001c410 con 0x7ffae8133fa0 2023-10-03T10:14:44.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.684+0000 7ffacffff700 1 -- 172.21.15.3:0/2394404976 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ffae0041290 con 0x7ffae8133fa0 2023-10-03T10:14:44.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.684+0000 7ffaee25c700 1 -- 172.21.15.3:0/2394404976 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae8123540 con 0x7ffae8133fa0 2023-10-03T10:14:44.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.690+0000 7ffacffff700 1 -- 172.21.15.3:0/2394404976 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffae0003e10 con 0x7ffae8133fa0 2023-10-03T10:14:44.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.777+0000 7fa4162d0700 1 Processor -- start 2023-10-03T10:14:44.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.777+0000 7fa4162d0700 1 -- start start 2023-10-03T10:14:44.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.778+0000 7fa4162d0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa4101338a0 0x7fa410133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:44.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.778+0000 7fa4162d0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa41009e930 0x7fa41013b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:44.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.778+0000 7fa4162d0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa41006ef30 con 0x7fa41009de50 2023-10-03T10:14:44.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.778+0000 7fa4162d0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa41006fef0 con 0x7fa4101338a0 2023-10-03T10:14:44.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.778+0000 7fa4162d0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4101334b0 con 0x7fa41009e930 2023-10-03T10:14:44.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.778+0000 7fa40f7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa41009de50 0x7fa41009e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54276/0 (socket says 172.21.15.3:54276) 2023-10-03T10:14:44.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.778+0000 7fa40f7fe700 1 -- 172.21.15.3:0/2233349828 learned_addr learned my addr 172.21.15.3:0/2233349828 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:44.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.778+0000 7fa41486d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa41009e930 0x7fa41013b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:44.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.779+0000 7fa40ffff700 1 --2- 172.21.15.3:0/2233349828 >> v2:172.21.15.3:3300/0 conn(0x7fa4101338a0 0x7fa410133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:44.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.779+0000 7fa40effd700 1 -- 172.21.15.3:0/2233349828 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2698794647 0 0) 0x7fa41006ef30 con 0x7fa41009de50 2023-10-03T10:14:44.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.779+0000 7fa40effd700 1 -- 172.21.15.3:0/2233349828 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3f80035b0 con 0x7fa41009de50 2023-10-03T10:14:44.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.779+0000 7fa41486d700 1 -- 172.21.15.3:0/2233349828 >> v2:172.21.15.3:3300/0 conn(0x7fa4101338a0 msgr2=0x7fa410133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:44.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.779+0000 7fa41486d700 1 --2- 172.21.15.3:0/2233349828 >> v2:172.21.15.3:3300/0 conn(0x7fa4101338a0 0x7fa410133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.779+0000 7fa41486d700 1 -- 172.21.15.3:0/2233349828 >> v1:172.21.15.3:6789/0 conn(0x7fa41009de50 legacy=0x7fa41009e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:44.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.779+0000 7fa41486d700 1 -- 172.21.15.3:0/2233349828 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41006fc30 con 0x7fa41009e930 2023-10-03T10:14:44.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.779+0000 7fa41486d700 1 --2- 172.21.15.3:0/2233349828 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa41009e930 0x7fa41013b740 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa408009d90 tx=0x7fa40800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fbdb13d7e8878f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:44.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.779+0000 7fa40effd700 1 -- 172.21.15.3:0/2233349828 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40800e070 con 0x7fa41009e930 2023-10-03T10:14:44.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.779+0000 7fa40effd700 1 -- 172.21.15.3:0/2233349828 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40800bb20 con 0x7fa41009e930 2023-10-03T10:14:44.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.779+0000 7fa40effd700 1 -- 172.21.15.3:0/2233349828 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa408012430 con 0x7fa41009e930 2023-10-03T10:14:44.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.780+0000 7fa4162d0700 1 -- 172.21.15.3:0/2233349828 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa41009e930 msgr2=0x7fa41013b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:44.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.780+0000 7fa4162d0700 1 --2- 172.21.15.3:0/2233349828 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa41009e930 0x7fa41013b740 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa408009d90 tx=0x7fa40800b330 comp rx=0 tx=0).stop 2023-10-03T10:14:44.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.780+0000 7fa4162d0700 1 -- 172.21.15.3:0/2233349828 shutdown_connections 2023-10-03T10:14:44.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.780+0000 7fa4162d0700 1 --2- 172.21.15.3:0/2233349828 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa41009e930 0x7fa41013b740 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.780+0000 7fa4162d0700 1 --2- 172.21.15.3:0/2233349828 >> v2:172.21.15.3:3300/0 conn(0x7fa4101338a0 0x7fa410133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.780+0000 7fa4162d0700 1 -- 172.21.15.3:0/2233349828 >> 172.21.15.3:0/2233349828 conn(0x7fa4100a2aa0 msgr2=0x7fa4100a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:44.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.780+0000 7fa4162d0700 1 -- 172.21.15.3:0/2233349828 shutdown_connections 2023-10-03T10:14:44.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.780+0000 7fa4162d0700 1 -- 172.21.15.3:0/2233349828 wait complete. 2023-10-03T10:14:44.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.781+0000 7fa4162d0700 1 Processor -- start 2023-10-03T10:14:44.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.781+0000 7fa4162d0700 1 -- start start 2023-10-03T10:14:44.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.782+0000 7fa4162d0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa41009de50 0x7fa41011d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:44.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.782+0000 7fa4162d0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa41009e930 0x7fa41011dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:44.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.782+0000 7fa4162d0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa4101338a0 0x7fa41011e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:44.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.782+0000 7fa4162d0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4101208f0 con 0x7fa41009e930 2023-10-03T10:14:44.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.782+0000 7fa4162d0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa41006f640 con 0x7fa41009de50 2023-10-03T10:14:44.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.782+0000 7fa4162d0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa4100a0c40 con 0x7fa4101338a0 2023-10-03T10:14:44.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.782+0000 7fa40ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa41009de50 0x7fa41011d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:44.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.782+0000 7fa40f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa41009e930 0x7fa41011dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:44.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.782+0000 7fa40ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa41009de50 0x7fa41011d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59042/0 (socket says 172.21.15.3:59042) 2023-10-03T10:14:44.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.782+0000 7fa40ffff700 1 -- 172.21.15.3:0/56011124 learned_addr learned my addr 172.21.15.3:0/56011124 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:44.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.782+0000 7fa41486d700 1 --2- 172.21.15.3:0/56011124 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa4101338a0 0x7fa41011e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:44.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.782+0000 7fa40ffff700 1 -- 172.21.15.3:0/56011124 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa4101338a0 msgr2=0x7fa41011e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:44.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.782+0000 7fa40ffff700 1 --2- 172.21.15.3:0/56011124 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa4101338a0 0x7fa41011e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.782+0000 7fa40ffff700 1 -- 172.21.15.3:0/56011124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa41009e930 msgr2=0x7fa41011dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:44.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.782+0000 7fa40ffff700 1 --2- 172.21.15.3:0/56011124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa41009e930 0x7fa41011dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.782+0000 7fa40ffff700 1 -- 172.21.15.3:0/56011124 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa408009a70 con 0x7fa41009de50 2023-10-03T10:14:44.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.782+0000 7fa40f7fe700 1 --2- 172.21.15.3:0/56011124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa41009e930 0x7fa41011dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:44.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.783+0000 7fa40ffff700 1 --2- 172.21.15.3:0/56011124 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa41009de50 0x7fa41011d770 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa40000e810 tx=0x7fa40000eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:44.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.783+0000 7fa40cff9700 1 -- 172.21.15.3:0/56011124 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40000c070 con 0x7fa41009de50 2023-10-03T10:14:44.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.783+0000 7fa40cff9700 1 -- 172.21.15.3:0/56011124 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa400013610 con 0x7fa41009de50 2023-10-03T10:14:44.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.783+0000 7fa4162d0700 1 -- 172.21.15.3:0/56011124 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa41011f250 con 0x7fa41009de50 2023-10-03T10:14:44.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.783+0000 7fa40cff9700 1 -- 172.21.15.3:0/56011124 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4000139c0 con 0x7fa41009de50 2023-10-03T10:14:44.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.783+0000 7fa4162d0700 1 -- 172.21.15.3:0/56011124 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa410121660 con 0x7fa41009de50 2023-10-03T10:14:44.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.784+0000 7fa4162d0700 1 -- 172.21.15.3:0/56011124 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa41011a470 con 0x7fa41009de50 2023-10-03T10:14:44.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.785+0000 7fa40cff9700 1 -- 172.21.15.3:0/56011124 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa400013b20 con 0x7fa41009de50 2023-10-03T10:14:44.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.785+0000 7fa40cff9700 1 -- 172.21.15.3:0/56011124 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa40003f420 con 0x7fa41009de50 2023-10-03T10:14:44.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.789+0000 7fa40cff9700 1 -- 172.21.15.3:0/56011124 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa400010b90 con 0x7fa41009de50 2023-10-03T10:14:44.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.817+0000 7fa44ea60700 1 -- 172.21.15.3:0/878896181 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa448074ea0 con 0x7fa4480a0a30 2023-10-03T10:14:44.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.818+0000 7fa444ff9700 1 -- 172.21.15.3:0/878896181 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa43802b090 con 0x7fa4480a0a30 2023-10-03T10:14:44.818 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:44.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.821+0000 7fa44ea60700 1 -- 172.21.15.3:0/878896181 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa4480a0a30 msgr2=0x7fa4481200e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:44.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.821+0000 7fa44ea60700 1 --2- 172.21.15.3:0/878896181 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa4480a0a30 0x7fa4481200e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa43800ec10 tx=0x7fa43800ef20 comp rx=0 tx=0).stop 2023-10-03T10:14:44.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.821+0000 7fa44ea60700 1 -- 172.21.15.3:0/878896181 shutdown_connections 2023-10-03T10:14:44.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.821+0000 7fa44ea60700 1 --2- 172.21.15.3:0/878896181 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa448133fa0 0x7fa448120b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.821+0000 7fa44ea60700 1 --2- 172.21.15.3:0/878896181 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa4480a0a30 0x7fa4481200e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.821+0000 7fa44ea60700 1 --2- 172.21.15.3:0/878896181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4480a1340 0x7fa448120620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.821+0000 7fa44ea60700 1 -- 172.21.15.3:0/878896181 >> 172.21.15.3:0/878896181 conn(0x7fa4480a6550 msgr2=0x7fa44809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:44.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.821+0000 7fa44ea60700 1 -- 172.21.15.3:0/878896181 shutdown_connections 2023-10-03T10:14:44.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.821+0000 7fa44ea60700 1 -- 172.21.15.3:0/878896181 wait complete. 2023-10-03T10:14:44.840 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:44.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.897+0000 7fdf2b0c3700 1 -- 172.21.15.3:0/1855687317 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fdf240a28e0 con 0x7fdf240a0a30 2023-10-03T10:14:44.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.898+0000 7fdf217fa700 1 -- 172.21.15.3:0/1855687317 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdf18017020 con 0x7fdf240a0a30 2023-10-03T10:14:44.898 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:44.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.900+0000 7fdf2b0c3700 1 -- 172.21.15.3:0/1855687317 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf240a0a30 msgr2=0x7fdf24120030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:44.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.900+0000 7fdf2b0c3700 1 --2- 172.21.15.3:0/1855687317 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf240a0a30 0x7fdf24120030 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800e9b0 tx=0x7fdf1800ecc0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.900+0000 7fdf2b0c3700 1 -- 172.21.15.3:0/1855687317 shutdown_connections 2023-10-03T10:14:44.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.900+0000 7fdf2b0c3700 1 --2- 172.21.15.3:0/1855687317 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf24133fa0 0x7fdf24120ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.900+0000 7fdf2b0c3700 1 --2- 172.21.15.3:0/1855687317 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf240a0a30 0x7fdf24120030 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.900+0000 7fdf2b0c3700 1 --2- 172.21.15.3:0/1855687317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf240a1340 0x7fdf24120570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.900+0000 7fdf2b0c3700 1 -- 172.21.15.3:0/1855687317 >> 172.21.15.3:0/1855687317 conn(0x7fdf240a6550 msgr2=0x7fdf241bb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:44.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.900+0000 7fdf2b0c3700 1 -- 172.21.15.3:0/1855687317 shutdown_connections 2023-10-03T10:14:44.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.900+0000 7fdf2b0c3700 1 -- 172.21.15.3:0/1855687317 wait complete. 2023-10-03T10:14:44.915 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:44.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.925+0000 7ffaee25c700 1 -- 172.21.15.3:0/2394404976 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ffae8074ea0 con 0x7ffae8133fa0 2023-10-03T10:14:44.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.925+0000 7ffacffff700 1 -- 172.21.15.3:0/2394404976 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffae001b3a0 con 0x7ffae8133fa0 2023-10-03T10:14:44.925 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.927+0000 7ffaee25c700 1 -- 172.21.15.3:0/2394404976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffae8133fa0 msgr2=0x7ffae8120b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.927+0000 7ffaee25c700 1 --2- 172.21.15.3:0/2394404976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffae8133fa0 0x7ffae8120b60 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ffae0015000 tx=0x7ffae0004ad0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.928+0000 7ffaee25c700 1 -- 172.21.15.3:0/2394404976 shutdown_connections 2023-10-03T10:14:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.928+0000 7ffaee25c700 1 --2- 172.21.15.3:0/2394404976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffae8133fa0 0x7ffae8120b60 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.928+0000 7ffaee25c700 1 --2- 172.21.15.3:0/2394404976 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffae80a0a30 0x7ffae81200e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.928+0000 7ffaee25c700 1 --2- 172.21.15.3:0/2394404976 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffae80a1340 0x7ffae8120620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:44.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.928+0000 7ffaee25c700 1 -- 172.21.15.3:0/2394404976 >> 172.21.15.3:0/2394404976 conn(0x7ffae80a6550 msgr2=0x7ffae809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:44.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.928+0000 7ffaee25c700 1 -- 172.21.15.3:0/2394404976 shutdown_connections 2023-10-03T10:14:44.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.928+0000 7ffaee25c700 1 -- 172.21.15.3:0/2394404976 wait complete. 2023-10-03T10:14:44.941 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:44.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:44.999+0000 7fa4162d0700 1 -- 172.21.15.3:0/56011124 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa410120d00 con 0x7fa41009de50 2023-10-03T10:14:45.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.000+0000 7fa40cff9700 1 -- 172.21.15.3:0/56011124 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa400010780 con 0x7fa41009de50 2023-10-03T10:14:45.000 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:45.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.002+0000 7fa4162d0700 1 -- 172.21.15.3:0/56011124 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa41009de50 msgr2=0x7fa41011d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:45.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.002+0000 7fa4162d0700 1 --2- 172.21.15.3:0/56011124 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa41009de50 0x7fa41011d770 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa40000e810 tx=0x7fa40000eb20 comp rx=0 tx=0).stop 2023-10-03T10:14:45.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.002+0000 7fa4162d0700 1 -- 172.21.15.3:0/56011124 shutdown_connections 2023-10-03T10:14:45.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.002+0000 7fa4162d0700 1 --2- 172.21.15.3:0/56011124 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa4101338a0 0x7fa41011e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:45.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.002+0000 7fa4162d0700 1 --2- 172.21.15.3:0/56011124 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa41009de50 0x7fa41011d770 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:45.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.002+0000 7fa4162d0700 1 --2- 172.21.15.3:0/56011124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa41009e930 0x7fa41011dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:45.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.002+0000 7fa4162d0700 1 -- 172.21.15.3:0/56011124 >> 172.21.15.3:0/56011124 conn(0x7fa4100a2aa0 msgr2=0x7fa4101baac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:45.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.002+0000 7fa4162d0700 1 -- 172.21.15.3:0/56011124 shutdown_connections 2023-10-03T10:14:45.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.002+0000 7fa4162d0700 1 -- 172.21.15.3:0/56011124 wait complete. 2023-10-03T10:14:45.016 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:45.841 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:45.916 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:45.942 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:45.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.992+0000 7fc87c33c700 1 Processor -- start 2023-10-03T10:14:45.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.992+0000 7fc87c33c700 1 -- start start 2023-10-03T10:14:45.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.993+0000 7fc87c33c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc87409e930 0x7fc87409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:45.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.993+0000 7fc87c33c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc87409f650 0x7fc874131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:45.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.993+0000 7fc87c33c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc874070940 con 0x7fc87409f240 2023-10-03T10:14:45.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.993+0000 7fc87c33c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87406fd60 con 0x7fc87409e930 2023-10-03T10:14:45.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.993+0000 7fc87c33c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc87406fa80 con 0x7fc87409f650 2023-10-03T10:14:45.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.993+0000 7fc87a0d8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc87409e930 0x7fc87409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:45.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.993+0000 7fc87a0d8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc87409e930 0x7fc87409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59046/0 (socket says 172.21.15.3:59046) 2023-10-03T10:14:45.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.993+0000 7fc87a0d8700 1 -- 172.21.15.3:0/2021142260 learned_addr learned my addr 172.21.15.3:0/2021142260 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:45.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.993+0000 7fc87a8d9700 1 --2- 172.21.15.3:0/2021142260 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc87409f650 0x7fc874131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:45.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.993+0000 7fc87a0d8700 1 -- 172.21.15.3:0/2021142260 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc87409f650 msgr2=0x7fc874131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:45.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.993+0000 7fc87a0d8700 1 --2- 172.21.15.3:0/2021142260 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc87409f650 0x7fc874131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:45.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.993+0000 7fc87a0d8700 1 -- 172.21.15.3:0/2021142260 >> v1:172.21.15.3:6789/0 conn(0x7fc87409f240 legacy=0x7fc8741317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:45.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.993+0000 7fc87a0d8700 1 -- 172.21.15.3:0/2021142260 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87409fb60 con 0x7fc87409e930 2023-10-03T10:14:45.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.994+0000 7fc87a0d8700 1 --2- 172.21.15.3:0/2021142260 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc87409e930 0x7fc87409ed00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc870009920 tx=0x7fc870009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e9f18fd03c773aa server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:45.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.994+0000 7fc8790d6700 1 -- 172.21.15.3:0/2021142260 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc870011830 con 0x7fc87409e930 2023-10-03T10:14:45.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.994+0000 7fc8790d6700 1 -- 172.21.15.3:0/2021142260 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc870011990 con 0x7fc87409e930 2023-10-03T10:14:45.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.994+0000 7fc8790d6700 1 -- 172.21.15.3:0/2021142260 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc870011d20 con 0x7fc87409e930 2023-10-03T10:14:45.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.995+0000 7fc87c33c700 1 -- 172.21.15.3:0/2021142260 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc87409e930 msgr2=0x7fc87409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:45.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.995+0000 7fc87c33c700 1 --2- 172.21.15.3:0/2021142260 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc87409e930 0x7fc87409ed00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc870009920 tx=0x7fc870009c30 comp rx=0 tx=0).stop 2023-10-03T10:14:45.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.995+0000 7fc87c33c700 1 -- 172.21.15.3:0/2021142260 shutdown_connections 2023-10-03T10:14:45.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.995+0000 7fc87c33c700 1 --2- 172.21.15.3:0/2021142260 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc87409e930 0x7fc87409ed00 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:45.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.995+0000 7fc87c33c700 1 --2- 172.21.15.3:0/2021142260 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc87409f650 0x7fc874131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:45.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.995+0000 7fc87c33c700 1 -- 172.21.15.3:0/2021142260 >> 172.21.15.3:0/2021142260 conn(0x7fc8740a4210 msgr2=0x7fc8740a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:45.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.995+0000 7fc87c33c700 1 -- 172.21.15.3:0/2021142260 shutdown_connections 2023-10-03T10:14:45.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.995+0000 7fc87c33c700 1 -- 172.21.15.3:0/2021142260 wait complete. 2023-10-03T10:14:45.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.997+0000 7fc87c33c700 1 Processor -- start 2023-10-03T10:14:45.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.997+0000 7fc87c33c700 1 -- start start 2023-10-03T10:14:45.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.997+0000 7fc87c33c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc87409e930 0x7fc87406b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:45.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.998+0000 7fc87c33c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc87409f240 0x7fc87406bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:45.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.998+0000 7fc87c33c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc87409f650 0x7fc87406c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:45.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.998+0000 7fc87c33c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8741a8190 con 0x7fc87409e930 2023-10-03T10:14:45.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.998+0000 7fc87c33c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8741b62c0 con 0x7fc87409f240 2023-10-03T10:14:45.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.998+0000 7fc87c33c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8741a8690 con 0x7fc87409f650 2023-10-03T10:14:45.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.998+0000 7fc8798d7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc87409f240 0x7fc87406bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:45.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.998+0000 7fc8798d7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc87409f240 0x7fc87406bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59050/0 (socket says 172.21.15.3:59050) 2023-10-03T10:14:45.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.998+0000 7fc8798d7700 1 -- 172.21.15.3:0/4275260070 learned_addr learned my addr 172.21.15.3:0/4275260070 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:46.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.998+0000 7fc87a0d8700 1 --2- 172.21.15.3:0/4275260070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc87409e930 0x7fc87406b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:46.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.998+0000 7fc87a8d9700 1 --2- 172.21.15.3:0/4275260070 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc87409f650 0x7fc87406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:46.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.998+0000 7fc8798d7700 1 -- 172.21.15.3:0/4275260070 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc87409f650 msgr2=0x7fc87406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:46.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.998+0000 7fc8798d7700 1 --2- 172.21.15.3:0/4275260070 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc87409f650 0x7fc87406c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.998+0000 7fc8798d7700 1 -- 172.21.15.3:0/4275260070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc87409e930 msgr2=0x7fc87406b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:46.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.998+0000 7fc8798d7700 1 --2- 172.21.15.3:0/4275260070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc87409e930 0x7fc87406b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.998+0000 7fc8798d7700 1 -- 172.21.15.3:0/4275260070 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc870009550 con 0x7fc87409f240 2023-10-03T10:14:46.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.998+0000 7fc8798d7700 1 --2- 172.21.15.3:0/4275260070 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc87409f240 0x7fc87406bdf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc86400c4b0 tx=0x7fc86400c7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:46.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.999+0000 7fc86affd700 1 -- 172.21.15.3:0/4275260070 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8640052d0 con 0x7fc87409f240 2023-10-03T10:14:46.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.999+0000 7fc86affd700 1 -- 172.21.15.3:0/4275260070 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc864005430 con 0x7fc87409f240 2023-10-03T10:14:46.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.999+0000 7fc86affd700 1 -- 172.21.15.3:0/4275260070 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8640038c0 con 0x7fc87409f240 2023-10-03T10:14:46.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.999+0000 7fc87c33c700 1 -- 172.21.15.3:0/4275260070 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8740a0190 con 0x7fc87409f240 2023-10-03T10:14:46.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:45.999+0000 7fc87c33c700 1 -- 172.21.15.3:0/4275260070 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8741a83d0 con 0x7fc87409f240 2023-10-03T10:14:46.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.000+0000 7fc86affd700 1 -- 172.21.15.3:0/4275260070 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc864003a20 con 0x7fc87409f240 2023-10-03T10:14:46.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.000+0000 7fc87c33c700 1 -- 172.21.15.3:0/4275260070 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc874074ea0 con 0x7fc87409f240 2023-10-03T10:14:46.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.000+0000 7fc86affd700 1 -- 172.21.15.3:0/4275260070 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc86403ea20 con 0x7fc87409f240 2023-10-03T10:14:46.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.005+0000 7fc86affd700 1 -- 172.21.15.3:0/4275260070 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc864018260 con 0x7fc87409f240 2023-10-03T10:14:46.016 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:46.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.072+0000 7fea66a56700 1 Processor -- start 2023-10-03T10:14:46.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.073+0000 7fea66a56700 1 -- start start 2023-10-03T10:14:46.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.073+0000 7fea66a56700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fea600a0a30 0x7fea600a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:46.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.073+0000 7fea66a56700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea600a1340 0x7fea60133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:46.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.073+0000 7fea66a56700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea60133fa0 0x7fea60134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:46.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.073+0000 7fea66a56700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea6009edd0 con 0x7fea600a0a30 2023-10-03T10:14:46.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.073+0000 7fea66a56700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea6009e1f0 con 0x7fea600a1340 2023-10-03T10:14:46.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.073+0000 7fea66a56700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea6009dc50 con 0x7fea60133fa0 2023-10-03T10:14:46.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.074+0000 7fea64ff3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea60133fa0 0x7fea60134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:46.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.074+0000 7fea5ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fea600a0a30 0x7fea600a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:46.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.074+0000 7fea64ff3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea60133fa0 0x7fea60134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57376/0 (socket says 172.21.15.3:57376) 2023-10-03T10:14:46.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.074+0000 7fea64ff3700 1 -- 172.21.15.3:0/3187367217 learned_addr learned my addr 172.21.15.3:0/3187367217 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:46.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.074+0000 7fea5f7fe700 1 --2- 172.21.15.3:0/3187367217 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea600a1340 0x7fea60133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:46.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.074+0000 7fea5f7fe700 1 -- 172.21.15.3:0/3187367217 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea60133fa0 msgr2=0x7fea60134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:46.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.074+0000 7fea5f7fe700 1 --2- 172.21.15.3:0/3187367217 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea60133fa0 0x7fea60134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.074+0000 7fea5f7fe700 1 -- 172.21.15.3:0/3187367217 >> v2:172.21.15.3:3300/0 conn(0x7fea600a0a30 msgr2=0x7fea600a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:46.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.074+0000 7fea5f7fe700 1 --2- 172.21.15.3:0/3187367217 >> v2:172.21.15.3:3300/0 conn(0x7fea600a0a30 0x7fea600a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.074+0000 7fea5f7fe700 1 -- 172.21.15.3:0/3187367217 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea600a1fe0 con 0x7fea600a1340 2023-10-03T10:14:46.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.074+0000 7fea64ff3700 1 --2- 172.21.15.3:0/3187367217 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea60133fa0 0x7fea60134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:46.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.074+0000 7fea5f7fe700 1 --2- 172.21.15.3:0/3187367217 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea600a1340 0x7fea60133a60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fea54009870 tx=0x7fea54009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a15f6729f760c136 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:46.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.075+0000 7fea5effd700 1 -- 172.21.15.3:0/3187367217 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea54011830 con 0x7fea600a1340 2023-10-03T10:14:46.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.075+0000 7fea5effd700 1 -- 172.21.15.3:0/3187367217 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea54011990 con 0x7fea600a1340 2023-10-03T10:14:46.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.075+0000 7fea5effd700 1 -- 172.21.15.3:0/3187367217 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea54011d20 con 0x7fea600a1340 2023-10-03T10:14:46.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.075+0000 7fea66a56700 1 -- 172.21.15.3:0/3187367217 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea600a1340 msgr2=0x7fea60133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:46.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.075+0000 7fea66a56700 1 --2- 172.21.15.3:0/3187367217 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea600a1340 0x7fea60133a60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fea54009870 tx=0x7fea54009b80 comp rx=0 tx=0).stop 2023-10-03T10:14:46.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.075+0000 7fea66a56700 1 -- 172.21.15.3:0/3187367217 shutdown_connections 2023-10-03T10:14:46.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.075+0000 7fea66a56700 1 --2- 172.21.15.3:0/3187367217 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea60133fa0 0x7fea60134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.076+0000 7fea66a56700 1 --2- 172.21.15.3:0/3187367217 >> v2:172.21.15.3:3300/0 conn(0x7fea600a0a30 0x7fea600a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.076+0000 7fea66a56700 1 --2- 172.21.15.3:0/3187367217 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea600a1340 0x7fea60133a60 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.076+0000 7fea66a56700 1 -- 172.21.15.3:0/3187367217 >> 172.21.15.3:0/3187367217 conn(0x7fea600a6550 msgr2=0x7fea600a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:46.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.076+0000 7fea66a56700 1 -- 172.21.15.3:0/3187367217 shutdown_connections 2023-10-03T10:14:46.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.076+0000 7fea66a56700 1 -- 172.21.15.3:0/3187367217 wait complete. 2023-10-03T10:14:46.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.077+0000 7fea66a56700 1 Processor -- start 2023-10-03T10:14:46.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.077+0000 7fea66a56700 1 -- start start 2023-10-03T10:14:46.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.078+0000 7fea66a56700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea600a0a30 0x7fea60120310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:46.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.078+0000 7fea66a56700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea600a1340 0x7fea60120850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:46.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.078+0000 7fea66a56700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea60133fa0 0x7fea60120d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:46.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.078+0000 7fea66a56700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea601217c0 con 0x7fea600a0a30 2023-10-03T10:14:46.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.078+0000 7fea66a56700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea60123e30 con 0x7fea60133fa0 2023-10-03T10:14:46.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.078+0000 7fea66a56700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea60123200 con 0x7fea600a1340 2023-10-03T10:14:46.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.078+0000 7fea5ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea600a0a30 0x7fea60120310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:46.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.078+0000 7fea5ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea600a0a30 0x7fea60120310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56192/0 (socket says 172.21.15.3:56192) 2023-10-03T10:14:46.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.078+0000 7fea5ffff700 1 -- 172.21.15.3:0/4269097761 learned_addr learned my addr 172.21.15.3:0/4269097761 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:46.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.078+0000 7fea64ff3700 1 --2- 172.21.15.3:0/4269097761 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea60133fa0 0x7fea60120d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:46.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.078+0000 7fea5f7fe700 1 --2- 172.21.15.3:0/4269097761 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea600a1340 0x7fea60120850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:46.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.079+0000 7fea5ffff700 1 -- 172.21.15.3:0/4269097761 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea600a1340 msgr2=0x7fea60120850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:46.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.079+0000 7fea5ffff700 1 --2- 172.21.15.3:0/4269097761 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea600a1340 0x7fea60120850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.079+0000 7fea5ffff700 1 -- 172.21.15.3:0/4269097761 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea60133fa0 msgr2=0x7fea60120d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:46.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.079+0000 7fea5ffff700 1 --2- 172.21.15.3:0/4269097761 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea60133fa0 0x7fea60120d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.079+0000 7fea5ffff700 1 -- 172.21.15.3:0/4269097761 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea58009b30 con 0x7fea600a0a30 2023-10-03T10:14:46.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.079+0000 7fea5ffff700 1 --2- 172.21.15.3:0/4269097761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea600a0a30 0x7fea60120310 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fea5000e9b0 tx=0x7fea5000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:46.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.079+0000 7fea5cff9700 1 -- 172.21.15.3:0/4269097761 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea50009710 con 0x7fea600a0a30 2023-10-03T10:14:46.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.079+0000 7fea5cff9700 1 -- 172.21.15.3:0/4269097761 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5000ee50 con 0x7fea600a0a30 2023-10-03T10:14:46.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.079+0000 7fea66a56700 1 -- 172.21.15.3:0/4269097761 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea540094a0 con 0x7fea600a0a30 2023-10-03T10:14:46.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.079+0000 7fea5cff9700 1 -- 172.21.15.3:0/4269097761 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5000ca50 con 0x7fea600a0a30 2023-10-03T10:14:46.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.080+0000 7fea66a56700 1 -- 172.21.15.3:0/4269097761 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea600a0220 con 0x7fea600a0a30 2023-10-03T10:14:46.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.080+0000 7fea5cff9700 1 -- 172.21.15.3:0/4269097761 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fea5000cbb0 con 0x7fea600a0a30 2023-10-03T10:14:46.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.081+0000 7fea5cff9700 1 -- 172.21.15.3:0/4269097761 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fea500421c0 con 0x7fea600a0a30 2023-10-03T10:14:46.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.082+0000 7fea66a56700 1 -- 172.21.15.3:0/4269097761 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea601237f0 con 0x7fea600a0a30 2023-10-03T10:14:46.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.091+0000 7fea5cff9700 1 -- 172.21.15.3:0/4269097761 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fea601237f0 con 0x7fea600a0a30 2023-10-03T10:14:46.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.126+0000 7f08e6616700 1 Processor -- start 2023-10-03T10:14:46.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.126+0000 7f08e6616700 1 -- start start 2023-10-03T10:14:46.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.127+0000 7f08e6616700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08e009e930 0x7f08e009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:46.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.127+0000 7f08e6616700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08e009f240 0x7f08e01317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:46.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.127+0000 7f08e6616700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f08e009f650 0x7f08e0131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:46.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.127+0000 7f08e6616700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08e0070940 con 0x7f08e009f650 2023-10-03T10:14:46.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.127+0000 7f08e6616700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e006fd60 con 0x7f08e009e930 2023-10-03T10:14:46.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.127+0000 7f08e6616700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f08e006fa80 con 0x7f08e009f240 2023-10-03T10:14:46.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.127+0000 7f08e4bb3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f08e009f650 0x7f08e0131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:46.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.127+0000 7f08df7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08e009f240 0x7f08e01317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:46.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.127+0000 7f08dffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08e009e930 0x7f08e009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:46.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.127+0000 7f08e4bb3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f08e009f650 0x7f08e0131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56206/0 (socket says 172.21.15.3:56206) 2023-10-03T10:14:46.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.127+0000 7f08df7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08e009f240 0x7f08e01317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57404/0 (socket says 172.21.15.3:57404) 2023-10-03T10:14:46.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.127+0000 7f08dffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08e009e930 0x7f08e009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59082/0 (socket says 172.21.15.3:59082) 2023-10-03T10:14:46.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.127+0000 7f08e4bb3700 1 -- 172.21.15.3:0/321649928 learned_addr learned my addr 172.21.15.3:0/321649928 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:46.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.128+0000 7f08dffff700 1 -- 172.21.15.3:0/321649928 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08e009f240 msgr2=0x7f08e01317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:46.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.128+0000 7f08dffff700 1 --2- 172.21.15.3:0/321649928 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08e009f240 0x7f08e01317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.128+0000 7f08dffff700 1 -- 172.21.15.3:0/321649928 >> v2:172.21.15.3:3300/0 conn(0x7f08e009f650 msgr2=0x7f08e0131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:46.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.128+0000 7f08dffff700 1 --2- 172.21.15.3:0/321649928 >> v2:172.21.15.3:3300/0 conn(0x7f08e009f650 0x7f08e0131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.128+0000 7f08dffff700 1 -- 172.21.15.3:0/321649928 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e009fb60 con 0x7f08e009e930 2023-10-03T10:14:46.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.128+0000 7f08e4bb3700 1 --2- 172.21.15.3:0/321649928 >> v2:172.21.15.3:3300/0 conn(0x7f08e009f650 0x7f08e0131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:46.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.128+0000 7f08dffff700 1 --2- 172.21.15.3:0/321649928 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08e009e930 0x7f08e009ed00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f08d0009840 tx=0x7f08d0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b85c562e8ad1580 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:46.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.128+0000 7f08deffd700 1 -- 172.21.15.3:0/321649928 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d0011830 con 0x7f08e009e930 2023-10-03T10:14:46.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.128+0000 7f08deffd700 1 -- 172.21.15.3:0/321649928 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d0011990 con 0x7f08e009e930 2023-10-03T10:14:46.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.128+0000 7f08deffd700 1 -- 172.21.15.3:0/321649928 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d0011d20 con 0x7f08e009e930 2023-10-03T10:14:46.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.129+0000 7f08e6616700 1 -- 172.21.15.3:0/321649928 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08e009e930 msgr2=0x7f08e009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:46.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.129+0000 7f08e6616700 1 --2- 172.21.15.3:0/321649928 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08e009e930 0x7f08e009ed00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f08d0009840 tx=0x7f08d0009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:46.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.129+0000 7f08e6616700 1 -- 172.21.15.3:0/321649928 shutdown_connections 2023-10-03T10:14:46.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.129+0000 7f08e6616700 1 --2- 172.21.15.3:0/321649928 >> v2:172.21.15.3:3300/0 conn(0x7f08e009f650 0x7f08e0131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.129+0000 7f08e6616700 1 --2- 172.21.15.3:0/321649928 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08e009e930 0x7f08e009ed00 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.129+0000 7f08e6616700 1 --2- 172.21.15.3:0/321649928 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08e009f240 0x7f08e01317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.129+0000 7f08e6616700 1 -- 172.21.15.3:0/321649928 >> 172.21.15.3:0/321649928 conn(0x7f08e00a4210 msgr2=0x7f08e00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:46.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.129+0000 7f08e6616700 1 -- 172.21.15.3:0/321649928 shutdown_connections 2023-10-03T10:14:46.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.130+0000 7f08e6616700 1 -- 172.21.15.3:0/321649928 wait complete. 2023-10-03T10:14:46.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.131+0000 7f08e6616700 1 Processor -- start 2023-10-03T10:14:46.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.131+0000 7f08e6616700 1 -- start start 2023-10-03T10:14:46.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.132+0000 7f08e6616700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08e009e930 0x7f08e01a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:46.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.132+0000 7f08e6616700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08e009f240 0x7f08e01a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:46.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.132+0000 7f08e6616700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08e009f650 0x7f08e01a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:46.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.132+0000 7f08e6616700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e01a5ec0 con 0x7f08e009f650 2023-10-03T10:14:46.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.132+0000 7f08e6616700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e01a6f80 con 0x7f08e009e930 2023-10-03T10:14:46.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.132+0000 7f08e6616700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f08e0071d20 con 0x7f08e009f240 2023-10-03T10:14:46.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.132+0000 7f08dffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08e009e930 0x7f08e01a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:46.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.132+0000 7f08dffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08e009e930 0x7f08e01a3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59084/0 (socket says 172.21.15.3:59084) 2023-10-03T10:14:46.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.132+0000 7f08dffff700 1 -- 172.21.15.3:0/2033274692 learned_addr learned my addr 172.21.15.3:0/2033274692 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:46.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.132+0000 7f08e4bb3700 1 --2- 172.21.15.3:0/2033274692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08e009f650 0x7f08e01a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:46.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.132+0000 7f08df7fe700 1 --2- 172.21.15.3:0/2033274692 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08e009f240 0x7f08e01a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:46.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.133+0000 7f08dffff700 1 -- 172.21.15.3:0/2033274692 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08e009f240 msgr2=0x7f08e01a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:46.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.133+0000 7f08dffff700 1 --2- 172.21.15.3:0/2033274692 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08e009f240 0x7f08e01a4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.133+0000 7f08dffff700 1 -- 172.21.15.3:0/2033274692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08e009f650 msgr2=0x7f08e01a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:46.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.133+0000 7f08dffff700 1 --2- 172.21.15.3:0/2033274692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08e009f650 0x7f08e01a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.133+0000 7f08dffff700 1 -- 172.21.15.3:0/2033274692 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d8009b30 con 0x7f08e009e930 2023-10-03T10:14:46.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.133+0000 7f08dffff700 1 --2- 172.21.15.3:0/2033274692 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08e009e930 0x7f08e01a3cd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f08d00090b0 tx=0x7f08d0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:46.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.133+0000 7f08dcff9700 1 -- 172.21.15.3:0/2033274692 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d0004450 con 0x7f08e009e930 2023-10-03T10:14:46.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.133+0000 7f08dcff9700 1 -- 172.21.15.3:0/2033274692 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d0027070 con 0x7f08e009e930 2023-10-03T10:14:46.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.133+0000 7f08dcff9700 1 -- 172.21.15.3:0/2033274692 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d00038e0 con 0x7f08e009e930 2023-10-03T10:14:46.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.133+0000 7f08e6616700 1 -- 172.21.15.3:0/2033274692 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d00094a0 con 0x7f08e009e930 2023-10-03T10:14:46.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.133+0000 7f08e6616700 1 -- 172.21.15.3:0/2033274692 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e00a0190 con 0x7f08e009e930 2023-10-03T10:14:46.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.134+0000 7f08dcff9700 1 -- 172.21.15.3:0/2033274692 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f08d0003a40 con 0x7f08e009e930 2023-10-03T10:14:46.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.135+0000 7f08dcff9700 1 -- 172.21.15.3:0/2033274692 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f08d0029070 con 0x7f08e009e930 2023-10-03T10:14:46.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.135+0000 7f08e6616700 1 -- 172.21.15.3:0/2033274692 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e01a71b0 con 0x7f08e009e930 2023-10-03T10:14:46.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.140+0000 7f08dcff9700 1 -- 172.21.15.3:0/2033274692 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f08e01a71b0 con 0x7f08e009e930 2023-10-03T10:14:46.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.166+0000 7fd3a2d36700 1 Processor -- start 2023-10-03T10:14:46.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.166+0000 7fd3a2d36700 1 -- start start 2023-10-03T10:14:46.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.167+0000 7fd3a2d36700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd39c09d6f0 0x7fd39c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:46.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.167+0000 7fd3a2d36700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd39c09e000 0x7fd39c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:46.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.167+0000 7fd3a2d36700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd39c09e410 0x7fd39c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:46.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.167+0000 7fd3a2d36700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd39c09f520 con 0x7fd39c09e000 2023-10-03T10:14:46.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.167+0000 7fd3a2d36700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c09f130 con 0x7fd39c09e410 2023-10-03T10:14:46.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.167+0000 7fd3a2d36700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c09e930 con 0x7fd39c09d6f0 2023-10-03T10:14:46.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.167+0000 7fd3a12d3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd39c09e410 0x7fd39c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:46.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.167+0000 7fd3a12d3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd39c09e410 0x7fd39c130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59094/0 (socket says 172.21.15.3:59094) 2023-10-03T10:14:46.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.167+0000 7fd3a12d3700 1 -- 172.21.15.3:0/41104360 learned_addr learned my addr 172.21.15.3:0/41104360 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:46.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.167+0000 7fd39bfff700 1 --2- 172.21.15.3:0/41104360 >> v2:172.21.15.3:3300/0 conn(0x7fd39c09e000 0x7fd39c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:46.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.167+0000 7fd3a0ad2700 1 --2- 172.21.15.3:0/41104360 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd39c09d6f0 0x7fd39c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:46.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.168+0000 7fd3a12d3700 1 -- 172.21.15.3:0/41104360 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd39c09d6f0 msgr2=0x7fd39c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:46.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.168+0000 7fd3a12d3700 1 --2- 172.21.15.3:0/41104360 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd39c09d6f0 0x7fd39c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.168+0000 7fd3a12d3700 1 -- 172.21.15.3:0/41104360 >> v2:172.21.15.3:3300/0 conn(0x7fd39c09e000 msgr2=0x7fd39c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:46.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.168+0000 7fd3a12d3700 1 --2- 172.21.15.3:0/41104360 >> v2:172.21.15.3:3300/0 conn(0x7fd39c09e000 0x7fd39c130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.168+0000 7fd3a12d3700 1 -- 172.21.15.3:0/41104360 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd39c09b2d0 con 0x7fd39c09e410 2023-10-03T10:14:46.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.168+0000 7fd39bfff700 1 --2- 172.21.15.3:0/41104360 >> v2:172.21.15.3:3300/0 conn(0x7fd39c09e000 0x7fd39c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:46.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.168+0000 7fd3a12d3700 1 --2- 172.21.15.3:0/41104360 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd39c09e410 0x7fd39c130ba0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd394009e50 tx=0x7fd39400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c46514d6fee56517 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:46.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.168+0000 7fd39b7fe700 1 -- 172.21.15.3:0/41104360 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39400e070 con 0x7fd39c09e410 2023-10-03T10:14:46.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.168+0000 7fd39b7fe700 1 -- 172.21.15.3:0/41104360 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39400b910 con 0x7fd39c09e410 2023-10-03T10:14:46.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.168+0000 7fd39b7fe700 1 -- 172.21.15.3:0/41104360 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39400bcc0 con 0x7fd39c09e410 2023-10-03T10:14:46.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.169+0000 7fd3a2d36700 1 -- 172.21.15.3:0/41104360 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd39c09e410 msgr2=0x7fd39c130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:46.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.169+0000 7fd3a2d36700 1 --2- 172.21.15.3:0/41104360 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd39c09e410 0x7fd39c130ba0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd394009e50 tx=0x7fd39400b330 comp rx=0 tx=0).stop 2023-10-03T10:14:46.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.169+0000 7fd3a2d36700 1 -- 172.21.15.3:0/41104360 shutdown_connections 2023-10-03T10:14:46.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.169+0000 7fd3a2d36700 1 --2- 172.21.15.3:0/41104360 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd39c09e410 0x7fd39c130ba0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.169+0000 7fd3a2d36700 1 --2- 172.21.15.3:0/41104360 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd39c09d6f0 0x7fd39c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.169+0000 7fd3a2d36700 1 --2- 172.21.15.3:0/41104360 >> v2:172.21.15.3:3300/0 conn(0x7fd39c09e000 0x7fd39c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.169+0000 7fd3a2d36700 1 -- 172.21.15.3:0/41104360 >> 172.21.15.3:0/41104360 conn(0x7fd39c066f50 msgr2=0x7fd39c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:46.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.169+0000 7fd3a2d36700 1 -- 172.21.15.3:0/41104360 shutdown_connections 2023-10-03T10:14:46.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.169+0000 7fd3a2d36700 1 -- 172.21.15.3:0/41104360 wait complete. 2023-10-03T10:14:46.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.170+0000 7fd3a2d36700 1 Processor -- start 2023-10-03T10:14:46.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.170+0000 7fd3a2d36700 1 -- start start 2023-10-03T10:14:46.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.171+0000 7fd3a2d36700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd39c09d6f0 0x7fd39c1c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:46.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.171+0000 7fd3a2d36700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd39c09e000 0x7fd39c1c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:46.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.171+0000 7fd3a2d36700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd39c09e410 0x7fd39c119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:46.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.171+0000 7fd3a2d36700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c11be70 con 0x7fd39c09d6f0 2023-10-03T10:14:46.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.171+0000 7fd3a2d36700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c1c5df0 con 0x7fd39c09e000 2023-10-03T10:14:46.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.171+0000 7fd3a2d36700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c130f20 con 0x7fd39c09e410 2023-10-03T10:14:46.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.171+0000 7fd3a0ad2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd39c09d6f0 0x7fd39c1c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:46.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.171+0000 7fd3a0ad2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd39c09d6f0 0x7fd39c1c61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56224/0 (socket says 172.21.15.3:56224) 2023-10-03T10:14:46.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.171+0000 7fd3a0ad2700 1 -- 172.21.15.3:0/818132723 learned_addr learned my addr 172.21.15.3:0/818132723 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:46.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.171+0000 7fd3a12d3700 1 --2- 172.21.15.3:0/818132723 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd39c09e410 0x7fd39c119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:46.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.171+0000 7fd39bfff700 1 --2- 172.21.15.3:0/818132723 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd39c09e000 0x7fd39c1c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:46.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.172+0000 7fd39bfff700 1 -- 172.21.15.3:0/818132723 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd39c09e410 msgr2=0x7fd39c119480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:46.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.172+0000 7fd39bfff700 1 --2- 172.21.15.3:0/818132723 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd39c09e410 0x7fd39c119480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.172+0000 7fd39bfff700 1 -- 172.21.15.3:0/818132723 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd39c09d6f0 msgr2=0x7fd39c1c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:46.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.172+0000 7fd39bfff700 1 --2- 172.21.15.3:0/818132723 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd39c09d6f0 0x7fd39c1c61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.172+0000 7fd39bfff700 1 -- 172.21.15.3:0/818132723 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3880094a0 con 0x7fd39c09e000 2023-10-03T10:14:46.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.172+0000 7fd39bfff700 1 --2- 172.21.15.3:0/818132723 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd39c09e000 0x7fd39c1c66f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd38800e980 tx=0x7fd38800ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:46.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.172+0000 7fd3997fa700 1 -- 172.21.15.3:0/818132723 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38800eed0 con 0x7fd39c09e000 2023-10-03T10:14:46.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.172+0000 7fd3997fa700 1 -- 172.21.15.3:0/818132723 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38800c850 con 0x7fd39c09e000 2023-10-03T10:14:46.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.172+0000 7fd3997fa700 1 -- 172.21.15.3:0/818132723 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38800cbe0 con 0x7fd39c09e000 2023-10-03T10:14:46.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.172+0000 7fd3a2d36700 1 -- 172.21.15.3:0/818132723 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd394009b30 con 0x7fd39c09e000 2023-10-03T10:14:46.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.172+0000 7fd3a2d36700 1 -- 172.21.15.3:0/818132723 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd39c11a950 con 0x7fd39c09e000 2023-10-03T10:14:46.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.173+0000 7fd3a2d36700 1 -- 172.21.15.3:0/818132723 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd39c074ea0 con 0x7fd39c09e000 2023-10-03T10:14:46.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.174+0000 7fd3997fa700 1 -- 172.21.15.3:0/818132723 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd38800cd40 con 0x7fd39c09e000 2023-10-03T10:14:46.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.174+0000 7fd3997fa700 1 -- 172.21.15.3:0/818132723 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd38803f870 con 0x7fd39c09e000 2023-10-03T10:14:46.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.179+0000 7fd3997fa700 1 -- 172.21.15.3:0/818132723 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd3880232f0 con 0x7fd39c09e000 2023-10-03T10:14:46.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.225+0000 7fc87c33c700 1 -- 172.21.15.3:0/4275260070 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc87406b250 con 0x7fc87409f240 2023-10-03T10:14:46.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.226+0000 7fc86affd700 1 -- 172.21.15.3:0/4275260070 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc864019a70 con 0x7fc87409f240 2023-10-03T10:14:46.226 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:46.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.229+0000 7fc87c33c700 1 -- 172.21.15.3:0/4275260070 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc87409f240 msgr2=0x7fc87406bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:46.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.229+0000 7fc87c33c700 1 --2- 172.21.15.3:0/4275260070 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc87409f240 0x7fc87406bdf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc86400c4b0 tx=0x7fc86400c7c0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.229+0000 7fc87c33c700 1 -- 172.21.15.3:0/4275260070 shutdown_connections 2023-10-03T10:14:46.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.229+0000 7fc87c33c700 1 --2- 172.21.15.3:0/4275260070 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc87409f650 0x7fc87406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.229+0000 7fc87c33c700 1 --2- 172.21.15.3:0/4275260070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc87409e930 0x7fc87406b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.229+0000 7fc87c33c700 1 --2- 172.21.15.3:0/4275260070 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc87409f240 0x7fc87406bdf0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.229+0000 7fc87c33c700 1 -- 172.21.15.3:0/4275260070 >> 172.21.15.3:0/4275260070 conn(0x7fc8740a4210 msgr2=0x7fc8741b8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:46.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.229+0000 7fc87c33c700 1 -- 172.21.15.3:0/4275260070 shutdown_connections 2023-10-03T10:14:46.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.229+0000 7fc87c33c700 1 -- 172.21.15.3:0/4275260070 wait complete. 2023-10-03T10:14:46.244 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:46.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.386+0000 7fea66a56700 1 -- 172.21.15.3:0/4269097761 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fea601237f0 con 0x7fea600a0a30 2023-10-03T10:14:46.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.386+0000 7fea5cff9700 1 -- 172.21.15.3:0/4269097761 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fea50017020 con 0x7fea600a0a30 2023-10-03T10:14:46.386 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:46.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.388+0000 7fd3a2d36700 1 -- 172.21.15.3:0/818132723 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd39c11bc10 con 0x7fd39c09e000 2023-10-03T10:14:46.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.388+0000 7fd3997fa700 1 -- 172.21.15.3:0/818132723 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd388021090 con 0x7fd39c09e000 2023-10-03T10:14:46.389 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:46.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.389+0000 7fea66a56700 1 -- 172.21.15.3:0/4269097761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea600a0a30 msgr2=0x7fea60120310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:46.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.389+0000 7fea66a56700 1 --2- 172.21.15.3:0/4269097761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea600a0a30 0x7fea60120310 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fea5000e9b0 tx=0x7fea5000ecc0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.389+0000 7fea66a56700 1 -- 172.21.15.3:0/4269097761 shutdown_connections 2023-10-03T10:14:46.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.389+0000 7fea66a56700 1 --2- 172.21.15.3:0/4269097761 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fea60133fa0 0x7fea60120d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.389+0000 7fea66a56700 1 --2- 172.21.15.3:0/4269097761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea600a0a30 0x7fea60120310 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.389+0000 7fea66a56700 1 --2- 172.21.15.3:0/4269097761 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fea600a1340 0x7fea60120850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.389+0000 7fea66a56700 1 -- 172.21.15.3:0/4269097761 >> 172.21.15.3:0/4269097761 conn(0x7fea600a6550 msgr2=0x7fea601bb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:46.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.390+0000 7fea66a56700 1 -- 172.21.15.3:0/4269097761 shutdown_connections 2023-10-03T10:14:46.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.390+0000 7fea66a56700 1 -- 172.21.15.3:0/4269097761 wait complete. 2023-10-03T10:14:46.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.391+0000 7fd3a2d36700 1 -- 172.21.15.3:0/818132723 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd39c09e000 msgr2=0x7fd39c1c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:46.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.391+0000 7fd3a2d36700 1 --2- 172.21.15.3:0/818132723 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd39c09e000 0x7fd39c1c66f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd38800e980 tx=0x7fd38800ec90 comp rx=0 tx=0).stop 2023-10-03T10:14:46.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.391+0000 7fd3a2d36700 1 -- 172.21.15.3:0/818132723 shutdown_connections 2023-10-03T10:14:46.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.391+0000 7fd3a2d36700 1 --2- 172.21.15.3:0/818132723 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd39c09e410 0x7fd39c119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.391+0000 7fd3a2d36700 1 --2- 172.21.15.3:0/818132723 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd39c09d6f0 0x7fd39c1c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.391+0000 7fd3a2d36700 1 --2- 172.21.15.3:0/818132723 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd39c09e000 0x7fd39c1c66f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.391+0000 7fd3a2d36700 1 -- 172.21.15.3:0/818132723 >> 172.21.15.3:0/818132723 conn(0x7fd39c066f50 msgr2=0x7fd39c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:46.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.391+0000 7fd3a2d36700 1 -- 172.21.15.3:0/818132723 shutdown_connections 2023-10-03T10:14:46.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.391+0000 7fd3a2d36700 1 -- 172.21.15.3:0/818132723 wait complete. 2023-10-03T10:14:46.406 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:46.410 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:46.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.429+0000 7f08e6616700 1 -- 172.21.15.3:0/2033274692 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f08e01a71b0 con 0x7f08e009e930 2023-10-03T10:14:46.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.429+0000 7f08dcff9700 1 -- 172.21.15.3:0/2033274692 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f08e01a71b0 con 0x7f08e009e930 2023-10-03T10:14:46.429 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:46.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.431+0000 7f08e6616700 1 -- 172.21.15.3:0/2033274692 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08e009e930 msgr2=0x7f08e01a3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:46.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.431+0000 7f08e6616700 1 --2- 172.21.15.3:0/2033274692 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08e009e930 0x7f08e01a3cd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f08d00090b0 tx=0x7f08d0004750 comp rx=0 tx=0).stop 2023-10-03T10:14:46.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.431+0000 7f08e6616700 1 -- 172.21.15.3:0/2033274692 shutdown_connections 2023-10-03T10:14:46.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.431+0000 7f08e6616700 1 --2- 172.21.15.3:0/2033274692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08e009f650 0x7f08e01a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.431+0000 7f08e6616700 1 --2- 172.21.15.3:0/2033274692 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08e009e930 0x7f08e01a3cd0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.431+0000 7f08e6616700 1 --2- 172.21.15.3:0/2033274692 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08e009f240 0x7f08e01a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:46.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.431+0000 7f08e6616700 1 -- 172.21.15.3:0/2033274692 >> 172.21.15.3:0/2033274692 conn(0x7f08e00a4210 msgr2=0x7f08e01b8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:46.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.431+0000 7f08e6616700 1 -- 172.21.15.3:0/2033274692 shutdown_connections 2023-10-03T10:14:46.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:46.431+0000 7f08e6616700 1 -- 172.21.15.3:0/2033274692 wait complete. 2023-10-03T10:14:46.445 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:47.244 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:47.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.391+0000 7f47b0c40700 1 Processor -- start 2023-10-03T10:14:47.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.391+0000 7f47b0c40700 1 -- start start 2023-10-03T10:14:47.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.392+0000 7f47b0c40700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f47ac09e930 0x7f47ac09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:47.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.392+0000 7f47b0c40700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f47ac1342f0 0x7f47ac13be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:47.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.392+0000 7f47b0c40700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47ac06c9f0 con 0x7f47ac09f240 2023-10-03T10:14:47.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.392+0000 7f47b0c40700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47ac06c670 con 0x7f47ac09e930 2023-10-03T10:14:47.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.392+0000 7f47b0c40700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac06bc90 con 0x7f47ac1342f0 2023-10-03T10:14:47.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.392+0000 7f47aa59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f47ac09e930 0x7f47ac09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:47.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.392+0000 7f47aad9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f47ac1342f0 0x7f47ac13be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:47.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.392+0000 7f47aa59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f47ac09e930 0x7f47ac09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56232/0 (socket says 172.21.15.3:56232) 2023-10-03T10:14:47.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.392+0000 7f47aad9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f47ac1342f0 0x7f47ac13be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59110/0 (socket says 172.21.15.3:59110) 2023-10-03T10:14:47.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.392+0000 7f47aa59c700 1 -- 172.21.15.3:0/3640574899 learned_addr learned my addr 172.21.15.3:0/3640574899 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:47.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.392+0000 7f47aa59c700 1 -- 172.21.15.3:0/3640574899 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f47ac1342f0 msgr2=0x7f47ac13be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:47.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.392+0000 7f47aa59c700 1 --2- 172.21.15.3:0/3640574899 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f47ac1342f0 0x7f47ac13be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.392+0000 7f47aa59c700 1 -- 172.21.15.3:0/3640574899 >> v1:172.21.15.3:6789/0 conn(0x7f47ac09f240 legacy=0x7f47ac133be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:47.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.392+0000 7f47aa59c700 1 -- 172.21.15.3:0/3640574899 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ac06d860 con 0x7f47ac09e930 2023-10-03T10:14:47.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.392+0000 7f47a959a700 1 -- 172.21.15.3:0/3640574899 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4004024375 0 0) 0x7f4794001110 con 0x7f47ac09f240 2023-10-03T10:14:47.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.392+0000 7f47aad9d700 1 --2- 172.21.15.3:0/3640574899 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f47ac1342f0 0x7f47ac13be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:47.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.393+0000 7f47aa59c700 1 --2- 172.21.15.3:0/3640574899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47ac09e930 0x7f47ac09ed00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f479c0098f0 tx=0x7f479c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd86b961b9d18bed server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:47.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.393+0000 7f47a959a700 1 -- 172.21.15.3:0/3640574899 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c011830 con 0x7f47ac09e930 2023-10-03T10:14:47.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.393+0000 7f47a959a700 1 -- 172.21.15.3:0/3640574899 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479c011990 con 0x7f47ac09e930 2023-10-03T10:14:47.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.393+0000 7f47a959a700 1 -- 172.21.15.3:0/3640574899 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c011d20 con 0x7f47ac09e930 2023-10-03T10:14:47.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.393+0000 7f47b0c40700 1 -- 172.21.15.3:0/3640574899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47ac09e930 msgr2=0x7f47ac09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:47.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.393+0000 7f47b0c40700 1 --2- 172.21.15.3:0/3640574899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47ac09e930 0x7f47ac09ed00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f479c0098f0 tx=0x7f479c009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:47.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.393+0000 7f47b0c40700 1 -- 172.21.15.3:0/3640574899 shutdown_connections 2023-10-03T10:14:47.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.393+0000 7f47b0c40700 1 --2- 172.21.15.3:0/3640574899 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f47ac1342f0 0x7f47ac13be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.393+0000 7f47b0c40700 1 --2- 172.21.15.3:0/3640574899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47ac09e930 0x7f47ac09ed00 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.393+0000 7f47b0c40700 1 -- 172.21.15.3:0/3640574899 >> 172.21.15.3:0/3640574899 conn(0x7f47ac0a4210 msgr2=0x7f47ac0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:47.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.393+0000 7f47b0c40700 1 -- 172.21.15.3:0/3640574899 shutdown_connections 2023-10-03T10:14:47.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.393+0000 7f47b0c40700 1 -- 172.21.15.3:0/3640574899 wait complete. 2023-10-03T10:14:47.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.395+0000 7f47b0c40700 1 Processor -- start 2023-10-03T10:14:47.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.395+0000 7f47b0c40700 1 -- start start 2023-10-03T10:14:47.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.395+0000 7f47b0c40700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47ac09e930 0x7f47ac122880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:47.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.395+0000 7f47b0c40700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f47ac09f240 0x7f47ac122dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:47.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.395+0000 7f47b0c40700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47ac1342f0 0x7f47ac123300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:47.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.395+0000 7f47b0c40700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac133480 con 0x7f47ac1342f0 2023-10-03T10:14:47.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.395+0000 7f47b0c40700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac125a80 con 0x7f47ac09f240 2023-10-03T10:14:47.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.395+0000 7f47b0c40700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac1244c0 con 0x7f47ac09e930 2023-10-03T10:14:47.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.395+0000 7f47a9d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f47ac09f240 0x7f47ac122dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:47.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.395+0000 7f47aa59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47ac09e930 0x7f47ac122880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:47.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.395+0000 7f47a9d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f47ac09f240 0x7f47ac122dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59116/0 (socket says 172.21.15.3:59116) 2023-10-03T10:14:47.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.395+0000 7f47aa59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47ac09e930 0x7f47ac122880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57424/0 (socket says 172.21.15.3:57424) 2023-10-03T10:14:47.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.395+0000 7f47a9d9b700 1 -- 172.21.15.3:0/2673205466 learned_addr learned my addr 172.21.15.3:0/2673205466 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:47.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.395+0000 7f47aad9d700 1 --2- 172.21.15.3:0/2673205466 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47ac1342f0 0x7f47ac123300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:47.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.396+0000 7f47a9d9b700 1 -- 172.21.15.3:0/2673205466 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47ac09e930 msgr2=0x7f47ac122880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:47.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.396+0000 7f47a9d9b700 1 --2- 172.21.15.3:0/2673205466 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47ac09e930 0x7f47ac122880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.396+0000 7f47a9d9b700 1 -- 172.21.15.3:0/2673205466 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47ac1342f0 msgr2=0x7f47ac123300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:47.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.396+0000 7f47a9d9b700 1 --2- 172.21.15.3:0/2673205466 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47ac1342f0 0x7f47ac123300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.396+0000 7f47a9d9b700 1 -- 172.21.15.3:0/2673205466 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a4009a70 con 0x7f47ac09f240 2023-10-03T10:14:47.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.396+0000 7f47aad9d700 1 --2- 172.21.15.3:0/2673205466 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47ac1342f0 0x7f47ac123300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:47.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.396+0000 7f47a9d9b700 1 --2- 172.21.15.3:0/2673205466 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f47ac09f240 0x7f47ac122dc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f479400c7b0 tx=0x7f479400cac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:47.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.396+0000 7f47a2ffd700 1 -- 172.21.15.3:0/2673205466 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47940054e0 con 0x7f47ac09f240 2023-10-03T10:14:47.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.396+0000 7f47a2ffd700 1 -- 172.21.15.3:0/2673205466 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4794003680 con 0x7f47ac09f240 2023-10-03T10:14:47.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.396+0000 7f47b0c40700 1 -- 172.21.15.3:0/2673205466 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f479c009550 con 0x7f47ac09f240 2023-10-03T10:14:47.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.396+0000 7f47a2ffd700 1 -- 172.21.15.3:0/2673205466 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4794003a30 con 0x7f47ac09f240 2023-10-03T10:14:47.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.396+0000 7f47b0c40700 1 -- 172.21.15.3:0/2673205466 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47ac06cbb0 con 0x7f47ac09f240 2023-10-03T10:14:47.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.397+0000 7f47a2ffd700 1 -- 172.21.15.3:0/2673205466 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4794003b90 con 0x7f47ac09f240 2023-10-03T10:14:47.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.397+0000 7f47a2ffd700 1 -- 172.21.15.3:0/2673205466 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f479403e970 con 0x7f47ac09f240 2023-10-03T10:14:47.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.397+0000 7f47b0c40700 1 -- 172.21.15.3:0/2673205466 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47ac06c9f0 con 0x7f47ac09f240 2023-10-03T10:14:47.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.402+0000 7f47a2ffd700 1 -- 172.21.15.3:0/2673205466 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f47ac06c9f0 con 0x7f47ac09f240 2023-10-03T10:14:47.407 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:47.410 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:47.446 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:47.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.562+0000 7f637d796700 1 Processor -- start 2023-10-03T10:14:47.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.562+0000 7f637d796700 1 -- start start 2023-10-03T10:14:47.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.563+0000 7f637d796700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f637809e930 0x7f637809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:47.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.563+0000 7f637d796700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f637809f240 0x7f63781317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:47.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.563+0000 7f637d796700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f637809f650 0x7f6378131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:47.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.563+0000 7f637d796700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6378070940 con 0x7f637809e930 2023-10-03T10:14:47.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.563+0000 7f637d796700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f637806fd60 con 0x7f637809f240 2023-10-03T10:14:47.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.563+0000 7f637d796700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f637806fa80 con 0x7f637809f650 2023-10-03T10:14:47.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.563+0000 7f63767fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f637809f240 0x7f63781317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:47.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.563+0000 7f6376ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f637809e930 0x7f637809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:47.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.563+0000 7f63767fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f637809f240 0x7f63781317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59132/0 (socket says 172.21.15.3:59132) 2023-10-03T10:14:47.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.563+0000 7f6376ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f637809e930 0x7f637809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56248/0 (socket says 172.21.15.3:56248) 2023-10-03T10:14:47.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.563+0000 7f63767fc700 1 -- 172.21.15.3:0/741692154 learned_addr learned my addr 172.21.15.3:0/741692154 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:47.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.563+0000 7f63777fe700 1 --2- 172.21.15.3:0/741692154 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f637809f650 0x7f6378131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:47.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.564+0000 7f63767fc700 1 -- 172.21.15.3:0/741692154 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f637809f650 msgr2=0x7f6378131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:47.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.564+0000 7f63767fc700 1 --2- 172.21.15.3:0/741692154 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f637809f650 0x7f6378131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.564+0000 7f63767fc700 1 -- 172.21.15.3:0/741692154 >> v2:172.21.15.3:3300/0 conn(0x7f637809e930 msgr2=0x7f637809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:47.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.564+0000 7f63767fc700 1 --2- 172.21.15.3:0/741692154 >> v2:172.21.15.3:3300/0 conn(0x7f637809e930 0x7f637809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.564+0000 7f63767fc700 1 -- 172.21.15.3:0/741692154 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637809fb60 con 0x7f637809f240 2023-10-03T10:14:47.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.564+0000 7f6376ffd700 1 --2- 172.21.15.3:0/741692154 >> v2:172.21.15.3:3300/0 conn(0x7f637809e930 0x7f637809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:47.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.564+0000 7f63777fe700 1 --2- 172.21.15.3:0/741692154 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f637809f650 0x7f6378131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:14:47.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.564+0000 7f63767fc700 1 --2- 172.21.15.3:0/741692154 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f637809f240 0x7f63781317f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f636c0098f0 tx=0x7f636c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f311e6cbec4ae66f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:47.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.564+0000 7f6375ffb700 1 -- 172.21.15.3:0/741692154 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c010a90 con 0x7f637809f240 2023-10-03T10:14:47.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.564+0000 7f6375ffb700 1 -- 172.21.15.3:0/741692154 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636c010bf0 con 0x7f637809f240 2023-10-03T10:14:47.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.564+0000 7f6375ffb700 1 -- 172.21.15.3:0/741692154 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c018440 con 0x7f637809f240 2023-10-03T10:14:47.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.565+0000 7f637d796700 1 -- 172.21.15.3:0/741692154 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f637809f240 msgr2=0x7f63781317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:47.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.565+0000 7f637d796700 1 --2- 172.21.15.3:0/741692154 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f637809f240 0x7f63781317f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f636c0098f0 tx=0x7f636c009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:47.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.565+0000 7f637d796700 1 -- 172.21.15.3:0/741692154 shutdown_connections 2023-10-03T10:14:47.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.565+0000 7f637d796700 1 --2- 172.21.15.3:0/741692154 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f637809f650 0x7f6378131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.565+0000 7f637d796700 1 --2- 172.21.15.3:0/741692154 >> v2:172.21.15.3:3300/0 conn(0x7f637809e930 0x7f637809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.565+0000 7f637d796700 1 --2- 172.21.15.3:0/741692154 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f637809f240 0x7f63781317f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.565+0000 7f637d796700 1 -- 172.21.15.3:0/741692154 >> 172.21.15.3:0/741692154 conn(0x7f63780a4210 msgr2=0x7f63780a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:47.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.565+0000 7f637d796700 1 -- 172.21.15.3:0/741692154 shutdown_connections 2023-10-03T10:14:47.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.565+0000 7f637d796700 1 -- 172.21.15.3:0/741692154 wait complete. 2023-10-03T10:14:47.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.567+0000 7f637d796700 1 Processor -- start 2023-10-03T10:14:47.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.567+0000 7f637d796700 1 -- start start 2023-10-03T10:14:47.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.568+0000 7f637d796700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f637809e930 0x7f637811d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:47.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.568+0000 7f637d796700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f637809f240 0x7f637811d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:47.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.568+0000 7f637d796700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f637809f650 0x7f637811dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:47.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.568+0000 7f637d796700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637811f740 con 0x7f637809e930 2023-10-03T10:14:47.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.568+0000 7f637d796700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6378071d20 con 0x7f637809f240 2023-10-03T10:14:47.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.568+0000 7f637d796700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f63780a2fd0 con 0x7f637809f650 2023-10-03T10:14:47.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.568+0000 7f6376ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f637809e930 0x7f637811d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:47.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.568+0000 7f6376ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f637809e930 0x7f637811d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56252/0 (socket says 172.21.15.3:56252) 2023-10-03T10:14:47.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.568+0000 7f6376ffd700 1 -- 172.21.15.3:0/889538614 learned_addr learned my addr 172.21.15.3:0/889538614 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:47.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.568+0000 7f63777fe700 1 --2- 172.21.15.3:0/889538614 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f637809f650 0x7f637811dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:47.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.568+0000 7f63767fc700 1 --2- 172.21.15.3:0/889538614 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f637809f240 0x7f637811d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:47.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.569+0000 7f63767fc700 1 -- 172.21.15.3:0/889538614 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f637809f650 msgr2=0x7f637811dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:47.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.569+0000 7f63767fc700 1 --2- 172.21.15.3:0/889538614 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f637809f650 0x7f637811dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.569+0000 7f63767fc700 1 -- 172.21.15.3:0/889538614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f637809e930 msgr2=0x7f637811d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:47.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.569+0000 7f63767fc700 1 --2- 172.21.15.3:0/889538614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f637809e930 0x7f637811d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.569+0000 7f63767fc700 1 -- 172.21.15.3:0/889538614 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63680094a0 con 0x7f637809f240 2023-10-03T10:14:47.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.569+0000 7f63777fe700 1 --2- 172.21.15.3:0/889538614 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f637809f650 0x7f637811dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:47.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.569+0000 7f63767fc700 1 --2- 172.21.15.3:0/889538614 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f637809f240 0x7f637811d7a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f636c01a000 tx=0x7f636c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:47.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.569+0000 7f635f7fe700 1 -- 172.21.15.3:0/889538614 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c003ab0 con 0x7f637809f240 2023-10-03T10:14:47.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.569+0000 7f637d796700 1 -- 172.21.15.3:0/889538614 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f636c009550 con 0x7f637809f240 2023-10-03T10:14:47.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.569+0000 7f637d796700 1 -- 172.21.15.3:0/889538614 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63780a0190 con 0x7f637809f240 2023-10-03T10:14:47.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.570+0000 7f635f7fe700 1 -- 172.21.15.3:0/889538614 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636c010a50 con 0x7f637809f240 2023-10-03T10:14:47.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.570+0000 7f635f7fe700 1 -- 172.21.15.3:0/889538614 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c003e10 con 0x7f637809f240 2023-10-03T10:14:47.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.570+0000 7f635f7fe700 1 -- 172.21.15.3:0/889538614 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f636c01f070 con 0x7f637809f240 2023-10-03T10:14:47.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.570+0000 7f635f7fe700 1 -- 172.21.15.3:0/889538614 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f636c044a40 con 0x7f637809f240 2023-10-03T10:14:47.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.571+0000 7f637d796700 1 -- 172.21.15.3:0/889538614 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63580052e0 con 0x7f637809f240 2023-10-03T10:14:47.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.581+0000 7f635f7fe700 1 -- 172.21.15.3:0/889538614 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f636c027080 con 0x7f637809f240 2023-10-03T10:14:47.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.596+0000 7faa067fc700 1 Processor -- start 2023-10-03T10:14:47.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.596+0000 7faa067fc700 1 -- start start 2023-10-03T10:14:47.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.596+0000 7faa067fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7faa0006a770 0x7faa0006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:47.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.596+0000 7faa067fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faa00131d10 0x7faa00139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:47.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.597+0000 7faa067fc700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa0009ece0 con 0x7faa0006b080 2023-10-03T10:14:47.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.597+0000 7faa067fc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa0009e760 con 0x7faa00131d10 2023-10-03T10:14:47.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.597+0000 7faa067fc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa0009e040 con 0x7faa0006a770 2023-10-03T10:14:47.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.597+0000 7fa9fffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7faa0006a770 0x7faa0006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:47.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.597+0000 7fa9fffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7faa0006a770 0x7faa0006ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57446/0 (socket says 172.21.15.3:57446) 2023-10-03T10:14:47.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.597+0000 7fa9fffff700 1 -- 172.21.15.3:0/1761162167 learned_addr learned my addr 172.21.15.3:0/1761162167 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:47.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.597+0000 7faa04d99700 1 --2- 172.21.15.3:0/1761162167 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faa00131d10 0x7faa00139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:47.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.597+0000 7fa9fffff700 1 -- 172.21.15.3:0/1761162167 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faa00131d10 msgr2=0x7faa00139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:47.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.597+0000 7fa9fffff700 1 --2- 172.21.15.3:0/1761162167 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faa00131d10 0x7faa00139860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.597+0000 7fa9fffff700 1 -- 172.21.15.3:0/1761162167 >> v1:172.21.15.3:6789/0 conn(0x7faa0006b080 legacy=0x7faa00131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:47.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.597+0000 7fa9fffff700 1 -- 172.21.15.3:0/1761162167 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0009f630 con 0x7faa0006a770 2023-10-03T10:14:47.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.597+0000 7fa9fffff700 1 --2- 172.21.15.3:0/1761162167 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7faa0006a770 0x7faa0006ab40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009920 tx=0x7fa9f0009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2176e7c7852695b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:47.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.598+0000 7fa9feffd700 1 -- 172.21.15.3:0/1761162167 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0011830 con 0x7faa0006a770 2023-10-03T10:14:47.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.598+0000 7fa9feffd700 1 -- 172.21.15.3:0/1761162167 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f0011990 con 0x7faa0006a770 2023-10-03T10:14:47.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.598+0000 7fa9feffd700 1 -- 172.21.15.3:0/1761162167 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0011d20 con 0x7faa0006a770 2023-10-03T10:14:47.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.598+0000 7faa067fc700 1 -- 172.21.15.3:0/1761162167 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7faa0006a770 msgr2=0x7faa0006ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:47.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.598+0000 7faa067fc700 1 --2- 172.21.15.3:0/1761162167 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7faa0006a770 0x7faa0006ab40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009920 tx=0x7fa9f0009c30 comp rx=0 tx=0).stop 2023-10-03T10:14:47.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.598+0000 7faa067fc700 1 -- 172.21.15.3:0/1761162167 shutdown_connections 2023-10-03T10:14:47.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.598+0000 7faa067fc700 1 --2- 172.21.15.3:0/1761162167 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faa00131d10 0x7faa00139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.598+0000 7faa067fc700 1 --2- 172.21.15.3:0/1761162167 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7faa0006a770 0x7faa0006ab40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.598+0000 7faa067fc700 1 -- 172.21.15.3:0/1761162167 >> 172.21.15.3:0/1761162167 conn(0x7faa00066f50 msgr2=0x7faa0009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:47.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.598+0000 7faa067fc700 1 -- 172.21.15.3:0/1761162167 shutdown_connections 2023-10-03T10:14:47.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.598+0000 7faa067fc700 1 -- 172.21.15.3:0/1761162167 wait complete. 2023-10-03T10:14:47.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.600+0000 7faa067fc700 1 Processor -- start 2023-10-03T10:14:47.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.600+0000 7faa067fc700 1 -- start start 2023-10-03T10:14:47.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.601+0000 7faa067fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7faa0006a770 0x7faa001c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:47.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.601+0000 7faa067fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faa0006b080 0x7faa001c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:47.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.601+0000 7faa067fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa00131d10 0x7faa0011e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:47.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.601+0000 7faa067fc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa00130e50 con 0x7faa00131d10 2023-10-03T10:14:47.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.601+0000 7faa067fc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa001b6ea0 con 0x7faa0006b080 2023-10-03T10:14:47.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.601+0000 7faa067fc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa001220e0 con 0x7faa0006a770 2023-10-03T10:14:47.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.601+0000 7fa9fffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7faa0006a770 0x7faa001c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:47.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.601+0000 7fa9fffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7faa0006a770 0x7faa001c4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57448/0 (socket says 172.21.15.3:57448) 2023-10-03T10:14:47.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.601+0000 7fa9fffff700 1 -- 172.21.15.3:0/166826621 learned_addr learned my addr 172.21.15.3:0/166826621 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:47.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.601+0000 7fa9ff7fe700 1 --2- 172.21.15.3:0/166826621 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faa0006b080 0x7faa001c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:47.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.601+0000 7faa04d99700 1 --2- 172.21.15.3:0/166826621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa00131d10 0x7faa0011e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:47.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.601+0000 7fa9ff7fe700 1 -- 172.21.15.3:0/166826621 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7faa0006a770 msgr2=0x7faa001c4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:47.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.601+0000 7fa9ff7fe700 1 --2- 172.21.15.3:0/166826621 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7faa0006a770 0x7faa001c4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.601+0000 7fa9ff7fe700 1 -- 172.21.15.3:0/166826621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa00131d10 msgr2=0x7faa0011e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:47.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.601+0000 7fa9ff7fe700 1 --2- 172.21.15.3:0/166826621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa00131d10 0x7faa0011e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.601+0000 7fa9ff7fe700 1 -- 172.21.15.3:0/166826621 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f0009550 con 0x7faa0006b080 2023-10-03T10:14:47.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.602+0000 7fa9ff7fe700 1 --2- 172.21.15.3:0/166826621 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faa0006b080 0x7faa001c4aa0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4000f60 tx=0x7fa9f400c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:47.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.602+0000 7fa9fcff9700 1 -- 172.21.15.3:0/166826621 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f40050d0 con 0x7faa0006b080 2023-10-03T10:14:47.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.602+0000 7fa9fcff9700 1 -- 172.21.15.3:0/166826621 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f4005230 con 0x7faa0006b080 2023-10-03T10:14:47.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.602+0000 7faa067fc700 1 -- 172.21.15.3:0/166826621 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa0009c290 con 0x7faa0006b080 2023-10-03T10:14:47.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.602+0000 7fa9fcff9700 1 -- 172.21.15.3:0/166826621 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4003680 con 0x7faa0006b080 2023-10-03T10:14:47.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.602+0000 7faa067fc700 1 -- 172.21.15.3:0/166826621 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa00071eb0 con 0x7faa0006b080 2023-10-03T10:14:47.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.603+0000 7fa9fcff9700 1 -- 172.21.15.3:0/166826621 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa9f40037e0 con 0x7faa0006b080 2023-10-03T10:14:47.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.603+0000 7faa067fc700 1 -- 172.21.15.3:0/166826621 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa00074ea0 con 0x7faa0006b080 2023-10-03T10:14:47.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.604+0000 7fa9fcff9700 1 -- 172.21.15.3:0/166826621 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa9f403e9e0 con 0x7faa0006b080 2023-10-03T10:14:47.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.608+0000 7fa9fcff9700 1 -- 172.21.15.3:0/166826621 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa9f4011590 con 0x7faa0006b080 2023-10-03T10:14:47.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.614+0000 7f8aef10b700 1 Processor -- start 2023-10-03T10:14:47.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.614+0000 7f8aef10b700 1 -- start start 2023-10-03T10:14:47.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.614+0000 7f8aef10b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ae809de00 0x7f8ae809e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:47.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.614+0000 7f8aef10b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8ae813d1f0 0x7f8ae813f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:47.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.614+0000 7f8aef10b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae809a810 con 0x7f8ae806a770 2023-10-03T10:14:47.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.614+0000 7f8aef10b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae806ab40 con 0x7f8ae809de00 2023-10-03T10:14:47.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.614+0000 7f8aef10b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae809ebc0 con 0x7f8ae813d1f0 2023-10-03T10:14:47.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.615+0000 7f8ae7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ae809de00 0x7f8ae809e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:47.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.615+0000 7f8aecea7700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8ae806a770 0x7f8ae809d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54314/0 (socket says 172.21.15.3:54314) 2023-10-03T10:14:47.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.615+0000 7f8aecea7700 1 -- 172.21.15.3:0/2263954475 learned_addr learned my addr 172.21.15.3:0/2263954475 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:47.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.615+0000 7f8aed6a8700 1 --2- 172.21.15.3:0/2263954475 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8ae813d1f0 0x7f8ae813f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:47.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.615+0000 7f8aed6a8700 1 -- 172.21.15.3:0/2263954475 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ae809de00 msgr2=0x7f8ae809e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:47.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.615+0000 7f8aed6a8700 1 --2- 172.21.15.3:0/2263954475 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ae809de00 0x7f8ae809e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.615+0000 7f8aed6a8700 1 -- 172.21.15.3:0/2263954475 >> v1:172.21.15.3:6789/0 conn(0x7f8ae806a770 legacy=0x7f8ae809d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:47.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.615+0000 7f8aed6a8700 1 -- 172.21.15.3:0/2263954475 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae809e920 con 0x7f8ae813d1f0 2023-10-03T10:14:47.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.615+0000 7f8aed6a8700 1 --2- 172.21.15.3:0/2263954475 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8ae813d1f0 0x7f8ae813f5d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0009dc0 tx=0x7f8ae000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b84692aee178bdcc server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:47.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.616+0000 7f8ae77fe700 1 -- 172.21.15.3:0/2263954475 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae000e070 con 0x7f8ae813d1f0 2023-10-03T10:14:47.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.616+0000 7f8ae77fe700 1 -- 172.21.15.3:0/2263954475 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae000bb20 con 0x7f8ae813d1f0 2023-10-03T10:14:47.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.616+0000 7f8ae77fe700 1 -- 172.21.15.3:0/2263954475 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0012430 con 0x7f8ae813d1f0 2023-10-03T10:14:47.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.616+0000 7f8aef10b700 1 -- 172.21.15.3:0/2263954475 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8ae813d1f0 msgr2=0x7f8ae813f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:47.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.616+0000 7f8aef10b700 1 --2- 172.21.15.3:0/2263954475 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8ae813d1f0 0x7f8ae813f5d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0009dc0 tx=0x7f8ae000b330 comp rx=0 tx=0).stop 2023-10-03T10:14:47.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.616+0000 7f8aef10b700 1 -- 172.21.15.3:0/2263954475 shutdown_connections 2023-10-03T10:14:47.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.616+0000 7f8aef10b700 1 --2- 172.21.15.3:0/2263954475 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8ae813d1f0 0x7f8ae813f5d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.616+0000 7f8aef10b700 1 --2- 172.21.15.3:0/2263954475 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ae809de00 0x7f8ae809e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.617+0000 7f8aef10b700 1 -- 172.21.15.3:0/2263954475 >> 172.21.15.3:0/2263954475 conn(0x7f8ae8066f50 msgr2=0x7f8ae809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:47.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.617+0000 7f8aef10b700 1 -- 172.21.15.3:0/2263954475 shutdown_connections 2023-10-03T10:14:47.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.617+0000 7f8aef10b700 1 -- 172.21.15.3:0/2263954475 wait complete. 2023-10-03T10:14:47.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.618+0000 7f8aef10b700 1 Processor -- start 2023-10-03T10:14:47.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.618+0000 7f8aef10b700 1 -- start start 2023-10-03T10:14:47.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.618+0000 7f8aef10b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae806a770 0x7f8ae8071770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:47.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.618+0000 7f8aef10b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8ae809de00 0x7f8ae813b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:47.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.618+0000 7f8aef10b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ae813d1f0 0x7f8ae813bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:47.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.618+0000 7f8aef10b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae811df40 con 0x7f8ae806a770 2023-10-03T10:14:47.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.618+0000 7f8aef10b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae809f1a0 con 0x7f8ae813d1f0 2023-10-03T10:14:47.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.618+0000 7f8aef10b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae811ea20 con 0x7f8ae809de00 2023-10-03T10:14:47.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.618+0000 7f8aecea7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae806a770 0x7f8ae8071770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:47.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.618+0000 7f8aecea7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae806a770 0x7f8ae8071770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56264/0 (socket says 172.21.15.3:56264) 2023-10-03T10:14:47.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.618+0000 7f8aecea7700 1 -- 172.21.15.3:0/2592299350 learned_addr learned my addr 172.21.15.3:0/2592299350 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:47.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.618+0000 7f8aed6a8700 1 --2- 172.21.15.3:0/2592299350 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ae813d1f0 0x7f8ae813bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:47.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.619+0000 7f8ae7fff700 1 --2- 172.21.15.3:0/2592299350 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8ae809de00 0x7f8ae813b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:47.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.619+0000 7f8aecea7700 1 -- 172.21.15.3:0/2592299350 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8ae809de00 msgr2=0x7f8ae813b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:47.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.619+0000 7f8aecea7700 1 --2- 172.21.15.3:0/2592299350 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8ae809de00 0x7f8ae813b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.619+0000 7f8aecea7700 1 -- 172.21.15.3:0/2592299350 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ae813d1f0 msgr2=0x7f8ae813bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:47.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.619+0000 7f8aecea7700 1 --2- 172.21.15.3:0/2592299350 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ae813d1f0 0x7f8ae813bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.619+0000 7f8aecea7700 1 -- 172.21.15.3:0/2592299350 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae0009a70 con 0x7f8ae806a770 2023-10-03T10:14:47.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.619+0000 7f8aecea7700 1 --2- 172.21.15.3:0/2592299350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae806a770 0x7f8ae8071770 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8adc00c520 tx=0x7f8adc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:47.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.620+0000 7f8ae57fa700 1 -- 172.21.15.3:0/2592299350 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc005d70 con 0x7f8ae806a770 2023-10-03T10:14:47.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.620+0000 7f8ae57fa700 1 -- 172.21.15.3:0/2592299350 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8adc0070d0 con 0x7f8ae806a770 2023-10-03T10:14:47.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.620+0000 7f8ae57fa700 1 -- 172.21.15.3:0/2592299350 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc007440 con 0x7f8ae806a770 2023-10-03T10:14:47.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.620+0000 7f8aef10b700 1 -- 172.21.15.3:0/2592299350 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae813cf20 con 0x7f8ae806a770 2023-10-03T10:14:47.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.620+0000 7f8aef10b700 1 -- 172.21.15.3:0/2592299350 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae80722a0 con 0x7f8ae806a770 2023-10-03T10:14:47.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.621+0000 7f8ae57fa700 1 -- 172.21.15.3:0/2592299350 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8adc0075a0 con 0x7f8ae806a770 2023-10-03T10:14:47.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.621+0000 7f8ae57fa700 1 -- 172.21.15.3:0/2592299350 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8adc03ec70 con 0x7f8ae806a770 2023-10-03T10:14:47.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.622+0000 7f8aef10b700 1 -- 172.21.15.3:0/2592299350 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae811ec40 con 0x7f8ae806a770 2023-10-03T10:14:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.632+0000 7f8ae57fa700 1 -- 172.21.15.3:0/2592299350 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8ae811ec40 con 0x7f8ae806a770 2023-10-03T10:14:47.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.640+0000 7f47b0c40700 1 -- 172.21.15.3:0/2673205466 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f47ac06c9f0 con 0x7f47ac09f240 2023-10-03T10:14:47.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.640+0000 7f47a2ffd700 1 -- 172.21.15.3:0/2673205466 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f47ac06c9f0 con 0x7f47ac09f240 2023-10-03T10:14:47.640 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:47.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.642+0000 7f47b0c40700 1 -- 172.21.15.3:0/2673205466 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f47ac09f240 msgr2=0x7f47ac122dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:47.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.642+0000 7f47b0c40700 1 --2- 172.21.15.3:0/2673205466 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f47ac09f240 0x7f47ac122dc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f479400c7b0 tx=0x7f479400cac0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.642+0000 7f47b0c40700 1 -- 172.21.15.3:0/2673205466 shutdown_connections 2023-10-03T10:14:47.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.642+0000 7f47b0c40700 1 --2- 172.21.15.3:0/2673205466 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47ac1342f0 0x7f47ac123300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.642+0000 7f47b0c40700 1 --2- 172.21.15.3:0/2673205466 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47ac09e930 0x7f47ac122880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.642+0000 7f47b0c40700 1 --2- 172.21.15.3:0/2673205466 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f47ac09f240 0x7f47ac122dc0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.642+0000 7f47b0c40700 1 -- 172.21.15.3:0/2673205466 >> 172.21.15.3:0/2673205466 conn(0x7f47ac0a4210 msgr2=0x7f47ac0a2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:47.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.642+0000 7f47b0c40700 1 -- 172.21.15.3:0/2673205466 shutdown_connections 2023-10-03T10:14:47.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.643+0000 7f47b0c40700 1 -- 172.21.15.3:0/2673205466 wait complete. 2023-10-03T10:14:47.658 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:47.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.804+0000 7f637d796700 1 -- 172.21.15.3:0/889538614 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6358005590 con 0x7f637809f240 2023-10-03T10:14:47.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.805+0000 7f635f7fe700 1 -- 172.21.15.3:0/889538614 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f636c01d020 con 0x7f637809f240 2023-10-03T10:14:47.805 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:47.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.807+0000 7f637d796700 1 -- 172.21.15.3:0/889538614 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f637809f240 msgr2=0x7f637811d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:47.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.807+0000 7f637d796700 1 --2- 172.21.15.3:0/889538614 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f637809f240 0x7f637811d7a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f636c01a000 tx=0x7f636c004750 comp rx=0 tx=0).stop 2023-10-03T10:14:47.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.807+0000 7f637d796700 1 -- 172.21.15.3:0/889538614 shutdown_connections 2023-10-03T10:14:47.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.807+0000 7f637d796700 1 --2- 172.21.15.3:0/889538614 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f637809f650 0x7f637811dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.807+0000 7f637d796700 1 --2- 172.21.15.3:0/889538614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f637809e930 0x7f637811d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.807+0000 7f637d796700 1 --2- 172.21.15.3:0/889538614 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f637809f240 0x7f637811d7a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.807+0000 7f637d796700 1 -- 172.21.15.3:0/889538614 >> 172.21.15.3:0/889538614 conn(0x7f63780a4210 msgr2=0x7f63780a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:47.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.807+0000 7f637d796700 1 -- 172.21.15.3:0/889538614 shutdown_connections 2023-10-03T10:14:47.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.807+0000 7f637d796700 1 -- 172.21.15.3:0/889538614 wait complete. 2023-10-03T10:14:47.821 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:47.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.821+0000 7faa067fc700 1 -- 172.21.15.3:0/166826621 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7faa00121f20 con 0x7faa0006b080 2023-10-03T10:14:47.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.822+0000 7fa9fcff9700 1 -- 172.21.15.3:0/166826621 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faa00121f20 con 0x7faa0006b080 2023-10-03T10:14:47.822 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:47.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.824+0000 7faa067fc700 1 -- 172.21.15.3:0/166826621 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faa0006b080 msgr2=0x7faa001c4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:47.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.824+0000 7faa067fc700 1 --2- 172.21.15.3:0/166826621 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faa0006b080 0x7faa001c4aa0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4000f60 tx=0x7fa9f400c730 comp rx=0 tx=0).stop 2023-10-03T10:14:47.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.824+0000 7faa067fc700 1 -- 172.21.15.3:0/166826621 shutdown_connections 2023-10-03T10:14:47.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.824+0000 7faa067fc700 1 --2- 172.21.15.3:0/166826621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa00131d10 0x7faa0011e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.824+0000 7faa067fc700 1 --2- 172.21.15.3:0/166826621 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7faa0006a770 0x7faa001c4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.824+0000 7faa067fc700 1 --2- 172.21.15.3:0/166826621 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faa0006b080 0x7faa001c4aa0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.824+0000 7faa067fc700 1 -- 172.21.15.3:0/166826621 >> 172.21.15.3:0/166826621 conn(0x7faa00066f50 msgr2=0x7faa000a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:47.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.824+0000 7faa067fc700 1 -- 172.21.15.3:0/166826621 shutdown_connections 2023-10-03T10:14:47.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.824+0000 7faa067fc700 1 -- 172.21.15.3:0/166826621 wait complete. 2023-10-03T10:14:47.837 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:47.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.841+0000 7f8aef10b700 1 -- 172.21.15.3:0/2592299350 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8ae811ec40 con 0x7f8ae806a770 2023-10-03T10:14:47.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.841+0000 7f8ae57fa700 1 -- 172.21.15.3:0/2592299350 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8ae811ec40 con 0x7f8ae806a770 2023-10-03T10:14:47.842 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:47.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.844+0000 7f8aef10b700 1 -- 172.21.15.3:0/2592299350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae806a770 msgr2=0x7f8ae8071770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:47.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.844+0000 7f8aef10b700 1 --2- 172.21.15.3:0/2592299350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae806a770 0x7f8ae8071770 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8adc00c520 tx=0x7f8adc00c830 comp rx=0 tx=0).stop 2023-10-03T10:14:47.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.844+0000 7f8aef10b700 1 -- 172.21.15.3:0/2592299350 shutdown_connections 2023-10-03T10:14:47.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.844+0000 7f8aef10b700 1 --2- 172.21.15.3:0/2592299350 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ae813d1f0 0x7f8ae813bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.844+0000 7f8aef10b700 1 --2- 172.21.15.3:0/2592299350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae806a770 0x7f8ae8071770 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.844+0000 7f8aef10b700 1 --2- 172.21.15.3:0/2592299350 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8ae809de00 0x7f8ae813b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:47.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.844+0000 7f8aef10b700 1 -- 172.21.15.3:0/2592299350 >> 172.21.15.3:0/2592299350 conn(0x7f8ae8066f50 msgr2=0x7f8ae809b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:47.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.844+0000 7f8aef10b700 1 -- 172.21.15.3:0/2592299350 shutdown_connections 2023-10-03T10:14:47.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:47.844+0000 7f8aef10b700 1 -- 172.21.15.3:0/2592299350 wait complete. 2023-10-03T10:14:47.857 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:48.659 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.805+0000 7f74d0b47700 1 Processor -- start 2023-10-03T10:14:48.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.805+0000 7f74d0b47700 1 -- start start 2023-10-03T10:14:48.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.806+0000 7f74d0b47700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74cc0a0a30 0x7f74cc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:48.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.806+0000 7f74d0b47700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f74cc0a1340 0x7f74cc133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:48.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.806+0000 7f74d0b47700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74cc133fa0 0x7f74cc134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:48.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.806+0000 7f74d0b47700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74cc09edd0 con 0x7f74cc0a1340 2023-10-03T10:14:48.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.806+0000 7f74d0b47700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc09e1f0 con 0x7f74cc133fa0 2023-10-03T10:14:48.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.806+0000 7f74d0b47700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc09dc50 con 0x7f74cc0a0a30 2023-10-03T10:14:48.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.806+0000 7f74cad9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74cc133fa0 0x7f74cc134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:48.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.806+0000 7f74c9d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f74cc0a1340 0x7f74cc133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:48.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.806+0000 7f74ca59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74cc0a0a30 0x7f74cc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:48.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.806+0000 7f74cad9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74cc133fa0 0x7f74cc134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59196/0 (socket says 172.21.15.3:59196) 2023-10-03T10:14:48.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.806+0000 7f74c9d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f74cc0a1340 0x7f74cc133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56272/0 (socket says 172.21.15.3:56272) 2023-10-03T10:14:48.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.806+0000 7f74ca59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74cc0a0a30 0x7f74cc0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57470/0 (socket says 172.21.15.3:57470) 2023-10-03T10:14:48.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.806+0000 7f74cad9d700 1 -- 172.21.15.3:0/333640658 learned_addr learned my addr 172.21.15.3:0/333640658 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:48.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.807+0000 7f74cad9d700 1 -- 172.21.15.3:0/333640658 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74cc0a0a30 msgr2=0x7f74cc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:48.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.807+0000 7f74cad9d700 1 --2- 172.21.15.3:0/333640658 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74cc0a0a30 0x7f74cc0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:48.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.807+0000 7f74cad9d700 1 -- 172.21.15.3:0/333640658 >> v2:172.21.15.3:3300/0 conn(0x7f74cc0a1340 msgr2=0x7f74cc133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:48.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.807+0000 7f74cad9d700 1 --2- 172.21.15.3:0/333640658 >> v2:172.21.15.3:3300/0 conn(0x7f74cc0a1340 0x7f74cc133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:48.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.807+0000 7f74cad9d700 1 -- 172.21.15.3:0/333640658 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74cc0a1fe0 con 0x7f74cc133fa0 2023-10-03T10:14:48.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.807+0000 7f74c9d9b700 1 --2- 172.21.15.3:0/333640658 >> v2:172.21.15.3:3300/0 conn(0x7f74cc0a1340 0x7f74cc133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:48.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.807+0000 7f74cad9d700 1 --2- 172.21.15.3:0/333640658 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74cc133fa0 0x7f74cc134410 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f74c4009e80 tx=0x7f74c400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ca42adb69646da2 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:48.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.807+0000 7f74c959a700 1 -- 172.21.15.3:0/333640658 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c400e070 con 0x7f74cc133fa0 2023-10-03T10:14:48.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.807+0000 7f74c959a700 1 -- 172.21.15.3:0/333640658 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c400b8d0 con 0x7f74cc133fa0 2023-10-03T10:14:48.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.807+0000 7f74c959a700 1 -- 172.21.15.3:0/333640658 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c400bc80 con 0x7f74cc133fa0 2023-10-03T10:14:48.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.808+0000 7f74d0b47700 1 -- 172.21.15.3:0/333640658 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74cc133fa0 msgr2=0x7f74cc134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:48.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.808+0000 7f74d0b47700 1 --2- 172.21.15.3:0/333640658 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74cc133fa0 0x7f74cc134410 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f74c4009e80 tx=0x7f74c400b330 comp rx=0 tx=0).stop 2023-10-03T10:14:48.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.808+0000 7f74d0b47700 1 -- 172.21.15.3:0/333640658 shutdown_connections 2023-10-03T10:14:48.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.808+0000 7f74d0b47700 1 --2- 172.21.15.3:0/333640658 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74cc133fa0 0x7f74cc134410 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:48.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.808+0000 7f74d0b47700 1 --2- 172.21.15.3:0/333640658 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74cc0a0a30 0x7f74cc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:48.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.808+0000 7f74d0b47700 1 --2- 172.21.15.3:0/333640658 >> v2:172.21.15.3:3300/0 conn(0x7f74cc0a1340 0x7f74cc133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:48.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.808+0000 7f74d0b47700 1 -- 172.21.15.3:0/333640658 >> 172.21.15.3:0/333640658 conn(0x7f74cc0a6550 msgr2=0x7f74cc0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:48.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.808+0000 7f74d0b47700 1 -- 172.21.15.3:0/333640658 shutdown_connections 2023-10-03T10:14:48.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.808+0000 7f74d0b47700 1 -- 172.21.15.3:0/333640658 wait complete. 2023-10-03T10:14:48.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.809+0000 7f74d0b47700 1 Processor -- start 2023-10-03T10:14:48.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.809+0000 7f74d0b47700 1 -- start start 2023-10-03T10:14:48.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.810+0000 7f74d0b47700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74cc0a0a30 0x7f74cc1200a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:48.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.810+0000 7f74d0b47700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74cc0a1340 0x7f74cc1205e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:48.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.810+0000 7f74d0b47700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74cc133fa0 0x7f74cc120b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:48.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.810+0000 7f74d0b47700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc1233e0 con 0x7f74cc133fa0 2023-10-03T10:14:48.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.810+0000 7f74d0b47700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc122eb0 con 0x7f74cc0a0a30 2023-10-03T10:14:48.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.810+0000 7f74d0b47700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc121d00 con 0x7f74cc0a1340 2023-10-03T10:14:48.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.810+0000 7f74ca59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74cc0a0a30 0x7f74cc1200a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:48.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.810+0000 7f74c9d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74cc0a1340 0x7f74cc1205e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:48.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.810+0000 7f74ca59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74cc0a0a30 0x7f74cc1200a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59202/0 (socket says 172.21.15.3:59202) 2023-10-03T10:14:48.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.810+0000 7f74ca59c700 1 -- 172.21.15.3:0/2401468943 learned_addr learned my addr 172.21.15.3:0/2401468943 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:48.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.810+0000 7f74cad9d700 1 --2- 172.21.15.3:0/2401468943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74cc133fa0 0x7f74cc120b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:48.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.811+0000 7f74c9d9b700 1 -- 172.21.15.3:0/2401468943 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74cc0a0a30 msgr2=0x7f74cc1200a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:48.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.811+0000 7f74c9d9b700 1 --2- 172.21.15.3:0/2401468943 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74cc0a0a30 0x7f74cc1200a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:48.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.811+0000 7f74c9d9b700 1 -- 172.21.15.3:0/2401468943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74cc133fa0 msgr2=0x7f74cc120b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:48.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.811+0000 7f74c9d9b700 1 --2- 172.21.15.3:0/2401468943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74cc133fa0 0x7f74cc120b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:48.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.811+0000 7f74c9d9b700 1 -- 172.21.15.3:0/2401468943 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b40094a0 con 0x7f74cc0a1340 2023-10-03T10:14:48.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.811+0000 7f74ca59c700 1 --2- 172.21.15.3:0/2401468943 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74cc0a0a30 0x7f74cc1200a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:48.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.811+0000 7f74c9d9b700 1 --2- 172.21.15.3:0/2401468943 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74cc0a1340 0x7f74cc1205e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f74b400e9e0 tx=0x7f74b400ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:48.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.811+0000 7f74c2ffd700 1 -- 172.21.15.3:0/2401468943 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b400ee80 con 0x7f74cc0a1340 2023-10-03T10:14:48.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.811+0000 7f74c2ffd700 1 -- 172.21.15.3:0/2401468943 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b400c850 con 0x7f74cc0a1340 2023-10-03T10:14:48.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.811+0000 7f74d0b47700 1 -- 172.21.15.3:0/2401468943 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c4009b30 con 0x7f74cc0a1340 2023-10-03T10:14:48.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.811+0000 7f74c2ffd700 1 -- 172.21.15.3:0/2401468943 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b400cc00 con 0x7f74cc0a1340 2023-10-03T10:14:48.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.811+0000 7f74d0b47700 1 -- 172.21.15.3:0/2401468943 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74cc1238e0 con 0x7f74cc0a1340 2023-10-03T10:14:48.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.812+0000 7f74c2ffd700 1 -- 172.21.15.3:0/2401468943 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f74b400cd60 con 0x7f74cc0a1340 2023-10-03T10:14:48.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.812+0000 7f74c2ffd700 1 -- 172.21.15.3:0/2401468943 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f74b403f880 con 0x7f74cc0a1340 2023-10-03T10:14:48.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.812+0000 7f74d0b47700 1 -- 172.21.15.3:0/2401468943 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74cc1238e0 con 0x7f74cc0a1340 2023-10-03T10:14:48.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.817+0000 7f74c2ffd700 1 -- 172.21.15.3:0/2401468943 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f74cc1238e0 con 0x7f74cc0a1340 2023-10-03T10:14:48.823 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:48.838 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:48.859 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:48.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.984+0000 7f7f387aa700 1 Processor -- start 2023-10-03T10:14:48.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.984+0000 7f7f387aa700 1 -- start start 2023-10-03T10:14:48.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.985+0000 7f7f387aa700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f30130670 0x7f7f30130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:48.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.985+0000 7f7f387aa700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7f300701a0 0x7f7f30138540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:48.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.985+0000 7f7f387aa700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f30130260 con 0x7f7f3006f680 2023-10-03T10:14:48.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.985+0000 7f7f387aa700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f30071ff0 con 0x7f7f300701a0 2023-10-03T10:14:48.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.985+0000 7f7f387aa700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3009ecf0 con 0x7f7f30130670 2023-10-03T10:14:48.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.985+0000 7f7f36546700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f30130670 0x7f7f30130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:48.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.985+0000 7f7f36546700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f30130670 0x7f7f30130a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57476/0 (socket says 172.21.15.3:57476) 2023-10-03T10:14:48.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.985+0000 7f7f36546700 1 -- 172.21.15.3:0/1111874586 learned_addr learned my addr 172.21.15.3:0/1111874586 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:48.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.985+0000 7f7f36d47700 1 --2- 172.21.15.3:0/1111874586 >> v2:172.21.15.3:3300/0 conn(0x7f7f300701a0 0x7f7f30138540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:48.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.986+0000 7f7f36d47700 1 -- 172.21.15.3:0/1111874586 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f30130670 msgr2=0x7f7f30130a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:48.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.986+0000 7f7f36d47700 1 --2- 172.21.15.3:0/1111874586 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f30130670 0x7f7f30130a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:48.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.986+0000 7f7f36d47700 1 -- 172.21.15.3:0/1111874586 >> v1:172.21.15.3:6789/0 conn(0x7f7f3006f680 legacy=0x7f7f3006fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:48.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.986+0000 7f7f36d47700 1 -- 172.21.15.3:0/1111874586 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f30070e70 con 0x7f7f300701a0 2023-10-03T10:14:48.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.986+0000 7f7f36d47700 1 --2- 172.21.15.3:0/1111874586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f300701a0 0x7f7f30138540 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7f28009d90 tx=0x7f7f2800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d248aab0785d10b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:48.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.986+0000 7f7f35544700 1 -- 172.21.15.3:0/1111874586 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2800e070 con 0x7f7f300701a0 2023-10-03T10:14:48.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.986+0000 7f7f35544700 1 -- 172.21.15.3:0/1111874586 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2800bb20 con 0x7f7f300701a0 2023-10-03T10:14:48.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.986+0000 7f7f35544700 1 -- 172.21.15.3:0/1111874586 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f28012430 con 0x7f7f300701a0 2023-10-03T10:14:48.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.987+0000 7f7f387aa700 1 -- 172.21.15.3:0/1111874586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f300701a0 msgr2=0x7f7f30138540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:48.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.987+0000 7f7f387aa700 1 --2- 172.21.15.3:0/1111874586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f300701a0 0x7f7f30138540 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7f28009d90 tx=0x7f7f2800b330 comp rx=0 tx=0).stop 2023-10-03T10:14:48.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.987+0000 7f7f387aa700 1 -- 172.21.15.3:0/1111874586 shutdown_connections 2023-10-03T10:14:48.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.987+0000 7f7f387aa700 1 --2- 172.21.15.3:0/1111874586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f300701a0 0x7f7f30138540 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:48.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.987+0000 7f7f387aa700 1 --2- 172.21.15.3:0/1111874586 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f30130670 0x7f7f30130a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:48.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.987+0000 7f7f387aa700 1 -- 172.21.15.3:0/1111874586 >> 172.21.15.3:0/1111874586 conn(0x7f7f300a2aa0 msgr2=0x7f7f300a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:48.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.987+0000 7f7f387aa700 1 -- 172.21.15.3:0/1111874586 shutdown_connections 2023-10-03T10:14:48.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.987+0000 7f7f387aa700 1 -- 172.21.15.3:0/1111874586 wait complete. 2023-10-03T10:14:48.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.988+0000 7f7f387aa700 1 Processor -- start 2023-10-03T10:14:48.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.988+0000 7f7f387aa700 1 -- start start 2023-10-03T10:14:48.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.989+0000 7f7f387aa700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f3006f680 0x7f7f301c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:48.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.989+0000 7f7f387aa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f300701a0 0x7f7f301c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:48.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.989+0000 7f7f387aa700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f30130670 0x7f7f301c4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:48.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.989+0000 7f7f387aa700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3009e4d0 con 0x7f7f300701a0 2023-10-03T10:14:48.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.989+0000 7f7f387aa700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f30127ec0 con 0x7f7f3006f680 2023-10-03T10:14:48.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.989+0000 7f7f387aa700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f301282f0 con 0x7f7f30130670 2023-10-03T10:14:48.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.989+0000 7f7f36546700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f3006f680 0x7f7f301c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:48.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.989+0000 7f7f36546700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f3006f680 0x7f7f301c8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59206/0 (socket says 172.21.15.3:59206) 2023-10-03T10:14:48.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.989+0000 7f7f36546700 1 -- 172.21.15.3:0/2708765544 learned_addr learned my addr 172.21.15.3:0/2708765544 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:48.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.989+0000 7f7f36546700 1 -- 172.21.15.3:0/2708765544 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f30130670 msgr2=0x7f7f301c4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:48.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.989+0000 7f7f36546700 1 --2- 172.21.15.3:0/2708765544 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f30130670 0x7f7f301c4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:48.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.989+0000 7f7f36546700 1 -- 172.21.15.3:0/2708765544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f300701a0 msgr2=0x7f7f301c88a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:14:48.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.989+0000 7f7f36546700 1 --2- 172.21.15.3:0/2708765544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f300701a0 0x7f7f301c88a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:48.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.989+0000 7f7f36546700 1 -- 172.21.15.3:0/2708765544 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f28009a70 con 0x7f7f3006f680 2023-10-03T10:14:48.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.990+0000 7f7f36546700 1 --2- 172.21.15.3:0/2708765544 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f3006f680 0x7f7f301c8360 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c00e940 tx=0x7f7f2c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:48.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.990+0000 7f7f26ffd700 1 -- 172.21.15.3:0/2708765544 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c0096a0 con 0x7f7f3006f680 2023-10-03T10:14:48.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.990+0000 7f7f26ffd700 1 -- 172.21.15.3:0/2708765544 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2c00c850 con 0x7f7f3006f680 2023-10-03T10:14:48.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.990+0000 7f7f387aa700 1 -- 172.21.15.3:0/2708765544 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f30128b50 con 0x7f7f3006f680 2023-10-03T10:14:48.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.990+0000 7f7f26ffd700 1 -- 172.21.15.3:0/2708765544 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c00cbc0 con 0x7f7f3006f680 2023-10-03T10:14:48.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.990+0000 7f7f387aa700 1 -- 172.21.15.3:0/2708765544 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f301c8de0 con 0x7f7f3006f680 2023-10-03T10:14:48.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.991+0000 7f7f26ffd700 1 -- 172.21.15.3:0/2708765544 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7f2c00cd20 con 0x7f7f3006f680 2023-10-03T10:14:48.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.991+0000 7f7f26ffd700 1 -- 172.21.15.3:0/2708765544 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7f2c03f780 con 0x7f7f3006f680 2023-10-03T10:14:48.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.991+0000 7f7f387aa700 1 -- 172.21.15.3:0/2708765544 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f30074ea0 con 0x7f7f3006f680 2023-10-03T10:14:48.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:48.996+0000 7f7f26ffd700 1 -- 172.21.15.3:0/2708765544 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7f2c0232f0 con 0x7f7f3006f680 2023-10-03T10:14:49.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.022+0000 7f9d774ff700 1 Processor -- start 2023-10-03T10:14:49.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.023+0000 7f9d774ff700 1 -- start start 2023-10-03T10:14:49.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.023+0000 7f9d774ff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d70130540 0x7f9d70130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:49.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.023+0000 7f9d774ff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9d70130e50 0x7f9d7006f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:49.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.024+0000 7f9d774ff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d7006fb20 0x7f9d7006ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:49.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.024+0000 7f9d774ff700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d700704b0 con 0x7f9d70130e50 2023-10-03T10:14:49.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.024+0000 7f9d774ff700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7009a390 con 0x7f9d7006fb20 2023-10-03T10:14:49.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.024+0000 7f9d774ff700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d700716a0 con 0x7f9d70130540 2023-10-03T10:14:49.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.024+0000 7f9d75a9c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d7006fb20 0x7f9d7006ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:49.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.024+0000 7f9d75a9c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d7006fb20 0x7f9d7006ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59218/0 (socket says 172.21.15.3:59218) 2023-10-03T10:14:49.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.024+0000 7f9d75a9c700 1 -- 172.21.15.3:0/2416882239 learned_addr learned my addr 172.21.15.3:0/2416882239 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:49.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.024+0000 7f9d74a9a700 1 --2- 172.21.15.3:0/2416882239 >> v2:172.21.15.3:3300/0 conn(0x7f9d70130e50 0x7f9d7006f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:49.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.024+0000 7f9d75a9c700 1 -- 172.21.15.3:0/2416882239 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d70130540 msgr2=0x7f9d70130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:49.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.024+0000 7f9d75a9c700 1 --2- 172.21.15.3:0/2416882239 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d70130540 0x7f9d70130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:49.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.024+0000 7f9d75a9c700 1 -- 172.21.15.3:0/2416882239 >> v2:172.21.15.3:3300/0 conn(0x7f9d70130e50 msgr2=0x7f9d7006f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:49.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.024+0000 7f9d75a9c700 1 --2- 172.21.15.3:0/2416882239 >> v2:172.21.15.3:3300/0 conn(0x7f9d70130e50 0x7f9d7006f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:49.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.024+0000 7f9d75a9c700 1 -- 172.21.15.3:0/2416882239 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d70071290 con 0x7f9d7006fb20 2023-10-03T10:14:49.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.024+0000 7f9d75a9c700 1 --2- 172.21.15.3:0/2416882239 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d7006fb20 0x7f9d7006ff70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9d68009e50 tx=0x7f9d6800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e2ab5963345f89b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:49.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.025+0000 7f9d67fff700 1 -- 172.21.15.3:0/2416882239 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6800e070 con 0x7f9d7006fb20 2023-10-03T10:14:49.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.025+0000 7f9d67fff700 1 -- 172.21.15.3:0/2416882239 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6800b910 con 0x7f9d7006fb20 2023-10-03T10:14:49.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.025+0000 7f9d67fff700 1 -- 172.21.15.3:0/2416882239 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6800bcc0 con 0x7f9d7006fb20 2023-10-03T10:14:49.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.026+0000 7fa2bd4b9700 1 Processor -- start 2023-10-03T10:14:49.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.026+0000 7fa2bd4b9700 1 -- start start 2023-10-03T10:14:49.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.026+0000 7f9d774ff700 1 -- 172.21.15.3:0/2416882239 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d7006fb20 msgr2=0x7f9d7006ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:49.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.026+0000 7f9d774ff700 1 --2- 172.21.15.3:0/2416882239 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d7006fb20 0x7f9d7006ff70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9d68009e50 tx=0x7f9d6800b330 comp rx=0 tx=0).stop 2023-10-03T10:14:49.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.026+0000 7f9d774ff700 1 -- 172.21.15.3:0/2416882239 shutdown_connections 2023-10-03T10:14:49.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.026+0000 7f9d774ff700 1 --2- 172.21.15.3:0/2416882239 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d7006fb20 0x7f9d7006ff70 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:49.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.026+0000 7f9d774ff700 1 --2- 172.21.15.3:0/2416882239 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d70130540 0x7f9d70130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:49.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.026+0000 7f9d774ff700 1 --2- 172.21.15.3:0/2416882239 >> v2:172.21.15.3:3300/0 conn(0x7f9d70130e50 0x7f9d7006f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:49.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.026+0000 7f9d774ff700 1 -- 172.21.15.3:0/2416882239 >> 172.21.15.3:0/2416882239 conn(0x7f9d700a2aa0 msgr2=0x7f9d700a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:49.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.026+0000 7fa2bd4b9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa2b81338c0 0x7fa2b8133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:49.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.027+0000 7fa2bd4b9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa2b81341d0 0x7fa2b809de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:49.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.027+0000 7fa2bd4b9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa2b809e390 0x7fa2b809e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:49.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.027+0000 7fa2bd4b9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2b8070c80 con 0x7fa2b81338c0 2023-10-03T10:14:49.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.027+0000 7fa2bd4b9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b809ed20 con 0x7fa2b809e390 2023-10-03T10:14:49.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.027+0000 7fa2bd4b9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b806f720 con 0x7fa2b81341d0 2023-10-03T10:14:49.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.027+0000 7fa2b6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa2b81338c0 0x7fa2b8133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:49.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.027+0000 7fa2b6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa2b81338c0 0x7fa2b8133c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56302/0 (socket says 172.21.15.3:56302) 2023-10-03T10:14:49.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.027+0000 7fa2b6ffd700 1 -- 172.21.15.3:0/2191883743 learned_addr learned my addr 172.21.15.3:0/2191883743 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:49.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.027+0000 7fa2b77fe700 1 --2- 172.21.15.3:0/2191883743 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa2b809e390 0x7fa2b809e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:49.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.027+0000 7fa2b67fc700 1 --2- 172.21.15.3:0/2191883743 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa2b81341d0 0x7fa2b809de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:49.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.027+0000 7f9d774ff700 1 -- 172.21.15.3:0/2416882239 shutdown_connections 2023-10-03T10:14:49.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.027+0000 7f9d774ff700 1 -- 172.21.15.3:0/2416882239 wait complete. 2023-10-03T10:14:49.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.027+0000 7fa2b6ffd700 1 -- 172.21.15.3:0/2191883743 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa2b81341d0 msgr2=0x7fa2b809de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:49.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.027+0000 7fa2b6ffd700 1 --2- 172.21.15.3:0/2191883743 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa2b81341d0 0x7fa2b809de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:49.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.027+0000 7fa2b6ffd700 1 -- 172.21.15.3:0/2191883743 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa2b809e390 msgr2=0x7fa2b809e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:49.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.027+0000 7fa2b6ffd700 1 --2- 172.21.15.3:0/2191883743 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa2b809e390 0x7fa2b809e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:49.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.027+0000 7fa2b6ffd700 1 -- 172.21.15.3:0/2191883743 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b806fce0 con 0x7fa2b81338c0 2023-10-03T10:14:49.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.027+0000 7fa2b6ffd700 1 --2- 172.21.15.3:0/2191883743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2b81338c0 0x7fa2b8133c90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80098f0 tx=0x7fa2a8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf17b84d802e53d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:49.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.028+0000 7fa2b5ffb700 1 -- 172.21.15.3:0/2191883743 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8011830 con 0x7fa2b81338c0 2023-10-03T10:14:49.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.028+0000 7fa2b5ffb700 1 -- 172.21.15.3:0/2191883743 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a8011990 con 0x7fa2b81338c0 2023-10-03T10:14:49.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.028+0000 7fa2b5ffb700 1 -- 172.21.15.3:0/2191883743 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8011d20 con 0x7fa2b81338c0 2023-10-03T10:14:49.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.028+0000 7fa2bd4b9700 1 -- 172.21.15.3:0/2191883743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2b81338c0 msgr2=0x7fa2b8133c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:49.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.028+0000 7fa2bd4b9700 1 --2- 172.21.15.3:0/2191883743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2b81338c0 0x7fa2b8133c90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80098f0 tx=0x7fa2a8009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:49.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.028+0000 7fa2bd4b9700 1 -- 172.21.15.3:0/2191883743 shutdown_connections 2023-10-03T10:14:49.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.028+0000 7fa2bd4b9700 1 --2- 172.21.15.3:0/2191883743 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa2b809e390 0x7fa2b809e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:49.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.028+0000 7fa2bd4b9700 1 --2- 172.21.15.3:0/2191883743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2b81338c0 0x7fa2b8133c90 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:49.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.028+0000 7fa2bd4b9700 1 --2- 172.21.15.3:0/2191883743 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa2b81341d0 0x7fa2b809de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:49.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.028+0000 7fa2bd4b9700 1 -- 172.21.15.3:0/2191883743 >> 172.21.15.3:0/2191883743 conn(0x7fa2b80a2aa0 msgr2=0x7fa2b80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:49.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.028+0000 7fa2bd4b9700 1 -- 172.21.15.3:0/2191883743 shutdown_connections 2023-10-03T10:14:49.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.028+0000 7fa2bd4b9700 1 -- 172.21.15.3:0/2191883743 wait complete. 2023-10-03T10:14:49.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.029+0000 7f9d774ff700 1 Processor -- start 2023-10-03T10:14:49.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.029+0000 7f9d774ff700 1 -- start start 2023-10-03T10:14:49.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.030+0000 7f9d774ff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d7006fb20 0x7f9d70118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:49.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.030+0000 7f9d774ff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d70130540 0x7f9d70119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:49.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.030+0000 7fa2bd4b9700 1 Processor -- start 2023-10-03T10:14:49.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.030+0000 7fa2bd4b9700 1 -- start start 2023-10-03T10:14:49.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.030+0000 7f9d774ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d70130e50 0x7f9d70119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:49.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.030+0000 7f9d774ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7011cae0 con 0x7f9d70130e50 2023-10-03T10:14:49.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.030+0000 7f9d774ff700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7011a840 con 0x7f9d7006fb20 2023-10-03T10:14:49.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.030+0000 7f9d774ff700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7011b900 con 0x7f9d70130540 2023-10-03T10:14:49.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.030+0000 7f9d75a9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d70130e50 0x7f9d70119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:49.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.030+0000 7f9d75a9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d70130e50 0x7f9d70119750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56304/0 (socket says 172.21.15.3:56304) 2023-10-03T10:14:49.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.030+0000 7f9d75a9c700 1 -- 172.21.15.3:0/1326739131 learned_addr learned my addr 172.21.15.3:0/1326739131 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:49.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.030+0000 7f9d7529b700 1 --2- 172.21.15.3:0/1326739131 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d7006fb20 0x7f9d70118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:49.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.030+0000 7fa2bd4b9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa2b809e390 0x7fa2b8118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:49.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.030+0000 7f9d75a9c700 1 -- 172.21.15.3:0/1326739131 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d70130540 msgr2=0x7f9d70119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:49.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.030+0000 7f9d75a9c700 1 --2- 172.21.15.3:0/1326739131 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d70130540 0x7f9d70119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:49.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.030+0000 7f9d75a9c700 1 -- 172.21.15.3:0/1326739131 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d7006fb20 msgr2=0x7f9d70118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:49.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.030+0000 7f9d75a9c700 1 --2- 172.21.15.3:0/1326739131 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d7006fb20 0x7f9d70118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:49.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.030+0000 7f9d75a9c700 1 -- 172.21.15.3:0/1326739131 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d68009b30 con 0x7f9d70130e50 2023-10-03T10:14:49.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.030+0000 7fa2bd4b9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2b81338c0 0x7fa2b8119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:49.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.030+0000 7fa2bd4b9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa2b81341d0 0x7fa2b8119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:49.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.030+0000 7fa2bd4b9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b811a860 con 0x7fa2b81338c0 2023-10-03T10:14:49.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.030+0000 7fa2bd4b9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b811c120 con 0x7fa2b81341d0 2023-10-03T10:14:49.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.030+0000 7fa2bd4b9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b811b4f0 con 0x7fa2b809e390 2023-10-03T10:14:49.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.031+0000 7fa2b6ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa2b809e390 0x7fa2b8118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:49.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.031+0000 7fa2b6ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa2b809e390 0x7fa2b8118cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57542/0 (socket says 172.21.15.3:57542) 2023-10-03T10:14:49.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.031+0000 7fa2b6ffd700 1 -- 172.21.15.3:0/2861220815 learned_addr learned my addr 172.21.15.3:0/2861220815 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:49.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.031+0000 7fa2b67fc700 1 --2- 172.21.15.3:0/2861220815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2b81338c0 0x7fa2b8119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:49.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.031+0000 7fa2b77fe700 1 --2- 172.21.15.3:0/2861220815 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa2b81341d0 0x7fa2b8119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:49.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.031+0000 7fa2b6ffd700 1 -- 172.21.15.3:0/2861220815 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa2b81341d0 msgr2=0x7fa2b8119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:49.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.031+0000 7fa2b6ffd700 1 --2- 172.21.15.3:0/2861220815 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa2b81341d0 0x7fa2b8119750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:49.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.031+0000 7fa2b6ffd700 1 -- 172.21.15.3:0/2861220815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2b81338c0 msgr2=0x7fa2b8119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:49.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.031+0000 7fa2b6ffd700 1 --2- 172.21.15.3:0/2861220815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2b81338c0 0x7fa2b8119210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:49.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.031+0000 7fa2b6ffd700 1 -- 172.21.15.3:0/2861220815 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a8009550 con 0x7fa2b809e390 2023-10-03T10:14:49.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.031+0000 7fa2b77fe700 1 --2- 172.21.15.3:0/2861220815 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa2b81341d0 0x7fa2b8119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:14:49.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.031+0000 7fa2b6ffd700 1 --2- 172.21.15.3:0/2861220815 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa2b809e390 0x7fa2b8118cd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8009f50 tx=0x7fa2a8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:49.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.031+0000 7f9d75a9c700 1 --2- 172.21.15.3:0/1326739131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d70130e50 0x7f9d70119750 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800cff0 tx=0x7f9d68004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:49.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.031+0000 7fa29f7fe700 1 -- 172.21.15.3:0/2861220815 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8004300 con 0x7fa2b809e390 2023-10-03T10:14:49.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.031+0000 7fa29f7fe700 1 -- 172.21.15.3:0/2861220815 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a801f070 con 0x7fa2b809e390 2023-10-03T10:14:49.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.031+0000 7fa29f7fe700 1 -- 172.21.15.3:0/2861220815 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a80036a0 con 0x7fa2b809e390 2023-10-03T10:14:49.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.032+0000 7f9d65ffb700 1 -- 172.21.15.3:0/1326739131 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d68003e30 con 0x7f9d70130e50 2023-10-03T10:14:49.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.032+0000 7f9d65ffb700 1 -- 172.21.15.3:0/1326739131 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d68004040 con 0x7f9d70130e50 2023-10-03T10:14:49.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.032+0000 7f9d65ffb700 1 -- 172.21.15.3:0/1326739131 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d680043d0 con 0x7f9d70130e50 2023-10-03T10:14:49.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.032+0000 7fa2bd4b9700 1 -- 172.21.15.3:0/2861220815 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b811ca90 con 0x7fa2b809e390 2023-10-03T10:14:49.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.032+0000 7fa2bd4b9700 1 -- 172.21.15.3:0/2861220815 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b8130d00 con 0x7fa2b809e390 2023-10-03T10:14:49.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.033+0000 7f9d774ff700 1 -- 172.21.15.3:0/1326739131 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d700718e0 con 0x7f9d70130e50 2023-10-03T10:14:49.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.033+0000 7fa29f7fe700 1 -- 172.21.15.3:0/2861220815 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa2a8021030 con 0x7fa2b809e390 2023-10-03T10:14:49.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.033+0000 7fa2bd4b9700 1 -- 172.21.15.3:0/2861220815 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b811bca0 con 0x7fa2b809e390 2023-10-03T10:14:49.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.033+0000 7fa29f7fe700 1 -- 172.21.15.3:0/2861220815 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa2a80408f0 con 0x7fa2b809e390 2023-10-03T10:14:49.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.033+0000 7f9d774ff700 1 -- 172.21.15.3:0/1326739131 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d7011a260 con 0x7f9d70130e50 2023-10-03T10:14:49.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.034+0000 7f9d65ffb700 1 -- 172.21.15.3:0/1326739131 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9d68004530 con 0x7f9d70130e50 2023-10-03T10:14:49.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.034+0000 7f9d65ffb700 1 -- 172.21.15.3:0/1326739131 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9d68048f50 con 0x7f9d70130e50 2023-10-03T10:14:49.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.035+0000 7f9d774ff700 1 -- 172.21.15.3:0/1326739131 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d7011bb30 con 0x7f9d70130e50 2023-10-03T10:14:49.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.039+0000 7fa29f7fe700 1 -- 172.21.15.3:0/2861220815 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa2b811bca0 con 0x7fa2b809e390 2023-10-03T10:14:49.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.040+0000 7f9d65ffb700 1 -- 172.21.15.3:0/1326739131 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9d7011bb30 con 0x7f9d70130e50 2023-10-03T10:14:49.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.055+0000 7f74d0b47700 1 -- 172.21.15.3:0/2401468943 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f74cc1238e0 con 0x7f74cc0a1340 2023-10-03T10:14:49.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.055+0000 7f74c2ffd700 1 -- 172.21.15.3:0/2401468943 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f74cc1238e0 con 0x7f74cc0a1340 2023-10-03T10:14:49.055 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:49.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.059+0000 7f74d0b47700 1 -- 172.21.15.3:0/2401468943 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74cc0a1340 msgr2=0x7f74cc1205e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:49.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.059+0000 7f74d0b47700 1 --2- 172.21.15.3:0/2401468943 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74cc0a1340 0x7f74cc1205e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f74b400e9e0 tx=0x7f74b400ecf0 comp rx=0 tx=0).stop 2023-10-03T10:14:49.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.059+0000 7f74d0b47700 1 -- 172.21.15.3:0/2401468943 shutdown_connections 2023-10-03T10:14:49.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.059+0000 7f74d0b47700 1 --2- 172.21.15.3:0/2401468943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74cc133fa0 0x7f74cc120b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:49.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.059+0000 7f74d0b47700 1 --2- 172.21.15.3:0/2401468943 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74cc0a0a30 0x7f74cc1200a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:49.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.059+0000 7f74d0b47700 1 --2- 172.21.15.3:0/2401468943 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74cc0a1340 0x7f74cc1205e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:49.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.059+0000 7f74d0b47700 1 -- 172.21.15.3:0/2401468943 >> 172.21.15.3:0/2401468943 conn(0x7f74cc0a6550 msgr2=0x7f74cc1baff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:49.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.059+0000 7f74d0b47700 1 -- 172.21.15.3:0/2401468943 shutdown_connections 2023-10-03T10:14:49.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.059+0000 7f74d0b47700 1 -- 172.21.15.3:0/2401468943 wait complete. 2023-10-03T10:14:49.074 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:49.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.214+0000 7f7f387aa700 1 -- 172.21.15.3:0/2708765544 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7f30072210 con 0x7f7f3006f680 2023-10-03T10:14:49.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.214+0000 7f7f26ffd700 1 -- 172.21.15.3:0/2708765544 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7f2c021030 con 0x7f7f3006f680 2023-10-03T10:14:49.214 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:49.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.216+0000 7f7f387aa700 1 -- 172.21.15.3:0/2708765544 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f3006f680 msgr2=0x7f7f301c8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:49.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.216+0000 7f7f387aa700 1 --2- 172.21.15.3:0/2708765544 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f3006f680 0x7f7f301c8360 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c00e940 tx=0x7f7f2c00ed00 comp rx=0 tx=0).stop 2023-10-03T10:14:49.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.216+0000 7f7f387aa700 1 -- 172.21.15.3:0/2708765544 shutdown_connections 2023-10-03T10:14:49.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.216+0000 7f7f387aa700 1 --2- 172.21.15.3:0/2708765544 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f30130670 0x7f7f301c4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:49.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.216+0000 7f7f387aa700 1 --2- 172.21.15.3:0/2708765544 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f3006f680 0x7f7f301c8360 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:49.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.216+0000 7f7f387aa700 1 --2- 172.21.15.3:0/2708765544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f300701a0 0x7f7f301c88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:49.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.216+0000 7f7f387aa700 1 -- 172.21.15.3:0/2708765544 >> 172.21.15.3:0/2708765544 conn(0x7f7f300a2aa0 msgr2=0x7f7f301250c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:49.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.217+0000 7f7f387aa700 1 -- 172.21.15.3:0/2708765544 shutdown_connections 2023-10-03T10:14:49.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.217+0000 7f7f387aa700 1 -- 172.21.15.3:0/2708765544 wait complete. 2023-10-03T10:14:49.230 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:49.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.250+0000 7fa2bd4b9700 1 -- 172.21.15.3:0/2861220815 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa2b811bca0 con 0x7fa2b809e390 2023-10-03T10:14:49.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.250+0000 7fa29f7fe700 1 -- 172.21.15.3:0/2861220815 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa2a8015070 con 0x7fa2b809e390 2023-10-03T10:14:49.250 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:49.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.253+0000 7fa2bd4b9700 1 -- 172.21.15.3:0/2861220815 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa2b809e390 msgr2=0x7fa2b8118cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:49.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.253+0000 7fa2bd4b9700 1 --2- 172.21.15.3:0/2861220815 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa2b809e390 0x7fa2b8118cd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8009f50 tx=0x7fa2a8004750 comp rx=0 tx=0).stop 2023-10-03T10:14:49.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.253+0000 7fa2bd4b9700 1 -- 172.21.15.3:0/2861220815 shutdown_connections 2023-10-03T10:14:49.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.253+0000 7fa2bd4b9700 1 --2- 172.21.15.3:0/2861220815 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa2b81341d0 0x7fa2b8119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:49.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.253+0000 7fa2bd4b9700 1 --2- 172.21.15.3:0/2861220815 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa2b809e390 0x7fa2b8118cd0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:49.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.253+0000 7fa2bd4b9700 1 --2- 172.21.15.3:0/2861220815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2b81338c0 0x7fa2b8119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:49.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.253+0000 7fa2bd4b9700 1 -- 172.21.15.3:0/2861220815 >> 172.21.15.3:0/2861220815 conn(0x7fa2b80a2aa0 msgr2=0x7fa2b812e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:49.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.253+0000 7fa2bd4b9700 1 -- 172.21.15.3:0/2861220815 shutdown_connections 2023-10-03T10:14:49.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.253+0000 7fa2bd4b9700 1 -- 172.21.15.3:0/2861220815 wait complete. 2023-10-03T10:14:49.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.264+0000 7f9d774ff700 1 -- 172.21.15.3:0/1326739131 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9d7011bb30 con 0x7f9d70130e50 2023-10-03T10:14:49.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.265+0000 7f9d65ffb700 1 -- 172.21.15.3:0/1326739131 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9d7011bb30 con 0x7f9d70130e50 2023-10-03T10:14:49.265 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:49.267 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:49.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.267+0000 7f9d774ff700 1 -- 172.21.15.3:0/1326739131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d70130e50 msgr2=0x7f9d70119750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:49.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.267+0000 7f9d774ff700 1 --2- 172.21.15.3:0/1326739131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d70130e50 0x7f9d70119750 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800cff0 tx=0x7f9d68004ad0 comp rx=0 tx=0).stop 2023-10-03T10:14:49.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.268+0000 7f9d774ff700 1 -- 172.21.15.3:0/1326739131 shutdown_connections 2023-10-03T10:14:49.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.268+0000 7f9d774ff700 1 --2- 172.21.15.3:0/1326739131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d70130e50 0x7f9d70119750 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:49.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.268+0000 7f9d774ff700 1 --2- 172.21.15.3:0/1326739131 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d7006fb20 0x7f9d70118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:49.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.268+0000 7f9d774ff700 1 --2- 172.21.15.3:0/1326739131 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d70130540 0x7f9d70119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:49.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.268+0000 7f9d774ff700 1 -- 172.21.15.3:0/1326739131 >> 172.21.15.3:0/1326739131 conn(0x7f9d700a2aa0 msgr2=0x7f9d7012d570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:49.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.268+0000 7f9d774ff700 1 -- 172.21.15.3:0/1326739131 shutdown_connections 2023-10-03T10:14:49.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:49.268+0000 7f9d774ff700 1 -- 172.21.15.3:0/1326739131 wait complete. 2023-10-03T10:14:49.281 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:50.075 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:50.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.222+0000 7f7f978de700 1 Processor -- start 2023-10-03T10:14:50.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.223+0000 7f7f978de700 1 -- start start 2023-10-03T10:14:50.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.223+0000 7f7f978de700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7f9009d6f0 0x7f7f9009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:50.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.223+0000 7f7f978de700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f9009e000 0x7f7f901339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:50.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.223+0000 7f7f978de700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f9009e410 0x7f7f90133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:50.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.223+0000 7f7f978de700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f9006b460 con 0x7f7f9009d6f0 2023-10-03T10:14:50.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.223+0000 7f7f978de700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9009b120 con 0x7f7f9009e410 2023-10-03T10:14:50.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.223+0000 7f7f978de700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9009a7e0 con 0x7f7f9009e000 2023-10-03T10:14:50.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.223+0000 7f7f9567a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7f9009d6f0 0x7f7f9009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:50.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.223+0000 7f7f95e7b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f9009e410 0x7f7f90133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:50.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.223+0000 7f7f95e7b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f9009e410 0x7f7f90133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59252/0 (socket says 172.21.15.3:59252) 2023-10-03T10:14:50.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.223+0000 7f7f95e7b700 1 -- 172.21.15.3:0/2072959498 learned_addr learned my addr 172.21.15.3:0/2072959498 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:50.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.223+0000 7f7f9567a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7f9009d6f0 0x7f7f9009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56316/0 (socket says 172.21.15.3:56316) 2023-10-03T10:14:50.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.223+0000 7f7f94e79700 1 --2- 172.21.15.3:0/2072959498 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f9009e000 0x7f7f901339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:50.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.224+0000 7f7f9567a700 1 -- 172.21.15.3:0/2072959498 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f9009e000 msgr2=0x7f7f901339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:50.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.224+0000 7f7f9567a700 1 --2- 172.21.15.3:0/2072959498 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f9009e000 0x7f7f901339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.224+0000 7f7f9567a700 1 -- 172.21.15.3:0/2072959498 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f9009e410 msgr2=0x7f7f90133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:50.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.224+0000 7f7f9567a700 1 --2- 172.21.15.3:0/2072959498 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f9009e410 0x7f7f90133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.224+0000 7f7f9567a700 1 -- 172.21.15.3:0/2072959498 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9009a500 con 0x7f7f9009d6f0 2023-10-03T10:14:50.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.224+0000 7f7f95e7b700 1 --2- 172.21.15.3:0/2072959498 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f9009e410 0x7f7f90133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:50.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.224+0000 7f7f94e79700 1 --2- 172.21.15.3:0/2072959498 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f9009e000 0x7f7f901339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:50.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.224+0000 7f7f9567a700 1 --2- 172.21.15.3:0/2072959498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f9009d6f0 0x7f7f9009dac0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c009840 tx=0x7f7f8c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c6b6c008f7738c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:50.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.224+0000 7f7f87fff700 1 -- 172.21.15.3:0/2072959498 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c011830 con 0x7f7f9009d6f0 2023-10-03T10:14:50.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.224+0000 7f7f87fff700 1 -- 172.21.15.3:0/2072959498 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8c011990 con 0x7f7f9009d6f0 2023-10-03T10:14:50.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.224+0000 7f7f87fff700 1 -- 172.21.15.3:0/2072959498 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c011d20 con 0x7f7f9009d6f0 2023-10-03T10:14:50.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.225+0000 7f7f978de700 1 -- 172.21.15.3:0/2072959498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f9009d6f0 msgr2=0x7f7f9009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:50.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.225+0000 7f7f978de700 1 --2- 172.21.15.3:0/2072959498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f9009d6f0 0x7f7f9009dac0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c009840 tx=0x7f7f8c009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:50.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.225+0000 7f7f978de700 1 -- 172.21.15.3:0/2072959498 shutdown_connections 2023-10-03T10:14:50.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.225+0000 7f7f978de700 1 --2- 172.21.15.3:0/2072959498 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f9009e410 0x7f7f90133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.225+0000 7f7f978de700 1 --2- 172.21.15.3:0/2072959498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f9009d6f0 0x7f7f9009dac0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.225+0000 7f7f978de700 1 --2- 172.21.15.3:0/2072959498 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f9009e000 0x7f7f901339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.225+0000 7f7f978de700 1 -- 172.21.15.3:0/2072959498 >> 172.21.15.3:0/2072959498 conn(0x7f7f90066f50 msgr2=0x7f7f9009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:50.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.225+0000 7f7f978de700 1 -- 172.21.15.3:0/2072959498 shutdown_connections 2023-10-03T10:14:50.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.225+0000 7f7f978de700 1 -- 172.21.15.3:0/2072959498 wait complete. 2023-10-03T10:14:50.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.226+0000 7f7f978de700 1 Processor -- start 2023-10-03T10:14:50.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.226+0000 7f7f978de700 1 -- start start 2023-10-03T10:14:50.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.226+0000 7f7f978de700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f9009e000 0x7f7f901a5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:50.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.226+0000 7f7f978de700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f9009e410 0x7f7f901a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:50.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.226+0000 7f7f978de700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f901a9f70 0x7f7f901a69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:50.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.226+0000 7f7f978de700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f901a9a70 con 0x7f7f9009e410 2023-10-03T10:14:50.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.226+0000 7f7f978de700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f901a8600 con 0x7f7f9009e000 2023-10-03T10:14:50.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.226+0000 7f7f978de700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f901a8e40 con 0x7f7f901a9f70 2023-10-03T10:14:50.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.226+0000 7f7f9567a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f9009e000 0x7f7f901a5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:50.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.227+0000 7f7f94e79700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f9009e410 0x7f7f901a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:50.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.227+0000 7f7f9567a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f9009e000 0x7f7f901a5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59266/0 (socket says 172.21.15.3:59266) 2023-10-03T10:14:50.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.227+0000 7f7f9567a700 1 -- 172.21.15.3:0/250672276 learned_addr learned my addr 172.21.15.3:0/250672276 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:50.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.227+0000 7f7f94e79700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f9009e410 0x7f7f901a6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56320/0 (socket says 172.21.15.3:56320) 2023-10-03T10:14:50.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.227+0000 7f7f95e7b700 1 --2- 172.21.15.3:0/250672276 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f901a9f70 0x7f7f901a69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:50.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.227+0000 7f7f9567a700 1 -- 172.21.15.3:0/250672276 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f901a9f70 msgr2=0x7f7f901a69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:50.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.227+0000 7f7f9567a700 1 --2- 172.21.15.3:0/250672276 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f901a9f70 0x7f7f901a69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.227+0000 7f7f9567a700 1 -- 172.21.15.3:0/250672276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f9009e410 msgr2=0x7f7f901a6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:50.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.227+0000 7f7f9567a700 1 --2- 172.21.15.3:0/250672276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f9009e410 0x7f7f901a6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.227+0000 7f7f9567a700 1 -- 172.21.15.3:0/250672276 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f800094a0 con 0x7f7f9009e000 2023-10-03T10:14:50.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.227+0000 7f7f94e79700 1 --2- 172.21.15.3:0/250672276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f9009e410 0x7f7f901a6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:50.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.227+0000 7f7f9567a700 1 --2- 172.21.15.3:0/250672276 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f9009e000 0x7f7f901a5f50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c009fd0 tx=0x7f7f8c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:50.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.227+0000 7f7f85ffb700 1 -- 172.21.15.3:0/250672276 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c004450 con 0x7f7f9009e000 2023-10-03T10:14:50.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.227+0000 7f7f85ffb700 1 -- 172.21.15.3:0/250672276 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8c01f070 con 0x7f7f9009e000 2023-10-03T10:14:50.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.227+0000 7f7f85ffb700 1 -- 172.21.15.3:0/250672276 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c0038e0 con 0x7f7f9009e000 2023-10-03T10:14:50.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.227+0000 7f7f978de700 1 -- 172.21.15.3:0/250672276 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f88009b30 con 0x7f7f9009e000 2023-10-03T10:14:50.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.228+0000 7f7f978de700 1 -- 172.21.15.3:0/250672276 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f8c0094a0 con 0x7f7f9009e000 2023-10-03T10:14:50.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.228+0000 7f7f85ffb700 1 -- 172.21.15.3:0/250672276 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7f8c003a40 con 0x7f7f9009e000 2023-10-03T10:14:50.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.228+0000 7f7f85ffb700 1 -- 172.21.15.3:0/250672276 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7f8c021070 con 0x7f7f9009e000 2023-10-03T10:14:50.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.229+0000 7f7f978de700 1 -- 172.21.15.3:0/250672276 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f901a95f0 con 0x7f7f9009e000 2023-10-03T10:14:50.233 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:50.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.234+0000 7f7f85ffb700 1 -- 172.21.15.3:0/250672276 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7f901a95f0 con 0x7f7f9009e000 2023-10-03T10:14:50.268 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:50.283 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:50.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.377+0000 7fd8a40a5700 1 Processor -- start 2023-10-03T10:14:50.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.377+0000 7fd8a40a5700 1 -- start start 2023-10-03T10:14:50.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.377+0000 7fd8a40a5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd89c06b250 0x7fd89c136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:50.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.377+0000 7fd8a40a5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd89c136d80 0x7fd89c139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:50.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.378+0000 7fd8a40a5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd89c09ece0 con 0x7fd89c06a770 2023-10-03T10:14:50.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.378+0000 7fd8a40a5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c06b640 con 0x7fd89c136d80 2023-10-03T10:14:50.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.378+0000 7fd8a40a5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c09e760 con 0x7fd89c06b250 2023-10-03T10:14:50.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.378+0000 7fd8a1640700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd89c06b250 0x7fd89c136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:50.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.378+0000 7fd8a1640700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd89c06b250 0x7fd89c136840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57558/0 (socket says 172.21.15.3:57558) 2023-10-03T10:14:50.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.378+0000 7fd8a1640700 1 -- 172.21.15.3:0/2506065267 learned_addr learned my addr 172.21.15.3:0/2506065267 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:50.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.378+0000 7fd8a2642700 1 --2- 172.21.15.3:0/2506065267 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd89c136d80 0x7fd89c139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:50.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.378+0000 7fd8a2642700 1 -- 172.21.15.3:0/2506065267 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd89c06b250 msgr2=0x7fd89c136840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:50.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.378+0000 7fd8a2642700 1 --2- 172.21.15.3:0/2506065267 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd89c06b250 0x7fd89c136840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.378+0000 7fd8a2642700 1 -- 172.21.15.3:0/2506065267 >> v1:172.21.15.3:6789/0 conn(0x7fd89c06a770 legacy=0x7fd89c06ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:50.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.378+0000 7fd8a2642700 1 -- 172.21.15.3:0/2506065267 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89c09e040 con 0x7fd89c136d80 2023-10-03T10:14:50.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.378+0000 7fd8a2642700 1 --2- 172.21.15.3:0/2506065267 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd89c136d80 0x7fd89c139160 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd894009dc0 tx=0x7fd89400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e81e5e32a30f3be server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:50.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.379+0000 7fd8a0e3f700 1 -- 172.21.15.3:0/2506065267 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89400e070 con 0x7fd89c136d80 2023-10-03T10:14:50.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.379+0000 7fd8a0e3f700 1 -- 172.21.15.3:0/2506065267 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89400bae0 con 0x7fd89c136d80 2023-10-03T10:14:50.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.379+0000 7fd8a0e3f700 1 -- 172.21.15.3:0/2506065267 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89400be90 con 0x7fd89c136d80 2023-10-03T10:14:50.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.379+0000 7fd8a40a5700 1 -- 172.21.15.3:0/2506065267 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd89c136d80 msgr2=0x7fd89c139160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:50.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.379+0000 7fd8a40a5700 1 --2- 172.21.15.3:0/2506065267 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd89c136d80 0x7fd89c139160 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd894009dc0 tx=0x7fd89400b330 comp rx=0 tx=0).stop 2023-10-03T10:14:50.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.380+0000 7fd8a40a5700 1 -- 172.21.15.3:0/2506065267 shutdown_connections 2023-10-03T10:14:50.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.380+0000 7fd8a40a5700 1 --2- 172.21.15.3:0/2506065267 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd89c136d80 0x7fd89c139160 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.380+0000 7fd8a40a5700 1 --2- 172.21.15.3:0/2506065267 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd89c06b250 0x7fd89c136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.380+0000 7fd8a40a5700 1 -- 172.21.15.3:0/2506065267 >> 172.21.15.3:0/2506065267 conn(0x7fd89c066f50 msgr2=0x7fd89c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:50.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.380+0000 7fd8a40a5700 1 -- 172.21.15.3:0/2506065267 shutdown_connections 2023-10-03T10:14:50.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.380+0000 7fd8a40a5700 1 -- 172.21.15.3:0/2506065267 wait complete. 2023-10-03T10:14:50.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.381+0000 7fd8a40a5700 1 Processor -- start 2023-10-03T10:14:50.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.381+0000 7fd8a40a5700 1 -- start start 2023-10-03T10:14:50.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.382+0000 7fd8a40a5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd89c06a770 0x7fd89c1327a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:50.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.382+0000 7fd8a40a5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd89c06b250 0x7fd89c132ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:50.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.382+0000 7fd8a40a5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd89c136d80 0x7fd89c133220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:50.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.382+0000 7fd8a40a5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c133c70 con 0x7fd89c06b250 2023-10-03T10:14:50.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.382+0000 7fd8a40a5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c136360 con 0x7fd89c06a770 2023-10-03T10:14:50.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.382+0000 7fd8a40a5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c071e60 con 0x7fd89c136d80 2023-10-03T10:14:50.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.382+0000 7fd8a1640700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd89c06b250 0x7fd89c132ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:50.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.382+0000 7fd8a1640700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd89c06b250 0x7fd89c132ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56332/0 (socket says 172.21.15.3:56332) 2023-10-03T10:14:50.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.382+0000 7fd8a1640700 1 -- 172.21.15.3:0/4523302 learned_addr learned my addr 172.21.15.3:0/4523302 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:50.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.382+0000 7fd8a1e41700 1 --2- 172.21.15.3:0/4523302 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd89c06a770 0x7fd89c1327a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:50.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.382+0000 7fd8a2642700 1 --2- 172.21.15.3:0/4523302 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd89c136d80 0x7fd89c133220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:50.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.383+0000 7fd8a1e41700 1 -- 172.21.15.3:0/4523302 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd89c136d80 msgr2=0x7fd89c133220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:50.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.383+0000 7fd8a1e41700 1 --2- 172.21.15.3:0/4523302 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd89c136d80 0x7fd89c133220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.383+0000 7fd8a1e41700 1 -- 172.21.15.3:0/4523302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd89c06b250 msgr2=0x7fd89c132ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:50.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.383+0000 7fd8a1e41700 1 --2- 172.21.15.3:0/4523302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd89c06b250 0x7fd89c132ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.383+0000 7fd8a1e41700 1 -- 172.21.15.3:0/4523302 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd894009a70 con 0x7fd89c06a770 2023-10-03T10:14:50.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.383+0000 7fd8a1e41700 1 --2- 172.21.15.3:0/4523302 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd89c06a770 0x7fd89c1327a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd89800c400 tx=0x7fd89800c710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:50.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.383+0000 7fd8927fc700 1 -- 172.21.15.3:0/4523302 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd898011790 con 0x7fd89c06a770 2023-10-03T10:14:50.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.383+0000 7fd8927fc700 1 -- 172.21.15.3:0/4523302 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8980118f0 con 0x7fd89c06a770 2023-10-03T10:14:50.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.383+0000 7fd8a40a5700 1 -- 172.21.15.3:0/4523302 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd89c09be90 con 0x7fd89c06a770 2023-10-03T10:14:50.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.383+0000 7fd8a40a5700 1 -- 172.21.15.3:0/4523302 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd89c135fe0 con 0x7fd89c06a770 2023-10-03T10:14:50.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.383+0000 7fd8927fc700 1 -- 172.21.15.3:0/4523302 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd898011ca0 con 0x7fd89c06a770 2023-10-03T10:14:50.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.384+0000 7fd8927fc700 1 -- 172.21.15.3:0/4523302 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd89801c460 con 0x7fd89c06a770 2023-10-03T10:14:50.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.385+0000 7fd8927fc700 1 -- 172.21.15.3:0/4523302 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd89803f410 con 0x7fd89c06a770 2023-10-03T10:14:50.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.385+0000 7fd8a40a5700 1 -- 172.21.15.3:0/4523302 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd89c134c20 con 0x7fd89c06a770 2023-10-03T10:14:50.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.394+0000 7fd8927fc700 1 -- 172.21.15.3:0/4523302 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd898011e00 con 0x7fd89c06a770 2023-10-03T10:14:50.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.425+0000 7f041134d700 1 Processor -- start 2023-10-03T10:14:50.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.425+0000 7f041134d700 1 -- start start 2023-10-03T10:14:50.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.425+0000 7f041134d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f040c0a0a30 0x7f040c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:50.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.425+0000 7f041134d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f040c09d630 0x7f040c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:50.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.425+0000 7f041134d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f040c0a2a70 con 0x7f040c133930 2023-10-03T10:14:50.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.425+0000 7f041134d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f040c0a01d0 con 0x7f040c09d630 2023-10-03T10:14:50.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.425+0000 7f041134d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f040c09e2b0 con 0x7f040c0a0a30 2023-10-03T10:14:50.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.426+0000 7f040affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f040c0a0a30 0x7f040c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:50.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.426+0000 7f040affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f040c0a0a30 0x7f040c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59288/0 (socket says 172.21.15.3:59288) 2023-10-03T10:14:50.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.426+0000 7f040affd700 1 -- 172.21.15.3:0/1757881159 learned_addr learned my addr 172.21.15.3:0/1757881159 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:50.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.426+0000 7f040a7fc700 1 --2- 172.21.15.3:0/1757881159 >> v2:172.21.15.3:3300/0 conn(0x7f040c09d630 0x7f040c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:50.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.427+0000 7f040affd700 1 -- 172.21.15.3:0/1757881159 >> v2:172.21.15.3:3300/0 conn(0x7f040c09d630 msgr2=0x7f040c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:50.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.427+0000 7f040affd700 1 --2- 172.21.15.3:0/1757881159 >> v2:172.21.15.3:3300/0 conn(0x7f040c09d630 0x7f040c0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.427+0000 7f040affd700 1 -- 172.21.15.3:0/1757881159 >> v1:172.21.15.3:6789/0 conn(0x7f040c133930 legacy=0x7f040c133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:50.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.427+0000 7f040affd700 1 -- 172.21.15.3:0/1757881159 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040c09dc50 con 0x7f040c0a0a30 2023-10-03T10:14:50.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.427+0000 7f040affd700 1 --2- 172.21.15.3:0/1757881159 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f040c0a0a30 0x7f040c0a0e00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f03fc0098f0 tx=0x7f03fc009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b919db203c9dc3f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:50.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.427+0000 7f0409ffb700 1 -- 172.21.15.3:0/1757881159 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc011830 con 0x7f040c0a0a30 2023-10-03T10:14:50.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.427+0000 7f0409ffb700 1 -- 172.21.15.3:0/1757881159 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03fc011990 con 0x7f040c0a0a30 2023-10-03T10:14:50.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.427+0000 7f0409ffb700 1 -- 172.21.15.3:0/1757881159 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc011d20 con 0x7f040c0a0a30 2023-10-03T10:14:50.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.428+0000 7f041134d700 1 -- 172.21.15.3:0/1757881159 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f040c0a0a30 msgr2=0x7f040c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:50.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.428+0000 7f041134d700 1 --2- 172.21.15.3:0/1757881159 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f040c0a0a30 0x7f040c0a0e00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f03fc0098f0 tx=0x7f03fc009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:50.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.428+0000 7f041134d700 1 -- 172.21.15.3:0/1757881159 shutdown_connections 2023-10-03T10:14:50.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.428+0000 7f041134d700 1 --2- 172.21.15.3:0/1757881159 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f040c0a0a30 0x7f040c0a0e00 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.428+0000 7f041134d700 1 --2- 172.21.15.3:0/1757881159 >> v2:172.21.15.3:3300/0 conn(0x7f040c09d630 0x7f040c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.428+0000 7f041134d700 1 -- 172.21.15.3:0/1757881159 >> 172.21.15.3:0/1757881159 conn(0x7f040c0a3050 msgr2=0x7f040c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:50.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.428+0000 7f041134d700 1 -- 172.21.15.3:0/1757881159 shutdown_connections 2023-10-03T10:14:50.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.428+0000 7f041134d700 1 -- 172.21.15.3:0/1757881159 wait complete. 2023-10-03T10:14:50.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.430+0000 7f041134d700 1 Processor -- start 2023-10-03T10:14:50.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.430+0000 7f041134d700 1 -- start start 2023-10-03T10:14:50.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.430+0000 7f041134d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f040c09d630 0x7f040c121290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:50.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.430+0000 7f041134d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f040c0a0a30 0x7f040c1217d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:50.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.431+0000 7f041134d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f040c133930 0x7f040c121d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:50.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.431+0000 7f041134d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040c122dc0 con 0x7f040c09d630 2023-10-03T10:14:50.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.431+0000 7f041134d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f040c123c50 con 0x7f040c133930 2023-10-03T10:14:50.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.431+0000 7f041134d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f040c124730 con 0x7f040c0a0a30 2023-10-03T10:14:50.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.431+0000 7f040a7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f040c0a0a30 0x7f040c1217d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:50.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.431+0000 7f040a7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f040c0a0a30 0x7f040c1217d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57576/0 (socket says 172.21.15.3:57576) 2023-10-03T10:14:50.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.431+0000 7f040a7fc700 1 -- 172.21.15.3:0/2561290009 learned_addr learned my addr 172.21.15.3:0/2561290009 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:50.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.431+0000 7f040b7fe700 1 --2- 172.21.15.3:0/2561290009 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f040c133930 0x7f040c121d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:50.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.431+0000 7f040affd700 1 --2- 172.21.15.3:0/2561290009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f040c09d630 0x7f040c121290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:50.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.431+0000 7f040a7fc700 1 -- 172.21.15.3:0/2561290009 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f040c133930 msgr2=0x7f040c121d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:50.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.431+0000 7f040a7fc700 1 --2- 172.21.15.3:0/2561290009 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f040c133930 0x7f040c121d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.431+0000 7f040a7fc700 1 -- 172.21.15.3:0/2561290009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f040c09d630 msgr2=0x7f040c121290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:50.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.431+0000 7f040a7fc700 1 --2- 172.21.15.3:0/2561290009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f040c09d630 0x7f040c121290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.431+0000 7f040a7fc700 1 -- 172.21.15.3:0/2561290009 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03fc009550 con 0x7f040c0a0a30 2023-10-03T10:14:50.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.431+0000 7f040affd700 1 --2- 172.21.15.3:0/2561290009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f040c09d630 0x7f040c121290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:50.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.431+0000 7f040a7fc700 1 --2- 172.21.15.3:0/2561290009 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f040c0a0a30 0x7f040c1217d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f040000e980 tx=0x7f040000ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:50.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.432+0000 7f03f37fe700 1 -- 172.21.15.3:0/2561290009 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0400009710 con 0x7f040c0a0a30 2023-10-03T10:14:50.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.432+0000 7f03f37fe700 1 -- 172.21.15.3:0/2561290009 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040000c690 con 0x7f040c0a0a30 2023-10-03T10:14:50.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.432+0000 7f03f37fe700 1 -- 172.21.15.3:0/2561290009 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040000ca20 con 0x7f040c0a0a30 2023-10-03T10:14:50.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.432+0000 7f041134d700 1 -- 172.21.15.3:0/2561290009 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f040c1249f0 con 0x7f040c0a0a30 2023-10-03T10:14:50.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.432+0000 7f041134d700 1 -- 172.21.15.3:0/2561290009 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f040c0a5130 con 0x7f040c0a0a30 2023-10-03T10:14:50.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.433+0000 7f041134d700 1 -- 172.21.15.3:0/2561290009 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f040c124010 con 0x7f040c0a0a30 2023-10-03T10:14:50.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.433+0000 7f03f37fe700 1 -- 172.21.15.3:0/2561290009 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f040000cb80 con 0x7f040c0a0a30 2023-10-03T10:14:50.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.433+0000 7f03f37fe700 1 -- 172.21.15.3:0/2561290009 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f040003fb20 con 0x7f040c0a0a30 2023-10-03T10:14:50.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.438+0000 7f03f37fe700 1 -- 172.21.15.3:0/2561290009 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f040001b3b0 con 0x7f040c0a0a30 2023-10-03T10:14:50.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.470+0000 7f71b9e8b700 1 Processor -- start 2023-10-03T10:14:50.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.470+0000 7f71b9e8b700 1 -- start start 2023-10-03T10:14:50.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.471+0000 7f71b9e8b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71b4133420 0x7f71b41337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:50.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.471+0000 7f71b9e8b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f71b4133e30 0x7f71b412ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:50.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.471+0000 7f71b9e8b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f71b4134260 0x7f71b41304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:50.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.471+0000 7f71b9e8b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71b406b990 con 0x7f71b4133e30 2023-10-03T10:14:50.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.471+0000 7f71b9e8b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b40a0590 con 0x7f71b4134260 2023-10-03T10:14:50.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.471+0000 7f71b9e8b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71b409fb80 con 0x7f71b4133420 2023-10-03T10:14:50.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.471+0000 7f71b2ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f71b4133e30 0x7f71b412ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:50.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.471+0000 7f71b2ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f71b4133e30 0x7f71b412ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56368/0 (socket says 172.21.15.3:56368) 2023-10-03T10:14:50.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.471+0000 7f71b2ffd700 1 -- 172.21.15.3:0/1858786788 learned_addr learned my addr 172.21.15.3:0/1858786788 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:50.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.472+0000 7f71b37fe700 1 --2- 172.21.15.3:0/1858786788 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71b4133420 0x7f71b41337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:50.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.472+0000 7f71b3fff700 1 --2- 172.21.15.3:0/1858786788 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f71b4134260 0x7f71b41304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:50.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.472+0000 7f71b2ffd700 1 -- 172.21.15.3:0/1858786788 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71b4133420 msgr2=0x7f71b41337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:50.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.472+0000 7f71b2ffd700 1 --2- 172.21.15.3:0/1858786788 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71b4133420 0x7f71b41337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.472+0000 7f71b2ffd700 1 -- 172.21.15.3:0/1858786788 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f71b4134260 msgr2=0x7f71b41304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:50.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.472+0000 7f71b2ffd700 1 --2- 172.21.15.3:0/1858786788 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f71b4134260 0x7f71b41304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.472+0000 7f71b2ffd700 1 -- 172.21.15.3:0/1858786788 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b40a0030 con 0x7f71b4133e30 2023-10-03T10:14:50.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.472+0000 7f71b2ffd700 1 --2- 172.21.15.3:0/1858786788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71b4133e30 0x7f71b412ff90 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f71a80098f0 tx=0x7f71a8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1e846cbc999446 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:50.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.472+0000 7f71b27fc700 1 -- 172.21.15.3:0/1858786788 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a8011830 con 0x7f71b4133e30 2023-10-03T10:14:50.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.472+0000 7f71b27fc700 1 -- 172.21.15.3:0/1858786788 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a8011990 con 0x7f71b4133e30 2023-10-03T10:14:50.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.472+0000 7f71b27fc700 1 -- 172.21.15.3:0/1858786788 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a8011d20 con 0x7f71b4133e30 2023-10-03T10:14:50.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.473+0000 7f71b9e8b700 1 -- 172.21.15.3:0/1858786788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71b4133e30 msgr2=0x7f71b412ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:50.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.473+0000 7f71b9e8b700 1 --2- 172.21.15.3:0/1858786788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71b4133e30 0x7f71b412ff90 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f71a80098f0 tx=0x7f71a8009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:50.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.473+0000 7f71b9e8b700 1 -- 172.21.15.3:0/1858786788 shutdown_connections 2023-10-03T10:14:50.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.473+0000 7f71b9e8b700 1 --2- 172.21.15.3:0/1858786788 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f71b4134260 0x7f71b41304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.473+0000 7f71b9e8b700 1 --2- 172.21.15.3:0/1858786788 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71b4133420 0x7f71b41337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.473+0000 7f71b9e8b700 1 --2- 172.21.15.3:0/1858786788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71b4133e30 0x7f71b412ff90 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.473+0000 7f71b9e8b700 1 -- 172.21.15.3:0/1858786788 >> 172.21.15.3:0/1858786788 conn(0x7f71b40a4210 msgr2=0x7f71b40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:50.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.473+0000 7f71b9e8b700 1 -- 172.21.15.3:0/1858786788 shutdown_connections 2023-10-03T10:14:50.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.473+0000 7f71b9e8b700 1 -- 172.21.15.3:0/1858786788 wait complete. 2023-10-03T10:14:50.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.474+0000 7f71b9e8b700 1 Processor -- start 2023-10-03T10:14:50.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.474+0000 7f71b9e8b700 1 -- start start 2023-10-03T10:14:50.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.475+0000 7f71b9e8b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71b4133420 0x7f71b41c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:50.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.475+0000 7f71b9e8b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f71b4133e30 0x7f71b41c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:50.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.475+0000 7f71b9e8b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71b4134260 0x7f71b409c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:50.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.475+0000 7f71b9e8b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b4120010 con 0x7f71b4133420 2023-10-03T10:14:50.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.475+0000 7f71b9e8b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b41212a0 con 0x7f71b4133e30 2023-10-03T10:14:50.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.475+0000 7f71b9e8b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71b4120260 con 0x7f71b4134260 2023-10-03T10:14:50.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.475+0000 7f71b37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71b4133420 0x7f71b41c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:50.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.475+0000 7f71b37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71b4133420 0x7f71b41c89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56378/0 (socket says 172.21.15.3:56378) 2023-10-03T10:14:50.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.475+0000 7f71b37fe700 1 -- 172.21.15.3:0/1683089641 learned_addr learned my addr 172.21.15.3:0/1683089641 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:50.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.475+0000 7f71b3fff700 1 --2- 172.21.15.3:0/1683089641 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71b4134260 0x7f71b409c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:50.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.475+0000 7f71b2ffd700 1 --2- 172.21.15.3:0/1683089641 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f71b4133e30 0x7f71b41c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:50.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.475+0000 7f71b37fe700 1 -- 172.21.15.3:0/1683089641 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71b4134260 msgr2=0x7f71b409c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:50.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.475+0000 7f71b37fe700 1 --2- 172.21.15.3:0/1683089641 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71b4134260 0x7f71b409c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.475+0000 7f71b37fe700 1 -- 172.21.15.3:0/1683089641 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f71b4133e30 msgr2=0x7f71b41c8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:50.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.475+0000 7f71b37fe700 1 --2- 172.21.15.3:0/1683089641 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f71b4133e30 0x7f71b41c8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.475+0000 7f71b37fe700 1 -- 172.21.15.3:0/1683089641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a8009550 con 0x7f71b4133420 2023-10-03T10:14:50.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.476+0000 7f71b3fff700 1 --2- 172.21.15.3:0/1683089641 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71b4134260 0x7f71b409c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:14:50.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.476+0000 7f71b37fe700 1 --2- 172.21.15.3:0/1683089641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71b4133420 0x7f71b41c89b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f71a400e810 tx=0x7f71a400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:50.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.476+0000 7f719bfff700 1 -- 172.21.15.3:0/1683089641 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4013610 con 0x7f71b4133420 2023-10-03T10:14:50.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.476+0000 7f719bfff700 1 -- 172.21.15.3:0/1683089641 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a4013770 con 0x7f71b4133420 2023-10-03T10:14:50.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.476+0000 7f71b9e8b700 1 -- 172.21.15.3:0/1683089641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b4120cd0 con 0x7f71b4133420 2023-10-03T10:14:50.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.476+0000 7f719bfff700 1 -- 172.21.15.3:0/1683089641 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4013b20 con 0x7f71b4133420 2023-10-03T10:14:50.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.476+0000 7f71b9e8b700 1 -- 172.21.15.3:0/1683089641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b411ea90 con 0x7f71b4133420 2023-10-03T10:14:50.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.477+0000 7f71b9e8b700 1 -- 172.21.15.3:0/1683089641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b409e100 con 0x7f71b4133420 2023-10-03T10:14:50.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.478+0000 7f719bfff700 1 -- 172.21.15.3:0/1683089641 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f71a4013c80 con 0x7f71b4133420 2023-10-03T10:14:50.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.478+0000 7f719bfff700 1 -- 172.21.15.3:0/1683089641 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f71a403f800 con 0x7f71b4133420 2023-10-03T10:14:50.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.487+0000 7f719bfff700 1 -- 172.21.15.3:0/1683089641 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f71a4026480 con 0x7f71b4133420 2023-10-03T10:14:50.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.522+0000 7f7f978de700 1 -- 172.21.15.3:0/250672276 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7f901a95f0 con 0x7f7f9009e000 2023-10-03T10:14:50.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.522+0000 7f7f85ffb700 1 -- 172.21.15.3:0/250672276 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7f901a95f0 con 0x7f7f9009e000 2023-10-03T10:14:50.523 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:50.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.525+0000 7f7f978de700 1 -- 172.21.15.3:0/250672276 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f9009e000 msgr2=0x7f7f901a5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:50.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.525+0000 7f7f978de700 1 --2- 172.21.15.3:0/250672276 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f9009e000 0x7f7f901a5f50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c009fd0 tx=0x7f7f8c004750 comp rx=0 tx=0).stop 2023-10-03T10:14:50.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.526+0000 7f7f978de700 1 -- 172.21.15.3:0/250672276 shutdown_connections 2023-10-03T10:14:50.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.526+0000 7f7f978de700 1 --2- 172.21.15.3:0/250672276 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f901a9f70 0x7f7f901a69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.526+0000 7f7f978de700 1 --2- 172.21.15.3:0/250672276 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f9009e000 0x7f7f901a5f50 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.526+0000 7f7f978de700 1 --2- 172.21.15.3:0/250672276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f9009e410 0x7f7f901a6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.526+0000 7f7f978de700 1 -- 172.21.15.3:0/250672276 >> 172.21.15.3:0/250672276 conn(0x7f7f90066f50 msgr2=0x7f7f901baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:50.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.526+0000 7f7f978de700 1 -- 172.21.15.3:0/250672276 shutdown_connections 2023-10-03T10:14:50.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.526+0000 7f7f978de700 1 -- 172.21.15.3:0/250672276 wait complete. 2023-10-03T10:14:50.545 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:50.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.650+0000 7f041134d700 1 -- 172.21.15.3:0/2561290009 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f040c124580 con 0x7f040c0a0a30 2023-10-03T10:14:50.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.650+0000 7f03f37fe700 1 -- 172.21.15.3:0/2561290009 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0400021070 con 0x7f040c0a0a30 2023-10-03T10:14:50.651 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:50.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.653+0000 7f041134d700 1 -- 172.21.15.3:0/2561290009 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f040c0a0a30 msgr2=0x7f040c1217d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:50.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.653+0000 7f041134d700 1 --2- 172.21.15.3:0/2561290009 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f040c0a0a30 0x7f040c1217d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f040000e980 tx=0x7f040000ec90 comp rx=0 tx=0).stop 2023-10-03T10:14:50.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.653+0000 7f041134d700 1 -- 172.21.15.3:0/2561290009 shutdown_connections 2023-10-03T10:14:50.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.653+0000 7f041134d700 1 --2- 172.21.15.3:0/2561290009 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f040c133930 0x7f040c121d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.653+0000 7f041134d700 1 --2- 172.21.15.3:0/2561290009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f040c09d630 0x7f040c121290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.653+0000 7f041134d700 1 --2- 172.21.15.3:0/2561290009 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f040c0a0a30 0x7f040c1217d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.653+0000 7f041134d700 1 -- 172.21.15.3:0/2561290009 >> 172.21.15.3:0/2561290009 conn(0x7f040c0a3050 msgr2=0x7f040c1331a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:50.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.653+0000 7f041134d700 1 -- 172.21.15.3:0/2561290009 shutdown_connections 2023-10-03T10:14:50.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.653+0000 7f041134d700 1 -- 172.21.15.3:0/2561290009 wait complete. 2023-10-03T10:14:50.667 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:50.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.674+0000 7fd8a40a5700 1 -- 172.21.15.3:0/4523302 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd89c074ea0 con 0x7fd89c06a770 2023-10-03T10:14:50.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.674+0000 7fd8927fc700 1 -- 172.21.15.3:0/4523302 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd89801a360 con 0x7fd89c06a770 2023-10-03T10:14:50.675 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:50.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.677+0000 7fd8a40a5700 1 -- 172.21.15.3:0/4523302 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd89c06a770 msgr2=0x7fd89c1327a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:50.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.677+0000 7fd8a40a5700 1 --2- 172.21.15.3:0/4523302 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd89c06a770 0x7fd89c1327a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd89800c400 tx=0x7fd89800c710 comp rx=0 tx=0).stop 2023-10-03T10:14:50.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.677+0000 7fd8a40a5700 1 -- 172.21.15.3:0/4523302 shutdown_connections 2023-10-03T10:14:50.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.677+0000 7fd8a40a5700 1 --2- 172.21.15.3:0/4523302 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd89c136d80 0x7fd89c133220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.677+0000 7fd8a40a5700 1 --2- 172.21.15.3:0/4523302 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd89c06a770 0x7fd89c1327a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.677+0000 7fd8a40a5700 1 --2- 172.21.15.3:0/4523302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd89c06b250 0x7fd89c132ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.677+0000 7fd8a40a5700 1 -- 172.21.15.3:0/4523302 >> 172.21.15.3:0/4523302 conn(0x7fd89c066f50 msgr2=0x7fd89c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:50.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.677+0000 7fd8a40a5700 1 -- 172.21.15.3:0/4523302 shutdown_connections 2023-10-03T10:14:50.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.677+0000 7fd8a40a5700 1 -- 172.21.15.3:0/4523302 wait complete. 2023-10-03T10:14:50.690 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.696+0000 7f71b9e8b700 1 -- 172.21.15.3:0/1683089641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f71b40a14e0 con 0x7f71b4133420 2023-10-03T10:14:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.696+0000 7f719bfff700 1 -- 172.21.15.3:0/1683089641 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f71a402b090 con 0x7f71b4133420 2023-10-03T10:14:50.696 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.698+0000 7f71b9e8b700 1 -- 172.21.15.3:0/1683089641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71b4133420 msgr2=0x7f71b41c89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.698+0000 7f71b9e8b700 1 --2- 172.21.15.3:0/1683089641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71b4133420 0x7f71b41c89b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f71a400e810 tx=0x7f71a400eb20 comp rx=0 tx=0).stop 2023-10-03T10:14:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.698+0000 7f71b9e8b700 1 -- 172.21.15.3:0/1683089641 shutdown_connections 2023-10-03T10:14:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.698+0000 7f71b9e8b700 1 --2- 172.21.15.3:0/1683089641 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71b4134260 0x7f71b409c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.698+0000 7f71b9e8b700 1 --2- 172.21.15.3:0/1683089641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71b4133420 0x7f71b41c89b0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.698+0000 7f71b9e8b700 1 --2- 172.21.15.3:0/1683089641 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f71b4133e30 0x7f71b41c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.698+0000 7f71b9e8b700 1 -- 172.21.15.3:0/1683089641 >> 172.21.15.3:0/1683089641 conn(0x7f71b40a4210 msgr2=0x7f71b412de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.698+0000 7f71b9e8b700 1 -- 172.21.15.3:0/1683089641 shutdown_connections 2023-10-03T10:14:50.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:50.698+0000 7f71b9e8b700 1 -- 172.21.15.3:0/1683089641 wait complete. 2023-10-03T10:14:50.712 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:51.547 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:51.667 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:51.691 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:51.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.694+0000 7f63d7ef7700 1 Processor -- start 2023-10-03T10:14:51.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.694+0000 7f63d7ef7700 1 -- start start 2023-10-03T10:14:51.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.694+0000 7f63d7ef7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f63d009d6f0 0x7f63d009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:51.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.694+0000 7f63d7ef7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f63d009e000 0x7f63d0130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:51.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.694+0000 7f63d7ef7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63d009e410 0x7f63d0130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:51.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.695+0000 7f63d7ef7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d009f520 con 0x7f63d009e000 2023-10-03T10:14:51.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.695+0000 7f63d7ef7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d009f130 con 0x7f63d009d6f0 2023-10-03T10:14:51.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.695+0000 7f63d7ef7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f63d009e930 con 0x7f63d009e410 2023-10-03T10:14:51.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.695+0000 7f63d5c93700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f63d009d6f0 0x7f63d009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:51.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.695+0000 7f63d6494700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63d009e410 0x7f63d0130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:51.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.695+0000 7f63d5492700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f63d009e000 0x7f63d0130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:51.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.695+0000 7f63d5c93700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f63d009d6f0 0x7f63d009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59326/0 (socket says 172.21.15.3:59326) 2023-10-03T10:14:51.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.695+0000 7f63d5492700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f63d009e000 0x7f63d0130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56382/0 (socket says 172.21.15.3:56382) 2023-10-03T10:14:51.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.695+0000 7f63d6494700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63d009e410 0x7f63d0130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57594/0 (socket says 172.21.15.3:57594) 2023-10-03T10:14:51.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.695+0000 7f63d5c93700 1 -- 172.21.15.3:0/291595458 learned_addr learned my addr 172.21.15.3:0/291595458 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:51.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.695+0000 7f63d5c93700 1 -- 172.21.15.3:0/291595458 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63d009e410 msgr2=0x7f63d0130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:51.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.695+0000 7f63d5c93700 1 --2- 172.21.15.3:0/291595458 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63d009e410 0x7f63d0130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:51.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.695+0000 7f63d5c93700 1 -- 172.21.15.3:0/291595458 >> v2:172.21.15.3:3300/0 conn(0x7f63d009e000 msgr2=0x7f63d0130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:51.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.695+0000 7f63d5c93700 1 --2- 172.21.15.3:0/291595458 >> v2:172.21.15.3:3300/0 conn(0x7f63d009e000 0x7f63d0130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:51.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.695+0000 7f63d5c93700 1 -- 172.21.15.3:0/291595458 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d009b2d0 con 0x7f63d009d6f0 2023-10-03T10:14:51.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.695+0000 7f63d6494700 1 --2- 172.21.15.3:0/291595458 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63d009e410 0x7f63d0130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:51.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.695+0000 7f63d5492700 1 --2- 172.21.15.3:0/291595458 >> v2:172.21.15.3:3300/0 conn(0x7f63d009e000 0x7f63d0130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:14:51.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.695+0000 7f63d5c93700 1 --2- 172.21.15.3:0/291595458 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f63d009d6f0 0x7f63d009dac0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f63cc0098f0 tx=0x7f63cc009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74f00bb423623e39 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:51.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.696+0000 7f63d4c91700 1 -- 172.21.15.3:0/291595458 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc011830 con 0x7f63d009d6f0 2023-10-03T10:14:51.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.696+0000 7f63d4c91700 1 -- 172.21.15.3:0/291595458 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63cc011990 con 0x7f63d009d6f0 2023-10-03T10:14:51.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.696+0000 7f63d4c91700 1 -- 172.21.15.3:0/291595458 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc011d20 con 0x7f63d009d6f0 2023-10-03T10:14:51.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.696+0000 7f63d7ef7700 1 -- 172.21.15.3:0/291595458 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f63d009d6f0 msgr2=0x7f63d009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:51.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.696+0000 7f63d7ef7700 1 --2- 172.21.15.3:0/291595458 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f63d009d6f0 0x7f63d009dac0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f63cc0098f0 tx=0x7f63cc009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:51.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.697+0000 7f63d7ef7700 1 -- 172.21.15.3:0/291595458 shutdown_connections 2023-10-03T10:14:51.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.697+0000 7f63d7ef7700 1 --2- 172.21.15.3:0/291595458 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63d009e410 0x7f63d0130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:51.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.697+0000 7f63d7ef7700 1 --2- 172.21.15.3:0/291595458 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f63d009d6f0 0x7f63d009dac0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:51.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.697+0000 7f63d7ef7700 1 --2- 172.21.15.3:0/291595458 >> v2:172.21.15.3:3300/0 conn(0x7f63d009e000 0x7f63d0130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:51.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.697+0000 7f63d7ef7700 1 -- 172.21.15.3:0/291595458 >> 172.21.15.3:0/291595458 conn(0x7f63d0066f50 msgr2=0x7f63d009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:51.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.697+0000 7f63d7ef7700 1 -- 172.21.15.3:0/291595458 shutdown_connections 2023-10-03T10:14:51.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.697+0000 7f63d7ef7700 1 -- 172.21.15.3:0/291595458 wait complete. 2023-10-03T10:14:51.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.698+0000 7f63d7ef7700 1 Processor -- start 2023-10-03T10:14:51.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.698+0000 7f63d7ef7700 1 -- start start 2023-10-03T10:14:51.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.698+0000 7f63d7ef7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f63d009d6f0 0x7f63d011e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:51.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.698+0000 7f63d7ef7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63d009e000 0x7f63d011e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:51.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.699+0000 7f63d7ef7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63d009e410 0x7f63d011ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:51.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.699+0000 7f63d7ef7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d011f5a0 con 0x7f63d009e410 2023-10-03T10:14:51.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.699+0000 7f63d7ef7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d0121c60 con 0x7f63d009d6f0 2023-10-03T10:14:51.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.699+0000 7f63d7ef7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f63d0120300 con 0x7f63d009e000 2023-10-03T10:14:51.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.699+0000 7f63d5492700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63d009e000 0x7f63d011e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:51.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.699+0000 7f63d5492700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63d009e000 0x7f63d011e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57596/0 (socket says 172.21.15.3:57596) 2023-10-03T10:14:51.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.699+0000 7f63d5492700 1 -- 172.21.15.3:0/3085171778 learned_addr learned my addr 172.21.15.3:0/3085171778 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:51.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.699+0000 7f63d6494700 1 --2- 172.21.15.3:0/3085171778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63d009e410 0x7f63d011ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:51.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.699+0000 7f63d5c93700 1 --2- 172.21.15.3:0/3085171778 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f63d009d6f0 0x7f63d011e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:51.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.699+0000 7f63d6494700 1 -- 172.21.15.3:0/3085171778 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63d009e000 msgr2=0x7f63d011e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.699+0000 7f63d6494700 1 --2- 172.21.15.3:0/3085171778 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63d009e000 0x7f63d011e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.699+0000 7f63d6494700 1 -- 172.21.15.3:0/3085171778 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f63d009d6f0 msgr2=0x7f63d011e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.699+0000 7f63d6494700 1 --2- 172.21.15.3:0/3085171778 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f63d009d6f0 0x7f63d011e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.699+0000 7f63d6494700 1 -- 172.21.15.3:0/3085171778 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c800ba90 con 0x7f63d009e410 2023-10-03T10:14:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.699+0000 7f63d5492700 1 --2- 172.21.15.3:0/3085171778 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63d009e000 0x7f63d011e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.699+0000 7f63d6494700 1 --2- 172.21.15.3:0/3085171778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63d009e410 0x7f63d011ebc0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f63c800bec0 tx=0x7f63c800f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.700+0000 7f63c67fc700 1 -- 172.21.15.3:0/3085171778 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8010040 con 0x7f63d009e410 2023-10-03T10:14:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.700+0000 7f63c67fc700 1 -- 172.21.15.3:0/3085171778 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c800fa30 con 0x7f63d009e410 2023-10-03T10:14:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.700+0000 7f63d7ef7700 1 -- 172.21.15.3:0/3085171778 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63cc009550 con 0x7f63d009e410 2023-10-03T10:14:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.700+0000 7f63c67fc700 1 -- 172.21.15.3:0/3085171778 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c800fde0 con 0x7f63d009e410 2023-10-03T10:14:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.700+0000 7f63d7ef7700 1 -- 172.21.15.3:0/3085171778 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d009c700 con 0x7f63d009e410 2023-10-03T10:14:51.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.701+0000 7f63d7ef7700 1 -- 172.21.15.3:0/3085171778 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d0074ea0 con 0x7f63d009e410 2023-10-03T10:14:51.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.701+0000 7f63c67fc700 1 -- 172.21.15.3:0/3085171778 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f63c8009540 con 0x7f63d009e410 2023-10-03T10:14:51.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.702+0000 7f63c67fc700 1 -- 172.21.15.3:0/3085171778 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f63c803ff00 con 0x7f63d009e410 2023-10-03T10:14:51.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.706+0000 7f63c67fc700 1 -- 172.21.15.3:0/3085171778 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f63c801d720 con 0x7f63d009e410 2023-10-03T10:14:51.713 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:51.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.813+0000 7f4f72c7f700 1 Processor -- start 2023-10-03T10:14:51.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.814+0000 7f4f72c7f700 1 -- start start 2023-10-03T10:14:51.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.814+0000 7f4f72c7f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f6c09d6f0 0x7f4f6c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:51.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.814+0000 7f4f72c7f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4f6c09e000 0x7f4f6c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:51.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.814+0000 7f4f72c7f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f6c09f130 con 0x7f4f6c09e410 2023-10-03T10:14:51.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.814+0000 7f4f72c7f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c09f520 con 0x7f4f6c09d6f0 2023-10-03T10:14:51.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.814+0000 7f4f72c7f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c09e930 con 0x7f4f6c09e000 2023-10-03T10:14:51.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.814+0000 7f4f7121c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4f6c09e410 0x7f4f6c130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54350/0 (socket says 172.21.15.3:54350) 2023-10-03T10:14:51.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.814+0000 7f4f7121c700 1 -- 172.21.15.3:0/2306799642 learned_addr learned my addr 172.21.15.3:0/2306799642 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:51.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.814+0000 7f4f70a1b700 1 --2- 172.21.15.3:0/2306799642 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f6c09d6f0 0x7f4f6c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:51.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.814+0000 7f4f6bfff700 1 --2- 172.21.15.3:0/2306799642 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4f6c09e000 0x7f4f6c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:51.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.815+0000 7f4f6b7fe700 1 -- 172.21.15.3:0/2306799642 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1549496656 0 0) 0x7f4f6c09f130 con 0x7f4f6c09e410 2023-10-03T10:14:51.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.815+0000 7f4f6b7fe700 1 -- 172.21.15.3:0/2306799642 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f5c0035b0 con 0x7f4f6c09e410 2023-10-03T10:14:51.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.815+0000 7f4f6bfff700 1 -- 172.21.15.3:0/2306799642 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f6c09d6f0 msgr2=0x7f4f6c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:51.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.815+0000 7f4f6bfff700 1 --2- 172.21.15.3:0/2306799642 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f6c09d6f0 0x7f4f6c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:51.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.815+0000 7f4f6bfff700 1 -- 172.21.15.3:0/2306799642 >> v1:172.21.15.3:6789/0 conn(0x7f4f6c09e410 legacy=0x7f4f6c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:51.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.815+0000 7f4f6bfff700 1 -- 172.21.15.3:0/2306799642 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6c09b2d0 con 0x7f4f6c09e000 2023-10-03T10:14:51.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.815+0000 7f4f6bfff700 1 --2- 172.21.15.3:0/2306799642 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4f6c09e000 0x7f4f6c130660 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4f580098a0 tx=0x7f4f58009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cad5cdd60a48c5f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:51.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.815+0000 7f4f6b7fe700 1 -- 172.21.15.3:0/2306799642 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f58011830 con 0x7f4f6c09e000 2023-10-03T10:14:51.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.815+0000 7f4f6b7fe700 1 -- 172.21.15.3:0/2306799642 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f58011990 con 0x7f4f6c09e000 2023-10-03T10:14:51.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.815+0000 7f4f6b7fe700 1 -- 172.21.15.3:0/2306799642 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f58011d20 con 0x7f4f6c09e000 2023-10-03T10:14:51.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.816+0000 7f4f72c7f700 1 -- 172.21.15.3:0/2306799642 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4f6c09e000 msgr2=0x7f4f6c130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:51.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.816+0000 7f4f72c7f700 1 --2- 172.21.15.3:0/2306799642 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4f6c09e000 0x7f4f6c130660 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4f580098a0 tx=0x7f4f58009bb0 comp rx=0 tx=0).stop 2023-10-03T10:14:51.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.816+0000 7f4f72c7f700 1 -- 172.21.15.3:0/2306799642 shutdown_connections 2023-10-03T10:14:51.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.816+0000 7f4f72c7f700 1 --2- 172.21.15.3:0/2306799642 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f6c09d6f0 0x7f4f6c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:51.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.816+0000 7f4f72c7f700 1 --2- 172.21.15.3:0/2306799642 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4f6c09e000 0x7f4f6c130660 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:51.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.816+0000 7f4f72c7f700 1 -- 172.21.15.3:0/2306799642 >> 172.21.15.3:0/2306799642 conn(0x7f4f6c066f50 msgr2=0x7f4f6c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:51.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.816+0000 7f4f72c7f700 1 -- 172.21.15.3:0/2306799642 shutdown_connections 2023-10-03T10:14:51.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.817+0000 7f4f72c7f700 1 -- 172.21.15.3:0/2306799642 wait complete. 2023-10-03T10:14:51.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.818+0000 7f4f72c7f700 1 Processor -- start 2023-10-03T10:14:51.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.818+0000 7f4f72c7f700 1 -- start start 2023-10-03T10:14:51.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.819+0000 7f4f72c7f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f6c09d6f0 0x7f4f6c139940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:51.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.819+0000 7f4f72c7f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4f6c09e000 0x7f4f6c139e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:51.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.819+0000 7f4f72c7f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f6c09e410 0x7f4f6c13a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:51.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.819+0000 7f4f72c7f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c13b3b0 con 0x7f4f6c09e410 2023-10-03T10:14:51.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.819+0000 7f4f72c7f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c13ab40 con 0x7f4f6c09d6f0 2023-10-03T10:14:51.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.819+0000 7f4f72c7f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c135cc0 con 0x7f4f6c09e000 2023-10-03T10:14:51.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.819+0000 7f4f70a1b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f6c09d6f0 0x7f4f6c139940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:51.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.819+0000 7f4f7121c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f6c09e410 0x7f4f6c13a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:51.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.819+0000 7f4f70a1b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f6c09d6f0 0x7f4f6c139940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59350/0 (socket says 172.21.15.3:59350) 2023-10-03T10:14:51.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.819+0000 7f4f70a1b700 1 -- 172.21.15.3:0/4151986231 learned_addr learned my addr 172.21.15.3:0/4151986231 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:51.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.819+0000 7f4f6bfff700 1 --2- 172.21.15.3:0/4151986231 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4f6c09e000 0x7f4f6c139e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:51.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.820+0000 7f4f70a1b700 1 -- 172.21.15.3:0/4151986231 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4f6c09e000 msgr2=0x7f4f6c139e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:51.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.820+0000 7f4f70a1b700 1 --2- 172.21.15.3:0/4151986231 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4f6c09e000 0x7f4f6c139e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:51.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.820+0000 7f4f70a1b700 1 -- 172.21.15.3:0/4151986231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f6c09e410 msgr2=0x7f4f6c13a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:51.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.820+0000 7f4f70a1b700 1 --2- 172.21.15.3:0/4151986231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f6c09e410 0x7f4f6c13a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:51.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.820+0000 7f4f70a1b700 1 -- 172.21.15.3:0/4151986231 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f580094a0 con 0x7f4f6c09d6f0 2023-10-03T10:14:51.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.820+0000 7f4f7121c700 1 --2- 172.21.15.3:0/4151986231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f6c09e410 0x7f4f6c13a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:51.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.820+0000 7f4f70a1b700 1 --2- 172.21.15.3:0/4151986231 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f6c09d6f0 0x7f4f6c139940 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000e9b0 tx=0x7f4f6000ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:51.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.820+0000 7f4f697fa700 1 -- 172.21.15.3:0/4151986231 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f60009710 con 0x7f4f6c09d6f0 2023-10-03T10:14:51.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.820+0000 7f4f697fa700 1 -- 172.21.15.3:0/4151986231 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f6000ee50 con 0x7f4f6c09d6f0 2023-10-03T10:14:51.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.820+0000 7f4f72c7f700 1 -- 172.21.15.3:0/4151986231 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f6c1372a0 con 0x7f4f6c09d6f0 2023-10-03T10:14:51.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.820+0000 7f4f697fa700 1 -- 172.21.15.3:0/4151986231 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6000ca50 con 0x7f4f6c09d6f0 2023-10-03T10:14:51.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.820+0000 7f4f72c7f700 1 -- 172.21.15.3:0/4151986231 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f6c0a1de0 con 0x7f4f6c09d6f0 2023-10-03T10:14:51.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.822+0000 7f4f72c7f700 1 -- 172.21.15.3:0/4151986231 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f6c1364f0 con 0x7f4f6c09d6f0 2023-10-03T10:14:51.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.822+0000 7f4f697fa700 1 -- 172.21.15.3:0/4151986231 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4f6000cbb0 con 0x7f4f6c09d6f0 2023-10-03T10:14:51.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.823+0000 7f4f697fa700 1 -- 172.21.15.3:0/4151986231 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4f6003fad0 con 0x7f4f6c09d6f0 2023-10-03T10:14:51.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.832+0000 7f4f697fa700 1 -- 172.21.15.3:0/4151986231 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4f6c1364f0 con 0x7f4f6c09d6f0 2023-10-03T10:14:51.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.843+0000 7f33bf172700 1 Processor -- start 2023-10-03T10:14:51.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.843+0000 7f33bf172700 1 -- start start 2023-10-03T10:14:51.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.844+0000 7f33bf172700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33b809d6f0 0x7f33b809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:51.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.844+0000 7f33bf172700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f33b809e000 0x7f33b8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:51.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.844+0000 7f33bf172700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f33b809e410 0x7f33b8130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:51.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.844+0000 7f33bf172700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b809f520 con 0x7f33b809e000 2023-10-03T10:14:51.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.844+0000 7f33bf172700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b809f130 con 0x7f33b809d6f0 2023-10-03T10:14:51.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.844+0000 7f33bf172700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f33b809e930 con 0x7f33b809e410 2023-10-03T10:14:51.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.844+0000 7f33bcf0e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33b809d6f0 0x7f33b809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:51.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.844+0000 7f33bcf0e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33b809d6f0 0x7f33b809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59362/0 (socket says 172.21.15.3:59362) 2023-10-03T10:14:51.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.844+0000 7f33bcf0e700 1 -- 172.21.15.3:0/406010090 learned_addr learned my addr 172.21.15.3:0/406010090 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:51.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.844+0000 7f33b7fff700 1 --2- 172.21.15.3:0/406010090 >> v2:172.21.15.3:3300/0 conn(0x7f33b809e000 0x7f33b8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:51.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.844+0000 7f33bd70f700 1 --2- 172.21.15.3:0/406010090 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f33b809e410 0x7f33b8130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:51.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.845+0000 7f33bcf0e700 1 -- 172.21.15.3:0/406010090 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f33b809e410 msgr2=0x7f33b8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:51.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.845+0000 7f33bcf0e700 1 --2- 172.21.15.3:0/406010090 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f33b809e410 0x7f33b8130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:51.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.845+0000 7f33bcf0e700 1 -- 172.21.15.3:0/406010090 >> v2:172.21.15.3:3300/0 conn(0x7f33b809e000 msgr2=0x7f33b8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:51.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.845+0000 7f33bcf0e700 1 --2- 172.21.15.3:0/406010090 >> v2:172.21.15.3:3300/0 conn(0x7f33b809e000 0x7f33b8130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:51.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.845+0000 7f33bcf0e700 1 -- 172.21.15.3:0/406010090 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b809b2d0 con 0x7f33b809d6f0 2023-10-03T10:14:51.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.845+0000 7f33bcf0e700 1 --2- 172.21.15.3:0/406010090 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33b809d6f0 0x7f33b809dac0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0098f0 tx=0x7f33ac009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70fca3ebf0eae25c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:51.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.845+0000 7f33b77fe700 1 -- 172.21.15.3:0/406010090 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac011830 con 0x7f33b809d6f0 2023-10-03T10:14:51.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.845+0000 7f33b77fe700 1 -- 172.21.15.3:0/406010090 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ac011990 con 0x7f33b809d6f0 2023-10-03T10:14:51.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.845+0000 7f33b77fe700 1 -- 172.21.15.3:0/406010090 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac011d20 con 0x7f33b809d6f0 2023-10-03T10:14:51.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.846+0000 7f33bf172700 1 -- 172.21.15.3:0/406010090 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33b809d6f0 msgr2=0x7f33b809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:51.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.846+0000 7f33bf172700 1 --2- 172.21.15.3:0/406010090 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33b809d6f0 0x7f33b809dac0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0098f0 tx=0x7f33ac009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:51.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.846+0000 7f33bf172700 1 -- 172.21.15.3:0/406010090 shutdown_connections 2023-10-03T10:14:51.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.846+0000 7f33bf172700 1 --2- 172.21.15.3:0/406010090 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f33b809e410 0x7f33b8130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:51.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.846+0000 7f33bf172700 1 --2- 172.21.15.3:0/406010090 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33b809d6f0 0x7f33b809dac0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:51.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.846+0000 7f33bf172700 1 --2- 172.21.15.3:0/406010090 >> v2:172.21.15.3:3300/0 conn(0x7f33b809e000 0x7f33b8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:51.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.846+0000 7f33bf172700 1 -- 172.21.15.3:0/406010090 >> 172.21.15.3:0/406010090 conn(0x7f33b8066f50 msgr2=0x7f33b809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:51.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.846+0000 7f33bf172700 1 -- 172.21.15.3:0/406010090 shutdown_connections 2023-10-03T10:14:51.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.846+0000 7f33bf172700 1 -- 172.21.15.3:0/406010090 wait complete. 2023-10-03T10:14:51.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.847+0000 7f33bf172700 1 Processor -- start 2023-10-03T10:14:51.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.847+0000 7f33bf172700 1 -- start start 2023-10-03T10:14:51.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.848+0000 7f33bf172700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f33b809d6f0 0x7f33b811e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:51.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.848+0000 7f33bf172700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33b809e000 0x7f33b811e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:51.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.848+0000 7f33bf172700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33b809e410 0x7f33b811eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:51.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.848+0000 7f33bf172700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b8121c70 con 0x7f33b809e410 2023-10-03T10:14:51.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.848+0000 7f33bf172700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b8120590 con 0x7f33b809e000 2023-10-03T10:14:51.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.848+0000 7f33bf172700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f33b8120270 con 0x7f33b809d6f0 2023-10-03T10:14:51.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.848+0000 7f33b7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33b809e000 0x7f33b811e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:51.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.848+0000 7f33b7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33b809e000 0x7f33b811e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59378/0 (socket says 172.21.15.3:59378) 2023-10-03T10:14:51.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.848+0000 7f33b7fff700 1 -- 172.21.15.3:0/768098624 learned_addr learned my addr 172.21.15.3:0/768098624 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:51.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.848+0000 7f33bcf0e700 1 --2- 172.21.15.3:0/768098624 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f33b809d6f0 0x7f33b811e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:51.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.848+0000 7f33bd70f700 1 --2- 172.21.15.3:0/768098624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33b809e410 0x7f33b811eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:51.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.849+0000 7f33b7fff700 1 -- 172.21.15.3:0/768098624 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f33b809d6f0 msgr2=0x7f33b811e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:51.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.849+0000 7f33b7fff700 1 --2- 172.21.15.3:0/768098624 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f33b809d6f0 0x7f33b811e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:51.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.849+0000 7f33b7fff700 1 -- 172.21.15.3:0/768098624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33b809e410 msgr2=0x7f33b811eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:51.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.849+0000 7f33b7fff700 1 --2- 172.21.15.3:0/768098624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33b809e410 0x7f33b811eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:51.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.849+0000 7f33b7fff700 1 -- 172.21.15.3:0/768098624 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ac009550 con 0x7f33b809e000 2023-10-03T10:14:51.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.849+0000 7f33b7fff700 1 --2- 172.21.15.3:0/768098624 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33b809e000 0x7f33b811e640 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f33a400e900 tx=0x7f33a400ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:51.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.849+0000 7f33b57fa700 1 -- 172.21.15.3:0/768098624 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a4013680 con 0x7f33b809e000 2023-10-03T10:14:51.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.849+0000 7f33b57fa700 1 -- 172.21.15.3:0/768098624 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a40137e0 con 0x7f33b809e000 2023-10-03T10:14:51.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.849+0000 7f33b57fa700 1 -- 172.21.15.3:0/768098624 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a4013b70 con 0x7f33b809e000 2023-10-03T10:14:51.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.849+0000 7f33bf172700 1 -- 172.21.15.3:0/768098624 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b80a3ad0 con 0x7f33b809e000 2023-10-03T10:14:51.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.849+0000 7f33bf172700 1 -- 172.21.15.3:0/768098624 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b81218f0 con 0x7f33b809e000 2023-10-03T10:14:51.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.851+0000 7f33b57fa700 1 -- 172.21.15.3:0/768098624 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f33a4013cd0 con 0x7f33b809e000 2023-10-03T10:14:51.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.851+0000 7f33bf172700 1 -- 172.21.15.3:0/768098624 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b811ac20 con 0x7f33b809e000 2023-10-03T10:14:51.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.851+0000 7f33b57fa700 1 -- 172.21.15.3:0/768098624 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f33a4042220 con 0x7f33b809e000 2023-10-03T10:14:51.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.856+0000 7f33b57fa700 1 -- 172.21.15.3:0/768098624 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f33a401be10 con 0x7f33b809e000 2023-10-03T10:14:51.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.877+0000 7efc66a69700 1 Processor -- start 2023-10-03T10:14:51.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.877+0000 7efc66a69700 1 -- start start 2023-10-03T10:14:51.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.878+0000 7efc66a69700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efc6009f410 0x7efc60136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:51.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.878+0000 7efc66a69700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efc60136fe0 0x7efc601393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:51.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.878+0000 7efc66a69700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc60070940 con 0x7efc6009e930 2023-10-03T10:14:51.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.878+0000 7efc66a69700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc6006fd60 con 0x7efc60136fe0 2023-10-03T10:14:51.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.878+0000 7efc66a69700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc6006fa80 con 0x7efc6009f410 2023-10-03T10:14:51.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.879+0000 7efc65006700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efc60136fe0 0x7efc601393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:51.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.879+0000 7efc5ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efc6009f410 0x7efc60136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:51.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.879+0000 7efc65006700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efc60136fe0 0x7efc601393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59394/0 (socket says 172.21.15.3:59394) 2023-10-03T10:14:51.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.879+0000 7efc5ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efc6009f410 0x7efc60136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:57644/0 (socket says 172.21.15.3:57644) 2023-10-03T10:14:51.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.879+0000 7efc65006700 1 -- 172.21.15.3:0/1734002193 learned_addr learned my addr 172.21.15.3:0/1734002193 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:51.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.879+0000 7efc5ffff700 1 -- 172.21.15.3:0/1734002193 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efc60136fe0 msgr2=0x7efc601393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:51.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.879+0000 7efc5ffff700 1 --2- 172.21.15.3:0/1734002193 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efc60136fe0 0x7efc601393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:51.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.879+0000 7efc5ffff700 1 -- 172.21.15.3:0/1734002193 >> v1:172.21.15.3:6789/0 conn(0x7efc6009e930 legacy=0x7efc6009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:51.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.879+0000 7efc5ffff700 1 -- 172.21.15.3:0/1734002193 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc6009fb60 con 0x7efc6009f410 2023-10-03T10:14:51.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.879+0000 7efc65006700 1 --2- 172.21.15.3:0/1734002193 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efc60136fe0 0x7efc601393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:51.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.879+0000 7efc5ffff700 1 --2- 172.21.15.3:0/1734002193 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efc6009f410 0x7efc60136aa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009870 tx=0x7efc4c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38db95024cba17d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:51.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.879+0000 7efc5f7fe700 1 -- 172.21.15.3:0/1734002193 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c011830 con 0x7efc6009f410 2023-10-03T10:14:51.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.880+0000 7efc5f7fe700 1 -- 172.21.15.3:0/1734002193 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4c011990 con 0x7efc6009f410 2023-10-03T10:14:51.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.880+0000 7efc5f7fe700 1 -- 172.21.15.3:0/1734002193 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c011d20 con 0x7efc6009f410 2023-10-03T10:14:51.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.880+0000 7efc66a69700 1 -- 172.21.15.3:0/1734002193 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efc6009f410 msgr2=0x7efc60136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:51.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.880+0000 7efc66a69700 1 --2- 172.21.15.3:0/1734002193 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efc6009f410 0x7efc60136aa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009870 tx=0x7efc4c009b80 comp rx=0 tx=0).stop 2023-10-03T10:14:51.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.880+0000 7efc66a69700 1 -- 172.21.15.3:0/1734002193 shutdown_connections 2023-10-03T10:14:51.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.880+0000 7efc66a69700 1 --2- 172.21.15.3:0/1734002193 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efc60136fe0 0x7efc601393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:51.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.880+0000 7efc66a69700 1 --2- 172.21.15.3:0/1734002193 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efc6009f410 0x7efc60136aa0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:51.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.880+0000 7efc66a69700 1 -- 172.21.15.3:0/1734002193 >> 172.21.15.3:0/1734002193 conn(0x7efc600a4210 msgr2=0x7efc600a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:51.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.880+0000 7efc66a69700 1 -- 172.21.15.3:0/1734002193 shutdown_connections 2023-10-03T10:14:51.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.880+0000 7efc66a69700 1 -- 172.21.15.3:0/1734002193 wait complete. 2023-10-03T10:14:51.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.882+0000 7efc66a69700 1 Processor -- start 2023-10-03T10:14:51.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.882+0000 7efc66a69700 1 -- start start 2023-10-03T10:14:51.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.883+0000 7efc66a69700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efc6009e930 0x7efc6006c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:51.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.883+0000 7efc66a69700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc6009f410 0x7efc6006c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:51.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.883+0000 7efc66a69700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efc60136fe0 0x7efc601c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:51.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.883+0000 7efc66a69700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc601321c0 con 0x7efc6009f410 2023-10-03T10:14:51.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.883+0000 7efc66a69700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc60121dc0 con 0x7efc6009e930 2023-10-03T10:14:51.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.883+0000 7efc66a69700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc6006bd70 con 0x7efc60136fe0 2023-10-03T10:14:51.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.883+0000 7efc64805700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efc6009e930 0x7efc6006c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:51.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.883+0000 7efc64805700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efc6009e930 0x7efc6006c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59398/0 (socket says 172.21.15.3:59398) 2023-10-03T10:14:51.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.883+0000 7efc64805700 1 -- 172.21.15.3:0/847352727 learned_addr learned my addr 172.21.15.3:0/847352727 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:51.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.883+0000 7efc65006700 1 --2- 172.21.15.3:0/847352727 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efc60136fe0 0x7efc601c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:51.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.883+0000 7efc5ffff700 1 --2- 172.21.15.3:0/847352727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc6009f410 0x7efc6006c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:51.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.883+0000 7efc65006700 1 -- 172.21.15.3:0/847352727 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efc6009e930 msgr2=0x7efc6006c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:51.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.883+0000 7efc65006700 1 --2- 172.21.15.3:0/847352727 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efc6009e930 0x7efc6006c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:51.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.883+0000 7efc65006700 1 -- 172.21.15.3:0/847352727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc6009f410 msgr2=0x7efc6006c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:51.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.883+0000 7efc65006700 1 --2- 172.21.15.3:0/847352727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc6009f410 0x7efc6006c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:51.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.883+0000 7efc65006700 1 -- 172.21.15.3:0/847352727 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc58009b30 con 0x7efc60136fe0 2023-10-03T10:14:51.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.884+0000 7efc65006700 1 --2- 172.21.15.3:0/847352727 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efc60136fe0 0x7efc601c8260 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efc600a07c0 tx=0x7efc5800ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:51.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.884+0000 7efc5d7fa700 1 -- 172.21.15.3:0/847352727 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5800ca10 con 0x7efc60136fe0 2023-10-03T10:14:51.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.884+0000 7efc5d7fa700 1 -- 172.21.15.3:0/847352727 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc5800f040 con 0x7efc60136fe0 2023-10-03T10:14:51.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.884+0000 7efc5d7fa700 1 -- 172.21.15.3:0/847352727 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5800cdb0 con 0x7efc60136fe0 2023-10-03T10:14:51.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.884+0000 7efc66a69700 1 -- 172.21.15.3:0/847352727 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc4c0094a0 con 0x7efc60136fe0 2023-10-03T10:14:51.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.884+0000 7efc66a69700 1 -- 172.21.15.3:0/847352727 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc60122440 con 0x7efc60136fe0 2023-10-03T10:14:51.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.885+0000 7efc5d7fa700 1 -- 172.21.15.3:0/847352727 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efc58014480 con 0x7efc60136fe0 2023-10-03T10:14:51.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.885+0000 7efc5d7fa700 1 -- 172.21.15.3:0/847352727 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7efc5803fbe0 con 0x7efc60136fe0 2023-10-03T10:14:51.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.885+0000 7efc66a69700 1 -- 172.21.15.3:0/847352727 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc600a30a0 con 0x7efc60136fe0 2023-10-03T10:14:51.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.895+0000 7efc5d7fa700 1 -- 172.21.15.3:0/847352727 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efc58012930 con 0x7efc60136fe0 2023-10-03T10:14:51.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.938+0000 7f63d7ef7700 1 -- 172.21.15.3:0/3085171778 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f63d0072310 con 0x7f63d009e410 2023-10-03T10:14:51.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.938+0000 7f63c67fc700 1 -- 172.21.15.3:0/3085171778 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f63c801d360 con 0x7f63d009e410 2023-10-03T10:14:51.939 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:51.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.941+0000 7f63d7ef7700 1 -- 172.21.15.3:0/3085171778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63d009e410 msgr2=0x7f63d011ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:51.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.941+0000 7f63d7ef7700 1 --2- 172.21.15.3:0/3085171778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63d009e410 0x7f63d011ebc0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f63c800bec0 tx=0x7f63c800f410 comp rx=0 tx=0).stop 2023-10-03T10:14:51.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.941+0000 7f63d7ef7700 1 -- 172.21.15.3:0/3085171778 shutdown_connections 2023-10-03T10:14:51.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.941+0000 7f63d7ef7700 1 --2- 172.21.15.3:0/3085171778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63d009e410 0x7f63d011ebc0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:51.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.941+0000 7f63d7ef7700 1 --2- 172.21.15.3:0/3085171778 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f63d009d6f0 0x7f63d011e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:51.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.941+0000 7f63d7ef7700 1 --2- 172.21.15.3:0/3085171778 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63d009e000 0x7f63d011e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:51.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.941+0000 7f63d7ef7700 1 -- 172.21.15.3:0/3085171778 >> 172.21.15.3:0/3085171778 conn(0x7f63d0066f50 msgr2=0x7f63d006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:51.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.941+0000 7f63d7ef7700 1 -- 172.21.15.3:0/3085171778 shutdown_connections 2023-10-03T10:14:51.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:51.941+0000 7f63d7ef7700 1 -- 172.21.15.3:0/3085171778 wait complete. 2023-10-03T10:14:51.955 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:52.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.069+0000 7f33bf172700 1 -- 172.21.15.3:0/768098624 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f33b8121730 con 0x7f33b809e000 2023-10-03T10:14:52.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.069+0000 7f33b57fa700 1 -- 172.21.15.3:0/768098624 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f33a401b3b0 con 0x7f33b809e000 2023-10-03T10:14:52.069 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:52.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.072+0000 7f33bf172700 1 -- 172.21.15.3:0/768098624 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33b809e000 msgr2=0x7f33b811e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:52.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.072+0000 7f33bf172700 1 --2- 172.21.15.3:0/768098624 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33b809e000 0x7f33b811e640 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f33a400e900 tx=0x7f33a400ec10 comp rx=0 tx=0).stop 2023-10-03T10:14:52.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.072+0000 7f33bf172700 1 -- 172.21.15.3:0/768098624 shutdown_connections 2023-10-03T10:14:52.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.072+0000 7f33bf172700 1 --2- 172.21.15.3:0/768098624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33b809e410 0x7f33b811eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:52.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.072+0000 7f33bf172700 1 --2- 172.21.15.3:0/768098624 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f33b809d6f0 0x7f33b811e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:52.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.072+0000 7f33bf172700 1 --2- 172.21.15.3:0/768098624 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33b809e000 0x7f33b811e640 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:52.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.072+0000 7f33bf172700 1 -- 172.21.15.3:0/768098624 >> 172.21.15.3:0/768098624 conn(0x7f33b8066f50 msgr2=0x7f33b8130890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:52.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.072+0000 7f33bf172700 1 -- 172.21.15.3:0/768098624 shutdown_connections 2023-10-03T10:14:52.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.073+0000 7f33bf172700 1 -- 172.21.15.3:0/768098624 wait complete. 2023-10-03T10:14:52.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.080+0000 7f4f72c7f700 1 -- 172.21.15.3:0/4151986231 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4f6c1364f0 con 0x7f4f6c09d6f0 2023-10-03T10:14:52.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.081+0000 7f4f697fa700 1 -- 172.21.15.3:0/4151986231 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4f6c1364f0 con 0x7f4f6c09d6f0 2023-10-03T10:14:52.081 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:52.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.083+0000 7f4f72c7f700 1 -- 172.21.15.3:0/4151986231 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f6c09d6f0 msgr2=0x7f4f6c139940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:52.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.083+0000 7f4f72c7f700 1 --2- 172.21.15.3:0/4151986231 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f6c09d6f0 0x7f4f6c139940 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000e9b0 tx=0x7f4f6000ecc0 comp rx=0 tx=0).stop 2023-10-03T10:14:52.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.083+0000 7f4f72c7f700 1 -- 172.21.15.3:0/4151986231 shutdown_connections 2023-10-03T10:14:52.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.083+0000 7f4f72c7f700 1 --2- 172.21.15.3:0/4151986231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f6c09e410 0x7f4f6c13a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:52.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.083+0000 7f4f72c7f700 1 --2- 172.21.15.3:0/4151986231 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f6c09d6f0 0x7f4f6c139940 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:52.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.083+0000 7f4f72c7f700 1 --2- 172.21.15.3:0/4151986231 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4f6c09e000 0x7f4f6c139e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:52.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.083+0000 7f4f72c7f700 1 -- 172.21.15.3:0/4151986231 >> 172.21.15.3:0/4151986231 conn(0x7f4f6c066f50 msgr2=0x7f4f6c070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:52.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.083+0000 7f4f72c7f700 1 -- 172.21.15.3:0/4151986231 shutdown_connections 2023-10-03T10:14:52.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.083+0000 7f4f72c7f700 1 -- 172.21.15.3:0/4151986231 wait complete. 2023-10-03T10:14:52.092 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:52.098 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:52.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.135+0000 7efc66a69700 1 -- 172.21.15.3:0/847352727 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7efc60074ea0 con 0x7efc60136fe0 2023-10-03T10:14:52.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.135+0000 7efc5d7fa700 1 -- 172.21.15.3:0/847352727 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efc58012340 con 0x7efc60136fe0 2023-10-03T10:14:52.135 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:52.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.137+0000 7efc66a69700 1 -- 172.21.15.3:0/847352727 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efc60136fe0 msgr2=0x7efc601c8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:52.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.137+0000 7efc66a69700 1 --2- 172.21.15.3:0/847352727 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efc60136fe0 0x7efc601c8260 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efc600a07c0 tx=0x7efc5800ef90 comp rx=0 tx=0).stop 2023-10-03T10:14:52.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.137+0000 7efc66a69700 1 -- 172.21.15.3:0/847352727 shutdown_connections 2023-10-03T10:14:52.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.137+0000 7efc66a69700 1 --2- 172.21.15.3:0/847352727 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efc60136fe0 0x7efc601c8260 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:52.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.137+0000 7efc66a69700 1 --2- 172.21.15.3:0/847352727 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efc6009e930 0x7efc6006c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:52.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.137+0000 7efc66a69700 1 --2- 172.21.15.3:0/847352727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc6009f410 0x7efc6006c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:52.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.137+0000 7efc66a69700 1 -- 172.21.15.3:0/847352727 >> 172.21.15.3:0/847352727 conn(0x7efc600a4210 msgr2=0x7efc6009bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:52.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.137+0000 7efc66a69700 1 -- 172.21.15.3:0/847352727 shutdown_connections 2023-10-03T10:14:52.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:52.137+0000 7efc66a69700 1 -- 172.21.15.3:0/847352727 wait complete. 2023-10-03T10:14:52.151 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:52.955 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:53.093 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:53.098 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:53.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.103+0000 7f691ae06700 1 Processor -- start 2023-10-03T10:14:53.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.103+0000 7f691ae06700 1 -- start start 2023-10-03T10:14:53.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.104+0000 7f691ae06700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f691409e930 0x7f691409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:53.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.104+0000 7f691ae06700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f691409f650 0x7f6914131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:53.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.104+0000 7f691ae06700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6914070940 con 0x7f691409f240 2023-10-03T10:14:53.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.104+0000 7f691ae06700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f691406fd60 con 0x7f691409f650 2023-10-03T10:14:53.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.104+0000 7f691ae06700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f691406fa80 con 0x7f691409e930 2023-10-03T10:14:53.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.104+0000 7f6918ba2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f691409e930 0x7f691409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:53.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.104+0000 7f69193a3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f691409f650 0x7f6914131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:53.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.104+0000 7f69193a3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f691409f650 0x7f6914131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56420/0 (socket says 172.21.15.3:56420) 2023-10-03T10:14:53.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.104+0000 7f6918ba2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f691409e930 0x7f691409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59408/0 (socket says 172.21.15.3:59408) 2023-10-03T10:14:53.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.104+0000 7f69193a3700 1 -- 172.21.15.3:0/3494743533 learned_addr learned my addr 172.21.15.3:0/3494743533 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:53.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.104+0000 7f6918ba2700 1 -- 172.21.15.3:0/3494743533 >> v2:172.21.15.3:3300/0 conn(0x7f691409f650 msgr2=0x7f6914131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:53.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.104+0000 7f6918ba2700 1 --2- 172.21.15.3:0/3494743533 >> v2:172.21.15.3:3300/0 conn(0x7f691409f650 0x7f6914131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.104+0000 7f6918ba2700 1 -- 172.21.15.3:0/3494743533 >> v1:172.21.15.3:6789/0 conn(0x7f691409f240 legacy=0x7f69141317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:53.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.104+0000 7f6918ba2700 1 -- 172.21.15.3:0/3494743533 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691409fb60 con 0x7f691409e930 2023-10-03T10:14:53.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.104+0000 7f69193a3700 1 --2- 172.21.15.3:0/3494743533 >> v2:172.21.15.3:3300/0 conn(0x7f691409f650 0x7f6914131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:53.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.105+0000 7f6918ba2700 1 --2- 172.21.15.3:0/3494743533 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f691409e930 0x7f691409ed00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6908009840 tx=0x7f6908009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a19b0a978d3f4940 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:53.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.105+0000 7f69137fe700 1 -- 172.21.15.3:0/3494743533 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6908011830 con 0x7f691409e930 2023-10-03T10:14:53.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.105+0000 7f69137fe700 1 -- 172.21.15.3:0/3494743533 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6908011990 con 0x7f691409e930 2023-10-03T10:14:53.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.105+0000 7f69137fe700 1 -- 172.21.15.3:0/3494743533 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6908011d20 con 0x7f691409e930 2023-10-03T10:14:53.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.106+0000 7f691ae06700 1 -- 172.21.15.3:0/3494743533 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f691409e930 msgr2=0x7f691409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:53.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.106+0000 7f691ae06700 1 --2- 172.21.15.3:0/3494743533 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f691409e930 0x7f691409ed00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6908009840 tx=0x7f6908009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:53.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.106+0000 7f691ae06700 1 -- 172.21.15.3:0/3494743533 shutdown_connections 2023-10-03T10:14:53.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.106+0000 7f691ae06700 1 --2- 172.21.15.3:0/3494743533 >> v2:172.21.15.3:3300/0 conn(0x7f691409f650 0x7f6914131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.106+0000 7f691ae06700 1 --2- 172.21.15.3:0/3494743533 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f691409e930 0x7f691409ed00 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.106+0000 7f691ae06700 1 -- 172.21.15.3:0/3494743533 >> 172.21.15.3:0/3494743533 conn(0x7f69140a4210 msgr2=0x7f69140a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:53.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.106+0000 7f691ae06700 1 -- 172.21.15.3:0/3494743533 shutdown_connections 2023-10-03T10:14:53.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.106+0000 7f691ae06700 1 -- 172.21.15.3:0/3494743533 wait complete. 2023-10-03T10:14:53.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.107+0000 7f691ae06700 1 Processor -- start 2023-10-03T10:14:53.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.107+0000 7f691ae06700 1 -- start start 2023-10-03T10:14:53.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.108+0000 7f691ae06700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f691409e930 0x7f69141a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:53.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.108+0000 7f691ae06700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f691409f240 0x7f69141a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:53.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.108+0000 7f691ae06700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f691409f650 0x7f69141a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:53.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.108+0000 7f691ae06700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69141a5120 con 0x7f691409e930 2023-10-03T10:14:53.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.108+0000 7f691ae06700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69141a5880 con 0x7f691409f240 2023-10-03T10:14:53.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.108+0000 7f691ae06700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f69141a5ea0 con 0x7f691409f650 2023-10-03T10:14:53.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.108+0000 7f6918ba2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f691409e930 0x7f69141a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:53.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.108+0000 7f6913fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f691409f240 0x7f69141a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:53.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.108+0000 7f6918ba2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f691409e930 0x7f69141a3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56430/0 (socket says 172.21.15.3:56430) 2023-10-03T10:14:53.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.108+0000 7f6913fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f691409f240 0x7f69141a41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59418/0 (socket says 172.21.15.3:59418) 2023-10-03T10:14:53.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.108+0000 7f6918ba2700 1 -- 172.21.15.3:0/470411561 learned_addr learned my addr 172.21.15.3:0/470411561 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:53.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.108+0000 7f6918ba2700 1 -- 172.21.15.3:0/470411561 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f691409f650 msgr2=0x7f69141a46f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:14:53.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.108+0000 7f6918ba2700 1 --2- 172.21.15.3:0/470411561 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f691409f650 0x7f69141a46f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.108+0000 7f6918ba2700 1 -- 172.21.15.3:0/470411561 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f691409f240 msgr2=0x7f69141a41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:53.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.108+0000 7f6918ba2700 1 --2- 172.21.15.3:0/470411561 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f691409f240 0x7f69141a41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.108+0000 7f6918ba2700 1 -- 172.21.15.3:0/470411561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c009b30 con 0x7f691409e930 2023-10-03T10:14:53.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.109+0000 7f6913fff700 1 --2- 172.21.15.3:0/470411561 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f691409f240 0x7f69141a41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:53.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.109+0000 7f6918ba2700 1 --2- 172.21.15.3:0/470411561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f691409e930 0x7f69141a3c70 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f69080090b0 tx=0x7f6908004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:53.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.109+0000 7f69117fa700 1 -- 172.21.15.3:0/470411561 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69080036a0 con 0x7f691409e930 2023-10-03T10:14:53.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.109+0000 7f691ae06700 1 -- 172.21.15.3:0/470411561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69080094a0 con 0x7f691409e930 2023-10-03T10:14:53.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.109+0000 7f69117fa700 1 -- 172.21.15.3:0/470411561 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690801f070 con 0x7f691409e930 2023-10-03T10:14:53.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.109+0000 7f69117fa700 1 -- 172.21.15.3:0/470411561 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6908003a40 con 0x7f691409e930 2023-10-03T10:14:53.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.109+0000 7f691ae06700 1 -- 172.21.15.3:0/470411561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69140a0190 con 0x7f691409e930 2023-10-03T10:14:53.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.110+0000 7f69117fa700 1 -- 172.21.15.3:0/470411561 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6908003ba0 con 0x7f691409e930 2023-10-03T10:14:53.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.110+0000 7f69117fa700 1 -- 172.21.15.3:0/470411561 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6908021070 con 0x7f691409e930 2023-10-03T10:14:53.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.110+0000 7f691ae06700 1 -- 172.21.15.3:0/470411561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69141a0b30 con 0x7f691409e930 2023-10-03T10:14:53.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.116+0000 7f69117fa700 1 -- 172.21.15.3:0/470411561 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f690801a430 con 0x7f691409e930 2023-10-03T10:14:53.152 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:53.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.240+0000 7f594622a700 1 Processor -- start 2023-10-03T10:14:53.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.240+0000 7f594622a700 1 -- start start 2023-10-03T10:14:53.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.241+0000 7f594622a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f594012cf60 0x7f594006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:53.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.241+0000 7f594622a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f594012f8b0 0x7f594012fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:53.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.241+0000 7f594622a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5940130260 0x7f5940136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:53.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.241+0000 7f594622a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f594012d330 con 0x7f594012f8b0 2023-10-03T10:14:53.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.241+0000 7f594622a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594012cd90 con 0x7f594012cf60 2023-10-03T10:14:53.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.241+0000 7f594622a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f59401306d0 con 0x7f5940130260 2023-10-03T10:14:53.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.241+0000 7f593f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f594012cf60 0x7f594006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:53.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.241+0000 7f593f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f594012cf60 0x7f594006f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42366/0 (socket says 172.21.15.3:42366) 2023-10-03T10:14:53.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.241+0000 7f593f7fe700 1 -- 172.21.15.3:0/2685579515 learned_addr learned my addr 172.21.15.3:0/2685579515 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:53.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.241+0000 7f593effd700 1 --2- 172.21.15.3:0/2685579515 >> v2:172.21.15.3:3300/0 conn(0x7f594012f8b0 0x7f594012fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:53.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.242+0000 7f593f7fe700 1 -- 172.21.15.3:0/2685579515 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5940130260 msgr2=0x7f5940136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:53.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.242+0000 7f593f7fe700 1 --2- 172.21.15.3:0/2685579515 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5940130260 0x7f5940136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.242+0000 7f593f7fe700 1 -- 172.21.15.3:0/2685579515 >> v2:172.21.15.3:3300/0 conn(0x7f594012f8b0 msgr2=0x7f594012fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:53.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.242+0000 7f593f7fe700 1 --2- 172.21.15.3:0/2685579515 >> v2:172.21.15.3:3300/0 conn(0x7f594012f8b0 0x7f594012fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.242+0000 7f593f7fe700 1 -- 172.21.15.3:0/2685579515 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5940071780 con 0x7f594012cf60 2023-10-03T10:14:53.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.242+0000 7f593f7fe700 1 --2- 172.21.15.3:0/2685579515 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f594012cf60 0x7f594006f5d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f59300098f0 tx=0x7f5930009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=273772ee35d7ae52 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:53.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.242+0000 7f593e7fc700 1 -- 172.21.15.3:0/2685579515 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5930011830 con 0x7f594012cf60 2023-10-03T10:14:53.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.242+0000 7f593e7fc700 1 -- 172.21.15.3:0/2685579515 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5930011990 con 0x7f594012cf60 2023-10-03T10:14:53.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.242+0000 7f593e7fc700 1 -- 172.21.15.3:0/2685579515 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5930011d20 con 0x7f594012cf60 2023-10-03T10:14:53.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.243+0000 7f594622a700 1 -- 172.21.15.3:0/2685579515 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f594012cf60 msgr2=0x7f594006f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:53.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.243+0000 7f594622a700 1 --2- 172.21.15.3:0/2685579515 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f594012cf60 0x7f594006f5d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f59300098f0 tx=0x7f5930009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:53.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.243+0000 7f594622a700 1 -- 172.21.15.3:0/2685579515 shutdown_connections 2023-10-03T10:14:53.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.243+0000 7f594622a700 1 --2- 172.21.15.3:0/2685579515 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5940130260 0x7f5940136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.243+0000 7f594622a700 1 --2- 172.21.15.3:0/2685579515 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f594012cf60 0x7f594006f5d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.243+0000 7f594622a700 1 --2- 172.21.15.3:0/2685579515 >> v2:172.21.15.3:3300/0 conn(0x7f594012f8b0 0x7f594012fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.243+0000 7f594622a700 1 -- 172.21.15.3:0/2685579515 >> 172.21.15.3:0/2685579515 conn(0x7f59400a2aa0 msgr2=0x7f59400a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:53.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.243+0000 7f594622a700 1 -- 172.21.15.3:0/2685579515 shutdown_connections 2023-10-03T10:14:53.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.243+0000 7f594622a700 1 -- 172.21.15.3:0/2685579515 wait complete. 2023-10-03T10:14:53.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.244+0000 7f594622a700 1 Processor -- start 2023-10-03T10:14:53.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.244+0000 7f594622a700 1 -- start start 2023-10-03T10:14:53.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.245+0000 7f594622a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f594012cf60 0x7f5940099d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:53.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.245+0000 7f594622a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f594012f8b0 0x7f594009a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:53.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.245+0000 7f594622a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5940130260 0x7f594009a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:53.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.245+0000 7f594622a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59401202f0 con 0x7f5940130260 2023-10-03T10:14:53.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.245+0000 7f594622a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594011fde0 con 0x7f594012f8b0 2023-10-03T10:14:53.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.245+0000 7f594622a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f594011fa80 con 0x7f594012cf60 2023-10-03T10:14:53.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.245+0000 7f593f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f594012cf60 0x7f5940099d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:53.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.245+0000 7f593f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f594012cf60 0x7f5940099d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55880/0 (socket says 172.21.15.3:55880) 2023-10-03T10:14:53.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.245+0000 7f593f7fe700 1 -- 172.21.15.3:0/1566784524 learned_addr learned my addr 172.21.15.3:0/1566784524 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:53.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.245+0000 7f593ffff700 1 --2- 172.21.15.3:0/1566784524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5940130260 0x7f594009a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:53.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.245+0000 7f593effd700 1 --2- 172.21.15.3:0/1566784524 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f594012f8b0 0x7f594009a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:53.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.245+0000 7f593ffff700 1 -- 172.21.15.3:0/1566784524 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f594012cf60 msgr2=0x7f5940099d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:53.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.245+0000 7f593ffff700 1 --2- 172.21.15.3:0/1566784524 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f594012cf60 0x7f5940099d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.245+0000 7f593ffff700 1 -- 172.21.15.3:0/1566784524 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f594012f8b0 msgr2=0x7f594009a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:53.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.245+0000 7f593ffff700 1 --2- 172.21.15.3:0/1566784524 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f594012f8b0 0x7f594009a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.245+0000 7f593ffff700 1 -- 172.21.15.3:0/1566784524 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5930009550 con 0x7f5940130260 2023-10-03T10:14:53.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.246+0000 7f593ffff700 1 --2- 172.21.15.3:0/1566784524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5940130260 0x7f594009a7d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f593800be20 tx=0x7f5938009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:53.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.246+0000 7f5927fff700 1 -- 172.21.15.3:0/1566784524 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593800cb90 con 0x7f5940130260 2023-10-03T10:14:53.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.246+0000 7f5927fff700 1 -- 172.21.15.3:0/1566784524 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593800cd10 con 0x7f5940130260 2023-10-03T10:14:53.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.246+0000 7f594622a700 1 -- 172.21.15.3:0/1566784524 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f594012f520 con 0x7f5940130260 2023-10-03T10:14:53.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.246+0000 7f5927fff700 1 -- 172.21.15.3:0/1566784524 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5938014680 con 0x7f5940130260 2023-10-03T10:14:53.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.246+0000 7f594622a700 1 -- 172.21.15.3:0/1566784524 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5940129b60 con 0x7f5940130260 2023-10-03T10:14:53.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.247+0000 7f594622a700 1 -- 172.21.15.3:0/1566784524 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5940074ea0 con 0x7f5940130260 2023-10-03T10:14:53.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.248+0000 7f5927fff700 1 -- 172.21.15.3:0/1566784524 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f593801d3b0 con 0x7f5940130260 2023-10-03T10:14:53.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.248+0000 7f5927fff700 1 -- 172.21.15.3:0/1566784524 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5938006e40 con 0x7f5940130260 2023-10-03T10:14:53.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.250+0000 7f2ae3f05700 1 Processor -- start 2023-10-03T10:14:53.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.250+0000 7f2ae3f05700 1 -- start start 2023-10-03T10:14:53.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.251+0000 7f2ae3f05700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2adc1305f0 0x7f2adc1309c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:53.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.251+0000 7f2ae3f05700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2adc070190 0x7f2adc138540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:53.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.251+0000 7f2ae3f05700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2adc1301c0 con 0x7f2adc06f660 2023-10-03T10:14:53.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.251+0000 7f2ae3f05700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2adc09a2e0 con 0x7f2adc1305f0 2023-10-03T10:14:53.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.251+0000 7f2ae3f05700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc071ff0 con 0x7f2adc070190 2023-10-03T10:14:53.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.251+0000 7f2ae14a0700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2adc06f660 0x7f2adc06fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48204/0 (socket says 172.21.15.3:48204) 2023-10-03T10:14:53.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.251+0000 7f2ae14a0700 1 -- 172.21.15.3:0/943541394 learned_addr learned my addr 172.21.15.3:0/943541394 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:53.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.252+0000 7f2ae24a2700 1 --2- 172.21.15.3:0/943541394 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2adc070190 0x7f2adc138540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:53.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.252+0000 7f2ae1ca1700 1 --2- 172.21.15.3:0/943541394 >> v2:172.21.15.3:3300/0 conn(0x7f2adc1305f0 0x7f2adc1309c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:53.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.252+0000 7f2ae24a2700 1 -- 172.21.15.3:0/943541394 >> v2:172.21.15.3:3300/0 conn(0x7f2adc1305f0 msgr2=0x7f2adc1309c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:53.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.252+0000 7f2ae24a2700 1 --2- 172.21.15.3:0/943541394 >> v2:172.21.15.3:3300/0 conn(0x7f2adc1305f0 0x7f2adc1309c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.252+0000 7f2ae24a2700 1 -- 172.21.15.3:0/943541394 >> v1:172.21.15.3:6789/0 conn(0x7f2adc06f660 legacy=0x7f2adc06fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:53.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.252+0000 7f2ae24a2700 1 -- 172.21.15.3:0/943541394 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2adc070e70 con 0x7f2adc070190 2023-10-03T10:14:53.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.252+0000 7f2ae24a2700 1 --2- 172.21.15.3:0/943541394 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2adc070190 0x7f2adc138540 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009dc0 tx=0x7f2ad400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b955d0b67c9e838d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:53.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.252+0000 7f2ae0c9f700 1 -- 172.21.15.3:0/943541394 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad400e070 con 0x7f2adc070190 2023-10-03T10:14:53.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.252+0000 7f2ae0c9f700 1 -- 172.21.15.3:0/943541394 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad400bae0 con 0x7f2adc070190 2023-10-03T10:14:53.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.252+0000 7f2ae0c9f700 1 -- 172.21.15.3:0/943541394 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad400be90 con 0x7f2adc070190 2023-10-03T10:14:53.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.253+0000 7f2ae3f05700 1 -- 172.21.15.3:0/943541394 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2adc070190 msgr2=0x7f2adc138540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:53.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.253+0000 7f2ae3f05700 1 --2- 172.21.15.3:0/943541394 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2adc070190 0x7f2adc138540 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009dc0 tx=0x7f2ad400b330 comp rx=0 tx=0).stop 2023-10-03T10:14:53.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.253+0000 7f2ae3f05700 1 -- 172.21.15.3:0/943541394 shutdown_connections 2023-10-03T10:14:53.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.253+0000 7f2ae3f05700 1 --2- 172.21.15.3:0/943541394 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2adc070190 0x7f2adc138540 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.253+0000 7f2ae3f05700 1 --2- 172.21.15.3:0/943541394 >> v2:172.21.15.3:3300/0 conn(0x7f2adc1305f0 0x7f2adc1309c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.253+0000 7f2ae3f05700 1 -- 172.21.15.3:0/943541394 >> 172.21.15.3:0/943541394 conn(0x7f2adc0a2aa0 msgr2=0x7f2adc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:53.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.253+0000 7f2ae3f05700 1 -- 172.21.15.3:0/943541394 shutdown_connections 2023-10-03T10:14:53.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.253+0000 7f5927fff700 1 -- 172.21.15.3:0/1566784524 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f593801c360 con 0x7f5940130260 2023-10-03T10:14:53.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.254+0000 7f2ae3f05700 1 -- 172.21.15.3:0/943541394 wait complete. 2023-10-03T10:14:53.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.255+0000 7f2ae3f05700 1 Processor -- start 2023-10-03T10:14:53.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.255+0000 7f2ae3f05700 1 -- start start 2023-10-03T10:14:53.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.256+0000 7f2ae3f05700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2adc06f660 0x7f2adc13b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:53.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.256+0000 7f2ae3f05700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2adc070190 0x7f2adc133d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:53.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.256+0000 7f2ae3f05700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2adc1305f0 0x7f2adc1342c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:53.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.256+0000 7f2ae3f05700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc136190 con 0x7f2adc070190 2023-10-03T10:14:53.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.256+0000 7f2ae3f05700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc1374e0 con 0x7f2adc06f660 2023-10-03T10:14:53.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.256+0000 7f2ae3f05700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc1359c0 con 0x7f2adc1305f0 2023-10-03T10:14:53.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.256+0000 7f2ae1ca1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2adc06f660 0x7f2adc13b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:53.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.256+0000 7f2ae1ca1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2adc06f660 0x7f2adc13b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42394/0 (socket says 172.21.15.3:42394) 2023-10-03T10:14:53.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.256+0000 7f2ae1ca1700 1 -- 172.21.15.3:0/3235205916 learned_addr learned my addr 172.21.15.3:0/3235205916 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:53.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.256+0000 7f2ae14a0700 1 --2- 172.21.15.3:0/3235205916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2adc070190 0x7f2adc133d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:53.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.256+0000 7f2ae24a2700 1 --2- 172.21.15.3:0/3235205916 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2adc1305f0 0x7f2adc1342c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:53.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.256+0000 7f2ae1ca1700 1 -- 172.21.15.3:0/3235205916 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2adc1305f0 msgr2=0x7f2adc1342c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:53.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.256+0000 7f2ae1ca1700 1 --2- 172.21.15.3:0/3235205916 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2adc1305f0 0x7f2adc1342c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.257+0000 7f2ae1ca1700 1 -- 172.21.15.3:0/3235205916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2adc070190 msgr2=0x7f2adc133d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:53.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.257+0000 7f2ae1ca1700 1 --2- 172.21.15.3:0/3235205916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2adc070190 0x7f2adc133d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.257+0000 7f2ae1ca1700 1 -- 172.21.15.3:0/3235205916 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad4009a70 con 0x7f2adc06f660 2023-10-03T10:14:53.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.257+0000 7f2ae1ca1700 1 --2- 172.21.15.3:0/3235205916 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2adc06f660 0x7f2adc13b090 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2ad800e9b0 tx=0x7f2ad800ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:53.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.257+0000 7f2ad27fc700 1 -- 172.21.15.3:0/3235205916 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8009710 con 0x7f2adc06f660 2023-10-03T10:14:53.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.257+0000 7f2ad27fc700 1 -- 172.21.15.3:0/3235205916 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad800ee50 con 0x7f2adc06f660 2023-10-03T10:14:53.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.257+0000 7f2ae3f05700 1 -- 172.21.15.3:0/3235205916 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2adc072360 con 0x7f2adc06f660 2023-10-03T10:14:53.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.257+0000 7f2ad27fc700 1 -- 172.21.15.3:0/3235205916 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad800ca50 con 0x7f2adc06f660 2023-10-03T10:14:53.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.257+0000 7f2ae3f05700 1 -- 172.21.15.3:0/3235205916 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2adc136b00 con 0x7f2adc06f660 2023-10-03T10:14:53.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.258+0000 7f2ae3f05700 1 -- 172.21.15.3:0/3235205916 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2adc09dd00 con 0x7f2adc06f660 2023-10-03T10:14:53.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.258+0000 7f2ad27fc700 1 -- 172.21.15.3:0/3235205916 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2ad800cbb0 con 0x7f2adc06f660 2023-10-03T10:14:53.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.259+0000 7f2ad27fc700 1 -- 172.21.15.3:0/3235205916 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2ad803f720 con 0x7f2adc06f660 2023-10-03T10:14:53.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.268+0000 7f2ad27fc700 1 -- 172.21.15.3:0/3235205916 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2ad80232f0 con 0x7f2adc06f660 2023-10-03T10:14:53.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.312+0000 7fce32ddd700 1 Processor -- start 2023-10-03T10:14:53.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.312+0000 7fce32ddd700 1 -- start start 2023-10-03T10:14:53.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.313+0000 7fce32ddd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce2c0a0a30 0x7fce2c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:53.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.313+0000 7fce32ddd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce2c09d630 0x7fce2c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:53.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.313+0000 7fce32ddd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce2c0a2a70 con 0x7fce2c133930 2023-10-03T10:14:53.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.313+0000 7fce32ddd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c0a01d0 con 0x7fce2c09d630 2023-10-03T10:14:53.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.313+0000 7fce32ddd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c09e2b0 con 0x7fce2c0a0a30 2023-10-03T10:14:53.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.313+0000 7fce3137a700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fce2c133930 0x7fce2c133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48208/0 (socket says 172.21.15.3:48208) 2023-10-03T10:14:53.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.313+0000 7fce3137a700 1 -- 172.21.15.3:0/340729546 learned_addr learned my addr 172.21.15.3:0/340729546 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:53.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.313+0000 7fce2bfff700 1 --2- 172.21.15.3:0/340729546 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce2c09d630 0x7fce2c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:53.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.313+0000 7fce30b79700 1 --2- 172.21.15.3:0/340729546 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce2c0a0a30 0x7fce2c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:53.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.314+0000 7fce2b7fe700 1 -- 172.21.15.3:0/340729546 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3057791656 0 0) 0x7fce2c0a2a70 con 0x7fce2c133930 2023-10-03T10:14:53.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.314+0000 7fce2b7fe700 1 -- 172.21.15.3:0/340729546 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce1c0035b0 con 0x7fce2c133930 2023-10-03T10:14:53.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.314+0000 7fce30b79700 1 -- 172.21.15.3:0/340729546 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce2c09d630 msgr2=0x7fce2c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:53.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.314+0000 7fce30b79700 1 --2- 172.21.15.3:0/340729546 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce2c09d630 0x7fce2c0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.314+0000 7fce30b79700 1 -- 172.21.15.3:0/340729546 >> v1:172.21.15.3:6789/0 conn(0x7fce2c133930 legacy=0x7fce2c133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:53.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.314+0000 7fce30b79700 1 -- 172.21.15.3:0/340729546 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2c09dc50 con 0x7fce2c0a0a30 2023-10-03T10:14:53.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.314+0000 7fce30b79700 1 --2- 172.21.15.3:0/340729546 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce2c0a0a30 0x7fce2c0a0e00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fce20009870 tx=0x7fce20009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bb12e4a8be9dbde server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:53.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.314+0000 7fce2b7fe700 1 -- 172.21.15.3:0/340729546 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce20011830 con 0x7fce2c0a0a30 2023-10-03T10:14:53.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.314+0000 7fce2b7fe700 1 -- 172.21.15.3:0/340729546 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce20011990 con 0x7fce2c0a0a30 2023-10-03T10:14:53.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.314+0000 7fce2b7fe700 1 -- 172.21.15.3:0/340729546 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce20011d00 con 0x7fce2c0a0a30 2023-10-03T10:14:53.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.315+0000 7fce32ddd700 1 -- 172.21.15.3:0/340729546 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce2c0a0a30 msgr2=0x7fce2c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:53.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.315+0000 7fce32ddd700 1 --2- 172.21.15.3:0/340729546 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce2c0a0a30 0x7fce2c0a0e00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fce20009870 tx=0x7fce20009b80 comp rx=0 tx=0).stop 2023-10-03T10:14:53.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.315+0000 7fce32ddd700 1 -- 172.21.15.3:0/340729546 shutdown_connections 2023-10-03T10:14:53.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.315+0000 7fce32ddd700 1 --2- 172.21.15.3:0/340729546 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce2c0a0a30 0x7fce2c0a0e00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.315+0000 7fce32ddd700 1 --2- 172.21.15.3:0/340729546 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce2c09d630 0x7fce2c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.315+0000 7fce32ddd700 1 -- 172.21.15.3:0/340729546 >> 172.21.15.3:0/340729546 conn(0x7fce2c0a3050 msgr2=0x7fce2c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:53.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.316+0000 7fce32ddd700 1 -- 172.21.15.3:0/340729546 shutdown_connections 2023-10-03T10:14:53.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.316+0000 7fce32ddd700 1 -- 172.21.15.3:0/340729546 wait complete. 2023-10-03T10:14:53.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.317+0000 7fce32ddd700 1 Processor -- start 2023-10-03T10:14:53.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.317+0000 7fce32ddd700 1 -- start start 2023-10-03T10:14:53.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.318+0000 7fce32ddd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce2c09d630 0x7fce2c1c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:53.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.318+0000 7fce32ddd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce2c0a0a30 0x7fce2c1c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:53.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.318+0000 7fce32ddd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce2c133930 0x7fce2c11e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:53.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.318+0000 7fce32ddd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c1c8ed0 con 0x7fce2c09d630 2023-10-03T10:14:53.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.318+0000 7fce32ddd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c0a5360 con 0x7fce2c133930 2023-10-03T10:14:53.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.318+0000 7fce32ddd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c1334d0 con 0x7fce2c0a0a30 2023-10-03T10:14:53.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.318+0000 7fce30b79700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce2c09d630 0x7fce2c1c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:53.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.318+0000 7fce30b79700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce2c09d630 0x7fce2c1c8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40974/0 (socket says 172.21.15.3:40974) 2023-10-03T10:14:53.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.318+0000 7fce30b79700 1 -- 172.21.15.3:0/3358295508 learned_addr learned my addr 172.21.15.3:0/3358295508 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:53.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.318+0000 7fce2bfff700 1 --2- 172.21.15.3:0/3358295508 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce2c0a0a30 0x7fce2c1c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:53.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.318+0000 7fce3137a700 1 --2- 172.21.15.3:0/3358295508 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce2c133930 0x7fce2c11e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:53.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.319+0000 7fce30b79700 1 -- 172.21.15.3:0/3358295508 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce2c0a0a30 msgr2=0x7fce2c1c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:53.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.319+0000 7fce30b79700 1 --2- 172.21.15.3:0/3358295508 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce2c0a0a30 0x7fce2c1c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.319+0000 7fce30b79700 1 -- 172.21.15.3:0/3358295508 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce2c133930 msgr2=0x7fce2c11e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:53.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.319+0000 7fce30b79700 1 --2- 172.21.15.3:0/3358295508 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce2c133930 0x7fce2c11e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.319+0000 7fce30b79700 1 -- 172.21.15.3:0/3358295508 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce200094a0 con 0x7fce2c09d630 2023-10-03T10:14:53.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.319+0000 7fce30b79700 1 --2- 172.21.15.3:0/3358295508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce2c09d630 0x7fce2c1c8450 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fce20009bc0 tx=0x7fce20004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:53.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.319+0000 7fce297fa700 1 -- 172.21.15.3:0/3358295508 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce20004550 con 0x7fce2c09d630 2023-10-03T10:14:53.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.319+0000 7fce297fa700 1 -- 172.21.15.3:0/3358295508 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2001f070 con 0x7fce2c09d630 2023-10-03T10:14:53.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.319+0000 7fce32ddd700 1 -- 172.21.15.3:0/3358295508 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce2c121e90 con 0x7fce2c09d630 2023-10-03T10:14:53.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.319+0000 7fce297fa700 1 -- 172.21.15.3:0/3358295508 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce200038e0 con 0x7fce2c09d630 2023-10-03T10:14:53.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.319+0000 7fce32ddd700 1 -- 172.21.15.3:0/3358295508 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce2c1bcff0 con 0x7fce2c09d630 2023-10-03T10:14:53.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.321+0000 7fce32ddd700 1 -- 172.21.15.3:0/3358295508 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce2c0747f0 con 0x7fce2c09d630 2023-10-03T10:14:53.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.321+0000 7fce297fa700 1 -- 172.21.15.3:0/3358295508 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fce20003a40 con 0x7fce2c09d630 2023-10-03T10:14:53.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.321+0000 7fce297fa700 1 -- 172.21.15.3:0/3358295508 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fce20021070 con 0x7fce2c09d630 2023-10-03T10:14:53.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.326+0000 7fce297fa700 1 -- 172.21.15.3:0/3358295508 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fce20047080 con 0x7fce2c09d630 2023-10-03T10:14:53.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.350+0000 7f691ae06700 1 -- 172.21.15.3:0/470411561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f69140a3570 con 0x7f691409e930 2023-10-03T10:14:53.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.351+0000 7f69117fa700 1 -- 172.21.15.3:0/470411561 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6908015020 con 0x7f691409e930 2023-10-03T10:14:53.351 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.353+0000 7f691ae06700 1 -- 172.21.15.3:0/470411561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f691409e930 msgr2=0x7f69141a3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.353+0000 7f691ae06700 1 --2- 172.21.15.3:0/470411561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f691409e930 0x7f69141a3c70 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f69080090b0 tx=0x7f6908004750 comp rx=0 tx=0).stop 2023-10-03T10:14:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.353+0000 7f691ae06700 1 -- 172.21.15.3:0/470411561 shutdown_connections 2023-10-03T10:14:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.353+0000 7f691ae06700 1 --2- 172.21.15.3:0/470411561 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f691409f650 0x7f69141a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.353+0000 7f691ae06700 1 --2- 172.21.15.3:0/470411561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f691409e930 0x7f69141a3c70 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.353+0000 7f691ae06700 1 --2- 172.21.15.3:0/470411561 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f691409f240 0x7f69141a41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.353+0000 7f691ae06700 1 -- 172.21.15.3:0/470411561 >> 172.21.15.3:0/470411561 conn(0x7f69140a4210 msgr2=0x7f69141b8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.354+0000 7f691ae06700 1 -- 172.21.15.3:0/470411561 shutdown_connections 2023-10-03T10:14:53.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.355+0000 7f691ae06700 1 -- 172.21.15.3:0/470411561 wait complete. 2023-10-03T10:14:53.369 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:53.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.471+0000 7f594622a700 1 -- 172.21.15.3:0/1566784524 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f59400713a0 con 0x7f5940130260 2023-10-03T10:14:53.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.472+0000 7f5927fff700 1 -- 172.21.15.3:0/1566784524 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5938019030 con 0x7f5940130260 2023-10-03T10:14:53.472 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:53.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.474+0000 7f594622a700 1 -- 172.21.15.3:0/1566784524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5940130260 msgr2=0x7f594009a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:53.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.474+0000 7f594622a700 1 --2- 172.21.15.3:0/1566784524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5940130260 0x7f594009a7d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f593800be20 tx=0x7f5938009ed0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.474+0000 7f594622a700 1 -- 172.21.15.3:0/1566784524 shutdown_connections 2023-10-03T10:14:53.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.474+0000 7f594622a700 1 --2- 172.21.15.3:0/1566784524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5940130260 0x7f594009a7d0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.474+0000 7f594622a700 1 --2- 172.21.15.3:0/1566784524 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f594012cf60 0x7f5940099d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.474+0000 7f594622a700 1 --2- 172.21.15.3:0/1566784524 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f594012f8b0 0x7f594009a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.474+0000 7f594622a700 1 -- 172.21.15.3:0/1566784524 >> 172.21.15.3:0/1566784524 conn(0x7f59400a2aa0 msgr2=0x7f594009e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:53.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.474+0000 7f594622a700 1 -- 172.21.15.3:0/1566784524 shutdown_connections 2023-10-03T10:14:53.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.474+0000 7f594622a700 1 -- 172.21.15.3:0/1566784524 wait complete. 2023-10-03T10:14:53.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.481+0000 7f2ae3f05700 1 -- 172.21.15.3:0/3235205916 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2adc074ea0 con 0x7f2adc06f660 2023-10-03T10:14:53.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.482+0000 7f2ad27fc700 1 -- 172.21.15.3:0/3235205916 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2ad801ae80 con 0x7f2adc06f660 2023-10-03T10:14:53.482 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:53.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.484+0000 7f2ae3f05700 1 -- 172.21.15.3:0/3235205916 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2adc06f660 msgr2=0x7f2adc13b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:53.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.484+0000 7f2ae3f05700 1 --2- 172.21.15.3:0/3235205916 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2adc06f660 0x7f2adc13b090 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2ad800e9b0 tx=0x7f2ad800ecc0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.484+0000 7f2ae3f05700 1 -- 172.21.15.3:0/3235205916 shutdown_connections 2023-10-03T10:14:53.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.484+0000 7f2ae3f05700 1 --2- 172.21.15.3:0/3235205916 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2adc1305f0 0x7f2adc1342c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.484+0000 7f2ae3f05700 1 --2- 172.21.15.3:0/3235205916 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2adc06f660 0x7f2adc13b090 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.484+0000 7f2ae3f05700 1 --2- 172.21.15.3:0/3235205916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2adc070190 0x7f2adc133d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.484+0000 7f2ae3f05700 1 -- 172.21.15.3:0/3235205916 >> 172.21.15.3:0/3235205916 conn(0x7f2adc0a2aa0 msgr2=0x7f2adc0a1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:53.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.484+0000 7f2ae3f05700 1 -- 172.21.15.3:0/3235205916 shutdown_connections 2023-10-03T10:14:53.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.484+0000 7f2ae3f05700 1 -- 172.21.15.3:0/3235205916 wait complete. 2023-10-03T10:14:53.489 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:53.499 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:53.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.536+0000 7fce32ddd700 1 -- 172.21.15.3:0/3358295508 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fce2c072980 con 0x7fce2c09d630 2023-10-03T10:14:53.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.537+0000 7fce297fa700 1 -- 172.21.15.3:0/3358295508 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fce20015070 con 0x7fce2c09d630 2023-10-03T10:14:53.537 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:53.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.539+0000 7fce32ddd700 1 -- 172.21.15.3:0/3358295508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce2c09d630 msgr2=0x7fce2c1c8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:53.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.539+0000 7fce32ddd700 1 --2- 172.21.15.3:0/3358295508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce2c09d630 0x7fce2c1c8450 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fce20009bc0 tx=0x7fce20004750 comp rx=0 tx=0).stop 2023-10-03T10:14:53.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.539+0000 7fce32ddd700 1 -- 172.21.15.3:0/3358295508 shutdown_connections 2023-10-03T10:14:53.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.539+0000 7fce32ddd700 1 --2- 172.21.15.3:0/3358295508 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce2c133930 0x7fce2c11e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.539+0000 7fce32ddd700 1 --2- 172.21.15.3:0/3358295508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce2c09d630 0x7fce2c1c8450 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.539+0000 7fce32ddd700 1 --2- 172.21.15.3:0/3358295508 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce2c0a0a30 0x7fce2c1c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:53.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.539+0000 7fce32ddd700 1 -- 172.21.15.3:0/3358295508 >> 172.21.15.3:0/3358295508 conn(0x7fce2c0a3050 msgr2=0x7fce2c1bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:53.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.539+0000 7fce32ddd700 1 -- 172.21.15.3:0/3358295508 shutdown_connections 2023-10-03T10:14:53.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:53.539+0000 7fce32ddd700 1 -- 172.21.15.3:0/3358295508 wait complete. 2023-10-03T10:14:53.553 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:54.369 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:54.490 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:54.499 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:54.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.521+0000 7fd6c4f5d700 1 Processor -- start 2023-10-03T10:14:54.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.521+0000 7fd6c4f5d700 1 -- start start 2023-10-03T10:14:54.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.522+0000 7fd6c4f5d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6c01338a0 0x7fd6c0133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:54.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.522+0000 7fd6c4f5d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd6c009e930 0x7fd6c013b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:54.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.522+0000 7fd6c4f5d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c006ef30 con 0x7fd6c009de50 2023-10-03T10:14:54.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.522+0000 7fd6c4f5d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6c006fef0 con 0x7fd6c009e930 2023-10-03T10:14:54.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.522+0000 7fd6c4f5d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c01334b0 con 0x7fd6c01338a0 2023-10-03T10:14:54.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.522+0000 7fd6be59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6c01338a0 0x7fd6c0133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:54.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.522+0000 7fd6be59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6c01338a0 0x7fd6c0133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42426/0 (socket says 172.21.15.3:42426) 2023-10-03T10:14:54.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.522+0000 7fd6bed9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd6c009e930 0x7fd6c013b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:54.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.522+0000 7fd6be59c700 1 -- 172.21.15.3:0/2619262955 learned_addr learned my addr 172.21.15.3:0/2619262955 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:54.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.522+0000 7fd6be59c700 1 -- 172.21.15.3:0/2619262955 >> v2:172.21.15.3:3300/0 conn(0x7fd6c009e930 msgr2=0x7fd6c013b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:54.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.522+0000 7fd6be59c700 1 --2- 172.21.15.3:0/2619262955 >> v2:172.21.15.3:3300/0 conn(0x7fd6c009e930 0x7fd6c013b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.522+0000 7fd6be59c700 1 -- 172.21.15.3:0/2619262955 >> v1:172.21.15.3:6789/0 conn(0x7fd6c009de50 legacy=0x7fd6c009e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:54.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.522+0000 7fd6be59c700 1 -- 172.21.15.3:0/2619262955 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c006fc30 con 0x7fd6c01338a0 2023-10-03T10:14:54.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.522+0000 7fd6bed9d700 1 --2- 172.21.15.3:0/2619262955 >> v2:172.21.15.3:3300/0 conn(0x7fd6c009e930 0x7fd6c013b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:54.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.523+0000 7fd6be59c700 1 --2- 172.21.15.3:0/2619262955 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6c01338a0 0x7fd6c0133c70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00098f0 tx=0x7fd6b0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c17248c5665cac3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:54.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.523+0000 7fd6bd59a700 1 -- 172.21.15.3:0/2619262955 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0011830 con 0x7fd6c01338a0 2023-10-03T10:14:54.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.523+0000 7fd6bd59a700 1 -- 172.21.15.3:0/2619262955 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b0011990 con 0x7fd6c01338a0 2023-10-03T10:14:54.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.523+0000 7fd6bd59a700 1 -- 172.21.15.3:0/2619262955 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0011d20 con 0x7fd6c01338a0 2023-10-03T10:14:54.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.524+0000 7fd6c4f5d700 1 -- 172.21.15.3:0/2619262955 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6c01338a0 msgr2=0x7fd6c0133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:54.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.524+0000 7fd6c4f5d700 1 --2- 172.21.15.3:0/2619262955 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6c01338a0 0x7fd6c0133c70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00098f0 tx=0x7fd6b0009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:54.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.524+0000 7fd6c4f5d700 1 -- 172.21.15.3:0/2619262955 shutdown_connections 2023-10-03T10:14:54.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.524+0000 7fd6c4f5d700 1 --2- 172.21.15.3:0/2619262955 >> v2:172.21.15.3:3300/0 conn(0x7fd6c009e930 0x7fd6c013b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.524+0000 7fd6c4f5d700 1 --2- 172.21.15.3:0/2619262955 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6c01338a0 0x7fd6c0133c70 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.524+0000 7fd6c4f5d700 1 -- 172.21.15.3:0/2619262955 >> 172.21.15.3:0/2619262955 conn(0x7fd6c00a2aa0 msgr2=0x7fd6c00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:54.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.524+0000 7fd6c4f5d700 1 -- 172.21.15.3:0/2619262955 shutdown_connections 2023-10-03T10:14:54.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.524+0000 7fd6c4f5d700 1 -- 172.21.15.3:0/2619262955 wait complete. 2023-10-03T10:14:54.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.525+0000 7fd6c4f5d700 1 Processor -- start 2023-10-03T10:14:54.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.525+0000 7fd6c4f5d700 1 -- start start 2023-10-03T10:14:54.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.525+0000 7fd6c4f5d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6c009de50 0x7fd6c011d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:54.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.526+0000 7fd6c4f5d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6c009e930 0x7fd6c011dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:54.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.526+0000 7fd6c4f5d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6c01338a0 0x7fd6c011e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:54.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.526+0000 7fd6c4f5d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c011f350 con 0x7fd6c009e930 2023-10-03T10:14:54.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.526+0000 7fd6c4f5d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c006f640 con 0x7fd6c01338a0 2023-10-03T10:14:54.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.526+0000 7fd6c4f5d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c0120f80 con 0x7fd6c009de50 2023-10-03T10:14:54.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.526+0000 7fd6be59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6c009de50 0x7fd6c011d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:54.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.526+0000 7fd6be59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6c009de50 0x7fd6c011d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55916/0 (socket says 172.21.15.3:55916) 2023-10-03T10:14:54.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.526+0000 7fd6be59c700 1 -- 172.21.15.3:0/3778441337 learned_addr learned my addr 172.21.15.3:0/3778441337 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:54.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.526+0000 7fd6bdd9b700 1 --2- 172.21.15.3:0/3778441337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6c009e930 0x7fd6c011dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:54.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.526+0000 7fd6bed9d700 1 --2- 172.21.15.3:0/3778441337 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6c01338a0 0x7fd6c011e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:54.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.526+0000 7fd6bed9d700 1 -- 172.21.15.3:0/3778441337 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6c009de50 msgr2=0x7fd6c011d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:54.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.526+0000 7fd6bed9d700 1 --2- 172.21.15.3:0/3778441337 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6c009de50 0x7fd6c011d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.526+0000 7fd6bed9d700 1 -- 172.21.15.3:0/3778441337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6c009e930 msgr2=0x7fd6c011dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:54.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.526+0000 7fd6bed9d700 1 --2- 172.21.15.3:0/3778441337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6c009e930 0x7fd6c011dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.526+0000 7fd6bed9d700 1 -- 172.21.15.3:0/3778441337 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b8009a70 con 0x7fd6c01338a0 2023-10-03T10:14:54.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.526+0000 7fd6bdd9b700 1 --2- 172.21.15.3:0/3778441337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6c009e930 0x7fd6c011dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:54.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.526+0000 7fd6be59c700 1 --2- 172.21.15.3:0/3778441337 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6c009de50 0x7fd6c011d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:54.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.527+0000 7fd6bed9d700 1 --2- 172.21.15.3:0/3778441337 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6c01338a0 0x7fd6c011e2a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800efc0 tx=0x7fd6b800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:54.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.527+0000 7fd6aeffd700 1 -- 172.21.15.3:0/3778441337 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b800ca90 con 0x7fd6c01338a0 2023-10-03T10:14:54.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.527+0000 7fd6aeffd700 1 -- 172.21.15.3:0/3778441337 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b800f040 con 0x7fd6c01338a0 2023-10-03T10:14:54.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.527+0000 7fd6c4f5d700 1 -- 172.21.15.3:0/3778441337 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b0009550 con 0x7fd6c01338a0 2023-10-03T10:14:54.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.527+0000 7fd6aeffd700 1 -- 172.21.15.3:0/3778441337 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b800ce30 con 0x7fd6c01338a0 2023-10-03T10:14:54.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.527+0000 7fd6c4f5d700 1 -- 172.21.15.3:0/3778441337 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c01205a0 con 0x7fd6c01338a0 2023-10-03T10:14:54.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.528+0000 7fd6aeffd700 1 -- 172.21.15.3:0/3778441337 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd6b8014480 con 0x7fd6c01338a0 2023-10-03T10:14:54.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.528+0000 7fd6aeffd700 1 -- 172.21.15.3:0/3778441337 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd6b803fc40 con 0x7fd6c01338a0 2023-10-03T10:14:54.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.528+0000 7fd6c4f5d700 1 -- 172.21.15.3:0/3778441337 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c011a470 con 0x7fd6c01338a0 2023-10-03T10:14:54.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.534+0000 7fd6aeffd700 1 -- 172.21.15.3:0/3778441337 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd6b8012930 con 0x7fd6c01338a0 2023-10-03T10:14:54.554 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:54.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.639+0000 7f629dfae700 1 Processor -- start 2023-10-03T10:14:54.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.639+0000 7f629dfae700 1 -- start start 2023-10-03T10:14:54.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.640+0000 7f629dfae700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f629806a770 0x7f629809d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:54.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.640+0000 7f629dfae700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f629809dc30 0x7f629809e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:54.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.640+0000 7f629dfae700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f629813c1e0 0x7f629813e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:54.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.640+0000 7f629dfae700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f629806ab40 con 0x7f629809dc30 2023-10-03T10:14:54.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.640+0000 7f629dfae700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f629809ebc0 con 0x7f629813c1e0 2023-10-03T10:14:54.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.640+0000 7f629dfae700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f629809a810 con 0x7f629806a770 2023-10-03T10:14:54.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.640+0000 7f6297fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f629813c1e0 0x7f629813e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:54.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.640+0000 7f6296ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f629809dc30 0x7f629809e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:54.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.640+0000 7f6297fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f629813c1e0 0x7f629813e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42432/0 (socket says 172.21.15.3:42432) 2023-10-03T10:14:54.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.640+0000 7f6297fff700 1 -- 172.21.15.3:0/1887820873 learned_addr learned my addr 172.21.15.3:0/1887820873 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:54.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.640+0000 7f62977fe700 1 --2- 172.21.15.3:0/1887820873 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f629806a770 0x7f629809d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:54.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.640+0000 7f6297fff700 1 -- 172.21.15.3:0/1887820873 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f629806a770 msgr2=0x7f629809d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:54.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.640+0000 7f6297fff700 1 --2- 172.21.15.3:0/1887820873 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f629806a770 0x7f629809d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.640+0000 7f6297fff700 1 -- 172.21.15.3:0/1887820873 >> v2:172.21.15.3:3300/0 conn(0x7f629809dc30 msgr2=0x7f629809e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:54.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.640+0000 7f6297fff700 1 --2- 172.21.15.3:0/1887820873 >> v2:172.21.15.3:3300/0 conn(0x7f629809dc30 0x7f629809e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.640+0000 7f6297fff700 1 -- 172.21.15.3:0/1887820873 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629809e920 con 0x7f629813c1e0 2023-10-03T10:14:54.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.641+0000 7f6297fff700 1 --2- 172.21.15.3:0/1887820873 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f629813c1e0 0x7f629813e5c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6290009e50 tx=0x7f629000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4722be3cd765428d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:54.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.641+0000 7f62967fc700 1 -- 172.21.15.3:0/1887820873 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629000e070 con 0x7f629813c1e0 2023-10-03T10:14:54.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.641+0000 7f62967fc700 1 -- 172.21.15.3:0/1887820873 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629000b910 con 0x7f629813c1e0 2023-10-03T10:14:54.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.641+0000 7f62967fc700 1 -- 172.21.15.3:0/1887820873 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629000bcc0 con 0x7f629813c1e0 2023-10-03T10:14:54.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.642+0000 7f629dfae700 1 -- 172.21.15.3:0/1887820873 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f629813c1e0 msgr2=0x7f629813e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:54.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.642+0000 7f629dfae700 1 --2- 172.21.15.3:0/1887820873 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f629813c1e0 0x7f629813e5c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6290009e50 tx=0x7f629000b330 comp rx=0 tx=0).stop 2023-10-03T10:14:54.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.642+0000 7f629dfae700 1 -- 172.21.15.3:0/1887820873 shutdown_connections 2023-10-03T10:14:54.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.642+0000 7f629dfae700 1 --2- 172.21.15.3:0/1887820873 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f629813c1e0 0x7f629813e5c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.642+0000 7f629dfae700 1 --2- 172.21.15.3:0/1887820873 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f629806a770 0x7f629809d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.642+0000 7f629dfae700 1 --2- 172.21.15.3:0/1887820873 >> v2:172.21.15.3:3300/0 conn(0x7f629809dc30 0x7f629809e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.642+0000 7f629dfae700 1 -- 172.21.15.3:0/1887820873 >> 172.21.15.3:0/1887820873 conn(0x7f6298066f50 msgr2=0x7f629809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:54.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.642+0000 7f629dfae700 1 -- 172.21.15.3:0/1887820873 shutdown_connections 2023-10-03T10:14:54.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.642+0000 7f629dfae700 1 -- 172.21.15.3:0/1887820873 wait complete. 2023-10-03T10:14:54.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.644+0000 7f629dfae700 1 Processor -- start 2023-10-03T10:14:54.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.644+0000 7f629dfae700 1 -- start start 2023-10-03T10:14:54.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.645+0000 7f629dfae700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f629806a770 0x7f62981205c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:54.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.645+0000 7f629dfae700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f629809dc30 0x7f629813a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:54.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.645+0000 7f629dfae700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f629813c1e0 0x7f629813abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:54.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.645+0000 7f629dfae700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629811fe70 con 0x7f629813c1e0 2023-10-03T10:14:54.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.645+0000 7f629dfae700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f629813bae0 con 0x7f629806a770 2023-10-03T10:14:54.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.645+0000 7f629dfae700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f629809f1a0 con 0x7f629809dc30 2023-10-03T10:14:54.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.645+0000 7f6296ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f629809dc30 0x7f629813a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:54.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.645+0000 7f6296ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f629809dc30 0x7f629813a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55938/0 (socket says 172.21.15.3:55938) 2023-10-03T10:14:54.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.645+0000 7f6296ffd700 1 -- 172.21.15.3:0/1456026911 learned_addr learned my addr 172.21.15.3:0/1456026911 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:54.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.645+0000 7f62977fe700 1 --2- 172.21.15.3:0/1456026911 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f629806a770 0x7f62981205c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:54.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.645+0000 7f6296ffd700 1 -- 172.21.15.3:0/1456026911 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f629806a770 msgr2=0x7f62981205c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:54.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.645+0000 7f6296ffd700 1 --2- 172.21.15.3:0/1456026911 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f629806a770 0x7f62981205c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.645+0000 7f6296ffd700 1 -- 172.21.15.3:0/1456026911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f629813c1e0 msgr2=0x7f629813abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:54.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.645+0000 7f6296ffd700 1 --2- 172.21.15.3:0/1456026911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f629813c1e0 0x7f629813abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.645+0000 7f6296ffd700 1 -- 172.21.15.3:0/1456026911 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6290009b30 con 0x7f629809dc30 2023-10-03T10:14:54.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.646+0000 7f6296ffd700 1 --2- 172.21.15.3:0/1456026911 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f629809dc30 0x7f629813a6a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f628c00e9b0 tx=0x7f628c00ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:54.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.646+0000 7f627ffff700 1 -- 172.21.15.3:0/1456026911 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628c009710 con 0x7f629809dc30 2023-10-03T10:14:54.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.646+0000 7f627ffff700 1 -- 172.21.15.3:0/1456026911 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628c00c850 con 0x7f629809dc30 2023-10-03T10:14:54.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.646+0000 7f627ffff700 1 -- 172.21.15.3:0/1456026911 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628c00cbe0 con 0x7f629809dc30 2023-10-03T10:14:54.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.646+0000 7f629dfae700 1 -- 172.21.15.3:0/1456026911 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62981200c0 con 0x7f629809dc30 2023-10-03T10:14:54.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.646+0000 7f629dfae700 1 -- 172.21.15.3:0/1456026911 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f629811d030 con 0x7f629809dc30 2023-10-03T10:14:54.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.647+0000 7f627ffff700 1 -- 172.21.15.3:0/1456026911 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f628c00cd40 con 0x7f629809dc30 2023-10-03T10:14:54.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.647+0000 7f629dfae700 1 -- 172.21.15.3:0/1456026911 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6298071ff0 con 0x7f629809dc30 2023-10-03T10:14:54.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.648+0000 7f627ffff700 1 -- 172.21.15.3:0/1456026911 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f628c03f5d0 con 0x7f629809dc30 2023-10-03T10:14:54.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.653+0000 7f627ffff700 1 -- 172.21.15.3:0/1456026911 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f628c013470 con 0x7f629809dc30 2023-10-03T10:14:54.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.668+0000 7fd93e5e8700 1 Processor -- start 2023-10-03T10:14:54.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.668+0000 7fd93e5e8700 1 -- start start 2023-10-03T10:14:54.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.669+0000 7fd93e5e8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd93809f410 0x7fd938136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:54.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.669+0000 7fd93e5e8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd938136fe0 0x7fd9381393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:54.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.669+0000 7fd93e5e8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd938070940 con 0x7fd93809e930 2023-10-03T10:14:54.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.669+0000 7fd93e5e8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd93806fd60 con 0x7fd93809f410 2023-10-03T10:14:54.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.669+0000 7fd93e5e8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd93806fa80 con 0x7fd938136fe0 2023-10-03T10:14:54.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.669+0000 7fd9377fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd93809f410 0x7fd938136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:54.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.669+0000 7fd9377fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd93809f410 0x7fd938136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41014/0 (socket says 172.21.15.3:41014) 2023-10-03T10:14:54.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.669+0000 7fd9377fe700 1 -- 172.21.15.3:0/1871182970 learned_addr learned my addr 172.21.15.3:0/1871182970 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:54.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.669+0000 7fd93cb85700 1 --2- 172.21.15.3:0/1871182970 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd938136fe0 0x7fd9381393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:54.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.669+0000 7fd9377fe700 1 -- 172.21.15.3:0/1871182970 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd938136fe0 msgr2=0x7fd9381393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:54.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.669+0000 7fd9377fe700 1 --2- 172.21.15.3:0/1871182970 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd938136fe0 0x7fd9381393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.669+0000 7fd9377fe700 1 -- 172.21.15.3:0/1871182970 >> v1:172.21.15.3:6789/0 conn(0x7fd93809e930 legacy=0x7fd93809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:54.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.669+0000 7fd9377fe700 1 -- 172.21.15.3:0/1871182970 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93809fb60 con 0x7fd93809f410 2023-10-03T10:14:54.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.670+0000 7fd9377fe700 1 --2- 172.21.15.3:0/1871182970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd93809f410 0x7fd938136aa0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fd92c0098f0 tx=0x7fd92c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31193bae1ad5fff5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:54.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.670+0000 7fd936ffd700 1 -- 172.21.15.3:0/1871182970 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c011830 con 0x7fd93809f410 2023-10-03T10:14:54.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.670+0000 7fd936ffd700 1 -- 172.21.15.3:0/1871182970 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92c011990 con 0x7fd93809f410 2023-10-03T10:14:54.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.670+0000 7fd936ffd700 1 -- 172.21.15.3:0/1871182970 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c011d20 con 0x7fd93809f410 2023-10-03T10:14:54.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.671+0000 7fd93e5e8700 1 -- 172.21.15.3:0/1871182970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd93809f410 msgr2=0x7fd938136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:54.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.671+0000 7fd93e5e8700 1 --2- 172.21.15.3:0/1871182970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd93809f410 0x7fd938136aa0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fd92c0098f0 tx=0x7fd92c009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:54.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.671+0000 7fd93e5e8700 1 -- 172.21.15.3:0/1871182970 shutdown_connections 2023-10-03T10:14:54.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.671+0000 7fd93e5e8700 1 --2- 172.21.15.3:0/1871182970 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd938136fe0 0x7fd9381393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.671+0000 7fd93e5e8700 1 --2- 172.21.15.3:0/1871182970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd93809f410 0x7fd938136aa0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.671+0000 7fd93e5e8700 1 -- 172.21.15.3:0/1871182970 >> 172.21.15.3:0/1871182970 conn(0x7fd9380a4210 msgr2=0x7fd9380a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:54.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.671+0000 7fd93e5e8700 1 -- 172.21.15.3:0/1871182970 shutdown_connections 2023-10-03T10:14:54.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.671+0000 7fd93e5e8700 1 -- 172.21.15.3:0/1871182970 wait complete. 2023-10-03T10:14:54.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.672+0000 7fd93e5e8700 1 Processor -- start 2023-10-03T10:14:54.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.672+0000 7fd93e5e8700 1 -- start start 2023-10-03T10:14:54.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.673+0000 7fd93e5e8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd93809e930 0x7fd93811f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:54.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.673+0000 7fd93e5e8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd93809f410 0x7fd93811fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:54.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.673+0000 7fd93e5e8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd938136fe0 0x7fd938120400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:54.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.673+0000 7fd93e5e8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd938071d20 con 0x7fd938136fe0 2023-10-03T10:14:54.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.673+0000 7fd93e5e8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9381216a0 con 0x7fd93809f410 2023-10-03T10:14:54.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.673+0000 7fd93e5e8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9381231c0 con 0x7fd93809e930 2023-10-03T10:14:54.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.673+0000 7fd937fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd93809e930 0x7fd93811f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:54.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.673+0000 7fd93cb85700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd938136fe0 0x7fd938120400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:54.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.673+0000 7fd9377fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd93809f410 0x7fd93811fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:54.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.673+0000 7fd937fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd93809e930 0x7fd93811f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55950/0 (socket says 172.21.15.3:55950) 2023-10-03T10:14:54.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.673+0000 7fd937fff700 1 -- 172.21.15.3:0/3052644895 learned_addr learned my addr 172.21.15.3:0/3052644895 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:54.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.674+0000 7fd937fff700 1 -- 172.21.15.3:0/3052644895 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd93809f410 msgr2=0x7fd93811fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:54.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.674+0000 7fd937fff700 1 --2- 172.21.15.3:0/3052644895 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd93809f410 0x7fd93811fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.674+0000 7fd937fff700 1 -- 172.21.15.3:0/3052644895 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd938136fe0 msgr2=0x7fd938120400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:54.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.674+0000 7fd937fff700 1 --2- 172.21.15.3:0/3052644895 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd938136fe0 0x7fd938120400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.674+0000 7fd937fff700 1 -- 172.21.15.3:0/3052644895 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92c009550 con 0x7fd93809e930 2023-10-03T10:14:54.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.674+0000 7fd9377fe700 1 --2- 172.21.15.3:0/3052644895 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd93809f410 0x7fd93811fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:54.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.674+0000 7fd937fff700 1 --2- 172.21.15.3:0/3052644895 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd93809e930 0x7fd93811f980 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd92800c510 tx=0x7fd92800c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:54.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.674+0000 7fd934ff9700 1 -- 172.21.15.3:0/3052644895 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9280052d0 con 0x7fd93809e930 2023-10-03T10:14:54.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.674+0000 7fd934ff9700 1 -- 172.21.15.3:0/3052644895 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd928005430 con 0x7fd93809e930 2023-10-03T10:14:54.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.674+0000 7fd93e5e8700 1 -- 172.21.15.3:0/3052644895 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9380a2f40 con 0x7fd93809e930 2023-10-03T10:14:54.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.674+0000 7fd934ff9700 1 -- 172.21.15.3:0/3052644895 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9280038c0 con 0x7fd93809e930 2023-10-03T10:14:54.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.675+0000 7fd93e5e8700 1 -- 172.21.15.3:0/3052644895 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9381234a0 con 0x7fd93809e930 2023-10-03T10:14:54.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.675+0000 7fd934ff9700 1 -- 172.21.15.3:0/3052644895 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd928003a20 con 0x7fd93809e930 2023-10-03T10:14:54.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.675+0000 7fd934ff9700 1 -- 172.21.15.3:0/3052644895 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd928003cd0 con 0x7fd93809e930 2023-10-03T10:14:54.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.676+0000 7fd93e5e8700 1 -- 172.21.15.3:0/3052644895 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd938074ea0 con 0x7fd93809e930 2023-10-03T10:14:54.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.681+0000 7fd934ff9700 1 -- 172.21.15.3:0/3052644895 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd928044060 con 0x7fd93809e930 2023-10-03T10:14:54.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.706+0000 7fca956bf700 1 Processor -- start 2023-10-03T10:14:54.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.706+0000 7fca956bf700 1 -- start start 2023-10-03T10:14:54.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.706+0000 7fca956bf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca9006a770 0x7fca9006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:54.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.706+0000 7fca956bf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fca9006b080 0x7fca90131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:54.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.706+0000 7fca956bf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca90131b40 0x7fca90131fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:54.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.706+0000 7fca956bf700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca9009ece0 con 0x7fca9006b080 2023-10-03T10:14:54.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.706+0000 7fca956bf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9009e760 con 0x7fca9006a770 2023-10-03T10:14:54.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.706+0000 7fca956bf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca9009e040 con 0x7fca90131b40 2023-10-03T10:14:54.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.707+0000 7fca8f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca90131b40 0x7fca90131fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:54.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.707+0000 7fca8f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca90131b40 0x7fca90131fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55964/0 (socket says 172.21.15.3:55964) 2023-10-03T10:14:54.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.707+0000 7fca8f7fe700 1 -- 172.21.15.3:0/4280414128 learned_addr learned my addr 172.21.15.3:0/4280414128 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:54.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.707+0000 7fca8e7fc700 1 --2- 172.21.15.3:0/4280414128 >> v2:172.21.15.3:3300/0 conn(0x7fca9006b080 0x7fca90131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:54.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.707+0000 7fca8effd700 1 --2- 172.21.15.3:0/4280414128 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca9006a770 0x7fca9006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:54.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.707+0000 7fca8effd700 1 -- 172.21.15.3:0/4280414128 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca90131b40 msgr2=0x7fca90131fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:54.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.707+0000 7fca8effd700 1 --2- 172.21.15.3:0/4280414128 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca90131b40 0x7fca90131fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.707+0000 7fca8effd700 1 -- 172.21.15.3:0/4280414128 >> v2:172.21.15.3:3300/0 conn(0x7fca9006b080 msgr2=0x7fca90131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:54.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.707+0000 7fca8effd700 1 --2- 172.21.15.3:0/4280414128 >> v2:172.21.15.3:3300/0 conn(0x7fca9006b080 0x7fca90131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.707+0000 7fca8effd700 1 -- 172.21.15.3:0/4280414128 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9009f630 con 0x7fca9006a770 2023-10-03T10:14:54.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.707+0000 7fca8e7fc700 1 --2- 172.21.15.3:0/4280414128 >> v2:172.21.15.3:3300/0 conn(0x7fca9006b080 0x7fca90131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:54.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.707+0000 7fca8effd700 1 --2- 172.21.15.3:0/4280414128 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca9006a770 0x7fca9006ab40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fca80009840 tx=0x7fca80009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92d784073ade62d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:54.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.708+0000 7fca8dffb700 1 -- 172.21.15.3:0/4280414128 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca80011830 con 0x7fca9006a770 2023-10-03T10:14:54.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.708+0000 7fca8dffb700 1 -- 172.21.15.3:0/4280414128 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca80011990 con 0x7fca9006a770 2023-10-03T10:14:54.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.708+0000 7fca8dffb700 1 -- 172.21.15.3:0/4280414128 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca80011d20 con 0x7fca9006a770 2023-10-03T10:14:54.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.708+0000 7fca956bf700 1 -- 172.21.15.3:0/4280414128 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca9006a770 msgr2=0x7fca9006ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:54.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.708+0000 7fca956bf700 1 --2- 172.21.15.3:0/4280414128 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca9006a770 0x7fca9006ab40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fca80009840 tx=0x7fca80009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:54.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.708+0000 7fca956bf700 1 -- 172.21.15.3:0/4280414128 shutdown_connections 2023-10-03T10:14:54.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.708+0000 7fca956bf700 1 --2- 172.21.15.3:0/4280414128 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca90131b40 0x7fca90131fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.708+0000 7fca956bf700 1 --2- 172.21.15.3:0/4280414128 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca9006a770 0x7fca9006ab40 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.708+0000 7fca956bf700 1 --2- 172.21.15.3:0/4280414128 >> v2:172.21.15.3:3300/0 conn(0x7fca9006b080 0x7fca90131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.708+0000 7fca956bf700 1 -- 172.21.15.3:0/4280414128 >> 172.21.15.3:0/4280414128 conn(0x7fca90066f50 msgr2=0x7fca9009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:54.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.709+0000 7fca956bf700 1 -- 172.21.15.3:0/4280414128 shutdown_connections 2023-10-03T10:14:54.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.709+0000 7fca956bf700 1 -- 172.21.15.3:0/4280414128 wait complete. 2023-10-03T10:14:54.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.710+0000 7fca956bf700 1 Processor -- start 2023-10-03T10:14:54.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.710+0000 7fca956bf700 1 -- start start 2023-10-03T10:14:54.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.711+0000 7fca956bf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca9006a770 0x7fca90120150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:54.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.711+0000 7fca956bf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca9006b080 0x7fca90120690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:54.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.711+0000 7fca956bf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca90131b40 0x7fca90120bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:54.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.711+0000 7fca956bf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca90123a20 con 0x7fca9006b080 2023-10-03T10:14:54.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.711+0000 7fca956bf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca90121d60 con 0x7fca90131b40 2023-10-03T10:14:54.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.711+0000 7fca956bf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca90123c70 con 0x7fca9006a770 2023-10-03T10:14:54.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.711+0000 7fca8e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca9006b080 0x7fca90120690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:54.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.711+0000 7fca8f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca90131b40 0x7fca90120bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:54.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.711+0000 7fca8e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca9006b080 0x7fca90120690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41040/0 (socket says 172.21.15.3:41040) 2023-10-03T10:14:54.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.711+0000 7fca8e7fc700 1 -- 172.21.15.3:0/4255308363 learned_addr learned my addr 172.21.15.3:0/4255308363 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:54.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.711+0000 7fca8effd700 1 --2- 172.21.15.3:0/4255308363 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca9006a770 0x7fca90120150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:54.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.712+0000 7fca8e7fc700 1 -- 172.21.15.3:0/4255308363 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca9006a770 msgr2=0x7fca90120150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:54.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.712+0000 7fca8e7fc700 1 --2- 172.21.15.3:0/4255308363 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca9006a770 0x7fca90120150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.712+0000 7fca8e7fc700 1 -- 172.21.15.3:0/4255308363 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca90131b40 msgr2=0x7fca90120bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:54.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.712+0000 7fca8e7fc700 1 --2- 172.21.15.3:0/4255308363 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca90131b40 0x7fca90120bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.712+0000 7fca8e7fc700 1 -- 172.21.15.3:0/4255308363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca840094a0 con 0x7fca9006b080 2023-10-03T10:14:54.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.712+0000 7fca8e7fc700 1 --2- 172.21.15.3:0/4255308363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca9006b080 0x7fca90120690 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fca8400ebb0 tx=0x7fca8400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:54.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.712+0000 7fca777fe700 1 -- 172.21.15.3:0/4255308363 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8400c850 con 0x7fca9006b080 2023-10-03T10:14:54.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.712+0000 7fca777fe700 1 -- 172.21.15.3:0/4255308363 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8400c9b0 con 0x7fca9006b080 2023-10-03T10:14:54.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.712+0000 7fca956bf700 1 -- 172.21.15.3:0/4255308363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca800094a0 con 0x7fca9006b080 2023-10-03T10:14:54.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.712+0000 7fca956bf700 1 -- 172.21.15.3:0/4255308363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca90130c70 con 0x7fca9006b080 2023-10-03T10:14:54.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.712+0000 7fca777fe700 1 -- 172.21.15.3:0/4255308363 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8400cd40 con 0x7fca9006b080 2023-10-03T10:14:54.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.713+0000 7fca777fe700 1 -- 172.21.15.3:0/4255308363 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fca84013470 con 0x7fca9006b080 2023-10-03T10:14:54.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.714+0000 7fca777fe700 1 -- 172.21.15.3:0/4255308363 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fca8403f750 con 0x7fca9006b080 2023-10-03T10:14:54.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.715+0000 7fca956bf700 1 -- 172.21.15.3:0/4255308363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca700052e0 con 0x7fca9006b080 2023-10-03T10:14:54.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.724+0000 7fca777fe700 1 -- 172.21.15.3:0/4255308363 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fca840232f0 con 0x7fca9006b080 2023-10-03T10:14:54.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.754+0000 7fd6c4f5d700 1 -- 172.21.15.3:0/3778441337 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd6c0074ea0 con 0x7fd6c01338a0 2023-10-03T10:14:54.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.755+0000 7fd6aeffd700 1 -- 172.21.15.3:0/3778441337 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd6b8012340 con 0x7fd6c01338a0 2023-10-03T10:14:54.755 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:54.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.757+0000 7fd6c4f5d700 1 -- 172.21.15.3:0/3778441337 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6c01338a0 msgr2=0x7fd6c011e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:54.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.757+0000 7fd6c4f5d700 1 --2- 172.21.15.3:0/3778441337 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6c01338a0 0x7fd6c011e2a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800efc0 tx=0x7fd6b800c3a0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.757+0000 7fd6c4f5d700 1 -- 172.21.15.3:0/3778441337 shutdown_connections 2023-10-03T10:14:54.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.757+0000 7fd6c4f5d700 1 --2- 172.21.15.3:0/3778441337 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6c01338a0 0x7fd6c011e2a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.757+0000 7fd6c4f5d700 1 --2- 172.21.15.3:0/3778441337 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6c009de50 0x7fd6c011d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.757+0000 7fd6c4f5d700 1 --2- 172.21.15.3:0/3778441337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6c009e930 0x7fd6c011dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.757+0000 7fd6c4f5d700 1 -- 172.21.15.3:0/3778441337 >> 172.21.15.3:0/3778441337 conn(0x7fd6c00a2aa0 msgr2=0x7fd6c01badd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:54.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.757+0000 7fd6c4f5d700 1 -- 172.21.15.3:0/3778441337 shutdown_connections 2023-10-03T10:14:54.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.757+0000 7fd6c4f5d700 1 -- 172.21.15.3:0/3778441337 wait complete. 2023-10-03T10:14:54.778 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:54.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.881+0000 7f629dfae700 1 -- 172.21.15.3:0/1456026911 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f629811d030 con 0x7f629809dc30 2023-10-03T10:14:54.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.881+0000 7f627ffff700 1 -- 172.21.15.3:0/1456026911 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f629811d030 con 0x7f629809dc30 2023-10-03T10:14:54.882 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:54.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.884+0000 7f629dfae700 1 -- 172.21.15.3:0/1456026911 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f629809dc30 msgr2=0x7f629813a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:54.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.884+0000 7f629dfae700 1 --2- 172.21.15.3:0/1456026911 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f629809dc30 0x7f629813a6a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f628c00e9b0 tx=0x7f628c00ed70 comp rx=0 tx=0).stop 2023-10-03T10:14:54.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.884+0000 7f629dfae700 1 -- 172.21.15.3:0/1456026911 shutdown_connections 2023-10-03T10:14:54.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.884+0000 7f629dfae700 1 --2- 172.21.15.3:0/1456026911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f629813c1e0 0x7f629813abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.884+0000 7f629dfae700 1 --2- 172.21.15.3:0/1456026911 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f629806a770 0x7f62981205c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.884+0000 7f629dfae700 1 --2- 172.21.15.3:0/1456026911 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f629809dc30 0x7f629813a6a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.884+0000 7f629dfae700 1 -- 172.21.15.3:0/1456026911 >> 172.21.15.3:0/1456026911 conn(0x7f6298066f50 msgr2=0x7f629813d460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:54.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.884+0000 7f629dfae700 1 -- 172.21.15.3:0/1456026911 shutdown_connections 2023-10-03T10:14:54.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.884+0000 7f629dfae700 1 -- 172.21.15.3:0/1456026911 wait complete. 2023-10-03T10:14:54.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.892+0000 7fd93e5e8700 1 -- 172.21.15.3:0/3052644895 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd93806b250 con 0x7fd93809e930 2023-10-03T10:14:54.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.892+0000 7fd934ff9700 1 -- 172.21.15.3:0/3052644895 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd928017030 con 0x7fd93809e930 2023-10-03T10:14:54.893 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:54.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.894+0000 7fd93e5e8700 1 -- 172.21.15.3:0/3052644895 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd93809e930 msgr2=0x7fd93811f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:54.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.894+0000 7fd93e5e8700 1 --2- 172.21.15.3:0/3052644895 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd93809e930 0x7fd93811f980 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd92800c510 tx=0x7fd92800c820 comp rx=0 tx=0).stop 2023-10-03T10:14:54.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.894+0000 7fd93e5e8700 1 -- 172.21.15.3:0/3052644895 shutdown_connections 2023-10-03T10:14:54.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.894+0000 7fd93e5e8700 1 --2- 172.21.15.3:0/3052644895 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd938136fe0 0x7fd938120400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.894+0000 7fd93e5e8700 1 --2- 172.21.15.3:0/3052644895 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd93809e930 0x7fd93811f980 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.894+0000 7fd93e5e8700 1 --2- 172.21.15.3:0/3052644895 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd93809f410 0x7fd93811fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.894+0000 7fd93e5e8700 1 -- 172.21.15.3:0/3052644895 >> 172.21.15.3:0/3052644895 conn(0x7fd9380a4210 msgr2=0x7fd93806b880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:54.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.895+0000 7fd93e5e8700 1 -- 172.21.15.3:0/3052644895 shutdown_connections 2023-10-03T10:14:54.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.895+0000 7fd93e5e8700 1 -- 172.21.15.3:0/3052644895 wait complete. 2023-10-03T10:14:54.899 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:54.909 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:54.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.934+0000 7fca956bf700 1 -- 172.21.15.3:0/4255308363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fca70005590 con 0x7fca9006b080 2023-10-03T10:14:54.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.934+0000 7fca777fe700 1 -- 172.21.15.3:0/4255308363 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fca84021030 con 0x7fca9006b080 2023-10-03T10:14:54.935 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:54.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.936+0000 7fca956bf700 1 -- 172.21.15.3:0/4255308363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca9006b080 msgr2=0x7fca90120690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:54.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.936+0000 7fca956bf700 1 --2- 172.21.15.3:0/4255308363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca9006b080 0x7fca90120690 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fca8400ebb0 tx=0x7fca8400eec0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.937+0000 7fca956bf700 1 -- 172.21.15.3:0/4255308363 shutdown_connections 2023-10-03T10:14:54.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.937+0000 7fca956bf700 1 --2- 172.21.15.3:0/4255308363 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca90131b40 0x7fca90120bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.937+0000 7fca956bf700 1 --2- 172.21.15.3:0/4255308363 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca9006a770 0x7fca90120150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.937+0000 7fca956bf700 1 --2- 172.21.15.3:0/4255308363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca9006b080 0x7fca90120690 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:54.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.937+0000 7fca956bf700 1 -- 172.21.15.3:0/4255308363 >> 172.21.15.3:0/4255308363 conn(0x7fca90066f50 msgr2=0x7fca900a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:54.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.937+0000 7fca956bf700 1 -- 172.21.15.3:0/4255308363 shutdown_connections 2023-10-03T10:14:54.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:54.937+0000 7fca956bf700 1 -- 172.21.15.3:0/4255308363 wait complete. 2023-10-03T10:14:54.951 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:55.779 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:55.899 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:55.910 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:55.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.925+0000 7f82684c8700 1 Processor -- start 2023-10-03T10:14:55.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.925+0000 7f82684c8700 1 -- start start 2023-10-03T10:14:55.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.926+0000 7f82684c8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f82601316c0 0x7f8260131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:55.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.926+0000 7f82684c8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f826006ae80 0x7f8260139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:55.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.926+0000 7f82684c8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f826009e130 con 0x7f826009d9f0 2023-10-03T10:14:55.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.926+0000 7f82684c8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82601312b0 con 0x7f82601316c0 2023-10-03T10:14:55.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.926+0000 7f82684c8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826009f6e0 con 0x7f826006ae80 2023-10-03T10:14:55.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.926+0000 7f8266a65700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f826006ae80 0x7f8260139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:55.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.926+0000 7f8266264700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f82601316c0 0x7f8260131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:55.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.926+0000 7f8266264700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f82601316c0 0x7f8260131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41056/0 (socket says 172.21.15.3:41056) 2023-10-03T10:14:55.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.926+0000 7f8266a65700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f826006ae80 0x7f8260139550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42490/0 (socket says 172.21.15.3:42490) 2023-10-03T10:14:55.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.926+0000 7f8266264700 1 -- 172.21.15.3:0/1020569968 learned_addr learned my addr 172.21.15.3:0/1020569968 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:55.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.927+0000 7f8266264700 1 -- 172.21.15.3:0/1020569968 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f826006ae80 msgr2=0x7f8260139550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:55.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.927+0000 7f8266264700 1 --2- 172.21.15.3:0/1020569968 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f826006ae80 0x7f8260139550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:55.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.927+0000 7f8266264700 1 -- 172.21.15.3:0/1020569968 >> v1:172.21.15.3:6789/0 conn(0x7f826009d9f0 legacy=0x7f826006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:55.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.927+0000 7f8266264700 1 -- 172.21.15.3:0/1020569968 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826009e760 con 0x7f82601316c0 2023-10-03T10:14:55.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.927+0000 7f8266264700 1 --2- 172.21.15.3:0/1020569968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82601316c0 0x7f8260131a90 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8250009e50 tx=0x7f825000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3c9584f608a0c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:55.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.927+0000 7f8265262700 1 -- 172.21.15.3:0/1020569968 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825000e070 con 0x7f82601316c0 2023-10-03T10:14:55.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.927+0000 7f8265262700 1 -- 172.21.15.3:0/1020569968 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825000b910 con 0x7f82601316c0 2023-10-03T10:14:55.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.927+0000 7f8265262700 1 -- 172.21.15.3:0/1020569968 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825000bcc0 con 0x7f82601316c0 2023-10-03T10:14:55.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.928+0000 7f82684c8700 1 -- 172.21.15.3:0/1020569968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82601316c0 msgr2=0x7f8260131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:55.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.928+0000 7f82684c8700 1 --2- 172.21.15.3:0/1020569968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82601316c0 0x7f8260131a90 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8250009e50 tx=0x7f825000b330 comp rx=0 tx=0).stop 2023-10-03T10:14:55.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.928+0000 7f82684c8700 1 -- 172.21.15.3:0/1020569968 shutdown_connections 2023-10-03T10:14:55.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.928+0000 7f82684c8700 1 --2- 172.21.15.3:0/1020569968 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f826006ae80 0x7f8260139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:55.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.928+0000 7f82684c8700 1 --2- 172.21.15.3:0/1020569968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82601316c0 0x7f8260131a90 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:55.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.928+0000 7f82684c8700 1 -- 172.21.15.3:0/1020569968 >> 172.21.15.3:0/1020569968 conn(0x7f8260066f50 msgr2=0x7f826009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:55.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.928+0000 7f82684c8700 1 -- 172.21.15.3:0/1020569968 shutdown_connections 2023-10-03T10:14:55.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.928+0000 7f82684c8700 1 -- 172.21.15.3:0/1020569968 wait complete. 2023-10-03T10:14:55.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.929+0000 7f82684c8700 1 Processor -- start 2023-10-03T10:14:55.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.930+0000 7f82684c8700 1 -- start start 2023-10-03T10:14:55.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.930+0000 7f82684c8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f826006ae80 0x7f8260070910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:55.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.930+0000 7f82684c8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f826009d9f0 0x7f8260070e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:55.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.930+0000 7f82684c8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82601316c0 0x7f8260071390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:55.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.930+0000 7f82684c8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826012b970 con 0x7f82601316c0 2023-10-03T10:14:55.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.930+0000 7f82684c8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826011aae0 con 0x7f826006ae80 2023-10-03T10:14:55.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.930+0000 7f82684c8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8260119cb0 con 0x7f826009d9f0 2023-10-03T10:14:55.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.930+0000 7f8266264700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f826006ae80 0x7f8260070910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:55.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.930+0000 7f8266264700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f826006ae80 0x7f8260070910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42494/0 (socket says 172.21.15.3:42494) 2023-10-03T10:14:55.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.930+0000 7f8266264700 1 -- 172.21.15.3:0/3104568545 learned_addr learned my addr 172.21.15.3:0/3104568545 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:55.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.930+0000 7f8266a65700 1 --2- 172.21.15.3:0/3104568545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82601316c0 0x7f8260071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:55.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.930+0000 7f8265a63700 1 --2- 172.21.15.3:0/3104568545 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f826009d9f0 0x7f8260070e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:55.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.931+0000 7f8266264700 1 -- 172.21.15.3:0/3104568545 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f826009d9f0 msgr2=0x7f8260070e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:55.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.931+0000 7f8266264700 1 --2- 172.21.15.3:0/3104568545 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f826009d9f0 0x7f8260070e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:55.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.931+0000 7f8266264700 1 -- 172.21.15.3:0/3104568545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82601316c0 msgr2=0x7f8260071390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:55.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.931+0000 7f8266264700 1 --2- 172.21.15.3:0/3104568545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82601316c0 0x7f8260071390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:55.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.931+0000 7f8266264700 1 -- 172.21.15.3:0/3104568545 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8250009b30 con 0x7f826006ae80 2023-10-03T10:14:55.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.931+0000 7f8266264700 1 --2- 172.21.15.3:0/3104568545 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f826006ae80 0x7f8260070910 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f82601313f0 tx=0x7f8250004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:55.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.931+0000 7f825effd700 1 -- 172.21.15.3:0/3104568545 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8250004680 con 0x7f826006ae80 2023-10-03T10:14:55.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.931+0000 7f825effd700 1 -- 172.21.15.3:0/3104568545 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82500047e0 con 0x7f826006ae80 2023-10-03T10:14:55.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.931+0000 7f82684c8700 1 -- 172.21.15.3:0/3104568545 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826011b690 con 0x7f826006ae80 2023-10-03T10:14:55.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.931+0000 7f825effd700 1 -- 172.21.15.3:0/3104568545 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825001e980 con 0x7f826006ae80 2023-10-03T10:14:55.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.931+0000 7f82684c8700 1 -- 172.21.15.3:0/3104568545 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8260072640 con 0x7f826006ae80 2023-10-03T10:14:55.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.932+0000 7f825effd700 1 -- 172.21.15.3:0/3104568545 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8250024070 con 0x7f826006ae80 2023-10-03T10:14:55.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.932+0000 7f825effd700 1 -- 172.21.15.3:0/3104568545 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8250045030 con 0x7f826006ae80 2023-10-03T10:14:55.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.933+0000 7f82684c8700 1 -- 172.21.15.3:0/3104568545 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82400052e0 con 0x7f826006ae80 2023-10-03T10:14:55.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:55.937+0000 7f825effd700 1 -- 172.21.15.3:0/3104568545 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f825001eae0 con 0x7f826006ae80 2023-10-03T10:14:55.952 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:56.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.054+0000 7f683dcb0700 1 Processor -- start 2023-10-03T10:14:56.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.054+0000 7f683dcb0700 1 -- start start 2023-10-03T10:14:56.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.055+0000 7f683dcb0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f68380a0a30 0x7f68380a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:56.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.055+0000 7f683dcb0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6838138d80 0x7f683813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:56.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.055+0000 7f683dcb0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68380a2a70 con 0x7f683809d630 2023-10-03T10:14:56.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.055+0000 7f683dcb0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68380a01d0 con 0x7f68380a0a30 2023-10-03T10:14:56.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.055+0000 7f683dcb0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f683809e2b0 con 0x7f6838138d80 2023-10-03T10:14:56.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.055+0000 7f68377fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f68380a0a30 0x7f68380a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:56.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.055+0000 7f6836ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f683809d630 0x7f68380a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48244/0 (socket says 172.21.15.3:48244) 2023-10-03T10:14:56.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.055+0000 7f68377fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f68380a0a30 0x7f68380a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42502/0 (socket says 172.21.15.3:42502) 2023-10-03T10:14:56.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.055+0000 7f6836ffd700 1 -- 172.21.15.3:0/2937221534 learned_addr learned my addr 172.21.15.3:0/2937221534 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:56.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.055+0000 7f6837fff700 1 --2- 172.21.15.3:0/2937221534 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6838138d80 0x7f683813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:56.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.055+0000 7f68367fc700 1 -- 172.21.15.3:0/2937221534 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2910901741 0 0) 0x7f68380a2a70 con 0x7f683809d630 2023-10-03T10:14:56.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.055+0000 7f68367fc700 1 -- 172.21.15.3:0/2937221534 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68200035b0 con 0x7f683809d630 2023-10-03T10:14:56.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.056+0000 7f68377fe700 1 -- 172.21.15.3:0/2937221534 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6838138d80 msgr2=0x7f683813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:56.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.056+0000 7f68377fe700 1 --2- 172.21.15.3:0/2937221534 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6838138d80 0x7f683813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.056+0000 7f68377fe700 1 -- 172.21.15.3:0/2937221534 >> v1:172.21.15.3:6789/0 conn(0x7f683809d630 legacy=0x7f68380a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:56.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.056+0000 7f68377fe700 1 -- 172.21.15.3:0/2937221534 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683809dc50 con 0x7f68380a0a30 2023-10-03T10:14:56.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.056+0000 7f6837fff700 1 --2- 172.21.15.3:0/2937221534 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6838138d80 0x7f683813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:56.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.056+0000 7f68377fe700 1 --2- 172.21.15.3:0/2937221534 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f68380a0a30 0x7f68380a0e00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6828009870 tx=0x7f6828009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8aabe1920a4786d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:56.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.056+0000 7f68367fc700 1 -- 172.21.15.3:0/2937221534 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6828011830 con 0x7f68380a0a30 2023-10-03T10:14:56.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.056+0000 7f68367fc700 1 -- 172.21.15.3:0/2937221534 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6828011990 con 0x7f68380a0a30 2023-10-03T10:14:56.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.056+0000 7f68367fc700 1 -- 172.21.15.3:0/2937221534 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6828011d20 con 0x7f68380a0a30 2023-10-03T10:14:56.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.057+0000 7f683dcb0700 1 -- 172.21.15.3:0/2937221534 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f68380a0a30 msgr2=0x7f68380a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:56.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.057+0000 7f683dcb0700 1 --2- 172.21.15.3:0/2937221534 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f68380a0a30 0x7f68380a0e00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6828009870 tx=0x7f6828009b80 comp rx=0 tx=0).stop 2023-10-03T10:14:56.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.057+0000 7f683dcb0700 1 -- 172.21.15.3:0/2937221534 shutdown_connections 2023-10-03T10:14:56.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.057+0000 7f683dcb0700 1 --2- 172.21.15.3:0/2937221534 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f68380a0a30 0x7f68380a0e00 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.057+0000 7f683dcb0700 1 --2- 172.21.15.3:0/2937221534 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6838138d80 0x7f683813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.057+0000 7f683dcb0700 1 -- 172.21.15.3:0/2937221534 >> 172.21.15.3:0/2937221534 conn(0x7f68380a3050 msgr2=0x7f68380a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:56.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.057+0000 7f683dcb0700 1 -- 172.21.15.3:0/2937221534 shutdown_connections 2023-10-03T10:14:56.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.057+0000 7f683dcb0700 1 -- 172.21.15.3:0/2937221534 wait complete. 2023-10-03T10:14:56.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.058+0000 7f683dcb0700 1 Processor -- start 2023-10-03T10:14:56.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.059+0000 7f683dcb0700 1 -- start start 2023-10-03T10:14:56.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.059+0000 7f683dcb0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f683809d630 0x7f6838133760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:56.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.059+0000 7f683dcb0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68380a0a30 0x7f6838133ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:56.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.059+0000 7f683dcb0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6838138d80 0x7f68381341e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:56.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.059+0000 7f683dcb0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68381365a0 con 0x7f68380a0a30 2023-10-03T10:14:56.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.059+0000 7f683dcb0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6838138b10 con 0x7f6838138d80 2023-10-03T10:14:56.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.059+0000 7f683dcb0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6838136f10 con 0x7f683809d630 2023-10-03T10:14:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.059+0000 7f68377fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f683809d630 0x7f6838133760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.059+0000 7f68377fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f683809d630 0x7f6838133760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56002/0 (socket says 172.21.15.3:56002) 2023-10-03T10:14:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.059+0000 7f68377fe700 1 -- 172.21.15.3:0/2769059427 learned_addr learned my addr 172.21.15.3:0/2769059427 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.059+0000 7f6836ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68380a0a30 0x7f6838133ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.059+0000 7f6837fff700 1 --2- 172.21.15.3:0/2769059427 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6838138d80 0x7f68381341e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.060+0000 7f6836ffd700 1 -- 172.21.15.3:0/2769059427 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f683809d630 msgr2=0x7f6838133760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.060+0000 7f6836ffd700 1 --2- 172.21.15.3:0/2769059427 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f683809d630 0x7f6838133760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.060+0000 7f6836ffd700 1 -- 172.21.15.3:0/2769059427 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6838138d80 msgr2=0x7f68381341e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.060+0000 7f6836ffd700 1 --2- 172.21.15.3:0/2769059427 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6838138d80 0x7f68381341e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.060+0000 7f6836ffd700 1 -- 172.21.15.3:0/2769059427 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6830009a70 con 0x7f68380a0a30 2023-10-03T10:14:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.060+0000 7f68377fe700 1 --2- 172.21.15.3:0/2769059427 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f683809d630 0x7f6838133760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:56.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.060+0000 7f6836ffd700 1 --2- 172.21.15.3:0/2769059427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68380a0a30 0x7f6838133ca0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f682c00c4c0 tx=0x7f682c00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:56.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.060+0000 7f681ffff700 1 -- 172.21.15.3:0/2769059427 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c005d70 con 0x7f68380a0a30 2023-10-03T10:14:56.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.060+0000 7f681ffff700 1 -- 172.21.15.3:0/2769059427 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682c0070d0 con 0x7f68380a0a30 2023-10-03T10:14:56.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.060+0000 7f681ffff700 1 -- 172.21.15.3:0/2769059427 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c007440 con 0x7f68380a0a30 2023-10-03T10:14:56.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.060+0000 7f683dcb0700 1 -- 172.21.15.3:0/2769059427 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68280094a0 con 0x7f68380a0a30 2023-10-03T10:14:56.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.060+0000 7f683dcb0700 1 -- 172.21.15.3:0/2769059427 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68381371f0 con 0x7f68380a0a30 2023-10-03T10:14:56.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.062+0000 7f683dcb0700 1 -- 172.21.15.3:0/2769059427 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6838137e30 con 0x7f68380a0a30 2023-10-03T10:14:56.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.066+0000 7f681ffff700 1 -- 172.21.15.3:0/2769059427 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f682c0075a0 con 0x7f68380a0a30 2023-10-03T10:14:56.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.066+0000 7f681ffff700 1 -- 172.21.15.3:0/2769059427 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f682c03eb60 con 0x7f68380a0a30 2023-10-03T10:14:56.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.067+0000 7f681ffff700 1 -- 172.21.15.3:0/2769059427 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f682c01a360 con 0x7f68380a0a30 2023-10-03T10:14:56.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.086+0000 7fd44309a700 1 Processor -- start 2023-10-03T10:14:56.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.086+0000 7fd44309a700 1 -- start start 2023-10-03T10:14:56.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.087+0000 7fd44309a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd43c09d6f0 0x7fd43c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:56.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.087+0000 7fd44309a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd43c09e410 0x7fd43c134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:56.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.087+0000 7fd44309a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd43c09b120 con 0x7fd43c09e000 2023-10-03T10:14:56.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.087+0000 7fd44309a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd43c06b460 con 0x7fd43c09e410 2023-10-03T10:14:56.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.087+0000 7fd44309a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c09a7e0 con 0x7fd43c09d6f0 2023-10-03T10:14:56.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.087+0000 7fd440e36700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd43c09d6f0 0x7fd43c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:56.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.087+0000 7fd43bfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd43c09e000 0x7fd43c1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48256/0 (socket says 172.21.15.3:48256) 2023-10-03T10:14:56.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.087+0000 7fd43bfff700 1 -- 172.21.15.3:0/2887314749 learned_addr learned my addr 172.21.15.3:0/2887314749 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:56.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.087+0000 7fd441637700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd43c09e410 0x7fd43c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:56.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.088+0000 7fd441637700 1 -- 172.21.15.3:0/2887314749 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd43c09d6f0 msgr2=0x7fd43c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:56.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.088+0000 7fd441637700 1 --2- 172.21.15.3:0/2887314749 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd43c09d6f0 0x7fd43c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.088+0000 7fd441637700 1 -- 172.21.15.3:0/2887314749 >> v1:172.21.15.3:6789/0 conn(0x7fd43c09e000 legacy=0x7fd43c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:56.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.088+0000 7fd441637700 1 -- 172.21.15.3:0/2887314749 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd43c09a500 con 0x7fd43c09e410 2023-10-03T10:14:56.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.088+0000 7fd441637700 1 --2- 172.21.15.3:0/2887314749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd43c09e410 0x7fd43c134100 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd434009d90 tx=0x7fd43400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48ee30f2f83d78 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:56.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.088+0000 7fd43b7fe700 1 -- 172.21.15.3:0/2887314749 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43400e070 con 0x7fd43c09e410 2023-10-03T10:14:56.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.088+0000 7fd43b7fe700 1 -- 172.21.15.3:0/2887314749 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43400bb20 con 0x7fd43c09e410 2023-10-03T10:14:56.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.088+0000 7fd43b7fe700 1 -- 172.21.15.3:0/2887314749 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd434012430 con 0x7fd43c09e410 2023-10-03T10:14:56.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.089+0000 7fd44309a700 1 -- 172.21.15.3:0/2887314749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd43c09e410 msgr2=0x7fd43c134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:56.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.089+0000 7fd44309a700 1 --2- 172.21.15.3:0/2887314749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd43c09e410 0x7fd43c134100 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd434009d90 tx=0x7fd43400b330 comp rx=0 tx=0).stop 2023-10-03T10:14:56.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.089+0000 7fd44309a700 1 -- 172.21.15.3:0/2887314749 shutdown_connections 2023-10-03T10:14:56.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.089+0000 7fd44309a700 1 --2- 172.21.15.3:0/2887314749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd43c09e410 0x7fd43c134100 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.089+0000 7fd44309a700 1 --2- 172.21.15.3:0/2887314749 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd43c09d6f0 0x7fd43c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.089+0000 7fd44309a700 1 -- 172.21.15.3:0/2887314749 >> 172.21.15.3:0/2887314749 conn(0x7fd43c066f50 msgr2=0x7fd43c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:56.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.090+0000 7fd44309a700 1 -- 172.21.15.3:0/2887314749 shutdown_connections 2023-10-03T10:14:56.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.090+0000 7fd44309a700 1 -- 172.21.15.3:0/2887314749 wait complete. 2023-10-03T10:14:56.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.091+0000 7fd44309a700 1 Processor -- start 2023-10-03T10:14:56.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.091+0000 7fd44309a700 1 -- start start 2023-10-03T10:14:56.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.092+0000 7fd44309a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd43c09d6f0 0x7fd43c1a5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:56.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.092+0000 7fd44309a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd43c09e000 0x7fd43c1a6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:56.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.092+0000 7fd44309a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd43c09e410 0x7fd43c1a6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:56.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.092+0000 7fd44309a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c1a8940 con 0x7fd43c09d6f0 2023-10-03T10:14:56.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.092+0000 7fd440e36700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd43c09d6f0 0x7fd43c1a5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:56.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.092+0000 7fd440e36700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd43c09d6f0 0x7fd43c1a5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41092/0 (socket says 172.21.15.3:41092) 2023-10-03T10:14:56.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.092+0000 7fd440e36700 1 -- 172.21.15.3:0/2473415661 learned_addr learned my addr 172.21.15.3:0/2473415661 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:56.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.092+0000 7fd43bfff700 1 --2- 172.21.15.3:0/2473415661 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd43c09e000 0x7fd43c1a6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:56.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.092+0000 7fd441637700 1 --2- 172.21.15.3:0/2473415661 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd43c09e410 0x7fd43c1a6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:56.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.093+0000 7fd44309a700 1 -- 172.21.15.3:0/2473415661 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c1a7470 con 0x7fd43c09e000 2023-10-03T10:14:56.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.093+0000 7fd44309a700 1 -- 172.21.15.3:0/2473415661 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c1a7b40 con 0x7fd43c09e410 2023-10-03T10:14:56.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.093+0000 7fd440e36700 1 -- 172.21.15.3:0/2473415661 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd43c09e410 msgr2=0x7fd43c1a6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:56.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.093+0000 7fd440e36700 1 --2- 172.21.15.3:0/2473415661 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd43c09e410 0x7fd43c1a6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.093+0000 7fd440e36700 1 -- 172.21.15.3:0/2473415661 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd43c09e000 msgr2=0x7fd43c1a6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:56.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.093+0000 7fd440e36700 1 --2- 172.21.15.3:0/2473415661 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd43c09e000 0x7fd43c1a6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.093+0000 7fd440e36700 1 -- 172.21.15.3:0/2473415661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd434009a70 con 0x7fd43c09d6f0 2023-10-03T10:14:56.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.093+0000 7fd43bfff700 1 --2- 172.21.15.3:0/2473415661 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd43c09e000 0x7fd43c1a6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:56.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.093+0000 7fd440e36700 1 --2- 172.21.15.3:0/2473415661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd43c09d6f0 0x7fd43c1a5fc0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd43000e9e0 tx=0x7fd43000ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:56.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.094+0000 7fd4397fa700 1 -- 172.21.15.3:0/2473415661 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd430009710 con 0x7fd43c09d6f0 2023-10-03T10:14:56.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.094+0000 7fd4397fa700 1 -- 172.21.15.3:0/2473415661 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43000ee80 con 0x7fd43c09d6f0 2023-10-03T10:14:56.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.094+0000 7fd44309a700 1 -- 172.21.15.3:0/2473415661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd43c1a8e00 con 0x7fd43c09d6f0 2023-10-03T10:14:56.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.094+0000 7fd4397fa700 1 -- 172.21.15.3:0/2473415661 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43000ca50 con 0x7fd43c09d6f0 2023-10-03T10:14:56.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.094+0000 7fd44309a700 1 -- 172.21.15.3:0/2473415661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd43c0a30e0 con 0x7fd43c09d6f0 2023-10-03T10:14:56.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.096+0000 7fd44309a700 1 -- 172.21.15.3:0/2473415661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd43c1a9890 con 0x7fd43c09d6f0 2023-10-03T10:14:56.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.096+0000 7fd4397fa700 1 -- 172.21.15.3:0/2473415661 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd43000cbb0 con 0x7fd43c09d6f0 2023-10-03T10:14:56.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.096+0000 7fd4397fa700 1 -- 172.21.15.3:0/2473415661 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd43003fad0 con 0x7fd43c09d6f0 2023-10-03T10:14:56.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.104+0000 7ff09702b700 1 Processor -- start 2023-10-03T10:14:56.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.104+0000 7ff09702b700 1 -- start start 2023-10-03T10:14:56.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.105+0000 7ff09702b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff09009e930 0x7ff09009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:56.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.105+0000 7ff09702b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff09009f240 0x7ff0901317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:56.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.105+0000 7ff09702b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff09009f650 0x7ff090131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:56.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.105+0000 7ff09702b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff090070940 con 0x7ff09009f650 2023-10-03T10:14:56.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.105+0000 7ff09702b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09006fd60 con 0x7ff09009f240 2023-10-03T10:14:56.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.105+0000 7ff09702b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff09006fa80 con 0x7ff09009e930 2023-10-03T10:14:56.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.105+0000 7ff094dc7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff09009e930 0x7ff09009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:56.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.105+0000 7ff094dc7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff09009e930 0x7ff09009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56026/0 (socket says 172.21.15.3:56026) 2023-10-03T10:14:56.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.105+0000 7ff094dc7700 1 -- 172.21.15.3:0/946685867 learned_addr learned my addr 172.21.15.3:0/946685867 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:56.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.105+0000 7ff08ffff700 1 --2- 172.21.15.3:0/946685867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff09009f240 0x7ff0901317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:56.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.106+0000 7fd4397fa700 1 -- 172.21.15.3:0/2473415661 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd43001b3b0 con 0x7fd43c09d6f0 2023-10-03T10:14:56.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.106+0000 7ff08ffff700 1 -- 172.21.15.3:0/946685867 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff09009e930 msgr2=0x7ff09009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:56.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.106+0000 7ff08ffff700 1 --2- 172.21.15.3:0/946685867 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff09009e930 0x7ff09009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.106+0000 7ff08ffff700 1 -- 172.21.15.3:0/946685867 >> v2:172.21.15.3:3300/0 conn(0x7ff09009f650 msgr2=0x7ff090131d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-03T10:14:56.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.106+0000 7ff08ffff700 1 --2- 172.21.15.3:0/946685867 >> v2:172.21.15.3:3300/0 conn(0x7ff09009f650 0x7ff090131d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.106+0000 7ff08ffff700 1 -- 172.21.15.3:0/946685867 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09009fb60 con 0x7ff09009f240 2023-10-03T10:14:56.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.106+0000 7ff08ffff700 1 --2- 172.21.15.3:0/946685867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff09009f240 0x7ff0901317f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00dc60 tx=0x7ff07c00df70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a846ce11dfed8101 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:56.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.106+0000 7ff08f7fe700 1 -- 172.21.15.3:0/946685867 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c00f8e0 con 0x7ff09009f240 2023-10-03T10:14:56.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.106+0000 7ff08f7fe700 1 -- 172.21.15.3:0/946685867 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07c00fa40 con 0x7ff09009f240 2023-10-03T10:14:56.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.106+0000 7ff08f7fe700 1 -- 172.21.15.3:0/946685867 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c00fdf0 con 0x7ff09009f240 2023-10-03T10:14:56.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.107+0000 7ff09702b700 1 -- 172.21.15.3:0/946685867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff09009f240 msgr2=0x7ff0901317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:56.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.107+0000 7ff09702b700 1 --2- 172.21.15.3:0/946685867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff09009f240 0x7ff0901317f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00dc60 tx=0x7ff07c00df70 comp rx=0 tx=0).stop 2023-10-03T10:14:56.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.108+0000 7ff09702b700 1 -- 172.21.15.3:0/946685867 shutdown_connections 2023-10-03T10:14:56.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.108+0000 7ff09702b700 1 --2- 172.21.15.3:0/946685867 >> v2:172.21.15.3:3300/0 conn(0x7ff09009f650 0x7ff090131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.108+0000 7ff09702b700 1 --2- 172.21.15.3:0/946685867 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff09009e930 0x7ff09009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.108+0000 7ff09702b700 1 --2- 172.21.15.3:0/946685867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff09009f240 0x7ff0901317f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.108+0000 7ff09702b700 1 -- 172.21.15.3:0/946685867 >> 172.21.15.3:0/946685867 conn(0x7ff0900a4210 msgr2=0x7ff0900a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:56.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.108+0000 7ff09702b700 1 -- 172.21.15.3:0/946685867 shutdown_connections 2023-10-03T10:14:56.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.108+0000 7ff09702b700 1 -- 172.21.15.3:0/946685867 wait complete. 2023-10-03T10:14:56.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.109+0000 7ff09702b700 1 Processor -- start 2023-10-03T10:14:56.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.109+0000 7ff09702b700 1 -- start start 2023-10-03T10:14:56.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.110+0000 7ff09702b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff09009e930 0x7ff09011d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:56.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.110+0000 7ff09702b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff09009f240 0x7ff09011d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:56.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.110+0000 7ff09702b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff09009f650 0x7ff09011de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:56.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.110+0000 7ff09702b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff090120e90 con 0x7ff09009f650 2023-10-03T10:14:56.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.110+0000 7ff09702b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0901206c0 con 0x7ff09009e930 2023-10-03T10:14:56.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.110+0000 7ff09702b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff090071d20 con 0x7ff09009f240 2023-10-03T10:14:56.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.110+0000 7ff08ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff09009f240 0x7ff09011d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:56.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.110+0000 7ff0955c8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff09009f650 0x7ff09011de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:56.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.110+0000 7ff0955c8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff09009f650 0x7ff09011de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41104/0 (socket says 172.21.15.3:41104) 2023-10-03T10:14:56.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.110+0000 7ff08ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff09009f240 0x7ff09011d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56028/0 (socket says 172.21.15.3:56028) 2023-10-03T10:14:56.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.110+0000 7ff0955c8700 1 -- 172.21.15.3:0/87636092 learned_addr learned my addr 172.21.15.3:0/87636092 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:56.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.111+0000 7ff094dc7700 1 --2- 172.21.15.3:0/87636092 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff09009e930 0x7ff09011d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:56.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.111+0000 7ff0955c8700 1 -- 172.21.15.3:0/87636092 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff09009f240 msgr2=0x7ff09011d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:56.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.111+0000 7ff0955c8700 1 --2- 172.21.15.3:0/87636092 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff09009f240 0x7ff09011d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.111+0000 7ff0955c8700 1 -- 172.21.15.3:0/87636092 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff09009e930 msgr2=0x7ff09011d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:56.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.111+0000 7ff0955c8700 1 --2- 172.21.15.3:0/87636092 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff09009e930 0x7ff09011d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.111+0000 7ff0955c8700 1 -- 172.21.15.3:0/87636092 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07c00d940 con 0x7ff09009f650 2023-10-03T10:14:56.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.111+0000 7ff0955c8700 1 --2- 172.21.15.3:0/87636092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff09009f650 0x7ff09011de80 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff08800ba10 tx=0x7ff08800bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:56.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.117+0000 7ff08d7fa700 1 -- 172.21.15.3:0/87636092 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08800c930 con 0x7ff09009f650 2023-10-03T10:14:56.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.117+0000 7ff08d7fa700 1 -- 172.21.15.3:0/87636092 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08800cad0 con 0x7ff09009f650 2023-10-03T10:14:56.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.117+0000 7ff09702b700 1 -- 172.21.15.3:0/87636092 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0900a0190 con 0x7ff09009f650 2023-10-03T10:14:56.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.117+0000 7ff08d7fa700 1 -- 172.21.15.3:0/87636092 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08800ce80 con 0x7ff09009f650 2023-10-03T10:14:56.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.117+0000 7ff09702b700 1 -- 172.21.15.3:0/87636092 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff09011f580 con 0x7ff09009f650 2023-10-03T10:14:56.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.118+0000 7ff09702b700 1 -- 172.21.15.3:0/87636092 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff090120a10 con 0x7ff09009f650 2023-10-03T10:14:56.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.118+0000 7ff08d7fa700 1 -- 172.21.15.3:0/87636092 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff088014440 con 0x7ff09009f650 2023-10-03T10:14:56.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.118+0000 7ff08d7fa700 1 -- 172.21.15.3:0/87636092 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff08803f3f0 con 0x7ff09009f650 2023-10-03T10:14:56.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.127+0000 7ff08d7fa700 1 -- 172.21.15.3:0/87636092 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff090120a10 con 0x7ff09009f650 2023-10-03T10:14:56.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.163+0000 7f82684c8700 1 -- 172.21.15.3:0/3104568545 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8240005590 con 0x7f826006ae80 2023-10-03T10:14:56.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.164+0000 7f825effd700 1 -- 172.21.15.3:0/3104568545 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8250035030 con 0x7f826006ae80 2023-10-03T10:14:56.164 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:56.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.166+0000 7f82684c8700 1 -- 172.21.15.3:0/3104568545 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f826006ae80 msgr2=0x7f8260070910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:56.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.166+0000 7f82684c8700 1 --2- 172.21.15.3:0/3104568545 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f826006ae80 0x7f8260070910 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f82601313f0 tx=0x7f8250004ad0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.166+0000 7f82684c8700 1 -- 172.21.15.3:0/3104568545 shutdown_connections 2023-10-03T10:14:56.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.166+0000 7f82684c8700 1 --2- 172.21.15.3:0/3104568545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82601316c0 0x7f8260071390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.166+0000 7f82684c8700 1 --2- 172.21.15.3:0/3104568545 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f826006ae80 0x7f8260070910 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.166+0000 7f82684c8700 1 --2- 172.21.15.3:0/3104568545 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f826009d9f0 0x7f8260070e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.166+0000 7f82684c8700 1 -- 172.21.15.3:0/3104568545 >> 172.21.15.3:0/3104568545 conn(0x7f8260066f50 msgr2=0x7f82600a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:56.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.167+0000 7f82684c8700 1 -- 172.21.15.3:0/3104568545 shutdown_connections 2023-10-03T10:14:56.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.167+0000 7f82684c8700 1 -- 172.21.15.3:0/3104568545 wait complete. 2023-10-03T10:14:56.182 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:56.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.330+0000 7f683dcb0700 1 -- 172.21.15.3:0/2769059427 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6838137dd0 con 0x7f68380a0a30 2023-10-03T10:14:56.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.330+0000 7f681ffff700 1 -- 172.21.15.3:0/2769059427 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f682c018270 con 0x7f68380a0a30 2023-10-03T10:14:56.331 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:56.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.332+0000 7f683dcb0700 1 -- 172.21.15.3:0/2769059427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68380a0a30 msgr2=0x7f6838133ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.332+0000 7f683dcb0700 1 --2- 172.21.15.3:0/2769059427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68380a0a30 0x7f6838133ca0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f682c00c4c0 tx=0x7f682c00c7d0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.332+0000 7f683dcb0700 1 -- 172.21.15.3:0/2769059427 shutdown_connections 2023-10-03T10:14:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.332+0000 7f683dcb0700 1 --2- 172.21.15.3:0/2769059427 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6838138d80 0x7f68381341e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.332+0000 7f683dcb0700 1 --2- 172.21.15.3:0/2769059427 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f683809d630 0x7f6838133760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.332+0000 7f683dcb0700 1 --2- 172.21.15.3:0/2769059427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68380a0a30 0x7f6838133ca0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.333+0000 7f683dcb0700 1 -- 172.21.15.3:0/2769059427 >> 172.21.15.3:0/2769059427 conn(0x7f68380a3050 msgr2=0x7f683813ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.333+0000 7f683dcb0700 1 -- 172.21.15.3:0/2769059427 shutdown_connections 2023-10-03T10:14:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.333+0000 7f683dcb0700 1 -- 172.21.15.3:0/2769059427 wait complete. 2023-10-03T10:14:56.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.341+0000 7ff09702b700 1 -- 172.21.15.3:0/87636092 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff090120a10 con 0x7ff09009f650 2023-10-03T10:14:56.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.342+0000 7ff08d7fa700 1 -- 172.21.15.3:0/87636092 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff090120a10 con 0x7ff09009f650 2023-10-03T10:14:56.342 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:56.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.345+0000 7ff09702b700 1 -- 172.21.15.3:0/87636092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff09009f650 msgr2=0x7ff09011de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:56.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.345+0000 7ff09702b700 1 --2- 172.21.15.3:0/87636092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff09009f650 0x7ff09011de80 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff08800ba10 tx=0x7ff08800bd20 comp rx=0 tx=0).stop 2023-10-03T10:14:56.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.345+0000 7ff09702b700 1 -- 172.21.15.3:0/87636092 shutdown_connections 2023-10-03T10:14:56.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.345+0000 7ff09702b700 1 --2- 172.21.15.3:0/87636092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff09009f650 0x7ff09011de80 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.345+0000 7ff09702b700 1 --2- 172.21.15.3:0/87636092 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff09009e930 0x7ff09011d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.345+0000 7ff09702b700 1 --2- 172.21.15.3:0/87636092 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff09009f240 0x7ff09011d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.345+0000 7ff09702b700 1 -- 172.21.15.3:0/87636092 >> 172.21.15.3:0/87636092 conn(0x7ff0900a4210 msgr2=0x7ff0901309f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:56.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.346+0000 7ff09702b700 1 -- 172.21.15.3:0/87636092 shutdown_connections 2023-10-03T10:14:56.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.346+0000 7ff09702b700 1 -- 172.21.15.3:0/87636092 wait complete. 2023-10-03T10:14:56.347 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:56.364 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:56.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.370+0000 7fd44309a700 1 -- 172.21.15.3:0/2473415661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd43c1a9a50 con 0x7fd43c09d6f0 2023-10-03T10:14:56.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.370+0000 7fd4397fa700 1 -- 172.21.15.3:0/2473415661 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd43c1a9a50 con 0x7fd43c09d6f0 2023-10-03T10:14:56.370 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:56.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.372+0000 7fd44309a700 1 -- 172.21.15.3:0/2473415661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd43c09d6f0 msgr2=0x7fd43c1a5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:56.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.372+0000 7fd44309a700 1 --2- 172.21.15.3:0/2473415661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd43c09d6f0 0x7fd43c1a5fc0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd43000e9e0 tx=0x7fd43000ecf0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.372+0000 7fd44309a700 1 -- 172.21.15.3:0/2473415661 shutdown_connections 2023-10-03T10:14:56.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.372+0000 7fd44309a700 1 --2- 172.21.15.3:0/2473415661 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd43c09e410 0x7fd43c1a6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.372+0000 7fd44309a700 1 --2- 172.21.15.3:0/2473415661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd43c09d6f0 0x7fd43c1a5fc0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.372+0000 7fd44309a700 1 --2- 172.21.15.3:0/2473415661 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd43c09e000 0x7fd43c1a6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:56.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.372+0000 7fd44309a700 1 -- 172.21.15.3:0/2473415661 >> 172.21.15.3:0/2473415661 conn(0x7fd43c066f50 msgr2=0x7fd43c1baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:56.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.372+0000 7fd44309a700 1 -- 172.21.15.3:0/2473415661 shutdown_connections 2023-10-03T10:14:56.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:56.372+0000 7fd44309a700 1 -- 172.21.15.3:0/2473415661 wait complete. 2023-10-03T10:14:56.386 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:57.182 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:57.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.331+0000 7fe7530bf700 1 Processor -- start 2023-10-03T10:14:57.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.331+0000 7fe7530bf700 1 -- start start 2023-10-03T10:14:57.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.332+0000 7fe7530bf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe74c09e930 0x7fe74c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:57.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.332+0000 7fe7530bf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe74c09f240 0x7fe74c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:57.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.332+0000 7fe7530bf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe74c09f650 0x7fe74c131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:57.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.332+0000 7fe7530bf700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe74c070940 con 0x7fe74c09f240 2023-10-03T10:14:57.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.332+0000 7fe7530bf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c06fd60 con 0x7fe74c09e930 2023-10-03T10:14:57.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.332+0000 7fe7530bf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c06fa80 con 0x7fe74c09f650 2023-10-03T10:14:57.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.332+0000 7fe75165c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe74c09f650 0x7fe74c131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:57.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.332+0000 7fe75165c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe74c09f650 0x7fe74c131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56034/0 (socket says 172.21.15.3:56034) 2023-10-03T10:14:57.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.332+0000 7fe75165c700 1 -- 172.21.15.3:0/666721849 learned_addr learned my addr 172.21.15.3:0/666721849 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:57.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.332+0000 7fe74bfff700 1 --2- 172.21.15.3:0/666721849 >> v2:172.21.15.3:3300/0 conn(0x7fe74c09f240 0x7fe74c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:57.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.332+0000 7fe750e5b700 1 --2- 172.21.15.3:0/666721849 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe74c09e930 0x7fe74c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:57.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.333+0000 7fe74bfff700 1 -- 172.21.15.3:0/666721849 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe74c09f650 msgr2=0x7fe74c131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:57.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.333+0000 7fe74bfff700 1 --2- 172.21.15.3:0/666721849 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe74c09f650 0x7fe74c131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.333+0000 7fe74bfff700 1 -- 172.21.15.3:0/666721849 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe74c09e930 msgr2=0x7fe74c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:57.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.333+0000 7fe74bfff700 1 --2- 172.21.15.3:0/666721849 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe74c09e930 0x7fe74c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.333+0000 7fe74bfff700 1 -- 172.21.15.3:0/666721849 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74c09fb60 con 0x7fe74c09f240 2023-10-03T10:14:57.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.333+0000 7fe75165c700 1 --2- 172.21.15.3:0/666721849 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe74c09f650 0x7fe74c131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:57.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.333+0000 7fe750e5b700 1 --2- 172.21.15.3:0/666721849 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe74c09e930 0x7fe74c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:57.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.333+0000 7fe74bfff700 1 --2- 172.21.15.3:0/666721849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe74c09f240 0x7fe74c1317f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe734009840 tx=0x7fe734009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4cdc518ef612bf server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:57.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.333+0000 7fe74b7fe700 1 -- 172.21.15.3:0/666721849 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe734011830 con 0x7fe74c09f240 2023-10-03T10:14:57.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.333+0000 7fe74b7fe700 1 -- 172.21.15.3:0/666721849 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe734011990 con 0x7fe74c09f240 2023-10-03T10:14:57.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.333+0000 7fe74b7fe700 1 -- 172.21.15.3:0/666721849 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe734011d20 con 0x7fe74c09f240 2023-10-03T10:14:57.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.333+0000 7fe7530bf700 1 -- 172.21.15.3:0/666721849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe74c09f240 msgr2=0x7fe74c1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:57.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.333+0000 7fe7530bf700 1 --2- 172.21.15.3:0/666721849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe74c09f240 0x7fe74c1317f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe734009840 tx=0x7fe734009b50 comp rx=0 tx=0).stop 2023-10-03T10:14:57.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.334+0000 7fe7530bf700 1 -- 172.21.15.3:0/666721849 shutdown_connections 2023-10-03T10:14:57.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.334+0000 7fe7530bf700 1 --2- 172.21.15.3:0/666721849 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe74c09f650 0x7fe74c131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.334+0000 7fe7530bf700 1 --2- 172.21.15.3:0/666721849 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe74c09e930 0x7fe74c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.334+0000 7fe7530bf700 1 --2- 172.21.15.3:0/666721849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe74c09f240 0x7fe74c1317f0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.334+0000 7fe7530bf700 1 -- 172.21.15.3:0/666721849 >> 172.21.15.3:0/666721849 conn(0x7fe74c0a4210 msgr2=0x7fe74c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:57.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.334+0000 7fe7530bf700 1 -- 172.21.15.3:0/666721849 shutdown_connections 2023-10-03T10:14:57.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.334+0000 7fe7530bf700 1 -- 172.21.15.3:0/666721849 wait complete. 2023-10-03T10:14:57.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.335+0000 7fe7530bf700 1 Processor -- start 2023-10-03T10:14:57.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.335+0000 7fe7530bf700 1 -- start start 2023-10-03T10:14:57.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.335+0000 7fe7530bf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe74c09e930 0x7fe74c1a3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:57.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.335+0000 7fe7530bf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe74c09f240 0x7fe74c1a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:57.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.335+0000 7fe7530bf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe74c09f650 0x7fe74c1a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:57.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.335+0000 7fe7530bf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c1a5330 con 0x7fe74c09e930 2023-10-03T10:14:57.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.335+0000 7fe7530bf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c1a5a00 con 0x7fe74c09f240 2023-10-03T10:14:57.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.335+0000 7fe7530bf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c1a6c50 con 0x7fe74c09f650 2023-10-03T10:14:57.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.335+0000 7fe750e5b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe74c09e930 0x7fe74c1a3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:57.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.335+0000 7fe74bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe74c09f240 0x7fe74c1a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:57.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.336+0000 7fe74bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe74c09f240 0x7fe74c1a43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42570/0 (socket says 172.21.15.3:42570) 2023-10-03T10:14:57.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.336+0000 7fe750e5b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe74c09e930 0x7fe74c1a3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41126/0 (socket says 172.21.15.3:41126) 2023-10-03T10:14:57.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.336+0000 7fe74bfff700 1 -- 172.21.15.3:0/1514500189 learned_addr learned my addr 172.21.15.3:0/1514500189 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:57.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.336+0000 7fe75165c700 1 --2- 172.21.15.3:0/1514500189 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe74c09f650 0x7fe74c1a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:57.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.336+0000 7fe750e5b700 1 -- 172.21.15.3:0/1514500189 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe74c09f650 msgr2=0x7fe74c1a4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:57.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.336+0000 7fe750e5b700 1 --2- 172.21.15.3:0/1514500189 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe74c09f650 0x7fe74c1a4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.336+0000 7fe750e5b700 1 -- 172.21.15.3:0/1514500189 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe74c09f240 msgr2=0x7fe74c1a43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:57.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.336+0000 7fe750e5b700 1 --2- 172.21.15.3:0/1514500189 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe74c09f240 0x7fe74c1a43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.336+0000 7fe750e5b700 1 -- 172.21.15.3:0/1514500189 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe73c0094a0 con 0x7fe74c09e930 2023-10-03T10:14:57.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.336+0000 7fe74bfff700 1 --2- 172.21.15.3:0/1514500189 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe74c09f240 0x7fe74c1a43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:57.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.336+0000 7fe750e5b700 1 --2- 172.21.15.3:0/1514500189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe74c09e930 0x7fe74c1a3e80 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe73c00e9b0 tx=0x7fe73c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:57.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.336+0000 7fe7497fa700 1 -- 172.21.15.3:0/1514500189 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73c00ee50 con 0x7fe74c09e930 2023-10-03T10:14:57.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.336+0000 7fe7497fa700 1 -- 172.21.15.3:0/1514500189 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73c004030 con 0x7fe74c09e930 2023-10-03T10:14:57.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.336+0000 7fe7497fa700 1 -- 172.21.15.3:0/1514500189 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73c0043c0 con 0x7fe74c09e930 2023-10-03T10:14:57.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.336+0000 7fe7530bf700 1 -- 172.21.15.3:0/1514500189 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe744009b30 con 0x7fe74c09e930 2023-10-03T10:14:57.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.336+0000 7fe7530bf700 1 -- 172.21.15.3:0/1514500189 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7340094a0 con 0x7fe74c09e930 2023-10-03T10:14:57.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.337+0000 7fe7497fa700 1 -- 172.21.15.3:0/1514500189 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe73c00c7f0 con 0x7fe74c09e930 2023-10-03T10:14:57.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.337+0000 7fe7497fa700 1 -- 172.21.15.3:0/1514500189 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe73c03f5b0 con 0x7fe74c09e930 2023-10-03T10:14:57.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.337+0000 7fe7530bf700 1 -- 172.21.15.3:0/1514500189 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe74c1a0b30 con 0x7fe74c09e930 2023-10-03T10:14:57.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.343+0000 7fe7497fa700 1 -- 172.21.15.3:0/1514500189 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe73c019d60 con 0x7fe74c09e930 2023-10-03T10:14:57.348 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:57.365 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:57.386 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:57.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.492+0000 7f88e2fac700 1 Processor -- start 2023-10-03T10:14:57.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.492+0000 7f88e2fac700 1 -- start start 2023-10-03T10:14:57.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.493+0000 7f88e2fac700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88dc09e1d0 0x7f88dc138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:57.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.493+0000 7f88e2fac700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88dc139170 0x7f88dc13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:57.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.493+0000 7f88e2fac700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88dc09b120 con 0x7f88dc09d6f0 2023-10-03T10:14:57.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.493+0000 7f88e2fac700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88dc06b460 con 0x7f88dc139170 2023-10-03T10:14:57.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.493+0000 7f88e2fac700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f88dc09a7e0 con 0x7f88dc09e1d0 2023-10-03T10:14:57.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.493+0000 7f88e1549700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88dc139170 0x7f88dc13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:57.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.493+0000 7f88e1549700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88dc139170 0x7f88dc13b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42586/0 (socket says 172.21.15.3:42586) 2023-10-03T10:14:57.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.493+0000 7f88e0d48700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f88dc09d6f0 0x7f88dc09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48262/0 (socket says 172.21.15.3:48262) 2023-10-03T10:14:57.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.493+0000 7f88e1549700 1 -- 172.21.15.3:0/1406381018 learned_addr learned my addr 172.21.15.3:0/1406381018 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:57.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.493+0000 7f88dbfff700 1 --2- 172.21.15.3:0/1406381018 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88dc09e1d0 0x7f88dc138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:57.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.494+0000 7f88e1549700 1 -- 172.21.15.3:0/1406381018 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88dc09e1d0 msgr2=0x7f88dc138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:57.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.494+0000 7f88e1549700 1 --2- 172.21.15.3:0/1406381018 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88dc09e1d0 0x7f88dc138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.494+0000 7f88e1549700 1 -- 172.21.15.3:0/1406381018 >> v1:172.21.15.3:6789/0 conn(0x7f88dc09d6f0 legacy=0x7f88dc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:57.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.494+0000 7f88e1549700 1 -- 172.21.15.3:0/1406381018 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88dc09a500 con 0x7f88dc139170 2023-10-03T10:14:57.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.494+0000 7f88dbfff700 1 --2- 172.21.15.3:0/1406381018 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88dc09e1d0 0x7f88dc138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:57.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.494+0000 7f88e1549700 1 --2- 172.21.15.3:0/1406381018 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88dc139170 0x7f88dc13b550 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f88d4009e80 tx=0x7f88d400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aa4f99c5f6d02c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:57.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.494+0000 7f88db7fe700 1 -- 172.21.15.3:0/1406381018 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d400e070 con 0x7f88dc139170 2023-10-03T10:14:57.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.494+0000 7f88db7fe700 1 -- 172.21.15.3:0/1406381018 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d400b910 con 0x7f88dc139170 2023-10-03T10:14:57.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.494+0000 7f88db7fe700 1 -- 172.21.15.3:0/1406381018 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d400bcc0 con 0x7f88dc139170 2023-10-03T10:14:57.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.495+0000 7f88e2fac700 1 -- 172.21.15.3:0/1406381018 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88dc139170 msgr2=0x7f88dc13b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:57.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.495+0000 7f88e2fac700 1 --2- 172.21.15.3:0/1406381018 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88dc139170 0x7f88dc13b550 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f88d4009e80 tx=0x7f88d400b330 comp rx=0 tx=0).stop 2023-10-03T10:14:57.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.495+0000 7f88e2fac700 1 -- 172.21.15.3:0/1406381018 shutdown_connections 2023-10-03T10:14:57.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.495+0000 7f88e2fac700 1 --2- 172.21.15.3:0/1406381018 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88dc139170 0x7f88dc13b550 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.495+0000 7f88e2fac700 1 --2- 172.21.15.3:0/1406381018 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88dc09e1d0 0x7f88dc138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.495+0000 7f88e2fac700 1 -- 172.21.15.3:0/1406381018 >> 172.21.15.3:0/1406381018 conn(0x7f88dc066f50 msgr2=0x7f88dc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:57.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.495+0000 7f88e2fac700 1 -- 172.21.15.3:0/1406381018 shutdown_connections 2023-10-03T10:14:57.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.496+0000 7f88e2fac700 1 -- 172.21.15.3:0/1406381018 wait complete. 2023-10-03T10:14:57.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.497+0000 7f88e2fac700 1 Processor -- start 2023-10-03T10:14:57.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.497+0000 7f88e2fac700 1 -- start start 2023-10-03T10:14:57.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.497+0000 7f88e2fac700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88dc09d6f0 0x7f88dc120790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:57.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.497+0000 7f88e2fac700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88dc09e1d0 0x7f88dc120cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:57.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.497+0000 7f88e2fac700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88dc139170 0x7f88dc121210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:57.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.497+0000 7f88e2fac700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88dc1360a0 con 0x7f88dc09d6f0 2023-10-03T10:14:57.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.497+0000 7f88e2fac700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88dc1242b0 con 0x7f88dc139170 2023-10-03T10:14:57.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.497+0000 7f88e2fac700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f88dc09c710 con 0x7f88dc09e1d0 2023-10-03T10:14:57.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.497+0000 7f88e1549700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88dc139170 0x7f88dc121210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:57.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.497+0000 7f88e1549700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88dc139170 0x7f88dc121210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42594/0 (socket says 172.21.15.3:42594) 2023-10-03T10:14:57.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.497+0000 7f88e1549700 1 -- 172.21.15.3:0/4178211985 learned_addr learned my addr 172.21.15.3:0/4178211985 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:57.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.498+0000 7f88dbfff700 1 --2- 172.21.15.3:0/4178211985 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88dc09e1d0 0x7f88dc120cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:57.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.498+0000 7f88e0d48700 1 --2- 172.21.15.3:0/4178211985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88dc09d6f0 0x7f88dc120790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:57.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.498+0000 7f88e1549700 1 -- 172.21.15.3:0/4178211985 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88dc09e1d0 msgr2=0x7f88dc120cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:57.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.498+0000 7f88e1549700 1 --2- 172.21.15.3:0/4178211985 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88dc09e1d0 0x7f88dc120cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.498+0000 7f88e1549700 1 -- 172.21.15.3:0/4178211985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88dc09d6f0 msgr2=0x7f88dc120790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:57.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.498+0000 7f88e1549700 1 --2- 172.21.15.3:0/4178211985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88dc09d6f0 0x7f88dc120790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.498+0000 7f88e1549700 1 -- 172.21.15.3:0/4178211985 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c80094a0 con 0x7f88dc139170 2023-10-03T10:14:57.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.498+0000 7f88e0d48700 1 --2- 172.21.15.3:0/4178211985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88dc09d6f0 0x7f88dc120790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:14:57.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.498+0000 7f88e1549700 1 --2- 172.21.15.3:0/4178211985 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88dc139170 0x7f88dc121210 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f88d4003f80 tx=0x7f88d4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:57.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.498+0000 7f88d97fa700 1 -- 172.21.15.3:0/4178211985 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d4004450 con 0x7f88dc139170 2023-10-03T10:14:57.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.498+0000 7f88d97fa700 1 -- 172.21.15.3:0/4178211985 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d40045b0 con 0x7f88dc139170 2023-10-03T10:14:57.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.498+0000 7f88e2fac700 1 -- 172.21.15.3:0/4178211985 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d4009b30 con 0x7f88dc139170 2023-10-03T10:14:57.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.498+0000 7f88d97fa700 1 -- 172.21.15.3:0/4178211985 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d4004920 con 0x7f88dc139170 2023-10-03T10:14:57.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.499+0000 7f88e2fac700 1 -- 172.21.15.3:0/4178211985 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88dc123d10 con 0x7f88dc139170 2023-10-03T10:14:57.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.500+0000 7f88d97fa700 1 -- 172.21.15.3:0/4178211985 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f88d401c410 con 0x7f88dc139170 2023-10-03T10:14:57.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.500+0000 7f88e2fac700 1 -- 172.21.15.3:0/4178211985 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88dc074ea0 con 0x7f88dc139170 2023-10-03T10:14:57.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.500+0000 7f88d97fa700 1 -- 172.21.15.3:0/4178211985 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f88d4041740 con 0x7f88dc139170 2023-10-03T10:14:57.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.505+0000 7f88d97fa700 1 -- 172.21.15.3:0/4178211985 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f88d402d7a0 con 0x7f88dc139170 2023-10-03T10:14:57.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.537+0000 7f366de90700 1 Processor -- start 2023-10-03T10:14:57.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.537+0000 7f366de90700 1 -- start start 2023-10-03T10:14:57.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.538+0000 7f366de90700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f366809e930 0x7f366809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:57.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.538+0000 7f366de90700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f366809f240 0x7f36681317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:57.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.538+0000 7f366de90700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3668070940 con 0x7f366809f650 2023-10-03T10:14:57.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.538+0000 7f366de90700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f366806fd60 con 0x7f366809f240 2023-10-03T10:14:57.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.538+0000 7f366de90700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f366806fa80 con 0x7f366809e930 2023-10-03T10:14:57.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.538+0000 7f3666ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f366809f240 0x7f36681317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:57.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.538+0000 7f36677fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f366809e930 0x7f366809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:57.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.538+0000 7f3666ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f366809f240 0x7f36681317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41146/0 (socket says 172.21.15.3:41146) 2023-10-03T10:14:57.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.538+0000 7f3666ffd700 1 -- 172.21.15.3:0/1317564577 learned_addr learned my addr 172.21.15.3:0/1317564577 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:57.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.538+0000 7f3666ffd700 1 -- 172.21.15.3:0/1317564577 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f366809e930 msgr2=0x7f366809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:57.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.538+0000 7f3666ffd700 1 --2- 172.21.15.3:0/1317564577 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f366809e930 0x7f366809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.538+0000 7f3666ffd700 1 -- 172.21.15.3:0/1317564577 >> v1:172.21.15.3:6789/0 conn(0x7f366809f650 legacy=0x7f3668131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:57.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.539+0000 7f3666ffd700 1 -- 172.21.15.3:0/1317564577 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f366809fb60 con 0x7f366809f240 2023-10-03T10:14:57.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.538+0000 7f36667fc700 1 -- 172.21.15.3:0/1317564577 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1223961294 0 0) 0x7f3668070940 con 0x7f366809f650 2023-10-03T10:14:57.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.539+0000 7f3666ffd700 1 --2- 172.21.15.3:0/1317564577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f366809f240 0x7f36681317f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f365c0098f0 tx=0x7f365c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d7270cd12daf63 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:57.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.539+0000 7f36667fc700 1 -- 172.21.15.3:0/1317564577 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c011830 con 0x7f366809f240 2023-10-03T10:14:57.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.539+0000 7f36667fc700 1 -- 172.21.15.3:0/1317564577 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365c011990 con 0x7f366809f240 2023-10-03T10:14:57.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.539+0000 7f36667fc700 1 -- 172.21.15.3:0/1317564577 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c011d20 con 0x7f366809f240 2023-10-03T10:14:57.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.540+0000 7f366de90700 1 -- 172.21.15.3:0/1317564577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f366809f240 msgr2=0x7f36681317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:57.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.540+0000 7f366de90700 1 --2- 172.21.15.3:0/1317564577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f366809f240 0x7f36681317f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f365c0098f0 tx=0x7f365c009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:57.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.540+0000 7f366de90700 1 -- 172.21.15.3:0/1317564577 shutdown_connections 2023-10-03T10:14:57.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.540+0000 7f366de90700 1 --2- 172.21.15.3:0/1317564577 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f366809e930 0x7f366809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.540+0000 7f366de90700 1 --2- 172.21.15.3:0/1317564577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f366809f240 0x7f36681317f0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.540+0000 7f366de90700 1 -- 172.21.15.3:0/1317564577 >> 172.21.15.3:0/1317564577 conn(0x7f36680a4210 msgr2=0x7f36680a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:57.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.540+0000 7f366de90700 1 -- 172.21.15.3:0/1317564577 shutdown_connections 2023-10-03T10:14:57.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.540+0000 7f366de90700 1 -- 172.21.15.3:0/1317564577 wait complete. 2023-10-03T10:14:57.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.542+0000 7f366de90700 1 Processor -- start 2023-10-03T10:14:57.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.542+0000 7f9574ace700 1 Processor -- start 2023-10-03T10:14:57.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.542+0000 7f9574ace700 1 -- start start 2023-10-03T10:14:57.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.542+0000 7f366de90700 1 -- start start 2023-10-03T10:14:57.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.542+0000 7f9574ace700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f957009d6f0 0x7f957009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:57.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.542+0000 7f9574ace700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f957009e000 0x7f95701339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:57.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.543+0000 7f9574ace700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f957009b120 con 0x7f957009e410 2023-10-03T10:14:57.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.543+0000 7f9574ace700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f957006b460 con 0x7f957009e000 2023-10-03T10:14:57.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.543+0000 7f9574ace700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f957009a7e0 con 0x7f957009d6f0 2023-10-03T10:14:57.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.543+0000 7f956ed9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f957009e410 0x7f9570133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48270/0 (socket says 172.21.15.3:48270) 2023-10-03T10:14:57.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.543+0000 7f956ed9d700 1 -- 172.21.15.3:0/1022141355 learned_addr learned my addr 172.21.15.3:0/1022141355 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:14:57.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.543+0000 7f956e59c700 1 --2- 172.21.15.3:0/1022141355 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f957009d6f0 0x7f957009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:57.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.543+0000 7f956d59a700 1 -- 172.21.15.3:0/1022141355 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1289959256 0 0) 0x7f957009b120 con 0x7f957009e410 2023-10-03T10:14:57.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.543+0000 7f956d59a700 1 -- 172.21.15.3:0/1022141355 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95580035b0 con 0x7f957009e410 2023-10-03T10:14:57.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.543+0000 7f956e59c700 1 -- 172.21.15.3:0/1022141355 >> v2:172.21.15.3:3300/0 conn(0x7f957009e000 msgr2=0x7f95701339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:57.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.543+0000 7f956e59c700 1 --2- 172.21.15.3:0/1022141355 >> v2:172.21.15.3:3300/0 conn(0x7f957009e000 0x7f95701339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.543+0000 7f956e59c700 1 -- 172.21.15.3:0/1022141355 >> v1:172.21.15.3:6789/0 conn(0x7f957009e410 legacy=0x7f9570133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:57.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.543+0000 7f956e59c700 1 -- 172.21.15.3:0/1022141355 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957009a500 con 0x7f957009d6f0 2023-10-03T10:14:57.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.543+0000 7f366de90700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f366809e930 0x7f366806b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:57.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.543+0000 7f366de90700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f366809f240 0x7f366806bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:57.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.543+0000 7f366de90700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f366809f650 0x7f366806c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:57.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.543+0000 7f366de90700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36681a8770 con 0x7f366809e930 2023-10-03T10:14:57.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.543+0000 7f366de90700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f366806d4c0 con 0x7f366809f650 2023-10-03T10:14:57.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.543+0000 7f366de90700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36681a9210 con 0x7f366809f240 2023-10-03T10:14:57.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.543+0000 7f3667fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f366809f650 0x7f366806c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:57.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.543+0000 7f3667fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f366809f650 0x7f366806c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42608/0 (socket says 172.21.15.3:42608) 2023-10-03T10:14:57.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.544+0000 7f956e59c700 1 --2- 172.21.15.3:0/1022141355 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f957009d6f0 0x7f957009dac0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9560009870 tx=0x7f9560009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a68c5c4277cb6ac0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:57.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.543+0000 7f3667fff700 1 -- 172.21.15.3:0/345303451 learned_addr learned my addr 172.21.15.3:0/345303451 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:57.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.543+0000 7f3666ffd700 1 --2- 172.21.15.3:0/345303451 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f366809f240 0x7f366806bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:57.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.543+0000 7f3667fff700 1 -- 172.21.15.3:0/345303451 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f366809f240 msgr2=0x7f366806bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:57.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.543+0000 7f3667fff700 1 --2- 172.21.15.3:0/345303451 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f366809f240 0x7f366806bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.543+0000 7f3667fff700 1 -- 172.21.15.3:0/345303451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f366809e930 msgr2=0x7f366806b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:57.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.543+0000 7f3667fff700 1 --2- 172.21.15.3:0/345303451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f366809e930 0x7f366806b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.543+0000 7f3667fff700 1 -- 172.21.15.3:0/345303451 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365c009550 con 0x7f366809f650 2023-10-03T10:14:57.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.544+0000 7f3667fff700 1 --2- 172.21.15.3:0/345303451 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f366809f650 0x7f366806c330 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3660001560 tx=0x7f366000cc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:57.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.544+0000 7f364ffff700 1 -- 172.21.15.3:0/345303451 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3660003e30 con 0x7f366809f650 2023-10-03T10:14:57.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.544+0000 7f364ffff700 1 -- 172.21.15.3:0/345303451 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3660005070 con 0x7f366809f650 2023-10-03T10:14:57.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.544+0000 7f956d59a700 1 -- 172.21.15.3:0/1022141355 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9560011830 con 0x7f957009d6f0 2023-10-03T10:14:57.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.544+0000 7f364ffff700 1 -- 172.21.15.3:0/345303451 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3660005420 con 0x7f366809f650 2023-10-03T10:14:57.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.544+0000 7f956d59a700 1 -- 172.21.15.3:0/1022141355 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9560011990 con 0x7f957009d6f0 2023-10-03T10:14:57.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.544+0000 7f956d59a700 1 -- 172.21.15.3:0/1022141355 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9560011d00 con 0x7f957009d6f0 2023-10-03T10:14:57.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.544+0000 7f366de90700 1 -- 172.21.15.3:0/345303451 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36680a0190 con 0x7f366809f650 2023-10-03T10:14:57.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.544+0000 7f366de90700 1 -- 172.21.15.3:0/345303451 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36680a3260 con 0x7f366809f650 2023-10-03T10:14:57.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.545+0000 7f364ffff700 1 -- 172.21.15.3:0/345303451 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3660005580 con 0x7f366809f650 2023-10-03T10:14:57.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.545+0000 7f366de90700 1 -- 172.21.15.3:0/345303451 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36681a0b30 con 0x7f366809f650 2023-10-03T10:14:57.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.546+0000 7f364ffff700 1 -- 172.21.15.3:0/345303451 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f366003ef50 con 0x7f366809f650 2023-10-03T10:14:57.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.546+0000 7f9574ace700 1 -- 172.21.15.3:0/1022141355 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f957009d6f0 msgr2=0x7f957009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:57.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.546+0000 7f9574ace700 1 --2- 172.21.15.3:0/1022141355 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f957009d6f0 0x7f957009dac0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9560009870 tx=0x7f9560009b80 comp rx=0 tx=0).stop 2023-10-03T10:14:57.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.547+0000 7f9574ace700 1 -- 172.21.15.3:0/1022141355 shutdown_connections 2023-10-03T10:14:57.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.547+0000 7f9574ace700 1 --2- 172.21.15.3:0/1022141355 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f957009d6f0 0x7f957009dac0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.547+0000 7f9574ace700 1 --2- 172.21.15.3:0/1022141355 >> v2:172.21.15.3:3300/0 conn(0x7f957009e000 0x7f95701339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.547+0000 7f9574ace700 1 -- 172.21.15.3:0/1022141355 >> 172.21.15.3:0/1022141355 conn(0x7f9570066f50 msgr2=0x7f957009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:57.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.547+0000 7f9574ace700 1 -- 172.21.15.3:0/1022141355 shutdown_connections 2023-10-03T10:14:57.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.547+0000 7f9574ace700 1 -- 172.21.15.3:0/1022141355 wait complete. 2023-10-03T10:14:57.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.548+0000 7f9574ace700 1 Processor -- start 2023-10-03T10:14:57.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.549+0000 7f9574ace700 1 -- start start 2023-10-03T10:14:57.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.549+0000 7f9574ace700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f957009d6f0 0x7f95701c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:57.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.549+0000 7f9574ace700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f957009e000 0x7f95701c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:57.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.549+0000 7f9574ace700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f957009e410 0x7f95701a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:57.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.549+0000 7f9574ace700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95701b81b0 con 0x7f957009d6f0 2023-10-03T10:14:57.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.549+0000 7f9574ace700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f957009c710 con 0x7f957009e410 2023-10-03T10:14:57.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.549+0000 7f9574ace700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95701a6b50 con 0x7f957009e000 2023-10-03T10:14:57.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.549+0000 7f956e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f957009d6f0 0x7f95701c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:57.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.550+0000 7f956e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f957009d6f0 0x7f95701c8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41174/0 (socket says 172.21.15.3:41174) 2023-10-03T10:14:57.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.550+0000 7f956e59c700 1 -- 172.21.15.3:0/2352817944 learned_addr learned my addr 172.21.15.3:0/2352817944 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:57.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.550+0000 7f956dd9b700 1 --2- 172.21.15.3:0/2352817944 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f957009e000 0x7f95701c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:57.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.550+0000 7f956ed9d700 1 --2- 172.21.15.3:0/2352817944 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f957009e410 0x7f95701a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:57.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.550+0000 7f956e59c700 1 -- 172.21.15.3:0/2352817944 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f957009e000 msgr2=0x7f95701c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:57.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.550+0000 7f956e59c700 1 --2- 172.21.15.3:0/2352817944 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f957009e000 0x7f95701c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.550+0000 7f956e59c700 1 -- 172.21.15.3:0/2352817944 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f957009e410 msgr2=0x7f95701a3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:57.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.550+0000 7f956e59c700 1 --2- 172.21.15.3:0/2352817944 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f957009e410 0x7f95701a3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.550+0000 7f956e59c700 1 -- 172.21.15.3:0/2352817944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95600094a0 con 0x7f957009d6f0 2023-10-03T10:14:57.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.550+0000 7f956ed9d700 1 --2- 172.21.15.3:0/2352817944 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f957009e410 0x7f95701a3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:57.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.550+0000 7f956e59c700 1 --2- 172.21.15.3:0/2352817944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f957009d6f0 0x7f95701c8580 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f95600090b0 tx=0x7f9560004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:57.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.550+0000 7f955effd700 1 -- 172.21.15.3:0/2352817944 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9560004550 con 0x7f957009d6f0 2023-10-03T10:14:57.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.551+0000 7f955effd700 1 -- 172.21.15.3:0/2352817944 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956001f070 con 0x7f957009d6f0 2023-10-03T10:14:57.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.551+0000 7f955effd700 1 -- 172.21.15.3:0/2352817944 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95600038e0 con 0x7f957009d6f0 2023-10-03T10:14:57.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.551+0000 7f9574ace700 1 -- 172.21.15.3:0/2352817944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95701b8540 con 0x7f957009d6f0 2023-10-03T10:14:57.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.551+0000 7f364ffff700 1 -- 172.21.15.3:0/345303451 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3660010d40 con 0x7f366809f650 2023-10-03T10:14:57.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.551+0000 7f9574ace700 1 -- 172.21.15.3:0/2352817944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95701a7180 con 0x7f957009d6f0 2023-10-03T10:14:57.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.552+0000 7f955effd700 1 -- 172.21.15.3:0/2352817944 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9560003a40 con 0x7f957009d6f0 2023-10-03T10:14:57.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.552+0000 7f9574ace700 1 -- 172.21.15.3:0/2352817944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95701a6d70 con 0x7f957009d6f0 2023-10-03T10:14:57.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.552+0000 7f955effd700 1 -- 172.21.15.3:0/2352817944 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9560021070 con 0x7f957009d6f0 2023-10-03T10:14:57.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.557+0000 7f955effd700 1 -- 172.21.15.3:0/2352817944 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9560046020 con 0x7f957009d6f0 2023-10-03T10:14:57.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.561+0000 7fe7530bf700 1 -- 172.21.15.3:0/1514500189 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe74c0a3550 con 0x7fe74c09e930 2023-10-03T10:14:57.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.562+0000 7fe7497fa700 1 -- 172.21.15.3:0/1514500189 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe73c029090 con 0x7fe74c09e930 2023-10-03T10:14:57.562 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:57.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.564+0000 7fe7530bf700 1 -- 172.21.15.3:0/1514500189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe74c09e930 msgr2=0x7fe74c1a3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:57.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.564+0000 7fe7530bf700 1 --2- 172.21.15.3:0/1514500189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe74c09e930 0x7fe74c1a3e80 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe73c00e9b0 tx=0x7fe73c00ecc0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.564+0000 7fe7530bf700 1 -- 172.21.15.3:0/1514500189 shutdown_connections 2023-10-03T10:14:57.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.564+0000 7fe7530bf700 1 --2- 172.21.15.3:0/1514500189 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe74c09f650 0x7fe74c1a4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.564+0000 7fe7530bf700 1 --2- 172.21.15.3:0/1514500189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe74c09e930 0x7fe74c1a3e80 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.564+0000 7fe7530bf700 1 --2- 172.21.15.3:0/1514500189 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe74c09f240 0x7fe74c1a43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.564+0000 7fe7530bf700 1 -- 172.21.15.3:0/1514500189 >> 172.21.15.3:0/1514500189 conn(0x7fe74c0a4210 msgr2=0x7fe74c1b8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:57.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.565+0000 7fe7530bf700 1 -- 172.21.15.3:0/1514500189 shutdown_connections 2023-10-03T10:14:57.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.565+0000 7fe7530bf700 1 -- 172.21.15.3:0/1514500189 wait complete. 2023-10-03T10:14:57.580 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:57.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.720+0000 7f88e2fac700 1 -- 172.21.15.3:0/4178211985 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f88dc122b90 con 0x7f88dc139170 2023-10-03T10:14:57.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.721+0000 7f88d97fa700 1 -- 172.21.15.3:0/4178211985 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f88d401b3a0 con 0x7f88dc139170 2023-10-03T10:14:57.721 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:57.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.723+0000 7f88e2fac700 1 -- 172.21.15.3:0/4178211985 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88dc139170 msgr2=0x7f88dc121210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:57.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.723+0000 7f88e2fac700 1 --2- 172.21.15.3:0/4178211985 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88dc139170 0x7f88dc121210 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f88d4003f80 tx=0x7f88d4004ad0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.723+0000 7f88e2fac700 1 -- 172.21.15.3:0/4178211985 shutdown_connections 2023-10-03T10:14:57.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.723+0000 7f88e2fac700 1 --2- 172.21.15.3:0/4178211985 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88dc139170 0x7f88dc121210 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.723+0000 7f88e2fac700 1 --2- 172.21.15.3:0/4178211985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88dc09d6f0 0x7f88dc120790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.723+0000 7f88e2fac700 1 --2- 172.21.15.3:0/4178211985 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88dc09e1d0 0x7f88dc120cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.723+0000 7f88e2fac700 1 -- 172.21.15.3:0/4178211985 >> 172.21.15.3:0/4178211985 conn(0x7f88dc066f50 msgr2=0x7f88dc136a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:57.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.723+0000 7f88e2fac700 1 -- 172.21.15.3:0/4178211985 shutdown_connections 2023-10-03T10:14:57.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.723+0000 7f88e2fac700 1 -- 172.21.15.3:0/4178211985 wait complete. 2023-10-03T10:14:57.737 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:57.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.759+0000 7f366de90700 1 -- 172.21.15.3:0/345303451 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3668074ea0 con 0x7f366809f650 2023-10-03T10:14:57.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.760+0000 7f364ffff700 1 -- 172.21.15.3:0/345303451 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3660010300 con 0x7f366809f650 2023-10-03T10:14:57.760 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:57.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.762+0000 7f366de90700 1 -- 172.21.15.3:0/345303451 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f366809f650 msgr2=0x7f366806c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:57.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.762+0000 7f366de90700 1 --2- 172.21.15.3:0/345303451 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f366809f650 0x7f366806c330 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3660001560 tx=0x7f366000cc00 comp rx=0 tx=0).stop 2023-10-03T10:14:57.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.762+0000 7f366de90700 1 -- 172.21.15.3:0/345303451 shutdown_connections 2023-10-03T10:14:57.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.762+0000 7f366de90700 1 --2- 172.21.15.3:0/345303451 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f366809f650 0x7f366806c330 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.762+0000 7f366de90700 1 --2- 172.21.15.3:0/345303451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f366809e930 0x7f366806b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.762+0000 7f366de90700 1 --2- 172.21.15.3:0/345303451 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f366809f240 0x7f366806bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.762+0000 7f366de90700 1 -- 172.21.15.3:0/345303451 >> 172.21.15.3:0/345303451 conn(0x7f36680a4210 msgr2=0x7f36681b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:57.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.762+0000 7f366de90700 1 -- 172.21.15.3:0/345303451 shutdown_connections 2023-10-03T10:14:57.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.762+0000 7f366de90700 1 -- 172.21.15.3:0/345303451 wait complete. 2023-10-03T10:14:57.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.764+0000 7f9574ace700 1 -- 172.21.15.3:0/2352817944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9570074ea0 con 0x7f957009d6f0 2023-10-03T10:14:57.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.764+0000 7f955effd700 1 -- 172.21.15.3:0/2352817944 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f956002b030 con 0x7f957009d6f0 2023-10-03T10:14:57.765 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:57.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.767+0000 7f9574ace700 1 -- 172.21.15.3:0/2352817944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f957009d6f0 msgr2=0x7f95701c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:57.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.767+0000 7f9574ace700 1 --2- 172.21.15.3:0/2352817944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f957009d6f0 0x7f95701c8580 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f95600090b0 tx=0x7f9560004750 comp rx=0 tx=0).stop 2023-10-03T10:14:57.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.767+0000 7f9574ace700 1 -- 172.21.15.3:0/2352817944 shutdown_connections 2023-10-03T10:14:57.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.767+0000 7f9574ace700 1 --2- 172.21.15.3:0/2352817944 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f957009e410 0x7f95701a3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.767+0000 7f9574ace700 1 --2- 172.21.15.3:0/2352817944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f957009d6f0 0x7f95701c8580 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.767+0000 7f9574ace700 1 --2- 172.21.15.3:0/2352817944 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f957009e000 0x7f95701c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:57.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.767+0000 7f9574ace700 1 -- 172.21.15.3:0/2352817944 >> 172.21.15.3:0/2352817944 conn(0x7f9570066f50 msgr2=0x7f957006f270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:57.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.767+0000 7f9574ace700 1 -- 172.21.15.3:0/2352817944 shutdown_connections 2023-10-03T10:14:57.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:57.767+0000 7f9574ace700 1 -- 172.21.15.3:0/2352817944 wait complete. 2023-10-03T10:14:57.778 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:57.782 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:58.581 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:14:58.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.726+0000 7f7aeac30700 1 Processor -- start 2023-10-03T10:14:58.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.726+0000 7f7aeac30700 1 -- start start 2023-10-03T10:14:58.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.727+0000 7f7aeac30700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7ae4133420 0x7f7ae41337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:58.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.727+0000 7f7aeac30700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ae4134260 0x7f7ae41306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:58.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.727+0000 7f7aeac30700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ae406b990 con 0x7f7ae4133e30 2023-10-03T10:14:58.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.727+0000 7f7aeac30700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae40a0590 con 0x7f7ae4133420 2023-10-03T10:14:58.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.727+0000 7f7aeac30700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae409fb80 con 0x7f7ae4134260 2023-10-03T10:14:58.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.727+0000 7f7ae89cc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7ae4133420 0x7f7ae41337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:58.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.727+0000 7f7ae91cd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ae4134260 0x7f7ae41306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:58.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.727+0000 7f7ae89cc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7ae4133420 0x7f7ae41337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41184/0 (socket says 172.21.15.3:41184) 2023-10-03T10:14:58.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.727+0000 7f7ae91cd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ae4134260 0x7f7ae41306a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42630/0 (socket says 172.21.15.3:42630) 2023-10-03T10:14:58.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.727+0000 7f7ae89cc700 1 -- 172.21.15.3:0/3664328937 learned_addr learned my addr 172.21.15.3:0/3664328937 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:58.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.728+0000 7f7ae91cd700 1 -- 172.21.15.3:0/3664328937 >> v2:172.21.15.3:3300/0 conn(0x7f7ae4133420 msgr2=0x7f7ae41337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:58.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.728+0000 7f7ae91cd700 1 --2- 172.21.15.3:0/3664328937 >> v2:172.21.15.3:3300/0 conn(0x7f7ae4133420 0x7f7ae41337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:58.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.728+0000 7f7ae91cd700 1 -- 172.21.15.3:0/3664328937 >> v1:172.21.15.3:6789/0 conn(0x7f7ae4133e30 legacy=0x7f7ae412ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:58.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.728+0000 7f7ae91cd700 1 -- 172.21.15.3:0/3664328937 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae40a0030 con 0x7f7ae4134260 2023-10-03T10:14:58.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.728+0000 7f7ae89cc700 1 --2- 172.21.15.3:0/3664328937 >> v2:172.21.15.3:3300/0 conn(0x7f7ae4133420 0x7f7ae41337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:14:58.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.728+0000 7f7ae91cd700 1 --2- 172.21.15.3:0/3664328937 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ae4134260 0x7f7ae41306a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009d90 tx=0x7f7adc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=214cb2e2dfb6426d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:58.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.728+0000 7f7ae37fe700 1 -- 172.21.15.3:0/3664328937 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc00e070 con 0x7f7ae4134260 2023-10-03T10:14:58.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.728+0000 7f7ae37fe700 1 -- 172.21.15.3:0/3664328937 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7adc00bb20 con 0x7f7ae4134260 2023-10-03T10:14:58.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.728+0000 7f7ae37fe700 1 -- 172.21.15.3:0/3664328937 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc012430 con 0x7f7ae4134260 2023-10-03T10:14:58.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.728+0000 7f7aeac30700 1 -- 172.21.15.3:0/3664328937 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ae4134260 msgr2=0x7f7ae41306a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:58.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.728+0000 7f7aeac30700 1 --2- 172.21.15.3:0/3664328937 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ae4134260 0x7f7ae41306a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009d90 tx=0x7f7adc00b330 comp rx=0 tx=0).stop 2023-10-03T10:14:58.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.729+0000 7f7aeac30700 1 -- 172.21.15.3:0/3664328937 shutdown_connections 2023-10-03T10:14:58.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.729+0000 7f7aeac30700 1 --2- 172.21.15.3:0/3664328937 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ae4134260 0x7f7ae41306a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:58.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.729+0000 7f7aeac30700 1 --2- 172.21.15.3:0/3664328937 >> v2:172.21.15.3:3300/0 conn(0x7f7ae4133420 0x7f7ae41337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:58.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.729+0000 7f7aeac30700 1 -- 172.21.15.3:0/3664328937 >> 172.21.15.3:0/3664328937 conn(0x7f7ae40a4210 msgr2=0x7f7ae40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:58.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.729+0000 7f7aeac30700 1 -- 172.21.15.3:0/3664328937 shutdown_connections 2023-10-03T10:14:58.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.729+0000 7f7aeac30700 1 -- 172.21.15.3:0/3664328937 wait complete. 2023-10-03T10:14:58.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.730+0000 7f7aeac30700 1 Processor -- start 2023-10-03T10:14:58.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.730+0000 7f7aeac30700 1 -- start start 2023-10-03T10:14:58.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.730+0000 7f7aeac30700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ae4133420 0x7f7ae409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:58.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.730+0000 7f7aeac30700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ae4133e30 0x7f7ae409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:58.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.730+0000 7f7aeac30700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ae4134260 0x7f7ae409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:58.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.730+0000 7f7aeac30700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae411fc20 con 0x7f7ae4134260 2023-10-03T10:14:58.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.730+0000 7f7aeac30700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae411fee0 con 0x7f7ae4133420 2023-10-03T10:14:58.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.730+0000 7f7aeac30700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae411f910 con 0x7f7ae4133e30 2023-10-03T10:14:58.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.730+0000 7f7ae89cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ae4133420 0x7f7ae409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:58.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.730+0000 7f7ae89cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ae4133420 0x7f7ae409c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42632/0 (socket says 172.21.15.3:42632) 2023-10-03T10:14:58.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.730+0000 7f7ae89cc700 1 -- 172.21.15.3:0/1878019684 learned_addr learned my addr 172.21.15.3:0/1878019684 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:58.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.730+0000 7f7ae3fff700 1 --2- 172.21.15.3:0/1878019684 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ae4133e30 0x7f7ae409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:58.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.730+0000 7f7ae91cd700 1 --2- 172.21.15.3:0/1878019684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ae4134260 0x7f7ae409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:58.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.731+0000 7f7ae3fff700 1 -- 172.21.15.3:0/1878019684 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ae4133420 msgr2=0x7f7ae409c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:58.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.731+0000 7f7ae3fff700 1 --2- 172.21.15.3:0/1878019684 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ae4133420 0x7f7ae409c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:58.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.731+0000 7f7ae3fff700 1 -- 172.21.15.3:0/1878019684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ae4134260 msgr2=0x7f7ae409cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:58.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.731+0000 7f7ae3fff700 1 --2- 172.21.15.3:0/1878019684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ae4134260 0x7f7ae409cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:58.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.731+0000 7f7ae3fff700 1 -- 172.21.15.3:0/1878019684 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad4009550 con 0x7f7ae4133e30 2023-10-03T10:14:58.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.731+0000 7f7ae3fff700 1 --2- 172.21.15.3:0/1878019684 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ae4133e30 0x7f7ae409c9c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8000f60 tx=0x7f7ad800c730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:58.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.731+0000 7f7ae17fa700 1 -- 172.21.15.3:0/1878019684 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad80052d0 con 0x7f7ae4133e30 2023-10-03T10:14:58.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.731+0000 7f7ae17fa700 1 -- 172.21.15.3:0/1878019684 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad8005430 con 0x7f7ae4133e30 2023-10-03T10:14:58.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.731+0000 7f7aeac30700 1 -- 172.21.15.3:0/1878019684 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7adc009a70 con 0x7f7ae4133e30 2023-10-03T10:14:58.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.731+0000 7f7ae17fa700 1 -- 172.21.15.3:0/1878019684 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad80038c0 con 0x7f7ae4133e30 2023-10-03T10:14:58.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.731+0000 7f7aeac30700 1 -- 172.21.15.3:0/1878019684 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae41201c0 con 0x7f7ae4133e30 2023-10-03T10:14:58.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.732+0000 7f7ae17fa700 1 -- 172.21.15.3:0/1878019684 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7ad8003a20 con 0x7f7ae4133e30 2023-10-03T10:14:58.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.732+0000 7f7ae17fa700 1 -- 172.21.15.3:0/1878019684 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7ad803e9d0 con 0x7f7ae4133e30 2023-10-03T10:14:58.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.732+0000 7f7aeac30700 1 -- 172.21.15.3:0/1878019684 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae4074ea0 con 0x7f7ae4133e30 2023-10-03T10:14:58.742 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:14:58.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.737+0000 7f7ae17fa700 1 -- 172.21.15.3:0/1878019684 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7ad8017080 con 0x7f7ae4133e30 2023-10-03T10:14:58.778 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:14:58.783 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:14:58.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.885+0000 7fa9ec8b9700 1 Processor -- start 2023-10-03T10:14:58.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.885+0000 7fa9ec8b9700 1 -- start start 2023-10-03T10:14:58.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.886+0000 7fa9ec8b9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9e8131810 0x7fa9e8131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:58.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.886+0000 7fa9ec8b9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9e8099a40 0x7fa9e8099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:58.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.886+0000 7fa9ec8b9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9e81313c0 con 0x7fa9e809a3b0 2023-10-03T10:14:58.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.886+0000 7fa9ec8b9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e809b580 con 0x7fa9e8099a40 2023-10-03T10:14:58.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.886+0000 7fa9ec8b9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e806e8c0 con 0x7fa9e8131810 2023-10-03T10:14:58.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.886+0000 7fa9e5d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9e8099a40 0x7fa9e8099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:58.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.886+0000 7fa9e659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9e8131810 0x7fa9e8131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:58.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.886+0000 7fa9e5d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9e8099a40 0x7fa9e8099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42638/0 (socket says 172.21.15.3:42638) 2023-10-03T10:14:58.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.886+0000 7fa9e5d9b700 1 -- 172.21.15.3:0/1646120838 learned_addr learned my addr 172.21.15.3:0/1646120838 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:58.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.886+0000 7fa9e5d9b700 1 -- 172.21.15.3:0/1646120838 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9e8131810 msgr2=0x7fa9e8131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:58.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.886+0000 7fa9e5d9b700 1 --2- 172.21.15.3:0/1646120838 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9e8131810 0x7fa9e8131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:58.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.886+0000 7fa9e5d9b700 1 -- 172.21.15.3:0/1646120838 >> v1:172.21.15.3:6789/0 conn(0x7fa9e809a3b0 legacy=0x7fa9e8138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:58.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.886+0000 7fa9e5d9b700 1 -- 172.21.15.3:0/1646120838 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e809a820 con 0x7fa9e8099a40 2023-10-03T10:14:58.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.887+0000 7fa9e5d9b700 1 --2- 172.21.15.3:0/1646120838 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9e8099a40 0x7fa9e8099e70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009920 tx=0x7fa9dc009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=131f50cb0d6590f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:58.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.887+0000 7fa9e559a700 1 -- 172.21.15.3:0/1646120838 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc00ba60 con 0x7fa9e8099a40 2023-10-03T10:14:58.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.887+0000 7fa9e559a700 1 -- 172.21.15.3:0/1646120838 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9dc00bbc0 con 0x7fa9e8099a40 2023-10-03T10:14:58.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.887+0000 7fa9e559a700 1 -- 172.21.15.3:0/1646120838 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc009e70 con 0x7fa9e8099a40 2023-10-03T10:14:58.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.888+0000 7fa9ec8b9700 1 -- 172.21.15.3:0/1646120838 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9e8099a40 msgr2=0x7fa9e8099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:58.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.888+0000 7fa9ec8b9700 1 --2- 172.21.15.3:0/1646120838 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9e8099a40 0x7fa9e8099e70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009920 tx=0x7fa9dc009c30 comp rx=0 tx=0).stop 2023-10-03T10:14:58.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.888+0000 7fa9ec8b9700 1 -- 172.21.15.3:0/1646120838 shutdown_connections 2023-10-03T10:14:58.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.888+0000 7fa9ec8b9700 1 --2- 172.21.15.3:0/1646120838 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9e8131810 0x7fa9e8131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:58.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.888+0000 7fa9ec8b9700 1 --2- 172.21.15.3:0/1646120838 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9e8099a40 0x7fa9e8099e70 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:58.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.888+0000 7fa9ec8b9700 1 -- 172.21.15.3:0/1646120838 >> 172.21.15.3:0/1646120838 conn(0x7fa9e80a2aa0 msgr2=0x7fa9e80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:58.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.888+0000 7fa9ec8b9700 1 -- 172.21.15.3:0/1646120838 shutdown_connections 2023-10-03T10:14:58.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.888+0000 7fa9ec8b9700 1 -- 172.21.15.3:0/1646120838 wait complete. 2023-10-03T10:14:58.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.890+0000 7fa9ec8b9700 1 Processor -- start 2023-10-03T10:14:58.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.890+0000 7fa9ec8b9700 1 -- start start 2023-10-03T10:14:58.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.890+0000 7fa9ec8b9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9e8099a40 0x7fa9e806ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:58.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.890+0000 7fa9ec8b9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9e809a3b0 0x7fa9e8070440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:58.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.891+0000 7fa9ec8b9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9e8131810 0x7fa9e81c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:58.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.891+0000 7fa9ec8b9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e8130a70 con 0x7fa9e8131810 2023-10-03T10:14:58.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.891+0000 7fa9ec8b9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e812ba80 con 0x7fa9e8099a40 2023-10-03T10:14:58.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.891+0000 7fa9ec8b9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e811e3f0 con 0x7fa9e809a3b0 2023-10-03T10:14:58.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.891+0000 7fa9e659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9e8099a40 0x7fa9e806ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:58.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.891+0000 7fa9e659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9e8099a40 0x7fa9e806ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42642/0 (socket says 172.21.15.3:42642) 2023-10-03T10:14:58.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.891+0000 7fa9e659c700 1 -- 172.21.15.3:0/2986981157 learned_addr learned my addr 172.21.15.3:0/2986981157 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:58.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.891+0000 7fa9e5d9b700 1 --2- 172.21.15.3:0/2986981157 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9e809a3b0 0x7fa9e8070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:58.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.891+0000 7fa9e6d9d700 1 --2- 172.21.15.3:0/2986981157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9e8131810 0x7fa9e81c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:58.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.891+0000 7fa9e5d9b700 1 -- 172.21.15.3:0/2986981157 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9e8099a40 msgr2=0x7fa9e806ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:58.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.891+0000 7fa9e5d9b700 1 --2- 172.21.15.3:0/2986981157 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9e8099a40 0x7fa9e806ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:58.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.891+0000 7fa9e5d9b700 1 -- 172.21.15.3:0/2986981157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9e8131810 msgr2=0x7fa9e81c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:58.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.891+0000 7fa9e5d9b700 1 --2- 172.21.15.3:0/2986981157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9e8131810 0x7fa9e81c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:58.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.891+0000 7fa9e5d9b700 1 -- 172.21.15.3:0/2986981157 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9dc009550 con 0x7fa9e809a3b0 2023-10-03T10:14:58.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.892+0000 7fa9e5d9b700 1 --2- 172.21.15.3:0/2986981157 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9e809a3b0 0x7fa9e8070440 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc016000 tx=0x7fa9dc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:58.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.892+0000 7fa9d6ffd700 1 -- 172.21.15.3:0/2986981157 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc004300 con 0x7fa9e809a3b0 2023-10-03T10:14:58.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.892+0000 7fa9d6ffd700 1 -- 172.21.15.3:0/2986981157 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9dc01f070 con 0x7fa9e809a3b0 2023-10-03T10:14:58.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.892+0000 7fa9ec8b9700 1 -- 172.21.15.3:0/2986981157 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e806eb00 con 0x7fa9e809a3b0 2023-10-03T10:14:58.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.892+0000 7fa9d6ffd700 1 -- 172.21.15.3:0/2986981157 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc0036a0 con 0x7fa9e809a3b0 2023-10-03T10:14:58.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.892+0000 7fa9ec8b9700 1 -- 172.21.15.3:0/2986981157 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e806fb10 con 0x7fa9e809a3b0 2023-10-03T10:14:58.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.893+0000 7fa9d6ffd700 1 -- 172.21.15.3:0/2986981157 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa9dc021030 con 0x7fa9e809a3b0 2023-10-03T10:14:58.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.893+0000 7fa9d6ffd700 1 -- 172.21.15.3:0/2986981157 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa9dc040550 con 0x7fa9e809a3b0 2023-10-03T10:14:58.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.893+0000 7fa9ec8b9700 1 -- 172.21.15.3:0/2986981157 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e8066990 con 0x7fa9e809a3b0 2023-10-03T10:14:58.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.903+0000 7fa9d6ffd700 1 -- 172.21.15.3:0/2986981157 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa9dc015070 con 0x7fa9e809a3b0 2023-10-03T10:14:58.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.930+0000 7f50dcdcb700 1 Processor -- start 2023-10-03T10:14:58.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.930+0000 7f50dcdcb700 1 -- start start 2023-10-03T10:14:58.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.931+0000 7f50dcdcb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50d809d6f0 0x7f50d809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:58.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.931+0000 7f50dcdcb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f50d809e000 0x7f50d81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:58.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.931+0000 7f50dcdcb700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d809b120 con 0x7f50d809e410 2023-10-03T10:14:58.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.931+0000 7f50dcdcb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d806b460 con 0x7f50d809d6f0 2023-10-03T10:14:58.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.931+0000 7f50dcdcb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f50d809a7e0 con 0x7f50d809e000 2023-10-03T10:14:58.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.931+0000 7f50d659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50d809d6f0 0x7f50d809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:58.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.931+0000 7f50d5d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f50d809e000 0x7f50d81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:58.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.931+0000 7f50d659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50d809d6f0 0x7f50d809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42644/0 (socket says 172.21.15.3:42644) 2023-10-03T10:14:58.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.931+0000 7f50d5d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f50d809e000 0x7f50d81339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56162/0 (socket says 172.21.15.3:56162) 2023-10-03T10:14:58.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.931+0000 7f50d659c700 1 -- 172.21.15.3:0/2551836726 learned_addr learned my addr 172.21.15.3:0/2551836726 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:58.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.931+0000 7f03510fe700 1 Processor -- start 2023-10-03T10:14:58.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.931+0000 7f03510fe700 1 -- start start 2023-10-03T10:14:58.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.931+0000 7f50d659c700 1 -- 172.21.15.3:0/2551836726 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f50d809e000 msgr2=0x7f50d81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:58.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.931+0000 7f50d659c700 1 --2- 172.21.15.3:0/2551836726 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f50d809e000 0x7f50d81339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:58.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.931+0000 7f50d659c700 1 -- 172.21.15.3:0/2551836726 >> v1:172.21.15.3:6789/0 conn(0x7f50d809e410 legacy=0x7f50d8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:58.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.931+0000 7f50d659c700 1 -- 172.21.15.3:0/2551836726 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d809a500 con 0x7f50d809d6f0 2023-10-03T10:14:58.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.932+0000 7f50d659c700 1 --2- 172.21.15.3:0/2551836726 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50d809d6f0 0x7f50d809dac0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009920 tx=0x7f50c8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a201a82fad90f51 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:58.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.932+0000 7f50d559a700 1 -- 172.21.15.3:0/2551836726 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8011830 con 0x7f50d809d6f0 2023-10-03T10:14:58.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.932+0000 7f03510fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f034c09d6f0 0x7f034c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:58.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.932+0000 7f03510fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f034c134100 0x7f034c13bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:58.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.932+0000 7f50d559a700 1 -- 172.21.15.3:0/2551836726 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c8011990 con 0x7f50d809d6f0 2023-10-03T10:14:58.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.932+0000 7f50d559a700 1 -- 172.21.15.3:0/2551836726 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8011d20 con 0x7f50d809d6f0 2023-10-03T10:14:58.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.932+0000 7f03510fe700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f034c09d2f0 con 0x7f034c09e000 2023-10-03T10:14:58.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.932+0000 7f03510fe700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f034c09a810 con 0x7f034c134100 2023-10-03T10:14:58.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.932+0000 7f03510fe700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f034c09a470 con 0x7f034c09d6f0 2023-10-03T10:14:58.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.932+0000 7f034ad9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f034c09d6f0 0x7f034c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:58.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.932+0000 7f034ad9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f034c09d6f0 0x7f034c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56178/0 (socket says 172.21.15.3:56178) 2023-10-03T10:14:58.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.932+0000 7f034ad9d700 1 -- 172.21.15.3:0/1136472889 learned_addr learned my addr 172.21.15.3:0/1136472889 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:58.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.932+0000 7f034b59e700 1 --2- 172.21.15.3:0/1136472889 >> v2:172.21.15.3:3300/0 conn(0x7f034c134100 0x7f034c13bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:58.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.933+0000 7f034ad9d700 1 -- 172.21.15.3:0/1136472889 >> v2:172.21.15.3:3300/0 conn(0x7f034c134100 msgr2=0x7f034c13bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:58.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.933+0000 7f034ad9d700 1 --2- 172.21.15.3:0/1136472889 >> v2:172.21.15.3:3300/0 conn(0x7f034c134100 0x7f034c13bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:58.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.933+0000 7f034ad9d700 1 -- 172.21.15.3:0/1136472889 >> v1:172.21.15.3:6789/0 conn(0x7f034c09e000 legacy=0x7f034c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:14:58.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.933+0000 7f034ad9d700 1 -- 172.21.15.3:0/1136472889 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034c09ec40 con 0x7f034c09d6f0 2023-10-03T10:14:58.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.933+0000 7f034ad9d700 1 --2- 172.21.15.3:0/1136472889 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f034c09d6f0 0x7f034c09dac0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f03440098f0 tx=0x7f0344009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faa50bc968c157fb server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:58.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.933+0000 7f0349d9b700 1 -- 172.21.15.3:0/1136472889 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344004230 con 0x7f034c09d6f0 2023-10-03T10:14:58.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.933+0000 7f0349d9b700 1 -- 172.21.15.3:0/1136472889 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0344004390 con 0x7f034c09d6f0 2023-10-03T10:14:58.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.933+0000 7f0349d9b700 1 -- 172.21.15.3:0/1136472889 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03440036a0 con 0x7f034c09d6f0 2023-10-03T10:14:58.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.933+0000 7f50dcdcb700 1 -- 172.21.15.3:0/2551836726 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50d809d6f0 msgr2=0x7f50d809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:58.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.933+0000 7f50dcdcb700 1 --2- 172.21.15.3:0/2551836726 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50d809d6f0 0x7f50d809dac0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009920 tx=0x7f50c8009c30 comp rx=0 tx=0).stop 2023-10-03T10:14:58.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.934+0000 7f50dcdcb700 1 -- 172.21.15.3:0/2551836726 shutdown_connections 2023-10-03T10:14:58.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.934+0000 7f50dcdcb700 1 --2- 172.21.15.3:0/2551836726 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50d809d6f0 0x7f50d809dac0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:58.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.934+0000 7f50dcdcb700 1 --2- 172.21.15.3:0/2551836726 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f50d809e000 0x7f50d81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:58.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.934+0000 7f50dcdcb700 1 -- 172.21.15.3:0/2551836726 >> 172.21.15.3:0/2551836726 conn(0x7f50d8066f50 msgr2=0x7f50d809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:58.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.934+0000 7f50dcdcb700 1 -- 172.21.15.3:0/2551836726 shutdown_connections 2023-10-03T10:14:58.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.934+0000 7f50dcdcb700 1 -- 172.21.15.3:0/2551836726 wait complete. 2023-10-03T10:14:58.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.934+0000 7f03510fe700 1 -- 172.21.15.3:0/1136472889 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f034c09d6f0 msgr2=0x7f034c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:58.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.934+0000 7f03510fe700 1 --2- 172.21.15.3:0/1136472889 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f034c09d6f0 0x7f034c09dac0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f03440098f0 tx=0x7f0344009c00 comp rx=0 tx=0).stop 2023-10-03T10:14:58.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.934+0000 7f03510fe700 1 -- 172.21.15.3:0/1136472889 shutdown_connections 2023-10-03T10:14:58.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.934+0000 7f03510fe700 1 --2- 172.21.15.3:0/1136472889 >> v2:172.21.15.3:3300/0 conn(0x7f034c134100 0x7f034c13bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:58.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.934+0000 7f03510fe700 1 --2- 172.21.15.3:0/1136472889 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f034c09d6f0 0x7f034c09dac0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:58.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.934+0000 7f03510fe700 1 -- 172.21.15.3:0/1136472889 >> 172.21.15.3:0/1136472889 conn(0x7f034c066f50 msgr2=0x7f034c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:58.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.934+0000 7f03510fe700 1 -- 172.21.15.3:0/1136472889 shutdown_connections 2023-10-03T10:14:58.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.934+0000 7f03510fe700 1 -- 172.21.15.3:0/1136472889 wait complete. 2023-10-03T10:14:58.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.935+0000 7f50dcdcb700 1 Processor -- start 2023-10-03T10:14:58.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.935+0000 7f50dcdcb700 1 -- start start 2023-10-03T10:14:58.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.936+0000 7f03510fe700 1 Processor -- start 2023-10-03T10:14:58.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.936+0000 7f03510fe700 1 -- start start 2023-10-03T10:14:58.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.936+0000 7f50dcdcb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f50d809d6f0 0x7f50d8122690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:58.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.937+0000 7f03510fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f034c09d6f0 0x7f034c1226d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:58.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.937+0000 7f03510fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f034c09e000 0x7f034c122c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:58.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.937+0000 7f03510fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f034c134100 0x7f034c123150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:58.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.937+0000 7f03510fe700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034c1331d0 con 0x7f034c09e000 2023-10-03T10:14:58.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.937+0000 7f03510fe700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f034c1242e0 con 0x7f034c134100 2023-10-03T10:14:58.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.937+0000 7f03510fe700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f034c125c50 con 0x7f034c09d6f0 2023-10-03T10:14:58.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.937+0000 7f034ad9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f034c09d6f0 0x7f034c1226d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:58.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.937+0000 7f034ad9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f034c09d6f0 0x7f034c1226d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56196/0 (socket says 172.21.15.3:56196) 2023-10-03T10:14:58.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.937+0000 7f034ad9d700 1 -- 172.21.15.3:0/1365576372 learned_addr learned my addr 172.21.15.3:0/1365576372 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:58.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.937+0000 7f034a59c700 1 --2- 172.21.15.3:0/1365576372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f034c09e000 0x7f034c122c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:58.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.936+0000 7f50dcdcb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50d809e000 0x7f50d8122bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:58.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.936+0000 7f50dcdcb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50d809e410 0x7f50d8123110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:14:58.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.937+0000 7f034ad9d700 1 -- 172.21.15.3:0/1365576372 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f034c134100 msgr2=0x7f034c123150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:58.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.937+0000 7f034ad9d700 1 --2- 172.21.15.3:0/1365576372 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f034c134100 0x7f034c123150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:58.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.937+0000 7f034ad9d700 1 -- 172.21.15.3:0/1365576372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f034c09e000 msgr2=0x7f034c122c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:58.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.937+0000 7f034ad9d700 1 --2- 172.21.15.3:0/1365576372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f034c09e000 0x7f034c122c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:58.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.937+0000 7f034ad9d700 1 -- 172.21.15.3:0/1365576372 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0344009550 con 0x7f034c09d6f0 2023-10-03T10:14:58.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.936+0000 7f50dcdcb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d81254b0 con 0x7f50d809e000 2023-10-03T10:14:58.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.936+0000 7f50dcdcb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d809c710 con 0x7f50d809e410 2023-10-03T10:14:58.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.936+0000 7f50dcdcb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f50d8124720 con 0x7f50d809d6f0 2023-10-03T10:14:58.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.936+0000 7f50d659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f50d809d6f0 0x7f50d8122690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:58.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.936+0000 7f50d659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f50d809d6f0 0x7f50d8122690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56192/0 (socket says 172.21.15.3:56192) 2023-10-03T10:14:58.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.936+0000 7f50d659c700 1 -- 172.21.15.3:0/3674412280 learned_addr learned my addr 172.21.15.3:0/3674412280 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:14:58.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.937+0000 7f50d6d9d700 1 --2- 172.21.15.3:0/3674412280 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50d809e410 0x7f50d8123110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:14:58.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.937+0000 7f50d659c700 1 -- 172.21.15.3:0/3674412280 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50d809e410 msgr2=0x7f50d8123110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:58.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.937+0000 7f50d659c700 1 --2- 172.21.15.3:0/3674412280 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50d809e410 0x7f50d8123110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:58.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.937+0000 7f50d659c700 1 -- 172.21.15.3:0/3674412280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50d809e000 msgr2=0x7f50d8122bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:58.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.937+0000 7f50d659c700 1 --2- 172.21.15.3:0/3674412280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50d809e000 0x7f50d8122bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:58.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.937+0000 7f50d659c700 1 -- 172.21.15.3:0/3674412280 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c8009550 con 0x7f50d809d6f0 2023-10-03T10:14:58.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.937+0000 7f50d659c700 1 --2- 172.21.15.3:0/3674412280 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f50d809d6f0 0x7f50d8122690 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009110 tx=0x7f50c8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:58.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.938+0000 7f50c6ffd700 1 -- 172.21.15.3:0/3674412280 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8004240 con 0x7f50d809d6f0 2023-10-03T10:14:58.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.937+0000 7f034ad9d700 1 --2- 172.21.15.3:0/1365576372 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f034c09d6f0 0x7f034c1226d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f034c09a5b0 tx=0x7f0344004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:14:58.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.938+0000 7f50c6ffd700 1 -- 172.21.15.3:0/3674412280 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c80043a0 con 0x7f50d809d6f0 2023-10-03T10:14:58.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.938+0000 7f50c6ffd700 1 -- 172.21.15.3:0/3674412280 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8022340 con 0x7f50d809d6f0 2023-10-03T10:14:58.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.938+0000 7f03437fe700 1 -- 172.21.15.3:0/1365576372 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344003cc0 con 0x7f034c09d6f0 2023-10-03T10:14:58.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.938+0000 7f03437fe700 1 -- 172.21.15.3:0/1365576372 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034401f070 con 0x7f034c09d6f0 2023-10-03T10:14:58.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.938+0000 7f03510fe700 1 -- 172.21.15.3:0/1365576372 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f034c1339d0 con 0x7f034c09d6f0 2023-10-03T10:14:58.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.938+0000 7f03437fe700 1 -- 172.21.15.3:0/1365576372 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034400b8a0 con 0x7f034c09d6f0 2023-10-03T10:14:58.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.938+0000 7f03510fe700 1 -- 172.21.15.3:0/1365576372 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034c125940 con 0x7f034c09d6f0 2023-10-03T10:14:58.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.938+0000 7f50dcdcb700 1 -- 172.21.15.3:0/3674412280 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d80a1d90 con 0x7f50d809d6f0 2023-10-03T10:14:58.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.938+0000 7f50dcdcb700 1 -- 172.21.15.3:0/3674412280 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d8125820 con 0x7f50d809d6f0 2023-10-03T10:14:58.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.939+0000 7f50dcdcb700 1 -- 172.21.15.3:0/3674412280 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d8125b50 con 0x7f50d809d6f0 2023-10-03T10:14:58.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.939+0000 7f03437fe700 1 -- 172.21.15.3:0/1365576372 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0344021030 con 0x7f034c09d6f0 2023-10-03T10:14:58.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.940+0000 7f50c6ffd700 1 -- 172.21.15.3:0/3674412280 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f50c80224a0 con 0x7f50d809d6f0 2023-10-03T10:14:58.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.940+0000 7f03437fe700 1 -- 172.21.15.3:0/1365576372 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0344040cd0 con 0x7f034c09d6f0 2023-10-03T10:14:58.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.940+0000 7f03510fe700 1 -- 172.21.15.3:0/1365576372 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03300052e0 con 0x7f034c09d6f0 2023-10-03T10:14:58.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.940+0000 7f50c6ffd700 1 -- 172.21.15.3:0/3674412280 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f50c8051b50 con 0x7f50d809d6f0 2023-10-03T10:14:58.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.949+0000 7f03437fe700 1 -- 172.21.15.3:0/1365576372 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f034400ba00 con 0x7f034c09d6f0 2023-10-03T10:14:58.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.958+0000 7f50c6ffd700 1 -- 172.21.15.3:0/3674412280 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f50d8125b50 con 0x7f50d809d6f0 2023-10-03T10:14:58.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.968+0000 7f7aeac30700 1 -- 172.21.15.3:0/1878019684 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7ae406b250 con 0x7f7ae4133e30 2023-10-03T10:14:58.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.969+0000 7f7ae17fa700 1 -- 172.21.15.3:0/1878019684 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7ad8011590 con 0x7f7ae4133e30 2023-10-03T10:14:58.969 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:14:58.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.971+0000 7f7aeac30700 1 -- 172.21.15.3:0/1878019684 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ae4133e30 msgr2=0x7f7ae409c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:58.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.971+0000 7f7aeac30700 1 --2- 172.21.15.3:0/1878019684 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ae4133e30 0x7f7ae409c9c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8000f60 tx=0x7f7ad800c730 comp rx=0 tx=0).stop 2023-10-03T10:14:58.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.971+0000 7f7aeac30700 1 -- 172.21.15.3:0/1878019684 shutdown_connections 2023-10-03T10:14:58.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.971+0000 7f7aeac30700 1 --2- 172.21.15.3:0/1878019684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ae4134260 0x7f7ae409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:58.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.971+0000 7f7aeac30700 1 --2- 172.21.15.3:0/1878019684 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ae4133420 0x7f7ae409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:58.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.971+0000 7f7aeac30700 1 --2- 172.21.15.3:0/1878019684 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ae4133e30 0x7f7ae409c9c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:58.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.971+0000 7f7aeac30700 1 -- 172.21.15.3:0/1878019684 >> 172.21.15.3:0/1878019684 conn(0x7f7ae40a4210 msgr2=0x7f7ae40a0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:58.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.972+0000 7f7aeac30700 1 -- 172.21.15.3:0/1878019684 shutdown_connections 2023-10-03T10:14:58.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:58.972+0000 7f7aeac30700 1 -- 172.21.15.3:0/1878019684 wait complete. 2023-10-03T10:14:58.987 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:14:59.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.122+0000 7fa9ec8b9700 1 -- 172.21.15.3:0/2986981157 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa9e809afa0 con 0x7fa9e809a3b0 2023-10-03T10:14:59.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.123+0000 7fa9d6ffd700 1 -- 172.21.15.3:0/2986981157 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa9dc003800 con 0x7fa9e809a3b0 2023-10-03T10:14:59.123 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:14:59.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.125+0000 7fa9ec8b9700 1 -- 172.21.15.3:0/2986981157 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9e809a3b0 msgr2=0x7fa9e8070440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:59.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.125+0000 7fa9ec8b9700 1 --2- 172.21.15.3:0/2986981157 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9e809a3b0 0x7fa9e8070440 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc016000 tx=0x7fa9dc004750 comp rx=0 tx=0).stop 2023-10-03T10:14:59.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.125+0000 7fa9ec8b9700 1 -- 172.21.15.3:0/2986981157 shutdown_connections 2023-10-03T10:14:59.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.125+0000 7fa9ec8b9700 1 --2- 172.21.15.3:0/2986981157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9e8131810 0x7fa9e81c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:59.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.125+0000 7fa9ec8b9700 1 --2- 172.21.15.3:0/2986981157 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9e8099a40 0x7fa9e806ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:59.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.125+0000 7fa9ec8b9700 1 --2- 172.21.15.3:0/2986981157 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9e809a3b0 0x7fa9e8070440 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:59.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.125+0000 7fa9ec8b9700 1 -- 172.21.15.3:0/2986981157 >> 172.21.15.3:0/2986981157 conn(0x7fa9e80a2aa0 msgr2=0x7fa9e80a12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:59.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.125+0000 7fa9ec8b9700 1 -- 172.21.15.3:0/2986981157 shutdown_connections 2023-10-03T10:14:59.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.125+0000 7fa9ec8b9700 1 -- 172.21.15.3:0/2986981157 wait complete. 2023-10-03T10:14:59.140 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:14:59.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.158+0000 7f03510fe700 1 -- 172.21.15.3:0/1365576372 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0330005590 con 0x7f034c09d6f0 2023-10-03T10:14:59.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.159+0000 7f03437fe700 1 -- 172.21.15.3:0/1365576372 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f03440233d0 con 0x7f034c09d6f0 2023-10-03T10:14:59.159 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:14:59.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.161+0000 7f03510fe700 1 -- 172.21.15.3:0/1365576372 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f034c09d6f0 msgr2=0x7f034c1226d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:59.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.161+0000 7f03510fe700 1 --2- 172.21.15.3:0/1365576372 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f034c09d6f0 0x7f034c1226d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f034c09a5b0 tx=0x7f0344004750 comp rx=0 tx=0).stop 2023-10-03T10:14:59.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.161+0000 7f03510fe700 1 -- 172.21.15.3:0/1365576372 shutdown_connections 2023-10-03T10:14:59.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.161+0000 7f03510fe700 1 --2- 172.21.15.3:0/1365576372 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f034c134100 0x7f034c123150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:59.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.161+0000 7f03510fe700 1 --2- 172.21.15.3:0/1365576372 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f034c09d6f0 0x7f034c1226d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:59.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.161+0000 7f03510fe700 1 --2- 172.21.15.3:0/1365576372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f034c09e000 0x7f034c122c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:59.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.161+0000 7f03510fe700 1 -- 172.21.15.3:0/1365576372 >> 172.21.15.3:0/1365576372 conn(0x7f034c066f50 msgr2=0x7f034c070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:59.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.161+0000 7f03510fe700 1 -- 172.21.15.3:0/1365576372 shutdown_connections 2023-10-03T10:14:59.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.161+0000 7f03510fe700 1 -- 172.21.15.3:0/1365576372 wait complete. 2023-10-03T10:14:59.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.167+0000 7f50dcdcb700 1 -- 172.21.15.3:0/3674412280 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f50d8125b50 con 0x7f50d809d6f0 2023-10-03T10:14:59.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.167+0000 7f50c6ffd700 1 -- 172.21.15.3:0/3674412280 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f50d8125b50 con 0x7f50d809d6f0 2023-10-03T10:14:59.167 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:14:59.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.170+0000 7f50dcdcb700 1 -- 172.21.15.3:0/3674412280 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f50d809d6f0 msgr2=0x7f50d8122690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:14:59.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.170+0000 7f50dcdcb700 1 --2- 172.21.15.3:0/3674412280 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f50d809d6f0 0x7f50d8122690 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009110 tx=0x7f50c8004750 comp rx=0 tx=0).stop 2023-10-03T10:14:59.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.170+0000 7f50dcdcb700 1 -- 172.21.15.3:0/3674412280 shutdown_connections 2023-10-03T10:14:59.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.170+0000 7f50dcdcb700 1 --2- 172.21.15.3:0/3674412280 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f50d809e410 0x7f50d8123110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:59.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.170+0000 7f50dcdcb700 1 --2- 172.21.15.3:0/3674412280 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f50d809d6f0 0x7f50d8122690 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:59.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.170+0000 7f50dcdcb700 1 --2- 172.21.15.3:0/3674412280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50d809e000 0x7f50d8122bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:14:59.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.170+0000 7f50dcdcb700 1 -- 172.21.15.3:0/3674412280 >> 172.21.15.3:0/3674412280 conn(0x7f50d8066f50 msgr2=0x7f50d80a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:14:59.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.170+0000 7f50dcdcb700 1 -- 172.21.15.3:0/3674412280 shutdown_connections 2023-10-03T10:14:59.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:14:59.170+0000 7f50dcdcb700 1 -- 172.21.15.3:0/3674412280 wait complete. 2023-10-03T10:14:59.176 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:14:59.184 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:14:59.988 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:00.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.134+0000 7f24ec2d8700 1 Processor -- start 2023-10-03T10:15:00.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.134+0000 7f24ec2d8700 1 -- start start 2023-10-03T10:15:00.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.135+0000 7f24ec2d8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24e409e1d0 0x7f24e4138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:00.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.135+0000 7f24ec2d8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f24e4139170 0x7f24e413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:00.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.135+0000 7f24ec2d8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24e409b120 con 0x7f24e409d6f0 2023-10-03T10:15:00.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.135+0000 7f24ec2d8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24e406b460 con 0x7f24e4139170 2023-10-03T10:15:00.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.135+0000 7f24ec2d8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e409a7e0 con 0x7f24e409e1d0 2023-10-03T10:15:00.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.135+0000 7f24e9873700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24e409e1d0 0x7f24e4138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:00.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.135+0000 7f24ea875700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f24e4139170 0x7f24e413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:00.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.135+0000 7f24ea875700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f24e4139170 0x7f24e413b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41252/0 (socket says 172.21.15.3:41252) 2023-10-03T10:15:00.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.135+0000 7f24e9873700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24e409e1d0 0x7f24e4138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42682/0 (socket says 172.21.15.3:42682) 2023-10-03T10:15:00.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.135+0000 7f24ea875700 1 -- 172.21.15.3:0/1455936323 learned_addr learned my addr 172.21.15.3:0/1455936323 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:00.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.135+0000 7f24ea875700 1 -- 172.21.15.3:0/1455936323 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24e409e1d0 msgr2=0x7f24e4138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:00.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.135+0000 7f24ea875700 1 --2- 172.21.15.3:0/1455936323 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24e409e1d0 0x7f24e4138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.135+0000 7f24ea875700 1 -- 172.21.15.3:0/1455936323 >> v1:172.21.15.3:6789/0 conn(0x7f24e409d6f0 legacy=0x7f24e409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:00.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.135+0000 7f24ea875700 1 -- 172.21.15.3:0/1455936323 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e409a500 con 0x7f24e4139170 2023-10-03T10:15:00.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.135+0000 7f24e9873700 1 --2- 172.21.15.3:0/1455936323 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24e409e1d0 0x7f24e4138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:00.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.135+0000 7f24ea875700 1 --2- 172.21.15.3:0/1455936323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24e4139170 0x7f24e413b550 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f24dc009e50 tx=0x7f24dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4228d14604c8c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:00.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.135+0000 7f24e9072700 1 -- 172.21.15.3:0/1455936323 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24dc00e070 con 0x7f24e4139170 2023-10-03T10:15:00.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.135+0000 7f24e9072700 1 -- 172.21.15.3:0/1455936323 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24dc00b910 con 0x7f24e4139170 2023-10-03T10:15:00.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.135+0000 7f24e9072700 1 -- 172.21.15.3:0/1455936323 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24dc00bcc0 con 0x7f24e4139170 2023-10-03T10:15:00.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.136+0000 7f24ec2d8700 1 -- 172.21.15.3:0/1455936323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24e4139170 msgr2=0x7f24e413b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:00.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.136+0000 7f24ec2d8700 1 --2- 172.21.15.3:0/1455936323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24e4139170 0x7f24e413b550 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f24dc009e50 tx=0x7f24dc00b330 comp rx=0 tx=0).stop 2023-10-03T10:15:00.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.136+0000 7f24ec2d8700 1 -- 172.21.15.3:0/1455936323 shutdown_connections 2023-10-03T10:15:00.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.136+0000 7f24ec2d8700 1 --2- 172.21.15.3:0/1455936323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24e4139170 0x7f24e413b550 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.136+0000 7f24ec2d8700 1 --2- 172.21.15.3:0/1455936323 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24e409e1d0 0x7f24e4138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.136+0000 7f24ec2d8700 1 -- 172.21.15.3:0/1455936323 >> 172.21.15.3:0/1455936323 conn(0x7f24e4066f50 msgr2=0x7f24e409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:00.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.136+0000 7f24ec2d8700 1 -- 172.21.15.3:0/1455936323 shutdown_connections 2023-10-03T10:15:00.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.136+0000 7f24ec2d8700 1 -- 172.21.15.3:0/1455936323 wait complete. 2023-10-03T10:15:00.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.137+0000 7f24ec2d8700 1 Processor -- start 2023-10-03T10:15:00.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.137+0000 7f24ec2d8700 1 -- start start 2023-10-03T10:15:00.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.138+0000 7f24ec2d8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24e409d6f0 0x7f24e41c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:00.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.138+0000 7f24ec2d8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24e409e1d0 0x7f24e41c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:00.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.138+0000 7f24ec2d8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24e4139170 0x7f24e411e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:00.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.138+0000 7f24ec2d8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e4120560 con 0x7f24e409e1d0 2023-10-03T10:15:00.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.138+0000 7f24ec2d8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e4120370 con 0x7f24e4139170 2023-10-03T10:15:00.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.138+0000 7f24ec2d8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24e4121ad0 con 0x7f24e409d6f0 2023-10-03T10:15:00.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.138+0000 7f24ea074700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24e409d6f0 0x7f24e41c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:00.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.138+0000 7f24ea875700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24e4139170 0x7f24e411e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:00.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.138+0000 7f24ea875700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24e4139170 0x7f24e411e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42698/0 (socket says 172.21.15.3:42698) 2023-10-03T10:15:00.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.138+0000 7f24ea074700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24e409d6f0 0x7f24e41c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56202/0 (socket says 172.21.15.3:56202) 2023-10-03T10:15:00.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.138+0000 7f24ea875700 1 -- 172.21.15.3:0/2462368559 learned_addr learned my addr 172.21.15.3:0/2462368559 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:00.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.138+0000 7f24e9873700 1 --2- 172.21.15.3:0/2462368559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24e409e1d0 0x7f24e41c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:00.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.138+0000 7f24ea875700 1 -- 172.21.15.3:0/2462368559 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24e409d6f0 msgr2=0x7f24e41c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:00.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.138+0000 7f24ea875700 1 --2- 172.21.15.3:0/2462368559 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24e409d6f0 0x7f24e41c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.138+0000 7f24ea875700 1 -- 172.21.15.3:0/2462368559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24e409e1d0 msgr2=0x7f24e41c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:00.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.138+0000 7f24ea875700 1 --2- 172.21.15.3:0/2462368559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24e409e1d0 0x7f24e41c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.138+0000 7f24ea875700 1 -- 172.21.15.3:0/2462368559 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d40094a0 con 0x7f24e4139170 2023-10-03T10:15:00.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.138+0000 7f24ea074700 1 --2- 172.21.15.3:0/2462368559 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24e409d6f0 0x7f24e41c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:00.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.138+0000 7f24ea875700 1 --2- 172.21.15.3:0/2462368559 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24e4139170 0x7f24e411e8e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f24dc004270 tx=0x7f24dc004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:00.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.138+0000 7f24daffd700 1 -- 172.21.15.3:0/2462368559 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24dc0048e0 con 0x7f24e4139170 2023-10-03T10:15:00.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.139+0000 7f24daffd700 1 -- 172.21.15.3:0/2462368559 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24dc0092e0 con 0x7f24e4139170 2023-10-03T10:15:00.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.139+0000 7f24ec2d8700 1 -- 172.21.15.3:0/2462368559 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24dc009b30 con 0x7f24e4139170 2023-10-03T10:15:00.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.139+0000 7f24daffd700 1 -- 172.21.15.3:0/2462368559 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24dc01c660 con 0x7f24e4139170 2023-10-03T10:15:00.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.139+0000 7f24ec2d8700 1 -- 172.21.15.3:0/2462368559 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e4121fa0 con 0x7f24e4139170 2023-10-03T10:15:00.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.139+0000 7f24daffd700 1 -- 172.21.15.3:0/2462368559 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f24dc01c7c0 con 0x7f24e4139170 2023-10-03T10:15:00.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.139+0000 7f24daffd700 1 -- 172.21.15.3:0/2462368559 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f24dc040fc0 con 0x7f24e4139170 2023-10-03T10:15:00.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.140+0000 7f24ec2d8700 1 -- 172.21.15.3:0/2462368559 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24e4074ea0 con 0x7f24e4139170 2023-10-03T10:15:00.144 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:00.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.145+0000 7f24daffd700 1 -- 172.21.15.3:0/2462368559 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f24dc0123f0 con 0x7f24e4139170 2023-10-03T10:15:00.177 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:00.184 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:00.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.290+0000 7fce27a8f700 1 Processor -- start 2023-10-03T10:15:00.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.291+0000 7fce27a8f700 1 -- start start 2023-10-03T10:15:00.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.291+0000 7fce27a8f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fce2009d6f0 0x7fce2009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:00.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.291+0000 7fce27a8f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce2009e000 0x7fce201339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:00.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.291+0000 7fce27a8f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce2009b120 con 0x7fce2009e410 2023-10-03T10:15:00.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.291+0000 7fce27a8f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce2006b460 con 0x7fce2009d6f0 2023-10-03T10:15:00.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.291+0000 7fce27a8f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce2009a7e0 con 0x7fce2009e000 2023-10-03T10:15:00.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.291+0000 7fce2582b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fce2009d6f0 0x7fce2009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:00.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.291+0000 7fce2582b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fce2009d6f0 0x7fce2009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41276/0 (socket says 172.21.15.3:41276) 2023-10-03T10:15:00.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.291+0000 7fce2582b700 1 -- 172.21.15.3:0/3682560021 learned_addr learned my addr 172.21.15.3:0/3682560021 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:00.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.291+0000 7fce2502a700 1 --2- 172.21.15.3:0/3682560021 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce2009e000 0x7fce201339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:00.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.292+0000 7fce2502a700 1 -- 172.21.15.3:0/3682560021 >> v2:172.21.15.3:3300/0 conn(0x7fce2009d6f0 msgr2=0x7fce2009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:00.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.292+0000 7fce2502a700 1 --2- 172.21.15.3:0/3682560021 >> v2:172.21.15.3:3300/0 conn(0x7fce2009d6f0 0x7fce2009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.292+0000 7fce2502a700 1 -- 172.21.15.3:0/3682560021 >> v1:172.21.15.3:6789/0 conn(0x7fce2009e410 legacy=0x7fce20133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:00.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.292+0000 7fce24829700 1 -- 172.21.15.3:0/3682560021 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 532971680 0 0) 0x7fce2009b120 con 0x7fce2009e410 2023-10-03T10:15:00.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.292+0000 7fce2502a700 1 -- 172.21.15.3:0/3682560021 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2009a500 con 0x7fce2009e000 2023-10-03T10:15:00.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.292+0000 7fce2502a700 1 --2- 172.21.15.3:0/3682560021 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce2009e000 0x7fce201339f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fce10009870 tx=0x7fce10009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a161fadb49b34708 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:00.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.292+0000 7fce24829700 1 -- 172.21.15.3:0/3682560021 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce10011830 con 0x7fce2009e000 2023-10-03T10:15:00.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.292+0000 7fce24829700 1 -- 172.21.15.3:0/3682560021 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce10011990 con 0x7fce2009e000 2023-10-03T10:15:00.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.292+0000 7fce24829700 1 -- 172.21.15.3:0/3682560021 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce10011d20 con 0x7fce2009e000 2023-10-03T10:15:00.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.293+0000 7fce27a8f700 1 -- 172.21.15.3:0/3682560021 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce2009e000 msgr2=0x7fce201339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:00.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.293+0000 7fce27a8f700 1 --2- 172.21.15.3:0/3682560021 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce2009e000 0x7fce201339f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fce10009870 tx=0x7fce10009b80 comp rx=0 tx=0).stop 2023-10-03T10:15:00.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.293+0000 7fce27a8f700 1 -- 172.21.15.3:0/3682560021 shutdown_connections 2023-10-03T10:15:00.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.293+0000 7fce27a8f700 1 --2- 172.21.15.3:0/3682560021 >> v2:172.21.15.3:3300/0 conn(0x7fce2009d6f0 0x7fce2009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.293+0000 7fce27a8f700 1 --2- 172.21.15.3:0/3682560021 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce2009e000 0x7fce201339f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.293+0000 7fce27a8f700 1 -- 172.21.15.3:0/3682560021 >> 172.21.15.3:0/3682560021 conn(0x7fce20066f50 msgr2=0x7fce2009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:00.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.293+0000 7fce27a8f700 1 -- 172.21.15.3:0/3682560021 shutdown_connections 2023-10-03T10:15:00.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.293+0000 7fce27a8f700 1 -- 172.21.15.3:0/3682560021 wait complete. 2023-10-03T10:15:00.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.294+0000 7fce27a8f700 1 Processor -- start 2023-10-03T10:15:00.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.295+0000 7fce27a8f700 1 -- start start 2023-10-03T10:15:00.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.295+0000 7fce27a8f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce2009d6f0 0x7fce201c5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:00.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.295+0000 7fce27a8f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce2009e000 0x7fce201c5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:00.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.295+0000 7fce27a8f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce2009e410 0x7fce201c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:00.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.295+0000 7fce27a8f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce201a60c0 con 0x7fce2009e410 2023-10-03T10:15:00.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.295+0000 7fce27a8f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce201b9530 con 0x7fce2009d6f0 2023-10-03T10:15:00.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.295+0000 7fce27a8f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce201a7a30 con 0x7fce2009e000 2023-10-03T10:15:00.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.295+0000 7fce2502a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce2009e000 0x7fce201c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:00.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.295+0000 7fce2502a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce2009e000 0x7fce201c5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56220/0 (socket says 172.21.15.3:56220) 2023-10-03T10:15:00.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.295+0000 7fce2502a700 1 -- 172.21.15.3:0/4087600621 learned_addr learned my addr 172.21.15.3:0/4087600621 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:00.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.295+0000 7fce2602c700 1 --2- 172.21.15.3:0/4087600621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce2009e410 0x7fce201c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:00.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.296+0000 7fce2502a700 1 -- 172.21.15.3:0/4087600621 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce2009d6f0 msgr2=0x7fce201c5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:00.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.296+0000 7fce2502a700 1 --2- 172.21.15.3:0/4087600621 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce2009d6f0 0x7fce201c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.296+0000 7fce2502a700 1 -- 172.21.15.3:0/4087600621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce2009e410 msgr2=0x7fce201c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:00.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.296+0000 7fce2502a700 1 --2- 172.21.15.3:0/4087600621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce2009e410 0x7fce201c8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.296+0000 7fce2502a700 1 -- 172.21.15.3:0/4087600621 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce100094a0 con 0x7fce2009e000 2023-10-03T10:15:00.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.296+0000 7fce2602c700 1 --2- 172.21.15.3:0/4087600621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce2009e410 0x7fce201c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:00.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.296+0000 7fce2502a700 1 --2- 172.21.15.3:0/4087600621 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce2009e000 0x7fce201c5860 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fce100090b0 tx=0x7fce10004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:00.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.296+0000 7fce167fc700 1 -- 172.21.15.3:0/4087600621 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce100036a0 con 0x7fce2009e000 2023-10-03T10:15:00.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.296+0000 7fce167fc700 1 -- 172.21.15.3:0/4087600621 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1001f070 con 0x7fce2009e000 2023-10-03T10:15:00.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.296+0000 7fce27a8f700 1 -- 172.21.15.3:0/4087600621 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce201a8450 con 0x7fce2009e000 2023-10-03T10:15:00.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.296+0000 7fce27a8f700 1 -- 172.21.15.3:0/4087600621 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce201a6820 con 0x7fce2009e000 2023-10-03T10:15:00.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.297+0000 7fce167fc700 1 -- 172.21.15.3:0/4087600621 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce10003a00 con 0x7fce2009e000 2023-10-03T10:15:00.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.298+0000 7fce27a8f700 1 -- 172.21.15.3:0/4087600621 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce080052e0 con 0x7fce2009e000 2023-10-03T10:15:00.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.298+0000 7fce167fc700 1 -- 172.21.15.3:0/4087600621 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fce10003b60 con 0x7fce2009e000 2023-10-03T10:15:00.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.299+0000 7fce167fc700 1 -- 172.21.15.3:0/4087600621 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fce10021070 con 0x7fce2009e000 2023-10-03T10:15:00.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.304+0000 7fce167fc700 1 -- 172.21.15.3:0/4087600621 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fce10046060 con 0x7fce2009e000 2023-10-03T10:15:00.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.332+0000 7f4274d43700 1 Processor -- start 2023-10-03T10:15:00.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.333+0000 7f4274d43700 1 -- start start 2023-10-03T10:15:00.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.333+0000 7f1157039700 1 Processor -- start 2023-10-03T10:15:00.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.333+0000 7f1157039700 1 -- start start 2023-10-03T10:15:00.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.333+0000 7f1157039700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f115009e930 0x7f115009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:00.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.333+0000 7f1157039700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f115009f650 0x7f1150131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:00.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.333+0000 7f1157039700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1150070940 con 0x7f115009f240 2023-10-03T10:15:00.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.333+0000 7f1157039700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f115006fd60 con 0x7f115009e930 2023-10-03T10:15:00.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.333+0000 7f1157039700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f115006fa80 con 0x7f115009f650 2023-10-03T10:15:00.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.333+0000 7f4274d43700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f427006c2f0 0x7f427009d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:00.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.333+0000 7f4274d43700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f427009dc30 0x7f427009e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:00.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.333+0000 7f4274d43700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4270099ef0 con 0x7f4270137ef0 2023-10-03T10:15:00.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.333+0000 7f4274d43700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f427006c6c0 con 0x7f427009dc30 2023-10-03T10:15:00.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.333+0000 7f4274d43700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f427009bb60 con 0x7f427006c2f0 2023-10-03T10:15:00.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.334+0000 7f11555d6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f115009f650 0x7f1150131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:00.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.334+0000 7f11555d6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f115009f650 0x7f1150131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56232/0 (socket says 172.21.15.3:56232) 2023-10-03T10:15:00.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.334+0000 7f11555d6700 1 -- 172.21.15.3:0/4239653426 learned_addr learned my addr 172.21.15.3:0/4239653426 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:00.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.334+0000 7f426ed9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4270137ef0 0x7f427013a2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48304/0 (socket says 172.21.15.3:48304) 2023-10-03T10:15:00.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.334+0000 7f426ed9d700 1 -- 172.21.15.3:0/2821062099 learned_addr learned my addr 172.21.15.3:0/2821062099 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:00.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.334+0000 7f426dd9b700 1 --2- 172.21.15.3:0/2821062099 >> v2:172.21.15.3:3300/0 conn(0x7f427009dc30 0x7f427009e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:00.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.334+0000 7f426d59a700 1 -- 172.21.15.3:0/2821062099 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473597946 0 0) 0x7f4270099ef0 con 0x7f4270137ef0 2023-10-03T10:15:00.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.334+0000 7f426d59a700 1 -- 172.21.15.3:0/2821062099 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42580035b0 con 0x7f4270137ef0 2023-10-03T10:15:00.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.334+0000 7f1154dd5700 1 --2- 172.21.15.3:0/4239653426 >> v2:172.21.15.3:3300/0 conn(0x7f115009e930 0x7f115009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:00.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.334+0000 7f11555d6700 1 -- 172.21.15.3:0/4239653426 >> v2:172.21.15.3:3300/0 conn(0x7f115009e930 msgr2=0x7f115009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:00.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.334+0000 7f11555d6700 1 --2- 172.21.15.3:0/4239653426 >> v2:172.21.15.3:3300/0 conn(0x7f115009e930 0x7f115009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.334+0000 7f11555d6700 1 -- 172.21.15.3:0/4239653426 >> v1:172.21.15.3:6789/0 conn(0x7f115009f240 legacy=0x7f11501317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:00.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.334+0000 7f11555d6700 1 -- 172.21.15.3:0/4239653426 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f115009fb60 con 0x7f115009f650 2023-10-03T10:15:00.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.334+0000 7f11555d6700 1 --2- 172.21.15.3:0/4239653426 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f115009f650 0x7f1150131f00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1148009e50 tx=0x7f114800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4337d0a8fdeca7f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:00.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.334+0000 7f426dd9b700 1 -- 172.21.15.3:0/2821062099 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f427006c2f0 msgr2=0x7f427009d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:00.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.334+0000 7f426dd9b700 1 --2- 172.21.15.3:0/2821062099 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f427006c2f0 0x7f427009d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.334+0000 7f426dd9b700 1 -- 172.21.15.3:0/2821062099 >> v1:172.21.15.3:6789/0 conn(0x7f4270137ef0 legacy=0x7f427013a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:00.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.334+0000 7f426dd9b700 1 -- 172.21.15.3:0/2821062099 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427009aca0 con 0x7f427009dc30 2023-10-03T10:15:00.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.335+0000 7f114f7fe700 1 -- 172.21.15.3:0/4239653426 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1148003c20 con 0x7f115009f650 2023-10-03T10:15:00.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.335+0000 7f426dd9b700 1 --2- 172.21.15.3:0/2821062099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f427009dc30 0x7f427009e040 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4264009840 tx=0x7f4264009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbbe1955abed74b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:00.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.335+0000 7f114f7fe700 1 -- 172.21.15.3:0/4239653426 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1148003d80 con 0x7f115009f650 2023-10-03T10:15:00.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.335+0000 7f114f7fe700 1 -- 172.21.15.3:0/4239653426 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114800bba0 con 0x7f115009f650 2023-10-03T10:15:00.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.335+0000 7f426d59a700 1 -- 172.21.15.3:0/2821062099 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4264011830 con 0x7f427009dc30 2023-10-03T10:15:00.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.335+0000 7f426d59a700 1 -- 172.21.15.3:0/2821062099 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4264011990 con 0x7f427009dc30 2023-10-03T10:15:00.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.335+0000 7f426d59a700 1 -- 172.21.15.3:0/2821062099 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4264011d00 con 0x7f427009dc30 2023-10-03T10:15:00.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.336+0000 7f1157039700 1 -- 172.21.15.3:0/4239653426 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f115009f650 msgr2=0x7f1150131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:00.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.336+0000 7f1157039700 1 --2- 172.21.15.3:0/4239653426 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f115009f650 0x7f1150131f00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1148009e50 tx=0x7f114800b330 comp rx=0 tx=0).stop 2023-10-03T10:15:00.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.336+0000 7f1157039700 1 -- 172.21.15.3:0/4239653426 shutdown_connections 2023-10-03T10:15:00.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.336+0000 7f1157039700 1 --2- 172.21.15.3:0/4239653426 >> v2:172.21.15.3:3300/0 conn(0x7f115009e930 0x7f115009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.336+0000 7f1157039700 1 --2- 172.21.15.3:0/4239653426 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f115009f650 0x7f1150131f00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.336+0000 7f1157039700 1 -- 172.21.15.3:0/4239653426 >> 172.21.15.3:0/4239653426 conn(0x7f11500a4210 msgr2=0x7f11500a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:00.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.336+0000 7f1157039700 1 -- 172.21.15.3:0/4239653426 shutdown_connections 2023-10-03T10:15:00.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.336+0000 7f1157039700 1 -- 172.21.15.3:0/4239653426 wait complete. 2023-10-03T10:15:00.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.336+0000 7f4274d43700 1 -- 172.21.15.3:0/2821062099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f427009dc30 msgr2=0x7f427009e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:00.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.336+0000 7f4274d43700 1 --2- 172.21.15.3:0/2821062099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f427009dc30 0x7f427009e040 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4264009840 tx=0x7f4264009b50 comp rx=0 tx=0).stop 2023-10-03T10:15:00.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.336+0000 7f4274d43700 1 -- 172.21.15.3:0/2821062099 shutdown_connections 2023-10-03T10:15:00.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.336+0000 7f4274d43700 1 --2- 172.21.15.3:0/2821062099 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f427006c2f0 0x7f427009d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.336+0000 7f4274d43700 1 --2- 172.21.15.3:0/2821062099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f427009dc30 0x7f427009e040 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.336+0000 7f4274d43700 1 -- 172.21.15.3:0/2821062099 >> 172.21.15.3:0/2821062099 conn(0x7f427009f470 msgr2=0x7f42700a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:00.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.336+0000 7f4274d43700 1 -- 172.21.15.3:0/2821062099 shutdown_connections 2023-10-03T10:15:00.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.337+0000 7f4274d43700 1 -- 172.21.15.3:0/2821062099 wait complete. 2023-10-03T10:15:00.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.337+0000 7f1157039700 1 Processor -- start 2023-10-03T10:15:00.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.337+0000 7f1157039700 1 -- start start 2023-10-03T10:15:00.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.338+0000 7f4274d43700 1 Processor -- start 2023-10-03T10:15:00.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.338+0000 7f4274d43700 1 -- start start 2023-10-03T10:15:00.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.338+0000 7f4274d43700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f427006c2f0 0x7f4270129430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:00.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.338+0000 7f4274d43700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f427009dc30 0x7f4270129970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:00.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.338+0000 7f4274d43700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4270137ef0 0x7f4270129eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:00.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.338+0000 7f4274d43700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427006ef70 con 0x7f4270137ef0 2023-10-03T10:15:00.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.338+0000 7f4274d43700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f427006e0a0 con 0x7f427006c2f0 2023-10-03T10:15:00.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.338+0000 7f4274d43700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f427013c710 con 0x7f427009dc30 2023-10-03T10:15:00.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.338+0000 7f1157039700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f115009e930 0x7f11501c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:00.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.338+0000 7f1157039700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f115009f240 0x7f11501c4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:00.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.338+0000 7f1157039700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f115009f650 0x7f11501c07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:00.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.338+0000 7f1157039700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11500a3410 con 0x7f115009e930 2023-10-03T10:15:00.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.338+0000 7f1157039700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1150071d20 con 0x7f115009f240 2023-10-03T10:15:00.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.338+0000 7f1157039700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11501a6030 con 0x7f115009f650 2023-10-03T10:15:00.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.339+0000 7f114ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f115009f240 0x7f11501c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:00.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.339+0000 7f114ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f115009f240 0x7f11501c4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42724/0 (socket says 172.21.15.3:42724) 2023-10-03T10:15:00.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.339+0000 7f114ffff700 1 -- 172.21.15.3:0/4087013335 learned_addr learned my addr 172.21.15.3:0/4087013335 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:00.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.338+0000 7f426e59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f427006c2f0 0x7f4270129430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:00.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.339+0000 7f426e59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f427006c2f0 0x7f4270129430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42708/0 (socket says 172.21.15.3:42708) 2023-10-03T10:15:00.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.339+0000 7f426e59c700 1 -- 172.21.15.3:0/2008827465 learned_addr learned my addr 172.21.15.3:0/2008827465 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:00.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.339+0000 7f11555d6700 1 --2- 172.21.15.3:0/4087013335 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f115009f650 0x7f11501c07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:00.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.339+0000 7f426dd9b700 1 --2- 172.21.15.3:0/2008827465 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f427009dc30 0x7f4270129970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:00.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.339+0000 7f426ed9d700 1 --2- 172.21.15.3:0/2008827465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4270137ef0 0x7f4270129eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:00.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.339+0000 7f426dd9b700 1 -- 172.21.15.3:0/2008827465 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f427006c2f0 msgr2=0x7f4270129430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:00.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.339+0000 7f426dd9b700 1 --2- 172.21.15.3:0/2008827465 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f427006c2f0 0x7f4270129430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.339+0000 7f426dd9b700 1 -- 172.21.15.3:0/2008827465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4270137ef0 msgr2=0x7f4270129eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:00.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.339+0000 7f426dd9b700 1 --2- 172.21.15.3:0/2008827465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4270137ef0 0x7f4270129eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.339+0000 7f426dd9b700 1 -- 172.21.15.3:0/2008827465 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42640094a0 con 0x7f427009dc30 2023-10-03T10:15:00.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.339+0000 7f426ed9d700 1 --2- 172.21.15.3:0/2008827465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4270137ef0 0x7f4270129eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:00.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.339+0000 7f114ffff700 1 -- 172.21.15.3:0/4087013335 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f115009f650 msgr2=0x7f11501c07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:00.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.339+0000 7f114ffff700 1 --2- 172.21.15.3:0/4087013335 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f115009f650 0x7f11501c07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.339+0000 7f114ffff700 1 -- 172.21.15.3:0/4087013335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f115009e930 msgr2=0x7f11501c42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:00.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.339+0000 7f114ffff700 1 --2- 172.21.15.3:0/4087013335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f115009e930 0x7f11501c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.339+0000 7f114ffff700 1 -- 172.21.15.3:0/4087013335 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1148009b30 con 0x7f115009f240 2023-10-03T10:15:00.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.340+0000 7f426dd9b700 1 --2- 172.21.15.3:0/2008827465 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f427009dc30 0x7f4270129970 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f427006aee0 tx=0x7f4264004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:00.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.340+0000 7f114ffff700 1 --2- 172.21.15.3:0/4087013335 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f115009f240 0x7f11501c4810 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f113c00c500 tx=0x7f113c00c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:00.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.340+0000 7f425effd700 1 -- 172.21.15.3:0/2008827465 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42640043c0 con 0x7f427009dc30 2023-10-03T10:15:00.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.340+0000 7f114d7fa700 1 -- 172.21.15.3:0/4087013335 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c011790 con 0x7f115009f240 2023-10-03T10:15:00.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.340+0000 7f425effd700 1 -- 172.21.15.3:0/2008827465 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426401f070 con 0x7f427009dc30 2023-10-03T10:15:00.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.340+0000 7f4274d43700 1 -- 172.21.15.3:0/2008827465 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f427012af70 con 0x7f427009dc30 2023-10-03T10:15:00.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.340+0000 7f425effd700 1 -- 172.21.15.3:0/2008827465 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42640036a0 con 0x7f427009dc30 2023-10-03T10:15:00.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.340+0000 7f4274d43700 1 -- 172.21.15.3:0/2008827465 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f427012b420 con 0x7f427009dc30 2023-10-03T10:15:00.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.340+0000 7f1157039700 1 -- 172.21.15.3:0/4087013335 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11500a0190 con 0x7f115009f240 2023-10-03T10:15:00.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.340+0000 7f114d7fa700 1 -- 172.21.15.3:0/4087013335 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113c0118f0 con 0x7f115009f240 2023-10-03T10:15:00.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.340+0000 7f114d7fa700 1 -- 172.21.15.3:0/4087013335 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c011ca0 con 0x7f115009f240 2023-10-03T10:15:00.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.340+0000 7f1157039700 1 -- 172.21.15.3:0/4087013335 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11501a5330 con 0x7f115009f240 2023-10-03T10:15:00.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.341+0000 7f114d7fa700 1 -- 172.21.15.3:0/4087013335 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f113c01c460 con 0x7f115009f240 2023-10-03T10:15:00.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.341+0000 7f4274d43700 1 -- 172.21.15.3:0/2008827465 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42700995b0 con 0x7f427009dc30 2023-10-03T10:15:00.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.341+0000 7f114d7fa700 1 -- 172.21.15.3:0/4087013335 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f113c03f4c0 con 0x7f115009f240 2023-10-03T10:15:00.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.341+0000 7f1157039700 1 -- 172.21.15.3:0/4087013335 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11501a6530 con 0x7f115009f240 2023-10-03T10:15:00.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.342+0000 7f425effd700 1 -- 172.21.15.3:0/2008827465 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4264021030 con 0x7f427009dc30 2023-10-03T10:15:00.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.342+0000 7f425effd700 1 -- 172.21.15.3:0/2008827465 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4264040d90 con 0x7f427009dc30 2023-10-03T10:15:00.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.346+0000 7f114d7fa700 1 -- 172.21.15.3:0/4087013335 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f11501a6530 con 0x7f115009f240 2023-10-03T10:15:00.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.351+0000 7f425effd700 1 -- 172.21.15.3:0/2008827465 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f42640239c0 con 0x7f427009dc30 2023-10-03T10:15:00.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.367+0000 7f24ec2d8700 1 -- 172.21.15.3:0/2462368559 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f24e40a29d0 con 0x7f24e4139170 2023-10-03T10:15:00.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.368+0000 7f24daffd700 1 -- 172.21.15.3:0/2462368559 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f24dc0243b0 con 0x7f24e4139170 2023-10-03T10:15:00.368 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:00.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.370+0000 7f24ec2d8700 1 -- 172.21.15.3:0/2462368559 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24e4139170 msgr2=0x7f24e411e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:00.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.370+0000 7f24ec2d8700 1 --2- 172.21.15.3:0/2462368559 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24e4139170 0x7f24e411e8e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f24dc004270 tx=0x7f24dc004ad0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.370+0000 7f24ec2d8700 1 -- 172.21.15.3:0/2462368559 shutdown_connections 2023-10-03T10:15:00.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.370+0000 7f24ec2d8700 1 --2- 172.21.15.3:0/2462368559 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24e4139170 0x7f24e411e8e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.370+0000 7f24ec2d8700 1 --2- 172.21.15.3:0/2462368559 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24e409d6f0 0x7f24e41c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.370+0000 7f24ec2d8700 1 --2- 172.21.15.3:0/2462368559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24e409e1d0 0x7f24e41c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.370+0000 7f24ec2d8700 1 -- 172.21.15.3:0/2462368559 >> 172.21.15.3:0/2462368559 conn(0x7f24e4066f50 msgr2=0x7f24e40a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:00.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.370+0000 7f24ec2d8700 1 -- 172.21.15.3:0/2462368559 shutdown_connections 2023-10-03T10:15:00.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.370+0000 7f24ec2d8700 1 -- 172.21.15.3:0/2462368559 wait complete. 2023-10-03T10:15:00.385 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:00.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.520+0000 7fce27a8f700 1 -- 172.21.15.3:0/4087600621 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fce08005590 con 0x7fce2009e000 2023-10-03T10:15:00.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.520+0000 7fce167fc700 1 -- 172.21.15.3:0/4087600621 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fce10015070 con 0x7fce2009e000 2023-10-03T10:15:00.521 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:00.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.522+0000 7fce27a8f700 1 -- 172.21.15.3:0/4087600621 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce2009e000 msgr2=0x7fce201c5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:00.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.522+0000 7fce27a8f700 1 --2- 172.21.15.3:0/4087600621 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce2009e000 0x7fce201c5860 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fce100090b0 tx=0x7fce10004750 comp rx=0 tx=0).stop 2023-10-03T10:15:00.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.523+0000 7fce27a8f700 1 -- 172.21.15.3:0/4087600621 shutdown_connections 2023-10-03T10:15:00.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.523+0000 7fce27a8f700 1 --2- 172.21.15.3:0/4087600621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce2009e410 0x7fce201c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.523+0000 7fce27a8f700 1 --2- 172.21.15.3:0/4087600621 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce2009d6f0 0x7fce201c5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.523+0000 7fce27a8f700 1 --2- 172.21.15.3:0/4087600621 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce2009e000 0x7fce201c5860 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.523+0000 7fce27a8f700 1 -- 172.21.15.3:0/4087600621 >> 172.21.15.3:0/4087600621 conn(0x7fce20066f50 msgr2=0x7fce201bb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:00.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.523+0000 7fce27a8f700 1 -- 172.21.15.3:0/4087600621 shutdown_connections 2023-10-03T10:15:00.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.523+0000 7fce27a8f700 1 -- 172.21.15.3:0/4087600621 wait complete. 2023-10-03T10:15:00.537 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:00.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.559+0000 7f1157039700 1 -- 172.21.15.3:0/4087013335 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f11501a6530 con 0x7f115009f240 2023-10-03T10:15:00.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.559+0000 7f4274d43700 1 -- 172.21.15.3:0/2008827465 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f42700747f0 con 0x7f427009dc30 2023-10-03T10:15:00.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.559+0000 7f114d7fa700 1 -- 172.21.15.3:0/4087013335 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f11501a6530 con 0x7f115009f240 2023-10-03T10:15:00.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.559+0000 7f425effd700 1 -- 172.21.15.3:0/2008827465 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4264015030 con 0x7f427009dc30 2023-10-03T10:15:00.560 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:00.561 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:00.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.562+0000 7f4274d43700 1 -- 172.21.15.3:0/2008827465 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f427009dc30 msgr2=0x7f4270129970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:00.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.562+0000 7f4274d43700 1 --2- 172.21.15.3:0/2008827465 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f427009dc30 0x7f4270129970 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f427006aee0 tx=0x7f4264004750 comp rx=0 tx=0).stop 2023-10-03T10:15:00.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.562+0000 7f4274d43700 1 -- 172.21.15.3:0/2008827465 shutdown_connections 2023-10-03T10:15:00.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.562+0000 7f4274d43700 1 --2- 172.21.15.3:0/2008827465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4270137ef0 0x7f4270129eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.562+0000 7f4274d43700 1 --2- 172.21.15.3:0/2008827465 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f427006c2f0 0x7f4270129430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.562+0000 7f4274d43700 1 --2- 172.21.15.3:0/2008827465 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f427009dc30 0x7f4270129970 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.562+0000 7f4274d43700 1 -- 172.21.15.3:0/2008827465 >> 172.21.15.3:0/2008827465 conn(0x7f427009f470 msgr2=0x7f42701393c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:00.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.562+0000 7f4274d43700 1 -- 172.21.15.3:0/2008827465 shutdown_connections 2023-10-03T10:15:00.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.562+0000 7f4274d43700 1 -- 172.21.15.3:0/2008827465 wait complete. 2023-10-03T10:15:00.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.562+0000 7f1157039700 1 -- 172.21.15.3:0/4087013335 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f115009f240 msgr2=0x7f11501c4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:00.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.562+0000 7f1157039700 1 --2- 172.21.15.3:0/4087013335 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f115009f240 0x7f11501c4810 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f113c00c500 tx=0x7f113c00c810 comp rx=0 tx=0).stop 2023-10-03T10:15:00.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.562+0000 7f1157039700 1 -- 172.21.15.3:0/4087013335 shutdown_connections 2023-10-03T10:15:00.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.562+0000 7f1157039700 1 --2- 172.21.15.3:0/4087013335 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f115009f650 0x7f11501c07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.562+0000 7f1157039700 1 --2- 172.21.15.3:0/4087013335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f115009e930 0x7f11501c42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.562+0000 7f1157039700 1 --2- 172.21.15.3:0/4087013335 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f115009f240 0x7f11501c4810 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:00.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.562+0000 7f1157039700 1 -- 172.21.15.3:0/4087013335 >> 172.21.15.3:0/4087013335 conn(0x7f11500a4210 msgr2=0x7f11501b68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:00.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.562+0000 7f1157039700 1 -- 172.21.15.3:0/4087013335 shutdown_connections 2023-10-03T10:15:00.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:00.562+0000 7f1157039700 1 -- 172.21.15.3:0/4087013335 wait complete. 2023-10-03T10:15:00.577 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:00.578 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:01.386 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:01.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.534+0000 7f8a893d3700 1 Processor -- start 2023-10-03T10:15:01.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.534+0000 7f8a893d3700 1 -- start start 2023-10-03T10:15:01.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.534+0000 7f8a893d3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8a8409d6f0 0x7f8a8409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:01.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.534+0000 7f8a893d3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a8409e000 0x7f8a8413bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:01.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.534+0000 7f8a893d3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a8413c470 0x7f8a8413e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:01.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.534+0000 7f8a893d3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a840992c0 con 0x7f8a8409d6f0 2023-10-03T10:15:01.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.534+0000 7f8a893d3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a84099520 con 0x7f8a8409e000 2023-10-03T10:15:01.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.534+0000 7f8a893d3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a840997e0 con 0x7f8a8413c470 2023-10-03T10:15:01.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.534+0000 7f8a837fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a8413c470 0x7f8a8413e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:01.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.534+0000 7f8a82ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8a8409d6f0 0x7f8a8409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:01.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.534+0000 7f8a827fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a8409e000 0x7f8a8413bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:01.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.534+0000 7f8a82ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8a8409d6f0 0x7f8a8409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41316/0 (socket says 172.21.15.3:41316) 2023-10-03T10:15:01.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.534+0000 7f8a837fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a8413c470 0x7f8a8413e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56256/0 (socket says 172.21.15.3:56256) 2023-10-03T10:15:01.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.534+0000 7f8a82ffd700 1 -- 172.21.15.3:0/3664979902 learned_addr learned my addr 172.21.15.3:0/3664979902 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:01.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.535+0000 7f8a837fe700 1 -- 172.21.15.3:0/3664979902 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a8409e000 msgr2=0x7f8a8413bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:01.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.535+0000 7f8a837fe700 1 --2- 172.21.15.3:0/3664979902 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a8409e000 0x7f8a8413bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.535+0000 7f8a837fe700 1 -- 172.21.15.3:0/3664979902 >> v2:172.21.15.3:3300/0 conn(0x7f8a8409d6f0 msgr2=0x7f8a8409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:01.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.535+0000 7f8a837fe700 1 --2- 172.21.15.3:0/3664979902 >> v2:172.21.15.3:3300/0 conn(0x7f8a8409d6f0 0x7f8a8409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.535+0000 7f8a837fe700 1 -- 172.21.15.3:0/3664979902 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a84099b30 con 0x7f8a8413c470 2023-10-03T10:15:01.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.535+0000 7f8a82ffd700 1 --2- 172.21.15.3:0/3664979902 >> v2:172.21.15.3:3300/0 conn(0x7f8a8409d6f0 0x7f8a8409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:01.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.535+0000 7f8a827fc700 1 --2- 172.21.15.3:0/3664979902 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a8409e000 0x7f8a8413bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:01.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.535+0000 7f8a837fe700 1 --2- 172.21.15.3:0/3664979902 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a8413c470 0x7f8a8413e850 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009dc0 tx=0x7f8a7400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8a159702f6bdab8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:01.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.535+0000 7f8a81ffb700 1 -- 172.21.15.3:0/3664979902 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7400e070 con 0x7f8a8413c470 2023-10-03T10:15:01.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.535+0000 7f8a81ffb700 1 -- 172.21.15.3:0/3664979902 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7400bae0 con 0x7f8a8413c470 2023-10-03T10:15:01.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.535+0000 7f8a81ffb700 1 -- 172.21.15.3:0/3664979902 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7400be90 con 0x7f8a8413c470 2023-10-03T10:15:01.538 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:01.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.536+0000 7f8a893d3700 1 -- 172.21.15.3:0/3664979902 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a8413c470 msgr2=0x7f8a8413e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:01.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.536+0000 7f8a893d3700 1 --2- 172.21.15.3:0/3664979902 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a8413c470 0x7f8a8413e850 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009dc0 tx=0x7f8a7400b330 comp rx=0 tx=0).stop 2023-10-03T10:15:01.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.536+0000 7f8a893d3700 1 -- 172.21.15.3:0/3664979902 shutdown_connections 2023-10-03T10:15:01.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.536+0000 7f8a893d3700 1 --2- 172.21.15.3:0/3664979902 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a8413c470 0x7f8a8413e850 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.536+0000 7f8a893d3700 1 --2- 172.21.15.3:0/3664979902 >> v2:172.21.15.3:3300/0 conn(0x7f8a8409d6f0 0x7f8a8409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.536+0000 7f8a893d3700 1 --2- 172.21.15.3:0/3664979902 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a8409e000 0x7f8a8413bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.536+0000 7f8a893d3700 1 -- 172.21.15.3:0/3664979902 >> 172.21.15.3:0/3664979902 conn(0x7f8a8409f470 msgr2=0x7f8a840a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:01.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.536+0000 7f8a893d3700 1 -- 172.21.15.3:0/3664979902 shutdown_connections 2023-10-03T10:15:01.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.536+0000 7f8a893d3700 1 -- 172.21.15.3:0/3664979902 wait complete. 2023-10-03T10:15:01.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.537+0000 7f8a893d3700 1 Processor -- start 2023-10-03T10:15:01.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.537+0000 7f8a893d3700 1 -- start start 2023-10-03T10:15:01.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.537+0000 7f8a893d3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a8409d6f0 0x7f8a840704f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:01.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.538+0000 7f8a893d3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a8409e000 0x7f8a84070a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:01.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.538+0000 7f8a893d3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a8413c470 0x7f8a84070f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:01.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.538+0000 7f8a893d3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a841287d0 con 0x7f8a8413c470 2023-10-03T10:15:01.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.538+0000 7f8a893d3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a84098c70 con 0x7f8a8409d6f0 2023-10-03T10:15:01.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.538+0000 7f8a893d3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8413c1a0 con 0x7f8a8409e000 2023-10-03T10:15:01.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.538+0000 7f8a82ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a8409d6f0 0x7f8a840704f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:01.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.538+0000 7f8a827fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a8409e000 0x7f8a84070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:01.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.538+0000 7f8a82ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a8409d6f0 0x7f8a840704f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42740/0 (socket says 172.21.15.3:42740) 2023-10-03T10:15:01.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.538+0000 7f8a82ffd700 1 -- 172.21.15.3:0/3006034839 learned_addr learned my addr 172.21.15.3:0/3006034839 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:01.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.538+0000 7f8a837fe700 1 --2- 172.21.15.3:0/3006034839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a8413c470 0x7f8a84070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:01.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.538+0000 7f8a82ffd700 1 -- 172.21.15.3:0/3006034839 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a8409e000 msgr2=0x7f8a84070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:01.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.538+0000 7f8a82ffd700 1 --2- 172.21.15.3:0/3006034839 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a8409e000 0x7f8a84070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.538+0000 7f8a82ffd700 1 -- 172.21.15.3:0/3006034839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a8413c470 msgr2=0x7f8a84070f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:01.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.538+0000 7f8a82ffd700 1 --2- 172.21.15.3:0/3006034839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a8413c470 0x7f8a84070f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.538+0000 7f8a82ffd700 1 -- 172.21.15.3:0/3006034839 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7c009550 con 0x7f8a8409d6f0 2023-10-03T10:15:01.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.538+0000 7f8a827fc700 1 --2- 172.21.15.3:0/3006034839 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a8409e000 0x7f8a84070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:01.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.538+0000 7f8a837fe700 1 --2- 172.21.15.3:0/3006034839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a8413c470 0x7f8a84070f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:01.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.538+0000 7f8a82ffd700 1 --2- 172.21.15.3:0/3006034839 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a8409d6f0 0x7f8a840704f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c00ebe0 tx=0x7f8a7c00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:01.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.538+0000 7f8a7b7fe700 1 -- 172.21.15.3:0/3006034839 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c00c850 con 0x7f8a8409d6f0 2023-10-03T10:15:01.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.538+0000 7f8a7b7fe700 1 -- 172.21.15.3:0/3006034839 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7c00c9b0 con 0x7f8a8409d6f0 2023-10-03T10:15:01.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.538+0000 7f8a7b7fe700 1 -- 172.21.15.3:0/3006034839 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c00cd40 con 0x7f8a8409d6f0 2023-10-03T10:15:01.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.538+0000 7f8a893d3700 1 -- 172.21.15.3:0/3006034839 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a6c0094a0 con 0x7f8a8409d6f0 2023-10-03T10:15:01.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.539+0000 7f8a893d3700 1 -- 172.21.15.3:0/3006034839 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a74009a70 con 0x7f8a8409d6f0 2023-10-03T10:15:01.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.539+0000 7f8a7b7fe700 1 -- 172.21.15.3:0/3006034839 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8a7c013470 con 0x7f8a8409d6f0 2023-10-03T10:15:01.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.539+0000 7f8a893d3700 1 -- 172.21.15.3:0/3006034839 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a84128190 con 0x7f8a8409d6f0 2023-10-03T10:15:01.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.540+0000 7f8a7b7fe700 1 -- 172.21.15.3:0/3006034839 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8a7c03f9c0 con 0x7f8a8409d6f0 2023-10-03T10:15:01.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.545+0000 7f8a7b7fe700 1 -- 172.21.15.3:0/3006034839 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8a7c017070 con 0x7f8a8409d6f0 2023-10-03T10:15:01.578 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:01.579 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:01.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.685+0000 7fa5d3e78700 1 Processor -- start 2023-10-03T10:15:01.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.685+0000 7fa5d3e78700 1 -- start start 2023-10-03T10:15:01.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.686+0000 7fa5d3e78700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa5cc131810 0x7fa5cc131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:01.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.686+0000 7fa5d3e78700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5cc099a40 0x7fa5cc099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:01.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.686+0000 7fa5d3e78700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa5cc09a3b0 0x7fa5cc138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:01.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.686+0000 7fa5d3e78700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5cc09b580 con 0x7fa5cc09a3b0 2023-10-03T10:15:01.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.686+0000 7fa5d3e78700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc1313c0 con 0x7fa5cc099a40 2023-10-03T10:15:01.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.686+0000 7fa5d3e78700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc06e8c0 con 0x7fa5cc131810 2023-10-03T10:15:01.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.686+0000 7fa5d2415700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa5cc09a3b0 0x7fa5cc138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:01.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.686+0000 7fa5d2415700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa5cc09a3b0 0x7fa5cc138650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41326/0 (socket says 172.21.15.3:41326) 2023-10-03T10:15:01.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.686+0000 7fa5d2415700 1 -- 172.21.15.3:0/1793001789 learned_addr learned my addr 172.21.15.3:0/1793001789 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:01.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.686+0000 7fa5d1413700 1 --2- 172.21.15.3:0/1793001789 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5cc099a40 0x7fa5cc099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:01.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.687+0000 7fa5d1c14700 1 --2- 172.21.15.3:0/1793001789 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa5cc131810 0x7fa5cc131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:01.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.687+0000 7fa5d2415700 1 -- 172.21.15.3:0/1793001789 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa5cc131810 msgr2=0x7fa5cc131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:01.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.687+0000 7fa5d2415700 1 --2- 172.21.15.3:0/1793001789 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa5cc131810 0x7fa5cc131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.687+0000 7fa5d2415700 1 -- 172.21.15.3:0/1793001789 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5cc099a40 msgr2=0x7fa5cc099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:01.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.687+0000 7fa5d2415700 1 --2- 172.21.15.3:0/1793001789 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5cc099a40 0x7fa5cc099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.687+0000 7fa5d2415700 1 -- 172.21.15.3:0/1793001789 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc09a820 con 0x7fa5cc09a3b0 2023-10-03T10:15:01.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.687+0000 7fa5d2415700 1 --2- 172.21.15.3:0/1793001789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5cc09a3b0 0x7fa5cc138650 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009e50 tx=0x7fa5c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef9281c338efed2 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:01.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.687+0000 7fa5d0c12700 1 -- 172.21.15.3:0/1793001789 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c400e070 con 0x7fa5cc09a3b0 2023-10-03T10:15:01.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.687+0000 7fa5d0c12700 1 -- 172.21.15.3:0/1793001789 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c400b910 con 0x7fa5cc09a3b0 2023-10-03T10:15:01.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.687+0000 7fa5d0c12700 1 -- 172.21.15.3:0/1793001789 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c400bcc0 con 0x7fa5cc09a3b0 2023-10-03T10:15:01.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.688+0000 7fa5d3e78700 1 -- 172.21.15.3:0/1793001789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5cc09a3b0 msgr2=0x7fa5cc138650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:01.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.688+0000 7fa5d3e78700 1 --2- 172.21.15.3:0/1793001789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5cc09a3b0 0x7fa5cc138650 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009e50 tx=0x7fa5c400b330 comp rx=0 tx=0).stop 2023-10-03T10:15:01.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.688+0000 7fa5d3e78700 1 -- 172.21.15.3:0/1793001789 shutdown_connections 2023-10-03T10:15:01.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.688+0000 7fa5d3e78700 1 --2- 172.21.15.3:0/1793001789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5cc09a3b0 0x7fa5cc138650 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.688+0000 7fa5d3e78700 1 --2- 172.21.15.3:0/1793001789 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa5cc131810 0x7fa5cc131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.688+0000 7fa5d3e78700 1 --2- 172.21.15.3:0/1793001789 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5cc099a40 0x7fa5cc099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.688+0000 7fa5d3e78700 1 -- 172.21.15.3:0/1793001789 >> 172.21.15.3:0/1793001789 conn(0x7fa5cc0a2aa0 msgr2=0x7fa5cc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:01.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.688+0000 7fa5d3e78700 1 -- 172.21.15.3:0/1793001789 shutdown_connections 2023-10-03T10:15:01.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.688+0000 7fa5d3e78700 1 -- 172.21.15.3:0/1793001789 wait complete. 2023-10-03T10:15:01.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.690+0000 7fa5d3e78700 1 Processor -- start 2023-10-03T10:15:01.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.690+0000 7fa5d3e78700 1 -- start start 2023-10-03T10:15:01.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.691+0000 7fa5d3e78700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5cc099a40 0x7fa5cc134fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:01.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.691+0000 7fa5d3e78700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5cc09a3b0 0x7fa5cc1354f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:01.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.691+0000 7fa5d3e78700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa5cc131810 0x7fa5cc135a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:01.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.691+0000 7fa5d3e78700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc1382b0 con 0x7fa5cc099a40 2023-10-03T10:15:01.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.691+0000 7fa5d3e78700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc136d90 con 0x7fa5cc09a3b0 2023-10-03T10:15:01.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.691+0000 7fa5d3e78700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc0717e0 con 0x7fa5cc131810 2023-10-03T10:15:01.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.691+0000 7fa5d1413700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5cc09a3b0 0x7fa5cc1354f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:01.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.691+0000 7fa5d1c14700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5cc099a40 0x7fa5cc134fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:01.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.691+0000 7fa5d1413700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5cc09a3b0 0x7fa5cc1354f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42754/0 (socket says 172.21.15.3:42754) 2023-10-03T10:15:01.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.691+0000 7fa5d1413700 1 -- 172.21.15.3:0/4214233385 learned_addr learned my addr 172.21.15.3:0/4214233385 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:01.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.691+0000 7fa5d2415700 1 --2- 172.21.15.3:0/4214233385 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa5cc131810 0x7fa5cc135a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:01.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.691+0000 7fa5d1413700 1 -- 172.21.15.3:0/4214233385 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa5cc131810 msgr2=0x7fa5cc135a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:01.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.691+0000 7fa5d1413700 1 --2- 172.21.15.3:0/4214233385 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa5cc131810 0x7fa5cc135a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.691+0000 7fa5d1413700 1 -- 172.21.15.3:0/4214233385 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5cc099a40 msgr2=0x7fa5cc134fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:01.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.691+0000 7fa5d1413700 1 --2- 172.21.15.3:0/4214233385 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5cc099a40 0x7fa5cc134fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.691+0000 7fa5d1413700 1 -- 172.21.15.3:0/4214233385 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c4009b30 con 0x7fa5cc09a3b0 2023-10-03T10:15:01.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.691+0000 7fa5d1c14700 1 --2- 172.21.15.3:0/4214233385 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5cc099a40 0x7fa5cc134fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:01.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.692+0000 7fa5d1413700 1 --2- 172.21.15.3:0/4214233385 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5cc09a3b0 0x7fa5cc1354f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00e9e0 tx=0x7fa5bc00ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:01.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.692+0000 7fa5c27fc700 1 -- 172.21.15.3:0/4214233385 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc009710 con 0x7fa5cc09a3b0 2023-10-03T10:15:01.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.692+0000 7fa5c27fc700 1 -- 172.21.15.3:0/4214233385 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5bc00c850 con 0x7fa5cc09a3b0 2023-10-03T10:15:01.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.692+0000 7fa5d3e78700 1 -- 172.21.15.3:0/4214233385 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5cc136b90 con 0x7fa5cc09a3b0 2023-10-03T10:15:01.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.692+0000 7fa5c27fc700 1 -- 172.21.15.3:0/4214233385 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc00cbe0 con 0x7fa5cc09a3b0 2023-10-03T10:15:01.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.692+0000 7fa5d3e78700 1 -- 172.21.15.3:0/4214233385 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5cc1365a0 con 0x7fa5cc09a3b0 2023-10-03T10:15:01.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.693+0000 7fa5c27fc700 1 -- 172.21.15.3:0/4214233385 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa5bc00cd40 con 0x7fa5cc09a3b0 2023-10-03T10:15:01.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.693+0000 7fa5d3e78700 1 -- 172.21.15.3:0/4214233385 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5cc074ea0 con 0x7fa5cc09a3b0 2023-10-03T10:15:01.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.693+0000 7fa5c27fc700 1 -- 172.21.15.3:0/4214233385 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa5bc03f7c0 con 0x7fa5cc09a3b0 2023-10-03T10:15:01.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.702+0000 7fa5c27fc700 1 -- 172.21.15.3:0/4214233385 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa5bc0232f0 con 0x7fa5cc09a3b0 2023-10-03T10:15:01.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.730+0000 7ff2d9b94700 1 Processor -- start 2023-10-03T10:15:01.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.730+0000 7ff2d9b94700 1 -- start start 2023-10-03T10:15:01.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.731+0000 7ff2d9b94700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2d409d6f0 0x7ff2d409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:01.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.731+0000 7ff2d9b94700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2d409e000 0x7ff2d41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:01.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.731+0000 7ff2d9b94700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2d409b120 con 0x7ff2d409e410 2023-10-03T10:15:01.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.731+0000 7ff2d9b94700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d406b460 con 0x7ff2d409e000 2023-10-03T10:15:01.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.731+0000 7ff2d9b94700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d409a7e0 con 0x7ff2d409d6f0 2023-10-03T10:15:01.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.731+0000 7ff2d37fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2d409d6f0 0x7ff2d409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:01.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.731+0000 7ff2d37fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2d409d6f0 0x7ff2d409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56286/0 (socket says 172.21.15.3:56286) 2023-10-03T10:15:01.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.731+0000 7ff2d37fe700 1 -- 172.21.15.3:0/3980920447 learned_addr learned my addr 172.21.15.3:0/3980920447 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:01.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.731+0000 7ff2d37fe700 1 -- 172.21.15.3:0/3980920447 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2d409e000 msgr2=0x7ff2d41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:01.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.731+0000 7ff2d37fe700 1 --2- 172.21.15.3:0/3980920447 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2d409e000 0x7ff2d41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.731+0000 7ff2d37fe700 1 -- 172.21.15.3:0/3980920447 >> v1:172.21.15.3:6789/0 conn(0x7ff2d409e410 legacy=0x7ff2d4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:01.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.731+0000 7ff2d37fe700 1 -- 172.21.15.3:0/3980920447 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d409a500 con 0x7ff2d409d6f0 2023-10-03T10:15:01.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.732+0000 7ff2d37fe700 1 --2- 172.21.15.3:0/3980920447 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2d409d6f0 0x7ff2d409dac0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4009950 tx=0x7ff2c4009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2f1433b4f27f881 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:01.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.732+0000 7ff2d27fc700 1 -- 172.21.15.3:0/3980920447 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4009ea0 con 0x7ff2d409d6f0 2023-10-03T10:15:01.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.732+0000 7ff2d27fc700 1 -- 172.21.15.3:0/3980920447 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c400ba20 con 0x7ff2d409d6f0 2023-10-03T10:15:01.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.732+0000 7ff2d27fc700 1 -- 172.21.15.3:0/3980920447 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c400bdb0 con 0x7ff2d409d6f0 2023-10-03T10:15:01.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.733+0000 7ff2d9b94700 1 -- 172.21.15.3:0/3980920447 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2d409d6f0 msgr2=0x7ff2d409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:01.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.733+0000 7ff2d9b94700 1 --2- 172.21.15.3:0/3980920447 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2d409d6f0 0x7ff2d409dac0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4009950 tx=0x7ff2c4009c60 comp rx=0 tx=0).stop 2023-10-03T10:15:01.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.733+0000 7ff2d9b94700 1 -- 172.21.15.3:0/3980920447 shutdown_connections 2023-10-03T10:15:01.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.733+0000 7ff2d9b94700 1 --2- 172.21.15.3:0/3980920447 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2d409d6f0 0x7ff2d409dac0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.733+0000 7ff2d9b94700 1 --2- 172.21.15.3:0/3980920447 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2d409e000 0x7ff2d41339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.733+0000 7ff2d9b94700 1 -- 172.21.15.3:0/3980920447 >> 172.21.15.3:0/3980920447 conn(0x7ff2d4066f50 msgr2=0x7ff2d409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:01.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.733+0000 7ff2d9b94700 1 -- 172.21.15.3:0/3980920447 shutdown_connections 2023-10-03T10:15:01.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.733+0000 7ff2d9b94700 1 -- 172.21.15.3:0/3980920447 wait complete. 2023-10-03T10:15:01.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.734+0000 7ff2d9b94700 1 Processor -- start 2023-10-03T10:15:01.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.734+0000 7ff2d9b94700 1 -- start start 2023-10-03T10:15:01.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.735+0000 7fa841226700 1 Processor -- start 2023-10-03T10:15:01.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.735+0000 7ff2d9b94700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2d409d6f0 0x7ff2d41225c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:01.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.735+0000 7ff2d9b94700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2d409e000 0x7ff2d4122b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:01.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.735+0000 7ff2d9b94700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2d409e410 0x7ff2d4123040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:01.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.735+0000 7ff2d9b94700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d4133860 con 0x7ff2d409d6f0 2023-10-03T10:15:01.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.735+0000 7ff2d9b94700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d4125fa0 con 0x7ff2d409e410 2023-10-03T10:15:01.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.735+0000 7ff2d9b94700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d40a2df0 con 0x7ff2d409e000 2023-10-03T10:15:01.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.735+0000 7fa841226700 1 -- start start 2023-10-03T10:15:01.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.735+0000 7ff2d37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2d409d6f0 0x7ff2d41225c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:01.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.735+0000 7ff2d37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2d409d6f0 0x7ff2d41225c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41342/0 (socket says 172.21.15.3:41342) 2023-10-03T10:15:01.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.735+0000 7ff2d37fe700 1 -- 172.21.15.3:0/3696709996 learned_addr learned my addr 172.21.15.3:0/3696709996 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:01.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.735+0000 7ff2d2ffd700 1 --2- 172.21.15.3:0/3696709996 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2d409e000 0x7ff2d4122b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:01.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.736+0000 7ff2d37fe700 1 -- 172.21.15.3:0/3696709996 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2d409e000 msgr2=0x7ff2d4122b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:01.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.736+0000 7ff2d37fe700 1 --2- 172.21.15.3:0/3696709996 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2d409e000 0x7ff2d4122b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.736+0000 7ff2d37fe700 1 -- 172.21.15.3:0/3696709996 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2d409e410 msgr2=0x7ff2d4123040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:01.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.736+0000 7ff2d37fe700 1 --2- 172.21.15.3:0/3696709996 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2d409e410 0x7ff2d4123040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.736+0000 7ff2d37fe700 1 -- 172.21.15.3:0/3696709996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c4009550 con 0x7ff2d409d6f0 2023-10-03T10:15:01.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.736+0000 7ff2d2ffd700 1 --2- 172.21.15.3:0/3696709996 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2d409e000 0x7ff2d4122b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:01.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.736+0000 7ff2d37fe700 1 --2- 172.21.15.3:0/3696709996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2d409d6f0 0x7ff2d41225c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4009870 tx=0x7ff2c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:01.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.736+0000 7fa841226700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa83c09eca0 0x7fa83c13cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:01.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.736+0000 7fa841226700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa83c06f050 0x7fa83c13f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:01.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.736+0000 7fa841226700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa83c070ee0 con 0x7fa83c06e570 2023-10-03T10:15:01.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.736+0000 7fa841226700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c071980 con 0x7fa83c06f050 2023-10-03T10:15:01.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.736+0000 7fa841226700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c071480 con 0x7fa83c09eca0 2023-10-03T10:15:01.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.736+0000 7ff2bbfff700 1 -- 172.21.15.3:0/3696709996 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c400be90 con 0x7ff2d409d6f0 2023-10-03T10:15:01.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.736+0000 7ff2bbfff700 1 -- 172.21.15.3:0/3696709996 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c40036a0 con 0x7ff2d409d6f0 2023-10-03T10:15:01.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.736+0000 7ff2bbfff700 1 -- 172.21.15.3:0/3696709996 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4003a50 con 0x7ff2d409d6f0 2023-10-03T10:15:01.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.736+0000 7ff2d9b94700 1 -- 172.21.15.3:0/3696709996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d409c710 con 0x7ff2d409d6f0 2023-10-03T10:15:01.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.736+0000 7ff2d9b94700 1 -- 172.21.15.3:0/3696709996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d4124e20 con 0x7ff2d409d6f0 2023-10-03T10:15:01.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.736+0000 7fa83a59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa83c09eca0 0x7fa83c13cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:01.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.736+0000 7fa83a59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa83c09eca0 0x7fa83c13cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56310/0 (socket says 172.21.15.3:56310) 2023-10-03T10:15:01.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.736+0000 7fa83a59c700 1 -- 172.21.15.3:0/4246431151 learned_addr learned my addr 172.21.15.3:0/4246431151 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:01.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.737+0000 7fa83b59e700 1 --2- 172.21.15.3:0/4246431151 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa83c06f050 0x7fa83c13f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:01.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.737+0000 7fa83a59c700 1 -- 172.21.15.3:0/4246431151 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa83c06f050 msgr2=0x7fa83c13f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:01.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.737+0000 7fa83a59c700 1 --2- 172.21.15.3:0/4246431151 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa83c06f050 0x7fa83c13f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.737+0000 7fa83a59c700 1 -- 172.21.15.3:0/4246431151 >> v1:172.21.15.3:6789/0 conn(0x7fa83c06e570 legacy=0x7fa83c06e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:01.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.737+0000 7fa83a59c700 1 -- 172.21.15.3:0/4246431151 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83c071b90 con 0x7fa83c09eca0 2023-10-03T10:15:01.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.737+0000 7fa83a59c700 1 --2- 172.21.15.3:0/4246431151 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa83c09eca0 0x7fa83c13cea0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa830009920 tx=0x7fa830009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bd61dc85d22f622 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:01.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.737+0000 7fa839d9b700 1 -- 172.21.15.3:0/4246431151 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa830011830 con 0x7fa83c09eca0 2023-10-03T10:15:01.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.737+0000 7ff2bbfff700 1 -- 172.21.15.3:0/3696709996 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff2c4003bb0 con 0x7ff2d409d6f0 2023-10-03T10:15:01.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.737+0000 7fa839d9b700 1 -- 172.21.15.3:0/4246431151 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa830011990 con 0x7fa83c09eca0 2023-10-03T10:15:01.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.737+0000 7fa839d9b700 1 -- 172.21.15.3:0/4246431151 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa830011d00 con 0x7fa83c09eca0 2023-10-03T10:15:01.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.738+0000 7ff2bbfff700 1 -- 172.21.15.3:0/3696709996 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff2c4040ca0 con 0x7ff2d409d6f0 2023-10-03T10:15:01.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.738+0000 7ff2d9b94700 1 -- 172.21.15.3:0/3696709996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2d411f030 con 0x7ff2d409d6f0 2023-10-03T10:15:01.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.738+0000 7fa841226700 1 -- 172.21.15.3:0/4246431151 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa83c09eca0 msgr2=0x7fa83c13cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:01.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.738+0000 7fa841226700 1 --2- 172.21.15.3:0/4246431151 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa83c09eca0 0x7fa83c13cea0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa830009920 tx=0x7fa830009c30 comp rx=0 tx=0).stop 2023-10-03T10:15:01.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.738+0000 7fa841226700 1 -- 172.21.15.3:0/4246431151 shutdown_connections 2023-10-03T10:15:01.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.738+0000 7fa841226700 1 --2- 172.21.15.3:0/4246431151 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa83c06f050 0x7fa83c13f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.738+0000 7fa841226700 1 --2- 172.21.15.3:0/4246431151 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa83c09eca0 0x7fa83c13cea0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.738+0000 7fa841226700 1 -- 172.21.15.3:0/4246431151 >> 172.21.15.3:0/4246431151 conn(0x7fa83c0a2aa0 msgr2=0x7fa83c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:01.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.738+0000 7fa841226700 1 -- 172.21.15.3:0/4246431151 shutdown_connections 2023-10-03T10:15:01.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.738+0000 7fa841226700 1 -- 172.21.15.3:0/4246431151 wait complete. 2023-10-03T10:15:01.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.740+0000 7fa841226700 1 Processor -- start 2023-10-03T10:15:01.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.740+0000 7fa841226700 1 -- start start 2023-10-03T10:15:01.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.740+0000 7fa841226700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa83c06e570 0x7fa83c1373c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:01.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.740+0000 7fa841226700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa83c06f050 0x7fa83c137900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:01.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.740+0000 7fa841226700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa83c1d4eb0 0x7fa83c1d5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:01.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.740+0000 7fa841226700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c070610 con 0x7fa83c1d4eb0 2023-10-03T10:15:01.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.740+0000 7fa841226700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c136f70 con 0x7fa83c06f050 2023-10-03T10:15:01.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.740+0000 7fa841226700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c139b80 con 0x7fa83c06e570 2023-10-03T10:15:01.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.741+0000 7fa83b59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa83c1d4eb0 0x7fa83c1d5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:01.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.741+0000 7fa83b59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa83c1d4eb0 0x7fa83c1d5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41358/0 (socket says 172.21.15.3:41358) 2023-10-03T10:15:01.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.741+0000 7fa83b59e700 1 -- 172.21.15.3:0/967916564 learned_addr learned my addr 172.21.15.3:0/967916564 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:01.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.741+0000 7fa83ad9d700 1 --2- 172.21.15.3:0/967916564 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa83c06e570 0x7fa83c1373c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:01.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.741+0000 7fa83a59c700 1 --2- 172.21.15.3:0/967916564 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa83c06f050 0x7fa83c137900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:01.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.741+0000 7fa83b59e700 1 -- 172.21.15.3:0/967916564 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa83c06e570 msgr2=0x7fa83c1373c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:01.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.741+0000 7fa83b59e700 1 --2- 172.21.15.3:0/967916564 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa83c06e570 0x7fa83c1373c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.741+0000 7fa83b59e700 1 -- 172.21.15.3:0/967916564 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa83c06f050 msgr2=0x7fa83c137900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:01.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.741+0000 7fa83b59e700 1 --2- 172.21.15.3:0/967916564 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa83c06f050 0x7fa83c137900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.741+0000 7fa83b59e700 1 -- 172.21.15.3:0/967916564 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa830009550 con 0x7fa83c1d4eb0 2023-10-03T10:15:01.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.741+0000 7fa83b59e700 1 --2- 172.21.15.3:0/967916564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa83c1d4eb0 0x7fa83c1d5330 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa83400efc0 tx=0x7fa83400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:01.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.744+0000 7fa8237fe700 1 -- 172.21.15.3:0/967916564 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa834009ea0 con 0x7fa83c1d4eb0 2023-10-03T10:15:01.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.744+0000 7fa8237fe700 1 -- 172.21.15.3:0/967916564 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83400f040 con 0x7fa83c1d4eb0 2023-10-03T10:15:01.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.744+0000 7fa8237fe700 1 -- 172.21.15.3:0/967916564 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83400ceb0 con 0x7fa83c1d4eb0 2023-10-03T10:15:01.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.744+0000 7fa841226700 1 -- 172.21.15.3:0/967916564 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa83c13a4e0 con 0x7fa83c1d4eb0 2023-10-03T10:15:01.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.744+0000 7fa841226700 1 -- 172.21.15.3:0/967916564 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa83c138e20 con 0x7fa83c1d4eb0 2023-10-03T10:15:01.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.744+0000 7ff2bbfff700 1 -- 172.21.15.3:0/3696709996 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff2c4003e60 con 0x7ff2d409d6f0 2023-10-03T10:15:01.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.745+0000 7fa8237fe700 1 -- 172.21.15.3:0/967916564 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa834014480 con 0x7fa83c1d4eb0 2023-10-03T10:15:01.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.745+0000 7fa8237fe700 1 -- 172.21.15.3:0/967916564 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa83403fe20 con 0x7fa83c1d4eb0 2023-10-03T10:15:01.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.745+0000 7fa841226700 1 -- 172.21.15.3:0/967916564 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa83c133c90 con 0x7fa83c1d4eb0 2023-10-03T10:15:01.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.755+0000 7fa8237fe700 1 -- 172.21.15.3:0/967916564 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa834018020 con 0x7fa83c1d4eb0 2023-10-03T10:15:01.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.768+0000 7f8a893d3700 1 -- 172.21.15.3:0/3006034839 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8a84128430 con 0x7f8a8409d6f0 2023-10-03T10:15:01.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.768+0000 7f8a7b7fe700 1 -- 172.21.15.3:0/3006034839 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8a84128430 con 0x7f8a8409d6f0 2023-10-03T10:15:01.770 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:01.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.770+0000 7f8a893d3700 1 -- 172.21.15.3:0/3006034839 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a8409d6f0 msgr2=0x7f8a840704f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:01.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.770+0000 7f8a893d3700 1 --2- 172.21.15.3:0/3006034839 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a8409d6f0 0x7f8a840704f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c00ebe0 tx=0x7f8a7c00eef0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.770+0000 7f8a893d3700 1 -- 172.21.15.3:0/3006034839 shutdown_connections 2023-10-03T10:15:01.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.770+0000 7f8a893d3700 1 --2- 172.21.15.3:0/3006034839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a8413c470 0x7f8a84070f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.770+0000 7f8a893d3700 1 --2- 172.21.15.3:0/3006034839 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a8409d6f0 0x7f8a840704f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.770+0000 7f8a893d3700 1 --2- 172.21.15.3:0/3006034839 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a8409e000 0x7f8a84070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.770+0000 7f8a893d3700 1 -- 172.21.15.3:0/3006034839 >> 172.21.15.3:0/3006034839 conn(0x7f8a8409f470 msgr2=0x7f8a8413c840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:01.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.770+0000 7f8a893d3700 1 -- 172.21.15.3:0/3006034839 shutdown_connections 2023-10-03T10:15:01.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.770+0000 7f8a893d3700 1 -- 172.21.15.3:0/3006034839 wait complete. 2023-10-03T10:15:01.791 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:01.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.929+0000 7fa5d3e78700 1 -- 172.21.15.3:0/4214233385 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa5cc071d80 con 0x7fa5cc09a3b0 2023-10-03T10:15:01.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.930+0000 7fa5c27fc700 1 -- 172.21.15.3:0/4214233385 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa5bc021070 con 0x7fa5cc09a3b0 2023-10-03T10:15:01.930 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:01.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.932+0000 7fa5d3e78700 1 -- 172.21.15.3:0/4214233385 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5cc09a3b0 msgr2=0x7fa5cc1354f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:01.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.932+0000 7fa5d3e78700 1 --2- 172.21.15.3:0/4214233385 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5cc09a3b0 0x7fa5cc1354f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00e9e0 tx=0x7fa5bc00ecf0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.932+0000 7fa5d3e78700 1 -- 172.21.15.3:0/4214233385 shutdown_connections 2023-10-03T10:15:01.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.932+0000 7fa5d3e78700 1 --2- 172.21.15.3:0/4214233385 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa5cc131810 0x7fa5cc135a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.932+0000 7fa5d3e78700 1 --2- 172.21.15.3:0/4214233385 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5cc099a40 0x7fa5cc134fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.932+0000 7fa5d3e78700 1 --2- 172.21.15.3:0/4214233385 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5cc09a3b0 0x7fa5cc1354f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.932+0000 7fa5d3e78700 1 -- 172.21.15.3:0/4214233385 >> 172.21.15.3:0/4214233385 conn(0x7fa5cc0a2aa0 msgr2=0x7fa5cc12f220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:01.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.932+0000 7fa5d3e78700 1 -- 172.21.15.3:0/4214233385 shutdown_connections 2023-10-03T10:15:01.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.932+0000 7fa5d3e78700 1 -- 172.21.15.3:0/4214233385 wait complete. 2023-10-03T10:15:01.946 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:01.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.975+0000 7ff2d9b94700 1 -- 172.21.15.3:0/3696709996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff2d4074ea0 con 0x7ff2d409d6f0 2023-10-03T10:15:01.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.976+0000 7ff2bbfff700 1 -- 172.21.15.3:0/3696709996 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff2c4019ba0 con 0x7ff2d409d6f0 2023-10-03T10:15:01.976 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:01.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.978+0000 7ff2d9b94700 1 -- 172.21.15.3:0/3696709996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2d409d6f0 msgr2=0x7ff2d41225c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:01.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.978+0000 7ff2d9b94700 1 --2- 172.21.15.3:0/3696709996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2d409d6f0 0x7ff2d41225c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4009870 tx=0x7ff2c4004750 comp rx=0 tx=0).stop 2023-10-03T10:15:01.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.978+0000 7ff2d9b94700 1 -- 172.21.15.3:0/3696709996 shutdown_connections 2023-10-03T10:15:01.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.978+0000 7ff2d9b94700 1 --2- 172.21.15.3:0/3696709996 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2d409e410 0x7ff2d4123040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.978+0000 7ff2d9b94700 1 --2- 172.21.15.3:0/3696709996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2d409d6f0 0x7ff2d41225c0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.978+0000 7ff2d9b94700 1 --2- 172.21.15.3:0/3696709996 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2d409e000 0x7ff2d4122b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.978+0000 7ff2d9b94700 1 -- 172.21.15.3:0/3696709996 >> 172.21.15.3:0/3696709996 conn(0x7ff2d4066f50 msgr2=0x7ff2d40a3550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:01.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.978+0000 7ff2d9b94700 1 -- 172.21.15.3:0/3696709996 shutdown_connections 2023-10-03T10:15:01.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.978+0000 7ff2d9b94700 1 -- 172.21.15.3:0/3696709996 wait complete. 2023-10-03T10:15:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.985+0000 7fa841226700 1 -- 172.21.15.3:0/967916564 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa83c074ea0 con 0x7fa83c1d4eb0 2023-10-03T10:15:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.986+0000 7fa8237fe700 1 -- 172.21.15.3:0/967916564 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa8340118b0 con 0x7fa83c1d4eb0 2023-10-03T10:15:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:01.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.988+0000 7fa841226700 1 -- 172.21.15.3:0/967916564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa83c1d4eb0 msgr2=0x7fa83c1d5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:01.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.988+0000 7fa841226700 1 --2- 172.21.15.3:0/967916564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa83c1d4eb0 0x7fa83c1d5330 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa83400efc0 tx=0x7fa83400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.988+0000 7fa841226700 1 -- 172.21.15.3:0/967916564 shutdown_connections 2023-10-03T10:15:01.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.988+0000 7fa841226700 1 --2- 172.21.15.3:0/967916564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa83c1d4eb0 0x7fa83c1d5330 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.988+0000 7fa841226700 1 --2- 172.21.15.3:0/967916564 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa83c06e570 0x7fa83c1373c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.988+0000 7fa841226700 1 --2- 172.21.15.3:0/967916564 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa83c06f050 0x7fa83c137900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:01.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.988+0000 7fa841226700 1 -- 172.21.15.3:0/967916564 >> 172.21.15.3:0/967916564 conn(0x7fa83c0a2aa0 msgr2=0x7fa83c09e270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:01.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.988+0000 7fa841226700 1 -- 172.21.15.3:0/967916564 shutdown_connections 2023-10-03T10:15:01.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:01.988+0000 7fa841226700 1 -- 172.21.15.3:0/967916564 wait complete. 2023-10-03T10:15:01.993 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:02.002 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:02.792 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:02.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.937+0000 7f25c81d2700 1 Processor -- start 2023-10-03T10:15:02.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.938+0000 7f25c81d2700 1 -- start start 2023-10-03T10:15:02.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.938+0000 7f25c81d2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f25c009e930 0x7f25c009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:02.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.938+0000 7f25c81d2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25c009f650 0x7f25c0131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:02.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.938+0000 7f25c81d2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25c0070940 con 0x7f25c009f240 2023-10-03T10:15:02.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.938+0000 7f25c81d2700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25c006fd60 con 0x7f25c009e930 2023-10-03T10:15:02.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.938+0000 7f25c81d2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25c006fa80 con 0x7f25c009f650 2023-10-03T10:15:02.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.938+0000 7f25c5f6e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f25c009e930 0x7f25c009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:02.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.938+0000 7f25c676f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25c009f650 0x7f25c0131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:02.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.938+0000 7f25c5f6e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f25c009e930 0x7f25c009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41370/0 (socket says 172.21.15.3:41370) 2023-10-03T10:15:02.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.938+0000 7f25c5f6e700 1 -- 172.21.15.3:0/2485513817 learned_addr learned my addr 172.21.15.3:0/2485513817 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:02.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.938+0000 7f25c676f700 1 -- 172.21.15.3:0/2485513817 >> v2:172.21.15.3:3300/0 conn(0x7f25c009e930 msgr2=0x7f25c009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:02.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.938+0000 7f25c676f700 1 --2- 172.21.15.3:0/2485513817 >> v2:172.21.15.3:3300/0 conn(0x7f25c009e930 0x7f25c009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:02.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.939+0000 7f25c676f700 1 -- 172.21.15.3:0/2485513817 >> v1:172.21.15.3:6789/0 conn(0x7f25c009f240 legacy=0x7f25c01317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:02.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.939+0000 7f25c676f700 1 -- 172.21.15.3:0/2485513817 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c009fb60 con 0x7f25c009f650 2023-10-03T10:15:02.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.939+0000 7f25c5f6e700 1 --2- 172.21.15.3:0/2485513817 >> v2:172.21.15.3:3300/0 conn(0x7f25c009e930 0x7f25c009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:02.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.939+0000 7f25c676f700 1 --2- 172.21.15.3:0/2485513817 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25c009f650 0x7f25c0131f00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009d90 tx=0x7f25b800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b11e963959b99bfe server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:02.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.939+0000 7f25c4f6c700 1 -- 172.21.15.3:0/2485513817 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b800e070 con 0x7f25c009f650 2023-10-03T10:15:02.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.939+0000 7f25c4f6c700 1 -- 172.21.15.3:0/2485513817 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b800bb20 con 0x7f25c009f650 2023-10-03T10:15:02.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.939+0000 7f25c4f6c700 1 -- 172.21.15.3:0/2485513817 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8012430 con 0x7f25c009f650 2023-10-03T10:15:02.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.939+0000 7f25c81d2700 1 -- 172.21.15.3:0/2485513817 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25c009f650 msgr2=0x7f25c0131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:02.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.939+0000 7f25c81d2700 1 --2- 172.21.15.3:0/2485513817 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25c009f650 0x7f25c0131f00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009d90 tx=0x7f25b800b330 comp rx=0 tx=0).stop 2023-10-03T10:15:02.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.939+0000 7f25c81d2700 1 -- 172.21.15.3:0/2485513817 shutdown_connections 2023-10-03T10:15:02.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.940+0000 7f25c81d2700 1 --2- 172.21.15.3:0/2485513817 >> v2:172.21.15.3:3300/0 conn(0x7f25c009e930 0x7f25c009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:02.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.940+0000 7f25c81d2700 1 --2- 172.21.15.3:0/2485513817 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25c009f650 0x7f25c0131f00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:02.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.940+0000 7f25c81d2700 1 -- 172.21.15.3:0/2485513817 >> 172.21.15.3:0/2485513817 conn(0x7f25c00a4210 msgr2=0x7f25c00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:02.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.940+0000 7f25c81d2700 1 -- 172.21.15.3:0/2485513817 shutdown_connections 2023-10-03T10:15:02.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.940+0000 7f25c81d2700 1 -- 172.21.15.3:0/2485513817 wait complete. 2023-10-03T10:15:02.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.941+0000 7f25c81d2700 1 Processor -- start 2023-10-03T10:15:02.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.941+0000 7f25c81d2700 1 -- start start 2023-10-03T10:15:02.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.941+0000 7f25c81d2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25c009e930 0x7f25c01c2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:02.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.941+0000 7f25c81d2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25c009f240 0x7f25c01c3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:02.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.941+0000 7f25c81d2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25c009f650 0x7f25c01c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:02.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.941+0000 7f25c81d2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c01a7420 con 0x7f25c009f650 2023-10-03T10:15:02.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.941+0000 7f25c81d2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c01a6960 con 0x7f25c009f240 2023-10-03T10:15:02.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.941+0000 7f25c81d2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25c01c3980 con 0x7f25c009e930 2023-10-03T10:15:02.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.941+0000 7f25c5f6e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25c009e930 0x7f25c01c2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:02.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.941+0000 7f25c676f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25c009f650 0x7f25c01c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:02.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.941+0000 7f25c5f6e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25c009e930 0x7f25c01c2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56340/0 (socket says 172.21.15.3:56340) 2023-10-03T10:15:02.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.941+0000 7f25c576d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25c009f240 0x7f25c01c3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:02.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.941+0000 7f25c5f6e700 1 -- 172.21.15.3:0/2461276838 learned_addr learned my addr 172.21.15.3:0/2461276838 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:02.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.942+0000 7f25c676f700 1 -- 172.21.15.3:0/2461276838 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25c009e930 msgr2=0x7f25c01c2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:02.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.942+0000 7f25c676f700 1 --2- 172.21.15.3:0/2461276838 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25c009e930 0x7f25c01c2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:02.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.942+0000 7f25c676f700 1 -- 172.21.15.3:0/2461276838 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25c009f240 msgr2=0x7f25c01c3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:02.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.942+0000 7f25c676f700 1 --2- 172.21.15.3:0/2461276838 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25c009f240 0x7f25c01c3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:02.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.942+0000 7f25c676f700 1 -- 172.21.15.3:0/2461276838 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc009550 con 0x7f25c009f650 2023-10-03T10:15:02.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.942+0000 7f25c5f6e700 1 --2- 172.21.15.3:0/2461276838 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25c009e930 0x7f25c01c2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:02.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.942+0000 7f25c676f700 1 --2- 172.21.15.3:0/2461276838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25c009f650 0x7f25c01c5f20 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f25b8016000 tx=0x7f25b8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:02.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.942+0000 7f25b27fc700 1 -- 172.21.15.3:0/2461276838 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8012430 con 0x7f25c009f650 2023-10-03T10:15:02.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.942+0000 7f25b27fc700 1 -- 172.21.15.3:0/2461276838 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b8020070 con 0x7f25c009f650 2023-10-03T10:15:02.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.942+0000 7f25c81d2700 1 -- 172.21.15.3:0/2461276838 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b8009a70 con 0x7f25c009f650 2023-10-03T10:15:02.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.942+0000 7f25b27fc700 1 -- 172.21.15.3:0/2461276838 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8004040 con 0x7f25c009f650 2023-10-03T10:15:02.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.942+0000 7f25c81d2700 1 -- 172.21.15.3:0/2461276838 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c00a0190 con 0x7f25c009f650 2023-10-03T10:15:02.948 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:02.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.943+0000 7f25c81d2700 1 -- 172.21.15.3:0/2461276838 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c01a5060 con 0x7f25c009f650 2023-10-03T10:15:02.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.943+0000 7f25b27fc700 1 -- 172.21.15.3:0/2461276838 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f25b80041a0 con 0x7f25c009f650 2023-10-03T10:15:02.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.944+0000 7f25b27fc700 1 -- 172.21.15.3:0/2461276838 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f25b8022070 con 0x7f25c009f650 2023-10-03T10:15:02.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:02.948+0000 7f25b27fc700 1 -- 172.21.15.3:0/2461276838 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f25b8011960 con 0x7f25c009f650 2023-10-03T10:15:02.994 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:03.003 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:03.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.102+0000 7f7007235700 1 Processor -- start 2023-10-03T10:15:03.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.102+0000 7f7007235700 1 -- start start 2023-10-03T10:15:03.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.103+0000 7f7007235700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7000131810 0x7f7000131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:03.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.103+0000 7f7007235700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7000099a40 0x7f7000099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:03.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.103+0000 7f7007235700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f700009a850 con 0x7f700009a3b0 2023-10-03T10:15:03.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.103+0000 7f7007235700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f700009b580 con 0x7f7000099a40 2023-10-03T10:15:03.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.103+0000 7f7007235700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f70001313c0 con 0x7f7000131810 2023-10-03T10:15:03.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.103+0000 7f6ffffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7000099a40 0x7f7000099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:03.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.103+0000 7f70057d2700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f700009a3b0 0x7f7000138650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48348/0 (socket says 172.21.15.3:48348) 2023-10-03T10:15:03.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.103+0000 7f70057d2700 1 -- 172.21.15.3:0/3749369260 learned_addr learned my addr 172.21.15.3:0/3749369260 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:03.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.103+0000 7f7004fd1700 1 --2- 172.21.15.3:0/3749369260 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7000131810 0x7f7000131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:03.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.104+0000 7f6fff7fe700 1 -- 172.21.15.3:0/3749369260 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2357670024 0 0) 0x7f700009a850 con 0x7f700009a3b0 2023-10-03T10:15:03.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.104+0000 7f6fff7fe700 1 -- 172.21.15.3:0/3749369260 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ff00035b0 con 0x7f700009a3b0 2023-10-03T10:15:03.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.104+0000 7f6ffffff700 1 -- 172.21.15.3:0/3749369260 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7000131810 msgr2=0x7f7000131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:03.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.104+0000 7f6ffffff700 1 --2- 172.21.15.3:0/3749369260 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7000131810 0x7f7000131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.104+0000 7f6ffffff700 1 -- 172.21.15.3:0/3749369260 >> v1:172.21.15.3:6789/0 conn(0x7f700009a3b0 legacy=0x7f7000138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:03.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.104+0000 7f6ffffff700 1 -- 172.21.15.3:0/3749369260 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700006e8c0 con 0x7f7000099a40 2023-10-03T10:15:03.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.104+0000 7f6ffffff700 1 --2- 172.21.15.3:0/3749369260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7000099a40 0x7f7000099e70 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6fec009870 tx=0x7f6fec009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557ffcb0b54efc77 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:03.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.104+0000 7f6fff7fe700 1 -- 172.21.15.3:0/3749369260 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec011830 con 0x7f7000099a40 2023-10-03T10:15:03.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.105+0000 7f6fff7fe700 1 -- 172.21.15.3:0/3749369260 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fec011990 con 0x7f7000099a40 2023-10-03T10:15:03.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.105+0000 7f6fff7fe700 1 -- 172.21.15.3:0/3749369260 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec011d20 con 0x7f7000099a40 2023-10-03T10:15:03.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.106+0000 7f7007235700 1 -- 172.21.15.3:0/3749369260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7000099a40 msgr2=0x7f7000099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:03.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.106+0000 7f7007235700 1 --2- 172.21.15.3:0/3749369260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7000099a40 0x7f7000099e70 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6fec009870 tx=0x7f6fec009b80 comp rx=0 tx=0).stop 2023-10-03T10:15:03.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.106+0000 7f7007235700 1 -- 172.21.15.3:0/3749369260 shutdown_connections 2023-10-03T10:15:03.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.106+0000 7f7007235700 1 --2- 172.21.15.3:0/3749369260 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7000131810 0x7f7000131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.106+0000 7f7007235700 1 --2- 172.21.15.3:0/3749369260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7000099a40 0x7f7000099e70 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.106+0000 7f7007235700 1 -- 172.21.15.3:0/3749369260 >> 172.21.15.3:0/3749369260 conn(0x7f70000a2aa0 msgr2=0x7f70000a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:03.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.106+0000 7f7007235700 1 -- 172.21.15.3:0/3749369260 shutdown_connections 2023-10-03T10:15:03.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.106+0000 7f7007235700 1 -- 172.21.15.3:0/3749369260 wait complete. 2023-10-03T10:15:03.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.107+0000 7f7007235700 1 Processor -- start 2023-10-03T10:15:03.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.107+0000 7f7007235700 1 -- start start 2023-10-03T10:15:03.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.108+0000 7f7007235700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7000099a40 0x7f7000119700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:03.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.108+0000 7f7007235700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f700009a3b0 0x7f7000119c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:03.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.108+0000 7f7007235700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7000131810 0x7f700011a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:03.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.108+0000 7f7007235700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700011bff0 con 0x7f7000131810 2023-10-03T10:15:03.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.108+0000 7f7007235700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70000a1a00 con 0x7f700009a3b0 2023-10-03T10:15:03.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.108+0000 7f7007235700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f700006eb00 con 0x7f7000099a40 2023-10-03T10:15:03.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.109+0000 7f7004fd1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7000099a40 0x7f7000119700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:03.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.109+0000 7f7004fd1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7000099a40 0x7f7000119700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56362/0 (socket says 172.21.15.3:56362) 2023-10-03T10:15:03.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.109+0000 7f7004fd1700 1 -- 172.21.15.3:0/3639798778 learned_addr learned my addr 172.21.15.3:0/3639798778 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:03.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.109+0000 7f70057d2700 1 --2- 172.21.15.3:0/3639798778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7000131810 0x7f700011a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:03.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.109+0000 7f6ffffff700 1 --2- 172.21.15.3:0/3639798778 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f700009a3b0 0x7f7000119c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:03.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.109+0000 7f7004fd1700 1 -- 172.21.15.3:0/3639798778 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f700009a3b0 msgr2=0x7f7000119c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:03.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.109+0000 7f7004fd1700 1 --2- 172.21.15.3:0/3639798778 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f700009a3b0 0x7f7000119c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.109+0000 7f7004fd1700 1 -- 172.21.15.3:0/3639798778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7000131810 msgr2=0x7f700011a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:03.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.109+0000 7f7004fd1700 1 --2- 172.21.15.3:0/3639798778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7000131810 0x7f700011a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.109+0000 7f7004fd1700 1 -- 172.21.15.3:0/3639798778 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fec0094a0 con 0x7f7000099a40 2023-10-03T10:15:03.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.109+0000 7f7004fd1700 1 --2- 172.21.15.3:0/3639798778 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7000099a40 0x7f7000119700 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400dd40 tx=0x7f6ff40103a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:03.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.109+0000 7f6ffd7fa700 1 -- 172.21.15.3:0/3639798778 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff400dbb0 con 0x7f7000099a40 2023-10-03T10:15:03.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.110+0000 7f6ffd7fa700 1 -- 172.21.15.3:0/3639798778 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff4013040 con 0x7f7000099a40 2023-10-03T10:15:03.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.110+0000 7f6ffd7fa700 1 -- 172.21.15.3:0/3639798778 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4010db0 con 0x7f7000099a40 2023-10-03T10:15:03.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.110+0000 7f7007235700 1 -- 172.21.15.3:0/3639798778 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700011b220 con 0x7f7000099a40 2023-10-03T10:15:03.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.110+0000 7f7007235700 1 -- 172.21.15.3:0/3639798778 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700011b970 con 0x7f7000099a40 2023-10-03T10:15:03.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.111+0000 7f7007235700 1 -- 172.21.15.3:0/3639798778 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f700011cd20 con 0x7f7000099a40 2023-10-03T10:15:03.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.111+0000 7f6ffd7fa700 1 -- 172.21.15.3:0/3639798778 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6ff4018480 con 0x7f7000099a40 2023-10-03T10:15:03.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.112+0000 7f6ffd7fa700 1 -- 172.21.15.3:0/3639798778 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6ff4043f00 con 0x7f7000099a40 2023-10-03T10:15:03.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.118+0000 7f6ffd7fa700 1 -- 172.21.15.3:0/3639798778 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6ff4020960 con 0x7f7000099a40 2023-10-03T10:15:03.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.153+0000 7fe83e513700 1 Processor -- start 2023-10-03T10:15:03.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.153+0000 7fe83e513700 1 -- start start 2023-10-03T10:15:03.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.154+0000 7fe83e513700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe8380a0a30 0x7fe8380a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:03.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.154+0000 7fe83e513700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe838138d80 0x7fe83813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:03.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.154+0000 7fe83e513700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8380a2a70 con 0x7fe83809d630 2023-10-03T10:15:03.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.154+0000 7fe83e513700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8380a01d0 con 0x7fe8380a0a30 2023-10-03T10:15:03.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.154+0000 7fe83e513700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe83809e2b0 con 0x7fe838138d80 2023-10-03T10:15:03.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.155+0000 7fe83cab0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe838138d80 0x7fe83813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:03.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.155+0000 7fe83cab0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe838138d80 0x7fe83813b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:56364/0 (socket says 172.21.15.3:56364) 2023-10-03T10:15:03.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.155+0000 7fe8377fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe83809d630 0x7fe8380a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48360/0 (socket says 172.21.15.3:48360) 2023-10-03T10:15:03.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.155+0000 7fe83cab0700 1 -- 172.21.15.3:0/1237818369 learned_addr learned my addr 172.21.15.3:0/1237818369 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:03.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.155+0000 7fe837fff700 1 --2- 172.21.15.3:0/1237818369 >> v2:172.21.15.3:3300/0 conn(0x7fe8380a0a30 0x7fe8380a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:03.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.155+0000 7fe83cab0700 1 -- 172.21.15.3:0/1237818369 >> v2:172.21.15.3:3300/0 conn(0x7fe8380a0a30 msgr2=0x7fe8380a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:03.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.155+0000 7fe83cab0700 1 --2- 172.21.15.3:0/1237818369 >> v2:172.21.15.3:3300/0 conn(0x7fe8380a0a30 0x7fe8380a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.155+0000 7fe83cab0700 1 -- 172.21.15.3:0/1237818369 >> v1:172.21.15.3:6789/0 conn(0x7fe83809d630 legacy=0x7fe8380a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:03.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.155+0000 7fe836ffd700 1 -- 172.21.15.3:0/1237818369 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2376431584 0 0) 0x7fe8380a2a70 con 0x7fe83809d630 2023-10-03T10:15:03.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.155+0000 7fe83cab0700 1 -- 172.21.15.3:0/1237818369 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83809dc50 con 0x7fe838138d80 2023-10-03T10:15:03.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.155+0000 7fe83cab0700 1 --2- 172.21.15.3:0/1237818369 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe838138d80 0x7fe83813b160 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe830009e50 tx=0x7fe83000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c66a5597c4a6038 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:03.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.156+0000 7fe836ffd700 1 -- 172.21.15.3:0/1237818369 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83000e070 con 0x7fe838138d80 2023-10-03T10:15:03.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.156+0000 7fe836ffd700 1 -- 172.21.15.3:0/1237818369 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83000b910 con 0x7fe838138d80 2023-10-03T10:15:03.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.156+0000 7fe836ffd700 1 -- 172.21.15.3:0/1237818369 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83000bcc0 con 0x7fe838138d80 2023-10-03T10:15:03.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.157+0000 7fe83e513700 1 -- 172.21.15.3:0/1237818369 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe838138d80 msgr2=0x7fe83813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:03.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.157+0000 7fe83e513700 1 --2- 172.21.15.3:0/1237818369 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe838138d80 0x7fe83813b160 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe830009e50 tx=0x7fe83000b330 comp rx=0 tx=0).stop 2023-10-03T10:15:03.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.157+0000 7fe83e513700 1 -- 172.21.15.3:0/1237818369 shutdown_connections 2023-10-03T10:15:03.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.157+0000 7fe83e513700 1 --2- 172.21.15.3:0/1237818369 >> v2:172.21.15.3:3300/0 conn(0x7fe8380a0a30 0x7fe8380a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.157+0000 7fe83e513700 1 --2- 172.21.15.3:0/1237818369 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe838138d80 0x7fe83813b160 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.157+0000 7fe83e513700 1 -- 172.21.15.3:0/1237818369 >> 172.21.15.3:0/1237818369 conn(0x7fe8380a3050 msgr2=0x7fe8380a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:03.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.157+0000 7fe83e513700 1 -- 172.21.15.3:0/1237818369 shutdown_connections 2023-10-03T10:15:03.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.157+0000 7fe83e513700 1 -- 172.21.15.3:0/1237818369 wait complete. 2023-10-03T10:15:03.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.158+0000 7fe83e513700 1 Processor -- start 2023-10-03T10:15:03.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.159+0000 7fe83e513700 1 -- start start 2023-10-03T10:15:03.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.159+0000 7fe83e513700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe83809d630 0x7fe838133700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:03.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.159+0000 7fe83e513700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8380a0a30 0x7fe838133c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:03.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.159+0000 7fe83e513700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe838138d80 0x7fe838134180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:03.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.159+0000 7fe83e513700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe838135b60 con 0x7fe8380a0a30 2023-10-03T10:15:03.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.159+0000 7fe83e513700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe838134af0 con 0x7fe838138d80 2023-10-03T10:15:03.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.159+0000 7fe83e513700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe83813aba0 con 0x7fe83809d630 2023-10-03T10:15:03.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.159+0000 7fe837fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe83809d630 0x7fe838133700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:03.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.160+0000 7fe837fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe83809d630 0x7fe838133700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38344/0 (socket says 172.21.15.3:38344) 2023-10-03T10:15:03.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.160+0000 7fe837fff700 1 -- 172.21.15.3:0/161623817 learned_addr learned my addr 172.21.15.3:0/161623817 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:03.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.160+0000 7fe8377fe700 1 --2- 172.21.15.3:0/161623817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8380a0a30 0x7fe838133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:03.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.160+0000 7fe83cab0700 1 --2- 172.21.15.3:0/161623817 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe838138d80 0x7fe838134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:03.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.160+0000 7fe837fff700 1 -- 172.21.15.3:0/161623817 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe838138d80 msgr2=0x7fe838134180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:03.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.160+0000 7fe837fff700 1 --2- 172.21.15.3:0/161623817 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe838138d80 0x7fe838134180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.160+0000 7fe837fff700 1 -- 172.21.15.3:0/161623817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8380a0a30 msgr2=0x7fe838133c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:03.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.160+0000 7fe837fff700 1 --2- 172.21.15.3:0/161623817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8380a0a30 0x7fe838133c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.160+0000 7fe837fff700 1 -- 172.21.15.3:0/161623817 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe830009b30 con 0x7fe83809d630 2023-10-03T10:15:03.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.160+0000 7fe837fff700 1 --2- 172.21.15.3:0/161623817 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe83809d630 0x7fe838133700 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe82800e9e0 tx=0x7fe82800ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:03.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.160+0000 7fe834ff9700 1 -- 172.21.15.3:0/161623817 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe828009710 con 0x7fe83809d630 2023-10-03T10:15:03.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.160+0000 7fe834ff9700 1 -- 172.21.15.3:0/161623817 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82800ee80 con 0x7fe83809d630 2023-10-03T10:15:03.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.160+0000 7fe83e513700 1 -- 172.21.15.3:0/161623817 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8380a0410 con 0x7fe83809d630 2023-10-03T10:15:03.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.161+0000 7fe834ff9700 1 -- 172.21.15.3:0/161623817 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82800c9f0 con 0x7fe83809d630 2023-10-03T10:15:03.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.161+0000 7fe83e513700 1 -- 172.21.15.3:0/161623817 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8381364e0 con 0x7fe83809d630 2023-10-03T10:15:03.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.161+0000 7fe834ff9700 1 -- 172.21.15.3:0/161623817 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe82800cb50 con 0x7fe83809d630 2023-10-03T10:15:03.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.162+0000 7fe834ff9700 1 -- 172.21.15.3:0/161623817 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe82803fad0 con 0x7fe83809d630 2023-10-03T10:15:03.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.162+0000 7fe83e513700 1 -- 172.21.15.3:0/161623817 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8380747f0 con 0x7fe83809d630 2023-10-03T10:15:03.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.168+0000 7fe834ff9700 1 -- 172.21.15.3:0/161623817 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe828021080 con 0x7fe83809d630 2023-10-03T10:15:03.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.170+0000 7f25c81d2700 1 -- 172.21.15.3:0/2461276838 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f25c01a5060 con 0x7f25c009f650 2023-10-03T10:15:03.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.171+0000 7f25b27fc700 1 -- 172.21.15.3:0/2461276838 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f25c01a5060 con 0x7f25c009f650 2023-10-03T10:15:03.171 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:03.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.174+0000 7f25c81d2700 1 -- 172.21.15.3:0/2461276838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25c009f650 msgr2=0x7f25c01c5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:03.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.174+0000 7f25c81d2700 1 --2- 172.21.15.3:0/2461276838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25c009f650 0x7f25c01c5f20 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f25b8016000 tx=0x7f25b8004ad0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.174+0000 7f25c81d2700 1 -- 172.21.15.3:0/2461276838 shutdown_connections 2023-10-03T10:15:03.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.175+0000 7f25c81d2700 1 --2- 172.21.15.3:0/2461276838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25c009f650 0x7f25c01c5f20 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.175+0000 7f25c81d2700 1 --2- 172.21.15.3:0/2461276838 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25c009e930 0x7f25c01c2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.175+0000 7f25c81d2700 1 --2- 172.21.15.3:0/2461276838 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25c009f240 0x7f25c01c3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.175+0000 7f25c81d2700 1 -- 172.21.15.3:0/2461276838 >> 172.21.15.3:0/2461276838 conn(0x7f25c00a4210 msgr2=0x7f25c01b8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:03.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.176+0000 7f25c81d2700 1 -- 172.21.15.3:0/2461276838 shutdown_connections 2023-10-03T10:15:03.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.176+0000 7f25c81d2700 1 -- 172.21.15.3:0/2461276838 wait complete. 2023-10-03T10:15:03.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.187+0000 7f371bedd700 1 Processor -- start 2023-10-03T10:15:03.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.187+0000 7f371bedd700 1 -- start start 2023-10-03T10:15:03.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.188+0000 7f371bedd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f371409d6f0 0x7f371409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:03.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.188+0000 7f371bedd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f371409e000 0x7f37141339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:03.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.188+0000 7f371bedd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f371409e410 0x7f3714133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:03.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.188+0000 7f371bedd700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f371406b460 con 0x7f371409e410 2023-10-03T10:15:03.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.188+0000 7f371bedd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f371409b120 con 0x7f371409e000 2023-10-03T10:15:03.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.188+0000 7f371bedd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f371409a7e0 con 0x7f371409d6f0 2023-10-03T10:15:03.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.188+0000 7f371a47a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f371409e410 0x7f3714133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:03.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.188+0000 7f371a47a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f371409e410 0x7f3714133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57020/0 (socket says 172.21.15.3:57020) 2023-10-03T10:15:03.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.188+0000 7f371a47a700 1 -- 172.21.15.3:0/1709377343 learned_addr learned my addr 172.21.15.3:0/1709377343 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:03.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.188+0000 7f3719c79700 1 --2- 172.21.15.3:0/1709377343 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f371409d6f0 0x7f371409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:03.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.188+0000 7f3719478700 1 --2- 172.21.15.3:0/1709377343 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f371409e000 0x7f37141339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:03.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.188+0000 7f3719478700 1 -- 172.21.15.3:0/1709377343 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f371409d6f0 msgr2=0x7f371409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:03.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.188+0000 7f3719478700 1 --2- 172.21.15.3:0/1709377343 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f371409d6f0 0x7f371409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.188+0000 7f3719478700 1 -- 172.21.15.3:0/1709377343 >> v2:172.21.15.3:3300/0 conn(0x7f371409e410 msgr2=0x7f3714133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:03.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.188+0000 7f3719478700 1 --2- 172.21.15.3:0/1709377343 >> v2:172.21.15.3:3300/0 conn(0x7f371409e410 0x7f3714133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.188+0000 7f3719478700 1 -- 172.21.15.3:0/1709377343 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371409a500 con 0x7f371409e000 2023-10-03T10:15:03.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.188+0000 7f371a47a700 1 --2- 172.21.15.3:0/1709377343 >> v2:172.21.15.3:3300/0 conn(0x7f371409e410 0x7f3714133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:03.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.188+0000 7f3719478700 1 --2- 172.21.15.3:0/1709377343 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f371409e000 0x7f37141339f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3704009870 tx=0x7f3704009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26f4a5ca7391863 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:03.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.189+0000 7f3718c77700 1 -- 172.21.15.3:0/1709377343 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3704011830 con 0x7f371409e000 2023-10-03T10:15:03.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.189+0000 7f3718c77700 1 -- 172.21.15.3:0/1709377343 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3704011990 con 0x7f371409e000 2023-10-03T10:15:03.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.189+0000 7f3718c77700 1 -- 172.21.15.3:0/1709377343 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3704011d20 con 0x7f371409e000 2023-10-03T10:15:03.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.189+0000 7f371bedd700 1 -- 172.21.15.3:0/1709377343 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f371409e000 msgr2=0x7f37141339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:03.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.189+0000 7f371bedd700 1 --2- 172.21.15.3:0/1709377343 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f371409e000 0x7f37141339f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3704009870 tx=0x7f3704009b80 comp rx=0 tx=0).stop 2023-10-03T10:15:03.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.190+0000 7f371bedd700 1 -- 172.21.15.3:0/1709377343 shutdown_connections 2023-10-03T10:15:03.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.190+0000 7f371bedd700 1 --2- 172.21.15.3:0/1709377343 >> v2:172.21.15.3:3300/0 conn(0x7f371409e410 0x7f3714133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.190+0000 7f371bedd700 1 --2- 172.21.15.3:0/1709377343 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f371409d6f0 0x7f371409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.190+0000 7f371bedd700 1 --2- 172.21.15.3:0/1709377343 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f371409e000 0x7f37141339f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.190+0000 7f371bedd700 1 -- 172.21.15.3:0/1709377343 >> 172.21.15.3:0/1709377343 conn(0x7f3714066f50 msgr2=0x7f371409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:03.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.190+0000 7f371bedd700 1 -- 172.21.15.3:0/1709377343 shutdown_connections 2023-10-03T10:15:03.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.190+0000 7f371bedd700 1 -- 172.21.15.3:0/1709377343 wait complete. 2023-10-03T10:15:03.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.191+0000 7f371bedd700 1 Processor -- start 2023-10-03T10:15:03.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.191+0000 7f371bedd700 1 -- start start 2023-10-03T10:15:03.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.191+0000 7f371bedd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f371409d6f0 0x7f37141a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:03.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.191+0000 7f371bedd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f371409e000 0x7f37141a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:03.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.191+0000 7f371bedd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f371409e410 0x7f37141c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:03.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.191+0000 7f371bedd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37141a7010 con 0x7f371409e000 2023-10-03T10:15:03.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.191+0000 7f371bedd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37141a3c80 con 0x7f371409e410 2023-10-03T10:15:03.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.191+0000 7f371bedd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f371409c710 con 0x7f371409d6f0 2023-10-03T10:15:03.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.192+0000 7f3719c79700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f371409d6f0 0x7f37141a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:03.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.192+0000 7f3719c79700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f371409d6f0 0x7f37141a4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38366/0 (socket says 172.21.15.3:38366) 2023-10-03T10:15:03.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.192+0000 7f3719c79700 1 -- 172.21.15.3:0/1548695570 learned_addr learned my addr 172.21.15.3:0/1548695570 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:03.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.192+0000 7f371a47a700 1 --2- 172.21.15.3:0/1548695570 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f371409e410 0x7f37141c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:03.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.192+0000 7f3719478700 1 --2- 172.21.15.3:0/1548695570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f371409e000 0x7f37141a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:03.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.192+0000 7f3719c79700 1 -- 172.21.15.3:0/1548695570 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f371409e410 msgr2=0x7f37141c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:03.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.192+0000 7f3719c79700 1 --2- 172.21.15.3:0/1548695570 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f371409e410 0x7f37141c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.192+0000 7f3719c79700 1 -- 172.21.15.3:0/1548695570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f371409e000 msgr2=0x7f37141a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:03.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.192+0000 7f3719c79700 1 --2- 172.21.15.3:0/1548695570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f371409e000 0x7f37141a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.192+0000 7f3719c79700 1 -- 172.21.15.3:0/1548695570 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c009b30 con 0x7f371409d6f0 2023-10-03T10:15:03.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.192+0000 7f371a47a700 1 --2- 172.21.15.3:0/1548695570 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f371409e410 0x7f37141c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:15:03.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.192+0000 7f3719c79700 1 --2- 172.21.15.3:0/1548695570 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f371409d6f0 0x7f37141a4140 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f371000e9e0 tx=0x7f371000ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:03.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.192+0000 7f370a7fc700 1 -- 172.21.15.3:0/1548695570 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3710009710 con 0x7f371409d6f0 2023-10-03T10:15:03.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.193+0000 7f370a7fc700 1 -- 172.21.15.3:0/1548695570 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371000c850 con 0x7f371409d6f0 2023-10-03T10:15:03.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.193+0000 7f370a7fc700 1 -- 172.21.15.3:0/1548695570 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371000cbc0 con 0x7f371409d6f0 2023-10-03T10:15:03.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.193+0000 7f371bedd700 1 -- 172.21.15.3:0/1548695570 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37040094a0 con 0x7f371409d6f0 2023-10-03T10:15:03.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.193+0000 7f371bedd700 1 -- 172.21.15.3:0/1548695570 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37141a5f50 con 0x7f371409d6f0 2023-10-03T10:15:03.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.193+0000 7f370a7fc700 1 -- 172.21.15.3:0/1548695570 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f371000cd20 con 0x7f371409d6f0 2023-10-03T10:15:03.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.194+0000 7f370a7fc700 1 -- 172.21.15.3:0/1548695570 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f371003f990 con 0x7f371409d6f0 2023-10-03T10:15:03.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.194+0000 7f371bedd700 1 -- 172.21.15.3:0/1548695570 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3714074ea0 con 0x7f371409d6f0 2023-10-03T10:15:03.201 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:03.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.199+0000 7f370a7fc700 1 -- 172.21.15.3:0/1548695570 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f371001be10 con 0x7f371409d6f0 2023-10-03T10:15:03.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.344+0000 7f7007235700 1 -- 172.21.15.3:0/3639798778 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f700011cd20 con 0x7f7000099a40 2023-10-03T10:15:03.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.345+0000 7f6ffd7fa700 1 -- 172.21.15.3:0/3639798778 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f700011cd20 con 0x7f7000099a40 2023-10-03T10:15:03.345 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:03.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.347+0000 7f7007235700 1 -- 172.21.15.3:0/3639798778 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7000099a40 msgr2=0x7f7000119700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:03.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.347+0000 7f7007235700 1 --2- 172.21.15.3:0/3639798778 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7000099a40 0x7f7000119700 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400dd40 tx=0x7f6ff40103a0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.347+0000 7f7007235700 1 -- 172.21.15.3:0/3639798778 shutdown_connections 2023-10-03T10:15:03.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.347+0000 7f7007235700 1 --2- 172.21.15.3:0/3639798778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7000131810 0x7f700011a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.347+0000 7f7007235700 1 --2- 172.21.15.3:0/3639798778 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7000099a40 0x7f7000119700 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.347+0000 7f7007235700 1 --2- 172.21.15.3:0/3639798778 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f700009a3b0 0x7f7000119c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.347+0000 7f7007235700 1 -- 172.21.15.3:0/3639798778 >> 172.21.15.3:0/3639798778 conn(0x7f70000a2aa0 msgr2=0x7f700012e440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:03.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.347+0000 7f7007235700 1 -- 172.21.15.3:0/3639798778 shutdown_connections 2023-10-03T10:15:03.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.347+0000 7f7007235700 1 -- 172.21.15.3:0/3639798778 wait complete. 2023-10-03T10:15:03.361 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:03.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.386+0000 7fe83e513700 1 -- 172.21.15.3:0/161623817 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe838072980 con 0x7fe83809d630 2023-10-03T10:15:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.387+0000 7fe834ff9700 1 -- 172.21.15.3:0/161623817 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe828017020 con 0x7fe83809d630 2023-10-03T10:15:03.387 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:03.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.389+0000 7fe83e513700 1 -- 172.21.15.3:0/161623817 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe83809d630 msgr2=0x7fe838133700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:03.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.389+0000 7fe83e513700 1 --2- 172.21.15.3:0/161623817 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe83809d630 0x7fe838133700 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe82800e9e0 tx=0x7fe82800ecf0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.389+0000 7fe83e513700 1 -- 172.21.15.3:0/161623817 shutdown_connections 2023-10-03T10:15:03.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.389+0000 7fe83e513700 1 --2- 172.21.15.3:0/161623817 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe838138d80 0x7fe838134180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.389+0000 7fe83e513700 1 --2- 172.21.15.3:0/161623817 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe83809d630 0x7fe838133700 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.389+0000 7fe83e513700 1 --2- 172.21.15.3:0/161623817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8380a0a30 0x7fe838133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.389+0000 7fe83e513700 1 -- 172.21.15.3:0/161623817 >> 172.21.15.3:0/161623817 conn(0x7fe8380a3050 msgr2=0x7fe83809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:03.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.389+0000 7fe83e513700 1 -- 172.21.15.3:0/161623817 shutdown_connections 2023-10-03T10:15:03.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.389+0000 7fe83e513700 1 -- 172.21.15.3:0/161623817 wait complete. 2023-10-03T10:15:03.403 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:03.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.405+0000 7f371bedd700 1 -- 172.21.15.3:0/1548695570 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f371406c360 con 0x7f371409d6f0 2023-10-03T10:15:03.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.406+0000 7f370a7fc700 1 -- 172.21.15.3:0/1548695570 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f371002b090 con 0x7f371409d6f0 2023-10-03T10:15:03.406 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:03.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.408+0000 7f371bedd700 1 -- 172.21.15.3:0/1548695570 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f371409d6f0 msgr2=0x7f37141a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:03.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.408+0000 7f371bedd700 1 --2- 172.21.15.3:0/1548695570 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f371409d6f0 0x7f37141a4140 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f371000e9e0 tx=0x7f371000ecf0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.408+0000 7f371bedd700 1 -- 172.21.15.3:0/1548695570 shutdown_connections 2023-10-03T10:15:03.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.408+0000 7f371bedd700 1 --2- 172.21.15.3:0/1548695570 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f371409e410 0x7f37141c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.408+0000 7f371bedd700 1 --2- 172.21.15.3:0/1548695570 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f371409d6f0 0x7f37141a4140 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.408+0000 7f371bedd700 1 --2- 172.21.15.3:0/1548695570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f371409e000 0x7f37141a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:03.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.408+0000 7f371bedd700 1 -- 172.21.15.3:0/1548695570 >> 172.21.15.3:0/1548695570 conn(0x7f3714066f50 msgr2=0x7f37141bbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:03.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.408+0000 7f371bedd700 1 -- 172.21.15.3:0/1548695570 shutdown_connections 2023-10-03T10:15:03.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:03.408+0000 7f371bedd700 1 -- 172.21.15.3:0/1548695570 wait complete. 2023-10-03T10:15:03.422 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:04.202 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:04.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.350+0000 7f67c451d700 1 Processor -- start 2023-10-03T10:15:04.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.350+0000 7f67c451d700 1 -- start start 2023-10-03T10:15:04.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.351+0000 7f67c451d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67bc09e1d0 0x7f67bc135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:04.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.351+0000 7f67c451d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f67bc135e50 0x7f67bc138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:04.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.351+0000 7f67c451d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67bc09f130 con 0x7f67bc09d6f0 2023-10-03T10:15:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.351+0000 7f67c451d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67bc09f520 con 0x7f67bc135e50 2023-10-03T10:15:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.351+0000 7f67c451d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc09e930 con 0x7f67bc09e1d0 2023-10-03T10:15:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.351+0000 7f67c22b9700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f67bc09d6f0 0x7f67bc09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:46986/0 (socket says 172.21.15.3:46986) 2023-10-03T10:15:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.351+0000 7f67c22b9700 1 -- 172.21.15.3:0/1847601122 learned_addr learned my addr 172.21.15.3:0/1847601122 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.351+0000 7f67c2aba700 1 --2- 172.21.15.3:0/1847601122 >> v2:172.21.15.3:3300/0 conn(0x7f67bc135e50 0x7f67bc138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.351+0000 7f67c1ab8700 1 --2- 172.21.15.3:0/1847601122 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67bc09e1d0 0x7f67bc135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.352+0000 7f67c1ab8700 1 -- 172.21.15.3:0/1847601122 >> v2:172.21.15.3:3300/0 conn(0x7f67bc135e50 msgr2=0x7f67bc138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.352+0000 7f67c1ab8700 1 --2- 172.21.15.3:0/1847601122 >> v2:172.21.15.3:3300/0 conn(0x7f67bc135e50 0x7f67bc138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.352+0000 7f67c1ab8700 1 -- 172.21.15.3:0/1847601122 >> v1:172.21.15.3:6789/0 conn(0x7f67bc09d6f0 legacy=0x7f67bc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.352+0000 7f67c1ab8700 1 -- 172.21.15.3:0/1847601122 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67bc09b2d0 con 0x7f67bc09e1d0 2023-10-03T10:15:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.352+0000 7f67c2aba700 1 --2- 172.21.15.3:0/1847601122 >> v2:172.21.15.3:3300/0 conn(0x7f67bc135e50 0x7f67bc138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.352+0000 7f67c1ab8700 1 --2- 172.21.15.3:0/1847601122 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67bc09e1d0 0x7f67bc135910 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009870 tx=0x7f67ac009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d3f99a5a002cc0d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:04.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.352+0000 7f67c12b7700 1 -- 172.21.15.3:0/1847601122 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac011830 con 0x7f67bc09e1d0 2023-10-03T10:15:04.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.352+0000 7f67c12b7700 1 -- 172.21.15.3:0/1847601122 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ac011990 con 0x7f67bc09e1d0 2023-10-03T10:15:04.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.352+0000 7f67c12b7700 1 -- 172.21.15.3:0/1847601122 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac011d20 con 0x7f67bc09e1d0 2023-10-03T10:15:04.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.353+0000 7f67c451d700 1 -- 172.21.15.3:0/1847601122 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67bc09e1d0 msgr2=0x7f67bc135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.353+0000 7f67c451d700 1 --2- 172.21.15.3:0/1847601122 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67bc09e1d0 0x7f67bc135910 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009870 tx=0x7f67ac009b80 comp rx=0 tx=0).stop 2023-10-03T10:15:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.353+0000 7f67c451d700 1 -- 172.21.15.3:0/1847601122 shutdown_connections 2023-10-03T10:15:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.353+0000 7f67c451d700 1 --2- 172.21.15.3:0/1847601122 >> v2:172.21.15.3:3300/0 conn(0x7f67bc135e50 0x7f67bc138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.353+0000 7f67c451d700 1 --2- 172.21.15.3:0/1847601122 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67bc09e1d0 0x7f67bc135910 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.353+0000 7f67c451d700 1 -- 172.21.15.3:0/1847601122 >> 172.21.15.3:0/1847601122 conn(0x7f67bc066f50 msgr2=0x7f67bc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.353+0000 7f67c451d700 1 -- 172.21.15.3:0/1847601122 shutdown_connections 2023-10-03T10:15:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.353+0000 7f67c451d700 1 -- 172.21.15.3:0/1847601122 wait complete. 2023-10-03T10:15:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.354+0000 7f67c451d700 1 Processor -- start 2023-10-03T10:15:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.354+0000 7f67c451d700 1 -- start start 2023-10-03T10:15:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.354+0000 7f67c451d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67bc09d6f0 0x7f67bc06f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.354+0000 7f67c451d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67bc09e1d0 0x7f67bc06f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:04.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.354+0000 7f67c451d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67bc135e50 0x7f67bc1c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:04.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.354+0000 7f67c451d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc124460 con 0x7f67bc09e1d0 2023-10-03T10:15:04.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.354+0000 7f67c451d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc1250a0 con 0x7f67bc09d6f0 2023-10-03T10:15:04.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.354+0000 7f67c451d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc06ed30 con 0x7f67bc135e50 2023-10-03T10:15:04.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.354+0000 7f67c22b9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67bc09d6f0 0x7f67bc06f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:04.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.354+0000 7f67c2aba700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67bc135e50 0x7f67bc1c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:04.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.355+0000 7f67c22b9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67bc09d6f0 0x7f67bc06f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51166/0 (socket says 172.21.15.3:51166) 2023-10-03T10:15:04.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.355+0000 7f67c2aba700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67bc135e50 0x7f67bc1c82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38388/0 (socket says 172.21.15.3:38388) 2023-10-03T10:15:04.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.355+0000 7f67c22b9700 1 -- 172.21.15.3:0/160625324 learned_addr learned my addr 172.21.15.3:0/160625324 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:04.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.355+0000 7f67c1ab8700 1 --2- 172.21.15.3:0/160625324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67bc09e1d0 0x7f67bc06f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:04.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.355+0000 7f67c2aba700 1 -- 172.21.15.3:0/160625324 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67bc09d6f0 msgr2=0x7f67bc06f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:04.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.355+0000 7f67c2aba700 1 --2- 172.21.15.3:0/160625324 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67bc09d6f0 0x7f67bc06f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.355+0000 7f67c2aba700 1 -- 172.21.15.3:0/160625324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67bc09e1d0 msgr2=0x7f67bc06f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:04.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.355+0000 7f67c2aba700 1 --2- 172.21.15.3:0/160625324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67bc09e1d0 0x7f67bc06f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.355+0000 7f67c2aba700 1 -- 172.21.15.3:0/160625324 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b4009a70 con 0x7f67bc135e50 2023-10-03T10:15:04.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.355+0000 7f67c1ab8700 1 --2- 172.21.15.3:0/160625324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67bc09e1d0 0x7f67bc06f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:04.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.355+0000 7f67c2aba700 1 --2- 172.21.15.3:0/160625324 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67bc135e50 0x7f67bc1c82f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f67b400ef90 tx=0x7f67b400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:04.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.355+0000 7f67b2ffd700 1 -- 172.21.15.3:0/160625324 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b400cc90 con 0x7f67bc135e50 2023-10-03T10:15:04.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.355+0000 7f67b2ffd700 1 -- 172.21.15.3:0/160625324 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b400f040 con 0x7f67bc135e50 2023-10-03T10:15:04.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.355+0000 7f67c451d700 1 -- 172.21.15.3:0/160625324 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67ac0094a0 con 0x7f67bc135e50 2023-10-03T10:15:04.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.355+0000 7f67b2ffd700 1 -- 172.21.15.3:0/160625324 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b4014480 con 0x7f67bc135e50 2023-10-03T10:15:04.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.355+0000 7f67c451d700 1 -- 172.21.15.3:0/160625324 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67bc0a1f50 con 0x7f67bc135e50 2023-10-03T10:15:04.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.356+0000 7f67b2ffd700 1 -- 172.21.15.3:0/160625324 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f67b4014600 con 0x7f67bc135e50 2023-10-03T10:15:04.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.356+0000 7f67b2ffd700 1 -- 172.21.15.3:0/160625324 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f67b403ff70 con 0x7f67bc135e50 2023-10-03T10:15:04.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.356+0000 7f67c451d700 1 -- 172.21.15.3:0/160625324 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67bc1251e0 con 0x7f67bc135e50 2023-10-03T10:15:04.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.361+0000 7f67b2ffd700 1 -- 172.21.15.3:0/160625324 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f67bc1251e0 con 0x7f67bc135e50 2023-10-03T10:15:04.362 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:04.404 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:04.424 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:04.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.504+0000 7fdffec42700 1 Processor -- start 2023-10-03T10:15:04.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.504+0000 7fdffec42700 1 -- start start 2023-10-03T10:15:04.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.504+0000 7fdffec42700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdff80a1510 0x7fdff8138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:04.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.504+0000 7fdffec42700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdff8139250 0x7fdff813b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:04.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.504+0000 7fdffec42700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff809edd0 con 0x7fdff80a0a30 2023-10-03T10:15:04.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.504+0000 7fdffec42700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff80a18e0 con 0x7fdff80a1510 2023-10-03T10:15:04.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.504+0000 7fdffec42700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff809e1f0 con 0x7fdff8139250 2023-10-03T10:15:04.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.505+0000 7fdffd1df700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdff8139250 0x7fdff813b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:04.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.505+0000 7fdff7fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdff80a1510 0x7fdff8138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:04.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.505+0000 7fdffd1df700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdff8139250 0x7fdff813b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51174/0 (socket says 172.21.15.3:51174) 2023-10-03T10:15:04.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.505+0000 7fdff7fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdff80a1510 0x7fdff8138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57058/0 (socket says 172.21.15.3:57058) 2023-10-03T10:15:04.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.505+0000 7fdffd1df700 1 -- 172.21.15.3:0/1489127181 learned_addr learned my addr 172.21.15.3:0/1489127181 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.505+0000 7fdff7fff700 1 -- 172.21.15.3:0/1489127181 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdff8139250 msgr2=0x7fdff813b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.505+0000 7fdff7fff700 1 --2- 172.21.15.3:0/1489127181 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdff8139250 0x7fdff813b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.505+0000 7fdff7fff700 1 -- 172.21.15.3:0/1489127181 >> v1:172.21.15.3:6789/0 conn(0x7fdff80a0a30 legacy=0x7fdff80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.505+0000 7fdff7fff700 1 -- 172.21.15.3:0/1489127181 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff809dc50 con 0x7fdff80a1510 2023-10-03T10:15:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.505+0000 7fdff7fff700 1 --2- 172.21.15.3:0/1489127181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdff80a1510 0x7fdff8138d10 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009840 tx=0x7fdfe4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119ff6e2f393da59 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.506+0000 7fdff77fe700 1 -- 172.21.15.3:0/1489127181 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4011830 con 0x7fdff80a1510 2023-10-03T10:15:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.506+0000 7fdff77fe700 1 -- 172.21.15.3:0/1489127181 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe4011990 con 0x7fdff80a1510 2023-10-03T10:15:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.506+0000 7fdff77fe700 1 -- 172.21.15.3:0/1489127181 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4011d00 con 0x7fdff80a1510 2023-10-03T10:15:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.506+0000 7fdffec42700 1 -- 172.21.15.3:0/1489127181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdff80a1510 msgr2=0x7fdff8138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:04.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.506+0000 7fdffec42700 1 --2- 172.21.15.3:0/1489127181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdff80a1510 0x7fdff8138d10 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009840 tx=0x7fdfe4009b50 comp rx=0 tx=0).stop 2023-10-03T10:15:04.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.506+0000 7fdffec42700 1 -- 172.21.15.3:0/1489127181 shutdown_connections 2023-10-03T10:15:04.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.506+0000 7fdffec42700 1 --2- 172.21.15.3:0/1489127181 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdff8139250 0x7fdff813b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.506+0000 7fdffec42700 1 --2- 172.21.15.3:0/1489127181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdff80a1510 0x7fdff8138d10 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.506+0000 7fdffec42700 1 -- 172.21.15.3:0/1489127181 >> 172.21.15.3:0/1489127181 conn(0x7fdff80a6550 msgr2=0x7fdff80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:04.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.507+0000 7fdffec42700 1 -- 172.21.15.3:0/1489127181 shutdown_connections 2023-10-03T10:15:04.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.507+0000 7fdffec42700 1 -- 172.21.15.3:0/1489127181 wait complete. 2023-10-03T10:15:04.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.508+0000 7fdffec42700 1 Processor -- start 2023-10-03T10:15:04.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.508+0000 7fdffec42700 1 -- start start 2023-10-03T10:15:04.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.508+0000 7fdffec42700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdff80a0a30 0x7fdff81c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:04.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.508+0000 7fdffec42700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdff80a1510 0x7fdff81c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:04.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.508+0000 7fdffec42700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdff8139250 0x7fdff81218e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:04.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.508+0000 7fdffec42700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff8123690 con 0x7fdff80a0a30 2023-10-03T10:15:04.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.508+0000 7fdffec42700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff81234a0 con 0x7fdff80a1510 2023-10-03T10:15:04.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.508+0000 7fdffec42700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdff81c92f0 con 0x7fdff8139250 2023-10-03T10:15:04.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.508+0000 7fdff7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdff80a1510 0x7fdff81c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:04.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.508+0000 7fdff7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdff80a1510 0x7fdff81c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51184/0 (socket says 172.21.15.3:51184) 2023-10-03T10:15:04.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.509+0000 7fdff7fff700 1 -- 172.21.15.3:0/2974764632 learned_addr learned my addr 172.21.15.3:0/2974764632 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:04.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.509+0000 7fdffc9de700 1 --2- 172.21.15.3:0/2974764632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdff80a0a30 0x7fdff81c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:04.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.509+0000 7fdffd1df700 1 --2- 172.21.15.3:0/2974764632 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdff8139250 0x7fdff81218e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:04.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.509+0000 7fdff7fff700 1 -- 172.21.15.3:0/2974764632 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdff8139250 msgr2=0x7fdff81218e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:04.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.509+0000 7fdff7fff700 1 --2- 172.21.15.3:0/2974764632 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdff8139250 0x7fdff81218e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.509+0000 7fdff7fff700 1 -- 172.21.15.3:0/2974764632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdff80a0a30 msgr2=0x7fdff81c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:04.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.509+0000 7fdff7fff700 1 --2- 172.21.15.3:0/2974764632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdff80a0a30 0x7fdff81c8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.509+0000 7fdff7fff700 1 -- 172.21.15.3:0/2974764632 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe40094a0 con 0x7fdff80a1510 2023-10-03T10:15:04.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.509+0000 7fdffd1df700 1 --2- 172.21.15.3:0/2974764632 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdff8139250 0x7fdff81218e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:15:04.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.509+0000 7fdffc9de700 1 --2- 172.21.15.3:0/2974764632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdff80a0a30 0x7fdff81c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:15:04.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.509+0000 7fdff7fff700 1 --2- 172.21.15.3:0/2974764632 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdff80a1510 0x7fdff81c8db0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4006f90 tx=0x7fdfe4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:04.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.509+0000 7fdff57fa700 1 -- 172.21.15.3:0/2974764632 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe40038a0 con 0x7fdff80a1510 2023-10-03T10:15:04.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.509+0000 7fdff57fa700 1 -- 172.21.15.3:0/2974764632 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe401f070 con 0x7fdff80a1510 2023-10-03T10:15:04.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.510+0000 7fdffec42700 1 -- 172.21.15.3:0/2974764632 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff8123a80 con 0x7fdff80a1510 2023-10-03T10:15:04.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.510+0000 7fdff57fa700 1 -- 172.21.15.3:0/2974764632 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4003c40 con 0x7fdff80a1510 2023-10-03T10:15:04.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.510+0000 7fdffec42700 1 -- 172.21.15.3:0/2974764632 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff8123c90 con 0x7fdff80a1510 2023-10-03T10:15:04.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.511+0000 7fdff57fa700 1 -- 172.21.15.3:0/2974764632 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdfe401b430 con 0x7fdff80a1510 2023-10-03T10:15:04.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.511+0000 7fdffec42700 1 -- 172.21.15.3:0/2974764632 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff80a1d00 con 0x7fdff80a1510 2023-10-03T10:15:04.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.511+0000 7fdff57fa700 1 -- 172.21.15.3:0/2974764632 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fdfe4021070 con 0x7fdff80a1510 2023-10-03T10:15:04.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.516+0000 7fdff57fa700 1 -- 172.21.15.3:0/2974764632 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdfe4010c00 con 0x7fdff80a1510 2023-10-03T10:15:04.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.551+0000 7f5fbbf24700 1 Processor -- start 2023-10-03T10:15:04.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.552+0000 7f5fbbf24700 1 -- start start 2023-10-03T10:15:04.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.552+0000 7f5fbbf24700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fb409d6f0 0x7f5fb409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:04.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.552+0000 7f5fbbf24700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5fb409e000 0x7f5fb4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:04.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.552+0000 7f5fbbf24700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fb409f130 con 0x7f5fb409e410 2023-10-03T10:15:04.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.552+0000 7f5fbbf24700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fb409f520 con 0x7f5fb409e000 2023-10-03T10:15:04.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.552+0000 7f5fbbf24700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb409e930 con 0x7f5fb409d6f0 2023-10-03T10:15:04.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.552+0000 7f5fb9cc0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fb409d6f0 0x7f5fb409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:04.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.552+0000 7f5fb94bf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5fb409e000 0x7f5fb4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:04.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.552+0000 7f5fb9cc0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fb409d6f0 0x7f5fb409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38414/0 (socket says 172.21.15.3:38414) 2023-10-03T10:15:04.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.552+0000 7f5fb9cc0700 1 -- 172.21.15.3:0/2707709530 learned_addr learned my addr 172.21.15.3:0/2707709530 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:04.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.553+0000 7f5fb9cc0700 1 -- 172.21.15.3:0/2707709530 >> v2:172.21.15.3:3300/0 conn(0x7f5fb409e000 msgr2=0x7f5fb4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:04.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.553+0000 7f5fb9cc0700 1 --2- 172.21.15.3:0/2707709530 >> v2:172.21.15.3:3300/0 conn(0x7f5fb409e000 0x7f5fb4130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.553+0000 7f5fb9cc0700 1 -- 172.21.15.3:0/2707709530 >> v1:172.21.15.3:6789/0 conn(0x7f5fb409e410 legacy=0x7f5fb4130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:04.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.553+0000 7f5fb9cc0700 1 -- 172.21.15.3:0/2707709530 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb409b2d0 con 0x7f5fb409d6f0 2023-10-03T10:15:04.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.553+0000 7f5fb8cbe700 1 -- 172.21.15.3:0/2707709530 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1080870441 0 0) 0x7f5fb409f130 con 0x7f5fb409e410 2023-10-03T10:15:04.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.553+0000 7f5fb94bf700 1 --2- 172.21.15.3:0/2707709530 >> v2:172.21.15.3:3300/0 conn(0x7f5fb409e000 0x7f5fb4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:04.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.553+0000 7f5fb9cc0700 1 --2- 172.21.15.3:0/2707709530 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fb409d6f0 0x7f5fb409dac0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00098f0 tx=0x7f5fb0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe3acb8e932cdb40 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:04.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.553+0000 7f5fb8cbe700 1 -- 172.21.15.3:0/2707709530 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0011830 con 0x7f5fb409d6f0 2023-10-03T10:15:04.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.553+0000 7f5fb8cbe700 1 -- 172.21.15.3:0/2707709530 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb0011990 con 0x7f5fb409d6f0 2023-10-03T10:15:04.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.553+0000 7f5fb8cbe700 1 -- 172.21.15.3:0/2707709530 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0011d20 con 0x7f5fb409d6f0 2023-10-03T10:15:04.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.554+0000 7f5fbbf24700 1 -- 172.21.15.3:0/2707709530 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fb409d6f0 msgr2=0x7f5fb409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:04.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.554+0000 7f5fbbf24700 1 --2- 172.21.15.3:0/2707709530 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fb409d6f0 0x7f5fb409dac0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00098f0 tx=0x7f5fb0009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:04.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.554+0000 7f5fbbf24700 1 -- 172.21.15.3:0/2707709530 shutdown_connections 2023-10-03T10:15:04.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.554+0000 7f5fbbf24700 1 --2- 172.21.15.3:0/2707709530 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fb409d6f0 0x7f5fb409dac0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.554+0000 7f5fbbf24700 1 --2- 172.21.15.3:0/2707709530 >> v2:172.21.15.3:3300/0 conn(0x7f5fb409e000 0x7f5fb4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.554+0000 7f5fbbf24700 1 -- 172.21.15.3:0/2707709530 >> 172.21.15.3:0/2707709530 conn(0x7f5fb4066f50 msgr2=0x7f5fb409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:04.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.554+0000 7f5fbbf24700 1 -- 172.21.15.3:0/2707709530 shutdown_connections 2023-10-03T10:15:04.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.554+0000 7f5fbbf24700 1 -- 172.21.15.3:0/2707709530 wait complete. 2023-10-03T10:15:04.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.556+0000 7f5fbbf24700 1 Processor -- start 2023-10-03T10:15:04.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.556+0000 7f5fbbf24700 1 -- start start 2023-10-03T10:15:04.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.557+0000 7f5fbbf24700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fb409d6f0 0x7f5fb411fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:04.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.557+0000 7f5fbbf24700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fb409e000 0x7f5fb4120530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:04.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.557+0000 7f5fbbf24700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fb409e410 0x7f5fb4120a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:04.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.557+0000 7f5fbbf24700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb4123b10 con 0x7f5fb409e410 2023-10-03T10:15:04.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.557+0000 7f5fbbf24700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb4123570 con 0x7f5fb409e000 2023-10-03T10:15:04.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.557+0000 7f5fbbf24700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb4121430 con 0x7f5fb409d6f0 2023-10-03T10:15:04.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.557+0000 7f5fb9cc0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fb409d6f0 0x7f5fb411fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:04.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.557+0000 7f5fb9cc0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fb409d6f0 0x7f5fb411fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38420/0 (socket says 172.21.15.3:38420) 2023-10-03T10:15:04.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.557+0000 7f5fb9cc0700 1 -- 172.21.15.3:0/3252478768 learned_addr learned my addr 172.21.15.3:0/3252478768 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:04.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.557+0000 7f5fba4c1700 1 --2- 172.21.15.3:0/3252478768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fb409e410 0x7f5fb4120a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:04.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.557+0000 7f5fb94bf700 1 --2- 172.21.15.3:0/3252478768 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fb409e000 0x7f5fb4120530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:04.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.557+0000 7f5fb9cc0700 1 -- 172.21.15.3:0/3252478768 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fb409e000 msgr2=0x7f5fb4120530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:04.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.557+0000 7f5fb9cc0700 1 --2- 172.21.15.3:0/3252478768 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fb409e000 0x7f5fb4120530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.557+0000 7f5fb9cc0700 1 -- 172.21.15.3:0/3252478768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fb409e410 msgr2=0x7f5fb4120a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:04.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.557+0000 7f5fb9cc0700 1 --2- 172.21.15.3:0/3252478768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fb409e410 0x7f5fb4120a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.557+0000 7f5fb9cc0700 1 -- 172.21.15.3:0/3252478768 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa40094a0 con 0x7f5fb409d6f0 2023-10-03T10:15:04.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.558+0000 7f5fb9cc0700 1 --2- 172.21.15.3:0/3252478768 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fb409d6f0 0x7f5fb411fff0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0009c40 tx=0x7f5fb0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:04.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.558+0000 7f5faa7fc700 1 -- 172.21.15.3:0/3252478768 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0004590 con 0x7f5fb409d6f0 2023-10-03T10:15:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.558+0000 7f5faa7fc700 1 -- 172.21.15.3:0/3252478768 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb001e070 con 0x7f5fb409d6f0 2023-10-03T10:15:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.558+0000 7f5fbbf24700 1 -- 172.21.15.3:0/3252478768 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb0009550 con 0x7f5fb409d6f0 2023-10-03T10:15:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.558+0000 7f5faa7fc700 1 -- 172.21.15.3:0/3252478768 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb00038c0 con 0x7f5fb409d6f0 2023-10-03T10:15:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.558+0000 7f5fbbf24700 1 -- 172.21.15.3:0/3252478768 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb40a1b80 con 0x7f5fb409d6f0 2023-10-03T10:15:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.559+0000 7f5faa7fc700 1 -- 172.21.15.3:0/3252478768 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5fb0003a20 con 0x7f5fb409d6f0 2023-10-03T10:15:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.559+0000 7f5fbbf24700 1 -- 172.21.15.3:0/3252478768 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb409c700 con 0x7f5fb409d6f0 2023-10-03T10:15:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.559+0000 7f5faa7fc700 1 -- 172.21.15.3:0/3252478768 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5fb00407d0 con 0x7f5fb409d6f0 2023-10-03T10:15:04.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.564+0000 7f5faa7fc700 1 -- 172.21.15.3:0/3252478768 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5fb0003cd0 con 0x7f5fb409d6f0 2023-10-03T10:15:04.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.577+0000 7f67c451d700 1 -- 172.21.15.3:0/160625324 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f67bc1251e0 con 0x7f67bc135e50 2023-10-03T10:15:04.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.578+0000 7f67b2ffd700 1 -- 172.21.15.3:0/160625324 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f67bc1251e0 con 0x7f67bc135e50 2023-10-03T10:15:04.578 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:04.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.580+0000 7f67c451d700 1 -- 172.21.15.3:0/160625324 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67bc135e50 msgr2=0x7f67bc1c82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:04.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.580+0000 7f67c451d700 1 --2- 172.21.15.3:0/160625324 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67bc135e50 0x7f67bc1c82f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f67b400ef90 tx=0x7f67b400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.581+0000 7f67c451d700 1 -- 172.21.15.3:0/160625324 shutdown_connections 2023-10-03T10:15:04.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.581+0000 7f67c451d700 1 --2- 172.21.15.3:0/160625324 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67bc135e50 0x7f67bc1c82f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.581+0000 7f67c451d700 1 --2- 172.21.15.3:0/160625324 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67bc09d6f0 0x7f67bc06f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.581+0000 7f67c451d700 1 --2- 172.21.15.3:0/160625324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67bc09e1d0 0x7f67bc06f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.581+0000 7f67c451d700 1 -- 172.21.15.3:0/160625324 >> 172.21.15.3:0/160625324 conn(0x7f67bc066f50 msgr2=0x7f67bc134840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:04.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.581+0000 7f67c451d700 1 -- 172.21.15.3:0/160625324 shutdown_connections 2023-10-03T10:15:04.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.581+0000 7f67c451d700 1 -- 172.21.15.3:0/160625324 wait complete. 2023-10-03T10:15:04.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.594+0000 7f28ae4cc700 1 Processor -- start 2023-10-03T10:15:04.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.594+0000 7f28ae4cc700 1 -- start start 2023-10-03T10:15:04.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.595+0000 7f28ae4cc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f28a806e570 0x7f28a806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:04.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.595+0000 7f28ae4cc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f28a806ee80 0x7f28a8134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:04.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.595+0000 7f28ae4cc700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28a8070ee0 con 0x7f28a809eca0 2023-10-03T10:15:04.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.595+0000 7f28ae4cc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28a8071480 con 0x7f28a806ee80 2023-10-03T10:15:04.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.595+0000 7f28ae4cc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28a8071b90 con 0x7f28a806e570 2023-10-03T10:15:04.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.595+0000 7f28a77fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f28a809eca0 0x7f28a8134610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47010/0 (socket says 172.21.15.3:47010) 2023-10-03T10:15:04.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.595+0000 7f28a77fe700 1 -- 172.21.15.3:0/3569839248 learned_addr learned my addr 172.21.15.3:0/3569839248 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:04.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.595+0000 7f28a7fff700 1 --2- 172.21.15.3:0/3569839248 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f28a806e570 0x7f28a806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:04.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.595+0000 7f28aca69700 1 --2- 172.21.15.3:0/3569839248 >> v2:172.21.15.3:3300/0 conn(0x7f28a806ee80 0x7f28a8134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:04.597 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:04.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.595+0000 7f28a7fff700 1 -- 172.21.15.3:0/3569839248 >> v2:172.21.15.3:3300/0 conn(0x7f28a806ee80 msgr2=0x7f28a8134d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:04.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.595+0000 7f28a7fff700 1 --2- 172.21.15.3:0/3569839248 >> v2:172.21.15.3:3300/0 conn(0x7f28a806ee80 0x7f28a8134d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.595+0000 7f28a7fff700 1 -- 172.21.15.3:0/3569839248 >> v1:172.21.15.3:6789/0 conn(0x7f28a809eca0 legacy=0x7f28a8134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:04.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.595+0000 7f28a7fff700 1 -- 172.21.15.3:0/3569839248 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a80711a0 con 0x7f28a806e570 2023-10-03T10:15:04.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.596+0000 7f28a7fff700 1 --2- 172.21.15.3:0/3569839248 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f28a806e570 0x7f28a806e940 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2898009840 tx=0x7f2898009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b4bc8e7abb82f4f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:04.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.596+0000 7f28a6ffd700 1 -- 172.21.15.3:0/3569839248 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2898004230 con 0x7f28a806e570 2023-10-03T10:15:04.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.596+0000 7f28a6ffd700 1 -- 172.21.15.3:0/3569839248 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2898004390 con 0x7f28a806e570 2023-10-03T10:15:04.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.596+0000 7f28a6ffd700 1 -- 172.21.15.3:0/3569839248 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28980036a0 con 0x7f28a806e570 2023-10-03T10:15:04.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.597+0000 7f28ae4cc700 1 -- 172.21.15.3:0/3569839248 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f28a806e570 msgr2=0x7f28a806e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:04.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.597+0000 7f28ae4cc700 1 --2- 172.21.15.3:0/3569839248 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f28a806e570 0x7f28a806e940 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2898009840 tx=0x7f2898009b50 comp rx=0 tx=0).stop 2023-10-03T10:15:04.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.597+0000 7f28ae4cc700 1 -- 172.21.15.3:0/3569839248 shutdown_connections 2023-10-03T10:15:04.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.597+0000 7f28ae4cc700 1 --2- 172.21.15.3:0/3569839248 >> v2:172.21.15.3:3300/0 conn(0x7f28a806ee80 0x7f28a8134d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.597+0000 7f28ae4cc700 1 --2- 172.21.15.3:0/3569839248 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f28a806e570 0x7f28a806e940 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.597+0000 7f28ae4cc700 1 -- 172.21.15.3:0/3569839248 >> 172.21.15.3:0/3569839248 conn(0x7f28a80a2aa0 msgr2=0x7f28a80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:04.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.597+0000 7f28ae4cc700 1 -- 172.21.15.3:0/3569839248 shutdown_connections 2023-10-03T10:15:04.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.597+0000 7f28ae4cc700 1 -- 172.21.15.3:0/3569839248 wait complete. 2023-10-03T10:15:04.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.598+0000 7f28ae4cc700 1 Processor -- start 2023-10-03T10:15:04.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.599+0000 7f28ae4cc700 1 -- start start 2023-10-03T10:15:04.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.599+0000 7f28ae4cc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28a806e570 0x7f28a8123260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:04.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.599+0000 7f28ae4cc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f28a806ee80 0x7f28a81237a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:04.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.599+0000 7f28ae4cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f28a8123ce0 0x7f28a81cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:04.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.599+0000 7f28ae4cc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a8135c90 con 0x7f28a806e570 2023-10-03T10:15:04.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.599+0000 7f28ae4cc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a8134190 con 0x7f28a8123ce0 2023-10-03T10:15:04.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.599+0000 7f28ae4cc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28a8124ef0 con 0x7f28a806ee80 2023-10-03T10:15:04.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.599+0000 7f28a7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28a806e570 0x7f28a8123260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:04.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.599+0000 7f28a77fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f28a806ee80 0x7f28a81237a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:04.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.599+0000 7f28a7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28a806e570 0x7f28a8123260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57100/0 (socket says 172.21.15.3:57100) 2023-10-03T10:15:04.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.599+0000 7f28a7fff700 1 -- 172.21.15.3:0/3989716630 learned_addr learned my addr 172.21.15.3:0/3989716630 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:04.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.600+0000 7f28aca69700 1 --2- 172.21.15.3:0/3989716630 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f28a8123ce0 0x7f28a81cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:04.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.600+0000 7f28a77fe700 1 -- 172.21.15.3:0/3989716630 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f28a8123ce0 msgr2=0x7f28a81cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:04.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.600+0000 7f28a77fe700 1 --2- 172.21.15.3:0/3989716630 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f28a8123ce0 0x7f28a81cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.600+0000 7f28a77fe700 1 -- 172.21.15.3:0/3989716630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28a806e570 msgr2=0x7f28a8123260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:04.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.600+0000 7f28a77fe700 1 --2- 172.21.15.3:0/3989716630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28a806e570 0x7f28a8123260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.600+0000 7f28a77fe700 1 -- 172.21.15.3:0/3989716630 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28980094a0 con 0x7f28a806ee80 2023-10-03T10:15:04.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.600+0000 7f28a7fff700 1 --2- 172.21.15.3:0/3989716630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28a806e570 0x7f28a8123260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:04.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.600+0000 7f28a77fe700 1 --2- 172.21.15.3:0/3989716630 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f28a806ee80 0x7f28a81237a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f289c00c750 tx=0x7f289c00ca60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:04.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.600+0000 7f28a4ff9700 1 -- 172.21.15.3:0/3989716630 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c004c80 con 0x7f28a806ee80 2023-10-03T10:15:04.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.600+0000 7f28a4ff9700 1 -- 172.21.15.3:0/3989716630 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f289c004de0 con 0x7f28a806ee80 2023-10-03T10:15:04.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.600+0000 7f28a4ff9700 1 -- 172.21.15.3:0/3989716630 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c007310 con 0x7f28a806ee80 2023-10-03T10:15:04.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.600+0000 7f28ae4cc700 1 -- 172.21.15.3:0/3989716630 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a8126750 con 0x7f28a806ee80 2023-10-03T10:15:04.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.600+0000 7f28ae4cc700 1 -- 172.21.15.3:0/3989716630 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a8070660 con 0x7f28a806ee80 2023-10-03T10:15:04.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.601+0000 7f28a4ff9700 1 -- 172.21.15.3:0/3989716630 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f289c007470 con 0x7f28a806ee80 2023-10-03T10:15:04.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.601+0000 7f28a4ff9700 1 -- 172.21.15.3:0/3989716630 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f289c03f350 con 0x7f28a806ee80 2023-10-03T10:15:04.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.602+0000 7f28ae4cc700 1 -- 172.21.15.3:0/3989716630 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28a8127540 con 0x7f28a806ee80 2023-10-03T10:15:04.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.607+0000 7f28a4ff9700 1 -- 172.21.15.3:0/3989716630 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f289c007720 con 0x7f28a806ee80 2023-10-03T10:15:04.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.732+0000 7fdffec42700 1 -- 172.21.15.3:0/2974764632 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdff8074ea0 con 0x7fdff80a1510 2023-10-03T10:15:04.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.732+0000 7fdff57fa700 1 -- 172.21.15.3:0/2974764632 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdfe402b030 con 0x7fdff80a1510 2023-10-03T10:15:04.733 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:04.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.734+0000 7fdffec42700 1 -- 172.21.15.3:0/2974764632 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdff80a1510 msgr2=0x7fdff81c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:04.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.734+0000 7fdffec42700 1 --2- 172.21.15.3:0/2974764632 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdff80a1510 0x7fdff81c8db0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4006f90 tx=0x7fdfe4004750 comp rx=0 tx=0).stop 2023-10-03T10:15:04.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.735+0000 7fdffec42700 1 -- 172.21.15.3:0/2974764632 shutdown_connections 2023-10-03T10:15:04.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.735+0000 7fdffec42700 1 --2- 172.21.15.3:0/2974764632 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdff8139250 0x7fdff81218e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.735+0000 7fdffec42700 1 --2- 172.21.15.3:0/2974764632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdff80a0a30 0x7fdff81c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.735+0000 7fdffec42700 1 --2- 172.21.15.3:0/2974764632 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdff80a1510 0x7fdff81c8db0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.735+0000 7fdffec42700 1 -- 172.21.15.3:0/2974764632 >> 172.21.15.3:0/2974764632 conn(0x7fdff80a6550 msgr2=0x7fdff8137b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:04.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.735+0000 7fdffec42700 1 -- 172.21.15.3:0/2974764632 shutdown_connections 2023-10-03T10:15:04.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.735+0000 7fdffec42700 1 -- 172.21.15.3:0/2974764632 wait complete. 2023-10-03T10:15:04.749 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:04.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.777+0000 7f5fbbf24700 1 -- 172.21.15.3:0/3252478768 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5fb4074ea0 con 0x7f5fb409d6f0 2023-10-03T10:15:04.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.777+0000 7f5faa7fc700 1 -- 172.21.15.3:0/3252478768 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5fb0015020 con 0x7f5fb409d6f0 2023-10-03T10:15:04.777 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:04.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.779+0000 7f5fbbf24700 1 -- 172.21.15.3:0/3252478768 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fb409d6f0 msgr2=0x7f5fb411fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:04.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.779+0000 7f5fbbf24700 1 --2- 172.21.15.3:0/3252478768 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fb409d6f0 0x7f5fb411fff0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0009c40 tx=0x7f5fb0004750 comp rx=0 tx=0).stop 2023-10-03T10:15:04.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.779+0000 7f5fbbf24700 1 -- 172.21.15.3:0/3252478768 shutdown_connections 2023-10-03T10:15:04.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.779+0000 7f5fbbf24700 1 --2- 172.21.15.3:0/3252478768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fb409e410 0x7f5fb4120a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.779+0000 7f5fbbf24700 1 --2- 172.21.15.3:0/3252478768 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fb409d6f0 0x7f5fb411fff0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.779+0000 7f5fbbf24700 1 --2- 172.21.15.3:0/3252478768 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fb409e000 0x7f5fb4120530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.779+0000 7f5fbbf24700 1 -- 172.21.15.3:0/3252478768 >> 172.21.15.3:0/3252478768 conn(0x7f5fb4066f50 msgr2=0x7f5fb4071110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:04.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.779+0000 7f5fbbf24700 1 -- 172.21.15.3:0/3252478768 shutdown_connections 2023-10-03T10:15:04.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.780+0000 7f5fbbf24700 1 -- 172.21.15.3:0/3252478768 wait complete. 2023-10-03T10:15:04.793 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:04.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.816+0000 7f28ae4cc700 1 -- 172.21.15.3:0/3989716630 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f28a8074ea0 con 0x7f28a806ee80 2023-10-03T10:15:04.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.816+0000 7f28a4ff9700 1 -- 172.21.15.3:0/3989716630 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f289c011360 con 0x7f28a806ee80 2023-10-03T10:15:04.817 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:04.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.819+0000 7f28ae4cc700 1 -- 172.21.15.3:0/3989716630 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f28a806ee80 msgr2=0x7f28a81237a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:04.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.819+0000 7f28ae4cc700 1 --2- 172.21.15.3:0/3989716630 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f28a806ee80 0x7f28a81237a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f289c00c750 tx=0x7f289c00ca60 comp rx=0 tx=0).stop 2023-10-03T10:15:04.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.819+0000 7f28ae4cc700 1 -- 172.21.15.3:0/3989716630 shutdown_connections 2023-10-03T10:15:04.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.819+0000 7f28ae4cc700 1 --2- 172.21.15.3:0/3989716630 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f28a8123ce0 0x7f28a81cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.819+0000 7f28ae4cc700 1 --2- 172.21.15.3:0/3989716630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28a806e570 0x7f28a8123260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.819+0000 7f28ae4cc700 1 --2- 172.21.15.3:0/3989716630 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f28a806ee80 0x7f28a81237a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:04.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.819+0000 7f28ae4cc700 1 -- 172.21.15.3:0/3989716630 >> 172.21.15.3:0/3989716630 conn(0x7f28a80a2aa0 msgr2=0x7f28a80a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:04.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.819+0000 7f28ae4cc700 1 -- 172.21.15.3:0/3989716630 shutdown_connections 2023-10-03T10:15:04.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:04.819+0000 7f28ae4cc700 1 -- 172.21.15.3:0/3989716630 wait complete. 2023-10-03T10:15:04.833 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:05.598 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:05.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.744+0000 7fa592749700 1 Processor -- start 2023-10-03T10:15:05.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.744+0000 7fa592749700 1 -- start start 2023-10-03T10:15:05.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.744+0000 7fa592749700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa58c06a770 0x7fa58c06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:05.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.744+0000 7fa592749700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa58c131d10 0x7fa58c139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:05.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.744+0000 7fa592749700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa58c09ece0 con 0x7fa58c06b080 2023-10-03T10:15:05.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.744+0000 7fa592749700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c09e760 con 0x7fa58c131d10 2023-10-03T10:15:05.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.744+0000 7fa592749700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c09e040 con 0x7fa58c06a770 2023-10-03T10:15:05.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.744+0000 7fa58bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa58c06a770 0x7fa58c06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:05.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.744+0000 7fa590ce6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa58c131d10 0x7fa58c139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:05.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.744+0000 7fa58b7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa58c06b080 0x7fa58c131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47016/0 (socket says 172.21.15.3:47016) 2023-10-03T10:15:05.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.744+0000 7fa58b7fe700 1 -- 172.21.15.3:0/3082018805 learned_addr learned my addr 172.21.15.3:0/3082018805 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:05.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.745+0000 7fa58affd700 1 -- 172.21.15.3:0/3082018805 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 701287480 0 0) 0x7fa58c09ece0 con 0x7fa58c06b080 2023-10-03T10:15:05.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.745+0000 7fa58bfff700 1 -- 172.21.15.3:0/3082018805 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa58c131d10 msgr2=0x7fa58c139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:05.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.745+0000 7fa58bfff700 1 --2- 172.21.15.3:0/3082018805 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa58c131d10 0x7fa58c139860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:05.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.745+0000 7fa58bfff700 1 -- 172.21.15.3:0/3082018805 >> v1:172.21.15.3:6789/0 conn(0x7fa58c06b080 legacy=0x7fa58c131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:05.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.745+0000 7fa58bfff700 1 -- 172.21.15.3:0/3082018805 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58c09f630 con 0x7fa58c06a770 2023-10-03T10:15:05.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.745+0000 7fa590ce6700 1 --2- 172.21.15.3:0/3082018805 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa58c131d10 0x7fa58c139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:05.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.745+0000 7fa58bfff700 1 --2- 172.21.15.3:0/3082018805 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa58c06a770 0x7fa58c06ab40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009870 tx=0x7fa57c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3a3aab2cdac0843 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:05.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.745+0000 7fa58affd700 1 -- 172.21.15.3:0/3082018805 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c011830 con 0x7fa58c06a770 2023-10-03T10:15:05.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.745+0000 7fa58affd700 1 -- 172.21.15.3:0/3082018805 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa57c011990 con 0x7fa58c06a770 2023-10-03T10:15:05.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.745+0000 7fa58affd700 1 -- 172.21.15.3:0/3082018805 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c011d20 con 0x7fa58c06a770 2023-10-03T10:15:05.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.746+0000 7fa592749700 1 -- 172.21.15.3:0/3082018805 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa58c06a770 msgr2=0x7fa58c06ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:05.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.746+0000 7fa592749700 1 --2- 172.21.15.3:0/3082018805 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa58c06a770 0x7fa58c06ab40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009870 tx=0x7fa57c009b80 comp rx=0 tx=0).stop 2023-10-03T10:15:05.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.746+0000 7fa592749700 1 -- 172.21.15.3:0/3082018805 shutdown_connections 2023-10-03T10:15:05.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.746+0000 7fa592749700 1 --2- 172.21.15.3:0/3082018805 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa58c131d10 0x7fa58c139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:05.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.746+0000 7fa592749700 1 --2- 172.21.15.3:0/3082018805 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa58c06a770 0x7fa58c06ab40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:05.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.746+0000 7fa592749700 1 -- 172.21.15.3:0/3082018805 >> 172.21.15.3:0/3082018805 conn(0x7fa58c066f50 msgr2=0x7fa58c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:05.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.746+0000 7fa592749700 1 -- 172.21.15.3:0/3082018805 shutdown_connections 2023-10-03T10:15:05.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.746+0000 7fa592749700 1 -- 172.21.15.3:0/3082018805 wait complete. 2023-10-03T10:15:05.749 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:05.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.747+0000 7fa592749700 1 Processor -- start 2023-10-03T10:15:05.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.747+0000 7fa592749700 1 -- start start 2023-10-03T10:15:05.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.748+0000 7fa592749700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa58c06a770 0x7fa58c120380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:05.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.748+0000 7fa592749700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa58c06b080 0x7fa58c1208c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:05.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.748+0000 7fa592749700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa58c131d10 0x7fa58c120e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:05.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.748+0000 7fa592749700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c123f30 con 0x7fa58c06a770 2023-10-03T10:15:05.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.748+0000 7fa592749700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c123580 con 0x7fa58c06b080 2023-10-03T10:15:05.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.748+0000 7fa592749700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c121fa0 con 0x7fa58c131d10 2023-10-03T10:15:05.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.748+0000 7fa58b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa58c06b080 0x7fa58c1208c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:05.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.748+0000 7fa590ce6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa58c131d10 0x7fa58c120e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:05.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.748+0000 7fa58bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa58c06a770 0x7fa58c120380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:05.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.748+0000 7fa590ce6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa58c131d10 0x7fa58c120e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38446/0 (socket says 172.21.15.3:38446) 2023-10-03T10:15:05.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.748+0000 7fa58b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa58c06b080 0x7fa58c1208c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51220/0 (socket says 172.21.15.3:51220) 2023-10-03T10:15:05.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.748+0000 7fa590ce6700 1 -- 172.21.15.3:0/1876769778 learned_addr learned my addr 172.21.15.3:0/1876769778 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:05.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.748+0000 7fa590ce6700 1 -- 172.21.15.3:0/1876769778 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa58c06b080 msgr2=0x7fa58c1208c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:05.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.748+0000 7fa590ce6700 1 --2- 172.21.15.3:0/1876769778 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa58c06b080 0x7fa58c1208c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:05.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.748+0000 7fa590ce6700 1 -- 172.21.15.3:0/1876769778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa58c06a770 msgr2=0x7fa58c120380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:05.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.748+0000 7fa590ce6700 1 --2- 172.21.15.3:0/1876769778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa58c06a770 0x7fa58c120380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:05.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.748+0000 7fa590ce6700 1 -- 172.21.15.3:0/1876769778 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa584009a70 con 0x7fa58c131d10 2023-10-03T10:15:05.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.748+0000 7fa58bfff700 1 --2- 172.21.15.3:0/1876769778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa58c06a770 0x7fa58c120380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:05.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.749+0000 7fa590ce6700 1 --2- 172.21.15.3:0/1876769778 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa58c131d10 0x7fa58c120e00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa58400efc0 tx=0x7fa58400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:05.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.749+0000 7fa588ff9700 1 -- 172.21.15.3:0/1876769778 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58400ca90 con 0x7fa58c131d10 2023-10-03T10:15:05.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.749+0000 7fa588ff9700 1 -- 172.21.15.3:0/1876769778 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58400f040 con 0x7fa58c131d10 2023-10-03T10:15:05.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.749+0000 7fa592749700 1 -- 172.21.15.3:0/1876769778 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa57c0094a0 con 0x7fa58c131d10 2023-10-03T10:15:05.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.749+0000 7fa588ff9700 1 -- 172.21.15.3:0/1876769778 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58400ce10 con 0x7fa58c131d10 2023-10-03T10:15:05.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.749+0000 7fa592749700 1 -- 172.21.15.3:0/1876769778 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa58c1312b0 con 0x7fa58c131d10 2023-10-03T10:15:05.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.750+0000 7fa588ff9700 1 -- 172.21.15.3:0/1876769778 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa584014480 con 0x7fa58c131d10 2023-10-03T10:15:05.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.750+0000 7fa588ff9700 1 -- 172.21.15.3:0/1876769778 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa58403fd70 con 0x7fa58c131d10 2023-10-03T10:15:05.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.750+0000 7fa592749700 1 -- 172.21.15.3:0/1876769778 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa58c123bc0 con 0x7fa58c131d10 2023-10-03T10:15:05.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.755+0000 7fa588ff9700 1 -- 172.21.15.3:0/1876769778 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa584012930 con 0x7fa58c131d10 2023-10-03T10:15:05.794 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:05.833 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:05.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.891+0000 7f8b2bf2d700 1 Processor -- start 2023-10-03T10:15:05.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.891+0000 7f8b2bf2d700 1 -- start start 2023-10-03T10:15:05.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.891+0000 7f8b2bf2d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b2406e570 0x7f8b2406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:05.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.891+0000 7f8b2bf2d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b2409eca0 0x7f8b2406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:05.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.891+0000 7f8b2bf2d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b24070ee0 con 0x7f8b24134610 2023-10-03T10:15:05.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.891+0000 7f8b2bf2d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b24071480 con 0x7f8b2409eca0 2023-10-03T10:15:05.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.891+0000 7f8b2bf2d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2406f3c0 con 0x7f8b2406e570 2023-10-03T10:15:05.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.892+0000 7f8b29cc9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b2406e570 0x7f8b2406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:05.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.892+0000 7f8b29cc9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b2406e570 0x7f8b2406e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38462/0 (socket says 172.21.15.3:38462) 2023-10-03T10:15:05.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.892+0000 7f8b29cc9700 1 -- 172.21.15.3:0/3932611964 learned_addr learned my addr 172.21.15.3:0/3932611964 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:05.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.892+0000 7f8b294c8700 1 --2- 172.21.15.3:0/3932611964 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b2409eca0 0x7f8b2406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:05.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.892+0000 7f8b294c8700 1 -- 172.21.15.3:0/3932611964 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b2406e570 msgr2=0x7f8b2406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:05.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.892+0000 7f8b294c8700 1 --2- 172.21.15.3:0/3932611964 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b2406e570 0x7f8b2406e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:05.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.892+0000 7f8b294c8700 1 -- 172.21.15.3:0/3932611964 >> v1:172.21.15.3:6789/0 conn(0x7f8b24134610 legacy=0x7f8b24134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:05.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.892+0000 7f8b294c8700 1 -- 172.21.15.3:0/3932611964 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b24071b90 con 0x7f8b2409eca0 2023-10-03T10:15:05.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.892+0000 7f8b28cc7700 1 -- 172.21.15.3:0/3932611964 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2719961342 0 0) 0x7f8b24070ee0 con 0x7f8b24134610 2023-10-03T10:15:05.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.892+0000 7f8b294c8700 1 --2- 172.21.15.3:0/3932611964 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b2409eca0 0x7f8b2406ee80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8b14009870 tx=0x7f8b14009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85cd106a1caf516e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:05.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.893+0000 7f8b28cc7700 1 -- 172.21.15.3:0/3932611964 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14011830 con 0x7f8b2409eca0 2023-10-03T10:15:05.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.893+0000 7f8b28cc7700 1 -- 172.21.15.3:0/3932611964 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b14011990 con 0x7f8b2409eca0 2023-10-03T10:15:05.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.893+0000 7f8b28cc7700 1 -- 172.21.15.3:0/3932611964 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14011d20 con 0x7f8b2409eca0 2023-10-03T10:15:05.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.894+0000 7f8b2bf2d700 1 -- 172.21.15.3:0/3932611964 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b2409eca0 msgr2=0x7f8b2406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:05.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.894+0000 7f8b2bf2d700 1 --2- 172.21.15.3:0/3932611964 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b2409eca0 0x7f8b2406ee80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8b14009870 tx=0x7f8b14009b80 comp rx=0 tx=0).stop 2023-10-03T10:15:05.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.894+0000 7f8b2bf2d700 1 -- 172.21.15.3:0/3932611964 shutdown_connections 2023-10-03T10:15:05.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.894+0000 7f8b2bf2d700 1 --2- 172.21.15.3:0/3932611964 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b2406e570 0x7f8b2406e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:05.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.894+0000 7f8b2bf2d700 1 --2- 172.21.15.3:0/3932611964 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b2409eca0 0x7f8b2406ee80 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:05.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.894+0000 7f8b2bf2d700 1 -- 172.21.15.3:0/3932611964 >> 172.21.15.3:0/3932611964 conn(0x7f8b240a2aa0 msgr2=0x7f8b240a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:05.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.894+0000 7f8b2bf2d700 1 -- 172.21.15.3:0/3932611964 shutdown_connections 2023-10-03T10:15:05.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.894+0000 7f8b2bf2d700 1 -- 172.21.15.3:0/3932611964 wait complete. 2023-10-03T10:15:05.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.895+0000 7f8b2bf2d700 1 Processor -- start 2023-10-03T10:15:05.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.896+0000 7f8b2bf2d700 1 -- start start 2023-10-03T10:15:05.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.896+0000 7f8b2bf2d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b2406e570 0x7f8b241340b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:05.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.896+0000 7f8b2bf2d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b24134610 0x7f8b24132e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:05.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.896+0000 7f8b2bf2d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b241333b0 0x7f8b24133820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:05.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.896+0000 7f8b2bf2d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b241401b0 con 0x7f8b2406e570 2023-10-03T10:15:05.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.896+0000 7f8b2bf2d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2413dea0 con 0x7f8b24134610 2023-10-03T10:15:05.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.896+0000 7f8b2bf2d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b24140830 con 0x7f8b241333b0 2023-10-03T10:15:05.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.896+0000 7f8b2a4ca700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b241333b0 0x7f8b24133820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:05.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.896+0000 7f8b2a4ca700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b241333b0 0x7f8b24133820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38476/0 (socket says 172.21.15.3:38476) 2023-10-03T10:15:05.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.896+0000 7f8b2a4ca700 1 -- 172.21.15.3:0/174847762 learned_addr learned my addr 172.21.15.3:0/174847762 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:05.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.897+0000 7f8b29cc9700 1 --2- 172.21.15.3:0/174847762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b2406e570 0x7f8b241340b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:05.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.897+0000 7f8b294c8700 1 --2- 172.21.15.3:0/174847762 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b24134610 0x7f8b24132e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:05.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.897+0000 7f8b29cc9700 1 -- 172.21.15.3:0/174847762 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b241333b0 msgr2=0x7f8b24133820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:05.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.897+0000 7f8b29cc9700 1 --2- 172.21.15.3:0/174847762 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b241333b0 0x7f8b24133820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:05.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.897+0000 7f8b29cc9700 1 -- 172.21.15.3:0/174847762 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b24134610 msgr2=0x7f8b24132e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:05.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.897+0000 7f8b29cc9700 1 --2- 172.21.15.3:0/174847762 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b24134610 0x7f8b24132e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:05.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.897+0000 7f8b29cc9700 1 -- 172.21.15.3:0/174847762 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b140094a0 con 0x7f8b2406e570 2023-10-03T10:15:05.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.897+0000 7f8b29cc9700 1 --2- 172.21.15.3:0/174847762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b2406e570 0x7f8b241340b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8b2000e8e0 tx=0x7f8b2000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:05.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.897+0000 7f8b2a4ca700 1 --2- 172.21.15.3:0/174847762 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b241333b0 0x7f8b24133820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:05.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.897+0000 7f8b1a7fc700 1 -- 172.21.15.3:0/174847762 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b20013680 con 0x7f8b2406e570 2023-10-03T10:15:05.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.897+0000 7f8b2bf2d700 1 -- 172.21.15.3:0/174847762 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b24070610 con 0x7f8b2406e570 2023-10-03T10:15:05.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.897+0000 7f8b1a7fc700 1 -- 172.21.15.3:0/174847762 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b200137e0 con 0x7f8b2406e570 2023-10-03T10:15:05.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.898+0000 7f8b2bf2d700 1 -- 172.21.15.3:0/174847762 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b2413ebd0 con 0x7f8b2406e570 2023-10-03T10:15:05.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.898+0000 7f8b1a7fc700 1 -- 172.21.15.3:0/174847762 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b20013b50 con 0x7f8b2406e570 2023-10-03T10:15:05.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.898+0000 7f8b1a7fc700 1 -- 172.21.15.3:0/174847762 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8b20013cb0 con 0x7f8b2406e570 2023-10-03T10:15:05.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.899+0000 7f8b1a7fc700 1 -- 172.21.15.3:0/174847762 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8b2003fb00 con 0x7f8b2406e570 2023-10-03T10:15:05.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.899+0000 7f8b2bf2d700 1 -- 172.21.15.3:0/174847762 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b0c0052e0 con 0x7f8b2406e570 2023-10-03T10:15:05.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.904+0000 7f8b1a7fc700 1 -- 172.21.15.3:0/174847762 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8b2003fe60 con 0x7f8b2406e570 2023-10-03T10:15:05.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.940+0000 7fd64e206700 1 Processor -- start 2023-10-03T10:15:05.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.940+0000 7fd64e206700 1 -- start start 2023-10-03T10:15:05.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.940+0000 7fd64e206700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd648133a80 0x7fd648133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:05.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.940+0000 7fd64e206700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd64812ff20 0x7fd648130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:05.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.940+0000 7fd64e206700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd648130870 0x7fd64813a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:05.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.940+0000 7fd64e206700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd64806c570 con 0x7fd648130870 2023-10-03T10:15:05.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.940+0000 7fd64e206700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd64806bc00 con 0x7fd64812ff20 2023-10-03T10:15:05.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.940+0000 7fd64e206700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6480a0040 con 0x7fd648133a80 2023-10-03T10:15:05.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.941+0000 7fd6477fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd648133a80 0x7fd648133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:05.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.941+0000 7fd646ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd64812ff20 0x7fd648130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:05.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.941+0000 7fd646ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd64812ff20 0x7fd648130330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51252/0 (socket says 172.21.15.3:51252) 2023-10-03T10:15:05.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.941+0000 7fd646ffd700 1 -- 172.21.15.3:0/123183035 learned_addr learned my addr 172.21.15.3:0/123183035 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:05.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.941+0000 7fd647fff700 1 --2- 172.21.15.3:0/123183035 >> v2:172.21.15.3:3300/0 conn(0x7fd648130870 0x7fd64813a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:05.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.941+0000 7fd6477fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd648133a80 0x7fd648133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38486/0 (socket says 172.21.15.3:38486) 2023-10-03T10:15:05.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.941+0000 7fd647fff700 1 -- 172.21.15.3:0/123183035 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd648133a80 msgr2=0x7fd648133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:05.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.941+0000 7fd647fff700 1 --2- 172.21.15.3:0/123183035 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd648133a80 0x7fd648133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:05.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.941+0000 7fd647fff700 1 -- 172.21.15.3:0/123183035 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd64812ff20 msgr2=0x7fd648130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:05.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.941+0000 7fd647fff700 1 --2- 172.21.15.3:0/123183035 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd64812ff20 0x7fd648130330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:05.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.941+0000 7fd647fff700 1 -- 172.21.15.3:0/123183035 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd648133860 con 0x7fd648130870 2023-10-03T10:15:05.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.941+0000 7fd647fff700 1 --2- 172.21.15.3:0/123183035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd648130870 0x7fd64813a840 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd640009d90 tx=0x7fd64000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feaa5830c8626886 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:05.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.942+0000 7fd6467fc700 1 -- 172.21.15.3:0/123183035 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64000e070 con 0x7fd648130870 2023-10-03T10:15:05.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.942+0000 7fd6467fc700 1 -- 172.21.15.3:0/123183035 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64000bb20 con 0x7fd648130870 2023-10-03T10:15:05.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.942+0000 7fd6467fc700 1 -- 172.21.15.3:0/123183035 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd640012430 con 0x7fd648130870 2023-10-03T10:15:05.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.942+0000 7fd64e206700 1 -- 172.21.15.3:0/123183035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd648130870 msgr2=0x7fd64813a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:05.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.942+0000 7fd64e206700 1 --2- 172.21.15.3:0/123183035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd648130870 0x7fd64813a840 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd640009d90 tx=0x7fd64000b330 comp rx=0 tx=0).stop 2023-10-03T10:15:05.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.942+0000 7fd64e206700 1 -- 172.21.15.3:0/123183035 shutdown_connections 2023-10-03T10:15:05.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.942+0000 7fd64e206700 1 --2- 172.21.15.3:0/123183035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd648130870 0x7fd64813a840 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:05.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.942+0000 7fd64e206700 1 --2- 172.21.15.3:0/123183035 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd648133a80 0x7fd648133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:05.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.942+0000 7fd64e206700 1 --2- 172.21.15.3:0/123183035 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd64812ff20 0x7fd648130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:05.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.942+0000 7fd64e206700 1 -- 172.21.15.3:0/123183035 >> 172.21.15.3:0/123183035 conn(0x7fd6480a4210 msgr2=0x7fd6480a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:05.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.943+0000 7fd64e206700 1 -- 172.21.15.3:0/123183035 shutdown_connections 2023-10-03T10:15:05.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.943+0000 7fd64e206700 1 -- 172.21.15.3:0/123183035 wait complete. 2023-10-03T10:15:05.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.944+0000 7fd64e206700 1 Processor -- start 2023-10-03T10:15:05.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.944+0000 7fd64e206700 1 -- start start 2023-10-03T10:15:05.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.944+0000 7fd64e206700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd64812ff20 0x7fd64809c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:05.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.944+0000 7fd64e206700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd648130870 0x7fd64809c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:05.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.944+0000 7fd64e206700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd648133a80 0x7fd64809ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:05.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.944+0000 7fd64e206700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64809df10 con 0x7fd648133a80 2023-10-03T10:15:05.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.944+0000 7fd64e206700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd64806cf30 con 0x7fd648130870 2023-10-03T10:15:05.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.944+0000 7fd64e206700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd64811fe10 con 0x7fd64812ff20 2023-10-03T10:15:05.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.945+0000 7fd647fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd648133a80 0x7fd64809ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:05.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.945+0000 7fd646ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd648130870 0x7fd64809c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:05.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.945+0000 7fd647fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd648133a80 0x7fd64809ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57136/0 (socket says 172.21.15.3:57136) 2023-10-03T10:15:05.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.945+0000 7fd647fff700 1 -- 172.21.15.3:0/727838929 learned_addr learned my addr 172.21.15.3:0/727838929 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:05.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.945+0000 7fd6477fe700 1 --2- 172.21.15.3:0/727838929 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd64812ff20 0x7fd64809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:05.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.945+0000 7fd646ffd700 1 -- 172.21.15.3:0/727838929 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd64812ff20 msgr2=0x7fd64809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:05.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.945+0000 7fd646ffd700 1 --2- 172.21.15.3:0/727838929 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd64812ff20 0x7fd64809c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:05.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.945+0000 7fd646ffd700 1 -- 172.21.15.3:0/727838929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd648133a80 msgr2=0x7fd64809ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:05.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.945+0000 7fd646ffd700 1 --2- 172.21.15.3:0/727838929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd648133a80 0x7fd64809ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:05.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.945+0000 7fd646ffd700 1 -- 172.21.15.3:0/727838929 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd640009a70 con 0x7fd648130870 2023-10-03T10:15:05.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.945+0000 7fd647fff700 1 --2- 172.21.15.3:0/727838929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd648133a80 0x7fd64809ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:05.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.945+0000 7fd646ffd700 1 --2- 172.21.15.3:0/727838929 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd648130870 0x7fd64809c940 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00e8b0 tx=0x7fd63c00ebc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:05.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.945+0000 7fd62ffff700 1 -- 172.21.15.3:0/727838929 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c013680 con 0x7fd648130870 2023-10-03T10:15:05.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.946+0000 7fd62ffff700 1 -- 172.21.15.3:0/727838929 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63c0137e0 con 0x7fd648130870 2023-10-03T10:15:05.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.946+0000 7fd62ffff700 1 -- 172.21.15.3:0/727838929 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c013b50 con 0x7fd648130870 2023-10-03T10:15:05.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.946+0000 7fd64e206700 1 -- 172.21.15.3:0/727838929 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd64812d7e0 con 0x7fd648130870 2023-10-03T10:15:05.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.946+0000 7fd64e206700 1 -- 172.21.15.3:0/727838929 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd648120b40 con 0x7fd648130870 2023-10-03T10:15:05.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.947+0000 7fd62ffff700 1 -- 172.21.15.3:0/727838929 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd63c013cb0 con 0x7fd648130870 2023-10-03T10:15:05.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.947+0000 7fd64e206700 1 -- 172.21.15.3:0/727838929 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd648074ea0 con 0x7fd648130870 2023-10-03T10:15:05.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.947+0000 7fd62ffff700 1 -- 172.21.15.3:0/727838929 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd63c03fa00 con 0x7fd648130870 2023-10-03T10:15:05.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.952+0000 7fd62ffff700 1 -- 172.21.15.3:0/727838929 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd63c01adc0 con 0x7fd648130870 2023-10-03T10:15:05.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.979+0000 7fa592749700 1 -- 172.21.15.3:0/1876769778 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa58c124660 con 0x7fa58c131d10 2023-10-03T10:15:05.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.979+0000 7fa588ff9700 1 -- 172.21.15.3:0/1876769778 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa584012340 con 0x7fa58c131d10 2023-10-03T10:15:05.981 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:05.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.982+0000 7fa592749700 1 -- 172.21.15.3:0/1876769778 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa58c131d10 msgr2=0x7fa58c120e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:05.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.982+0000 7fa592749700 1 --2- 172.21.15.3:0/1876769778 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa58c131d10 0x7fa58c120e00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa58400efc0 tx=0x7fa58400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:15:05.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.982+0000 7fa592749700 1 -- 172.21.15.3:0/1876769778 shutdown_connections 2023-10-03T10:15:05.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.982+0000 7fa592749700 1 --2- 172.21.15.3:0/1876769778 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa58c131d10 0x7fa58c120e00 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:05.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.982+0000 7fa592749700 1 --2- 172.21.15.3:0/1876769778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa58c06a770 0x7fa58c120380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:05.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.982+0000 7fa592749700 1 --2- 172.21.15.3:0/1876769778 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa58c06b080 0x7fa58c1208c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:05.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.982+0000 7fa592749700 1 -- 172.21.15.3:0/1876769778 >> 172.21.15.3:0/1876769778 conn(0x7fa58c066f50 msgr2=0x7fa58c070070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:05.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.983+0000 7fa592749700 1 -- 172.21.15.3:0/1876769778 shutdown_connections 2023-10-03T10:15:05.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.983+0000 7fa592749700 1 -- 172.21.15.3:0/1876769778 wait complete. 2023-10-03T10:15:05.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.986+0000 7fc7136df700 1 Processor -- start 2023-10-03T10:15:05.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.987+0000 7fc7136df700 1 -- start start 2023-10-03T10:15:05.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.987+0000 7fc7136df700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc70c09e1d0 0x7fc70c135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:05.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.987+0000 7fc7136df700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc70c135e50 0x7fc70c138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:05.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.987+0000 7fc7136df700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc70c09f130 con 0x7fc70c09d6f0 2023-10-03T10:15:05.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.987+0000 7fc7136df700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc70c09f520 con 0x7fc70c135e50 2023-10-03T10:15:05.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.987+0000 7fc7136df700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c09e930 con 0x7fc70c09e1d0 2023-10-03T10:15:05.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.987+0000 7fc711c7c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc70c135e50 0x7fc70c138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:05.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.987+0000 7fc711c7c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc70c135e50 0x7fc70c138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57152/0 (socket says 172.21.15.3:57152) 2023-10-03T10:15:05.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.987+0000 7fc711c7c700 1 -- 172.21.15.3:0/1326422194 learned_addr learned my addr 172.21.15.3:0/1326422194 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:05.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.988+0000 7fc710c7a700 1 --2- 172.21.15.3:0/1326422194 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc70c09e1d0 0x7fc70c135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:05.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.988+0000 7fc711c7c700 1 -- 172.21.15.3:0/1326422194 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc70c09e1d0 msgr2=0x7fc70c135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:05.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.988+0000 7fc711c7c700 1 --2- 172.21.15.3:0/1326422194 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc70c09e1d0 0x7fc70c135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:05.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.988+0000 7fc711c7c700 1 -- 172.21.15.3:0/1326422194 >> v1:172.21.15.3:6789/0 conn(0x7fc70c09d6f0 legacy=0x7fc70c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:05.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.988+0000 7fc711c7c700 1 -- 172.21.15.3:0/1326422194 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70c09b2d0 con 0x7fc70c135e50 2023-10-03T10:15:05.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.988+0000 7fc711c7c700 1 --2- 172.21.15.3:0/1326422194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc70c135e50 0x7fc70c138230 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fc70400d180 tx=0x7fc70400d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99aafe93ebb5bbf3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:05.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.988+0000 7fc703fff700 1 -- 172.21.15.3:0/1326422194 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc704010070 con 0x7fc70c135e50 2023-10-03T10:15:05.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.988+0000 7fc703fff700 1 -- 172.21.15.3:0/1326422194 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70400ddb0 con 0x7fc70c135e50 2023-10-03T10:15:05.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.988+0000 7fc703fff700 1 -- 172.21.15.3:0/1326422194 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70400b310 con 0x7fc70c135e50 2023-10-03T10:15:05.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.989+0000 7fc7136df700 1 -- 172.21.15.3:0/1326422194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc70c135e50 msgr2=0x7fc70c138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:05.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.989+0000 7fc7136df700 1 --2- 172.21.15.3:0/1326422194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc70c135e50 0x7fc70c138230 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fc70400d180 tx=0x7fc70400d490 comp rx=0 tx=0).stop 2023-10-03T10:15:05.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.989+0000 7fc7136df700 1 -- 172.21.15.3:0/1326422194 shutdown_connections 2023-10-03T10:15:05.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.989+0000 7fc7136df700 1 --2- 172.21.15.3:0/1326422194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc70c135e50 0x7fc70c138230 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:05.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.989+0000 7fc7136df700 1 --2- 172.21.15.3:0/1326422194 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc70c09e1d0 0x7fc70c135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:05.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.989+0000 7fc7136df700 1 -- 172.21.15.3:0/1326422194 >> 172.21.15.3:0/1326422194 conn(0x7fc70c066f50 msgr2=0x7fc70c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:05.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.989+0000 7fc7136df700 1 -- 172.21.15.3:0/1326422194 shutdown_connections 2023-10-03T10:15:05.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.989+0000 7fc7136df700 1 -- 172.21.15.3:0/1326422194 wait complete. 2023-10-03T10:15:05.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.990+0000 7fc7136df700 1 Processor -- start 2023-10-03T10:15:05.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.990+0000 7fc7136df700 1 -- start start 2023-10-03T10:15:05.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.991+0000 7fc7136df700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc70c09d6f0 0x7fc70c11e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:05.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.991+0000 7fc7136df700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc70c09e1d0 0x7fc70c11e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:05.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.991+0000 7fc7136df700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc70c135e50 0x7fc70c1c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:05.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.991+0000 7fc7136df700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c11fbd0 con 0x7fc70c09e1d0 2023-10-03T10:15:05.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.991+0000 7fc7136df700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c130060 con 0x7fc70c09d6f0 2023-10-03T10:15:05.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.991+0000 7fc7136df700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c09c700 con 0x7fc70c135e50 2023-10-03T10:15:05.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.991+0000 7fc710c7a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc70c09e1d0 0x7fc70c11e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:05.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.991+0000 7fc710c7a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc70c09e1d0 0x7fc70c11e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57158/0 (socket says 172.21.15.3:57158) 2023-10-03T10:15:05.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.991+0000 7fc710c7a700 1 -- 172.21.15.3:0/3142981844 learned_addr learned my addr 172.21.15.3:0/3142981844 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:05.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.991+0000 7fc71147b700 1 --2- 172.21.15.3:0/3142981844 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc70c09d6f0 0x7fc70c11e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:05.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.991+0000 7fc710c7a700 1 -- 172.21.15.3:0/3142981844 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc70c135e50 msgr2=0x7fc70c1c8170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:15:05.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.991+0000 7fc710c7a700 1 --2- 172.21.15.3:0/3142981844 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc70c135e50 0x7fc70c1c8170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:05.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.991+0000 7fc710c7a700 1 -- 172.21.15.3:0/3142981844 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc70c09d6f0 msgr2=0x7fc70c11e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:05.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.991+0000 7fc710c7a700 1 --2- 172.21.15.3:0/3142981844 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc70c09d6f0 0x7fc70c11e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:05.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.991+0000 7fc710c7a700 1 -- 172.21.15.3:0/3142981844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70400bd40 con 0x7fc70c09e1d0 2023-10-03T10:15:05.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.992+0000 7fc710c7a700 1 --2- 172.21.15.3:0/3142981844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc70c09e1d0 0x7fc70c11e680 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00e840 tx=0x7fc6fc00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:05.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.992+0000 7fc701ffb700 1 -- 172.21.15.3:0/3142981844 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc013610 con 0x7fc70c09e1d0 2023-10-03T10:15:05.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.992+0000 7fc701ffb700 1 -- 172.21.15.3:0/3142981844 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6fc013770 con 0x7fc70c09e1d0 2023-10-03T10:15:05.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.992+0000 7fc701ffb700 1 -- 172.21.15.3:0/3142981844 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc013b00 con 0x7fc70c09e1d0 2023-10-03T10:15:05.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.992+0000 7fc7136df700 1 -- 172.21.15.3:0/3142981844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc70c0a1cc0 con 0x7fc70c09e1d0 2023-10-03T10:15:05.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.992+0000 7fc7136df700 1 -- 172.21.15.3:0/3142981844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc70c120c70 con 0x7fc70c09e1d0 2023-10-03T10:15:05.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.993+0000 7fc701ffb700 1 -- 172.21.15.3:0/3142981844 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc6fc013c60 con 0x7fc70c09e1d0 2023-10-03T10:15:05.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.994+0000 7fc701ffb700 1 -- 172.21.15.3:0/3142981844 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc6fc03f510 con 0x7fc70c09e1d0 2023-10-03T10:15:05.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:05.994+0000 7fc7136df700 1 -- 172.21.15.3:0/3142981844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc70c120e90 con 0x7fc70c09e1d0 2023-10-03T10:15:05.999 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:06.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.003+0000 7fc701ffb700 1 -- 172.21.15.3:0/3142981844 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc70c120e90 con 0x7fc70c09e1d0 2023-10-03T10:15:06.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.123+0000 7f8b2bf2d700 1 -- 172.21.15.3:0/174847762 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8b0c005150 con 0x7f8b2406e570 2023-10-03T10:15:06.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.124+0000 7f8b1a7fc700 1 -- 172.21.15.3:0/174847762 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8b20021020 con 0x7f8b2406e570 2023-10-03T10:15:06.124 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.126+0000 7f8b2bf2d700 1 -- 172.21.15.3:0/174847762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b2406e570 msgr2=0x7f8b241340b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.126+0000 7f8b2bf2d700 1 --2- 172.21.15.3:0/174847762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b2406e570 0x7f8b241340b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8b2000e8e0 tx=0x7f8b2000ebf0 comp rx=0 tx=0).stop 2023-10-03T10:15:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.126+0000 7f8b2bf2d700 1 -- 172.21.15.3:0/174847762 shutdown_connections 2023-10-03T10:15:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.126+0000 7f8b2bf2d700 1 --2- 172.21.15.3:0/174847762 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b241333b0 0x7f8b24133820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.126+0000 7f8b2bf2d700 1 --2- 172.21.15.3:0/174847762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b2406e570 0x7f8b241340b0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.126+0000 7f8b2bf2d700 1 --2- 172.21.15.3:0/174847762 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b24134610 0x7f8b24132e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.126+0000 7f8b2bf2d700 1 -- 172.21.15.3:0/174847762 >> 172.21.15.3:0/174847762 conn(0x7f8b240a2aa0 msgr2=0x7f8b241381f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.126+0000 7f8b2bf2d700 1 -- 172.21.15.3:0/174847762 shutdown_connections 2023-10-03T10:15:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.126+0000 7f8b2bf2d700 1 -- 172.21.15.3:0/174847762 wait complete. 2023-10-03T10:15:06.141 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:06.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.170+0000 7fd64e206700 1 -- 172.21.15.3:0/727838929 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd6480a3260 con 0x7fd648130870 2023-10-03T10:15:06.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.171+0000 7fd62ffff700 1 -- 172.21.15.3:0/727838929 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd63c021070 con 0x7fd648130870 2023-10-03T10:15:06.171 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:06.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.173+0000 7fd64e206700 1 -- 172.21.15.3:0/727838929 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd648130870 msgr2=0x7fd64809c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:06.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.173+0000 7fd64e206700 1 --2- 172.21.15.3:0/727838929 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd648130870 0x7fd64809c940 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00e8b0 tx=0x7fd63c00ebc0 comp rx=0 tx=0).stop 2023-10-03T10:15:06.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.173+0000 7fd64e206700 1 -- 172.21.15.3:0/727838929 shutdown_connections 2023-10-03T10:15:06.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.173+0000 7fd64e206700 1 --2- 172.21.15.3:0/727838929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd648133a80 0x7fd64809ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:06.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.173+0000 7fd64e206700 1 --2- 172.21.15.3:0/727838929 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd64812ff20 0x7fd64809c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:06.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.173+0000 7fd64e206700 1 --2- 172.21.15.3:0/727838929 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd648130870 0x7fd64809c940 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:06.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.173+0000 7fd64e206700 1 -- 172.21.15.3:0/727838929 >> 172.21.15.3:0/727838929 conn(0x7fd6480a4210 msgr2=0x7fd64809e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:06.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.173+0000 7fd64e206700 1 -- 172.21.15.3:0/727838929 shutdown_connections 2023-10-03T10:15:06.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.173+0000 7fd64e206700 1 -- 172.21.15.3:0/727838929 wait complete. 2023-10-03T10:15:06.187 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:06.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.216+0000 7fc7136df700 1 -- 172.21.15.3:0/3142981844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc70c120e90 con 0x7fc70c09e1d0 2023-10-03T10:15:06.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.216+0000 7fc701ffb700 1 -- 172.21.15.3:0/3142981844 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc70c120e90 con 0x7fc70c09e1d0 2023-10-03T10:15:06.217 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:06.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.219+0000 7fc7136df700 1 -- 172.21.15.3:0/3142981844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc70c09e1d0 msgr2=0x7fc70c11e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:06.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.219+0000 7fc7136df700 1 --2- 172.21.15.3:0/3142981844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc70c09e1d0 0x7fc70c11e680 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00e840 tx=0x7fc6fc00eb50 comp rx=0 tx=0).stop 2023-10-03T10:15:06.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.219+0000 7fc7136df700 1 -- 172.21.15.3:0/3142981844 shutdown_connections 2023-10-03T10:15:06.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.219+0000 7fc7136df700 1 --2- 172.21.15.3:0/3142981844 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc70c135e50 0x7fc70c1c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:06.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.219+0000 7fc7136df700 1 --2- 172.21.15.3:0/3142981844 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc70c09d6f0 0x7fc70c11e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:06.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.219+0000 7fc7136df700 1 --2- 172.21.15.3:0/3142981844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc70c09e1d0 0x7fc70c11e680 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:06.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.219+0000 7fc7136df700 1 -- 172.21.15.3:0/3142981844 >> 172.21.15.3:0/3142981844 conn(0x7fc70c066f50 msgr2=0x7fc70c132940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:06.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.219+0000 7fc7136df700 1 -- 172.21.15.3:0/3142981844 shutdown_connections 2023-10-03T10:15:06.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:06.219+0000 7fc7136df700 1 -- 172.21.15.3:0/3142981844 wait complete. 2023-10-03T10:15:06.233 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:07.000 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:07.142 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:07.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.145+0000 7fbb8494c700 1 Processor -- start 2023-10-03T10:15:07.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.145+0000 7fbb8494c700 1 -- start start 2023-10-03T10:15:07.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.145+0000 7fbb8494c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb80130b30 0x7fbb80138890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:07.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.145+0000 7fbb8494c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb80138dd0 0x7fbb8013b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:07.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.145+0000 7fbb8494c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb80099ef0 con 0x7fbb80071b40 2023-10-03T10:15:07.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.145+0000 7fbb8494c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb80099500 con 0x7fbb80130b30 2023-10-03T10:15:07.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.145+0000 7fbb8494c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb80099c10 con 0x7fbb80138dd0 2023-10-03T10:15:07.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.146+0000 7fbb7e59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbb80071b40 0x7fbb80130420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47046/0 (socket says 172.21.15.3:47046) 2023-10-03T10:15:07.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.146+0000 7fbb7dd9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb80130b30 0x7fbb80138890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:07.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.146+0000 7fbb7ed9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb80138dd0 0x7fbb8013b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:07.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.146+0000 7fbb7e59c700 1 -- 172.21.15.3:0/211590028 learned_addr learned my addr 172.21.15.3:0/211590028 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:07.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.146+0000 7fbb7d59a700 1 -- 172.21.15.3:0/211590028 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2493121283 0 0) 0x7fbb80099ef0 con 0x7fbb80071b40 2023-10-03T10:15:07.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.146+0000 7fbb7d59a700 1 -- 172.21.15.3:0/211590028 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb680035b0 con 0x7fbb80071b40 2023-10-03T10:15:07.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.146+0000 7fbb7ed9d700 1 -- 172.21.15.3:0/211590028 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb80130b30 msgr2=0x7fbb80138890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:07.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.146+0000 7fbb7ed9d700 1 --2- 172.21.15.3:0/211590028 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb80130b30 0x7fbb80138890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.146+0000 7fbb7ed9d700 1 -- 172.21.15.3:0/211590028 >> v1:172.21.15.3:6789/0 conn(0x7fbb80071b40 legacy=0x7fbb80130420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:07.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.146+0000 7fbb7ed9d700 1 -- 172.21.15.3:0/211590028 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb80130f20 con 0x7fbb80138dd0 2023-10-03T10:15:07.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.146+0000 7fbb7dd9b700 1 --2- 172.21.15.3:0/211590028 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb80130b30 0x7fbb80138890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:07.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.146+0000 7fbb7ed9d700 1 --2- 172.21.15.3:0/211590028 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb80138dd0 0x7fbb8013b1b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbb78009dc0 tx=0x7fbb7800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a8be164a0c6e79d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:07.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.146+0000 7fbb7d59a700 1 -- 172.21.15.3:0/211590028 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7800e070 con 0x7fbb80138dd0 2023-10-03T10:15:07.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.146+0000 7fbb7d59a700 1 -- 172.21.15.3:0/211590028 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7800bb20 con 0x7fbb80138dd0 2023-10-03T10:15:07.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.146+0000 7fbb7d59a700 1 -- 172.21.15.3:0/211590028 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb78012430 con 0x7fbb80138dd0 2023-10-03T10:15:07.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.147+0000 7fbb8494c700 1 -- 172.21.15.3:0/211590028 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb80138dd0 msgr2=0x7fbb8013b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:07.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.147+0000 7fbb8494c700 1 --2- 172.21.15.3:0/211590028 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb80138dd0 0x7fbb8013b1b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbb78009dc0 tx=0x7fbb7800b330 comp rx=0 tx=0).stop 2023-10-03T10:15:07.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.147+0000 7fbb8494c700 1 -- 172.21.15.3:0/211590028 shutdown_connections 2023-10-03T10:15:07.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.147+0000 7fbb8494c700 1 --2- 172.21.15.3:0/211590028 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb80138dd0 0x7fbb8013b1b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.147+0000 7fbb8494c700 1 --2- 172.21.15.3:0/211590028 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb80130b30 0x7fbb80138890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.147+0000 7fbb8494c700 1 -- 172.21.15.3:0/211590028 >> 172.21.15.3:0/211590028 conn(0x7fbb8009e7e0 msgr2=0x7fbb8009ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:07.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.147+0000 7fbb8494c700 1 -- 172.21.15.3:0/211590028 shutdown_connections 2023-10-03T10:15:07.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.147+0000 7fbb8494c700 1 -- 172.21.15.3:0/211590028 wait complete. 2023-10-03T10:15:07.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.148+0000 7fbb8494c700 1 Processor -- start 2023-10-03T10:15:07.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.148+0000 7fbb8494c700 1 -- start start 2023-10-03T10:15:07.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.149+0000 7fbb8494c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb80071b40 0x7fbb80126e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:07.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.149+0000 7fbb8494c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb80130b30 0x7fbb80127340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:07.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.149+0000 7fbb8494c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb80138dd0 0x7fbb80127880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:07.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.149+0000 7fbb8494c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb80128ee0 con 0x7fbb80138dd0 2023-10-03T10:15:07.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.149+0000 7fbb8494c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8012fcc0 con 0x7fbb80071b40 2023-10-03T10:15:07.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.149+0000 7fbb8494c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb801287e0 con 0x7fbb80130b30 2023-10-03T10:15:07.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.149+0000 7fbb7dd9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb80130b30 0x7fbb80127340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:07.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.149+0000 7fbb7e59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb80071b40 0x7fbb80126e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:07.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.149+0000 7fbb7dd9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb80130b30 0x7fbb80127340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38534/0 (socket says 172.21.15.3:38534) 2023-10-03T10:15:07.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.149+0000 7fbb7dd9b700 1 -- 172.21.15.3:0/2185466935 learned_addr learned my addr 172.21.15.3:0/2185466935 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:07.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.149+0000 7fbb7e59c700 1 -- 172.21.15.3:0/2185466935 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb80130b30 msgr2=0x7fbb80127340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:07.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.149+0000 7fbb7e59c700 1 --2- 172.21.15.3:0/2185466935 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb80130b30 0x7fbb80127340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.149+0000 7fbb7e59c700 1 -- 172.21.15.3:0/2185466935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb80138dd0 msgr2=0x7fbb80127880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:07.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.149+0000 7fbb7e59c700 1 --2- 172.21.15.3:0/2185466935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb80138dd0 0x7fbb80127880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.149+0000 7fbb7e59c700 1 -- 172.21.15.3:0/2185466935 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb740094a0 con 0x7fbb80071b40 2023-10-03T10:15:07.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.149+0000 7fbb7dd9b700 1 --2- 172.21.15.3:0/2185466935 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb80130b30 0x7fbb80127340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:07.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.150+0000 7fbb7e59c700 1 --2- 172.21.15.3:0/2185466935 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb80071b40 0x7fbb80126e00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbb7000c460 tx=0x7fbb7000c770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:07.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.150+0000 7fbb6effd700 1 -- 172.21.15.3:0/2185466935 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb70005d70 con 0x7fbb80071b40 2023-10-03T10:15:07.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.150+0000 7fbb6effd700 1 -- 172.21.15.3:0/2185466935 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb700070d0 con 0x7fbb80071b40 2023-10-03T10:15:07.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.150+0000 7fbb8494c700 1 -- 172.21.15.3:0/2185466935 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb78009a70 con 0x7fbb80071b40 2023-10-03T10:15:07.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.150+0000 7fbb6effd700 1 -- 172.21.15.3:0/2185466935 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb70007460 con 0x7fbb80071b40 2023-10-03T10:15:07.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.151+0000 7fbb8494c700 1 -- 172.21.15.3:0/2185466935 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb80128170 con 0x7fbb80071b40 2023-10-03T10:15:07.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.151+0000 7fbb6effd700 1 -- 172.21.15.3:0/2185466935 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbb700075c0 con 0x7fbb80071b40 2023-10-03T10:15:07.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.151+0000 7fbb8494c700 1 -- 172.21.15.3:0/2185466935 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb800747f0 con 0x7fbb80071b40 2023-10-03T10:15:07.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.152+0000 7fbb6effd700 1 -- 172.21.15.3:0/2185466935 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fbb7003eb20 con 0x7fbb80071b40 2023-10-03T10:15:07.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.157+0000 7fbb6effd700 1 -- 172.21.15.3:0/2185466935 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbb7001bb10 con 0x7fbb80071b40 2023-10-03T10:15:07.188 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:07.233 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:07.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.289+0000 7f04283d1700 1 Processor -- start 2023-10-03T10:15:07.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.290+0000 7f04283d1700 1 -- start start 2023-10-03T10:15:07.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.291+0000 7f04283d1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0420136830 0x7f0420138c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:07.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.291+0000 7f04283d1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0420139150 0x7f042013b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:07.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.291+0000 7f04283d1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f042006f720 con 0x7f04201338c0 2023-10-03T10:15:07.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.291+0000 7f04283d1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0420070c80 con 0x7f0420139150 2023-10-03T10:15:07.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.291+0000 7f04283d1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f042006fce0 con 0x7f0420136830 2023-10-03T10:15:07.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.291+0000 7f042596c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0420136830 0x7f0420138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:07.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.291+0000 7f042616d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f04201338c0 0x7f0420133c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47052/0 (socket says 172.21.15.3:47052) 2023-10-03T10:15:07.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.291+0000 7f042616d700 1 -- 172.21.15.3:0/2105231266 learned_addr learned my addr 172.21.15.3:0/2105231266 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:07.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.291+0000 7f042696e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0420139150 0x7f042013b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:07.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.291+0000 7f042696e700 1 -- 172.21.15.3:0/2105231266 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0420136830 msgr2=0x7f0420138c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:07.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.291+0000 7f042696e700 1 --2- 172.21.15.3:0/2105231266 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0420136830 0x7f0420138c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.291+0000 7f042696e700 1 -- 172.21.15.3:0/2105231266 >> v1:172.21.15.3:6789/0 conn(0x7f04201338c0 legacy=0x7f0420133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:07.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.291+0000 7f042696e700 1 -- 172.21.15.3:0/2105231266 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04201336a0 con 0x7f0420139150 2023-10-03T10:15:07.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.291+0000 7f042596c700 1 --2- 172.21.15.3:0/2105231266 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0420136830 0x7f0420138c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:07.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.292+0000 7f042696e700 1 --2- 172.21.15.3:0/2105231266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0420139150 0x7f042013b530 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0418009d90 tx=0x7f041800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54749194106a98b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:07.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.292+0000 7f042516b700 1 -- 172.21.15.3:0/2105231266 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041800e070 con 0x7f0420139150 2023-10-03T10:15:07.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.292+0000 7f042516b700 1 -- 172.21.15.3:0/2105231266 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041800bb20 con 0x7f0420139150 2023-10-03T10:15:07.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.292+0000 7f042516b700 1 -- 172.21.15.3:0/2105231266 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0418012430 con 0x7f0420139150 2023-10-03T10:15:07.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.293+0000 7f04283d1700 1 -- 172.21.15.3:0/2105231266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0420139150 msgr2=0x7f042013b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:07.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.293+0000 7f04283d1700 1 --2- 172.21.15.3:0/2105231266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0420139150 0x7f042013b530 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0418009d90 tx=0x7f041800b330 comp rx=0 tx=0).stop 2023-10-03T10:15:07.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.293+0000 7f04283d1700 1 -- 172.21.15.3:0/2105231266 shutdown_connections 2023-10-03T10:15:07.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.293+0000 7f04283d1700 1 --2- 172.21.15.3:0/2105231266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0420139150 0x7f042013b530 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.293+0000 7f04283d1700 1 --2- 172.21.15.3:0/2105231266 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0420136830 0x7f0420138c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.293+0000 7f04283d1700 1 -- 172.21.15.3:0/2105231266 >> 172.21.15.3:0/2105231266 conn(0x7f04200a2aa0 msgr2=0x7f04200a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:07.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.293+0000 7f04283d1700 1 -- 172.21.15.3:0/2105231266 shutdown_connections 2023-10-03T10:15:07.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.293+0000 7f04283d1700 1 -- 172.21.15.3:0/2105231266 wait complete. 2023-10-03T10:15:07.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.294+0000 7f04283d1700 1 Processor -- start 2023-10-03T10:15:07.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.294+0000 7f04283d1700 1 -- start start 2023-10-03T10:15:07.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.294+0000 7f04283d1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f04201338c0 0x7f042012dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:07.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.294+0000 7f04283d1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0420136830 0x7f042012e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:07.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.294+0000 7f04283d1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0420139150 0x7f0420118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:07.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.294+0000 7f04283d1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042011aed0 con 0x7f0420136830 2023-10-03T10:15:07.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.295+0000 7f04283d1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0420136070 con 0x7f0420139150 2023-10-03T10:15:07.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.295+0000 7f04283d1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f042011a9c0 con 0x7f04201338c0 2023-10-03T10:15:07.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.295+0000 7f042596c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0420136830 0x7f042012e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:07.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.295+0000 7f042596c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0420136830 0x7f042012e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57186/0 (socket says 172.21.15.3:57186) 2023-10-03T10:15:07.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.295+0000 7f042596c700 1 -- 172.21.15.3:0/1089538218 learned_addr learned my addr 172.21.15.3:0/1089538218 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:07.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.295+0000 7f042696e700 1 --2- 172.21.15.3:0/1089538218 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0420139150 0x7f0420118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:07.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.295+0000 7f042616d700 1 --2- 172.21.15.3:0/1089538218 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f04201338c0 0x7f042012dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:07.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.295+0000 7f042596c700 1 -- 172.21.15.3:0/1089538218 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f04201338c0 msgr2=0x7f042012dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:07.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.295+0000 7f042596c700 1 --2- 172.21.15.3:0/1089538218 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f04201338c0 0x7f042012dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.295+0000 7f042596c700 1 -- 172.21.15.3:0/1089538218 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0420139150 msgr2=0x7f0420118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:07.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.295+0000 7f042596c700 1 --2- 172.21.15.3:0/1089538218 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0420139150 0x7f0420118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.295+0000 7f042596c700 1 -- 172.21.15.3:0/1089538218 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04100094a0 con 0x7f0420136830 2023-10-03T10:15:07.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.295+0000 7f042616d700 1 --2- 172.21.15.3:0/1089538218 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f04201338c0 0x7f042012dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:07.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.295+0000 7f042696e700 1 --2- 172.21.15.3:0/1089538218 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0420139150 0x7f0420118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:07.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.296+0000 7f042596c700 1 --2- 172.21.15.3:0/1089538218 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0420136830 0x7f042012e1d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f041000ebe0 tx=0x7f041000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:07.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.296+0000 7f0416ffd700 1 -- 172.21.15.3:0/1089538218 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04100097a0 con 0x7f0420136830 2023-10-03T10:15:07.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.296+0000 7f0416ffd700 1 -- 172.21.15.3:0/1089538218 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041000c850 con 0x7f0420136830 2023-10-03T10:15:07.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.296+0000 7f04283d1700 1 -- 172.21.15.3:0/1089538218 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0418009a70 con 0x7f0420136830 2023-10-03T10:15:07.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.296+0000 7f0416ffd700 1 -- 172.21.15.3:0/1089538218 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041000cbe0 con 0x7f0420136830 2023-10-03T10:15:07.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.296+0000 7f04283d1700 1 -- 172.21.15.3:0/1089538218 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f042011b110 con 0x7f0420136830 2023-10-03T10:15:07.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.297+0000 7f0416ffd700 1 -- 172.21.15.3:0/1089538218 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f041000cd40 con 0x7f0420136830 2023-10-03T10:15:07.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.297+0000 7f04283d1700 1 -- 172.21.15.3:0/1089538218 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0420074ea0 con 0x7f0420136830 2023-10-03T10:15:07.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.297+0000 7f0416ffd700 1 -- 172.21.15.3:0/1089538218 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f041003f440 con 0x7f0420136830 2023-10-03T10:15:07.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.307+0000 7f0416ffd700 1 -- 172.21.15.3:0/1089538218 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0410017080 con 0x7f0420136830 2023-10-03T10:15:07.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.342+0000 7f43b5954700 1 Processor -- start 2023-10-03T10:15:07.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.342+0000 7f43b5954700 1 -- start start 2023-10-03T10:15:07.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.343+0000 7f43b5954700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f43b009d6f0 0x7f43b009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:07.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.343+0000 7f43b5954700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f43b009e410 0x7f43b0130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:07.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.343+0000 7f43b5954700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b009f130 con 0x7f43b009e000 2023-10-03T10:15:07.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.343+0000 7f43b5954700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b009f520 con 0x7f43b009e410 2023-10-03T10:15:07.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.343+0000 7f43b5954700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b009e930 con 0x7f43b009d6f0 2023-10-03T10:15:07.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.343+0000 7f43aeffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f43b009d6f0 0x7f43b009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:07.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.343+0000 7f43af7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f43b009e410 0x7f43b0130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:07.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.343+0000 7f43a7fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f43b009e000 0x7f43b0130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47058/0 (socket says 172.21.15.3:47058) 2023-10-03T10:15:07.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.343+0000 7f43a7fff700 1 -- 172.21.15.3:0/42239837 learned_addr learned my addr 172.21.15.3:0/42239837 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:07.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.344+0000 7f43af7fe700 1 -- 172.21.15.3:0/42239837 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f43b009d6f0 msgr2=0x7f43b009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:07.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.344+0000 7f43af7fe700 1 --2- 172.21.15.3:0/42239837 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f43b009d6f0 0x7f43b009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.344+0000 7f43af7fe700 1 -- 172.21.15.3:0/42239837 >> v1:172.21.15.3:6789/0 conn(0x7f43b009e000 legacy=0x7f43b0130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:07.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.344+0000 7f43af7fe700 1 -- 172.21.15.3:0/42239837 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b009b2d0 con 0x7f43b009e410 2023-10-03T10:15:07.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.344+0000 7f43af7fe700 1 --2- 172.21.15.3:0/42239837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43b009e410 0x7f43b0130d70 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f43a8009d90 tx=0x7f43a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2871fa6c04185401 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:07.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.344+0000 7f43ae7fc700 1 -- 172.21.15.3:0/42239837 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a800e070 con 0x7f43b009e410 2023-10-03T10:15:07.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.344+0000 7f43ae7fc700 1 -- 172.21.15.3:0/42239837 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a800bb20 con 0x7f43b009e410 2023-10-03T10:15:07.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.344+0000 7f43ae7fc700 1 -- 172.21.15.3:0/42239837 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a8012430 con 0x7f43b009e410 2023-10-03T10:15:07.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.345+0000 7f43b5954700 1 -- 172.21.15.3:0/42239837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43b009e410 msgr2=0x7f43b0130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:07.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.345+0000 7f43b5954700 1 --2- 172.21.15.3:0/42239837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43b009e410 0x7f43b0130d70 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f43a8009d90 tx=0x7f43a800b330 comp rx=0 tx=0).stop 2023-10-03T10:15:07.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.345+0000 7f43b5954700 1 -- 172.21.15.3:0/42239837 shutdown_connections 2023-10-03T10:15:07.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.345+0000 7f43b5954700 1 --2- 172.21.15.3:0/42239837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43b009e410 0x7f43b0130d70 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.345+0000 7f43b5954700 1 --2- 172.21.15.3:0/42239837 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f43b009d6f0 0x7f43b009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.345+0000 7f43b5954700 1 -- 172.21.15.3:0/42239837 >> 172.21.15.3:0/42239837 conn(0x7f43b0066f50 msgr2=0x7f43b009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:07.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.346+0000 7f43b5954700 1 -- 172.21.15.3:0/42239837 shutdown_connections 2023-10-03T10:15:07.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.346+0000 7f43b5954700 1 -- 172.21.15.3:0/42239837 wait complete. 2023-10-03T10:15:07.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.347+0000 7f43b5954700 1 Processor -- start 2023-10-03T10:15:07.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.347+0000 7f43b5954700 1 -- start start 2023-10-03T10:15:07.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.347+0000 7f43b5954700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f43b009d6f0 0x7f43b012f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:07.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.347+0000 7f43b5954700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43b009e000 0x7f43b012dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:07.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.347+0000 7f43b5954700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f43b009e410 0x7f43b012e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:07.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.347+0000 7f43b5954700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b01318a0 con 0x7f43b009e000 2023-10-03T10:15:07.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.347+0000 7f43b5954700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b009c700 con 0x7f43b009e410 2023-10-03T10:15:07.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.347+0000 7f43b5954700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f43b0131ab0 con 0x7f43b009d6f0 2023-10-03T10:15:07.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.347+0000 7f43af7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f43b009e410 0x7f43b012e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:07.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.347+0000 7f43af7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f43b009e410 0x7f43b012e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51312/0 (socket says 172.21.15.3:51312) 2023-10-03T10:15:07.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.347+0000 7f43af7fe700 1 -- 172.21.15.3:0/289137301 learned_addr learned my addr 172.21.15.3:0/289137301 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:07.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.348+0000 7f43a7fff700 1 --2- 172.21.15.3:0/289137301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43b009e000 0x7f43b012dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:07.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.348+0000 7f43aeffd700 1 --2- 172.21.15.3:0/289137301 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f43b009d6f0 0x7f43b012f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:07.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.348+0000 7f43af7fe700 1 -- 172.21.15.3:0/289137301 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f43b009d6f0 msgr2=0x7f43b012f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:07.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.348+0000 7f43af7fe700 1 --2- 172.21.15.3:0/289137301 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f43b009d6f0 0x7f43b012f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.348+0000 7f43af7fe700 1 -- 172.21.15.3:0/289137301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43b009e000 msgr2=0x7f43b012dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:07.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.348+0000 7f43af7fe700 1 --2- 172.21.15.3:0/289137301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43b009e000 0x7f43b012dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.348+0000 7f43af7fe700 1 -- 172.21.15.3:0/289137301 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a8009a70 con 0x7f43b009e410 2023-10-03T10:15:07.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.348+0000 7f43aeffd700 1 --2- 172.21.15.3:0/289137301 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f43b009d6f0 0x7f43b012f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:15:07.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.348+0000 7f43af7fe700 1 --2- 172.21.15.3:0/289137301 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f43b009e410 0x7f43b012e250 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f43a8014040 tx=0x7f43a8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:07.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.348+0000 7f43acff9700 1 -- 172.21.15.3:0/289137301 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a8012430 con 0x7f43b009e410 2023-10-03T10:15:07.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.348+0000 7f43acff9700 1 -- 172.21.15.3:0/289137301 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a8020070 con 0x7f43b009e410 2023-10-03T10:15:07.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.348+0000 7f43acff9700 1 -- 172.21.15.3:0/289137301 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a8003ea0 con 0x7f43b009e410 2023-10-03T10:15:07.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.349+0000 7f43b5954700 1 -- 172.21.15.3:0/289137301 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b00725c0 con 0x7f43b009e410 2023-10-03T10:15:07.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.349+0000 7f43b5954700 1 -- 172.21.15.3:0/289137301 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b0130e80 con 0x7f43b009e410 2023-10-03T10:15:07.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.350+0000 7f43acff9700 1 -- 172.21.15.3:0/289137301 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f43a8004280 con 0x7f43b009e410 2023-10-03T10:15:07.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.350+0000 7f43acff9700 1 -- 172.21.15.3:0/289137301 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f43a8022070 con 0x7f43b009e410 2023-10-03T10:15:07.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.350+0000 7f43b5954700 1 -- 172.21.15.3:0/289137301 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b0130a30 con 0x7f43b009e410 2023-10-03T10:15:07.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.355+0000 7f43acff9700 1 -- 172.21.15.3:0/289137301 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f43a802c020 con 0x7f43b009e410 2023-10-03T10:15:07.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.384+0000 7fbb8494c700 1 -- 172.21.15.3:0/2185466935 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbb8006b250 con 0x7fbb80071b40 2023-10-03T10:15:07.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.384+0000 7fbb6effd700 1 -- 172.21.15.3:0/2185466935 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbb700192c0 con 0x7fbb80071b40 2023-10-03T10:15:07.385 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:07.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.387+0000 7fbb8494c700 1 -- 172.21.15.3:0/2185466935 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb80071b40 msgr2=0x7fbb80126e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:07.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.387+0000 7fbb8494c700 1 --2- 172.21.15.3:0/2185466935 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb80071b40 0x7fbb80126e00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbb7000c460 tx=0x7fbb7000c770 comp rx=0 tx=0).stop 2023-10-03T10:15:07.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.388+0000 7fbb8494c700 1 -- 172.21.15.3:0/2185466935 shutdown_connections 2023-10-03T10:15:07.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.388+0000 7fbb8494c700 1 --2- 172.21.15.3:0/2185466935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb80138dd0 0x7fbb80127880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.388+0000 7fbb8494c700 1 --2- 172.21.15.3:0/2185466935 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb80071b40 0x7fbb80126e00 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.388+0000 7fbb8494c700 1 --2- 172.21.15.3:0/2185466935 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb80130b30 0x7fbb80127340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.388+0000 7fbb8494c700 1 -- 172.21.15.3:0/2185466935 >> 172.21.15.3:0/2185466935 conn(0x7fbb8009e7e0 msgr2=0x7fbb8013a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:07.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.390+0000 7fbb8494c700 1 -- 172.21.15.3:0/2185466935 shutdown_connections 2023-10-03T10:15:07.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.390+0000 7fbb8494c700 1 -- 172.21.15.3:0/2185466935 wait complete. 2023-10-03T10:15:07.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.395+0000 7fd05a2e9700 1 Processor -- start 2023-10-03T10:15:07.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.395+0000 7fd05a2e9700 1 -- start start 2023-10-03T10:15:07.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.396+0000 7fd05a2e9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd05409e1d0 0x7fd054138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:07.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.396+0000 7fd05a2e9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd054139170 0x7fd05413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:07.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.396+0000 7fd05a2e9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd05409b120 con 0x7fd05409d6f0 2023-10-03T10:15:07.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.396+0000 7fd05a2e9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd05406b460 con 0x7fd05409e1d0 2023-10-03T10:15:07.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.396+0000 7fd05a2e9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd05409a7e0 con 0x7fd054139170 2023-10-03T10:15:07.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.396+0000 7fd053fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd05409d6f0 0x7fd05409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47060/0 (socket says 172.21.15.3:47060) 2023-10-03T10:15:07.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.396+0000 7fd053fff700 1 -- 172.21.15.3:0/1831895091 learned_addr learned my addr 172.21.15.3:0/1831895091 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:07.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.396+0000 7fd0537fe700 1 --2- 172.21.15.3:0/1831895091 >> v2:172.21.15.3:3300/0 conn(0x7fd05409e1d0 0x7fd054138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:07.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.396+0000 7fd058886700 1 --2- 172.21.15.3:0/1831895091 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd054139170 0x7fd05413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:07.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.397+0000 7fd0537fe700 1 -- 172.21.15.3:0/1831895091 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd054139170 msgr2=0x7fd05413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:07.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.397+0000 7fd0537fe700 1 --2- 172.21.15.3:0/1831895091 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd054139170 0x7fd05413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.397+0000 7fd0537fe700 1 -- 172.21.15.3:0/1831895091 >> v1:172.21.15.3:6789/0 conn(0x7fd05409d6f0 legacy=0x7fd05409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:07.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.397+0000 7fd0537fe700 1 -- 172.21.15.3:0/1831895091 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd05409a500 con 0x7fd05409e1d0 2023-10-03T10:15:07.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.397+0000 7fd0537fe700 1 --2- 172.21.15.3:0/1831895091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd05409e1d0 0x7fd054138c30 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd048009870 tx=0x7fd048009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f6bfb01d6cde3e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:07.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.397+0000 7fd052ffd700 1 -- 172.21.15.3:0/1831895091 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd048011830 con 0x7fd05409e1d0 2023-10-03T10:15:07.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.397+0000 7fd052ffd700 1 -- 172.21.15.3:0/1831895091 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd048011990 con 0x7fd05409e1d0 2023-10-03T10:15:07.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.397+0000 7fd052ffd700 1 -- 172.21.15.3:0/1831895091 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd048011d20 con 0x7fd05409e1d0 2023-10-03T10:15:07.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.398+0000 7fd05a2e9700 1 -- 172.21.15.3:0/1831895091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd05409e1d0 msgr2=0x7fd054138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:07.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.398+0000 7fd05a2e9700 1 --2- 172.21.15.3:0/1831895091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd05409e1d0 0x7fd054138c30 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd048009870 tx=0x7fd048009b80 comp rx=0 tx=0).stop 2023-10-03T10:15:07.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.398+0000 7fd05a2e9700 1 -- 172.21.15.3:0/1831895091 shutdown_connections 2023-10-03T10:15:07.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.398+0000 7fd05a2e9700 1 --2- 172.21.15.3:0/1831895091 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd054139170 0x7fd05413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.398+0000 7fd05a2e9700 1 --2- 172.21.15.3:0/1831895091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd05409e1d0 0x7fd054138c30 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.398+0000 7fd05a2e9700 1 -- 172.21.15.3:0/1831895091 >> 172.21.15.3:0/1831895091 conn(0x7fd054066f50 msgr2=0x7fd05409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:07.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.398+0000 7fd05a2e9700 1 -- 172.21.15.3:0/1831895091 shutdown_connections 2023-10-03T10:15:07.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.398+0000 7fd05a2e9700 1 -- 172.21.15.3:0/1831895091 wait complete. 2023-10-03T10:15:07.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.399+0000 7fd05a2e9700 1 Processor -- start 2023-10-03T10:15:07.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.400+0000 7fd05a2e9700 1 -- start start 2023-10-03T10:15:07.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.400+0000 7fd05a2e9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd05409d6f0 0x7fd0541c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:07.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.400+0000 7fd05a2e9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd05409e1d0 0x7fd0541c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:07.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.400+0000 7fd05a2e9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd054139170 0x7fd05411e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:07.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.400+0000 7fd05a2e9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0540a1a50 con 0x7fd05409d6f0 2023-10-03T10:15:07.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.400+0000 7fd05a2e9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd054121d20 con 0x7fd05409e1d0 2023-10-03T10:15:07.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.400+0000 7fd05a2e9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd05409c710 con 0x7fd054139170 2023-10-03T10:15:07.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.400+0000 7fd0537fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd05409e1d0 0x7fd0541c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:07.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.400+0000 7fd053fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd05409d6f0 0x7fd0541c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:07.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.400+0000 7fd0537fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd05409e1d0 0x7fd0541c8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51326/0 (socket says 172.21.15.3:51326) 2023-10-03T10:15:07.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.400+0000 7fd053fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd05409d6f0 0x7fd0541c84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57228/0 (socket says 172.21.15.3:57228) 2023-10-03T10:15:07.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.400+0000 7fd0537fe700 1 -- 172.21.15.3:0/1426747164 learned_addr learned my addr 172.21.15.3:0/1426747164 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:07.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.400+0000 7fd058886700 1 --2- 172.21.15.3:0/1426747164 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd054139170 0x7fd05411e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:07.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.401+0000 7fd053fff700 1 -- 172.21.15.3:0/1426747164 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd054139170 msgr2=0x7fd05411e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:07.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.401+0000 7fd053fff700 1 --2- 172.21.15.3:0/1426747164 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd054139170 0x7fd05411e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.401+0000 7fd053fff700 1 -- 172.21.15.3:0/1426747164 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd05409e1d0 msgr2=0x7fd0541c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:07.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.401+0000 7fd053fff700 1 --2- 172.21.15.3:0/1426747164 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd05409e1d0 0x7fd0541c8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.401+0000 7fd053fff700 1 -- 172.21.15.3:0/1426747164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0480094a0 con 0x7fd05409d6f0 2023-10-03T10:15:07.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.401+0000 7fd0537fe700 1 --2- 172.21.15.3:0/1426747164 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd05409e1d0 0x7fd0541c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:07.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.401+0000 7fd053fff700 1 --2- 172.21.15.3:0/1426747164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd05409d6f0 0x7fd0541c84f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd04400c530 tx=0x7fd04400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:07.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.401+0000 7fd050ff9700 1 -- 172.21.15.3:0/1426747164 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04400cea0 con 0x7fd05409d6f0 2023-10-03T10:15:07.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.401+0000 7fd050ff9700 1 -- 172.21.15.3:0/1426747164 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd044005d70 con 0x7fd05409d6f0 2023-10-03T10:15:07.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.401+0000 7fd05a2e9700 1 -- 172.21.15.3:0/1426747164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd054134670 con 0x7fd05409d6f0 2023-10-03T10:15:07.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.401+0000 7fd050ff9700 1 -- 172.21.15.3:0/1426747164 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0440072d0 con 0x7fd05409d6f0 2023-10-03T10:15:07.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.402+0000 7fd05a2e9700 1 -- 172.21.15.3:0/1426747164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd054122a50 con 0x7fd05409d6f0 2023-10-03T10:15:07.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.402+0000 7fd050ff9700 1 -- 172.21.15.3:0/1426747164 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd044007430 con 0x7fd05409d6f0 2023-10-03T10:15:07.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.403+0000 7fd050ff9700 1 -- 172.21.15.3:0/1426747164 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd04403eab0 con 0x7fd05409d6f0 2023-10-03T10:15:07.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.403+0000 7fd05a2e9700 1 -- 172.21.15.3:0/1426747164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd054074ea0 con 0x7fd05409d6f0 2023-10-03T10:15:07.408 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:07.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.408+0000 7fd050ff9700 1 -- 172.21.15.3:0/1426747164 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd044017070 con 0x7fd05409d6f0 2023-10-03T10:15:07.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.523+0000 7f04283d1700 1 -- 172.21.15.3:0/1089538218 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f042011c1f0 con 0x7f0420136830 2023-10-03T10:15:07.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.523+0000 7f0416ffd700 1 -- 172.21.15.3:0/1089538218 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f04100102b0 con 0x7f0420136830 2023-10-03T10:15:07.524 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:07.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.526+0000 7f04283d1700 1 -- 172.21.15.3:0/1089538218 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0420136830 msgr2=0x7f042012e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:07.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.526+0000 7f04283d1700 1 --2- 172.21.15.3:0/1089538218 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0420136830 0x7f042012e1d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f041000ebe0 tx=0x7f041000eef0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.526+0000 7f04283d1700 1 -- 172.21.15.3:0/1089538218 shutdown_connections 2023-10-03T10:15:07.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.526+0000 7f04283d1700 1 --2- 172.21.15.3:0/1089538218 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0420139150 0x7f0420118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.526+0000 7f04283d1700 1 --2- 172.21.15.3:0/1089538218 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f04201338c0 0x7f042012dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.526+0000 7f04283d1700 1 --2- 172.21.15.3:0/1089538218 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0420136830 0x7f042012e1d0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.526+0000 7f04283d1700 1 -- 172.21.15.3:0/1089538218 >> 172.21.15.3:0/1089538218 conn(0x7f04200a2aa0 msgr2=0x7f042012f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:07.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.526+0000 7f04283d1700 1 -- 172.21.15.3:0/1089538218 shutdown_connections 2023-10-03T10:15:07.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.526+0000 7f04283d1700 1 -- 172.21.15.3:0/1089538218 wait complete. 2023-10-03T10:15:07.540 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:07.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.580+0000 7f43b5954700 1 -- 172.21.15.3:0/289137301 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f43b0074ea0 con 0x7f43b009e410 2023-10-03T10:15:07.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.580+0000 7f43acff9700 1 -- 172.21.15.3:0/289137301 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f43a801c400 con 0x7f43b009e410 2023-10-03T10:15:07.583 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:07.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.583+0000 7f43b5954700 1 -- 172.21.15.3:0/289137301 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f43b009e410 msgr2=0x7f43b012e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:07.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.583+0000 7f43b5954700 1 --2- 172.21.15.3:0/289137301 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f43b009e410 0x7f43b012e250 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f43a8014040 tx=0x7f43a8004ad0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.583+0000 7f43b5954700 1 -- 172.21.15.3:0/289137301 shutdown_connections 2023-10-03T10:15:07.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.583+0000 7f43b5954700 1 --2- 172.21.15.3:0/289137301 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f43b009e410 0x7f43b012e250 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.583+0000 7f43b5954700 1 --2- 172.21.15.3:0/289137301 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f43b009d6f0 0x7f43b012f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.583+0000 7f43b5954700 1 --2- 172.21.15.3:0/289137301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43b009e000 0x7f43b012dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.583+0000 7f43b5954700 1 -- 172.21.15.3:0/289137301 >> 172.21.15.3:0/289137301 conn(0x7f43b0066f50 msgr2=0x7f43b00a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:07.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.583+0000 7f43b5954700 1 -- 172.21.15.3:0/289137301 shutdown_connections 2023-10-03T10:15:07.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.583+0000 7f43b5954700 1 -- 172.21.15.3:0/289137301 wait complete. 2023-10-03T10:15:07.597 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:07.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.616+0000 7fd05a2e9700 1 -- 172.21.15.3:0/1426747164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd05409f420 con 0x7fd05409d6f0 2023-10-03T10:15:07.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.616+0000 7fd050ff9700 1 -- 172.21.15.3:0/1426747164 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd044017030 con 0x7fd05409d6f0 2023-10-03T10:15:07.616 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:07.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.618+0000 7fd05a2e9700 1 -- 172.21.15.3:0/1426747164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd05409d6f0 msgr2=0x7fd0541c84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:07.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.618+0000 7fd05a2e9700 1 --2- 172.21.15.3:0/1426747164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd05409d6f0 0x7fd0541c84f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd04400c530 tx=0x7fd04400c840 comp rx=0 tx=0).stop 2023-10-03T10:15:07.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.618+0000 7fd05a2e9700 1 -- 172.21.15.3:0/1426747164 shutdown_connections 2023-10-03T10:15:07.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.618+0000 7fd05a2e9700 1 --2- 172.21.15.3:0/1426747164 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd054139170 0x7fd05411e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.618+0000 7fd05a2e9700 1 --2- 172.21.15.3:0/1426747164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd05409d6f0 0x7fd0541c84f0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.618+0000 7fd05a2e9700 1 --2- 172.21.15.3:0/1426747164 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd05409e1d0 0x7fd0541c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:07.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.618+0000 7fd05a2e9700 1 -- 172.21.15.3:0/1426747164 >> 172.21.15.3:0/1426747164 conn(0x7fd054066f50 msgr2=0x7fd05409b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:07.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.618+0000 7fd05a2e9700 1 -- 172.21.15.3:0/1426747164 shutdown_connections 2023-10-03T10:15:07.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:07.618+0000 7fd05a2e9700 1 -- 172.21.15.3:0/1426747164 wait complete. 2023-10-03T10:15:07.632 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:08.410 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:08.541 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:08.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.555+0000 7f597c78a700 1 Processor -- start 2023-10-03T10:15:08.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.555+0000 7f597c78a700 1 -- start start 2023-10-03T10:15:08.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.556+0000 7f597c78a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f59740a0a30 0x7f59740a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:08.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.556+0000 7f597c78a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f597409d630 0x7f59740a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:08.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.556+0000 7f597c78a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5974133930 0x7f5974133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:08.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.556+0000 7f597c78a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59740a2a70 con 0x7f59740a0a30 2023-10-03T10:15:08.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.556+0000 7f597c78a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59740a01d0 con 0x7f597409d630 2023-10-03T10:15:08.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.556+0000 7f597c78a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f597409e2b0 con 0x7f5974133930 2023-10-03T10:15:08.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.556+0000 7f5979d25700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f597409d630 0x7f59740a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:08.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.556+0000 7f597ad27700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5974133930 0x7f5974133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:08.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.556+0000 7f597a526700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f59740a0a30 0x7f59740a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:08.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.556+0000 7f5979d25700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f597409d630 0x7f59740a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51332/0 (socket says 172.21.15.3:51332) 2023-10-03T10:15:08.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.556+0000 7f597ad27700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5974133930 0x7f5974133db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38584/0 (socket says 172.21.15.3:38584) 2023-10-03T10:15:08.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.556+0000 7f597a526700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f59740a0a30 0x7f59740a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57244/0 (socket says 172.21.15.3:57244) 2023-10-03T10:15:08.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.556+0000 7f5979d25700 1 -- 172.21.15.3:0/82450258 learned_addr learned my addr 172.21.15.3:0/82450258 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:08.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.556+0000 7f5979d25700 1 -- 172.21.15.3:0/82450258 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5974133930 msgr2=0x7f5974133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:08.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.556+0000 7f5979d25700 1 --2- 172.21.15.3:0/82450258 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5974133930 0x7f5974133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.556+0000 7f5979d25700 1 -- 172.21.15.3:0/82450258 >> v2:172.21.15.3:3300/0 conn(0x7f59740a0a30 msgr2=0x7f59740a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:08.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.556+0000 7f5979d25700 1 --2- 172.21.15.3:0/82450258 >> v2:172.21.15.3:3300/0 conn(0x7f59740a0a30 0x7f59740a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.556+0000 7f5979d25700 1 -- 172.21.15.3:0/82450258 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597409dc50 con 0x7f597409d630 2023-10-03T10:15:08.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.556+0000 7f597a526700 1 --2- 172.21.15.3:0/82450258 >> v2:172.21.15.3:3300/0 conn(0x7f59740a0a30 0x7f59740a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:08.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.557+0000 7f5979d25700 1 --2- 172.21.15.3:0/82450258 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f597409d630 0x7f59740a1340 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f59680098f0 tx=0x7f5968009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffe44f19cc7d0422 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:08.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.557+0000 7f5979524700 1 -- 172.21.15.3:0/82450258 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5968011830 con 0x7f597409d630 2023-10-03T10:15:08.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.557+0000 7f5979524700 1 -- 172.21.15.3:0/82450258 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5968011990 con 0x7f597409d630 2023-10-03T10:15:08.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.557+0000 7f5979524700 1 -- 172.21.15.3:0/82450258 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5968011d20 con 0x7f597409d630 2023-10-03T10:15:08.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.557+0000 7f597c78a700 1 -- 172.21.15.3:0/82450258 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f597409d630 msgr2=0x7f59740a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:08.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.557+0000 7f597c78a700 1 --2- 172.21.15.3:0/82450258 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f597409d630 0x7f59740a1340 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f59680098f0 tx=0x7f5968009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:08.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.557+0000 7f597c78a700 1 -- 172.21.15.3:0/82450258 shutdown_connections 2023-10-03T10:15:08.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.557+0000 7f597c78a700 1 --2- 172.21.15.3:0/82450258 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5974133930 0x7f5974133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.557+0000 7f597c78a700 1 --2- 172.21.15.3:0/82450258 >> v2:172.21.15.3:3300/0 conn(0x7f59740a0a30 0x7f59740a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.558+0000 7f597c78a700 1 --2- 172.21.15.3:0/82450258 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f597409d630 0x7f59740a1340 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.558+0000 7f597c78a700 1 -- 172.21.15.3:0/82450258 >> 172.21.15.3:0/82450258 conn(0x7f59740a3050 msgr2=0x7f59740a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:08.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.558+0000 7f597c78a700 1 -- 172.21.15.3:0/82450258 shutdown_connections 2023-10-03T10:15:08.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.558+0000 7f597c78a700 1 -- 172.21.15.3:0/82450258 wait complete. 2023-10-03T10:15:08.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.559+0000 7f597c78a700 1 Processor -- start 2023-10-03T10:15:08.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.559+0000 7f597c78a700 1 -- start start 2023-10-03T10:15:08.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.559+0000 7f597c78a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f597409d630 0x7f597411f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:08.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.559+0000 7f597c78a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59740a0a30 0x7f597411fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:08.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.559+0000 7f597c78a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5974133930 0x7f59741c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:08.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.559+0000 7f597c78a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5974121fa0 con 0x7f59740a0a30 2023-10-03T10:15:08.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.559+0000 7f597c78a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5974122640 con 0x7f5974133930 2023-10-03T10:15:08.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.559+0000 7f597c78a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f597411ef70 con 0x7f597409d630 2023-10-03T10:15:08.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.559+0000 7f597a526700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f597409d630 0x7f597411f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:08.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.559+0000 7f597ad27700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5974133930 0x7f59741c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:08.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.559+0000 7f597a526700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f597409d630 0x7f597411f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38598/0 (socket says 172.21.15.3:38598) 2023-10-03T10:15:08.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.559+0000 7f597a526700 1 -- 172.21.15.3:0/440449756 learned_addr learned my addr 172.21.15.3:0/440449756 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:08.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.559+0000 7f5979d25700 1 --2- 172.21.15.3:0/440449756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59740a0a30 0x7f597411fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:08.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.560+0000 7f5979d25700 1 -- 172.21.15.3:0/440449756 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f597409d630 msgr2=0x7f597411f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:08.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.560+0000 7f5979d25700 1 --2- 172.21.15.3:0/440449756 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f597409d630 0x7f597411f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.560+0000 7f5979d25700 1 -- 172.21.15.3:0/440449756 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5974133930 msgr2=0x7f59741c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:08.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.560+0000 7f5979d25700 1 --2- 172.21.15.3:0/440449756 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5974133930 0x7f59741c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.560+0000 7f5979d25700 1 -- 172.21.15.3:0/440449756 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5964009a70 con 0x7f59740a0a30 2023-10-03T10:15:08.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.560+0000 7f597a526700 1 --2- 172.21.15.3:0/440449756 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f597409d630 0x7f597411f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:08.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.560+0000 7f597ad27700 1 --2- 172.21.15.3:0/440449756 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5974133930 0x7f59741c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:08.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.560+0000 7f5979d25700 1 --2- 172.21.15.3:0/440449756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59740a0a30 0x7f597411fa50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5968016000 tx=0x7f5968004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:08.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.560+0000 7f5972ffd700 1 -- 172.21.15.3:0/440449756 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5968004450 con 0x7f59740a0a30 2023-10-03T10:15:08.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.560+0000 7f5972ffd700 1 -- 172.21.15.3:0/440449756 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59680045b0 con 0x7f59740a0a30 2023-10-03T10:15:08.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.560+0000 7f597c78a700 1 -- 172.21.15.3:0/440449756 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5968009550 con 0x7f59740a0a30 2023-10-03T10:15:08.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.560+0000 7f5972ffd700 1 -- 172.21.15.3:0/440449756 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59680038c0 con 0x7f59740a0a30 2023-10-03T10:15:08.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.560+0000 7f597c78a700 1 -- 172.21.15.3:0/440449756 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59740a0410 con 0x7f59740a0a30 2023-10-03T10:15:08.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.561+0000 7f5972ffd700 1 -- 172.21.15.3:0/440449756 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5968021070 con 0x7f59740a0a30 2023-10-03T10:15:08.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.561+0000 7f5972ffd700 1 -- 172.21.15.3:0/440449756 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f59680408d0 con 0x7f59740a0a30 2023-10-03T10:15:08.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.561+0000 7f597c78a700 1 -- 172.21.15.3:0/440449756 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5974120690 con 0x7f59740a0a30 2023-10-03T10:15:08.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.566+0000 7f5972ffd700 1 -- 172.21.15.3:0/440449756 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5968015070 con 0x7f59740a0a30 2023-10-03T10:15:08.598 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:08.633 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:08.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.683+0000 7f432a960700 1 Processor -- start 2023-10-03T10:15:08.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.684+0000 7f432a960700 1 -- start start 2023-10-03T10:15:08.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.684+0000 7f432a960700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f432406a770 0x7f432409d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:08.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.684+0000 7f432a960700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f432409dc30 0x7f432409e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:08.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.684+0000 7f432a960700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f432409a810 con 0x7f432413c1e0 2023-10-03T10:15:08.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.684+0000 7f432a960700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f432406ab40 con 0x7f432406a770 2023-10-03T10:15:08.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.684+0000 7f432a960700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f432409ebc0 con 0x7f432409dc30 2023-10-03T10:15:08.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.685+0000 7f43237fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f432409dc30 0x7f432409e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:08.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.685+0000 7f4328efd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f432413c1e0 0x7f432413e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47072/0 (socket says 172.21.15.3:47072) 2023-10-03T10:15:08.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.685+0000 7f4328efd700 1 -- 172.21.15.3:0/2010619655 learned_addr learned my addr 172.21.15.3:0/2010619655 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:08.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.685+0000 7f4323fff700 1 --2- 172.21.15.3:0/2010619655 >> v2:172.21.15.3:3300/0 conn(0x7f432406a770 0x7f432409d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:08.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.685+0000 7f43237fe700 1 -- 172.21.15.3:0/2010619655 >> v2:172.21.15.3:3300/0 conn(0x7f432406a770 msgr2=0x7f432409d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:08.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.685+0000 7f43237fe700 1 --2- 172.21.15.3:0/2010619655 >> v2:172.21.15.3:3300/0 conn(0x7f432406a770 0x7f432409d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.685+0000 7f43237fe700 1 -- 172.21.15.3:0/2010619655 >> v1:172.21.15.3:6789/0 conn(0x7f432413c1e0 legacy=0x7f432413e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:08.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.685+0000 7f43237fe700 1 -- 172.21.15.3:0/2010619655 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432409e920 con 0x7f432409dc30 2023-10-03T10:15:08.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.685+0000 7f4323fff700 1 --2- 172.21.15.3:0/2010619655 >> v2:172.21.15.3:3300/0 conn(0x7f432406a770 0x7f432409d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:08.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.685+0000 7f43237fe700 1 --2- 172.21.15.3:0/2010619655 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f432409dc30 0x7f432409e080 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4318009840 tx=0x7f4318009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b8cd9fbaa446bc3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:08.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.685+0000 7f4322ffd700 1 -- 172.21.15.3:0/2010619655 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4318011830 con 0x7f432409dc30 2023-10-03T10:15:08.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.686+0000 7f4322ffd700 1 -- 172.21.15.3:0/2010619655 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4318011990 con 0x7f432409dc30 2023-10-03T10:15:08.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.686+0000 7f4322ffd700 1 -- 172.21.15.3:0/2010619655 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4318011d20 con 0x7f432409dc30 2023-10-03T10:15:08.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.686+0000 7f432a960700 1 -- 172.21.15.3:0/2010619655 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f432409dc30 msgr2=0x7f432409e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:08.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.686+0000 7f432a960700 1 --2- 172.21.15.3:0/2010619655 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f432409dc30 0x7f432409e080 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4318009840 tx=0x7f4318009b50 comp rx=0 tx=0).stop 2023-10-03T10:15:08.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.686+0000 7f432a960700 1 -- 172.21.15.3:0/2010619655 shutdown_connections 2023-10-03T10:15:08.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.686+0000 7f432a960700 1 --2- 172.21.15.3:0/2010619655 >> v2:172.21.15.3:3300/0 conn(0x7f432406a770 0x7f432409d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.686+0000 7f432a960700 1 --2- 172.21.15.3:0/2010619655 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f432409dc30 0x7f432409e080 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.686+0000 7f432a960700 1 -- 172.21.15.3:0/2010619655 >> 172.21.15.3:0/2010619655 conn(0x7f4324066f50 msgr2=0x7f432409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:08.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.686+0000 7f432a960700 1 -- 172.21.15.3:0/2010619655 shutdown_connections 2023-10-03T10:15:08.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.686+0000 7f432a960700 1 -- 172.21.15.3:0/2010619655 wait complete. 2023-10-03T10:15:08.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.687+0000 7f432a960700 1 Processor -- start 2023-10-03T10:15:08.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.688+0000 7f432a960700 1 -- start start 2023-10-03T10:15:08.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.688+0000 7f432a960700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f432406a770 0x7f4324071500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:08.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.688+0000 7f432a960700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f432409dc30 0x7f432413a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:08.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.688+0000 7f432a960700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f432413c1e0 0x7f432413aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:08.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.688+0000 7f432a960700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432409f1a0 con 0x7f432413c1e0 2023-10-03T10:15:08.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.688+0000 7f432a960700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432411ad40 con 0x7f432409dc30 2023-10-03T10:15:08.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.688+0000 7f432a960700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f432413bc10 con 0x7f432406a770 2023-10-03T10:15:08.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.688+0000 7f43237fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f432409dc30 0x7f432413a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:08.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.688+0000 7f43237fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f432409dc30 0x7f432413a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51352/0 (socket says 172.21.15.3:51352) 2023-10-03T10:15:08.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.688+0000 7f4323fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f432406a770 0x7f4324071500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:08.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.688+0000 7f43237fe700 1 -- 172.21.15.3:0/1442869203 learned_addr learned my addr 172.21.15.3:0/1442869203 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:08.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.689+0000 7f4328efd700 1 --2- 172.21.15.3:0/1442869203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f432413c1e0 0x7f432413aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:08.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.689+0000 7f43237fe700 1 -- 172.21.15.3:0/1442869203 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f432406a770 msgr2=0x7f4324071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:08.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.689+0000 7f43237fe700 1 --2- 172.21.15.3:0/1442869203 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f432406a770 0x7f4324071500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.689+0000 7f43237fe700 1 -- 172.21.15.3:0/1442869203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f432413c1e0 msgr2=0x7f432413aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:08.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.689+0000 7f43237fe700 1 --2- 172.21.15.3:0/1442869203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f432413c1e0 0x7f432413aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.689+0000 7f43237fe700 1 -- 172.21.15.3:0/1442869203 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43140094a0 con 0x7f432409dc30 2023-10-03T10:15:08.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.689+0000 7f43237fe700 1 --2- 172.21.15.3:0/1442869203 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f432409dc30 0x7f432413a4c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4318009fd0 tx=0x7f4318004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:08.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.689+0000 7f4320ff9700 1 -- 172.21.15.3:0/1442869203 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4318004550 con 0x7f432409dc30 2023-10-03T10:15:08.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.689+0000 7f4320ff9700 1 -- 172.21.15.3:0/1442869203 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431801f070 con 0x7f432409dc30 2023-10-03T10:15:08.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.689+0000 7f432a960700 1 -- 172.21.15.3:0/1442869203 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43180094a0 con 0x7f432409dc30 2023-10-03T10:15:08.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.689+0000 7f4320ff9700 1 -- 172.21.15.3:0/1442869203 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43180038e0 con 0x7f432409dc30 2023-10-03T10:15:08.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.690+0000 7f432a960700 1 -- 172.21.15.3:0/1442869203 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4324119480 con 0x7f432409dc30 2023-10-03T10:15:08.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.691+0000 7f4320ff9700 1 -- 172.21.15.3:0/1442869203 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4318003a40 con 0x7f432409dc30 2023-10-03T10:15:08.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.691+0000 7f432a960700 1 -- 172.21.15.3:0/1442869203 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4324074ea0 con 0x7f432409dc30 2023-10-03T10:15:08.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.691+0000 7f4320ff9700 1 -- 172.21.15.3:0/1442869203 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4318021070 con 0x7f432409dc30 2023-10-03T10:15:08.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.696+0000 7f4320ff9700 1 -- 172.21.15.3:0/1442869203 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f431802f3a0 con 0x7f432409dc30 2023-10-03T10:15:08.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.754+0000 7fb4219ed700 1 Processor -- start 2023-10-03T10:15:08.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.754+0000 7fb4219ed700 1 -- start start 2023-10-03T10:15:08.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.754+0000 7fb4219ed700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb41c09e930 0x7fb41c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:08.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.754+0000 7fb4219ed700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb41c09f650 0x7fb41c131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:08.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.754+0000 7fb4219ed700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb41c070940 con 0x7fb41c09f240 2023-10-03T10:15:08.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.754+0000 7fb4219ed700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c06fd60 con 0x7fb41c09f650 2023-10-03T10:15:08.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.754+0000 7fb4219ed700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c06fa80 con 0x7fb41c09e930 2023-10-03T10:15:08.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.754+0000 7fb41affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb41c09e930 0x7fb41c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:08.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.754+0000 7fb41b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb41c09f650 0x7fb41c131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:08.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.754+0000 7fb41affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb41c09e930 0x7fb41c09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38632/0 (socket says 172.21.15.3:38632) 2023-10-03T10:15:08.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.755+0000 7fb41affd700 1 -- 172.21.15.3:0/3917436923 learned_addr learned my addr 172.21.15.3:0/3917436923 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:08.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.755+0000 7fb41b7fe700 1 -- 172.21.15.3:0/3917436923 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb41c09e930 msgr2=0x7fb41c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:08.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.755+0000 7fb41b7fe700 1 --2- 172.21.15.3:0/3917436923 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb41c09e930 0x7fb41c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.755+0000 7fb41b7fe700 1 -- 172.21.15.3:0/3917436923 >> v1:172.21.15.3:6789/0 conn(0x7fb41c09f240 legacy=0x7fb41c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:08.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.755+0000 7fb41b7fe700 1 -- 172.21.15.3:0/3917436923 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c09fb60 con 0x7fb41c09f650 2023-10-03T10:15:08.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.755+0000 7fb41b7fe700 1 --2- 172.21.15.3:0/3917436923 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb41c09f650 0x7fb41c131f00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb414009dc0 tx=0x7fb41400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8054697a4a623a71 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:08.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.755+0000 7fb419ffb700 1 -- 172.21.15.3:0/3917436923 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41400e070 con 0x7fb41c09f650 2023-10-03T10:15:08.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.755+0000 7fb419ffb700 1 -- 172.21.15.3:0/3917436923 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb414003a20 con 0x7fb41c09f650 2023-10-03T10:15:08.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.755+0000 7fb419ffb700 1 -- 172.21.15.3:0/3917436923 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb414003dd0 con 0x7fb41c09f650 2023-10-03T10:15:08.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.756+0000 7fb4219ed700 1 -- 172.21.15.3:0/3917436923 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb41c09f650 msgr2=0x7fb41c131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:08.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.756+0000 7fb4219ed700 1 --2- 172.21.15.3:0/3917436923 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb41c09f650 0x7fb41c131f00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb414009dc0 tx=0x7fb41400b330 comp rx=0 tx=0).stop 2023-10-03T10:15:08.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.756+0000 7fb4219ed700 1 -- 172.21.15.3:0/3917436923 shutdown_connections 2023-10-03T10:15:08.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.756+0000 7fb4219ed700 1 --2- 172.21.15.3:0/3917436923 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb41c09f650 0x7fb41c131f00 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.756+0000 7fb4219ed700 1 --2- 172.21.15.3:0/3917436923 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb41c09e930 0x7fb41c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.756+0000 7fb4219ed700 1 -- 172.21.15.3:0/3917436923 >> 172.21.15.3:0/3917436923 conn(0x7fb41c0a4210 msgr2=0x7fb41c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:08.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.756+0000 7fb4219ed700 1 -- 172.21.15.3:0/3917436923 shutdown_connections 2023-10-03T10:15:08.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.756+0000 7fb4219ed700 1 -- 172.21.15.3:0/3917436923 wait complete. 2023-10-03T10:15:08.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.758+0000 7fb4219ed700 1 Processor -- start 2023-10-03T10:15:08.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.758+0000 7fb4219ed700 1 -- start start 2023-10-03T10:15:08.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.759+0000 7fb4219ed700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb41c09e930 0x7fb41c1b6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:08.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.759+0000 7fb4219ed700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb41c09f240 0x7fb41c1a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:08.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.759+0000 7fb4219ed700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb41c09f650 0x7fb41c1a4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:08.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.759+0000 7fb4219ed700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c1a80c0 con 0x7fb41c09f240 2023-10-03T10:15:08.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.759+0000 7fb4219ed700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c1a7700 con 0x7fb41c09e930 2023-10-03T10:15:08.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.759+0000 7fb4219ed700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c1a6e90 con 0x7fb41c09f650 2023-10-03T10:15:08.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.759+0000 7fb41b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb41c09f650 0x7fb41c1a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:08.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.759+0000 7fb41affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb41c09e930 0x7fb41c1b6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:08.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.759+0000 7fb41b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb41c09f650 0x7fb41c1a4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38646/0 (socket says 172.21.15.3:38646) 2023-10-03T10:15:08.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.759+0000 7fb41b7fe700 1 -- 172.21.15.3:0/149253678 learned_addr learned my addr 172.21.15.3:0/149253678 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:08.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.760+0000 7fb41a7fc700 1 --2- 172.21.15.3:0/149253678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb41c09f240 0x7fb41c1a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:08.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.760+0000 7fb41affd700 1 -- 172.21.15.3:0/149253678 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb41c09f650 msgr2=0x7fb41c1a4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:08.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.760+0000 7fb41affd700 1 --2- 172.21.15.3:0/149253678 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb41c09f650 0x7fb41c1a4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.760+0000 7fb41affd700 1 -- 172.21.15.3:0/149253678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb41c09f240 msgr2=0x7fb41c1a3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:08.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.760+0000 7fb41affd700 1 --2- 172.21.15.3:0/149253678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb41c09f240 0x7fb41c1a3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.760+0000 7fb41affd700 1 -- 172.21.15.3:0/149253678 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb414009a70 con 0x7fb41c09e930 2023-10-03T10:15:08.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.760+0000 7fb41a7fc700 1 --2- 172.21.15.3:0/149253678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb41c09f240 0x7fb41c1a3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:15:08.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.760+0000 7fb41affd700 1 --2- 172.21.15.3:0/149253678 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb41c09e930 0x7fb41c1b6fd0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb40c00e910 tx=0x7fb40c00ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:08.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.760+0000 7fb403fff700 1 -- 172.21.15.3:0/149253678 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c00ee60 con 0x7fb41c09e930 2023-10-03T10:15:08.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.760+0000 7fb403fff700 1 -- 172.21.15.3:0/149253678 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40c00c850 con 0x7fb41c09e930 2023-10-03T10:15:08.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.760+0000 7fb4219ed700 1 -- 172.21.15.3:0/149253678 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb41c0a0190 con 0x7fb41c09e930 2023-10-03T10:15:08.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.760+0000 7fb403fff700 1 -- 172.21.15.3:0/149253678 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c00cbe0 con 0x7fb41c09e930 2023-10-03T10:15:08.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.760+0000 7fb4219ed700 1 -- 172.21.15.3:0/149253678 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41c071d20 con 0x7fb41c09e930 2023-10-03T10:15:08.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.761+0000 7fb403fff700 1 -- 172.21.15.3:0/149253678 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb40c00cd40 con 0x7fb41c09e930 2023-10-03T10:15:08.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.762+0000 7fb403fff700 1 -- 172.21.15.3:0/149253678 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb40c012d90 con 0x7fb41c09e930 2023-10-03T10:15:08.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.762+0000 7fb4219ed700 1 -- 172.21.15.3:0/149253678 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb41c1a5850 con 0x7fb41c09e930 2023-10-03T10:15:08.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.769+0000 7fb403fff700 1 -- 172.21.15.3:0/149253678 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb40c021080 con 0x7fb41c09e930 2023-10-03T10:15:08.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.786+0000 7f53c8f9f700 1 Processor -- start 2023-10-03T10:15:08.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.787+0000 7f53c8f9f700 1 -- start start 2023-10-03T10:15:08.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.787+0000 7f53c8f9f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53c409e930 0x7f53c409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:08.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.787+0000 7f53c8f9f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f53c409f240 0x7f53c41317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:08.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.787+0000 7f53c8f9f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f53c409f650 0x7f53c4131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:08.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.787+0000 7f53c8f9f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c4070940 con 0x7f53c409f650 2023-10-03T10:15:08.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.787+0000 7f53c8f9f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c406fd60 con 0x7f53c409f240 2023-10-03T10:15:08.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.787+0000 7f53c8f9f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53c406fa80 con 0x7f53c409e930 2023-10-03T10:15:08.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.787+0000 7f53c1d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f53c409f240 0x7f53c41317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:08.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.787+0000 7f53c1d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f53c409f240 0x7f53c41317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51392/0 (socket says 172.21.15.3:51392) 2023-10-03T10:15:08.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.788+0000 7f53c1d9b700 1 -- 172.21.15.3:0/1124915577 learned_addr learned my addr 172.21.15.3:0/1124915577 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:08.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.788+0000 7f53c259c700 1 --2- 172.21.15.3:0/1124915577 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53c409e930 0x7f53c409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:08.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.788+0000 7f53c2d9d700 1 --2- 172.21.15.3:0/1124915577 >> v2:172.21.15.3:3300/0 conn(0x7f53c409f650 0x7f53c4131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:08.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.788+0000 7f53c1d9b700 1 -- 172.21.15.3:0/1124915577 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53c409e930 msgr2=0x7f53c409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:08.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.788+0000 7f53c1d9b700 1 --2- 172.21.15.3:0/1124915577 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53c409e930 0x7f53c409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.788+0000 7f53c1d9b700 1 -- 172.21.15.3:0/1124915577 >> v2:172.21.15.3:3300/0 conn(0x7f53c409f650 msgr2=0x7f53c4131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:08.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.788+0000 7f53c1d9b700 1 --2- 172.21.15.3:0/1124915577 >> v2:172.21.15.3:3300/0 conn(0x7f53c409f650 0x7f53c4131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.788+0000 7f53c1d9b700 1 -- 172.21.15.3:0/1124915577 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c409fb60 con 0x7f53c409f240 2023-10-03T10:15:08.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.788+0000 7f53c1d9b700 1 --2- 172.21.15.3:0/1124915577 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f53c409f240 0x7f53c41317f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f53b80098f0 tx=0x7f53b8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42ec7427d7cb1529 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:08.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.788+0000 7f53c159a700 1 -- 172.21.15.3:0/1124915577 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b8011830 con 0x7f53c409f240 2023-10-03T10:15:08.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.788+0000 7f53c159a700 1 -- 172.21.15.3:0/1124915577 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b8011990 con 0x7f53c409f240 2023-10-03T10:15:08.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.788+0000 7f53c159a700 1 -- 172.21.15.3:0/1124915577 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b8011d20 con 0x7f53c409f240 2023-10-03T10:15:08.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.789+0000 7f53c8f9f700 1 -- 172.21.15.3:0/1124915577 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f53c409f240 msgr2=0x7f53c41317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:08.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.789+0000 7f53c8f9f700 1 --2- 172.21.15.3:0/1124915577 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f53c409f240 0x7f53c41317f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f53b80098f0 tx=0x7f53b8009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:08.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.789+0000 7f53c8f9f700 1 -- 172.21.15.3:0/1124915577 shutdown_connections 2023-10-03T10:15:08.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.789+0000 7f53c8f9f700 1 --2- 172.21.15.3:0/1124915577 >> v2:172.21.15.3:3300/0 conn(0x7f53c409f650 0x7f53c4131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.789+0000 7f53c8f9f700 1 --2- 172.21.15.3:0/1124915577 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53c409e930 0x7f53c409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.789+0000 7f53c8f9f700 1 --2- 172.21.15.3:0/1124915577 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f53c409f240 0x7f53c41317f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.789+0000 7f53c8f9f700 1 -- 172.21.15.3:0/1124915577 >> 172.21.15.3:0/1124915577 conn(0x7f53c40a4210 msgr2=0x7f53c40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:08.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.789+0000 7f53c8f9f700 1 -- 172.21.15.3:0/1124915577 shutdown_connections 2023-10-03T10:15:08.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.789+0000 7f53c8f9f700 1 -- 172.21.15.3:0/1124915577 wait complete. 2023-10-03T10:15:08.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.791+0000 7f53c8f9f700 1 Processor -- start 2023-10-03T10:15:08.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.791+0000 7f53c8f9f700 1 -- start start 2023-10-03T10:15:08.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.791+0000 7f53c8f9f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f53c409e930 0x7f53c41aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:08.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.791+0000 7f53c8f9f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53c409f240 0x7f53c41aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:08.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.791+0000 7f53c8f9f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53c409f650 0x7f53c41aaf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:08.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.791+0000 7f53c8f9f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c41b5020 con 0x7f53c409f240 2023-10-03T10:15:08.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.791+0000 7f53c8f9f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c41ac0c0 con 0x7f53c409e930 2023-10-03T10:15:08.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.791+0000 7f53c8f9f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53c40a34a0 con 0x7f53c409f650 2023-10-03T10:15:08.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.791+0000 7f53c2d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53c409f650 0x7f53c41aaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:08.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.791+0000 7f53c2d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53c409f650 0x7f53c41aaf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38672/0 (socket says 172.21.15.3:38672) 2023-10-03T10:15:08.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.791+0000 7f53c2d9d700 1 -- 172.21.15.3:0/908003763 learned_addr learned my addr 172.21.15.3:0/908003763 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:08.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.792+0000 7f53c2d9d700 1 -- 172.21.15.3:0/908003763 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f53c409e930 msgr2=0x7f53c41aa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:08.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.792+0000 7f53c2d9d700 1 --2- 172.21.15.3:0/908003763 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f53c409e930 0x7f53c41aa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.792+0000 7f53c2d9d700 1 -- 172.21.15.3:0/908003763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53c409f240 msgr2=0x7f53c41aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:08.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.792+0000 7f53c2d9d700 1 --2- 172.21.15.3:0/908003763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53c409f240 0x7f53c41aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.792+0000 7f53c2d9d700 1 -- 172.21.15.3:0/908003763 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b8009550 con 0x7f53c409f650 2023-10-03T10:15:08.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.792+0000 7f53c2d9d700 1 --2- 172.21.15.3:0/908003763 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53c409f650 0x7f53c41aaf30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00d380 tx=0x7f53bc00d740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:08.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.792+0000 7f53b2ffd700 1 -- 172.21.15.3:0/908003763 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc00dee0 con 0x7f53c409f650 2023-10-03T10:15:08.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.792+0000 7f53b2ffd700 1 -- 172.21.15.3:0/908003763 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53bc013070 con 0x7f53c409f650 2023-10-03T10:15:08.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.792+0000 7f53c8f9f700 1 -- 172.21.15.3:0/908003763 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c40a0190 con 0x7f53c409f650 2023-10-03T10:15:08.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.792+0000 7f53b2ffd700 1 -- 172.21.15.3:0/908003763 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc01a8c0 con 0x7f53c409f650 2023-10-03T10:15:08.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.792+0000 7f53c8f9f700 1 -- 172.21.15.3:0/908003763 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c41add40 con 0x7f53c409f650 2023-10-03T10:15:08.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.793+0000 7f53c8f9f700 1 -- 172.21.15.3:0/908003763 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c4074ea0 con 0x7f53c409f650 2023-10-03T10:15:08.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.794+0000 7f597c78a700 1 -- 172.21.15.3:0/440449756 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5974122480 con 0x7f59740a0a30 2023-10-03T10:15:08.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.794+0000 7f5972ffd700 1 -- 172.21.15.3:0/440449756 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5974122480 con 0x7f59740a0a30 2023-10-03T10:15:08.797 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:08.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.794+0000 7f53b2ffd700 1 -- 172.21.15.3:0/908003763 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f53bc01aa20 con 0x7f53c409f650 2023-10-03T10:15:08.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.794+0000 7f53b2ffd700 1 -- 172.21.15.3:0/908003763 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f53bc015070 con 0x7f53c409f650 2023-10-03T10:15:08.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.797+0000 7f597c78a700 1 -- 172.21.15.3:0/440449756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59740a0a30 msgr2=0x7f597411fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:08.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.797+0000 7f597c78a700 1 --2- 172.21.15.3:0/440449756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59740a0a30 0x7f597411fa50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5968016000 tx=0x7f5968004750 comp rx=0 tx=0).stop 2023-10-03T10:15:08.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.798+0000 7f597c78a700 1 -- 172.21.15.3:0/440449756 shutdown_connections 2023-10-03T10:15:08.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.798+0000 7f597c78a700 1 --2- 172.21.15.3:0/440449756 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5974133930 0x7f59741c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.798+0000 7f597c78a700 1 --2- 172.21.15.3:0/440449756 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f597409d630 0x7f597411f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.798+0000 7f597c78a700 1 --2- 172.21.15.3:0/440449756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59740a0a30 0x7f597411fa50 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.798+0000 7f597c78a700 1 -- 172.21.15.3:0/440449756 >> 172.21.15.3:0/440449756 conn(0x7f59740a3050 msgr2=0x7f59741bbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:08.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.798+0000 7f597c78a700 1 -- 172.21.15.3:0/440449756 shutdown_connections 2023-10-03T10:15:08.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.800+0000 7f597c78a700 1 -- 172.21.15.3:0/440449756 wait complete. 2023-10-03T10:15:08.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.800+0000 7f53b2ffd700 1 -- 172.21.15.3:0/908003763 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f53bc022d70 con 0x7f53c409f650 2023-10-03T10:15:08.814 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:08.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.909+0000 7f432a960700 1 -- 172.21.15.3:0/1442869203 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f432406c360 con 0x7f432409dc30 2023-10-03T10:15:08.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.909+0000 7f4320ff9700 1 -- 172.21.15.3:0/1442869203 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f431802b030 con 0x7f432409dc30 2023-10-03T10:15:08.909 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:08.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.911+0000 7f432a960700 1 -- 172.21.15.3:0/1442869203 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f432409dc30 msgr2=0x7f432413a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:08.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.911+0000 7f432a960700 1 --2- 172.21.15.3:0/1442869203 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f432409dc30 0x7f432413a4c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4318009fd0 tx=0x7f4318004750 comp rx=0 tx=0).stop 2023-10-03T10:15:08.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.911+0000 7f432a960700 1 -- 172.21.15.3:0/1442869203 shutdown_connections 2023-10-03T10:15:08.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.911+0000 7f432a960700 1 --2- 172.21.15.3:0/1442869203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f432413c1e0 0x7f432413aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.911+0000 7f432a960700 1 --2- 172.21.15.3:0/1442869203 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f432406a770 0x7f4324071500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.911+0000 7f432a960700 1 --2- 172.21.15.3:0/1442869203 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f432409dc30 0x7f432413a4c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.911+0000 7f432a960700 1 -- 172.21.15.3:0/1442869203 >> 172.21.15.3:0/1442869203 conn(0x7f4324066f50 msgr2=0x7f432412e060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:08.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.911+0000 7f432a960700 1 -- 172.21.15.3:0/1442869203 shutdown_connections 2023-10-03T10:15:08.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.911+0000 7f432a960700 1 -- 172.21.15.3:0/1442869203 wait complete. 2023-10-03T10:15:08.925 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:08.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.979+0000 7fb4219ed700 1 -- 172.21.15.3:0/149253678 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb41c09ffe0 con 0x7fb41c09e930 2023-10-03T10:15:08.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.980+0000 7fb403fff700 1 -- 172.21.15.3:0/149253678 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb40c017020 con 0x7fb41c09e930 2023-10-03T10:15:08.980 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:08.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.982+0000 7fb4219ed700 1 -- 172.21.15.3:0/149253678 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb41c09e930 msgr2=0x7fb41c1b6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:08.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.982+0000 7fb4219ed700 1 --2- 172.21.15.3:0/149253678 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb41c09e930 0x7fb41c1b6fd0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb40c00e910 tx=0x7fb40c00ec20 comp rx=0 tx=0).stop 2023-10-03T10:15:08.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.982+0000 7fb4219ed700 1 -- 172.21.15.3:0/149253678 shutdown_connections 2023-10-03T10:15:08.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.982+0000 7fb4219ed700 1 --2- 172.21.15.3:0/149253678 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb41c09f650 0x7fb41c1a4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.982+0000 7fb4219ed700 1 --2- 172.21.15.3:0/149253678 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb41c09e930 0x7fb41c1b6fd0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.982+0000 7fb4219ed700 1 --2- 172.21.15.3:0/149253678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb41c09f240 0x7fb41c1a3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:08.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.982+0000 7fb4219ed700 1 -- 172.21.15.3:0/149253678 >> 172.21.15.3:0/149253678 conn(0x7fb41c0a4210 msgr2=0x7fb41c1b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:08.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.982+0000 7fb4219ed700 1 -- 172.21.15.3:0/149253678 shutdown_connections 2023-10-03T10:15:08.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:08.982+0000 7fb4219ed700 1 -- 172.21.15.3:0/149253678 wait complete. 2023-10-03T10:15:08.996 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:09.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.009+0000 7f53c8f9f700 1 -- 172.21.15.3:0/908003763 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f53c41add40 con 0x7f53c409f650 2023-10-03T10:15:09.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.010+0000 7f53b2ffd700 1 -- 172.21.15.3:0/908003763 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f53c41add40 con 0x7f53c409f650 2023-10-03T10:15:09.010 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:09.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.012+0000 7f53c8f9f700 1 -- 172.21.15.3:0/908003763 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53c409f650 msgr2=0x7f53c41aaf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:09.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.012+0000 7f53c8f9f700 1 --2- 172.21.15.3:0/908003763 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53c409f650 0x7f53c41aaf30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00d380 tx=0x7f53bc00d740 comp rx=0 tx=0).stop 2023-10-03T10:15:09.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.012+0000 7f53c8f9f700 1 -- 172.21.15.3:0/908003763 shutdown_connections 2023-10-03T10:15:09.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.012+0000 7f53c8f9f700 1 --2- 172.21.15.3:0/908003763 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53c409f650 0x7f53c41aaf30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:09.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.012+0000 7f53c8f9f700 1 --2- 172.21.15.3:0/908003763 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f53c409e930 0x7f53c41aa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:09.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.012+0000 7f53c8f9f700 1 --2- 172.21.15.3:0/908003763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53c409f240 0x7f53c41aa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:09.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.012+0000 7f53c8f9f700 1 -- 172.21.15.3:0/908003763 >> 172.21.15.3:0/908003763 conn(0x7f53c40a4210 msgr2=0x7f53c41b6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:09.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.012+0000 7f53c8f9f700 1 -- 172.21.15.3:0/908003763 shutdown_connections 2023-10-03T10:15:09.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.012+0000 7f53c8f9f700 1 -- 172.21.15.3:0/908003763 wait complete. 2023-10-03T10:15:09.026 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:09.815 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:09.926 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:09.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.964+0000 7f9410618700 1 Processor -- start 2023-10-03T10:15:09.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.964+0000 7f9410618700 1 -- start start 2023-10-03T10:15:09.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.965+0000 7f9410618700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f940809d6f0 0x7f940809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:09.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.965+0000 7f9410618700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f940809e000 0x7f9408130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:09.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.965+0000 7f9410618700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f940809f130 con 0x7f940809e410 2023-10-03T10:15:09.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.965+0000 7f9410618700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f940809f520 con 0x7f940809d6f0 2023-10-03T10:15:09.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.965+0000 7f9410618700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f940809e930 con 0x7f940809e000 2023-10-03T10:15:09.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.965+0000 7f940e3b4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f940809d6f0 0x7f940809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:09.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.965+0000 7f940dbb3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f940809e000 0x7f9408130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:09.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.965+0000 7f940e3b4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f940809d6f0 0x7f940809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57320/0 (socket says 172.21.15.3:57320) 2023-10-03T10:15:09.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.965+0000 7f940e3b4700 1 -- 172.21.15.3:0/1259901924 learned_addr learned my addr 172.21.15.3:0/1259901924 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:09.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.966+0000 7f940e3b4700 1 -- 172.21.15.3:0/1259901924 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f940809e000 msgr2=0x7f9408130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:09.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.966+0000 7f940e3b4700 1 --2- 172.21.15.3:0/1259901924 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f940809e000 0x7f9408130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:09.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.966+0000 7f940e3b4700 1 -- 172.21.15.3:0/1259901924 >> v1:172.21.15.3:6789/0 conn(0x7f940809e410 legacy=0x7f9408130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:09.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.966+0000 7f940e3b4700 1 -- 172.21.15.3:0/1259901924 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f940809b2d0 con 0x7f940809d6f0 2023-10-03T10:15:09.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.966+0000 7f940d3b2700 1 -- 172.21.15.3:0/1259901924 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2892576649 0 0) 0x7f940809f130 con 0x7f940809e410 2023-10-03T10:15:09.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.966+0000 7f940dbb3700 1 --2- 172.21.15.3:0/1259901924 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f940809e000 0x7f9408130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:09.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.966+0000 7f940e3b4700 1 --2- 172.21.15.3:0/1259901924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f940809d6f0 0x7f940809dac0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f93f800bb90 tx=0x7f93f800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=814d0337c8ed354 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:09.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.966+0000 7f940d3b2700 1 -- 172.21.15.3:0/1259901924 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f800dd30 con 0x7f940809d6f0 2023-10-03T10:15:09.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.966+0000 7f940d3b2700 1 -- 172.21.15.3:0/1259901924 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f800de90 con 0x7f940809d6f0 2023-10-03T10:15:09.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.966+0000 7f940d3b2700 1 -- 172.21.15.3:0/1259901924 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f8010690 con 0x7f940809d6f0 2023-10-03T10:15:09.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.967+0000 7f9410618700 1 -- 172.21.15.3:0/1259901924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f940809d6f0 msgr2=0x7f940809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:09.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.967+0000 7f9410618700 1 --2- 172.21.15.3:0/1259901924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f940809d6f0 0x7f940809dac0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f93f800bb90 tx=0x7f93f800bea0 comp rx=0 tx=0).stop 2023-10-03T10:15:09.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.967+0000 7f9410618700 1 -- 172.21.15.3:0/1259901924 shutdown_connections 2023-10-03T10:15:09.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.967+0000 7f9410618700 1 --2- 172.21.15.3:0/1259901924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f940809d6f0 0x7f940809dac0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:09.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.967+0000 7f9410618700 1 --2- 172.21.15.3:0/1259901924 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f940809e000 0x7f9408130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:09.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.967+0000 7f9410618700 1 -- 172.21.15.3:0/1259901924 >> 172.21.15.3:0/1259901924 conn(0x7f9408066f50 msgr2=0x7f940809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:09.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.967+0000 7f9410618700 1 -- 172.21.15.3:0/1259901924 shutdown_connections 2023-10-03T10:15:09.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.967+0000 7f9410618700 1 -- 172.21.15.3:0/1259901924 wait complete. 2023-10-03T10:15:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.968+0000 7f9410618700 1 Processor -- start 2023-10-03T10:15:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.968+0000 7f9410618700 1 -- start start 2023-10-03T10:15:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.969+0000 7f9410618700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f940809d6f0 0x7f9408122410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.969+0000 7f9410618700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f940809e000 0x7f9408122950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.969+0000 7f9410618700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f940809e410 0x7f9408122e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.969+0000 7f9410618700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9408125f30 con 0x7f940809e410 2023-10-03T10:15:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.969+0000 7f9410618700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9408124020 con 0x7f940809e000 2023-10-03T10:15:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.969+0000 7f9410618700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94080a2070 con 0x7f940809d6f0 2023-10-03T10:15:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.969+0000 7f940ebb5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f940809e410 0x7f9408122e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.969+0000 7f940ebb5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f940809e410 0x7f9408122e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57326/0 (socket says 172.21.15.3:57326) 2023-10-03T10:15:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.969+0000 7f940ebb5700 1 -- 172.21.15.3:0/3665650709 learned_addr learned my addr 172.21.15.3:0/3665650709 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.969+0000 7f940e3b4700 1 --2- 172.21.15.3:0/3665650709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f940809d6f0 0x7f9408122410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.969+0000 7f940dbb3700 1 --2- 172.21.15.3:0/3665650709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f940809e000 0x7f9408122950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.969+0000 7f940e3b4700 1 -- 172.21.15.3:0/3665650709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f940809e000 msgr2=0x7f9408122950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.969+0000 7f940e3b4700 1 --2- 172.21.15.3:0/3665650709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f940809e000 0x7f9408122950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.969+0000 7f940e3b4700 1 -- 172.21.15.3:0/3665650709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f940809e410 msgr2=0x7f9408122e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.969+0000 7f940e3b4700 1 --2- 172.21.15.3:0/3665650709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f940809e410 0x7f9408122e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.969+0000 7f940e3b4700 1 -- 172.21.15.3:0/3665650709 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93fc0094a0 con 0x7f940809d6f0 2023-10-03T10:15:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.969+0000 7f940ebb5700 1 --2- 172.21.15.3:0/3665650709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f940809e410 0x7f9408122e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.969+0000 7f940e3b4700 1 --2- 172.21.15.3:0/3665650709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f940809d6f0 0x7f9408122410 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f93f8018000 tx=0x7f93f8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.969+0000 7f9406ffd700 1 -- 172.21.15.3:0/3665650709 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f8003b70 con 0x7f940809d6f0 2023-10-03T10:15:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.970+0000 7f9406ffd700 1 -- 172.21.15.3:0/3665650709 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f8003cd0 con 0x7f940809d6f0 2023-10-03T10:15:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.970+0000 7f9410618700 1 -- 172.21.15.3:0/3665650709 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f800b7f0 con 0x7f940809d6f0 2023-10-03T10:15:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.970+0000 7f9406ffd700 1 -- 172.21.15.3:0/3665650709 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f8010690 con 0x7f940809d6f0 2023-10-03T10:15:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.970+0000 7f9410618700 1 -- 172.21.15.3:0/3665650709 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94081247e0 con 0x7f940809d6f0 2023-10-03T10:15:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.970+0000 7f9406ffd700 1 -- 172.21.15.3:0/3665650709 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f93f802c430 con 0x7f940809d6f0 2023-10-03T10:15:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.971+0000 7f9406ffd700 1 -- 172.21.15.3:0/3665650709 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f93f8040fe0 con 0x7f940809d6f0 2023-10-03T10:15:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.971+0000 7f9410618700 1 -- 172.21.15.3:0/3665650709 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9408125ab0 con 0x7f940809d6f0 2023-10-03T10:15:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:09.976+0000 7f9406ffd700 1 -- 172.21.15.3:0/3665650709 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9408125ab0 con 0x7f940809d6f0 2023-10-03T10:15:09.997 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:10.027 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:10.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.070+0000 7f6d209e2700 1 Processor -- start 2023-10-03T10:15:10.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.070+0000 7f6d209e2700 1 -- start start 2023-10-03T10:15:10.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.071+0000 7f6d209e2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d1c09d6f0 0x7f6d1c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:10.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.071+0000 7f6d209e2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6d1c09e000 0x7f6d1c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:10.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.071+0000 7f6d209e2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d1c09b120 con 0x7f6d1c09e410 2023-10-03T10:15:10.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.071+0000 7f6d209e2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c06b460 con 0x7f6d1c09d6f0 2023-10-03T10:15:10.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.071+0000 7f6d209e2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c09a7e0 con 0x7f6d1c09e000 2023-10-03T10:15:10.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.071+0000 7f6d1a59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d1c09d6f0 0x7f6d1c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:10.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.071+0000 7f6d1a59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d1c09d6f0 0x7f6d1c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51420/0 (socket says 172.21.15.3:51420) 2023-10-03T10:15:10.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.071+0000 7f6d1a59c700 1 -- 172.21.15.3:0/3360234670 learned_addr learned my addr 172.21.15.3:0/3360234670 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:10.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.072+0000 7f6d19d9b700 1 --2- 172.21.15.3:0/3360234670 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6d1c09e000 0x7f6d1c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:10.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.072+0000 7f6d1a59c700 1 -- 172.21.15.3:0/3360234670 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6d1c09e000 msgr2=0x7f6d1c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:10.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.072+0000 7f6d1a59c700 1 --2- 172.21.15.3:0/3360234670 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6d1c09e000 0x7f6d1c1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:10.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.072+0000 7f6d1a59c700 1 -- 172.21.15.3:0/3360234670 >> v1:172.21.15.3:6789/0 conn(0x7f6d1c09e410 legacy=0x7f6d1c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:10.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.072+0000 7f6d1959a700 1 -- 172.21.15.3:0/3360234670 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2253462947 0 0) 0x7f6d1c09b120 con 0x7f6d1c09e410 2023-10-03T10:15:10.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.072+0000 7f6d1a59c700 1 -- 172.21.15.3:0/3360234670 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1c09a500 con 0x7f6d1c09d6f0 2023-10-03T10:15:10.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.072+0000 7f6d1a59c700 1 --2- 172.21.15.3:0/3360234670 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d1c09d6f0 0x7f6d1c09dac0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009920 tx=0x7f6d0c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31eb5b605dcb81e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:10.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.073+0000 7f6d1959a700 1 -- 172.21.15.3:0/3360234670 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c011830 con 0x7f6d1c09d6f0 2023-10-03T10:15:10.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.073+0000 7f6d1959a700 1 -- 172.21.15.3:0/3360234670 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0c011990 con 0x7f6d1c09d6f0 2023-10-03T10:15:10.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.073+0000 7f6d1959a700 1 -- 172.21.15.3:0/3360234670 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c011d20 con 0x7f6d1c09d6f0 2023-10-03T10:15:10.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.074+0000 7f6d209e2700 1 -- 172.21.15.3:0/3360234670 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d1c09d6f0 msgr2=0x7f6d1c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:10.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.074+0000 7f6d209e2700 1 --2- 172.21.15.3:0/3360234670 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d1c09d6f0 0x7f6d1c09dac0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009920 tx=0x7f6d0c009c30 comp rx=0 tx=0).stop 2023-10-03T10:15:10.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.074+0000 7f6d209e2700 1 -- 172.21.15.3:0/3360234670 shutdown_connections 2023-10-03T10:15:10.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.074+0000 7f6d209e2700 1 --2- 172.21.15.3:0/3360234670 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d1c09d6f0 0x7f6d1c09dac0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:10.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.074+0000 7f6d209e2700 1 --2- 172.21.15.3:0/3360234670 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6d1c09e000 0x7f6d1c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:10.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.074+0000 7f6d209e2700 1 -- 172.21.15.3:0/3360234670 >> 172.21.15.3:0/3360234670 conn(0x7f6d1c066f50 msgr2=0x7f6d1c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:10.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.074+0000 7f6d209e2700 1 -- 172.21.15.3:0/3360234670 shutdown_connections 2023-10-03T10:15:10.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.074+0000 7f6d209e2700 1 -- 172.21.15.3:0/3360234670 wait complete. 2023-10-03T10:15:10.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.076+0000 7f6d209e2700 1 Processor -- start 2023-10-03T10:15:10.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.076+0000 7f6d209e2700 1 -- start start 2023-10-03T10:15:10.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.077+0000 7f6d209e2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6d1c09d6f0 0x7f6d1c1c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:10.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.077+0000 7f6d209e2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d1c09e000 0x7f6d1c1c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:10.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.077+0000 7f6d209e2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d1c09e410 0x7f6d1c1c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:10.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.077+0000 7f6d209e2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c1a3d80 con 0x7f6d1c09e410 2023-10-03T10:15:10.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.077+0000 7f6d209e2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c1a7110 con 0x7f6d1c09e000 2023-10-03T10:15:10.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.077+0000 7f6d209e2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c1a5450 con 0x7f6d1c09d6f0 2023-10-03T10:15:10.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.077+0000 7f6d1a59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6d1c09d6f0 0x7f6d1c1c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:10.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.077+0000 7f6d1a59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6d1c09d6f0 0x7f6d1c1c8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38702/0 (socket says 172.21.15.3:38702) 2023-10-03T10:15:10.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.077+0000 7f6d1a59c700 1 -- 172.21.15.3:0/3073969462 learned_addr learned my addr 172.21.15.3:0/3073969462 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:10.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.077+0000 7f6d19d9b700 1 --2- 172.21.15.3:0/3073969462 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d1c09e000 0x7f6d1c1c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:10.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.077+0000 7f6d19d9b700 1 -- 172.21.15.3:0/3073969462 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6d1c09d6f0 msgr2=0x7f6d1c1c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:10.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.077+0000 7f6d19d9b700 1 --2- 172.21.15.3:0/3073969462 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6d1c09d6f0 0x7f6d1c1c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:10.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.077+0000 7f6d19d9b700 1 -- 172.21.15.3:0/3073969462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d1c09e410 msgr2=0x7f6d1c1c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:10.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.077+0000 7f6d19d9b700 1 --2- 172.21.15.3:0/3073969462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d1c09e410 0x7f6d1c1c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:10.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.077+0000 7f6d19d9b700 1 -- 172.21.15.3:0/3073969462 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d0c009550 con 0x7f6d1c09e000 2023-10-03T10:15:10.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.078+0000 7f6d1a59c700 1 --2- 172.21.15.3:0/3073969462 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6d1c09d6f0 0x7f6d1c1c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:10.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.078+0000 7f6d19d9b700 1 --2- 172.21.15.3:0/3073969462 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d1c09e000 0x7f6d1c1c8740 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000e900 tx=0x7f6d1000ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:10.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.078+0000 7f6d0affd700 1 -- 172.21.15.3:0/3073969462 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1000ee50 con 0x7f6d1c09e000 2023-10-03T10:15:10.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.078+0000 7f6d0affd700 1 -- 172.21.15.3:0/3073969462 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1000c850 con 0x7f6d1c09e000 2023-10-03T10:15:10.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.078+0000 7f6d0affd700 1 -- 172.21.15.3:0/3073969462 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1000cbc0 con 0x7f6d1c09e000 2023-10-03T10:15:10.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.078+0000 7f6d209e2700 1 -- 172.21.15.3:0/3073969462 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1c0a1e70 con 0x7f6d1c09e000 2023-10-03T10:15:10.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.078+0000 7f6d209e2700 1 -- 172.21.15.3:0/3073969462 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1c1a5c40 con 0x7f6d1c09e000 2023-10-03T10:15:10.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.079+0000 7f6d0affd700 1 -- 172.21.15.3:0/3073969462 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6d1000cd20 con 0x7f6d1c09e000 2023-10-03T10:15:10.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.079+0000 7f6d209e2700 1 -- 172.21.15.3:0/3073969462 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d1c074ea0 con 0x7f6d1c09e000 2023-10-03T10:15:10.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.080+0000 7f6d0affd700 1 -- 172.21.15.3:0/3073969462 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6d1003f7b0 con 0x7f6d1c09e000 2023-10-03T10:15:10.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.084+0000 7f6d0affd700 1 -- 172.21.15.3:0/3073969462 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6d100232f0 con 0x7f6d1c09e000 2023-10-03T10:15:10.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.141+0000 7fb853d9a700 1 Processor -- start 2023-10-03T10:15:10.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.142+0000 7fb853d9a700 1 -- start start 2023-10-03T10:15:10.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.143+0000 7fb853d9a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb84c0a1510 0x7fb84c138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:10.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.143+0000 7fb853d9a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb84c139250 0x7fb84c13b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:10.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.143+0000 7fb853d9a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb84c09edd0 con 0x7fb84c0a0a30 2023-10-03T10:15:10.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.143+0000 7fb853d9a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb84c0a18e0 con 0x7fb84c139250 2023-10-03T10:15:10.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.143+0000 7fb853d9a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c09e1f0 con 0x7fb84c0a1510 2023-10-03T10:15:10.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.143+0000 7fb851335700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb84c0a1510 0x7fb84c138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:10.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.143+0000 7fb851b36700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb84c0a0a30 0x7fb84c0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47108/0 (socket says 172.21.15.3:47108) 2023-10-03T10:15:10.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.143+0000 7fb851b36700 1 -- 172.21.15.3:0/550566794 learned_addr learned my addr 172.21.15.3:0/550566794 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:10.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.143+0000 7fb852337700 1 --2- 172.21.15.3:0/550566794 >> v2:172.21.15.3:3300/0 conn(0x7fb84c139250 0x7fb84c13b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:10.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.143+0000 7fb851335700 1 -- 172.21.15.3:0/550566794 >> v2:172.21.15.3:3300/0 conn(0x7fb84c139250 msgr2=0x7fb84c13b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:10.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.143+0000 7fb851335700 1 --2- 172.21.15.3:0/550566794 >> v2:172.21.15.3:3300/0 conn(0x7fb84c139250 0x7fb84c13b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:10.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.143+0000 7fb851335700 1 -- 172.21.15.3:0/550566794 >> v1:172.21.15.3:6789/0 conn(0x7fb84c0a0a30 legacy=0x7fb84c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:10.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.143+0000 7fb851335700 1 -- 172.21.15.3:0/550566794 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb84c09dc50 con 0x7fb84c0a1510 2023-10-03T10:15:10.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.143+0000 7fb851335700 1 --2- 172.21.15.3:0/550566794 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb84c0a1510 0x7fb84c138d10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009870 tx=0x7fb83c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a6e5a4a1c1b9c65 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:10.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.144+0000 7fb850b34700 1 -- 172.21.15.3:0/550566794 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83c011830 con 0x7fb84c0a1510 2023-10-03T10:15:10.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.144+0000 7fb850b34700 1 -- 172.21.15.3:0/550566794 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83c011990 con 0x7fb84c0a1510 2023-10-03T10:15:10.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.144+0000 7fb850b34700 1 -- 172.21.15.3:0/550566794 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83c011d20 con 0x7fb84c0a1510 2023-10-03T10:15:10.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.145+0000 7fb853d9a700 1 -- 172.21.15.3:0/550566794 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb84c0a1510 msgr2=0x7fb84c138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:10.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.145+0000 7fb853d9a700 1 --2- 172.21.15.3:0/550566794 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb84c0a1510 0x7fb84c138d10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009870 tx=0x7fb83c009b80 comp rx=0 tx=0).stop 2023-10-03T10:15:10.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.145+0000 7fb853d9a700 1 -- 172.21.15.3:0/550566794 shutdown_connections 2023-10-03T10:15:10.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.145+0000 7fb853d9a700 1 --2- 172.21.15.3:0/550566794 >> v2:172.21.15.3:3300/0 conn(0x7fb84c139250 0x7fb84c13b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:10.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.145+0000 7fb853d9a700 1 --2- 172.21.15.3:0/550566794 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb84c0a1510 0x7fb84c138d10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:10.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.145+0000 7fb853d9a700 1 -- 172.21.15.3:0/550566794 >> 172.21.15.3:0/550566794 conn(0x7fb84c0a6550 msgr2=0x7fb84c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:10.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.145+0000 7fb853d9a700 1 -- 172.21.15.3:0/550566794 shutdown_connections 2023-10-03T10:15:10.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.145+0000 7fb853d9a700 1 -- 172.21.15.3:0/550566794 wait complete. 2023-10-03T10:15:10.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.146+0000 7fb853d9a700 1 Processor -- start 2023-10-03T10:15:10.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.146+0000 7fb853d9a700 1 -- start start 2023-10-03T10:15:10.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.147+0000 7fb853d9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb84c0a0a30 0x7fb84c1218c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:10.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.147+0000 7fb853d9a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb84c0a1510 0x7fb84c121e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:10.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.147+0000 7fb853d9a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb84c139250 0x7fb84c122340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:10.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.147+0000 7fb853d9a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c1236c0 con 0x7fb84c0a0a30 2023-10-03T10:15:10.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.147+0000 7fb853d9a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c1234d0 con 0x7fb84c139250 2023-10-03T10:15:10.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.147+0000 7fb853d9a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c1253e0 con 0x7fb84c0a1510 2023-10-03T10:15:10.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.147+0000 7fb851335700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb84c0a1510 0x7fb84c121e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:10.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.147+0000 7fb851335700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb84c0a1510 0x7fb84c121e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38724/0 (socket says 172.21.15.3:38724) 2023-10-03T10:15:10.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.148+0000 7fb851335700 1 -- 172.21.15.3:0/2607481665 learned_addr learned my addr 172.21.15.3:0/2607481665 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:10.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.148+0000 7fb852337700 1 --2- 172.21.15.3:0/2607481665 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb84c139250 0x7fb84c122340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:10.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.148+0000 7fb851b36700 1 --2- 172.21.15.3:0/2607481665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb84c0a0a30 0x7fb84c1218c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:10.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.148+0000 7fb851335700 1 -- 172.21.15.3:0/2607481665 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb84c139250 msgr2=0x7fb84c122340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:10.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.148+0000 7fb851335700 1 --2- 172.21.15.3:0/2607481665 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb84c139250 0x7fb84c122340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:10.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.148+0000 7fb851335700 1 -- 172.21.15.3:0/2607481665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb84c0a0a30 msgr2=0x7fb84c1218c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:10.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.148+0000 7fb851335700 1 --2- 172.21.15.3:0/2607481665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb84c0a0a30 0x7fb84c1218c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:10.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.148+0000 7fb851335700 1 -- 172.21.15.3:0/2607481665 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb83c0094a0 con 0x7fb84c0a1510 2023-10-03T10:15:10.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.148+0000 7fb851b36700 1 --2- 172.21.15.3:0/2607481665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb84c0a0a30 0x7fb84c1218c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:15:10.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.148+0000 7fb851335700 1 --2- 172.21.15.3:0/2607481665 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb84c0a1510 0x7fb84c121e00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009840 tx=0x7fb83c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:10.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.148+0000 7fb8427fc700 1 -- 172.21.15.3:0/2607481665 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83c004590 con 0x7fb84c0a1510 2023-10-03T10:15:10.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.148+0000 7fb8427fc700 1 -- 172.21.15.3:0/2607481665 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83c01f070 con 0x7fb84c0a1510 2023-10-03T10:15:10.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.148+0000 7fb853d9a700 1 -- 172.21.15.3:0/2607481665 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb84c138bd0 con 0x7fb84c0a1510 2023-10-03T10:15:10.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.148+0000 7fb8427fc700 1 -- 172.21.15.3:0/2607481665 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83c0038e0 con 0x7fb84c0a1510 2023-10-03T10:15:10.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.149+0000 7fb8427fc700 1 -- 172.21.15.3:0/2607481665 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb83c003a40 con 0x7fb84c0a1510 2023-10-03T10:15:10.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.150+0000 7fb853d9a700 1 -- 172.21.15.3:0/2607481665 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb84c123cc0 con 0x7fb84c0a1510 2023-10-03T10:15:10.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.151+0000 7fb82ffff700 1 -- 172.21.15.3:0/2607481665 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb84c074ea0 con 0x7fb84c0a1510 2023-10-03T10:15:10.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.151+0000 7fb8427fc700 1 -- 172.21.15.3:0/2607481665 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb83c021070 con 0x7fb84c0a1510 2023-10-03T10:15:10.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.156+0000 7fb8427fc700 1 -- 172.21.15.3:0/2607481665 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb83c026480 con 0x7fb84c0a1510 2023-10-03T10:15:10.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.193+0000 7f9410618700 1 -- 172.21.15.3:0/3665650709 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9408125ab0 con 0x7f940809d6f0 2023-10-03T10:15:10.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.194+0000 7f9406ffd700 1 -- 172.21.15.3:0/3665650709 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f93f8003e40 con 0x7f940809d6f0 2023-10-03T10:15:10.194 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:10.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.196+0000 7f9410618700 1 -- 172.21.15.3:0/3665650709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f940809d6f0 msgr2=0x7f9408122410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:10.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.196+0000 7f9410618700 1 --2- 172.21.15.3:0/3665650709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f940809d6f0 0x7f9408122410 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f93f8018000 tx=0x7f93f8004750 comp rx=0 tx=0).stop 2023-10-03T10:15:10.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.196+0000 7f9410618700 1 -- 172.21.15.3:0/3665650709 shutdown_connections 2023-10-03T10:15:10.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.196+0000 7f9410618700 1 --2- 172.21.15.3:0/3665650709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f940809e410 0x7f9408122e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:10.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.196+0000 7f9410618700 1 --2- 172.21.15.3:0/3665650709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f940809d6f0 0x7f9408122410 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:10.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.196+0000 7f9410618700 1 --2- 172.21.15.3:0/3665650709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f940809e000 0x7f9408122950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:10.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.196+0000 7f9410618700 1 -- 172.21.15.3:0/3665650709 >> 172.21.15.3:0/3665650709 conn(0x7f9408066f50 msgr2=0x7f9408130a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:10.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.196+0000 7f9410618700 1 -- 172.21.15.3:0/3665650709 shutdown_connections 2023-10-03T10:15:10.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.196+0000 7f9410618700 1 -- 172.21.15.3:0/3665650709 wait complete. 2023-10-03T10:15:10.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.209+0000 7f0f0842c700 1 Processor -- start 2023-10-03T10:15:10.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.209+0000 7f0f0842c700 1 -- start start 2023-10-03T10:15:10.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.210+0000 7f0f0842c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f0012cf60 0x7f0f0006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:10.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.210+0000 7f0f0842c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f00130430 0x7f0f00137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:10.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.210+0000 7f0f0842c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f0012cd90 con 0x7f0f0012f8b0 2023-10-03T10:15:10.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.210+0000 7f0f0842c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0012d330 con 0x7f0f00130430 2023-10-03T10:15:10.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.210+0000 7f0f0842c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f00071780 con 0x7f0f0012cf60 2023-10-03T10:15:10.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.210+0000 7f0f059c7700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0f0012f8b0 0x7f0f0012fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47114/0 (socket says 172.21.15.3:47114) 2023-10-03T10:15:10.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.210+0000 7f0f059c7700 1 -- 172.21.15.3:0/1520034212 learned_addr learned my addr 172.21.15.3:0/1520034212 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:10.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.211+0000 7f0f051c6700 1 -- 172.21.15.3:0/1520034212 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4001951260 0 0) 0x7f0f0012cd90 con 0x7f0f0012f8b0 2023-10-03T10:15:10.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.211+0000 7f0f051c6700 1 -- 172.21.15.3:0/1520034212 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0eec0035b0 con 0x7f0f0012f8b0 2023-10-03T10:15:10.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.211+0000 7f0f061c8700 1 --2- 172.21.15.3:0/1520034212 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f0012cf60 0x7f0f0006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:10.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.211+0000 7f0f069c9700 1 --2- 172.21.15.3:0/1520034212 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f00130430 0x7f0f00137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:10.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.211+0000 7f0f051c6700 1 -- 172.21.15.3:0/1520034212 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 261657364 0 0) 0x7f0eec0035b0 con 0x7f0f0012f8b0 2023-10-03T10:15:10.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.211+0000 7f0f061c8700 1 -- 172.21.15.3:0/1520034212 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f00130430 msgr2=0x7f0f00137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:10.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.211+0000 7f0f061c8700 1 --2- 172.21.15.3:0/1520034212 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f00130430 0x7f0f00137c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:10.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.211+0000 7f0f061c8700 1 -- 172.21.15.3:0/1520034212 >> v1:172.21.15.3:6789/0 conn(0x7f0f0012f8b0 legacy=0x7f0f0012fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:10.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.211+0000 7f0f061c8700 1 -- 172.21.15.3:0/1520034212 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f00071a40 con 0x7f0f0012cf60 2023-10-03T10:15:10.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.211+0000 7f0f069c9700 1 --2- 172.21.15.3:0/1520034212 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f00130430 0x7f0f00137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:10.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.212+0000 7f0f061c8700 1 --2- 172.21.15.3:0/1520034212 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f0012cf60 0x7f0f0006f5d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0efc009870 tx=0x7f0efc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f5f23d7e0d89854 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:10.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.212+0000 7f0f051c6700 1 -- 172.21.15.3:0/1520034212 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0001670 con 0x7f0f0012cf60 2023-10-03T10:15:10.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.212+0000 7f0f051c6700 1 -- 172.21.15.3:0/1520034212 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0efc004030 con 0x7f0f0012cf60 2023-10-03T10:15:10.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.212+0000 7f0f051c6700 1 -- 172.21.15.3:0/1520034212 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0efc0043c0 con 0x7f0f0012cf60 2023-10-03T10:15:10.217 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:10.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.213+0000 7f0f0842c700 1 -- 172.21.15.3:0/1520034212 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f0012cf60 msgr2=0x7f0f0006f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:10.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.213+0000 7f0f0842c700 1 --2- 172.21.15.3:0/1520034212 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f0012cf60 0x7f0f0006f5d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0efc009870 tx=0x7f0efc009b80 comp rx=0 tx=0).stop 2023-10-03T10:15:10.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.213+0000 7f0f0842c700 1 -- 172.21.15.3:0/1520034212 shutdown_connections 2023-10-03T10:15:10.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.213+0000 7f0f0842c700 1 --2- 172.21.15.3:0/1520034212 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f00130430 0x7f0f00137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:10.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.213+0000 7f0f0842c700 1 --2- 172.21.15.3:0/1520034212 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f0012cf60 0x7f0f0006f5d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:10.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.213+0000 7f0f0842c700 1 -- 172.21.15.3:0/1520034212 >> 172.21.15.3:0/1520034212 conn(0x7f0f000a2aa0 msgr2=0x7f0f000a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:10.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.213+0000 7f0f0842c700 1 -- 172.21.15.3:0/1520034212 shutdown_connections 2023-10-03T10:15:10.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.214+0000 7f0f0842c700 1 -- 172.21.15.3:0/1520034212 wait complete. 2023-10-03T10:15:10.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.215+0000 7f0f0842c700 1 Processor -- start 2023-10-03T10:15:10.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.215+0000 7f0f0842c700 1 -- start start 2023-10-03T10:15:10.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.216+0000 7f0f0842c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f0012cf60 0x7f0f001309e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:10.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.216+0000 7f0f0842c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f0012f8b0 0x7f0f00130f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:10.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.216+0000 7f0f0842c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f00130430 0x7f0f00131460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:10.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.216+0000 7f0f0842c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f00133100 con 0x7f0f00130430 2023-10-03T10:15:10.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.216+0000 7f0f0842c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0013b370 con 0x7f0f0012f8b0 2023-10-03T10:15:10.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.216+0000 7f0f0842c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0013b830 con 0x7f0f0012cf60 2023-10-03T10:15:10.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.216+0000 7f0f059c7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f0012f8b0 0x7f0f00130f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:10.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.216+0000 7f0f059c7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f0012f8b0 0x7f0f00130f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51464/0 (socket says 172.21.15.3:51464) 2023-10-03T10:15:10.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.216+0000 7f0f059c7700 1 -- 172.21.15.3:0/3656348902 learned_addr learned my addr 172.21.15.3:0/3656348902 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:10.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.216+0000 7f0f069c9700 1 --2- 172.21.15.3:0/3656348902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f00130430 0x7f0f00131460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:10.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.216+0000 7f0f061c8700 1 --2- 172.21.15.3:0/3656348902 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f0012cf60 0x7f0f001309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:10.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.216+0000 7f0f059c7700 1 -- 172.21.15.3:0/3656348902 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f0012cf60 msgr2=0x7f0f001309e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:10.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.216+0000 7f0f059c7700 1 --2- 172.21.15.3:0/3656348902 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f0012cf60 0x7f0f001309e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:10.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.216+0000 7f0f059c7700 1 -- 172.21.15.3:0/3656348902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f00130430 msgr2=0x7f0f00131460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:10.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.216+0000 7f0f059c7700 1 --2- 172.21.15.3:0/3656348902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f00130430 0x7f0f00131460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:10.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.216+0000 7f0f059c7700 1 -- 172.21.15.3:0/3656348902 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef8009a70 con 0x7f0f0012f8b0 2023-10-03T10:15:10.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.216+0000 7f0f069c9700 1 --2- 172.21.15.3:0/3656348902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f00130430 0x7f0f00131460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:10.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.217+0000 7f0f059c7700 1 --2- 172.21.15.3:0/3656348902 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f0012f8b0 0x7f0f00130f20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0ef000c750 tx=0x7f0ef000ca60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:10.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.217+0000 7f0ef6ffd700 1 -- 172.21.15.3:0/3656348902 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0004c80 con 0x7f0f0012f8b0 2023-10-03T10:15:10.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.217+0000 7f0ef6ffd700 1 -- 172.21.15.3:0/3656348902 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ef0004de0 con 0x7f0f0012f8b0 2023-10-03T10:15:10.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.217+0000 7f0ef6ffd700 1 -- 172.21.15.3:0/3656348902 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0007310 con 0x7f0f0012f8b0 2023-10-03T10:15:10.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.217+0000 7f0f0842c700 1 -- 172.21.15.3:0/3656348902 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0efc0094a0 con 0x7f0f0012f8b0 2023-10-03T10:15:10.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.217+0000 7f0f0842c700 1 -- 172.21.15.3:0/3656348902 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f0009ec10 con 0x7f0f0012f8b0 2023-10-03T10:15:10.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.218+0000 7f0ef6ffd700 1 -- 172.21.15.3:0/3656348902 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0ef0007470 con 0x7f0f0012f8b0 2023-10-03T10:15:10.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.218+0000 7f0ef6ffd700 1 -- 172.21.15.3:0/3656348902 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0ef003f440 con 0x7f0f0012f8b0 2023-10-03T10:15:10.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.218+0000 7f0f0842c700 1 -- 172.21.15.3:0/3656348902 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f00133250 con 0x7f0f0012f8b0 2023-10-03T10:15:10.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.227+0000 7f0ef6ffd700 1 -- 172.21.15.3:0/3656348902 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0ef00123b0 con 0x7f0f0012f8b0 2023-10-03T10:15:10.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.316+0000 7f6d209e2700 1 -- 172.21.15.3:0/3073969462 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6d1c1a6f70 con 0x7f6d1c09e000 2023-10-03T10:15:10.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.317+0000 7f6d0affd700 1 -- 172.21.15.3:0/3073969462 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6d1c1a6f70 con 0x7f6d1c09e000 2023-10-03T10:15:10.317 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:10.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.319+0000 7f6d209e2700 1 -- 172.21.15.3:0/3073969462 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d1c09e000 msgr2=0x7f6d1c1c8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:10.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.319+0000 7f6d209e2700 1 --2- 172.21.15.3:0/3073969462 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d1c09e000 0x7f6d1c1c8740 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000e900 tx=0x7f6d1000ec10 comp rx=0 tx=0).stop 2023-10-03T10:15:10.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.319+0000 7f6d209e2700 1 -- 172.21.15.3:0/3073969462 shutdown_connections 2023-10-03T10:15:10.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.319+0000 7f6d209e2700 1 --2- 172.21.15.3:0/3073969462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d1c09e410 0x7f6d1c1c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:10.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.319+0000 7f6d209e2700 1 --2- 172.21.15.3:0/3073969462 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6d1c09d6f0 0x7f6d1c1c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:10.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.319+0000 7f6d209e2700 1 --2- 172.21.15.3:0/3073969462 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d1c09e000 0x7f6d1c1c8740 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:10.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.319+0000 7f6d209e2700 1 -- 172.21.15.3:0/3073969462 >> 172.21.15.3:0/3073969462 conn(0x7f6d1c066f50 msgr2=0x7f6d1c1bbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:10.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.319+0000 7f6d209e2700 1 -- 172.21.15.3:0/3073969462 shutdown_connections 2023-10-03T10:15:10.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.319+0000 7f6d209e2700 1 -- 172.21.15.3:0/3073969462 wait complete. 2023-10-03T10:15:10.334 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:10.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.380+0000 7fb82ffff700 1 -- 172.21.15.3:0/2607481665 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb84c0a28e0 con 0x7fb84c0a1510 2023-10-03T10:15:10.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.380+0000 7fb8427fc700 1 -- 172.21.15.3:0/2607481665 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb83c015030 con 0x7fb84c0a1510 2023-10-03T10:15:10.381 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:10.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.383+0000 7fb82ffff700 1 -- 172.21.15.3:0/2607481665 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb84c0a1510 msgr2=0x7fb84c121e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:10.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.383+0000 7fb82ffff700 1 --2- 172.21.15.3:0/2607481665 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb84c0a1510 0x7fb84c121e00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009840 tx=0x7fb83c004750 comp rx=0 tx=0).stop 2023-10-03T10:15:10.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.383+0000 7fb82ffff700 1 -- 172.21.15.3:0/2607481665 shutdown_connections 2023-10-03T10:15:10.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.383+0000 7fb82ffff700 1 --2- 172.21.15.3:0/2607481665 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb84c139250 0x7fb84c122340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:10.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.383+0000 7fb82ffff700 1 --2- 172.21.15.3:0/2607481665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb84c0a0a30 0x7fb84c1218c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:10.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.383+0000 7fb82ffff700 1 --2- 172.21.15.3:0/2607481665 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb84c0a1510 0x7fb84c121e00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:10.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.383+0000 7fb82ffff700 1 -- 172.21.15.3:0/2607481665 >> 172.21.15.3:0/2607481665 conn(0x7fb84c0a6550 msgr2=0x7fb84c137b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:10.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.383+0000 7fb82ffff700 1 -- 172.21.15.3:0/2607481665 shutdown_connections 2023-10-03T10:15:10.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.383+0000 7fb82ffff700 1 -- 172.21.15.3:0/2607481665 wait complete. 2023-10-03T10:15:10.397 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:10.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.433+0000 7f0f0842c700 1 -- 172.21.15.3:0/3656348902 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0f00074ea0 con 0x7f0f0012f8b0 2023-10-03T10:15:10.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.434+0000 7f0ef6ffd700 1 -- 172.21.15.3:0/3656348902 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0ef001b340 con 0x7f0f0012f8b0 2023-10-03T10:15:10.434 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:10.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.436+0000 7f0f0842c700 1 -- 172.21.15.3:0/3656348902 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f0012f8b0 msgr2=0x7f0f00130f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:10.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.436+0000 7f0f0842c700 1 --2- 172.21.15.3:0/3656348902 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f0012f8b0 0x7f0f00130f20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0ef000c750 tx=0x7f0ef000ca60 comp rx=0 tx=0).stop 2023-10-03T10:15:10.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.436+0000 7f0f0842c700 1 -- 172.21.15.3:0/3656348902 shutdown_connections 2023-10-03T10:15:10.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.436+0000 7f0f0842c700 1 --2- 172.21.15.3:0/3656348902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f00130430 0x7f0f00131460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:10.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.436+0000 7f0f0842c700 1 --2- 172.21.15.3:0/3656348902 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f0012cf60 0x7f0f001309e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:10.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.436+0000 7f0f0842c700 1 --2- 172.21.15.3:0/3656348902 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f0012f8b0 0x7f0f00130f20 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:10.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.436+0000 7f0f0842c700 1 -- 172.21.15.3:0/3656348902 >> 172.21.15.3:0/3656348902 conn(0x7f0f000a2aa0 msgr2=0x7f0f0006e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:10.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.436+0000 7f0f0842c700 1 -- 172.21.15.3:0/3656348902 shutdown_connections 2023-10-03T10:15:10.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:10.436+0000 7f0f0842c700 1 -- 172.21.15.3:0/3656348902 wait complete. 2023-10-03T10:15:10.450 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:11.218 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:11.335 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:11.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.366+0000 7f340252c700 1 Processor -- start 2023-10-03T10:15:11.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.366+0000 7f340252c700 1 -- start start 2023-10-03T10:15:11.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.367+0000 7f340252c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f33fc1345a0 0x7f33fc136980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:11.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.367+0000 7f340252c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33fc136ec0 0x7f33fc1392a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:11.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.367+0000 7f340252c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33fc099620 con 0x7f33fc0984f0 2023-10-03T10:15:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.367+0000 7f340252c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33fc09a9e0 con 0x7f33fc1345a0 2023-10-03T10:15:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.367+0000 7f340252c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc09bce0 con 0x7f33fc136ec0 2023-10-03T10:15:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.367+0000 7f3400ac9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33fc136ec0 0x7f33fc1392a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.367+0000 7f33fb7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f33fc1345a0 0x7f33fc136980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.367+0000 7f3400ac9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33fc136ec0 0x7f33fc1392a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51476/0 (socket says 172.21.15.3:51476) 2023-10-03T10:15:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.367+0000 7f33fb7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f33fc1345a0 0x7f33fc136980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57362/0 (socket says 172.21.15.3:57362) 2023-10-03T10:15:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.367+0000 7f3400ac9700 1 -- 172.21.15.3:0/3375598206 learned_addr learned my addr 172.21.15.3:0/3375598206 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.367+0000 7f3400ac9700 1 -- 172.21.15.3:0/3375598206 >> v2:172.21.15.3:3300/0 conn(0x7f33fc1345a0 msgr2=0x7f33fc136980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.367+0000 7f3400ac9700 1 --2- 172.21.15.3:0/3375598206 >> v2:172.21.15.3:3300/0 conn(0x7f33fc1345a0 0x7f33fc136980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.367+0000 7f3400ac9700 1 -- 172.21.15.3:0/3375598206 >> v1:172.21.15.3:6789/0 conn(0x7f33fc0984f0 legacy=0x7f33fc0988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.368+0000 7f3400ac9700 1 -- 172.21.15.3:0/3375598206 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33fc09aca0 con 0x7f33fc136ec0 2023-10-03T10:15:11.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.368+0000 7f33fb7fe700 1 --2- 172.21.15.3:0/3375598206 >> v2:172.21.15.3:3300/0 conn(0x7f33fc1345a0 0x7f33fc136980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:11.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.368+0000 7f3400ac9700 1 --2- 172.21.15.3:0/3375598206 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33fc136ec0 0x7f33fc1392a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009e50 tx=0x7f33f400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd9465eb7d106c59 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:11.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.368+0000 7f33faffd700 1 -- 172.21.15.3:0/3375598206 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f400e070 con 0x7f33fc136ec0 2023-10-03T10:15:11.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.368+0000 7f33faffd700 1 -- 172.21.15.3:0/3375598206 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f400b910 con 0x7f33fc136ec0 2023-10-03T10:15:11.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.368+0000 7f33faffd700 1 -- 172.21.15.3:0/3375598206 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f400bcc0 con 0x7f33fc136ec0 2023-10-03T10:15:11.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.369+0000 7f340252c700 1 -- 172.21.15.3:0/3375598206 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33fc136ec0 msgr2=0x7f33fc1392a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:11.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.369+0000 7f340252c700 1 --2- 172.21.15.3:0/3375598206 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33fc136ec0 0x7f33fc1392a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009e50 tx=0x7f33f400b330 comp rx=0 tx=0).stop 2023-10-03T10:15:11.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.369+0000 7f340252c700 1 -- 172.21.15.3:0/3375598206 shutdown_connections 2023-10-03T10:15:11.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.369+0000 7f340252c700 1 --2- 172.21.15.3:0/3375598206 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33fc136ec0 0x7f33fc1392a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.369+0000 7f340252c700 1 --2- 172.21.15.3:0/3375598206 >> v2:172.21.15.3:3300/0 conn(0x7f33fc1345a0 0x7f33fc136980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.369+0000 7f340252c700 1 -- 172.21.15.3:0/3375598206 >> 172.21.15.3:0/3375598206 conn(0x7f33fc09f470 msgr2=0x7f33fc0a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:11.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.369+0000 7f340252c700 1 -- 172.21.15.3:0/3375598206 shutdown_connections 2023-10-03T10:15:11.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.369+0000 7f340252c700 1 -- 172.21.15.3:0/3375598206 wait complete. 2023-10-03T10:15:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.370+0000 7f340252c700 1 Processor -- start 2023-10-03T10:15:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.370+0000 7f340252c700 1 -- start start 2023-10-03T10:15:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.370+0000 7f340252c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f33fc0984f0 0x7f33fc126e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.370+0000 7f340252c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33fc1345a0 0x7f33fc127340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.370+0000 7f340252c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33fc136ec0 0x7f33fc127880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.370+0000 7f340252c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc0a1d90 con 0x7f33fc1345a0 2023-10-03T10:15:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.370+0000 7f340252c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc09a2b0 con 0x7f33fc136ec0 2023-10-03T10:15:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.370+0000 7f340252c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc131fd0 con 0x7f33fc0984f0 2023-10-03T10:15:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.371+0000 7f3400ac9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33fc136ec0 0x7f33fc127880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.371+0000 7f33fbfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f33fc0984f0 0x7f33fc126e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.371+0000 7f3400ac9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33fc136ec0 0x7f33fc127880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51482/0 (socket says 172.21.15.3:51482) 2023-10-03T10:15:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.371+0000 7f3400ac9700 1 -- 172.21.15.3:0/1003157553 learned_addr learned my addr 172.21.15.3:0/1003157553 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.371+0000 7f33fb7fe700 1 --2- 172.21.15.3:0/1003157553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33fc1345a0 0x7f33fc127340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.371+0000 7f33fb7fe700 1 -- 172.21.15.3:0/1003157553 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f33fc0984f0 msgr2=0x7f33fc126e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.371+0000 7f33fb7fe700 1 --2- 172.21.15.3:0/1003157553 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f33fc0984f0 0x7f33fc126e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.371+0000 7f33fb7fe700 1 -- 172.21.15.3:0/1003157553 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33fc136ec0 msgr2=0x7f33fc127880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.371+0000 7f33fb7fe700 1 --2- 172.21.15.3:0/1003157553 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33fc136ec0 0x7f33fc127880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.371+0000 7f33fb7fe700 1 -- 172.21.15.3:0/1003157553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f00094a0 con 0x7f33fc1345a0 2023-10-03T10:15:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.371+0000 7f3400ac9700 1 --2- 172.21.15.3:0/1003157553 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33fc136ec0 0x7f33fc127880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.371+0000 7f33fbfff700 1 --2- 172.21.15.3:0/1003157553 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f33fc0984f0 0x7f33fc126e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:11.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.371+0000 7f33fb7fe700 1 --2- 172.21.15.3:0/1003157553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33fc1345a0 0x7f33fc127340 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f33f000e9b0 tx=0x7f33f000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:11.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.371+0000 7f33f8ff9700 1 -- 172.21.15.3:0/1003157553 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f000ee50 con 0x7f33fc1345a0 2023-10-03T10:15:11.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.372+0000 7f33f8ff9700 1 -- 172.21.15.3:0/1003157553 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f000c850 con 0x7f33fc1345a0 2023-10-03T10:15:11.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.372+0000 7f33f8ff9700 1 -- 172.21.15.3:0/1003157553 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f000cc00 con 0x7f33fc1345a0 2023-10-03T10:15:11.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.372+0000 7f340252c700 1 -- 172.21.15.3:0/1003157553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f4009b30 con 0x7f33fc1345a0 2023-10-03T10:15:11.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.372+0000 7f340252c700 1 -- 172.21.15.3:0/1003157553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33fc131640 con 0x7f33fc1345a0 2023-10-03T10:15:11.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.373+0000 7f33f8ff9700 1 -- 172.21.15.3:0/1003157553 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f33f0013470 con 0x7f33fc1345a0 2023-10-03T10:15:11.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.373+0000 7f340252c700 1 -- 172.21.15.3:0/1003157553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33fc0747f0 con 0x7f33fc1345a0 2023-10-03T10:15:11.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.373+0000 7f33f8ff9700 1 -- 172.21.15.3:0/1003157553 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f33f003fab0 con 0x7f33fc1345a0 2023-10-03T10:15:11.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.382+0000 7f33f8ff9700 1 -- 172.21.15.3:0/1003157553 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f33f0013720 con 0x7f33fc1345a0 2023-10-03T10:15:11.397 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:11.451 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:11.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.483+0000 7f8c27cdd700 1 Processor -- start 2023-10-03T10:15:11.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.484+0000 7f8c27cdd700 1 -- start start 2023-10-03T10:15:11.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.484+0000 7f8c27cdd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c20130750 0x7f8c20130b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:11.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.484+0000 7f8c27cdd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c2009e250 0x7f8c201384c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:11.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.484+0000 7f8c27cdd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c2009c190 con 0x7f8c2009d6f0 2023-10-03T10:15:11.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.484+0000 7f8c27cdd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2009b090 con 0x7f8c2009e250 2023-10-03T10:15:11.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.484+0000 7f8c27cdd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2009a6c0 con 0x7f8c20130750 2023-10-03T10:15:11.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.484+0000 7f8c2627a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c2009e250 0x7f8c201384c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:11.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.484+0000 7f8c25a79700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c20130750 0x7f8c20130b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:11.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.484+0000 7f8c2627a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c2009e250 0x7f8c201384c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51484/0 (socket says 172.21.15.3:51484) 2023-10-03T10:15:11.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.484+0000 7f8c2627a700 1 -- 172.21.15.3:0/1080635308 learned_addr learned my addr 172.21.15.3:0/1080635308 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:11.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.485+0000 7f8c25a79700 1 -- 172.21.15.3:0/1080635308 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c2009e250 msgr2=0x7f8c201384c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:11.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.485+0000 7f8c25a79700 1 --2- 172.21.15.3:0/1080635308 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c2009e250 0x7f8c201384c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.485+0000 7f8c25a79700 1 -- 172.21.15.3:0/1080635308 >> v1:172.21.15.3:6789/0 conn(0x7f8c2009d6f0 legacy=0x7f8c2009db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:11.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.485+0000 7f8c25a79700 1 -- 172.21.15.3:0/1080635308 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c20131060 con 0x7f8c20130750 2023-10-03T10:15:11.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.485+0000 7f8c2627a700 1 --2- 172.21.15.3:0/1080635308 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c2009e250 0x7f8c201384c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:11.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.485+0000 7f8c25a79700 1 --2- 172.21.15.3:0/1080635308 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c20130750 0x7f8c20130b20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c009870 tx=0x7f8c1c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13c50c0f0c217f44 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:11.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.485+0000 7f8c24a77700 1 -- 172.21.15.3:0/1080635308 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c011830 con 0x7f8c20130750 2023-10-03T10:15:11.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.485+0000 7f8c24a77700 1 -- 172.21.15.3:0/1080635308 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c1c011990 con 0x7f8c20130750 2023-10-03T10:15:11.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.485+0000 7f8c24a77700 1 -- 172.21.15.3:0/1080635308 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c011d20 con 0x7f8c20130750 2023-10-03T10:15:11.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.486+0000 7f8c27cdd700 1 -- 172.21.15.3:0/1080635308 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c20130750 msgr2=0x7f8c20130b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.486+0000 7f8c27cdd700 1 --2- 172.21.15.3:0/1080635308 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c20130750 0x7f8c20130b20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c009870 tx=0x7f8c1c009b80 comp rx=0 tx=0).stop 2023-10-03T10:15:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.486+0000 7f8c27cdd700 1 -- 172.21.15.3:0/1080635308 shutdown_connections 2023-10-03T10:15:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.486+0000 7f8c27cdd700 1 --2- 172.21.15.3:0/1080635308 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c2009e250 0x7f8c201384c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.486+0000 7f8c27cdd700 1 --2- 172.21.15.3:0/1080635308 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c20130750 0x7f8c20130b20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.486+0000 7f8c27cdd700 1 -- 172.21.15.3:0/1080635308 >> 172.21.15.3:0/1080635308 conn(0x7f8c2009f470 msgr2=0x7f8c200a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.486+0000 7f8c27cdd700 1 -- 172.21.15.3:0/1080635308 shutdown_connections 2023-10-03T10:15:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.486+0000 7f8c27cdd700 1 -- 172.21.15.3:0/1080635308 wait complete. 2023-10-03T10:15:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.487+0000 7f8c27cdd700 1 Processor -- start 2023-10-03T10:15:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.487+0000 7f8c27cdd700 1 -- start start 2023-10-03T10:15:11.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.488+0000 7f8c27cdd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c2009d6f0 0x7f8c20135e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:11.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.488+0000 7f8c27cdd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c2009e250 0x7f8c20136380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:11.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.488+0000 7f8c27cdd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c20130750 0x7f8c201368c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:11.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.488+0000 7f8c27cdd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c201271f0 con 0x7f8c2009e250 2023-10-03T10:15:11.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.488+0000 7f8c27cdd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c20137870 con 0x7f8c2009d6f0 2023-10-03T10:15:11.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.488+0000 7f8c27cdd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c200a0190 con 0x7f8c20130750 2023-10-03T10:15:11.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.488+0000 7f8c2627a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c20130750 0x7f8c201368c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:11.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.488+0000 7f8c2627a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c20130750 0x7f8c201368c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38772/0 (socket says 172.21.15.3:38772) 2023-10-03T10:15:11.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.488+0000 7f8c2627a700 1 -- 172.21.15.3:0/3886629730 learned_addr learned my addr 172.21.15.3:0/3886629730 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:11.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.488+0000 7f8c25278700 1 --2- 172.21.15.3:0/3886629730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c2009e250 0x7f8c20136380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:11.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.488+0000 7f8c2627a700 1 -- 172.21.15.3:0/3886629730 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c2009d6f0 msgr2=0x7f8c20135e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:11.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.488+0000 7f8c2627a700 1 --2- 172.21.15.3:0/3886629730 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c2009d6f0 0x7f8c20135e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.488+0000 7f8c2627a700 1 -- 172.21.15.3:0/3886629730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c2009e250 msgr2=0x7f8c20136380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:11.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.488+0000 7f8c2627a700 1 --2- 172.21.15.3:0/3886629730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c2009e250 0x7f8c20136380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.488+0000 7f8c2627a700 1 -- 172.21.15.3:0/3886629730 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c18009b30 con 0x7f8c20130750 2023-10-03T10:15:11.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.489+0000 7f8c25278700 1 --2- 172.21.15.3:0/3886629730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c2009e250 0x7f8c20136380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:15:11.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.489+0000 7f8c2627a700 1 --2- 172.21.15.3:0/3886629730 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c20130750 0x7f8c201368c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800ef90 tx=0x7f8c1800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:11.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.489+0000 7f8c167fc700 1 -- 172.21.15.3:0/3886629730 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c18018070 con 0x7f8c20130750 2023-10-03T10:15:11.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.489+0000 7f8c167fc700 1 -- 172.21.15.3:0/3886629730 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c1800f040 con 0x7f8c20130750 2023-10-03T10:15:11.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.489+0000 7f8c167fc700 1 -- 172.21.15.3:0/3886629730 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c18003c60 con 0x7f8c20130750 2023-10-03T10:15:11.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.489+0000 7f8c27cdd700 1 -- 172.21.15.3:0/3886629730 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1c0094a0 con 0x7f8c20130750 2023-10-03T10:15:11.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.489+0000 7f8c27cdd700 1 -- 172.21.15.3:0/3886629730 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c20137f50 con 0x7f8c20130750 2023-10-03T10:15:11.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.490+0000 7f8c167fc700 1 -- 172.21.15.3:0/3886629730 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8c1800c9f0 con 0x7f8c20130750 2023-10-03T10:15:11.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.490+0000 7f8c27cdd700 1 -- 172.21.15.3:0/3886629730 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c20132a10 con 0x7f8c20130750 2023-10-03T10:15:11.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.491+0000 7f8c167fc700 1 -- 172.21.15.3:0/3886629730 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8c180135a0 con 0x7f8c20130750 2023-10-03T10:15:11.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.496+0000 7f8c167fc700 1 -- 172.21.15.3:0/3886629730 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8c1800cca0 con 0x7f8c20130750 2023-10-03T10:15:11.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.552+0000 7f14874e0700 1 Processor -- start 2023-10-03T10:15:11.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.552+0000 7f14874e0700 1 -- start start 2023-10-03T10:15:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.553+0000 7f14874e0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f14800a0a30 0x7f14800a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.553+0000 7f14874e0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1480134170 0x7f148013bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.553+0000 7f14874e0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f148009edd0 con 0x7f14800a1340 2023-10-03T10:15:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.553+0000 7f14874e0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f148009e1f0 con 0x7f14800a0a30 2023-10-03T10:15:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.553+0000 7f14874e0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f148009dc50 con 0x7f1480134170 2023-10-03T10:15:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.553+0000 7f148527c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f14800a0a30 0x7f14800a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.553+0000 7f1484a7b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f14800a1340 0x7f1480133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47142/0 (socket says 172.21.15.3:47142) 2023-10-03T10:15:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.553+0000 7f148527c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f14800a0a30 0x7f14800a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57384/0 (socket says 172.21.15.3:57384) 2023-10-03T10:15:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.553+0000 7f1484a7b700 1 -- 172.21.15.3:0/386342822 learned_addr learned my addr 172.21.15.3:0/386342822 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.554+0000 7f1485a7d700 1 --2- 172.21.15.3:0/386342822 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1480134170 0x7f148013bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:11.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.554+0000 7f148527c700 1 -- 172.21.15.3:0/386342822 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1480134170 msgr2=0x7f148013bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:11.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.554+0000 7f148527c700 1 --2- 172.21.15.3:0/386342822 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1480134170 0x7f148013bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.554+0000 7f148527c700 1 -- 172.21.15.3:0/386342822 >> v1:172.21.15.3:6789/0 conn(0x7f14800a1340 legacy=0x7f1480133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:11.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.554+0000 7f148527c700 1 -- 172.21.15.3:0/386342822 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14800a1fe0 con 0x7f14800a0a30 2023-10-03T10:15:11.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.554+0000 7f1485a7d700 1 --2- 172.21.15.3:0/386342822 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1480134170 0x7f148013bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:11.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.554+0000 7f148527c700 1 --2- 172.21.15.3:0/386342822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14800a0a30 0x7f14800a0e00 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f147c009840 tx=0x7f147c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8aee7b2c32311e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:11.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.554+0000 7f1477fff700 1 -- 172.21.15.3:0/386342822 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c004230 con 0x7f14800a0a30 2023-10-03T10:15:11.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.555+0000 7f1477fff700 1 -- 172.21.15.3:0/386342822 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147c004390 con 0x7f14800a0a30 2023-10-03T10:15:11.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.555+0000 7f1477fff700 1 -- 172.21.15.3:0/386342822 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c0036a0 con 0x7f14800a0a30 2023-10-03T10:15:11.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.555+0000 7f14874e0700 1 -- 172.21.15.3:0/386342822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14800a0a30 msgr2=0x7f14800a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:11.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.555+0000 7f14874e0700 1 --2- 172.21.15.3:0/386342822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14800a0a30 0x7f14800a0e00 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f147c009840 tx=0x7f147c009b50 comp rx=0 tx=0).stop 2023-10-03T10:15:11.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.556+0000 7f14874e0700 1 -- 172.21.15.3:0/386342822 shutdown_connections 2023-10-03T10:15:11.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.556+0000 7f14874e0700 1 --2- 172.21.15.3:0/386342822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14800a0a30 0x7f14800a0e00 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.556+0000 7f14874e0700 1 --2- 172.21.15.3:0/386342822 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1480134170 0x7f148013bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.556+0000 7f14874e0700 1 -- 172.21.15.3:0/386342822 >> 172.21.15.3:0/386342822 conn(0x7f14800a6550 msgr2=0x7f14800a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:11.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.556+0000 7f14874e0700 1 -- 172.21.15.3:0/386342822 shutdown_connections 2023-10-03T10:15:11.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.556+0000 7f14874e0700 1 -- 172.21.15.3:0/386342822 wait complete. 2023-10-03T10:15:11.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.557+0000 7f14874e0700 1 Processor -- start 2023-10-03T10:15:11.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.557+0000 7f14874e0700 1 -- start start 2023-10-03T10:15:11.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.558+0000 7f14874e0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f14800a0a30 0x7f148011e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:11.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.558+0000 7f14874e0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f14800a1340 0x7f148011ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:11.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.558+0000 7f14874e0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1480134170 0x7f148011f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:11.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.558+0000 7f14874e0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14801203e0 con 0x7f1480134170 2023-10-03T10:15:11.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.558+0000 7f14874e0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1480121f00 con 0x7f14800a1340 2023-10-03T10:15:11.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.558+0000 7f14874e0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14800a4f40 con 0x7f14800a0a30 2023-10-03T10:15:11.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.558+0000 7f148527c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f14800a0a30 0x7f148011e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:11.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.558+0000 7f148527c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f14800a0a30 0x7f148011e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38792/0 (socket says 172.21.15.3:38792) 2023-10-03T10:15:11.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.558+0000 7f148527c700 1 -- 172.21.15.3:0/1443493810 learned_addr learned my addr 172.21.15.3:0/1443493810 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:11.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.558+0000 7f1484a7b700 1 --2- 172.21.15.3:0/1443493810 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f14800a1340 0x7f148011ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:11.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.558+0000 7f1485a7d700 1 --2- 172.21.15.3:0/1443493810 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1480134170 0x7f148011f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:11.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.559+0000 7f148527c700 1 -- 172.21.15.3:0/1443493810 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f14800a1340 msgr2=0x7f148011ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:11.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.559+0000 7f148527c700 1 --2- 172.21.15.3:0/1443493810 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f14800a1340 0x7f148011ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.559+0000 7f148527c700 1 -- 172.21.15.3:0/1443493810 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1480134170 msgr2=0x7f148011f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:11.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.559+0000 7f148527c700 1 --2- 172.21.15.3:0/1443493810 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1480134170 0x7f148011f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.559+0000 7f148527c700 1 -- 172.21.15.3:0/1443493810 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1478009a70 con 0x7f14800a0a30 2023-10-03T10:15:11.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.559+0000 7f148527c700 1 --2- 172.21.15.3:0/1443493810 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f14800a0a30 0x7f148011e750 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f147c0090b0 tx=0x7f147c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:11.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.559+0000 7f1475ffb700 1 -- 172.21.15.3:0/1443493810 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c003d80 con 0x7f14800a0a30 2023-10-03T10:15:11.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.559+0000 7f1475ffb700 1 -- 172.21.15.3:0/1443493810 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147c01f070 con 0x7f14800a0a30 2023-10-03T10:15:11.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.559+0000 7f1475ffb700 1 -- 172.21.15.3:0/1443493810 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c011870 con 0x7f14800a0a30 2023-10-03T10:15:11.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.559+0000 7f14874e0700 1 -- 172.21.15.3:0/1443493810 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f147c0094a0 con 0x7f14800a0a30 2023-10-03T10:15:11.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.559+0000 7f14874e0700 1 -- 172.21.15.3:0/1443493810 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14800a25e0 con 0x7f14800a0a30 2023-10-03T10:15:11.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.561+0000 7f14874e0700 1 -- 172.21.15.3:0/1443493810 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1480121ba0 con 0x7f14800a0a30 2023-10-03T10:15:11.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.562+0000 7f1475ffb700 1 -- 172.21.15.3:0/1443493810 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f147c021030 con 0x7f14800a0a30 2023-10-03T10:15:11.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.562+0000 7f1475ffb700 1 -- 172.21.15.3:0/1443493810 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f147c040ba0 con 0x7f14800a0a30 2023-10-03T10:15:11.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.566+0000 7f1475ffb700 1 -- 172.21.15.3:0/1443493810 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f147c0239c0 con 0x7f14800a0a30 2023-10-03T10:15:11.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.595+0000 7f340252c700 1 -- 172.21.15.3:0/1003157553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f33fc06b250 con 0x7f33fc1345a0 2023-10-03T10:15:11.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.596+0000 7f33f8ff9700 1 -- 172.21.15.3:0/1003157553 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f33f0017070 con 0x7f33fc1345a0 2023-10-03T10:15:11.596 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:11.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.598+0000 7f340252c700 1 -- 172.21.15.3:0/1003157553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33fc1345a0 msgr2=0x7f33fc127340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:11.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.598+0000 7f340252c700 1 --2- 172.21.15.3:0/1003157553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33fc1345a0 0x7f33fc127340 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f33f000e9b0 tx=0x7f33f000ecc0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.598+0000 7f340252c700 1 -- 172.21.15.3:0/1003157553 shutdown_connections 2023-10-03T10:15:11.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.598+0000 7f340252c700 1 --2- 172.21.15.3:0/1003157553 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33fc136ec0 0x7f33fc127880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.598+0000 7f340252c700 1 --2- 172.21.15.3:0/1003157553 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f33fc0984f0 0x7f33fc126e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.598+0000 7f340252c700 1 --2- 172.21.15.3:0/1003157553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33fc1345a0 0x7f33fc127340 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.598+0000 7f340252c700 1 -- 172.21.15.3:0/1003157553 >> 172.21.15.3:0/1003157553 conn(0x7f33fc09f470 msgr2=0x7f33fc09c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:11.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.598+0000 7f340252c700 1 -- 172.21.15.3:0/1003157553 shutdown_connections 2023-10-03T10:15:11.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.598+0000 7f340252c700 1 -- 172.21.15.3:0/1003157553 wait complete. 2023-10-03T10:15:11.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.606+0000 7feaca6d0700 1 Processor -- start 2023-10-03T10:15:11.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.606+0000 7feaca6d0700 1 -- start start 2023-10-03T10:15:11.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.607+0000 7feaca6d0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feac4133540 0x7feac4135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:11.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.607+0000 7feaca6d0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feac4135e60 0x7feac4138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:11.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.607+0000 7feaca6d0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac40716a0 con 0x7feac4130540 2023-10-03T10:15:11.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.607+0000 7feaca6d0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac4071290 con 0x7feac4133540 2023-10-03T10:15:11.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.607+0000 7feaca6d0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feac4130340 con 0x7feac4135e60 2023-10-03T10:15:11.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.607+0000 7feac3fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7feac4130540 0x7feac4130910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47144/0 (socket says 172.21.15.3:47144) 2023-10-03T10:15:11.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.607+0000 7feac8c6d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feac4135e60 0x7feac4138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:11.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.607+0000 7feac3fff700 1 -- 172.21.15.3:0/3983554493 learned_addr learned my addr 172.21.15.3:0/3983554493 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:11.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.607+0000 7feac37fe700 1 --2- 172.21.15.3:0/3983554493 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feac4133540 0x7feac4135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:11.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.607+0000 7feac8c6d700 1 -- 172.21.15.3:0/3983554493 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feac4133540 msgr2=0x7feac4135920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:11.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.607+0000 7feac8c6d700 1 --2- 172.21.15.3:0/3983554493 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feac4133540 0x7feac4135920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.607+0000 7feac8c6d700 1 -- 172.21.15.3:0/3983554493 >> v1:172.21.15.3:6789/0 conn(0x7feac4130540 legacy=0x7feac4130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:11.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.607+0000 7feac2ffd700 1 -- 172.21.15.3:0/3983554493 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1385777957 0 0) 0x7feac40716a0 con 0x7feac4130540 2023-10-03T10:15:11.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.607+0000 7feac8c6d700 1 -- 172.21.15.3:0/3983554493 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac4070b80 con 0x7feac4135e60 2023-10-03T10:15:11.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.608+0000 7feac8c6d700 1 --2- 172.21.15.3:0/3983554493 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feac4135e60 0x7feac4138240 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7feabc009dc0 tx=0x7feabc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8e9c19cb1a86b50 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:11.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.609+0000 7feac2ffd700 1 -- 172.21.15.3:0/3983554493 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc00e070 con 0x7feac4135e60 2023-10-03T10:15:11.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.609+0000 7feac2ffd700 1 -- 172.21.15.3:0/3983554493 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feabc00bb20 con 0x7feac4135e60 2023-10-03T10:15:11.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.609+0000 7feac2ffd700 1 -- 172.21.15.3:0/3983554493 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc012430 con 0x7feac4135e60 2023-10-03T10:15:11.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.609+0000 7feaca6d0700 1 -- 172.21.15.3:0/3983554493 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feac4135e60 msgr2=0x7feac4138240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:11.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.609+0000 7feaca6d0700 1 --2- 172.21.15.3:0/3983554493 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feac4135e60 0x7feac4138240 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7feabc009dc0 tx=0x7feabc00b330 comp rx=0 tx=0).stop 2023-10-03T10:15:11.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.610+0000 7feaca6d0700 1 -- 172.21.15.3:0/3983554493 shutdown_connections 2023-10-03T10:15:11.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.610+0000 7feaca6d0700 1 --2- 172.21.15.3:0/3983554493 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feac4135e60 0x7feac4138240 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.610+0000 7feaca6d0700 1 --2- 172.21.15.3:0/3983554493 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feac4133540 0x7feac4135920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.610+0000 7feaca6d0700 1 -- 172.21.15.3:0/3983554493 >> 172.21.15.3:0/3983554493 conn(0x7feac40a2aa0 msgr2=0x7feac40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:11.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.610+0000 7feaca6d0700 1 -- 172.21.15.3:0/3983554493 shutdown_connections 2023-10-03T10:15:11.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.610+0000 7feaca6d0700 1 -- 172.21.15.3:0/3983554493 wait complete. 2023-10-03T10:15:11.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.611+0000 7feaca6d0700 1 Processor -- start 2023-10-03T10:15:11.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.611+0000 7feaca6d0700 1 -- start start 2023-10-03T10:15:11.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.611+0000 7feaca6d0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feac4130540 0x7feac4136230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:11.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.611+0000 7feaca6d0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feac4133540 0x7feac4136770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:11.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.611+0000 7feaca6d0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feac4135e60 0x7feac4136cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:11.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.611+0000 7feaca6d0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac40718e0 con 0x7feac4133540 2023-10-03T10:15:11.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.611+0000 7feaca6d0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac409aca0 con 0x7feac4135e60 2023-10-03T10:15:11.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.611+0000 7feaca6d0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feac4138ef0 con 0x7feac4130540 2023-10-03T10:15:11.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.612+0000 7feac3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feac4130540 0x7feac4136230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:11.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.612+0000 7feac3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feac4130540 0x7feac4136230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:38806/0 (socket says 172.21.15.3:38806) 2023-10-03T10:15:11.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.612+0000 7feac3fff700 1 -- 172.21.15.3:0/3406473270 learned_addr learned my addr 172.21.15.3:0/3406473270 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:11.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.612+0000 7feac37fe700 1 --2- 172.21.15.3:0/3406473270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feac4133540 0x7feac4136770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:11.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.612+0000 7feac3fff700 1 -- 172.21.15.3:0/3406473270 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feac4135e60 msgr2=0x7feac4136cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:11.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.612+0000 7feac3fff700 1 --2- 172.21.15.3:0/3406473270 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feac4135e60 0x7feac4136cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.612+0000 7feac3fff700 1 -- 172.21.15.3:0/3406473270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feac4133540 msgr2=0x7feac4136770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:11.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.612+0000 7feac3fff700 1 --2- 172.21.15.3:0/3406473270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feac4133540 0x7feac4136770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.612+0000 7feac3fff700 1 -- 172.21.15.3:0/3406473270 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feabc009a70 con 0x7feac4130540 2023-10-03T10:15:11.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.612+0000 7feac37fe700 1 --2- 172.21.15.3:0/3406473270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feac4133540 0x7feac4136770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:11.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.612+0000 7feac3fff700 1 --2- 172.21.15.3:0/3406473270 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feac4130540 0x7feac4136230 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7feab400c520 tx=0x7feab400c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:11.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.612+0000 7feac0ff9700 1 -- 172.21.15.3:0/3406473270 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4004c80 con 0x7feac4130540 2023-10-03T10:15:11.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.613+0000 7feac0ff9700 1 -- 172.21.15.3:0/3406473270 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab4004de0 con 0x7feac4130540 2023-10-03T10:15:11.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.613+0000 7feac0ff9700 1 -- 172.21.15.3:0/3406473270 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4007310 con 0x7feac4130540 2023-10-03T10:15:11.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.613+0000 7feaca6d0700 1 -- 172.21.15.3:0/3406473270 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac4138690 con 0x7feac4130540 2023-10-03T10:15:11.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.613+0000 7feaca6d0700 1 -- 172.21.15.3:0/3406473270 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac41391b0 con 0x7feac4130540 2023-10-03T10:15:11.615 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:11.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.614+0000 7feaca6d0700 1 -- 172.21.15.3:0/3406473270 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac41381f0 con 0x7feac4130540 2023-10-03T10:15:11.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.614+0000 7feac0ff9700 1 -- 172.21.15.3:0/3406473270 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feab4007470 con 0x7feac4130540 2023-10-03T10:15:11.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.614+0000 7feac0ff9700 1 -- 172.21.15.3:0/3406473270 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7feab403f3b0 con 0x7feac4130540 2023-10-03T10:15:11.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.624+0000 7feac0ff9700 1 -- 172.21.15.3:0/3406473270 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feab40106d0 con 0x7feac4130540 2023-10-03T10:15:11.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.711+0000 7f8c27cdd700 1 -- 172.21.15.3:0/3886629730 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8c200747f0 con 0x7f8c20130750 2023-10-03T10:15:11.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.711+0000 7f8c167fc700 1 -- 172.21.15.3:0/3886629730 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8c180133e0 con 0x7f8c20130750 2023-10-03T10:15:11.711 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:11.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.713+0000 7f8c27cdd700 1 -- 172.21.15.3:0/3886629730 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c20130750 msgr2=0x7f8c201368c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:11.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.713+0000 7f8c27cdd700 1 --2- 172.21.15.3:0/3886629730 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c20130750 0x7f8c201368c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800ef90 tx=0x7f8c1800c3a0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.713+0000 7f8c27cdd700 1 -- 172.21.15.3:0/3886629730 shutdown_connections 2023-10-03T10:15:11.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.713+0000 7f8c27cdd700 1 --2- 172.21.15.3:0/3886629730 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c20130750 0x7f8c201368c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.713+0000 7f8c27cdd700 1 --2- 172.21.15.3:0/3886629730 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c2009d6f0 0x7f8c20135e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.713+0000 7f8c27cdd700 1 --2- 172.21.15.3:0/3886629730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c2009e250 0x7f8c20136380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.714+0000 7f8c27cdd700 1 -- 172.21.15.3:0/3886629730 >> 172.21.15.3:0/3886629730 conn(0x7f8c2009f470 msgr2=0x7f8c200717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:11.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.714+0000 7f8c27cdd700 1 -- 172.21.15.3:0/3886629730 shutdown_connections 2023-10-03T10:15:11.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.714+0000 7f8c27cdd700 1 -- 172.21.15.3:0/3886629730 wait complete. 2023-10-03T10:15:11.728 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:11.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.787+0000 7f14874e0700 1 -- 172.21.15.3:0/1443493810 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1480121ba0 con 0x7f14800a0a30 2023-10-03T10:15:11.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.787+0000 7f1475ffb700 1 -- 172.21.15.3:0/1443493810 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1480121ba0 con 0x7f14800a0a30 2023-10-03T10:15:11.787 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:11.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.790+0000 7f14874e0700 1 -- 172.21.15.3:0/1443493810 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f14800a0a30 msgr2=0x7f148011e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:11.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.790+0000 7f14874e0700 1 --2- 172.21.15.3:0/1443493810 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f14800a0a30 0x7f148011e750 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f147c0090b0 tx=0x7f147c004750 comp rx=0 tx=0).stop 2023-10-03T10:15:11.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.790+0000 7f14874e0700 1 -- 172.21.15.3:0/1443493810 shutdown_connections 2023-10-03T10:15:11.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.790+0000 7f14874e0700 1 --2- 172.21.15.3:0/1443493810 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1480134170 0x7f148011f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.790+0000 7f14874e0700 1 --2- 172.21.15.3:0/1443493810 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f14800a0a30 0x7f148011e750 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.790+0000 7f14874e0700 1 --2- 172.21.15.3:0/1443493810 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f14800a1340 0x7f148011ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.790+0000 7f14874e0700 1 -- 172.21.15.3:0/1443493810 >> 172.21.15.3:0/1443493810 conn(0x7f14800a6550 msgr2=0x7f148009c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:11.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.790+0000 7f14874e0700 1 -- 172.21.15.3:0/1443493810 shutdown_connections 2023-10-03T10:15:11.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.790+0000 7f14874e0700 1 -- 172.21.15.3:0/1443493810 wait complete. 2023-10-03T10:15:11.803 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:11.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.833+0000 7feaca6d0700 1 -- 172.21.15.3:0/3406473270 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7feac4070ea0 con 0x7feac4130540 2023-10-03T10:15:11.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.833+0000 7feac0ff9700 1 -- 172.21.15.3:0/3406473270 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feab4011360 con 0x7feac4130540 2023-10-03T10:15:11.833 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:11.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.835+0000 7feaca6d0700 1 -- 172.21.15.3:0/3406473270 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feac4130540 msgr2=0x7feac4136230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:11.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.835+0000 7feaca6d0700 1 --2- 172.21.15.3:0/3406473270 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feac4130540 0x7feac4136230 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7feab400c520 tx=0x7feab400c830 comp rx=0 tx=0).stop 2023-10-03T10:15:11.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.835+0000 7feaca6d0700 1 -- 172.21.15.3:0/3406473270 shutdown_connections 2023-10-03T10:15:11.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.835+0000 7feaca6d0700 1 --2- 172.21.15.3:0/3406473270 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feac4135e60 0x7feac4136cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.835+0000 7feaca6d0700 1 --2- 172.21.15.3:0/3406473270 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feac4130540 0x7feac4136230 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.835+0000 7feaca6d0700 1 --2- 172.21.15.3:0/3406473270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feac4133540 0x7feac4136770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:11.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.835+0000 7feaca6d0700 1 -- 172.21.15.3:0/3406473270 >> 172.21.15.3:0/3406473270 conn(0x7feac40a2aa0 msgr2=0x7feac4134fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:11.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.836+0000 7feaca6d0700 1 -- 172.21.15.3:0/3406473270 shutdown_connections 2023-10-03T10:15:11.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:11.836+0000 7feaca6d0700 1 -- 172.21.15.3:0/3406473270 wait complete. 2023-10-03T10:15:11.849 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:12.616 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:12.729 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:12.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.763+0000 7f25a6ca9700 1 Processor -- start 2023-10-03T10:15:12.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.763+0000 7f25a6ca9700 1 -- start start 2023-10-03T10:15:12.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.764+0000 7f25a6ca9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f25a0130540 0x7f25a0130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:12.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.764+0000 7f25a6ca9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25a006fcf0 0x7f25a0138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:12.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.764+0000 7f25a6ca9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a00716a0 con 0x7f25a0130e50 2023-10-03T10:15:12.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.764+0000 7f25a6ca9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a009a390 con 0x7f25a0130540 2023-10-03T10:15:12.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.764+0000 7f25a6ca9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25a0071290 con 0x7f25a006fcf0 2023-10-03T10:15:12.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.764+0000 7f25a4a45700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f25a0130540 0x7f25a0130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:12.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.764+0000 7f25a4a45700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f25a0130540 0x7f25a0130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57398/0 (socket says 172.21.15.3:57398) 2023-10-03T10:15:12.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.764+0000 7f25a5246700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25a006fcf0 0x7f25a0138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:12.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.764+0000 7f25a4a45700 1 -- 172.21.15.3:0/3213467138 learned_addr learned my addr 172.21.15.3:0/3213467138 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:12.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.764+0000 7f25a4a45700 1 -- 172.21.15.3:0/3213467138 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25a006fcf0 msgr2=0x7f25a0138570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:12.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.764+0000 7f25a4a45700 1 --2- 172.21.15.3:0/3213467138 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25a006fcf0 0x7f25a0138570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:12.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.764+0000 7f25a4a45700 1 -- 172.21.15.3:0/3213467138 >> v1:172.21.15.3:6789/0 conn(0x7f25a0130e50 legacy=0x7f25a006f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:12.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.764+0000 7f25a4a45700 1 -- 172.21.15.3:0/3213467138 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a0130340 con 0x7f25a0130540 2023-10-03T10:15:12.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.764+0000 7f25a5246700 1 --2- 172.21.15.3:0/3213467138 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25a006fcf0 0x7f25a0138570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:12.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.765+0000 7f25a4a45700 1 --2- 172.21.15.3:0/3213467138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a0130540 0x7f25a0130910 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f25940098f0 tx=0x7f2594009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=429f71f9d96ae98e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:12.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.765+0000 7f259f7fe700 1 -- 172.21.15.3:0/3213467138 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2594011830 con 0x7f25a0130540 2023-10-03T10:15:12.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.765+0000 7f259f7fe700 1 -- 172.21.15.3:0/3213467138 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2594011990 con 0x7f25a0130540 2023-10-03T10:15:12.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.765+0000 7f259f7fe700 1 -- 172.21.15.3:0/3213467138 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2594011d20 con 0x7f25a0130540 2023-10-03T10:15:12.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.765+0000 7f25a6ca9700 1 -- 172.21.15.3:0/3213467138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a0130540 msgr2=0x7f25a0130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:12.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.765+0000 7f25a6ca9700 1 --2- 172.21.15.3:0/3213467138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a0130540 0x7f25a0130910 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f25940098f0 tx=0x7f2594009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:12.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.766+0000 7f25a6ca9700 1 -- 172.21.15.3:0/3213467138 shutdown_connections 2023-10-03T10:15:12.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.766+0000 7f25a6ca9700 1 --2- 172.21.15.3:0/3213467138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a0130540 0x7f25a0130910 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:12.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.766+0000 7f25a6ca9700 1 --2- 172.21.15.3:0/3213467138 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25a006fcf0 0x7f25a0138570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:12.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.766+0000 7f25a6ca9700 1 -- 172.21.15.3:0/3213467138 >> 172.21.15.3:0/3213467138 conn(0x7f25a00a2aa0 msgr2=0x7f25a00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:12.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.766+0000 7f25a6ca9700 1 -- 172.21.15.3:0/3213467138 shutdown_connections 2023-10-03T10:15:12.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.766+0000 7f25a6ca9700 1 -- 172.21.15.3:0/3213467138 wait complete. 2023-10-03T10:15:12.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.767+0000 7f25a6ca9700 1 Processor -- start 2023-10-03T10:15:12.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.767+0000 7f25a6ca9700 1 -- start start 2023-10-03T10:15:12.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.768+0000 7f25a6ca9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25a006fcf0 0x7f25a01b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:12.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.768+0000 7f25a6ca9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25a0130540 0x7f25a01b8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:12.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.768+0000 7f25a6ca9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a0130e50 0x7f25a01b86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:12.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.768+0000 7f25a6ca9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a011aad0 con 0x7f25a0130e50 2023-10-03T10:15:12.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.768+0000 7f25a6ca9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a00718e0 con 0x7f25a006fcf0 2023-10-03T10:15:12.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.768+0000 7f25a6ca9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25a0119270 con 0x7f25a0130540 2023-10-03T10:15:12.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.768+0000 7f25a4a45700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25a006fcf0 0x7f25a01b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:12.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.768+0000 7f259ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25a0130540 0x7f25a01b8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:12.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.768+0000 7f25a4a45700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25a006fcf0 0x7f25a01b7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51514/0 (socket says 172.21.15.3:51514) 2023-10-03T10:15:12.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.768+0000 7f25a4a45700 1 -- 172.21.15.3:0/1321965345 learned_addr learned my addr 172.21.15.3:0/1321965345 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:12.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.768+0000 7f25a5246700 1 --2- 172.21.15.3:0/1321965345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a0130e50 0x7f25a01b86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:12.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.768+0000 7f259ffff700 1 -- 172.21.15.3:0/1321965345 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25a006fcf0 msgr2=0x7f25a01b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:12.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.768+0000 7f259ffff700 1 --2- 172.21.15.3:0/1321965345 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25a006fcf0 0x7f25a01b7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:12.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.768+0000 7f259ffff700 1 -- 172.21.15.3:0/1321965345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a0130e50 msgr2=0x7f25a01b86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:12.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.768+0000 7f259ffff700 1 --2- 172.21.15.3:0/1321965345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a0130e50 0x7f25a01b86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:12.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.768+0000 7f259ffff700 1 -- 172.21.15.3:0/1321965345 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2598009a70 con 0x7f25a0130540 2023-10-03T10:15:12.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.769+0000 7f25a4a45700 1 --2- 172.21.15.3:0/1321965345 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25a006fcf0 0x7f25a01b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:12.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.769+0000 7f259ffff700 1 --2- 172.21.15.3:0/1321965345 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25a0130540 0x7f25a01b8160 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f258c000f60 tx=0x7f258c00c730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:12.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.769+0000 7f259d7fa700 1 -- 172.21.15.3:0/1321965345 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c0052d0 con 0x7f25a0130540 2023-10-03T10:15:12.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.769+0000 7f259d7fa700 1 -- 172.21.15.3:0/1321965345 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258c005430 con 0x7f25a0130540 2023-10-03T10:15:12.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.769+0000 7f25a6ca9700 1 -- 172.21.15.3:0/1321965345 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2594009550 con 0x7f25a0130540 2023-10-03T10:15:12.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.769+0000 7f259d7fa700 1 -- 172.21.15.3:0/1321965345 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c0038c0 con 0x7f25a0130540 2023-10-03T10:15:12.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.769+0000 7f25a6ca9700 1 -- 172.21.15.3:0/1321965345 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a011b070 con 0x7f25a0130540 2023-10-03T10:15:12.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.770+0000 7f259d7fa700 1 -- 172.21.15.3:0/1321965345 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f258c003a20 con 0x7f25a0130540 2023-10-03T10:15:12.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.770+0000 7f25a6ca9700 1 -- 172.21.15.3:0/1321965345 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a011ac40 con 0x7f25a0130540 2023-10-03T10:15:12.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.770+0000 7f259d7fa700 1 -- 172.21.15.3:0/1321965345 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f258c03e9e0 con 0x7f25a0130540 2023-10-03T10:15:12.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.775+0000 7f259d7fa700 1 -- 172.21.15.3:0/1321965345 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f258c011590 con 0x7f25a0130540 2023-10-03T10:15:12.805 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:12.850 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:12.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.891+0000 7f1cfc95f700 1 Processor -- start 2023-10-03T10:15:12.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.891+0000 7f1cfc95f700 1 -- start start 2023-10-03T10:15:12.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.892+0000 7f1cfc95f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cf809e1d0 0x7f1cf8138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:12.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.892+0000 7f1cfc95f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1cf8139170 0x7f1cf813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:12.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.892+0000 7f1cfc95f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cf809b120 con 0x7f1cf809d6f0 2023-10-03T10:15:12.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.892+0000 7f1cfc95f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cf806b460 con 0x7f1cf8139170 2023-10-03T10:15:12.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.892+0000 7f1cfc95f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf809a7e0 con 0x7f1cf809e1d0 2023-10-03T10:15:12.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.892+0000 7f1cf5d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cf809e1d0 0x7f1cf8138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:12.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.892+0000 7f1cf5d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cf809e1d0 0x7f1cf8138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51516/0 (socket says 172.21.15.3:51516) 2023-10-03T10:15:12.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.892+0000 7f1cf5d9b700 1 -- 172.21.15.3:0/3588561441 learned_addr learned my addr 172.21.15.3:0/3588561441 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:12.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.893+0000 7f1cf6d9d700 1 --2- 172.21.15.3:0/3588561441 >> v2:172.21.15.3:3300/0 conn(0x7f1cf8139170 0x7f1cf813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:12.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.893+0000 7f1cf5d9b700 1 -- 172.21.15.3:0/3588561441 >> v2:172.21.15.3:3300/0 conn(0x7f1cf8139170 msgr2=0x7f1cf813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:12.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.893+0000 7f1cf5d9b700 1 --2- 172.21.15.3:0/3588561441 >> v2:172.21.15.3:3300/0 conn(0x7f1cf8139170 0x7f1cf813b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:12.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.893+0000 7f1cf5d9b700 1 -- 172.21.15.3:0/3588561441 >> v1:172.21.15.3:6789/0 conn(0x7f1cf809d6f0 legacy=0x7f1cf809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:12.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.893+0000 7f1cf5d9b700 1 -- 172.21.15.3:0/3588561441 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf809a500 con 0x7f1cf809e1d0 2023-10-03T10:15:12.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.893+0000 7f1cf6d9d700 1 --2- 172.21.15.3:0/3588561441 >> v2:172.21.15.3:3300/0 conn(0x7f1cf8139170 0x7f1cf813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:15:12.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.893+0000 7f1cf5d9b700 1 --2- 172.21.15.3:0/3588561441 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cf809e1d0 0x7f1cf8138c30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009920 tx=0x7f1cec009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b845cf6efd568d4b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:12.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.893+0000 7f1cf559a700 1 -- 172.21.15.3:0/3588561441 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec00ba20 con 0x7f1cf809e1d0 2023-10-03T10:15:12.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.893+0000 7f1cf559a700 1 -- 172.21.15.3:0/3588561441 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cec00bb80 con 0x7f1cf809e1d0 2023-10-03T10:15:12.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.893+0000 7f1cf559a700 1 -- 172.21.15.3:0/3588561441 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec009e70 con 0x7f1cf809e1d0 2023-10-03T10:15:12.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.894+0000 7f1cfc95f700 1 -- 172.21.15.3:0/3588561441 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cf809e1d0 msgr2=0x7f1cf8138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:12.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.894+0000 7f1cfc95f700 1 --2- 172.21.15.3:0/3588561441 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cf809e1d0 0x7f1cf8138c30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009920 tx=0x7f1cec009c30 comp rx=0 tx=0).stop 2023-10-03T10:15:12.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.895+0000 7f1cfc95f700 1 -- 172.21.15.3:0/3588561441 shutdown_connections 2023-10-03T10:15:12.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.895+0000 7f1cfc95f700 1 --2- 172.21.15.3:0/3588561441 >> v2:172.21.15.3:3300/0 conn(0x7f1cf8139170 0x7f1cf813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:12.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.895+0000 7f1cfc95f700 1 --2- 172.21.15.3:0/3588561441 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cf809e1d0 0x7f1cf8138c30 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:12.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.895+0000 7f1cfc95f700 1 -- 172.21.15.3:0/3588561441 >> 172.21.15.3:0/3588561441 conn(0x7f1cf8066f50 msgr2=0x7f1cf809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:12.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.895+0000 7f1cfc95f700 1 -- 172.21.15.3:0/3588561441 shutdown_connections 2023-10-03T10:15:12.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.895+0000 7f1cfc95f700 1 -- 172.21.15.3:0/3588561441 wait complete. 2023-10-03T10:15:12.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.896+0000 7f1cfc95f700 1 Processor -- start 2023-10-03T10:15:12.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.896+0000 7f1cfc95f700 1 -- start start 2023-10-03T10:15:12.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.896+0000 7f1cfc95f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1cf809d6f0 0x7f1cf81c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:12.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.896+0000 7f1cfc95f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cf809e1d0 0x7f1cf81c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:12.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.896+0000 7f1cfc95f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cf8139170 0x7f1cf811e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:12.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.896+0000 7f1cfc95f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf81220a0 con 0x7f1cf8139170 2023-10-03T10:15:12.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.896+0000 7f1cfc95f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf8122280 con 0x7f1cf809e1d0 2023-10-03T10:15:12.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.896+0000 7f1cfc95f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf81be250 con 0x7f1cf809d6f0 2023-10-03T10:15:12.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.897+0000 7f1cf5d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cf809e1d0 0x7f1cf81c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:12.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.897+0000 7f1cf5d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cf809e1d0 0x7f1cf81c8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51532/0 (socket says 172.21.15.3:51532) 2023-10-03T10:15:12.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.897+0000 7f1cf5d9b700 1 -- 172.21.15.3:0/2716573710 learned_addr learned my addr 172.21.15.3:0/2716573710 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:12.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.897+0000 7f1cf5d9b700 1 -- 172.21.15.3:0/2716573710 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1cf809d6f0 msgr2=0x7f1cf81c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:12.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.897+0000 7f1cf5d9b700 1 --2- 172.21.15.3:0/2716573710 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1cf809d6f0 0x7f1cf81c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:12.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.897+0000 7f1cf5d9b700 1 -- 172.21.15.3:0/2716573710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cf8139170 msgr2=0x7f1cf811e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:12.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.897+0000 7f1cf5d9b700 1 --2- 172.21.15.3:0/2716573710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cf8139170 0x7f1cf811e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:12.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.897+0000 7f1cf5d9b700 1 -- 172.21.15.3:0/2716573710 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cec009550 con 0x7f1cf809e1d0 2023-10-03T10:15:12.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.897+0000 7f1cf5d9b700 1 --2- 172.21.15.3:0/2716573710 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cf809e1d0 0x7f1cf81c8bf0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009870 tx=0x7f1cec004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:12.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.897+0000 7f1ce6ffd700 1 -- 172.21.15.3:0/2716573710 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec004400 con 0x7f1cf809e1d0 2023-10-03T10:15:12.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.897+0000 7f1ce6ffd700 1 -- 172.21.15.3:0/2716573710 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cec01f070 con 0x7f1cf809e1d0 2023-10-03T10:15:12.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.898+0000 7f1cfc95f700 1 -- 172.21.15.3:0/2716573710 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf80a29f0 con 0x7f1cf809e1d0 2023-10-03T10:15:12.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.898+0000 7f1ce6ffd700 1 -- 172.21.15.3:0/2716573710 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0038e0 con 0x7f1cf809e1d0 2023-10-03T10:15:12.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.898+0000 7f1cfc95f700 1 -- 172.21.15.3:0/2716573710 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf809c710 con 0x7f1cf809e1d0 2023-10-03T10:15:12.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.899+0000 7f1cfc95f700 1 -- 172.21.15.3:0/2716573710 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cf8074ea0 con 0x7f1cf809e1d0 2023-10-03T10:15:12.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.899+0000 7f1ce6ffd700 1 -- 172.21.15.3:0/2716573710 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1cec021030 con 0x7f1cf809e1d0 2023-10-03T10:15:12.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.900+0000 7f1ce6ffd700 1 -- 172.21.15.3:0/2716573710 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1cec040b60 con 0x7f1cf809e1d0 2023-10-03T10:15:12.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.909+0000 7f1ce6ffd700 1 -- 172.21.15.3:0/2716573710 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1cec0199a0 con 0x7f1cf809e1d0 2023-10-03T10:15:12.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.969+0000 7f44c35a9700 1 Processor -- start 2023-10-03T10:15:12.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.969+0000 7f44c35a9700 1 -- start start 2023-10-03T10:15:12.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.970+0000 7f44c35a9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44bc130670 0x7f44bc130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:12.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.970+0000 7f44c35a9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44bc06f680 0x7f44bc06fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:12.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.970+0000 7f44c35a9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44bc06ffd0 0x7f44bc137530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:12.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.970+0000 7f44c35a9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44bc09a350 con 0x7f44bc130670 2023-10-03T10:15:12.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.970+0000 7f44c35a9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc130260 con 0x7f44bc06f680 2023-10-03T10:15:12.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.970+0000 7f44c35a9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc071ff0 con 0x7f44bc06ffd0 2023-10-03T10:15:12.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.970+0000 7f44c0b44700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44bc06f680 0x7f44bc06fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:12.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.970+0000 7f44c1345700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44bc130670 0x7f44bc130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:12.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.970+0000 7f44c1345700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44bc130670 0x7f44bc130a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57446/0 (socket says 172.21.15.3:57446) 2023-10-03T10:15:12.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.970+0000 7f44c0b44700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44bc06f680 0x7f44bc06fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51546/0 (socket says 172.21.15.3:51546) 2023-10-03T10:15:12.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.970+0000 7f44c1345700 1 -- 172.21.15.3:0/2652630808 learned_addr learned my addr 172.21.15.3:0/2652630808 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:12.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.970+0000 7f44c1b46700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44bc06ffd0 0x7f44bc137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:12.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.971+0000 7f44c1345700 1 -- 172.21.15.3:0/2652630808 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44bc06ffd0 msgr2=0x7f44bc137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:12.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.971+0000 7f44c1345700 1 --2- 172.21.15.3:0/2652630808 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44bc06ffd0 0x7f44bc137530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:12.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.971+0000 7f44c1345700 1 -- 172.21.15.3:0/2652630808 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44bc06f680 msgr2=0x7f44bc06fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:12.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.971+0000 7f44c1345700 1 --2- 172.21.15.3:0/2652630808 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44bc06f680 0x7f44bc06fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:12.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.971+0000 7f44c1345700 1 -- 172.21.15.3:0/2652630808 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44bc09ecf0 con 0x7f44bc130670 2023-10-03T10:15:12.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.971+0000 7f44c0b44700 1 --2- 172.21.15.3:0/2652630808 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44bc06f680 0x7f44bc06fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:12.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.971+0000 7f44c1345700 1 --2- 172.21.15.3:0/2652630808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44bc130670 0x7f44bc130a40 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f44b80098f0 tx=0x7f44b8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2755c8dac4948a4e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:12.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.971+0000 7f44b3fff700 1 -- 172.21.15.3:0/2652630808 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b8011830 con 0x7f44bc130670 2023-10-03T10:15:12.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.971+0000 7f44b3fff700 1 -- 172.21.15.3:0/2652630808 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b8011990 con 0x7f44bc130670 2023-10-03T10:15:12.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.971+0000 7f44b3fff700 1 -- 172.21.15.3:0/2652630808 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b8011d20 con 0x7f44bc130670 2023-10-03T10:15:12.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.972+0000 7f44c35a9700 1 -- 172.21.15.3:0/2652630808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44bc130670 msgr2=0x7f44bc130a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:12.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.972+0000 7f44c35a9700 1 --2- 172.21.15.3:0/2652630808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44bc130670 0x7f44bc130a40 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f44b80098f0 tx=0x7f44b8009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:12.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.972+0000 7f44c35a9700 1 -- 172.21.15.3:0/2652630808 shutdown_connections 2023-10-03T10:15:12.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.972+0000 7f44c35a9700 1 --2- 172.21.15.3:0/2652630808 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44bc06ffd0 0x7f44bc137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:12.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.972+0000 7f44c35a9700 1 --2- 172.21.15.3:0/2652630808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44bc130670 0x7f44bc130a40 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:12.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.972+0000 7f44c35a9700 1 --2- 172.21.15.3:0/2652630808 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44bc06f680 0x7f44bc06fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:12.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.972+0000 7f44c35a9700 1 -- 172.21.15.3:0/2652630808 >> 172.21.15.3:0/2652630808 conn(0x7f44bc0a2aa0 msgr2=0x7f44bc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:12.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.973+0000 7f44c35a9700 1 -- 172.21.15.3:0/2652630808 shutdown_connections 2023-10-03T10:15:12.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.973+0000 7f44c35a9700 1 -- 172.21.15.3:0/2652630808 wait complete. 2023-10-03T10:15:12.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.974+0000 7f44c35a9700 1 Processor -- start 2023-10-03T10:15:12.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.974+0000 7f44c35a9700 1 -- start start 2023-10-03T10:15:12.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.974+0000 7f44c35a9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44bc06f680 0x7f44bc11daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:12.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.974+0000 7f44c35a9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44bc06ffd0 0x7f44bc11dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:12.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.974+0000 7f44c35a9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44bc130670 0x7f44bc11e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:12.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.974+0000 7f44c35a9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc1214e0 con 0x7f44bc06ffd0 2023-10-03T10:15:12.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.974+0000 7f44c35a9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc11f5d0 con 0x7f44bc130670 2023-10-03T10:15:12.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.974+0000 7f44c35a9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc09d010 con 0x7f44bc06f680 2023-10-03T10:15:12.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.974+0000 7f44c1b46700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44bc130670 0x7f44bc11e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:12.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.974+0000 7f44c1b46700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44bc130670 0x7f44bc11e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51548/0 (socket says 172.21.15.3:51548) 2023-10-03T10:15:12.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.974+0000 7f44c1b46700 1 -- 172.21.15.3:0/3700542983 learned_addr learned my addr 172.21.15.3:0/3700542983 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:12.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.974+0000 7f44c1345700 1 --2- 172.21.15.3:0/3700542983 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44bc06f680 0x7f44bc11daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:12.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.975+0000 7f44c0b44700 1 --2- 172.21.15.3:0/3700542983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44bc06ffd0 0x7f44bc11dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:12.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.975+0000 7f44c1b46700 1 -- 172.21.15.3:0/3700542983 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44bc06f680 msgr2=0x7f44bc11daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:12.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.975+0000 7f44c1b46700 1 --2- 172.21.15.3:0/3700542983 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44bc06f680 0x7f44bc11daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:12.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.975+0000 7f44c1b46700 1 -- 172.21.15.3:0/3700542983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44bc06ffd0 msgr2=0x7f44bc11dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:12.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.975+0000 7f44c1b46700 1 --2- 172.21.15.3:0/3700542983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44bc06ffd0 0x7f44bc11dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:12.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.975+0000 7f44c1b46700 1 -- 172.21.15.3:0/3700542983 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ac0094a0 con 0x7f44bc130670 2023-10-03T10:15:12.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.975+0000 7f44c1b46700 1 --2- 172.21.15.3:0/3700542983 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44bc130670 0x7f44bc11e520 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f44b400efc0 tx=0x7f44b400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:12.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.975+0000 7f44b1ffb700 1 -- 172.21.15.3:0/3700542983 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4009ea0 con 0x7f44bc130670 2023-10-03T10:15:12.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.975+0000 7f44b1ffb700 1 -- 172.21.15.3:0/3700542983 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b400f040 con 0x7f44bc130670 2023-10-03T10:15:12.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.975+0000 7f44b1ffb700 1 -- 172.21.15.3:0/3700542983 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4014480 con 0x7f44bc130670 2023-10-03T10:15:12.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.975+0000 7f44c35a9700 1 -- 172.21.15.3:0/3700542983 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b8009550 con 0x7f44bc130670 2023-10-03T10:15:12.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.976+0000 7f44c35a9700 1 -- 172.21.15.3:0/3700542983 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44bc11fd90 con 0x7f44bc130670 2023-10-03T10:15:12.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.977+0000 7f44b1ffb700 1 -- 172.21.15.3:0/3700542983 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f44b40145e0 con 0x7f44bc130670 2023-10-03T10:15:12.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.977+0000 7f44b1ffb700 1 -- 172.21.15.3:0/3700542983 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f44b403ff20 con 0x7f44bc130670 2023-10-03T10:15:12.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.977+0000 7f44c35a9700 1 -- 172.21.15.3:0/3700542983 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44bc074ea0 con 0x7f44bc130670 2023-10-03T10:15:12.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.982+0000 7f44b1ffb700 1 -- 172.21.15.3:0/3700542983 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f44b4018070 con 0x7f44bc130670 2023-10-03T10:15:12.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:12.999+0000 7f1547863700 1 Processor -- start 2023-10-03T10:15:13.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.000+0000 7f1547863700 1 -- start start 2023-10-03T10:15:13.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.000+0000 7f1547863700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f154009e1d0 0x7f1540135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:13.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.000+0000 7f1547863700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1540135e50 0x7f1540138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:13.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.000+0000 7f1547863700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f154009f130 con 0x7f154009d6f0 2023-10-03T10:15:13.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.000+0000 7f1547863700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f154009f520 con 0x7f1540135e50 2023-10-03T10:15:13.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.000+0000 7f1547863700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f154009e930 con 0x7f154009e1d0 2023-10-03T10:15:13.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.000+0000 7f15455ff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f154009d6f0 0x7f154009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47176/0 (socket says 172.21.15.3:47176) 2023-10-03T10:15:13.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.000+0000 7f15455ff700 1 -- 172.21.15.3:0/4155638231 learned_addr learned my addr 172.21.15.3:0/4155638231 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:13.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.000+0000 7f1545e00700 1 --2- 172.21.15.3:0/4155638231 >> v2:172.21.15.3:3300/0 conn(0x7f1540135e50 0x7f1540138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:13.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.000+0000 7f1544dfe700 1 --2- 172.21.15.3:0/4155638231 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f154009e1d0 0x7f1540135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:13.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.001+0000 7f1544dfe700 1 -- 172.21.15.3:0/4155638231 >> v2:172.21.15.3:3300/0 conn(0x7f1540135e50 msgr2=0x7f1540138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:13.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.001+0000 7f1537fff700 1 -- 172.21.15.3:0/4155638231 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1394621723 0 0) 0x7f154009f130 con 0x7f154009d6f0 2023-10-03T10:15:13.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.001+0000 7f1544dfe700 1 --2- 172.21.15.3:0/4155638231 >> v2:172.21.15.3:3300/0 conn(0x7f1540135e50 0x7f1540138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:13.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.001+0000 7f1544dfe700 1 -- 172.21.15.3:0/4155638231 >> v1:172.21.15.3:6789/0 conn(0x7f154009d6f0 legacy=0x7f154009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:13.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.001+0000 7f1544dfe700 1 -- 172.21.15.3:0/4155638231 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154009b2d0 con 0x7f154009e1d0 2023-10-03T10:15:13.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.001+0000 7f1544dfe700 1 --2- 172.21.15.3:0/4155638231 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f154009e1d0 0x7f1540135910 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1530009870 tx=0x7f1530009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e633a3e277ce053d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:13.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.001+0000 7f1537fff700 1 -- 172.21.15.3:0/4155638231 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1530010a90 con 0x7f154009e1d0 2023-10-03T10:15:13.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.001+0000 7f1537fff700 1 -- 172.21.15.3:0/4155638231 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1530010bf0 con 0x7f154009e1d0 2023-10-03T10:15:13.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.001+0000 7f1537fff700 1 -- 172.21.15.3:0/4155638231 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1530018440 con 0x7f154009e1d0 2023-10-03T10:15:13.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.002+0000 7f1547863700 1 -- 172.21.15.3:0/4155638231 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f154009e1d0 msgr2=0x7f1540135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:13.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.002+0000 7f1547863700 1 --2- 172.21.15.3:0/4155638231 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f154009e1d0 0x7f1540135910 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1530009870 tx=0x7f1530009b80 comp rx=0 tx=0).stop 2023-10-03T10:15:13.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.002+0000 7f1547863700 1 -- 172.21.15.3:0/4155638231 shutdown_connections 2023-10-03T10:15:13.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.002+0000 7f1547863700 1 --2- 172.21.15.3:0/4155638231 >> v2:172.21.15.3:3300/0 conn(0x7f1540135e50 0x7f1540138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:13.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.002+0000 7f1547863700 1 --2- 172.21.15.3:0/4155638231 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f154009e1d0 0x7f1540135910 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:13.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.002+0000 7f1547863700 1 -- 172.21.15.3:0/4155638231 >> 172.21.15.3:0/4155638231 conn(0x7f1540066f50 msgr2=0x7f154009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:13.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.002+0000 7f1547863700 1 -- 172.21.15.3:0/4155638231 shutdown_connections 2023-10-03T10:15:13.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.003+0000 7f1547863700 1 -- 172.21.15.3:0/4155638231 wait complete. 2023-10-03T10:15:13.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.004+0000 7f1547863700 1 Processor -- start 2023-10-03T10:15:13.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.004+0000 7f1547863700 1 -- start start 2023-10-03T10:15:13.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.004+0000 7f1547863700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f154009d6f0 0x7f154011b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:13.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.004+0000 7f1547863700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f154009e1d0 0x7f154011b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:13.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.004+0000 7f1547863700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1540135e50 0x7f154011bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:13.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.005+0000 7f1547863700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154011ee00 con 0x7f1540135e50 2023-10-03T10:15:13.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.005+0000 7f1547863700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1540131d20 con 0x7f154009d6f0 2023-10-03T10:15:13.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.005+0000 7f1547863700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f154011eb20 con 0x7f154009e1d0 2023-10-03T10:15:13.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.005+0000 7f15455ff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f154009d6f0 0x7f154011b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:13.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.005+0000 7f15455ff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f154009d6f0 0x7f154011b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51562/0 (socket says 172.21.15.3:51562) 2023-10-03T10:15:13.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.005+0000 7f15455ff700 1 -- 172.21.15.3:0/845011303 learned_addr learned my addr 172.21.15.3:0/845011303 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:13.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.005+0000 7f1544dfe700 1 --2- 172.21.15.3:0/845011303 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f154009e1d0 0x7f154011b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:13.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.005+0000 7f1545e00700 1 --2- 172.21.15.3:0/845011303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1540135e50 0x7f154011bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:13.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.005+0000 7f1544dfe700 1 -- 172.21.15.3:0/845011303 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f154009d6f0 msgr2=0x7f154011b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:13.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.005+0000 7f1544dfe700 1 --2- 172.21.15.3:0/845011303 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f154009d6f0 0x7f154011b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:13.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.005+0000 7f1544dfe700 1 -- 172.21.15.3:0/845011303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1540135e50 msgr2=0x7f154011bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:13.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.005+0000 7f1544dfe700 1 --2- 172.21.15.3:0/845011303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1540135e50 0x7f154011bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:13.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.005+0000 7f1544dfe700 1 -- 172.21.15.3:0/845011303 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15300094a0 con 0x7f154009e1d0 2023-10-03T10:15:13.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.005+0000 7f1545e00700 1 --2- 172.21.15.3:0/845011303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1540135e50 0x7f154011bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:15:13.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.005+0000 7f1544dfe700 1 --2- 172.21.15.3:0/845011303 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f154009e1d0 0x7f154011b820 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1530009870 tx=0x7f1530004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:13.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.006+0000 7f1535ffb700 1 -- 172.21.15.3:0/845011303 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1530023620 con 0x7f154009e1d0 2023-10-03T10:15:13.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.006+0000 7f1535ffb700 1 -- 172.21.15.3:0/845011303 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1530023780 con 0x7f154009e1d0 2023-10-03T10:15:13.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.006+0000 7f1547863700 1 -- 172.21.15.3:0/845011303 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154011cef0 con 0x7f154009e1d0 2023-10-03T10:15:13.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.006+0000 7f1535ffb700 1 -- 172.21.15.3:0/845011303 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1530023b10 con 0x7f154009e1d0 2023-10-03T10:15:13.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.006+0000 7f1547863700 1 -- 172.21.15.3:0/845011303 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154009c700 con 0x7f154009e1d0 2023-10-03T10:15:13.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.007+0000 7f1535ffb700 1 -- 172.21.15.3:0/845011303 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f153001f040 con 0x7f154009e1d0 2023-10-03T10:15:13.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.007+0000 7f1547863700 1 -- 172.21.15.3:0/845011303 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1540074ea0 con 0x7f154009e1d0 2023-10-03T10:15:13.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.007+0000 7f1535ffb700 1 -- 172.21.15.3:0/845011303 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1530045070 con 0x7f154009e1d0 2023-10-03T10:15:13.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.017+0000 7f1535ffb700 1 -- 172.21.15.3:0/845011303 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1530023c70 con 0x7f154009e1d0 2023-10-03T10:15:13.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.018+0000 7f25a6ca9700 1 -- 172.21.15.3:0/1321965345 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f25a0074ea0 con 0x7f25a0130540 2023-10-03T10:15:13.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.019+0000 7f259d7fa700 1 -- 172.21.15.3:0/1321965345 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f258c017020 con 0x7f25a0130540 2023-10-03T10:15:13.019 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:13.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.021+0000 7f25a6ca9700 1 -- 172.21.15.3:0/1321965345 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25a0130540 msgr2=0x7f25a01b8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:13.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.021+0000 7f25a6ca9700 1 --2- 172.21.15.3:0/1321965345 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25a0130540 0x7f25a01b8160 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f258c000f60 tx=0x7f258c00c730 comp rx=0 tx=0).stop 2023-10-03T10:15:13.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.021+0000 7f25a6ca9700 1 -- 172.21.15.3:0/1321965345 shutdown_connections 2023-10-03T10:15:13.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.021+0000 7f25a6ca9700 1 --2- 172.21.15.3:0/1321965345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a0130e50 0x7f25a01b86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:13.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.021+0000 7f25a6ca9700 1 --2- 172.21.15.3:0/1321965345 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25a006fcf0 0x7f25a01b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:13.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.021+0000 7f25a6ca9700 1 --2- 172.21.15.3:0/1321965345 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25a0130540 0x7f25a01b8160 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:13.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.021+0000 7f25a6ca9700 1 -- 172.21.15.3:0/1321965345 >> 172.21.15.3:0/1321965345 conn(0x7f25a00a2aa0 msgr2=0x7f25a012d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:13.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.022+0000 7f25a6ca9700 1 -- 172.21.15.3:0/1321965345 shutdown_connections 2023-10-03T10:15:13.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.022+0000 7f25a6ca9700 1 -- 172.21.15.3:0/1321965345 wait complete. 2023-10-03T10:15:13.036 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:13.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.148+0000 7f1cfc95f700 1 -- 172.21.15.3:0/2716573710 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1cf809f420 con 0x7f1cf809e1d0 2023-10-03T10:15:13.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.148+0000 7f1ce6ffd700 1 -- 172.21.15.3:0/2716573710 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1cec032090 con 0x7f1cf809e1d0 2023-10-03T10:15:13.148 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:13.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.150+0000 7f1cfc95f700 1 -- 172.21.15.3:0/2716573710 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cf809e1d0 msgr2=0x7f1cf81c8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:13.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.150+0000 7f1cfc95f700 1 --2- 172.21.15.3:0/2716573710 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cf809e1d0 0x7f1cf81c8bf0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009870 tx=0x7f1cec004750 comp rx=0 tx=0).stop 2023-10-03T10:15:13.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.150+0000 7f1cfc95f700 1 -- 172.21.15.3:0/2716573710 shutdown_connections 2023-10-03T10:15:13.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.150+0000 7f1cfc95f700 1 --2- 172.21.15.3:0/2716573710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cf8139170 0x7f1cf811e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:13.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.150+0000 7f1cfc95f700 1 --2- 172.21.15.3:0/2716573710 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1cf809d6f0 0x7f1cf81c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:13.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.150+0000 7f1cfc95f700 1 --2- 172.21.15.3:0/2716573710 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cf809e1d0 0x7f1cf81c8bf0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:13.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.150+0000 7f1cfc95f700 1 -- 172.21.15.3:0/2716573710 >> 172.21.15.3:0/2716573710 conn(0x7f1cf8066f50 msgr2=0x7f1cf80a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:13.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.151+0000 7f1cfc95f700 1 -- 172.21.15.3:0/2716573710 shutdown_connections 2023-10-03T10:15:13.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.151+0000 7f1cfc95f700 1 -- 172.21.15.3:0/2716573710 wait complete. 2023-10-03T10:15:13.164 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:13.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.196+0000 7f44c35a9700 1 -- 172.21.15.3:0/3700542983 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f44bc072210 con 0x7f44bc130670 2023-10-03T10:15:13.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.196+0000 7f44b1ffb700 1 -- 172.21.15.3:0/3700542983 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f44b402b090 con 0x7f44bc130670 2023-10-03T10:15:13.196 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:13.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.198+0000 7f44c35a9700 1 -- 172.21.15.3:0/3700542983 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44bc130670 msgr2=0x7f44bc11e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:13.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.198+0000 7f44c35a9700 1 --2- 172.21.15.3:0/3700542983 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44bc130670 0x7f44bc11e520 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f44b400efc0 tx=0x7f44b400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:15:13.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.198+0000 7f44c35a9700 1 -- 172.21.15.3:0/3700542983 shutdown_connections 2023-10-03T10:15:13.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.198+0000 7f44c35a9700 1 --2- 172.21.15.3:0/3700542983 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44bc130670 0x7f44bc11e520 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:13.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.198+0000 7f44c35a9700 1 --2- 172.21.15.3:0/3700542983 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44bc06f680 0x7f44bc11daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:13.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.198+0000 7f44c35a9700 1 --2- 172.21.15.3:0/3700542983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44bc06ffd0 0x7f44bc11dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:13.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.198+0000 7f44c35a9700 1 -- 172.21.15.3:0/3700542983 >> 172.21.15.3:0/3700542983 conn(0x7f44bc0a2aa0 msgr2=0x7f44bc06e570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:13.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.198+0000 7f44c35a9700 1 -- 172.21.15.3:0/3700542983 shutdown_connections 2023-10-03T10:15:13.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.199+0000 7f44c35a9700 1 -- 172.21.15.3:0/3700542983 wait complete. 2023-10-03T10:15:13.213 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:13.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.225+0000 7f1547863700 1 -- 172.21.15.3:0/845011303 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f154006c360 con 0x7f154009e1d0 2023-10-03T10:15:13.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.225+0000 7f1535ffb700 1 -- 172.21.15.3:0/845011303 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f153001d020 con 0x7f154009e1d0 2023-10-03T10:15:13.225 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:13.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.227+0000 7f1547863700 1 -- 172.21.15.3:0/845011303 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f154009e1d0 msgr2=0x7f154011b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:13.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.227+0000 7f1547863700 1 --2- 172.21.15.3:0/845011303 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f154009e1d0 0x7f154011b820 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1530009870 tx=0x7f1530004750 comp rx=0 tx=0).stop 2023-10-03T10:15:13.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.227+0000 7f1547863700 1 -- 172.21.15.3:0/845011303 shutdown_connections 2023-10-03T10:15:13.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.227+0000 7f1547863700 1 --2- 172.21.15.3:0/845011303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1540135e50 0x7f154011bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:13.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.227+0000 7f1547863700 1 --2- 172.21.15.3:0/845011303 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f154009d6f0 0x7f154011b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:13.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.227+0000 7f1547863700 1 --2- 172.21.15.3:0/845011303 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f154009e1d0 0x7f154011b820 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:13.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.227+0000 7f1547863700 1 -- 172.21.15.3:0/845011303 >> 172.21.15.3:0/845011303 conn(0x7f1540066f50 msgr2=0x7f1540132680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:13.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.227+0000 7f1547863700 1 -- 172.21.15.3:0/845011303 shutdown_connections 2023-10-03T10:15:13.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:13.227+0000 7f1547863700 1 -- 172.21.15.3:0/845011303 wait complete. 2023-10-03T10:15:13.241 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:14.037 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:14.165 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:14.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.183+0000 7feff6138700 1 Processor -- start 2023-10-03T10:15:14.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.183+0000 7feff6138700 1 -- start start 2023-10-03T10:15:14.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.184+0000 7feff6138700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feff009d6f0 0x7feff009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:14.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.184+0000 7feff6138700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feff009e000 0x7feff0130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:14.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.184+0000 7feff6138700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff009f130 con 0x7feff009e410 2023-10-03T10:15:14.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.184+0000 7feff6138700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feff009f520 con 0x7feff009d6f0 2023-10-03T10:15:14.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.184+0000 7feff6138700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff009e930 con 0x7feff009e000 2023-10-03T10:15:14.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.184+0000 7fefeeffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feff009e000 0x7feff0130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:14.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.184+0000 7fefef7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feff009d6f0 0x7feff009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:14.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.184+0000 7fefef7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feff009d6f0 0x7feff009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42296/0 (socket says 172.21.15.3:42296) 2023-10-03T10:15:14.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.184+0000 7fefeeffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feff009e000 0x7feff0130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50408/0 (socket says 172.21.15.3:50408) 2023-10-03T10:15:14.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.184+0000 7fefef7fe700 1 -- 172.21.15.3:0/3552417463 learned_addr learned my addr 172.21.15.3:0/3552417463 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:14.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.184+0000 7fefef7fe700 1 -- 172.21.15.3:0/3552417463 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feff009e000 msgr2=0x7feff0130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:14.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.184+0000 7fefef7fe700 1 --2- 172.21.15.3:0/3552417463 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feff009e000 0x7feff0130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.184+0000 7fefef7fe700 1 -- 172.21.15.3:0/3552417463 >> v1:172.21.15.3:6789/0 conn(0x7feff009e410 legacy=0x7feff0130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:14.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.184+0000 7fefef7fe700 1 -- 172.21.15.3:0/3552417463 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff009b2d0 con 0x7feff009d6f0 2023-10-03T10:15:14.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.184+0000 7fefeeffd700 1 --2- 172.21.15.3:0/3552417463 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feff009e000 0x7feff0130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:14.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.185+0000 7fefef7fe700 1 --2- 172.21.15.3:0/3552417463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feff009d6f0 0x7feff009dac0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fefe00098f0 tx=0x7fefe0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22db65b058a9ef9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:14.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.185+0000 7fefee7fc700 1 -- 172.21.15.3:0/3552417463 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe0011830 con 0x7feff009d6f0 2023-10-03T10:15:14.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.185+0000 7fefee7fc700 1 -- 172.21.15.3:0/3552417463 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe0011990 con 0x7feff009d6f0 2023-10-03T10:15:14.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.185+0000 7fefee7fc700 1 -- 172.21.15.3:0/3552417463 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe0011d20 con 0x7feff009d6f0 2023-10-03T10:15:14.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.185+0000 7feff6138700 1 -- 172.21.15.3:0/3552417463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feff009d6f0 msgr2=0x7feff009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:14.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.185+0000 7feff6138700 1 --2- 172.21.15.3:0/3552417463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feff009d6f0 0x7feff009dac0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fefe00098f0 tx=0x7fefe0009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:14.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.185+0000 7feff6138700 1 -- 172.21.15.3:0/3552417463 shutdown_connections 2023-10-03T10:15:14.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.185+0000 7feff6138700 1 --2- 172.21.15.3:0/3552417463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feff009d6f0 0x7feff009dac0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.185+0000 7feff6138700 1 --2- 172.21.15.3:0/3552417463 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feff009e000 0x7feff0130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.185+0000 7feff6138700 1 -- 172.21.15.3:0/3552417463 >> 172.21.15.3:0/3552417463 conn(0x7feff0066f50 msgr2=0x7feff009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:14.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.185+0000 7feff6138700 1 -- 172.21.15.3:0/3552417463 shutdown_connections 2023-10-03T10:15:14.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.185+0000 7feff6138700 1 -- 172.21.15.3:0/3552417463 wait complete. 2023-10-03T10:15:14.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.186+0000 7feff6138700 1 Processor -- start 2023-10-03T10:15:14.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.187+0000 7feff6138700 1 -- start start 2023-10-03T10:15:14.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.187+0000 7feff6138700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feff009d6f0 0x7feff0139bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:14.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.187+0000 7feff6138700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feff009e000 0x7feff013a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:14.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.187+0000 7feff6138700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feff009e410 0x7feff013a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:14.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.187+0000 7feff6138700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff01370c0 con 0x7feff009e000 2023-10-03T10:15:14.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.187+0000 7feff6138700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff0138280 con 0x7feff009d6f0 2023-10-03T10:15:14.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.187+0000 7feff6138700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feff009c700 con 0x7feff009e410 2023-10-03T10:15:14.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.187+0000 7fefef7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feff009d6f0 0x7feff0139bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:14.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.187+0000 7fefeffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feff009e410 0x7feff013a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:14.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.187+0000 7fefef7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feff009d6f0 0x7feff0139bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50424/0 (socket says 172.21.15.3:50424) 2023-10-03T10:15:14.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.187+0000 7fefef7fe700 1 -- 172.21.15.3:0/3511391250 learned_addr learned my addr 172.21.15.3:0/3511391250 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:14.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.187+0000 7fefeeffd700 1 --2- 172.21.15.3:0/3511391250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feff009e000 0x7feff013a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:14.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.188+0000 7fefef7fe700 1 -- 172.21.15.3:0/3511391250 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feff009e410 msgr2=0x7feff013a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:14.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.188+0000 7fefef7fe700 1 --2- 172.21.15.3:0/3511391250 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feff009e410 0x7feff013a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.188+0000 7fefef7fe700 1 -- 172.21.15.3:0/3511391250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feff009e000 msgr2=0x7feff013a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:14.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.188+0000 7fefef7fe700 1 --2- 172.21.15.3:0/3511391250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feff009e000 0x7feff013a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.188+0000 7fefef7fe700 1 -- 172.21.15.3:0/3511391250 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe40094a0 con 0x7feff009d6f0 2023-10-03T10:15:14.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.188+0000 7fefef7fe700 1 --2- 172.21.15.3:0/3511391250 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feff009d6f0 0x7feff0139bb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fefe0000c00 tx=0x7fefe0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:14.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.188+0000 7fefd7fff700 1 -- 172.21.15.3:0/3511391250 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe00036a0 con 0x7feff009d6f0 2023-10-03T10:15:14.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.188+0000 7fefd7fff700 1 -- 172.21.15.3:0/3511391250 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe0003800 con 0x7feff009d6f0 2023-10-03T10:15:14.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.188+0000 7feff6138700 1 -- 172.21.15.3:0/3511391250 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe0009550 con 0x7feff009d6f0 2023-10-03T10:15:14.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.188+0000 7fefd7fff700 1 -- 172.21.15.3:0/3511391250 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe0003bb0 con 0x7feff009d6f0 2023-10-03T10:15:14.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.188+0000 7feff6138700 1 -- 172.21.15.3:0/3511391250 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff0137920 con 0x7feff009d6f0 2023-10-03T10:15:14.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.189+0000 7fefd7fff700 1 -- 172.21.15.3:0/3511391250 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fefe0021070 con 0x7feff009d6f0 2023-10-03T10:15:14.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.189+0000 7fefd7fff700 1 -- 172.21.15.3:0/3511391250 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fefe0040b80 con 0x7feff009d6f0 2023-10-03T10:15:14.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.189+0000 7feff6138700 1 -- 172.21.15.3:0/3511391250 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff0138990 con 0x7feff009d6f0 2023-10-03T10:15:14.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.194+0000 7fefd7fff700 1 -- 172.21.15.3:0/3511391250 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fefe0015030 con 0x7feff009d6f0 2023-10-03T10:15:14.214 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:14.242 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:14.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.319+0000 7f6a72f1b700 1 Processor -- start 2023-10-03T10:15:14.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.319+0000 7f6a72f1b700 1 -- start start 2023-10-03T10:15:14.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.319+0000 7f6a72f1b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a6c06a770 0x7f6a6c09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:14.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.319+0000 7f6a72f1b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6a6c13d1f0 0x7f6a6c13f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:14.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.319+0000 7f6a72f1b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a6c09a810 con 0x7f6a6c09dc30 2023-10-03T10:15:14.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.319+0000 7f6a72f1b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a6c06ab40 con 0x7f6a6c13d1f0 2023-10-03T10:15:14.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.319+0000 7f6a72f1b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c09ebc0 con 0x7f6a6c06a770 2023-10-03T10:15:14.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.320+0000 7f6a6bfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6a6c09dc30 0x7f6a6c09e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47578/0 (socket says 172.21.15.3:47578) 2023-10-03T10:15:14.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.320+0000 7f6a6bfff700 1 -- 172.21.15.3:0/977810652 learned_addr learned my addr 172.21.15.3:0/977810652 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:14.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.320+0000 7f6a714b8700 1 --2- 172.21.15.3:0/977810652 >> v2:172.21.15.3:3300/0 conn(0x7f6a6c13d1f0 0x7f6a6c13f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:14.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.320+0000 7f6a70cb7700 1 --2- 172.21.15.3:0/977810652 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a6c06a770 0x7f6a6c09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:14.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.320+0000 7f6a714b8700 1 -- 172.21.15.3:0/977810652 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a6c06a770 msgr2=0x7f6a6c09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:14.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.320+0000 7f6a714b8700 1 --2- 172.21.15.3:0/977810652 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a6c06a770 0x7f6a6c09d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.320+0000 7f6a714b8700 1 -- 172.21.15.3:0/977810652 >> v1:172.21.15.3:6789/0 conn(0x7f6a6c09dc30 legacy=0x7f6a6c09e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:14.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.320+0000 7f6a6b7fe700 1 -- 172.21.15.3:0/977810652 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 881270248 0 0) 0x7f6a6c09a810 con 0x7f6a6c09dc30 2023-10-03T10:15:14.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.320+0000 7f6a714b8700 1 -- 172.21.15.3:0/977810652 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6c09e920 con 0x7f6a6c13d1f0 2023-10-03T10:15:14.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.320+0000 7f6a70cb7700 1 --2- 172.21.15.3:0/977810652 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a6c06a770 0x7f6a6c09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:15:14.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.320+0000 7f6a714b8700 1 --2- 172.21.15.3:0/977810652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a6c13d1f0 0x7f6a6c13f5d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6a64009dc0 tx=0x7f6a6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e066220b2100376 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:14.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.320+0000 7f6a6b7fe700 1 -- 172.21.15.3:0/977810652 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6400e070 con 0x7f6a6c13d1f0 2023-10-03T10:15:14.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.321+0000 7f6a6b7fe700 1 -- 172.21.15.3:0/977810652 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6400bb90 con 0x7f6a6c13d1f0 2023-10-03T10:15:14.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.321+0000 7f6a6b7fe700 1 -- 172.21.15.3:0/977810652 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a64012430 con 0x7f6a6c13d1f0 2023-10-03T10:15:14.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.321+0000 7f6a72f1b700 1 -- 172.21.15.3:0/977810652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a6c13d1f0 msgr2=0x7f6a6c13f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:14.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.321+0000 7f6a72f1b700 1 --2- 172.21.15.3:0/977810652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a6c13d1f0 0x7f6a6c13f5d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6a64009dc0 tx=0x7f6a6400b330 comp rx=0 tx=0).stop 2023-10-03T10:15:14.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.322+0000 7f6a72f1b700 1 -- 172.21.15.3:0/977810652 shutdown_connections 2023-10-03T10:15:14.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.322+0000 7f6a72f1b700 1 --2- 172.21.15.3:0/977810652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a6c13d1f0 0x7f6a6c13f5d0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.322+0000 7f6a72f1b700 1 --2- 172.21.15.3:0/977810652 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a6c06a770 0x7f6a6c09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.322+0000 7f6a72f1b700 1 -- 172.21.15.3:0/977810652 >> 172.21.15.3:0/977810652 conn(0x7f6a6c066f50 msgr2=0x7f6a6c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:14.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.322+0000 7f6a72f1b700 1 -- 172.21.15.3:0/977810652 shutdown_connections 2023-10-03T10:15:14.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.322+0000 7f6a72f1b700 1 -- 172.21.15.3:0/977810652 wait complete. 2023-10-03T10:15:14.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.323+0000 7f6a72f1b700 1 Processor -- start 2023-10-03T10:15:14.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.323+0000 7f6a72f1b700 1 -- start start 2023-10-03T10:15:14.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.323+0000 7f6a72f1b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a6c06a770 0x7f6a6c1cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:14.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.323+0000 7f6a72f1b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a6c09dc30 0x7f6a6c1ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:14.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.323+0000 7f6a72f1b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6a6c13d1f0 0x7f6a6c1cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:14.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.323+0000 7f6a72f1b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c11c770 con 0x7f6a6c06a770 2023-10-03T10:15:14.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.323+0000 7f6a72f1b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c09f1a0 con 0x7f6a6c09dc30 2023-10-03T10:15:14.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.323+0000 7f6a72f1b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c0724a0 con 0x7f6a6c13d1f0 2023-10-03T10:15:14.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.324+0000 7f6a70cb7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a6c06a770 0x7f6a6c1cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:14.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.324+0000 7f6a70cb7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a6c06a770 0x7f6a6c1cc9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42324/0 (socket says 172.21.15.3:42324) 2023-10-03T10:15:14.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.324+0000 7f6a70cb7700 1 -- 172.21.15.3:0/3778899529 learned_addr learned my addr 172.21.15.3:0/3778899529 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:14.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.324+0000 7f6a714b8700 1 --2- 172.21.15.3:0/3778899529 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6a6c13d1f0 0x7f6a6c1cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:14.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.324+0000 7f6a6bfff700 1 --2- 172.21.15.3:0/3778899529 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a6c09dc30 0x7f6a6c1ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:14.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.324+0000 7f6a70cb7700 1 -- 172.21.15.3:0/3778899529 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6a6c13d1f0 msgr2=0x7f6a6c1cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:14.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.324+0000 7f6a70cb7700 1 --2- 172.21.15.3:0/3778899529 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6a6c13d1f0 0x7f6a6c1cd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.324+0000 7f6a70cb7700 1 -- 172.21.15.3:0/3778899529 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a6c09dc30 msgr2=0x7f6a6c1ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:14.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.324+0000 7f6a70cb7700 1 --2- 172.21.15.3:0/3778899529 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a6c09dc30 0x7f6a6c1ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.324+0000 7f6a70cb7700 1 -- 172.21.15.3:0/3778899529 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a64009a70 con 0x7f6a6c06a770 2023-10-03T10:15:14.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.324+0000 7f6a70cb7700 1 --2- 172.21.15.3:0/3778899529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a6c06a770 0x7f6a6c1cc9c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000c750 tx=0x7f6a6000ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:14.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.325+0000 7f6a697fa700 1 -- 172.21.15.3:0/3778899529 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a60011800 con 0x7f6a6c06a770 2023-10-03T10:15:14.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.325+0000 7f6a697fa700 1 -- 172.21.15.3:0/3778899529 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a60011960 con 0x7f6a6c06a770 2023-10-03T10:15:14.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.325+0000 7f6a72f1b700 1 -- 172.21.15.3:0/3778899529 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a6c11cfa0 con 0x7f6a6c06a770 2023-10-03T10:15:14.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.325+0000 7f6a72f1b700 1 -- 172.21.15.3:0/3778899529 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6c0708b0 con 0x7f6a6c06a770 2023-10-03T10:15:14.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.325+0000 7f6a697fa700 1 -- 172.21.15.3:0/3778899529 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a60011cd0 con 0x7f6a6c06a770 2023-10-03T10:15:14.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.326+0000 7f6a697fa700 1 -- 172.21.15.3:0/3778899529 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6a60021350 con 0x7f6a6c06a770 2023-10-03T10:15:14.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.326+0000 7f6a697fa700 1 -- 172.21.15.3:0/3778899529 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6a6003f620 con 0x7f6a6c06a770 2023-10-03T10:15:14.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.326+0000 7f6a72f1b700 1 -- 172.21.15.3:0/3778899529 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a4c0052e0 con 0x7f6a6c06a770 2023-10-03T10:15:14.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.331+0000 7f6a697fa700 1 -- 172.21.15.3:0/3778899529 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6a60011e30 con 0x7f6a6c06a770 2023-10-03T10:15:14.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.368+0000 7f382f70f700 1 Processor -- start 2023-10-03T10:15:14.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.369+0000 7f382f70f700 1 -- start start 2023-10-03T10:15:14.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.369+0000 7f382f70f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f382809e930 0x7f382809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:14.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.369+0000 7f382f70f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f382809f240 0x7f38281317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:14.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.369+0000 7f382f70f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3828070940 con 0x7f382809f650 2023-10-03T10:15:14.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.369+0000 7f382f70f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f382806fd60 con 0x7f382809e930 2023-10-03T10:15:14.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.369+0000 7f382f70f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f382806fa80 con 0x7f382809f240 2023-10-03T10:15:14.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.370+0000 7f382ccaa700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f382809f240 0x7f38281317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:14.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.370+0000 7f382ccaa700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f382809f240 0x7f38281317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53718/0 (socket says 172.21.15.3:53718) 2023-10-03T10:15:14.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.370+0000 7f382dcac700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f382809f650 0x7f3828131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47592/0 (socket says 172.21.15.3:47592) 2023-10-03T10:15:14.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.370+0000 7f382ccaa700 1 -- 172.21.15.3:0/2933492822 learned_addr learned my addr 172.21.15.3:0/2933492822 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:14.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.370+0000 7f382d4ab700 1 --2- 172.21.15.3:0/2933492822 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f382809e930 0x7f382809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:14.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.370+0000 7f382ccaa700 1 -- 172.21.15.3:0/2933492822 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f382809e930 msgr2=0x7f382809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:14.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.370+0000 7f382ccaa700 1 --2- 172.21.15.3:0/2933492822 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f382809e930 0x7f382809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.370+0000 7f382ccaa700 1 -- 172.21.15.3:0/2933492822 >> v1:172.21.15.3:6789/0 conn(0x7f382809f650 legacy=0x7f3828131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:14.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.370+0000 7f382ccaa700 1 -- 172.21.15.3:0/2933492822 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f382809fb60 con 0x7f382809f240 2023-10-03T10:15:14.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.370+0000 7f382ccaa700 1 --2- 172.21.15.3:0/2933492822 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f382809f240 0x7f38281317f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3818009920 tx=0x7f3818009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=deb0b311cab96e1e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:14.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.370+0000 7f381ffff700 1 -- 172.21.15.3:0/2933492822 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3818011830 con 0x7f382809f240 2023-10-03T10:15:14.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.371+0000 7f381ffff700 1 -- 172.21.15.3:0/2933492822 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3818011990 con 0x7f382809f240 2023-10-03T10:15:14.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.371+0000 7f381ffff700 1 -- 172.21.15.3:0/2933492822 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3818011d20 con 0x7f382809f240 2023-10-03T10:15:14.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.371+0000 7f382f70f700 1 -- 172.21.15.3:0/2933492822 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f382809f240 msgr2=0x7f38281317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:14.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.371+0000 7f382f70f700 1 --2- 172.21.15.3:0/2933492822 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f382809f240 0x7f38281317f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3818009920 tx=0x7f3818009c30 comp rx=0 tx=0).stop 2023-10-03T10:15:14.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.372+0000 7f382f70f700 1 -- 172.21.15.3:0/2933492822 shutdown_connections 2023-10-03T10:15:14.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.372+0000 7f382f70f700 1 --2- 172.21.15.3:0/2933492822 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f382809e930 0x7f382809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.372+0000 7f382f70f700 1 --2- 172.21.15.3:0/2933492822 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f382809f240 0x7f38281317f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.372+0000 7f382f70f700 1 -- 172.21.15.3:0/2933492822 >> 172.21.15.3:0/2933492822 conn(0x7f38280a4210 msgr2=0x7f38280a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:14.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.372+0000 7f382f70f700 1 -- 172.21.15.3:0/2933492822 shutdown_connections 2023-10-03T10:15:14.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.372+0000 7f382f70f700 1 -- 172.21.15.3:0/2933492822 wait complete. 2023-10-03T10:15:14.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.373+0000 7f382f70f700 1 Processor -- start 2023-10-03T10:15:14.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.373+0000 7f382f70f700 1 -- start start 2023-10-03T10:15:14.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.374+0000 7f382f70f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f382809e930 0x7f382806b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:14.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.374+0000 7f382f70f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f382809f240 0x7f382806bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:14.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.374+0000 7f382f70f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f382809f650 0x7f382806c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:14.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.374+0000 7f382f70f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38281b61f0 con 0x7f382809f650 2023-10-03T10:15:14.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.374+0000 7f382f70f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38281a83a0 con 0x7f382809f240 2023-10-03T10:15:14.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.374+0000 7f382f70f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38281a78c0 con 0x7f382809e930 2023-10-03T10:15:14.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.374+0000 7f382d4ab700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f382809e930 0x7f382806b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:14.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.374+0000 7f382d4ab700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f382809e930 0x7f382806b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53730/0 (socket says 172.21.15.3:53730) 2023-10-03T10:15:14.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.374+0000 7f382d4ab700 1 -- 172.21.15.3:0/3939434416 learned_addr learned my addr 172.21.15.3:0/3939434416 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:14.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.374+0000 7f382ccaa700 1 --2- 172.21.15.3:0/3939434416 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f382809f240 0x7f382806bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:14.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.374+0000 7f382dcac700 1 --2- 172.21.15.3:0/3939434416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f382809f650 0x7f382806c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:14.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.374+0000 7f382d4ab700 1 -- 172.21.15.3:0/3939434416 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f382809f240 msgr2=0x7f382806bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:14.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.374+0000 7f382d4ab700 1 --2- 172.21.15.3:0/3939434416 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f382809f240 0x7f382806bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.374+0000 7f382d4ab700 1 -- 172.21.15.3:0/3939434416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f382809f650 msgr2=0x7f382806c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:14.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.374+0000 7f382d4ab700 1 --2- 172.21.15.3:0/3939434416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f382809f650 0x7f382806c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.374+0000 7f382d4ab700 1 -- 172.21.15.3:0/3939434416 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3818009550 con 0x7f382809e930 2023-10-03T10:15:14.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.375+0000 7f382d4ab700 1 --2- 172.21.15.3:0/3939434416 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f382809e930 0x7f382806b8b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f382400e9b0 tx=0x7f382400ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:14.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.375+0000 7f381dffb700 1 -- 172.21.15.3:0/3939434416 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382400ee50 con 0x7f382809e930 2023-10-03T10:15:14.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.375+0000 7f381dffb700 1 -- 172.21.15.3:0/3939434416 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382400c850 con 0x7f382809e930 2023-10-03T10:15:14.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.375+0000 7f381dffb700 1 -- 172.21.15.3:0/3939434416 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382400cbc0 con 0x7f382809e930 2023-10-03T10:15:14.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.375+0000 7f382f70f700 1 -- 172.21.15.3:0/3939434416 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38280a0190 con 0x7f382809e930 2023-10-03T10:15:14.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.375+0000 7f382f70f700 1 -- 172.21.15.3:0/3939434416 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38281a80d0 con 0x7f382809e930 2023-10-03T10:15:14.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.376+0000 7f382f70f700 1 -- 172.21.15.3:0/3939434416 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38280a38e0 con 0x7f382809e930 2023-10-03T10:15:14.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.376+0000 7f381dffb700 1 -- 172.21.15.3:0/3939434416 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f382400cd20 con 0x7f382809e930 2023-10-03T10:15:14.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.377+0000 7f381dffb700 1 -- 172.21.15.3:0/3939434416 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f382403faa0 con 0x7f382809e930 2023-10-03T10:15:14.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.386+0000 7f381dffb700 1 -- 172.21.15.3:0/3939434416 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f382401ab50 con 0x7f382809e930 2023-10-03T10:15:14.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.404+0000 7fc118d87700 1 Processor -- start 2023-10-03T10:15:14.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.404+0000 7fc118d87700 1 -- start start 2023-10-03T10:15:14.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.405+0000 7fc118d87700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc11406a770 0x7fc11409d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:14.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.405+0000 7fc118d87700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc11413d1f0 0x7fc11413f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:14.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.405+0000 7fc118d87700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc11409a810 con 0x7fc11409dc30 2023-10-03T10:15:14.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.405+0000 7fc118d87700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc11406ab40 con 0x7fc11406a770 2023-10-03T10:15:14.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.405+0000 7fc118d87700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc11409ebc0 con 0x7fc11413d1f0 2023-10-03T10:15:14.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.405+0000 7fc11259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc11406a770 0x7fc11409d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:14.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.405+0000 7fc11259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc11406a770 0x7fc11409d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42346/0 (socket says 172.21.15.3:42346) 2023-10-03T10:15:14.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.405+0000 7fc11259c700 1 -- 172.21.15.3:0/985452406 learned_addr learned my addr 172.21.15.3:0/985452406 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:14.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.405+0000 7fc112d9d700 1 --2- 172.21.15.3:0/985452406 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc11413d1f0 0x7fc11413f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:14.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.405+0000 7fc112d9d700 1 -- 172.21.15.3:0/985452406 >> v2:172.21.15.3:3300/0 conn(0x7fc11406a770 msgr2=0x7fc11409d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:14.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.405+0000 7fc112d9d700 1 --2- 172.21.15.3:0/985452406 >> v2:172.21.15.3:3300/0 conn(0x7fc11406a770 0x7fc11409d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.405+0000 7fc112d9d700 1 -- 172.21.15.3:0/985452406 >> v1:172.21.15.3:6789/0 conn(0x7fc11409dc30 legacy=0x7fc11409e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:14.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.405+0000 7fc112d9d700 1 -- 172.21.15.3:0/985452406 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11409e920 con 0x7fc11413d1f0 2023-10-03T10:15:14.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.405+0000 7fc11159a700 1 -- 172.21.15.3:0/985452406 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1348461715 0 0) 0x7fc11409a810 con 0x7fc11409dc30 2023-10-03T10:15:14.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.406+0000 7fc11259c700 1 --2- 172.21.15.3:0/985452406 >> v2:172.21.15.3:3300/0 conn(0x7fc11406a770 0x7fc11409d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:14.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.406+0000 7fc112d9d700 1 --2- 172.21.15.3:0/985452406 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc11413d1f0 0x7fc11413f5d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009d90 tx=0x7fc10c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f9c738d925c27a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:14.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.406+0000 7fc11159a700 1 -- 172.21.15.3:0/985452406 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c00e070 con 0x7fc11413d1f0 2023-10-03T10:15:14.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.406+0000 7fc11159a700 1 -- 172.21.15.3:0/985452406 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10c00bb20 con 0x7fc11413d1f0 2023-10-03T10:15:14.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.406+0000 7fc11159a700 1 -- 172.21.15.3:0/985452406 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c012430 con 0x7fc11413d1f0 2023-10-03T10:15:14.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.407+0000 7fc118d87700 1 -- 172.21.15.3:0/985452406 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc11413d1f0 msgr2=0x7fc11413f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:14.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.407+0000 7fc118d87700 1 --2- 172.21.15.3:0/985452406 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc11413d1f0 0x7fc11413f5d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009d90 tx=0x7fc10c00b330 comp rx=0 tx=0).stop 2023-10-03T10:15:14.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.407+0000 7fc118d87700 1 -- 172.21.15.3:0/985452406 shutdown_connections 2023-10-03T10:15:14.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.407+0000 7fc118d87700 1 --2- 172.21.15.3:0/985452406 >> v2:172.21.15.3:3300/0 conn(0x7fc11406a770 0x7fc11409d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.407+0000 7fc118d87700 1 --2- 172.21.15.3:0/985452406 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc11413d1f0 0x7fc11413f5d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.407+0000 7fc118d87700 1 -- 172.21.15.3:0/985452406 >> 172.21.15.3:0/985452406 conn(0x7fc114066f50 msgr2=0x7fc11409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:14.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.407+0000 7fc118d87700 1 -- 172.21.15.3:0/985452406 shutdown_connections 2023-10-03T10:15:14.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.407+0000 7fc118d87700 1 -- 172.21.15.3:0/985452406 wait complete. 2023-10-03T10:15:14.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.409+0000 7fc118d87700 1 Processor -- start 2023-10-03T10:15:14.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.409+0000 7fc118d87700 1 -- start start 2023-10-03T10:15:14.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.409+0000 7fc118d87700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc11406a770 0x7fc11411c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:14.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.409+0000 7fc118d87700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc11409dc30 0x7fc11413b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:14.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.409+0000 7fc118d87700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc11413d1f0 0x7fc11413bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:14.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.409+0000 7fc118d87700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc114135cd0 con 0x7fc11409dc30 2023-10-03T10:15:14.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.409+0000 7fc118d87700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11411ef00 con 0x7fc11413d1f0 2023-10-03T10:15:14.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.410+0000 7fc118d87700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc11409f1a0 con 0x7fc11406a770 2023-10-03T10:15:14.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.410+0000 7fc11259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc11406a770 0x7fc11411c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:14.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.410+0000 7fc11259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc11406a770 0x7fc11411c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53744/0 (socket says 172.21.15.3:53744) 2023-10-03T10:15:14.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.410+0000 7fc111d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc11409dc30 0x7fc11413b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:14.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.410+0000 7fc11259c700 1 -- 172.21.15.3:0/2550024038 learned_addr learned my addr 172.21.15.3:0/2550024038 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:14.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.410+0000 7fc112d9d700 1 --2- 172.21.15.3:0/2550024038 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc11413d1f0 0x7fc11413bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:14.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.410+0000 7fc11259c700 1 -- 172.21.15.3:0/2550024038 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc11413d1f0 msgr2=0x7fc11413bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:14.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.410+0000 7fc11259c700 1 --2- 172.21.15.3:0/2550024038 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc11413d1f0 0x7fc11413bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.410+0000 7fc11259c700 1 -- 172.21.15.3:0/2550024038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc11409dc30 msgr2=0x7fc11413b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:14.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.410+0000 7fc11259c700 1 --2- 172.21.15.3:0/2550024038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc11409dc30 0x7fc11413b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.410+0000 7fc11259c700 1 -- 172.21.15.3:0/2550024038 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc104009550 con 0x7fc11406a770 2023-10-03T10:15:14.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.410+0000 7fc11259c700 1 --2- 172.21.15.3:0/2550024038 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc11406a770 0x7fc11411c2b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc10400ebe0 tx=0x7fc10400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:14.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.411+0000 7fc102ffd700 1 -- 172.21.15.3:0/2550024038 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10400c850 con 0x7fc11406a770 2023-10-03T10:15:14.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.411+0000 7fc102ffd700 1 -- 172.21.15.3:0/2550024038 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10400c9b0 con 0x7fc11406a770 2023-10-03T10:15:14.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.411+0000 7fc118d87700 1 -- 172.21.15.3:0/2550024038 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc10c009a70 con 0x7fc11406a770 2023-10-03T10:15:14.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.411+0000 7fc102ffd700 1 -- 172.21.15.3:0/2550024038 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10400cd20 con 0x7fc11406a770 2023-10-03T10:15:14.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.411+0000 7fc118d87700 1 -- 172.21.15.3:0/2550024038 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11413ce50 con 0x7fc11406a770 2023-10-03T10:15:14.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.412+0000 7fc118d87700 1 -- 172.21.15.3:0/2550024038 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11411e8c0 con 0x7fc11406a770 2023-10-03T10:15:14.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.413+0000 7fc102ffd700 1 -- 172.21.15.3:0/2550024038 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc104004020 con 0x7fc11406a770 2023-10-03T10:15:14.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.413+0000 7fc102ffd700 1 -- 172.21.15.3:0/2550024038 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc104040160 con 0x7fc11406a770 2023-10-03T10:15:14.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.421+0000 7fc102ffd700 1 -- 172.21.15.3:0/2550024038 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc104024350 con 0x7fc11406a770 2023-10-03T10:15:14.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.428+0000 7feff6138700 1 -- 172.21.15.3:0/3511391250 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7feff0074ea0 con 0x7feff009d6f0 2023-10-03T10:15:14.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.428+0000 7fefd7fff700 1 -- 172.21.15.3:0/3511391250 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fefe0003d10 con 0x7feff009d6f0 2023-10-03T10:15:14.428 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:14.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.430+0000 7feff6138700 1 -- 172.21.15.3:0/3511391250 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feff009d6f0 msgr2=0x7feff0139bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:14.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.430+0000 7feff6138700 1 --2- 172.21.15.3:0/3511391250 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feff009d6f0 0x7feff0139bb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fefe0000c00 tx=0x7fefe0004750 comp rx=0 tx=0).stop 2023-10-03T10:15:14.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.431+0000 7feff6138700 1 -- 172.21.15.3:0/3511391250 shutdown_connections 2023-10-03T10:15:14.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.431+0000 7feff6138700 1 --2- 172.21.15.3:0/3511391250 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feff009e410 0x7feff013a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.431+0000 7feff6138700 1 --2- 172.21.15.3:0/3511391250 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feff009d6f0 0x7feff0139bb0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.431+0000 7feff6138700 1 --2- 172.21.15.3:0/3511391250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feff009e000 0x7feff013a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.431+0000 7feff6138700 1 -- 172.21.15.3:0/3511391250 >> 172.21.15.3:0/3511391250 conn(0x7feff0066f50 msgr2=0x7feff00a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:14.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.431+0000 7feff6138700 1 -- 172.21.15.3:0/3511391250 shutdown_connections 2023-10-03T10:15:14.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.431+0000 7feff6138700 1 -- 172.21.15.3:0/3511391250 wait complete. 2023-10-03T10:15:14.445 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:14.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.554+0000 7f6a72f1b700 1 -- 172.21.15.3:0/3778899529 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6a4c005590 con 0x7f6a6c06a770 2023-10-03T10:15:14.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.555+0000 7f6a697fa700 1 -- 172.21.15.3:0/3778899529 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6a60017af0 con 0x7f6a6c06a770 2023-10-03T10:15:14.555 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:14.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.557+0000 7f6a72f1b700 1 -- 172.21.15.3:0/3778899529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a6c06a770 msgr2=0x7f6a6c1cc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:14.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.557+0000 7f6a72f1b700 1 --2- 172.21.15.3:0/3778899529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a6c06a770 0x7f6a6c1cc9c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000c750 tx=0x7f6a6000ca60 comp rx=0 tx=0).stop 2023-10-03T10:15:14.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.557+0000 7f6a72f1b700 1 -- 172.21.15.3:0/3778899529 shutdown_connections 2023-10-03T10:15:14.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.557+0000 7f6a72f1b700 1 --2- 172.21.15.3:0/3778899529 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6a6c13d1f0 0x7f6a6c1cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.557+0000 7f6a72f1b700 1 --2- 172.21.15.3:0/3778899529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a6c06a770 0x7f6a6c1cc9c0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.557+0000 7f6a72f1b700 1 --2- 172.21.15.3:0/3778899529 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a6c09dc30 0x7f6a6c1ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.557+0000 7f6a72f1b700 1 -- 172.21.15.3:0/3778899529 >> 172.21.15.3:0/3778899529 conn(0x7f6a6c066f50 msgr2=0x7f6a6c13c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:14.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.557+0000 7f6a72f1b700 1 -- 172.21.15.3:0/3778899529 shutdown_connections 2023-10-03T10:15:14.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.557+0000 7f6a72f1b700 1 -- 172.21.15.3:0/3778899529 wait complete. 2023-10-03T10:15:14.571 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:14.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.596+0000 7f382f70f700 1 -- 172.21.15.3:0/3939434416 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3828074ea0 con 0x7f382809e930 2023-10-03T10:15:14.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.596+0000 7f381dffb700 1 -- 172.21.15.3:0/3939434416 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3824017020 con 0x7f382809e930 2023-10-03T10:15:14.597 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:14.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.599+0000 7f382f70f700 1 -- 172.21.15.3:0/3939434416 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f382809e930 msgr2=0x7f382806b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:14.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.599+0000 7f382f70f700 1 --2- 172.21.15.3:0/3939434416 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f382809e930 0x7f382806b8b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f382400e9b0 tx=0x7f382400ecc0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.599+0000 7f382f70f700 1 -- 172.21.15.3:0/3939434416 shutdown_connections 2023-10-03T10:15:14.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.599+0000 7f382f70f700 1 --2- 172.21.15.3:0/3939434416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f382809f650 0x7f382806c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.599+0000 7f382f70f700 1 --2- 172.21.15.3:0/3939434416 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f382809e930 0x7f382806b8b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.599+0000 7f382f70f700 1 --2- 172.21.15.3:0/3939434416 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f382809f240 0x7f382806bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.599+0000 7f382f70f700 1 -- 172.21.15.3:0/3939434416 >> 172.21.15.3:0/3939434416 conn(0x7f38280a4210 msgr2=0x7f38281b7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:14.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.599+0000 7f382f70f700 1 -- 172.21.15.3:0/3939434416 shutdown_connections 2023-10-03T10:15:14.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.599+0000 7f382f70f700 1 -- 172.21.15.3:0/3939434416 wait complete. 2023-10-03T10:15:14.613 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:14.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.640+0000 7fc118d87700 1 -- 172.21.15.3:0/2550024038 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc114074ea0 con 0x7fc11406a770 2023-10-03T10:15:14.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.640+0000 7fc102ffd700 1 -- 172.21.15.3:0/2550024038 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc104021070 con 0x7fc11406a770 2023-10-03T10:15:14.640 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:14.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.642+0000 7fc118d87700 1 -- 172.21.15.3:0/2550024038 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc11406a770 msgr2=0x7fc11411c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:14.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.642+0000 7fc118d87700 1 --2- 172.21.15.3:0/2550024038 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc11406a770 0x7fc11411c2b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc10400ebe0 tx=0x7fc10400eef0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.642+0000 7fc118d87700 1 -- 172.21.15.3:0/2550024038 shutdown_connections 2023-10-03T10:15:14.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.642+0000 7fc118d87700 1 --2- 172.21.15.3:0/2550024038 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc11413d1f0 0x7fc11413bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.642+0000 7fc118d87700 1 --2- 172.21.15.3:0/2550024038 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc11406a770 0x7fc11411c2b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.642+0000 7fc118d87700 1 --2- 172.21.15.3:0/2550024038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc11409dc30 0x7fc11413b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:14.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.642+0000 7fc118d87700 1 -- 172.21.15.3:0/2550024038 >> 172.21.15.3:0/2550024038 conn(0x7fc114066f50 msgr2=0x7fc11409b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:14.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.643+0000 7fc118d87700 1 -- 172.21.15.3:0/2550024038 shutdown_connections 2023-10-03T10:15:14.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:14.643+0000 7fc118d87700 1 -- 172.21.15.3:0/2550024038 wait complete. 2023-10-03T10:15:14.656 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:15.447 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:15.572 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:15.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.594+0000 7f8a476a4700 1 Processor -- start 2023-10-03T10:15:15.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.595+0000 7f8a476a4700 1 -- start start 2023-10-03T10:15:15.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.595+0000 7f8a476a4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a4009e930 0x7f8a4009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:15.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.595+0000 7f8a476a4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8a4009f650 0x7f8a40131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:15.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.595+0000 7f8a476a4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a40070940 con 0x7f8a4009f240 2023-10-03T10:15:15.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.595+0000 7f8a476a4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a4006fd60 con 0x7f8a4009f650 2023-10-03T10:15:15.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.595+0000 7f8a476a4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4006fa80 con 0x7f8a4009e930 2023-10-03T10:15:15.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.595+0000 7f8a45440700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a4009e930 0x7f8a4009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:15.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.595+0000 7f8a44c3f700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8a4009f240 0x7f8a401317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47616/0 (socket says 172.21.15.3:47616) 2023-10-03T10:15:15.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.595+0000 7f8a44c3f700 1 -- 172.21.15.3:0/4062363731 learned_addr learned my addr 172.21.15.3:0/4062363731 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:15.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.595+0000 7f8a45c41700 1 --2- 172.21.15.3:0/4062363731 >> v2:172.21.15.3:3300/0 conn(0x7f8a4009f650 0x7f8a40131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:15.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.596+0000 7f8a37fff700 1 -- 172.21.15.3:0/4062363731 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1097204331 0 0) 0x7f8a40070940 con 0x7f8a4009f240 2023-10-03T10:15:15.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.596+0000 7f8a45440700 1 -- 172.21.15.3:0/4062363731 >> v2:172.21.15.3:3300/0 conn(0x7f8a4009f650 msgr2=0x7f8a40131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:15.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.596+0000 7f8a45440700 1 --2- 172.21.15.3:0/4062363731 >> v2:172.21.15.3:3300/0 conn(0x7f8a4009f650 0x7f8a40131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.596+0000 7f8a45440700 1 -- 172.21.15.3:0/4062363731 >> v1:172.21.15.3:6789/0 conn(0x7f8a4009f240 legacy=0x7f8a401317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:15.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.596+0000 7f8a45440700 1 -- 172.21.15.3:0/4062363731 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4009fb60 con 0x7f8a4009e930 2023-10-03T10:15:15.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.596+0000 7f8a45440700 1 --2- 172.21.15.3:0/4062363731 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a4009e930 0x7f8a4009ed00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009840 tx=0x7f8a3c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aab469372a004b30 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:15.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.596+0000 7f8a37fff700 1 -- 172.21.15.3:0/4062363731 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c011830 con 0x7f8a4009e930 2023-10-03T10:15:15.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.596+0000 7f8a37fff700 1 -- 172.21.15.3:0/4062363731 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3c011990 con 0x7f8a4009e930 2023-10-03T10:15:15.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.596+0000 7f8a37fff700 1 -- 172.21.15.3:0/4062363731 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c011d20 con 0x7f8a4009e930 2023-10-03T10:15:15.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.597+0000 7f8a476a4700 1 -- 172.21.15.3:0/4062363731 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a4009e930 msgr2=0x7f8a4009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:15.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.597+0000 7f8a476a4700 1 --2- 172.21.15.3:0/4062363731 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a4009e930 0x7f8a4009ed00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009840 tx=0x7f8a3c009b50 comp rx=0 tx=0).stop 2023-10-03T10:15:15.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.597+0000 7f8a476a4700 1 -- 172.21.15.3:0/4062363731 shutdown_connections 2023-10-03T10:15:15.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.597+0000 7f8a476a4700 1 --2- 172.21.15.3:0/4062363731 >> v2:172.21.15.3:3300/0 conn(0x7f8a4009f650 0x7f8a40131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.597+0000 7f8a476a4700 1 --2- 172.21.15.3:0/4062363731 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a4009e930 0x7f8a4009ed00 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.597+0000 7f8a476a4700 1 -- 172.21.15.3:0/4062363731 >> 172.21.15.3:0/4062363731 conn(0x7f8a400a4210 msgr2=0x7f8a400a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:15.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.597+0000 7f8a476a4700 1 -- 172.21.15.3:0/4062363731 shutdown_connections 2023-10-03T10:15:15.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.597+0000 7f8a476a4700 1 -- 172.21.15.3:0/4062363731 wait complete. 2023-10-03T10:15:15.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.598+0000 7f8a476a4700 1 Processor -- start 2023-10-03T10:15:15.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.598+0000 7f8a476a4700 1 -- start start 2023-10-03T10:15:15.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.598+0000 7f8a476a4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a4009e930 0x7f8a401c2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:15.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.598+0000 7f8a476a4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a4009f240 0x7f8a401c3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:15.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.598+0000 7f8a476a4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a4009f650 0x7f8a401c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:15.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.598+0000 7f8a476a4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a401a4f80 con 0x7f8a4009f240 2023-10-03T10:15:15.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.598+0000 7f8a476a4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a401a75f0 con 0x7f8a4009f650 2023-10-03T10:15:15.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.598+0000 7f8a476a4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a401c38a0 con 0x7f8a4009e930 2023-10-03T10:15:15.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.599+0000 7f8a44c3f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a4009f240 0x7f8a401c3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:15.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.599+0000 7f8a45c41700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a4009f650 0x7f8a401c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:15.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.599+0000 7f8a45c41700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a4009f650 0x7f8a401c5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50478/0 (socket says 172.21.15.3:50478) 2023-10-03T10:15:15.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.599+0000 7f8a44c3f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a4009f240 0x7f8a401c3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42364/0 (socket says 172.21.15.3:42364) 2023-10-03T10:15:15.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.599+0000 7f8a45c41700 1 -- 172.21.15.3:0/2844259737 learned_addr learned my addr 172.21.15.3:0/2844259737 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:15.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.599+0000 7f8a45440700 1 --2- 172.21.15.3:0/2844259737 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a4009e930 0x7f8a401c2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:15.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.599+0000 7f8a45c41700 1 -- 172.21.15.3:0/2844259737 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a4009e930 msgr2=0x7f8a401c2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:15.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.599+0000 7f8a45c41700 1 --2- 172.21.15.3:0/2844259737 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a4009e930 0x7f8a401c2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.599+0000 7f8a45c41700 1 -- 172.21.15.3:0/2844259737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a4009f240 msgr2=0x7f8a401c3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:15.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.599+0000 7f8a45c41700 1 --2- 172.21.15.3:0/2844259737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a4009f240 0x7f8a401c3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.599+0000 7f8a45c41700 1 -- 172.21.15.3:0/2844259737 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a3c0094a0 con 0x7f8a4009f650 2023-10-03T10:15:15.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.599+0000 7f8a44c3f700 1 --2- 172.21.15.3:0/2844259737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a4009f240 0x7f8a401c3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:15.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.599+0000 7f8a45c41700 1 --2- 172.21.15.3:0/2844259737 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a4009f650 0x7f8a401c5e40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8a3800efc0 tx=0x7f8a3800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:15.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.599+0000 7f8a35ffb700 1 -- 172.21.15.3:0/2844259737 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a38009ea0 con 0x7f8a4009f650 2023-10-03T10:15:15.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.599+0000 7f8a35ffb700 1 -- 172.21.15.3:0/2844259737 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3800f040 con 0x7f8a4009f650 2023-10-03T10:15:15.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.599+0000 7f8a476a4700 1 -- 172.21.15.3:0/2844259737 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a400a0190 con 0x7f8a4009f650 2023-10-03T10:15:15.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.600+0000 7f8a35ffb700 1 -- 172.21.15.3:0/2844259737 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3800ced0 con 0x7f8a4009f650 2023-10-03T10:15:15.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.600+0000 7f8a476a4700 1 -- 172.21.15.3:0/2844259737 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a401a6930 con 0x7f8a4009f650 2023-10-03T10:15:15.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.600+0000 7f8a35ffb700 1 -- 172.21.15.3:0/2844259737 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8a38014480 con 0x7f8a4009f650 2023-10-03T10:15:15.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.601+0000 7f8a35ffb700 1 -- 172.21.15.3:0/2844259737 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8a3803fe70 con 0x7f8a4009f650 2023-10-03T10:15:15.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.601+0000 7f8a476a4700 1 -- 172.21.15.3:0/2844259737 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a40074ea0 con 0x7f8a4009f650 2023-10-03T10:15:15.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.606+0000 7f8a35ffb700 1 -- 172.21.15.3:0/2844259737 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8a38018020 con 0x7f8a4009f650 2023-10-03T10:15:15.614 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:15.657 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:15.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.717+0000 7fcaf83d6700 1 Processor -- start 2023-10-03T10:15:15.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.717+0000 7fcaf83d6700 1 -- start start 2023-10-03T10:15:15.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.717+0000 7fcaf83d6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcaf01338a0 0x7fcaf0133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:15.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.717+0000 7fcaf83d6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcaf009e930 0x7fcaf013b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:15.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.717+0000 7fcaf83d6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf006ef30 con 0x7fcaf009de50 2023-10-03T10:15:15.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.717+0000 7fcaf83d6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf006fef0 con 0x7fcaf01338a0 2023-10-03T10:15:15.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.717+0000 7fcaf83d6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf01334b0 con 0x7fcaf009e930 2023-10-03T10:15:15.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.718+0000 7fcaf6172700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcaf01338a0 0x7fcaf0133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:15.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.718+0000 7fcaf6172700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcaf01338a0 0x7fcaf0133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42368/0 (socket says 172.21.15.3:42368) 2023-10-03T10:15:15.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.718+0000 7fcaf6172700 1 -- 172.21.15.3:0/3762229876 learned_addr learned my addr 172.21.15.3:0/3762229876 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:15.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.718+0000 7fcaf6973700 1 --2- 172.21.15.3:0/3762229876 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcaf009e930 0x7fcaf013b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:15.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.718+0000 7fcaf6172700 1 -- 172.21.15.3:0/3762229876 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcaf009e930 msgr2=0x7fcaf013b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:15.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.718+0000 7fcaf6172700 1 --2- 172.21.15.3:0/3762229876 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcaf009e930 0x7fcaf013b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.718+0000 7fcaf6172700 1 -- 172.21.15.3:0/3762229876 >> v1:172.21.15.3:6789/0 conn(0x7fcaf009de50 legacy=0x7fcaf009e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:15.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.718+0000 7fcaf6172700 1 -- 172.21.15.3:0/3762229876 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf006fc30 con 0x7fcaf01338a0 2023-10-03T10:15:15.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.718+0000 7fcaf5170700 1 -- 172.21.15.3:0/3762229876 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4210743416 0 0) 0x7fcaf006ef30 con 0x7fcaf009de50 2023-10-03T10:15:15.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.718+0000 7fcaf6973700 1 --2- 172.21.15.3:0/3762229876 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcaf009e930 0x7fcaf013b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:15.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.719+0000 7fcaf6172700 1 --2- 172.21.15.3:0/3762229876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaf01338a0 0x7fcaf0133c70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0098f0 tx=0x7fcaec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f616af0fbe279930 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:15.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.719+0000 7fcaf5170700 1 -- 172.21.15.3:0/3762229876 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec011830 con 0x7fcaf01338a0 2023-10-03T10:15:15.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.719+0000 7fcaf5170700 1 -- 172.21.15.3:0/3762229876 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaec011990 con 0x7fcaf01338a0 2023-10-03T10:15:15.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.719+0000 7fcaf5170700 1 -- 172.21.15.3:0/3762229876 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec011d20 con 0x7fcaf01338a0 2023-10-03T10:15:15.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.719+0000 7fcaf83d6700 1 -- 172.21.15.3:0/3762229876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaf01338a0 msgr2=0x7fcaf0133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:15.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.719+0000 7fcaf83d6700 1 --2- 172.21.15.3:0/3762229876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaf01338a0 0x7fcaf0133c70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0098f0 tx=0x7fcaec009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:15.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.720+0000 7fcaf83d6700 1 -- 172.21.15.3:0/3762229876 shutdown_connections 2023-10-03T10:15:15.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.720+0000 7fcaf83d6700 1 --2- 172.21.15.3:0/3762229876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaf01338a0 0x7fcaf0133c70 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.720+0000 7fcaf83d6700 1 --2- 172.21.15.3:0/3762229876 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcaf009e930 0x7fcaf013b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.720+0000 7fcaf83d6700 1 -- 172.21.15.3:0/3762229876 >> 172.21.15.3:0/3762229876 conn(0x7fcaf00a2aa0 msgr2=0x7fcaf00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:15.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.720+0000 7fcaf83d6700 1 -- 172.21.15.3:0/3762229876 shutdown_connections 2023-10-03T10:15:15.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.720+0000 7fcaf83d6700 1 -- 172.21.15.3:0/3762229876 wait complete. 2023-10-03T10:15:15.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.721+0000 7fcaf83d6700 1 Processor -- start 2023-10-03T10:15:15.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.721+0000 7fcaf83d6700 1 -- start start 2023-10-03T10:15:15.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.721+0000 7fcaf83d6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcaf009de50 0x7fcaf011d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:15.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.721+0000 7fcaf83d6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaf009e930 0x7fcaf011dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:15.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.721+0000 7fcaf83d6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcaf01338a0 0x7fcaf011e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:15.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.721+0000 7fcaf83d6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf011f320 con 0x7fcaf009e930 2023-10-03T10:15:15.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.721+0000 7fcaf83d6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf0120570 con 0x7fcaf01338a0 2023-10-03T10:15:15.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.721+0000 7fcaf83d6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf011f910 con 0x7fcaf009de50 2023-10-03T10:15:15.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.722+0000 7fcaf6172700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcaf009de50 0x7fcaf011d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:15.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.722+0000 7fcaf6172700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcaf009de50 0x7fcaf011d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53766/0 (socket says 172.21.15.3:53766) 2023-10-03T10:15:15.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.722+0000 7fcaf6172700 1 -- 172.21.15.3:0/1877716478 learned_addr learned my addr 172.21.15.3:0/1877716478 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:15.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.722+0000 7fcaf5971700 1 --2- 172.21.15.3:0/1877716478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaf009e930 0x7fcaf011dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:15.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.722+0000 7fcaf6973700 1 --2- 172.21.15.3:0/1877716478 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcaf01338a0 0x7fcaf011e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:15.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.722+0000 7fcaf5971700 1 -- 172.21.15.3:0/1877716478 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcaf009de50 msgr2=0x7fcaf011d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:15.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.722+0000 7fcaf5971700 1 --2- 172.21.15.3:0/1877716478 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcaf009de50 0x7fcaf011d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.722+0000 7fcaf5971700 1 -- 172.21.15.3:0/1877716478 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcaf01338a0 msgr2=0x7fcaf011e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:15.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.722+0000 7fcaf5971700 1 --2- 172.21.15.3:0/1877716478 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcaf01338a0 0x7fcaf011e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.722+0000 7fcaf5971700 1 -- 172.21.15.3:0/1877716478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae8009a70 con 0x7fcaf009e930 2023-10-03T10:15:15.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.722+0000 7fcaf6973700 1 --2- 172.21.15.3:0/1877716478 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcaf01338a0 0x7fcaf011e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:15:15.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.722+0000 7fcaf5971700 1 --2- 172.21.15.3:0/1877716478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaf009e930 0x7fcaf011dd80 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fcae000c3e0 tx=0x7fcae000c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:15.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.722+0000 7fcae6ffd700 1 -- 172.21.15.3:0/1877716478 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae000cea0 con 0x7fcaf009e930 2023-10-03T10:15:15.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.722+0000 7fcae6ffd700 1 -- 172.21.15.3:0/1877716478 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae00054e0 con 0x7fcaf009e930 2023-10-03T10:15:15.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.722+0000 7fcae6ffd700 1 -- 172.21.15.3:0/1877716478 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae00038a0 con 0x7fcaf009e930 2023-10-03T10:15:15.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.723+0000 7fcaf83d6700 1 -- 172.21.15.3:0/1877716478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaec009550 con 0x7fcaf009e930 2023-10-03T10:15:15.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.723+0000 7fcaf83d6700 1 -- 172.21.15.3:0/1877716478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf006f640 con 0x7fcaf009e930 2023-10-03T10:15:15.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.724+0000 7fcae6ffd700 1 -- 172.21.15.3:0/1877716478 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcae0003a00 con 0x7fcaf009e930 2023-10-03T10:15:15.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.724+0000 7fcaf83d6700 1 -- 172.21.15.3:0/1877716478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf0074ea0 con 0x7fcaf009e930 2023-10-03T10:15:15.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.724+0000 7fcae6ffd700 1 -- 172.21.15.3:0/1877716478 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fcae003ebb0 con 0x7fcaf009e930 2023-10-03T10:15:15.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.734+0000 7fcae6ffd700 1 -- 172.21.15.3:0/1877716478 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcae0017070 con 0x7fcaf009e930 2023-10-03T10:15:15.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.762+0000 7efcb2b67700 1 Processor -- start 2023-10-03T10:15:15.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.762+0000 7efcb2b67700 1 -- start start 2023-10-03T10:15:15.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.763+0000 7efcb2b67700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efcac133540 0x7efcac135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:15.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.763+0000 7efcb2b67700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efcac135e60 0x7efcac138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:15.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.763+0000 7efcb2b67700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcac0716a0 con 0x7efcac130540 2023-10-03T10:15:15.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.763+0000 7efcb2b67700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcac071290 con 0x7efcac135e60 2023-10-03T10:15:15.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.763+0000 7efcb2b67700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcac130340 con 0x7efcac133540 2023-10-03T10:15:15.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.763+0000 7efcb1104700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efcac135e60 0x7efcac138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:15.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.763+0000 7efcb1104700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efcac135e60 0x7efcac138240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42392/0 (socket says 172.21.15.3:42392) 2023-10-03T10:15:15.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.763+0000 7efcb1104700 1 -- 172.21.15.3:0/4285238882 learned_addr learned my addr 172.21.15.3:0/4285238882 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:15.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.763+0000 7efcb0903700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7efcac130540 0x7efcac130910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47642/0 (socket says 172.21.15.3:47642) 2023-10-03T10:15:15.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.763+0000 7efcab7fe700 1 -- 172.21.15.3:0/4285238882 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3617335176 0 0) 0x7efcac0716a0 con 0x7efcac130540 2023-10-03T10:15:15.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.763+0000 7efcabfff700 1 --2- 172.21.15.3:0/4285238882 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efcac133540 0x7efcac135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:15.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.764+0000 7efcb1104700 1 -- 172.21.15.3:0/4285238882 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efcac133540 msgr2=0x7efcac135920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:15.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.764+0000 7efcb1104700 1 --2- 172.21.15.3:0/4285238882 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efcac133540 0x7efcac135920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.764+0000 7efcb1104700 1 -- 172.21.15.3:0/4285238882 >> v1:172.21.15.3:6789/0 conn(0x7efcac130540 legacy=0x7efcac130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:15.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.764+0000 7efcb1104700 1 -- 172.21.15.3:0/4285238882 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac070b80 con 0x7efcac135e60 2023-10-03T10:15:15.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.764+0000 7efcabfff700 1 --2- 172.21.15.3:0/4285238882 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efcac133540 0x7efcac135920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:15:15.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.764+0000 7efcb1104700 1 --2- 172.21.15.3:0/4285238882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcac135e60 0x7efcac138240 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7efca400d150 tx=0x7efca400d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe3582925d7866a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:15.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.764+0000 7efcab7fe700 1 -- 172.21.15.3:0/4285238882 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca4010070 con 0x7efcac135e60 2023-10-03T10:15:15.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.764+0000 7efcab7fe700 1 -- 172.21.15.3:0/4285238882 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca400ddc0 con 0x7efcac135e60 2023-10-03T10:15:15.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.764+0000 7efcab7fe700 1 -- 172.21.15.3:0/4285238882 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca400b310 con 0x7efcac135e60 2023-10-03T10:15:15.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.765+0000 7efcb2b67700 1 -- 172.21.15.3:0/4285238882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcac135e60 msgr2=0x7efcac138240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:15.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.765+0000 7efcb2b67700 1 --2- 172.21.15.3:0/4285238882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcac135e60 0x7efcac138240 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7efca400d150 tx=0x7efca400d460 comp rx=0 tx=0).stop 2023-10-03T10:15:15.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.765+0000 7efcb2b67700 1 -- 172.21.15.3:0/4285238882 shutdown_connections 2023-10-03T10:15:15.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.765+0000 7efcb2b67700 1 --2- 172.21.15.3:0/4285238882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcac135e60 0x7efcac138240 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.765+0000 7efcb2b67700 1 --2- 172.21.15.3:0/4285238882 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efcac133540 0x7efcac135920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.765+0000 7efcb2b67700 1 -- 172.21.15.3:0/4285238882 >> 172.21.15.3:0/4285238882 conn(0x7efcac0a2aa0 msgr2=0x7efcac0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:15.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.765+0000 7efcb2b67700 1 -- 172.21.15.3:0/4285238882 shutdown_connections 2023-10-03T10:15:15.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.765+0000 7efcb2b67700 1 -- 172.21.15.3:0/4285238882 wait complete. 2023-10-03T10:15:15.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.767+0000 7efcb2b67700 1 Processor -- start 2023-10-03T10:15:15.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.767+0000 7efcb2b67700 1 -- start start 2023-10-03T10:15:15.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.768+0000 7efcb2b67700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efcac130540 0x7efcac09c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:15.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.768+0000 7efcb2b67700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efcac133540 0x7efcac09ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:15.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.768+0000 7efcb2b67700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcac135e60 0x7efcac09cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:15.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.768+0000 7efcb2b67700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcac11a450 con 0x7efcac135e60 2023-10-03T10:15:15.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.768+0000 7efcb2b67700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcac1359c0 con 0x7efcac133540 2023-10-03T10:15:15.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.768+0000 7efcb2b67700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efcac0718e0 con 0x7efcac130540 2023-10-03T10:15:15.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.768+0000 7efcabfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efcac133540 0x7efcac09ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:15.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.768+0000 7efcabfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efcac133540 0x7efcac09ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50512/0 (socket says 172.21.15.3:50512) 2023-10-03T10:15:15.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.768+0000 7efcabfff700 1 -- 172.21.15.3:0/2300924356 learned_addr learned my addr 172.21.15.3:0/2300924356 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:15.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.768+0000 7efcb1104700 1 --2- 172.21.15.3:0/2300924356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcac135e60 0x7efcac09cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:15.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.768+0000 7efcb0903700 1 --2- 172.21.15.3:0/2300924356 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efcac130540 0x7efcac09c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:15.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.769+0000 7efcabfff700 1 -- 172.21.15.3:0/2300924356 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efcac130540 msgr2=0x7efcac09c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:15.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.769+0000 7efcabfff700 1 --2- 172.21.15.3:0/2300924356 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efcac130540 0x7efcac09c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.769+0000 7efcabfff700 1 -- 172.21.15.3:0/2300924356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcac135e60 msgr2=0x7efcac09cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:15.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.769+0000 7efcabfff700 1 --2- 172.21.15.3:0/2300924356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcac135e60 0x7efcac09cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.769+0000 7efcabfff700 1 -- 172.21.15.3:0/2300924356 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca400bd40 con 0x7efcac133540 2023-10-03T10:15:15.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.769+0000 7efcabfff700 1 --2- 172.21.15.3:0/2300924356 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efcac133540 0x7efcac09ca50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7efc9800c750 tx=0x7efc9800ca60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:15.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.769+0000 7efca97fa700 1 -- 172.21.15.3:0/2300924356 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc98011800 con 0x7efcac133540 2023-10-03T10:15:15.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.769+0000 7efca97fa700 1 -- 172.21.15.3:0/2300924356 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc98011960 con 0x7efcac133540 2023-10-03T10:15:15.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.769+0000 7efcb2b67700 1 -- 172.21.15.3:0/2300924356 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcac119010 con 0x7efcac133540 2023-10-03T10:15:15.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.769+0000 7efca97fa700 1 -- 172.21.15.3:0/2300924356 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc98011cd0 con 0x7efcac133540 2023-10-03T10:15:15.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.769+0000 7efcb2b67700 1 -- 172.21.15.3:0/2300924356 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcac11a730 con 0x7efcac133540 2023-10-03T10:15:15.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.770+0000 7efcb2b67700 1 -- 172.21.15.3:0/2300924356 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcac06d5a0 con 0x7efcac133540 2023-10-03T10:15:15.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.770+0000 7efca97fa700 1 -- 172.21.15.3:0/2300924356 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efc98021350 con 0x7efcac133540 2023-10-03T10:15:15.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.771+0000 7efca97fa700 1 -- 172.21.15.3:0/2300924356 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7efc9803f680 con 0x7efcac133540 2023-10-03T10:15:15.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.780+0000 7efca97fa700 1 -- 172.21.15.3:0/2300924356 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efc98011e30 con 0x7efcac133540 2023-10-03T10:15:15.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.811+0000 7f8988b0f700 1 Processor -- start 2023-10-03T10:15:15.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.811+0000 7f8988b0f700 1 -- start start 2023-10-03T10:15:15.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.812+0000 7f8988b0f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f898409e1d0 0x7f8984135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:15.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.812+0000 7f8988b0f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8984135e50 0x7f8984138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:15.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.812+0000 7f8988b0f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f898409f130 con 0x7f898409d6f0 2023-10-03T10:15:15.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.812+0000 7f8988b0f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f898409f520 con 0x7f8984135e50 2023-10-03T10:15:15.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.812+0000 7f8988b0f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f898409e930 con 0x7f898409e1d0 2023-10-03T10:15:15.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.812+0000 7f897bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f898409e1d0 0x7f8984135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:15.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.812+0000 7f897bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f898409e1d0 0x7f8984135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50524/0 (socket says 172.21.15.3:50524) 2023-10-03T10:15:15.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.812+0000 7f897bfff700 1 -- 172.21.15.3:0/1304381335 learned_addr learned my addr 172.21.15.3:0/1304381335 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:15.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.812+0000 7f8982d9d700 1 --2- 172.21.15.3:0/1304381335 >> v2:172.21.15.3:3300/0 conn(0x7f8984135e50 0x7f8984138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:15.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.812+0000 7f897bfff700 1 -- 172.21.15.3:0/1304381335 >> v2:172.21.15.3:3300/0 conn(0x7f8984135e50 msgr2=0x7f8984138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:15.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.812+0000 7f897bfff700 1 --2- 172.21.15.3:0/1304381335 >> v2:172.21.15.3:3300/0 conn(0x7f8984135e50 0x7f8984138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.812+0000 7f897bfff700 1 -- 172.21.15.3:0/1304381335 >> v1:172.21.15.3:6789/0 conn(0x7f898409d6f0 legacy=0x7f898409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:15.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.812+0000 7f897bfff700 1 -- 172.21.15.3:0/1304381335 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898409b2d0 con 0x7f898409e1d0 2023-10-03T10:15:15.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.812+0000 7f897bfff700 1 --2- 172.21.15.3:0/1304381335 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f898409e1d0 0x7f8984135910 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f896c00b9d0 tx=0x7f896c00bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bfa30cf1d4eb605 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:15.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.813+0000 7f8981d9b700 1 -- 172.21.15.3:0/1304381335 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c00cc20 con 0x7f898409e1d0 2023-10-03T10:15:15.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.813+0000 7f8981d9b700 1 -- 172.21.15.3:0/1304381335 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f896c00cd80 con 0x7f898409e1d0 2023-10-03T10:15:15.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.813+0000 7f8981d9b700 1 -- 172.21.15.3:0/1304381335 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c00f680 con 0x7f898409e1d0 2023-10-03T10:15:15.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.813+0000 7f8988b0f700 1 -- 172.21.15.3:0/1304381335 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f898409e1d0 msgr2=0x7f8984135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:15.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.813+0000 7f8988b0f700 1 --2- 172.21.15.3:0/1304381335 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f898409e1d0 0x7f8984135910 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f896c00b9d0 tx=0x7f896c00bce0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.813+0000 7f8988b0f700 1 -- 172.21.15.3:0/1304381335 shutdown_connections 2023-10-03T10:15:15.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.813+0000 7f8988b0f700 1 --2- 172.21.15.3:0/1304381335 >> v2:172.21.15.3:3300/0 conn(0x7f8984135e50 0x7f8984138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.813+0000 7f8988b0f700 1 --2- 172.21.15.3:0/1304381335 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f898409e1d0 0x7f8984135910 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.813+0000 7f8988b0f700 1 -- 172.21.15.3:0/1304381335 >> 172.21.15.3:0/1304381335 conn(0x7f8984066f50 msgr2=0x7f898409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:15.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.814+0000 7f8988b0f700 1 -- 172.21.15.3:0/1304381335 shutdown_connections 2023-10-03T10:15:15.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.814+0000 7f8988b0f700 1 -- 172.21.15.3:0/1304381335 wait complete. 2023-10-03T10:15:15.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.815+0000 7f8988b0f700 1 Processor -- start 2023-10-03T10:15:15.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.815+0000 7f8988b0f700 1 -- start start 2023-10-03T10:15:15.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.815+0000 7f8988b0f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f898409d6f0 0x7f8984119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:15.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.816+0000 7f8988b0f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f898409e1d0 0x7f89841199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:15.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.816+0000 7f8988b0f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8984135e50 0x7f8984119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:15.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.816+0000 7f8988b0f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898409c700 con 0x7f898409d6f0 2023-10-03T10:15:15.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.816+0000 7f8988b0f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f898411b0f0 con 0x7f898409e1d0 2023-10-03T10:15:15.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.816+0000 7f8988b0f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89840a1e20 con 0x7f8984135e50 2023-10-03T10:15:15.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.816+0000 7f898259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f898409d6f0 0x7f8984119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:15.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.816+0000 7f898259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f898409d6f0 0x7f8984119480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42412/0 (socket says 172.21.15.3:42412) 2023-10-03T10:15:15.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.816+0000 7f898259c700 1 -- 172.21.15.3:0/43263989 learned_addr learned my addr 172.21.15.3:0/43263989 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:15.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.816+0000 7f8982d9d700 1 --2- 172.21.15.3:0/43263989 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8984135e50 0x7f8984119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:15.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.816+0000 7f897bfff700 1 --2- 172.21.15.3:0/43263989 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f898409e1d0 0x7f89841199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:15.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.817+0000 7f897bfff700 1 -- 172.21.15.3:0/43263989 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8984135e50 msgr2=0x7f8984119f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:15.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.817+0000 7f897bfff700 1 --2- 172.21.15.3:0/43263989 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8984135e50 0x7f8984119f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.817+0000 7f897bfff700 1 -- 172.21.15.3:0/43263989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f898409d6f0 msgr2=0x7f8984119480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:15.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.817+0000 7f897bfff700 1 --2- 172.21.15.3:0/43263989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f898409d6f0 0x7f8984119480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.817+0000 7f897bfff700 1 -- 172.21.15.3:0/43263989 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f896c00b600 con 0x7f898409e1d0 2023-10-03T10:15:15.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.817+0000 7f897bfff700 1 --2- 172.21.15.3:0/43263989 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f898409e1d0 0x7f89841199c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f896c016000 tx=0x7f896c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:15.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.817+0000 7f897affd700 1 -- 172.21.15.3:0/43263989 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c01c070 con 0x7f898409e1d0 2023-10-03T10:15:15.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.817+0000 7f897affd700 1 -- 172.21.15.3:0/43263989 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f896c00be70 con 0x7f898409e1d0 2023-10-03T10:15:15.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.817+0000 7f8988b0f700 1 -- 172.21.15.3:0/43263989 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f898411b7b0 con 0x7f898409e1d0 2023-10-03T10:15:15.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.817+0000 7f897affd700 1 -- 172.21.15.3:0/43263989 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c004270 con 0x7f898409e1d0 2023-10-03T10:15:15.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.817+0000 7f8988b0f700 1 -- 172.21.15.3:0/43263989 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898411d3f0 con 0x7f898409e1d0 2023-10-03T10:15:15.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.818+0000 7f897affd700 1 -- 172.21.15.3:0/43263989 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f896c0043d0 con 0x7f898409e1d0 2023-10-03T10:15:15.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.819+0000 7f897affd700 1 -- 172.21.15.3:0/43263989 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f896c040da0 con 0x7f898409e1d0 2023-10-03T10:15:15.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.819+0000 7f8988b0f700 1 -- 172.21.15.3:0/43263989 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8984074ea0 con 0x7f898409e1d0 2023-10-03T10:15:15.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.829+0000 7f897affd700 1 -- 172.21.15.3:0/43263989 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f896c017400 con 0x7f898409e1d0 2023-10-03T10:15:15.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.835+0000 7f8a476a4700 1 -- 172.21.15.3:0/2844259737 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8a401a56e0 con 0x7f8a4009f650 2023-10-03T10:15:15.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.835+0000 7f8a35ffb700 1 -- 172.21.15.3:0/2844259737 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8a3801a680 con 0x7f8a4009f650 2023-10-03T10:15:15.836 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:15.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.838+0000 7f8a476a4700 1 -- 172.21.15.3:0/2844259737 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a4009f650 msgr2=0x7f8a401c5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:15.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.838+0000 7f8a476a4700 1 --2- 172.21.15.3:0/2844259737 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a4009f650 0x7f8a401c5e40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8a3800efc0 tx=0x7f8a3800c3a0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.838+0000 7f8a476a4700 1 -- 172.21.15.3:0/2844259737 shutdown_connections 2023-10-03T10:15:15.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.838+0000 7f8a476a4700 1 --2- 172.21.15.3:0/2844259737 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a4009f650 0x7f8a401c5e40 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.838+0000 7f8a476a4700 1 --2- 172.21.15.3:0/2844259737 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a4009e930 0x7f8a401c2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.838+0000 7f8a476a4700 1 --2- 172.21.15.3:0/2844259737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a4009f240 0x7f8a401c3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.838+0000 7f8a476a4700 1 -- 172.21.15.3:0/2844259737 >> 172.21.15.3:0/2844259737 conn(0x7f8a400a4210 msgr2=0x7f8a401b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:15.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.838+0000 7f8a476a4700 1 -- 172.21.15.3:0/2844259737 shutdown_connections 2023-10-03T10:15:15.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.838+0000 7f8a476a4700 1 -- 172.21.15.3:0/2844259737 wait complete. 2023-10-03T10:15:15.853 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:15.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.969+0000 7fcaf83d6700 1 -- 172.21.15.3:0/1877716478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcaf006b9e0 con 0x7fcaf009e930 2023-10-03T10:15:15.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.969+0000 7fcae6ffd700 1 -- 172.21.15.3:0/1877716478 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcae0017020 con 0x7fcaf009e930 2023-10-03T10:15:15.970 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:15.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.972+0000 7fcaf83d6700 1 -- 172.21.15.3:0/1877716478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaf009e930 msgr2=0x7fcaf011dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:15.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.972+0000 7fcaf83d6700 1 --2- 172.21.15.3:0/1877716478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaf009e930 0x7fcaf011dd80 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fcae000c3e0 tx=0x7fcae000c6f0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.972+0000 7fcaf83d6700 1 -- 172.21.15.3:0/1877716478 shutdown_connections 2023-10-03T10:15:15.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.972+0000 7fcaf83d6700 1 --2- 172.21.15.3:0/1877716478 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcaf01338a0 0x7fcaf011e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.972+0000 7fcaf83d6700 1 --2- 172.21.15.3:0/1877716478 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcaf009de50 0x7fcaf011d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.972+0000 7fcaf83d6700 1 --2- 172.21.15.3:0/1877716478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaf009e930 0x7fcaf011dd80 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:15.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.972+0000 7fcaf83d6700 1 -- 172.21.15.3:0/1877716478 >> 172.21.15.3:0/1877716478 conn(0x7fcaf00a2aa0 msgr2=0x7fcaf01b9410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:15.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.972+0000 7fcaf83d6700 1 -- 172.21.15.3:0/1877716478 shutdown_connections 2023-10-03T10:15:15.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.972+0000 7fcaf83d6700 1 -- 172.21.15.3:0/1877716478 wait complete. 2023-10-03T10:15:15.986 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:15.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.996+0000 7efcb2b67700 1 -- 172.21.15.3:0/2300924356 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7efcac11a0f0 con 0x7efcac133540 2023-10-03T10:15:15.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.997+0000 7efca97fa700 1 -- 172.21.15.3:0/2300924356 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efcac11a0f0 con 0x7efcac133540 2023-10-03T10:15:15.997 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:15.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.999+0000 7efcb2b67700 1 -- 172.21.15.3:0/2300924356 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efcac133540 msgr2=0x7efcac09ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:16.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.999+0000 7efcb2b67700 1 --2- 172.21.15.3:0/2300924356 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efcac133540 0x7efcac09ca50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7efc9800c750 tx=0x7efc9800ca60 comp rx=0 tx=0).stop 2023-10-03T10:15:16.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.999+0000 7efcb2b67700 1 -- 172.21.15.3:0/2300924356 shutdown_connections 2023-10-03T10:15:16.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.999+0000 7efcb2b67700 1 --2- 172.21.15.3:0/2300924356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcac135e60 0x7efcac09cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:16.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.999+0000 7efcb2b67700 1 --2- 172.21.15.3:0/2300924356 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efcac130540 0x7efcac09c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:16.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.999+0000 7efcb2b67700 1 --2- 172.21.15.3:0/2300924356 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efcac133540 0x7efcac09ca50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:16.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.999+0000 7efcb2b67700 1 -- 172.21.15.3:0/2300924356 >> 172.21.15.3:0/2300924356 conn(0x7efcac0a2aa0 msgr2=0x7efcac131300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:16.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.999+0000 7efcb2b67700 1 -- 172.21.15.3:0/2300924356 shutdown_connections 2023-10-03T10:15:16.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:15.999+0000 7efcb2b67700 1 -- 172.21.15.3:0/2300924356 wait complete. 2023-10-03T10:15:16.013 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:16.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:16.037+0000 7f8988b0f700 1 -- 172.21.15.3:0/43263989 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f898411d6a0 con 0x7f898409e1d0 2023-10-03T10:15:16.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:16.037+0000 7f897affd700 1 -- 172.21.15.3:0/43263989 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f896c026020 con 0x7f898409e1d0 2023-10-03T10:15:16.038 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:16.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:16.039+0000 7f8988b0f700 1 -- 172.21.15.3:0/43263989 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f898409e1d0 msgr2=0x7f89841199c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:16.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:16.039+0000 7f8988b0f700 1 --2- 172.21.15.3:0/43263989 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f898409e1d0 0x7f89841199c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f896c016000 tx=0x7f896c004750 comp rx=0 tx=0).stop 2023-10-03T10:15:16.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:16.039+0000 7f8988b0f700 1 -- 172.21.15.3:0/43263989 shutdown_connections 2023-10-03T10:15:16.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:16.039+0000 7f8988b0f700 1 --2- 172.21.15.3:0/43263989 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8984135e50 0x7f8984119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:16.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:16.039+0000 7f8988b0f700 1 --2- 172.21.15.3:0/43263989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f898409d6f0 0x7f8984119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:16.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:16.039+0000 7f8988b0f700 1 --2- 172.21.15.3:0/43263989 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f898409e1d0 0x7f89841199c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:16.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:16.039+0000 7f8988b0f700 1 -- 172.21.15.3:0/43263989 >> 172.21.15.3:0/43263989 conn(0x7f8984066f50 msgr2=0x7f8984132a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:16.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:16.039+0000 7f8988b0f700 1 -- 172.21.15.3:0/43263989 shutdown_connections 2023-10-03T10:15:16.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:16.039+0000 7f8988b0f700 1 -- 172.21.15.3:0/43263989 wait complete. 2023-10-03T10:15:16.053 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:16.853 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:16.987 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:16.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:16.999+0000 7f67d8a48700 1 Processor -- start 2023-10-03T10:15:16.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:16.999+0000 7f67d8a48700 1 -- start start 2023-10-03T10:15:17.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.000+0000 7f67d8a48700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f67d406b250 0x7f67d4136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:17.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.000+0000 7f67d8a48700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67d4136d80 0x7f67d4139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:17.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.000+0000 7f67d8a48700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67d409ece0 con 0x7f67d406a770 2023-10-03T10:15:17.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.000+0000 7f67d8a48700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d409e760 con 0x7f67d406b250 2023-10-03T10:15:17.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.000+0000 7f67d8a48700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67d409e040 con 0x7f67d4136d80 2023-10-03T10:15:17.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.000+0000 7f67d1d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f67d406b250 0x7f67d4136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:17.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.000+0000 7f67d2d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67d4136d80 0x7f67d4139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:17.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.000+0000 7f67d1d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f67d406b250 0x7f67d4136840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42416/0 (socket says 172.21.15.3:42416) 2023-10-03T10:15:17.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.000+0000 7f67d1d9b700 1 -- 172.21.15.3:0/3138867145 learned_addr learned my addr 172.21.15.3:0/3138867145 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:17.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.000+0000 7f67d2d9d700 1 -- 172.21.15.3:0/3138867145 >> v2:172.21.15.3:3300/0 conn(0x7f67d406b250 msgr2=0x7f67d4136840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:17.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.000+0000 7f67d2d9d700 1 --2- 172.21.15.3:0/3138867145 >> v2:172.21.15.3:3300/0 conn(0x7f67d406b250 0x7f67d4136840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.000+0000 7f67d2d9d700 1 -- 172.21.15.3:0/3138867145 >> v1:172.21.15.3:6789/0 conn(0x7f67d406a770 legacy=0x7f67d406ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:17.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.000+0000 7f67d2d9d700 1 -- 172.21.15.3:0/3138867145 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d409f630 con 0x7f67d4136d80 2023-10-03T10:15:17.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.001+0000 7f67d2d9d700 1 --2- 172.21.15.3:0/3138867145 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67d4136d80 0x7f67d4139160 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009d90 tx=0x7f67cc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67143e7ca7d57777 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:17.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.001+0000 7f67d159a700 1 -- 172.21.15.3:0/3138867145 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67cc00e070 con 0x7f67d4136d80 2023-10-03T10:15:17.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.001+0000 7f67d159a700 1 -- 172.21.15.3:0/3138867145 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67cc00bb20 con 0x7f67d4136d80 2023-10-03T10:15:17.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.001+0000 7f67d159a700 1 -- 172.21.15.3:0/3138867145 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67cc012430 con 0x7f67d4136d80 2023-10-03T10:15:17.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.001+0000 7f67d8a48700 1 -- 172.21.15.3:0/3138867145 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67d4136d80 msgr2=0x7f67d4139160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:17.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.001+0000 7f67d8a48700 1 --2- 172.21.15.3:0/3138867145 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67d4136d80 0x7f67d4139160 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009d90 tx=0x7f67cc00b330 comp rx=0 tx=0).stop 2023-10-03T10:15:17.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.002+0000 7f67d8a48700 1 -- 172.21.15.3:0/3138867145 shutdown_connections 2023-10-03T10:15:17.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.002+0000 7f67d8a48700 1 --2- 172.21.15.3:0/3138867145 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67d4136d80 0x7f67d4139160 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.002+0000 7f67d8a48700 1 --2- 172.21.15.3:0/3138867145 >> v2:172.21.15.3:3300/0 conn(0x7f67d406b250 0x7f67d4136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.002+0000 7f67d8a48700 1 -- 172.21.15.3:0/3138867145 >> 172.21.15.3:0/3138867145 conn(0x7f67d4066f50 msgr2=0x7f67d409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:17.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.002+0000 7f67d8a48700 1 -- 172.21.15.3:0/3138867145 shutdown_connections 2023-10-03T10:15:17.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.002+0000 7f67d8a48700 1 -- 172.21.15.3:0/3138867145 wait complete. 2023-10-03T10:15:17.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.003+0000 7f67d8a48700 1 Processor -- start 2023-10-03T10:15:17.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.003+0000 7f67d8a48700 1 -- start start 2023-10-03T10:15:17.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.003+0000 7f67d8a48700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67d406a770 0x7f67d4122470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:17.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.003+0000 7f67d8a48700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67d406b250 0x7f67d41229b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:17.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.003+0000 7f67d8a48700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67d4136d80 0x7f67d41c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:17.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.003+0000 7f67d8a48700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d4125590 con 0x7f67d406a770 2023-10-03T10:15:17.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.003+0000 7f67d8a48700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d4121ed0 con 0x7f67d406b250 2023-10-03T10:15:17.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.003+0000 7f67d8a48700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67d409c250 con 0x7f67d4136d80 2023-10-03T10:15:17.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.004+0000 7f67d2d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67d4136d80 0x7f67d41c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:17.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.004+0000 7f67d2d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67d4136d80 0x7f67d41c81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53800/0 (socket says 172.21.15.3:53800) 2023-10-03T10:15:17.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.004+0000 7f67d2d9d700 1 -- 172.21.15.3:0/339974834 learned_addr learned my addr 172.21.15.3:0/339974834 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:17.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.004+0000 7f67d1d9b700 1 --2- 172.21.15.3:0/339974834 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67d406b250 0x7f67d41229b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:17.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.004+0000 7f67d259c700 1 --2- 172.21.15.3:0/339974834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67d406a770 0x7f67d4122470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:17.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.004+0000 7f67d2d9d700 1 -- 172.21.15.3:0/339974834 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67d406b250 msgr2=0x7f67d41229b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:17.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.004+0000 7f67d2d9d700 1 --2- 172.21.15.3:0/339974834 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67d406b250 0x7f67d41229b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.004+0000 7f67d2d9d700 1 -- 172.21.15.3:0/339974834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67d406a770 msgr2=0x7f67d4122470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:17.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.004+0000 7f67d2d9d700 1 --2- 172.21.15.3:0/339974834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67d406a770 0x7f67d4122470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.004+0000 7f67d2d9d700 1 -- 172.21.15.3:0/339974834 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67cc009a70 con 0x7f67d4136d80 2023-10-03T10:15:17.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.004+0000 7f67d2d9d700 1 --2- 172.21.15.3:0/339974834 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67d4136d80 0x7f67d41c81a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f67d409e180 tx=0x7f67cc004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:17.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.004+0000 7f67c2ffd700 1 -- 172.21.15.3:0/339974834 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67cc012430 con 0x7f67d4136d80 2023-10-03T10:15:17.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.004+0000 7f67c2ffd700 1 -- 172.21.15.3:0/339974834 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67cc020070 con 0x7f67d4136d80 2023-10-03T10:15:17.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.004+0000 7f67c2ffd700 1 -- 172.21.15.3:0/339974834 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67cc003ea0 con 0x7f67d4136d80 2023-10-03T10:15:17.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.004+0000 7f67d8a48700 1 -- 172.21.15.3:0/339974834 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d4071e60 con 0x7f67d4136d80 2023-10-03T10:15:17.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.005+0000 7f67d8a48700 1 -- 172.21.15.3:0/339974834 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d4123580 con 0x7f67d4136d80 2023-10-03T10:15:17.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.005+0000 7f67c2ffd700 1 -- 172.21.15.3:0/339974834 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f67cc004040 con 0x7f67d4136d80 2023-10-03T10:15:17.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.005+0000 7f67d8a48700 1 -- 172.21.15.3:0/339974834 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d41253d0 con 0x7f67d4136d80 2023-10-03T10:15:17.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.006+0000 7f67c2ffd700 1 -- 172.21.15.3:0/339974834 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f67cc022070 con 0x7f67d4136d80 2023-10-03T10:15:17.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.011+0000 7f67c2ffd700 1 -- 172.21.15.3:0/339974834 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f67d41253d0 con 0x7f67d4136d80 2023-10-03T10:15:17.014 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:17.053 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:17.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.132+0000 7f88c6cd6700 1 Processor -- start 2023-10-03T10:15:17.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.132+0000 7f88c6cd6700 1 -- start start 2023-10-03T10:15:17.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.132+0000 7f88c6cd6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88c012cf60 0x7f88c006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:17.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.132+0000 7f88c6cd6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88c012f8b0 0x7f88c012fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:17.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.132+0000 7f88c6cd6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c012cd90 con 0x7f88c0130260 2023-10-03T10:15:17.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.132+0000 7f88c6cd6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c012d330 con 0x7f88c012f8b0 2023-10-03T10:15:17.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.132+0000 7f88c6cd6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f88c01306d0 con 0x7f88c012cf60 2023-10-03T10:15:17.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.132+0000 7f88bffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88c012f8b0 0x7f88c012fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:17.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.133+0000 7f88bffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88c012f8b0 0x7f88c012fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50552/0 (socket says 172.21.15.3:50552) 2023-10-03T10:15:17.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.133+0000 7f88bffff700 1 -- 172.21.15.3:0/3472158779 learned_addr learned my addr 172.21.15.3:0/3472158779 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:17.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.133+0000 7f88c4a72700 1 --2- 172.21.15.3:0/3472158779 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88c012cf60 0x7f88c006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:17.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.133+0000 7f88bffff700 1 -- 172.21.15.3:0/3472158779 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88c012cf60 msgr2=0x7f88c006f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:17.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.133+0000 7f88bffff700 1 --2- 172.21.15.3:0/3472158779 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88c012cf60 0x7f88c006f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.133+0000 7f88bffff700 1 -- 172.21.15.3:0/3472158779 >> v1:172.21.15.3:6789/0 conn(0x7f88c0130260 legacy=0x7f88c0136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:17.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.133+0000 7f88bffff700 1 -- 172.21.15.3:0/3472158779 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c0071780 con 0x7f88c012f8b0 2023-10-03T10:15:17.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.133+0000 7f88bffff700 1 --2- 172.21.15.3:0/3472158779 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88c012f8b0 0x7f88c012fd20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00aac0 tx=0x7f88ac00add0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7caf190c24706b41 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:17.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.133+0000 7f88bf7fe700 1 -- 172.21.15.3:0/3472158779 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac00cb70 con 0x7f88c012f8b0 2023-10-03T10:15:17.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.133+0000 7f88bf7fe700 1 -- 172.21.15.3:0/3472158779 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ac00ccd0 con 0x7f88c012f8b0 2023-10-03T10:15:17.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.133+0000 7f88bf7fe700 1 -- 172.21.15.3:0/3472158779 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac010640 con 0x7f88c012f8b0 2023-10-03T10:15:17.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.134+0000 7f88c6cd6700 1 -- 172.21.15.3:0/3472158779 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88c012f8b0 msgr2=0x7f88c012fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:17.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.134+0000 7f88c6cd6700 1 --2- 172.21.15.3:0/3472158779 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88c012f8b0 0x7f88c012fd20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00aac0 tx=0x7f88ac00add0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.134+0000 7f88c6cd6700 1 -- 172.21.15.3:0/3472158779 shutdown_connections 2023-10-03T10:15:17.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.134+0000 7f88c6cd6700 1 --2- 172.21.15.3:0/3472158779 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88c012cf60 0x7f88c006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.134+0000 7f88c6cd6700 1 --2- 172.21.15.3:0/3472158779 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88c012f8b0 0x7f88c012fd20 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.134+0000 7f88c6cd6700 1 -- 172.21.15.3:0/3472158779 >> 172.21.15.3:0/3472158779 conn(0x7f88c00a2aa0 msgr2=0x7f88c00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:17.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.134+0000 7f88c6cd6700 1 -- 172.21.15.3:0/3472158779 shutdown_connections 2023-10-03T10:15:17.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.134+0000 7f88c6cd6700 1 -- 172.21.15.3:0/3472158779 wait complete. 2023-10-03T10:15:17.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.136+0000 7f88c6cd6700 1 Processor -- start 2023-10-03T10:15:17.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.136+0000 7f88c6cd6700 1 -- start start 2023-10-03T10:15:17.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.137+0000 7f88c6cd6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88c012cf60 0x7f88c009bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:17.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.137+0000 7f88c6cd6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88c012f8b0 0x7f88c01c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:17.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.137+0000 7f88c6cd6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c0130260 0x7f88c01c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:17.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.137+0000 7f88c6cd6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c0129900 con 0x7f88c0130260 2023-10-03T10:15:17.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.137+0000 7f88c6cd6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c009b690 con 0x7f88c012cf60 2023-10-03T10:15:17.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.137+0000 7f88c6cd6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f88c0129fa0 con 0x7f88c012f8b0 2023-10-03T10:15:17.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.137+0000 7f88c4a72700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88c012cf60 0x7f88c009bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:17.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.137+0000 7f88c4a72700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88c012cf60 0x7f88c009bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50558/0 (socket says 172.21.15.3:50558) 2023-10-03T10:15:17.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.137+0000 7f88c4a72700 1 -- 172.21.15.3:0/1399713108 learned_addr learned my addr 172.21.15.3:0/1399713108 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:17.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.137+0000 7f88bffff700 1 --2- 172.21.15.3:0/1399713108 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88c012f8b0 0x7f88c01c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:17.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.137+0000 7f88c5273700 1 --2- 172.21.15.3:0/1399713108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c0130260 0x7f88c01c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:17.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.138+0000 7f88c4a72700 1 -- 172.21.15.3:0/1399713108 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88c012f8b0 msgr2=0x7f88c01c3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:17.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.138+0000 7f88c4a72700 1 --2- 172.21.15.3:0/1399713108 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88c012f8b0 0x7f88c01c3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.138+0000 7f88c4a72700 1 -- 172.21.15.3:0/1399713108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c0130260 msgr2=0x7f88c01c44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:17.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.138+0000 7f88c4a72700 1 --2- 172.21.15.3:0/1399713108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c0130260 0x7f88c01c44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.138+0000 7f88c4a72700 1 -- 172.21.15.3:0/1399713108 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ac00a6f0 con 0x7f88c012cf60 2023-10-03T10:15:17.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.138+0000 7f88c5273700 1 --2- 172.21.15.3:0/1399713108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c0130260 0x7f88c01c44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:15:17.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.138+0000 7f88c4a72700 1 --2- 172.21.15.3:0/1399713108 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88c012cf60 0x7f88c009bd80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f88b400da80 tx=0x7f88b400dd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:17.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.138+0000 7f88bd7fa700 1 -- 172.21.15.3:0/1399713108 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b400a9d0 con 0x7f88c012cf60 2023-10-03T10:15:17.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.138+0000 7f88bd7fa700 1 -- 172.21.15.3:0/1399713108 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b400ab30 con 0x7f88c012cf60 2023-10-03T10:15:17.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.138+0000 7f88bd7fa700 1 -- 172.21.15.3:0/1399713108 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4019480 con 0x7f88c012cf60 2023-10-03T10:15:17.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.138+0000 7f88c6cd6700 1 -- 172.21.15.3:0/1399713108 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c009e6c0 con 0x7f88c012cf60 2023-10-03T10:15:17.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.138+0000 7f88c6cd6700 1 -- 172.21.15.3:0/1399713108 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c0119200 con 0x7f88c012cf60 2023-10-03T10:15:17.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.139+0000 7f88bd7fa700 1 -- 172.21.15.3:0/1399713108 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f88b40195e0 con 0x7f88c012cf60 2023-10-03T10:15:17.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.139+0000 7f88c6cd6700 1 -- 172.21.15.3:0/1399713108 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c0074ea0 con 0x7f88c012cf60 2023-10-03T10:15:17.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.140+0000 7f88bd7fa700 1 -- 172.21.15.3:0/1399713108 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f88b4014070 con 0x7f88c012cf60 2023-10-03T10:15:17.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.144+0000 7f88bd7fa700 1 -- 172.21.15.3:0/1399713108 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f88b4017970 con 0x7f88c012cf60 2023-10-03T10:15:17.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.170+0000 7f97eea8a700 1 Processor -- start 2023-10-03T10:15:17.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.170+0000 7f97eea8a700 1 -- start start 2023-10-03T10:15:17.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.171+0000 7f97eea8a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f97e809d6f0 0x7f97e809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:17.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.171+0000 7f97eea8a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f97e809e440 0x7f97e8134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:17.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.171+0000 7f97eea8a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97e809b120 con 0x7f97e809e000 2023-10-03T10:15:17.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.171+0000 7f97eea8a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97e806b460 con 0x7f97e809d6f0 2023-10-03T10:15:17.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.171+0000 7f97eea8a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97e809a7e0 con 0x7f97e809e440 2023-10-03T10:15:17.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.171+0000 7f97ed027700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f97e809e440 0x7f97e8134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.171+0000 7f97ed027700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f97e809e440 0x7f97e8134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53824/0 (socket says 172.21.15.3:53824) 2023-10-03T10:15:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.171+0000 7f97ed027700 1 -- 172.21.15.3:0/1141025066 learned_addr learned my addr 172.21.15.3:0/1141025066 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.172+0000 7f97ec826700 1 --2- 172.21.15.3:0/1141025066 >> v2:172.21.15.3:3300/0 conn(0x7f97e809d6f0 0x7f97e809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.172+0000 7f97ed027700 1 -- 172.21.15.3:0/1141025066 >> v2:172.21.15.3:3300/0 conn(0x7f97e809d6f0 msgr2=0x7f97e809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.172+0000 7f97ed027700 1 --2- 172.21.15.3:0/1141025066 >> v2:172.21.15.3:3300/0 conn(0x7f97e809d6f0 0x7f97e809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.172+0000 7f97ed027700 1 -- 172.21.15.3:0/1141025066 >> v1:172.21.15.3:6789/0 conn(0x7f97e809e000 legacy=0x7f97e81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.172+0000 7f97ed027700 1 -- 172.21.15.3:0/1141025066 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e809a500 con 0x7f97e809e440 2023-10-03T10:15:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.172+0000 7f97ed027700 1 --2- 172.21.15.3:0/1141025066 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f97e809e440 0x7f97e8134100 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009e80 tx=0x7f97e000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7784ca75fffeaaf3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:17.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.172+0000 7f97e77fe700 1 -- 172.21.15.3:0/1141025066 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e000e070 con 0x7f97e809e440 2023-10-03T10:15:17.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.172+0000 7f97e77fe700 1 -- 172.21.15.3:0/1141025066 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e000b8d0 con 0x7f97e809e440 2023-10-03T10:15:17.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.172+0000 7f97e77fe700 1 -- 172.21.15.3:0/1141025066 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e000bc80 con 0x7f97e809e440 2023-10-03T10:15:17.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.173+0000 7f97eea8a700 1 -- 172.21.15.3:0/1141025066 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f97e809e440 msgr2=0x7f97e8134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:17.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.173+0000 7f97eea8a700 1 --2- 172.21.15.3:0/1141025066 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f97e809e440 0x7f97e8134100 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009e80 tx=0x7f97e000b330 comp rx=0 tx=0).stop 2023-10-03T10:15:17.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.173+0000 7f97eea8a700 1 -- 172.21.15.3:0/1141025066 shutdown_connections 2023-10-03T10:15:17.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.173+0000 7f97eea8a700 1 --2- 172.21.15.3:0/1141025066 >> v2:172.21.15.3:3300/0 conn(0x7f97e809d6f0 0x7f97e809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.173+0000 7f97eea8a700 1 --2- 172.21.15.3:0/1141025066 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f97e809e440 0x7f97e8134100 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.173+0000 7f97eea8a700 1 -- 172.21.15.3:0/1141025066 >> 172.21.15.3:0/1141025066 conn(0x7f97e8066f50 msgr2=0x7f97e809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:17.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.173+0000 7f97eea8a700 1 -- 172.21.15.3:0/1141025066 shutdown_connections 2023-10-03T10:15:17.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.173+0000 7f97eea8a700 1 -- 172.21.15.3:0/1141025066 wait complete. 2023-10-03T10:15:17.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.175+0000 7f97eea8a700 1 Processor -- start 2023-10-03T10:15:17.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.175+0000 7f97eea8a700 1 -- start start 2023-10-03T10:15:17.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.176+0000 7f97eea8a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f97e809d6f0 0x7f97e81c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:17.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.176+0000 7f97eea8a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97e809e000 0x7f97e81c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:17.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.176+0000 7f97eea8a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f97e809e440 0x7f97e81c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:17.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.176+0000 7f97eea8a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e81a80b0 con 0x7f97e809e000 2023-10-03T10:15:17.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.176+0000 7f97eea8a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e80a1f90 con 0x7f97e809d6f0 2023-10-03T10:15:17.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.176+0000 7f97eea8a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97e81b9550 con 0x7f97e809e440 2023-10-03T10:15:17.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.176+0000 7f97ed027700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f97e809e440 0x7f97e81c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:17.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.176+0000 7f97ed027700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f97e809e440 0x7f97e81c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53836/0 (socket says 172.21.15.3:53836) 2023-10-03T10:15:17.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.176+0000 7f97ed027700 1 -- 172.21.15.3:0/1499138618 learned_addr learned my addr 172.21.15.3:0/1499138618 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:17.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.176+0000 7f97ec826700 1 --2- 172.21.15.3:0/1499138618 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f97e809d6f0 0x7f97e81c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:17.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.176+0000 7f97e7fff700 1 --2- 172.21.15.3:0/1499138618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97e809e000 0x7f97e81c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:17.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.176+0000 7f97ed027700 1 -- 172.21.15.3:0/1499138618 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f97e809d6f0 msgr2=0x7f97e81c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:17.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.176+0000 7f97ed027700 1 --2- 172.21.15.3:0/1499138618 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f97e809d6f0 0x7f97e81c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.176+0000 7f97ed027700 1 -- 172.21.15.3:0/1499138618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97e809e000 msgr2=0x7f97e81c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:17.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.176+0000 7f97ed027700 1 --2- 172.21.15.3:0/1499138618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97e809e000 0x7f97e81c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.176+0000 7f97ed027700 1 -- 172.21.15.3:0/1499138618 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e0009b30 con 0x7f97e809e440 2023-10-03T10:15:17.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.177+0000 7f97ed027700 1 --2- 172.21.15.3:0/1499138618 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f97e809e440 0x7f97e81c49d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f97e809a920 tx=0x7f97e0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:17.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.177+0000 7f97e57fa700 1 -- 172.21.15.3:0/1499138618 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e00048b0 con 0x7f97e809e440 2023-10-03T10:15:17.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.177+0000 7f97e57fa700 1 -- 172.21.15.3:0/1499138618 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e00092e0 con 0x7f97e809e440 2023-10-03T10:15:17.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.177+0000 7f97eea8a700 1 -- 172.21.15.3:0/1499138618 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e81a8980 con 0x7f97e809e440 2023-10-03T10:15:17.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.177+0000 7f97e57fa700 1 -- 172.21.15.3:0/1499138618 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e001c680 con 0x7f97e809e440 2023-10-03T10:15:17.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.177+0000 7f97eea8a700 1 -- 172.21.15.3:0/1499138618 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e81a79c0 con 0x7f97e809e440 2023-10-03T10:15:17.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.178+0000 7f97e57fa700 1 -- 172.21.15.3:0/1499138618 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f97e001c7e0 con 0x7f97e809e440 2023-10-03T10:15:17.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.178+0000 7f97e57fa700 1 -- 172.21.15.3:0/1499138618 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f97e0040e20 con 0x7f97e809e440 2023-10-03T10:15:17.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.178+0000 7f97eea8a700 1 -- 172.21.15.3:0/1499138618 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97e81a79c0 con 0x7f97e809e440 2023-10-03T10:15:17.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.188+0000 7f97e57fa700 1 -- 172.21.15.3:0/1499138618 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f97e81a79c0 con 0x7f97e809e440 2023-10-03T10:15:17.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.210+0000 7f65731d9700 1 Processor -- start 2023-10-03T10:15:17.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.211+0000 7f65731d9700 1 -- start start 2023-10-03T10:15:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.211+0000 7f65731d9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f656c09d6f0 0x7f656c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.211+0000 7f65731d9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f656c09e000 0x7f656c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.211+0000 7f65731d9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f656c09f130 con 0x7f656c09e410 2023-10-03T10:15:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.211+0000 7f65731d9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656c09f520 con 0x7f656c09e000 2023-10-03T10:15:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.211+0000 7f65731d9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f656c09e930 con 0x7f656c09d6f0 2023-10-03T10:15:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.211+0000 7f6570f75700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f656c09d6f0 0x7f656c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.211+0000 7f6570f75700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f656c09d6f0 0x7f656c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53852/0 (socket says 172.21.15.3:53852) 2023-10-03T10:15:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.211+0000 7f6570f75700 1 -- 172.21.15.3:0/1381221939 learned_addr learned my addr 172.21.15.3:0/1381221939 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.211+0000 7f656bfff700 1 --2- 172.21.15.3:0/1381221939 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f656c09e000 0x7f656c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:17.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.212+0000 7f6570f75700 1 -- 172.21.15.3:0/1381221939 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f656c09e000 msgr2=0x7f656c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:17.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.212+0000 7f6570f75700 1 --2- 172.21.15.3:0/1381221939 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f656c09e000 0x7f656c130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.212+0000 7f6570f75700 1 -- 172.21.15.3:0/1381221939 >> v1:172.21.15.3:6789/0 conn(0x7f656c09e410 legacy=0x7f656c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:17.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.212+0000 7f6570f75700 1 -- 172.21.15.3:0/1381221939 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656c09b2d0 con 0x7f656c09d6f0 2023-10-03T10:15:17.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.212+0000 7f6570f75700 1 --2- 172.21.15.3:0/1381221939 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f656c09d6f0 0x7f656c09dac0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6560009950 tx=0x7f6560009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83b349225041dc76 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:17.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.212+0000 7f656b7fe700 1 -- 172.21.15.3:0/1381221939 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656000f8a0 con 0x7f656c09d6f0 2023-10-03T10:15:17.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.212+0000 7f656b7fe700 1 -- 172.21.15.3:0/1381221939 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f656000fa00 con 0x7f656c09d6f0 2023-10-03T10:15:17.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.212+0000 7f656b7fe700 1 -- 172.21.15.3:0/1381221939 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656000fdb0 con 0x7f656c09d6f0 2023-10-03T10:15:17.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.213+0000 7f65731d9700 1 -- 172.21.15.3:0/1381221939 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f656c09d6f0 msgr2=0x7f656c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:17.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.213+0000 7f65731d9700 1 --2- 172.21.15.3:0/1381221939 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f656c09d6f0 0x7f656c09dac0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6560009950 tx=0x7f6560009c60 comp rx=0 tx=0).stop 2023-10-03T10:15:17.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.213+0000 7f65731d9700 1 -- 172.21.15.3:0/1381221939 shutdown_connections 2023-10-03T10:15:17.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.213+0000 7f65731d9700 1 --2- 172.21.15.3:0/1381221939 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f656c09d6f0 0x7f656c09dac0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.213+0000 7f65731d9700 1 --2- 172.21.15.3:0/1381221939 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f656c09e000 0x7f656c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.213+0000 7f65731d9700 1 -- 172.21.15.3:0/1381221939 >> 172.21.15.3:0/1381221939 conn(0x7f656c066f50 msgr2=0x7f656c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.213+0000 7f65731d9700 1 -- 172.21.15.3:0/1381221939 shutdown_connections 2023-10-03T10:15:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.213+0000 7f65731d9700 1 -- 172.21.15.3:0/1381221939 wait complete. 2023-10-03T10:15:17.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.214+0000 7f65731d9700 1 Processor -- start 2023-10-03T10:15:17.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.214+0000 7f65731d9700 1 -- start start 2023-10-03T10:15:17.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.215+0000 7f65731d9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f656c09d6f0 0x7f656c1229e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:17.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.215+0000 7f65731d9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f656c09e000 0x7f656c122f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:17.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.215+0000 7f65731d9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f656c09e410 0x7f656c123460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:17.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.215+0000 7f65731d9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c0a1d90 con 0x7f656c09e000 2023-10-03T10:15:17.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.215+0000 7f65731d9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656c1245c0 con 0x7f656c09e410 2023-10-03T10:15:17.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.215+0000 7f65731d9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f656c126150 con 0x7f656c09d6f0 2023-10-03T10:15:17.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.215+0000 7f6570f75700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f656c09d6f0 0x7f656c1229e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:17.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.215+0000 7f6570f75700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f656c09d6f0 0x7f656c1229e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53854/0 (socket says 172.21.15.3:53854) 2023-10-03T10:15:17.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.215+0000 7f6570f75700 1 -- 172.21.15.3:0/4094958610 learned_addr learned my addr 172.21.15.3:0/4094958610 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:17.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.215+0000 7f6571776700 1 --2- 172.21.15.3:0/4094958610 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f656c09e410 0x7f656c123460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:17.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.215+0000 7f6570f75700 1 -- 172.21.15.3:0/4094958610 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f656c09e410 msgr2=0x7f656c123460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:17.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.215+0000 7f6570f75700 1 --2- 172.21.15.3:0/4094958610 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f656c09e410 0x7f656c123460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.215+0000 7f6570f75700 1 -- 172.21.15.3:0/4094958610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f656c09e000 msgr2=0x7f656c122f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:17.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.215+0000 7f6570f75700 1 --2- 172.21.15.3:0/4094958610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f656c09e000 0x7f656c122f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.215+0000 7f6570f75700 1 -- 172.21.15.3:0/4094958610 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6560009550 con 0x7f656c09d6f0 2023-10-03T10:15:17.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.216+0000 7f6570f75700 1 --2- 172.21.15.3:0/4094958610 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f656c09d6f0 0x7f656c1229e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6560009ca0 tx=0x7f6560004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:17.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.216+0000 7f65697fa700 1 -- 172.21.15.3:0/4094958610 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560003ab0 con 0x7f656c09d6f0 2023-10-03T10:15:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.216+0000 7f65697fa700 1 -- 172.21.15.3:0/4094958610 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6560003c30 con 0x7f656c09d6f0 2023-10-03T10:15:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.216+0000 7f65697fa700 1 -- 172.21.15.3:0/4094958610 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560006bd0 con 0x7f656c09d6f0 2023-10-03T10:15:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.216+0000 7f65731d9700 1 -- 172.21.15.3:0/4094958610 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f656c09c700 con 0x7f656c09d6f0 2023-10-03T10:15:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.216+0000 7f65731d9700 1 -- 172.21.15.3:0/4094958610 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f656c126bc0 con 0x7f656c09d6f0 2023-10-03T10:15:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.217+0000 7f65697fa700 1 -- 172.21.15.3:0/4094958610 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f656002a440 con 0x7f656c09d6f0 2023-10-03T10:15:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.217+0000 7f65697fa700 1 -- 172.21.15.3:0/4094958610 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6560040810 con 0x7f656c09d6f0 2023-10-03T10:15:17.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.217+0000 7f65731d9700 1 -- 172.21.15.3:0/4094958610 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f656c074ea0 con 0x7f656c09d6f0 2023-10-03T10:15:17.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.222+0000 7f65697fa700 1 -- 172.21.15.3:0/4094958610 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6560017710 con 0x7f656c09d6f0 2023-10-03T10:15:17.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.289+0000 7f67d8a48700 1 -- 172.21.15.3:0/339974834 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f67d41253d0 con 0x7f67d4136d80 2023-10-03T10:15:17.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.289+0000 7f67c2ffd700 1 -- 172.21.15.3:0/339974834 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f67d41253d0 con 0x7f67d4136d80 2023-10-03T10:15:17.290 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:17.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.292+0000 7f67d8a48700 1 -- 172.21.15.3:0/339974834 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67d4136d80 msgr2=0x7f67d41c81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:17.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.292+0000 7f67d8a48700 1 --2- 172.21.15.3:0/339974834 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67d4136d80 0x7f67d41c81a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f67d409e180 tx=0x7f67cc004ad0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.292+0000 7f67d8a48700 1 -- 172.21.15.3:0/339974834 shutdown_connections 2023-10-03T10:15:17.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.292+0000 7f67d8a48700 1 --2- 172.21.15.3:0/339974834 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67d4136d80 0x7f67d41c81a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.292+0000 7f67d8a48700 1 --2- 172.21.15.3:0/339974834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67d406a770 0x7f67d4122470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.292+0000 7f67d8a48700 1 --2- 172.21.15.3:0/339974834 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67d406b250 0x7f67d41229b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.292+0000 7f67d8a48700 1 -- 172.21.15.3:0/339974834 >> 172.21.15.3:0/339974834 conn(0x7f67d4066f50 msgr2=0x7f67d41bbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:17.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.293+0000 7f67d8a48700 1 -- 172.21.15.3:0/339974834 shutdown_connections 2023-10-03T10:15:17.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.293+0000 7f67d8a48700 1 -- 172.21.15.3:0/339974834 wait complete. 2023-10-03T10:15:17.312 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:17.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.363+0000 7f88c6cd6700 1 -- 172.21.15.3:0/1399713108 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f88c00713a0 con 0x7f88c012cf60 2023-10-03T10:15:17.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.364+0000 7f88bd7fa700 1 -- 172.21.15.3:0/1399713108 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f88b401e030 con 0x7f88c012cf60 2023-10-03T10:15:17.364 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:17.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.366+0000 7f88c6cd6700 1 -- 172.21.15.3:0/1399713108 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88c012cf60 msgr2=0x7f88c009bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:17.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.366+0000 7f88c6cd6700 1 --2- 172.21.15.3:0/1399713108 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88c012cf60 0x7f88c009bd80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f88b400da80 tx=0x7f88b400dd90 comp rx=0 tx=0).stop 2023-10-03T10:15:17.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.366+0000 7f88c6cd6700 1 -- 172.21.15.3:0/1399713108 shutdown_connections 2023-10-03T10:15:17.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.366+0000 7f88c6cd6700 1 --2- 172.21.15.3:0/1399713108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c0130260 0x7f88c01c44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.366+0000 7f88c6cd6700 1 --2- 172.21.15.3:0/1399713108 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88c012cf60 0x7f88c009bd80 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.366+0000 7f88c6cd6700 1 --2- 172.21.15.3:0/1399713108 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88c012f8b0 0x7f88c01c3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.366+0000 7f88c6cd6700 1 -- 172.21.15.3:0/1399713108 >> 172.21.15.3:0/1399713108 conn(0x7f88c00a2aa0 msgr2=0x7f88c006e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:17.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.367+0000 7f88c6cd6700 1 -- 172.21.15.3:0/1399713108 shutdown_connections 2023-10-03T10:15:17.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.367+0000 7f88c6cd6700 1 -- 172.21.15.3:0/1399713108 wait complete. 2023-10-03T10:15:17.381 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:17.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.437+0000 7f65731d9700 1 -- 172.21.15.3:0/4094958610 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f656c125df0 con 0x7f656c09d6f0 2023-10-03T10:15:17.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.438+0000 7f65697fa700 1 -- 172.21.15.3:0/4094958610 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f65600293a0 con 0x7f656c09d6f0 2023-10-03T10:15:17.438 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:17.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.440+0000 7f65731d9700 1 -- 172.21.15.3:0/4094958610 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f656c09d6f0 msgr2=0x7f656c1229e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:17.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.440+0000 7f65731d9700 1 --2- 172.21.15.3:0/4094958610 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f656c09d6f0 0x7f656c1229e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6560009ca0 tx=0x7f6560004750 comp rx=0 tx=0).stop 2023-10-03T10:15:17.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.440+0000 7f65731d9700 1 -- 172.21.15.3:0/4094958610 shutdown_connections 2023-10-03T10:15:17.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.440+0000 7f65731d9700 1 --2- 172.21.15.3:0/4094958610 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f656c09e410 0x7f656c123460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.440+0000 7f65731d9700 1 --2- 172.21.15.3:0/4094958610 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f656c09d6f0 0x7f656c1229e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.440+0000 7f65731d9700 1 --2- 172.21.15.3:0/4094958610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f656c09e000 0x7f656c122f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.440+0000 7f65731d9700 1 -- 172.21.15.3:0/4094958610 >> 172.21.15.3:0/4094958610 conn(0x7f656c066f50 msgr2=0x7f656c06f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:17.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.440+0000 7f65731d9700 1 -- 172.21.15.3:0/4094958610 shutdown_connections 2023-10-03T10:15:17.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.440+0000 7f65731d9700 1 -- 172.21.15.3:0/4094958610 wait complete. 2023-10-03T10:15:17.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.443+0000 7f97eea8a700 1 -- 172.21.15.3:0/1499138618 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f97e81a79c0 con 0x7f97e809e440 2023-10-03T10:15:17.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.444+0000 7f97e57fa700 1 -- 172.21.15.3:0/1499138618 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f97e81a79c0 con 0x7f97e809e440 2023-10-03T10:15:17.444 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:17.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.446+0000 7f97eea8a700 1 -- 172.21.15.3:0/1499138618 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f97e809e440 msgr2=0x7f97e81c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:17.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.446+0000 7f97eea8a700 1 --2- 172.21.15.3:0/1499138618 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f97e809e440 0x7f97e81c49d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f97e809a920 tx=0x7f97e0004ad0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.446+0000 7f97eea8a700 1 -- 172.21.15.3:0/1499138618 shutdown_connections 2023-10-03T10:15:17.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.446+0000 7f97eea8a700 1 --2- 172.21.15.3:0/1499138618 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f97e809e440 0x7f97e81c49d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.446+0000 7f97eea8a700 1 --2- 172.21.15.3:0/1499138618 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f97e809d6f0 0x7f97e81c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.446+0000 7f97eea8a700 1 --2- 172.21.15.3:0/1499138618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97e809e000 0x7f97e81c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:17.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.446+0000 7f97eea8a700 1 -- 172.21.15.3:0/1499138618 >> 172.21.15.3:0/1499138618 conn(0x7f97e8066f50 msgr2=0x7f97e81bad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:17.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.446+0000 7f97eea8a700 1 -- 172.21.15.3:0/1499138618 shutdown_connections 2023-10-03T10:15:17.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:17.446+0000 7f97eea8a700 1 -- 172.21.15.3:0/1499138618 wait complete. 2023-10-03T10:15:17.455 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:17.461 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:18.314 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:18.382 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:18.456 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:18.461 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:18.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.461+0000 7f27a6236700 1 Processor -- start 2023-10-03T10:15:18.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.462+0000 7f27a6236700 1 -- start start 2023-10-03T10:15:18.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.462+0000 7f27a6236700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f27a009d6f0 0x7f27a009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:18.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.462+0000 7f27a6236700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f27a009e000 0x7f27a0130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:18.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.462+0000 7f27a6236700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27a009f130 con 0x7f27a009e410 2023-10-03T10:15:18.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.462+0000 7f27a6236700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a009f520 con 0x7f27a009e000 2023-10-03T10:15:18.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.462+0000 7f27a6236700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f27a009e930 con 0x7f27a009d6f0 2023-10-03T10:15:18.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.462+0000 7f279effd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f27a009e000 0x7f27a0130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:18.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.462+0000 7f279effd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f27a009e000 0x7f27a0130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50578/0 (socket says 172.21.15.3:50578) 2023-10-03T10:15:18.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.462+0000 7f279effd700 1 -- 172.21.15.3:0/707898181 learned_addr learned my addr 172.21.15.3:0/707898181 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:18.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.462+0000 7f279ffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f27a009e410 0x7f27a0130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47704/0 (socket says 172.21.15.3:47704) 2023-10-03T10:15:18.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.462+0000 7f279f7fe700 1 --2- 172.21.15.3:0/707898181 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f27a009d6f0 0x7f27a009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:18.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.463+0000 7f279effd700 1 -- 172.21.15.3:0/707898181 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f27a009d6f0 msgr2=0x7f27a009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:18.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.463+0000 7f279effd700 1 --2- 172.21.15.3:0/707898181 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f27a009d6f0 0x7f27a009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.463+0000 7f279effd700 1 -- 172.21.15.3:0/707898181 >> v1:172.21.15.3:6789/0 conn(0x7f27a009e410 legacy=0x7f27a0130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:18.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.463+0000 7f279effd700 1 -- 172.21.15.3:0/707898181 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a009b2d0 con 0x7f27a009e000 2023-10-03T10:15:18.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.463+0000 7f279f7fe700 1 --2- 172.21.15.3:0/707898181 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f27a009d6f0 0x7f27a009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:18.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.463+0000 7f279effd700 1 --2- 172.21.15.3:0/707898181 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f27a009e000 0x7f27a0130660 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2794009920 tx=0x7f2794009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9745da7cf90185d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:18.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.463+0000 7f279e7fc700 1 -- 172.21.15.3:0/707898181 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2794011830 con 0x7f27a009e000 2023-10-03T10:15:18.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.463+0000 7f279e7fc700 1 -- 172.21.15.3:0/707898181 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2794011990 con 0x7f27a009e000 2023-10-03T10:15:18.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.463+0000 7f279e7fc700 1 -- 172.21.15.3:0/707898181 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2794011d20 con 0x7f27a009e000 2023-10-03T10:15:18.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.464+0000 7f27a6236700 1 -- 172.21.15.3:0/707898181 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f27a009e000 msgr2=0x7f27a0130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:18.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.464+0000 7f27a6236700 1 --2- 172.21.15.3:0/707898181 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f27a009e000 0x7f27a0130660 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2794009920 tx=0x7f2794009c30 comp rx=0 tx=0).stop 2023-10-03T10:15:18.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.464+0000 7f27a6236700 1 -- 172.21.15.3:0/707898181 shutdown_connections 2023-10-03T10:15:18.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.464+0000 7f27a6236700 1 --2- 172.21.15.3:0/707898181 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f27a009d6f0 0x7f27a009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.464+0000 7f27a6236700 1 --2- 172.21.15.3:0/707898181 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f27a009e000 0x7f27a0130660 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.464+0000 7f27a6236700 1 -- 172.21.15.3:0/707898181 >> 172.21.15.3:0/707898181 conn(0x7f27a0066f50 msgr2=0x7f27a009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:18.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.464+0000 7f27a6236700 1 -- 172.21.15.3:0/707898181 shutdown_connections 2023-10-03T10:15:18.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.464+0000 7f27a6236700 1 -- 172.21.15.3:0/707898181 wait complete. 2023-10-03T10:15:18.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.465+0000 7f27a6236700 1 Processor -- start 2023-10-03T10:15:18.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.466+0000 7f27a6236700 1 -- start start 2023-10-03T10:15:18.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.466+0000 7f27a6236700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27a009d6f0 0x7f27a01224b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:18.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.466+0000 7f27a6236700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f27a009e000 0x7f27a01229f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:18.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.466+0000 7f27a6236700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f27a009e410 0x7f27a0122f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:18.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.466+0000 7f27a6236700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a0130360 con 0x7f27a009d6f0 2023-10-03T10:15:18.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.466+0000 7f27a6236700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a00a1d00 con 0x7f27a009e000 2023-10-03T10:15:18.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.466+0000 7f27a6236700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f27a0124080 con 0x7f27a009e410 2023-10-03T10:15:18.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.467+0000 7f279effd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f27a009e000 0x7f27a01229f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:18.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.467+0000 7f279f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27a009d6f0 0x7f27a01224b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:18.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.467+0000 7f279effd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f27a009e000 0x7f27a01229f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50580/0 (socket says 172.21.15.3:50580) 2023-10-03T10:15:18.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.467+0000 7f279effd700 1 -- 172.21.15.3:0/2867610183 learned_addr learned my addr 172.21.15.3:0/2867610183 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:18.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.467+0000 7f279ffff700 1 --2- 172.21.15.3:0/2867610183 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f27a009e410 0x7f27a0122f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:18.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.467+0000 7f279f7fe700 1 -- 172.21.15.3:0/2867610183 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f27a009e410 msgr2=0x7f27a0122f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:18.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.467+0000 7f279f7fe700 1 --2- 172.21.15.3:0/2867610183 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f27a009e410 0x7f27a0122f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.467+0000 7f279f7fe700 1 -- 172.21.15.3:0/2867610183 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f27a009e000 msgr2=0x7f27a01229f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:18.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.467+0000 7f279f7fe700 1 --2- 172.21.15.3:0/2867610183 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f27a009e000 0x7f27a01229f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.467+0000 7f279f7fe700 1 -- 172.21.15.3:0/2867610183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f279000b740 con 0x7f27a009d6f0 2023-10-03T10:15:18.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.467+0000 7f279f7fe700 1 --2- 172.21.15.3:0/2867610183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27a009d6f0 0x7f27a01224b0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f279000eda0 tx=0x7f279000d630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:18.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.468+0000 7f2787fff700 1 -- 172.21.15.3:0/2867610183 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279000dd90 con 0x7f27a009d6f0 2023-10-03T10:15:18.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.468+0000 7f2787fff700 1 -- 172.21.15.3:0/2867610183 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2790010440 con 0x7f27a009d6f0 2023-10-03T10:15:18.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.468+0000 7f27a6236700 1 -- 172.21.15.3:0/2867610183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2794009550 con 0x7f27a009d6f0 2023-10-03T10:15:18.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.468+0000 7f2787fff700 1 -- 172.21.15.3:0/2867610183 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27900107f0 con 0x7f27a009d6f0 2023-10-03T10:15:18.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.468+0000 7f27a6236700 1 -- 172.21.15.3:0/2867610183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27a01247d0 con 0x7f27a009d6f0 2023-10-03T10:15:18.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.469+0000 7f2787fff700 1 -- 172.21.15.3:0/2867610183 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2790010950 con 0x7f27a009d6f0 2023-10-03T10:15:18.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.469+0000 7f2787fff700 1 -- 172.21.15.3:0/2867610183 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f279003fd20 con 0x7f27a009d6f0 2023-10-03T10:15:18.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.469+0000 7f27a6236700 1 -- 172.21.15.3:0/2867610183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27a01258b0 con 0x7f27a009d6f0 2023-10-03T10:15:18.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.475+0000 7f2787fff700 1 -- 172.21.15.3:0/2867610183 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f279001d020 con 0x7f27a009d6f0 2023-10-03T10:15:18.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.538+0000 7f3c73f46700 1 Processor -- start 2023-10-03T10:15:18.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.538+0000 7f3c73f46700 1 -- start start 2023-10-03T10:15:18.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.539+0000 7f3c73f46700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3c6c09d6f0 0x7f3c6c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:18.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.539+0000 7f3c73f46700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3c6c09e000 0x7f3c6c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:18.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.539+0000 7f3c73f46700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c6c09f130 con 0x7f3c6c09e410 2023-10-03T10:15:18.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.539+0000 7f3c73f46700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c09f520 con 0x7f3c6c09d6f0 2023-10-03T10:15:18.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.539+0000 7f3c73f46700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c09e930 con 0x7f3c6c09e000 2023-10-03T10:15:18.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.540+0000 7f3c71ce2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3c6c09d6f0 0x7f3c6c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:18.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.540+0000 7f3c724e3700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3c6c09e410 0x7f3c6c130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47716/0 (socket says 172.21.15.3:47716) 2023-10-03T10:15:18.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.540+0000 7f3c71ce2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3c6c09d6f0 0x7f3c6c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50594/0 (socket says 172.21.15.3:50594) 2023-10-03T10:15:18.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.540+0000 7f3c724e3700 1 -- 172.21.15.3:0/3861960258 learned_addr learned my addr 172.21.15.3:0/3861960258 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:18.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.540+0000 7f3c714e1700 1 --2- 172.21.15.3:0/3861960258 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3c6c09e000 0x7f3c6c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:18.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.540+0000 7f3c71ce2700 1 -- 172.21.15.3:0/3861960258 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3c6c09e000 msgr2=0x7f3c6c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:18.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.540+0000 7f3c71ce2700 1 --2- 172.21.15.3:0/3861960258 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3c6c09e000 0x7f3c6c130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.540+0000 7f3c71ce2700 1 -- 172.21.15.3:0/3861960258 >> v1:172.21.15.3:6789/0 conn(0x7f3c6c09e410 legacy=0x7f3c6c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:18.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.540+0000 7f3c71ce2700 1 -- 172.21.15.3:0/3861960258 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c6c09b2d0 con 0x7f3c6c09d6f0 2023-10-03T10:15:18.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.540+0000 7f3c71ce2700 1 --2- 172.21.15.3:0/3861960258 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3c6c09d6f0 0x7f3c6c09dac0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009870 tx=0x7f3c68009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c39837186e348e0e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.540+0000 7f3c70ce0700 1 -- 172.21.15.3:0/3861960258 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c68004230 con 0x7f3c6c09d6f0 2023-10-03T10:15:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.541+0000 7f3c70ce0700 1 -- 172.21.15.3:0/3861960258 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c68004390 con 0x7f3c6c09d6f0 2023-10-03T10:15:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.541+0000 7f3c70ce0700 1 -- 172.21.15.3:0/3861960258 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c680036a0 con 0x7f3c6c09d6f0 2023-10-03T10:15:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.541+0000 7f3c73f46700 1 -- 172.21.15.3:0/3861960258 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3c6c09d6f0 msgr2=0x7f3c6c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.541+0000 7f3c73f46700 1 --2- 172.21.15.3:0/3861960258 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3c6c09d6f0 0x7f3c6c09dac0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009870 tx=0x7f3c68009b80 comp rx=0 tx=0).stop 2023-10-03T10:15:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.542+0000 7f3c73f46700 1 -- 172.21.15.3:0/3861960258 shutdown_connections 2023-10-03T10:15:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.542+0000 7f3c73f46700 1 --2- 172.21.15.3:0/3861960258 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3c6c09d6f0 0x7f3c6c09dac0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.542+0000 7f3c73f46700 1 --2- 172.21.15.3:0/3861960258 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3c6c09e000 0x7f3c6c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.542+0000 7f3c73f46700 1 -- 172.21.15.3:0/3861960258 >> 172.21.15.3:0/3861960258 conn(0x7f3c6c066f50 msgr2=0x7f3c6c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.542+0000 7f3c73f46700 1 -- 172.21.15.3:0/3861960258 shutdown_connections 2023-10-03T10:15:18.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.542+0000 7f3c73f46700 1 -- 172.21.15.3:0/3861960258 wait complete. 2023-10-03T10:15:18.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.543+0000 7f3c73f46700 1 Processor -- start 2023-10-03T10:15:18.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.543+0000 7f3c73f46700 1 -- start start 2023-10-03T10:15:18.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.543+0000 7f3c73f46700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3c6c09d6f0 0x7f3c6c120350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:18.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.543+0000 7f3c73f46700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6c09e000 0x7f3c6c120890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:18.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.543+0000 7f3c73f46700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3c6c09e410 0x7f3c6c120dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:18.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.543+0000 7f3c73f46700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c121f60 con 0x7f3c6c09e000 2023-10-03T10:15:18.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.543+0000 7f3c73f46700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c1227f0 con 0x7f3c6c09d6f0 2023-10-03T10:15:18.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.543+0000 7f3c73f46700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c1217b0 con 0x7f3c6c09e410 2023-10-03T10:15:18.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.544+0000 7f3c71ce2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3c6c09d6f0 0x7f3c6c120350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:18.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.544+0000 7f3c71ce2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3c6c09d6f0 0x7f3c6c120350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50596/0 (socket says 172.21.15.3:50596) 2023-10-03T10:15:18.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.544+0000 7f3c71ce2700 1 -- 172.21.15.3:0/4258975004 learned_addr learned my addr 172.21.15.3:0/4258975004 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:18.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.544+0000 7f3c724e3700 1 --2- 172.21.15.3:0/4258975004 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3c6c09e410 0x7f3c6c120dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:18.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.544+0000 7f3c714e1700 1 --2- 172.21.15.3:0/4258975004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6c09e000 0x7f3c6c120890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:18.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.544+0000 7f3c71ce2700 1 -- 172.21.15.3:0/4258975004 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3c6c09e410 msgr2=0x7f3c6c120dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:18.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.544+0000 7f3c71ce2700 1 --2- 172.21.15.3:0/4258975004 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3c6c09e410 0x7f3c6c120dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.544+0000 7f3c71ce2700 1 -- 172.21.15.3:0/4258975004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6c09e000 msgr2=0x7f3c6c120890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:18.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.544+0000 7f3c71ce2700 1 --2- 172.21.15.3:0/4258975004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6c09e000 0x7f3c6c120890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.544+0000 7f3c71ce2700 1 -- 172.21.15.3:0/4258975004 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c680094a0 con 0x7f3c6c09d6f0 2023-10-03T10:15:18.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.544+0000 7f3c714e1700 1 --2- 172.21.15.3:0/4258975004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6c09e000 0x7f3c6c120890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:15:18.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.544+0000 7f3c71ce2700 1 --2- 172.21.15.3:0/4258975004 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3c6c09d6f0 0x7f3c6c120350 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009ed0 tx=0x7f3c68004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:18.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.544+0000 7f3c627fc700 1 -- 172.21.15.3:0/4258975004 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c68003db0 con 0x7f3c6c09d6f0 2023-10-03T10:15:18.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.545+0000 7f3c627fc700 1 -- 172.21.15.3:0/4258975004 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6801f070 con 0x7f3c6c09d6f0 2023-10-03T10:15:18.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.545+0000 7f3c73f46700 1 -- 172.21.15.3:0/4258975004 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c6c06e840 con 0x7f3c6c09d6f0 2023-10-03T10:15:18.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.545+0000 7f3c73f46700 1 -- 172.21.15.3:0/4258975004 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c6c09c700 con 0x7f3c6c09d6f0 2023-10-03T10:15:18.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.545+0000 7f3c627fc700 1 -- 172.21.15.3:0/4258975004 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c68011870 con 0x7f3c6c09d6f0 2023-10-03T10:15:18.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.546+0000 7f3c627fc700 1 -- 172.21.15.3:0/4258975004 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3c68021030 con 0x7f3c6c09d6f0 2023-10-03T10:15:18.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.546+0000 7f3c73f46700 1 -- 172.21.15.3:0/4258975004 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c540052e0 con 0x7f3c6c09d6f0 2023-10-03T10:15:18.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.546+0000 7f3c627fc700 1 -- 172.21.15.3:0/4258975004 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3c68040930 con 0x7f3c6c09d6f0 2023-10-03T10:15:18.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.551+0000 7f3c627fc700 1 -- 172.21.15.3:0/4258975004 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3c68011db0 con 0x7f3c6c09d6f0 2023-10-03T10:15:18.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.608+0000 7ffa1ff71700 1 Processor -- start 2023-10-03T10:15:18.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.609+0000 7ffa1ff71700 1 -- start start 2023-10-03T10:15:18.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.609+0000 7ffa1ff71700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffa180a0a30 0x7ffa180a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:18.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.609+0000 7ffa1ff71700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa1809d630 0x7ffa180a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:18.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.609+0000 7ffa1ff71700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa180a2a70 con 0x7ffa18133930 2023-10-03T10:15:18.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.609+0000 7ffa1ff71700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa180a01d0 con 0x7ffa1809d630 2023-10-03T10:15:18.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.609+0000 7ffa1ff71700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1809e2b0 con 0x7ffa180a0a30 2023-10-03T10:15:18.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.609+0000 7ffa1e50e700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ffa18133930 0x7ffa18133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47732/0 (socket says 172.21.15.3:47732) 2023-10-03T10:15:18.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.609+0000 7ffa1e50e700 1 -- 172.21.15.3:0/3909917660 learned_addr learned my addr 172.21.15.3:0/3909917660 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:18.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.609+0000 7ffa1d50c700 1 --2- 172.21.15.3:0/3909917660 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa1809d630 0x7ffa180a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:18.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.609+0000 7ffa1dd0d700 1 --2- 172.21.15.3:0/3909917660 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffa180a0a30 0x7ffa180a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:18.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.610+0000 7ffa1d50c700 1 -- 172.21.15.3:0/3909917660 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffa180a0a30 msgr2=0x7ffa180a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:18.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.610+0000 7ffa1d50c700 1 --2- 172.21.15.3:0/3909917660 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffa180a0a30 0x7ffa180a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.610+0000 7ffa1d50c700 1 -- 172.21.15.3:0/3909917660 >> v1:172.21.15.3:6789/0 conn(0x7ffa18133930 legacy=0x7ffa18133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:18.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.610+0000 7ffa1d50c700 1 -- 172.21.15.3:0/3909917660 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1809dc50 con 0x7ffa1809d630 2023-10-03T10:15:18.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.610+0000 7ffa1d50c700 1 --2- 172.21.15.3:0/3909917660 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa1809d630 0x7ffa180a1340 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009870 tx=0x7ffa08009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f0ebd5c852bc295 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:18.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.610+0000 7ffa1cd0b700 1 -- 172.21.15.3:0/3909917660 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa08011830 con 0x7ffa1809d630 2023-10-03T10:15:18.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.610+0000 7ffa1cd0b700 1 -- 172.21.15.3:0/3909917660 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa08011990 con 0x7ffa1809d630 2023-10-03T10:15:18.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.610+0000 7ffa1cd0b700 1 -- 172.21.15.3:0/3909917660 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa08011d20 con 0x7ffa1809d630 2023-10-03T10:15:18.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.611+0000 7ffa1ff71700 1 -- 172.21.15.3:0/3909917660 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa1809d630 msgr2=0x7ffa180a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:18.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.611+0000 7ffa1ff71700 1 --2- 172.21.15.3:0/3909917660 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa1809d630 0x7ffa180a1340 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009870 tx=0x7ffa08009b80 comp rx=0 tx=0).stop 2023-10-03T10:15:18.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.611+0000 7ffa1ff71700 1 -- 172.21.15.3:0/3909917660 shutdown_connections 2023-10-03T10:15:18.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.611+0000 7ffa1ff71700 1 --2- 172.21.15.3:0/3909917660 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffa180a0a30 0x7ffa180a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.611+0000 7ffa1ff71700 1 --2- 172.21.15.3:0/3909917660 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa1809d630 0x7ffa180a1340 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.611+0000 7ffa1ff71700 1 -- 172.21.15.3:0/3909917660 >> 172.21.15.3:0/3909917660 conn(0x7ffa180a3050 msgr2=0x7ffa180a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:18.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.611+0000 7ffa1ff71700 1 -- 172.21.15.3:0/3909917660 shutdown_connections 2023-10-03T10:15:18.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.612+0000 7ffa1ff71700 1 -- 172.21.15.3:0/3909917660 wait complete. 2023-10-03T10:15:18.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.613+0000 7ffa1ff71700 1 Processor -- start 2023-10-03T10:15:18.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.613+0000 7ffa1ff71700 1 -- start start 2023-10-03T10:15:18.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.614+0000 7ffa1ff71700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa1809d630 0x7ffa181ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:18.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.614+0000 7ffa1ff71700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffa180a0a30 0x7ffa181cd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:18.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.614+0000 7ffa1ff71700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa18133930 0x7ffa1806dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:18.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.614+0000 7ffa1ff71700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa180a5040 con 0x7ffa1809d630 2023-10-03T10:15:18.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.614+0000 7ffa1ff71700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1806fbf0 con 0x7ffa18133930 2023-10-03T10:15:18.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.614+0000 7ffa1ff71700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa18132410 con 0x7ffa180a0a30 2023-10-03T10:15:18.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.614+0000 7ffa1d50c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffa180a0a30 0x7ffa181cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:18.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.614+0000 7ffa1d50c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffa180a0a30 0x7ffa181cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53910/0 (socket says 172.21.15.3:53910) 2023-10-03T10:15:18.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.614+0000 7ffa1d50c700 1 -- 172.21.15.3:0/1994350544 learned_addr learned my addr 172.21.15.3:0/1994350544 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:18.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.614+0000 7ffa1e50e700 1 --2- 172.21.15.3:0/1994350544 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa18133930 0x7ffa1806dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:18.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.614+0000 7ffa1dd0d700 1 --2- 172.21.15.3:0/1994350544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa1809d630 0x7ffa181ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:18.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.615+0000 7ffa1d50c700 1 -- 172.21.15.3:0/1994350544 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa18133930 msgr2=0x7ffa1806dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:18.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.615+0000 7ffa1d50c700 1 --2- 172.21.15.3:0/1994350544 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa18133930 0x7ffa1806dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.615+0000 7ffa1d50c700 1 -- 172.21.15.3:0/1994350544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa1809d630 msgr2=0x7ffa181ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:18.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.615+0000 7ffa1d50c700 1 --2- 172.21.15.3:0/1994350544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa1809d630 0x7ffa181ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.615+0000 7ffa1d50c700 1 -- 172.21.15.3:0/1994350544 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa080094a0 con 0x7ffa180a0a30 2023-10-03T10:15:18.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.615+0000 7ffa1d50c700 1 --2- 172.21.15.3:0/1994350544 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffa180a0a30 0x7ffa181cd140 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009bc0 tx=0x7ffa08004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:18.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.615+0000 7ffa0e7fc700 1 -- 172.21.15.3:0/1994350544 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa08004450 con 0x7ffa180a0a30 2023-10-03T10:15:18.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.615+0000 7ffa0e7fc700 1 -- 172.21.15.3:0/1994350544 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa0801f070 con 0x7ffa180a0a30 2023-10-03T10:15:18.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.615+0000 7ffa0e7fc700 1 -- 172.21.15.3:0/1994350544 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa080038c0 con 0x7ffa180a0a30 2023-10-03T10:15:18.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.615+0000 7ffa1ff71700 1 -- 172.21.15.3:0/1994350544 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa18071270 con 0x7ffa180a0a30 2023-10-03T10:15:18.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.615+0000 7ffa1ff71700 1 -- 172.21.15.3:0/1994350544 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa181cd680 con 0x7ffa180a0a30 2023-10-03T10:15:18.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.616+0000 7f9bd9dee700 1 Processor -- start 2023-10-03T10:15:18.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.616+0000 7f9bd9dee700 1 -- start start 2023-10-03T10:15:18.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.616+0000 7ffa1ff71700 1 -- 172.21.15.3:0/1994350544 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa180710b0 con 0x7ffa180a0a30 2023-10-03T10:15:18.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.617+0000 7f9bd9dee700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9bd41316c0 0x7f9bd4131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:18.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.617+0000 7f9bd9dee700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9bd409d9f0 0x7f9bd406a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:18.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.617+0000 7f9bd9dee700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9bd406acb0 0x7f9bd406b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:18.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.617+0000 7f9bd9dee700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bd406b630 con 0x7f9bd409d9f0 2023-10-03T10:15:18.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.617+0000 7ffa0e7fc700 1 -- 172.21.15.3:0/1994350544 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffa08003a20 con 0x7ffa180a0a30 2023-10-03T10:15:18.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.617+0000 7f9bd9dee700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd409e130 con 0x7f9bd41316c0 2023-10-03T10:15:18.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.617+0000 7f9bd9dee700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd41312b0 con 0x7f9bd406acb0 2023-10-03T10:15:18.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.617+0000 7ffa0e7fc700 1 -- 172.21.15.3:0/1994350544 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ffa08021070 con 0x7ffa180a0a30 2023-10-03T10:15:18.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.617+0000 7f9bd37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9bd41316c0 0x7f9bd4131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:18.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.617+0000 7f9bd37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9bd41316c0 0x7f9bd4131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50608/0 (socket says 172.21.15.3:50608) 2023-10-03T10:15:18.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.617+0000 7f9bd37fe700 1 -- 172.21.15.3:0/1770132212 learned_addr learned my addr 172.21.15.3:0/1770132212 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:18.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.617+0000 7f9bd3fff700 1 --2- 172.21.15.3:0/1770132212 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9bd406acb0 0x7f9bd406b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:18.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.618+0000 7f9bd37fe700 1 -- 172.21.15.3:0/1770132212 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9bd406acb0 msgr2=0x7f9bd406b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:18.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.618+0000 7f9bd37fe700 1 --2- 172.21.15.3:0/1770132212 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9bd406acb0 0x7f9bd406b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.618+0000 7f9bd37fe700 1 -- 172.21.15.3:0/1770132212 >> v2:172.21.15.3:3300/0 conn(0x7f9bd409d9f0 msgr2=0x7f9bd406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:18.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.618+0000 7f9bd37fe700 1 --2- 172.21.15.3:0/1770132212 >> v2:172.21.15.3:3300/0 conn(0x7f9bd409d9f0 0x7f9bd406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.618+0000 7f9bd37fe700 1 -- 172.21.15.3:0/1770132212 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd409f6e0 con 0x7f9bd41316c0 2023-10-03T10:15:18.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.618+0000 7f9bd37fe700 1 --2- 172.21.15.3:0/1770132212 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9bd41316c0 0x7f9bd4131a90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc009920 tx=0x7f9bbc009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d49300a8abd7d4be server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:18.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.618+0000 7f9bd27fc700 1 -- 172.21.15.3:0/1770132212 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc011830 con 0x7f9bd41316c0 2023-10-03T10:15:18.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.618+0000 7f9bd27fc700 1 -- 172.21.15.3:0/1770132212 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bbc011990 con 0x7f9bd41316c0 2023-10-03T10:15:18.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.618+0000 7f9bd27fc700 1 -- 172.21.15.3:0/1770132212 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc011d20 con 0x7f9bd41316c0 2023-10-03T10:15:18.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.619+0000 7f9bd9dee700 1 -- 172.21.15.3:0/1770132212 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9bd41316c0 msgr2=0x7f9bd4131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:18.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.619+0000 7f9bd9dee700 1 --2- 172.21.15.3:0/1770132212 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9bd41316c0 0x7f9bd4131a90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc009920 tx=0x7f9bbc009c30 comp rx=0 tx=0).stop 2023-10-03T10:15:18.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.619+0000 7f9bd9dee700 1 -- 172.21.15.3:0/1770132212 shutdown_connections 2023-10-03T10:15:18.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.619+0000 7f9bd9dee700 1 --2- 172.21.15.3:0/1770132212 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9bd406acb0 0x7f9bd406b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.619+0000 7f9bd9dee700 1 --2- 172.21.15.3:0/1770132212 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9bd41316c0 0x7f9bd4131a90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.619+0000 7f9bd9dee700 1 --2- 172.21.15.3:0/1770132212 >> v2:172.21.15.3:3300/0 conn(0x7f9bd409d9f0 0x7f9bd406a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.619+0000 7f9bd9dee700 1 -- 172.21.15.3:0/1770132212 >> 172.21.15.3:0/1770132212 conn(0x7f9bd4066f50 msgr2=0x7f9bd409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:18.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.619+0000 7f9bd9dee700 1 -- 172.21.15.3:0/1770132212 shutdown_connections 2023-10-03T10:15:18.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.619+0000 7f9bd9dee700 1 -- 172.21.15.3:0/1770132212 wait complete. 2023-10-03T10:15:18.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.620+0000 7f9bd9dee700 1 Processor -- start 2023-10-03T10:15:18.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.620+0000 7f9bd9dee700 1 -- start start 2023-10-03T10:15:18.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.621+0000 7f9bd9dee700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9bd406acb0 0x7f9bd4070aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:18.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.621+0000 7f9bd9dee700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9bd409d9f0 0x7f9bd4070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:18.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.621+0000 7f9bd9dee700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9bd41316c0 0x7f9bd4071520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:18.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.621+0000 7f9bd9dee700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd411aa40 con 0x7f9bd409d9f0 2023-10-03T10:15:18.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.621+0000 7f9bd9dee700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd4072740 con 0x7f9bd406acb0 2023-10-03T10:15:18.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.621+0000 7f9bd9dee700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd4119630 con 0x7f9bd41316c0 2023-10-03T10:15:18.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.621+0000 7f9bd37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9bd406acb0 0x7f9bd4070aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:18.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.621+0000 7f9bd37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9bd406acb0 0x7f9bd4070aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50612/0 (socket says 172.21.15.3:50612) 2023-10-03T10:15:18.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.621+0000 7f9bd37fe700 1 -- 172.21.15.3:0/966418543 learned_addr learned my addr 172.21.15.3:0/966418543 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:18.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.621+0000 7f9bd2ffd700 1 --2- 172.21.15.3:0/966418543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9bd409d9f0 0x7f9bd4070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:18.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.621+0000 7f9bd3fff700 1 --2- 172.21.15.3:0/966418543 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9bd41316c0 0x7f9bd4071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:18.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.622+0000 7f9bd3fff700 1 -- 172.21.15.3:0/966418543 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9bd406acb0 msgr2=0x7f9bd4070aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:18.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.622+0000 7f9bd3fff700 1 --2- 172.21.15.3:0/966418543 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9bd406acb0 0x7f9bd4070aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.622+0000 7f9bd3fff700 1 -- 172.21.15.3:0/966418543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9bd409d9f0 msgr2=0x7f9bd4070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:18.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.622+0000 7f9bd3fff700 1 --2- 172.21.15.3:0/966418543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9bd409d9f0 0x7f9bd4070fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.622+0000 7f9bd3fff700 1 -- 172.21.15.3:0/966418543 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bbc009550 con 0x7f9bd41316c0 2023-10-03T10:15:18.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.622+0000 7f9bd37fe700 1 --2- 172.21.15.3:0/966418543 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9bd406acb0 0x7f9bd4070aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:18.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.622+0000 7f9bd3fff700 1 --2- 172.21.15.3:0/966418543 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9bd41316c0 0x7f9bd4071520 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc00efc0 tx=0x7f9bcc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:18.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.626+0000 7f9bcbfff700 1 -- 172.21.15.3:0/966418543 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc00cc90 con 0x7f9bd41316c0 2023-10-03T10:15:18.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.626+0000 7f9bcbfff700 1 -- 172.21.15.3:0/966418543 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bcc00f040 con 0x7f9bd41316c0 2023-10-03T10:15:18.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.626+0000 7f9bcbfff700 1 -- 172.21.15.3:0/966418543 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc0146c0 con 0x7f9bd41316c0 2023-10-03T10:15:18.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.626+0000 7f9bd9dee700 1 -- 172.21.15.3:0/966418543 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd411afe0 con 0x7f9bd41316c0 2023-10-03T10:15:18.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.626+0000 7f9bd9dee700 1 -- 172.21.15.3:0/966418543 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd411ad00 con 0x7f9bd41316c0 2023-10-03T10:15:18.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.626+0000 7ffa0e7fc700 1 -- 172.21.15.3:0/1994350544 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffa080045b0 con 0x7ffa180a0a30 2023-10-03T10:15:18.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.627+0000 7f9bcbfff700 1 -- 172.21.15.3:0/966418543 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9bcc014820 con 0x7f9bd41316c0 2023-10-03T10:15:18.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.628+0000 7f9bcbfff700 1 -- 172.21.15.3:0/966418543 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9bcc040120 con 0x7f9bd41316c0 2023-10-03T10:15:18.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.628+0000 7f9bd9dee700 1 -- 172.21.15.3:0/966418543 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd4074ea0 con 0x7f9bd41316c0 2023-10-03T10:15:18.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.637+0000 7f9bcbfff700 1 -- 172.21.15.3:0/966418543 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9bcc014ad0 con 0x7f9bd41316c0 2023-10-03T10:15:18.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.722+0000 7f27a6236700 1 -- 172.21.15.3:0/2867610183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f27a0074ea0 con 0x7f27a009d6f0 2023-10-03T10:15:18.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.722+0000 7f2787fff700 1 -- 172.21.15.3:0/2867610183 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2790018400 con 0x7f27a009d6f0 2023-10-03T10:15:18.723 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:18.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.725+0000 7f27a6236700 1 -- 172.21.15.3:0/2867610183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27a009d6f0 msgr2=0x7f27a01224b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:18.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.725+0000 7f27a6236700 1 --2- 172.21.15.3:0/2867610183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27a009d6f0 0x7f27a01224b0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f279000eda0 tx=0x7f279000d630 comp rx=0 tx=0).stop 2023-10-03T10:15:18.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.725+0000 7f27a6236700 1 -- 172.21.15.3:0/2867610183 shutdown_connections 2023-10-03T10:15:18.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.725+0000 7f27a6236700 1 --2- 172.21.15.3:0/2867610183 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f27a009e410 0x7f27a0122f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.725+0000 7f27a6236700 1 --2- 172.21.15.3:0/2867610183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27a009d6f0 0x7f27a01224b0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.725+0000 7f27a6236700 1 --2- 172.21.15.3:0/2867610183 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f27a009e000 0x7f27a01229f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.725+0000 7f27a6236700 1 -- 172.21.15.3:0/2867610183 >> 172.21.15.3:0/2867610183 conn(0x7f27a0066f50 msgr2=0x7f27a0130b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:18.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.725+0000 7f27a6236700 1 -- 172.21.15.3:0/2867610183 shutdown_connections 2023-10-03T10:15:18.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.725+0000 7f27a6236700 1 -- 172.21.15.3:0/2867610183 wait complete. 2023-10-03T10:15:18.739 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:18.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.771+0000 7f3c73f46700 1 -- 172.21.15.3:0/4258975004 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3c54005590 con 0x7f3c6c09d6f0 2023-10-03T10:15:18.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.772+0000 7f3c627fc700 1 -- 172.21.15.3:0/4258975004 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3c680193c0 con 0x7f3c6c09d6f0 2023-10-03T10:15:18.772 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:18.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.774+0000 7f3c73f46700 1 -- 172.21.15.3:0/4258975004 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3c6c09d6f0 msgr2=0x7f3c6c120350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:18.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.774+0000 7f3c73f46700 1 --2- 172.21.15.3:0/4258975004 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3c6c09d6f0 0x7f3c6c120350 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009ed0 tx=0x7f3c68004750 comp rx=0 tx=0).stop 2023-10-03T10:15:18.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.774+0000 7f3c73f46700 1 -- 172.21.15.3:0/4258975004 shutdown_connections 2023-10-03T10:15:18.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.774+0000 7f3c73f46700 1 --2- 172.21.15.3:0/4258975004 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3c6c09e410 0x7f3c6c120dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.774+0000 7f3c73f46700 1 --2- 172.21.15.3:0/4258975004 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3c6c09d6f0 0x7f3c6c120350 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.774+0000 7f3c73f46700 1 --2- 172.21.15.3:0/4258975004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6c09e000 0x7f3c6c120890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.774+0000 7f3c73f46700 1 -- 172.21.15.3:0/4258975004 >> 172.21.15.3:0/4258975004 conn(0x7f3c6c066f50 msgr2=0x7f3c6c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:18.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.774+0000 7f3c73f46700 1 -- 172.21.15.3:0/4258975004 shutdown_connections 2023-10-03T10:15:18.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.774+0000 7f3c73f46700 1 -- 172.21.15.3:0/4258975004 wait complete. 2023-10-03T10:15:18.788 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:18.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.832+0000 7ffa1ff71700 1 -- 172.21.15.3:0/1994350544 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ffa180a0410 con 0x7ffa180a0a30 2023-10-03T10:15:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.833+0000 7ffa0e7fc700 1 -- 172.21.15.3:0/1994350544 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffa08034090 con 0x7ffa180a0a30 2023-10-03T10:15:18.833 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.835+0000 7ffa1ff71700 1 -- 172.21.15.3:0/1994350544 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffa180a0a30 msgr2=0x7ffa181cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.835+0000 7ffa1ff71700 1 --2- 172.21.15.3:0/1994350544 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffa180a0a30 0x7ffa181cd140 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009bc0 tx=0x7ffa08004750 comp rx=0 tx=0).stop 2023-10-03T10:15:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.835+0000 7ffa1ff71700 1 -- 172.21.15.3:0/1994350544 shutdown_connections 2023-10-03T10:15:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.835+0000 7ffa1ff71700 1 --2- 172.21.15.3:0/1994350544 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa18133930 0x7ffa1806dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.835+0000 7ffa1ff71700 1 --2- 172.21.15.3:0/1994350544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa1809d630 0x7ffa181ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.835+0000 7ffa1ff71700 1 --2- 172.21.15.3:0/1994350544 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffa180a0a30 0x7ffa181cd140 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.835+0000 7ffa1ff71700 1 -- 172.21.15.3:0/1994350544 >> 172.21.15.3:0/1994350544 conn(0x7ffa180a3050 msgr2=0x7ffa18133210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.835+0000 7ffa1ff71700 1 -- 172.21.15.3:0/1994350544 shutdown_connections 2023-10-03T10:15:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.835+0000 7ffa1ff71700 1 -- 172.21.15.3:0/1994350544 wait complete. 2023-10-03T10:15:18.848 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:18.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.865+0000 7f9bd9dee700 1 -- 172.21.15.3:0/966418543 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9bd409e980 con 0x7f9bd41316c0 2023-10-03T10:15:18.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.865+0000 7f9bcbfff700 1 -- 172.21.15.3:0/966418543 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9bcc02b090 con 0x7f9bd41316c0 2023-10-03T10:15:18.866 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:18.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.868+0000 7f9bd9dee700 1 -- 172.21.15.3:0/966418543 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9bd41316c0 msgr2=0x7f9bd4071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:18.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.868+0000 7f9bd9dee700 1 --2- 172.21.15.3:0/966418543 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9bd41316c0 0x7f9bd4071520 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc00efc0 tx=0x7f9bcc00c3a0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.868+0000 7f9bd9dee700 1 -- 172.21.15.3:0/966418543 shutdown_connections 2023-10-03T10:15:18.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.868+0000 7f9bd9dee700 1 --2- 172.21.15.3:0/966418543 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9bd41316c0 0x7f9bd4071520 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.868+0000 7f9bd9dee700 1 --2- 172.21.15.3:0/966418543 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9bd406acb0 0x7f9bd4070aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.868+0000 7f9bd9dee700 1 --2- 172.21.15.3:0/966418543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9bd409d9f0 0x7f9bd4070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:18.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.868+0000 7f9bd9dee700 1 -- 172.21.15.3:0/966418543 >> 172.21.15.3:0/966418543 conn(0x7f9bd4066f50 msgr2=0x7f9bd40a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:18.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.868+0000 7f9bd9dee700 1 -- 172.21.15.3:0/966418543 shutdown_connections 2023-10-03T10:15:18.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:18.868+0000 7f9bd9dee700 1 -- 172.21.15.3:0/966418543 wait complete. 2023-10-03T10:15:18.882 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:19.740 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:19.788 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:19.849 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:19.882 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:19.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.890+0000 7f48d8e0e700 1 Processor -- start 2023-10-03T10:15:19.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.890+0000 7f48d8e0e700 1 -- start start 2023-10-03T10:15:19.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.891+0000 7f48d8e0e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48d41338c0 0x7f48d4133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:19.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.891+0000 7f48d8e0e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48d41341d0 0x7f48d409de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:19.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.891+0000 7f48d8e0e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f48d409e390 0x7f48d409e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:19.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.891+0000 7f48d8e0e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d4070c80 con 0x7f48d409e390 2023-10-03T10:15:19.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.891+0000 7f48d8e0e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d406f720 con 0x7f48d41338c0 2023-10-03T10:15:19.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.891+0000 7f48d8e0e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f48d406fce0 con 0x7f48d41341d0 2023-10-03T10:15:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.892+0000 7f48d259c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48d41338c0 0x7f48d4133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.892+0000 7f48d259c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48d41338c0 0x7f48d4133c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50616/0 (socket says 172.21.15.3:50616) 2023-10-03T10:15:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.892+0000 7f48d259c700 1 -- 172.21.15.3:0/4235859871 learned_addr learned my addr 172.21.15.3:0/4235859871 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.892+0000 7f48d1d9b700 1 --2- 172.21.15.3:0/4235859871 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48d41341d0 0x7f48d409de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.892+0000 7f48d259c700 1 -- 172.21.15.3:0/4235859871 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48d41341d0 msgr2=0x7f48d409de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.892+0000 7f48d259c700 1 --2- 172.21.15.3:0/4235859871 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48d41341d0 0x7f48d409de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.892+0000 7f48d259c700 1 -- 172.21.15.3:0/4235859871 >> v2:172.21.15.3:3300/0 conn(0x7f48d409e390 msgr2=0x7f48d409e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.892+0000 7f48d259c700 1 --2- 172.21.15.3:0/4235859871 >> v2:172.21.15.3:3300/0 conn(0x7f48d409e390 0x7f48d409e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.892+0000 7f48d259c700 1 -- 172.21.15.3:0/4235859871 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d41336a0 con 0x7f48d41338c0 2023-10-03T10:15:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.892+0000 7f48d259c700 1 --2- 172.21.15.3:0/4235859871 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48d41338c0 0x7f48d4133c90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f48c40098f0 tx=0x7f48c4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f82dcea92e18580 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.892+0000 7f48d159a700 1 -- 172.21.15.3:0/4235859871 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c4011830 con 0x7f48d41338c0 2023-10-03T10:15:19.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.892+0000 7f48d159a700 1 -- 172.21.15.3:0/4235859871 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c4011990 con 0x7f48d41338c0 2023-10-03T10:15:19.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.892+0000 7f48d159a700 1 -- 172.21.15.3:0/4235859871 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c4011d20 con 0x7f48d41338c0 2023-10-03T10:15:19.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.893+0000 7f48d8e0e700 1 -- 172.21.15.3:0/4235859871 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48d41338c0 msgr2=0x7f48d4133c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:19.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.893+0000 7f48d8e0e700 1 --2- 172.21.15.3:0/4235859871 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48d41338c0 0x7f48d4133c90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f48c40098f0 tx=0x7f48c4009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:19.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.893+0000 7f48d8e0e700 1 -- 172.21.15.3:0/4235859871 shutdown_connections 2023-10-03T10:15:19.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.893+0000 7f48d8e0e700 1 --2- 172.21.15.3:0/4235859871 >> v2:172.21.15.3:3300/0 conn(0x7f48d409e390 0x7f48d409e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:19.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.893+0000 7f48d8e0e700 1 --2- 172.21.15.3:0/4235859871 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48d41338c0 0x7f48d4133c90 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:19.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.893+0000 7f48d8e0e700 1 --2- 172.21.15.3:0/4235859871 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48d41341d0 0x7f48d409de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:19.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.893+0000 7f48d8e0e700 1 -- 172.21.15.3:0/4235859871 >> 172.21.15.3:0/4235859871 conn(0x7f48d40a2aa0 msgr2=0x7f48d40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:19.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.893+0000 7f48d8e0e700 1 -- 172.21.15.3:0/4235859871 shutdown_connections 2023-10-03T10:15:19.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.893+0000 7f48d8e0e700 1 -- 172.21.15.3:0/4235859871 wait complete. 2023-10-03T10:15:19.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.895+0000 7f48d8e0e700 1 Processor -- start 2023-10-03T10:15:19.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.895+0000 7f48d8e0e700 1 -- start start 2023-10-03T10:15:19.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.896+0000 7f48d8e0e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48d409e390 0x7f48d411f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:19.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.896+0000 7f48d8e0e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48d41338c0 0x7f48d411fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:19.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.896+0000 7f48d8e0e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48d41341d0 0x7f48d41203b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:19.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.896+0000 7f48d8e0e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d4123450 con 0x7f48d409e390 2023-10-03T10:15:19.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.896+0000 7f48d8e0e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d41320a0 con 0x7f48d41341d0 2023-10-03T10:15:19.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.896+0000 7f48d8e0e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f48d406f960 con 0x7f48d41338c0 2023-10-03T10:15:19.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.896+0000 7f48d259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48d409e390 0x7f48d411f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:19.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.896+0000 7f48d259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48d409e390 0x7f48d411f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42536/0 (socket says 172.21.15.3:42536) 2023-10-03T10:15:19.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.896+0000 7f48d259c700 1 -- 172.21.15.3:0/4042430028 learned_addr learned my addr 172.21.15.3:0/4042430028 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:19.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.896+0000 7f48d2d9d700 1 --2- 172.21.15.3:0/4042430028 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48d41341d0 0x7f48d41203b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:19.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.897+0000 7f48d1d9b700 1 --2- 172.21.15.3:0/4042430028 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48d41338c0 0x7f48d411fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:19.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.897+0000 7f48d259c700 1 -- 172.21.15.3:0/4042430028 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48d41338c0 msgr2=0x7f48d411fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:19.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.897+0000 7f48d259c700 1 --2- 172.21.15.3:0/4042430028 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48d41338c0 0x7f48d411fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:19.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.897+0000 7f48d259c700 1 -- 172.21.15.3:0/4042430028 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48d41341d0 msgr2=0x7f48d41203b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:19.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.897+0000 7f48d259c700 1 --2- 172.21.15.3:0/4042430028 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48d41341d0 0x7f48d41203b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:19.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.897+0000 7f48d259c700 1 -- 172.21.15.3:0/4042430028 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c4009550 con 0x7f48d409e390 2023-10-03T10:15:19.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.897+0000 7f48d1d9b700 1 --2- 172.21.15.3:0/4042430028 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48d41338c0 0x7f48d411fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:15:19.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.897+0000 7f48d259c700 1 --2- 172.21.15.3:0/4042430028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48d409e390 0x7f48d411f930 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f48c4000c00 tx=0x7f48c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:19.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.897+0000 7f48c2ffd700 1 -- 172.21.15.3:0/4042430028 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c4004450 con 0x7f48d409e390 2023-10-03T10:15:19.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.897+0000 7f48c2ffd700 1 -- 172.21.15.3:0/4042430028 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c40045b0 con 0x7f48d409e390 2023-10-03T10:15:19.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.897+0000 7f48c2ffd700 1 -- 172.21.15.3:0/4042430028 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c40038c0 con 0x7f48d409e390 2023-10-03T10:15:19.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.897+0000 7f48d8e0e700 1 -- 172.21.15.3:0/4042430028 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d4122790 con 0x7f48d409e390 2023-10-03T10:15:19.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.897+0000 7f48d8e0e700 1 -- 172.21.15.3:0/4042430028 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d4122be0 con 0x7f48d409e390 2023-10-03T10:15:19.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.899+0000 7f48d8e0e700 1 -- 172.21.15.3:0/4042430028 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d4122dc0 con 0x7f48d409e390 2023-10-03T10:15:19.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.899+0000 7f48c2ffd700 1 -- 172.21.15.3:0/4042430028 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f48c4021070 con 0x7f48d409e390 2023-10-03T10:15:19.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.899+0000 7f48c2ffd700 1 -- 172.21.15.3:0/4042430028 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f48c4040bf0 con 0x7f48d409e390 2023-10-03T10:15:19.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.904+0000 7f48c2ffd700 1 -- 172.21.15.3:0/4042430028 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f48c402d760 con 0x7f48d409e390 2023-10-03T10:15:19.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.938+0000 7ff6084d4700 1 Processor -- start 2023-10-03T10:15:19.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.938+0000 7ff6084d4700 1 -- start start 2023-10-03T10:15:19.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.939+0000 7ff6084d4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff60009d6f0 0x7ff60009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:19.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.939+0000 7ff6084d4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff60009e000 0x7ff600130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:19.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.939+0000 7ff6084d4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff60009e410 0x7ff600130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:19.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.939+0000 7ff6084d4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff60009f520 con 0x7ff60009d6f0 2023-10-03T10:15:19.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.939+0000 7ff6084d4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60009f130 con 0x7ff60009e000 2023-10-03T10:15:19.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.939+0000 7ff6084d4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff60009e930 con 0x7ff60009e410 2023-10-03T10:15:19.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.939+0000 7ff605a6f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff60009e000 0x7ff600130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:19.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.939+0000 7ff606a71700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff60009e410 0x7ff600130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:19.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.939+0000 7ff606a71700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff60009e410 0x7ff600130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53962/0 (socket says 172.21.15.3:53962) 2023-10-03T10:15:19.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.939+0000 7ff605a6f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff60009e000 0x7ff600130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50640/0 (socket says 172.21.15.3:50640) 2023-10-03T10:15:19.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.939+0000 7ff606a71700 1 -- 172.21.15.3:0/2652256709 learned_addr learned my addr 172.21.15.3:0/2652256709 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:19.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.939+0000 7ff606270700 1 --2- 172.21.15.3:0/2652256709 >> v2:172.21.15.3:3300/0 conn(0x7ff60009d6f0 0x7ff60009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:19.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.939+0000 7ff605a6f700 1 -- 172.21.15.3:0/2652256709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff60009e410 msgr2=0x7ff600130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:19.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.939+0000 7ff605a6f700 1 --2- 172.21.15.3:0/2652256709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff60009e410 0x7ff600130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:19.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.939+0000 7ff605a6f700 1 -- 172.21.15.3:0/2652256709 >> v2:172.21.15.3:3300/0 conn(0x7ff60009d6f0 msgr2=0x7ff60009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:19.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.939+0000 7ff605a6f700 1 --2- 172.21.15.3:0/2652256709 >> v2:172.21.15.3:3300/0 conn(0x7ff60009d6f0 0x7ff60009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:19.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.939+0000 7ff605a6f700 1 -- 172.21.15.3:0/2652256709 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60009b2d0 con 0x7ff60009e000 2023-10-03T10:15:19.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.940+0000 7ff605a6f700 1 --2- 172.21.15.3:0/2652256709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff60009e000 0x7ff600130660 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0009840 tx=0x7ff5f0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a56ebf2a7dd8b8ab server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:19.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.940+0000 7ff60526e700 1 -- 172.21.15.3:0/2652256709 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0011830 con 0x7ff60009e000 2023-10-03T10:15:19.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.940+0000 7ff60526e700 1 -- 172.21.15.3:0/2652256709 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f0011990 con 0x7ff60009e000 2023-10-03T10:15:19.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.940+0000 7ff60526e700 1 -- 172.21.15.3:0/2652256709 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0011d20 con 0x7ff60009e000 2023-10-03T10:15:19.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.941+0000 7ff6084d4700 1 -- 172.21.15.3:0/2652256709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff60009e000 msgr2=0x7ff600130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:19.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.941+0000 7ff6084d4700 1 --2- 172.21.15.3:0/2652256709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff60009e000 0x7ff600130660 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0009840 tx=0x7ff5f0009b50 comp rx=0 tx=0).stop 2023-10-03T10:15:19.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.941+0000 7ff6084d4700 1 -- 172.21.15.3:0/2652256709 shutdown_connections 2023-10-03T10:15:19.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.941+0000 7ff6084d4700 1 --2- 172.21.15.3:0/2652256709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff60009e410 0x7ff600130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:19.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.941+0000 7ff6084d4700 1 --2- 172.21.15.3:0/2652256709 >> v2:172.21.15.3:3300/0 conn(0x7ff60009d6f0 0x7ff60009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:19.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.941+0000 7ff6084d4700 1 --2- 172.21.15.3:0/2652256709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff60009e000 0x7ff600130660 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:19.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.941+0000 7ff6084d4700 1 -- 172.21.15.3:0/2652256709 >> 172.21.15.3:0/2652256709 conn(0x7ff600066f50 msgr2=0x7ff60009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:19.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.941+0000 7ff6084d4700 1 -- 172.21.15.3:0/2652256709 shutdown_connections 2023-10-03T10:15:19.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.941+0000 7ff6084d4700 1 -- 172.21.15.3:0/2652256709 wait complete. 2023-10-03T10:15:19.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.942+0000 7ff6084d4700 1 Processor -- start 2023-10-03T10:15:19.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.942+0000 7ff6084d4700 1 -- start start 2023-10-03T10:15:19.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.943+0000 7ff6084d4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff60009d6f0 0x7ff60011dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:19.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.943+0000 7ff6084d4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff60009e000 0x7ff60011e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:19.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.943+0000 7ff6084d4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff60009e410 0x7ff60011ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:19.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.943+0000 7ff6084d4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60011f4a0 con 0x7ff60009d6f0 2023-10-03T10:15:19.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.943+0000 7ff6084d4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff600121b10 con 0x7ff60009e410 2023-10-03T10:15:19.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.943+0000 7ff6084d4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6001212a0 con 0x7ff60009e000 2023-10-03T10:15:19.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.943+0000 7ff606270700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff60009d6f0 0x7ff60011dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:19.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.943+0000 7ff606270700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff60009d6f0 0x7ff60011dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42556/0 (socket says 172.21.15.3:42556) 2023-10-03T10:15:19.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.943+0000 7ff606270700 1 -- 172.21.15.3:0/1843928748 learned_addr learned my addr 172.21.15.3:0/1843928748 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:19.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.943+0000 7ff606a71700 1 --2- 172.21.15.3:0/1843928748 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff60009e410 0x7ff60011ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:19.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.943+0000 7ff606270700 1 -- 172.21.15.3:0/1843928748 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff60009e000 msgr2=0x7ff60011e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:19.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.943+0000 7ff606270700 1 --2- 172.21.15.3:0/1843928748 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff60009e000 0x7ff60011e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:19.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.943+0000 7ff606270700 1 -- 172.21.15.3:0/1843928748 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff60009e410 msgr2=0x7ff60011ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:19.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.943+0000 7ff606270700 1 --2- 172.21.15.3:0/1843928748 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff60009e410 0x7ff60011ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:19.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.943+0000 7ff606270700 1 -- 172.21.15.3:0/1843928748 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f00094a0 con 0x7ff60009d6f0 2023-10-03T10:15:19.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.943+0000 7ff606a71700 1 --2- 172.21.15.3:0/1843928748 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff60009e410 0x7ff60011ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:19.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.943+0000 7ff606270700 1 --2- 172.21.15.3:0/1843928748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff60009d6f0 0x7ff60011dff0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00e930 tx=0x7ff5fc00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:19.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.944+0000 7ff5f6ffd700 1 -- 172.21.15.3:0/1843928748 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc00ee80 con 0x7ff60009d6f0 2023-10-03T10:15:19.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.944+0000 7ff6084d4700 1 -- 172.21.15.3:0/1843928748 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6000a3b10 con 0x7ff60009d6f0 2023-10-03T10:15:19.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.944+0000 7ff6084d4700 1 -- 172.21.15.3:0/1843928748 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff600120e50 con 0x7ff60009d6f0 2023-10-03T10:15:19.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.944+0000 7ff5f6ffd700 1 -- 172.21.15.3:0/1843928748 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5fc00c850 con 0x7ff60009d6f0 2023-10-03T10:15:19.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.944+0000 7ff5f6ffd700 1 -- 172.21.15.3:0/1843928748 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc00cbc0 con 0x7ff60009d6f0 2023-10-03T10:15:19.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.945+0000 7ff5f6ffd700 1 -- 172.21.15.3:0/1843928748 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff5fc00cd20 con 0x7ff60009d6f0 2023-10-03T10:15:19.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.945+0000 7ff5f6ffd700 1 -- 172.21.15.3:0/1843928748 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff5fc03fe20 con 0x7ff60009d6f0 2023-10-03T10:15:19.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.945+0000 7ff6084d4700 1 -- 172.21.15.3:0/1843928748 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e80052e0 con 0x7ff60009d6f0 2023-10-03T10:15:19.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:19.955+0000 7ff5f6ffd700 1 -- 172.21.15.3:0/1843928748 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff5fc017070 con 0x7ff60009d6f0 2023-10-03T10:15:20.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.006+0000 7f096831c700 1 Processor -- start 2023-10-03T10:15:20.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.006+0000 7f096831c700 1 -- start start 2023-10-03T10:15:20.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.007+0000 7f096831c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0960133540 0x7f0960135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:20.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.007+0000 7f096831c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0960135e60 0x7f0960138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:20.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.007+0000 7f096831c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09600716a0 con 0x7f0960130540 2023-10-03T10:15:20.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.007+0000 7f096831c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0960071290 con 0x7f0960133540 2023-10-03T10:15:20.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.007+0000 7f096831c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0960130340 con 0x7f0960135e60 2023-10-03T10:15:20.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.007+0000 7f09668b9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0960135e60 0x7f0960138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:20.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.007+0000 7f09658b7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0960133540 0x7f0960135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:20.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.007+0000 7f09668b9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0960135e60 0x7f0960138240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53984/0 (socket says 172.21.15.3:53984) 2023-10-03T10:15:20.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.007+0000 7f09658b7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0960133540 0x7f0960135920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42560/0 (socket says 172.21.15.3:42560) 2023-10-03T10:15:20.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.007+0000 7f09668b9700 1 -- 172.21.15.3:0/1377455361 learned_addr learned my addr 172.21.15.3:0/1377455361 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:20.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.008+0000 7f09658b7700 1 -- 172.21.15.3:0/1377455361 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0960135e60 msgr2=0x7f0960138240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:20.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.008+0000 7f09658b7700 1 --2- 172.21.15.3:0/1377455361 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0960135e60 0x7f0960138240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:20.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.008+0000 7f09658b7700 1 -- 172.21.15.3:0/1377455361 >> v1:172.21.15.3:6789/0 conn(0x7f0960130540 legacy=0x7f0960130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:20.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.008+0000 7f09658b7700 1 -- 172.21.15.3:0/1377455361 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0960070b80 con 0x7f0960133540 2023-10-03T10:15:20.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.008+0000 7f09658b7700 1 --2- 172.21.15.3:0/1377455361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0960133540 0x7f0960135920 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f0950009840 tx=0x7f0950009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41898b2553f77184 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:20.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.008+0000 7f09650b6700 1 -- 172.21.15.3:0/1377455361 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0950011830 con 0x7f0960133540 2023-10-03T10:15:20.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.008+0000 7f09650b6700 1 -- 172.21.15.3:0/1377455361 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0950011990 con 0x7f0960133540 2023-10-03T10:15:20.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.008+0000 7f09650b6700 1 -- 172.21.15.3:0/1377455361 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0950011d20 con 0x7f0960133540 2023-10-03T10:15:20.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.009+0000 7f096831c700 1 -- 172.21.15.3:0/1377455361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0960133540 msgr2=0x7f0960135920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:20.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.009+0000 7f096831c700 1 --2- 172.21.15.3:0/1377455361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0960133540 0x7f0960135920 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f0950009840 tx=0x7f0950009b50 comp rx=0 tx=0).stop 2023-10-03T10:15:20.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.009+0000 7f096831c700 1 -- 172.21.15.3:0/1377455361 shutdown_connections 2023-10-03T10:15:20.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.009+0000 7f096831c700 1 --2- 172.21.15.3:0/1377455361 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0960135e60 0x7f0960138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:20.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.009+0000 7f096831c700 1 --2- 172.21.15.3:0/1377455361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0960133540 0x7f0960135920 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:20.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.009+0000 7f096831c700 1 -- 172.21.15.3:0/1377455361 >> 172.21.15.3:0/1377455361 conn(0x7f09600a2aa0 msgr2=0x7f09600a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:20.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.009+0000 7f096831c700 1 -- 172.21.15.3:0/1377455361 shutdown_connections 2023-10-03T10:15:20.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.009+0000 7f096831c700 1 -- 172.21.15.3:0/1377455361 wait complete. 2023-10-03T10:15:20.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.011+0000 7f096831c700 1 Processor -- start 2023-10-03T10:15:20.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.011+0000 7f096831c700 1 -- start start 2023-10-03T10:15:20.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.011+0000 7f096831c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0960130540 0x7f096012cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:20.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.011+0000 7f096831c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0960133540 0x7f096012d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:20.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.011+0000 7f096831c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0960135e60 0x7f09601c6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:20.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.011+0000 7f096831c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096011aba0 con 0x7f0960133540 2023-10-03T10:15:20.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.011+0000 7f096831c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096011a970 con 0x7f0960130540 2023-10-03T10:15:20.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.011+0000 7f096831c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f09600a0580 con 0x7f0960135e60 2023-10-03T10:15:20.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.012+0000 7f09658b7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0960133540 0x7f096012d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:20.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.012+0000 7f09658b7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0960133540 0x7f096012d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42570/0 (socket says 172.21.15.3:42570) 2023-10-03T10:15:20.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.012+0000 7f09658b7700 1 -- 172.21.15.3:0/744039013 learned_addr learned my addr 172.21.15.3:0/744039013 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:20.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.012+0000 7f09660b8700 1 --2- 172.21.15.3:0/744039013 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0960130540 0x7f096012cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:20.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.012+0000 7f09668b9700 1 --2- 172.21.15.3:0/744039013 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0960135e60 0x7f09601c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:20.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.012+0000 7f09658b7700 1 -- 172.21.15.3:0/744039013 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0960135e60 msgr2=0x7f09601c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:20.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.012+0000 7f09658b7700 1 --2- 172.21.15.3:0/744039013 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0960135e60 0x7f09601c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:20.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.012+0000 7f09658b7700 1 -- 172.21.15.3:0/744039013 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0960130540 msgr2=0x7f096012cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:20.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.012+0000 7f09658b7700 1 --2- 172.21.15.3:0/744039013 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0960130540 0x7f096012cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:20.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.012+0000 7f09658b7700 1 -- 172.21.15.3:0/744039013 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09500094a0 con 0x7f0960133540 2023-10-03T10:15:20.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.012+0000 7f09658b7700 1 --2- 172.21.15.3:0/744039013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0960133540 0x7f096012d080 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f0950013000 tx=0x7f0950004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:20.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.012+0000 7f0956ffd700 1 -- 172.21.15.3:0/744039013 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0950004480 con 0x7f0960133540 2023-10-03T10:15:20.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.012+0000 7f0956ffd700 1 -- 172.21.15.3:0/744039013 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095001f070 con 0x7f0960133540 2023-10-03T10:15:20.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.012+0000 7f0956ffd700 1 -- 172.21.15.3:0/744039013 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09500038e0 con 0x7f0960133540 2023-10-03T10:15:20.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.013+0000 7f096831c700 1 -- 172.21.15.3:0/744039013 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09600718e0 con 0x7f0960133540 2023-10-03T10:15:20.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.013+0000 7f096831c700 1 -- 172.21.15.3:0/744039013 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096012d5c0 con 0x7f0960133540 2023-10-03T10:15:20.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.014+0000 7f096831c700 1 -- 172.21.15.3:0/744039013 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f096011c750 con 0x7f0960133540 2023-10-03T10:15:20.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.014+0000 7f0956ffd700 1 -- 172.21.15.3:0/744039013 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0950021030 con 0x7f0960133540 2023-10-03T10:15:20.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.014+0000 7f0956ffd700 1 -- 172.21.15.3:0/744039013 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f09500409f0 con 0x7f0960133540 2023-10-03T10:15:20.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.023+0000 7f0956ffd700 1 -- 172.21.15.3:0/744039013 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f096011c750 con 0x7f0960133540 2023-10-03T10:15:20.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.056+0000 7f771eaba700 1 Processor -- start 2023-10-03T10:15:20.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.056+0000 7f771eaba700 1 -- start start 2023-10-03T10:15:20.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.057+0000 7f771eaba700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f77180a1510 0x7f7718138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:20.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.057+0000 7f771eaba700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7718139250 0x7f771813b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:20.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.057+0000 7f771eaba700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f771809edd0 con 0x7f77180a0a30 2023-10-03T10:15:20.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.057+0000 7f771eaba700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77180a18e0 con 0x7f77180a1510 2023-10-03T10:15:20.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.057+0000 7f771eaba700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771809e1f0 con 0x7f7718139250 2023-10-03T10:15:20.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.057+0000 7f771d057700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7718139250 0x7f771813b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:20.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.057+0000 7f7717fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f77180a1510 0x7f7718138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:20.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.057+0000 7f771c856700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f77180a0a30 0x7f77180a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47748/0 (socket says 172.21.15.3:47748) 2023-10-03T10:15:20.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.057+0000 7f771c856700 1 -- 172.21.15.3:0/1799072215 learned_addr learned my addr 172.21.15.3:0/1799072215 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:20.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.057+0000 7f771d057700 1 -- 172.21.15.3:0/1799072215 >> v2:172.21.15.3:3300/0 conn(0x7f77180a1510 msgr2=0x7f7718138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:20.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.057+0000 7f771d057700 1 --2- 172.21.15.3:0/1799072215 >> v2:172.21.15.3:3300/0 conn(0x7f77180a1510 0x7f7718138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:20.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.057+0000 7f771d057700 1 -- 172.21.15.3:0/1799072215 >> v1:172.21.15.3:6789/0 conn(0x7f77180a0a30 legacy=0x7f77180a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:20.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.057+0000 7f771d057700 1 -- 172.21.15.3:0/1799072215 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f771809dc50 con 0x7f7718139250 2023-10-03T10:15:20.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.057+0000 7f77177fe700 1 -- 172.21.15.3:0/1799072215 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 820526420 0 0) 0x7f771809edd0 con 0x7f77180a0a30 2023-10-03T10:15:20.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.057+0000 7f7717fff700 1 --2- 172.21.15.3:0/1799072215 >> v2:172.21.15.3:3300/0 conn(0x7f77180a1510 0x7f7718138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:20.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.058+0000 7f771d057700 1 --2- 172.21.15.3:0/1799072215 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7718139250 0x7f771813b630 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7710009d90 tx=0x7f771000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d075f7261f95494d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:20.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.058+0000 7f77177fe700 1 -- 172.21.15.3:0/1799072215 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771000e070 con 0x7f7718139250 2023-10-03T10:15:20.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.058+0000 7f77177fe700 1 -- 172.21.15.3:0/1799072215 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f771000bb20 con 0x7f7718139250 2023-10-03T10:15:20.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.058+0000 7f77177fe700 1 -- 172.21.15.3:0/1799072215 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7710012430 con 0x7f7718139250 2023-10-03T10:15:20.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.059+0000 7f771eaba700 1 -- 172.21.15.3:0/1799072215 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7718139250 msgr2=0x7f771813b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:20.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.059+0000 7f771eaba700 1 --2- 172.21.15.3:0/1799072215 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7718139250 0x7f771813b630 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7710009d90 tx=0x7f771000b330 comp rx=0 tx=0).stop 2023-10-03T10:15:20.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.059+0000 7f771eaba700 1 -- 172.21.15.3:0/1799072215 shutdown_connections 2023-10-03T10:15:20.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.059+0000 7f771eaba700 1 --2- 172.21.15.3:0/1799072215 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7718139250 0x7f771813b630 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:20.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.059+0000 7f771eaba700 1 --2- 172.21.15.3:0/1799072215 >> v2:172.21.15.3:3300/0 conn(0x7f77180a1510 0x7f7718138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:20.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.059+0000 7f771eaba700 1 -- 172.21.15.3:0/1799072215 >> 172.21.15.3:0/1799072215 conn(0x7f77180a6550 msgr2=0x7f77180a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:20.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.059+0000 7f771eaba700 1 -- 172.21.15.3:0/1799072215 shutdown_connections 2023-10-03T10:15:20.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.059+0000 7f771eaba700 1 -- 172.21.15.3:0/1799072215 wait complete. 2023-10-03T10:15:20.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.060+0000 7f771eaba700 1 Processor -- start 2023-10-03T10:15:20.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.060+0000 7f771eaba700 1 -- start start 2023-10-03T10:15:20.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.061+0000 7f771eaba700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77180a0a30 0x7f771811e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:20.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.061+0000 7f771eaba700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77180a1510 0x7f771811ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:20.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.061+0000 7f771eaba700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7718139250 0x7f771811efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:20.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.061+0000 7f771eaba700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7718120ed0 con 0x7f77180a1510 2023-10-03T10:15:20.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.061+0000 7f771eaba700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7718120140 con 0x7f7718139250 2023-10-03T10:15:20.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.061+0000 7f771eaba700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7718121fe0 con 0x7f77180a0a30 2023-10-03T10:15:20.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.061+0000 7f771c856700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77180a0a30 0x7f771811e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:20.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.061+0000 7f771c856700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77180a0a30 0x7f771811e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54000/0 (socket says 172.21.15.3:54000) 2023-10-03T10:15:20.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.061+0000 7f771c856700 1 -- 172.21.15.3:0/3063115515 learned_addr learned my addr 172.21.15.3:0/3063115515 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:20.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.062+0000 7f7717fff700 1 --2- 172.21.15.3:0/3063115515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77180a1510 0x7f771811ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:20.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.062+0000 7f771d057700 1 --2- 172.21.15.3:0/3063115515 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7718139250 0x7f771811efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:20.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.062+0000 7f771c856700 1 -- 172.21.15.3:0/3063115515 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7718139250 msgr2=0x7f771811efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:20.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.062+0000 7f771c856700 1 --2- 172.21.15.3:0/3063115515 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7718139250 0x7f771811efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:20.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.062+0000 7f771c856700 1 -- 172.21.15.3:0/3063115515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77180a1510 msgr2=0x7f771811ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:20.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.062+0000 7f771c856700 1 --2- 172.21.15.3:0/3063115515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77180a1510 0x7f771811ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:20.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.062+0000 7f771c856700 1 -- 172.21.15.3:0/3063115515 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77040094a0 con 0x7f77180a0a30 2023-10-03T10:15:20.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.062+0000 7f771c856700 1 --2- 172.21.15.3:0/3063115515 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77180a0a30 0x7f771811e530 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f770c00c550 tx=0x7f770c00c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:20.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.062+0000 7f77157fa700 1 -- 172.21.15.3:0/3063115515 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c004c80 con 0x7f77180a0a30 2023-10-03T10:15:20.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.063+0000 7f77157fa700 1 -- 172.21.15.3:0/3063115515 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770c004e00 con 0x7f77180a0a30 2023-10-03T10:15:20.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.063+0000 7f77157fa700 1 -- 172.21.15.3:0/3063115515 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c007310 con 0x7f77180a0a30 2023-10-03T10:15:20.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.063+0000 7f771eaba700 1 -- 172.21.15.3:0/3063115515 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7710009a70 con 0x7f77180a0a30 2023-10-03T10:15:20.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.063+0000 7f771eaba700 1 -- 172.21.15.3:0/3063115515 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77181213f0 con 0x7f77180a0a30 2023-10-03T10:15:20.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.064+0000 7f77157fa700 1 -- 172.21.15.3:0/3063115515 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f770c007470 con 0x7f77180a0a30 2023-10-03T10:15:20.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.064+0000 7f771eaba700 1 -- 172.21.15.3:0/3063115515 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7718074ea0 con 0x7f77180a0a30 2023-10-03T10:15:20.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.064+0000 7f77157fa700 1 -- 172.21.15.3:0/3063115515 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f770c03f2d0 con 0x7f77180a0a30 2023-10-03T10:15:20.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.074+0000 7f77157fa700 1 -- 172.21.15.3:0/3063115515 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f770c0102c0 con 0x7f77180a0a30 2023-10-03T10:15:20.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.124+0000 7f48d8e0e700 1 -- 172.21.15.3:0/4042430028 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f48d4122fd0 con 0x7f48d409e390 2023-10-03T10:15:20.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.124+0000 7f48c2ffd700 1 -- 172.21.15.3:0/4042430028 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f48d4122fd0 con 0x7f48d409e390 2023-10-03T10:15:20.125 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:20.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.126+0000 7f48d8e0e700 1 -- 172.21.15.3:0/4042430028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48d409e390 msgr2=0x7f48d411f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:20.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.126+0000 7f48d8e0e700 1 --2- 172.21.15.3:0/4042430028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48d409e390 0x7f48d411f930 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f48c4000c00 tx=0x7f48c4004750 comp rx=0 tx=0).stop 2023-10-03T10:15:20.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.126+0000 7f48d8e0e700 1 -- 172.21.15.3:0/4042430028 shutdown_connections 2023-10-03T10:15:20.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.127+0000 7f48d8e0e700 1 --2- 172.21.15.3:0/4042430028 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48d41341d0 0x7f48d41203b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:20.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.127+0000 7f48d8e0e700 1 --2- 172.21.15.3:0/4042430028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48d409e390 0x7f48d411f930 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:20.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.127+0000 7f48d8e0e700 1 --2- 172.21.15.3:0/4042430028 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48d41338c0 0x7f48d411fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:20.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.127+0000 7f48d8e0e700 1 -- 172.21.15.3:0/4042430028 >> 172.21.15.3:0/4042430028 conn(0x7f48d40a2aa0 msgr2=0x7f48d40a0620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:20.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.127+0000 7f48d8e0e700 1 -- 172.21.15.3:0/4042430028 shutdown_connections 2023-10-03T10:15:20.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.127+0000 7f48d8e0e700 1 -- 172.21.15.3:0/4042430028 wait complete. 2023-10-03T10:15:20.141 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:20.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.182+0000 7ff6084d4700 1 -- 172.21.15.3:0/1843928748 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff5e8005150 con 0x7ff60009d6f0 2023-10-03T10:15:20.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.182+0000 7ff5f6ffd700 1 -- 172.21.15.3:0/1843928748 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff5fc024350 con 0x7ff60009d6f0 2023-10-03T10:15:20.183 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:20.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.185+0000 7ff6084d4700 1 -- 172.21.15.3:0/1843928748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff60009d6f0 msgr2=0x7ff60011dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:20.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.185+0000 7ff6084d4700 1 --2- 172.21.15.3:0/1843928748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff60009d6f0 0x7ff60011dff0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00e930 tx=0x7ff5fc00ec40 comp rx=0 tx=0).stop 2023-10-03T10:15:20.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.185+0000 7ff6084d4700 1 -- 172.21.15.3:0/1843928748 shutdown_connections 2023-10-03T10:15:20.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.185+0000 7ff6084d4700 1 --2- 172.21.15.3:0/1843928748 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff60009e410 0x7ff60011ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:20.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.185+0000 7ff6084d4700 1 --2- 172.21.15.3:0/1843928748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff60009d6f0 0x7ff60011dff0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:20.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.185+0000 7ff6084d4700 1 --2- 172.21.15.3:0/1843928748 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff60009e000 0x7ff60011e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:20.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.185+0000 7ff6084d4700 1 -- 172.21.15.3:0/1843928748 >> 172.21.15.3:0/1843928748 conn(0x7ff600066f50 msgr2=0x7ff6001309b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:20.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.185+0000 7ff6084d4700 1 -- 172.21.15.3:0/1843928748 shutdown_connections 2023-10-03T10:15:20.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.185+0000 7ff6084d4700 1 -- 172.21.15.3:0/1843928748 wait complete. 2023-10-03T10:15:20.199 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:20.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.268+0000 7f096831c700 1 -- 172.21.15.3:0/744039013 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f096011c750 con 0x7f0960133540 2023-10-03T10:15:20.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.269+0000 7f0956ffd700 1 -- 172.21.15.3:0/744039013 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f096011c750 con 0x7f0960133540 2023-10-03T10:15:20.269 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:20.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.271+0000 7f096831c700 1 -- 172.21.15.3:0/744039013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0960133540 msgr2=0x7f096012d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:20.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.271+0000 7f096831c700 1 --2- 172.21.15.3:0/744039013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0960133540 0x7f096012d080 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f0950013000 tx=0x7f0950004750 comp rx=0 tx=0).stop 2023-10-03T10:15:20.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.271+0000 7f096831c700 1 -- 172.21.15.3:0/744039013 shutdown_connections 2023-10-03T10:15:20.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.271+0000 7f096831c700 1 --2- 172.21.15.3:0/744039013 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0960135e60 0x7f09601c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:20.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.271+0000 7f096831c700 1 --2- 172.21.15.3:0/744039013 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0960130540 0x7f096012cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:20.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.271+0000 7f096831c700 1 --2- 172.21.15.3:0/744039013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0960133540 0x7f096012d080 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:20.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.271+0000 7f096831c700 1 -- 172.21.15.3:0/744039013 >> 172.21.15.3:0/744039013 conn(0x7f09600a2aa0 msgr2=0x7f0960131300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:20.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.271+0000 7f096831c700 1 -- 172.21.15.3:0/744039013 shutdown_connections 2023-10-03T10:15:20.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.271+0000 7f096831c700 1 -- 172.21.15.3:0/744039013 wait complete. 2023-10-03T10:15:20.285 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:20.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.310+0000 7f771eaba700 1 -- 172.21.15.3:0/3063115515 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7718121390 con 0x7f77180a0a30 2023-10-03T10:15:20.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.310+0000 7f77157fa700 1 -- 172.21.15.3:0/3063115515 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f770c011360 con 0x7f77180a0a30 2023-10-03T10:15:20.311 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:20.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.313+0000 7f771eaba700 1 -- 172.21.15.3:0/3063115515 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77180a0a30 msgr2=0x7f771811e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:20.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.313+0000 7f771eaba700 1 --2- 172.21.15.3:0/3063115515 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77180a0a30 0x7f771811e530 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f770c00c550 tx=0x7f770c00c860 comp rx=0 tx=0).stop 2023-10-03T10:15:20.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.313+0000 7f771eaba700 1 -- 172.21.15.3:0/3063115515 shutdown_connections 2023-10-03T10:15:20.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.313+0000 7f771eaba700 1 --2- 172.21.15.3:0/3063115515 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7718139250 0x7f771811efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:20.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.313+0000 7f771eaba700 1 --2- 172.21.15.3:0/3063115515 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77180a0a30 0x7f771811e530 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:20.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.313+0000 7f771eaba700 1 --2- 172.21.15.3:0/3063115515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77180a1510 0x7f771811ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:20.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.313+0000 7f771eaba700 1 -- 172.21.15.3:0/3063115515 >> 172.21.15.3:0/3063115515 conn(0x7f77180a6550 msgr2=0x7f77180a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:20.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.313+0000 7f771eaba700 1 -- 172.21.15.3:0/3063115515 shutdown_connections 2023-10-03T10:15:20.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:20.313+0000 7f771eaba700 1 -- 172.21.15.3:0/3063115515 wait complete. 2023-10-03T10:15:20.327 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:21.143 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:21.200 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:21.286 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:21.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.294+0000 7f4a0b89d700 1 Processor -- start 2023-10-03T10:15:21.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.294+0000 7f4a0b89d700 1 -- start start 2023-10-03T10:15:21.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.295+0000 7f4a0b89d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4a0409e930 0x7f4a0409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:21.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.295+0000 7f4a0b89d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a0409f240 0x7f4a041317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:21.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.295+0000 7f4a0b89d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a0409f650 0x7f4a04131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:21.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.295+0000 7f4a0b89d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a04070940 con 0x7f4a0409e930 2023-10-03T10:15:21.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.295+0000 7f4a0b89d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0406fd60 con 0x7f4a0409f240 2023-10-03T10:15:21.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.295+0000 7f4a0b89d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0406fa80 con 0x7f4a0409f650 2023-10-03T10:15:21.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.295+0000 7f4a08e38700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a0409f240 0x7f4a041317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:21.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.295+0000 7f4a09e3a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a0409f650 0x7f4a04131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:21.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.295+0000 7f4a08e38700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a0409f240 0x7f4a041317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50682/0 (socket says 172.21.15.3:50682) 2023-10-03T10:15:21.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.295+0000 7f4a08e38700 1 -- 172.21.15.3:0/1976632789 learned_addr learned my addr 172.21.15.3:0/1976632789 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:21.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.295+0000 7f4a08e38700 1 -- 172.21.15.3:0/1976632789 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a0409f650 msgr2=0x7f4a04131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:21.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.295+0000 7f4a08e38700 1 --2- 172.21.15.3:0/1976632789 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a0409f650 0x7f4a04131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.295+0000 7f4a08e38700 1 -- 172.21.15.3:0/1976632789 >> v2:172.21.15.3:3300/0 conn(0x7f4a0409e930 msgr2=0x7f4a0409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:21.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.295+0000 7f4a08e38700 1 --2- 172.21.15.3:0/1976632789 >> v2:172.21.15.3:3300/0 conn(0x7f4a0409e930 0x7f4a0409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.295+0000 7f4a08e38700 1 -- 172.21.15.3:0/1976632789 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a0409fb60 con 0x7f4a0409f240 2023-10-03T10:15:21.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.295+0000 7f4a09e3a700 1 --2- 172.21.15.3:0/1976632789 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a0409f650 0x7f4a04131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:21.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.296+0000 7f4a08e38700 1 --2- 172.21.15.3:0/1976632789 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a0409f240 0x7f4a041317f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f49f40098f0 tx=0x7f49f4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92d885503ed11cf1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:21.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.296+0000 7f49fbfff700 1 -- 172.21.15.3:0/1976632789 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f4011830 con 0x7f4a0409f240 2023-10-03T10:15:21.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.296+0000 7f49fbfff700 1 -- 172.21.15.3:0/1976632789 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f4011990 con 0x7f4a0409f240 2023-10-03T10:15:21.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.296+0000 7f49fbfff700 1 -- 172.21.15.3:0/1976632789 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f4011d20 con 0x7f4a0409f240 2023-10-03T10:15:21.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.297+0000 7f4a0b89d700 1 -- 172.21.15.3:0/1976632789 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a0409f240 msgr2=0x7f4a041317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:21.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.297+0000 7f4a0b89d700 1 --2- 172.21.15.3:0/1976632789 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a0409f240 0x7f4a041317f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f49f40098f0 tx=0x7f49f4009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:21.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.297+0000 7f4a0b89d700 1 -- 172.21.15.3:0/1976632789 shutdown_connections 2023-10-03T10:15:21.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.297+0000 7f4a0b89d700 1 --2- 172.21.15.3:0/1976632789 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a0409f650 0x7f4a04131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.297+0000 7f4a0b89d700 1 --2- 172.21.15.3:0/1976632789 >> v2:172.21.15.3:3300/0 conn(0x7f4a0409e930 0x7f4a0409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.297+0000 7f4a0b89d700 1 --2- 172.21.15.3:0/1976632789 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a0409f240 0x7f4a041317f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.297+0000 7f4a0b89d700 1 -- 172.21.15.3:0/1976632789 >> 172.21.15.3:0/1976632789 conn(0x7f4a040a4210 msgr2=0x7f4a040a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:21.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.297+0000 7f4a0b89d700 1 -- 172.21.15.3:0/1976632789 shutdown_connections 2023-10-03T10:15:21.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.298+0000 7f4a0b89d700 1 -- 172.21.15.3:0/1976632789 wait complete. 2023-10-03T10:15:21.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.299+0000 7f4a0b89d700 1 Processor -- start 2023-10-03T10:15:21.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.299+0000 7f4a0b89d700 1 -- start start 2023-10-03T10:15:21.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.299+0000 7f4a0b89d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a0409e930 0x7f4a041c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:21.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.299+0000 7f4a0b89d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a0409f240 0x7f4a041c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:21.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.299+0000 7f4a0b89d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a0409f650 0x7f4a041c1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:21.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.299+0000 7f4a0b89d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a04131d80 con 0x7f4a0409e930 2023-10-03T10:15:21.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.299+0000 7f4a0b89d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a040a3630 con 0x7f4a0409f650 2023-10-03T10:15:21.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.299+0000 7f4a0b89d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a04071d20 con 0x7f4a0409f240 2023-10-03T10:15:21.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.299+0000 7f4a08e38700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a0409f240 0x7f4a041c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:21.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.299+0000 7f4a08e38700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a0409f240 0x7f4a041c8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54008/0 (socket says 172.21.15.3:54008) 2023-10-03T10:15:21.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.299+0000 7f4a08e38700 1 -- 172.21.15.3:0/3531473573 learned_addr learned my addr 172.21.15.3:0/3531473573 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:21.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.300+0000 7f4a08e38700 1 -- 172.21.15.3:0/3531473573 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a0409f650 msgr2=0x7f4a041c1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:21.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.300+0000 7f4a08e38700 1 --2- 172.21.15.3:0/3531473573 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a0409f650 0x7f4a041c1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.300+0000 7f4a08e38700 1 -- 172.21.15.3:0/3531473573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a0409e930 msgr2=0x7f4a041c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:21.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.300+0000 7f4a08e38700 1 --2- 172.21.15.3:0/3531473573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a0409e930 0x7f4a041c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.300+0000 7f4a08e38700 1 -- 172.21.15.3:0/3531473573 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49fc009a70 con 0x7f4a0409f240 2023-10-03T10:15:21.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.300+0000 7f4a08e38700 1 --2- 172.21.15.3:0/3531473573 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a0409f240 0x7f4a041c8d60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f49f4000c00 tx=0x7f49f4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:21.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.300+0000 7f49f9ffb700 1 -- 172.21.15.3:0/3531473573 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f4004400 con 0x7f4a0409f240 2023-10-03T10:15:21.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.301+0000 7f49f9ffb700 1 -- 172.21.15.3:0/3531473573 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f401f070 con 0x7f4a0409f240 2023-10-03T10:15:21.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.301+0000 7f49f9ffb700 1 -- 172.21.15.3:0/3531473573 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f40038e0 con 0x7f4a0409f240 2023-10-03T10:15:21.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.301+0000 7f4a0b89d700 1 -- 172.21.15.3:0/3531473573 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f4009550 con 0x7f4a0409f240 2023-10-03T10:15:21.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.301+0000 7f4a0b89d700 1 -- 172.21.15.3:0/3531473573 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a040a0190 con 0x7f4a0409f240 2023-10-03T10:15:21.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.303+0000 7f49f9ffb700 1 -- 172.21.15.3:0/3531473573 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f49f4021070 con 0x7f4a0409f240 2023-10-03T10:15:21.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.303+0000 7f49f9ffb700 1 -- 172.21.15.3:0/3531473573 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f49f40408b0 con 0x7f4a0409f240 2023-10-03T10:15:21.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.303+0000 7f4a0b89d700 1 -- 172.21.15.3:0/3531473573 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a04132bc0 con 0x7f4a0409f240 2023-10-03T10:15:21.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.309+0000 7f49f9ffb700 1 -- 172.21.15.3:0/3531473573 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f49f4015030 con 0x7f4a0409f240 2023-10-03T10:15:21.328 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:21.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.349+0000 7f12b8438700 1 Processor -- start 2023-10-03T10:15:21.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.349+0000 7f12b8438700 1 -- start start 2023-10-03T10:15:21.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.350+0000 7f12b8438700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12b0136710 0x7f12b0138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:21.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.350+0000 7f12b8438700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f12b0139030 0x7f12b013b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:21.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.350+0000 7f12b8438700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b006ef30 con 0x7f12b01338a0 2023-10-03T10:15:21.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.350+0000 7f12b8438700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12b006fef0 con 0x7f12b0139030 2023-10-03T10:15:21.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.350+0000 7f12b8438700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b01334b0 con 0x7f12b0136710 2023-10-03T10:15:21.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.350+0000 7f12b61d4700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f12b01338a0 0x7f12b0133c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47762/0 (socket says 172.21.15.3:47762) 2023-10-03T10:15:21.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.350+0000 7f12b61d4700 1 -- 172.21.15.3:0/1282661154 learned_addr learned my addr 172.21.15.3:0/1282661154 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:21.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.350+0000 7f12b59d3700 1 --2- 172.21.15.3:0/1282661154 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12b0136710 0x7f12b0138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:21.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.350+0000 7f12b69d5700 1 --2- 172.21.15.3:0/1282661154 >> v2:172.21.15.3:3300/0 conn(0x7f12b0139030 0x7f12b013b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:21.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.351+0000 7f12b59d3700 1 -- 172.21.15.3:0/1282661154 >> v2:172.21.15.3:3300/0 conn(0x7f12b0139030 msgr2=0x7f12b013b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:21.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.351+0000 7f12b59d3700 1 --2- 172.21.15.3:0/1282661154 >> v2:172.21.15.3:3300/0 conn(0x7f12b0139030 0x7f12b013b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.351+0000 7f12b59d3700 1 -- 172.21.15.3:0/1282661154 >> v1:172.21.15.3:6789/0 conn(0x7f12b01338a0 legacy=0x7f12b0133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:21.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.351+0000 7f12b59d3700 1 -- 172.21.15.3:0/1282661154 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b006fc30 con 0x7f12b0136710 2023-10-03T10:15:21.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.351+0000 7f12b69d5700 1 --2- 172.21.15.3:0/1282661154 >> v2:172.21.15.3:3300/0 conn(0x7f12b0139030 0x7f12b013b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:21.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.351+0000 7f12b59d3700 1 --2- 172.21.15.3:0/1282661154 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12b0136710 0x7f12b0138af0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009840 tx=0x7f12a0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adfa5ed768ea7071 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:21.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.351+0000 7f12b51d2700 1 -- 172.21.15.3:0/1282661154 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a0011830 con 0x7f12b0136710 2023-10-03T10:15:21.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.351+0000 7f12b51d2700 1 -- 172.21.15.3:0/1282661154 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a0011990 con 0x7f12b0136710 2023-10-03T10:15:21.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.351+0000 7f12b51d2700 1 -- 172.21.15.3:0/1282661154 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a0011d20 con 0x7f12b0136710 2023-10-03T10:15:21.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.352+0000 7f12b8438700 1 -- 172.21.15.3:0/1282661154 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12b0136710 msgr2=0x7f12b0138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:21.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.352+0000 7f12b8438700 1 --2- 172.21.15.3:0/1282661154 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12b0136710 0x7f12b0138af0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009840 tx=0x7f12a0009b50 comp rx=0 tx=0).stop 2023-10-03T10:15:21.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.353+0000 7f12b8438700 1 -- 172.21.15.3:0/1282661154 shutdown_connections 2023-10-03T10:15:21.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.353+0000 7f12b8438700 1 --2- 172.21.15.3:0/1282661154 >> v2:172.21.15.3:3300/0 conn(0x7f12b0139030 0x7f12b013b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.353+0000 7f12b8438700 1 --2- 172.21.15.3:0/1282661154 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12b0136710 0x7f12b0138af0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.353+0000 7f12b8438700 1 -- 172.21.15.3:0/1282661154 >> 172.21.15.3:0/1282661154 conn(0x7f12b00a2aa0 msgr2=0x7f12b00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:21.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.353+0000 7f12b8438700 1 -- 172.21.15.3:0/1282661154 shutdown_connections 2023-10-03T10:15:21.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.353+0000 7f12b8438700 1 -- 172.21.15.3:0/1282661154 wait complete. 2023-10-03T10:15:21.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.354+0000 7f12b8438700 1 Processor -- start 2023-10-03T10:15:21.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.354+0000 7f12b8438700 1 -- start start 2023-10-03T10:15:21.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.355+0000 7f12b8438700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12b01338a0 0x7f12b0119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:21.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.355+0000 7f12b8438700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12b0136710 0x7f12b011a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:21.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.355+0000 7f12b8438700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12b0139030 0x7f12b011aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:21.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.355+0000 7f12b8438700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b006f6d0 con 0x7f12b01338a0 2023-10-03T10:15:21.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.355+0000 7f12b8438700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b011bb10 con 0x7f12b0136710 2023-10-03T10:15:21.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.355+0000 7f12b8438700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12b0138e20 con 0x7f12b0139030 2023-10-03T10:15:21.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.355+0000 7f12b59d3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12b0136710 0x7f12b011a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:21.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.355+0000 7f12b59d3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12b0136710 0x7f12b011a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50704/0 (socket says 172.21.15.3:50704) 2023-10-03T10:15:21.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.355+0000 7f12b59d3700 1 -- 172.21.15.3:0/4202085682 learned_addr learned my addr 172.21.15.3:0/4202085682 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:21.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.355+0000 7f12b61d4700 1 --2- 172.21.15.3:0/4202085682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12b01338a0 0x7f12b0119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:21.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.355+0000 7f12b69d5700 1 --2- 172.21.15.3:0/4202085682 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12b0139030 0x7f12b011aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:21.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.356+0000 7f12b69d5700 1 -- 172.21.15.3:0/4202085682 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12b0136710 msgr2=0x7f12b011a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:21.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.356+0000 7f12b69d5700 1 --2- 172.21.15.3:0/4202085682 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12b0136710 0x7f12b011a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.356+0000 7f12b69d5700 1 -- 172.21.15.3:0/4202085682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12b01338a0 msgr2=0x7f12b0119f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:21.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.356+0000 7f12b69d5700 1 --2- 172.21.15.3:0/4202085682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12b01338a0 0x7f12b0119f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.356+0000 7f12b69d5700 1 -- 172.21.15.3:0/4202085682 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a8009a70 con 0x7f12b0139030 2023-10-03T10:15:21.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.356+0000 7f12b61d4700 1 --2- 172.21.15.3:0/4202085682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12b01338a0 0x7f12b0119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:21.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.356+0000 7f12b59d3700 1 --2- 172.21.15.3:0/4202085682 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12b0136710 0x7f12b011a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:21.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.356+0000 7f12b69d5700 1 --2- 172.21.15.3:0/4202085682 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12b0139030 0x7f12b011aa00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f12a800efc0 tx=0x7f12a800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:21.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.356+0000 7f12a6ffd700 1 -- 172.21.15.3:0/4202085682 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a800ca90 con 0x7f12b0139030 2023-10-03T10:15:21.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.356+0000 7f12a6ffd700 1 -- 172.21.15.3:0/4202085682 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a800f040 con 0x7f12b0139030 2023-10-03T10:15:21.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.356+0000 7f12a6ffd700 1 -- 172.21.15.3:0/4202085682 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a800ce30 con 0x7f12b0139030 2023-10-03T10:15:21.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.357+0000 7f12b8438700 1 -- 172.21.15.3:0/4202085682 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a00094a0 con 0x7f12b0139030 2023-10-03T10:15:21.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.357+0000 7f12b8438700 1 -- 172.21.15.3:0/4202085682 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b011c180 con 0x7f12b0139030 2023-10-03T10:15:21.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.357+0000 7f12a6ffd700 1 -- 172.21.15.3:0/4202085682 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f12a8014480 con 0x7f12b0139030 2023-10-03T10:15:21.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.358+0000 7f12a6ffd700 1 -- 172.21.15.3:0/4202085682 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f12a803fca0 con 0x7f12b0139030 2023-10-03T10:15:21.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.358+0000 7f12b8438700 1 -- 172.21.15.3:0/4202085682 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b0074ea0 con 0x7f12b0139030 2023-10-03T10:15:21.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.364+0000 7f12a6ffd700 1 -- 172.21.15.3:0/4202085682 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f12a8012910 con 0x7f12b0139030 2023-10-03T10:15:21.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.437+0000 7fb6bcf2c700 1 Processor -- start 2023-10-03T10:15:21.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.437+0000 7fb6bcf2c700 1 -- start start 2023-10-03T10:15:21.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.437+0000 7fb6bcf2c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb6b809d6f0 0x7fb6b809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:21.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.437+0000 7fb6bcf2c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb6b809e000 0x7fb6b81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:21.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.437+0000 7fb6bcf2c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6b809b120 con 0x7fb6b809e410 2023-10-03T10:15:21.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.437+0000 7fb6bcf2c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b806b460 con 0x7fb6b809d6f0 2023-10-03T10:15:21.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.437+0000 7fb6bcf2c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b809a7e0 con 0x7fb6b809e000 2023-10-03T10:15:21.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.438+0000 7fb6b659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb6b809d6f0 0x7fb6b809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:21.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.438+0000 7fb6b5d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb6b809e000 0x7fb6b81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:21.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.438+0000 7fb6b659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb6b809d6f0 0x7fb6b809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50712/0 (socket says 172.21.15.3:50712) 2023-10-03T10:15:21.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.438+0000 7fb6b5d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb6b809e000 0x7fb6b81339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54016/0 (socket says 172.21.15.3:54016) 2023-10-03T10:15:21.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.438+0000 7fb6b659c700 1 -- 172.21.15.3:0/3165220918 learned_addr learned my addr 172.21.15.3:0/3165220918 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:21.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.438+0000 7fb6b659c700 1 -- 172.21.15.3:0/3165220918 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb6b809e000 msgr2=0x7fb6b81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:21.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.438+0000 7fb6b659c700 1 --2- 172.21.15.3:0/3165220918 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb6b809e000 0x7fb6b81339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.438+0000 7fb6b659c700 1 -- 172.21.15.3:0/3165220918 >> v1:172.21.15.3:6789/0 conn(0x7fb6b809e410 legacy=0x7fb6b8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:21.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.438+0000 7fb6b559a700 1 -- 172.21.15.3:0/3165220918 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2036456768 0 0) 0x7fb6b809b120 con 0x7fb6b809e410 2023-10-03T10:15:21.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.438+0000 7fb6b659c700 1 -- 172.21.15.3:0/3165220918 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b809a500 con 0x7fb6b809d6f0 2023-10-03T10:15:21.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.438+0000 7fb6b5d9b700 1 --2- 172.21.15.3:0/3165220918 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb6b809e000 0x7fb6b81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:21.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.439+0000 7fb6b659c700 1 --2- 172.21.15.3:0/3165220918 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb6b809d6f0 0x7fb6b809dac0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009920 tx=0x7fb6a8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cfe9974aa20a0cf server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:21.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.439+0000 7fb6b559a700 1 -- 172.21.15.3:0/3165220918 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a8011830 con 0x7fb6b809d6f0 2023-10-03T10:15:21.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.439+0000 7fb6b559a700 1 -- 172.21.15.3:0/3165220918 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a8011990 con 0x7fb6b809d6f0 2023-10-03T10:15:21.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.439+0000 7fb6b559a700 1 -- 172.21.15.3:0/3165220918 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a8011d20 con 0x7fb6b809d6f0 2023-10-03T10:15:21.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.439+0000 7fb6bcf2c700 1 -- 172.21.15.3:0/3165220918 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb6b809d6f0 msgr2=0x7fb6b809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:21.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.439+0000 7fb6bcf2c700 1 --2- 172.21.15.3:0/3165220918 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb6b809d6f0 0x7fb6b809dac0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009920 tx=0x7fb6a8009c30 comp rx=0 tx=0).stop 2023-10-03T10:15:21.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.440+0000 7fb6bcf2c700 1 -- 172.21.15.3:0/3165220918 shutdown_connections 2023-10-03T10:15:21.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.440+0000 7fb6bcf2c700 1 --2- 172.21.15.3:0/3165220918 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb6b809d6f0 0x7fb6b809dac0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.440+0000 7fb6bcf2c700 1 --2- 172.21.15.3:0/3165220918 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb6b809e000 0x7fb6b81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.440+0000 7fb6bcf2c700 1 -- 172.21.15.3:0/3165220918 >> 172.21.15.3:0/3165220918 conn(0x7fb6b8066f50 msgr2=0x7fb6b809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:21.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.440+0000 7fb6bcf2c700 1 -- 172.21.15.3:0/3165220918 shutdown_connections 2023-10-03T10:15:21.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.440+0000 7fb6bcf2c700 1 -- 172.21.15.3:0/3165220918 wait complete. 2023-10-03T10:15:21.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.441+0000 7fb6bcf2c700 1 Processor -- start 2023-10-03T10:15:21.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.442+0000 7fb6bcf2c700 1 -- start start 2023-10-03T10:15:21.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.442+0000 7fb6bcf2c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6b809d6f0 0x7fb6b8122360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:21.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.442+0000 7fb6bcf2c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb6b809e000 0x7fb6b81228a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:21.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.442+0000 7fb6bcf2c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb6b809e410 0x7fb6b8122de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:21.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.442+0000 7fb6bcf2c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b8124e30 con 0x7fb6b809d6f0 2023-10-03T10:15:21.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.442+0000 7fb6bcf2c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b81336f0 con 0x7fb6b809e410 2023-10-03T10:15:21.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.442+0000 7fb6bcf2c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b809c710 con 0x7fb6b809e000 2023-10-03T10:15:21.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.442+0000 7fb6b5d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb6b809e000 0x7fb6b81228a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:21.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.442+0000 7fb6b5d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb6b809e000 0x7fb6b81228a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54026/0 (socket says 172.21.15.3:54026) 2023-10-03T10:15:21.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.442+0000 7fb6b5d9b700 1 -- 172.21.15.3:0/2737207559 learned_addr learned my addr 172.21.15.3:0/2737207559 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:21.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.442+0000 7fb6b6d9d700 1 --2- 172.21.15.3:0/2737207559 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb6b809e410 0x7fb6b8122de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:21.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.442+0000 7fb6b659c700 1 --2- 172.21.15.3:0/2737207559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6b809d6f0 0x7fb6b8122360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:21.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.443+0000 7fb6b5d9b700 1 -- 172.21.15.3:0/2737207559 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb6b809e410 msgr2=0x7fb6b8122de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:21.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.443+0000 7fb6b5d9b700 1 --2- 172.21.15.3:0/2737207559 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb6b809e410 0x7fb6b8122de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.443+0000 7fb6b5d9b700 1 -- 172.21.15.3:0/2737207559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6b809d6f0 msgr2=0x7fb6b8122360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:21.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.443+0000 7fb6b5d9b700 1 --2- 172.21.15.3:0/2737207559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6b809d6f0 0x7fb6b8122360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.443+0000 7fb6b5d9b700 1 -- 172.21.15.3:0/2737207559 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ac0094a0 con 0x7fb6b809e000 2023-10-03T10:15:21.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.443+0000 7fb6b5d9b700 1 --2- 172.21.15.3:0/2737207559 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb6b809e000 0x7fb6b81228a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac00ebe0 tx=0x7fb6ac00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:21.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.443+0000 7fb6a6ffd700 1 -- 172.21.15.3:0/2737207559 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac00c850 con 0x7fb6b809e000 2023-10-03T10:15:21.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.443+0000 7fb6a6ffd700 1 -- 172.21.15.3:0/2737207559 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6ac00c9b0 con 0x7fb6b809e000 2023-10-03T10:15:21.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.443+0000 7fb6bcf2c700 1 -- 172.21.15.3:0/2737207559 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6a8009550 con 0x7fb6b809e000 2023-10-03T10:15:21.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.443+0000 7fb6bcf2c700 1 -- 172.21.15.3:0/2737207559 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b80a19c0 con 0x7fb6b809e000 2023-10-03T10:15:21.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.443+0000 7fb6a6ffd700 1 -- 172.21.15.3:0/2737207559 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac00cd20 con 0x7fb6b809e000 2023-10-03T10:15:21.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.445+0000 7fb6bcf2c700 1 -- 172.21.15.3:0/2737207559 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb69c0052e0 con 0x7fb6b809e000 2023-10-03T10:15:21.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.445+0000 7fb6a6ffd700 1 -- 172.21.15.3:0/2737207559 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb6ac013470 con 0x7fb6b809e000 2023-10-03T10:15:21.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.445+0000 7fb6a6ffd700 1 -- 172.21.15.3:0/2737207559 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb6ac03ff50 con 0x7fb6b809e000 2023-10-03T10:15:21.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.454+0000 7fb6a6ffd700 1 -- 172.21.15.3:0/2737207559 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb6ac013720 con 0x7fb6b809e000 2023-10-03T10:15:21.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.479+0000 7ff970703700 1 Processor -- start 2023-10-03T10:15:21.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.479+0000 7ff970703700 1 -- start start 2023-10-03T10:15:21.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.480+0000 7ff970703700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9680a0a30 0x7ff9680a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:21.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.480+0000 7ff970703700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff968138d80 0x7ff96813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:21.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.480+0000 7ff970703700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9680a2a70 con 0x7ff96809d630 2023-10-03T10:15:21.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.480+0000 7ff970703700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9680a01d0 con 0x7ff968138d80 2023-10-03T10:15:21.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.480+0000 7ff970703700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff96809e2b0 con 0x7ff9680a0a30 2023-10-03T10:15:21.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.480+0000 7ff96e49f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9680a0a30 0x7ff9680a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:21.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.480+0000 7ff96e49f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9680a0a30 0x7ff9680a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54030/0 (socket says 172.21.15.3:54030) 2023-10-03T10:15:21.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.480+0000 7ff96e49f700 1 -- 172.21.15.3:0/551643753 learned_addr learned my addr 172.21.15.3:0/551643753 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:21.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.480+0000 7ff96eca0700 1 --2- 172.21.15.3:0/551643753 >> v2:172.21.15.3:3300/0 conn(0x7ff968138d80 0x7ff96813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:21.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.480+0000 7ff96e49f700 1 -- 172.21.15.3:0/551643753 >> v2:172.21.15.3:3300/0 conn(0x7ff968138d80 msgr2=0x7ff96813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:21.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.481+0000 7ff96e49f700 1 --2- 172.21.15.3:0/551643753 >> v2:172.21.15.3:3300/0 conn(0x7ff968138d80 0x7ff96813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.481+0000 7ff96e49f700 1 -- 172.21.15.3:0/551643753 >> v1:172.21.15.3:6789/0 conn(0x7ff96809d630 legacy=0x7ff9680a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:21.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.481+0000 7ff96e49f700 1 -- 172.21.15.3:0/551643753 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff96809dc50 con 0x7ff9680a0a30 2023-10-03T10:15:21.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.481+0000 7ff96e49f700 1 --2- 172.21.15.3:0/551643753 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9680a0a30 0x7ff9680a0e00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff9640098f0 tx=0x7ff964009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd36240ea61d8cbc server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:21.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.481+0000 7ff96d49d700 1 -- 172.21.15.3:0/551643753 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff964011830 con 0x7ff9680a0a30 2023-10-03T10:15:21.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.481+0000 7ff96d49d700 1 -- 172.21.15.3:0/551643753 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff964011990 con 0x7ff9680a0a30 2023-10-03T10:15:21.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.481+0000 7ff96d49d700 1 -- 172.21.15.3:0/551643753 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff964011d20 con 0x7ff9680a0a30 2023-10-03T10:15:21.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.482+0000 7ff970703700 1 -- 172.21.15.3:0/551643753 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9680a0a30 msgr2=0x7ff9680a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:21.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.482+0000 7ff970703700 1 --2- 172.21.15.3:0/551643753 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9680a0a30 0x7ff9680a0e00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff9640098f0 tx=0x7ff964009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:21.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.482+0000 7ff970703700 1 -- 172.21.15.3:0/551643753 shutdown_connections 2023-10-03T10:15:21.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.482+0000 7ff970703700 1 --2- 172.21.15.3:0/551643753 >> v2:172.21.15.3:3300/0 conn(0x7ff968138d80 0x7ff96813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.482+0000 7ff970703700 1 --2- 172.21.15.3:0/551643753 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9680a0a30 0x7ff9680a0e00 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.482+0000 7ff970703700 1 -- 172.21.15.3:0/551643753 >> 172.21.15.3:0/551643753 conn(0x7ff9680a3050 msgr2=0x7ff9680a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:21.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.482+0000 7ff970703700 1 -- 172.21.15.3:0/551643753 shutdown_connections 2023-10-03T10:15:21.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.482+0000 7ff970703700 1 -- 172.21.15.3:0/551643753 wait complete. 2023-10-03T10:15:21.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.484+0000 7ff970703700 1 Processor -- start 2023-10-03T10:15:21.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.484+0000 7ff970703700 1 -- start start 2023-10-03T10:15:21.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.484+0000 7ff970703700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff96809d630 0x7ff9681208f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:21.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.484+0000 7ff970703700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9680a0a30 0x7ff968120e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:21.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.484+0000 7ff970703700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff968138d80 0x7ff968121370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:21.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.484+0000 7ff970703700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff968123700 con 0x7ff96809d630 2023-10-03T10:15:21.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.484+0000 7ff970703700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9680a0410 con 0x7ff968138d80 2023-10-03T10:15:21.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.484+0000 7ff970703700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff9681224b0 con 0x7ff9680a0a30 2023-10-03T10:15:21.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.485+0000 7ff96e49f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff96809d630 0x7ff9681208f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:21.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.485+0000 7ff96e49f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff96809d630 0x7ff9681208f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42642/0 (socket says 172.21.15.3:42642) 2023-10-03T10:15:21.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.485+0000 7ff96e49f700 1 -- 172.21.15.3:0/2601100247 learned_addr learned my addr 172.21.15.3:0/2601100247 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:21.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.485+0000 7ff96dc9e700 1 --2- 172.21.15.3:0/2601100247 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9680a0a30 0x7ff968120e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:21.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.485+0000 7ff96eca0700 1 --2- 172.21.15.3:0/2601100247 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff968138d80 0x7ff968121370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:21.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.485+0000 7ff96e49f700 1 -- 172.21.15.3:0/2601100247 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9680a0a30 msgr2=0x7ff968120e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:21.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.485+0000 7ff96e49f700 1 --2- 172.21.15.3:0/2601100247 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9680a0a30 0x7ff968120e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.485+0000 7ff96e49f700 1 -- 172.21.15.3:0/2601100247 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff968138d80 msgr2=0x7ff968121370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:21.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.485+0000 7ff96e49f700 1 --2- 172.21.15.3:0/2601100247 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff968138d80 0x7ff968121370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.485+0000 7ff96e49f700 1 -- 172.21.15.3:0/2601100247 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff964009550 con 0x7ff96809d630 2023-10-03T10:15:21.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.485+0000 7ff96e49f700 1 --2- 172.21.15.3:0/2601100247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff96809d630 0x7ff9681208f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ff964009c40 tx=0x7ff964004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:21.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.486+0000 7ff95effd700 1 -- 172.21.15.3:0/2601100247 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff964004400 con 0x7ff96809d630 2023-10-03T10:15:21.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.486+0000 7ff95effd700 1 -- 172.21.15.3:0/2601100247 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96401f070 con 0x7ff96809d630 2023-10-03T10:15:21.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.486+0000 7ff970703700 1 -- 172.21.15.3:0/2601100247 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9681240e0 con 0x7ff96809d630 2023-10-03T10:15:21.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.486+0000 7ff95effd700 1 -- 172.21.15.3:0/2601100247 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9640038e0 con 0x7ff96809d630 2023-10-03T10:15:21.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.486+0000 7ff970703700 1 -- 172.21.15.3:0/2601100247 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff968124b50 con 0x7ff96809d630 2023-10-03T10:15:21.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.487+0000 7ff95effd700 1 -- 172.21.15.3:0/2601100247 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff964021030 con 0x7ff96809d630 2023-10-03T10:15:21.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.487+0000 7ff95effd700 1 -- 172.21.15.3:0/2601100247 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff964040940 con 0x7ff96809d630 2023-10-03T10:15:21.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.487+0000 7ff970703700 1 -- 172.21.15.3:0/2601100247 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9680747f0 con 0x7ff96809d630 2023-10-03T10:15:21.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.496+0000 7ff95effd700 1 -- 172.21.15.3:0/2601100247 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff964015080 con 0x7ff96809d630 2023-10-03T10:15:21.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.543+0000 7f4a0b89d700 1 -- 172.21.15.3:0/3531473573 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4a04074ea0 con 0x7f4a0409f240 2023-10-03T10:15:21.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.543+0000 7f49f9ffb700 1 -- 172.21.15.3:0/3531473573 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f49f4004560 con 0x7f4a0409f240 2023-10-03T10:15:21.543 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:21.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.547+0000 7f4a0b89d700 1 -- 172.21.15.3:0/3531473573 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a0409f240 msgr2=0x7f4a041c8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:21.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.547+0000 7f4a0b89d700 1 --2- 172.21.15.3:0/3531473573 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a0409f240 0x7f4a041c8d60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f49f4000c00 tx=0x7f49f4004750 comp rx=0 tx=0).stop 2023-10-03T10:15:21.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.547+0000 7f4a0b89d700 1 -- 172.21.15.3:0/3531473573 shutdown_connections 2023-10-03T10:15:21.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.547+0000 7f4a0b89d700 1 --2- 172.21.15.3:0/3531473573 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a0409f650 0x7f4a041c1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.547+0000 7f4a0b89d700 1 --2- 172.21.15.3:0/3531473573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a0409e930 0x7f4a041c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.547+0000 7f4a0b89d700 1 --2- 172.21.15.3:0/3531473573 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a0409f240 0x7f4a041c8d60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.547+0000 7f4a0b89d700 1 -- 172.21.15.3:0/3531473573 >> 172.21.15.3:0/3531473573 conn(0x7f4a040a4210 msgr2=0x7f4a0406c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:21.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.547+0000 7f4a0b89d700 1 -- 172.21.15.3:0/3531473573 shutdown_connections 2023-10-03T10:15:21.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.547+0000 7f4a0b89d700 1 -- 172.21.15.3:0/3531473573 wait complete. 2023-10-03T10:15:21.567 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:21.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.602+0000 7f12b8438700 1 -- 172.21.15.3:0/4202085682 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f12b011d260 con 0x7f12b0139030 2023-10-03T10:15:21.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.603+0000 7f12a6ffd700 1 -- 172.21.15.3:0/4202085682 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f12a8012340 con 0x7f12b0139030 2023-10-03T10:15:21.603 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:21.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.605+0000 7f12b8438700 1 -- 172.21.15.3:0/4202085682 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12b0139030 msgr2=0x7f12b011aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:21.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.605+0000 7f12b8438700 1 --2- 172.21.15.3:0/4202085682 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12b0139030 0x7f12b011aa00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f12a800efc0 tx=0x7f12a800c3a0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.605+0000 7f12b8438700 1 -- 172.21.15.3:0/4202085682 shutdown_connections 2023-10-03T10:15:21.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.605+0000 7f12b8438700 1 --2- 172.21.15.3:0/4202085682 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12b0139030 0x7f12b011aa00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.605+0000 7f12b8438700 1 --2- 172.21.15.3:0/4202085682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12b01338a0 0x7f12b0119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.605+0000 7f12b8438700 1 --2- 172.21.15.3:0/4202085682 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12b0136710 0x7f12b011a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.605+0000 7f12b8438700 1 -- 172.21.15.3:0/4202085682 >> 172.21.15.3:0/4202085682 conn(0x7f12b00a2aa0 msgr2=0x7f12b0130050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:21.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.605+0000 7f12b8438700 1 -- 172.21.15.3:0/4202085682 shutdown_connections 2023-10-03T10:15:21.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.605+0000 7f12b8438700 1 -- 172.21.15.3:0/4202085682 wait complete. 2023-10-03T10:15:21.619 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:21.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.675+0000 7fb6bcf2c700 1 -- 172.21.15.3:0/2737207559 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb69c005860 con 0x7fb6b809e000 2023-10-03T10:15:21.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.675+0000 7fb6a6ffd700 1 -- 172.21.15.3:0/2737207559 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb6ac021030 con 0x7fb6b809e000 2023-10-03T10:15:21.676 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:21.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.678+0000 7fb6bcf2c700 1 -- 172.21.15.3:0/2737207559 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb6b809e000 msgr2=0x7fb6b81228a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:21.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.678+0000 7fb6bcf2c700 1 --2- 172.21.15.3:0/2737207559 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb6b809e000 0x7fb6b81228a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac00ebe0 tx=0x7fb6ac00eef0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.678+0000 7fb6bcf2c700 1 -- 172.21.15.3:0/2737207559 shutdown_connections 2023-10-03T10:15:21.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.678+0000 7fb6bcf2c700 1 --2- 172.21.15.3:0/2737207559 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb6b809e410 0x7fb6b8122de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.678+0000 7fb6bcf2c700 1 --2- 172.21.15.3:0/2737207559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6b809d6f0 0x7fb6b8122360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.678+0000 7fb6bcf2c700 1 --2- 172.21.15.3:0/2737207559 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb6b809e000 0x7fb6b81228a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.678+0000 7fb6bcf2c700 1 -- 172.21.15.3:0/2737207559 >> 172.21.15.3:0/2737207559 conn(0x7fb6b8066f50 msgr2=0x7fb6b8133e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:21.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.678+0000 7fb6bcf2c700 1 -- 172.21.15.3:0/2737207559 shutdown_connections 2023-10-03T10:15:21.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.678+0000 7fb6bcf2c700 1 -- 172.21.15.3:0/2737207559 wait complete. 2023-10-03T10:15:21.691 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:21.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.706+0000 7ff970703700 1 -- 172.21.15.3:0/2601100247 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff968072980 con 0x7ff96809d630 2023-10-03T10:15:21.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.707+0000 7ff95effd700 1 -- 172.21.15.3:0/2601100247 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff964004560 con 0x7ff96809d630 2023-10-03T10:15:21.707 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:21.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.709+0000 7ff970703700 1 -- 172.21.15.3:0/2601100247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff96809d630 msgr2=0x7ff9681208f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:21.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.709+0000 7ff970703700 1 --2- 172.21.15.3:0/2601100247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff96809d630 0x7ff9681208f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ff964009c40 tx=0x7ff964004750 comp rx=0 tx=0).stop 2023-10-03T10:15:21.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.709+0000 7ff970703700 1 -- 172.21.15.3:0/2601100247 shutdown_connections 2023-10-03T10:15:21.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.709+0000 7ff970703700 1 --2- 172.21.15.3:0/2601100247 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff968138d80 0x7ff968121370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.709+0000 7ff970703700 1 --2- 172.21.15.3:0/2601100247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff96809d630 0x7ff9681208f0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.709+0000 7ff970703700 1 --2- 172.21.15.3:0/2601100247 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9680a0a30 0x7ff968120e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:21.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.709+0000 7ff970703700 1 -- 172.21.15.3:0/2601100247 >> 172.21.15.3:0/2601100247 conn(0x7ff9680a3050 msgr2=0x7ff96809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:21.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.709+0000 7ff970703700 1 -- 172.21.15.3:0/2601100247 shutdown_connections 2023-10-03T10:15:21.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:21.709+0000 7ff970703700 1 -- 172.21.15.3:0/2601100247 wait complete. 2023-10-03T10:15:21.722 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:22.568 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:22.620 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:22.693 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:22.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.720+0000 7f64a2f24700 1 Processor -- start 2023-10-03T10:15:22.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.720+0000 7f64a2f24700 1 -- start start 2023-10-03T10:15:22.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.721+0000 7f64a2f24700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f649c09d6f0 0x7f649c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:22.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.721+0000 7f64a2f24700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f649c09e000 0x7f649c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:22.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.721+0000 7f64a2f24700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f649c09b120 con 0x7f649c09e410 2023-10-03T10:15:22.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.721+0000 7f64a2f24700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f649c06b460 con 0x7f649c09d6f0 2023-10-03T10:15:22.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.721+0000 7f64a2f24700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f649c09a7e0 con 0x7f649c09e000 2023-10-03T10:15:22.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.721+0000 7f649bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f649c09e000 0x7f649c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:22.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.721+0000 7f64a14c1700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f649c09e410 0x7f649c133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47788/0 (socket says 172.21.15.3:47788) 2023-10-03T10:15:22.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.721+0000 7f64a14c1700 1 -- 172.21.15.3:0/249893762 learned_addr learned my addr 172.21.15.3:0/249893762 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:22.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.721+0000 7f64a0cc0700 1 --2- 172.21.15.3:0/249893762 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f649c09d6f0 0x7f649c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:22.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.722+0000 7f649bfff700 1 -- 172.21.15.3:0/249893762 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f649c09d6f0 msgr2=0x7f649c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:22.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.722+0000 7f649bfff700 1 --2- 172.21.15.3:0/249893762 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f649c09d6f0 0x7f649c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:22.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.722+0000 7f649bfff700 1 -- 172.21.15.3:0/249893762 >> v1:172.21.15.3:6789/0 conn(0x7f649c09e410 legacy=0x7f649c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:22.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.722+0000 7f649bfff700 1 -- 172.21.15.3:0/249893762 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f649c09a500 con 0x7f649c09e000 2023-10-03T10:15:22.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.722+0000 7f649b7fe700 1 -- 172.21.15.3:0/249893762 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1808979026 0 0) 0x7f649c09b120 con 0x7f649c09e410 2023-10-03T10:15:22.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.722+0000 7f649bfff700 1 --2- 172.21.15.3:0/249893762 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f649c09e000 0x7f649c1339f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f64880098a0 tx=0x7f6488009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=490d5f2cf592db8a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:22.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.722+0000 7f649b7fe700 1 -- 172.21.15.3:0/249893762 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6488011830 con 0x7f649c09e000 2023-10-03T10:15:22.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.722+0000 7f649b7fe700 1 -- 172.21.15.3:0/249893762 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6488011990 con 0x7f649c09e000 2023-10-03T10:15:22.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.722+0000 7f649b7fe700 1 -- 172.21.15.3:0/249893762 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6488011d20 con 0x7f649c09e000 2023-10-03T10:15:22.724 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:22.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.723+0000 7f64a2f24700 1 -- 172.21.15.3:0/249893762 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f649c09e000 msgr2=0x7f649c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:22.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.723+0000 7f64a2f24700 1 --2- 172.21.15.3:0/249893762 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f649c09e000 0x7f649c1339f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f64880098a0 tx=0x7f6488009bb0 comp rx=0 tx=0).stop 2023-10-03T10:15:22.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.723+0000 7f64a2f24700 1 -- 172.21.15.3:0/249893762 shutdown_connections 2023-10-03T10:15:22.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.723+0000 7f64a2f24700 1 --2- 172.21.15.3:0/249893762 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f649c09d6f0 0x7f649c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:22.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.723+0000 7f64a2f24700 1 --2- 172.21.15.3:0/249893762 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f649c09e000 0x7f649c1339f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:22.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.723+0000 7f64a2f24700 1 -- 172.21.15.3:0/249893762 >> 172.21.15.3:0/249893762 conn(0x7f649c066f50 msgr2=0x7f649c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:22.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.723+0000 7f64a2f24700 1 -- 172.21.15.3:0/249893762 shutdown_connections 2023-10-03T10:15:22.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.723+0000 7f64a2f24700 1 -- 172.21.15.3:0/249893762 wait complete. 2023-10-03T10:15:22.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.724+0000 7f64a2f24700 1 Processor -- start 2023-10-03T10:15:22.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.724+0000 7f64a2f24700 1 -- start start 2023-10-03T10:15:22.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.725+0000 7f64a2f24700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649c09d6f0 0x7f649c1a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:22.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.725+0000 7f64a2f24700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f649c09e000 0x7f649c1a5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:22.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.725+0000 7f64a2f24700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f649c09e410 0x7f649c1c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:22.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.725+0000 7f64a2f24700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649c1b94f0 con 0x7f649c09d6f0 2023-10-03T10:15:22.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.725+0000 7f64a2f24700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f649c1a4e70 con 0x7f649c09e410 2023-10-03T10:15:22.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.725+0000 7f64a2f24700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f649c1b9290 con 0x7f649c09e000 2023-10-03T10:15:22.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.725+0000 7f64a0cc0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649c09d6f0 0x7f649c1a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:22.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.725+0000 7f64a0cc0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649c09d6f0 0x7f649c1a5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42646/0 (socket says 172.21.15.3:42646) 2023-10-03T10:15:22.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.725+0000 7f64a0cc0700 1 -- 172.21.15.3:0/36307553 learned_addr learned my addr 172.21.15.3:0/36307553 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:22.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.725+0000 7f64a14c1700 1 --2- 172.21.15.3:0/36307553 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f649c09e410 0x7f649c1c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:22.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.726+0000 7f64a14c1700 1 -- 172.21.15.3:0/36307553 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f649c09e000 msgr2=0x7f649c1a5860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:15:22.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.726+0000 7f64a14c1700 1 --2- 172.21.15.3:0/36307553 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f649c09e000 0x7f649c1a5860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:22.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.726+0000 7f64a14c1700 1 -- 172.21.15.3:0/36307553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649c09d6f0 msgr2=0x7f649c1a5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:22.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.726+0000 7f64a14c1700 1 --2- 172.21.15.3:0/36307553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649c09d6f0 0x7f649c1a5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:22.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.726+0000 7f64a14c1700 1 -- 172.21.15.3:0/36307553 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64880094a0 con 0x7f649c09e410 2023-10-03T10:15:22.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.726+0000 7f64a0cc0700 1 --2- 172.21.15.3:0/36307553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649c09d6f0 0x7f649c1a5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:22.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.726+0000 7f64a14c1700 1 --2- 172.21.15.3:0/36307553 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f649c09e410 0x7f649c1c8150 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f649400cbe0 tx=0x7f649400cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:22.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.726+0000 7f64997fa700 1 -- 172.21.15.3:0/36307553 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649400ab10 con 0x7f649c09e410 2023-10-03T10:15:22.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.726+0000 7f64997fa700 1 -- 172.21.15.3:0/36307553 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649400ac70 con 0x7f649c09e410 2023-10-03T10:15:22.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.726+0000 7f64a2f24700 1 -- 172.21.15.3:0/36307553 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f649c1a8140 con 0x7f649c09e410 2023-10-03T10:15:22.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.726+0000 7f64997fa700 1 -- 172.21.15.3:0/36307553 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64940070d0 con 0x7f649c09e410 2023-10-03T10:15:22.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.726+0000 7f64a2f24700 1 -- 172.21.15.3:0/36307553 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f649c0a1d00 con 0x7f649c09e410 2023-10-03T10:15:22.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.727+0000 7f64a2f24700 1 -- 172.21.15.3:0/36307553 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f649c1a7db0 con 0x7f649c09e410 2023-10-03T10:15:22.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.728+0000 7f64997fa700 1 -- 172.21.15.3:0/36307553 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6494007230 con 0x7f649c09e410 2023-10-03T10:15:22.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.728+0000 7f64997fa700 1 -- 172.21.15.3:0/36307553 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f649403f780 con 0x7f649c09e410 2023-10-03T10:15:22.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.733+0000 7f64997fa700 1 -- 172.21.15.3:0/36307553 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f649401a690 con 0x7f649c09e410 2023-10-03T10:15:22.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.773+0000 7f4622f87700 1 Processor -- start 2023-10-03T10:15:22.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.773+0000 7f4622f87700 1 -- start start 2023-10-03T10:15:22.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.773+0000 7f4622f87700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f461c130540 0x7f461c130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:22.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.773+0000 7f4622f87700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f461c06fcf0 0x7f461c138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:22.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.773+0000 7f4622f87700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f461c0716a0 con 0x7f461c130e50 2023-10-03T10:15:22.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.773+0000 7f4622f87700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f461c09a390 con 0x7f461c130540 2023-10-03T10:15:22.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.773+0000 7f4622f87700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461c071290 con 0x7f461c06fcf0 2023-10-03T10:15:22.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.773+0000 7f4620d23700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f461c130540 0x7f461c130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:22.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.774+0000 7f4620d23700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f461c130540 0x7f461c130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42660/0 (socket says 172.21.15.3:42660) 2023-10-03T10:15:22.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.774+0000 7f4620d23700 1 -- 172.21.15.3:0/4274652239 learned_addr learned my addr 172.21.15.3:0/4274652239 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:22.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.774+0000 7f4621524700 1 --2- 172.21.15.3:0/4274652239 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f461c06fcf0 0x7f461c138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:22.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.774+0000 7f4621524700 1 -- 172.21.15.3:0/4274652239 >> v2:172.21.15.3:3300/0 conn(0x7f461c130540 msgr2=0x7f461c130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:22.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.774+0000 7f4621524700 1 --2- 172.21.15.3:0/4274652239 >> v2:172.21.15.3:3300/0 conn(0x7f461c130540 0x7f461c130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:22.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.774+0000 7f4621524700 1 -- 172.21.15.3:0/4274652239 >> v1:172.21.15.3:6789/0 conn(0x7f461c130e50 legacy=0x7f461c06f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:22.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.774+0000 7f4621524700 1 -- 172.21.15.3:0/4274652239 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c130340 con 0x7f461c06fcf0 2023-10-03T10:15:22.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.774+0000 7f4620d23700 1 --2- 172.21.15.3:0/4274652239 >> v2:172.21.15.3:3300/0 conn(0x7f461c130540 0x7f461c130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:22.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.774+0000 7f4621524700 1 --2- 172.21.15.3:0/4274652239 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f461c06fcf0 0x7f461c138570 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4614009dc0 tx=0x7f461400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f38d11fa96fe2dde server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:22.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.774+0000 7f461b7fe700 1 -- 172.21.15.3:0/4274652239 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461400e070 con 0x7f461c06fcf0 2023-10-03T10:15:22.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.774+0000 7f461b7fe700 1 -- 172.21.15.3:0/4274652239 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461400bae0 con 0x7f461c06fcf0 2023-10-03T10:15:22.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.774+0000 7f461b7fe700 1 -- 172.21.15.3:0/4274652239 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461400be90 con 0x7f461c06fcf0 2023-10-03T10:15:22.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.775+0000 7f4622f87700 1 -- 172.21.15.3:0/4274652239 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f461c06fcf0 msgr2=0x7f461c138570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:22.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.775+0000 7f4622f87700 1 --2- 172.21.15.3:0/4274652239 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f461c06fcf0 0x7f461c138570 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4614009dc0 tx=0x7f461400b330 comp rx=0 tx=0).stop 2023-10-03T10:15:22.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.775+0000 7f4622f87700 1 -- 172.21.15.3:0/4274652239 shutdown_connections 2023-10-03T10:15:22.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.775+0000 7f4622f87700 1 --2- 172.21.15.3:0/4274652239 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f461c06fcf0 0x7f461c138570 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:22.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.775+0000 7f4622f87700 1 --2- 172.21.15.3:0/4274652239 >> v2:172.21.15.3:3300/0 conn(0x7f461c130540 0x7f461c130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:22.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.775+0000 7f4622f87700 1 -- 172.21.15.3:0/4274652239 >> 172.21.15.3:0/4274652239 conn(0x7f461c0a2aa0 msgr2=0x7f461c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:22.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.775+0000 7f4622f87700 1 -- 172.21.15.3:0/4274652239 shutdown_connections 2023-10-03T10:15:22.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.775+0000 7f4622f87700 1 -- 172.21.15.3:0/4274652239 wait complete. 2023-10-03T10:15:22.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.776+0000 7f4622f87700 1 Processor -- start 2023-10-03T10:15:22.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.776+0000 7f4622f87700 1 -- start start 2023-10-03T10:15:22.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.777+0000 7f4622f87700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f461c06fcf0 0x7f461c11a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:22.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.777+0000 7f4622f87700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f461c130540 0x7f461c11ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:22.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.777+0000 7f4622f87700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f461c130e50 0x7f461c11b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:22.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.777+0000 7f4622f87700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c11e2f0 con 0x7f461c06fcf0 2023-10-03T10:15:22.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.777+0000 7f4622f87700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461c12dc80 con 0x7f461c130540 2023-10-03T10:15:22.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.777+0000 7f4622f87700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f461c09ce40 con 0x7f461c130e50 2023-10-03T10:15:22.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.777+0000 7f4621524700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f461c130e50 0x7f461c11b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:22.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.777+0000 7f461bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f461c130540 0x7f461c11ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:22.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.777+0000 7f4621524700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f461c130e50 0x7f461c11b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54062/0 (socket says 172.21.15.3:54062) 2023-10-03T10:15:22.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.777+0000 7f4621524700 1 -- 172.21.15.3:0/4267555416 learned_addr learned my addr 172.21.15.3:0/4267555416 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:22.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.778+0000 7f461bfff700 1 -- 172.21.15.3:0/4267555416 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f461c130e50 msgr2=0x7f461c11b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:22.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.778+0000 7f461bfff700 1 --2- 172.21.15.3:0/4267555416 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f461c130e50 0x7f461c11b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:22.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.778+0000 7f461bfff700 1 -- 172.21.15.3:0/4267555416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f461c06fcf0 msgr2=0x7f461c11a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:22.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.778+0000 7f461bfff700 1 --2- 172.21.15.3:0/4267555416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f461c06fcf0 0x7f461c11a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:22.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.778+0000 7f461bfff700 1 -- 172.21.15.3:0/4267555416 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4610009550 con 0x7f461c130540 2023-10-03T10:15:22.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.778+0000 7f461bfff700 1 --2- 172.21.15.3:0/4267555416 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f461c130540 0x7f461c11ada0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f460800c450 tx=0x7f460800c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:22.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.778+0000 7f46197fa700 1 -- 172.21.15.3:0/4267555416 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46080052d0 con 0x7f461c130540 2023-10-03T10:15:22.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.778+0000 7f4622f87700 1 -- 172.21.15.3:0/4267555416 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4614009a70 con 0x7f461c130540 2023-10-03T10:15:22.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.778+0000 7f46197fa700 1 -- 172.21.15.3:0/4267555416 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4608005430 con 0x7f461c130540 2023-10-03T10:15:22.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.778+0000 7f46197fa700 1 -- 172.21.15.3:0/4267555416 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46080038c0 con 0x7f461c130540 2023-10-03T10:15:22.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.778+0000 7f4622f87700 1 -- 172.21.15.3:0/4267555416 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461c11d1e0 con 0x7f461c130540 2023-10-03T10:15:22.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.779+0000 7f46197fa700 1 -- 172.21.15.3:0/4267555416 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4608003a20 con 0x7f461c130540 2023-10-03T10:15:22.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.779+0000 7f4622f87700 1 -- 172.21.15.3:0/4267555416 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f461c1b79e0 con 0x7f461c130540 2023-10-03T10:15:22.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.780+0000 7f46197fa700 1 -- 172.21.15.3:0/4267555416 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f460803ebf0 con 0x7f461c130540 2023-10-03T10:15:22.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.784+0000 7f46197fa700 1 -- 172.21.15.3:0/4267555416 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4608018260 con 0x7f461c130540 2023-10-03T10:15:22.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.850+0000 7f67fa84e700 1 Processor -- start 2023-10-03T10:15:22.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.850+0000 7f67fa84e700 1 -- start start 2023-10-03T10:15:22.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.851+0000 7f67fa84e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67f41338a0 0x7f67f4133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:22.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.851+0000 7f67fa84e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f67f409de50 0x7f67f409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:22.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.851+0000 7f67fa84e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67f409e760 0x7f67f413a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:22.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.851+0000 7f67fa84e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67f406fef0 con 0x7f67f409de50 2023-10-03T10:15:22.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.851+0000 7f67fa84e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f406ef30 con 0x7f67f409e760 2023-10-03T10:15:22.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.851+0000 7f67fa84e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67f41334b0 con 0x7f67f41338a0 2023-10-03T10:15:22.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.851+0000 7f67f37fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f67f409de50 0x7f67f409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:22.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.851+0000 7f67f37fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f67f409de50 0x7f67f409e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42684/0 (socket says 172.21.15.3:42684) 2023-10-03T10:15:22.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.851+0000 7f67f3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67f41338a0 0x7f67f4133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:22.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.851+0000 7f67f37fe700 1 -- 172.21.15.3:0/1128993928 learned_addr learned my addr 172.21.15.3:0/1128993928 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:22.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.851+0000 7f67f8deb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67f409e760 0x7f67f413a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:22.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.852+0000 7f67f3fff700 1 -- 172.21.15.3:0/1128993928 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67f409e760 msgr2=0x7f67f413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:22.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.852+0000 7f67f3fff700 1 --2- 172.21.15.3:0/1128993928 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67f409e760 0x7f67f413a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:22.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.852+0000 7f67f3fff700 1 -- 172.21.15.3:0/1128993928 >> v2:172.21.15.3:3300/0 conn(0x7f67f409de50 msgr2=0x7f67f409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:22.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.852+0000 7f67f3fff700 1 --2- 172.21.15.3:0/1128993928 >> v2:172.21.15.3:3300/0 conn(0x7f67f409de50 0x7f67f409e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:22.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.852+0000 7f67f3fff700 1 -- 172.21.15.3:0/1128993928 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f406fc30 con 0x7f67f41338a0 2023-10-03T10:15:22.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.852+0000 7f67f8deb700 1 --2- 172.21.15.3:0/1128993928 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67f409e760 0x7f67f413a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:22.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.852+0000 7f67f3fff700 1 --2- 172.21.15.3:0/1128993928 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67f41338a0 0x7f67f4133c70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009840 tx=0x7f67e4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f78c14f83d3d0964 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:22.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.852+0000 7f67f2ffd700 1 -- 172.21.15.3:0/1128993928 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e4011830 con 0x7f67f41338a0 2023-10-03T10:15:22.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.852+0000 7f67f2ffd700 1 -- 172.21.15.3:0/1128993928 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e4011990 con 0x7f67f41338a0 2023-10-03T10:15:22.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.852+0000 7f67f2ffd700 1 -- 172.21.15.3:0/1128993928 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e4011d20 con 0x7f67f41338a0 2023-10-03T10:15:22.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.853+0000 7f67fa84e700 1 -- 172.21.15.3:0/1128993928 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67f41338a0 msgr2=0x7f67f4133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:22.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.853+0000 7f67fa84e700 1 --2- 172.21.15.3:0/1128993928 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67f41338a0 0x7f67f4133c70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009840 tx=0x7f67e4009b50 comp rx=0 tx=0).stop 2023-10-03T10:15:22.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.853+0000 7f67fa84e700 1 -- 172.21.15.3:0/1128993928 shutdown_connections 2023-10-03T10:15:22.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.853+0000 7f67fa84e700 1 --2- 172.21.15.3:0/1128993928 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67f409e760 0x7f67f413a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:22.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.853+0000 7f67fa84e700 1 --2- 172.21.15.3:0/1128993928 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67f41338a0 0x7f67f4133c70 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:22.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.853+0000 7f67fa84e700 1 --2- 172.21.15.3:0/1128993928 >> v2:172.21.15.3:3300/0 conn(0x7f67f409de50 0x7f67f409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:22.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.853+0000 7f67fa84e700 1 -- 172.21.15.3:0/1128993928 >> 172.21.15.3:0/1128993928 conn(0x7f67f40a2aa0 msgr2=0x7f67f40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:22.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.853+0000 7f67fa84e700 1 -- 172.21.15.3:0/1128993928 shutdown_connections 2023-10-03T10:15:22.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.853+0000 7f67fa84e700 1 -- 172.21.15.3:0/1128993928 wait complete. 2023-10-03T10:15:22.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.854+0000 7f67fa84e700 1 Processor -- start 2023-10-03T10:15:22.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.855+0000 7f67fa84e700 1 -- start start 2023-10-03T10:15:22.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.855+0000 7f67fa84e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67f409de50 0x7f67f411e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:22.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.855+0000 7f67fa84e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67f409e760 0x7f67f411eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:22.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.855+0000 7f67fa84e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67f41338a0 0x7f67f411f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:22.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.855+0000 7f67fa84e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f4122540 con 0x7f67f409e760 2023-10-03T10:15:22.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.855+0000 7f67fa84e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f4121c30 con 0x7f67f409de50 2023-10-03T10:15:22.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.855+0000 7f67fa84e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67f40a0fe0 con 0x7f67f41338a0 2023-10-03T10:15:22.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.855+0000 7f67f37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67f409e760 0x7f67f411eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:22.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.855+0000 7f67f37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67f409e760 0x7f67f411eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42690/0 (socket says 172.21.15.3:42690) 2023-10-03T10:15:22.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.855+0000 7f67f37fe700 1 -- 172.21.15.3:0/309670661 learned_addr learned my addr 172.21.15.3:0/309670661 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:22.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.856+0000 7f67f8deb700 1 --2- 172.21.15.3:0/309670661 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67f41338a0 0x7f67f411f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:22.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.856+0000 7f67f3fff700 1 --2- 172.21.15.3:0/309670661 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67f409de50 0x7f67f411e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:22.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.856+0000 7f67f37fe700 1 -- 172.21.15.3:0/309670661 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67f41338a0 msgr2=0x7f67f411f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:22.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.856+0000 7f67f37fe700 1 --2- 172.21.15.3:0/309670661 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67f41338a0 0x7f67f411f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:22.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.856+0000 7f67f37fe700 1 -- 172.21.15.3:0/309670661 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67f409de50 msgr2=0x7f67f411e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:22.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.856+0000 7f67f37fe700 1 --2- 172.21.15.3:0/309670661 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67f409de50 0x7f67f411e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:22.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.856+0000 7f67f37fe700 1 -- 172.21.15.3:0/309670661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ec009a70 con 0x7f67f409e760 2023-10-03T10:15:22.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.856+0000 7f67f3fff700 1 --2- 172.21.15.3:0/309670661 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67f409de50 0x7f67f411e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:15:22.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.856+0000 7f67f37fe700 1 --2- 172.21.15.3:0/309670661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67f409e760 0x7f67f411eec0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f67e800e800 tx=0x7f67e800ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:22.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.856+0000 7f67f0ff9700 1 -- 172.21.15.3:0/309670661 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e8013680 con 0x7f67f409e760 2023-10-03T10:15:22.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.856+0000 7f67f0ff9700 1 -- 172.21.15.3:0/309670661 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e80137e0 con 0x7f67f409e760 2023-10-03T10:15:22.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.856+0000 7f67fa84e700 1 -- 172.21.15.3:0/309670661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e40094a0 con 0x7f67f409e760 2023-10-03T10:15:22.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.856+0000 7f67f0ff9700 1 -- 172.21.15.3:0/309670661 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e8013b70 con 0x7f67f409e760 2023-10-03T10:15:22.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.856+0000 7f67fa84e700 1 -- 172.21.15.3:0/309670661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f41221c0 con 0x7f67f409e760 2023-10-03T10:15:22.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.857+0000 7f67f0ff9700 1 -- 172.21.15.3:0/309670661 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f67e8013cd0 con 0x7f67f409e760 2023-10-03T10:15:22.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.858+0000 7f67f0ff9700 1 -- 172.21.15.3:0/309670661 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f67e803f610 con 0x7f67f409e760 2023-10-03T10:15:22.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.858+0000 7f67fa84e700 1 -- 172.21.15.3:0/309670661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f4074ea0 con 0x7f67f409e760 2023-10-03T10:15:22.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.867+0000 7f67f0ff9700 1 -- 172.21.15.3:0/309670661 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f67e801ba00 con 0x7f67f409e760 2023-10-03T10:15:22.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.877+0000 7f760de56700 1 Processor -- start 2023-10-03T10:15:22.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.877+0000 7f760de56700 1 -- start start 2023-10-03T10:15:22.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.878+0000 7f760de56700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f76080a1510 0x7f7608138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:22.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.878+0000 7f760de56700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7608139250 0x7f760813b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:22.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.878+0000 7f760de56700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f760809edd0 con 0x7f76080a0a30 2023-10-03T10:15:22.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.878+0000 7f760de56700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76080a18e0 con 0x7f76080a1510 2023-10-03T10:15:22.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.878+0000 7f760de56700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f760809e1f0 con 0x7f7608139250 2023-10-03T10:15:22.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.878+0000 7f7607fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7608139250 0x7f760813b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:22.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.878+0000 7f7607fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7608139250 0x7f760813b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50808/0 (socket says 172.21.15.3:50808) 2023-10-03T10:15:22.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.878+0000 7f7607fff700 1 -- 172.21.15.3:0/3493574952 learned_addr learned my addr 172.21.15.3:0/3493574952 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:22.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.878+0000 7f7607fff700 1 -- 172.21.15.3:0/3493574952 >> v2:172.21.15.3:3300/0 conn(0x7f76080a1510 msgr2=0x7f7608138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:22.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.878+0000 7f7607fff700 1 --2- 172.21.15.3:0/3493574952 >> v2:172.21.15.3:3300/0 conn(0x7f76080a1510 0x7f7608138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:22.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.878+0000 7f7607fff700 1 -- 172.21.15.3:0/3493574952 >> v1:172.21.15.3:6789/0 conn(0x7f76080a0a30 legacy=0x7f76080a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:22.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.878+0000 7f7607fff700 1 -- 172.21.15.3:0/3493574952 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f760809dc50 con 0x7f7608139250 2023-10-03T10:15:22.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.878+0000 7f76067fc700 1 -- 172.21.15.3:0/3493574952 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 284878405 0 0) 0x7f760809edd0 con 0x7f76080a0a30 2023-10-03T10:15:22.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.879+0000 7f7607fff700 1 --2- 172.21.15.3:0/3493574952 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7608139250 0x7f760813b630 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7600009e50 tx=0x7f760000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=497767e456af05dc server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:22.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.879+0000 7f76067fc700 1 -- 172.21.15.3:0/3493574952 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760000e070 con 0x7f7608139250 2023-10-03T10:15:22.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.879+0000 7f76067fc700 1 -- 172.21.15.3:0/3493574952 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f760000b9c0 con 0x7f7608139250 2023-10-03T10:15:22.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.879+0000 7f76067fc700 1 -- 172.21.15.3:0/3493574952 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760000bd70 con 0x7f7608139250 2023-10-03T10:15:22.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.880+0000 7f760de56700 1 -- 172.21.15.3:0/3493574952 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7608139250 msgr2=0x7f760813b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:22.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.880+0000 7f760de56700 1 --2- 172.21.15.3:0/3493574952 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7608139250 0x7f760813b630 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7600009e50 tx=0x7f760000b330 comp rx=0 tx=0).stop 2023-10-03T10:15:22.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.880+0000 7f760de56700 1 -- 172.21.15.3:0/3493574952 shutdown_connections 2023-10-03T10:15:22.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.880+0000 7f760de56700 1 --2- 172.21.15.3:0/3493574952 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7608139250 0x7f760813b630 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:22.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.880+0000 7f760de56700 1 --2- 172.21.15.3:0/3493574952 >> v2:172.21.15.3:3300/0 conn(0x7f76080a1510 0x7f7608138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:22.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.880+0000 7f760de56700 1 -- 172.21.15.3:0/3493574952 >> 172.21.15.3:0/3493574952 conn(0x7f76080a6550 msgr2=0x7f76080a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:22.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.880+0000 7f760de56700 1 -- 172.21.15.3:0/3493574952 shutdown_connections 2023-10-03T10:15:22.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.880+0000 7f760de56700 1 -- 172.21.15.3:0/3493574952 wait complete. 2023-10-03T10:15:22.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.881+0000 7f760de56700 1 Processor -- start 2023-10-03T10:15:22.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.881+0000 7f760de56700 1 -- start start 2023-10-03T10:15:22.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.882+0000 7f760de56700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76080a0a30 0x7f7608133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:22.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.882+0000 7f760de56700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76080a1510 0x7f7608134310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:22.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.882+0000 7f760de56700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7608139250 0x7f7608134850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:22.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.882+0000 7f760de56700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7608136410 con 0x7f7608139250 2023-10-03T10:15:22.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.882+0000 7f760de56700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76081390e0 con 0x7f76080a1510 2023-10-03T10:15:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.882+0000 7f760de56700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76081365f0 con 0x7f76080a0a30 2023-10-03T10:15:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.882+0000 7f76077fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76080a0a30 0x7f7608133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.882+0000 7f76077fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76080a0a30 0x7f7608133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54080/0 (socket says 172.21.15.3:54080) 2023-10-03T10:15:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.882+0000 7f76077fe700 1 -- 172.21.15.3:0/4142969457 learned_addr learned my addr 172.21.15.3:0/4142969457 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.882+0000 7f7607fff700 1 --2- 172.21.15.3:0/4142969457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7608139250 0x7f7608134850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.882+0000 7f7606ffd700 1 --2- 172.21.15.3:0/4142969457 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76080a1510 0x7f7608134310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.882+0000 7f76077fe700 1 -- 172.21.15.3:0/4142969457 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76080a1510 msgr2=0x7f7608134310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.882+0000 7f76077fe700 1 --2- 172.21.15.3:0/4142969457 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76080a1510 0x7f7608134310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.882+0000 7f76077fe700 1 -- 172.21.15.3:0/4142969457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7608139250 msgr2=0x7f7608134850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.882+0000 7f76077fe700 1 --2- 172.21.15.3:0/4142969457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7608139250 0x7f7608134850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.882+0000 7f76077fe700 1 -- 172.21.15.3:0/4142969457 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7600009b30 con 0x7f76080a0a30 2023-10-03T10:15:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.882+0000 7f76077fe700 1 --2- 172.21.15.3:0/4142969457 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76080a0a30 0x7f7608133dd0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f75f8001eb0 tx=0x7f75f800c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.883+0000 7f75effff700 1 -- 172.21.15.3:0/4142969457 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f80054e0 con 0x7f76080a0a30 2023-10-03T10:15:22.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.883+0000 7f75effff700 1 -- 172.21.15.3:0/4142969457 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f8003680 con 0x7f76080a0a30 2023-10-03T10:15:22.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.883+0000 7f75effff700 1 -- 172.21.15.3:0/4142969457 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f80039f0 con 0x7f76080a0a30 2023-10-03T10:15:22.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.883+0000 7f760de56700 1 -- 172.21.15.3:0/4142969457 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76080a0220 con 0x7f76080a0a30 2023-10-03T10:15:22.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.883+0000 7f760de56700 1 -- 172.21.15.3:0/4142969457 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7608135100 con 0x7f76080a0a30 2023-10-03T10:15:22.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.884+0000 7f760de56700 1 -- 172.21.15.3:0/4142969457 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7608074ea0 con 0x7f76080a0a30 2023-10-03T10:15:22.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.885+0000 7f75effff700 1 -- 172.21.15.3:0/4142969457 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f75f8003b50 con 0x7f76080a0a30 2023-10-03T10:15:22.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.885+0000 7f75effff700 1 -- 172.21.15.3:0/4142969457 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f75f803eb40 con 0x7f76080a0a30 2023-10-03T10:15:22.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.894+0000 7f75effff700 1 -- 172.21.15.3:0/4142969457 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f75f8012400 con 0x7f76080a0a30 2023-10-03T10:15:22.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.964+0000 7f64a2f24700 1 -- 172.21.15.3:0/36307553 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f649c074ea0 con 0x7f649c09e410 2023-10-03T10:15:22.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.965+0000 7f64997fa700 1 -- 172.21.15.3:0/36307553 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6494018020 con 0x7f649c09e410 2023-10-03T10:15:22.965 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:22.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.967+0000 7f64a2f24700 1 -- 172.21.15.3:0/36307553 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f649c09e410 msgr2=0x7f649c1c8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:22.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.967+0000 7f64a2f24700 1 --2- 172.21.15.3:0/36307553 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f649c09e410 0x7f649c1c8150 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f649400cbe0 tx=0x7f649400cef0 comp rx=0 tx=0).stop 2023-10-03T10:15:22.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.968+0000 7f64a2f24700 1 -- 172.21.15.3:0/36307553 shutdown_connections 2023-10-03T10:15:22.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.968+0000 7f64a2f24700 1 --2- 172.21.15.3:0/36307553 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f649c09e410 0x7f649c1c8150 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:22.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.968+0000 7f64a2f24700 1 --2- 172.21.15.3:0/36307553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649c09d6f0 0x7f649c1a5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:22.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.968+0000 7f64a2f24700 1 --2- 172.21.15.3:0/36307553 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f649c09e000 0x7f649c1a5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:22.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.968+0000 7f64a2f24700 1 -- 172.21.15.3:0/36307553 >> 172.21.15.3:0/36307553 conn(0x7f649c066f50 msgr2=0x7f649c06f140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:22.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.968+0000 7f64a2f24700 1 -- 172.21.15.3:0/36307553 shutdown_connections 2023-10-03T10:15:22.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:22.968+0000 7f64a2f24700 1 -- 172.21.15.3:0/36307553 wait complete. 2023-10-03T10:15:22.982 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:23.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.001+0000 7f4622f87700 1 -- 172.21.15.3:0/4267555416 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f461c074ea0 con 0x7f461c130540 2023-10-03T10:15:23.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.002+0000 7f46197fa700 1 -- 172.21.15.3:0/4267555416 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4608017020 con 0x7f461c130540 2023-10-03T10:15:23.002 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:23.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.004+0000 7f4622f87700 1 -- 172.21.15.3:0/4267555416 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f461c130540 msgr2=0x7f461c11ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:23.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.004+0000 7f4622f87700 1 --2- 172.21.15.3:0/4267555416 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f461c130540 0x7f461c11ada0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f460800c450 tx=0x7f460800c760 comp rx=0 tx=0).stop 2023-10-03T10:15:23.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.004+0000 7f4622f87700 1 -- 172.21.15.3:0/4267555416 shutdown_connections 2023-10-03T10:15:23.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.004+0000 7f4622f87700 1 --2- 172.21.15.3:0/4267555416 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f461c130e50 0x7f461c11b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:23.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.004+0000 7f4622f87700 1 --2- 172.21.15.3:0/4267555416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f461c06fcf0 0x7f461c11a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:23.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.004+0000 7f4622f87700 1 --2- 172.21.15.3:0/4267555416 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f461c130540 0x7f461c11ada0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:23.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.004+0000 7f4622f87700 1 -- 172.21.15.3:0/4267555416 >> 172.21.15.3:0/4267555416 conn(0x7f461c0a2aa0 msgr2=0x7f461c0a1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:23.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.004+0000 7f4622f87700 1 -- 172.21.15.3:0/4267555416 shutdown_connections 2023-10-03T10:15:23.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.004+0000 7f4622f87700 1 -- 172.21.15.3:0/4267555416 wait complete. 2023-10-03T10:15:23.018 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:23.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.113+0000 7f67fa84e700 1 -- 172.21.15.3:0/309670661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f67f406b9e0 con 0x7f67f409e760 2023-10-03T10:15:23.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.113+0000 7f67f0ff9700 1 -- 172.21.15.3:0/309670661 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f67e8021030 con 0x7f67f409e760 2023-10-03T10:15:23.114 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:23.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.115+0000 7f67fa84e700 1 -- 172.21.15.3:0/309670661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67f409e760 msgr2=0x7f67f411eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:23.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.115+0000 7f67fa84e700 1 --2- 172.21.15.3:0/309670661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67f409e760 0x7f67f411eec0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f67e800e800 tx=0x7f67e800ebc0 comp rx=0 tx=0).stop 2023-10-03T10:15:23.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.116+0000 7f67fa84e700 1 -- 172.21.15.3:0/309670661 shutdown_connections 2023-10-03T10:15:23.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.116+0000 7f67fa84e700 1 --2- 172.21.15.3:0/309670661 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67f41338a0 0x7f67f411f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:23.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.116+0000 7f67fa84e700 1 --2- 172.21.15.3:0/309670661 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67f409de50 0x7f67f411e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:23.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.116+0000 7f67fa84e700 1 --2- 172.21.15.3:0/309670661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67f409e760 0x7f67f411eec0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:23.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.116+0000 7f67fa84e700 1 -- 172.21.15.3:0/309670661 >> 172.21.15.3:0/309670661 conn(0x7f67f40a2aa0 msgr2=0x7f67f409ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:23.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.116+0000 7f67fa84e700 1 -- 172.21.15.3:0/309670661 shutdown_connections 2023-10-03T10:15:23.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.116+0000 7f67fa84e700 1 -- 172.21.15.3:0/309670661 wait complete. 2023-10-03T10:15:23.129 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:23.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.136+0000 7f760de56700 1 -- 172.21.15.3:0/4142969457 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f76080a28e0 con 0x7f76080a0a30 2023-10-03T10:15:23.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.136+0000 7f75effff700 1 -- 172.21.15.3:0/4142969457 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f75f8017020 con 0x7f76080a0a30 2023-10-03T10:15:23.137 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:23.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.138+0000 7f760de56700 1 -- 172.21.15.3:0/4142969457 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76080a0a30 msgr2=0x7f7608133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:23.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.138+0000 7f760de56700 1 --2- 172.21.15.3:0/4142969457 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76080a0a30 0x7f7608133dd0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f75f8001eb0 tx=0x7f75f800c800 comp rx=0 tx=0).stop 2023-10-03T10:15:23.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.139+0000 7f760de56700 1 -- 172.21.15.3:0/4142969457 shutdown_connections 2023-10-03T10:15:23.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.139+0000 7f760de56700 1 --2- 172.21.15.3:0/4142969457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7608139250 0x7f7608134850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:23.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.139+0000 7f760de56700 1 --2- 172.21.15.3:0/4142969457 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76080a0a30 0x7f7608133dd0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:23.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.139+0000 7f760de56700 1 --2- 172.21.15.3:0/4142969457 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76080a1510 0x7f7608134310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:23.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.139+0000 7f760de56700 1 -- 172.21.15.3:0/4142969457 >> 172.21.15.3:0/4142969457 conn(0x7f76080a6550 msgr2=0x7f7608132a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:23.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.139+0000 7f760de56700 1 -- 172.21.15.3:0/4142969457 shutdown_connections 2023-10-03T10:15:23.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:23.139+0000 7f760de56700 1 -- 172.21.15.3:0/4142969457 wait complete. 2023-10-03T10:15:23.152 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:23.984 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:24.019 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:24.130 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:24.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.132+0000 7fa8d741a700 1 Processor -- start 2023-10-03T10:15:24.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.133+0000 7fa8d741a700 1 -- start start 2023-10-03T10:15:24.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.133+0000 7fa8d741a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa8d009d6f0 0x7fa8d009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:24.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.133+0000 7fa8d741a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa8d009e000 0x7fa8d01339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:24.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.133+0000 7fa8d741a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa8d009e410 0x7fa8d0133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:24.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.133+0000 7fa8d741a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8d006b460 con 0x7fa8d009e410 2023-10-03T10:15:24.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.133+0000 7fa8d741a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d009b120 con 0x7fa8d009e000 2023-10-03T10:15:24.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.133+0000 7fa8d741a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d009a7e0 con 0x7fa8d009d6f0 2023-10-03T10:15:24.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.133+0000 7fa8d51b6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa8d009d6f0 0x7fa8d009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:24.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.133+0000 7fa8d49b5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa8d009e000 0x7fa8d01339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:24.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.133+0000 7fa8d51b6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa8d009d6f0 0x7fa8d009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43438/0 (socket says 172.21.15.3:43438) 2023-10-03T10:15:24.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.133+0000 7fa8d51b6700 1 -- 172.21.15.3:0/1698047329 learned_addr learned my addr 172.21.15.3:0/1698047329 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.133+0000 7fa8d59b7700 1 --2- 172.21.15.3:0/1698047329 >> v2:172.21.15.3:3300/0 conn(0x7fa8d009e410 0x7fa8d0133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.134+0000 7fa8d49b5700 1 -- 172.21.15.3:0/1698047329 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa8d009d6f0 msgr2=0x7fa8d009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.134+0000 7fa8d49b5700 1 --2- 172.21.15.3:0/1698047329 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa8d009d6f0 0x7fa8d009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.134+0000 7fa8d49b5700 1 -- 172.21.15.3:0/1698047329 >> v2:172.21.15.3:3300/0 conn(0x7fa8d009e410 msgr2=0x7fa8d0133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.134+0000 7fa8d49b5700 1 --2- 172.21.15.3:0/1698047329 >> v2:172.21.15.3:3300/0 conn(0x7fa8d009e410 0x7fa8d0133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.134+0000 7fa8d49b5700 1 -- 172.21.15.3:0/1698047329 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d009a500 con 0x7fa8d009e000 2023-10-03T10:15:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.134+0000 7fa8d51b6700 1 --2- 172.21.15.3:0/1698047329 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa8d009d6f0 0x7fa8d009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.134+0000 7fa8d49b5700 1 --2- 172.21.15.3:0/1698047329 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa8d009e000 0x7fa8d01339f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009870 tx=0x7fa8c0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af0ca85a93d1ba68 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.134+0000 7fa8c7fff700 1 -- 172.21.15.3:0/1698047329 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0011830 con 0x7fa8d009e000 2023-10-03T10:15:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.134+0000 7fa8c7fff700 1 -- 172.21.15.3:0/1698047329 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c0011990 con 0x7fa8d009e000 2023-10-03T10:15:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.134+0000 7fa8c7fff700 1 -- 172.21.15.3:0/1698047329 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0011d20 con 0x7fa8d009e000 2023-10-03T10:15:24.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.135+0000 7fa8d741a700 1 -- 172.21.15.3:0/1698047329 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa8d009e000 msgr2=0x7fa8d01339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:24.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.135+0000 7fa8d741a700 1 --2- 172.21.15.3:0/1698047329 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa8d009e000 0x7fa8d01339f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009870 tx=0x7fa8c0009b80 comp rx=0 tx=0).stop 2023-10-03T10:15:24.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.135+0000 7fa8d741a700 1 -- 172.21.15.3:0/1698047329 shutdown_connections 2023-10-03T10:15:24.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.135+0000 7fa8d741a700 1 --2- 172.21.15.3:0/1698047329 >> v2:172.21.15.3:3300/0 conn(0x7fa8d009e410 0x7fa8d0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.135+0000 7fa8d741a700 1 --2- 172.21.15.3:0/1698047329 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa8d009d6f0 0x7fa8d009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.135+0000 7fa8d741a700 1 --2- 172.21.15.3:0/1698047329 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa8d009e000 0x7fa8d01339f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.135+0000 7fa8d741a700 1 -- 172.21.15.3:0/1698047329 >> 172.21.15.3:0/1698047329 conn(0x7fa8d0066f50 msgr2=0x7fa8d009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.135+0000 7fa8d741a700 1 -- 172.21.15.3:0/1698047329 shutdown_connections 2023-10-03T10:15:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.135+0000 7fa8d741a700 1 -- 172.21.15.3:0/1698047329 wait complete. 2023-10-03T10:15:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.137+0000 7fa8d741a700 1 Processor -- start 2023-10-03T10:15:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.137+0000 7fa8d741a700 1 -- start start 2023-10-03T10:15:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.138+0000 7fa8d741a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa8d009d6f0 0x7fa8d01a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.138+0000 7fa8d741a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d009e000 0x7fa8d01a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.138+0000 7fa8d741a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa8d009e410 0x7fa8d01a6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.138+0000 7fa8d741a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d01a96c0 con 0x7fa8d009e000 2023-10-03T10:15:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.138+0000 7fa8d741a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d01a9910 con 0x7fa8d009d6f0 2023-10-03T10:15:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.138+0000 7fa8d741a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d009c6d0 con 0x7fa8d009e410 2023-10-03T10:15:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.138+0000 7fa8d51b6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa8d009d6f0 0x7fa8d01a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.138+0000 7fa8d51b6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa8d009d6f0 0x7fa8d01a5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46420/0 (socket says 172.21.15.3:46420) 2023-10-03T10:15:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.138+0000 7fa8d51b6700 1 -- 172.21.15.3:0/1808660849 learned_addr learned my addr 172.21.15.3:0/1808660849 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.138+0000 7fa8d49b5700 1 --2- 172.21.15.3:0/1808660849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d009e000 0x7fa8d01a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.138+0000 7fa8d59b7700 1 --2- 172.21.15.3:0/1808660849 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa8d009e410 0x7fa8d01a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.138+0000 7fa8d49b5700 1 -- 172.21.15.3:0/1808660849 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa8d009e410 msgr2=0x7fa8d01a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.138+0000 7fa8d49b5700 1 --2- 172.21.15.3:0/1808660849 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa8d009e410 0x7fa8d01a6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.138+0000 7fa8d49b5700 1 -- 172.21.15.3:0/1808660849 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa8d009d6f0 msgr2=0x7fa8d01a5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.138+0000 7fa8d49b5700 1 --2- 172.21.15.3:0/1808660849 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa8d009d6f0 0x7fa8d01a5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.138+0000 7fa8d49b5700 1 -- 172.21.15.3:0/1808660849 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8cc009550 con 0x7fa8d009e000 2023-10-03T10:15:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.138+0000 7fa8d59b7700 1 --2- 172.21.15.3:0/1808660849 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa8d009e410 0x7fa8d01a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:15:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.138+0000 7fa8d51b6700 1 --2- 172.21.15.3:0/1808660849 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa8d009d6f0 0x7fa8d01a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:24.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.138+0000 7fa8d49b5700 1 --2- 172.21.15.3:0/1808660849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d009e000 0x7fa8d01a6400 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0016000 tx=0x7fa8c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:24.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.139+0000 7fa8c5ffb700 1 -- 172.21.15.3:0/1808660849 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0004510 con 0x7fa8d009e000 2023-10-03T10:15:24.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.139+0000 7fa8c5ffb700 1 -- 172.21.15.3:0/1808660849 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c001f070 con 0x7fa8d009e000 2023-10-03T10:15:24.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.139+0000 7fa8c5ffb700 1 -- 172.21.15.3:0/1808660849 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c00038c0 con 0x7fa8d009e000 2023-10-03T10:15:24.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.139+0000 7fa8d741a700 1 -- 172.21.15.3:0/1808660849 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c00094a0 con 0x7fa8d009e000 2023-10-03T10:15:24.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.139+0000 7fa8d741a700 1 -- 172.21.15.3:0/1808660849 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d00a30e0 con 0x7fa8d009e000 2023-10-03T10:15:24.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.140+0000 7fa8c5ffb700 1 -- 172.21.15.3:0/1808660849 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa8c0003a20 con 0x7fa8d009e000 2023-10-03T10:15:24.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.140+0000 7fa8c5ffb700 1 -- 172.21.15.3:0/1808660849 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa8c0021070 con 0x7fa8d009e000 2023-10-03T10:15:24.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.140+0000 7fa8d741a700 1 -- 172.21.15.3:0/1808660849 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8d01a7b00 con 0x7fa8d009e000 2023-10-03T10:15:24.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.146+0000 7fa8c5ffb700 1 -- 172.21.15.3:0/1808660849 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa8c0015080 con 0x7fa8d009e000 2023-10-03T10:15:24.153 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:24.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.176+0000 7f742e920700 1 Processor -- start 2023-10-03T10:15:24.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.176+0000 7f742e920700 1 -- start start 2023-10-03T10:15:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.176+0000 7f742e920700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7428131810 0x7f7428131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.176+0000 7f742e920700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7428099a40 0x7f7428099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.176+0000 7f742e920700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74281313c0 con 0x7f742809a3b0 2023-10-03T10:15:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.176+0000 7f742e920700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f742809b580 con 0x7f7428131810 2023-10-03T10:15:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.176+0000 7f742e920700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742806e8c0 con 0x7f7428099a40 2023-10-03T10:15:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.177+0000 7f7427fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7428131810 0x7f7428131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.177+0000 7f742cebd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f742809a3b0 0x7f7428138650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39642/0 (socket says 172.21.15.3:39642) 2023-10-03T10:15:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.177+0000 7f742cebd700 1 -- 172.21.15.3:0/2528138734 learned_addr learned my addr 172.21.15.3:0/2528138734 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.177+0000 7f74277fe700 1 --2- 172.21.15.3:0/2528138734 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7428099a40 0x7f7428099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:24.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.177+0000 7f74277fe700 1 -- 172.21.15.3:0/2528138734 >> v2:172.21.15.3:3300/0 conn(0x7f7428131810 msgr2=0x7f7428131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:24.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.177+0000 7f74277fe700 1 --2- 172.21.15.3:0/2528138734 >> v2:172.21.15.3:3300/0 conn(0x7f7428131810 0x7f7428131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.177+0000 7f74277fe700 1 -- 172.21.15.3:0/2528138734 >> v1:172.21.15.3:6789/0 conn(0x7f742809a3b0 legacy=0x7f7428138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:24.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.177+0000 7f74277fe700 1 -- 172.21.15.3:0/2528138734 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f742809a830 con 0x7f7428099a40 2023-10-03T10:15:24.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.177+0000 7f7426ffd700 1 -- 172.21.15.3:0/2528138734 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4054129963 0 0) 0x7f74281313c0 con 0x7f742809a3b0 2023-10-03T10:15:24.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.177+0000 7f74277fe700 1 --2- 172.21.15.3:0/2528138734 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7428099a40 0x7f7428099e70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f741c009870 tx=0x7f741c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cbc7902a06a632 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:24.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.178+0000 7f7426ffd700 1 -- 172.21.15.3:0/2528138734 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c011830 con 0x7f7428099a40 2023-10-03T10:15:24.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.178+0000 7f7426ffd700 1 -- 172.21.15.3:0/2528138734 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741c011990 con 0x7f7428099a40 2023-10-03T10:15:24.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.178+0000 7f7426ffd700 1 -- 172.21.15.3:0/2528138734 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c011d20 con 0x7f7428099a40 2023-10-03T10:15:24.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.178+0000 7f742e920700 1 -- 172.21.15.3:0/2528138734 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7428099a40 msgr2=0x7f7428099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:24.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.178+0000 7f742e920700 1 --2- 172.21.15.3:0/2528138734 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7428099a40 0x7f7428099e70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f741c009870 tx=0x7f741c009b80 comp rx=0 tx=0).stop 2023-10-03T10:15:24.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.179+0000 7f742e920700 1 -- 172.21.15.3:0/2528138734 shutdown_connections 2023-10-03T10:15:24.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.179+0000 7f742e920700 1 --2- 172.21.15.3:0/2528138734 >> v2:172.21.15.3:3300/0 conn(0x7f7428131810 0x7f7428131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.179+0000 7f742e920700 1 --2- 172.21.15.3:0/2528138734 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7428099a40 0x7f7428099e70 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.179+0000 7f742e920700 1 -- 172.21.15.3:0/2528138734 >> 172.21.15.3:0/2528138734 conn(0x7f74280a2aa0 msgr2=0x7f74280a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:24.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.179+0000 7f742e920700 1 -- 172.21.15.3:0/2528138734 shutdown_connections 2023-10-03T10:15:24.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.179+0000 7f742e920700 1 -- 172.21.15.3:0/2528138734 wait complete. 2023-10-03T10:15:24.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.180+0000 7f742e920700 1 Processor -- start 2023-10-03T10:15:24.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.180+0000 7f742e920700 1 -- start start 2023-10-03T10:15:24.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.180+0000 7f742e920700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7428099a40 0x7f742811ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:24.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.180+0000 7f742e920700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f742809a3b0 0x7f742811f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:24.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.181+0000 7f742e920700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7428131810 0x7f742811f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:24.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.181+0000 7f742e920700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7428122270 con 0x7f7428099a40 2023-10-03T10:15:24.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.181+0000 7f742e920700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742806eb00 con 0x7f742809a3b0 2023-10-03T10:15:24.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.181+0000 7f742e920700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74281201e0 con 0x7f7428131810 2023-10-03T10:15:24.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.181+0000 7f7427fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7428099a40 0x7f742811ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:24.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.181+0000 7f7427fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7428099a40 0x7f742811ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45008/0 (socket says 172.21.15.3:45008) 2023-10-03T10:15:24.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.181+0000 7f7427fff700 1 -- 172.21.15.3:0/2960323973 learned_addr learned my addr 172.21.15.3:0/2960323973 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:24.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.181+0000 7f74277fe700 1 --2- 172.21.15.3:0/2960323973 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f742809a3b0 0x7f742811f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.181+0000 7f74277fe700 1 -- 172.21.15.3:0/2960323973 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7428131810 msgr2=0x7f742811f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.181+0000 7f74277fe700 1 --2- 172.21.15.3:0/2960323973 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7428131810 0x7f742811f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.181+0000 7f74277fe700 1 -- 172.21.15.3:0/2960323973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7428099a40 msgr2=0x7f742811ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.181+0000 7f74277fe700 1 --2- 172.21.15.3:0/2960323973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7428099a40 0x7f742811ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.181+0000 7f74277fe700 1 -- 172.21.15.3:0/2960323973 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f741c0094a0 con 0x7f742809a3b0 2023-10-03T10:15:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.181+0000 7f74277fe700 1 --2- 172.21.15.3:0/2960323973 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f742809a3b0 0x7f742811f2c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f741c016000 tx=0x7f741c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.182+0000 7f7424ff9700 1 -- 172.21.15.3:0/2960323973 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c0036a0 con 0x7f742809a3b0 2023-10-03T10:15:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.182+0000 7f7424ff9700 1 -- 172.21.15.3:0/2960323973 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741c01f070 con 0x7f742809a3b0 2023-10-03T10:15:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.182+0000 7f742e920700 1 -- 172.21.15.3:0/2960323973 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7428122810 con 0x7f742809a3b0 2023-10-03T10:15:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.182+0000 7f742e920700 1 -- 172.21.15.3:0/2960323973 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74280a19f0 con 0x7f742809a3b0 2023-10-03T10:15:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.182+0000 7f7424ff9700 1 -- 172.21.15.3:0/2960323973 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c003a20 con 0x7f742809a3b0 2023-10-03T10:15:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.183+0000 7f742e920700 1 -- 172.21.15.3:0/2960323973 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74080052e0 con 0x7f742809a3b0 2023-10-03T10:15:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.184+0000 7f7424ff9700 1 -- 172.21.15.3:0/2960323973 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f741c003b80 con 0x7f742809a3b0 2023-10-03T10:15:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.184+0000 7f7424ff9700 1 -- 172.21.15.3:0/2960323973 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f741c021070 con 0x7f742809a3b0 2023-10-03T10:15:24.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.193+0000 7f7424ff9700 1 -- 172.21.15.3:0/2960323973 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f741c003e30 con 0x7f742809a3b0 2023-10-03T10:15:24.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.278+0000 7eff1ff3d700 1 Processor -- start 2023-10-03T10:15:24.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.279+0000 7eff1ff3d700 1 -- start start 2023-10-03T10:15:24.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.279+0000 7eff1ff3d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7eff1806a770 0x7eff1806ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:24.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.279+0000 7eff1ff3d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff1806b080 0x7eff18131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:24.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.279+0000 7eff1ff3d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff1809ece0 con 0x7eff18131b40 2023-10-03T10:15:24.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.279+0000 7eff1ff3d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff1809e760 con 0x7eff1806a770 2023-10-03T10:15:24.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.279+0000 7eff1ff3d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff1809e040 con 0x7eff1806b080 2023-10-03T10:15:24.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.280+0000 7eff1dcd9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7eff1806a770 0x7eff1806ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:24.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.280+0000 7eff1e4da700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7eff18131b40 0x7eff18138850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39650/0 (socket says 172.21.15.3:39650) 2023-10-03T10:15:24.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.280+0000 7eff1e4da700 1 -- 172.21.15.3:0/2711562212 learned_addr learned my addr 172.21.15.3:0/2711562212 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:24.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.280+0000 7eff1d4d8700 1 --2- 172.21.15.3:0/2711562212 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff1806b080 0x7eff18131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:24.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.280+0000 7eff1dcd9700 1 -- 172.21.15.3:0/2711562212 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff1806b080 msgr2=0x7eff18131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:24.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.280+0000 7eff1dcd9700 1 --2- 172.21.15.3:0/2711562212 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff1806b080 0x7eff18131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.280+0000 7eff1dcd9700 1 -- 172.21.15.3:0/2711562212 >> v1:172.21.15.3:6789/0 conn(0x7eff18131b40 legacy=0x7eff18138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:24.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.280+0000 7eff1dcd9700 1 -- 172.21.15.3:0/2711562212 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff1809f630 con 0x7eff1806a770 2023-10-03T10:15:24.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.280+0000 7eff1ccd7700 1 -- 172.21.15.3:0/2711562212 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807259672 0 0) 0x7eff1809ece0 con 0x7eff18131b40 2023-10-03T10:15:24.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.280+0000 7eff1dcd9700 1 --2- 172.21.15.3:0/2711562212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff1806a770 0x7eff1806ab40 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7eff14009840 tx=0x7eff14009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc47a6b1d706b7e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:24.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.281+0000 7eff1ccd7700 1 -- 172.21.15.3:0/2711562212 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff14011830 con 0x7eff1806a770 2023-10-03T10:15:24.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.281+0000 7eff1ccd7700 1 -- 172.21.15.3:0/2711562212 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff14011990 con 0x7eff1806a770 2023-10-03T10:15:24.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.281+0000 7eff1ccd7700 1 -- 172.21.15.3:0/2711562212 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff14011d20 con 0x7eff1806a770 2023-10-03T10:15:24.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.281+0000 7eff1ff3d700 1 -- 172.21.15.3:0/2711562212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff1806a770 msgr2=0x7eff1806ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:24.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.281+0000 7eff1ff3d700 1 --2- 172.21.15.3:0/2711562212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff1806a770 0x7eff1806ab40 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7eff14009840 tx=0x7eff14009b50 comp rx=0 tx=0).stop 2023-10-03T10:15:24.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.281+0000 7eff1ff3d700 1 -- 172.21.15.3:0/2711562212 shutdown_connections 2023-10-03T10:15:24.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.281+0000 7eff1ff3d700 1 --2- 172.21.15.3:0/2711562212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff1806a770 0x7eff1806ab40 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.281+0000 7eff1ff3d700 1 --2- 172.21.15.3:0/2711562212 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff1806b080 0x7eff18131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.281+0000 7eff1ff3d700 1 -- 172.21.15.3:0/2711562212 >> 172.21.15.3:0/2711562212 conn(0x7eff18066f50 msgr2=0x7eff1809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:24.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.282+0000 7eff1ff3d700 1 -- 172.21.15.3:0/2711562212 shutdown_connections 2023-10-03T10:15:24.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.282+0000 7eff1ff3d700 1 -- 172.21.15.3:0/2711562212 wait complete. 2023-10-03T10:15:24.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.283+0000 7eff1ff3d700 1 Processor -- start 2023-10-03T10:15:24.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.283+0000 7eff1ff3d700 1 -- start start 2023-10-03T10:15:24.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.284+0000 7eff1ff3d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff1806a770 0x7eff18120810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:24.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.284+0000 7eff1ff3d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff1806b080 0x7eff18120e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:24.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.284+0000 7eff1ff3d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff18131b40 0x7eff18121370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:24.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.284+0000 7eff1ff3d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff18122e50 con 0x7eff1806b080 2023-10-03T10:15:24.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.284+0000 7eff1ff3d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff18123bc0 con 0x7eff18131b40 2023-10-03T10:15:24.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.284+0000 7eff1ff3d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff18123e10 con 0x7eff1806a770 2023-10-03T10:15:24.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.284+0000 7eff1e4da700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff18131b40 0x7eff18121370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:24.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.284+0000 7eff1dcd9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff1806a770 0x7eff18120810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:24.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.284+0000 7eff1e4da700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff18131b40 0x7eff18121370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46432/0 (socket says 172.21.15.3:46432) 2023-10-03T10:15:24.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.284+0000 7eff1e4da700 1 -- 172.21.15.3:0/2827173101 learned_addr learned my addr 172.21.15.3:0/2827173101 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:24.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.284+0000 7eff1d4d8700 1 --2- 172.21.15.3:0/2827173101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff1806b080 0x7eff18120e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:24.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.284+0000 7eff1e4da700 1 -- 172.21.15.3:0/2827173101 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff1806a770 msgr2=0x7eff18120810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:24.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.284+0000 7eff1e4da700 1 --2- 172.21.15.3:0/2827173101 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff1806a770 0x7eff18120810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.284+0000 7eff1e4da700 1 -- 172.21.15.3:0/2827173101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff1806b080 msgr2=0x7eff18120e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:24.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.284+0000 7eff1e4da700 1 --2- 172.21.15.3:0/2827173101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff1806b080 0x7eff18120e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.284+0000 7eff1e4da700 1 -- 172.21.15.3:0/2827173101 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff140094a0 con 0x7eff18131b40 2023-10-03T10:15:24.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.285+0000 7eff1e4da700 1 --2- 172.21.15.3:0/2827173101 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff18131b40 0x7eff18121370 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7eff1000cc70 tx=0x7eff1000cf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:24.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.285+0000 7eff0e7fc700 1 -- 172.21.15.3:0/2827173101 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1000ab70 con 0x7eff18131b40 2023-10-03T10:15:24.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.285+0000 7eff0e7fc700 1 -- 172.21.15.3:0/2827173101 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1000acf0 con 0x7eff18131b40 2023-10-03T10:15:24.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.285+0000 7eff0e7fc700 1 -- 172.21.15.3:0/2827173101 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff10007310 con 0x7eff18131b40 2023-10-03T10:15:24.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.285+0000 7eff1ff3d700 1 -- 172.21.15.3:0/2827173101 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff18124880 con 0x7eff18131b40 2023-10-03T10:15:24.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.285+0000 7eff1ff3d700 1 -- 172.21.15.3:0/2827173101 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff18121a90 con 0x7eff18131b40 2023-10-03T10:15:24.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.286+0000 7eff1ff3d700 1 -- 172.21.15.3:0/2827173101 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff18074ea0 con 0x7eff18131b40 2023-10-03T10:15:24.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.286+0000 7eff0e7fc700 1 -- 172.21.15.3:0/2827173101 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7eff10007470 con 0x7eff18131b40 2023-10-03T10:15:24.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.286+0000 7eff0e7fc700 1 -- 172.21.15.3:0/2827173101 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7eff1003f2b0 con 0x7eff18131b40 2023-10-03T10:15:24.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.292+0000 7eff0e7fc700 1 -- 172.21.15.3:0/2827173101 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7eff1000ae60 con 0x7eff18131b40 2023-10-03T10:15:24.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.301+0000 7f680f2a9700 1 Processor -- start 2023-10-03T10:15:24.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.301+0000 7f680f2a9700 1 -- start start 2023-10-03T10:15:24.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.302+0000 7f680f2a9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6808134710 0x7f6808136af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:24.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.302+0000 7f680f2a9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6808137030 0x7f6808139410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:24.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.302+0000 7f680f2a9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6808131510 con 0x7f68081316d0 2023-10-03T10:15:24.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.302+0000 7f680f2a9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6808099e70 con 0x7f6808134710 2023-10-03T10:15:24.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.302+0000 7f680f2a9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6808066680 con 0x7f6808137030 2023-10-03T10:15:24.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.302+0000 7f680d846700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6808137030 0x7f6808139410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:24.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.303+0000 7f680d846700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6808137030 0x7f6808139410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46442/0 (socket says 172.21.15.3:46442) 2023-10-03T10:15:24.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.303+0000 7f680d846700 1 -- 172.21.15.3:0/442499923 learned_addr learned my addr 172.21.15.3:0/442499923 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:24.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.303+0000 7f680c844700 1 --2- 172.21.15.3:0/442499923 >> v2:172.21.15.3:3300/0 conn(0x7f6808134710 0x7f6808136af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:24.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.303+0000 7f680d846700 1 -- 172.21.15.3:0/442499923 >> v2:172.21.15.3:3300/0 conn(0x7f6808134710 msgr2=0x7f6808136af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:24.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.303+0000 7f680d846700 1 --2- 172.21.15.3:0/442499923 >> v2:172.21.15.3:3300/0 conn(0x7f6808134710 0x7f6808136af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.303+0000 7f680d846700 1 -- 172.21.15.3:0/442499923 >> v1:172.21.15.3:6789/0 conn(0x7f68081316d0 legacy=0x7f6808131aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:24.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.303+0000 7f680d846700 1 -- 172.21.15.3:0/442499923 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680809ee00 con 0x7f6808137030 2023-10-03T10:15:24.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.303+0000 7f680d846700 1 --2- 172.21.15.3:0/442499923 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6808137030 0x7f6808139410 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6800009e50 tx=0x7f680000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dd946b0effbb47e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:24.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.303+0000 7f67fffff700 1 -- 172.21.15.3:0/442499923 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680000bb10 con 0x7f6808137030 2023-10-03T10:15:24.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.303+0000 7f67fffff700 1 -- 172.21.15.3:0/442499923 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680000bc70 con 0x7f6808137030 2023-10-03T10:15:24.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.303+0000 7f67fffff700 1 -- 172.21.15.3:0/442499923 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68000104d0 con 0x7f6808137030 2023-10-03T10:15:24.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.304+0000 7f680f2a9700 1 -- 172.21.15.3:0/442499923 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6808137030 msgr2=0x7f6808139410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:24.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.304+0000 7f680f2a9700 1 --2- 172.21.15.3:0/442499923 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6808137030 0x7f6808139410 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6800009e50 tx=0x7f680000b330 comp rx=0 tx=0).stop 2023-10-03T10:15:24.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.304+0000 7f680f2a9700 1 -- 172.21.15.3:0/442499923 shutdown_connections 2023-10-03T10:15:24.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.304+0000 7f680f2a9700 1 --2- 172.21.15.3:0/442499923 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6808137030 0x7f6808139410 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.304+0000 7f680f2a9700 1 --2- 172.21.15.3:0/442499923 >> v2:172.21.15.3:3300/0 conn(0x7f6808134710 0x7f6808136af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.304+0000 7f680f2a9700 1 -- 172.21.15.3:0/442499923 >> 172.21.15.3:0/442499923 conn(0x7f68080a2aa0 msgr2=0x7f68080a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:24.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.305+0000 7f680f2a9700 1 -- 172.21.15.3:0/442499923 shutdown_connections 2023-10-03T10:15:24.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.305+0000 7f680f2a9700 1 -- 172.21.15.3:0/442499923 wait complete. 2023-10-03T10:15:24.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.306+0000 7f680f2a9700 1 Processor -- start 2023-10-03T10:15:24.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.306+0000 7f680f2a9700 1 -- start start 2023-10-03T10:15:24.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.306+0000 7f680f2a9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68081316d0 0x7f680809b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:24.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.306+0000 7f680f2a9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6808134710 0x7f680809bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:24.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.306+0000 7f680f2a9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6808137030 0x7f680809c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:24.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.306+0000 7f680f2a9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6808118f30 con 0x7f68081316d0 2023-10-03T10:15:24.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.306+0000 7f680f2a9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6808119110 con 0x7f6808134710 2023-10-03T10:15:24.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.306+0000 7f680f2a9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f680809a0b0 con 0x7f6808137030 2023-10-03T10:15:24.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.306+0000 7f680c844700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6808134710 0x7f680809bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:24.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.306+0000 7f680c844700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6808134710 0x7f680809bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46450/0 (socket says 172.21.15.3:46450) 2023-10-03T10:15:24.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.306+0000 7f680c844700 1 -- 172.21.15.3:0/716484671 learned_addr learned my addr 172.21.15.3:0/716484671 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:24.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.307+0000 7f680d846700 1 --2- 172.21.15.3:0/716484671 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6808137030 0x7f680809c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:24.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.307+0000 7f680c844700 1 -- 172.21.15.3:0/716484671 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6808137030 msgr2=0x7f680809c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:24.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.307+0000 7f680c844700 1 --2- 172.21.15.3:0/716484671 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6808137030 0x7f680809c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.307+0000 7f680d045700 1 --2- 172.21.15.3:0/716484671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68081316d0 0x7f680809b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:24.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.307+0000 7f680c844700 1 -- 172.21.15.3:0/716484671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68081316d0 msgr2=0x7f680809b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:24.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.307+0000 7f680c844700 1 --2- 172.21.15.3:0/716484671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68081316d0 0x7f680809b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.307+0000 7f680c844700 1 -- 172.21.15.3:0/716484671 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6800009b30 con 0x7f6808134710 2023-10-03T10:15:24.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.307+0000 7f680d846700 1 --2- 172.21.15.3:0/716484671 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6808137030 0x7f680809c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:15:24.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.307+0000 7f680d045700 1 --2- 172.21.15.3:0/716484671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68081316d0 0x7f680809b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:15:24.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.307+0000 7f680c844700 1 --2- 172.21.15.3:0/716484671 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6808134710 0x7f680809bcb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f67f800e9b0 tx=0x7f67f800ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:24.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.307+0000 7f67fdffb700 1 -- 172.21.15.3:0/716484671 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f8009710 con 0x7f6808134710 2023-10-03T10:15:24.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.307+0000 7f67fdffb700 1 -- 172.21.15.3:0/716484671 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f800c850 con 0x7f6808134710 2023-10-03T10:15:24.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.307+0000 7f680f2a9700 1 -- 172.21.15.3:0/716484671 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f680809d510 con 0x7f6808134710 2023-10-03T10:15:24.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.307+0000 7f67fdffb700 1 -- 172.21.15.3:0/716484671 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f800cbe0 con 0x7f6808134710 2023-10-03T10:15:24.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.308+0000 7f680f2a9700 1 -- 172.21.15.3:0/716484671 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6808119b80 con 0x7f6808134710 2023-10-03T10:15:24.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.309+0000 7f67fdffb700 1 -- 172.21.15.3:0/716484671 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f67f800cd40 con 0x7f6808134710 2023-10-03T10:15:24.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.309+0000 7f67fdffb700 1 -- 172.21.15.3:0/716484671 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f67f803f880 con 0x7f6808134710 2023-10-03T10:15:24.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.309+0000 7f680f2a9700 1 -- 172.21.15.3:0/716484671 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6808074ea0 con 0x7f6808134710 2023-10-03T10:15:24.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.318+0000 7f67fdffb700 1 -- 172.21.15.3:0/716484671 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f67f80232f0 con 0x7f6808134710 2023-10-03T10:15:24.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.377+0000 7fa8d741a700 1 -- 172.21.15.3:0/1808660849 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa8d0074ea0 con 0x7fa8d009e000 2023-10-03T10:15:24.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.377+0000 7fa8c5ffb700 1 -- 172.21.15.3:0/1808660849 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa8c0035090 con 0x7fa8d009e000 2023-10-03T10:15:24.378 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:24.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.380+0000 7fa8d741a700 1 -- 172.21.15.3:0/1808660849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d009e000 msgr2=0x7fa8d01a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:24.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.380+0000 7fa8d741a700 1 --2- 172.21.15.3:0/1808660849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d009e000 0x7fa8d01a6400 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0016000 tx=0x7fa8c0004750 comp rx=0 tx=0).stop 2023-10-03T10:15:24.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.380+0000 7fa8d741a700 1 -- 172.21.15.3:0/1808660849 shutdown_connections 2023-10-03T10:15:24.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.380+0000 7fa8d741a700 1 --2- 172.21.15.3:0/1808660849 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa8d009e410 0x7fa8d01a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.380+0000 7fa8d741a700 1 --2- 172.21.15.3:0/1808660849 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa8d009d6f0 0x7fa8d01a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.380+0000 7fa8d741a700 1 --2- 172.21.15.3:0/1808660849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d009e000 0x7fa8d01a6400 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.380+0000 7fa8d741a700 1 -- 172.21.15.3:0/1808660849 >> 172.21.15.3:0/1808660849 conn(0x7fa8d0066f50 msgr2=0x7fa8d01baeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:24.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.380+0000 7fa8d741a700 1 -- 172.21.15.3:0/1808660849 shutdown_connections 2023-10-03T10:15:24.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.380+0000 7fa8d741a700 1 -- 172.21.15.3:0/1808660849 wait complete. 2023-10-03T10:15:24.394 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:24.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.408+0000 7f742e920700 1 -- 172.21.15.3:0/2960323973 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7408005860 con 0x7f742809a3b0 2023-10-03T10:15:24.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.409+0000 7f7424ff9700 1 -- 172.21.15.3:0/2960323973 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f741c015020 con 0x7f742809a3b0 2023-10-03T10:15:24.409 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.411+0000 7f742e920700 1 -- 172.21.15.3:0/2960323973 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f742809a3b0 msgr2=0x7f742811f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.411+0000 7f742e920700 1 --2- 172.21.15.3:0/2960323973 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f742809a3b0 0x7f742811f2c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f741c016000 tx=0x7f741c004750 comp rx=0 tx=0).stop 2023-10-03T10:15:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.411+0000 7f742e920700 1 -- 172.21.15.3:0/2960323973 shutdown_connections 2023-10-03T10:15:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.411+0000 7f742e920700 1 --2- 172.21.15.3:0/2960323973 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7428131810 0x7f742811f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.411+0000 7f742e920700 1 --2- 172.21.15.3:0/2960323973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7428099a40 0x7f742811ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.411+0000 7f742e920700 1 --2- 172.21.15.3:0/2960323973 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f742809a3b0 0x7f742811f2c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.411+0000 7f742e920700 1 -- 172.21.15.3:0/2960323973 >> 172.21.15.3:0/2960323973 conn(0x7f74280a2aa0 msgr2=0x7f7428130140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.411+0000 7f742e920700 1 -- 172.21.15.3:0/2960323973 shutdown_connections 2023-10-03T10:15:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.411+0000 7f742e920700 1 -- 172.21.15.3:0/2960323973 wait complete. 2023-10-03T10:15:24.425 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:24.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.504+0000 7eff1ff3d700 1 -- 172.21.15.3:0/2827173101 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7eff1806c360 con 0x7eff18131b40 2023-10-03T10:15:24.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.504+0000 7eff0e7fc700 1 -- 172.21.15.3:0/2827173101 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7eff10018020 con 0x7eff18131b40 2023-10-03T10:15:24.505 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:24.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.507+0000 7eff1ff3d700 1 -- 172.21.15.3:0/2827173101 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff18131b40 msgr2=0x7eff18121370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:24.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.507+0000 7eff1ff3d700 1 --2- 172.21.15.3:0/2827173101 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff18131b40 0x7eff18121370 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7eff1000cc70 tx=0x7eff1000cf80 comp rx=0 tx=0).stop 2023-10-03T10:15:24.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.507+0000 7eff1ff3d700 1 -- 172.21.15.3:0/2827173101 shutdown_connections 2023-10-03T10:15:24.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.507+0000 7eff1ff3d700 1 --2- 172.21.15.3:0/2827173101 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff18131b40 0x7eff18121370 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.507+0000 7eff1ff3d700 1 --2- 172.21.15.3:0/2827173101 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff1806a770 0x7eff18120810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.507+0000 7eff1ff3d700 1 --2- 172.21.15.3:0/2827173101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff1806b080 0x7eff18120e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.507+0000 7eff1ff3d700 1 -- 172.21.15.3:0/2827173101 >> 172.21.15.3:0/2827173101 conn(0x7eff18066f50 msgr2=0x7eff1806ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:24.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.507+0000 7eff1ff3d700 1 -- 172.21.15.3:0/2827173101 shutdown_connections 2023-10-03T10:15:24.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.507+0000 7eff1ff3d700 1 -- 172.21.15.3:0/2827173101 wait complete. 2023-10-03T10:15:24.520 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:24.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.531+0000 7f680f2a9700 1 -- 172.21.15.3:0/716484671 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f680806b9e0 con 0x7f6808134710 2023-10-03T10:15:24.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.532+0000 7f67fdffb700 1 -- 172.21.15.3:0/716484671 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f67f801bba0 con 0x7f6808134710 2023-10-03T10:15:24.532 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.534+0000 7f680f2a9700 1 -- 172.21.15.3:0/716484671 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6808134710 msgr2=0x7f680809bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.534+0000 7f680f2a9700 1 --2- 172.21.15.3:0/716484671 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6808134710 0x7f680809bcb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f67f800e9b0 tx=0x7f67f800ed70 comp rx=0 tx=0).stop 2023-10-03T10:15:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.534+0000 7f680f2a9700 1 -- 172.21.15.3:0/716484671 shutdown_connections 2023-10-03T10:15:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.534+0000 7f680f2a9700 1 --2- 172.21.15.3:0/716484671 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6808137030 0x7f680809c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.534+0000 7f680f2a9700 1 --2- 172.21.15.3:0/716484671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68081316d0 0x7f680809b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.534+0000 7f680f2a9700 1 --2- 172.21.15.3:0/716484671 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6808134710 0x7f680809bcb0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.534+0000 7f680f2a9700 1 -- 172.21.15.3:0/716484671 >> 172.21.15.3:0/716484671 conn(0x7f68080a2aa0 msgr2=0x7f680809f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.534+0000 7f680f2a9700 1 -- 172.21.15.3:0/716484671 shutdown_connections 2023-10-03T10:15:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:24.534+0000 7f680f2a9700 1 -- 172.21.15.3:0/716484671 wait complete. 2023-10-03T10:15:24.548 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:25.396 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:25.425 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:25.521 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:25.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.544+0000 7fe5aaf5e700 1 Processor -- start 2023-10-03T10:15:25.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.544+0000 7fe5aaf5e700 1 -- start start 2023-10-03T10:15:25.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.545+0000 7fe5aaf5e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe5a4136710 0x7fe5a4138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:25.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.545+0000 7fe5aaf5e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5a4139030 0x7fe5a413b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:25.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.545+0000 7fe5aaf5e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5a406ef30 con 0x7fe5a41338a0 2023-10-03T10:15:25.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.545+0000 7fe5aaf5e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5a406fef0 con 0x7fe5a4136710 2023-10-03T10:15:25.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.545+0000 7fe5aaf5e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a41334b0 con 0x7fe5a4139030 2023-10-03T10:15:25.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.545+0000 7fe5a3fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe5a4136710 0x7fe5a4138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:25.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.545+0000 7fe5a3fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe5a4136710 0x7fe5a4138af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45046/0 (socket says 172.21.15.3:45046) 2023-10-03T10:15:25.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.545+0000 7fe5a3fff700 1 -- 172.21.15.3:0/2864713342 learned_addr learned my addr 172.21.15.3:0/2864713342 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:25.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.545+0000 7fe5a94fb700 1 --2- 172.21.15.3:0/2864713342 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5a4139030 0x7fe5a413b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:25.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.546+0000 7fe5a3fff700 1 -- 172.21.15.3:0/2864713342 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5a4139030 msgr2=0x7fe5a413b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:25.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.546+0000 7fe5a3fff700 1 --2- 172.21.15.3:0/2864713342 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5a4139030 0x7fe5a413b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.546+0000 7fe5a3fff700 1 -- 172.21.15.3:0/2864713342 >> v1:172.21.15.3:6789/0 conn(0x7fe5a41338a0 legacy=0x7fe5a4133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:25.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.546+0000 7fe5a3fff700 1 -- 172.21.15.3:0/2864713342 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a406fc30 con 0x7fe5a4136710 2023-10-03T10:15:25.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.546+0000 7fe5a94fb700 1 --2- 172.21.15.3:0/2864713342 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5a4139030 0x7fe5a413b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.546+0000 7fe5a3fff700 1 --2- 172.21.15.3:0/2864713342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5a4136710 0x7fe5a4138af0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe5900098f0 tx=0x7fe590009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d4764209f14d6a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.546+0000 7fe5a37fe700 1 -- 172.21.15.3:0/2864713342 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe590011830 con 0x7fe5a4136710 2023-10-03T10:15:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.546+0000 7fe5a37fe700 1 -- 172.21.15.3:0/2864713342 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe590011990 con 0x7fe5a4136710 2023-10-03T10:15:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.546+0000 7fe5a37fe700 1 -- 172.21.15.3:0/2864713342 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe590011d20 con 0x7fe5a4136710 2023-10-03T10:15:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.547+0000 7fe5aaf5e700 1 -- 172.21.15.3:0/2864713342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5a4136710 msgr2=0x7fe5a4138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.547+0000 7fe5aaf5e700 1 --2- 172.21.15.3:0/2864713342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5a4136710 0x7fe5a4138af0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe5900098f0 tx=0x7fe590009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.547+0000 7fe5aaf5e700 1 -- 172.21.15.3:0/2864713342 shutdown_connections 2023-10-03T10:15:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.547+0000 7fe5aaf5e700 1 --2- 172.21.15.3:0/2864713342 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5a4139030 0x7fe5a413b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.547+0000 7fe5aaf5e700 1 --2- 172.21.15.3:0/2864713342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5a4136710 0x7fe5a4138af0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.547+0000 7fe5aaf5e700 1 -- 172.21.15.3:0/2864713342 >> 172.21.15.3:0/2864713342 conn(0x7fe5a40a2aa0 msgr2=0x7fe5a40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.547+0000 7fe5aaf5e700 1 -- 172.21.15.3:0/2864713342 shutdown_connections 2023-10-03T10:15:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.547+0000 7fe5aaf5e700 1 -- 172.21.15.3:0/2864713342 wait complete. 2023-10-03T10:15:25.549 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:25.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.549+0000 7fe5aaf5e700 1 Processor -- start 2023-10-03T10:15:25.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.549+0000 7fe5aaf5e700 1 -- start start 2023-10-03T10:15:25.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.550+0000 7fe5aaf5e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5a41338a0 0x7fe5a411ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:25.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.550+0000 7fe5aaf5e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5a4136710 0x7fe5a411b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:25.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.550+0000 7fe5aaf5e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5a4139030 0x7fe5a411b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:25.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.550+0000 7fe5aaf5e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a411cad0 con 0x7fe5a4139030 2023-10-03T10:15:25.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.550+0000 7fe5aaf5e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a411e330 con 0x7fe5a41338a0 2023-10-03T10:15:25.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.550+0000 7fe5aaf5e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a4138b00 con 0x7fe5a4136710 2023-10-03T10:15:25.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.550+0000 7fe5a8cfa700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5a41338a0 0x7fe5a411ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:25.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.550+0000 7fe5a94fb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5a4139030 0x7fe5a411b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:25.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.550+0000 7fe5a8cfa700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5a41338a0 0x7fe5a411ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46458/0 (socket says 172.21.15.3:46458) 2023-10-03T10:15:25.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.550+0000 7fe5a94fb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5a4139030 0x7fe5a411b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45054/0 (socket says 172.21.15.3:45054) 2023-10-03T10:15:25.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.550+0000 7fe5a8cfa700 1 -- 172.21.15.3:0/953999389 learned_addr learned my addr 172.21.15.3:0/953999389 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:25.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.550+0000 7fe5a3fff700 1 --2- 172.21.15.3:0/953999389 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5a4136710 0x7fe5a411b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:25.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.551+0000 7fe5a94fb700 1 -- 172.21.15.3:0/953999389 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5a4136710 msgr2=0x7fe5a411b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:25.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.551+0000 7fe5a94fb700 1 --2- 172.21.15.3:0/953999389 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5a4136710 0x7fe5a411b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.551+0000 7fe5a94fb700 1 -- 172.21.15.3:0/953999389 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5a41338a0 msgr2=0x7fe5a411ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:25.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.551+0000 7fe5a94fb700 1 --2- 172.21.15.3:0/953999389 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5a41338a0 0x7fe5a411ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.551+0000 7fe5a94fb700 1 -- 172.21.15.3:0/953999389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59c009a70 con 0x7fe5a4139030 2023-10-03T10:15:25.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.551+0000 7fe5a3fff700 1 --2- 172.21.15.3:0/953999389 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5a4136710 0x7fe5a411b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:25.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.551+0000 7fe5a8cfa700 1 --2- 172.21.15.3:0/953999389 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5a41338a0 0x7fe5a411ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:25.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.551+0000 7fe5a94fb700 1 --2- 172.21.15.3:0/953999389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5a4139030 0x7fe5a411b8d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe59c00efc0 tx=0x7fe59c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:25.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.551+0000 7fe5a17fa700 1 -- 172.21.15.3:0/953999389 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c00ca90 con 0x7fe5a4139030 2023-10-03T10:15:25.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.551+0000 7fe5a17fa700 1 -- 172.21.15.3:0/953999389 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59c00f040 con 0x7fe5a4139030 2023-10-03T10:15:25.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.551+0000 7fe5aaf5e700 1 -- 172.21.15.3:0/953999389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe590009550 con 0x7fe5a4139030 2023-10-03T10:15:25.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.551+0000 7fe5a17fa700 1 -- 172.21.15.3:0/953999389 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c00ce30 con 0x7fe5a4139030 2023-10-03T10:15:25.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.551+0000 7fe5aaf5e700 1 -- 172.21.15.3:0/953999389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5a411f060 con 0x7fe5a4139030 2023-10-03T10:15:25.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.552+0000 7fe5a17fa700 1 -- 172.21.15.3:0/953999389 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe59c014480 con 0x7fe5a4139030 2023-10-03T10:15:25.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.552+0000 7fe5a17fa700 1 -- 172.21.15.3:0/953999389 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe59c03fcf0 con 0x7fe5a4139030 2023-10-03T10:15:25.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.553+0000 7fe5aaf5e700 1 -- 172.21.15.3:0/953999389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5a4074ea0 con 0x7fe5a4139030 2023-10-03T10:15:25.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.558+0000 7fe5a17fa700 1 -- 172.21.15.3:0/953999389 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe59c012910 con 0x7fe5a4139030 2023-10-03T10:15:25.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.595+0000 7f07e81cd700 1 Processor -- start 2023-10-03T10:15:25.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.595+0000 7f07e81cd700 1 -- start start 2023-10-03T10:15:25.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.595+0000 7f07e81cd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07e009d6f0 0x7f07e009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:25.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.595+0000 7f07e81cd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07e009e000 0x7f07e0130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:25.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.596+0000 7f07e81cd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e009f130 con 0x7f07e009e410 2023-10-03T10:15:25.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.596+0000 7f07e81cd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e009f520 con 0x7f07e009e000 2023-10-03T10:15:25.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.596+0000 7f07e81cd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f07e009e930 con 0x7f07e009d6f0 2023-10-03T10:15:25.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.596+0000 7f07e5768700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07e009e000 0x7f07e0130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:25.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.596+0000 7f07e5768700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07e009e000 0x7f07e0130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46464/0 (socket says 172.21.15.3:46464) 2023-10-03T10:15:25.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.596+0000 7f07e5768700 1 -- 172.21.15.3:0/19842308 learned_addr learned my addr 172.21.15.3:0/19842308 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:25.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.596+0000 7f07e5768700 1 -- 172.21.15.3:0/19842308 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07e009d6f0 msgr2=0x7f07e009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:25.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.596+0000 7f07e5768700 1 --2- 172.21.15.3:0/19842308 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07e009d6f0 0x7f07e009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.596+0000 7f07e5768700 1 -- 172.21.15.3:0/19842308 >> v1:172.21.15.3:6789/0 conn(0x7f07e009e410 legacy=0x7f07e0130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:25.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.596+0000 7f07e5768700 1 -- 172.21.15.3:0/19842308 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e009b2d0 con 0x7f07e009e000 2023-10-03T10:15:25.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.596+0000 7f07e5768700 1 --2- 172.21.15.3:0/19842308 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07e009e000 0x7f07e0130660 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f07d000bbc0 tx=0x7f07d000bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=526d3586b5f6a786 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:25.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.597+0000 7f07e4f67700 1 -- 172.21.15.3:0/19842308 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d000dd50 con 0x7f07e009e000 2023-10-03T10:15:25.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.597+0000 7f07e4f67700 1 -- 172.21.15.3:0/19842308 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d0004030 con 0x7f07e009e000 2023-10-03T10:15:25.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.597+0000 7f07e4f67700 1 -- 172.21.15.3:0/19842308 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d00043a0 con 0x7f07e009e000 2023-10-03T10:15:25.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.597+0000 7f07e81cd700 1 -- 172.21.15.3:0/19842308 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07e009e000 msgr2=0x7f07e0130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:25.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.597+0000 7f07e81cd700 1 --2- 172.21.15.3:0/19842308 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07e009e000 0x7f07e0130660 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f07d000bbc0 tx=0x7f07d000bed0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.597+0000 7f07e81cd700 1 -- 172.21.15.3:0/19842308 shutdown_connections 2023-10-03T10:15:25.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.597+0000 7f07e81cd700 1 --2- 172.21.15.3:0/19842308 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07e009d6f0 0x7f07e009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.597+0000 7f07e81cd700 1 --2- 172.21.15.3:0/19842308 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07e009e000 0x7f07e0130660 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.597+0000 7f07e81cd700 1 -- 172.21.15.3:0/19842308 >> 172.21.15.3:0/19842308 conn(0x7f07e0066f50 msgr2=0x7f07e009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:25.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.598+0000 7f07e81cd700 1 -- 172.21.15.3:0/19842308 shutdown_connections 2023-10-03T10:15:25.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.598+0000 7f07e81cd700 1 -- 172.21.15.3:0/19842308 wait complete. 2023-10-03T10:15:25.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.599+0000 7f07e81cd700 1 Processor -- start 2023-10-03T10:15:25.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.599+0000 7f07e81cd700 1 -- start start 2023-10-03T10:15:25.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.599+0000 7f07e81cd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07e009d6f0 0x7f07e0122500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:25.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.599+0000 7f07e81cd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07e009e000 0x7f07e0122a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:25.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.599+0000 7f07e81cd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07e009e410 0x7f07e0122f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:25.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.599+0000 7f07e81cd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e0125fd0 con 0x7f07e009e000 2023-10-03T10:15:25.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.599+0000 7f07e81cd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e0123940 con 0x7f07e009e410 2023-10-03T10:15:25.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.599+0000 7f07e81cd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f07e01240c0 con 0x7f07e009d6f0 2023-10-03T10:15:25.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.599+0000 7f07e676a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07e009e410 0x7f07e0122f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:25.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.599+0000 7f07e676a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07e009e410 0x7f07e0122f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46470/0 (socket says 172.21.15.3:46470) 2023-10-03T10:15:25.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.599+0000 7f07e676a700 1 -- 172.21.15.3:0/3913007805 learned_addr learned my addr 172.21.15.3:0/3913007805 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:25.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.600+0000 7f07e5768700 1 --2- 172.21.15.3:0/3913007805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07e009e000 0x7f07e0122a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:25.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.600+0000 7f07e5f69700 1 --2- 172.21.15.3:0/3913007805 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07e009d6f0 0x7f07e0122500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:25.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.600+0000 7f07e5768700 1 -- 172.21.15.3:0/3913007805 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07e009d6f0 msgr2=0x7f07e0122500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:25.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.600+0000 7f07e5768700 1 --2- 172.21.15.3:0/3913007805 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07e009d6f0 0x7f07e0122500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.600+0000 7f07e5768700 1 -- 172.21.15.3:0/3913007805 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07e009e410 msgr2=0x7f07e0122f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:25.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.600+0000 7f07e5768700 1 --2- 172.21.15.3:0/3913007805 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07e009e410 0x7f07e0122f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.600+0000 7f07e5768700 1 -- 172.21.15.3:0/3913007805 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d000b7f0 con 0x7f07e009e000 2023-10-03T10:15:25.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.600+0000 7f07e5768700 1 --2- 172.21.15.3:0/3913007805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07e009e000 0x7f07e0122a40 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f07d0017000 tx=0x7f07d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:25.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.601+0000 7f07d67fc700 1 -- 172.21.15.3:0/3913007805 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d00045b0 con 0x7f07e009e000 2023-10-03T10:15:25.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.601+0000 7f07d67fc700 1 -- 172.21.15.3:0/3913007805 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d0010470 con 0x7f07e009e000 2023-10-03T10:15:25.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.601+0000 7f07d67fc700 1 -- 172.21.15.3:0/3913007805 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d0010840 con 0x7f07e009e000 2023-10-03T10:15:25.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.601+0000 7f07e81cd700 1 -- 172.21.15.3:0/3913007805 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e00a1c20 con 0x7f07e009e000 2023-10-03T10:15:25.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.601+0000 7f07e81cd700 1 -- 172.21.15.3:0/3913007805 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e0124870 con 0x7f07e009e000 2023-10-03T10:15:25.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.602+0000 7f07d67fc700 1 -- 172.21.15.3:0/3913007805 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f07d00193f0 con 0x7f07e009e000 2023-10-03T10:15:25.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.603+0000 7f07d67fc700 1 -- 172.21.15.3:0/3913007805 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f07d0041640 con 0x7f07e009e000 2023-10-03T10:15:25.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.603+0000 7f07e81cd700 1 -- 172.21.15.3:0/3913007805 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e0074ea0 con 0x7f07e009e000 2023-10-03T10:15:25.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.608+0000 7f07d67fc700 1 -- 172.21.15.3:0/3913007805 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f07d0018990 con 0x7f07e009e000 2023-10-03T10:15:25.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.680+0000 7f19c41f2700 1 Processor -- start 2023-10-03T10:15:25.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.681+0000 7f19c41f2700 1 -- start start 2023-10-03T10:15:25.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.681+0000 7f19c41f2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19bc09e930 0x7f19bc09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:25.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.681+0000 7f19c41f2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f19bc09f240 0x7f19bc1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:25.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.681+0000 7f19c41f2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19bc070940 con 0x7f19bc09f650 2023-10-03T10:15:25.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.681+0000 7f19c41f2700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19bc06fd60 con 0x7f19bc09f240 2023-10-03T10:15:25.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.681+0000 7f19c41f2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc06fa80 con 0x7f19bc09e930 2023-10-03T10:15:25.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.682+0000 7f19c178d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f19bc09f240 0x7f19bc1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:25.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.682+0000 7f19c178d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f19bc09f240 0x7f19bc1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45078/0 (socket says 172.21.15.3:45078) 2023-10-03T10:15:25.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.682+0000 7f19c178d700 1 -- 172.21.15.3:0/2075249656 learned_addr learned my addr 172.21.15.3:0/2075249656 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:25.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.682+0000 7f19c1f8e700 1 --2- 172.21.15.3:0/2075249656 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19bc09e930 0x7f19bc09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:25.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.682+0000 7f19c178d700 1 -- 172.21.15.3:0/2075249656 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19bc09e930 msgr2=0x7f19bc09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:25.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.682+0000 7f19c178d700 1 --2- 172.21.15.3:0/2075249656 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19bc09e930 0x7f19bc09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.682+0000 7f19c178d700 1 -- 172.21.15.3:0/2075249656 >> v1:172.21.15.3:6789/0 conn(0x7f19bc09f650 legacy=0x7f19bc131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:25.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.682+0000 7f19c178d700 1 -- 172.21.15.3:0/2075249656 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19bc09fb60 con 0x7f19bc09f240 2023-10-03T10:15:25.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.682+0000 7f19c178d700 1 --2- 172.21.15.3:0/2075249656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19bc09f240 0x7f19bc1317f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f19ac0098f0 tx=0x7f19ac009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29970baa50dd064 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:25.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.683+0000 7f19c0f8c700 1 -- 172.21.15.3:0/2075249656 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac004230 con 0x7f19bc09f240 2023-10-03T10:15:25.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.683+0000 7f19c0f8c700 1 -- 172.21.15.3:0/2075249656 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ac004390 con 0x7f19bc09f240 2023-10-03T10:15:25.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.683+0000 7f19c0f8c700 1 -- 172.21.15.3:0/2075249656 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac0036a0 con 0x7f19bc09f240 2023-10-03T10:15:25.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.684+0000 7f19c41f2700 1 -- 172.21.15.3:0/2075249656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19bc09f240 msgr2=0x7f19bc1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:25.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.684+0000 7f19c41f2700 1 --2- 172.21.15.3:0/2075249656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19bc09f240 0x7f19bc1317f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f19ac0098f0 tx=0x7f19ac009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:25.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.684+0000 7f19c41f2700 1 -- 172.21.15.3:0/2075249656 shutdown_connections 2023-10-03T10:15:25.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.684+0000 7f19c41f2700 1 --2- 172.21.15.3:0/2075249656 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19bc09e930 0x7f19bc09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.684+0000 7f19c41f2700 1 --2- 172.21.15.3:0/2075249656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19bc09f240 0x7f19bc1317f0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.684+0000 7f19c41f2700 1 -- 172.21.15.3:0/2075249656 >> 172.21.15.3:0/2075249656 conn(0x7f19bc0a4210 msgr2=0x7f19bc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:25.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.684+0000 7f19c41f2700 1 -- 172.21.15.3:0/2075249656 shutdown_connections 2023-10-03T10:15:25.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.684+0000 7f19c41f2700 1 -- 172.21.15.3:0/2075249656 wait complete. 2023-10-03T10:15:25.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.685+0000 7f19c41f2700 1 Processor -- start 2023-10-03T10:15:25.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.685+0000 7f19c41f2700 1 -- start start 2023-10-03T10:15:25.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.686+0000 7f19c41f2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19bc09e930 0x7f19bc06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:25.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.686+0000 7f19c41f2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19bc09f240 0x7f19bc06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:25.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.686+0000 7f19c41f2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19bc09f650 0x7f19bc06c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:25.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.686+0000 7f19c41f2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc1a99d0 con 0x7f19bc09e930 2023-10-03T10:15:25.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.686+0000 7f19c41f2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc1a8970 con 0x7f19bc09f650 2023-10-03T10:15:25.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.686+0000 7f19c41f2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc0a36e0 con 0x7f19bc09f240 2023-10-03T10:15:25.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.686+0000 7f19c178d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19bc09f240 0x7f19bc06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:25.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.686+0000 7f19c178d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19bc09f240 0x7f19bc06bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43532/0 (socket says 172.21.15.3:43532) 2023-10-03T10:15:25.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.686+0000 7f19c178d700 1 -- 172.21.15.3:0/1434035067 learned_addr learned my addr 172.21.15.3:0/1434035067 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:25.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.686+0000 7f19c278f700 1 --2- 172.21.15.3:0/1434035067 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19bc09f650 0x7f19bc06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:25.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.686+0000 7f19c178d700 1 -- 172.21.15.3:0/1434035067 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19bc09f650 msgr2=0x7f19bc06c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:25.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.686+0000 7f19c178d700 1 --2- 172.21.15.3:0/1434035067 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19bc09f650 0x7f19bc06c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.686+0000 7f19c178d700 1 -- 172.21.15.3:0/1434035067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19bc09e930 msgr2=0x7f19bc06b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:25.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.686+0000 7f19c178d700 1 --2- 172.21.15.3:0/1434035067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19bc09e930 0x7f19bc06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.686+0000 7f19c178d700 1 -- 172.21.15.3:0/1434035067 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ac009550 con 0x7f19bc09f240 2023-10-03T10:15:25.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.686+0000 7f19c278f700 1 --2- 172.21.15.3:0/1434035067 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19bc09f650 0x7f19bc06c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:25.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.686+0000 7f19c178d700 1 --2- 172.21.15.3:0/1434035067 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19bc09f240 0x7f19bc06bdf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f19ac009130 tx=0x7f19ac004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:25.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.687+0000 7f19b27fc700 1 -- 172.21.15.3:0/1434035067 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac003c10 con 0x7f19bc09f240 2023-10-03T10:15:25.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.687+0000 7f19b27fc700 1 -- 172.21.15.3:0/1434035067 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ac01f070 con 0x7f19bc09f240 2023-10-03T10:15:25.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.687+0000 7f19c41f2700 1 -- 172.21.15.3:0/1434035067 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19bc0a0190 con 0x7f19bc09f240 2023-10-03T10:15:25.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.687+0000 7f19b27fc700 1 -- 172.21.15.3:0/1434035067 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac011630 con 0x7f19bc09f240 2023-10-03T10:15:25.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.687+0000 7f19c41f2700 1 -- 172.21.15.3:0/1434035067 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19bc06d480 con 0x7f19bc09f240 2023-10-03T10:15:25.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.688+0000 7f19b27fc700 1 -- 172.21.15.3:0/1434035067 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f19ac021030 con 0x7f19bc09f240 2023-10-03T10:15:25.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.688+0000 7f19b27fc700 1 -- 172.21.15.3:0/1434035067 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f19ac040ae0 con 0x7f19bc09f240 2023-10-03T10:15:25.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.688+0000 7f19c41f2700 1 -- 172.21.15.3:0/1434035067 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19bc074ea0 con 0x7f19bc09f240 2023-10-03T10:15:25.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.696+0000 7f2031deb700 1 Processor -- start 2023-10-03T10:15:25.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.696+0000 7f2031deb700 1 -- start start 2023-10-03T10:15:25.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.696+0000 7f2031deb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f202c12f640 0x7f202c12fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:25.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.696+0000 7f2031deb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f202c12ff50 0x7f202c095070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:25.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.696+0000 7f2031deb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f202c0955b0 0x7f202c095a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:25.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.696+0000 7f2031deb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f202c09eb40 con 0x7f202c12ff50 2023-10-03T10:15:25.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.696+0000 7f2031deb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202c09c580 con 0x7f202c12f640 2023-10-03T10:15:25.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.696+0000 7f2031deb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f202c09b180 con 0x7f202c0955b0 2023-10-03T10:15:25.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.696+0000 7f202affd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f202c12ff50 0x7f202c095070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:25.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.697+0000 7f202affd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f202c12ff50 0x7f202c095070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45084/0 (socket says 172.21.15.3:45084) 2023-10-03T10:15:25.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.697+0000 7f202affd700 1 -- 172.21.15.3:0/1079013162 learned_addr learned my addr 172.21.15.3:0/1079013162 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:25.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.697+0000 7f202bfff700 1 --2- 172.21.15.3:0/1079013162 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f202c0955b0 0x7f202c095a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:25.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.697+0000 7f202b7fe700 1 --2- 172.21.15.3:0/1079013162 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f202c12f640 0x7f202c12fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:25.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.697+0000 7f202affd700 1 -- 172.21.15.3:0/1079013162 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f202c0955b0 msgr2=0x7f202c095a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:25.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.697+0000 7f202affd700 1 --2- 172.21.15.3:0/1079013162 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f202c0955b0 0x7f202c095a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.697+0000 7f202affd700 1 -- 172.21.15.3:0/1079013162 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f202c12f640 msgr2=0x7f202c12fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:25.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.697+0000 7f202affd700 1 --2- 172.21.15.3:0/1079013162 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f202c12f640 0x7f202c12fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.697+0000 7f202affd700 1 -- 172.21.15.3:0/1079013162 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202c09aca0 con 0x7f202c12ff50 2023-10-03T10:15:25.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.697+0000 7f202b7fe700 1 --2- 172.21.15.3:0/1079013162 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f202c12f640 0x7f202c12fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:15:25.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.697+0000 7f202affd700 1 --2- 172.21.15.3:0/1079013162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c12ff50 0x7f202c095070 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f202000a9c0 tx=0x7f202000a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf0ff7d06688819 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:25.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.697+0000 7f202a7fc700 1 -- 172.21.15.3:0/1079013162 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202000ad20 con 0x7f202c12ff50 2023-10-03T10:15:25.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.697+0000 7f202a7fc700 1 -- 172.21.15.3:0/1079013162 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202000ae80 con 0x7f202c12ff50 2023-10-03T10:15:25.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.697+0000 7f202a7fc700 1 -- 172.21.15.3:0/1079013162 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202000cb20 con 0x7f202c12ff50 2023-10-03T10:15:25.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.698+0000 7f19b27fc700 1 -- 172.21.15.3:0/1434035067 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f19ac0239c0 con 0x7f19bc09f240 2023-10-03T10:15:25.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.698+0000 7f2031deb700 1 -- 172.21.15.3:0/1079013162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c12ff50 msgr2=0x7f202c095070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:25.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.698+0000 7f2031deb700 1 --2- 172.21.15.3:0/1079013162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c12ff50 0x7f202c095070 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f202000a9c0 tx=0x7f202000a2d0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.698+0000 7f2031deb700 1 -- 172.21.15.3:0/1079013162 shutdown_connections 2023-10-03T10:15:25.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.698+0000 7f2031deb700 1 --2- 172.21.15.3:0/1079013162 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f202c0955b0 0x7f202c095a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.698+0000 7f2031deb700 1 --2- 172.21.15.3:0/1079013162 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f202c12f640 0x7f202c12fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.698+0000 7f2031deb700 1 --2- 172.21.15.3:0/1079013162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c12ff50 0x7f202c095070 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.698+0000 7f2031deb700 1 -- 172.21.15.3:0/1079013162 >> 172.21.15.3:0/1079013162 conn(0x7f202c0a2360 msgr2=0x7f202c0a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:25.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.698+0000 7f2031deb700 1 -- 172.21.15.3:0/1079013162 shutdown_connections 2023-10-03T10:15:25.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.698+0000 7f2031deb700 1 -- 172.21.15.3:0/1079013162 wait complete. 2023-10-03T10:15:25.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.699+0000 7f2031deb700 1 Processor -- start 2023-10-03T10:15:25.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.699+0000 7f2031deb700 1 -- start start 2023-10-03T10:15:25.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.700+0000 7f2031deb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f202c0955b0 0x7f202c06f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:25.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.700+0000 7f2031deb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f202c12f640 0x7f202c06fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:25.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.700+0000 7f2031deb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c12ff50 0x7f202c070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:25.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.700+0000 7f2031deb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c071940 con 0x7f202c12ff50 2023-10-03T10:15:25.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.700+0000 7f2031deb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202c127680 con 0x7f202c0955b0 2023-10-03T10:15:25.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.700+0000 7f2031deb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f202c09a750 con 0x7f202c12f640 2023-10-03T10:15:25.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.700+0000 7f202affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f202c12f640 0x7f202c06fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:25.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.700+0000 7f202affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f202c12f640 0x7f202c06fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43564/0 (socket says 172.21.15.3:43564) 2023-10-03T10:15:25.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.700+0000 7f202affd700 1 -- 172.21.15.3:0/268958764 learned_addr learned my addr 172.21.15.3:0/268958764 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:25.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.700+0000 7f202bfff700 1 --2- 172.21.15.3:0/268958764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c12ff50 0x7f202c070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:25.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.701+0000 7f202affd700 1 -- 172.21.15.3:0/268958764 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f202c0955b0 msgr2=0x7f202c06f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:25.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.701+0000 7f202affd700 1 --2- 172.21.15.3:0/268958764 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f202c0955b0 0x7f202c06f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.701+0000 7f202affd700 1 -- 172.21.15.3:0/268958764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c12ff50 msgr2=0x7f202c070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:25.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.701+0000 7f202affd700 1 --2- 172.21.15.3:0/268958764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c12ff50 0x7f202c070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.701+0000 7f202affd700 1 -- 172.21.15.3:0/268958764 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202000a5f0 con 0x7f202c12f640 2023-10-03T10:15:25.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.701+0000 7f202affd700 1 --2- 172.21.15.3:0/268958764 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f202c12f640 0x7f202c06fd20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2020000c00 tx=0x7f2020004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:25.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.701+0000 7f2013fff700 1 -- 172.21.15.3:0/268958764 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20200038a0 con 0x7f202c12f640 2023-10-03T10:15:25.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.701+0000 7f2013fff700 1 -- 172.21.15.3:0/268958764 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2020003a00 con 0x7f202c12f640 2023-10-03T10:15:25.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.701+0000 7f2031deb700 1 -- 172.21.15.3:0/268958764 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202c071420 con 0x7f202c12f640 2023-10-03T10:15:25.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.701+0000 7f2013fff700 1 -- 172.21.15.3:0/268958764 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2020003db0 con 0x7f202c12f640 2023-10-03T10:15:25.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.702+0000 7f2013fff700 1 -- 172.21.15.3:0/268958764 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f202001c070 con 0x7f202c12f640 2023-10-03T10:15:25.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.702+0000 7f2031deb700 1 -- 172.21.15.3:0/268958764 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f202c127e60 con 0x7f202c12f640 2023-10-03T10:15:25.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.702+0000 7f2013fff700 1 -- 172.21.15.3:0/268958764 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2020015e20 con 0x7f202c12f640 2023-10-03T10:15:25.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.703+0000 7f2031deb700 1 -- 172.21.15.3:0/268958764 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f202c1277c0 con 0x7f202c12f640 2023-10-03T10:15:25.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.709+0000 7f2013fff700 1 -- 172.21.15.3:0/268958764 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f202c1277c0 con 0x7f202c12f640 2023-10-03T10:15:25.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.811+0000 7fe5aaf5e700 1 -- 172.21.15.3:0/953999389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe5a406b9e0 con 0x7fe5a4139030 2023-10-03T10:15:25.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.812+0000 7fe5a17fa700 1 -- 172.21.15.3:0/953999389 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe59c009cc0 con 0x7fe5a4139030 2023-10-03T10:15:25.812 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:25.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.815+0000 7fe5aaf5e700 1 -- 172.21.15.3:0/953999389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5a4139030 msgr2=0x7fe5a411b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:25.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.815+0000 7fe5aaf5e700 1 --2- 172.21.15.3:0/953999389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5a4139030 0x7fe5a411b8d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe59c00efc0 tx=0x7fe59c00c3a0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.816+0000 7fe5aaf5e700 1 -- 172.21.15.3:0/953999389 shutdown_connections 2023-10-03T10:15:25.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.816+0000 7fe5aaf5e700 1 --2- 172.21.15.3:0/953999389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5a4139030 0x7fe5a411b8d0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.816+0000 7fe5aaf5e700 1 --2- 172.21.15.3:0/953999389 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5a41338a0 0x7fe5a411ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.816+0000 7fe5aaf5e700 1 --2- 172.21.15.3:0/953999389 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5a4136710 0x7fe5a411b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.816+0000 7fe5aaf5e700 1 -- 172.21.15.3:0/953999389 >> 172.21.15.3:0/953999389 conn(0x7fe5a40a2aa0 msgr2=0x7fe5a4137360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:25.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.816+0000 7fe5aaf5e700 1 -- 172.21.15.3:0/953999389 shutdown_connections 2023-10-03T10:15:25.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.816+0000 7fe5aaf5e700 1 -- 172.21.15.3:0/953999389 wait complete. 2023-10-03T10:15:25.830 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:25.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.844+0000 7f07e81cd700 1 -- 172.21.15.3:0/3913007805 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f07e009c700 con 0x7f07e009e000 2023-10-03T10:15:25.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.844+0000 7f07d67fc700 1 -- 172.21.15.3:0/3913007805 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f07d0006d80 con 0x7f07e009e000 2023-10-03T10:15:25.844 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:25.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.846+0000 7f07e81cd700 1 -- 172.21.15.3:0/3913007805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07e009e000 msgr2=0x7f07e0122a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:25.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.846+0000 7f07e81cd700 1 --2- 172.21.15.3:0/3913007805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07e009e000 0x7f07e0122a40 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f07d0017000 tx=0x7f07d0004750 comp rx=0 tx=0).stop 2023-10-03T10:15:25.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.846+0000 7f07e81cd700 1 -- 172.21.15.3:0/3913007805 shutdown_connections 2023-10-03T10:15:25.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.846+0000 7f07e81cd700 1 --2- 172.21.15.3:0/3913007805 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f07e009e410 0x7f07e0122f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.846+0000 7f07e81cd700 1 --2- 172.21.15.3:0/3913007805 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f07e009d6f0 0x7f07e0122500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.847+0000 7f07e81cd700 1 --2- 172.21.15.3:0/3913007805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07e009e000 0x7f07e0122a40 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.847+0000 7f07e81cd700 1 -- 172.21.15.3:0/3913007805 >> 172.21.15.3:0/3913007805 conn(0x7f07e0066f50 msgr2=0x7f07e00710c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:25.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.847+0000 7f07e81cd700 1 -- 172.21.15.3:0/3913007805 shutdown_connections 2023-10-03T10:15:25.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.847+0000 7f07e81cd700 1 -- 172.21.15.3:0/3913007805 wait complete. 2023-10-03T10:15:25.861 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:25.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.933+0000 7f19c41f2700 1 -- 172.21.15.3:0/1434035067 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f19bc074ea0 con 0x7f19bc09f240 2023-10-03T10:15:25.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.933+0000 7f2031deb700 1 -- 172.21.15.3:0/268958764 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f202c1277c0 con 0x7f202c12f640 2023-10-03T10:15:25.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.933+0000 7f19b27fc700 1 -- 172.21.15.3:0/1434035067 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f19ac01b440 con 0x7f19bc09f240 2023-10-03T10:15:25.934 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:25.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.933+0000 7f2013fff700 1 -- 172.21.15.3:0/268958764 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f202c1277c0 con 0x7f202c12f640 2023-10-03T10:15:25.934 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:25.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.935+0000 7f19c41f2700 1 -- 172.21.15.3:0/1434035067 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19bc09f240 msgr2=0x7f19bc06bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:25.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.935+0000 7f19c41f2700 1 --2- 172.21.15.3:0/1434035067 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19bc09f240 0x7f19bc06bdf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f19ac009130 tx=0x7f19ac004750 comp rx=0 tx=0).stop 2023-10-03T10:15:25.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.935+0000 7f2031deb700 1 -- 172.21.15.3:0/268958764 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f202c12f640 msgr2=0x7f202c06fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:25.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.935+0000 7f2031deb700 1 --2- 172.21.15.3:0/268958764 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f202c12f640 0x7f202c06fd20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2020000c00 tx=0x7f2020004750 comp rx=0 tx=0).stop 2023-10-03T10:15:25.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.936+0000 7f2031deb700 1 -- 172.21.15.3:0/268958764 shutdown_connections 2023-10-03T10:15:25.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.936+0000 7f2031deb700 1 --2- 172.21.15.3:0/268958764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c12ff50 0x7f202c070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.936+0000 7f2031deb700 1 --2- 172.21.15.3:0/268958764 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f202c0955b0 0x7f202c06f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.936+0000 7f2031deb700 1 --2- 172.21.15.3:0/268958764 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f202c12f640 0x7f202c06fd20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.936+0000 7f19c41f2700 1 -- 172.21.15.3:0/1434035067 shutdown_connections 2023-10-03T10:15:25.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.936+0000 7f19c41f2700 1 --2- 172.21.15.3:0/1434035067 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19bc09f650 0x7f19bc06c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.936+0000 7f2031deb700 1 -- 172.21.15.3:0/268958764 >> 172.21.15.3:0/268958764 conn(0x7f202c0a2360 msgr2=0x7f202c12eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:25.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.936+0000 7f19c41f2700 1 --2- 172.21.15.3:0/1434035067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19bc09e930 0x7f19bc06b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.936+0000 7f19c41f2700 1 --2- 172.21.15.3:0/1434035067 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19bc09f240 0x7f19bc06bdf0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:25.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.936+0000 7f19c41f2700 1 -- 172.21.15.3:0/1434035067 >> 172.21.15.3:0/1434035067 conn(0x7f19bc0a4210 msgr2=0x7f19bc1b8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:25.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.936+0000 7f19c41f2700 1 -- 172.21.15.3:0/1434035067 shutdown_connections 2023-10-03T10:15:25.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.936+0000 7f19c41f2700 1 -- 172.21.15.3:0/1434035067 wait complete. 2023-10-03T10:15:25.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.936+0000 7f2031deb700 1 -- 172.21.15.3:0/268958764 shutdown_connections 2023-10-03T10:15:25.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:25.936+0000 7f2031deb700 1 -- 172.21.15.3:0/268958764 wait complete. 2023-10-03T10:15:25.951 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:25.951 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:26.832 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:26.862 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:26.952 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:26.952 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:26.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.991+0000 7f8a0dcac700 1 Processor -- start 2023-10-03T10:15:26.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.991+0000 7f8a0dcac700 1 -- start start 2023-10-03T10:15:26.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.992+0000 7f8a0dcac700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8a081316d0 0x7f8a08131aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:26.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.992+0000 7f8a0dcac700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a0806ec60 0x7f8a08139740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:26.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.992+0000 7f8a0dcac700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a08131510 con 0x7f8a08131fe0 2023-10-03T10:15:26.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.992+0000 7f8a0dcac700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a08099e70 con 0x7f8a081316d0 2023-10-03T10:15:26.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.992+0000 7f8a0dcac700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a08066680 con 0x7f8a0806ec60 2023-10-03T10:15:26.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.992+0000 7f8a07fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a0806ec60 0x7f8a08139740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:26.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.992+0000 7f8a07fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a0806ec60 0x7f8a08139740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46508/0 (socket says 172.21.15.3:46508) 2023-10-03T10:15:26.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.992+0000 7f8a07fff700 1 -- 172.21.15.3:0/3173757773 learned_addr learned my addr 172.21.15.3:0/3173757773 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:26.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.992+0000 7f8a077fe700 1 --2- 172.21.15.3:0/3173757773 >> v2:172.21.15.3:3300/0 conn(0x7f8a081316d0 0x7f8a08131aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:26.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.992+0000 7f8a07fff700 1 -- 172.21.15.3:0/3173757773 >> v2:172.21.15.3:3300/0 conn(0x7f8a081316d0 msgr2=0x7f8a08131aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:26.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.992+0000 7f8a07fff700 1 --2- 172.21.15.3:0/3173757773 >> v2:172.21.15.3:3300/0 conn(0x7f8a081316d0 0x7f8a08131aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:26.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.992+0000 7f8a07fff700 1 -- 172.21.15.3:0/3173757773 >> v1:172.21.15.3:6789/0 conn(0x7f8a08131fe0 legacy=0x7f8a0806e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:26.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.992+0000 7f8a07fff700 1 -- 172.21.15.3:0/3173757773 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0809ee00 con 0x7f8a0806ec60 2023-10-03T10:15:26.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.993+0000 7f8a07fff700 1 --2- 172.21.15.3:0/3173757773 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a0806ec60 0x7f8a08139740 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8a00009e50 tx=0x7f8a0000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78d83f4196e49ee server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:26.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.993+0000 7f8a067fc700 1 -- 172.21.15.3:0/3173757773 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0000e070 con 0x7f8a0806ec60 2023-10-03T10:15:26.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.993+0000 7f8a067fc700 1 -- 172.21.15.3:0/3173757773 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0000b910 con 0x7f8a0806ec60 2023-10-03T10:15:26.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.993+0000 7f8a067fc700 1 -- 172.21.15.3:0/3173757773 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0000bcc0 con 0x7f8a0806ec60 2023-10-03T10:15:26.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.994+0000 7f8a0dcac700 1 -- 172.21.15.3:0/3173757773 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a0806ec60 msgr2=0x7f8a08139740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:26.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.994+0000 7f8a0dcac700 1 --2- 172.21.15.3:0/3173757773 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a0806ec60 0x7f8a08139740 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8a00009e50 tx=0x7f8a0000b330 comp rx=0 tx=0).stop 2023-10-03T10:15:26.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.994+0000 7f8a0dcac700 1 -- 172.21.15.3:0/3173757773 shutdown_connections 2023-10-03T10:15:26.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.994+0000 7f8a0dcac700 1 --2- 172.21.15.3:0/3173757773 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a0806ec60 0x7f8a08139740 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:26.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.994+0000 7f8a0dcac700 1 --2- 172.21.15.3:0/3173757773 >> v2:172.21.15.3:3300/0 conn(0x7f8a081316d0 0x7f8a08131aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:26.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.994+0000 7f8a0dcac700 1 -- 172.21.15.3:0/3173757773 >> 172.21.15.3:0/3173757773 conn(0x7f8a080a2aa0 msgr2=0x7f8a080a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:26.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.994+0000 7f8a0dcac700 1 -- 172.21.15.3:0/3173757773 shutdown_connections 2023-10-03T10:15:26.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.994+0000 7f8a0dcac700 1 -- 172.21.15.3:0/3173757773 wait complete. 2023-10-03T10:15:26.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.995+0000 7f8a0dcac700 1 Processor -- start 2023-10-03T10:15:26.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.995+0000 7f8a0dcac700 1 -- start start 2023-10-03T10:15:26.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.996+0000 7f8a0dcac700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a0806ec60 0x7f8a08127f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:26.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.996+0000 7f8a0dcac700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a081316d0 0x7f8a08128440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:26.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.996+0000 7f8a0dcac700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a08131fe0 0x7f8a081c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:26.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.996+0000 7f8a0dcac700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a081293f0 con 0x7f8a08131fe0 2023-10-03T10:15:26.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.996+0000 7f8a0dcac700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0812a850 con 0x7f8a081316d0 2023-10-03T10:15:26.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.996+0000 7f8a0dcac700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a08127ab0 con 0x7f8a0806ec60 2023-10-03T10:15:26.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.996+0000 7f8a06ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a081316d0 0x7f8a08128440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:26.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.996+0000 7f8a077fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a0806ec60 0x7f8a08127f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:26.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.996+0000 7f8a06ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a081316d0 0x7f8a08128440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46518/0 (socket says 172.21.15.3:46518) 2023-10-03T10:15:26.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.996+0000 7f8a06ffd700 1 -- 172.21.15.3:0/1222760579 learned_addr learned my addr 172.21.15.3:0/1222760579 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:26.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.996+0000 7f8a07fff700 1 --2- 172.21.15.3:0/1222760579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a08131fe0 0x7f8a081c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:26.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.996+0000 7f8a06ffd700 1 -- 172.21.15.3:0/1222760579 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a0806ec60 msgr2=0x7f8a08127f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:26.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.996+0000 7f8a06ffd700 1 --2- 172.21.15.3:0/1222760579 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a0806ec60 0x7f8a08127f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:26.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.996+0000 7f8a06ffd700 1 -- 172.21.15.3:0/1222760579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a08131fe0 msgr2=0x7f8a081c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:26.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.996+0000 7f8a06ffd700 1 --2- 172.21.15.3:0/1222760579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a08131fe0 0x7f8a081c80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:26.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.996+0000 7f8a06ffd700 1 -- 172.21.15.3:0/1222760579 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a00009b30 con 0x7f8a081316d0 2023-10-03T10:15:26.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.997+0000 7f8a06ffd700 1 --2- 172.21.15.3:0/1222760579 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a081316d0 0x7f8a08128440 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00c4e0 tx=0x7f89fc00c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:26.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.997+0000 7f89effff700 1 -- 172.21.15.3:0/1222760579 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc0052d0 con 0x7f8a081316d0 2023-10-03T10:15:26.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.997+0000 7f89effff700 1 -- 172.21.15.3:0/1222760579 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89fc005430 con 0x7f8a081316d0 2023-10-03T10:15:26.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.997+0000 7f8a0dcac700 1 -- 172.21.15.3:0/1222760579 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0812aaa0 con 0x7f8a081316d0 2023-10-03T10:15:27.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.997+0000 7f89effff700 1 -- 172.21.15.3:0/1222760579 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc0038c0 con 0x7f8a081316d0 2023-10-03T10:15:27.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.997+0000 7f8a0dcac700 1 -- 172.21.15.3:0/1222760579 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a0812ad80 con 0x7f8a081316d0 2023-10-03T10:15:27.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.998+0000 7f8a0dcac700 1 -- 172.21.15.3:0/1222760579 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a08074ea0 con 0x7f8a081316d0 2023-10-03T10:15:27.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.999+0000 7f89effff700 1 -- 172.21.15.3:0/1222760579 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f89fc003a20 con 0x7f8a081316d0 2023-10-03T10:15:27.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:26.999+0000 7f89effff700 1 -- 172.21.15.3:0/1222760579 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f89fc03ef10 con 0x7f8a081316d0 2023-10-03T10:15:27.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.008+0000 7f89effff700 1 -- 172.21.15.3:0/1222760579 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f89fc018260 con 0x7f8a081316d0 2023-10-03T10:15:27.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.027+0000 7fc9c61c5700 1 Processor -- start 2023-10-03T10:15:27.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.027+0000 7fc9c61c5700 1 -- start start 2023-10-03T10:15:27.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.028+0000 7fc9c61c5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9c0130670 0x7fc9c0130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:27.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.028+0000 7fc9c61c5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc9c00701a0 0x7fc9c0138540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:27.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.028+0000 7fc9c61c5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9c0130260 con 0x7fc9c006f680 2023-10-03T10:15:27.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.028+0000 7fc9c61c5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9c0071ff0 con 0x7fc9c00701a0 2023-10-03T10:15:27.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.028+0000 7fc9c61c5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c009ecf0 con 0x7fc9c0130670 2023-10-03T10:15:27.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.028+0000 7fc9bf7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9c0130670 0x7fc9c0130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:27.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.028+0000 7fc9bf7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9c0130670 0x7fc9c0130a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43594/0 (socket says 172.21.15.3:43594) 2023-10-03T10:15:27.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.028+0000 7fc9bf7fe700 1 -- 172.21.15.3:0/857242410 learned_addr learned my addr 172.21.15.3:0/857242410 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:27.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.029+0000 7fc9bffff700 1 --2- 172.21.15.3:0/857242410 >> v2:172.21.15.3:3300/0 conn(0x7fc9c00701a0 0x7fc9c0138540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:27.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.029+0000 7fc9bf7fe700 1 -- 172.21.15.3:0/857242410 >> v2:172.21.15.3:3300/0 conn(0x7fc9c00701a0 msgr2=0x7fc9c0138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:27.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.029+0000 7fc9bf7fe700 1 --2- 172.21.15.3:0/857242410 >> v2:172.21.15.3:3300/0 conn(0x7fc9c00701a0 0x7fc9c0138540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.029+0000 7fc9bf7fe700 1 -- 172.21.15.3:0/857242410 >> v1:172.21.15.3:6789/0 conn(0x7fc9c006f680 legacy=0x7fc9c006fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:27.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.029+0000 7fc9bf7fe700 1 -- 172.21.15.3:0/857242410 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c0070e70 con 0x7fc9c0130670 2023-10-03T10:15:27.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.029+0000 7fc9bffff700 1 --2- 172.21.15.3:0/857242410 >> v2:172.21.15.3:3300/0 conn(0x7fc9c00701a0 0x7fc9c0138540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:15:27.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.029+0000 7fc9bf7fe700 1 --2- 172.21.15.3:0/857242410 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9c0130670 0x7fc9c0130a40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc9b00098f0 tx=0x7fc9b0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7350ca83ad2fd63 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:27.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.029+0000 7fc9be7fc700 1 -- 172.21.15.3:0/857242410 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b0004230 con 0x7fc9c0130670 2023-10-03T10:15:27.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.029+0000 7fc9be7fc700 1 -- 172.21.15.3:0/857242410 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b0004390 con 0x7fc9c0130670 2023-10-03T10:15:27.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.029+0000 7fc9be7fc700 1 -- 172.21.15.3:0/857242410 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b00036a0 con 0x7fc9c0130670 2023-10-03T10:15:27.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.030+0000 7fc9c61c5700 1 -- 172.21.15.3:0/857242410 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9c0130670 msgr2=0x7fc9c0130a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:27.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.030+0000 7fc9c61c5700 1 --2- 172.21.15.3:0/857242410 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9c0130670 0x7fc9c0130a40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc9b00098f0 tx=0x7fc9b0009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:27.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.030+0000 7fc9c61c5700 1 -- 172.21.15.3:0/857242410 shutdown_connections 2023-10-03T10:15:27.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.030+0000 7fc9c61c5700 1 --2- 172.21.15.3:0/857242410 >> v2:172.21.15.3:3300/0 conn(0x7fc9c00701a0 0x7fc9c0138540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.030+0000 7fc9c61c5700 1 --2- 172.21.15.3:0/857242410 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9c0130670 0x7fc9c0130a40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.030+0000 7fc9c61c5700 1 -- 172.21.15.3:0/857242410 >> 172.21.15.3:0/857242410 conn(0x7fc9c00a2aa0 msgr2=0x7fc9c00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:27.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.030+0000 7fc9c61c5700 1 -- 172.21.15.3:0/857242410 shutdown_connections 2023-10-03T10:15:27.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.030+0000 7fc9c61c5700 1 -- 172.21.15.3:0/857242410 wait complete. 2023-10-03T10:15:27.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.032+0000 7fc9c61c5700 1 Processor -- start 2023-10-03T10:15:27.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.032+0000 7fc9c61c5700 1 -- start start 2023-10-03T10:15:27.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.033+0000 7fc9c61c5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9c006f680 0x7fc9c01349c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:27.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.033+0000 7fc9c61c5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9c00701a0 0x7fc9c01c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:27.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.033+0000 7fc9c61c5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9c0130670 0x7fc9c01c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:27.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.033+0000 7fc9c61c5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c0136290 con 0x7fc9c0130670 2023-10-03T10:15:27.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.033+0000 7fc9c61c5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c0136e90 con 0x7fc9c00701a0 2023-10-03T10:15:27.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.033+0000 7fc9c61c5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c0136630 con 0x7fc9c006f680 2023-10-03T10:15:27.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.033+0000 7fc9bffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9c0130670 0x7fc9c01c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:27.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.033+0000 7fc9bffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9c0130670 0x7fc9c01c8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45140/0 (socket says 172.21.15.3:45140) 2023-10-03T10:15:27.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.033+0000 7fc9bffff700 1 -- 172.21.15.3:0/2304715836 learned_addr learned my addr 172.21.15.3:0/2304715836 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:27.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.033+0000 7fc9bf7fe700 1 --2- 172.21.15.3:0/2304715836 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9c006f680 0x7fc9c01349c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:27.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.033+0000 7fc9bf7fe700 1 -- 172.21.15.3:0/2304715836 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9c00701a0 msgr2=0x7fc9c01c8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:27.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.033+0000 7fc9bf7fe700 1 --2- 172.21.15.3:0/2304715836 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9c00701a0 0x7fc9c01c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.033+0000 7fc9bf7fe700 1 -- 172.21.15.3:0/2304715836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9c0130670 msgr2=0x7fc9c01c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:27.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.033+0000 7fc9bf7fe700 1 --2- 172.21.15.3:0/2304715836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9c0130670 0x7fc9c01c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.033+0000 7fc9bf7fe700 1 -- 172.21.15.3:0/2304715836 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b0009550 con 0x7fc9c006f680 2023-10-03T10:15:27.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.033+0000 7fc9bffff700 1 --2- 172.21.15.3:0/2304715836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9c0130670 0x7fc9c01c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:27.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.034+0000 7fc9bf7fe700 1 --2- 172.21.15.3:0/2304715836 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9c006f680 0x7fc9c01349c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0015000 tx=0x7fc9b0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:27.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.034+0000 7fc9a7fff700 1 -- 172.21.15.3:0/2304715836 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b0003d00 con 0x7fc9c006f680 2023-10-03T10:15:27.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.034+0000 7fc9a7fff700 1 -- 172.21.15.3:0/2304715836 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b001f070 con 0x7fc9c006f680 2023-10-03T10:15:27.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.034+0000 7fc9a7fff700 1 -- 172.21.15.3:0/2304715836 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b000b8a0 con 0x7fc9c006f680 2023-10-03T10:15:27.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.034+0000 7fc9c61c5700 1 -- 172.21.15.3:0/2304715836 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9c0071820 con 0x7fc9c006f680 2023-10-03T10:15:27.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.034+0000 7fc9c61c5700 1 -- 172.21.15.3:0/2304715836 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9c0135520 con 0x7fc9c006f680 2023-10-03T10:15:27.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.035+0000 7fc9a7fff700 1 -- 172.21.15.3:0/2304715836 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc9b0021030 con 0x7fc9c006f680 2023-10-03T10:15:27.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.035+0000 7fc9a7fff700 1 -- 172.21.15.3:0/2304715836 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc9b0040f40 con 0x7fc9c006f680 2023-10-03T10:15:27.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.035+0000 7fc9c61c5700 1 -- 172.21.15.3:0/2304715836 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9c0074ea0 con 0x7fc9c006f680 2023-10-03T10:15:27.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.045+0000 7fc9a7fff700 1 -- 172.21.15.3:0/2304715836 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc9b001a440 con 0x7fc9c006f680 2023-10-03T10:15:27.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.107+0000 7f6f33a7f700 1 Processor -- start 2023-10-03T10:15:27.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.108+0000 7f6f33a7f700 1 -- start start 2023-10-03T10:15:27.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.108+0000 7f6f33a7f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6f2c09d6f0 0x7f6f2c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:27.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.108+0000 7f6f33a7f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f2c09e000 0x7f6f2c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:27.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.108+0000 7f6f33a7f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f2c09e410 0x7f6f2c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:27.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.108+0000 7f6f33a7f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f2c09f520 con 0x7f6f2c09d6f0 2023-10-03T10:15:27.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.108+0000 7f6f33a7f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c09f130 con 0x7f6f2c09e000 2023-10-03T10:15:27.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.108+0000 7f6f33a7f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c09e930 con 0x7f6f2c09e410 2023-10-03T10:15:27.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.109+0000 7f6f3101a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f2c09e000 0x7f6f2c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:27.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.109+0000 7f6f3181b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6f2c09d6f0 0x7f6f2c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:27.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.109+0000 7f6f3101a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f2c09e000 0x7f6f2c130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46540/0 (socket says 172.21.15.3:46540) 2023-10-03T10:15:27.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.109+0000 7f6f3101a700 1 -- 172.21.15.3:0/2784948012 learned_addr learned my addr 172.21.15.3:0/2784948012 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:27.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.109+0000 7f6f3201c700 1 --2- 172.21.15.3:0/2784948012 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f2c09e410 0x7f6f2c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:27.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.109+0000 7f6f3101a700 1 -- 172.21.15.3:0/2784948012 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f2c09e410 msgr2=0x7f6f2c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:27.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.109+0000 7f6f3101a700 1 --2- 172.21.15.3:0/2784948012 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f2c09e410 0x7f6f2c130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.109+0000 7f6f3101a700 1 -- 172.21.15.3:0/2784948012 >> v2:172.21.15.3:3300/0 conn(0x7f6f2c09d6f0 msgr2=0x7f6f2c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:27.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.109+0000 7f6f3101a700 1 --2- 172.21.15.3:0/2784948012 >> v2:172.21.15.3:3300/0 conn(0x7f6f2c09d6f0 0x7f6f2c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.109+0000 7f6f3101a700 1 -- 172.21.15.3:0/2784948012 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f2c09b2d0 con 0x7f6f2c09e000 2023-10-03T10:15:27.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.109+0000 7f6f3201c700 1 --2- 172.21.15.3:0/2784948012 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f2c09e410 0x7f6f2c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:27.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.109+0000 7f6f3101a700 1 --2- 172.21.15.3:0/2784948012 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f2c09e000 0x7f6f2c130660 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00b9a0 tx=0x7f6f1c00bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ec481ba73315737 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:27.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.110+0000 7f6f30819700 1 -- 172.21.15.3:0/2784948012 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c00caa0 con 0x7f6f2c09e000 2023-10-03T10:15:27.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.110+0000 7f6f30819700 1 -- 172.21.15.3:0/2784948012 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1c00cc00 con 0x7f6f2c09e000 2023-10-03T10:15:27.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.110+0000 7f6f30819700 1 -- 172.21.15.3:0/2784948012 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c00f460 con 0x7f6f2c09e000 2023-10-03T10:15:27.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.110+0000 7efd935ec700 1 Processor -- start 2023-10-03T10:15:27.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.110+0000 7f6f33a7f700 1 -- 172.21.15.3:0/2784948012 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f2c09e000 msgr2=0x7f6f2c130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:27.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.110+0000 7f6f33a7f700 1 --2- 172.21.15.3:0/2784948012 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f2c09e000 0x7f6f2c130660 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00b9a0 tx=0x7f6f1c00bcb0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.110+0000 7efd935ec700 1 -- start start 2023-10-03T10:15:27.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.111+0000 7f6f33a7f700 1 -- 172.21.15.3:0/2784948012 shutdown_connections 2023-10-03T10:15:27.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.111+0000 7f6f33a7f700 1 --2- 172.21.15.3:0/2784948012 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f2c09e410 0x7f6f2c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.111+0000 7f6f33a7f700 1 --2- 172.21.15.3:0/2784948012 >> v2:172.21.15.3:3300/0 conn(0x7f6f2c09d6f0 0x7f6f2c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.111+0000 7f6f33a7f700 1 --2- 172.21.15.3:0/2784948012 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f2c09e000 0x7f6f2c130660 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.111+0000 7f6f33a7f700 1 -- 172.21.15.3:0/2784948012 >> 172.21.15.3:0/2784948012 conn(0x7f6f2c066f50 msgr2=0x7f6f2c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:27.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.111+0000 7f6f33a7f700 1 -- 172.21.15.3:0/2784948012 shutdown_connections 2023-10-03T10:15:27.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.111+0000 7f6f33a7f700 1 -- 172.21.15.3:0/2784948012 wait complete. 2023-10-03T10:15:27.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.111+0000 7efd935ec700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efd8c131810 0x7efd8c131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:27.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.111+0000 7efd935ec700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd8c099a40 0x7efd8c099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:27.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.111+0000 7efd935ec700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd8c09a3b0 0x7efd8c138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:27.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.111+0000 7efd935ec700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd8c1313c0 con 0x7efd8c131810 2023-10-03T10:15:27.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.111+0000 7efd935ec700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c06e8c0 con 0x7efd8c09a3b0 2023-10-03T10:15:27.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.111+0000 7efd935ec700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c071d20 con 0x7efd8c099a40 2023-10-03T10:15:27.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.112+0000 7efd90b87700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd8c099a40 0x7efd8c099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:27.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.112+0000 7efd90b87700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd8c099a40 0x7efd8c099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43608/0 (socket says 172.21.15.3:43608) 2023-10-03T10:15:27.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.112+0000 7efd90b87700 1 -- 172.21.15.3:0/450594071 learned_addr learned my addr 172.21.15.3:0/450594071 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:27.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.112+0000 7f6f33a7f700 1 Processor -- start 2023-10-03T10:15:27.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.112+0000 7efd91388700 1 --2- 172.21.15.3:0/450594071 >> v2:172.21.15.3:3300/0 conn(0x7efd8c131810 0x7efd8c131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:27.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.112+0000 7efd90b87700 1 -- 172.21.15.3:0/450594071 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd8c09a3b0 msgr2=0x7efd8c138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:27.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.112+0000 7efd90b87700 1 --2- 172.21.15.3:0/450594071 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd8c09a3b0 0x7efd8c138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.112+0000 7efd90b87700 1 -- 172.21.15.3:0/450594071 >> v2:172.21.15.3:3300/0 conn(0x7efd8c131810 msgr2=0x7efd8c131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:27.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.112+0000 7efd90b87700 1 --2- 172.21.15.3:0/450594071 >> v2:172.21.15.3:3300/0 conn(0x7efd8c131810 0x7efd8c131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.112+0000 7efd90b87700 1 -- 172.21.15.3:0/450594071 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8c0721a0 con 0x7efd8c099a40 2023-10-03T10:15:27.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.112+0000 7efd91388700 1 --2- 172.21.15.3:0/450594071 >> v2:172.21.15.3:3300/0 conn(0x7efd8c131810 0x7efd8c131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:15:27.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.112+0000 7efd90b87700 1 --2- 172.21.15.3:0/450594071 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd8c099a40 0x7efd8c099e70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efd7c0098f0 tx=0x7efd7c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f78db085104fb7b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:27.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.112+0000 7efd83fff700 1 -- 172.21.15.3:0/450594071 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c00b8a0 con 0x7efd8c099a40 2023-10-03T10:15:27.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.112+0000 7f6f33a7f700 1 -- start start 2023-10-03T10:15:27.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.112+0000 7efd83fff700 1 -- 172.21.15.3:0/450594071 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7c00ba00 con 0x7efd8c099a40 2023-10-03T10:15:27.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.113+0000 7efd83fff700 1 -- 172.21.15.3:0/450594071 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c00bd90 con 0x7efd8c099a40 2023-10-03T10:15:27.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.113+0000 7f6f33a7f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f2c09d6f0 0x7f6f2c1cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:27.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.113+0000 7f6f33a7f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f2c09e000 0x7f6f2c1ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:27.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.113+0000 7f6f33a7f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f2c09e410 0x7f6f2c1cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:27.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.113+0000 7f6f33a7f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c128480 con 0x7f6f2c09e410 2023-10-03T10:15:27.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.113+0000 7f6f33a7f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c128ad0 con 0x7f6f2c09d6f0 2023-10-03T10:15:27.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.113+0000 7f6f33a7f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c126cf0 con 0x7f6f2c09e000 2023-10-03T10:15:27.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.113+0000 7f6f3101a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f2c09e000 0x7f6f2c1ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:27.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.113+0000 7f6f3101a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f2c09e000 0x7f6f2c1ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43612/0 (socket says 172.21.15.3:43612) 2023-10-03T10:15:27.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.113+0000 7f6f3101a700 1 -- 172.21.15.3:0/842815651 learned_addr learned my addr 172.21.15.3:0/842815651 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:27.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.113+0000 7f6f3201c700 1 --2- 172.21.15.3:0/842815651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f2c09e410 0x7f6f2c1cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:27.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.113+0000 7efd935ec700 1 -- 172.21.15.3:0/450594071 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd8c099a40 msgr2=0x7efd8c099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:27.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.113+0000 7efd935ec700 1 --2- 172.21.15.3:0/450594071 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd8c099a40 0x7efd8c099e70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efd7c0098f0 tx=0x7efd7c009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:27.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.113+0000 7f6f3181b700 1 --2- 172.21.15.3:0/842815651 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f2c09d6f0 0x7f6f2c1cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:27.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.113+0000 7f6f3101a700 1 -- 172.21.15.3:0/842815651 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f2c09d6f0 msgr2=0x7f6f2c1cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:27.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.113+0000 7f6f3101a700 1 --2- 172.21.15.3:0/842815651 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f2c09d6f0 0x7f6f2c1cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.113+0000 7f6f3101a700 1 -- 172.21.15.3:0/842815651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f2c09e410 msgr2=0x7f6f2c1cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:27.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.113+0000 7f6f3101a700 1 --2- 172.21.15.3:0/842815651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f2c09e410 0x7f6f2c1cd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.113+0000 7f6f3101a700 1 -- 172.21.15.3:0/842815651 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f24009a70 con 0x7f6f2c09e000 2023-10-03T10:15:27.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.114+0000 7f6f3181b700 1 --2- 172.21.15.3:0/842815651 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f2c09d6f0 0x7f6f2c1cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:15:27.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.113+0000 7efd935ec700 1 -- 172.21.15.3:0/450594071 shutdown_connections 2023-10-03T10:15:27.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.113+0000 7efd935ec700 1 --2- 172.21.15.3:0/450594071 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd8c09a3b0 0x7efd8c138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.113+0000 7efd935ec700 1 --2- 172.21.15.3:0/450594071 >> v2:172.21.15.3:3300/0 conn(0x7efd8c131810 0x7efd8c131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.113+0000 7efd935ec700 1 --2- 172.21.15.3:0/450594071 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd8c099a40 0x7efd8c099e70 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.113+0000 7efd935ec700 1 -- 172.21.15.3:0/450594071 >> 172.21.15.3:0/450594071 conn(0x7efd8c0a2aa0 msgr2=0x7efd8c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:27.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.114+0000 7efd935ec700 1 -- 172.21.15.3:0/450594071 shutdown_connections 2023-10-03T10:15:27.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.114+0000 7efd935ec700 1 -- 172.21.15.3:0/450594071 wait complete. 2023-10-03T10:15:27.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.114+0000 7f6f3101a700 1 --2- 172.21.15.3:0/842815651 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f2c09e000 0x7f6f2c1ccac0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00b340 tx=0x7f6f1c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:27.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.114+0000 7f6f227fc700 1 -- 172.21.15.3:0/842815651 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c01c070 con 0x7f6f2c09e000 2023-10-03T10:15:27.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.114+0000 7f6f227fc700 1 -- 172.21.15.3:0/842815651 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1c00c8a0 con 0x7f6f2c09e000 2023-10-03T10:15:27.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.114+0000 7f6f33a7f700 1 -- 172.21.15.3:0/842815651 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f1c00b600 con 0x7f6f2c09e000 2023-10-03T10:15:27.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.114+0000 7f6f227fc700 1 -- 172.21.15.3:0/842815651 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c004270 con 0x7f6f2c09e000 2023-10-03T10:15:27.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.114+0000 7f6f33a7f700 1 -- 172.21.15.3:0/842815651 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f2c12a0b0 con 0x7f6f2c09e000 2023-10-03T10:15:27.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.115+0000 7efd935ec700 1 Processor -- start 2023-10-03T10:15:27.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.115+0000 7efd935ec700 1 -- start start 2023-10-03T10:15:27.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.116+0000 7f6f33a7f700 1 -- 172.21.15.3:0/842815651 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f2c12a3e0 con 0x7f6f2c09e000 2023-10-03T10:15:27.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.117+0000 7f6f227fc700 1 -- 172.21.15.3:0/842815651 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6f1c0043d0 con 0x7f6f2c09e000 2023-10-03T10:15:27.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.117+0000 7f6f227fc700 1 -- 172.21.15.3:0/842815651 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6f1c040ec0 con 0x7f6f2c09e000 2023-10-03T10:15:27.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.116+0000 7efd935ec700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd8c099a40 0x7efd8c118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:27.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.116+0000 7efd935ec700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd8c09a3b0 0x7efd8c119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:27.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.116+0000 7efd935ec700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd8c131810 0x7efd8c119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:27.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.116+0000 7efd935ec700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c12bcf0 con 0x7efd8c09a3b0 2023-10-03T10:15:27.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.116+0000 7efd935ec700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c11be00 con 0x7efd8c131810 2023-10-03T10:15:27.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.116+0000 7efd935ec700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c0a1cb0 con 0x7efd8c099a40 2023-10-03T10:15:27.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.116+0000 7efd90b87700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd8c09a3b0 0x7efd8c119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:27.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.116+0000 7efd91b89700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd8c131810 0x7efd8c119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:27.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.116+0000 7efd90b87700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd8c09a3b0 0x7efd8c119210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45168/0 (socket says 172.21.15.3:45168) 2023-10-03T10:15:27.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.116+0000 7efd90b87700 1 -- 172.21.15.3:0/3047494194 learned_addr learned my addr 172.21.15.3:0/3047494194 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:27.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.116+0000 7efd91388700 1 --2- 172.21.15.3:0/3047494194 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd8c099a40 0x7efd8c118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:27.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.117+0000 7efd91b89700 1 -- 172.21.15.3:0/3047494194 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd8c099a40 msgr2=0x7efd8c118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:27.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.117+0000 7efd91b89700 1 --2- 172.21.15.3:0/3047494194 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd8c099a40 0x7efd8c118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.117+0000 7efd91b89700 1 -- 172.21.15.3:0/3047494194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd8c09a3b0 msgr2=0x7efd8c119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:27.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.117+0000 7efd91b89700 1 --2- 172.21.15.3:0/3047494194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd8c09a3b0 0x7efd8c119210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.117+0000 7efd91b89700 1 -- 172.21.15.3:0/3047494194 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd7c009550 con 0x7efd8c131810 2023-10-03T10:15:27.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.117+0000 7efd91b89700 1 --2- 172.21.15.3:0/3047494194 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd8c131810 0x7efd8c119750 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7efd8400be20 tx=0x7efd84009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:27.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.117+0000 7efd81ffb700 1 -- 172.21.15.3:0/3047494194 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8400cc10 con 0x7efd8c131810 2023-10-03T10:15:27.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.117+0000 7efd935ec700 1 -- 172.21.15.3:0/3047494194 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd8c11ac80 con 0x7efd8c131810 2023-10-03T10:15:27.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.117+0000 7efd81ffb700 1 -- 172.21.15.3:0/3047494194 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8400cd90 con 0x7efd8c131810 2023-10-03T10:15:27.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.117+0000 7efd81ffb700 1 -- 172.21.15.3:0/3047494194 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd84014680 con 0x7efd8c131810 2023-10-03T10:15:27.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.118+0000 7efd935ec700 1 -- 172.21.15.3:0/3047494194 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd8c11cb30 con 0x7efd8c131810 2023-10-03T10:15:27.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.119+0000 7efd935ec700 1 -- 172.21.15.3:0/3047494194 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd8c11bf90 con 0x7efd8c131810 2023-10-03T10:15:27.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.120+0000 7efd81ffb700 1 -- 172.21.15.3:0/3047494194 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efd8401e440 con 0x7efd8c131810 2023-10-03T10:15:27.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.120+0000 7efd81ffb700 1 -- 172.21.15.3:0/3047494194 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7efd84006e40 con 0x7efd8c131810 2023-10-03T10:15:27.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.125+0000 7efd81ffb700 1 -- 172.21.15.3:0/3047494194 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efd8401b2c0 con 0x7efd8c131810 2023-10-03T10:15:27.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.126+0000 7f6f227fc700 1 -- 172.21.15.3:0/842815651 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6f1c0036a0 con 0x7f6f2c09e000 2023-10-03T10:15:27.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.234+0000 7f8a0dcac700 1 -- 172.21.15.3:0/1222760579 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8a0806b9e0 con 0x7f8a081316d0 2023-10-03T10:15:27.235 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:27.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.235+0000 7f89effff700 1 -- 172.21.15.3:0/1222760579 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f89fc017030 con 0x7f8a081316d0 2023-10-03T10:15:27.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.238+0000 7f8a0dcac700 1 -- 172.21.15.3:0/1222760579 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a081316d0 msgr2=0x7f8a08128440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:27.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.238+0000 7f8a0dcac700 1 --2- 172.21.15.3:0/1222760579 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a081316d0 0x7f8a08128440 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00c4e0 tx=0x7f89fc00c7f0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.238+0000 7f8a0dcac700 1 -- 172.21.15.3:0/1222760579 shutdown_connections 2023-10-03T10:15:27.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.238+0000 7f8a0dcac700 1 --2- 172.21.15.3:0/1222760579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a08131fe0 0x7f8a081c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.238+0000 7f8a0dcac700 1 --2- 172.21.15.3:0/1222760579 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a0806ec60 0x7f8a08127f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.238+0000 7f8a0dcac700 1 --2- 172.21.15.3:0/1222760579 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a081316d0 0x7f8a08128440 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.238+0000 7f8a0dcac700 1 -- 172.21.15.3:0/1222760579 >> 172.21.15.3:0/1222760579 conn(0x7f8a080a2aa0 msgr2=0x7f8a0809f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:27.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.239+0000 7f8a0dcac700 1 -- 172.21.15.3:0/1222760579 shutdown_connections 2023-10-03T10:15:27.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.239+0000 7f8a0dcac700 1 -- 172.21.15.3:0/1222760579 wait complete. 2023-10-03T10:15:27.258 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:27.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.258+0000 7fc9c61c5700 1 -- 172.21.15.3:0/2304715836 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc9c0074ea0 con 0x7fc9c006f680 2023-10-03T10:15:27.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.259+0000 7fc9a7fff700 1 -- 172.21.15.3:0/2304715836 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc9b0019bb0 con 0x7fc9c006f680 2023-10-03T10:15:27.259 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:27.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.262+0000 7fc9c61c5700 1 -- 172.21.15.3:0/2304715836 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9c006f680 msgr2=0x7fc9c01349c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:27.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.262+0000 7fc9c61c5700 1 --2- 172.21.15.3:0/2304715836 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9c006f680 0x7fc9c01349c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0015000 tx=0x7fc9b0004750 comp rx=0 tx=0).stop 2023-10-03T10:15:27.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.262+0000 7fc9c61c5700 1 -- 172.21.15.3:0/2304715836 shutdown_connections 2023-10-03T10:15:27.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.262+0000 7fc9c61c5700 1 --2- 172.21.15.3:0/2304715836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9c0130670 0x7fc9c01c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.262+0000 7fc9c61c5700 1 --2- 172.21.15.3:0/2304715836 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9c006f680 0x7fc9c01349c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.262+0000 7fc9c61c5700 1 --2- 172.21.15.3:0/2304715836 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9c00701a0 0x7fc9c01c8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.262+0000 7fc9c61c5700 1 -- 172.21.15.3:0/2304715836 >> 172.21.15.3:0/2304715836 conn(0x7fc9c00a2aa0 msgr2=0x7fc9c00a1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:27.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.262+0000 7fc9c61c5700 1 -- 172.21.15.3:0/2304715836 shutdown_connections 2023-10-03T10:15:27.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.262+0000 7fc9c61c5700 1 -- 172.21.15.3:0/2304715836 wait complete. 2023-10-03T10:15:27.276 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:27.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.332+0000 7efd935ec700 1 -- 172.21.15.3:0/3047494194 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7efd8c11bf90 con 0x7efd8c131810 2023-10-03T10:15:27.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.332+0000 7efd81ffb700 1 -- 172.21.15.3:0/3047494194 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efd8c11bf90 con 0x7efd8c131810 2023-10-03T10:15:27.333 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:27.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.335+0000 7efd935ec700 1 -- 172.21.15.3:0/3047494194 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd8c131810 msgr2=0x7efd8c119750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:27.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.335+0000 7efd935ec700 1 --2- 172.21.15.3:0/3047494194 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd8c131810 0x7efd8c119750 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7efd8400be20 tx=0x7efd84009ed0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.335+0000 7efd935ec700 1 -- 172.21.15.3:0/3047494194 shutdown_connections 2023-10-03T10:15:27.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.335+0000 7efd935ec700 1 --2- 172.21.15.3:0/3047494194 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd8c131810 0x7efd8c119750 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.335+0000 7efd935ec700 1 --2- 172.21.15.3:0/3047494194 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd8c099a40 0x7efd8c118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.335+0000 7efd935ec700 1 --2- 172.21.15.3:0/3047494194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd8c09a3b0 0x7efd8c119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.335+0000 7efd935ec700 1 -- 172.21.15.3:0/3047494194 >> 172.21.15.3:0/3047494194 conn(0x7efd8c0a2aa0 msgr2=0x7efd8c12e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:27.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.335+0000 7efd935ec700 1 -- 172.21.15.3:0/3047494194 shutdown_connections 2023-10-03T10:15:27.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.335+0000 7efd935ec700 1 -- 172.21.15.3:0/3047494194 wait complete. 2023-10-03T10:15:27.349 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:27.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.348+0000 7f6f33a7f700 1 -- 172.21.15.3:0/842815651 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6f2c129d50 con 0x7f6f2c09e000 2023-10-03T10:15:27.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.348+0000 7f6f227fc700 1 -- 172.21.15.3:0/842815651 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6f2c129d50 con 0x7f6f2c09e000 2023-10-03T10:15:27.349 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:27.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.351+0000 7f6f33a7f700 1 -- 172.21.15.3:0/842815651 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f2c09e000 msgr2=0x7f6f2c1ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:27.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.351+0000 7f6f33a7f700 1 --2- 172.21.15.3:0/842815651 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f2c09e000 0x7f6f2c1ccac0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00b340 tx=0x7f6f1c004750 comp rx=0 tx=0).stop 2023-10-03T10:15:27.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.351+0000 7f6f33a7f700 1 -- 172.21.15.3:0/842815651 shutdown_connections 2023-10-03T10:15:27.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.351+0000 7f6f33a7f700 1 --2- 172.21.15.3:0/842815651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f2c09e410 0x7f6f2c1cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.351+0000 7f6f33a7f700 1 --2- 172.21.15.3:0/842815651 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f2c09d6f0 0x7f6f2c1cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.351+0000 7f6f33a7f700 1 --2- 172.21.15.3:0/842815651 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f2c09e000 0x7f6f2c1ccac0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:27.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.351+0000 7f6f33a7f700 1 -- 172.21.15.3:0/842815651 >> 172.21.15.3:0/842815651 conn(0x7f6f2c066f50 msgr2=0x7f6f2c070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:27.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.351+0000 7f6f33a7f700 1 -- 172.21.15.3:0/842815651 shutdown_connections 2023-10-03T10:15:27.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:27.351+0000 7f6f33a7f700 1 -- 172.21.15.3:0/842815651 wait complete. 2023-10-03T10:15:27.365 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:28.259 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:28.277 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:28.349 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:28.365 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:28.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.412+0000 7f6f37a73700 1 Processor -- start 2023-10-03T10:15:28.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.413+0000 7f6f37a73700 1 -- start start 2023-10-03T10:15:28.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.413+0000 7f6f37a73700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f3006a770 0x7f6f3009d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:28.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.413+0000 7f6f37a73700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f3009dc30 0x7f6f3009e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:28.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.413+0000 7f6f37a73700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6f3013c1e0 0x7f6f3013e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:28.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.413+0000 7f6f37a73700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f3006ab40 con 0x7f6f3013c1e0 2023-10-03T10:15:28.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.413+0000 7f6f37a73700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3009ebc0 con 0x7f6f3009dc30 2023-10-03T10:15:28.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.413+0000 7f6f37a73700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3009a810 con 0x7f6f3006a770 2023-10-03T10:15:28.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.414+0000 7f6f3580f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f3006a770 0x7f6f3009d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:28.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.414+0000 7f6f3580f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f3006a770 0x7f6f3009d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43630/0 (socket says 172.21.15.3:43630) 2023-10-03T10:15:28.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.414+0000 7f6f3580f700 1 -- 172.21.15.3:0/1007196977 learned_addr learned my addr 172.21.15.3:0/1007196977 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:28.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.414+0000 7f6f36010700 1 --2- 172.21.15.3:0/1007196977 >> v2:172.21.15.3:3300/0 conn(0x7f6f3013c1e0 0x7f6f3013e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:28.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.414+0000 7f6f3500e700 1 --2- 172.21.15.3:0/1007196977 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f3009dc30 0x7f6f3009e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:28.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.414+0000 7f6f3580f700 1 -- 172.21.15.3:0/1007196977 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f3009dc30 msgr2=0x7f6f3009e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:28.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.414+0000 7f6f3580f700 1 --2- 172.21.15.3:0/1007196977 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f3009dc30 0x7f6f3009e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.414+0000 7f6f3580f700 1 -- 172.21.15.3:0/1007196977 >> v2:172.21.15.3:3300/0 conn(0x7f6f3013c1e0 msgr2=0x7f6f3013e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:28.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.414+0000 7f6f3580f700 1 --2- 172.21.15.3:0/1007196977 >> v2:172.21.15.3:3300/0 conn(0x7f6f3013c1e0 0x7f6f3013e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.414+0000 7f6f3580f700 1 -- 172.21.15.3:0/1007196977 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3009e920 con 0x7f6f3006a770 2023-10-03T10:15:28.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.414+0000 7f6f36010700 1 --2- 172.21.15.3:0/1007196977 >> v2:172.21.15.3:3300/0 conn(0x7f6f3013c1e0 0x7f6f3013e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:28.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.414+0000 7f6f3580f700 1 --2- 172.21.15.3:0/1007196977 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f3006a770 0x7f6f3009d6f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009920 tx=0x7f6f2c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf680d4892e303a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:28.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.415+0000 7f6f3480d700 1 -- 172.21.15.3:0/1007196977 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c011830 con 0x7f6f3006a770 2023-10-03T10:15:28.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.415+0000 7f6f3480d700 1 -- 172.21.15.3:0/1007196977 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2c011990 con 0x7f6f3006a770 2023-10-03T10:15:28.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.415+0000 7f6f3480d700 1 -- 172.21.15.3:0/1007196977 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c011d20 con 0x7f6f3006a770 2023-10-03T10:15:28.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.415+0000 7f6f37a73700 1 -- 172.21.15.3:0/1007196977 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f3006a770 msgr2=0x7f6f3009d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:28.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.415+0000 7f6f37a73700 1 --2- 172.21.15.3:0/1007196977 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f3006a770 0x7f6f3009d6f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009920 tx=0x7f6f2c009c30 comp rx=0 tx=0).stop 2023-10-03T10:15:28.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.416+0000 7f6f37a73700 1 -- 172.21.15.3:0/1007196977 shutdown_connections 2023-10-03T10:15:28.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.416+0000 7f6f37a73700 1 --2- 172.21.15.3:0/1007196977 >> v2:172.21.15.3:3300/0 conn(0x7f6f3013c1e0 0x7f6f3013e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.416+0000 7f6f37a73700 1 --2- 172.21.15.3:0/1007196977 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f3006a770 0x7f6f3009d6f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.416+0000 7f6f37a73700 1 --2- 172.21.15.3:0/1007196977 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f3009dc30 0x7f6f3009e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.416+0000 7f6f37a73700 1 -- 172.21.15.3:0/1007196977 >> 172.21.15.3:0/1007196977 conn(0x7f6f30066f50 msgr2=0x7f6f3009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:28.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.420+0000 7f6f37a73700 1 -- 172.21.15.3:0/1007196977 shutdown_connections 2023-10-03T10:15:28.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.420+0000 7f6f37a73700 1 -- 172.21.15.3:0/1007196977 wait complete. 2023-10-03T10:15:28.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.421+0000 7f6f37a73700 1 Processor -- start 2023-10-03T10:15:28.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.421+0000 7f6f37a73700 1 -- start start 2023-10-03T10:15:28.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.422+0000 7f6f37a73700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f3006a770 0x7f6f3013bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:28.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.422+0000 7f6f37a73700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f3009dc30 0x7f6f3013a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:28.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.422+0000 7f6f37a73700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f3013c1e0 0x7f6f3013a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:28.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.422+0000 7f6f37a73700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f300a3980 con 0x7f6f3006a770 2023-10-03T10:15:28.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.422+0000 7f6f37a73700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3013c080 con 0x7f6f3009dc30 2023-10-03T10:15:28.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.422+0000 7f6f37a73700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3009f1a0 con 0x7f6f3013c1e0 2023-10-03T10:15:28.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.422+0000 7f6f3580f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f3006a770 0x7f6f3013bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:28.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.422+0000 7f6f3580f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f3006a770 0x7f6f3013bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45184/0 (socket says 172.21.15.3:45184) 2023-10-03T10:15:28.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.422+0000 7f6f3580f700 1 -- 172.21.15.3:0/1163768345 learned_addr learned my addr 172.21.15.3:0/1163768345 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:28.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.422+0000 7f6f36010700 1 --2- 172.21.15.3:0/1163768345 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f3013c1e0 0x7f6f3013a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:28.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.422+0000 7f6f3580f700 1 -- 172.21.15.3:0/1163768345 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f3013c1e0 msgr2=0x7f6f3013a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:28.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.422+0000 7f6f3580f700 1 --2- 172.21.15.3:0/1163768345 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f3013c1e0 0x7f6f3013a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.422+0000 7f6f3580f700 1 -- 172.21.15.3:0/1163768345 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f3009dc30 msgr2=0x7f6f3013a460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:15:28.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.422+0000 7f6f3580f700 1 --2- 172.21.15.3:0/1163768345 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f3009dc30 0x7f6f3013a460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.422+0000 7f6f3580f700 1 -- 172.21.15.3:0/1163768345 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f28009a70 con 0x7f6f3006a770 2023-10-03T10:15:28.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.422+0000 7f6f36010700 1 --2- 172.21.15.3:0/1163768345 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f3013c1e0 0x7f6f3013a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:28.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.423+0000 7f6f3580f700 1 --2- 172.21.15.3:0/1163768345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f3006a770 0x7f6f3013bb40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c000c00 tx=0x7f6f2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:28.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.423+0000 7f6f267fc700 1 -- 172.21.15.3:0/1163768345 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c004240 con 0x7f6f3006a770 2023-10-03T10:15:28.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.423+0000 7f6f267fc700 1 -- 172.21.15.3:0/1163768345 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2c01f070 con 0x7f6f3006a770 2023-10-03T10:15:28.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.423+0000 7f6f37a73700 1 -- 172.21.15.3:0/1163768345 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f2c009550 con 0x7f6f3006a770 2023-10-03T10:15:28.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.423+0000 7f6f267fc700 1 -- 172.21.15.3:0/1163768345 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c0045e0 con 0x7f6f3006a770 2023-10-03T10:15:28.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.423+0000 7f6f37a73700 1 -- 172.21.15.3:0/1163768345 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f30072210 con 0x7f6f3006a770 2023-10-03T10:15:28.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.424+0000 7f6f37a73700 1 -- 172.21.15.3:0/1163768345 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f3011f6a0 con 0x7f6f3006a770 2023-10-03T10:15:28.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.425+0000 7f6f267fc700 1 -- 172.21.15.3:0/1163768345 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6f2c021030 con 0x7f6f3006a770 2023-10-03T10:15:28.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.425+0000 7f6f267fc700 1 -- 172.21.15.3:0/1163768345 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6f2c040d90 con 0x7f6f3006a770 2023-10-03T10:15:28.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.431+0000 7f6f267fc700 1 -- 172.21.15.3:0/1163768345 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6f2c003e00 con 0x7f6f3006a770 2023-10-03T10:15:28.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.458+0000 7f2975415700 1 Processor -- start 2023-10-03T10:15:28.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.458+0000 7f2975415700 1 -- start start 2023-10-03T10:15:28.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.458+0000 7f2975415700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f297009d6f0 0x7f297009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:28.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.458+0000 7f2975415700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f297009e000 0x7f2970130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.459+0000 7f2975415700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f297009e410 0x7f2970130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.459+0000 7f2975415700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f297009f520 con 0x7f297009e410 2023-10-03T10:15:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.459+0000 7f2975415700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f297009f130 con 0x7f297009d6f0 2023-10-03T10:15:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.459+0000 7f2975415700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f297009e930 con 0x7f297009e000 2023-10-03T10:15:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.459+0000 7f296e7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f297009e000 0x7f2970130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.459+0000 7f296effd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f297009d6f0 0x7f297009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.459+0000 7f296e7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f297009e000 0x7f2970130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43650/0 (socket says 172.21.15.3:43650) 2023-10-03T10:15:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.459+0000 7f296e7fc700 1 -- 172.21.15.3:0/1047169567 learned_addr learned my addr 172.21.15.3:0/1047169567 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.459+0000 7f296f7fe700 1 --2- 172.21.15.3:0/1047169567 >> v2:172.21.15.3:3300/0 conn(0x7f297009e410 0x7f2970130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.459+0000 7f296e7fc700 1 -- 172.21.15.3:0/1047169567 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f297009d6f0 msgr2=0x7f297009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.459+0000 7f296e7fc700 1 --2- 172.21.15.3:0/1047169567 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f297009d6f0 0x7f297009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.459+0000 7f296e7fc700 1 -- 172.21.15.3:0/1047169567 >> v2:172.21.15.3:3300/0 conn(0x7f297009e410 msgr2=0x7f2970130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:28.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.459+0000 7f296e7fc700 1 --2- 172.21.15.3:0/1047169567 >> v2:172.21.15.3:3300/0 conn(0x7f297009e410 0x7f2970130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.459+0000 7f296e7fc700 1 -- 172.21.15.3:0/1047169567 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297009b2d0 con 0x7f297009e000 2023-10-03T10:15:28.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.459+0000 7f296effd700 1 --2- 172.21.15.3:0/1047169567 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f297009d6f0 0x7f297009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:28.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.459+0000 7f296f7fe700 1 --2- 172.21.15.3:0/1047169567 >> v2:172.21.15.3:3300/0 conn(0x7f297009e410 0x7f2970130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:28.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.460+0000 7f296e7fc700 1 --2- 172.21.15.3:0/1047169567 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f297009e000 0x7f2970130660 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f296400bbc0 tx=0x7f296400bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2495a2a7257d2219 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:28.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.460+0000 7f296dffb700 1 -- 172.21.15.3:0/1047169567 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296400dcf0 con 0x7f297009e000 2023-10-03T10:15:28.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.460+0000 7f296dffb700 1 -- 172.21.15.3:0/1047169567 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296400de50 con 0x7f297009e000 2023-10-03T10:15:28.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.460+0000 7f296dffb700 1 -- 172.21.15.3:0/1047169567 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2964010690 con 0x7f297009e000 2023-10-03T10:15:28.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.461+0000 7f2975415700 1 -- 172.21.15.3:0/1047169567 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f297009e000 msgr2=0x7f2970130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:28.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.461+0000 7f2975415700 1 --2- 172.21.15.3:0/1047169567 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f297009e000 0x7f2970130660 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f296400bbc0 tx=0x7f296400bed0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.461+0000 7f2975415700 1 -- 172.21.15.3:0/1047169567 shutdown_connections 2023-10-03T10:15:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.461+0000 7f2975415700 1 --2- 172.21.15.3:0/1047169567 >> v2:172.21.15.3:3300/0 conn(0x7f297009e410 0x7f2970130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.461+0000 7f2975415700 1 --2- 172.21.15.3:0/1047169567 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f297009d6f0 0x7f297009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.461+0000 7f2975415700 1 --2- 172.21.15.3:0/1047169567 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f297009e000 0x7f2970130660 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.461+0000 7f2975415700 1 -- 172.21.15.3:0/1047169567 >> 172.21.15.3:0/1047169567 conn(0x7f2970066f50 msgr2=0x7f297009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.461+0000 7f2975415700 1 -- 172.21.15.3:0/1047169567 shutdown_connections 2023-10-03T10:15:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.461+0000 7f2975415700 1 -- 172.21.15.3:0/1047169567 wait complete. 2023-10-03T10:15:28.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.462+0000 7f2975415700 1 Processor -- start 2023-10-03T10:15:28.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.462+0000 7f2975415700 1 -- start start 2023-10-03T10:15:28.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.463+0000 7f2975415700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f297009d6f0 0x7f29701224c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:28.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.463+0000 7f2975415700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f297009e000 0x7f2970122a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:28.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.463+0000 7f2975415700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f297009e410 0x7f2970122f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:28.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.463+0000 7f2975415700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29701303e0 con 0x7f297009d6f0 2023-10-03T10:15:28.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.463+0000 7f2975415700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29701241e0 con 0x7f297009e000 2023-10-03T10:15:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.463+0000 7f2975415700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2970125a40 con 0x7f297009e410 2023-10-03T10:15:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.463+0000 7f296f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f297009e410 0x7f2970122f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.463+0000 7f296f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f297009e410 0x7f2970122f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43664/0 (socket says 172.21.15.3:43664) 2023-10-03T10:15:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.463+0000 7f296e7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f297009e000 0x7f2970122a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.463+0000 7f296f7fe700 1 -- 172.21.15.3:0/2813768614 learned_addr learned my addr 172.21.15.3:0/2813768614 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.463+0000 7f296effd700 1 --2- 172.21.15.3:0/2813768614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f297009d6f0 0x7f29701224c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.464+0000 7f296f7fe700 1 -- 172.21.15.3:0/2813768614 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f297009e000 msgr2=0x7f2970122a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.464+0000 7f296f7fe700 1 --2- 172.21.15.3:0/2813768614 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f297009e000 0x7f2970122a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.464+0000 7f296f7fe700 1 -- 172.21.15.3:0/2813768614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f297009d6f0 msgr2=0x7f29701224c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.464+0000 7f296f7fe700 1 --2- 172.21.15.3:0/2813768614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f297009d6f0 0x7f29701224c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.464+0000 7f296f7fe700 1 -- 172.21.15.3:0/2813768614 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29600094a0 con 0x7f297009e410 2023-10-03T10:15:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.464+0000 7f296f7fe700 1 --2- 172.21.15.3:0/2813768614 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f297009e410 0x7f2970122f40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f296800efc0 tx=0x7f296800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:28.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.464+0000 7f29577fe700 1 -- 172.21.15.3:0/2813768614 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296800ca90 con 0x7f297009e410 2023-10-03T10:15:28.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.464+0000 7f29577fe700 1 -- 172.21.15.3:0/2813768614 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296800f040 con 0x7f297009e410 2023-10-03T10:15:28.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.464+0000 7f29577fe700 1 -- 172.21.15.3:0/2813768614 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296800ce30 con 0x7f297009e410 2023-10-03T10:15:28.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.464+0000 7f2975415700 1 -- 172.21.15.3:0/2813768614 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2968009a70 con 0x7f297009e410 2023-10-03T10:15:28.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.464+0000 7f2975415700 1 -- 172.21.15.3:0/2813768614 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296400b7f0 con 0x7f297009e410 2023-10-03T10:15:28.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.465+0000 7f29577fe700 1 -- 172.21.15.3:0/2813768614 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2968014480 con 0x7f297009e410 2023-10-03T10:15:28.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.466+0000 7f29577fe700 1 -- 172.21.15.3:0/2813768614 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f296801c570 con 0x7f297009e410 2023-10-03T10:15:28.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.466+0000 7f2975415700 1 -- 172.21.15.3:0/2813768614 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2970074ea0 con 0x7f297009e410 2023-10-03T10:15:28.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.471+0000 7f29577fe700 1 -- 172.21.15.3:0/2813768614 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2968012930 con 0x7f297009e410 2023-10-03T10:15:28.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.509+0000 7f7716376700 1 Processor -- start 2023-10-03T10:15:28.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.509+0000 7f7716376700 1 -- start start 2023-10-03T10:15:28.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.510+0000 7f7716376700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f771009d6f0 0x7f771009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:28.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.510+0000 7f7716376700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f771009e000 0x7f7710130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:28.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.510+0000 7f7716376700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f771009e410 0x7f7710130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:28.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.510+0000 7f7716376700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f771009f520 con 0x7f771009d6f0 2023-10-03T10:15:28.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.510+0000 7f7716376700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771009f130 con 0x7f771009e000 2023-10-03T10:15:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.510+0000 7f7716376700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f771009e930 con 0x7f771009e410 2023-10-03T10:15:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.510+0000 7f7714913700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f771009e410 0x7f7710130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.510+0000 7f7714913700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f771009e410 0x7f7710130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43672/0 (socket says 172.21.15.3:43672) 2023-10-03T10:15:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.510+0000 7f7714913700 1 -- 172.21.15.3:0/2143535910 learned_addr learned my addr 172.21.15.3:0/2143535910 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.510+0000 7f770f7fe700 1 --2- 172.21.15.3:0/2143535910 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f771009e000 0x7f7710130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.510+0000 7f770ffff700 1 --2- 172.21.15.3:0/2143535910 >> v2:172.21.15.3:3300/0 conn(0x7f771009d6f0 0x7f771009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:28.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.510+0000 7f7714913700 1 -- 172.21.15.3:0/2143535910 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f771009e000 msgr2=0x7f7710130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:28.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.510+0000 7f7714913700 1 --2- 172.21.15.3:0/2143535910 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f771009e000 0x7f7710130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.510+0000 7f7714913700 1 -- 172.21.15.3:0/2143535910 >> v2:172.21.15.3:3300/0 conn(0x7f771009d6f0 msgr2=0x7f771009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.510+0000 7f7714913700 1 --2- 172.21.15.3:0/2143535910 >> v2:172.21.15.3:3300/0 conn(0x7f771009d6f0 0x7f771009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.510+0000 7f7714913700 1 -- 172.21.15.3:0/2143535910 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f771009b2d0 con 0x7f771009e410 2023-10-03T10:15:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.510+0000 7f770f7fe700 1 --2- 172.21.15.3:0/2143535910 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f771009e000 0x7f7710130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.510+0000 7f770ffff700 1 --2- 172.21.15.3:0/2143535910 >> v2:172.21.15.3:3300/0 conn(0x7f771009d6f0 0x7f771009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.511+0000 7f7714913700 1 --2- 172.21.15.3:0/2143535910 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f771009e410 0x7f7710130ba0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f770800d180 tx=0x7f770800d490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=109af68318f09d8d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.511+0000 7f770effd700 1 -- 172.21.15.3:0/2143535910 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7708010070 con 0x7f771009e410 2023-10-03T10:15:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.511+0000 7f770effd700 1 -- 172.21.15.3:0/2143535910 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770800dd00 con 0x7f771009e410 2023-10-03T10:15:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.511+0000 7f770effd700 1 -- 172.21.15.3:0/2143535910 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770800b310 con 0x7f771009e410 2023-10-03T10:15:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.512+0000 7f7716376700 1 -- 172.21.15.3:0/2143535910 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f771009e410 msgr2=0x7f7710130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.512+0000 7f7716376700 1 --2- 172.21.15.3:0/2143535910 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f771009e410 0x7f7710130ba0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f770800d180 tx=0x7f770800d490 comp rx=0 tx=0).stop 2023-10-03T10:15:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.512+0000 7f7716376700 1 -- 172.21.15.3:0/2143535910 shutdown_connections 2023-10-03T10:15:28.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.512+0000 7f7716376700 1 --2- 172.21.15.3:0/2143535910 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f771009e410 0x7f7710130ba0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.512+0000 7f7716376700 1 --2- 172.21.15.3:0/2143535910 >> v2:172.21.15.3:3300/0 conn(0x7f771009d6f0 0x7f771009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.512+0000 7f7716376700 1 --2- 172.21.15.3:0/2143535910 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f771009e000 0x7f7710130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.512+0000 7f7716376700 1 -- 172.21.15.3:0/2143535910 >> 172.21.15.3:0/2143535910 conn(0x7f7710066f50 msgr2=0x7f771009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:28.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.512+0000 7f7716376700 1 -- 172.21.15.3:0/2143535910 shutdown_connections 2023-10-03T10:15:28.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.512+0000 7f7716376700 1 -- 172.21.15.3:0/2143535910 wait complete. 2023-10-03T10:15:28.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.514+0000 7f7716376700 1 Processor -- start 2023-10-03T10:15:28.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.514+0000 7f7716376700 1 -- start start 2023-10-03T10:15:28.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.515+0000 7f7716376700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f771009d6f0 0x7f771011ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:28.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.515+0000 7f7716376700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f771009e000 0x7f771011e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:28.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.515+0000 7f7716376700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f771009e410 0x7f771011e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.515+0000 7f7716376700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771011f260 con 0x7f771009e000 2023-10-03T10:15:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.515+0000 7f7716376700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771011f9c0 con 0x7f771009e410 2023-10-03T10:15:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.515+0000 7f7716376700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7710120c10 con 0x7f771009d6f0 2023-10-03T10:15:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.515+0000 7f770ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f771009d6f0 0x7f771011ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.515+0000 7f7714913700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f771009e410 0x7f771011e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.515+0000 7f770ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f771009d6f0 0x7f771011ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43678/0 (socket says 172.21.15.3:43678) 2023-10-03T10:15:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.515+0000 7f770ffff700 1 -- 172.21.15.3:0/1385895279 learned_addr learned my addr 172.21.15.3:0/1385895279 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.515+0000 7f770f7fe700 1 --2- 172.21.15.3:0/1385895279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f771009e000 0x7f771011e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.515+0000 7f770ffff700 1 -- 172.21.15.3:0/1385895279 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f771009e410 msgr2=0x7f771011e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.515+0000 7f770ffff700 1 --2- 172.21.15.3:0/1385895279 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f771009e410 0x7f771011e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.515+0000 7f770ffff700 1 -- 172.21.15.3:0/1385895279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f771009e000 msgr2=0x7f771011e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.515+0000 7f770ffff700 1 --2- 172.21.15.3:0/1385895279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f771009e000 0x7f771011e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.515+0000 7f770ffff700 1 -- 172.21.15.3:0/1385895279 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77000094a0 con 0x7f771009d6f0 2023-10-03T10:15:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.515+0000 7f7714913700 1 --2- 172.21.15.3:0/1385895279 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f771009e410 0x7f771011e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.516+0000 7f770ffff700 1 --2- 172.21.15.3:0/1385895279 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f771009d6f0 0x7f771011ddb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f770000ec10 tx=0x7f770000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.516+0000 7f770cff9700 1 -- 172.21.15.3:0/1385895279 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7700009780 con 0x7f771009d6f0 2023-10-03T10:15:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.516+0000 7f770cff9700 1 -- 172.21.15.3:0/1385895279 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770000c850 con 0x7f771009d6f0 2023-10-03T10:15:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.516+0000 7f7716376700 1 -- 172.21.15.3:0/1385895279 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77040094a0 con 0x7f771009d6f0 2023-10-03T10:15:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.516+0000 7f770cff9700 1 -- 172.21.15.3:0/1385895279 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770000cbe0 con 0x7f771009d6f0 2023-10-03T10:15:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.517+0000 7f7716376700 1 -- 172.21.15.3:0/1385895279 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f770800bd40 con 0x7f771009d6f0 2023-10-03T10:15:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.517+0000 7f770cff9700 1 -- 172.21.15.3:0/1385895279 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f770000cd40 con 0x7f771009d6f0 2023-10-03T10:15:28.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.518+0000 7f7716376700 1 -- 172.21.15.3:0/1385895279 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77100a1660 con 0x7f771009d6f0 2023-10-03T10:15:28.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.518+0000 7f770cff9700 1 -- 172.21.15.3:0/1385895279 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f770001bd80 con 0x7f771009d6f0 2023-10-03T10:15:28.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.528+0000 7f770cff9700 1 -- 172.21.15.3:0/1385895279 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7700044060 con 0x7f771009d6f0 2023-10-03T10:15:28.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.557+0000 7f044c432700 1 Processor -- start 2023-10-03T10:15:28.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.557+0000 7f044c432700 1 -- start start 2023-10-03T10:15:28.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.558+0000 7f044c432700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f044409d630 0x7f04440a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:28.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.558+0000 7f044c432700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0444138d80 0x7f044413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:28.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.558+0000 7f044c432700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04440a2a70 con 0x7f04440a0a30 2023-10-03T10:15:28.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.558+0000 7f044c432700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04440a01d0 con 0x7f044409d630 2023-10-03T10:15:28.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.558+0000 7f044c432700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f044409e2b0 con 0x7f0444138d80 2023-10-03T10:15:28.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.558+0000 7f044a1ce700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f04440a0a30 0x7f04440a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39696/0 (socket says 172.21.15.3:39696) 2023-10-03T10:15:28.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.558+0000 7f044a1ce700 1 -- 172.21.15.3:0/2642876920 learned_addr learned my addr 172.21.15.3:0/2642876920 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:28.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.558+0000 7f04499cd700 1 --2- 172.21.15.3:0/2642876920 >> v2:172.21.15.3:3300/0 conn(0x7f044409d630 0x7f04440a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:28.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.558+0000 7f044a9cf700 1 --2- 172.21.15.3:0/2642876920 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0444138d80 0x7f044413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:28.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.559+0000 7f04491cc700 1 -- 172.21.15.3:0/2642876920 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2948948595 0 0) 0x7f04440a2a70 con 0x7f04440a0a30 2023-10-03T10:15:28.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.559+0000 7f04491cc700 1 -- 172.21.15.3:0/2642876920 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04300035b0 con 0x7f04440a0a30 2023-10-03T10:15:28.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.559+0000 7f04499cd700 1 -- 172.21.15.3:0/2642876920 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0444138d80 msgr2=0x7f044413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:28.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.559+0000 7f04499cd700 1 --2- 172.21.15.3:0/2642876920 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0444138d80 0x7f044413b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.559+0000 7f04499cd700 1 -- 172.21.15.3:0/2642876920 >> v1:172.21.15.3:6789/0 conn(0x7f04440a0a30 legacy=0x7f04440a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:28.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.559+0000 7f04499cd700 1 -- 172.21.15.3:0/2642876920 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f044409dc50 con 0x7f044409d630 2023-10-03T10:15:28.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.559+0000 7f04499cd700 1 --2- 172.21.15.3:0/2642876920 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f044409d630 0x7f04440a1510 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0434009840 tx=0x7f0434009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bdf016eac339bec server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:28.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.559+0000 7f04491cc700 1 -- 172.21.15.3:0/2642876920 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0434011830 con 0x7f044409d630 2023-10-03T10:15:28.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.559+0000 7f04491cc700 1 -- 172.21.15.3:0/2642876920 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0434011990 con 0x7f044409d630 2023-10-03T10:15:28.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.559+0000 7f04491cc700 1 -- 172.21.15.3:0/2642876920 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0434011d20 con 0x7f044409d630 2023-10-03T10:15:28.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.560+0000 7f044c432700 1 -- 172.21.15.3:0/2642876920 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f044409d630 msgr2=0x7f04440a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:28.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.560+0000 7f044c432700 1 --2- 172.21.15.3:0/2642876920 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f044409d630 0x7f04440a1510 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0434009840 tx=0x7f0434009b50 comp rx=0 tx=0).stop 2023-10-03T10:15:28.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.560+0000 7f044c432700 1 -- 172.21.15.3:0/2642876920 shutdown_connections 2023-10-03T10:15:28.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.560+0000 7f044c432700 1 --2- 172.21.15.3:0/2642876920 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0444138d80 0x7f044413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.560+0000 7f044c432700 1 --2- 172.21.15.3:0/2642876920 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f044409d630 0x7f04440a1510 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.560+0000 7f044c432700 1 -- 172.21.15.3:0/2642876920 >> 172.21.15.3:0/2642876920 conn(0x7f04440a3050 msgr2=0x7f04440a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:28.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.560+0000 7f044c432700 1 -- 172.21.15.3:0/2642876920 shutdown_connections 2023-10-03T10:15:28.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.560+0000 7f044c432700 1 -- 172.21.15.3:0/2642876920 wait complete. 2023-10-03T10:15:28.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.562+0000 7f044c432700 1 Processor -- start 2023-10-03T10:15:28.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.562+0000 7f044c432700 1 -- start start 2023-10-03T10:15:28.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.562+0000 7f044c432700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f044409d630 0x7f0444133c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:28.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.562+0000 7f044c432700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f04440a0a30 0x7f0444134170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:28.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.562+0000 7f044c432700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0444138d80 0x7f04441346b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:28.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.562+0000 7f044c432700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0444137200 con 0x7f044409d630 2023-10-03T10:15:28.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.562+0000 7f044c432700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04440a0410 con 0x7f0444138d80 2023-10-03T10:15:28.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.562+0000 7f044c432700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f04441373e0 con 0x7f04440a0a30 2023-10-03T10:15:28.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.563+0000 7f044a9cf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0444138d80 0x7f04441346b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:28.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.563+0000 7f044a9cf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0444138d80 0x7f04441346b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46644/0 (socket says 172.21.15.3:46644) 2023-10-03T10:15:28.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.563+0000 7f044a9cf700 1 -- 172.21.15.3:0/267974021 learned_addr learned my addr 172.21.15.3:0/267974021 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:28.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.563+0000 7f044a1ce700 1 --2- 172.21.15.3:0/267974021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f044409d630 0x7f0444133c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:28.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.563+0000 7f044a9cf700 1 -- 172.21.15.3:0/267974021 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f04440a0a30 msgr2=0x7f0444134170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:28.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.563+0000 7f044a9cf700 1 --2- 172.21.15.3:0/267974021 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f04440a0a30 0x7f0444134170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.563+0000 7f044a9cf700 1 -- 172.21.15.3:0/267974021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f044409d630 msgr2=0x7f0444133c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:28.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.563+0000 7f044a9cf700 1 --2- 172.21.15.3:0/267974021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f044409d630 0x7f0444133c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.563+0000 7f044a9cf700 1 -- 172.21.15.3:0/267974021 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04340094a0 con 0x7f0444138d80 2023-10-03T10:15:28.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.564+0000 7f044a9cf700 1 --2- 172.21.15.3:0/267974021 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0444138d80 0x7f04441346b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f043c009e30 tx=0x7f043c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:28.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.564+0000 7f043affd700 1 -- 172.21.15.3:0/267974021 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043c009ca0 con 0x7f0444138d80 2023-10-03T10:15:28.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.564+0000 7f043affd700 1 -- 172.21.15.3:0/267974021 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f043c00f040 con 0x7f0444138d80 2023-10-03T10:15:28.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.564+0000 7f044c432700 1 -- 172.21.15.3:0/267974021 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04441383b0 con 0x7f0444138d80 2023-10-03T10:15:28.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.564+0000 7f043affd700 1 -- 172.21.15.3:0/267974021 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043c014480 con 0x7f0444138d80 2023-10-03T10:15:28.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.564+0000 7f044c432700 1 -- 172.21.15.3:0/267974021 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0444135f20 con 0x7f0444138d80 2023-10-03T10:15:28.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.565+0000 7f043affd700 1 -- 172.21.15.3:0/267974021 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f043c0145e0 con 0x7f0444138d80 2023-10-03T10:15:28.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.565+0000 7f044c432700 1 -- 172.21.15.3:0/267974021 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04440747f0 con 0x7f0444138d80 2023-10-03T10:15:28.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.565+0000 7f043affd700 1 -- 172.21.15.3:0/267974021 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f043c0404e0 con 0x7f0444138d80 2023-10-03T10:15:28.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.575+0000 7f043affd700 1 -- 172.21.15.3:0/267974021 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f043c007900 con 0x7f0444138d80 2023-10-03T10:15:28.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.649+0000 7f6f37a73700 1 -- 172.21.15.3:0/1163768345 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6f30074ea0 con 0x7f6f3006a770 2023-10-03T10:15:28.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.650+0000 7f6f267fc700 1 -- 172.21.15.3:0/1163768345 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6f2c015070 con 0x7f6f3006a770 2023-10-03T10:15:28.650 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:28.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.652+0000 7f6f37a73700 1 -- 172.21.15.3:0/1163768345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f3006a770 msgr2=0x7f6f3013bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:28.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.652+0000 7f6f37a73700 1 --2- 172.21.15.3:0/1163768345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f3006a770 0x7f6f3013bb40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c000c00 tx=0x7f6f2c004750 comp rx=0 tx=0).stop 2023-10-03T10:15:28.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.652+0000 7f6f37a73700 1 -- 172.21.15.3:0/1163768345 shutdown_connections 2023-10-03T10:15:28.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.652+0000 7f6f37a73700 1 --2- 172.21.15.3:0/1163768345 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f3013c1e0 0x7f6f3013a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.652+0000 7f6f37a73700 1 --2- 172.21.15.3:0/1163768345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f3006a770 0x7f6f3013bb40 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.652+0000 7f6f37a73700 1 --2- 172.21.15.3:0/1163768345 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f3009dc30 0x7f6f3013a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.652+0000 7f6f37a73700 1 -- 172.21.15.3:0/1163768345 >> 172.21.15.3:0/1163768345 conn(0x7f6f30066f50 msgr2=0x7f6f300a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:28.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.652+0000 7f6f37a73700 1 -- 172.21.15.3:0/1163768345 shutdown_connections 2023-10-03T10:15:28.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.652+0000 7f6f37a73700 1 -- 172.21.15.3:0/1163768345 wait complete. 2023-10-03T10:15:28.667 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:28.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.738+0000 7f7716376700 1 -- 172.21.15.3:0/1385895279 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f771011ac20 con 0x7f771009d6f0 2023-10-03T10:15:28.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.738+0000 7f770cff9700 1 -- 172.21.15.3:0/1385895279 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7700017030 con 0x7f771009d6f0 2023-10-03T10:15:28.739 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:28.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.741+0000 7f7716376700 1 -- 172.21.15.3:0/1385895279 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f771009d6f0 msgr2=0x7f771011ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:28.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.741+0000 7f7716376700 1 --2- 172.21.15.3:0/1385895279 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f771009d6f0 0x7f771011ddb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f770000ec10 tx=0x7f770000ef20 comp rx=0 tx=0).stop 2023-10-03T10:15:28.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.741+0000 7f7716376700 1 -- 172.21.15.3:0/1385895279 shutdown_connections 2023-10-03T10:15:28.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.741+0000 7f7716376700 1 --2- 172.21.15.3:0/1385895279 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f771009e410 0x7f771011e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.741+0000 7f7716376700 1 --2- 172.21.15.3:0/1385895279 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f771009d6f0 0x7f771011ddb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.741+0000 7f7716376700 1 --2- 172.21.15.3:0/1385895279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f771009e000 0x7f771011e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.741+0000 7f7716376700 1 -- 172.21.15.3:0/1385895279 >> 172.21.15.3:0/1385895279 conn(0x7f7710066f50 msgr2=0x7f7710130800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:28.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.741+0000 7f7716376700 1 -- 172.21.15.3:0/1385895279 shutdown_connections 2023-10-03T10:15:28.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.741+0000 7f7716376700 1 -- 172.21.15.3:0/1385895279 wait complete. 2023-10-03T10:15:28.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.750+0000 7f2975415700 1 -- 172.21.15.3:0/2813768614 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f29701248c0 con 0x7f297009e410 2023-10-03T10:15:28.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.750+0000 7f29577fe700 1 -- 172.21.15.3:0/2813768614 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f296801c3b0 con 0x7f297009e410 2023-10-03T10:15:28.751 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:28.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.753+0000 7f2975415700 1 -- 172.21.15.3:0/2813768614 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f297009e410 msgr2=0x7f2970122f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:28.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.753+0000 7f2975415700 1 --2- 172.21.15.3:0/2813768614 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f297009e410 0x7f2970122f40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f296800efc0 tx=0x7f296800c3a0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.753+0000 7f2975415700 1 -- 172.21.15.3:0/2813768614 shutdown_connections 2023-10-03T10:15:28.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.753+0000 7f2975415700 1 --2- 172.21.15.3:0/2813768614 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f297009e410 0x7f2970122f40 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.753+0000 7f2975415700 1 --2- 172.21.15.3:0/2813768614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f297009d6f0 0x7f29701224c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.753+0000 7f2975415700 1 --2- 172.21.15.3:0/2813768614 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f297009e000 0x7f2970122a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.753+0000 7f2975415700 1 -- 172.21.15.3:0/2813768614 >> 172.21.15.3:0/2813768614 conn(0x7f2970066f50 msgr2=0x7f297006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:28.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.753+0000 7f2975415700 1 -- 172.21.15.3:0/2813768614 shutdown_connections 2023-10-03T10:15:28.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.753+0000 7f2975415700 1 -- 172.21.15.3:0/2813768614 wait complete. 2023-10-03T10:15:28.755 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:28.767 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:28.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.812+0000 7f044c432700 1 -- 172.21.15.3:0/267974021 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f04440a4c10 con 0x7f0444138d80 2023-10-03T10:15:28.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.812+0000 7f043affd700 1 -- 172.21.15.3:0/267974021 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f043c018030 con 0x7f0444138d80 2023-10-03T10:15:28.813 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:28.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.815+0000 7f044c432700 1 -- 172.21.15.3:0/267974021 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0444138d80 msgr2=0x7f04441346b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:28.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.815+0000 7f044c432700 1 --2- 172.21.15.3:0/267974021 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0444138d80 0x7f04441346b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f043c009e30 tx=0x7f043c00c3a0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.815+0000 7f044c432700 1 -- 172.21.15.3:0/267974021 shutdown_connections 2023-10-03T10:15:28.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.815+0000 7f044c432700 1 --2- 172.21.15.3:0/267974021 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0444138d80 0x7f04441346b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.815+0000 7f044c432700 1 --2- 172.21.15.3:0/267974021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f044409d630 0x7f0444133c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.815+0000 7f044c432700 1 --2- 172.21.15.3:0/267974021 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f04440a0a30 0x7f0444134170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:28.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.815+0000 7f044c432700 1 -- 172.21.15.3:0/267974021 >> 172.21.15.3:0/267974021 conn(0x7f04440a3050 msgr2=0x7f044409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:28.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.815+0000 7f044c432700 1 -- 172.21.15.3:0/267974021 shutdown_connections 2023-10-03T10:15:28.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:28.815+0000 7f044c432700 1 -- 172.21.15.3:0/267974021 wait complete. 2023-10-03T10:15:28.828 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:29.667 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:29.756 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:29.768 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:29.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.815+0000 7f87192dd700 1 Processor -- start 2023-10-03T10:15:29.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.815+0000 7f87192dd700 1 -- start start 2023-10-03T10:15:29.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.816+0000 7f87192dd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8714134630 0x7f8714136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:29.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.816+0000 7f87192dd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8714136f50 0x7f8714139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:29.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.816+0000 7f87192dd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871409b4f0 con 0x7f8714131810 2023-10-03T10:15:29.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.816+0000 7f87192dd700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87141313c0 con 0x7f8714134630 2023-10-03T10:15:29.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.816+0000 7f87192dd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871406e8c0 con 0x7f8714136f50 2023-10-03T10:15:29.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.816+0000 7f87137fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8714136f50 0x7f8714139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:29.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.816+0000 7f87137fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8714136f50 0x7f8714139330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46654/0 (socket says 172.21.15.3:46654) 2023-10-03T10:15:29.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.816+0000 7f87137fe700 1 -- 172.21.15.3:0/2449878340 learned_addr learned my addr 172.21.15.3:0/2449878340 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:29.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.816+0000 7f87127fc700 1 --2- 172.21.15.3:0/2449878340 >> v2:172.21.15.3:3300/0 conn(0x7f8714134630 0x7f8714136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:29.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.817+0000 7f87137fe700 1 -- 172.21.15.3:0/2449878340 >> v2:172.21.15.3:3300/0 conn(0x7f8714134630 msgr2=0x7f8714136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:29.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.817+0000 7f87137fe700 1 --2- 172.21.15.3:0/2449878340 >> v2:172.21.15.3:3300/0 conn(0x7f8714134630 0x7f8714136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:29.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.817+0000 7f87137fe700 1 -- 172.21.15.3:0/2449878340 >> v1:172.21.15.3:6789/0 conn(0x7f8714131810 legacy=0x7f8714131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:29.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.817+0000 7f87137fe700 1 -- 172.21.15.3:0/2449878340 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8714071d20 con 0x7f8714136f50 2023-10-03T10:15:29.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.817+0000 7f87137fe700 1 --2- 172.21.15.3:0/2449878340 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8714136f50 0x7f8714139330 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f870c009e50 tx=0x7f870c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef509bbced02833f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:29.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.817+0000 7f8711ffb700 1 -- 172.21.15.3:0/2449878340 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c00e070 con 0x7f8714136f50 2023-10-03T10:15:29.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.817+0000 7f8711ffb700 1 -- 172.21.15.3:0/2449878340 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870c00b9c0 con 0x7f8714136f50 2023-10-03T10:15:29.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.817+0000 7f8711ffb700 1 -- 172.21.15.3:0/2449878340 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c00bd70 con 0x7f8714136f50 2023-10-03T10:15:29.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.818+0000 7f87192dd700 1 -- 172.21.15.3:0/2449878340 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8714136f50 msgr2=0x7f8714139330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:29.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.818+0000 7f87192dd700 1 --2- 172.21.15.3:0/2449878340 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8714136f50 0x7f8714139330 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f870c009e50 tx=0x7f870c00b330 comp rx=0 tx=0).stop 2023-10-03T10:15:29.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.818+0000 7f87192dd700 1 -- 172.21.15.3:0/2449878340 shutdown_connections 2023-10-03T10:15:29.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.818+0000 7f87192dd700 1 --2- 172.21.15.3:0/2449878340 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8714136f50 0x7f8714139330 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:29.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.818+0000 7f87192dd700 1 --2- 172.21.15.3:0/2449878340 >> v2:172.21.15.3:3300/0 conn(0x7f8714134630 0x7f8714136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:29.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.818+0000 7f87192dd700 1 -- 172.21.15.3:0/2449878340 >> 172.21.15.3:0/2449878340 conn(0x7f87140a2aa0 msgr2=0x7f87140a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:29.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.818+0000 7f87192dd700 1 -- 172.21.15.3:0/2449878340 shutdown_connections 2023-10-03T10:15:29.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.818+0000 7f87192dd700 1 -- 172.21.15.3:0/2449878340 wait complete. 2023-10-03T10:15:29.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.819+0000 7f87192dd700 1 Processor -- start 2023-10-03T10:15:29.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.819+0000 7f87192dd700 1 -- start start 2023-10-03T10:15:29.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.820+0000 7f87192dd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8714131810 0x7f871412ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:29.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.820+0000 7f87192dd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8714134630 0x7f871412e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:29.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.820+0000 7f87192dd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8714136f50 0x7f87141c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:29.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.820+0000 7f87192dd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8714136b70 con 0x7f8714134630 2023-10-03T10:15:29.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.820+0000 7f87192dd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8714071220 con 0x7f8714136f50 2023-10-03T10:15:29.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.820+0000 7f87192dd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f871411e4e0 con 0x7f8714131810 2023-10-03T10:15:29.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.820+0000 7f87127fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8714134630 0x7f871412e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:29.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.820+0000 7f87137fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8714136f50 0x7f87141c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:29.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.820+0000 7f87127fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8714134630 0x7f871412e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45274/0 (socket says 172.21.15.3:45274) 2023-10-03T10:15:29.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.820+0000 7f87137fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8714136f50 0x7f87141c8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46664/0 (socket says 172.21.15.3:46664) 2023-10-03T10:15:29.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.820+0000 7f87127fc700 1 -- 172.21.15.3:0/3985426106 learned_addr learned my addr 172.21.15.3:0/3985426106 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:29.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.820+0000 7f8712ffd700 1 --2- 172.21.15.3:0/3985426106 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8714131810 0x7f871412ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:29.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.820+0000 7f87127fc700 1 -- 172.21.15.3:0/3985426106 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8714131810 msgr2=0x7f871412ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:29.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.820+0000 7f87127fc700 1 --2- 172.21.15.3:0/3985426106 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8714131810 0x7f871412ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:29.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.820+0000 7f87127fc700 1 -- 172.21.15.3:0/3985426106 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8714136f50 msgr2=0x7f87141c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:29.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.820+0000 7f87127fc700 1 --2- 172.21.15.3:0/3985426106 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8714136f50 0x7f87141c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:29.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.820+0000 7f87127fc700 1 -- 172.21.15.3:0/3985426106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870c009b30 con 0x7f8714134630 2023-10-03T10:15:29.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.820+0000 7f87137fe700 1 --2- 172.21.15.3:0/3985426106 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8714136f50 0x7f87141c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:29.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.820+0000 7f87127fc700 1 --2- 172.21.15.3:0/3985426106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8714134630 0x7f871412e300 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f870800e7e0 tx=0x7f870800eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:29.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.821+0000 7f86fb7fe700 1 -- 172.21.15.3:0/3985426106 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870800c070 con 0x7f8714134630 2023-10-03T10:15:29.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.821+0000 7f86fb7fe700 1 -- 172.21.15.3:0/3985426106 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8708013650 con 0x7f8714134630 2023-10-03T10:15:29.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.821+0000 7f87192dd700 1 -- 172.21.15.3:0/3985426106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f871411d360 con 0x7f8714134630 2023-10-03T10:15:29.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.821+0000 7f86fb7fe700 1 -- 172.21.15.3:0/3985426106 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8708013a00 con 0x7f8714134630 2023-10-03T10:15:29.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.821+0000 7f87192dd700 1 -- 172.21.15.3:0/3985426106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f871406eb00 con 0x7f8714134630 2023-10-03T10:15:29.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.821+0000 7f86fb7fe700 1 -- 172.21.15.3:0/3985426106 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8708013b60 con 0x7f8714134630 2023-10-03T10:15:29.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.822+0000 7f86fb7fe700 1 -- 172.21.15.3:0/3985426106 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f870803f220 con 0x7f8714134630 2023-10-03T10:15:29.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.822+0000 7f87192dd700 1 -- 172.21.15.3:0/3985426106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8714074ea0 con 0x7f8714134630 2023-10-03T10:15:29.829 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:29.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.829+0000 7f86fb7fe700 1 -- 172.21.15.3:0/3985426106 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f870801b3b0 con 0x7f8714134630 2023-10-03T10:15:29.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.909+0000 7fc95a60c700 1 Processor -- start 2023-10-03T10:15:29.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.909+0000 7fc95a60c700 1 -- start start 2023-10-03T10:15:29.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.910+0000 7fc95a60c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc95409d6f0 0x7fc95409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:29.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.910+0000 7fc95a60c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc95409e410 0x7fc954130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:29.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.910+0000 7fc95a60c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc95409f130 con 0x7fc95409e000 2023-10-03T10:15:29.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.910+0000 7fc95a60c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc95409f520 con 0x7fc95409d6f0 2023-10-03T10:15:29.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.910+0000 7fc95a60c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95409e930 con 0x7fc95409e410 2023-10-03T10:15:29.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.910+0000 7fc953fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc95409d6f0 0x7fc95409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:29.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.910+0000 7fc958ba9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc95409e410 0x7fc954130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:29.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.910+0000 7fc94bfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc95409e000 0x7fc954130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39720/0 (socket says 172.21.15.3:39720) 2023-10-03T10:15:29.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.910+0000 7fc94bfff700 1 -- 172.21.15.3:0/521267102 learned_addr learned my addr 172.21.15.3:0/521267102 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:29.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.911+0000 7fc958ba9700 1 -- 172.21.15.3:0/521267102 >> v2:172.21.15.3:3300/0 conn(0x7fc95409d6f0 msgr2=0x7fc95409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:29.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.911+0000 7fc958ba9700 1 --2- 172.21.15.3:0/521267102 >> v2:172.21.15.3:3300/0 conn(0x7fc95409d6f0 0x7fc95409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:29.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.911+0000 7fc958ba9700 1 -- 172.21.15.3:0/521267102 >> v1:172.21.15.3:6789/0 conn(0x7fc95409e000 legacy=0x7fc954130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:29.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.911+0000 7fc958ba9700 1 -- 172.21.15.3:0/521267102 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95409b2d0 con 0x7fc95409e410 2023-10-03T10:15:29.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.911+0000 7fc958ba9700 1 --2- 172.21.15.3:0/521267102 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc95409e410 0x7fc954130d70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009d90 tx=0x7fc94c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb638599f826301a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:29.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.911+0000 7fc9537fe700 1 -- 172.21.15.3:0/521267102 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c00e070 con 0x7fc95409e410 2023-10-03T10:15:29.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.911+0000 7fc9537fe700 1 -- 172.21.15.3:0/521267102 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94c00bb20 con 0x7fc95409e410 2023-10-03T10:15:29.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.911+0000 7fc9537fe700 1 -- 172.21.15.3:0/521267102 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c012430 con 0x7fc95409e410 2023-10-03T10:15:29.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.912+0000 7fc95a60c700 1 -- 172.21.15.3:0/521267102 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc95409e410 msgr2=0x7fc954130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:29.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.912+0000 7fc95a60c700 1 --2- 172.21.15.3:0/521267102 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc95409e410 0x7fc954130d70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009d90 tx=0x7fc94c00b330 comp rx=0 tx=0).stop 2023-10-03T10:15:29.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.912+0000 7fc95a60c700 1 -- 172.21.15.3:0/521267102 shutdown_connections 2023-10-03T10:15:29.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.912+0000 7fc95a60c700 1 --2- 172.21.15.3:0/521267102 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc95409e410 0x7fc954130d70 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:29.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.912+0000 7fc95a60c700 1 --2- 172.21.15.3:0/521267102 >> v2:172.21.15.3:3300/0 conn(0x7fc95409d6f0 0x7fc95409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:29.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.912+0000 7fc95a60c700 1 -- 172.21.15.3:0/521267102 >> 172.21.15.3:0/521267102 conn(0x7fc954066f50 msgr2=0x7fc95409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:29.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.913+0000 7fc95a60c700 1 -- 172.21.15.3:0/521267102 shutdown_connections 2023-10-03T10:15:29.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.913+0000 7fc95a60c700 1 -- 172.21.15.3:0/521267102 wait complete. 2023-10-03T10:15:29.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.914+0000 7fc95a60c700 1 Processor -- start 2023-10-03T10:15:29.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.914+0000 7fc95a60c700 1 -- start start 2023-10-03T10:15:29.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.915+0000 7fc95a60c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc95409d6f0 0x7fc954120090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:29.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.915+0000 7fc95a60c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc95409e000 0x7fc9541205d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:29.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.915+0000 7fc95a60c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc95409e410 0x7fc954120b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:29.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.915+0000 7fc95a60c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9541222a0 con 0x7fc95409e410 2023-10-03T10:15:29.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.915+0000 7fc95a60c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9540a1ca0 con 0x7fc95409d6f0 2023-10-03T10:15:29.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.915+0000 7fc95a60c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9541232f0 con 0x7fc95409e000 2023-10-03T10:15:29.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.915+0000 7fc953fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc95409d6f0 0x7fc954120090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:29.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.915+0000 7fc953fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc95409d6f0 0x7fc954120090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46682/0 (socket says 172.21.15.3:46682) 2023-10-03T10:15:29.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.915+0000 7fc953fff700 1 -- 172.21.15.3:0/97003145 learned_addr learned my addr 172.21.15.3:0/97003145 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:29.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.915+0000 7fc94bfff700 1 --2- 172.21.15.3:0/97003145 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc95409e000 0x7fc9541205d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:29.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.915+0000 7fc958ba9700 1 --2- 172.21.15.3:0/97003145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc95409e410 0x7fc954120b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:29.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.915+0000 7fc94bfff700 1 -- 172.21.15.3:0/97003145 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc95409d6f0 msgr2=0x7fc954120090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:29.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.915+0000 7fc94bfff700 1 --2- 172.21.15.3:0/97003145 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc95409d6f0 0x7fc954120090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:29.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.916+0000 7fc94bfff700 1 -- 172.21.15.3:0/97003145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc95409e410 msgr2=0x7fc954120b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:29.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.916+0000 7fc94bfff700 1 --2- 172.21.15.3:0/97003145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc95409e410 0x7fc954120b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:29.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.916+0000 7fc94bfff700 1 -- 172.21.15.3:0/97003145 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc94c009a70 con 0x7fc95409e000 2023-10-03T10:15:29.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.916+0000 7fc953fff700 1 --2- 172.21.15.3:0/97003145 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc95409d6f0 0x7fc954120090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:29.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.916+0000 7fc94bfff700 1 --2- 172.21.15.3:0/97003145 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc95409e000 0x7fc9541205d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc93c00c500 tx=0x7fc93c00c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:29.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.916+0000 7fc9517fa700 1 -- 172.21.15.3:0/97003145 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c005d70 con 0x7fc95409e000 2023-10-03T10:15:29.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.916+0000 7fc9517fa700 1 -- 172.21.15.3:0/97003145 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93c0070d0 con 0x7fc95409e000 2023-10-03T10:15:29.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.916+0000 7fc9517fa700 1 -- 172.21.15.3:0/97003145 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c007440 con 0x7fc95409e000 2023-10-03T10:15:29.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.916+0000 7fc95a60c700 1 -- 172.21.15.3:0/97003145 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc954123c00 con 0x7fc95409e000 2023-10-03T10:15:29.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.916+0000 7fc95a60c700 1 -- 172.21.15.3:0/97003145 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc954123880 con 0x7fc95409e000 2023-10-03T10:15:29.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.917+0000 7fc9517fa700 1 -- 172.21.15.3:0/97003145 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc93c0075a0 con 0x7fc95409e000 2023-10-03T10:15:29.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.918+0000 7fc95a60c700 1 -- 172.21.15.3:0/97003145 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc954074ea0 con 0x7fc95409e000 2023-10-03T10:15:29.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.918+0000 7fc9517fa700 1 -- 172.21.15.3:0/97003145 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc93c03ea00 con 0x7fc95409e000 2023-10-03T10:15:29.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.926+0000 7fc9517fa700 1 -- 172.21.15.3:0/97003145 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc93c01b8d0 con 0x7fc95409e000 2023-10-03T10:15:29.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.956+0000 7f8a0d252700 1 Processor -- start 2023-10-03T10:15:29.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.956+0000 7f8a0d252700 1 -- start start 2023-10-03T10:15:29.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.957+0000 7f8a0d252700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a08133420 0x7f8a081337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:29.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.957+0000 7f8a0d252700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a08134260 0x7f8a081306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:29.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.957+0000 7f8a0d252700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a0806b990 con 0x7f8a08133e30 2023-10-03T10:15:29.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.957+0000 7f8a0d252700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a080a0590 con 0x7f8a08134260 2023-10-03T10:15:29.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.957+0000 7f8a0d252700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0809fb80 con 0x7f8a08133420 2023-10-03T10:15:29.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.957+0000 7f8a06d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a08133420 0x7f8a081337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:29.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.957+0000 7f8a0659c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8a08133e30 0x7f8a0812ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39734/0 (socket says 172.21.15.3:39734) 2023-10-03T10:15:29.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.957+0000 7f8a0659c700 1 -- 172.21.15.3:0/849319135 learned_addr learned my addr 172.21.15.3:0/849319135 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:29.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.957+0000 7f8a0759e700 1 --2- 172.21.15.3:0/849319135 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a08134260 0x7f8a081306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:29.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.957+0000 7f8a06d9d700 1 -- 172.21.15.3:0/849319135 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a08134260 msgr2=0x7f8a081306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:29.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.957+0000 7f8a06d9d700 1 --2- 172.21.15.3:0/849319135 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a08134260 0x7f8a081306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:29.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.957+0000 7f8a06d9d700 1 -- 172.21.15.3:0/849319135 >> v1:172.21.15.3:6789/0 conn(0x7f8a08133e30 legacy=0x7f8a0812ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:29.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.958+0000 7f8a06d9d700 1 -- 172.21.15.3:0/849319135 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a080a0030 con 0x7f8a08133420 2023-10-03T10:15:29.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.957+0000 7f8a05d9b700 1 -- 172.21.15.3:0/849319135 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2353124709 0 0) 0x7f8a0806b990 con 0x7f8a08133e30 2023-10-03T10:15:29.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.958+0000 7f8a0759e700 1 --2- 172.21.15.3:0/849319135 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a08134260 0x7f8a081306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:29.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.958+0000 7f8a06d9d700 1 --2- 172.21.15.3:0/849319135 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a08133420 0x7f8a081337f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009870 tx=0x7f89f8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5eb748412b2020 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:29.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.958+0000 7f8a05d9b700 1 -- 172.21.15.3:0/849319135 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f8011830 con 0x7f8a08133420 2023-10-03T10:15:29.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.958+0000 7f8a05d9b700 1 -- 172.21.15.3:0/849319135 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f8011990 con 0x7f8a08133420 2023-10-03T10:15:29.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.958+0000 7f8a05d9b700 1 -- 172.21.15.3:0/849319135 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f8011d20 con 0x7f8a08133420 2023-10-03T10:15:29.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.959+0000 7f8a0d252700 1 -- 172.21.15.3:0/849319135 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a08133420 msgr2=0x7f8a081337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:29.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.959+0000 7f8a0d252700 1 --2- 172.21.15.3:0/849319135 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a08133420 0x7f8a081337f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009870 tx=0x7f89f8009b80 comp rx=0 tx=0).stop 2023-10-03T10:15:29.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.959+0000 7f8a0d252700 1 -- 172.21.15.3:0/849319135 shutdown_connections 2023-10-03T10:15:29.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.959+0000 7f8a0d252700 1 --2- 172.21.15.3:0/849319135 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a08134260 0x7f8a081306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:29.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.959+0000 7f8a0d252700 1 --2- 172.21.15.3:0/849319135 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a08133420 0x7f8a081337f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:29.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.959+0000 7f8a0d252700 1 -- 172.21.15.3:0/849319135 >> 172.21.15.3:0/849319135 conn(0x7f8a080a4210 msgr2=0x7f8a080a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:29.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.959+0000 7f8a0d252700 1 -- 172.21.15.3:0/849319135 shutdown_connections 2023-10-03T10:15:29.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.959+0000 7f8a0d252700 1 -- 172.21.15.3:0/849319135 wait complete. 2023-10-03T10:15:29.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.961+0000 7f8a0d252700 1 Processor -- start 2023-10-03T10:15:29.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.961+0000 7f8a0d252700 1 -- start start 2023-10-03T10:15:29.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.962+0000 7f8a0d252700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a08133420 0x7f8a0809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:29.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.962+0000 7f8a0d252700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a08133e30 0x7f8a0809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:29.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.962+0000 7f8a0d252700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a08134260 0x7f8a0809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:29.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.962+0000 7f8a0d252700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0812d960 con 0x7f8a08133420 2023-10-03T10:15:29.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.962+0000 7f8a0d252700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0811ec20 con 0x7f8a08134260 2023-10-03T10:15:29.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.962+0000 7f8a0d252700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a081201e0 con 0x7f8a08133e30 2023-10-03T10:15:29.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.962+0000 7f8a06d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a08133420 0x7f8a0809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:29.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.962+0000 7f8a06d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a08133420 0x7f8a0809c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45308/0 (socket says 172.21.15.3:45308) 2023-10-03T10:15:29.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.962+0000 7f8a06d9d700 1 -- 172.21.15.3:0/197630134 learned_addr learned my addr 172.21.15.3:0/197630134 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:29.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.962+0000 7f8a0659c700 1 --2- 172.21.15.3:0/197630134 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a08133e30 0x7f8a0809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:29.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.962+0000 7f8a0759e700 1 --2- 172.21.15.3:0/197630134 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a08134260 0x7f8a0809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:29.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.962+0000 7f8a06d9d700 1 -- 172.21.15.3:0/197630134 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a08133e30 msgr2=0x7f8a0809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:29.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.962+0000 7f8a06d9d700 1 --2- 172.21.15.3:0/197630134 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a08133e30 0x7f8a0809c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:29.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.962+0000 7f8a06d9d700 1 -- 172.21.15.3:0/197630134 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a08134260 msgr2=0x7f8a0809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:29.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.962+0000 7f8a06d9d700 1 --2- 172.21.15.3:0/197630134 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a08134260 0x7f8a0809cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:29.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.962+0000 7f8a06d9d700 1 -- 172.21.15.3:0/197630134 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a00009a70 con 0x7f8a08133420 2023-10-03T10:15:29.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.962+0000 7f8a0759e700 1 --2- 172.21.15.3:0/197630134 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a08134260 0x7f8a0809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:29.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.963+0000 7f8a06d9d700 1 --2- 172.21.15.3:0/197630134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a08133420 0x7f8a0809c480 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009840 tx=0x7f89f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:29.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.963+0000 7f89ef7fe700 1 -- 172.21.15.3:0/197630134 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f80045d0 con 0x7f8a08133420 2023-10-03T10:15:29.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.963+0000 7f89ef7fe700 1 -- 172.21.15.3:0/197630134 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f801f070 con 0x7f8a08133420 2023-10-03T10:15:29.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.963+0000 7f8a0d252700 1 -- 172.21.15.3:0/197630134 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f80094a0 con 0x7f8a08133420 2023-10-03T10:15:29.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.963+0000 7f89ef7fe700 1 -- 172.21.15.3:0/197630134 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f80038c0 con 0x7f8a08133420 2023-10-03T10:15:29.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.963+0000 7f8a0d252700 1 -- 172.21.15.3:0/197630134 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a08131860 con 0x7f8a08133420 2023-10-03T10:15:29.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.964+0000 7f89ef7fe700 1 -- 172.21.15.3:0/197630134 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f89f8003a20 con 0x7f8a08133420 2023-10-03T10:15:29.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.964+0000 7f89ef7fe700 1 -- 172.21.15.3:0/197630134 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f89f8021070 con 0x7f8a08133420 2023-10-03T10:15:29.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.964+0000 7f8a0d252700 1 -- 172.21.15.3:0/197630134 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a08074ea0 con 0x7f8a08133420 2023-10-03T10:15:29.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.973+0000 7f6592881700 1 Processor -- start 2023-10-03T10:15:29.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.973+0000 7f6592881700 1 -- start start 2023-10-03T10:15:29.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.974+0000 7f6592881700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f658c09d6f0 0x7f658c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:29.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.974+0000 7f89ef7fe700 1 -- 172.21.15.3:0/197630134 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f89f8015020 con 0x7f8a08133420 2023-10-03T10:15:29.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.974+0000 7f6592881700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f658c09e000 0x7f658c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:29.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.974+0000 7f6592881700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f658c09e410 0x7f658c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:29.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.974+0000 7f6592881700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f658c09f520 con 0x7f658c09e410 2023-10-03T10:15:29.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.974+0000 7f6592881700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f658c09f130 con 0x7f658c09e000 2023-10-03T10:15:29.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.974+0000 7f6592881700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f658c09e930 con 0x7f658c09d6f0 2023-10-03T10:15:29.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.974+0000 7f6590e1e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f658c09e410 0x7f658c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:29.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.974+0000 7f658bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f658c09d6f0 0x7f658c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:29.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.974+0000 7f6590e1e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f658c09e410 0x7f658c130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45316/0 (socket says 172.21.15.3:45316) 2023-10-03T10:15:29.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.974+0000 7f6590e1e700 1 -- 172.21.15.3:0/3325961732 learned_addr learned my addr 172.21.15.3:0/3325961732 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:29.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.974+0000 7f658b7fe700 1 --2- 172.21.15.3:0/3325961732 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f658c09e000 0x7f658c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:29.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.975+0000 7f658bfff700 1 -- 172.21.15.3:0/3325961732 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f658c09e000 msgr2=0x7f658c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:29.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.975+0000 7f658bfff700 1 --2- 172.21.15.3:0/3325961732 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f658c09e000 0x7f658c130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:29.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.975+0000 7f658bfff700 1 -- 172.21.15.3:0/3325961732 >> v2:172.21.15.3:3300/0 conn(0x7f658c09e410 msgr2=0x7f658c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:29.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.975+0000 7f658bfff700 1 --2- 172.21.15.3:0/3325961732 >> v2:172.21.15.3:3300/0 conn(0x7f658c09e410 0x7f658c130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:29.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.975+0000 7f658bfff700 1 -- 172.21.15.3:0/3325961732 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f658c09b2d0 con 0x7f658c09d6f0 2023-10-03T10:15:29.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.975+0000 7f6590e1e700 1 --2- 172.21.15.3:0/3325961732 >> v2:172.21.15.3:3300/0 conn(0x7f658c09e410 0x7f658c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:29.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.976+0000 7f658bfff700 1 --2- 172.21.15.3:0/3325961732 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f658c09d6f0 0x7f658c09dac0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f657c009870 tx=0x7f657c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5e89de21e0c17b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:29.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.976+0000 7f658affd700 1 -- 172.21.15.3:0/3325961732 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657c00f8a0 con 0x7f658c09d6f0 2023-10-03T10:15:29.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.976+0000 7f658affd700 1 -- 172.21.15.3:0/3325961732 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657c00fa00 con 0x7f658c09d6f0 2023-10-03T10:15:29.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.976+0000 7f658affd700 1 -- 172.21.15.3:0/3325961732 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657c00fd90 con 0x7f658c09d6f0 2023-10-03T10:15:29.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.977+0000 7f6592881700 1 -- 172.21.15.3:0/3325961732 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f658c09d6f0 msgr2=0x7f658c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:29.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.977+0000 7f6592881700 1 --2- 172.21.15.3:0/3325961732 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f658c09d6f0 0x7f658c09dac0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f657c009870 tx=0x7f657c009b80 comp rx=0 tx=0).stop 2023-10-03T10:15:29.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.977+0000 7f6592881700 1 -- 172.21.15.3:0/3325961732 shutdown_connections 2023-10-03T10:15:29.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.977+0000 7f6592881700 1 --2- 172.21.15.3:0/3325961732 >> v2:172.21.15.3:3300/0 conn(0x7f658c09e410 0x7f658c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:29.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.977+0000 7f6592881700 1 --2- 172.21.15.3:0/3325961732 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f658c09d6f0 0x7f658c09dac0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:29.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.977+0000 7f6592881700 1 --2- 172.21.15.3:0/3325961732 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f658c09e000 0x7f658c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:29.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.977+0000 7f6592881700 1 -- 172.21.15.3:0/3325961732 >> 172.21.15.3:0/3325961732 conn(0x7f658c066f50 msgr2=0x7f658c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:29.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.977+0000 7f6592881700 1 -- 172.21.15.3:0/3325961732 shutdown_connections 2023-10-03T10:15:29.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.977+0000 7f6592881700 1 -- 172.21.15.3:0/3325961732 wait complete. 2023-10-03T10:15:29.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.978+0000 7f6592881700 1 Processor -- start 2023-10-03T10:15:29.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.978+0000 7f6592881700 1 -- start start 2023-10-03T10:15:29.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.979+0000 7f6592881700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f658c09d6f0 0x7f658c11e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:29.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.979+0000 7f6592881700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f658c09e000 0x7f658c11e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:29.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.979+0000 7f6592881700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f658c09e410 0x7f658c11eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:29.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.979+0000 7f6592881700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658c09c7b0 con 0x7f658c09e410 2023-10-03T10:15:29.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.979+0000 7f6592881700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f658c120c10 con 0x7f658c09e000 2023-10-03T10:15:29.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.979+0000 7f6592881700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f658c11fdc0 con 0x7f658c09d6f0 2023-10-03T10:15:29.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.979+0000 7f658bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f658c09d6f0 0x7f658c11e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:29.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.979+0000 7f658b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f658c09e000 0x7f658c11e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:29.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.979+0000 7f658bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f658c09d6f0 0x7f658c11e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43736/0 (socket says 172.21.15.3:43736) 2023-10-03T10:15:29.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.979+0000 7f658bfff700 1 -- 172.21.15.3:0/2329748555 learned_addr learned my addr 172.21.15.3:0/2329748555 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:29.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.979+0000 7f6590e1e700 1 --2- 172.21.15.3:0/2329748555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f658c09e410 0x7f658c11eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:29.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.979+0000 7f658b7fe700 1 -- 172.21.15.3:0/2329748555 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f658c09d6f0 msgr2=0x7f658c11e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:29.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.979+0000 7f658b7fe700 1 --2- 172.21.15.3:0/2329748555 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f658c09d6f0 0x7f658c11e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:29.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.979+0000 7f658b7fe700 1 -- 172.21.15.3:0/2329748555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f658c09e410 msgr2=0x7f658c11eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:29.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.979+0000 7f658b7fe700 1 --2- 172.21.15.3:0/2329748555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f658c09e410 0x7f658c11eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:29.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.979+0000 7f658b7fe700 1 -- 172.21.15.3:0/2329748555 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6584009b30 con 0x7f658c09e000 2023-10-03T10:15:29.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.980+0000 7f658b7fe700 1 --2- 172.21.15.3:0/2329748555 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f658c09e000 0x7f658c11e650 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f658000e870 tx=0x7f658000eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:29.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.980+0000 7f6588ff9700 1 -- 172.21.15.3:0/2329748555 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6580013680 con 0x7f658c09e000 2023-10-03T10:15:29.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.980+0000 7f6588ff9700 1 -- 172.21.15.3:0/2329748555 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65800137e0 con 0x7f658c09e000 2023-10-03T10:15:29.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.980+0000 7f6588ff9700 1 -- 172.21.15.3:0/2329748555 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6580013b70 con 0x7f658c09e000 2023-10-03T10:15:29.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.980+0000 7f6592881700 1 -- 172.21.15.3:0/2329748555 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f657c0094a0 con 0x7f658c09e000 2023-10-03T10:15:29.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.980+0000 7f6592881700 1 -- 172.21.15.3:0/2329748555 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f658c121c30 con 0x7f658c09e000 2023-10-03T10:15:29.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.981+0000 7f6588ff9700 1 -- 172.21.15.3:0/2329748555 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6580013cd0 con 0x7f658c09e000 2023-10-03T10:15:29.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.981+0000 7f6592881700 1 -- 172.21.15.3:0/2329748555 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f658c074ea0 con 0x7f658c09e000 2023-10-03T10:15:29.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.981+0000 7f6588ff9700 1 -- 172.21.15.3:0/2329748555 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6580010d30 con 0x7f658c09e000 2023-10-03T10:15:29.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:29.986+0000 7f6588ff9700 1 -- 172.21.15.3:0/2329748555 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6580017070 con 0x7f658c09e000 2023-10-03T10:15:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.109+0000 7f87192dd700 1 -- 172.21.15.3:0/3985426106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f871406b9e0 con 0x7f8714134630 2023-10-03T10:15:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.109+0000 7f86fb7fe700 1 -- 172.21.15.3:0/3985426106 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f87080112f0 con 0x7f8714134630 2023-10-03T10:15:30.110 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:30.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.112+0000 7f87192dd700 1 -- 172.21.15.3:0/3985426106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8714134630 msgr2=0x7f871412e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:30.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.112+0000 7f87192dd700 1 --2- 172.21.15.3:0/3985426106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8714134630 0x7f871412e300 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f870800e7e0 tx=0x7f870800eaf0 comp rx=0 tx=0).stop 2023-10-03T10:15:30.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.113+0000 7f87192dd700 1 -- 172.21.15.3:0/3985426106 shutdown_connections 2023-10-03T10:15:30.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.113+0000 7f87192dd700 1 --2- 172.21.15.3:0/3985426106 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8714136f50 0x7f87141c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:30.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.113+0000 7f87192dd700 1 --2- 172.21.15.3:0/3985426106 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8714131810 0x7f871412ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:30.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.113+0000 7f87192dd700 1 --2- 172.21.15.3:0/3985426106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8714134630 0x7f871412e300 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:30.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.113+0000 7f87192dd700 1 -- 172.21.15.3:0/3985426106 >> 172.21.15.3:0/3985426106 conn(0x7f87140a2aa0 msgr2=0x7f871412f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:30.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.113+0000 7f87192dd700 1 -- 172.21.15.3:0/3985426106 shutdown_connections 2023-10-03T10:15:30.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.113+0000 7f87192dd700 1 -- 172.21.15.3:0/3985426106 wait complete. 2023-10-03T10:15:30.132 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:30.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.152+0000 7fc95a60c700 1 -- 172.21.15.3:0/97003145 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc95406c360 con 0x7fc95409e000 2023-10-03T10:15:30.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.152+0000 7fc9517fa700 1 -- 172.21.15.3:0/97003145 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc93c017030 con 0x7fc95409e000 2023-10-03T10:15:30.153 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:30.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.155+0000 7fc95a60c700 1 -- 172.21.15.3:0/97003145 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc95409e000 msgr2=0x7fc9541205d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:30.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.155+0000 7fc95a60c700 1 --2- 172.21.15.3:0/97003145 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc95409e000 0x7fc9541205d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc93c00c500 tx=0x7fc93c00c810 comp rx=0 tx=0).stop 2023-10-03T10:15:30.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.155+0000 7fc95a60c700 1 -- 172.21.15.3:0/97003145 shutdown_connections 2023-10-03T10:15:30.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.155+0000 7fc95a60c700 1 --2- 172.21.15.3:0/97003145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc95409e410 0x7fc954120b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:30.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.155+0000 7fc95a60c700 1 --2- 172.21.15.3:0/97003145 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc95409d6f0 0x7fc954120090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:30.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.155+0000 7fc95a60c700 1 --2- 172.21.15.3:0/97003145 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc95409e000 0x7fc9541205d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:30.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.155+0000 7fc95a60c700 1 -- 172.21.15.3:0/97003145 >> 172.21.15.3:0/97003145 conn(0x7fc954066f50 msgr2=0x7fc95406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:30.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.155+0000 7fc95a60c700 1 -- 172.21.15.3:0/97003145 shutdown_connections 2023-10-03T10:15:30.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.155+0000 7fc95a60c700 1 -- 172.21.15.3:0/97003145 wait complete. 2023-10-03T10:15:30.169 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:30.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.204+0000 7f6592881700 1 -- 172.21.15.3:0/2329748555 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f658c1303b0 con 0x7f658c09e000 2023-10-03T10:15:30.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.204+0000 7f6588ff9700 1 -- 172.21.15.3:0/2329748555 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6580017030 con 0x7f658c09e000 2023-10-03T10:15:30.204 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:30.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.206+0000 7f6592881700 1 -- 172.21.15.3:0/2329748555 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f658c09e000 msgr2=0x7f658c11e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:30.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.206+0000 7f6592881700 1 --2- 172.21.15.3:0/2329748555 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f658c09e000 0x7f658c11e650 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f658000e870 tx=0x7f658000eb80 comp rx=0 tx=0).stop 2023-10-03T10:15:30.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.206+0000 7f6592881700 1 -- 172.21.15.3:0/2329748555 shutdown_connections 2023-10-03T10:15:30.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.206+0000 7f6592881700 1 --2- 172.21.15.3:0/2329748555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f658c09e410 0x7f658c11eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:30.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.206+0000 7f6592881700 1 --2- 172.21.15.3:0/2329748555 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f658c09d6f0 0x7f658c11e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:30.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.206+0000 7f6592881700 1 --2- 172.21.15.3:0/2329748555 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f658c09e000 0x7f658c11e650 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:30.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.206+0000 7f6592881700 1 -- 172.21.15.3:0/2329748555 >> 172.21.15.3:0/2329748555 conn(0x7f658c066f50 msgr2=0x7f658c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:30.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.207+0000 7f6592881700 1 -- 172.21.15.3:0/2329748555 shutdown_connections 2023-10-03T10:15:30.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.207+0000 7f6592881700 1 -- 172.21.15.3:0/2329748555 wait complete. 2023-10-03T10:15:30.222 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:30.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.244+0000 7f8a0d252700 1 -- 172.21.15.3:0/197630134 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8a0809dcd0 con 0x7f8a08133420 2023-10-03T10:15:30.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.244+0000 7f89ef7fe700 1 -- 172.21.15.3:0/197630134 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f89f8024380 con 0x7f8a08133420 2023-10-03T10:15:30.245 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:30.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.246+0000 7f8a0d252700 1 -- 172.21.15.3:0/197630134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a08133420 msgr2=0x7f8a0809c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:30.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.246+0000 7f8a0d252700 1 --2- 172.21.15.3:0/197630134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a08133420 0x7f8a0809c480 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009840 tx=0x7f89f8004750 comp rx=0 tx=0).stop 2023-10-03T10:15:30.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.246+0000 7f8a0d252700 1 -- 172.21.15.3:0/197630134 shutdown_connections 2023-10-03T10:15:30.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.246+0000 7f8a0d252700 1 --2- 172.21.15.3:0/197630134 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a08134260 0x7f8a0809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:30.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.246+0000 7f8a0d252700 1 --2- 172.21.15.3:0/197630134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a08133420 0x7f8a0809c480 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:30.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.246+0000 7f8a0d252700 1 --2- 172.21.15.3:0/197630134 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a08133e30 0x7f8a0809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:30.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.246+0000 7f8a0d252700 1 -- 172.21.15.3:0/197630134 >> 172.21.15.3:0/197630134 conn(0x7f8a080a4210 msgr2=0x7f8a080a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:30.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.247+0000 7f8a0d252700 1 -- 172.21.15.3:0/197630134 shutdown_connections 2023-10-03T10:15:30.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:30.247+0000 7f8a0d252700 1 -- 172.21.15.3:0/197630134 wait complete. 2023-10-03T10:15:30.260 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:31.133 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:31.169 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:31.223 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:31.261 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:31.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.286+0000 7f431c1ca700 1 Processor -- start 2023-10-03T10:15:31.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.287+0000 7f431c1ca700 1 -- start start 2023-10-03T10:15:31.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.287+0000 7f431c1ca700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f431406e570 0x7f431406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:31.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.287+0000 7f431c1ca700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f431409eca0 0x7f431406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:31.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.287+0000 7f431c1ca700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4314134610 0x7f4314134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:31.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.287+0000 7f431c1ca700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4314070ee0 con 0x7f4314134610 2023-10-03T10:15:31.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.287+0000 7f431c1ca700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4314071480 con 0x7f431409eca0 2023-10-03T10:15:31.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.287+0000 7f431c1ca700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f431406f3c0 con 0x7f431406e570 2023-10-03T10:15:31.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.287+0000 7f4319765700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f431409eca0 0x7f431406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:31.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.287+0000 7f4319765700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f431409eca0 0x7f431406ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46714/0 (socket says 172.21.15.3:46714) 2023-10-03T10:15:31.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.287+0000 7f4319765700 1 -- 172.21.15.3:0/3648803894 learned_addr learned my addr 172.21.15.3:0/3648803894 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:31.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.287+0000 7f4319f66700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f431406e570 0x7f431406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:31.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.288+0000 7f431a767700 1 --2- 172.21.15.3:0/3648803894 >> v2:172.21.15.3:3300/0 conn(0x7f4314134610 0x7f4314134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:31.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.288+0000 7f4319765700 1 -- 172.21.15.3:0/3648803894 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f431406e570 msgr2=0x7f431406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:31.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.288+0000 7f4319765700 1 --2- 172.21.15.3:0/3648803894 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f431406e570 0x7f431406e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.288+0000 7f4319765700 1 -- 172.21.15.3:0/3648803894 >> v2:172.21.15.3:3300/0 conn(0x7f4314134610 msgr2=0x7f4314134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:31.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.288+0000 7f4319765700 1 --2- 172.21.15.3:0/3648803894 >> v2:172.21.15.3:3300/0 conn(0x7f4314134610 0x7f4314134ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.288+0000 7f4319765700 1 -- 172.21.15.3:0/3648803894 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4314071b90 con 0x7f431409eca0 2023-10-03T10:15:31.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.288+0000 7f4319765700 1 --2- 172.21.15.3:0/3648803894 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f431409eca0 0x7f431406ee80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f43040098f0 tx=0x7f4304009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3ccdb626d7de9a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:31.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.288+0000 7f4318f64700 1 -- 172.21.15.3:0/3648803894 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4304011830 con 0x7f431409eca0 2023-10-03T10:15:31.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.288+0000 7f4318f64700 1 -- 172.21.15.3:0/3648803894 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4304011990 con 0x7f431409eca0 2023-10-03T10:15:31.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.288+0000 7f4318f64700 1 -- 172.21.15.3:0/3648803894 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4304011d20 con 0x7f431409eca0 2023-10-03T10:15:31.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.289+0000 7f431c1ca700 1 -- 172.21.15.3:0/3648803894 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f431409eca0 msgr2=0x7f431406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:31.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.289+0000 7f431c1ca700 1 --2- 172.21.15.3:0/3648803894 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f431409eca0 0x7f431406ee80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f43040098f0 tx=0x7f4304009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:31.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.289+0000 7f431c1ca700 1 -- 172.21.15.3:0/3648803894 shutdown_connections 2023-10-03T10:15:31.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.289+0000 7f431c1ca700 1 --2- 172.21.15.3:0/3648803894 >> v2:172.21.15.3:3300/0 conn(0x7f4314134610 0x7f4314134ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.289+0000 7f431c1ca700 1 --2- 172.21.15.3:0/3648803894 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f431406e570 0x7f431406e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.289+0000 7f431c1ca700 1 --2- 172.21.15.3:0/3648803894 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f431409eca0 0x7f431406ee80 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.289+0000 7f431c1ca700 1 -- 172.21.15.3:0/3648803894 >> 172.21.15.3:0/3648803894 conn(0x7f43140a2aa0 msgr2=0x7f43140a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:31.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.289+0000 7f431c1ca700 1 -- 172.21.15.3:0/3648803894 shutdown_connections 2023-10-03T10:15:31.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.289+0000 7f431c1ca700 1 -- 172.21.15.3:0/3648803894 wait complete. 2023-10-03T10:15:31.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.290+0000 7f431c1ca700 1 Processor -- start 2023-10-03T10:15:31.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.290+0000 7f431c1ca700 1 -- start start 2023-10-03T10:15:31.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.291+0000 7f431c1ca700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f431406e570 0x7f4314127250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:31.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.291+0000 7f431c1ca700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4314134610 0x7f4314127790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:31.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.291+0000 7f431c1ca700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4314125ea0 0x7f4314126340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:31.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.291+0000 7f431c1ca700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431413b710 con 0x7f4314134610 2023-10-03T10:15:31.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.291+0000 7f431c1ca700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431413d3d0 con 0x7f4314125ea0 2023-10-03T10:15:31.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.291+0000 7f431c1ca700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f43140a1d30 con 0x7f431406e570 2023-10-03T10:15:31.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.291+0000 7f4319765700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4314134610 0x7f4314127790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:31.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.291+0000 7f4319765700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4314134610 0x7f4314127790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45342/0 (socket says 172.21.15.3:45342) 2023-10-03T10:15:31.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.291+0000 7f4319765700 1 -- 172.21.15.3:0/858158222 learned_addr learned my addr 172.21.15.3:0/858158222 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:31.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.291+0000 7f431a767700 1 --2- 172.21.15.3:0/858158222 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4314125ea0 0x7f4314126340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:31.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.291+0000 7f4319f66700 1 --2- 172.21.15.3:0/858158222 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f431406e570 0x7f4314127250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:31.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.292+0000 7f431a767700 1 -- 172.21.15.3:0/858158222 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f431406e570 msgr2=0x7f4314127250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:31.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.292+0000 7f431a767700 1 --2- 172.21.15.3:0/858158222 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f431406e570 0x7f4314127250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.292+0000 7f431a767700 1 -- 172.21.15.3:0/858158222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4314134610 msgr2=0x7f4314127790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:31.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.292+0000 7f431a767700 1 --2- 172.21.15.3:0/858158222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4314134610 0x7f4314127790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.292+0000 7f431a767700 1 -- 172.21.15.3:0/858158222 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4304009550 con 0x7f4314125ea0 2023-10-03T10:15:31.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.292+0000 7f4319765700 1 --2- 172.21.15.3:0/858158222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4314134610 0x7f4314127790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:31.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.292+0000 7f431a767700 1 --2- 172.21.15.3:0/858158222 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4314125ea0 0x7f4314126340 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f430c00efc0 tx=0x7f430c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:31.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.292+0000 7f430a7fc700 1 -- 172.21.15.3:0/858158222 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c00cc90 con 0x7f4314125ea0 2023-10-03T10:15:31.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.292+0000 7f431c1ca700 1 -- 172.21.15.3:0/858158222 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4314070610 con 0x7f4314125ea0 2023-10-03T10:15:31.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.292+0000 7f430a7fc700 1 -- 172.21.15.3:0/858158222 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430c00f040 con 0x7f4314125ea0 2023-10-03T10:15:31.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.292+0000 7f430a7fc700 1 -- 172.21.15.3:0/858158222 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c0146c0 con 0x7f4314125ea0 2023-10-03T10:15:31.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.292+0000 7f431c1ca700 1 -- 172.21.15.3:0/858158222 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431413ddd0 con 0x7f4314125ea0 2023-10-03T10:15:31.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.293+0000 7f430a7fc700 1 -- 172.21.15.3:0/858158222 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f430c014820 con 0x7f4314125ea0 2023-10-03T10:15:31.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.293+0000 7f431c1ca700 1 -- 172.21.15.3:0/858158222 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4314074ea0 con 0x7f4314125ea0 2023-10-03T10:15:31.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.294+0000 7f430a7fc700 1 -- 172.21.15.3:0/858158222 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f430c0401a0 con 0x7f4314125ea0 2023-10-03T10:15:31.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.298+0000 7f430a7fc700 1 -- 172.21.15.3:0/858158222 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f430c014ad0 con 0x7f4314125ea0 2023-10-03T10:15:31.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.338+0000 7f05b47f7700 1 Processor -- start 2023-10-03T10:15:31.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.338+0000 7f05b47f7700 1 -- start start 2023-10-03T10:15:31.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.339+0000 7f05b47f7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f05ac09d6f0 0x7f05ac09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:31.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.339+0000 7f05b47f7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f05ac09e410 0x7f05ac134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:31.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.339+0000 7f05b47f7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ac09b120 con 0x7f05ac09e000 2023-10-03T10:15:31.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.339+0000 7f05b47f7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac06b460 con 0x7f05ac09d6f0 2023-10-03T10:15:31.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.339+0000 7f05b47f7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac09a7e0 con 0x7f05ac09e410 2023-10-03T10:15:31.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.339+0000 7f05b2d94700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f05ac09e410 0x7f05ac134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:31.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.339+0000 7f05b2d94700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f05ac09e410 0x7f05ac134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43766/0 (socket says 172.21.15.3:43766) 2023-10-03T10:15:31.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.339+0000 7f05b2d94700 1 -- 172.21.15.3:0/2602669420 learned_addr learned my addr 172.21.15.3:0/2602669420 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:31.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.339+0000 7f05b2593700 1 --2- 172.21.15.3:0/2602669420 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f05ac09d6f0 0x7f05ac09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:31.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.340+0000 7f05b2d94700 1 -- 172.21.15.3:0/2602669420 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f05ac09d6f0 msgr2=0x7f05ac09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:31.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.340+0000 7f05b2d94700 1 --2- 172.21.15.3:0/2602669420 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f05ac09d6f0 0x7f05ac09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.340+0000 7f05b2d94700 1 -- 172.21.15.3:0/2602669420 >> v1:172.21.15.3:6789/0 conn(0x7f05ac09e000 legacy=0x7f05ac1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:31.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.340+0000 7f05b2d94700 1 -- 172.21.15.3:0/2602669420 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ac09a500 con 0x7f05ac09e410 2023-10-03T10:15:31.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.340+0000 7f05b2593700 1 --2- 172.21.15.3:0/2602669420 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f05ac09d6f0 0x7f05ac09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:15:31.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.340+0000 7f05b2d94700 1 --2- 172.21.15.3:0/2602669420 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f05ac09e410 0x7f05ac134100 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009eb0 tx=0x7f05a400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ba8e88d8dd55814 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:31.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.340+0000 7f05b1591700 1 -- 172.21.15.3:0/2602669420 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a400e070 con 0x7f05ac09e410 2023-10-03T10:15:31.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.340+0000 7f05b1591700 1 -- 172.21.15.3:0/2602669420 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a400b980 con 0x7f05ac09e410 2023-10-03T10:15:31.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.340+0000 7f05b1591700 1 -- 172.21.15.3:0/2602669420 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a400bd30 con 0x7f05ac09e410 2023-10-03T10:15:31.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.341+0000 7f05b47f7700 1 -- 172.21.15.3:0/2602669420 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f05ac09e410 msgr2=0x7f05ac134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:31.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.341+0000 7f05b47f7700 1 --2- 172.21.15.3:0/2602669420 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f05ac09e410 0x7f05ac134100 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009eb0 tx=0x7f05a400b330 comp rx=0 tx=0).stop 2023-10-03T10:15:31.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.341+0000 7f05b47f7700 1 -- 172.21.15.3:0/2602669420 shutdown_connections 2023-10-03T10:15:31.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.341+0000 7f05b47f7700 1 --2- 172.21.15.3:0/2602669420 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f05ac09d6f0 0x7f05ac09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.341+0000 7f05b47f7700 1 --2- 172.21.15.3:0/2602669420 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f05ac09e410 0x7f05ac134100 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.341+0000 7f05b47f7700 1 -- 172.21.15.3:0/2602669420 >> 172.21.15.3:0/2602669420 conn(0x7f05ac066f50 msgr2=0x7f05ac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:31.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.341+0000 7f05b47f7700 1 -- 172.21.15.3:0/2602669420 shutdown_connections 2023-10-03T10:15:31.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.341+0000 7f05b47f7700 1 -- 172.21.15.3:0/2602669420 wait complete. 2023-10-03T10:15:31.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.342+0000 7f05b47f7700 1 Processor -- start 2023-10-03T10:15:31.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.343+0000 7f05b47f7700 1 -- start start 2023-10-03T10:15:31.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.343+0000 7f05b47f7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05ac09d6f0 0x7f05ac12d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:31.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.343+0000 7f05b47f7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f05ac09e000 0x7f05ac12d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:31.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.343+0000 7f05b47f7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f05ac09e410 0x7f05ac1d4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:31.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.343+0000 7f05b47f7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac1301d0 con 0x7f05ac09d6f0 2023-10-03T10:15:31.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.343+0000 7f05b47f7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac1303b0 con 0x7f05ac09e000 2023-10-03T10:15:31.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.343+0000 7f05b47f7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac12ce80 con 0x7f05ac09e410 2023-10-03T10:15:31.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.343+0000 7f05b2d94700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f05ac09e410 0x7f05ac1d4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:31.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.343+0000 7f05b2593700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05ac09d6f0 0x7f05ac12d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:31.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.343+0000 7f05b2d94700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f05ac09e410 0x7f05ac1d4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43778/0 (socket says 172.21.15.3:43778) 2023-10-03T10:15:31.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.343+0000 7f05b2d94700 1 -- 172.21.15.3:0/3481967966 learned_addr learned my addr 172.21.15.3:0/3481967966 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:31.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.344+0000 7f05b1d92700 1 --2- 172.21.15.3:0/3481967966 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f05ac09e000 0x7f05ac12d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:31.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.344+0000 7f05b2d94700 1 -- 172.21.15.3:0/3481967966 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f05ac09e000 msgr2=0x7f05ac12d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:31.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.344+0000 7f05b2d94700 1 --2- 172.21.15.3:0/3481967966 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f05ac09e000 0x7f05ac12d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.344+0000 7f05b2d94700 1 -- 172.21.15.3:0/3481967966 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05ac09d6f0 msgr2=0x7f05ac12d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:31.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.344+0000 7f05b2d94700 1 --2- 172.21.15.3:0/3481967966 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05ac09d6f0 0x7f05ac12d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.344+0000 7f05b2d94700 1 -- 172.21.15.3:0/3481967966 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a4009b30 con 0x7f05ac09e410 2023-10-03T10:15:31.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.344+0000 7f05b1d92700 1 --2- 172.21.15.3:0/3481967966 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f05ac09e000 0x7f05ac12d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:15:31.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.344+0000 7f05b2593700 1 --2- 172.21.15.3:0/3481967966 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05ac09d6f0 0x7f05ac12d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:31.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.344+0000 7f05b2d94700 1 --2- 172.21.15.3:0/3481967966 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f05ac09e410 0x7f05ac1d4f60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f05a400b730 tx=0x7f05a4004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:31.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.344+0000 7f05a2ffd700 1 -- 172.21.15.3:0/3481967966 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a4004450 con 0x7f05ac09e410 2023-10-03T10:15:31.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.344+0000 7f05a2ffd700 1 -- 172.21.15.3:0/3481967966 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a40045d0 con 0x7f05ac09e410 2023-10-03T10:15:31.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.344+0000 7f05a2ffd700 1 -- 172.21.15.3:0/3481967966 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a4025460 con 0x7f05ac09e410 2023-10-03T10:15:31.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.345+0000 7f05b47f7700 1 -- 172.21.15.3:0/3481967966 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ac0721c0 con 0x7f05ac09e410 2023-10-03T10:15:31.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.345+0000 7f05b47f7700 1 -- 172.21.15.3:0/3481967966 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ac12e9f0 con 0x7f05ac09e410 2023-10-03T10:15:31.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.346+0000 7f05b47f7700 1 -- 172.21.15.3:0/3481967966 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ac074ea0 con 0x7f05ac09e410 2023-10-03T10:15:31.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.346+0000 7f05a2ffd700 1 -- 172.21.15.3:0/3481967966 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f05a401c410 con 0x7f05ac09e410 2023-10-03T10:15:31.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.347+0000 7f05a2ffd700 1 -- 172.21.15.3:0/3481967966 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f05a4041090 con 0x7f05ac09e410 2023-10-03T10:15:31.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.356+0000 7f05a2ffd700 1 -- 172.21.15.3:0/3481967966 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f05a402d350 con 0x7f05ac09e410 2023-10-03T10:15:31.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.395+0000 7effe33a0700 1 Processor -- start 2023-10-03T10:15:31.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.395+0000 7effe33a0700 1 -- start start 2023-10-03T10:15:31.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.396+0000 7effe33a0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7effdc09e930 0x7effdc09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:31.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.396+0000 7effe33a0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effdc09f240 0x7effdc1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:31.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.396+0000 7effe33a0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effdc070940 con 0x7effdc09f650 2023-10-03T10:15:31.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.396+0000 7effe33a0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effdc06fd60 con 0x7effdc09e930 2023-10-03T10:15:31.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.396+0000 7effe33a0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7effdc06fa80 con 0x7effdc09f240 2023-10-03T10:15:31.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.396+0000 7effe093b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effdc09f240 0x7effdc1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:31.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.396+0000 7effe113c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7effdc09e930 0x7effdc09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:31.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.396+0000 7effe093b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effdc09f240 0x7effdc1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43788/0 (socket says 172.21.15.3:43788) 2023-10-03T10:15:31.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.396+0000 7effe193d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7effdc09f650 0x7effdc131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39758/0 (socket says 172.21.15.3:39758) 2023-10-03T10:15:31.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.396+0000 7effe093b700 1 -- 172.21.15.3:0/20960918 learned_addr learned my addr 172.21.15.3:0/20960918 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:31.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.397+0000 7effe093b700 1 -- 172.21.15.3:0/20960918 >> v2:172.21.15.3:3300/0 conn(0x7effdc09e930 msgr2=0x7effdc09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:31.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.397+0000 7effe093b700 1 --2- 172.21.15.3:0/20960918 >> v2:172.21.15.3:3300/0 conn(0x7effdc09e930 0x7effdc09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.397+0000 7effe093b700 1 -- 172.21.15.3:0/20960918 >> v1:172.21.15.3:6789/0 conn(0x7effdc09f650 legacy=0x7effdc131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:31.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.397+0000 7effd3fff700 1 -- 172.21.15.3:0/20960918 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1540694152 0 0) 0x7effdc070940 con 0x7effdc09f650 2023-10-03T10:15:31.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.397+0000 7effe093b700 1 -- 172.21.15.3:0/20960918 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effdc09fb60 con 0x7effdc09f240 2023-10-03T10:15:31.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.397+0000 7effe113c700 1 --2- 172.21.15.3:0/20960918 >> v2:172.21.15.3:3300/0 conn(0x7effdc09e930 0x7effdc09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:31.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.397+0000 7effe093b700 1 --2- 172.21.15.3:0/20960918 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effdc09f240 0x7effdc1317f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7effcc0098f0 tx=0x7effcc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecbc0c9c28d33e0f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:31.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.397+0000 7effd3fff700 1 -- 172.21.15.3:0/20960918 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effcc011830 con 0x7effdc09f240 2023-10-03T10:15:31.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.397+0000 7effd3fff700 1 -- 172.21.15.3:0/20960918 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effcc011990 con 0x7effdc09f240 2023-10-03T10:15:31.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.397+0000 7effd3fff700 1 -- 172.21.15.3:0/20960918 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effcc011d20 con 0x7effdc09f240 2023-10-03T10:15:31.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.398+0000 7effe33a0700 1 -- 172.21.15.3:0/20960918 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effdc09f240 msgr2=0x7effdc1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:31.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.398+0000 7effe33a0700 1 --2- 172.21.15.3:0/20960918 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effdc09f240 0x7effdc1317f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7effcc0098f0 tx=0x7effcc009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:31.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.398+0000 7effe33a0700 1 -- 172.21.15.3:0/20960918 shutdown_connections 2023-10-03T10:15:31.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.398+0000 7effe33a0700 1 --2- 172.21.15.3:0/20960918 >> v2:172.21.15.3:3300/0 conn(0x7effdc09e930 0x7effdc09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.398+0000 7effe33a0700 1 --2- 172.21.15.3:0/20960918 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effdc09f240 0x7effdc1317f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.398+0000 7effe33a0700 1 -- 172.21.15.3:0/20960918 >> 172.21.15.3:0/20960918 conn(0x7effdc0a4210 msgr2=0x7effdc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:31.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.398+0000 7effe33a0700 1 -- 172.21.15.3:0/20960918 shutdown_connections 2023-10-03T10:15:31.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.398+0000 7effe33a0700 1 -- 172.21.15.3:0/20960918 wait complete. 2023-10-03T10:15:31.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.399+0000 7effe33a0700 1 Processor -- start 2023-10-03T10:15:31.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.399+0000 7effe33a0700 1 -- start start 2023-10-03T10:15:31.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.400+0000 7effe33a0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7effdc09e930 0x7effdc1a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:31.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.400+0000 7effe33a0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effdc09f240 0x7effdc1a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:31.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.400+0000 7effe33a0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effdc09f650 0x7effdc1a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:31.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.400+0000 7effe33a0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effdc1a5f50 con 0x7effdc09f240 2023-10-03T10:15:31.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.400+0000 7effe33a0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effdc1a5920 con 0x7effdc09e930 2023-10-03T10:15:31.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.400+0000 7effe33a0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7effdc071d20 con 0x7effdc09f650 2023-10-03T10:15:31.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.400+0000 7effe193d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effdc09f650 0x7effdc1a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:31.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.400+0000 7effe193d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effdc09f650 0x7effdc1a4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43802/0 (socket says 172.21.15.3:43802) 2023-10-03T10:15:31.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.400+0000 7effe193d700 1 -- 172.21.15.3:0/2954990087 learned_addr learned my addr 172.21.15.3:0/2954990087 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:31.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.400+0000 7effe093b700 1 --2- 172.21.15.3:0/2954990087 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effdc09f240 0x7effdc1a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:31.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.400+0000 7effe193d700 1 -- 172.21.15.3:0/2954990087 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7effdc09e930 msgr2=0x7effdc1a3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:31.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.400+0000 7effe193d700 1 --2- 172.21.15.3:0/2954990087 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7effdc09e930 0x7effdc1a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.400+0000 7effe193d700 1 -- 172.21.15.3:0/2954990087 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effdc09f240 msgr2=0x7effdc1a42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:31.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.400+0000 7effe193d700 1 --2- 172.21.15.3:0/2954990087 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effdc09f240 0x7effdc1a42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.400+0000 7effe193d700 1 -- 172.21.15.3:0/2954990087 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd80094a0 con 0x7effdc09f650 2023-10-03T10:15:31.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.401+0000 7effe193d700 1 --2- 172.21.15.3:0/2954990087 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effdc09f650 0x7effdc1a4820 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7effd40146a0 tx=0x7effd40149b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:31.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.401+0000 7effd1ffb700 1 -- 172.21.15.3:0/2954990087 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd4017070 con 0x7effdc09f650 2023-10-03T10:15:31.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.401+0000 7effd1ffb700 1 -- 172.21.15.3:0/2954990087 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd4012320 con 0x7effdc09f650 2023-10-03T10:15:31.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.401+0000 7effe33a0700 1 -- 172.21.15.3:0/2954990087 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effcc009550 con 0x7effdc09f650 2023-10-03T10:15:31.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.401+0000 7effd1ffb700 1 -- 172.21.15.3:0/2954990087 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd40126d0 con 0x7effdc09f650 2023-10-03T10:15:31.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.401+0000 7effe33a0700 1 -- 172.21.15.3:0/2954990087 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effdc0a0190 con 0x7effdc09f650 2023-10-03T10:15:31.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.402+0000 7effe33a0700 1 -- 172.21.15.3:0/2954990087 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effdc1a7760 con 0x7effdc09f650 2023-10-03T10:15:31.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.402+0000 7effd1ffb700 1 -- 172.21.15.3:0/2954990087 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7effd4010850 con 0x7effdc09f650 2023-10-03T10:15:31.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.403+0000 7effd1ffb700 1 -- 172.21.15.3:0/2954990087 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7effd4047b00 con 0x7effdc09f650 2023-10-03T10:15:31.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.408+0000 7effd1ffb700 1 -- 172.21.15.3:0/2954990087 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7effd401b330 con 0x7effdc09f650 2023-10-03T10:15:31.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.422+0000 7f6dcd79a700 1 Processor -- start 2023-10-03T10:15:31.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.422+0000 7f6dcd79a700 1 -- start start 2023-10-03T10:15:31.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.422+0000 7f6dcd79a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6dc806a770 0x7f6dc809d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:31.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.422+0000 7f6dcd79a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6dc809dc30 0x7f6dc809e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:31.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.422+0000 7f6dcd79a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dc809a810 con 0x7f6dc813c1e0 2023-10-03T10:15:31.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.422+0000 7f6dcd79a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc806ab40 con 0x7f6dc809dc30 2023-10-03T10:15:31.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.422+0000 7f6dcd79a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc809ebc0 con 0x7f6dc806a770 2023-10-03T10:15:31.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.422+0000 7f6dc67fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6dc809dc30 0x7f6dc809e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:31.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.423+0000 7f6dc67fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6dc809dc30 0x7f6dc809e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45362/0 (socket says 172.21.15.3:45362) 2023-10-03T10:15:31.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.423+0000 7f6dc67fc700 1 -- 172.21.15.3:0/185020640 learned_addr learned my addr 172.21.15.3:0/185020640 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:31.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.423+0000 7f6dc77fe700 1 --1- 172.21.15.3:0/185020640 >> v1:172.21.15.3:6789/0 conn(0x7f6dc813c1e0 0x7f6dc813e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39766/0 (socket says 172.21.15.3:39766) 2023-10-03T10:15:31.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.423+0000 7f6dc67fc700 1 -- 172.21.15.3:0/185020640 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6dc806a770 msgr2=0x7f6dc809d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:31.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.423+0000 7f6dc67fc700 1 --2- 172.21.15.3:0/185020640 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6dc806a770 0x7f6dc809d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.423+0000 7f6dc67fc700 1 -- 172.21.15.3:0/185020640 >> v1:172.21.15.3:6789/0 conn(0x7f6dc813c1e0 legacy=0x7f6dc813e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:31.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.423+0000 7f6dc67fc700 1 -- 172.21.15.3:0/185020640 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc809e920 con 0x7f6dc809dc30 2023-10-03T10:15:31.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.423+0000 7f6dc5ffb700 1 -- 172.21.15.3:0/185020640 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 807723428 0 0) 0x7f6dc809a810 con 0x7f6dc813c1e0 2023-10-03T10:15:31.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.423+0000 7f6dc67fc700 1 --2- 172.21.15.3:0/185020640 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dc809dc30 0x7f6dc809e080 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc009920 tx=0x7f6dbc009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9afefb35c843aa24 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:31.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.423+0000 7f6dc5ffb700 1 -- 172.21.15.3:0/185020640 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc00ba20 con 0x7f6dc809dc30 2023-10-03T10:15:31.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.424+0000 7f6dc5ffb700 1 -- 172.21.15.3:0/185020640 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dbc00bb80 con 0x7f6dc809dc30 2023-10-03T10:15:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.424+0000 7f6dc5ffb700 1 -- 172.21.15.3:0/185020640 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc009e70 con 0x7f6dc809dc30 2023-10-03T10:15:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.425+0000 7f6dcd79a700 1 -- 172.21.15.3:0/185020640 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dc809dc30 msgr2=0x7f6dc809e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.425+0000 7f6dcd79a700 1 --2- 172.21.15.3:0/185020640 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dc809dc30 0x7f6dc809e080 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc009920 tx=0x7f6dbc009c30 comp rx=0 tx=0).stop 2023-10-03T10:15:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.425+0000 7f6dcd79a700 1 -- 172.21.15.3:0/185020640 shutdown_connections 2023-10-03T10:15:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.425+0000 7f6dcd79a700 1 --2- 172.21.15.3:0/185020640 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6dc806a770 0x7f6dc809d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.425+0000 7f6dcd79a700 1 --2- 172.21.15.3:0/185020640 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dc809dc30 0x7f6dc809e080 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.425+0000 7f6dcd79a700 1 -- 172.21.15.3:0/185020640 >> 172.21.15.3:0/185020640 conn(0x7f6dc8066f50 msgr2=0x7f6dc809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.425+0000 7f6dcd79a700 1 -- 172.21.15.3:0/185020640 shutdown_connections 2023-10-03T10:15:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.425+0000 7f6dcd79a700 1 -- 172.21.15.3:0/185020640 wait complete. 2023-10-03T10:15:31.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.427+0000 7f6dcd79a700 1 Processor -- start 2023-10-03T10:15:31.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.427+0000 7f6dcd79a700 1 -- start start 2023-10-03T10:15:31.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.427+0000 7f6dcd79a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dc806a770 0x7f6dc811aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:31.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.427+0000 7f6dcd79a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6dc809dc30 0x7f6dc813a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:31.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.427+0000 7f6dcd79a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6dc813c1e0 0x7f6dc813a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:31.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.427+0000 7f6dcd79a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc809bfb0 con 0x7f6dc806a770 2023-10-03T10:15:31.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.427+0000 7f6dcd79a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc8134c00 con 0x7f6dc813c1e0 2023-10-03T10:15:31.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.427+0000 7f6dcd79a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc811d890 con 0x7f6dc809dc30 2023-10-03T10:15:31.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.428+0000 7f6dc6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dc806a770 0x7f6dc811aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:31.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.428+0000 7f6dc67fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6dc809dc30 0x7f6dc813a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:31.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.428+0000 7f6dc6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dc806a770 0x7f6dc811aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45372/0 (socket says 172.21.15.3:45372) 2023-10-03T10:15:31.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.428+0000 7f6dc6ffd700 1 -- 172.21.15.3:0/1597465908 learned_addr learned my addr 172.21.15.3:0/1597465908 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:31.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.428+0000 7f6dc77fe700 1 --2- 172.21.15.3:0/1597465908 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6dc813c1e0 0x7f6dc813a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:31.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.428+0000 7f6dc67fc700 1 -- 172.21.15.3:0/1597465908 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6dc813c1e0 msgr2=0x7f6dc813a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:31.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.428+0000 7f6dc67fc700 1 --2- 172.21.15.3:0/1597465908 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6dc813c1e0 0x7f6dc813a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.428+0000 7f6dc67fc700 1 -- 172.21.15.3:0/1597465908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dc806a770 msgr2=0x7f6dc811aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.428+0000 7f6dc67fc700 1 --2- 172.21.15.3:0/1597465908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dc806a770 0x7f6dc811aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.428+0000 7f6dc67fc700 1 -- 172.21.15.3:0/1597465908 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dbc009550 con 0x7f6dc809dc30 2023-10-03T10:15:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.428+0000 7f6dc6ffd700 1 --2- 172.21.15.3:0/1597465908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dc806a770 0x7f6dc811aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.428+0000 7f6dc67fc700 1 --2- 172.21.15.3:0/1597465908 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6dc809dc30 0x7f6dc813a4a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc016000 tx=0x7f6dbc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.429+0000 7f6daf7fe700 1 -- 172.21.15.3:0/1597465908 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc004400 con 0x7f6dc809dc30 2023-10-03T10:15:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.429+0000 7f6daf7fe700 1 -- 172.21.15.3:0/1597465908 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dbc01f070 con 0x7f6dc809dc30 2023-10-03T10:15:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.429+0000 7f6dcd79a700 1 -- 172.21.15.3:0/1597465908 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc811bd70 con 0x7f6dc809dc30 2023-10-03T10:15:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.429+0000 7f6daf7fe700 1 -- 172.21.15.3:0/1597465908 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc0038e0 con 0x7f6dc809dc30 2023-10-03T10:15:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.429+0000 7f6dcd79a700 1 -- 172.21.15.3:0/1597465908 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc809f1a0 con 0x7f6dc809dc30 2023-10-03T10:15:31.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.430+0000 7f6daf7fe700 1 -- 172.21.15.3:0/1597465908 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6dbc021030 con 0x7f6dc809dc30 2023-10-03T10:15:31.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.430+0000 7f6dcd79a700 1 -- 172.21.15.3:0/1597465908 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dc811cea0 con 0x7f6dc809dc30 2023-10-03T10:15:31.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.430+0000 7f6daf7fe700 1 -- 172.21.15.3:0/1597465908 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6dbc040db0 con 0x7f6dc809dc30 2023-10-03T10:15:31.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.440+0000 7f6daf7fe700 1 -- 172.21.15.3:0/1597465908 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6dbc0199a0 con 0x7f6dc809dc30 2023-10-03T10:15:31.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.525+0000 7f431c1ca700 1 -- 172.21.15.3:0/858158222 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f431413cfe0 con 0x7f4314125ea0 2023-10-03T10:15:31.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.526+0000 7f430a7fc700 1 -- 172.21.15.3:0/858158222 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f430c007330 con 0x7f4314125ea0 2023-10-03T10:15:31.526 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:31.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.528+0000 7f431c1ca700 1 -- 172.21.15.3:0/858158222 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4314125ea0 msgr2=0x7f4314126340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:31.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.528+0000 7f431c1ca700 1 --2- 172.21.15.3:0/858158222 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4314125ea0 0x7f4314126340 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f430c00efc0 tx=0x7f430c00c3a0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.528+0000 7f431c1ca700 1 -- 172.21.15.3:0/858158222 shutdown_connections 2023-10-03T10:15:31.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.528+0000 7f431c1ca700 1 --2- 172.21.15.3:0/858158222 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4314125ea0 0x7f4314126340 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.528+0000 7f431c1ca700 1 --2- 172.21.15.3:0/858158222 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f431406e570 0x7f4314127250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.528+0000 7f431c1ca700 1 --2- 172.21.15.3:0/858158222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4314134610 0x7f4314127790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.528+0000 7f431c1ca700 1 -- 172.21.15.3:0/858158222 >> 172.21.15.3:0/858158222 conn(0x7f43140a2aa0 msgr2=0x7f4314099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:31.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.528+0000 7f431c1ca700 1 -- 172.21.15.3:0/858158222 shutdown_connections 2023-10-03T10:15:31.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.528+0000 7f431c1ca700 1 -- 172.21.15.3:0/858158222 wait complete. 2023-10-03T10:15:31.543 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:31.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.575+0000 7f05b47f7700 1 -- 172.21.15.3:0/3481967966 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f05ac071fa0 con 0x7f05ac09e410 2023-10-03T10:15:31.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.575+0000 7f05a2ffd700 1 -- 172.21.15.3:0/3481967966 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f05a402a020 con 0x7f05ac09e410 2023-10-03T10:15:31.576 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:31.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.578+0000 7f05b47f7700 1 -- 172.21.15.3:0/3481967966 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f05ac09e410 msgr2=0x7f05ac1d4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:31.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.578+0000 7f05b47f7700 1 --2- 172.21.15.3:0/3481967966 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f05ac09e410 0x7f05ac1d4f60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f05a400b730 tx=0x7f05a4004ad0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.578+0000 7f05b47f7700 1 -- 172.21.15.3:0/3481967966 shutdown_connections 2023-10-03T10:15:31.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.578+0000 7f05b47f7700 1 --2- 172.21.15.3:0/3481967966 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f05ac09e410 0x7f05ac1d4f60 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.578+0000 7f05b47f7700 1 --2- 172.21.15.3:0/3481967966 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05ac09d6f0 0x7f05ac12d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.578+0000 7f05b47f7700 1 --2- 172.21.15.3:0/3481967966 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f05ac09e000 0x7f05ac12d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.578+0000 7f05b47f7700 1 -- 172.21.15.3:0/3481967966 >> 172.21.15.3:0/3481967966 conn(0x7f05ac066f50 msgr2=0x7f05ac1c89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:31.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.579+0000 7f05b47f7700 1 -- 172.21.15.3:0/3481967966 shutdown_connections 2023-10-03T10:15:31.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.579+0000 7f05b47f7700 1 -- 172.21.15.3:0/3481967966 wait complete. 2023-10-03T10:15:31.593 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:31.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.622+0000 7effe33a0700 1 -- 172.21.15.3:0/2954990087 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7effdc1a0b30 con 0x7effdc09f650 2023-10-03T10:15:31.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.622+0000 7effd1ffb700 1 -- 172.21.15.3:0/2954990087 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7effd4022020 con 0x7effdc09f650 2023-10-03T10:15:31.622 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:31.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.624+0000 7effe33a0700 1 -- 172.21.15.3:0/2954990087 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effdc09f650 msgr2=0x7effdc1a4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:31.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.624+0000 7effe33a0700 1 --2- 172.21.15.3:0/2954990087 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effdc09f650 0x7effdc1a4820 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7effd40146a0 tx=0x7effd40149b0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.624+0000 7effe33a0700 1 -- 172.21.15.3:0/2954990087 shutdown_connections 2023-10-03T10:15:31.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.624+0000 7effe33a0700 1 --2- 172.21.15.3:0/2954990087 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effdc09f650 0x7effdc1a4820 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.624+0000 7effe33a0700 1 --2- 172.21.15.3:0/2954990087 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7effdc09e930 0x7effdc1a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.624+0000 7effe33a0700 1 --2- 172.21.15.3:0/2954990087 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effdc09f240 0x7effdc1a42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.624+0000 7effe33a0700 1 -- 172.21.15.3:0/2954990087 >> 172.21.15.3:0/2954990087 conn(0x7effdc0a4210 msgr2=0x7effdc1b8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:31.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.624+0000 7effe33a0700 1 -- 172.21.15.3:0/2954990087 shutdown_connections 2023-10-03T10:15:31.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.624+0000 7effe33a0700 1 -- 172.21.15.3:0/2954990087 wait complete. 2023-10-03T10:15:31.638 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:31.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.648+0000 7f6dcd79a700 1 -- 172.21.15.3:0/1597465908 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6dc811d290 con 0x7f6dc809dc30 2023-10-03T10:15:31.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.649+0000 7f6daf7fe700 1 -- 172.21.15.3:0/1597465908 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6dbc01b430 con 0x7f6dc809dc30 2023-10-03T10:15:31.649 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:31.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.651+0000 7f6dcd79a700 1 -- 172.21.15.3:0/1597465908 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6dc809dc30 msgr2=0x7f6dc813a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:31.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.651+0000 7f6dcd79a700 1 --2- 172.21.15.3:0/1597465908 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6dc809dc30 0x7f6dc813a4a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc016000 tx=0x7f6dbc004750 comp rx=0 tx=0).stop 2023-10-03T10:15:31.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.651+0000 7f6dcd79a700 1 -- 172.21.15.3:0/1597465908 shutdown_connections 2023-10-03T10:15:31.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.651+0000 7f6dcd79a700 1 --2- 172.21.15.3:0/1597465908 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6dc813c1e0 0x7f6dc813a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.651+0000 7f6dcd79a700 1 --2- 172.21.15.3:0/1597465908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dc806a770 0x7f6dc811aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.651+0000 7f6dcd79a700 1 --2- 172.21.15.3:0/1597465908 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6dc809dc30 0x7f6dc813a4a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:31.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.651+0000 7f6dcd79a700 1 -- 172.21.15.3:0/1597465908 >> 172.21.15.3:0/1597465908 conn(0x7f6dc8066f50 msgr2=0x7f6dc8072150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:31.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.651+0000 7f6dcd79a700 1 -- 172.21.15.3:0/1597465908 shutdown_connections 2023-10-03T10:15:31.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:31.651+0000 7f6dcd79a700 1 -- 172.21.15.3:0/1597465908 wait complete. 2023-10-03T10:15:31.665 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:32.544 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:32.594 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:32.639 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:32.665 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:32.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.697+0000 7f346d44c700 1 Processor -- start 2023-10-03T10:15:32.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.697+0000 7f346d44c700 1 -- start start 2023-10-03T10:15:32.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.697+0000 7f346d44c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f346809de00 0x7f346809e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:32.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.697+0000 7f346d44c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f346813d1f0 0x7f346813f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:32.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.698+0000 7f346d44c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f346809a810 con 0x7f346806a770 2023-10-03T10:15:32.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.698+0000 7f346d44c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f346806ab40 con 0x7f346813d1f0 2023-10-03T10:15:32.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.698+0000 7f346d44c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346809ebc0 con 0x7f346809de00 2023-10-03T10:15:32.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.698+0000 7f34667fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f346809de00 0x7f346809e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:32.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.698+0000 7f34667fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f346809de00 0x7f346809e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46778/0 (socket says 172.21.15.3:46778) 2023-10-03T10:15:32.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.698+0000 7f34667fc700 1 -- 172.21.15.3:0/1258973651 learned_addr learned my addr 172.21.15.3:0/1258973651 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:32.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.698+0000 7f34677fe700 1 --2- 172.21.15.3:0/1258973651 >> v2:172.21.15.3:3300/0 conn(0x7f346813d1f0 0x7f346813f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:32.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.698+0000 7f34667fc700 1 -- 172.21.15.3:0/1258973651 >> v2:172.21.15.3:3300/0 conn(0x7f346813d1f0 msgr2=0x7f346813f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:32.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.698+0000 7f34667fc700 1 --2- 172.21.15.3:0/1258973651 >> v2:172.21.15.3:3300/0 conn(0x7f346813d1f0 0x7f346813f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:32.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.698+0000 7f3465ffb700 1 -- 172.21.15.3:0/1258973651 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2156298613 0 0) 0x7f346809a810 con 0x7f346806a770 2023-10-03T10:15:32.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.698+0000 7f34667fc700 1 -- 172.21.15.3:0/1258973651 >> v1:172.21.15.3:6789/0 conn(0x7f346806a770 legacy=0x7f346809d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:32.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.698+0000 7f34667fc700 1 -- 172.21.15.3:0/1258973651 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f346809e920 con 0x7f346809de00 2023-10-03T10:15:32.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.699+0000 7f34667fc700 1 --2- 172.21.15.3:0/1258973651 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f346809de00 0x7f346809e250 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f345c0098f0 tx=0x7f345c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2bf943d343ed52c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:32.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.699+0000 7f3465ffb700 1 -- 172.21.15.3:0/1258973651 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345c011830 con 0x7f346809de00 2023-10-03T10:15:32.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.699+0000 7f3465ffb700 1 -- 172.21.15.3:0/1258973651 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f345c011990 con 0x7f346809de00 2023-10-03T10:15:32.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.699+0000 7f3465ffb700 1 -- 172.21.15.3:0/1258973651 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345c011d20 con 0x7f346809de00 2023-10-03T10:15:32.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.700+0000 7f346d44c700 1 -- 172.21.15.3:0/1258973651 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f346809de00 msgr2=0x7f346809e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:32.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.700+0000 7f346d44c700 1 --2- 172.21.15.3:0/1258973651 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f346809de00 0x7f346809e250 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f345c0098f0 tx=0x7f345c009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:32.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.700+0000 7f346d44c700 1 -- 172.21.15.3:0/1258973651 shutdown_connections 2023-10-03T10:15:32.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.700+0000 7f346d44c700 1 --2- 172.21.15.3:0/1258973651 >> v2:172.21.15.3:3300/0 conn(0x7f346813d1f0 0x7f346813f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:32.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.700+0000 7f346d44c700 1 --2- 172.21.15.3:0/1258973651 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f346809de00 0x7f346809e250 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:32.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.700+0000 7f346d44c700 1 -- 172.21.15.3:0/1258973651 >> 172.21.15.3:0/1258973651 conn(0x7f3468066f50 msgr2=0x7f346809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:32.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.700+0000 7f346d44c700 1 -- 172.21.15.3:0/1258973651 shutdown_connections 2023-10-03T10:15:32.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.700+0000 7f346d44c700 1 -- 172.21.15.3:0/1258973651 wait complete. 2023-10-03T10:15:32.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.701+0000 7f346d44c700 1 Processor -- start 2023-10-03T10:15:32.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.701+0000 7f346d44c700 1 -- start start 2023-10-03T10:15:32.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.702+0000 7f346d44c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f346806a770 0x7f3468070bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:32.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.702+0000 7f346d44c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f346809de00 0x7f3468071130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:32.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.702+0000 7f346d44c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f346813d1f0 0x7f3468071670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:32.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.702+0000 7f346d44c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346806e9d0 con 0x7f346809de00 2023-10-03T10:15:32.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.702+0000 7f346d44c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3468070010 con 0x7f346806a770 2023-10-03T10:15:32.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.702+0000 7f346d44c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f346806f330 con 0x7f346813d1f0 2023-10-03T10:15:32.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.702+0000 7f3466ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f346806a770 0x7f3468070bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:32.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.702+0000 7f34677fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f346813d1f0 0x7f3468071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:32.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.702+0000 7f3466ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f346806a770 0x7f3468070bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46784/0 (socket says 172.21.15.3:46784) 2023-10-03T10:15:32.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.702+0000 7f34677fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f346813d1f0 0x7f3468071670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43838/0 (socket says 172.21.15.3:43838) 2023-10-03T10:15:32.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.702+0000 7f3466ffd700 1 -- 172.21.15.3:0/3099478183 learned_addr learned my addr 172.21.15.3:0/3099478183 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:32.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.702+0000 7f34677fe700 1 -- 172.21.15.3:0/3099478183 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f346806a770 msgr2=0x7f3468070bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:32.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.702+0000 7f34677fe700 1 --2- 172.21.15.3:0/3099478183 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f346806a770 0x7f3468070bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:32.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.702+0000 7f34677fe700 1 -- 172.21.15.3:0/3099478183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f346809de00 msgr2=0x7f3468071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:32.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.702+0000 7f34677fe700 1 --2- 172.21.15.3:0/3099478183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f346809de00 0x7f3468071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:32.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.702+0000 7f34677fe700 1 -- 172.21.15.3:0/3099478183 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f345c009550 con 0x7f346813d1f0 2023-10-03T10:15:32.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.702+0000 7f3466ffd700 1 --2- 172.21.15.3:0/3099478183 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f346806a770 0x7f3468070bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:32.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.703+0000 7f34677fe700 1 --2- 172.21.15.3:0/3099478183 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f346813d1f0 0x7f3468071670 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3460009d90 tx=0x7f346000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:32.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.703+0000 7f344f7fe700 1 -- 172.21.15.3:0/3099478183 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3460009c00 con 0x7f346813d1f0 2023-10-03T10:15:32.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.703+0000 7f344f7fe700 1 -- 172.21.15.3:0/3099478183 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f346000f040 con 0x7f346813d1f0 2023-10-03T10:15:32.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.703+0000 7f346d44c700 1 -- 172.21.15.3:0/3099478183 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3468072730 con 0x7f346813d1f0 2023-10-03T10:15:32.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.703+0000 7f344f7fe700 1 -- 172.21.15.3:0/3099478183 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3460014480 con 0x7f346813d1f0 2023-10-03T10:15:32.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.703+0000 7f346d44c700 1 -- 172.21.15.3:0/3099478183 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34680707a0 con 0x7f346813d1f0 2023-10-03T10:15:32.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.704+0000 7f344f7fe700 1 -- 172.21.15.3:0/3099478183 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f34600145e0 con 0x7f346813d1f0 2023-10-03T10:15:32.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.704+0000 7f344f7fe700 1 -- 172.21.15.3:0/3099478183 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3460040390 con 0x7f346813d1f0 2023-10-03T10:15:32.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.704+0000 7f346d44c700 1 -- 172.21.15.3:0/3099478183 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f346806f9d0 con 0x7f346813d1f0 2023-10-03T10:15:32.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.710+0000 7f344f7fe700 1 -- 172.21.15.3:0/3099478183 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f346806f9d0 con 0x7f346813d1f0 2023-10-03T10:15:32.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.754+0000 7fbd79ba7700 1 Processor -- start 2023-10-03T10:15:32.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.754+0000 7fbd79ba7700 1 -- start start 2023-10-03T10:15:32.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.754+0000 7fbd79ba7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbd7409e930 0x7fbd7409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:32.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.755+0000 7fbd79ba7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7409f240 0x7fbd741317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:32.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.755+0000 7fbd79ba7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd74070940 con 0x7fbd7409f650 2023-10-03T10:15:32.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.755+0000 7fbd79ba7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd7406fd60 con 0x7fbd7409e930 2023-10-03T10:15:32.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.755+0000 7fbd79ba7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7406fa80 con 0x7fbd7409f240 2023-10-03T10:15:32.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.755+0000 7fbd72ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7409f240 0x7fbd741317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:32.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.755+0000 7fbd72ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7409f240 0x7fbd741317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46792/0 (socket says 172.21.15.3:46792) 2023-10-03T10:15:32.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.755+0000 7fbd72ffd700 1 -- 172.21.15.3:0/408806905 learned_addr learned my addr 172.21.15.3:0/408806905 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:32.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.755+0000 7fbd737fe700 1 --2- 172.21.15.3:0/408806905 >> v2:172.21.15.3:3300/0 conn(0x7fbd7409e930 0x7fbd7409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:32.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.755+0000 7fbd72ffd700 1 -- 172.21.15.3:0/408806905 >> v2:172.21.15.3:3300/0 conn(0x7fbd7409e930 msgr2=0x7fbd7409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:32.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.755+0000 7fbd72ffd700 1 --2- 172.21.15.3:0/408806905 >> v2:172.21.15.3:3300/0 conn(0x7fbd7409e930 0x7fbd7409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:32.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.755+0000 7fbd72ffd700 1 -- 172.21.15.3:0/408806905 >> v1:172.21.15.3:6789/0 conn(0x7fbd7409f650 legacy=0x7fbd74131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:32.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.755+0000 7fbd72ffd700 1 -- 172.21.15.3:0/408806905 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7409fb60 con 0x7fbd7409f240 2023-10-03T10:15:32.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.755+0000 7fbd72ffd700 1 --2- 172.21.15.3:0/408806905 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7409f240 0x7fbd741317f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbd680098f0 tx=0x7fbd68009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=958b7ed71c3d1416 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:32.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.756+0000 7fbd727fc700 1 -- 172.21.15.3:0/408806905 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd68011830 con 0x7fbd7409f240 2023-10-03T10:15:32.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.756+0000 7fbd727fc700 1 -- 172.21.15.3:0/408806905 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd68011990 con 0x7fbd7409f240 2023-10-03T10:15:32.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.756+0000 7fbd727fc700 1 -- 172.21.15.3:0/408806905 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd68011d20 con 0x7fbd7409f240 2023-10-03T10:15:32.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.757+0000 7fbd79ba7700 1 -- 172.21.15.3:0/408806905 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7409f240 msgr2=0x7fbd741317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:32.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.757+0000 7fbd79ba7700 1 --2- 172.21.15.3:0/408806905 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7409f240 0x7fbd741317f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbd680098f0 tx=0x7fbd68009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:32.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.757+0000 7fbd79ba7700 1 -- 172.21.15.3:0/408806905 shutdown_connections 2023-10-03T10:15:32.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.757+0000 7fbd79ba7700 1 --2- 172.21.15.3:0/408806905 >> v2:172.21.15.3:3300/0 conn(0x7fbd7409e930 0x7fbd7409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:32.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.757+0000 7fbd79ba7700 1 --2- 172.21.15.3:0/408806905 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7409f240 0x7fbd741317f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:32.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.757+0000 7fbd79ba7700 1 -- 172.21.15.3:0/408806905 >> 172.21.15.3:0/408806905 conn(0x7fbd740a4210 msgr2=0x7fbd740a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:32.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.757+0000 7fbd79ba7700 1 -- 172.21.15.3:0/408806905 shutdown_connections 2023-10-03T10:15:32.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.757+0000 7fbd79ba7700 1 -- 172.21.15.3:0/408806905 wait complete. 2023-10-03T10:15:32.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.758+0000 7fbd79ba7700 1 Processor -- start 2023-10-03T10:15:32.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.759+0000 7fbd79ba7700 1 -- start start 2023-10-03T10:15:32.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.759+0000 7fbd79ba7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409e930 0x7fbd741a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:32.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.759+0000 7fbd79ba7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd7409f240 0x7fbd741a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:32.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.759+0000 7fbd79ba7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7409f650 0x7fbd741a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:32.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.760+0000 7fbd79ba7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd741a5e30 con 0x7fbd7409f240 2023-10-03T10:15:32.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.760+0000 7fbd79ba7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd741a6ef0 con 0x7fbd7409f650 2023-10-03T10:15:32.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.760+0000 7fbd79ba7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd74071d20 con 0x7fbd7409e930 2023-10-03T10:15:32.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.760+0000 7fbd73fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7409f650 0x7fbd741a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:32.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.760+0000 7fbd73fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7409f650 0x7fbd741a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46800/0 (socket says 172.21.15.3:46800) 2023-10-03T10:15:32.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.760+0000 7fbd73fff700 1 -- 172.21.15.3:0/1314316083 learned_addr learned my addr 172.21.15.3:0/1314316083 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:32.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.760+0000 7fbd72ffd700 1 --2- 172.21.15.3:0/1314316083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd7409f240 0x7fbd741a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:32.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.760+0000 7fbd73fff700 1 -- 172.21.15.3:0/1314316083 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409e930 msgr2=0x7fbd741a3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:32.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.760+0000 7fbd73fff700 1 --2- 172.21.15.3:0/1314316083 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409e930 0x7fbd741a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:32.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.760+0000 7fbd73fff700 1 -- 172.21.15.3:0/1314316083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd7409f240 msgr2=0x7fbd741a4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:32.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.760+0000 7fbd73fff700 1 --2- 172.21.15.3:0/1314316083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd7409f240 0x7fbd741a4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:32.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.760+0000 7fbd73fff700 1 -- 172.21.15.3:0/1314316083 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd68009550 con 0x7fbd7409f650 2023-10-03T10:15:32.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.760+0000 7fbd73fff700 1 --2- 172.21.15.3:0/1314316083 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7409f650 0x7fbd741a46c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00cc50 tx=0x7fbd6c00cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:32.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.761+0000 7fbd5bfff700 1 -- 172.21.15.3:0/1314316083 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c003c00 con 0x7fbd7409f650 2023-10-03T10:15:32.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.761+0000 7fbd5bfff700 1 -- 172.21.15.3:0/1314316083 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6c003d60 con 0x7fbd7409f650 2023-10-03T10:15:32.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.761+0000 7fbd5bfff700 1 -- 172.21.15.3:0/1314316083 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c00ab60 con 0x7fbd7409f650 2023-10-03T10:15:32.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.761+0000 7fbd79ba7700 1 -- 172.21.15.3:0/1314316083 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd740a0190 con 0x7fbd7409f650 2023-10-03T10:15:32.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.761+0000 7fbd79ba7700 1 -- 172.21.15.3:0/1314316083 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd741a5850 con 0x7fbd7409f650 2023-10-03T10:15:32.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.762+0000 7fbd5bfff700 1 -- 172.21.15.3:0/1314316083 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbd6c00acc0 con 0x7fbd7409f650 2023-10-03T10:15:32.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.762+0000 7fbd5bfff700 1 -- 172.21.15.3:0/1314316083 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fbd6c03f800 con 0x7fbd7409f650 2023-10-03T10:15:32.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.763+0000 7fbd79ba7700 1 -- 172.21.15.3:0/1314316083 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd741a0b30 con 0x7fbd7409f650 2023-10-03T10:15:32.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.772+0000 7fbd5bfff700 1 -- 172.21.15.3:0/1314316083 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbd6c0198f0 con 0x7fbd7409f650 2023-10-03T10:15:32.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.789+0000 7f4f9c786700 1 Processor -- start 2023-10-03T10:15:32.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.789+0000 7f4f9c786700 1 -- start start 2023-10-03T10:15:32.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.789+0000 7f4f9c786700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f9409d6f0 0x7f4f9409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:32.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.790+0000 7f4f9c786700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4f9409e410 0x7f4f94134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:32.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.790+0000 7f4f9c786700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f9409b120 con 0x7f4f9409e000 2023-10-03T10:15:32.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.790+0000 7f4f9c786700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f9406b460 con 0x7f4f9409e410 2023-10-03T10:15:32.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.790+0000 7f4f9c786700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9409a7e0 con 0x7f4f9409d6f0 2023-10-03T10:15:32.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.790+0000 7f4f9a522700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f9409d6f0 0x7f4f9409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:32.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.790+0000 7f4f9a522700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f9409d6f0 0x7f4f9409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46814/0 (socket says 172.21.15.3:46814) 2023-10-03T10:15:32.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.790+0000 7f4f9a522700 1 -- 172.21.15.3:0/4085361532 learned_addr learned my addr 172.21.15.3:0/4085361532 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:32.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.790+0000 7f4f9ad23700 1 --2- 172.21.15.3:0/4085361532 >> v2:172.21.15.3:3300/0 conn(0x7f4f9409e410 0x7f4f94134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:32.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.790+0000 7f4f9a522700 1 -- 172.21.15.3:0/4085361532 >> v2:172.21.15.3:3300/0 conn(0x7f4f9409e410 msgr2=0x7f4f94134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:32.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.790+0000 7f4f9a522700 1 --2- 172.21.15.3:0/4085361532 >> v2:172.21.15.3:3300/0 conn(0x7f4f9409e410 0x7f4f94134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:32.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.790+0000 7f4f9a522700 1 -- 172.21.15.3:0/4085361532 >> v1:172.21.15.3:6789/0 conn(0x7f4f9409e000 legacy=0x7f4f941339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:32.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.790+0000 7f4f9a522700 1 -- 172.21.15.3:0/4085361532 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9409a500 con 0x7f4f9409d6f0 2023-10-03T10:15:32.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.790+0000 7f4f9a522700 1 --2- 172.21.15.3:0/4085361532 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f9409d6f0 0x7f4f9409dac0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009920 tx=0x7f4f90009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc676b9781f70273 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:32.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.791+0000 7f4f99520700 1 -- 172.21.15.3:0/4085361532 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f90011830 con 0x7f4f9409d6f0 2023-10-03T10:15:32.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.791+0000 7f4f99520700 1 -- 172.21.15.3:0/4085361532 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f90011990 con 0x7f4f9409d6f0 2023-10-03T10:15:32.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.791+0000 7f4f99520700 1 -- 172.21.15.3:0/4085361532 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f90011d20 con 0x7f4f9409d6f0 2023-10-03T10:15:32.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.791+0000 7f4f9c786700 1 -- 172.21.15.3:0/4085361532 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f9409d6f0 msgr2=0x7f4f9409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:32.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.791+0000 7f4f9c786700 1 --2- 172.21.15.3:0/4085361532 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f9409d6f0 0x7f4f9409dac0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009920 tx=0x7f4f90009c30 comp rx=0 tx=0).stop 2023-10-03T10:15:32.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.792+0000 7f4f9c786700 1 -- 172.21.15.3:0/4085361532 shutdown_connections 2023-10-03T10:15:32.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.792+0000 7f4f9c786700 1 --2- 172.21.15.3:0/4085361532 >> v2:172.21.15.3:3300/0 conn(0x7f4f9409e410 0x7f4f94134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:32.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.792+0000 7f4f9c786700 1 --2- 172.21.15.3:0/4085361532 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f9409d6f0 0x7f4f9409dac0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:32.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.792+0000 7f4f9c786700 1 -- 172.21.15.3:0/4085361532 >> 172.21.15.3:0/4085361532 conn(0x7f4f94066f50 msgr2=0x7f4f9409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:32.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.792+0000 7f4f9c786700 1 -- 172.21.15.3:0/4085361532 shutdown_connections 2023-10-03T10:15:32.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.792+0000 7f4f9c786700 1 -- 172.21.15.3:0/4085361532 wait complete. 2023-10-03T10:15:32.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.793+0000 7f4f9c786700 1 Processor -- start 2023-10-03T10:15:32.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.793+0000 7f4f9c786700 1 -- start start 2023-10-03T10:15:32.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.793+0000 7f4f9c786700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4f9409d6f0 0x7f4f941a5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:32.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.793+0000 7f4f9c786700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f9409e000 0x7f4f941a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:32.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.793+0000 7f4f9c786700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f9409e410 0x7f4f941a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:32.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.793+0000 7f4f9c786700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f941a8da0 con 0x7f4f9409e410 2023-10-03T10:15:32.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.793+0000 7f4f9c786700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f941a7410 con 0x7f4f9409e000 2023-10-03T10:15:32.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.793+0000 7f4f9c786700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f941a97f0 con 0x7f4f9409d6f0 2023-10-03T10:15:32.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.793+0000 7f4f99d21700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f9409e000 0x7f4f941a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:32.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.793+0000 7f4f99d21700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f9409e000 0x7f4f941a64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46822/0 (socket says 172.21.15.3:46822) 2023-10-03T10:15:32.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.793+0000 7f4f99d21700 1 -- 172.21.15.3:0/801328708 learned_addr learned my addr 172.21.15.3:0/801328708 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:32.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.793+0000 7f4f9ad23700 1 --2- 172.21.15.3:0/801328708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f9409e410 0x7f4f941a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:32.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.794+0000 7f4f99d21700 1 -- 172.21.15.3:0/801328708 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4f9409d6f0 msgr2=0x7f4f941a5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:32.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.794+0000 7f4f99d21700 1 --2- 172.21.15.3:0/801328708 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4f9409d6f0 0x7f4f941a5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:32.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.794+0000 7f4f99d21700 1 -- 172.21.15.3:0/801328708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f9409e410 msgr2=0x7f4f941a69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:32.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.794+0000 7f4f99d21700 1 --2- 172.21.15.3:0/801328708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f9409e410 0x7f4f941a69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:32.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.794+0000 7f4f99d21700 1 -- 172.21.15.3:0/801328708 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f90009550 con 0x7f4f9409e000 2023-10-03T10:15:32.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.794+0000 7f4f99d21700 1 --2- 172.21.15.3:0/801328708 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f9409e000 0x7f4f941a64a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400c4b0 tx=0x7f4f8400c7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:32.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.794+0000 7f4f8affd700 1 -- 172.21.15.3:0/801328708 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f840052d0 con 0x7f4f9409e000 2023-10-03T10:15:32.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.794+0000 7f4f8affd700 1 -- 172.21.15.3:0/801328708 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f84005430 con 0x7f4f9409e000 2023-10-03T10:15:32.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.794+0000 7f4f9c786700 1 -- 172.21.15.3:0/801328708 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f941a88e0 con 0x7f4f9409e000 2023-10-03T10:15:32.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.794+0000 7f4f8affd700 1 -- 172.21.15.3:0/801328708 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f840038c0 con 0x7f4f9409e000 2023-10-03T10:15:32.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.794+0000 7f4f9c786700 1 -- 172.21.15.3:0/801328708 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f9409c710 con 0x7f4f9409e000 2023-10-03T10:15:32.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.795+0000 7f4f9c786700 1 -- 172.21.15.3:0/801328708 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f941a9930 con 0x7f4f9409e000 2023-10-03T10:15:32.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.795+0000 7f4f8affd700 1 -- 172.21.15.3:0/801328708 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4f84003a20 con 0x7f4f9409e000 2023-10-03T10:15:32.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.796+0000 7f4f8affd700 1 -- 172.21.15.3:0/801328708 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4f8403ec20 con 0x7f4f9409e000 2023-10-03T10:15:32.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.800+0000 7f4f8affd700 1 -- 172.21.15.3:0/801328708 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4f84018260 con 0x7f4f9409e000 2023-10-03T10:15:32.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.833+0000 7f32a1df2700 1 Processor -- start 2023-10-03T10:15:32.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.833+0000 7f32a1df2700 1 -- start start 2023-10-03T10:15:32.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.834+0000 7f32a1df2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f329c09e930 0x7f329c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:32.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.834+0000 7f32a1df2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f329c09f240 0x7f329c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:32.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.834+0000 7f32a1df2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f329c09f650 0x7f329c131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:32.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.834+0000 7f32a1df2700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f329c070940 con 0x7f329c09f650 2023-10-03T10:15:32.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.834+0000 7f32a1df2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f329c06fd60 con 0x7f329c09e930 2023-10-03T10:15:32.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.834+0000 7f32a1df2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f329c06fa80 con 0x7f329c09f240 2023-10-03T10:15:32.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.834+0000 7f329b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f329c09e930 0x7f329c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:32.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.834+0000 7f329b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f329c09e930 0x7f329c09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46834/0 (socket says 172.21.15.3:46834) 2023-10-03T10:15:32.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.834+0000 7f329b7fe700 1 -- 172.21.15.3:0/921375544 learned_addr learned my addr 172.21.15.3:0/921375544 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:32.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.834+0000 7f329bfff700 1 --2- 172.21.15.3:0/921375544 >> v2:172.21.15.3:3300/0 conn(0x7f329c09f650 0x7f329c131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:32.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.834+0000 7f329affd700 1 --2- 172.21.15.3:0/921375544 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f329c09f240 0x7f329c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:32.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.835+0000 7f329bfff700 1 -- 172.21.15.3:0/921375544 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f329c09f240 msgr2=0x7f329c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:32.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.835+0000 7f329bfff700 1 --2- 172.21.15.3:0/921375544 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f329c09f240 0x7f329c1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:32.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.835+0000 7f329bfff700 1 -- 172.21.15.3:0/921375544 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f329c09e930 msgr2=0x7f329c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:32.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.835+0000 7f329bfff700 1 --2- 172.21.15.3:0/921375544 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f329c09e930 0x7f329c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:32.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.835+0000 7f329bfff700 1 -- 172.21.15.3:0/921375544 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f329c09fb60 con 0x7f329c09f650 2023-10-03T10:15:32.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.835+0000 7f329b7fe700 1 --2- 172.21.15.3:0/921375544 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f329c09e930 0x7f329c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:32.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.835+0000 7f329bfff700 1 --2- 172.21.15.3:0/921375544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f329c09f650 0x7f329c131d30 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f3294009d90 tx=0x7f329400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33075d858534333 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:32.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.835+0000 7f329a7fc700 1 -- 172.21.15.3:0/921375544 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329400e070 con 0x7f329c09f650 2023-10-03T10:15:32.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.835+0000 7f329a7fc700 1 -- 172.21.15.3:0/921375544 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329400bb20 con 0x7f329c09f650 2023-10-03T10:15:32.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.835+0000 7f329a7fc700 1 -- 172.21.15.3:0/921375544 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3294012430 con 0x7f329c09f650 2023-10-03T10:15:32.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.836+0000 7f32a1df2700 1 -- 172.21.15.3:0/921375544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f329c09f650 msgr2=0x7f329c131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:32.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.836+0000 7f32a1df2700 1 --2- 172.21.15.3:0/921375544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f329c09f650 0x7f329c131d30 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f3294009d90 tx=0x7f329400b330 comp rx=0 tx=0).stop 2023-10-03T10:15:32.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.836+0000 7f32a1df2700 1 -- 172.21.15.3:0/921375544 shutdown_connections 2023-10-03T10:15:32.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.836+0000 7f32a1df2700 1 --2- 172.21.15.3:0/921375544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f329c09f650 0x7f329c131d30 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:32.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.836+0000 7f32a1df2700 1 --2- 172.21.15.3:0/921375544 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f329c09e930 0x7f329c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:32.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.836+0000 7f32a1df2700 1 --2- 172.21.15.3:0/921375544 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f329c09f240 0x7f329c1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:32.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.836+0000 7f32a1df2700 1 -- 172.21.15.3:0/921375544 >> 172.21.15.3:0/921375544 conn(0x7f329c0a4210 msgr2=0x7f329c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:32.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.836+0000 7f32a1df2700 1 -- 172.21.15.3:0/921375544 shutdown_connections 2023-10-03T10:15:32.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.836+0000 7f32a1df2700 1 -- 172.21.15.3:0/921375544 wait complete. 2023-10-03T10:15:32.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.838+0000 7f32a1df2700 1 Processor -- start 2023-10-03T10:15:32.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.838+0000 7f32a1df2700 1 -- start start 2023-10-03T10:15:32.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.839+0000 7f32a1df2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f329c09e930 0x7f329c06b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:32.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.839+0000 7f32a1df2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f329c09f240 0x7f329c06bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:32.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.839+0000 7f32a1df2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f329c09f650 0x7f329c06c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:32.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.839+0000 7f32a1df2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c1b7270 con 0x7f329c09f650 2023-10-03T10:15:32.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.839+0000 7f32a1df2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f329c06cc80 con 0x7f329c09e930 2023-10-03T10:15:32.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.839+0000 7f32a1df2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f329c1a8150 con 0x7f329c09f240 2023-10-03T10:15:32.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.839+0000 7f329b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f329c09e930 0x7f329c06b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:32.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.839+0000 7f329b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f329c09e930 0x7f329c06b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46842/0 (socket says 172.21.15.3:46842) 2023-10-03T10:15:32.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.839+0000 7f329b7fe700 1 -- 172.21.15.3:0/3571796458 learned_addr learned my addr 172.21.15.3:0/3571796458 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:32.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.839+0000 7f329bfff700 1 --2- 172.21.15.3:0/3571796458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f329c09f650 0x7f329c06c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:32.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.839+0000 7f329affd700 1 --2- 172.21.15.3:0/3571796458 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f329c09f240 0x7f329c06bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:32.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.839+0000 7f329b7fe700 1 -- 172.21.15.3:0/3571796458 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f329c09f240 msgr2=0x7f329c06bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:32.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.839+0000 7f329b7fe700 1 --2- 172.21.15.3:0/3571796458 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f329c09f240 0x7f329c06bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:32.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.839+0000 7f329b7fe700 1 -- 172.21.15.3:0/3571796458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f329c09f650 msgr2=0x7f329c06c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:32.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.839+0000 7f329b7fe700 1 --2- 172.21.15.3:0/3571796458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f329c09f650 0x7f329c06c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:32.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.839+0000 7f329b7fe700 1 -- 172.21.15.3:0/3571796458 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328c009550 con 0x7f329c09e930 2023-10-03T10:15:32.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.839+0000 7f329bfff700 1 --2- 172.21.15.3:0/3571796458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f329c09f650 0x7f329c06c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:15:32.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.840+0000 7f329b7fe700 1 --2- 172.21.15.3:0/3571796458 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f329c09e930 0x7f329c06b880 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f328c00ebb0 tx=0x7f328c00ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.840+0000 7f3283fff700 1 -- 172.21.15.3:0/3571796458 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c004030 con 0x7f329c09e930 2023-10-03T10:15:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.840+0000 7f3283fff700 1 -- 172.21.15.3:0/3571796458 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f328c004190 con 0x7f329c09e930 2023-10-03T10:15:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.840+0000 7f32a1df2700 1 -- 172.21.15.3:0/3571796458 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3294009a70 con 0x7f329c09e930 2023-10-03T10:15:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.840+0000 7f32a1df2700 1 -- 172.21.15.3:0/3571796458 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329c0a0190 con 0x7f329c09e930 2023-10-03T10:15:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.841+0000 7f3283fff700 1 -- 172.21.15.3:0/3571796458 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c004520 con 0x7f329c09e930 2023-10-03T10:15:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.841+0000 7f3283fff700 1 -- 172.21.15.3:0/3571796458 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f328c00c850 con 0x7f329c09e930 2023-10-03T10:15:32.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.841+0000 7f3283fff700 1 -- 172.21.15.3:0/3571796458 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f328c03f7a0 con 0x7f329c09e930 2023-10-03T10:15:32.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.841+0000 7f32a1df2700 1 -- 172.21.15.3:0/3571796458 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f327c0052e0 con 0x7f329c09e930 2023-10-03T10:15:32.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:32.846+0000 7f3283fff700 1 -- 172.21.15.3:0/3571796458 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f328c0036a0 con 0x7f329c09e930 2023-10-03T10:15:33.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.009+0000 7f346d44c700 1 -- 172.21.15.3:0/3099478183 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f346806f9d0 con 0x7f346813d1f0 2023-10-03T10:15:33.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.009+0000 7f344f7fe700 1 -- 172.21.15.3:0/3099478183 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f346806f9d0 con 0x7f346813d1f0 2023-10-03T10:15:33.010 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:33.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.011+0000 7f4f9c786700 1 -- 172.21.15.3:0/801328708 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4f94074ea0 con 0x7f4f9409e000 2023-10-03T10:15:33.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.011+0000 7f4f8affd700 1 -- 172.21.15.3:0/801328708 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4f84017020 con 0x7f4f9409e000 2023-10-03T10:15:33.012 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:33.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.012+0000 7f346d44c700 1 -- 172.21.15.3:0/3099478183 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f346813d1f0 msgr2=0x7f3468071670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:33.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.012+0000 7f346d44c700 1 --2- 172.21.15.3:0/3099478183 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f346813d1f0 0x7f3468071670 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3460009d90 tx=0x7f346000c3a0 comp rx=0 tx=0).stop 2023-10-03T10:15:33.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.012+0000 7f346d44c700 1 -- 172.21.15.3:0/3099478183 shutdown_connections 2023-10-03T10:15:33.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.012+0000 7f346d44c700 1 --2- 172.21.15.3:0/3099478183 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f346813d1f0 0x7f3468071670 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:33.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.012+0000 7f346d44c700 1 --2- 172.21.15.3:0/3099478183 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f346806a770 0x7f3468070bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:33.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.012+0000 7f346d44c700 1 --2- 172.21.15.3:0/3099478183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f346809de00 0x7f3468071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:33.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.012+0000 7f346d44c700 1 -- 172.21.15.3:0/3099478183 >> 172.21.15.3:0/3099478183 conn(0x7f3468066f50 msgr2=0x7f346813c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:33.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.014+0000 7f346d44c700 1 -- 172.21.15.3:0/3099478183 shutdown_connections 2023-10-03T10:15:33.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.014+0000 7f346d44c700 1 -- 172.21.15.3:0/3099478183 wait complete. 2023-10-03T10:15:33.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.014+0000 7f4f9c786700 1 -- 172.21.15.3:0/801328708 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f9409e000 msgr2=0x7f4f941a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:33.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.014+0000 7f4f9c786700 1 --2- 172.21.15.3:0/801328708 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f9409e000 0x7f4f941a64a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400c4b0 tx=0x7f4f8400c7c0 comp rx=0 tx=0).stop 2023-10-03T10:15:33.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.014+0000 7f4f9c786700 1 -- 172.21.15.3:0/801328708 shutdown_connections 2023-10-03T10:15:33.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.014+0000 7f4f9c786700 1 --2- 172.21.15.3:0/801328708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f9409e410 0x7f4f941a69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:33.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.014+0000 7f4f9c786700 1 --2- 172.21.15.3:0/801328708 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4f9409d6f0 0x7f4f941a5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:33.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.014+0000 7f4f9c786700 1 --2- 172.21.15.3:0/801328708 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f9409e000 0x7f4f941a64a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:33.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.014+0000 7f4f9c786700 1 -- 172.21.15.3:0/801328708 >> 172.21.15.3:0/801328708 conn(0x7f4f94066f50 msgr2=0x7f4f941baed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:33.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.014+0000 7f4f9c786700 1 -- 172.21.15.3:0/801328708 shutdown_connections 2023-10-03T10:15:33.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.014+0000 7f4f9c786700 1 -- 172.21.15.3:0/801328708 wait complete. 2023-10-03T10:15:33.035 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:33.035 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.053+0000 7f32a1df2700 1 -- 172.21.15.3:0/3571796458 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f327c005590 con 0x7f329c09e930 2023-10-03T10:15:33.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.054+0000 7f3283fff700 1 -- 172.21.15.3:0/3571796458 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f328c020020 con 0x7f329c09e930 2023-10-03T10:15:33.054 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:33.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.056+0000 7f32a1df2700 1 -- 172.21.15.3:0/3571796458 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f329c09e930 msgr2=0x7f329c06b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:33.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.056+0000 7f32a1df2700 1 --2- 172.21.15.3:0/3571796458 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f329c09e930 0x7f329c06b880 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f328c00ebb0 tx=0x7f328c00ef70 comp rx=0 tx=0).stop 2023-10-03T10:15:33.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.056+0000 7f32a1df2700 1 -- 172.21.15.3:0/3571796458 shutdown_connections 2023-10-03T10:15:33.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.056+0000 7f32a1df2700 1 --2- 172.21.15.3:0/3571796458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f329c09f650 0x7f329c06c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:33.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.056+0000 7f32a1df2700 1 --2- 172.21.15.3:0/3571796458 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f329c09e930 0x7f329c06b880 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:33.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.056+0000 7f32a1df2700 1 --2- 172.21.15.3:0/3571796458 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f329c09f240 0x7f329c06bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:33.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.056+0000 7f32a1df2700 1 -- 172.21.15.3:0/3571796458 >> 172.21.15.3:0/3571796458 conn(0x7f329c0a4210 msgr2=0x7f329c1b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:33.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.056+0000 7f32a1df2700 1 -- 172.21.15.3:0/3571796458 shutdown_connections 2023-10-03T10:15:33.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.056+0000 7f32a1df2700 1 -- 172.21.15.3:0/3571796458 wait complete. 2023-10-03T10:15:33.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.056+0000 7fbd79ba7700 1 -- 172.21.15.3:0/1314316083 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbd74074ea0 con 0x7fbd7409f650 2023-10-03T10:15:33.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.057+0000 7fbd5bfff700 1 -- 172.21.15.3:0/1314316083 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbd6c019300 con 0x7fbd7409f650 2023-10-03T10:15:33.058 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:33.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.059+0000 7fbd79ba7700 1 -- 172.21.15.3:0/1314316083 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7409f650 msgr2=0x7fbd741a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:33.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.059+0000 7fbd79ba7700 1 --2- 172.21.15.3:0/1314316083 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7409f650 0x7fbd741a46c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00cc50 tx=0x7fbd6c00cf60 comp rx=0 tx=0).stop 2023-10-03T10:15:33.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.059+0000 7fbd79ba7700 1 -- 172.21.15.3:0/1314316083 shutdown_connections 2023-10-03T10:15:33.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.059+0000 7fbd79ba7700 1 --2- 172.21.15.3:0/1314316083 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7409f650 0x7fbd741a46c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:33.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.059+0000 7fbd79ba7700 1 --2- 172.21.15.3:0/1314316083 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409e930 0x7fbd741a3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:33.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.059+0000 7fbd79ba7700 1 --2- 172.21.15.3:0/1314316083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd7409f240 0x7fbd741a4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:33.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.059+0000 7fbd79ba7700 1 -- 172.21.15.3:0/1314316083 >> 172.21.15.3:0/1314316083 conn(0x7fbd740a4210 msgr2=0x7fbd741b8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:33.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.059+0000 7fbd79ba7700 1 -- 172.21.15.3:0/1314316083 shutdown_connections 2023-10-03T10:15:33.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:33.059+0000 7fbd79ba7700 1 -- 172.21.15.3:0/1314316083 wait complete. 2023-10-03T10:15:33.071 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:33.073 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:34.036 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:34.036 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:34.071 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:34.074 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:34.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.190+0000 7f382b720700 1 Processor -- start 2023-10-03T10:15:34.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.190+0000 7f382b720700 1 -- start start 2023-10-03T10:15:34.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.191+0000 7f382b720700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f382409d6f0 0x7f382409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:34.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.191+0000 7f382b720700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f382409e000 0x7f38241339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:34.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.191+0000 7f382b720700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f382409b120 con 0x7f382409e410 2023-10-03T10:15:34.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.191+0000 7f382b720700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f382406b460 con 0x7f382409e000 2023-10-03T10:15:34.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.191+0000 7f382b720700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f382409a7e0 con 0x7f382409d6f0 2023-10-03T10:15:34.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.191+0000 7f38294bc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f382409d6f0 0x7f382409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:34.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.191+0000 7f3828cbb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f382409e000 0x7f38241339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:34.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.191+0000 7f38294bc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f382409d6f0 0x7f382409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41760/0 (socket says 172.21.15.3:41760) 2023-10-03T10:15:34.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.191+0000 7f38294bc700 1 -- 172.21.15.3:0/3712431408 learned_addr learned my addr 172.21.15.3:0/3712431408 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:34.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.192+0000 7f38294bc700 1 -- 172.21.15.3:0/3712431408 >> v2:172.21.15.3:3300/0 conn(0x7f382409e000 msgr2=0x7f38241339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:34.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.192+0000 7f38294bc700 1 --2- 172.21.15.3:0/3712431408 >> v2:172.21.15.3:3300/0 conn(0x7f382409e000 0x7f38241339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.192+0000 7f38294bc700 1 -- 172.21.15.3:0/3712431408 >> v1:172.21.15.3:6789/0 conn(0x7f382409e410 legacy=0x7f3824133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:34.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.192+0000 7f38294bc700 1 -- 172.21.15.3:0/3712431408 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f382409a500 con 0x7f382409d6f0 2023-10-03T10:15:34.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.192+0000 7f38294bc700 1 --2- 172.21.15.3:0/3712431408 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f382409d6f0 0x7f382409dac0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3820009920 tx=0x7f3820009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c07aab78695ab22 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:34.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.192+0000 7f381bfff700 1 -- 172.21.15.3:0/3712431408 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3820011830 con 0x7f382409d6f0 2023-10-03T10:15:34.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.192+0000 7f381bfff700 1 -- 172.21.15.3:0/3712431408 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3820011990 con 0x7f382409d6f0 2023-10-03T10:15:34.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.192+0000 7f381bfff700 1 -- 172.21.15.3:0/3712431408 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3820011d20 con 0x7f382409d6f0 2023-10-03T10:15:34.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.193+0000 7f382b720700 1 -- 172.21.15.3:0/3712431408 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f382409d6f0 msgr2=0x7f382409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:34.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.193+0000 7f382b720700 1 --2- 172.21.15.3:0/3712431408 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f382409d6f0 0x7f382409dac0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3820009920 tx=0x7f3820009c30 comp rx=0 tx=0).stop 2023-10-03T10:15:34.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.193+0000 7f382b720700 1 -- 172.21.15.3:0/3712431408 shutdown_connections 2023-10-03T10:15:34.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.193+0000 7f382b720700 1 --2- 172.21.15.3:0/3712431408 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f382409d6f0 0x7f382409dac0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.193+0000 7f382b720700 1 --2- 172.21.15.3:0/3712431408 >> v2:172.21.15.3:3300/0 conn(0x7f382409e000 0x7f38241339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.193+0000 7f382b720700 1 -- 172.21.15.3:0/3712431408 >> 172.21.15.3:0/3712431408 conn(0x7f3824066f50 msgr2=0x7f382409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:34.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.193+0000 7f382b720700 1 -- 172.21.15.3:0/3712431408 shutdown_connections 2023-10-03T10:15:34.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.193+0000 7f382b720700 1 -- 172.21.15.3:0/3712431408 wait complete. 2023-10-03T10:15:34.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.195+0000 7f382b720700 1 Processor -- start 2023-10-03T10:15:34.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.195+0000 7f382b720700 1 -- start start 2023-10-03T10:15:34.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.196+0000 7f382b720700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f382409d6f0 0x7f38241a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:34.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.196+0000 7f382b720700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f382409e000 0x7f38241a6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:34.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.196+0000 7f382b720700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f382409e410 0x7f38241a6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:34.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.196+0000 7f382b720700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38241a9a60 con 0x7f382409d6f0 2023-10-03T10:15:34.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.196+0000 7f382b720700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38241a7bf0 con 0x7f382409e410 2023-10-03T10:15:34.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.196+0000 7f382b720700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38240a33d0 con 0x7f382409e000 2023-10-03T10:15:34.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.196+0000 7f38294bc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f382409d6f0 0x7f38241a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:34.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.196+0000 7f38294bc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f382409d6f0 0x7f38241a5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41964/0 (socket says 172.21.15.3:41964) 2023-10-03T10:15:34.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.196+0000 7f38294bc700 1 -- 172.21.15.3:0/2317383319 learned_addr learned my addr 172.21.15.3:0/2317383319 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:34.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.196+0000 7f3828cbb700 1 --2- 172.21.15.3:0/2317383319 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f382409e000 0x7f38241a6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:34.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.196+0000 7f3829cbd700 1 --2- 172.21.15.3:0/2317383319 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f382409e410 0x7f38241a6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:34.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.196+0000 7f38294bc700 1 -- 172.21.15.3:0/2317383319 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f382409e000 msgr2=0x7f38241a6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:34.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.196+0000 7f38294bc700 1 --2- 172.21.15.3:0/2317383319 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f382409e000 0x7f38241a6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.196+0000 7f38294bc700 1 -- 172.21.15.3:0/2317383319 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f382409e410 msgr2=0x7f38241a6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:34.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.196+0000 7f38294bc700 1 --2- 172.21.15.3:0/2317383319 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f382409e410 0x7f38241a6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.196+0000 7f38294bc700 1 -- 172.21.15.3:0/2317383319 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3820009550 con 0x7f382409d6f0 2023-10-03T10:15:34.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.196+0000 7f38294bc700 1 --2- 172.21.15.3:0/2317383319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f382409d6f0 0x7f38241a5fd0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f3820009870 tx=0x7f3820004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:34.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.197+0000 7f3819ffb700 1 -- 172.21.15.3:0/2317383319 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3820004380 con 0x7f382409d6f0 2023-10-03T10:15:34.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.197+0000 7f3819ffb700 1 -- 172.21.15.3:0/2317383319 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382001f070 con 0x7f382409d6f0 2023-10-03T10:15:34.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.197+0000 7f3819ffb700 1 -- 172.21.15.3:0/2317383319 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38200036a0 con 0x7f382409d6f0 2023-10-03T10:15:34.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.198+0000 7f2b73adf700 1 Processor -- start 2023-10-03T10:15:34.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.198+0000 7f382b720700 1 -- 172.21.15.3:0/2317383319 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f382409c710 con 0x7f382409d6f0 2023-10-03T10:15:34.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.198+0000 7f382b720700 1 -- 172.21.15.3:0/2317383319 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38241a8950 con 0x7f382409d6f0 2023-10-03T10:15:34.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.198+0000 7f2b73adf700 1 -- start start 2023-10-03T10:15:34.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.199+0000 7f2b73adf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2b6c09e930 0x7f2b6c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:34.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.199+0000 7f2b73adf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b6c09f240 0x7f2b6c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:34.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.199+0000 7f2b73adf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b6c09f650 0x7f2b6c131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:34.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.199+0000 7f2b73adf700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b6c070940 con 0x7f2b6c09e930 2023-10-03T10:15:34.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.199+0000 7f2b73adf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c06fd60 con 0x7f2b6c09f650 2023-10-03T10:15:34.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.199+0000 7f2b73adf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c06fa80 con 0x7f2b6c09f240 2023-10-03T10:15:34.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.199+0000 7f2b7207c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b6c09f650 0x7f2b6c131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:34.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.199+0000 7f2b7207c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b6c09f650 0x7f2b6c131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50300/0 (socket says 172.21.15.3:50300) 2023-10-03T10:15:34.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.199+0000 7f2b7207c700 1 -- 172.21.15.3:0/2914314807 learned_addr learned my addr 172.21.15.3:0/2914314807 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:34.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.199+0000 7f2b7207c700 1 -- 172.21.15.3:0/2914314807 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b6c09f240 msgr2=0x7f2b6c1317f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-03T10:15:34.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.199+0000 7f3819ffb700 1 -- 172.21.15.3:0/2317383319 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3820021030 con 0x7f382409d6f0 2023-10-03T10:15:34.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.199+0000 7f2b7207c700 1 --2- 172.21.15.3:0/2914314807 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b6c09f240 0x7f2b6c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.199+0000 7f2b7207c700 1 -- 172.21.15.3:0/2914314807 >> v2:172.21.15.3:3300/0 conn(0x7f2b6c09e930 msgr2=0x7f2b6c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:34.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.199+0000 7f2b7207c700 1 --2- 172.21.15.3:0/2914314807 >> v2:172.21.15.3:3300/0 conn(0x7f2b6c09e930 0x7f2b6c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.199+0000 7f382b720700 1 -- 172.21.15.3:0/2317383319 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3824074ea0 con 0x7f382409d6f0 2023-10-03T10:15:34.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.199+0000 7f2b7207c700 1 -- 172.21.15.3:0/2914314807 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6c09fb60 con 0x7f2b6c09f650 2023-10-03T10:15:34.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.200+0000 7f2b7207c700 1 --2- 172.21.15.3:0/2914314807 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b6c09f650 0x7f2b6c131d30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009e50 tx=0x7f2b6400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4afb42283fb762f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:34.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.200+0000 7f3819ffb700 1 -- 172.21.15.3:0/2317383319 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3820040a20 con 0x7f382409d6f0 2023-10-03T10:15:34.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.200+0000 7f2b70879700 1 -- 172.21.15.3:0/2914314807 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6400e070 con 0x7f2b6c09f650 2023-10-03T10:15:34.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.200+0000 7f2b70879700 1 -- 172.21.15.3:0/2914314807 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6400b9c0 con 0x7f2b6c09f650 2023-10-03T10:15:34.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.200+0000 7f2b70879700 1 -- 172.21.15.3:0/2914314807 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6400bd70 con 0x7f2b6c09f650 2023-10-03T10:15:34.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.201+0000 7f2b73adf700 1 -- 172.21.15.3:0/2914314807 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b6c09f650 msgr2=0x7f2b6c131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:34.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.201+0000 7f2b73adf700 1 --2- 172.21.15.3:0/2914314807 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b6c09f650 0x7f2b6c131d30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009e50 tx=0x7f2b6400b330 comp rx=0 tx=0).stop 2023-10-03T10:15:34.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.201+0000 7f2b73adf700 1 -- 172.21.15.3:0/2914314807 shutdown_connections 2023-10-03T10:15:34.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.201+0000 7f2b73adf700 1 --2- 172.21.15.3:0/2914314807 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b6c09f650 0x7f2b6c131d30 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.201+0000 7f2b73adf700 1 --2- 172.21.15.3:0/2914314807 >> v2:172.21.15.3:3300/0 conn(0x7f2b6c09e930 0x7f2b6c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.201+0000 7f2b73adf700 1 --2- 172.21.15.3:0/2914314807 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b6c09f240 0x7f2b6c1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.201+0000 7f2b73adf700 1 -- 172.21.15.3:0/2914314807 >> 172.21.15.3:0/2914314807 conn(0x7f2b6c0a4210 msgr2=0x7f2b6c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:34.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.201+0000 7f2b73adf700 1 -- 172.21.15.3:0/2914314807 shutdown_connections 2023-10-03T10:15:34.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.201+0000 7f2b73adf700 1 -- 172.21.15.3:0/2914314807 wait complete. 2023-10-03T10:15:34.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.203+0000 7f2b73adf700 1 Processor -- start 2023-10-03T10:15:34.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.203+0000 7f2b73adf700 1 -- start start 2023-10-03T10:15:34.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.204+0000 7f2b73adf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b6c09e930 0x7f2b6c1a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:34.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.204+0000 7f2b73adf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b6c09f240 0x7f2b6c1a4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:34.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.204+0000 7f2b73adf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b6c09f650 0x7f2b6c1a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:34.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.204+0000 7f2b73adf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c1a80f0 con 0x7f2b6c09e930 2023-10-03T10:15:34.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.204+0000 7f2b73adf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c1a58c0 con 0x7f2b6c09f240 2023-10-03T10:15:34.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.204+0000 7f2b73adf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c1a7450 con 0x7f2b6c09f650 2023-10-03T10:15:34.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.204+0000 7f2b7107a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b6c09f240 0x7f2b6c1a4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:34.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.204+0000 7f2b7187b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b6c09e930 0x7f2b6c1a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:34.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.204+0000 7f2b7187b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b6c09e930 0x7f2b6c1a3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41986/0 (socket says 172.21.15.3:41986) 2023-10-03T10:15:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.204+0000 7f2b7187b700 1 -- 172.21.15.3:0/2978369220 learned_addr learned my addr 172.21.15.3:0/2978369220 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.204+0000 7f2b7207c700 1 --2- 172.21.15.3:0/2978369220 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b6c09f650 0x7f2b6c1a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.204+0000 7f2b7107a700 1 -- 172.21.15.3:0/2978369220 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b6c09f650 msgr2=0x7f2b6c1a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.204+0000 7f2b7107a700 1 --2- 172.21.15.3:0/2978369220 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b6c09f650 0x7f2b6c1a4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.204+0000 7f2b7107a700 1 -- 172.21.15.3:0/2978369220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b6c09e930 msgr2=0x7f2b6c1a3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.204+0000 7f2b7107a700 1 --2- 172.21.15.3:0/2978369220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b6c09e930 0x7f2b6c1a3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.204+0000 7f2b7107a700 1 -- 172.21.15.3:0/2978369220 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b64009b30 con 0x7f2b6c09f240 2023-10-03T10:15:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.205+0000 7f2b7107a700 1 --2- 172.21.15.3:0/2978369220 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b6c09f240 0x7f2b6c1a4150 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00b480 tx=0x7f2b5c00b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.205+0000 7f2b627fc700 1 -- 172.21.15.3:0/2978369220 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c011830 con 0x7f2b6c09f240 2023-10-03T10:15:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.205+0000 7f2b627fc700 1 -- 172.21.15.3:0/2978369220 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5c011990 con 0x7f2b6c09f240 2023-10-03T10:15:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.205+0000 7f2b73adf700 1 -- 172.21.15.3:0/2978369220 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b6c0a0190 con 0x7f2b6c09f240 2023-10-03T10:15:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.205+0000 7f2b627fc700 1 -- 172.21.15.3:0/2978369220 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c011d40 con 0x7f2b6c09f240 2023-10-03T10:15:34.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.205+0000 7f2b73adf700 1 -- 172.21.15.3:0/2978369220 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b6c1a6a70 con 0x7f2b6c09f240 2023-10-03T10:15:34.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.206+0000 7f3819ffb700 1 -- 172.21.15.3:0/2317383319 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f38200239c0 con 0x7f382409d6f0 2023-10-03T10:15:34.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.206+0000 7f2b627fc700 1 -- 172.21.15.3:0/2978369220 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2b5c01a400 con 0x7f2b6c09f240 2023-10-03T10:15:34.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.206+0000 7f2b627fc700 1 -- 172.21.15.3:0/2978369220 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2b5c03f6e0 con 0x7f2b6c09f240 2023-10-03T10:15:34.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.206+0000 7f2b73adf700 1 -- 172.21.15.3:0/2978369220 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b6c074ea0 con 0x7f2b6c09f240 2023-10-03T10:15:34.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.211+0000 7f2b627fc700 1 -- 172.21.15.3:0/2978369220 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2b5c015070 con 0x7f2b6c09f240 2023-10-03T10:15:34.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.232+0000 7f414379e700 1 Processor -- start 2023-10-03T10:15:34.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.232+0000 7f414379e700 1 -- start start 2023-10-03T10:15:34.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.233+0000 7f414379e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f413c09e1d0 0x7f413c135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:34.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.233+0000 7f414379e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f413c135e50 0x7f413c138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:34.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.233+0000 7f414379e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f413c09f130 con 0x7f413c09d6f0 2023-10-03T10:15:34.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.233+0000 7f414379e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f413c09f520 con 0x7f413c135e50 2023-10-03T10:15:34.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.233+0000 7f414379e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f413c09e930 con 0x7f413c09e1d0 2023-10-03T10:15:34.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.233+0000 7f4140d39700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f413c09e1d0 0x7f413c135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:34.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.233+0000 7f4141d3b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f413c135e50 0x7f413c138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:34.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.233+0000 7f4140d39700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f413c09e1d0 0x7f413c135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41786/0 (socket says 172.21.15.3:41786) 2023-10-03T10:15:34.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.233+0000 7f4140d39700 1 -- 172.21.15.3:0/2492390027 learned_addr learned my addr 172.21.15.3:0/2492390027 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:34.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.233+0000 7f4140d39700 1 -- 172.21.15.3:0/2492390027 >> v2:172.21.15.3:3300/0 conn(0x7f413c135e50 msgr2=0x7f413c138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:34.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.233+0000 7f4140d39700 1 --2- 172.21.15.3:0/2492390027 >> v2:172.21.15.3:3300/0 conn(0x7f413c135e50 0x7f413c138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.233+0000 7f4140d39700 1 -- 172.21.15.3:0/2492390027 >> v1:172.21.15.3:6789/0 conn(0x7f413c09d6f0 legacy=0x7f413c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:34.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.233+0000 7f4140d39700 1 -- 172.21.15.3:0/2492390027 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413c09b2d0 con 0x7f413c09e1d0 2023-10-03T10:15:34.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.234+0000 7f4140d39700 1 --2- 172.21.15.3:0/2492390027 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f413c09e1d0 0x7f413c135910 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f412c009920 tx=0x7f412c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be8a51addbca76ef server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:34.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.235+0000 7f4133fff700 1 -- 172.21.15.3:0/2492390027 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c004230 con 0x7f413c09e1d0 2023-10-03T10:15:34.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.235+0000 7f4133fff700 1 -- 172.21.15.3:0/2492390027 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412c004390 con 0x7f413c09e1d0 2023-10-03T10:15:34.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.235+0000 7f4133fff700 1 -- 172.21.15.3:0/2492390027 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c0036a0 con 0x7f413c09e1d0 2023-10-03T10:15:34.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.235+0000 7f414379e700 1 -- 172.21.15.3:0/2492390027 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f413c09e1d0 msgr2=0x7f413c135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:34.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.235+0000 7f414379e700 1 --2- 172.21.15.3:0/2492390027 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f413c09e1d0 0x7f413c135910 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f412c009920 tx=0x7f412c009c30 comp rx=0 tx=0).stop 2023-10-03T10:15:34.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.235+0000 7f414379e700 1 -- 172.21.15.3:0/2492390027 shutdown_connections 2023-10-03T10:15:34.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.235+0000 7f414379e700 1 --2- 172.21.15.3:0/2492390027 >> v2:172.21.15.3:3300/0 conn(0x7f413c135e50 0x7f413c138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.235+0000 7f414379e700 1 --2- 172.21.15.3:0/2492390027 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f413c09e1d0 0x7f413c135910 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.235+0000 7f414379e700 1 -- 172.21.15.3:0/2492390027 >> 172.21.15.3:0/2492390027 conn(0x7f413c066f50 msgr2=0x7f413c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:34.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.236+0000 7f414379e700 1 -- 172.21.15.3:0/2492390027 shutdown_connections 2023-10-03T10:15:34.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.236+0000 7f414379e700 1 -- 172.21.15.3:0/2492390027 wait complete. 2023-10-03T10:15:34.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.237+0000 7f414379e700 1 Processor -- start 2023-10-03T10:15:34.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.237+0000 7f414379e700 1 -- start start 2023-10-03T10:15:34.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.237+0000 7f414379e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f413c09d6f0 0x7f413c1329f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:34.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.237+0000 7f414379e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f413c09e1d0 0x7f413c132f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:34.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.237+0000 7f414379e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f413c135e50 0x7f413c133470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:34.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.237+0000 7f414379e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413c119600 con 0x7f413c135e50 2023-10-03T10:15:34.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.237+0000 7f414379e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f413c0a2310 con 0x7f413c09e1d0 2023-10-03T10:15:34.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.237+0000 7f414379e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f413c134bb0 con 0x7f413c09d6f0 2023-10-03T10:15:34.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.238+0000 7f4141d3b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f413c135e50 0x7f413c133470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:34.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.238+0000 7f4140d39700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f413c09e1d0 0x7f413c132f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:34.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.238+0000 7f4141d3b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f413c135e50 0x7f413c133470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42006/0 (socket says 172.21.15.3:42006) 2023-10-03T10:15:34.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.238+0000 7f4141d3b700 1 -- 172.21.15.3:0/113693660 learned_addr learned my addr 172.21.15.3:0/113693660 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:34.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.238+0000 7f414153a700 1 --2- 172.21.15.3:0/113693660 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f413c09d6f0 0x7f413c1329f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:34.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.238+0000 7f4141d3b700 1 -- 172.21.15.3:0/113693660 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f413c09d6f0 msgr2=0x7f413c1329f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:34.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.238+0000 7f4141d3b700 1 --2- 172.21.15.3:0/113693660 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f413c09d6f0 0x7f413c1329f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.238+0000 7f4141d3b700 1 -- 172.21.15.3:0/113693660 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f413c09e1d0 msgr2=0x7f413c132f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:34.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.238+0000 7f4141d3b700 1 --2- 172.21.15.3:0/113693660 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f413c09e1d0 0x7f413c132f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.238+0000 7f4141d3b700 1 -- 172.21.15.3:0/113693660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412c009550 con 0x7f413c135e50 2023-10-03T10:15:34.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.238+0000 7f4140d39700 1 --2- 172.21.15.3:0/113693660 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f413c09e1d0 0x7f413c132f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:34.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.238+0000 7f4141d3b700 1 --2- 172.21.15.3:0/113693660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f413c135e50 0x7f413c133470 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f413400ef90 tx=0x7f413400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:34.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.238+0000 7f4131ffb700 1 -- 172.21.15.3:0/113693660 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413400cd40 con 0x7f413c135e50 2023-10-03T10:15:34.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.239+0000 7f4131ffb700 1 -- 172.21.15.3:0/113693660 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f413400f040 con 0x7f413c135e50 2023-10-03T10:15:34.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.239+0000 7f4131ffb700 1 -- 172.21.15.3:0/113693660 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413400cea0 con 0x7f413c135e50 2023-10-03T10:15:34.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.239+0000 7f414379e700 1 -- 172.21.15.3:0/113693660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f413c119c40 con 0x7f413c135e50 2023-10-03T10:15:34.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.239+0000 7f414379e700 1 -- 172.21.15.3:0/113693660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f413c1198c0 con 0x7f413c135e50 2023-10-03T10:15:34.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.240+0000 7f4131ffb700 1 -- 172.21.15.3:0/113693660 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f41340146a0 con 0x7f413c135e50 2023-10-03T10:15:34.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.240+0000 7f4131ffb700 1 -- 172.21.15.3:0/113693660 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f41340403f0 con 0x7f413c135e50 2023-10-03T10:15:34.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.241+0000 7f414379e700 1 -- 172.21.15.3:0/113693660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f413c09c790 con 0x7f413c135e50 2023-10-03T10:15:34.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.244+0000 7f2f0fafe700 1 Processor -- start 2023-10-03T10:15:34.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.245+0000 7f2f0fafe700 1 -- start start 2023-10-03T10:15:34.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.245+0000 7f2f0fafe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2f0806e570 0x7f2f0806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:34.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.245+0000 7f2f0fafe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2f0809eca0 0x7f2f0806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:34.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.245+0000 7f2f0fafe700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f08070ee0 con 0x7f2f08134610 2023-10-03T10:15:34.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.245+0000 7f2f0fafe700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f08071480 con 0x7f2f0809eca0 2023-10-03T10:15:34.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.245+0000 7f2f0fafe700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0806f3c0 con 0x7f2f0806e570 2023-10-03T10:15:34.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.245+0000 7f2f0d89a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2f0806e570 0x7f2f0806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:34.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.246+0000 7f2f0d099700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2f0809eca0 0x7f2f0806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:34.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.246+0000 7f2f0d89a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2f0806e570 0x7f2f0806e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50332/0 (socket says 172.21.15.3:50332) 2023-10-03T10:15:34.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.246+0000 7f2f0d89a700 1 -- 172.21.15.3:0/3378501040 learned_addr learned my addr 172.21.15.3:0/3378501040 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:34.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.246+0000 7f2f0d89a700 1 -- 172.21.15.3:0/3378501040 >> v2:172.21.15.3:3300/0 conn(0x7f2f0809eca0 msgr2=0x7f2f0806ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:34.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.246+0000 7f2f0d89a700 1 --2- 172.21.15.3:0/3378501040 >> v2:172.21.15.3:3300/0 conn(0x7f2f0809eca0 0x7f2f0806ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.246+0000 7f2f0d89a700 1 -- 172.21.15.3:0/3378501040 >> v1:172.21.15.3:6789/0 conn(0x7f2f08134610 legacy=0x7f2f08134a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:34.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.246+0000 7f2f0d89a700 1 -- 172.21.15.3:0/3378501040 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f08071b90 con 0x7f2f0806e570 2023-10-03T10:15:34.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.246+0000 7f2f0d099700 1 --2- 172.21.15.3:0/3378501040 >> v2:172.21.15.3:3300/0 conn(0x7f2f0809eca0 0x7f2f0806ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:34.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.246+0000 7f2f0d89a700 1 --2- 172.21.15.3:0/3378501040 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2f0806e570 0x7f2f0806e940 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2f040098f0 tx=0x7f2f04009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d859f0fe9058a8f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:34.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.246+0000 7f2f0c898700 1 -- 172.21.15.3:0/3378501040 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f04011830 con 0x7f2f0806e570 2023-10-03T10:15:34.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.246+0000 7f2f0c898700 1 -- 172.21.15.3:0/3378501040 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f04011990 con 0x7f2f0806e570 2023-10-03T10:15:34.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.246+0000 7f2f0c898700 1 -- 172.21.15.3:0/3378501040 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f04011d20 con 0x7f2f0806e570 2023-10-03T10:15:34.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.247+0000 7f2f0fafe700 1 -- 172.21.15.3:0/3378501040 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2f0806e570 msgr2=0x7f2f0806e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:34.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.247+0000 7f2f0fafe700 1 --2- 172.21.15.3:0/3378501040 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2f0806e570 0x7f2f0806e940 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2f040098f0 tx=0x7f2f04009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:34.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.247+0000 7f2f0fafe700 1 -- 172.21.15.3:0/3378501040 shutdown_connections 2023-10-03T10:15:34.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.247+0000 7f2f0fafe700 1 --2- 172.21.15.3:0/3378501040 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2f0806e570 0x7f2f0806e940 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.247+0000 7f2f0fafe700 1 --2- 172.21.15.3:0/3378501040 >> v2:172.21.15.3:3300/0 conn(0x7f2f0809eca0 0x7f2f0806ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.247+0000 7f2f0fafe700 1 -- 172.21.15.3:0/3378501040 >> 172.21.15.3:0/3378501040 conn(0x7f2f080a2aa0 msgr2=0x7f2f080a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:34.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.250+0000 7f2f0fafe700 1 -- 172.21.15.3:0/3378501040 shutdown_connections 2023-10-03T10:15:34.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.250+0000 7f2f0fafe700 1 -- 172.21.15.3:0/3378501040 wait complete. 2023-10-03T10:15:34.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.250+0000 7f4131ffb700 1 -- 172.21.15.3:0/113693660 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f413c09c790 con 0x7f413c135e50 2023-10-03T10:15:34.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.252+0000 7f2f0fafe700 1 Processor -- start 2023-10-03T10:15:34.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.252+0000 7f2f0fafe700 1 -- start start 2023-10-03T10:15:34.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.252+0000 7f2f0fafe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2f0806e570 0x7f2f08140150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:34.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.252+0000 7f2f0fafe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f08134610 0x7f2f0813e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:34.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.252+0000 7f2f0fafe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2f0813ec50 0x7f2f0813f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:34.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.253+0000 7f2f0fafe700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f081344d0 con 0x7f2f08134610 2023-10-03T10:15:34.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.253+0000 7f2f0fafe700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f080a2170 con 0x7f2f0813ec50 2023-10-03T10:15:34.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.253+0000 7f2f0fafe700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0813cdc0 con 0x7f2f0806e570 2023-10-03T10:15:34.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.253+0000 7f2f0e09b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2f0813ec50 0x7f2f0813f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:34.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.253+0000 7f2f0e09b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2f0813ec50 0x7f2f0813f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50342/0 (socket says 172.21.15.3:50342) 2023-10-03T10:15:34.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.253+0000 7f2f0e09b700 1 -- 172.21.15.3:0/3909735066 learned_addr learned my addr 172.21.15.3:0/3909735066 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:34.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.253+0000 7f2f0d89a700 1 --2- 172.21.15.3:0/3909735066 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2f0806e570 0x7f2f08140150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:34.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.253+0000 7f2f0e09b700 1 -- 172.21.15.3:0/3909735066 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2f0806e570 msgr2=0x7f2f08140150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:34.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.253+0000 7f2f0e09b700 1 --2- 172.21.15.3:0/3909735066 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2f0806e570 0x7f2f08140150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.253+0000 7f2f0e09b700 1 -- 172.21.15.3:0/3909735066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f08134610 msgr2=0x7f2f0813e710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:15:34.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.253+0000 7f2f0e09b700 1 --2- 172.21.15.3:0/3909735066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f08134610 0x7f2f0813e710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.253+0000 7f2f0e09b700 1 -- 172.21.15.3:0/3909735066 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef80094a0 con 0x7f2f0813ec50 2023-10-03T10:15:34.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.253+0000 7f2f0e09b700 1 --2- 172.21.15.3:0/3909735066 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2f0813ec50 0x7f2f0813f0b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000cc50 tx=0x7f2f0000cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:34.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.254+0000 7f2efe7fc700 1 -- 172.21.15.3:0/3909735066 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f00003c00 con 0x7f2f0813ec50 2023-10-03T10:15:34.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.254+0000 7f2efe7fc700 1 -- 172.21.15.3:0/3909735066 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f00003d60 con 0x7f2f0813ec50 2023-10-03T10:15:34.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.254+0000 7f2efe7fc700 1 -- 172.21.15.3:0/3909735066 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0000ab60 con 0x7f2f0813ec50 2023-10-03T10:15:34.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.254+0000 7f2f0fafe700 1 -- 172.21.15.3:0/3909735066 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f04009550 con 0x7f2f0813ec50 2023-10-03T10:15:34.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.254+0000 7f2f0fafe700 1 -- 172.21.15.3:0/3909735066 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f08070610 con 0x7f2f0813ec50 2023-10-03T10:15:34.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.255+0000 7f2efe7fc700 1 -- 172.21.15.3:0/3909735066 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2f0000acc0 con 0x7f2f0813ec50 2023-10-03T10:15:34.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.255+0000 7f2efe7fc700 1 -- 172.21.15.3:0/3909735066 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2f000421c0 con 0x7f2f0813ec50 2023-10-03T10:15:34.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.255+0000 7f2f0fafe700 1 -- 172.21.15.3:0/3909735066 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f0813f700 con 0x7f2f0813ec50 2023-10-03T10:15:34.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.264+0000 7f2efe7fc700 1 -- 172.21.15.3:0/3909735066 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2f00044060 con 0x7f2f0813ec50 2023-10-03T10:15:34.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.424+0000 7f2b73adf700 1 -- 172.21.15.3:0/2978369220 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2b6c1a70f0 con 0x7f2b6c09f240 2023-10-03T10:15:34.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.425+0000 7f2b627fc700 1 -- 172.21.15.3:0/2978369220 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2b5c00f2e0 con 0x7f2b6c09f240 2023-10-03T10:15:34.425 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:34.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.428+0000 7f2b73adf700 1 -- 172.21.15.3:0/2978369220 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b6c09f240 msgr2=0x7f2b6c1a4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:34.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.428+0000 7f2b73adf700 1 --2- 172.21.15.3:0/2978369220 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b6c09f240 0x7f2b6c1a4150 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00b480 tx=0x7f2b5c00b790 comp rx=0 tx=0).stop 2023-10-03T10:15:34.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.428+0000 7f2b73adf700 1 -- 172.21.15.3:0/2978369220 shutdown_connections 2023-10-03T10:15:34.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.428+0000 7f2b73adf700 1 --2- 172.21.15.3:0/2978369220 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b6c09f650 0x7f2b6c1a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.428+0000 7f2b73adf700 1 --2- 172.21.15.3:0/2978369220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b6c09e930 0x7f2b6c1a3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.428+0000 7f2b73adf700 1 --2- 172.21.15.3:0/2978369220 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b6c09f240 0x7f2b6c1a4150 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.428+0000 7f2b73adf700 1 -- 172.21.15.3:0/2978369220 >> 172.21.15.3:0/2978369220 conn(0x7f2b6c0a4210 msgr2=0x7f2b6c1b8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:34.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.428+0000 7f2b73adf700 1 -- 172.21.15.3:0/2978369220 shutdown_connections 2023-10-03T10:15:34.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.428+0000 7f2b73adf700 1 -- 172.21.15.3:0/2978369220 wait complete. 2023-10-03T10:15:34.447 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:34.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.465+0000 7f414379e700 1 -- 172.21.15.3:0/113693660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f413c09c790 con 0x7f413c135e50 2023-10-03T10:15:34.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.466+0000 7f382b720700 1 -- 172.21.15.3:0/2317383319 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f382406c360 con 0x7f382409d6f0 2023-10-03T10:15:34.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.466+0000 7f4131ffb700 1 -- 172.21.15.3:0/113693660 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f413c09c790 con 0x7f413c135e50 2023-10-03T10:15:34.467 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:34.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.466+0000 7f3819ffb700 1 -- 172.21.15.3:0/2317383319 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3820015070 con 0x7f382409d6f0 2023-10-03T10:15:34.467 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:34.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.469+0000 7f414379e700 1 -- 172.21.15.3:0/113693660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f413c135e50 msgr2=0x7f413c133470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:34.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.469+0000 7f414379e700 1 --2- 172.21.15.3:0/113693660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f413c135e50 0x7f413c133470 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f413400ef90 tx=0x7f413400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.469+0000 7f414379e700 1 -- 172.21.15.3:0/113693660 shutdown_connections 2023-10-03T10:15:34.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.469+0000 7f414379e700 1 --2- 172.21.15.3:0/113693660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f413c135e50 0x7f413c133470 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.469+0000 7f414379e700 1 --2- 172.21.15.3:0/113693660 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f413c09d6f0 0x7f413c1329f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.469+0000 7f414379e700 1 --2- 172.21.15.3:0/113693660 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f413c09e1d0 0x7f413c132f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.469+0000 7f414379e700 1 -- 172.21.15.3:0/113693660 >> 172.21.15.3:0/113693660 conn(0x7f413c066f50 msgr2=0x7f413c06e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:34.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.469+0000 7f382b720700 1 -- 172.21.15.3:0/2317383319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f382409d6f0 msgr2=0x7f38241a5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:34.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.469+0000 7f382b720700 1 --2- 172.21.15.3:0/2317383319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f382409d6f0 0x7f38241a5fd0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f3820009870 tx=0x7f3820004750 comp rx=0 tx=0).stop 2023-10-03T10:15:34.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.469+0000 7f414379e700 1 -- 172.21.15.3:0/113693660 shutdown_connections 2023-10-03T10:15:34.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.470+0000 7f414379e700 1 -- 172.21.15.3:0/113693660 wait complete. 2023-10-03T10:15:34.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.470+0000 7f382b720700 1 -- 172.21.15.3:0/2317383319 shutdown_connections 2023-10-03T10:15:34.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.470+0000 7f382b720700 1 --2- 172.21.15.3:0/2317383319 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f382409e410 0x7f38241a6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.470+0000 7f382b720700 1 --2- 172.21.15.3:0/2317383319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f382409d6f0 0x7f38241a5fd0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.470+0000 7f382b720700 1 --2- 172.21.15.3:0/2317383319 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f382409e000 0x7f38241a6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.470+0000 7f382b720700 1 -- 172.21.15.3:0/2317383319 >> 172.21.15.3:0/2317383319 conn(0x7f3824066f50 msgr2=0x7f38241bafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:34.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.470+0000 7f382b720700 1 -- 172.21.15.3:0/2317383319 shutdown_connections 2023-10-03T10:15:34.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.470+0000 7f382b720700 1 -- 172.21.15.3:0/2317383319 wait complete. 2023-10-03T10:15:34.487 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:34.487 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:34.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.494+0000 7f2f0fafe700 1 -- 172.21.15.3:0/3909735066 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2f08074ea0 con 0x7f2f0813ec50 2023-10-03T10:15:34.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.495+0000 7f2efe7fc700 1 -- 172.21.15.3:0/3909735066 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2f000050d0 con 0x7f2f0813ec50 2023-10-03T10:15:34.495 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:34.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.497+0000 7f2f0fafe700 1 -- 172.21.15.3:0/3909735066 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2f0813ec50 msgr2=0x7f2f0813f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:34.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.497+0000 7f2f0fafe700 1 --2- 172.21.15.3:0/3909735066 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2f0813ec50 0x7f2f0813f0b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000cc50 tx=0x7f2f0000cf60 comp rx=0 tx=0).stop 2023-10-03T10:15:34.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.497+0000 7f2f0fafe700 1 -- 172.21.15.3:0/3909735066 shutdown_connections 2023-10-03T10:15:34.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.497+0000 7f2f0fafe700 1 --2- 172.21.15.3:0/3909735066 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2f0813ec50 0x7f2f0813f0b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.497+0000 7f2f0fafe700 1 --2- 172.21.15.3:0/3909735066 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2f0806e570 0x7f2f08140150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.497+0000 7f2f0fafe700 1 --2- 172.21.15.3:0/3909735066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f08134610 0x7f2f0813e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:34.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.497+0000 7f2f0fafe700 1 -- 172.21.15.3:0/3909735066 >> 172.21.15.3:0/3909735066 conn(0x7f2f080a2aa0 msgr2=0x7f2f0809e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:34.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.497+0000 7f2f0fafe700 1 -- 172.21.15.3:0/3909735066 shutdown_connections 2023-10-03T10:15:34.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:34.497+0000 7f2f0fafe700 1 -- 172.21.15.3:0/3909735066 wait complete. 2023-10-03T10:15:34.513 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:35.448 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:35.487 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:35.488 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:35.515 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:35.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.612+0000 7f5044fc0700 1 Processor -- start 2023-10-03T10:15:35.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.612+0000 7f5044fc0700 1 -- start start 2023-10-03T10:15:35.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.613+0000 7f5044fc0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5040133420 0x7f50401337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:35.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.613+0000 7f5044fc0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5040133e30 0x7f504012ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:35.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.613+0000 7f5044fc0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f504006b990 con 0x7f5040134260 2023-10-03T10:15:35.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.613+0000 7f5044fc0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50400a0590 con 0x7f5040133e30 2023-10-03T10:15:35.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.613+0000 7f5044fc0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504009fb80 con 0x7f5040133420 2023-10-03T10:15:35.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.613+0000 7f503dd9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5040133e30 0x7f504012ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:35.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.613+0000 7f503dd9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5040133e30 0x7f504012ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42030/0 (socket says 172.21.15.3:42030) 2023-10-03T10:15:35.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.613+0000 7f503dd9b700 1 -- 172.21.15.3:0/2405383762 learned_addr learned my addr 172.21.15.3:0/2405383762 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:35.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.613+0000 7f503e59c700 1 --2- 172.21.15.3:0/2405383762 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5040133420 0x7f50401337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:35.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.614+0000 7f503d59a700 1 -- 172.21.15.3:0/2405383762 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2583000422 0 0) 0x7f504006b990 con 0x7f5040134260 2023-10-03T10:15:35.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.614+0000 7f503d59a700 1 -- 172.21.15.3:0/2405383762 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50280035b0 con 0x7f5040134260 2023-10-03T10:15:35.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.614+0000 7f503e59c700 1 -- 172.21.15.3:0/2405383762 >> v2:172.21.15.3:3300/0 conn(0x7f5040133e30 msgr2=0x7f504012ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:35.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.614+0000 7f503e59c700 1 --2- 172.21.15.3:0/2405383762 >> v2:172.21.15.3:3300/0 conn(0x7f5040133e30 0x7f504012ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.614+0000 7f503e59c700 1 -- 172.21.15.3:0/2405383762 >> v1:172.21.15.3:6789/0 conn(0x7f5040134260 legacy=0x7f50401304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:35.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.614+0000 7f503e59c700 1 -- 172.21.15.3:0/2405383762 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50400a0030 con 0x7f5040133420 2023-10-03T10:15:35.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.614+0000 7f503dd9b700 1 --2- 172.21.15.3:0/2405383762 >> v2:172.21.15.3:3300/0 conn(0x7f5040133e30 0x7f504012ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:15:35.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.614+0000 7f503e59c700 1 --2- 172.21.15.3:0/2405383762 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5040133420 0x7f50401337f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5030009840 tx=0x7f5030009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bdc4c49c8b8499f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:35.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.614+0000 7f503d59a700 1 -- 172.21.15.3:0/2405383762 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5030011830 con 0x7f5040133420 2023-10-03T10:15:35.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.614+0000 7f503d59a700 1 -- 172.21.15.3:0/2405383762 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5030011990 con 0x7f5040133420 2023-10-03T10:15:35.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.614+0000 7f503d59a700 1 -- 172.21.15.3:0/2405383762 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5030011d20 con 0x7f5040133420 2023-10-03T10:15:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.615+0000 7f5044fc0700 1 -- 172.21.15.3:0/2405383762 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5040133420 msgr2=0x7f50401337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.615+0000 7f5044fc0700 1 --2- 172.21.15.3:0/2405383762 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5040133420 0x7f50401337f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5030009840 tx=0x7f5030009b50 comp rx=0 tx=0).stop 2023-10-03T10:15:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.616+0000 7f5044fc0700 1 -- 172.21.15.3:0/2405383762 shutdown_connections 2023-10-03T10:15:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.616+0000 7f5044fc0700 1 --2- 172.21.15.3:0/2405383762 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5040133420 0x7f50401337f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.616+0000 7f5044fc0700 1 --2- 172.21.15.3:0/2405383762 >> v2:172.21.15.3:3300/0 conn(0x7f5040133e30 0x7f504012ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.616+0000 7f5044fc0700 1 -- 172.21.15.3:0/2405383762 >> 172.21.15.3:0/2405383762 conn(0x7f50400a4210 msgr2=0x7f50400a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.616+0000 7f5044fc0700 1 -- 172.21.15.3:0/2405383762 shutdown_connections 2023-10-03T10:15:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.616+0000 7f5044fc0700 1 -- 172.21.15.3:0/2405383762 wait complete. 2023-10-03T10:15:35.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.617+0000 7f5044fc0700 1 Processor -- start 2023-10-03T10:15:35.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.617+0000 7f5044fc0700 1 -- start start 2023-10-03T10:15:35.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.618+0000 7f5044fc0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5040133420 0x7f504009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:35.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.618+0000 7f5044fc0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5040133e30 0x7f504009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:35.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.618+0000 7f5044fc0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5040134260 0x7f504009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:35.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.618+0000 7f5044fc0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504011f320 con 0x7f5040133e30 2023-10-03T10:15:35.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.618+0000 7f5044fc0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504011f770 con 0x7f5040133420 2023-10-03T10:15:35.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.618+0000 7f5044fc0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f504011fd00 con 0x7f5040134260 2023-10-03T10:15:35.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.618+0000 7f503e59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5040133420 0x7f504009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:35.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.618+0000 7f503e59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5040133420 0x7f504009c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50352/0 (socket says 172.21.15.3:50352) 2023-10-03T10:15:35.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.618+0000 7f503e59c700 1 -- 172.21.15.3:0/2527701165 learned_addr learned my addr 172.21.15.3:0/2527701165 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:35.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.618+0000 7f503dd9b700 1 --2- 172.21.15.3:0/2527701165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5040133e30 0x7f504009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:35.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.619+0000 7f503e59c700 1 -- 172.21.15.3:0/2527701165 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5040134260 msgr2=0x7f504009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:35.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.619+0000 7f503e59c700 1 --2- 172.21.15.3:0/2527701165 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5040134260 0x7f504009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.619+0000 7f503e59c700 1 -- 172.21.15.3:0/2527701165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5040133e30 msgr2=0x7f504009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:35.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.619+0000 7f503e59c700 1 --2- 172.21.15.3:0/2527701165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5040133e30 0x7f504009c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.619+0000 7f503e59c700 1 -- 172.21.15.3:0/2527701165 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50300094a0 con 0x7f5040133420 2023-10-03T10:15:35.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.619+0000 7f503dd9b700 1 --2- 172.21.15.3:0/2527701165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5040133e30 0x7f504009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:35.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.619+0000 7f503e59c700 1 --2- 172.21.15.3:0/2527701165 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5040133420 0x7f504009c480 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5030009fd0 tx=0x7f5030004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:35.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.619+0000 7f502effd700 1 -- 172.21.15.3:0/2527701165 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5030004480 con 0x7f5040133420 2023-10-03T10:15:35.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.619+0000 7f502effd700 1 -- 172.21.15.3:0/2527701165 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503001f070 con 0x7f5040133420 2023-10-03T10:15:35.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.619+0000 7f5044fc0700 1 -- 172.21.15.3:0/2527701165 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f504011ea90 con 0x7f5040133420 2023-10-03T10:15:35.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.619+0000 7f502effd700 1 -- 172.21.15.3:0/2527701165 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50300038e0 con 0x7f5040133420 2023-10-03T10:15:35.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.621+0000 7f502effd700 1 -- 172.21.15.3:0/2527701165 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5030021030 con 0x7f5040133420 2023-10-03T10:15:35.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.621+0000 7f5044fc0700 1 -- 172.21.15.3:0/2527701165 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f504012d890 con 0x7f5040133420 2023-10-03T10:15:35.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.621+0000 7f502effd700 1 -- 172.21.15.3:0/2527701165 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f50300453b0 con 0x7f5040133420 2023-10-03T10:15:35.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.622+0000 7f5044fc0700 1 -- 172.21.15.3:0/2527701165 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5040074ea0 con 0x7f5040133420 2023-10-03T10:15:35.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.627+0000 7f502effd700 1 -- 172.21.15.3:0/2527701165 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f503001a7d0 con 0x7f5040133420 2023-10-03T10:15:35.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.647+0000 7fdc14822700 1 Processor -- start 2023-10-03T10:15:35.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.647+0000 7fdc14822700 1 -- start start 2023-10-03T10:15:35.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.648+0000 7fdc14822700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc10131810 0x7fdc10131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:35.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.648+0000 7fdc14822700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdc10099a40 0x7fdc10099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:35.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.648+0000 7fdc14822700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc1009a850 con 0x7fdc1009a3b0 2023-10-03T10:15:35.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.648+0000 7fdc14822700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1009b580 con 0x7fdc10099a40 2023-10-03T10:15:35.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.648+0000 7fdc14822700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc101313c0 con 0x7fdc10131810 2023-10-03T10:15:35.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.648+0000 7fdc0e59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc10131810 0x7fdc10131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:35.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.648+0000 7fdc0e59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc10131810 0x7fdc10131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50368/0 (socket says 172.21.15.3:50368) 2023-10-03T10:15:35.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.648+0000 7fdc0e59c700 1 -- 172.21.15.3:0/2638624833 learned_addr learned my addr 172.21.15.3:0/2638624833 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:35.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.648+0000 7fdc0dd9b700 1 --2- 172.21.15.3:0/2638624833 >> v2:172.21.15.3:3300/0 conn(0x7fdc10099a40 0x7fdc10099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:35.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.649+0000 7fdc0e59c700 1 -- 172.21.15.3:0/2638624833 >> v2:172.21.15.3:3300/0 conn(0x7fdc10099a40 msgr2=0x7fdc10099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:35.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.649+0000 7fdc0e59c700 1 --2- 172.21.15.3:0/2638624833 >> v2:172.21.15.3:3300/0 conn(0x7fdc10099a40 0x7fdc10099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.649+0000 7fdc0e59c700 1 -- 172.21.15.3:0/2638624833 >> v1:172.21.15.3:6789/0 conn(0x7fdc1009a3b0 legacy=0x7fdc10138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:35.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.649+0000 7fdc0e59c700 1 -- 172.21.15.3:0/2638624833 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1006e8c0 con 0x7fdc10131810 2023-10-03T10:15:35.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.649+0000 7fdc0e59c700 1 --2- 172.21.15.3:0/2638624833 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc10131810 0x7fdc10131be0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdc00009920 tx=0x7fdc00009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cbc5ac1d0e539dc server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:35.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.649+0000 7fdc0d59a700 1 -- 172.21.15.3:0/2638624833 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc00011830 con 0x7fdc10131810 2023-10-03T10:15:35.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.649+0000 7fdc0d59a700 1 -- 172.21.15.3:0/2638624833 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc00011990 con 0x7fdc10131810 2023-10-03T10:15:35.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.649+0000 7fdc0d59a700 1 -- 172.21.15.3:0/2638624833 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc00011d20 con 0x7fdc10131810 2023-10-03T10:15:35.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.650+0000 7fdc14822700 1 -- 172.21.15.3:0/2638624833 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc10131810 msgr2=0x7fdc10131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:35.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.650+0000 7fdc14822700 1 --2- 172.21.15.3:0/2638624833 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc10131810 0x7fdc10131be0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdc00009920 tx=0x7fdc00009c30 comp rx=0 tx=0).stop 2023-10-03T10:15:35.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.650+0000 7fdc14822700 1 -- 172.21.15.3:0/2638624833 shutdown_connections 2023-10-03T10:15:35.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.650+0000 7fdc14822700 1 --2- 172.21.15.3:0/2638624833 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc10131810 0x7fdc10131be0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.650+0000 7fdc14822700 1 --2- 172.21.15.3:0/2638624833 >> v2:172.21.15.3:3300/0 conn(0x7fdc10099a40 0x7fdc10099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.650+0000 7fdc14822700 1 -- 172.21.15.3:0/2638624833 >> 172.21.15.3:0/2638624833 conn(0x7fdc100a2aa0 msgr2=0x7fdc100a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:35.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.651+0000 7fdc14822700 1 -- 172.21.15.3:0/2638624833 shutdown_connections 2023-10-03T10:15:35.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.651+0000 7fdc14822700 1 -- 172.21.15.3:0/2638624833 wait complete. 2023-10-03T10:15:35.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.652+0000 7f25284ba700 1 Processor -- start 2023-10-03T10:15:35.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.652+0000 7fdc14822700 1 Processor -- start 2023-10-03T10:15:35.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.652+0000 7fdc14822700 1 -- start start 2023-10-03T10:15:35.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.652+0000 7f25284ba700 1 -- start start 2023-10-03T10:15:35.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.652+0000 7f25284ba700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f252009eca0 0x7f252013cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:35.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.652+0000 7f25284ba700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f252006f050 0x7f252013f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:35.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.652+0000 7f25284ba700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2520070ee0 con 0x7f252006e570 2023-10-03T10:15:35.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.652+0000 7f25284ba700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2520071980 con 0x7f252006f050 2023-10-03T10:15:35.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.652+0000 7f25284ba700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2520071480 con 0x7f252009eca0 2023-10-03T10:15:35.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.653+0000 7f2526256700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f252006e570 0x7f252006e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34348/0 (socket says 172.21.15.3:34348) 2023-10-03T10:15:35.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.653+0000 7f2526256700 1 -- 172.21.15.3:0/1884127680 learned_addr learned my addr 172.21.15.3:0/1884127680 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:35.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.653+0000 7f2525a55700 1 --2- 172.21.15.3:0/1884127680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f252009eca0 0x7f252013cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:35.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.653+0000 7f2525254700 1 -- 172.21.15.3:0/1884127680 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3069519309 0 0) 0x7f2520070ee0 con 0x7f252006e570 2023-10-03T10:15:35.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.653+0000 7f2525254700 1 -- 172.21.15.3:0/1884127680 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f250c0035b0 con 0x7f252006e570 2023-10-03T10:15:35.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.653+0000 7fdc14822700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc10099a40 0x7fdc101197b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:35.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.653+0000 7fdc14822700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc1009a3b0 0x7fdc10119cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:35.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.653+0000 7fdc14822700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc10131810 0x7fdc1011a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:35.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.653+0000 7fdc14822700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1012cba0 con 0x7fdc10131810 2023-10-03T10:15:35.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.653+0000 7fdc14822700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1011ac10 con 0x7fdc1009a3b0 2023-10-03T10:15:35.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.653+0000 7fdc14822700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1011c610 con 0x7fdc10099a40 2023-10-03T10:15:35.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.653+0000 7fdc0e59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc10099a40 0x7fdc101197b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:35.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.653+0000 7f2526a57700 1 --2- 172.21.15.3:0/1884127680 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f252006f050 0x7f252013f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:35.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.653+0000 7fdc0e59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc10099a40 0x7fdc101197b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41830/0 (socket says 172.21.15.3:41830) 2023-10-03T10:15:35.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.653+0000 7fdc0e59c700 1 -- 172.21.15.3:0/1148318615 learned_addr learned my addr 172.21.15.3:0/1148318615 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:35.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.653+0000 7f2525a55700 1 -- 172.21.15.3:0/1884127680 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f252006f050 msgr2=0x7f252013f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:35.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.654+0000 7f2525a55700 1 --2- 172.21.15.3:0/1884127680 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f252006f050 0x7f252013f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.654+0000 7f2525a55700 1 -- 172.21.15.3:0/1884127680 >> v1:172.21.15.3:6789/0 conn(0x7f252006e570 legacy=0x7f252006e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:35.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.654+0000 7f2525a55700 1 -- 172.21.15.3:0/1884127680 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2520071b90 con 0x7f252009eca0 2023-10-03T10:15:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.654+0000 7fdc0e59c700 1 -- 172.21.15.3:0/1148318615 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc1009a3b0 msgr2=0x7fdc10119cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.654+0000 7fdc0e59c700 1 --2- 172.21.15.3:0/1148318615 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc1009a3b0 0x7fdc10119cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.654+0000 7fdc0e59c700 1 -- 172.21.15.3:0/1148318615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc10131810 msgr2=0x7fdc1011a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.654+0000 7fdc0e59c700 1 --2- 172.21.15.3:0/1148318615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc10131810 0x7fdc1011a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.654+0000 7fdc0e59c700 1 -- 172.21.15.3:0/1148318615 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc00009550 con 0x7fdc10099a40 2023-10-03T10:15:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.654+0000 7fdc0e59c700 1 --2- 172.21.15.3:0/1148318615 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc10099a40 0x7fdc101197b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdc00009c70 tx=0x7fdc00004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:35.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.654+0000 7f2525a55700 1 --2- 172.21.15.3:0/1884127680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f252009eca0 0x7f252013cea0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2510009870 tx=0x7f2510009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=308863a3526a2cd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:35.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.654+0000 7fdbfeffd700 1 -- 172.21.15.3:0/1148318615 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc00004400 con 0x7fdc10099a40 2023-10-03T10:15:35.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.654+0000 7fdbfeffd700 1 -- 172.21.15.3:0/1148318615 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0001f070 con 0x7fdc10099a40 2023-10-03T10:15:35.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.654+0000 7fdbfeffd700 1 -- 172.21.15.3:0/1148318615 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc000038e0 con 0x7fdc10099a40 2023-10-03T10:15:35.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.654+0000 7f2525254700 1 -- 172.21.15.3:0/1884127680 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2510011830 con 0x7f252009eca0 2023-10-03T10:15:35.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.654+0000 7f2525254700 1 -- 172.21.15.3:0/1884127680 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2510011990 con 0x7f252009eca0 2023-10-03T10:15:35.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.654+0000 7f2525254700 1 -- 172.21.15.3:0/1884127680 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2510011d20 con 0x7f252009eca0 2023-10-03T10:15:35.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.655+0000 7f25284ba700 1 -- 172.21.15.3:0/1884127680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f252009eca0 msgr2=0x7f252013cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:35.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.655+0000 7f25284ba700 1 --2- 172.21.15.3:0/1884127680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f252009eca0 0x7f252013cea0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2510009870 tx=0x7f2510009b80 comp rx=0 tx=0).stop 2023-10-03T10:15:35.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.655+0000 7fdc14822700 1 -- 172.21.15.3:0/1148318615 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1011b9a0 con 0x7fdc10099a40 2023-10-03T10:15:35.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.655+0000 7f25284ba700 1 -- 172.21.15.3:0/1884127680 shutdown_connections 2023-10-03T10:15:35.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.655+0000 7f25284ba700 1 --2- 172.21.15.3:0/1884127680 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f252006f050 0x7f252013f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.655+0000 7f25284ba700 1 --2- 172.21.15.3:0/1884127680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f252009eca0 0x7f252013cea0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.655+0000 7f25284ba700 1 -- 172.21.15.3:0/1884127680 >> 172.21.15.3:0/1884127680 conn(0x7f25200a2aa0 msgr2=0x7f25200a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:35.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.655+0000 7f25284ba700 1 -- 172.21.15.3:0/1884127680 shutdown_connections 2023-10-03T10:15:35.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.655+0000 7f25284ba700 1 -- 172.21.15.3:0/1884127680 wait complete. 2023-10-03T10:15:35.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.655+0000 7fdc14822700 1 -- 172.21.15.3:0/1148318615 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc100715f0 con 0x7fdc10099a40 2023-10-03T10:15:35.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.656+0000 7f25284ba700 1 Processor -- start 2023-10-03T10:15:35.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.657+0000 7f25284ba700 1 -- start start 2023-10-03T10:15:35.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.657+0000 7fdc14822700 1 -- 172.21.15.3:0/1148318615 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc100a1c60 con 0x7fdc10099a40 2023-10-03T10:15:35.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.657+0000 7f25284ba700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f252006e570 0x7f25201371e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:35.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.657+0000 7f25284ba700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f252006f050 0x7f2520137720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:35.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.657+0000 7fdbfeffd700 1 -- 172.21.15.3:0/1148318615 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdc00021030 con 0x7fdc10099a40 2023-10-03T10:15:35.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.657+0000 7f25284ba700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2520137c60 0x7f25201308a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:35.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.657+0000 7f25284ba700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2520118e50 con 0x7f252006f050 2023-10-03T10:15:35.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.657+0000 7f25284ba700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f252013ecd0 con 0x7f2520137c60 2023-10-03T10:15:35.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.657+0000 7f25284ba700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f252011a5d0 con 0x7f252006e570 2023-10-03T10:15:35.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.657+0000 7f2526a57700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2520137c60 0x7f25201308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:35.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.657+0000 7f2526a57700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2520137c60 0x7f25201308a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50404/0 (socket says 172.21.15.3:50404) 2023-10-03T10:15:35.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.657+0000 7f2526256700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f252006e570 0x7f25201371e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:35.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.657+0000 7f2526a57700 1 -- 172.21.15.3:0/2511156857 learned_addr learned my addr 172.21.15.3:0/2511156857 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:35.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.658+0000 7f2525a55700 1 --2- 172.21.15.3:0/2511156857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f252006f050 0x7f2520137720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:35.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.658+0000 7f2526256700 1 -- 172.21.15.3:0/2511156857 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2520137c60 msgr2=0x7f25201308a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:35.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.658+0000 7f2526256700 1 --2- 172.21.15.3:0/2511156857 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2520137c60 0x7f25201308a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.658+0000 7f2526256700 1 -- 172.21.15.3:0/2511156857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f252006f050 msgr2=0x7f2520137720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:35.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.658+0000 7fdbfeffd700 1 -- 172.21.15.3:0/1148318615 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fdc00040a90 con 0x7fdc10099a40 2023-10-03T10:15:35.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.658+0000 7f2526256700 1 --2- 172.21.15.3:0/2511156857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f252006f050 0x7f2520137720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.658+0000 7f2526256700 1 -- 172.21.15.3:0/2511156857 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25100094a0 con 0x7f252006e570 2023-10-03T10:15:35.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.658+0000 7f2525a55700 1 --2- 172.21.15.3:0/2511156857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f252006f050 0x7f2520137720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:15:35.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.658+0000 7f2526256700 1 --2- 172.21.15.3:0/2511156857 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f252006e570 0x7f25201371e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f251c00c4c0 tx=0x7f251c00c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:35.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.667+0000 7f2516ffd700 1 -- 172.21.15.3:0/2511156857 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c005d70 con 0x7f252006e570 2023-10-03T10:15:35.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.667+0000 7f2516ffd700 1 -- 172.21.15.3:0/2511156857 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f251c0070d0 con 0x7f252006e570 2023-10-03T10:15:35.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.667+0000 7f25284ba700 1 -- 172.21.15.3:0/2511156857 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2520131940 con 0x7f252006e570 2023-10-03T10:15:35.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.667+0000 7f25284ba700 1 -- 172.21.15.3:0/2511156857 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2520070610 con 0x7f252006e570 2023-10-03T10:15:35.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.667+0000 7f2516ffd700 1 -- 172.21.15.3:0/2511156857 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c007440 con 0x7f252006e570 2023-10-03T10:15:35.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.667+0000 7fdbfeffd700 1 -- 172.21.15.3:0/1148318615 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdc000199a0 con 0x7fdc10099a40 2023-10-03T10:15:35.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.668+0000 7f2516ffd700 1 -- 172.21.15.3:0/2511156857 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f251c0075a0 con 0x7f252006e570 2023-10-03T10:15:35.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.668+0000 7f25284ba700 1 -- 172.21.15.3:0/2511156857 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25080052e0 con 0x7f252006e570 2023-10-03T10:15:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.668+0000 7f2516ffd700 1 -- 172.21.15.3:0/2511156857 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f251c03ebb0 con 0x7f252006e570 2023-10-03T10:15:35.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.673+0000 7f2516ffd700 1 -- 172.21.15.3:0/2511156857 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f251c019b40 con 0x7f252006e570 2023-10-03T10:15:35.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.694+0000 7f39fe121700 1 Processor -- start 2023-10-03T10:15:35.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.694+0000 7f39fe121700 1 -- start start 2023-10-03T10:15:35.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.695+0000 7f39fe121700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f39f81337f0 0x7f39f8138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:35.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.695+0000 7f39fe121700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f39f8138dc0 0x7f39f813b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:35.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.695+0000 7f39fe121700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f806b990 con 0x7f39f8133420 2023-10-03T10:15:35.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.695+0000 7f39fe121700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39f809b0e0 con 0x7f39f81337f0 2023-10-03T10:15:35.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.695+0000 7f39fe121700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f39f8133c20 con 0x7f39f8138dc0 2023-10-03T10:15:35.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.695+0000 7f39f6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f39f81337f0 0x7f39f8138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:35.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.695+0000 7f39f7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f39f8138dc0 0x7f39f813b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:35.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.696+0000 7f39f6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f39f81337f0 0x7f39f8138880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42082/0 (socket says 172.21.15.3:42082) 2023-10-03T10:15:35.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.696+0000 7f39f7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f39f8138dc0 0x7f39f813b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41838/0 (socket says 172.21.15.3:41838) 2023-10-03T10:15:35.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.696+0000 7f39f6ffd700 1 -- 172.21.15.3:0/3209572331 learned_addr learned my addr 172.21.15.3:0/3209572331 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:35.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.696+0000 7f39f7fff700 1 -- 172.21.15.3:0/3209572331 >> v2:172.21.15.3:3300/0 conn(0x7f39f81337f0 msgr2=0x7f39f8138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:35.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.696+0000 7f39f7fff700 1 --2- 172.21.15.3:0/3209572331 >> v2:172.21.15.3:3300/0 conn(0x7f39f81337f0 0x7f39f8138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.696+0000 7f39f7fff700 1 -- 172.21.15.3:0/3209572331 >> v1:172.21.15.3:6789/0 conn(0x7f39f8133420 legacy=0x7f39f8133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:35.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.696+0000 7f39f7fff700 1 -- 172.21.15.3:0/3209572331 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f80a0590 con 0x7f39f8138dc0 2023-10-03T10:15:35.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.696+0000 7f39f6ffd700 1 --2- 172.21.15.3:0/3209572331 >> v2:172.21.15.3:3300/0 conn(0x7f39f81337f0 0x7f39f8138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:35.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.696+0000 7f39f7fff700 1 --2- 172.21.15.3:0/3209572331 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f39f8138dc0 0x7f39f813b1a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f39f0009d90 tx=0x7f39f000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ff50d3fa2a0041f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:35.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.696+0000 7f39f67fc700 1 -- 172.21.15.3:0/3209572331 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f000e070 con 0x7f39f8138dc0 2023-10-03T10:15:35.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.696+0000 7f39f67fc700 1 -- 172.21.15.3:0/3209572331 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f000bb20 con 0x7f39f8138dc0 2023-10-03T10:15:35.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.696+0000 7f39f67fc700 1 -- 172.21.15.3:0/3209572331 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f0012430 con 0x7f39f8138dc0 2023-10-03T10:15:35.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.697+0000 7f39fe121700 1 -- 172.21.15.3:0/3209572331 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f39f8138dc0 msgr2=0x7f39f813b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:35.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.697+0000 7f39fe121700 1 --2- 172.21.15.3:0/3209572331 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f39f8138dc0 0x7f39f813b1a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f39f0009d90 tx=0x7f39f000b330 comp rx=0 tx=0).stop 2023-10-03T10:15:35.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.697+0000 7f39fe121700 1 -- 172.21.15.3:0/3209572331 shutdown_connections 2023-10-03T10:15:35.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.697+0000 7f39fe121700 1 --2- 172.21.15.3:0/3209572331 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f39f8138dc0 0x7f39f813b1a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.697+0000 7f39fe121700 1 --2- 172.21.15.3:0/3209572331 >> v2:172.21.15.3:3300/0 conn(0x7f39f81337f0 0x7f39f8138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.697+0000 7f39fe121700 1 -- 172.21.15.3:0/3209572331 >> 172.21.15.3:0/3209572331 conn(0x7f39f80a4210 msgr2=0x7f39f80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:35.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.697+0000 7f39fe121700 1 -- 172.21.15.3:0/3209572331 shutdown_connections 2023-10-03T10:15:35.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.697+0000 7f39fe121700 1 -- 172.21.15.3:0/3209572331 wait complete. 2023-10-03T10:15:35.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.699+0000 7f39fe121700 1 Processor -- start 2023-10-03T10:15:35.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.699+0000 7f39fe121700 1 -- start start 2023-10-03T10:15:35.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.700+0000 7f39fe121700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f39f8133420 0x7f39f809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:35.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.700+0000 7f39fe121700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f39f81337f0 0x7f39f809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:35.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.700+0000 7f39fe121700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39f8138dc0 0x7f39f809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:35.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.700+0000 7f39fe121700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f81388a0 con 0x7f39f8138dc0 2023-10-03T10:15:35.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.700+0000 7f39fe121700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f812eff0 con 0x7f39f81337f0 2023-10-03T10:15:35.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.700+0000 7f39fe121700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f39f8130490 con 0x7f39f8133420 2023-10-03T10:15:35.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.700+0000 7f39f77fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f39f8133420 0x7f39f809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:35.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.700+0000 7f39f77fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f39f8133420 0x7f39f809c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41846/0 (socket says 172.21.15.3:41846) 2023-10-03T10:15:35.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.700+0000 7f39f77fe700 1 -- 172.21.15.3:0/4286102271 learned_addr learned my addr 172.21.15.3:0/4286102271 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:35.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.700+0000 7f39f6ffd700 1 --2- 172.21.15.3:0/4286102271 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f39f81337f0 0x7f39f809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:35.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.700+0000 7f39f7fff700 1 --2- 172.21.15.3:0/4286102271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39f8138dc0 0x7f39f809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:35.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.700+0000 7f39f77fe700 1 -- 172.21.15.3:0/4286102271 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f39f81337f0 msgr2=0x7f39f809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:35.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.700+0000 7f39f77fe700 1 --2- 172.21.15.3:0/4286102271 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f39f81337f0 0x7f39f809c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.700+0000 7f39f77fe700 1 -- 172.21.15.3:0/4286102271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39f8138dc0 msgr2=0x7f39f809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:35.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.700+0000 7f39f77fe700 1 --2- 172.21.15.3:0/4286102271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39f8138dc0 0x7f39f809cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.700+0000 7f39f77fe700 1 -- 172.21.15.3:0/4286102271 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ec009550 con 0x7f39f8133420 2023-10-03T10:15:35.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.701+0000 7f39f77fe700 1 --2- 172.21.15.3:0/4286102271 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f39f8133420 0x7f39f809c480 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f39e800c4e0 tx=0x7f39e800c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:35.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.701+0000 7f39dffff700 1 -- 172.21.15.3:0/4286102271 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e80052d0 con 0x7f39f8133420 2023-10-03T10:15:35.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.701+0000 7f39dffff700 1 -- 172.21.15.3:0/4286102271 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e8005430 con 0x7f39f8133420 2023-10-03T10:15:35.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.701+0000 7f39fe121700 1 -- 172.21.15.3:0/4286102271 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f0009a70 con 0x7f39f8133420 2023-10-03T10:15:35.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.701+0000 7f39dffff700 1 -- 172.21.15.3:0/4286102271 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e80038c0 con 0x7f39f8133420 2023-10-03T10:15:35.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.701+0000 7f39fe121700 1 -- 172.21.15.3:0/4286102271 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f81309b0 con 0x7f39f8133420 2023-10-03T10:15:35.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.702+0000 7f39dffff700 1 -- 172.21.15.3:0/4286102271 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f39e8003a20 con 0x7f39f8133420 2023-10-03T10:15:35.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.702+0000 7f39dffff700 1 -- 172.21.15.3:0/4286102271 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f39e803eac0 con 0x7f39f8133420 2023-10-03T10:15:35.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.703+0000 7f39fe121700 1 -- 172.21.15.3:0/4286102271 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f8074ea0 con 0x7f39f8133420 2023-10-03T10:15:35.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.708+0000 7f39dffff700 1 -- 172.21.15.3:0/4286102271 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f39e803ce60 con 0x7f39f8133420 2023-10-03T10:15:35.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.881+0000 7f5044fc0700 1 -- 172.21.15.3:0/2527701165 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f504011ff10 con 0x7f5040133420 2023-10-03T10:15:35.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.881+0000 7f502effd700 1 -- 172.21.15.3:0/2527701165 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5030032090 con 0x7f5040133420 2023-10-03T10:15:35.882 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:35.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.884+0000 7f5044fc0700 1 -- 172.21.15.3:0/2527701165 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5040133420 msgr2=0x7f504009c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:35.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.884+0000 7f5044fc0700 1 --2- 172.21.15.3:0/2527701165 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5040133420 0x7f504009c480 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5030009fd0 tx=0x7f5030004750 comp rx=0 tx=0).stop 2023-10-03T10:15:35.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.884+0000 7f5044fc0700 1 -- 172.21.15.3:0/2527701165 shutdown_connections 2023-10-03T10:15:35.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.884+0000 7f5044fc0700 1 --2- 172.21.15.3:0/2527701165 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5040134260 0x7f504009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.884+0000 7f5044fc0700 1 --2- 172.21.15.3:0/2527701165 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5040133420 0x7f504009c480 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.884+0000 7f5044fc0700 1 --2- 172.21.15.3:0/2527701165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5040133e30 0x7f504009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.884+0000 7f5044fc0700 1 -- 172.21.15.3:0/2527701165 >> 172.21.15.3:0/2527701165 conn(0x7f50400a4210 msgr2=0x7f50400a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:35.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.884+0000 7f5044fc0700 1 -- 172.21.15.3:0/2527701165 shutdown_connections 2023-10-03T10:15:35.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.884+0000 7f5044fc0700 1 -- 172.21.15.3:0/2527701165 wait complete. 2023-10-03T10:15:35.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.887+0000 7f25284ba700 1 -- 172.21.15.3:0/2511156857 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2508005590 con 0x7f252006e570 2023-10-03T10:15:35.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.887+0000 7f2516ffd700 1 -- 172.21.15.3:0/2511156857 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f251c017070 con 0x7f252006e570 2023-10-03T10:15:35.887 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:35.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.890+0000 7f25284ba700 1 -- 172.21.15.3:0/2511156857 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f252006e570 msgr2=0x7f25201371e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:35.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.890+0000 7f25284ba700 1 --2- 172.21.15.3:0/2511156857 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f252006e570 0x7f25201371e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f251c00c4c0 tx=0x7f251c00c7d0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.892+0000 7f25284ba700 1 -- 172.21.15.3:0/2511156857 shutdown_connections 2023-10-03T10:15:35.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.892+0000 7f25284ba700 1 --2- 172.21.15.3:0/2511156857 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2520137c60 0x7f25201308a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.892+0000 7f25284ba700 1 --2- 172.21.15.3:0/2511156857 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f252006e570 0x7f25201371e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.892+0000 7f25284ba700 1 --2- 172.21.15.3:0/2511156857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f252006f050 0x7f2520137720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.892+0000 7f25284ba700 1 -- 172.21.15.3:0/2511156857 >> 172.21.15.3:0/2511156857 conn(0x7f25200a2aa0 msgr2=0x7f252009ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:35.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.892+0000 7f25284ba700 1 -- 172.21.15.3:0/2511156857 shutdown_connections 2023-10-03T10:15:35.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.892+0000 7f25284ba700 1 -- 172.21.15.3:0/2511156857 wait complete. 2023-10-03T10:15:35.899 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:35.911 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:35.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.913+0000 7fdc14822700 1 -- 172.21.15.3:0/1148318615 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdc1006eb00 con 0x7fdc10099a40 2023-10-03T10:15:35.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.913+0000 7fdbfeffd700 1 -- 172.21.15.3:0/1148318615 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdc00015030 con 0x7fdc10099a40 2023-10-03T10:15:35.914 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:35.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.917+0000 7fdc14822700 1 -- 172.21.15.3:0/1148318615 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc10099a40 msgr2=0x7fdc101197b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:35.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.917+0000 7fdc14822700 1 --2- 172.21.15.3:0/1148318615 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc10099a40 0x7fdc101197b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdc00009c70 tx=0x7fdc00004750 comp rx=0 tx=0).stop 2023-10-03T10:15:35.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.917+0000 7fdc14822700 1 -- 172.21.15.3:0/1148318615 shutdown_connections 2023-10-03T10:15:35.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.917+0000 7fdc14822700 1 --2- 172.21.15.3:0/1148318615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc10131810 0x7fdc1011a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.917+0000 7fdc14822700 1 --2- 172.21.15.3:0/1148318615 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc10099a40 0x7fdc101197b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.917+0000 7fdc14822700 1 --2- 172.21.15.3:0/1148318615 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc1009a3b0 0x7fdc10119cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.917+0000 7fdc14822700 1 -- 172.21.15.3:0/1148318615 >> 172.21.15.3:0/1148318615 conn(0x7fdc100a2aa0 msgr2=0x7fdc1012e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:35.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.917+0000 7fdc14822700 1 -- 172.21.15.3:0/1148318615 shutdown_connections 2023-10-03T10:15:35.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.917+0000 7fdc14822700 1 -- 172.21.15.3:0/1148318615 wait complete. 2023-10-03T10:15:35.935 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:35.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.936+0000 7f39fe121700 1 -- 172.21.15.3:0/4286102271 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f39f806b250 con 0x7f39f8133420 2023-10-03T10:15:35.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.936+0000 7f39dffff700 1 -- 172.21.15.3:0/4286102271 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f39e8017020 con 0x7f39f8133420 2023-10-03T10:15:35.936 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:35.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.938+0000 7f39fe121700 1 -- 172.21.15.3:0/4286102271 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f39f8133420 msgr2=0x7f39f809c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:35.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.938+0000 7f39fe121700 1 --2- 172.21.15.3:0/4286102271 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f39f8133420 0x7f39f809c480 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f39e800c4e0 tx=0x7f39e800c7f0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.938+0000 7f39fe121700 1 -- 172.21.15.3:0/4286102271 shutdown_connections 2023-10-03T10:15:35.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.938+0000 7f39fe121700 1 --2- 172.21.15.3:0/4286102271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39f8138dc0 0x7f39f809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.938+0000 7f39fe121700 1 --2- 172.21.15.3:0/4286102271 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f39f8133420 0x7f39f809c480 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.938+0000 7f39fe121700 1 --2- 172.21.15.3:0/4286102271 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f39f81337f0 0x7f39f809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:35.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.938+0000 7f39fe121700 1 -- 172.21.15.3:0/4286102271 >> 172.21.15.3:0/4286102271 conn(0x7f39f80a4210 msgr2=0x7f39f80a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:35.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.938+0000 7f39fe121700 1 -- 172.21.15.3:0/4286102271 shutdown_connections 2023-10-03T10:15:35.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:35.938+0000 7f39fe121700 1 -- 172.21.15.3:0/4286102271 wait complete. 2023-10-03T10:15:35.952 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:36.900 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:36.912 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:36.935 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:36.953 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:37.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.056+0000 7f318d82c700 1 Processor -- start 2023-10-03T10:15:37.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.056+0000 7f318d82c700 1 -- start start 2023-10-03T10:15:37.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.057+0000 7f318d82c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f318809e1d0 0x7f3188138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:37.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.057+0000 7f318d82c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3188139170 0x7f318813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:37.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.057+0000 7f318d82c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f318809b120 con 0x7f318809d6f0 2023-10-03T10:15:37.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.057+0000 7f318d82c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f318806b460 con 0x7f318809e1d0 2023-10-03T10:15:37.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.057+0000 7f318d82c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f318809a7e0 con 0x7f3188139170 2023-10-03T10:15:37.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.057+0000 7f31877fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3188139170 0x7f318813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:37.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.057+0000 7f31867fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f318809e1d0 0x7f3188138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:37.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.057+0000 7f31877fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3188139170 0x7f318813b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41860/0 (socket says 172.21.15.3:41860) 2023-10-03T10:15:37.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.057+0000 7f31877fe700 1 -- 172.21.15.3:0/3792964544 learned_addr learned my addr 172.21.15.3:0/3792964544 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:37.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.057+0000 7f31867fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f318809e1d0 0x7f3188138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42096/0 (socket says 172.21.15.3:42096) 2023-10-03T10:15:37.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.057+0000 7f31877fe700 1 -- 172.21.15.3:0/3792964544 >> v2:172.21.15.3:3300/0 conn(0x7f318809e1d0 msgr2=0x7f3188138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:37.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.057+0000 7f31877fe700 1 --2- 172.21.15.3:0/3792964544 >> v2:172.21.15.3:3300/0 conn(0x7f318809e1d0 0x7f3188138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.058+0000 7f31877fe700 1 -- 172.21.15.3:0/3792964544 >> v1:172.21.15.3:6789/0 conn(0x7f318809d6f0 legacy=0x7f318809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:37.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.058+0000 7f31877fe700 1 -- 172.21.15.3:0/3792964544 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318809a500 con 0x7f3188139170 2023-10-03T10:15:37.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.058+0000 7f3185ffb700 1 -- 172.21.15.3:0/3792964544 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2189892967 0 0) 0x7f318809b120 con 0x7f318809d6f0 2023-10-03T10:15:37.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.058+0000 7f31877fe700 1 --2- 172.21.15.3:0/3792964544 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3188139170 0x7f318813b550 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3180009e80 tx=0x7f318000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46ec8fa18de27888 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:37.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.058+0000 7f3185ffb700 1 -- 172.21.15.3:0/3792964544 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318000e070 con 0x7f3188139170 2023-10-03T10:15:37.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.058+0000 7f3185ffb700 1 -- 172.21.15.3:0/3792964544 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318000b8d0 con 0x7f3188139170 2023-10-03T10:15:37.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.058+0000 7f3185ffb700 1 -- 172.21.15.3:0/3792964544 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318000bc80 con 0x7f3188139170 2023-10-03T10:15:37.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.059+0000 7f318d82c700 1 -- 172.21.15.3:0/3792964544 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3188139170 msgr2=0x7f318813b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:37.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.059+0000 7f318d82c700 1 --2- 172.21.15.3:0/3792964544 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3188139170 0x7f318813b550 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3180009e80 tx=0x7f318000b330 comp rx=0 tx=0).stop 2023-10-03T10:15:37.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.059+0000 7f318d82c700 1 -- 172.21.15.3:0/3792964544 shutdown_connections 2023-10-03T10:15:37.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.059+0000 7f318d82c700 1 --2- 172.21.15.3:0/3792964544 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3188139170 0x7f318813b550 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.059+0000 7f318d82c700 1 --2- 172.21.15.3:0/3792964544 >> v2:172.21.15.3:3300/0 conn(0x7f318809e1d0 0x7f3188138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.059+0000 7f318d82c700 1 -- 172.21.15.3:0/3792964544 >> 172.21.15.3:0/3792964544 conn(0x7f3188066f50 msgr2=0x7f318809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.059+0000 7f318d82c700 1 -- 172.21.15.3:0/3792964544 shutdown_connections 2023-10-03T10:15:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.059+0000 7f318d82c700 1 -- 172.21.15.3:0/3792964544 wait complete. 2023-10-03T10:15:37.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.060+0000 7f318d82c700 1 Processor -- start 2023-10-03T10:15:37.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.060+0000 7f318d82c700 1 -- start start 2023-10-03T10:15:37.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.061+0000 7f318d82c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f318809d6f0 0x7f3188121740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:37.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.061+0000 7f318d82c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f318809e1d0 0x7f3188121c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:37.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.061+0000 7f318d82c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3188139170 0x7f31881221c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:37.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.061+0000 7f318d82c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3188124260 con 0x7f318809e1d0 2023-10-03T10:15:37.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.061+0000 7f318d82c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3188124f80 con 0x7f3188139170 2023-10-03T10:15:37.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.061+0000 7f318d82c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31880a3ab0 con 0x7f318809d6f0 2023-10-03T10:15:37.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.061+0000 7f31867fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f318809e1d0 0x7f3188121c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:37.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.061+0000 7f3186ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f318809d6f0 0x7f3188121740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:37.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.061+0000 7f31867fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f318809e1d0 0x7f3188121c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42104/0 (socket says 172.21.15.3:42104) 2023-10-03T10:15:37.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.061+0000 7f31867fc700 1 -- 172.21.15.3:0/3790271646 learned_addr learned my addr 172.21.15.3:0/3790271646 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:37.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.061+0000 7f3186ffd700 1 -- 172.21.15.3:0/3790271646 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3188139170 msgr2=0x7f31881221c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:37.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.061+0000 7f3186ffd700 1 --2- 172.21.15.3:0/3790271646 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3188139170 0x7f31881221c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.061+0000 7f3186ffd700 1 -- 172.21.15.3:0/3790271646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f318809e1d0 msgr2=0x7f3188121c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:37.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.061+0000 7f3186ffd700 1 --2- 172.21.15.3:0/3790271646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f318809e1d0 0x7f3188121c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.061+0000 7f3186ffd700 1 -- 172.21.15.3:0/3790271646 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3180009b30 con 0x7f318809d6f0 2023-10-03T10:15:37.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.061+0000 7f31867fc700 1 --2- 172.21.15.3:0/3790271646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f318809e1d0 0x7f3188121c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:37.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.062+0000 7f3186ffd700 1 --2- 172.21.15.3:0/3790271646 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f318809d6f0 0x7f3188121740 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f317800c460 tx=0x7f317800c770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:37.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.062+0000 7f316ffff700 1 -- 172.21.15.3:0/3790271646 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31780054e0 con 0x7f318809d6f0 2023-10-03T10:15:37.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.062+0000 7f316ffff700 1 -- 172.21.15.3:0/3790271646 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3178003680 con 0x7f318809d6f0 2023-10-03T10:15:37.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.062+0000 7f316ffff700 1 -- 172.21.15.3:0/3790271646 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3178003a30 con 0x7f318809d6f0 2023-10-03T10:15:37.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.062+0000 7f318d82c700 1 -- 172.21.15.3:0/3790271646 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f318809c710 con 0x7f318809d6f0 2023-10-03T10:15:37.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.062+0000 7f318d82c700 1 -- 172.21.15.3:0/3790271646 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31881259f0 con 0x7f318809d6f0 2023-10-03T10:15:37.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.063+0000 7f316ffff700 1 -- 172.21.15.3:0/3790271646 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3178003b90 con 0x7f318809d6f0 2023-10-03T10:15:37.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.063+0000 7f316ffff700 1 -- 172.21.15.3:0/3790271646 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f317803eb10 con 0x7f318809d6f0 2023-10-03T10:15:37.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.063+0000 7f318d82c700 1 -- 172.21.15.3:0/3790271646 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31881250c0 con 0x7f318809d6f0 2023-10-03T10:15:37.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.069+0000 7f316ffff700 1 -- 172.21.15.3:0/3790271646 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f31881250c0 con 0x7f318809d6f0 2023-10-03T10:15:37.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.094+0000 7fa86e6ec700 1 Processor -- start 2023-10-03T10:15:37.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.094+0000 7fa86e6ec700 1 -- start start 2023-10-03T10:15:37.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.094+0000 7fa86e6ec700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa868133420 0x7fa8681337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:37.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.094+0000 7fa86e6ec700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa868133e30 0x7fa86812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:37.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.094+0000 7fa86e6ec700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa86806b990 con 0x7fa868134260 2023-10-03T10:15:37.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.094+0000 7fa86e6ec700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8680a0590 con 0x7fa868133420 2023-10-03T10:15:37.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.094+0000 7fa86e6ec700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86809fb80 con 0x7fa868133e30 2023-10-03T10:15:37.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.095+0000 7fa8677fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa868133e30 0x7fa86812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:37.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.095+0000 7fa8677fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa868133e30 0x7fa86812ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50426/0 (socket says 172.21.15.3:50426) 2023-10-03T10:15:37.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.095+0000 7fa8677fe700 1 -- 172.21.15.3:0/1425996683 learned_addr learned my addr 172.21.15.3:0/1425996683 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:37.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.095+0000 7fa8677fe700 1 -- 172.21.15.3:0/1425996683 >> v2:172.21.15.3:3300/0 conn(0x7fa868133420 msgr2=0x7fa8681337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:37.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.095+0000 7fa8677fe700 1 --2- 172.21.15.3:0/1425996683 >> v2:172.21.15.3:3300/0 conn(0x7fa868133420 0x7fa8681337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.095+0000 7fa8677fe700 1 -- 172.21.15.3:0/1425996683 >> v1:172.21.15.3:6789/0 conn(0x7fa868134260 legacy=0x7fa8681304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:37.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.095+0000 7fa8677fe700 1 -- 172.21.15.3:0/1425996683 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8680a0030 con 0x7fa868133e30 2023-10-03T10:15:37.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.095+0000 7fa8677fe700 1 --2- 172.21.15.3:0/1425996683 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa868133e30 0x7fa86812ff90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0098f0 tx=0x7fa85c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ece93449512e7ee5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:37.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.095+0000 7fa866ffd700 1 -- 172.21.15.3:0/1425996683 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c004230 con 0x7fa868133e30 2023-10-03T10:15:37.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.095+0000 7fa866ffd700 1 -- 172.21.15.3:0/1425996683 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85c004390 con 0x7fa868133e30 2023-10-03T10:15:37.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.095+0000 7fa866ffd700 1 -- 172.21.15.3:0/1425996683 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c0036a0 con 0x7fa868133e30 2023-10-03T10:15:37.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.096+0000 7fa86e6ec700 1 -- 172.21.15.3:0/1425996683 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa868133e30 msgr2=0x7fa86812ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:37.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.096+0000 7fa86e6ec700 1 --2- 172.21.15.3:0/1425996683 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa868133e30 0x7fa86812ff90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0098f0 tx=0x7fa85c009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:37.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.096+0000 7fa86e6ec700 1 -- 172.21.15.3:0/1425996683 shutdown_connections 2023-10-03T10:15:37.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.096+0000 7fa86e6ec700 1 --2- 172.21.15.3:0/1425996683 >> v2:172.21.15.3:3300/0 conn(0x7fa868133420 0x7fa8681337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.096+0000 7fa86e6ec700 1 --2- 172.21.15.3:0/1425996683 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa868133e30 0x7fa86812ff90 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.096+0000 7fa86e6ec700 1 -- 172.21.15.3:0/1425996683 >> 172.21.15.3:0/1425996683 conn(0x7fa8680a4210 msgr2=0x7fa8680a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:37.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.096+0000 7fa86e6ec700 1 -- 172.21.15.3:0/1425996683 shutdown_connections 2023-10-03T10:15:37.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.096+0000 7fa86e6ec700 1 -- 172.21.15.3:0/1425996683 wait complete. 2023-10-03T10:15:37.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.098+0000 7fa86e6ec700 1 Processor -- start 2023-10-03T10:15:37.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.098+0000 7fa86e6ec700 1 -- start start 2023-10-03T10:15:37.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.099+0000 7fa86e6ec700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa868133420 0x7fa86809c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:37.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.099+0000 7fa86e6ec700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa868133e30 0x7fa86809cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:37.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.099+0000 7fa86e6ec700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa868134260 0x7fa86809d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:37.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.099+0000 7fa86e6ec700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86809e2f0 con 0x7fa868134260 2023-10-03T10:15:37.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.099+0000 7fa86e6ec700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86811ff70 con 0x7fa868133e30 2023-10-03T10:15:37.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.099+0000 7fa86e6ec700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa86812d940 con 0x7fa868133420 2023-10-03T10:15:37.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.099+0000 7fa867fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa868133420 0x7fa86809c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:37.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.099+0000 7fa867fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa868133420 0x7fa86809c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41876/0 (socket says 172.21.15.3:41876) 2023-10-03T10:15:37.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.099+0000 7fa867fff700 1 -- 172.21.15.3:0/3379348414 learned_addr learned my addr 172.21.15.3:0/3379348414 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:37.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.099+0000 7fa8677fe700 1 --2- 172.21.15.3:0/3379348414 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa868133e30 0x7fa86809cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:37.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.099+0000 7fa86cc89700 1 --2- 172.21.15.3:0/3379348414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa868134260 0x7fa86809d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:37.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.100+0000 7fa867fff700 1 -- 172.21.15.3:0/3379348414 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa868133e30 msgr2=0x7fa86809cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:37.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.100+0000 7fa867fff700 1 --2- 172.21.15.3:0/3379348414 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa868133e30 0x7fa86809cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.100+0000 7fa867fff700 1 -- 172.21.15.3:0/3379348414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa868134260 msgr2=0x7fa86809d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:37.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.100+0000 7fa867fff700 1 --2- 172.21.15.3:0/3379348414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa868134260 0x7fa86809d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.100+0000 7fa867fff700 1 -- 172.21.15.3:0/3379348414 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85c009550 con 0x7fa868133420 2023-10-03T10:15:37.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.100+0000 7fa8677fe700 1 --2- 172.21.15.3:0/3379348414 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa868133e30 0x7fa86809cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:37.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.100+0000 7fa867fff700 1 --2- 172.21.15.3:0/3379348414 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa868133420 0x7fa86809c6e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa85800b530 tx=0x7fa85800b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:37.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.100+0000 7fa864ff9700 1 -- 172.21.15.3:0/3379348414 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa858011830 con 0x7fa868133420 2023-10-03T10:15:37.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.100+0000 7fa864ff9700 1 -- 172.21.15.3:0/3379348414 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa858011990 con 0x7fa868133420 2023-10-03T10:15:37.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.100+0000 7fa86e6ec700 1 -- 172.21.15.3:0/3379348414 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa86812db80 con 0x7fa868133420 2023-10-03T10:15:37.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.100+0000 7fa864ff9700 1 -- 172.21.15.3:0/3379348414 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa858011d40 con 0x7fa868133420 2023-10-03T10:15:37.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.100+0000 7fa86e6ec700 1 -- 172.21.15.3:0/3379348414 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa868120510 con 0x7fa868133420 2023-10-03T10:15:37.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.101+0000 7fa86e6ec700 1 -- 172.21.15.3:0/3379348414 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa868120370 con 0x7fa868133420 2023-10-03T10:15:37.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.102+0000 7fa864ff9700 1 -- 172.21.15.3:0/3379348414 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa85801a400 con 0x7fa868133420 2023-10-03T10:15:37.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.102+0000 7fa864ff9700 1 -- 172.21.15.3:0/3379348414 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa85803fa90 con 0x7fa868133420 2023-10-03T10:15:37.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.104+0000 7f2b5a882700 1 Processor -- start 2023-10-03T10:15:37.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.104+0000 7f2b5a882700 1 -- start start 2023-10-03T10:15:37.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.105+0000 7f2b5a882700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2b540a0090 0x7f2b54136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:37.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.105+0000 7f2b5a882700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b5409d130 0x7f2b54139170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:37.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.105+0000 7f2b5a882700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b5409ba70 con 0x7f2b5409c650 2023-10-03T10:15:37.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.105+0000 7f2b5a882700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b5409e8f0 con 0x7f2b540a0090 2023-10-03T10:15:37.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.105+0000 7f2b5a882700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b540a0760 con 0x7f2b5409d130 2023-10-03T10:15:37.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.105+0000 7f2b53fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2b5409c650 0x7f2b5409ca20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34372/0 (socket says 172.21.15.3:34372) 2023-10-03T10:15:37.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.105+0000 7f2b53fff700 1 -- 172.21.15.3:0/2231274631 learned_addr learned my addr 172.21.15.3:0/2231274631 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:37.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.105+0000 7f2b58e1f700 1 --2- 172.21.15.3:0/2231274631 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b5409d130 0x7f2b54139170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:37.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.105+0000 7f2b537fe700 1 --2- 172.21.15.3:0/2231274631 >> v2:172.21.15.3:3300/0 conn(0x7f2b540a0090 0x7f2b54136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:37.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.105+0000 7f2b52ffd700 1 -- 172.21.15.3:0/2231274631 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2489458711 0 0) 0x7f2b5409ba70 con 0x7f2b5409c650 2023-10-03T10:15:37.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.105+0000 7f2b52ffd700 1 -- 172.21.15.3:0/2231274631 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b3c0035b0 con 0x7f2b5409c650 2023-10-03T10:15:37.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.105+0000 7f2b58e1f700 1 -- 172.21.15.3:0/2231274631 >> v2:172.21.15.3:3300/0 conn(0x7f2b540a0090 msgr2=0x7f2b54136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:37.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.105+0000 7f2b58e1f700 1 --2- 172.21.15.3:0/2231274631 >> v2:172.21.15.3:3300/0 conn(0x7f2b540a0090 0x7f2b54136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.105+0000 7f2b58e1f700 1 -- 172.21.15.3:0/2231274631 >> v1:172.21.15.3:6789/0 conn(0x7f2b5409c650 legacy=0x7f2b5409ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:37.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.105+0000 7f2b58e1f700 1 -- 172.21.15.3:0/2231274631 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5409a080 con 0x7f2b5409d130 2023-10-03T10:15:37.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.106+0000 7f2b58e1f700 1 --2- 172.21.15.3:0/2231274631 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b5409d130 0x7f2b54139170 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c009d90 tx=0x7f2b4c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80c79026d875ae44 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:37.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.111+0000 7f2b52ffd700 1 -- 172.21.15.3:0/2231274631 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c00e070 con 0x7f2b5409d130 2023-10-03T10:15:37.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.111+0000 7f2b52ffd700 1 -- 172.21.15.3:0/2231274631 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4c00bb20 con 0x7f2b5409d130 2023-10-03T10:15:37.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.111+0000 7f2b52ffd700 1 -- 172.21.15.3:0/2231274631 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c012430 con 0x7f2b5409d130 2023-10-03T10:15:37.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.111+0000 7fa864ff9700 1 -- 172.21.15.3:0/3379348414 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa858022b40 con 0x7fa868133420 2023-10-03T10:15:37.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.112+0000 7f2b5a882700 1 -- 172.21.15.3:0/2231274631 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b5409d130 msgr2=0x7f2b54139170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:37.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.112+0000 7f2b5a882700 1 --2- 172.21.15.3:0/2231274631 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b5409d130 0x7f2b54139170 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c009d90 tx=0x7f2b4c00b330 comp rx=0 tx=0).stop 2023-10-03T10:15:37.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.112+0000 7f2b5a882700 1 -- 172.21.15.3:0/2231274631 shutdown_connections 2023-10-03T10:15:37.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.112+0000 7f2b5a882700 1 --2- 172.21.15.3:0/2231274631 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b5409d130 0x7f2b54139170 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.112+0000 7f2b5a882700 1 --2- 172.21.15.3:0/2231274631 >> v2:172.21.15.3:3300/0 conn(0x7f2b540a0090 0x7f2b54136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.112+0000 7f2b5a882700 1 -- 172.21.15.3:0/2231274631 >> 172.21.15.3:0/2231274631 conn(0x7f2b540a3f70 msgr2=0x7f2b540a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:37.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.112+0000 7f2b5a882700 1 -- 172.21.15.3:0/2231274631 shutdown_connections 2023-10-03T10:15:37.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.112+0000 7f2b5a882700 1 -- 172.21.15.3:0/2231274631 wait complete. 2023-10-03T10:15:37.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.113+0000 7f2b5a882700 1 Processor -- start 2023-10-03T10:15:37.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.113+0000 7f2b5a882700 1 -- start start 2023-10-03T10:15:37.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.114+0000 7f2b5a882700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b5409c650 0x7f2b541c0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:37.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.114+0000 7f2b5a882700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b5409d130 0x7f2b541c1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:37.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.114+0000 7f2b5a882700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b540a0090 0x7f2b5411a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:37.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.114+0000 7f2b5a882700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b541c18a0 con 0x7f2b540a0090 2023-10-03T10:15:37.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.114+0000 7f2b5a882700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b541c09d0 con 0x7f2b5409d130 2023-10-03T10:15:37.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.114+0000 7f2b5a882700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5409d980 con 0x7f2b5409c650 2023-10-03T10:15:37.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.114+0000 7f2b537fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b5409d130 0x7f2b541c1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:37.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.114+0000 7f2b537fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b5409d130 0x7f2b541c1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50442/0 (socket says 172.21.15.3:50442) 2023-10-03T10:15:37.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.114+0000 7f2b537fe700 1 -- 172.21.15.3:0/3151272405 learned_addr learned my addr 172.21.15.3:0/3151272405 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:37.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.114+0000 7f2b58e1f700 1 --2- 172.21.15.3:0/3151272405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b540a0090 0x7f2b5411a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:37.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.114+0000 7f2b53fff700 1 --2- 172.21.15.3:0/3151272405 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b5409c650 0x7f2b541c0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:37.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.114+0000 7f2b537fe700 1 -- 172.21.15.3:0/3151272405 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b5409c650 msgr2=0x7f2b541c0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:37.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.114+0000 7f2b537fe700 1 --2- 172.21.15.3:0/3151272405 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b5409c650 0x7f2b541c0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.114+0000 7f2b537fe700 1 -- 172.21.15.3:0/3151272405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b540a0090 msgr2=0x7f2b5411a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:37.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.114+0000 7f2b537fe700 1 --2- 172.21.15.3:0/3151272405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b540a0090 0x7f2b5411a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.114+0000 7f2b537fe700 1 -- 172.21.15.3:0/3151272405 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b4c009a70 con 0x7f2b5409d130 2023-10-03T10:15:37.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.114+0000 7f2b58e1f700 1 --2- 172.21.15.3:0/3151272405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b540a0090 0x7f2b5411a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:37.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.115+0000 7f2b537fe700 1 --2- 172.21.15.3:0/3151272405 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b5409d130 0x7f2b541c1360 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800e9e0 tx=0x7f2b4800ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:37.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.115+0000 7f2b50ff9700 1 -- 172.21.15.3:0/3151272405 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48009710 con 0x7f2b5409d130 2023-10-03T10:15:37.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.115+0000 7f2b50ff9700 1 -- 172.21.15.3:0/3151272405 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4800c850 con 0x7f2b5409d130 2023-10-03T10:15:37.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.115+0000 7f2b50ff9700 1 -- 172.21.15.3:0/3151272405 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4800cbc0 con 0x7f2b5409d130 2023-10-03T10:15:37.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.116+0000 7f2b5a882700 1 -- 172.21.15.3:0/3151272405 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b5411d210 con 0x7f2b5409d130 2023-10-03T10:15:37.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.116+0000 7f2b5a882700 1 -- 172.21.15.3:0/3151272405 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b5411c580 con 0x7f2b5409d130 2023-10-03T10:15:37.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.117+0000 7f2b50ff9700 1 -- 172.21.15.3:0/3151272405 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2b4800cd20 con 0x7f2b5409d130 2023-10-03T10:15:37.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.117+0000 7f2b5a882700 1 -- 172.21.15.3:0/3151272405 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b540747f0 con 0x7f2b5409d130 2023-10-03T10:15:37.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.117+0000 7f2b50ff9700 1 -- 172.21.15.3:0/3151272405 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2b4803f8d0 con 0x7f2b5409d130 2023-10-03T10:15:37.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.127+0000 7f2b50ff9700 1 -- 172.21.15.3:0/3151272405 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2b480232f0 con 0x7f2b5409d130 2023-10-03T10:15:37.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.144+0000 7fe383219700 1 Processor -- start 2023-10-03T10:15:37.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.144+0000 7fe383219700 1 -- start start 2023-10-03T10:15:37.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.144+0000 7fe383219700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe37c09c650 0x7fe37c09ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:37.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.144+0000 7fe383219700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe37c0a0090 0x7fe37c09cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:37.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.144+0000 7fe383219700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe37c09ba70 con 0x7fe37c131970 2023-10-03T10:15:37.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.144+0000 7fe383219700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe37c09d4a0 con 0x7fe37c09c650 2023-10-03T10:15:37.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.144+0000 7fe383219700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c09e8f0 con 0x7fe37c0a0090 2023-10-03T10:15:37.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.144+0000 7fe37bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe37c0a0090 0x7fe37c09cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:37.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.145+0000 7fe37bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe37c0a0090 0x7fe37c09cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41900/0 (socket says 172.21.15.3:41900) 2023-10-03T10:15:37.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.145+0000 7fe37bfff700 1 -- 172.21.15.3:0/3700217967 learned_addr learned my addr 172.21.15.3:0/3700217967 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:37.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.145+0000 7fe380fb5700 1 --2- 172.21.15.3:0/3700217967 >> v2:172.21.15.3:3300/0 conn(0x7fe37c09c650 0x7fe37c09ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:37.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.145+0000 7fe37bfff700 1 -- 172.21.15.3:0/3700217967 >> v2:172.21.15.3:3300/0 conn(0x7fe37c09c650 msgr2=0x7fe37c09ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:37.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.145+0000 7fe37bfff700 1 --2- 172.21.15.3:0/3700217967 >> v2:172.21.15.3:3300/0 conn(0x7fe37c09c650 0x7fe37c09ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.145+0000 7fe37bfff700 1 -- 172.21.15.3:0/3700217967 >> v1:172.21.15.3:6789/0 conn(0x7fe37c131970 legacy=0x7fe37c131db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:37.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.145+0000 7fe37bfff700 1 -- 172.21.15.3:0/3700217967 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37c0a0760 con 0x7fe37c0a0090 2023-10-03T10:15:37.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.145+0000 7fe380fb5700 1 --2- 172.21.15.3:0/3700217967 >> v2:172.21.15.3:3300/0 conn(0x7fe37c09c650 0x7fe37c09ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:37.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.145+0000 7fe37bfff700 1 --2- 172.21.15.3:0/3700217967 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe37c0a0090 0x7fe37c09cf60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe3680098f0 tx=0x7fe368009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf1d0835999ad2d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:37.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.145+0000 7fe37b7fe700 1 -- 172.21.15.3:0/3700217967 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe368011830 con 0x7fe37c0a0090 2023-10-03T10:15:37.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.145+0000 7fe37b7fe700 1 -- 172.21.15.3:0/3700217967 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe368011990 con 0x7fe37c0a0090 2023-10-03T10:15:37.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.145+0000 7fe37b7fe700 1 -- 172.21.15.3:0/3700217967 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe368011d20 con 0x7fe37c0a0090 2023-10-03T10:15:37.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.146+0000 7fe383219700 1 -- 172.21.15.3:0/3700217967 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe37c0a0090 msgr2=0x7fe37c09cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:37.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.146+0000 7fe383219700 1 --2- 172.21.15.3:0/3700217967 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe37c0a0090 0x7fe37c09cf60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe3680098f0 tx=0x7fe368009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:37.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.146+0000 7fe383219700 1 -- 172.21.15.3:0/3700217967 shutdown_connections 2023-10-03T10:15:37.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.146+0000 7fe383219700 1 --2- 172.21.15.3:0/3700217967 >> v2:172.21.15.3:3300/0 conn(0x7fe37c09c650 0x7fe37c09ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.146+0000 7fe383219700 1 --2- 172.21.15.3:0/3700217967 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe37c0a0090 0x7fe37c09cf60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.146+0000 7fe383219700 1 -- 172.21.15.3:0/3700217967 >> 172.21.15.3:0/3700217967 conn(0x7fe37c0a3f70 msgr2=0x7fe37c0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:37.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.147+0000 7fe383219700 1 -- 172.21.15.3:0/3700217967 shutdown_connections 2023-10-03T10:15:37.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.147+0000 7fe383219700 1 -- 172.21.15.3:0/3700217967 wait complete. 2023-10-03T10:15:37.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.148+0000 7fe383219700 1 Processor -- start 2023-10-03T10:15:37.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.148+0000 7fe383219700 1 -- start start 2023-10-03T10:15:37.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.149+0000 7fe383219700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe37c09c650 0x7fe37c06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:37.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.149+0000 7fe383219700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe37c0a0090 0x7fe37c06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:37.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.149+0000 7fe383219700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe37c131970 0x7fe37c06c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:37.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.149+0000 7fe383219700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c121c80 con 0x7fe37c131970 2023-10-03T10:15:37.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.149+0000 7fe383219700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c1216e0 con 0x7fe37c0a0090 2023-10-03T10:15:37.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.149+0000 7fe383219700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c120c00 con 0x7fe37c09c650 2023-10-03T10:15:37.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.149+0000 7fe3817b6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe37c131970 0x7fe37c06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:37.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.149+0000 7fe3817b6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe37c131970 0x7fe37c06c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42148/0 (socket says 172.21.15.3:42148) 2023-10-03T10:15:37.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.149+0000 7fe3817b6700 1 -- 172.21.15.3:0/2366619867 learned_addr learned my addr 172.21.15.3:0/2366619867 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:37.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.149+0000 7fe380fb5700 1 --2- 172.21.15.3:0/2366619867 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe37c09c650 0x7fe37c06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:37.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.150+0000 7fe37bfff700 1 --2- 172.21.15.3:0/2366619867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe37c0a0090 0x7fe37c06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:37.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.150+0000 7fe3817b6700 1 -- 172.21.15.3:0/2366619867 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe37c09c650 msgr2=0x7fe37c06b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:37.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.150+0000 7fe3817b6700 1 --2- 172.21.15.3:0/2366619867 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe37c09c650 0x7fe37c06b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.150+0000 7fe3817b6700 1 -- 172.21.15.3:0/2366619867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe37c0a0090 msgr2=0x7fe37c06bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:37.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.150+0000 7fe3817b6700 1 --2- 172.21.15.3:0/2366619867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe37c0a0090 0x7fe37c06bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.150+0000 7fe3817b6700 1 -- 172.21.15.3:0/2366619867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3700094a0 con 0x7fe37c131970 2023-10-03T10:15:37.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.150+0000 7fe37bfff700 1 --2- 172.21.15.3:0/2366619867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe37c0a0090 0x7fe37c06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:15:37.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.150+0000 7fe3817b6700 1 --2- 172.21.15.3:0/2366619867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe37c131970 0x7fe37c06c330 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fe37400cc80 tx=0x7fe37400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:37.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.150+0000 7fe3797fa700 1 -- 172.21.15.3:0/2366619867 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374003c00 con 0x7fe37c131970 2023-10-03T10:15:37.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.150+0000 7fe3797fa700 1 -- 172.21.15.3:0/2366619867 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe374003d60 con 0x7fe37c131970 2023-10-03T10:15:37.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.150+0000 7fe383219700 1 -- 172.21.15.3:0/2366619867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe368009550 con 0x7fe37c131970 2023-10-03T10:15:37.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.150+0000 7fe3797fa700 1 -- 172.21.15.3:0/2366619867 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37400ab30 con 0x7fe37c131970 2023-10-03T10:15:37.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.150+0000 7fe383219700 1 -- 172.21.15.3:0/2366619867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe37c12f580 con 0x7fe37c131970 2023-10-03T10:15:37.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.151+0000 7fe3797fa700 1 -- 172.21.15.3:0/2366619867 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe37400ac90 con 0x7fe37c131970 2023-10-03T10:15:37.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.151+0000 7fe3797fa700 1 -- 172.21.15.3:0/2366619867 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe37403f5c0 con 0x7fe37c131970 2023-10-03T10:15:37.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.152+0000 7fe383219700 1 -- 172.21.15.3:0/2366619867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe37c12f580 con 0x7fe37c131970 2023-10-03T10:15:37.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.161+0000 7fe3797fa700 1 -- 172.21.15.3:0/2366619867 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe37c12f580 con 0x7fe37c131970 2023-10-03T10:15:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.339+0000 7fa86e6ec700 1 -- 172.21.15.3:0/3379348414 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa868120370 con 0x7fa868133420 2023-10-03T10:15:37.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.340+0000 7fa864ff9700 1 -- 172.21.15.3:0/3379348414 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa868120370 con 0x7fa868133420 2023-10-03T10:15:37.340 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:37.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.343+0000 7fa86e6ec700 1 -- 172.21.15.3:0/3379348414 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa868133420 msgr2=0x7fa86809c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:37.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.343+0000 7fa86e6ec700 1 --2- 172.21.15.3:0/3379348414 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa868133420 0x7fa86809c6e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa85800b530 tx=0x7fa85800b840 comp rx=0 tx=0).stop 2023-10-03T10:15:37.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.343+0000 7fa86e6ec700 1 -- 172.21.15.3:0/3379348414 shutdown_connections 2023-10-03T10:15:37.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.343+0000 7fa86e6ec700 1 --2- 172.21.15.3:0/3379348414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa868134260 0x7fa86809d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.343+0000 7fa86e6ec700 1 --2- 172.21.15.3:0/3379348414 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa868133420 0x7fa86809c6e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.343+0000 7fa86e6ec700 1 --2- 172.21.15.3:0/3379348414 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa868133e30 0x7fa86809cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.343+0000 7fa86e6ec700 1 -- 172.21.15.3:0/3379348414 >> 172.21.15.3:0/3379348414 conn(0x7fa8680a4210 msgr2=0x7fa86812e170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:37.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.343+0000 7fa86e6ec700 1 -- 172.21.15.3:0/3379348414 shutdown_connections 2023-10-03T10:15:37.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.343+0000 7fa86e6ec700 1 -- 172.21.15.3:0/3379348414 wait complete. 2023-10-03T10:15:37.357 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:37.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.365+0000 7f318d82c700 1 -- 172.21.15.3:0/3790271646 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f31881250c0 con 0x7f318809d6f0 2023-10-03T10:15:37.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.365+0000 7f316ffff700 1 -- 172.21.15.3:0/3790271646 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f31881250c0 con 0x7f318809d6f0 2023-10-03T10:15:37.365 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:37.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.369+0000 7f318d82c700 1 -- 172.21.15.3:0/3790271646 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f318809d6f0 msgr2=0x7f3188121740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:37.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.369+0000 7f318d82c700 1 --2- 172.21.15.3:0/3790271646 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f318809d6f0 0x7f3188121740 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f317800c460 tx=0x7f317800c770 comp rx=0 tx=0).stop 2023-10-03T10:15:37.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.369+0000 7f318d82c700 1 -- 172.21.15.3:0/3790271646 shutdown_connections 2023-10-03T10:15:37.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.369+0000 7f318d82c700 1 --2- 172.21.15.3:0/3790271646 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3188139170 0x7f31881221c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.369+0000 7f318d82c700 1 --2- 172.21.15.3:0/3790271646 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f318809d6f0 0x7f3188121740 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.369+0000 7f318d82c700 1 --2- 172.21.15.3:0/3790271646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f318809e1d0 0x7f3188121c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.369+0000 7f318d82c700 1 -- 172.21.15.3:0/3790271646 >> 172.21.15.3:0/3790271646 conn(0x7f3188066f50 msgr2=0x7f3188070c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:37.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.369+0000 7f318d82c700 1 -- 172.21.15.3:0/3790271646 shutdown_connections 2023-10-03T10:15:37.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.369+0000 7f318d82c700 1 -- 172.21.15.3:0/3790271646 wait complete. 2023-10-03T10:15:37.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.377+0000 7fe383219700 1 -- 172.21.15.3:0/2366619867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe37c12f580 con 0x7fe37c131970 2023-10-03T10:15:37.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.378+0000 7fe3797fa700 1 -- 172.21.15.3:0/2366619867 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe37c12f580 con 0x7fe37c131970 2023-10-03T10:15:37.378 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:37.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.380+0000 7fe383219700 1 -- 172.21.15.3:0/2366619867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe37c131970 msgr2=0x7fe37c06c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:37.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.380+0000 7fe383219700 1 --2- 172.21.15.3:0/2366619867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe37c131970 0x7fe37c06c330 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fe37400cc80 tx=0x7fe37400cf90 comp rx=0 tx=0).stop 2023-10-03T10:15:37.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.380+0000 7fe383219700 1 -- 172.21.15.3:0/2366619867 shutdown_connections 2023-10-03T10:15:37.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.380+0000 7fe383219700 1 --2- 172.21.15.3:0/2366619867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe37c131970 0x7fe37c06c330 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.380+0000 7fe383219700 1 --2- 172.21.15.3:0/2366619867 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe37c09c650 0x7fe37c06b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.380+0000 7fe383219700 1 --2- 172.21.15.3:0/2366619867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe37c0a0090 0x7fe37c06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.380+0000 7fe383219700 1 -- 172.21.15.3:0/2366619867 >> 172.21.15.3:0/2366619867 conn(0x7fe37c0a3f70 msgr2=0x7fe37c1b8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:37.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.380+0000 7fe383219700 1 -- 172.21.15.3:0/2366619867 shutdown_connections 2023-10-03T10:15:37.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.380+0000 7fe383219700 1 -- 172.21.15.3:0/2366619867 wait complete. 2023-10-03T10:15:37.389 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:37.394 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:37.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.410+0000 7f2b5a882700 1 -- 172.21.15.3:0/3151272405 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2b5411d710 con 0x7f2b5409d130 2023-10-03T10:15:37.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.411+0000 7f2b50ff9700 1 -- 172.21.15.3:0/3151272405 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2b4801ae80 con 0x7f2b5409d130 2023-10-03T10:15:37.411 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:37.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.413+0000 7f2b5a882700 1 -- 172.21.15.3:0/3151272405 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b5409d130 msgr2=0x7f2b541c1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:37.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.413+0000 7f2b5a882700 1 --2- 172.21.15.3:0/3151272405 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b5409d130 0x7f2b541c1360 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800e9e0 tx=0x7f2b4800ecf0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.413+0000 7f2b5a882700 1 -- 172.21.15.3:0/3151272405 shutdown_connections 2023-10-03T10:15:37.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.413+0000 7f2b5a882700 1 --2- 172.21.15.3:0/3151272405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b540a0090 0x7f2b5411a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.413+0000 7f2b5a882700 1 --2- 172.21.15.3:0/3151272405 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b5409c650 0x7f2b541c0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.413+0000 7f2b5a882700 1 --2- 172.21.15.3:0/3151272405 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b5409d130 0x7f2b541c1360 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:37.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.413+0000 7f2b5a882700 1 -- 172.21.15.3:0/3151272405 >> 172.21.15.3:0/3151272405 conn(0x7f2b540a3f70 msgr2=0x7f2b5412edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:37.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.413+0000 7f2b5a882700 1 -- 172.21.15.3:0/3151272405 shutdown_connections 2023-10-03T10:15:37.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:37.413+0000 7f2b5a882700 1 -- 172.21.15.3:0/3151272405 wait complete. 2023-10-03T10:15:37.427 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:38.358 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:38.390 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:38.395 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:38.428 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:38.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.523+0000 7fd415c63700 1 Processor -- start 2023-10-03T10:15:38.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.523+0000 7fd415c63700 1 -- start start 2023-10-03T10:15:38.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.523+0000 7fd415c63700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd41006a770 0x7fd41009d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:38.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.523+0000 7fd415c63700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd41009dc30 0x7fd41009e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:38.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.523+0000 7fd415c63700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd41009a810 con 0x7fd41013c1e0 2023-10-03T10:15:38.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.523+0000 7fd415c63700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd41006ab40 con 0x7fd41009dc30 2023-10-03T10:15:38.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.523+0000 7fd415c63700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd41009ebc0 con 0x7fd41006a770 2023-10-03T10:15:38.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.524+0000 7fd40f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd41006a770 0x7fd41009d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:38.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.524+0000 7fd40f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd41006a770 0x7fd41009d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50470/0 (socket says 172.21.15.3:50470) 2023-10-03T10:15:38.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.524+0000 7fd40f7fe700 1 -- 172.21.15.3:0/3390016021 learned_addr learned my addr 172.21.15.3:0/3390016021 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:38.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.524+0000 7fd40effd700 1 --2- 172.21.15.3:0/3390016021 >> v2:172.21.15.3:3300/0 conn(0x7fd41009dc30 0x7fd41009e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:38.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.524+0000 7fd40effd700 1 -- 172.21.15.3:0/3390016021 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd41006a770 msgr2=0x7fd41009d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:38.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.524+0000 7fd40effd700 1 --2- 172.21.15.3:0/3390016021 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd41006a770 0x7fd41009d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.524+0000 7fd40effd700 1 -- 172.21.15.3:0/3390016021 >> v1:172.21.15.3:6789/0 conn(0x7fd41013c1e0 legacy=0x7fd41013e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:38.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.524+0000 7fd40effd700 1 -- 172.21.15.3:0/3390016021 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41009e920 con 0x7fd41009dc30 2023-10-03T10:15:38.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.524+0000 7fd40f7fe700 1 --2- 172.21.15.3:0/3390016021 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd41006a770 0x7fd41009d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:38.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.525+0000 7fd40effd700 1 --2- 172.21.15.3:0/3390016021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd41009dc30 0x7fd41009e080 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd404009870 tx=0x7fd404009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936f27947f64d14 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:38.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.525+0000 7fd40e7fc700 1 -- 172.21.15.3:0/3390016021 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd404011830 con 0x7fd41009dc30 2023-10-03T10:15:38.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.525+0000 7fd40e7fc700 1 -- 172.21.15.3:0/3390016021 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd404011990 con 0x7fd41009dc30 2023-10-03T10:15:38.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.525+0000 7fd40e7fc700 1 -- 172.21.15.3:0/3390016021 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd404011d20 con 0x7fd41009dc30 2023-10-03T10:15:38.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.526+0000 7fd415c63700 1 -- 172.21.15.3:0/3390016021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd41009dc30 msgr2=0x7fd41009e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:38.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.526+0000 7fd415c63700 1 --2- 172.21.15.3:0/3390016021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd41009dc30 0x7fd41009e080 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd404009870 tx=0x7fd404009b80 comp rx=0 tx=0).stop 2023-10-03T10:15:38.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.526+0000 7fd415c63700 1 -- 172.21.15.3:0/3390016021 shutdown_connections 2023-10-03T10:15:38.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.526+0000 7fd415c63700 1 --2- 172.21.15.3:0/3390016021 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd41006a770 0x7fd41009d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.526+0000 7fd415c63700 1 --2- 172.21.15.3:0/3390016021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd41009dc30 0x7fd41009e080 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.526+0000 7fd415c63700 1 -- 172.21.15.3:0/3390016021 >> 172.21.15.3:0/3390016021 conn(0x7fd410066f50 msgr2=0x7fd41009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:38.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.526+0000 7fd415c63700 1 -- 172.21.15.3:0/3390016021 shutdown_connections 2023-10-03T10:15:38.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.526+0000 7fd415c63700 1 -- 172.21.15.3:0/3390016021 wait complete. 2023-10-03T10:15:38.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.527+0000 7fd415c63700 1 Processor -- start 2023-10-03T10:15:38.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.528+0000 7fd415c63700 1 -- start start 2023-10-03T10:15:38.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.528+0000 7fd415c63700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd41006a770 0x7fd41011b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:38.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.528+0000 7fd415c63700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd41009dc30 0x7fd41013a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:38.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.528+0000 7fd415c63700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd41013c1e0 0x7fd41013aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:38.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.528+0000 7fd415c63700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4100a2d90 con 0x7fd41013c1e0 2023-10-03T10:15:38.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.528+0000 7fd415c63700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd41011ca30 con 0x7fd41006a770 2023-10-03T10:15:38.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.528+0000 7fd415c63700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd41011bf40 con 0x7fd41009dc30 2023-10-03T10:15:38.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.528+0000 7fd40f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd41006a770 0x7fd41011b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:38.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.528+0000 7fd40f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd41006a770 0x7fd41011b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50476/0 (socket says 172.21.15.3:50476) 2023-10-03T10:15:38.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.528+0000 7fd40f7fe700 1 -- 172.21.15.3:0/3279666160 learned_addr learned my addr 172.21.15.3:0/3279666160 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:38.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.528+0000 7fd40effd700 1 --2- 172.21.15.3:0/3279666160 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd41009dc30 0x7fd41013a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:38.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.529+0000 7fd40ffff700 1 --2- 172.21.15.3:0/3279666160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd41013c1e0 0x7fd41013aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:38.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.529+0000 7fd40effd700 1 -- 172.21.15.3:0/3279666160 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd41006a770 msgr2=0x7fd41011b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:38.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.529+0000 7fd40effd700 1 --2- 172.21.15.3:0/3279666160 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd41006a770 0x7fd41011b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.529+0000 7fd40effd700 1 -- 172.21.15.3:0/3279666160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd41013c1e0 msgr2=0x7fd41013aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:38.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.529+0000 7fd40effd700 1 --2- 172.21.15.3:0/3279666160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd41013c1e0 0x7fd41013aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.529+0000 7fd40effd700 1 -- 172.21.15.3:0/3279666160 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd400009550 con 0x7fd41009dc30 2023-10-03T10:15:38.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.529+0000 7fd40f7fe700 1 --2- 172.21.15.3:0/3279666160 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd41006a770 0x7fd41011b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:38.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.529+0000 7fd40effd700 1 --2- 172.21.15.3:0/3279666160 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd41009dc30 0x7fd41013a530 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd404016000 tx=0x7fd404004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:38.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.529+0000 7fd3f7fff700 1 -- 172.21.15.3:0/3279666160 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4040036a0 con 0x7fd41009dc30 2023-10-03T10:15:38.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.529+0000 7fd3f7fff700 1 -- 172.21.15.3:0/3279666160 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40401f070 con 0x7fd41009dc30 2023-10-03T10:15:38.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.529+0000 7fd3f7fff700 1 -- 172.21.15.3:0/3279666160 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd404003a40 con 0x7fd41009dc30 2023-10-03T10:15:38.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.529+0000 7fd415c63700 1 -- 172.21.15.3:0/3279666160 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4040094a0 con 0x7fd41009dc30 2023-10-03T10:15:38.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.530+0000 7fd415c63700 1 -- 172.21.15.3:0/3279666160 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41013bc30 con 0x7fd41009dc30 2023-10-03T10:15:38.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.531+0000 7fd3f7fff700 1 -- 172.21.15.3:0/3279666160 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd404003ba0 con 0x7fd41009dc30 2023-10-03T10:15:38.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.531+0000 7fd3f7fff700 1 -- 172.21.15.3:0/3279666160 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd404021070 con 0x7fd41009dc30 2023-10-03T10:15:38.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.531+0000 7fd415c63700 1 -- 172.21.15.3:0/3279666160 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd41009f1a0 con 0x7fd41009dc30 2023-10-03T10:15:38.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.537+0000 7fd3f7fff700 1 -- 172.21.15.3:0/3279666160 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd404015070 con 0x7fd41009dc30 2023-10-03T10:15:38.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.554+0000 7f570078a700 1 Processor -- start 2023-10-03T10:15:38.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.554+0000 7f2e8497c700 1 Processor -- start 2023-10-03T10:15:38.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.554+0000 7f570078a700 1 -- start start 2023-10-03T10:15:38.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.554+0000 7f2e8497c700 1 -- start start 2023-10-03T10:15:38.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.555+0000 7f570078a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56f809d6f0 0x7f56f809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:38.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.555+0000 7f2e8497c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e8009d6f0 0x7f2e8009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:38.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.555+0000 7f570078a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56f809e410 0x7f56f8130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:38.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.555+0000 7f570078a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f809f130 con 0x7f56f809e000 2023-10-03T10:15:38.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.555+0000 7f570078a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f809f520 con 0x7f56f809e410 2023-10-03T10:15:38.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.555+0000 7f570078a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56f809e930 con 0x7f56f809d6f0 2023-10-03T10:15:38.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.555+0000 7f56fe526700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56f809d6f0 0x7f56f809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:38.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.555+0000 7f56fdd25700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f56f809e000 0x7f56f8130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34408/0 (socket says 172.21.15.3:34408) 2023-10-03T10:15:38.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.555+0000 7f56fdd25700 1 -- 172.21.15.3:0/2481705775 learned_addr learned my addr 172.21.15.3:0/2481705775 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:38.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.555+0000 7f2e8497c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e8009e000 0x7f2e80130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:38.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.555+0000 7f2e8497c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2e8009e410 0x7f2e80130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:38.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.555+0000 7f2e8497c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e8009f520 con 0x7f2e8009e410 2023-10-03T10:15:38.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.555+0000 7f2e8497c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8009f130 con 0x7f2e8009d6f0 2023-10-03T10:15:38.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.555+0000 7f2e8497c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8009e930 con 0x7f2e8009e000 2023-10-03T10:15:38.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.555+0000 7f56fd524700 1 -- 172.21.15.3:0/2481705775 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4236144588 0 0) 0x7f56f809f130 con 0x7f56f809e000 2023-10-03T10:15:38.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.555+0000 7f56fd524700 1 -- 172.21.15.3:0/2481705775 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56e40035b0 con 0x7f56f809e000 2023-10-03T10:15:38.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.555+0000 7f56fed27700 1 --2- 172.21.15.3:0/2481705775 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56f809e410 0x7f56f8130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:38.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.555+0000 7f2e7dd9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e8009e000 0x7f2e80130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:38.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.555+0000 7f2e7dd9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e8009e000 0x7f2e80130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41942/0 (socket says 172.21.15.3:41942) 2023-10-03T10:15:38.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.555+0000 7f2e7dd9b700 1 -- 172.21.15.3:0/3018108651 learned_addr learned my addr 172.21.15.3:0/3018108651 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:38.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.555+0000 7f2e7ed9d700 1 --2- 172.21.15.3:0/3018108651 >> v2:172.21.15.3:3300/0 conn(0x7f2e8009e410 0x7f2e80130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:38.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.556+0000 7f56fe526700 1 -- 172.21.15.3:0/2481705775 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56f809e410 msgr2=0x7f56f8130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:38.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.556+0000 7f56fe526700 1 --2- 172.21.15.3:0/2481705775 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56f809e410 0x7f56f8130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.556+0000 7f56fe526700 1 -- 172.21.15.3:0/2481705775 >> v1:172.21.15.3:6789/0 conn(0x7f56f809e000 legacy=0x7f56f8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:38.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.556+0000 7f56fe526700 1 -- 172.21.15.3:0/2481705775 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f809b2d0 con 0x7f56f809d6f0 2023-10-03T10:15:38.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.556+0000 7f2e7e59c700 1 --2- 172.21.15.3:0/3018108651 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e8009d6f0 0x7f2e8009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:38.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.556+0000 7f2e7dd9b700 1 -- 172.21.15.3:0/3018108651 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e8009d6f0 msgr2=0x7f2e8009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:38.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.556+0000 7f2e7dd9b700 1 --2- 172.21.15.3:0/3018108651 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e8009d6f0 0x7f2e8009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.556+0000 7f2e7dd9b700 1 -- 172.21.15.3:0/3018108651 >> v2:172.21.15.3:3300/0 conn(0x7f2e8009e410 msgr2=0x7f2e80130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:38.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.556+0000 7f2e7dd9b700 1 --2- 172.21.15.3:0/3018108651 >> v2:172.21.15.3:3300/0 conn(0x7f2e8009e410 0x7f2e80130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.556+0000 7f2e7dd9b700 1 -- 172.21.15.3:0/3018108651 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8009b2d0 con 0x7f2e8009e000 2023-10-03T10:15:38.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.556+0000 7f56fe526700 1 --2- 172.21.15.3:0/2481705775 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56f809d6f0 0x7f56f809dac0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f56f4009870 tx=0x7f56f4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ad51fe5d0ee47a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:38.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.556+0000 7f2e7dd9b700 1 --2- 172.21.15.3:0/3018108651 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e8009e000 0x7f2e80130660 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2e7400b9d0 tx=0x7f2e7400bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5ea34536e7da905 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:38.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.556+0000 7f2e7d59a700 1 -- 172.21.15.3:0/3018108651 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7400cc20 con 0x7f2e8009e000 2023-10-03T10:15:38.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.556+0000 7f2e7d59a700 1 -- 172.21.15.3:0/3018108651 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e7400cd80 con 0x7f2e8009e000 2023-10-03T10:15:38.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.556+0000 7f2e7d59a700 1 -- 172.21.15.3:0/3018108651 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7400f680 con 0x7f2e8009e000 2023-10-03T10:15:38.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.556+0000 7f56fd524700 1 -- 172.21.15.3:0/2481705775 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f4004230 con 0x7f56f809d6f0 2023-10-03T10:15:38.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.556+0000 7f56fd524700 1 -- 172.21.15.3:0/2481705775 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f4004390 con 0x7f56f809d6f0 2023-10-03T10:15:38.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.556+0000 7f56fd524700 1 -- 172.21.15.3:0/2481705775 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f40036a0 con 0x7f56f809d6f0 2023-10-03T10:15:38.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.557+0000 7f2e8497c700 1 -- 172.21.15.3:0/3018108651 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e8009e000 msgr2=0x7f2e80130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:38.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.557+0000 7f570078a700 1 -- 172.21.15.3:0/2481705775 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56f809d6f0 msgr2=0x7f56f809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:38.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.557+0000 7f570078a700 1 --2- 172.21.15.3:0/2481705775 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56f809d6f0 0x7f56f809dac0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f56f4009870 tx=0x7f56f4009b80 comp rx=0 tx=0).stop 2023-10-03T10:15:38.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.557+0000 7f2e8497c700 1 --2- 172.21.15.3:0/3018108651 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e8009e000 0x7f2e80130660 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2e7400b9d0 tx=0x7f2e7400bce0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.557+0000 7f570078a700 1 -- 172.21.15.3:0/2481705775 shutdown_connections 2023-10-03T10:15:38.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.557+0000 7f570078a700 1 --2- 172.21.15.3:0/2481705775 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56f809e410 0x7f56f8130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.557+0000 7f570078a700 1 --2- 172.21.15.3:0/2481705775 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56f809d6f0 0x7f56f809dac0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.557+0000 7f570078a700 1 -- 172.21.15.3:0/2481705775 >> 172.21.15.3:0/2481705775 conn(0x7f56f8066f50 msgr2=0x7f56f809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.557+0000 7f2e8497c700 1 -- 172.21.15.3:0/3018108651 shutdown_connections 2023-10-03T10:15:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.557+0000 7f570078a700 1 -- 172.21.15.3:0/2481705775 shutdown_connections 2023-10-03T10:15:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.557+0000 7f2e8497c700 1 --2- 172.21.15.3:0/3018108651 >> v2:172.21.15.3:3300/0 conn(0x7f2e8009e410 0x7f2e80130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.557+0000 7f2e8497c700 1 --2- 172.21.15.3:0/3018108651 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e8009d6f0 0x7f2e8009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.557+0000 7f2e8497c700 1 --2- 172.21.15.3:0/3018108651 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e8009e000 0x7f2e80130660 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.557+0000 7f2e8497c700 1 -- 172.21.15.3:0/3018108651 >> 172.21.15.3:0/3018108651 conn(0x7f2e80066f50 msgr2=0x7f2e8009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:38.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.558+0000 7f2e8497c700 1 -- 172.21.15.3:0/3018108651 shutdown_connections 2023-10-03T10:15:38.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.558+0000 7f2e8497c700 1 -- 172.21.15.3:0/3018108651 wait complete. 2023-10-03T10:15:38.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.558+0000 7f570078a700 1 -- 172.21.15.3:0/2481705775 wait complete. 2023-10-03T10:15:38.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.559+0000 7f570078a700 1 Processor -- start 2023-10-03T10:15:38.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.559+0000 7f570078a700 1 -- start start 2023-10-03T10:15:38.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.559+0000 7f570078a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56f809d6f0 0x7f56f811df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:38.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.559+0000 7f570078a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56f809e000 0x7f56f811e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:38.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.559+0000 7f570078a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56f809e410 0x7f56f811e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:38.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.559+0000 7f570078a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f8121730 con 0x7f56f809e410 2023-10-03T10:15:38.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.559+0000 7f570078a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f8072150 con 0x7f56f809e000 2023-10-03T10:15:38.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.559+0000 7f570078a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56f811faa0 con 0x7f56f809d6f0 2023-10-03T10:15:38.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.559+0000 7f2e8497c700 1 Processor -- start 2023-10-03T10:15:38.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.559+0000 7f56fed27700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56f809e410 0x7f56f811e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:38.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.559+0000 7f56fed27700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56f809e410 0x7f56f811e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42182/0 (socket says 172.21.15.3:42182) 2023-10-03T10:15:38.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.559+0000 7f56fed27700 1 -- 172.21.15.3:0/2578996853 learned_addr learned my addr 172.21.15.3:0/2578996853 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:38.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.559+0000 7f56fe526700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56f809d6f0 0x7f56f811df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:38.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.559+0000 7f56fdd25700 1 --2- 172.21.15.3:0/2578996853 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56f809e000 0x7f56f811e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:38.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.560+0000 7f56fed27700 1 -- 172.21.15.3:0/2578996853 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56f809d6f0 msgr2=0x7f56f811df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:38.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.560+0000 7f56fed27700 1 --2- 172.21.15.3:0/2578996853 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56f809d6f0 0x7f56f811df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.560+0000 7f56fed27700 1 -- 172.21.15.3:0/2578996853 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56f809e000 msgr2=0x7f56f811e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:38.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.560+0000 7f56fed27700 1 --2- 172.21.15.3:0/2578996853 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56f809e000 0x7f56f811e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.560+0000 7f56fed27700 1 -- 172.21.15.3:0/2578996853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f40094a0 con 0x7f56f809e410 2023-10-03T10:15:38.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.560+0000 7f56fed27700 1 --2- 172.21.15.3:0/2578996853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56f809e410 0x7f56f811e9e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f56f0007e70 tx=0x7f56f000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:38.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.560+0000 7f56eeffd700 1 -- 172.21.15.3:0/2578996853 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f000ed30 con 0x7f56f809e410 2023-10-03T10:15:38.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.560+0000 7f56eeffd700 1 -- 172.21.15.3:0/2578996853 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f000eeb0 con 0x7f56f809e410 2023-10-03T10:15:38.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.560+0000 7f570078a700 1 -- 172.21.15.3:0/2578996853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f809c700 con 0x7f56f809e410 2023-10-03T10:15:38.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.560+0000 7f570078a700 1 -- 172.21.15.3:0/2578996853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f81201f0 con 0x7f56f809e410 2023-10-03T10:15:38.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.560+0000 7f56eeffd700 1 -- 172.21.15.3:0/2578996853 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f0014640 con 0x7f56f809e410 2023-10-03T10:15:38.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.560+0000 7f2e8497c700 1 -- start start 2023-10-03T10:15:38.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.560+0000 7f2e8497c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e8009d6f0 0x7f2e80127170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:38.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.560+0000 7f2e8497c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e8009e000 0x7f2e801276b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:38.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.560+0000 7f2e8497c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e8009e410 0x7f2e801cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:38.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.560+0000 7f2e8497c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8012a0d0 con 0x7f2e8009e410 2023-10-03T10:15:38.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.560+0000 7f2e8497c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e80128a50 con 0x7f2e8009e000 2023-10-03T10:15:38.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.560+0000 7f2e8497c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e801284a0 con 0x7f2e8009d6f0 2023-10-03T10:15:38.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.561+0000 7f2e7dd9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e8009e000 0x7f2e801276b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:38.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.561+0000 7f2e7dd9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e8009e000 0x7f2e801276b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50516/0 (socket says 172.21.15.3:50516) 2023-10-03T10:15:38.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.561+0000 7f2e7dd9b700 1 -- 172.21.15.3:0/744776437 learned_addr learned my addr 172.21.15.3:0/744776437 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:38.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.561+0000 7f2e7dd9b700 1 -- 172.21.15.3:0/744776437 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e8009d6f0 msgr2=0x7f2e80127170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:15:38.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.561+0000 7f2e7dd9b700 1 --2- 172.21.15.3:0/744776437 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e8009d6f0 0x7f2e80127170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.561+0000 7f2e7dd9b700 1 -- 172.21.15.3:0/744776437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e8009e410 msgr2=0x7f2e801cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:38.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.561+0000 7f2e7dd9b700 1 --2- 172.21.15.3:0/744776437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e8009e410 0x7f2e801cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.561+0000 7f2e7dd9b700 1 -- 172.21.15.3:0/744776437 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7400b600 con 0x7f2e8009e000 2023-10-03T10:15:38.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.561+0000 7f2e7dd9b700 1 --2- 172.21.15.3:0/744776437 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e8009e000 0x7f2e801276b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2e74000c00 tx=0x7f2e74004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:38.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.562+0000 7f570078a700 1 -- 172.21.15.3:0/2578996853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e00052e0 con 0x7f56f809e410 2023-10-03T10:15:38.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.562+0000 7f2e6effd700 1 -- 172.21.15.3:0/744776437 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7401c070 con 0x7f2e8009e000 2023-10-03T10:15:38.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.562+0000 7f2e6effd700 1 -- 172.21.15.3:0/744776437 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e7400be70 con 0x7f2e8009e000 2023-10-03T10:15:38.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.562+0000 7f2e8497c700 1 -- 172.21.15.3:0/744776437 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e80128c90 con 0x7f2e8009e000 2023-10-03T10:15:38.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.562+0000 7f2e6effd700 1 -- 172.21.15.3:0/744776437 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e74022420 con 0x7f2e8009e000 2023-10-03T10:15:38.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.562+0000 7f2e8497c700 1 -- 172.21.15.3:0/744776437 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e80126ca0 con 0x7f2e8009e000 2023-10-03T10:15:38.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.563+0000 7f56eeffd700 1 -- 172.21.15.3:0/2578996853 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f56f00147a0 con 0x7f56f809e410 2023-10-03T10:15:38.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.563+0000 7f2e6effd700 1 -- 172.21.15.3:0/744776437 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2e74020070 con 0x7f2e8009e000 2023-10-03T10:15:38.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.563+0000 7f2e8497c700 1 -- 172.21.15.3:0/744776437 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e80074ea0 con 0x7f2e8009e000 2023-10-03T10:15:38.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.563+0000 7f2e6effd700 1 -- 172.21.15.3:0/744776437 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2e74040b20 con 0x7f2e8009e000 2023-10-03T10:15:38.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.572+0000 7f56eeffd700 1 -- 172.21.15.3:0/2578996853 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f56f0013df0 con 0x7f56f809e410 2023-10-03T10:15:38.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.572+0000 7f56eeffd700 1 -- 172.21.15.3:0/2578996853 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f56f0014e30 con 0x7f56f809e410 2023-10-03T10:15:38.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.572+0000 7f2e6effd700 1 -- 172.21.15.3:0/744776437 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2e74017400 con 0x7f2e8009e000 2023-10-03T10:15:38.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.631+0000 7f814d048700 1 Processor -- start 2023-10-03T10:15:38.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.631+0000 7f814d048700 1 -- start start 2023-10-03T10:15:38.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.632+0000 7f814d048700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f814809f410 0x7f8148136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:38.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.632+0000 7f814d048700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8148136fe0 0x7f81481393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:38.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.632+0000 7f814d048700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8148070940 con 0x7f814809e930 2023-10-03T10:15:38.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.632+0000 7f814d048700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f814806fd60 con 0x7f814809f410 2023-10-03T10:15:38.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.632+0000 7f814d048700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814806fa80 con 0x7f8148136fe0 2023-10-03T10:15:38.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.632+0000 7f814759e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8148136fe0 0x7f81481393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:38.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.632+0000 7f814759e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8148136fe0 0x7f81481393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50522/0 (socket says 172.21.15.3:50522) 2023-10-03T10:15:38.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.632+0000 7f814759e700 1 -- 172.21.15.3:0/1116351134 learned_addr learned my addr 172.21.15.3:0/1116351134 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:38.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.633+0000 7f814659c700 1 --2- 172.21.15.3:0/1116351134 >> v2:172.21.15.3:3300/0 conn(0x7f814809f410 0x7f8148136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:38.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.633+0000 7f814759e700 1 -- 172.21.15.3:0/1116351134 >> v2:172.21.15.3:3300/0 conn(0x7f814809f410 msgr2=0x7f8148136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:38.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.633+0000 7f8145d9b700 1 -- 172.21.15.3:0/1116351134 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592261790 0 0) 0x7f8148070940 con 0x7f814809e930 2023-10-03T10:15:38.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.633+0000 7f814759e700 1 --2- 172.21.15.3:0/1116351134 >> v2:172.21.15.3:3300/0 conn(0x7f814809f410 0x7f8148136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.633+0000 7f814759e700 1 -- 172.21.15.3:0/1116351134 >> v1:172.21.15.3:6789/0 conn(0x7f814809e930 legacy=0x7f814809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:38.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.633+0000 7f814759e700 1 -- 172.21.15.3:0/1116351134 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814809fb60 con 0x7f8148136fe0 2023-10-03T10:15:38.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.633+0000 7f814659c700 1 --2- 172.21.15.3:0/1116351134 >> v2:172.21.15.3:3300/0 conn(0x7f814809f410 0x7f8148136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:15:38.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.633+0000 7f814759e700 1 --2- 172.21.15.3:0/1116351134 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8148136fe0 0x7f81481393c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8140009e50 tx=0x7f814000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ba4873bcec339c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:38.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.633+0000 7f8145d9b700 1 -- 172.21.15.3:0/1116351134 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814000e070 con 0x7f8148136fe0 2023-10-03T10:15:38.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.633+0000 7f8145d9b700 1 -- 172.21.15.3:0/1116351134 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814000b9c0 con 0x7f8148136fe0 2023-10-03T10:15:38.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.633+0000 7f8145d9b700 1 -- 172.21.15.3:0/1116351134 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814000bd70 con 0x7f8148136fe0 2023-10-03T10:15:38.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.634+0000 7f814d048700 1 -- 172.21.15.3:0/1116351134 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8148136fe0 msgr2=0x7f81481393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:38.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.634+0000 7f814d048700 1 --2- 172.21.15.3:0/1116351134 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8148136fe0 0x7f81481393c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8140009e50 tx=0x7f814000b330 comp rx=0 tx=0).stop 2023-10-03T10:15:38.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.634+0000 7f814d048700 1 -- 172.21.15.3:0/1116351134 shutdown_connections 2023-10-03T10:15:38.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.634+0000 7f814d048700 1 --2- 172.21.15.3:0/1116351134 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8148136fe0 0x7f81481393c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.634+0000 7f814d048700 1 --2- 172.21.15.3:0/1116351134 >> v2:172.21.15.3:3300/0 conn(0x7f814809f410 0x7f8148136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.634+0000 7f814d048700 1 -- 172.21.15.3:0/1116351134 >> 172.21.15.3:0/1116351134 conn(0x7f81480a4210 msgr2=0x7f81480a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:38.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.634+0000 7f814d048700 1 -- 172.21.15.3:0/1116351134 shutdown_connections 2023-10-03T10:15:38.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.634+0000 7f814d048700 1 -- 172.21.15.3:0/1116351134 wait complete. 2023-10-03T10:15:38.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.635+0000 7f814d048700 1 Processor -- start 2023-10-03T10:15:38.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.635+0000 7f814d048700 1 -- start start 2023-10-03T10:15:38.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.636+0000 7f814d048700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f814809e930 0x7f814811f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:38.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.636+0000 7f814d048700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f814809f410 0x7f814811fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:38.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.636+0000 7f814d048700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8148136fe0 0x7f8148120040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:38.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.636+0000 7f814d048700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8148123260 con 0x7f814809e930 2023-10-03T10:15:38.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.636+0000 7f814d048700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8148071d20 con 0x7f8148136fe0 2023-10-03T10:15:38.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.636+0000 7f814d048700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81481211d0 con 0x7f814809f410 2023-10-03T10:15:38.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.636+0000 7f8146d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f814809e930 0x7f814811f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:38.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.636+0000 7f8146d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f814809e930 0x7f814811f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42206/0 (socket says 172.21.15.3:42206) 2023-10-03T10:15:38.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.636+0000 7f8146d9d700 1 -- 172.21.15.3:0/1525542762 learned_addr learned my addr 172.21.15.3:0/1525542762 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:38.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.637+0000 7f814759e700 1 --2- 172.21.15.3:0/1525542762 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8148136fe0 0x7f8148120040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:38.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.637+0000 7f814659c700 1 --2- 172.21.15.3:0/1525542762 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f814809f410 0x7f814811fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:38.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.637+0000 7f8146d9d700 1 -- 172.21.15.3:0/1525542762 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f814809f410 msgr2=0x7f814811fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:38.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.637+0000 7f8146d9d700 1 --2- 172.21.15.3:0/1525542762 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f814809f410 0x7f814811fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.637+0000 7f8146d9d700 1 -- 172.21.15.3:0/1525542762 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8148136fe0 msgr2=0x7f8148120040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:38.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.637+0000 7f8146d9d700 1 --2- 172.21.15.3:0/1525542762 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8148136fe0 0x7f8148120040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.637+0000 7f8146d9d700 1 -- 172.21.15.3:0/1525542762 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8140009b30 con 0x7f814809e930 2023-10-03T10:15:38.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.637+0000 7f814759e700 1 --2- 172.21.15.3:0/1525542762 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8148136fe0 0x7f8148120040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:38.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.637+0000 7f8146d9d700 1 --2- 172.21.15.3:0/1525542762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f814809e930 0x7f814811f5c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8138001eb0 tx=0x7f813800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:38.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.637+0000 7f812f7fe700 1 -- 172.21.15.3:0/1525542762 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81380054e0 con 0x7f814809e930 2023-10-03T10:15:38.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.637+0000 7f814d048700 1 -- 172.21.15.3:0/1525542762 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81480a31d0 con 0x7f814809e930 2023-10-03T10:15:38.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.638+0000 7f812f7fe700 1 -- 172.21.15.3:0/1525542762 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8138003680 con 0x7f814809e930 2023-10-03T10:15:38.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.638+0000 7f814d048700 1 -- 172.21.15.3:0/1525542762 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8148122890 con 0x7f814809e930 2023-10-03T10:15:38.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.638+0000 7f812f7fe700 1 -- 172.21.15.3:0/1525542762 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8138003a30 con 0x7f814809e930 2023-10-03T10:15:38.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.639+0000 7f814d048700 1 -- 172.21.15.3:0/1525542762 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81280052e0 con 0x7f814809e930 2023-10-03T10:15:38.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.640+0000 7f812f7fe700 1 -- 172.21.15.3:0/1525542762 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8138003b90 con 0x7f814809e930 2023-10-03T10:15:38.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.640+0000 7f812f7fe700 1 -- 172.21.15.3:0/1525542762 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f813803eb50 con 0x7f814809e930 2023-10-03T10:15:38.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.645+0000 7f812f7fe700 1 -- 172.21.15.3:0/1525542762 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f813801b3b0 con 0x7f814809e930 2023-10-03T10:15:38.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.785+0000 7fd415c63700 1 -- 172.21.15.3:0/3279666160 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd410074ea0 con 0x7fd41009dc30 2023-10-03T10:15:38.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.786+0000 7fd3f7fff700 1 -- 172.21.15.3:0/3279666160 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd40402b030 con 0x7fd41009dc30 2023-10-03T10:15:38.786 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:38.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.788+0000 7fd415c63700 1 -- 172.21.15.3:0/3279666160 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd41009dc30 msgr2=0x7fd41013a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:38.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.788+0000 7fd415c63700 1 --2- 172.21.15.3:0/3279666160 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd41009dc30 0x7fd41013a530 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd404016000 tx=0x7fd404004750 comp rx=0 tx=0).stop 2023-10-03T10:15:38.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.788+0000 7fd415c63700 1 -- 172.21.15.3:0/3279666160 shutdown_connections 2023-10-03T10:15:38.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.788+0000 7fd415c63700 1 --2- 172.21.15.3:0/3279666160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd41013c1e0 0x7fd41013aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.788+0000 7fd415c63700 1 --2- 172.21.15.3:0/3279666160 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd41006a770 0x7fd41011b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.788+0000 7fd415c63700 1 --2- 172.21.15.3:0/3279666160 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd41009dc30 0x7fd41013a530 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.788+0000 7fd415c63700 1 -- 172.21.15.3:0/3279666160 >> 172.21.15.3:0/3279666160 conn(0x7fd410066f50 msgr2=0x7fd410070c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:38.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.788+0000 7fd415c63700 1 -- 172.21.15.3:0/3279666160 shutdown_connections 2023-10-03T10:15:38.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.788+0000 7fd415c63700 1 -- 172.21.15.3:0/3279666160 wait complete. 2023-10-03T10:15:38.803 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:38.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.811+0000 7f2e8497c700 1 -- 172.21.15.3:0/744776437 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2e80129d70 con 0x7f2e8009e000 2023-10-03T10:15:38.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.812+0000 7f2e6effd700 1 -- 172.21.15.3:0/744776437 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2e80129d70 con 0x7f2e8009e000 2023-10-03T10:15:38.812 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:38.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.813+0000 7f570078a700 1 -- 172.21.15.3:0/2578996853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f56e0005860 con 0x7f56f809e410 2023-10-03T10:15:38.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.814+0000 7f56eeffd700 1 -- 172.21.15.3:0/2578996853 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f56f002c090 con 0x7f56f809e410 2023-10-03T10:15:38.814 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:38.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.814+0000 7f2e8497c700 1 -- 172.21.15.3:0/744776437 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e8009e000 msgr2=0x7f2e801276b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:38.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.814+0000 7f2e8497c700 1 --2- 172.21.15.3:0/744776437 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e8009e000 0x7f2e801276b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2e74000c00 tx=0x7f2e74004750 comp rx=0 tx=0).stop 2023-10-03T10:15:38.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.814+0000 7f2e8497c700 1 -- 172.21.15.3:0/744776437 shutdown_connections 2023-10-03T10:15:38.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.814+0000 7f2e8497c700 1 --2- 172.21.15.3:0/744776437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e8009e410 0x7f2e801cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.814+0000 7f2e8497c700 1 --2- 172.21.15.3:0/744776437 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e8009d6f0 0x7f2e80127170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.814+0000 7f2e8497c700 1 --2- 172.21.15.3:0/744776437 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e8009e000 0x7f2e801276b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.814+0000 7f2e8497c700 1 -- 172.21.15.3:0/744776437 >> 172.21.15.3:0/744776437 conn(0x7f2e80066f50 msgr2=0x7f2e800a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:38.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.814+0000 7f2e8497c700 1 -- 172.21.15.3:0/744776437 shutdown_connections 2023-10-03T10:15:38.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.814+0000 7f2e8497c700 1 -- 172.21.15.3:0/744776437 wait complete. 2023-10-03T10:15:38.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.816+0000 7f570078a700 1 -- 172.21.15.3:0/2578996853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56f809e410 msgr2=0x7f56f811e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:38.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.816+0000 7f570078a700 1 --2- 172.21.15.3:0/2578996853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56f809e410 0x7f56f811e9e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f56f0007e70 tx=0x7f56f000e3c0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.816+0000 7f570078a700 1 -- 172.21.15.3:0/2578996853 shutdown_connections 2023-10-03T10:15:38.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.816+0000 7f570078a700 1 --2- 172.21.15.3:0/2578996853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56f809e410 0x7f56f811e9e0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.816+0000 7f570078a700 1 --2- 172.21.15.3:0/2578996853 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56f809d6f0 0x7f56f811df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.816+0000 7f570078a700 1 --2- 172.21.15.3:0/2578996853 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56f809e000 0x7f56f811e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.816+0000 7f570078a700 1 -- 172.21.15.3:0/2578996853 >> 172.21.15.3:0/2578996853 conn(0x7f56f8066f50 msgr2=0x7f56f8070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:38.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.816+0000 7f570078a700 1 -- 172.21.15.3:0/2578996853 shutdown_connections 2023-10-03T10:15:38.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.816+0000 7f570078a700 1 -- 172.21.15.3:0/2578996853 wait complete. 2023-10-03T10:15:38.830 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:38.831 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:38.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.856+0000 7f814d048700 1 -- 172.21.15.3:0/1525542762 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8128005590 con 0x7f814809e930 2023-10-03T10:15:38.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.856+0000 7f812f7fe700 1 -- 172.21.15.3:0/1525542762 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8138017020 con 0x7f814809e930 2023-10-03T10:15:38.856 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:38.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.858+0000 7f814d048700 1 -- 172.21.15.3:0/1525542762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f814809e930 msgr2=0x7f814811f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:38.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.858+0000 7f814d048700 1 --2- 172.21.15.3:0/1525542762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f814809e930 0x7f814811f5c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8138001eb0 tx=0x7f813800c800 comp rx=0 tx=0).stop 2023-10-03T10:15:38.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.858+0000 7f814d048700 1 -- 172.21.15.3:0/1525542762 shutdown_connections 2023-10-03T10:15:38.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.858+0000 7f814d048700 1 --2- 172.21.15.3:0/1525542762 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8148136fe0 0x7f8148120040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.858+0000 7f814d048700 1 --2- 172.21.15.3:0/1525542762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f814809e930 0x7f814811f5c0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.858+0000 7f814d048700 1 --2- 172.21.15.3:0/1525542762 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f814809f410 0x7f814811fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:38.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.858+0000 7f814d048700 1 -- 172.21.15.3:0/1525542762 >> 172.21.15.3:0/1525542762 conn(0x7f81480a4210 msgr2=0x7f814809b430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:38.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.858+0000 7f814d048700 1 -- 172.21.15.3:0/1525542762 shutdown_connections 2023-10-03T10:15:38.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:38.858+0000 7f814d048700 1 -- 172.21.15.3:0/1525542762 wait complete. 2023-10-03T10:15:38.872 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:39.803 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:39.831 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:39.832 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:39.872 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:39.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.965+0000 7f09e716d700 1 Processor -- start 2023-10-03T10:15:39.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.965+0000 7f09e716d700 1 -- start start 2023-10-03T10:15:39.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.966+0000 7f09e716d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f09e009ffb0 0x7f09e009f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:39.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.966+0000 7f09e716d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09e0138e30 0x7f09e013b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:39.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.966+0000 7f09e716d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09e00a06b0 con 0x7f09e009e850 2023-10-03T10:15:39.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.966+0000 7f09e716d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e006b920 con 0x7f09e0138e30 2023-10-03T10:15:39.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.966+0000 7f09e716d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f09e009aa30 con 0x7f09e009ffb0 2023-10-03T10:15:39.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.966+0000 7f09dffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f09e009ffb0 0x7f09e009f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:39.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.966+0000 7f09e4f09700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f09e009e850 0x7f09e009ec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34434/0 (socket says 172.21.15.3:34434) 2023-10-03T10:15:39.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.966+0000 7f09e4f09700 1 -- 172.21.15.3:0/645046750 learned_addr learned my addr 172.21.15.3:0/645046750 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:39.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.966+0000 7f09e570a700 1 --2- 172.21.15.3:0/645046750 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09e0138e30 0x7f09e013b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:39.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.967+0000 7f09e570a700 1 -- 172.21.15.3:0/645046750 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f09e009ffb0 msgr2=0x7f09e009f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:39.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.967+0000 7f09e570a700 1 --2- 172.21.15.3:0/645046750 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f09e009ffb0 0x7f09e009f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:39.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.967+0000 7f09e570a700 1 -- 172.21.15.3:0/645046750 >> v1:172.21.15.3:6789/0 conn(0x7f09e009e850 legacy=0x7f09e009ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:39.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.967+0000 7f09e570a700 1 -- 172.21.15.3:0/645046750 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e006caa0 con 0x7f09e0138e30 2023-10-03T10:15:39.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.967+0000 7f09e570a700 1 --2- 172.21.15.3:0/645046750 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09e0138e30 0x7f09e013b210 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f09d8009dc0 tx=0x7f09d800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc1aff18f05afa7d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:39.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.967+0000 7f09df7fe700 1 -- 172.21.15.3:0/645046750 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d800e070 con 0x7f09e0138e30 2023-10-03T10:15:39.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.967+0000 7f09df7fe700 1 -- 172.21.15.3:0/645046750 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d800bb20 con 0x7f09e0138e30 2023-10-03T10:15:39.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.967+0000 7f09df7fe700 1 -- 172.21.15.3:0/645046750 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d8012430 con 0x7f09e0138e30 2023-10-03T10:15:39.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.968+0000 7f09e716d700 1 -- 172.21.15.3:0/645046750 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09e0138e30 msgr2=0x7f09e013b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:39.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.968+0000 7f09e716d700 1 --2- 172.21.15.3:0/645046750 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09e0138e30 0x7f09e013b210 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f09d8009dc0 tx=0x7f09d800b330 comp rx=0 tx=0).stop 2023-10-03T10:15:39.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.968+0000 7f09e716d700 1 -- 172.21.15.3:0/645046750 shutdown_connections 2023-10-03T10:15:39.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.968+0000 7f09e716d700 1 --2- 172.21.15.3:0/645046750 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09e0138e30 0x7f09e013b210 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:39.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.968+0000 7f09e716d700 1 --2- 172.21.15.3:0/645046750 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f09e009ffb0 0x7f09e009f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:39.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.968+0000 7f09e716d700 1 -- 172.21.15.3:0/645046750 >> 172.21.15.3:0/645046750 conn(0x7f09e00a3f70 msgr2=0x7f09e00a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:39.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.968+0000 7f09e716d700 1 -- 172.21.15.3:0/645046750 shutdown_connections 2023-10-03T10:15:39.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.968+0000 7f09e716d700 1 -- 172.21.15.3:0/645046750 wait complete. 2023-10-03T10:15:39.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.969+0000 7f09e716d700 1 Processor -- start 2023-10-03T10:15:39.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.969+0000 7f09e716d700 1 -- start start 2023-10-03T10:15:39.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.970+0000 7f09e716d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f09e009e850 0x7f09e0120a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:39.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.970+0000 7f09e716d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09e009ffb0 0x7f09e0120f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:39.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.970+0000 7f09e716d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09e0138e30 0x7f09e0121480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:39.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.970+0000 7f09e716d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e0124060 con 0x7f09e0138e30 2023-10-03T10:15:39.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.970+0000 7f09e716d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e01382c0 con 0x7f09e009ffb0 2023-10-03T10:15:39.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.970+0000 7f09e716d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f09e0124240 con 0x7f09e009e850 2023-10-03T10:15:39.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.970+0000 7f09dffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09e009ffb0 0x7f09e0120f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:39.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.970+0000 7f09dffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09e009ffb0 0x7f09e0120f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50562/0 (socket says 172.21.15.3:50562) 2023-10-03T10:15:39.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.970+0000 7f09dffff700 1 -- 172.21.15.3:0/2586199597 learned_addr learned my addr 172.21.15.3:0/2586199597 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:39.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.970+0000 7f09e4f09700 1 --2- 172.21.15.3:0/2586199597 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f09e009e850 0x7f09e0120a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:39.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.970+0000 7f09e570a700 1 --2- 172.21.15.3:0/2586199597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09e0138e30 0x7f09e0121480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:39.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.971+0000 7f09dffff700 1 -- 172.21.15.3:0/2586199597 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f09e009e850 msgr2=0x7f09e0120a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:39.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.971+0000 7f09dffff700 1 --2- 172.21.15.3:0/2586199597 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f09e009e850 0x7f09e0120a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:39.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.971+0000 7f09dffff700 1 -- 172.21.15.3:0/2586199597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09e0138e30 msgr2=0x7f09e0121480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:39.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.971+0000 7f09dffff700 1 --2- 172.21.15.3:0/2586199597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09e0138e30 0x7f09e0121480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:39.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.971+0000 7f09dffff700 1 -- 172.21.15.3:0/2586199597 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d8009a70 con 0x7f09e009ffb0 2023-10-03T10:15:39.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.971+0000 7f09e570a700 1 --2- 172.21.15.3:0/2586199597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09e0138e30 0x7f09e0121480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:15:39.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.971+0000 7f09e4f09700 1 --2- 172.21.15.3:0/2586199597 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f09e009e850 0x7f09e0120a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:15:39.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.971+0000 7f09dffff700 1 --2- 172.21.15.3:0/2586199597 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09e009ffb0 0x7f09e0120f40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00ea10 tx=0x7f09cc00edd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:39.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.971+0000 7f09dd7fa700 1 -- 172.21.15.3:0/2586199597 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc009710 con 0x7f09e009ffb0 2023-10-03T10:15:39.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.971+0000 7f09dd7fa700 1 -- 172.21.15.3:0/2586199597 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09cc00c910 con 0x7f09e009ffb0 2023-10-03T10:15:39.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.971+0000 7f09e716d700 1 -- 172.21.15.3:0/2586199597 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e0123520 con 0x7f09e009ffb0 2023-10-03T10:15:39.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.971+0000 7f09dd7fa700 1 -- 172.21.15.3:0/2586199597 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc00cc80 con 0x7f09e009ffb0 2023-10-03T10:15:39.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.971+0000 7f09e716d700 1 -- 172.21.15.3:0/2586199597 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e00a3530 con 0x7f09e009ffb0 2023-10-03T10:15:39.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.972+0000 7f09dd7fa700 1 -- 172.21.15.3:0/2586199597 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f09cc013470 con 0x7f09e009ffb0 2023-10-03T10:15:39.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.972+0000 7f09dd7fa700 1 -- 172.21.15.3:0/2586199597 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f09cc03fbc0 con 0x7f09e009ffb0 2023-10-03T10:15:39.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.973+0000 7f09e716d700 1 -- 172.21.15.3:0/2586199597 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e0123520 con 0x7f09e009ffb0 2023-10-03T10:15:39.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.982+0000 7f09dd7fa700 1 -- 172.21.15.3:0/2586199597 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f09e0123520 con 0x7f09e009ffb0 2023-10-03T10:15:39.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.996+0000 7f5eff751700 1 Processor -- start 2023-10-03T10:15:39.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.996+0000 7f5eff751700 1 -- start start 2023-10-03T10:15:39.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.996+0000 7f5eff751700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5ef806a770 0x7f5ef809d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:39.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.996+0000 7f5eff751700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ef809dc30 0x7f5ef809e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:39.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.996+0000 7f5eff751700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ef813c1e0 0x7f5ef813e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:39.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.996+0000 7f5eff751700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef806ab40 con 0x7f5ef806a770 2023-10-03T10:15:39.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.996+0000 7f5eff751700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef809ebc0 con 0x7f5ef809dc30 2023-10-03T10:15:39.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.996+0000 7f5eff751700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef809a810 con 0x7f5ef813c1e0 2023-10-03T10:15:39.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.997+0000 7f5efdcee700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ef813c1e0 0x7f5ef813e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:39.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.997+0000 7f5efd4ed700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5ef806a770 0x7f5ef809d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:39.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.997+0000 7f5efdcee700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ef813c1e0 0x7f5ef813e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41984/0 (socket says 172.21.15.3:41984) 2023-10-03T10:15:39.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.997+0000 7f5efd4ed700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5ef806a770 0x7f5ef809d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42228/0 (socket says 172.21.15.3:42228) 2023-10-03T10:15:39.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.997+0000 7f5efdcee700 1 -- 172.21.15.3:0/3615306127 learned_addr learned my addr 172.21.15.3:0/3615306127 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:39.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.997+0000 7f5efccec700 1 --2- 172.21.15.3:0/3615306127 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ef809dc30 0x7f5ef809e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:39.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.997+0000 7f5efccec700 1 -- 172.21.15.3:0/3615306127 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ef813c1e0 msgr2=0x7f5ef813e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:40.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.997+0000 7f5efccec700 1 --2- 172.21.15.3:0/3615306127 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ef813c1e0 0x7f5ef813e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.997+0000 7f5efccec700 1 -- 172.21.15.3:0/3615306127 >> v2:172.21.15.3:3300/0 conn(0x7f5ef806a770 msgr2=0x7f5ef809d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:40.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.997+0000 7f5efccec700 1 --2- 172.21.15.3:0/3615306127 >> v2:172.21.15.3:3300/0 conn(0x7f5ef806a770 0x7f5ef809d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.997+0000 7f5efccec700 1 -- 172.21.15.3:0/3615306127 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef809e920 con 0x7f5ef809dc30 2023-10-03T10:15:40.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.997+0000 7f5efccec700 1 --2- 172.21.15.3:0/3615306127 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ef809dc30 0x7f5ef809e080 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8009840 tx=0x7f5ee8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b53085203441409 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:40.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.998+0000 7f5eeffff700 1 -- 172.21.15.3:0/3615306127 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8011830 con 0x7f5ef809dc30 2023-10-03T10:15:40.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.998+0000 7f5eeffff700 1 -- 172.21.15.3:0/3615306127 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee8011990 con 0x7f5ef809dc30 2023-10-03T10:15:40.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.998+0000 7f5eeffff700 1 -- 172.21.15.3:0/3615306127 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8011d20 con 0x7f5ef809dc30 2023-10-03T10:15:40.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.998+0000 7f5eff751700 1 -- 172.21.15.3:0/3615306127 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ef809dc30 msgr2=0x7f5ef809e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:40.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.998+0000 7f5eff751700 1 --2- 172.21.15.3:0/3615306127 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ef809dc30 0x7f5ef809e080 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8009840 tx=0x7f5ee8009b50 comp rx=0 tx=0).stop 2023-10-03T10:15:40.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.998+0000 7f5eff751700 1 -- 172.21.15.3:0/3615306127 shutdown_connections 2023-10-03T10:15:40.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.998+0000 7f5eff751700 1 --2- 172.21.15.3:0/3615306127 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ef813c1e0 0x7f5ef813e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.999+0000 7f5eff751700 1 --2- 172.21.15.3:0/3615306127 >> v2:172.21.15.3:3300/0 conn(0x7f5ef806a770 0x7f5ef809d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.999+0000 7f5eff751700 1 --2- 172.21.15.3:0/3615306127 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ef809dc30 0x7f5ef809e080 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.999+0000 7f5eff751700 1 -- 172.21.15.3:0/3615306127 >> 172.21.15.3:0/3615306127 conn(0x7f5ef8066f50 msgr2=0x7f5ef809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:40.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.999+0000 7f5eff751700 1 -- 172.21.15.3:0/3615306127 shutdown_connections 2023-10-03T10:15:40.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:39.999+0000 7f5eff751700 1 -- 172.21.15.3:0/3615306127 wait complete. 2023-10-03T10:15:40.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.000+0000 7f5eff751700 1 Processor -- start 2023-10-03T10:15:40.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.000+0000 7f5eff751700 1 -- start start 2023-10-03T10:15:40.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.001+0000 7f5eff751700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ef806a770 0x7f5ef813b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:40.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.001+0000 7f5eff751700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ef809dc30 0x7f5ef811d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:40.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.001+0000 7f5eff751700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ef813c1e0 0x7f5ef811db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:40.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.001+0000 7f5eff751700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef8072510 con 0x7f5ef813c1e0 2023-10-03T10:15:40.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.001+0000 7f5eff751700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef811fcb0 con 0x7f5ef809dc30 2023-10-03T10:15:40.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.001+0000 7f5eff751700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef80a3800 con 0x7f5ef806a770 2023-10-03T10:15:40.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.001+0000 7f5efccec700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ef809dc30 0x7f5ef811d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:40.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.001+0000 7f5efccec700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ef809dc30 0x7f5ef811d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50584/0 (socket says 172.21.15.3:50584) 2023-10-03T10:15:40.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.001+0000 7f5efccec700 1 -- 172.21.15.3:0/745258874 learned_addr learned my addr 172.21.15.3:0/745258874 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:40.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.001+0000 7f5efd4ed700 1 --2- 172.21.15.3:0/745258874 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ef806a770 0x7f5ef813b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:40.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.001+0000 7f5efccec700 1 -- 172.21.15.3:0/745258874 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ef806a770 msgr2=0x7f5ef813b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:40.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.001+0000 7f5efccec700 1 --2- 172.21.15.3:0/745258874 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ef806a770 0x7f5ef813b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.001+0000 7f5efccec700 1 -- 172.21.15.3:0/745258874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ef813c1e0 msgr2=0x7f5ef811db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:40.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.001+0000 7f5efccec700 1 --2- 172.21.15.3:0/745258874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ef813c1e0 0x7f5ef811db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.001+0000 7f5efccec700 1 -- 172.21.15.3:0/745258874 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee80094a0 con 0x7f5ef809dc30 2023-10-03T10:15:40.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.001+0000 7f5efd4ed700 1 --2- 172.21.15.3:0/745258874 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ef806a770 0x7f5ef813b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:15:40.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.002+0000 7f5efccec700 1 --2- 172.21.15.3:0/745258874 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ef809dc30 0x7f5ef811d5e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8009fd0 tx=0x7f5ee8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:40.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.002+0000 7f5eedffb700 1 -- 172.21.15.3:0/745258874 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8004510 con 0x7f5ef809dc30 2023-10-03T10:15:40.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.002+0000 7f5eedffb700 1 -- 172.21.15.3:0/745258874 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee801f070 con 0x7f5ef809dc30 2023-10-03T10:15:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.002+0000 7f5eff751700 1 -- 172.21.15.3:0/745258874 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef809f1a0 con 0x7f5ef809dc30 2023-10-03T10:15:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.002+0000 7f5eedffb700 1 -- 172.21.15.3:0/745258874 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee80038a0 con 0x7f5ef809dc30 2023-10-03T10:15:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.002+0000 7f5eff751700 1 -- 172.21.15.3:0/745258874 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef811ff90 con 0x7f5ef809dc30 2023-10-03T10:15:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.003+0000 7f8de5c17700 1 Processor -- start 2023-10-03T10:15:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.003+0000 7f8de5c17700 1 -- start start 2023-10-03T10:15:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.003+0000 7f8de5c17700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8de006a770 0x7f8de009d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.003+0000 7f8de5c17700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8de009dc30 0x7f8de009e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:40.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.003+0000 7f8de5c17700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8de013c1e0 0x7f8de013e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:40.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.003+0000 7f8de5c17700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8de006ab40 con 0x7f8de009dc30 2023-10-03T10:15:40.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.003+0000 7f8de5c17700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de009ebc0 con 0x7f8de006a770 2023-10-03T10:15:40.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.003+0000 7f8de5c17700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8de009a810 con 0x7f8de013c1e0 2023-10-03T10:15:40.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.004+0000 7f5eff751700 1 -- 172.21.15.3:0/745258874 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef8074ea0 con 0x7f5ef809dc30 2023-10-03T10:15:40.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.004+0000 7f8ddf7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8de006a770 0x7f8de009d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:40.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.004+0000 7f8ddf7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8de006a770 0x7f8de009d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50598/0 (socket says 172.21.15.3:50598) 2023-10-03T10:15:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.004+0000 7f8ddf7fe700 1 -- 172.21.15.3:0/2122038986 learned_addr learned my addr 172.21.15.3:0/2122038986 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.004+0000 7f8ddffff700 1 --2- 172.21.15.3:0/2122038986 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8de013c1e0 0x7f8de013e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.004+0000 7f8ddf7fe700 1 -- 172.21.15.3:0/2122038986 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8de013c1e0 msgr2=0x7f8de013e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.004+0000 7f8ddf7fe700 1 --2- 172.21.15.3:0/2122038986 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8de013c1e0 0x7f8de013e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.004+0000 7f8ddf7fe700 1 -- 172.21.15.3:0/2122038986 >> v2:172.21.15.3:3300/0 conn(0x7f8de009dc30 msgr2=0x7f8de009e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.004+0000 7f8ddf7fe700 1 --2- 172.21.15.3:0/2122038986 >> v2:172.21.15.3:3300/0 conn(0x7f8de009dc30 0x7f8de009e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.004+0000 7f8ddf7fe700 1 -- 172.21.15.3:0/2122038986 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de009e920 con 0x7f8de006a770 2023-10-03T10:15:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.004+0000 7f8ddf7fe700 1 --2- 172.21.15.3:0/2122038986 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8de006a770 0x7f8de009d6f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8dd00098f0 tx=0x7f8dd0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=908a3bd0049319ee server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:40.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.008+0000 7f5eedffb700 1 -- 172.21.15.3:0/745258874 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5ee8003a00 con 0x7f5ef809dc30 2023-10-03T10:15:40.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.008+0000 7f5eedffb700 1 -- 172.21.15.3:0/745258874 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5ee8021070 con 0x7f5ef809dc30 2023-10-03T10:15:40.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.013+0000 7f8dde7fc700 1 -- 172.21.15.3:0/2122038986 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd000b8a0 con 0x7f8de006a770 2023-10-03T10:15:40.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.013+0000 7f8dde7fc700 1 -- 172.21.15.3:0/2122038986 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd000ba00 con 0x7f8de006a770 2023-10-03T10:15:40.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.013+0000 7f8dde7fc700 1 -- 172.21.15.3:0/2122038986 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd000bd70 con 0x7f8de006a770 2023-10-03T10:15:40.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.013+0000 7f5eedffb700 1 -- 172.21.15.3:0/745258874 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5ee8024970 con 0x7f5ef809dc30 2023-10-03T10:15:40.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.014+0000 7f8de5c17700 1 -- 172.21.15.3:0/2122038986 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8de006a770 msgr2=0x7f8de009d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:40.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.014+0000 7f8de5c17700 1 --2- 172.21.15.3:0/2122038986 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8de006a770 0x7f8de009d6f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8dd00098f0 tx=0x7f8dd0009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:40.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.014+0000 7f8de5c17700 1 -- 172.21.15.3:0/2122038986 shutdown_connections 2023-10-03T10:15:40.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.014+0000 7f8de5c17700 1 --2- 172.21.15.3:0/2122038986 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8de013c1e0 0x7f8de013e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.014+0000 7f8de5c17700 1 --2- 172.21.15.3:0/2122038986 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8de006a770 0x7f8de009d6f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.014+0000 7f8de5c17700 1 --2- 172.21.15.3:0/2122038986 >> v2:172.21.15.3:3300/0 conn(0x7f8de009dc30 0x7f8de009e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.014+0000 7f8de5c17700 1 -- 172.21.15.3:0/2122038986 >> 172.21.15.3:0/2122038986 conn(0x7f8de0066f50 msgr2=0x7f8de009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:40.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.014+0000 7f8de5c17700 1 -- 172.21.15.3:0/2122038986 shutdown_connections 2023-10-03T10:15:40.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.014+0000 7f8de5c17700 1 -- 172.21.15.3:0/2122038986 wait complete. 2023-10-03T10:15:40.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.015+0000 7f8de5c17700 1 Processor -- start 2023-10-03T10:15:40.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.016+0000 7f8de5c17700 1 -- start start 2023-10-03T10:15:40.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.016+0000 7f8de5c17700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8de006a770 0x7f8de013a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:40.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.016+0000 7f8de5c17700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8de009dc30 0x7f8de013aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:40.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.016+0000 7f8de5c17700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8de013c1e0 0x7f8de013b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:40.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.016+0000 7f8de5c17700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de009f230 con 0x7f8de009dc30 2023-10-03T10:15:40.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.016+0000 7f8de5c17700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de011fab0 con 0x7f8de013c1e0 2023-10-03T10:15:40.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.016+0000 7f8de5c17700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8de013ba80 con 0x7f8de006a770 2023-10-03T10:15:40.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.016+0000 7f8ddffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8de013c1e0 0x7f8de013b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:40.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.016+0000 7f8ddffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8de013c1e0 0x7f8de013b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50602/0 (socket says 172.21.15.3:50602) 2023-10-03T10:15:40.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.016+0000 7f8ddffff700 1 -- 172.21.15.3:0/463300188 learned_addr learned my addr 172.21.15.3:0/463300188 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:40.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.016+0000 7f8ddeffd700 1 --2- 172.21.15.3:0/463300188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8de009dc30 0x7f8de013aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:40.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.017+0000 7f8ddf7fe700 1 --2- 172.21.15.3:0/463300188 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8de006a770 0x7f8de013a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:40.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.017+0000 7f8ddffff700 1 -- 172.21.15.3:0/463300188 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8de006a770 msgr2=0x7f8de013a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:40.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.017+0000 7f8ddffff700 1 --2- 172.21.15.3:0/463300188 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8de006a770 0x7f8de013a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.017+0000 7f8ddffff700 1 -- 172.21.15.3:0/463300188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8de009dc30 msgr2=0x7f8de013aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:40.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.017+0000 7f8ddffff700 1 --2- 172.21.15.3:0/463300188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8de009dc30 0x7f8de013aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.017+0000 7f8ddffff700 1 -- 172.21.15.3:0/463300188 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd0009550 con 0x7f8de013c1e0 2023-10-03T10:15:40.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.017+0000 7f8ddf7fe700 1 --2- 172.21.15.3:0/463300188 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8de006a770 0x7f8de013a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:15:40.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.017+0000 7f8ddffff700 1 --2- 172.21.15.3:0/463300188 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8de013c1e0 0x7f8de013b020 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8007ea0 tx=0x7f8dd800e3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:40.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.017+0000 7f8dc7fff700 1 -- 172.21.15.3:0/463300188 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd800eea0 con 0x7f8de013c1e0 2023-10-03T10:15:40.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.017+0000 7f8dc7fff700 1 -- 172.21.15.3:0/463300188 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd8014420 con 0x7f8de013c1e0 2023-10-03T10:15:40.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.017+0000 7f8de5c17700 1 -- 172.21.15.3:0/463300188 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8de011f390 con 0x7f8de013c1e0 2023-10-03T10:15:40.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.017+0000 7f8dc7fff700 1 -- 172.21.15.3:0/463300188 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd80147b0 con 0x7f8de013c1e0 2023-10-03T10:15:40.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.017+0000 7f8de5c17700 1 -- 172.21.15.3:0/463300188 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de011ffc0 con 0x7f8de013c1e0 2023-10-03T10:15:40.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.018+0000 7f8dc7fff700 1 -- 172.21.15.3:0/463300188 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8dd8014910 con 0x7f8de013c1e0 2023-10-03T10:15:40.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.019+0000 7f8dc7fff700 1 -- 172.21.15.3:0/463300188 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8dd803ff30 con 0x7f8de013c1e0 2023-10-03T10:15:40.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.019+0000 7f8de5c17700 1 -- 172.21.15.3:0/463300188 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8de011fbf0 con 0x7f8de013c1e0 2023-10-03T10:15:40.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.028+0000 7f8dc7fff700 1 -- 172.21.15.3:0/463300188 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8dd801bbd0 con 0x7f8de013c1e0 2023-10-03T10:15:40.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.058+0000 7f3cedbbb700 1 Processor -- start 2023-10-03T10:15:40.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.058+0000 7f3cedbbb700 1 -- start start 2023-10-03T10:15:40.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.058+0000 7f3cedbbb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3ce809d6f0 0x7f3ce809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:40.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.058+0000 7f3cedbbb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ce809e000 0x7f3ce81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:40.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.058+0000 7f3cedbbb700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce809b120 con 0x7f3ce809e410 2023-10-03T10:15:40.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.058+0000 7f3cedbbb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce806b460 con 0x7f3ce809d6f0 2023-10-03T10:15:40.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.058+0000 7f3cedbbb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce809a7e0 con 0x7f3ce809e000 2023-10-03T10:15:40.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.058+0000 7f3ce6ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ce809e000 0x7f3ce81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:40.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.058+0000 7f3ce77fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3ce809d6f0 0x7f3ce809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:40.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.058+0000 7f3ce6ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ce809e000 0x7f3ce81339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50604/0 (socket says 172.21.15.3:50604) 2023-10-03T10:15:40.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.059+0000 7f3ce6ffd700 1 -- 172.21.15.3:0/3123558738 learned_addr learned my addr 172.21.15.3:0/3123558738 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:40.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.059+0000 7f3ce6ffd700 1 -- 172.21.15.3:0/3123558738 >> v2:172.21.15.3:3300/0 conn(0x7f3ce809d6f0 msgr2=0x7f3ce809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:40.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.059+0000 7f3ce6ffd700 1 --2- 172.21.15.3:0/3123558738 >> v2:172.21.15.3:3300/0 conn(0x7f3ce809d6f0 0x7f3ce809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.059+0000 7f3ce6ffd700 1 -- 172.21.15.3:0/3123558738 >> v1:172.21.15.3:6789/0 conn(0x7f3ce809e410 legacy=0x7f3ce8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:40.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.059+0000 7f3ce67fc700 1 -- 172.21.15.3:0/3123558738 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2752425629 0 0) 0x7f3ce809b120 con 0x7f3ce809e410 2023-10-03T10:15:40.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.059+0000 7f3ce6ffd700 1 -- 172.21.15.3:0/3123558738 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce809a500 con 0x7f3ce809e000 2023-10-03T10:15:40.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.059+0000 7f3ce6ffd700 1 --2- 172.21.15.3:0/3123558738 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ce809e000 0x7f3ce81339f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc009920 tx=0x7f3cdc009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8c93f46c5911ecb server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:40.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.059+0000 7f3ce67fc700 1 -- 172.21.15.3:0/3123558738 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc011830 con 0x7f3ce809e000 2023-10-03T10:15:40.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.059+0000 7f3ce67fc700 1 -- 172.21.15.3:0/3123558738 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cdc011990 con 0x7f3ce809e000 2023-10-03T10:15:40.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.059+0000 7f3ce67fc700 1 -- 172.21.15.3:0/3123558738 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc011d00 con 0x7f3ce809e000 2023-10-03T10:15:40.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.060+0000 7f3cedbbb700 1 -- 172.21.15.3:0/3123558738 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ce809e000 msgr2=0x7f3ce81339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:40.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.060+0000 7f3cedbbb700 1 --2- 172.21.15.3:0/3123558738 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ce809e000 0x7f3ce81339f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc009920 tx=0x7f3cdc009c30 comp rx=0 tx=0).stop 2023-10-03T10:15:40.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.060+0000 7f3cedbbb700 1 -- 172.21.15.3:0/3123558738 shutdown_connections 2023-10-03T10:15:40.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.060+0000 7f3cedbbb700 1 --2- 172.21.15.3:0/3123558738 >> v2:172.21.15.3:3300/0 conn(0x7f3ce809d6f0 0x7f3ce809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.060+0000 7f3cedbbb700 1 --2- 172.21.15.3:0/3123558738 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ce809e000 0x7f3ce81339f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.060+0000 7f3cedbbb700 1 -- 172.21.15.3:0/3123558738 >> 172.21.15.3:0/3123558738 conn(0x7f3ce8066f50 msgr2=0x7f3ce809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:40.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.060+0000 7f3cedbbb700 1 -- 172.21.15.3:0/3123558738 shutdown_connections 2023-10-03T10:15:40.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.060+0000 7f3cedbbb700 1 -- 172.21.15.3:0/3123558738 wait complete. 2023-10-03T10:15:40.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.061+0000 7f3cedbbb700 1 Processor -- start 2023-10-03T10:15:40.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.062+0000 7f3cedbbb700 1 -- start start 2023-10-03T10:15:40.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.062+0000 7f3cedbbb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ce809d6f0 0x7f3ce81c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:40.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.062+0000 7f3cedbbb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ce809e000 0x7f3ce81c5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:40.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.062+0000 7f3cedbbb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ce809e410 0x7f3ce81c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:40.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.062+0000 7f3cedbbb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce809c7a0 con 0x7f3ce809e410 2023-10-03T10:15:40.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.062+0000 7f3cedbbb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce81a7110 con 0x7f3ce809e000 2023-10-03T10:15:40.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.062+0000 7f3cedbbb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce81a8b70 con 0x7f3ce809d6f0 2023-10-03T10:15:40.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.062+0000 7f3ce6ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ce809e000 0x7f3ce81c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:40.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.062+0000 7f3ce6ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ce809e000 0x7f3ce81c5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50610/0 (socket says 172.21.15.3:50610) 2023-10-03T10:15:40.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.062+0000 7f3ce6ffd700 1 -- 172.21.15.3:0/617447816 learned_addr learned my addr 172.21.15.3:0/617447816 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:40.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.062+0000 7f3ce77fe700 1 --2- 172.21.15.3:0/617447816 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ce809d6f0 0x7f3ce81c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:40.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.062+0000 7f3ce7fff700 1 --2- 172.21.15.3:0/617447816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ce809e410 0x7f3ce81c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:40.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.063+0000 7f3ce6ffd700 1 -- 172.21.15.3:0/617447816 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ce809d6f0 msgr2=0x7f3ce81c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:40.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.063+0000 7f3ce6ffd700 1 --2- 172.21.15.3:0/617447816 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ce809d6f0 0x7f3ce81c50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.063+0000 7f3ce6ffd700 1 -- 172.21.15.3:0/617447816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ce809e410 msgr2=0x7f3ce81c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:40.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.063+0000 7f3ce6ffd700 1 --2- 172.21.15.3:0/617447816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ce809e410 0x7f3ce81c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.063+0000 7f3ce6ffd700 1 -- 172.21.15.3:0/617447816 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cdc009550 con 0x7f3ce809e000 2023-10-03T10:15:40.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.063+0000 7f3ce6ffd700 1 --2- 172.21.15.3:0/617447816 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ce809e000 0x7f3ce81c5630 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3ce809a920 tx=0x7f3cdc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:40.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.063+0000 7f3ccffff700 1 -- 172.21.15.3:0/617447816 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc004240 con 0x7f3ce809e000 2023-10-03T10:15:40.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.063+0000 7f3ccffff700 1 -- 172.21.15.3:0/617447816 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cdc01f070 con 0x7f3ce809e000 2023-10-03T10:15:40.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.063+0000 7f3ccffff700 1 -- 172.21.15.3:0/617447816 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc0036a0 con 0x7f3ce809e000 2023-10-03T10:15:40.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.063+0000 7f3cedbbb700 1 -- 172.21.15.3:0/617447816 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce81a9220 con 0x7f3ce809e000 2023-10-03T10:15:40.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.063+0000 7f3cedbbb700 1 -- 172.21.15.3:0/617447816 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce81a9f50 con 0x7f3ce809e000 2023-10-03T10:15:40.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.064+0000 7f3cedbbb700 1 -- 172.21.15.3:0/617447816 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce8074ea0 con 0x7f3ce809e000 2023-10-03T10:15:40.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.065+0000 7f3ccffff700 1 -- 172.21.15.3:0/617447816 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3cdc021030 con 0x7f3ce809e000 2023-10-03T10:15:40.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.065+0000 7f3ccffff700 1 -- 172.21.15.3:0/617447816 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3cdc041080 con 0x7f3ce809e000 2023-10-03T10:15:40.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.070+0000 7f3ccffff700 1 -- 172.21.15.3:0/617447816 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3cdc0239c0 con 0x7f3ce809e000 2023-10-03T10:15:40.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.225+0000 7f5eff751700 1 -- 172.21.15.3:0/745258874 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5ef806c360 con 0x7f5ef809dc30 2023-10-03T10:15:40.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.226+0000 7f5eedffb700 1 -- 172.21.15.3:0/745258874 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5ee8034090 con 0x7f5ef809dc30 2023-10-03T10:15:40.226 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:40.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.228+0000 7f5eff751700 1 -- 172.21.15.3:0/745258874 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ef809dc30 msgr2=0x7f5ef811d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:40.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.228+0000 7f5eff751700 1 --2- 172.21.15.3:0/745258874 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ef809dc30 0x7f5ef811d5e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8009fd0 tx=0x7f5ee8004750 comp rx=0 tx=0).stop 2023-10-03T10:15:40.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.228+0000 7f5eff751700 1 -- 172.21.15.3:0/745258874 shutdown_connections 2023-10-03T10:15:40.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.228+0000 7f5eff751700 1 --2- 172.21.15.3:0/745258874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ef813c1e0 0x7f5ef811db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.228+0000 7f5eff751700 1 --2- 172.21.15.3:0/745258874 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ef806a770 0x7f5ef813b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.228+0000 7f5eff751700 1 --2- 172.21.15.3:0/745258874 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ef809dc30 0x7f5ef811d5e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.228+0000 7f5eff751700 1 -- 172.21.15.3:0/745258874 >> 172.21.15.3:0/745258874 conn(0x7f5ef8066f50 msgr2=0x7f5ef813c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:40.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.228+0000 7f5eff751700 1 -- 172.21.15.3:0/745258874 shutdown_connections 2023-10-03T10:15:40.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.228+0000 7f5eff751700 1 -- 172.21.15.3:0/745258874 wait complete. 2023-10-03T10:15:40.247 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:40.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.263+0000 7f09e716d700 1 -- 172.21.15.3:0/2586199597 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f09e0123520 con 0x7f09e009ffb0 2023-10-03T10:15:40.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.263+0000 7f09dd7fa700 1 -- 172.21.15.3:0/2586199597 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f09e0123520 con 0x7f09e009ffb0 2023-10-03T10:15:40.263 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:40.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.265+0000 7f09e716d700 1 -- 172.21.15.3:0/2586199597 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09e009ffb0 msgr2=0x7f09e0120f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:40.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.265+0000 7f09e716d700 1 --2- 172.21.15.3:0/2586199597 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09e009ffb0 0x7f09e0120f40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00ea10 tx=0x7f09cc00edd0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.265+0000 7f09e716d700 1 -- 172.21.15.3:0/2586199597 shutdown_connections 2023-10-03T10:15:40.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.265+0000 7f09e716d700 1 --2- 172.21.15.3:0/2586199597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09e0138e30 0x7f09e0121480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.265+0000 7f09e716d700 1 --2- 172.21.15.3:0/2586199597 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f09e009e850 0x7f09e0120a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.265+0000 7f09e716d700 1 --2- 172.21.15.3:0/2586199597 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09e009ffb0 0x7f09e0120f40 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.265+0000 7f09e716d700 1 -- 172.21.15.3:0/2586199597 >> 172.21.15.3:0/2586199597 conn(0x7f09e00a3f70 msgr2=0x7f09e00a1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:40.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.265+0000 7f09e716d700 1 -- 172.21.15.3:0/2586199597 shutdown_connections 2023-10-03T10:15:40.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.265+0000 7f09e716d700 1 -- 172.21.15.3:0/2586199597 wait complete. 2023-10-03T10:15:40.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.277+0000 7f3cedbbb700 1 -- 172.21.15.3:0/617447816 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3ce806c360 con 0x7f3ce809e000 2023-10-03T10:15:40.280 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:40.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.280+0000 7f3ccffff700 1 -- 172.21.15.3:0/617447816 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3cdc01b430 con 0x7f3ce809e000 2023-10-03T10:15:40.281 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:40.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.282+0000 7f3cedbbb700 1 -- 172.21.15.3:0/617447816 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ce809e000 msgr2=0x7f3ce81c5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:40.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.282+0000 7f3cedbbb700 1 --2- 172.21.15.3:0/617447816 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ce809e000 0x7f3ce81c5630 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3ce809a920 tx=0x7f3cdc004750 comp rx=0 tx=0).stop 2023-10-03T10:15:40.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.282+0000 7f3cedbbb700 1 -- 172.21.15.3:0/617447816 shutdown_connections 2023-10-03T10:15:40.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.282+0000 7f3cedbbb700 1 --2- 172.21.15.3:0/617447816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ce809e410 0x7f3ce81c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.282+0000 7f3cedbbb700 1 --2- 172.21.15.3:0/617447816 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ce809d6f0 0x7f3ce81c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.282+0000 7f3cedbbb700 1 --2- 172.21.15.3:0/617447816 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ce809e000 0x7f3ce81c5630 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.282+0000 7f3cedbbb700 1 -- 172.21.15.3:0/617447816 >> 172.21.15.3:0/617447816 conn(0x7f3ce8066f50 msgr2=0x7f3ce81bac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:40.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.282+0000 7f3cedbbb700 1 -- 172.21.15.3:0/617447816 shutdown_connections 2023-10-03T10:15:40.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.282+0000 7f3cedbbb700 1 -- 172.21.15.3:0/617447816 wait complete. 2023-10-03T10:15:40.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.293+0000 7f8de5c17700 1 -- 172.21.15.3:0/463300188 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8de0074ea0 con 0x7f8de013c1e0 2023-10-03T10:15:40.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.294+0000 7f8dc7fff700 1 -- 172.21.15.3:0/463300188 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8dd8013e40 con 0x7f8de013c1e0 2023-10-03T10:15:40.294 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:40.297 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:40.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.296+0000 7f8de5c17700 1 -- 172.21.15.3:0/463300188 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8de013c1e0 msgr2=0x7f8de013b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:40.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.296+0000 7f8de5c17700 1 --2- 172.21.15.3:0/463300188 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8de013c1e0 0x7f8de013b020 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8007ea0 tx=0x7f8dd800e3c0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.296+0000 7f8de5c17700 1 -- 172.21.15.3:0/463300188 shutdown_connections 2023-10-03T10:15:40.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.296+0000 7f8de5c17700 1 --2- 172.21.15.3:0/463300188 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8de013c1e0 0x7f8de013b020 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.296+0000 7f8de5c17700 1 --2- 172.21.15.3:0/463300188 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8de006a770 0x7f8de013a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.296+0000 7f8de5c17700 1 --2- 172.21.15.3:0/463300188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8de009dc30 0x7f8de013aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:40.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.296+0000 7f8de5c17700 1 -- 172.21.15.3:0/463300188 >> 172.21.15.3:0/463300188 conn(0x7f8de0066f50 msgr2=0x7f8de009b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:40.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.296+0000 7f8de5c17700 1 -- 172.21.15.3:0/463300188 shutdown_connections 2023-10-03T10:15:40.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:40.296+0000 7f8de5c17700 1 -- 172.21.15.3:0/463300188 wait complete. 2023-10-03T10:15:40.310 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:41.248 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:41.281 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:41.297 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:41.311 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:41.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.410+0000 7feffa77f700 1 Processor -- start 2023-10-03T10:15:41.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.410+0000 7feffa77f700 1 -- start start 2023-10-03T10:15:41.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.411+0000 7feffa77f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feff40a0a30 0x7feff40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:41.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.411+0000 7feffa77f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feff40a1340 0x7feff4133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:41.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.411+0000 7feffa77f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feff4133fa0 0x7feff4134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:41.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.411+0000 7feffa77f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feff409edd0 con 0x7feff40a1340 2023-10-03T10:15:41.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.411+0000 7feffa77f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff409e1f0 con 0x7feff40a0a30 2023-10-03T10:15:41.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.411+0000 7feffa77f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feff409dc50 con 0x7feff4133fa0 2023-10-03T10:15:41.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.411+0000 7feff8d1c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feff4133fa0 0x7feff4134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:41.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.411+0000 7feff8d1c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feff4133fa0 0x7feff4134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42044/0 (socket says 172.21.15.3:42044) 2023-10-03T10:15:41.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.411+0000 7feff8d1c700 1 -- 172.21.15.3:0/511537547 learned_addr learned my addr 172.21.15.3:0/511537547 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:41.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.411+0000 7feff37fe700 1 --2- 172.21.15.3:0/511537547 >> v2:172.21.15.3:3300/0 conn(0x7feff40a1340 0x7feff4133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:41.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.412+0000 7feff8d1c700 1 -- 172.21.15.3:0/511537547 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feff40a0a30 msgr2=0x7feff40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:41.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.412+0000 7feff8d1c700 1 --2- 172.21.15.3:0/511537547 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feff40a0a30 0x7feff40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.412+0000 7feff8d1c700 1 -- 172.21.15.3:0/511537547 >> v2:172.21.15.3:3300/0 conn(0x7feff40a1340 msgr2=0x7feff4133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:41.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.412+0000 7feff8d1c700 1 --2- 172.21.15.3:0/511537547 >> v2:172.21.15.3:3300/0 conn(0x7feff40a1340 0x7feff4133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.412+0000 7feff8d1c700 1 -- 172.21.15.3:0/511537547 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff40a1fe0 con 0x7feff4133fa0 2023-10-03T10:15:41.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.412+0000 7feff8d1c700 1 --2- 172.21.15.3:0/511537547 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feff4133fa0 0x7feff4134410 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fefec009e80 tx=0x7fefec00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=797388cd3a6878ba server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:41.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.412+0000 7feff2ffd700 1 -- 172.21.15.3:0/511537547 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec00e070 con 0x7feff4133fa0 2023-10-03T10:15:41.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.412+0000 7feff2ffd700 1 -- 172.21.15.3:0/511537547 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefec00b8d0 con 0x7feff4133fa0 2023-10-03T10:15:41.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.412+0000 7feff2ffd700 1 -- 172.21.15.3:0/511537547 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec00bc80 con 0x7feff4133fa0 2023-10-03T10:15:41.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.413+0000 7feffa77f700 1 -- 172.21.15.3:0/511537547 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feff4133fa0 msgr2=0x7feff4134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:41.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.413+0000 7feffa77f700 1 --2- 172.21.15.3:0/511537547 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feff4133fa0 0x7feff4134410 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fefec009e80 tx=0x7fefec00b330 comp rx=0 tx=0).stop 2023-10-03T10:15:41.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.413+0000 7feffa77f700 1 -- 172.21.15.3:0/511537547 shutdown_connections 2023-10-03T10:15:41.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.413+0000 7feffa77f700 1 --2- 172.21.15.3:0/511537547 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feff4133fa0 0x7feff4134410 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.413+0000 7feffa77f700 1 --2- 172.21.15.3:0/511537547 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feff40a0a30 0x7feff40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.413+0000 7feffa77f700 1 --2- 172.21.15.3:0/511537547 >> v2:172.21.15.3:3300/0 conn(0x7feff40a1340 0x7feff4133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.413+0000 7feffa77f700 1 -- 172.21.15.3:0/511537547 >> 172.21.15.3:0/511537547 conn(0x7feff40a6550 msgr2=0x7feff40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:41.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.413+0000 7feffa77f700 1 -- 172.21.15.3:0/511537547 shutdown_connections 2023-10-03T10:15:41.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.413+0000 7feffa77f700 1 -- 172.21.15.3:0/511537547 wait complete. 2023-10-03T10:15:41.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.415+0000 7feffa77f700 1 Processor -- start 2023-10-03T10:15:41.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.415+0000 7feffa77f700 1 -- start start 2023-10-03T10:15:41.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.416+0000 7feffa77f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feff40a0a30 0x7feff4120390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:41.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.416+0000 7feffa77f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feff40a1340 0x7feff41208d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:41.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.416+0000 7feffa77f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feff4133fa0 0x7feff4120e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:41.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.416+0000 7feffa77f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff4133780 con 0x7feff40a0a30 2023-10-03T10:15:41.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.416+0000 7feffa77f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff41225c0 con 0x7feff40a1340 2023-10-03T10:15:41.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.416+0000 7feffa77f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feff41231f0 con 0x7feff4133fa0 2023-10-03T10:15:41.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.416+0000 7feff3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feff40a0a30 0x7feff4120390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:41.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.416+0000 7feff3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feff40a0a30 0x7feff4120390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42278/0 (socket says 172.21.15.3:42278) 2023-10-03T10:15:41.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.416+0000 7feff3fff700 1 -- 172.21.15.3:0/1083580110 learned_addr learned my addr 172.21.15.3:0/1083580110 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:41.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.416+0000 7feff8d1c700 1 --2- 172.21.15.3:0/1083580110 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feff4133fa0 0x7feff4120e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:41.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.417+0000 7feff37fe700 1 --2- 172.21.15.3:0/1083580110 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feff40a1340 0x7feff41208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:41.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.417+0000 7feff8d1c700 1 -- 172.21.15.3:0/1083580110 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feff40a1340 msgr2=0x7feff41208d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:41.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.417+0000 7feff8d1c700 1 --2- 172.21.15.3:0/1083580110 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feff40a1340 0x7feff41208d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.417+0000 7feff8d1c700 1 -- 172.21.15.3:0/1083580110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feff40a0a30 msgr2=0x7feff4120390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:41.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.417+0000 7feff8d1c700 1 --2- 172.21.15.3:0/1083580110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feff40a0a30 0x7feff4120390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.417+0000 7feff8d1c700 1 -- 172.21.15.3:0/1083580110 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefec009b30 con 0x7feff4133fa0 2023-10-03T10:15:41.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.417+0000 7feff37fe700 1 --2- 172.21.15.3:0/1083580110 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feff40a1340 0x7feff41208d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:15:41.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.417+0000 7feff8d1c700 1 --2- 172.21.15.3:0/1083580110 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feff4133fa0 0x7feff4120e10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fefec015000 tx=0x7fefec004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:41.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.418+0000 7feff0ff9700 1 -- 172.21.15.3:0/1083580110 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec004450 con 0x7feff4133fa0 2023-10-03T10:15:41.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.418+0000 7feff0ff9700 1 -- 172.21.15.3:0/1083580110 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefec0045f0 con 0x7feff4133fa0 2023-10-03T10:15:41.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.418+0000 7feffa77f700 1 -- 172.21.15.3:0/1083580110 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff4123640 con 0x7feff4133fa0 2023-10-03T10:15:41.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.418+0000 7feff0ff9700 1 -- 172.21.15.3:0/1083580110 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec0099b0 con 0x7feff4133fa0 2023-10-03T10:15:41.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.418+0000 7feffa77f700 1 -- 172.21.15.3:0/1083580110 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff40a0220 con 0x7feff4133fa0 2023-10-03T10:15:41.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.419+0000 7feff0ff9700 1 -- 172.21.15.3:0/1083580110 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fefec01c410 con 0x7feff4133fa0 2023-10-03T10:15:41.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.419+0000 7feffa77f700 1 -- 172.21.15.3:0/1083580110 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff40a6c20 con 0x7feff4133fa0 2023-10-03T10:15:41.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.419+0000 7feff0ff9700 1 -- 172.21.15.3:0/1083580110 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fefec041250 con 0x7feff4133fa0 2023-10-03T10:15:41.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.424+0000 7feff0ff9700 1 -- 172.21.15.3:0/1083580110 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fefec01cbe0 con 0x7feff4133fa0 2023-10-03T10:15:41.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.442+0000 7f0e9a53e700 1 Processor -- start 2023-10-03T10:15:41.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.442+0000 7f0e9a53e700 1 -- start start 2023-10-03T10:15:41.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.442+0000 7f0e9a53e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e9406e570 0x7f0e9406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:41.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.442+0000 7f0e9a53e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0e9409eca0 0x7f0e9406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:41.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.442+0000 7f0e9a53e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e94070ee0 con 0x7f0e94134610 2023-10-03T10:15:41.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.442+0000 7f0e9a53e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e94071480 con 0x7f0e9409eca0 2023-10-03T10:15:41.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.442+0000 7f0e9a53e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9406f3c0 con 0x7f0e9406e570 2023-10-03T10:15:41.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.443+0000 7f0e937fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0e9409eca0 0x7f0e9406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:41.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.443+0000 7f0e93fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e9406e570 0x7f0e9406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:41.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.443+0000 7f0e937fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0e9409eca0 0x7f0e9406ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42286/0 (socket says 172.21.15.3:42286) 2023-10-03T10:15:41.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.443+0000 7f0e93fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e9406e570 0x7f0e9406e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42048/0 (socket says 172.21.15.3:42048) 2023-10-03T10:15:41.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.443+0000 7f0e937fe700 1 -- 172.21.15.3:0/722527490 learned_addr learned my addr 172.21.15.3:0/722527490 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:41.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.443+0000 7f0e937fe700 1 -- 172.21.15.3:0/722527490 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e9406e570 msgr2=0x7f0e9406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:41.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.443+0000 7f0e937fe700 1 --2- 172.21.15.3:0/722527490 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e9406e570 0x7f0e9406e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.443+0000 7f0e937fe700 1 -- 172.21.15.3:0/722527490 >> v1:172.21.15.3:6789/0 conn(0x7f0e94134610 legacy=0x7f0e94134a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:41.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.443+0000 7f0e92ffd700 1 -- 172.21.15.3:0/722527490 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 244223464 0 0) 0x7f0e94070ee0 con 0x7f0e94134610 2023-10-03T10:15:41.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.443+0000 7f0e937fe700 1 -- 172.21.15.3:0/722527490 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e94071b90 con 0x7f0e9409eca0 2023-10-03T10:15:41.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.443+0000 7f0e937fe700 1 --2- 172.21.15.3:0/722527490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e9409eca0 0x7f0e9406ee80 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f0e880098f0 tx=0x7f0e88009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed9d94b452bc9ce server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:41.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.444+0000 7f0e92ffd700 1 -- 172.21.15.3:0/722527490 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e88011830 con 0x7f0e9409eca0 2023-10-03T10:15:41.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.444+0000 7f0e92ffd700 1 -- 172.21.15.3:0/722527490 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e88011990 con 0x7f0e9409eca0 2023-10-03T10:15:41.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.444+0000 7f0e92ffd700 1 -- 172.21.15.3:0/722527490 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e88011d20 con 0x7f0e9409eca0 2023-10-03T10:15:41.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.444+0000 7f0e9a53e700 1 -- 172.21.15.3:0/722527490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e9409eca0 msgr2=0x7f0e9406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:41.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.444+0000 7f0e9a53e700 1 --2- 172.21.15.3:0/722527490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e9409eca0 0x7f0e9406ee80 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f0e880098f0 tx=0x7f0e88009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:41.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.445+0000 7f0e9a53e700 1 -- 172.21.15.3:0/722527490 shutdown_connections 2023-10-03T10:15:41.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.445+0000 7f0e9a53e700 1 --2- 172.21.15.3:0/722527490 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e9406e570 0x7f0e9406e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.445+0000 7f0e9a53e700 1 --2- 172.21.15.3:0/722527490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e9409eca0 0x7f0e9406ee80 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.445+0000 7f0e9a53e700 1 -- 172.21.15.3:0/722527490 >> 172.21.15.3:0/722527490 conn(0x7f0e940a2aa0 msgr2=0x7f0e940a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:41.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.445+0000 7f0e9a53e700 1 -- 172.21.15.3:0/722527490 shutdown_connections 2023-10-03T10:15:41.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.445+0000 7f0e9a53e700 1 -- 172.21.15.3:0/722527490 wait complete. 2023-10-03T10:15:41.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.446+0000 7f0e9a53e700 1 Processor -- start 2023-10-03T10:15:41.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.446+0000 7f0e9a53e700 1 -- start start 2023-10-03T10:15:41.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.447+0000 7f0e9a53e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e9406e570 0x7f0e9413b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:41.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.447+0000 7f0e9a53e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e94134610 0x7f0e94132650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:41.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.447+0000 7f0e9a53e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0e9413e2f0 0x7f0e94132b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:41.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.447+0000 7f0e9a53e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e941344d0 con 0x7f0e9406e570 2023-10-03T10:15:41.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.447+0000 7f0e9a53e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9413dbf0 con 0x7f0e9413e2f0 2023-10-03T10:15:41.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.447+0000 7f0e9a53e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9413ddf0 con 0x7f0e94134610 2023-10-03T10:15:41.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.447+0000 7f0e98adb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0e9413e2f0 0x7f0e94132b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:41.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.447+0000 7f0e93fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e9406e570 0x7f0e9413b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:41.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.447+0000 7f0e98adb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0e9413e2f0 0x7f0e94132b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50636/0 (socket says 172.21.15.3:50636) 2023-10-03T10:15:41.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.447+0000 7f0e98adb700 1 -- 172.21.15.3:0/2021231177 learned_addr learned my addr 172.21.15.3:0/2021231177 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:41.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.447+0000 7f0e937fe700 1 --2- 172.21.15.3:0/2021231177 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e94134610 0x7f0e94132650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:41.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.448+0000 7f0e98adb700 1 -- 172.21.15.3:0/2021231177 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e94134610 msgr2=0x7f0e94132650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:41.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.448+0000 7f0e98adb700 1 --2- 172.21.15.3:0/2021231177 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e94134610 0x7f0e94132650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.448+0000 7f0e98adb700 1 -- 172.21.15.3:0/2021231177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e9406e570 msgr2=0x7f0e9413b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:41.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.448+0000 7f0e98adb700 1 --2- 172.21.15.3:0/2021231177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e9406e570 0x7f0e9413b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.448+0000 7f0e98adb700 1 -- 172.21.15.3:0/2021231177 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e88009550 con 0x7f0e9413e2f0 2023-10-03T10:15:41.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.448+0000 7f0e93fff700 1 --2- 172.21.15.3:0/2021231177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e9406e570 0x7f0e9413b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:41.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.448+0000 7f0e98adb700 1 --2- 172.21.15.3:0/2021231177 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0e9413e2f0 0x7f0e94132b90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c00c840 tx=0x7f0e8c00cb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:41.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.448+0000 7f0e90ff9700 1 -- 172.21.15.3:0/2021231177 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c003e30 con 0x7f0e9413e2f0 2023-10-03T10:15:41.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.448+0000 7f0e90ff9700 1 -- 172.21.15.3:0/2021231177 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8c005070 con 0x7f0e9413e2f0 2023-10-03T10:15:41.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.448+0000 7f0e9a53e700 1 -- 172.21.15.3:0/2021231177 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e9413bf00 con 0x7f0e9413e2f0 2023-10-03T10:15:41.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.448+0000 7f0e90ff9700 1 -- 172.21.15.3:0/2021231177 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c005400 con 0x7f0e9413e2f0 2023-10-03T10:15:41.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.448+0000 7f0e9a53e700 1 -- 172.21.15.3:0/2021231177 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e940a1e60 con 0x7f0e9413e2f0 2023-10-03T10:15:41.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.450+0000 7f0e9a53e700 1 -- 172.21.15.3:0/2021231177 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e94135100 con 0x7f0e9413e2f0 2023-10-03T10:15:41.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.450+0000 7f0e90ff9700 1 -- 172.21.15.3:0/2021231177 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0e8c005560 con 0x7f0e9413e2f0 2023-10-03T10:15:41.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.451+0000 7f0e90ff9700 1 -- 172.21.15.3:0/2021231177 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0e8c03f140 con 0x7f0e9413e2f0 2023-10-03T10:15:41.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.459+0000 7f0e90ff9700 1 -- 172.21.15.3:0/2021231177 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0e8c019b40 con 0x7f0e9413e2f0 2023-10-03T10:15:41.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.484+0000 7fbbeb1c7700 1 Processor -- start 2023-10-03T10:15:41.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.485+0000 7fbbeb1c7700 1 -- start start 2023-10-03T10:15:41.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.485+0000 7fbbeb1c7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbbe40a0a30 0x7fbbe40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:41.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.485+0000 7fbbeb1c7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbbe40a1340 0x7fbbe4133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:41.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.485+0000 7fbbeb1c7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe409edd0 con 0x7fbbe4133fa0 2023-10-03T10:15:41.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.485+0000 7fbbeb1c7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe409e1f0 con 0x7fbbe40a0a30 2023-10-03T10:15:41.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.485+0000 7fbbeb1c7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe409dc50 con 0x7fbbe40a1340 2023-10-03T10:15:41.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.485+0000 7fbbe8f63700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbbe40a0a30 0x7fbbe40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:41.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.485+0000 7fbbe8f63700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbbe40a0a30 0x7fbbe40a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50652/0 (socket says 172.21.15.3:50652) 2023-10-03T10:15:41.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.485+0000 7fbbe9764700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbbe4133fa0 0x7fbbe413ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34470/0 (socket says 172.21.15.3:34470) 2023-10-03T10:15:41.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.485+0000 7fbbe8f63700 1 -- 172.21.15.3:0/2687841389 learned_addr learned my addr 172.21.15.3:0/2687841389 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:41.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.486+0000 7fbbe8f63700 1 -- 172.21.15.3:0/2687841389 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbbe40a1340 msgr2=0x7fbbe4133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:41.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.486+0000 7fbbe8f63700 1 --2- 172.21.15.3:0/2687841389 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbbe40a1340 0x7fbbe4133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.486+0000 7fbbe8f63700 1 -- 172.21.15.3:0/2687841389 >> v1:172.21.15.3:6789/0 conn(0x7fbbe4133fa0 legacy=0x7fbbe413ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:41.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.486+0000 7fbbe8f63700 1 -- 172.21.15.3:0/2687841389 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe40a1fe0 con 0x7fbbe40a0a30 2023-10-03T10:15:41.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.486+0000 7fbbe8f63700 1 --2- 172.21.15.3:0/2687841389 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbbe40a0a30 0x7fbbe40a0e00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009920 tx=0x7fbbd8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6837c512a9faabf server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:41.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.486+0000 7fbbe37fe700 1 -- 172.21.15.3:0/2687841389 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8004230 con 0x7fbbe40a0a30 2023-10-03T10:15:41.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.486+0000 7fbbe37fe700 1 -- 172.21.15.3:0/2687841389 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd8004390 con 0x7fbbe40a0a30 2023-10-03T10:15:41.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.486+0000 7fbbe37fe700 1 -- 172.21.15.3:0/2687841389 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8009e70 con 0x7fbbe40a0a30 2023-10-03T10:15:41.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.487+0000 7fbbeb1c7700 1 -- 172.21.15.3:0/2687841389 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbbe40a0a30 msgr2=0x7fbbe40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:41.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.487+0000 7fbbeb1c7700 1 --2- 172.21.15.3:0/2687841389 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbbe40a0a30 0x7fbbe40a0e00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009920 tx=0x7fbbd8009c30 comp rx=0 tx=0).stop 2023-10-03T10:15:41.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.487+0000 7fbbeb1c7700 1 -- 172.21.15.3:0/2687841389 shutdown_connections 2023-10-03T10:15:41.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.487+0000 7fbbeb1c7700 1 --2- 172.21.15.3:0/2687841389 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbbe40a0a30 0x7fbbe40a0e00 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.487+0000 7fbbeb1c7700 1 --2- 172.21.15.3:0/2687841389 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbbe40a1340 0x7fbbe4133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.487+0000 7fbbeb1c7700 1 -- 172.21.15.3:0/2687841389 >> 172.21.15.3:0/2687841389 conn(0x7fbbe40a6550 msgr2=0x7fbbe40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:41.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.490+0000 7fbbeb1c7700 1 -- 172.21.15.3:0/2687841389 shutdown_connections 2023-10-03T10:15:41.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.490+0000 7fbbeb1c7700 1 -- 172.21.15.3:0/2687841389 wait complete. 2023-10-03T10:15:41.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.491+0000 7fbbeb1c7700 1 Processor -- start 2023-10-03T10:15:41.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.491+0000 7fbbeb1c7700 1 -- start start 2023-10-03T10:15:41.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.492+0000 7fbbeb1c7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbbe40a0a30 0x7fbbe411f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:41.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.492+0000 7fbbeb1c7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbe40a1340 0x7fbbe41c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:41.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.492+0000 7fbbeb1c7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbbe4133fa0 0x7fbbe41c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:41.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.492+0000 7fbbeb1c7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe41212f0 con 0x7fbbe40a1340 2023-10-03T10:15:41.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.492+0000 7fbbeb1c7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe4133300 con 0x7fbbe4133fa0 2023-10-03T10:15:41.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.492+0000 7fbbeb1c7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe4121ef0 con 0x7fbbe40a0a30 2023-10-03T10:15:41.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.492+0000 7fbbe8f63700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbbe40a0a30 0x7fbbe411f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:41.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.492+0000 7fbbe8f63700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbbe40a0a30 0x7fbbe411f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42080/0 (socket says 172.21.15.3:42080) 2023-10-03T10:15:41.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.492+0000 7fbbe8f63700 1 -- 172.21.15.3:0/1200825766 learned_addr learned my addr 172.21.15.3:0/1200825766 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:41.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.492+0000 7fbbe3fff700 1 --2- 172.21.15.3:0/1200825766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbe40a1340 0x7fbbe41c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:41.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.492+0000 7fbbe9764700 1 --2- 172.21.15.3:0/1200825766 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbbe4133fa0 0x7fbbe41c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:41.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.493+0000 7fbbe8f63700 1 -- 172.21.15.3:0/1200825766 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbbe4133fa0 msgr2=0x7fbbe41c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:41.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.493+0000 7fbbe8f63700 1 --2- 172.21.15.3:0/1200825766 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbbe4133fa0 0x7fbbe41c8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.493+0000 7fbbe8f63700 1 -- 172.21.15.3:0/1200825766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbe40a1340 msgr2=0x7fbbe41c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:41.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.493+0000 7fbbe8f63700 1 --2- 172.21.15.3:0/1200825766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbe40a1340 0x7fbbe41c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.493+0000 7fbbe8f63700 1 -- 172.21.15.3:0/1200825766 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd8009550 con 0x7fbbe40a0a30 2023-10-03T10:15:41.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.493+0000 7fbbe3fff700 1 --2- 172.21.15.3:0/1200825766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbe40a1340 0x7fbbe41c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:41.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.493+0000 7fbbe8f63700 1 --2- 172.21.15.3:0/1200825766 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbbe40a0a30 0x7fbbe411f5a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009130 tx=0x7fbbd8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:41.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.493+0000 7fbbe17fa700 1 -- 172.21.15.3:0/1200825766 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd800bcf0 con 0x7fbbe40a0a30 2023-10-03T10:15:41.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.493+0000 7fbbe17fa700 1 -- 172.21.15.3:0/1200825766 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd800be50 con 0x7fbbe40a0a30 2023-10-03T10:15:41.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.493+0000 7fbbeb1c7700 1 -- 172.21.15.3:0/1200825766 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe40a25e0 con 0x7fbbe40a0a30 2023-10-03T10:15:41.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.493+0000 7fbbe17fa700 1 -- 172.21.15.3:0/1200825766 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd80038e0 con 0x7fbbe40a0a30 2023-10-03T10:15:41.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.493+0000 7fbbeb1c7700 1 -- 172.21.15.3:0/1200825766 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe41221b0 con 0x7fbbe40a0a30 2023-10-03T10:15:41.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.494+0000 7fbbe17fa700 1 -- 172.21.15.3:0/1200825766 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbbd8022070 con 0x7fbbe40a0a30 2023-10-03T10:15:41.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.494+0000 7fbbeb1c7700 1 -- 172.21.15.3:0/1200825766 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe41bbfe0 con 0x7fbbe40a0a30 2023-10-03T10:15:41.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.494+0000 7fbbe17fa700 1 -- 172.21.15.3:0/1200825766 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fbbd8041180 con 0x7fbbe40a0a30 2023-10-03T10:15:41.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.501+0000 7f7fc0427700 1 Processor -- start 2023-10-03T10:15:41.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.501+0000 7f7fc0427700 1 -- start start 2023-10-03T10:15:41.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.502+0000 7f7fc0427700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7fb81338a0 0x7f7fb8133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:41.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.502+0000 7f7fc0427700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7fb809de50 0x7f7fb809e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:41.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.502+0000 7f7fc0427700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7fb809e760 0x7f7fb813a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:41.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.502+0000 7f7fc0427700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fb806fef0 con 0x7f7fb81338a0 2023-10-03T10:15:41.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.502+0000 7f7fc0427700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb806ef30 con 0x7f7fb809e760 2023-10-03T10:15:41.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.502+0000 7f7fc0427700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb81334b0 con 0x7f7fb809de50 2023-10-03T10:15:41.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.502+0000 7f7fbe1c3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7fb81338a0 0x7f7fb8133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:41.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.502+0000 7f7fbd9c2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7fb809de50 0x7f7fb809e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:41.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.502+0000 7fbbe17fa700 1 -- 172.21.15.3:0/1200825766 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbbd8003a40 con 0x7fbbe40a0a30 2023-10-03T10:15:41.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.502+0000 7f7fbe1c3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7fb81338a0 0x7f7fb8133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42312/0 (socket says 172.21.15.3:42312) 2023-10-03T10:15:41.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.502+0000 7f7fbd9c2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7fb809de50 0x7f7fb809e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42084/0 (socket says 172.21.15.3:42084) 2023-10-03T10:15:41.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.502+0000 7f7fbe1c3700 1 -- 172.21.15.3:0/116084014 learned_addr learned my addr 172.21.15.3:0/116084014 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:41.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.503+0000 7f7fbe9c4700 1 --2- 172.21.15.3:0/116084014 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7fb809e760 0x7f7fb813a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:41.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.503+0000 7f7fbe1c3700 1 -- 172.21.15.3:0/116084014 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7fb809de50 msgr2=0x7f7fb809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:41.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.503+0000 7f7fbe1c3700 1 --2- 172.21.15.3:0/116084014 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7fb809de50 0x7f7fb809e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.503+0000 7f7fbe1c3700 1 -- 172.21.15.3:0/116084014 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7fb809e760 msgr2=0x7f7fb813a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:41.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.503+0000 7f7fbe1c3700 1 --2- 172.21.15.3:0/116084014 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7fb809e760 0x7f7fb813a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.503+0000 7f7fbe1c3700 1 -- 172.21.15.3:0/116084014 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb806fc30 con 0x7f7fb81338a0 2023-10-03T10:15:41.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.503+0000 7f7fbe9c4700 1 --2- 172.21.15.3:0/116084014 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7fb809e760 0x7f7fb813a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:15:41.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.503+0000 7f7fbe1c3700 1 --2- 172.21.15.3:0/116084014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fb81338a0 0x7f7fb8133c70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f7fb40098f0 tx=0x7f7fb4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d12b968dc4caf9a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:41.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.503+0000 7f7fbd1c1700 1 -- 172.21.15.3:0/116084014 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb400b8a0 con 0x7f7fb81338a0 2023-10-03T10:15:41.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.503+0000 7f7fbd1c1700 1 -- 172.21.15.3:0/116084014 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb400ba00 con 0x7f7fb81338a0 2023-10-03T10:15:41.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.503+0000 7f7fbd1c1700 1 -- 172.21.15.3:0/116084014 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb400bd90 con 0x7f7fb81338a0 2023-10-03T10:15:41.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.504+0000 7f7fc0427700 1 -- 172.21.15.3:0/116084014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fb81338a0 msgr2=0x7f7fb8133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:41.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.504+0000 7f7fc0427700 1 --2- 172.21.15.3:0/116084014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fb81338a0 0x7f7fb8133c70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f7fb40098f0 tx=0x7f7fb4009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:41.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.504+0000 7f7fc0427700 1 -- 172.21.15.3:0/116084014 shutdown_connections 2023-10-03T10:15:41.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.504+0000 7f7fc0427700 1 --2- 172.21.15.3:0/116084014 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7fb809e760 0x7f7fb813a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.504+0000 7f7fc0427700 1 --2- 172.21.15.3:0/116084014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fb81338a0 0x7f7fb8133c70 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.504+0000 7f7fc0427700 1 --2- 172.21.15.3:0/116084014 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7fb809de50 0x7f7fb809e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.504+0000 7f7fc0427700 1 -- 172.21.15.3:0/116084014 >> 172.21.15.3:0/116084014 conn(0x7f7fb80a2aa0 msgr2=0x7f7fb80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:41.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.504+0000 7f7fc0427700 1 -- 172.21.15.3:0/116084014 shutdown_connections 2023-10-03T10:15:41.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.504+0000 7f7fc0427700 1 -- 172.21.15.3:0/116084014 wait complete. 2023-10-03T10:15:41.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.505+0000 7f7fc0427700 1 Processor -- start 2023-10-03T10:15:41.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.505+0000 7f7fc0427700 1 -- start start 2023-10-03T10:15:41.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.506+0000 7f7fc0427700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fb809de50 0x7f7fb811b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.506+0000 7f7fc0427700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7fb809e760 0x7f7fb811ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.506+0000 7f7fc0427700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7fb81338a0 0x7f7fb811bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.506+0000 7f7fc0427700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb811f9f0 con 0x7f7fb809de50 2023-10-03T10:15:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.506+0000 7f7fc0427700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb811f030 con 0x7f7fb81338a0 2023-10-03T10:15:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.506+0000 7f7fc0427700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb811e7c0 con 0x7f7fb809e760 2023-10-03T10:15:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.506+0000 7f7fbd9c2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7fb809e760 0x7f7fb811ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.506+0000 7f7fbd9c2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7fb809e760 0x7f7fb811ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42088/0 (socket says 172.21.15.3:42088) 2023-10-03T10:15:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.506+0000 7f7fbd9c2700 1 -- 172.21.15.3:0/4268669758 learned_addr learned my addr 172.21.15.3:0/4268669758 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.506+0000 7f7fbe9c4700 1 --2- 172.21.15.3:0/4268669758 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7fb81338a0 0x7f7fb811bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.506+0000 7f7fbe1c3700 1 --2- 172.21.15.3:0/4268669758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fb809de50 0x7f7fb811b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:41.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.506+0000 7f7fbd9c2700 1 -- 172.21.15.3:0/4268669758 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7fb81338a0 msgr2=0x7f7fb811bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:41.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.506+0000 7f7fbd9c2700 1 --2- 172.21.15.3:0/4268669758 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7fb81338a0 0x7f7fb811bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.506+0000 7f7fbd9c2700 1 -- 172.21.15.3:0/4268669758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fb809de50 msgr2=0x7f7fb811b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:41.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.506+0000 7f7fbd9c2700 1 --2- 172.21.15.3:0/4268669758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fb809de50 0x7f7fb811b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.506+0000 7f7fbd9c2700 1 -- 172.21.15.3:0/4268669758 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb4009550 con 0x7f7fb809e760 2023-10-03T10:15:41.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.506+0000 7f7fbe9c4700 1 --2- 172.21.15.3:0/4268669758 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7fb81338a0 0x7f7fb811bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:41.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.507+0000 7f7fbd9c2700 1 --2- 172.21.15.3:0/4268669758 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7fb809e760 0x7f7fb811ba50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800d830 tx=0x7f7fa800db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:41.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.507+0000 7f7faeffd700 1 -- 172.21.15.3:0/4268669758 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa80125f0 con 0x7f7fb809e760 2023-10-03T10:15:41.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.507+0000 7f7faeffd700 1 -- 172.21.15.3:0/4268669758 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa8012750 con 0x7f7fb809e760 2023-10-03T10:15:41.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.507+0000 7f7faeffd700 1 -- 172.21.15.3:0/4268669758 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa8012b00 con 0x7f7fb809e760 2023-10-03T10:15:41.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.507+0000 7f7fc0427700 1 -- 172.21.15.3:0/4268669758 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb811e370 con 0x7f7fb809e760 2023-10-03T10:15:41.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.507+0000 7f7fc0427700 1 -- 172.21.15.3:0/4268669758 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb80a0fa0 con 0x7f7fb809e760 2023-10-03T10:15:41.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.508+0000 7f7fc0427700 1 -- 172.21.15.3:0/4268669758 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb8074ea0 con 0x7f7fb809e760 2023-10-03T10:15:41.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.508+0000 7f7faeffd700 1 -- 172.21.15.3:0/4268669758 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7fa8012c60 con 0x7f7fb809e760 2023-10-03T10:15:41.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.509+0000 7f7faeffd700 1 -- 172.21.15.3:0/4268669758 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7fa80401c0 con 0x7f7fb809e760 2023-10-03T10:15:41.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.513+0000 7f7faeffd700 1 -- 172.21.15.3:0/4268669758 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7fa801b770 con 0x7f7fb809e760 2023-10-03T10:15:41.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.676+0000 7feffa77f700 1 -- 172.21.15.3:0/1083580110 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7feff4074ea0 con 0x7feff4133fa0 2023-10-03T10:15:41.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.677+0000 7feff0ff9700 1 -- 172.21.15.3:0/1083580110 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fefec02a030 con 0x7feff4133fa0 2023-10-03T10:15:41.677 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:41.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.679+0000 7feffa77f700 1 -- 172.21.15.3:0/1083580110 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feff4133fa0 msgr2=0x7feff4120e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:41.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.679+0000 7feffa77f700 1 --2- 172.21.15.3:0/1083580110 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feff4133fa0 0x7feff4120e10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fefec015000 tx=0x7fefec004ad0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.679+0000 7feffa77f700 1 -- 172.21.15.3:0/1083580110 shutdown_connections 2023-10-03T10:15:41.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.679+0000 7feffa77f700 1 --2- 172.21.15.3:0/1083580110 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feff4133fa0 0x7feff4120e10 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.679+0000 7feffa77f700 1 --2- 172.21.15.3:0/1083580110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feff40a0a30 0x7feff4120390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.679+0000 7feffa77f700 1 --2- 172.21.15.3:0/1083580110 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feff40a1340 0x7feff41208d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.679+0000 7feffa77f700 1 -- 172.21.15.3:0/1083580110 >> 172.21.15.3:0/1083580110 conn(0x7feff40a6550 msgr2=0x7feff409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:41.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.679+0000 7feffa77f700 1 -- 172.21.15.3:0/1083580110 shutdown_connections 2023-10-03T10:15:41.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.679+0000 7feffa77f700 1 -- 172.21.15.3:0/1083580110 wait complete. 2023-10-03T10:15:41.693 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:41.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.709+0000 7f0e9a53e700 1 -- 172.21.15.3:0/2021231177 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0e94074ea0 con 0x7f0e9413e2f0 2023-10-03T10:15:41.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.709+0000 7f0e90ff9700 1 -- 172.21.15.3:0/2021231177 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0e8c017030 con 0x7f0e9413e2f0 2023-10-03T10:15:41.709 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:41.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.711+0000 7f0e9a53e700 1 -- 172.21.15.3:0/2021231177 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0e9413e2f0 msgr2=0x7f0e94132b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:41.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.711+0000 7f0e9a53e700 1 --2- 172.21.15.3:0/2021231177 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0e9413e2f0 0x7f0e94132b90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c00c840 tx=0x7f0e8c00cb50 comp rx=0 tx=0).stop 2023-10-03T10:15:41.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.711+0000 7f0e9a53e700 1 -- 172.21.15.3:0/2021231177 shutdown_connections 2023-10-03T10:15:41.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.711+0000 7f0e9a53e700 1 --2- 172.21.15.3:0/2021231177 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0e9413e2f0 0x7f0e94132b90 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.711+0000 7f0e9a53e700 1 --2- 172.21.15.3:0/2021231177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e9406e570 0x7f0e9413b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.711+0000 7f0e9a53e700 1 --2- 172.21.15.3:0/2021231177 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e94134610 0x7f0e94132650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.711+0000 7f0e9a53e700 1 -- 172.21.15.3:0/2021231177 >> 172.21.15.3:0/2021231177 conn(0x7f0e940a2aa0 msgr2=0x7f0e94138000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:41.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.712+0000 7f0e9a53e700 1 -- 172.21.15.3:0/2021231177 shutdown_connections 2023-10-03T10:15:41.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.712+0000 7f0e9a53e700 1 -- 172.21.15.3:0/2021231177 wait complete. 2023-10-03T10:15:41.726 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:41.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.727+0000 7fbbeb1c7700 1 -- 172.21.15.3:0/1200825766 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbbe4074ea0 con 0x7fbbe40a0a30 2023-10-03T10:15:41.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.728+0000 7fbbe17fa700 1 -- 172.21.15.3:0/1200825766 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbbd8033020 con 0x7fbbe40a0a30 2023-10-03T10:15:41.728 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:41.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.730+0000 7fbbeb1c7700 1 -- 172.21.15.3:0/1200825766 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbbe40a0a30 msgr2=0x7fbbe411f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:41.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.730+0000 7fbbeb1c7700 1 --2- 172.21.15.3:0/1200825766 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbbe40a0a30 0x7fbbe411f5a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009130 tx=0x7fbbd8004750 comp rx=0 tx=0).stop 2023-10-03T10:15:41.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.730+0000 7fbbeb1c7700 1 -- 172.21.15.3:0/1200825766 shutdown_connections 2023-10-03T10:15:41.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.730+0000 7fbbeb1c7700 1 --2- 172.21.15.3:0/1200825766 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbbe4133fa0 0x7fbbe41c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.730+0000 7fbbeb1c7700 1 --2- 172.21.15.3:0/1200825766 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbbe40a0a30 0x7fbbe411f5a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.730+0000 7fbbeb1c7700 1 --2- 172.21.15.3:0/1200825766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbe40a1340 0x7fbbe41c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.730+0000 7fbbeb1c7700 1 -- 172.21.15.3:0/1200825766 >> 172.21.15.3:0/1200825766 conn(0x7fbbe40a6550 msgr2=0x7fbbe41b9120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:41.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.730+0000 7fbbeb1c7700 1 -- 172.21.15.3:0/1200825766 shutdown_connections 2023-10-03T10:15:41.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.730+0000 7fbbeb1c7700 1 -- 172.21.15.3:0/1200825766 wait complete. 2023-10-03T10:15:41.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.734+0000 7f7fc0427700 1 -- 172.21.15.3:0/4268669758 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7fb806b9e0 con 0x7f7fb809e760 2023-10-03T10:15:41.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.734+0000 7f7faeffd700 1 -- 172.21.15.3:0/4268669758 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7fa801b360 con 0x7f7fb809e760 2023-10-03T10:15:41.734 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:41.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.737+0000 7f7fc0427700 1 -- 172.21.15.3:0/4268669758 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7fb809e760 msgr2=0x7f7fb811ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:41.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.737+0000 7f7fc0427700 1 --2- 172.21.15.3:0/4268669758 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7fb809e760 0x7f7fb811ba50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800d830 tx=0x7f7fa800db40 comp rx=0 tx=0).stop 2023-10-03T10:15:41.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.737+0000 7f7fc0427700 1 -- 172.21.15.3:0/4268669758 shutdown_connections 2023-10-03T10:15:41.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.737+0000 7f7fc0427700 1 --2- 172.21.15.3:0/4268669758 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7fb81338a0 0x7f7fb811bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.737+0000 7f7fc0427700 1 --2- 172.21.15.3:0/4268669758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fb809de50 0x7f7fb811b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.737+0000 7f7fc0427700 1 --2- 172.21.15.3:0/4268669758 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7fb809e760 0x7f7fb811ba50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:41.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.737+0000 7f7fc0427700 1 -- 172.21.15.3:0/4268669758 >> 172.21.15.3:0/4268669758 conn(0x7f7fb80a2aa0 msgr2=0x7f7fb8130430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:41.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.737+0000 7f7fc0427700 1 -- 172.21.15.3:0/4268669758 shutdown_connections 2023-10-03T10:15:41.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:41.737+0000 7f7fc0427700 1 -- 172.21.15.3:0/4268669758 wait complete. 2023-10-03T10:15:41.745 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:41.751 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:42.694 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:42.727 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:42.746 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:42.752 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:42.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.861+0000 7f7acd9b5700 1 Processor -- start 2023-10-03T10:15:42.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.861+0000 7f7acd9b5700 1 -- start start 2023-10-03T10:15:42.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.862+0000 7f7acd9b5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7ac806a770 0x7f7ac806ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:42.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.862+0000 7f7acd9b5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ac8131d10 0x7f7ac8139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:42.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.862+0000 7f7acd9b5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ac809ece0 con 0x7f7ac806b080 2023-10-03T10:15:42.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.862+0000 7f7acd9b5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ac809e760 con 0x7f7ac806a770 2023-10-03T10:15:42.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.862+0000 7f7acd9b5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac809e040 con 0x7f7ac8131d10 2023-10-03T10:15:42.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.862+0000 7f7ac6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7ac806a770 0x7f7ac806ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:42.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.862+0000 7f7ac67fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7ac806b080 0x7f7ac8131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34480/0 (socket says 172.21.15.3:34480) 2023-10-03T10:15:42.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.862+0000 7f7ac6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7ac806a770 0x7f7ac806ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42326/0 (socket says 172.21.15.3:42326) 2023-10-03T10:15:42.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.862+0000 7f7ac67fc700 1 -- 172.21.15.3:0/1488613372 learned_addr learned my addr 172.21.15.3:0/1488613372 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:42.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.862+0000 7f7ac77fe700 1 --2- 172.21.15.3:0/1488613372 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ac8131d10 0x7f7ac8139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:42.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.862+0000 7f7ac5ffb700 1 -- 172.21.15.3:0/1488613372 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 509146518 0 0) 0x7f7ac809ece0 con 0x7f7ac806b080 2023-10-03T10:15:42.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.862+0000 7f7ac5ffb700 1 -- 172.21.15.3:0/1488613372 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ab00035b0 con 0x7f7ac806b080 2023-10-03T10:15:42.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.863+0000 7f7ac6ffd700 1 -- 172.21.15.3:0/1488613372 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ac8131d10 msgr2=0x7f7ac8139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:42.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.863+0000 7f7ac6ffd700 1 --2- 172.21.15.3:0/1488613372 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ac8131d10 0x7f7ac8139860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:42.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.863+0000 7f7ac6ffd700 1 -- 172.21.15.3:0/1488613372 >> v1:172.21.15.3:6789/0 conn(0x7f7ac806b080 legacy=0x7f7ac8131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:42.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.863+0000 7f7ac6ffd700 1 -- 172.21.15.3:0/1488613372 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac809f630 con 0x7f7ac806a770 2023-10-03T10:15:42.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.863+0000 7f7ac6ffd700 1 --2- 172.21.15.3:0/1488613372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ac806a770 0x7f7ac806ab40 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009840 tx=0x7f7ab8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ce3788400a50b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:42.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.863+0000 7f7ac5ffb700 1 -- 172.21.15.3:0/1488613372 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8010a90 con 0x7f7ac806a770 2023-10-03T10:15:42.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.863+0000 7f7ac5ffb700 1 -- 172.21.15.3:0/1488613372 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab8010bf0 con 0x7f7ac806a770 2023-10-03T10:15:42.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.863+0000 7f7ac5ffb700 1 -- 172.21.15.3:0/1488613372 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8018440 con 0x7f7ac806a770 2023-10-03T10:15:42.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.864+0000 7f7acd9b5700 1 -- 172.21.15.3:0/1488613372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ac806a770 msgr2=0x7f7ac806ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:42.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.864+0000 7f7acd9b5700 1 --2- 172.21.15.3:0/1488613372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ac806a770 0x7f7ac806ab40 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009840 tx=0x7f7ab8009b50 comp rx=0 tx=0).stop 2023-10-03T10:15:42.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.864+0000 7f7acd9b5700 1 -- 172.21.15.3:0/1488613372 shutdown_connections 2023-10-03T10:15:42.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.864+0000 7f7acd9b5700 1 --2- 172.21.15.3:0/1488613372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ac806a770 0x7f7ac806ab40 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:42.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.864+0000 7f7acd9b5700 1 --2- 172.21.15.3:0/1488613372 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ac8131d10 0x7f7ac8139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:42.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.864+0000 7f7acd9b5700 1 -- 172.21.15.3:0/1488613372 >> 172.21.15.3:0/1488613372 conn(0x7f7ac8066f50 msgr2=0x7f7ac809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:42.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.864+0000 7f7acd9b5700 1 -- 172.21.15.3:0/1488613372 shutdown_connections 2023-10-03T10:15:42.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.864+0000 7f7acd9b5700 1 -- 172.21.15.3:0/1488613372 wait complete. 2023-10-03T10:15:42.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.865+0000 7f7acd9b5700 1 Processor -- start 2023-10-03T10:15:42.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.865+0000 7f7acd9b5700 1 -- start start 2023-10-03T10:15:42.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.866+0000 7f7acd9b5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ac806a770 0x7f7ac811e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:42.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.866+0000 7f7acd9b5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ac806b080 0x7f7ac811e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:42.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.866+0000 7f7acd9b5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ac8131d10 0x7f7ac811ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:42.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.866+0000 7f7acd9b5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac8121d40 con 0x7f7ac806a770 2023-10-03T10:15:42.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.866+0000 7f7acd9b5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac8131200 con 0x7f7ac806b080 2023-10-03T10:15:42.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.866+0000 7f7acd9b5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac811fed0 con 0x7f7ac8131d10 2023-10-03T10:15:42.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.866+0000 7f7ac67fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ac806b080 0x7f7ac811e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:42.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.866+0000 7f7ac67fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ac806b080 0x7f7ac811e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50684/0 (socket says 172.21.15.3:50684) 2023-10-03T10:15:42.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.866+0000 7f7ac67fc700 1 -- 172.21.15.3:0/2710631846 learned_addr learned my addr 172.21.15.3:0/2710631846 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:42.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.866+0000 7f7ac6ffd700 1 --2- 172.21.15.3:0/2710631846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ac806a770 0x7f7ac811e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:42.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.866+0000 7f7ac77fe700 1 --2- 172.21.15.3:0/2710631846 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ac8131d10 0x7f7ac811ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:42.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.867+0000 7f7ac67fc700 1 -- 172.21.15.3:0/2710631846 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ac8131d10 msgr2=0x7f7ac811ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:42.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.867+0000 7f7ac67fc700 1 --2- 172.21.15.3:0/2710631846 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ac8131d10 0x7f7ac811ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:42.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.867+0000 7f7ac67fc700 1 -- 172.21.15.3:0/2710631846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ac806a770 msgr2=0x7f7ac811e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:42.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.867+0000 7f7ac67fc700 1 --2- 172.21.15.3:0/2710631846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ac806a770 0x7f7ac811e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:42.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.867+0000 7f7ac67fc700 1 -- 172.21.15.3:0/2710631846 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab80094a0 con 0x7f7ac806b080 2023-10-03T10:15:42.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.867+0000 7f7ac6ffd700 1 --2- 172.21.15.3:0/2710631846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ac806a770 0x7f7ac811e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:42.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.867+0000 7f7ac67fc700 1 --2- 172.21.15.3:0/2710631846 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ac806b080 0x7f7ac811e7b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00c700 tx=0x7f7abc00ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:42.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.867+0000 7f7aaffff700 1 -- 172.21.15.3:0/2710631846 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc005d70 con 0x7f7ac806b080 2023-10-03T10:15:42.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.867+0000 7f7aaffff700 1 -- 172.21.15.3:0/2710631846 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7abc0070d0 con 0x7f7ac806b080 2023-10-03T10:15:42.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.867+0000 7f7acd9b5700 1 -- 172.21.15.3:0/2710631846 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ac80a2980 con 0x7f7ac806b080 2023-10-03T10:15:42.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.867+0000 7f7acd9b5700 1 -- 172.21.15.3:0/2710631846 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac81224d0 con 0x7f7ac806b080 2023-10-03T10:15:42.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.867+0000 7f7aaffff700 1 -- 172.21.15.3:0/2710631846 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc007440 con 0x7f7ac806b080 2023-10-03T10:15:42.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.868+0000 7f7aaffff700 1 -- 172.21.15.3:0/2710631846 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7abc0075a0 con 0x7f7ac806b080 2023-10-03T10:15:42.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.869+0000 7f7acd9b5700 1 -- 172.21.15.3:0/2710631846 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ac81217f0 con 0x7f7ac806b080 2023-10-03T10:15:42.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.869+0000 7f7aaffff700 1 -- 172.21.15.3:0/2710631846 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7abc03ebd0 con 0x7f7ac806b080 2023-10-03T10:15:42.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.874+0000 7f7aaffff700 1 -- 172.21.15.3:0/2710631846 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7abc017020 con 0x7f7ac806b080 2023-10-03T10:15:42.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.899+0000 7f472e4c4700 1 Processor -- start 2023-10-03T10:15:42.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.899+0000 7f472e4c4700 1 -- start start 2023-10-03T10:15:42.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.900+0000 7f472e4c4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f47280a0a30 0x7f47280a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:42.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.900+0000 7f472e4c4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4728134170 0x7f472813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:42.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.900+0000 7f472e4c4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f472809edd0 con 0x7f47280a1340 2023-10-03T10:15:42.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.900+0000 7f472e4c4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f472809e1f0 con 0x7f47280a0a30 2023-10-03T10:15:42.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.900+0000 7f472e4c4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f472809dc50 con 0x7f4728134170 2023-10-03T10:15:42.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.900+0000 7f472ca61700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4728134170 0x7f472813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:42.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.900+0000 7f472ca61700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4728134170 0x7f472813bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50698/0 (socket says 172.21.15.3:50698) 2023-10-03T10:15:42.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.900+0000 7f472ca61700 1 -- 172.21.15.3:0/2034883529 learned_addr learned my addr 172.21.15.3:0/2034883529 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:42.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.900+0000 7f4727fff700 1 --2- 172.21.15.3:0/2034883529 >> v2:172.21.15.3:3300/0 conn(0x7f47280a0a30 0x7f47280a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:42.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.901+0000 7f472ca61700 1 -- 172.21.15.3:0/2034883529 >> v2:172.21.15.3:3300/0 conn(0x7f47280a0a30 msgr2=0x7f47280a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:42.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.901+0000 7f472ca61700 1 --2- 172.21.15.3:0/2034883529 >> v2:172.21.15.3:3300/0 conn(0x7f47280a0a30 0x7f47280a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:42.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.901+0000 7f472ca61700 1 -- 172.21.15.3:0/2034883529 >> v1:172.21.15.3:6789/0 conn(0x7f47280a1340 legacy=0x7f4728133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:42.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.901+0000 7f472ca61700 1 -- 172.21.15.3:0/2034883529 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47280a1fe0 con 0x7f4728134170 2023-10-03T10:15:42.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.901+0000 7f472ca61700 1 --2- 172.21.15.3:0/2034883529 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4728134170 0x7f472813bd30 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4720009e50 tx=0x7f472000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=435ded45099230a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:42.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.901+0000 7f4726ffd700 1 -- 172.21.15.3:0/2034883529 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472000e070 con 0x7f4728134170 2023-10-03T10:15:42.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.901+0000 7f4726ffd700 1 -- 172.21.15.3:0/2034883529 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f472000b910 con 0x7f4728134170 2023-10-03T10:15:42.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.901+0000 7f4726ffd700 1 -- 172.21.15.3:0/2034883529 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472000bc80 con 0x7f4728134170 2023-10-03T10:15:42.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.902+0000 7f472e4c4700 1 -- 172.21.15.3:0/2034883529 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4728134170 msgr2=0x7f472813bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:42.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.902+0000 7f472e4c4700 1 --2- 172.21.15.3:0/2034883529 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4728134170 0x7f472813bd30 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4720009e50 tx=0x7f472000b330 comp rx=0 tx=0).stop 2023-10-03T10:15:42.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.902+0000 7f472e4c4700 1 -- 172.21.15.3:0/2034883529 shutdown_connections 2023-10-03T10:15:42.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.902+0000 7f472e4c4700 1 --2- 172.21.15.3:0/2034883529 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4728134170 0x7f472813bd30 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:42.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.902+0000 7f472e4c4700 1 --2- 172.21.15.3:0/2034883529 >> v2:172.21.15.3:3300/0 conn(0x7f47280a0a30 0x7f47280a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:42.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.902+0000 7f472e4c4700 1 -- 172.21.15.3:0/2034883529 >> 172.21.15.3:0/2034883529 conn(0x7f47280a6550 msgr2=0x7f47280a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:42.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.903+0000 7f472e4c4700 1 -- 172.21.15.3:0/2034883529 shutdown_connections 2023-10-03T10:15:42.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.903+0000 7f472e4c4700 1 -- 172.21.15.3:0/2034883529 wait complete. 2023-10-03T10:15:42.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.904+0000 7f472e4c4700 1 Processor -- start 2023-10-03T10:15:42.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.904+0000 7f472e4c4700 1 -- start start 2023-10-03T10:15:42.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.904+0000 7f472e4c4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47280a0a30 0x7f472811e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:42.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.904+0000 7f472e4c4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47280a1340 0x7f472811ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:42.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.904+0000 7f472e4c4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4728134170 0x7f472811f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:42.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.904+0000 7f472e4c4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47280a4e60 con 0x7f47280a0a30 2023-10-03T10:15:42.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.904+0000 7f472e4c4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47280a0220 con 0x7f4728134170 2023-10-03T10:15:42.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.904+0000 7f472e4c4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f47281209c0 con 0x7f47280a1340 2023-10-03T10:15:42.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.905+0000 7f472ca61700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4728134170 0x7f472811f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:42.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.905+0000 7f472ca61700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4728134170 0x7f472811f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50700/0 (socket says 172.21.15.3:50700) 2023-10-03T10:15:42.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.905+0000 7f472ca61700 1 -- 172.21.15.3:0/2775827213 learned_addr learned my addr 172.21.15.3:0/2775827213 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:42.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.905+0000 7f47277fe700 1 --2- 172.21.15.3:0/2775827213 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47280a1340 0x7f472811ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:42.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.905+0000 7f4727fff700 1 --2- 172.21.15.3:0/2775827213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47280a0a30 0x7f472811e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:42.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.905+0000 7f47277fe700 1 -- 172.21.15.3:0/2775827213 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4728134170 msgr2=0x7f472811f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:42.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.905+0000 7f47277fe700 1 --2- 172.21.15.3:0/2775827213 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4728134170 0x7f472811f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:42.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.905+0000 7f47277fe700 1 -- 172.21.15.3:0/2775827213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47280a0a30 msgr2=0x7f472811e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:42.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.905+0000 7f47277fe700 1 --2- 172.21.15.3:0/2775827213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47280a0a30 0x7f472811e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:42.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.905+0000 7f47277fe700 1 -- 172.21.15.3:0/2775827213 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4720009b30 con 0x7f47280a1340 2023-10-03T10:15:42.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.905+0000 7f4727fff700 1 --2- 172.21.15.3:0/2775827213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47280a0a30 0x7f472811e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:15:42.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.905+0000 7f47277fe700 1 --2- 172.21.15.3:0/2775827213 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47280a1340 0x7f472811ec50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f471c000f60 tx=0x7f471c00c730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:42.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.906+0000 7f4724ff9700 1 -- 172.21.15.3:0/2775827213 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c0052d0 con 0x7f47280a1340 2023-10-03T10:15:42.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.906+0000 7f4724ff9700 1 -- 172.21.15.3:0/2775827213 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f471c005430 con 0x7f47280a1340 2023-10-03T10:15:42.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.906+0000 7f472e4c4700 1 -- 172.21.15.3:0/2775827213 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47280a25e0 con 0x7f47280a1340 2023-10-03T10:15:42.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.906+0000 7f4724ff9700 1 -- 172.21.15.3:0/2775827213 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c0038c0 con 0x7f47280a1340 2023-10-03T10:15:42.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.906+0000 7f472e4c4700 1 -- 172.21.15.3:0/2775827213 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4728120200 con 0x7f47280a1340 2023-10-03T10:15:42.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.907+0000 7f472e4c4700 1 -- 172.21.15.3:0/2775827213 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4728122930 con 0x7f47280a1340 2023-10-03T10:15:42.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.908+0000 7f4724ff9700 1 -- 172.21.15.3:0/2775827213 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f471c003a20 con 0x7f47280a1340 2023-10-03T10:15:42.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.908+0000 7f4724ff9700 1 -- 172.21.15.3:0/2775827213 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f471c03ef10 con 0x7f47280a1340 2023-10-03T10:15:42.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.912+0000 7f4724ff9700 1 -- 172.21.15.3:0/2775827213 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f471c018260 con 0x7f47280a1340 2023-10-03T10:15:42.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.942+0000 7f347e8cf700 1 Processor -- start 2023-10-03T10:15:42.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.942+0000 7f347e8cf700 1 -- start start 2023-10-03T10:15:42.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.943+0000 7f347e8cf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f347809ac90 0x7f347813cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:42.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.943+0000 7f347e8cf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f347813d240 0x7f347813f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:42.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.943+0000 7f347e8cf700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3478134980 con 0x7f347806c0e0 2023-10-03T10:15:42.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.943+0000 7f347e8cf700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f347806c4b0 con 0x7f347813d240 2023-10-03T10:15:42.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.943+0000 7f347e8cf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f347809b0e0 con 0x7f347809ac90 2023-10-03T10:15:42.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.943+0000 7f3477fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f347806c0e0 0x7f3478134ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34500/0 (socket says 172.21.15.3:34500) 2023-10-03T10:15:42.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.943+0000 7f3477fff700 1 -- 172.21.15.3:0/3386043934 learned_addr learned my addr 172.21.15.3:0/3386043934 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:42.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.943+0000 7f34777fe700 1 --2- 172.21.15.3:0/3386043934 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f347809ac90 0x7f347813cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:42.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.943+0000 7f347ce6c700 1 --2- 172.21.15.3:0/3386043934 >> v2:172.21.15.3:3300/0 conn(0x7f347813d240 0x7f347813f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:42.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.943+0000 7f34777fe700 1 -- 172.21.15.3:0/3386043934 >> v2:172.21.15.3:3300/0 conn(0x7f347813d240 msgr2=0x7f347813f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:42.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.943+0000 7f34777fe700 1 --2- 172.21.15.3:0/3386043934 >> v2:172.21.15.3:3300/0 conn(0x7f347813d240 0x7f347813f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:42.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.943+0000 7f34777fe700 1 -- 172.21.15.3:0/3386043934 >> v1:172.21.15.3:6789/0 conn(0x7f347806c0e0 legacy=0x7f3478134ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:42.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.943+0000 7f34777fe700 1 -- 172.21.15.3:0/3386043934 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34780a0600 con 0x7f347809ac90 2023-10-03T10:15:42.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.943+0000 7f347ce6c700 1 --2- 172.21.15.3:0/3386043934 >> v2:172.21.15.3:3300/0 conn(0x7f347813d240 0x7f347813f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:42.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.944+0000 7f34777fe700 1 --2- 172.21.15.3:0/3386043934 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f347809ac90 0x7f347813cd00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f346c009840 tx=0x7f346c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fe5a0acd999ad70 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:42.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.944+0000 7f3476ffd700 1 -- 172.21.15.3:0/3386043934 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346c011830 con 0x7f347809ac90 2023-10-03T10:15:42.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.944+0000 7f3476ffd700 1 -- 172.21.15.3:0/3386043934 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f346c011990 con 0x7f347809ac90 2023-10-03T10:15:42.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.944+0000 7f3476ffd700 1 -- 172.21.15.3:0/3386043934 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346c011d20 con 0x7f347809ac90 2023-10-03T10:15:42.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.945+0000 7f347e8cf700 1 -- 172.21.15.3:0/3386043934 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f347809ac90 msgr2=0x7f347813cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:42.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.945+0000 7f347e8cf700 1 --2- 172.21.15.3:0/3386043934 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f347809ac90 0x7f347813cd00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f346c009840 tx=0x7f346c009b50 comp rx=0 tx=0).stop 2023-10-03T10:15:42.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.945+0000 7f347e8cf700 1 -- 172.21.15.3:0/3386043934 shutdown_connections 2023-10-03T10:15:42.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.945+0000 7f347e8cf700 1 --2- 172.21.15.3:0/3386043934 >> v2:172.21.15.3:3300/0 conn(0x7f347813d240 0x7f347813f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:42.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.945+0000 7f347e8cf700 1 --2- 172.21.15.3:0/3386043934 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f347809ac90 0x7f347813cd00 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:42.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.945+0000 7f347e8cf700 1 -- 172.21.15.3:0/3386043934 >> 172.21.15.3:0/3386043934 conn(0x7f34780a4210 msgr2=0x7f34780a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:42.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.945+0000 7f347e8cf700 1 -- 172.21.15.3:0/3386043934 shutdown_connections 2023-10-03T10:15:42.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.945+0000 7f347e8cf700 1 -- 172.21.15.3:0/3386043934 wait complete. 2023-10-03T10:15:42.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.946+0000 7f347e8cf700 1 Processor -- start 2023-10-03T10:15:42.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.946+0000 7f347e8cf700 1 -- start start 2023-10-03T10:15:42.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.946+0000 7f347e8cf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f347806c0e0 0x7f3478134010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:42.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.946+0000 7f347e8cf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f347809ac90 0x7f34781d88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:42.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.946+0000 7f347e8cf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f347813d240 0x7f34781dadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:42.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.946+0000 7f347e8cf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347813f630 con 0x7f347813d240 2023-10-03T10:15:42.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.946+0000 7f347e8cf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3478138120 con 0x7f347806c0e0 2023-10-03T10:15:42.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.946+0000 7f347e8cf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3478138d20 con 0x7f347809ac90 2023-10-03T10:15:42.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.947+0000 7f34777fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f347809ac90 0x7f34781d88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:42.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.947+0000 7f34777fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f347809ac90 0x7f34781d88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42132/0 (socket says 172.21.15.3:42132) 2023-10-03T10:15:42.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.947+0000 7f34777fe700 1 -- 172.21.15.3:0/3590088532 learned_addr learned my addr 172.21.15.3:0/3590088532 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.947+0000 7f347ce6c700 1 --2- 172.21.15.3:0/3590088532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f347813d240 0x7f34781dadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.947+0000 7f3477fff700 1 --2- 172.21.15.3:0/3590088532 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f347806c0e0 0x7f3478134010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.947+0000 7f34777fe700 1 -- 172.21.15.3:0/3590088532 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f347806c0e0 msgr2=0x7f3478134010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.947+0000 7f34777fe700 1 --2- 172.21.15.3:0/3590088532 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f347806c0e0 0x7f3478134010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.947+0000 7f34777fe700 1 -- 172.21.15.3:0/3590088532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f347813d240 msgr2=0x7f34781dadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.947+0000 7f34777fe700 1 --2- 172.21.15.3:0/3590088532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f347813d240 0x7f34781dadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.947+0000 7f34777fe700 1 -- 172.21.15.3:0/3590088532 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3470009a70 con 0x7f347809ac90 2023-10-03T10:15:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.947+0000 7f34777fe700 1 --2- 172.21.15.3:0/3590088532 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f347809ac90 0x7f34781d88a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f346c0090b0 tx=0x7f346c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.947+0000 7f3474ff9700 1 -- 172.21.15.3:0/3590088532 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346c0036a0 con 0x7f347809ac90 2023-10-03T10:15:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.948+0000 7f3474ff9700 1 -- 172.21.15.3:0/3590088532 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f346c01e070 con 0x7f347809ac90 2023-10-03T10:15:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.948+0000 7f3474ff9700 1 -- 172.21.15.3:0/3590088532 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346c003a60 con 0x7f347809ac90 2023-10-03T10:15:42.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.948+0000 7f347e8cf700 1 -- 172.21.15.3:0/3590088532 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f346c0094a0 con 0x7f347809ac90 2023-10-03T10:15:42.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.948+0000 7f347e8cf700 1 -- 172.21.15.3:0/3590088532 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f347809e310 con 0x7f347809ac90 2023-10-03T10:15:42.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.949+0000 7f3474ff9700 1 -- 172.21.15.3:0/3590088532 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f346c003bc0 con 0x7f347809ac90 2023-10-03T10:15:42.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.949+0000 7f347e8cf700 1 -- 172.21.15.3:0/3590088532 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f347809dc80 con 0x7f347809ac90 2023-10-03T10:15:42.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.949+0000 7f3474ff9700 1 -- 172.21.15.3:0/3590088532 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f346c020070 con 0x7f347809ac90 2023-10-03T10:15:42.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.951+0000 7fb3e3221700 1 Processor -- start 2023-10-03T10:15:42.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.951+0000 7fb3e3221700 1 -- start start 2023-10-03T10:15:42.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.952+0000 7fb3e3221700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb3dc09e930 0x7fb3dc09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:42.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.952+0000 7fb3e3221700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb3dc09f240 0x7fb3dc1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:42.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.952+0000 7fb3e3221700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb3dc09f650 0x7fb3dc131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:42.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.952+0000 7fb3e3221700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3dc070940 con 0x7fb3dc09f650 2023-10-03T10:15:42.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.952+0000 7fb3e3221700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc06fd60 con 0x7fb3dc09e930 2023-10-03T10:15:42.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.952+0000 7fb3e3221700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc06fa80 con 0x7fb3dc09f240 2023-10-03T10:15:42.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.952+0000 7fb3e0fbd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb3dc09e930 0x7fb3dc09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:42.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.952+0000 7fb3e0fbd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb3dc09e930 0x7fb3dc09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50728/0 (socket says 172.21.15.3:50728) 2023-10-03T10:15:42.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.952+0000 7fb3e0fbd700 1 -- 172.21.15.3:0/1531793665 learned_addr learned my addr 172.21.15.3:0/1531793665 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:42.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.952+0000 7fb3dbfff700 1 --2- 172.21.15.3:0/1531793665 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb3dc09f240 0x7fb3dc1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:42.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.952+0000 7fb3e17be700 1 --2- 172.21.15.3:0/1531793665 >> v2:172.21.15.3:3300/0 conn(0x7fb3dc09f650 0x7fb3dc131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:42.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.952+0000 7fb3e0fbd700 1 -- 172.21.15.3:0/1531793665 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb3dc09f240 msgr2=0x7fb3dc1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:42.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.952+0000 7fb3e0fbd700 1 --2- 172.21.15.3:0/1531793665 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb3dc09f240 0x7fb3dc1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:42.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.952+0000 7fb3e0fbd700 1 -- 172.21.15.3:0/1531793665 >> v2:172.21.15.3:3300/0 conn(0x7fb3dc09f650 msgr2=0x7fb3dc131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:42.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.952+0000 7fb3e0fbd700 1 --2- 172.21.15.3:0/1531793665 >> v2:172.21.15.3:3300/0 conn(0x7fb3dc09f650 0x7fb3dc131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:42.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.952+0000 7fb3e0fbd700 1 -- 172.21.15.3:0/1531793665 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3dc09fb60 con 0x7fb3dc09e930 2023-10-03T10:15:42.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.952+0000 7fb3e0fbd700 1 --2- 172.21.15.3:0/1531793665 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb3dc09e930 0x7fb3dc09ed00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00098f0 tx=0x7fb3d0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3c3665653dd529f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:42.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.953+0000 7fb3db7fe700 1 -- 172.21.15.3:0/1531793665 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d000b8a0 con 0x7fb3dc09e930 2023-10-03T10:15:42.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.953+0000 7fb3db7fe700 1 -- 172.21.15.3:0/1531793665 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d000ba00 con 0x7fb3dc09e930 2023-10-03T10:15:42.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.953+0000 7fb3db7fe700 1 -- 172.21.15.3:0/1531793665 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d000bd90 con 0x7fb3dc09e930 2023-10-03T10:15:42.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.954+0000 7fb3e3221700 1 -- 172.21.15.3:0/1531793665 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb3dc09e930 msgr2=0x7fb3dc09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:42.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.954+0000 7fb3e3221700 1 --2- 172.21.15.3:0/1531793665 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb3dc09e930 0x7fb3dc09ed00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00098f0 tx=0x7fb3d0009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:42.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.954+0000 7fb3e3221700 1 -- 172.21.15.3:0/1531793665 shutdown_connections 2023-10-03T10:15:42.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.954+0000 7fb3e3221700 1 --2- 172.21.15.3:0/1531793665 >> v2:172.21.15.3:3300/0 conn(0x7fb3dc09f650 0x7fb3dc131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:42.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.954+0000 7fb3e3221700 1 --2- 172.21.15.3:0/1531793665 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb3dc09e930 0x7fb3dc09ed00 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:42.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.954+0000 7fb3e3221700 1 --2- 172.21.15.3:0/1531793665 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb3dc09f240 0x7fb3dc1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:42.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.954+0000 7fb3e3221700 1 -- 172.21.15.3:0/1531793665 >> 172.21.15.3:0/1531793665 conn(0x7fb3dc0a4210 msgr2=0x7fb3dc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:42.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.954+0000 7fb3e3221700 1 -- 172.21.15.3:0/1531793665 shutdown_connections 2023-10-03T10:15:42.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.954+0000 7fb3e3221700 1 -- 172.21.15.3:0/1531793665 wait complete. 2023-10-03T10:15:42.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.955+0000 7f3474ff9700 1 -- 172.21.15.3:0/3590088532 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f347809dc80 con 0x7f347809ac90 2023-10-03T10:15:42.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.955+0000 7fb3e3221700 1 Processor -- start 2023-10-03T10:15:42.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.955+0000 7fb3e3221700 1 -- start start 2023-10-03T10:15:42.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.956+0000 7fb3e3221700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb3dc09e930 0x7fb3dc1c8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:42.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.956+0000 7fb3e3221700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3dc09f240 0x7fb3dc1c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:42.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.956+0000 7fb3e3221700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb3dc09f650 0x7fb3dc1c4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:42.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.956+0000 7fb3e3221700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc136730 con 0x7fb3dc09f240 2023-10-03T10:15:42.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.956+0000 7fb3e3221700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc137cb0 con 0x7fb3dc09e930 2023-10-03T10:15:42.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.956+0000 7fb3e3221700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc0a33a0 con 0x7fb3dc09f650 2023-10-03T10:15:42.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.957+0000 7fb3e0fbd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb3dc09e930 0x7fb3dc1c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:42.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.957+0000 7fb3e0fbd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb3dc09e930 0x7fb3dc1c8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50738/0 (socket says 172.21.15.3:50738) 2023-10-03T10:15:42.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.957+0000 7fb3e0fbd700 1 -- 172.21.15.3:0/574291132 learned_addr learned my addr 172.21.15.3:0/574291132 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:42.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.957+0000 7fb3dbfff700 1 --2- 172.21.15.3:0/574291132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3dc09f240 0x7fb3dc1c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:42.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.957+0000 7fb3e17be700 1 --2- 172.21.15.3:0/574291132 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb3dc09f650 0x7fb3dc1c4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:42.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.957+0000 7fb3e0fbd700 1 -- 172.21.15.3:0/574291132 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb3dc09f650 msgr2=0x7fb3dc1c4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:42.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.957+0000 7fb3e0fbd700 1 --2- 172.21.15.3:0/574291132 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb3dc09f650 0x7fb3dc1c4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:42.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.957+0000 7fb3e0fbd700 1 -- 172.21.15.3:0/574291132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3dc09f240 msgr2=0x7fb3dc1c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:42.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.957+0000 7fb3e0fbd700 1 --2- 172.21.15.3:0/574291132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3dc09f240 0x7fb3dc1c8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:42.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.957+0000 7fb3e0fbd700 1 -- 172.21.15.3:0/574291132 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d0009550 con 0x7fb3dc09e930 2023-10-03T10:15:42.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.957+0000 7fb3e0fbd700 1 --2- 172.21.15.3:0/574291132 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb3dc09e930 0x7fb3dc1c8720 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0009c40 tx=0x7fb3d0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:42.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.957+0000 7fb3d97fa700 1 -- 172.21.15.3:0/574291132 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0004380 con 0x7fb3dc09e930 2023-10-03T10:15:42.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.957+0000 7fb3d97fa700 1 -- 172.21.15.3:0/574291132 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d001f070 con 0x7fb3dc09e930 2023-10-03T10:15:42.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.957+0000 7fb3d97fa700 1 -- 172.21.15.3:0/574291132 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d00036a0 con 0x7fb3dc09e930 2023-10-03T10:15:42.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.958+0000 7fb3e3221700 1 -- 172.21.15.3:0/574291132 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3dc0a0190 con 0x7fb3dc09e930 2023-10-03T10:15:42.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.958+0000 7fb3e3221700 1 -- 172.21.15.3:0/574291132 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3dc136e00 con 0x7fb3dc09e930 2023-10-03T10:15:42.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.959+0000 7fb3e3221700 1 -- 172.21.15.3:0/574291132 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3dc1386d0 con 0x7fb3dc09e930 2023-10-03T10:15:42.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.959+0000 7fb3d97fa700 1 -- 172.21.15.3:0/574291132 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb3d0021030 con 0x7fb3dc09e930 2023-10-03T10:15:42.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.959+0000 7fb3d97fa700 1 -- 172.21.15.3:0/574291132 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb3d00409f0 con 0x7fb3dc09e930 2023-10-03T10:15:42.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:42.964+0000 7fb3d97fa700 1 -- 172.21.15.3:0/574291132 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb3dc1386d0 con 0x7fb3dc09e930 2023-10-03T10:15:43.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.092+0000 7f7acd9b5700 1 -- 172.21.15.3:0/2710631846 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7ac8074ea0 con 0x7f7ac806b080 2023-10-03T10:15:43.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.092+0000 7f7aaffff700 1 -- 172.21.15.3:0/2710631846 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7abc019d20 con 0x7f7ac806b080 2023-10-03T10:15:43.093 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:43.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.095+0000 7f7acd9b5700 1 -- 172.21.15.3:0/2710631846 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ac806b080 msgr2=0x7f7ac811e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:43.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.095+0000 7f7acd9b5700 1 --2- 172.21.15.3:0/2710631846 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ac806b080 0x7f7ac811e7b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00c700 tx=0x7f7abc00ca10 comp rx=0 tx=0).stop 2023-10-03T10:15:43.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.095+0000 7f7acd9b5700 1 -- 172.21.15.3:0/2710631846 shutdown_connections 2023-10-03T10:15:43.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.095+0000 7f7acd9b5700 1 --2- 172.21.15.3:0/2710631846 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ac8131d10 0x7f7ac811ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:43.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.095+0000 7f7acd9b5700 1 --2- 172.21.15.3:0/2710631846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ac806a770 0x7f7ac811e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:43.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.095+0000 7f7acd9b5700 1 --2- 172.21.15.3:0/2710631846 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ac806b080 0x7f7ac811e7b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:43.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.095+0000 7f7acd9b5700 1 -- 172.21.15.3:0/2710631846 >> 172.21.15.3:0/2710631846 conn(0x7f7ac8066f50 msgr2=0x7f7ac80a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:43.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.095+0000 7f7acd9b5700 1 -- 172.21.15.3:0/2710631846 shutdown_connections 2023-10-03T10:15:43.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.095+0000 7f7acd9b5700 1 -- 172.21.15.3:0/2710631846 wait complete. 2023-10-03T10:15:43.110 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:43.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.129+0000 7f472e4c4700 1 -- 172.21.15.3:0/2775827213 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4728074ea0 con 0x7f47280a1340 2023-10-03T10:15:43.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.129+0000 7f4724ff9700 1 -- 172.21.15.3:0/2775827213 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f471c017030 con 0x7f47280a1340 2023-10-03T10:15:43.129 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:43.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.131+0000 7f472e4c4700 1 -- 172.21.15.3:0/2775827213 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47280a1340 msgr2=0x7f472811ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:43.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.131+0000 7f472e4c4700 1 --2- 172.21.15.3:0/2775827213 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47280a1340 0x7f472811ec50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f471c000f60 tx=0x7f471c00c730 comp rx=0 tx=0).stop 2023-10-03T10:15:43.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.131+0000 7f472e4c4700 1 -- 172.21.15.3:0/2775827213 shutdown_connections 2023-10-03T10:15:43.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.131+0000 7f472e4c4700 1 --2- 172.21.15.3:0/2775827213 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4728134170 0x7f472811f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:43.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.131+0000 7f472e4c4700 1 --2- 172.21.15.3:0/2775827213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47280a0a30 0x7f472811e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:43.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.131+0000 7f472e4c4700 1 --2- 172.21.15.3:0/2775827213 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47280a1340 0x7f472811ec50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:43.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.131+0000 7f472e4c4700 1 -- 172.21.15.3:0/2775827213 >> 172.21.15.3:0/2775827213 conn(0x7f47280a6550 msgr2=0x7f472809ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:43.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.132+0000 7f472e4c4700 1 -- 172.21.15.3:0/2775827213 shutdown_connections 2023-10-03T10:15:43.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.132+0000 7f472e4c4700 1 -- 172.21.15.3:0/2775827213 wait complete. 2023-10-03T10:15:43.146 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:43.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.181+0000 7f347e8cf700 1 -- 172.21.15.3:0/3590088532 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f347809e310 con 0x7f347809ac90 2023-10-03T10:15:43.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.181+0000 7f3474ff9700 1 -- 172.21.15.3:0/3590088532 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f347809e310 con 0x7f347809ac90 2023-10-03T10:15:43.182 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:43.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.184+0000 7f347e8cf700 1 -- 172.21.15.3:0/3590088532 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f347809ac90 msgr2=0x7f34781d88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:43.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.184+0000 7f347e8cf700 1 --2- 172.21.15.3:0/3590088532 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f347809ac90 0x7f34781d88a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f346c0090b0 tx=0x7f346c004750 comp rx=0 tx=0).stop 2023-10-03T10:15:43.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.184+0000 7f347e8cf700 1 -- 172.21.15.3:0/3590088532 shutdown_connections 2023-10-03T10:15:43.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.184+0000 7f347e8cf700 1 --2- 172.21.15.3:0/3590088532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f347813d240 0x7f34781dadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:43.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.184+0000 7f347e8cf700 1 --2- 172.21.15.3:0/3590088532 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f347806c0e0 0x7f3478134010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:43.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.184+0000 7f347e8cf700 1 --2- 172.21.15.3:0/3590088532 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f347809ac90 0x7f34781d88a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:43.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.184+0000 7f347e8cf700 1 -- 172.21.15.3:0/3590088532 >> 172.21.15.3:0/3590088532 conn(0x7f34780a4210 msgr2=0x7f34780a0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:43.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.184+0000 7f347e8cf700 1 -- 172.21.15.3:0/3590088532 shutdown_connections 2023-10-03T10:15:43.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.184+0000 7f347e8cf700 1 -- 172.21.15.3:0/3590088532 wait complete. 2023-10-03T10:15:43.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.186+0000 7fb3e3221700 1 -- 172.21.15.3:0/574291132 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb3dc1386d0 con 0x7fb3dc09e930 2023-10-03T10:15:43.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.186+0000 7fb3d97fa700 1 -- 172.21.15.3:0/574291132 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb3dc1386d0 con 0x7fb3dc09e930 2023-10-03T10:15:43.187 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:43.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.189+0000 7fb3e3221700 1 -- 172.21.15.3:0/574291132 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb3dc09e930 msgr2=0x7fb3dc1c8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:43.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.189+0000 7fb3e3221700 1 --2- 172.21.15.3:0/574291132 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb3dc09e930 0x7fb3dc1c8720 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0009c40 tx=0x7fb3d0004750 comp rx=0 tx=0).stop 2023-10-03T10:15:43.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.189+0000 7fb3e3221700 1 -- 172.21.15.3:0/574291132 shutdown_connections 2023-10-03T10:15:43.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.189+0000 7fb3e3221700 1 --2- 172.21.15.3:0/574291132 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb3dc09f650 0x7fb3dc1c4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:43.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.189+0000 7fb3e3221700 1 --2- 172.21.15.3:0/574291132 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb3dc09e930 0x7fb3dc1c8720 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:43.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.189+0000 7fb3e3221700 1 --2- 172.21.15.3:0/574291132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3dc09f240 0x7fb3dc1c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:43.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.189+0000 7fb3e3221700 1 -- 172.21.15.3:0/574291132 >> 172.21.15.3:0/574291132 conn(0x7fb3dc0a4210 msgr2=0x7fb3dc132d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:43.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.189+0000 7fb3e3221700 1 -- 172.21.15.3:0/574291132 shutdown_connections 2023-10-03T10:15:43.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:43.189+0000 7fb3e3221700 1 -- 172.21.15.3:0/574291132 wait complete. 2023-10-03T10:15:43.199 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:43.203 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:44.111 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:44.147 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:44.200 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:44.204 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:44.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.270+0000 7fee66d44700 1 Processor -- start 2023-10-03T10:15:44.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.270+0000 7fee66d44700 1 -- start start 2023-10-03T10:15:44.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.271+0000 7fee66d44700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee6009d6f0 0x7fee6009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:44.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.271+0000 7fee66d44700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fee6009e410 0x7fee60134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:44.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.271+0000 7fee66d44700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee6009b120 con 0x7fee6009e000 2023-10-03T10:15:44.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.271+0000 7fee66d44700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee6006b460 con 0x7fee6009e410 2023-10-03T10:15:44.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.271+0000 7fee66d44700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6009a7e0 con 0x7fee6009d6f0 2023-10-03T10:15:44.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.271+0000 7fee5ffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fee6009e000 0x7fee601339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57204/0 (socket says 172.21.15.3:57204) 2023-10-03T10:15:44.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.271+0000 7fee5ffff700 1 -- 172.21.15.3:0/530060069 learned_addr learned my addr 172.21.15.3:0/530060069 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:44.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.271+0000 7fee652e1700 1 --2- 172.21.15.3:0/530060069 >> v2:172.21.15.3:3300/0 conn(0x7fee6009e410 0x7fee60134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:44.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.271+0000 7fee652e1700 1 -- 172.21.15.3:0/530060069 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee6009d6f0 msgr2=0x7fee6009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:44.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.271+0000 7fee652e1700 1 --2- 172.21.15.3:0/530060069 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee6009d6f0 0x7fee6009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.271+0000 7fee652e1700 1 -- 172.21.15.3:0/530060069 >> v1:172.21.15.3:6789/0 conn(0x7fee6009e000 legacy=0x7fee601339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:44.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.271+0000 7fee652e1700 1 -- 172.21.15.3:0/530060069 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6009a500 con 0x7fee6009e410 2023-10-03T10:15:44.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.272+0000 7fee652e1700 1 --2- 172.21.15.3:0/530060069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee6009e410 0x7fee60134100 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fee58009d90 tx=0x7fee5800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3dc1b44dfa1dcb server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:44.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.272+0000 7fee5f7fe700 1 -- 172.21.15.3:0/530060069 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5800e070 con 0x7fee6009e410 2023-10-03T10:15:44.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.272+0000 7fee5f7fe700 1 -- 172.21.15.3:0/530060069 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5800bbd0 con 0x7fee6009e410 2023-10-03T10:15:44.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.272+0000 7fee5f7fe700 1 -- 172.21.15.3:0/530060069 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee58012430 con 0x7fee6009e410 2023-10-03T10:15:44.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.273+0000 7fee66d44700 1 -- 172.21.15.3:0/530060069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee6009e410 msgr2=0x7fee60134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:44.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.273+0000 7fee66d44700 1 --2- 172.21.15.3:0/530060069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee6009e410 0x7fee60134100 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fee58009d90 tx=0x7fee5800b330 comp rx=0 tx=0).stop 2023-10-03T10:15:44.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.273+0000 7fee66d44700 1 -- 172.21.15.3:0/530060069 shutdown_connections 2023-10-03T10:15:44.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.273+0000 7fee66d44700 1 --2- 172.21.15.3:0/530060069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee6009e410 0x7fee60134100 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.273+0000 7fee66d44700 1 --2- 172.21.15.3:0/530060069 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee6009d6f0 0x7fee6009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.273+0000 7fee66d44700 1 -- 172.21.15.3:0/530060069 >> 172.21.15.3:0/530060069 conn(0x7fee60066f50 msgr2=0x7fee6009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:44.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.273+0000 7fee66d44700 1 -- 172.21.15.3:0/530060069 shutdown_connections 2023-10-03T10:15:44.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.273+0000 7fee66d44700 1 -- 172.21.15.3:0/530060069 wait complete. 2023-10-03T10:15:44.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.274+0000 7fee66d44700 1 Processor -- start 2023-10-03T10:15:44.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.274+0000 7fee66d44700 1 -- start start 2023-10-03T10:15:44.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.275+0000 7fee66d44700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee6009d6f0 0x7fee601a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:44.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.275+0000 7fee66d44700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee6009e000 0x7fee601a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:44.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.275+0000 7fee66d44700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee6009e410 0x7fee601a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:44.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.275+0000 7fee66d44700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee601a6500 con 0x7fee6009e410 2023-10-03T10:15:44.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.275+0000 7fee66d44700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee601a8810 con 0x7fee6009e000 2023-10-03T10:15:44.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.275+0000 7fee66d44700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee600a30f0 con 0x7fee6009d6f0 2023-10-03T10:15:44.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.275+0000 7fee652e1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee6009e410 0x7fee601a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:44.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.275+0000 7fee652e1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee6009e410 0x7fee601a5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55422/0 (socket says 172.21.15.3:55422) 2023-10-03T10:15:44.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.275+0000 7fee652e1700 1 -- 172.21.15.3:0/3143618042 learned_addr learned my addr 172.21.15.3:0/3143618042 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:44.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.276+0000 7fee64ae0700 1 --2- 172.21.15.3:0/3143618042 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee6009d6f0 0x7fee601a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:44.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.276+0000 7fee5ffff700 1 --2- 172.21.15.3:0/3143618042 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee6009e000 0x7fee601a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:44.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.276+0000 7fee652e1700 1 -- 172.21.15.3:0/3143618042 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee6009d6f0 msgr2=0x7fee601a50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:44.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.276+0000 7fee652e1700 1 --2- 172.21.15.3:0/3143618042 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee6009d6f0 0x7fee601a50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.276+0000 7fee652e1700 1 -- 172.21.15.3:0/3143618042 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee6009e000 msgr2=0x7fee601a55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:44.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.276+0000 7fee652e1700 1 --2- 172.21.15.3:0/3143618042 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee6009e000 0x7fee601a55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.276+0000 7fee652e1700 1 -- 172.21.15.3:0/3143618042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee58009a70 con 0x7fee6009e410 2023-10-03T10:15:44.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.276+0000 7fee652e1700 1 --2- 172.21.15.3:0/3143618042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee6009e410 0x7fee601a5b20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fee58004270 tx=0x7fee58004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:44.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.277+0000 7fee5d7fa700 1 -- 172.21.15.3:0/3143618042 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee58004850 con 0x7fee6009e410 2023-10-03T10:15:44.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.277+0000 7fee5d7fa700 1 -- 172.21.15.3:0/3143618042 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee58020070 con 0x7fee6009e410 2023-10-03T10:15:44.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.277+0000 7fee5d7fa700 1 -- 172.21.15.3:0/3143618042 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee580276c0 con 0x7fee6009e410 2023-10-03T10:15:44.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.277+0000 7fee66d44700 1 -- 172.21.15.3:0/3143618042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee6009c710 con 0x7fee6009e410 2023-10-03T10:15:44.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.277+0000 7fee66d44700 1 -- 172.21.15.3:0/3143618042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee601a7e30 con 0x7fee6009e410 2023-10-03T10:15:44.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.278+0000 7fee5d7fa700 1 -- 172.21.15.3:0/3143618042 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fee58012430 con 0x7fee6009e410 2023-10-03T10:15:44.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.278+0000 7fee5d7fa700 1 -- 172.21.15.3:0/3143618042 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fee58022070 con 0x7fee6009e410 2023-10-03T10:15:44.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.278+0000 7fee66d44700 1 -- 172.21.15.3:0/3143618042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee60074ea0 con 0x7fee6009e410 2023-10-03T10:15:44.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.284+0000 7fee5d7fa700 1 -- 172.21.15.3:0/3143618042 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fee580128c0 con 0x7fee6009e410 2023-10-03T10:15:44.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.310+0000 7f1d254a5700 1 Processor -- start 2023-10-03T10:15:44.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.310+0000 7f1d254a5700 1 -- start start 2023-10-03T10:15:44.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.311+0000 7f1d254a5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1d20133a80 0x7f1d20133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:44.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.311+0000 7f1d254a5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1d20130a70 0x7f1d2013b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:44.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.311+0000 7f1d254a5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d2006bc00 con 0x7f1d2012ff20 2023-10-03T10:15:44.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.311+0000 7f1d254a5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d2006c570 con 0x7f1d20133a80 2023-10-03T10:15:44.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.311+0000 7f1d254a5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d200a0040 con 0x7f1d20130a70 2023-10-03T10:15:44.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.311+0000 7f1d1e7fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1d2012ff20 0x7f1d20130360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57206/0 (socket says 172.21.15.3:57206) 2023-10-03T10:15:44.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.311+0000 7f1d1e7fc700 1 -- 172.21.15.3:0/792137504 learned_addr learned my addr 172.21.15.3:0/792137504 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:44.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.312+0000 7f1d1f7fe700 1 --2- 172.21.15.3:0/792137504 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1d20130a70 0x7f1d2013b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:44.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.312+0000 7f1d1dffb700 1 -- 172.21.15.3:0/792137504 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1337322167 0 0) 0x7f1d2006bc00 con 0x7f1d2012ff20 2023-10-03T10:15:44.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.312+0000 7f1d1dffb700 1 -- 172.21.15.3:0/792137504 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d080035b0 con 0x7f1d2012ff20 2023-10-03T10:15:44.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.312+0000 7f1d1f7fe700 1 -- 172.21.15.3:0/792137504 >> v2:172.21.15.3:3300/0 conn(0x7f1d20133a80 msgr2=0x7f1d20133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:44.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.312+0000 7f1d1f7fe700 1 --2- 172.21.15.3:0/792137504 >> v2:172.21.15.3:3300/0 conn(0x7f1d20133a80 0x7f1d20133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.312+0000 7f1d1f7fe700 1 -- 172.21.15.3:0/792137504 >> v1:172.21.15.3:6789/0 conn(0x7f1d2012ff20 legacy=0x7f1d20130360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:44.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.312+0000 7f1d1f7fe700 1 -- 172.21.15.3:0/792137504 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d20133860 con 0x7f1d20130a70 2023-10-03T10:15:44.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.312+0000 7f1d1f7fe700 1 --2- 172.21.15.3:0/792137504 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1d20130a70 0x7f1d2013b850 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1d18009d90 tx=0x7f1d1800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a3122aeb6f3f01b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:44.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.312+0000 7f1d1dffb700 1 -- 172.21.15.3:0/792137504 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1800e070 con 0x7f1d20130a70 2023-10-03T10:15:44.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.312+0000 7f1d1dffb700 1 -- 172.21.15.3:0/792137504 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1800bb20 con 0x7f1d20130a70 2023-10-03T10:15:44.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.312+0000 7f1d1dffb700 1 -- 172.21.15.3:0/792137504 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d18012430 con 0x7f1d20130a70 2023-10-03T10:15:44.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.313+0000 7f1d254a5700 1 -- 172.21.15.3:0/792137504 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1d20130a70 msgr2=0x7f1d2013b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:44.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.313+0000 7f1d254a5700 1 --2- 172.21.15.3:0/792137504 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1d20130a70 0x7f1d2013b850 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1d18009d90 tx=0x7f1d1800b330 comp rx=0 tx=0).stop 2023-10-03T10:15:44.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.313+0000 7f1d254a5700 1 -- 172.21.15.3:0/792137504 shutdown_connections 2023-10-03T10:15:44.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.313+0000 7f1d254a5700 1 --2- 172.21.15.3:0/792137504 >> v2:172.21.15.3:3300/0 conn(0x7f1d20133a80 0x7f1d20133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.313+0000 7f1d254a5700 1 --2- 172.21.15.3:0/792137504 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1d20130a70 0x7f1d2013b850 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.313+0000 7f1d254a5700 1 -- 172.21.15.3:0/792137504 >> 172.21.15.3:0/792137504 conn(0x7f1d200a4210 msgr2=0x7f1d200a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:44.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.313+0000 7f1d254a5700 1 -- 172.21.15.3:0/792137504 shutdown_connections 2023-10-03T10:15:44.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.313+0000 7f1d254a5700 1 -- 172.21.15.3:0/792137504 wait complete. 2023-10-03T10:15:44.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.315+0000 7f1d254a5700 1 Processor -- start 2023-10-03T10:15:44.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.315+0000 7f1d254a5700 1 -- start start 2023-10-03T10:15:44.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.315+0000 7f1d254a5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1d2012ff20 0x7f1d2009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:44.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.315+0000 7f1d254a5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1d20130a70 0x7f1d2009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:44.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.315+0000 7f1d254a5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d20133a80 0x7f1d2009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:44.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.315+0000 7f1d254a5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d20120f50 con 0x7f1d20133a80 2023-10-03T10:15:44.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.315+0000 7f1d254a5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2011fd30 con 0x7f1d20130a70 2023-10-03T10:15:44.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.315+0000 7f1d254a5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d200a2030 con 0x7f1d2012ff20 2023-10-03T10:15:44.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.316+0000 7f1d1e7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1d20130a70 0x7f1d2009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:44.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.316+0000 7f1d1e7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1d20130a70 0x7f1d2009c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34580/0 (socket says 172.21.15.3:34580) 2023-10-03T10:15:44.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.316+0000 7f1d1e7fc700 1 -- 172.21.15.3:0/2527768867 learned_addr learned my addr 172.21.15.3:0/2527768867 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:44.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.316+0000 7f1d1effd700 1 --2- 172.21.15.3:0/2527768867 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1d2012ff20 0x7f1d2009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:44.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.316+0000 7f1d1e7fc700 1 -- 172.21.15.3:0/2527768867 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1d2012ff20 msgr2=0x7f1d2009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:44.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.316+0000 7f1d1e7fc700 1 --2- 172.21.15.3:0/2527768867 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1d2012ff20 0x7f1d2009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.316+0000 7f1d1e7fc700 1 -- 172.21.15.3:0/2527768867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d20133a80 msgr2=0x7f1d2009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:44.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.316+0000 7f1d1f7fe700 1 --2- 172.21.15.3:0/2527768867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d20133a80 0x7f1d2009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:44.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.316+0000 7f1d1e7fc700 1 --2- 172.21.15.3:0/2527768867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d20133a80 0x7f1d2009cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.316+0000 7f1d1e7fc700 1 -- 172.21.15.3:0/2527768867 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d18009a70 con 0x7f1d20130a70 2023-10-03T10:15:44.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.316+0000 7f1d1effd700 1 --2- 172.21.15.3:0/2527768867 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1d2012ff20 0x7f1d2009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:44.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.316+0000 7f1d1f7fe700 1 --2- 172.21.15.3:0/2527768867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d20133a80 0x7f1d2009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:15:44.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.316+0000 7f1d1e7fc700 1 --2- 172.21.15.3:0/2527768867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1d20130a70 0x7f1d2009c9c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400c700 tx=0x7f1d1400ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:44.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.316+0000 7f1d077fe700 1 -- 172.21.15.3:0/2527768867 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d14005d70 con 0x7f1d20130a70 2023-10-03T10:15:44.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.317+0000 7f1d077fe700 1 -- 172.21.15.3:0/2527768867 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d140070d0 con 0x7f1d20130a70 2023-10-03T10:15:44.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.317+0000 7f1d254a5700 1 -- 172.21.15.3:0/2527768867 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d2012db30 con 0x7f1d20130a70 2023-10-03T10:15:44.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.317+0000 7f1d077fe700 1 -- 172.21.15.3:0/2527768867 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d14007460 con 0x7f1d20130a70 2023-10-03T10:15:44.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.317+0000 7f1d254a5700 1 -- 172.21.15.3:0/2527768867 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d20120a60 con 0x7f1d20130a70 2023-10-03T10:15:44.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.318+0000 7f1d077fe700 1 -- 172.21.15.3:0/2527768867 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1d140075c0 con 0x7f1d20130a70 2023-10-03T10:15:44.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.318+0000 7f1d077fe700 1 -- 172.21.15.3:0/2527768867 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1d1403ebf0 con 0x7f1d20130a70 2023-10-03T10:15:44.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.318+0000 7f1d254a5700 1 -- 172.21.15.3:0/2527768867 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d20134310 con 0x7f1d20130a70 2023-10-03T10:15:44.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.323+0000 7f1d077fe700 1 -- 172.21.15.3:0/2527768867 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1d14017020 con 0x7f1d20130a70 2023-10-03T10:15:44.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.373+0000 7ffb9f6e7700 1 Processor -- start 2023-10-03T10:15:44.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.374+0000 7ffb9f6e7700 1 -- start start 2023-10-03T10:15:44.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.374+0000 7ffb9f6e7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffb98131810 0x7ffb98131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:44.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.374+0000 7ffb9f6e7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffb9809a580 0x7ffb98139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:44.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.374+0000 7ffb9f6e7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb981313c0 con 0x7ffb98099a40 2023-10-03T10:15:44.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.374+0000 7ffb9f6e7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb9806e8c0 con 0x7ffb98131810 2023-10-03T10:15:44.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.374+0000 7ffb9f6e7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb98071d20 con 0x7ffb9809a580 2023-10-03T10:15:44.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.375+0000 7ffb9d483700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffb98131810 0x7ffb98131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:44.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.375+0000 7ffb9d483700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffb98131810 0x7ffb98131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55460/0 (socket says 172.21.15.3:55460) 2023-10-03T10:15:44.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.375+0000 7ffb9d483700 1 -- 172.21.15.3:0/2149173955 learned_addr learned my addr 172.21.15.3:0/2149173955 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:44.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.375+0000 7ffb9dc84700 1 --2- 172.21.15.3:0/2149173955 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffb9809a580 0x7ffb98139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:44.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.375+0000 7ffb9d483700 1 -- 172.21.15.3:0/2149173955 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffb9809a580 msgr2=0x7ffb98139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:44.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.375+0000 7ffb9d483700 1 --2- 172.21.15.3:0/2149173955 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffb9809a580 0x7ffb98139660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.375+0000 7ffb9d483700 1 -- 172.21.15.3:0/2149173955 >> v1:172.21.15.3:6789/0 conn(0x7ffb98099a40 legacy=0x7ffb98099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:44.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.375+0000 7ffb9d483700 1 -- 172.21.15.3:0/2149173955 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb980721a0 con 0x7ffb98131810 2023-10-03T10:15:44.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.375+0000 7ffb9d483700 1 --2- 172.21.15.3:0/2149173955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb98131810 0x7ffb98131be0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ffb940098f0 tx=0x7ffb94009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87a65cfd57e5d14 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:44.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.375+0000 7ffb8ffff700 1 -- 172.21.15.3:0/2149173955 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb94011830 con 0x7ffb98131810 2023-10-03T10:15:44.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.375+0000 7ffb8ffff700 1 -- 172.21.15.3:0/2149173955 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb94011990 con 0x7ffb98131810 2023-10-03T10:15:44.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.375+0000 7ffb8ffff700 1 -- 172.21.15.3:0/2149173955 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb94011d20 con 0x7ffb98131810 2023-10-03T10:15:44.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.376+0000 7ffb9f6e7700 1 -- 172.21.15.3:0/2149173955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb98131810 msgr2=0x7ffb98131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:44.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.376+0000 7ffb9f6e7700 1 --2- 172.21.15.3:0/2149173955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb98131810 0x7ffb98131be0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ffb940098f0 tx=0x7ffb94009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:44.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.376+0000 7ffb9f6e7700 1 -- 172.21.15.3:0/2149173955 shutdown_connections 2023-10-03T10:15:44.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.376+0000 7ffb9f6e7700 1 --2- 172.21.15.3:0/2149173955 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffb9809a580 0x7ffb98139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.376+0000 7ffb9f6e7700 1 --2- 172.21.15.3:0/2149173955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb98131810 0x7ffb98131be0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.376+0000 7ffb9f6e7700 1 -- 172.21.15.3:0/2149173955 >> 172.21.15.3:0/2149173955 conn(0x7ffb980a2aa0 msgr2=0x7ffb980a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:44.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.376+0000 7ffb9f6e7700 1 -- 172.21.15.3:0/2149173955 shutdown_connections 2023-10-03T10:15:44.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.376+0000 7ffb9f6e7700 1 -- 172.21.15.3:0/2149173955 wait complete. 2023-10-03T10:15:44.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.377+0000 7ffb9f6e7700 1 Processor -- start 2023-10-03T10:15:44.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.378+0000 7ffb9f6e7700 1 -- start start 2023-10-03T10:15:44.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.378+0000 7ffb9f6e7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb98099a40 0x7ffb9811b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:44.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.378+0000 7ffb9f6e7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffb9809a580 0x7ffb9811bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:44.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.378+0000 7ffb9f6e7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffb98131810 0x7ffb9811c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:44.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.378+0000 7ffb9f6e7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9811d320 con 0x7ffb98099a40 2023-10-03T10:15:44.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.378+0000 7ffb9f6e7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9811e570 con 0x7ffb9809a580 2023-10-03T10:15:44.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.378+0000 7ffb9f6e7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9811d910 con 0x7ffb98131810 2023-10-03T10:15:44.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.378+0000 7ffb9d483700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb98099a40 0x7ffb9811b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:44.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.378+0000 7ffb9d483700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb98099a40 0x7ffb9811b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55474/0 (socket says 172.21.15.3:55474) 2023-10-03T10:15:44.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.378+0000 7ffb9cc82700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffb9809a580 0x7ffb9811bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:44.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.378+0000 7ffb9d483700 1 -- 172.21.15.3:0/354110657 learned_addr learned my addr 172.21.15.3:0/354110657 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:44.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.379+0000 7ffb9cc82700 1 -- 172.21.15.3:0/354110657 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffb98131810 msgr2=0x7ffb9811c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:44.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.379+0000 7ffb9cc82700 1 --2- 172.21.15.3:0/354110657 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffb98131810 0x7ffb9811c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.379+0000 7ffb9cc82700 1 -- 172.21.15.3:0/354110657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb98099a40 msgr2=0x7ffb9811b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:44.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.379+0000 7ffb9cc82700 1 --2- 172.21.15.3:0/354110657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb98099a40 0x7ffb9811b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.379+0000 7ffb9cc82700 1 -- 172.21.15.3:0/354110657 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb94009550 con 0x7ffb9809a580 2023-10-03T10:15:44.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.379+0000 7ffb9d483700 1 --2- 172.21.15.3:0/354110657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb98099a40 0x7ffb9811b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:44.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.379+0000 7ffb9cc82700 1 --2- 172.21.15.3:0/354110657 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffb9809a580 0x7ffb9811bce0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ffb88000f60 tx=0x7ffb8800c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:44.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.379+0000 7ffb8dffb700 1 -- 172.21.15.3:0/354110657 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb880052d0 con 0x7ffb9809a580 2023-10-03T10:15:44.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.379+0000 7ffb8dffb700 1 -- 172.21.15.3:0/354110657 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb88005430 con 0x7ffb9809a580 2023-10-03T10:15:44.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.379+0000 7ffb9f6e7700 1 -- 172.21.15.3:0/354110657 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb9811e9c0 con 0x7ffb9809a580 2023-10-03T10:15:44.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.379+0000 7ffb8dffb700 1 -- 172.21.15.3:0/354110657 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb880038c0 con 0x7ffb9809a580 2023-10-03T10:15:44.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.379+0000 7ffb9f6e7700 1 -- 172.21.15.3:0/354110657 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb9811ef50 con 0x7ffb9809a580 2023-10-03T10:15:44.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.380+0000 7ffb8dffb700 1 -- 172.21.15.3:0/354110657 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffb88003a20 con 0x7ffb9809a580 2023-10-03T10:15:44.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.380+0000 7ffb8dffb700 1 -- 172.21.15.3:0/354110657 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ffb8803e9e0 con 0x7ffb9809a580 2023-10-03T10:15:44.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.381+0000 7ffb9f6e7700 1 -- 172.21.15.3:0/354110657 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb98074ea0 con 0x7ffb9809a580 2023-10-03T10:15:44.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.386+0000 7ffb8dffb700 1 -- 172.21.15.3:0/354110657 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffb88011590 con 0x7ffb9809a580 2023-10-03T10:15:44.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.403+0000 7fa34df34700 1 Processor -- start 2023-10-03T10:15:44.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.403+0000 7fa34df34700 1 -- start start 2023-10-03T10:15:44.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.403+0000 7fa34df34700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3480a0a30 0x7fa3480a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:44.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.403+0000 7fa34df34700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa348138d80 0x7fa34813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:44.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.403+0000 7fa34df34700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3480a2a70 con 0x7fa34809d630 2023-10-03T10:15:44.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.404+0000 7fa34df34700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3480a01d0 con 0x7fa348138d80 2023-10-03T10:15:44.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.404+0000 7fa34df34700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa34809e2b0 con 0x7fa3480a0a30 2023-10-03T10:15:44.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.404+0000 7fa3477fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3480a0a30 0x7fa3480a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:44.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.404+0000 7fa3477fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3480a0a30 0x7fa3480a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34616/0 (socket says 172.21.15.3:34616) 2023-10-03T10:15:44.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.404+0000 7fa3477fe700 1 -- 172.21.15.3:0/3435505982 learned_addr learned my addr 172.21.15.3:0/3435505982 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:44.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.404+0000 7fa347fff700 1 --2- 172.21.15.3:0/3435505982 >> v2:172.21.15.3:3300/0 conn(0x7fa348138d80 0x7fa34813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:44.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.404+0000 7fa3477fe700 1 -- 172.21.15.3:0/3435505982 >> v2:172.21.15.3:3300/0 conn(0x7fa348138d80 msgr2=0x7fa34813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:44.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.404+0000 7fa3477fe700 1 --2- 172.21.15.3:0/3435505982 >> v2:172.21.15.3:3300/0 conn(0x7fa348138d80 0x7fa34813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.404+0000 7fa3477fe700 1 -- 172.21.15.3:0/3435505982 >> v1:172.21.15.3:6789/0 conn(0x7fa34809d630 legacy=0x7fa3480a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:44.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.404+0000 7fa3477fe700 1 -- 172.21.15.3:0/3435505982 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa34809dc50 con 0x7fa3480a0a30 2023-10-03T10:15:44.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.404+0000 7fa347fff700 1 --2- 172.21.15.3:0/3435505982 >> v2:172.21.15.3:3300/0 conn(0x7fa348138d80 0x7fa34813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:15:44.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.405+0000 7fa3477fe700 1 --2- 172.21.15.3:0/3435505982 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3480a0a30 0x7fa3480a0e00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa3380098f0 tx=0x7fa338009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f98c3488e1792a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:44.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.405+0000 7fa3467fc700 1 -- 172.21.15.3:0/3435505982 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33800b8a0 con 0x7fa3480a0a30 2023-10-03T10:15:44.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.405+0000 7fa3467fc700 1 -- 172.21.15.3:0/3435505982 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33800ba00 con 0x7fa3480a0a30 2023-10-03T10:15:44.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.405+0000 7fa3467fc700 1 -- 172.21.15.3:0/3435505982 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33800bd90 con 0x7fa3480a0a30 2023-10-03T10:15:44.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.406+0000 7fa34df34700 1 -- 172.21.15.3:0/3435505982 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3480a0a30 msgr2=0x7fa3480a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:44.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.406+0000 7fa34df34700 1 --2- 172.21.15.3:0/3435505982 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3480a0a30 0x7fa3480a0e00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa3380098f0 tx=0x7fa338009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:44.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.406+0000 7fa34df34700 1 -- 172.21.15.3:0/3435505982 shutdown_connections 2023-10-03T10:15:44.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.406+0000 7fa34df34700 1 --2- 172.21.15.3:0/3435505982 >> v2:172.21.15.3:3300/0 conn(0x7fa348138d80 0x7fa34813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.406+0000 7fa34df34700 1 --2- 172.21.15.3:0/3435505982 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3480a0a30 0x7fa3480a0e00 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.406+0000 7fa34df34700 1 -- 172.21.15.3:0/3435505982 >> 172.21.15.3:0/3435505982 conn(0x7fa3480a3050 msgr2=0x7fa3480a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:44.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.406+0000 7fa34df34700 1 -- 172.21.15.3:0/3435505982 shutdown_connections 2023-10-03T10:15:44.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.406+0000 7fa34df34700 1 -- 172.21.15.3:0/3435505982 wait complete. 2023-10-03T10:15:44.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.407+0000 7fa34df34700 1 Processor -- start 2023-10-03T10:15:44.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.408+0000 7fa34df34700 1 -- start start 2023-10-03T10:15:44.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.408+0000 7fa34df34700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa34809d630 0x7fa348120870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:44.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.408+0000 7fa34df34700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3480a0a30 0x7fa348120db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:44.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.408+0000 7fa34df34700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa348138d80 0x7fa3481212f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:44.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.408+0000 7fa34df34700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa348123670 con 0x7fa348138d80 2023-10-03T10:15:44.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.408+0000 7fa34df34700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa348122c10 con 0x7fa3480a0a30 2023-10-03T10:15:44.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.408+0000 7fa34df34700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa3481242c0 con 0x7fa34809d630 2023-10-03T10:15:44.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.408+0000 7fa3477fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa34809d630 0x7fa348120870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:44.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.408+0000 7fa3477fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa34809d630 0x7fa348120870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:46844/0 (socket says 172.21.15.3:46844) 2023-10-03T10:15:44.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.408+0000 7fa3477fe700 1 -- 172.21.15.3:0/2930907500 learned_addr learned my addr 172.21.15.3:0/2930907500 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:44.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.409+0000 7fa346ffd700 1 --2- 172.21.15.3:0/2930907500 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3480a0a30 0x7fa348120db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:44.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.409+0000 7fa347fff700 1 --2- 172.21.15.3:0/2930907500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa348138d80 0x7fa3481212f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:44.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.409+0000 7fa3477fe700 1 -- 172.21.15.3:0/2930907500 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3480a0a30 msgr2=0x7fa348120db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:44.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.409+0000 7fa3477fe700 1 --2- 172.21.15.3:0/2930907500 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3480a0a30 0x7fa348120db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.409+0000 7fa3477fe700 1 -- 172.21.15.3:0/2930907500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa348138d80 msgr2=0x7fa3481212f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:44.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.409+0000 7fa3477fe700 1 --2- 172.21.15.3:0/2930907500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa348138d80 0x7fa3481212f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.409+0000 7fa3477fe700 1 -- 172.21.15.3:0/2930907500 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa338009550 con 0x7fa34809d630 2023-10-03T10:15:44.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.409+0000 7fa3477fe700 1 --2- 172.21.15.3:0/2930907500 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa34809d630 0x7fa348120870 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa338009c40 tx=0x7fa338004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:44.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.409+0000 7fa32ffff700 1 -- 172.21.15.3:0/2930907500 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa338004480 con 0x7fa34809d630 2023-10-03T10:15:44.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.409+0000 7fa32ffff700 1 -- 172.21.15.3:0/2930907500 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33801f070 con 0x7fa34809d630 2023-10-03T10:15:44.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.409+0000 7fa34df34700 1 -- 172.21.15.3:0/2930907500 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa34813b070 con 0x7fa34809d630 2023-10-03T10:15:44.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.409+0000 7fa32ffff700 1 -- 172.21.15.3:0/2930907500 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3380038c0 con 0x7fa34809d630 2023-10-03T10:15:44.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.409+0000 7fa34df34700 1 -- 172.21.15.3:0/2930907500 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa348123fe0 con 0x7fa34809d630 2023-10-03T10:15:44.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.410+0000 7fa32ffff700 1 -- 172.21.15.3:0/2930907500 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa338021030 con 0x7fa34809d630 2023-10-03T10:15:44.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.411+0000 7fa32ffff700 1 -- 172.21.15.3:0/2930907500 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa3380409f0 con 0x7fa34809d630 2023-10-03T10:15:44.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.412+0000 7fa34df34700 1 -- 172.21.15.3:0/2930907500 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3480747f0 con 0x7fa34809d630 2023-10-03T10:15:44.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.417+0000 7fa32ffff700 1 -- 172.21.15.3:0/2930907500 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa338003cb0 con 0x7fa34809d630 2023-10-03T10:15:44.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.536+0000 7f1d254a5700 1 -- 172.21.15.3:0/2527768867 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1d20074ea0 con 0x7f1d20130a70 2023-10-03T10:15:44.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.537+0000 7f1d077fe700 1 -- 172.21.15.3:0/2527768867 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1d1400e280 con 0x7f1d20130a70 2023-10-03T10:15:44.537 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:44.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.540+0000 7f1d254a5700 1 -- 172.21.15.3:0/2527768867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1d20130a70 msgr2=0x7f1d2009c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:44.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.540+0000 7f1d254a5700 1 --2- 172.21.15.3:0/2527768867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1d20130a70 0x7f1d2009c9c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400c700 tx=0x7f1d1400ca10 comp rx=0 tx=0).stop 2023-10-03T10:15:44.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.540+0000 7f1d254a5700 1 -- 172.21.15.3:0/2527768867 shutdown_connections 2023-10-03T10:15:44.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.540+0000 7f1d254a5700 1 --2- 172.21.15.3:0/2527768867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d20133a80 0x7f1d2009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.540+0000 7f1d254a5700 1 --2- 172.21.15.3:0/2527768867 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1d2012ff20 0x7f1d2009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.540+0000 7f1d254a5700 1 --2- 172.21.15.3:0/2527768867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1d20130a70 0x7f1d2009c9c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.540+0000 7f1d254a5700 1 -- 172.21.15.3:0/2527768867 >> 172.21.15.3:0/2527768867 conn(0x7f1d200a4210 msgr2=0x7f1d2009e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:44.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.541+0000 7f1d254a5700 1 -- 172.21.15.3:0/2527768867 shutdown_connections 2023-10-03T10:15:44.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.541+0000 7f1d254a5700 1 -- 172.21.15.3:0/2527768867 wait complete. 2023-10-03T10:15:44.555 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:44.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.581+0000 7fee66d44700 1 -- 172.21.15.3:0/3143618042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fee6006c360 con 0x7fee6009e410 2023-10-03T10:15:44.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.582+0000 7fee5d7fa700 1 -- 172.21.15.3:0/3143618042 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fee58035090 con 0x7fee6009e410 2023-10-03T10:15:44.582 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:44.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.584+0000 7fee66d44700 1 -- 172.21.15.3:0/3143618042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee6009e410 msgr2=0x7fee601a5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:44.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.584+0000 7fee66d44700 1 --2- 172.21.15.3:0/3143618042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee6009e410 0x7fee601a5b20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fee58004270 tx=0x7fee58004ad0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.584+0000 7fee66d44700 1 -- 172.21.15.3:0/3143618042 shutdown_connections 2023-10-03T10:15:44.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.584+0000 7fee66d44700 1 --2- 172.21.15.3:0/3143618042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee6009e410 0x7fee601a5b20 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.584+0000 7fee66d44700 1 --2- 172.21.15.3:0/3143618042 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee6009d6f0 0x7fee601a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.584+0000 7fee66d44700 1 --2- 172.21.15.3:0/3143618042 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee6009e000 0x7fee601a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.584+0000 7fee66d44700 1 -- 172.21.15.3:0/3143618042 >> 172.21.15.3:0/3143618042 conn(0x7fee60066f50 msgr2=0x7fee600a3640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:44.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.584+0000 7fee66d44700 1 -- 172.21.15.3:0/3143618042 shutdown_connections 2023-10-03T10:15:44.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.584+0000 7fee66d44700 1 -- 172.21.15.3:0/3143618042 wait complete. 2023-10-03T10:15:44.600 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:44.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.600+0000 7ffb9f6e7700 1 -- 172.21.15.3:0/354110657 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ffb9806b9e0 con 0x7ffb9809a580 2023-10-03T10:15:44.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.601+0000 7ffb8dffb700 1 -- 172.21.15.3:0/354110657 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffb88017030 con 0x7ffb9809a580 2023-10-03T10:15:44.601 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:44.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.603+0000 7ffb9f6e7700 1 -- 172.21.15.3:0/354110657 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffb9809a580 msgr2=0x7ffb9811bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:44.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.603+0000 7ffb9f6e7700 1 --2- 172.21.15.3:0/354110657 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffb9809a580 0x7ffb9811bce0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ffb88000f60 tx=0x7ffb8800c730 comp rx=0 tx=0).stop 2023-10-03T10:15:44.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.603+0000 7ffb9f6e7700 1 -- 172.21.15.3:0/354110657 shutdown_connections 2023-10-03T10:15:44.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.603+0000 7ffb9f6e7700 1 --2- 172.21.15.3:0/354110657 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffb98131810 0x7ffb9811c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.603+0000 7ffb9f6e7700 1 --2- 172.21.15.3:0/354110657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb98099a40 0x7ffb9811b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.603+0000 7ffb9f6e7700 1 --2- 172.21.15.3:0/354110657 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffb9809a580 0x7ffb9811bce0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.603+0000 7ffb9f6e7700 1 -- 172.21.15.3:0/354110657 >> 172.21.15.3:0/354110657 conn(0x7ffb980a2aa0 msgr2=0x7ffb981b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:44.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.603+0000 7ffb9f6e7700 1 -- 172.21.15.3:0/354110657 shutdown_connections 2023-10-03T10:15:44.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.603+0000 7ffb9f6e7700 1 -- 172.21.15.3:0/354110657 wait complete. 2023-10-03T10:15:44.617 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:44.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.672+0000 7fa34df34700 1 -- 172.21.15.3:0/2930907500 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa348123db0 con 0x7fa34809d630 2023-10-03T10:15:44.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.673+0000 7fa32ffff700 1 -- 172.21.15.3:0/2930907500 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa338023750 con 0x7fa34809d630 2023-10-03T10:15:44.673 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:44.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.675+0000 7fa34df34700 1 -- 172.21.15.3:0/2930907500 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa34809d630 msgr2=0x7fa348120870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:44.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.675+0000 7fa34df34700 1 --2- 172.21.15.3:0/2930907500 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa34809d630 0x7fa348120870 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa338009c40 tx=0x7fa338004750 comp rx=0 tx=0).stop 2023-10-03T10:15:44.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.675+0000 7fa34df34700 1 -- 172.21.15.3:0/2930907500 shutdown_connections 2023-10-03T10:15:44.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.675+0000 7fa34df34700 1 --2- 172.21.15.3:0/2930907500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa348138d80 0x7fa3481212f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.675+0000 7fa34df34700 1 --2- 172.21.15.3:0/2930907500 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa34809d630 0x7fa348120870 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.675+0000 7fa34df34700 1 --2- 172.21.15.3:0/2930907500 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3480a0a30 0x7fa348120db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:44.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.675+0000 7fa34df34700 1 -- 172.21.15.3:0/2930907500 >> 172.21.15.3:0/2930907500 conn(0x7fa3480a3050 msgr2=0x7fa34809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:44.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.675+0000 7fa34df34700 1 -- 172.21.15.3:0/2930907500 shutdown_connections 2023-10-03T10:15:44.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:44.675+0000 7fa34df34700 1 -- 172.21.15.3:0/2930907500 wait complete. 2023-10-03T10:15:44.689 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:45.556 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:45.603 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:45.617 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:45.690 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:45.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.754+0000 7f0cbaf55700 1 Processor -- start 2023-10-03T10:15:45.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.754+0000 7fe3a6ddd700 1 Processor -- start 2023-10-03T10:15:45.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.754+0000 7f0cbaf55700 1 -- start start 2023-10-03T10:15:45.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.755+0000 7f0cbaf55700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0cb41316c0 0x7f0cb4131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:45.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.755+0000 7f0cbaf55700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cb409d9f0 0x7f0cb406a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:45.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.755+0000 7f0cbaf55700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cb409e130 con 0x7f0cb406acb0 2023-10-03T10:15:45.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.755+0000 7f0cbaf55700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cb41312b0 con 0x7f0cb41316c0 2023-10-03T10:15:45.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.755+0000 7f0cbaf55700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb409f6e0 con 0x7f0cb409d9f0 2023-10-03T10:15:45.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.754+0000 7fe3a6ddd700 1 -- start start 2023-10-03T10:15:45.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.755+0000 7fe3a6ddd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a00a0a30 0x7fe3a00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:45.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.755+0000 7fe3a6ddd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3a0138d80 0x7fe3a013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:45.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.755+0000 7fe3a6ddd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a00a2a70 con 0x7fe3a009d630 2023-10-03T10:15:45.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.755+0000 7fe3a6ddd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a00a01d0 con 0x7fe3a00a0a30 2023-10-03T10:15:45.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.755+0000 7fe3a6ddd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a009e2b0 con 0x7fe3a0138d80 2023-10-03T10:15:45.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.755+0000 7fe3a4b79700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a00a0a30 0x7fe3a00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:45.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.755+0000 7fe3a4b79700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a00a0a30 0x7fe3a00a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34630/0 (socket says 172.21.15.3:34630) 2023-10-03T10:15:45.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.755+0000 7fe3a4b79700 1 -- 172.21.15.3:0/1315667283 learned_addr learned my addr 172.21.15.3:0/1315667283 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:45.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.755+0000 7fe3a537a700 1 --2- 172.21.15.3:0/1315667283 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3a0138d80 0x7fe3a013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:45.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.755+0000 7f0cb94f2700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0cb406acb0 0x7f0cb4138540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57232/0 (socket says 172.21.15.3:57232) 2023-10-03T10:15:45.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.755+0000 7f0cb94f2700 1 -- 172.21.15.3:0/3298448288 learned_addr learned my addr 172.21.15.3:0/3298448288 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:45.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.755+0000 7fe3a4b79700 1 -- 172.21.15.3:0/1315667283 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3a0138d80 msgr2=0x7fe3a013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:45.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.755+0000 7fe3a4b79700 1 --2- 172.21.15.3:0/1315667283 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3a0138d80 0x7fe3a013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:45.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.755+0000 7fe3a4b79700 1 -- 172.21.15.3:0/1315667283 >> v1:172.21.15.3:6789/0 conn(0x7fe3a009d630 legacy=0x7fe3a00a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:45.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.755+0000 7fe3a4b79700 1 -- 172.21.15.3:0/1315667283 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a009dc50 con 0x7fe3a00a0a30 2023-10-03T10:15:45.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.756+0000 7f0cb8cf1700 1 --2- 172.21.15.3:0/3298448288 >> v2:172.21.15.3:3300/0 conn(0x7f0cb41316c0 0x7f0cb4131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:45.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.756+0000 7fe3a4b79700 1 --2- 172.21.15.3:0/1315667283 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a00a0a30 0x7fe3a00a0e00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe390009920 tx=0x7fe390009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25890ad3fa284715 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:45.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.756+0000 7f0cb3fff700 1 --2- 172.21.15.3:0/3298448288 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cb409d9f0 0x7f0cb406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:45.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.756+0000 7fe39f7fe700 1 -- 172.21.15.3:0/1315667283 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe390011830 con 0x7fe3a00a0a30 2023-10-03T10:15:45.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.756+0000 7fe39f7fe700 1 -- 172.21.15.3:0/1315667283 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe390011990 con 0x7fe3a00a0a30 2023-10-03T10:15:45.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.756+0000 7fe39f7fe700 1 -- 172.21.15.3:0/1315667283 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe390011d20 con 0x7fe3a00a0a30 2023-10-03T10:15:45.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.757+0000 7fe3a6ddd700 1 -- 172.21.15.3:0/1315667283 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a00a0a30 msgr2=0x7fe3a00a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:45.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.757+0000 7fe3a6ddd700 1 --2- 172.21.15.3:0/1315667283 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a00a0a30 0x7fe3a00a0e00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe390009920 tx=0x7fe390009c30 comp rx=0 tx=0).stop 2023-10-03T10:15:45.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.756+0000 7f0cb3fff700 1 -- 172.21.15.3:0/3298448288 >> v2:172.21.15.3:3300/0 conn(0x7f0cb41316c0 msgr2=0x7f0cb4131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:45.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.756+0000 7f0cb3fff700 1 --2- 172.21.15.3:0/3298448288 >> v2:172.21.15.3:3300/0 conn(0x7f0cb41316c0 0x7f0cb4131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:45.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.756+0000 7f0cb3fff700 1 -- 172.21.15.3:0/3298448288 >> v1:172.21.15.3:6789/0 conn(0x7f0cb406acb0 legacy=0x7f0cb4138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:45.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.756+0000 7f0cb37fe700 1 -- 172.21.15.3:0/3298448288 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4205019302 0 0) 0x7f0cac001490 con 0x7f0cb406acb0 2023-10-03T10:15:45.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.756+0000 7f0cb3fff700 1 -- 172.21.15.3:0/3298448288 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb409e760 con 0x7f0cb409d9f0 2023-10-03T10:15:45.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.756+0000 7f0cb8cf1700 1 --2- 172.21.15.3:0/3298448288 >> v2:172.21.15.3:3300/0 conn(0x7f0cb41316c0 0x7f0cb4131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:45.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.756+0000 7f0cb3fff700 1 --2- 172.21.15.3:0/3298448288 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cb409d9f0 0x7f0cb406a770 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009840 tx=0x7f0ca0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=739326efb577d02c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:45.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.757+0000 7f0cb37fe700 1 -- 172.21.15.3:0/3298448288 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0011830 con 0x7f0cb409d9f0 2023-10-03T10:15:45.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.757+0000 7f0cb37fe700 1 -- 172.21.15.3:0/3298448288 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca0011990 con 0x7f0cb409d9f0 2023-10-03T10:15:45.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.757+0000 7f0cb37fe700 1 -- 172.21.15.3:0/3298448288 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0011d20 con 0x7f0cb409d9f0 2023-10-03T10:15:45.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.757+0000 7fe3a6ddd700 1 -- 172.21.15.3:0/1315667283 shutdown_connections 2023-10-03T10:15:45.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.757+0000 7fe3a6ddd700 1 --2- 172.21.15.3:0/1315667283 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a00a0a30 0x7fe3a00a0e00 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:45.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.757+0000 7fe3a6ddd700 1 --2- 172.21.15.3:0/1315667283 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3a0138d80 0x7fe3a013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:45.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.757+0000 7fe3a6ddd700 1 -- 172.21.15.3:0/1315667283 >> 172.21.15.3:0/1315667283 conn(0x7fe3a00a3050 msgr2=0x7fe3a00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:45.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.757+0000 7fe3a6ddd700 1 -- 172.21.15.3:0/1315667283 shutdown_connections 2023-10-03T10:15:45.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.757+0000 7fe3a6ddd700 1 -- 172.21.15.3:0/1315667283 wait complete. 2023-10-03T10:15:45.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.757+0000 7f0cbaf55700 1 -- 172.21.15.3:0/3298448288 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cb409d9f0 msgr2=0x7f0cb406a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:45.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.757+0000 7f0cbaf55700 1 --2- 172.21.15.3:0/3298448288 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cb409d9f0 0x7f0cb406a770 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009840 tx=0x7f0ca0009b50 comp rx=0 tx=0).stop 2023-10-03T10:15:45.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.758+0000 7f0cbaf55700 1 -- 172.21.15.3:0/3298448288 shutdown_connections 2023-10-03T10:15:45.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.758+0000 7f0cbaf55700 1 --2- 172.21.15.3:0/3298448288 >> v2:172.21.15.3:3300/0 conn(0x7f0cb41316c0 0x7f0cb4131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:45.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.758+0000 7f0cbaf55700 1 --2- 172.21.15.3:0/3298448288 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cb409d9f0 0x7f0cb406a770 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:45.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.758+0000 7f0cbaf55700 1 -- 172.21.15.3:0/3298448288 >> 172.21.15.3:0/3298448288 conn(0x7f0cb4066f50 msgr2=0x7f0cb409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:45.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.758+0000 7f0cbaf55700 1 -- 172.21.15.3:0/3298448288 shutdown_connections 2023-10-03T10:15:45.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.758+0000 7f0cbaf55700 1 -- 172.21.15.3:0/3298448288 wait complete. 2023-10-03T10:15:45.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.759+0000 7fe3a6ddd700 1 Processor -- start 2023-10-03T10:15:45.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.759+0000 7f0cbaf55700 1 Processor -- start 2023-10-03T10:15:45.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.759+0000 7fe3a6ddd700 1 -- start start 2023-10-03T10:15:45.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.759+0000 7f0cbaf55700 1 -- start start 2023-10-03T10:15:45.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.759+0000 7f0cbaf55700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cb406acb0 0x7f0cb41cca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:45.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.759+0000 7f0cbaf55700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0cb409d9f0 0x7f0cb41ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:45.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.759+0000 7f0cbaf55700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cb41316c0 0x7f0cb41294d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:45.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.759+0000 7f0cbaf55700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb411a460 con 0x7f0cb41316c0 2023-10-03T10:15:45.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.759+0000 7f0cbaf55700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb41cc660 con 0x7f0cb406acb0 2023-10-03T10:15:45.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.759+0000 7f0cbaf55700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb40a35f0 con 0x7f0cb409d9f0 2023-10-03T10:15:45.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.760+0000 7f0cb8cf1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cb406acb0 0x7f0cb41cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:45.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.760+0000 7f0cb8cf1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cb406acb0 0x7f0cb41cca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34658/0 (socket says 172.21.15.3:34658) 2023-10-03T10:15:45.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.760+0000 7f0cb8cf1700 1 -- 172.21.15.3:0/710026566 learned_addr learned my addr 172.21.15.3:0/710026566 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:45.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.760+0000 7fe3a6ddd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a009d630 0x7fe3a0133a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:45.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.760+0000 7fe3a6ddd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3a00a0a30 0x7fe3a0133f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:45.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.760+0000 7fe3a6ddd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3a0138d80 0x7fe3a0134480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:45.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.760+0000 7fe3a6ddd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a0138150 con 0x7fe3a00a0a30 2023-10-03T10:15:45.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.760+0000 7fe3a6ddd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a01371f0 con 0x7fe3a009d630 2023-10-03T10:15:45.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.760+0000 7fe3a6ddd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a00a4d70 con 0x7fe3a0138d80 2023-10-03T10:15:45.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.760+0000 7fe39ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3a00a0a30 0x7fe3a0133f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:45.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.760+0000 7fe39ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3a00a0a30 0x7fe3a0133f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55534/0 (socket says 172.21.15.3:55534) 2023-10-03T10:15:45.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.760+0000 7fe3a4b79700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a009d630 0x7fe3a0133a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:45.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.760+0000 7fe39ffff700 1 -- 172.21.15.3:0/1490792717 learned_addr learned my addr 172.21.15.3:0/1490792717 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:45.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.760+0000 7f0cb94f2700 1 --2- 172.21.15.3:0/710026566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cb41316c0 0x7f0cb41294d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:45.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.760+0000 7f0cb3fff700 1 --2- 172.21.15.3:0/710026566 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0cb409d9f0 0x7f0cb41ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:45.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.760+0000 7fe3a4b79700 1 -- 172.21.15.3:0/1490792717 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3a0138d80 msgr2=0x7fe3a0134480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:45.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.760+0000 7fe3a4b79700 1 --2- 172.21.15.3:0/1490792717 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3a0138d80 0x7fe3a0134480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:45.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.760+0000 7fe3a4b79700 1 -- 172.21.15.3:0/1490792717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3a00a0a30 msgr2=0x7fe3a0133f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:45.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.760+0000 7fe3a4b79700 1 --2- 172.21.15.3:0/1490792717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3a00a0a30 0x7fe3a0133f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:45.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.760+0000 7fe3a4b79700 1 -- 172.21.15.3:0/1490792717 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe390009550 con 0x7fe3a009d630 2023-10-03T10:15:45.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.761+0000 7fe3a4b79700 1 --2- 172.21.15.3:0/1490792717 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a009d630 0x7fe3a0133a00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe390016000 tx=0x7fe390004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:45.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.761+0000 7f0cb94f2700 1 -- 172.21.15.3:0/710026566 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0cb409d9f0 msgr2=0x7f0cb41ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:45.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.761+0000 7f0cb94f2700 1 --2- 172.21.15.3:0/710026566 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0cb409d9f0 0x7f0cb41ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:45.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.761+0000 7f0cb94f2700 1 -- 172.21.15.3:0/710026566 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cb406acb0 msgr2=0x7f0cb41cca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:45.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.761+0000 7f0cb94f2700 1 --2- 172.21.15.3:0/710026566 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cb406acb0 0x7f0cb41cca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:45.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.761+0000 7f0cb94f2700 1 -- 172.21.15.3:0/710026566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca80094a0 con 0x7f0cb41316c0 2023-10-03T10:15:45.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.761+0000 7fe39d7fa700 1 -- 172.21.15.3:0/1490792717 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe390004510 con 0x7fe3a009d630 2023-10-03T10:15:45.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.761+0000 7fe39d7fa700 1 -- 172.21.15.3:0/1490792717 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39001f070 con 0x7fe3a009d630 2023-10-03T10:15:45.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.761+0000 7fe39d7fa700 1 -- 172.21.15.3:0/1490792717 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3900038c0 con 0x7fe3a009d630 2023-10-03T10:15:45.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.761+0000 7f0cb94f2700 1 --2- 172.21.15.3:0/710026566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cb41316c0 0x7f0cb41294d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f0cac000f80 tx=0x7f0cac00cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:45.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.761+0000 7fe3a6ddd700 1 -- 172.21.15.3:0/1490792717 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a00a0410 con 0x7fe3a009d630 2023-10-03T10:15:45.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.761+0000 7fe3a6ddd700 1 -- 172.21.15.3:0/1490792717 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a0134dd0 con 0x7fe3a009d630 2023-10-03T10:15:45.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.762+0000 7f0cb17fa700 1 -- 172.21.15.3:0/710026566 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac00aaa0 con 0x7f0cb41316c0 2023-10-03T10:15:45.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.762+0000 7f0cb17fa700 1 -- 172.21.15.3:0/710026566 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cac00ac20 con 0x7f0cb41316c0 2023-10-03T10:15:45.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.762+0000 7f0cb17fa700 1 -- 172.21.15.3:0/710026566 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0070d0 con 0x7f0cb41316c0 2023-10-03T10:15:45.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.762+0000 7fe39d7fa700 1 -- 172.21.15.3:0/1490792717 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe390003a20 con 0x7fe3a009d630 2023-10-03T10:15:45.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.762+0000 7fe39d7fa700 1 -- 172.21.15.3:0/1490792717 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe390021070 con 0x7fe3a009d630 2023-10-03T10:15:45.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.763+0000 7fe3a6ddd700 1 -- 172.21.15.3:0/1490792717 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3800052e0 con 0x7fe3a009d630 2023-10-03T10:15:45.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.762+0000 7f0cbaf55700 1 -- 172.21.15.3:0/710026566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca00094a0 con 0x7f0cb41316c0 2023-10-03T10:15:45.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.762+0000 7f0cbaf55700 1 -- 172.21.15.3:0/710026566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb4119dc0 con 0x7f0cb41316c0 2023-10-03T10:15:45.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.763+0000 7f0cbaf55700 1 -- 172.21.15.3:0/710026566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb4074ea0 con 0x7f0cb41316c0 2023-10-03T10:15:45.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.764+0000 7f0cb17fa700 1 -- 172.21.15.3:0/710026566 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0cac007230 con 0x7f0cb41316c0 2023-10-03T10:15:45.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.764+0000 7f0cb17fa700 1 -- 172.21.15.3:0/710026566 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0cac03f8c0 con 0x7f0cb41316c0 2023-10-03T10:15:45.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.768+0000 7fe39d7fa700 1 -- 172.21.15.3:0/1490792717 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe39001a430 con 0x7fe3a009d630 2023-10-03T10:15:45.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.773+0000 7f0cb17fa700 1 -- 172.21.15.3:0/710026566 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0cac01b360 con 0x7f0cb41316c0 2023-10-03T10:15:45.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.800+0000 7fdabf15c700 1 Processor -- start 2023-10-03T10:15:45.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.800+0000 7fdabf15c700 1 -- start start 2023-10-03T10:15:45.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.800+0000 7fdabf15c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdab809e1d0 0x7fdab8135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:45.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.800+0000 7fdabf15c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdab8135e50 0x7fdab8138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:45.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.800+0000 7fdabf15c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdab809f130 con 0x7fdab809d6f0 2023-10-03T10:15:45.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.800+0000 7fdabf15c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdab809f520 con 0x7fdab8135e50 2023-10-03T10:15:45.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.800+0000 7fdabf15c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdab809e930 con 0x7fdab809e1d0 2023-10-03T10:15:45.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.800+0000 7fdabd6f9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdab8135e50 0x7fdab8138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:45.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.800+0000 7fdabd6f9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdab8135e50 0x7fdab8138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55536/0 (socket says 172.21.15.3:55536) 2023-10-03T10:15:45.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.800+0000 7fdabd6f9700 1 -- 172.21.15.3:0/3003872547 learned_addr learned my addr 172.21.15.3:0/3003872547 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:45.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.801+0000 7fdab7fff700 1 --2- 172.21.15.3:0/3003872547 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdab809e1d0 0x7fdab8135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:45.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.801+0000 7fdab7fff700 1 -- 172.21.15.3:0/3003872547 >> v2:172.21.15.3:3300/0 conn(0x7fdab8135e50 msgr2=0x7fdab8138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:45.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.801+0000 7fdab77fe700 1 -- 172.21.15.3:0/3003872547 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1711569751 0 0) 0x7fdab809f130 con 0x7fdab809d6f0 2023-10-03T10:15:45.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.801+0000 7fdab7fff700 1 --2- 172.21.15.3:0/3003872547 >> v2:172.21.15.3:3300/0 conn(0x7fdab8135e50 0x7fdab8138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:45.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.801+0000 7fdab7fff700 1 -- 172.21.15.3:0/3003872547 >> v1:172.21.15.3:6789/0 conn(0x7fdab809d6f0 legacy=0x7fdab809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:45.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.801+0000 7fdab7fff700 1 -- 172.21.15.3:0/3003872547 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab809b2d0 con 0x7fdab809e1d0 2023-10-03T10:15:45.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.801+0000 7fdabd6f9700 1 --2- 172.21.15.3:0/3003872547 >> v2:172.21.15.3:3300/0 conn(0x7fdab8135e50 0x7fdab8138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:45.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.801+0000 7fdab7fff700 1 --2- 172.21.15.3:0/3003872547 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdab809e1d0 0x7fdab8135910 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4009870 tx=0x7fdaa4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7bf302212e8ad35 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:45.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.801+0000 7fdab77fe700 1 -- 172.21.15.3:0/3003872547 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4011830 con 0x7fdab809e1d0 2023-10-03T10:15:45.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.802+0000 7fdab77fe700 1 -- 172.21.15.3:0/3003872547 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa4011990 con 0x7fdab809e1d0 2023-10-03T10:15:45.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.802+0000 7fdab77fe700 1 -- 172.21.15.3:0/3003872547 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4011d20 con 0x7fdab809e1d0 2023-10-03T10:15:45.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.802+0000 7fdabf15c700 1 -- 172.21.15.3:0/3003872547 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdab809e1d0 msgr2=0x7fdab8135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:45.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.802+0000 7fdabf15c700 1 --2- 172.21.15.3:0/3003872547 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdab809e1d0 0x7fdab8135910 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4009870 tx=0x7fdaa4009b80 comp rx=0 tx=0).stop 2023-10-03T10:15:45.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.802+0000 7fdabf15c700 1 -- 172.21.15.3:0/3003872547 shutdown_connections 2023-10-03T10:15:45.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.802+0000 7fdabf15c700 1 --2- 172.21.15.3:0/3003872547 >> v2:172.21.15.3:3300/0 conn(0x7fdab8135e50 0x7fdab8138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:45.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.802+0000 7fdabf15c700 1 --2- 172.21.15.3:0/3003872547 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdab809e1d0 0x7fdab8135910 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:45.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.802+0000 7fdabf15c700 1 -- 172.21.15.3:0/3003872547 >> 172.21.15.3:0/3003872547 conn(0x7fdab8066f50 msgr2=0x7fdab809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:45.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.802+0000 7fdabf15c700 1 -- 172.21.15.3:0/3003872547 shutdown_connections 2023-10-03T10:15:45.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.802+0000 7fdabf15c700 1 -- 172.21.15.3:0/3003872547 wait complete. 2023-10-03T10:15:45.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.803+0000 7fdabf15c700 1 Processor -- start 2023-10-03T10:15:45.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.804+0000 7fdabf15c700 1 -- start start 2023-10-03T10:15:45.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.804+0000 7fdabf15c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdab809d6f0 0x7fdab811d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:45.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.804+0000 7fdabf15c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdab809e1d0 0x7fdab811da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:45.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.804+0000 7fdabf15c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdab8135e50 0x7fdab811df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:45.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.804+0000 7fdabf15c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab811e9e0 con 0x7fdab809e1d0 2023-10-03T10:15:45.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.804+0000 7fdabf15c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab811f120 con 0x7fdab809d6f0 2023-10-03T10:15:45.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.804+0000 7fdabf15c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdab8121030 con 0x7fdab8135e50 2023-10-03T10:15:45.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.804+0000 7fdabd6f9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdab8135e50 0x7fdab811df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:45.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.804+0000 7fdabcef8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdab809d6f0 0x7fdab811d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.805+0000 7fdabd6f9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdab8135e50 0x7fdab811df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:46896/0 (socket says 172.21.15.3:46896) 2023-10-03T10:15:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.805+0000 7fdabd6f9700 1 -- 172.21.15.3:0/2952081903 learned_addr learned my addr 172.21.15.3:0/2952081903 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.805+0000 7fdab7fff700 1 --2- 172.21.15.3:0/2952081903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdab809e1d0 0x7fdab811da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.805+0000 7fdabd6f9700 1 -- 172.21.15.3:0/2952081903 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdab809d6f0 msgr2=0x7fdab811d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.805+0000 7fdabd6f9700 1 --2- 172.21.15.3:0/2952081903 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdab809d6f0 0x7fdab811d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.805+0000 7fdabd6f9700 1 -- 172.21.15.3:0/2952081903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdab809e1d0 msgr2=0x7fdab811da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.805+0000 7fdabd6f9700 1 --2- 172.21.15.3:0/2952081903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdab809e1d0 0x7fdab811da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.805+0000 7fdabd6f9700 1 -- 172.21.15.3:0/2952081903 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab0009b30 con 0x7fdab8135e50 2023-10-03T10:15:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.805+0000 7fdabcef8700 1 --2- 172.21.15.3:0/2952081903 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdab809d6f0 0x7fdab811d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.805+0000 7fdab7fff700 1 --2- 172.21.15.3:0/2952081903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdab809e1d0 0x7fdab811da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:15:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.805+0000 7fdabd6f9700 1 --2- 172.21.15.3:0/2952081903 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdab8135e50 0x7fdab811df90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdab000ef90 tx=0x7fdab000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.805+0000 7fdab57fa700 1 -- 172.21.15.3:0/2952081903 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab0003c20 con 0x7fdab8135e50 2023-10-03T10:15:45.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.805+0000 7fdab57fa700 1 -- 172.21.15.3:0/2952081903 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab000f040 con 0x7fdab8135e50 2023-10-03T10:15:45.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.805+0000 7fdab57fa700 1 -- 172.21.15.3:0/2952081903 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab000c930 con 0x7fdab8135e50 2023-10-03T10:15:45.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.806+0000 7fdabf15c700 1 -- 172.21.15.3:0/2952081903 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa40094a0 con 0x7fdab8135e50 2023-10-03T10:15:45.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.806+0000 7fdabf15c700 1 -- 172.21.15.3:0/2952081903 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab8120d50 con 0x7fdab8135e50 2023-10-03T10:15:45.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.807+0000 7fdab57fa700 1 -- 172.21.15.3:0/2952081903 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdab000ca90 con 0x7fdab8135e50 2023-10-03T10:15:45.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.807+0000 7fdab57fa700 1 -- 172.21.15.3:0/2952081903 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fdab003fb40 con 0x7fdab8135e50 2023-10-03T10:15:45.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.807+0000 7fdabf15c700 1 -- 172.21.15.3:0/2952081903 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdab81209f0 con 0x7fdab8135e50 2023-10-03T10:15:45.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.817+0000 7fdab57fa700 1 -- 172.21.15.3:0/2952081903 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdab81209f0 con 0x7fdab8135e50 2023-10-03T10:15:45.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.854+0000 7f48fe33c700 1 Processor -- start 2023-10-03T10:15:45.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.854+0000 7f48fe33c700 1 -- start start 2023-10-03T10:15:45.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.855+0000 7f48fe33c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48f809e1d0 0x7f48f8138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:45.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.855+0000 7f48fe33c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48f8139170 0x7f48f813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:45.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.855+0000 7f48fe33c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f809b120 con 0x7f48f809d6f0 2023-10-03T10:15:45.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.855+0000 7f48fe33c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f806b460 con 0x7f48f809e1d0 2023-10-03T10:15:45.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.855+0000 7f48fe33c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f48f809a7e0 con 0x7f48f8139170 2023-10-03T10:15:45.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.855+0000 7f48fc8d9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48f8139170 0x7f48f813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:45.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.855+0000 7f48f77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48f809e1d0 0x7f48f8138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:45.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.855+0000 7f48fc8d9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48f8139170 0x7f48f813b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:46902/0 (socket says 172.21.15.3:46902) 2023-10-03T10:15:45.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.855+0000 7f48f77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48f809e1d0 0x7f48f8138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34682/0 (socket says 172.21.15.3:34682) 2023-10-03T10:15:45.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.855+0000 7f48fc8d9700 1 -- 172.21.15.3:0/3656487642 learned_addr learned my addr 172.21.15.3:0/3656487642 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:45.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.855+0000 7f48fc8d9700 1 -- 172.21.15.3:0/3656487642 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48f809e1d0 msgr2=0x7f48f8138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:45.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.855+0000 7f48fc8d9700 1 --2- 172.21.15.3:0/3656487642 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48f809e1d0 0x7f48f8138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:45.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.855+0000 7f48fc8d9700 1 -- 172.21.15.3:0/3656487642 >> v1:172.21.15.3:6789/0 conn(0x7f48f809d6f0 legacy=0x7f48f809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:45.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.855+0000 7f48fc8d9700 1 -- 172.21.15.3:0/3656487642 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f809a500 con 0x7f48f8139170 2023-10-03T10:15:45.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.856+0000 7f48fc8d9700 1 --2- 172.21.15.3:0/3656487642 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48f8139170 0x7f48f813b550 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f48f0009eb0 tx=0x7f48f000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d369720491d6ac25 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:45.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.856+0000 7f48f6ffd700 1 -- 172.21.15.3:0/3656487642 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f000e070 con 0x7f48f8139170 2023-10-03T10:15:45.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.856+0000 7f48f6ffd700 1 -- 172.21.15.3:0/3656487642 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f0003a20 con 0x7f48f8139170 2023-10-03T10:15:45.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.856+0000 7f48f6ffd700 1 -- 172.21.15.3:0/3656487642 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f0003db0 con 0x7f48f8139170 2023-10-03T10:15:45.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.857+0000 7f48fe33c700 1 -- 172.21.15.3:0/3656487642 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48f8139170 msgr2=0x7f48f813b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:45.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.857+0000 7f48fe33c700 1 --2- 172.21.15.3:0/3656487642 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48f8139170 0x7f48f813b550 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f48f0009eb0 tx=0x7f48f000b330 comp rx=0 tx=0).stop 2023-10-03T10:15:45.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.857+0000 7f48fe33c700 1 -- 172.21.15.3:0/3656487642 shutdown_connections 2023-10-03T10:15:45.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.857+0000 7f48fe33c700 1 --2- 172.21.15.3:0/3656487642 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48f8139170 0x7f48f813b550 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:45.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.857+0000 7f48fe33c700 1 --2- 172.21.15.3:0/3656487642 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48f809e1d0 0x7f48f8138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:45.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.857+0000 7f48fe33c700 1 -- 172.21.15.3:0/3656487642 >> 172.21.15.3:0/3656487642 conn(0x7f48f8066f50 msgr2=0x7f48f809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:45.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.857+0000 7f48fe33c700 1 -- 172.21.15.3:0/3656487642 shutdown_connections 2023-10-03T10:15:45.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.857+0000 7f48fe33c700 1 -- 172.21.15.3:0/3656487642 wait complete. 2023-10-03T10:15:45.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.858+0000 7f48fe33c700 1 Processor -- start 2023-10-03T10:15:45.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.858+0000 7f48fe33c700 1 -- start start 2023-10-03T10:15:45.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.859+0000 7f48fe33c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48f809d6f0 0x7f48f811e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:45.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.859+0000 7f48fe33c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48f809e1d0 0x7f48f811ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:45.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.859+0000 7f48fe33c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48f8139170 0x7f48f811f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:45.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.859+0000 7f48fe33c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f81206e0 con 0x7f48f809d6f0 2023-10-03T10:15:45.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.859+0000 7f48fe33c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f8122690 con 0x7f48f809e1d0 2023-10-03T10:15:45.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.859+0000 7f48fe33c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f48f8122190 con 0x7f48f8139170 2023-10-03T10:15:45.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.859+0000 7f48f7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48f809d6f0 0x7f48f811e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:45.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.859+0000 7f48fc8d9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48f8139170 0x7f48f811f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:45.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.859+0000 7f48f7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48f809d6f0 0x7f48f811e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55550/0 (socket says 172.21.15.3:55550) 2023-10-03T10:15:45.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.859+0000 7f48fc8d9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48f8139170 0x7f48f811f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:46910/0 (socket says 172.21.15.3:46910) 2023-10-03T10:15:45.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.859+0000 7f48f7fff700 1 -- 172.21.15.3:0/3424253609 learned_addr learned my addr 172.21.15.3:0/3424253609 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.859+0000 7f48f77fe700 1 --2- 172.21.15.3:0/3424253609 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48f809e1d0 0x7f48f811ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.859+0000 7f48f7fff700 1 -- 172.21.15.3:0/3424253609 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48f8139170 msgr2=0x7f48f811f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.859+0000 7f48f7fff700 1 --2- 172.21.15.3:0/3424253609 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48f8139170 0x7f48f811f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.859+0000 7f48f7fff700 1 -- 172.21.15.3:0/3424253609 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48f809e1d0 msgr2=0x7f48f811ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.859+0000 7f48f7fff700 1 --2- 172.21.15.3:0/3424253609 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48f809e1d0 0x7f48f811ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.859+0000 7f48f7fff700 1 -- 172.21.15.3:0/3424253609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f0009b30 con 0x7f48f809d6f0 2023-10-03T10:15:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.860+0000 7f48f77fe700 1 --2- 172.21.15.3:0/3424253609 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48f809e1d0 0x7f48f811ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:15:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.860+0000 7f48f7fff700 1 --2- 172.21.15.3:0/3424253609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48f809d6f0 0x7f48f811e950 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f48e800c750 tx=0x7f48e800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:45.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.860+0000 7f48f4ff9700 1 -- 172.21.15.3:0/3424253609 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e8005480 con 0x7f48f809d6f0 2023-10-03T10:15:45.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.860+0000 7f48f4ff9700 1 -- 172.21.15.3:0/3424253609 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e8003680 con 0x7f48f809d6f0 2023-10-03T10:15:45.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.860+0000 7f48fe33c700 1 -- 172.21.15.3:0/3424253609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f8121c00 con 0x7f48f809d6f0 2023-10-03T10:15:45.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.860+0000 7f48f4ff9700 1 -- 172.21.15.3:0/3424253609 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e8003a30 con 0x7f48f809d6f0 2023-10-03T10:15:45.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.860+0000 7f48fe33c700 1 -- 172.21.15.3:0/3424253609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f8120d50 con 0x7f48f809d6f0 2023-10-03T10:15:45.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.861+0000 7f48f4ff9700 1 -- 172.21.15.3:0/3424253609 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f48e8003b90 con 0x7f48f809d6f0 2023-10-03T10:15:45.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.861+0000 7f48fe33c700 1 -- 172.21.15.3:0/3424253609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48f8074ea0 con 0x7f48f809d6f0 2023-10-03T10:15:45.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.861+0000 7f48f4ff9700 1 -- 172.21.15.3:0/3424253609 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f48e803f160 con 0x7f48f809d6f0 2023-10-03T10:15:45.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.866+0000 7f48f4ff9700 1 -- 172.21.15.3:0/3424253609 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f48e8003e40 con 0x7f48f809d6f0 2023-10-03T10:15:45.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.984+0000 7f0cbaf55700 1 -- 172.21.15.3:0/710026566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0cb409e130 con 0x7f0cb41316c0 2023-10-03T10:15:45.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.984+0000 7f0cb17fa700 1 -- 172.21.15.3:0/710026566 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0cb409e130 con 0x7f0cb41316c0 2023-10-03T10:15:45.985 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:45.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.986+0000 7fe3a6ddd700 1 -- 172.21.15.3:0/1490792717 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe380005590 con 0x7fe3a009d630 2023-10-03T10:15:45.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.986+0000 7fe39d7fa700 1 -- 172.21.15.3:0/1490792717 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe390015020 con 0x7fe3a009d630 2023-10-03T10:15:45.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.987+0000 7f0cbaf55700 1 -- 172.21.15.3:0/710026566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cb41316c0 msgr2=0x7f0cb41294d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:45.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.987+0000 7f0cbaf55700 1 --2- 172.21.15.3:0/710026566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cb41316c0 0x7f0cb41294d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f0cac000f80 tx=0x7f0cac00cee0 comp rx=0 tx=0).stop 2023-10-03T10:15:45.987 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:45.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.987+0000 7f0cbaf55700 1 -- 172.21.15.3:0/710026566 shutdown_connections 2023-10-03T10:15:45.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.987+0000 7f0cbaf55700 1 --2- 172.21.15.3:0/710026566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cb41316c0 0x7f0cb41294d0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:45.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.987+0000 7f0cbaf55700 1 --2- 172.21.15.3:0/710026566 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cb406acb0 0x7f0cb41cca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:45.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.987+0000 7f0cbaf55700 1 --2- 172.21.15.3:0/710026566 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0cb409d9f0 0x7f0cb41ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:45.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.987+0000 7f0cbaf55700 1 -- 172.21.15.3:0/710026566 >> 172.21.15.3:0/710026566 conn(0x7f0cb4066f50 msgr2=0x7f0cb41300e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:45.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.987+0000 7f0cbaf55700 1 -- 172.21.15.3:0/710026566 shutdown_connections 2023-10-03T10:15:45.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.987+0000 7f0cbaf55700 1 -- 172.21.15.3:0/710026566 wait complete. 2023-10-03T10:15:45.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.990+0000 7fe3a6ddd700 1 -- 172.21.15.3:0/1490792717 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a009d630 msgr2=0x7fe3a0133a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:45.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.990+0000 7fe3a6ddd700 1 --2- 172.21.15.3:0/1490792717 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a009d630 0x7fe3a0133a00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe390016000 tx=0x7fe390004750 comp rx=0 tx=0).stop 2023-10-03T10:15:45.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.990+0000 7fe3a6ddd700 1 -- 172.21.15.3:0/1490792717 shutdown_connections 2023-10-03T10:15:45.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.990+0000 7fe3a6ddd700 1 --2- 172.21.15.3:0/1490792717 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3a0138d80 0x7fe3a0134480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:45.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.990+0000 7fe3a6ddd700 1 --2- 172.21.15.3:0/1490792717 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a009d630 0x7fe3a0133a00 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:45.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.990+0000 7fe3a6ddd700 1 --2- 172.21.15.3:0/1490792717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3a00a0a30 0x7fe3a0133f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:45.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.990+0000 7fe3a6ddd700 1 -- 172.21.15.3:0/1490792717 >> 172.21.15.3:0/1490792717 conn(0x7fe3a00a3050 msgr2=0x7fe3a01398e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:45.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.990+0000 7fe3a6ddd700 1 -- 172.21.15.3:0/1490792717 shutdown_connections 2023-10-03T10:15:45.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:45.990+0000 7fe3a6ddd700 1 -- 172.21.15.3:0/1490792717 wait complete. 2023-10-03T10:15:46.002 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:46.011 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:46.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:46.043+0000 7fdabf15c700 1 -- 172.21.15.3:0/2952081903 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdab81209f0 con 0x7fdab8135e50 2023-10-03T10:15:46.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:46.043+0000 7fdab57fa700 1 -- 172.21.15.3:0/2952081903 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdab81209f0 con 0x7fdab8135e50 2023-10-03T10:15:46.043 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:46.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:46.045+0000 7fdabf15c700 1 -- 172.21.15.3:0/2952081903 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdab8135e50 msgr2=0x7fdab811df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:46.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:46.045+0000 7fdabf15c700 1 --2- 172.21.15.3:0/2952081903 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdab8135e50 0x7fdab811df90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdab000ef90 tx=0x7fdab000c3a0 comp rx=0 tx=0).stop 2023-10-03T10:15:46.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:46.046+0000 7fdabf15c700 1 -- 172.21.15.3:0/2952081903 shutdown_connections 2023-10-03T10:15:46.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:46.046+0000 7fdabf15c700 1 --2- 172.21.15.3:0/2952081903 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdab8135e50 0x7fdab811df90 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:46.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:46.046+0000 7fdabf15c700 1 --2- 172.21.15.3:0/2952081903 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdab809d6f0 0x7fdab811d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:46.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:46.046+0000 7fdabf15c700 1 --2- 172.21.15.3:0/2952081903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdab809e1d0 0x7fdab811da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:46.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:46.046+0000 7fdabf15c700 1 -- 172.21.15.3:0/2952081903 >> 172.21.15.3:0/2952081903 conn(0x7fdab8066f50 msgr2=0x7fdab8071110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:46.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:46.046+0000 7fdabf15c700 1 -- 172.21.15.3:0/2952081903 shutdown_connections 2023-10-03T10:15:46.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:46.046+0000 7fdabf15c700 1 -- 172.21.15.3:0/2952081903 wait complete. 2023-10-03T10:15:46.060 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:46.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:46.083+0000 7f48fe33c700 1 -- 172.21.15.3:0/3424253609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f48f806c360 con 0x7f48f809d6f0 2023-10-03T10:15:46.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:46.084+0000 7f48f4ff9700 1 -- 172.21.15.3:0/3424253609 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f48e801a360 con 0x7f48f809d6f0 2023-10-03T10:15:46.084 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:46.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:46.086+0000 7f48fe33c700 1 -- 172.21.15.3:0/3424253609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48f809d6f0 msgr2=0x7f48f811e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:46.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:46.086+0000 7f48fe33c700 1 --2- 172.21.15.3:0/3424253609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48f809d6f0 0x7f48f811e950 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f48e800c750 tx=0x7f48e800ca60 comp rx=0 tx=0).stop 2023-10-03T10:15:46.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:46.086+0000 7f48fe33c700 1 -- 172.21.15.3:0/3424253609 shutdown_connections 2023-10-03T10:15:46.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:46.086+0000 7f48fe33c700 1 --2- 172.21.15.3:0/3424253609 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48f8139170 0x7f48f811f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:46.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:46.086+0000 7f48fe33c700 1 --2- 172.21.15.3:0/3424253609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48f809d6f0 0x7f48f811e950 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:46.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:46.086+0000 7f48fe33c700 1 --2- 172.21.15.3:0/3424253609 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48f809e1d0 0x7f48f811ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:46.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:46.086+0000 7f48fe33c700 1 -- 172.21.15.3:0/3424253609 >> 172.21.15.3:0/3424253609 conn(0x7f48f8066f50 msgr2=0x7f48f8070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:46.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:46.086+0000 7f48fe33c700 1 -- 172.21.15.3:0/3424253609 shutdown_connections 2023-10-03T10:15:46.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:46.086+0000 7f48fe33c700 1 -- 172.21.15.3:0/3424253609 wait complete. 2023-10-03T10:15:46.100 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:47.003 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:47.012 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:47.061 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:47.101 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:47.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.158+0000 7f3b79b1a700 1 Processor -- start 2023-10-03T10:15:47.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.158+0000 7f3b79b1a700 1 -- start start 2023-10-03T10:15:47.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.159+0000 7f3b79b1a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b7409d6f0 0x7f3b7409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:47.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.159+0000 7f3b79b1a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3b7409e410 0x7f3b74130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:47.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.159+0000 7f3b79b1a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b7409f130 con 0x7f3b7409e000 2023-10-03T10:15:47.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.159+0000 7f3b79b1a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b7409f520 con 0x7f3b7409e410 2023-10-03T10:15:47.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.159+0000 7f3b79b1a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7409e930 con 0x7f3b7409d6f0 2023-10-03T10:15:47.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.159+0000 7f3b737fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b7409d6f0 0x7f3b7409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:47.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.159+0000 7f3b73fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3b7409e410 0x7f3b74130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:47.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.159+0000 7f3b737fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b7409d6f0 0x7f3b7409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:46912/0 (socket says 172.21.15.3:46912) 2023-10-03T10:15:47.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.159+0000 7f3b737fe700 1 -- 172.21.15.3:0/697702927 learned_addr learned my addr 172.21.15.3:0/697702927 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:47.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.160+0000 7f3b737fe700 1 -- 172.21.15.3:0/697702927 >> v2:172.21.15.3:3300/0 conn(0x7f3b7409e410 msgr2=0x7f3b74130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:47.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.160+0000 7f3b737fe700 1 --2- 172.21.15.3:0/697702927 >> v2:172.21.15.3:3300/0 conn(0x7f3b7409e410 0x7f3b74130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.160+0000 7f3b737fe700 1 -- 172.21.15.3:0/697702927 >> v1:172.21.15.3:6789/0 conn(0x7f3b7409e000 legacy=0x7f3b74130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:47.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.160+0000 7f3b737fe700 1 -- 172.21.15.3:0/697702927 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7409b2d0 con 0x7f3b7409d6f0 2023-10-03T10:15:47.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.160+0000 7f3b737fe700 1 --2- 172.21.15.3:0/697702927 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b7409d6f0 0x7f3b7409dac0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3b640098f0 tx=0x7f3b64009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e189c178afe29ef server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:47.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.160+0000 7f3b727fc700 1 -- 172.21.15.3:0/697702927 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b64011830 con 0x7f3b7409d6f0 2023-10-03T10:15:47.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.160+0000 7f3b727fc700 1 -- 172.21.15.3:0/697702927 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b64011990 con 0x7f3b7409d6f0 2023-10-03T10:15:47.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.160+0000 7f3b727fc700 1 -- 172.21.15.3:0/697702927 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b64011d20 con 0x7f3b7409d6f0 2023-10-03T10:15:47.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.160+0000 7f3b73fff700 1 --2- 172.21.15.3:0/697702927 >> v2:172.21.15.3:3300/0 conn(0x7f3b7409e410 0x7f3b74130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:47.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.161+0000 7f3b79b1a700 1 -- 172.21.15.3:0/697702927 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b7409d6f0 msgr2=0x7f3b7409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:47.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.161+0000 7f3b79b1a700 1 --2- 172.21.15.3:0/697702927 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b7409d6f0 0x7f3b7409dac0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3b640098f0 tx=0x7f3b64009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:47.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.161+0000 7f3b79b1a700 1 -- 172.21.15.3:0/697702927 shutdown_connections 2023-10-03T10:15:47.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.161+0000 7f3b79b1a700 1 --2- 172.21.15.3:0/697702927 >> v2:172.21.15.3:3300/0 conn(0x7f3b7409e410 0x7f3b74130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.161+0000 7f3b79b1a700 1 --2- 172.21.15.3:0/697702927 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b7409d6f0 0x7f3b7409dac0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.161+0000 7f3b79b1a700 1 -- 172.21.15.3:0/697702927 >> 172.21.15.3:0/697702927 conn(0x7f3b74066f50 msgr2=0x7f3b7409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:47.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.161+0000 7f3b79b1a700 1 -- 172.21.15.3:0/697702927 shutdown_connections 2023-10-03T10:15:47.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.161+0000 7f3b79b1a700 1 -- 172.21.15.3:0/697702927 wait complete. 2023-10-03T10:15:47.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.163+0000 7f3b79b1a700 1 Processor -- start 2023-10-03T10:15:47.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.163+0000 7f3b79b1a700 1 -- start start 2023-10-03T10:15:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.164+0000 7f3b79b1a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b7409d6f0 0x7f3b741301b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.164+0000 7f3b79b1a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b7409e000 0x7f3b741306f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.164+0000 7f3b79b1a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b7409e410 0x7f3b74130c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.164+0000 7f3b79b1a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7412dd00 con 0x7f3b7409e410 2023-10-03T10:15:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.164+0000 7f3b79b1a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7412ed80 con 0x7f3b7409e000 2023-10-03T10:15:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.164+0000 7f3b79b1a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7412e930 con 0x7f3b7409d6f0 2023-10-03T10:15:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.164+0000 7f3b73fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b7409e410 0x7f3b74130c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.164+0000 7f3b72ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b7409e000 0x7f3b741306f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.164+0000 7f3b737fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b7409d6f0 0x7f3b741301b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.164+0000 7f3b73fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b7409e410 0x7f3b74130c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55566/0 (socket says 172.21.15.3:55566) 2023-10-03T10:15:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.164+0000 7f3b73fff700 1 -- 172.21.15.3:0/947130709 learned_addr learned my addr 172.21.15.3:0/947130709 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.165+0000 7f3b72ffd700 1 -- 172.21.15.3:0/947130709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b7409d6f0 msgr2=0x7f3b741301b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.165+0000 7f3b72ffd700 1 --2- 172.21.15.3:0/947130709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b7409d6f0 0x7f3b741301b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.165+0000 7f3b72ffd700 1 -- 172.21.15.3:0/947130709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b7409e410 msgr2=0x7f3b74130c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:47.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.165+0000 7f3b72ffd700 1 --2- 172.21.15.3:0/947130709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b7409e410 0x7f3b74130c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.165+0000 7f3b72ffd700 1 -- 172.21.15.3:0/947130709 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b6c009a70 con 0x7f3b7409e000 2023-10-03T10:15:47.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.165+0000 7f3b73fff700 1 --2- 172.21.15.3:0/947130709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b7409e410 0x7f3b74130c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:47.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.165+0000 7f3b72ffd700 1 --2- 172.21.15.3:0/947130709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b7409e000 0x7f3b741306f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3b68000f60 tx=0x7f3b6800c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:47.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.166+0000 7f3b5bfff700 1 -- 172.21.15.3:0/947130709 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b680050d0 con 0x7f3b7409e000 2023-10-03T10:15:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.166+0000 7f3b5bfff700 1 -- 172.21.15.3:0/947130709 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b68005230 con 0x7f3b7409e000 2023-10-03T10:15:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.166+0000 7f3b5bfff700 1 -- 172.21.15.3:0/947130709 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b68003680 con 0x7f3b7409e000 2023-10-03T10:15:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.166+0000 7f3b79b1a700 1 -- 172.21.15.3:0/947130709 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b64009550 con 0x7f3b7409e000 2023-10-03T10:15:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.166+0000 7f3b79b1a700 1 -- 172.21.15.3:0/947130709 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b7412f640 con 0x7f3b7409e000 2023-10-03T10:15:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.168+0000 7f3b79b1a700 1 -- 172.21.15.3:0/947130709 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b74074ea0 con 0x7f3b7409e000 2023-10-03T10:15:47.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.170+0000 7f3b5bfff700 1 -- 172.21.15.3:0/947130709 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3b680037e0 con 0x7f3b7409e000 2023-10-03T10:15:47.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.171+0000 7f3b5bfff700 1 -- 172.21.15.3:0/947130709 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3b6803ec30 con 0x7f3b7409e000 2023-10-03T10:15:47.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.179+0000 7f3b5bfff700 1 -- 172.21.15.3:0/947130709 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3b68003a90 con 0x7f3b7409e000 2023-10-03T10:15:47.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.202+0000 7fb4c0633700 1 Processor -- start 2023-10-03T10:15:47.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.202+0000 7fb4c0633700 1 -- start start 2023-10-03T10:15:47.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.203+0000 7fb4c0633700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb4b8130540 0x7fb4b8130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:47.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.203+0000 7fb4c0633700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb4b806fcf0 0x7fb4b8138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:47.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.203+0000 7fb4c0633700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b80716a0 con 0x7fb4b8130e50 2023-10-03T10:15:47.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.203+0000 7fb4c0633700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b809a390 con 0x7fb4b8130540 2023-10-03T10:15:47.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.203+0000 7fb4c0633700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b8071290 con 0x7fb4b806fcf0 2023-10-03T10:15:47.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.203+0000 7fb4be3cf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb4b8130540 0x7fb4b8130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:47.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.203+0000 7fb4bebd0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb4b806fcf0 0x7fb4b8138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:47.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.203+0000 7fb4be3cf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb4b8130540 0x7fb4b8130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55580/0 (socket says 172.21.15.3:55580) 2023-10-03T10:15:47.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.203+0000 7fb4be3cf700 1 -- 172.21.15.3:0/2513315811 learned_addr learned my addr 172.21.15.3:0/2513315811 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:47.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.203+0000 7fb4be3cf700 1 -- 172.21.15.3:0/2513315811 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb4b806fcf0 msgr2=0x7fb4b8138570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:47.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.203+0000 7fb4be3cf700 1 --2- 172.21.15.3:0/2513315811 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb4b806fcf0 0x7fb4b8138570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.203+0000 7fb4be3cf700 1 -- 172.21.15.3:0/2513315811 >> v1:172.21.15.3:6789/0 conn(0x7fb4b8130e50 legacy=0x7fb4b806f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:47.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.203+0000 7fb4be3cf700 1 -- 172.21.15.3:0/2513315811 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b8130340 con 0x7fb4b8130540 2023-10-03T10:15:47.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.203+0000 7fb4bebd0700 1 --2- 172.21.15.3:0/2513315811 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb4b806fcf0 0x7fb4b8138570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:47.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.204+0000 7fb4be3cf700 1 --2- 172.21.15.3:0/2513315811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4b8130540 0x7fb4b8130910 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fb4b40098f0 tx=0x7fb4b4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca863b8e3b4a4146 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:47.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.204+0000 7fb4bd3cd700 1 -- 172.21.15.3:0/2513315811 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4010a90 con 0x7fb4b8130540 2023-10-03T10:15:47.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.204+0000 7fb4bd3cd700 1 -- 172.21.15.3:0/2513315811 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b4010bf0 con 0x7fb4b8130540 2023-10-03T10:15:47.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.204+0000 7fb4bd3cd700 1 -- 172.21.15.3:0/2513315811 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4018440 con 0x7fb4b8130540 2023-10-03T10:15:47.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.205+0000 7fb4c0633700 1 -- 172.21.15.3:0/2513315811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4b8130540 msgr2=0x7fb4b8130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:47.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.205+0000 7fb4c0633700 1 --2- 172.21.15.3:0/2513315811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4b8130540 0x7fb4b8130910 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fb4b40098f0 tx=0x7fb4b4009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:47.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.205+0000 7fb4c0633700 1 -- 172.21.15.3:0/2513315811 shutdown_connections 2023-10-03T10:15:47.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.205+0000 7fb4c0633700 1 --2- 172.21.15.3:0/2513315811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4b8130540 0x7fb4b8130910 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.205+0000 7fb4c0633700 1 --2- 172.21.15.3:0/2513315811 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb4b806fcf0 0x7fb4b8138570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.205+0000 7fb4c0633700 1 -- 172.21.15.3:0/2513315811 >> 172.21.15.3:0/2513315811 conn(0x7fb4b80a2aa0 msgr2=0x7fb4b80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:47.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.205+0000 7fb4c0633700 1 -- 172.21.15.3:0/2513315811 shutdown_connections 2023-10-03T10:15:47.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.205+0000 7fb4c0633700 1 -- 172.21.15.3:0/2513315811 wait complete. 2023-10-03T10:15:47.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.207+0000 7fb4c0633700 1 Processor -- start 2023-10-03T10:15:47.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.207+0000 7fb4c0633700 1 -- start start 2023-10-03T10:15:47.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.208+0000 7fb4c0633700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb4b806fcf0 0x7fb4b8118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:47.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.208+0000 7fb4c0633700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4b8130540 0x7fb4b8119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:47.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.208+0000 7fb4c0633700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4b8130e50 0x7fb4b8119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:47.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.208+0000 7fb4c0633700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b811c020 con 0x7fb4b8130540 2023-10-03T10:15:47.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.208+0000 7fb4c0633700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b811a1b0 con 0x7fb4b8130e50 2023-10-03T10:15:47.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.208+0000 7fb4c0633700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b80a0ce0 con 0x7fb4b806fcf0 2023-10-03T10:15:47.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.208+0000 7fb4bdbce700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4b8130540 0x7fb4b8119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:47.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.208+0000 7fb4bdbce700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4b8130540 0x7fb4b8119210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55582/0 (socket says 172.21.15.3:55582) 2023-10-03T10:15:47.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.208+0000 7fb4bdbce700 1 -- 172.21.15.3:0/1797159689 learned_addr learned my addr 172.21.15.3:0/1797159689 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:47.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.208+0000 7fb4be3cf700 1 --2- 172.21.15.3:0/1797159689 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb4b806fcf0 0x7fb4b8118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:47.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.208+0000 7fb4bebd0700 1 --2- 172.21.15.3:0/1797159689 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4b8130e50 0x7fb4b8119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:47.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.208+0000 7fb4bdbce700 1 -- 172.21.15.3:0/1797159689 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb4b806fcf0 msgr2=0x7fb4b8118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:47.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.208+0000 7fb4bdbce700 1 --2- 172.21.15.3:0/1797159689 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb4b806fcf0 0x7fb4b8118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.208+0000 7fb4bdbce700 1 -- 172.21.15.3:0/1797159689 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4b8130e50 msgr2=0x7fb4b8119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:47.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.208+0000 7fb4bdbce700 1 --2- 172.21.15.3:0/1797159689 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4b8130e50 0x7fb4b8119750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.208+0000 7fb4bdbce700 1 -- 172.21.15.3:0/1797159689 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b0009a70 con 0x7fb4b8130540 2023-10-03T10:15:47.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.209+0000 7fb4bdbce700 1 --2- 172.21.15.3:0/1797159689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4b8130540 0x7fb4b8119210 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800c4e0 tx=0x7fb4a800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:47.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.209+0000 7fb4aeffd700 1 -- 172.21.15.3:0/1797159689 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a80052d0 con 0x7fb4b8130540 2023-10-03T10:15:47.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.209+0000 7fb4aeffd700 1 -- 172.21.15.3:0/1797159689 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a8005430 con 0x7fb4b8130540 2023-10-03T10:15:47.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.209+0000 7fb4c0633700 1 -- 172.21.15.3:0/1797159689 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b4009550 con 0x7fb4b8130540 2023-10-03T10:15:47.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.209+0000 7fb4aeffd700 1 -- 172.21.15.3:0/1797159689 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a80038c0 con 0x7fb4b8130540 2023-10-03T10:15:47.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.209+0000 7fb4c0633700 1 -- 172.21.15.3:0/1797159689 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b811aea0 con 0x7fb4b8130540 2023-10-03T10:15:47.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.211+0000 7fb4c0633700 1 -- 172.21.15.3:0/1797159689 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b811a8f0 con 0x7fb4b8130540 2023-10-03T10:15:47.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.214+0000 7fb4aeffd700 1 -- 172.21.15.3:0/1797159689 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb4a8003a20 con 0x7fb4b8130540 2023-10-03T10:15:47.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.214+0000 7fb4aeffd700 1 -- 172.21.15.3:0/1797159689 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb4a803ed40 con 0x7fb4b8130540 2023-10-03T10:15:47.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.219+0000 7fb4aeffd700 1 -- 172.21.15.3:0/1797159689 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb4a8045080 con 0x7fb4b8130540 2023-10-03T10:15:47.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.252+0000 7f4487051700 1 Processor -- start 2023-10-03T10:15:47.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.252+0000 7f4487051700 1 -- start start 2023-10-03T10:15:47.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.253+0000 7f4487051700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f448009e1d0 0x7f4480138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:47.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.253+0000 7f4487051700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4480139170 0x7f448013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:47.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.253+0000 7f4487051700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f448009b120 con 0x7f448009d6f0 2023-10-03T10:15:47.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.253+0000 7f4487051700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f448006b460 con 0x7f4480139170 2023-10-03T10:15:47.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.253+0000 7f4487051700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f448009a7e0 con 0x7f448009e1d0 2023-10-03T10:15:47.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.253+0000 7f44855ee700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4480139170 0x7f448013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:47.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.253+0000 7f44855ee700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4480139170 0x7f448013b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34730/0 (socket says 172.21.15.3:34730) 2023-10-03T10:15:47.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.253+0000 7f44855ee700 1 -- 172.21.15.3:0/2969132342 learned_addr learned my addr 172.21.15.3:0/2969132342 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:47.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.253+0000 7f4484ded700 1 --1- 172.21.15.3:0/2969132342 >> v1:172.21.15.3:6789/0 conn(0x7f448009d6f0 0x7f448009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57260/0 (socket says 172.21.15.3:57260) 2023-10-03T10:15:47.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.253+0000 7f447ffff700 1 --2- 172.21.15.3:0/2969132342 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f448009e1d0 0x7f4480138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:47.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.253+0000 7f44855ee700 1 -- 172.21.15.3:0/2969132342 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f448009e1d0 msgr2=0x7f4480138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:47.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.253+0000 7f44855ee700 1 --2- 172.21.15.3:0/2969132342 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f448009e1d0 0x7f4480138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.253+0000 7f44855ee700 1 -- 172.21.15.3:0/2969132342 >> v1:172.21.15.3:6789/0 conn(0x7f448009d6f0 legacy=0x7f448009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:47.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.253+0000 7f44855ee700 1 -- 172.21.15.3:0/2969132342 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448009a500 con 0x7f4480139170 2023-10-03T10:15:47.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.253+0000 7f447f7fe700 1 -- 172.21.15.3:0/2969132342 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4117850393 0 0) 0x7f448009b120 con 0x7f448009d6f0 2023-10-03T10:15:47.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.254+0000 7f44855ee700 1 --2- 172.21.15.3:0/2969132342 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4480139170 0x7f448013b550 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4478009e80 tx=0x7f447800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1d818cadd2a83d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:47.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.254+0000 7f447f7fe700 1 -- 172.21.15.3:0/2969132342 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447800e070 con 0x7f4480139170 2023-10-03T10:15:47.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.254+0000 7f447f7fe700 1 -- 172.21.15.3:0/2969132342 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447800b910 con 0x7f4480139170 2023-10-03T10:15:47.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.254+0000 7f447f7fe700 1 -- 172.21.15.3:0/2969132342 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447800bcc0 con 0x7f4480139170 2023-10-03T10:15:47.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.255+0000 7f4487051700 1 -- 172.21.15.3:0/2969132342 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4480139170 msgr2=0x7f448013b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:47.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.255+0000 7f4487051700 1 --2- 172.21.15.3:0/2969132342 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4480139170 0x7f448013b550 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4478009e80 tx=0x7f447800b330 comp rx=0 tx=0).stop 2023-10-03T10:15:47.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.255+0000 7f4487051700 1 -- 172.21.15.3:0/2969132342 shutdown_connections 2023-10-03T10:15:47.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.255+0000 7f4487051700 1 --2- 172.21.15.3:0/2969132342 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4480139170 0x7f448013b550 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.255+0000 7f4487051700 1 --2- 172.21.15.3:0/2969132342 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f448009e1d0 0x7f4480138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.255+0000 7f4487051700 1 -- 172.21.15.3:0/2969132342 >> 172.21.15.3:0/2969132342 conn(0x7f4480066f50 msgr2=0x7f448009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:47.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.255+0000 7f4487051700 1 -- 172.21.15.3:0/2969132342 shutdown_connections 2023-10-03T10:15:47.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.255+0000 7f4487051700 1 -- 172.21.15.3:0/2969132342 wait complete. 2023-10-03T10:15:47.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.257+0000 7f4487051700 1 Processor -- start 2023-10-03T10:15:47.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.257+0000 7f4487051700 1 -- start start 2023-10-03T10:15:47.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.258+0000 7f4487051700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f448009d6f0 0x7f4480127da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:47.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.258+0000 7f4487051700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f448009e1d0 0x7f44801282e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:47.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.258+0000 7f4487051700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4480139170 0x7f44801cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:47.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.258+0000 7f4487051700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448012aef0 con 0x7f448009d6f0 2023-10-03T10:15:47.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.258+0000 7f4487051700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4480138d30 con 0x7f4480139170 2023-10-03T10:15:47.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.258+0000 7f4487051700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f448009c710 con 0x7f448009e1d0 2023-10-03T10:15:47.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.258+0000 7f44855ee700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4480139170 0x7f44801cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:47.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.258+0000 7f44855ee700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4480139170 0x7f44801cc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34732/0 (socket says 172.21.15.3:34732) 2023-10-03T10:15:47.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.258+0000 7f44855ee700 1 -- 172.21.15.3:0/3971426823 learned_addr learned my addr 172.21.15.3:0/3971426823 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:47.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.258+0000 7f447ffff700 1 --2- 172.21.15.3:0/3971426823 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f448009e1d0 0x7f44801282e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:47.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.258+0000 7f4484ded700 1 --2- 172.21.15.3:0/3971426823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f448009d6f0 0x7f4480127da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:47.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.258+0000 7f44855ee700 1 -- 172.21.15.3:0/3971426823 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f448009e1d0 msgr2=0x7f44801282e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:47.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.258+0000 7f44855ee700 1 --2- 172.21.15.3:0/3971426823 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f448009e1d0 0x7f44801282e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.258+0000 7f44855ee700 1 -- 172.21.15.3:0/3971426823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f448009d6f0 msgr2=0x7f4480127da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:47.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.258+0000 7f44855ee700 1 --2- 172.21.15.3:0/3971426823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f448009d6f0 0x7f4480127da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.258+0000 7f44855ee700 1 -- 172.21.15.3:0/3971426823 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4478009b30 con 0x7f4480139170 2023-10-03T10:15:47.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.259+0000 7f44855ee700 1 --2- 172.21.15.3:0/3971426823 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4480139170 0x7f44801cc620 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f447800b680 tx=0x7f4478004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:47.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.259+0000 7f447d7fa700 1 -- 172.21.15.3:0/3971426823 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44780092e0 con 0x7f4480139170 2023-10-03T10:15:47.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.259+0000 7f447d7fa700 1 -- 172.21.15.3:0/3971426823 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4478025460 con 0x7f4480139170 2023-10-03T10:15:47.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.259+0000 7f447d7fa700 1 -- 172.21.15.3:0/3971426823 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4478025830 con 0x7f4480139170 2023-10-03T10:15:47.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.259+0000 7f4487051700 1 -- 172.21.15.3:0/3971426823 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4480128820 con 0x7f4480139170 2023-10-03T10:15:47.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.259+0000 7f4487051700 1 -- 172.21.15.3:0/3971426823 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4480072070 con 0x7f4480139170 2023-10-03T10:15:47.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.260+0000 7f447d7fa700 1 -- 172.21.15.3:0/3971426823 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4478025990 con 0x7f4480139170 2023-10-03T10:15:47.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.261+0000 7f447d7fa700 1 -- 172.21.15.3:0/3971426823 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4478041340 con 0x7f4480139170 2023-10-03T10:15:47.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.261+0000 7f446affd700 1 -- 172.21.15.3:0/3971426823 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4480074ea0 con 0x7f4480139170 2023-10-03T10:15:47.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.270+0000 7f447d7fa700 1 -- 172.21.15.3:0/3971426823 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4478023360 con 0x7f4480139170 2023-10-03T10:15:47.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.288+0000 7f4fd9962700 1 Processor -- start 2023-10-03T10:15:47.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.288+0000 7f4fd9962700 1 -- start start 2023-10-03T10:15:47.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.289+0000 7f4fd9962700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fd406a770 0x7f4fd409d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:47.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.289+0000 7f4fd9962700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4fd409dc30 0x7f4fd409e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:47.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.289+0000 7f4fd9962700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fd409a810 con 0x7f4fd413c1e0 2023-10-03T10:15:47.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.289+0000 7f4fd9962700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fd406ab40 con 0x7f4fd409dc30 2023-10-03T10:15:47.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.289+0000 7f4fd9962700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd409ebc0 con 0x7f4fd406a770 2023-10-03T10:15:47.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.289+0000 7f4fd2ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fd406a770 0x7f4fd409d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:47.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.289+0000 7f4fd2ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fd406a770 0x7f4fd409d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34748/0 (socket says 172.21.15.3:34748) 2023-10-03T10:15:47.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.289+0000 7f4fd2ffd700 1 -- 172.21.15.3:0/3206788081 learned_addr learned my addr 172.21.15.3:0/3206788081 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:47.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.289+0000 7f4fd27fc700 1 --2- 172.21.15.3:0/3206788081 >> v2:172.21.15.3:3300/0 conn(0x7f4fd409dc30 0x7f4fd409e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:47.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.289+0000 7f4fd2ffd700 1 -- 172.21.15.3:0/3206788081 >> v2:172.21.15.3:3300/0 conn(0x7f4fd409dc30 msgr2=0x7f4fd409e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:47.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.289+0000 7f4fd2ffd700 1 --2- 172.21.15.3:0/3206788081 >> v2:172.21.15.3:3300/0 conn(0x7f4fd409dc30 0x7f4fd409e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.289+0000 7f4fd2ffd700 1 -- 172.21.15.3:0/3206788081 >> v1:172.21.15.3:6789/0 conn(0x7f4fd413c1e0 legacy=0x7f4fd413e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:47.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.289+0000 7f4fd2ffd700 1 -- 172.21.15.3:0/3206788081 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd409e920 con 0x7f4fd406a770 2023-10-03T10:15:47.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.289+0000 7f4fd2ffd700 1 --2- 172.21.15.3:0/3206788081 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fd406a770 0x7f4fd409d6f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4009920 tx=0x7f4fc4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c35e843c5700463b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:47.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.290+0000 7f4fd1ffb700 1 -- 172.21.15.3:0/3206788081 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4011830 con 0x7f4fd406a770 2023-10-03T10:15:47.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.290+0000 7f4fd1ffb700 1 -- 172.21.15.3:0/3206788081 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc4011990 con 0x7f4fd406a770 2023-10-03T10:15:47.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.290+0000 7f4fd1ffb700 1 -- 172.21.15.3:0/3206788081 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4011d20 con 0x7f4fd406a770 2023-10-03T10:15:47.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.290+0000 7f4fd9962700 1 -- 172.21.15.3:0/3206788081 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fd406a770 msgr2=0x7f4fd409d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:47.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.290+0000 7f4fd9962700 1 --2- 172.21.15.3:0/3206788081 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fd406a770 0x7f4fd409d6f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4009920 tx=0x7f4fc4009c30 comp rx=0 tx=0).stop 2023-10-03T10:15:47.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.290+0000 7f4fd9962700 1 -- 172.21.15.3:0/3206788081 shutdown_connections 2023-10-03T10:15:47.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.290+0000 7f4fd9962700 1 --2- 172.21.15.3:0/3206788081 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fd406a770 0x7f4fd409d6f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.290+0000 7f4fd9962700 1 --2- 172.21.15.3:0/3206788081 >> v2:172.21.15.3:3300/0 conn(0x7f4fd409dc30 0x7f4fd409e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.290+0000 7f4fd9962700 1 -- 172.21.15.3:0/3206788081 >> 172.21.15.3:0/3206788081 conn(0x7f4fd4066f50 msgr2=0x7f4fd409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:47.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.290+0000 7f4fd9962700 1 -- 172.21.15.3:0/3206788081 shutdown_connections 2023-10-03T10:15:47.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.290+0000 7f4fd9962700 1 -- 172.21.15.3:0/3206788081 wait complete. 2023-10-03T10:15:47.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.291+0000 7f4fd9962700 1 Processor -- start 2023-10-03T10:15:47.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.292+0000 7f4fd9962700 1 -- start start 2023-10-03T10:15:47.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.292+0000 7f4fd9962700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fd406a770 0x7f4fd413e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:47.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.292+0000 7f4fd9962700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fd409dc30 0x7f4fd413a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:47.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.292+0000 7f4fd9962700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fd413c1e0 0x7f4fd413a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:47.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.292+0000 7f4fd9962700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd4134b90 con 0x7f4fd406a770 2023-10-03T10:15:47.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.292+0000 7f4fd9962700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd413bd40 con 0x7f4fd413c1e0 2023-10-03T10:15:47.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.292+0000 7f4fd9962700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd409f1a0 con 0x7f4fd409dc30 2023-10-03T10:15:47.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.292+0000 7f4fd2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fd406a770 0x7f4fd413e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:47.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.292+0000 7f4fd27fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fd409dc30 0x7f4fd413a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:47.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.292+0000 7f4fd2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fd406a770 0x7f4fd413e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55616/0 (socket says 172.21.15.3:55616) 2023-10-03T10:15:47.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.292+0000 7f4fd2ffd700 1 -- 172.21.15.3:0/2384468641 learned_addr learned my addr 172.21.15.3:0/2384468641 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:47.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.293+0000 7f4fd37fe700 1 --2- 172.21.15.3:0/2384468641 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fd413c1e0 0x7f4fd413a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:47.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.293+0000 7f4fd2ffd700 1 -- 172.21.15.3:0/2384468641 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fd409dc30 msgr2=0x7f4fd413a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:47.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.293+0000 7f4fd2ffd700 1 --2- 172.21.15.3:0/2384468641 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fd409dc30 0x7f4fd413a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.293+0000 7f4fd2ffd700 1 -- 172.21.15.3:0/2384468641 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fd413c1e0 msgr2=0x7f4fd413a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:47.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.293+0000 7f4fd2ffd700 1 --2- 172.21.15.3:0/2384468641 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fd413c1e0 0x7f4fd413a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.293+0000 7f4fd2ffd700 1 -- 172.21.15.3:0/2384468641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc4009550 con 0x7f4fd406a770 2023-10-03T10:15:47.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.293+0000 7f4fd27fc700 1 --2- 172.21.15.3:0/2384468641 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fd409dc30 0x7f4fd413a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:47.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.293+0000 7f4fd2ffd700 1 --2- 172.21.15.3:0/2384468641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fd406a770 0x7f4fd413e030 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4000c00 tx=0x7f4fc4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:47.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.293+0000 7f4fbbfff700 1 -- 172.21.15.3:0/2384468641 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4004450 con 0x7f4fd406a770 2023-10-03T10:15:47.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.293+0000 7f4fbbfff700 1 -- 172.21.15.3:0/2384468641 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc40045d0 con 0x7f4fd406a770 2023-10-03T10:15:47.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.293+0000 7f4fbbfff700 1 -- 172.21.15.3:0/2384468641 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc40038e0 con 0x7f4fd406a770 2023-10-03T10:15:47.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.293+0000 7f4fd9962700 1 -- 172.21.15.3:0/2384468641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd411dae0 con 0x7f4fd406a770 2023-10-03T10:15:47.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.294+0000 7f4fd9962700 1 -- 172.21.15.3:0/2384468641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd413fef0 con 0x7f4fd406a770 2023-10-03T10:15:47.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.295+0000 7f4fbbfff700 1 -- 172.21.15.3:0/2384468641 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4fc4003a40 con 0x7f4fd406a770 2023-10-03T10:15:47.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.295+0000 7f4fd9962700 1 -- 172.21.15.3:0/2384468641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd406b430 con 0x7f4fd406a770 2023-10-03T10:15:47.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.295+0000 7f4fbbfff700 1 -- 172.21.15.3:0/2384468641 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4fc4021030 con 0x7f4fd406a770 2023-10-03T10:15:47.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.304+0000 7f4fbbfff700 1 -- 172.21.15.3:0/2384468641 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4fc401a430 con 0x7f4fd406a770 2023-10-03T10:15:47.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.406+0000 7f3b79b1a700 1 -- 172.21.15.3:0/947130709 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3b7412f210 con 0x7f3b7409e000 2023-10-03T10:15:47.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.407+0000 7f3b5bfff700 1 -- 172.21.15.3:0/947130709 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3b68017020 con 0x7f3b7409e000 2023-10-03T10:15:47.407 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:47.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.410+0000 7f3b79b1a700 1 -- 172.21.15.3:0/947130709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b7409e000 msgr2=0x7f3b741306f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:47.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.410+0000 7f3b79b1a700 1 --2- 172.21.15.3:0/947130709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b7409e000 0x7f3b741306f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3b68000f60 tx=0x7f3b6800c730 comp rx=0 tx=0).stop 2023-10-03T10:15:47.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.410+0000 7f3b79b1a700 1 -- 172.21.15.3:0/947130709 shutdown_connections 2023-10-03T10:15:47.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.410+0000 7f3b79b1a700 1 --2- 172.21.15.3:0/947130709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b7409e410 0x7f3b74130c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.410+0000 7f3b79b1a700 1 --2- 172.21.15.3:0/947130709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b7409d6f0 0x7f3b741301b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.410+0000 7f3b79b1a700 1 --2- 172.21.15.3:0/947130709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b7409e000 0x7f3b741306f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.410+0000 7f3b79b1a700 1 -- 172.21.15.3:0/947130709 >> 172.21.15.3:0/947130709 conn(0x7f3b74066f50 msgr2=0x7f3b740a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:47.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.410+0000 7f3b79b1a700 1 -- 172.21.15.3:0/947130709 shutdown_connections 2023-10-03T10:15:47.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.410+0000 7f3b79b1a700 1 -- 172.21.15.3:0/947130709 wait complete. 2023-10-03T10:15:47.429 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:47.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.494+0000 7fb4c0633700 1 -- 172.21.15.3:0/1797159689 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb4b8074ea0 con 0x7fb4b8130540 2023-10-03T10:15:47.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.494+0000 7fb4aeffd700 1 -- 172.21.15.3:0/1797159689 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb4a8017020 con 0x7fb4b8130540 2023-10-03T10:15:47.494 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:47.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.496+0000 7fb4c0633700 1 -- 172.21.15.3:0/1797159689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4b8130540 msgr2=0x7fb4b8119210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:47.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.496+0000 7fb4c0633700 1 --2- 172.21.15.3:0/1797159689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4b8130540 0x7fb4b8119210 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800c4e0 tx=0x7fb4a800c8a0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.496+0000 7fb4c0633700 1 -- 172.21.15.3:0/1797159689 shutdown_connections 2023-10-03T10:15:47.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.496+0000 7fb4c0633700 1 --2- 172.21.15.3:0/1797159689 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4b8130e50 0x7fb4b8119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.496+0000 7fb4c0633700 1 --2- 172.21.15.3:0/1797159689 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb4b806fcf0 0x7fb4b8118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.496+0000 7fb4c0633700 1 --2- 172.21.15.3:0/1797159689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4b8130540 0x7fb4b8119210 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.496+0000 7fb4c0633700 1 -- 172.21.15.3:0/1797159689 >> 172.21.15.3:0/1797159689 conn(0x7fb4b80a2aa0 msgr2=0x7fb4b812d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:47.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.496+0000 7fb4c0633700 1 -- 172.21.15.3:0/1797159689 shutdown_connections 2023-10-03T10:15:47.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.496+0000 7fb4c0633700 1 -- 172.21.15.3:0/1797159689 wait complete. 2023-10-03T10:15:47.510 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:47.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.514+0000 7f4fd9962700 1 -- 172.21.15.3:0/2384468641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4fd411de70 con 0x7f4fd406a770 2023-10-03T10:15:47.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.514+0000 7f4fbbfff700 1 -- 172.21.15.3:0/2384468641 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4fc4015070 con 0x7f4fd406a770 2023-10-03T10:15:47.514 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:47.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.516+0000 7f4fd9962700 1 -- 172.21.15.3:0/2384468641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fd406a770 msgr2=0x7f4fd413e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:47.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.516+0000 7f4fd9962700 1 --2- 172.21.15.3:0/2384468641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fd406a770 0x7f4fd413e030 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4000c00 tx=0x7f4fc4004750 comp rx=0 tx=0).stop 2023-10-03T10:15:47.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.516+0000 7f4fd9962700 1 -- 172.21.15.3:0/2384468641 shutdown_connections 2023-10-03T10:15:47.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.516+0000 7f4fd9962700 1 --2- 172.21.15.3:0/2384468641 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fd413c1e0 0x7f4fd413a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.516+0000 7f4fd9962700 1 --2- 172.21.15.3:0/2384468641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fd406a770 0x7f4fd413e030 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.516+0000 7f4fd9962700 1 --2- 172.21.15.3:0/2384468641 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fd409dc30 0x7f4fd413a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.516+0000 7f4fd9962700 1 -- 172.21.15.3:0/2384468641 >> 172.21.15.3:0/2384468641 conn(0x7f4fd4066f50 msgr2=0x7f4fd4072240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:47.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.516+0000 7f4fd9962700 1 -- 172.21.15.3:0/2384468641 shutdown_connections 2023-10-03T10:15:47.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.516+0000 7f4fd9962700 1 -- 172.21.15.3:0/2384468641 wait complete. 2023-10-03T10:15:47.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.519+0000 7f446affd700 1 -- 172.21.15.3:0/3971426823 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4480071e50 con 0x7f4480139170 2023-10-03T10:15:47.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.519+0000 7f447d7fa700 1 -- 172.21.15.3:0/3971426823 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4478011b80 con 0x7f4480139170 2023-10-03T10:15:47.519 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.521+0000 7f446affd700 1 -- 172.21.15.3:0/3971426823 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4480139170 msgr2=0x7f44801cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.521+0000 7f446affd700 1 --2- 172.21.15.3:0/3971426823 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4480139170 0x7f44801cc620 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f447800b680 tx=0x7f4478004ad0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.521+0000 7f446affd700 1 -- 172.21.15.3:0/3971426823 shutdown_connections 2023-10-03T10:15:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.521+0000 7f446affd700 1 --2- 172.21.15.3:0/3971426823 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4480139170 0x7f44801cc620 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.521+0000 7f446affd700 1 --2- 172.21.15.3:0/3971426823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f448009d6f0 0x7f4480127da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.521+0000 7f446affd700 1 --2- 172.21.15.3:0/3971426823 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f448009e1d0 0x7f44801282e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.521+0000 7f446affd700 1 -- 172.21.15.3:0/3971426823 >> 172.21.15.3:0/3971426823 conn(0x7f4480066f50 msgr2=0x7f4480070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.522+0000 7f446affd700 1 -- 172.21.15.3:0/3971426823 shutdown_connections 2023-10-03T10:15:47.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:47.522+0000 7f446affd700 1 -- 172.21.15.3:0/3971426823 wait complete. 2023-10-03T10:15:47.531 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:47.536 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:48.430 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:48.511 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:48.533 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:48.537 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:48.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.585+0000 7fe83ba0b700 1 Processor -- start 2023-10-03T10:15:48.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.586+0000 7fe83ba0b700 1 -- start start 2023-10-03T10:15:48.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.586+0000 7fe83ba0b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe83409e930 0x7fe83409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:48.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.586+0000 7fe83ba0b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe83409f650 0x7fe834131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:48.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.586+0000 7fe83ba0b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe834070940 con 0x7fe83409f240 2023-10-03T10:15:48.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.586+0000 7fe83ba0b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83406fd60 con 0x7fe83409e930 2023-10-03T10:15:48.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.586+0000 7fe83ba0b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe83406fa80 con 0x7fe83409f650 2023-10-03T10:15:48.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.586+0000 7fe8397a7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe83409e930 0x7fe83409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:48.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.586+0000 7fe8397a7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe83409e930 0x7fe83409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34766/0 (socket says 172.21.15.3:34766) 2023-10-03T10:15:48.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.586+0000 7fe8397a7700 1 -- 172.21.15.3:0/1950226254 learned_addr learned my addr 172.21.15.3:0/1950226254 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:48.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.586+0000 7fe838fa6700 1 --1- 172.21.15.3:0/1950226254 >> v1:172.21.15.3:6789/0 conn(0x7fe83409f240 0x7fe8341317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57280/0 (socket says 172.21.15.3:57280) 2023-10-03T10:15:48.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.587+0000 7fe8397a7700 1 -- 172.21.15.3:0/1950226254 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe83409f650 msgr2=0x7fe834131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:48.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.587+0000 7fe82bfff700 1 -- 172.21.15.3:0/1950226254 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2021088477 0 0) 0x7fe834070940 con 0x7fe83409f240 2023-10-03T10:15:48.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.587+0000 7fe8397a7700 1 --2- 172.21.15.3:0/1950226254 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe83409f650 0x7fe834131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.587+0000 7fe8397a7700 1 -- 172.21.15.3:0/1950226254 >> v1:172.21.15.3:6789/0 conn(0x7fe83409f240 legacy=0x7fe8341317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:48.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.587+0000 7fe8397a7700 1 -- 172.21.15.3:0/1950226254 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83409fb60 con 0x7fe83409e930 2023-10-03T10:15:48.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.587+0000 7fe8397a7700 1 --2- 172.21.15.3:0/1950226254 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe83409e930 0x7fe83409ed00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe830009920 tx=0x7fe830009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ac248d8c7718dd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:48.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.587+0000 7fe82bfff700 1 -- 172.21.15.3:0/1950226254 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83000ba60 con 0x7fe83409e930 2023-10-03T10:15:48.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.587+0000 7fe82bfff700 1 -- 172.21.15.3:0/1950226254 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83000bbc0 con 0x7fe83409e930 2023-10-03T10:15:48.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.587+0000 7fe82bfff700 1 -- 172.21.15.3:0/1950226254 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe830009e70 con 0x7fe83409e930 2023-10-03T10:15:48.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.588+0000 7fe83ba0b700 1 -- 172.21.15.3:0/1950226254 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe83409e930 msgr2=0x7fe83409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:48.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.588+0000 7fe83ba0b700 1 --2- 172.21.15.3:0/1950226254 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe83409e930 0x7fe83409ed00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe830009920 tx=0x7fe830009c30 comp rx=0 tx=0).stop 2023-10-03T10:15:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.588+0000 7fe83ba0b700 1 -- 172.21.15.3:0/1950226254 shutdown_connections 2023-10-03T10:15:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.588+0000 7fe83ba0b700 1 --2- 172.21.15.3:0/1950226254 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe83409e930 0x7fe83409ed00 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.588+0000 7fe83ba0b700 1 --2- 172.21.15.3:0/1950226254 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe83409f650 0x7fe834131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.588+0000 7fe83ba0b700 1 -- 172.21.15.3:0/1950226254 >> 172.21.15.3:0/1950226254 conn(0x7fe8340a4210 msgr2=0x7fe8340a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.589+0000 7fe83ba0b700 1 -- 172.21.15.3:0/1950226254 shutdown_connections 2023-10-03T10:15:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.589+0000 7fe83ba0b700 1 -- 172.21.15.3:0/1950226254 wait complete. 2023-10-03T10:15:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.590+0000 7fe83ba0b700 1 Processor -- start 2023-10-03T10:15:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.590+0000 7fe83ba0b700 1 -- start start 2023-10-03T10:15:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.591+0000 7fe83ba0b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe83409e930 0x7fe8341a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.591+0000 7fe83ba0b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe83409f240 0x7fe8341a41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.591+0000 7fe83ba0b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe83409f650 0x7fe8341a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.591+0000 7fe83ba0b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8341a6f40 con 0x7fe83409f650 2023-10-03T10:15:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.591+0000 7fe83ba0b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe834071d20 con 0x7fe83409f240 2023-10-03T10:15:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.591+0000 7fe83ba0b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe8340a34a0 con 0x7fe83409e930 2023-10-03T10:15:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.591+0000 7fe839fa8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe83409f650 0x7fe8341a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.591+0000 7fe8397a7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe83409e930 0x7fe8341a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.591+0000 7fe839fa8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe83409f650 0x7fe8341a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55624/0 (socket says 172.21.15.3:55624) 2023-10-03T10:15:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.591+0000 7fe839fa8700 1 -- 172.21.15.3:0/1444833183 learned_addr learned my addr 172.21.15.3:0/1444833183 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.592+0000 7fe838fa6700 1 --2- 172.21.15.3:0/1444833183 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe83409f240 0x7fe8341a41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:48.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.592+0000 7fe839fa8700 1 -- 172.21.15.3:0/1444833183 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe83409e930 msgr2=0x7fe8341a3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:48.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.592+0000 7fe839fa8700 1 --2- 172.21.15.3:0/1444833183 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe83409e930 0x7fe8341a3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.592+0000 7fe839fa8700 1 -- 172.21.15.3:0/1444833183 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe83409f240 msgr2=0x7fe8341a41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:48.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.592+0000 7fe839fa8700 1 --2- 172.21.15.3:0/1444833183 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe83409f240 0x7fe8341a41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.592+0000 7fe839fa8700 1 -- 172.21.15.3:0/1444833183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe830009550 con 0x7fe83409f650 2023-10-03T10:15:48.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.592+0000 7fe838fa6700 1 --2- 172.21.15.3:0/1444833183 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe83409f240 0x7fe8341a41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:15:48.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.592+0000 7fe839fa8700 1 --2- 172.21.15.3:0/1444833183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe83409f650 0x7fe8341a4710 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00b9e0 tx=0x7fe82c00bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:48.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.592+0000 7fe829ffb700 1 -- 172.21.15.3:0/1444833183 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82c00c930 con 0x7fe83409f650 2023-10-03T10:15:48.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.592+0000 7fe829ffb700 1 -- 172.21.15.3:0/1444833183 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82c00cad0 con 0x7fe83409f650 2023-10-03T10:15:48.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.592+0000 7fe829ffb700 1 -- 172.21.15.3:0/1444833183 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82c00ce80 con 0x7fe83409f650 2023-10-03T10:15:48.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.592+0000 7fe83ba0b700 1 -- 172.21.15.3:0/1444833183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8340a0190 con 0x7fe83409f650 2023-10-03T10:15:48.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.593+0000 7fe83ba0b700 1 -- 172.21.15.3:0/1444833183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8341a6af0 con 0x7fe83409f650 2023-10-03T10:15:48.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.594+0000 7fe829ffb700 1 -- 172.21.15.3:0/1444833183 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe82c014440 con 0x7fe83409f650 2023-10-03T10:15:48.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.594+0000 7fe829ffb700 1 -- 172.21.15.3:0/1444833183 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe82c006e40 con 0x7fe83409f650 2023-10-03T10:15:48.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.594+0000 7fe83ba0b700 1 -- 172.21.15.3:0/1444833183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8341a0b30 con 0x7fe83409f650 2023-10-03T10:15:48.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.605+0000 7fe829ffb700 1 -- 172.21.15.3:0/1444833183 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe82c019080 con 0x7fe83409f650 2023-10-03T10:15:48.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.663+0000 7f421280b700 1 Processor -- start 2023-10-03T10:15:48.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.663+0000 7f421280b700 1 -- start start 2023-10-03T10:15:48.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.663+0000 7f421280b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f420c09d6f0 0x7f420c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:48.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.663+0000 7f421280b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f420c09e000 0x7f420c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:48.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.664+0000 7f421280b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f420c09f130 con 0x7f420c09e410 2023-10-03T10:15:48.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.664+0000 7f421280b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f420c09f520 con 0x7f420c09d6f0 2023-10-03T10:15:48.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.664+0000 7f421280b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f420c09e930 con 0x7f420c09e000 2023-10-03T10:15:48.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.664+0000 7f420bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f420c09d6f0 0x7f420c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:48.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.664+0000 7f4210da8700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f420c09e410 0x7f420c130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57292/0 (socket says 172.21.15.3:57292) 2023-10-03T10:15:48.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.664+0000 7f4210da8700 1 -- 172.21.15.3:0/591826312 learned_addr learned my addr 172.21.15.3:0/591826312 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:48.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.664+0000 7f420b7fe700 1 --2- 172.21.15.3:0/591826312 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f420c09e000 0x7f420c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:48.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.664+0000 7f420affd700 1 -- 172.21.15.3:0/591826312 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2253499163 0 0) 0x7f420c09f130 con 0x7f420c09e410 2023-10-03T10:15:48.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.664+0000 7f420affd700 1 -- 172.21.15.3:0/591826312 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41f40035b0 con 0x7f420c09e410 2023-10-03T10:15:48.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.664+0000 7f420bfff700 1 -- 172.21.15.3:0/591826312 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f420c09e000 msgr2=0x7f420c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:48.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.664+0000 7f420bfff700 1 --2- 172.21.15.3:0/591826312 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f420c09e000 0x7f420c130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.664+0000 7f420bfff700 1 -- 172.21.15.3:0/591826312 >> v1:172.21.15.3:6789/0 conn(0x7f420c09e410 legacy=0x7f420c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:48.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.664+0000 7f420bfff700 1 -- 172.21.15.3:0/591826312 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420c09b2d0 con 0x7f420c09d6f0 2023-10-03T10:15:48.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.665+0000 7f420bfff700 1 --2- 172.21.15.3:0/591826312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f420c09d6f0 0x7f420c09dac0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009840 tx=0x7f41fc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a559e048c9ca986 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:48.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.665+0000 7f420affd700 1 -- 172.21.15.3:0/591826312 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc00f8a0 con 0x7f420c09d6f0 2023-10-03T10:15:48.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.665+0000 7f420affd700 1 -- 172.21.15.3:0/591826312 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41fc00fa00 con 0x7f420c09d6f0 2023-10-03T10:15:48.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.665+0000 7f420affd700 1 -- 172.21.15.3:0/591826312 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc00fd90 con 0x7f420c09d6f0 2023-10-03T10:15:48.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.665+0000 7f421280b700 1 -- 172.21.15.3:0/591826312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f420c09d6f0 msgr2=0x7f420c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:48.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.665+0000 7f421280b700 1 --2- 172.21.15.3:0/591826312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f420c09d6f0 0x7f420c09dac0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009840 tx=0x7f41fc009b50 comp rx=0 tx=0).stop 2023-10-03T10:15:48.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.666+0000 7f421280b700 1 -- 172.21.15.3:0/591826312 shutdown_connections 2023-10-03T10:15:48.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.666+0000 7f421280b700 1 --2- 172.21.15.3:0/591826312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f420c09d6f0 0x7f420c09dac0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.666+0000 7f421280b700 1 --2- 172.21.15.3:0/591826312 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f420c09e000 0x7f420c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.666+0000 7f421280b700 1 -- 172.21.15.3:0/591826312 >> 172.21.15.3:0/591826312 conn(0x7f420c066f50 msgr2=0x7f420c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:48.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.666+0000 7f421280b700 1 -- 172.21.15.3:0/591826312 shutdown_connections 2023-10-03T10:15:48.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.666+0000 7f421280b700 1 -- 172.21.15.3:0/591826312 wait complete. 2023-10-03T10:15:48.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.667+0000 7f421280b700 1 Processor -- start 2023-10-03T10:15:48.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.667+0000 7f421280b700 1 -- start start 2023-10-03T10:15:48.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.667+0000 7f421280b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f420c09d6f0 0x7f420c1221b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:48.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.667+0000 7f421280b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f420c09e000 0x7f420c1226f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:48.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.668+0000 7f421280b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f420c09e410 0x7f420c122c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:48.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.668+0000 7f421280b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420c124310 con 0x7f420c09e000 2023-10-03T10:15:48.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.668+0000 7f421280b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f420c0a2000 con 0x7f420c09e410 2023-10-03T10:15:48.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.668+0000 7f421280b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f420c09c700 con 0x7f420c09d6f0 2023-10-03T10:15:48.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.668+0000 7f4210da8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f420c09e410 0x7f420c122c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:48.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.668+0000 7f4210da8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f420c09e410 0x7f420c122c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34802/0 (socket says 172.21.15.3:34802) 2023-10-03T10:15:48.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.668+0000 7f4210da8700 1 -- 172.21.15.3:0/3401149709 learned_addr learned my addr 172.21.15.3:0/3401149709 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:48.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.668+0000 7f420bfff700 1 --2- 172.21.15.3:0/3401149709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f420c09d6f0 0x7f420c1221b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:48.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.668+0000 7f420b7fe700 1 --2- 172.21.15.3:0/3401149709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f420c09e000 0x7f420c1226f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:48.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.668+0000 7f4210da8700 1 -- 172.21.15.3:0/3401149709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f420c09d6f0 msgr2=0x7f420c1221b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:48.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.668+0000 7f4210da8700 1 --2- 172.21.15.3:0/3401149709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f420c09d6f0 0x7f420c1221b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.668+0000 7f4210da8700 1 -- 172.21.15.3:0/3401149709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f420c09e000 msgr2=0x7f420c1226f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:48.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.668+0000 7f4210da8700 1 --2- 172.21.15.3:0/3401149709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f420c09e000 0x7f420c1226f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.668+0000 7f4210da8700 1 -- 172.21.15.3:0/3401149709 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41fc0094a0 con 0x7f420c09e410 2023-10-03T10:15:48.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.668+0000 7f4210da8700 1 --2- 172.21.15.3:0/3401149709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f420c09e410 0x7f420c122c30 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f420400cb40 tx=0x7f420400ce50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:48.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.669+0000 7f4208ff9700 1 -- 172.21.15.3:0/3401149709 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420400a9e0 con 0x7f420c09e410 2023-10-03T10:15:48.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.669+0000 7f4208ff9700 1 -- 172.21.15.3:0/3401149709 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420400ab40 con 0x7f420c09e410 2023-10-03T10:15:48.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.669+0000 7f421280b700 1 -- 172.21.15.3:0/3401149709 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f420c125410 con 0x7f420c09e410 2023-10-03T10:15:48.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.669+0000 7f4208ff9700 1 -- 172.21.15.3:0/3401149709 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4204012400 con 0x7f420c09e410 2023-10-03T10:15:48.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.669+0000 7f421280b700 1 -- 172.21.15.3:0/3401149709 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f420c124560 con 0x7f420c09e410 2023-10-03T10:15:48.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.670+0000 7f421280b700 1 -- 172.21.15.3:0/3401149709 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f420c074ea0 con 0x7f420c09e410 2023-10-03T10:15:48.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.670+0000 7f4208ff9700 1 -- 172.21.15.3:0/3401149709 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f420401b3b0 con 0x7f420c09e410 2023-10-03T10:15:48.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.670+0000 7f4208ff9700 1 -- 172.21.15.3:0/3401149709 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f420403f2b0 con 0x7f420c09e410 2023-10-03T10:15:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.679+0000 7f4208ff9700 1 -- 172.21.15.3:0/3401149709 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f42040192c0 con 0x7f420c09e410 2023-10-03T10:15:48.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.705+0000 7fb2c6b4a700 1 Processor -- start 2023-10-03T10:15:48.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.705+0000 7fb2c6b4a700 1 -- start start 2023-10-03T10:15:48.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.706+0000 7fb2c6b4a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb2c0134630 0x7fb2c0136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:48.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.706+0000 7fb2c6b4a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb2c0136f50 0x7fb2c0139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:48.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.706+0000 7fb2c6b4a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2c009b4f0 con 0x7fb2c0131810 2023-10-03T10:15:48.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.706+0000 7fb2c6b4a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c01313c0 con 0x7fb2c0134630 2023-10-03T10:15:48.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.706+0000 7fb2c6b4a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c006e8c0 con 0x7fb2c0136f50 2023-10-03T10:15:48.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.706+0000 7fb2bffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb2c0134630 0x7fb2c0136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:48.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.706+0000 7fb2bffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb2c0134630 0x7fb2c0136a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34818/0 (socket says 172.21.15.3:34818) 2023-10-03T10:15:48.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.706+0000 7fb2bffff700 1 -- 172.21.15.3:0/461434246 learned_addr learned my addr 172.21.15.3:0/461434246 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:48.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.706+0000 7fb2c50e7700 1 --2- 172.21.15.3:0/461434246 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb2c0136f50 0x7fb2c0139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:48.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.706+0000 7fb2bffff700 1 -- 172.21.15.3:0/461434246 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb2c0136f50 msgr2=0x7fb2c0139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:48.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.706+0000 7fb2bffff700 1 --2- 172.21.15.3:0/461434246 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb2c0136f50 0x7fb2c0139330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.706+0000 7fb2bffff700 1 -- 172.21.15.3:0/461434246 >> v1:172.21.15.3:6789/0 conn(0x7fb2c0131810 legacy=0x7fb2c0131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:48.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.706+0000 7fb2bffff700 1 -- 172.21.15.3:0/461434246 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c0071d20 con 0x7fb2c0134630 2023-10-03T10:15:48.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.707+0000 7fb2bffff700 1 --2- 172.21.15.3:0/461434246 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb2c0134630 0x7fb2c0136a10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac009920 tx=0x7fb2ac009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f3b01a1b8e6cdb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:48.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.707+0000 7fb2bf7fe700 1 -- 172.21.15.3:0/461434246 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac004230 con 0x7fb2c0134630 2023-10-03T10:15:48.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.707+0000 7fb2bf7fe700 1 -- 172.21.15.3:0/461434246 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ac004390 con 0x7fb2c0134630 2023-10-03T10:15:48.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.707+0000 7fb2bf7fe700 1 -- 172.21.15.3:0/461434246 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac0036a0 con 0x7fb2c0134630 2023-10-03T10:15:48.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.707+0000 7fb2c6b4a700 1 -- 172.21.15.3:0/461434246 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb2c0134630 msgr2=0x7fb2c0136a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:48.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.707+0000 7fb2c6b4a700 1 --2- 172.21.15.3:0/461434246 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb2c0134630 0x7fb2c0136a10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac009920 tx=0x7fb2ac009c30 comp rx=0 tx=0).stop 2023-10-03T10:15:48.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.708+0000 7fb2c6b4a700 1 -- 172.21.15.3:0/461434246 shutdown_connections 2023-10-03T10:15:48.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.708+0000 7fb2c6b4a700 1 --2- 172.21.15.3:0/461434246 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb2c0136f50 0x7fb2c0139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.708+0000 7fb2c6b4a700 1 --2- 172.21.15.3:0/461434246 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb2c0134630 0x7fb2c0136a10 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.708+0000 7fb2c6b4a700 1 -- 172.21.15.3:0/461434246 >> 172.21.15.3:0/461434246 conn(0x7fb2c00a2aa0 msgr2=0x7fb2c00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:48.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.708+0000 7fb2c6b4a700 1 -- 172.21.15.3:0/461434246 shutdown_connections 2023-10-03T10:15:48.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.708+0000 7fb2c6b4a700 1 -- 172.21.15.3:0/461434246 wait complete. 2023-10-03T10:15:48.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.709+0000 7fb2c6b4a700 1 Processor -- start 2023-10-03T10:15:48.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.709+0000 7fb2c6b4a700 1 -- start start 2023-10-03T10:15:48.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.710+0000 7fb2c6b4a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb2c0131810 0x7fb2c01c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:48.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.710+0000 7fb2c6b4a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb2c0134630 0x7fb2c01c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:48.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.710+0000 7fb2c6b4a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2c0136f50 0x7fb2c01c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:48.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.710+0000 7fb2c6b4a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c011dd50 con 0x7fb2c0136f50 2023-10-03T10:15:48.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.710+0000 7fb2c6b4a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c009e5b0 con 0x7fb2c0131810 2023-10-03T10:15:48.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.710+0000 7fb2c6b4a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c011d3b0 con 0x7fb2c0134630 2023-10-03T10:15:48.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.710+0000 7fb2c48e6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb2c0131810 0x7fb2c01c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:48.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.710+0000 7fb2c48e6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb2c0131810 0x7fb2c01c8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34820/0 (socket says 172.21.15.3:34820) 2023-10-03T10:15:48.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.710+0000 7fb2c48e6700 1 -- 172.21.15.3:0/1260616485 learned_addr learned my addr 172.21.15.3:0/1260616485 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:48.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.710+0000 7fb2bffff700 1 --2- 172.21.15.3:0/1260616485 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb2c0134630 0x7fb2c01c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:48.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.710+0000 7fb2bffff700 1 -- 172.21.15.3:0/1260616485 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb2c0131810 msgr2=0x7fb2c01c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:48.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.710+0000 7fb2bffff700 1 --2- 172.21.15.3:0/1260616485 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb2c0131810 0x7fb2c01c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.710+0000 7fb2bffff700 1 -- 172.21.15.3:0/1260616485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2c0136f50 msgr2=0x7fb2c01c8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:48.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.710+0000 7fb2bffff700 1 --2- 172.21.15.3:0/1260616485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2c0136f50 0x7fb2c01c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.710+0000 7fb2bffff700 1 -- 172.21.15.3:0/1260616485 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ac009550 con 0x7fb2c0134630 2023-10-03T10:15:48.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.710+0000 7fb2c48e6700 1 --2- 172.21.15.3:0/1260616485 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb2c0131810 0x7fb2c01c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:48.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.711+0000 7fb2bffff700 1 --2- 172.21.15.3:0/1260616485 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb2c0134630 0x7fb2c01c89b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb2c006dd30 tx=0x7fb2ac004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:48.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.711+0000 7fb2bd7fa700 1 -- 172.21.15.3:0/1260616485 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac003da0 con 0x7fb2c0134630 2023-10-03T10:15:48.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.711+0000 7fb2bd7fa700 1 -- 172.21.15.3:0/1260616485 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ac011630 con 0x7fb2c0134630 2023-10-03T10:15:48.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.711+0000 7fb2c6b4a700 1 -- 172.21.15.3:0/1260616485 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c011ece0 con 0x7fb2c0134630 2023-10-03T10:15:48.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.711+0000 7fb2bd7fa700 1 -- 172.21.15.3:0/1260616485 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac0119e0 con 0x7fb2c0134630 2023-10-03T10:15:48.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.711+0000 7fb2c6b4a700 1 -- 172.21.15.3:0/1260616485 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c011ea00 con 0x7fb2c0134630 2023-10-03T10:15:48.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.712+0000 7fb2bd7fa700 1 -- 172.21.15.3:0/1260616485 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb2ac021070 con 0x7fb2c0134630 2023-10-03T10:15:48.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.712+0000 7fb2bd7fa700 1 -- 172.21.15.3:0/1260616485 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb2ac040bc0 con 0x7fb2c0134630 2023-10-03T10:15:48.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.712+0000 7fb2c6b4a700 1 -- 172.21.15.3:0/1260616485 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c0074ea0 con 0x7fb2c0134630 2023-10-03T10:15:48.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.717+0000 7fb2bd7fa700 1 -- 172.21.15.3:0/1260616485 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb2ac00b3e0 con 0x7fb2c0134630 2023-10-03T10:15:48.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.726+0000 7fa5ab0e7700 1 Processor -- start 2023-10-03T10:15:48.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.726+0000 7fa5ab0e7700 1 -- start start 2023-10-03T10:15:48.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.726+0000 7fa5ab0e7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5a412cf60 0x7fa5a406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:48.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.727+0000 7fa5ab0e7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa5a4130430 0x7fa5a4137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:48.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.727+0000 7fa5ab0e7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5a412cd90 con 0x7fa5a412f8b0 2023-10-03T10:15:48.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.727+0000 7fa5ab0e7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5a412d330 con 0x7fa5a4130430 2023-10-03T10:15:48.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.727+0000 7fa5ab0e7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a4071780 con 0x7fa5a412cf60 2023-10-03T10:15:48.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.727+0000 7fa5a8e83700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5a412cf60 0x7fa5a406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:48.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.727+0000 7fa5a8e83700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5a412cf60 0x7fa5a406f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34826/0 (socket says 172.21.15.3:34826) 2023-10-03T10:15:48.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.727+0000 7fa5a8e83700 1 -- 172.21.15.3:0/2913469715 learned_addr learned my addr 172.21.15.3:0/2913469715 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:48.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.727+0000 7fa5a9684700 1 --2- 172.21.15.3:0/2913469715 >> v2:172.21.15.3:3300/0 conn(0x7fa5a4130430 0x7fa5a4137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:48.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.727+0000 7fa5a8e83700 1 -- 172.21.15.3:0/2913469715 >> v2:172.21.15.3:3300/0 conn(0x7fa5a4130430 msgr2=0x7fa5a4137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:48.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.727+0000 7fa5a8e83700 1 --2- 172.21.15.3:0/2913469715 >> v2:172.21.15.3:3300/0 conn(0x7fa5a4130430 0x7fa5a4137c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.727+0000 7fa5a8e83700 1 -- 172.21.15.3:0/2913469715 >> v1:172.21.15.3:6789/0 conn(0x7fa5a412f8b0 legacy=0x7fa5a412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:48.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.727+0000 7fa5a8e83700 1 -- 172.21.15.3:0/2913469715 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a4071a40 con 0x7fa5a412cf60 2023-10-03T10:15:48.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.727+0000 7fa5a8e83700 1 --2- 172.21.15.3:0/2913469715 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5a412cf60 0x7fa5a406f5d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa5980131e0 tx=0x7fa5980134f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3d607a061c7c97e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:48.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.728+0000 7fa5a37fe700 1 -- 172.21.15.3:0/2913469715 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa598016070 con 0x7fa5a412cf60 2023-10-03T10:15:48.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.728+0000 7fa5a37fe700 1 -- 172.21.15.3:0/2913469715 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa598013e30 con 0x7fa5a412cf60 2023-10-03T10:15:48.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.728+0000 7fa5a37fe700 1 -- 172.21.15.3:0/2913469715 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5980112f0 con 0x7fa5a412cf60 2023-10-03T10:15:48.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.728+0000 7fa5ab0e7700 1 -- 172.21.15.3:0/2913469715 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5a412cf60 msgr2=0x7fa5a406f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:48.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.728+0000 7fa5ab0e7700 1 --2- 172.21.15.3:0/2913469715 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5a412cf60 0x7fa5a406f5d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa5980131e0 tx=0x7fa5980134f0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.728+0000 7fa5ab0e7700 1 -- 172.21.15.3:0/2913469715 shutdown_connections 2023-10-03T10:15:48.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.728+0000 7fa5ab0e7700 1 --2- 172.21.15.3:0/2913469715 >> v2:172.21.15.3:3300/0 conn(0x7fa5a4130430 0x7fa5a4137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.728+0000 7fa5ab0e7700 1 --2- 172.21.15.3:0/2913469715 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5a412cf60 0x7fa5a406f5d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.728+0000 7fa5ab0e7700 1 -- 172.21.15.3:0/2913469715 >> 172.21.15.3:0/2913469715 conn(0x7fa5a40a2aa0 msgr2=0x7fa5a40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:48.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.728+0000 7fa5ab0e7700 1 -- 172.21.15.3:0/2913469715 shutdown_connections 2023-10-03T10:15:48.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.728+0000 7fa5ab0e7700 1 -- 172.21.15.3:0/2913469715 wait complete. 2023-10-03T10:15:48.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.729+0000 7fa5ab0e7700 1 Processor -- start 2023-10-03T10:15:48.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.729+0000 7fa5ab0e7700 1 -- start start 2023-10-03T10:15:48.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.730+0000 7fa5ab0e7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5a412cf60 0x7fa5a409bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:48.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.730+0000 7fa5ab0e7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa5a412f8b0 0x7fa5a409c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:48.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.730+0000 7fa5ab0e7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5a4130430 0x7fa5a41c4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:48.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.730+0000 7fa5ab0e7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a409b7b0 con 0x7fa5a412cf60 2023-10-03T10:15:48.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.730+0000 7fa5ab0e7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a409e780 con 0x7fa5a4130430 2023-10-03T10:15:48.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.730+0000 7fa5ab0e7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a409c660 con 0x7fa5a412f8b0 2023-10-03T10:15:48.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.730+0000 7fa5a9684700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5a4130430 0x7fa5a41c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:48.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.730+0000 7fa5a9684700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5a4130430 0x7fa5a41c4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34836/0 (socket says 172.21.15.3:34836) 2023-10-03T10:15:48.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.730+0000 7fa5a9684700 1 -- 172.21.15.3:0/2905677314 learned_addr learned my addr 172.21.15.3:0/2905677314 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:48.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.730+0000 7fa5a8e83700 1 --2- 172.21.15.3:0/2905677314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5a412cf60 0x7fa5a409bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:48.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.731+0000 7fa5a3fff700 1 --2- 172.21.15.3:0/2905677314 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa5a412f8b0 0x7fa5a409c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:48.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.731+0000 7fa5a8e83700 1 -- 172.21.15.3:0/2905677314 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa5a412f8b0 msgr2=0x7fa5a409c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:48.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.731+0000 7fa5a8e83700 1 --2- 172.21.15.3:0/2905677314 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa5a412f8b0 0x7fa5a409c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.731+0000 7fa5a8e83700 1 -- 172.21.15.3:0/2905677314 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5a4130430 msgr2=0x7fa5a41c4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:48.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.731+0000 7fa5a8e83700 1 --2- 172.21.15.3:0/2905677314 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5a4130430 0x7fa5a41c4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.731+0000 7fa5a8e83700 1 -- 172.21.15.3:0/2905677314 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa598011df0 con 0x7fa5a412cf60 2023-10-03T10:15:48.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.731+0000 7fa5a9684700 1 --2- 172.21.15.3:0/2905677314 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5a4130430 0x7fa5a41c4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:48.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.731+0000 7fa5a8e83700 1 --2- 172.21.15.3:0/2905677314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5a412cf60 0x7fa5a409bbe0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fa59801d000 tx=0x7fa598004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:48.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.732+0000 7fa5a17fa700 1 -- 172.21.15.3:0/2905677314 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa598004050 con 0x7fa5a412cf60 2023-10-03T10:15:48.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.732+0000 7fa5a17fa700 1 -- 172.21.15.3:0/2905677314 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5980041d0 con 0x7fa5a412cf60 2023-10-03T10:15:48.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.732+0000 7fa5ab0e7700 1 -- 172.21.15.3:0/2905677314 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a409d670 con 0x7fa5a412cf60 2023-10-03T10:15:48.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.732+0000 7fa5a17fa700 1 -- 172.21.15.3:0/2905677314 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa598004580 con 0x7fa5a412cf60 2023-10-03T10:15:48.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.732+0000 7fa5ab0e7700 1 -- 172.21.15.3:0/2905677314 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a4129fa0 con 0x7fa5a412cf60 2023-10-03T10:15:48.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.733+0000 7fa5ab0e7700 1 -- 172.21.15.3:0/2905677314 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a409e910 con 0x7fa5a412cf60 2023-10-03T10:15:48.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.734+0000 7fa5a17fa700 1 -- 172.21.15.3:0/2905677314 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa5980036a0 con 0x7fa5a412cf60 2023-10-03T10:15:48.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.734+0000 7fa5a17fa700 1 -- 172.21.15.3:0/2905677314 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa5980491d0 con 0x7fa5a412cf60 2023-10-03T10:15:48.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.743+0000 7fa5a17fa700 1 -- 172.21.15.3:0/2905677314 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa598019750 con 0x7fa5a412cf60 2023-10-03T10:15:48.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.843+0000 7fe83ba0b700 1 -- 172.21.15.3:0/1444833183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe8341a7350 con 0x7fe83409f650 2023-10-03T10:15:48.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.843+0000 7fe829ffb700 1 -- 172.21.15.3:0/1444833183 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe82c0133a0 con 0x7fe83409f650 2023-10-03T10:15:48.843 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:48.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.845+0000 7fe83ba0b700 1 -- 172.21.15.3:0/1444833183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe83409f650 msgr2=0x7fe8341a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:48.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.845+0000 7fe83ba0b700 1 --2- 172.21.15.3:0/1444833183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe83409f650 0x7fe8341a4710 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00b9e0 tx=0x7fe82c00bcf0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.845+0000 7fe83ba0b700 1 -- 172.21.15.3:0/1444833183 shutdown_connections 2023-10-03T10:15:48.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.845+0000 7fe83ba0b700 1 --2- 172.21.15.3:0/1444833183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe83409f650 0x7fe8341a4710 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.845+0000 7fe83ba0b700 1 --2- 172.21.15.3:0/1444833183 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe83409e930 0x7fe8341a3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.845+0000 7fe83ba0b700 1 --2- 172.21.15.3:0/1444833183 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe83409f240 0x7fe8341a41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.845+0000 7fe83ba0b700 1 -- 172.21.15.3:0/1444833183 >> 172.21.15.3:0/1444833183 conn(0x7fe8340a4210 msgr2=0x7fe8341b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:48.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.846+0000 7fe83ba0b700 1 -- 172.21.15.3:0/1444833183 shutdown_connections 2023-10-03T10:15:48.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.846+0000 7fe83ba0b700 1 -- 172.21.15.3:0/1444833183 wait complete. 2023-10-03T10:15:48.860 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:48.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.895+0000 7f421280b700 1 -- 172.21.15.3:0/3401149709 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f420c06c360 con 0x7f420c09e410 2023-10-03T10:15:48.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.895+0000 7f4208ff9700 1 -- 172.21.15.3:0/3401149709 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4204017070 con 0x7f420c09e410 2023-10-03T10:15:48.896 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:48.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.898+0000 7f421280b700 1 -- 172.21.15.3:0/3401149709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f420c09e410 msgr2=0x7f420c122c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:48.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.898+0000 7f421280b700 1 --2- 172.21.15.3:0/3401149709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f420c09e410 0x7f420c122c30 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f420400cb40 tx=0x7f420400ce50 comp rx=0 tx=0).stop 2023-10-03T10:15:48.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.898+0000 7f421280b700 1 -- 172.21.15.3:0/3401149709 shutdown_connections 2023-10-03T10:15:48.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.898+0000 7f421280b700 1 --2- 172.21.15.3:0/3401149709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f420c09e410 0x7f420c122c30 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.898+0000 7f421280b700 1 --2- 172.21.15.3:0/3401149709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f420c09d6f0 0x7f420c1221b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.898+0000 7f421280b700 1 --2- 172.21.15.3:0/3401149709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f420c09e000 0x7f420c1226f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.898+0000 7f421280b700 1 -- 172.21.15.3:0/3401149709 >> 172.21.15.3:0/3401149709 conn(0x7f420c066f50 msgr2=0x7f420c130ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:48.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.898+0000 7f421280b700 1 -- 172.21.15.3:0/3401149709 shutdown_connections 2023-10-03T10:15:48.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.898+0000 7f421280b700 1 -- 172.21.15.3:0/3401149709 wait complete. 2023-10-03T10:15:48.912 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:48.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.935+0000 7fb2c6b4a700 1 -- 172.21.15.3:0/1260616485 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb2c006b9e0 con 0x7fb2c0134630 2023-10-03T10:15:48.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.936+0000 7fb2bd7fa700 1 -- 172.21.15.3:0/1260616485 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb2ac019bb0 con 0x7fb2c0134630 2023-10-03T10:15:48.936 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:48.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.938+0000 7fb2c6b4a700 1 -- 172.21.15.3:0/1260616485 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb2c0134630 msgr2=0x7fb2c01c89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:48.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.938+0000 7fb2c6b4a700 1 --2- 172.21.15.3:0/1260616485 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb2c0134630 0x7fb2c01c89b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb2c006dd30 tx=0x7fb2ac004750 comp rx=0 tx=0).stop 2023-10-03T10:15:48.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.938+0000 7fb2c6b4a700 1 -- 172.21.15.3:0/1260616485 shutdown_connections 2023-10-03T10:15:48.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.938+0000 7fb2c6b4a700 1 --2- 172.21.15.3:0/1260616485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2c0136f50 0x7fb2c01c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.938+0000 7fb2c6b4a700 1 --2- 172.21.15.3:0/1260616485 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb2c0131810 0x7fb2c01c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.938+0000 7fb2c6b4a700 1 --2- 172.21.15.3:0/1260616485 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb2c0134630 0x7fb2c01c89b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.938+0000 7fb2c6b4a700 1 -- 172.21.15.3:0/1260616485 >> 172.21.15.3:0/1260616485 conn(0x7fb2c00a2aa0 msgr2=0x7fb2c0133200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:48.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.938+0000 7fb2c6b4a700 1 -- 172.21.15.3:0/1260616485 shutdown_connections 2023-10-03T10:15:48.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.938+0000 7fb2c6b4a700 1 -- 172.21.15.3:0/1260616485 wait complete. 2023-10-03T10:15:48.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.948+0000 7fa5ab0e7700 1 -- 172.21.15.3:0/2905677314 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa5a409e8c0 con 0x7fa5a412cf60 2023-10-03T10:15:48.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.948+0000 7fa5a17fa700 1 -- 172.21.15.3:0/2905677314 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa59803a020 con 0x7fa5a412cf60 2023-10-03T10:15:48.948 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:48.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.950+0000 7fa5ab0e7700 1 -- 172.21.15.3:0/2905677314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5a412cf60 msgr2=0x7fa5a409bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:48.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.950+0000 7fa5ab0e7700 1 --2- 172.21.15.3:0/2905677314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5a412cf60 0x7fa5a409bbe0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fa59801d000 tx=0x7fa598004750 comp rx=0 tx=0).stop 2023-10-03T10:15:48.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.951+0000 7fa5ab0e7700 1 -- 172.21.15.3:0/2905677314 shutdown_connections 2023-10-03T10:15:48.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.951+0000 7fa5ab0e7700 1 --2- 172.21.15.3:0/2905677314 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5a4130430 0x7fa5a41c4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.951+0000 7fa5ab0e7700 1 --2- 172.21.15.3:0/2905677314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5a412cf60 0x7fa5a409bbe0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.951+0000 7fa5ab0e7700 1 --2- 172.21.15.3:0/2905677314 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa5a412f8b0 0x7fa5a409c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:48.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.951+0000 7fa5ab0e7700 1 -- 172.21.15.3:0/2905677314 >> 172.21.15.3:0/2905677314 conn(0x7fa5a40a2aa0 msgr2=0x7fa5a406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:48.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.951+0000 7fa5ab0e7700 1 -- 172.21.15.3:0/2905677314 shutdown_connections 2023-10-03T10:15:48.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:48.951+0000 7fa5ab0e7700 1 -- 172.21.15.3:0/2905677314 wait complete. 2023-10-03T10:15:48.952 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:48.964 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:49.860 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:49.913 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:49.953 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:49.966 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:50.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.015+0000 7f8902f85700 1 Processor -- start 2023-10-03T10:15:50.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.015+0000 7f8902f85700 1 -- start start 2023-10-03T10:15:50.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.016+0000 7f8902f85700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f88fc131810 0x7f88fc131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:50.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.016+0000 7f8902f85700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88fc099a40 0x7f88fc099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:50.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.016+0000 7f8902f85700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88fc09a3b0 0x7f88fc138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:50.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.016+0000 7f8902f85700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88fc1313c0 con 0x7f88fc131810 2023-10-03T10:15:50.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.016+0000 7f8902f85700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc06e8c0 con 0x7f88fc099a40 2023-10-03T10:15:50.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.016+0000 7f8902f85700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc071d20 con 0x7f88fc09a3b0 2023-10-03T10:15:50.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.016+0000 7f8901522700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88fc09a3b0 0x7f88fc138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:50.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.016+0000 7f88fbfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88fc099a40 0x7f88fc099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:50.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.016+0000 7f88fbfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88fc099a40 0x7f88fc099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34844/0 (socket says 172.21.15.3:34844) 2023-10-03T10:15:50.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.016+0000 7f8901522700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88fc09a3b0 0x7f88fc138650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47022/0 (socket says 172.21.15.3:47022) 2023-10-03T10:15:50.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.016+0000 7f88fbfff700 1 -- 172.21.15.3:0/2234172386 learned_addr learned my addr 172.21.15.3:0/2234172386 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:50.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.017+0000 7f8900d21700 1 --2- 172.21.15.3:0/2234172386 >> v2:172.21.15.3:3300/0 conn(0x7f88fc131810 0x7f88fc131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:50.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.017+0000 7f8901522700 1 -- 172.21.15.3:0/2234172386 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88fc099a40 msgr2=0x7f88fc099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:50.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.017+0000 7f8901522700 1 --2- 172.21.15.3:0/2234172386 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88fc099a40 0x7f88fc099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.017+0000 7f8901522700 1 -- 172.21.15.3:0/2234172386 >> v2:172.21.15.3:3300/0 conn(0x7f88fc131810 msgr2=0x7f88fc131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:50.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.017+0000 7f8901522700 1 --2- 172.21.15.3:0/2234172386 >> v2:172.21.15.3:3300/0 conn(0x7f88fc131810 0x7f88fc131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.017+0000 7f8901522700 1 -- 172.21.15.3:0/2234172386 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88fc0721a0 con 0x7f88fc09a3b0 2023-10-03T10:15:50.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.017+0000 7f8901522700 1 --2- 172.21.15.3:0/2234172386 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88fc09a3b0 0x7f88fc138650 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f88f4009d90 tx=0x7f88f400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b11b77eb4050ef6f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:50.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.017+0000 7f88fbfff700 1 --2- 172.21.15.3:0/2234172386 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88fc099a40 0x7f88fc099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:15:50.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.017+0000 7f88fb7fe700 1 -- 172.21.15.3:0/2234172386 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f400e070 con 0x7f88fc09a3b0 2023-10-03T10:15:50.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.017+0000 7f88fb7fe700 1 -- 172.21.15.3:0/2234172386 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f400bb20 con 0x7f88fc09a3b0 2023-10-03T10:15:50.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.017+0000 7f88fb7fe700 1 -- 172.21.15.3:0/2234172386 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f4012430 con 0x7f88fc09a3b0 2023-10-03T10:15:50.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.018+0000 7f8902f85700 1 -- 172.21.15.3:0/2234172386 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88fc09a3b0 msgr2=0x7f88fc138650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:50.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.018+0000 7f8902f85700 1 --2- 172.21.15.3:0/2234172386 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88fc09a3b0 0x7f88fc138650 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f88f4009d90 tx=0x7f88f400b330 comp rx=0 tx=0).stop 2023-10-03T10:15:50.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.018+0000 7f8902f85700 1 -- 172.21.15.3:0/2234172386 shutdown_connections 2023-10-03T10:15:50.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.019+0000 7f8902f85700 1 --2- 172.21.15.3:0/2234172386 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88fc09a3b0 0x7f88fc138650 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.019+0000 7f8902f85700 1 --2- 172.21.15.3:0/2234172386 >> v2:172.21.15.3:3300/0 conn(0x7f88fc131810 0x7f88fc131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.019+0000 7f8902f85700 1 --2- 172.21.15.3:0/2234172386 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88fc099a40 0x7f88fc099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.019+0000 7f8902f85700 1 -- 172.21.15.3:0/2234172386 >> 172.21.15.3:0/2234172386 conn(0x7f88fc0a2aa0 msgr2=0x7f88fc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:50.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.019+0000 7f8902f85700 1 -- 172.21.15.3:0/2234172386 shutdown_connections 2023-10-03T10:15:50.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.019+0000 7f8902f85700 1 -- 172.21.15.3:0/2234172386 wait complete. 2023-10-03T10:15:50.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.020+0000 7f8902f85700 1 Processor -- start 2023-10-03T10:15:50.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.020+0000 7f8902f85700 1 -- start start 2023-10-03T10:15:50.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.021+0000 7f8902f85700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88fc099a40 0x7f88fc06f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:50.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.021+0000 7f8902f85700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88fc09a3b0 0x7f88fc06fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:50.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.021+0000 7f8902f85700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88fc131810 0x7f88fc070060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:50.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.021+0000 7f8902f85700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc11eff0 con 0x7f88fc099a40 2023-10-03T10:15:50.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.021+0000 7f8902f85700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc1307e0 con 0x7f88fc09a3b0 2023-10-03T10:15:50.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.021+0000 7f8902f85700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc06eb00 con 0x7f88fc131810 2023-10-03T10:15:50.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.021+0000 7f8900d21700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88fc099a40 0x7f88fc06f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:50.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.021+0000 7f8901522700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88fc131810 0x7f88fc070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:50.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.022+0000 7f8900d21700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88fc099a40 0x7f88fc06f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55686/0 (socket says 172.21.15.3:55686) 2023-10-03T10:15:50.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.022+0000 7f8900d21700 1 -- 172.21.15.3:0/3855100699 learned_addr learned my addr 172.21.15.3:0/3855100699 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:50.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.022+0000 7f88fbfff700 1 --2- 172.21.15.3:0/3855100699 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88fc09a3b0 0x7f88fc06fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:50.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.022+0000 7f8901522700 1 -- 172.21.15.3:0/3855100699 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88fc09a3b0 msgr2=0x7f88fc06fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:50.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.022+0000 7f8901522700 1 --2- 172.21.15.3:0/3855100699 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88fc09a3b0 0x7f88fc06fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.022+0000 7f8901522700 1 -- 172.21.15.3:0/3855100699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88fc099a40 msgr2=0x7f88fc06f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:50.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.022+0000 7f8901522700 1 --2- 172.21.15.3:0/3855100699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88fc099a40 0x7f88fc06f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.022+0000 7f8901522700 1 -- 172.21.15.3:0/3855100699 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f4009a70 con 0x7f88fc131810 2023-10-03T10:15:50.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.022+0000 7f8901522700 1 --2- 172.21.15.3:0/3855100699 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88fc131810 0x7f88fc070060 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f88f4016000 tx=0x7f88f4004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:50.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.023+0000 7f88f97fa700 1 -- 172.21.15.3:0/3855100699 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f4012430 con 0x7f88fc131810 2023-10-03T10:15:50.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.023+0000 7f88f97fa700 1 -- 172.21.15.3:0/3855100699 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f4004240 con 0x7f88fc131810 2023-10-03T10:15:50.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.023+0000 7f88f97fa700 1 -- 172.21.15.3:0/3855100699 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f40045f0 con 0x7f88fc131810 2023-10-03T10:15:50.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.023+0000 7f8902f85700 1 -- 172.21.15.3:0/3855100699 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88fc071290 con 0x7f88fc131810 2023-10-03T10:15:50.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.023+0000 7f8902f85700 1 -- 172.21.15.3:0/3855100699 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88fc11fa60 con 0x7f88fc131810 2023-10-03T10:15:50.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.024+0000 7f8902f85700 1 -- 172.21.15.3:0/3855100699 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88fc074ea0 con 0x7f88fc131810 2023-10-03T10:15:50.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.024+0000 7f88f97fa700 1 -- 172.21.15.3:0/3855100699 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f88f4004750 con 0x7f88fc131810 2023-10-03T10:15:50.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.025+0000 7f88f97fa700 1 -- 172.21.15.3:0/3855100699 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f88f40413a0 con 0x7f88fc131810 2023-10-03T10:15:50.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.034+0000 7f88f97fa700 1 -- 172.21.15.3:0/3855100699 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f88f4025370 con 0x7f88fc131810 2023-10-03T10:15:50.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.073+0000 7f27a494e700 1 Processor -- start 2023-10-03T10:15:50.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.074+0000 7f27a494e700 1 -- start start 2023-10-03T10:15:50.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.074+0000 7f27a494e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f27a00a0a30 0x7f27a00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:50.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.074+0000 7f27a494e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f27a00a1340 0x7f27a0133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:50.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.074+0000 7f27a494e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f27a0133fa0 0x7f27a0134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:50.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.074+0000 7f27a494e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27a009edd0 con 0x7f27a00a1340 2023-10-03T10:15:50.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.074+0000 7f27a494e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a009e1f0 con 0x7f27a00a0a30 2023-10-03T10:15:50.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.074+0000 7f27a494e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f27a009dc50 con 0x7f27a0133fa0 2023-10-03T10:15:50.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.075+0000 7f279ed9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f27a0133fa0 0x7f27a0134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:50.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.075+0000 7f279ed9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f27a0133fa0 0x7f27a0134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47040/0 (socket says 172.21.15.3:47040) 2023-10-03T10:15:50.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.075+0000 7f279ed9d700 1 -- 172.21.15.3:0/2566488902 learned_addr learned my addr 172.21.15.3:0/2566488902 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:50.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.075+0000 7f279e59c700 1 --2- 172.21.15.3:0/2566488902 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f27a00a0a30 0x7f27a00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:50.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.075+0000 7f279dd9b700 1 --2- 172.21.15.3:0/2566488902 >> v2:172.21.15.3:3300/0 conn(0x7f27a00a1340 0x7f27a0133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:50.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.075+0000 7f279ed9d700 1 -- 172.21.15.3:0/2566488902 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f27a00a0a30 msgr2=0x7f27a00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:50.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.075+0000 7f279ed9d700 1 --2- 172.21.15.3:0/2566488902 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f27a00a0a30 0x7f27a00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.075+0000 7f279ed9d700 1 -- 172.21.15.3:0/2566488902 >> v2:172.21.15.3:3300/0 conn(0x7f27a00a1340 msgr2=0x7f27a0133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:50.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.075+0000 7f279ed9d700 1 --2- 172.21.15.3:0/2566488902 >> v2:172.21.15.3:3300/0 conn(0x7f27a00a1340 0x7f27a0133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.075+0000 7f279ed9d700 1 -- 172.21.15.3:0/2566488902 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a00a1fe0 con 0x7f27a0133fa0 2023-10-03T10:15:50.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.075+0000 7f279ed9d700 1 --2- 172.21.15.3:0/2566488902 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f27a0133fa0 0x7f27a0134410 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2798009e80 tx=0x7f279800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3666dcec5d076a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:50.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.075+0000 7f279d59a700 1 -- 172.21.15.3:0/2566488902 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279800e070 con 0x7f27a0133fa0 2023-10-03T10:15:50.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.076+0000 7f279d59a700 1 -- 172.21.15.3:0/2566488902 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279800b8d0 con 0x7f27a0133fa0 2023-10-03T10:15:50.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.076+0000 7f279d59a700 1 -- 172.21.15.3:0/2566488902 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279800bc80 con 0x7f27a0133fa0 2023-10-03T10:15:50.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.077+0000 7f27a494e700 1 -- 172.21.15.3:0/2566488902 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f27a0133fa0 msgr2=0x7f27a0134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:50.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.077+0000 7f27a494e700 1 --2- 172.21.15.3:0/2566488902 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f27a0133fa0 0x7f27a0134410 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2798009e80 tx=0x7f279800b330 comp rx=0 tx=0).stop 2023-10-03T10:15:50.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.077+0000 7f27a494e700 1 -- 172.21.15.3:0/2566488902 shutdown_connections 2023-10-03T10:15:50.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.077+0000 7f27a494e700 1 --2- 172.21.15.3:0/2566488902 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f27a0133fa0 0x7f27a0134410 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.077+0000 7f27a494e700 1 --2- 172.21.15.3:0/2566488902 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f27a00a0a30 0x7f27a00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.077+0000 7f27a494e700 1 --2- 172.21.15.3:0/2566488902 >> v2:172.21.15.3:3300/0 conn(0x7f27a00a1340 0x7f27a0133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.077+0000 7f27a494e700 1 -- 172.21.15.3:0/2566488902 >> 172.21.15.3:0/2566488902 conn(0x7f27a00a6550 msgr2=0x7f27a00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:50.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.077+0000 7f27a494e700 1 -- 172.21.15.3:0/2566488902 shutdown_connections 2023-10-03T10:15:50.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.077+0000 7f27a494e700 1 -- 172.21.15.3:0/2566488902 wait complete. 2023-10-03T10:15:50.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.078+0000 7f27a494e700 1 Processor -- start 2023-10-03T10:15:50.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.078+0000 7f27a494e700 1 -- start start 2023-10-03T10:15:50.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.079+0000 7f27a494e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f27a00a0a30 0x7f27a0120760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:50.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.079+0000 7f27a494e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27a00a1340 0x7f27a0120ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:50.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.079+0000 7f27a494e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f27a0133fa0 0x7f27a01211e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:50.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.079+0000 7f27a494e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a0121c10 con 0x7f27a00a1340 2023-10-03T10:15:50.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.079+0000 7f27a494e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a0122290 con 0x7f27a0133fa0 2023-10-03T10:15:50.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.079+0000 7f27a494e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f27a01234e0 con 0x7f27a00a0a30 2023-10-03T10:15:50.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.079+0000 7f279ed9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f27a0133fa0 0x7f27a01211e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:50.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.079+0000 7f279ed9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f27a0133fa0 0x7f27a01211e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34860/0 (socket says 172.21.15.3:34860) 2023-10-03T10:15:50.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.079+0000 7f279ed9d700 1 -- 172.21.15.3:0/2516969226 learned_addr learned my addr 172.21.15.3:0/2516969226 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:50.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.079+0000 7f279dd9b700 1 --2- 172.21.15.3:0/2516969226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27a00a1340 0x7f27a0120ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:50.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.079+0000 7f279e59c700 1 --2- 172.21.15.3:0/2516969226 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f27a00a0a30 0x7f27a0120760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:50.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.079+0000 7f279ed9d700 1 -- 172.21.15.3:0/2516969226 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f27a00a0a30 msgr2=0x7f27a0120760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:50.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.079+0000 7f279ed9d700 1 --2- 172.21.15.3:0/2516969226 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f27a00a0a30 0x7f27a0120760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.079+0000 7f279ed9d700 1 -- 172.21.15.3:0/2516969226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27a00a1340 msgr2=0x7f27a0120ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:50.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.079+0000 7f279ed9d700 1 --2- 172.21.15.3:0/2516969226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27a00a1340 0x7f27a0120ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.080+0000 7f279ed9d700 1 -- 172.21.15.3:0/2516969226 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2798009b30 con 0x7f27a0133fa0 2023-10-03T10:15:50.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.080+0000 7f279ed9d700 1 --2- 172.21.15.3:0/2516969226 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f27a0133fa0 0x7f27a01211e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f27a009dd90 tx=0x7f2798004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:50.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.080+0000 7f278effd700 1 -- 172.21.15.3:0/2516969226 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2798004450 con 0x7f27a0133fa0 2023-10-03T10:15:50.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.080+0000 7f278effd700 1 -- 172.21.15.3:0/2516969226 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27980045b0 con 0x7f27a0133fa0 2023-10-03T10:15:50.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.080+0000 7f27a494e700 1 -- 172.21.15.3:0/2516969226 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27a0123ec0 con 0x7f27a0133fa0 2023-10-03T10:15:50.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.080+0000 7f278effd700 1 -- 172.21.15.3:0/2516969226 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2798004960 con 0x7f27a0133fa0 2023-10-03T10:15:50.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.080+0000 7f27a494e700 1 -- 172.21.15.3:0/2516969226 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27a00a0220 con 0x7f27a0133fa0 2023-10-03T10:15:50.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.081+0000 7f278effd700 1 -- 172.21.15.3:0/2516969226 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f279802d420 con 0x7f27a0133fa0 2023-10-03T10:15:50.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.082+0000 7f278effd700 1 -- 172.21.15.3:0/2516969226 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2798051fd0 con 0x7f27a0133fa0 2023-10-03T10:15:50.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.082+0000 7f27a494e700 1 -- 172.21.15.3:0/2516969226 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27a0074ea0 con 0x7f27a0133fa0 2023-10-03T10:15:50.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.087+0000 7f278effd700 1 -- 172.21.15.3:0/2516969226 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f279803e350 con 0x7f27a0133fa0 2023-10-03T10:15:50.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.118+0000 7f192bab4700 1 Processor -- start 2023-10-03T10:15:50.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.118+0000 7f192bab4700 1 -- start start 2023-10-03T10:15:50.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.118+0000 7f192bab4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f192409d6f0 0x7f192409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:50.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.118+0000 7f192bab4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f192409e000 0x7f19241339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.118+0000 7f192bab4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f192409e410 0x7f1924133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.118+0000 7f192bab4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f192406b460 con 0x7f192409d6f0 2023-10-03T10:15:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.118+0000 7f192bab4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f192409b120 con 0x7f192409e410 2023-10-03T10:15:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.118+0000 7f192bab4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f192409a7e0 con 0x7f192409e000 2023-10-03T10:15:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.119+0000 7f192904f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f192409e000 0x7f19241339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.119+0000 7f192904f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f192409e000 0x7f19241339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47068/0 (socket says 172.21.15.3:47068) 2023-10-03T10:15:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.119+0000 7f192904f700 1 -- 172.21.15.3:0/3390170777 learned_addr learned my addr 172.21.15.3:0/3390170777 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.119+0000 7f192a051700 1 --2- 172.21.15.3:0/3390170777 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f192409e410 0x7f1924133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.119+0000 7f1929850700 1 --2- 172.21.15.3:0/3390170777 >> v2:172.21.15.3:3300/0 conn(0x7f192409d6f0 0x7f192409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.119+0000 7f192904f700 1 -- 172.21.15.3:0/3390170777 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f192409e410 msgr2=0x7f1924133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.119+0000 7f192904f700 1 --2- 172.21.15.3:0/3390170777 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f192409e410 0x7f1924133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.119+0000 7f192904f700 1 -- 172.21.15.3:0/3390170777 >> v2:172.21.15.3:3300/0 conn(0x7f192409d6f0 msgr2=0x7f192409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.119+0000 7f192904f700 1 --2- 172.21.15.3:0/3390170777 >> v2:172.21.15.3:3300/0 conn(0x7f192409d6f0 0x7f192409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.119+0000 7f192904f700 1 -- 172.21.15.3:0/3390170777 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192409a500 con 0x7f192409e000 2023-10-03T10:15:50.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.119+0000 7f192a051700 1 --2- 172.21.15.3:0/3390170777 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f192409e410 0x7f1924133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:50.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.119+0000 7f1929850700 1 --2- 172.21.15.3:0/3390170777 >> v2:172.21.15.3:3300/0 conn(0x7f192409d6f0 0x7f192409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:15:50.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.119+0000 7f192904f700 1 --2- 172.21.15.3:0/3390170777 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f192409e000 0x7f19241339f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1914009920 tx=0x7f1914009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6de0fbf463f8ab21 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:50.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.120+0000 7f192884e700 1 -- 172.21.15.3:0/3390170777 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1914011830 con 0x7f192409e000 2023-10-03T10:15:50.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.120+0000 7f192884e700 1 -- 172.21.15.3:0/3390170777 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1914011990 con 0x7f192409e000 2023-10-03T10:15:50.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.120+0000 7f192884e700 1 -- 172.21.15.3:0/3390170777 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1914011d20 con 0x7f192409e000 2023-10-03T10:15:50.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.120+0000 7f192bab4700 1 -- 172.21.15.3:0/3390170777 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f192409e000 msgr2=0x7f19241339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:50.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.120+0000 7f192bab4700 1 --2- 172.21.15.3:0/3390170777 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f192409e000 0x7f19241339f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1914009920 tx=0x7f1914009c30 comp rx=0 tx=0).stop 2023-10-03T10:15:50.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.121+0000 7f192bab4700 1 -- 172.21.15.3:0/3390170777 shutdown_connections 2023-10-03T10:15:50.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.121+0000 7f192bab4700 1 --2- 172.21.15.3:0/3390170777 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f192409e410 0x7f1924133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.121+0000 7f192bab4700 1 --2- 172.21.15.3:0/3390170777 >> v2:172.21.15.3:3300/0 conn(0x7f192409d6f0 0x7f192409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.121+0000 7f192bab4700 1 --2- 172.21.15.3:0/3390170777 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f192409e000 0x7f19241339f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.121+0000 7f192bab4700 1 -- 172.21.15.3:0/3390170777 >> 172.21.15.3:0/3390170777 conn(0x7f1924066f50 msgr2=0x7f192409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:50.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.121+0000 7f192bab4700 1 -- 172.21.15.3:0/3390170777 shutdown_connections 2023-10-03T10:15:50.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.121+0000 7f192bab4700 1 -- 172.21.15.3:0/3390170777 wait complete. 2023-10-03T10:15:50.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.122+0000 7f192bab4700 1 Processor -- start 2023-10-03T10:15:50.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.122+0000 7f192bab4700 1 -- start start 2023-10-03T10:15:50.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.122+0000 7f192bab4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f192409d6f0 0x7f19241a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:50.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.122+0000 7f192bab4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f192409e000 0x7f19241a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:50.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.122+0000 7f192bab4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f192409e410 0x7f19241a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:50.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.122+0000 7f192bab4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19241aa250 con 0x7f192409e410 2023-10-03T10:15:50.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.122+0000 7f192bab4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19241a9890 con 0x7f192409d6f0 2023-10-03T10:15:50.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.122+0000 7f192bab4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f19241a8c60 con 0x7f192409e000 2023-10-03T10:15:50.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.122+0000 7f1929850700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f192409d6f0 0x7f19241a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:50.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.122+0000 7f1929850700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f192409d6f0 0x7f19241a5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34886/0 (socket says 172.21.15.3:34886) 2023-10-03T10:15:50.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.122+0000 7f1929850700 1 -- 172.21.15.3:0/3918192037 learned_addr learned my addr 172.21.15.3:0/3918192037 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:50.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.123+0000 7f192a051700 1 --2- 172.21.15.3:0/3918192037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f192409e410 0x7f19241a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:50.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.123+0000 7f192904f700 1 --2- 172.21.15.3:0/3918192037 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f192409e000 0x7f19241a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:50.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.123+0000 7f1929850700 1 -- 172.21.15.3:0/3918192037 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f192409e000 msgr2=0x7f19241a62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:50.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.123+0000 7f1929850700 1 --2- 172.21.15.3:0/3918192037 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f192409e000 0x7f19241a62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.123+0000 7f1929850700 1 -- 172.21.15.3:0/3918192037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f192409e410 msgr2=0x7f19241a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:50.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.123+0000 7f1929850700 1 --2- 172.21.15.3:0/3918192037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f192409e410 0x7f19241a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.123+0000 7f1929850700 1 -- 172.21.15.3:0/3918192037 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191c009a70 con 0x7f192409d6f0 2023-10-03T10:15:50.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.123+0000 7f192904f700 1 --2- 172.21.15.3:0/3918192037 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f192409e000 0x7f19241a62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:15:50.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.123+0000 7f1929850700 1 --2- 172.21.15.3:0/3918192037 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f192409d6f0 0x7f19241a5d70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f192000c780 tx=0x7f192000cb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:50.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.124+0000 7f191a7fc700 1 -- 172.21.15.3:0/3918192037 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920011800 con 0x7f192409d6f0 2023-10-03T10:15:50.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.124+0000 7f191a7fc700 1 -- 172.21.15.3:0/3918192037 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1920011960 con 0x7f192409d6f0 2023-10-03T10:15:50.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.124+0000 7f191a7fc700 1 -- 172.21.15.3:0/3918192037 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920011cd0 con 0x7f192409d6f0 2023-10-03T10:15:50.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.124+0000 7f192bab4700 1 -- 172.21.15.3:0/3918192037 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1914009550 con 0x7f192409d6f0 2023-10-03T10:15:50.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.124+0000 7f192bab4700 1 -- 172.21.15.3:0/3918192037 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19240a2000 con 0x7f192409d6f0 2023-10-03T10:15:50.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.125+0000 7f191a7fc700 1 -- 172.21.15.3:0/3918192037 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1920021350 con 0x7f192409d6f0 2023-10-03T10:15:50.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.125+0000 7f192bab4700 1 -- 172.21.15.3:0/3918192037 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f192409c770 con 0x7f192409d6f0 2023-10-03T10:15:50.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.125+0000 7f191a7fc700 1 -- 172.21.15.3:0/3918192037 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f192003f7b0 con 0x7f192409d6f0 2023-10-03T10:15:50.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.134+0000 7f191a7fc700 1 -- 172.21.15.3:0/3918192037 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1920011e30 con 0x7f192409d6f0 2023-10-03T10:15:50.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.156+0000 7f1d13b32700 1 Processor -- start 2023-10-03T10:15:50.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.156+0000 7f1d13b32700 1 -- start start 2023-10-03T10:15:50.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.157+0000 7f1d13b32700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1d0c136820 0x7f1d0c138c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:50.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.157+0000 7f1d13b32700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1d0c139140 0x7f1d0c13b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:50.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.157+0000 7f1d13b32700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d0c06bc00 con 0x7f1d0c133a80 2023-10-03T10:15:50.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.157+0000 7f1d13b32700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c06c570 con 0x7f1d0c136820 2023-10-03T10:15:50.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.157+0000 7f1d13b32700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c0a0040 con 0x7f1d0c139140 2023-10-03T10:15:50.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.157+0000 7f1d118ce700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1d0c133a80 0x7f1d0c133e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57318/0 (socket says 172.21.15.3:57318) 2023-10-03T10:15:50.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.157+0000 7f1d118ce700 1 -- 172.21.15.3:0/4245890770 learned_addr learned my addr 172.21.15.3:0/4245890770 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:50.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.157+0000 7f1d110cd700 1 --2- 172.21.15.3:0/4245890770 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1d0c136820 0x7f1d0c138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:50.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.157+0000 7f1d108cc700 1 -- 172.21.15.3:0/4245890770 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 796749465 0 0) 0x7f1d0c06bc00 con 0x7f1d0c133a80 2023-10-03T10:15:50.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.157+0000 7f1d108cc700 1 -- 172.21.15.3:0/4245890770 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cf80035b0 con 0x7f1d0c133a80 2023-10-03T10:15:50.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.157+0000 7f1d110cd700 1 -- 172.21.15.3:0/4245890770 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1d0c139140 msgr2=0x7f1d0c13b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:50.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.157+0000 7f1d110cd700 1 --2- 172.21.15.3:0/4245890770 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1d0c139140 0x7f1d0c13b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.158+0000 7f1d110cd700 1 -- 172.21.15.3:0/4245890770 >> v1:172.21.15.3:6789/0 conn(0x7f1d0c133a80 legacy=0x7f1d0c133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:50.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.158+0000 7f1d110cd700 1 -- 172.21.15.3:0/4245890770 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0c133860 con 0x7f1d0c136820 2023-10-03T10:15:50.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.158+0000 7f1d110cd700 1 --2- 172.21.15.3:0/4245890770 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1d0c136820 0x7f1d0c138c00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc009870 tx=0x7f1cfc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fff0c2e3af56da93 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:50.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.158+0000 7f1d108cc700 1 -- 172.21.15.3:0/4245890770 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc011830 con 0x7f1d0c136820 2023-10-03T10:15:50.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.158+0000 7f1d108cc700 1 -- 172.21.15.3:0/4245890770 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cfc011990 con 0x7f1d0c136820 2023-10-03T10:15:50.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.158+0000 7f1d108cc700 1 -- 172.21.15.3:0/4245890770 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc011d20 con 0x7f1d0c136820 2023-10-03T10:15:50.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.159+0000 7f1d13b32700 1 -- 172.21.15.3:0/4245890770 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1d0c136820 msgr2=0x7f1d0c138c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:50.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.159+0000 7f1d13b32700 1 --2- 172.21.15.3:0/4245890770 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1d0c136820 0x7f1d0c138c00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc009870 tx=0x7f1cfc009b80 comp rx=0 tx=0).stop 2023-10-03T10:15:50.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.159+0000 7f1d13b32700 1 -- 172.21.15.3:0/4245890770 shutdown_connections 2023-10-03T10:15:50.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.159+0000 7f1d13b32700 1 --2- 172.21.15.3:0/4245890770 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1d0c139140 0x7f1d0c13b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.159+0000 7f1d13b32700 1 --2- 172.21.15.3:0/4245890770 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1d0c136820 0x7f1d0c138c00 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.159+0000 7f1d13b32700 1 -- 172.21.15.3:0/4245890770 >> 172.21.15.3:0/4245890770 conn(0x7f1d0c0a4210 msgr2=0x7f1d0c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:50.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.159+0000 7f1d13b32700 1 -- 172.21.15.3:0/4245890770 shutdown_connections 2023-10-03T10:15:50.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.159+0000 7f1d13b32700 1 -- 172.21.15.3:0/4245890770 wait complete. 2023-10-03T10:15:50.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.160+0000 7f1d13b32700 1 Processor -- start 2023-10-03T10:15:50.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.160+0000 7f1d13b32700 1 -- start start 2023-10-03T10:15:50.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.161+0000 7f1d13b32700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d0c133a80 0x7f1d0c06f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:50.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.161+0000 7f1d13b32700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1d0c136820 0x7f1d0c06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:50.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.161+0000 7f1d13b32700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1d0c139140 0x7f1d0c0703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:50.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.161+0000 7f1d13b32700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c072280 con 0x7f1d0c133a80 2023-10-03T10:15:50.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.161+0000 7f1d13b32700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c09bc20 con 0x7f1d0c139140 2023-10-03T10:15:50.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.161+0000 7f1d13b32700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c09e3a0 con 0x7f1d0c136820 2023-10-03T10:15:50.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.161+0000 7f1d118ce700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d0c133a80 0x7f1d0c06f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:50.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.161+0000 7f1d118ce700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d0c133a80 0x7f1d0c06f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55718/0 (socket says 172.21.15.3:55718) 2023-10-03T10:15:50.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.161+0000 7f1d118ce700 1 -- 172.21.15.3:0/2155969981 learned_addr learned my addr 172.21.15.3:0/2155969981 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:50.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.161+0000 7f1d110cd700 1 --2- 172.21.15.3:0/2155969981 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1d0c136820 0x7f1d0c06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.161+0000 7f1d120cf700 1 --2- 172.21.15.3:0/2155969981 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1d0c139140 0x7f1d0c0703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.161+0000 7f1d118ce700 1 -- 172.21.15.3:0/2155969981 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1d0c136820 msgr2=0x7f1d0c06fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.161+0000 7f1d118ce700 1 --2- 172.21.15.3:0/2155969981 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1d0c136820 0x7f1d0c06fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.161+0000 7f1d118ce700 1 -- 172.21.15.3:0/2155969981 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1d0c139140 msgr2=0x7f1d0c0703d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.161+0000 7f1d118ce700 1 --2- 172.21.15.3:0/2155969981 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1d0c139140 0x7f1d0c0703d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.161+0000 7f1d118ce700 1 -- 172.21.15.3:0/2155969981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cfc0094a0 con 0x7f1d0c133a80 2023-10-03T10:15:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.162+0000 7f1d118ce700 1 --2- 172.21.15.3:0/2155969981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d0c133a80 0x7f1d0c06f950 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800c700 tx=0x7f1d0800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.162+0000 7f1d027fc700 1 -- 172.21.15.3:0/2155969981 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d08005d70 con 0x7f1d0c133a80 2023-10-03T10:15:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.162+0000 7f1d027fc700 1 -- 172.21.15.3:0/2155969981 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d080070d0 con 0x7f1d0c133a80 2023-10-03T10:15:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.162+0000 7f1d13b32700 1 -- 172.21.15.3:0/2155969981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d0c09dee0 con 0x7f1d0c133a80 2023-10-03T10:15:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.162+0000 7f1d027fc700 1 -- 172.21.15.3:0/2155969981 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d08007460 con 0x7f1d0c133a80 2023-10-03T10:15:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.162+0000 7f1d13b32700 1 -- 172.21.15.3:0/2155969981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d0c072010 con 0x7f1d0c133a80 2023-10-03T10:15:50.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.163+0000 7f1d027fc700 1 -- 172.21.15.3:0/2155969981 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1d080075c0 con 0x7f1d0c133a80 2023-10-03T10:15:50.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.164+0000 7f1d027fc700 1 -- 172.21.15.3:0/2155969981 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1d0803ec80 con 0x7f1d0c133a80 2023-10-03T10:15:50.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.164+0000 7f1d13b32700 1 -- 172.21.15.3:0/2155969981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d0c074ea0 con 0x7f1d0c133a80 2023-10-03T10:15:50.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.169+0000 7f1d027fc700 1 -- 172.21.15.3:0/2155969981 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1d08017080 con 0x7f1d0c133a80 2023-10-03T10:15:50.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.264+0000 7f8902f85700 1 -- 172.21.15.3:0/3855100699 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f88fc06b9e0 con 0x7f88fc131810 2023-10-03T10:15:50.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.264+0000 7f88f97fa700 1 -- 172.21.15.3:0/3855100699 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f88f402b020 con 0x7f88fc131810 2023-10-03T10:15:50.263 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:50.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.266+0000 7f8902f85700 1 -- 172.21.15.3:0/3855100699 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88fc131810 msgr2=0x7f88fc070060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:50.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.266+0000 7f8902f85700 1 --2- 172.21.15.3:0/3855100699 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88fc131810 0x7f88fc070060 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f88f4016000 tx=0x7f88f4004ad0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.266+0000 7f8902f85700 1 -- 172.21.15.3:0/3855100699 shutdown_connections 2023-10-03T10:15:50.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.266+0000 7f8902f85700 1 --2- 172.21.15.3:0/3855100699 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88fc131810 0x7f88fc070060 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.266+0000 7f8902f85700 1 --2- 172.21.15.3:0/3855100699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88fc099a40 0x7f88fc06f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.266+0000 7f8902f85700 1 --2- 172.21.15.3:0/3855100699 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88fc09a3b0 0x7f88fc06fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.266+0000 7f8902f85700 1 -- 172.21.15.3:0/3855100699 >> 172.21.15.3:0/3855100699 conn(0x7f88fc0a2aa0 msgr2=0x7f88fc12e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:50.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.267+0000 7f8902f85700 1 -- 172.21.15.3:0/3855100699 shutdown_connections 2023-10-03T10:15:50.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.267+0000 7f8902f85700 1 -- 172.21.15.3:0/3855100699 wait complete. 2023-10-03T10:15:50.285 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:50.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.310+0000 7f27a494e700 1 -- 172.21.15.3:0/2516969226 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f27a00a28e0 con 0x7f27a0133fa0 2023-10-03T10:15:50.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.310+0000 7f278effd700 1 -- 172.21.15.3:0/2516969226 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f279803b020 con 0x7f27a0133fa0 2023-10-03T10:15:50.309 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.313+0000 7f27a494e700 1 -- 172.21.15.3:0/2516969226 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f27a0133fa0 msgr2=0x7f27a01211e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.313+0000 7f27a494e700 1 --2- 172.21.15.3:0/2516969226 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f27a0133fa0 0x7f27a01211e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f27a009dd90 tx=0x7f2798004ad0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.313+0000 7f27a494e700 1 -- 172.21.15.3:0/2516969226 shutdown_connections 2023-10-03T10:15:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.313+0000 7f27a494e700 1 --2- 172.21.15.3:0/2516969226 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f27a0133fa0 0x7f27a01211e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.313+0000 7f27a494e700 1 --2- 172.21.15.3:0/2516969226 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f27a00a0a30 0x7f27a0120760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.313+0000 7f27a494e700 1 --2- 172.21.15.3:0/2516969226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27a00a1340 0x7f27a0120ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.313+0000 7f27a494e700 1 -- 172.21.15.3:0/2516969226 >> 172.21.15.3:0/2516969226 conn(0x7f27a00a6550 msgr2=0x7f27a009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:50.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.313+0000 7f27a494e700 1 -- 172.21.15.3:0/2516969226 shutdown_connections 2023-10-03T10:15:50.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.313+0000 7f27a494e700 1 -- 172.21.15.3:0/2516969226 wait complete. 2023-10-03T10:15:50.328 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:50.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.352+0000 7f192bab4700 1 -- 172.21.15.3:0/3918192037 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f19241a9430 con 0x7f192409d6f0 2023-10-03T10:15:50.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.353+0000 7f191a7fc700 1 -- 172.21.15.3:0/3918192037 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1920018b40 con 0x7f192409d6f0 2023-10-03T10:15:50.352 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:50.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.355+0000 7f192bab4700 1 -- 172.21.15.3:0/3918192037 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f192409d6f0 msgr2=0x7f19241a5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:50.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.355+0000 7f192bab4700 1 --2- 172.21.15.3:0/3918192037 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f192409d6f0 0x7f19241a5d70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f192000c780 tx=0x7f192000cb40 comp rx=0 tx=0).stop 2023-10-03T10:15:50.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.355+0000 7f192bab4700 1 -- 172.21.15.3:0/3918192037 shutdown_connections 2023-10-03T10:15:50.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.355+0000 7f192bab4700 1 --2- 172.21.15.3:0/3918192037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f192409e410 0x7f19241a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.355+0000 7f192bab4700 1 --2- 172.21.15.3:0/3918192037 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f192409d6f0 0x7f19241a5d70 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.355+0000 7f192bab4700 1 --2- 172.21.15.3:0/3918192037 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f192409e000 0x7f19241a62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.355+0000 7f192bab4700 1 -- 172.21.15.3:0/3918192037 >> 172.21.15.3:0/3918192037 conn(0x7f1924066f50 msgr2=0x7f19241babc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:50.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.355+0000 7f192bab4700 1 -- 172.21.15.3:0/3918192037 shutdown_connections 2023-10-03T10:15:50.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.355+0000 7f192bab4700 1 -- 172.21.15.3:0/3918192037 wait complete. 2023-10-03T10:15:50.368 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:50.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.377+0000 7f1d13b32700 1 -- 172.21.15.3:0/2155969981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1d0c06b250 con 0x7f1d0c133a80 2023-10-03T10:15:50.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.378+0000 7f1d027fc700 1 -- 172.21.15.3:0/2155969981 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1d0800e280 con 0x7f1d0c133a80 2023-10-03T10:15:50.377 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:50.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.380+0000 7f1d13b32700 1 -- 172.21.15.3:0/2155969981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d0c133a80 msgr2=0x7f1d0c06f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:50.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.380+0000 7f1d13b32700 1 --2- 172.21.15.3:0/2155969981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d0c133a80 0x7f1d0c06f950 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800c700 tx=0x7f1d0800ca10 comp rx=0 tx=0).stop 2023-10-03T10:15:50.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.380+0000 7f1d13b32700 1 -- 172.21.15.3:0/2155969981 shutdown_connections 2023-10-03T10:15:50.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.380+0000 7f1d13b32700 1 --2- 172.21.15.3:0/2155969981 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1d0c139140 0x7f1d0c0703d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.380+0000 7f1d13b32700 1 --2- 172.21.15.3:0/2155969981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d0c133a80 0x7f1d0c06f950 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.380+0000 7f1d13b32700 1 --2- 172.21.15.3:0/2155969981 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1d0c136820 0x7f1d0c06fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:50.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.380+0000 7f1d13b32700 1 -- 172.21.15.3:0/2155969981 >> 172.21.15.3:0/2155969981 conn(0x7f1d0c0a4210 msgr2=0x7f1d0c131960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:50.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.380+0000 7f1d13b32700 1 -- 172.21.15.3:0/2155969981 shutdown_connections 2023-10-03T10:15:50.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:50.380+0000 7f1d13b32700 1 -- 172.21.15.3:0/2155969981 wait complete. 2023-10-03T10:15:50.393 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:51.286 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:51.329 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:51.369 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:51.394 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:51.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.440+0000 7ff789675700 1 Processor -- start 2023-10-03T10:15:51.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.440+0000 7ff789675700 1 -- start start 2023-10-03T10:15:51.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.441+0000 7ff789675700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff78409d6f0 0x7ff78409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:51.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.441+0000 7ff789675700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78409e000 0x7ff784130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:51.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.441+0000 7ff789675700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78409e410 0x7ff784130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:51.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.441+0000 7ff789675700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff78409f520 con 0x7ff78409d6f0 2023-10-03T10:15:51.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.441+0000 7ff789675700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78409f130 con 0x7ff78409e000 2023-10-03T10:15:51.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.441+0000 7ff789675700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff78409e930 con 0x7ff78409e410 2023-10-03T10:15:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.441+0000 7ff782ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff78409d6f0 0x7ff78409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.441+0000 7ff7827fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78409e000 0x7ff784130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.441+0000 7ff782ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff78409d6f0 0x7ff78409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55734/0 (socket says 172.21.15.3:55734) 2023-10-03T10:15:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.441+0000 7ff782ffd700 1 -- 172.21.15.3:0/1835428659 learned_addr learned my addr 172.21.15.3:0/1835428659 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.441+0000 7ff7837fe700 1 --2- 172.21.15.3:0/1835428659 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78409e410 0x7ff784130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:51.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.441+0000 7ff7837fe700 1 -- 172.21.15.3:0/1835428659 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78409e000 msgr2=0x7ff784130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:51.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.441+0000 7ff7837fe700 1 --2- 172.21.15.3:0/1835428659 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78409e000 0x7ff784130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.441+0000 7ff7837fe700 1 -- 172.21.15.3:0/1835428659 >> v2:172.21.15.3:3300/0 conn(0x7ff78409d6f0 msgr2=0x7ff78409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:51.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.441+0000 7ff7837fe700 1 --2- 172.21.15.3:0/1835428659 >> v2:172.21.15.3:3300/0 conn(0x7ff78409d6f0 0x7ff78409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.441+0000 7ff7837fe700 1 -- 172.21.15.3:0/1835428659 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78409b2d0 con 0x7ff78409e410 2023-10-03T10:15:51.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.442+0000 7ff782ffd700 1 --2- 172.21.15.3:0/1835428659 >> v2:172.21.15.3:3300/0 conn(0x7ff78409d6f0 0x7ff78409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:15:51.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.442+0000 7ff7837fe700 1 --2- 172.21.15.3:0/1835428659 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78409e410 0x7ff784130ba0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff77c009dc0 tx=0x7ff77c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a69aa97acd7164fe server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:51.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.442+0000 7ff781ffb700 1 -- 172.21.15.3:0/1835428659 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c00e070 con 0x7ff78409e410 2023-10-03T10:15:51.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.442+0000 7ff781ffb700 1 -- 172.21.15.3:0/1835428659 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77c00bae0 con 0x7ff78409e410 2023-10-03T10:15:51.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.442+0000 7ff781ffb700 1 -- 172.21.15.3:0/1835428659 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c00be90 con 0x7ff78409e410 2023-10-03T10:15:51.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.443+0000 7ff789675700 1 -- 172.21.15.3:0/1835428659 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78409e410 msgr2=0x7ff784130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:51.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.443+0000 7ff789675700 1 --2- 172.21.15.3:0/1835428659 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78409e410 0x7ff784130ba0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff77c009dc0 tx=0x7ff77c00b330 comp rx=0 tx=0).stop 2023-10-03T10:15:51.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.443+0000 7ff789675700 1 -- 172.21.15.3:0/1835428659 shutdown_connections 2023-10-03T10:15:51.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.443+0000 7ff789675700 1 --2- 172.21.15.3:0/1835428659 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78409e410 0x7ff784130ba0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.443+0000 7ff789675700 1 --2- 172.21.15.3:0/1835428659 >> v2:172.21.15.3:3300/0 conn(0x7ff78409d6f0 0x7ff78409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.443+0000 7ff789675700 1 --2- 172.21.15.3:0/1835428659 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78409e000 0x7ff784130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.443+0000 7ff789675700 1 -- 172.21.15.3:0/1835428659 >> 172.21.15.3:0/1835428659 conn(0x7ff784066f50 msgr2=0x7ff78409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:51.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.443+0000 7ff789675700 1 -- 172.21.15.3:0/1835428659 shutdown_connections 2023-10-03T10:15:51.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.443+0000 7ff789675700 1 -- 172.21.15.3:0/1835428659 wait complete. 2023-10-03T10:15:51.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.444+0000 7ff789675700 1 Processor -- start 2023-10-03T10:15:51.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.444+0000 7ff789675700 1 -- start start 2023-10-03T10:15:51.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.445+0000 7ff789675700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78409d6f0 0x7ff784122230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:51.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.445+0000 7ff789675700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78409e000 0x7ff784122770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:51.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.445+0000 7ff789675700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78409e410 0x7ff784122cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:51.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.445+0000 7ff789675700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff784123f50 con 0x7ff78409e410 2023-10-03T10:15:51.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.445+0000 7ff789675700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff784125a70 con 0x7ff78409e000 2023-10-03T10:15:51.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.445+0000 7ff789675700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff7840a3b10 con 0x7ff78409d6f0 2023-10-03T10:15:51.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.445+0000 7ff782ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78409d6f0 0x7ff784122230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:51.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.445+0000 7ff782ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78409d6f0 0x7ff784122230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47116/0 (socket says 172.21.15.3:47116) 2023-10-03T10:15:51.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.445+0000 7ff782ffd700 1 -- 172.21.15.3:0/1489579598 learned_addr learned my addr 172.21.15.3:0/1489579598 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:51.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.445+0000 7ff7827fc700 1 --2- 172.21.15.3:0/1489579598 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78409e000 0x7ff784122770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:51.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.445+0000 7ff7837fe700 1 --2- 172.21.15.3:0/1489579598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78409e410 0x7ff784122cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:51.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.445+0000 7ff782ffd700 1 -- 172.21.15.3:0/1489579598 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78409e000 msgr2=0x7ff784122770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:51.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.445+0000 7ff782ffd700 1 --2- 172.21.15.3:0/1489579598 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78409e000 0x7ff784122770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.445+0000 7ff782ffd700 1 -- 172.21.15.3:0/1489579598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78409e410 msgr2=0x7ff784122cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:51.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.445+0000 7ff782ffd700 1 --2- 172.21.15.3:0/1489579598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78409e410 0x7ff784122cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.445+0000 7ff782ffd700 1 -- 172.21.15.3:0/1489579598 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77c009a70 con 0x7ff78409d6f0 2023-10-03T10:15:51.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.445+0000 7ff7827fc700 1 --2- 172.21.15.3:0/1489579598 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78409e000 0x7ff784122770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:51.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.445+0000 7ff7837fe700 1 --2- 172.21.15.3:0/1489579598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78409e410 0x7ff784122cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:51.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.446+0000 7ff782ffd700 1 --2- 172.21.15.3:0/1489579598 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78409d6f0 0x7ff784122230 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff77400d9f0 tx=0x7ff77400dd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:51.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.446+0000 7ff76b7fe700 1 -- 172.21.15.3:0/1489579598 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff774004230 con 0x7ff78409d6f0 2023-10-03T10:15:51.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.446+0000 7ff76b7fe700 1 -- 172.21.15.3:0/1489579598 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff774004390 con 0x7ff78409d6f0 2023-10-03T10:15:51.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.446+0000 7ff76b7fe700 1 -- 172.21.15.3:0/1489579598 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77400de90 con 0x7ff78409d6f0 2023-10-03T10:15:51.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.446+0000 7ff789675700 1 -- 172.21.15.3:0/1489579598 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff784124630 con 0x7ff78409d6f0 2023-10-03T10:15:51.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.446+0000 7ff789675700 1 -- 172.21.15.3:0/1489579598 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7841264e0 con 0x7ff78409d6f0 2023-10-03T10:15:51.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.447+0000 7ff76b7fe700 1 -- 172.21.15.3:0/1489579598 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff77400a7d0 con 0x7ff78409d6f0 2023-10-03T10:15:51.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.447+0000 7ff76b7fe700 1 -- 172.21.15.3:0/1489579598 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff77403f670 con 0x7ff78409d6f0 2023-10-03T10:15:51.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.448+0000 7ff789675700 1 -- 172.21.15.3:0/1489579598 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff784074ea0 con 0x7ff78409d6f0 2023-10-03T10:15:51.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.453+0000 7ff76b7fe700 1 -- 172.21.15.3:0/1489579598 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff7740036a0 con 0x7ff78409d6f0 2023-10-03T10:15:51.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.480+0000 7ffa196ef700 1 Processor -- start 2023-10-03T10:15:51.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.481+0000 7ffa196ef700 1 -- start start 2023-10-03T10:15:51.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.481+0000 7ffa196ef700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffa1409d6f0 0x7ffa1409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:51.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.481+0000 7ffa196ef700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffa1409e000 0x7ffa14130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:51.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.481+0000 7ffa196ef700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa1409f130 con 0x7ffa1409e410 2023-10-03T10:15:51.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.481+0000 7ffa196ef700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa1409f520 con 0x7ffa1409e000 2023-10-03T10:15:51.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.481+0000 7ffa196ef700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1409e930 con 0x7ffa1409d6f0 2023-10-03T10:15:51.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.481+0000 7ffa127fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffa1409e000 0x7ffa14130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:51.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.481+0000 7ffa12ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffa1409d6f0 0x7ffa1409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:51.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.481+0000 7ffa127fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffa1409e000 0x7ffa14130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55748/0 (socket says 172.21.15.3:55748) 2023-10-03T10:15:51.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.481+0000 7ffa127fc700 1 -- 172.21.15.3:0/809091387 learned_addr learned my addr 172.21.15.3:0/809091387 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:51.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.482+0000 7ffa127fc700 1 -- 172.21.15.3:0/809091387 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffa1409d6f0 msgr2=0x7ffa1409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:51.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.482+0000 7ffa127fc700 1 --2- 172.21.15.3:0/809091387 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffa1409d6f0 0x7ffa1409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.482+0000 7ffa127fc700 1 -- 172.21.15.3:0/809091387 >> v1:172.21.15.3:6789/0 conn(0x7ffa1409e410 legacy=0x7ffa14130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:51.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.482+0000 7ffa127fc700 1 -- 172.21.15.3:0/809091387 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1409b2d0 con 0x7ffa1409e000 2023-10-03T10:15:51.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.482+0000 7ffa12ffd700 1 --2- 172.21.15.3:0/809091387 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffa1409d6f0 0x7ffa1409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:51.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.482+0000 7ffa127fc700 1 --2- 172.21.15.3:0/809091387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa1409e000 0x7ffa14130660 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800b9a0 tx=0x7ffa0800bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9cd8426200b5688 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:51.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.482+0000 7ffa11ffb700 1 -- 172.21.15.3:0/809091387 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0800caa0 con 0x7ffa1409e000 2023-10-03T10:15:51.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.482+0000 7ffa11ffb700 1 -- 172.21.15.3:0/809091387 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa0800cc00 con 0x7ffa1409e000 2023-10-03T10:15:51.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.482+0000 7ffa11ffb700 1 -- 172.21.15.3:0/809091387 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0800f460 con 0x7ffa1409e000 2023-10-03T10:15:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.483+0000 7ffa196ef700 1 -- 172.21.15.3:0/809091387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa1409e000 msgr2=0x7ffa14130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.483+0000 7ffa196ef700 1 --2- 172.21.15.3:0/809091387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa1409e000 0x7ffa14130660 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800b9a0 tx=0x7ffa0800bcb0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.483+0000 7ffa196ef700 1 -- 172.21.15.3:0/809091387 shutdown_connections 2023-10-03T10:15:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.483+0000 7ffa196ef700 1 --2- 172.21.15.3:0/809091387 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffa1409d6f0 0x7ffa1409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.483+0000 7ffa196ef700 1 --2- 172.21.15.3:0/809091387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa1409e000 0x7ffa14130660 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.483+0000 7ffa196ef700 1 -- 172.21.15.3:0/809091387 >> 172.21.15.3:0/809091387 conn(0x7ffa14066f50 msgr2=0x7ffa1409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.483+0000 7ffa196ef700 1 -- 172.21.15.3:0/809091387 shutdown_connections 2023-10-03T10:15:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.483+0000 7ffa196ef700 1 -- 172.21.15.3:0/809091387 wait complete. 2023-10-03T10:15:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.484+0000 7ffa196ef700 1 Processor -- start 2023-10-03T10:15:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.485+0000 7ffa196ef700 1 -- start start 2023-10-03T10:15:51.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.485+0000 7ffa196ef700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa1409d6f0 0x7ffa1411df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:51.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.485+0000 7ffa196ef700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa1409e000 0x7ffa1411e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:51.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.485+0000 7ffa196ef700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffa1409e410 0x7ffa1411ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:51.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.485+0000 7ffa196ef700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1411f450 con 0x7ffa1409e000 2023-10-03T10:15:51.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.485+0000 7ffa196ef700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa14121aa0 con 0x7ffa1409d6f0 2023-10-03T10:15:51.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.485+0000 7ffa196ef700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa141201b0 con 0x7ffa1409e410 2023-10-03T10:15:51.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.485+0000 7ffa12ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa1409d6f0 0x7ffa1411df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.485+0000 7ffa127fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa1409e000 0x7ffa1411e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.485+0000 7ffa12ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa1409d6f0 0x7ffa1411df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34944/0 (socket says 172.21.15.3:34944) 2023-10-03T10:15:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.485+0000 7ffa12ffd700 1 -- 172.21.15.3:0/1868577211 learned_addr learned my addr 172.21.15.3:0/1868577211 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.485+0000 7ffa137fe700 1 --2- 172.21.15.3:0/1868577211 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffa1409e410 0x7ffa1411ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.486+0000 7ffa127fc700 1 -- 172.21.15.3:0/1868577211 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffa1409e410 msgr2=0x7ffa1411ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.486+0000 7ffa127fc700 1 --2- 172.21.15.3:0/1868577211 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffa1409e410 0x7ffa1411ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.486+0000 7ffa127fc700 1 -- 172.21.15.3:0/1868577211 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa1409d6f0 msgr2=0x7ffa1411df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.486+0000 7ffa127fc700 1 --2- 172.21.15.3:0/1868577211 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa1409d6f0 0x7ffa1411df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.486+0000 7ffa127fc700 1 -- 172.21.15.3:0/1868577211 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0400d940 con 0x7ffa1409e000 2023-10-03T10:15:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.486+0000 7ffa127fc700 1 --2- 172.21.15.3:0/1868577211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa1409e000 0x7ffa1411e4c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ffa1406d540 tx=0x7ffa08004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.486+0000 7ff9fb7fe700 1 -- 172.21.15.3:0/1868577211 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0801c070 con 0x7ffa1409e000 2023-10-03T10:15:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.486+0000 7ff9fb7fe700 1 -- 172.21.15.3:0/1868577211 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa0800c8a0 con 0x7ffa1409e000 2023-10-03T10:15:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.486+0000 7ffa196ef700 1 -- 172.21.15.3:0/1868577211 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa0800b600 con 0x7ffa1409e000 2023-10-03T10:15:51.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.486+0000 7ff9fb7fe700 1 -- 172.21.15.3:0/1868577211 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa08004270 con 0x7ffa1409e000 2023-10-03T10:15:51.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.486+0000 7ffa196ef700 1 -- 172.21.15.3:0/1868577211 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa1409c700 con 0x7ffa1409e000 2023-10-03T10:15:51.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.488+0000 7ffa196ef700 1 -- 172.21.15.3:0/1868577211 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa14121620 con 0x7ffa1409e000 2023-10-03T10:15:51.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.488+0000 7ff9fb7fe700 1 -- 172.21.15.3:0/1868577211 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffa080043d0 con 0x7ffa1409e000 2023-10-03T10:15:51.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.488+0000 7ff9fb7fe700 1 -- 172.21.15.3:0/1868577211 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ffa0802a3b0 con 0x7ffa1409e000 2023-10-03T10:15:51.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.493+0000 7ff9fb7fe700 1 -- 172.21.15.3:0/1868577211 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffa08029360 con 0x7ffa1409e000 2023-10-03T10:15:51.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.523+0000 7f79f4910700 1 Processor -- start 2023-10-03T10:15:51.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.523+0000 7f79f4910700 1 -- start start 2023-10-03T10:15:51.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.524+0000 7f79f4910700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f79f01337f0 0x7f79f0138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:51.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.524+0000 7f79f4910700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79f0138dc0 0x7f79f013b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:51.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.524+0000 7f79f4910700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f006b990 con 0x7f79f0133420 2023-10-03T10:15:51.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.524+0000 7f79f4910700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f009b0e0 con 0x7f79f01337f0 2023-10-03T10:15:51.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.524+0000 7f79f4910700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f0133c20 con 0x7f79f0138dc0 2023-10-03T10:15:51.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.524+0000 7f79ee59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f79f0133420 0x7f79f0133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57330/0 (socket says 172.21.15.3:57330) 2023-10-03T10:15:51.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.524+0000 7f79ee59c700 1 -- 172.21.15.3:0/3976976965 learned_addr learned my addr 172.21.15.3:0/3976976965 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:51.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.524+0000 7f79edd9b700 1 --2- 172.21.15.3:0/3976976965 >> v2:172.21.15.3:3300/0 conn(0x7f79f01337f0 0x7f79f0138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:51.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.525+0000 7f79eed9d700 1 --2- 172.21.15.3:0/3976976965 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79f0138dc0 0x7f79f013b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:51.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.525+0000 7f79edd9b700 1 -- 172.21.15.3:0/3976976965 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79f0138dc0 msgr2=0x7f79f013b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:51.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.525+0000 7f79edd9b700 1 --2- 172.21.15.3:0/3976976965 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79f0138dc0 0x7f79f013b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.525+0000 7f79edd9b700 1 -- 172.21.15.3:0/3976976965 >> v1:172.21.15.3:6789/0 conn(0x7f79f0133420 legacy=0x7f79f0133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:51.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.525+0000 7f79edd9b700 1 -- 172.21.15.3:0/3976976965 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f00a0590 con 0x7f79f01337f0 2023-10-03T10:15:51.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.525+0000 7f79ed59a700 1 -- 172.21.15.3:0/3976976965 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1752676618 0 0) 0x7f79f006b990 con 0x7f79f0133420 2023-10-03T10:15:51.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.525+0000 7f79edd9b700 1 --2- 172.21.15.3:0/3976976965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79f01337f0 0x7f79f0138880 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f79e4009840 tx=0x7f79e4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e70f74bdb61c8c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:51.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.525+0000 7f79ed59a700 1 -- 172.21.15.3:0/3976976965 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e4011830 con 0x7f79f01337f0 2023-10-03T10:15:51.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.525+0000 7f79ed59a700 1 -- 172.21.15.3:0/3976976965 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e4011990 con 0x7f79f01337f0 2023-10-03T10:15:51.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.525+0000 7f79ed59a700 1 -- 172.21.15.3:0/3976976965 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e4011d20 con 0x7f79f01337f0 2023-10-03T10:15:51.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.526+0000 7f79f4910700 1 -- 172.21.15.3:0/3976976965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79f01337f0 msgr2=0x7f79f0138880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:51.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.526+0000 7f79f4910700 1 --2- 172.21.15.3:0/3976976965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79f01337f0 0x7f79f0138880 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f79e4009840 tx=0x7f79e4009b50 comp rx=0 tx=0).stop 2023-10-03T10:15:51.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.526+0000 7f79f4910700 1 -- 172.21.15.3:0/3976976965 shutdown_connections 2023-10-03T10:15:51.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.526+0000 7f79f4910700 1 --2- 172.21.15.3:0/3976976965 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79f0138dc0 0x7f79f013b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.526+0000 7f79f4910700 1 --2- 172.21.15.3:0/3976976965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79f01337f0 0x7f79f0138880 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.526+0000 7f79f4910700 1 -- 172.21.15.3:0/3976976965 >> 172.21.15.3:0/3976976965 conn(0x7f79f00a4210 msgr2=0x7f79f00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:51.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.526+0000 7f79f4910700 1 -- 172.21.15.3:0/3976976965 shutdown_connections 2023-10-03T10:15:51.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.526+0000 7f79f4910700 1 -- 172.21.15.3:0/3976976965 wait complete. 2023-10-03T10:15:51.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.527+0000 7f79f4910700 1 Processor -- start 2023-10-03T10:15:51.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.527+0000 7f79f4910700 1 -- start start 2023-10-03T10:15:51.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.528+0000 7f79f4910700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79f0133420 0x7f79f012ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:51.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.528+0000 7f79f4910700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79f01337f0 0x7f79f012f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:51.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.528+0000 7f79f4910700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79f0138dc0 0x7f79f012f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:51.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.528+0000 7f79f4910700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f009c920 con 0x7f79f01337f0 2023-10-03T10:15:51.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.528+0000 7f79f4910700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f009e400 con 0x7f79f0133420 2023-10-03T10:15:51.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.528+0000 7f79f4910700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79f01389c0 con 0x7f79f0138dc0 2023-10-03T10:15:51.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.528+0000 7f79eed9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79f0138dc0 0x7f79f012f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:51.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.528+0000 7f79eed9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79f0138dc0 0x7f79f012f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47150/0 (socket says 172.21.15.3:47150) 2023-10-03T10:15:51.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.528+0000 7f79eed9d700 1 -- 172.21.15.3:0/3506293843 learned_addr learned my addr 172.21.15.3:0/3506293843 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:51.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.528+0000 7f79ee59c700 1 --2- 172.21.15.3:0/3506293843 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79f0133420 0x7f79f012ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:51.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.528+0000 7f79eed9d700 1 -- 172.21.15.3:0/3506293843 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79f0133420 msgr2=0x7f79f012ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:51.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.528+0000 7f79eed9d700 1 --2- 172.21.15.3:0/3506293843 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79f0133420 0x7f79f012ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.529+0000 7f79eed9d700 1 -- 172.21.15.3:0/3506293843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79f01337f0 msgr2=0x7f79f012f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:51.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.528+0000 7f79edd9b700 1 --2- 172.21.15.3:0/3506293843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79f01337f0 0x7f79f012f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:51.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.529+0000 7f79eed9d700 1 --2- 172.21.15.3:0/3506293843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79f01337f0 0x7f79f012f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.529+0000 7f79eed9d700 1 -- 172.21.15.3:0/3506293843 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e40094a0 con 0x7f79f0138dc0 2023-10-03T10:15:51.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.529+0000 7f79edd9b700 1 --2- 172.21.15.3:0/3506293843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79f01337f0 0x7f79f012f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:15:51.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.529+0000 7f79eed9d700 1 --2- 172.21.15.3:0/3506293843 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79f0138dc0 0x7f79f012f8d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f79e8007e30 tx=0x7f79e800e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:51.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.529+0000 7f79deffd700 1 -- 172.21.15.3:0/3506293843 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e800eea0 con 0x7f79f0138dc0 2023-10-03T10:15:51.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.529+0000 7f79deffd700 1 -- 172.21.15.3:0/3506293843 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e8014420 con 0x7f79f0138dc0 2023-10-03T10:15:51.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.529+0000 7f79deffd700 1 -- 172.21.15.3:0/3506293843 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e80147d0 con 0x7f79f0138dc0 2023-10-03T10:15:51.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.529+0000 7f79f4910700 1 -- 172.21.15.3:0/3506293843 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f009beb0 con 0x7f79f0138dc0 2023-10-03T10:15:51.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.529+0000 7f79f4910700 1 -- 172.21.15.3:0/3506293843 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f009d390 con 0x7f79f0138dc0 2023-10-03T10:15:51.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.530+0000 7f79deffd700 1 -- 172.21.15.3:0/3506293843 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f79e8014930 con 0x7f79f0138dc0 2023-10-03T10:15:51.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.530+0000 7f79deffd700 1 -- 172.21.15.3:0/3506293843 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f79e803ff10 con 0x7f79f0138dc0 2023-10-03T10:15:51.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.531+0000 7f79f4910700 1 -- 172.21.15.3:0/3506293843 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f009c760 con 0x7f79f0138dc0 2023-10-03T10:15:51.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.540+0000 7f79deffd700 1 -- 172.21.15.3:0/3506293843 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f79e8014be0 con 0x7f79f0138dc0 2023-10-03T10:15:51.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.565+0000 7fd74eae2700 1 Processor -- start 2023-10-03T10:15:51.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.565+0000 7fd74eae2700 1 -- start start 2023-10-03T10:15:51.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.565+0000 7fd74eae2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd74809f410 0x7fd748136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:51.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.565+0000 7fd74eae2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd748136fe0 0x7fd7481393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:51.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.566+0000 7fd74eae2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd748070940 con 0x7fd74809e930 2023-10-03T10:15:51.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.566+0000 7fd74eae2700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd74806fd60 con 0x7fd748136fe0 2023-10-03T10:15:51.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.566+0000 7fd74eae2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd74806fa80 con 0x7fd74809f410 2023-10-03T10:15:51.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.566+0000 7fd747fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd74809f410 0x7fd748136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:51.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.566+0000 7fd74c87e700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd74809e930 0x7fd74809ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57346/0 (socket says 172.21.15.3:57346) 2023-10-03T10:15:51.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.566+0000 7fd747fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd74809f410 0x7fd748136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47158/0 (socket says 172.21.15.3:47158) 2023-10-03T10:15:51.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.566+0000 7fd74c87e700 1 -- 172.21.15.3:0/2105778735 learned_addr learned my addr 172.21.15.3:0/2105778735 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:51.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.566+0000 7fd74d07f700 1 --2- 172.21.15.3:0/2105778735 >> v2:172.21.15.3:3300/0 conn(0x7fd748136fe0 0x7fd7481393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:51.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.566+0000 7fd747fff700 1 -- 172.21.15.3:0/2105778735 >> v2:172.21.15.3:3300/0 conn(0x7fd748136fe0 msgr2=0x7fd7481393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:51.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.566+0000 7fd747fff700 1 --2- 172.21.15.3:0/2105778735 >> v2:172.21.15.3:3300/0 conn(0x7fd748136fe0 0x7fd7481393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.566+0000 7fd747fff700 1 -- 172.21.15.3:0/2105778735 >> v1:172.21.15.3:6789/0 conn(0x7fd74809e930 legacy=0x7fd74809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:51.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.566+0000 7fd747fff700 1 -- 172.21.15.3:0/2105778735 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74809fb60 con 0x7fd74809f410 2023-10-03T10:15:51.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.566+0000 7fd747fff700 1 --2- 172.21.15.3:0/2105778735 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd74809f410 0x7fd748136aa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd734009840 tx=0x7fd734009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6cad10bdf9cfe48 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:51.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.567+0000 7fd7477fe700 1 -- 172.21.15.3:0/2105778735 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd734011830 con 0x7fd74809f410 2023-10-03T10:15:51.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.567+0000 7fd7477fe700 1 -- 172.21.15.3:0/2105778735 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd734011990 con 0x7fd74809f410 2023-10-03T10:15:51.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.567+0000 7fd7477fe700 1 -- 172.21.15.3:0/2105778735 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd734011d20 con 0x7fd74809f410 2023-10-03T10:15:51.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.567+0000 7fd74eae2700 1 -- 172.21.15.3:0/2105778735 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd74809f410 msgr2=0x7fd748136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:51.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.567+0000 7fd74eae2700 1 --2- 172.21.15.3:0/2105778735 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd74809f410 0x7fd748136aa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd734009840 tx=0x7fd734009b50 comp rx=0 tx=0).stop 2023-10-03T10:15:51.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.567+0000 7fd74eae2700 1 -- 172.21.15.3:0/2105778735 shutdown_connections 2023-10-03T10:15:51.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.567+0000 7fd74eae2700 1 --2- 172.21.15.3:0/2105778735 >> v2:172.21.15.3:3300/0 conn(0x7fd748136fe0 0x7fd7481393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.567+0000 7fd74eae2700 1 --2- 172.21.15.3:0/2105778735 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd74809f410 0x7fd748136aa0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.567+0000 7fd74eae2700 1 -- 172.21.15.3:0/2105778735 >> 172.21.15.3:0/2105778735 conn(0x7fd7480a4210 msgr2=0x7fd7480a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:51.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.567+0000 7fd74eae2700 1 -- 172.21.15.3:0/2105778735 shutdown_connections 2023-10-03T10:15:51.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.568+0000 7fd74eae2700 1 -- 172.21.15.3:0/2105778735 wait complete. 2023-10-03T10:15:51.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.568+0000 7fd74eae2700 1 Processor -- start 2023-10-03T10:15:51.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.569+0000 7fd74eae2700 1 -- start start 2023-10-03T10:15:51.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.569+0000 7fd74eae2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd74809e930 0x7fd74806c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:51.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.569+0000 7fd74eae2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd74809f410 0x7fd74806c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:51.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.569+0000 7fd74eae2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd748136fe0 0x7fd7481c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:51.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.569+0000 7fd74eae2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74806bd70 con 0x7fd74809f410 2023-10-03T10:15:51.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.569+0000 7fd74eae2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd748122080 con 0x7fd748136fe0 2023-10-03T10:15:51.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.569+0000 7fd74eae2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd748132420 con 0x7fd74809e930 2023-10-03T10:15:51.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.569+0000 7fd74c87e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd74809e930 0x7fd74806c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:51.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.569+0000 7fd74c87e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd74809e930 0x7fd74806c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47162/0 (socket says 172.21.15.3:47162) 2023-10-03T10:15:51.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.569+0000 7fd74c87e700 1 -- 172.21.15.3:0/4241433262 learned_addr learned my addr 172.21.15.3:0/4241433262 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:51.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.569+0000 7fd74d07f700 1 --2- 172.21.15.3:0/4241433262 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd748136fe0 0x7fd7481c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:51.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.570+0000 7fd747fff700 1 --2- 172.21.15.3:0/4241433262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd74809f410 0x7fd74806c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:51.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.570+0000 7fd74c87e700 1 -- 172.21.15.3:0/4241433262 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd748136fe0 msgr2=0x7fd7481c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:51.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.570+0000 7fd74c87e700 1 --2- 172.21.15.3:0/4241433262 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd748136fe0 0x7fd7481c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.570+0000 7fd74c87e700 1 -- 172.21.15.3:0/4241433262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd74809f410 msgr2=0x7fd74806c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:51.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.570+0000 7fd74c87e700 1 --2- 172.21.15.3:0/4241433262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd74809f410 0x7fd74806c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.570+0000 7fd74c87e700 1 -- 172.21.15.3:0/4241433262 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7340094a0 con 0x7fd74809e930 2023-10-03T10:15:51.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.570+0000 7fd74c87e700 1 --2- 172.21.15.3:0/4241433262 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd74809e930 0x7fd74806c0e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd73c00c520 tx=0x7fd73c00c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:51.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.570+0000 7fd7457fa700 1 -- 172.21.15.3:0/4241433262 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73c005d70 con 0x7fd74809e930 2023-10-03T10:15:51.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.570+0000 7fd7457fa700 1 -- 172.21.15.3:0/4241433262 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73c0070d0 con 0x7fd74809e930 2023-10-03T10:15:51.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.570+0000 7fd74eae2700 1 -- 172.21.15.3:0/4241433262 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd748120c40 con 0x7fd74809e930 2023-10-03T10:15:51.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.570+0000 7fd7457fa700 1 -- 172.21.15.3:0/4241433262 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73c007440 con 0x7fd74809e930 2023-10-03T10:15:51.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.570+0000 7fd74eae2700 1 -- 172.21.15.3:0/4241433262 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd748122af0 con 0x7fd74809e930 2023-10-03T10:15:51.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.571+0000 7fd74eae2700 1 -- 172.21.15.3:0/4241433262 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd748074ea0 con 0x7fd74809e930 2023-10-03T10:15:51.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.572+0000 7fd7457fa700 1 -- 172.21.15.3:0/4241433262 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd73c0075a0 con 0x7fd74809e930 2023-10-03T10:15:51.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.572+0000 7fd7457fa700 1 -- 172.21.15.3:0/4241433262 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd73c040bb0 con 0x7fd74809e930 2023-10-03T10:15:51.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.577+0000 7fd7457fa700 1 -- 172.21.15.3:0/4241433262 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd73c044060 con 0x7fd74809e930 2023-10-03T10:15:51.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.674+0000 7ff789675700 1 -- 172.21.15.3:0/1489579598 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff78406c360 con 0x7ff78409d6f0 2023-10-03T10:15:51.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.674+0000 7ff76b7fe700 1 -- 172.21.15.3:0/1489579598 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff77400eb50 con 0x7ff78409d6f0 2023-10-03T10:15:51.675 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:51.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.677+0000 7ff789675700 1 -- 172.21.15.3:0/1489579598 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78409d6f0 msgr2=0x7ff784122230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:51.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.677+0000 7ff789675700 1 --2- 172.21.15.3:0/1489579598 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78409d6f0 0x7ff784122230 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff77400d9f0 tx=0x7ff77400dd00 comp rx=0 tx=0).stop 2023-10-03T10:15:51.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.677+0000 7ff789675700 1 -- 172.21.15.3:0/1489579598 shutdown_connections 2023-10-03T10:15:51.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.677+0000 7ff789675700 1 --2- 172.21.15.3:0/1489579598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78409e410 0x7ff784122cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.677+0000 7ff789675700 1 --2- 172.21.15.3:0/1489579598 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78409d6f0 0x7ff784122230 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.677+0000 7ff789675700 1 --2- 172.21.15.3:0/1489579598 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78409e000 0x7ff784122770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.677+0000 7ff789675700 1 -- 172.21.15.3:0/1489579598 >> 172.21.15.3:0/1489579598 conn(0x7ff784066f50 msgr2=0x7ff784130890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:51.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.677+0000 7ff789675700 1 -- 172.21.15.3:0/1489579598 shutdown_connections 2023-10-03T10:15:51.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.677+0000 7ff789675700 1 -- 172.21.15.3:0/1489579598 wait complete. 2023-10-03T10:15:51.691 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:51.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.706+0000 7ffa196ef700 1 -- 172.21.15.3:0/1868577211 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ffa14074ea0 con 0x7ffa1409e000 2023-10-03T10:15:51.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.706+0000 7ff9fb7fe700 1 -- 172.21.15.3:0/1868577211 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffa08026070 con 0x7ffa1409e000 2023-10-03T10:15:51.707 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:51.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.709+0000 7ffa196ef700 1 -- 172.21.15.3:0/1868577211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa1409e000 msgr2=0x7ffa1411e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:51.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.709+0000 7ffa196ef700 1 --2- 172.21.15.3:0/1868577211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa1409e000 0x7ffa1411e4c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ffa1406d540 tx=0x7ffa08004750 comp rx=0 tx=0).stop 2023-10-03T10:15:51.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.709+0000 7ffa196ef700 1 -- 172.21.15.3:0/1868577211 shutdown_connections 2023-10-03T10:15:51.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.709+0000 7ffa196ef700 1 --2- 172.21.15.3:0/1868577211 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffa1409e410 0x7ffa1411ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.709+0000 7ffa196ef700 1 --2- 172.21.15.3:0/1868577211 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa1409d6f0 0x7ffa1411df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.709+0000 7ffa196ef700 1 --2- 172.21.15.3:0/1868577211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa1409e000 0x7ffa1411e4c0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.709+0000 7ffa196ef700 1 -- 172.21.15.3:0/1868577211 >> 172.21.15.3:0/1868577211 conn(0x7ffa14066f50 msgr2=0x7ffa140710c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:51.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.709+0000 7ffa196ef700 1 -- 172.21.15.3:0/1868577211 shutdown_connections 2023-10-03T10:15:51.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.709+0000 7ffa196ef700 1 -- 172.21.15.3:0/1868577211 wait complete. 2023-10-03T10:15:51.723 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:51.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.756+0000 7f79f4910700 1 -- 172.21.15.3:0/3506293843 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f79f0074ea0 con 0x7f79f0138dc0 2023-10-03T10:15:51.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.756+0000 7f79deffd700 1 -- 172.21.15.3:0/3506293843 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f79e801bb00 con 0x7f79f0138dc0 2023-10-03T10:15:51.757 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:51.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.759+0000 7f79f4910700 1 -- 172.21.15.3:0/3506293843 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79f0138dc0 msgr2=0x7f79f012f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:51.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.759+0000 7f79f4910700 1 --2- 172.21.15.3:0/3506293843 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79f0138dc0 0x7f79f012f8d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f79e8007e30 tx=0x7f79e800e3c0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.759+0000 7f79f4910700 1 -- 172.21.15.3:0/3506293843 shutdown_connections 2023-10-03T10:15:51.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.759+0000 7f79f4910700 1 --2- 172.21.15.3:0/3506293843 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79f0138dc0 0x7f79f012f8d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.759+0000 7f79f4910700 1 --2- 172.21.15.3:0/3506293843 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79f0133420 0x7f79f012ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.759+0000 7f79f4910700 1 --2- 172.21.15.3:0/3506293843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79f01337f0 0x7f79f012f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.759+0000 7f79f4910700 1 -- 172.21.15.3:0/3506293843 >> 172.21.15.3:0/3506293843 conn(0x7f79f00a4210 msgr2=0x7f79f00a0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:51.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.759+0000 7f79f4910700 1 -- 172.21.15.3:0/3506293843 shutdown_connections 2023-10-03T10:15:51.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.759+0000 7f79f4910700 1 -- 172.21.15.3:0/3506293843 wait complete. 2023-10-03T10:15:51.773 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:51.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.787+0000 7fd74eae2700 1 -- 172.21.15.3:0/4241433262 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd74806b250 con 0x7fd74809e930 2023-10-03T10:15:51.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.787+0000 7fd7457fa700 1 -- 172.21.15.3:0/4241433262 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd73c044020 con 0x7fd74809e930 2023-10-03T10:15:51.787 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:51.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.789+0000 7fd74eae2700 1 -- 172.21.15.3:0/4241433262 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd74809e930 msgr2=0x7fd74806c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:51.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.789+0000 7fd74eae2700 1 --2- 172.21.15.3:0/4241433262 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd74809e930 0x7fd74806c0e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd73c00c520 tx=0x7fd73c00c830 comp rx=0 tx=0).stop 2023-10-03T10:15:51.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.789+0000 7fd74eae2700 1 -- 172.21.15.3:0/4241433262 shutdown_connections 2023-10-03T10:15:51.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.789+0000 7fd74eae2700 1 --2- 172.21.15.3:0/4241433262 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd748136fe0 0x7fd7481c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.789+0000 7fd74eae2700 1 --2- 172.21.15.3:0/4241433262 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd74809e930 0x7fd74806c0e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.789+0000 7fd74eae2700 1 --2- 172.21.15.3:0/4241433262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd74809f410 0x7fd74806c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:51.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.789+0000 7fd74eae2700 1 -- 172.21.15.3:0/4241433262 >> 172.21.15.3:0/4241433262 conn(0x7fd7480a4210 msgr2=0x7fd7480a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:51.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.790+0000 7fd74eae2700 1 -- 172.21.15.3:0/4241433262 shutdown_connections 2023-10-03T10:15:51.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:51.790+0000 7fd74eae2700 1 -- 172.21.15.3:0/4241433262 wait complete. 2023-10-03T10:15:51.803 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:52.693 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:52.723 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:52.774 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:52.803 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:52.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.844+0000 7f28597a3700 1 Processor -- start 2023-10-03T10:15:52.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.845+0000 7f28597a3700 1 -- start start 2023-10-03T10:15:52.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.845+0000 7f28597a3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f285412cf60 0x7f285406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:52.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.845+0000 7f28597a3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2854130430 0x7f2854137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:52.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.845+0000 7f28597a3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f285412cd90 con 0x7f285412f8b0 2023-10-03T10:15:52.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.845+0000 7f28597a3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f285412d330 con 0x7f2854130430 2023-10-03T10:15:52.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.845+0000 7f28597a3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2854071780 con 0x7f285412cf60 2023-10-03T10:15:52.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.845+0000 7f28537fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2854130430 0x7f2854137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:52.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.845+0000 7f28537fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2854130430 0x7f2854137c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55806/0 (socket says 172.21.15.3:55806) 2023-10-03T10:15:52.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.845+0000 7f28537fe700 1 -- 172.21.15.3:0/1888849637 learned_addr learned my addr 172.21.15.3:0/1888849637 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:52.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.845+0000 7f2852ffd700 1 --2- 172.21.15.3:0/1888849637 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f285412cf60 0x7f285406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:52.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.846+0000 7f2852ffd700 1 -- 172.21.15.3:0/1888849637 >> v2:172.21.15.3:3300/0 conn(0x7f2854130430 msgr2=0x7f2854137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:52.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.846+0000 7f2852ffd700 1 --2- 172.21.15.3:0/1888849637 >> v2:172.21.15.3:3300/0 conn(0x7f2854130430 0x7f2854137c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:52.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.846+0000 7f2852ffd700 1 -- 172.21.15.3:0/1888849637 >> v1:172.21.15.3:6789/0 conn(0x7f285412f8b0 legacy=0x7f285412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:52.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.846+0000 7f2851ffb700 1 -- 172.21.15.3:0/1888849637 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2662035953 0 0) 0x7f285412cd90 con 0x7f285412f8b0 2023-10-03T10:15:52.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.846+0000 7f2852ffd700 1 -- 172.21.15.3:0/1888849637 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2854071a40 con 0x7f285412cf60 2023-10-03T10:15:52.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.846+0000 7f2852ffd700 1 --2- 172.21.15.3:0/1888849637 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f285412cf60 0x7f285406f5d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2844009840 tx=0x7f2844009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f106d5262b23975 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:52.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.846+0000 7f2851ffb700 1 -- 172.21.15.3:0/1888849637 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2844011830 con 0x7f285412cf60 2023-10-03T10:15:52.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.847+0000 7f2851ffb700 1 -- 172.21.15.3:0/1888849637 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2844011990 con 0x7f285412cf60 2023-10-03T10:15:52.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.847+0000 7f2851ffb700 1 -- 172.21.15.3:0/1888849637 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2844011d20 con 0x7f285412cf60 2023-10-03T10:15:52.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.847+0000 7f28597a3700 1 -- 172.21.15.3:0/1888849637 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f285412cf60 msgr2=0x7f285406f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:52.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.847+0000 7f28597a3700 1 --2- 172.21.15.3:0/1888849637 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f285412cf60 0x7f285406f5d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2844009840 tx=0x7f2844009b50 comp rx=0 tx=0).stop 2023-10-03T10:15:52.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.847+0000 7f28597a3700 1 -- 172.21.15.3:0/1888849637 shutdown_connections 2023-10-03T10:15:52.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.847+0000 7f28597a3700 1 --2- 172.21.15.3:0/1888849637 >> v2:172.21.15.3:3300/0 conn(0x7f2854130430 0x7f2854137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:52.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.847+0000 7f28597a3700 1 --2- 172.21.15.3:0/1888849637 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f285412cf60 0x7f285406f5d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:52.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.847+0000 7f28597a3700 1 -- 172.21.15.3:0/1888849637 >> 172.21.15.3:0/1888849637 conn(0x7f28540a2aa0 msgr2=0x7f28540a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:52.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.848+0000 7f28597a3700 1 -- 172.21.15.3:0/1888849637 shutdown_connections 2023-10-03T10:15:52.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.848+0000 7f28597a3700 1 -- 172.21.15.3:0/1888849637 wait complete. 2023-10-03T10:15:52.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.849+0000 7f28597a3700 1 Processor -- start 2023-10-03T10:15:52.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.849+0000 7f28597a3700 1 -- start start 2023-10-03T10:15:52.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.849+0000 7f28597a3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285412cf60 0x7f28541c4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:52.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.849+0000 7f28597a3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285412f8b0 0x7f28541c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:52.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.849+0000 7f28597a3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2854130430 0x7f285409b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:52.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.849+0000 7f28597a3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285409e940 con 0x7f285412cf60 2023-10-03T10:15:52.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.849+0000 7f28597a3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2854129950 con 0x7f285412f8b0 2023-10-03T10:15:52.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.849+0000 7f28597a3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2854129fa0 con 0x7f2854130430 2023-10-03T10:15:52.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.850+0000 7f2852ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285412cf60 0x7f28541c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:52.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.850+0000 7f28527fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285412f8b0 0x7f28541c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:52.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.850+0000 7f2852ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285412cf60 0x7f28541c4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55818/0 (socket says 172.21.15.3:55818) 2023-10-03T10:15:52.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.850+0000 7f28527fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285412f8b0 0x7f28541c48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:34988/0 (socket says 172.21.15.3:34988) 2023-10-03T10:15:52.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.850+0000 7f2852ffd700 1 -- 172.21.15.3:0/2168367118 learned_addr learned my addr 172.21.15.3:0/2168367118 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:52.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.850+0000 7f28537fe700 1 --2- 172.21.15.3:0/2168367118 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2854130430 0x7f285409b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:52.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.850+0000 7f28527fc700 1 -- 172.21.15.3:0/2168367118 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2854130430 msgr2=0x7f285409b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:52.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.850+0000 7f28527fc700 1 --2- 172.21.15.3:0/2168367118 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2854130430 0x7f285409b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:52.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.850+0000 7f28527fc700 1 -- 172.21.15.3:0/2168367118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285412cf60 msgr2=0x7f28541c4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:52.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.850+0000 7f28527fc700 1 --2- 172.21.15.3:0/2168367118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285412cf60 0x7f28541c4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:52.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.850+0000 7f28527fc700 1 -- 172.21.15.3:0/2168367118 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28440094a0 con 0x7f285412f8b0 2023-10-03T10:15:52.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.850+0000 7f2852ffd700 1 --2- 172.21.15.3:0/2168367118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285412cf60 0x7f28541c4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:52.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.850+0000 7f28537fe700 1 --2- 172.21.15.3:0/2168367118 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2854130430 0x7f285409b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:15:52.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.850+0000 7f28527fc700 1 --2- 172.21.15.3:0/2168367118 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285412f8b0 0x7f28541c48b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f28480018a0 tx=0x7f2848001bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:52.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.851+0000 7f283b7fe700 1 -- 172.21.15.3:0/2168367118 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284800e370 con 0x7f285412f8b0 2023-10-03T10:15:52.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.851+0000 7f283b7fe700 1 -- 172.21.15.3:0/2168367118 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28480050d0 con 0x7f285412f8b0 2023-10-03T10:15:52.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.851+0000 7f28597a3700 1 -- 172.21.15.3:0/2168367118 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f285412da40 con 0x7f285412f8b0 2023-10-03T10:15:52.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.851+0000 7f283b7fe700 1 -- 172.21.15.3:0/2168367118 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2848005480 con 0x7f285412f8b0 2023-10-03T10:15:52.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.851+0000 7f28597a3700 1 -- 172.21.15.3:0/2168367118 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f285409d0f0 con 0x7f285412f8b0 2023-10-03T10:15:52.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.852+0000 7f283b7fe700 1 -- 172.21.15.3:0/2168367118 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2848003680 con 0x7f285412f8b0 2023-10-03T10:15:52.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.852+0000 7f283b7fe700 1 -- 172.21.15.3:0/2168367118 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f284803eee0 con 0x7f285412f8b0 2023-10-03T10:15:52.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.852+0000 7f28597a3700 1 -- 172.21.15.3:0/2168367118 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2854074ea0 con 0x7f285412f8b0 2023-10-03T10:15:52.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.859+0000 7f283b7fe700 1 -- 172.21.15.3:0/2168367118 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2848003930 con 0x7f285412f8b0 2023-10-03T10:15:52.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.895+0000 7fca449ba700 1 Processor -- start 2023-10-03T10:15:52.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.895+0000 7fca449ba700 1 -- start start 2023-10-03T10:15:52.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.896+0000 7fca449ba700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca4006c0e0 0x7fca4009ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:52.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.897+0000 7fca449ba700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fca401380f0 0x7fca4013fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:52.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.897+0000 7fca449ba700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca40134980 con 0x7fca40134ec0 2023-10-03T10:15:52.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.897+0000 7fca449ba700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca4006c4b0 con 0x7fca401380f0 2023-10-03T10:15:52.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.897+0000 7fca449ba700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca400a09c0 con 0x7fca4006c0e0 2023-10-03T10:15:52.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.897+0000 7fca3e59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca4006c0e0 0x7fca4009ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:52.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.897+0000 7fca3e59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca4006c0e0 0x7fca4009ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:35002/0 (socket says 172.21.15.3:35002) 2023-10-03T10:15:52.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.897+0000 7fca3e59c700 1 -- 172.21.15.3:0/3870269793 learned_addr learned my addr 172.21.15.3:0/3870269793 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:52.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.897+0000 7fca3ed9d700 1 --2- 172.21.15.3:0/3870269793 >> v2:172.21.15.3:3300/0 conn(0x7fca401380f0 0x7fca4013fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:52.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.897+0000 7fca3e59c700 1 -- 172.21.15.3:0/3870269793 >> v2:172.21.15.3:3300/0 conn(0x7fca401380f0 msgr2=0x7fca4013fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:52.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.897+0000 7fca3e59c700 1 --2- 172.21.15.3:0/3870269793 >> v2:172.21.15.3:3300/0 conn(0x7fca401380f0 0x7fca4013fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:52.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.897+0000 7fca3e59c700 1 -- 172.21.15.3:0/3870269793 >> v1:172.21.15.3:6789/0 conn(0x7fca40134ec0 legacy=0x7fca401379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:52.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.897+0000 7fca3e59c700 1 -- 172.21.15.3:0/3870269793 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca400a0600 con 0x7fca4006c0e0 2023-10-03T10:15:52.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.897+0000 7fca3e59c700 1 --2- 172.21.15.3:0/3870269793 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca4006c0e0 0x7fca4009ac90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fca300098f0 tx=0x7fca30009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ca657c4a700f60d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:52.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.898+0000 7fca3d59a700 1 -- 172.21.15.3:0/3870269793 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca30011830 con 0x7fca4006c0e0 2023-10-03T10:15:52.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.898+0000 7fca3d59a700 1 -- 172.21.15.3:0/3870269793 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca30011990 con 0x7fca4006c0e0 2023-10-03T10:15:52.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.898+0000 7fca3d59a700 1 -- 172.21.15.3:0/3870269793 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca30011d20 con 0x7fca4006c0e0 2023-10-03T10:15:52.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.899+0000 7fca449ba700 1 -- 172.21.15.3:0/3870269793 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca4006c0e0 msgr2=0x7fca4009ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:52.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.899+0000 7fca449ba700 1 --2- 172.21.15.3:0/3870269793 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca4006c0e0 0x7fca4009ac90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fca300098f0 tx=0x7fca30009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:52.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.899+0000 7fca449ba700 1 -- 172.21.15.3:0/3870269793 shutdown_connections 2023-10-03T10:15:52.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.899+0000 7fca449ba700 1 --2- 172.21.15.3:0/3870269793 >> v2:172.21.15.3:3300/0 conn(0x7fca401380f0 0x7fca4013fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:52.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.899+0000 7fca449ba700 1 --2- 172.21.15.3:0/3870269793 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca4006c0e0 0x7fca4009ac90 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:52.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.899+0000 7fca449ba700 1 -- 172.21.15.3:0/3870269793 >> 172.21.15.3:0/3870269793 conn(0x7fca400a4210 msgr2=0x7fca400a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:52.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.899+0000 7fca449ba700 1 -- 172.21.15.3:0/3870269793 shutdown_connections 2023-10-03T10:15:52.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.899+0000 7fca449ba700 1 -- 172.21.15.3:0/3870269793 wait complete. 2023-10-03T10:15:52.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.900+0000 7fca449ba700 1 Processor -- start 2023-10-03T10:15:52.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.900+0000 7fca449ba700 1 -- start start 2023-10-03T10:15:52.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.901+0000 7fca449ba700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca4006c0e0 0x7fca401ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:52.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.901+0000 7fca449ba700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca40134ec0 0x7fca401cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:52.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.901+0000 7fca449ba700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca401380f0 0x7fca4011a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:52.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.901+0000 7fca449ba700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca40134470 con 0x7fca4006c0e0 2023-10-03T10:15:52.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.901+0000 7fca449ba700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca40133f60 con 0x7fca40134ec0 2023-10-03T10:15:52.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.901+0000 7fca449ba700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca400a03f0 con 0x7fca401380f0 2023-10-03T10:15:52.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.901+0000 7fca3e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca4006c0e0 0x7fca401ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:52.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.901+0000 7fca3e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca4006c0e0 0x7fca401ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55834/0 (socket says 172.21.15.3:55834) 2023-10-03T10:15:52.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.901+0000 7fca3e59c700 1 -- 172.21.15.3:0/3408893475 learned_addr learned my addr 172.21.15.3:0/3408893475 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:52.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.901+0000 7fca3ed9d700 1 --2- 172.21.15.3:0/3408893475 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca401380f0 0x7fca4011a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:52.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.901+0000 7fca3dd9b700 1 --2- 172.21.15.3:0/3408893475 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca40134ec0 0x7fca401cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:52.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.901+0000 7fca3e59c700 1 -- 172.21.15.3:0/3408893475 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca401380f0 msgr2=0x7fca4011a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:52.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.901+0000 7fca3e59c700 1 --2- 172.21.15.3:0/3408893475 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca401380f0 0x7fca4011a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:52.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.901+0000 7fca3e59c700 1 -- 172.21.15.3:0/3408893475 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca40134ec0 msgr2=0x7fca401cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:52.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.901+0000 7fca3e59c700 1 --2- 172.21.15.3:0/3408893475 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca40134ec0 0x7fca401cd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:52.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.901+0000 7fca3e59c700 1 -- 172.21.15.3:0/3408893475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca30009550 con 0x7fca4006c0e0 2023-10-03T10:15:52.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.902+0000 7fca3e59c700 1 --2- 172.21.15.3:0/3408893475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca4006c0e0 0x7fca401ccd50 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fca30009c40 tx=0x7fca30004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:52.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.902+0000 7fca2effd700 1 -- 172.21.15.3:0/3408893475 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca30004590 con 0x7fca4006c0e0 2023-10-03T10:15:52.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.902+0000 7fca2effd700 1 -- 172.21.15.3:0/3408893475 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3001e070 con 0x7fca4006c0e0 2023-10-03T10:15:52.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.902+0000 7fca2effd700 1 -- 172.21.15.3:0/3408893475 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca300038e0 con 0x7fca4006c0e0 2023-10-03T10:15:52.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.903+0000 7fca449ba700 1 -- 172.21.15.3:0/3408893475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca4011c540 con 0x7fca4006c0e0 2023-10-03T10:15:52.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.903+0000 7fca449ba700 1 -- 172.21.15.3:0/3408893475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4011e550 con 0x7fca4006c0e0 2023-10-03T10:15:52.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.904+0000 7fca449ba700 1 -- 172.21.15.3:0/3408893475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca40074ea0 con 0x7fca4006c0e0 2023-10-03T10:15:52.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.904+0000 7fca2effd700 1 -- 172.21.15.3:0/3408893475 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fca30003a40 con 0x7fca4006c0e0 2023-10-03T10:15:52.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.906+0000 7fca2effd700 1 -- 172.21.15.3:0/3408893475 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fca30020070 con 0x7fca4006c0e0 2023-10-03T10:15:52.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.913+0000 7fca2effd700 1 -- 172.21.15.3:0/3408893475 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fca30033340 con 0x7fca4006c0e0 2023-10-03T10:15:52.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.929+0000 7fc0e1696700 1 Processor -- start 2023-10-03T10:15:52.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.929+0000 7fc0e1696700 1 -- start start 2023-10-03T10:15:52.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.930+0000 7fc0e1696700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc0dc09d6f0 0x7fc0dc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:52.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.930+0000 7fc0e1696700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0dc09e410 0x7fc0dc130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:52.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.930+0000 7fc0e1696700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0dc09f130 con 0x7fc0dc09e000 2023-10-03T10:15:52.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.930+0000 7fc0e1696700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0dc09f520 con 0x7fc0dc09d6f0 2023-10-03T10:15:52.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.930+0000 7fc0e1696700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc09e930 con 0x7fc0dc09e410 2023-10-03T10:15:52.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.930+0000 7fc0db7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0dc09e410 0x7fc0dc130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:52.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.930+0000 7fc0db7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0dc09e410 0x7fc0dc130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47182/0 (socket says 172.21.15.3:47182) 2023-10-03T10:15:52.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.930+0000 7fc0db7fe700 1 -- 172.21.15.3:0/2781910163 learned_addr learned my addr 172.21.15.3:0/2781910163 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:52.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.930+0000 7fc0daffd700 1 --2- 172.21.15.3:0/2781910163 >> v2:172.21.15.3:3300/0 conn(0x7fc0dc09d6f0 0x7fc0dc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:52.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.930+0000 7fc0db7fe700 1 -- 172.21.15.3:0/2781910163 >> v2:172.21.15.3:3300/0 conn(0x7fc0dc09d6f0 msgr2=0x7fc0dc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:52.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.930+0000 7fc0db7fe700 1 --2- 172.21.15.3:0/2781910163 >> v2:172.21.15.3:3300/0 conn(0x7fc0dc09d6f0 0x7fc0dc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:52.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.930+0000 7fc0db7fe700 1 -- 172.21.15.3:0/2781910163 >> v1:172.21.15.3:6789/0 conn(0x7fc0dc09e000 legacy=0x7fc0dc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:52.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.930+0000 7fc0db7fe700 1 -- 172.21.15.3:0/2781910163 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0dc09b2d0 con 0x7fc0dc09e410 2023-10-03T10:15:52.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.931+0000 7fc0db7fe700 1 --2- 172.21.15.3:0/2781910163 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0dc09e410 0x7fc0dc130d70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4009e50 tx=0x7fc0d400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd9840fd3748580a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:52.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.931+0000 7fc0d9ffb700 1 -- 172.21.15.3:0/2781910163 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d400e070 con 0x7fc0dc09e410 2023-10-03T10:15:52.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.931+0000 7fc0d9ffb700 1 -- 172.21.15.3:0/2781910163 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d400b910 con 0x7fc0dc09e410 2023-10-03T10:15:52.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.931+0000 7fc0d9ffb700 1 -- 172.21.15.3:0/2781910163 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d400bcc0 con 0x7fc0dc09e410 2023-10-03T10:15:52.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.932+0000 7fc0e1696700 1 -- 172.21.15.3:0/2781910163 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0dc09e410 msgr2=0x7fc0dc130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:52.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.932+0000 7fc0e1696700 1 --2- 172.21.15.3:0/2781910163 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0dc09e410 0x7fc0dc130d70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4009e50 tx=0x7fc0d400b330 comp rx=0 tx=0).stop 2023-10-03T10:15:52.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.932+0000 7fc0e1696700 1 -- 172.21.15.3:0/2781910163 shutdown_connections 2023-10-03T10:15:52.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.932+0000 7fc0e1696700 1 --2- 172.21.15.3:0/2781910163 >> v2:172.21.15.3:3300/0 conn(0x7fc0dc09d6f0 0x7fc0dc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:52.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.932+0000 7fc0e1696700 1 --2- 172.21.15.3:0/2781910163 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0dc09e410 0x7fc0dc130d70 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:52.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.932+0000 7fc0e1696700 1 -- 172.21.15.3:0/2781910163 >> 172.21.15.3:0/2781910163 conn(0x7fc0dc066f50 msgr2=0x7fc0dc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:52.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.932+0000 7fc0e1696700 1 -- 172.21.15.3:0/2781910163 shutdown_connections 2023-10-03T10:15:52.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.932+0000 7fc0e1696700 1 -- 172.21.15.3:0/2781910163 wait complete. 2023-10-03T10:15:52.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.934+0000 7fc0e1696700 1 Processor -- start 2023-10-03T10:15:52.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.934+0000 7fc0e1696700 1 -- start start 2023-10-03T10:15:52.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.935+0000 7fc0e1696700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0dc09d6f0 0x7fc0dc12fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:52.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.935+0000 7fc0e1696700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc0dc09e000 0x7fc0dc1c8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:52.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.935+0000 7fc0e1696700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0dc09e410 0x7fc0dc1c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:52.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.935+0000 7fc0e1696700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc131040 con 0x7fc0dc09e410 2023-10-03T10:15:52.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.935+0000 7fc0e1696700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc132af0 con 0x7fc0dc09e000 2023-10-03T10:15:52.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.935+0000 7fc0e1696700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc0721e0 con 0x7fc0dc09d6f0 2023-10-03T10:15:52.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.935+0000 7fc0daffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0dc09d6f0 0x7fc0dc12fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:52.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.935+0000 7fc0daffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0dc09d6f0 0x7fc0dc12fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47188/0 (socket says 172.21.15.3:47188) 2023-10-03T10:15:52.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.935+0000 7fc0daffd700 1 -- 172.21.15.3:0/1641614756 learned_addr learned my addr 172.21.15.3:0/1641614756 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:52.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.935+0000 7fc0da7fc700 1 --2- 172.21.15.3:0/1641614756 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc0dc09e000 0x7fc0dc1c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:52.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.935+0000 7fc0db7fe700 1 --2- 172.21.15.3:0/1641614756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0dc09e410 0x7fc0dc1c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:52.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.935+0000 7fc0daffd700 1 -- 172.21.15.3:0/1641614756 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc0dc09e000 msgr2=0x7fc0dc1c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:52.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.935+0000 7fc0daffd700 1 --2- 172.21.15.3:0/1641614756 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc0dc09e000 0x7fc0dc1c8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:52.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.935+0000 7fc0daffd700 1 -- 172.21.15.3:0/1641614756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0dc09e410 msgr2=0x7fc0dc1c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:52.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.935+0000 7fc0daffd700 1 --2- 172.21.15.3:0/1641614756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0dc09e410 0x7fc0dc1c8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:52.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.935+0000 7fc0daffd700 1 -- 172.21.15.3:0/1641614756 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d4009b30 con 0x7fc0dc09d6f0 2023-10-03T10:15:52.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.936+0000 7fc0db7fe700 1 --2- 172.21.15.3:0/1641614756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0dc09e410 0x7fc0dc1c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:52.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.936+0000 7fc0daffd700 1 --2- 172.21.15.3:0/1641614756 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0dc09d6f0 0x7fc0dc12fa50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc00e9e0 tx=0x7fc0cc00ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:52.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.936+0000 7fc0c37fe700 1 -- 172.21.15.3:0/1641614756 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc009710 con 0x7fc0dc09d6f0 2023-10-03T10:15:52.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.936+0000 7fc0c37fe700 1 -- 172.21.15.3:0/1641614756 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0cc00ee80 con 0x7fc0dc09d6f0 2023-10-03T10:15:52.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.936+0000 7fc0c37fe700 1 -- 172.21.15.3:0/1641614756 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc00ca50 con 0x7fc0dc09d6f0 2023-10-03T10:15:52.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.936+0000 7fc0e1696700 1 -- 172.21.15.3:0/1641614756 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0dc132350 con 0x7fc0dc09d6f0 2023-10-03T10:15:52.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.936+0000 7fc0e1696700 1 -- 172.21.15.3:0/1641614756 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0dc130650 con 0x7fc0dc09d6f0 2023-10-03T10:15:52.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.937+0000 7fc0c37fe700 1 -- 172.21.15.3:0/1641614756 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc0cc00cbb0 con 0x7fc0dc09d6f0 2023-10-03T10:15:52.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.937+0000 7fc0e1696700 1 -- 172.21.15.3:0/1641614756 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0dc09c760 con 0x7fc0dc09d6f0 2023-10-03T10:15:52.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.937+0000 7fc0c37fe700 1 -- 172.21.15.3:0/1641614756 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc0cc03f760 con 0x7fc0dc09d6f0 2023-10-03T10:15:52.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.947+0000 7fc0c37fe700 1 -- 172.21.15.3:0/1641614756 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc0cc0232f0 con 0x7fc0dc09d6f0 2023-10-03T10:15:52.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.987+0000 7fdc1d3a3700 1 Processor -- start 2023-10-03T10:15:52.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.987+0000 7fdc1d3a3700 1 -- start start 2023-10-03T10:15:52.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.987+0000 7fdc1d3a3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdc1809e1d0 0x7fdc18135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:52.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.987+0000 7fdc1d3a3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc18135e50 0x7fdc18138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:52.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.987+0000 7fdc1d3a3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc1809f130 con 0x7fdc1809d6f0 2023-10-03T10:15:52.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.987+0000 7fdc1d3a3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1809f520 con 0x7fdc1809e1d0 2023-10-03T10:15:52.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.987+0000 7fdc1d3a3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1809e930 con 0x7fdc18135e50 2023-10-03T10:15:52.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.987+0000 7fdc177fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc18135e50 0x7fdc18138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:52.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.987+0000 7fdc177fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc18135e50 0x7fdc18138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:35036/0 (socket says 172.21.15.3:35036) 2023-10-03T10:15:52.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.987+0000 7fdc177fe700 1 -- 172.21.15.3:0/807184720 learned_addr learned my addr 172.21.15.3:0/807184720 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:52.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.987+0000 7fdc167fc700 1 --2- 172.21.15.3:0/807184720 >> v2:172.21.15.3:3300/0 conn(0x7fdc1809e1d0 0x7fdc18135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:52.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.988+0000 7fdc177fe700 1 -- 172.21.15.3:0/807184720 >> v2:172.21.15.3:3300/0 conn(0x7fdc1809e1d0 msgr2=0x7fdc18135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:52.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.988+0000 7fdc177fe700 1 --2- 172.21.15.3:0/807184720 >> v2:172.21.15.3:3300/0 conn(0x7fdc1809e1d0 0x7fdc18135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:52.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.988+0000 7fdc177fe700 1 -- 172.21.15.3:0/807184720 >> v1:172.21.15.3:6789/0 conn(0x7fdc1809d6f0 legacy=0x7fdc1809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:52.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.988+0000 7fdc177fe700 1 -- 172.21.15.3:0/807184720 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1809b2d0 con 0x7fdc18135e50 2023-10-03T10:15:52.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.988+0000 7fdc177fe700 1 --2- 172.21.15.3:0/807184720 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc18135e50 0x7fdc18138230 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000d150 tx=0x7fdc1000d460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab28f5c16f9bcb31 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:52.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.988+0000 7fdc15ffb700 1 -- 172.21.15.3:0/807184720 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc10010070 con 0x7fdc18135e50 2023-10-03T10:15:52.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.988+0000 7fdc15ffb700 1 -- 172.21.15.3:0/807184720 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1000dd10 con 0x7fdc18135e50 2023-10-03T10:15:52.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.988+0000 7fdc15ffb700 1 -- 172.21.15.3:0/807184720 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1000b310 con 0x7fdc18135e50 2023-10-03T10:15:52.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.989+0000 7fdc1d3a3700 1 -- 172.21.15.3:0/807184720 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc18135e50 msgr2=0x7fdc18138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:52.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.989+0000 7fdc1d3a3700 1 --2- 172.21.15.3:0/807184720 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc18135e50 0x7fdc18138230 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000d150 tx=0x7fdc1000d460 comp rx=0 tx=0).stop 2023-10-03T10:15:52.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.989+0000 7fdc1d3a3700 1 -- 172.21.15.3:0/807184720 shutdown_connections 2023-10-03T10:15:52.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.989+0000 7fdc1d3a3700 1 --2- 172.21.15.3:0/807184720 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc18135e50 0x7fdc18138230 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:52.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.989+0000 7fdc1d3a3700 1 --2- 172.21.15.3:0/807184720 >> v2:172.21.15.3:3300/0 conn(0x7fdc1809e1d0 0x7fdc18135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:52.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.989+0000 7fdc1d3a3700 1 -- 172.21.15.3:0/807184720 >> 172.21.15.3:0/807184720 conn(0x7fdc18066f50 msgr2=0x7fdc1809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:52.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.989+0000 7fdc1d3a3700 1 -- 172.21.15.3:0/807184720 shutdown_connections 2023-10-03T10:15:52.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.989+0000 7fdc1d3a3700 1 -- 172.21.15.3:0/807184720 wait complete. 2023-10-03T10:15:52.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.990+0000 7fdc1d3a3700 1 Processor -- start 2023-10-03T10:15:52.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.990+0000 7fdc1d3a3700 1 -- start start 2023-10-03T10:15:52.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.991+0000 7fdc1d3a3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc1809d6f0 0x7fdc181c4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:52.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.991+0000 7fdc1d3a3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc1809e1d0 0x7fdc181c4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:52.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.991+0000 7fdc1d3a3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc18135e50 0x7fdc18119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:52.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.991+0000 7fdc1d3a3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1811b1f0 con 0x7fdc1809e1d0 2023-10-03T10:15:52.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.991+0000 7fdc1d3a3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1811b000 con 0x7fdc1809d6f0 2023-10-03T10:15:52.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.991+0000 7fdc1d3a3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc180a1e20 con 0x7fdc18135e50 2023-10-03T10:15:52.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.991+0000 7fdc16ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc1809d6f0 0x7fdc181c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:52.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.991+0000 7fdc16ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc1809d6f0 0x7fdc181c4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:35040/0 (socket says 172.21.15.3:35040) 2023-10-03T10:15:52.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.991+0000 7fdc16ffd700 1 -- 172.21.15.3:0/1926237027 learned_addr learned my addr 172.21.15.3:0/1926237027 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:52.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.991+0000 7fdc167fc700 1 --2- 172.21.15.3:0/1926237027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc1809e1d0 0x7fdc181c4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:52.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.992+0000 7fdc16ffd700 1 -- 172.21.15.3:0/1926237027 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc18135e50 msgr2=0x7fdc18119480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:52.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.992+0000 7fdc16ffd700 1 --2- 172.21.15.3:0/1926237027 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc18135e50 0x7fdc18119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:52.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.992+0000 7fdc16ffd700 1 -- 172.21.15.3:0/1926237027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc1809e1d0 msgr2=0x7fdc181c4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:52.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.992+0000 7fdc16ffd700 1 --2- 172.21.15.3:0/1926237027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc1809e1d0 0x7fdc181c4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:52.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.992+0000 7fdc16ffd700 1 -- 172.21.15.3:0/1926237027 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1000bd40 con 0x7fdc1809d6f0 2023-10-03T10:15:52.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.992+0000 7fdc167fc700 1 --2- 172.21.15.3:0/1926237027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc1809e1d0 0x7fdc181c4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:52.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.992+0000 7fdc16ffd700 1 --2- 172.21.15.3:0/1926237027 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc1809d6f0 0x7fdc181c4500 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800c480 tx=0x7fdc0800c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:52.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.992+0000 7fdbff7fe700 1 -- 172.21.15.3:0/1926237027 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc080052d0 con 0x7fdc1809d6f0 2023-10-03T10:15:52.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.992+0000 7fdbff7fe700 1 -- 172.21.15.3:0/1926237027 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc08005430 con 0x7fdc1809d6f0 2023-10-03T10:15:52.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.992+0000 7fdc1d3a3700 1 -- 172.21.15.3:0/1926237027 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1811c970 con 0x7fdc1809d6f0 2023-10-03T10:15:52.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.992+0000 7fdbff7fe700 1 -- 172.21.15.3:0/1926237027 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc080038c0 con 0x7fdc1809d6f0 2023-10-03T10:15:52.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.992+0000 7fdc1d3a3700 1 -- 172.21.15.3:0/1926237027 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1811cc30 con 0x7fdc1809d6f0 2023-10-03T10:15:52.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.993+0000 7fdbff7fe700 1 -- 172.21.15.3:0/1926237027 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdc08003a20 con 0x7fdc1809d6f0 2023-10-03T10:15:52.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.993+0000 7fdbff7fe700 1 -- 172.21.15.3:0/1926237027 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fdc0803eac0 con 0x7fdc1809d6f0 2023-10-03T10:15:52.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:52.993+0000 7fdc1d3a3700 1 -- 172.21.15.3:0/1926237027 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc18074ea0 con 0x7fdc1809d6f0 2023-10-03T10:15:53.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.003+0000 7fdbff7fe700 1 -- 172.21.15.3:0/1926237027 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdc08017070 con 0x7fdc1809d6f0 2023-10-03T10:15:53.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.074+0000 7f28597a3700 1 -- 172.21.15.3:0/2168367118 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f28540713a0 con 0x7f285412f8b0 2023-10-03T10:15:53.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.074+0000 7f283b7fe700 1 -- 172.21.15.3:0/2168367118 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2848027090 con 0x7f285412f8b0 2023-10-03T10:15:53.074 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:53.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.077+0000 7f28597a3700 1 -- 172.21.15.3:0/2168367118 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285412f8b0 msgr2=0x7f28541c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:53.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.077+0000 7f28597a3700 1 --2- 172.21.15.3:0/2168367118 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285412f8b0 0x7f28541c48b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f28480018a0 tx=0x7f2848001bb0 comp rx=0 tx=0).stop 2023-10-03T10:15:53.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.077+0000 7f28597a3700 1 -- 172.21.15.3:0/2168367118 shutdown_connections 2023-10-03T10:15:53.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.077+0000 7f28597a3700 1 --2- 172.21.15.3:0/2168367118 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2854130430 0x7f285409b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:53.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.077+0000 7f28597a3700 1 --2- 172.21.15.3:0/2168367118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285412cf60 0x7f28541c4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:53.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.077+0000 7f28597a3700 1 --2- 172.21.15.3:0/2168367118 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285412f8b0 0x7f28541c48b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:53.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.077+0000 7f28597a3700 1 -- 172.21.15.3:0/2168367118 >> 172.21.15.3:0/2168367118 conn(0x7f28540a2aa0 msgr2=0x7f28541b7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:53.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.077+0000 7f28597a3700 1 -- 172.21.15.3:0/2168367118 shutdown_connections 2023-10-03T10:15:53.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.077+0000 7f28597a3700 1 -- 172.21.15.3:0/2168367118 wait complete. 2023-10-03T10:15:53.091 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:53.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.133+0000 7fca449ba700 1 -- 172.21.15.3:0/3408893475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fca4011dc90 con 0x7fca4006c0e0 2023-10-03T10:15:53.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.133+0000 7fca2effd700 1 -- 172.21.15.3:0/3408893475 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fca3004fe60 con 0x7fca4006c0e0 2023-10-03T10:15:53.134 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:53.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.135+0000 7fca449ba700 1 -- 172.21.15.3:0/3408893475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca4006c0e0 msgr2=0x7fca401ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:53.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.135+0000 7fca449ba700 1 --2- 172.21.15.3:0/3408893475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca4006c0e0 0x7fca401ccd50 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fca30009c40 tx=0x7fca30004750 comp rx=0 tx=0).stop 2023-10-03T10:15:53.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.135+0000 7fca449ba700 1 -- 172.21.15.3:0/3408893475 shutdown_connections 2023-10-03T10:15:53.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.135+0000 7fca449ba700 1 --2- 172.21.15.3:0/3408893475 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca401380f0 0x7fca4011a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:53.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.135+0000 7fca449ba700 1 --2- 172.21.15.3:0/3408893475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca4006c0e0 0x7fca401ccd50 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:53.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.135+0000 7fca449ba700 1 --2- 172.21.15.3:0/3408893475 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca40134ec0 0x7fca401cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:53.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.135+0000 7fca449ba700 1 -- 172.21.15.3:0/3408893475 >> 172.21.15.3:0/3408893475 conn(0x7fca400a4210 msgr2=0x7fca400a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:53.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.136+0000 7fca449ba700 1 -- 172.21.15.3:0/3408893475 shutdown_connections 2023-10-03T10:15:53.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.136+0000 7fca449ba700 1 -- 172.21.15.3:0/3408893475 wait complete. 2023-10-03T10:15:53.149 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:53.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.170+0000 7fc0e1696700 1 -- 172.21.15.3:0/1641614756 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc0dc074ea0 con 0x7fc0dc09d6f0 2023-10-03T10:15:53.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.171+0000 7fc0c37fe700 1 -- 172.21.15.3:0/1641614756 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc0cc01ae80 con 0x7fc0dc09d6f0 2023-10-03T10:15:53.171 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:53.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.173+0000 7fc0e1696700 1 -- 172.21.15.3:0/1641614756 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0dc09d6f0 msgr2=0x7fc0dc12fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:53.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.173+0000 7fc0e1696700 1 --2- 172.21.15.3:0/1641614756 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0dc09d6f0 0x7fc0dc12fa50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc00e9e0 tx=0x7fc0cc00ecf0 comp rx=0 tx=0).stop 2023-10-03T10:15:53.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.173+0000 7fc0e1696700 1 -- 172.21.15.3:0/1641614756 shutdown_connections 2023-10-03T10:15:53.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.173+0000 7fc0e1696700 1 --2- 172.21.15.3:0/1641614756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0dc09e410 0x7fc0dc1c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:53.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.173+0000 7fc0e1696700 1 --2- 172.21.15.3:0/1641614756 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0dc09d6f0 0x7fc0dc12fa50 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:53.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.173+0000 7fc0e1696700 1 --2- 172.21.15.3:0/1641614756 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc0dc09e000 0x7fc0dc1c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:53.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.173+0000 7fc0e1696700 1 -- 172.21.15.3:0/1641614756 >> 172.21.15.3:0/1641614756 conn(0x7fc0dc066f50 msgr2=0x7fc0dc0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:53.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.173+0000 7fc0e1696700 1 -- 172.21.15.3:0/1641614756 shutdown_connections 2023-10-03T10:15:53.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.173+0000 7fc0e1696700 1 -- 172.21.15.3:0/1641614756 wait complete. 2023-10-03T10:15:53.187 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:53.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.210+0000 7fdc1d3a3700 1 -- 172.21.15.3:0/1926237027 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdc1806c360 con 0x7fdc1809d6f0 2023-10-03T10:15:53.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.211+0000 7fdbff7fe700 1 -- 172.21.15.3:0/1926237027 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdc08017070 con 0x7fdc1809d6f0 2023-10-03T10:15:53.211 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:53.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.213+0000 7fdc1d3a3700 1 -- 172.21.15.3:0/1926237027 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc1809d6f0 msgr2=0x7fdc181c4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:53.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.213+0000 7fdc1d3a3700 1 --2- 172.21.15.3:0/1926237027 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc1809d6f0 0x7fdc181c4500 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800c480 tx=0x7fdc0800c790 comp rx=0 tx=0).stop 2023-10-03T10:15:53.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.213+0000 7fdc1d3a3700 1 -- 172.21.15.3:0/1926237027 shutdown_connections 2023-10-03T10:15:53.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.213+0000 7fdc1d3a3700 1 --2- 172.21.15.3:0/1926237027 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdc18135e50 0x7fdc18119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:53.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.213+0000 7fdc1d3a3700 1 --2- 172.21.15.3:0/1926237027 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdc1809d6f0 0x7fdc181c4500 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:53.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.213+0000 7fdc1d3a3700 1 --2- 172.21.15.3:0/1926237027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc1809e1d0 0x7fdc181c4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:53.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.213+0000 7fdc1d3a3700 1 -- 172.21.15.3:0/1926237027 >> 172.21.15.3:0/1926237027 conn(0x7fdc18066f50 msgr2=0x7fdc1806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:53.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.213+0000 7fdc1d3a3700 1 -- 172.21.15.3:0/1926237027 shutdown_connections 2023-10-03T10:15:53.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:53.213+0000 7fdc1d3a3700 1 -- 172.21.15.3:0/1926237027 wait complete. 2023-10-03T10:15:53.227 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:54.093 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:54.150 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:54.188 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:54.227 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:54.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.243+0000 7f4ccfa14700 1 Processor -- start 2023-10-03T10:15:54.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.243+0000 7f4ccfa14700 1 -- start start 2023-10-03T10:15:54.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.244+0000 7f4ccfa14700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cc809d6f0 0x7f4cc809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:54.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.244+0000 7f4ccfa14700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cc809e000 0x7f4cc81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:54.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.244+0000 7f4ccfa14700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4cc809e410 0x7f4cc8133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:54.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.244+0000 7f4ccfa14700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cc806b460 con 0x7f4cc809e410 2023-10-03T10:15:54.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.244+0000 7f4ccfa14700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc809b120 con 0x7f4cc809e000 2023-10-03T10:15:54.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.244+0000 7f4ccfa14700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc809a7e0 con 0x7f4cc809d6f0 2023-10-03T10:15:54.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.244+0000 7f4cccfaf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cc809e000 0x7f4cc81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:54.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.244+0000 7f4cccfaf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cc809e000 0x7f4cc81339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38806/0 (socket says 172.21.15.3:38806) 2023-10-03T10:15:54.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.244+0000 7f4cccfaf700 1 -- 172.21.15.3:0/3774580444 learned_addr learned my addr 172.21.15.3:0/3774580444 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:54.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.244+0000 7f4ccdfb1700 1 --2- 172.21.15.3:0/3774580444 >> v2:172.21.15.3:3300/0 conn(0x7f4cc809e410 0x7f4cc8133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:54.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.244+0000 7f4ccd7b0700 1 --2- 172.21.15.3:0/3774580444 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cc809d6f0 0x7f4cc809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:54.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.244+0000 7f4ccdfb1700 1 -- 172.21.15.3:0/3774580444 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cc809d6f0 msgr2=0x7f4cc809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:54.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.245+0000 7f4ccdfb1700 1 --2- 172.21.15.3:0/3774580444 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cc809d6f0 0x7f4cc809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.245+0000 7f4ccdfb1700 1 -- 172.21.15.3:0/3774580444 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cc809e000 msgr2=0x7f4cc81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:54.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.245+0000 7f4ccdfb1700 1 --2- 172.21.15.3:0/3774580444 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cc809e000 0x7f4cc81339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.245+0000 7f4ccdfb1700 1 -- 172.21.15.3:0/3774580444 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc809a500 con 0x7f4cc809e410 2023-10-03T10:15:54.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.245+0000 7f4ccdfb1700 1 --2- 172.21.15.3:0/3774580444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cc809e410 0x7f4cc8133f30 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0009d90 tx=0x7f4cc000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e8803bbcd6da10 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:54.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.245+0000 7f4cbffff700 1 -- 172.21.15.3:0/3774580444 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc000e070 con 0x7f4cc809e410 2023-10-03T10:15:54.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.245+0000 7f4cbffff700 1 -- 172.21.15.3:0/3774580444 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc000bb20 con 0x7f4cc809e410 2023-10-03T10:15:54.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.245+0000 7f4cbffff700 1 -- 172.21.15.3:0/3774580444 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0012430 con 0x7f4cc809e410 2023-10-03T10:15:54.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.246+0000 7f4ccfa14700 1 -- 172.21.15.3:0/3774580444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cc809e410 msgr2=0x7f4cc8133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:54.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.246+0000 7f4ccfa14700 1 --2- 172.21.15.3:0/3774580444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cc809e410 0x7f4cc8133f30 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0009d90 tx=0x7f4cc000b330 comp rx=0 tx=0).stop 2023-10-03T10:15:54.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.246+0000 7f4ccfa14700 1 -- 172.21.15.3:0/3774580444 shutdown_connections 2023-10-03T10:15:54.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.246+0000 7f4ccfa14700 1 --2- 172.21.15.3:0/3774580444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cc809e410 0x7f4cc8133f30 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.246+0000 7f4ccfa14700 1 --2- 172.21.15.3:0/3774580444 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cc809d6f0 0x7f4cc809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.246+0000 7f4ccfa14700 1 --2- 172.21.15.3:0/3774580444 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cc809e000 0x7f4cc81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.246+0000 7f4ccfa14700 1 -- 172.21.15.3:0/3774580444 >> 172.21.15.3:0/3774580444 conn(0x7f4cc8066f50 msgr2=0x7f4cc809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:54.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.246+0000 7f4ccfa14700 1 -- 172.21.15.3:0/3774580444 shutdown_connections 2023-10-03T10:15:54.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.246+0000 7f4ccfa14700 1 -- 172.21.15.3:0/3774580444 wait complete. 2023-10-03T10:15:54.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.248+0000 7f4ccfa14700 1 Processor -- start 2023-10-03T10:15:54.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.248+0000 7f4ccfa14700 1 -- start start 2023-10-03T10:15:54.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.248+0000 7f4ccfa14700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cc809d6f0 0x7f4cc81196f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:54.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.248+0000 7f4ccfa14700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cc809e000 0x7f4cc8119c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:54.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.248+0000 7f4ccfa14700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cc809e410 0x7f4cc811a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:54.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.248+0000 7f4ccfa14700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc811b5e0 con 0x7f4cc809d6f0 2023-10-03T10:15:54.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.248+0000 7f4ccfa14700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc80a3090 con 0x7f4cc809e410 2023-10-03T10:15:54.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.248+0000 7f4ccfa14700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc809c710 con 0x7f4cc809e000 2023-10-03T10:15:54.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.249+0000 7f4cccfaf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cc809e000 0x7f4cc8119c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:54.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.249+0000 7f4cccfaf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cc809e000 0x7f4cc8119c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:58266/0 (socket says 172.21.15.3:58266) 2023-10-03T10:15:54.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.249+0000 7f4cccfaf700 1 -- 172.21.15.3:0/1014530789 learned_addr learned my addr 172.21.15.3:0/1014530789 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:54.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.249+0000 7f4ccdfb1700 1 --2- 172.21.15.3:0/1014530789 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cc809e410 0x7f4cc811a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:54.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.249+0000 7f4ccd7b0700 1 --2- 172.21.15.3:0/1014530789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cc809d6f0 0x7f4cc81196f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:54.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.249+0000 7f4cccfaf700 1 -- 172.21.15.3:0/1014530789 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cc809e410 msgr2=0x7f4cc811a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:54.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.249+0000 7f4cccfaf700 1 --2- 172.21.15.3:0/1014530789 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cc809e410 0x7f4cc811a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.249+0000 7f4cccfaf700 1 -- 172.21.15.3:0/1014530789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cc809d6f0 msgr2=0x7f4cc81196f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:54.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.249+0000 7f4cccfaf700 1 --2- 172.21.15.3:0/1014530789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cc809d6f0 0x7f4cc81196f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.249+0000 7f4cccfaf700 1 -- 172.21.15.3:0/1014530789 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc0009a70 con 0x7f4cc809e000 2023-10-03T10:15:54.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.249+0000 7f4ccd7b0700 1 --2- 172.21.15.3:0/1014530789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cc809d6f0 0x7f4cc81196f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:15:54.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.249+0000 7f4cccfaf700 1 --2- 172.21.15.3:0/1014530789 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cc809e000 0x7f4cc8119c30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4cb800e8e0 tx=0x7f4cb800ebf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:54.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.250+0000 7f4cbdffb700 1 -- 172.21.15.3:0/1014530789 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8013680 con 0x7f4cc809e000 2023-10-03T10:15:54.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.250+0000 7f4cbdffb700 1 -- 172.21.15.3:0/1014530789 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb80137e0 con 0x7f4cc809e000 2023-10-03T10:15:54.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.250+0000 7f4ccfa14700 1 -- 172.21.15.3:0/1014530789 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc811c670 con 0x7f4cc809e000 2023-10-03T10:15:54.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.250+0000 7f4cbdffb700 1 -- 172.21.15.3:0/1014530789 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8013b50 con 0x7f4cc809e000 2023-10-03T10:15:54.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.250+0000 7f4ccfa14700 1 -- 172.21.15.3:0/1014530789 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc811b7c0 con 0x7f4cc809e000 2023-10-03T10:15:54.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.251+0000 7f4ccfa14700 1 -- 172.21.15.3:0/1014530789 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cc8074ea0 con 0x7f4cc809e000 2023-10-03T10:15:54.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.252+0000 7f4cbdffb700 1 -- 172.21.15.3:0/1014530789 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4cb8013cb0 con 0x7f4cc809e000 2023-10-03T10:15:54.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.252+0000 7f4cbdffb700 1 -- 172.21.15.3:0/1014530789 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4cb803ff80 con 0x7f4cc809e000 2023-10-03T10:15:54.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.257+0000 7f4cbdffb700 1 -- 172.21.15.3:0/1014530789 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4cb8024b40 con 0x7f4cc809e000 2023-10-03T10:15:54.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.298+0000 7f2cee332700 1 Processor -- start 2023-10-03T10:15:54.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.298+0000 7f2cee332700 1 -- start start 2023-10-03T10:15:54.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.298+0000 7f2cee332700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2ce809e1d0 0x7f2ce8135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:54.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.298+0000 7f2cee332700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ce8135e50 0x7f2ce8138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:54.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.298+0000 7f2cee332700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce809f130 con 0x7f2ce809d6f0 2023-10-03T10:15:54.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.298+0000 7f2cee332700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ce809f520 con 0x7f2ce809e1d0 2023-10-03T10:15:54.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.298+0000 7f2cee332700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce809e930 con 0x7f2ce8135e50 2023-10-03T10:15:54.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.299+0000 7f2ce77fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2ce809e1d0 0x7f2ce8135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:54.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.299+0000 7f2cec8cf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ce8135e50 0x7f2ce8138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:54.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.299+0000 7f2ce77fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2ce809e1d0 0x7f2ce8135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60846/0 (socket says 172.21.15.3:60846) 2023-10-03T10:15:54.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.299+0000 7f2ce77fe700 1 -- 172.21.15.3:0/2072600231 learned_addr learned my addr 172.21.15.3:0/2072600231 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:54.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.299+0000 7f2cec8cf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ce8135e50 0x7f2ce8138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38834/0 (socket says 172.21.15.3:38834) 2023-10-03T10:15:54.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.299+0000 7f2ce77fe700 1 -- 172.21.15.3:0/2072600231 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ce8135e50 msgr2=0x7f2ce8138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:54.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.299+0000 7f2ce77fe700 1 --2- 172.21.15.3:0/2072600231 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ce8135e50 0x7f2ce8138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.299+0000 7f2ce6ffd700 1 -- 172.21.15.3:0/2072600231 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 301611144 0 0) 0x7f2ce809f130 con 0x7f2ce809d6f0 2023-10-03T10:15:54.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.299+0000 7f2ce77fe700 1 -- 172.21.15.3:0/2072600231 >> v1:172.21.15.3:6789/0 conn(0x7f2ce809d6f0 legacy=0x7f2ce809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:54.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.299+0000 7f2ce77fe700 1 -- 172.21.15.3:0/2072600231 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce809b2d0 con 0x7f2ce809e1d0 2023-10-03T10:15:54.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.300+0000 7f2ce77fe700 1 --2- 172.21.15.3:0/2072600231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce809e1d0 0x7f2ce8135910 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc0098f0 tx=0x7f2cdc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5cb8b8e357b76 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:54.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.300+0000 7f2ce6ffd700 1 -- 172.21.15.3:0/2072600231 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc011830 con 0x7f2ce809e1d0 2023-10-03T10:15:54.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.300+0000 7f2ce6ffd700 1 -- 172.21.15.3:0/2072600231 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cdc011990 con 0x7f2ce809e1d0 2023-10-03T10:15:54.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.300+0000 7f2ce6ffd700 1 -- 172.21.15.3:0/2072600231 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc011d20 con 0x7f2ce809e1d0 2023-10-03T10:15:54.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.301+0000 7f2cee332700 1 -- 172.21.15.3:0/2072600231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce809e1d0 msgr2=0x7f2ce8135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:54.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.301+0000 7f2cee332700 1 --2- 172.21.15.3:0/2072600231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce809e1d0 0x7f2ce8135910 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc0098f0 tx=0x7f2cdc009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:54.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.301+0000 7f2cee332700 1 -- 172.21.15.3:0/2072600231 shutdown_connections 2023-10-03T10:15:54.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.301+0000 7f2cee332700 1 --2- 172.21.15.3:0/2072600231 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ce8135e50 0x7f2ce8138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.301+0000 7f2cee332700 1 --2- 172.21.15.3:0/2072600231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce809e1d0 0x7f2ce8135910 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.301+0000 7f2cee332700 1 -- 172.21.15.3:0/2072600231 >> 172.21.15.3:0/2072600231 conn(0x7f2ce8066f50 msgr2=0x7f2ce809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:54.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.301+0000 7f2cee332700 1 -- 172.21.15.3:0/2072600231 shutdown_connections 2023-10-03T10:15:54.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.301+0000 7f2cee332700 1 -- 172.21.15.3:0/2072600231 wait complete. 2023-10-03T10:15:54.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.302+0000 7f2cee332700 1 Processor -- start 2023-10-03T10:15:54.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.303+0000 7f2cee332700 1 -- start start 2023-10-03T10:15:54.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.303+0000 7f2cee332700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ce809d6f0 0x7f2ce81347e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:54.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.303+0000 7f2cee332700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ce809e1d0 0x7f2ce8134d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:54.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.303+0000 7f2cee332700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce8135e50 0x7f2ce8135260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:54.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.303+0000 7f2cee332700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce811b630 con 0x7f2ce8135e50 2023-10-03T10:15:54.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.303+0000 7f2cee332700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce811a230 con 0x7f2ce809d6f0 2023-10-03T10:15:54.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.303+0000 7f2cee332700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce8119980 con 0x7f2ce809e1d0 2023-10-03T10:15:54.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.304+0000 7f2ce7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ce809d6f0 0x7f2ce81347e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:54.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.304+0000 7f2ce77fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ce809e1d0 0x7f2ce8134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:54.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.304+0000 7f2ce77fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ce809e1d0 0x7f2ce8134d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:58272/0 (socket says 172.21.15.3:58272) 2023-10-03T10:15:54.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.304+0000 7f2ce77fe700 1 -- 172.21.15.3:0/2275128632 learned_addr learned my addr 172.21.15.3:0/2275128632 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:54.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.304+0000 7f2ce7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ce809d6f0 0x7f2ce81347e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38844/0 (socket says 172.21.15.3:38844) 2023-10-03T10:15:54.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.304+0000 7f2cec8cf700 1 --2- 172.21.15.3:0/2275128632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce8135e50 0x7f2ce8135260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:54.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.304+0000 7f2ce77fe700 1 -- 172.21.15.3:0/2275128632 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ce809d6f0 msgr2=0x7f2ce81347e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:54.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.304+0000 7f2ce77fe700 1 --2- 172.21.15.3:0/2275128632 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ce809d6f0 0x7f2ce81347e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.304+0000 7f2ce77fe700 1 -- 172.21.15.3:0/2275128632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce8135e50 msgr2=0x7f2ce8135260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:54.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.304+0000 7f2ce77fe700 1 --2- 172.21.15.3:0/2275128632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce8135e50 0x7f2ce8135260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.304+0000 7f2ce77fe700 1 -- 172.21.15.3:0/2275128632 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc009550 con 0x7f2ce809e1d0 2023-10-03T10:15:54.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.304+0000 7f2ce7fff700 1 --2- 172.21.15.3:0/2275128632 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ce809d6f0 0x7f2ce81347e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:54.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.304+0000 7f2cec8cf700 1 --2- 172.21.15.3:0/2275128632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce8135e50 0x7f2ce8135260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:54.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.304+0000 7f2ce77fe700 1 --2- 172.21.15.3:0/2275128632 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ce809e1d0 0x7f2ce8134d20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009130 tx=0x7f2cdc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:54.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.305+0000 7f2ce4ff9700 1 -- 172.21.15.3:0/2275128632 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc004510 con 0x7f2ce809e1d0 2023-10-03T10:15:54.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.305+0000 7f2ce4ff9700 1 -- 172.21.15.3:0/2275128632 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cdc01e070 con 0x7f2ce809e1d0 2023-10-03T10:15:54.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.305+0000 7f2cee332700 1 -- 172.21.15.3:0/2275128632 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce809c700 con 0x7f2ce809e1d0 2023-10-03T10:15:54.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.305+0000 7f2ce4ff9700 1 -- 172.21.15.3:0/2275128632 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc0038c0 con 0x7f2ce809e1d0 2023-10-03T10:15:54.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.305+0000 7f2cee332700 1 -- 172.21.15.3:0/2275128632 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce811b000 con 0x7f2ce809e1d0 2023-10-03T10:15:54.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.306+0000 7f2ce4ff9700 1 -- 172.21.15.3:0/2275128632 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2cdc003a20 con 0x7f2ce809e1d0 2023-10-03T10:15:54.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.306+0000 7f2ce4ff9700 1 -- 172.21.15.3:0/2275128632 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2cdc040690 con 0x7f2ce809e1d0 2023-10-03T10:15:54.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.306+0000 7f2cee332700 1 -- 172.21.15.3:0/2275128632 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce8071ff0 con 0x7f2ce809e1d0 2023-10-03T10:15:54.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.315+0000 7f2ce4ff9700 1 -- 172.21.15.3:0/2275128632 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2cdc015080 con 0x7f2ce809e1d0 2023-10-03T10:15:54.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.352+0000 7fcb8dfad700 1 Processor -- start 2023-10-03T10:15:54.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.352+0000 7fcb8dfad700 1 -- start start 2023-10-03T10:15:54.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.353+0000 7fcb8dfad700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb8809d6f0 0x7fcb8809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:54.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.353+0000 7fcb8dfad700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcb8809e410 0x7fcb88130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:54.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.353+0000 7fcb8dfad700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb8809f130 con 0x7fcb8809e000 2023-10-03T10:15:54.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.353+0000 7fcb8dfad700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb8809f520 con 0x7fcb8809e410 2023-10-03T10:15:54.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.353+0000 7fcb8dfad700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8809e930 con 0x7fcb8809d6f0 2023-10-03T10:15:54.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.353+0000 7fcb86ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcb8809e000 0x7fcb88130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36086/0 (socket says 172.21.15.3:36086) 2023-10-03T10:15:54.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.353+0000 7fcb877fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb8809d6f0 0x7fcb8809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:54.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.353+0000 7fcb86ffd700 1 -- 172.21.15.3:0/1331083416 learned_addr learned my addr 172.21.15.3:0/1331083416 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:54.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.353+0000 7fcb87fff700 1 --2- 172.21.15.3:0/1331083416 >> v2:172.21.15.3:3300/0 conn(0x7fcb8809e410 0x7fcb88130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:54.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.353+0000 7fcb877fe700 1 -- 172.21.15.3:0/1331083416 >> v2:172.21.15.3:3300/0 conn(0x7fcb8809e410 msgr2=0x7fcb88130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:54.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.353+0000 7fcb877fe700 1 --2- 172.21.15.3:0/1331083416 >> v2:172.21.15.3:3300/0 conn(0x7fcb8809e410 0x7fcb88130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.353+0000 7fcb877fe700 1 -- 172.21.15.3:0/1331083416 >> v1:172.21.15.3:6789/0 conn(0x7fcb8809e000 legacy=0x7fcb88130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:54.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.353+0000 7fcb877fe700 1 -- 172.21.15.3:0/1331083416 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8809b2d0 con 0x7fcb8809d6f0 2023-10-03T10:15:54.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.354+0000 7fcb877fe700 1 --2- 172.21.15.3:0/1331083416 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb8809d6f0 0x7fcb8809dac0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009840 tx=0x7fcb78009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa86b4dfb9816336 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:54.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.354+0000 7fcb867fc700 1 -- 172.21.15.3:0/1331083416 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7800f8a0 con 0x7fcb8809d6f0 2023-10-03T10:15:54.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.354+0000 7fcb867fc700 1 -- 172.21.15.3:0/1331083416 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7800fa00 con 0x7fcb8809d6f0 2023-10-03T10:15:54.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.354+0000 7fcb867fc700 1 -- 172.21.15.3:0/1331083416 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7800fd90 con 0x7fcb8809d6f0 2023-10-03T10:15:54.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.355+0000 7fcb8dfad700 1 -- 172.21.15.3:0/1331083416 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb8809d6f0 msgr2=0x7fcb8809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:54.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.355+0000 7fcb8dfad700 1 --2- 172.21.15.3:0/1331083416 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb8809d6f0 0x7fcb8809dac0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009840 tx=0x7fcb78009b50 comp rx=0 tx=0).stop 2023-10-03T10:15:54.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.355+0000 7fcb8dfad700 1 -- 172.21.15.3:0/1331083416 shutdown_connections 2023-10-03T10:15:54.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.355+0000 7fcb8dfad700 1 --2- 172.21.15.3:0/1331083416 >> v2:172.21.15.3:3300/0 conn(0x7fcb8809e410 0x7fcb88130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.355+0000 7fcb8dfad700 1 --2- 172.21.15.3:0/1331083416 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb8809d6f0 0x7fcb8809dac0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.355+0000 7fcb8dfad700 1 -- 172.21.15.3:0/1331083416 >> 172.21.15.3:0/1331083416 conn(0x7fcb88066f50 msgr2=0x7fcb8809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:54.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.355+0000 7fcb8dfad700 1 -- 172.21.15.3:0/1331083416 shutdown_connections 2023-10-03T10:15:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.355+0000 7fcb8dfad700 1 -- 172.21.15.3:0/1331083416 wait complete. 2023-10-03T10:15:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.357+0000 7fcb8dfad700 1 Processor -- start 2023-10-03T10:15:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.357+0000 7fcb8dfad700 1 -- start start 2023-10-03T10:15:54.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.358+0000 7fcb8dfad700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb8809d6f0 0x7fcb88122500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:54.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.358+0000 7fcb8dfad700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb8809e000 0x7fcb88122a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:54.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.358+0000 7fcb8dfad700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb8809e410 0x7fcb88122f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:54.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.358+0000 7fcb8dfad700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8809c700 con 0x7fcb8809e000 2023-10-03T10:15:54.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.358+0000 7fcb8dfad700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb88124150 con 0x7fcb8809e410 2023-10-03T10:15:54.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.358+0000 7fcb8dfad700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb88125c70 con 0x7fcb8809d6f0 2023-10-03T10:15:54.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.358+0000 7fcb86ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb8809e000 0x7fcb88122a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:54.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.358+0000 7fcb86ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb8809e000 0x7fcb88122a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60870/0 (socket says 172.21.15.3:60870) 2023-10-03T10:15:54.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.358+0000 7fcb86ffd700 1 -- 172.21.15.3:0/1941185214 learned_addr learned my addr 172.21.15.3:0/1941185214 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:54.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.358+0000 7fcb877fe700 1 --2- 172.21.15.3:0/1941185214 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb8809d6f0 0x7fcb88122500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:54.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.358+0000 7fcb86ffd700 1 -- 172.21.15.3:0/1941185214 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb8809d6f0 msgr2=0x7fcb88122500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:54.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.358+0000 7fcb86ffd700 1 --2- 172.21.15.3:0/1941185214 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb8809d6f0 0x7fcb88122500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.358+0000 7fcb86ffd700 1 -- 172.21.15.3:0/1941185214 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb8809e410 msgr2=0x7fcb88122f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:54.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.358+0000 7fcb86ffd700 1 --2- 172.21.15.3:0/1941185214 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb8809e410 0x7fcb88122f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.358+0000 7fcb86ffd700 1 -- 172.21.15.3:0/1941185214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb780094a0 con 0x7fcb8809e000 2023-10-03T10:15:54.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.358+0000 7fcb86ffd700 1 --2- 172.21.15.3:0/1941185214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb8809e000 0x7fcb88122a40 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00c500 tx=0x7fcb7c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:54.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.359+0000 7fcb6ffff700 1 -- 172.21.15.3:0/1941185214 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c005d70 con 0x7fcb8809e000 2023-10-03T10:15:54.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.359+0000 7fcb6ffff700 1 -- 172.21.15.3:0/1941185214 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7c0070d0 con 0x7fcb8809e000 2023-10-03T10:15:54.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.359+0000 7fcb8dfad700 1 -- 172.21.15.3:0/1941185214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb88124830 con 0x7fcb8809e000 2023-10-03T10:15:54.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.359+0000 7fcb6ffff700 1 -- 172.21.15.3:0/1941185214 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c007440 con 0x7fcb8809e000 2023-10-03T10:15:54.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.359+0000 7fcb8dfad700 1 -- 172.21.15.3:0/1941185214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb88126650 con 0x7fcb8809e000 2023-10-03T10:15:54.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.360+0000 7fcb6ffff700 1 -- 172.21.15.3:0/1941185214 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcb7c0075a0 con 0x7fcb8809e000 2023-10-03T10:15:54.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.360+0000 7fcb8dfad700 1 -- 172.21.15.3:0/1941185214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb88074ea0 con 0x7fcb8809e000 2023-10-03T10:15:54.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.360+0000 7fcb6ffff700 1 -- 172.21.15.3:0/1941185214 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fcb7c03ea50 con 0x7fcb8809e000 2023-10-03T10:15:54.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.365+0000 7fcb6ffff700 1 -- 172.21.15.3:0/1941185214 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcb7c019b40 con 0x7fcb8809e000 2023-10-03T10:15:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.390+0000 7f8bf5f6c700 1 Processor -- start 2023-10-03T10:15:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.390+0000 7f8bf5f6c700 1 -- start start 2023-10-03T10:15:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.390+0000 7f8bf5f6c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8bf006a770 0x7f8bf006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:54.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.390+0000 7f8bf5f6c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8bf006b080 0x7f8bf0131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:54.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.390+0000 7f8bf5f6c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf009ece0 con 0x7f8bf0131b40 2023-10-03T10:15:54.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.390+0000 7f8bf5f6c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf009e760 con 0x7f8bf006b080 2023-10-03T10:15:54.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.390+0000 7f8bf5f6c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf009e040 con 0x7f8bf006a770 2023-10-03T10:15:54.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.390+0000 7f8bef7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8bf006a770 0x7f8bf006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:54.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.390+0000 7f8bef7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8bf006a770 0x7f8bf006ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:58290/0 (socket says 172.21.15.3:58290) 2023-10-03T10:15:54.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.390+0000 7f8bef7fe700 1 -- 172.21.15.3:0/4167008101 learned_addr learned my addr 172.21.15.3:0/4167008101 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:54.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.391+0000 7f8bef7fe700 1 -- 172.21.15.3:0/4167008101 >> v2:172.21.15.3:3300/0 conn(0x7f8bf006b080 msgr2=0x7f8bf0131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:54.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.391+0000 7f8bef7fe700 1 --2- 172.21.15.3:0/4167008101 >> v2:172.21.15.3:3300/0 conn(0x7f8bf006b080 0x7f8bf0131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.391+0000 7f8bef7fe700 1 -- 172.21.15.3:0/4167008101 >> v1:172.21.15.3:6789/0 conn(0x7f8bf0131b40 legacy=0x7f8bf0138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:54.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.391+0000 7f8bef7fe700 1 -- 172.21.15.3:0/4167008101 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf009f630 con 0x7f8bf006a770 2023-10-03T10:15:54.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.391+0000 7f8bef7fe700 1 --2- 172.21.15.3:0/4167008101 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8bf006a770 0x7f8bf006ab40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8be00098f0 tx=0x7f8be0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f58e7d50e57b336 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:54.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.391+0000 7f8bee7fc700 1 -- 172.21.15.3:0/4167008101 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be0010a90 con 0x7f8bf006a770 2023-10-03T10:15:54.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.391+0000 7f8bee7fc700 1 -- 172.21.15.3:0/4167008101 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be0010bf0 con 0x7f8bf006a770 2023-10-03T10:15:54.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.391+0000 7f8bee7fc700 1 -- 172.21.15.3:0/4167008101 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be0018440 con 0x7f8bf006a770 2023-10-03T10:15:54.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.392+0000 7f8bf5f6c700 1 -- 172.21.15.3:0/4167008101 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8bf006a770 msgr2=0x7f8bf006ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:54.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.392+0000 7f8bf5f6c700 1 --2- 172.21.15.3:0/4167008101 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8bf006a770 0x7f8bf006ab40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8be00098f0 tx=0x7f8be0009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:54.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.392+0000 7f8bf5f6c700 1 -- 172.21.15.3:0/4167008101 shutdown_connections 2023-10-03T10:15:54.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.392+0000 7f8bf5f6c700 1 --2- 172.21.15.3:0/4167008101 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8bf006a770 0x7f8bf006ab40 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.392+0000 7f8bf5f6c700 1 --2- 172.21.15.3:0/4167008101 >> v2:172.21.15.3:3300/0 conn(0x7f8bf006b080 0x7f8bf0131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.392+0000 7f8bf5f6c700 1 -- 172.21.15.3:0/4167008101 >> 172.21.15.3:0/4167008101 conn(0x7f8bf0066f50 msgr2=0x7f8bf009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:54.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.392+0000 7f8bf5f6c700 1 -- 172.21.15.3:0/4167008101 shutdown_connections 2023-10-03T10:15:54.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.392+0000 7f8bf5f6c700 1 -- 172.21.15.3:0/4167008101 wait complete. 2023-10-03T10:15:54.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.393+0000 7f8bf5f6c700 1 Processor -- start 2023-10-03T10:15:54.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.393+0000 7f8bf5f6c700 1 -- start start 2023-10-03T10:15:54.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.394+0000 7f8bf5f6c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8bf006a770 0x7f8bf0130440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:54.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.394+0000 7f8bf5f6c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bf006b080 0x7f8bf0130980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:54.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.394+0000 7f8bf5f6c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8bf0131b40 0x7f8bf01c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:54.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.394+0000 7f8bf5f6c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf011eb80 con 0x7f8bf006b080 2023-10-03T10:15:54.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.394+0000 7f8bf5f6c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf0120290 con 0x7f8bf0131b40 2023-10-03T10:15:54.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.394+0000 7f8bf5f6c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf0121c00 con 0x7f8bf006a770 2023-10-03T10:15:54.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.394+0000 7f8bef7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8bf006a770 0x7f8bf0130440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:54.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.394+0000 7f8beeffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bf006b080 0x7f8bf0130980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:54.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.394+0000 7f8bef7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8bf006a770 0x7f8bf0130440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:58300/0 (socket says 172.21.15.3:58300) 2023-10-03T10:15:54.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.394+0000 7f8bef7fe700 1 -- 172.21.15.3:0/1456326116 learned_addr learned my addr 172.21.15.3:0/1456326116 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:54.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.394+0000 7f8beffff700 1 --2- 172.21.15.3:0/1456326116 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8bf0131b40 0x7f8bf01c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:54.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.394+0000 7f8beeffd700 1 -- 172.21.15.3:0/1456326116 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8bf006a770 msgr2=0x7f8bf0130440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:54.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.394+0000 7f8beeffd700 1 --2- 172.21.15.3:0/1456326116 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8bf006a770 0x7f8bf0130440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.394+0000 7f8beeffd700 1 -- 172.21.15.3:0/1456326116 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8bf0131b40 msgr2=0x7f8bf01c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:54.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.394+0000 7f8beeffd700 1 --2- 172.21.15.3:0/1456326116 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8bf0131b40 0x7f8bf01c3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.395+0000 7f8beeffd700 1 -- 172.21.15.3:0/1456326116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be0009550 con 0x7f8bf006b080 2023-10-03T10:15:54.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.395+0000 7f8beeffd700 1 --2- 172.21.15.3:0/1456326116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bf006b080 0x7f8bf0130980 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f8bf009f8c0 tx=0x7f8be400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:54.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.395+0000 7f8bd7fff700 1 -- 172.21.15.3:0/1456326116 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be4011830 con 0x7f8bf006b080 2023-10-03T10:15:54.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.395+0000 7f8bd7fff700 1 -- 172.21.15.3:0/1456326116 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be4011990 con 0x7f8bf006b080 2023-10-03T10:15:54.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.395+0000 7f8bd7fff700 1 -- 172.21.15.3:0/1456326116 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be4011d20 con 0x7f8bf006b080 2023-10-03T10:15:54.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.396+0000 7f8bf5f6c700 1 -- 172.21.15.3:0/1456326116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf0071e60 con 0x7f8bf006b080 2023-10-03T10:15:54.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.396+0000 7f8bf5f6c700 1 -- 172.21.15.3:0/1456326116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf01311a0 con 0x7f8bf006b080 2023-10-03T10:15:54.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.397+0000 7f8bf5f6c700 1 -- 172.21.15.3:0/1456326116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf0074ea0 con 0x7f8bf006b080 2023-10-03T10:15:54.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.397+0000 7f8bd7fff700 1 -- 172.21.15.3:0/1456326116 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8be401a400 con 0x7f8bf006b080 2023-10-03T10:15:54.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.397+0000 7f8bd7fff700 1 -- 172.21.15.3:0/1456326116 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8be403f4f0 con 0x7f8bf006b080 2023-10-03T10:15:54.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.406+0000 7f8bd7fff700 1 -- 172.21.15.3:0/1456326116 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8be400f2e0 con 0x7f8bf006b080 2023-10-03T10:15:54.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.472+0000 7f4ccfa14700 1 -- 172.21.15.3:0/1014530789 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4cc811ccc0 con 0x7f4cc809e000 2023-10-03T10:15:54.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.473+0000 7f4cbdffb700 1 -- 172.21.15.3:0/1014530789 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4cb8024b40 con 0x7f4cc809e000 2023-10-03T10:15:54.473 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:54.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.475+0000 7f4ccfa14700 1 -- 172.21.15.3:0/1014530789 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cc809e000 msgr2=0x7f4cc8119c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:54.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.475+0000 7f4ccfa14700 1 --2- 172.21.15.3:0/1014530789 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cc809e000 0x7f4cc8119c30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4cb800e8e0 tx=0x7f4cb800ebf0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.475+0000 7f4ccfa14700 1 -- 172.21.15.3:0/1014530789 shutdown_connections 2023-10-03T10:15:54.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.475+0000 7f4ccfa14700 1 --2- 172.21.15.3:0/1014530789 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cc809e410 0x7f4cc811a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.475+0000 7f4ccfa14700 1 --2- 172.21.15.3:0/1014530789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cc809d6f0 0x7f4cc81196f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.475+0000 7f4ccfa14700 1 --2- 172.21.15.3:0/1014530789 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cc809e000 0x7f4cc8119c30 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.475+0000 7f4ccfa14700 1 -- 172.21.15.3:0/1014530789 >> 172.21.15.3:0/1014530789 conn(0x7f4cc8066f50 msgr2=0x7f4cc81badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:54.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.475+0000 7f4ccfa14700 1 -- 172.21.15.3:0/1014530789 shutdown_connections 2023-10-03T10:15:54.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.475+0000 7f4ccfa14700 1 -- 172.21.15.3:0/1014530789 wait complete. 2023-10-03T10:15:54.490 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:54.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.554+0000 7f2cee332700 1 -- 172.21.15.3:0/2275128632 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2ce8074ea0 con 0x7f2ce809e1d0 2023-10-03T10:15:54.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.554+0000 7f2ce4ff9700 1 -- 172.21.15.3:0/2275128632 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2cdc003cd0 con 0x7f2ce809e1d0 2023-10-03T10:15:54.555 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:54.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.557+0000 7f2cee332700 1 -- 172.21.15.3:0/2275128632 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ce809e1d0 msgr2=0x7f2ce8134d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:54.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.557+0000 7f2cee332700 1 --2- 172.21.15.3:0/2275128632 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ce809e1d0 0x7f2ce8134d20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009130 tx=0x7f2cdc004750 comp rx=0 tx=0).stop 2023-10-03T10:15:54.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.557+0000 7f2cee332700 1 -- 172.21.15.3:0/2275128632 shutdown_connections 2023-10-03T10:15:54.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.557+0000 7f2cee332700 1 --2- 172.21.15.3:0/2275128632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce8135e50 0x7f2ce8135260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.557+0000 7f2cee332700 1 --2- 172.21.15.3:0/2275128632 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ce809d6f0 0x7f2ce81347e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.557+0000 7f2cee332700 1 --2- 172.21.15.3:0/2275128632 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ce809e1d0 0x7f2ce8134d20 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.557+0000 7f2cee332700 1 -- 172.21.15.3:0/2275128632 >> 172.21.15.3:0/2275128632 conn(0x7f2ce8066f50 msgr2=0x7f2ce809fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:54.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.557+0000 7f2cee332700 1 -- 172.21.15.3:0/2275128632 shutdown_connections 2023-10-03T10:15:54.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.557+0000 7f2cee332700 1 -- 172.21.15.3:0/2275128632 wait complete. 2023-10-03T10:15:54.571 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:54.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.586+0000 7fcb8dfad700 1 -- 172.21.15.3:0/1941185214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcb8806c360 con 0x7fcb8809e000 2023-10-03T10:15:54.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.586+0000 7fcb6ffff700 1 -- 172.21.15.3:0/1941185214 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcb7c017030 con 0x7fcb8809e000 2023-10-03T10:15:54.586 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:54.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.588+0000 7fcb8dfad700 1 -- 172.21.15.3:0/1941185214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb8809e000 msgr2=0x7fcb88122a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:54.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.588+0000 7fcb8dfad700 1 --2- 172.21.15.3:0/1941185214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb8809e000 0x7fcb88122a40 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00c500 tx=0x7fcb7c00c810 comp rx=0 tx=0).stop 2023-10-03T10:15:54.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.588+0000 7fcb8dfad700 1 -- 172.21.15.3:0/1941185214 shutdown_connections 2023-10-03T10:15:54.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.588+0000 7fcb8dfad700 1 --2- 172.21.15.3:0/1941185214 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb8809e410 0x7fcb88122f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.588+0000 7fcb8dfad700 1 --2- 172.21.15.3:0/1941185214 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb8809d6f0 0x7fcb88122500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.588+0000 7fcb8dfad700 1 --2- 172.21.15.3:0/1941185214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb8809e000 0x7fcb88122a40 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.588+0000 7fcb8dfad700 1 -- 172.21.15.3:0/1941185214 >> 172.21.15.3:0/1941185214 conn(0x7fcb88066f50 msgr2=0x7fcb8806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:54.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.589+0000 7fcb8dfad700 1 -- 172.21.15.3:0/1941185214 shutdown_connections 2023-10-03T10:15:54.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.589+0000 7fcb8dfad700 1 -- 172.21.15.3:0/1941185214 wait complete. 2023-10-03T10:15:54.602 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:54.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.615+0000 7f8bf5f6c700 1 -- 172.21.15.3:0/1456326116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8bf0121dd0 con 0x7f8bf006b080 2023-10-03T10:15:54.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.615+0000 7f8bd7fff700 1 -- 172.21.15.3:0/1456326116 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8be4021ab0 con 0x7f8bf006b080 2023-10-03T10:15:54.615 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:54.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.617+0000 7f8bf5f6c700 1 -- 172.21.15.3:0/1456326116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bf006b080 msgr2=0x7f8bf0130980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:54.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.617+0000 7f8bf5f6c700 1 --2- 172.21.15.3:0/1456326116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bf006b080 0x7f8bf0130980 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f8bf009f8c0 tx=0x7f8be400b730 comp rx=0 tx=0).stop 2023-10-03T10:15:54.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.617+0000 7f8bf5f6c700 1 -- 172.21.15.3:0/1456326116 shutdown_connections 2023-10-03T10:15:54.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.617+0000 7f8bf5f6c700 1 --2- 172.21.15.3:0/1456326116 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8bf0131b40 0x7f8bf01c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.617+0000 7f8bf5f6c700 1 --2- 172.21.15.3:0/1456326116 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8bf006a770 0x7f8bf0130440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.617+0000 7f8bf5f6c700 1 --2- 172.21.15.3:0/1456326116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bf006b080 0x7f8bf0130980 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:54.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.617+0000 7f8bf5f6c700 1 -- 172.21.15.3:0/1456326116 >> 172.21.15.3:0/1456326116 conn(0x7f8bf0066f50 msgr2=0x7f8bf00a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:54.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.617+0000 7f8bf5f6c700 1 -- 172.21.15.3:0/1456326116 shutdown_connections 2023-10-03T10:15:54.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:54.617+0000 7f8bf5f6c700 1 -- 172.21.15.3:0/1456326116 wait complete. 2023-10-03T10:15:54.631 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:55.491 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:55.572 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:55.603 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:55.631 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:55.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.638+0000 7f9dde778700 1 Processor -- start 2023-10-03T10:15:55.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.638+0000 7f9dde778700 1 -- start start 2023-10-03T10:15:55.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.639+0000 7f9dde778700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9dd809d6f0 0x7f9dd809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:55.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.639+0000 7f9dde778700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9dd809e410 0x7f9dd8134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:55.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.639+0000 7f9dde778700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dd809b120 con 0x7f9dd809e000 2023-10-03T10:15:55.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.639+0000 7f9dde778700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dd806b460 con 0x7f9dd809e410 2023-10-03T10:15:55.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.639+0000 7f9dde778700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd809a7e0 con 0x7f9dd809d6f0 2023-10-03T10:15:55.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.639+0000 7f9dd77fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9dd809e000 0x7f9dd81339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36100/0 (socket says 172.21.15.3:36100) 2023-10-03T10:15:55.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.639+0000 7f9dd77fe700 1 -- 172.21.15.3:0/1568751976 learned_addr learned my addr 172.21.15.3:0/1568751976 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:55.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.639+0000 7f9ddcd15700 1 --2- 172.21.15.3:0/1568751976 >> v2:172.21.15.3:3300/0 conn(0x7f9dd809e410 0x7f9dd8134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:55.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.639+0000 7f9dd7fff700 1 --2- 172.21.15.3:0/1568751976 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9dd809d6f0 0x7f9dd809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:55.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.639+0000 7f9ddcd15700 1 -- 172.21.15.3:0/1568751976 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9dd809d6f0 msgr2=0x7f9dd809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:55.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.639+0000 7f9ddcd15700 1 --2- 172.21.15.3:0/1568751976 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9dd809d6f0 0x7f9dd809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:55.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.640+0000 7f9ddcd15700 1 -- 172.21.15.3:0/1568751976 >> v1:172.21.15.3:6789/0 conn(0x7f9dd809e000 legacy=0x7f9dd81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:55.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.640+0000 7f9ddcd15700 1 -- 172.21.15.3:0/1568751976 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd809a500 con 0x7f9dd809e410 2023-10-03T10:15:55.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.640+0000 7f9ddcd15700 1 --2- 172.21.15.3:0/1568751976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dd809e410 0x7f9dd8134100 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009d90 tx=0x7f9dd000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86fec0da6255a307 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:55.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.640+0000 7f9dd6ffd700 1 -- 172.21.15.3:0/1568751976 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd000e070 con 0x7f9dd809e410 2023-10-03T10:15:55.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.640+0000 7f9dd6ffd700 1 -- 172.21.15.3:0/1568751976 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd000bbd0 con 0x7f9dd809e410 2023-10-03T10:15:55.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.640+0000 7f9dd6ffd700 1 -- 172.21.15.3:0/1568751976 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0012430 con 0x7f9dd809e410 2023-10-03T10:15:55.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.641+0000 7f9dde778700 1 -- 172.21.15.3:0/1568751976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dd809e410 msgr2=0x7f9dd8134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:55.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.641+0000 7f9dde778700 1 --2- 172.21.15.3:0/1568751976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dd809e410 0x7f9dd8134100 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009d90 tx=0x7f9dd000b330 comp rx=0 tx=0).stop 2023-10-03T10:15:55.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.641+0000 7f9dde778700 1 -- 172.21.15.3:0/1568751976 shutdown_connections 2023-10-03T10:15:55.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.641+0000 7f9dde778700 1 --2- 172.21.15.3:0/1568751976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dd809e410 0x7f9dd8134100 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:55.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.641+0000 7f9dde778700 1 --2- 172.21.15.3:0/1568751976 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9dd809d6f0 0x7f9dd809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:55.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.641+0000 7f9dde778700 1 -- 172.21.15.3:0/1568751976 >> 172.21.15.3:0/1568751976 conn(0x7f9dd8066f50 msgr2=0x7f9dd809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:55.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.641+0000 7f9dde778700 1 -- 172.21.15.3:0/1568751976 shutdown_connections 2023-10-03T10:15:55.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.641+0000 7f9dde778700 1 -- 172.21.15.3:0/1568751976 wait complete. 2023-10-03T10:15:55.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.642+0000 7f9dde778700 1 Processor -- start 2023-10-03T10:15:55.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.642+0000 7f9dde778700 1 -- start start 2023-10-03T10:15:55.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.643+0000 7f9dde778700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9dd809d6f0 0x7f9dd81a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:55.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.643+0000 7f9dde778700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9dd809e000 0x7f9dd81a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:55.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.643+0000 7f9dde778700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dd809e410 0x7f9dd81a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:55.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.643+0000 7f9dde778700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd81a7ac0 con 0x7f9dd809e410 2023-10-03T10:15:55.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.643+0000 7f9dde778700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd809c710 con 0x7f9dd809d6f0 2023-10-03T10:15:55.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.643+0000 7f9dde778700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd81a96f0 con 0x7f9dd809e000 2023-10-03T10:15:55.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.643+0000 7f9dd77fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9dd809e000 0x7f9dd81a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:55.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.643+0000 7f9dd7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9dd809d6f0 0x7f9dd81a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:55.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.643+0000 7f9dd77fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9dd809e000 0x7f9dd81a65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:58328/0 (socket says 172.21.15.3:58328) 2023-10-03T10:15:55.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.643+0000 7f9dd77fe700 1 -- 172.21.15.3:0/4252743008 learned_addr learned my addr 172.21.15.3:0/4252743008 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:55.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.643+0000 7f9dd7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9dd809d6f0 0x7f9dd81a60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38872/0 (socket says 172.21.15.3:38872) 2023-10-03T10:15:55.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.643+0000 7f9ddcd15700 1 --2- 172.21.15.3:0/4252743008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dd809e410 0x7f9dd81a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:55.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.644+0000 7f9ddcd15700 1 -- 172.21.15.3:0/4252743008 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9dd809e000 msgr2=0x7f9dd81a65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:55.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.644+0000 7f9ddcd15700 1 --2- 172.21.15.3:0/4252743008 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9dd809e000 0x7f9dd81a65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:55.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.644+0000 7f9ddcd15700 1 -- 172.21.15.3:0/4252743008 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9dd809d6f0 msgr2=0x7f9dd81a60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:55.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.644+0000 7f9ddcd15700 1 --2- 172.21.15.3:0/4252743008 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9dd809d6f0 0x7f9dd81a60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:55.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.644+0000 7f9ddcd15700 1 -- 172.21.15.3:0/4252743008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd0009a70 con 0x7f9dd809e410 2023-10-03T10:15:55.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.644+0000 7f9ddcd15700 1 --2- 172.21.15.3:0/4252743008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dd809e410 0x7f9dd81a6b20 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009d90 tx=0x7f9dd0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:55.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.644+0000 7f9dd4ff9700 1 -- 172.21.15.3:0/4252743008 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0012430 con 0x7f9dd809e410 2023-10-03T10:15:55.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.644+0000 7f9dd4ff9700 1 -- 172.21.15.3:0/4252743008 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd0020070 con 0x7f9dd809e410 2023-10-03T10:15:55.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.644+0000 7f9dd4ff9700 1 -- 172.21.15.3:0/4252743008 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0009410 con 0x7f9dd809e410 2023-10-03T10:15:55.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.645+0000 7f9dde778700 1 -- 172.21.15.3:0/4252743008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd80a30d0 con 0x7f9dd809e410 2023-10-03T10:15:55.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.645+0000 7f9dde778700 1 -- 172.21.15.3:0/4252743008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd81a9160 con 0x7f9dd809e410 2023-10-03T10:15:55.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.646+0000 7f9dd4ff9700 1 -- 172.21.15.3:0/4252743008 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9dd0004040 con 0x7f9dd809e410 2023-10-03T10:15:55.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.646+0000 7f9dd4ff9700 1 -- 172.21.15.3:0/4252743008 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9dd0022070 con 0x7f9dd809e410 2023-10-03T10:15:55.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.646+0000 7f9dde778700 1 -- 172.21.15.3:0/4252743008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dd8074ea0 con 0x7f9dd809e410 2023-10-03T10:15:55.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.653+0000 7f9dd4ff9700 1 -- 172.21.15.3:0/4252743008 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9dd00269d0 con 0x7f9dd809e410 2023-10-03T10:15:55.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.728+0000 7f5fc54a0700 1 Processor -- start 2023-10-03T10:15:55.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.728+0000 7f5fc54a0700 1 -- start start 2023-10-03T10:15:55.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.729+0000 7f5fc54a0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5fc009e930 0x7f5fc009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:55.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.729+0000 7f5fc54a0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fc009f240 0x7f5fc01317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:55.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.729+0000 7f5fc54a0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fc009f650 0x7f5fc0131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:55.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.729+0000 7f5fc54a0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fc0070940 con 0x7f5fc009e930 2023-10-03T10:15:55.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.729+0000 7f5fc54a0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc006fd60 con 0x7f5fc009f650 2023-10-03T10:15:55.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.729+0000 7f5fc54a0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc006fa80 con 0x7f5fc009f240 2023-10-03T10:15:55.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.729+0000 7f5fbf7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fc009f650 0x7f5fc0131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:55.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.729+0000 7f5fbe7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fc009f240 0x7f5fc01317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:55.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.729+0000 7f5fbf7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fc009f650 0x7f5fc0131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38884/0 (socket says 172.21.15.3:38884) 2023-10-03T10:15:55.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.729+0000 7f5fbf7fe700 1 -- 172.21.15.3:0/1471693316 learned_addr learned my addr 172.21.15.3:0/1471693316 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:55.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.729+0000 7f5fbeffd700 1 --2- 172.21.15.3:0/1471693316 >> v2:172.21.15.3:3300/0 conn(0x7f5fc009e930 0x7f5fc009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:55.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.729+0000 7f5fbf7fe700 1 -- 172.21.15.3:0/1471693316 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fc009f240 msgr2=0x7f5fc01317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:55.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.729+0000 7f5fbf7fe700 1 --2- 172.21.15.3:0/1471693316 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fc009f240 0x7f5fc01317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:55.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.729+0000 7f5fbf7fe700 1 -- 172.21.15.3:0/1471693316 >> v2:172.21.15.3:3300/0 conn(0x7f5fc009e930 msgr2=0x7f5fc009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:55.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.729+0000 7f5fbf7fe700 1 --2- 172.21.15.3:0/1471693316 >> v2:172.21.15.3:3300/0 conn(0x7f5fc009e930 0x7f5fc009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:55.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.729+0000 7f5fbf7fe700 1 -- 172.21.15.3:0/1471693316 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc009fb60 con 0x7f5fc009f650 2023-10-03T10:15:55.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.729+0000 7f5fbe7fc700 1 --2- 172.21.15.3:0/1471693316 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fc009f240 0x7f5fc01317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:55.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.730+0000 7f5fbf7fe700 1 --2- 172.21.15.3:0/1471693316 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fc009f650 0x7f5fc0131d30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8009e50 tx=0x7f5fb800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de999cdcf3afe3b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:55.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.730+0000 7f5fbdffb700 1 -- 172.21.15.3:0/1471693316 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb800e070 con 0x7f5fc009f650 2023-10-03T10:15:55.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.730+0000 7f5fbdffb700 1 -- 172.21.15.3:0/1471693316 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb800b910 con 0x7f5fc009f650 2023-10-03T10:15:55.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.730+0000 7f5fbdffb700 1 -- 172.21.15.3:0/1471693316 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb800bcc0 con 0x7f5fc009f650 2023-10-03T10:15:55.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.731+0000 7f5fc54a0700 1 -- 172.21.15.3:0/1471693316 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fc009f650 msgr2=0x7f5fc0131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:55.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.731+0000 7f5fc54a0700 1 --2- 172.21.15.3:0/1471693316 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fc009f650 0x7f5fc0131d30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8009e50 tx=0x7f5fb800b330 comp rx=0 tx=0).stop 2023-10-03T10:15:55.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.731+0000 7f5fc54a0700 1 -- 172.21.15.3:0/1471693316 shutdown_connections 2023-10-03T10:15:55.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.731+0000 7f5fc54a0700 1 --2- 172.21.15.3:0/1471693316 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fc009f650 0x7f5fc0131d30 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:55.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.731+0000 7f5fc54a0700 1 --2- 172.21.15.3:0/1471693316 >> v2:172.21.15.3:3300/0 conn(0x7f5fc009e930 0x7f5fc009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:55.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.731+0000 7f5fc54a0700 1 --2- 172.21.15.3:0/1471693316 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fc009f240 0x7f5fc01317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:55.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.731+0000 7f5fc54a0700 1 -- 172.21.15.3:0/1471693316 >> 172.21.15.3:0/1471693316 conn(0x7f5fc00a4210 msgr2=0x7f5fc00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:55.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.731+0000 7f5fc54a0700 1 -- 172.21.15.3:0/1471693316 shutdown_connections 2023-10-03T10:15:55.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.731+0000 7f5fc54a0700 1 -- 172.21.15.3:0/1471693316 wait complete. 2023-10-03T10:15:55.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.733+0000 7f5fc54a0700 1 Processor -- start 2023-10-03T10:15:55.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.733+0000 7f5fc54a0700 1 -- start start 2023-10-03T10:15:55.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.733+0000 7f5fc54a0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fc009e930 0x7f5fc01c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:55.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.733+0000 7f5fc54a0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fc009f240 0x7f5fc01c44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:55.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.733+0000 7f5fc54a0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fc009f650 0x7f5fc01c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:55.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.733+0000 7f5fc54a0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc01a0ce0 con 0x7f5fc009e930 2023-10-03T10:15:55.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.733+0000 7f5fc54a0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc009e400 con 0x7f5fc009f240 2023-10-03T10:15:55.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.733+0000 7f5fc54a0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc019f660 con 0x7f5fc009f650 2023-10-03T10:15:55.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.734+0000 7f5fbeffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fc009e930 0x7f5fc01c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:55.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.734+0000 7f5fbeffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fc009e930 0x7f5fc01c3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60922/0 (socket says 172.21.15.3:60922) 2023-10-03T10:15:55.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.734+0000 7f5fbeffd700 1 -- 172.21.15.3:0/3119974906 learned_addr learned my addr 172.21.15.3:0/3119974906 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:55.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.734+0000 7f5fbe7fc700 1 --2- 172.21.15.3:0/3119974906 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fc009f240 0x7f5fc01c44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:55.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.734+0000 7f5fbf7fe700 1 --2- 172.21.15.3:0/3119974906 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fc009f650 0x7f5fc01c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:55.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.734+0000 7f5fbeffd700 1 -- 172.21.15.3:0/3119974906 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fc009f650 msgr2=0x7f5fc01c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:55.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.734+0000 7f5fbeffd700 1 --2- 172.21.15.3:0/3119974906 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fc009f650 0x7f5fc01c49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:55.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.734+0000 7f5fbeffd700 1 -- 172.21.15.3:0/3119974906 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fc009f240 msgr2=0x7f5fc01c44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:55.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.734+0000 7f5fbeffd700 1 --2- 172.21.15.3:0/3119974906 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fc009f240 0x7f5fc01c44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:55.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.734+0000 7f5fbeffd700 1 -- 172.21.15.3:0/3119974906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb400d940 con 0x7f5fc009e930 2023-10-03T10:15:55.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.734+0000 7f5fbeffd700 1 --2- 172.21.15.3:0/3119974906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fc009e930 0x7f5fc01c3f70 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000e930 tx=0x7f5fb000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:55.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.735+0000 7f5fa77fe700 1 -- 172.21.15.3:0/3119974906 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0013680 con 0x7f5fc009e930 2023-10-03T10:15:55.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.735+0000 7f5fc54a0700 1 -- 172.21.15.3:0/3119974906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb8009b30 con 0x7f5fc009e930 2023-10-03T10:15:55.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.735+0000 7f5fa77fe700 1 -- 172.21.15.3:0/3119974906 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb00137e0 con 0x7f5fc009e930 2023-10-03T10:15:55.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.735+0000 7f5fa77fe700 1 -- 172.21.15.3:0/3119974906 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0013b50 con 0x7f5fc009e930 2023-10-03T10:15:55.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.735+0000 7f5fc54a0700 1 -- 172.21.15.3:0/3119974906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc00a0190 con 0x7f5fc009e930 2023-10-03T10:15:55.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.736+0000 7f5fa77fe700 1 -- 172.21.15.3:0/3119974906 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5fb0013cb0 con 0x7f5fc009e930 2023-10-03T10:15:55.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.736+0000 7f5fa77fe700 1 -- 172.21.15.3:0/3119974906 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5fb003f7f0 con 0x7f5fc009e930 2023-10-03T10:15:55.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.737+0000 7f5fc54a0700 1 -- 172.21.15.3:0/3119974906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fc0074ea0 con 0x7f5fc009e930 2023-10-03T10:15:55.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.746+0000 7f5fa77fe700 1 -- 172.21.15.3:0/3119974906 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5fb0017020 con 0x7f5fc009e930 2023-10-03T10:15:55.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.776+0000 7f3c20465700 1 Processor -- start 2023-10-03T10:15:55.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.776+0000 7f3c20465700 1 -- start start 2023-10-03T10:15:55.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.777+0000 7f3c20465700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3c18136710 0x7f3c18138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:55.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.777+0000 7f3c20465700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3c18139030 0x7f3c1813b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:55.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.777+0000 7f3c20465700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c1806ef30 con 0x7f3c181338a0 2023-10-03T10:15:55.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.777+0000 7f3c20465700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c1806fef0 con 0x7f3c18136710 2023-10-03T10:15:55.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.777+0000 7f3c20465700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3c181334b0 con 0x7f3c18139030 2023-10-03T10:15:55.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.777+0000 7f3c1ea02700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3c18139030 0x7f3c1813b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:55.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.777+0000 7f3c1ea02700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3c18139030 0x7f3c1813b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:58366/0 (socket says 172.21.15.3:58366) 2023-10-03T10:15:55.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.777+0000 7f3c1ea02700 1 -- 172.21.15.3:0/3762894863 learned_addr learned my addr 172.21.15.3:0/3762894863 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:55.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.777+0000 7f3c1da00700 1 --2- 172.21.15.3:0/3762894863 >> v2:172.21.15.3:3300/0 conn(0x7f3c18136710 0x7f3c18138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:55.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.778+0000 7f3c1da00700 1 -- 172.21.15.3:0/3762894863 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3c18139030 msgr2=0x7f3c1813b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:55.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.778+0000 7f3c1da00700 1 --2- 172.21.15.3:0/3762894863 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3c18139030 0x7f3c1813b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:55.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.778+0000 7f3c1da00700 1 -- 172.21.15.3:0/3762894863 >> v1:172.21.15.3:6789/0 conn(0x7f3c181338a0 legacy=0x7f3c18133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:55.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.778+0000 7f3c1da00700 1 -- 172.21.15.3:0/3762894863 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c1806fc30 con 0x7f3c18136710 2023-10-03T10:15:55.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.778+0000 7f3c1ea02700 1 --2- 172.21.15.3:0/3762894863 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3c18139030 0x7f3c1813b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:55.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.778+0000 7f3c1da00700 1 --2- 172.21.15.3:0/3762894863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c18136710 0x7f3c18138af0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009840 tx=0x7f3c08009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68cd662d66d6a0f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:55.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.778+0000 7f3c1d1ff700 1 -- 172.21.15.3:0/3762894863 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c08011830 con 0x7f3c18136710 2023-10-03T10:15:55.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.778+0000 7f3c1d1ff700 1 -- 172.21.15.3:0/3762894863 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c08011990 con 0x7f3c18136710 2023-10-03T10:15:55.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.778+0000 7f3c1d1ff700 1 -- 172.21.15.3:0/3762894863 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c08011d20 con 0x7f3c18136710 2023-10-03T10:15:55.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.779+0000 7f3c20465700 1 -- 172.21.15.3:0/3762894863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c18136710 msgr2=0x7f3c18138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:55.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.779+0000 7f3c20465700 1 --2- 172.21.15.3:0/3762894863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c18136710 0x7f3c18138af0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009840 tx=0x7f3c08009b50 comp rx=0 tx=0).stop 2023-10-03T10:15:55.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.779+0000 7f3c20465700 1 -- 172.21.15.3:0/3762894863 shutdown_connections 2023-10-03T10:15:55.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.779+0000 7f3c20465700 1 --2- 172.21.15.3:0/3762894863 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3c18139030 0x7f3c1813b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:55.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.779+0000 7f3c20465700 1 --2- 172.21.15.3:0/3762894863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c18136710 0x7f3c18138af0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:55.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.779+0000 7f3c20465700 1 -- 172.21.15.3:0/3762894863 >> 172.21.15.3:0/3762894863 conn(0x7f3c180a2aa0 msgr2=0x7f3c180a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:55.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.779+0000 7f3c20465700 1 -- 172.21.15.3:0/3762894863 shutdown_connections 2023-10-03T10:15:55.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.779+0000 7f3c20465700 1 -- 172.21.15.3:0/3762894863 wait complete. 2023-10-03T10:15:55.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.781+0000 7f3c20465700 1 Processor -- start 2023-10-03T10:15:55.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.781+0000 7f3c20465700 1 -- start start 2023-10-03T10:15:55.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.782+0000 7f3c20465700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3c181338a0 0x7f3c1811ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:55.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.782+0000 7f3c20465700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3c18136710 0x7f3c1811b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:55.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.782+0000 7f3c20465700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c18139030 0x7f3c1811b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:55.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.782+0000 7f3c20465700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1811cc30 con 0x7f3c18139030 2023-10-03T10:15:55.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.782+0000 7f3c20465700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1811ca40 con 0x7f3c181338a0 2023-10-03T10:15:55.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.782+0000 7f3c20465700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1811e950 con 0x7f3c18136710 2023-10-03T10:15:55.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.782+0000 7f3c1e201700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3c181338a0 0x7f3c1811ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:55.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.782+0000 7f3c1ea02700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c18139030 0x7f3c1811b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:55.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.782+0000 7f3c1e201700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3c181338a0 0x7f3c1811ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38890/0 (socket says 172.21.15.3:38890) 2023-10-03T10:15:55.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.782+0000 7f3c1e201700 1 -- 172.21.15.3:0/592461852 learned_addr learned my addr 172.21.15.3:0/592461852 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:55.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.782+0000 7f3c1da00700 1 --2- 172.21.15.3:0/592461852 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3c18136710 0x7f3c1811b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:55.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.782+0000 7f3c1e201700 1 -- 172.21.15.3:0/592461852 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3c18136710 msgr2=0x7f3c1811b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:55.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.782+0000 7f3c1e201700 1 --2- 172.21.15.3:0/592461852 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3c18136710 0x7f3c1811b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:55.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.782+0000 7f3c1e201700 1 -- 172.21.15.3:0/592461852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c18139030 msgr2=0x7f3c1811b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:55.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.782+0000 7f3c1e201700 1 --2- 172.21.15.3:0/592461852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c18139030 0x7f3c1811b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:55.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.782+0000 7f3c1e201700 1 -- 172.21.15.3:0/592461852 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c10009b30 con 0x7f3c181338a0 2023-10-03T10:15:55.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.782+0000 7f3c1ea02700 1 --2- 172.21.15.3:0/592461852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c18139030 0x7f3c1811b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:55.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.783+0000 7f3c1e201700 1 --2- 172.21.15.3:0/592461852 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3c181338a0 0x7f3c1811ae30 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3c1400c4e0 tx=0x7f3c1400c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:55.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.783+0000 7f3c0effd700 1 -- 172.21.15.3:0/592461852 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c140052d0 con 0x7f3c181338a0 2023-10-03T10:15:55.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.783+0000 7f3c0effd700 1 -- 172.21.15.3:0/592461852 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c14005430 con 0x7f3c181338a0 2023-10-03T10:15:55.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.783+0000 7f3c20465700 1 -- 172.21.15.3:0/592461852 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c080094a0 con 0x7f3c181338a0 2023-10-03T10:15:55.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.783+0000 7f3c0effd700 1 -- 172.21.15.3:0/592461852 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c140038c0 con 0x7f3c181338a0 2023-10-03T10:15:55.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.783+0000 7f3c20465700 1 -- 172.21.15.3:0/592461852 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c1811e670 con 0x7f3c181338a0 2023-10-03T10:15:55.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.784+0000 7f3c20465700 1 -- 172.21.15.3:0/592461852 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c1811d290 con 0x7f3c181338a0 2023-10-03T10:15:55.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.784+0000 7f3c0effd700 1 -- 172.21.15.3:0/592461852 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3c14003a20 con 0x7f3c181338a0 2023-10-03T10:15:55.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.785+0000 7f3c0effd700 1 -- 172.21.15.3:0/592461852 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3c1403ebc0 con 0x7f3c181338a0 2023-10-03T10:15:55.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.787+0000 7f7881fbb700 1 Processor -- start 2023-10-03T10:15:55.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.787+0000 7f7881fbb700 1 -- start start 2023-10-03T10:15:55.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.787+0000 7f7881fbb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f787c134630 0x7f787c136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:55.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.788+0000 7f7881fbb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f787c136f50 0x7f787c139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:55.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.788+0000 7f7881fbb700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f787c071a00 con 0x7f787c131630 2023-10-03T10:15:55.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.788+0000 7f7881fbb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787c131430 con 0x7f787c134630 2023-10-03T10:15:55.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.788+0000 7f7881fbb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f787c09a510 con 0x7f787c136f50 2023-10-03T10:15:55.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.788+0000 7f787b7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f787c131630 0x7f787c131a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36130/0 (socket says 172.21.15.3:36130) 2023-10-03T10:15:55.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.788+0000 7f787b7fe700 1 -- 172.21.15.3:0/1447267603 learned_addr learned my addr 172.21.15.3:0/1447267603 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:55.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.788+0000 7f787bfff700 1 --2- 172.21.15.3:0/1447267603 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f787c136f50 0x7f787c139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:55.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.788+0000 7f787affd700 1 --2- 172.21.15.3:0/1447267603 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f787c134630 0x7f787c136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:55.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.788+0000 7f787bfff700 1 -- 172.21.15.3:0/1447267603 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f787c134630 msgr2=0x7f787c136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:55.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.788+0000 7f787bfff700 1 --2- 172.21.15.3:0/1447267603 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f787c134630 0x7f787c136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:55.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.788+0000 7f787bfff700 1 -- 172.21.15.3:0/1447267603 >> v1:172.21.15.3:6789/0 conn(0x7f787c131630 legacy=0x7f787c131a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:55.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.788+0000 7f787bfff700 1 -- 172.21.15.3:0/1447267603 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787c0721c0 con 0x7f787c136f50 2023-10-03T10:15:55.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.788+0000 7f787bfff700 1 --2- 172.21.15.3:0/1447267603 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f787c136f50 0x7f787c139330 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7874009dc0 tx=0x7f787400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df65b6519fb61f58 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:55.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.789+0000 7f787a7fc700 1 -- 172.21.15.3:0/1447267603 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787400e070 con 0x7f787c136f50 2023-10-03T10:15:55.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.789+0000 7f787a7fc700 1 -- 172.21.15.3:0/1447267603 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787400bb20 con 0x7f787c136f50 2023-10-03T10:15:55.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.789+0000 7f787a7fc700 1 -- 172.21.15.3:0/1447267603 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7874012430 con 0x7f787c136f50 2023-10-03T10:15:55.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.790+0000 7f7881fbb700 1 -- 172.21.15.3:0/1447267603 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f787c136f50 msgr2=0x7f787c139330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:55.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.790+0000 7f7881fbb700 1 --2- 172.21.15.3:0/1447267603 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f787c136f50 0x7f787c139330 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7874009dc0 tx=0x7f787400b330 comp rx=0 tx=0).stop 2023-10-03T10:15:55.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.790+0000 7f7881fbb700 1 -- 172.21.15.3:0/1447267603 shutdown_connections 2023-10-03T10:15:55.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.790+0000 7f7881fbb700 1 --2- 172.21.15.3:0/1447267603 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f787c136f50 0x7f787c139330 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:55.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.790+0000 7f7881fbb700 1 --2- 172.21.15.3:0/1447267603 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f787c134630 0x7f787c136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:55.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.790+0000 7f7881fbb700 1 -- 172.21.15.3:0/1447267603 >> 172.21.15.3:0/1447267603 conn(0x7f787c0a2aa0 msgr2=0x7f787c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:55.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.790+0000 7f7881fbb700 1 -- 172.21.15.3:0/1447267603 shutdown_connections 2023-10-03T10:15:55.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.790+0000 7f7881fbb700 1 -- 172.21.15.3:0/1447267603 wait complete. 2023-10-03T10:15:55.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.791+0000 7f3c0effd700 1 -- 172.21.15.3:0/592461852 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3c14018260 con 0x7f3c181338a0 2023-10-03T10:15:55.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.791+0000 7f7881fbb700 1 Processor -- start 2023-10-03T10:15:55.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.791+0000 7f7881fbb700 1 -- start start 2023-10-03T10:15:55.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.792+0000 7f7881fbb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f787c131630 0x7f787c118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:55.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.792+0000 7f7881fbb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f787c134630 0x7f787c119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:55.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.792+0000 7f7881fbb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f787c136f50 0x7f787c119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:55.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.792+0000 7f7881fbb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c09e3a0 con 0x7f787c131630 2023-10-03T10:15:55.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.792+0000 7f7881fbb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787c136d40 con 0x7f787c134630 2023-10-03T10:15:55.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.792+0000 7f7881fbb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f787c099f20 con 0x7f787c136f50 2023-10-03T10:15:55.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.792+0000 7f787affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f787c134630 0x7f787c119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:55.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.792+0000 7f787affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f787c134630 0x7f787c119210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38896/0 (socket says 172.21.15.3:38896) 2023-10-03T10:15:55.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.792+0000 7f787affd700 1 -- 172.21.15.3:0/1956776835 learned_addr learned my addr 172.21.15.3:0/1956776835 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:55.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.793+0000 7f787affd700 1 -- 172.21.15.3:0/1956776835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f787c136f50 msgr2=0x7f787c119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:55.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.793+0000 7f787affd700 1 --2- 172.21.15.3:0/1956776835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f787c136f50 0x7f787c119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:55.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.793+0000 7f787affd700 1 -- 172.21.15.3:0/1956776835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f787c131630 msgr2=0x7f787c118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:55.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.793+0000 7f787affd700 1 --2- 172.21.15.3:0/1956776835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f787c131630 0x7f787c118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:55.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.793+0000 7f787affd700 1 -- 172.21.15.3:0/1956776835 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7874009a70 con 0x7f787c134630 2023-10-03T10:15:55.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.793+0000 7f787affd700 1 --2- 172.21.15.3:0/1956776835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f787c134630 0x7f787c119210 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f787000e9b0 tx=0x7f787000ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:55.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.793+0000 7f7863fff700 1 -- 172.21.15.3:0/1956776835 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870009710 con 0x7f787c134630 2023-10-03T10:15:55.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.793+0000 7f7863fff700 1 -- 172.21.15.3:0/1956776835 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7870013860 con 0x7f787c134630 2023-10-03T10:15:55.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.793+0000 7f7881fbb700 1 -- 172.21.15.3:0/1956776835 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f787c11b480 con 0x7f787c134630 2023-10-03T10:15:55.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.793+0000 7f7863fff700 1 -- 172.21.15.3:0/1956776835 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870013bd0 con 0x7f787c134630 2023-10-03T10:15:55.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.793+0000 7f7881fbb700 1 -- 172.21.15.3:0/1956776835 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f787c09db40 con 0x7f787c134630 2023-10-03T10:15:55.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.794+0000 7f7863fff700 1 -- 172.21.15.3:0/1956776835 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7870013d30 con 0x7f787c134630 2023-10-03T10:15:55.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.794+0000 7f7881fbb700 1 -- 172.21.15.3:0/1956776835 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f787c074ea0 con 0x7f787c134630 2023-10-03T10:15:55.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.795+0000 7f7863fff700 1 -- 172.21.15.3:0/1956776835 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7870043880 con 0x7f787c134630 2023-10-03T10:15:55.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.804+0000 7f7863fff700 1 -- 172.21.15.3:0/1956776835 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f787001b440 con 0x7f787c134630 2023-10-03T10:15:55.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.949+0000 7f9dde778700 1 -- 172.21.15.3:0/4252743008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9dd806c360 con 0x7f9dd809e410 2023-10-03T10:15:55.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.950+0000 7f9dd4ff9700 1 -- 172.21.15.3:0/4252743008 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9dd00263e0 con 0x7f9dd809e410 2023-10-03T10:15:55.950 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:55.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.953+0000 7f9dde778700 1 -- 172.21.15.3:0/4252743008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dd809e410 msgr2=0x7f9dd81a6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:55.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.953+0000 7f9dde778700 1 --2- 172.21.15.3:0/4252743008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dd809e410 0x7f9dd81a6b20 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009d90 tx=0x7f9dd0004ad0 comp rx=0 tx=0).stop 2023-10-03T10:15:55.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.953+0000 7f9dde778700 1 -- 172.21.15.3:0/4252743008 shutdown_connections 2023-10-03T10:15:55.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.953+0000 7f9dde778700 1 --2- 172.21.15.3:0/4252743008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dd809e410 0x7f9dd81a6b20 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:55.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.953+0000 7f9dde778700 1 --2- 172.21.15.3:0/4252743008 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9dd809d6f0 0x7f9dd81a60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:55.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.953+0000 7f9dde778700 1 --2- 172.21.15.3:0/4252743008 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9dd809e000 0x7f9dd81a65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:55.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.953+0000 7f9dde778700 1 -- 172.21.15.3:0/4252743008 >> 172.21.15.3:0/4252743008 conn(0x7f9dd8066f50 msgr2=0x7f9dd81baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:55.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.954+0000 7f9dde778700 1 -- 172.21.15.3:0/4252743008 shutdown_connections 2023-10-03T10:15:55.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:55.954+0000 7f9dde778700 1 -- 172.21.15.3:0/4252743008 wait complete. 2023-10-03T10:15:55.968 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:56.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.001+0000 7f3c20465700 1 -- 172.21.15.3:0/592461852 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3c1806ea40 con 0x7f3c181338a0 2023-10-03T10:15:56.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.001+0000 7f3c0effd700 1 -- 172.21.15.3:0/592461852 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3c14017020 con 0x7f3c181338a0 2023-10-03T10:15:56.002 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:56.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.003+0000 7f3c20465700 1 -- 172.21.15.3:0/592461852 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3c181338a0 msgr2=0x7f3c1811ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:56.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.003+0000 7f3c20465700 1 --2- 172.21.15.3:0/592461852 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3c181338a0 0x7f3c1811ae30 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3c1400c4e0 tx=0x7f3c1400c7f0 comp rx=0 tx=0).stop 2023-10-03T10:15:56.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.003+0000 7f3c20465700 1 -- 172.21.15.3:0/592461852 shutdown_connections 2023-10-03T10:15:56.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.003+0000 7f3c20465700 1 --2- 172.21.15.3:0/592461852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c18139030 0x7f3c1811b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:56.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.004+0000 7f3c20465700 1 --2- 172.21.15.3:0/592461852 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3c181338a0 0x7f3c1811ae30 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:56.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.004+0000 7f3c20465700 1 --2- 172.21.15.3:0/592461852 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3c18136710 0x7f3c1811b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:56.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.004+0000 7f3c20465700 1 -- 172.21.15.3:0/592461852 >> 172.21.15.3:0/592461852 conn(0x7f3c180a2aa0 msgr2=0x7f3c181344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:56.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.004+0000 7f3c20465700 1 -- 172.21.15.3:0/592461852 shutdown_connections 2023-10-03T10:15:56.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.004+0000 7f3c20465700 1 -- 172.21.15.3:0/592461852 wait complete. 2023-10-03T10:15:56.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.006+0000 7f5fc54a0700 1 -- 172.21.15.3:0/3119974906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5fc006d800 con 0x7f5fc009e930 2023-10-03T10:15:56.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.006+0000 7f5fa77fe700 1 -- 172.21.15.3:0/3119974906 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5fb000c970 con 0x7f5fc009e930 2023-10-03T10:15:56.007 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:56.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.009+0000 7f5fc54a0700 1 -- 172.21.15.3:0/3119974906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fc009e930 msgr2=0x7f5fc01c3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:56.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.009+0000 7f5fc54a0700 1 --2- 172.21.15.3:0/3119974906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fc009e930 0x7f5fc01c3f70 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000e930 tx=0x7f5fb000ec40 comp rx=0 tx=0).stop 2023-10-03T10:15:56.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.009+0000 7f5fc54a0700 1 -- 172.21.15.3:0/3119974906 shutdown_connections 2023-10-03T10:15:56.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.009+0000 7f5fc54a0700 1 --2- 172.21.15.3:0/3119974906 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fc009f650 0x7f5fc01c49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:56.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.009+0000 7f5fc54a0700 1 --2- 172.21.15.3:0/3119974906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fc009e930 0x7f5fc01c3f70 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:56.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.009+0000 7f5fc54a0700 1 --2- 172.21.15.3:0/3119974906 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fc009f240 0x7f5fc01c44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:56.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.009+0000 7f5fc54a0700 1 -- 172.21.15.3:0/3119974906 >> 172.21.15.3:0/3119974906 conn(0x7f5fc00a4210 msgr2=0x7f5fc01b7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:56.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.009+0000 7f5fc54a0700 1 -- 172.21.15.3:0/3119974906 shutdown_connections 2023-10-03T10:15:56.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.009+0000 7f5fc54a0700 1 -- 172.21.15.3:0/3119974906 wait complete. 2023-10-03T10:15:56.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.010+0000 7f7881fbb700 1 -- 172.21.15.3:0/1956776835 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f787c06b9e0 con 0x7f787c134630 2023-10-03T10:15:56.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.011+0000 7f7863fff700 1 -- 172.21.15.3:0/1956776835 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7870020070 con 0x7f787c134630 2023-10-03T10:15:56.011 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:56.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.013+0000 7f7881fbb700 1 -- 172.21.15.3:0/1956776835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f787c134630 msgr2=0x7f787c119210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:56.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.013+0000 7f7881fbb700 1 --2- 172.21.15.3:0/1956776835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f787c134630 0x7f787c119210 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f787000e9b0 tx=0x7f787000ed70 comp rx=0 tx=0).stop 2023-10-03T10:15:56.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.013+0000 7f7881fbb700 1 -- 172.21.15.3:0/1956776835 shutdown_connections 2023-10-03T10:15:56.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.013+0000 7f7881fbb700 1 --2- 172.21.15.3:0/1956776835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f787c136f50 0x7f787c119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:56.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.013+0000 7f7881fbb700 1 --2- 172.21.15.3:0/1956776835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f787c131630 0x7f787c118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:56.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.013+0000 7f7881fbb700 1 --2- 172.21.15.3:0/1956776835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f787c134630 0x7f787c119210 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:56.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.013+0000 7f7881fbb700 1 -- 172.21.15.3:0/1956776835 >> 172.21.15.3:0/1956776835 conn(0x7f787c0a2aa0 msgr2=0x7f787c12e030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:56.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.014+0000 7f7881fbb700 1 -- 172.21.15.3:0/1956776835 shutdown_connections 2023-10-03T10:15:56.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:56.014+0000 7f7881fbb700 1 -- 172.21.15.3:0/1956776835 wait complete. 2023-10-03T10:15:56.021 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:56.026 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:56.029 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:56.969 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:57.022 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:57.026 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:57.030 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:57.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.121+0000 7f66d7139700 1 Processor -- start 2023-10-03T10:15:57.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.121+0000 7f66d7139700 1 -- start start 2023-10-03T10:15:57.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.121+0000 7f66d7139700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66d009d6f0 0x7f66d009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:57.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.121+0000 7f66d7139700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66d009e000 0x7f66d01339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:57.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.121+0000 7f66d7139700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66d009b120 con 0x7f66d009e410 2023-10-03T10:15:57.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.121+0000 7f66d7139700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d006b460 con 0x7f66d009d6f0 2023-10-03T10:15:57.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.121+0000 7f66d7139700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66d009a7e0 con 0x7f66d009e000 2023-10-03T10:15:57.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.122+0000 7f66d4ed5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66d009d6f0 0x7f66d009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:57.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.122+0000 7f66d56d6700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f66d009e410 0x7f66d0133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36138/0 (socket says 172.21.15.3:36138) 2023-10-03T10:15:57.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.122+0000 7f66d56d6700 1 -- 172.21.15.3:0/3272467665 learned_addr learned my addr 172.21.15.3:0/3272467665 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:57.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.122+0000 7f66cffff700 1 --2- 172.21.15.3:0/3272467665 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66d009e000 0x7f66d01339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:57.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.122+0000 7f66d4ed5700 1 -- 172.21.15.3:0/3272467665 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66d009e000 msgr2=0x7f66d01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:57.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.122+0000 7f66d4ed5700 1 --2- 172.21.15.3:0/3272467665 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66d009e000 0x7f66d01339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.122+0000 7f66d4ed5700 1 -- 172.21.15.3:0/3272467665 >> v1:172.21.15.3:6789/0 conn(0x7f66d009e410 legacy=0x7f66d0133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:57.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.122+0000 7f66d4ed5700 1 -- 172.21.15.3:0/3272467665 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d009a500 con 0x7f66d009d6f0 2023-10-03T10:15:57.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.122+0000 7f66cf7fe700 1 -- 172.21.15.3:0/3272467665 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3810578876 0 0) 0x7f66d009b120 con 0x7f66d009e410 2023-10-03T10:15:57.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.122+0000 7f66cffff700 1 --2- 172.21.15.3:0/3272467665 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66d009e000 0x7f66d01339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:57.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.122+0000 7f66d4ed5700 1 --2- 172.21.15.3:0/3272467665 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66d009d6f0 0x7f66d009dac0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f66c4009870 tx=0x7f66c4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a5c266da64f7d9e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:57.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.123+0000 7f66cf7fe700 1 -- 172.21.15.3:0/3272467665 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c4011830 con 0x7f66d009d6f0 2023-10-03T10:15:57.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.123+0000 7f66cf7fe700 1 -- 172.21.15.3:0/3272467665 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c4011990 con 0x7f66d009d6f0 2023-10-03T10:15:57.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.123+0000 7f66cf7fe700 1 -- 172.21.15.3:0/3272467665 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c4011d20 con 0x7f66d009d6f0 2023-10-03T10:15:57.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.124+0000 7f66d7139700 1 -- 172.21.15.3:0/3272467665 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66d009d6f0 msgr2=0x7f66d009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:57.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.124+0000 7f66d7139700 1 --2- 172.21.15.3:0/3272467665 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66d009d6f0 0x7f66d009dac0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f66c4009870 tx=0x7f66c4009b80 comp rx=0 tx=0).stop 2023-10-03T10:15:57.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.124+0000 7f66d7139700 1 -- 172.21.15.3:0/3272467665 shutdown_connections 2023-10-03T10:15:57.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.124+0000 7f66d7139700 1 --2- 172.21.15.3:0/3272467665 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66d009d6f0 0x7f66d009dac0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.124+0000 7f66d7139700 1 --2- 172.21.15.3:0/3272467665 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66d009e000 0x7f66d01339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.124+0000 7f66d7139700 1 -- 172.21.15.3:0/3272467665 >> 172.21.15.3:0/3272467665 conn(0x7f66d0066f50 msgr2=0x7f66d009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:57.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.124+0000 7f66d7139700 1 -- 172.21.15.3:0/3272467665 shutdown_connections 2023-10-03T10:15:57.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.124+0000 7f66d7139700 1 -- 172.21.15.3:0/3272467665 wait complete. 2023-10-03T10:15:57.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.125+0000 7f66d7139700 1 Processor -- start 2023-10-03T10:15:57.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.125+0000 7f66d7139700 1 -- start start 2023-10-03T10:15:57.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.126+0000 7f66d7139700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66d009d6f0 0x7f66d01224c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:57.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.126+0000 7f66d7139700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66d009e000 0x7f66d0122a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:57.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.126+0000 7f66d7139700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66d009e410 0x7f66d0122f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:57.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.126+0000 7f66d7139700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d0124f40 con 0x7f66d009e410 2023-10-03T10:15:57.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.126+0000 7f66d7139700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d0125f50 con 0x7f66d009d6f0 2023-10-03T10:15:57.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.126+0000 7f66d7139700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66d0125a40 con 0x7f66d009e000 2023-10-03T10:15:57.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.126+0000 7f66d4ed5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66d009d6f0 0x7f66d01224c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:57.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.126+0000 7f66d4ed5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66d009d6f0 0x7f66d01224c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38910/0 (socket says 172.21.15.3:38910) 2023-10-03T10:15:57.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.126+0000 7f66d4ed5700 1 -- 172.21.15.3:0/2353461275 learned_addr learned my addr 172.21.15.3:0/2353461275 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:57.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.127+0000 7f66cffff700 1 --2- 172.21.15.3:0/2353461275 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66d009e000 0x7f66d0122a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:57.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.127+0000 7f66d56d6700 1 --2- 172.21.15.3:0/2353461275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66d009e410 0x7f66d0122f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:57.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.127+0000 7f66cffff700 1 -- 172.21.15.3:0/2353461275 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66d009d6f0 msgr2=0x7f66d01224c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:57.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.127+0000 7f66cffff700 1 --2- 172.21.15.3:0/2353461275 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66d009d6f0 0x7f66d01224c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.127+0000 7f66cffff700 1 -- 172.21.15.3:0/2353461275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66d009e410 msgr2=0x7f66d0122f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:57.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.127+0000 7f66cffff700 1 --2- 172.21.15.3:0/2353461275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66d009e410 0x7f66d0122f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.127+0000 7f66cffff700 1 -- 172.21.15.3:0/2353461275 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66bc0094a0 con 0x7f66d009e000 2023-10-03T10:15:57.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.127+0000 7f66cffff700 1 --2- 172.21.15.3:0/2353461275 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66d009e000 0x7f66d0122a00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00e9b0 tx=0x7f66bc00ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:57.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.127+0000 7f66cd7fa700 1 -- 172.21.15.3:0/2353461275 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc00ee50 con 0x7f66d009e000 2023-10-03T10:15:57.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.127+0000 7f66cd7fa700 1 -- 172.21.15.3:0/2353461275 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66bc00c850 con 0x7f66d009e000 2023-10-03T10:15:57.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.127+0000 7f66d7139700 1 -- 172.21.15.3:0/2353461275 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c40094a0 con 0x7f66d009e000 2023-10-03T10:15:57.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.127+0000 7f66cd7fa700 1 -- 172.21.15.3:0/2353461275 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc00cbe0 con 0x7f66d009e000 2023-10-03T10:15:57.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.128+0000 7f66d7139700 1 -- 172.21.15.3:0/2353461275 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d0133800 con 0x7f66d009e000 2023-10-03T10:15:57.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.128+0000 7f66cd7fa700 1 -- 172.21.15.3:0/2353461275 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f66bc00cd40 con 0x7f66d009e000 2023-10-03T10:15:57.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.129+0000 7f66cd7fa700 1 -- 172.21.15.3:0/2353461275 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f66bc03f820 con 0x7f66d009e000 2023-10-03T10:15:57.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.129+0000 7f66d7139700 1 -- 172.21.15.3:0/2353461275 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66d011f030 con 0x7f66d009e000 2023-10-03T10:15:57.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.138+0000 7f66cd7fa700 1 -- 172.21.15.3:0/2353461275 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f66bc0232f0 con 0x7f66d009e000 2023-10-03T10:15:57.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.175+0000 7fb19f8c2700 1 Processor -- start 2023-10-03T10:15:57.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.175+0000 7fb19f8c2700 1 -- start start 2023-10-03T10:15:57.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.176+0000 7fb19f8c2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb198133540 0x7fb198135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:57.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.176+0000 7fb19f8c2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb198135e60 0x7fb198138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:57.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.176+0000 7fb19f8c2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1980716a0 con 0x7fb198130540 2023-10-03T10:15:57.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.176+0000 7fb19f8c2700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb198071290 con 0x7fb198133540 2023-10-03T10:15:57.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.176+0000 7fb19f8c2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb198130340 con 0x7fb198135e60 2023-10-03T10:15:57.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.176+0000 7fb19ce5d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb198133540 0x7fb198135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:57.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.176+0000 7fb19ce5d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb198133540 0x7fb198135920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60952/0 (socket says 172.21.15.3:60952) 2023-10-03T10:15:57.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.176+0000 7fb19ce5d700 1 -- 172.21.15.3:0/4190369273 learned_addr learned my addr 172.21.15.3:0/4190369273 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:57.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.176+0000 7fb19de5f700 1 --2- 172.21.15.3:0/4190369273 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb198135e60 0x7fb198138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:57.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.177+0000 7fb19ce5d700 1 -- 172.21.15.3:0/4190369273 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb198135e60 msgr2=0x7fb198138240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:57.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.177+0000 7fb19ce5d700 1 --2- 172.21.15.3:0/4190369273 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb198135e60 0x7fb198138240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.177+0000 7fb19ce5d700 1 -- 172.21.15.3:0/4190369273 >> v1:172.21.15.3:6789/0 conn(0x7fb198130540 legacy=0x7fb198130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:57.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.177+0000 7fb19ce5d700 1 -- 172.21.15.3:0/4190369273 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb198070b80 con 0x7fb198133540 2023-10-03T10:15:57.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.177+0000 7fb19ce5d700 1 --2- 172.21.15.3:0/4190369273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb198133540 0x7fb198135920 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fb1880098f0 tx=0x7fb188009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feefbf7be04da57b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:57.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.178+0000 7fb18ffff700 1 -- 172.21.15.3:0/4190369273 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb188011830 con 0x7fb198133540 2023-10-03T10:15:57.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.178+0000 7fb18ffff700 1 -- 172.21.15.3:0/4190369273 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb188011990 con 0x7fb198133540 2023-10-03T10:15:57.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.178+0000 7fb18ffff700 1 -- 172.21.15.3:0/4190369273 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb188011d20 con 0x7fb198133540 2023-10-03T10:15:57.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.178+0000 7fb19f8c2700 1 -- 172.21.15.3:0/4190369273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb198133540 msgr2=0x7fb198135920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:57.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.178+0000 7fb19f8c2700 1 --2- 172.21.15.3:0/4190369273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb198133540 0x7fb198135920 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fb1880098f0 tx=0x7fb188009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:57.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.179+0000 7fb19f8c2700 1 -- 172.21.15.3:0/4190369273 shutdown_connections 2023-10-03T10:15:57.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.179+0000 7fb19f8c2700 1 --2- 172.21.15.3:0/4190369273 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb198135e60 0x7fb198138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.179+0000 7fb19f8c2700 1 --2- 172.21.15.3:0/4190369273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb198133540 0x7fb198135920 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.179+0000 7fb19f8c2700 1 -- 172.21.15.3:0/4190369273 >> 172.21.15.3:0/4190369273 conn(0x7fb1980a2aa0 msgr2=0x7fb1980a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:57.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.179+0000 7fb19f8c2700 1 -- 172.21.15.3:0/4190369273 shutdown_connections 2023-10-03T10:15:57.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.179+0000 7fb19f8c2700 1 -- 172.21.15.3:0/4190369273 wait complete. 2023-10-03T10:15:57.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.180+0000 7fb19f8c2700 1 Processor -- start 2023-10-03T10:15:57.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.180+0000 7fb19f8c2700 1 -- start start 2023-10-03T10:15:57.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.180+0000 7fb19f8c2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb198130540 0x7fb19811a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:57.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.180+0000 7fb19f8c2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb198133540 0x7fb19809c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:57.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.180+0000 7fb19f8c2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb198135e60 0x7fb19809c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:57.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.180+0000 7fb19f8c2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19812cc30 con 0x7fb198130540 2023-10-03T10:15:57.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.180+0000 7fb19f8c2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb19811a1d0 con 0x7fb198135e60 2023-10-03T10:15:57.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.180+0000 7fb19f8c2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb198119080 con 0x7fb198133540 2023-10-03T10:15:57.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.181+0000 7fb19de5f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb198135e60 0x7fb19809c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:57.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.181+0000 7fb19de5f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb198135e60 0x7fb19809c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38926/0 (socket says 172.21.15.3:38926) 2023-10-03T10:15:57.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.181+0000 7fb19de5f700 1 -- 172.21.15.3:0/2282469403 learned_addr learned my addr 172.21.15.3:0/2282469403 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:57.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.181+0000 7fb19ce5d700 1 --2- 172.21.15.3:0/2282469403 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb198133540 0x7fb19809c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:57.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.181+0000 7fb19d65e700 1 --2- 172.21.15.3:0/2282469403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb198130540 0x7fb19811a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:57.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.181+0000 7fb19ce5d700 1 -- 172.21.15.3:0/2282469403 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb198135e60 msgr2=0x7fb19809c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:57.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.181+0000 7fb19ce5d700 1 --2- 172.21.15.3:0/2282469403 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb198135e60 0x7fb19809c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.181+0000 7fb19ce5d700 1 -- 172.21.15.3:0/2282469403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb198130540 msgr2=0x7fb19811a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:57.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.181+0000 7fb19ce5d700 1 --2- 172.21.15.3:0/2282469403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb198130540 0x7fb19811a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.181+0000 7fb19ce5d700 1 -- 172.21.15.3:0/2282469403 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb188009550 con 0x7fb198133540 2023-10-03T10:15:57.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.181+0000 7fb19ce5d700 1 --2- 172.21.15.3:0/2282469403 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb198133540 0x7fb19809c4b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb188016000 tx=0x7fb188004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:57.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.182+0000 7fb18dffb700 1 -- 172.21.15.3:0/2282469403 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1880036a0 con 0x7fb198133540 2023-10-03T10:15:57.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.182+0000 7fb18dffb700 1 -- 172.21.15.3:0/2282469403 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb188003800 con 0x7fb198133540 2023-10-03T10:15:57.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.182+0000 7fb19f8c2700 1 -- 172.21.15.3:0/2282469403 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb198119310 con 0x7fb198133540 2023-10-03T10:15:57.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.182+0000 7fb18dffb700 1 -- 172.21.15.3:0/2282469403 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb188003bb0 con 0x7fb198133540 2023-10-03T10:15:57.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.182+0000 7fb19f8c2700 1 -- 172.21.15.3:0/2282469403 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb19806e770 con 0x7fb198133540 2023-10-03T10:15:57.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.183+0000 7fb18dffb700 1 -- 172.21.15.3:0/2282469403 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb188021070 con 0x7fb198133540 2023-10-03T10:15:57.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.183+0000 7fb18dffb700 1 -- 172.21.15.3:0/2282469403 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb188040a60 con 0x7fb198133540 2023-10-03T10:15:57.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.184+0000 7fb19f8c2700 1 -- 172.21.15.3:0/2282469403 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb198074ea0 con 0x7fb198133540 2023-10-03T10:15:57.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.194+0000 7fb18dffb700 1 -- 172.21.15.3:0/2282469403 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb188015080 con 0x7fb198133540 2023-10-03T10:15:57.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.216+0000 7f5f3f257700 1 Processor -- start 2023-10-03T10:15:57.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.216+0000 7f5f3f257700 1 -- start start 2023-10-03T10:15:57.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.217+0000 7f5f3f257700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f380a0a30 0x7f5f380a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:57.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.217+0000 7f5f3f257700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f380a1340 0x7f5f38133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:57.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.217+0000 7f5f3f257700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5f38133fa0 0x7f5f38134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:57.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.217+0000 7f5f3f257700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f3809edd0 con 0x7f5f38133fa0 2023-10-03T10:15:57.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.217+0000 7f5f3f257700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3809e1f0 con 0x7f5f380a1340 2023-10-03T10:15:57.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.217+0000 7f5f3f257700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3809dc50 con 0x7f5f380a0a30 2023-10-03T10:15:57.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.217+0000 7f5f3cff3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f380a0a30 0x7f5f380a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:57.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.217+0000 7f5f3cff3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f380a0a30 0x7f5f380a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:58432/0 (socket says 172.21.15.3:58432) 2023-10-03T10:15:57.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.217+0000 7f5f3cff3700 1 -- 172.21.15.3:0/2510189404 learned_addr learned my addr 172.21.15.3:0/2510189404 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:57.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.217+0000 7f5f37fff700 1 --2- 172.21.15.3:0/2510189404 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f380a1340 0x7f5f38133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:57.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.217+0000 7f5f3d7f4700 1 --2- 172.21.15.3:0/2510189404 >> v2:172.21.15.3:3300/0 conn(0x7f5f38133fa0 0x7f5f38134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:57.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.217+0000 7f5f3cff3700 1 -- 172.21.15.3:0/2510189404 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f380a1340 msgr2=0x7f5f38133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:57.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.217+0000 7f5f3cff3700 1 --2- 172.21.15.3:0/2510189404 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f380a1340 0x7f5f38133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.217+0000 7f5f3cff3700 1 -- 172.21.15.3:0/2510189404 >> v2:172.21.15.3:3300/0 conn(0x7f5f38133fa0 msgr2=0x7f5f38134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:57.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.217+0000 7f5f3cff3700 1 --2- 172.21.15.3:0/2510189404 >> v2:172.21.15.3:3300/0 conn(0x7f5f38133fa0 0x7f5f38134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.217+0000 7f5f3cff3700 1 -- 172.21.15.3:0/2510189404 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f380a1fe0 con 0x7f5f380a0a30 2023-10-03T10:15:57.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.217+0000 7f5f3d7f4700 1 --2- 172.21.15.3:0/2510189404 >> v2:172.21.15.3:3300/0 conn(0x7f5f38133fa0 0x7f5f38134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:57.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.217+0000 7f5f37fff700 1 --2- 172.21.15.3:0/2510189404 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f380a1340 0x7f5f38133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:57.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.217+0000 7f5f3cff3700 1 --2- 172.21.15.3:0/2510189404 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f380a0a30 0x7f5f380a0e00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c0098f0 tx=0x7f5f2c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=542146c958d8efa4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:57.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.218+0000 7f5f377fe700 1 -- 172.21.15.3:0/2510189404 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c011830 con 0x7f5f380a0a30 2023-10-03T10:15:57.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.218+0000 7f5f377fe700 1 -- 172.21.15.3:0/2510189404 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2c011990 con 0x7f5f380a0a30 2023-10-03T10:15:57.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.218+0000 7f5f377fe700 1 -- 172.21.15.3:0/2510189404 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c011d20 con 0x7f5f380a0a30 2023-10-03T10:15:57.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.218+0000 7fa6b6236700 1 Processor -- start 2023-10-03T10:15:57.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.219+0000 7f5f3f257700 1 -- 172.21.15.3:0/2510189404 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f380a0a30 msgr2=0x7f5f380a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:57.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.219+0000 7f5f3f257700 1 --2- 172.21.15.3:0/2510189404 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f380a0a30 0x7f5f380a0e00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c0098f0 tx=0x7f5f2c009c00 comp rx=0 tx=0).stop 2023-10-03T10:15:57.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.219+0000 7fa6b6236700 1 -- start start 2023-10-03T10:15:57.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.219+0000 7f5f3f257700 1 -- 172.21.15.3:0/2510189404 shutdown_connections 2023-10-03T10:15:57.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.219+0000 7f5f3f257700 1 --2- 172.21.15.3:0/2510189404 >> v2:172.21.15.3:3300/0 conn(0x7f5f38133fa0 0x7f5f38134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.219+0000 7f5f3f257700 1 --2- 172.21.15.3:0/2510189404 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f380a0a30 0x7f5f380a0e00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.219+0000 7f5f3f257700 1 --2- 172.21.15.3:0/2510189404 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f380a1340 0x7f5f38133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.219+0000 7f5f3f257700 1 -- 172.21.15.3:0/2510189404 >> 172.21.15.3:0/2510189404 conn(0x7f5f380a6550 msgr2=0x7f5f380a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:57.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.219+0000 7fa6b6236700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa6b009d6f0 0x7fa6b009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:57.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.219+0000 7fa6b6236700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa6b009ace0 0x7fa6b009e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:57.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.219+0000 7fa6b6236700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa6b01339f0 0x7fa6b0133e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:57.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.219+0000 7fa6b6236700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6b009b150 con 0x7fa6b009d6f0 2023-10-03T10:15:57.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.219+0000 7fa6b6236700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b009a500 con 0x7fa6b01339f0 2023-10-03T10:15:57.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.219+0000 7fa6b6236700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b009e540 con 0x7fa6b009ace0 2023-10-03T10:15:57.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.220+0000 7f5f3f257700 1 -- 172.21.15.3:0/2510189404 shutdown_connections 2023-10-03T10:15:57.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.220+0000 7fa6aeffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa6b009ace0 0x7fa6b009e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:57.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.220+0000 7fa6affff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa6b01339f0 0x7fa6b0133e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:57.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.220+0000 7fa6aeffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa6b009ace0 0x7fa6b009e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:58440/0 (socket says 172.21.15.3:58440) 2023-10-03T10:15:57.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.220+0000 7fa6aeffd700 1 -- 172.21.15.3:0/1296497176 learned_addr learned my addr 172.21.15.3:0/1296497176 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:57.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.220+0000 7fa6af7fe700 1 --2- 172.21.15.3:0/1296497176 >> v2:172.21.15.3:3300/0 conn(0x7fa6b009d6f0 0x7fa6b009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:57.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.220+0000 7fa6af7fe700 1 -- 172.21.15.3:0/1296497176 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa6b009ace0 msgr2=0x7fa6b009e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:57.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.220+0000 7fa6af7fe700 1 --2- 172.21.15.3:0/1296497176 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa6b009ace0 0x7fa6b009e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.220+0000 7fa6af7fe700 1 -- 172.21.15.3:0/1296497176 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa6b01339f0 msgr2=0x7fa6b0133e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:57.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.220+0000 7fa6af7fe700 1 --2- 172.21.15.3:0/1296497176 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa6b01339f0 0x7fa6b0133e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.220+0000 7fa6af7fe700 1 -- 172.21.15.3:0/1296497176 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b009f720 con 0x7fa6b009d6f0 2023-10-03T10:15:57.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.220+0000 7fa6af7fe700 1 --2- 172.21.15.3:0/1296497176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6b009d6f0 0x7fa6b009dac0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009840 tx=0x7fa6a0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562cc2d6d0a5b624 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:57.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.220+0000 7fa6ae7fc700 1 -- 172.21.15.3:0/1296497176 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0011830 con 0x7fa6b009d6f0 2023-10-03T10:15:57.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.221+0000 7fa6ae7fc700 1 -- 172.21.15.3:0/1296497176 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a0011990 con 0x7fa6b009d6f0 2023-10-03T10:15:57.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.221+0000 7fa6ae7fc700 1 -- 172.21.15.3:0/1296497176 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0011d20 con 0x7fa6b009d6f0 2023-10-03T10:15:57.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.220+0000 7f5f3f257700 1 -- 172.21.15.3:0/2510189404 wait complete. 2023-10-03T10:15:57.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.221+0000 7f5f3f257700 1 Processor -- start 2023-10-03T10:15:57.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.221+0000 7f5f3f257700 1 -- start start 2023-10-03T10:15:57.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.221+0000 7fa6b6236700 1 -- 172.21.15.3:0/1296497176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6b009d6f0 msgr2=0x7fa6b009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:57.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.221+0000 7fa6b6236700 1 --2- 172.21.15.3:0/1296497176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6b009d6f0 0x7fa6b009dac0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009840 tx=0x7fa6a0009b50 comp rx=0 tx=0).stop 2023-10-03T10:15:57.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.222+0000 7fa6b6236700 1 -- 172.21.15.3:0/1296497176 shutdown_connections 2023-10-03T10:15:57.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.222+0000 7fa6b6236700 1 --2- 172.21.15.3:0/1296497176 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa6b01339f0 0x7fa6b0133e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.222+0000 7fa6b6236700 1 --2- 172.21.15.3:0/1296497176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6b009d6f0 0x7fa6b009dac0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.222+0000 7fa6b6236700 1 --2- 172.21.15.3:0/1296497176 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa6b009ace0 0x7fa6b009e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.222+0000 7fa6b6236700 1 -- 172.21.15.3:0/1296497176 >> 172.21.15.3:0/1296497176 conn(0x7fa6b0066f50 msgr2=0x7fa6b009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:57.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.222+0000 7fa6b6236700 1 -- 172.21.15.3:0/1296497176 shutdown_connections 2023-10-03T10:15:57.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.222+0000 7fa6b6236700 1 -- 172.21.15.3:0/1296497176 wait complete. 2023-10-03T10:15:57.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.222+0000 7f5f3f257700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f380a0a30 0x7f5f381200a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:57.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.222+0000 7f5f3f257700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f380a1340 0x7f5f381205e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:57.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.222+0000 7f5f3f257700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f38133fa0 0x7f5f38120b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:57.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.222+0000 7f5f3f257700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f38123bc0 con 0x7f5f380a0a30 2023-10-03T10:15:57.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.222+0000 7f5f3f257700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f38122f90 con 0x7f5f38133fa0 2023-10-03T10:15:57.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.222+0000 7f5f3f257700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f380a34f0 con 0x7f5f380a1340 2023-10-03T10:15:57.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.222+0000 7f5f3cff3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f380a0a30 0x7f5f381200a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:57.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.222+0000 7f5f3cff3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f380a0a30 0x7f5f381200a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60992/0 (socket says 172.21.15.3:60992) 2023-10-03T10:15:57.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.222+0000 7f5f3cff3700 1 -- 172.21.15.3:0/1056586357 learned_addr learned my addr 172.21.15.3:0/1056586357 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:57.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.222+0000 7f5f37fff700 1 --2- 172.21.15.3:0/1056586357 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f380a1340 0x7f5f381205e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:57.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.222+0000 7f5f3d7f4700 1 --2- 172.21.15.3:0/1056586357 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f38133fa0 0x7f5f38120b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:57.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.222+0000 7f5f3cff3700 1 -- 172.21.15.3:0/1056586357 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f380a1340 msgr2=0x7f5f381205e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:57.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.223+0000 7f5f3cff3700 1 --2- 172.21.15.3:0/1056586357 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f380a1340 0x7f5f381205e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.223+0000 7f5f3cff3700 1 -- 172.21.15.3:0/1056586357 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f38133fa0 msgr2=0x7f5f38120b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:57.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.223+0000 7f5f3cff3700 1 --2- 172.21.15.3:0/1056586357 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f38133fa0 0x7f5f38120b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.223+0000 7f5f3cff3700 1 -- 172.21.15.3:0/1056586357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f240094a0 con 0x7f5f380a0a30 2023-10-03T10:15:57.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.223+0000 7f5f37fff700 1 --2- 172.21.15.3:0/1056586357 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f380a1340 0x7f5f381205e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:15:57.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.223+0000 7f5f3cff3700 1 --2- 172.21.15.3:0/1056586357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f380a0a30 0x7f5f381200a0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c000c00 tx=0x7f5f2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:57.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.223+0000 7f5f357fa700 1 -- 172.21.15.3:0/1056586357 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c004300 con 0x7f5f380a0a30 2023-10-03T10:15:57.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.223+0000 7f5f357fa700 1 -- 172.21.15.3:0/1056586357 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2c01f070 con 0x7f5f380a0a30 2023-10-03T10:15:57.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.223+0000 7f5f3f257700 1 -- 172.21.15.3:0/1056586357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f30009a70 con 0x7f5f380a0a30 2023-10-03T10:15:57.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.223+0000 7f5f357fa700 1 -- 172.21.15.3:0/1056586357 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c0036a0 con 0x7f5f380a0a30 2023-10-03T10:15:57.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.223+0000 7f5f3f257700 1 -- 172.21.15.3:0/1056586357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2c009550 con 0x7f5f380a0a30 2023-10-03T10:15:57.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.223+0000 7fa6b6236700 1 Processor -- start 2023-10-03T10:15:57.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.224+0000 7f5f3f257700 1 -- 172.21.15.3:0/1056586357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f38121d10 con 0x7f5f380a0a30 2023-10-03T10:15:57.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.224+0000 7f5f357fa700 1 -- 172.21.15.3:0/1056586357 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5f2c021030 con 0x7f5f380a0a30 2023-10-03T10:15:57.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.224+0000 7fa6b6236700 1 -- start start 2023-10-03T10:15:57.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.225+0000 7f5f357fa700 1 -- 172.21.15.3:0/1056586357 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5f2c0408b0 con 0x7f5f380a0a30 2023-10-03T10:15:57.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.225+0000 7fa6b6236700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6b009ace0 0x7fa6b011f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:57.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.225+0000 7fa6b6236700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa6b009d6f0 0x7fa6b011f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:57.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.225+0000 7fa6b6236700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa6b01339f0 0x7fa6b01c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:57.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.225+0000 7fa6b6236700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b0121fe0 con 0x7fa6b009ace0 2023-10-03T10:15:57.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.225+0000 7fa6b6236700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b0133360 con 0x7fa6b01339f0 2023-10-03T10:15:57.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.225+0000 7fa6b6236700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b011eee0 con 0x7fa6b009d6f0 2023-10-03T10:15:57.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.225+0000 7fa6af7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6b009ace0 0x7fa6b011f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:57.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.225+0000 7fa6af7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6b009ace0 0x7fa6b011f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32772/0 (socket says 172.21.15.3:32772) 2023-10-03T10:15:57.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.225+0000 7fa6af7fe700 1 -- 172.21.15.3:0/1231713969 learned_addr learned my addr 172.21.15.3:0/1231713969 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:57.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.225+0000 7fa6aeffd700 1 --2- 172.21.15.3:0/1231713969 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa6b009d6f0 0x7fa6b011f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:57.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.226+0000 7fa6af7fe700 1 -- 172.21.15.3:0/1231713969 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa6b009d6f0 msgr2=0x7fa6b011f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:57.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.226+0000 7fa6af7fe700 1 --2- 172.21.15.3:0/1231713969 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa6b009d6f0 0x7fa6b011f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.226+0000 7fa6af7fe700 1 -- 172.21.15.3:0/1231713969 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa6b01339f0 msgr2=0x7fa6b01c8010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:15:57.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.226+0000 7fa6af7fe700 1 --2- 172.21.15.3:0/1231713969 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa6b01339f0 0x7fa6b01c8010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.226+0000 7fa6af7fe700 1 -- 172.21.15.3:0/1231713969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a00094a0 con 0x7fa6b009ace0 2023-10-03T10:15:57.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.226+0000 7fa6af7fe700 1 --2- 172.21.15.3:0/1231713969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6b009ace0 0x7fa6b011f480 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0000c00 tx=0x7fa6a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:57.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.230+0000 7fa697fff700 1 -- 172.21.15.3:0/1231713969 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0004240 con 0x7fa6b009ace0 2023-10-03T10:15:57.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.230+0000 7fa697fff700 1 -- 172.21.15.3:0/1231713969 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a001f070 con 0x7fa6b009ace0 2023-10-03T10:15:57.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.230+0000 7fa697fff700 1 -- 172.21.15.3:0/1231713969 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a00045a0 con 0x7fa6b009ace0 2023-10-03T10:15:57.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.230+0000 7f5f357fa700 1 -- 172.21.15.3:0/1056586357 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5f2c0199a0 con 0x7f5f380a0a30 2023-10-03T10:15:57.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.231+0000 7fa6b6236700 1 -- 172.21.15.3:0/1231713969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b0122210 con 0x7fa6b009ace0 2023-10-03T10:15:57.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.231+0000 7fa6b6236700 1 -- 172.21.15.3:0/1231713969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b00a2700 con 0x7fa6b009ace0 2023-10-03T10:15:57.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.232+0000 7fa697fff700 1 -- 172.21.15.3:0/1231713969 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa6a0021030 con 0x7fa6b009ace0 2023-10-03T10:15:57.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.232+0000 7fa6b6236700 1 -- 172.21.15.3:0/1231713969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6b0120bb0 con 0x7fa6b009ace0 2023-10-03T10:15:57.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.232+0000 7fa697fff700 1 -- 172.21.15.3:0/1231713969 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa6a0040870 con 0x7fa6b009ace0 2023-10-03T10:15:57.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.237+0000 7fa697fff700 1 -- 172.21.15.3:0/1231713969 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa6a0015080 con 0x7fa6b009ace0 2023-10-03T10:15:57.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.407+0000 7f66d7139700 1 -- 172.21.15.3:0/2353461275 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f66d0074ea0 con 0x7f66d009e000 2023-10-03T10:15:57.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.407+0000 7f66cd7fa700 1 -- 172.21.15.3:0/2353461275 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f66bc02b090 con 0x7f66d009e000 2023-10-03T10:15:57.408 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:57.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.410+0000 7f66d7139700 1 -- 172.21.15.3:0/2353461275 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66d009e000 msgr2=0x7f66d0122a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:57.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.410+0000 7f66d7139700 1 --2- 172.21.15.3:0/2353461275 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66d009e000 0x7f66d0122a00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00e9b0 tx=0x7f66bc00ecc0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.410+0000 7f66d7139700 1 -- 172.21.15.3:0/2353461275 shutdown_connections 2023-10-03T10:15:57.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.410+0000 7f66d7139700 1 --2- 172.21.15.3:0/2353461275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66d009e410 0x7f66d0122f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.410+0000 7f66d7139700 1 --2- 172.21.15.3:0/2353461275 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66d009d6f0 0x7f66d01224c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.410+0000 7f66d7139700 1 --2- 172.21.15.3:0/2353461275 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66d009e000 0x7f66d0122a00 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.410+0000 7f66d7139700 1 -- 172.21.15.3:0/2353461275 >> 172.21.15.3:0/2353461275 conn(0x7f66d0066f50 msgr2=0x7f66d0133e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:57.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.410+0000 7f66d7139700 1 -- 172.21.15.3:0/2353461275 shutdown_connections 2023-10-03T10:15:57.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.410+0000 7f66d7139700 1 -- 172.21.15.3:0/2353461275 wait complete. 2023-10-03T10:15:57.424 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:57.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.437+0000 7f5f3f257700 1 -- 172.21.15.3:0/1056586357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5f38074ea0 con 0x7f5f380a0a30 2023-10-03T10:15:57.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.438+0000 7f5f357fa700 1 -- 172.21.15.3:0/1056586357 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5f2c015030 con 0x7f5f380a0a30 2023-10-03T10:15:57.438 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:57.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.440+0000 7f5f3f257700 1 -- 172.21.15.3:0/1056586357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f380a0a30 msgr2=0x7f5f381200a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:57.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.440+0000 7f5f3f257700 1 --2- 172.21.15.3:0/1056586357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f380a0a30 0x7f5f381200a0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c000c00 tx=0x7f5f2c004750 comp rx=0 tx=0).stop 2023-10-03T10:15:57.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.440+0000 7f5f3f257700 1 -- 172.21.15.3:0/1056586357 shutdown_connections 2023-10-03T10:15:57.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.440+0000 7f5f3f257700 1 --2- 172.21.15.3:0/1056586357 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f38133fa0 0x7f5f38120b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.440+0000 7f5f3f257700 1 --2- 172.21.15.3:0/1056586357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f380a0a30 0x7f5f381200a0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.440+0000 7f5f3f257700 1 --2- 172.21.15.3:0/1056586357 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f380a1340 0x7f5f381205e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.440+0000 7f5f3f257700 1 -- 172.21.15.3:0/1056586357 >> 172.21.15.3:0/1056586357 conn(0x7f5f380a6550 msgr2=0x7f5f381bae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:57.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.440+0000 7f5f3f257700 1 -- 172.21.15.3:0/1056586357 shutdown_connections 2023-10-03T10:15:57.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.440+0000 7f5f3f257700 1 -- 172.21.15.3:0/1056586357 wait complete. 2023-10-03T10:15:57.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.443+0000 7fa6b6236700 1 -- 172.21.15.3:0/1231713969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa6b00747f0 con 0x7fa6b009ace0 2023-10-03T10:15:57.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.443+0000 7fa697fff700 1 -- 172.21.15.3:0/1231713969 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa6a00199a0 con 0x7fa6b009ace0 2023-10-03T10:15:57.444 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:57.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.446+0000 7fa6b6236700 1 -- 172.21.15.3:0/1231713969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6b009ace0 msgr2=0x7fa6b011f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:57.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.446+0000 7fa6b6236700 1 --2- 172.21.15.3:0/1231713969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6b009ace0 0x7fa6b011f480 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0000c00 tx=0x7fa6a0004750 comp rx=0 tx=0).stop 2023-10-03T10:15:57.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.446+0000 7fa6b6236700 1 -- 172.21.15.3:0/1231713969 shutdown_connections 2023-10-03T10:15:57.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.446+0000 7fa6b6236700 1 --2- 172.21.15.3:0/1231713969 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa6b01339f0 0x7fa6b01c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.446+0000 7fa6b6236700 1 --2- 172.21.15.3:0/1231713969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6b009ace0 0x7fa6b011f480 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.446+0000 7fa6b6236700 1 --2- 172.21.15.3:0/1231713969 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa6b009d6f0 0x7fa6b011f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.446+0000 7fa6b6236700 1 -- 172.21.15.3:0/1231713969 >> 172.21.15.3:0/1231713969 conn(0x7fa6b0066f50 msgr2=0x7fa6b00a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:57.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.446+0000 7fa6b6236700 1 -- 172.21.15.3:0/1231713969 shutdown_connections 2023-10-03T10:15:57.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.446+0000 7fa6b6236700 1 -- 172.21.15.3:0/1231713969 wait complete. 2023-10-03T10:15:57.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.448+0000 7fb19f8c2700 1 -- 172.21.15.3:0/2282469403 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb19806b9e0 con 0x7fb198133540 2023-10-03T10:15:57.448 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:57.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.448+0000 7fb18dffb700 1 -- 172.21.15.3:0/2282469403 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb18800b3e0 con 0x7fb198133540 2023-10-03T10:15:57.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.452+0000 7fb19f8c2700 1 -- 172.21.15.3:0/2282469403 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb198133540 msgr2=0x7fb19809c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:57.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.452+0000 7fb19f8c2700 1 --2- 172.21.15.3:0/2282469403 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb198133540 0x7fb19809c4b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb188016000 tx=0x7fb188004750 comp rx=0 tx=0).stop 2023-10-03T10:15:57.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.452+0000 7fb19f8c2700 1 -- 172.21.15.3:0/2282469403 shutdown_connections 2023-10-03T10:15:57.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.452+0000 7fb19f8c2700 1 --2- 172.21.15.3:0/2282469403 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb198135e60 0x7fb19809c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.452+0000 7fb19f8c2700 1 --2- 172.21.15.3:0/2282469403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb198130540 0x7fb19811a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.452+0000 7fb19f8c2700 1 --2- 172.21.15.3:0/2282469403 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb198133540 0x7fb19809c4b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:57.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.452+0000 7fb19f8c2700 1 -- 172.21.15.3:0/2282469403 >> 172.21.15.3:0/2282469403 conn(0x7fb1980a2aa0 msgr2=0x7fb19812d020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:57.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.453+0000 7fb19f8c2700 1 -- 172.21.15.3:0/2282469403 shutdown_connections 2023-10-03T10:15:57.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:57.453+0000 7fb19f8c2700 1 -- 172.21.15.3:0/2282469403 wait complete. 2023-10-03T10:15:57.459 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:57.463 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:57.469 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:58.426 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:58.459 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:58.464 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:15:58.470 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:58.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.600+0000 7f3928b05700 1 Processor -- start 2023-10-03T10:15:58.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.600+0000 7f3928b05700 1 -- start start 2023-10-03T10:15:58.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.601+0000 7f3928b05700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f392409d6f0 0x7f392409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:58.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.601+0000 7f3928b05700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f392409e000 0x7f39241339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:58.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.601+0000 7f3928b05700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f392409b120 con 0x7f392409e410 2023-10-03T10:15:58.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.601+0000 7f3928b05700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f392406b460 con 0x7f392409d6f0 2023-10-03T10:15:58.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.601+0000 7f3928b05700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f392409a7e0 con 0x7f392409e000 2023-10-03T10:15:58.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.602+0000 7f392259c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f392409d6f0 0x7f392409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:58.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.602+0000 7f3922d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f392409e410 0x7f3924133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36160/0 (socket says 172.21.15.3:36160) 2023-10-03T10:15:58.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.602+0000 7f3922d9d700 1 -- 172.21.15.3:0/2715588863 learned_addr learned my addr 172.21.15.3:0/2715588863 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:58.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.602+0000 7f3921d9b700 1 --2- 172.21.15.3:0/2715588863 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f392409e000 0x7f39241339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:58.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.602+0000 7f3921d9b700 1 -- 172.21.15.3:0/2715588863 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f392409d6f0 msgr2=0x7f392409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:58.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.602+0000 7f3921d9b700 1 --2- 172.21.15.3:0/2715588863 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f392409d6f0 0x7f392409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.602+0000 7f3921d9b700 1 -- 172.21.15.3:0/2715588863 >> v1:172.21.15.3:6789/0 conn(0x7f392409e410 legacy=0x7f3924133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:58.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.602+0000 7f392159a700 1 -- 172.21.15.3:0/2715588863 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1096258811 0 0) 0x7f392409b120 con 0x7f392409e410 2023-10-03T10:15:58.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.602+0000 7f3921d9b700 1 -- 172.21.15.3:0/2715588863 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392409a500 con 0x7f392409e000 2023-10-03T10:15:58.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.602+0000 7f392259c700 1 --2- 172.21.15.3:0/2715588863 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f392409d6f0 0x7f392409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:58.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.602+0000 7f3921d9b700 1 --2- 172.21.15.3:0/2715588863 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f392409e000 0x7f39241339f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f39180098a0 tx=0x7f3918009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89e3c6c76a9475b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:58.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.603+0000 7f392159a700 1 -- 172.21.15.3:0/2715588863 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3918011830 con 0x7f392409e000 2023-10-03T10:15:58.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.603+0000 7f392159a700 1 -- 172.21.15.3:0/2715588863 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3918011990 con 0x7f392409e000 2023-10-03T10:15:58.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.603+0000 7f392159a700 1 -- 172.21.15.3:0/2715588863 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3918011d20 con 0x7f392409e000 2023-10-03T10:15:58.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.604+0000 7f3928b05700 1 -- 172.21.15.3:0/2715588863 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f392409e000 msgr2=0x7f39241339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:58.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.604+0000 7f3928b05700 1 --2- 172.21.15.3:0/2715588863 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f392409e000 0x7f39241339f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f39180098a0 tx=0x7f3918009bb0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.604+0000 7f3928b05700 1 -- 172.21.15.3:0/2715588863 shutdown_connections 2023-10-03T10:15:58.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.604+0000 7f3928b05700 1 --2- 172.21.15.3:0/2715588863 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f392409d6f0 0x7f392409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.604+0000 7f3928b05700 1 --2- 172.21.15.3:0/2715588863 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f392409e000 0x7f39241339f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.604+0000 7f3928b05700 1 -- 172.21.15.3:0/2715588863 >> 172.21.15.3:0/2715588863 conn(0x7f3924066f50 msgr2=0x7f392409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:58.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.604+0000 7f3928b05700 1 -- 172.21.15.3:0/2715588863 shutdown_connections 2023-10-03T10:15:58.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.604+0000 7f3928b05700 1 -- 172.21.15.3:0/2715588863 wait complete. 2023-10-03T10:15:58.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.606+0000 7f3928b05700 1 Processor -- start 2023-10-03T10:15:58.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.606+0000 7f3928b05700 1 -- start start 2023-10-03T10:15:58.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.607+0000 7f3928b05700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f392409d6f0 0x7f39241a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:58.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.607+0000 7f3928b05700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f392409e000 0x7f39241a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:58.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.607+0000 7f3928b05700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f392409e410 0x7f39241c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:58.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.607+0000 7f3928b05700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39241b91f0 con 0x7f392409e000 2023-10-03T10:15:58.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.607+0000 7f3928b05700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39241a4d40 con 0x7f392409d6f0 2023-10-03T10:15:58.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.607+0000 7f3928b05700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f39241a6470 con 0x7f392409e410 2023-10-03T10:15:58.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.607+0000 7f392259c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f392409d6f0 0x7f39241a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:58.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.607+0000 7f392259c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f392409d6f0 0x7f39241a5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38964/0 (socket says 172.21.15.3:38964) 2023-10-03T10:15:58.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.607+0000 7f392259c700 1 -- 172.21.15.3:0/2668735941 learned_addr learned my addr 172.21.15.3:0/2668735941 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:58.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.607+0000 7f3921d9b700 1 --2- 172.21.15.3:0/2668735941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f392409e000 0x7f39241a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:58.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.607+0000 7f3922d9d700 1 --2- 172.21.15.3:0/2668735941 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f392409e410 0x7f39241c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:58.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.607+0000 7f3921d9b700 1 -- 172.21.15.3:0/2668735941 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f392409e410 msgr2=0x7f39241c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:58.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.607+0000 7f3921d9b700 1 --2- 172.21.15.3:0/2668735941 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f392409e410 0x7f39241c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.607+0000 7f3921d9b700 1 -- 172.21.15.3:0/2668735941 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f392409d6f0 msgr2=0x7f39241a5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:58.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.607+0000 7f3921d9b700 1 --2- 172.21.15.3:0/2668735941 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f392409d6f0 0x7f39241a5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.607+0000 7f3921d9b700 1 -- 172.21.15.3:0/2668735941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39140094a0 con 0x7f392409e000 2023-10-03T10:15:58.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.607+0000 7f392259c700 1 --2- 172.21.15.3:0/2668735941 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f392409d6f0 0x7f39241a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:58.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.608+0000 7f3921d9b700 1 --2- 172.21.15.3:0/2668735941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f392409e000 0x7f39241a5670 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f3918016000 tx=0x7f3918004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:58.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.608+0000 7f3912ffd700 1 -- 172.21.15.3:0/2668735941 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3918004450 con 0x7f392409e000 2023-10-03T10:15:58.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.608+0000 7f3912ffd700 1 -- 172.21.15.3:0/2668735941 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391801f070 con 0x7f392409e000 2023-10-03T10:15:58.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.608+0000 7f3912ffd700 1 -- 172.21.15.3:0/2668735941 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39180038e0 con 0x7f392409e000 2023-10-03T10:15:58.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.608+0000 7f3928b05700 1 -- 172.21.15.3:0/2668735941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39180094a0 con 0x7f392409e000 2023-10-03T10:15:58.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.608+0000 7f3928b05700 1 -- 172.21.15.3:0/2668735941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f392409c710 con 0x7f392409e000 2023-10-03T10:15:58.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.609+0000 7f3928b05700 1 -- 172.21.15.3:0/2668735941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3924074ea0 con 0x7f392409e000 2023-10-03T10:15:58.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.610+0000 7f3912ffd700 1 -- 172.21.15.3:0/2668735941 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3918003a40 con 0x7f392409e000 2023-10-03T10:15:58.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.610+0000 7f3912ffd700 1 -- 172.21.15.3:0/2668735941 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3918021070 con 0x7f392409e000 2023-10-03T10:15:58.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.618+0000 7f3912ffd700 1 -- 172.21.15.3:0/2668735941 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f39180045b0 con 0x7f392409e000 2023-10-03T10:15:58.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.620+0000 7fdfa8e16700 1 Processor -- start 2023-10-03T10:15:58.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.620+0000 7fdfa8e16700 1 -- start start 2023-10-03T10:15:58.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.621+0000 7fdfa8e16700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdfa4131810 0x7fdfa4131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:58.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.621+0000 7fdfa8e16700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfa4099a40 0x7fdfa4099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:58.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.621+0000 7fdfa8e16700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdfa409a3b0 0x7fdfa4138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:58.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.621+0000 7fdfa8e16700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfa41313c0 con 0x7fdfa4131810 2023-10-03T10:15:58.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.621+0000 7fdfa8e16700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa406e8c0 con 0x7fdfa4099a40 2023-10-03T10:15:58.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.621+0000 7fdfa8e16700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa4071d20 con 0x7fdfa409a3b0 2023-10-03T10:15:58.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.621+0000 7fdfa2d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdfa409a3b0 0x7fdfa4138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:58.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.621+0000 7fdfa2d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdfa409a3b0 0x7fdfa4138650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:58498/0 (socket says 172.21.15.3:58498) 2023-10-03T10:15:58.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.621+0000 7fdfa2d9d700 1 -- 172.21.15.3:0/237313311 learned_addr learned my addr 172.21.15.3:0/237313311 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:58.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.621+0000 7fdf9bfff700 1 --2- 172.21.15.3:0/237313311 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfa4099a40 0x7fdfa4099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:58.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.621+0000 7fdfa2d9d700 1 -- 172.21.15.3:0/237313311 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfa4099a40 msgr2=0x7fdfa4099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:58.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.621+0000 7fdfa2d9d700 1 --2- 172.21.15.3:0/237313311 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfa4099a40 0x7fdfa4099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.621+0000 7fdfa2d9d700 1 -- 172.21.15.3:0/237313311 >> v2:172.21.15.3:3300/0 conn(0x7fdfa4131810 msgr2=0x7fdfa4131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:58.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.621+0000 7fdfa2d9d700 1 --2- 172.21.15.3:0/237313311 >> v2:172.21.15.3:3300/0 conn(0x7fdfa4131810 0x7fdfa4131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.621+0000 7fdfa2d9d700 1 -- 172.21.15.3:0/237313311 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa40721a0 con 0x7fdfa409a3b0 2023-10-03T10:15:58.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.622+0000 7fdfa2d9d700 1 --2- 172.21.15.3:0/237313311 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdfa409a3b0 0x7fdfa4138650 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009e50 tx=0x7fdf9c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aed50f6e107e8052 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:58.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.622+0000 7fdfa1d9b700 1 -- 172.21.15.3:0/237313311 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c00e070 con 0x7fdfa409a3b0 2023-10-03T10:15:58.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.622+0000 7fdfa1d9b700 1 -- 172.21.15.3:0/237313311 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9c00b910 con 0x7fdfa409a3b0 2023-10-03T10:15:58.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.622+0000 7fdfa1d9b700 1 -- 172.21.15.3:0/237313311 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c00bcc0 con 0x7fdfa409a3b0 2023-10-03T10:15:58.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.623+0000 7fdfa8e16700 1 -- 172.21.15.3:0/237313311 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdfa409a3b0 msgr2=0x7fdfa4138650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:58.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.623+0000 7fdfa8e16700 1 --2- 172.21.15.3:0/237313311 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdfa409a3b0 0x7fdfa4138650 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009e50 tx=0x7fdf9c00b330 comp rx=0 tx=0).stop 2023-10-03T10:15:58.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.623+0000 7fdfa8e16700 1 -- 172.21.15.3:0/237313311 shutdown_connections 2023-10-03T10:15:58.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.623+0000 7fdfa8e16700 1 --2- 172.21.15.3:0/237313311 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdfa409a3b0 0x7fdfa4138650 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.623+0000 7fdfa8e16700 1 --2- 172.21.15.3:0/237313311 >> v2:172.21.15.3:3300/0 conn(0x7fdfa4131810 0x7fdfa4131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.623+0000 7fdfa8e16700 1 --2- 172.21.15.3:0/237313311 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfa4099a40 0x7fdfa4099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.623+0000 7fdfa8e16700 1 -- 172.21.15.3:0/237313311 >> 172.21.15.3:0/237313311 conn(0x7fdfa40a2aa0 msgr2=0x7fdfa40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:58.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.623+0000 7fdfa8e16700 1 -- 172.21.15.3:0/237313311 shutdown_connections 2023-10-03T10:15:58.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.623+0000 7fdfa8e16700 1 -- 172.21.15.3:0/237313311 wait complete. 2023-10-03T10:15:58.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.625+0000 7fdfa8e16700 1 Processor -- start 2023-10-03T10:15:58.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.625+0000 7fdfa8e16700 1 -- start start 2023-10-03T10:15:58.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.625+0000 7fdfa8e16700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfa4099a40 0x7fdfa411b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:58.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.625+0000 7fdfa8e16700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdfa409a3b0 0x7fdfa411bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:58.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.625+0000 7fdfa8e16700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfa4131810 0x7fdfa411c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:58.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.625+0000 7fdfa8e16700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa411d2f0 con 0x7fdfa4099a40 2023-10-03T10:15:58.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.625+0000 7fdfa8e16700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa411d910 con 0x7fdfa4131810 2023-10-03T10:15:58.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.625+0000 7fdfa8e16700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa411e540 con 0x7fdfa409a3b0 2023-10-03T10:15:58.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.626+0000 7fdfa259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfa4099a40 0x7fdfa411b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:58.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.626+0000 7fdfa259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfa4099a40 0x7fdfa411b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32794/0 (socket says 172.21.15.3:32794) 2023-10-03T10:15:58.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.626+0000 7fdfa259c700 1 -- 172.21.15.3:0/3089732399 learned_addr learned my addr 172.21.15.3:0/3089732399 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:58.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.626+0000 7fdfa2d9d700 1 --2- 172.21.15.3:0/3089732399 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfa4131810 0x7fdfa411c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:58.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.626+0000 7fdfa259c700 1 -- 172.21.15.3:0/3089732399 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdfa409a3b0 msgr2=0x7fdfa411bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:58.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.626+0000 7fdfa259c700 1 --2- 172.21.15.3:0/3089732399 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdfa409a3b0 0x7fdfa411bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.626+0000 7fdfa259c700 1 -- 172.21.15.3:0/3089732399 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfa4131810 msgr2=0x7fdfa411c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:58.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.626+0000 7fdfa259c700 1 --2- 172.21.15.3:0/3089732399 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfa4131810 0x7fdfa411c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.626+0000 7fdfa259c700 1 -- 172.21.15.3:0/3089732399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9c009b30 con 0x7fdfa4099a40 2023-10-03T10:15:58.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.626+0000 7fdfa259c700 1 --2- 172.21.15.3:0/3089732399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfa4099a40 0x7fdfa411b730 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fdf9400b560 tx=0x7fdf9400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:58.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.627+0000 7fdf9affd700 1 -- 172.21.15.3:0/3089732399 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94011830 con 0x7fdfa4099a40 2023-10-03T10:15:58.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.627+0000 7fdf9affd700 1 -- 172.21.15.3:0/3089732399 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf94011990 con 0x7fdfa4099a40 2023-10-03T10:15:58.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.627+0000 7fdfa8e16700 1 -- 172.21.15.3:0/3089732399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa411ef20 con 0x7fdfa4099a40 2023-10-03T10:15:58.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.627+0000 7fdfa8e16700 1 -- 172.21.15.3:0/3089732399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa406eb00 con 0x7fdfa4099a40 2023-10-03T10:15:58.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.627+0000 7fdf9affd700 1 -- 172.21.15.3:0/3089732399 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94011d20 con 0x7fdfa4099a40 2023-10-03T10:15:58.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.628+0000 7fdde1aa2700 1 Processor -- start 2023-10-03T10:15:58.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.628+0000 7fdf9affd700 1 -- 172.21.15.3:0/3089732399 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdf9401a400 con 0x7fdfa4099a40 2023-10-03T10:15:58.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.628+0000 7fdfa8e16700 1 -- 172.21.15.3:0/3089732399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf880052e0 con 0x7fdfa4099a40 2023-10-03T10:15:58.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.628+0000 7fdf9affd700 1 -- 172.21.15.3:0/3089732399 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fdf9403f6f0 con 0x7fdfa4099a40 2023-10-03T10:15:58.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.628+0000 7fdde1aa2700 1 -- start start 2023-10-03T10:15:58.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.628+0000 7fdde1aa2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdddc06c0e0 0x7fdddc09ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:58.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.628+0000 7fdde1aa2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdddc134ec0 0x7fdddc1379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:58.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.628+0000 7fdde1aa2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdddc134980 con 0x7fdddc137f20 2023-10-03T10:15:58.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.628+0000 7fdde1aa2700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdddc06c4b0 con 0x7fdddc134ec0 2023-10-03T10:15:58.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.628+0000 7fdde1aa2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc0a09c0 con 0x7fdddc06c0e0 2023-10-03T10:15:58.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.629+0000 7fdddbfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdddc137f20 0x7fdddc13ed10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36174/0 (socket says 172.21.15.3:36174) 2023-10-03T10:15:58.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.629+0000 7fdddaffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdddc134ec0 0x7fdddc1379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:58.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.629+0000 7fdddbfff700 1 -- 172.21.15.3:0/2253619355 learned_addr learned my addr 172.21.15.3:0/2253619355 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:15:58.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.629+0000 7fdddb7fe700 1 --2- 172.21.15.3:0/2253619355 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdddc06c0e0 0x7fdddc09ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:58.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.629+0000 7fdddaffd700 1 -- 172.21.15.3:0/2253619355 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdddc06c0e0 msgr2=0x7fdddc09ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:58.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.629+0000 7fdddaffd700 1 --2- 172.21.15.3:0/2253619355 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdddc06c0e0 0x7fdddc09ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.629+0000 7fdddaffd700 1 -- 172.21.15.3:0/2253619355 >> v1:172.21.15.3:6789/0 conn(0x7fdddc137f20 legacy=0x7fdddc13ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:58.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.629+0000 7fdddaffd700 1 -- 172.21.15.3:0/2253619355 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdddc0a0600 con 0x7fdddc134ec0 2023-10-03T10:15:58.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.629+0000 7fdddaffd700 1 --2- 172.21.15.3:0/2253619355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdddc134ec0 0x7fdddc1379e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fddcc009840 tx=0x7fddcc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fcc772f724349e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:58.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.634+0000 7fddda7fc700 1 -- 172.21.15.3:0/2253619355 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddcc011830 con 0x7fdddc134ec0 2023-10-03T10:15:58.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.634+0000 7fddda7fc700 1 -- 172.21.15.3:0/2253619355 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddcc011990 con 0x7fdddc134ec0 2023-10-03T10:15:58.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.634+0000 7fddda7fc700 1 -- 172.21.15.3:0/2253619355 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddcc011d20 con 0x7fdddc134ec0 2023-10-03T10:15:58.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.634+0000 7fdf9affd700 1 -- 172.21.15.3:0/3089732399 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdf94015070 con 0x7fdfa4099a40 2023-10-03T10:15:58.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.635+0000 7fdde1aa2700 1 -- 172.21.15.3:0/2253619355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdddc134ec0 msgr2=0x7fdddc1379e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:58.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.635+0000 7fdde1aa2700 1 --2- 172.21.15.3:0/2253619355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdddc134ec0 0x7fdddc1379e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fddcc009840 tx=0x7fddcc009b50 comp rx=0 tx=0).stop 2023-10-03T10:15:58.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.635+0000 7fdde1aa2700 1 -- 172.21.15.3:0/2253619355 shutdown_connections 2023-10-03T10:15:58.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.635+0000 7fdde1aa2700 1 --2- 172.21.15.3:0/2253619355 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdddc06c0e0 0x7fdddc09ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.635+0000 7fdde1aa2700 1 --2- 172.21.15.3:0/2253619355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdddc134ec0 0x7fdddc1379e0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.635+0000 7fdde1aa2700 1 -- 172.21.15.3:0/2253619355 >> 172.21.15.3:0/2253619355 conn(0x7fdddc0a4210 msgr2=0x7fdddc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:58.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.635+0000 7fdde1aa2700 1 -- 172.21.15.3:0/2253619355 shutdown_connections 2023-10-03T10:15:58.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.635+0000 7fdde1aa2700 1 -- 172.21.15.3:0/2253619355 wait complete. 2023-10-03T10:15:58.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.637+0000 7fdde1aa2700 1 Processor -- start 2023-10-03T10:15:58.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.637+0000 7fdde1aa2700 1 -- start start 2023-10-03T10:15:58.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.638+0000 7fdde1aa2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdddc06c0e0 0x7fdddc1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:58.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.638+0000 7fdde1aa2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdddc134ec0 0x7fdddc133f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:58.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.638+0000 7fdde1aa2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdddc137f20 0x7fdddc132280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:58.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.638+0000 7fdde1aa2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc11bb30 con 0x7fdddc134ec0 2023-10-03T10:15:58.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.638+0000 7fdde1aa2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc11c280 con 0x7fdddc137f20 2023-10-03T10:15:58.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.638+0000 7fdde1aa2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc11c8a0 con 0x7fdddc06c0e0 2023-10-03T10:15:58.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.638+0000 7fdddaffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdddc134ec0 0x7fdddc133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:58.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.638+0000 7fdddbfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdddc137f20 0x7fdddc132280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:58.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.638+0000 7fdddaffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdddc134ec0 0x7fdddc133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32818/0 (socket says 172.21.15.3:32818) 2023-10-03T10:15:58.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.638+0000 7fdddaffd700 1 -- 172.21.15.3:0/1230340638 learned_addr learned my addr 172.21.15.3:0/1230340638 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:58.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.638+0000 7fdddaffd700 1 -- 172.21.15.3:0/1230340638 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdddc06c0e0 msgr2=0x7fdddc1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:58.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.638+0000 7fdddaffd700 1 --2- 172.21.15.3:0/1230340638 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdddc06c0e0 0x7fdddc1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.638+0000 7fdddaffd700 1 -- 172.21.15.3:0/1230340638 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdddc137f20 msgr2=0x7fdddc132280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:58.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.638+0000 7fdddaffd700 1 --2- 172.21.15.3:0/1230340638 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdddc137f20 0x7fdddc132280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.638+0000 7fdddaffd700 1 -- 172.21.15.3:0/1230340638 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddcc0094a0 con 0x7fdddc134ec0 2023-10-03T10:15:58.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.638+0000 7fdddaffd700 1 --2- 172.21.15.3:0/1230340638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdddc134ec0 0x7fdddc133f30 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fddcc000c00 tx=0x7fddcc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:58.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.639+0000 7fddc3fff700 1 -- 172.21.15.3:0/1230340638 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddcc0036a0 con 0x7fdddc134ec0 2023-10-03T10:15:58.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.639+0000 7fddc3fff700 1 -- 172.21.15.3:0/1230340638 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddcc01f070 con 0x7fdddc134ec0 2023-10-03T10:15:58.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.639+0000 7fddc3fff700 1 -- 172.21.15.3:0/1230340638 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddcc003a60 con 0x7fdddc134ec0 2023-10-03T10:15:58.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.639+0000 7fdde1aa2700 1 -- 172.21.15.3:0/1230340638 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdddc11d560 con 0x7fdddc134ec0 2023-10-03T10:15:58.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.640+0000 7fdde1aa2700 1 -- 172.21.15.3:0/1230340638 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdddc137990 con 0x7fdddc134ec0 2023-10-03T10:15:58.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.641+0000 7fddc3fff700 1 -- 172.21.15.3:0/1230340638 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fddcc003bc0 con 0x7fdddc134ec0 2023-10-03T10:15:58.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.641+0000 7fdde1aa2700 1 -- 172.21.15.3:0/1230340638 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdddc11db50 con 0x7fdddc134ec0 2023-10-03T10:15:58.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.641+0000 7fddc3fff700 1 -- 172.21.15.3:0/1230340638 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fddcc021070 con 0x7fdddc134ec0 2023-10-03T10:15:58.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.646+0000 7fddc3fff700 1 -- 172.21.15.3:0/1230340638 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdddc11db50 con 0x7fdddc134ec0 2023-10-03T10:15:58.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.649+0000 7fb54035c700 1 Processor -- start 2023-10-03T10:15:58.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.649+0000 7fb54035c700 1 -- start start 2023-10-03T10:15:58.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.650+0000 7fb54035c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb53809d6f0 0x7fb53809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:58.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.650+0000 7fb54035c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb53809e410 0x7fb538130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:58.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.650+0000 7fb54035c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb53809f130 con 0x7fb53809e000 2023-10-03T10:15:58.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.650+0000 7fb54035c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb53809f520 con 0x7fb53809e410 2023-10-03T10:15:58.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.650+0000 7fb54035c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb53809e930 con 0x7fb53809d6f0 2023-10-03T10:15:58.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.650+0000 7fb53e0f8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb53809d6f0 0x7fb53809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:58.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.650+0000 7fb53e0f8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb53809d6f0 0x7fb53809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:58520/0 (socket says 172.21.15.3:58520) 2023-10-03T10:15:58.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.650+0000 7fb53e0f8700 1 -- 172.21.15.3:0/449010198 learned_addr learned my addr 172.21.15.3:0/449010198 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:58.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.650+0000 7fb53e8f9700 1 --2- 172.21.15.3:0/449010198 >> v2:172.21.15.3:3300/0 conn(0x7fb53809e410 0x7fb538130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:58.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.650+0000 7fb53e0f8700 1 -- 172.21.15.3:0/449010198 >> v2:172.21.15.3:3300/0 conn(0x7fb53809e410 msgr2=0x7fb538130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:15:58.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.650+0000 7fb53e0f8700 1 --2- 172.21.15.3:0/449010198 >> v2:172.21.15.3:3300/0 conn(0x7fb53809e410 0x7fb538130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.650+0000 7fb53e0f8700 1 -- 172.21.15.3:0/449010198 >> v1:172.21.15.3:6789/0 conn(0x7fb53809e000 legacy=0x7fb538130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:58.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.650+0000 7fb53e0f8700 1 -- 172.21.15.3:0/449010198 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb53809b2d0 con 0x7fb53809d6f0 2023-10-03T10:15:58.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.651+0000 7fb53e8f9700 1 --2- 172.21.15.3:0/449010198 >> v2:172.21.15.3:3300/0 conn(0x7fb53809e410 0x7fb538130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:15:58.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.651+0000 7fb53e0f8700 1 --2- 172.21.15.3:0/449010198 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb53809d6f0 0x7fb53809dac0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb53400bb90 tx=0x7fb53400bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29a1d22e5754f94c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:58.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.651+0000 7fb53d0f6700 1 -- 172.21.15.3:0/449010198 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53400dd30 con 0x7fb53809d6f0 2023-10-03T10:15:58.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.651+0000 7fb53d0f6700 1 -- 172.21.15.3:0/449010198 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53400de90 con 0x7fb53809d6f0 2023-10-03T10:15:58.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.651+0000 7fb53d0f6700 1 -- 172.21.15.3:0/449010198 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb534010690 con 0x7fb53809d6f0 2023-10-03T10:15:58.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.652+0000 7fb54035c700 1 -- 172.21.15.3:0/449010198 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb53809d6f0 msgr2=0x7fb53809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:58.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.652+0000 7fb54035c700 1 --2- 172.21.15.3:0/449010198 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb53809d6f0 0x7fb53809dac0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb53400bb90 tx=0x7fb53400bea0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.652+0000 7fb54035c700 1 -- 172.21.15.3:0/449010198 shutdown_connections 2023-10-03T10:15:58.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.652+0000 7fb54035c700 1 --2- 172.21.15.3:0/449010198 >> v2:172.21.15.3:3300/0 conn(0x7fb53809e410 0x7fb538130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.652+0000 7fb54035c700 1 --2- 172.21.15.3:0/449010198 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb53809d6f0 0x7fb53809dac0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.652+0000 7fb54035c700 1 -- 172.21.15.3:0/449010198 >> 172.21.15.3:0/449010198 conn(0x7fb538066f50 msgr2=0x7fb53809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:58.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.652+0000 7fb54035c700 1 -- 172.21.15.3:0/449010198 shutdown_connections 2023-10-03T10:15:58.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.652+0000 7fb54035c700 1 -- 172.21.15.3:0/449010198 wait complete. 2023-10-03T10:15:58.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.654+0000 7fb54035c700 1 Processor -- start 2023-10-03T10:15:58.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.654+0000 7fb54035c700 1 -- start start 2023-10-03T10:15:58.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.654+0000 7fb54035c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb53809d6f0 0x7fb5381b7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:58.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.655+0000 7fb54035c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb53809e000 0x7fb5381b7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:58.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.655+0000 7fb53e0f8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb53809d6f0 0x7fb5381b7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:58.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.655+0000 7fb54035c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb53809e410 0x7fb5381b8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:15:58.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.655+0000 7fb54035c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5381baab0 con 0x7fb53809e410 2023-10-03T10:15:58.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.655+0000 7fb54035c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5381bac90 con 0x7fb53809d6f0 2023-10-03T10:15:58.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.655+0000 7fb54035c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb5380a1e20 con 0x7fb53809e000 2023-10-03T10:15:58.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.655+0000 7fb53e0f8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb53809d6f0 0x7fb5381b7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39000/0 (socket says 172.21.15.3:39000) 2023-10-03T10:15:58.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.655+0000 7fb53e0f8700 1 -- 172.21.15.3:0/3281779710 learned_addr learned my addr 172.21.15.3:0/3281779710 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:15:58.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.655+0000 7fb53e8f9700 1 --2- 172.21.15.3:0/3281779710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb53809e410 0x7fb5381b8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:58.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.655+0000 7fb53d8f7700 1 --2- 172.21.15.3:0/3281779710 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb53809e000 0x7fb5381b7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:15:58.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.655+0000 7fb53e0f8700 1 -- 172.21.15.3:0/3281779710 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb53809e000 msgr2=0x7fb5381b7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:58.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.655+0000 7fb53e0f8700 1 --2- 172.21.15.3:0/3281779710 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb53809e000 0x7fb5381b7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.655+0000 7fb53e0f8700 1 -- 172.21.15.3:0/3281779710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb53809e410 msgr2=0x7fb5381b8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:58.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.655+0000 7fb53e0f8700 1 --2- 172.21.15.3:0/3281779710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb53809e410 0x7fb5381b8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.655+0000 7fb53e0f8700 1 -- 172.21.15.3:0/3281779710 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb530009a70 con 0x7fb53809d6f0 2023-10-03T10:15:58.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.656+0000 7fb53e0f8700 1 --2- 172.21.15.3:0/3281779710 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb53809d6f0 0x7fb5381b7610 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb534000c00 tx=0x7fb534004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:15:58.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.656+0000 7fb52effd700 1 -- 172.21.15.3:0/3281779710 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb534003b70 con 0x7fb53809d6f0 2023-10-03T10:15:58.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.656+0000 7fb54035c700 1 -- 172.21.15.3:0/3281779710 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb53400b7f0 con 0x7fb53809d6f0 2023-10-03T10:15:58.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.656+0000 7fb52effd700 1 -- 172.21.15.3:0/3281779710 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb534003cd0 con 0x7fb53809d6f0 2023-10-03T10:15:58.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.656+0000 7fb52effd700 1 -- 172.21.15.3:0/3281779710 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5340106b0 con 0x7fb53809d6f0 2023-10-03T10:15:58.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.656+0000 7fb54035c700 1 -- 172.21.15.3:0/3281779710 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5381b95d0 con 0x7fb53809d6f0 2023-10-03T10:15:58.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.657+0000 7fb52effd700 1 -- 172.21.15.3:0/3281779710 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb534019400 con 0x7fb53809d6f0 2023-10-03T10:15:58.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.657+0000 7fb52effd700 1 -- 172.21.15.3:0/3281779710 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb534040b30 con 0x7fb53809d6f0 2023-10-03T10:15:58.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.657+0000 7fb54035c700 1 -- 172.21.15.3:0/3281779710 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb53809c700 con 0x7fb53809d6f0 2023-10-03T10:15:58.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.663+0000 7fb52effd700 1 -- 172.21.15.3:0/3281779710 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb534003e40 con 0x7fb53809d6f0 2023-10-03T10:15:58.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.834+0000 7f3928b05700 1 -- 172.21.15.3:0/2668735941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f39241a82b0 con 0x7f392409e000 2023-10-03T10:15:58.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.835+0000 7f3912ffd700 1 -- 172.21.15.3:0/2668735941 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3918015070 con 0x7f392409e000 2023-10-03T10:15:58.835 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:15:58.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.837+0000 7f3928b05700 1 -- 172.21.15.3:0/2668735941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f392409e000 msgr2=0x7f39241a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:58.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.837+0000 7f3928b05700 1 --2- 172.21.15.3:0/2668735941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f392409e000 0x7f39241a5670 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f3918016000 tx=0x7f3918004750 comp rx=0 tx=0).stop 2023-10-03T10:15:58.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.837+0000 7f3928b05700 1 -- 172.21.15.3:0/2668735941 shutdown_connections 2023-10-03T10:15:58.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.837+0000 7f3928b05700 1 --2- 172.21.15.3:0/2668735941 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f392409e410 0x7f39241c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.837+0000 7f3928b05700 1 --2- 172.21.15.3:0/2668735941 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f392409d6f0 0x7f39241a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.837+0000 7f3928b05700 1 --2- 172.21.15.3:0/2668735941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f392409e000 0x7f39241a5670 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.837+0000 7f3928b05700 1 -- 172.21.15.3:0/2668735941 >> 172.21.15.3:0/2668735941 conn(0x7f3924066f50 msgr2=0x7f39240a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:58.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.837+0000 7f3928b05700 1 -- 172.21.15.3:0/2668735941 shutdown_connections 2023-10-03T10:15:58.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.837+0000 7f3928b05700 1 -- 172.21.15.3:0/2668735941 wait complete. 2023-10-03T10:15:58.852 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:15:58.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.856+0000 7fdde1aa2700 1 -- 172.21.15.3:0/1230340638 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdddc11db50 con 0x7fdddc134ec0 2023-10-03T10:15:58.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.856+0000 7fddc3fff700 1 -- 172.21.15.3:0/1230340638 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdddc11db50 con 0x7fdddc134ec0 2023-10-03T10:15:58.856 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:15:58.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.858+0000 7fdde1aa2700 1 -- 172.21.15.3:0/1230340638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdddc134ec0 msgr2=0x7fdddc133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:58.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.858+0000 7fdde1aa2700 1 --2- 172.21.15.3:0/1230340638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdddc134ec0 0x7fdddc133f30 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fddcc000c00 tx=0x7fddcc004750 comp rx=0 tx=0).stop 2023-10-03T10:15:58.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.858+0000 7fdde1aa2700 1 -- 172.21.15.3:0/1230340638 shutdown_connections 2023-10-03T10:15:58.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.858+0000 7fdde1aa2700 1 --2- 172.21.15.3:0/1230340638 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdddc137f20 0x7fdddc132280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.858+0000 7fdde1aa2700 1 --2- 172.21.15.3:0/1230340638 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdddc06c0e0 0x7fdddc1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.858+0000 7fdde1aa2700 1 --2- 172.21.15.3:0/1230340638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdddc134ec0 0x7fdddc133f30 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.858+0000 7fdde1aa2700 1 -- 172.21.15.3:0/1230340638 >> 172.21.15.3:0/1230340638 conn(0x7fdddc0a4210 msgr2=0x7fdddc06c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:58.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.859+0000 7fdde1aa2700 1 -- 172.21.15.3:0/1230340638 shutdown_connections 2023-10-03T10:15:58.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.859+0000 7fdde1aa2700 1 -- 172.21.15.3:0/1230340638 wait complete. 2023-10-03T10:15:58.872 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:15:58.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.881+0000 7fdfa8e16700 1 -- 172.21.15.3:0/3089732399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fdf88005590 con 0x7fdfa4099a40 2023-10-03T10:15:58.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.882+0000 7fdf9affd700 1 -- 172.21.15.3:0/3089732399 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdf94028090 con 0x7fdfa4099a40 2023-10-03T10:15:58.882 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:15:58.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.884+0000 7fdfa8e16700 1 -- 172.21.15.3:0/3089732399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfa4099a40 msgr2=0x7fdfa411b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:58.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.884+0000 7fdfa8e16700 1 --2- 172.21.15.3:0/3089732399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfa4099a40 0x7fdfa411b730 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fdf9400b560 tx=0x7fdf9400b870 comp rx=0 tx=0).stop 2023-10-03T10:15:58.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.884+0000 7fdfa8e16700 1 -- 172.21.15.3:0/3089732399 shutdown_connections 2023-10-03T10:15:58.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.884+0000 7fdfa8e16700 1 --2- 172.21.15.3:0/3089732399 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfa4131810 0x7fdfa411c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.884+0000 7fdfa8e16700 1 --2- 172.21.15.3:0/3089732399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfa4099a40 0x7fdfa411b730 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.884+0000 7fdfa8e16700 1 --2- 172.21.15.3:0/3089732399 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdfa409a3b0 0x7fdfa411bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.884+0000 7fdfa8e16700 1 -- 172.21.15.3:0/3089732399 >> 172.21.15.3:0/3089732399 conn(0x7fdfa40a2aa0 msgr2=0x7fdfa4070ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:58.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.884+0000 7fdfa8e16700 1 -- 172.21.15.3:0/3089732399 shutdown_connections 2023-10-03T10:15:58.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.884+0000 7fdfa8e16700 1 -- 172.21.15.3:0/3089732399 wait complete. 2023-10-03T10:15:58.898 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:15:58.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.901+0000 7fb54035c700 1 -- 172.21.15.3:0/3281779710 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb538074ea0 con 0x7fb53809d6f0 2023-10-03T10:15:58.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.902+0000 7fb52effd700 1 -- 172.21.15.3:0/3281779710 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb534018360 con 0x7fb53809d6f0 2023-10-03T10:15:58.902 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:15:58.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.904+0000 7fb54035c700 1 -- 172.21.15.3:0/3281779710 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb53809d6f0 msgr2=0x7fb5381b7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:15:58.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.904+0000 7fb54035c700 1 --2- 172.21.15.3:0/3281779710 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb53809d6f0 0x7fb5381b7610 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb534000c00 tx=0x7fb534004750 comp rx=0 tx=0).stop 2023-10-03T10:15:58.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.904+0000 7fb54035c700 1 -- 172.21.15.3:0/3281779710 shutdown_connections 2023-10-03T10:15:58.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.904+0000 7fb54035c700 1 --2- 172.21.15.3:0/3281779710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb53809e410 0x7fb5381b8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.904+0000 7fb54035c700 1 --2- 172.21.15.3:0/3281779710 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb53809d6f0 0x7fb5381b7610 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.904+0000 7fb54035c700 1 --2- 172.21.15.3:0/3281779710 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb53809e000 0x7fb5381b7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:15:58.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.904+0000 7fb54035c700 1 -- 172.21.15.3:0/3281779710 >> 172.21.15.3:0/3281779710 conn(0x7fb538066f50 msgr2=0x7fb53806f100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:15:58.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.904+0000 7fb54035c700 1 -- 172.21.15.3:0/3281779710 shutdown_connections 2023-10-03T10:15:58.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:15:58.904+0000 7fb54035c700 1 -- 172.21.15.3:0/3281779710 wait complete. 2023-10-03T10:15:58.918 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:15:59.853 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:15:59.873 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:15:59.901 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:15:59.919 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:00.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.004+0000 7f41fe78a700 1 Processor -- start 2023-10-03T10:16:00.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.004+0000 7f41fe78a700 1 -- start start 2023-10-03T10:16:00.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.005+0000 7f41fe78a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f41f81338a0 0x7f41f8133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:00.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.005+0000 7f41fe78a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f41f809de50 0x7f41f809e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:00.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.005+0000 7f41fe78a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f806ef30 con 0x7f41f809e760 2023-10-03T10:16:00.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.005+0000 7f41fe78a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f806fef0 con 0x7f41f809de50 2023-10-03T10:16:00.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.005+0000 7f41fe78a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f41f81334b0 con 0x7f41f81338a0 2023-10-03T10:16:00.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.005+0000 7f41fcd27700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f41f809e760 0x7f41f813a730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36180/0 (socket says 172.21.15.3:36180) 2023-10-03T10:16:00.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.005+0000 7f41fcd27700 1 -- 172.21.15.3:0/1368480325 learned_addr learned my addr 172.21.15.3:0/1368480325 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:00.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.005+0000 7f41f77fe700 1 --2- 172.21.15.3:0/1368480325 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f41f809de50 0x7f41f809e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:00.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.005+0000 7f41f7fff700 1 --2- 172.21.15.3:0/1368480325 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f41f81338a0 0x7f41f8133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:00.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.006+0000 7f41f77fe700 1 -- 172.21.15.3:0/1368480325 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f41f81338a0 msgr2=0x7f41f8133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:00.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.006+0000 7f41f77fe700 1 --2- 172.21.15.3:0/1368480325 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f41f81338a0 0x7f41f8133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.006+0000 7f41f77fe700 1 -- 172.21.15.3:0/1368480325 >> v1:172.21.15.3:6789/0 conn(0x7f41f809e760 legacy=0x7f41f813a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:00.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.006+0000 7f41f77fe700 1 -- 172.21.15.3:0/1368480325 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f806fc30 con 0x7f41f809de50 2023-10-03T10:16:00.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.006+0000 7f41f77fe700 1 --2- 172.21.15.3:0/1368480325 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f41f809de50 0x7f41f809e220 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009870 tx=0x7f41ec009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=524ad4bb947ecbb5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:00.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.006+0000 7f41f6ffd700 1 -- 172.21.15.3:0/1368480325 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ec011830 con 0x7f41f809de50 2023-10-03T10:16:00.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.006+0000 7f41f6ffd700 1 -- 172.21.15.3:0/1368480325 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ec011990 con 0x7f41f809de50 2023-10-03T10:16:00.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.006+0000 7f41f6ffd700 1 -- 172.21.15.3:0/1368480325 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ec011d20 con 0x7f41f809de50 2023-10-03T10:16:00.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.007+0000 7f41fe78a700 1 -- 172.21.15.3:0/1368480325 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f41f809de50 msgr2=0x7f41f809e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:00.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.007+0000 7f41fe78a700 1 --2- 172.21.15.3:0/1368480325 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f41f809de50 0x7f41f809e220 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009870 tx=0x7f41ec009b80 comp rx=0 tx=0).stop 2023-10-03T10:16:00.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.007+0000 7f41fe78a700 1 -- 172.21.15.3:0/1368480325 shutdown_connections 2023-10-03T10:16:00.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.007+0000 7f41fe78a700 1 --2- 172.21.15.3:0/1368480325 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f41f81338a0 0x7f41f8133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.007+0000 7f41fe78a700 1 --2- 172.21.15.3:0/1368480325 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f41f809de50 0x7f41f809e220 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.007+0000 7f41fe78a700 1 -- 172.21.15.3:0/1368480325 >> 172.21.15.3:0/1368480325 conn(0x7f41f80a2aa0 msgr2=0x7f41f80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:00.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.007+0000 7f41fe78a700 1 -- 172.21.15.3:0/1368480325 shutdown_connections 2023-10-03T10:16:00.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.007+0000 7f41fe78a700 1 -- 172.21.15.3:0/1368480325 wait complete. 2023-10-03T10:16:00.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.009+0000 7f41fe78a700 1 Processor -- start 2023-10-03T10:16:00.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.009+0000 7f41fe78a700 1 -- start start 2023-10-03T10:16:00.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.010+0000 7f41fe78a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f41f809de50 0x7f41f811eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:00.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.010+0000 7f41fe78a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f41f809e760 0x7f41f811f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:00.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.010+0000 7f41fe78a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41f81338a0 0x7f41f811f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:00.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.010+0000 7f41fe78a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f8121ea0 con 0x7f41f81338a0 2023-10-03T10:16:00.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.010+0000 7f41fe78a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f812f690 con 0x7f41f809de50 2023-10-03T10:16:00.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.010+0000 7f41fe78a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f41f8122080 con 0x7f41f809e760 2023-10-03T10:16:00.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.010+0000 7f41f7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f41f809de50 0x7f41f811eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:00.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.010+0000 7f41f7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f41f809de50 0x7f41f811eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39024/0 (socket says 172.21.15.3:39024) 2023-10-03T10:16:00.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.010+0000 7f41f7fff700 1 -- 172.21.15.3:0/811701887 learned_addr learned my addr 172.21.15.3:0/811701887 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:00.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.010+0000 7f41fcd27700 1 --2- 172.21.15.3:0/811701887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41f81338a0 0x7f41f811f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:00.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.010+0000 7f41f77fe700 1 --2- 172.21.15.3:0/811701887 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f41f809e760 0x7f41f811f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:00.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.010+0000 7f41f7fff700 1 -- 172.21.15.3:0/811701887 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f41f809e760 msgr2=0x7f41f811f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:00.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.010+0000 7f41f7fff700 1 --2- 172.21.15.3:0/811701887 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f41f809e760 0x7f41f811f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.010+0000 7f41f7fff700 1 -- 172.21.15.3:0/811701887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41f81338a0 msgr2=0x7f41f811f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:00.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.010+0000 7f41f7fff700 1 --2- 172.21.15.3:0/811701887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41f81338a0 0x7f41f811f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.010+0000 7f41f7fff700 1 -- 172.21.15.3:0/811701887 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ec0094a0 con 0x7f41f809de50 2023-10-03T10:16:00.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.011+0000 7f41f7fff700 1 --2- 172.21.15.3:0/811701887 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f41f809de50 0x7f41f811eb50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f41e800e9b0 tx=0x7f41e800ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:00.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.011+0000 7f41f4ff9700 1 -- 172.21.15.3:0/811701887 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e8009710 con 0x7f41f809de50 2023-10-03T10:16:00.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.011+0000 7f41f4ff9700 1 -- 172.21.15.3:0/811701887 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e800c850 con 0x7f41f809de50 2023-10-03T10:16:00.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.011+0000 7f41f4ff9700 1 -- 172.21.15.3:0/811701887 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e800cbc0 con 0x7f41f809de50 2023-10-03T10:16:00.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.011+0000 7f41fe78a700 1 -- 172.21.15.3:0/811701887 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f80a10d0 con 0x7f41f809de50 2023-10-03T10:16:00.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.011+0000 7f41fe78a700 1 -- 172.21.15.3:0/811701887 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f8122340 con 0x7f41f809de50 2023-10-03T10:16:00.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.012+0000 7f41f4ff9700 1 -- 172.21.15.3:0/811701887 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f41e800cd20 con 0x7f41f809de50 2023-10-03T10:16:00.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.012+0000 7f41fe78a700 1 -- 172.21.15.3:0/811701887 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f8074ea0 con 0x7f41f809de50 2023-10-03T10:16:00.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.013+0000 7f41f4ff9700 1 -- 172.21.15.3:0/811701887 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f41e803f8b0 con 0x7f41f809de50 2023-10-03T10:16:00.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.022+0000 7f41f4ff9700 1 -- 172.21.15.3:0/811701887 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f41e801bc10 con 0x7f41f809de50 2023-10-03T10:16:00.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.056+0000 7f46f2d3e700 1 Processor -- start 2023-10-03T10:16:00.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.056+0000 7f46f2d3e700 1 -- start start 2023-10-03T10:16:00.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.057+0000 7f46f2d3e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46ec09d6f0 0x7f46ec09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:00.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.057+0000 7f46f2d3e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46ec09e410 0x7f46ec134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:00.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.057+0000 7f46f2d3e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46ec09b120 con 0x7f46ec09e000 2023-10-03T10:16:00.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.057+0000 7f46f2d3e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec06b460 con 0x7f46ec09d6f0 2023-10-03T10:16:00.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.057+0000 7f46f2d3e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec09a7e0 con 0x7f46ec09e410 2023-10-03T10:16:00.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.057+0000 7f46ebfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f46ec09e000 0x7f46ec1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36188/0 (socket says 172.21.15.3:36188) 2023-10-03T10:16:00.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.057+0000 7f46ebfff700 1 -- 172.21.15.3:0/2218053797 learned_addr learned my addr 172.21.15.3:0/2218053797 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:00.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.057+0000 7f46f0ada700 1 --2- 172.21.15.3:0/2218053797 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46ec09d6f0 0x7f46ec09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:00.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.058+0000 7f46f12db700 1 --2- 172.21.15.3:0/2218053797 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46ec09e410 0x7f46ec134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:00.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.058+0000 7f46eb7fe700 1 -- 172.21.15.3:0/2218053797 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 168327402 0 0) 0x7f46ec09b120 con 0x7f46ec09e000 2023-10-03T10:16:00.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.058+0000 7f46eb7fe700 1 -- 172.21.15.3:0/2218053797 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46dc0035b0 con 0x7f46ec09e000 2023-10-03T10:16:00.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.058+0000 7f46f0ada700 1 -- 172.21.15.3:0/2218053797 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46ec09e410 msgr2=0x7f46ec134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:00.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.058+0000 7f46f0ada700 1 --2- 172.21.15.3:0/2218053797 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46ec09e410 0x7f46ec134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.058+0000 7f46f0ada700 1 -- 172.21.15.3:0/2218053797 >> v1:172.21.15.3:6789/0 conn(0x7f46ec09e000 legacy=0x7f46ec1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:00.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.058+0000 7f46f0ada700 1 -- 172.21.15.3:0/2218053797 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ec09a500 con 0x7f46ec09d6f0 2023-10-03T10:16:00.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.058+0000 7f46f0ada700 1 --2- 172.21.15.3:0/2218053797 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46ec09d6f0 0x7f46ec09dac0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f46e0009870 tx=0x7f46e0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa12b50e5c129457 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:00.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.058+0000 7f46eb7fe700 1 -- 172.21.15.3:0/2218053797 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e0011830 con 0x7f46ec09d6f0 2023-10-03T10:16:00.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.058+0000 7f46eb7fe700 1 -- 172.21.15.3:0/2218053797 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e0011990 con 0x7f46ec09d6f0 2023-10-03T10:16:00.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.058+0000 7f46eb7fe700 1 -- 172.21.15.3:0/2218053797 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e0011d20 con 0x7f46ec09d6f0 2023-10-03T10:16:00.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.059+0000 7f46f2d3e700 1 -- 172.21.15.3:0/2218053797 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46ec09d6f0 msgr2=0x7f46ec09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:00.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.059+0000 7f46f2d3e700 1 --2- 172.21.15.3:0/2218053797 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46ec09d6f0 0x7f46ec09dac0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f46e0009870 tx=0x7f46e0009b80 comp rx=0 tx=0).stop 2023-10-03T10:16:00.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.059+0000 7f46f2d3e700 1 -- 172.21.15.3:0/2218053797 shutdown_connections 2023-10-03T10:16:00.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.059+0000 7f46f2d3e700 1 --2- 172.21.15.3:0/2218053797 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46ec09d6f0 0x7f46ec09dac0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.059+0000 7f46f2d3e700 1 --2- 172.21.15.3:0/2218053797 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46ec09e410 0x7f46ec134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.059+0000 7f46f2d3e700 1 -- 172.21.15.3:0/2218053797 >> 172.21.15.3:0/2218053797 conn(0x7f46ec066f50 msgr2=0x7f46ec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:00.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.060+0000 7f46f2d3e700 1 -- 172.21.15.3:0/2218053797 shutdown_connections 2023-10-03T10:16:00.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.060+0000 7f46f2d3e700 1 -- 172.21.15.3:0/2218053797 wait complete. 2023-10-03T10:16:00.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.061+0000 7f46f2d3e700 1 Processor -- start 2023-10-03T10:16:00.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.061+0000 7f46f2d3e700 1 -- start start 2023-10-03T10:16:00.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.061+0000 7f46f2d3e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46ec09d6f0 0x7f46ec1223f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:00.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.061+0000 7f46f2d3e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46ec09e000 0x7f46ec122930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:00.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.061+0000 7f46f2d3e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46ec09e410 0x7f46ec122e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:00.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.061+0000 7f46f2d3e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec124cd0 con 0x7f46ec09e410 2023-10-03T10:16:00.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.061+0000 7f46f2d3e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec1336b0 con 0x7f46ec09e000 2023-10-03T10:16:00.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.061+0000 7f46f2d3e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec125de0 con 0x7f46ec09d6f0 2023-10-03T10:16:00.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.062+0000 7f46f0ada700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46ec09d6f0 0x7f46ec1223f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:00.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.062+0000 7f46f0ada700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46ec09d6f0 0x7f46ec1223f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:58560/0 (socket says 172.21.15.3:58560) 2023-10-03T10:16:00.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.062+0000 7f46f0ada700 1 -- 172.21.15.3:0/847117642 learned_addr learned my addr 172.21.15.3:0/847117642 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:00.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.062+0000 7f46f12db700 1 --2- 172.21.15.3:0/847117642 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46ec09e410 0x7f46ec122e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:00.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.062+0000 7f46ebfff700 1 --2- 172.21.15.3:0/847117642 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46ec09e000 0x7f46ec122930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:00.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.062+0000 7f46f0ada700 1 -- 172.21.15.3:0/847117642 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46ec09e000 msgr2=0x7f46ec122930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:00.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.062+0000 7f46f0ada700 1 --2- 172.21.15.3:0/847117642 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46ec09e000 0x7f46ec122930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.062+0000 7f46f0ada700 1 -- 172.21.15.3:0/847117642 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46ec09e410 msgr2=0x7f46ec122e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:00.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.062+0000 7f46f0ada700 1 --2- 172.21.15.3:0/847117642 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46ec09e410 0x7f46ec122e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.062+0000 7f46f0ada700 1 -- 172.21.15.3:0/847117642 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e00094a0 con 0x7f46ec09d6f0 2023-10-03T10:16:00.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.062+0000 7f46f0ada700 1 --2- 172.21.15.3:0/847117642 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46ec09d6f0 0x7f46ec1223f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f46e00090b0 tx=0x7f46e0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:00.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.063+0000 7f46e97fa700 1 -- 172.21.15.3:0/847117642 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e0004510 con 0x7f46ec09d6f0 2023-10-03T10:16:00.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.063+0000 7f46e97fa700 1 -- 172.21.15.3:0/847117642 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e001f070 con 0x7f46ec09d6f0 2023-10-03T10:16:00.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.063+0000 7f46e97fa700 1 -- 172.21.15.3:0/847117642 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e00038c0 con 0x7f46ec09d6f0 2023-10-03T10:16:00.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.063+0000 7f46f2d3e700 1 -- 172.21.15.3:0/847117642 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46ec125840 con 0x7f46ec09d6f0 2023-10-03T10:16:00.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.063+0000 7f46f2d3e700 1 -- 172.21.15.3:0/847117642 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46ec125b00 con 0x7f46ec09d6f0 2023-10-03T10:16:00.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.064+0000 7f46e97fa700 1 -- 172.21.15.3:0/847117642 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f46e0003a20 con 0x7f46ec09d6f0 2023-10-03T10:16:00.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.064+0000 7f46f2d3e700 1 -- 172.21.15.3:0/847117642 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46ec074ea0 con 0x7f46ec09d6f0 2023-10-03T10:16:00.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.064+0000 7f46e97fa700 1 -- 172.21.15.3:0/847117642 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f46e0021070 con 0x7f46ec09d6f0 2023-10-03T10:16:00.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.070+0000 7f46e97fa700 1 -- 172.21.15.3:0/847117642 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f46e0015080 con 0x7f46ec09d6f0 2023-10-03T10:16:00.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.092+0000 7f9b9b9f0700 1 Processor -- start 2023-10-03T10:16:00.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.092+0000 7f9b9b9f0700 1 -- start start 2023-10-03T10:16:00.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.093+0000 7f9b9b9f0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9b9409e930 0x7f9b9409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:00.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.093+0000 7f9b9b9f0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b9409f650 0x7f9b94131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:00.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.093+0000 7f9b9b9f0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b94070940 con 0x7f9b9409f240 2023-10-03T10:16:00.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.093+0000 7f9b9b9f0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b9406fd60 con 0x7f9b9409e930 2023-10-03T10:16:00.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.093+0000 7f9b9b9f0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9406fa80 con 0x7f9b9409f650 2023-10-03T10:16:00.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.093+0000 7f9b9978c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9b9409e930 0x7f9b9409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:00.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.093+0000 7f9b9978c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9b9409e930 0x7f9b9409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32876/0 (socket says 172.21.15.3:32876) 2023-10-03T10:16:00.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.093+0000 7f9b9978c700 1 -- 172.21.15.3:0/1230433111 learned_addr learned my addr 172.21.15.3:0/1230433111 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:00.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.093+0000 7f9b99f8d700 1 --2- 172.21.15.3:0/1230433111 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b9409f650 0x7f9b94131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:00.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.094+0000 7f9b99f8d700 1 -- 172.21.15.3:0/1230433111 >> v2:172.21.15.3:3300/0 conn(0x7f9b9409e930 msgr2=0x7f9b9409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:00.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.094+0000 7f9b99f8d700 1 --2- 172.21.15.3:0/1230433111 >> v2:172.21.15.3:3300/0 conn(0x7f9b9409e930 0x7f9b9409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.094+0000 7f9b99f8d700 1 -- 172.21.15.3:0/1230433111 >> v1:172.21.15.3:6789/0 conn(0x7f9b9409f240 legacy=0x7f9b941317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:00.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.094+0000 7f9b99f8d700 1 -- 172.21.15.3:0/1230433111 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9409fb60 con 0x7f9b9409f650 2023-10-03T10:16:00.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.094+0000 7f9b8bfff700 1 -- 172.21.15.3:0/1230433111 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 867969627 0 0) 0x7f9b94070940 con 0x7f9b9409f240 2023-10-03T10:16:00.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.094+0000 7f9b99f8d700 1 --2- 172.21.15.3:0/1230433111 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b9409f650 0x7f9b94131f00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c009d90 tx=0x7f9b8c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15ff8d4e212e1ac3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:00.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.094+0000 7f9b8bfff700 1 -- 172.21.15.3:0/1230433111 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c00e070 con 0x7f9b9409f650 2023-10-03T10:16:00.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.094+0000 7f9b8bfff700 1 -- 172.21.15.3:0/1230433111 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8c00bb20 con 0x7f9b9409f650 2023-10-03T10:16:00.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.094+0000 7f9b8bfff700 1 -- 172.21.15.3:0/1230433111 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c012430 con 0x7f9b9409f650 2023-10-03T10:16:00.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.095+0000 7f9b9b9f0700 1 -- 172.21.15.3:0/1230433111 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b9409f650 msgr2=0x7f9b94131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:00.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.095+0000 7f9b9b9f0700 1 --2- 172.21.15.3:0/1230433111 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b9409f650 0x7f9b94131f00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c009d90 tx=0x7f9b8c00b330 comp rx=0 tx=0).stop 2023-10-03T10:16:00.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.095+0000 7f9b9b9f0700 1 -- 172.21.15.3:0/1230433111 shutdown_connections 2023-10-03T10:16:00.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.095+0000 7f9b9b9f0700 1 --2- 172.21.15.3:0/1230433111 >> v2:172.21.15.3:3300/0 conn(0x7f9b9409e930 0x7f9b9409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.095+0000 7f9b9b9f0700 1 --2- 172.21.15.3:0/1230433111 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b9409f650 0x7f9b94131f00 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.095+0000 7f9b9b9f0700 1 -- 172.21.15.3:0/1230433111 >> 172.21.15.3:0/1230433111 conn(0x7f9b940a4210 msgr2=0x7f9b940a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:00.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.095+0000 7f9b9b9f0700 1 -- 172.21.15.3:0/1230433111 shutdown_connections 2023-10-03T10:16:00.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.095+0000 7f9b9b9f0700 1 -- 172.21.15.3:0/1230433111 wait complete. 2023-10-03T10:16:00.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.096+0000 7f9b9b9f0700 1 Processor -- start 2023-10-03T10:16:00.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.096+0000 7f9b9b9f0700 1 -- start start 2023-10-03T10:16:00.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.097+0000 7f9b9b9f0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b9409e930 0x7f9b941c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:00.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.097+0000 7f9b9b9f0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9b9409f240 0x7f9b941c3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:00.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.097+0000 7f9b9b9f0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b9409f650 0x7f9b941c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:00.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.097+0000 7f9b9b9f0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b941a58a0 con 0x7f9b9409f650 2023-10-03T10:16:00.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.097+0000 7f9b9b9f0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b941a56b0 con 0x7f9b9409f240 2023-10-03T10:16:00.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.097+0000 7f9b9b9f0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9b940a32f0 con 0x7f9b9409e930 2023-10-03T10:16:00.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.097+0000 7f9b9978c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b9409e930 0x7f9b941c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:00.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.097+0000 7f9b9978c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b9409e930 0x7f9b941c2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:58592/0 (socket says 172.21.15.3:58592) 2023-10-03T10:16:00.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.097+0000 7f9b9978c700 1 -- 172.21.15.3:0/1499196080 learned_addr learned my addr 172.21.15.3:0/1499196080 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:00.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.097+0000 7f9b98f8b700 1 --2- 172.21.15.3:0/1499196080 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9b9409f240 0x7f9b941c3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:00.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.097+0000 7f9b99f8d700 1 --2- 172.21.15.3:0/1499196080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b9409f650 0x7f9b941c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:00.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.097+0000 7f9b9978c700 1 -- 172.21.15.3:0/1499196080 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9b9409f240 msgr2=0x7f9b941c3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:00.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.097+0000 7f9b9978c700 1 --2- 172.21.15.3:0/1499196080 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9b9409f240 0x7f9b941c3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.097+0000 7f9b9978c700 1 -- 172.21.15.3:0/1499196080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b9409f650 msgr2=0x7f9b941c5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:00.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.097+0000 7f9b9978c700 1 --2- 172.21.15.3:0/1499196080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b9409f650 0x7f9b941c5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.097+0000 7f9b9978c700 1 -- 172.21.15.3:0/1499196080 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8c009a70 con 0x7f9b9409e930 2023-10-03T10:16:00.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.098+0000 7f9b98f8b700 1 --2- 172.21.15.3:0/1499196080 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9b9409f240 0x7f9b941c3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:00.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.098+0000 7f9b9978c700 1 --2- 172.21.15.3:0/1499196080 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b9409e930 0x7f9b941c2e50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000e940 tx=0x7f9b9000ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:00.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.098+0000 7f9b89ffb700 1 -- 172.21.15.3:0/1499196080 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b900096a0 con 0x7f9b9409e930 2023-10-03T10:16:00.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.098+0000 7f9b89ffb700 1 -- 172.21.15.3:0/1499196080 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b90013680 con 0x7f9b9409e930 2023-10-03T10:16:00.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.098+0000 7f9b9b9f0700 1 -- 172.21.15.3:0/1499196080 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b940a0190 con 0x7f9b9409e930 2023-10-03T10:16:00.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.098+0000 7f9b89ffb700 1 -- 172.21.15.3:0/1499196080 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b90013a30 con 0x7f9b9409e930 2023-10-03T10:16:00.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.098+0000 7f9b9b9f0700 1 -- 172.21.15.3:0/1499196080 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b941a6900 con 0x7f9b9409e930 2023-10-03T10:16:00.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.099+0000 7f9b9b9f0700 1 -- 172.21.15.3:0/1499196080 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b94074ea0 con 0x7f9b9409e930 2023-10-03T10:16:00.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.100+0000 7f9b89ffb700 1 -- 172.21.15.3:0/1499196080 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9b90013b90 con 0x7f9b9409e930 2023-10-03T10:16:00.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.100+0000 7f9b89ffb700 1 -- 172.21.15.3:0/1499196080 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9b9003fe10 con 0x7f9b9409e930 2023-10-03T10:16:00.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.104+0000 7fe14fcf3700 1 Processor -- start 2023-10-03T10:16:00.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.105+0000 7f9b89ffb700 1 -- 172.21.15.3:0/1499196080 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9b9000c450 con 0x7f9b9409e930 2023-10-03T10:16:00.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.104+0000 7fe14fcf3700 1 -- start start 2023-10-03T10:16:00.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.105+0000 7fe14fcf3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe14806e570 0x7fe14806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:00.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.105+0000 7fe14fcf3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe14809eca0 0x7fe14806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:00.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.105+0000 7fe14fcf3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe148070ee0 con 0x7fe148134610 2023-10-03T10:16:00.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.105+0000 7fe14fcf3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe148071480 con 0x7fe14806e570 2023-10-03T10:16:00.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.105+0000 7fe14fcf3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe14806f3c0 con 0x7fe14809eca0 2023-10-03T10:16:00.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.105+0000 7fe14da8f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe14806e570 0x7fe14806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:00.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.105+0000 7fe14da8f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe14806e570 0x7fe14806e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39064/0 (socket says 172.21.15.3:39064) 2023-10-03T10:16:00.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.105+0000 7fe14da8f700 1 -- 172.21.15.3:0/4269253574 learned_addr learned my addr 172.21.15.3:0/4269253574 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:00.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.105+0000 7fe14d28e700 1 --2- 172.21.15.3:0/4269253574 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe14809eca0 0x7fe14806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:00.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.105+0000 7fe14d28e700 1 -- 172.21.15.3:0/4269253574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe14806e570 msgr2=0x7fe14806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:00.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.105+0000 7fe14d28e700 1 --2- 172.21.15.3:0/4269253574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe14806e570 0x7fe14806e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.105+0000 7fe14d28e700 1 -- 172.21.15.3:0/4269253574 >> v1:172.21.15.3:6789/0 conn(0x7fe148134610 legacy=0x7fe148134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:00.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.105+0000 7fe14ca8d700 1 -- 172.21.15.3:0/4269253574 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1252932285 0 0) 0x7fe148070ee0 con 0x7fe148134610 2023-10-03T10:16:00.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.105+0000 7fe14d28e700 1 -- 172.21.15.3:0/4269253574 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe148071b90 con 0x7fe14809eca0 2023-10-03T10:16:00.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.106+0000 7fe14da8f700 1 --2- 172.21.15.3:0/4269253574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe14806e570 0x7fe14806e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:00.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.106+0000 7fe14d28e700 1 --2- 172.21.15.3:0/4269253574 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe14809eca0 0x7fe14806ee80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe138009870 tx=0x7fe138009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=104cbd29fafb7e83 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:00.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.106+0000 7fe14ca8d700 1 -- 172.21.15.3:0/4269253574 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138011830 con 0x7fe14809eca0 2023-10-03T10:16:00.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.106+0000 7fe14ca8d700 1 -- 172.21.15.3:0/4269253574 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe138011990 con 0x7fe14809eca0 2023-10-03T10:16:00.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.106+0000 7fe14ca8d700 1 -- 172.21.15.3:0/4269253574 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138011d20 con 0x7fe14809eca0 2023-10-03T10:16:00.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.107+0000 7fe14fcf3700 1 -- 172.21.15.3:0/4269253574 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe14809eca0 msgr2=0x7fe14806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:00.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.107+0000 7fe14fcf3700 1 --2- 172.21.15.3:0/4269253574 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe14809eca0 0x7fe14806ee80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe138009870 tx=0x7fe138009b80 comp rx=0 tx=0).stop 2023-10-03T10:16:00.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.108+0000 7fe14fcf3700 1 -- 172.21.15.3:0/4269253574 shutdown_connections 2023-10-03T10:16:00.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.108+0000 7fe14fcf3700 1 --2- 172.21.15.3:0/4269253574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe14806e570 0x7fe14806e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.108+0000 7fe14fcf3700 1 --2- 172.21.15.3:0/4269253574 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe14809eca0 0x7fe14806ee80 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.108+0000 7fe14fcf3700 1 -- 172.21.15.3:0/4269253574 >> 172.21.15.3:0/4269253574 conn(0x7fe1480a2aa0 msgr2=0x7fe1480a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:00.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.108+0000 7fe14fcf3700 1 -- 172.21.15.3:0/4269253574 shutdown_connections 2023-10-03T10:16:00.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.108+0000 7fe14fcf3700 1 -- 172.21.15.3:0/4269253574 wait complete. 2023-10-03T10:16:00.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.109+0000 7fe14fcf3700 1 Processor -- start 2023-10-03T10:16:00.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.109+0000 7fe14fcf3700 1 -- start start 2023-10-03T10:16:00.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.110+0000 7fe14fcf3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe14806e570 0x7fe1481214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:00.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.110+0000 7fe14fcf3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe148134610 0x7fe148121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:00.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.110+0000 7fe14fcf3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe14809e560 0x7fe148121f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:00.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.110+0000 7fe14fcf3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1480a1fd0 con 0x7fe148134610 2023-10-03T10:16:00.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.110+0000 7fe14fcf3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1481242a0 con 0x7fe14809e560 2023-10-03T10:16:00.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.110+0000 7fe14fcf3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe148070610 con 0x7fe14806e570 2023-10-03T10:16:00.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.110+0000 7fe14da8f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe14806e570 0x7fe1481214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:00.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.110+0000 7fe14da8f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe14806e570 0x7fe1481214e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:58602/0 (socket says 172.21.15.3:58602) 2023-10-03T10:16:00.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.110+0000 7fe14da8f700 1 -- 172.21.15.3:0/102544848 learned_addr learned my addr 172.21.15.3:0/102544848 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:00.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.110+0000 7fe14d28e700 1 --2- 172.21.15.3:0/102544848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe148134610 0x7fe148121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:00.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.110+0000 7fe14e290700 1 --2- 172.21.15.3:0/102544848 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe14809e560 0x7fe148121f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:00.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.111+0000 7fe14d28e700 1 -- 172.21.15.3:0/102544848 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe14806e570 msgr2=0x7fe1481214e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:00.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.111+0000 7fe14d28e700 1 --2- 172.21.15.3:0/102544848 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe14806e570 0x7fe1481214e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.111+0000 7fe14d28e700 1 -- 172.21.15.3:0/102544848 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe14809e560 msgr2=0x7fe148121f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:00.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.111+0000 7fe14d28e700 1 --2- 172.21.15.3:0/102544848 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe14809e560 0x7fe148121f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.111+0000 7fe14d28e700 1 -- 172.21.15.3:0/102544848 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe144009550 con 0x7fe148134610 2023-10-03T10:16:00.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.111+0000 7fe14d28e700 1 --2- 172.21.15.3:0/102544848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe148134610 0x7fe148121a20 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe138016000 tx=0x7fe138004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:00.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.111+0000 7fe13e7fc700 1 -- 172.21.15.3:0/102544848 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1380045d0 con 0x7fe148134610 2023-10-03T10:16:00.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.111+0000 7fe13e7fc700 1 -- 172.21.15.3:0/102544848 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13801f070 con 0x7fe148134610 2023-10-03T10:16:00.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.111+0000 7fe14fcf3700 1 -- 172.21.15.3:0/102544848 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1380094a0 con 0x7fe148134610 2023-10-03T10:16:00.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.111+0000 7fe13e7fc700 1 -- 172.21.15.3:0/102544848 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1380038a0 con 0x7fe148134610 2023-10-03T10:16:00.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.111+0000 7fe14fcf3700 1 -- 172.21.15.3:0/102544848 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe148122aa0 con 0x7fe148134610 2023-10-03T10:16:00.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.113+0000 7fe13e7fc700 1 -- 172.21.15.3:0/102544848 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe138003a00 con 0x7fe148134610 2023-10-03T10:16:00.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.113+0000 7fe12bfff700 1 -- 172.21.15.3:0/102544848 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1240052e0 con 0x7fe148134610 2023-10-03T10:16:00.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.113+0000 7fe13e7fc700 1 -- 172.21.15.3:0/102544848 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe138021070 con 0x7fe148134610 2023-10-03T10:16:00.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.123+0000 7fe13e7fc700 1 -- 172.21.15.3:0/102544848 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe138010de0 con 0x7fe148134610 2023-10-03T10:16:00.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.253+0000 7f41fe78a700 1 -- 172.21.15.3:0/811701887 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f41f806b9e0 con 0x7f41f809de50 2023-10-03T10:16:00.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.253+0000 7f41f4ff9700 1 -- 172.21.15.3:0/811701887 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f41e8021090 con 0x7f41f809de50 2023-10-03T10:16:00.254 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:00.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.256+0000 7f41fe78a700 1 -- 172.21.15.3:0/811701887 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f41f809de50 msgr2=0x7f41f811eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:00.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.256+0000 7f41fe78a700 1 --2- 172.21.15.3:0/811701887 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f41f809de50 0x7f41f811eb50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f41e800e9b0 tx=0x7f41e800ecc0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.256+0000 7f41fe78a700 1 -- 172.21.15.3:0/811701887 shutdown_connections 2023-10-03T10:16:00.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.256+0000 7f41fe78a700 1 --2- 172.21.15.3:0/811701887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41f81338a0 0x7f41f811f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.256+0000 7f41fe78a700 1 --2- 172.21.15.3:0/811701887 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f41f809de50 0x7f41f811eb50 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.256+0000 7f41fe78a700 1 --2- 172.21.15.3:0/811701887 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f41f809e760 0x7f41f811f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.256+0000 7f41fe78a700 1 -- 172.21.15.3:0/811701887 >> 172.21.15.3:0/811701887 conn(0x7f41f80a2aa0 msgr2=0x7f41f809ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:00.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.256+0000 7f41fe78a700 1 -- 172.21.15.3:0/811701887 shutdown_connections 2023-10-03T10:16:00.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.256+0000 7f41fe78a700 1 -- 172.21.15.3:0/811701887 wait complete. 2023-10-03T10:16:00.270 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:00.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.288+0000 7f46f2d3e700 1 -- 172.21.15.3:0/847117642 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f46ec123820 con 0x7f46ec09d6f0 2023-10-03T10:16:00.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.289+0000 7f46e97fa700 1 -- 172.21.15.3:0/847117642 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f46e0035090 con 0x7f46ec09d6f0 2023-10-03T10:16:00.289 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:00.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.291+0000 7f46f2d3e700 1 -- 172.21.15.3:0/847117642 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46ec09d6f0 msgr2=0x7f46ec1223f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:00.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.291+0000 7f46f2d3e700 1 --2- 172.21.15.3:0/847117642 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46ec09d6f0 0x7f46ec1223f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f46e00090b0 tx=0x7f46e0004750 comp rx=0 tx=0).stop 2023-10-03T10:16:00.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.291+0000 7f46f2d3e700 1 -- 172.21.15.3:0/847117642 shutdown_connections 2023-10-03T10:16:00.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.291+0000 7f46f2d3e700 1 --2- 172.21.15.3:0/847117642 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46ec09e410 0x7f46ec122e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.291+0000 7f46f2d3e700 1 --2- 172.21.15.3:0/847117642 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46ec09d6f0 0x7f46ec1223f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.291+0000 7f46f2d3e700 1 --2- 172.21.15.3:0/847117642 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46ec09e000 0x7f46ec122930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.291+0000 7f46f2d3e700 1 -- 172.21.15.3:0/847117642 >> 172.21.15.3:0/847117642 conn(0x7f46ec066f50 msgr2=0x7f46ec09b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:00.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.291+0000 7f46f2d3e700 1 -- 172.21.15.3:0/847117642 shutdown_connections 2023-10-03T10:16:00.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.291+0000 7f46f2d3e700 1 -- 172.21.15.3:0/847117642 wait complete. 2023-10-03T10:16:00.307 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:00.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.341+0000 7f9b9b9f0700 1 -- 172.21.15.3:0/1499196080 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9b9406b250 con 0x7f9b9409e930 2023-10-03T10:16:00.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.341+0000 7f9b89ffb700 1 -- 172.21.15.3:0/1499196080 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9b90021070 con 0x7f9b9409e930 2023-10-03T10:16:00.342 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:00.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.343+0000 7f9b9b9f0700 1 -- 172.21.15.3:0/1499196080 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b9409e930 msgr2=0x7f9b941c2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:00.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.343+0000 7f9b9b9f0700 1 --2- 172.21.15.3:0/1499196080 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b9409e930 0x7f9b941c2e50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000e940 tx=0x7f9b9000ec50 comp rx=0 tx=0).stop 2023-10-03T10:16:00.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.343+0000 7f9b9b9f0700 1 -- 172.21.15.3:0/1499196080 shutdown_connections 2023-10-03T10:16:00.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.343+0000 7f9b9b9f0700 1 --2- 172.21.15.3:0/1499196080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b9409f650 0x7f9b941c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.343+0000 7f9b9b9f0700 1 --2- 172.21.15.3:0/1499196080 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b9409e930 0x7f9b941c2e50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.343+0000 7f9b9b9f0700 1 --2- 172.21.15.3:0/1499196080 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9b9409f240 0x7f9b941c3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.344+0000 7f9b9b9f0700 1 -- 172.21.15.3:0/1499196080 >> 172.21.15.3:0/1499196080 conn(0x7f9b940a4210 msgr2=0x7f9b941b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:00.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.344+0000 7f9b9b9f0700 1 -- 172.21.15.3:0/1499196080 shutdown_connections 2023-10-03T10:16:00.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.344+0000 7f9b9b9f0700 1 -- 172.21.15.3:0/1499196080 wait complete. 2023-10-03T10:16:00.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.352+0000 7fe12bfff700 1 -- 172.21.15.3:0/102544848 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe124005590 con 0x7fe148134610 2023-10-03T10:16:00.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.353+0000 7fe13e7fc700 1 -- 172.21.15.3:0/102544848 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe138015070 con 0x7fe148134610 2023-10-03T10:16:00.353 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:00.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.355+0000 7fe12bfff700 1 -- 172.21.15.3:0/102544848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe148134610 msgr2=0x7fe148121a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:00.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.355+0000 7fe12bfff700 1 --2- 172.21.15.3:0/102544848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe148134610 0x7fe148121a20 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe138016000 tx=0x7fe138004750 comp rx=0 tx=0).stop 2023-10-03T10:16:00.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.355+0000 7fe12bfff700 1 -- 172.21.15.3:0/102544848 shutdown_connections 2023-10-03T10:16:00.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.355+0000 7fe12bfff700 1 --2- 172.21.15.3:0/102544848 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe14809e560 0x7fe148121f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.355+0000 7fe12bfff700 1 --2- 172.21.15.3:0/102544848 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe14806e570 0x7fe1481214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.355+0000 7fe12bfff700 1 --2- 172.21.15.3:0/102544848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe148134610 0x7fe148121a20 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:00.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.355+0000 7fe12bfff700 1 -- 172.21.15.3:0/102544848 >> 172.21.15.3:0/102544848 conn(0x7fe1480a2aa0 msgr2=0x7fe1481bf070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:00.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.355+0000 7fe12bfff700 1 -- 172.21.15.3:0/102544848 shutdown_connections 2023-10-03T10:16:00.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:00.355+0000 7fe12bfff700 1 -- 172.21.15.3:0/102544848 wait complete. 2023-10-03T10:16:00.357 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:00.369 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:01.272 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:01.308 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:01.358 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:01.370 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:01.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.425+0000 7fbc6f7e4700 1 Processor -- start 2023-10-03T10:16:01.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.425+0000 7fbc6f7e4700 1 -- start start 2023-10-03T10:16:01.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.426+0000 7fbc6f7e4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc6806e570 0x7fbc6806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:01.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.426+0000 7fbc6f7e4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbc6809eca0 0x7fbc6806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:01.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.426+0000 7fbc6f7e4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc68134610 0x7fbc68134a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:01.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.426+0000 7fbc6f7e4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc68070ee0 con 0x7fbc6809eca0 2023-10-03T10:16:01.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.426+0000 7fbc6f7e4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc68071480 con 0x7fbc6806e570 2023-10-03T10:16:01.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.426+0000 7fbc6f7e4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6806f3c0 con 0x7fbc68134610 2023-10-03T10:16:01.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.426+0000 7fbc6d580700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc6806e570 0x7fbc6806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:01.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.426+0000 7fbc6dd81700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc68134610 0x7fbc68134a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:01.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.426+0000 7fbc6cd7f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbc6809eca0 0x7fbc6806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:01.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.426+0000 7fbc6d580700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc6806e570 0x7fbc6806e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39078/0 (socket says 172.21.15.3:39078) 2023-10-03T10:16:01.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.426+0000 7fbc6d580700 1 -- 172.21.15.3:0/3231367659 learned_addr learned my addr 172.21.15.3:0/3231367659 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:01.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.426+0000 7fbc6d580700 1 -- 172.21.15.3:0/3231367659 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc68134610 msgr2=0x7fbc68134a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:01.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.426+0000 7fbc6d580700 1 --2- 172.21.15.3:0/3231367659 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc68134610 0x7fbc68134a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.426+0000 7fbc6d580700 1 -- 172.21.15.3:0/3231367659 >> v2:172.21.15.3:3300/0 conn(0x7fbc6809eca0 msgr2=0x7fbc6806ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:01.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.426+0000 7fbc6d580700 1 --2- 172.21.15.3:0/3231367659 >> v2:172.21.15.3:3300/0 conn(0x7fbc6809eca0 0x7fbc6806ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.426+0000 7fbc6d580700 1 -- 172.21.15.3:0/3231367659 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc68071b90 con 0x7fbc6806e570 2023-10-03T10:16:01.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.426+0000 7fbc6cd7f700 1 --2- 172.21.15.3:0/3231367659 >> v2:172.21.15.3:3300/0 conn(0x7fbc6809eca0 0x7fbc6806ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:01.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.427+0000 7fbc6d580700 1 --2- 172.21.15.3:0/3231367659 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc6806e570 0x7fbc6806e940 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbc640098f0 tx=0x7fbc64009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43e095060b2e71f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:01.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.427+0000 7fbc5ffff700 1 -- 172.21.15.3:0/3231367659 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64011830 con 0x7fbc6806e570 2023-10-03T10:16:01.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.427+0000 7fbc5ffff700 1 -- 172.21.15.3:0/3231367659 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc64011990 con 0x7fbc6806e570 2023-10-03T10:16:01.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.427+0000 7fbc5ffff700 1 -- 172.21.15.3:0/3231367659 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64011d20 con 0x7fbc6806e570 2023-10-03T10:16:01.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.428+0000 7fbc6f7e4700 1 -- 172.21.15.3:0/3231367659 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc6806e570 msgr2=0x7fbc6806e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:01.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.428+0000 7fbc6f7e4700 1 --2- 172.21.15.3:0/3231367659 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc6806e570 0x7fbc6806e940 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbc640098f0 tx=0x7fbc64009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:01.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.428+0000 7fbc6f7e4700 1 -- 172.21.15.3:0/3231367659 shutdown_connections 2023-10-03T10:16:01.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.428+0000 7fbc6f7e4700 1 --2- 172.21.15.3:0/3231367659 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc68134610 0x7fbc68134a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.428+0000 7fbc6f7e4700 1 --2- 172.21.15.3:0/3231367659 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc6806e570 0x7fbc6806e940 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.428+0000 7fbc6f7e4700 1 --2- 172.21.15.3:0/3231367659 >> v2:172.21.15.3:3300/0 conn(0x7fbc6809eca0 0x7fbc6806ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.428+0000 7fbc6f7e4700 1 -- 172.21.15.3:0/3231367659 >> 172.21.15.3:0/3231367659 conn(0x7fbc680a2aa0 msgr2=0x7fbc680a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:01.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.428+0000 7fbc6f7e4700 1 -- 172.21.15.3:0/3231367659 shutdown_connections 2023-10-03T10:16:01.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.428+0000 7fbc6f7e4700 1 -- 172.21.15.3:0/3231367659 wait complete. 2023-10-03T10:16:01.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.429+0000 7fbc6f7e4700 1 Processor -- start 2023-10-03T10:16:01.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.430+0000 7fbc6f7e4700 1 -- start start 2023-10-03T10:16:01.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.430+0000 7fbc6f7e4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc6806e570 0x7fbc681276a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:01.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.430+0000 7fbc6f7e4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc68134610 0x7fbc68139920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:01.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.430+0000 7fbc6f7e4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc68139e60 0x7fbc6813a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:01.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.430+0000 7fbc6f7e4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6813e3d0 con 0x7fbc6806e570 2023-10-03T10:16:01.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.430+0000 7fbc6f7e4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6813b620 con 0x7fbc68139e60 2023-10-03T10:16:01.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.430+0000 7fbc6f7e4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbc68127be0 con 0x7fbc68134610 2023-10-03T10:16:01.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.430+0000 7fbc6d580700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc6806e570 0x7fbc681276a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:01.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.430+0000 7fbc6d580700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc6806e570 0x7fbc681276a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32910/0 (socket says 172.21.15.3:32910) 2023-10-03T10:16:01.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.430+0000 7fbc6cd7f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc68134610 0x7fbc68139920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:01.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.430+0000 7fbc6d580700 1 -- 172.21.15.3:0/903631280 learned_addr learned my addr 172.21.15.3:0/903631280 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:01.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.431+0000 7fbc6d580700 1 -- 172.21.15.3:0/903631280 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc68134610 msgr2=0x7fbc68139920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:01.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.431+0000 7fbc6d580700 1 --2- 172.21.15.3:0/903631280 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc68134610 0x7fbc68139920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.431+0000 7fbc6d580700 1 -- 172.21.15.3:0/903631280 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc68139e60 msgr2=0x7fbc6813a2e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:16:01.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.431+0000 7fbc6d580700 1 --2- 172.21.15.3:0/903631280 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc68139e60 0x7fbc6813a2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.431+0000 7fbc6d580700 1 -- 172.21.15.3:0/903631280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc580094a0 con 0x7fbc6806e570 2023-10-03T10:16:01.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.431+0000 7fbc6cd7f700 1 --2- 172.21.15.3:0/903631280 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc68134610 0x7fbc68139920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:01.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.431+0000 7fbc6d580700 1 --2- 172.21.15.3:0/903631280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc6806e570 0x7fbc681276a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fbc680715c0 tx=0x7fbc64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:01.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.431+0000 7fbc5dffb700 1 -- 172.21.15.3:0/903631280 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64004240 con 0x7fbc6806e570 2023-10-03T10:16:01.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.431+0000 7fbc5dffb700 1 -- 172.21.15.3:0/903631280 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc6401f070 con 0x7fbc6806e570 2023-10-03T10:16:01.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.431+0000 7fbc5dffb700 1 -- 172.21.15.3:0/903631280 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc640045e0 con 0x7fbc6806e570 2023-10-03T10:16:01.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.431+0000 7fbc6f7e4700 1 -- 172.21.15.3:0/903631280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc64009550 con 0x7fbc6806e570 2023-10-03T10:16:01.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.432+0000 7fbc6f7e4700 1 -- 172.21.15.3:0/903631280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc6813d0d0 con 0x7fbc6806e570 2023-10-03T10:16:01.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.433+0000 7fbc5dffb700 1 -- 172.21.15.3:0/903631280 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbc64021030 con 0x7fbc6806e570 2023-10-03T10:16:01.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.433+0000 7fbc5dffb700 1 -- 172.21.15.3:0/903631280 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fbc64040780 con 0x7fbc6806e570 2023-10-03T10:16:01.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.433+0000 7fbc6f7e4700 1 -- 172.21.15.3:0/903631280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc68074ea0 con 0x7fbc6806e570 2023-10-03T10:16:01.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.443+0000 7fbc5dffb700 1 -- 172.21.15.3:0/903631280 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbc64015080 con 0x7fbc6806e570 2023-10-03T10:16:01.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.464+0000 7f4897bc0700 1 Processor -- start 2023-10-03T10:16:01.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.465+0000 7f4897bc0700 1 -- start start 2023-10-03T10:16:01.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.465+0000 7f4897bc0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f489009d6f0 0x7f489009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:01.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.465+0000 7f4897bc0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f489009e000 0x7f48901339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:01.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.465+0000 7f4897bc0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f489009b120 con 0x7f489009e410 2023-10-03T10:16:01.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.465+0000 7f4897bc0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489006b460 con 0x7f489009e000 2023-10-03T10:16:01.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.465+0000 7f4897bc0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f489009a7e0 con 0x7f489009d6f0 2023-10-03T10:16:01.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.465+0000 7f489515b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f489009e000 0x7f48901339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:01.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.465+0000 7f489595c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f489009d6f0 0x7f489009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:01.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.465+0000 7f489615d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f489009e410 0x7f4890133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36208/0 (socket says 172.21.15.3:36208) 2023-10-03T10:16:01.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.465+0000 7f489615d700 1 -- 172.21.15.3:0/2876361073 learned_addr learned my addr 172.21.15.3:0/2876361073 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:01.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.466+0000 7f489515b700 1 -- 172.21.15.3:0/2876361073 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f489009d6f0 msgr2=0x7f489009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:01.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.466+0000 7f489515b700 1 --2- 172.21.15.3:0/2876361073 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f489009d6f0 0x7f489009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.466+0000 7f489515b700 1 -- 172.21.15.3:0/2876361073 >> v1:172.21.15.3:6789/0 conn(0x7f489009e410 legacy=0x7f4890133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:01.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.466+0000 7f489515b700 1 -- 172.21.15.3:0/2876361073 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489009a500 con 0x7f489009e000 2023-10-03T10:16:01.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.466+0000 7f489515b700 1 --2- 172.21.15.3:0/2876361073 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f489009e000 0x7f48901339f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4880009870 tx=0x7f4880009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74b91b1f7664b594 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:01.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.466+0000 7f489495a700 1 -- 172.21.15.3:0/2876361073 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4880011830 con 0x7f489009e000 2023-10-03T10:16:01.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.466+0000 7f489495a700 1 -- 172.21.15.3:0/2876361073 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4880011990 con 0x7f489009e000 2023-10-03T10:16:01.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.466+0000 7f489495a700 1 -- 172.21.15.3:0/2876361073 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4880011d20 con 0x7f489009e000 2023-10-03T10:16:01.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.467+0000 7f4897bc0700 1 -- 172.21.15.3:0/2876361073 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f489009e000 msgr2=0x7f48901339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:01.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.467+0000 7f4897bc0700 1 --2- 172.21.15.3:0/2876361073 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f489009e000 0x7f48901339f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4880009870 tx=0x7f4880009b80 comp rx=0 tx=0).stop 2023-10-03T10:16:01.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.467+0000 7f4897bc0700 1 -- 172.21.15.3:0/2876361073 shutdown_connections 2023-10-03T10:16:01.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.467+0000 7f4897bc0700 1 --2- 172.21.15.3:0/2876361073 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f489009d6f0 0x7f489009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.467+0000 7f4897bc0700 1 --2- 172.21.15.3:0/2876361073 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f489009e000 0x7f48901339f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.467+0000 7f4897bc0700 1 -- 172.21.15.3:0/2876361073 >> 172.21.15.3:0/2876361073 conn(0x7f4890066f50 msgr2=0x7f489009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:01.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.467+0000 7f4897bc0700 1 -- 172.21.15.3:0/2876361073 shutdown_connections 2023-10-03T10:16:01.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.467+0000 7f4897bc0700 1 -- 172.21.15.3:0/2876361073 wait complete. 2023-10-03T10:16:01.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.468+0000 7f4897bc0700 1 Processor -- start 2023-10-03T10:16:01.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.468+0000 7f4897bc0700 1 -- start start 2023-10-03T10:16:01.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.469+0000 7f4897bc0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f489009d6f0 0x7f4890122510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:01.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.469+0000 7f4897bc0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f489009e000 0x7f4890122a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:01.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.469+0000 7f4897bc0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f489009e410 0x7f4890122f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:01.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.469+0000 7f4897bc0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48900a2d90 con 0x7f489009e410 2023-10-03T10:16:01.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.469+0000 7f4897bc0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489009c710 con 0x7f489009e000 2023-10-03T10:16:01.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.469+0000 7f4897bc0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4890124f40 con 0x7f489009d6f0 2023-10-03T10:16:01.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.469+0000 7f489595c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f489009d6f0 0x7f4890122510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:01.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.469+0000 7f489595c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f489009d6f0 0x7f4890122510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:58648/0 (socket says 172.21.15.3:58648) 2023-10-03T10:16:01.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.469+0000 7f489595c700 1 -- 172.21.15.3:0/3775166 learned_addr learned my addr 172.21.15.3:0/3775166 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:01.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.469+0000 7f489515b700 1 --2- 172.21.15.3:0/3775166 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f489009e000 0x7f4890122a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:01.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.469+0000 7f489615d700 1 --2- 172.21.15.3:0/3775166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f489009e410 0x7f4890122f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:01.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.469+0000 7f489595c700 1 -- 172.21.15.3:0/3775166 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f489009e000 msgr2=0x7f4890122a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:01.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.469+0000 7f489595c700 1 --2- 172.21.15.3:0/3775166 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f489009e000 0x7f4890122a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.469+0000 7f489595c700 1 -- 172.21.15.3:0/3775166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f489009e410 msgr2=0x7f4890122f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:01.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.469+0000 7f489595c700 1 --2- 172.21.15.3:0/3775166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f489009e410 0x7f4890122f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.469+0000 7f489595c700 1 -- 172.21.15.3:0/3775166 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48800094a0 con 0x7f489009d6f0 2023-10-03T10:16:01.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.470+0000 7f489595c700 1 --2- 172.21.15.3:0/3775166 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f489009d6f0 0x7f4890122510 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f488c00e9b0 tx=0x7f488c00ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:01.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.470+0000 7f48867fc700 1 -- 172.21.15.3:0/3775166 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c009710 con 0x7f489009d6f0 2023-10-03T10:16:01.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.470+0000 7f48867fc700 1 -- 172.21.15.3:0/3775166 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488c00ee50 con 0x7f489009d6f0 2023-10-03T10:16:01.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.470+0000 7f4897bc0700 1 -- 172.21.15.3:0/3775166 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48901257c0 con 0x7f489009d6f0 2023-10-03T10:16:01.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.470+0000 7f48867fc700 1 -- 172.21.15.3:0/3775166 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c00ca50 con 0x7f489009d6f0 2023-10-03T10:16:01.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.470+0000 7f4897bc0700 1 -- 172.21.15.3:0/3775166 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48901267c0 con 0x7f489009d6f0 2023-10-03T10:16:01.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.471+0000 7f48867fc700 1 -- 172.21.15.3:0/3775166 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f488c00cbb0 con 0x7f489009d6f0 2023-10-03T10:16:01.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.471+0000 7f4897bc0700 1 -- 172.21.15.3:0/3775166 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4890074ea0 con 0x7f489009d6f0 2023-10-03T10:16:01.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.471+0000 7f48867fc700 1 -- 172.21.15.3:0/3775166 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f488c03f8f0 con 0x7f489009d6f0 2023-10-03T10:16:01.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.480+0000 7f48867fc700 1 -- 172.21.15.3:0/3775166 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f488c0232f0 con 0x7f489009d6f0 2023-10-03T10:16:01.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.515+0000 7ff480d37700 1 Processor -- start 2023-10-03T10:16:01.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.515+0000 7ff480d37700 1 -- start start 2023-10-03T10:16:01.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.516+0000 7ff480d37700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff47c0a1510 0x7ff47c138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:01.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.516+0000 7ff480d37700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff47c139250 0x7ff47c13b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:01.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.516+0000 7ff480d37700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff47c09edd0 con 0x7ff47c0a0a30 2023-10-03T10:16:01.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.516+0000 7ff480d37700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff47c0a18e0 con 0x7ff47c139250 2023-10-03T10:16:01.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.516+0000 7ff480d37700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c09e1f0 con 0x7ff47c0a1510 2023-10-03T10:16:01.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.516+0000 7ff47ad9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff47c139250 0x7ff47c13b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:01.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.516+0000 7ff47ad9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff47c139250 0x7ff47c13b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32916/0 (socket says 172.21.15.3:32916) 2023-10-03T10:16:01.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.516+0000 7ff47ad9d700 1 -- 172.21.15.3:0/290581100 learned_addr learned my addr 172.21.15.3:0/290581100 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:01.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.516+0000 7ff479d9b700 1 --2- 172.21.15.3:0/290581100 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff47c0a1510 0x7ff47c138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:01.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.517+0000 7ff47ad9d700 1 -- 172.21.15.3:0/290581100 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff47c0a1510 msgr2=0x7ff47c138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:01.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.517+0000 7ff47ad9d700 1 --2- 172.21.15.3:0/290581100 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff47c0a1510 0x7ff47c138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.517+0000 7ff47ad9d700 1 -- 172.21.15.3:0/290581100 >> v1:172.21.15.3:6789/0 conn(0x7ff47c0a0a30 legacy=0x7ff47c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:01.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.517+0000 7ff47ad9d700 1 -- 172.21.15.3:0/290581100 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47c09dc50 con 0x7ff47c139250 2023-10-03T10:16:01.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.517+0000 7ff47ad9d700 1 --2- 172.21.15.3:0/290581100 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff47c139250 0x7ff47c13b630 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff474009e80 tx=0x7ff47400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9550aba974dbf6 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:01.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.517+0000 7ff47959a700 1 -- 172.21.15.3:0/290581100 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47400e070 con 0x7ff47c139250 2023-10-03T10:16:01.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.517+0000 7ff47959a700 1 -- 172.21.15.3:0/290581100 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47400b8d0 con 0x7ff47c139250 2023-10-03T10:16:01.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.517+0000 7ff47959a700 1 -- 172.21.15.3:0/290581100 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47400bc80 con 0x7ff47c139250 2023-10-03T10:16:01.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.518+0000 7ff480d37700 1 -- 172.21.15.3:0/290581100 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff47c139250 msgr2=0x7ff47c13b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:01.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.518+0000 7ff480d37700 1 --2- 172.21.15.3:0/290581100 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff47c139250 0x7ff47c13b630 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff474009e80 tx=0x7ff47400b330 comp rx=0 tx=0).stop 2023-10-03T10:16:01.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.518+0000 7ff480d37700 1 -- 172.21.15.3:0/290581100 shutdown_connections 2023-10-03T10:16:01.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.518+0000 7ff480d37700 1 --2- 172.21.15.3:0/290581100 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff47c139250 0x7ff47c13b630 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.518+0000 7ff480d37700 1 --2- 172.21.15.3:0/290581100 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff47c0a1510 0x7ff47c138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.518+0000 7ff480d37700 1 -- 172.21.15.3:0/290581100 >> 172.21.15.3:0/290581100 conn(0x7ff47c0a6550 msgr2=0x7ff47c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:01.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.518+0000 7ff480d37700 1 -- 172.21.15.3:0/290581100 shutdown_connections 2023-10-03T10:16:01.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.518+0000 7ff480d37700 1 -- 172.21.15.3:0/290581100 wait complete. 2023-10-03T10:16:01.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.519+0000 7ff480d37700 1 Processor -- start 2023-10-03T10:16:01.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.519+0000 7ff480d37700 1 -- start start 2023-10-03T10:16:01.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.520+0000 7ff480d37700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff47c0a0a30 0x7ff47c1c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:01.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.520+0000 7ff480d37700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff47c0a1510 0x7ff47c1c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:01.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.520+0000 7ff480d37700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff47c139250 0x7ff47c1227d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:01.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.520+0000 7ff480d37700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c1243e0 con 0x7ff47c139250 2023-10-03T10:16:01.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.520+0000 7ff480d37700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c1263d0 con 0x7ff47c0a0a30 2023-10-03T10:16:01.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.520+0000 7ff480d37700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c0a0220 con 0x7ff47c0a1510 2023-10-03T10:16:01.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.520+0000 7ff479d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff47c0a1510 0x7ff47c1c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:01.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.520+0000 7ff47ad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff47c139250 0x7ff47c1227d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:01.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.520+0000 7ff479d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff47c0a1510 0x7ff47c1c8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:58668/0 (socket says 172.21.15.3:58668) 2023-10-03T10:16:01.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.520+0000 7ff47ad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff47c139250 0x7ff47c1227d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32928/0 (socket says 172.21.15.3:32928) 2023-10-03T10:16:01.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.520+0000 7ff479d9b700 1 -- 172.21.15.3:0/2529900080 learned_addr learned my addr 172.21.15.3:0/2529900080 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:01.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.520+0000 7ff47ad9d700 1 -- 172.21.15.3:0/2529900080 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff47c0a1510 msgr2=0x7ff47c1c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:01.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.520+0000 7ff47ad9d700 1 --2- 172.21.15.3:0/2529900080 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff47c0a1510 0x7ff47c1c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.520+0000 7ff47ad9d700 1 -- 172.21.15.3:0/2529900080 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff47c0a0a30 msgr2=0x7ff47c1c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:01.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.520+0000 7ff47ad9d700 1 --2- 172.21.15.3:0/2529900080 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff47c0a0a30 0x7ff47c1c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.520+0000 7ff47ad9d700 1 -- 172.21.15.3:0/2529900080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff474009b30 con 0x7ff47c139250 2023-10-03T10:16:01.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.520+0000 7ff479d9b700 1 --2- 172.21.15.3:0/2529900080 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff47c0a1510 0x7ff47c1c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:01.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.521+0000 7ff47ad9d700 1 --2- 172.21.15.3:0/2529900080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff47c139250 0x7ff47c1227d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff474015000 tx=0x7ff474004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:01.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.521+0000 7ff46affd700 1 -- 172.21.15.3:0/2529900080 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4740048b0 con 0x7ff47c139250 2023-10-03T10:16:01.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.521+0000 7ff46affd700 1 -- 172.21.15.3:0/2529900080 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4740092e0 con 0x7ff47c139250 2023-10-03T10:16:01.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.521+0000 7ff480d37700 1 -- 172.21.15.3:0/2529900080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff47c1386a0 con 0x7ff47c139250 2023-10-03T10:16:01.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.521+0000 7ff46affd700 1 -- 172.21.15.3:0/2529900080 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47401c650 con 0x7ff47c139250 2023-10-03T10:16:01.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.521+0000 7ff480d37700 1 -- 172.21.15.3:0/2529900080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff47c1389e0 con 0x7ff47c139250 2023-10-03T10:16:01.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.522+0000 7ff46affd700 1 -- 172.21.15.3:0/2529900080 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff47401c7b0 con 0x7ff47c139250 2023-10-03T10:16:01.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.522+0000 7ff46affd700 1 -- 172.21.15.3:0/2529900080 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff474041230 con 0x7ff47c139250 2023-10-03T10:16:01.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.523+0000 7ff480d37700 1 -- 172.21.15.3:0/2529900080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff47c074ea0 con 0x7ff47c139250 2023-10-03T10:16:01.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.528+0000 7ff46affd700 1 -- 172.21.15.3:0/2529900080 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff47402c920 con 0x7ff47c139250 2023-10-03T10:16:01.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.607+0000 7f96787cd700 1 Processor -- start 2023-10-03T10:16:01.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.607+0000 7f96787cd700 1 -- start start 2023-10-03T10:16:01.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.608+0000 7f96787cd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f96700a0a30 0x7f96700a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:01.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.608+0000 7f96787cd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f96700a1340 0x7f9670133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:01.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.608+0000 7f96787cd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9670133fa0 0x7f9670134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:01.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.608+0000 7f96787cd700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f967009edd0 con 0x7f96700a1340 2023-10-03T10:16:01.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.608+0000 7f96787cd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f967009e1f0 con 0x7f9670133fa0 2023-10-03T10:16:01.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.608+0000 7f96787cd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f967009dc50 con 0x7f96700a0a30 2023-10-03T10:16:01.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.608+0000 7f9676d6a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9670133fa0 0x7f9670134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:01.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.608+0000 7f9675d68700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f96700a1340 0x7f9670133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:01.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.608+0000 7f9676d6a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9670133fa0 0x7f9670134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39124/0 (socket says 172.21.15.3:39124) 2023-10-03T10:16:01.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.608+0000 7f9676d6a700 1 -- 172.21.15.3:0/2007574533 learned_addr learned my addr 172.21.15.3:0/2007574533 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:01.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.608+0000 7f9676569700 1 --2- 172.21.15.3:0/2007574533 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f96700a0a30 0x7f96700a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:01.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.608+0000 7f9675d68700 1 -- 172.21.15.3:0/2007574533 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f96700a0a30 msgr2=0x7f96700a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:01.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.608+0000 7f9675d68700 1 --2- 172.21.15.3:0/2007574533 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f96700a0a30 0x7f96700a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.608+0000 7f9675d68700 1 -- 172.21.15.3:0/2007574533 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9670133fa0 msgr2=0x7f9670134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:01.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.608+0000 7f9675d68700 1 --2- 172.21.15.3:0/2007574533 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9670133fa0 0x7f9670134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.608+0000 7f9675d68700 1 -- 172.21.15.3:0/2007574533 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96700a1fe0 con 0x7f96700a1340 2023-10-03T10:16:01.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.609+0000 7f9676d6a700 1 --2- 172.21.15.3:0/2007574533 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9670133fa0 0x7f9670134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:01.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.609+0000 7f9675d68700 1 --2- 172.21.15.3:0/2007574533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96700a1340 0x7f9670133a60 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f9660009840 tx=0x7f9660009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddea54b88f1a9c99 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:01.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.609+0000 7f9675567700 1 -- 172.21.15.3:0/2007574533 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9660011830 con 0x7f96700a1340 2023-10-03T10:16:01.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.609+0000 7f9675567700 1 -- 172.21.15.3:0/2007574533 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9660011990 con 0x7f96700a1340 2023-10-03T10:16:01.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.609+0000 7f9675567700 1 -- 172.21.15.3:0/2007574533 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9660011d20 con 0x7f96700a1340 2023-10-03T10:16:01.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.610+0000 7f96787cd700 1 -- 172.21.15.3:0/2007574533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96700a1340 msgr2=0x7f9670133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:01.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.610+0000 7f96787cd700 1 --2- 172.21.15.3:0/2007574533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96700a1340 0x7f9670133a60 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f9660009840 tx=0x7f9660009b50 comp rx=0 tx=0).stop 2023-10-03T10:16:01.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.610+0000 7f96787cd700 1 -- 172.21.15.3:0/2007574533 shutdown_connections 2023-10-03T10:16:01.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.610+0000 7f96787cd700 1 --2- 172.21.15.3:0/2007574533 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9670133fa0 0x7f9670134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.610+0000 7f96787cd700 1 --2- 172.21.15.3:0/2007574533 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f96700a0a30 0x7f96700a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.610+0000 7f96787cd700 1 --2- 172.21.15.3:0/2007574533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96700a1340 0x7f9670133a60 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.610+0000 7f96787cd700 1 -- 172.21.15.3:0/2007574533 >> 172.21.15.3:0/2007574533 conn(0x7f96700a6550 msgr2=0x7f96700a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:01.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.610+0000 7f96787cd700 1 -- 172.21.15.3:0/2007574533 shutdown_connections 2023-10-03T10:16:01.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.610+0000 7f96787cd700 1 -- 172.21.15.3:0/2007574533 wait complete. 2023-10-03T10:16:01.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.611+0000 7f96787cd700 1 Processor -- start 2023-10-03T10:16:01.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.611+0000 7f96787cd700 1 -- start start 2023-10-03T10:16:01.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.612+0000 7f96787cd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f96700a0a30 0x7f9670120270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:01.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.612+0000 7f96787cd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96700a1340 0x7f96701207b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:01.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.612+0000 7f96787cd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9670133fa0 0x7f9670120cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:01.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.612+0000 7f96787cd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9670121740 con 0x7f96700a1340 2023-10-03T10:16:01.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.612+0000 7f96787cd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9670133660 con 0x7f9670133fa0 2023-10-03T10:16:01.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.612+0000 7f96787cd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f96701230d0 con 0x7f96700a0a30 2023-10-03T10:16:01.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.612+0000 7f9676569700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f96700a0a30 0x7f9670120270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:01.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.612+0000 7f9676569700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f96700a0a30 0x7f9670120270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:58680/0 (socket says 172.21.15.3:58680) 2023-10-03T10:16:01.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.612+0000 7f9676569700 1 -- 172.21.15.3:0/557344715 learned_addr learned my addr 172.21.15.3:0/557344715 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:01.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.612+0000 7f9676d6a700 1 --2- 172.21.15.3:0/557344715 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9670133fa0 0x7f9670120cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:01.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.613+0000 7f9676569700 1 -- 172.21.15.3:0/557344715 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9670133fa0 msgr2=0x7f9670120cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:01.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.613+0000 7f9676569700 1 --2- 172.21.15.3:0/557344715 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9670133fa0 0x7f9670120cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.613+0000 7f9676569700 1 -- 172.21.15.3:0/557344715 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96700a1340 msgr2=0x7f96701207b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:01.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.613+0000 7f9676569700 1 --2- 172.21.15.3:0/557344715 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96700a1340 0x7f96701207b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.613+0000 7f9676569700 1 -- 172.21.15.3:0/557344715 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9668009b30 con 0x7f96700a0a30 2023-10-03T10:16:01.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.613+0000 7f9676569700 1 --2- 172.21.15.3:0/557344715 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f96700a0a30 0x7f9670120270 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f966c00e9e0 tx=0x7f966c00ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:01.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.613+0000 7f9666ffd700 1 -- 172.21.15.3:0/557344715 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966c009710 con 0x7f96700a0a30 2023-10-03T10:16:01.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.613+0000 7f96787cd700 1 -- 172.21.15.3:0/557344715 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96600094a0 con 0x7f96700a0a30 2023-10-03T10:16:01.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.613+0000 7f9666ffd700 1 -- 172.21.15.3:0/557344715 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f966c00c850 con 0x7f96700a0a30 2023-10-03T10:16:01.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.613+0000 7f9666ffd700 1 -- 172.21.15.3:0/557344715 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966c00cbc0 con 0x7f96700a0a30 2023-10-03T10:16:01.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.613+0000 7f96787cd700 1 -- 172.21.15.3:0/557344715 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9670123570 con 0x7f96700a0a30 2023-10-03T10:16:01.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.614+0000 7f96787cd700 1 -- 172.21.15.3:0/557344715 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9670074ea0 con 0x7f96700a0a30 2023-10-03T10:16:01.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.615+0000 7f9666ffd700 1 -- 172.21.15.3:0/557344715 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f966c00cd20 con 0x7f96700a0a30 2023-10-03T10:16:01.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.615+0000 7f9666ffd700 1 -- 172.21.15.3:0/557344715 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f966c03fbe0 con 0x7f96700a0a30 2023-10-03T10:16:01.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.624+0000 7f9666ffd700 1 -- 172.21.15.3:0/557344715 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f966c01adc0 con 0x7f96700a0a30 2023-10-03T10:16:01.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.692+0000 7f4897bc0700 1 -- 172.21.15.3:0/3775166 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f489006c360 con 0x7f489009d6f0 2023-10-03T10:16:01.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.692+0000 7f48867fc700 1 -- 172.21.15.3:0/3775166 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f488c017070 con 0x7f489009d6f0 2023-10-03T10:16:01.693 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:01.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.694+0000 7f4897bc0700 1 -- 172.21.15.3:0/3775166 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f489009d6f0 msgr2=0x7f4890122510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:01.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.694+0000 7f4897bc0700 1 --2- 172.21.15.3:0/3775166 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f489009d6f0 0x7f4890122510 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f488c00e9b0 tx=0x7f488c00ecc0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.695+0000 7f4897bc0700 1 -- 172.21.15.3:0/3775166 shutdown_connections 2023-10-03T10:16:01.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.695+0000 7f4897bc0700 1 --2- 172.21.15.3:0/3775166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f489009e410 0x7f4890122f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.695+0000 7f4897bc0700 1 --2- 172.21.15.3:0/3775166 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f489009d6f0 0x7f4890122510 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.695+0000 7f4897bc0700 1 --2- 172.21.15.3:0/3775166 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f489009e000 0x7f4890122a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.695+0000 7f4897bc0700 1 -- 172.21.15.3:0/3775166 >> 172.21.15.3:0/3775166 conn(0x7f4890066f50 msgr2=0x7f4890071080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:01.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.695+0000 7f4897bc0700 1 -- 172.21.15.3:0/3775166 shutdown_connections 2023-10-03T10:16:01.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.695+0000 7f4897bc0700 1 -- 172.21.15.3:0/3775166 wait complete. 2023-10-03T10:16:01.709 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:01.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.735+0000 7ff480d37700 1 -- 172.21.15.3:0/2529900080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff47c0a28e0 con 0x7ff47c139250 2023-10-03T10:16:01.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.736+0000 7ff46affd700 1 -- 172.21.15.3:0/2529900080 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff474033090 con 0x7ff47c139250 2023-10-03T10:16:01.736 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:01.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.738+0000 7ff480d37700 1 -- 172.21.15.3:0/2529900080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff47c139250 msgr2=0x7ff47c1227d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:01.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.738+0000 7ff480d37700 1 --2- 172.21.15.3:0/2529900080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff47c139250 0x7ff47c1227d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff474015000 tx=0x7ff474004ad0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.738+0000 7ff480d37700 1 -- 172.21.15.3:0/2529900080 shutdown_connections 2023-10-03T10:16:01.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.738+0000 7ff480d37700 1 --2- 172.21.15.3:0/2529900080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff47c139250 0x7ff47c1227d0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.738+0000 7ff480d37700 1 --2- 172.21.15.3:0/2529900080 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff47c0a0a30 0x7ff47c1c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.738+0000 7ff480d37700 1 --2- 172.21.15.3:0/2529900080 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff47c0a1510 0x7ff47c1c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.738+0000 7ff480d37700 1 -- 172.21.15.3:0/2529900080 >> 172.21.15.3:0/2529900080 conn(0x7ff47c0a6550 msgr2=0x7ff47c0a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:01.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.738+0000 7ff480d37700 1 -- 172.21.15.3:0/2529900080 shutdown_connections 2023-10-03T10:16:01.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.738+0000 7ff480d37700 1 -- 172.21.15.3:0/2529900080 wait complete. 2023-10-03T10:16:01.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.741+0000 7fbc6f7e4700 1 -- 172.21.15.3:0/903631280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbc68071100 con 0x7fbc6806e570 2023-10-03T10:16:01.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.741+0000 7fbc5dffb700 1 -- 172.21.15.3:0/903631280 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbc6402d440 con 0x7fbc6806e570 2023-10-03T10:16:01.741 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:01.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.744+0000 7fbc6f7e4700 1 -- 172.21.15.3:0/903631280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc6806e570 msgr2=0x7fbc681276a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:01.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.744+0000 7fbc6f7e4700 1 --2- 172.21.15.3:0/903631280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc6806e570 0x7fbc681276a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fbc680715c0 tx=0x7fbc64004750 comp rx=0 tx=0).stop 2023-10-03T10:16:01.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.744+0000 7fbc6f7e4700 1 -- 172.21.15.3:0/903631280 shutdown_connections 2023-10-03T10:16:01.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.744+0000 7fbc6f7e4700 1 --2- 172.21.15.3:0/903631280 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc68139e60 0x7fbc6813a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.744+0000 7fbc6f7e4700 1 --2- 172.21.15.3:0/903631280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc6806e570 0x7fbc681276a0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.744+0000 7fbc6f7e4700 1 --2- 172.21.15.3:0/903631280 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc68134610 0x7fbc68139920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.744+0000 7fbc6f7e4700 1 -- 172.21.15.3:0/903631280 >> 172.21.15.3:0/903631280 conn(0x7fbc680a2aa0 msgr2=0x7fbc68099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:01.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.745+0000 7fbc6f7e4700 1 -- 172.21.15.3:0/903631280 shutdown_connections 2023-10-03T10:16:01.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.745+0000 7fbc6f7e4700 1 -- 172.21.15.3:0/903631280 wait complete. 2023-10-03T10:16:01.754 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:01.760 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:01.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.831+0000 7f96787cd700 1 -- 172.21.15.3:0/557344715 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f96700a6c20 con 0x7f96700a0a30 2023-10-03T10:16:01.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.832+0000 7f9666ffd700 1 -- 172.21.15.3:0/557344715 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f966c021070 con 0x7f96700a0a30 2023-10-03T10:16:01.832 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:01.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.834+0000 7f96787cd700 1 -- 172.21.15.3:0/557344715 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f96700a0a30 msgr2=0x7f9670120270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:01.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.834+0000 7f96787cd700 1 --2- 172.21.15.3:0/557344715 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f96700a0a30 0x7f9670120270 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f966c00e9e0 tx=0x7f966c00ecf0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.834+0000 7f96787cd700 1 -- 172.21.15.3:0/557344715 shutdown_connections 2023-10-03T10:16:01.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.834+0000 7f96787cd700 1 --2- 172.21.15.3:0/557344715 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9670133fa0 0x7f9670120cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.834+0000 7f96787cd700 1 --2- 172.21.15.3:0/557344715 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f96700a0a30 0x7f9670120270 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.834+0000 7f96787cd700 1 --2- 172.21.15.3:0/557344715 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96700a1340 0x7f96701207b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:01.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.834+0000 7f96787cd700 1 -- 172.21.15.3:0/557344715 >> 172.21.15.3:0/557344715 conn(0x7f96700a6550 msgr2=0x7f96701baf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:01.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.834+0000 7f96787cd700 1 -- 172.21.15.3:0/557344715 shutdown_connections 2023-10-03T10:16:01.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:01.834+0000 7f96787cd700 1 -- 172.21.15.3:0/557344715 wait complete. 2023-10-03T10:16:01.848 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:02.710 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:02.755 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:02.761 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:02.849 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:02.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.868+0000 7f991c37a700 1 Processor -- start 2023-10-03T10:16:02.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.868+0000 7f991c37a700 1 -- start start 2023-10-03T10:16:02.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.869+0000 7f991c37a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9914136710 0x7f9914138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:02.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.869+0000 7f991c37a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9914139030 0x7f991413b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:02.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.869+0000 7f991c37a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991406ef30 con 0x7f99141338a0 2023-10-03T10:16:02.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.869+0000 7f991c37a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991406fef0 con 0x7f9914139030 2023-10-03T10:16:02.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.869+0000 7f991c37a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99141334b0 con 0x7f9914136710 2023-10-03T10:16:02.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.870+0000 7f991a917700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9914139030 0x7f991413b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:02.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.870+0000 7f991a917700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9914139030 0x7f991413b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32952/0 (socket says 172.21.15.3:32952) 2023-10-03T10:16:02.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.870+0000 7f991a917700 1 -- 172.21.15.3:0/2815465305 learned_addr learned my addr 172.21.15.3:0/2815465305 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:02.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.870+0000 7f991a917700 1 -- 172.21.15.3:0/2815465305 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9914136710 msgr2=0x7f9914138af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:02.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.870+0000 7f991a917700 1 --2- 172.21.15.3:0/2815465305 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9914136710 0x7f9914138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:02.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.870+0000 7f991a917700 1 -- 172.21.15.3:0/2815465305 >> v1:172.21.15.3:6789/0 conn(0x7f99141338a0 legacy=0x7f9914133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:02.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.870+0000 7f991a917700 1 -- 172.21.15.3:0/2815465305 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991406fc30 con 0x7f9914139030 2023-10-03T10:16:02.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.870+0000 7f991a917700 1 --2- 172.21.15.3:0/2815465305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9914139030 0x7f991413b410 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f990c009e50 tx=0x7f990c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6683ed6fa4a46221 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:02.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.870+0000 7f9919114700 1 -- 172.21.15.3:0/2815465305 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c00e070 con 0x7f9914139030 2023-10-03T10:16:02.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.870+0000 7f9919114700 1 -- 172.21.15.3:0/2815465305 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990c003a20 con 0x7f9914139030 2023-10-03T10:16:02.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.870+0000 7f9919114700 1 -- 172.21.15.3:0/2815465305 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c003dd0 con 0x7f9914139030 2023-10-03T10:16:02.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.871+0000 7f991c37a700 1 -- 172.21.15.3:0/2815465305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9914139030 msgr2=0x7f991413b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:02.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.871+0000 7f991c37a700 1 --2- 172.21.15.3:0/2815465305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9914139030 0x7f991413b410 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f990c009e50 tx=0x7f990c00b330 comp rx=0 tx=0).stop 2023-10-03T10:16:02.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.871+0000 7f991c37a700 1 -- 172.21.15.3:0/2815465305 shutdown_connections 2023-10-03T10:16:02.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.871+0000 7f991c37a700 1 --2- 172.21.15.3:0/2815465305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9914139030 0x7f991413b410 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:02.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.871+0000 7f991c37a700 1 --2- 172.21.15.3:0/2815465305 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9914136710 0x7f9914138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:02.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.871+0000 7f991c37a700 1 -- 172.21.15.3:0/2815465305 >> 172.21.15.3:0/2815465305 conn(0x7f99140a2aa0 msgr2=0x7f99140a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:02.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.871+0000 7f991c37a700 1 -- 172.21.15.3:0/2815465305 shutdown_connections 2023-10-03T10:16:02.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.871+0000 7f991c37a700 1 -- 172.21.15.3:0/2815465305 wait complete. 2023-10-03T10:16:02.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.872+0000 7f991c37a700 1 Processor -- start 2023-10-03T10:16:02.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.872+0000 7f991c37a700 1 -- start start 2023-10-03T10:16:02.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.873+0000 7f991c37a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f99141338a0 0x7f9914118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:02.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.873+0000 7f991c37a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9914136710 0x7f9914119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:02.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.873+0000 7f991c37a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9914139030 0x7f9914119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:02.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.873+0000 7f991c37a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991411b3d0 con 0x7f9914136710 2023-10-03T10:16:02.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.873+0000 7f991c37a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991411c0f0 con 0x7f99141338a0 2023-10-03T10:16:02.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.873+0000 7f991c37a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f991413b6b0 con 0x7f9914139030 2023-10-03T10:16:02.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.873+0000 7f991a116700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f99141338a0 0x7f9914118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:02.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.873+0000 7f991a116700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f99141338a0 0x7f9914118cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39136/0 (socket says 172.21.15.3:39136) 2023-10-03T10:16:02.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.873+0000 7f991a116700 1 -- 172.21.15.3:0/1252309107 learned_addr learned my addr 172.21.15.3:0/1252309107 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:02.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.873+0000 7f991a917700 1 --2- 172.21.15.3:0/1252309107 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9914139030 0x7f9914119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:02.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.873+0000 7f9919915700 1 --2- 172.21.15.3:0/1252309107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9914136710 0x7f9914119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:02.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.874+0000 7f991a116700 1 -- 172.21.15.3:0/1252309107 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9914139030 msgr2=0x7f9914119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:02.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.874+0000 7f991a116700 1 --2- 172.21.15.3:0/1252309107 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9914139030 0x7f9914119750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:02.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.874+0000 7f991a116700 1 -- 172.21.15.3:0/1252309107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9914136710 msgr2=0x7f9914119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:02.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.874+0000 7f991a116700 1 --2- 172.21.15.3:0/1252309107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9914136710 0x7f9914119210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:02.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.874+0000 7f991a116700 1 -- 172.21.15.3:0/1252309107 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990c009b30 con 0x7f99141338a0 2023-10-03T10:16:02.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.874+0000 7f991a116700 1 --2- 172.21.15.3:0/1252309107 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f99141338a0 0x7f9914118cd0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f991000c4d0 tx=0x7f991000c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:02.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.874+0000 7f990affd700 1 -- 172.21.15.3:0/1252309107 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9910011790 con 0x7f99141338a0 2023-10-03T10:16:02.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.874+0000 7f990affd700 1 -- 172.21.15.3:0/1252309107 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99100118f0 con 0x7f99141338a0 2023-10-03T10:16:02.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.874+0000 7f991c37a700 1 -- 172.21.15.3:0/1252309107 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991406f640 con 0x7f99141338a0 2023-10-03T10:16:02.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.874+0000 7f990affd700 1 -- 172.21.15.3:0/1252309107 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9910011ca0 con 0x7f99141338a0 2023-10-03T10:16:02.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.874+0000 7f991c37a700 1 -- 172.21.15.3:0/1252309107 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991411c3d0 con 0x7f99141338a0 2023-10-03T10:16:02.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.875+0000 7f991c37a700 1 -- 172.21.15.3:0/1252309107 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98fc0052e0 con 0x7f99141338a0 2023-10-03T10:16:02.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.876+0000 7f990affd700 1 -- 172.21.15.3:0/1252309107 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9910026430 con 0x7f99141338a0 2023-10-03T10:16:02.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.876+0000 7f990affd700 1 -- 172.21.15.3:0/1252309107 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f991003ff40 con 0x7f99141338a0 2023-10-03T10:16:02.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.885+0000 7f990affd700 1 -- 172.21.15.3:0/1252309107 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9910026e40 con 0x7f99141338a0 2023-10-03T10:16:02.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.910+0000 7fe33f5cc700 1 Processor -- start 2023-10-03T10:16:02.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.910+0000 7fe33f5cc700 1 -- start start 2023-10-03T10:16:02.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.911+0000 7fe33f5cc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe33809d6f0 0x7fe33809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:02.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.911+0000 7fe33f5cc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe33809e000 0x7fe3381339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:02.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.911+0000 7fe33f5cc700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe33809b120 con 0x7fe33809e410 2023-10-03T10:16:02.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.911+0000 7fe33f5cc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe33806b460 con 0x7fe33809d6f0 2023-10-03T10:16:02.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.911+0000 7fe33f5cc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe33809a7e0 con 0x7fe33809e000 2023-10-03T10:16:02.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.911+0000 7fe33d368700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe33809d6f0 0x7fe33809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:02.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.911+0000 7fe33d368700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe33809d6f0 0x7fe33809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32964/0 (socket says 172.21.15.3:32964) 2023-10-03T10:16:02.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.911+0000 7fe33d368700 1 -- 172.21.15.3:0/1567181836 learned_addr learned my addr 172.21.15.3:0/1567181836 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:02.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.911+0000 7fe33cb67700 1 --2- 172.21.15.3:0/1567181836 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe33809e000 0x7fe3381339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:02.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.912+0000 7fe33cb67700 1 -- 172.21.15.3:0/1567181836 >> v2:172.21.15.3:3300/0 conn(0x7fe33809d6f0 msgr2=0x7fe33809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:02.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.912+0000 7fe33cb67700 1 --2- 172.21.15.3:0/1567181836 >> v2:172.21.15.3:3300/0 conn(0x7fe33809d6f0 0x7fe33809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:02.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.912+0000 7fe33cb67700 1 -- 172.21.15.3:0/1567181836 >> v1:172.21.15.3:6789/0 conn(0x7fe33809e410 legacy=0x7fe338133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:02.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.912+0000 7fe33cb67700 1 -- 172.21.15.3:0/1567181836 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33809a500 con 0x7fe33809e000 2023-10-03T10:16:02.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.912+0000 7fe33cb67700 1 --2- 172.21.15.3:0/1567181836 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe33809e000 0x7fe3381339f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe328009870 tx=0x7fe328009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cb7d0adbcec6ff1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:02.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.912+0000 7fe32ffff700 1 -- 172.21.15.3:0/1567181836 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe328011830 con 0x7fe33809e000 2023-10-03T10:16:02.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.912+0000 7fe32ffff700 1 -- 172.21.15.3:0/1567181836 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe328011990 con 0x7fe33809e000 2023-10-03T10:16:02.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.912+0000 7fe32ffff700 1 -- 172.21.15.3:0/1567181836 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe328011d20 con 0x7fe33809e000 2023-10-03T10:16:02.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.913+0000 7fe33f5cc700 1 -- 172.21.15.3:0/1567181836 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe33809e000 msgr2=0x7fe3381339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:02.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.913+0000 7fe33f5cc700 1 --2- 172.21.15.3:0/1567181836 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe33809e000 0x7fe3381339f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe328009870 tx=0x7fe328009b80 comp rx=0 tx=0).stop 2023-10-03T10:16:02.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.913+0000 7fe33f5cc700 1 -- 172.21.15.3:0/1567181836 shutdown_connections 2023-10-03T10:16:02.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.913+0000 7fe33f5cc700 1 --2- 172.21.15.3:0/1567181836 >> v2:172.21.15.3:3300/0 conn(0x7fe33809d6f0 0x7fe33809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:02.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.913+0000 7fe33f5cc700 1 --2- 172.21.15.3:0/1567181836 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe33809e000 0x7fe3381339f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:02.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.913+0000 7fe33f5cc700 1 -- 172.21.15.3:0/1567181836 >> 172.21.15.3:0/1567181836 conn(0x7fe338066f50 msgr2=0x7fe33809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:02.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.913+0000 7fe33f5cc700 1 -- 172.21.15.3:0/1567181836 shutdown_connections 2023-10-03T10:16:02.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.913+0000 7fe33f5cc700 1 -- 172.21.15.3:0/1567181836 wait complete. 2023-10-03T10:16:02.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.914+0000 7fe33f5cc700 1 Processor -- start 2023-10-03T10:16:02.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.914+0000 7fe33f5cc700 1 -- start start 2023-10-03T10:16:02.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.915+0000 7fe33f5cc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe33809d6f0 0x7fe3381a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:02.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.915+0000 7fe33f5cc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe33809e000 0x7fe3381a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:02.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.915+0000 7fe33f5cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe33809e410 0x7fe3381a6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:02.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.915+0000 7fe33f5cc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3381a8c00 con 0x7fe33809d6f0 2023-10-03T10:16:02.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.915+0000 7fe33f5cc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3381aa210 con 0x7fe33809e410 2023-10-03T10:16:02.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.915+0000 7fe33f5cc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe3381a9850 con 0x7fe33809e000 2023-10-03T10:16:02.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.915+0000 7fe33cb67700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe33809e000 0x7fe3381a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:02.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.915+0000 7fe33cb67700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe33809e000 0x7fe3381a6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:58724/0 (socket says 172.21.15.3:58724) 2023-10-03T10:16:02.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.915+0000 7fe33cb67700 1 -- 172.21.15.3:0/1851491169 learned_addr learned my addr 172.21.15.3:0/1851491169 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:02.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.915+0000 7fe33db69700 1 --2- 172.21.15.3:0/1851491169 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe33809e410 0x7fe3381a6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:02.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.915+0000 7fe33d368700 1 --2- 172.21.15.3:0/1851491169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe33809d6f0 0x7fe3381a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:02.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.915+0000 7fe33cb67700 1 -- 172.21.15.3:0/1851491169 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe33809e410 msgr2=0x7fe3381a6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:02.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.915+0000 7fe33cb67700 1 --2- 172.21.15.3:0/1851491169 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe33809e410 0x7fe3381a6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:02.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.915+0000 7fe33cb67700 1 -- 172.21.15.3:0/1851491169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe33809d6f0 msgr2=0x7fe3381a5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:02.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.915+0000 7fe33cb67700 1 --2- 172.21.15.3:0/1851491169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe33809d6f0 0x7fe3381a5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:02.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.915+0000 7fe33cb67700 1 -- 172.21.15.3:0/1851491169 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3280094a0 con 0x7fe33809e000 2023-10-03T10:16:02.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.916+0000 7fe33cb67700 1 --2- 172.21.15.3:0/1851491169 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe33809e000 0x7fe3381a6310 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe3280097c0 tx=0x7fe328004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:02.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.916+0000 7fe32dffb700 1 -- 172.21.15.3:0/1851491169 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3280036a0 con 0x7fe33809e000 2023-10-03T10:16:02.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.916+0000 7fe32dffb700 1 -- 172.21.15.3:0/1851491169 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe32801f070 con 0x7fe33809e000 2023-10-03T10:16:02.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.916+0000 7fe33f5cc700 1 -- 172.21.15.3:0/1851491169 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3381a79e0 con 0x7fe33809e000 2023-10-03T10:16:02.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.916+0000 7fe32dffb700 1 -- 172.21.15.3:0/1851491169 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe328003a20 con 0x7fe33809e000 2023-10-03T10:16:02.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.916+0000 7fe33f5cc700 1 -- 172.21.15.3:0/1851491169 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3380a1fe0 con 0x7fe33809e000 2023-10-03T10:16:02.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.917+0000 7fe32dffb700 1 -- 172.21.15.3:0/1851491169 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe328003b80 con 0x7fe33809e000 2023-10-03T10:16:02.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.918+0000 7fe32dffb700 1 -- 172.21.15.3:0/1851491169 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe328021070 con 0x7fe33809e000 2023-10-03T10:16:02.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.918+0000 7fe33f5cc700 1 -- 172.21.15.3:0/1851491169 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3381a8800 con 0x7fe33809e000 2023-10-03T10:16:02.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.928+0000 7fe32dffb700 1 -- 172.21.15.3:0/1851491169 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe328015030 con 0x7fe33809e000 2023-10-03T10:16:02.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.951+0000 7fae43a88700 1 Processor -- start 2023-10-03T10:16:02.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.952+0000 7fae43a88700 1 -- start start 2023-10-03T10:16:02.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.952+0000 7fae43a88700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae3c09de00 0x7fae3c09e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:02.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.952+0000 7fae43a88700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fae3c13d1f0 0x7fae3c13f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:02.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.952+0000 7fae43a88700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae3c09a810 con 0x7fae3c06a770 2023-10-03T10:16:02.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.952+0000 7fae43a88700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae3c06ab40 con 0x7fae3c13d1f0 2023-10-03T10:16:02.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.952+0000 7fae43a88700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c09ebc0 con 0x7fae3c09de00 2023-10-03T10:16:02.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.952+0000 7fae41023700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae3c09de00 0x7fae3c09e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:02.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.952+0000 7fae41023700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae3c09de00 0x7fae3c09e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:58740/0 (socket says 172.21.15.3:58740) 2023-10-03T10:16:02.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.952+0000 7fae41023700 1 -- 172.21.15.3:0/3680915876 learned_addr learned my addr 172.21.15.3:0/3680915876 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:02.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.952+0000 7fae42025700 1 --2- 172.21.15.3:0/3680915876 >> v2:172.21.15.3:3300/0 conn(0x7fae3c13d1f0 0x7fae3c13f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:02.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.953+0000 7fae41023700 1 -- 172.21.15.3:0/3680915876 >> v2:172.21.15.3:3300/0 conn(0x7fae3c13d1f0 msgr2=0x7fae3c13f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:02.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.953+0000 7fae41023700 1 --2- 172.21.15.3:0/3680915876 >> v2:172.21.15.3:3300/0 conn(0x7fae3c13d1f0 0x7fae3c13f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:02.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.953+0000 7fae41023700 1 -- 172.21.15.3:0/3680915876 >> v1:172.21.15.3:6789/0 conn(0x7fae3c06a770 legacy=0x7fae3c09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:02.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.953+0000 7fae41023700 1 -- 172.21.15.3:0/3680915876 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c09e920 con 0x7fae3c09de00 2023-10-03T10:16:02.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.953+0000 7fae41023700 1 --2- 172.21.15.3:0/3680915876 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae3c09de00 0x7fae3c09e250 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0098f0 tx=0x7fae2c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adc1dc406dc16b91 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:02.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.953+0000 7fae40822700 1 -- 172.21.15.3:0/3680915876 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c011830 con 0x7fae3c09de00 2023-10-03T10:16:02.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.953+0000 7fae40822700 1 -- 172.21.15.3:0/3680915876 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2c011990 con 0x7fae3c09de00 2023-10-03T10:16:02.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.953+0000 7fae40822700 1 -- 172.21.15.3:0/3680915876 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c011d20 con 0x7fae3c09de00 2023-10-03T10:16:02.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.954+0000 7fae43a88700 1 -- 172.21.15.3:0/3680915876 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae3c09de00 msgr2=0x7fae3c09e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:02.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.954+0000 7fae43a88700 1 --2- 172.21.15.3:0/3680915876 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae3c09de00 0x7fae3c09e250 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0098f0 tx=0x7fae2c009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:02.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.954+0000 7fae43a88700 1 -- 172.21.15.3:0/3680915876 shutdown_connections 2023-10-03T10:16:02.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.954+0000 7fae43a88700 1 --2- 172.21.15.3:0/3680915876 >> v2:172.21.15.3:3300/0 conn(0x7fae3c13d1f0 0x7fae3c13f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:02.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.954+0000 7fae43a88700 1 --2- 172.21.15.3:0/3680915876 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae3c09de00 0x7fae3c09e250 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:02.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.954+0000 7fae43a88700 1 -- 172.21.15.3:0/3680915876 >> 172.21.15.3:0/3680915876 conn(0x7fae3c066f50 msgr2=0x7fae3c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:02.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.954+0000 7fae43a88700 1 -- 172.21.15.3:0/3680915876 shutdown_connections 2023-10-03T10:16:02.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.954+0000 7fae43a88700 1 -- 172.21.15.3:0/3680915876 wait complete. 2023-10-03T10:16:02.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.955+0000 7fae43a88700 1 Processor -- start 2023-10-03T10:16:02.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.955+0000 7fae43a88700 1 -- start start 2023-10-03T10:16:02.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.956+0000 7fae43a88700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fae3c06a770 0x7fae3c11c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:02.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.956+0000 7fae43a88700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae3c09de00 0x7fae3c13b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:02.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.956+0000 7fae43a88700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae3c13d1f0 0x7fae3c13bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:02.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.956+0000 7fae43a88700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c09f210 con 0x7fae3c13d1f0 2023-10-03T10:16:02.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.956+0000 7fae43a88700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c11d1f0 con 0x7fae3c06a770 2023-10-03T10:16:02.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.956+0000 7fae43a88700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c11e410 con 0x7fae3c09de00 2023-10-03T10:16:02.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.956+0000 7fae41824700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fae3c06a770 0x7fae3c11c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:02.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.956+0000 7fae41824700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fae3c06a770 0x7fae3c11c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39156/0 (socket says 172.21.15.3:39156) 2023-10-03T10:16:02.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.956+0000 7fae41824700 1 -- 172.21.15.3:0/2557603655 learned_addr learned my addr 172.21.15.3:0/2557603655 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:02.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.956+0000 7fae41023700 1 --2- 172.21.15.3:0/2557603655 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae3c09de00 0x7fae3c13b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:02.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.956+0000 7fae42025700 1 --2- 172.21.15.3:0/2557603655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae3c13d1f0 0x7fae3c13bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:02.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.957+0000 7fae41824700 1 -- 172.21.15.3:0/2557603655 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae3c09de00 msgr2=0x7fae3c13b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:02.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.957+0000 7fae41824700 1 --2- 172.21.15.3:0/2557603655 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae3c09de00 0x7fae3c13b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:02.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.957+0000 7fae41824700 1 -- 172.21.15.3:0/2557603655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae3c13d1f0 msgr2=0x7fae3c13bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:02.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.957+0000 7fae41824700 1 --2- 172.21.15.3:0/2557603655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae3c13d1f0 0x7fae3c13bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:02.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.957+0000 7fae41824700 1 -- 172.21.15.3:0/2557603655 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2c009550 con 0x7fae3c06a770 2023-10-03T10:16:02.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.957+0000 7fae41824700 1 --2- 172.21.15.3:0/2557603655 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fae3c06a770 0x7fae3c11c5e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fae3800c4e0 tx=0x7fae3800c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:02.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.957+0000 7fae327fc700 1 -- 172.21.15.3:0/2557603655 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae380052d0 con 0x7fae3c06a770 2023-10-03T10:16:02.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.957+0000 7fae327fc700 1 -- 172.21.15.3:0/2557603655 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae38005430 con 0x7fae3c06a770 2023-10-03T10:16:02.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.957+0000 7fae327fc700 1 -- 172.21.15.3:0/2557603655 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae380038c0 con 0x7fae3c06a770 2023-10-03T10:16:02.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.957+0000 7fae43a88700 1 -- 172.21.15.3:0/2557603655 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3c11eac0 con 0x7fae3c06a770 2023-10-03T10:16:02.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.957+0000 7fae43a88700 1 -- 172.21.15.3:0/2557603655 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3c11df50 con 0x7fae3c06a770 2023-10-03T10:16:02.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.958+0000 7fae327fc700 1 -- 172.21.15.3:0/2557603655 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fae38003a20 con 0x7fae3c06a770 2023-10-03T10:16:02.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.958+0000 7fae43a88700 1 -- 172.21.15.3:0/2557603655 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae3c11efc0 con 0x7fae3c06a770 2023-10-03T10:16:02.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.959+0000 7fae327fc700 1 -- 172.21.15.3:0/2557603655 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fae3803eab0 con 0x7fae3c06a770 2023-10-03T10:16:02.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:02.964+0000 7fae327fc700 1 -- 172.21.15.3:0/2557603655 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fae38018260 con 0x7fae3c06a770 2023-10-03T10:16:03.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.001+0000 7f1b4f53b700 1 Processor -- start 2023-10-03T10:16:03.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.002+0000 7f1b4f53b700 1 -- start start 2023-10-03T10:16:03.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.002+0000 7f1b4f53b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1b4809e930 0x7f1b4809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:03.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.002+0000 7f1b4f53b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b4809f650 0x7f1b48131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:03.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.002+0000 7f1b4f53b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b48070940 con 0x7f1b4809f240 2023-10-03T10:16:03.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.002+0000 7f1b4f53b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b4806fd60 con 0x7f1b4809e930 2023-10-03T10:16:03.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.002+0000 7f1b4f53b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4806fa80 con 0x7f1b4809f650 2023-10-03T10:16:03.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.003+0000 7f1b4d2d7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1b4809e930 0x7f1b4809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:03.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.003+0000 7f1b4d2d7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1b4809e930 0x7f1b4809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32994/0 (socket says 172.21.15.3:32994) 2023-10-03T10:16:03.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.003+0000 7f1b4d2d7700 1 -- 172.21.15.3:0/1145714187 learned_addr learned my addr 172.21.15.3:0/1145714187 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:03.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.003+0000 7f1b4dad8700 1 --2- 172.21.15.3:0/1145714187 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b4809f650 0x7f1b48131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:03.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.003+0000 7f1b4dad8700 1 -- 172.21.15.3:0/1145714187 >> v2:172.21.15.3:3300/0 conn(0x7f1b4809e930 msgr2=0x7f1b4809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:03.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.003+0000 7f1b4dad8700 1 --2- 172.21.15.3:0/1145714187 >> v2:172.21.15.3:3300/0 conn(0x7f1b4809e930 0x7f1b4809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:03.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.003+0000 7f1b4dad8700 1 -- 172.21.15.3:0/1145714187 >> v1:172.21.15.3:6789/0 conn(0x7f1b4809f240 legacy=0x7f1b481317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:03.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.003+0000 7f1b3ffff700 1 -- 172.21.15.3:0/1145714187 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4043495634 0 0) 0x7f1b48070940 con 0x7f1b4809f240 2023-10-03T10:16:03.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.003+0000 7f1b4dad8700 1 -- 172.21.15.3:0/1145714187 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4809fb60 con 0x7f1b4809f650 2023-10-03T10:16:03.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.003+0000 7f1b4d2d7700 1 --2- 172.21.15.3:0/1145714187 >> v2:172.21.15.3:3300/0 conn(0x7f1b4809e930 0x7f1b4809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:03.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.003+0000 7f1b4dad8700 1 --2- 172.21.15.3:0/1145714187 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b4809f650 0x7f1b48131f00 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1b40009d90 tx=0x7f1b4000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2472c055c19801e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:03.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.004+0000 7f1b3ffff700 1 -- 172.21.15.3:0/1145714187 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4000e070 con 0x7f1b4809f650 2023-10-03T10:16:03.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.004+0000 7f1b3ffff700 1 -- 172.21.15.3:0/1145714187 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4000bb20 con 0x7f1b4809f650 2023-10-03T10:16:03.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.004+0000 7f1b3ffff700 1 -- 172.21.15.3:0/1145714187 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b40012430 con 0x7f1b4809f650 2023-10-03T10:16:03.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.005+0000 7f1b4f53b700 1 -- 172.21.15.3:0/1145714187 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b4809f650 msgr2=0x7f1b48131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:03.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.005+0000 7f1b4f53b700 1 --2- 172.21.15.3:0/1145714187 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b4809f650 0x7f1b48131f00 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1b40009d90 tx=0x7f1b4000b330 comp rx=0 tx=0).stop 2023-10-03T10:16:03.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.005+0000 7f1b4f53b700 1 -- 172.21.15.3:0/1145714187 shutdown_connections 2023-10-03T10:16:03.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.005+0000 7f1b4f53b700 1 --2- 172.21.15.3:0/1145714187 >> v2:172.21.15.3:3300/0 conn(0x7f1b4809e930 0x7f1b4809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:03.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.005+0000 7f1b4f53b700 1 --2- 172.21.15.3:0/1145714187 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b4809f650 0x7f1b48131f00 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:03.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.005+0000 7f1b4f53b700 1 -- 172.21.15.3:0/1145714187 >> 172.21.15.3:0/1145714187 conn(0x7f1b480a4210 msgr2=0x7f1b480a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:03.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.005+0000 7f1b4f53b700 1 -- 172.21.15.3:0/1145714187 shutdown_connections 2023-10-03T10:16:03.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.005+0000 7f1b4f53b700 1 -- 172.21.15.3:0/1145714187 wait complete. 2023-10-03T10:16:03.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.006+0000 7f1b4f53b700 1 Processor -- start 2023-10-03T10:16:03.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.006+0000 7f1b4f53b700 1 -- start start 2023-10-03T10:16:03.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.007+0000 7f1b4f53b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b4809e930 0x7f1b4806b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:03.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.007+0000 7f1b4f53b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b4809f240 0x7f1b4806bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:03.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.007+0000 7f1b4f53b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b4809f650 0x7f1b4806c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:03.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.007+0000 7f1b4f53b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4806cd80 con 0x7f1b4809f650 2023-10-03T10:16:03.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.007+0000 7f1b4f53b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b481b7200 con 0x7f1b4809e930 2023-10-03T10:16:03.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.007+0000 7f1b4f53b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1b481a93b0 con 0x7f1b4809f240 2023-10-03T10:16:03.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.007+0000 7f1b4d2d7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b4809e930 0x7f1b4806b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:03.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.007+0000 7f1b4cad6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b4809f240 0x7f1b4806bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:03.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.007+0000 7f1b4d2d7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b4809e930 0x7f1b4806b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39172/0 (socket says 172.21.15.3:39172) 2023-10-03T10:16:03.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.007+0000 7f1b4d2d7700 1 -- 172.21.15.3:0/1305848848 learned_addr learned my addr 172.21.15.3:0/1305848848 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:03.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.007+0000 7f1b4dad8700 1 --2- 172.21.15.3:0/1305848848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b4809f650 0x7f1b4806c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:03.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.008+0000 7f1b4cad6700 1 -- 172.21.15.3:0/1305848848 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b4809e930 msgr2=0x7f1b4806b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:03.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.008+0000 7f1b4cad6700 1 --2- 172.21.15.3:0/1305848848 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b4809e930 0x7f1b4806b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:03.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.008+0000 7f1b4cad6700 1 -- 172.21.15.3:0/1305848848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b4809f650 msgr2=0x7f1b4806c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:03.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.008+0000 7f1b4cad6700 1 --2- 172.21.15.3:0/1305848848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b4809f650 0x7f1b4806c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:03.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.008+0000 7f1b4cad6700 1 -- 172.21.15.3:0/1305848848 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b44009550 con 0x7f1b4809f240 2023-10-03T10:16:03.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.008+0000 7f1b4cad6700 1 --2- 172.21.15.3:0/1305848848 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b4809f240 0x7f1b4806bdf0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800c430 tx=0x7f1b3800c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:03.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.008+0000 7f1b3dffb700 1 -- 172.21.15.3:0/1305848848 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3800ce40 con 0x7f1b4809f240 2023-10-03T10:16:03.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.008+0000 7f1b3dffb700 1 -- 172.21.15.3:0/1305848848 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b380052e0 con 0x7f1b4809f240 2023-10-03T10:16:03.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.008+0000 7f1b4f53b700 1 -- 172.21.15.3:0/1305848848 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b40009a70 con 0x7f1b4809f240 2023-10-03T10:16:03.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.008+0000 7f1b3dffb700 1 -- 172.21.15.3:0/1305848848 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b38003680 con 0x7f1b4809f240 2023-10-03T10:16:03.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.009+0000 7f1b4f53b700 1 -- 172.21.15.3:0/1305848848 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b480a0190 con 0x7f1b4809f240 2023-10-03T10:16:03.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.010+0000 7f1b4f53b700 1 -- 172.21.15.3:0/1305848848 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b481a8c90 con 0x7f1b4809f240 2023-10-03T10:16:03.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.010+0000 7f1b3dffb700 1 -- 172.21.15.3:0/1305848848 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1b380037e0 con 0x7f1b4809f240 2023-10-03T10:16:03.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.010+0000 7f1b3dffb700 1 -- 172.21.15.3:0/1305848848 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1b38051890 con 0x7f1b4809f240 2023-10-03T10:16:03.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.020+0000 7f1b3dffb700 1 -- 172.21.15.3:0/1305848848 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1b38055060 con 0x7f1b4809f240 2023-10-03T10:16:03.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.103+0000 7f991c37a700 1 -- 172.21.15.3:0/1252309107 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f98fc005860 con 0x7f99141338a0 2023-10-03T10:16:03.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.104+0000 7f990affd700 1 -- 172.21.15.3:0/1252309107 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f991002a090 con 0x7f99141338a0 2023-10-03T10:16:03.104 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:03.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.106+0000 7f991c37a700 1 -- 172.21.15.3:0/1252309107 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f99141338a0 msgr2=0x7f9914118cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:03.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.106+0000 7f991c37a700 1 --2- 172.21.15.3:0/1252309107 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f99141338a0 0x7f9914118cd0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f991000c4d0 tx=0x7f991000c7e0 comp rx=0 tx=0).stop 2023-10-03T10:16:03.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.107+0000 7f991c37a700 1 -- 172.21.15.3:0/1252309107 shutdown_connections 2023-10-03T10:16:03.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.107+0000 7f991c37a700 1 --2- 172.21.15.3:0/1252309107 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9914139030 0x7f9914119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:03.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.107+0000 7f991c37a700 1 --2- 172.21.15.3:0/1252309107 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f99141338a0 0x7f9914118cd0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:03.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.107+0000 7f991c37a700 1 --2- 172.21.15.3:0/1252309107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9914136710 0x7f9914119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:03.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.107+0000 7f991c37a700 1 -- 172.21.15.3:0/1252309107 >> 172.21.15.3:0/1252309107 conn(0x7f99140a2aa0 msgr2=0x7f99140a1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:03.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.107+0000 7f991c37a700 1 -- 172.21.15.3:0/1252309107 shutdown_connections 2023-10-03T10:16:03.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.107+0000 7f991c37a700 1 -- 172.21.15.3:0/1252309107 wait complete. 2023-10-03T10:16:03.122 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:03.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.142+0000 7fe33f5cc700 1 -- 172.21.15.3:0/1851491169 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe338074ea0 con 0x7fe33809e000 2023-10-03T10:16:03.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.142+0000 7fe32dffb700 1 -- 172.21.15.3:0/1851491169 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe328026480 con 0x7fe33809e000 2023-10-03T10:16:03.142 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:03.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.144+0000 7fe33f5cc700 1 -- 172.21.15.3:0/1851491169 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe33809e000 msgr2=0x7fe3381a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:03.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.144+0000 7fe33f5cc700 1 --2- 172.21.15.3:0/1851491169 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe33809e000 0x7fe3381a6310 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe3280097c0 tx=0x7fe328004750 comp rx=0 tx=0).stop 2023-10-03T10:16:03.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.144+0000 7fe33f5cc700 1 -- 172.21.15.3:0/1851491169 shutdown_connections 2023-10-03T10:16:03.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.144+0000 7fe33f5cc700 1 --2- 172.21.15.3:0/1851491169 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe33809e410 0x7fe3381a6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:03.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.144+0000 7fe33f5cc700 1 --2- 172.21.15.3:0/1851491169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe33809d6f0 0x7fe3381a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:03.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.144+0000 7fe33f5cc700 1 --2- 172.21.15.3:0/1851491169 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe33809e000 0x7fe3381a6310 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:03.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.144+0000 7fe33f5cc700 1 -- 172.21.15.3:0/1851491169 >> 172.21.15.3:0/1851491169 conn(0x7fe338066f50 msgr2=0x7fe3381bacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:03.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.145+0000 7fe33f5cc700 1 -- 172.21.15.3:0/1851491169 shutdown_connections 2023-10-03T10:16:03.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.145+0000 7fe33f5cc700 1 -- 172.21.15.3:0/1851491169 wait complete. 2023-10-03T10:16:03.158 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:03.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.177+0000 7fae43a88700 1 -- 172.21.15.3:0/2557603655 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fae3c074ea0 con 0x7fae3c06a770 2023-10-03T10:16:03.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.177+0000 7fae327fc700 1 -- 172.21.15.3:0/2557603655 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fae38017070 con 0x7fae3c06a770 2023-10-03T10:16:03.177 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:03.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.179+0000 7fae43a88700 1 -- 172.21.15.3:0/2557603655 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fae3c06a770 msgr2=0x7fae3c11c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:03.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.179+0000 7fae43a88700 1 --2- 172.21.15.3:0/2557603655 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fae3c06a770 0x7fae3c11c5e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fae3800c4e0 tx=0x7fae3800c7f0 comp rx=0 tx=0).stop 2023-10-03T10:16:03.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.179+0000 7fae43a88700 1 -- 172.21.15.3:0/2557603655 shutdown_connections 2023-10-03T10:16:03.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.179+0000 7fae43a88700 1 --2- 172.21.15.3:0/2557603655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae3c13d1f0 0x7fae3c13bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:03.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.179+0000 7fae43a88700 1 --2- 172.21.15.3:0/2557603655 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fae3c06a770 0x7fae3c11c5e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:03.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.179+0000 7fae43a88700 1 --2- 172.21.15.3:0/2557603655 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae3c09de00 0x7fae3c13b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:03.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.179+0000 7fae43a88700 1 -- 172.21.15.3:0/2557603655 >> 172.21.15.3:0/2557603655 conn(0x7fae3c066f50 msgr2=0x7fae3c09b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:03.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.179+0000 7fae43a88700 1 -- 172.21.15.3:0/2557603655 shutdown_connections 2023-10-03T10:16:03.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.179+0000 7fae43a88700 1 -- 172.21.15.3:0/2557603655 wait complete. 2023-10-03T10:16:03.193 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:03.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.229+0000 7f1b4f53b700 1 -- 172.21.15.3:0/1305848848 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1b481a0b30 con 0x7f1b4809f240 2023-10-03T10:16:03.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.229+0000 7f1b3dffb700 1 -- 172.21.15.3:0/1305848848 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1b38055070 con 0x7f1b4809f240 2023-10-03T10:16:03.230 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:03.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.231+0000 7f1b4f53b700 1 -- 172.21.15.3:0/1305848848 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b4809f240 msgr2=0x7f1b4806bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:03.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.231+0000 7f1b4f53b700 1 --2- 172.21.15.3:0/1305848848 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b4809f240 0x7f1b4806bdf0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800c430 tx=0x7f1b3800c740 comp rx=0 tx=0).stop 2023-10-03T10:16:03.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.232+0000 7f1b4f53b700 1 -- 172.21.15.3:0/1305848848 shutdown_connections 2023-10-03T10:16:03.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.232+0000 7f1b4f53b700 1 --2- 172.21.15.3:0/1305848848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b4809f650 0x7f1b4806c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:03.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.232+0000 7f1b4f53b700 1 --2- 172.21.15.3:0/1305848848 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b4809e930 0x7f1b4806b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:03.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.232+0000 7f1b4f53b700 1 --2- 172.21.15.3:0/1305848848 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b4809f240 0x7f1b4806bdf0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:03.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.232+0000 7f1b4f53b700 1 -- 172.21.15.3:0/1305848848 >> 172.21.15.3:0/1305848848 conn(0x7f1b480a4210 msgr2=0x7f1b481b8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:03.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.232+0000 7f1b4f53b700 1 -- 172.21.15.3:0/1305848848 shutdown_connections 2023-10-03T10:16:03.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:03.232+0000 7f1b4f53b700 1 -- 172.21.15.3:0/1305848848 wait complete. 2023-10-03T10:16:03.245 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:04.123 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:04.159 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:04.194 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:04.246 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:04.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.275+0000 7f1ee6d11700 1 Processor -- start 2023-10-03T10:16:04.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.275+0000 7f1ee6d11700 1 -- start start 2023-10-03T10:16:04.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.276+0000 7f1ee6d11700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1ee006e570 0x7f1ee006e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:04.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.276+0000 7f1ee6d11700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ee009eca0 0x7f1ee006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:04.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.276+0000 7f1ee6d11700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ee0134610 0x7f1ee0134a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:04.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.276+0000 7f1ee6d11700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee0070ee0 con 0x7f1ee006e570 2023-10-03T10:16:04.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.276+0000 7f1ee6d11700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee0071480 con 0x7f1ee009eca0 2023-10-03T10:16:04.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.276+0000 7f1ee6d11700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee006f3c0 con 0x7f1ee0134610 2023-10-03T10:16:04.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.276+0000 7f1edffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ee009eca0 0x7f1ee006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:04.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.276+0000 7f1ee52ae700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ee0134610 0x7f1ee0134a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:04.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.276+0000 7f1ee52ae700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ee0134610 0x7f1ee0134a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52120/0 (socket says 172.21.15.3:52120) 2023-10-03T10:16:04.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.276+0000 7f1edffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ee009eca0 0x7f1ee006ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53012/0 (socket says 172.21.15.3:53012) 2023-10-03T10:16:04.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.276+0000 7f1ee52ae700 1 -- 172.21.15.3:0/1172811630 learned_addr learned my addr 172.21.15.3:0/1172811630 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:04.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.276+0000 7f1ee4aad700 1 --2- 172.21.15.3:0/1172811630 >> v2:172.21.15.3:3300/0 conn(0x7f1ee006e570 0x7f1ee006e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:04.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.276+0000 7f1ee52ae700 1 -- 172.21.15.3:0/1172811630 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ee009eca0 msgr2=0x7f1ee006ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:04.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.276+0000 7f1ee52ae700 1 --2- 172.21.15.3:0/1172811630 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ee009eca0 0x7f1ee006ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.276+0000 7f1ee52ae700 1 -- 172.21.15.3:0/1172811630 >> v2:172.21.15.3:3300/0 conn(0x7f1ee006e570 msgr2=0x7f1ee006e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:04.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.276+0000 7f1ee52ae700 1 --2- 172.21.15.3:0/1172811630 >> v2:172.21.15.3:3300/0 conn(0x7f1ee006e570 0x7f1ee006e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.276+0000 7f1ee52ae700 1 -- 172.21.15.3:0/1172811630 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee0071b90 con 0x7f1ee0134610 2023-10-03T10:16:04.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.276+0000 7f1ee4aad700 1 --2- 172.21.15.3:0/1172811630 >> v2:172.21.15.3:3300/0 conn(0x7f1ee006e570 0x7f1ee006e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:16:04.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.277+0000 7f1ee52ae700 1 --2- 172.21.15.3:0/1172811630 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ee0134610 0x7f1ee0134a90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8009e50 tx=0x7f1ed800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16114c6abb1b3038 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:04.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.277+0000 7f1edf7fe700 1 -- 172.21.15.3:0/1172811630 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed800e070 con 0x7f1ee0134610 2023-10-03T10:16:04.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.277+0000 7f1edf7fe700 1 -- 172.21.15.3:0/1172811630 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed800b910 con 0x7f1ee0134610 2023-10-03T10:16:04.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.277+0000 7f1edf7fe700 1 -- 172.21.15.3:0/1172811630 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed800bcc0 con 0x7f1ee0134610 2023-10-03T10:16:04.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.278+0000 7f1ee6d11700 1 -- 172.21.15.3:0/1172811630 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ee0134610 msgr2=0x7f1ee0134a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:04.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.278+0000 7f1ee6d11700 1 --2- 172.21.15.3:0/1172811630 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ee0134610 0x7f1ee0134a90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8009e50 tx=0x7f1ed800b330 comp rx=0 tx=0).stop 2023-10-03T10:16:04.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.278+0000 7f1ee6d11700 1 -- 172.21.15.3:0/1172811630 shutdown_connections 2023-10-03T10:16:04.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.278+0000 7f1ee6d11700 1 --2- 172.21.15.3:0/1172811630 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ee0134610 0x7f1ee0134a90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.278+0000 7f1ee6d11700 1 --2- 172.21.15.3:0/1172811630 >> v2:172.21.15.3:3300/0 conn(0x7f1ee006e570 0x7f1ee006e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.278+0000 7f1ee6d11700 1 --2- 172.21.15.3:0/1172811630 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ee009eca0 0x7f1ee006ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.278+0000 7f1ee6d11700 1 -- 172.21.15.3:0/1172811630 >> 172.21.15.3:0/1172811630 conn(0x7f1ee00a2aa0 msgr2=0x7f1ee00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:04.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.278+0000 7f1ee6d11700 1 -- 172.21.15.3:0/1172811630 shutdown_connections 2023-10-03T10:16:04.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.278+0000 7f1ee6d11700 1 -- 172.21.15.3:0/1172811630 wait complete. 2023-10-03T10:16:04.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.279+0000 7f1ee6d11700 1 Processor -- start 2023-10-03T10:16:04.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.279+0000 7f1ee6d11700 1 -- start start 2023-10-03T10:16:04.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.280+0000 7f1ee6d11700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ee006e570 0x7f1ee0127530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:04.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.280+0000 7f1ee6d11700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ee0134610 0x7f1ee0125ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:04.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.280+0000 7f1ee6d11700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ee01263e0 0x7f1ee0126880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:04.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.280+0000 7f1ee6d11700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee00a1e30 con 0x7f1ee0134610 2023-10-03T10:16:04.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.280+0000 7f1ee6d11700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee013e4b0 con 0x7f1ee01263e0 2023-10-03T10:16:04.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.280+0000 7f1ee6d11700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee013be60 con 0x7f1ee006e570 2023-10-03T10:16:04.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.280+0000 7f1edffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ee0134610 0x7f1ee0125ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:04.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.280+0000 7f1ee52ae700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ee01263e0 0x7f1ee0126880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:04.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.280+0000 7f1edffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ee0134610 0x7f1ee0125ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52908/0 (socket says 172.21.15.3:52908) 2023-10-03T10:16:04.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.280+0000 7f1edffff700 1 -- 172.21.15.3:0/581957001 learned_addr learned my addr 172.21.15.3:0/581957001 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:04.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.280+0000 7f1ee52ae700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ee01263e0 0x7f1ee0126880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53020/0 (socket says 172.21.15.3:53020) 2023-10-03T10:16:04.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.280+0000 7f1ee4aad700 1 --2- 172.21.15.3:0/581957001 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ee006e570 0x7f1ee0127530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:04.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.281+0000 7f1ee52ae700 1 -- 172.21.15.3:0/581957001 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ee006e570 msgr2=0x7f1ee0127530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:04.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.281+0000 7f1ee52ae700 1 --2- 172.21.15.3:0/581957001 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ee006e570 0x7f1ee0127530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.281+0000 7f1ee52ae700 1 -- 172.21.15.3:0/581957001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ee0134610 msgr2=0x7f1ee0125ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:04.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.281+0000 7f1ee52ae700 1 --2- 172.21.15.3:0/581957001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ee0134610 0x7f1ee0125ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.281+0000 7f1ee52ae700 1 -- 172.21.15.3:0/581957001 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed8009b30 con 0x7f1ee01263e0 2023-10-03T10:16:04.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.281+0000 7f1ee52ae700 1 --2- 172.21.15.3:0/581957001 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ee01263e0 0x7f1ee0126880 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8015000 tx=0x7f1ed8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:04.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.281+0000 7f1edd7fa700 1 -- 172.21.15.3:0/581957001 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8004450 con 0x7f1ee01263e0 2023-10-03T10:16:04.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.281+0000 7f1edd7fa700 1 -- 172.21.15.3:0/581957001 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed8004610 con 0x7f1ee01263e0 2023-10-03T10:16:04.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.281+0000 7f1ee6d11700 1 -- 172.21.15.3:0/581957001 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee013d4e0 con 0x7f1ee01263e0 2023-10-03T10:16:04.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.281+0000 7f1ee6d11700 1 -- 172.21.15.3:0/581957001 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee013df50 con 0x7f1ee01263e0 2023-10-03T10:16:04.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.281+0000 7f1edd7fa700 1 -- 172.21.15.3:0/581957001 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed80092e0 con 0x7f1ee01263e0 2023-10-03T10:16:04.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.282+0000 7f1edd7fa700 1 -- 172.21.15.3:0/581957001 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1ed8025460 con 0x7f1ee01263e0 2023-10-03T10:16:04.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.282+0000 7f1ee6d11700 1 -- 172.21.15.3:0/581957001 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec00052e0 con 0x7f1ee01263e0 2023-10-03T10:16:04.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.283+0000 7f1edd7fa700 1 -- 172.21.15.3:0/581957001 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1ed8041510 con 0x7f1ee01263e0 2023-10-03T10:16:04.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.288+0000 7f1edd7fa700 1 -- 172.21.15.3:0/581957001 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1ed8025c30 con 0x7f1ee01263e0 2023-10-03T10:16:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.317+0000 7f67b75a5700 1 Processor -- start 2023-10-03T10:16:04.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.317+0000 7f67b75a5700 1 -- start start 2023-10-03T10:16:04.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.317+0000 7f67b75a5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67b0130540 0x7f67b0130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:04.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.318+0000 7f67b75a5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f67b0130e50 0x7f67b006f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:04.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.318+0000 7f67b75a5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67b006fb20 0x7f67b006ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:04.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.318+0000 7f67b75a5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67b00704b0 con 0x7f67b0130e50 2023-10-03T10:16:04.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.318+0000 7f67b75a5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b009a390 con 0x7f67b006fb20 2023-10-03T10:16:04.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.318+0000 7f67b75a5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67b00716a0 con 0x7f67b0130540 2023-10-03T10:16:04.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.318+0000 7f67b5341700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67b0130540 0x7f67b0130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:04.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.318+0000 7f67b5341700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67b0130540 0x7f67b0130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52146/0 (socket says 172.21.15.3:52146) 2023-10-03T10:16:04.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.318+0000 7f67b5341700 1 -- 172.21.15.3:0/2444987290 learned_addr learned my addr 172.21.15.3:0/2444987290 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:04.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.318+0000 7f67b5b42700 1 --2- 172.21.15.3:0/2444987290 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67b006fb20 0x7f67b006ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:04.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.318+0000 7f67b4b40700 1 --2- 172.21.15.3:0/2444987290 >> v2:172.21.15.3:3300/0 conn(0x7f67b0130e50 0x7f67b006f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:04.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.318+0000 7f67b5341700 1 -- 172.21.15.3:0/2444987290 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67b006fb20 msgr2=0x7f67b006ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:04.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.318+0000 7f67b5341700 1 --2- 172.21.15.3:0/2444987290 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67b006fb20 0x7f67b006ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.318+0000 7f67b5341700 1 -- 172.21.15.3:0/2444987290 >> v2:172.21.15.3:3300/0 conn(0x7f67b0130e50 msgr2=0x7f67b006f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:04.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.318+0000 7f67b5341700 1 --2- 172.21.15.3:0/2444987290 >> v2:172.21.15.3:3300/0 conn(0x7f67b0130e50 0x7f67b006f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.318+0000 7f67b5341700 1 -- 172.21.15.3:0/2444987290 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b0071290 con 0x7f67b0130540 2023-10-03T10:16:04.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.318+0000 7f67b5341700 1 --2- 172.21.15.3:0/2444987290 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67b0130540 0x7f67b0130910 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00bbc0 tx=0x7f67ac00bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80968f4936da158b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:04.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.319+0000 7f67a7fff700 1 -- 172.21.15.3:0/2444987290 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac00dcf0 con 0x7f67b0130540 2023-10-03T10:16:04.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.319+0000 7f67a7fff700 1 -- 172.21.15.3:0/2444987290 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ac00de50 con 0x7f67b0130540 2023-10-03T10:16:04.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.319+0000 7f67a7fff700 1 -- 172.21.15.3:0/2444987290 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac010690 con 0x7f67b0130540 2023-10-03T10:16:04.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.320+0000 7f67b75a5700 1 -- 172.21.15.3:0/2444987290 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67b0130540 msgr2=0x7f67b0130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:04.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.320+0000 7f67b75a5700 1 --2- 172.21.15.3:0/2444987290 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67b0130540 0x7f67b0130910 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00bbc0 tx=0x7f67ac00bed0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.320+0000 7f67b75a5700 1 -- 172.21.15.3:0/2444987290 shutdown_connections 2023-10-03T10:16:04.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.320+0000 7f67b75a5700 1 --2- 172.21.15.3:0/2444987290 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67b006fb20 0x7f67b006ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.320+0000 7f67b75a5700 1 --2- 172.21.15.3:0/2444987290 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67b0130540 0x7f67b0130910 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.320+0000 7f67b75a5700 1 --2- 172.21.15.3:0/2444987290 >> v2:172.21.15.3:3300/0 conn(0x7f67b0130e50 0x7f67b006f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.320+0000 7f67b75a5700 1 -- 172.21.15.3:0/2444987290 >> 172.21.15.3:0/2444987290 conn(0x7f67b00a2aa0 msgr2=0x7f67b00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:04.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.320+0000 7f67b75a5700 1 -- 172.21.15.3:0/2444987290 shutdown_connections 2023-10-03T10:16:04.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.320+0000 7f67b75a5700 1 -- 172.21.15.3:0/2444987290 wait complete. 2023-10-03T10:16:04.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.321+0000 7f67b75a5700 1 Processor -- start 2023-10-03T10:16:04.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.322+0000 7f67b75a5700 1 -- start start 2023-10-03T10:16:04.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.322+0000 7f67b75a5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67b006fb20 0x7f67b011fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:04.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.322+0000 7f67b75a5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67b0130540 0x7f67b011ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:04.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.322+0000 7f67b75a5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67b0130e50 0x7f67b0120480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:04.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.322+0000 7f67b75a5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b0121e50 con 0x7f67b0130e50 2023-10-03T10:16:04.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.322+0000 7f67b75a5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b0121610 con 0x7f67b0130540 2023-10-03T10:16:04.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.322+0000 7f67b75a5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67b00a0a50 con 0x7f67b006fb20 2023-10-03T10:16:04.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.323+0000 7f67b5341700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67b006fb20 0x7f67b011fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:04.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.323+0000 7f67b4b40700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67b0130540 0x7f67b011ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:04.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.323+0000 7f67b5341700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67b006fb20 0x7f67b011fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52148/0 (socket says 172.21.15.3:52148) 2023-10-03T10:16:04.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.323+0000 7f67b5341700 1 -- 172.21.15.3:0/1696210 learned_addr learned my addr 172.21.15.3:0/1696210 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:04.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.323+0000 7f67b5b42700 1 --2- 172.21.15.3:0/1696210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67b0130e50 0x7f67b0120480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:04.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.323+0000 7f67b5341700 1 -- 172.21.15.3:0/1696210 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67b0130540 msgr2=0x7f67b011ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:04.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.323+0000 7f67b5341700 1 --2- 172.21.15.3:0/1696210 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67b0130540 0x7f67b011ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.323+0000 7f67b5341700 1 -- 172.21.15.3:0/1696210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67b0130e50 msgr2=0x7f67b0120480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:04.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.323+0000 7f67b5341700 1 --2- 172.21.15.3:0/1696210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67b0130e50 0x7f67b0120480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.323+0000 7f67b5341700 1 -- 172.21.15.3:0/1696210 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ac00b7f0 con 0x7f67b006fb20 2023-10-03T10:16:04.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.323+0000 7f67b5341700 1 --2- 172.21.15.3:0/1696210 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67b006fb20 0x7f67b011fa00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00bb10 tx=0x7f67ac004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:04.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.323+0000 7f67a5ffb700 1 -- 172.21.15.3:0/1696210 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac01d070 con 0x7f67b006fb20 2023-10-03T10:16:04.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.324+0000 7f67a5ffb700 1 -- 172.21.15.3:0/1696210 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ac0038b0 con 0x7f67b006fb20 2023-10-03T10:16:04.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.324+0000 7f67a5ffb700 1 -- 172.21.15.3:0/1696210 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac003c60 con 0x7f67b006fb20 2023-10-03T10:16:04.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.324+0000 7f67b75a5700 1 -- 172.21.15.3:0/1696210 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b01234b0 con 0x7f67b006fb20 2023-10-03T10:16:04.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.324+0000 7f67b75a5700 1 -- 172.21.15.3:0/1696210 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b0122f60 con 0x7f67b006fb20 2023-10-03T10:16:04.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.325+0000 7f67b75a5700 1 -- 172.21.15.3:0/1696210 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b0074ea0 con 0x7f67b006fb20 2023-10-03T10:16:04.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.325+0000 7f67a5ffb700 1 -- 172.21.15.3:0/1696210 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f67ac02b430 con 0x7f67b006fb20 2023-10-03T10:16:04.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.325+0000 7f67a5ffb700 1 -- 172.21.15.3:0/1696210 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f67ac040730 con 0x7f67b006fb20 2023-10-03T10:16:04.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.330+0000 7f67a5ffb700 1 -- 172.21.15.3:0/1696210 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f67ac02a7a0 con 0x7f67b006fb20 2023-10-03T10:16:04.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.359+0000 7f88a1418700 1 Processor -- start 2023-10-03T10:16:04.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.359+0000 7f88a1418700 1 -- start start 2023-10-03T10:16:04.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.360+0000 7f88a1418700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f889c09d6f0 0x7f889c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:04.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.360+0000 7f88a1418700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f889c09e410 0x7f889c130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:04.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.360+0000 7f88a1418700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f889c09f130 con 0x7f889c09e000 2023-10-03T10:16:04.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.360+0000 7f88a1418700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f889c09f520 con 0x7f889c09d6f0 2023-10-03T10:16:04.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.360+0000 7f88a1418700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889c09e930 con 0x7f889c09e410 2023-10-03T10:16:04.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.360+0000 7f889affd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f889c09d6f0 0x7f889c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:04.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.360+0000 7f889affd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f889c09d6f0 0x7f889c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52928/0 (socket says 172.21.15.3:52928) 2023-10-03T10:16:04.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.360+0000 7f889a7fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f889c09e000 0x7f889c130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38886/0 (socket says 172.21.15.3:38886) 2023-10-03T10:16:04.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.360+0000 7f889affd700 1 -- 172.21.15.3:0/275219466 learned_addr learned my addr 172.21.15.3:0/275219466 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:04.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.360+0000 7f889b7fe700 1 --2- 172.21.15.3:0/275219466 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f889c09e410 0x7f889c130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:04.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.360+0000 7f889b7fe700 1 -- 172.21.15.3:0/275219466 >> v2:172.21.15.3:3300/0 conn(0x7f889c09d6f0 msgr2=0x7f889c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:04.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.360+0000 7f889b7fe700 1 --2- 172.21.15.3:0/275219466 >> v2:172.21.15.3:3300/0 conn(0x7f889c09d6f0 0x7f889c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.360+0000 7f889b7fe700 1 -- 172.21.15.3:0/275219466 >> v1:172.21.15.3:6789/0 conn(0x7f889c09e000 legacy=0x7f889c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:04.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.360+0000 7f889b7fe700 1 -- 172.21.15.3:0/275219466 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889c09b2d0 con 0x7f889c09e410 2023-10-03T10:16:04.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.361+0000 7f889b7fe700 1 --2- 172.21.15.3:0/275219466 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f889c09e410 0x7f889c130d70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8894009d90 tx=0x7f889400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41c3581b788e92db server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:04.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.361+0000 7f8899ffb700 1 -- 172.21.15.3:0/275219466 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889400e070 con 0x7f889c09e410 2023-10-03T10:16:04.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.361+0000 7f8899ffb700 1 -- 172.21.15.3:0/275219466 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f889400bb20 con 0x7f889c09e410 2023-10-03T10:16:04.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.361+0000 7f8899ffb700 1 -- 172.21.15.3:0/275219466 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8894012430 con 0x7f889c09e410 2023-10-03T10:16:04.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.362+0000 7f88a1418700 1 -- 172.21.15.3:0/275219466 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f889c09e410 msgr2=0x7f889c130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:04.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.362+0000 7f88a1418700 1 --2- 172.21.15.3:0/275219466 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f889c09e410 0x7f889c130d70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8894009d90 tx=0x7f889400b330 comp rx=0 tx=0).stop 2023-10-03T10:16:04.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.362+0000 7f88a1418700 1 -- 172.21.15.3:0/275219466 shutdown_connections 2023-10-03T10:16:04.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.362+0000 7f88a1418700 1 --2- 172.21.15.3:0/275219466 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f889c09e410 0x7f889c130d70 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.362+0000 7f88a1418700 1 --2- 172.21.15.3:0/275219466 >> v2:172.21.15.3:3300/0 conn(0x7f889c09d6f0 0x7f889c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.362+0000 7f88a1418700 1 -- 172.21.15.3:0/275219466 >> 172.21.15.3:0/275219466 conn(0x7f889c066f50 msgr2=0x7f889c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:04.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.362+0000 7f88a1418700 1 -- 172.21.15.3:0/275219466 shutdown_connections 2023-10-03T10:16:04.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.362+0000 7f88a1418700 1 -- 172.21.15.3:0/275219466 wait complete. 2023-10-03T10:16:04.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.363+0000 7f88a1418700 1 Processor -- start 2023-10-03T10:16:04.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.363+0000 7f88a1418700 1 -- start start 2023-10-03T10:16:04.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.364+0000 7f88a1418700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f889c09d6f0 0x7f889c11df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:04.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.364+0000 7f88a1418700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f889c09e000 0x7f889c11e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:04.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.364+0000 7f88a1418700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f889c09e410 0x7f889c11e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:04.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.364+0000 7f88a1418700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c121780 con 0x7f889c09e000 2023-10-03T10:16:04.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.364+0000 7f88a1418700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889c130280 con 0x7f889c09d6f0 2023-10-03T10:16:04.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.364+0000 7f88a1418700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f889c1200d0 con 0x7f889c09e410 2023-10-03T10:16:04.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.364+0000 7f889affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f889c09d6f0 0x7f889c11df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:04.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.364+0000 7f889affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f889c09d6f0 0x7f889c11df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53076/0 (socket says 172.21.15.3:53076) 2023-10-03T10:16:04.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.364+0000 7f889affd700 1 -- 172.21.15.3:0/1557401904 learned_addr learned my addr 172.21.15.3:0/1557401904 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:04.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.364+0000 7f889a7fc700 1 --2- 172.21.15.3:0/1557401904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f889c09e000 0x7f889c11e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:04.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.364+0000 7f889affd700 1 -- 172.21.15.3:0/1557401904 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f889c09e410 msgr2=0x7f889c11e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:04.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.364+0000 7f889affd700 1 --2- 172.21.15.3:0/1557401904 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f889c09e410 0x7f889c11e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.364+0000 7f889affd700 1 -- 172.21.15.3:0/1557401904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f889c09e000 msgr2=0x7f889c11e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:04.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.364+0000 7f889affd700 1 --2- 172.21.15.3:0/1557401904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f889c09e000 0x7f889c11e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.364+0000 7f889affd700 1 -- 172.21.15.3:0/1557401904 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8894009a70 con 0x7f889c09d6f0 2023-10-03T10:16:04.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.364+0000 7f889affd700 1 --2- 172.21.15.3:0/1557401904 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f889c09d6f0 0x7f889c11df30 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f888c00dcd0 tx=0x7f888c0100c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:04.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.365+0000 7f88837fe700 1 -- 172.21.15.3:0/1557401904 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c00db40 con 0x7f889c09d6f0 2023-10-03T10:16:04.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.365+0000 7f88a1418700 1 -- 172.21.15.3:0/1557401904 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f889c120d00 con 0x7f889c09d6f0 2023-10-03T10:16:04.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.365+0000 7f88837fe700 1 -- 172.21.15.3:0/1557401904 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888c013040 con 0x7f889c09d6f0 2023-10-03T10:16:04.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.365+0000 7f88837fe700 1 -- 172.21.15.3:0/1557401904 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c010c10 con 0x7f889c09d6f0 2023-10-03T10:16:04.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.366+0000 7f88a1418700 1 -- 172.21.15.3:0/1557401904 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f889c09c700 con 0x7f889c09d6f0 2023-10-03T10:16:04.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.366+0000 7f88837fe700 1 -- 172.21.15.3:0/1557401904 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f888c010d70 con 0x7f889c09d6f0 2023-10-03T10:16:04.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.366+0000 7f88837fe700 1 -- 172.21.15.3:0/1557401904 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f888c043170 con 0x7f889c09d6f0 2023-10-03T10:16:04.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.367+0000 7f88a1418700 1 -- 172.21.15.3:0/1557401904 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f889c11ac20 con 0x7f889c09d6f0 2023-10-03T10:16:04.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.376+0000 7f88837fe700 1 -- 172.21.15.3:0/1557401904 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f888c044060 con 0x7f889c09d6f0 2023-10-03T10:16:04.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.408+0000 7fab1cc7e700 1 Processor -- start 2023-10-03T10:16:04.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.408+0000 7fab1cc7e700 1 -- start start 2023-10-03T10:16:04.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.409+0000 7fab1cc7e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fab18130200 0x7fab1806f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:04.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.409+0000 7fab1cc7e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab181305d0 0x7fab1809de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:04.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.409+0000 7fab1cc7e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab1809fb40 con 0x7fab18130a20 2023-10-03T10:16:04.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.409+0000 7fab1cc7e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab1806fc00 con 0x7fab18130200 2023-10-03T10:16:04.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.409+0000 7fab1cc7e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1812fdd0 con 0x7fab181305d0 2023-10-03T10:16:04.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.409+0000 7fab15d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab181305d0 0x7fab1809de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:04.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.409+0000 7fab15d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab181305d0 0x7fab1809de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53080/0 (socket says 172.21.15.3:53080) 2023-10-03T10:16:04.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.409+0000 7fab15d9b700 1 -- 172.21.15.3:0/672958077 learned_addr learned my addr 172.21.15.3:0/672958077 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:04.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.409+0000 7fab1659c700 1 --2- 172.21.15.3:0/672958077 >> v2:172.21.15.3:3300/0 conn(0x7fab18130200 0x7fab1806f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:04.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.409+0000 7fab15d9b700 1 -- 172.21.15.3:0/672958077 >> v2:172.21.15.3:3300/0 conn(0x7fab18130200 msgr2=0x7fab1806f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:04.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.409+0000 7fab15d9b700 1 --2- 172.21.15.3:0/672958077 >> v2:172.21.15.3:3300/0 conn(0x7fab18130200 0x7fab1806f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.409+0000 7fab15d9b700 1 -- 172.21.15.3:0/672958077 >> v1:172.21.15.3:6789/0 conn(0x7fab18130a20 legacy=0x7fab1809e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:04.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.409+0000 7fab15d9b700 1 -- 172.21.15.3:0/672958077 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab18066680 con 0x7fab181305d0 2023-10-03T10:16:04.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.410+0000 7fab15d9b700 1 --2- 172.21.15.3:0/672958077 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab181305d0 0x7fab1809de50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009920 tx=0x7fab0c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=195f57c81f7b5523 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:04.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.410+0000 7fab1559a700 1 -- 172.21.15.3:0/672958077 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c00ba20 con 0x7fab181305d0 2023-10-03T10:16:04.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.410+0000 7fab1559a700 1 -- 172.21.15.3:0/672958077 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0c00bb80 con 0x7fab181305d0 2023-10-03T10:16:04.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.410+0000 7fab1559a700 1 -- 172.21.15.3:0/672958077 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c009e70 con 0x7fab181305d0 2023-10-03T10:16:04.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.411+0000 7fab1cc7e700 1 -- 172.21.15.3:0/672958077 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab181305d0 msgr2=0x7fab1809de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:04.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.411+0000 7fab1cc7e700 1 --2- 172.21.15.3:0/672958077 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab181305d0 0x7fab1809de50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009920 tx=0x7fab0c009c30 comp rx=0 tx=0).stop 2023-10-03T10:16:04.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.411+0000 7fab1cc7e700 1 -- 172.21.15.3:0/672958077 shutdown_connections 2023-10-03T10:16:04.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.411+0000 7fab1cc7e700 1 --2- 172.21.15.3:0/672958077 >> v2:172.21.15.3:3300/0 conn(0x7fab18130200 0x7fab1806f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.411+0000 7fab1cc7e700 1 --2- 172.21.15.3:0/672958077 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab181305d0 0x7fab1809de50 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.411+0000 7fab1cc7e700 1 -- 172.21.15.3:0/672958077 >> 172.21.15.3:0/672958077 conn(0x7fab180a27f0 msgr2=0x7fab180a2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:04.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.411+0000 7fab1cc7e700 1 -- 172.21.15.3:0/672958077 shutdown_connections 2023-10-03T10:16:04.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.411+0000 7fab1cc7e700 1 -- 172.21.15.3:0/672958077 wait complete. 2023-10-03T10:16:04.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.412+0000 7fab1cc7e700 1 Processor -- start 2023-10-03T10:16:04.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.412+0000 7fab1cc7e700 1 -- start start 2023-10-03T10:16:04.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.413+0000 7fab1cc7e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab18130200 0x7fab18118c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:04.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.413+0000 7fab1cc7e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab181305d0 0x7fab181191a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:04.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.413+0000 7fab1cc7e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab18130a20 0x7fab181196e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:04.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.413+0000 7fab1cc7e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1813a120 con 0x7fab18130200 2023-10-03T10:16:04.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.413+0000 7fab1cc7e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab18139f30 con 0x7fab18130a20 2023-10-03T10:16:04.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.413+0000 7fab1cc7e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fab1811b440 con 0x7fab181305d0 2023-10-03T10:16:04.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.413+0000 7fab1659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab18130200 0x7fab18118c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:04.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.413+0000 7fab1659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab18130200 0x7fab18118c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52966/0 (socket says 172.21.15.3:52966) 2023-10-03T10:16:04.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.413+0000 7fab1659c700 1 -- 172.21.15.3:0/657339863 learned_addr learned my addr 172.21.15.3:0/657339863 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:04.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.413+0000 7fab15d9b700 1 --2- 172.21.15.3:0/657339863 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab181305d0 0x7fab181191a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:04.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.413+0000 7fab1659c700 1 -- 172.21.15.3:0/657339863 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab181305d0 msgr2=0x7fab181191a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:04.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.413+0000 7fab1659c700 1 --2- 172.21.15.3:0/657339863 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab181305d0 0x7fab181191a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.413+0000 7fab1659c700 1 -- 172.21.15.3:0/657339863 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab18130a20 msgr2=0x7fab181196e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:04.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.413+0000 7fab1659c700 1 --2- 172.21.15.3:0/657339863 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab18130a20 0x7fab181196e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.413+0000 7fab1659c700 1 -- 172.21.15.3:0/657339863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab0c009550 con 0x7fab18130200 2023-10-03T10:16:04.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.413+0000 7fab1659c700 1 --2- 172.21.15.3:0/657339863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab18130200 0x7fab18118c60 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fab0800e810 tx=0x7fab0800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:04.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.414+0000 7fab06ffd700 1 -- 172.21.15.3:0/657339863 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab08013610 con 0x7fab18130200 2023-10-03T10:16:04.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.414+0000 7fab06ffd700 1 -- 172.21.15.3:0/657339863 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab08013770 con 0x7fab18130200 2023-10-03T10:16:04.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.414+0000 7fab1cc7e700 1 -- 172.21.15.3:0/657339863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1811a350 con 0x7fab18130200 2023-10-03T10:16:04.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.414+0000 7fab06ffd700 1 -- 172.21.15.3:0/657339863 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab08013b20 con 0x7fab18130200 2023-10-03T10:16:04.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.414+0000 7fab1cc7e700 1 -- 172.21.15.3:0/657339863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1811b160 con 0x7fab18130200 2023-10-03T10:16:04.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.415+0000 7fab06ffd700 1 -- 172.21.15.3:0/657339863 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fab08013c80 con 0x7fab18130200 2023-10-03T10:16:04.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.415+0000 7fab06ffd700 1 -- 172.21.15.3:0/657339863 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fab0803f360 con 0x7fab18130200 2023-10-03T10:16:04.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.416+0000 7fab1cc7e700 1 -- 172.21.15.3:0/657339863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab1804f2a0 con 0x7fab18130200 2023-10-03T10:16:04.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.421+0000 7fab06ffd700 1 -- 172.21.15.3:0/657339863 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fab08012370 con 0x7fab18130200 2023-10-03T10:16:04.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.502+0000 7f1ee6d11700 1 -- 172.21.15.3:0/581957001 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1ec0005590 con 0x7f1ee01263e0 2023-10-03T10:16:04.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.502+0000 7f1edd7fa700 1 -- 172.21.15.3:0/581957001 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1ed801ad10 con 0x7f1ee01263e0 2023-10-03T10:16:04.503 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:04.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.506+0000 7f1ee6d11700 1 -- 172.21.15.3:0/581957001 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ee01263e0 msgr2=0x7f1ee0126880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.506+0000 7f1ee6d11700 1 --2- 172.21.15.3:0/581957001 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ee01263e0 0x7f1ee0126880 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8015000 tx=0x7f1ed8004ad0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.506+0000 7f1ee6d11700 1 -- 172.21.15.3:0/581957001 shutdown_connections 2023-10-03T10:16:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.506+0000 7f1ee6d11700 1 --2- 172.21.15.3:0/581957001 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ee01263e0 0x7f1ee0126880 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.506+0000 7f1ee6d11700 1 --2- 172.21.15.3:0/581957001 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ee006e570 0x7f1ee0127530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.506+0000 7f1ee6d11700 1 --2- 172.21.15.3:0/581957001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ee0134610 0x7f1ee0125ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.506+0000 7f1ee6d11700 1 -- 172.21.15.3:0/581957001 >> 172.21.15.3:0/581957001 conn(0x7f1ee00a2aa0 msgr2=0x7f1ee0099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.506+0000 7f1ee6d11700 1 -- 172.21.15.3:0/581957001 shutdown_connections 2023-10-03T10:16:04.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.506+0000 7f1ee6d11700 1 -- 172.21.15.3:0/581957001 wait complete. 2023-10-03T10:16:04.521 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:04.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.550+0000 7f67b75a5700 1 -- 172.21.15.3:0/1696210 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f67b0123230 con 0x7f67b006fb20 2023-10-03T10:16:04.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.550+0000 7f67a5ffb700 1 -- 172.21.15.3:0/1696210 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f67ac027070 con 0x7f67b006fb20 2023-10-03T10:16:04.551 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:04.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.554+0000 7f67b75a5700 1 -- 172.21.15.3:0/1696210 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67b006fb20 msgr2=0x7f67b011fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:04.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.554+0000 7f67b75a5700 1 --2- 172.21.15.3:0/1696210 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67b006fb20 0x7f67b011fa00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00bb10 tx=0x7f67ac004750 comp rx=0 tx=0).stop 2023-10-03T10:16:04.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.554+0000 7f67b75a5700 1 -- 172.21.15.3:0/1696210 shutdown_connections 2023-10-03T10:16:04.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.554+0000 7f67b75a5700 1 --2- 172.21.15.3:0/1696210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67b0130e50 0x7f67b0120480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.554+0000 7f67b75a5700 1 --2- 172.21.15.3:0/1696210 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f67b006fb20 0x7f67b011fa00 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.554+0000 7f67b75a5700 1 --2- 172.21.15.3:0/1696210 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f67b0130540 0x7f67b011ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.554+0000 7f67b75a5700 1 -- 172.21.15.3:0/1696210 >> 172.21.15.3:0/1696210 conn(0x7f67b00a2aa0 msgr2=0x7f67b012fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:04.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.554+0000 7f67b75a5700 1 -- 172.21.15.3:0/1696210 shutdown_connections 2023-10-03T10:16:04.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.554+0000 7f67b75a5700 1 -- 172.21.15.3:0/1696210 wait complete. 2023-10-03T10:16:04.569 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:04.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.596+0000 7f88a1418700 1 -- 172.21.15.3:0/1557401904 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f889c074ea0 con 0x7f889c09d6f0 2023-10-03T10:16:04.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.596+0000 7f88837fe700 1 -- 172.21.15.3:0/1557401904 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f888c014c20 con 0x7f889c09d6f0 2023-10-03T10:16:04.597 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:04.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.599+0000 7f88a1418700 1 -- 172.21.15.3:0/1557401904 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f889c09d6f0 msgr2=0x7f889c11df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:04.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.599+0000 7f88a1418700 1 --2- 172.21.15.3:0/1557401904 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f889c09d6f0 0x7f889c11df30 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f888c00dcd0 tx=0x7f888c0100c0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.599+0000 7f88a1418700 1 -- 172.21.15.3:0/1557401904 shutdown_connections 2023-10-03T10:16:04.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.599+0000 7f88a1418700 1 --2- 172.21.15.3:0/1557401904 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f889c09e410 0x7f889c11e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.599+0000 7f88a1418700 1 --2- 172.21.15.3:0/1557401904 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f889c09d6f0 0x7f889c11df30 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.599+0000 7f88a1418700 1 --2- 172.21.15.3:0/1557401904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f889c09e000 0x7f889c11e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.599+0000 7f88a1418700 1 -- 172.21.15.3:0/1557401904 >> 172.21.15.3:0/1557401904 conn(0x7f889c066f50 msgr2=0x7f889c130b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:04.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.599+0000 7f88a1418700 1 -- 172.21.15.3:0/1557401904 shutdown_connections 2023-10-03T10:16:04.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.599+0000 7f88a1418700 1 -- 172.21.15.3:0/1557401904 wait complete. 2023-10-03T10:16:04.613 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:04.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.633+0000 7fab1cc7e700 1 -- 172.21.15.3:0/657339863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fab1806b9a0 con 0x7fab18130200 2023-10-03T10:16:04.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.633+0000 7fab06ffd700 1 -- 172.21.15.3:0/657339863 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fab08010310 con 0x7fab18130200 2023-10-03T10:16:04.634 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:04.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.636+0000 7fab1cc7e700 1 -- 172.21.15.3:0/657339863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab18130200 msgr2=0x7fab18118c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:04.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.636+0000 7fab1cc7e700 1 --2- 172.21.15.3:0/657339863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab18130200 0x7fab18118c60 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fab0800e810 tx=0x7fab0800eb20 comp rx=0 tx=0).stop 2023-10-03T10:16:04.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.636+0000 7fab1cc7e700 1 -- 172.21.15.3:0/657339863 shutdown_connections 2023-10-03T10:16:04.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.636+0000 7fab1cc7e700 1 --2- 172.21.15.3:0/657339863 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab18130a20 0x7fab181196e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.636+0000 7fab1cc7e700 1 --2- 172.21.15.3:0/657339863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab18130200 0x7fab18118c60 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.636+0000 7fab1cc7e700 1 --2- 172.21.15.3:0/657339863 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab181305d0 0x7fab181191a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:04.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.636+0000 7fab1cc7e700 1 -- 172.21.15.3:0/657339863 >> 172.21.15.3:0/657339863 conn(0x7fab180a27f0 msgr2=0x7fab18070ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:04.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.636+0000 7fab1cc7e700 1 -- 172.21.15.3:0/657339863 shutdown_connections 2023-10-03T10:16:04.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:04.636+0000 7fab1cc7e700 1 -- 172.21.15.3:0/657339863 wait complete. 2023-10-03T10:16:04.650 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:05.521 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:05.570 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:05.613 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:05.651 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:05.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.674+0000 7f92b33e5700 1 Processor -- start 2023-10-03T10:16:05.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.675+0000 7f92b33e5700 1 -- start start 2023-10-03T10:16:05.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.675+0000 7f92b33e5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f92ac131810 0x7f92ac131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:05.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.675+0000 7f92b33e5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f92ac099a40 0x7f92ac099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:05.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.675+0000 7f92b33e5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92ac1313c0 con 0x7f92ac09a3b0 2023-10-03T10:16:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.675+0000 7f92b33e5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92ac09b580 con 0x7f92ac131810 2023-10-03T10:16:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.675+0000 7f92b33e5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac06e8c0 con 0x7f92ac099a40 2023-10-03T10:16:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.676+0000 7f92b1181700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f92ac131810 0x7f92ac131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.676+0000 7f92b1181700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f92ac131810 0x7f92ac131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52976/0 (socket says 172.21.15.3:52976) 2023-10-03T10:16:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.676+0000 7f92b1181700 1 -- 172.21.15.3:0/2133678528 learned_addr learned my addr 172.21.15.3:0/2133678528 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.676+0000 7f92b0980700 1 --2- 172.21.15.3:0/2133678528 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f92ac099a40 0x7f92ac099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.676+0000 7f92b1181700 1 -- 172.21.15.3:0/2133678528 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f92ac099a40 msgr2=0x7f92ac099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.676+0000 7f92b1181700 1 --2- 172.21.15.3:0/2133678528 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f92ac099a40 0x7f92ac099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.676+0000 7f92b1181700 1 -- 172.21.15.3:0/2133678528 >> v1:172.21.15.3:6789/0 conn(0x7f92ac09a3b0 legacy=0x7f92ac138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.676+0000 7f92b1181700 1 -- 172.21.15.3:0/2133678528 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac09a830 con 0x7f92ac131810 2023-10-03T10:16:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.676+0000 7f92b1181700 1 --2- 172.21.15.3:0/2133678528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92ac131810 0x7f92ac131be0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f92a80098f0 tx=0x7f92a8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b17e85e90d3f172 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.676+0000 7f92a3fff700 1 -- 172.21.15.3:0/2133678528 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a800b8a0 con 0x7f92ac131810 2023-10-03T10:16:05.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.676+0000 7f92a3fff700 1 -- 172.21.15.3:0/2133678528 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a800ba00 con 0x7f92ac131810 2023-10-03T10:16:05.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.676+0000 7f92a3fff700 1 -- 172.21.15.3:0/2133678528 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a800bd90 con 0x7f92ac131810 2023-10-03T10:16:05.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.677+0000 7f92b33e5700 1 -- 172.21.15.3:0/2133678528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92ac131810 msgr2=0x7f92ac131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:05.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.677+0000 7f92b33e5700 1 --2- 172.21.15.3:0/2133678528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92ac131810 0x7f92ac131be0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f92a80098f0 tx=0x7f92a8009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:05.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.677+0000 7f92b33e5700 1 -- 172.21.15.3:0/2133678528 shutdown_connections 2023-10-03T10:16:05.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.677+0000 7f92b33e5700 1 --2- 172.21.15.3:0/2133678528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92ac131810 0x7f92ac131be0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:05.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.677+0000 7f92b33e5700 1 --2- 172.21.15.3:0/2133678528 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f92ac099a40 0x7f92ac099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:05.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.677+0000 7f92b33e5700 1 -- 172.21.15.3:0/2133678528 >> 172.21.15.3:0/2133678528 conn(0x7f92ac0a2aa0 msgr2=0x7f92ac0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:05.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.678+0000 7f92b33e5700 1 -- 172.21.15.3:0/2133678528 shutdown_connections 2023-10-03T10:16:05.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.678+0000 7f92b33e5700 1 -- 172.21.15.3:0/2133678528 wait complete. 2023-10-03T10:16:05.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.679+0000 7f92b33e5700 1 Processor -- start 2023-10-03T10:16:05.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.679+0000 7f92b33e5700 1 -- start start 2023-10-03T10:16:05.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.679+0000 7f92b33e5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f92ac099a40 0x7f92ac11b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:05.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.679+0000 7f92b33e5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f92ac09a3b0 0x7f92ac11bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:05.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.679+0000 7f92b33e5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92ac131810 0x7f92ac11c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:05.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.679+0000 7f92b33e5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac11f1b0 con 0x7f92ac131810 2023-10-03T10:16:05.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.679+0000 7f92b33e5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac11d880 con 0x7f92ac099a40 2023-10-03T10:16:05.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.679+0000 7f92b33e5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac11e940 con 0x7f92ac09a3b0 2023-10-03T10:16:05.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.679+0000 7f92b1181700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f92ac099a40 0x7f92ac11b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:05.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.679+0000 7f92b1181700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f92ac099a40 0x7f92ac11b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53112/0 (socket says 172.21.15.3:53112) 2023-10-03T10:16:05.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.679+0000 7f92b1181700 1 -- 172.21.15.3:0/212286458 learned_addr learned my addr 172.21.15.3:0/212286458 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:05.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.679+0000 7f92b0980700 1 --2- 172.21.15.3:0/212286458 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f92ac09a3b0 0x7f92ac11bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:05.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.680+0000 7f92b1982700 1 --2- 172.21.15.3:0/212286458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92ac131810 0x7f92ac11c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:05.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.680+0000 7f92b1181700 1 -- 172.21.15.3:0/212286458 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f92ac09a3b0 msgr2=0x7f92ac11bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:05.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.680+0000 7f92b1181700 1 --2- 172.21.15.3:0/212286458 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f92ac09a3b0 0x7f92ac11bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:05.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.680+0000 7f92b1181700 1 -- 172.21.15.3:0/212286458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92ac131810 msgr2=0x7f92ac11c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:05.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.680+0000 7f92b1181700 1 --2- 172.21.15.3:0/212286458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92ac131810 0x7f92ac11c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:05.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.680+0000 7f92b1181700 1 -- 172.21.15.3:0/212286458 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a8009550 con 0x7f92ac099a40 2023-10-03T10:16:05.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.680+0000 7f92b1181700 1 --2- 172.21.15.3:0/212286458 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f92ac099a40 0x7f92ac11b690 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009c40 tx=0x7f92a8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:05.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.680+0000 7f92a1ffb700 1 -- 172.21.15.3:0/212286458 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a8004450 con 0x7f92ac099a40 2023-10-03T10:16:05.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.680+0000 7f92a1ffb700 1 -- 172.21.15.3:0/212286458 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a80045b0 con 0x7f92ac099a40 2023-10-03T10:16:05.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.680+0000 7f92b33e5700 1 -- 172.21.15.3:0/212286458 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92ac06eb00 con 0x7f92ac099a40 2023-10-03T10:16:05.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.680+0000 7f92a1ffb700 1 -- 172.21.15.3:0/212286458 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a801fa60 con 0x7f92ac099a40 2023-10-03T10:16:05.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.681+0000 7f92b33e5700 1 -- 172.21.15.3:0/212286458 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92ac11d2a0 con 0x7f92ac099a40 2023-10-03T10:16:05.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.682+0000 7f92a1ffb700 1 -- 172.21.15.3:0/212286458 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f92a8025070 con 0x7f92ac099a40 2023-10-03T10:16:05.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.682+0000 7f92b33e5700 1 -- 172.21.15.3:0/212286458 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92ac11e550 con 0x7f92ac099a40 2023-10-03T10:16:05.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.682+0000 7f92a1ffb700 1 -- 172.21.15.3:0/212286458 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f92a8044ed0 con 0x7f92ac099a40 2023-10-03T10:16:05.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.687+0000 7f92a1ffb700 1 -- 172.21.15.3:0/212286458 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f92a801fe00 con 0x7f92ac099a40 2023-10-03T10:16:05.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.728+0000 7ff13d838700 1 Processor -- start 2023-10-03T10:16:05.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.728+0000 7ff13d838700 1 -- start start 2023-10-03T10:16:05.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.729+0000 7ff13d838700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff13809d6f0 0x7ff13809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:05.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.729+0000 7ff13d838700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff138134100 0x7ff13813bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:05.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.729+0000 7ff13d838700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff13809d2f0 con 0x7ff13809e000 2023-10-03T10:16:05.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.729+0000 7ff13d838700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff13809a810 con 0x7ff13809d6f0 2023-10-03T10:16:05.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.729+0000 7ff13d838700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13809a470 con 0x7ff138134100 2023-10-03T10:16:05.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.729+0000 7ff136ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff13809d6f0 0x7ff13809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:05.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.729+0000 7ff136ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff13809d6f0 0x7ff13809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52990/0 (socket says 172.21.15.3:52990) 2023-10-03T10:16:05.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.729+0000 7ff136ffd700 1 -- 172.21.15.3:0/2249504572 learned_addr learned my addr 172.21.15.3:0/2249504572 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:05.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.729+0000 7ff1377fe700 1 --2- 172.21.15.3:0/2249504572 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff138134100 0x7ff13813bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:05.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.729+0000 7ff136ffd700 1 -- 172.21.15.3:0/2249504572 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff138134100 msgr2=0x7ff13813bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:05.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.729+0000 7ff136ffd700 1 --2- 172.21.15.3:0/2249504572 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff138134100 0x7ff13813bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:05.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.729+0000 7ff136ffd700 1 -- 172.21.15.3:0/2249504572 >> v1:172.21.15.3:6789/0 conn(0x7ff13809e000 legacy=0x7ff1381339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:05.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.729+0000 7ff136ffd700 1 -- 172.21.15.3:0/2249504572 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13809ec40 con 0x7ff13809d6f0 2023-10-03T10:16:05.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.730+0000 7ff136ffd700 1 --2- 172.21.15.3:0/2249504572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff13809d6f0 0x7ff13809dac0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7ff1280098f0 tx=0x7ff128009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb89f50eae4b056e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:05.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.730+0000 7ff135ffb700 1 -- 172.21.15.3:0/2249504572 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff128011830 con 0x7ff13809d6f0 2023-10-03T10:16:05.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.730+0000 7ff135ffb700 1 -- 172.21.15.3:0/2249504572 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff128011990 con 0x7ff13809d6f0 2023-10-03T10:16:05.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.730+0000 7ff135ffb700 1 -- 172.21.15.3:0/2249504572 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff128011d20 con 0x7ff13809d6f0 2023-10-03T10:16:05.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.731+0000 7ff13d838700 1 -- 172.21.15.3:0/2249504572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff13809d6f0 msgr2=0x7ff13809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:05.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.731+0000 7ff13d838700 1 --2- 172.21.15.3:0/2249504572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff13809d6f0 0x7ff13809dac0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7ff1280098f0 tx=0x7ff128009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:05.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.731+0000 7ff13d838700 1 -- 172.21.15.3:0/2249504572 shutdown_connections 2023-10-03T10:16:05.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.731+0000 7ff13d838700 1 --2- 172.21.15.3:0/2249504572 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff138134100 0x7ff13813bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:05.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.731+0000 7ff13d838700 1 --2- 172.21.15.3:0/2249504572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff13809d6f0 0x7ff13809dac0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:05.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.731+0000 7ff13d838700 1 -- 172.21.15.3:0/2249504572 >> 172.21.15.3:0/2249504572 conn(0x7ff138066f50 msgr2=0x7ff13809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:05.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.731+0000 7ff13d838700 1 -- 172.21.15.3:0/2249504572 shutdown_connections 2023-10-03T10:16:05.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.731+0000 7ff13d838700 1 -- 172.21.15.3:0/2249504572 wait complete. 2023-10-03T10:16:05.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.733+0000 7ff13d838700 1 Processor -- start 2023-10-03T10:16:05.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.733+0000 7ff13d838700 1 -- start start 2023-10-03T10:16:05.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.734+0000 7ff13d838700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff13809d6f0 0x7ff138122890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:05.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.734+0000 7ff13d838700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff13809e000 0x7ff138122dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:05.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.734+0000 7ff13d838700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff138134100 0x7ff138123310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:05.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.734+0000 7ff13d838700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1381261b0 con 0x7ff138134100 2023-10-03T10:16:05.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.734+0000 7ff13d838700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1381338f0 con 0x7ff13809e000 2023-10-03T10:16:05.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.734+0000 7ff13d838700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff1381333b0 con 0x7ff13809d6f0 2023-10-03T10:16:05.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.734+0000 7ff1377fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff138134100 0x7ff138123310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:05.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.734+0000 7ff1377fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff138134100 0x7ff138123310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52996/0 (socket says 172.21.15.3:52996) 2023-10-03T10:16:05.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.734+0000 7ff1377fe700 1 -- 172.21.15.3:0/3422233757 learned_addr learned my addr 172.21.15.3:0/3422233757 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:05.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.734+0000 7ff1367fc700 1 --2- 172.21.15.3:0/3422233757 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff13809e000 0x7ff138122dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:05.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.735+0000 7ff136ffd700 1 --2- 172.21.15.3:0/3422233757 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff13809d6f0 0x7ff138122890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:05.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.735+0000 7ff1377fe700 1 -- 172.21.15.3:0/3422233757 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff13809d6f0 msgr2=0x7ff138122890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:05.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.735+0000 7ff1377fe700 1 --2- 172.21.15.3:0/3422233757 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff13809d6f0 0x7ff138122890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:05.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.735+0000 7ff1377fe700 1 -- 172.21.15.3:0/3422233757 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff13809e000 msgr2=0x7ff138122dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:05.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.735+0000 7ff1377fe700 1 --2- 172.21.15.3:0/3422233757 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff13809e000 0x7ff138122dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:05.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.735+0000 7ff1377fe700 1 -- 172.21.15.3:0/3422233757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff128009550 con 0x7ff138134100 2023-10-03T10:16:05.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.735+0000 7ff136ffd700 1 --2- 172.21.15.3:0/3422233757 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff13809d6f0 0x7ff138122890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:16:05.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.735+0000 7ff1377fe700 1 --2- 172.21.15.3:0/3422233757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff138134100 0x7ff138123310 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff13000efc0 tx=0x7ff13000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:05.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.735+0000 7ff11ffff700 1 -- 172.21.15.3:0/3422233757 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff130009ea0 con 0x7ff138134100 2023-10-03T10:16:05.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.735+0000 7ff11ffff700 1 -- 172.21.15.3:0/3422233757 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13000f040 con 0x7ff138134100 2023-10-03T10:16:05.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.735+0000 7ff13d838700 1 -- 172.21.15.3:0/3422233757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff13809c640 con 0x7ff138134100 2023-10-03T10:16:05.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.735+0000 7ff11ffff700 1 -- 172.21.15.3:0/3422233757 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff130014480 con 0x7ff138134100 2023-10-03T10:16:05.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.735+0000 7ff13d838700 1 -- 172.21.15.3:0/3422233757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff138126b40 con 0x7ff138134100 2023-10-03T10:16:05.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.736+0000 7ff11ffff700 1 -- 172.21.15.3:0/3422233757 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff1300145e0 con 0x7ff138134100 2023-10-03T10:16:05.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.737+0000 7ff11ffff700 1 -- 172.21.15.3:0/3422233757 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff1300402e0 con 0x7ff138134100 2023-10-03T10:16:05.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.737+0000 7ff13d838700 1 -- 172.21.15.3:0/3422233757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff138074ea0 con 0x7ff138134100 2023-10-03T10:16:05.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.742+0000 7ff11ffff700 1 -- 172.21.15.3:0/3422233757 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff130007920 con 0x7ff138134100 2023-10-03T10:16:05.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.765+0000 7f7d2f439700 1 Processor -- start 2023-10-03T10:16:05.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.765+0000 7f7d2f439700 1 -- start start 2023-10-03T10:16:05.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.765+0000 7f7d2f439700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d2809e1d0 0x7f7d28138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:05.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.765+0000 7f7d2f439700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7d28139170 0x7f7d2813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:05.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.765+0000 7f7d2f439700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d2809b120 con 0x7f7d2809d6f0 2023-10-03T10:16:05.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.765+0000 7f7d2f439700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d2806b460 con 0x7f7d28139170 2023-10-03T10:16:05.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.765+0000 7f7d2f439700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2809a7e0 con 0x7f7d2809e1d0 2023-10-03T10:16:05.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.766+0000 7f7d2d9d6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7d28139170 0x7f7d2813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:05.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.766+0000 7f7d2c9d4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d2809e1d0 0x7f7d28138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:05.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.766+0000 7f7d2c9d4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d2809e1d0 0x7f7d28138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52192/0 (socket says 172.21.15.3:52192) 2023-10-03T10:16:05.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.766+0000 7f7d2d9d6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7d28139170 0x7f7d2813b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53012/0 (socket says 172.21.15.3:53012) 2023-10-03T10:16:05.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.766+0000 7f7d2c9d4700 1 -- 172.21.15.3:0/1965946777 learned_addr learned my addr 172.21.15.3:0/1965946777 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:05.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.766+0000 7f7d2c9d4700 1 -- 172.21.15.3:0/1965946777 >> v2:172.21.15.3:3300/0 conn(0x7f7d28139170 msgr2=0x7f7d2813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:05.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.766+0000 7f7d2c9d4700 1 --2- 172.21.15.3:0/1965946777 >> v2:172.21.15.3:3300/0 conn(0x7f7d28139170 0x7f7d2813b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:05.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.766+0000 7f7d2c9d4700 1 -- 172.21.15.3:0/1965946777 >> v1:172.21.15.3:6789/0 conn(0x7f7d2809d6f0 legacy=0x7f7d2809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:05.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.766+0000 7f7d2c9d4700 1 -- 172.21.15.3:0/1965946777 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2809a500 con 0x7f7d2809e1d0 2023-10-03T10:16:05.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.766+0000 7f7d2c9d4700 1 --2- 172.21.15.3:0/1965946777 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d2809e1d0 0x7f7d28138c30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7d18009920 tx=0x7f7d18009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd3b72119e920518 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:05.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.767+0000 7f7d1ffff700 1 -- 172.21.15.3:0/1965946777 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1800ba20 con 0x7f7d2809e1d0 2023-10-03T10:16:05.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.767+0000 7f7d1ffff700 1 -- 172.21.15.3:0/1965946777 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1800bb80 con 0x7f7d2809e1d0 2023-10-03T10:16:05.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.767+0000 7f7d1ffff700 1 -- 172.21.15.3:0/1965946777 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d18009e70 con 0x7f7d2809e1d0 2023-10-03T10:16:05.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.767+0000 7f7d2f439700 1 -- 172.21.15.3:0/1965946777 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d2809e1d0 msgr2=0x7f7d28138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:05.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.767+0000 7f7d2f439700 1 --2- 172.21.15.3:0/1965946777 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d2809e1d0 0x7f7d28138c30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7d18009920 tx=0x7f7d18009c30 comp rx=0 tx=0).stop 2023-10-03T10:16:05.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.767+0000 7f7d2f439700 1 -- 172.21.15.3:0/1965946777 shutdown_connections 2023-10-03T10:16:05.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.767+0000 7f7d2f439700 1 --2- 172.21.15.3:0/1965946777 >> v2:172.21.15.3:3300/0 conn(0x7f7d28139170 0x7f7d2813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:05.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.767+0000 7f7d2f439700 1 --2- 172.21.15.3:0/1965946777 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d2809e1d0 0x7f7d28138c30 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:05.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.767+0000 7f7d2f439700 1 -- 172.21.15.3:0/1965946777 >> 172.21.15.3:0/1965946777 conn(0x7f7d28066f50 msgr2=0x7f7d2809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:05.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.768+0000 7f7d2f439700 1 -- 172.21.15.3:0/1965946777 shutdown_connections 2023-10-03T10:16:05.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.768+0000 7f7d2f439700 1 -- 172.21.15.3:0/1965946777 wait complete. 2023-10-03T10:16:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.769+0000 7f7d2f439700 1 Processor -- start 2023-10-03T10:16:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.769+0000 7f7d2f439700 1 -- start start 2023-10-03T10:16:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.769+0000 7f7d2f439700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d2809d6f0 0x7f7d28119720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.769+0000 7f7d2f439700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d2809e1d0 0x7f7d28119c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.769+0000 7f7d2f439700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d28139170 0x7f7d281cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.769+0000 7f7d2f439700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2811d160 con 0x7f7d2809d6f0 2023-10-03T10:16:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.769+0000 7f7d2f439700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2811cdd0 con 0x7f7d2809e1d0 2023-10-03T10:16:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.769+0000 7f7d2f439700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2811a910 con 0x7f7d28139170 2023-10-03T10:16:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.770+0000 7f7d2d1d5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d2809d6f0 0x7f7d28119720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.770+0000 7f7d2d1d5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d2809d6f0 0x7f7d28119720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53026/0 (socket says 172.21.15.3:53026) 2023-10-03T10:16:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.770+0000 7f7d2d1d5700 1 -- 172.21.15.3:0/1259619793 learned_addr learned my addr 172.21.15.3:0/1259619793 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.770+0000 7f7d2d9d6700 1 --2- 172.21.15.3:0/1259619793 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d28139170 0x7f7d281cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.770+0000 7f7d2d9d6700 1 -- 172.21.15.3:0/1259619793 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d2809e1d0 msgr2=0x7f7d28119c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.770+0000 7f7d2d9d6700 1 --2- 172.21.15.3:0/1259619793 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d2809e1d0 0x7f7d28119c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.770+0000 7f7d2d9d6700 1 -- 172.21.15.3:0/1259619793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d2809d6f0 msgr2=0x7f7d28119720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.770+0000 7f7d2d9d6700 1 --2- 172.21.15.3:0/1259619793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d2809d6f0 0x7f7d28119720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.770+0000 7f7d2d9d6700 1 -- 172.21.15.3:0/1259619793 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d18009550 con 0x7f7d28139170 2023-10-03T10:16:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.770+0000 7f7d2d1d5700 1 --2- 172.21.15.3:0/1259619793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d2809d6f0 0x7f7d28119720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.770+0000 7f7d2d9d6700 1 --2- 172.21.15.3:0/1259619793 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d28139170 0x7f7d281cc790 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7d20007d90 tx=0x7f7d2000e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.771+0000 7f7d1dffb700 1 -- 172.21.15.3:0/1259619793 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d20018070 con 0x7f7d28139170 2023-10-03T10:16:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.771+0000 7f7d1dffb700 1 -- 172.21.15.3:0/1259619793 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2000ebe0 con 0x7f7d28139170 2023-10-03T10:16:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.771+0000 7f7d1dffb700 1 -- 172.21.15.3:0/1259619793 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d20003a20 con 0x7f7d28139170 2023-10-03T10:16:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.771+0000 7f7d2f439700 1 -- 172.21.15.3:0/1259619793 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2811b2a0 con 0x7f7d28139170 2023-10-03T10:16:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.772+0000 7f7d2f439700 1 -- 172.21.15.3:0/1259619793 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2811c160 con 0x7f7d28139170 2023-10-03T10:16:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.772+0000 7f7d1dffb700 1 -- 172.21.15.3:0/1259619793 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7d20014420 con 0x7f7d28139170 2023-10-03T10:16:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.772+0000 7f7d1dffb700 1 -- 172.21.15.3:0/1259619793 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7d2003f550 con 0x7f7d28139170 2023-10-03T10:16:05.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.773+0000 7f7d2f439700 1 -- 172.21.15.3:0/1259619793 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d2809e5a0 con 0x7f7d28139170 2023-10-03T10:16:05.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.778+0000 7f7d1dffb700 1 -- 172.21.15.3:0/1259619793 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7d200128d0 con 0x7f7d28139170 2023-10-03T10:16:05.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.808+0000 7f9aef63e700 1 Processor -- start 2023-10-03T10:16:05.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.809+0000 7f9aef63e700 1 -- start start 2023-10-03T10:16:05.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.809+0000 7f9aef63e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ae80a0a30 0x7f9ae80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:05.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.809+0000 7f9aef63e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9ae809d630 0x7f9ae80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:05.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.809+0000 7f9aef63e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ae8133930 0x7f9ae8133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:05.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.809+0000 7f9aef63e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae80a2a70 con 0x7f9ae809d630 2023-10-03T10:16:05.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.809+0000 7f9aef63e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae80a01d0 con 0x7f9ae80a0a30 2023-10-03T10:16:05.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.809+0000 7f9aef63e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae809e2b0 con 0x7f9ae8133930 2023-10-03T10:16:05.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.809+0000 7f9aed3da700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ae80a0a30 0x7f9ae80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:05.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.809+0000 7f9aed3da700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ae80a0a30 0x7f9ae80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53146/0 (socket says 172.21.15.3:53146) 2023-10-03T10:16:05.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.809+0000 7f9aed3da700 1 -- 172.21.15.3:0/273745728 learned_addr learned my addr 172.21.15.3:0/273745728 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:05.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.809+0000 7f9aedbdb700 1 --2- 172.21.15.3:0/273745728 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ae8133930 0x7f9ae8133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:05.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.810+0000 7f9aecbd9700 1 --2- 172.21.15.3:0/273745728 >> v2:172.21.15.3:3300/0 conn(0x7f9ae809d630 0x7f9ae80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:05.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.810+0000 7f9aed3da700 1 -- 172.21.15.3:0/273745728 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ae8133930 msgr2=0x7f9ae8133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:05.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.810+0000 7f9aed3da700 1 --2- 172.21.15.3:0/273745728 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ae8133930 0x7f9ae8133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:05.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.810+0000 7f9aed3da700 1 -- 172.21.15.3:0/273745728 >> v2:172.21.15.3:3300/0 conn(0x7f9ae809d630 msgr2=0x7f9ae80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:05.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.810+0000 7f9aed3da700 1 --2- 172.21.15.3:0/273745728 >> v2:172.21.15.3:3300/0 conn(0x7f9ae809d630 0x7f9ae80a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:05.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.810+0000 7f9aed3da700 1 -- 172.21.15.3:0/273745728 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae809dc50 con 0x7f9ae80a0a30 2023-10-03T10:16:05.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.810+0000 7f9aedbdb700 1 --2- 172.21.15.3:0/273745728 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ae8133930 0x7f9ae8133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:05.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.810+0000 7f9aed3da700 1 --2- 172.21.15.3:0/273745728 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ae80a0a30 0x7f9ae80a0e00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40098f0 tx=0x7f9ae4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8d444b7ac892208 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:05.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.810+0000 7f9adffff700 1 -- 172.21.15.3:0/273745728 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4011830 con 0x7f9ae80a0a30 2023-10-03T10:16:05.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.810+0000 7f9adffff700 1 -- 172.21.15.3:0/273745728 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae4011990 con 0x7f9ae80a0a30 2023-10-03T10:16:05.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.810+0000 7f9adffff700 1 -- 172.21.15.3:0/273745728 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4011d20 con 0x7f9ae80a0a30 2023-10-03T10:16:05.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.811+0000 7f9aef63e700 1 -- 172.21.15.3:0/273745728 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ae80a0a30 msgr2=0x7f9ae80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:05.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.811+0000 7f9aef63e700 1 --2- 172.21.15.3:0/273745728 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ae80a0a30 0x7f9ae80a0e00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40098f0 tx=0x7f9ae4009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:05.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.811+0000 7f9aef63e700 1 -- 172.21.15.3:0/273745728 shutdown_connections 2023-10-03T10:16:05.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.811+0000 7f9aef63e700 1 --2- 172.21.15.3:0/273745728 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ae8133930 0x7f9ae8133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:05.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.811+0000 7f9aef63e700 1 --2- 172.21.15.3:0/273745728 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ae80a0a30 0x7f9ae80a0e00 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:05.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.811+0000 7f9aef63e700 1 --2- 172.21.15.3:0/273745728 >> v2:172.21.15.3:3300/0 conn(0x7f9ae809d630 0x7f9ae80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:05.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.811+0000 7f9aef63e700 1 -- 172.21.15.3:0/273745728 >> 172.21.15.3:0/273745728 conn(0x7f9ae80a3050 msgr2=0x7f9ae80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:05.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.811+0000 7f9aef63e700 1 -- 172.21.15.3:0/273745728 shutdown_connections 2023-10-03T10:16:05.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.811+0000 7f9aef63e700 1 -- 172.21.15.3:0/273745728 wait complete. 2023-10-03T10:16:05.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.813+0000 7f9aef63e700 1 Processor -- start 2023-10-03T10:16:05.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.813+0000 7f9aef63e700 1 -- start start 2023-10-03T10:16:05.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.814+0000 7f9aef63e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ae809d630 0x7f9ae81c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:05.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.814+0000 7f9aef63e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ae80a0a30 0x7f9ae81c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:05.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.814+0000 7f9aef63e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ae8133930 0x7f9ae81c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:05.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.814+0000 7f9aef63e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae8120a90 con 0x7f9ae80a0a30 2023-10-03T10:16:05.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.814+0000 7f9aef63e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae80a41c0 con 0x7f9ae809d630 2023-10-03T10:16:05.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.814+0000 7f9aef63e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae81205f0 con 0x7f9ae8133930 2023-10-03T10:16:05.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.814+0000 7f9aed3da700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ae809d630 0x7f9ae81c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:05.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.814+0000 7f9aecbd9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ae80a0a30 0x7f9ae81c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:05.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.814+0000 7f9aed3da700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ae809d630 0x7f9ae81c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53156/0 (socket says 172.21.15.3:53156) 2023-10-03T10:16:05.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.814+0000 7f9aed3da700 1 -- 172.21.15.3:0/1012156279 learned_addr learned my addr 172.21.15.3:0/1012156279 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:05.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.814+0000 7f9aedbdb700 1 --2- 172.21.15.3:0/1012156279 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ae8133930 0x7f9ae81c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:05.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.814+0000 7f9aecbd9700 1 -- 172.21.15.3:0/1012156279 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ae8133930 msgr2=0x7f9ae81c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:05.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.814+0000 7f9aecbd9700 1 --2- 172.21.15.3:0/1012156279 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ae8133930 0x7f9ae81c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:05.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.814+0000 7f9aecbd9700 1 -- 172.21.15.3:0/1012156279 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ae809d630 msgr2=0x7f9ae81c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:05.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.814+0000 7f9aecbd9700 1 --2- 172.21.15.3:0/1012156279 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ae809d630 0x7f9ae81c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:05.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.814+0000 7f9aecbd9700 1 -- 172.21.15.3:0/1012156279 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae0009a70 con 0x7f9ae80a0a30 2023-10-03T10:16:05.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.814+0000 7f9aed3da700 1 --2- 172.21.15.3:0/1012156279 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ae809d630 0x7f9ae81c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:05.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.815+0000 7f9aecbd9700 1 --2- 172.21.15.3:0/1012156279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ae80a0a30 0x7f9ae81c85d0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f9ad800e7e0 tx=0x7f9ad800eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:05.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.815+0000 7f9addffb700 1 -- 172.21.15.3:0/1012156279 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad800c070 con 0x7f9ae80a0a30 2023-10-03T10:16:05.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.815+0000 7f9addffb700 1 -- 172.21.15.3:0/1012156279 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad8004020 con 0x7f9ae80a0a30 2023-10-03T10:16:05.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.815+0000 7f9aef63e700 1 -- 172.21.15.3:0/1012156279 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae4009550 con 0x7f9ae80a0a30 2023-10-03T10:16:05.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.815+0000 7f9addffb700 1 -- 172.21.15.3:0/1012156279 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad80043b0 con 0x7f9ae80a0a30 2023-10-03T10:16:05.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.815+0000 7f9aef63e700 1 -- 172.21.15.3:0/1012156279 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae8120d80 con 0x7f9ae80a0a30 2023-10-03T10:16:05.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.816+0000 7f9addffb700 1 -- 172.21.15.3:0/1012156279 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9ad80036a0 con 0x7f9ae80a0a30 2023-10-03T10:16:05.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.816+0000 7f9addffb700 1 -- 172.21.15.3:0/1012156279 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9ad803eee0 con 0x7f9ae80a0a30 2023-10-03T10:16:05.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.816+0000 7f9aef63e700 1 -- 172.21.15.3:0/1012156279 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae8121db0 con 0x7f9ae80a0a30 2023-10-03T10:16:05.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.821+0000 7f9addffb700 1 -- 172.21.15.3:0/1012156279 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9ad801c410 con 0x7f9ae80a0a30 2023-10-03T10:16:05.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.910+0000 7f92b33e5700 1 -- 172.21.15.3:0/212286458 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f92ac074ea0 con 0x7f92ac099a40 2023-10-03T10:16:05.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.910+0000 7f92a1ffb700 1 -- 172.21.15.3:0/212286458 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f92a8028e60 con 0x7f92ac099a40 2023-10-03T10:16:05.911 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:05.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.914+0000 7f92b33e5700 1 -- 172.21.15.3:0/212286458 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f92ac099a40 msgr2=0x7f92ac11b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:05.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.914+0000 7f92b33e5700 1 --2- 172.21.15.3:0/212286458 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f92ac099a40 0x7f92ac11b690 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009c40 tx=0x7f92a8004750 comp rx=0 tx=0).stop 2023-10-03T10:16:05.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.914+0000 7f92b33e5700 1 -- 172.21.15.3:0/212286458 shutdown_connections 2023-10-03T10:16:05.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.914+0000 7f92b33e5700 1 --2- 172.21.15.3:0/212286458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92ac131810 0x7f92ac11c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:05.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.914+0000 7f92b33e5700 1 --2- 172.21.15.3:0/212286458 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f92ac099a40 0x7f92ac11b690 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:05.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.914+0000 7f92b33e5700 1 --2- 172.21.15.3:0/212286458 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f92ac09a3b0 0x7f92ac11bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:05.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.914+0000 7f92b33e5700 1 -- 172.21.15.3:0/212286458 >> 172.21.15.3:0/212286458 conn(0x7f92ac0a2aa0 msgr2=0x7f92ac1b8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:05.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.914+0000 7f92b33e5700 1 -- 172.21.15.3:0/212286458 shutdown_connections 2023-10-03T10:16:05.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:05.914+0000 7f92b33e5700 1 -- 172.21.15.3:0/212286458 wait complete. 2023-10-03T10:16:05.928 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:06.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.011+0000 7ff13d838700 1 -- 172.21.15.3:0/3422233757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff13806c360 con 0x7ff138134100 2023-10-03T10:16:06.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.011+0000 7ff11ffff700 1 -- 172.21.15.3:0/3422233757 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff13002c090 con 0x7ff138134100 2023-10-03T10:16:06.012 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:06.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.013+0000 7ff13d838700 1 -- 172.21.15.3:0/3422233757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff138134100 msgr2=0x7ff138123310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:06.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.013+0000 7ff13d838700 1 --2- 172.21.15.3:0/3422233757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff138134100 0x7ff138123310 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff13000efc0 tx=0x7ff13000c3a0 comp rx=0 tx=0).stop 2023-10-03T10:16:06.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.014+0000 7ff13d838700 1 -- 172.21.15.3:0/3422233757 shutdown_connections 2023-10-03T10:16:06.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.014+0000 7ff13d838700 1 --2- 172.21.15.3:0/3422233757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff138134100 0x7ff138123310 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:06.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.014+0000 7ff13d838700 1 --2- 172.21.15.3:0/3422233757 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff13809d6f0 0x7ff138122890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:06.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.014+0000 7ff13d838700 1 --2- 172.21.15.3:0/3422233757 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff13809e000 0x7ff138122dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:06.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.014+0000 7ff13d838700 1 -- 172.21.15.3:0/3422233757 >> 172.21.15.3:0/3422233757 conn(0x7ff138066f50 msgr2=0x7ff1380a1830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:06.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.014+0000 7ff13d838700 1 -- 172.21.15.3:0/3422233757 shutdown_connections 2023-10-03T10:16:06.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.014+0000 7ff13d838700 1 -- 172.21.15.3:0/3422233757 wait complete. 2023-10-03T10:16:06.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.029+0000 7f9aef63e700 1 -- 172.21.15.3:0/1012156279 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9ae81224b0 con 0x7f9ae80a0a30 2023-10-03T10:16:06.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.029+0000 7f9addffb700 1 -- 172.21.15.3:0/1012156279 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9ad801b3b0 con 0x7f9ae80a0a30 2023-10-03T10:16:06.030 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:06.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.032+0000 7f9aef63e700 1 -- 172.21.15.3:0/1012156279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ae80a0a30 msgr2=0x7f9ae81c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:06.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.032+0000 7f9aef63e700 1 --2- 172.21.15.3:0/1012156279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ae80a0a30 0x7f9ae81c85d0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f9ad800e7e0 tx=0x7f9ad800eaf0 comp rx=0 tx=0).stop 2023-10-03T10:16:06.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.032+0000 7f9aef63e700 1 -- 172.21.15.3:0/1012156279 shutdown_connections 2023-10-03T10:16:06.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.032+0000 7f9aef63e700 1 --2- 172.21.15.3:0/1012156279 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ae8133930 0x7f9ae81c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:06.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.032+0000 7f9aef63e700 1 --2- 172.21.15.3:0/1012156279 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ae809d630 0x7f9ae81c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:06.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.032+0000 7f9aef63e700 1 --2- 172.21.15.3:0/1012156279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ae80a0a30 0x7f9ae81c85d0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:06.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.032+0000 7f9aef63e700 1 -- 172.21.15.3:0/1012156279 >> 172.21.15.3:0/1012156279 conn(0x7f9ae80a3050 msgr2=0x7f9ae809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:06.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.032+0000 7f9aef63e700 1 -- 172.21.15.3:0/1012156279 shutdown_connections 2023-10-03T10:16:06.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.032+0000 7f9aef63e700 1 -- 172.21.15.3:0/1012156279 wait complete. 2023-10-03T10:16:06.034 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:06.046 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:06.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.048+0000 7f7d2f439700 1 -- 172.21.15.3:0/1259619793 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7d2811b240 con 0x7f7d28139170 2023-10-03T10:16:06.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.048+0000 7f7d1dffb700 1 -- 172.21.15.3:0/1259619793 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7d200122e0 con 0x7f7d28139170 2023-10-03T10:16:06.048 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:06.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.051+0000 7f7d2f439700 1 -- 172.21.15.3:0/1259619793 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d28139170 msgr2=0x7f7d281cc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:06.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.051+0000 7f7d2f439700 1 --2- 172.21.15.3:0/1259619793 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d28139170 0x7f7d281cc790 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7d20007d90 tx=0x7f7d2000e3c0 comp rx=0 tx=0).stop 2023-10-03T10:16:06.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.051+0000 7f7d2f439700 1 -- 172.21.15.3:0/1259619793 shutdown_connections 2023-10-03T10:16:06.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.051+0000 7f7d2f439700 1 --2- 172.21.15.3:0/1259619793 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d28139170 0x7f7d281cc790 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:06.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.051+0000 7f7d2f439700 1 --2- 172.21.15.3:0/1259619793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d2809d6f0 0x7f7d28119720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:06.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.051+0000 7f7d2f439700 1 --2- 172.21.15.3:0/1259619793 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d2809e1d0 0x7f7d28119c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:06.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.051+0000 7f7d2f439700 1 -- 172.21.15.3:0/1259619793 >> 172.21.15.3:0/1259619793 conn(0x7f7d28066f50 msgr2=0x7f7d28070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:06.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.051+0000 7f7d2f439700 1 -- 172.21.15.3:0/1259619793 shutdown_connections 2023-10-03T10:16:06.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:06.051+0000 7f7d2f439700 1 -- 172.21.15.3:0/1259619793 wait complete. 2023-10-03T10:16:06.064 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:06.929 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:07.035 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:07.046 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:07.065 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:07.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.076+0000 7fce43a2c700 1 Processor -- start 2023-10-03T10:16:07.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.076+0000 7fce43a2c700 1 -- start start 2023-10-03T10:16:07.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.077+0000 7fce43a2c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce3c09e930 0x7fce3c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:07.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.077+0000 7fce43a2c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce3c09f240 0x7fce3c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:07.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.077+0000 7fce43a2c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fce3c09f650 0x7fce3c131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:07.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.077+0000 7fce43a2c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce3c070940 con 0x7fce3c09f650 2023-10-03T10:16:07.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.077+0000 7fce43a2c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c06fd60 con 0x7fce3c09f240 2023-10-03T10:16:07.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.077+0000 7fce43a2c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c06fa80 con 0x7fce3c09e930 2023-10-03T10:16:07.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.077+0000 7fce40fc7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce3c09f240 0x7fce3c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:07.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.077+0000 7fce417c8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce3c09e930 0x7fce3c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:07.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.077+0000 7fce40fc7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce3c09f240 0x7fce3c1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53162/0 (socket says 172.21.15.3:53162) 2023-10-03T10:16:07.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.077+0000 7fce40fc7700 1 -- 172.21.15.3:0/2414495634 learned_addr learned my addr 172.21.15.3:0/2414495634 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:07.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.077+0000 7fce41fc9700 1 --2- 172.21.15.3:0/2414495634 >> v2:172.21.15.3:3300/0 conn(0x7fce3c09f650 0x7fce3c131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:07.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.077+0000 7fce40fc7700 1 -- 172.21.15.3:0/2414495634 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce3c09e930 msgr2=0x7fce3c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:07.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.077+0000 7fce40fc7700 1 --2- 172.21.15.3:0/2414495634 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce3c09e930 0x7fce3c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.077+0000 7fce40fc7700 1 -- 172.21.15.3:0/2414495634 >> v2:172.21.15.3:3300/0 conn(0x7fce3c09f650 msgr2=0x7fce3c131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:07.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.077+0000 7fce40fc7700 1 --2- 172.21.15.3:0/2414495634 >> v2:172.21.15.3:3300/0 conn(0x7fce3c09f650 0x7fce3c131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.077+0000 7fce40fc7700 1 -- 172.21.15.3:0/2414495634 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3c09fb60 con 0x7fce3c09f240 2023-10-03T10:16:07.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.077+0000 7fce417c8700 1 --2- 172.21.15.3:0/2414495634 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce3c09e930 0x7fce3c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:07.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.077+0000 7fce41fc9700 1 --2- 172.21.15.3:0/2414495634 >> v2:172.21.15.3:3300/0 conn(0x7fce3c09f650 0x7fce3c131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:07.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.078+0000 7fce40fc7700 1 --2- 172.21.15.3:0/2414495634 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce3c09f240 0x7fce3c1317f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0098f0 tx=0x7fce2c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d56db992d7b148e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:07.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.078+0000 7fce33fff700 1 -- 172.21.15.3:0/2414495634 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c011830 con 0x7fce3c09f240 2023-10-03T10:16:07.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.078+0000 7fce33fff700 1 -- 172.21.15.3:0/2414495634 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2c011990 con 0x7fce3c09f240 2023-10-03T10:16:07.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.078+0000 7fce33fff700 1 -- 172.21.15.3:0/2414495634 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c011d20 con 0x7fce3c09f240 2023-10-03T10:16:07.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.079+0000 7fce43a2c700 1 -- 172.21.15.3:0/2414495634 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce3c09f240 msgr2=0x7fce3c1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:07.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.079+0000 7fce43a2c700 1 --2- 172.21.15.3:0/2414495634 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce3c09f240 0x7fce3c1317f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0098f0 tx=0x7fce2c009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:07.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.079+0000 7fce43a2c700 1 -- 172.21.15.3:0/2414495634 shutdown_connections 2023-10-03T10:16:07.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.079+0000 7fce43a2c700 1 --2- 172.21.15.3:0/2414495634 >> v2:172.21.15.3:3300/0 conn(0x7fce3c09f650 0x7fce3c131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.079+0000 7fce43a2c700 1 --2- 172.21.15.3:0/2414495634 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce3c09e930 0x7fce3c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.079+0000 7fce43a2c700 1 --2- 172.21.15.3:0/2414495634 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce3c09f240 0x7fce3c1317f0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.079+0000 7fce43a2c700 1 -- 172.21.15.3:0/2414495634 >> 172.21.15.3:0/2414495634 conn(0x7fce3c0a4210 msgr2=0x7fce3c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:07.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.079+0000 7fce43a2c700 1 -- 172.21.15.3:0/2414495634 shutdown_connections 2023-10-03T10:16:07.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.079+0000 7fce43a2c700 1 -- 172.21.15.3:0/2414495634 wait complete. 2023-10-03T10:16:07.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.080+0000 7fce43a2c700 1 Processor -- start 2023-10-03T10:16:07.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.080+0000 7fce43a2c700 1 -- start start 2023-10-03T10:16:07.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.081+0000 7fce43a2c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce3c09e930 0x7fce3c1a7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:07.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.081+0000 7fce43a2c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce3c09f240 0x7fce3c1a7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:07.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.081+0000 7fce43a2c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce3c09f650 0x7fce3c1a7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:07.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.081+0000 7fce43a2c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c1b8690 con 0x7fce3c09e930 2023-10-03T10:16:07.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.081+0000 7fce43a2c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c1a9d60 con 0x7fce3c09f650 2023-10-03T10:16:07.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.081+0000 7fce43a2c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c1aa840 con 0x7fce3c09f240 2023-10-03T10:16:07.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.081+0000 7fce417c8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce3c09e930 0x7fce3c1a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:07.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.081+0000 7fce417c8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce3c09e930 0x7fce3c1a7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53060/0 (socket says 172.21.15.3:53060) 2023-10-03T10:16:07.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.081+0000 7fce417c8700 1 -- 172.21.15.3:0/2460260869 learned_addr learned my addr 172.21.15.3:0/2460260869 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:07.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.081+0000 7fce40fc7700 1 --2- 172.21.15.3:0/2460260869 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce3c09f240 0x7fce3c1a7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:07.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.081+0000 7fce41fc9700 1 --2- 172.21.15.3:0/2460260869 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce3c09f650 0x7fce3c1a7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:07.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.082+0000 7fce40fc7700 1 -- 172.21.15.3:0/2460260869 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce3c09f650 msgr2=0x7fce3c1a7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:07.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.082+0000 7fce40fc7700 1 --2- 172.21.15.3:0/2460260869 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce3c09f650 0x7fce3c1a7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.082+0000 7fce40fc7700 1 -- 172.21.15.3:0/2460260869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce3c09e930 msgr2=0x7fce3c1a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:07.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.082+0000 7fce40fc7700 1 --2- 172.21.15.3:0/2460260869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce3c09e930 0x7fce3c1a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.082+0000 7fce40fc7700 1 -- 172.21.15.3:0/2460260869 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce380094a0 con 0x7fce3c09f240 2023-10-03T10:16:07.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.082+0000 7fce40fc7700 1 --2- 172.21.15.3:0/2460260869 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce3c09f240 0x7fce3c1a7890 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fce2c016000 tx=0x7fce2c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:07.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.082+0000 7fce31ffb700 1 -- 172.21.15.3:0/2460260869 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c004240 con 0x7fce3c09f240 2023-10-03T10:16:07.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.082+0000 7fce31ffb700 1 -- 172.21.15.3:0/2460260869 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2c01f070 con 0x7fce3c09f240 2023-10-03T10:16:07.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.082+0000 7fce43a2c700 1 -- 172.21.15.3:0/2460260869 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce34009a70 con 0x7fce3c09f240 2023-10-03T10:16:07.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.082+0000 7fce31ffb700 1 -- 172.21.15.3:0/2460260869 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c0036a0 con 0x7fce3c09f240 2023-10-03T10:16:07.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.082+0000 7fce43a2c700 1 -- 172.21.15.3:0/2460260869 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce2c009550 con 0x7fce3c09f240 2023-10-03T10:16:07.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.083+0000 7fce31ffb700 1 -- 172.21.15.3:0/2460260869 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fce2c021030 con 0x7fce3c09f240 2023-10-03T10:16:07.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.084+0000 7fce31ffb700 1 -- 172.21.15.3:0/2460260869 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fce2c040a90 con 0x7fce3c09f240 2023-10-03T10:16:07.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.084+0000 7fce43a2c700 1 -- 172.21.15.3:0/2460260869 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce3c071d20 con 0x7fce3c09f240 2023-10-03T10:16:07.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.091+0000 7fce31ffb700 1 -- 172.21.15.3:0/2460260869 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fce2c0199a0 con 0x7fce3c09f240 2023-10-03T10:16:07.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.191+0000 7f706d669700 1 Processor -- start 2023-10-03T10:16:07.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.191+0000 7f706d669700 1 -- start start 2023-10-03T10:16:07.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.192+0000 7f706d669700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f706809e1d0 0x7f7068138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:07.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.192+0000 7f706d669700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7068139170 0x7f706813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:07.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.192+0000 7f706d669700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f706809b120 con 0x7f706809d6f0 2023-10-03T10:16:07.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.192+0000 7f706d669700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706806b460 con 0x7f706809e1d0 2023-10-03T10:16:07.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.192+0000 7f706d669700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f706809a7e0 con 0x7f7068139170 2023-10-03T10:16:07.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.192+0000 7f70677fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7068139170 0x7f706813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:07.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.192+0000 7f7066ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f706809d6f0 0x7f706809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38924/0 (socket says 172.21.15.3:38924) 2023-10-03T10:16:07.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.192+0000 7f70677fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7068139170 0x7f706813b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52254/0 (socket says 172.21.15.3:52254) 2023-10-03T10:16:07.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.192+0000 7f7066ffd700 1 -- 172.21.15.3:0/1112070913 learned_addr learned my addr 172.21.15.3:0/1112070913 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:07.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.192+0000 7f70667fc700 1 --2- 172.21.15.3:0/1112070913 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f706809e1d0 0x7f7068138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:07.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.192+0000 7f7065ffb700 1 -- 172.21.15.3:0/1112070913 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3623872747 0 0) 0x7f706809b120 con 0x7f706809d6f0 2023-10-03T10:16:07.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.192+0000 7f7065ffb700 1 -- 172.21.15.3:0/1112070913 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70500035b0 con 0x7f706809d6f0 2023-10-03T10:16:07.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.192+0000 7f70677fe700 1 -- 172.21.15.3:0/1112070913 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f706809e1d0 msgr2=0x7f7068138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:07.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.192+0000 7f70677fe700 1 --2- 172.21.15.3:0/1112070913 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f706809e1d0 0x7f7068138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.192+0000 7f70677fe700 1 -- 172.21.15.3:0/1112070913 >> v1:172.21.15.3:6789/0 conn(0x7f706809d6f0 legacy=0x7f706809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:07.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.192+0000 7f70677fe700 1 -- 172.21.15.3:0/1112070913 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706809a500 con 0x7f7068139170 2023-10-03T10:16:07.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.193+0000 7f70677fe700 1 --2- 172.21.15.3:0/1112070913 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7068139170 0x7f706813b550 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7060009df0 tx=0x7f706000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2357fa1c3c6b47a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:07.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.193+0000 7f7065ffb700 1 -- 172.21.15.3:0/1112070913 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706000e070 con 0x7f7068139170 2023-10-03T10:16:07.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.193+0000 7f7065ffb700 1 -- 172.21.15.3:0/1112070913 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706000baa0 con 0x7f7068139170 2023-10-03T10:16:07.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.193+0000 7f7065ffb700 1 -- 172.21.15.3:0/1112070913 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706000be10 con 0x7f7068139170 2023-10-03T10:16:07.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.194+0000 7f706d669700 1 -- 172.21.15.3:0/1112070913 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7068139170 msgr2=0x7f706813b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:07.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.194+0000 7f706d669700 1 --2- 172.21.15.3:0/1112070913 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7068139170 0x7f706813b550 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7060009df0 tx=0x7f706000b330 comp rx=0 tx=0).stop 2023-10-03T10:16:07.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.194+0000 7f706d669700 1 -- 172.21.15.3:0/1112070913 shutdown_connections 2023-10-03T10:16:07.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.194+0000 7f706d669700 1 --2- 172.21.15.3:0/1112070913 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7068139170 0x7f706813b550 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.194+0000 7f706d669700 1 --2- 172.21.15.3:0/1112070913 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f706809e1d0 0x7f7068138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.194+0000 7f706d669700 1 -- 172.21.15.3:0/1112070913 >> 172.21.15.3:0/1112070913 conn(0x7f7068066f50 msgr2=0x7f706809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:07.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.194+0000 7f706d669700 1 -- 172.21.15.3:0/1112070913 shutdown_connections 2023-10-03T10:16:07.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.194+0000 7f706d669700 1 -- 172.21.15.3:0/1112070913 wait complete. 2023-10-03T10:16:07.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.196+0000 7f706d669700 1 Processor -- start 2023-10-03T10:16:07.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.196+0000 7f706d669700 1 -- start start 2023-10-03T10:16:07.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.197+0000 7f706d669700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f706809d6f0 0x7f7068120660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:07.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.197+0000 7f706d669700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f706809e1d0 0x7f7068120ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:07.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.197+0000 7f706d669700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7068139170 0x7f70681210e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:07.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.197+0000 7f706d669700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7068124d60 con 0x7f706809e1d0 2023-10-03T10:16:07.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.197+0000 7f706d669700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7068123530 con 0x7f7068139170 2023-10-03T10:16:07.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.197+0000 7f706d669700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f706809c710 con 0x7f706809d6f0 2023-10-03T10:16:07.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.197+0000 7f70667fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f706809e1d0 0x7f7068120ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:07.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.197+0000 7f70667fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f706809e1d0 0x7f7068120ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53072/0 (socket says 172.21.15.3:53072) 2023-10-03T10:16:07.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.197+0000 7f70667fc700 1 -- 172.21.15.3:0/4219883850 learned_addr learned my addr 172.21.15.3:0/4219883850 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:07.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.197+0000 7f7066ffd700 1 --2- 172.21.15.3:0/4219883850 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f706809d6f0 0x7f7068120660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:07.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.197+0000 7f70677fe700 1 --2- 172.21.15.3:0/4219883850 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7068139170 0x7f70681210e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:07.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.197+0000 7f70667fc700 1 -- 172.21.15.3:0/4219883850 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f706809d6f0 msgr2=0x7f7068120660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:07.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.197+0000 7f70667fc700 1 --2- 172.21.15.3:0/4219883850 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f706809d6f0 0x7f7068120660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.197+0000 7f70667fc700 1 -- 172.21.15.3:0/4219883850 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7068139170 msgr2=0x7f70681210e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:07.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.197+0000 7f70667fc700 1 --2- 172.21.15.3:0/4219883850 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7068139170 0x7f70681210e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.197+0000 7f70667fc700 1 -- 172.21.15.3:0/4219883850 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7060009a70 con 0x7f706809e1d0 2023-10-03T10:16:07.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.198+0000 7f70667fc700 1 --2- 172.21.15.3:0/4219883850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f706809e1d0 0x7f7068120ba0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f705c00e9b0 tx=0x7f705c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:07.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.198+0000 7f704f7fe700 1 -- 172.21.15.3:0/4219883850 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c009710 con 0x7f706809e1d0 2023-10-03T10:16:07.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.198+0000 7f704f7fe700 1 -- 172.21.15.3:0/4219883850 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705c00ee50 con 0x7f706809e1d0 2023-10-03T10:16:07.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.198+0000 7f706d669700 1 -- 172.21.15.3:0/4219883850 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7068123be0 con 0x7f706809e1d0 2023-10-03T10:16:07.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.198+0000 7f704f7fe700 1 -- 172.21.15.3:0/4219883850 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c00ca50 con 0x7f706809e1d0 2023-10-03T10:16:07.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.198+0000 7f706d669700 1 -- 172.21.15.3:0/4219883850 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7068124910 con 0x7f706809e1d0 2023-10-03T10:16:07.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.199+0000 7f704f7fe700 1 -- 172.21.15.3:0/4219883850 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f705c00cbb0 con 0x7f706809e1d0 2023-10-03T10:16:07.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.199+0000 7f704f7fe700 1 -- 172.21.15.3:0/4219883850 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f705c03f720 con 0x7f706809e1d0 2023-10-03T10:16:07.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.200+0000 7f706d669700 1 -- 172.21.15.3:0/4219883850 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7068074ea0 con 0x7f706809e1d0 2023-10-03T10:16:07.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.205+0000 7f704f7fe700 1 -- 172.21.15.3:0/4219883850 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f705c0232f0 con 0x7f706809e1d0 2023-10-03T10:16:07.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.234+0000 7fe6bab4d700 1 Processor -- start 2023-10-03T10:16:07.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.234+0000 7fe6bab4d700 1 -- start start 2023-10-03T10:16:07.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.234+0000 7fe6bab4d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe6b4066410 0x7fe6b406e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:07.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.234+0000 7fe6bab4d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6b406efa0 0x7fe6b412fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:07.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.234+0000 7fe6bab4d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b4099b50 con 0x7fe6b406eb80 2023-10-03T10:16:07.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.234+0000 7fe6bab4d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b40667e0 con 0x7fe6b4066410 2023-10-03T10:16:07.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.234+0000 7fe6bab4d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b406f3e0 con 0x7fe6b406efa0 2023-10-03T10:16:07.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.235+0000 7fe6b3fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe6b406eb80 0x7fe6b412f320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38934/0 (socket says 172.21.15.3:38934) 2023-10-03T10:16:07.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.235+0000 7fe6b3fff700 1 -- 172.21.15.3:0/1537555536 learned_addr learned my addr 172.21.15.3:0/1537555536 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:07.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.235+0000 7fe6b90ea700 1 --2- 172.21.15.3:0/1537555536 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6b406efa0 0x7fe6b412fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:07.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.235+0000 7fe6b88e9700 1 --2- 172.21.15.3:0/1537555536 >> v2:172.21.15.3:3300/0 conn(0x7fe6b4066410 0x7fe6b406e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:07.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.235+0000 7fe6b90ea700 1 -- 172.21.15.3:0/1537555536 >> v2:172.21.15.3:3300/0 conn(0x7fe6b4066410 msgr2=0x7fe6b406e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:07.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.235+0000 7fe6b90ea700 1 --2- 172.21.15.3:0/1537555536 >> v2:172.21.15.3:3300/0 conn(0x7fe6b4066410 0x7fe6b406e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.235+0000 7fe6b90ea700 1 -- 172.21.15.3:0/1537555536 >> v1:172.21.15.3:6789/0 conn(0x7fe6b406eb80 legacy=0x7fe6b412f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:07.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.235+0000 7fe6b90ea700 1 -- 172.21.15.3:0/1537555536 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b4099f60 con 0x7fe6b406efa0 2023-10-03T10:16:07.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.235+0000 7fe6b88e9700 1 --2- 172.21.15.3:0/1537555536 >> v2:172.21.15.3:3300/0 conn(0x7fe6b4066410 0x7fe6b406e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:07.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.235+0000 7fe6b90ea700 1 --2- 172.21.15.3:0/1537555536 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6b406efa0 0x7fe6b412fa30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac009d90 tx=0x7fe6ac00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73412c93aaa9224 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:07.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.236+0000 7fabe20f8700 1 Processor -- start 2023-10-03T10:16:07.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.236+0000 7fabe20f8700 1 -- start start 2023-10-03T10:16:07.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.236+0000 7fabe20f8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fabdc12c9a0 0x7fabdc12cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:07.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.236+0000 7fabe20f8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fabdc130380 0x7fabdc137c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:07.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.236+0000 7fabe20f8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabdc09a420 con 0x7fabdc12f880 2023-10-03T10:16:07.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.236+0000 7fabe20f8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc09b890 con 0x7fabdc130380 2023-10-03T10:16:07.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.236+0000 7fabe20f8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc09b2f0 con 0x7fabdc12c9a0 2023-10-03T10:16:07.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.236+0000 7fabdb7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fabdc12c9a0 0x7fabdc12cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:07.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.236+0000 7fabdb7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fabdc12c9a0 0x7fabdc12cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52270/0 (socket says 172.21.15.3:52270) 2023-10-03T10:16:07.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.237+0000 7fabdb7fe700 1 -- 172.21.15.3:0/1423929656 learned_addr learned my addr 172.21.15.3:0/1423929656 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:07.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.236+0000 7fe6b37fe700 1 -- 172.21.15.3:0/1537555536 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac00e070 con 0x7fe6b406efa0 2023-10-03T10:16:07.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.237+0000 7fe6b37fe700 1 -- 172.21.15.3:0/1537555536 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ac00bb20 con 0x7fe6b406efa0 2023-10-03T10:16:07.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.237+0000 7fe6b37fe700 1 -- 172.21.15.3:0/1537555536 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac012430 con 0x7fe6b406efa0 2023-10-03T10:16:07.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.237+0000 7fabdbfff700 1 --2- 172.21.15.3:0/1423929656 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fabdc130380 0x7fabdc137c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.237+0000 7fabdb7fe700 1 -- 172.21.15.3:0/1423929656 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fabdc130380 msgr2=0x7fabdc137c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.237+0000 7fabdb7fe700 1 --2- 172.21.15.3:0/1423929656 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fabdc130380 0x7fabdc137c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.237+0000 7fabdb7fe700 1 -- 172.21.15.3:0/1423929656 >> v1:172.21.15.3:6789/0 conn(0x7fabdc12f880 legacy=0x7fabdc12fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.237+0000 7fabdb7fe700 1 -- 172.21.15.3:0/1423929656 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabdc09a870 con 0x7fabdc12c9a0 2023-10-03T10:16:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.237+0000 7fabda7fc700 1 -- 172.21.15.3:0/1423929656 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3137409319 0 0) 0x7fabdc09a420 con 0x7fabdc12f880 2023-10-03T10:16:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.237+0000 7fabdb7fe700 1 --2- 172.21.15.3:0/1423929656 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fabdc12c9a0 0x7fabdc12cd70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00bbc0 tx=0x7fabcc00bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dea6206cdea09b61 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.237+0000 7fabda7fc700 1 -- 172.21.15.3:0/1423929656 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc00dc80 con 0x7fabdc12c9a0 2023-10-03T10:16:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.237+0000 7fabda7fc700 1 -- 172.21.15.3:0/1423929656 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabcc00dde0 con 0x7fabdc12c9a0 2023-10-03T10:16:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.237+0000 7fabda7fc700 1 -- 172.21.15.3:0/1423929656 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc017630 con 0x7fabdc12c9a0 2023-10-03T10:16:07.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.238+0000 7fe6bab4d700 1 -- 172.21.15.3:0/1537555536 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6b406efa0 msgr2=0x7fe6b412fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:07.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.238+0000 7fe6bab4d700 1 --2- 172.21.15.3:0/1537555536 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6b406efa0 0x7fe6b412fa30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac009d90 tx=0x7fe6ac00b330 comp rx=0 tx=0).stop 2023-10-03T10:16:07.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.238+0000 7fe6bab4d700 1 -- 172.21.15.3:0/1537555536 shutdown_connections 2023-10-03T10:16:07.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.238+0000 7fe6bab4d700 1 --2- 172.21.15.3:0/1537555536 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6b406efa0 0x7fe6b412fa30 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.238+0000 7fe6bab4d700 1 --2- 172.21.15.3:0/1537555536 >> v2:172.21.15.3:3300/0 conn(0x7fe6b4066410 0x7fe6b406e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.238+0000 7fe6bab4d700 1 -- 172.21.15.3:0/1537555536 >> 172.21.15.3:0/1537555536 conn(0x7fe6b40a2aa0 msgr2=0x7fe6b40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:07.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.238+0000 7fe6bab4d700 1 -- 172.21.15.3:0/1537555536 shutdown_connections 2023-10-03T10:16:07.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.238+0000 7fe6bab4d700 1 -- 172.21.15.3:0/1537555536 wait complete. 2023-10-03T10:16:07.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.238+0000 7fabe20f8700 1 -- 172.21.15.3:0/1423929656 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fabdc12c9a0 msgr2=0x7fabdc12cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:07.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.238+0000 7fabe20f8700 1 --2- 172.21.15.3:0/1423929656 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fabdc12c9a0 0x7fabdc12cd70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00bbc0 tx=0x7fabcc00bed0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.239+0000 7fabe20f8700 1 -- 172.21.15.3:0/1423929656 shutdown_connections 2023-10-03T10:16:07.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.239+0000 7fabe20f8700 1 --2- 172.21.15.3:0/1423929656 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fabdc130380 0x7fabdc137c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.239+0000 7fabe20f8700 1 --2- 172.21.15.3:0/1423929656 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fabdc12c9a0 0x7fabdc12cd70 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.239+0000 7fabe20f8700 1 -- 172.21.15.3:0/1423929656 >> 172.21.15.3:0/1423929656 conn(0x7fabdc0a2360 msgr2=0x7fabdc0a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:07.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.239+0000 7fe6bab4d700 1 Processor -- start 2023-10-03T10:16:07.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.239+0000 7fabe20f8700 1 -- 172.21.15.3:0/1423929656 shutdown_connections 2023-10-03T10:16:07.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.239+0000 7fe6bab4d700 1 -- start start 2023-10-03T10:16:07.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.239+0000 7fabe20f8700 1 -- 172.21.15.3:0/1423929656 wait complete. 2023-10-03T10:16:07.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.240+0000 7fe6bab4d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6b4066410 0x7fe6b41ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:07.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.240+0000 7fe6bab4d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6b406eb80 0x7fe6b41cd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:07.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.240+0000 7fe6bab4d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe6b406efa0 0x7fe6b4118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:07.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.240+0000 7fe6bab4d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b409a180 con 0x7fe6b4066410 2023-10-03T10:16:07.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.240+0000 7fe6bab4d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b411a660 con 0x7fe6b406eb80 2023-10-03T10:16:07.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.240+0000 7fe6bab4d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b411aac0 con 0x7fe6b406efa0 2023-10-03T10:16:07.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.240+0000 7fe6b88e9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6b4066410 0x7fe6b41ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:07.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.240+0000 7fe6b88e9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6b4066410 0x7fe6b41ccbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53102/0 (socket says 172.21.15.3:53102) 2023-10-03T10:16:07.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.240+0000 7fe6b88e9700 1 -- 172.21.15.3:0/3995132348 learned_addr learned my addr 172.21.15.3:0/3995132348 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:07.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.240+0000 7fe6b90ea700 1 --2- 172.21.15.3:0/3995132348 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe6b406efa0 0x7fe6b4118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:07.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.240+0000 7fe6b88e9700 1 -- 172.21.15.3:0/3995132348 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe6b406efa0 msgr2=0x7fe6b4118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:07.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.240+0000 7fe6b88e9700 1 --2- 172.21.15.3:0/3995132348 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe6b406efa0 0x7fe6b4118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.240+0000 7fe6b88e9700 1 -- 172.21.15.3:0/3995132348 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6b406eb80 msgr2=0x7fe6b41cd120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:16:07.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.240+0000 7fe6b88e9700 1 --2- 172.21.15.3:0/3995132348 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6b406eb80 0x7fe6b41cd120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.240+0000 7fe6b88e9700 1 -- 172.21.15.3:0/3995132348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a80094a0 con 0x7fe6b4066410 2023-10-03T10:16:07.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.241+0000 7fe6b88e9700 1 --2- 172.21.15.3:0/3995132348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6b4066410 0x7fe6b41ccbe0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800ebe0 tx=0x7fe6a800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:07.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.241+0000 7fabe20f8700 1 Processor -- start 2023-10-03T10:16:07.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.241+0000 7fabe20f8700 1 -- start start 2023-10-03T10:16:07.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.241+0000 7fe6b17fa700 1 -- 172.21.15.3:0/3995132348 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a800c850 con 0x7fe6b4066410 2023-10-03T10:16:07.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.241+0000 7fe6b17fa700 1 -- 172.21.15.3:0/3995132348 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a800c9b0 con 0x7fe6b4066410 2023-10-03T10:16:07.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.241+0000 7fe6b17fa700 1 -- 172.21.15.3:0/3995132348 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a800cd40 con 0x7fe6b4066410 2023-10-03T10:16:07.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.241+0000 7fe6bab4d700 1 -- 172.21.15.3:0/3995132348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6ac009a70 con 0x7fe6b4066410 2023-10-03T10:16:07.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.241+0000 7fe6bab4d700 1 -- 172.21.15.3:0/3995132348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b41cd660 con 0x7fe6b4066410 2023-10-03T10:16:07.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.242+0000 7fabe20f8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabdc12c9a0 0x7fabdc1c4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:07.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.242+0000 7fabe20f8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fabdc12f880 0x7fabdc1c54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:07.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.242+0000 7fabe20f8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fabdc130380 0x7fabdc1c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:07.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.242+0000 7fabe20f8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc08bf40 con 0x7fabdc12c9a0 2023-10-03T10:16:07.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.242+0000 7fabe20f8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc08c120 con 0x7fabdc12f880 2023-10-03T10:16:07.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.242+0000 7fabe20f8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc12c440 con 0x7fabdc130380 2023-10-03T10:16:07.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.242+0000 7fabdb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabdc12c9a0 0x7fabdc1c4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:07.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.242+0000 7fabdb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabdc12c9a0 0x7fabdc1c4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53106/0 (socket says 172.21.15.3:53106) 2023-10-03T10:16:07.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.242+0000 7fabdb7fe700 1 -- 172.21.15.3:0/3708524351 learned_addr learned my addr 172.21.15.3:0/3708524351 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:07.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.242+0000 7fabdb7fe700 1 -- 172.21.15.3:0/3708524351 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fabdc130380 msgr2=0x7fabdc1c80f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:16:07.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.242+0000 7fabdb7fe700 1 --2- 172.21.15.3:0/3708524351 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fabdc130380 0x7fabdc1c80f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.242+0000 7fabdb7fe700 1 -- 172.21.15.3:0/3708524351 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fabdc12f880 msgr2=0x7fabdc1c54f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:16:07.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.242+0000 7fabdb7fe700 1 --2- 172.21.15.3:0/3708524351 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fabdc12f880 0x7fabdc1c54f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.242+0000 7fabdb7fe700 1 -- 172.21.15.3:0/3708524351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabcc00b7f0 con 0x7fabdc12c9a0 2023-10-03T10:16:07.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.242+0000 7fabdb7fe700 1 --2- 172.21.15.3:0/3708524351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabdc12c9a0 0x7fabdc1c4fb0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fabdc09b430 tx=0x7fabcc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:07.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.243+0000 7fabc3fff700 1 -- 172.21.15.3:0/3708524351 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc009050 con 0x7fabdc12c9a0 2023-10-03T10:16:07.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.243+0000 7fabc3fff700 1 -- 172.21.15.3:0/3708524351 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabcc00a040 con 0x7fabdc12c9a0 2023-10-03T10:16:07.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.243+0000 7fabc3fff700 1 -- 172.21.15.3:0/3708524351 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc021850 con 0x7fabdc12c9a0 2023-10-03T10:16:07.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.242+0000 7fe6b17fa700 1 -- 172.21.15.3:0/3995132348 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe6a8013470 con 0x7fe6b4066410 2023-10-03T10:16:07.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.243+0000 7fe6b17fa700 1 -- 172.21.15.3:0/3995132348 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe6a803f7d0 con 0x7fe6b4066410 2023-10-03T10:16:07.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.243+0000 7fe6bab4d700 1 -- 172.21.15.3:0/3995132348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6940052e0 con 0x7fe6b4066410 2023-10-03T10:16:07.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.243+0000 7fabe20f8700 1 -- 172.21.15.3:0/3708524351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabdc09c5f0 con 0x7fabdc12c9a0 2023-10-03T10:16:07.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.244+0000 7fabe20f8700 1 -- 172.21.15.3:0/3708524351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabdc09ad80 con 0x7fabdc12c9a0 2023-10-03T10:16:07.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.245+0000 7fabe20f8700 1 -- 172.21.15.3:0/3708524351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabdc074ea0 con 0x7fabdc12c9a0 2023-10-03T10:16:07.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.248+0000 7fe6b17fa700 1 -- 172.21.15.3:0/3995132348 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe6a8013720 con 0x7fe6b4066410 2023-10-03T10:16:07.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.249+0000 7fabc3fff700 1 -- 172.21.15.3:0/3708524351 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fabcc015070 con 0x7fabdc12c9a0 2023-10-03T10:16:07.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.249+0000 7fabc3fff700 1 -- 172.21.15.3:0/3708524351 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fabcc041980 con 0x7fabdc12c9a0 2023-10-03T10:16:07.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.253+0000 7fabc3fff700 1 -- 172.21.15.3:0/3708524351 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fabcc006c60 con 0x7fabdc12c9a0 2023-10-03T10:16:07.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.334+0000 7fce43a2c700 1 -- 172.21.15.3:0/2460260869 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fce3c074ea0 con 0x7fce3c09f240 2023-10-03T10:16:07.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.335+0000 7fce31ffb700 1 -- 172.21.15.3:0/2460260869 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fce2c0193b0 con 0x7fce3c09f240 2023-10-03T10:16:07.335 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:07.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.337+0000 7fce43a2c700 1 -- 172.21.15.3:0/2460260869 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce3c09f240 msgr2=0x7fce3c1a7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:07.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.337+0000 7fce43a2c700 1 --2- 172.21.15.3:0/2460260869 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce3c09f240 0x7fce3c1a7890 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fce2c016000 tx=0x7fce2c004750 comp rx=0 tx=0).stop 2023-10-03T10:16:07.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.337+0000 7fce43a2c700 1 -- 172.21.15.3:0/2460260869 shutdown_connections 2023-10-03T10:16:07.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.337+0000 7fce43a2c700 1 --2- 172.21.15.3:0/2460260869 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce3c09f650 0x7fce3c1a7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.337+0000 7fce43a2c700 1 --2- 172.21.15.3:0/2460260869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce3c09e930 0x7fce3c1a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.337+0000 7fce43a2c700 1 --2- 172.21.15.3:0/2460260869 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fce3c09f240 0x7fce3c1a7890 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.337+0000 7fce43a2c700 1 -- 172.21.15.3:0/2460260869 >> 172.21.15.3:0/2460260869 conn(0x7fce3c0a4210 msgr2=0x7fce3c06b880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:07.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.337+0000 7fce43a2c700 1 -- 172.21.15.3:0/2460260869 shutdown_connections 2023-10-03T10:16:07.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.337+0000 7fce43a2c700 1 -- 172.21.15.3:0/2460260869 wait complete. 2023-10-03T10:16:07.352 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:07.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.422+0000 7f706d669700 1 -- 172.21.15.3:0/4219883850 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7068136890 con 0x7f706809e1d0 2023-10-03T10:16:07.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.423+0000 7f704f7fe700 1 -- 172.21.15.3:0/4219883850 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f705c01ae80 con 0x7f706809e1d0 2023-10-03T10:16:07.423 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:07.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.425+0000 7f706d669700 1 -- 172.21.15.3:0/4219883850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f706809e1d0 msgr2=0x7f7068120ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:07.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.425+0000 7f706d669700 1 --2- 172.21.15.3:0/4219883850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f706809e1d0 0x7f7068120ba0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f705c00e9b0 tx=0x7f705c00ecc0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.425+0000 7f706d669700 1 -- 172.21.15.3:0/4219883850 shutdown_connections 2023-10-03T10:16:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.425+0000 7f706d669700 1 --2- 172.21.15.3:0/4219883850 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7068139170 0x7f70681210e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.425+0000 7f706d669700 1 --2- 172.21.15.3:0/4219883850 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f706809d6f0 0x7f7068120660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.425+0000 7f706d669700 1 --2- 172.21.15.3:0/4219883850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f706809e1d0 0x7f7068120ba0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.425+0000 7f706d669700 1 -- 172.21.15.3:0/4219883850 >> 172.21.15.3:0/4219883850 conn(0x7f7068066f50 msgr2=0x7f7068070c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.425+0000 7f706d669700 1 -- 172.21.15.3:0/4219883850 shutdown_connections 2023-10-03T10:16:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.425+0000 7f706d669700 1 -- 172.21.15.3:0/4219883850 wait complete. 2023-10-03T10:16:07.439 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:07.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.461+0000 7fabe20f8700 1 -- 172.21.15.3:0/3708524351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fabdc12c050 con 0x7fabdc12c9a0 2023-10-03T10:16:07.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.461+0000 7fabc3fff700 1 -- 172.21.15.3:0/3708524351 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fabcc032380 con 0x7fabdc12c9a0 2023-10-03T10:16:07.461 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:07.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.463+0000 7fabe20f8700 1 -- 172.21.15.3:0/3708524351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabdc12c9a0 msgr2=0x7fabdc1c4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:07.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.463+0000 7fabe20f8700 1 --2- 172.21.15.3:0/3708524351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabdc12c9a0 0x7fabdc1c4fb0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fabdc09b430 tx=0x7fabcc004750 comp rx=0 tx=0).stop 2023-10-03T10:16:07.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.463+0000 7fabe20f8700 1 -- 172.21.15.3:0/3708524351 shutdown_connections 2023-10-03T10:16:07.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.463+0000 7fabe20f8700 1 --2- 172.21.15.3:0/3708524351 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fabdc130380 0x7fabdc1c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.463+0000 7fabe20f8700 1 --2- 172.21.15.3:0/3708524351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabdc12c9a0 0x7fabdc1c4fb0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.463+0000 7fabe20f8700 1 --2- 172.21.15.3:0/3708524351 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fabdc12f880 0x7fabdc1c54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.463+0000 7fabe20f8700 1 -- 172.21.15.3:0/3708524351 >> 172.21.15.3:0/3708524351 conn(0x7fabdc0a2360 msgr2=0x7fabdc1b58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:07.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.463+0000 7fabe20f8700 1 -- 172.21.15.3:0/3708524351 shutdown_connections 2023-10-03T10:16:07.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.463+0000 7fabe20f8700 1 -- 172.21.15.3:0/3708524351 wait complete. 2023-10-03T10:16:07.477 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:07.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.477+0000 7fe6bab4d700 1 -- 172.21.15.3:0/3995132348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe694005150 con 0x7fe6b4066410 2023-10-03T10:16:07.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.477+0000 7fe6b17fa700 1 -- 172.21.15.3:0/3995132348 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe6a8021020 con 0x7fe6b4066410 2023-10-03T10:16:07.478 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:07.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.479+0000 7fe6bab4d700 1 -- 172.21.15.3:0/3995132348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6b4066410 msgr2=0x7fe6b41ccbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:07.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.479+0000 7fe6bab4d700 1 --2- 172.21.15.3:0/3995132348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6b4066410 0x7fe6b41ccbe0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800ebe0 tx=0x7fe6a800eef0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.479+0000 7fe6bab4d700 1 -- 172.21.15.3:0/3995132348 shutdown_connections 2023-10-03T10:16:07.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.479+0000 7fe6bab4d700 1 --2- 172.21.15.3:0/3995132348 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe6b406efa0 0x7fe6b4118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.479+0000 7fe6bab4d700 1 --2- 172.21.15.3:0/3995132348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6b4066410 0x7fe6b41ccbe0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.479+0000 7fe6bab4d700 1 --2- 172.21.15.3:0/3995132348 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6b406eb80 0x7fe6b41cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:07.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.479+0000 7fe6bab4d700 1 -- 172.21.15.3:0/3995132348 >> 172.21.15.3:0/3995132348 conn(0x7fe6b40a2aa0 msgr2=0x7fe6b412e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:07.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.479+0000 7fe6bab4d700 1 -- 172.21.15.3:0/3995132348 shutdown_connections 2023-10-03T10:16:07.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:07.479+0000 7fe6bab4d700 1 -- 172.21.15.3:0/3995132348 wait complete. 2023-10-03T10:16:07.493 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:08.352 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:08.439 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:08.478 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:08.494 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:08.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.499+0000 7f0dd116d700 1 Processor -- start 2023-10-03T10:16:08.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.499+0000 7f0dd116d700 1 -- start start 2023-10-03T10:16:08.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.500+0000 7f0dd116d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dcc130540 0x7f0dcc130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:08.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.500+0000 7f0dd116d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0dcc06fcf0 0x7f0dcc138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:08.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.500+0000 7f0dd116d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dcc0716a0 con 0x7f0dcc130e50 2023-10-03T10:16:08.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.500+0000 7f0dd116d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dcc09a390 con 0x7f0dcc06fcf0 2023-10-03T10:16:08.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.500+0000 7f0dd116d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc071290 con 0x7f0dcc130540 2023-10-03T10:16:08.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.500+0000 7f0dcb59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0dcc06fcf0 0x7f0dcc138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:08.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.500+0000 7f0dcb59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0dcc06fcf0 0x7f0dcc138570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53114/0 (socket says 172.21.15.3:53114) 2023-10-03T10:16:08.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.500+0000 7f0dcb59e700 1 -- 172.21.15.3:0/2722906423 learned_addr learned my addr 172.21.15.3:0/2722906423 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:08.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.500+0000 7f0dcad9d700 1 --2- 172.21.15.3:0/2722906423 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dcc130540 0x7f0dcc130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.500+0000 7f0dcb59e700 1 -- 172.21.15.3:0/2722906423 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dcc130540 msgr2=0x7f0dcc130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.500+0000 7f0dcb59e700 1 --2- 172.21.15.3:0/2722906423 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dcc130540 0x7f0dcc130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.500+0000 7f0dcb59e700 1 -- 172.21.15.3:0/2722906423 >> v1:172.21.15.3:6789/0 conn(0x7f0dcc130e50 legacy=0x7f0dcc06f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.500+0000 7f0dcb59e700 1 -- 172.21.15.3:0/2722906423 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dcc130340 con 0x7f0dcc06fcf0 2023-10-03T10:16:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.501+0000 7f0dcb59e700 1 --2- 172.21.15.3:0/2722906423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dcc06fcf0 0x7f0dcc138570 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009e50 tx=0x7f0dc400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c3e2c029aa2d96 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.501+0000 7f0dc9d9b700 1 -- 172.21.15.3:0/2722906423 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc400e070 con 0x7f0dcc06fcf0 2023-10-03T10:16:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.501+0000 7f0dc9d9b700 1 -- 172.21.15.3:0/2722906423 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc400b910 con 0x7f0dcc06fcf0 2023-10-03T10:16:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.501+0000 7f0dc9d9b700 1 -- 172.21.15.3:0/2722906423 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc400bcc0 con 0x7f0dcc06fcf0 2023-10-03T10:16:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.502+0000 7f0dd116d700 1 -- 172.21.15.3:0/2722906423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dcc06fcf0 msgr2=0x7f0dcc138570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.502+0000 7f0dd116d700 1 --2- 172.21.15.3:0/2722906423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dcc06fcf0 0x7f0dcc138570 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009e50 tx=0x7f0dc400b330 comp rx=0 tx=0).stop 2023-10-03T10:16:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.502+0000 7f0dd116d700 1 -- 172.21.15.3:0/2722906423 shutdown_connections 2023-10-03T10:16:08.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.502+0000 7f0dd116d700 1 --2- 172.21.15.3:0/2722906423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dcc06fcf0 0x7f0dcc138570 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.502+0000 7f0dd116d700 1 --2- 172.21.15.3:0/2722906423 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dcc130540 0x7f0dcc130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.502+0000 7f0dd116d700 1 -- 172.21.15.3:0/2722906423 >> 172.21.15.3:0/2722906423 conn(0x7f0dcc0a2aa0 msgr2=0x7f0dcc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:08.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.502+0000 7f0dd116d700 1 -- 172.21.15.3:0/2722906423 shutdown_connections 2023-10-03T10:16:08.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.502+0000 7f0dd116d700 1 -- 172.21.15.3:0/2722906423 wait complete. 2023-10-03T10:16:08.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.503+0000 7f0dd116d700 1 Processor -- start 2023-10-03T10:16:08.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.503+0000 7f0dd116d700 1 -- start start 2023-10-03T10:16:08.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.504+0000 7f0dd116d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dcc06fcf0 0x7f0dcc09a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:08.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.504+0000 7f0dd116d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dcc130540 0x7f0dcc09ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:08.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.504+0000 7f0dd116d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dcc130e50 0x7f0dcc09b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:08.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.504+0000 7f0dd116d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc0718e0 con 0x7f0dcc130540 2023-10-03T10:16:08.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.504+0000 7f0dd116d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc123160 con 0x7f0dcc130e50 2023-10-03T10:16:08.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.504+0000 7f0dd116d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc09c270 con 0x7f0dcc06fcf0 2023-10-03T10:16:08.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.504+0000 7f0dcad9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dcc06fcf0 0x7f0dcc09a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:08.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.504+0000 7f0dcad9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dcc06fcf0 0x7f0dcc09a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52276/0 (socket says 172.21.15.3:52276) 2023-10-03T10:16:08.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.504+0000 7f0dcad9d700 1 -- 172.21.15.3:0/3798046405 learned_addr learned my addr 172.21.15.3:0/3798046405 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:08.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.504+0000 7f0dca59c700 1 --2- 172.21.15.3:0/3798046405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dcc130540 0x7f0dcc09ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:08.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.504+0000 7f0dcb59e700 1 --2- 172.21.15.3:0/3798046405 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dcc130e50 0x7f0dcc09b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:08.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.505+0000 7f0dca59c700 1 -- 172.21.15.3:0/3798046405 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dcc06fcf0 msgr2=0x7f0dcc09a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:08.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.505+0000 7f0dca59c700 1 --2- 172.21.15.3:0/3798046405 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dcc06fcf0 0x7f0dcc09a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.505+0000 7f0dca59c700 1 -- 172.21.15.3:0/3798046405 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dcc130e50 msgr2=0x7f0dcc09b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:08.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.505+0000 7f0dca59c700 1 --2- 172.21.15.3:0/3798046405 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dcc130e50 0x7f0dcc09b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.505+0000 7f0dca59c700 1 -- 172.21.15.3:0/3798046405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc4009b30 con 0x7f0dcc130540 2023-10-03T10:16:08.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.505+0000 7f0dcb59e700 1 --2- 172.21.15.3:0/3798046405 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dcc130e50 0x7f0dcc09b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:08.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.505+0000 7f0dca59c700 1 --2- 172.21.15.3:0/3798046405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dcc130540 0x7f0dcc09ab00 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0000f60 tx=0x7f0dc000c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:08.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.505+0000 7f0db37fe700 1 -- 172.21.15.3:0/3798046405 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc00050d0 con 0x7f0dcc130540 2023-10-03T10:16:08.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.505+0000 7f0db37fe700 1 -- 172.21.15.3:0/3798046405 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc0005230 con 0x7f0dcc130540 2023-10-03T10:16:08.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.505+0000 7f0dd116d700 1 -- 172.21.15.3:0/3798046405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dcc12db70 con 0x7f0dcc130540 2023-10-03T10:16:08.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.505+0000 7f0db37fe700 1 -- 172.21.15.3:0/3798046405 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0003680 con 0x7f0dcc130540 2023-10-03T10:16:08.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.505+0000 7f0dd116d700 1 -- 172.21.15.3:0/3798046405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dcc12d6b0 con 0x7f0dcc130540 2023-10-03T10:16:08.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.506+0000 7f0db37fe700 1 -- 172.21.15.3:0/3798046405 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0dc00037e0 con 0x7f0dcc130540 2023-10-03T10:16:08.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.506+0000 7f0db37fe700 1 -- 172.21.15.3:0/3798046405 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0dc003eaf0 con 0x7f0dcc130540 2023-10-03T10:16:08.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.507+0000 7f0dd116d700 1 -- 172.21.15.3:0/3798046405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dcc130910 con 0x7f0dcc130540 2023-10-03T10:16:08.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.514+0000 7f0db37fe700 1 -- 172.21.15.3:0/3798046405 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0dc0011590 con 0x7f0dcc130540 2023-10-03T10:16:08.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.594+0000 7fade0676700 1 Processor -- start 2023-10-03T10:16:08.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.595+0000 7fade0676700 1 -- start start 2023-10-03T10:16:08.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.596+0000 7fade0676700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fadd806a770 0x7fadd806ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:08.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.596+0000 7fade0676700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fadd806b080 0x7fadd8131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:08.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.596+0000 7fade0676700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadd809ece0 con 0x7fadd8131b40 2023-10-03T10:16:08.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.596+0000 7fade0676700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadd809e760 con 0x7fadd806a770 2023-10-03T10:16:08.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.596+0000 7fade0676700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fadd809e040 con 0x7fadd806b080 2023-10-03T10:16:08.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.596+0000 7fadde412700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fadd806a770 0x7fadd806ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:08.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.596+0000 7fadde412700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fadd806a770 0x7fadd806ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53126/0 (socket says 172.21.15.3:53126) 2023-10-03T10:16:08.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.596+0000 7fadde412700 1 -- 172.21.15.3:0/1948566753 learned_addr learned my addr 172.21.15.3:0/1948566753 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:08.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.596+0000 7fadddc11700 1 --2- 172.21.15.3:0/1948566753 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fadd806b080 0x7fadd8131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:08.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.596+0000 7fadde412700 1 -- 172.21.15.3:0/1948566753 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fadd806b080 msgr2=0x7fadd8131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:08.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.596+0000 7fadde412700 1 --2- 172.21.15.3:0/1948566753 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fadd806b080 0x7fadd8131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.596+0000 7fadde412700 1 -- 172.21.15.3:0/1948566753 >> v1:172.21.15.3:6789/0 conn(0x7fadd8131b40 legacy=0x7fadd8138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:08.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.596+0000 7fadde412700 1 -- 172.21.15.3:0/1948566753 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd809f630 con 0x7fadd806a770 2023-10-03T10:16:08.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.597+0000 7fadde412700 1 --2- 172.21.15.3:0/1948566753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadd806a770 0x7fadd806ab40 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fadd40098f0 tx=0x7fadd4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=976184c52923d86a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:08.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.597+0000 7faddd410700 1 -- 172.21.15.3:0/1948566753 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd400b8a0 con 0x7fadd806a770 2023-10-03T10:16:08.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.597+0000 7faddd410700 1 -- 172.21.15.3:0/1948566753 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd400ba00 con 0x7fadd806a770 2023-10-03T10:16:08.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.597+0000 7faddd410700 1 -- 172.21.15.3:0/1948566753 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd400bd90 con 0x7fadd806a770 2023-10-03T10:16:08.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.598+0000 7fade0676700 1 -- 172.21.15.3:0/1948566753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadd806a770 msgr2=0x7fadd806ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:08.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.598+0000 7fade0676700 1 --2- 172.21.15.3:0/1948566753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadd806a770 0x7fadd806ab40 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fadd40098f0 tx=0x7fadd4009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:08.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.598+0000 7fade0676700 1 -- 172.21.15.3:0/1948566753 shutdown_connections 2023-10-03T10:16:08.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.598+0000 7fade0676700 1 --2- 172.21.15.3:0/1948566753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadd806a770 0x7fadd806ab40 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.598+0000 7fade0676700 1 --2- 172.21.15.3:0/1948566753 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fadd806b080 0x7fadd8131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.598+0000 7fade0676700 1 -- 172.21.15.3:0/1948566753 >> 172.21.15.3:0/1948566753 conn(0x7fadd8066f50 msgr2=0x7fadd809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:08.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.598+0000 7fade0676700 1 -- 172.21.15.3:0/1948566753 shutdown_connections 2023-10-03T10:16:08.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.598+0000 7fade0676700 1 -- 172.21.15.3:0/1948566753 wait complete. 2023-10-03T10:16:08.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.599+0000 7fade0676700 1 Processor -- start 2023-10-03T10:16:08.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.599+0000 7fade0676700 1 -- start start 2023-10-03T10:16:08.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.599+0000 7fade0676700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fadd806a770 0x7fadd811c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:08.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.599+0000 7fade0676700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadd806b080 0x7fadd811c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:08.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.599+0000 7fade0676700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fadd8131b40 0x7fadd811cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:08.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.600+0000 7fade0676700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd8071f10 con 0x7fadd806b080 2023-10-03T10:16:08.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.600+0000 7fade0676700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd8131040 con 0x7fadd806a770 2023-10-03T10:16:08.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.600+0000 7fade0676700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fadd811fa10 con 0x7fadd8131b40 2023-10-03T10:16:08.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.600+0000 7faddec13700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fadd8131b40 0x7fadd811cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:08.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.600+0000 7faddec13700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fadd8131b40 0x7fadd811cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52288/0 (socket says 172.21.15.3:52288) 2023-10-03T10:16:08.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.600+0000 7faddec13700 1 -- 172.21.15.3:0/1996853719 learned_addr learned my addr 172.21.15.3:0/1996853719 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:08.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.600+0000 7fadddc11700 1 --2- 172.21.15.3:0/1996853719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadd806b080 0x7fadd811c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:08.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.600+0000 7faddec13700 1 -- 172.21.15.3:0/1996853719 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fadd806a770 msgr2=0x7fadd811c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:08.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.600+0000 7faddec13700 1 --2- 172.21.15.3:0/1996853719 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fadd806a770 0x7fadd811c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.600+0000 7faddec13700 1 -- 172.21.15.3:0/1996853719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadd806b080 msgr2=0x7fadd811c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:08.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.600+0000 7faddec13700 1 --2- 172.21.15.3:0/1996853719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadd806b080 0x7fadd811c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.600+0000 7faddec13700 1 -- 172.21.15.3:0/1996853719 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd4009550 con 0x7fadd8131b40 2023-10-03T10:16:08.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.600+0000 7faddec13700 1 --2- 172.21.15.3:0/1996853719 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fadd8131b40 0x7fadd811cf10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fadd000cc80 tx=0x7fadd000cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:08.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.601+0000 7fadceffd700 1 -- 172.21.15.3:0/1996853719 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd0003c00 con 0x7fadd8131b40 2023-10-03T10:16:08.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.601+0000 7fadceffd700 1 -- 172.21.15.3:0/1996853719 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd0003d60 con 0x7fadd8131b40 2023-10-03T10:16:08.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.601+0000 7fade0676700 1 -- 172.21.15.3:0/1996853719 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd8131510 con 0x7fadd8131b40 2023-10-03T10:16:08.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.601+0000 7fadceffd700 1 -- 172.21.15.3:0/1996853719 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd000ab50 con 0x7fadd8131b40 2023-10-03T10:16:08.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.601+0000 7fade0676700 1 -- 172.21.15.3:0/1996853719 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd811e0a0 con 0x7fadd8131b40 2023-10-03T10:16:08.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.602+0000 7fadceffd700 1 -- 172.21.15.3:0/1996853719 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fadd000acb0 con 0x7fadd8131b40 2023-10-03T10:16:08.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.602+0000 7fadceffd700 1 -- 172.21.15.3:0/1996853719 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fadd003f740 con 0x7fadd8131b40 2023-10-03T10:16:08.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.602+0000 7fade0676700 1 -- 172.21.15.3:0/1996853719 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd811fc30 con 0x7fadd8131b40 2023-10-03T10:16:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.612+0000 7fadceffd700 1 -- 172.21.15.3:0/1996853719 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fadd00198f0 con 0x7fadd8131b40 2023-10-03T10:16:08.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.641+0000 7fcb32546700 1 Processor -- start 2023-10-03T10:16:08.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.641+0000 7fcb32546700 1 -- start start 2023-10-03T10:16:08.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.642+0000 7fcb32546700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcb2c130540 0x7fcb2c130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:08.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.642+0000 7fcb32546700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb2c130e50 0x7fcb2c06f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:08.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.642+0000 7fcb32546700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb2c0716a0 con 0x7fcb2c06fb20 2023-10-03T10:16:08.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.642+0000 7fcb32546700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb2c09a390 con 0x7fcb2c130540 2023-10-03T10:16:08.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.642+0000 7fcb32546700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c071290 con 0x7fcb2c130e50 2023-10-03T10:16:08.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.642+0000 7fcb2b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb2c130e50 0x7fcb2c06f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:08.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.642+0000 7fcb2b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb2c130e50 0x7fcb2c06f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52296/0 (socket says 172.21.15.3:52296) 2023-10-03T10:16:08.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.642+0000 7fcb2b7fe700 1 -- 172.21.15.3:0/3842231369 learned_addr learned my addr 172.21.15.3:0/3842231369 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:08.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.642+0000 7fcb2bfff700 1 --2- 172.21.15.3:0/3842231369 >> v2:172.21.15.3:3300/0 conn(0x7fcb2c130540 0x7fcb2c130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:08.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.642+0000 7fcb2b7fe700 1 -- 172.21.15.3:0/3842231369 >> v2:172.21.15.3:3300/0 conn(0x7fcb2c130540 msgr2=0x7fcb2c130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:08.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.642+0000 7fcb2b7fe700 1 --2- 172.21.15.3:0/3842231369 >> v2:172.21.15.3:3300/0 conn(0x7fcb2c130540 0x7fcb2c130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.642+0000 7fcb2b7fe700 1 -- 172.21.15.3:0/3842231369 >> v1:172.21.15.3:6789/0 conn(0x7fcb2c06fb20 legacy=0x7fcb2c137560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:08.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.642+0000 7fcb2b7fe700 1 -- 172.21.15.3:0/3842231369 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb2c130340 con 0x7fcb2c130e50 2023-10-03T10:16:08.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.643+0000 7fcb2b7fe700 1 --2- 172.21.15.3:0/3842231369 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb2c130e50 0x7fcb2c06f5e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000de80 tx=0x7fcb2000f330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f39d21f9d3d7eef server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:08.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.643+0000 7fcb2affd700 1 -- 172.21.15.3:0/3842231369 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb20012070 con 0x7fcb2c130e50 2023-10-03T10:16:08.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.643+0000 7fcb2affd700 1 -- 172.21.15.3:0/3842231369 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2000f980 con 0x7fcb2c130e50 2023-10-03T10:16:08.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.643+0000 7fcb2affd700 1 -- 172.21.15.3:0/3842231369 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2000fd30 con 0x7fcb2c130e50 2023-10-03T10:16:08.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.644+0000 7fcb32546700 1 -- 172.21.15.3:0/3842231369 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb2c130e50 msgr2=0x7fcb2c06f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:08.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.644+0000 7fcb32546700 1 --2- 172.21.15.3:0/3842231369 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb2c130e50 0x7fcb2c06f5e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000de80 tx=0x7fcb2000f330 comp rx=0 tx=0).stop 2023-10-03T10:16:08.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.644+0000 7fcb32546700 1 -- 172.21.15.3:0/3842231369 shutdown_connections 2023-10-03T10:16:08.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.644+0000 7fcb32546700 1 --2- 172.21.15.3:0/3842231369 >> v2:172.21.15.3:3300/0 conn(0x7fcb2c130540 0x7fcb2c130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.644+0000 7fcb32546700 1 --2- 172.21.15.3:0/3842231369 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb2c130e50 0x7fcb2c06f5e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.644+0000 7fcb32546700 1 -- 172.21.15.3:0/3842231369 >> 172.21.15.3:0/3842231369 conn(0x7fcb2c0a2aa0 msgr2=0x7fcb2c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:08.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.644+0000 7fcb32546700 1 -- 172.21.15.3:0/3842231369 shutdown_connections 2023-10-03T10:16:08.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.644+0000 7fcb32546700 1 -- 172.21.15.3:0/3842231369 wait complete. 2023-10-03T10:16:08.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.645+0000 7fcb32546700 1 Processor -- start 2023-10-03T10:16:08.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.646+0000 7fcb32546700 1 -- start start 2023-10-03T10:16:08.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.646+0000 7fcb32546700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb2c06fb20 0x7fcb2c11b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:08.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.646+0000 7fcb32546700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb2c130540 0x7fcb2c11bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:08.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.646+0000 7fcb32546700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb2c130e50 0x7fcb2c11c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:08.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.646+0000 7fcb32546700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c11d9f0 con 0x7fcb2c06fb20 2023-10-03T10:16:08.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.646+0000 7fcb32546700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c12db70 con 0x7fcb2c130540 2023-10-03T10:16:08.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.646+0000 7fcb32546700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c0718e0 con 0x7fcb2c130e50 2023-10-03T10:16:08.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.646+0000 7fcb30ae3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb2c130e50 0x7fcb2c11c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:08.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.646+0000 7fcb30ae3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb2c130e50 0x7fcb2c11c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52306/0 (socket says 172.21.15.3:52306) 2023-10-03T10:16:08.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.646+0000 7fcb30ae3700 1 -- 172.21.15.3:0/3170257160 learned_addr learned my addr 172.21.15.3:0/3170257160 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:08.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.647+0000 7fcb2bfff700 1 --2- 172.21.15.3:0/3170257160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb2c06fb20 0x7fcb2c11b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:08.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.647+0000 7fcb2b7fe700 1 --2- 172.21.15.3:0/3170257160 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb2c130540 0x7fcb2c11bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:08.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.647+0000 7fcb30ae3700 1 -- 172.21.15.3:0/3170257160 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb2c130540 msgr2=0x7fcb2c11bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:08.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.647+0000 7fcb30ae3700 1 --2- 172.21.15.3:0/3170257160 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb2c130540 0x7fcb2c11bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.647+0000 7fcb30ae3700 1 -- 172.21.15.3:0/3170257160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb2c06fb20 msgr2=0x7fcb2c11b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:08.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.647+0000 7fcb30ae3700 1 --2- 172.21.15.3:0/3170257160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb2c06fb20 0x7fcb2c11b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.647+0000 7fcb30ae3700 1 -- 172.21.15.3:0/3170257160 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb2000db30 con 0x7fcb2c130e50 2023-10-03T10:16:08.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.647+0000 7fcb2bfff700 1 --2- 172.21.15.3:0/3170257160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb2c06fb20 0x7fcb2c11b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:16:08.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.647+0000 7fcb30ae3700 1 --2- 172.21.15.3:0/3170257160 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb2c130e50 0x7fcb2c11c320 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400cc80 tx=0x7fcb2400a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:08.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.647+0000 7fcb28ff9700 1 -- 172.21.15.3:0/3170257160 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2400ad40 con 0x7fcb2c130e50 2023-10-03T10:16:08.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.647+0000 7fcb28ff9700 1 -- 172.21.15.3:0/3170257160 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2400ce80 con 0x7fcb2c130e50 2023-10-03T10:16:08.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.647+0000 7fcb28ff9700 1 -- 172.21.15.3:0/3170257160 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2400aea0 con 0x7fcb2c130e50 2023-10-03T10:16:08.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.647+0000 7fcb32546700 1 -- 172.21.15.3:0/3170257160 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb2c11efd0 con 0x7fcb2c130e50 2023-10-03T10:16:08.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.648+0000 7fcb32546700 1 -- 172.21.15.3:0/3170257160 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb2c11e5f0 con 0x7fcb2c130e50 2023-10-03T10:16:08.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.650+0000 7ff735e6a700 1 Processor -- start 2023-10-03T10:16:08.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.650+0000 7ff735e6a700 1 -- start start 2023-10-03T10:16:08.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.650+0000 7ff735e6a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff73009d6f0 0x7ff73009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:08.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.650+0000 7ff735e6a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff73009e000 0x7ff730130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:08.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.650+0000 7ff735e6a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff73009e410 0x7ff730130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:08.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.650+0000 7ff735e6a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff73009f520 con 0x7ff73009e000 2023-10-03T10:16:08.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.650+0000 7ff735e6a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73009f130 con 0x7ff73009d6f0 2023-10-03T10:16:08.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.650+0000 7ff735e6a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff73009e930 con 0x7ff73009e410 2023-10-03T10:16:08.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.650+0000 7ff72effd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff73009e000 0x7ff730130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:08.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.650+0000 7ff72effd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff73009e000 0x7ff730130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53158/0 (socket says 172.21.15.3:53158) 2023-10-03T10:16:08.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.650+0000 7ff72effd700 1 -- 172.21.15.3:0/802039712 learned_addr learned my addr 172.21.15.3:0/802039712 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:08.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.651+0000 7ff72effd700 1 -- 172.21.15.3:0/802039712 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff73009e410 msgr2=0x7ff730130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:08.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.651+0000 7ff72effd700 1 --2- 172.21.15.3:0/802039712 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff73009e410 0x7ff730130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.651+0000 7ff72effd700 1 -- 172.21.15.3:0/802039712 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff73009d6f0 msgr2=0x7ff73009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:08.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.651+0000 7ff72effd700 1 --2- 172.21.15.3:0/802039712 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff73009d6f0 0x7ff73009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.651+0000 7ff72effd700 1 -- 172.21.15.3:0/802039712 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73009b2d0 con 0x7ff73009e000 2023-10-03T10:16:08.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.651+0000 7ff72effd700 1 --2- 172.21.15.3:0/802039712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff73009e000 0x7ff730130660 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff724009920 tx=0x7ff724009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0cbd8620aaa0894 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:08.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.651+0000 7ff72e7fc700 1 -- 172.21.15.3:0/802039712 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72400ba20 con 0x7ff73009e000 2023-10-03T10:16:08.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.651+0000 7ff72e7fc700 1 -- 172.21.15.3:0/802039712 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72400bb80 con 0x7ff73009e000 2023-10-03T10:16:08.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.651+0000 7ff72e7fc700 1 -- 172.21.15.3:0/802039712 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff724009e70 con 0x7ff73009e000 2023-10-03T10:16:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.652+0000 7ff735e6a700 1 -- 172.21.15.3:0/802039712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff73009e000 msgr2=0x7ff730130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.652+0000 7ff735e6a700 1 --2- 172.21.15.3:0/802039712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff73009e000 0x7ff730130660 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff724009920 tx=0x7ff724009c30 comp rx=0 tx=0).stop 2023-10-03T10:16:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.652+0000 7ff735e6a700 1 -- 172.21.15.3:0/802039712 shutdown_connections 2023-10-03T10:16:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.652+0000 7ff735e6a700 1 --2- 172.21.15.3:0/802039712 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff73009e410 0x7ff730130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.652+0000 7ff735e6a700 1 --2- 172.21.15.3:0/802039712 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff73009d6f0 0x7ff73009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.652+0000 7ff735e6a700 1 --2- 172.21.15.3:0/802039712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff73009e000 0x7ff730130660 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.652+0000 7ff735e6a700 1 -- 172.21.15.3:0/802039712 >> 172.21.15.3:0/802039712 conn(0x7ff730066f50 msgr2=0x7ff73009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.652+0000 7ff735e6a700 1 -- 172.21.15.3:0/802039712 shutdown_connections 2023-10-03T10:16:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.652+0000 7ff735e6a700 1 -- 172.21.15.3:0/802039712 wait complete. 2023-10-03T10:16:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.654+0000 7ff735e6a700 1 Processor -- start 2023-10-03T10:16:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.654+0000 7ff735e6a700 1 -- start start 2023-10-03T10:16:08.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.655+0000 7ff735e6a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff73009d6f0 0x7ff730137920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:08.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.655+0000 7ff735e6a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff73009e000 0x7ff730137e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.655+0000 7ff72f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff73009d6f0 0x7ff730137920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.655+0000 7ff72f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff73009d6f0 0x7ff730137920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53270/0 (socket says 172.21.15.3:53270) 2023-10-03T10:16:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.655+0000 7ff72effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff73009e000 0x7ff730137e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.655+0000 7ff72effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff73009e000 0x7ff730137e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52314/0 (socket says 172.21.15.3:52314) 2023-10-03T10:16:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.655+0000 7ff735e6a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff73009e410 0x7ff7301383a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.655+0000 7ff72f7fe700 1 -- 172.21.15.3:0/1475993678 learned_addr learned my addr 172.21.15.3:0/1475993678 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.655+0000 7ff735e6a700 1 -- 172.21.15.3:0/1475993678 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73011acc0 con 0x7ff73009e410 2023-10-03T10:16:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.655+0000 7ff735e6a700 1 -- 172.21.15.3:0/1475993678 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7300a1d90 con 0x7ff73009d6f0 2023-10-03T10:16:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.655+0000 7ff735e6a700 1 -- 172.21.15.3:0/1475993678 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff730139400 con 0x7ff73009e000 2023-10-03T10:16:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.655+0000 7ff72ffff700 1 --2- 172.21.15.3:0/1475993678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff73009e410 0x7ff7301383a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.656+0000 7ff72f7fe700 1 -- 172.21.15.3:0/1475993678 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff73009e000 msgr2=0x7ff730137e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.656+0000 7ff72f7fe700 1 --2- 172.21.15.3:0/1475993678 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff73009e000 0x7ff730137e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.656+0000 7ff72f7fe700 1 -- 172.21.15.3:0/1475993678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff73009e410 msgr2=0x7ff7301383a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.656+0000 7ff72f7fe700 1 --2- 172.21.15.3:0/1475993678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff73009e410 0x7ff7301383a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.656+0000 7ff72f7fe700 1 -- 172.21.15.3:0/1475993678 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff724009550 con 0x7ff73009d6f0 2023-10-03T10:16:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.656+0000 7ff72effd700 1 --2- 172.21.15.3:0/1475993678 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff73009e000 0x7ff730137e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.656+0000 7ff72ffff700 1 --2- 172.21.15.3:0/1475993678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff73009e410 0x7ff7301383a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:16:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.656+0000 7ff72f7fe700 1 --2- 172.21.15.3:0/1475993678 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff73009d6f0 0x7ff730137920 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff72000b560 tx=0x7ff72000b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.656+0000 7ff717fff700 1 -- 172.21.15.3:0/1475993678 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720011830 con 0x7ff73009d6f0 2023-10-03T10:16:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.656+0000 7ff717fff700 1 -- 172.21.15.3:0/1475993678 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7200119b0 con 0x7ff73009d6f0 2023-10-03T10:16:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.657+0000 7ff735e6a700 1 -- 172.21.15.3:0/1475993678 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff73009c700 con 0x7ff73009d6f0 2023-10-03T10:16:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.657+0000 7ff717fff700 1 -- 172.21.15.3:0/1475993678 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720011d60 con 0x7ff73009d6f0 2023-10-03T10:16:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.657+0000 7ff735e6a700 1 -- 172.21.15.3:0/1475993678 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff73011b450 con 0x7ff73009d6f0 2023-10-03T10:16:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.658+0000 7ff717fff700 1 -- 172.21.15.3:0/1475993678 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff72001a400 con 0x7ff73009d6f0 2023-10-03T10:16:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.658+0000 7ff735e6a700 1 -- 172.21.15.3:0/1475993678 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff730074ea0 con 0x7ff73009d6f0 2023-10-03T10:16:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.658+0000 7ff717fff700 1 -- 172.21.15.3:0/1475993678 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff72003f6c0 con 0x7ff73009d6f0 2023-10-03T10:16:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.648+0000 7fcb32546700 1 -- 172.21.15.3:0/3170257160 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb2c09dd40 con 0x7fcb2c130e50 2023-10-03T10:16:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.659+0000 7fcb28ff9700 1 -- 172.21.15.3:0/3170257160 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcb24003c40 con 0x7fcb2c130e50 2023-10-03T10:16:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.659+0000 7fcb28ff9700 1 -- 172.21.15.3:0/3170257160 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fcb240402f0 con 0x7fcb2c130e50 2023-10-03T10:16:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.659+0000 7fcb28ff9700 1 -- 172.21.15.3:0/3170257160 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcb240407c0 con 0x7fcb2c130e50 2023-10-03T10:16:08.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.665+0000 7ff717fff700 1 -- 172.21.15.3:0/1475993678 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff7200193b0 con 0x7ff73009d6f0 2023-10-03T10:16:08.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.732+0000 7f0dd116d700 1 -- 172.21.15.3:0/3798046405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0dcc074ea0 con 0x7f0dcc130540 2023-10-03T10:16:08.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.733+0000 7f0db37fe700 1 -- 172.21.15.3:0/3798046405 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0dc00192c0 con 0x7f0dcc130540 2023-10-03T10:16:08.733 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:08.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.736+0000 7f0dd116d700 1 -- 172.21.15.3:0/3798046405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dcc130540 msgr2=0x7f0dcc09ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:08.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.736+0000 7f0dd116d700 1 --2- 172.21.15.3:0/3798046405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dcc130540 0x7f0dcc09ab00 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0000f60 tx=0x7f0dc000c730 comp rx=0 tx=0).stop 2023-10-03T10:16:08.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.736+0000 7f0dd116d700 1 -- 172.21.15.3:0/3798046405 shutdown_connections 2023-10-03T10:16:08.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.736+0000 7f0dd116d700 1 --2- 172.21.15.3:0/3798046405 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dcc130e50 0x7f0dcc09b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.736+0000 7f0dd116d700 1 --2- 172.21.15.3:0/3798046405 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dcc06fcf0 0x7f0dcc09a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.736+0000 7f0dd116d700 1 --2- 172.21.15.3:0/3798046405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dcc130540 0x7f0dcc09ab00 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.736+0000 7f0dd116d700 1 -- 172.21.15.3:0/3798046405 >> 172.21.15.3:0/3798046405 conn(0x7f0dcc0a2aa0 msgr2=0x7f0dcc09df40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:08.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.736+0000 7f0dd116d700 1 -- 172.21.15.3:0/3798046405 shutdown_connections 2023-10-03T10:16:08.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.736+0000 7f0dd116d700 1 -- 172.21.15.3:0/3798046405 wait complete. 2023-10-03T10:16:08.756 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:08.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.844+0000 7fade0676700 1 -- 172.21.15.3:0/1996853719 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fadd811fe30 con 0x7fadd8131b40 2023-10-03T10:16:08.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.844+0000 7fadceffd700 1 -- 172.21.15.3:0/1996853719 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fadd0019300 con 0x7fadd8131b40 2023-10-03T10:16:08.845 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:08.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.847+0000 7fade0676700 1 -- 172.21.15.3:0/1996853719 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fadd8131b40 msgr2=0x7fadd811cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:08.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.847+0000 7fade0676700 1 --2- 172.21.15.3:0/1996853719 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fadd8131b40 0x7fadd811cf10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fadd000cc80 tx=0x7fadd000cf90 comp rx=0 tx=0).stop 2023-10-03T10:16:08.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.847+0000 7fade0676700 1 -- 172.21.15.3:0/1996853719 shutdown_connections 2023-10-03T10:16:08.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.847+0000 7fade0676700 1 --2- 172.21.15.3:0/1996853719 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fadd8131b40 0x7fadd811cf10 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.847+0000 7fade0676700 1 --2- 172.21.15.3:0/1996853719 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fadd806a770 0x7fadd811c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.847+0000 7fade0676700 1 --2- 172.21.15.3:0/1996853719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadd806b080 0x7fadd811c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.847+0000 7fade0676700 1 -- 172.21.15.3:0/1996853719 >> 172.21.15.3:0/1996853719 conn(0x7fadd8066f50 msgr2=0x7fadd80a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:08.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.847+0000 7fade0676700 1 -- 172.21.15.3:0/1996853719 shutdown_connections 2023-10-03T10:16:08.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.847+0000 7fade0676700 1 -- 172.21.15.3:0/1996853719 wait complete. 2023-10-03T10:16:08.861 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:08.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.866+0000 7fcb32546700 1 -- 172.21.15.3:0/3170257160 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcb2c11f1e0 con 0x7fcb2c130e50 2023-10-03T10:16:08.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.866+0000 7fcb28ff9700 1 -- 172.21.15.3:0/3170257160 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcb24029090 con 0x7fcb2c130e50 2023-10-03T10:16:08.867 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:08.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.869+0000 7fcb32546700 1 -- 172.21.15.3:0/3170257160 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb2c130e50 msgr2=0x7fcb2c11c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:08.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.869+0000 7fcb32546700 1 --2- 172.21.15.3:0/3170257160 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb2c130e50 0x7fcb2c11c320 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400cc80 tx=0x7fcb2400a340 comp rx=0 tx=0).stop 2023-10-03T10:16:08.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.869+0000 7fcb32546700 1 -- 172.21.15.3:0/3170257160 shutdown_connections 2023-10-03T10:16:08.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.869+0000 7fcb32546700 1 --2- 172.21.15.3:0/3170257160 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb2c130e50 0x7fcb2c11c320 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.869+0000 7fcb32546700 1 --2- 172.21.15.3:0/3170257160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb2c06fb20 0x7fcb2c11b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.869+0000 7fcb32546700 1 --2- 172.21.15.3:0/3170257160 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb2c130540 0x7fcb2c11bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.869+0000 7fcb32546700 1 -- 172.21.15.3:0/3170257160 >> 172.21.15.3:0/3170257160 conn(0x7fcb2c0a2aa0 msgr2=0x7fcb2c12fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:08.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.869+0000 7fcb32546700 1 -- 172.21.15.3:0/3170257160 shutdown_connections 2023-10-03T10:16:08.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.869+0000 7fcb32546700 1 -- 172.21.15.3:0/3170257160 wait complete. 2023-10-03T10:16:08.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.875+0000 7ff735e6a700 1 -- 172.21.15.3:0/1475993678 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff730071f30 con 0x7ff73009d6f0 2023-10-03T10:16:08.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.875+0000 7ff717fff700 1 -- 172.21.15.3:0/1475993678 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff720028030 con 0x7ff73009d6f0 2023-10-03T10:16:08.876 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:08.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.878+0000 7ff735e6a700 1 -- 172.21.15.3:0/1475993678 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff73009d6f0 msgr2=0x7ff730137920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:08.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.878+0000 7ff735e6a700 1 --2- 172.21.15.3:0/1475993678 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff73009d6f0 0x7ff730137920 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff72000b560 tx=0x7ff72000b870 comp rx=0 tx=0).stop 2023-10-03T10:16:08.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.878+0000 7ff735e6a700 1 -- 172.21.15.3:0/1475993678 shutdown_connections 2023-10-03T10:16:08.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.878+0000 7ff735e6a700 1 --2- 172.21.15.3:0/1475993678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff73009e410 0x7ff7301383a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.878+0000 7ff735e6a700 1 --2- 172.21.15.3:0/1475993678 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff73009d6f0 0x7ff730137920 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.878+0000 7ff735e6a700 1 --2- 172.21.15.3:0/1475993678 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff73009e000 0x7ff730137e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:08.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.878+0000 7ff735e6a700 1 -- 172.21.15.3:0/1475993678 >> 172.21.15.3:0/1475993678 conn(0x7ff730066f50 msgr2=0x7ff730070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:08.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.878+0000 7ff735e6a700 1 -- 172.21.15.3:0/1475993678 shutdown_connections 2023-10-03T10:16:08.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:08.878+0000 7ff735e6a700 1 -- 172.21.15.3:0/1475993678 wait complete. 2023-10-03T10:16:08.884 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:08.892 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:09.757 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:09.861 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:09.885 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:09.893 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:09.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.904+0000 7fb80bc64700 1 Processor -- start 2023-10-03T10:16:09.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.905+0000 7fb80bc64700 1 -- start start 2023-10-03T10:16:09.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.905+0000 7fb80bc64700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb80409d630 0x7fb8040a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:09.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.905+0000 7fb80bc64700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb804138d80 0x7fb80413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:09.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.905+0000 7fb80bc64700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8040a2a70 con 0x7fb8040a0a30 2023-10-03T10:16:09.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.905+0000 7fb80bc64700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8040a01d0 con 0x7fb804138d80 2023-10-03T10:16:09.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.905+0000 7fb80bc64700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb80409e2b0 con 0x7fb80409d630 2023-10-03T10:16:09.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.905+0000 7fb8091ff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb80409d630 0x7fb8040a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:09.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.905+0000 7fb8091ff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb80409d630 0x7fb8040a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52322/0 (socket says 172.21.15.3:52322) 2023-10-03T10:16:09.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.905+0000 7fb8091ff700 1 -- 172.21.15.3:0/1532017409 learned_addr learned my addr 172.21.15.3:0/1532017409 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:09.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.905+0000 7fb80a201700 1 --2- 172.21.15.3:0/1532017409 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb804138d80 0x7fb80413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:09.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.906+0000 7fb8091ff700 1 -- 172.21.15.3:0/1532017409 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb804138d80 msgr2=0x7fb80413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:09.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.906+0000 7fb8091ff700 1 --2- 172.21.15.3:0/1532017409 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb804138d80 0x7fb80413b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:09.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.906+0000 7fb8091ff700 1 -- 172.21.15.3:0/1532017409 >> v1:172.21.15.3:6789/0 conn(0x7fb8040a0a30 legacy=0x7fb8040a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:09.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.906+0000 7fb8089fe700 1 -- 172.21.15.3:0/1532017409 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3489340840 0 0) 0x7fb8040a2a70 con 0x7fb8040a0a30 2023-10-03T10:16:09.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.906+0000 7fb8091ff700 1 -- 172.21.15.3:0/1532017409 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80409dc50 con 0x7fb80409d630 2023-10-03T10:16:09.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.906+0000 7fb8091ff700 1 --2- 172.21.15.3:0/1532017409 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb80409d630 0x7fb8040a1510 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009920 tx=0x7fb7f4009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d564053a5d99ddc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:09.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.906+0000 7fb8089fe700 1 -- 172.21.15.3:0/1532017409 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4011830 con 0x7fb80409d630 2023-10-03T10:16:09.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.906+0000 7fb8089fe700 1 -- 172.21.15.3:0/1532017409 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f4011990 con 0x7fb80409d630 2023-10-03T10:16:09.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.906+0000 7fb8089fe700 1 -- 172.21.15.3:0/1532017409 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4011d20 con 0x7fb80409d630 2023-10-03T10:16:09.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.907+0000 7fb80bc64700 1 -- 172.21.15.3:0/1532017409 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb80409d630 msgr2=0x7fb8040a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:09.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.907+0000 7fb80bc64700 1 --2- 172.21.15.3:0/1532017409 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb80409d630 0x7fb8040a1510 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009920 tx=0x7fb7f4009c30 comp rx=0 tx=0).stop 2023-10-03T10:16:09.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.907+0000 7fb80bc64700 1 -- 172.21.15.3:0/1532017409 shutdown_connections 2023-10-03T10:16:09.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.907+0000 7fb80bc64700 1 --2- 172.21.15.3:0/1532017409 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb804138d80 0x7fb80413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:09.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.907+0000 7fb80bc64700 1 --2- 172.21.15.3:0/1532017409 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb80409d630 0x7fb8040a1510 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:09.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.907+0000 7fb80bc64700 1 -- 172.21.15.3:0/1532017409 >> 172.21.15.3:0/1532017409 conn(0x7fb8040a3050 msgr2=0x7fb8040a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:09.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.907+0000 7fb80bc64700 1 -- 172.21.15.3:0/1532017409 shutdown_connections 2023-10-03T10:16:09.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.907+0000 7fb80bc64700 1 -- 172.21.15.3:0/1532017409 wait complete. 2023-10-03T10:16:09.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.909+0000 7fb80bc64700 1 Processor -- start 2023-10-03T10:16:09.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.909+0000 7fb80bc64700 1 -- start start 2023-10-03T10:16:09.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.910+0000 7fb80bc64700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80409d630 0x7fb80412c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:09.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.910+0000 7fb80bc64700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb8040a0a30 0x7fb80412c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:09.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.910+0000 7fb80bc64700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb804138d80 0x7fb80412ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:09.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.910+0000 7fb80bc64700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80412e730 con 0x7fb80409d630 2023-10-03T10:16:09.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.910+0000 7fb80bc64700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80412d9c0 con 0x7fb804138d80 2023-10-03T10:16:09.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.910+0000 7fb80bc64700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb8040a0410 con 0x7fb8040a0a30 2023-10-03T10:16:09.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.910+0000 7fb809a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80409d630 0x7fb80412c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:09.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.910+0000 7fb809a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80409d630 0x7fb80412c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53184/0 (socket says 172.21.15.3:53184) 2023-10-03T10:16:09.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.910+0000 7fb809a00700 1 -- 172.21.15.3:0/1122759523 learned_addr learned my addr 172.21.15.3:0/1122759523 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:09.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.910+0000 7fb8091ff700 1 --2- 172.21.15.3:0/1122759523 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb8040a0a30 0x7fb80412c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:09.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.910+0000 7fb809a00700 1 -- 172.21.15.3:0/1122759523 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb8040a0a30 msgr2=0x7fb80412c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:09.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.910+0000 7fb809a00700 1 --2- 172.21.15.3:0/1122759523 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb8040a0a30 0x7fb80412c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:09.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.910+0000 7fb809a00700 1 -- 172.21.15.3:0/1122759523 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb804138d80 msgr2=0x7fb80412ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:09.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.910+0000 7fb809a00700 1 --2- 172.21.15.3:0/1122759523 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb804138d80 0x7fb80412ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:09.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.910+0000 7fb809a00700 1 -- 172.21.15.3:0/1122759523 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f4009550 con 0x7fb80409d630 2023-10-03T10:16:09.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.911+0000 7fb809a00700 1 --2- 172.21.15.3:0/1122759523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80409d630 0x7fb80412c230 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fb800001eb0 tx=0x7fb80000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:09.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.911+0000 7fb7fa7fc700 1 -- 172.21.15.3:0/1122759523 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8000054e0 con 0x7fb80409d630 2023-10-03T10:16:09.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.911+0000 7fb7fa7fc700 1 -- 172.21.15.3:0/1122759523 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb800003680 con 0x7fb80409d630 2023-10-03T10:16:09.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.911+0000 7fb7fa7fc700 1 -- 172.21.15.3:0/1122759523 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8000039f0 con 0x7fb80409d630 2023-10-03T10:16:09.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.911+0000 7fb80bc64700 1 -- 172.21.15.3:0/1122759523 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb80412f070 con 0x7fb80409d630 2023-10-03T10:16:09.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.911+0000 7fb80bc64700 1 -- 172.21.15.3:0/1122759523 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb80412ed90 con 0x7fb80409d630 2023-10-03T10:16:09.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.913+0000 7fb80bc64700 1 -- 172.21.15.3:0/1122759523 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8040747f0 con 0x7fb80409d630 2023-10-03T10:16:09.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.913+0000 7fb7fa7fc700 1 -- 172.21.15.3:0/1122759523 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb800003b50 con 0x7fb80409d630 2023-10-03T10:16:09.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.913+0000 7fb7fa7fc700 1 -- 172.21.15.3:0/1122759523 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb80003ea90 con 0x7fb80409d630 2023-10-03T10:16:09.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:09.920+0000 7fb7fa7fc700 1 -- 172.21.15.3:0/1122759523 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb800012400 con 0x7fb80409d630 2023-10-03T10:16:10.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.046+0000 7fe9a0096700 1 Processor -- start 2023-10-03T10:16:10.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.047+0000 7fe0a2f4a700 1 Processor -- start 2023-10-03T10:16:10.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.047+0000 7fe9a0096700 1 -- start start 2023-10-03T10:16:10.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.047+0000 7fe0a2f4a700 1 -- start start 2023-10-03T10:16:10.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.047+0000 7fe9a0096700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe99812cf60 0x7fe99806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:10.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.047+0000 7fe0a2f4a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe09c09d5c0 0x7fe09c138d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:10.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.048+0000 7fe0a2f4a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe09c0a14a0 0x7fe09c13b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:10.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.047+0000 7fe9a0096700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe99812f8b0 0x7fe99812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:10.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.048+0000 7fe9a0096700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe99812cd90 con 0x7fe998130260 2023-10-03T10:16:10.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.048+0000 7fe0a2f4a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe09c09ea10 con 0x7fe09c0a09c0 2023-10-03T10:16:10.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.048+0000 7fe0a2f4a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe09c09e210 con 0x7fe09c09d5c0 2023-10-03T10:16:10.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.048+0000 7fe0a2f4a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c0a26f0 con 0x7fe09c0a14a0 2023-10-03T10:16:10.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.048+0000 7fe9a0096700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe99812d330 con 0x7fe99812cf60 2023-10-03T10:16:10.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.048+0000 7fe9a0096700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9981306d0 con 0x7fe99812f8b0 2023-10-03T10:16:10.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.048+0000 7fe0a14e7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe09c0a14a0 0x7fe09c13b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:10.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.048+0000 7fe0a14e7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe09c0a14a0 0x7fe09c13b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52330/0 (socket says 172.21.15.3:52330) 2023-10-03T10:16:10.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.048+0000 7fe0a14e7700 1 -- 172.21.15.3:0/841127207 learned_addr learned my addr 172.21.15.3:0/841127207 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:10.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.048+0000 7fe99d631700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe99812f8b0 0x7fe99812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:10.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.048+0000 7fe99d631700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe99812f8b0 0x7fe99812fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52346/0 (socket says 172.21.15.3:52346) 2023-10-03T10:16:10.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.048+0000 7fe99d631700 1 -- 172.21.15.3:0/1678229413 learned_addr learned my addr 172.21.15.3:0/1678229413 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:10.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.048+0000 7fe99d631700 1 -- 172.21.15.3:0/1678229413 >> v2:172.21.15.3:3300/0 conn(0x7fe99812cf60 msgr2=0x7fe99806f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:10.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.048+0000 7fe99d631700 1 --2- 172.21.15.3:0/1678229413 >> v2:172.21.15.3:3300/0 conn(0x7fe99812cf60 0x7fe99806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.048+0000 7fe99d631700 1 -- 172.21.15.3:0/1678229413 >> v1:172.21.15.3:6789/0 conn(0x7fe998130260 legacy=0x7fe998136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:10.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.048+0000 7fe99d631700 1 -- 172.21.15.3:0/1678229413 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe998071780 con 0x7fe99812f8b0 2023-10-03T10:16:10.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.048+0000 7fe0a14e7700 1 -- 172.21.15.3:0/841127207 >> v2:172.21.15.3:3300/0 conn(0x7fe09c09d5c0 msgr2=0x7fe09c138d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:10.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.048+0000 7fe0a14e7700 1 --2- 172.21.15.3:0/841127207 >> v2:172.21.15.3:3300/0 conn(0x7fe09c09d5c0 0x7fe09c138d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.048+0000 7fe0a14e7700 1 -- 172.21.15.3:0/841127207 >> v1:172.21.15.3:6789/0 conn(0x7fe09c0a09c0 legacy=0x7fe09c0a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:10.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.048+0000 7fe0a14e7700 1 -- 172.21.15.3:0/841127207 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09c09db70 con 0x7fe09c0a14a0 2023-10-03T10:16:10.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.048+0000 7fe99d631700 1 --2- 172.21.15.3:0/1678229413 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe99812f8b0 0x7fe99812fd20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe9880098f0 tx=0x7fe988009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d8da82547fe8dc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:10.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.049+0000 7fe0a14e7700 1 --2- 172.21.15.3:0/841127207 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe09c0a14a0 0x7fe09c13b2c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe094009e50 tx=0x7fe09400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb1238b3e64c53b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:10.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.049+0000 7fe99ce30700 1 -- 172.21.15.3:0/1678229413 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe988004230 con 0x7fe99812f8b0 2023-10-03T10:16:10.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.049+0000 7fe99ce30700 1 -- 172.21.15.3:0/1678229413 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe988004390 con 0x7fe99812f8b0 2023-10-03T10:16:10.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.049+0000 7fe99ce30700 1 -- 172.21.15.3:0/1678229413 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9880036a0 con 0x7fe99812f8b0 2023-10-03T10:16:10.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.049+0000 7fe09b7fe700 1 -- 172.21.15.3:0/841127207 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09400e070 con 0x7fe09c0a14a0 2023-10-03T10:16:10.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.049+0000 7fe09b7fe700 1 -- 172.21.15.3:0/841127207 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe094003a20 con 0x7fe09c0a14a0 2023-10-03T10:16:10.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.049+0000 7fe09b7fe700 1 -- 172.21.15.3:0/841127207 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe094003dd0 con 0x7fe09c0a14a0 2023-10-03T10:16:10.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.050+0000 7fe9a0096700 1 -- 172.21.15.3:0/1678229413 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe99812f8b0 msgr2=0x7fe99812fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:10.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.050+0000 7fe9a0096700 1 --2- 172.21.15.3:0/1678229413 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe99812f8b0 0x7fe99812fd20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe9880098f0 tx=0x7fe988009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:10.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.050+0000 7fe9a0096700 1 -- 172.21.15.3:0/1678229413 shutdown_connections 2023-10-03T10:16:10.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.050+0000 7fe9a0096700 1 --2- 172.21.15.3:0/1678229413 >> v2:172.21.15.3:3300/0 conn(0x7fe99812cf60 0x7fe99806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.050+0000 7fe9a0096700 1 --2- 172.21.15.3:0/1678229413 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe99812f8b0 0x7fe99812fd20 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.050+0000 7fe9a0096700 1 -- 172.21.15.3:0/1678229413 >> 172.21.15.3:0/1678229413 conn(0x7fe9980a2aa0 msgr2=0x7fe9980a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:10.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.050+0000 7fe0a2f4a700 1 -- 172.21.15.3:0/841127207 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe09c0a14a0 msgr2=0x7fe09c13b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:10.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.050+0000 7fe0a2f4a700 1 --2- 172.21.15.3:0/841127207 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe09c0a14a0 0x7fe09c13b2c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe094009e50 tx=0x7fe09400b330 comp rx=0 tx=0).stop 2023-10-03T10:16:10.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.050+0000 7fe0a2f4a700 1 -- 172.21.15.3:0/841127207 shutdown_connections 2023-10-03T10:16:10.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.050+0000 7fe0a2f4a700 1 --2- 172.21.15.3:0/841127207 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe09c0a14a0 0x7fe09c13b2c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.050+0000 7fe0a2f4a700 1 --2- 172.21.15.3:0/841127207 >> v2:172.21.15.3:3300/0 conn(0x7fe09c09d5c0 0x7fe09c138d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.050+0000 7fe0a2f4a700 1 -- 172.21.15.3:0/841127207 >> 172.21.15.3:0/841127207 conn(0x7fe09c0a2fe0 msgr2=0x7fe09c0a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:10.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.050+0000 7fe9a0096700 1 -- 172.21.15.3:0/1678229413 shutdown_connections 2023-10-03T10:16:10.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.050+0000 7fe0a2f4a700 1 -- 172.21.15.3:0/841127207 shutdown_connections 2023-10-03T10:16:10.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.050+0000 7fe0a2f4a700 1 -- 172.21.15.3:0/841127207 wait complete. 2023-10-03T10:16:10.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.050+0000 7fe9a0096700 1 -- 172.21.15.3:0/1678229413 wait complete. 2023-10-03T10:16:10.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.051+0000 7fe0a2f4a700 1 Processor -- start 2023-10-03T10:16:10.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.051+0000 7fe9a0096700 1 Processor -- start 2023-10-03T10:16:10.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.052+0000 7fe0a2f4a700 1 -- start start 2023-10-03T10:16:10.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.052+0000 7fe9a0096700 1 -- start start 2023-10-03T10:16:10.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.052+0000 7fe9a0096700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe99812cf60 0x7fe99809b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:10.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.052+0000 7fe9a0096700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe99812f8b0 0x7fe99809bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:10.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.052+0000 7fe9a0096700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe998130260 0x7fe9981c4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:10.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.052+0000 7fe9a0096700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99809b510 con 0x7fe998130260 2023-10-03T10:16:10.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.052+0000 7fe9a0096700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe998129760 con 0x7fe99812cf60 2023-10-03T10:16:10.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.052+0000 7fe9a0096700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe99812f5d0 con 0x7fe99812f8b0 2023-10-03T10:16:10.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.052+0000 7fe0a2f4a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe09c09d5c0 0x7fe09c11c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:10.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.052+0000 7fe0a2f4a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe09c0a09c0 0x7fe09c11ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:10.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.052+0000 7fe99de32700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe99812cf60 0x7fe99809b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:10.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.052+0000 7fe0a2f4a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe09c0a14a0 0x7fe09c11d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:10.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.052+0000 7fe99de32700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe99812cf60 0x7fe99809b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53292/0 (socket says 172.21.15.3:53292) 2023-10-03T10:16:10.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.052+0000 7fe99de32700 1 -- 172.21.15.3:0/3755029727 learned_addr learned my addr 172.21.15.3:0/3755029727 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:10.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.052+0000 7fe99d631700 1 --2- 172.21.15.3:0/3755029727 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe99812f8b0 0x7fe99809bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:10.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.052+0000 7fe99e633700 1 --2- 172.21.15.3:0/3755029727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe998130260 0x7fe9981c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:10.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.053+0000 7fe99de32700 1 -- 172.21.15.3:0/3755029727 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe99812f8b0 msgr2=0x7fe99809bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:10.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.053+0000 7fe99de32700 1 --2- 172.21.15.3:0/3755029727 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe99812f8b0 0x7fe99809bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.053+0000 7fe99de32700 1 -- 172.21.15.3:0/3755029727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe998130260 msgr2=0x7fe9981c4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:10.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.053+0000 7fe99de32700 1 --2- 172.21.15.3:0/3755029727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe998130260 0x7fe9981c4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.053+0000 7fe99de32700 1 -- 172.21.15.3:0/3755029727 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe988009550 con 0x7fe99812cf60 2023-10-03T10:16:10.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.053+0000 7fe99de32700 1 --2- 172.21.15.3:0/3755029727 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe99812cf60 0x7fe99809b9c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe99400b560 tx=0x7fe99400b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:10.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.053+0000 7fe98e7fc700 1 -- 172.21.15.3:0/3755029727 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe994011830 con 0x7fe99812cf60 2023-10-03T10:16:10.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.053+0000 7fe98e7fc700 1 -- 172.21.15.3:0/3755029727 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe994011990 con 0x7fe99812cf60 2023-10-03T10:16:10.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.053+0000 7fe98e7fc700 1 -- 172.21.15.3:0/3755029727 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe994011d40 con 0x7fe99812cf60 2023-10-03T10:16:10.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.053+0000 7fe9a0096700 1 -- 172.21.15.3:0/3755029727 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe998129fa0 con 0x7fe99812cf60 2023-10-03T10:16:10.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.054+0000 7fe9a0096700 1 -- 172.21.15.3:0/3755029727 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe99809e560 con 0x7fe99812cf60 2023-10-03T10:16:10.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.055+0000 7fe98e7fc700 1 -- 172.21.15.3:0/3755029727 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe99401a400 con 0x7fe99812cf60 2023-10-03T10:16:10.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.055+0000 7fe9a0096700 1 -- 172.21.15.3:0/3755029727 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe998074ea0 con 0x7fe99812cf60 2023-10-03T10:16:10.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.055+0000 7fe98e7fc700 1 -- 172.21.15.3:0/3755029727 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe99403f6d0 con 0x7fe99812cf60 2023-10-03T10:16:10.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.052+0000 7fe0a2f4a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c11ef00 con 0x7fe09c09d5c0 2023-10-03T10:16:10.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.052+0000 7fe0a2f4a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c11e9d0 con 0x7fe09c0a09c0 2023-10-03T10:16:10.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.052+0000 7fe0a2f4a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c11e100 con 0x7fe09c0a14a0 2023-10-03T10:16:10.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.053+0000 7fe0a14e7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe09c0a14a0 0x7fe09c11d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:10.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.053+0000 7fe0a14e7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe09c0a14a0 0x7fe09c11d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52370/0 (socket says 172.21.15.3:52370) 2023-10-03T10:16:10.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.053+0000 7fe0a14e7700 1 -- 172.21.15.3:0/3738839398 learned_addr learned my addr 172.21.15.3:0/3738839398 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:10.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.053+0000 7fe09bfff700 1 --2- 172.21.15.3:0/3738839398 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe09c0a09c0 0x7fe09c11ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:10.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.053+0000 7fe0a14e7700 1 -- 172.21.15.3:0/3738839398 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe09c0a09c0 msgr2=0x7fe09c11ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:10.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.053+0000 7fe0a14e7700 1 --2- 172.21.15.3:0/3738839398 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe09c0a09c0 0x7fe09c11ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.053+0000 7fe0a14e7700 1 -- 172.21.15.3:0/3738839398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe09c09d5c0 msgr2=0x7fe09c11c7b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:16:10.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.053+0000 7fe0a14e7700 1 --2- 172.21.15.3:0/3738839398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe09c09d5c0 0x7fe09c11c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.053+0000 7fe0a14e7700 1 -- 172.21.15.3:0/3738839398 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe094009b30 con 0x7fe09c0a14a0 2023-10-03T10:16:10.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.053+0000 7fe09bfff700 1 --2- 172.21.15.3:0/3738839398 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe09c0a09c0 0x7fe09c11ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:16:10.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.054+0000 7fe0a14e7700 1 --2- 172.21.15.3:0/3738839398 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe09c0a14a0 0x7fe09c11d230 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0a2830 tx=0x7fe094004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:10.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.054+0000 7fe0997fa700 1 -- 172.21.15.3:0/3738839398 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe094004240 con 0x7fe09c0a14a0 2023-10-03T10:16:10.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.054+0000 7fe0997fa700 1 -- 172.21.15.3:0/3738839398 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0940043c0 con 0x7fe09c0a14a0 2023-10-03T10:16:10.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.054+0000 7fe0997fa700 1 -- 172.21.15.3:0/3738839398 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe094004770 con 0x7fe09c0a14a0 2023-10-03T10:16:10.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.054+0000 7fe0a2f4a700 1 -- 172.21.15.3:0/3738839398 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe09c120070 con 0x7fe09c0a14a0 2023-10-03T10:16:10.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.054+0000 7fe0a2f4a700 1 -- 172.21.15.3:0/3738839398 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe09c11f3c0 con 0x7fe09c0a14a0 2023-10-03T10:16:10.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.055+0000 7fe0997fa700 1 -- 172.21.15.3:0/3738839398 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe094019070 con 0x7fe09c0a14a0 2023-10-03T10:16:10.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.056+0000 7fe0a2f4a700 1 -- 172.21.15.3:0/3738839398 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe09c11f9d0 con 0x7fe09c0a14a0 2023-10-03T10:16:10.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.056+0000 7fe0997fa700 1 -- 172.21.15.3:0/3738839398 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe094024a10 con 0x7fe09c0a14a0 2023-10-03T10:16:10.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.060+0000 7fe98e7fc700 1 -- 172.21.15.3:0/3755029727 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe99400f8d0 con 0x7fe99812cf60 2023-10-03T10:16:10.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.061+0000 7fe0997fa700 1 -- 172.21.15.3:0/3738839398 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe09c11f9d0 con 0x7fe09c0a14a0 2023-10-03T10:16:10.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.062+0000 7f4145a18700 1 Processor -- start 2023-10-03T10:16:10.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.062+0000 7f4145a18700 1 -- start start 2023-10-03T10:16:10.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.063+0000 7f4145a18700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f414009d6f0 0x7f414009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:10.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.063+0000 7f4145a18700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f414009e000 0x7f4140130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:10.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.063+0000 7f4145a18700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f414009e410 0x7f4140130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:10.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.063+0000 7f4145a18700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f414009f520 con 0x7f414009d6f0 2023-10-03T10:16:10.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.063+0000 7f4145a18700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f414009f130 con 0x7f414009e410 2023-10-03T10:16:10.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.063+0000 7f4145a18700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f414009e930 con 0x7f414009e000 2023-10-03T10:16:10.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.063+0000 7f413e7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f414009e000 0x7f4140130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:10.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.063+0000 7f413e7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f414009e000 0x7f4140130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52372/0 (socket says 172.21.15.3:52372) 2023-10-03T10:16:10.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.063+0000 7f413e7fc700 1 -- 172.21.15.3:0/8881016 learned_addr learned my addr 172.21.15.3:0/8881016 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:10.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.063+0000 7f413effd700 1 --2- 172.21.15.3:0/8881016 >> v2:172.21.15.3:3300/0 conn(0x7f414009d6f0 0x7f414009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:10.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.063+0000 7f413f7fe700 1 --2- 172.21.15.3:0/8881016 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f414009e410 0x7f4140130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:10.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.064+0000 7f413effd700 1 -- 172.21.15.3:0/8881016 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f414009e000 msgr2=0x7f4140130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:10.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.064+0000 7f413effd700 1 --2- 172.21.15.3:0/8881016 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f414009e000 0x7f4140130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.064+0000 7f413effd700 1 -- 172.21.15.3:0/8881016 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f414009e410 msgr2=0x7f4140130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:10.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.064+0000 7f413effd700 1 --2- 172.21.15.3:0/8881016 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f414009e410 0x7f4140130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.064+0000 7f413effd700 1 -- 172.21.15.3:0/8881016 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f414009b2d0 con 0x7f414009d6f0 2023-10-03T10:16:10.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.064+0000 7f413effd700 1 --2- 172.21.15.3:0/8881016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f414009d6f0 0x7f414009dac0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f413000dc90 tx=0x7f4130008050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876da86542e0b2a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:10.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.064+0000 7f413dffb700 1 -- 172.21.15.3:0/8881016 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413000b070 con 0x7f414009d6f0 2023-10-03T10:16:10.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.064+0000 7f413dffb700 1 -- 172.21.15.3:0/8881016 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4130008a00 con 0x7f414009d6f0 2023-10-03T10:16:10.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.064+0000 7f413dffb700 1 -- 172.21.15.3:0/8881016 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4130008d90 con 0x7f414009d6f0 2023-10-03T10:16:10.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.065+0000 7f4145a18700 1 -- 172.21.15.3:0/8881016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f414009d6f0 msgr2=0x7f414009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:10.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.065+0000 7f4145a18700 1 --2- 172.21.15.3:0/8881016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f414009d6f0 0x7f414009dac0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f413000dc90 tx=0x7f4130008050 comp rx=0 tx=0).stop 2023-10-03T10:16:10.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.065+0000 7f4145a18700 1 -- 172.21.15.3:0/8881016 shutdown_connections 2023-10-03T10:16:10.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.065+0000 7f4145a18700 1 --2- 172.21.15.3:0/8881016 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f414009e410 0x7f4140130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.065+0000 7f4145a18700 1 --2- 172.21.15.3:0/8881016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f414009d6f0 0x7f414009dac0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.065+0000 7f4145a18700 1 --2- 172.21.15.3:0/8881016 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f414009e000 0x7f4140130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.065+0000 7f4145a18700 1 -- 172.21.15.3:0/8881016 >> 172.21.15.3:0/8881016 conn(0x7f4140066f50 msgr2=0x7f414009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:10.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.065+0000 7f4145a18700 1 -- 172.21.15.3:0/8881016 shutdown_connections 2023-10-03T10:16:10.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.065+0000 7f4145a18700 1 -- 172.21.15.3:0/8881016 wait complete. 2023-10-03T10:16:10.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.066+0000 7f4145a18700 1 Processor -- start 2023-10-03T10:16:10.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.066+0000 7f4145a18700 1 -- start start 2023-10-03T10:16:10.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.067+0000 7f4145a18700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f414009d6f0 0x7f414011e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:10.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.067+0000 7f4145a18700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f414009e000 0x7f414011e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:10.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.067+0000 7f4145a18700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f414009e410 0x7f414011ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:10.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.067+0000 7f4145a18700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4140121640 con 0x7f414009d6f0 2023-10-03T10:16:10.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.067+0000 7f4145a18700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f414011f730 con 0x7f414009e000 2023-10-03T10:16:10.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.067+0000 7f4145a18700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f41401204d0 con 0x7f414009e410 2023-10-03T10:16:10.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.067+0000 7f413effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f414009d6f0 0x7f414011e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:10.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.067+0000 7f413effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f414009d6f0 0x7f414011e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53222/0 (socket says 172.21.15.3:53222) 2023-10-03T10:16:10.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.067+0000 7f413effd700 1 -- 172.21.15.3:0/907241646 learned_addr learned my addr 172.21.15.3:0/907241646 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:10.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.067+0000 7f413f7fe700 1 --2- 172.21.15.3:0/907241646 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f414009e410 0x7f414011ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:10.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.067+0000 7f413e7fc700 1 --2- 172.21.15.3:0/907241646 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f414009e000 0x7f414011e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:10.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.067+0000 7f413effd700 1 -- 172.21.15.3:0/907241646 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f414009e410 msgr2=0x7f414011ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:10.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.067+0000 7f413effd700 1 --2- 172.21.15.3:0/907241646 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f414009e410 0x7f414011ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.067+0000 7f413effd700 1 -- 172.21.15.3:0/907241646 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f414009e000 msgr2=0x7f414011e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:10.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.067+0000 7f413effd700 1 --2- 172.21.15.3:0/907241646 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f414009e000 0x7f414011e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.067+0000 7f413effd700 1 -- 172.21.15.3:0/907241646 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413000d940 con 0x7f414009d6f0 2023-10-03T10:16:10.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.067+0000 7f413e7fc700 1 --2- 172.21.15.3:0/907241646 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f414009e000 0x7f414011e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:16:10.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.067+0000 7f413effd700 1 --2- 172.21.15.3:0/907241646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f414009d6f0 0x7f414011e2a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f41300083a0 tx=0x7f4130004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:10.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.068+0000 7f4127fff700 1 -- 172.21.15.3:0/907241646 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413001b070 con 0x7f414009d6f0 2023-10-03T10:16:10.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.068+0000 7f4145a18700 1 -- 172.21.15.3:0/907241646 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4140072590 con 0x7f414009d6f0 2023-10-03T10:16:10.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.068+0000 7f4145a18700 1 -- 172.21.15.3:0/907241646 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4140121ae0 con 0x7f414009d6f0 2023-10-03T10:16:10.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.068+0000 7f4127fff700 1 -- 172.21.15.3:0/907241646 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4130008800 con 0x7f414009d6f0 2023-10-03T10:16:10.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.068+0000 7f4127fff700 1 -- 172.21.15.3:0/907241646 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413000b070 con 0x7f414009d6f0 2023-10-03T10:16:10.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.069+0000 7f4127fff700 1 -- 172.21.15.3:0/907241646 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4130004030 con 0x7f414009d6f0 2023-10-03T10:16:10.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.069+0000 7f4127fff700 1 -- 172.21.15.3:0/907241646 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4130040680 con 0x7f414009d6f0 2023-10-03T10:16:10.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.071+0000 7f4145a18700 1 -- 172.21.15.3:0/907241646 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4140074ea0 con 0x7f414009d6f0 2023-10-03T10:16:10.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.080+0000 7f4127fff700 1 -- 172.21.15.3:0/907241646 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f41300042e0 con 0x7f414009d6f0 2023-10-03T10:16:10.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.199+0000 7fb80bc64700 1 -- 172.21.15.3:0/1122759523 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb80412d220 con 0x7fb80409d630 2023-10-03T10:16:10.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.200+0000 7fb7fa7fc700 1 -- 172.21.15.3:0/1122759523 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb800017020 con 0x7fb80409d630 2023-10-03T10:16:10.200 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:10.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.202+0000 7fb80bc64700 1 -- 172.21.15.3:0/1122759523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80409d630 msgr2=0x7fb80412c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:10.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.202+0000 7fb80bc64700 1 --2- 172.21.15.3:0/1122759523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80409d630 0x7fb80412c230 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fb800001eb0 tx=0x7fb80000c800 comp rx=0 tx=0).stop 2023-10-03T10:16:10.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.202+0000 7fb80bc64700 1 -- 172.21.15.3:0/1122759523 shutdown_connections 2023-10-03T10:16:10.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.202+0000 7fb80bc64700 1 --2- 172.21.15.3:0/1122759523 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb804138d80 0x7fb80412ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.202+0000 7fb80bc64700 1 --2- 172.21.15.3:0/1122759523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80409d630 0x7fb80412c230 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.202+0000 7fb80bc64700 1 --2- 172.21.15.3:0/1122759523 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb8040a0a30 0x7fb80412c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.202+0000 7fb80bc64700 1 -- 172.21.15.3:0/1122759523 >> 172.21.15.3:0/1122759523 conn(0x7fb8040a3050 msgr2=0x7fb8040a4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:10.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.202+0000 7fb80bc64700 1 -- 172.21.15.3:0/1122759523 shutdown_connections 2023-10-03T10:16:10.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.202+0000 7fb80bc64700 1 -- 172.21.15.3:0/1122759523 wait complete. 2023-10-03T10:16:10.217 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:10.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.268+0000 7fe9a0096700 1 -- 172.21.15.3:0/3755029727 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe99809cbf0 con 0x7fe99812cf60 2023-10-03T10:16:10.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.269+0000 7fe98e7fc700 1 -- 172.21.15.3:0/3755029727 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe994022350 con 0x7fe99812cf60 2023-10-03T10:16:10.269 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:10.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.271+0000 7fe9a0096700 1 -- 172.21.15.3:0/3755029727 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe99812cf60 msgr2=0x7fe99809b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:10.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.271+0000 7fe9a0096700 1 --2- 172.21.15.3:0/3755029727 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe99812cf60 0x7fe99809b9c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe99400b560 tx=0x7fe99400b870 comp rx=0 tx=0).stop 2023-10-03T10:16:10.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.271+0000 7fe9a0096700 1 -- 172.21.15.3:0/3755029727 shutdown_connections 2023-10-03T10:16:10.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.271+0000 7fe9a0096700 1 --2- 172.21.15.3:0/3755029727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe998130260 0x7fe9981c4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.271+0000 7fe9a0096700 1 --2- 172.21.15.3:0/3755029727 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe99812cf60 0x7fe99809b9c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.271+0000 7fe9a0096700 1 --2- 172.21.15.3:0/3755029727 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe99812f8b0 0x7fe99809bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.271+0000 7fe9a0096700 1 -- 172.21.15.3:0/3755029727 >> 172.21.15.3:0/3755029727 conn(0x7fe9980a2aa0 msgr2=0x7fe99806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:10.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.271+0000 7fe9a0096700 1 -- 172.21.15.3:0/3755029727 shutdown_connections 2023-10-03T10:16:10.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.271+0000 7fe9a0096700 1 -- 172.21.15.3:0/3755029727 wait complete. 2023-10-03T10:16:10.285 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:10.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.308+0000 7fe0a2f4a700 1 -- 172.21.15.3:0/3738839398 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe09c11f9d0 con 0x7fe09c0a14a0 2023-10-03T10:16:10.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.308+0000 7fe0997fa700 1 -- 172.21.15.3:0/3738839398 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe09c11f9d0 con 0x7fe09c0a14a0 2023-10-03T10:16:10.308 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:10.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.310+0000 7fe0a2f4a700 1 -- 172.21.15.3:0/3738839398 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe09c0a14a0 msgr2=0x7fe09c11d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:10.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.310+0000 7fe0a2f4a700 1 --2- 172.21.15.3:0/3738839398 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe09c0a14a0 0x7fe09c11d230 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0a2830 tx=0x7fe094004ad0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.311+0000 7fe0a2f4a700 1 -- 172.21.15.3:0/3738839398 shutdown_connections 2023-10-03T10:16:10.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.311+0000 7fe0a2f4a700 1 --2- 172.21.15.3:0/3738839398 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe09c0a14a0 0x7fe09c11d230 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.311+0000 7fe0a2f4a700 1 --2- 172.21.15.3:0/3738839398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe09c09d5c0 0x7fe09c11c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.311+0000 7fe0a2f4a700 1 --2- 172.21.15.3:0/3738839398 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe09c0a09c0 0x7fe09c11ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.311+0000 7fe0a2f4a700 1 -- 172.21.15.3:0/3738839398 >> 172.21.15.3:0/3738839398 conn(0x7fe09c0a2fe0 msgr2=0x7fe09c09a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:10.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.311+0000 7fe0a2f4a700 1 -- 172.21.15.3:0/3738839398 shutdown_connections 2023-10-03T10:16:10.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.311+0000 7fe0a2f4a700 1 -- 172.21.15.3:0/3738839398 wait complete. 2023-10-03T10:16:10.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.318+0000 7f4145a18700 1 -- 172.21.15.3:0/907241646 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4140072070 con 0x7f414009d6f0 2023-10-03T10:16:10.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.318+0000 7f4127fff700 1 -- 172.21.15.3:0/907241646 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4130031090 con 0x7f414009d6f0 2023-10-03T10:16:10.318 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:10.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.320+0000 7f4145a18700 1 -- 172.21.15.3:0/907241646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f414009d6f0 msgr2=0x7f414011e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:10.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.320+0000 7f4145a18700 1 --2- 172.21.15.3:0/907241646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f414009d6f0 0x7f414011e2a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f41300083a0 tx=0x7f4130004750 comp rx=0 tx=0).stop 2023-10-03T10:16:10.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.320+0000 7f4145a18700 1 -- 172.21.15.3:0/907241646 shutdown_connections 2023-10-03T10:16:10.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.320+0000 7f4145a18700 1 --2- 172.21.15.3:0/907241646 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f414009e410 0x7f414011ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.320+0000 7f4145a18700 1 --2- 172.21.15.3:0/907241646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f414009d6f0 0x7f414011e2a0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.320+0000 7f4145a18700 1 --2- 172.21.15.3:0/907241646 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f414009e000 0x7f414011e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:10.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.320+0000 7f4145a18700 1 -- 172.21.15.3:0/907241646 >> 172.21.15.3:0/907241646 conn(0x7f4140066f50 msgr2=0x7f41400a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:10.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.321+0000 7f4145a18700 1 -- 172.21.15.3:0/907241646 shutdown_connections 2023-10-03T10:16:10.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:10.321+0000 7f4145a18700 1 -- 172.21.15.3:0/907241646 wait complete. 2023-10-03T10:16:10.325 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:10.335 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:11.218 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:11.286 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:11.326 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:11.336 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:11.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.363+0000 7fd95b481700 1 Processor -- start 2023-10-03T10:16:11.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.364+0000 7fd95b481700 1 -- start start 2023-10-03T10:16:11.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.364+0000 7fd95b481700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd95406a770 0x7fd95406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:11.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.364+0000 7fd95b481700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd954131d10 0x7fd954139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:11.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.364+0000 7fd95b481700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd95409ece0 con 0x7fd95406b080 2023-10-03T10:16:11.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.364+0000 7fd95b481700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd95409e760 con 0x7fd954131d10 2023-10-03T10:16:11.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.364+0000 7fd95b481700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd95409e040 con 0x7fd95406a770 2023-10-03T10:16:11.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.364+0000 7fd958a1c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd95406b080 0x7fd954131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38982/0 (socket says 172.21.15.3:38982) 2023-10-03T10:16:11.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.365+0000 7fd958a1c700 1 -- 172.21.15.3:0/676113125 learned_addr learned my addr 172.21.15.3:0/676113125 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:11.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.365+0000 7fd95921d700 1 --2- 172.21.15.3:0/676113125 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd95406a770 0x7fd95406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:11.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.365+0000 7fd959a1e700 1 --2- 172.21.15.3:0/676113125 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd954131d10 0x7fd954139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:11.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.365+0000 7fd95921d700 1 -- 172.21.15.3:0/676113125 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd954131d10 msgr2=0x7fd954139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:11.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.365+0000 7fd95921d700 1 --2- 172.21.15.3:0/676113125 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd954131d10 0x7fd954139860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.365+0000 7fd95921d700 1 -- 172.21.15.3:0/676113125 >> v1:172.21.15.3:6789/0 conn(0x7fd95406b080 legacy=0x7fd954131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.365+0000 7fd95921d700 1 -- 172.21.15.3:0/676113125 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95409f630 con 0x7fd95406a770 2023-10-03T10:16:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.365+0000 7fd94bfff700 1 -- 172.21.15.3:0/676113125 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3721909285 0 0) 0x7fd95409ece0 con 0x7fd95406b080 2023-10-03T10:16:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.365+0000 7fd959a1e700 1 --2- 172.21.15.3:0/676113125 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd954131d10 0x7fd954139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.365+0000 7fd95921d700 1 --2- 172.21.15.3:0/676113125 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd95406a770 0x7fd95406ab40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd950009870 tx=0x7fd950009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4d03e8b54374414 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.365+0000 7fd94bfff700 1 -- 172.21.15.3:0/676113125 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd950011830 con 0x7fd95406a770 2023-10-03T10:16:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.365+0000 7fd94bfff700 1 -- 172.21.15.3:0/676113125 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd950011990 con 0x7fd95406a770 2023-10-03T10:16:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.366+0000 7fd94bfff700 1 -- 172.21.15.3:0/676113125 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd950011d20 con 0x7fd95406a770 2023-10-03T10:16:11.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.366+0000 7fd95b481700 1 -- 172.21.15.3:0/676113125 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd95406a770 msgr2=0x7fd95406ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:11.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.366+0000 7fd95b481700 1 --2- 172.21.15.3:0/676113125 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd95406a770 0x7fd95406ab40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd950009870 tx=0x7fd950009b80 comp rx=0 tx=0).stop 2023-10-03T10:16:11.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.366+0000 7fd95b481700 1 -- 172.21.15.3:0/676113125 shutdown_connections 2023-10-03T10:16:11.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.366+0000 7fd95b481700 1 --2- 172.21.15.3:0/676113125 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd954131d10 0x7fd954139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.366+0000 7fd95b481700 1 --2- 172.21.15.3:0/676113125 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd95406a770 0x7fd95406ab40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.366+0000 7fd95b481700 1 -- 172.21.15.3:0/676113125 >> 172.21.15.3:0/676113125 conn(0x7fd954066f50 msgr2=0x7fd95409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:11.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.367+0000 7fd95b481700 1 -- 172.21.15.3:0/676113125 shutdown_connections 2023-10-03T10:16:11.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.367+0000 7fd95b481700 1 -- 172.21.15.3:0/676113125 wait complete. 2023-10-03T10:16:11.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.368+0000 7fd95b481700 1 Processor -- start 2023-10-03T10:16:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.368+0000 7fd95b481700 1 -- start start 2023-10-03T10:16:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.368+0000 7fd95b481700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd95406a770 0x7fd9541c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.368+0000 7fd95b481700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd95406b080 0x7fd9541c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.368+0000 7fd95b481700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd954131d10 0x7fd954120510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.368+0000 7fd95b481700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd954122210 con 0x7fd95406b080 2023-10-03T10:16:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.368+0000 7fd95b481700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd95406ecf0 con 0x7fd95406a770 2023-10-03T10:16:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.368+0000 7fd95b481700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd954071e60 con 0x7fd954131d10 2023-10-03T10:16:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.369+0000 7fd959a1e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd954131d10 0x7fd954120510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.369+0000 7fd959a1e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd954131d10 0x7fd954120510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52398/0 (socket says 172.21.15.3:52398) 2023-10-03T10:16:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.369+0000 7fd958a1c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd95406b080 0x7fd9541c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.369+0000 7fd959a1e700 1 -- 172.21.15.3:0/2313787510 learned_addr learned my addr 172.21.15.3:0/2313787510 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:11.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.369+0000 7fd95921d700 1 --2- 172.21.15.3:0/2313787510 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd95406a770 0x7fd9541c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:11.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.369+0000 7fd959a1e700 1 -- 172.21.15.3:0/2313787510 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd95406a770 msgr2=0x7fd9541c62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:11.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.369+0000 7fd959a1e700 1 --2- 172.21.15.3:0/2313787510 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd95406a770 0x7fd9541c62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.369+0000 7fd959a1e700 1 -- 172.21.15.3:0/2313787510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd95406b080 msgr2=0x7fd9541c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:11.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.369+0000 7fd959a1e700 1 --2- 172.21.15.3:0/2313787510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd95406b080 0x7fd9541c6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.369+0000 7fd959a1e700 1 -- 172.21.15.3:0/2313787510 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd94c009a70 con 0x7fd954131d10 2023-10-03T10:16:11.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.369+0000 7fd95921d700 1 --2- 172.21.15.3:0/2313787510 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd95406a770 0x7fd9541c62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:11.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.369+0000 7fd959a1e700 1 --2- 172.21.15.3:0/2313787510 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd954131d10 0x7fd954120510 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd94c00efc0 tx=0x7fd94c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:11.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.370+0000 7fd949ffb700 1 -- 172.21.15.3:0/2313787510 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94c00ca90 con 0x7fd954131d10 2023-10-03T10:16:11.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.370+0000 7fd949ffb700 1 -- 172.21.15.3:0/2313787510 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd94c00f040 con 0x7fd954131d10 2023-10-03T10:16:11.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.370+0000 7fd95b481700 1 -- 172.21.15.3:0/2313787510 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9500094a0 con 0x7fd954131d10 2023-10-03T10:16:11.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.370+0000 7fd95b481700 1 -- 172.21.15.3:0/2313787510 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd954122880 con 0x7fd954131d10 2023-10-03T10:16:11.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.371+0000 7fd949ffb700 1 -- 172.21.15.3:0/2313787510 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94c00ce10 con 0x7fd954131d10 2023-10-03T10:16:11.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.371+0000 7fd949ffb700 1 -- 172.21.15.3:0/2313787510 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd94c014480 con 0x7fd954131d10 2023-10-03T10:16:11.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.371+0000 7fd949ffb700 1 -- 172.21.15.3:0/2313787510 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd94c03fd70 con 0x7fd954131d10 2023-10-03T10:16:11.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.372+0000 7fd95b481700 1 -- 172.21.15.3:0/2313787510 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9340052e0 con 0x7fd954131d10 2023-10-03T10:16:11.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.378+0000 7fd949ffb700 1 -- 172.21.15.3:0/2313787510 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd94c012930 con 0x7fd954131d10 2023-10-03T10:16:11.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.433+0000 7f525a090700 1 Processor -- start 2023-10-03T10:16:11.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.433+0000 7f525a090700 1 -- start start 2023-10-03T10:16:11.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.433+0000 7f525a090700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f525409d6f0 0x7f525409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:11.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.433+0000 7f525a090700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f525409e440 0x7f5254134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:11.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.433+0000 7f525a090700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f525409b120 con 0x7f525409e000 2023-10-03T10:16:11.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.433+0000 7f525a090700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f525406b460 con 0x7f525409d6f0 2023-10-03T10:16:11.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.433+0000 7f525a090700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525409a7e0 con 0x7f525409e440 2023-10-03T10:16:11.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.434+0000 7f52537fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f525409d6f0 0x7f525409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:11.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.434+0000 7f52537fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f525409d6f0 0x7f525409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53244/0 (socket says 172.21.15.3:53244) 2023-10-03T10:16:11.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.434+0000 7f52537fe700 1 -- 172.21.15.3:0/186981590 learned_addr learned my addr 172.21.15.3:0/186981590 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:11.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.434+0000 7f5253fff700 1 --2- 172.21.15.3:0/186981590 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f525409e440 0x7f5254134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:11.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.434+0000 7f52537fe700 1 -- 172.21.15.3:0/186981590 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f525409e440 msgr2=0x7f5254134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:11.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.434+0000 7f52537fe700 1 --2- 172.21.15.3:0/186981590 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f525409e440 0x7f5254134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.434+0000 7f52537fe700 1 -- 172.21.15.3:0/186981590 >> v1:172.21.15.3:6789/0 conn(0x7f525409e000 legacy=0x7f52541339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:11.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.434+0000 7f52537fe700 1 -- 172.21.15.3:0/186981590 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525409a500 con 0x7f525409d6f0 2023-10-03T10:16:11.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.434+0000 7f52537fe700 1 --2- 172.21.15.3:0/186981590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f525409d6f0 0x7f525409dac0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f5244009920 tx=0x7f5244009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d40041d169e63d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:11.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.434+0000 7f52527fc700 1 -- 172.21.15.3:0/186981590 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5244011830 con 0x7f525409d6f0 2023-10-03T10:16:11.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.435+0000 7f52527fc700 1 -- 172.21.15.3:0/186981590 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5244011990 con 0x7f525409d6f0 2023-10-03T10:16:11.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.435+0000 7f52527fc700 1 -- 172.21.15.3:0/186981590 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5244011d20 con 0x7f525409d6f0 2023-10-03T10:16:11.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.436+0000 7f525a090700 1 -- 172.21.15.3:0/186981590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f525409d6f0 msgr2=0x7f525409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:11.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.436+0000 7f525a090700 1 --2- 172.21.15.3:0/186981590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f525409d6f0 0x7f525409dac0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f5244009920 tx=0x7f5244009c30 comp rx=0 tx=0).stop 2023-10-03T10:16:11.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.436+0000 7f525a090700 1 -- 172.21.15.3:0/186981590 shutdown_connections 2023-10-03T10:16:11.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.436+0000 7f525a090700 1 --2- 172.21.15.3:0/186981590 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f525409e440 0x7f5254134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.436+0000 7f525a090700 1 --2- 172.21.15.3:0/186981590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f525409d6f0 0x7f525409dac0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.436+0000 7f525a090700 1 -- 172.21.15.3:0/186981590 >> 172.21.15.3:0/186981590 conn(0x7f5254066f50 msgr2=0x7f525409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:11.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.436+0000 7f525a090700 1 -- 172.21.15.3:0/186981590 shutdown_connections 2023-10-03T10:16:11.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.436+0000 7f525a090700 1 -- 172.21.15.3:0/186981590 wait complete. 2023-10-03T10:16:11.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.437+0000 7f525a090700 1 Processor -- start 2023-10-03T10:16:11.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.438+0000 7f525a090700 1 -- start start 2023-10-03T10:16:11.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.438+0000 7f525a090700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f525409d6f0 0x7f52541a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:11.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.438+0000 7f525a090700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f525409e000 0x7f52541a64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:11.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.438+0000 7f525a090700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f525409e440 0x7f52541a6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:11.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.438+0000 7f525a090700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525409c710 con 0x7f525409e440 2023-10-03T10:16:11.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.438+0000 7f525a090700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52541a81b0 con 0x7f525409e000 2023-10-03T10:16:11.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.438+0000 7f525a090700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f52541a7b60 con 0x7f525409d6f0 2023-10-03T10:16:11.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.439+0000 7f5253fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f525409e440 0x7f52541a6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:11.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.439+0000 7f5253fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f525409e440 0x7f52541a6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53246/0 (socket says 172.21.15.3:53246) 2023-10-03T10:16:11.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.439+0000 7f5253fff700 1 -- 172.21.15.3:0/1575437845 learned_addr learned my addr 172.21.15.3:0/1575437845 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:11.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.439+0000 7f52537fe700 1 --2- 172.21.15.3:0/1575437845 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f525409d6f0 0x7f52541a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:11.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.439+0000 7f5252ffd700 1 --2- 172.21.15.3:0/1575437845 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f525409e000 0x7f52541a64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:11.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.439+0000 7f5253fff700 1 -- 172.21.15.3:0/1575437845 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f525409d6f0 msgr2=0x7f52541a5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:11.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.439+0000 7f5253fff700 1 --2- 172.21.15.3:0/1575437845 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f525409d6f0 0x7f52541a5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.439+0000 7f5253fff700 1 -- 172.21.15.3:0/1575437845 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f525409e000 msgr2=0x7f52541a64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:11.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.439+0000 7f5253fff700 1 --2- 172.21.15.3:0/1575437845 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f525409e000 0x7f52541a64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.439+0000 7f5253fff700 1 -- 172.21.15.3:0/1575437845 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5244009550 con 0x7f525409e440 2023-10-03T10:16:11.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.439+0000 7f5252ffd700 1 --2- 172.21.15.3:0/1575437845 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f525409e000 0x7f52541a64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:11.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.439+0000 7f5253fff700 1 --2- 172.21.15.3:0/1575437845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f525409e440 0x7f52541a6a20 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f524c009dd0 tx=0x7f524c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:11.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.440+0000 7f523bfff700 1 -- 172.21.15.3:0/1575437845 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c009c40 con 0x7f525409e440 2023-10-03T10:16:11.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.440+0000 7f523bfff700 1 -- 172.21.15.3:0/1575437845 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524c00f040 con 0x7f525409e440 2023-10-03T10:16:11.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.440+0000 7f523bfff700 1 -- 172.21.15.3:0/1575437845 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c00ceb0 con 0x7f525409e440 2023-10-03T10:16:11.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.440+0000 7f525a090700 1 -- 172.21.15.3:0/1575437845 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52541a9a70 con 0x7f525409e440 2023-10-03T10:16:11.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.440+0000 7f525a090700 1 -- 172.21.15.3:0/1575437845 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52540a22a0 con 0x7f525409e440 2023-10-03T10:16:11.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.441+0000 7f525a090700 1 -- 172.21.15.3:0/1575437845 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52541a9340 con 0x7f525409e440 2023-10-03T10:16:11.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.442+0000 7f523bfff700 1 -- 172.21.15.3:0/1575437845 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f524c014480 con 0x7f525409e440 2023-10-03T10:16:11.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.442+0000 7f523bfff700 1 -- 172.21.15.3:0/1575437845 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f524c040750 con 0x7f525409e440 2023-10-03T10:16:11.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.447+0000 7f523bfff700 1 -- 172.21.15.3:0/1575437845 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f524c012930 con 0x7f525409e440 2023-10-03T10:16:11.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.476+0000 7f4fc6d53700 1 Processor -- start 2023-10-03T10:16:11.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.476+0000 7f4fc6d53700 1 -- start start 2023-10-03T10:16:11.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.477+0000 7f4fc6d53700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4fc00a0a30 0x7f4fc00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:11.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.477+0000 7f4fc6d53700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fc009d630 0x7f4fc00a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:11.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.477+0000 7f4fc6d53700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc00a2a70 con 0x7f4fc0133930 2023-10-03T10:16:11.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.477+0000 7f4fc6d53700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fc00a01d0 con 0x7f4fc00a0a30 2023-10-03T10:16:11.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.477+0000 7f4fc6d53700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc009e2b0 con 0x7f4fc009d630 2023-10-03T10:16:11.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.477+0000 7f4fb7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fc009d630 0x7f4fc00a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:11.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.477+0000 7f4fb7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fc009d630 0x7f4fc00a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53356/0 (socket says 172.21.15.3:53356) 2023-10-03T10:16:11.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.477+0000 7f4fb7fff700 1 -- 172.21.15.3:0/1311175714 learned_addr learned my addr 172.21.15.3:0/1311175714 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:11.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.477+0000 7f4fc4aef700 1 --2- 172.21.15.3:0/1311175714 >> v2:172.21.15.3:3300/0 conn(0x7f4fc00a0a30 0x7f4fc00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:11.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.478+0000 7f4fb7fff700 1 -- 172.21.15.3:0/1311175714 >> v2:172.21.15.3:3300/0 conn(0x7f4fc00a0a30 msgr2=0x7f4fc00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:11.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.478+0000 7f4fb7fff700 1 --2- 172.21.15.3:0/1311175714 >> v2:172.21.15.3:3300/0 conn(0x7f4fc00a0a30 0x7f4fc00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.478+0000 7f4fb7fff700 1 -- 172.21.15.3:0/1311175714 >> v1:172.21.15.3:6789/0 conn(0x7f4fc0133930 legacy=0x7f4fc0133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:11.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.478+0000 7f4fb7fff700 1 -- 172.21.15.3:0/1311175714 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc009dc50 con 0x7f4fc009d630 2023-10-03T10:16:11.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.478+0000 7f4fb7fff700 1 --2- 172.21.15.3:0/1311175714 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fc009d630 0x7f4fc00a1340 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4fb00098f0 tx=0x7f4fb0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c9b1a23c3377068 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:11.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.478+0000 7f4fb77fe700 1 -- 172.21.15.3:0/1311175714 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0011830 con 0x7f4fc009d630 2023-10-03T10:16:11.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.478+0000 7f4fb77fe700 1 -- 172.21.15.3:0/1311175714 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb0011990 con 0x7f4fc009d630 2023-10-03T10:16:11.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.478+0000 7f4fb77fe700 1 -- 172.21.15.3:0/1311175714 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0011d20 con 0x7f4fc009d630 2023-10-03T10:16:11.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.479+0000 7f4fc6d53700 1 -- 172.21.15.3:0/1311175714 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fc009d630 msgr2=0x7f4fc00a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:11.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.479+0000 7f4fc6d53700 1 --2- 172.21.15.3:0/1311175714 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fc009d630 0x7f4fc00a1340 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4fb00098f0 tx=0x7f4fb0009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:11.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.479+0000 7f4fc6d53700 1 -- 172.21.15.3:0/1311175714 shutdown_connections 2023-10-03T10:16:11.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.479+0000 7f4fc6d53700 1 --2- 172.21.15.3:0/1311175714 >> v2:172.21.15.3:3300/0 conn(0x7f4fc00a0a30 0x7f4fc00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.479+0000 7f4fc6d53700 1 --2- 172.21.15.3:0/1311175714 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fc009d630 0x7f4fc00a1340 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.479+0000 7f4fc6d53700 1 -- 172.21.15.3:0/1311175714 >> 172.21.15.3:0/1311175714 conn(0x7f4fc00a3050 msgr2=0x7f4fc00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:11.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.479+0000 7f4fc6d53700 1 -- 172.21.15.3:0/1311175714 shutdown_connections 2023-10-03T10:16:11.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.479+0000 7f4fc6d53700 1 -- 172.21.15.3:0/1311175714 wait complete. 2023-10-03T10:16:11.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.480+0000 7f4fc6d53700 1 Processor -- start 2023-10-03T10:16:11.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.480+0000 7f4fc6d53700 1 -- start start 2023-10-03T10:16:11.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.481+0000 7f4fc6d53700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fc009d630 0x7f4fc0121100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:11.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.481+0000 7f4fc6d53700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fc00a0a30 0x7f4fc0121640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:11.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.481+0000 7f4fc6d53700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fc0133930 0x7f4fc0121b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:11.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.481+0000 7f4fc6d53700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc01225b0 con 0x7f4fc009d630 2023-10-03T10:16:11.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.481+0000 7f4fc6d53700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc0124bd0 con 0x7f4fc00a0a30 2023-10-03T10:16:11.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.481+0000 7f4fc6d53700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc0124630 con 0x7f4fc0133930 2023-10-03T10:16:11.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.481+0000 7f4fc52f0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fc0133930 0x7f4fc0121b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:11.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.481+0000 7f4fc52f0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fc0133930 0x7f4fc0121b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52420/0 (socket says 172.21.15.3:52420) 2023-10-03T10:16:11.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.481+0000 7f4fc52f0700 1 -- 172.21.15.3:0/162494546 learned_addr learned my addr 172.21.15.3:0/162494546 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:11.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.481+0000 7f4fc4aef700 1 --2- 172.21.15.3:0/162494546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fc009d630 0x7f4fc0121100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:11.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.481+0000 7f4fb7fff700 1 --2- 172.21.15.3:0/162494546 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fc00a0a30 0x7f4fc0121640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:11.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.481+0000 7f4fc52f0700 1 -- 172.21.15.3:0/162494546 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fc00a0a30 msgr2=0x7f4fc0121640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:11.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.481+0000 7f4fc52f0700 1 --2- 172.21.15.3:0/162494546 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fc00a0a30 0x7f4fc0121640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.481+0000 7f4fc52f0700 1 -- 172.21.15.3:0/162494546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fc009d630 msgr2=0x7f4fc0121100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:11.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.481+0000 7f4fc52f0700 1 --2- 172.21.15.3:0/162494546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fc009d630 0x7f4fc0121100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.481+0000 7f4fc52f0700 1 -- 172.21.15.3:0/162494546 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb0009550 con 0x7f4fc0133930 2023-10-03T10:16:11.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.481+0000 7f4fb7fff700 1 --2- 172.21.15.3:0/162494546 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fc00a0a30 0x7f4fc0121640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:16:11.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.482+0000 7f4fc52f0700 1 --2- 172.21.15.3:0/162494546 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fc0133930 0x7f4fc0121b80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800cc50 tx=0x7f4fb800cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:11.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.482+0000 7f4fb57fa700 1 -- 172.21.15.3:0/162494546 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8003c00 con 0x7f4fc0133930 2023-10-03T10:16:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.482+0000 7f4fb57fa700 1 -- 172.21.15.3:0/162494546 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb8003d60 con 0x7f4fc0133930 2023-10-03T10:16:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.482+0000 7f4fc6d53700 1 -- 172.21.15.3:0/162494546 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc0122dc0 con 0x7f4fc0133930 2023-10-03T10:16:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.482+0000 7f4fb57fa700 1 -- 172.21.15.3:0/162494546 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb800ab60 con 0x7f4fc0133930 2023-10-03T10:16:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.482+0000 7f4fc6d53700 1 -- 172.21.15.3:0/162494546 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc0123ac0 con 0x7f4fc0133930 2023-10-03T10:16:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.483+0000 7f4fb57fa700 1 -- 172.21.15.3:0/162494546 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4fb800acc0 con 0x7f4fc0133930 2023-10-03T10:16:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.483+0000 7f4fb57fa700 1 -- 172.21.15.3:0/162494546 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4fb803f5b0 con 0x7f4fc0133930 2023-10-03T10:16:11.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.484+0000 7f4fc6d53700 1 -- 172.21.15.3:0/162494546 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc011df60 con 0x7f4fc0133930 2023-10-03T10:16:11.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.489+0000 7f4fb57fa700 1 -- 172.21.15.3:0/162494546 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4fb8012420 con 0x7f4fc0133930 2023-10-03T10:16:11.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.547+0000 7fa427f05700 1 Processor -- start 2023-10-03T10:16:11.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.548+0000 7fa427f05700 1 -- start start 2023-10-03T10:16:11.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.549+0000 7fa427f05700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa4200a0a30 0x7fa4200a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:11.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.549+0000 7fa427f05700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa42009d630 0x7fa4200a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:11.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.549+0000 7fa427f05700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa420133930 0x7fa420133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:11.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.549+0000 7fa427f05700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4200a2a70 con 0x7fa4200a0a30 2023-10-03T10:16:11.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.549+0000 7fa427f05700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4200a01d0 con 0x7fa420133930 2023-10-03T10:16:11.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.549+0000 7fa427f05700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa42009e2b0 con 0x7fa42009d630 2023-10-03T10:16:11.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.549+0000 7fa4254a0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa42009d630 0x7fa4200a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:11.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.549+0000 7fa4254a0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa42009d630 0x7fa4200a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52436/0 (socket says 172.21.15.3:52436) 2023-10-03T10:16:11.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.549+0000 7fa4254a0700 1 -- 172.21.15.3:0/1755035214 learned_addr learned my addr 172.21.15.3:0/1755035214 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:11.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.549+0000 7fa4264a2700 1 --2- 172.21.15.3:0/1755035214 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa420133930 0x7fa420133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:11.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.549+0000 7fa425ca1700 1 --2- 172.21.15.3:0/1755035214 >> v2:172.21.15.3:3300/0 conn(0x7fa4200a0a30 0x7fa4200a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:11.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.549+0000 7fa4254a0700 1 -- 172.21.15.3:0/1755035214 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa420133930 msgr2=0x7fa420133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:11.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.549+0000 7fa4254a0700 1 --2- 172.21.15.3:0/1755035214 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa420133930 0x7fa420133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.549+0000 7fa4254a0700 1 -- 172.21.15.3:0/1755035214 >> v2:172.21.15.3:3300/0 conn(0x7fa4200a0a30 msgr2=0x7fa4200a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:11.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.549+0000 7fa4254a0700 1 --2- 172.21.15.3:0/1755035214 >> v2:172.21.15.3:3300/0 conn(0x7fa4200a0a30 0x7fa4200a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.549+0000 7fa4254a0700 1 -- 172.21.15.3:0/1755035214 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42009dc50 con 0x7fa42009d630 2023-10-03T10:16:11.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.550+0000 7fa425ca1700 1 --2- 172.21.15.3:0/1755035214 >> v2:172.21.15.3:3300/0 conn(0x7fa4200a0a30 0x7fa4200a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:16:11.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.550+0000 7fa4254a0700 1 --2- 172.21.15.3:0/1755035214 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa42009d630 0x7fa4200a1340 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa4100098f0 tx=0x7fa410009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a1b1eff8cb56970 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:11.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.550+0000 7fa424c9f700 1 -- 172.21.15.3:0/1755035214 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa410014e90 con 0x7fa42009d630 2023-10-03T10:16:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.550+0000 7fa424c9f700 1 -- 172.21.15.3:0/1755035214 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41001c440 con 0x7fa42009d630 2023-10-03T10:16:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.550+0000 7fa424c9f700 1 -- 172.21.15.3:0/1755035214 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41001c7d0 con 0x7fa42009d630 2023-10-03T10:16:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.551+0000 7fa427f05700 1 -- 172.21.15.3:0/1755035214 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa42009d630 msgr2=0x7fa4200a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.551+0000 7fa427f05700 1 --2- 172.21.15.3:0/1755035214 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa42009d630 0x7fa4200a1340 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa4100098f0 tx=0x7fa410009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:11.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.551+0000 7fa427f05700 1 -- 172.21.15.3:0/1755035214 shutdown_connections 2023-10-03T10:16:11.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.551+0000 7fa427f05700 1 --2- 172.21.15.3:0/1755035214 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa420133930 0x7fa420133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.551+0000 7fa427f05700 1 --2- 172.21.15.3:0/1755035214 >> v2:172.21.15.3:3300/0 conn(0x7fa4200a0a30 0x7fa4200a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.551+0000 7fa427f05700 1 --2- 172.21.15.3:0/1755035214 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa42009d630 0x7fa4200a1340 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.551+0000 7fa427f05700 1 -- 172.21.15.3:0/1755035214 >> 172.21.15.3:0/1755035214 conn(0x7fa4200a3050 msgr2=0x7fa4200a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:11.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.551+0000 7fa427f05700 1 -- 172.21.15.3:0/1755035214 shutdown_connections 2023-10-03T10:16:11.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.551+0000 7fa427f05700 1 -- 172.21.15.3:0/1755035214 wait complete. 2023-10-03T10:16:11.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.552+0000 7fa427f05700 1 Processor -- start 2023-10-03T10:16:11.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.553+0000 7fa427f05700 1 -- start start 2023-10-03T10:16:11.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.553+0000 7fa427f05700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa42009d630 0x7fa42011c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:11.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.553+0000 7fa427f05700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4200a0a30 0x7fa42011cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:11.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.553+0000 7fa427f05700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa420133930 0x7fa42011d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:11.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.553+0000 7fa427f05700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa420071c60 con 0x7fa4200a0a30 2023-10-03T10:16:11.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.553+0000 7fa427f05700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42011f490 con 0x7fa420133930 2023-10-03T10:16:11.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.553+0000 7fa427f05700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa42011d970 con 0x7fa42009d630 2023-10-03T10:16:11.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.553+0000 7fa425ca1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa42009d630 0x7fa42011c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:11.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.553+0000 7fa4264a2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa420133930 0x7fa42011d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:11.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.553+0000 7fa425ca1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa42009d630 0x7fa42011c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52440/0 (socket says 172.21.15.3:52440) 2023-10-03T10:16:11.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.553+0000 7fa4264a2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa420133930 0x7fa42011d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53368/0 (socket says 172.21.15.3:53368) 2023-10-03T10:16:11.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.553+0000 7fa425ca1700 1 -- 172.21.15.3:0/104929996 learned_addr learned my addr 172.21.15.3:0/104929996 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:11.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.554+0000 7fa4254a0700 1 --2- 172.21.15.3:0/104929996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4200a0a30 0x7fa42011cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:11.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.554+0000 7fa4264a2700 1 -- 172.21.15.3:0/104929996 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa42009d630 msgr2=0x7fa42011c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:11.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.554+0000 7fa4264a2700 1 --2- 172.21.15.3:0/104929996 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa42009d630 0x7fa42011c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.554+0000 7fa4264a2700 1 -- 172.21.15.3:0/104929996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4200a0a30 msgr2=0x7fa42011cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:11.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.554+0000 7fa4264a2700 1 --2- 172.21.15.3:0/104929996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4200a0a30 0x7fa42011cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.554+0000 7fa4264a2700 1 -- 172.21.15.3:0/104929996 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa410009550 con 0x7fa420133930 2023-10-03T10:16:11.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.554+0000 7fa4264a2700 1 --2- 172.21.15.3:0/104929996 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa420133930 0x7fa42011d240 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa418009d90 tx=0x7fa41800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:11.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.554+0000 7fa4167fc700 1 -- 172.21.15.3:0/104929996 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa418009c00 con 0x7fa420133930 2023-10-03T10:16:11.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.555+0000 7fa4167fc700 1 -- 172.21.15.3:0/104929996 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41800f040 con 0x7fa420133930 2023-10-03T10:16:11.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.555+0000 7fa4167fc700 1 -- 172.21.15.3:0/104929996 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa418014480 con 0x7fa420133930 2023-10-03T10:16:11.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.555+0000 7fa427f05700 1 -- 172.21.15.3:0/104929996 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa42011ef20 con 0x7fa420133930 2023-10-03T10:16:11.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.555+0000 7fa427f05700 1 -- 172.21.15.3:0/104929996 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa420071160 con 0x7fa420133930 2023-10-03T10:16:11.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.556+0000 7fa4167fc700 1 -- 172.21.15.3:0/104929996 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa4180145e0 con 0x7fa420133930 2023-10-03T10:16:11.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.556+0000 7fa4167fc700 1 -- 172.21.15.3:0/104929996 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa418040300 con 0x7fa420133930 2023-10-03T10:16:11.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.556+0000 7fa427f05700 1 -- 172.21.15.3:0/104929996 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4200747f0 con 0x7fa420133930 2023-10-03T10:16:11.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.561+0000 7fa4167fc700 1 -- 172.21.15.3:0/104929996 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa418018070 con 0x7fa420133930 2023-10-03T10:16:11.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.600+0000 7fd95b481700 1 -- 172.21.15.3:0/2313787510 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd934005590 con 0x7fd954131d10 2023-10-03T10:16:11.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.600+0000 7fd949ffb700 1 -- 172.21.15.3:0/2313787510 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd94c02a090 con 0x7fd954131d10 2023-10-03T10:16:11.601 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:11.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.603+0000 7fd95b481700 1 -- 172.21.15.3:0/2313787510 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd954131d10 msgr2=0x7fd954120510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:11.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.603+0000 7fd95b481700 1 --2- 172.21.15.3:0/2313787510 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd954131d10 0x7fd954120510 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd94c00efc0 tx=0x7fd94c00c3a0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.603+0000 7fd95b481700 1 -- 172.21.15.3:0/2313787510 shutdown_connections 2023-10-03T10:16:11.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.603+0000 7fd95b481700 1 --2- 172.21.15.3:0/2313787510 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd954131d10 0x7fd954120510 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.603+0000 7fd95b481700 1 --2- 172.21.15.3:0/2313787510 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd95406a770 0x7fd9541c62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.603+0000 7fd95b481700 1 --2- 172.21.15.3:0/2313787510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd95406b080 0x7fd9541c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.603+0000 7fd95b481700 1 -- 172.21.15.3:0/2313787510 >> 172.21.15.3:0/2313787510 conn(0x7fd954066f50 msgr2=0x7fd954070000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:11.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.603+0000 7fd95b481700 1 -- 172.21.15.3:0/2313787510 shutdown_connections 2023-10-03T10:16:11.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.603+0000 7fd95b481700 1 -- 172.21.15.3:0/2313787510 wait complete. 2023-10-03T10:16:11.617 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:11.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.700+0000 7f4fc6d53700 1 -- 172.21.15.3:0/162494546 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4fc0124850 con 0x7f4fc0133930 2023-10-03T10:16:11.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.701+0000 7f4fb57fa700 1 -- 172.21.15.3:0/162494546 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4fc0124850 con 0x7f4fc0133930 2023-10-03T10:16:11.702 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.703+0000 7f4fc6d53700 1 -- 172.21.15.3:0/162494546 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fc0133930 msgr2=0x7f4fc0121b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.703+0000 7f4fc6d53700 1 --2- 172.21.15.3:0/162494546 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fc0133930 0x7f4fc0121b80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800cc50 tx=0x7f4fb800cf60 comp rx=0 tx=0).stop 2023-10-03T10:16:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.703+0000 7f4fc6d53700 1 -- 172.21.15.3:0/162494546 shutdown_connections 2023-10-03T10:16:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.703+0000 7f4fc6d53700 1 --2- 172.21.15.3:0/162494546 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4fc0133930 0x7f4fc0121b80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.703+0000 7f4fc6d53700 1 --2- 172.21.15.3:0/162494546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fc009d630 0x7f4fc0121100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.703+0000 7f4fc6d53700 1 --2- 172.21.15.3:0/162494546 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4fc00a0a30 0x7f4fc0121640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.703+0000 7f4fc6d53700 1 -- 172.21.15.3:0/162494546 >> 172.21.15.3:0/162494546 conn(0x7f4fc00a3050 msgr2=0x7f4fc0133210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.704+0000 7f4fc6d53700 1 -- 172.21.15.3:0/162494546 shutdown_connections 2023-10-03T10:16:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.704+0000 7f4fc6d53700 1 -- 172.21.15.3:0/162494546 wait complete. 2023-10-03T10:16:11.717 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:11.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.739+0000 7f525a090700 1 -- 172.21.15.3:0/1575437845 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5254074ea0 con 0x7f525409e440 2023-10-03T10:16:11.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.740+0000 7f523bfff700 1 -- 172.21.15.3:0/1575437845 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f524c02a070 con 0x7f525409e440 2023-10-03T10:16:11.740 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:11.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.742+0000 7f525a090700 1 -- 172.21.15.3:0/1575437845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f525409e440 msgr2=0x7f52541a6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:11.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.742+0000 7f525a090700 1 --2- 172.21.15.3:0/1575437845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f525409e440 0x7f52541a6a20 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f524c009dd0 tx=0x7f524c00c3a0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.742+0000 7f525a090700 1 -- 172.21.15.3:0/1575437845 shutdown_connections 2023-10-03T10:16:11.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.742+0000 7f525a090700 1 --2- 172.21.15.3:0/1575437845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f525409e440 0x7f52541a6a20 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.742+0000 7f525a090700 1 --2- 172.21.15.3:0/1575437845 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f525409d6f0 0x7f52541a5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.742+0000 7f525a090700 1 --2- 172.21.15.3:0/1575437845 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f525409e000 0x7f52541a64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.742+0000 7f525a090700 1 -- 172.21.15.3:0/1575437845 >> 172.21.15.3:0/1575437845 conn(0x7f5254066f50 msgr2=0x7f52541baed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:11.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.742+0000 7f525a090700 1 -- 172.21.15.3:0/1575437845 shutdown_connections 2023-10-03T10:16:11.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.742+0000 7f525a090700 1 -- 172.21.15.3:0/1575437845 wait complete. 2023-10-03T10:16:11.756 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:11.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.805+0000 7fa427f05700 1 -- 172.21.15.3:0/104929996 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa420072980 con 0x7fa420133930 2023-10-03T10:16:11.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.805+0000 7fa4167fc700 1 -- 172.21.15.3:0/104929996 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa41802c090 con 0x7fa420133930 2023-10-03T10:16:11.806 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:11.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.808+0000 7fa427f05700 1 -- 172.21.15.3:0/104929996 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa420133930 msgr2=0x7fa42011d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:11.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.808+0000 7fa427f05700 1 --2- 172.21.15.3:0/104929996 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa420133930 0x7fa42011d240 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa418009d90 tx=0x7fa41800c3a0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.808+0000 7fa427f05700 1 -- 172.21.15.3:0/104929996 shutdown_connections 2023-10-03T10:16:11.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.808+0000 7fa427f05700 1 --2- 172.21.15.3:0/104929996 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa420133930 0x7fa42011d240 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.808+0000 7fa427f05700 1 --2- 172.21.15.3:0/104929996 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa42009d630 0x7fa42011c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.808+0000 7fa427f05700 1 --2- 172.21.15.3:0/104929996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4200a0a30 0x7fa42011cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:11.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.808+0000 7fa427f05700 1 -- 172.21.15.3:0/104929996 >> 172.21.15.3:0/104929996 conn(0x7fa4200a3050 msgr2=0x7fa420132d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:11.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.808+0000 7fa427f05700 1 -- 172.21.15.3:0/104929996 shutdown_connections 2023-10-03T10:16:11.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:11.808+0000 7fa427f05700 1 -- 172.21.15.3:0/104929996 wait complete. 2023-10-03T10:16:11.821 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:12.618 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:12.719 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:12.757 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:12.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.765+0000 7fbee76cf700 1 Processor -- start 2023-10-03T10:16:12.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.766+0000 7fbee76cf700 1 -- start start 2023-10-03T10:16:12.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.766+0000 7fbee76cf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbee009d6f0 0x7fbee009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:12.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.766+0000 7fbee76cf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbee009e000 0x7fbee0130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:12.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.766+0000 7fbee76cf700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbee009f130 con 0x7fbee009e410 2023-10-03T10:16:12.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.766+0000 7fbee76cf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee009f520 con 0x7fbee009e000 2023-10-03T10:16:12.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.766+0000 7fbee76cf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbee009e930 con 0x7fbee009d6f0 2023-10-03T10:16:12.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.766+0000 7fbee5c6c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbee009e410 0x7fbee0130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38996/0 (socket says 172.21.15.3:38996) 2023-10-03T10:16:12.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.766+0000 7fbee5c6c700 1 -- 172.21.15.3:0/227946936 learned_addr learned my addr 172.21.15.3:0/227946936 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:12.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.766+0000 7fbee546b700 1 --2- 172.21.15.3:0/227946936 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbee009d6f0 0x7fbee009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:12.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.766+0000 7fbee4c6a700 1 --2- 172.21.15.3:0/227946936 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbee009e000 0x7fbee0130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:12.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.767+0000 7fbee546b700 1 -- 172.21.15.3:0/227946936 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbee009e000 msgr2=0x7fbee0130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:12.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.767+0000 7fbee546b700 1 --2- 172.21.15.3:0/227946936 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbee009e000 0x7fbee0130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:12.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.767+0000 7fbee546b700 1 -- 172.21.15.3:0/227946936 >> v1:172.21.15.3:6789/0 conn(0x7fbee009e410 legacy=0x7fbee0130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:12.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.767+0000 7fbed7fff700 1 -- 172.21.15.3:0/227946936 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 689632862 0 0) 0x7fbee009f130 con 0x7fbee009e410 2023-10-03T10:16:12.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.767+0000 7fbee546b700 1 -- 172.21.15.3:0/227946936 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee009b2d0 con 0x7fbee009d6f0 2023-10-03T10:16:12.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.767+0000 7fbee4c6a700 1 --2- 172.21.15.3:0/227946936 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbee009e000 0x7fbee0130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:12.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.767+0000 7fbee546b700 1 --2- 172.21.15.3:0/227946936 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbee009d6f0 0x7fbee009dac0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0098a0 tx=0x7fbedc009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a7eea6adaa649d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:12.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.767+0000 7fbed7fff700 1 -- 172.21.15.3:0/227946936 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc011830 con 0x7fbee009d6f0 2023-10-03T10:16:12.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.767+0000 7fbed7fff700 1 -- 172.21.15.3:0/227946936 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbedc011990 con 0x7fbee009d6f0 2023-10-03T10:16:12.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.767+0000 7fbed7fff700 1 -- 172.21.15.3:0/227946936 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc011d20 con 0x7fbee009d6f0 2023-10-03T10:16:12.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.768+0000 7fbee76cf700 1 -- 172.21.15.3:0/227946936 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbee009d6f0 msgr2=0x7fbee009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:12.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.768+0000 7fbee76cf700 1 --2- 172.21.15.3:0/227946936 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbee009d6f0 0x7fbee009dac0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0098a0 tx=0x7fbedc009bb0 comp rx=0 tx=0).stop 2023-10-03T10:16:12.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.768+0000 7fbee76cf700 1 -- 172.21.15.3:0/227946936 shutdown_connections 2023-10-03T10:16:12.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.768+0000 7fbee76cf700 1 --2- 172.21.15.3:0/227946936 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbee009d6f0 0x7fbee009dac0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:12.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.768+0000 7fbee76cf700 1 --2- 172.21.15.3:0/227946936 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbee009e000 0x7fbee0130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:12.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.768+0000 7fbee76cf700 1 -- 172.21.15.3:0/227946936 >> 172.21.15.3:0/227946936 conn(0x7fbee0066f50 msgr2=0x7fbee009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:12.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.768+0000 7fbee76cf700 1 -- 172.21.15.3:0/227946936 shutdown_connections 2023-10-03T10:16:12.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.768+0000 7fbee76cf700 1 -- 172.21.15.3:0/227946936 wait complete. 2023-10-03T10:16:12.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.770+0000 7fbee76cf700 1 Processor -- start 2023-10-03T10:16:12.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.770+0000 7fbee76cf700 1 -- start start 2023-10-03T10:16:12.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.770+0000 7fbee76cf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbee009d6f0 0x7fbee011de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:12.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.770+0000 7fbee76cf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbee009e000 0x7fbee011e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:12.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.770+0000 7fbee76cf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee009e410 0x7fbee011e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:12.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.770+0000 7fbee76cf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee011fa20 con 0x7fbee009e410 2023-10-03T10:16:12.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.771+0000 7fbee76cf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee0120c70 con 0x7fbee009d6f0 2023-10-03T10:16:12.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.771+0000 7fbee76cf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbee0120010 con 0x7fbee009e000 2023-10-03T10:16:12.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.771+0000 7fbee4c6a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbee009e000 0x7fbee011e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:12.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.771+0000 7fbee4c6a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbee009e000 0x7fbee011e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52462/0 (socket says 172.21.15.3:52462) 2023-10-03T10:16:12.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.771+0000 7fbee4c6a700 1 -- 172.21.15.3:0/3094902139 learned_addr learned my addr 172.21.15.3:0/3094902139 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:12.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.771+0000 7fbee546b700 1 --2- 172.21.15.3:0/3094902139 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbee009d6f0 0x7fbee011de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:12.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.771+0000 7fbee5c6c700 1 --2- 172.21.15.3:0/3094902139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee009e410 0x7fbee011e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:12.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.771+0000 7fbee4c6a700 1 -- 172.21.15.3:0/3094902139 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbee009d6f0 msgr2=0x7fbee011de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:12.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.771+0000 7fbee4c6a700 1 --2- 172.21.15.3:0/3094902139 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbee009d6f0 0x7fbee011de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:12.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.771+0000 7fbee4c6a700 1 -- 172.21.15.3:0/3094902139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee009e410 msgr2=0x7fbee011e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:12.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.771+0000 7fbee4c6a700 1 --2- 172.21.15.3:0/3094902139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee009e410 0x7fbee011e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:12.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.771+0000 7fbee4c6a700 1 -- 172.21.15.3:0/3094902139 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed00094a0 con 0x7fbee009e000 2023-10-03T10:16:12.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.771+0000 7fbee546b700 1 --2- 172.21.15.3:0/3094902139 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbee009d6f0 0x7fbee011de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:12.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.771+0000 7fbee4c6a700 1 --2- 172.21.15.3:0/3094902139 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbee009e000 0x7fbee011e350 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbed000ebb0 tx=0x7fbed000eec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:12.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.772+0000 7fbed5ffb700 1 -- 172.21.15.3:0/3094902139 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed000c850 con 0x7fbee009e000 2023-10-03T10:16:12.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.772+0000 7fbed5ffb700 1 -- 172.21.15.3:0/3094902139 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed000c9b0 con 0x7fbee009e000 2023-10-03T10:16:12.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.772+0000 7fbee76cf700 1 -- 172.21.15.3:0/3094902139 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbedc0094a0 con 0x7fbee009e000 2023-10-03T10:16:12.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.772+0000 7fbed5ffb700 1 -- 172.21.15.3:0/3094902139 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed000cd40 con 0x7fbee009e000 2023-10-03T10:16:12.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.772+0000 7fbee76cf700 1 -- 172.21.15.3:0/3094902139 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee009c700 con 0x7fbee009e000 2023-10-03T10:16:12.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.773+0000 7fbed5ffb700 1 -- 172.21.15.3:0/3094902139 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbed0013470 con 0x7fbee009e000 2023-10-03T10:16:12.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.773+0000 7fbed5ffb700 1 -- 172.21.15.3:0/3094902139 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fbed003fb30 con 0x7fbee009e000 2023-10-03T10:16:12.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.773+0000 7fbee76cf700 1 -- 172.21.15.3:0/3094902139 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbee011ac20 con 0x7fbee009e000 2023-10-03T10:16:12.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.779+0000 7fbed5ffb700 1 -- 172.21.15.3:0/3094902139 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbed001b3b0 con 0x7fbee009e000 2023-10-03T10:16:12.822 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:12.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.876+0000 7fd9249be700 1 Processor -- start 2023-10-03T10:16:12.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.876+0000 7fd9249be700 1 -- start start 2023-10-03T10:16:12.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.877+0000 7fd9249be700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd92009e1d0 0x7fd920138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:12.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.877+0000 7fd9249be700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd920139170 0x7fd92013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:12.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.877+0000 7fd9249be700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd92009d2f0 con 0x7fd92009d6f0 2023-10-03T10:16:12.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.878+0000 7fd9249be700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd92009b0f0 con 0x7fd92009e1d0 2023-10-03T10:16:12.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.878+0000 7fd9249be700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd92009a810 con 0x7fd920139170 2023-10-03T10:16:12.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.878+0000 7fd91e59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd92009d6f0 0x7fd92009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38998/0 (socket says 172.21.15.3:38998) 2023-10-03T10:16:12.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.878+0000 7fd91e59c700 1 -- 172.21.15.3:0/549832393 learned_addr learned my addr 172.21.15.3:0/549832393 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:12.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.878+0000 7fd91ed9d700 1 --2- 172.21.15.3:0/549832393 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd920139170 0x7fd92013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:12.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.878+0000 7fd91dd9b700 1 --2- 172.21.15.3:0/549832393 >> v2:172.21.15.3:3300/0 conn(0x7fd92009e1d0 0x7fd920138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:12.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.878+0000 7fd91ed9d700 1 -- 172.21.15.3:0/549832393 >> v2:172.21.15.3:3300/0 conn(0x7fd92009e1d0 msgr2=0x7fd920138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:12.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.878+0000 7fd91ed9d700 1 --2- 172.21.15.3:0/549832393 >> v2:172.21.15.3:3300/0 conn(0x7fd92009e1d0 0x7fd920138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:12.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.878+0000 7fd91ed9d700 1 -- 172.21.15.3:0/549832393 >> v1:172.21.15.3:6789/0 conn(0x7fd92009d6f0 legacy=0x7fd92009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:12.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.878+0000 7fd91ed9d700 1 -- 172.21.15.3:0/549832393 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92009a470 con 0x7fd920139170 2023-10-03T10:16:12.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.878+0000 7fd91ed9d700 1 --2- 172.21.15.3:0/549832393 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd920139170 0x7fd92013b550 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd918009d90 tx=0x7fd91800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e9b7ed00ad8d1d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:12.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.879+0000 7fd91d59a700 1 -- 172.21.15.3:0/549832393 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91800e070 con 0x7fd920139170 2023-10-03T10:16:12.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.879+0000 7fd91d59a700 1 -- 172.21.15.3:0/549832393 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd91800bb20 con 0x7fd920139170 2023-10-03T10:16:12.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.879+0000 7fd91d59a700 1 -- 172.21.15.3:0/549832393 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd918012430 con 0x7fd920139170 2023-10-03T10:16:12.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.880+0000 7fd9249be700 1 -- 172.21.15.3:0/549832393 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd920139170 msgr2=0x7fd92013b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:12.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.880+0000 7fd9249be700 1 --2- 172.21.15.3:0/549832393 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd920139170 0x7fd92013b550 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd918009d90 tx=0x7fd91800b330 comp rx=0 tx=0).stop 2023-10-03T10:16:12.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.880+0000 7fd9249be700 1 -- 172.21.15.3:0/549832393 shutdown_connections 2023-10-03T10:16:12.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.880+0000 7fd9249be700 1 --2- 172.21.15.3:0/549832393 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd920139170 0x7fd92013b550 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:12.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.880+0000 7fd9249be700 1 --2- 172.21.15.3:0/549832393 >> v2:172.21.15.3:3300/0 conn(0x7fd92009e1d0 0x7fd920138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:12.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.880+0000 7fd9249be700 1 -- 172.21.15.3:0/549832393 >> 172.21.15.3:0/549832393 conn(0x7fd920066f50 msgr2=0x7fd92009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:12.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.884+0000 7fd9249be700 1 -- 172.21.15.3:0/549832393 shutdown_connections 2023-10-03T10:16:12.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.884+0000 7fd9249be700 1 -- 172.21.15.3:0/549832393 wait complete. 2023-10-03T10:16:12.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.885+0000 7fd9249be700 1 Processor -- start 2023-10-03T10:16:12.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.885+0000 7fd9249be700 1 -- start start 2023-10-03T10:16:12.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.885+0000 7fd9249be700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd92009d6f0 0x7fd92011e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:12.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.885+0000 7fd9249be700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd92009e1d0 0x7fd92011ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:12.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.885+0000 7fd9249be700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd920139170 0x7fd92011f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:12.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.885+0000 7fd9249be700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92009c640 con 0x7fd92009e1d0 2023-10-03T10:16:12.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.885+0000 7fd9249be700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9201203c0 con 0x7fd92009d6f0 2023-10-03T10:16:12.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.885+0000 7fd9249be700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9201222d0 con 0x7fd920139170 2023-10-03T10:16:12.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.886+0000 7fd91e59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd92009d6f0 0x7fd92011e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:12.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.886+0000 7fd91e59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd92009d6f0 0x7fd92011e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53398/0 (socket says 172.21.15.3:53398) 2023-10-03T10:16:12.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.886+0000 7fd91e59c700 1 -- 172.21.15.3:0/379661969 learned_addr learned my addr 172.21.15.3:0/379661969 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:12.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.886+0000 7fd91ed9d700 1 --2- 172.21.15.3:0/379661969 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd920139170 0x7fd92011f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:12.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.886+0000 7fd91e59c700 1 -- 172.21.15.3:0/379661969 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd920139170 msgr2=0x7fd92011f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:12.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.886+0000 7fd91e59c700 1 --2- 172.21.15.3:0/379661969 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd920139170 0x7fd92011f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:12.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.886+0000 7fd91dd9b700 1 --2- 172.21.15.3:0/379661969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd92009e1d0 0x7fd92011ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:12.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.886+0000 7fd91e59c700 1 -- 172.21.15.3:0/379661969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd92009e1d0 msgr2=0x7fd92011ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:12.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.886+0000 7fd91e59c700 1 --2- 172.21.15.3:0/379661969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd92009e1d0 0x7fd92011ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:12.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.886+0000 7fd91e59c700 1 -- 172.21.15.3:0/379661969 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd918009a70 con 0x7fd92009d6f0 2023-10-03T10:16:12.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.886+0000 7fd91dd9b700 1 --2- 172.21.15.3:0/379661969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd92009e1d0 0x7fd92011ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:16:12.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.887+0000 7fd91e59c700 1 --2- 172.21.15.3:0/379661969 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd92009d6f0 0x7fd92011e7b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd91000c480 tx=0x7fd91000c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:12.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.887+0000 7fd90effd700 1 -- 172.21.15.3:0/379661969 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910005d70 con 0x7fd92009d6f0 2023-10-03T10:16:12.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.887+0000 7fd90effd700 1 -- 172.21.15.3:0/379661969 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9100070d0 con 0x7fd92009d6f0 2023-10-03T10:16:12.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.887+0000 7fd9249be700 1 -- 172.21.15.3:0/379661969 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9201346a0 con 0x7fd92009d6f0 2023-10-03T10:16:12.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.887+0000 7fd90effd700 1 -- 172.21.15.3:0/379661969 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910007440 con 0x7fd92009d6f0 2023-10-03T10:16:12.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.887+0000 7fd9249be700 1 -- 172.21.15.3:0/379661969 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd920121ff0 con 0x7fd92009d6f0 2023-10-03T10:16:12.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.888+0000 7fd90effd700 1 -- 172.21.15.3:0/379661969 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd9100075a0 con 0x7fd92009d6f0 2023-10-03T10:16:12.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.888+0000 7fd90effd700 1 -- 172.21.15.3:0/379661969 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd91003eaf0 con 0x7fd92009d6f0 2023-10-03T10:16:12.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.888+0000 7fd9249be700 1 -- 172.21.15.3:0/379661969 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd920074ea0 con 0x7fd92009d6f0 2023-10-03T10:16:12.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.898+0000 7fd90effd700 1 -- 172.21.15.3:0/379661969 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd910017070 con 0x7fd92009d6f0 2023-10-03T10:16:12.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.922+0000 7f3626081700 1 Processor -- start 2023-10-03T10:16:12.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.922+0000 7f3626081700 1 -- start start 2023-10-03T10:16:12.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.923+0000 7f3626081700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3620131810 0x7f3620131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:12.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.923+0000 7f3626081700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f362009a580 0x7f3620139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:12.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.923+0000 7f3626081700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36201313c0 con 0x7f3620099a40 2023-10-03T10:16:12.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.923+0000 7f3626081700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362006e8c0 con 0x7f362009a580 2023-10-03T10:16:12.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.923+0000 7f3626081700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3620071d20 con 0x7f3620131810 2023-10-03T10:16:12.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.923+0000 7f361effd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3620099a40 0x7f3620099e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39000/0 (socket says 172.21.15.3:39000) 2023-10-03T10:16:12.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.923+0000 7f361effd700 1 -- 172.21.15.3:0/2580003085 learned_addr learned my addr 172.21.15.3:0/2580003085 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:12.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.924+0000 7f361ffff700 1 --2- 172.21.15.3:0/2580003085 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f362009a580 0x7f3620139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:12.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.924+0000 7f361f7fe700 1 --2- 172.21.15.3:0/2580003085 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3620131810 0x7f3620131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:12.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.924+0000 7f361e7fc700 1 -- 172.21.15.3:0/2580003085 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 984697874 0 0) 0x7f36201313c0 con 0x7f3620099a40 2023-10-03T10:16:12.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.924+0000 7f361e7fc700 1 -- 172.21.15.3:0/2580003085 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36080079c0 con 0x7f3620099a40 2023-10-03T10:16:12.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.924+0000 7f361ffff700 1 -- 172.21.15.3:0/2580003085 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3620131810 msgr2=0x7f3620131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:12.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.924+0000 7f361ffff700 1 --2- 172.21.15.3:0/2580003085 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3620131810 0x7f3620131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:12.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.924+0000 7f361ffff700 1 -- 172.21.15.3:0/2580003085 >> v1:172.21.15.3:6789/0 conn(0x7f3620099a40 legacy=0x7f3620099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:12.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.924+0000 7f361ffff700 1 -- 172.21.15.3:0/2580003085 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36200721a0 con 0x7f362009a580 2023-10-03T10:16:12.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.924+0000 7f361ffff700 1 --2- 172.21.15.3:0/2580003085 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f362009a580 0x7f3620139660 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3618009dc0 tx=0x7f361800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80d35a28643cd73f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:12.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.925+0000 7f361e7fc700 1 -- 172.21.15.3:0/2580003085 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361800e070 con 0x7f362009a580 2023-10-03T10:16:12.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.925+0000 7f361e7fc700 1 -- 172.21.15.3:0/2580003085 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361800bb20 con 0x7f362009a580 2023-10-03T10:16:12.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.925+0000 7f361e7fc700 1 -- 172.21.15.3:0/2580003085 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3618012430 con 0x7f362009a580 2023-10-03T10:16:12.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.925+0000 7f3626081700 1 -- 172.21.15.3:0/2580003085 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f362009a580 msgr2=0x7f3620139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:12.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.925+0000 7f3626081700 1 --2- 172.21.15.3:0/2580003085 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f362009a580 0x7f3620139660 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3618009dc0 tx=0x7f361800b330 comp rx=0 tx=0).stop 2023-10-03T10:16:12.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.925+0000 7f3626081700 1 -- 172.21.15.3:0/2580003085 shutdown_connections 2023-10-03T10:16:12.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.925+0000 7f3626081700 1 --2- 172.21.15.3:0/2580003085 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f362009a580 0x7f3620139660 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:12.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.925+0000 7f3626081700 1 --2- 172.21.15.3:0/2580003085 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3620131810 0x7f3620131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:12.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.925+0000 7f3626081700 1 -- 172.21.15.3:0/2580003085 >> 172.21.15.3:0/2580003085 conn(0x7f36200a2aa0 msgr2=0x7f36200a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:12.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.926+0000 7f3626081700 1 -- 172.21.15.3:0/2580003085 shutdown_connections 2023-10-03T10:16:12.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.926+0000 7f3626081700 1 -- 172.21.15.3:0/2580003085 wait complete. 2023-10-03T10:16:12.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.927+0000 7f3626081700 1 Processor -- start 2023-10-03T10:16:12.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.927+0000 7f3626081700 1 -- start start 2023-10-03T10:16:12.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.928+0000 7f3626081700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3620099a40 0x7f36201196d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:12.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.928+0000 7f3626081700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f362009a580 0x7f3620119c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:12.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.928+0000 7f361f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3620099a40 0x7f36201196d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:12.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.928+0000 7f361f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3620099a40 0x7f36201196d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53304/0 (socket says 172.21.15.3:53304) 2023-10-03T10:16:12.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.928+0000 7f361effd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f362009a580 0x7f3620119c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:12.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.928+0000 7f3626081700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3620131810 0x7f362011a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:12.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.928+0000 7f361effd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f362009a580 0x7f3620119c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53408/0 (socket says 172.21.15.3:53408) 2023-10-03T10:16:12.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.928+0000 7f3626081700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362011b250 con 0x7f3620099a40 2023-10-03T10:16:12.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.928+0000 7f361effd700 1 -- 172.21.15.3:0/2243668447 learned_addr learned my addr 172.21.15.3:0/2243668447 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:12.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.928+0000 7f3626081700 1 -- 172.21.15.3:0/2243668447 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362011b870 con 0x7f362009a580 2023-10-03T10:16:12.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.928+0000 7f3626081700 1 -- 172.21.15.3:0/2243668447 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f362011c4a0 con 0x7f3620131810 2023-10-03T10:16:12.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.928+0000 7f361ffff700 1 --2- 172.21.15.3:0/2243668447 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3620131810 0x7f362011a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:12.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.929+0000 7f361effd700 1 -- 172.21.15.3:0/2243668447 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3620131810 msgr2=0x7f362011a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:12.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.929+0000 7f361effd700 1 --2- 172.21.15.3:0/2243668447 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3620131810 0x7f362011a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:12.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.929+0000 7f361effd700 1 -- 172.21.15.3:0/2243668447 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3620099a40 msgr2=0x7f36201196d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:12.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.929+0000 7f361effd700 1 --2- 172.21.15.3:0/2243668447 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3620099a40 0x7f36201196d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:12.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.929+0000 7f361effd700 1 -- 172.21.15.3:0/2243668447 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3618009a70 con 0x7f362009a580 2023-10-03T10:16:12.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.929+0000 7f361ffff700 1 --2- 172.21.15.3:0/2243668447 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3620131810 0x7f362011a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:12.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.929+0000 7f361effd700 1 --2- 172.21.15.3:0/2243668447 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f362009a580 0x7f3620119c10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f361400c700 tx=0x7f361400ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:12.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.929+0000 7f3607fff700 1 -- 172.21.15.3:0/2243668447 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3614005d70 con 0x7f362009a580 2023-10-03T10:16:12.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.929+0000 7f3607fff700 1 -- 172.21.15.3:0/2243668447 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36140070d0 con 0x7f362009a580 2023-10-03T10:16:12.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.929+0000 7f3626081700 1 -- 172.21.15.3:0/2243668447 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362011ab50 con 0x7f362009a580 2023-10-03T10:16:12.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.929+0000 7f3607fff700 1 -- 172.21.15.3:0/2243668447 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3614007460 con 0x7f362009a580 2023-10-03T10:16:12.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.929+0000 7f3626081700 1 -- 172.21.15.3:0/2243668447 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36200a1b90 con 0x7f362009a580 2023-10-03T10:16:12.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.931+0000 7f3626081700 1 -- 172.21.15.3:0/2243668447 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362006eb60 con 0x7f362009a580 2023-10-03T10:16:12.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.931+0000 7f3607fff700 1 -- 172.21.15.3:0/2243668447 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f36140075c0 con 0x7f362009a580 2023-10-03T10:16:12.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.931+0000 7f3607fff700 1 -- 172.21.15.3:0/2243668447 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f361403ec40 con 0x7f362009a580 2023-10-03T10:16:12.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.940+0000 7f3607fff700 1 -- 172.21.15.3:0/2243668447 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f361400f260 con 0x7f362009a580 2023-10-03T10:16:12.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.987+0000 7f38aa7fb700 1 Processor -- start 2023-10-03T10:16:12.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.987+0000 7f38aa7fb700 1 -- start start 2023-10-03T10:16:12.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.988+0000 7f38aa7fb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f38a409e1d0 0x7f38a4135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:12.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.988+0000 7f38aa7fb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38a4135e50 0x7f38a4138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:12.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.988+0000 7f38aa7fb700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a409f130 con 0x7f38a409d6f0 2023-10-03T10:16:12.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.988+0000 7f38aa7fb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a409f520 con 0x7f38a409e1d0 2023-10-03T10:16:12.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.988+0000 7f38aa7fb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38a409e930 con 0x7f38a4135e50 2023-10-03T10:16:12.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.988+0000 7f38a37fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f38a409e1d0 0x7f38a4135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:12.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.988+0000 7f38a37fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f38a409e1d0 0x7f38a4135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53308/0 (socket says 172.21.15.3:53308) 2023-10-03T10:16:12.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.988+0000 7f38a37fe700 1 -- 172.21.15.3:0/603266508 learned_addr learned my addr 172.21.15.3:0/603266508 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:12.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.988+0000 7f38a8d98700 1 --2- 172.21.15.3:0/603266508 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38a4135e50 0x7f38a4138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:12.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.989+0000 7f38a8d98700 1 -- 172.21.15.3:0/603266508 >> v2:172.21.15.3:3300/0 conn(0x7f38a409e1d0 msgr2=0x7f38a4135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:12.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.989+0000 7f38a8d98700 1 --2- 172.21.15.3:0/603266508 >> v2:172.21.15.3:3300/0 conn(0x7f38a409e1d0 0x7f38a4135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:12.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.989+0000 7f38a8d98700 1 -- 172.21.15.3:0/603266508 >> v1:172.21.15.3:6789/0 conn(0x7f38a409d6f0 legacy=0x7f38a409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:12.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.989+0000 7f38a8d98700 1 -- 172.21.15.3:0/603266508 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a409b2d0 con 0x7f38a4135e50 2023-10-03T10:16:12.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.989+0000 7f38a8d98700 1 --2- 172.21.15.3:0/603266508 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38a4135e50 0x7f38a4138230 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f389c009d90 tx=0x7f389c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60dcdd400511601a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:12.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.989+0000 7f38a2ffd700 1 -- 172.21.15.3:0/603266508 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c00e070 con 0x7f38a4135e50 2023-10-03T10:16:12.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.989+0000 7f38a2ffd700 1 -- 172.21.15.3:0/603266508 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c00bb20 con 0x7f38a4135e50 2023-10-03T10:16:12.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.989+0000 7f38a2ffd700 1 -- 172.21.15.3:0/603266508 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c012430 con 0x7f38a4135e50 2023-10-03T10:16:12.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.990+0000 7f38aa7fb700 1 -- 172.21.15.3:0/603266508 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38a4135e50 msgr2=0x7f38a4138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:12.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.990+0000 7f38aa7fb700 1 --2- 172.21.15.3:0/603266508 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38a4135e50 0x7f38a4138230 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f389c009d90 tx=0x7f389c00b330 comp rx=0 tx=0).stop 2023-10-03T10:16:12.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.990+0000 7f38aa7fb700 1 -- 172.21.15.3:0/603266508 shutdown_connections 2023-10-03T10:16:12.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.990+0000 7f38aa7fb700 1 --2- 172.21.15.3:0/603266508 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38a4135e50 0x7f38a4138230 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:12.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.990+0000 7f38aa7fb700 1 --2- 172.21.15.3:0/603266508 >> v2:172.21.15.3:3300/0 conn(0x7f38a409e1d0 0x7f38a4135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:12.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.990+0000 7f38aa7fb700 1 -- 172.21.15.3:0/603266508 >> 172.21.15.3:0/603266508 conn(0x7f38a4066f50 msgr2=0x7f38a409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:12.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.990+0000 7f38aa7fb700 1 -- 172.21.15.3:0/603266508 shutdown_connections 2023-10-03T10:16:12.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.990+0000 7f38aa7fb700 1 -- 172.21.15.3:0/603266508 wait complete. 2023-10-03T10:16:12.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.991+0000 7f38aa7fb700 1 Processor -- start 2023-10-03T10:16:12.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.991+0000 7f38aa7fb700 1 -- start start 2023-10-03T10:16:12.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.992+0000 7f38aa7fb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38a409d6f0 0x7f38a411e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:12.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.992+0000 7f38aa7fb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38a409e1d0 0x7f38a411e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:12.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.992+0000 7f38aa7fb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38a4135e50 0x7f38a41c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:12.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.992+0000 7f38aa7fb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a411dd70 con 0x7f38a4135e50 2023-10-03T10:16:12.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.992+0000 7f38aa7fb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a41303a0 con 0x7f38a409e1d0 2023-10-03T10:16:12.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.992+0000 7f38aa7fb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38a4130160 con 0x7f38a409d6f0 2023-10-03T10:16:12.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.992+0000 7f38a3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38a409d6f0 0x7f38a411e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:12.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.992+0000 7f38a3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38a409d6f0 0x7f38a411e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52542/0 (socket says 172.21.15.3:52542) 2023-10-03T10:16:12.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.992+0000 7f38a3fff700 1 -- 172.21.15.3:0/3367980386 learned_addr learned my addr 172.21.15.3:0/3367980386 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:12.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.992+0000 7f38a37fe700 1 --2- 172.21.15.3:0/3367980386 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38a409e1d0 0x7f38a411e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:12.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.992+0000 7f38a8d98700 1 --2- 172.21.15.3:0/3367980386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38a4135e50 0x7f38a41c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:12.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.992+0000 7f38a37fe700 1 -- 172.21.15.3:0/3367980386 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38a409d6f0 msgr2=0x7f38a411e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:12.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.992+0000 7f38a37fe700 1 --2- 172.21.15.3:0/3367980386 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38a409d6f0 0x7f38a411e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:12.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.992+0000 7f38a37fe700 1 -- 172.21.15.3:0/3367980386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38a4135e50 msgr2=0x7f38a41c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:12.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.992+0000 7f38a37fe700 1 --2- 172.21.15.3:0/3367980386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38a4135e50 0x7f38a41c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:12.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.992+0000 7f38a37fe700 1 -- 172.21.15.3:0/3367980386 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389c009a70 con 0x7f38a409e1d0 2023-10-03T10:16:12.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.993+0000 7f38a37fe700 1 --2- 172.21.15.3:0/3367980386 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38a409e1d0 0x7f38a411e660 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f389800ea30 tx=0x7f389800ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:12.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.993+0000 7f38a0ff9700 1 -- 172.21.15.3:0/3367980386 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389800dc50 con 0x7f38a409e1d0 2023-10-03T10:16:12.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.993+0000 7f38a0ff9700 1 -- 172.21.15.3:0/3367980386 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389800ddb0 con 0x7f38a409e1d0 2023-10-03T10:16:12.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.993+0000 7f38aa7fb700 1 -- 172.21.15.3:0/3367980386 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a40a1d00 con 0x7f38a409e1d0 2023-10-03T10:16:12.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.993+0000 7f38aa7fb700 1 -- 172.21.15.3:0/3367980386 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a411f400 con 0x7f38a409e1d0 2023-10-03T10:16:12.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.993+0000 7f38a0ff9700 1 -- 172.21.15.3:0/3367980386 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3898010640 con 0x7f38a409e1d0 2023-10-03T10:16:13.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.994+0000 7f38aa7fb700 1 -- 172.21.15.3:0/3367980386 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38840052e0 con 0x7f38a409e1d0 2023-10-03T10:16:13.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.995+0000 7f38a0ff9700 1 -- 172.21.15.3:0/3367980386 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f38980107a0 con 0x7f38a409e1d0 2023-10-03T10:16:13.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:12.995+0000 7f38a0ff9700 1 -- 172.21.15.3:0/3367980386 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f389803fd20 con 0x7f38a409e1d0 2023-10-03T10:16:13.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.004+0000 7f38a0ff9700 1 -- 172.21.15.3:0/3367980386 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3898017be0 con 0x7f38a409e1d0 2023-10-03T10:16:13.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.022+0000 7fbee76cf700 1 -- 172.21.15.3:0/3094902139 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbee0074ea0 con 0x7fbee009e000 2023-10-03T10:16:13.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.022+0000 7fbed5ffb700 1 -- 172.21.15.3:0/3094902139 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbed0017020 con 0x7fbee009e000 2023-10-03T10:16:13.022 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:13.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.026+0000 7fbee76cf700 1 -- 172.21.15.3:0/3094902139 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbee009e000 msgr2=0x7fbee011e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:13.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.026+0000 7fbee76cf700 1 --2- 172.21.15.3:0/3094902139 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbee009e000 0x7fbee011e350 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbed000ebb0 tx=0x7fbed000eec0 comp rx=0 tx=0).stop 2023-10-03T10:16:13.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.026+0000 7fbee76cf700 1 -- 172.21.15.3:0/3094902139 shutdown_connections 2023-10-03T10:16:13.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.026+0000 7fbee76cf700 1 --2- 172.21.15.3:0/3094902139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee009e410 0x7fbee011e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:13.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.026+0000 7fbee76cf700 1 --2- 172.21.15.3:0/3094902139 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbee009d6f0 0x7fbee011de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:13.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.026+0000 7fbee76cf700 1 --2- 172.21.15.3:0/3094902139 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbee009e000 0x7fbee011e350 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:13.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.026+0000 7fbee76cf700 1 -- 172.21.15.3:0/3094902139 >> 172.21.15.3:0/3094902139 conn(0x7fbee0066f50 msgr2=0x7fbee0130a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:13.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.026+0000 7fbee76cf700 1 -- 172.21.15.3:0/3094902139 shutdown_connections 2023-10-03T10:16:13.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.026+0000 7fbee76cf700 1 -- 172.21.15.3:0/3094902139 wait complete. 2023-10-03T10:16:13.045 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:13.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.125+0000 7fd9249be700 1 -- 172.21.15.3:0/379661969 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd92006c360 con 0x7fd92009d6f0 2023-10-03T10:16:13.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.125+0000 7fd90effd700 1 -- 172.21.15.3:0/379661969 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd910019d20 con 0x7fd92009d6f0 2023-10-03T10:16:13.125 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:13.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.128+0000 7fd9249be700 1 -- 172.21.15.3:0/379661969 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd92009d6f0 msgr2=0x7fd92011e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:13.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.128+0000 7fd9249be700 1 --2- 172.21.15.3:0/379661969 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd92009d6f0 0x7fd92011e7b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd91000c480 tx=0x7fd91000c790 comp rx=0 tx=0).stop 2023-10-03T10:16:13.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.128+0000 7fd9249be700 1 -- 172.21.15.3:0/379661969 shutdown_connections 2023-10-03T10:16:13.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.129+0000 7fd9249be700 1 --2- 172.21.15.3:0/379661969 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd920139170 0x7fd92011f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:13.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.129+0000 7fd9249be700 1 --2- 172.21.15.3:0/379661969 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd92009d6f0 0x7fd92011e7b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:13.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.129+0000 7fd9249be700 1 --2- 172.21.15.3:0/379661969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd92009e1d0 0x7fd92011ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:13.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.129+0000 7fd9249be700 1 -- 172.21.15.3:0/379661969 >> 172.21.15.3:0/379661969 conn(0x7fd920066f50 msgr2=0x7fd9200a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:13.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.129+0000 7fd9249be700 1 -- 172.21.15.3:0/379661969 shutdown_connections 2023-10-03T10:16:13.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.129+0000 7fd9249be700 1 -- 172.21.15.3:0/379661969 wait complete. 2023-10-03T10:16:13.143 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:13.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.150+0000 7f3626081700 1 -- 172.21.15.3:0/2243668447 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f362011cde0 con 0x7f362009a580 2023-10-03T10:16:13.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.150+0000 7f3607fff700 1 -- 172.21.15.3:0/2243668447 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3614017070 con 0x7f362009a580 2023-10-03T10:16:13.150 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:13.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.152+0000 7f3626081700 1 -- 172.21.15.3:0/2243668447 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f362009a580 msgr2=0x7f3620119c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:13.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.152+0000 7f3626081700 1 --2- 172.21.15.3:0/2243668447 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f362009a580 0x7f3620119c10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f361400c700 tx=0x7f361400ca10 comp rx=0 tx=0).stop 2023-10-03T10:16:13.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.152+0000 7f3626081700 1 -- 172.21.15.3:0/2243668447 shutdown_connections 2023-10-03T10:16:13.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.152+0000 7f3626081700 1 --2- 172.21.15.3:0/2243668447 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3620131810 0x7f362011a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:13.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.152+0000 7f3626081700 1 --2- 172.21.15.3:0/2243668447 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3620099a40 0x7f36201196d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:13.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.152+0000 7f3626081700 1 --2- 172.21.15.3:0/2243668447 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f362009a580 0x7f3620119c10 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:13.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.152+0000 7f3626081700 1 -- 172.21.15.3:0/2243668447 >> 172.21.15.3:0/2243668447 conn(0x7f36200a2aa0 msgr2=0x7f362012e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:13.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.152+0000 7f3626081700 1 -- 172.21.15.3:0/2243668447 shutdown_connections 2023-10-03T10:16:13.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.152+0000 7f3626081700 1 -- 172.21.15.3:0/2243668447 wait complete. 2023-10-03T10:16:13.166 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:13.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.212+0000 7f38aa7fb700 1 -- 172.21.15.3:0/3367980386 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3884005860 con 0x7f38a409e1d0 2023-10-03T10:16:13.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.212+0000 7f38a0ff9700 1 -- 172.21.15.3:0/3367980386 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f38980177d0 con 0x7f38a409e1d0 2023-10-03T10:16:13.212 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:13.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.214+0000 7f38aa7fb700 1 -- 172.21.15.3:0/3367980386 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38a409e1d0 msgr2=0x7f38a411e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:13.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.214+0000 7f38aa7fb700 1 --2- 172.21.15.3:0/3367980386 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38a409e1d0 0x7f38a411e660 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f389800ea30 tx=0x7f389800ed40 comp rx=0 tx=0).stop 2023-10-03T10:16:13.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.214+0000 7f38aa7fb700 1 -- 172.21.15.3:0/3367980386 shutdown_connections 2023-10-03T10:16:13.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.214+0000 7f38aa7fb700 1 --2- 172.21.15.3:0/3367980386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38a4135e50 0x7f38a41c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:13.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.214+0000 7f38aa7fb700 1 --2- 172.21.15.3:0/3367980386 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38a409d6f0 0x7f38a411e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:13.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.214+0000 7f38aa7fb700 1 --2- 172.21.15.3:0/3367980386 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38a409e1d0 0x7f38a411e660 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:13.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.214+0000 7f38aa7fb700 1 -- 172.21.15.3:0/3367980386 >> 172.21.15.3:0/3367980386 conn(0x7f38a4066f50 msgr2=0x7f38a40a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:13.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.214+0000 7f38aa7fb700 1 -- 172.21.15.3:0/3367980386 shutdown_connections 2023-10-03T10:16:13.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:13.214+0000 7f38aa7fb700 1 -- 172.21.15.3:0/3367980386 wait complete. 2023-10-03T10:16:13.228 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:14.046 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:14.143 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:14.167 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:14.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.193+0000 7f77aa961700 1 Processor -- start 2023-10-03T10:16:14.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.193+0000 7f77aa961700 1 -- start start 2023-10-03T10:16:14.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.194+0000 7f77aa961700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77a409e1d0 0x7f77a4135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:14.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.194+0000 7f77aa961700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77a4135e50 0x7f77a4138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:14.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.194+0000 7f77aa961700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a409f130 con 0x7f77a409d6f0 2023-10-03T10:16:14.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.194+0000 7f77aa961700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a409f520 con 0x7f77a409e1d0 2023-10-03T10:16:14.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.194+0000 7f77aa961700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77a409e930 con 0x7f77a4135e50 2023-10-03T10:16:14.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.194+0000 7f77a3fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f77a409d6f0 0x7f77a409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:41262/0 (socket says 172.21.15.3:41262) 2023-10-03T10:16:14.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.194+0000 7f77a3fff700 1 -- 172.21.15.3:0/888575008 learned_addr learned my addr 172.21.15.3:0/888575008 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:14.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.194+0000 7f77a37fe700 1 --2- 172.21.15.3:0/888575008 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77a409e1d0 0x7f77a4135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:14.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.195+0000 7f77a8efe700 1 --2- 172.21.15.3:0/888575008 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77a4135e50 0x7f77a4138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:14.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.195+0000 7f77a2ffd700 1 -- 172.21.15.3:0/888575008 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3964710117 0 0) 0x7f77a409f130 con 0x7f77a409d6f0 2023-10-03T10:16:14.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.195+0000 7f77a37fe700 1 -- 172.21.15.3:0/888575008 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77a4135e50 msgr2=0x7f77a4138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:14.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.195+0000 7f77a37fe700 1 --2- 172.21.15.3:0/888575008 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77a4135e50 0x7f77a4138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.195+0000 7f77a37fe700 1 -- 172.21.15.3:0/888575008 >> v1:172.21.15.3:6789/0 conn(0x7f77a409d6f0 legacy=0x7f77a409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:14.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.195+0000 7f77a37fe700 1 -- 172.21.15.3:0/888575008 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a409b2d0 con 0x7f77a409e1d0 2023-10-03T10:16:14.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.195+0000 7f77a37fe700 1 --2- 172.21.15.3:0/888575008 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77a409e1d0 0x7f77a4135910 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7798009870 tx=0x7f7798009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f11396e4b5e454f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:14.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.195+0000 7f77a8efe700 1 --2- 172.21.15.3:0/888575008 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77a4135e50 0x7f77a4138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:16:14.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.195+0000 7f77a2ffd700 1 -- 172.21.15.3:0/888575008 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7798011830 con 0x7f77a409e1d0 2023-10-03T10:16:14.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.195+0000 7f77a2ffd700 1 -- 172.21.15.3:0/888575008 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7798011990 con 0x7f77a409e1d0 2023-10-03T10:16:14.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.195+0000 7f77a2ffd700 1 -- 172.21.15.3:0/888575008 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7798011d00 con 0x7f77a409e1d0 2023-10-03T10:16:14.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.196+0000 7f77aa961700 1 -- 172.21.15.3:0/888575008 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77a409e1d0 msgr2=0x7f77a4135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:14.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.196+0000 7f77aa961700 1 --2- 172.21.15.3:0/888575008 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77a409e1d0 0x7f77a4135910 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7798009870 tx=0x7f7798009b80 comp rx=0 tx=0).stop 2023-10-03T10:16:14.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.196+0000 7f77aa961700 1 -- 172.21.15.3:0/888575008 shutdown_connections 2023-10-03T10:16:14.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.196+0000 7f77aa961700 1 --2- 172.21.15.3:0/888575008 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77a4135e50 0x7f77a4138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.196+0000 7f77aa961700 1 --2- 172.21.15.3:0/888575008 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77a409e1d0 0x7f77a4135910 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.196+0000 7f77aa961700 1 -- 172.21.15.3:0/888575008 >> 172.21.15.3:0/888575008 conn(0x7f77a4066f50 msgr2=0x7f77a409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:14.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.196+0000 7f77aa961700 1 -- 172.21.15.3:0/888575008 shutdown_connections 2023-10-03T10:16:14.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.196+0000 7f77aa961700 1 -- 172.21.15.3:0/888575008 wait complete. 2023-10-03T10:16:14.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.197+0000 7f77aa961700 1 Processor -- start 2023-10-03T10:16:14.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.198+0000 7f77aa961700 1 -- start start 2023-10-03T10:16:14.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.198+0000 7f77aa961700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77a409d6f0 0x7f77a4119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:14.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.198+0000 7f77aa961700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77a409e1d0 0x7f77a41199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:14.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.198+0000 7f77aa961700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77a4135e50 0x7f77a4119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:14.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.198+0000 7f77aa961700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a41300c0 con 0x7f77a4135e50 2023-10-03T10:16:14.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.198+0000 7f77aa961700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a411cc00 con 0x7f77a409e1d0 2023-10-03T10:16:14.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.198+0000 7f77aa961700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77a411c4f0 con 0x7f77a409d6f0 2023-10-03T10:16:14.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.198+0000 7f77a37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77a409e1d0 0x7f77a41199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:14.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.198+0000 7f77a37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77a409e1d0 0x7f77a41199c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44164/0 (socket says 172.21.15.3:44164) 2023-10-03T10:16:14.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.198+0000 7f77a37fe700 1 -- 172.21.15.3:0/1986544727 learned_addr learned my addr 172.21.15.3:0/1986544727 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:14.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.198+0000 7f77a8efe700 1 --2- 172.21.15.3:0/1986544727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77a4135e50 0x7f77a4119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:14.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.198+0000 7f77a3fff700 1 --2- 172.21.15.3:0/1986544727 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77a409d6f0 0x7f77a4119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:14.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.199+0000 7f77a8efe700 1 -- 172.21.15.3:0/1986544727 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77a409d6f0 msgr2=0x7f77a4119480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:14.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.199+0000 7f77a8efe700 1 --2- 172.21.15.3:0/1986544727 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77a409d6f0 0x7f77a4119480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.199+0000 7f77a8efe700 1 -- 172.21.15.3:0/1986544727 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77a409e1d0 msgr2=0x7f77a41199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:14.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.199+0000 7f77a8efe700 1 --2- 172.21.15.3:0/1986544727 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77a409e1d0 0x7f77a41199c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.199+0000 7f77a8efe700 1 -- 172.21.15.3:0/1986544727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77980094a0 con 0x7f77a4135e50 2023-10-03T10:16:14.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.199+0000 7f77a8efe700 1 --2- 172.21.15.3:0/1986544727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77a4135e50 0x7f77a4119f00 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f779c007e60 tx=0x7f779c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:14.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.199+0000 7f77a37fe700 1 --2- 172.21.15.3:0/1986544727 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77a409e1d0 0x7f77a41199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:14.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.199+0000 7f77a0ff9700 1 -- 172.21.15.3:0/1986544727 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c00dd20 con 0x7f77a4135e50 2023-10-03T10:16:14.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.199+0000 7f77aa961700 1 -- 172.21.15.3:0/1986544727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a411aef0 con 0x7f77a4135e50 2023-10-03T10:16:14.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.199+0000 7f77a0ff9700 1 -- 172.21.15.3:0/1986544727 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779c00f040 con 0x7f77a4135e50 2023-10-03T10:16:14.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.199+0000 7f77a0ff9700 1 -- 172.21.15.3:0/1986544727 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c003c40 con 0x7f77a4135e50 2023-10-03T10:16:14.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.199+0000 7f77aa961700 1 -- 172.21.15.3:0/1986544727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a40a1d50 con 0x7f77a4135e50 2023-10-03T10:16:14.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.200+0000 7f77a0ff9700 1 -- 172.21.15.3:0/1986544727 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f779c004030 con 0x7f77a4135e50 2023-10-03T10:16:14.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.200+0000 7f77aa961700 1 -- 172.21.15.3:0/1986544727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a4074ea0 con 0x7f77a4135e50 2023-10-03T10:16:14.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.201+0000 7f77a0ff9700 1 -- 172.21.15.3:0/1986544727 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f779c03f6d0 con 0x7f77a4135e50 2023-10-03T10:16:14.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.207+0000 7f77a0ff9700 1 -- 172.21.15.3:0/1986544727 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f779c0043d0 con 0x7f77a4135e50 2023-10-03T10:16:14.229 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:14.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.291+0000 7f773ef25700 1 Processor -- start 2023-10-03T10:16:14.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.291+0000 7f773ef25700 1 -- start start 2023-10-03T10:16:14.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.291+0000 7f773ef25700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f773806e570 0x7f773806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:14.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.291+0000 7f773ef25700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f773806ee80 0x7f7738134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:14.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.291+0000 7f773ef25700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7738070ee0 con 0x7f773809eca0 2023-10-03T10:16:14.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.291+0000 7f773ef25700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7738071480 con 0x7f773806ee80 2023-10-03T10:16:14.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.291+0000 7f773ef25700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7738071b90 con 0x7f773806e570 2023-10-03T10:16:14.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.292+0000 7f7737fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f773809eca0 0x7f7738134610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:41274/0 (socket says 172.21.15.3:41274) 2023-10-03T10:16:14.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.292+0000 7f7737fff700 1 -- 172.21.15.3:0/2783202383 learned_addr learned my addr 172.21.15.3:0/2783202383 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:14.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.292+0000 7f773ccc1700 1 --2- 172.21.15.3:0/2783202383 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f773806e570 0x7f773806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:14.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.292+0000 7f773d4c2700 1 --2- 172.21.15.3:0/2783202383 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f773806ee80 0x7f7738134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:14.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.292+0000 7f77377fe700 1 -- 172.21.15.3:0/2783202383 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1449702702 0 0) 0x7f7738070ee0 con 0x7f773809eca0 2023-10-03T10:16:14.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.292+0000 7f77377fe700 1 -- 172.21.15.3:0/2783202383 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77280035b0 con 0x7f773809eca0 2023-10-03T10:16:14.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.292+0000 7f773ccc1700 1 -- 172.21.15.3:0/2783202383 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f773806ee80 msgr2=0x7f7738134d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:14.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.292+0000 7f773ccc1700 1 --2- 172.21.15.3:0/2783202383 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f773806ee80 0x7f7738134d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.292+0000 7f773ccc1700 1 -- 172.21.15.3:0/2783202383 >> v1:172.21.15.3:6789/0 conn(0x7f773809eca0 legacy=0x7f7738134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:14.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.292+0000 7f773ccc1700 1 -- 172.21.15.3:0/2783202383 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77380711a0 con 0x7f773806e570 2023-10-03T10:16:14.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.293+0000 7f773ccc1700 1 --2- 172.21.15.3:0/2783202383 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f773806e570 0x7f773806e940 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f772c009870 tx=0x7f772c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d59acd29f5bf37bc server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:14.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.293+0000 7f77377fe700 1 -- 172.21.15.3:0/2783202383 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c011830 con 0x7f773806e570 2023-10-03T10:16:14.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.293+0000 7f77377fe700 1 -- 172.21.15.3:0/2783202383 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772c011990 con 0x7f773806e570 2023-10-03T10:16:14.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.293+0000 7f77377fe700 1 -- 172.21.15.3:0/2783202383 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c011d20 con 0x7f773806e570 2023-10-03T10:16:14.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.294+0000 7f773ef25700 1 -- 172.21.15.3:0/2783202383 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f773806e570 msgr2=0x7f773806e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:14.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.294+0000 7f773ef25700 1 --2- 172.21.15.3:0/2783202383 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f773806e570 0x7f773806e940 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f772c009870 tx=0x7f772c009b80 comp rx=0 tx=0).stop 2023-10-03T10:16:14.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.294+0000 7f773ef25700 1 -- 172.21.15.3:0/2783202383 shutdown_connections 2023-10-03T10:16:14.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.294+0000 7f773ef25700 1 --2- 172.21.15.3:0/2783202383 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f773806ee80 0x7f7738134d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.294+0000 7f773ef25700 1 --2- 172.21.15.3:0/2783202383 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f773806e570 0x7f773806e940 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.294+0000 7f773ef25700 1 -- 172.21.15.3:0/2783202383 >> 172.21.15.3:0/2783202383 conn(0x7f77380a2aa0 msgr2=0x7f77380a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:14.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.294+0000 7f773ef25700 1 -- 172.21.15.3:0/2783202383 shutdown_connections 2023-10-03T10:16:14.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.294+0000 7f773ef25700 1 -- 172.21.15.3:0/2783202383 wait complete. 2023-10-03T10:16:14.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.296+0000 7f773ef25700 1 Processor -- start 2023-10-03T10:16:14.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.296+0000 7f773ef25700 1 -- start start 2023-10-03T10:16:14.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.296+0000 7f773ef25700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f773806e570 0x7f773813af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:14.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.296+0000 7f773ef25700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f773806ee80 0x7f7738125130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:14.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.296+0000 7f773ef25700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7738125670 0x7f7738125ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:14.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.296+0000 7f773ef25700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7738070680 con 0x7f773806e570 2023-10-03T10:16:14.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.296+0000 7f773ef25700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f773813d8b0 con 0x7f773806ee80 2023-10-03T10:16:14.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.296+0000 7f773ef25700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f773813d310 con 0x7f7738125670 2023-10-03T10:16:14.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.297+0000 7f773d4c2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7738125670 0x7f7738125ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:14.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.297+0000 7f773d4c2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7738125670 0x7f7738125ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:32942/0 (socket says 172.21.15.3:32942) 2023-10-03T10:16:14.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.297+0000 7f773d4c2700 1 -- 172.21.15.3:0/2161698783 learned_addr learned my addr 172.21.15.3:0/2161698783 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:14.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.297+0000 7f7737fff700 1 --2- 172.21.15.3:0/2161698783 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f773806ee80 0x7f7738125130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:14.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.297+0000 7f773ccc1700 1 --2- 172.21.15.3:0/2161698783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f773806e570 0x7f773813af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:14.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.297+0000 7f773d4c2700 1 -- 172.21.15.3:0/2161698783 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f773806ee80 msgr2=0x7f7738125130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:14.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.297+0000 7f773d4c2700 1 --2- 172.21.15.3:0/2161698783 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f773806ee80 0x7f7738125130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.297+0000 7f773d4c2700 1 -- 172.21.15.3:0/2161698783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f773806e570 msgr2=0x7f773813af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:14.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.297+0000 7f773d4c2700 1 --2- 172.21.15.3:0/2161698783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f773806e570 0x7f773813af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.297+0000 7f773d4c2700 1 -- 172.21.15.3:0/2161698783 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772c0094a0 con 0x7f7738125670 2023-10-03T10:16:14.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.297+0000 7f7737fff700 1 --2- 172.21.15.3:0/2161698783 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f773806ee80 0x7f7738125130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:14.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.297+0000 7f773ccc1700 1 --2- 172.21.15.3:0/2161698783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f773806e570 0x7f773813af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:14.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.297+0000 7f773d4c2700 1 --2- 172.21.15.3:0/2161698783 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7738125670 0x7f7738125ad0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f773000efc0 tx=0x7f773000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:14.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.298+0000 7f77357fa700 1 -- 172.21.15.3:0/2161698783 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7730009ea0 con 0x7f7738125670 2023-10-03T10:16:14.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.298+0000 7f77357fa700 1 -- 172.21.15.3:0/2161698783 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773000f040 con 0x7f7738125670 2023-10-03T10:16:14.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.298+0000 7f773ef25700 1 -- 172.21.15.3:0/2161698783 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7738126840 con 0x7f7738125670 2023-10-03T10:16:14.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.298+0000 7f77357fa700 1 -- 172.21.15.3:0/2161698783 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773000ced0 con 0x7f7738125670 2023-10-03T10:16:14.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.298+0000 7f773ef25700 1 -- 172.21.15.3:0/2161698783 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f773813e040 con 0x7f7738125670 2023-10-03T10:16:14.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.299+0000 7f77357fa700 1 -- 172.21.15.3:0/2161698783 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7730014480 con 0x7f7738125670 2023-10-03T10:16:14.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.299+0000 7f773ef25700 1 -- 172.21.15.3:0/2161698783 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7738137590 con 0x7f7738125670 2023-10-03T10:16:14.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.299+0000 7f77357fa700 1 -- 172.21.15.3:0/2161698783 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f773003ff00 con 0x7f7738125670 2023-10-03T10:16:14.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.309+0000 7f77357fa700 1 -- 172.21.15.3:0/2161698783 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f773001a8b0 con 0x7f7738125670 2023-10-03T10:16:14.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.343+0000 7fd9f00c6700 1 Processor -- start 2023-10-03T10:16:14.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.344+0000 7fd9f00c6700 1 -- start start 2023-10-03T10:16:14.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.344+0000 7fd9f00c6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd9e809d6f0 0x7fd9e809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:14.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.344+0000 7fd9f00c6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9e809e000 0x7fd9e8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:14.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.344+0000 7fd9f00c6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e809f130 con 0x7fd9e809e410 2023-10-03T10:16:14.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.344+0000 7fd9f00c6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e809f520 con 0x7fd9e809d6f0 2023-10-03T10:16:14.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.344+0000 7fd9f00c6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e809e930 con 0x7fd9e809e000 2023-10-03T10:16:14.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.345+0000 7fd9ed661700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9e809e000 0x7fd9e8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:14.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.345+0000 7fd9ede62700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd9e809d6f0 0x7fd9e809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:14.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.345+0000 7fd9ed661700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9e809e000 0x7fd9e8130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44198/0 (socket says 172.21.15.3:44198) 2023-10-03T10:16:14.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.345+0000 7fd9ede62700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd9e809d6f0 0x7fd9e809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49672/0 (socket says 172.21.15.3:49672) 2023-10-03T10:16:14.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.345+0000 7fd9ed661700 1 -- 172.21.15.3:0/1379541321 learned_addr learned my addr 172.21.15.3:0/1379541321 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:14.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.345+0000 7fd9ed661700 1 -- 172.21.15.3:0/1379541321 >> v2:172.21.15.3:3300/0 conn(0x7fd9e809d6f0 msgr2=0x7fd9e809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:14.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.345+0000 7fd9ed661700 1 --2- 172.21.15.3:0/1379541321 >> v2:172.21.15.3:3300/0 conn(0x7fd9e809d6f0 0x7fd9e809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.345+0000 7fd9ed661700 1 -- 172.21.15.3:0/1379541321 >> v1:172.21.15.3:6789/0 conn(0x7fd9e809e410 legacy=0x7fd9e8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:14.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.345+0000 7fd9ed661700 1 -- 172.21.15.3:0/1379541321 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e809b2d0 con 0x7fd9e809e000 2023-10-03T10:16:14.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.345+0000 7fd9ed661700 1 --2- 172.21.15.3:0/1379541321 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9e809e000 0x7fd9e8130660 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800bb90 tx=0x7fd9d800bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44f0bc1b9dd9bfc3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:14.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.346+0000 7fd9ece60700 1 -- 172.21.15.3:0/1379541321 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d800dd30 con 0x7fd9e809e000 2023-10-03T10:16:14.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.346+0000 7fd9ece60700 1 -- 172.21.15.3:0/1379541321 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d800de90 con 0x7fd9e809e000 2023-10-03T10:16:14.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.346+0000 7fd9ece60700 1 -- 172.21.15.3:0/1379541321 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8010690 con 0x7fd9e809e000 2023-10-03T10:16:14.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.346+0000 7fd9f00c6700 1 -- 172.21.15.3:0/1379541321 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9e809e000 msgr2=0x7fd9e8130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:14.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.346+0000 7fd9f00c6700 1 --2- 172.21.15.3:0/1379541321 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9e809e000 0x7fd9e8130660 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800bb90 tx=0x7fd9d800bea0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.346+0000 7fd9f00c6700 1 -- 172.21.15.3:0/1379541321 shutdown_connections 2023-10-03T10:16:14.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.346+0000 7fd9f00c6700 1 --2- 172.21.15.3:0/1379541321 >> v2:172.21.15.3:3300/0 conn(0x7fd9e809d6f0 0x7fd9e809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.346+0000 7fd9f00c6700 1 --2- 172.21.15.3:0/1379541321 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9e809e000 0x7fd9e8130660 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.346+0000 7fd9f00c6700 1 -- 172.21.15.3:0/1379541321 >> 172.21.15.3:0/1379541321 conn(0x7fd9e8066f50 msgr2=0x7fd9e809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:14.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.347+0000 7fd9f00c6700 1 -- 172.21.15.3:0/1379541321 shutdown_connections 2023-10-03T10:16:14.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.347+0000 7fd9f00c6700 1 -- 172.21.15.3:0/1379541321 wait complete. 2023-10-03T10:16:14.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.348+0000 7fd9f00c6700 1 Processor -- start 2023-10-03T10:16:14.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.348+0000 7fd9f00c6700 1 -- start start 2023-10-03T10:16:14.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.349+0000 7fd9f00c6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9e809d6f0 0x7fd9e8122470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:14.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.349+0000 7fd9f00c6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9e809e000 0x7fd9e81229b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:14.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.349+0000 7fd9f00c6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9e809e410 0x7fd9e8122ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:14.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.349+0000 7fd9f00c6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e8125ed0 con 0x7fd9e809d6f0 2023-10-03T10:16:14.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.349+0000 7fd9f00c6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e809c700 con 0x7fd9e809e000 2023-10-03T10:16:14.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.349+0000 7fd9f00c6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e80a1d00 con 0x7fd9e809e410 2023-10-03T10:16:14.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.349+0000 7fd9ed661700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9e809e000 0x7fd9e81229b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:14.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.349+0000 7fd9ed661700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9e809e000 0x7fd9e81229b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44206/0 (socket says 172.21.15.3:44206) 2023-10-03T10:16:14.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.349+0000 7fd9ede62700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9e809d6f0 0x7fd9e8122470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:14.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.349+0000 7fd9ed661700 1 -- 172.21.15.3:0/1197308362 learned_addr learned my addr 172.21.15.3:0/1197308362 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:14.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.349+0000 7fd9ee663700 1 --2- 172.21.15.3:0/1197308362 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9e809e410 0x7fd9e8122ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:14.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.350+0000 7fd9ed661700 1 -- 172.21.15.3:0/1197308362 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9e809e410 msgr2=0x7fd9e8122ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:14.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.350+0000 7fd9ed661700 1 --2- 172.21.15.3:0/1197308362 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9e809e410 0x7fd9e8122ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.350+0000 7fd9ed661700 1 -- 172.21.15.3:0/1197308362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9e809d6f0 msgr2=0x7fd9e8122470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:14.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.350+0000 7fd9ed661700 1 --2- 172.21.15.3:0/1197308362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9e809d6f0 0x7fd9e8122470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.350+0000 7fd9ed661700 1 -- 172.21.15.3:0/1197308362 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d800b7f0 con 0x7fd9e809e000 2023-10-03T10:16:14.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.350+0000 7fd9ede62700 1 --2- 172.21.15.3:0/1197308362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9e809d6f0 0x7fd9e8122470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:14.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.350+0000 7fd9ed661700 1 --2- 172.21.15.3:0/1197308362 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9e809e000 0x7fd9e81229b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8000c00 tx=0x7fd9d8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:14.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.350+0000 7fd9de7fc700 1 -- 172.21.15.3:0/1197308362 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8003bf0 con 0x7fd9e809e000 2023-10-03T10:16:14.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.350+0000 7fd9de7fc700 1 -- 172.21.15.3:0/1197308362 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d8003d50 con 0x7fd9e809e000 2023-10-03T10:16:14.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.350+0000 7fd9de7fc700 1 -- 172.21.15.3:0/1197308362 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d80106b0 con 0x7fd9e809e000 2023-10-03T10:16:14.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.350+0000 7fd9f00c6700 1 -- 172.21.15.3:0/1197308362 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e8124060 con 0x7fd9e809e000 2023-10-03T10:16:14.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.350+0000 7fd9f00c6700 1 -- 172.21.15.3:0/1197308362 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e8125590 con 0x7fd9e809e000 2023-10-03T10:16:14.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.351+0000 7fd9de7fc700 1 -- 172.21.15.3:0/1197308362 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd9d8019400 con 0x7fd9e809e000 2023-10-03T10:16:14.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.351+0000 7fd9de7fc700 1 -- 172.21.15.3:0/1197308362 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd9d8040b10 con 0x7fd9e809e000 2023-10-03T10:16:14.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.352+0000 7fd9f00c6700 1 -- 172.21.15.3:0/1197308362 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e8125b10 con 0x7fd9e809e000 2023-10-03T10:16:14.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.361+0000 7fd9de7fc700 1 -- 172.21.15.3:0/1197308362 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd9d803ee60 con 0x7fd9e809e000 2023-10-03T10:16:14.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.404+0000 7f35f17be700 1 Processor -- start 2023-10-03T10:16:14.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.404+0000 7f35f17be700 1 -- start start 2023-10-03T10:16:14.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.405+0000 7f35f17be700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35ec131810 0x7f35ec131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:14.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.405+0000 7f35f17be700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35ec099a40 0x7f35ec099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:14.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.405+0000 7f35f17be700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35ec1313c0 con 0x7f35ec09a3b0 2023-10-03T10:16:14.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.405+0000 7f35f17be700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec09b580 con 0x7f35ec099a40 2023-10-03T10:16:14.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.405+0000 7f35f17be700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec06e8c0 con 0x7f35ec131810 2023-10-03T10:16:14.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.405+0000 7f35eb7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f35ec09a3b0 0x7f35ec138650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:41292/0 (socket says 172.21.15.3:41292) 2023-10-03T10:16:14.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.405+0000 7f35eb7fe700 1 -- 172.21.15.3:0/1807577082 learned_addr learned my addr 172.21.15.3:0/1807577082 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:14.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.405+0000 7f35eaffd700 1 --2- 172.21.15.3:0/1807577082 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35ec131810 0x7f35ec131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:14.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.405+0000 7f35ea7fc700 1 --2- 172.21.15.3:0/1807577082 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35ec099a40 0x7f35ec099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:14.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.405+0000 7f35eaffd700 1 -- 172.21.15.3:0/1807577082 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35ec099a40 msgr2=0x7f35ec099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:14.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.405+0000 7f35eaffd700 1 --2- 172.21.15.3:0/1807577082 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35ec099a40 0x7f35ec099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.405+0000 7f35eaffd700 1 -- 172.21.15.3:0/1807577082 >> v1:172.21.15.3:6789/0 conn(0x7f35ec09a3b0 legacy=0x7f35ec138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:14.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.405+0000 7f35eaffd700 1 -- 172.21.15.3:0/1807577082 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ec09a820 con 0x7f35ec131810 2023-10-03T10:16:14.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.405+0000 7f35eaffd700 1 --2- 172.21.15.3:0/1807577082 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35ec131810 0x7f35ec131be0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f35dc009870 tx=0x7f35dc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38205b5cc81bbcc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:14.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.406+0000 7f35e9ffb700 1 -- 172.21.15.3:0/1807577082 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc011830 con 0x7f35ec131810 2023-10-03T10:16:14.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.406+0000 7f35e9ffb700 1 -- 172.21.15.3:0/1807577082 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35dc011990 con 0x7f35ec131810 2023-10-03T10:16:14.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.406+0000 7f35e9ffb700 1 -- 172.21.15.3:0/1807577082 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc011d20 con 0x7f35ec131810 2023-10-03T10:16:14.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.407+0000 7f35f17be700 1 -- 172.21.15.3:0/1807577082 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35ec131810 msgr2=0x7f35ec131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:14.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.407+0000 7f35f17be700 1 --2- 172.21.15.3:0/1807577082 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35ec131810 0x7f35ec131be0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f35dc009870 tx=0x7f35dc009b80 comp rx=0 tx=0).stop 2023-10-03T10:16:14.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.407+0000 7f35f17be700 1 -- 172.21.15.3:0/1807577082 shutdown_connections 2023-10-03T10:16:14.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.407+0000 7f35f17be700 1 --2- 172.21.15.3:0/1807577082 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35ec131810 0x7f35ec131be0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.407+0000 7f35f17be700 1 --2- 172.21.15.3:0/1807577082 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35ec099a40 0x7f35ec099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.407+0000 7f35f17be700 1 -- 172.21.15.3:0/1807577082 >> 172.21.15.3:0/1807577082 conn(0x7f35ec0a2aa0 msgr2=0x7f35ec0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:14.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.407+0000 7f35f17be700 1 -- 172.21.15.3:0/1807577082 shutdown_connections 2023-10-03T10:16:14.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.407+0000 7f35f17be700 1 -- 172.21.15.3:0/1807577082 wait complete. 2023-10-03T10:16:14.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.409+0000 7f35f17be700 1 Processor -- start 2023-10-03T10:16:14.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.409+0000 7f35f17be700 1 -- start start 2023-10-03T10:16:14.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.409+0000 7f35f17be700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35ec099a40 0x7f35ec11b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:14.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.409+0000 7f35f17be700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35ec09a3b0 0x7f35ec11bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:14.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.409+0000 7f35f17be700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35ec131810 0x7f35ec11c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:14.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.409+0000 7f35f17be700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec11cac0 con 0x7f35ec09a3b0 2023-10-03T10:16:14.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.409+0000 7f35f17be700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec11f0e0 con 0x7f35ec131810 2023-10-03T10:16:14.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.409+0000 7f35f17be700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec11e490 con 0x7f35ec099a40 2023-10-03T10:16:14.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.410+0000 7f35eb7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35ec131810 0x7f35ec11c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:14.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.410+0000 7f35eaffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35ec099a40 0x7f35ec11b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:14.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.410+0000 7f35eaffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35ec099a40 0x7f35ec11b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:32972/0 (socket says 172.21.15.3:32972) 2023-10-03T10:16:14.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.410+0000 7f35eaffd700 1 -- 172.21.15.3:0/1843600726 learned_addr learned my addr 172.21.15.3:0/1843600726 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:14.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.410+0000 7f35eb7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35ec131810 0x7f35ec11c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44218/0 (socket says 172.21.15.3:44218) 2023-10-03T10:16:14.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.410+0000 7f35ea7fc700 1 --2- 172.21.15.3:0/1843600726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35ec09a3b0 0x7f35ec11bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:14.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.410+0000 7f35eb7fe700 1 -- 172.21.15.3:0/1843600726 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35ec099a40 msgr2=0x7f35ec11b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:14.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.410+0000 7f35eb7fe700 1 --2- 172.21.15.3:0/1843600726 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35ec099a40 0x7f35ec11b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.410+0000 7f35eb7fe700 1 -- 172.21.15.3:0/1843600726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35ec09a3b0 msgr2=0x7f35ec11bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:14.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.410+0000 7f35eb7fe700 1 --2- 172.21.15.3:0/1843600726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35ec09a3b0 0x7f35ec11bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.410+0000 7f35eb7fe700 1 -- 172.21.15.3:0/1843600726 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35dc0094a0 con 0x7f35ec131810 2023-10-03T10:16:14.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.410+0000 7f35eaffd700 1 --2- 172.21.15.3:0/1843600726 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35ec099a40 0x7f35ec11b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:14.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.410+0000 7f35eb7fe700 1 --2- 172.21.15.3:0/1843600726 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35ec131810 0x7f35ec11c090 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f35e4002220 tx=0x7f35e400cd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:14.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.410+0000 7f35d37fe700 1 -- 172.21.15.3:0/1843600726 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e4015d70 con 0x7f35ec131810 2023-10-03T10:16:14.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.411+0000 7f35f17be700 1 -- 172.21.15.3:0/1843600726 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35ec130a00 con 0x7f35ec131810 2023-10-03T10:16:14.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.411+0000 7f35f17be700 1 -- 172.21.15.3:0/1843600726 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ec06eb50 con 0x7f35ec131810 2023-10-03T10:16:14.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.411+0000 7f35d37fe700 1 -- 172.21.15.3:0/1843600726 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e400a5d0 con 0x7f35ec131810 2023-10-03T10:16:14.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.411+0000 7f35d37fe700 1 -- 172.21.15.3:0/1843600726 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e400a940 con 0x7f35ec131810 2023-10-03T10:16:14.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.412+0000 7f35d37fe700 1 -- 172.21.15.3:0/1843600726 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f35e401b070 con 0x7f35ec131810 2023-10-03T10:16:14.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.412+0000 7f35d37fe700 1 -- 172.21.15.3:0/1843600726 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f35e40479f0 con 0x7f35ec131810 2023-10-03T10:16:14.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.412+0000 7f35f17be700 1 -- 172.21.15.3:0/1843600726 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35cc0052e0 con 0x7f35ec131810 2023-10-03T10:16:14.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.421+0000 7f35d37fe700 1 -- 172.21.15.3:0/1843600726 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f35e401e470 con 0x7f35ec131810 2023-10-03T10:16:14.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.445+0000 7f77aa961700 1 -- 172.21.15.3:0/1986544727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f77a406c360 con 0x7f77a4135e50 2023-10-03T10:16:14.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.445+0000 7f77a0ff9700 1 -- 172.21.15.3:0/1986544727 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f779c01a070 con 0x7f77a4135e50 2023-10-03T10:16:14.445 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:14.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.448+0000 7f77aa961700 1 -- 172.21.15.3:0/1986544727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77a4135e50 msgr2=0x7f77a4119f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:14.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.448+0000 7f77aa961700 1 --2- 172.21.15.3:0/1986544727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77a4135e50 0x7f77a4119f00 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f779c007e60 tx=0x7f779c00d3b0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.448+0000 7f77aa961700 1 -- 172.21.15.3:0/1986544727 shutdown_connections 2023-10-03T10:16:14.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.448+0000 7f77aa961700 1 --2- 172.21.15.3:0/1986544727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77a4135e50 0x7f77a4119f00 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.448+0000 7f77aa961700 1 --2- 172.21.15.3:0/1986544727 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77a409d6f0 0x7f77a4119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.448+0000 7f77aa961700 1 --2- 172.21.15.3:0/1986544727 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77a409e1d0 0x7f77a41199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.449+0000 7f77aa961700 1 -- 172.21.15.3:0/1986544727 >> 172.21.15.3:0/1986544727 conn(0x7f77a4066f50 msgr2=0x7f77a4132a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:14.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.449+0000 7f77aa961700 1 -- 172.21.15.3:0/1986544727 shutdown_connections 2023-10-03T10:16:14.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.449+0000 7f77aa961700 1 -- 172.21.15.3:0/1986544727 wait complete. 2023-10-03T10:16:14.464 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:14.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.531+0000 7f773ef25700 1 -- 172.21.15.3:0/2161698783 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7738074ea0 con 0x7f7738125670 2023-10-03T10:16:14.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.531+0000 7f77357fa700 1 -- 172.21.15.3:0/2161698783 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7730018030 con 0x7f7738125670 2023-10-03T10:16:14.532 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:14.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.534+0000 7f773ef25700 1 -- 172.21.15.3:0/2161698783 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7738125670 msgr2=0x7f7738125ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:14.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.534+0000 7f773ef25700 1 --2- 172.21.15.3:0/2161698783 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7738125670 0x7f7738125ad0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f773000efc0 tx=0x7f773000c3a0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.534+0000 7f773ef25700 1 -- 172.21.15.3:0/2161698783 shutdown_connections 2023-10-03T10:16:14.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.534+0000 7f773ef25700 1 --2- 172.21.15.3:0/2161698783 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7738125670 0x7f7738125ad0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.534+0000 7f773ef25700 1 --2- 172.21.15.3:0/2161698783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f773806e570 0x7f773813af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.534+0000 7f773ef25700 1 --2- 172.21.15.3:0/2161698783 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f773806ee80 0x7f7738125130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.534+0000 7f773ef25700 1 -- 172.21.15.3:0/2161698783 >> 172.21.15.3:0/2161698783 conn(0x7f77380a2aa0 msgr2=0x7f77380a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:14.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.534+0000 7f773ef25700 1 -- 172.21.15.3:0/2161698783 shutdown_connections 2023-10-03T10:16:14.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.534+0000 7f773ef25700 1 -- 172.21.15.3:0/2161698783 wait complete. 2023-10-03T10:16:14.548 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:14.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.586+0000 7fd9f00c6700 1 -- 172.21.15.3:0/1197308362 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd9e8126720 con 0x7fd9e809e000 2023-10-03T10:16:14.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.586+0000 7fd9de7fc700 1 -- 172.21.15.3:0/1197308362 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd9d80196b0 con 0x7fd9e809e000 2023-10-03T10:16:14.587 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:14.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.589+0000 7fd9f00c6700 1 -- 172.21.15.3:0/1197308362 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9e809e000 msgr2=0x7fd9e81229b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:14.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.589+0000 7fd9f00c6700 1 --2- 172.21.15.3:0/1197308362 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9e809e000 0x7fd9e81229b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8000c00 tx=0x7fd9d8004750 comp rx=0 tx=0).stop 2023-10-03T10:16:14.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.589+0000 7fd9f00c6700 1 -- 172.21.15.3:0/1197308362 shutdown_connections 2023-10-03T10:16:14.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.589+0000 7fd9f00c6700 1 --2- 172.21.15.3:0/1197308362 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9e809e410 0x7fd9e8122ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.589+0000 7fd9f00c6700 1 --2- 172.21.15.3:0/1197308362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9e809d6f0 0x7fd9e8122470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.589+0000 7fd9f00c6700 1 --2- 172.21.15.3:0/1197308362 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9e809e000 0x7fd9e81229b0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.589+0000 7fd9f00c6700 1 -- 172.21.15.3:0/1197308362 >> 172.21.15.3:0/1197308362 conn(0x7fd9e8066f50 msgr2=0x7fd9e806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:14.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.589+0000 7fd9f00c6700 1 -- 172.21.15.3:0/1197308362 shutdown_connections 2023-10-03T10:16:14.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.589+0000 7fd9f00c6700 1 -- 172.21.15.3:0/1197308362 wait complete. 2023-10-03T10:16:14.602 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:14.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.646+0000 7f35f17be700 1 -- 172.21.15.3:0/1843600726 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f35cc005590 con 0x7f35ec131810 2023-10-03T10:16:14.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.646+0000 7f35d37fe700 1 -- 172.21.15.3:0/1843600726 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f35e4007860 con 0x7f35ec131810 2023-10-03T10:16:14.646 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:14.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.648+0000 7f35f17be700 1 -- 172.21.15.3:0/1843600726 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35ec131810 msgr2=0x7f35ec11c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:14.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.648+0000 7f35f17be700 1 --2- 172.21.15.3:0/1843600726 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35ec131810 0x7f35ec11c090 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f35e4002220 tx=0x7f35e400cd80 comp rx=0 tx=0).stop 2023-10-03T10:16:14.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.648+0000 7f35f17be700 1 -- 172.21.15.3:0/1843600726 shutdown_connections 2023-10-03T10:16:14.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.648+0000 7f35f17be700 1 --2- 172.21.15.3:0/1843600726 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35ec131810 0x7f35ec11c090 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.648+0000 7f35f17be700 1 --2- 172.21.15.3:0/1843600726 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35ec099a40 0x7f35ec11b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.648+0000 7f35f17be700 1 --2- 172.21.15.3:0/1843600726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35ec09a3b0 0x7f35ec11bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:14.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.648+0000 7f35f17be700 1 -- 172.21.15.3:0/1843600726 >> 172.21.15.3:0/1843600726 conn(0x7f35ec0a2aa0 msgr2=0x7f35ec1b8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:14.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.649+0000 7f35f17be700 1 -- 172.21.15.3:0/1843600726 shutdown_connections 2023-10-03T10:16:14.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:14.649+0000 7f35f17be700 1 -- 172.21.15.3:0/1843600726 wait complete. 2023-10-03T10:16:14.662 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:15.465 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:15.549 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:15.603 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:15.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.613+0000 7f007ed57700 1 Processor -- start 2023-10-03T10:16:15.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.613+0000 7f007ed57700 1 -- start start 2023-10-03T10:16:15.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.614+0000 7f007ed57700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f007809d6f0 0x7f007809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:15.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.614+0000 7f007ed57700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f007809e000 0x7f00781339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:15.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.614+0000 7f007ed57700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f007809b120 con 0x7f007809e410 2023-10-03T10:16:15.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.614+0000 7f007ed57700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f007806b460 con 0x7f007809d6f0 2023-10-03T10:16:15.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.614+0000 7f007ed57700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f007809a7e0 con 0x7f007809e000 2023-10-03T10:16:15.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.614+0000 7f0077fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f007809e000 0x7f00781339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:15.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.614+0000 7f0077fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f007809e000 0x7f00781339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:32988/0 (socket says 172.21.15.3:32988) 2023-10-03T10:16:15.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.614+0000 7f0077fff700 1 -- 172.21.15.3:0/1480793783 learned_addr learned my addr 172.21.15.3:0/1480793783 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:15.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.614+0000 7f007caf3700 1 --2- 172.21.15.3:0/1480793783 >> v2:172.21.15.3:3300/0 conn(0x7f007809d6f0 0x7f007809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:15.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.614+0000 7f0077fff700 1 -- 172.21.15.3:0/1480793783 >> v2:172.21.15.3:3300/0 conn(0x7f007809d6f0 msgr2=0x7f007809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:15.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.614+0000 7f0077fff700 1 --2- 172.21.15.3:0/1480793783 >> v2:172.21.15.3:3300/0 conn(0x7f007809d6f0 0x7f007809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.614+0000 7f0077fff700 1 -- 172.21.15.3:0/1480793783 >> v1:172.21.15.3:6789/0 conn(0x7f007809e410 legacy=0x7f0078133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:15.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.614+0000 7f0077fff700 1 -- 172.21.15.3:0/1480793783 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f007809a500 con 0x7f007809e000 2023-10-03T10:16:15.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.615+0000 7f0077fff700 1 --2- 172.21.15.3:0/1480793783 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f007809e000 0x7f00781339f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0064009920 tx=0x7f0064009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a093891be9f79c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:15.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.615+0000 7f00777fe700 1 -- 172.21.15.3:0/1480793783 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0064011830 con 0x7f007809e000 2023-10-03T10:16:15.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.615+0000 7f00777fe700 1 -- 172.21.15.3:0/1480793783 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0064011990 con 0x7f007809e000 2023-10-03T10:16:15.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.615+0000 7f00777fe700 1 -- 172.21.15.3:0/1480793783 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0064011d20 con 0x7f007809e000 2023-10-03T10:16:15.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.616+0000 7f007ed57700 1 -- 172.21.15.3:0/1480793783 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f007809e000 msgr2=0x7f00781339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:15.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.616+0000 7f007ed57700 1 --2- 172.21.15.3:0/1480793783 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f007809e000 0x7f00781339f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0064009920 tx=0x7f0064009c30 comp rx=0 tx=0).stop 2023-10-03T10:16:15.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.616+0000 7f007ed57700 1 -- 172.21.15.3:0/1480793783 shutdown_connections 2023-10-03T10:16:15.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.616+0000 7f007ed57700 1 --2- 172.21.15.3:0/1480793783 >> v2:172.21.15.3:3300/0 conn(0x7f007809d6f0 0x7f007809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.616+0000 7f007ed57700 1 --2- 172.21.15.3:0/1480793783 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f007809e000 0x7f00781339f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.616+0000 7f007ed57700 1 -- 172.21.15.3:0/1480793783 >> 172.21.15.3:0/1480793783 conn(0x7f0078066f50 msgr2=0x7f007809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:15.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.616+0000 7f007ed57700 1 -- 172.21.15.3:0/1480793783 shutdown_connections 2023-10-03T10:16:15.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.616+0000 7f007ed57700 1 -- 172.21.15.3:0/1480793783 wait complete. 2023-10-03T10:16:15.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.617+0000 7f007ed57700 1 Processor -- start 2023-10-03T10:16:15.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.617+0000 7f007ed57700 1 -- start start 2023-10-03T10:16:15.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.618+0000 7f007ed57700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f007809d6f0 0x7f00781a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:15.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.618+0000 7f007ed57700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f007809e000 0x7f00781a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:15.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.618+0000 7f007ed57700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f007809e410 0x7f00781a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:15.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.618+0000 7f007ed57700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00781a8600 con 0x7f007809e000 2023-10-03T10:16:15.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.618+0000 7f007ed57700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00781a9230 con 0x7f007809e410 2023-10-03T10:16:15.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.618+0000 7f007ed57700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f00781a7bb0 con 0x7f007809d6f0 2023-10-03T10:16:15.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.618+0000 7f007caf3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f007809d6f0 0x7f00781a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:15.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.618+0000 7f007caf3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f007809d6f0 0x7f00781a4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:32990/0 (socket says 172.21.15.3:32990) 2023-10-03T10:16:15.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.618+0000 7f007caf3700 1 -- 172.21.15.3:0/421891585 learned_addr learned my addr 172.21.15.3:0/421891585 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:15.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.618+0000 7f0077fff700 1 --2- 172.21.15.3:0/421891585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f007809e000 0x7f00781a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:15.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.618+0000 7f007d2f4700 1 --2- 172.21.15.3:0/421891585 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f007809e410 0x7f00781a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:15.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.618+0000 7f0077fff700 1 -- 172.21.15.3:0/421891585 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f007809d6f0 msgr2=0x7f00781a4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:15.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.619+0000 7f0077fff700 1 --2- 172.21.15.3:0/421891585 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f007809d6f0 0x7f00781a4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.619+0000 7f0077fff700 1 -- 172.21.15.3:0/421891585 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f007809e410 msgr2=0x7f00781a57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:15.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.619+0000 7f0077fff700 1 --2- 172.21.15.3:0/421891585 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f007809e410 0x7f00781a57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.619+0000 7f0077fff700 1 -- 172.21.15.3:0/421891585 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0064009550 con 0x7f007809e000 2023-10-03T10:16:15.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.619+0000 7f007caf3700 1 --2- 172.21.15.3:0/421891585 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f007809d6f0 0x7f00781a4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:15.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.619+0000 7f0077fff700 1 --2- 172.21.15.3:0/421891585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f007809e000 0x7f00781a5290 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f007809ac00 tx=0x7f0064004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:15.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.619+0000 7f00757fa700 1 -- 172.21.15.3:0/421891585 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0064004450 con 0x7f007809e000 2023-10-03T10:16:15.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.619+0000 7f00757fa700 1 -- 172.21.15.3:0/421891585 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00640036a0 con 0x7f007809e000 2023-10-03T10:16:15.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.619+0000 7f007ed57700 1 -- 172.21.15.3:0/421891585 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00781a8870 con 0x7f007809e000 2023-10-03T10:16:15.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.619+0000 7f00757fa700 1 -- 172.21.15.3:0/421891585 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00640045f0 con 0x7f007809e000 2023-10-03T10:16:15.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.619+0000 7f007ed57700 1 -- 172.21.15.3:0/421891585 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00780a2460 con 0x7f007809e000 2023-10-03T10:16:15.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.620+0000 7f00757fa700 1 -- 172.21.15.3:0/421891585 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0064003a50 con 0x7f007809e000 2023-10-03T10:16:15.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.620+0000 7f00757fa700 1 -- 172.21.15.3:0/421891585 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0064021030 con 0x7f007809e000 2023-10-03T10:16:15.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.620+0000 7f007ed57700 1 -- 172.21.15.3:0/421891585 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00781a6220 con 0x7f007809e000 2023-10-03T10:16:15.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.625+0000 7f00757fa700 1 -- 172.21.15.3:0/421891585 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f00781a6220 con 0x7f007809e000 2023-10-03T10:16:15.663 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:15.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.695+0000 7f285fccd700 1 Processor -- start 2023-10-03T10:16:15.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.695+0000 7f285fccd700 1 -- start start 2023-10-03T10:16:15.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.695+0000 7f285fccd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f285809d6f0 0x7f285809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:15.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.695+0000 7f285fccd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285809e000 0x7f28581339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:15.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.695+0000 7f285fccd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f285809e410 0x7f2858133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:15.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.696+0000 7f285fccd700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f285806b460 con 0x7f285809d6f0 2023-10-03T10:16:15.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.696+0000 7f285fccd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f285809b120 con 0x7f285809e000 2023-10-03T10:16:15.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.696+0000 7f285fccd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f285809a7e0 con 0x7f285809e410 2023-10-03T10:16:15.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.696+0000 7f285d268700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285809e000 0x7f28581339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:15.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.696+0000 7f285d268700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285809e000 0x7f28581339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44228/0 (socket says 172.21.15.3:44228) 2023-10-03T10:16:15.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.696+0000 7f285d268700 1 -- 172.21.15.3:0/938567874 learned_addr learned my addr 172.21.15.3:0/938567874 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:15.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.696+0000 7f285e26a700 1 --2- 172.21.15.3:0/938567874 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f285809e410 0x7f2858133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:15.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.696+0000 7f285da69700 1 --2- 172.21.15.3:0/938567874 >> v2:172.21.15.3:3300/0 conn(0x7f285809d6f0 0x7f285809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:15.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.696+0000 7f285d268700 1 -- 172.21.15.3:0/938567874 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f285809e410 msgr2=0x7f2858133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:15.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.696+0000 7f285d268700 1 --2- 172.21.15.3:0/938567874 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f285809e410 0x7f2858133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.696+0000 7f285d268700 1 -- 172.21.15.3:0/938567874 >> v2:172.21.15.3:3300/0 conn(0x7f285809d6f0 msgr2=0x7f285809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:15.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.696+0000 7f285d268700 1 --2- 172.21.15.3:0/938567874 >> v2:172.21.15.3:3300/0 conn(0x7f285809d6f0 0x7f285809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.696+0000 7f285d268700 1 -- 172.21.15.3:0/938567874 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f285809a500 con 0x7f285809e000 2023-10-03T10:16:15.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.696+0000 7f285d268700 1 --2- 172.21.15.3:0/938567874 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285809e000 0x7f28581339f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2848009920 tx=0x7f2848009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65c70bd246979158 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:15.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.696+0000 7f285da69700 1 --2- 172.21.15.3:0/938567874 >> v2:172.21.15.3:3300/0 conn(0x7f285809d6f0 0x7f285809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:16:15.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.697+0000 7f285ca67700 1 -- 172.21.15.3:0/938567874 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2848011830 con 0x7f285809e000 2023-10-03T10:16:15.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.697+0000 7f285ca67700 1 -- 172.21.15.3:0/938567874 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2848011990 con 0x7f285809e000 2023-10-03T10:16:15.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.697+0000 7f285ca67700 1 -- 172.21.15.3:0/938567874 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2848011d00 con 0x7f285809e000 2023-10-03T10:16:15.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.698+0000 7f285fccd700 1 -- 172.21.15.3:0/938567874 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285809e000 msgr2=0x7f28581339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:15.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.698+0000 7f285fccd700 1 --2- 172.21.15.3:0/938567874 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285809e000 0x7f28581339f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2848009920 tx=0x7f2848009c30 comp rx=0 tx=0).stop 2023-10-03T10:16:15.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.698+0000 7f285fccd700 1 -- 172.21.15.3:0/938567874 shutdown_connections 2023-10-03T10:16:15.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.698+0000 7f285fccd700 1 --2- 172.21.15.3:0/938567874 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f285809e410 0x7f2858133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.698+0000 7f285fccd700 1 --2- 172.21.15.3:0/938567874 >> v2:172.21.15.3:3300/0 conn(0x7f285809d6f0 0x7f285809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.698+0000 7f285fccd700 1 --2- 172.21.15.3:0/938567874 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285809e000 0x7f28581339f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.698+0000 7f285fccd700 1 -- 172.21.15.3:0/938567874 >> 172.21.15.3:0/938567874 conn(0x7f2858066f50 msgr2=0x7f285809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:15.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.698+0000 7f285fccd700 1 -- 172.21.15.3:0/938567874 shutdown_connections 2023-10-03T10:16:15.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.698+0000 7f285fccd700 1 -- 172.21.15.3:0/938567874 wait complete. 2023-10-03T10:16:15.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.700+0000 7f285fccd700 1 Processor -- start 2023-10-03T10:16:15.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.700+0000 7f285fccd700 1 -- start start 2023-10-03T10:16:15.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.700+0000 7f285fccd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285809d6f0 0x7f28581a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:15.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.700+0000 7f285fccd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f285809e000 0x7f28581a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:15.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.700+0000 7f285fccd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285809e410 0x7f28581a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:15.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.701+0000 7f285fccd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28581aa1f0 con 0x7f285809d6f0 2023-10-03T10:16:15.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.701+0000 7f285fccd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28581a7920 con 0x7f285809e410 2023-10-03T10:16:15.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.701+0000 7f285fccd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28581a7f40 con 0x7f285809e000 2023-10-03T10:16:15.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.701+0000 7f285da69700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285809d6f0 0x7f28581a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:15.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.701+0000 7f285da69700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285809d6f0 0x7f28581a5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49724/0 (socket says 172.21.15.3:49724) 2023-10-03T10:16:15.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.701+0000 7f285da69700 1 -- 172.21.15.3:0/621177798 learned_addr learned my addr 172.21.15.3:0/621177798 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:15.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.701+0000 7f285d268700 1 --2- 172.21.15.3:0/621177798 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f285809e000 0x7f28581a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:15.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.701+0000 7f285da69700 1 -- 172.21.15.3:0/621177798 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f285809e000 msgr2=0x7f28581a62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:15.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.701+0000 7f285da69700 1 --2- 172.21.15.3:0/621177798 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f285809e000 0x7f28581a62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.701+0000 7f285da69700 1 -- 172.21.15.3:0/621177798 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285809e410 msgr2=0x7f28581a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:15.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.701+0000 7f285da69700 1 --2- 172.21.15.3:0/621177798 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285809e410 0x7f28581a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.701+0000 7f285da69700 1 -- 172.21.15.3:0/621177798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2848009550 con 0x7f285809d6f0 2023-10-03T10:16:15.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.701+0000 7f285da69700 1 --2- 172.21.15.3:0/621177798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285809d6f0 0x7f28581a5d60 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f285400e840 tx=0x7f285400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:15.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.702+0000 7f284e7fc700 1 -- 172.21.15.3:0/621177798 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2854013610 con 0x7f285809d6f0 2023-10-03T10:16:15.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.702+0000 7f284e7fc700 1 -- 172.21.15.3:0/621177798 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2854013770 con 0x7f285809d6f0 2023-10-03T10:16:15.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.702+0000 7f285fccd700 1 -- 172.21.15.3:0/621177798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28581a71c0 con 0x7f285809d6f0 2023-10-03T10:16:15.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.702+0000 7f284e7fc700 1 -- 172.21.15.3:0/621177798 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2854013b20 con 0x7f285809d6f0 2023-10-03T10:16:15.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.702+0000 7f285fccd700 1 -- 172.21.15.3:0/621177798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28580a2000 con 0x7f285809d6f0 2023-10-03T10:16:15.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.703+0000 7f284e7fc700 1 -- 172.21.15.3:0/621177798 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2854013c80 con 0x7f285809d6f0 2023-10-03T10:16:15.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.703+0000 7f284e7fc700 1 -- 172.21.15.3:0/621177798 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f285403f420 con 0x7f285809d6f0 2023-10-03T10:16:15.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.703+0000 7f285fccd700 1 -- 172.21.15.3:0/621177798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f285809c710 con 0x7f285809d6f0 2023-10-03T10:16:15.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.713+0000 7f284e7fc700 1 -- 172.21.15.3:0/621177798 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2854021080 con 0x7f285809d6f0 2023-10-03T10:16:15.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.750+0000 7f6bc678d700 1 Processor -- start 2023-10-03T10:16:15.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.751+0000 7f6bc678d700 1 -- start start 2023-10-03T10:16:15.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.751+0000 7f6bc678d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6bc00a0a30 0x7f6bc00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:15.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.751+0000 7f6bc678d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6bc009d630 0x7f6bc00a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:15.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.751+0000 7f6bc678d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6bc0133930 0x7f6bc0133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:15.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.751+0000 7f6bc678d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bc00a2a70 con 0x7f6bc009d630 2023-10-03T10:16:15.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.751+0000 7f6bc678d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc00a01d0 con 0x7f6bc0133930 2023-10-03T10:16:15.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.751+0000 7f6bc678d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc009e2b0 con 0x7f6bc00a0a30 2023-10-03T10:16:15.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.751+0000 7f6bc4d2a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6bc0133930 0x7f6bc0133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:15.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.751+0000 7f6bc4d2a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6bc0133930 0x7f6bc0133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44242/0 (socket says 172.21.15.3:44242) 2023-10-03T10:16:15.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.751+0000 7f6bc4d2a700 1 -- 172.21.15.3:0/245156912 learned_addr learned my addr 172.21.15.3:0/245156912 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:15.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.751+0000 7f6bbffff700 1 --2- 172.21.15.3:0/245156912 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6bc00a0a30 0x7f6bc00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:15.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.752+0000 7f6bbf7fe700 1 --2- 172.21.15.3:0/245156912 >> v2:172.21.15.3:3300/0 conn(0x7f6bc009d630 0x7f6bc00a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:15.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.752+0000 7f6bc4d2a700 1 -- 172.21.15.3:0/245156912 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6bc00a0a30 msgr2=0x7f6bc00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:15.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.752+0000 7f6bc4d2a700 1 --2- 172.21.15.3:0/245156912 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6bc00a0a30 0x7f6bc00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.752+0000 7f6bc4d2a700 1 -- 172.21.15.3:0/245156912 >> v2:172.21.15.3:3300/0 conn(0x7f6bc009d630 msgr2=0x7f6bc00a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:15.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.752+0000 7f6bc4d2a700 1 --2- 172.21.15.3:0/245156912 >> v2:172.21.15.3:3300/0 conn(0x7f6bc009d630 0x7f6bc00a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.752+0000 7f6bc4d2a700 1 -- 172.21.15.3:0/245156912 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc009dc50 con 0x7f6bc0133930 2023-10-03T10:16:15.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.752+0000 7f6bc4d2a700 1 --2- 172.21.15.3:0/245156912 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6bc0133930 0x7f6bc0133dd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8009e50 tx=0x7f6bb800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31015b87dc3bc2e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:15.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.752+0000 7f6bbeffd700 1 -- 172.21.15.3:0/245156912 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb800e070 con 0x7f6bc0133930 2023-10-03T10:16:15.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.752+0000 7f6bbeffd700 1 -- 172.21.15.3:0/245156912 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb800b910 con 0x7f6bc0133930 2023-10-03T10:16:15.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.752+0000 7f6bbeffd700 1 -- 172.21.15.3:0/245156912 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb800bcc0 con 0x7f6bc0133930 2023-10-03T10:16:15.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.753+0000 7f6bc678d700 1 -- 172.21.15.3:0/245156912 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6bc0133930 msgr2=0x7f6bc0133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:15.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.753+0000 7f6bc678d700 1 --2- 172.21.15.3:0/245156912 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6bc0133930 0x7f6bc0133dd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8009e50 tx=0x7f6bb800b330 comp rx=0 tx=0).stop 2023-10-03T10:16:15.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.753+0000 7f6bc678d700 1 -- 172.21.15.3:0/245156912 shutdown_connections 2023-10-03T10:16:15.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.753+0000 7f6bc678d700 1 --2- 172.21.15.3:0/245156912 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6bc0133930 0x7f6bc0133dd0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.753+0000 7f6bc678d700 1 --2- 172.21.15.3:0/245156912 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6bc00a0a30 0x7f6bc00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.753+0000 7f6bc678d700 1 --2- 172.21.15.3:0/245156912 >> v2:172.21.15.3:3300/0 conn(0x7f6bc009d630 0x7f6bc00a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.753+0000 7f6bc678d700 1 -- 172.21.15.3:0/245156912 >> 172.21.15.3:0/245156912 conn(0x7f6bc00a3050 msgr2=0x7f6bc00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:15.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.753+0000 7f6bc678d700 1 -- 172.21.15.3:0/245156912 shutdown_connections 2023-10-03T10:16:15.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.753+0000 7f6bc678d700 1 -- 172.21.15.3:0/245156912 wait complete. 2023-10-03T10:16:15.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.754+0000 7f6bc678d700 1 Processor -- start 2023-10-03T10:16:15.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.755+0000 7f6bc678d700 1 -- start start 2023-10-03T10:16:15.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.755+0000 7f6bc678d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6bc009d630 0x7f6bc01c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:15.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.755+0000 7f6bc678d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bc00a0a30 0x7f6bc01c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:15.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.755+0000 7f6bc678d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6bc0133930 0x7f6bc01c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:15.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.755+0000 7f6bc678d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc0133530 con 0x7f6bc00a0a30 2023-10-03T10:16:15.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.755+0000 7f6bc678d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc0121f20 con 0x7f6bc0133930 2023-10-03T10:16:15.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.755+0000 7f6bc678d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc0120730 con 0x7f6bc009d630 2023-10-03T10:16:15.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.756+0000 7f6bbf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bc00a0a30 0x7f6bc01c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:15.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.756+0000 7f6bbf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bc00a0a30 0x7f6bc01c84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49736/0 (socket says 172.21.15.3:49736) 2023-10-03T10:16:15.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.756+0000 7f6bbf7fe700 1 -- 172.21.15.3:0/2080672777 learned_addr learned my addr 172.21.15.3:0/2080672777 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:15.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.756+0000 7f6bc4d2a700 1 --2- 172.21.15.3:0/2080672777 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6bc0133930 0x7f6bc01c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:15.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.756+0000 7f6bbffff700 1 --2- 172.21.15.3:0/2080672777 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6bc009d630 0x7f6bc01c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:15.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.756+0000 7f6bc4d2a700 1 -- 172.21.15.3:0/2080672777 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6bc009d630 msgr2=0x7f6bc01c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:15.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.756+0000 7f6bc4d2a700 1 --2- 172.21.15.3:0/2080672777 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6bc009d630 0x7f6bc01c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.756+0000 7f6bc4d2a700 1 -- 172.21.15.3:0/2080672777 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bc00a0a30 msgr2=0x7f6bc01c84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:15.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.756+0000 7f6bc4d2a700 1 --2- 172.21.15.3:0/2080672777 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bc00a0a30 0x7f6bc01c84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.756+0000 7f6bc4d2a700 1 -- 172.21.15.3:0/2080672777 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb8009b30 con 0x7f6bc0133930 2023-10-03T10:16:15.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.756+0000 7f6bbffff700 1 --2- 172.21.15.3:0/2080672777 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6bc009d630 0x7f6bc01c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:15.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.756+0000 7f6bc4d2a700 1 --2- 172.21.15.3:0/2080672777 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6bc0133930 0x7f6bc01c8a00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8015000 tx=0x7f6bb8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:15.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.756+0000 7f6bbcff9700 1 -- 172.21.15.3:0/2080672777 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8004450 con 0x7f6bc0133930 2023-10-03T10:16:15.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.757+0000 7f6bbcff9700 1 -- 172.21.15.3:0/2080672777 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb80045f0 con 0x7f6bc0133930 2023-10-03T10:16:15.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.757+0000 7f6bc678d700 1 -- 172.21.15.3:0/2080672777 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bc00a5d70 con 0x7f6bc0133930 2023-10-03T10:16:15.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.757+0000 7f6bbcff9700 1 -- 172.21.15.3:0/2080672777 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb80092e0 con 0x7f6bc0133930 2023-10-03T10:16:15.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.757+0000 7f6bc678d700 1 -- 172.21.15.3:0/2080672777 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc0122c50 con 0x7f6bc0133930 2023-10-03T10:16:15.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.758+0000 7f6bbcff9700 1 -- 172.21.15.3:0/2080672777 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6bb8025460 con 0x7f6bc0133930 2023-10-03T10:16:15.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.758+0000 7f6bbcff9700 1 -- 172.21.15.3:0/2080672777 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6bb8041070 con 0x7f6bc0133930 2023-10-03T10:16:15.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.758+0000 7f6bc678d700 1 -- 172.21.15.3:0/2080672777 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bc00747f0 con 0x7f6bc0133930 2023-10-03T10:16:15.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.763+0000 7f6bbcff9700 1 -- 172.21.15.3:0/2080672777 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6bb802a020 con 0x7f6bc0133930 2023-10-03T10:16:15.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.812+0000 7fded0e6f700 1 Processor -- start 2023-10-03T10:16:15.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.812+0000 7fded0e6f700 1 -- start start 2023-10-03T10:16:15.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.813+0000 7fded0e6f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdecc0a0a30 0x7fdecc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:15.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.813+0000 7fded0e6f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdecc09d630 0x7fdecc0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:15.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.813+0000 7fded0e6f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdecc133930 0x7fdecc133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:15.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.813+0000 7fded0e6f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdecc0a2a70 con 0x7fdecc133930 2023-10-03T10:16:15.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.813+0000 7fded0e6f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc0a01d0 con 0x7fdecc09d630 2023-10-03T10:16:15.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.813+0000 7fded0e6f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc09e2b0 con 0x7fdecc0a0a30 2023-10-03T10:16:15.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.813+0000 7fdec9d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdecc09d630 0x7fdecc0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:15.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.813+0000 7fdec9d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdecc09d630 0x7fdecc0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44246/0 (socket says 172.21.15.3:44246) 2023-10-03T10:16:15.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.813+0000 7fdec9d9b700 1 -- 172.21.15.3:0/584728857 learned_addr learned my addr 172.21.15.3:0/584728857 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:15.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.813+0000 7fdeca59c700 1 --2- 172.21.15.3:0/584728857 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdecc0a0a30 0x7fdecc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:15.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.813+0000 7fdecad9d700 1 --2- 172.21.15.3:0/584728857 >> v2:172.21.15.3:3300/0 conn(0x7fdecc133930 0x7fdecc133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:15.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.814+0000 7fdec9d9b700 1 -- 172.21.15.3:0/584728857 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdecc0a0a30 msgr2=0x7fdecc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:15.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.814+0000 7fdec9d9b700 1 --2- 172.21.15.3:0/584728857 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdecc0a0a30 0x7fdecc0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.814+0000 7fdec9d9b700 1 -- 172.21.15.3:0/584728857 >> v2:172.21.15.3:3300/0 conn(0x7fdecc133930 msgr2=0x7fdecc133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:15.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.814+0000 7fdec9d9b700 1 --2- 172.21.15.3:0/584728857 >> v2:172.21.15.3:3300/0 conn(0x7fdecc133930 0x7fdecc133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.814+0000 7fdec9d9b700 1 -- 172.21.15.3:0/584728857 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdecc09dc50 con 0x7fdecc09d630 2023-10-03T10:16:15.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.814+0000 7fdecad9d700 1 --2- 172.21.15.3:0/584728857 >> v2:172.21.15.3:3300/0 conn(0x7fdecc133930 0x7fdecc133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:15.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.814+0000 7fdec9d9b700 1 --2- 172.21.15.3:0/584728857 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdecc09d630 0x7fdecc0a1340 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fdec00098f0 tx=0x7fdec0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5f3061fd4649b04 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:15.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.814+0000 7fdec959a700 1 -- 172.21.15.3:0/584728857 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec0011830 con 0x7fdecc09d630 2023-10-03T10:16:15.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.814+0000 7fdec959a700 1 -- 172.21.15.3:0/584728857 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec0011990 con 0x7fdecc09d630 2023-10-03T10:16:15.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.814+0000 7fdec959a700 1 -- 172.21.15.3:0/584728857 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec0011d20 con 0x7fdecc09d630 2023-10-03T10:16:15.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.815+0000 7fded0e6f700 1 -- 172.21.15.3:0/584728857 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdecc09d630 msgr2=0x7fdecc0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:15.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.815+0000 7fded0e6f700 1 --2- 172.21.15.3:0/584728857 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdecc09d630 0x7fdecc0a1340 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fdec00098f0 tx=0x7fdec0009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:15.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.815+0000 7fded0e6f700 1 -- 172.21.15.3:0/584728857 shutdown_connections 2023-10-03T10:16:15.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.815+0000 7fded0e6f700 1 --2- 172.21.15.3:0/584728857 >> v2:172.21.15.3:3300/0 conn(0x7fdecc133930 0x7fdecc133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.815+0000 7fded0e6f700 1 --2- 172.21.15.3:0/584728857 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdecc0a0a30 0x7fdecc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.815+0000 7fded0e6f700 1 --2- 172.21.15.3:0/584728857 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdecc09d630 0x7fdecc0a1340 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.815+0000 7fded0e6f700 1 -- 172.21.15.3:0/584728857 >> 172.21.15.3:0/584728857 conn(0x7fdecc0a3050 msgr2=0x7fdecc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:15.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.815+0000 7fded0e6f700 1 -- 172.21.15.3:0/584728857 shutdown_connections 2023-10-03T10:16:15.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.815+0000 7fded0e6f700 1 -- 172.21.15.3:0/584728857 wait complete. 2023-10-03T10:16:15.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.817+0000 7fded0e6f700 1 Processor -- start 2023-10-03T10:16:15.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.817+0000 7fded0e6f700 1 -- start start 2023-10-03T10:16:15.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.818+0000 7fded0e6f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdecc09d630 0x7fdecc11f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:15.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.818+0000 7fded0e6f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdecc0a0a30 0x7fdecc11fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:15.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.818+0000 7fded0e6f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdecc133930 0x7fdecc1c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:15.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.818+0000 7fded0e6f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc0a0410 con 0x7fdecc0a0a30 2023-10-03T10:16:15.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.818+0000 7fded0e6f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc122640 con 0x7fdecc09d630 2023-10-03T10:16:15.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.818+0000 7fded0e6f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc11ef20 con 0x7fdecc133930 2023-10-03T10:16:15.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.818+0000 7fdeca59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdecc09d630 0x7fdecc11f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:15.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.818+0000 7fdeca59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdecc09d630 0x7fdecc11f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44258/0 (socket says 172.21.15.3:44258) 2023-10-03T10:16:15.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.818+0000 7fdeca59c700 1 -- 172.21.15.3:0/1839505680 learned_addr learned my addr 172.21.15.3:0/1839505680 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:15.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.818+0000 7fdecad9d700 1 --2- 172.21.15.3:0/1839505680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdecc133930 0x7fdecc1c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:15.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.818+0000 7fdeca59c700 1 -- 172.21.15.3:0/1839505680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdecc133930 msgr2=0x7fdecc1c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:15.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.818+0000 7fdeca59c700 1 --2- 172.21.15.3:0/1839505680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdecc133930 0x7fdecc1c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.818+0000 7fdeca59c700 1 -- 172.21.15.3:0/1839505680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdecc0a0a30 msgr2=0x7fdecc11fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:15.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.818+0000 7fdeca59c700 1 --2- 172.21.15.3:0/1839505680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdecc0a0a30 0x7fdecc11fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.818+0000 7fdeca59c700 1 -- 172.21.15.3:0/1839505680 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec4009a70 con 0x7fdecc09d630 2023-10-03T10:16:15.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.818+0000 7fdecad9d700 1 --2- 172.21.15.3:0/1839505680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdecc133930 0x7fdecc1c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:16:15.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.819+0000 7fdeca59c700 1 --2- 172.21.15.3:0/1839505680 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdecc09d630 0x7fdecc11f4c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fdebc00e810 tx=0x7fdebc00ebd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:15.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.819+0000 7fdebaffd700 1 -- 172.21.15.3:0/1839505680 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc013610 con 0x7fdecc09d630 2023-10-03T10:16:15.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.819+0000 7fdebaffd700 1 -- 172.21.15.3:0/1839505680 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdebc013770 con 0x7fdecc09d630 2023-10-03T10:16:15.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.819+0000 7fdebaffd700 1 -- 172.21.15.3:0/1839505680 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc013b20 con 0x7fdecc09d630 2023-10-03T10:16:15.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.819+0000 7fded0e6f700 1 -- 172.21.15.3:0/1839505680 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec0009550 con 0x7fdecc09d630 2023-10-03T10:16:15.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.819+0000 7fded0e6f700 1 -- 172.21.15.3:0/1839505680 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdecc121fa0 con 0x7fdecc09d630 2023-10-03T10:16:15.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.820+0000 7fded0e6f700 1 -- 172.21.15.3:0/1839505680 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdecc120690 con 0x7fdecc09d630 2023-10-03T10:16:15.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.821+0000 7fdebaffd700 1 -- 172.21.15.3:0/1839505680 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdebc013c80 con 0x7fdecc09d630 2023-10-03T10:16:15.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.821+0000 7fdebaffd700 1 -- 172.21.15.3:0/1839505680 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fdebc03f9d0 con 0x7fdecc09d630 2023-10-03T10:16:15.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.825+0000 7fdebaffd700 1 -- 172.21.15.3:0/1839505680 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdebc026480 con 0x7fdecc09d630 2023-10-03T10:16:15.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.915+0000 7f007ed57700 1 -- 172.21.15.3:0/421891585 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f00781a6220 con 0x7f007809e000 2023-10-03T10:16:15.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.916+0000 7f00757fa700 1 -- 172.21.15.3:0/421891585 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f00781a6220 con 0x7f007809e000 2023-10-03T10:16:15.916 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:15.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.919+0000 7f007ed57700 1 -- 172.21.15.3:0/421891585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f007809e000 msgr2=0x7f00781a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:15.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.919+0000 7f007ed57700 1 --2- 172.21.15.3:0/421891585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f007809e000 0x7f00781a5290 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f007809ac00 tx=0x7f0064004750 comp rx=0 tx=0).stop 2023-10-03T10:16:15.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.919+0000 7f007ed57700 1 -- 172.21.15.3:0/421891585 shutdown_connections 2023-10-03T10:16:15.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.919+0000 7f007ed57700 1 --2- 172.21.15.3:0/421891585 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f007809e410 0x7f00781a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.919+0000 7f007ed57700 1 --2- 172.21.15.3:0/421891585 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f007809d6f0 0x7f00781a4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.919+0000 7f007ed57700 1 --2- 172.21.15.3:0/421891585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f007809e000 0x7f00781a5290 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.919+0000 7f007ed57700 1 -- 172.21.15.3:0/421891585 >> 172.21.15.3:0/421891585 conn(0x7f0078066f50 msgr2=0x7f00781b9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:15.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.919+0000 7f007ed57700 1 -- 172.21.15.3:0/421891585 shutdown_connections 2023-10-03T10:16:15.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.919+0000 7f007ed57700 1 -- 172.21.15.3:0/421891585 wait complete. 2023-10-03T10:16:15.938 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:15.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.975+0000 7f6bc678d700 1 -- 172.21.15.3:0/2080672777 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6bc0072980 con 0x7f6bc0133930 2023-10-03T10:16:15.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.975+0000 7f6bbcff9700 1 -- 172.21.15.3:0/2080672777 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6bb8033090 con 0x7f6bc0133930 2023-10-03T10:16:15.975 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:15.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.978+0000 7f6bc678d700 1 -- 172.21.15.3:0/2080672777 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6bc0133930 msgr2=0x7f6bc01c8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:15.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.978+0000 7f6bc678d700 1 --2- 172.21.15.3:0/2080672777 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6bc0133930 0x7f6bc01c8a00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8015000 tx=0x7f6bb8004ad0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.979+0000 7f6bc678d700 1 -- 172.21.15.3:0/2080672777 shutdown_connections 2023-10-03T10:16:15.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.979+0000 7f6bc678d700 1 --2- 172.21.15.3:0/2080672777 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6bc0133930 0x7f6bc01c8a00 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.979+0000 7f6bc678d700 1 --2- 172.21.15.3:0/2080672777 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6bc009d630 0x7f6bc01c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.979+0000 7f6bc678d700 1 --2- 172.21.15.3:0/2080672777 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bc00a0a30 0x7f6bc01c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:15.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.979+0000 7f6bc678d700 1 -- 172.21.15.3:0/2080672777 >> 172.21.15.3:0/2080672777 conn(0x7f6bc00a3050 msgr2=0x7f6bc009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:15.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.979+0000 7f6bc678d700 1 -- 172.21.15.3:0/2080672777 shutdown_connections 2023-10-03T10:16:15.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:15.979+0000 7f6bc678d700 1 -- 172.21.15.3:0/2080672777 wait complete. 2023-10-03T10:16:15.997 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:16.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:16.007+0000 7f285fccd700 1 -- 172.21.15.3:0/621177798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2858074ea0 con 0x7f285809d6f0 2023-10-03T10:16:16.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:16.008+0000 7f284e7fc700 1 -- 172.21.15.3:0/621177798 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2854012370 con 0x7f285809d6f0 2023-10-03T10:16:16.008 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:16.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:16.010+0000 7f285fccd700 1 -- 172.21.15.3:0/621177798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285809d6f0 msgr2=0x7f28581a5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:16.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:16.010+0000 7f285fccd700 1 --2- 172.21.15.3:0/621177798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285809d6f0 0x7f28581a5d60 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f285400e840 tx=0x7f285400eb50 comp rx=0 tx=0).stop 2023-10-03T10:16:16.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:16.010+0000 7f285fccd700 1 -- 172.21.15.3:0/621177798 shutdown_connections 2023-10-03T10:16:16.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:16.010+0000 7f285fccd700 1 --2- 172.21.15.3:0/621177798 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285809e410 0x7f28581a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:16.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:16.010+0000 7f285fccd700 1 --2- 172.21.15.3:0/621177798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285809d6f0 0x7f28581a5d60 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:16.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:16.010+0000 7f285fccd700 1 --2- 172.21.15.3:0/621177798 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f285809e000 0x7f28581a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:16.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:16.010+0000 7f285fccd700 1 -- 172.21.15.3:0/621177798 >> 172.21.15.3:0/621177798 conn(0x7f2858066f50 msgr2=0x7f28581bad10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:16.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:16.010+0000 7f285fccd700 1 -- 172.21.15.3:0/621177798 shutdown_connections 2023-10-03T10:16:16.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:16.010+0000 7f285fccd700 1 -- 172.21.15.3:0/621177798 wait complete. 2023-10-03T10:16:16.024 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:16.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:16.041+0000 7fded0e6f700 1 -- 172.21.15.3:0/1839505680 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdecc0747f0 con 0x7fdecc09d630 2023-10-03T10:16:16.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:16.041+0000 7fdebaffd700 1 -- 172.21.15.3:0/1839505680 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdebc021070 con 0x7fdecc09d630 2023-10-03T10:16:16.042 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:16.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:16.043+0000 7fded0e6f700 1 -- 172.21.15.3:0/1839505680 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdecc09d630 msgr2=0x7fdecc11f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:16.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:16.043+0000 7fded0e6f700 1 --2- 172.21.15.3:0/1839505680 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdecc09d630 0x7fdecc11f4c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fdebc00e810 tx=0x7fdebc00ebd0 comp rx=0 tx=0).stop 2023-10-03T10:16:16.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:16.043+0000 7fded0e6f700 1 -- 172.21.15.3:0/1839505680 shutdown_connections 2023-10-03T10:16:16.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:16.043+0000 7fded0e6f700 1 --2- 172.21.15.3:0/1839505680 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdecc133930 0x7fdecc1c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:16.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:16.043+0000 7fded0e6f700 1 --2- 172.21.15.3:0/1839505680 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdecc09d630 0x7fdecc11f4c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:16.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:16.043+0000 7fded0e6f700 1 --2- 172.21.15.3:0/1839505680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdecc0a0a30 0x7fdecc11fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:16.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:16.043+0000 7fded0e6f700 1 -- 172.21.15.3:0/1839505680 >> 172.21.15.3:0/1839505680 conn(0x7fdecc0a3050 msgr2=0x7fdecc09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:16.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:16.043+0000 7fded0e6f700 1 -- 172.21.15.3:0/1839505680 shutdown_connections 2023-10-03T10:16:16.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:16.043+0000 7fded0e6f700 1 -- 172.21.15.3:0/1839505680 wait complete. 2023-10-03T10:16:16.057 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:16.939 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:16.999 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:17.025 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:17.057 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:17.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.096+0000 7fc4feee9700 1 Processor -- start 2023-10-03T10:16:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.096+0000 7fc4feee9700 1 -- start start 2023-10-03T10:16:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.096+0000 7fc4feee9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc4f806c0e0 0x7fc4f809ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.096+0000 7fc4feee9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc4f8134ec0 0x7fc4f81379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.096+0000 7fc4feee9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc4f8137f20 0x7fc4f8138370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.096+0000 7fc4feee9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f806c4b0 con 0x7fc4f806c0e0 2023-10-03T10:16:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.096+0000 7fc4feee9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f81388b0 con 0x7fc4f8134ec0 2023-10-03T10:16:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.096+0000 7fc4feee9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f80a09c0 con 0x7fc4f8137f20 2023-10-03T10:16:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.096+0000 7fc4f7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc4f8134ec0 0x7fc4f81379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:17.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.096+0000 7fc4f7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc4f8134ec0 0x7fc4f81379e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44268/0 (socket says 172.21.15.3:44268) 2023-10-03T10:16:17.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.097+0000 7fc4f7fff700 1 -- 172.21.15.3:0/2182465475 learned_addr learned my addr 172.21.15.3:0/2182465475 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:17.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.097+0000 7fc4fcc85700 1 --2- 172.21.15.3:0/2182465475 >> v2:172.21.15.3:3300/0 conn(0x7fc4f806c0e0 0x7fc4f809ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:17.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.097+0000 7fc4f7fff700 1 -- 172.21.15.3:0/2182465475 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc4f8137f20 msgr2=0x7fc4f8138370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:17.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.097+0000 7fc4f7fff700 1 --2- 172.21.15.3:0/2182465475 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc4f8137f20 0x7fc4f8138370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.097+0000 7fc4f7fff700 1 -- 172.21.15.3:0/2182465475 >> v2:172.21.15.3:3300/0 conn(0x7fc4f806c0e0 msgr2=0x7fc4f809ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:17.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.097+0000 7fc4f7fff700 1 --2- 172.21.15.3:0/2182465475 >> v2:172.21.15.3:3300/0 conn(0x7fc4f806c0e0 0x7fc4f809ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.097+0000 7fc4f7fff700 1 -- 172.21.15.3:0/2182465475 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f8134980 con 0x7fc4f8134ec0 2023-10-03T10:16:17.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.097+0000 7fc4f7fff700 1 --2- 172.21.15.3:0/2182465475 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc4f8134ec0 0x7fc4f81379e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc4e40098f0 tx=0x7fc4e4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=819eaecfab5506fb server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:17.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.097+0000 7fc4f77fe700 1 -- 172.21.15.3:0/2182465475 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e400b8a0 con 0x7fc4f8134ec0 2023-10-03T10:16:17.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.097+0000 7fc4f77fe700 1 -- 172.21.15.3:0/2182465475 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e400ba00 con 0x7fc4f8134ec0 2023-10-03T10:16:17.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.097+0000 7fc4f77fe700 1 -- 172.21.15.3:0/2182465475 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e400bd90 con 0x7fc4f8134ec0 2023-10-03T10:16:17.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.098+0000 7fc4feee9700 1 -- 172.21.15.3:0/2182465475 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc4f8134ec0 msgr2=0x7fc4f81379e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:17.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.098+0000 7fc4feee9700 1 --2- 172.21.15.3:0/2182465475 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc4f8134ec0 0x7fc4f81379e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc4e40098f0 tx=0x7fc4e4009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:17.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.098+0000 7fc4feee9700 1 -- 172.21.15.3:0/2182465475 shutdown_connections 2023-10-03T10:16:17.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.098+0000 7fc4feee9700 1 --2- 172.21.15.3:0/2182465475 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc4f8137f20 0x7fc4f8138370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.098+0000 7fc4feee9700 1 --2- 172.21.15.3:0/2182465475 >> v2:172.21.15.3:3300/0 conn(0x7fc4f806c0e0 0x7fc4f809ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.098+0000 7fc4feee9700 1 --2- 172.21.15.3:0/2182465475 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc4f8134ec0 0x7fc4f81379e0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.098+0000 7fc4feee9700 1 -- 172.21.15.3:0/2182465475 >> 172.21.15.3:0/2182465475 conn(0x7fc4f80a4210 msgr2=0x7fc4f80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:17.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.098+0000 7fc4feee9700 1 -- 172.21.15.3:0/2182465475 shutdown_connections 2023-10-03T10:16:17.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.099+0000 7fc4feee9700 1 -- 172.21.15.3:0/2182465475 wait complete. 2023-10-03T10:16:17.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.100+0000 7fc4feee9700 1 Processor -- start 2023-10-03T10:16:17.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.100+0000 7fc4feee9700 1 -- start start 2023-10-03T10:16:17.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.101+0000 7fc4feee9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4f806c0e0 0x7fc4f8134100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:17.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.101+0000 7fc4feee9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc4f8134ec0 0x7fc4f811a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:17.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.101+0000 7fc4feee9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc4f8137f20 0x7fc4f811ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:17.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.101+0000 7fc4feee9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f811c6c0 con 0x7fc4f806c0e0 2023-10-03T10:16:17.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.101+0000 7fc4feee9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f811c4d0 con 0x7fc4f8137f20 2023-10-03T10:16:17.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.101+0000 7fc4feee9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f8134640 con 0x7fc4f8134ec0 2023-10-03T10:16:17.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.101+0000 7fc4fcc85700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4f806c0e0 0x7fc4f8134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:17.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.101+0000 7fc4fd486700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc4f8137f20 0x7fc4f811ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:17.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.101+0000 7fc4fcc85700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4f806c0e0 0x7fc4f8134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49768/0 (socket says 172.21.15.3:49768) 2023-10-03T10:16:17.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.101+0000 7fc4fcc85700 1 -- 172.21.15.3:0/177879095 learned_addr learned my addr 172.21.15.3:0/177879095 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:17.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.101+0000 7fc4f7fff700 1 --2- 172.21.15.3:0/177879095 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc4f8134ec0 0x7fc4f811a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:17.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.101+0000 7fc4fd486700 1 -- 172.21.15.3:0/177879095 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc4f8134ec0 msgr2=0x7fc4f811a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:17.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.101+0000 7fc4fd486700 1 --2- 172.21.15.3:0/177879095 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc4f8134ec0 0x7fc4f811a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.101+0000 7fc4fd486700 1 -- 172.21.15.3:0/177879095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4f806c0e0 msgr2=0x7fc4f8134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:17.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.101+0000 7fc4fd486700 1 --2- 172.21.15.3:0/177879095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4f806c0e0 0x7fc4f8134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.101+0000 7fc4fd486700 1 -- 172.21.15.3:0/177879095 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e4009550 con 0x7fc4f8137f20 2023-10-03T10:16:17.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.101+0000 7fc4fcc85700 1 --2- 172.21.15.3:0/177879095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4f806c0e0 0x7fc4f8134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:17.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.102+0000 7fc4fd486700 1 --2- 172.21.15.3:0/177879095 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc4f8137f20 0x7fc4f811ae60 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc4f000be50 tx=0x7fc4f0009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:17.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.102+0000 7fc4f57fa700 1 -- 172.21.15.3:0/177879095 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f000cb90 con 0x7fc4f8137f20 2023-10-03T10:16:17.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.102+0000 7fc4f57fa700 1 -- 172.21.15.3:0/177879095 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f000cd10 con 0x7fc4f8137f20 2023-10-03T10:16:17.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.102+0000 7fc4feee9700 1 -- 172.21.15.3:0/177879095 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f811cd70 con 0x7fc4f8137f20 2023-10-03T10:16:17.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.102+0000 7fc4f57fa700 1 -- 172.21.15.3:0/177879095 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0014680 con 0x7fc4f8137f20 2023-10-03T10:16:17.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.102+0000 7fc4feee9700 1 -- 172.21.15.3:0/177879095 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f80a03f0 con 0x7fc4f8137f20 2023-10-03T10:16:17.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.103+0000 7fc4f57fa700 1 -- 172.21.15.3:0/177879095 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc4f001d3b0 con 0x7fc4f8137f20 2023-10-03T10:16:17.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.103+0000 7fc4feee9700 1 -- 172.21.15.3:0/177879095 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f8074ea0 con 0x7fc4f8137f20 2023-10-03T10:16:17.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.103+0000 7fc4f57fa700 1 -- 172.21.15.3:0/177879095 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc4f0006e40 con 0x7fc4f8137f20 2023-10-03T10:16:17.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.109+0000 7fc4f57fa700 1 -- 172.21.15.3:0/177879095 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc4f00076c0 con 0x7fc4f8137f20 2023-10-03T10:16:17.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.165+0000 7f03b94e1700 1 Processor -- start 2023-10-03T10:16:17.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.165+0000 7f03b94e1700 1 -- start start 2023-10-03T10:16:17.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.166+0000 7f03b94e1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f03b409d6f0 0x7f03b409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:17.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.166+0000 7f03b94e1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f03b409e000 0x7f03b41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:17.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.166+0000 7f03b94e1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f03b4133f30 0x7f03b4134390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:17.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.166+0000 7f03b94e1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03b409b0f0 con 0x7f03b409d6f0 2023-10-03T10:16:17.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.166+0000 7f03b94e1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b409d2f0 con 0x7f03b409e000 2023-10-03T10:16:17.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.166+0000 7f03b94e1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03b409a810 con 0x7f03b4133f30 2023-10-03T10:16:17.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.166+0000 7f03b27fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f03b409e000 0x7f03b41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:17.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.166+0000 7f03b37fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f03b4133f30 0x7f03b4134390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:17.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.166+0000 7f03b27fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f03b409e000 0x7f03b41339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44288/0 (socket says 172.21.15.3:44288) 2023-10-03T10:16:17.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.166+0000 7f03b2ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f03b409d6f0 0x7f03b409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:17.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.166+0000 7f03b27fc700 1 -- 172.21.15.3:0/2347837869 learned_addr learned my addr 172.21.15.3:0/2347837869 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:17.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.166+0000 7f03b37fe700 1 --2- 172.21.15.3:0/2347837869 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f03b4133f30 0x7f03b4134390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:33098/0 (socket says 172.21.15.3:33098) 2023-10-03T10:16:17.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.167+0000 7f03b27fc700 1 -- 172.21.15.3:0/2347837869 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f03b4133f30 msgr2=0x7f03b4134390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:17.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.167+0000 7f03b27fc700 1 --2- 172.21.15.3:0/2347837869 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f03b4133f30 0x7f03b4134390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.167+0000 7f03b27fc700 1 -- 172.21.15.3:0/2347837869 >> v2:172.21.15.3:3300/0 conn(0x7f03b409d6f0 msgr2=0x7f03b409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:17.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.167+0000 7f03b27fc700 1 --2- 172.21.15.3:0/2347837869 >> v2:172.21.15.3:3300/0 conn(0x7f03b409d6f0 0x7f03b409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.167+0000 7f03b27fc700 1 -- 172.21.15.3:0/2347837869 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b409a470 con 0x7f03b409e000 2023-10-03T10:16:17.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.167+0000 7f03b27fc700 1 --2- 172.21.15.3:0/2347837869 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f03b409e000 0x7f03b41339f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f03a80098f0 tx=0x7f03a8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13c61055119c8d81 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:17.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.167+0000 7f03b1ffb700 1 -- 172.21.15.3:0/2347837869 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8011830 con 0x7f03b409e000 2023-10-03T10:16:17.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.167+0000 7f03b1ffb700 1 -- 172.21.15.3:0/2347837869 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a8011990 con 0x7f03b409e000 2023-10-03T10:16:17.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.167+0000 7f03b1ffb700 1 -- 172.21.15.3:0/2347837869 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8011d20 con 0x7f03b409e000 2023-10-03T10:16:17.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.168+0000 7f03b94e1700 1 -- 172.21.15.3:0/2347837869 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f03b409e000 msgr2=0x7f03b41339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:17.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.168+0000 7f03b94e1700 1 --2- 172.21.15.3:0/2347837869 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f03b409e000 0x7f03b41339f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f03a80098f0 tx=0x7f03a8009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:17.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.168+0000 7f03b94e1700 1 -- 172.21.15.3:0/2347837869 shutdown_connections 2023-10-03T10:16:17.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.168+0000 7f03b94e1700 1 --2- 172.21.15.3:0/2347837869 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f03b4133f30 0x7f03b4134390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.168+0000 7f03b94e1700 1 --2- 172.21.15.3:0/2347837869 >> v2:172.21.15.3:3300/0 conn(0x7f03b409d6f0 0x7f03b409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.168+0000 7f03b94e1700 1 --2- 172.21.15.3:0/2347837869 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f03b409e000 0x7f03b41339f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.168+0000 7f03b94e1700 1 -- 172.21.15.3:0/2347837869 >> 172.21.15.3:0/2347837869 conn(0x7f03b4066f50 msgr2=0x7f03b409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:17.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.168+0000 7f03b94e1700 1 -- 172.21.15.3:0/2347837869 shutdown_connections 2023-10-03T10:16:17.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.168+0000 7f03b94e1700 1 -- 172.21.15.3:0/2347837869 wait complete. 2023-10-03T10:16:17.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.170+0000 7f03b94e1700 1 Processor -- start 2023-10-03T10:16:17.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.170+0000 7f03b94e1700 1 -- start start 2023-10-03T10:16:17.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.171+0000 7f03b94e1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03b409d6f0 0x7f03b411e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:17.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.171+0000 7f03b94e1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f03b409e000 0x7f03b411eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:17.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.171+0000 7f03b94e1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f03b4133f30 0x7f03b41c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:17.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.171+0000 7f03b94e1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b41216a0 con 0x7f03b409d6f0 2023-10-03T10:16:17.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.171+0000 7f03b94e1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b411dfe0 con 0x7f03b4133f30 2023-10-03T10:16:17.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.171+0000 7f03b94e1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03b411f6b0 con 0x7f03b409e000 2023-10-03T10:16:17.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.171+0000 7f03b27fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f03b409e000 0x7f03b411eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:17.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.171+0000 7f03b37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f03b4133f30 0x7f03b41c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.171+0000 7f03b2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03b409d6f0 0x7f03b411e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.171+0000 7f03b27fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f03b409e000 0x7f03b411eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:33108/0 (socket says 172.21.15.3:33108) 2023-10-03T10:16:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.171+0000 7f03b27fc700 1 -- 172.21.15.3:0/629792506 learned_addr learned my addr 172.21.15.3:0/629792506 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.171+0000 7f03b37fe700 1 -- 172.21.15.3:0/629792506 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f03b409e000 msgr2=0x7f03b411eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.171+0000 7f03b37fe700 1 --2- 172.21.15.3:0/629792506 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f03b409e000 0x7f03b411eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.171+0000 7f03b37fe700 1 -- 172.21.15.3:0/629792506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03b409d6f0 msgr2=0x7f03b411e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.171+0000 7f03b37fe700 1 --2- 172.21.15.3:0/629792506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03b409d6f0 0x7f03b411e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.171+0000 7f03b37fe700 1 -- 172.21.15.3:0/629792506 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a8009550 con 0x7f03b4133f30 2023-10-03T10:16:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.171+0000 7f03b2ffd700 1 --2- 172.21.15.3:0/629792506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03b409d6f0 0x7f03b411e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:16:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.172+0000 7f03b37fe700 1 --2- 172.21.15.3:0/629792506 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f03b4133f30 0x7f03b41c8040 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00efc0 tx=0x7f03ac00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:17.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.172+0000 7f039b7fe700 1 -- 172.21.15.3:0/629792506 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac00cd40 con 0x7f03b4133f30 2023-10-03T10:16:17.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.172+0000 7f039b7fe700 1 -- 172.21.15.3:0/629792506 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03ac00f040 con 0x7f03b4133f30 2023-10-03T10:16:17.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.172+0000 7f039b7fe700 1 -- 172.21.15.3:0/629792506 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac00cea0 con 0x7f03b4133f30 2023-10-03T10:16:17.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.172+0000 7f03b94e1700 1 -- 172.21.15.3:0/629792506 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b409bed0 con 0x7f03b4133f30 2023-10-03T10:16:17.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.172+0000 7f03b94e1700 1 -- 172.21.15.3:0/629792506 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b4121000 con 0x7f03b4133f30 2023-10-03T10:16:17.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.173+0000 7f039b7fe700 1 -- 172.21.15.3:0/629792506 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f03ac0146c0 con 0x7f03b4133f30 2023-10-03T10:16:17.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.173+0000 7f039b7fe700 1 -- 172.21.15.3:0/629792506 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f03ac0401c0 con 0x7f03b4133f30 2023-10-03T10:16:17.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.173+0000 7f03b94e1700 1 -- 172.21.15.3:0/629792506 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03b4074ea0 con 0x7f03b4133f30 2023-10-03T10:16:17.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.178+0000 7f039b7fe700 1 -- 172.21.15.3:0/629792506 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f03ac018020 con 0x7f03b4133f30 2023-10-03T10:16:17.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.195+0000 7f6eab8dd700 1 Processor -- start 2023-10-03T10:16:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.196+0000 7f6eab8dd700 1 -- start start 2023-10-03T10:16:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.196+0000 7f6eab8dd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ea409d6f0 0x7f6ea409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:17.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.196+0000 7f6eab8dd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ea409e000 0x7f6ea4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:17.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.196+0000 7f6eab8dd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ea409f130 con 0x7f6ea409e410 2023-10-03T10:16:17.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.196+0000 7f6eab8dd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea409f520 con 0x7f6ea409e000 2023-10-03T10:16:17.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.196+0000 7f6eab8dd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea409e930 con 0x7f6ea409d6f0 2023-10-03T10:16:17.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.196+0000 7f6ea8e78700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ea409e000 0x7f6ea4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:17.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.196+0000 7f6ea9e7a700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6ea409e410 0x7f6ea4130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:41300/0 (socket says 172.21.15.3:41300) 2023-10-03T10:16:17.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.196+0000 7f6ea9e7a700 1 -- 172.21.15.3:0/2619943285 learned_addr learned my addr 172.21.15.3:0/2619943285 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:17.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.197+0000 7f6ea9679700 1 --2- 172.21.15.3:0/2619943285 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ea409d6f0 0x7f6ea409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:17.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.197+0000 7f6ea9679700 1 -- 172.21.15.3:0/2619943285 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ea409e000 msgr2=0x7f6ea4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:17.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.197+0000 7f6ea9679700 1 --2- 172.21.15.3:0/2619943285 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ea409e000 0x7f6ea4130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.197+0000 7f6ea9679700 1 -- 172.21.15.3:0/2619943285 >> v1:172.21.15.3:6789/0 conn(0x7f6ea409e410 legacy=0x7f6ea4130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:17.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.197+0000 7f6ea9679700 1 -- 172.21.15.3:0/2619943285 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea409b2d0 con 0x7f6ea409d6f0 2023-10-03T10:16:17.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.197+0000 7f6e9bfff700 1 -- 172.21.15.3:0/2619943285 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 538066784 0 0) 0x7f6ea409f130 con 0x7f6ea409e410 2023-10-03T10:16:17.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.197+0000 7f6ea8e78700 1 --2- 172.21.15.3:0/2619943285 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ea409e000 0x7f6ea4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:17.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.197+0000 7f6ea9679700 1 --2- 172.21.15.3:0/2619943285 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ea409d6f0 0x7f6ea409dac0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6ea00098a0 tx=0x7f6ea0009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7906454931466822 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:17.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.197+0000 7f6e9bfff700 1 -- 172.21.15.3:0/2619943285 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea000d980 con 0x7f6ea409d6f0 2023-10-03T10:16:17.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.197+0000 7f6e9bfff700 1 -- 172.21.15.3:0/2619943285 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea000dae0 con 0x7f6ea409d6f0 2023-10-03T10:16:17.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.197+0000 7f6e9bfff700 1 -- 172.21.15.3:0/2619943285 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea000de70 con 0x7f6ea409d6f0 2023-10-03T10:16:17.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.198+0000 7f6eab8dd700 1 -- 172.21.15.3:0/2619943285 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ea409d6f0 msgr2=0x7f6ea409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:17.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.198+0000 7f6eab8dd700 1 --2- 172.21.15.3:0/2619943285 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ea409d6f0 0x7f6ea409dac0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6ea00098a0 tx=0x7f6ea0009bb0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.198+0000 7f6eab8dd700 1 -- 172.21.15.3:0/2619943285 shutdown_connections 2023-10-03T10:16:17.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.198+0000 7f6eab8dd700 1 --2- 172.21.15.3:0/2619943285 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ea409d6f0 0x7f6ea409dac0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.198+0000 7f6eab8dd700 1 --2- 172.21.15.3:0/2619943285 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ea409e000 0x7f6ea4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.198+0000 7f6eab8dd700 1 -- 172.21.15.3:0/2619943285 >> 172.21.15.3:0/2619943285 conn(0x7f6ea4066f50 msgr2=0x7f6ea409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:17.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.198+0000 7f6eab8dd700 1 -- 172.21.15.3:0/2619943285 shutdown_connections 2023-10-03T10:16:17.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.198+0000 7f6eab8dd700 1 -- 172.21.15.3:0/2619943285 wait complete. 2023-10-03T10:16:17.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.199+0000 7f6eab8dd700 1 Processor -- start 2023-10-03T10:16:17.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.200+0000 7f6eab8dd700 1 -- start start 2023-10-03T10:16:17.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.200+0000 7f6eab8dd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ea409d6f0 0x7f6ea4122250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:17.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.200+0000 7f6eab8dd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ea409e000 0x7f6ea4122790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:17.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.200+0000 7f6eab8dd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ea409e410 0x7f6ea4122cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:17.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.200+0000 7f6eab8dd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea40a3bb0 con 0x7f6ea409d6f0 2023-10-03T10:16:17.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.200+0000 7f6eab8dd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea409c700 con 0x7f6ea409e000 2023-10-03T10:16:17.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.200+0000 7f6eab8dd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea4125500 con 0x7f6ea409e410 2023-10-03T10:16:17.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.200+0000 7f6ea9679700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ea409d6f0 0x7f6ea4122250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:17.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.200+0000 7f6ea9679700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ea409d6f0 0x7f6ea4122250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49804/0 (socket says 172.21.15.3:49804) 2023-10-03T10:16:17.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.200+0000 7f6ea9679700 1 -- 172.21.15.3:0/1467762609 learned_addr learned my addr 172.21.15.3:0/1467762609 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:17.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.200+0000 7f6ea9e7a700 1 --2- 172.21.15.3:0/1467762609 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ea409e410 0x7f6ea4122cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:17.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.200+0000 7f6ea8e78700 1 --2- 172.21.15.3:0/1467762609 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ea409e000 0x7f6ea4122790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:17.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.201+0000 7f6ea8e78700 1 -- 172.21.15.3:0/1467762609 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ea409e410 msgr2=0x7f6ea4122cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:17.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.201+0000 7f6ea8e78700 1 --2- 172.21.15.3:0/1467762609 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ea409e410 0x7f6ea4122cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.201+0000 7f6ea8e78700 1 -- 172.21.15.3:0/1467762609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ea409d6f0 msgr2=0x7f6ea4122250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:17.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.201+0000 7f6ea8e78700 1 --2- 172.21.15.3:0/1467762609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ea409d6f0 0x7f6ea4122250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.201+0000 7f6ea8e78700 1 -- 172.21.15.3:0/1467762609 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e940094a0 con 0x7f6ea409e000 2023-10-03T10:16:17.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.201+0000 7f6ea9e7a700 1 --2- 172.21.15.3:0/1467762609 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ea409e410 0x7f6ea4122cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:17.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.201+0000 7f6ea9679700 1 --2- 172.21.15.3:0/1467762609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ea409d6f0 0x7f6ea4122250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:16:17.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.201+0000 7f6ea8e78700 1 --2- 172.21.15.3:0/1467762609 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ea409e000 0x7f6ea4122790 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400e980 tx=0x7f6e9400ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:17.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.201+0000 7f6e99ffb700 1 -- 172.21.15.3:0/1467762609 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9400eed0 con 0x7f6ea409e000 2023-10-03T10:16:17.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.201+0000 7f6e99ffb700 1 -- 172.21.15.3:0/1467762609 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9400c850 con 0x7f6ea409e000 2023-10-03T10:16:17.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.201+0000 7f6eab8dd700 1 -- 172.21.15.3:0/1467762609 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea00094a0 con 0x7f6ea409e000 2023-10-03T10:16:17.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.201+0000 7f6e99ffb700 1 -- 172.21.15.3:0/1467762609 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9400cbe0 con 0x7f6ea409e000 2023-10-03T10:16:17.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.202+0000 7f6eab8dd700 1 -- 172.21.15.3:0/1467762609 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea4124610 con 0x7f6ea409e000 2023-10-03T10:16:17.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.203+0000 7f6e99ffb700 1 -- 172.21.15.3:0/1467762609 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6e9400cd40 con 0x7f6ea409e000 2023-10-03T10:16:17.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.203+0000 7f6eab8dd700 1 -- 172.21.15.3:0/1467762609 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea411ac20 con 0x7f6ea409e000 2023-10-03T10:16:17.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.203+0000 7f6e99ffb700 1 -- 172.21.15.3:0/1467762609 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6e9401bd80 con 0x7f6ea409e000 2023-10-03T10:16:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.212+0000 7f6e99ffb700 1 -- 172.21.15.3:0/1467762609 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6e94017070 con 0x7f6ea409e000 2023-10-03T10:16:17.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.253+0000 7fc5a05c0700 1 Processor -- start 2023-10-03T10:16:17.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.254+0000 7fc5a05c0700 1 -- start start 2023-10-03T10:16:17.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.254+0000 7fc5a05c0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc598133a80 0x7fc598133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:17.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.254+0000 7fc5a05c0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc59812ff20 0x7fc598130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:17.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.254+0000 7fc5a05c0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc598130870 0x7fc59813a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:17.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.254+0000 7fc5a05c0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc59806c570 con 0x7fc598130870 2023-10-03T10:16:17.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.254+0000 7fc5a05c0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc59806bc00 con 0x7fc59812ff20 2023-10-03T10:16:17.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.254+0000 7fc5a05c0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc5980a0040 con 0x7fc598133a80 2023-10-03T10:16:17.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.255+0000 7fc59e35c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc598133a80 0x7fc598133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:17.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.255+0000 7fc59eb5d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc598130870 0x7fc59813a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:17.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.255+0000 7fc59db5b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc59812ff20 0x7fc598130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:17.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.255+0000 7fc59eb5d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc598130870 0x7fc59813a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49814/0 (socket says 172.21.15.3:49814) 2023-10-03T10:16:17.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.255+0000 7fc59db5b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc59812ff20 0x7fc598130330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44310/0 (socket says 172.21.15.3:44310) 2023-10-03T10:16:17.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.255+0000 7fc59eb5d700 1 -- 172.21.15.3:0/2553920174 learned_addr learned my addr 172.21.15.3:0/2553920174 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:17.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.255+0000 7fc59e35c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc598133a80 0x7fc598133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:33122/0 (socket says 172.21.15.3:33122) 2023-10-03T10:16:17.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.255+0000 7fc59eb5d700 1 -- 172.21.15.3:0/2553920174 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc598133a80 msgr2=0x7fc598133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:17.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.255+0000 7fc59eb5d700 1 --2- 172.21.15.3:0/2553920174 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc598133a80 0x7fc598133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.255+0000 7fc59eb5d700 1 -- 172.21.15.3:0/2553920174 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc59812ff20 msgr2=0x7fc598130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:17.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.255+0000 7fc59eb5d700 1 --2- 172.21.15.3:0/2553920174 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc59812ff20 0x7fc598130330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.255+0000 7fc59eb5d700 1 -- 172.21.15.3:0/2553920174 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc598133860 con 0x7fc598130870 2023-10-03T10:16:17.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.255+0000 7fc59db5b700 1 --2- 172.21.15.3:0/2553920174 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc59812ff20 0x7fc598130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:17.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.255+0000 7fc59eb5d700 1 --2- 172.21.15.3:0/2553920174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc598130870 0x7fc59813a840 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fc590009e50 tx=0x7fc59000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14659c896479f2d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:17.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.256+0000 7fc59d35a700 1 -- 172.21.15.3:0/2553920174 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59000e070 con 0x7fc598130870 2023-10-03T10:16:17.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.256+0000 7fc59d35a700 1 -- 172.21.15.3:0/2553920174 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59000b910 con 0x7fc598130870 2023-10-03T10:16:17.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.256+0000 7fc59d35a700 1 -- 172.21.15.3:0/2553920174 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59000bcc0 con 0x7fc598130870 2023-10-03T10:16:17.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.256+0000 7fc5a05c0700 1 -- 172.21.15.3:0/2553920174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc598130870 msgr2=0x7fc59813a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:17.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.256+0000 7fc5a05c0700 1 --2- 172.21.15.3:0/2553920174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc598130870 0x7fc59813a840 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fc590009e50 tx=0x7fc59000b330 comp rx=0 tx=0).stop 2023-10-03T10:16:17.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.257+0000 7fc5a05c0700 1 -- 172.21.15.3:0/2553920174 shutdown_connections 2023-10-03T10:16:17.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.257+0000 7fc5a05c0700 1 --2- 172.21.15.3:0/2553920174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc598130870 0x7fc59813a840 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.257+0000 7fc5a05c0700 1 --2- 172.21.15.3:0/2553920174 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc598133a80 0x7fc598133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.257+0000 7fc5a05c0700 1 --2- 172.21.15.3:0/2553920174 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc59812ff20 0x7fc598130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.257+0000 7fc5a05c0700 1 -- 172.21.15.3:0/2553920174 >> 172.21.15.3:0/2553920174 conn(0x7fc5980a4210 msgr2=0x7fc5980a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:17.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.257+0000 7fc5a05c0700 1 -- 172.21.15.3:0/2553920174 shutdown_connections 2023-10-03T10:16:17.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.257+0000 7fc5a05c0700 1 -- 172.21.15.3:0/2553920174 wait complete. 2023-10-03T10:16:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.258+0000 7fc5a05c0700 1 Processor -- start 2023-10-03T10:16:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.258+0000 7fc5a05c0700 1 -- start start 2023-10-03T10:16:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.258+0000 7fc5a05c0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc59812ff20 0x7fc59809c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.258+0000 7fc5a05c0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc598130870 0x7fc59809c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.258+0000 7fc5a05c0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc598133a80 0x7fc59809ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.258+0000 7fc5a05c0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59809df10 con 0x7fc598130870 2023-10-03T10:16:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.258+0000 7fc5a05c0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc59806cf30 con 0x7fc598133a80 2023-10-03T10:16:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.258+0000 7fc5a05c0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc59811fe10 con 0x7fc59812ff20 2023-10-03T10:16:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.258+0000 7fc59db5b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc598130870 0x7fc59809c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.258+0000 7fc59db5b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc598130870 0x7fc59809c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49820/0 (socket says 172.21.15.3:49820) 2023-10-03T10:16:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.258+0000 7fc59db5b700 1 -- 172.21.15.3:0/3583826705 learned_addr learned my addr 172.21.15.3:0/3583826705 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.259+0000 7fc59eb5d700 1 --2- 172.21.15.3:0/3583826705 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc598133a80 0x7fc59809ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.259+0000 7fc59e35c700 1 --2- 172.21.15.3:0/3583826705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc59812ff20 0x7fc59809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:17.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.259+0000 7fc59eb5d700 1 -- 172.21.15.3:0/3583826705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc59812ff20 msgr2=0x7fc59809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:17.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.259+0000 7fc59eb5d700 1 --2- 172.21.15.3:0/3583826705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc59812ff20 0x7fc59809c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.259+0000 7fc59eb5d700 1 -- 172.21.15.3:0/3583826705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc598130870 msgr2=0x7fc59809c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:17.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.259+0000 7fc59eb5d700 1 --2- 172.21.15.3:0/3583826705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc598130870 0x7fc59809c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.259+0000 7fc59eb5d700 1 -- 172.21.15.3:0/3583826705 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5880094a0 con 0x7fc598133a80 2023-10-03T10:16:17.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.259+0000 7fc59eb5d700 1 --2- 172.21.15.3:0/3583826705 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc598133a80 0x7fc59809ce80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc590015000 tx=0x7fc590004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:17.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.259+0000 7fc58effd700 1 -- 172.21.15.3:0/3583826705 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5900092e0 con 0x7fc598133a80 2023-10-03T10:16:17.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.259+0000 7fc58effd700 1 -- 172.21.15.3:0/3583826705 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc590004040 con 0x7fc598133a80 2023-10-03T10:16:17.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.259+0000 7fc5a05c0700 1 -- 172.21.15.3:0/3583826705 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc590009b30 con 0x7fc598133a80 2023-10-03T10:16:17.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.260+0000 7fc5a05c0700 1 -- 172.21.15.3:0/3583826705 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc59811fb20 con 0x7fc598133a80 2023-10-03T10:16:17.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.260+0000 7fc58effd700 1 -- 172.21.15.3:0/3583826705 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5900043f0 con 0x7fc598133a80 2023-10-03T10:16:17.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.261+0000 7fc5a05c0700 1 -- 172.21.15.3:0/3583826705 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5800052e0 con 0x7fc598133a80 2023-10-03T10:16:17.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.261+0000 7fc58effd700 1 -- 172.21.15.3:0/3583826705 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc590004550 con 0x7fc598133a80 2023-10-03T10:16:17.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.261+0000 7fc58effd700 1 -- 172.21.15.3:0/3583826705 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc5900407c0 con 0x7fc598133a80 2023-10-03T10:16:17.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.270+0000 7fc58effd700 1 -- 172.21.15.3:0/3583826705 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc590019350 con 0x7fc598133a80 2023-10-03T10:16:17.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.338+0000 7fc4feee9700 1 -- 172.21.15.3:0/177879095 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc4f806b250 con 0x7fc4f8137f20 2023-10-03T10:16:17.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.338+0000 7fc4f57fa700 1 -- 172.21.15.3:0/177879095 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc4f0019030 con 0x7fc4f8137f20 2023-10-03T10:16:17.338 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:17.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.340+0000 7fc4feee9700 1 -- 172.21.15.3:0/177879095 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc4f8137f20 msgr2=0x7fc4f811ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:17.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.340+0000 7fc4feee9700 1 --2- 172.21.15.3:0/177879095 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc4f8137f20 0x7fc4f811ae60 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc4f000be50 tx=0x7fc4f0009ed0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.340+0000 7fc4feee9700 1 -- 172.21.15.3:0/177879095 shutdown_connections 2023-10-03T10:16:17.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.340+0000 7fc4feee9700 1 --2- 172.21.15.3:0/177879095 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc4f8137f20 0x7fc4f811ae60 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.340+0000 7fc4feee9700 1 --2- 172.21.15.3:0/177879095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4f806c0e0 0x7fc4f8134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.340+0000 7fc4feee9700 1 --2- 172.21.15.3:0/177879095 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc4f8134ec0 0x7fc4f811a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.340+0000 7fc4feee9700 1 -- 172.21.15.3:0/177879095 >> 172.21.15.3:0/177879095 conn(0x7fc4f80a4210 msgr2=0x7fc4f8099c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:17.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.340+0000 7fc4feee9700 1 -- 172.21.15.3:0/177879095 shutdown_connections 2023-10-03T10:16:17.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.341+0000 7fc4feee9700 1 -- 172.21.15.3:0/177879095 wait complete. 2023-10-03T10:16:17.355 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:17.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.427+0000 7f6eab8dd700 1 -- 172.21.15.3:0/1467762609 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6ea4074ea0 con 0x7f6ea409e000 2023-10-03T10:16:17.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.428+0000 7f6e99ffb700 1 -- 172.21.15.3:0/1467762609 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6e94017030 con 0x7f6ea409e000 2023-10-03T10:16:17.428 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.430+0000 7f6eab8dd700 1 -- 172.21.15.3:0/1467762609 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ea409e000 msgr2=0x7f6ea4122790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.430+0000 7f6eab8dd700 1 --2- 172.21.15.3:0/1467762609 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ea409e000 0x7f6ea4122790 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400e980 tx=0x7f6e9400ec90 comp rx=0 tx=0).stop 2023-10-03T10:16:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.430+0000 7f6eab8dd700 1 -- 172.21.15.3:0/1467762609 shutdown_connections 2023-10-03T10:16:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.430+0000 7f6eab8dd700 1 --2- 172.21.15.3:0/1467762609 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ea409e410 0x7f6ea4122cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.430+0000 7f6eab8dd700 1 --2- 172.21.15.3:0/1467762609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ea409d6f0 0x7f6ea4122250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.430+0000 7f6eab8dd700 1 --2- 172.21.15.3:0/1467762609 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ea409e000 0x7f6ea4122790 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.430+0000 7f6eab8dd700 1 -- 172.21.15.3:0/1467762609 >> 172.21.15.3:0/1467762609 conn(0x7f6ea4066f50 msgr2=0x7f6ea4130900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.431+0000 7f6eab8dd700 1 -- 172.21.15.3:0/1467762609 shutdown_connections 2023-10-03T10:16:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.431+0000 7f6eab8dd700 1 -- 172.21.15.3:0/1467762609 wait complete. 2023-10-03T10:16:17.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.442+0000 7f03b94e1700 1 -- 172.21.15.3:0/629792506 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f03b4132530 con 0x7f03b4133f30 2023-10-03T10:16:17.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.442+0000 7f039b7fe700 1 -- 172.21.15.3:0/629792506 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f03ac02b090 con 0x7f03b4133f30 2023-10-03T10:16:17.443 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:17.444 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:17.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.446+0000 7f03b94e1700 1 -- 172.21.15.3:0/629792506 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f03b4133f30 msgr2=0x7f03b41c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:17.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.446+0000 7f03b94e1700 1 --2- 172.21.15.3:0/629792506 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f03b4133f30 0x7f03b41c8040 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00efc0 tx=0x7f03ac00c3a0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.446+0000 7f03b94e1700 1 -- 172.21.15.3:0/629792506 shutdown_connections 2023-10-03T10:16:17.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.446+0000 7f03b94e1700 1 --2- 172.21.15.3:0/629792506 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f03b4133f30 0x7f03b41c8040 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.446+0000 7f03b94e1700 1 --2- 172.21.15.3:0/629792506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03b409d6f0 0x7f03b411e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.446+0000 7f03b94e1700 1 --2- 172.21.15.3:0/629792506 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f03b409e000 0x7f03b411eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.446+0000 7f03b94e1700 1 -- 172.21.15.3:0/629792506 >> 172.21.15.3:0/629792506 conn(0x7f03b4066f50 msgr2=0x7f03b41bbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:17.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.446+0000 7f03b94e1700 1 -- 172.21.15.3:0/629792506 shutdown_connections 2023-10-03T10:16:17.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.446+0000 7f03b94e1700 1 -- 172.21.15.3:0/629792506 wait complete. 2023-10-03T10:16:17.464 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:17.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.536+0000 7fc5a05c0700 1 -- 172.21.15.3:0/3583826705 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc580005590 con 0x7fc598133a80 2023-10-03T10:16:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.536+0000 7fc58effd700 1 -- 172.21.15.3:0/3583826705 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc590020070 con 0x7fc598133a80 2023-10-03T10:16:17.537 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:17.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.539+0000 7fc5a05c0700 1 -- 172.21.15.3:0/3583826705 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc598133a80 msgr2=0x7fc59809ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:17.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.539+0000 7fc5a05c0700 1 --2- 172.21.15.3:0/3583826705 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc598133a80 0x7fc59809ce80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc590015000 tx=0x7fc590004ad0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.539+0000 7fc5a05c0700 1 -- 172.21.15.3:0/3583826705 shutdown_connections 2023-10-03T10:16:17.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.539+0000 7fc5a05c0700 1 --2- 172.21.15.3:0/3583826705 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc598133a80 0x7fc59809ce80 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.539+0000 7fc5a05c0700 1 --2- 172.21.15.3:0/3583826705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc59812ff20 0x7fc59809c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.539+0000 7fc5a05c0700 1 --2- 172.21.15.3:0/3583826705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc598130870 0x7fc59809c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:17.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.539+0000 7fc5a05c0700 1 -- 172.21.15.3:0/3583826705 >> 172.21.15.3:0/3583826705 conn(0x7fc5980a4210 msgr2=0x7fc59809e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:17.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.539+0000 7fc5a05c0700 1 -- 172.21.15.3:0/3583826705 shutdown_connections 2023-10-03T10:16:17.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:17.539+0000 7fc5a05c0700 1 -- 172.21.15.3:0/3583826705 wait complete. 2023-10-03T10:16:17.553 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:18.355 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:18.445 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:18.466 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:18.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.503+0000 7f85320a4700 1 Processor -- start 2023-10-03T10:16:18.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.504+0000 7f85320a4700 1 -- start start 2023-10-03T10:16:18.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.504+0000 7f85320a4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f852c09d6f0 0x7f852c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:18.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.504+0000 7f85320a4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f852c09e410 0x7f852c130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:18.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.504+0000 7f85320a4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f852c09f130 con 0x7f852c09e000 2023-10-03T10:16:18.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.504+0000 7f85320a4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852c09f520 con 0x7f852c09e410 2023-10-03T10:16:18.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.504+0000 7f85320a4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f852c09e930 con 0x7f852c09d6f0 2023-10-03T10:16:18.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.504+0000 7f852b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f852c09d6f0 0x7f852c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:18.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.504+0000 7f852b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f852c09d6f0 0x7f852c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:33138/0 (socket says 172.21.15.3:33138) 2023-10-03T10:16:18.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.504+0000 7f852b7fe700 1 -- 172.21.15.3:0/2536190334 learned_addr learned my addr 172.21.15.3:0/2536190334 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:18.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.504+0000 7f852affd700 1 --1- 172.21.15.3:0/2536190334 >> v1:172.21.15.3:6789/0 conn(0x7f852c09e000 0x7f852c130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:41314/0 (socket says 172.21.15.3:41314) 2023-10-03T10:16:18.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.505+0000 7f852a7fc700 1 -- 172.21.15.3:0/2536190334 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2911866220 0 0) 0x7f852c09f130 con 0x7f852c09e000 2023-10-03T10:16:18.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.505+0000 7f852b7fe700 1 -- 172.21.15.3:0/2536190334 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f852c09e410 msgr2=0x7f852c130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:18.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.505+0000 7f852bfff700 1 --2- 172.21.15.3:0/2536190334 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f852c09e410 0x7f852c130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:18.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.505+0000 7f852b7fe700 1 --2- 172.21.15.3:0/2536190334 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f852c09e410 0x7f852c130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.505+0000 7f852b7fe700 1 -- 172.21.15.3:0/2536190334 >> v1:172.21.15.3:6789/0 conn(0x7f852c09e000 legacy=0x7f852c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:18.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.505+0000 7f852b7fe700 1 -- 172.21.15.3:0/2536190334 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852c09b2d0 con 0x7f852c09d6f0 2023-10-03T10:16:18.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.505+0000 7f852bfff700 1 --2- 172.21.15.3:0/2536190334 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f852c09e410 0x7f852c130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:16:18.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.505+0000 7f852b7fe700 1 --2- 172.21.15.3:0/2536190334 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f852c09d6f0 0x7f852c09dac0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f851c009920 tx=0x7f851c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6b14b6988d99195 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:18.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.505+0000 7f852a7fc700 1 -- 172.21.15.3:0/2536190334 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c00ba60 con 0x7f852c09d6f0 2023-10-03T10:16:18.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.505+0000 7f852a7fc700 1 -- 172.21.15.3:0/2536190334 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851c00bbc0 con 0x7f852c09d6f0 2023-10-03T10:16:18.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.505+0000 7f852a7fc700 1 -- 172.21.15.3:0/2536190334 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c009e70 con 0x7f852c09d6f0 2023-10-03T10:16:18.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.506+0000 7f85320a4700 1 -- 172.21.15.3:0/2536190334 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f852c09d6f0 msgr2=0x7f852c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:18.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.506+0000 7f85320a4700 1 --2- 172.21.15.3:0/2536190334 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f852c09d6f0 0x7f852c09dac0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f851c009920 tx=0x7f851c009c30 comp rx=0 tx=0).stop 2023-10-03T10:16:18.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.506+0000 7f85320a4700 1 -- 172.21.15.3:0/2536190334 shutdown_connections 2023-10-03T10:16:18.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.506+0000 7f85320a4700 1 --2- 172.21.15.3:0/2536190334 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f852c09e410 0x7f852c130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.506+0000 7f85320a4700 1 --2- 172.21.15.3:0/2536190334 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f852c09d6f0 0x7f852c09dac0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.506+0000 7f85320a4700 1 -- 172.21.15.3:0/2536190334 >> 172.21.15.3:0/2536190334 conn(0x7f852c066f50 msgr2=0x7f852c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:18.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.506+0000 7f85320a4700 1 -- 172.21.15.3:0/2536190334 shutdown_connections 2023-10-03T10:16:18.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.506+0000 7f85320a4700 1 -- 172.21.15.3:0/2536190334 wait complete. 2023-10-03T10:16:18.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.507+0000 7f85320a4700 1 Processor -- start 2023-10-03T10:16:18.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.508+0000 7f85320a4700 1 -- start start 2023-10-03T10:16:18.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.508+0000 7f85320a4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f852c09d6f0 0x7f852c1221b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:18.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.508+0000 7f85320a4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f852c09e000 0x7f852c1226f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:18.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.508+0000 7f85320a4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f852c09e410 0x7f852c122c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:18.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.508+0000 7f85320a4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c124410 con 0x7f852c09e410 2023-10-03T10:16:18.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.508+0000 7f85320a4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852c123ed0 con 0x7f852c09e000 2023-10-03T10:16:18.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.508+0000 7f85320a4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f852c1259f0 con 0x7f852c09d6f0 2023-10-03T10:16:18.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.508+0000 7f852bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f852c09e410 0x7f852c122c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:18.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.508+0000 7f852affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f852c09e000 0x7f852c1226f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:18.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.508+0000 7f852bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f852c09e410 0x7f852c122c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49834/0 (socket says 172.21.15.3:49834) 2023-10-03T10:16:18.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.508+0000 7f852affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f852c09e000 0x7f852c1226f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44330/0 (socket says 172.21.15.3:44330) 2023-10-03T10:16:18.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.508+0000 7f852bfff700 1 -- 172.21.15.3:0/4206465823 learned_addr learned my addr 172.21.15.3:0/4206465823 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:18.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.508+0000 7f852b7fe700 1 --2- 172.21.15.3:0/4206465823 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f852c09d6f0 0x7f852c1221b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:18.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.509+0000 7f852bfff700 1 -- 172.21.15.3:0/4206465823 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f852c09d6f0 msgr2=0x7f852c1221b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:18.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.509+0000 7f852bfff700 1 --2- 172.21.15.3:0/4206465823 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f852c09d6f0 0x7f852c1221b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.509+0000 7f852bfff700 1 -- 172.21.15.3:0/4206465823 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f852c09e000 msgr2=0x7f852c1226f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:18.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.509+0000 7f852bfff700 1 --2- 172.21.15.3:0/4206465823 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f852c09e000 0x7f852c1226f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.509+0000 7f852bfff700 1 -- 172.21.15.3:0/4206465823 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851c009550 con 0x7f852c09e410 2023-10-03T10:16:18.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.509+0000 7f852affd700 1 --2- 172.21.15.3:0/4206465823 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f852c09e000 0x7f852c1226f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:18.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.509+0000 7f852bfff700 1 --2- 172.21.15.3:0/4206465823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f852c09e410 0x7f852c122c30 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f852400cef0 tx=0x7f852400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:18.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.509+0000 7f8513fff700 1 -- 172.21.15.3:0/4206465823 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852400ad00 con 0x7f852c09e410 2023-10-03T10:16:18.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.509+0000 7f8513fff700 1 -- 172.21.15.3:0/4206465823 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852400ae60 con 0x7f852c09e410 2023-10-03T10:16:18.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.509+0000 7f85320a4700 1 -- 172.21.15.3:0/4206465823 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f852c072150 con 0x7f852c09e410 2023-10-03T10:16:18.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.509+0000 7f8513fff700 1 -- 172.21.15.3:0/4206465823 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8524015680 con 0x7f852c09e410 2023-10-03T10:16:18.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.509+0000 7f85320a4700 1 -- 172.21.15.3:0/4206465823 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f852c126460 con 0x7f852c09e410 2023-10-03T10:16:18.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.511+0000 7f85320a4700 1 -- 172.21.15.3:0/4206465823 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f852c074ea0 con 0x7f852c09e410 2023-10-03T10:16:18.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.511+0000 7f8513fff700 1 -- 172.21.15.3:0/4206465823 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f852400d430 con 0x7f852c09e410 2023-10-03T10:16:18.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.511+0000 7f8513fff700 1 -- 172.21.15.3:0/4206465823 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f852403fbb0 con 0x7f852c09e410 2023-10-03T10:16:18.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.517+0000 7f8513fff700 1 -- 172.21.15.3:0/4206465823 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8524008910 con 0x7f852c09e410 2023-10-03T10:16:18.553 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:18.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.599+0000 7f9563128700 1 Processor -- start 2023-10-03T10:16:18.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.600+0000 7f9563128700 1 -- start start 2023-10-03T10:16:18.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.600+0000 7f9563128700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f955c09e930 0x7f955c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:18.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.600+0000 7f9563128700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f955c09f650 0x7f955c131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:18.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.600+0000 7f9563128700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955c070940 con 0x7f955c09f240 2023-10-03T10:16:18.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.601+0000 7f9563128700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955c06fd60 con 0x7f955c09f650 2023-10-03T10:16:18.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.601+0000 7f9563128700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f955c06fa80 con 0x7f955c09e930 2023-10-03T10:16:18.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.601+0000 7f9560ec4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f955c09e930 0x7f955c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:18.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.601+0000 7f95616c5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f955c09f650 0x7f955c131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:18.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.601+0000 7f9560ec4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f955c09e930 0x7f955c09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:33158/0 (socket says 172.21.15.3:33158) 2023-10-03T10:16:18.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.601+0000 7f9560ec4700 1 -- 172.21.15.3:0/742594981 learned_addr learned my addr 172.21.15.3:0/742594981 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:18.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.601+0000 7f95616c5700 1 -- 172.21.15.3:0/742594981 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f955c09e930 msgr2=0x7f955c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:18.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.601+0000 7f95616c5700 1 --2- 172.21.15.3:0/742594981 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f955c09e930 0x7f955c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.601+0000 7f95616c5700 1 -- 172.21.15.3:0/742594981 >> v1:172.21.15.3:6789/0 conn(0x7f955c09f240 legacy=0x7f955c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:18.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.601+0000 7f95616c5700 1 -- 172.21.15.3:0/742594981 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955c09fb60 con 0x7f955c09f650 2023-10-03T10:16:18.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.601+0000 7f9560ec4700 1 --2- 172.21.15.3:0/742594981 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f955c09e930 0x7f955c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:18.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.601+0000 7f95616c5700 1 --2- 172.21.15.3:0/742594981 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f955c09f650 0x7f955c131f00 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9554009dc0 tx=0x7f955400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd3bf1465fcea7c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:18.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.602+0000 7f955b7fe700 1 -- 172.21.15.3:0/742594981 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955400e070 con 0x7f955c09f650 2023-10-03T10:16:18.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.602+0000 7f955b7fe700 1 -- 172.21.15.3:0/742594981 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955400bae0 con 0x7f955c09f650 2023-10-03T10:16:18.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.602+0000 7f955b7fe700 1 -- 172.21.15.3:0/742594981 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955400be90 con 0x7f955c09f650 2023-10-03T10:16:18.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.603+0000 7f9563128700 1 -- 172.21.15.3:0/742594981 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f955c09f650 msgr2=0x7f955c131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:18.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.603+0000 7f9563128700 1 --2- 172.21.15.3:0/742594981 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f955c09f650 0x7f955c131f00 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9554009dc0 tx=0x7f955400b330 comp rx=0 tx=0).stop 2023-10-03T10:16:18.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.603+0000 7f9563128700 1 -- 172.21.15.3:0/742594981 shutdown_connections 2023-10-03T10:16:18.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.603+0000 7f9563128700 1 --2- 172.21.15.3:0/742594981 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f955c09f650 0x7f955c131f00 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.603+0000 7f9563128700 1 --2- 172.21.15.3:0/742594981 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f955c09e930 0x7f955c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.603+0000 7f9563128700 1 -- 172.21.15.3:0/742594981 >> 172.21.15.3:0/742594981 conn(0x7f955c0a4210 msgr2=0x7f955c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:18.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.603+0000 7f9563128700 1 -- 172.21.15.3:0/742594981 shutdown_connections 2023-10-03T10:16:18.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.603+0000 7f9563128700 1 -- 172.21.15.3:0/742594981 wait complete. 2023-10-03T10:16:18.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.604+0000 7f9563128700 1 Processor -- start 2023-10-03T10:16:18.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.604+0000 7f9563128700 1 -- start start 2023-10-03T10:16:18.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.605+0000 7f9563128700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f955c09e930 0x7f955c06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:18.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.605+0000 7f9563128700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f955c09f240 0x7f955c06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:18.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.605+0000 7f9563128700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f955c09f650 0x7f955c06c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:18.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.605+0000 7f9563128700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c1b7280 con 0x7f955c09f650 2023-10-03T10:16:18.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.605+0000 7f9563128700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955c1a9430 con 0x7f955c09e930 2023-10-03T10:16:18.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.605+0000 7f9563128700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f955c1a8950 con 0x7f955c09f240 2023-10-03T10:16:18.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.606+0000 7f95616c5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f955c09f650 0x7f955c06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:18.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.606+0000 7f95616c5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f955c09f650 0x7f955c06c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49848/0 (socket says 172.21.15.3:49848) 2023-10-03T10:16:18.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.606+0000 7f95616c5700 1 -- 172.21.15.3:0/741499577 learned_addr learned my addr 172.21.15.3:0/741499577 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:18.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.606+0000 7f9560ec4700 1 --2- 172.21.15.3:0/741499577 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f955c09e930 0x7f955c06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:18.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.606+0000 7f955bfff700 1 --2- 172.21.15.3:0/741499577 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f955c09f240 0x7f955c06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:18.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.606+0000 7f9560ec4700 1 -- 172.21.15.3:0/741499577 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f955c09f240 msgr2=0x7f955c06bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:18.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.606+0000 7f9560ec4700 1 --2- 172.21.15.3:0/741499577 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f955c09f240 0x7f955c06bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.606+0000 7f9560ec4700 1 -- 172.21.15.3:0/741499577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f955c09f650 msgr2=0x7f955c06c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:18.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.606+0000 7f9560ec4700 1 --2- 172.21.15.3:0/741499577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f955c09f650 0x7f955c06c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.606+0000 7f9560ec4700 1 -- 172.21.15.3:0/741499577 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9550009550 con 0x7f955c09e930 2023-10-03T10:16:18.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.606+0000 7f95616c5700 1 --2- 172.21.15.3:0/741499577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f955c09f650 0x7f955c06c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:18.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.606+0000 7f9560ec4700 1 --2- 172.21.15.3:0/741499577 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f955c09e930 0x7f955c06b8b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f955000e9d0 tx=0x7f955000ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:18.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.607+0000 7f95597fa700 1 -- 172.21.15.3:0/741499577 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9550004030 con 0x7f955c09e930 2023-10-03T10:16:18.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.607+0000 7f95597fa700 1 -- 172.21.15.3:0/741499577 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9550004190 con 0x7f955c09e930 2023-10-03T10:16:18.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.607+0000 7f9563128700 1 -- 172.21.15.3:0/741499577 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9554009a70 con 0x7f955c09e930 2023-10-03T10:16:18.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.607+0000 7f95597fa700 1 -- 172.21.15.3:0/741499577 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9550004520 con 0x7f955c09e930 2023-10-03T10:16:18.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.607+0000 7f9563128700 1 -- 172.21.15.3:0/741499577 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f955c0a0190 con 0x7f955c09e930 2023-10-03T10:16:18.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.608+0000 7f95597fa700 1 -- 172.21.15.3:0/741499577 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f955000c850 con 0x7f955c09e930 2023-10-03T10:16:18.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.608+0000 7f95597fa700 1 -- 172.21.15.3:0/741499577 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f955003f910 con 0x7f955c09e930 2023-10-03T10:16:18.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.608+0000 7f9563128700 1 -- 172.21.15.3:0/741499577 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f955c1a8d10 con 0x7f955c09e930 2023-10-03T10:16:18.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.614+0000 7f95597fa700 1 -- 172.21.15.3:0/741499577 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f95500036a0 con 0x7f955c09e930 2023-10-03T10:16:18.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.638+0000 7ff896ad4700 1 Processor -- start 2023-10-03T10:16:18.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.638+0000 7ff896ad4700 1 -- start start 2023-10-03T10:16:18.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.639+0000 7ff896ad4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff89009e930 0x7ff89009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:18.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.639+0000 7ff896ad4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff89009f240 0x7ff8901317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:18.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.639+0000 7ff896ad4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff890070940 con 0x7ff89009f650 2023-10-03T10:16:18.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.639+0000 7ff896ad4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff89006fd60 con 0x7ff89009f240 2023-10-03T10:16:18.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.639+0000 7ff896ad4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff89006fa80 con 0x7ff89009e930 2023-10-03T10:16:18.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.639+0000 7ff894870700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff89009e930 0x7ff89009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:18.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.639+0000 7ff894870700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff89009e930 0x7ff89009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44352/0 (socket says 172.21.15.3:44352) 2023-10-03T10:16:18.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.639+0000 7ff894870700 1 -- 172.21.15.3:0/3013408674 learned_addr learned my addr 172.21.15.3:0/3013408674 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:18.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.639+0000 7ff88ffff700 1 --2- 172.21.15.3:0/3013408674 >> v2:172.21.15.3:3300/0 conn(0x7ff89009f240 0x7ff8901317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:18.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.640+0000 7ff88ffff700 1 -- 172.21.15.3:0/3013408674 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff89009e930 msgr2=0x7ff89009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:18.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.640+0000 7ff88ffff700 1 --2- 172.21.15.3:0/3013408674 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff89009e930 0x7ff89009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.640+0000 7ff88ffff700 1 -- 172.21.15.3:0/3013408674 >> v1:172.21.15.3:6789/0 conn(0x7ff89009f650 legacy=0x7ff890131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:18.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.640+0000 7ff88ffff700 1 -- 172.21.15.3:0/3013408674 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff89009fb60 con 0x7ff89009f240 2023-10-03T10:16:18.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.640+0000 7ff88f7fe700 1 -- 172.21.15.3:0/3013408674 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 78535716 0 0) 0x7ff890070940 con 0x7ff89009f650 2023-10-03T10:16:18.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.640+0000 7ff894870700 1 --2- 172.21.15.3:0/3013408674 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff89009e930 0x7ff89009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:18.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.640+0000 7ff88ffff700 1 --2- 172.21.15.3:0/3013408674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff89009f240 0x7ff8901317f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7ff87c009840 tx=0x7ff87c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f0de0845404721 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:18.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.640+0000 7ff88f7fe700 1 -- 172.21.15.3:0/3013408674 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c011830 con 0x7ff89009f240 2023-10-03T10:16:18.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.640+0000 7ff88f7fe700 1 -- 172.21.15.3:0/3013408674 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87c011990 con 0x7ff89009f240 2023-10-03T10:16:18.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.640+0000 7ff88f7fe700 1 -- 172.21.15.3:0/3013408674 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c011d20 con 0x7ff89009f240 2023-10-03T10:16:18.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.641+0000 7ff896ad4700 1 -- 172.21.15.3:0/3013408674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff89009f240 msgr2=0x7ff8901317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:18.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.641+0000 7ff896ad4700 1 --2- 172.21.15.3:0/3013408674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff89009f240 0x7ff8901317f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7ff87c009840 tx=0x7ff87c009b50 comp rx=0 tx=0).stop 2023-10-03T10:16:18.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.641+0000 7ff896ad4700 1 -- 172.21.15.3:0/3013408674 shutdown_connections 2023-10-03T10:16:18.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.641+0000 7ff896ad4700 1 --2- 172.21.15.3:0/3013408674 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff89009e930 0x7ff89009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.641+0000 7ff896ad4700 1 --2- 172.21.15.3:0/3013408674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff89009f240 0x7ff8901317f0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.641+0000 7ff896ad4700 1 -- 172.21.15.3:0/3013408674 >> 172.21.15.3:0/3013408674 conn(0x7ff8900a4210 msgr2=0x7ff8900a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:18.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.642+0000 7ff896ad4700 1 -- 172.21.15.3:0/3013408674 shutdown_connections 2023-10-03T10:16:18.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.642+0000 7ff896ad4700 1 -- 172.21.15.3:0/3013408674 wait complete. 2023-10-03T10:16:18.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.643+0000 7ff896ad4700 1 Processor -- start 2023-10-03T10:16:18.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.643+0000 7ff896ad4700 1 -- start start 2023-10-03T10:16:18.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.644+0000 7ff896ad4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff89009e930 0x7ff8901a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:18.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.644+0000 7ff896ad4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff89009f240 0x7ff8901a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:18.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.644+0000 7ff896ad4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff89009f650 0x7ff8901a48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:18.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.644+0000 7ff896ad4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8901a76a0 con 0x7ff89009e930 2023-10-03T10:16:18.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.644+0000 7ff896ad4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff890071d20 con 0x7ff89009f650 2023-10-03T10:16:18.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.644+0000 7ff896ad4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8901a59c0 con 0x7ff89009f240 2023-10-03T10:16:18.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.644+0000 7ff894870700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff89009e930 0x7ff8901a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:18.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.644+0000 7ff894870700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff89009e930 0x7ff8901a3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49866/0 (socket says 172.21.15.3:49866) 2023-10-03T10:16:18.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.644+0000 7ff894870700 1 -- 172.21.15.3:0/1002927759 learned_addr learned my addr 172.21.15.3:0/1002927759 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:18.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.644+0000 7ff895071700 1 --2- 172.21.15.3:0/1002927759 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff89009f650 0x7ff8901a48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:18.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.645+0000 7ff88ffff700 1 --2- 172.21.15.3:0/1002927759 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff89009f240 0x7ff8901a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:18.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.645+0000 7ff894870700 1 -- 172.21.15.3:0/1002927759 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff89009f240 msgr2=0x7ff8901a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:18.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.645+0000 7ff894870700 1 --2- 172.21.15.3:0/1002927759 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff89009f240 0x7ff8901a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.645+0000 7ff894870700 1 -- 172.21.15.3:0/1002927759 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff89009f650 msgr2=0x7ff8901a48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:18.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.645+0000 7ff894870700 1 --2- 172.21.15.3:0/1002927759 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff89009f650 0x7ff8901a48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.645+0000 7ff894870700 1 -- 172.21.15.3:0/1002927759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff884009550 con 0x7ff89009e930 2023-10-03T10:16:18.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.645+0000 7ff894870700 1 --2- 172.21.15.3:0/1002927759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff89009e930 0x7ff8901a3e50 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7ff88400ebe0 tx=0x7ff88400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:18.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.645+0000 7ff88d7fa700 1 -- 172.21.15.3:0/1002927759 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88400c850 con 0x7ff89009e930 2023-10-03T10:16:18.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.645+0000 7ff88d7fa700 1 -- 172.21.15.3:0/1002927759 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88400c9b0 con 0x7ff89009e930 2023-10-03T10:16:18.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.646+0000 7ff896ad4700 1 -- 172.21.15.3:0/1002927759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff87c0094a0 con 0x7ff89009e930 2023-10-03T10:16:18.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.646+0000 7ff88d7fa700 1 -- 172.21.15.3:0/1002927759 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88400cd40 con 0x7ff89009e930 2023-10-03T10:16:18.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.646+0000 7ff896ad4700 1 -- 172.21.15.3:0/1002927759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8900a0190 con 0x7ff89009e930 2023-10-03T10:16:18.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.647+0000 7ff88d7fa700 1 -- 172.21.15.3:0/1002927759 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff884013470 con 0x7ff89009e930 2023-10-03T10:16:18.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.647+0000 7ff88d7fa700 1 -- 172.21.15.3:0/1002927759 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff88403f890 con 0x7ff89009e930 2023-10-03T10:16:18.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.647+0000 7ff896ad4700 1 -- 172.21.15.3:0/1002927759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8901a0b30 con 0x7ff89009e930 2023-10-03T10:16:18.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.653+0000 7ff88d7fa700 1 -- 172.21.15.3:0/1002927759 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff8840232f0 con 0x7ff89009e930 2023-10-03T10:16:18.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.707+0000 7f4cc224a700 1 Processor -- start 2023-10-03T10:16:18.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.707+0000 7f4cc224a700 1 -- start start 2023-10-03T10:16:18.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.707+0000 7f4cc224a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cbc09d6f0 0x7f4cbc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:18.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.708+0000 7f4cc224a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cbc09e410 0x7f4cbc130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:18.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.708+0000 7f4cc224a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cbc09f130 con 0x7f4cbc09e000 2023-10-03T10:16:18.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.708+0000 7f4cc224a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc09f520 con 0x7f4cbc09e410 2023-10-03T10:16:18.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.708+0000 7f4cc224a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc09e930 con 0x7f4cbc09d6f0 2023-10-03T10:16:18.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.708+0000 7f4cbaffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4cbc09e000 0x7f4cbc130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:41336/0 (socket says 172.21.15.3:41336) 2023-10-03T10:16:18.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.708+0000 7f4cbaffd700 1 -- 172.21.15.3:0/517381285 learned_addr learned my addr 172.21.15.3:0/517381285 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:18.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.708+0000 7f4cbbfff700 1 --2- 172.21.15.3:0/517381285 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cbc09e410 0x7f4cbc130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:18.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.708+0000 7f4cbb7fe700 1 --2- 172.21.15.3:0/517381285 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cbc09d6f0 0x7f4cbc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:18.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.708+0000 7f4cbbfff700 1 -- 172.21.15.3:0/517381285 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cbc09d6f0 msgr2=0x7f4cbc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:18.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.708+0000 7f4cbbfff700 1 --2- 172.21.15.3:0/517381285 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cbc09d6f0 0x7f4cbc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.708+0000 7f4cba7fc700 1 -- 172.21.15.3:0/517381285 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 797132187 0 0) 0x7f4cbc09f130 con 0x7f4cbc09e000 2023-10-03T10:16:18.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.708+0000 7f4cbbfff700 1 -- 172.21.15.3:0/517381285 >> v1:172.21.15.3:6789/0 conn(0x7f4cbc09e000 legacy=0x7f4cbc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:18.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.708+0000 7f4cbbfff700 1 -- 172.21.15.3:0/517381285 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cbc09b2d0 con 0x7f4cbc09e410 2023-10-03T10:16:18.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.709+0000 7f4cbbfff700 1 --2- 172.21.15.3:0/517381285 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cbc09e410 0x7f4cbc130d70 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400bfd0 tx=0x7f4cb400d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97868f8528593410 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:18.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.709+0000 7f4cba7fc700 1 -- 172.21.15.3:0/517381285 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4010070 con 0x7f4cbc09e410 2023-10-03T10:16:18.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.709+0000 7f4cba7fc700 1 -- 172.21.15.3:0/517381285 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb400b2e0 con 0x7f4cbc09e410 2023-10-03T10:16:18.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.709+0000 7f4cba7fc700 1 -- 172.21.15.3:0/517381285 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb400dd20 con 0x7f4cbc09e410 2023-10-03T10:16:18.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.709+0000 7f4cc224a700 1 -- 172.21.15.3:0/517381285 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cbc09e410 msgr2=0x7f4cbc130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:18.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.709+0000 7f4cc224a700 1 --2- 172.21.15.3:0/517381285 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cbc09e410 0x7f4cbc130d70 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400bfd0 tx=0x7f4cb400d330 comp rx=0 tx=0).stop 2023-10-03T10:16:18.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.710+0000 7f4cc224a700 1 -- 172.21.15.3:0/517381285 shutdown_connections 2023-10-03T10:16:18.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.710+0000 7f4cc224a700 1 --2- 172.21.15.3:0/517381285 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cbc09e410 0x7f4cbc130d70 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.710+0000 7f4cc224a700 1 --2- 172.21.15.3:0/517381285 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cbc09d6f0 0x7f4cbc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.710+0000 7f4cc224a700 1 -- 172.21.15.3:0/517381285 >> 172.21.15.3:0/517381285 conn(0x7f4cbc066f50 msgr2=0x7f4cbc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:18.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.710+0000 7f4cc224a700 1 -- 172.21.15.3:0/517381285 shutdown_connections 2023-10-03T10:16:18.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.710+0000 7f4cc224a700 1 -- 172.21.15.3:0/517381285 wait complete. 2023-10-03T10:16:18.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.711+0000 7f4cc224a700 1 Processor -- start 2023-10-03T10:16:18.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.711+0000 7f4cc224a700 1 -- start start 2023-10-03T10:16:18.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.711+0000 7f4cc224a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cbc09d6f0 0x7f4cbc132970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:18.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.711+0000 7f4cc224a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cbc09e000 0x7f4cbc132eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:18.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.711+0000 7f4cc224a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cbc09e410 0x7f4cbc1c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:18.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.711+0000 7f4cc224a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc135ab0 con 0x7f4cbc09d6f0 2023-10-03T10:16:18.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.711+0000 7f4cc224a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc1322e0 con 0x7f4cbc09e410 2023-10-03T10:16:18.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.711+0000 7f4cc224a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc09c700 con 0x7f4cbc09e000 2023-10-03T10:16:18.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.712+0000 7f4cbb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cbc09d6f0 0x7f4cbc132970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:18.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.712+0000 7f4cbb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cbc09d6f0 0x7f4cbc132970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49880/0 (socket says 172.21.15.3:49880) 2023-10-03T10:16:18.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.712+0000 7f4cbb7fe700 1 -- 172.21.15.3:0/3709211344 learned_addr learned my addr 172.21.15.3:0/3709211344 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:18.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.712+0000 7f4cbbfff700 1 --2- 172.21.15.3:0/3709211344 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cbc09e410 0x7f4cbc1c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:18.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.712+0000 7f4cbaffd700 1 --2- 172.21.15.3:0/3709211344 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cbc09e000 0x7f4cbc132eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:18.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.712+0000 7f4cbbfff700 1 -- 172.21.15.3:0/3709211344 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cbc09e000 msgr2=0x7f4cbc132eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:18.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.712+0000 7f4cbbfff700 1 --2- 172.21.15.3:0/3709211344 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cbc09e000 0x7f4cbc132eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.712+0000 7f4cbbfff700 1 -- 172.21.15.3:0/3709211344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cbc09d6f0 msgr2=0x7f4cbc132970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:18.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.712+0000 7f4cbbfff700 1 --2- 172.21.15.3:0/3709211344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cbc09d6f0 0x7f4cbc132970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.712+0000 7f4cbbfff700 1 -- 172.21.15.3:0/3709211344 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb400bc80 con 0x7f4cbc09e410 2023-10-03T10:16:18.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.712+0000 7f4cbb7fe700 1 --2- 172.21.15.3:0/3709211344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cbc09d6f0 0x7f4cbc132970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:18.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.712+0000 7f4cbbfff700 1 --2- 172.21.15.3:0/3709211344 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cbc09e410 0x7f4cbc1c7f80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc0a3e50 tx=0x7f4cb4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:18.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.713+0000 7f4ca3fff700 1 -- 172.21.15.3:0/3709211344 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4009300 con 0x7f4cbc09e410 2023-10-03T10:16:18.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.713+0000 7f4ca3fff700 1 -- 172.21.15.3:0/3709211344 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb4004250 con 0x7f4cbc09e410 2023-10-03T10:16:18.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.713+0000 7f4cc224a700 1 -- 172.21.15.3:0/3709211344 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cbc132520 con 0x7f4cbc09e410 2023-10-03T10:16:18.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.713+0000 7f4ca3fff700 1 -- 172.21.15.3:0/3709211344 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4004600 con 0x7f4cbc09e410 2023-10-03T10:16:18.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.713+0000 7f4cc224a700 1 -- 172.21.15.3:0/3709211344 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cbc136190 con 0x7f4cbc09e410 2023-10-03T10:16:18.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.714+0000 7f4cc224a700 1 -- 172.21.15.3:0/3709211344 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cbc135580 con 0x7f4cbc09e410 2023-10-03T10:16:18.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.714+0000 7f4ca3fff700 1 -- 172.21.15.3:0/3709211344 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4cb4009050 con 0x7f4cbc09e410 2023-10-03T10:16:18.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.714+0000 7f4ca3fff700 1 -- 172.21.15.3:0/3709211344 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4cb4040d60 con 0x7f4cbc09e410 2023-10-03T10:16:18.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.719+0000 7f4ca3fff700 1 -- 172.21.15.3:0/3709211344 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4cb401cb40 con 0x7f4cbc09e410 2023-10-03T10:16:18.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.754+0000 7f85320a4700 1 -- 172.21.15.3:0/4206465823 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f852c1300d0 con 0x7f852c09e410 2023-10-03T10:16:18.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.754+0000 7f8513fff700 1 -- 172.21.15.3:0/4206465823 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f852401a070 con 0x7f852c09e410 2023-10-03T10:16:18.755 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:18.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.757+0000 7f85320a4700 1 -- 172.21.15.3:0/4206465823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f852c09e410 msgr2=0x7f852c122c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:18.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.757+0000 7f85320a4700 1 --2- 172.21.15.3:0/4206465823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f852c09e410 0x7f852c122c30 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f852400cef0 tx=0x7f852400cf20 comp rx=0 tx=0).stop 2023-10-03T10:16:18.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.757+0000 7f85320a4700 1 -- 172.21.15.3:0/4206465823 shutdown_connections 2023-10-03T10:16:18.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.757+0000 7f85320a4700 1 --2- 172.21.15.3:0/4206465823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f852c09e410 0x7f852c122c30 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.757+0000 7f85320a4700 1 --2- 172.21.15.3:0/4206465823 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f852c09d6f0 0x7f852c1221b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.757+0000 7f85320a4700 1 --2- 172.21.15.3:0/4206465823 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f852c09e000 0x7f852c1226f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.757+0000 7f85320a4700 1 -- 172.21.15.3:0/4206465823 >> 172.21.15.3:0/4206465823 conn(0x7f852c066f50 msgr2=0x7f852c130b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:18.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.758+0000 7f85320a4700 1 -- 172.21.15.3:0/4206465823 shutdown_connections 2023-10-03T10:16:18.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.758+0000 7f85320a4700 1 -- 172.21.15.3:0/4206465823 wait complete. 2023-10-03T10:16:18.777 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:18.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.853+0000 7f9563128700 1 -- 172.21.15.3:0/741499577 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f955c0a3660 con 0x7f955c09e930 2023-10-03T10:16:18.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.853+0000 7f95597fa700 1 -- 172.21.15.3:0/741499577 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9550013e60 con 0x7f955c09e930 2023-10-03T10:16:18.853 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:18.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.856+0000 7f9563128700 1 -- 172.21.15.3:0/741499577 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f955c09e930 msgr2=0x7f955c06b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:18.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.856+0000 7f9563128700 1 --2- 172.21.15.3:0/741499577 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f955c09e930 0x7f955c06b8b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f955000e9d0 tx=0x7f955000ece0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.856+0000 7f9563128700 1 -- 172.21.15.3:0/741499577 shutdown_connections 2023-10-03T10:16:18.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.856+0000 7f9563128700 1 --2- 172.21.15.3:0/741499577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f955c09f650 0x7f955c06c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.856+0000 7f9563128700 1 --2- 172.21.15.3:0/741499577 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f955c09e930 0x7f955c06b8b0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.856+0000 7f9563128700 1 --2- 172.21.15.3:0/741499577 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f955c09f240 0x7f955c06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.856+0000 7f9563128700 1 -- 172.21.15.3:0/741499577 >> 172.21.15.3:0/741499577 conn(0x7f955c0a4210 msgr2=0x7f955c1b8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:18.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.856+0000 7f9563128700 1 -- 172.21.15.3:0/741499577 shutdown_connections 2023-10-03T10:16:18.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.856+0000 7f9563128700 1 -- 172.21.15.3:0/741499577 wait complete. 2023-10-03T10:16:18.870 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:18.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.883+0000 7ff896ad4700 1 -- 172.21.15.3:0/1002927759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff890074ea0 con 0x7ff89009e930 2023-10-03T10:16:18.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.883+0000 7ff88d7fa700 1 -- 172.21.15.3:0/1002927759 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff884017020 con 0x7ff89009e930 2023-10-03T10:16:18.884 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:18.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.886+0000 7ff896ad4700 1 -- 172.21.15.3:0/1002927759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff89009e930 msgr2=0x7ff8901a3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:18.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.886+0000 7ff896ad4700 1 --2- 172.21.15.3:0/1002927759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff89009e930 0x7ff8901a3e50 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7ff88400ebe0 tx=0x7ff88400eef0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.886+0000 7ff896ad4700 1 -- 172.21.15.3:0/1002927759 shutdown_connections 2023-10-03T10:16:18.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.886+0000 7ff896ad4700 1 --2- 172.21.15.3:0/1002927759 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff89009f650 0x7ff8901a48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.886+0000 7ff896ad4700 1 --2- 172.21.15.3:0/1002927759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff89009e930 0x7ff8901a3e50 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.886+0000 7ff896ad4700 1 --2- 172.21.15.3:0/1002927759 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff89009f240 0x7ff8901a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.886+0000 7ff896ad4700 1 -- 172.21.15.3:0/1002927759 >> 172.21.15.3:0/1002927759 conn(0x7ff8900a4210 msgr2=0x7ff8901b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:18.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.886+0000 7ff896ad4700 1 -- 172.21.15.3:0/1002927759 shutdown_connections 2023-10-03T10:16:18.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.886+0000 7ff896ad4700 1 -- 172.21.15.3:0/1002927759 wait complete. 2023-10-03T10:16:18.899 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:18.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.938+0000 7f4cc224a700 1 -- 172.21.15.3:0/3709211344 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4cbc074ea0 con 0x7f4cbc09e410 2023-10-03T10:16:18.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.938+0000 7f4ca3fff700 1 -- 172.21.15.3:0/3709211344 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4cb401cb40 con 0x7f4cbc09e410 2023-10-03T10:16:18.939 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:18.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.941+0000 7f4cc224a700 1 -- 172.21.15.3:0/3709211344 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cbc09e410 msgr2=0x7f4cbc1c7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:18.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.941+0000 7f4cc224a700 1 --2- 172.21.15.3:0/3709211344 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cbc09e410 0x7f4cbc1c7f80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc0a3e50 tx=0x7f4cb4004ad0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.941+0000 7f4cc224a700 1 -- 172.21.15.3:0/3709211344 shutdown_connections 2023-10-03T10:16:18.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.941+0000 7f4cc224a700 1 --2- 172.21.15.3:0/3709211344 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cbc09e410 0x7f4cbc1c7f80 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.941+0000 7f4cc224a700 1 --2- 172.21.15.3:0/3709211344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cbc09d6f0 0x7f4cbc132970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.941+0000 7f4cc224a700 1 --2- 172.21.15.3:0/3709211344 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cbc09e000 0x7f4cbc132eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:18.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.941+0000 7f4cc224a700 1 -- 172.21.15.3:0/3709211344 >> 172.21.15.3:0/3709211344 conn(0x7f4cbc066f50 msgr2=0x7f4cbc070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:18.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.941+0000 7f4cc224a700 1 -- 172.21.15.3:0/3709211344 shutdown_connections 2023-10-03T10:16:18.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:18.941+0000 7f4cc224a700 1 -- 172.21.15.3:0/3709211344 wait complete. 2023-10-03T10:16:18.955 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:19.778 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:19.871 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:19.900 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:19.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.926+0000 7face4fdb700 1 Processor -- start 2023-10-03T10:16:19.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.926+0000 7face4fdb700 1 -- start start 2023-10-03T10:16:19.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.927+0000 7face4fdb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7face009e1d0 0x7face0138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:19.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.927+0000 7face4fdb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7face0139170 0x7face013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:19.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.927+0000 7face4fdb700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face009b120 con 0x7face009d6f0 2023-10-03T10:16:19.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.927+0000 7face4fdb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7face006b460 con 0x7face009e1d0 2023-10-03T10:16:19.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.927+0000 7face4fdb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face009a7e0 con 0x7face0139170 2023-10-03T10:16:19.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.927+0000 7facddd9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7face009e1d0 0x7face0138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:19.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.927+0000 7facddd9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7face009e1d0 0x7face0138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49882/0 (socket says 172.21.15.3:49882) 2023-10-03T10:16:19.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.927+0000 7facded9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7face0139170 0x7face013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:19.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.927+0000 7facddd9b700 1 -- 172.21.15.3:0/3376719575 learned_addr learned my addr 172.21.15.3:0/3376719575 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:19.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.928+0000 7facded9d700 1 -- 172.21.15.3:0/3376719575 >> v2:172.21.15.3:3300/0 conn(0x7face009e1d0 msgr2=0x7face0138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:19.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.928+0000 7facded9d700 1 --2- 172.21.15.3:0/3376719575 >> v2:172.21.15.3:3300/0 conn(0x7face009e1d0 0x7face0138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:19.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.928+0000 7facded9d700 1 -- 172.21.15.3:0/3376719575 >> v1:172.21.15.3:6789/0 conn(0x7face009d6f0 legacy=0x7face009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:19.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.928+0000 7facded9d700 1 -- 172.21.15.3:0/3376719575 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face009a500 con 0x7face0139170 2023-10-03T10:16:19.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.928+0000 7facddd9b700 1 --2- 172.21.15.3:0/3376719575 >> v2:172.21.15.3:3300/0 conn(0x7face009e1d0 0x7face0138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:19.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.928+0000 7facded9d700 1 --2- 172.21.15.3:0/3376719575 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7face0139170 0x7face013b550 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7facd8009dc0 tx=0x7facd800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f57436536b815f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:19.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.928+0000 7facdd59a700 1 -- 172.21.15.3:0/3376719575 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd800e070 con 0x7face0139170 2023-10-03T10:16:19.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.928+0000 7facdd59a700 1 -- 172.21.15.3:0/3376719575 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd800bae0 con 0x7face0139170 2023-10-03T10:16:19.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.928+0000 7facdd59a700 1 -- 172.21.15.3:0/3376719575 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd800be90 con 0x7face0139170 2023-10-03T10:16:19.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.929+0000 7face4fdb700 1 -- 172.21.15.3:0/3376719575 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7face0139170 msgr2=0x7face013b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:19.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.929+0000 7face4fdb700 1 --2- 172.21.15.3:0/3376719575 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7face0139170 0x7face013b550 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7facd8009dc0 tx=0x7facd800b330 comp rx=0 tx=0).stop 2023-10-03T10:16:19.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.929+0000 7face4fdb700 1 -- 172.21.15.3:0/3376719575 shutdown_connections 2023-10-03T10:16:19.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.929+0000 7face4fdb700 1 --2- 172.21.15.3:0/3376719575 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7face0139170 0x7face013b550 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:19.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.929+0000 7face4fdb700 1 --2- 172.21.15.3:0/3376719575 >> v2:172.21.15.3:3300/0 conn(0x7face009e1d0 0x7face0138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:19.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.929+0000 7face4fdb700 1 -- 172.21.15.3:0/3376719575 >> 172.21.15.3:0/3376719575 conn(0x7face0066f50 msgr2=0x7face009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:19.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.929+0000 7face4fdb700 1 -- 172.21.15.3:0/3376719575 shutdown_connections 2023-10-03T10:16:19.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.929+0000 7face4fdb700 1 -- 172.21.15.3:0/3376719575 wait complete. 2023-10-03T10:16:19.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.931+0000 7face4fdb700 1 Processor -- start 2023-10-03T10:16:19.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.931+0000 7face4fdb700 1 -- start start 2023-10-03T10:16:19.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.932+0000 7face4fdb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7face009d6f0 0x7face01c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:19.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.932+0000 7face4fdb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face009e1d0 0x7face01c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:19.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.932+0000 7face4fdb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7face0139170 0x7face011e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:19.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.932+0000 7face4fdb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face0122110 con 0x7face009e1d0 2023-10-03T10:16:19.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.932+0000 7face4fdb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face0120400 con 0x7face009d6f0 2023-10-03T10:16:19.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.932+0000 7face4fdb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7face0121d70 con 0x7face0139170 2023-10-03T10:16:19.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.932+0000 7facde59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7face009d6f0 0x7face01c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:19.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.932+0000 7facde59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7face009d6f0 0x7face01c84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44378/0 (socket says 172.21.15.3:44378) 2023-10-03T10:16:19.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.932+0000 7facde59c700 1 -- 172.21.15.3:0/2926777121 learned_addr learned my addr 172.21.15.3:0/2926777121 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:19.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.932+0000 7facddd9b700 1 --2- 172.21.15.3:0/2926777121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face009e1d0 0x7face01c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:19.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.932+0000 7facded9d700 1 --2- 172.21.15.3:0/2926777121 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7face0139170 0x7face011e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:19.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.932+0000 7facde59c700 1 -- 172.21.15.3:0/2926777121 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7face0139170 msgr2=0x7face011e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:19.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.932+0000 7facde59c700 1 --2- 172.21.15.3:0/2926777121 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7face0139170 0x7face011e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:19.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.932+0000 7facde59c700 1 -- 172.21.15.3:0/2926777121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face009e1d0 msgr2=0x7face01c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:19.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.932+0000 7facde59c700 1 --2- 172.21.15.3:0/2926777121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face009e1d0 0x7face01c8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:19.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.933+0000 7facde59c700 1 -- 172.21.15.3:0/2926777121 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd4009550 con 0x7face009d6f0 2023-10-03T10:16:19.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.933+0000 7facde59c700 1 --2- 172.21.15.3:0/2926777121 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7face009d6f0 0x7face01c84d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7facd000c4e0 tx=0x7facd000c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:19.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.933+0000 7facceffd700 1 -- 172.21.15.3:0/2926777121 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd00052d0 con 0x7face009d6f0 2023-10-03T10:16:19.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.933+0000 7facceffd700 1 -- 172.21.15.3:0/2926777121 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd0005430 con 0x7face009d6f0 2023-10-03T10:16:19.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.933+0000 7face4fdb700 1 -- 172.21.15.3:0/2926777121 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd8009a70 con 0x7face009d6f0 2023-10-03T10:16:19.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.933+0000 7facceffd700 1 -- 172.21.15.3:0/2926777121 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd00038c0 con 0x7face009d6f0 2023-10-03T10:16:19.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.933+0000 7face4fdb700 1 -- 172.21.15.3:0/2926777121 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face0122aa0 con 0x7face009d6f0 2023-10-03T10:16:19.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.934+0000 7facceffd700 1 -- 172.21.15.3:0/2926777121 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7facd0003a20 con 0x7face009d6f0 2023-10-03T10:16:19.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.934+0000 7facceffd700 1 -- 172.21.15.3:0/2926777121 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7facd003eaa0 con 0x7face009d6f0 2023-10-03T10:16:19.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.934+0000 7face4fdb700 1 -- 172.21.15.3:0/2926777121 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face0074ea0 con 0x7face009d6f0 2023-10-03T10:16:19.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:19.939+0000 7facceffd700 1 -- 172.21.15.3:0/2926777121 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7facd0017080 con 0x7face009d6f0 2023-10-03T10:16:19.955 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:20.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.018+0000 7fbc8d29e700 1 Processor -- start 2023-10-03T10:16:20.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.018+0000 7fbc8d29e700 1 -- start start 2023-10-03T10:16:20.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.018+0000 7fbc8d29e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbc8809e930 0x7fbc8809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:20.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.018+0000 7fbc8d29e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc8809f240 0x7fbc881317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:20.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.018+0000 7fbc8d29e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc8809f650 0x7fbc88131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:20.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.018+0000 7fbc8d29e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc88070940 con 0x7fbc8809e930 2023-10-03T10:16:20.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.018+0000 7fbc8d29e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8806fd60 con 0x7fbc8809f650 2023-10-03T10:16:20.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.018+0000 7fbc8d29e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8806fa80 con 0x7fbc8809f240 2023-10-03T10:16:20.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.019+0000 7fbc86ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbc8809e930 0x7fbc8809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:20.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.019+0000 7fbc86ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbc8809e930 0x7fbc8809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49902/0 (socket says 172.21.15.3:49902) 2023-10-03T10:16:20.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.019+0000 7fbc86ffd700 1 -- 172.21.15.3:0/3831303781 learned_addr learned my addr 172.21.15.3:0/3831303781 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:20.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.019+0000 7fbc877fe700 1 --2- 172.21.15.3:0/3831303781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc8809f650 0x7fbc88131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:20.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.019+0000 7fbc867fc700 1 --2- 172.21.15.3:0/3831303781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc8809f240 0x7fbc881317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:20.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.019+0000 7fbc86ffd700 1 -- 172.21.15.3:0/3831303781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc8809f240 msgr2=0x7fbc881317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:20.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.019+0000 7fbc86ffd700 1 --2- 172.21.15.3:0/3831303781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc8809f240 0x7fbc881317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.019+0000 7fbc86ffd700 1 -- 172.21.15.3:0/3831303781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc8809f650 msgr2=0x7fbc88131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:20.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.019+0000 7fbc86ffd700 1 --2- 172.21.15.3:0/3831303781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc8809f650 0x7fbc88131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.019+0000 7fbc86ffd700 1 -- 172.21.15.3:0/3831303781 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc8809fb60 con 0x7fbc8809e930 2023-10-03T10:16:20.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.020+0000 7fbc86ffd700 1 --2- 172.21.15.3:0/3831303781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc8809e930 0x7fbc8809ed00 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fbc780098f0 tx=0x7fbc78009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3311175881e7164 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:20.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.020+0000 7fbc85ffb700 1 -- 172.21.15.3:0/3831303781 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78011830 con 0x7fbc8809e930 2023-10-03T10:16:20.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.020+0000 7fbc85ffb700 1 -- 172.21.15.3:0/3831303781 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc78011990 con 0x7fbc8809e930 2023-10-03T10:16:20.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.020+0000 7fbc85ffb700 1 -- 172.21.15.3:0/3831303781 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78011d20 con 0x7fbc8809e930 2023-10-03T10:16:20.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.020+0000 7fbc8d29e700 1 -- 172.21.15.3:0/3831303781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc8809e930 msgr2=0x7fbc8809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:20.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.020+0000 7fbc8d29e700 1 --2- 172.21.15.3:0/3831303781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc8809e930 0x7fbc8809ed00 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fbc780098f0 tx=0x7fbc78009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:20.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.021+0000 7fbc8d29e700 1 -- 172.21.15.3:0/3831303781 shutdown_connections 2023-10-03T10:16:20.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.021+0000 7fbc8d29e700 1 --2- 172.21.15.3:0/3831303781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc8809f650 0x7fbc88131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.021+0000 7fbc8d29e700 1 --2- 172.21.15.3:0/3831303781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc8809e930 0x7fbc8809ed00 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.021+0000 7fbc8d29e700 1 --2- 172.21.15.3:0/3831303781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc8809f240 0x7fbc881317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.021+0000 7fbc8d29e700 1 -- 172.21.15.3:0/3831303781 >> 172.21.15.3:0/3831303781 conn(0x7fbc880a4210 msgr2=0x7fbc880a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:20.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.021+0000 7fbc8d29e700 1 -- 172.21.15.3:0/3831303781 shutdown_connections 2023-10-03T10:16:20.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.021+0000 7fbc8d29e700 1 -- 172.21.15.3:0/3831303781 wait complete. 2023-10-03T10:16:20.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.022+0000 7fbc8d29e700 1 Processor -- start 2023-10-03T10:16:20.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.022+0000 7fbc8d29e700 1 -- start start 2023-10-03T10:16:20.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.023+0000 7fbc8d29e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc8809e930 0x7fbc8806b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:20.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.023+0000 7fbc8d29e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc8809f240 0x7fbc8806bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:20.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.023+0000 7fbc8d29e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc8809f650 0x7fbc8806c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:20.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.023+0000 7fbc8d29e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8806d4c0 con 0x7fbc8809e930 2023-10-03T10:16:20.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.023+0000 7fbc8d29e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc881a92f0 con 0x7fbc8809f240 2023-10-03T10:16:20.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.023+0000 7fbc8d29e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8806cd10 con 0x7fbc8809f650 2023-10-03T10:16:20.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.023+0000 7fbc86ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc8809e930 0x7fbc8806b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:20.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.023+0000 7fbc86ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc8809e930 0x7fbc8806b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49910/0 (socket says 172.21.15.3:49910) 2023-10-03T10:16:20.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.023+0000 7fbc86ffd700 1 -- 172.21.15.3:0/3870388649 learned_addr learned my addr 172.21.15.3:0/3870388649 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:20.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.023+0000 7fbc877fe700 1 --2- 172.21.15.3:0/3870388649 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc8809f650 0x7fbc8806c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:20.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.023+0000 7fbc867fc700 1 --2- 172.21.15.3:0/3870388649 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc8809f240 0x7fbc8806bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:20.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.023+0000 7fbc86ffd700 1 -- 172.21.15.3:0/3870388649 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc8809f650 msgr2=0x7fbc8806c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:20.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.023+0000 7fbc86ffd700 1 --2- 172.21.15.3:0/3870388649 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc8809f650 0x7fbc8806c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.023+0000 7fbc86ffd700 1 -- 172.21.15.3:0/3870388649 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc8809f240 msgr2=0x7fbc8806bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:20.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.023+0000 7fbc86ffd700 1 --2- 172.21.15.3:0/3870388649 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc8809f240 0x7fbc8806bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.023+0000 7fbc86ffd700 1 -- 172.21.15.3:0/3870388649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc78009550 con 0x7fbc8809e930 2023-10-03T10:16:20.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.023+0000 7fbc877fe700 1 --2- 172.21.15.3:0/3870388649 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc8809f650 0x7fbc8806c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:20.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.023+0000 7fbc86ffd700 1 --2- 172.21.15.3:0/3870388649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc8809e930 0x7fbc8806b8b0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fbc78009c40 tx=0x7fbc78004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:20.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.024+0000 7fbc6f7fe700 1 -- 172.21.15.3:0/3870388649 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78004240 con 0x7fbc8809e930 2023-10-03T10:16:20.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.024+0000 7fbc6f7fe700 1 -- 172.21.15.3:0/3870388649 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7801f070 con 0x7fbc8809e930 2023-10-03T10:16:20.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.024+0000 7fbc6f7fe700 1 -- 172.21.15.3:0/3870388649 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc780036a0 con 0x7fbc8809e930 2023-10-03T10:16:20.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.024+0000 7fbc8d29e700 1 -- 172.21.15.3:0/3870388649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc880a0190 con 0x7fbc8809e930 2023-10-03T10:16:20.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.024+0000 7fbc8d29e700 1 -- 172.21.15.3:0/3870388649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc881a9020 con 0x7fbc8809e930 2023-10-03T10:16:20.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.025+0000 7fbc6f7fe700 1 -- 172.21.15.3:0/3870388649 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbc78021030 con 0x7fbc8809e930 2023-10-03T10:16:20.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.025+0000 7fbc6f7fe700 1 -- 172.21.15.3:0/3870388649 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fbc78040830 con 0x7fbc8809e930 2023-10-03T10:16:20.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.025+0000 7fbc8d29e700 1 -- 172.21.15.3:0/3870388649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc881a0b30 con 0x7fbc8809e930 2023-10-03T10:16:20.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.030+0000 7fbc6f7fe700 1 -- 172.21.15.3:0/3870388649 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbc780199a0 con 0x7fbc8809e930 2023-10-03T10:16:20.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.069+0000 7f6f3f235700 1 Processor -- start 2023-10-03T10:16:20.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.069+0000 7f6f3f235700 1 -- start start 2023-10-03T10:16:20.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.070+0000 7f6f3f235700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f3809e930 0x7f6f3809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:20.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.070+0000 7f6f3f235700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6f3809f240 0x7f6f381317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:20.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.070+0000 7f6f3f235700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f3809f650 0x7f6f38131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:20.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.070+0000 7f6f3f235700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f38070940 con 0x7f6f3809f240 2023-10-03T10:16:20.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.070+0000 7f6f3f235700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3806fd60 con 0x7f6f3809e930 2023-10-03T10:16:20.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.070+0000 7f6f3f235700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3806fa80 con 0x7f6f3809f650 2023-10-03T10:16:20.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.070+0000 7f6f3cfd1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f3809e930 0x7f6f3809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:20.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.070+0000 7f6f3cfd1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f3809e930 0x7f6f3809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44410/0 (socket says 172.21.15.3:44410) 2023-10-03T10:16:20.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.070+0000 7f6f3cfd1700 1 -- 172.21.15.3:0/3959281105 learned_addr learned my addr 172.21.15.3:0/3959281105 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:20.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.070+0000 7f6f37fff700 1 --2- 172.21.15.3:0/3959281105 >> v2:172.21.15.3:3300/0 conn(0x7f6f3809f240 0x7f6f381317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:20.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.070+0000 7f6f3d7d2700 1 --2- 172.21.15.3:0/3959281105 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f3809f650 0x7f6f38131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:20.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.071+0000 7f6f3cfd1700 1 -- 172.21.15.3:0/3959281105 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f3809f650 msgr2=0x7f6f38131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:20.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.071+0000 7f6f3cfd1700 1 --2- 172.21.15.3:0/3959281105 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f3809f650 0x7f6f38131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.071+0000 7f6f3cfd1700 1 -- 172.21.15.3:0/3959281105 >> v2:172.21.15.3:3300/0 conn(0x7f6f3809f240 msgr2=0x7f6f381317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:20.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.071+0000 7f6f3cfd1700 1 --2- 172.21.15.3:0/3959281105 >> v2:172.21.15.3:3300/0 conn(0x7f6f3809f240 0x7f6f381317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.071+0000 7f6f3cfd1700 1 -- 172.21.15.3:0/3959281105 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3809fb60 con 0x7f6f3809e930 2023-10-03T10:16:20.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.071+0000 7f6f37fff700 1 --2- 172.21.15.3:0/3959281105 >> v2:172.21.15.3:3300/0 conn(0x7f6f3809f240 0x7f6f381317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:20.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.071+0000 7f6f3cfd1700 1 --2- 172.21.15.3:0/3959281105 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f3809e930 0x7f6f3809ed00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c0098f0 tx=0x7f6f2c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0fa4b625cf046c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:20.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.071+0000 7f6f377fe700 1 -- 172.21.15.3:0/3959281105 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c011830 con 0x7f6f3809e930 2023-10-03T10:16:20.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.071+0000 7f6f377fe700 1 -- 172.21.15.3:0/3959281105 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2c011990 con 0x7f6f3809e930 2023-10-03T10:16:20.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.071+0000 7f6f377fe700 1 -- 172.21.15.3:0/3959281105 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c011d20 con 0x7f6f3809e930 2023-10-03T10:16:20.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.072+0000 7f6f3f235700 1 -- 172.21.15.3:0/3959281105 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f3809e930 msgr2=0x7f6f3809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:20.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.072+0000 7f6f3f235700 1 --2- 172.21.15.3:0/3959281105 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f3809e930 0x7f6f3809ed00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c0098f0 tx=0x7f6f2c009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:20.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.072+0000 7f6f3f235700 1 -- 172.21.15.3:0/3959281105 shutdown_connections 2023-10-03T10:16:20.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.072+0000 7f6f3f235700 1 --2- 172.21.15.3:0/3959281105 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f3809f650 0x7f6f38131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.072+0000 7f6f3f235700 1 --2- 172.21.15.3:0/3959281105 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f3809e930 0x7f6f3809ed00 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.072+0000 7f6f3f235700 1 --2- 172.21.15.3:0/3959281105 >> v2:172.21.15.3:3300/0 conn(0x7f6f3809f240 0x7f6f381317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.072+0000 7f6f3f235700 1 -- 172.21.15.3:0/3959281105 >> 172.21.15.3:0/3959281105 conn(0x7f6f380a4210 msgr2=0x7f6f380a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:20.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.072+0000 7f6f3f235700 1 -- 172.21.15.3:0/3959281105 shutdown_connections 2023-10-03T10:16:20.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.072+0000 7f6f3f235700 1 -- 172.21.15.3:0/3959281105 wait complete. 2023-10-03T10:16:20.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.073+0000 7f6f3f235700 1 Processor -- start 2023-10-03T10:16:20.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.073+0000 7f6f3f235700 1 -- start start 2023-10-03T10:16:20.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.074+0000 7f6f3f235700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f3809e930 0x7f6f381310b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:20.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.074+0000 7f6f3f235700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f3809f240 0x7f6f381315f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:20.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.074+0000 7f6f3f235700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f3809f650 0x7f6f38131b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:20.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.074+0000 7f6f3f235700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f381330c0 con 0x7f6f3809e930 2023-10-03T10:16:20.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.074+0000 7f6f3f235700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f38132ed0 con 0x7f6f3809f650 2023-10-03T10:16:20.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.074+0000 7f6f3f235700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f38071d20 con 0x7f6f3809f240 2023-10-03T10:16:20.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.074+0000 7f6f3cfd1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f3809e930 0x7f6f381310b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:20.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.074+0000 7f6f3d7d2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f3809f650 0x7f6f38131b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:20.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.074+0000 7f6f3cfd1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f3809e930 0x7f6f381310b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49920/0 (socket says 172.21.15.3:49920) 2023-10-03T10:16:20.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.074+0000 7f6f3cfd1700 1 -- 172.21.15.3:0/2830344249 learned_addr learned my addr 172.21.15.3:0/2830344249 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:20.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.074+0000 7f6f37fff700 1 --2- 172.21.15.3:0/2830344249 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f3809f240 0x7f6f381315f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:20.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.074+0000 7f6f3cfd1700 1 -- 172.21.15.3:0/2830344249 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f3809f240 msgr2=0x7f6f381315f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:20.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.074+0000 7f6f3cfd1700 1 --2- 172.21.15.3:0/2830344249 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f3809f240 0x7f6f381315f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.074+0000 7f6f3cfd1700 1 -- 172.21.15.3:0/2830344249 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f3809f650 msgr2=0x7f6f38131b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:20.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.074+0000 7f6f3cfd1700 1 --2- 172.21.15.3:0/2830344249 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f3809f650 0x7f6f38131b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.074+0000 7f6f3cfd1700 1 -- 172.21.15.3:0/2830344249 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f240094a0 con 0x7f6f3809e930 2023-10-03T10:16:20.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.074+0000 7f6f3cfd1700 1 --2- 172.21.15.3:0/2830344249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f3809e930 0x7f6f381310b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009c40 tx=0x7f6f2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:20.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.075+0000 7f6f357fa700 1 -- 172.21.15.3:0/2830344249 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c004240 con 0x7f6f3809e930 2023-10-03T10:16:20.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.075+0000 7f6f3f235700 1 -- 172.21.15.3:0/2830344249 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f2c009550 con 0x7f6f3809e930 2023-10-03T10:16:20.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.075+0000 7f6f357fa700 1 -- 172.21.15.3:0/2830344249 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2c01f070 con 0x7f6f3809e930 2023-10-03T10:16:20.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.075+0000 7f6f357fa700 1 -- 172.21.15.3:0/2830344249 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c0045e0 con 0x7f6f3809e930 2023-10-03T10:16:20.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.075+0000 7f6f3f235700 1 -- 172.21.15.3:0/2830344249 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f380a0190 con 0x7f6f3809e930 2023-10-03T10:16:20.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.076+0000 7f6f357fa700 1 -- 172.21.15.3:0/2830344249 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6f2c021030 con 0x7f6f3809e930 2023-10-03T10:16:20.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.076+0000 7f6f3f235700 1 -- 172.21.15.3:0/2830344249 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f3804ea50 con 0x7f6f3809e930 2023-10-03T10:16:20.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.076+0000 7f6f357fa700 1 -- 172.21.15.3:0/2830344249 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6f2c040830 con 0x7f6f3809e930 2023-10-03T10:16:20.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.086+0000 7f6f357fa700 1 -- 172.21.15.3:0/2830344249 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6f2c0197f0 con 0x7f6f3809e930 2023-10-03T10:16:20.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.100+0000 7fa25f47f700 1 Processor -- start 2023-10-03T10:16:20.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.101+0000 7fa25f47f700 1 -- start start 2023-10-03T10:16:20.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.101+0000 7fa25f47f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa25809de00 0x7fa25809e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:20.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.101+0000 7fa25f47f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa25813d1f0 0x7fa25813f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:20.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.101+0000 7fa25f47f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa25809a810 con 0x7fa25806a770 2023-10-03T10:16:20.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.101+0000 7fa25f47f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa25806ab40 con 0x7fa25809de00 2023-10-03T10:16:20.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.101+0000 7fa25f47f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25809ebc0 con 0x7fa25813d1f0 2023-10-03T10:16:20.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.101+0000 7fa25ca1a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa25809de00 0x7fa25809e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:20.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.101+0000 7fa25ca1a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa25809de00 0x7fa25809e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49922/0 (socket says 172.21.15.3:49922) 2023-10-03T10:16:20.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.101+0000 7fa25ca1a700 1 -- 172.21.15.3:0/4088570913 learned_addr learned my addr 172.21.15.3:0/4088570913 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:20.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.101+0000 7fa25da1c700 1 --2- 172.21.15.3:0/4088570913 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa25813d1f0 0x7fa25813f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:20.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.102+0000 7fa25da1c700 1 -- 172.21.15.3:0/4088570913 >> v2:172.21.15.3:3300/0 conn(0x7fa25809de00 msgr2=0x7fa25809e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:20.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.102+0000 7fa25da1c700 1 --2- 172.21.15.3:0/4088570913 >> v2:172.21.15.3:3300/0 conn(0x7fa25809de00 0x7fa25809e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.102+0000 7fa25da1c700 1 -- 172.21.15.3:0/4088570913 >> v1:172.21.15.3:6789/0 conn(0x7fa25806a770 legacy=0x7fa25809d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:20.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.102+0000 7fa24ffff700 1 -- 172.21.15.3:0/4088570913 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1451413905 0 0) 0x7fa25809a810 con 0x7fa25806a770 2023-10-03T10:16:20.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.102+0000 7fa25da1c700 1 -- 172.21.15.3:0/4088570913 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25809e920 con 0x7fa25813d1f0 2023-10-03T10:16:20.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.102+0000 7fa25ca1a700 1 --2- 172.21.15.3:0/4088570913 >> v2:172.21.15.3:3300/0 conn(0x7fa25809de00 0x7fa25809e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:20.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.102+0000 7fa25da1c700 1 --2- 172.21.15.3:0/4088570913 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa25813d1f0 0x7fa25813f5d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa250009d90 tx=0x7fa25000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ded3da4cd210a1c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:20.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.102+0000 7fa24ffff700 1 -- 172.21.15.3:0/4088570913 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25000e070 con 0x7fa25813d1f0 2023-10-03T10:16:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.102+0000 7fa24ffff700 1 -- 172.21.15.3:0/4088570913 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25000bb20 con 0x7fa25813d1f0 2023-10-03T10:16:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.102+0000 7fa24ffff700 1 -- 172.21.15.3:0/4088570913 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250012430 con 0x7fa25813d1f0 2023-10-03T10:16:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.103+0000 7fa25f47f700 1 -- 172.21.15.3:0/4088570913 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa25813d1f0 msgr2=0x7fa25813f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.103+0000 7fa25f47f700 1 --2- 172.21.15.3:0/4088570913 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa25813d1f0 0x7fa25813f5d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa250009d90 tx=0x7fa25000b330 comp rx=0 tx=0).stop 2023-10-03T10:16:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.103+0000 7fa25f47f700 1 -- 172.21.15.3:0/4088570913 shutdown_connections 2023-10-03T10:16:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.103+0000 7fa25f47f700 1 --2- 172.21.15.3:0/4088570913 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa25813d1f0 0x7fa25813f5d0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.103+0000 7fa25f47f700 1 --2- 172.21.15.3:0/4088570913 >> v2:172.21.15.3:3300/0 conn(0x7fa25809de00 0x7fa25809e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.103+0000 7fa25f47f700 1 -- 172.21.15.3:0/4088570913 >> 172.21.15.3:0/4088570913 conn(0x7fa258066f50 msgr2=0x7fa25809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.103+0000 7fa25f47f700 1 -- 172.21.15.3:0/4088570913 shutdown_connections 2023-10-03T10:16:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.103+0000 7fa25f47f700 1 -- 172.21.15.3:0/4088570913 wait complete. 2023-10-03T10:16:20.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.105+0000 7fa25f47f700 1 Processor -- start 2023-10-03T10:16:20.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.105+0000 7fa25f47f700 1 -- start start 2023-10-03T10:16:20.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.106+0000 7fa25f47f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa25806a770 0x7fa25811c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:20.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.106+0000 7fa25f47f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa25809de00 0x7fa25813b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:20.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.106+0000 7fa25f47f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa25813d1f0 0x7fa25813ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.106+0000 7fa25f47f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25811d130 con 0x7fa25813d1f0 2023-10-03T10:16:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.106+0000 7fa25f47f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25811cf40 con 0x7fa25809de00 2023-10-03T10:16:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.106+0000 7fa25f47f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa2580a0f30 con 0x7fa25806a770 2023-10-03T10:16:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.106+0000 7fa25ca1a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa25809de00 0x7fa25813b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.106+0000 7fa25ca1a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa25809de00 0x7fa25813b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44426/0 (socket says 172.21.15.3:44426) 2023-10-03T10:16:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.106+0000 7fa25ca1a700 1 -- 172.21.15.3:0/1469863994 learned_addr learned my addr 172.21.15.3:0/1469863994 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.106+0000 7fa25da1c700 1 --2- 172.21.15.3:0/1469863994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa25813d1f0 0x7fa25813ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.106+0000 7fa25d21b700 1 --2- 172.21.15.3:0/1469863994 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa25806a770 0x7fa25811c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.106+0000 7fa25ca1a700 1 -- 172.21.15.3:0/1469863994 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa25806a770 msgr2=0x7fa25811c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.106+0000 7fa25ca1a700 1 --2- 172.21.15.3:0/1469863994 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa25806a770 0x7fa25811c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.106+0000 7fa25ca1a700 1 -- 172.21.15.3:0/1469863994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa25813d1f0 msgr2=0x7fa25813ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:20.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.106+0000 7fa25ca1a700 1 --2- 172.21.15.3:0/1469863994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa25813d1f0 0x7fa25813ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.106+0000 7fa25ca1a700 1 -- 172.21.15.3:0/1469863994 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa248009550 con 0x7fa25809de00 2023-10-03T10:16:20.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.106+0000 7fa25da1c700 1 --2- 172.21.15.3:0/1469863994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa25813d1f0 0x7fa25813ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:20.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.106+0000 7fa25ca1a700 1 --2- 172.21.15.3:0/1469863994 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa25809de00 0x7fa25813b530 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa24800ebe0 tx=0x7fa24800eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:20.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.107+0000 7fa24dffb700 1 -- 172.21.15.3:0/1469863994 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24800c850 con 0x7fa25809de00 2023-10-03T10:16:20.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.107+0000 7fa24dffb700 1 -- 172.21.15.3:0/1469863994 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24800c9b0 con 0x7fa25809de00 2023-10-03T10:16:20.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.107+0000 7fa25f47f700 1 -- 172.21.15.3:0/1469863994 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa250009a70 con 0x7fa25809de00 2023-10-03T10:16:20.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.107+0000 7fa24dffb700 1 -- 172.21.15.3:0/1469863994 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24800cd40 con 0x7fa25809de00 2023-10-03T10:16:20.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.107+0000 7fa25f47f700 1 -- 172.21.15.3:0/1469863994 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa25809f1a0 con 0x7fa25809de00 2023-10-03T10:16:20.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.108+0000 7fa24dffb700 1 -- 172.21.15.3:0/1469863994 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa248013470 con 0x7fa25809de00 2023-10-03T10:16:20.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.108+0000 7fa25f47f700 1 -- 172.21.15.3:0/1469863994 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa258074ea0 con 0x7fa25809de00 2023-10-03T10:16:20.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.108+0000 7fa24dffb700 1 -- 172.21.15.3:0/1469863994 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa24803f970 con 0x7fa25809de00 2023-10-03T10:16:20.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.118+0000 7fa24dffb700 1 -- 172.21.15.3:0/1469863994 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa248017080 con 0x7fa25809de00 2023-10-03T10:16:20.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.165+0000 7face4fdb700 1 -- 172.21.15.3:0/2926777121 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7face009f420 con 0x7face009d6f0 2023-10-03T10:16:20.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.166+0000 7facceffd700 1 -- 172.21.15.3:0/2926777121 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7facd0019e60 con 0x7face009d6f0 2023-10-03T10:16:20.166 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:20.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.169+0000 7face4fdb700 1 -- 172.21.15.3:0/2926777121 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7face009d6f0 msgr2=0x7face01c84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:20.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.169+0000 7face4fdb700 1 --2- 172.21.15.3:0/2926777121 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7face009d6f0 0x7face01c84d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7facd000c4e0 tx=0x7facd000c7f0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.169+0000 7face4fdb700 1 -- 172.21.15.3:0/2926777121 shutdown_connections 2023-10-03T10:16:20.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.169+0000 7face4fdb700 1 --2- 172.21.15.3:0/2926777121 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7face0139170 0x7face011e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.170+0000 7face4fdb700 1 --2- 172.21.15.3:0/2926777121 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7face009d6f0 0x7face01c84d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.170+0000 7face4fdb700 1 --2- 172.21.15.3:0/2926777121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face009e1d0 0x7face01c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.170+0000 7face4fdb700 1 -- 172.21.15.3:0/2926777121 >> 172.21.15.3:0/2926777121 conn(0x7face0066f50 msgr2=0x7face00a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:20.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.170+0000 7face4fdb700 1 -- 172.21.15.3:0/2926777121 shutdown_connections 2023-10-03T10:16:20.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.170+0000 7face4fdb700 1 -- 172.21.15.3:0/2926777121 wait complete. 2023-10-03T10:16:20.185 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:20.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.261+0000 7fbc8d29e700 1 -- 172.21.15.3:0/3870388649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbc880a35c0 con 0x7fbc8809e930 2023-10-03T10:16:20.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.262+0000 7fbc6f7fe700 1 -- 172.21.15.3:0/3870388649 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbc780193b0 con 0x7fbc8809e930 2023-10-03T10:16:20.262 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:20.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.264+0000 7fbc8d29e700 1 -- 172.21.15.3:0/3870388649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc8809e930 msgr2=0x7fbc8806b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:20.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.264+0000 7fbc8d29e700 1 --2- 172.21.15.3:0/3870388649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc8809e930 0x7fbc8806b8b0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fbc78009c40 tx=0x7fbc78004750 comp rx=0 tx=0).stop 2023-10-03T10:16:20.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.264+0000 7fbc8d29e700 1 -- 172.21.15.3:0/3870388649 shutdown_connections 2023-10-03T10:16:20.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.264+0000 7fbc8d29e700 1 --2- 172.21.15.3:0/3870388649 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc8809f650 0x7fbc8806c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.264+0000 7fbc8d29e700 1 --2- 172.21.15.3:0/3870388649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc8809e930 0x7fbc8806b8b0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.264+0000 7fbc8d29e700 1 --2- 172.21.15.3:0/3870388649 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc8809f240 0x7fbc8806bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.264+0000 7fbc8d29e700 1 -- 172.21.15.3:0/3870388649 >> 172.21.15.3:0/3870388649 conn(0x7fbc880a4210 msgr2=0x7fbc881b8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:20.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.264+0000 7fbc8d29e700 1 -- 172.21.15.3:0/3870388649 shutdown_connections 2023-10-03T10:16:20.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.264+0000 7fbc8d29e700 1 -- 172.21.15.3:0/3870388649 wait complete. 2023-10-03T10:16:20.279 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:20.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.303+0000 7f6f3f235700 1 -- 172.21.15.3:0/2830344249 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6f38074ea0 con 0x7f6f3809e930 2023-10-03T10:16:20.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.303+0000 7f6f357fa700 1 -- 172.21.15.3:0/2830344249 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6f2c01b430 con 0x7f6f3809e930 2023-10-03T10:16:20.303 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:20.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.305+0000 7f6f3f235700 1 -- 172.21.15.3:0/2830344249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f3809e930 msgr2=0x7f6f381310b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:20.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.305+0000 7f6f3f235700 1 --2- 172.21.15.3:0/2830344249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f3809e930 0x7f6f381310b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009c40 tx=0x7f6f2c004750 comp rx=0 tx=0).stop 2023-10-03T10:16:20.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.306+0000 7f6f3f235700 1 -- 172.21.15.3:0/2830344249 shutdown_connections 2023-10-03T10:16:20.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.306+0000 7f6f3f235700 1 --2- 172.21.15.3:0/2830344249 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6f3809f650 0x7f6f38131b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.306+0000 7f6f3f235700 1 --2- 172.21.15.3:0/2830344249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f3809e930 0x7f6f381310b0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.306+0000 7f6f3f235700 1 --2- 172.21.15.3:0/2830344249 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6f3809f240 0x7f6f381315f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.306+0000 7f6f3f235700 1 -- 172.21.15.3:0/2830344249 >> 172.21.15.3:0/2830344249 conn(0x7f6f380a4210 msgr2=0x7f6f3812c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:20.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.306+0000 7f6f3f235700 1 -- 172.21.15.3:0/2830344249 shutdown_connections 2023-10-03T10:16:20.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.306+0000 7f6f3f235700 1 -- 172.21.15.3:0/2830344249 wait complete. 2023-10-03T10:16:20.319 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:20.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.330+0000 7fa25f47f700 1 -- 172.21.15.3:0/1469863994 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa25811e200 con 0x7fa25809de00 2023-10-03T10:16:20.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.330+0000 7fa24dffb700 1 -- 172.21.15.3:0/1469863994 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa248021020 con 0x7fa25809de00 2023-10-03T10:16:20.331 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:20.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.333+0000 7fa25f47f700 1 -- 172.21.15.3:0/1469863994 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa25809de00 msgr2=0x7fa25813b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:20.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.333+0000 7fa25f47f700 1 --2- 172.21.15.3:0/1469863994 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa25809de00 0x7fa25813b530 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa24800ebe0 tx=0x7fa24800eef0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.333+0000 7fa25f47f700 1 -- 172.21.15.3:0/1469863994 shutdown_connections 2023-10-03T10:16:20.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.333+0000 7fa25f47f700 1 --2- 172.21.15.3:0/1469863994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa25813d1f0 0x7fa25813ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.333+0000 7fa25f47f700 1 --2- 172.21.15.3:0/1469863994 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa25806a770 0x7fa25811c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.333+0000 7fa25f47f700 1 --2- 172.21.15.3:0/1469863994 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa25809de00 0x7fa25813b530 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:20.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.333+0000 7fa25f47f700 1 -- 172.21.15.3:0/1469863994 >> 172.21.15.3:0/1469863994 conn(0x7fa258066f50 msgr2=0x7fa25809b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:20.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.333+0000 7fa25f47f700 1 -- 172.21.15.3:0/1469863994 shutdown_connections 2023-10-03T10:16:20.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:20.333+0000 7fa25f47f700 1 -- 172.21.15.3:0/1469863994 wait complete. 2023-10-03T10:16:20.346 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:21.186 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:21.279 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:21.321 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:21.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.334+0000 7f2b68641700 1 Processor -- start 2023-10-03T10:16:21.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.334+0000 7f2b68641700 1 -- start start 2023-10-03T10:16:21.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.335+0000 7f2b68641700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b6009e1d0 0x7f2b60138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:21.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.335+0000 7f2b68641700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b60139170 0x7f2b6013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:21.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.335+0000 7f2b68641700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b6009b120 con 0x7f2b6009d6f0 2023-10-03T10:16:21.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.335+0000 7f2b68641700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6006b460 con 0x7f2b6009e1d0 2023-10-03T10:16:21.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.335+0000 7f2b68641700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6009a7e0 con 0x7f2b60139170 2023-10-03T10:16:21.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.335+0000 7f2b65bdc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b6009e1d0 0x7f2b60138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:21.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.335+0000 7f2b65bdc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b6009e1d0 0x7f2b60138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44442/0 (socket says 172.21.15.3:44442) 2023-10-03T10:16:21.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.335+0000 7f2b65bdc700 1 -- 172.21.15.3:0/672231732 learned_addr learned my addr 172.21.15.3:0/672231732 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:21.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.335+0000 7f2b66bde700 1 --2- 172.21.15.3:0/672231732 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b60139170 0x7f2b6013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:21.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.335+0000 7f2b66bde700 1 -- 172.21.15.3:0/672231732 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b6009e1d0 msgr2=0x7f2b60138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:21.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.335+0000 7f2b66bde700 1 --2- 172.21.15.3:0/672231732 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b6009e1d0 0x7f2b60138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.335+0000 7f2b66bde700 1 -- 172.21.15.3:0/672231732 >> v1:172.21.15.3:6789/0 conn(0x7f2b6009d6f0 legacy=0x7f2b6009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:21.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.335+0000 7f2b653db700 1 -- 172.21.15.3:0/672231732 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900081552 0 0) 0x7f2b6009b120 con 0x7f2b6009d6f0 2023-10-03T10:16:21.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.335+0000 7f2b66bde700 1 -- 172.21.15.3:0/672231732 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6009a500 con 0x7f2b60139170 2023-10-03T10:16:21.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.336+0000 7f2b66bde700 1 --2- 172.21.15.3:0/672231732 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b60139170 0x7f2b6013b550 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2b58009df0 tx=0x7f2b5800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c8132b97a4f4daa server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:21.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.336+0000 7f2b653db700 1 -- 172.21.15.3:0/672231732 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5800e070 con 0x7f2b60139170 2023-10-03T10:16:21.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.336+0000 7f2b653db700 1 -- 172.21.15.3:0/672231732 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5800bae0 con 0x7f2b60139170 2023-10-03T10:16:21.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.336+0000 7f2b653db700 1 -- 172.21.15.3:0/672231732 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5800be70 con 0x7f2b60139170 2023-10-03T10:16:21.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.337+0000 7f2b68641700 1 -- 172.21.15.3:0/672231732 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b60139170 msgr2=0x7f2b6013b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:21.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.337+0000 7f2b68641700 1 --2- 172.21.15.3:0/672231732 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b60139170 0x7f2b6013b550 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2b58009df0 tx=0x7f2b5800b330 comp rx=0 tx=0).stop 2023-10-03T10:16:21.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.337+0000 7f2b68641700 1 -- 172.21.15.3:0/672231732 shutdown_connections 2023-10-03T10:16:21.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.337+0000 7f2b68641700 1 --2- 172.21.15.3:0/672231732 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b60139170 0x7f2b6013b550 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.337+0000 7f2b68641700 1 --2- 172.21.15.3:0/672231732 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b6009e1d0 0x7f2b60138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.337+0000 7f2b68641700 1 -- 172.21.15.3:0/672231732 >> 172.21.15.3:0/672231732 conn(0x7f2b60066f50 msgr2=0x7f2b6009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:21.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.337+0000 7f2b68641700 1 -- 172.21.15.3:0/672231732 shutdown_connections 2023-10-03T10:16:21.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.337+0000 7f2b68641700 1 -- 172.21.15.3:0/672231732 wait complete. 2023-10-03T10:16:21.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.338+0000 7f2b68641700 1 Processor -- start 2023-10-03T10:16:21.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.338+0000 7f2b68641700 1 -- start start 2023-10-03T10:16:21.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.339+0000 7f2b68641700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b6009d6f0 0x7f2b6011e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:21.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.339+0000 7f2b68641700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b6009e1d0 0x7f2b6011ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:21.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.339+0000 7f2b68641700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b60139170 0x7f2b6011f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:21.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.339+0000 7f2b68641700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b60120a60 con 0x7f2b6009d6f0 2023-10-03T10:16:21.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.339+0000 7f2b68641700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b60120190 con 0x7f2b60139170 2023-10-03T10:16:21.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.339+0000 7f2b68641700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b601220a0 con 0x7f2b6009e1d0 2023-10-03T10:16:21.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.339+0000 7f2b663dd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b6009d6f0 0x7f2b6011e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:21.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.339+0000 7f2b663dd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b6009d6f0 0x7f2b6011e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49938/0 (socket says 172.21.15.3:49938) 2023-10-03T10:16:21.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.339+0000 7f2b663dd700 1 -- 172.21.15.3:0/1341567494 learned_addr learned my addr 172.21.15.3:0/1341567494 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:21.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.339+0000 7f2b65bdc700 1 --2- 172.21.15.3:0/1341567494 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b6009e1d0 0x7f2b6011ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:21.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.339+0000 7f2b66bde700 1 --2- 172.21.15.3:0/1341567494 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b60139170 0x7f2b6011f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:21.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.339+0000 7f2b663dd700 1 -- 172.21.15.3:0/1341567494 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b6009e1d0 msgr2=0x7f2b6011ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:21.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.339+0000 7f2b663dd700 1 --2- 172.21.15.3:0/1341567494 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b6009e1d0 0x7f2b6011ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.339+0000 7f2b663dd700 1 -- 172.21.15.3:0/1341567494 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b60139170 msgr2=0x7f2b6011f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:21.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.339+0000 7f2b663dd700 1 --2- 172.21.15.3:0/1341567494 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b60139170 0x7f2b6011f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.339+0000 7f2b663dd700 1 -- 172.21.15.3:0/1341567494 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b58009a70 con 0x7f2b6009d6f0 2023-10-03T10:16:21.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.339+0000 7f2b663dd700 1 --2- 172.21.15.3:0/1341567494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b6009d6f0 0x7f2b6011e8e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00c520 tx=0x7f2b5c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:21.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.340+0000 7f2b56ffd700 1 -- 172.21.15.3:0/1341567494 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c0054e0 con 0x7f2b6009d6f0 2023-10-03T10:16:21.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.340+0000 7f2b56ffd700 1 -- 172.21.15.3:0/1341567494 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5c003680 con 0x7f2b6009d6f0 2023-10-03T10:16:21.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.340+0000 7f2b68641700 1 -- 172.21.15.3:0/1341567494 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b60121450 con 0x7f2b6009d6f0 2023-10-03T10:16:21.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.340+0000 7f2b56ffd700 1 -- 172.21.15.3:0/1341567494 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c0039f0 con 0x7f2b6009d6f0 2023-10-03T10:16:21.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.340+0000 7f2b68641700 1 -- 172.21.15.3:0/1341567494 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b60121dc0 con 0x7f2b6009d6f0 2023-10-03T10:16:21.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.341+0000 7f2b56ffd700 1 -- 172.21.15.3:0/1341567494 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2b5c003b50 con 0x7f2b6009d6f0 2023-10-03T10:16:21.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.341+0000 7f2b56ffd700 1 -- 172.21.15.3:0/1341567494 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2b5c03eaa0 con 0x7f2b6009d6f0 2023-10-03T10:16:21.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.341+0000 7f2b68641700 1 -- 172.21.15.3:0/1341567494 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b60121c20 con 0x7f2b6009d6f0 2023-10-03T10:16:21.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.347+0000 7f2b56ffd700 1 -- 172.21.15.3:0/1341567494 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2b60121c20 con 0x7f2b6009d6f0 2023-10-03T10:16:21.347 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:21.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.425+0000 7f991db49700 1 Processor -- start 2023-10-03T10:16:21.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.426+0000 7f991db49700 1 -- start start 2023-10-03T10:16:21.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.426+0000 7f991db49700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9918134630 0x7f9918136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:21.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.426+0000 7f991db49700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9918136f50 0x7f9918139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:21.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.426+0000 7f991db49700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991809b4f0 con 0x7f9918131810 2023-10-03T10:16:21.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.426+0000 7f991db49700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99181313c0 con 0x7f9918136f50 2023-10-03T10:16:21.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.426+0000 7f991db49700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f991806e8c0 con 0x7f9918134630 2023-10-03T10:16:21.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.426+0000 7f99177fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9918131810 0x7f9918131be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:41382/0 (socket says 172.21.15.3:41382) 2023-10-03T10:16:21.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.426+0000 7f99177fe700 1 -- 172.21.15.3:0/1944848490 learned_addr learned my addr 172.21.15.3:0/1944848490 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:21.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.426+0000 7f9917fff700 1 --2- 172.21.15.3:0/1944848490 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9918136f50 0x7f9918139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:21.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.426+0000 7f9916ffd700 1 --2- 172.21.15.3:0/1944848490 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9918134630 0x7f9918136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:21.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.427+0000 7f9916ffd700 1 -- 172.21.15.3:0/1944848490 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9918136f50 msgr2=0x7f9918139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:21.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.427+0000 7f9916ffd700 1 --2- 172.21.15.3:0/1944848490 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9918136f50 0x7f9918139330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.427+0000 7f9916ffd700 1 -- 172.21.15.3:0/1944848490 >> v1:172.21.15.3:6789/0 conn(0x7f9918131810 legacy=0x7f9918131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:21.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.427+0000 7f9916ffd700 1 -- 172.21.15.3:0/1944848490 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9918071d20 con 0x7f9918134630 2023-10-03T10:16:21.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.427+0000 7f9916ffd700 1 --2- 172.21.15.3:0/1944848490 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9918134630 0x7f9918136a10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f990c009870 tx=0x7f990c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e10a05272d8c2b3e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:21.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.427+0000 7f99167fc700 1 -- 172.21.15.3:0/1944848490 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c011830 con 0x7f9918134630 2023-10-03T10:16:21.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.427+0000 7f99167fc700 1 -- 172.21.15.3:0/1944848490 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990c011990 con 0x7f9918134630 2023-10-03T10:16:21.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.427+0000 7f99167fc700 1 -- 172.21.15.3:0/1944848490 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c011d20 con 0x7f9918134630 2023-10-03T10:16:21.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.428+0000 7f991db49700 1 -- 172.21.15.3:0/1944848490 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9918134630 msgr2=0x7f9918136a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:21.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.428+0000 7f991db49700 1 --2- 172.21.15.3:0/1944848490 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9918134630 0x7f9918136a10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f990c009870 tx=0x7f990c009b80 comp rx=0 tx=0).stop 2023-10-03T10:16:21.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.428+0000 7f991db49700 1 -- 172.21.15.3:0/1944848490 shutdown_connections 2023-10-03T10:16:21.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.428+0000 7f991db49700 1 --2- 172.21.15.3:0/1944848490 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9918136f50 0x7f9918139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.428+0000 7f991db49700 1 --2- 172.21.15.3:0/1944848490 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9918134630 0x7f9918136a10 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.428+0000 7f991db49700 1 -- 172.21.15.3:0/1944848490 >> 172.21.15.3:0/1944848490 conn(0x7f99180a2aa0 msgr2=0x7f99180a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:21.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.428+0000 7f991db49700 1 -- 172.21.15.3:0/1944848490 shutdown_connections 2023-10-03T10:16:21.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.428+0000 7f991db49700 1 -- 172.21.15.3:0/1944848490 wait complete. 2023-10-03T10:16:21.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.429+0000 7f991db49700 1 Processor -- start 2023-10-03T10:16:21.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.430+0000 7f991db49700 1 -- start start 2023-10-03T10:16:21.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.430+0000 7f991db49700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9918131810 0x7f99181c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:21.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.430+0000 7f991db49700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9918134630 0x7f99181c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:21.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.430+0000 7f991db49700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9918136f50 0x7f9918118f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:21.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.430+0000 7f991db49700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991811caa0 con 0x7f9918134630 2023-10-03T10:16:21.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.430+0000 7f991db49700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991809ff00 con 0x7f9918136f50 2023-10-03T10:16:21.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.430+0000 7f991db49700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f991811c5f0 con 0x7f9918131810 2023-10-03T10:16:21.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.430+0000 7f9917fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9918136f50 0x7f9918118f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:21.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.430+0000 7f99177fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9918131810 0x7f99181c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:21.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.430+0000 7f9916ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9918134630 0x7f99181c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:21.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.430+0000 7f99177fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9918131810 0x7f99181c6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:33294/0 (socket says 172.21.15.3:33294) 2023-10-03T10:16:21.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.430+0000 7f9917fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9918136f50 0x7f9918118f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44454/0 (socket says 172.21.15.3:44454) 2023-10-03T10:16:21.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.430+0000 7f99177fe700 1 -- 172.21.15.3:0/3425913555 learned_addr learned my addr 172.21.15.3:0/3425913555 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:21.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.430+0000 7f9916ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9918134630 0x7f99181c6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49944/0 (socket says 172.21.15.3:49944) 2023-10-03T10:16:21.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.431+0000 7f99177fe700 1 -- 172.21.15.3:0/3425913555 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9918136f50 msgr2=0x7f9918118f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:21.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.431+0000 7f99177fe700 1 --2- 172.21.15.3:0/3425913555 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9918136f50 0x7f9918118f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.431+0000 7f99177fe700 1 -- 172.21.15.3:0/3425913555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9918134630 msgr2=0x7f99181c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:21.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.431+0000 7f99177fe700 1 --2- 172.21.15.3:0/3425913555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9918134630 0x7f99181c6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.431+0000 7f99177fe700 1 -- 172.21.15.3:0/3425913555 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990c0094a0 con 0x7f9918131810 2023-10-03T10:16:21.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.431+0000 7f9917fff700 1 --2- 172.21.15.3:0/3425913555 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9918136f50 0x7f9918118f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:21.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.431+0000 7f99177fe700 1 --2- 172.21.15.3:0/3425913555 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9918131810 0x7f99181c6550 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f990800c480 tx=0x7f990800c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:21.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.431+0000 7f98fffff700 1 -- 172.21.15.3:0/3425913555 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9908005d70 con 0x7f9918131810 2023-10-03T10:16:21.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.431+0000 7f98fffff700 1 -- 172.21.15.3:0/3425913555 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990800ce90 con 0x7f9918131810 2023-10-03T10:16:21.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.431+0000 7f991db49700 1 -- 172.21.15.3:0/3425913555 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99181bc140 con 0x7f9918131810 2023-10-03T10:16:21.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.431+0000 7f98fffff700 1 -- 172.21.15.3:0/3425913555 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99080072d0 con 0x7f9918131810 2023-10-03T10:16:21.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.431+0000 7f991db49700 1 -- 172.21.15.3:0/3425913555 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991812ec10 con 0x7f9918131810 2023-10-03T10:16:21.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.432+0000 7f98fffff700 1 -- 172.21.15.3:0/3425913555 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9908007430 con 0x7f9918131810 2023-10-03T10:16:21.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.433+0000 7f991db49700 1 -- 172.21.15.3:0/3425913555 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f991811c220 con 0x7f9918131810 2023-10-03T10:16:21.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.433+0000 7f98fffff700 1 -- 172.21.15.3:0/3425913555 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f990803e9a0 con 0x7f9918131810 2023-10-03T10:16:21.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.442+0000 7f98fffff700 1 -- 172.21.15.3:0/3425913555 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9908017020 con 0x7f9918131810 2023-10-03T10:16:21.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.475+0000 7fb97ce39700 1 Processor -- start 2023-10-03T10:16:21.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.475+0000 7fb97ce39700 1 -- start start 2023-10-03T10:16:21.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.476+0000 7fb97ce39700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb97806a770 0x7fb97809d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:21.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.476+0000 7fb97ce39700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb97813d1f0 0x7fb97813f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:21.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.476+0000 7fb97ce39700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb97809a810 con 0x7fb97809dc30 2023-10-03T10:16:21.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.476+0000 7fb97ce39700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb97806ab40 con 0x7fb97813d1f0 2023-10-03T10:16:21.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.476+0000 7fb97ce39700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb97809ebc0 con 0x7fb97806a770 2023-10-03T10:16:21.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.476+0000 7fb97659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb97806a770 0x7fb97809d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:21.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.476+0000 7fb97659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb97806a770 0x7fb97809d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:33310/0 (socket says 172.21.15.3:33310) 2023-10-03T10:16:21.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.476+0000 7fb975d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb97809dc30 0x7fb97809e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:41388/0 (socket says 172.21.15.3:41388) 2023-10-03T10:16:21.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.476+0000 7fb97659c700 1 -- 172.21.15.3:0/2447151968 learned_addr learned my addr 172.21.15.3:0/2447151968 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:21.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.476+0000 7fb976d9d700 1 --2- 172.21.15.3:0/2447151968 >> v2:172.21.15.3:3300/0 conn(0x7fb97813d1f0 0x7fb97813f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:21.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.476+0000 7fb97659c700 1 -- 172.21.15.3:0/2447151968 >> v2:172.21.15.3:3300/0 conn(0x7fb97813d1f0 msgr2=0x7fb97813f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:21.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.476+0000 7fb97659c700 1 --2- 172.21.15.3:0/2447151968 >> v2:172.21.15.3:3300/0 conn(0x7fb97813d1f0 0x7fb97813f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.476+0000 7fb97659c700 1 -- 172.21.15.3:0/2447151968 >> v1:172.21.15.3:6789/0 conn(0x7fb97809dc30 legacy=0x7fb97809e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:21.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.476+0000 7fb97659c700 1 -- 172.21.15.3:0/2447151968 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97809e920 con 0x7fb97806a770 2023-10-03T10:16:21.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.476+0000 7fb976d9d700 1 --2- 172.21.15.3:0/2447151968 >> v2:172.21.15.3:3300/0 conn(0x7fb97813d1f0 0x7fb97813f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:21.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.477+0000 7fb97659c700 1 --2- 172.21.15.3:0/2447151968 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb97806a770 0x7fb97809d6f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb968009920 tx=0x7fb968009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4efcf3024579ddd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:21.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.477+0000 7fb97559a700 1 -- 172.21.15.3:0/2447151968 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb968011830 con 0x7fb97806a770 2023-10-03T10:16:21.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.477+0000 7fb97559a700 1 -- 172.21.15.3:0/2447151968 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb968011990 con 0x7fb97806a770 2023-10-03T10:16:21.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.477+0000 7fb97559a700 1 -- 172.21.15.3:0/2447151968 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb968011d20 con 0x7fb97806a770 2023-10-03T10:16:21.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.478+0000 7fb97ce39700 1 -- 172.21.15.3:0/2447151968 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb97806a770 msgr2=0x7fb97809d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:21.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.478+0000 7fb97ce39700 1 --2- 172.21.15.3:0/2447151968 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb97806a770 0x7fb97809d6f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb968009920 tx=0x7fb968009c30 comp rx=0 tx=0).stop 2023-10-03T10:16:21.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.478+0000 7fb97ce39700 1 -- 172.21.15.3:0/2447151968 shutdown_connections 2023-10-03T10:16:21.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.478+0000 7fb97ce39700 1 --2- 172.21.15.3:0/2447151968 >> v2:172.21.15.3:3300/0 conn(0x7fb97813d1f0 0x7fb97813f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.478+0000 7fb97ce39700 1 --2- 172.21.15.3:0/2447151968 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb97806a770 0x7fb97809d6f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.478+0000 7fb97ce39700 1 -- 172.21.15.3:0/2447151968 >> 172.21.15.3:0/2447151968 conn(0x7fb978066f50 msgr2=0x7fb97809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:21.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.478+0000 7fb97ce39700 1 -- 172.21.15.3:0/2447151968 shutdown_connections 2023-10-03T10:16:21.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.478+0000 7fb97ce39700 1 -- 172.21.15.3:0/2447151968 wait complete. 2023-10-03T10:16:21.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.480+0000 7fb97ce39700 1 Processor -- start 2023-10-03T10:16:21.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.480+0000 7fb97ce39700 1 -- start start 2023-10-03T10:16:21.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.481+0000 7fb97ce39700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb97806a770 0x7fb9781260a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:21.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.481+0000 7fb97ce39700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb97809dc30 0x7fb9781265e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:21.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.481+0000 7fb97ce39700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb97813d1f0 0x7fb978126b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:21.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.481+0000 7fb97ce39700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb978128a00 con 0x7fb97813d1f0 2023-10-03T10:16:21.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.481+0000 7fb97ce39700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb978129330 con 0x7fb97806a770 2023-10-03T10:16:21.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.481+0000 7fb97ce39700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb978072800 con 0x7fb97809dc30 2023-10-03T10:16:21.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.481+0000 7fb976d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb97813d1f0 0x7fb978126b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:21.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.481+0000 7fb976d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb97813d1f0 0x7fb978126b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49962/0 (socket says 172.21.15.3:49962) 2023-10-03T10:16:21.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.481+0000 7fb976d9d700 1 -- 172.21.15.3:0/2558599562 learned_addr learned my addr 172.21.15.3:0/2558599562 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:21.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.481+0000 7fb975d9b700 1 --2- 172.21.15.3:0/2558599562 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb97809dc30 0x7fb9781265e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:21.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.481+0000 7fb97659c700 1 --2- 172.21.15.3:0/2558599562 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb97806a770 0x7fb9781260a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:21.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.481+0000 7fb976d9d700 1 -- 172.21.15.3:0/2558599562 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb97809dc30 msgr2=0x7fb9781265e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:21.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.481+0000 7fb976d9d700 1 --2- 172.21.15.3:0/2558599562 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb97809dc30 0x7fb9781265e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.481+0000 7fb976d9d700 1 -- 172.21.15.3:0/2558599562 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb97806a770 msgr2=0x7fb9781260a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:21.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.481+0000 7fb976d9d700 1 --2- 172.21.15.3:0/2558599562 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb97806a770 0x7fb9781260a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.481+0000 7fb976d9d700 1 -- 172.21.15.3:0/2558599562 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb970009a70 con 0x7fb97813d1f0 2023-10-03T10:16:21.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.482+0000 7fb976d9d700 1 --2- 172.21.15.3:0/2558599562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb97813d1f0 0x7fb978126b20 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fb97000efc0 tx=0x7fb97000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:21.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.482+0000 7fb966ffd700 1 -- 172.21.15.3:0/2558599562 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb970009d70 con 0x7fb97813d1f0 2023-10-03T10:16:21.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.482+0000 7fb97ce39700 1 -- 172.21.15.3:0/2558599562 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb968009550 con 0x7fb97813d1f0 2023-10-03T10:16:21.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.482+0000 7fb966ffd700 1 -- 172.21.15.3:0/2558599562 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97000f040 con 0x7fb97813d1f0 2023-10-03T10:16:21.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.482+0000 7fb966ffd700 1 -- 172.21.15.3:0/2558599562 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb970014480 con 0x7fb97813d1f0 2023-10-03T10:16:21.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.482+0000 7fb97ce39700 1 -- 172.21.15.3:0/2558599562 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb978127440 con 0x7fb97813d1f0 2023-10-03T10:16:21.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.483+0000 7fb97ce39700 1 -- 172.21.15.3:0/2558599562 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9781282d0 con 0x7fb97813d1f0 2023-10-03T10:16:21.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.483+0000 7fb966ffd700 1 -- 172.21.15.3:0/2558599562 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb9700145e0 con 0x7fb97813d1f0 2023-10-03T10:16:21.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.483+0000 7fb966ffd700 1 -- 172.21.15.3:0/2558599562 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb9700405f0 con 0x7fb97813d1f0 2023-10-03T10:16:21.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.493+0000 7fb966ffd700 1 -- 172.21.15.3:0/2558599562 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb970007510 con 0x7fb97813d1f0 2023-10-03T10:16:21.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.495+0000 7f31e170e700 1 Processor -- start 2023-10-03T10:16:21.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.495+0000 7f31e170e700 1 -- start start 2023-10-03T10:16:21.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.496+0000 7f31e170e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31dc09e930 0x7f31dc09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:21.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.496+0000 7f31e170e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f31dc09f240 0x7f31dc1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:21.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.496+0000 7f31e170e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31dc070940 con 0x7f31dc09f650 2023-10-03T10:16:21.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.496+0000 7f31e170e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31dc06fd60 con 0x7f31dc09f240 2023-10-03T10:16:21.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.496+0000 7f31e170e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc06fa80 con 0x7f31dc09e930 2023-10-03T10:16:21.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.496+0000 7f31da7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f31dc09f240 0x7f31dc1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:21.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.496+0000 7f31da7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f31dc09f240 0x7f31dc1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49964/0 (socket says 172.21.15.3:49964) 2023-10-03T10:16:21.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.496+0000 7f31da7fc700 1 -- 172.21.15.3:0/1996456161 learned_addr learned my addr 172.21.15.3:0/1996456161 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:21.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.496+0000 7f31db7fe700 1 --1- 172.21.15.3:0/1996456161 >> v1:172.21.15.3:6789/0 conn(0x7f31dc09f650 0x7f31dc131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:41390/0 (socket says 172.21.15.3:41390) 2023-10-03T10:16:21.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.497+0000 7f31da7fc700 1 -- 172.21.15.3:0/1996456161 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31dc09e930 msgr2=0x7f31dc09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:21.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.497+0000 7f31da7fc700 1 --2- 172.21.15.3:0/1996456161 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31dc09e930 0x7f31dc09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.497+0000 7f31da7fc700 1 -- 172.21.15.3:0/1996456161 >> v1:172.21.15.3:6789/0 conn(0x7f31dc09f650 legacy=0x7f31dc131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:21.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.497+0000 7f31da7fc700 1 -- 172.21.15.3:0/1996456161 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31dc09fb60 con 0x7f31dc09f240 2023-10-03T10:16:21.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.497+0000 7f31d9ffb700 1 -- 172.21.15.3:0/1996456161 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3956445972 0 0) 0x7f31dc070940 con 0x7f31dc09f650 2023-10-03T10:16:21.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.497+0000 7f31da7fc700 1 --2- 172.21.15.3:0/1996456161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31dc09f240 0x7f31dc1317f0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f31d00098f0 tx=0x7f31d0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f807a3fb9b32d00 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:21.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.497+0000 7f31d9ffb700 1 -- 172.21.15.3:0/1996456161 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d000b8a0 con 0x7f31dc09f240 2023-10-03T10:16:21.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.497+0000 7f31d9ffb700 1 -- 172.21.15.3:0/1996456161 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d000ba00 con 0x7f31dc09f240 2023-10-03T10:16:21.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.497+0000 7f31d9ffb700 1 -- 172.21.15.3:0/1996456161 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d000bd90 con 0x7f31dc09f240 2023-10-03T10:16:21.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.498+0000 7f31e170e700 1 -- 172.21.15.3:0/1996456161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31dc09f240 msgr2=0x7f31dc1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:21.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.498+0000 7f31e170e700 1 --2- 172.21.15.3:0/1996456161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31dc09f240 0x7f31dc1317f0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f31d00098f0 tx=0x7f31d0009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:21.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.498+0000 7f31e170e700 1 -- 172.21.15.3:0/1996456161 shutdown_connections 2023-10-03T10:16:21.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.498+0000 7f31e170e700 1 --2- 172.21.15.3:0/1996456161 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31dc09e930 0x7f31dc09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.498+0000 7f31e170e700 1 --2- 172.21.15.3:0/1996456161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31dc09f240 0x7f31dc1317f0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.498+0000 7f31e170e700 1 -- 172.21.15.3:0/1996456161 >> 172.21.15.3:0/1996456161 conn(0x7f31dc0a4210 msgr2=0x7f31dc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:21.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.498+0000 7f31e170e700 1 -- 172.21.15.3:0/1996456161 shutdown_connections 2023-10-03T10:16:21.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.498+0000 7f31e170e700 1 -- 172.21.15.3:0/1996456161 wait complete. 2023-10-03T10:16:21.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.500+0000 7f31e170e700 1 Processor -- start 2023-10-03T10:16:21.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.500+0000 7f31e170e700 1 -- start start 2023-10-03T10:16:21.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.500+0000 7f31e170e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31dc09e930 0x7f31dc06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:21.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.500+0000 7f31e170e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31dc09f240 0x7f31dc06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:21.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.500+0000 7f31e170e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31dc09f650 0x7f31dc06c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:21.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.500+0000 7f31e170e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc1a98a0 con 0x7f31dc09f240 2023-10-03T10:16:21.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.500+0000 7f31e170e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc1a9030 con 0x7f31dc09e930 2023-10-03T10:16:21.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.500+0000 7f31e170e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc1a9300 con 0x7f31dc09f650 2023-10-03T10:16:21.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.501+0000 7f31da7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31dc09f240 0x7f31dc06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:21.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.501+0000 7f31da7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31dc09f240 0x7f31dc06bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49972/0 (socket says 172.21.15.3:49972) 2023-10-03T10:16:21.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.501+0000 7f31da7fc700 1 -- 172.21.15.3:0/1055331182 learned_addr learned my addr 172.21.15.3:0/1055331182 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:21.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.501+0000 7f31db7fe700 1 --2- 172.21.15.3:0/1055331182 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31dc09f650 0x7f31dc06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:21.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.501+0000 7f31da7fc700 1 -- 172.21.15.3:0/1055331182 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31dc09f650 msgr2=0x7f31dc06c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:21.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.501+0000 7f31da7fc700 1 --2- 172.21.15.3:0/1055331182 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31dc09f650 0x7f31dc06c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.501+0000 7f31da7fc700 1 -- 172.21.15.3:0/1055331182 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31dc09e930 msgr2=0x7f31dc06b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:21.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.501+0000 7f31da7fc700 1 --2- 172.21.15.3:0/1055331182 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31dc09e930 0x7f31dc06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.501+0000 7f31da7fc700 1 -- 172.21.15.3:0/1055331182 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d0009550 con 0x7f31dc09f240 2023-10-03T10:16:21.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.501+0000 7f31da7fc700 1 --2- 172.21.15.3:0/1055331182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31dc09f240 0x7f31dc06bdf0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f31d0000c00 tx=0x7f31d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:21.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.502+0000 7f31c37fe700 1 -- 172.21.15.3:0/1055331182 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d0004550 con 0x7f31dc09f240 2023-10-03T10:16:21.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.502+0000 7f31e170e700 1 -- 172.21.15.3:0/1055331182 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31dc0a0190 con 0x7f31dc09f240 2023-10-03T10:16:21.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.502+0000 7f31e170e700 1 -- 172.21.15.3:0/1055331182 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31dc06d430 con 0x7f31dc09f240 2023-10-03T10:16:21.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.502+0000 7f31c37fe700 1 -- 172.21.15.3:0/1055331182 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d001f070 con 0x7f31dc09f240 2023-10-03T10:16:21.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.502+0000 7f31c37fe700 1 -- 172.21.15.3:0/1055331182 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d00038a0 con 0x7f31dc09f240 2023-10-03T10:16:21.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.503+0000 7f31c37fe700 1 -- 172.21.15.3:0/1055331182 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f31d0003a00 con 0x7f31dc09f240 2023-10-03T10:16:21.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.503+0000 7f31c37fe700 1 -- 172.21.15.3:0/1055331182 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f31d0021070 con 0x7f31dc09f240 2023-10-03T10:16:21.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.504+0000 7f31e170e700 1 -- 172.21.15.3:0/1055331182 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31bc0052e0 con 0x7f31dc09f240 2023-10-03T10:16:21.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.513+0000 7f31c37fe700 1 -- 172.21.15.3:0/1055331182 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f31d001a950 con 0x7f31dc09f240 2023-10-03T10:16:21.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.568+0000 7f2b68641700 1 -- 172.21.15.3:0/1341567494 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2b60121c20 con 0x7f2b6009d6f0 2023-10-03T10:16:21.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.569+0000 7f2b56ffd700 1 -- 172.21.15.3:0/1341567494 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2b60121c20 con 0x7f2b6009d6f0 2023-10-03T10:16:21.569 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:21.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.572+0000 7f2b68641700 1 -- 172.21.15.3:0/1341567494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b6009d6f0 msgr2=0x7f2b6011e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:21.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.572+0000 7f2b68641700 1 --2- 172.21.15.3:0/1341567494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b6009d6f0 0x7f2b6011e8e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00c520 tx=0x7f2b5c00c830 comp rx=0 tx=0).stop 2023-10-03T10:16:21.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.573+0000 7f2b68641700 1 -- 172.21.15.3:0/1341567494 shutdown_connections 2023-10-03T10:16:21.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.573+0000 7f2b68641700 1 --2- 172.21.15.3:0/1341567494 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2b60139170 0x7f2b6011f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.573+0000 7f2b68641700 1 --2- 172.21.15.3:0/1341567494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b6009d6f0 0x7f2b6011e8e0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.573+0000 7f2b68641700 1 --2- 172.21.15.3:0/1341567494 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2b6009e1d0 0x7f2b6011ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.573+0000 7f2b68641700 1 -- 172.21.15.3:0/1341567494 >> 172.21.15.3:0/1341567494 conn(0x7f2b60066f50 msgr2=0x7f2b60136b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:21.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.573+0000 7f2b68641700 1 -- 172.21.15.3:0/1341567494 shutdown_connections 2023-10-03T10:16:21.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.573+0000 7f2b68641700 1 -- 172.21.15.3:0/1341567494 wait complete. 2023-10-03T10:16:21.587 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:21.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.702+0000 7fb97ce39700 1 -- 172.21.15.3:0/2558599562 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb978074ea0 con 0x7fb97813d1f0 2023-10-03T10:16:21.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.703+0000 7fb966ffd700 1 -- 172.21.15.3:0/2558599562 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb970007330 con 0x7fb97813d1f0 2023-10-03T10:16:21.703 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:21.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.706+0000 7fb97ce39700 1 -- 172.21.15.3:0/2558599562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb97813d1f0 msgr2=0x7fb978126b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:21.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.706+0000 7fb97ce39700 1 --2- 172.21.15.3:0/2558599562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb97813d1f0 0x7fb978126b20 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fb97000efc0 tx=0x7fb97000c3a0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.706+0000 7fb97ce39700 1 -- 172.21.15.3:0/2558599562 shutdown_connections 2023-10-03T10:16:21.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.706+0000 7fb97ce39700 1 --2- 172.21.15.3:0/2558599562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb97813d1f0 0x7fb978126b20 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.706+0000 7fb97ce39700 1 --2- 172.21.15.3:0/2558599562 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb97806a770 0x7fb9781260a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.706+0000 7fb97ce39700 1 --2- 172.21.15.3:0/2558599562 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb97809dc30 0x7fb9781265e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.706+0000 7fb97ce39700 1 -- 172.21.15.3:0/2558599562 >> 172.21.15.3:0/2558599562 conn(0x7fb978066f50 msgr2=0x7fb9780a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:21.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.706+0000 7fb97ce39700 1 -- 172.21.15.3:0/2558599562 shutdown_connections 2023-10-03T10:16:21.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.706+0000 7fb97ce39700 1 -- 172.21.15.3:0/2558599562 wait complete. 2023-10-03T10:16:21.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.717+0000 7f991db49700 1 -- 172.21.15.3:0/3425913555 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9918074ea0 con 0x7f9918131810 2023-10-03T10:16:21.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.717+0000 7f98fffff700 1 -- 172.21.15.3:0/3425913555 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f990801b3b0 con 0x7f9918131810 2023-10-03T10:16:21.718 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:21.720 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:21.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.721+0000 7f991db49700 1 -- 172.21.15.3:0/3425913555 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9918131810 msgr2=0x7f99181c6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:21.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.721+0000 7f991db49700 1 --2- 172.21.15.3:0/3425913555 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9918131810 0x7f99181c6550 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f990800c480 tx=0x7f990800c790 comp rx=0 tx=0).stop 2023-10-03T10:16:21.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.721+0000 7f991db49700 1 -- 172.21.15.3:0/3425913555 shutdown_connections 2023-10-03T10:16:21.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.721+0000 7f991db49700 1 --2- 172.21.15.3:0/3425913555 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9918136f50 0x7f9918118f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.721+0000 7f991db49700 1 --2- 172.21.15.3:0/3425913555 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9918131810 0x7f99181c6550 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.721+0000 7f991db49700 1 --2- 172.21.15.3:0/3425913555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9918134630 0x7f99181c6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.721+0000 7f991db49700 1 -- 172.21.15.3:0/3425913555 >> 172.21.15.3:0/3425913555 conn(0x7f99180a2aa0 msgr2=0x7f991812f930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:21.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.721+0000 7f991db49700 1 -- 172.21.15.3:0/3425913555 shutdown_connections 2023-10-03T10:16:21.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.721+0000 7f991db49700 1 -- 172.21.15.3:0/3425913555 wait complete. 2023-10-03T10:16:21.739 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:21.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.783+0000 7f31e170e700 1 -- 172.21.15.3:0/1055331182 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f31bc005590 con 0x7f31dc09f240 2023-10-03T10:16:21.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.783+0000 7f31c37fe700 1 -- 172.21.15.3:0/1055331182 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f31d0010b70 con 0x7f31dc09f240 2023-10-03T10:16:21.783 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:21.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.785+0000 7f31e170e700 1 -- 172.21.15.3:0/1055331182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31dc09f240 msgr2=0x7f31dc06bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:21.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.785+0000 7f31e170e700 1 --2- 172.21.15.3:0/1055331182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31dc09f240 0x7f31dc06bdf0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f31d0000c00 tx=0x7f31d0004750 comp rx=0 tx=0).stop 2023-10-03T10:16:21.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.786+0000 7f31e170e700 1 -- 172.21.15.3:0/1055331182 shutdown_connections 2023-10-03T10:16:21.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.786+0000 7f31e170e700 1 --2- 172.21.15.3:0/1055331182 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31dc09f650 0x7f31dc06c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.786+0000 7f31e170e700 1 --2- 172.21.15.3:0/1055331182 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31dc09e930 0x7f31dc06b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.786+0000 7f31e170e700 1 --2- 172.21.15.3:0/1055331182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31dc09f240 0x7f31dc06bdf0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:21.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.786+0000 7f31e170e700 1 -- 172.21.15.3:0/1055331182 >> 172.21.15.3:0/1055331182 conn(0x7f31dc0a4210 msgr2=0x7f31dc1b8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:21.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.786+0000 7f31e170e700 1 -- 172.21.15.3:0/1055331182 shutdown_connections 2023-10-03T10:16:21.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:21.786+0000 7f31e170e700 1 -- 172.21.15.3:0/1055331182 wait complete. 2023-10-03T10:16:21.799 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:22.589 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:22.721 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:22.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.734+0000 7f3bc77af700 1 Processor -- start 2023-10-03T10:16:22.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.734+0000 7f3bc77af700 1 -- start start 2023-10-03T10:16:22.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.735+0000 7f3bc77af700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3bc0136830 0x7f3bc0138c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:22.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.735+0000 7f3bc77af700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3bc0139150 0x7f3bc013b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:22.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.735+0000 7f3bc77af700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bc006f720 con 0x7f3bc01338c0 2023-10-03T10:16:22.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.735+0000 7f3bc77af700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc0070c80 con 0x7f3bc0136830 2023-10-03T10:16:22.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.735+0000 7f3bc77af700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc006fce0 con 0x7f3bc0139150 2023-10-03T10:16:22.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.735+0000 7f3bc5d4c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3bc0139150 0x7f3bc013b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:22.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.735+0000 7f3bc4d4a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3bc0136830 0x7f3bc0138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:22.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.735+0000 7f3bc5d4c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3bc0139150 0x7f3bc013b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44496/0 (socket says 172.21.15.3:44496) 2023-10-03T10:16:22.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.735+0000 7f3bc5d4c700 1 -- 172.21.15.3:0/1973940963 learned_addr learned my addr 172.21.15.3:0/1973940963 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:22.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.735+0000 7f3bc4d4a700 1 -- 172.21.15.3:0/1973940963 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3bc0139150 msgr2=0x7f3bc013b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:22.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.735+0000 7f3bc4d4a700 1 --2- 172.21.15.3:0/1973940963 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3bc0139150 0x7f3bc013b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.735+0000 7f3bc4d4a700 1 -- 172.21.15.3:0/1973940963 >> v1:172.21.15.3:6789/0 conn(0x7f3bc01338c0 legacy=0x7f3bc0133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:22.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.735+0000 7f3bc4d4a700 1 -- 172.21.15.3:0/1973940963 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc01336a0 con 0x7f3bc0136830 2023-10-03T10:16:22.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.735+0000 7f3bc5d4c700 1 --2- 172.21.15.3:0/1973940963 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3bc0139150 0x7f3bc013b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:22.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.735+0000 7f3bc4d4a700 1 --2- 172.21.15.3:0/1973940963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bc0136830 0x7f3bc0138c10 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009840 tx=0x7f3bb0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0d118f75c5de35 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:22.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.736+0000 7f3bb7fff700 1 -- 172.21.15.3:0/1973940963 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0011830 con 0x7f3bc0136830 2023-10-03T10:16:22.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.736+0000 7f3bb7fff700 1 -- 172.21.15.3:0/1973940963 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb0011990 con 0x7f3bc0136830 2023-10-03T10:16:22.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.736+0000 7f3bb7fff700 1 -- 172.21.15.3:0/1973940963 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0011d20 con 0x7f3bc0136830 2023-10-03T10:16:22.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.736+0000 7f3bc77af700 1 -- 172.21.15.3:0/1973940963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bc0136830 msgr2=0x7f3bc0138c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:22.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.736+0000 7f3bc77af700 1 --2- 172.21.15.3:0/1973940963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bc0136830 0x7f3bc0138c10 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009840 tx=0x7f3bb0009b50 comp rx=0 tx=0).stop 2023-10-03T10:16:22.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.736+0000 7f3bc77af700 1 -- 172.21.15.3:0/1973940963 shutdown_connections 2023-10-03T10:16:22.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.736+0000 7f3bc77af700 1 --2- 172.21.15.3:0/1973940963 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3bc0139150 0x7f3bc013b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.736+0000 7f3bc77af700 1 --2- 172.21.15.3:0/1973940963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bc0136830 0x7f3bc0138c10 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.736+0000 7f3bc77af700 1 -- 172.21.15.3:0/1973940963 >> 172.21.15.3:0/1973940963 conn(0x7f3bc00a2aa0 msgr2=0x7f3bc00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:22.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.737+0000 7f3bc77af700 1 -- 172.21.15.3:0/1973940963 shutdown_connections 2023-10-03T10:16:22.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.737+0000 7f3bc77af700 1 -- 172.21.15.3:0/1973940963 wait complete. 2023-10-03T10:16:22.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.738+0000 7f3bc77af700 1 Processor -- start 2023-10-03T10:16:22.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.738+0000 7f3bc77af700 1 -- start start 2023-10-03T10:16:22.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.738+0000 7f3bc77af700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3bc01338c0 0x7f3bc0119c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:22.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.738+0000 7f3bc77af700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3bc0136830 0x7f3bc011a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:22.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.738+0000 7f3bc77af700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bc0139150 0x7f3bc01c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:22.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.738+0000 7f3bc77af700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc011c9a0 con 0x7f3bc0139150 2023-10-03T10:16:22.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.738+0000 7f3bc77af700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc0119640 con 0x7f3bc01338c0 2023-10-03T10:16:22.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.738+0000 7f3bc77af700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc006f960 con 0x7f3bc0136830 2023-10-03T10:16:22.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.738+0000 7f3bc5d4c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bc0139150 0x7f3bc01c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:22.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.738+0000 7f3bc554b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3bc01338c0 0x7f3bc0119c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:22.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.738+0000 7f3bc5d4c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bc0139150 0x7f3bc01c85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50000/0 (socket says 172.21.15.3:50000) 2023-10-03T10:16:22.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.738+0000 7f3bc5d4c700 1 -- 172.21.15.3:0/1638776242 learned_addr learned my addr 172.21.15.3:0/1638776242 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:22.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.738+0000 7f3bc4d4a700 1 --2- 172.21.15.3:0/1638776242 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3bc0136830 0x7f3bc011a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:22.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.739+0000 7f3bc5d4c700 1 -- 172.21.15.3:0/1638776242 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3bc0136830 msgr2=0x7f3bc011a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:22.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.739+0000 7f3bc5d4c700 1 --2- 172.21.15.3:0/1638776242 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3bc0136830 0x7f3bc011a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.739+0000 7f3bc5d4c700 1 -- 172.21.15.3:0/1638776242 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3bc01338c0 msgr2=0x7f3bc0119c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:22.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.739+0000 7f3bc5d4c700 1 --2- 172.21.15.3:0/1638776242 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3bc01338c0 0x7f3bc0119c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.739+0000 7f3bc5d4c700 1 -- 172.21.15.3:0/1638776242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb8009b30 con 0x7f3bc0139150 2023-10-03T10:16:22.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.739+0000 7f3bc554b700 1 --2- 172.21.15.3:0/1638776242 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3bc01338c0 0x7f3bc0119c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:22.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.739+0000 7f3bc5d4c700 1 --2- 172.21.15.3:0/1638776242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bc0139150 0x7f3bc01c85c0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8009ef0 tx=0x7f3bb800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:22.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.739+0000 7f3bb5ffb700 1 -- 172.21.15.3:0/1638776242 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb800cb40 con 0x7f3bc0139150 2023-10-03T10:16:22.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.739+0000 7f3bb5ffb700 1 -- 172.21.15.3:0/1638776242 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb800f040 con 0x7f3bc0139150 2023-10-03T10:16:22.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.739+0000 7f3bc77af700 1 -- 172.21.15.3:0/1638776242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb00094a0 con 0x7f3bc0139150 2023-10-03T10:16:22.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.739+0000 7f3bb5ffb700 1 -- 172.21.15.3:0/1638776242 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb800cee0 con 0x7f3bc0139150 2023-10-03T10:16:22.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.739+0000 7f3bc77af700 1 -- 172.21.15.3:0/1638776242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc0136110 con 0x7f3bc0139150 2023-10-03T10:16:22.742 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:22.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.740+0000 7f3bb5ffb700 1 -- 172.21.15.3:0/1638776242 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3bb8014480 con 0x7f3bc0139150 2023-10-03T10:16:22.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.740+0000 7f3bb5ffb700 1 -- 172.21.15.3:0/1638776242 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3bb803fbe0 con 0x7f3bc0139150 2023-10-03T10:16:22.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.741+0000 7f3bc77af700 1 -- 172.21.15.3:0/1638776242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc0074ea0 con 0x7f3bc0139150 2023-10-03T10:16:22.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.746+0000 7f3bb5ffb700 1 -- 172.21.15.3:0/1638776242 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3bb8014910 con 0x7f3bc0139150 2023-10-03T10:16:22.800 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:22.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.865+0000 7fa00d5af700 1 Processor -- start 2023-10-03T10:16:22.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.865+0000 7fa00d5af700 1 -- start start 2023-10-03T10:16:22.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.866+0000 7fa00d5af700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa00806b250 0x7fa008136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:22.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.866+0000 7fa00d5af700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa008136d80 0x7fa008139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:22.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.866+0000 7fa00d5af700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa00809ece0 con 0x7fa00806a770 2023-10-03T10:16:22.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.866+0000 7fa00d5af700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00806b640 con 0x7fa008136d80 2023-10-03T10:16:22.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.866+0000 7fa00d5af700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa00809e760 con 0x7fa00806b250 2023-10-03T10:16:22.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.866+0000 7fa0067fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa00806b250 0x7fa008136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:22.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.866+0000 7fa0077fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa008136d80 0x7fa008139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:22.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.866+0000 7fa0067fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa00806b250 0x7fa008136840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:33338/0 (socket says 172.21.15.3:33338) 2023-10-03T10:16:22.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.866+0000 7fa0067fc700 1 -- 172.21.15.3:0/45178208 learned_addr learned my addr 172.21.15.3:0/45178208 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.866+0000 7fa005ffb700 1 -- 172.21.15.3:0/45178208 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1080326900 0 0) 0x7fa00809ece0 con 0x7fa00806a770 2023-10-03T10:16:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.866+0000 7fa005ffb700 1 -- 172.21.15.3:0/45178208 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ff00035b0 con 0x7fa00806a770 2023-10-03T10:16:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.866+0000 7fa0077fe700 1 -- 172.21.15.3:0/45178208 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa00806b250 msgr2=0x7fa008136840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.867+0000 7fa0077fe700 1 --2- 172.21.15.3:0/45178208 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa00806b250 0x7fa008136840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.867+0000 7fa0077fe700 1 -- 172.21.15.3:0/45178208 >> v1:172.21.15.3:6789/0 conn(0x7fa00806a770 legacy=0x7fa00806ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.867+0000 7fa0077fe700 1 -- 172.21.15.3:0/45178208 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00809e040 con 0x7fa008136d80 2023-10-03T10:16:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.867+0000 7fa0067fc700 1 --2- 172.21.15.3:0/45178208 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa00806b250 0x7fa008136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.867+0000 7fa0077fe700 1 --2- 172.21.15.3:0/45178208 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa008136d80 0x7fa008139160 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa000009dc0 tx=0x7fa00000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5a92ac6f3cdf03c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.867+0000 7fa005ffb700 1 -- 172.21.15.3:0/45178208 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00000e070 con 0x7fa008136d80 2023-10-03T10:16:22.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.867+0000 7fa005ffb700 1 -- 172.21.15.3:0/45178208 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00000bb20 con 0x7fa008136d80 2023-10-03T10:16:22.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.867+0000 7fa005ffb700 1 -- 172.21.15.3:0/45178208 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa000012430 con 0x7fa008136d80 2023-10-03T10:16:22.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.868+0000 7fa00d5af700 1 -- 172.21.15.3:0/45178208 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa008136d80 msgr2=0x7fa008139160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:22.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.868+0000 7fa00d5af700 1 --2- 172.21.15.3:0/45178208 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa008136d80 0x7fa008139160 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa000009dc0 tx=0x7fa00000b330 comp rx=0 tx=0).stop 2023-10-03T10:16:22.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.868+0000 7fa00d5af700 1 -- 172.21.15.3:0/45178208 shutdown_connections 2023-10-03T10:16:22.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.868+0000 7fa00d5af700 1 --2- 172.21.15.3:0/45178208 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa008136d80 0x7fa008139160 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.868+0000 7fa00d5af700 1 --2- 172.21.15.3:0/45178208 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa00806b250 0x7fa008136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.868+0000 7fa00d5af700 1 -- 172.21.15.3:0/45178208 >> 172.21.15.3:0/45178208 conn(0x7fa008066f50 msgr2=0x7fa00809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:22.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.868+0000 7fa00d5af700 1 -- 172.21.15.3:0/45178208 shutdown_connections 2023-10-03T10:16:22.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.868+0000 7fa00d5af700 1 -- 172.21.15.3:0/45178208 wait complete. 2023-10-03T10:16:22.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.870+0000 7fa00d5af700 1 Processor -- start 2023-10-03T10:16:22.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.870+0000 7fa00d5af700 1 -- start start 2023-10-03T10:16:22.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.870+0000 7fa00d5af700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa00806a770 0x7fa00809b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:22.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.870+0000 7fa00d5af700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa00806b250 0x7fa0081c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:22.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.870+0000 7fa00d5af700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa008136d80 0x7fa0081c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:22.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.870+0000 7fa00d5af700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0081350c0 con 0x7fa008136d80 2023-10-03T10:16:22.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.870+0000 7fa00d5af700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa008123fc0 con 0x7fa00806b250 2023-10-03T10:16:22.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.870+0000 7fa00d5af700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa00809aa90 con 0x7fa00806a770 2023-10-03T10:16:22.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.870+0000 7fa0067fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa00806b250 0x7fa0081c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:22.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.871+0000 7fa0067fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa00806b250 0x7fa0081c8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44528/0 (socket says 172.21.15.3:44528) 2023-10-03T10:16:22.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.871+0000 7fa0067fc700 1 -- 172.21.15.3:0/1313371645 learned_addr learned my addr 172.21.15.3:0/1313371645 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:22.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.871+0000 7fa006ffd700 1 --2- 172.21.15.3:0/1313371645 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa00806a770 0x7fa00809b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:22.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.871+0000 7fa0077fe700 1 --2- 172.21.15.3:0/1313371645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa008136d80 0x7fa0081c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.871+0000 7fa0067fc700 1 -- 172.21.15.3:0/1313371645 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa00806a770 msgr2=0x7fa00809b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.871+0000 7fa0067fc700 1 --2- 172.21.15.3:0/1313371645 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa00806a770 0x7fa00809b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.871+0000 7fa0067fc700 1 -- 172.21.15.3:0/1313371645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa008136d80 msgr2=0x7fa0081c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.871+0000 7fa0067fc700 1 --2- 172.21.15.3:0/1313371645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa008136d80 0x7fa0081c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.871+0000 7fa0067fc700 1 -- 172.21.15.3:0/1313371645 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ffc009550 con 0x7fa00806b250 2023-10-03T10:16:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.871+0000 7fa0067fc700 1 --2- 172.21.15.3:0/1313371645 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa00806b250 0x7fa0081c8000 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc00ebe0 tx=0x7f9ffc00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.871+0000 7f9fef7fe700 1 -- 172.21.15.3:0/1313371645 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc00c850 con 0x7fa00806b250 2023-10-03T10:16:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.872+0000 7f9fef7fe700 1 -- 172.21.15.3:0/1313371645 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ffc00c9b0 con 0x7fa00806b250 2023-10-03T10:16:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.872+0000 7f9fef7fe700 1 -- 172.21.15.3:0/1313371645 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc00cd40 con 0x7fa00806b250 2023-10-03T10:16:22.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.872+0000 7fa00d5af700 1 -- 172.21.15.3:0/1313371645 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa000009a70 con 0x7fa00806b250 2023-10-03T10:16:22.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.872+0000 7fa00d5af700 1 -- 172.21.15.3:0/1313371645 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0081259f0 con 0x7fa00806b250 2023-10-03T10:16:22.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.873+0000 7fa00d5af700 1 -- 172.21.15.3:0/1313371645 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa008074ea0 con 0x7fa00806b250 2023-10-03T10:16:22.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.873+0000 7f9fef7fe700 1 -- 172.21.15.3:0/1313371645 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9ffc013470 con 0x7fa00806b250 2023-10-03T10:16:22.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.873+0000 7f9fef7fe700 1 -- 172.21.15.3:0/1313371645 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9ffc03f980 con 0x7fa00806b250 2023-10-03T10:16:22.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.878+0000 7f9fef7fe700 1 -- 172.21.15.3:0/1313371645 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9ffc0232f0 con 0x7fa00806b250 2023-10-03T10:16:22.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.891+0000 7ff55416d700 1 Processor -- start 2023-10-03T10:16:22.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.891+0000 7ff55416d700 1 -- start start 2023-10-03T10:16:22.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.892+0000 7ff55416d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff54c09d6f0 0x7ff54c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:22.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.892+0000 7ff55416d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff54c09e000 0x7ff54c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:22.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.892+0000 7ff55416d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff54c09e410 0x7ff54c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:22.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.892+0000 7ff55416d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff54c09f520 con 0x7ff54c09e000 2023-10-03T10:16:22.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.892+0000 7ff55416d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c09f130 con 0x7ff54c09e410 2023-10-03T10:16:22.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.892+0000 7ff55416d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c09e930 con 0x7ff54c09d6f0 2023-10-03T10:16:22.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.892+0000 7ff551708700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff54c09e000 0x7ff54c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:22.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.892+0000 7ff551708700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff54c09e000 0x7ff54c130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50016/0 (socket says 172.21.15.3:50016) 2023-10-03T10:16:22.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.892+0000 7ff551708700 1 -- 172.21.15.3:0/1994736829 learned_addr learned my addr 172.21.15.3:0/1994736829 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:22.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.892+0000 7ff551f09700 1 --2- 172.21.15.3:0/1994736829 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff54c09d6f0 0x7ff54c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:22.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.892+0000 7ff55270a700 1 --2- 172.21.15.3:0/1994736829 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff54c09e410 0x7ff54c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:22.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.893+0000 7ff551708700 1 -- 172.21.15.3:0/1994736829 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff54c09d6f0 msgr2=0x7ff54c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:22.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.893+0000 7ff551708700 1 --2- 172.21.15.3:0/1994736829 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff54c09d6f0 0x7ff54c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.893+0000 7ff551708700 1 -- 172.21.15.3:0/1994736829 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff54c09e410 msgr2=0x7ff54c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:22.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.893+0000 7ff551708700 1 --2- 172.21.15.3:0/1994736829 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff54c09e410 0x7ff54c130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.893+0000 7ff551708700 1 -- 172.21.15.3:0/1994736829 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54c09b2d0 con 0x7ff54c09e000 2023-10-03T10:16:22.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.893+0000 7ff55270a700 1 --2- 172.21.15.3:0/1994736829 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff54c09e410 0x7ff54c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:16:22.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.893+0000 7ff551708700 1 --2- 172.21.15.3:0/1994736829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff54c09e000 0x7ff54c130660 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009920 tx=0x7ff53c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac73103cba9a4b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:22.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.893+0000 7ff550f07700 1 -- 172.21.15.3:0/1994736829 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c011830 con 0x7ff54c09e000 2023-10-03T10:16:22.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.893+0000 7ff550f07700 1 -- 172.21.15.3:0/1994736829 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53c011990 con 0x7ff54c09e000 2023-10-03T10:16:22.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.893+0000 7ff550f07700 1 -- 172.21.15.3:0/1994736829 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c011d20 con 0x7ff54c09e000 2023-10-03T10:16:22.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.894+0000 7ff55416d700 1 -- 172.21.15.3:0/1994736829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff54c09e000 msgr2=0x7ff54c130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:22.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.894+0000 7ff55416d700 1 --2- 172.21.15.3:0/1994736829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff54c09e000 0x7ff54c130660 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009920 tx=0x7ff53c009c30 comp rx=0 tx=0).stop 2023-10-03T10:16:22.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.894+0000 7ff55416d700 1 -- 172.21.15.3:0/1994736829 shutdown_connections 2023-10-03T10:16:22.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.894+0000 7ff55416d700 1 --2- 172.21.15.3:0/1994736829 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff54c09e410 0x7ff54c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.894+0000 7ff55416d700 1 --2- 172.21.15.3:0/1994736829 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff54c09d6f0 0x7ff54c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.894+0000 7ff55416d700 1 --2- 172.21.15.3:0/1994736829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff54c09e000 0x7ff54c130660 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.894+0000 7ff55416d700 1 -- 172.21.15.3:0/1994736829 >> 172.21.15.3:0/1994736829 conn(0x7ff54c066f50 msgr2=0x7ff54c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:22.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.894+0000 7ff55416d700 1 -- 172.21.15.3:0/1994736829 shutdown_connections 2023-10-03T10:16:22.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.894+0000 7ff55416d700 1 -- 172.21.15.3:0/1994736829 wait complete. 2023-10-03T10:16:22.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.896+0000 7ff55416d700 1 Processor -- start 2023-10-03T10:16:22.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.896+0000 7ff55416d700 1 -- start start 2023-10-03T10:16:22.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.897+0000 7ff55416d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff54c09d6f0 0x7ff54c1c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:22.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.897+0000 7ff55416d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff54c09e000 0x7ff54c1c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:22.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.897+0000 7ff55416d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff54c09e410 0x7ff54c131650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:22.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.897+0000 7ff55416d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c1c8210 con 0x7ff54c09d6f0 2023-10-03T10:16:22.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.897+0000 7ff55416d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c134170 con 0x7ff54c09e000 2023-10-03T10:16:22.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.897+0000 7ff55416d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c09c700 con 0x7ff54c09e410 2023-10-03T10:16:22.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.897+0000 7ff55270a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff54c09e410 0x7ff54c131650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:22.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.897+0000 7ff55270a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff54c09e410 0x7ff54c131650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:33374/0 (socket says 172.21.15.3:33374) 2023-10-03T10:16:22.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.897+0000 7ff55270a700 1 -- 172.21.15.3:0/78950240 learned_addr learned my addr 172.21.15.3:0/78950240 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:22.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.897+0000 7ff551f09700 1 --2- 172.21.15.3:0/78950240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff54c09d6f0 0x7ff54c1c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:22.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.897+0000 7ff551708700 1 --2- 172.21.15.3:0/78950240 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff54c09e000 0x7ff54c1c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:22.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.897+0000 7ff55270a700 1 -- 172.21.15.3:0/78950240 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff54c09e000 msgr2=0x7ff54c1c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:22.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.897+0000 7ff55270a700 1 --2- 172.21.15.3:0/78950240 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff54c09e000 0x7ff54c1c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.897+0000 7ff55270a700 1 -- 172.21.15.3:0/78950240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff54c09d6f0 msgr2=0x7ff54c1c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:22.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.897+0000 7ff55270a700 1 --2- 172.21.15.3:0/78950240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff54c09d6f0 0x7ff54c1c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.897+0000 7ff55270a700 1 -- 172.21.15.3:0/78950240 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53c009550 con 0x7ff54c09e410 2023-10-03T10:16:22.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.897+0000 7ff551f09700 1 --2- 172.21.15.3:0/78950240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff54c09d6f0 0x7ff54c1c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:22.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.898+0000 7ff55270a700 1 --2- 172.21.15.3:0/78950240 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff54c09e410 0x7ff54c131650 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff544007ef0 tx=0x7ff54400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:22.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.898+0000 7ff5427fc700 1 -- 172.21.15.3:0/78950240 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54400dd80 con 0x7ff54c09e410 2023-10-03T10:16:22.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.898+0000 7ff5427fc700 1 -- 172.21.15.3:0/78950240 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54400f040 con 0x7ff54c09e410 2023-10-03T10:16:22.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.898+0000 7ff5427fc700 1 -- 172.21.15.3:0/78950240 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff544003c60 con 0x7ff54c09e410 2023-10-03T10:16:22.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.898+0000 7ff55416d700 1 -- 172.21.15.3:0/78950240 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54c1c9050 con 0x7ff54c09e410 2023-10-03T10:16:22.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.898+0000 7ff55416d700 1 -- 172.21.15.3:0/78950240 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54c133470 con 0x7ff54c09e410 2023-10-03T10:16:22.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.899+0000 7ff5427fc700 1 -- 172.21.15.3:0/78950240 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff544004030 con 0x7ff54c09e410 2023-10-03T10:16:22.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.899+0000 7ff5427fc700 1 -- 172.21.15.3:0/78950240 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff54403f7f0 con 0x7ff54c09e410 2023-10-03T10:16:22.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.899+0000 7ff55416d700 1 -- 172.21.15.3:0/78950240 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff54c131090 con 0x7ff54c09e410 2023-10-03T10:16:22.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.905+0000 7ff5427fc700 1 -- 172.21.15.3:0/78950240 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff54401a020 con 0x7ff54c09e410 2023-10-03T10:16:22.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.964+0000 7f3bc77af700 1 -- 172.21.15.3:0/1638776242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3bc0072120 con 0x7f3bc0139150 2023-10-03T10:16:22.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.965+0000 7f3bb5ffb700 1 -- 172.21.15.3:0/1638776242 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3bb8012340 con 0x7f3bc0139150 2023-10-03T10:16:22.965 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:22.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.967+0000 7f3bc77af700 1 -- 172.21.15.3:0/1638776242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bc0139150 msgr2=0x7f3bc01c85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:22.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.967+0000 7f3bc77af700 1 --2- 172.21.15.3:0/1638776242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bc0139150 0x7f3bc01c85c0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8009ef0 tx=0x7f3bb800c3a0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.967+0000 7fab7c15c700 1 Processor -- start 2023-10-03T10:16:22.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.967+0000 7f3bc77af700 1 -- 172.21.15.3:0/1638776242 shutdown_connections 2023-10-03T10:16:22.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.967+0000 7f3bc77af700 1 --2- 172.21.15.3:0/1638776242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bc0139150 0x7f3bc01c85c0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.967+0000 7f3bc77af700 1 --2- 172.21.15.3:0/1638776242 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3bc01338c0 0x7f3bc0119c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.967+0000 7f3bc77af700 1 --2- 172.21.15.3:0/1638776242 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3bc0136830 0x7f3bc011a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.967+0000 7f3bc77af700 1 -- 172.21.15.3:0/1638776242 >> 172.21.15.3:0/1638776242 conn(0x7f3bc00a2aa0 msgr2=0x7f3bc012f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:22.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.967+0000 7f3bc77af700 1 -- 172.21.15.3:0/1638776242 shutdown_connections 2023-10-03T10:16:22.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.967+0000 7f3bc77af700 1 -- 172.21.15.3:0/1638776242 wait complete. 2023-10-03T10:16:22.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.967+0000 7fab7c15c700 1 -- start start 2023-10-03T10:16:22.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.968+0000 7fab7c15c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fab7409d6f0 0x7fab7409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:22.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.968+0000 7fab7c15c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab7409e000 0x7fab74130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:22.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.968+0000 7fab7c15c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab7409e410 0x7fab74130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:22.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.968+0000 7fab7c15c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab7409f520 con 0x7fab7409d6f0 2023-10-03T10:16:22.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.968+0000 7fab7c15c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab7409f130 con 0x7fab7409e000 2023-10-03T10:16:22.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.968+0000 7fab7c15c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fab7409e930 con 0x7fab7409e410 2023-10-03T10:16:22.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.968+0000 7fab796f7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab7409e000 0x7fab74130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:22.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.968+0000 7fab7a6f9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab7409e410 0x7fab74130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:22.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.968+0000 7fab796f7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab7409e000 0x7fab74130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44554/0 (socket says 172.21.15.3:44554) 2023-10-03T10:16:22.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.968+0000 7fab796f7700 1 -- 172.21.15.3:0/1373629290 learned_addr learned my addr 172.21.15.3:0/1373629290 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:22.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.968+0000 7fab79ef8700 1 --2- 172.21.15.3:0/1373629290 >> v2:172.21.15.3:3300/0 conn(0x7fab7409d6f0 0x7fab7409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:22.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.969+0000 7fab7a6f9700 1 -- 172.21.15.3:0/1373629290 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab7409e000 msgr2=0x7fab74130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:22.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.969+0000 7fab7a6f9700 1 --2- 172.21.15.3:0/1373629290 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab7409e000 0x7fab74130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.969+0000 7fab7a6f9700 1 -- 172.21.15.3:0/1373629290 >> v2:172.21.15.3:3300/0 conn(0x7fab7409d6f0 msgr2=0x7fab7409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:22.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.969+0000 7fab7a6f9700 1 --2- 172.21.15.3:0/1373629290 >> v2:172.21.15.3:3300/0 conn(0x7fab7409d6f0 0x7fab7409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.969+0000 7fab7a6f9700 1 -- 172.21.15.3:0/1373629290 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7409b2d0 con 0x7fab7409e410 2023-10-03T10:16:22.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.969+0000 7fab7a6f9700 1 --2- 172.21.15.3:0/1373629290 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab7409e410 0x7fab74130ba0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00be60 tx=0x7fab6c0092f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61f929035204109b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:22.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.969+0000 7fab78ef6700 1 -- 172.21.15.3:0/1373629290 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c00c850 con 0x7fab7409e410 2023-10-03T10:16:22.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.969+0000 7fab78ef6700 1 -- 172.21.15.3:0/1373629290 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6c00c9b0 con 0x7fab7409e410 2023-10-03T10:16:22.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.969+0000 7fab78ef6700 1 -- 172.21.15.3:0/1373629290 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c00cd60 con 0x7fab7409e410 2023-10-03T10:16:22.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.970+0000 7fab7c15c700 1 -- 172.21.15.3:0/1373629290 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab7409e410 msgr2=0x7fab74130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:22.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.970+0000 7fab7c15c700 1 --2- 172.21.15.3:0/1373629290 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab7409e410 0x7fab74130ba0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00be60 tx=0x7fab6c0092f0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.970+0000 7fab7c15c700 1 -- 172.21.15.3:0/1373629290 shutdown_connections 2023-10-03T10:16:22.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.970+0000 7fab7c15c700 1 --2- 172.21.15.3:0/1373629290 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab7409e410 0x7fab74130ba0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.970+0000 7fab7c15c700 1 --2- 172.21.15.3:0/1373629290 >> v2:172.21.15.3:3300/0 conn(0x7fab7409d6f0 0x7fab7409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.970+0000 7fab7c15c700 1 --2- 172.21.15.3:0/1373629290 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab7409e000 0x7fab74130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.970+0000 7fab7c15c700 1 -- 172.21.15.3:0/1373629290 >> 172.21.15.3:0/1373629290 conn(0x7fab74066f50 msgr2=0x7fab7409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:22.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.970+0000 7fab7c15c700 1 -- 172.21.15.3:0/1373629290 shutdown_connections 2023-10-03T10:16:22.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.970+0000 7fab7c15c700 1 -- 172.21.15.3:0/1373629290 wait complete. 2023-10-03T10:16:22.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.972+0000 7fab7c15c700 1 Processor -- start 2023-10-03T10:16:22.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.972+0000 7fab7c15c700 1 -- start start 2023-10-03T10:16:22.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.973+0000 7fab7c15c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab7409d6f0 0x7fab74127040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:22.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.973+0000 7fab7c15c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab7409e000 0x7fab74127580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:22.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.973+0000 7fab7c15c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab7409e410 0x7fab741cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:22.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.973+0000 7fab7c15c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7412a0b0 con 0x7fab7409e410 2023-10-03T10:16:22.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.973+0000 7fab7c15c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab7412a290 con 0x7fab7409d6f0 2023-10-03T10:16:22.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.973+0000 7fab7c15c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fab74126c50 con 0x7fab7409e000 2023-10-03T10:16:22.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.973+0000 7fab796f7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab7409e000 0x7fab74127580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:22.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.973+0000 7fab796f7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab7409e000 0x7fab74127580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:33394/0 (socket says 172.21.15.3:33394) 2023-10-03T10:16:22.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.973+0000 7fab796f7700 1 -- 172.21.15.3:0/1449417965 learned_addr learned my addr 172.21.15.3:0/1449417965 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:22.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.973+0000 7fab7a6f9700 1 --2- 172.21.15.3:0/1449417965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab7409e410 0x7fab741cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:22.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.974+0000 7fab7a6f9700 1 -- 172.21.15.3:0/1449417965 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab7409e000 msgr2=0x7fab74127580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:22.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.974+0000 7fab7a6f9700 1 --2- 172.21.15.3:0/1449417965 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab7409e000 0x7fab74127580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.974+0000 7fab7a6f9700 1 -- 172.21.15.3:0/1449417965 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab7409d6f0 msgr2=0x7fab74127040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:22.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.974+0000 7fab7a6f9700 1 --2- 172.21.15.3:0/1449417965 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab7409d6f0 0x7fab74127040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:22.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.974+0000 7fab7a6f9700 1 -- 172.21.15.3:0/1449417965 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c00ba90 con 0x7fab7409e410 2023-10-03T10:16:22.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.974+0000 7fab796f7700 1 --2- 172.21.15.3:0/1449417965 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab7409e000 0x7fab74127580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:22.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.974+0000 7fab7a6f9700 1 --2- 172.21.15.3:0/1449417965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab7409e410 0x7fab741cc470 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fab7409f6f0 tx=0x7fab6c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:22.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.974+0000 7fab6a7fc700 1 -- 172.21.15.3:0/1449417965 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c00c850 con 0x7fab7409e410 2023-10-03T10:16:22.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.974+0000 7fab6a7fc700 1 -- 172.21.15.3:0/1449417965 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6c009db0 con 0x7fab7409e410 2023-10-03T10:16:22.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.974+0000 7fab7c15c700 1 -- 172.21.15.3:0/1449417965 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab74129a40 con 0x7fab7409e410 2023-10-03T10:16:22.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.974+0000 7fab6a7fc700 1 -- 172.21.15.3:0/1449417965 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c01d650 con 0x7fab7409e410 2023-10-03T10:16:22.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.974+0000 7fab7c15c700 1 -- 172.21.15.3:0/1449417965 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab74128380 con 0x7fab7409e410 2023-10-03T10:16:22.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.976+0000 7fab7c15c700 1 -- 172.21.15.3:0/1449417965 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab74074ea0 con 0x7fab7409e410 2023-10-03T10:16:22.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.976+0000 7fab6a7fc700 1 -- 172.21.15.3:0/1449417965 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fab6c01a070 con 0x7fab7409e410 2023-10-03T10:16:22.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.976+0000 7fab6a7fc700 1 -- 172.21.15.3:0/1449417965 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fab6c041590 con 0x7fab7409e410 2023-10-03T10:16:22.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:22.984+0000 7fab6a7fc700 1 -- 172.21.15.3:0/1449417965 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fab6c01d7b0 con 0x7fab7409e410 2023-10-03T10:16:22.986 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:23.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.094+0000 7fa00d5af700 1 -- 172.21.15.3:0/1313371645 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa00809f6b0 con 0x7fa00806b250 2023-10-03T10:16:23.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.094+0000 7f9fef7fe700 1 -- 172.21.15.3:0/1313371645 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9ffc021090 con 0x7fa00806b250 2023-10-03T10:16:23.094 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:23.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.096+0000 7fa00d5af700 1 -- 172.21.15.3:0/1313371645 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa00806b250 msgr2=0x7fa0081c8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:23.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.096+0000 7fa00d5af700 1 --2- 172.21.15.3:0/1313371645 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa00806b250 0x7fa0081c8000 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc00ebe0 tx=0x7f9ffc00eef0 comp rx=0 tx=0).stop 2023-10-03T10:16:23.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.097+0000 7fa00d5af700 1 -- 172.21.15.3:0/1313371645 shutdown_connections 2023-10-03T10:16:23.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.097+0000 7fa00d5af700 1 --2- 172.21.15.3:0/1313371645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa008136d80 0x7fa0081c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:23.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.097+0000 7fa00d5af700 1 --2- 172.21.15.3:0/1313371645 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa00806a770 0x7fa00809b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:23.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.097+0000 7fa00d5af700 1 --2- 172.21.15.3:0/1313371645 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa00806b250 0x7fa0081c8000 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:23.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.097+0000 7fa00d5af700 1 -- 172.21.15.3:0/1313371645 >> 172.21.15.3:0/1313371645 conn(0x7fa008066f50 msgr2=0x7fa00806ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:23.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.097+0000 7fa00d5af700 1 -- 172.21.15.3:0/1313371645 shutdown_connections 2023-10-03T10:16:23.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.097+0000 7fa00d5af700 1 -- 172.21.15.3:0/1313371645 wait complete. 2023-10-03T10:16:23.111 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:23.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.141+0000 7ff55416d700 1 -- 172.21.15.3:0/78950240 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff54c074ea0 con 0x7ff54c09e410 2023-10-03T10:16:23.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.141+0000 7ff5427fc700 1 -- 172.21.15.3:0/78950240 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff54401e3b0 con 0x7ff54c09e410 2023-10-03T10:16:23.142 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:23.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.144+0000 7ff55416d700 1 -- 172.21.15.3:0/78950240 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff54c09e410 msgr2=0x7ff54c131650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:23.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.144+0000 7ff55416d700 1 --2- 172.21.15.3:0/78950240 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff54c09e410 0x7ff54c131650 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff544007ef0 tx=0x7ff54400d3b0 comp rx=0 tx=0).stop 2023-10-03T10:16:23.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.144+0000 7ff55416d700 1 -- 172.21.15.3:0/78950240 shutdown_connections 2023-10-03T10:16:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.144+0000 7ff55416d700 1 --2- 172.21.15.3:0/78950240 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff54c09e410 0x7ff54c131650 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.144+0000 7ff55416d700 1 --2- 172.21.15.3:0/78950240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff54c09d6f0 0x7ff54c1c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.144+0000 7ff55416d700 1 --2- 172.21.15.3:0/78950240 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff54c09e000 0x7ff54c1c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.144+0000 7ff55416d700 1 -- 172.21.15.3:0/78950240 >> 172.21.15.3:0/78950240 conn(0x7ff54c066f50 msgr2=0x7ff54c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.144+0000 7ff55416d700 1 -- 172.21.15.3:0/78950240 shutdown_connections 2023-10-03T10:16:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.144+0000 7ff55416d700 1 -- 172.21.15.3:0/78950240 wait complete. 2023-10-03T10:16:23.157 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:23.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.190+0000 7fab7c15c700 1 -- 172.21.15.3:0/1449417965 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fab7406c360 con 0x7fab7409e410 2023-10-03T10:16:23.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.190+0000 7fab6a7fc700 1 -- 172.21.15.3:0/1449417965 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fab6c02b030 con 0x7fab7409e410 2023-10-03T10:16:23.191 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:23.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.192+0000 7fab7c15c700 1 -- 172.21.15.3:0/1449417965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab7409e410 msgr2=0x7fab741cc470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:23.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.192+0000 7fab7c15c700 1 --2- 172.21.15.3:0/1449417965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab7409e410 0x7fab741cc470 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fab7409f6f0 tx=0x7fab6c004ad0 comp rx=0 tx=0).stop 2023-10-03T10:16:23.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.193+0000 7fab7c15c700 1 -- 172.21.15.3:0/1449417965 shutdown_connections 2023-10-03T10:16:23.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.193+0000 7fab7c15c700 1 --2- 172.21.15.3:0/1449417965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab7409e410 0x7fab741cc470 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:23.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.193+0000 7fab7c15c700 1 --2- 172.21.15.3:0/1449417965 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab7409d6f0 0x7fab74127040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:23.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.193+0000 7fab7c15c700 1 --2- 172.21.15.3:0/1449417965 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab7409e000 0x7fab74127580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:23.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.193+0000 7fab7c15c700 1 -- 172.21.15.3:0/1449417965 >> 172.21.15.3:0/1449417965 conn(0x7fab74066f50 msgr2=0x7fab74070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:23.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.193+0000 7fab7c15c700 1 -- 172.21.15.3:0/1449417965 shutdown_connections 2023-10-03T10:16:23.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:23.193+0000 7fab7c15c700 1 -- 172.21.15.3:0/1449417965 wait complete. 2023-10-03T10:16:23.206 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:23.988 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:24.112 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:24.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.133+0000 7f3cb41cf700 1 Processor -- start 2023-10-03T10:16:24.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.134+0000 7f3cb41cf700 1 -- start start 2023-10-03T10:16:24.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.134+0000 7f3cb41cf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3cac06a770 0x7f3cac06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.134+0000 7f3cb41cf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3cac06b080 0x7f3cac131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.134+0000 7f3cb41cf700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cac09ece0 con 0x7f3cac131b40 2023-10-03T10:16:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.134+0000 7f3cb41cf700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cac09e760 con 0x7f3cac06a770 2023-10-03T10:16:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.134+0000 7f3cb41cf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac09e040 con 0x7f3cac06b080 2023-10-03T10:16:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.134+0000 7f3cb176a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3cac06b080 0x7f3cac131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.134+0000 7f3cb1f6b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3cac06a770 0x7f3cac06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.134+0000 7f3cb176a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3cac06b080 0x7f3cac131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37640/0 (socket says 172.21.15.3:37640) 2023-10-03T10:16:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.134+0000 7f3cb1f6b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3cac06a770 0x7f3cac06ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42350/0 (socket says 172.21.15.3:42350) 2023-10-03T10:16:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.134+0000 7f3cb176a700 1 -- 172.21.15.3:0/1335452852 learned_addr learned my addr 172.21.15.3:0/1335452852 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.135+0000 7f3cb176a700 1 -- 172.21.15.3:0/1335452852 >> v2:172.21.15.3:3300/0 conn(0x7f3cac06a770 msgr2=0x7f3cac06ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.135+0000 7f3cb176a700 1 --2- 172.21.15.3:0/1335452852 >> v2:172.21.15.3:3300/0 conn(0x7f3cac06a770 0x7f3cac06ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.135+0000 7f3cb176a700 1 -- 172.21.15.3:0/1335452852 >> v1:172.21.15.3:6789/0 conn(0x7f3cac131b40 legacy=0x7f3cac138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:24.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.135+0000 7f3cb176a700 1 -- 172.21.15.3:0/1335452852 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cac09f630 con 0x7f3cac06b080 2023-10-03T10:16:24.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.135+0000 7f3cb0f69700 1 -- 172.21.15.3:0/1335452852 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1678735812 0 0) 0x7f3cac09ece0 con 0x7f3cac131b40 2023-10-03T10:16:24.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.135+0000 7f3cb1f6b700 1 --2- 172.21.15.3:0/1335452852 >> v2:172.21.15.3:3300/0 conn(0x7f3cac06a770 0x7f3cac06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:24.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.135+0000 7f3cb176a700 1 --2- 172.21.15.3:0/1335452852 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3cac06b080 0x7f3cac131600 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3ca00098f0 tx=0x7f3ca0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5ddfe80eb6b0027 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:24.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.135+0000 7f3cb0f69700 1 -- 172.21.15.3:0/1335452852 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0011830 con 0x7f3cac06b080 2023-10-03T10:16:24.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.135+0000 7f3cb0f69700 1 -- 172.21.15.3:0/1335452852 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca0011990 con 0x7f3cac06b080 2023-10-03T10:16:24.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.135+0000 7f3cb0f69700 1 -- 172.21.15.3:0/1335452852 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0011d20 con 0x7f3cac06b080 2023-10-03T10:16:24.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.136+0000 7f3cb41cf700 1 -- 172.21.15.3:0/1335452852 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3cac06b080 msgr2=0x7f3cac131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:24.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.136+0000 7f3cb41cf700 1 --2- 172.21.15.3:0/1335452852 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3cac06b080 0x7f3cac131600 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3ca00098f0 tx=0x7f3ca0009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:24.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.136+0000 7f3cb41cf700 1 -- 172.21.15.3:0/1335452852 shutdown_connections 2023-10-03T10:16:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.136+0000 7f3cb41cf700 1 --2- 172.21.15.3:0/1335452852 >> v2:172.21.15.3:3300/0 conn(0x7f3cac06a770 0x7f3cac06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.136+0000 7f3cb41cf700 1 --2- 172.21.15.3:0/1335452852 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3cac06b080 0x7f3cac131600 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.136+0000 7f3cb41cf700 1 -- 172.21.15.3:0/1335452852 >> 172.21.15.3:0/1335452852 conn(0x7f3cac066f50 msgr2=0x7f3cac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.136+0000 7f3cb41cf700 1 -- 172.21.15.3:0/1335452852 shutdown_connections 2023-10-03T10:16:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.136+0000 7f3cb41cf700 1 -- 172.21.15.3:0/1335452852 wait complete. 2023-10-03T10:16:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.137+0000 7f3cb41cf700 1 Processor -- start 2023-10-03T10:16:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.137+0000 7f3cb41cf700 1 -- start start 2023-10-03T10:16:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.138+0000 7f3cb41cf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cac06a770 0x7f3cac1c5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.138+0000 7f3cb41cf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3cac06b080 0x7f3cac1c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.138+0000 7f3cb41cf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3cac131b40 0x7f3cac1c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.138+0000 7f3cb41cf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac1c5c10 con 0x7f3cac06a770 2023-10-03T10:16:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.138+0000 7f3cb41cf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac09c010 con 0x7f3cac06b080 2023-10-03T10:16:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.138+0000 7f3cb41cf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac125c40 con 0x7f3cac131b40 2023-10-03T10:16:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.138+0000 7f3cb1f6b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cac06a770 0x7f3cac1c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.138+0000 7f3cb176a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3cac06b080 0x7f3cac1c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.138+0000 7f3cb1f6b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cac06a770 0x7f3cac1c5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42366/0 (socket says 172.21.15.3:42366) 2023-10-03T10:16:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.138+0000 7f3cb176a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3cac06b080 0x7f3cac1c56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37654/0 (socket says 172.21.15.3:37654) 2023-10-03T10:16:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.138+0000 7f3cb1f6b700 1 -- 172.21.15.3:0/429663128 learned_addr learned my addr 172.21.15.3:0/429663128 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.138+0000 7f3cb276c700 1 --2- 172.21.15.3:0/429663128 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3cac131b40 0x7f3cac1c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.138+0000 7f3cb176a700 1 -- 172.21.15.3:0/429663128 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3cac131b40 msgr2=0x7f3cac1c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.138+0000 7f3cb176a700 1 --2- 172.21.15.3:0/429663128 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3cac131b40 0x7f3cac1c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.138+0000 7f3cb176a700 1 -- 172.21.15.3:0/429663128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cac06a770 msgr2=0x7f3cac1c5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.138+0000 7f3cb176a700 1 --2- 172.21.15.3:0/429663128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cac06a770 0x7f3cac1c5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.138+0000 7f3cb176a700 1 -- 172.21.15.3:0/429663128 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca80094a0 con 0x7f3cac06b080 2023-10-03T10:16:24.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.138+0000 7f3cb276c700 1 --2- 172.21.15.3:0/429663128 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3cac131b40 0x7f3cac1c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:24.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.138+0000 7f3cb1f6b700 1 --2- 172.21.15.3:0/429663128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cac06a770 0x7f3cac1c5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:24.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.138+0000 7f3cb176a700 1 --2- 172.21.15.3:0/429663128 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3cac06b080 0x7f3cac1c56d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0016000 tx=0x7f3ca0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:24.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.139+0000 7f3c9e7fc700 1 -- 172.21.15.3:0/429663128 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0004300 con 0x7f3cac06b080 2023-10-03T10:16:24.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.139+0000 7f3c9e7fc700 1 -- 172.21.15.3:0/429663128 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca001f070 con 0x7f3cac06b080 2023-10-03T10:16:24.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.139+0000 7f3cb41cf700 1 -- 172.21.15.3:0/429663128 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca0009550 con 0x7f3cac06b080 2023-10-03T10:16:24.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.139+0000 7f3c9e7fc700 1 -- 172.21.15.3:0/429663128 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca00036a0 con 0x7f3cac06b080 2023-10-03T10:16:24.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.139+0000 7f3cb41cf700 1 -- 172.21.15.3:0/429663128 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cac1315e0 con 0x7f3cac06b080 2023-10-03T10:16:24.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.140+0000 7f3c9e7fc700 1 -- 172.21.15.3:0/429663128 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3ca0021030 con 0x7f3cac06b080 2023-10-03T10:16:24.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.140+0000 7f3c9e7fc700 1 -- 172.21.15.3:0/429663128 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3ca00408b0 con 0x7f3cac06b080 2023-10-03T10:16:24.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.140+0000 7f3cb41cf700 1 -- 172.21.15.3:0/429663128 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cac074ea0 con 0x7f3cac06b080 2023-10-03T10:16:24.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.145+0000 7f3c9e7fc700 1 -- 172.21.15.3:0/429663128 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3ca0015070 con 0x7f3cac06b080 2023-10-03T10:16:24.158 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:24.207 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:24.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.265+0000 7f62d55c1700 1 Processor -- start 2023-10-03T10:16:24.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.265+0000 7f62d55c1700 1 -- start start 2023-10-03T10:16:24.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.265+0000 7f62d55c1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f62d0133420 0x7f62d01337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:24.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.265+0000 7f62d55c1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f62d0133e30 0x7f62d012ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:24.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.265+0000 7f62d55c1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f62d0134260 0x7f62d01304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:24.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.265+0000 7f62d55c1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d006b990 con 0x7f62d0133420 2023-10-03T10:16:24.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.265+0000 7f62d55c1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d00a0590 con 0x7f62d0133e30 2023-10-03T10:16:24.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.265+0000 7f62d55c1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f62d009fb80 con 0x7f62d0134260 2023-10-03T10:16:24.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.266+0000 7f62ce7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f62d0133e30 0x7f62d012ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:24.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.266+0000 7f62ceffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f62d0133420 0x7f62d01337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:24.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.266+0000 7f62ce7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f62d0133e30 0x7f62d012ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37670/0 (socket says 172.21.15.3:37670) 2023-10-03T10:16:24.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.266+0000 7f62ceffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f62d0133420 0x7f62d01337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42382/0 (socket says 172.21.15.3:42382) 2023-10-03T10:16:24.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.266+0000 7f62ce7fc700 1 -- 172.21.15.3:0/1617052994 learned_addr learned my addr 172.21.15.3:0/1617052994 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:24.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.266+0000 7f62cf7fe700 1 --2- 172.21.15.3:0/1617052994 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f62d0134260 0x7f62d01304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:24.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.266+0000 7f62ceffd700 1 -- 172.21.15.3:0/1617052994 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f62d0134260 msgr2=0x7f62d01304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:24.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.266+0000 7f62ceffd700 1 --2- 172.21.15.3:0/1617052994 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f62d0134260 0x7f62d01304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.266+0000 7f62ceffd700 1 -- 172.21.15.3:0/1617052994 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f62d0133e30 msgr2=0x7f62d012ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:24.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.266+0000 7f62ceffd700 1 --2- 172.21.15.3:0/1617052994 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f62d0133e30 0x7f62d012ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.266+0000 7f62ceffd700 1 -- 172.21.15.3:0/1617052994 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d00a0030 con 0x7f62d0133420 2023-10-03T10:16:24.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.266+0000 7f62ceffd700 1 --2- 172.21.15.3:0/1617052994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62d0133420 0x7f62d01337f0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009840 tx=0x7f62c0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d7006f7b5f40012 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:24.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.267+0000 7f62cdffb700 1 -- 172.21.15.3:0/1617052994 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c0011830 con 0x7f62d0133420 2023-10-03T10:16:24.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.267+0000 7f62cdffb700 1 -- 172.21.15.3:0/1617052994 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c0011990 con 0x7f62d0133420 2023-10-03T10:16:24.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.267+0000 7f62cdffb700 1 -- 172.21.15.3:0/1617052994 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c0011d20 con 0x7f62d0133420 2023-10-03T10:16:24.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.267+0000 7f62d55c1700 1 -- 172.21.15.3:0/1617052994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62d0133420 msgr2=0x7f62d01337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:24.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.267+0000 7f62d55c1700 1 --2- 172.21.15.3:0/1617052994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62d0133420 0x7f62d01337f0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009840 tx=0x7f62c0009b50 comp rx=0 tx=0).stop 2023-10-03T10:16:24.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.268+0000 7f62d55c1700 1 -- 172.21.15.3:0/1617052994 shutdown_connections 2023-10-03T10:16:24.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.268+0000 7f62d55c1700 1 --2- 172.21.15.3:0/1617052994 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f62d0134260 0x7f62d01304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.268+0000 7f62d55c1700 1 --2- 172.21.15.3:0/1617052994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62d0133420 0x7f62d01337f0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.268+0000 7f62d55c1700 1 --2- 172.21.15.3:0/1617052994 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f62d0133e30 0x7f62d012ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.268+0000 7f62d55c1700 1 -- 172.21.15.3:0/1617052994 >> 172.21.15.3:0/1617052994 conn(0x7f62d00a4210 msgr2=0x7f62d00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:24.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.268+0000 7f62d55c1700 1 -- 172.21.15.3:0/1617052994 shutdown_connections 2023-10-03T10:16:24.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.268+0000 7f62d55c1700 1 -- 172.21.15.3:0/1617052994 wait complete. 2023-10-03T10:16:24.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.269+0000 7f62d55c1700 1 Processor -- start 2023-10-03T10:16:24.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.269+0000 7f62d55c1700 1 -- start start 2023-10-03T10:16:24.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.269+0000 7f62d55c1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62d0133420 0x7f62d009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:24.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.269+0000 7f62d55c1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f62d0133e30 0x7f62d009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:24.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.269+0000 7f62d55c1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f62d0134260 0x7f62d009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:24.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.269+0000 7f62d55c1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d0131470 con 0x7f62d0133420 2023-10-03T10:16:24.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.269+0000 7f62d55c1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d0120170 con 0x7f62d0133e30 2023-10-03T10:16:24.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.269+0000 7f62d55c1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f62d009d950 con 0x7f62d0134260 2023-10-03T10:16:24.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.270+0000 7f62ceffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62d0133420 0x7f62d009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:24.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.270+0000 7f62ceffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62d0133420 0x7f62d009c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42398/0 (socket says 172.21.15.3:42398) 2023-10-03T10:16:24.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.270+0000 7f62ceffd700 1 -- 172.21.15.3:0/2681296292 learned_addr learned my addr 172.21.15.3:0/2681296292 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:24.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.270+0000 7f62cf7fe700 1 --2- 172.21.15.3:0/2681296292 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f62d0134260 0x7f62d009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:24.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.270+0000 7f62ce7fc700 1 --2- 172.21.15.3:0/2681296292 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f62d0133e30 0x7f62d009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:24.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.270+0000 7f62ceffd700 1 -- 172.21.15.3:0/2681296292 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f62d0134260 msgr2=0x7f62d009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:24.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.270+0000 7f62ceffd700 1 --2- 172.21.15.3:0/2681296292 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f62d0134260 0x7f62d009cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.270+0000 7f62ceffd700 1 -- 172.21.15.3:0/2681296292 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f62d0133e30 msgr2=0x7f62d009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:24.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.270+0000 7f62ceffd700 1 --2- 172.21.15.3:0/2681296292 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f62d0133e30 0x7f62d009c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.270+0000 7f62ceffd700 1 -- 172.21.15.3:0/2681296292 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c00094a0 con 0x7f62d0133420 2023-10-03T10:16:24.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.270+0000 7f62ceffd700 1 --2- 172.21.15.3:0/2681296292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62d0133420 0x7f62d009c480 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009fd0 tx=0x7f62c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:24.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.271+0000 7f62b77fe700 1 -- 172.21.15.3:0/2681296292 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c0004240 con 0x7f62d0133420 2023-10-03T10:16:24.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.271+0000 7f62b77fe700 1 -- 172.21.15.3:0/2681296292 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c001f070 con 0x7f62d0133420 2023-10-03T10:16:24.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.271+0000 7f62b77fe700 1 -- 172.21.15.3:0/2681296292 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c00036a0 con 0x7f62d0133420 2023-10-03T10:16:24.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.271+0000 7f62d55c1700 1 -- 172.21.15.3:0/2681296292 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d0120810 con 0x7f62d0133420 2023-10-03T10:16:24.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.271+0000 7f62d55c1700 1 -- 172.21.15.3:0/2681296292 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d011fc30 con 0x7f62d0133420 2023-10-03T10:16:24.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.272+0000 7f62d55c1700 1 -- 172.21.15.3:0/2681296292 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d009dfe0 con 0x7f62d0133420 2023-10-03T10:16:24.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.272+0000 7f62b77fe700 1 -- 172.21.15.3:0/2681296292 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f62c0021030 con 0x7f62d0133420 2023-10-03T10:16:24.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.272+0000 7f62b77fe700 1 -- 172.21.15.3:0/2681296292 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f62c00408a0 con 0x7f62d0133420 2023-10-03T10:16:24.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.281+0000 7f62b77fe700 1 -- 172.21.15.3:0/2681296292 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f62c00239c0 con 0x7f62d0133420 2023-10-03T10:16:24.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.325+0000 7fe6b7478700 1 Processor -- start 2023-10-03T10:16:24.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.326+0000 7fe6b7478700 1 -- start start 2023-10-03T10:16:24.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.326+0000 7fe6b7478700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6b009d6f0 0x7fe6b009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:24.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.326+0000 7fe6b7478700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe6b009e410 0x7fe6b0130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:24.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.326+0000 7fe6b7478700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b009f130 con 0x7fe6b009e000 2023-10-03T10:16:24.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.326+0000 7fe6b7478700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b009f520 con 0x7fe6b009e410 2023-10-03T10:16:24.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.326+0000 7fe6b7478700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b009e930 con 0x7fe6b009d6f0 2023-10-03T10:16:24.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.327+0000 7fe6b5214700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6b009d6f0 0x7fe6b009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:24.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.327+0000 7fe6b5a15700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe6b009e410 0x7fe6b0130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:24.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.327+0000 7fe6b4a13700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe6b009e000 0x7fe6b0130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35348/0 (socket says 172.21.15.3:35348) 2023-10-03T10:16:24.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.327+0000 7fe6b4a13700 1 -- 172.21.15.3:0/1744114225 learned_addr learned my addr 172.21.15.3:0/1744114225 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:24.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.327+0000 7fe6b5a15700 1 -- 172.21.15.3:0/1744114225 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6b009d6f0 msgr2=0x7fe6b009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:24.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.327+0000 7fe6b5a15700 1 --2- 172.21.15.3:0/1744114225 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6b009d6f0 0x7fe6b009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.327+0000 7fe6b5a15700 1 -- 172.21.15.3:0/1744114225 >> v1:172.21.15.3:6789/0 conn(0x7fe6b009e000 legacy=0x7fe6b0130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:24.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.327+0000 7fe6a7fff700 1 -- 172.21.15.3:0/1744114225 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 684790782 0 0) 0x7fe6b009f130 con 0x7fe6b009e000 2023-10-03T10:16:24.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.327+0000 7fe6b5a15700 1 -- 172.21.15.3:0/1744114225 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b009b2d0 con 0x7fe6b009e410 2023-10-03T10:16:24.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.327+0000 7fe6b5a15700 1 --2- 172.21.15.3:0/1744114225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6b009e410 0x7fe6b0130d70 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8009d90 tx=0x7fe6a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cf3f5d8956d513 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:24.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.327+0000 7fe6a7fff700 1 -- 172.21.15.3:0/1744114225 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a800e070 con 0x7fe6b009e410 2023-10-03T10:16:24.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.328+0000 7fe6a7fff700 1 -- 172.21.15.3:0/1744114225 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a800bb20 con 0x7fe6b009e410 2023-10-03T10:16:24.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.328+0000 7fe6a7fff700 1 -- 172.21.15.3:0/1744114225 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8012430 con 0x7fe6b009e410 2023-10-03T10:16:24.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.328+0000 7fe6b7478700 1 -- 172.21.15.3:0/1744114225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6b009e410 msgr2=0x7fe6b0130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:24.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.328+0000 7fe6b7478700 1 --2- 172.21.15.3:0/1744114225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6b009e410 0x7fe6b0130d70 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8009d90 tx=0x7fe6a800b330 comp rx=0 tx=0).stop 2023-10-03T10:16:24.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.328+0000 7fe6b7478700 1 -- 172.21.15.3:0/1744114225 shutdown_connections 2023-10-03T10:16:24.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.328+0000 7fe6b7478700 1 --2- 172.21.15.3:0/1744114225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6b009e410 0x7fe6b0130d70 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.328+0000 7fe6b7478700 1 --2- 172.21.15.3:0/1744114225 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6b009d6f0 0x7fe6b009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.328+0000 7fe6b7478700 1 -- 172.21.15.3:0/1744114225 >> 172.21.15.3:0/1744114225 conn(0x7fe6b0066f50 msgr2=0x7fe6b009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:24.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.328+0000 7fe6b7478700 1 -- 172.21.15.3:0/1744114225 shutdown_connections 2023-10-03T10:16:24.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.328+0000 7fe6b7478700 1 -- 172.21.15.3:0/1744114225 wait complete. 2023-10-03T10:16:24.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.330+0000 7fe6b7478700 1 Processor -- start 2023-10-03T10:16:24.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.330+0000 7fe6b7478700 1 -- start start 2023-10-03T10:16:24.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.331+0000 7fe6b7478700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6b009d6f0 0x7fe6b011f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:24.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.331+0000 7fe6b7478700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6b009e000 0x7fe6b011f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:24.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.331+0000 7fe6b7478700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe6b009e410 0x7fe6b011fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:24.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.331+0000 7fe6b7478700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b0122a30 con 0x7fe6b009d6f0 2023-10-03T10:16:24.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.331+0000 7fe6b7478700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b0120f10 con 0x7fe6b009e000 2023-10-03T10:16:24.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.331+0000 7fe6b7478700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b0121640 con 0x7fe6b009e410 2023-10-03T10:16:24.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.331+0000 7fe6b4a13700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6b009e000 0x7fe6b011f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:24.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.331+0000 7fe6b4a13700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6b009e000 0x7fe6b011f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37696/0 (socket says 172.21.15.3:37696) 2023-10-03T10:16:24.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.331+0000 7fe6b4a13700 1 -- 172.21.15.3:0/1319429270 learned_addr learned my addr 172.21.15.3:0/1319429270 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:24.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.331+0000 7fe6b5a15700 1 --2- 172.21.15.3:0/1319429270 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe6b009e410 0x7fe6b011fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:24.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.331+0000 7fe6b5214700 1 --2- 172.21.15.3:0/1319429270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6b009d6f0 0x7fe6b011f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:24.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.331+0000 7fe6b4a13700 1 -- 172.21.15.3:0/1319429270 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe6b009e410 msgr2=0x7fe6b011fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:24.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.332+0000 7fe6b4a13700 1 --2- 172.21.15.3:0/1319429270 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe6b009e410 0x7fe6b011fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.332+0000 7fe6b4a13700 1 -- 172.21.15.3:0/1319429270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6b009d6f0 msgr2=0x7fe6b011f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:24.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.332+0000 7fe6b4a13700 1 --2- 172.21.15.3:0/1319429270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6b009d6f0 0x7fe6b011f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.332+0000 7fe6b4a13700 1 -- 172.21.15.3:0/1319429270 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a8009a70 con 0x7fe6b009e000 2023-10-03T10:16:24.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.332+0000 7fe6b4a13700 1 --2- 172.21.15.3:0/1319429270 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6b009e000 0x7fe6b011f730 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe6a000c520 tx=0x7fe6a000c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:24.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.332+0000 7fe6a5ffb700 1 -- 172.21.15.3:0/1319429270 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0004c80 con 0x7fe6b009e000 2023-10-03T10:16:24.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.332+0000 7fe6a5ffb700 1 -- 172.21.15.3:0/1319429270 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a0015070 con 0x7fe6b009e000 2023-10-03T10:16:24.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.332+0000 7fe6b7478700 1 -- 172.21.15.3:0/1319429270 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b0072640 con 0x7fe6b009e000 2023-10-03T10:16:24.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.332+0000 7fe6a5ffb700 1 -- 172.21.15.3:0/1319429270 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a00070d0 con 0x7fe6b009e000 2023-10-03T10:16:24.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.332+0000 7fe6b7478700 1 -- 172.21.15.3:0/1319429270 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b01234a0 con 0x7fe6b009e000 2023-10-03T10:16:24.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.333+0000 7fe6a5ffb700 1 -- 172.21.15.3:0/1319429270 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe6a0007230 con 0x7fe6b009e000 2023-10-03T10:16:24.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.334+0000 7fe6a5ffb700 1 -- 172.21.15.3:0/1319429270 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe6a0045030 con 0x7fe6b009e000 2023-10-03T10:16:24.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.334+0000 7fe6b7478700 1 -- 172.21.15.3:0/1319429270 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b0074ea0 con 0x7fe6b009e000 2023-10-03T10:16:24.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.343+0000 7fe6a5ffb700 1 -- 172.21.15.3:0/1319429270 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe6a0018c70 con 0x7fe6b009e000 2023-10-03T10:16:24.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.355+0000 7fb987f09700 1 Processor -- start 2023-10-03T10:16:24.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.355+0000 7fb987f09700 1 -- start start 2023-10-03T10:16:24.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.356+0000 7fb987f09700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb98009f410 0x7fb980136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:24.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.356+0000 7fb987f09700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb980136fe0 0x7fb9801393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:24.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.356+0000 7fb987f09700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb980070940 con 0x7fb98009e930 2023-10-03T10:16:24.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.356+0000 7fb987f09700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb98006fd60 con 0x7fb98009f410 2023-10-03T10:16:24.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.356+0000 7fb987f09700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb98006fa80 con 0x7fb980136fe0 2023-10-03T10:16:24.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.356+0000 7fb9854a4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb98009f410 0x7fb980136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:24.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.356+0000 7fb985ca5700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb98009e930 0x7fb98009ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35358/0 (socket says 172.21.15.3:35358) 2023-10-03T10:16:24.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.356+0000 7fb985ca5700 1 -- 172.21.15.3:0/2617172209 learned_addr learned my addr 172.21.15.3:0/2617172209 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:24.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.356+0000 7fb9854a4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb98009f410 0x7fb980136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42438/0 (socket says 172.21.15.3:42438) 2023-10-03T10:16:24.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.356+0000 7fb9864a6700 1 --2- 172.21.15.3:0/2617172209 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb980136fe0 0x7fb9801393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:24.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.356+0000 7fb9854a4700 1 -- 172.21.15.3:0/2617172209 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb980136fe0 msgr2=0x7fb9801393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:24.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.356+0000 7fb9854a4700 1 --2- 172.21.15.3:0/2617172209 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb980136fe0 0x7fb9801393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.356+0000 7fb9854a4700 1 -- 172.21.15.3:0/2617172209 >> v1:172.21.15.3:6789/0 conn(0x7fb98009e930 legacy=0x7fb98009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:24.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.357+0000 7fb9854a4700 1 -- 172.21.15.3:0/2617172209 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98009fb60 con 0x7fb98009f410 2023-10-03T10:16:24.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.357+0000 7fb9854a4700 1 --2- 172.21.15.3:0/2617172209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb98009f410 0x7fb980136aa0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fb970009840 tx=0x7fb970009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc290652a2521cd server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:24.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.357+0000 7fb984ca3700 1 -- 172.21.15.3:0/2617172209 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb970011830 con 0x7fb98009f410 2023-10-03T10:16:24.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.357+0000 7fb984ca3700 1 -- 172.21.15.3:0/2617172209 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb970011990 con 0x7fb98009f410 2023-10-03T10:16:24.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.357+0000 7fb984ca3700 1 -- 172.21.15.3:0/2617172209 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb970011d20 con 0x7fb98009f410 2023-10-03T10:16:24.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.358+0000 7fb987f09700 1 -- 172.21.15.3:0/2617172209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb98009f410 msgr2=0x7fb980136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:24.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.358+0000 7fb987f09700 1 --2- 172.21.15.3:0/2617172209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb98009f410 0x7fb980136aa0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fb970009840 tx=0x7fb970009b50 comp rx=0 tx=0).stop 2023-10-03T10:16:24.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.358+0000 7fb987f09700 1 -- 172.21.15.3:0/2617172209 shutdown_connections 2023-10-03T10:16:24.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.358+0000 7fb987f09700 1 --2- 172.21.15.3:0/2617172209 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb980136fe0 0x7fb9801393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.358+0000 7fb987f09700 1 --2- 172.21.15.3:0/2617172209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb98009f410 0x7fb980136aa0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.358+0000 7fb987f09700 1 -- 172.21.15.3:0/2617172209 >> 172.21.15.3:0/2617172209 conn(0x7fb9800a4210 msgr2=0x7fb9800a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:24.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.358+0000 7fb987f09700 1 -- 172.21.15.3:0/2617172209 shutdown_connections 2023-10-03T10:16:24.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.358+0000 7fb987f09700 1 -- 172.21.15.3:0/2617172209 wait complete. 2023-10-03T10:16:24.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.359+0000 7fb987f09700 1 Processor -- start 2023-10-03T10:16:24.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.359+0000 7fb987f09700 1 -- start start 2023-10-03T10:16:24.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.360+0000 7fb987f09700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb98009e930 0x7fb980131a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:24.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.360+0000 7fb987f09700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb98009f410 0x7fb980131fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:24.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.360+0000 7fb987f09700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb980136fe0 0x7fb980132510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:24.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.360+0000 7fb987f09700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb980135480 con 0x7fb98009f410 2023-10-03T10:16:24.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.360+0000 7fb987f09700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9801347c0 con 0x7fb98009e930 2023-10-03T10:16:24.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.360+0000 7fb987f09700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb98009b220 con 0x7fb980136fe0 2023-10-03T10:16:24.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.360+0000 7fb985ca5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb98009e930 0x7fb980131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:24.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.360+0000 7fb985ca5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb98009e930 0x7fb980131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37704/0 (socket says 172.21.15.3:37704) 2023-10-03T10:16:24.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.360+0000 7fb985ca5700 1 -- 172.21.15.3:0/1428674967 learned_addr learned my addr 172.21.15.3:0/1428674967 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:24.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.360+0000 7fb9864a6700 1 --2- 172.21.15.3:0/1428674967 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb980136fe0 0x7fb980132510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:24.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.360+0000 7fb9854a4700 1 --2- 172.21.15.3:0/1428674967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb98009f410 0x7fb980131fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:24.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.361+0000 7fb985ca5700 1 -- 172.21.15.3:0/1428674967 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb980136fe0 msgr2=0x7fb980132510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:24.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.361+0000 7fb985ca5700 1 --2- 172.21.15.3:0/1428674967 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb980136fe0 0x7fb980132510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.361+0000 7fb985ca5700 1 -- 172.21.15.3:0/1428674967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb98009f410 msgr2=0x7fb980131fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:24.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.361+0000 7fb985ca5700 1 --2- 172.21.15.3:0/1428674967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb98009f410 0x7fb980131fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.361+0000 7fb985ca5700 1 -- 172.21.15.3:0/1428674967 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9700094a0 con 0x7fb98009e930 2023-10-03T10:16:24.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.361+0000 7fb985ca5700 1 --2- 172.21.15.3:0/1428674967 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb98009e930 0x7fb980131a90 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00c520 tx=0x7fb97c00c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:24.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.361+0000 7fb9767fc700 1 -- 172.21.15.3:0/1428674967 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97c005d70 con 0x7fb98009e930 2023-10-03T10:16:24.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.361+0000 7fb9767fc700 1 -- 172.21.15.3:0/1428674967 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97c0070d0 con 0x7fb98009e930 2023-10-03T10:16:24.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.361+0000 7fb987f09700 1 -- 172.21.15.3:0/1428674967 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb980133dd0 con 0x7fb98009e930 2023-10-03T10:16:24.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.361+0000 7fb9767fc700 1 -- 172.21.15.3:0/1428674967 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97c007440 con 0x7fb98009e930 2023-10-03T10:16:24.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.361+0000 7fb987f09700 1 -- 172.21.15.3:0/1428674967 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb980134c10 con 0x7fb98009e930 2023-10-03T10:16:24.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.362+0000 7fb9767fc700 1 -- 172.21.15.3:0/1428674967 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb97c0075a0 con 0x7fb98009e930 2023-10-03T10:16:24.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.362+0000 7fb987f09700 1 -- 172.21.15.3:0/1428674967 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb980074ea0 con 0x7fb98009e930 2023-10-03T10:16:24.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.362+0000 7fb9767fc700 1 -- 172.21.15.3:0/1428674967 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb97c03eb30 con 0x7fb98009e930 2023-10-03T10:16:24.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.367+0000 7fb9767fc700 1 -- 172.21.15.3:0/1428674967 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb97c017020 con 0x7fb98009e930 2023-10-03T10:16:24.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.371+0000 7f3cb41cf700 1 -- 172.21.15.3:0/429663128 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3cac126070 con 0x7f3cac06b080 2023-10-03T10:16:24.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.372+0000 7f3c9e7fc700 1 -- 172.21.15.3:0/429663128 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3ca00239c0 con 0x7f3cac06b080 2023-10-03T10:16:24.372 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:24.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.374+0000 7f3cb41cf700 1 -- 172.21.15.3:0/429663128 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3cac06b080 msgr2=0x7f3cac1c56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:24.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.374+0000 7f3cb41cf700 1 --2- 172.21.15.3:0/429663128 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3cac06b080 0x7f3cac1c56d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0016000 tx=0x7f3ca0004750 comp rx=0 tx=0).stop 2023-10-03T10:16:24.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.374+0000 7f3cb41cf700 1 -- 172.21.15.3:0/429663128 shutdown_connections 2023-10-03T10:16:24.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.374+0000 7f3cb41cf700 1 --2- 172.21.15.3:0/429663128 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3cac131b40 0x7f3cac1c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.374+0000 7f3cb41cf700 1 --2- 172.21.15.3:0/429663128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cac06a770 0x7f3cac1c5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.374+0000 7f3cb41cf700 1 --2- 172.21.15.3:0/429663128 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3cac06b080 0x7f3cac1c56d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.374+0000 7f3cb41cf700 1 -- 172.21.15.3:0/429663128 >> 172.21.15.3:0/429663128 conn(0x7f3cac066f50 msgr2=0x7f3cac06ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:24.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.377+0000 7f3cb41cf700 1 -- 172.21.15.3:0/429663128 shutdown_connections 2023-10-03T10:16:24.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.378+0000 7f3cb41cf700 1 -- 172.21.15.3:0/429663128 wait complete. 2023-10-03T10:16:24.392 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.535+0000 7f62d55c1700 1 -- 172.21.15.3:0/2681296292 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f62d0074ea0 con 0x7f62d0133420 2023-10-03T10:16:24.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.536+0000 7f62b77fe700 1 -- 172.21.15.3:0/2681296292 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f62c00233d0 con 0x7f62d0133420 2023-10-03T10:16:24.536 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:24.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.538+0000 7f62d55c1700 1 -- 172.21.15.3:0/2681296292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62d0133420 msgr2=0x7f62d009c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:24.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.538+0000 7f62d55c1700 1 --2- 172.21.15.3:0/2681296292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62d0133420 0x7f62d009c480 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009fd0 tx=0x7f62c0004750 comp rx=0 tx=0).stop 2023-10-03T10:16:24.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.538+0000 7f62d55c1700 1 -- 172.21.15.3:0/2681296292 shutdown_connections 2023-10-03T10:16:24.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.538+0000 7f62d55c1700 1 --2- 172.21.15.3:0/2681296292 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f62d0134260 0x7f62d009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.538+0000 7f62d55c1700 1 --2- 172.21.15.3:0/2681296292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62d0133420 0x7f62d009c480 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.538+0000 7f62d55c1700 1 --2- 172.21.15.3:0/2681296292 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f62d0133e30 0x7f62d009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.538+0000 7f62d55c1700 1 -- 172.21.15.3:0/2681296292 >> 172.21.15.3:0/2681296292 conn(0x7f62d00a4210 msgr2=0x7f62d0130250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:24.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.538+0000 7f62d55c1700 1 -- 172.21.15.3:0/2681296292 shutdown_connections 2023-10-03T10:16:24.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.538+0000 7f62d55c1700 1 -- 172.21.15.3:0/2681296292 wait complete. 2023-10-03T10:16:24.552 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:24.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.575+0000 7fb987f09700 1 -- 172.21.15.3:0/1428674967 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb98006d870 con 0x7fb98009e930 2023-10-03T10:16:24.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.576+0000 7fb9767fc700 1 -- 172.21.15.3:0/1428674967 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb97c019b40 con 0x7fb98009e930 2023-10-03T10:16:24.576 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:24.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.578+0000 7fb987f09700 1 -- 172.21.15.3:0/1428674967 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb98009e930 msgr2=0x7fb980131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:24.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.578+0000 7fb987f09700 1 --2- 172.21.15.3:0/1428674967 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb98009e930 0x7fb980131a90 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00c520 tx=0x7fb97c00c830 comp rx=0 tx=0).stop 2023-10-03T10:16:24.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.578+0000 7fb987f09700 1 -- 172.21.15.3:0/1428674967 shutdown_connections 2023-10-03T10:16:24.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.578+0000 7fb987f09700 1 --2- 172.21.15.3:0/1428674967 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb980136fe0 0x7fb980132510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.578+0000 7fb987f09700 1 --2- 172.21.15.3:0/1428674967 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb98009e930 0x7fb980131a90 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.578+0000 7fb987f09700 1 --2- 172.21.15.3:0/1428674967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb98009f410 0x7fb980131fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.578+0000 7fb987f09700 1 -- 172.21.15.3:0/1428674967 >> 172.21.15.3:0/1428674967 conn(0x7fb9800a4210 msgr2=0x7fb98006ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:24.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.578+0000 7fb987f09700 1 -- 172.21.15.3:0/1428674967 shutdown_connections 2023-10-03T10:16:24.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.578+0000 7fb987f09700 1 -- 172.21.15.3:0/1428674967 wait complete. 2023-10-03T10:16:24.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.582+0000 7fe6b7478700 1 -- 172.21.15.3:0/1319429270 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe6b00720c0 con 0x7fe6b009e000 2023-10-03T10:16:24.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.583+0000 7fe6a5ffb700 1 -- 172.21.15.3:0/1319429270 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe6a0018a90 con 0x7fe6b009e000 2023-10-03T10:16:24.583 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:24.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.585+0000 7fe6b7478700 1 -- 172.21.15.3:0/1319429270 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6b009e000 msgr2=0x7fe6b011f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:24.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.585+0000 7fe6b7478700 1 --2- 172.21.15.3:0/1319429270 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6b009e000 0x7fe6b011f730 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe6a000c520 tx=0x7fe6a000c830 comp rx=0 tx=0).stop 2023-10-03T10:16:24.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.585+0000 7fe6b7478700 1 -- 172.21.15.3:0/1319429270 shutdown_connections 2023-10-03T10:16:24.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.585+0000 7fe6b7478700 1 --2- 172.21.15.3:0/1319429270 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe6b009e410 0x7fe6b011fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.585+0000 7fe6b7478700 1 --2- 172.21.15.3:0/1319429270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6b009d6f0 0x7fe6b011f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.585+0000 7fe6b7478700 1 --2- 172.21.15.3:0/1319429270 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6b009e000 0x7fe6b011f730 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:24.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.585+0000 7fe6b7478700 1 -- 172.21.15.3:0/1319429270 >> 172.21.15.3:0/1319429270 conn(0x7fe6b0066f50 msgr2=0x7fe6b00a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:24.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.585+0000 7fe6b7478700 1 -- 172.21.15.3:0/1319429270 shutdown_connections 2023-10-03T10:16:24.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:24.585+0000 7fe6b7478700 1 -- 172.21.15.3:0/1319429270 wait complete. 2023-10-03T10:16:24.593 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:24.599 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:25.394 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:25.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.541+0000 7f1e7c70c700 1 Processor -- start 2023-10-03T10:16:25.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.541+0000 7f1e7c70c700 1 -- start start 2023-10-03T10:16:25.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.541+0000 7f1e7c70c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1e7409d6f0 0x7f1e7409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:25.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.541+0000 7f1e7c70c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e7409e000 0x7f1e741339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:25.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.541+0000 7f1e7c70c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7409b120 con 0x7f1e7409e410 2023-10-03T10:16:25.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.541+0000 7f1e7c70c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e7406b460 con 0x7f1e7409d6f0 2023-10-03T10:16:25.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.541+0000 7f1e7c70c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7409a7e0 con 0x7f1e7409e000 2023-10-03T10:16:25.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.541+0000 7f1e79ca7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e7409e000 0x7f1e741339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:25.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.542+0000 7f1e79ca7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e7409e000 0x7f1e741339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37710/0 (socket says 172.21.15.3:37710) 2023-10-03T10:16:25.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.542+0000 7f1e79ca7700 1 -- 172.21.15.3:0/2942799408 learned_addr learned my addr 172.21.15.3:0/2942799408 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:25.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.542+0000 7f1e7a4a8700 1 --2- 172.21.15.3:0/2942799408 >> v2:172.21.15.3:3300/0 conn(0x7f1e7409d6f0 0x7f1e7409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:25.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.542+0000 7f1e79ca7700 1 -- 172.21.15.3:0/2942799408 >> v2:172.21.15.3:3300/0 conn(0x7f1e7409d6f0 msgr2=0x7f1e7409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:25.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.542+0000 7f1e79ca7700 1 --2- 172.21.15.3:0/2942799408 >> v2:172.21.15.3:3300/0 conn(0x7f1e7409d6f0 0x7f1e7409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.542+0000 7f1e79ca7700 1 -- 172.21.15.3:0/2942799408 >> v1:172.21.15.3:6789/0 conn(0x7f1e7409e410 legacy=0x7f1e74133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:25.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.542+0000 7f1e79ca7700 1 -- 172.21.15.3:0/2942799408 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e7409a500 con 0x7f1e7409e000 2023-10-03T10:16:25.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.542+0000 7f1e79ca7700 1 --2- 172.21.15.3:0/2942799408 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e7409e000 0x7f1e741339f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f1e64009920 tx=0x7f1e64009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4de0db699347bf7a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:25.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.542+0000 7f1e794a6700 1 -- 172.21.15.3:0/2942799408 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e64011830 con 0x7f1e7409e000 2023-10-03T10:16:25.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.542+0000 7f1e794a6700 1 -- 172.21.15.3:0/2942799408 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e64011990 con 0x7f1e7409e000 2023-10-03T10:16:25.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.542+0000 7f1e794a6700 1 -- 172.21.15.3:0/2942799408 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e64011d20 con 0x7f1e7409e000 2023-10-03T10:16:25.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.543+0000 7f1e7c70c700 1 -- 172.21.15.3:0/2942799408 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e7409e000 msgr2=0x7f1e741339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:25.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.543+0000 7f1e7c70c700 1 --2- 172.21.15.3:0/2942799408 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e7409e000 0x7f1e741339f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f1e64009920 tx=0x7f1e64009c30 comp rx=0 tx=0).stop 2023-10-03T10:16:25.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.543+0000 7f1e7c70c700 1 -- 172.21.15.3:0/2942799408 shutdown_connections 2023-10-03T10:16:25.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.543+0000 7f1e7c70c700 1 --2- 172.21.15.3:0/2942799408 >> v2:172.21.15.3:3300/0 conn(0x7f1e7409d6f0 0x7f1e7409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.543+0000 7f1e7c70c700 1 --2- 172.21.15.3:0/2942799408 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e7409e000 0x7f1e741339f0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.543+0000 7f1e7c70c700 1 -- 172.21.15.3:0/2942799408 >> 172.21.15.3:0/2942799408 conn(0x7f1e74066f50 msgr2=0x7f1e7409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:25.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.543+0000 7f1e7c70c700 1 -- 172.21.15.3:0/2942799408 shutdown_connections 2023-10-03T10:16:25.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.543+0000 7f1e7c70c700 1 -- 172.21.15.3:0/2942799408 wait complete. 2023-10-03T10:16:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.544+0000 7f1e7c70c700 1 Processor -- start 2023-10-03T10:16:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.544+0000 7f1e7c70c700 1 -- start start 2023-10-03T10:16:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.544+0000 7f1e7c70c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e7409d6f0 0x7f1e741a4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.544+0000 7f1e7c70c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e7409e000 0x7f1e741a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.544+0000 7f1e7c70c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e7409e410 0x7f1e741a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.544+0000 7f1e7c70c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e741a8720 con 0x7f1e7409e410 2023-10-03T10:16:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.544+0000 7f1e7c70c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e741a6330 con 0x7f1e7409d6f0 2023-10-03T10:16:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.544+0000 7f1e7c70c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e740a3240 con 0x7f1e7409e000 2023-10-03T10:16:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.544+0000 7f1e7a4a8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e7409d6f0 0x7f1e741a4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.544+0000 7f1e7a4a8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e7409d6f0 0x7f1e741a4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37724/0 (socket says 172.21.15.3:37724) 2023-10-03T10:16:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.545+0000 7f1e7a4a8700 1 -- 172.21.15.3:0/4290430581 learned_addr learned my addr 172.21.15.3:0/4290430581 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.545+0000 7f1e7aca9700 1 --2- 172.21.15.3:0/4290430581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e7409e410 0x7f1e741a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.545+0000 7f1e79ca7700 1 --2- 172.21.15.3:0/4290430581 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e7409e000 0x7f1e741a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.545+0000 7f1e7a4a8700 1 -- 172.21.15.3:0/4290430581 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e7409e000 msgr2=0x7f1e741a54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.545+0000 7f1e7a4a8700 1 --2- 172.21.15.3:0/4290430581 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e7409e000 0x7f1e741a54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.545+0000 7f1e7a4a8700 1 -- 172.21.15.3:0/4290430581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e7409e410 msgr2=0x7f1e741a59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:25.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.545+0000 7f1e7a4a8700 1 --2- 172.21.15.3:0/4290430581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e7409e410 0x7f1e741a59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.545+0000 7f1e7a4a8700 1 -- 172.21.15.3:0/4290430581 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e64009550 con 0x7f1e7409d6f0 2023-10-03T10:16:25.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.545+0000 7f1e7a4a8700 1 --2- 172.21.15.3:0/4290430581 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e7409d6f0 0x7f1e741a4f60 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000e8a0 tx=0x7f1e7000ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:25.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.545+0000 7f1e6affd700 1 -- 172.21.15.3:0/4290430581 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e70009590 con 0x7f1e7409d6f0 2023-10-03T10:16:25.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.545+0000 7f1e6affd700 1 -- 172.21.15.3:0/4290430581 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e70013680 con 0x7f1e7409d6f0 2023-10-03T10:16:25.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.545+0000 7f1e7c70c700 1 -- 172.21.15.3:0/4290430581 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e741a6c00 con 0x7f1e7409d6f0 2023-10-03T10:16:25.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.545+0000 7f1e6affd700 1 -- 172.21.15.3:0/4290430581 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e700139f0 con 0x7f1e7409d6f0 2023-10-03T10:16:25.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.545+0000 7f1e7c70c700 1 -- 172.21.15.3:0/4290430581 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e741a7d40 con 0x7f1e7409d6f0 2023-10-03T10:16:25.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.546+0000 7f1e6affd700 1 -- 172.21.15.3:0/4290430581 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1e70013b50 con 0x7f1e7409d6f0 2023-10-03T10:16:25.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.546+0000 7f1e7c70c700 1 -- 172.21.15.3:0/4290430581 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e74074ea0 con 0x7f1e7409d6f0 2023-10-03T10:16:25.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.546+0000 7f1e6affd700 1 -- 172.21.15.3:0/4290430581 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1e7003f960 con 0x7f1e7409d6f0 2023-10-03T10:16:25.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.551+0000 7f1e6affd700 1 -- 172.21.15.3:0/4290430581 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1e7001adc0 con 0x7f1e7409d6f0 2023-10-03T10:16:25.553 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:25.594 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:25.600 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:25.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.696+0000 7f59d98d8700 1 Processor -- start 2023-10-03T10:16:25.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.697+0000 7f59d98d8700 1 -- start start 2023-10-03T10:16:25.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.697+0000 7f59d98d8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59d409d6f0 0x7f59d409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:25.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.697+0000 7f59d98d8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f59d409e000 0x7f59d4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:25.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.697+0000 7f59d98d8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59d409e410 0x7f59d4130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:25.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.697+0000 7f59d98d8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d409f520 con 0x7f59d409e000 2023-10-03T10:16:25.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.697+0000 7f59d98d8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d409f130 con 0x7f59d409d6f0 2023-10-03T10:16:25.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.697+0000 7f59d98d8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f59d409e930 con 0x7f59d409e410 2023-10-03T10:16:25.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.697+0000 7f59d37fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59d409e410 0x7f59d4130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:25.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.697+0000 7f59d37fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59d409e410 0x7f59d4130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51386/0 (socket says 172.21.15.3:51386) 2023-10-03T10:16:25.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.697+0000 7f59d37fe700 1 -- 172.21.15.3:0/1046399665 learned_addr learned my addr 172.21.15.3:0/1046399665 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:25.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.697+0000 7f59d27fc700 1 --2- 172.21.15.3:0/1046399665 >> v2:172.21.15.3:3300/0 conn(0x7f59d409e000 0x7f59d4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:25.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.697+0000 7f59d2ffd700 1 --2- 172.21.15.3:0/1046399665 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59d409d6f0 0x7f59d409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:25.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.698+0000 7f59d37fe700 1 -- 172.21.15.3:0/1046399665 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59d409d6f0 msgr2=0x7f59d409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:25.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.698+0000 7f59d37fe700 1 --2- 172.21.15.3:0/1046399665 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59d409d6f0 0x7f59d409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.698+0000 7f59d37fe700 1 -- 172.21.15.3:0/1046399665 >> v2:172.21.15.3:3300/0 conn(0x7f59d409e000 msgr2=0x7f59d4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:25.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.698+0000 7f59d37fe700 1 --2- 172.21.15.3:0/1046399665 >> v2:172.21.15.3:3300/0 conn(0x7f59d409e000 0x7f59d4130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.698+0000 7f59d37fe700 1 -- 172.21.15.3:0/1046399665 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d409b2d0 con 0x7f59d409e410 2023-10-03T10:16:25.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.698+0000 7f59d27fc700 1 --2- 172.21.15.3:0/1046399665 >> v2:172.21.15.3:3300/0 conn(0x7f59d409e000 0x7f59d4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:25.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.698+0000 7f59d37fe700 1 --2- 172.21.15.3:0/1046399665 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59d409e410 0x7f59d4130ba0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009e80 tx=0x7f59cc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcab7009c3e75634 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:25.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.698+0000 7f59d1ffb700 1 -- 172.21.15.3:0/1046399665 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc00bad0 con 0x7f59d409e410 2023-10-03T10:16:25.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.698+0000 7f59d1ffb700 1 -- 172.21.15.3:0/1046399665 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59cc00bc30 con 0x7f59d409e410 2023-10-03T10:16:25.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.698+0000 7f59d1ffb700 1 -- 172.21.15.3:0/1046399665 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc011460 con 0x7f59d409e410 2023-10-03T10:16:25.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.699+0000 7f59d98d8700 1 -- 172.21.15.3:0/1046399665 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59d409e410 msgr2=0x7f59d4130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:25.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.699+0000 7f59d98d8700 1 --2- 172.21.15.3:0/1046399665 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59d409e410 0x7f59d4130ba0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009e80 tx=0x7f59cc00b330 comp rx=0 tx=0).stop 2023-10-03T10:16:25.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.699+0000 7f59d98d8700 1 -- 172.21.15.3:0/1046399665 shutdown_connections 2023-10-03T10:16:25.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.699+0000 7f59d98d8700 1 --2- 172.21.15.3:0/1046399665 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59d409e410 0x7f59d4130ba0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.699+0000 7f59d98d8700 1 --2- 172.21.15.3:0/1046399665 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59d409d6f0 0x7f59d409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.699+0000 7f59d98d8700 1 --2- 172.21.15.3:0/1046399665 >> v2:172.21.15.3:3300/0 conn(0x7f59d409e000 0x7f59d4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.699+0000 7f59d98d8700 1 -- 172.21.15.3:0/1046399665 >> 172.21.15.3:0/1046399665 conn(0x7f59d4066f50 msgr2=0x7f59d409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:25.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.699+0000 7f59d98d8700 1 -- 172.21.15.3:0/1046399665 shutdown_connections 2023-10-03T10:16:25.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.699+0000 7f59d98d8700 1 -- 172.21.15.3:0/1046399665 wait complete. 2023-10-03T10:16:25.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.700+0000 7f59d98d8700 1 Processor -- start 2023-10-03T10:16:25.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.701+0000 7f59d98d8700 1 -- start start 2023-10-03T10:16:25.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.701+0000 7f59d98d8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59d409d6f0 0x7f59d4122180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:25.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.701+0000 7f59d98d8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59d409e000 0x7f59d41226c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:25.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.701+0000 7f59d98d8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59d409e410 0x7f59d4122c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:25.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.701+0000 7f59d98d8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d41243b0 con 0x7f59d409e410 2023-10-03T10:16:25.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.701+0000 7f59d98d8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d409c700 con 0x7f59d409e000 2023-10-03T10:16:25.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.701+0000 7f59d98d8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f59d40a16b0 con 0x7f59d409d6f0 2023-10-03T10:16:25.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.702+0000 7f59d2ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59d409d6f0 0x7f59d4122180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:25.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.702+0000 7f59d2ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59d409d6f0 0x7f59d4122180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51396/0 (socket says 172.21.15.3:51396) 2023-10-03T10:16:25.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.702+0000 7f59d2ffd700 1 -- 172.21.15.3:0/3089424369 learned_addr learned my addr 172.21.15.3:0/3089424369 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:25.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.702+0000 7f59d37fe700 1 --2- 172.21.15.3:0/3089424369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59d409e410 0x7f59d4122c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:25.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.702+0000 7f59d27fc700 1 --2- 172.21.15.3:0/3089424369 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59d409e000 0x7f59d41226c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:25.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.702+0000 7f59d37fe700 1 -- 172.21.15.3:0/3089424369 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59d409d6f0 msgr2=0x7f59d4122180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:25.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.702+0000 7f59d37fe700 1 --2- 172.21.15.3:0/3089424369 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59d409d6f0 0x7f59d4122180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.702+0000 7f59d37fe700 1 -- 172.21.15.3:0/3089424369 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59d409e000 msgr2=0x7f59d41226c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:25.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.702+0000 7f59d37fe700 1 --2- 172.21.15.3:0/3089424369 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59d409e000 0x7f59d41226c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.702+0000 7f59d37fe700 1 -- 172.21.15.3:0/3089424369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c80094a0 con 0x7f59d409e410 2023-10-03T10:16:25.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.702+0000 7f59d37fe700 1 --2- 172.21.15.3:0/3089424369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59d409e410 0x7f59d4122c00 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f59cc017000 tx=0x7f59cc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:25.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.703+0000 7f59bbfff700 1 -- 172.21.15.3:0/3089424369 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc004800 con 0x7f59d409e410 2023-10-03T10:16:25.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.703+0000 7f59bbfff700 1 -- 172.21.15.3:0/3089424369 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59cc0092e0 con 0x7f59d409e410 2023-10-03T10:16:25.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.703+0000 7f59bbfff700 1 -- 172.21.15.3:0/3089424369 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc0116a0 con 0x7f59d409e410 2023-10-03T10:16:25.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.703+0000 7f59d98d8700 1 -- 172.21.15.3:0/3089424369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59cc009b30 con 0x7f59d409e410 2023-10-03T10:16:25.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.703+0000 7f59d98d8700 1 -- 172.21.15.3:0/3089424369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d4124580 con 0x7f59d409e410 2023-10-03T10:16:25.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.704+0000 7f59d98d8700 1 -- 172.21.15.3:0/3089424369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59d4125660 con 0x7f59d409e410 2023-10-03T10:16:25.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.704+0000 7f59bbfff700 1 -- 172.21.15.3:0/3089424369 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f59cc011800 con 0x7f59d409e410 2023-10-03T10:16:25.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.705+0000 7f59bbfff700 1 -- 172.21.15.3:0/3089424369 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f59cc0411e0 con 0x7f59d409e410 2023-10-03T10:16:25.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.710+0000 7f59bbfff700 1 -- 172.21.15.3:0/3089424369 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f59d4125660 con 0x7f59d409e410 2023-10-03T10:16:25.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.742+0000 7fa94c368700 1 Processor -- start 2023-10-03T10:16:25.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.742+0000 7fa94c368700 1 -- start start 2023-10-03T10:16:25.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.743+0000 7fa94c368700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa94409d6f0 0x7fa94409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:25.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.743+0000 7fa94c368700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa94409e410 0x7fa944130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:25.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.743+0000 7fa94c368700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa94409f130 con 0x7fa94409e000 2023-10-03T10:16:25.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.743+0000 7fa94c368700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa94409f520 con 0x7fa94409d6f0 2023-10-03T10:16:25.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.743+0000 7fa94c368700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa94409e930 con 0x7fa94409e410 2023-10-03T10:16:25.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.743+0000 7fa949903700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa94409e000 0x7fa944130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35382/0 (socket says 172.21.15.3:35382) 2023-10-03T10:16:25.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.743+0000 7fa949903700 1 -- 172.21.15.3:0/3650189980 learned_addr learned my addr 172.21.15.3:0/3650189980 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:25.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.744+0000 7fa94a104700 1 --2- 172.21.15.3:0/3650189980 >> v2:172.21.15.3:3300/0 conn(0x7fa94409d6f0 0x7fa94409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:25.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.744+0000 7fa94a905700 1 --2- 172.21.15.3:0/3650189980 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa94409e410 0x7fa944130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:25.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.744+0000 7fa949102700 1 -- 172.21.15.3:0/3650189980 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3687819778 0 0) 0x7fa94409f130 con 0x7fa94409e000 2023-10-03T10:16:25.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.744+0000 7fa949102700 1 -- 172.21.15.3:0/3650189980 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9300035b0 con 0x7fa94409e000 2023-10-03T10:16:25.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.744+0000 7fa94a905700 1 -- 172.21.15.3:0/3650189980 >> v2:172.21.15.3:3300/0 conn(0x7fa94409d6f0 msgr2=0x7fa94409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:25.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.744+0000 7fa94a905700 1 --2- 172.21.15.3:0/3650189980 >> v2:172.21.15.3:3300/0 conn(0x7fa94409d6f0 0x7fa94409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.744+0000 7fa94a905700 1 -- 172.21.15.3:0/3650189980 >> v1:172.21.15.3:6789/0 conn(0x7fa94409e000 legacy=0x7fa944130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:25.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.744+0000 7fa94a905700 1 -- 172.21.15.3:0/3650189980 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94409b2d0 con 0x7fa94409e410 2023-10-03T10:16:25.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.744+0000 7fa94a104700 1 --2- 172.21.15.3:0/3650189980 >> v2:172.21.15.3:3300/0 conn(0x7fa94409d6f0 0x7fa94409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:25.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.744+0000 7fa94a905700 1 --2- 172.21.15.3:0/3650189980 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa94409e410 0x7fa944130d70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009d90 tx=0x7fa93c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0a4e6aad1b7a559 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:25.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.744+0000 7fa949102700 1 -- 172.21.15.3:0/3650189980 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c00e070 con 0x7fa94409e410 2023-10-03T10:16:25.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.745+0000 7fa949102700 1 -- 172.21.15.3:0/3650189980 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93c00bb20 con 0x7fa94409e410 2023-10-03T10:16:25.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.745+0000 7fa949102700 1 -- 172.21.15.3:0/3650189980 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c012430 con 0x7fa94409e410 2023-10-03T10:16:25.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.746+0000 7fa94c368700 1 -- 172.21.15.3:0/3650189980 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa94409e410 msgr2=0x7fa944130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:25.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.746+0000 7fa94c368700 1 --2- 172.21.15.3:0/3650189980 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa94409e410 0x7fa944130d70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009d90 tx=0x7fa93c00b330 comp rx=0 tx=0).stop 2023-10-03T10:16:25.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.746+0000 7fa94c368700 1 -- 172.21.15.3:0/3650189980 shutdown_connections 2023-10-03T10:16:25.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.746+0000 7fa94c368700 1 --2- 172.21.15.3:0/3650189980 >> v2:172.21.15.3:3300/0 conn(0x7fa94409d6f0 0x7fa94409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.746+0000 7fa94c368700 1 --2- 172.21.15.3:0/3650189980 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa94409e410 0x7fa944130d70 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.746+0000 7fa94c368700 1 -- 172.21.15.3:0/3650189980 >> 172.21.15.3:0/3650189980 conn(0x7fa944066f50 msgr2=0x7fa94409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:25.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.746+0000 7fa94c368700 1 -- 172.21.15.3:0/3650189980 shutdown_connections 2023-10-03T10:16:25.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.746+0000 7fa94c368700 1 -- 172.21.15.3:0/3650189980 wait complete. 2023-10-03T10:16:25.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.747+0000 7fa94c368700 1 Processor -- start 2023-10-03T10:16:25.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.747+0000 7fa94c368700 1 -- start start 2023-10-03T10:16:25.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.748+0000 7fa94c368700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa94409d6f0 0x7fa944122180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:25.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.748+0000 7fa94c368700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa94409e000 0x7fa9441226c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:25.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.748+0000 7fa94c368700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa94409e410 0x7fa944122c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:25.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.748+0000 7fa94c368700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa944130490 con 0x7fa94409e000 2023-10-03T10:16:25.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.748+0000 7fa94c368700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa944125670 con 0x7fa94409d6f0 2023-10-03T10:16:25.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.748+0000 7fa94c368700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa944124560 con 0x7fa94409e410 2023-10-03T10:16:25.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.748+0000 7fa94a104700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa94409d6f0 0x7fa944122180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.748+0000 7fa94a104700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa94409d6f0 0x7fa944122180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37752/0 (socket says 172.21.15.3:37752) 2023-10-03T10:16:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.748+0000 7fa94a104700 1 -- 172.21.15.3:0/3894980456 learned_addr learned my addr 172.21.15.3:0/3894980456 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.749+0000 7fa94a905700 1 --2- 172.21.15.3:0/3894980456 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa94409e410 0x7fa944122c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.749+0000 7fa949903700 1 --2- 172.21.15.3:0/3894980456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa94409e000 0x7fa9441226c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.749+0000 7fa94a104700 1 -- 172.21.15.3:0/3894980456 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa94409e410 msgr2=0x7fa944122c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.749+0000 7fa94a104700 1 --2- 172.21.15.3:0/3894980456 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa94409e410 0x7fa944122c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.749+0000 7fa94a104700 1 -- 172.21.15.3:0/3894980456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa94409e000 msgr2=0x7fa9441226c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.749+0000 7fa94a104700 1 --2- 172.21.15.3:0/3894980456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa94409e000 0x7fa9441226c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.749+0000 7fa94a104700 1 -- 172.21.15.3:0/3894980456 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9400094a0 con 0x7fa94409d6f0 2023-10-03T10:16:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.749+0000 7fa94a905700 1 --2- 172.21.15.3:0/3894980456 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa94409e410 0x7fa944122c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:16:25.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.749+0000 7fa94a104700 1 --2- 172.21.15.3:0/3894980456 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa94409d6f0 0x7fa944122180 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa94000fde0 tx=0x7fa94000d440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.749+0000 7fa93affd700 1 -- 172.21.15.3:0/3894980456 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94000c070 con 0x7fa94409d6f0 2023-10-03T10:16:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.749+0000 7fa93affd700 1 -- 172.21.15.3:0/3894980456 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94000dc50 con 0x7fa94409d6f0 2023-10-03T10:16:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.749+0000 7fa94c368700 1 -- 172.21.15.3:0/3894980456 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa93c009a70 con 0x7fa94409d6f0 2023-10-03T10:16:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.749+0000 7fa93affd700 1 -- 172.21.15.3:0/3894980456 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94001a480 con 0x7fa94409d6f0 2023-10-03T10:16:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.750+0000 7fa94c368700 1 -- 172.21.15.3:0/3894980456 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94409c700 con 0x7fa94409d6f0 2023-10-03T10:16:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.751+0000 7fa93affd700 1 -- 172.21.15.3:0/3894980456 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa940011450 con 0x7fa94409d6f0 2023-10-03T10:16:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.751+0000 7fa94c368700 1 -- 172.21.15.3:0/3894980456 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa94411ac20 con 0x7fa94409d6f0 2023-10-03T10:16:25.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.751+0000 7fa93affd700 1 -- 172.21.15.3:0/3894980456 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa940015070 con 0x7fa94409d6f0 2023-10-03T10:16:25.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.760+0000 7fa93affd700 1 -- 172.21.15.3:0/3894980456 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa9400233a0 con 0x7fa94409d6f0 2023-10-03T10:16:25.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.770+0000 7f1e7c70c700 1 -- 172.21.15.3:0/4290430581 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1e7406c360 con 0x7f1e7409d6f0 2023-10-03T10:16:25.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.770+0000 7f1e6affd700 1 -- 172.21.15.3:0/4290430581 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1e70021070 con 0x7f1e7409d6f0 2023-10-03T10:16:25.771 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:25.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.774+0000 7f1e7c70c700 1 -- 172.21.15.3:0/4290430581 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e7409d6f0 msgr2=0x7f1e741a4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:25.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.774+0000 7f1e7c70c700 1 --2- 172.21.15.3:0/4290430581 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e7409d6f0 0x7f1e741a4f60 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000e8a0 tx=0x7f1e7000ebb0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.774+0000 7f1e7c70c700 1 -- 172.21.15.3:0/4290430581 shutdown_connections 2023-10-03T10:16:25.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.774+0000 7f1e7c70c700 1 --2- 172.21.15.3:0/4290430581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e7409e410 0x7f1e741a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.774+0000 7f1e7c70c700 1 --2- 172.21.15.3:0/4290430581 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e7409d6f0 0x7f1e741a4f60 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.774+0000 7f1e7c70c700 1 --2- 172.21.15.3:0/4290430581 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e7409e000 0x7f1e741a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.774+0000 7f1e7c70c700 1 -- 172.21.15.3:0/4290430581 >> 172.21.15.3:0/4290430581 conn(0x7f1e74066f50 msgr2=0x7f1e741bb010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:25.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.775+0000 7f1e7c70c700 1 -- 172.21.15.3:0/4290430581 shutdown_connections 2023-10-03T10:16:25.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.775+0000 7f1e7c70c700 1 -- 172.21.15.3:0/4290430581 wait complete. 2023-10-03T10:16:25.789 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:25.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.795+0000 7f69c34a2700 1 Processor -- start 2023-10-03T10:16:25.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.796+0000 7f69c34a2700 1 -- start start 2023-10-03T10:16:25.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.796+0000 7f69c34a2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f69bc134630 0x7f69bc136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:25.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.796+0000 7f69c34a2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f69bc136f50 0x7f69bc139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:25.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.796+0000 7f69c34a2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69bc09b4f0 con 0x7f69bc131810 2023-10-03T10:16:25.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.796+0000 7f69c34a2700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69bc1313c0 con 0x7f69bc136f50 2023-10-03T10:16:25.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.796+0000 7f69c34a2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69bc06e8c0 con 0x7f69bc134630 2023-10-03T10:16:25.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.796+0000 7f69c1a3f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f69bc136f50 0x7f69bc139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:25.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.796+0000 7f69c0a3d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f69bc134630 0x7f69bc136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:25.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.796+0000 7f69c123e700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f69bc131810 0x7f69bc131be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35386/0 (socket says 172.21.15.3:35386) 2023-10-03T10:16:25.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.796+0000 7f69c123e700 1 -- 172.21.15.3:0/2036914636 learned_addr learned my addr 172.21.15.3:0/2036914636 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:25.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.796+0000 7f69c0a3d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f69bc134630 0x7f69bc136a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37758/0 (socket says 172.21.15.3:37758) 2023-10-03T10:16:25.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.797+0000 7f69c1a3f700 1 -- 172.21.15.3:0/2036914636 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f69bc134630 msgr2=0x7f69bc136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:25.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.797+0000 7f69c1a3f700 1 --2- 172.21.15.3:0/2036914636 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f69bc134630 0x7f69bc136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.797+0000 7f69c1a3f700 1 -- 172.21.15.3:0/2036914636 >> v1:172.21.15.3:6789/0 conn(0x7f69bc131810 legacy=0x7f69bc131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:25.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.797+0000 7f69b3fff700 1 -- 172.21.15.3:0/2036914636 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4286616260 0 0) 0x7f69bc09b4f0 con 0x7f69bc131810 2023-10-03T10:16:25.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.797+0000 7f69c1a3f700 1 -- 172.21.15.3:0/2036914636 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69bc071d20 con 0x7f69bc136f50 2023-10-03T10:16:25.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.797+0000 7f69c1a3f700 1 --2- 172.21.15.3:0/2036914636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69bc136f50 0x7f69bc139330 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009d90 tx=0x7f69b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48b4d0058aefe90 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:25.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.797+0000 7f69b3fff700 1 -- 172.21.15.3:0/2036914636 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b400e070 con 0x7f69bc136f50 2023-10-03T10:16:25.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.797+0000 7f69b3fff700 1 -- 172.21.15.3:0/2036914636 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b400bb20 con 0x7f69bc136f50 2023-10-03T10:16:25.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.797+0000 7f69b3fff700 1 -- 172.21.15.3:0/2036914636 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b4012430 con 0x7f69bc136f50 2023-10-03T10:16:25.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.798+0000 7f69c34a2700 1 -- 172.21.15.3:0/2036914636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69bc136f50 msgr2=0x7f69bc139330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:25.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.798+0000 7f69c34a2700 1 --2- 172.21.15.3:0/2036914636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69bc136f50 0x7f69bc139330 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009d90 tx=0x7f69b400b330 comp rx=0 tx=0).stop 2023-10-03T10:16:25.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.798+0000 7f69c34a2700 1 -- 172.21.15.3:0/2036914636 shutdown_connections 2023-10-03T10:16:25.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.798+0000 7f69c34a2700 1 --2- 172.21.15.3:0/2036914636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69bc136f50 0x7f69bc139330 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.798+0000 7f69c34a2700 1 --2- 172.21.15.3:0/2036914636 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f69bc134630 0x7f69bc136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.798+0000 7f69c34a2700 1 -- 172.21.15.3:0/2036914636 >> 172.21.15.3:0/2036914636 conn(0x7f69bc0a2aa0 msgr2=0x7f69bc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:25.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.798+0000 7f69c34a2700 1 -- 172.21.15.3:0/2036914636 shutdown_connections 2023-10-03T10:16:25.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.798+0000 7f69c34a2700 1 -- 172.21.15.3:0/2036914636 wait complete. 2023-10-03T10:16:25.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.799+0000 7f69c34a2700 1 Processor -- start 2023-10-03T10:16:25.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.799+0000 7f69c34a2700 1 -- start start 2023-10-03T10:16:25.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.800+0000 7f69c34a2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f69bc131810 0x7f69bc118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:25.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.800+0000 7f69c34a2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69bc134630 0x7f69bc119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:25.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.800+0000 7f69c34a2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f69bc136f50 0x7f69bc119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:25.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.800+0000 7f69c34a2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69bc12e460 con 0x7f69bc134630 2023-10-03T10:16:25.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.800+0000 7f69c34a2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69bc12ea20 con 0x7f69bc136f50 2023-10-03T10:16:25.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.800+0000 7f69c34a2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f69bc132fd0 con 0x7f69bc131810 2023-10-03T10:16:25.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.800+0000 7f69c1a3f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f69bc136f50 0x7f69bc119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:25.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.800+0000 7f69c123e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f69bc131810 0x7f69bc118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:25.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.800+0000 7f69c1a3f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f69bc136f50 0x7f69bc119750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37770/0 (socket says 172.21.15.3:37770) 2023-10-03T10:16:25.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.800+0000 7f69c1a3f700 1 -- 172.21.15.3:0/1873031714 learned_addr learned my addr 172.21.15.3:0/1873031714 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:25.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.800+0000 7f69c0a3d700 1 --2- 172.21.15.3:0/1873031714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69bc134630 0x7f69bc119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:25.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.800+0000 7f69c1a3f700 1 -- 172.21.15.3:0/1873031714 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f69bc131810 msgr2=0x7f69bc118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:25.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.800+0000 7f69c1a3f700 1 --2- 172.21.15.3:0/1873031714 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f69bc131810 0x7f69bc118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.800+0000 7f69c1a3f700 1 -- 172.21.15.3:0/1873031714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69bc134630 msgr2=0x7f69bc119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:25.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.800+0000 7f69c1a3f700 1 --2- 172.21.15.3:0/1873031714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69bc134630 0x7f69bc119210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.800+0000 7f69c1a3f700 1 -- 172.21.15.3:0/1873031714 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b4009a70 con 0x7f69bc136f50 2023-10-03T10:16:25.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.801+0000 7f69c1a3f700 1 --2- 172.21.15.3:0/1873031714 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f69bc136f50 0x7f69bc119750 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f69b4013000 tx=0x7f69b4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:25.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.801+0000 7f69b1ffb700 1 -- 172.21.15.3:0/1873031714 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b4012430 con 0x7f69bc136f50 2023-10-03T10:16:25.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.801+0000 7f69b1ffb700 1 -- 172.21.15.3:0/1873031714 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b4020070 con 0x7f69bc136f50 2023-10-03T10:16:25.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.801+0000 7f69c34a2700 1 -- 172.21.15.3:0/1873031714 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69bc11afa0 con 0x7f69bc136f50 2023-10-03T10:16:25.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.801+0000 7f69b1ffb700 1 -- 172.21.15.3:0/1873031714 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b4004280 con 0x7f69bc136f50 2023-10-03T10:16:25.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.801+0000 7f69c34a2700 1 -- 172.21.15.3:0/1873031714 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69bc06eb00 con 0x7f69bc136f50 2023-10-03T10:16:25.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.802+0000 7f69b1ffb700 1 -- 172.21.15.3:0/1873031714 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f69b40043e0 con 0x7f69bc136f50 2023-10-03T10:16:25.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.802+0000 7f69c34a2700 1 -- 172.21.15.3:0/1873031714 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69bc074ea0 con 0x7f69bc136f50 2023-10-03T10:16:25.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.803+0000 7f69b1ffb700 1 -- 172.21.15.3:0/1873031714 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f69b4022070 con 0x7f69bc136f50 2023-10-03T10:16:25.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.812+0000 7f69b1ffb700 1 -- 172.21.15.3:0/1873031714 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f69b401b400 con 0x7f69bc136f50 2023-10-03T10:16:25.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.935+0000 7f59d98d8700 1 -- 172.21.15.3:0/3089424369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f59d4125660 con 0x7f59d409e410 2023-10-03T10:16:25.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.935+0000 7f59bbfff700 1 -- 172.21.15.3:0/3089424369 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f59d4125660 con 0x7f59d409e410 2023-10-03T10:16:25.936 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:25.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.939+0000 7f59d98d8700 1 -- 172.21.15.3:0/3089424369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59d409e410 msgr2=0x7f59d4122c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:25.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.939+0000 7f59d98d8700 1 --2- 172.21.15.3:0/3089424369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59d409e410 0x7f59d4122c00 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f59cc017000 tx=0x7f59cc004ad0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.939+0000 7f59d98d8700 1 -- 172.21.15.3:0/3089424369 shutdown_connections 2023-10-03T10:16:25.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.939+0000 7f59d98d8700 1 --2- 172.21.15.3:0/3089424369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59d409e410 0x7f59d4122c00 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.939+0000 7f59d98d8700 1 --2- 172.21.15.3:0/3089424369 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59d409d6f0 0x7f59d4122180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.939+0000 7f59d98d8700 1 --2- 172.21.15.3:0/3089424369 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59d409e000 0x7f59d41226c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.939+0000 7f59d98d8700 1 -- 172.21.15.3:0/3089424369 >> 172.21.15.3:0/3089424369 conn(0x7f59d4066f50 msgr2=0x7f59d4130800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:25.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.939+0000 7f59d98d8700 1 -- 172.21.15.3:0/3089424369 shutdown_connections 2023-10-03T10:16:25.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.939+0000 7f59d98d8700 1 -- 172.21.15.3:0/3089424369 wait complete. 2023-10-03T10:16:25.953 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:25.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.976+0000 7fa94c368700 1 -- 172.21.15.3:0/3894980456 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa944074ea0 con 0x7fa94409d6f0 2023-10-03T10:16:25.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.976+0000 7fa93affd700 1 -- 172.21.15.3:0/3894980456 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa94001f070 con 0x7fa94409d6f0 2023-10-03T10:16:25.977 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:25.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.979+0000 7fa94c368700 1 -- 172.21.15.3:0/3894980456 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa94409d6f0 msgr2=0x7fa944122180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:25.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.979+0000 7fa94c368700 1 --2- 172.21.15.3:0/3894980456 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa94409d6f0 0x7fa944122180 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa94000fde0 tx=0x7fa94000d440 comp rx=0 tx=0).stop 2023-10-03T10:16:25.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.979+0000 7fa94c368700 1 -- 172.21.15.3:0/3894980456 shutdown_connections 2023-10-03T10:16:25.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.979+0000 7fa94c368700 1 --2- 172.21.15.3:0/3894980456 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa94409e410 0x7fa944122c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.979+0000 7fa94c368700 1 --2- 172.21.15.3:0/3894980456 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa94409d6f0 0x7fa944122180 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.979+0000 7fa94c368700 1 --2- 172.21.15.3:0/3894980456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa94409e000 0x7fa9441226c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:25.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.979+0000 7fa94c368700 1 -- 172.21.15.3:0/3894980456 >> 172.21.15.3:0/3894980456 conn(0x7fa944066f50 msgr2=0x7fa944130a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:25.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.979+0000 7fa94c368700 1 -- 172.21.15.3:0/3894980456 shutdown_connections 2023-10-03T10:16:25.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:25.979+0000 7fa94c368700 1 -- 172.21.15.3:0/3894980456 wait complete. 2023-10-03T10:16:25.993 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:26.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.020+0000 7f69c34a2700 1 -- 172.21.15.3:0/1873031714 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f69bc11c430 con 0x7f69bc136f50 2023-10-03T10:16:26.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.021+0000 7f69b1ffb700 1 -- 172.21.15.3:0/1873031714 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f69b402c070 con 0x7f69bc136f50 2023-10-03T10:16:26.021 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:26.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.023+0000 7f69c34a2700 1 -- 172.21.15.3:0/1873031714 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f69bc136f50 msgr2=0x7f69bc119750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:26.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.023+0000 7f69c34a2700 1 --2- 172.21.15.3:0/1873031714 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f69bc136f50 0x7f69bc119750 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f69b4013000 tx=0x7f69b4004ad0 comp rx=0 tx=0).stop 2023-10-03T10:16:26.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.023+0000 7f69c34a2700 1 -- 172.21.15.3:0/1873031714 shutdown_connections 2023-10-03T10:16:26.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.023+0000 7f69c34a2700 1 --2- 172.21.15.3:0/1873031714 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f69bc136f50 0x7f69bc119750 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:26.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.023+0000 7f69c34a2700 1 --2- 172.21.15.3:0/1873031714 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f69bc131810 0x7f69bc118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:26.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.023+0000 7f69c34a2700 1 --2- 172.21.15.3:0/1873031714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69bc134630 0x7f69bc119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:26.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.023+0000 7f69c34a2700 1 -- 172.21.15.3:0/1873031714 >> 172.21.15.3:0/1873031714 conn(0x7f69bc0a2aa0 msgr2=0x7f69bc09f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:26.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.023+0000 7f69c34a2700 1 -- 172.21.15.3:0/1873031714 shutdown_connections 2023-10-03T10:16:26.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.023+0000 7f69c34a2700 1 -- 172.21.15.3:0/1873031714 wait complete. 2023-10-03T10:16:26.037 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:26.790 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:26.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.936+0000 7f48a7987700 1 Processor -- start 2023-10-03T10:16:26.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.936+0000 7f48a7987700 1 -- start start 2023-10-03T10:16:26.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.937+0000 7f48a7987700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48a0136830 0x7f48a0138c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:26.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.937+0000 7f48a7987700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48a0139150 0x7f48a013b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:26.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.937+0000 7f48a7987700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a006f720 con 0x7f48a01338c0 2023-10-03T10:16:26.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.937+0000 7f48a7987700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a0070c80 con 0x7f48a0136830 2023-10-03T10:16:26.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.937+0000 7f48a7987700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f48a006fce0 con 0x7f48a0139150 2023-10-03T10:16:26.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.937+0000 7f48a4f22700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48a0136830 0x7f48a0138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:26.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.937+0000 7f48a5723700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f48a01338c0 0x7f48a0133c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35390/0 (socket says 172.21.15.3:35390) 2023-10-03T10:16:26.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.937+0000 7f48a5f24700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48a0139150 0x7f48a013b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:26.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.937+0000 7f48a5723700 1 -- 172.21.15.3:0/3703923916 learned_addr learned my addr 172.21.15.3:0/3703923916 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:26.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.937+0000 7f4893fff700 1 -- 172.21.15.3:0/3703923916 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4194227061 0 0) 0x7f48a006f720 con 0x7f48a01338c0 2023-10-03T10:16:26.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.937+0000 7f4893fff700 1 -- 172.21.15.3:0/3703923916 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f488c0035b0 con 0x7f48a01338c0 2023-10-03T10:16:26.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.937+0000 7f48a5f24700 1 -- 172.21.15.3:0/3703923916 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48a0136830 msgr2=0x7f48a0138c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:26.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.937+0000 7f48a5f24700 1 --2- 172.21.15.3:0/3703923916 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48a0136830 0x7f48a0138c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:26.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.937+0000 7f48a5f24700 1 -- 172.21.15.3:0/3703923916 >> v1:172.21.15.3:6789/0 conn(0x7f48a01338c0 legacy=0x7f48a0133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:26.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.937+0000 7f48a5f24700 1 -- 172.21.15.3:0/3703923916 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a01336a0 con 0x7f48a0139150 2023-10-03T10:16:26.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.937+0000 7f48a4f22700 1 --2- 172.21.15.3:0/3703923916 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48a0136830 0x7f48a0138c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:26.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.938+0000 7f48a5f24700 1 --2- 172.21.15.3:0/3703923916 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48a0139150 0x7f48a013b530 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4898009dc0 tx=0x7f489800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d465938f5fe5528 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:26.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.938+0000 7f4893fff700 1 -- 172.21.15.3:0/3703923916 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489800e070 con 0x7f48a0139150 2023-10-03T10:16:26.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.938+0000 7f4893fff700 1 -- 172.21.15.3:0/3703923916 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489800bb20 con 0x7f48a0139150 2023-10-03T10:16:26.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.938+0000 7f4893fff700 1 -- 172.21.15.3:0/3703923916 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4898012430 con 0x7f48a0139150 2023-10-03T10:16:26.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.938+0000 7f48a7987700 1 -- 172.21.15.3:0/3703923916 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48a0139150 msgr2=0x7f48a013b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:26.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.938+0000 7f48a7987700 1 --2- 172.21.15.3:0/3703923916 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48a0139150 0x7f48a013b530 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4898009dc0 tx=0x7f489800b330 comp rx=0 tx=0).stop 2023-10-03T10:16:26.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.938+0000 7f48a7987700 1 -- 172.21.15.3:0/3703923916 shutdown_connections 2023-10-03T10:16:26.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.938+0000 7f48a7987700 1 --2- 172.21.15.3:0/3703923916 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48a0139150 0x7f48a013b530 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:26.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.938+0000 7f48a7987700 1 --2- 172.21.15.3:0/3703923916 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48a0136830 0x7f48a0138c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:26.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.938+0000 7f48a7987700 1 -- 172.21.15.3:0/3703923916 >> 172.21.15.3:0/3703923916 conn(0x7f48a00a2aa0 msgr2=0x7f48a00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:26.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.939+0000 7f48a7987700 1 -- 172.21.15.3:0/3703923916 shutdown_connections 2023-10-03T10:16:26.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.939+0000 7f48a7987700 1 -- 172.21.15.3:0/3703923916 wait complete. 2023-10-03T10:16:26.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.940+0000 7f48a7987700 1 Processor -- start 2023-10-03T10:16:26.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.940+0000 7f48a7987700 1 -- start start 2023-10-03T10:16:26.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.940+0000 7f48a7987700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48a01338c0 0x7f48a0119ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:26.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.940+0000 7f48a7987700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a0136830 0x7f48a011a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:26.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.940+0000 7f48a7987700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48a0139150 0x7f48a011a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:26.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.940+0000 7f48a7987700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a012f560 con 0x7f48a0136830 2023-10-03T10:16:26.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.940+0000 7f48a7987700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a012fb40 con 0x7f48a0139150 2023-10-03T10:16:26.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.940+0000 7f48a7987700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f48a0138f20 con 0x7f48a01338c0 2023-10-03T10:16:26.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.940+0000 7f48a5723700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48a01338c0 0x7f48a0119ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:26.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.940+0000 7f48a4f22700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a0136830 0x7f48a011a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:26.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.940+0000 7f48a5f24700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48a0139150 0x7f48a011a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:26.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.940+0000 7f48a5723700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48a01338c0 0x7f48a0119ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51446/0 (socket says 172.21.15.3:51446) 2023-10-03T10:16:26.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.940+0000 7f48a4f22700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a0136830 0x7f48a011a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42548/0 (socket says 172.21.15.3:42548) 2023-10-03T10:16:26.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.940+0000 7f48a5723700 1 -- 172.21.15.3:0/3421329303 learned_addr learned my addr 172.21.15.3:0/3421329303 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:26.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.940+0000 7f48a5f24700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48a0139150 0x7f48a011a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37792/0 (socket says 172.21.15.3:37792) 2023-10-03T10:16:26.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.941+0000 7f48a5723700 1 -- 172.21.15.3:0/3421329303 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48a0139150 msgr2=0x7f48a011a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:26.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.941+0000 7f48a5723700 1 --2- 172.21.15.3:0/3421329303 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48a0139150 0x7f48a011a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:26.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.941+0000 7f48a5723700 1 -- 172.21.15.3:0/3421329303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a0136830 msgr2=0x7f48a011a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:26.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.941+0000 7f48a5723700 1 --2- 172.21.15.3:0/3421329303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a0136830 0x7f48a011a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:26.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.941+0000 7f48a5723700 1 -- 172.21.15.3:0/3421329303 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489c0094a0 con 0x7f48a01338c0 2023-10-03T10:16:26.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.941+0000 7f48a5f24700 1 --2- 172.21.15.3:0/3421329303 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48a0139150 0x7f48a011a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:26.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.941+0000 7f48a4f22700 1 --2- 172.21.15.3:0/3421329303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a0136830 0x7f48a011a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:26.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.941+0000 7f48a5723700 1 --2- 172.21.15.3:0/3421329303 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48a01338c0 0x7f48a0119ca0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f489400c700 tx=0x7f489400ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:26.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.941+0000 7f4891ffb700 1 -- 172.21.15.3:0/3421329303 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4894005d70 con 0x7f48a01338c0 2023-10-03T10:16:26.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.941+0000 7f4891ffb700 1 -- 172.21.15.3:0/3421329303 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48940070d0 con 0x7f48a01338c0 2023-10-03T10:16:26.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.941+0000 7f48a7987700 1 -- 172.21.15.3:0/3421329303 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4898009a70 con 0x7f48a01338c0 2023-10-03T10:16:26.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.941+0000 7f4891ffb700 1 -- 172.21.15.3:0/3421329303 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4894007460 con 0x7f48a01338c0 2023-10-03T10:16:26.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.941+0000 7f48a7987700 1 -- 172.21.15.3:0/3421329303 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a011c0c0 con 0x7f48a01338c0 2023-10-03T10:16:26.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.942+0000 7f4891ffb700 1 -- 172.21.15.3:0/3421329303 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f48940075c0 con 0x7f48a01338c0 2023-10-03T10:16:26.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.942+0000 7f48a7987700 1 -- 172.21.15.3:0/3421329303 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a006f9c0 con 0x7f48a01338c0 2023-10-03T10:16:26.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.942+0000 7f4891ffb700 1 -- 172.21.15.3:0/3421329303 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f489403ec90 con 0x7f48a01338c0 2023-10-03T10:16:26.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:26.947+0000 7f4891ffb700 1 -- 172.21.15.3:0/3421329303 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4894018260 con 0x7f48a01338c0 2023-10-03T10:16:26.954 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:26.993 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:27.038 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:27.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.095+0000 7f5e866d0700 1 Processor -- start 2023-10-03T10:16:27.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.095+0000 7f5e866d0700 1 -- start start 2023-10-03T10:16:27.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.096+0000 7f5e866d0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5e801337f0 0x7f5e80138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:27.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.096+0000 7f5e866d0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e80138dc0 0x7f5e8013b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:27.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.096+0000 7f5e866d0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e8006b990 con 0x7f5e80133420 2023-10-03T10:16:27.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.096+0000 7f5e866d0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e8009b0e0 con 0x7f5e801337f0 2023-10-03T10:16:27.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.096+0000 7f5e866d0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e80133c20 con 0x7f5e80138dc0 2023-10-03T10:16:27.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.096+0000 7f5e7f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5e801337f0 0x7f5e80138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:27.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.096+0000 7f5e84c6d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e80138dc0 0x7f5e8013b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:27.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.096+0000 7f5e84c6d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e80138dc0 0x7f5e8013b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51452/0 (socket says 172.21.15.3:51452) 2023-10-03T10:16:27.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.096+0000 7f5e84c6d700 1 -- 172.21.15.3:0/1779144732 learned_addr learned my addr 172.21.15.3:0/1779144732 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:27.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.096+0000 7f5e7f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5e801337f0 0x7f5e80138880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42554/0 (socket says 172.21.15.3:42554) 2023-10-03T10:16:27.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.097+0000 7f5e84c6d700 1 -- 172.21.15.3:0/1779144732 >> v2:172.21.15.3:3300/0 conn(0x7f5e801337f0 msgr2=0x7f5e80138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:27.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.097+0000 7f5e84c6d700 1 --2- 172.21.15.3:0/1779144732 >> v2:172.21.15.3:3300/0 conn(0x7f5e801337f0 0x7f5e80138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.097+0000 7f5e84c6d700 1 -- 172.21.15.3:0/1779144732 >> v1:172.21.15.3:6789/0 conn(0x7f5e80133420 legacy=0x7f5e80133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:27.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.097+0000 7f5e84c6d700 1 -- 172.21.15.3:0/1779144732 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e800a0590 con 0x7f5e80138dc0 2023-10-03T10:16:27.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.097+0000 7f5e84c6d700 1 --2- 172.21.15.3:0/1779144732 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e80138dc0 0x7f5e8013b1a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009e50 tx=0x7f5e7800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e66852f4fc9f184c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:27.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.097+0000 7f5e7effd700 1 -- 172.21.15.3:0/1779144732 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7800e070 con 0x7f5e80138dc0 2023-10-03T10:16:27.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.097+0000 7f5e7effd700 1 -- 172.21.15.3:0/1779144732 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7800b910 con 0x7f5e80138dc0 2023-10-03T10:16:27.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.097+0000 7f5e7effd700 1 -- 172.21.15.3:0/1779144732 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7800bca0 con 0x7f5e80138dc0 2023-10-03T10:16:27.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.098+0000 7f5e866d0700 1 -- 172.21.15.3:0/1779144732 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e80138dc0 msgr2=0x7f5e8013b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:27.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.098+0000 7f5e866d0700 1 --2- 172.21.15.3:0/1779144732 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e80138dc0 0x7f5e8013b1a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009e50 tx=0x7f5e7800b330 comp rx=0 tx=0).stop 2023-10-03T10:16:27.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.098+0000 7f5e866d0700 1 -- 172.21.15.3:0/1779144732 shutdown_connections 2023-10-03T10:16:27.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.098+0000 7f5e866d0700 1 --2- 172.21.15.3:0/1779144732 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e80138dc0 0x7f5e8013b1a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.098+0000 7f5e866d0700 1 --2- 172.21.15.3:0/1779144732 >> v2:172.21.15.3:3300/0 conn(0x7f5e801337f0 0x7f5e80138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.098+0000 7f5e866d0700 1 -- 172.21.15.3:0/1779144732 >> 172.21.15.3:0/1779144732 conn(0x7f5e800a4210 msgr2=0x7f5e800a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:27.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.098+0000 7f5e866d0700 1 -- 172.21.15.3:0/1779144732 shutdown_connections 2023-10-03T10:16:27.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.098+0000 7f5e866d0700 1 -- 172.21.15.3:0/1779144732 wait complete. 2023-10-03T10:16:27.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.099+0000 7f5e866d0700 1 Processor -- start 2023-10-03T10:16:27.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.099+0000 7f5e866d0700 1 -- start start 2023-10-03T10:16:27.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.100+0000 7f5e866d0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e80133420 0x7f5e800709a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:27.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.100+0000 7f5e866d0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e801337f0 0x7f5e80070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:27.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.100+0000 7f5e866d0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e80138dc0 0x7f5e80071420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:27.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.100+0000 7f5e866d0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8009bde0 con 0x7f5e801337f0 2023-10-03T10:16:27.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.100+0000 7f5e866d0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8009d900 con 0x7f5e80138dc0 2023-10-03T10:16:27.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.100+0000 7f5e866d0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8009c600 con 0x7f5e80133420 2023-10-03T10:16:27.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.100+0000 7f5e7ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e80133420 0x7f5e800709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:27.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.100+0000 7f5e7ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e80133420 0x7f5e800709a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51466/0 (socket says 172.21.15.3:51466) 2023-10-03T10:16:27.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.100+0000 7f5e7ffff700 1 -- 172.21.15.3:0/2025604083 learned_addr learned my addr 172.21.15.3:0/2025604083 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.100+0000 7f5e7f7fe700 1 --2- 172.21.15.3:0/2025604083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e801337f0 0x7f5e80070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.100+0000 7f5e84c6d700 1 --2- 172.21.15.3:0/2025604083 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e80138dc0 0x7f5e80071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.100+0000 7f5e7ffff700 1 -- 172.21.15.3:0/2025604083 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e80138dc0 msgr2=0x7f5e80071420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.100+0000 7f5e7ffff700 1 --2- 172.21.15.3:0/2025604083 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e80138dc0 0x7f5e80071420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.100+0000 7f5e7ffff700 1 -- 172.21.15.3:0/2025604083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e801337f0 msgr2=0x7f5e80070ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.101+0000 7f5e7ffff700 1 --2- 172.21.15.3:0/2025604083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e801337f0 0x7f5e80070ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.101+0000 7f5e7ffff700 1 -- 172.21.15.3:0/2025604083 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e78009b30 con 0x7f5e80133420 2023-10-03T10:16:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.101+0000 7f5e7f7fe700 1 --2- 172.21.15.3:0/2025604083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e801337f0 0x7f5e80070ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.101+0000 7f5e7ffff700 1 --2- 172.21.15.3:0/2025604083 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e80133420 0x7f5e800709a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000c4e0 tx=0x7f5e7000c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.101+0000 7f5e7cff9700 1 -- 172.21.15.3:0/2025604083 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e700052d0 con 0x7f5e80133420 2023-10-03T10:16:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.101+0000 7f5e7cff9700 1 -- 172.21.15.3:0/2025604083 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7000ce90 con 0x7f5e80133420 2023-10-03T10:16:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.101+0000 7f5e866d0700 1 -- 172.21.15.3:0/2025604083 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e801388a0 con 0x7f5e80133420 2023-10-03T10:16:27.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.101+0000 7f5e7cff9700 1 -- 172.21.15.3:0/2025604083 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e70003680 con 0x7f5e80133420 2023-10-03T10:16:27.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.101+0000 7f5e866d0700 1 -- 172.21.15.3:0/2025604083 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8009d330 con 0x7f5e80133420 2023-10-03T10:16:27.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.102+0000 7f5e7cff9700 1 -- 172.21.15.3:0/2025604083 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5e700037e0 con 0x7f5e80133420 2023-10-03T10:16:27.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.102+0000 7f5e7cff9700 1 -- 172.21.15.3:0/2025604083 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5e7003eaa0 con 0x7f5e80133420 2023-10-03T10:16:27.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.102+0000 7f5e866d0700 1 -- 172.21.15.3:0/2025604083 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e80074ea0 con 0x7f5e80133420 2023-10-03T10:16:27.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.108+0000 7f5e7cff9700 1 -- 172.21.15.3:0/2025604083 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5e70011590 con 0x7f5e80133420 2023-10-03T10:16:27.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.140+0000 7fe0b2896700 1 Processor -- start 2023-10-03T10:16:27.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.140+0000 7fe0b2896700 1 -- start start 2023-10-03T10:16:27.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.141+0000 7fe0b2896700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe0ac09d6f0 0x7fe0ac09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:27.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.141+0000 7fe0b2896700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe0ac09e000 0x7fe0ac130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:27.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.141+0000 7fe0b2896700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe0ac09e410 0x7fe0ac130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:27.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.141+0000 7fe0b2896700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0ac09f520 con 0x7fe0ac09e000 2023-10-03T10:16:27.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.141+0000 7fe0b2896700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac09f130 con 0x7fe0ac09e410 2023-10-03T10:16:27.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.141+0000 7fe0b2896700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac09e930 con 0x7fe0ac09d6f0 2023-10-03T10:16:27.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.141+0000 7fe0b0e33700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe0ac09e410 0x7fe0ac130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:27.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.141+0000 7fe0b0e33700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe0ac09e410 0x7fe0ac130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37814/0 (socket says 172.21.15.3:37814) 2023-10-03T10:16:27.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.141+0000 7fe0b0e33700 1 -- 172.21.15.3:0/2332402758 learned_addr learned my addr 172.21.15.3:0/2332402758 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:27.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.141+0000 7fe0abfff700 1 --2- 172.21.15.3:0/2332402758 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe0ac09d6f0 0x7fe0ac09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:27.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.141+0000 7fe0ab7fe700 1 --2- 172.21.15.3:0/2332402758 >> v2:172.21.15.3:3300/0 conn(0x7fe0ac09e000 0x7fe0ac130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:27.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.141+0000 7fe0b0e33700 1 -- 172.21.15.3:0/2332402758 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe0ac09d6f0 msgr2=0x7fe0ac09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:27.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.141+0000 7fe0b0e33700 1 --2- 172.21.15.3:0/2332402758 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe0ac09d6f0 0x7fe0ac09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.141+0000 7fe0b0e33700 1 -- 172.21.15.3:0/2332402758 >> v2:172.21.15.3:3300/0 conn(0x7fe0ac09e000 msgr2=0x7fe0ac130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:27.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.141+0000 7fe0b0e33700 1 --2- 172.21.15.3:0/2332402758 >> v2:172.21.15.3:3300/0 conn(0x7fe0ac09e000 0x7fe0ac130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.141+0000 7fe0b0e33700 1 -- 172.21.15.3:0/2332402758 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ac09b2d0 con 0x7fe0ac09e410 2023-10-03T10:16:27.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.141+0000 7fe0abfff700 1 --2- 172.21.15.3:0/2332402758 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe0ac09d6f0 0x7fe0ac09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:27.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.141+0000 7fe0ab7fe700 1 --2- 172.21.15.3:0/2332402758 >> v2:172.21.15.3:3300/0 conn(0x7fe0ac09e000 0x7fe0ac130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:27.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.142+0000 7fe0b0e33700 1 --2- 172.21.15.3:0/2332402758 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe0ac09e410 0x7fe0ac130ba0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009e50 tx=0x7fe0a400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cf1e66606d4211 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:27.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.142+0000 7fe0aaffd700 1 -- 172.21.15.3:0/2332402758 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a400e070 con 0x7fe0ac09e410 2023-10-03T10:16:27.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.142+0000 7fe0aaffd700 1 -- 172.21.15.3:0/2332402758 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a400b910 con 0x7fe0ac09e410 2023-10-03T10:16:27.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.142+0000 7fe0aaffd700 1 -- 172.21.15.3:0/2332402758 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a400bcc0 con 0x7fe0ac09e410 2023-10-03T10:16:27.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.143+0000 7fe0b2896700 1 -- 172.21.15.3:0/2332402758 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe0ac09e410 msgr2=0x7fe0ac130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:27.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.143+0000 7fe0b2896700 1 --2- 172.21.15.3:0/2332402758 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe0ac09e410 0x7fe0ac130ba0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009e50 tx=0x7fe0a400b330 comp rx=0 tx=0).stop 2023-10-03T10:16:27.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.143+0000 7fe0b2896700 1 -- 172.21.15.3:0/2332402758 shutdown_connections 2023-10-03T10:16:27.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.143+0000 7fe0b2896700 1 --2- 172.21.15.3:0/2332402758 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe0ac09e410 0x7fe0ac130ba0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.143+0000 7fe0b2896700 1 --2- 172.21.15.3:0/2332402758 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe0ac09d6f0 0x7fe0ac09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.143+0000 7fe0b2896700 1 --2- 172.21.15.3:0/2332402758 >> v2:172.21.15.3:3300/0 conn(0x7fe0ac09e000 0x7fe0ac130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.143+0000 7fe0b2896700 1 -- 172.21.15.3:0/2332402758 >> 172.21.15.3:0/2332402758 conn(0x7fe0ac066f50 msgr2=0x7fe0ac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:27.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.143+0000 7fe0b2896700 1 -- 172.21.15.3:0/2332402758 shutdown_connections 2023-10-03T10:16:27.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.143+0000 7fe0b2896700 1 -- 172.21.15.3:0/2332402758 wait complete. 2023-10-03T10:16:27.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.144+0000 7fe0b2896700 1 Processor -- start 2023-10-03T10:16:27.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.144+0000 7fe0b2896700 1 -- start start 2023-10-03T10:16:27.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.145+0000 7fe0b2896700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0ac09d6f0 0x7fe0ac137920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:27.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.145+0000 7fe0b2896700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe0ac09e000 0x7fe0ac137e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:27.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.145+0000 7fe0b2896700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe0ac09e410 0x7fe0ac1383a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:27.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.145+0000 7fe0b2896700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac11a1c0 con 0x7fe0ac09d6f0 2023-10-03T10:16:27.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.145+0000 7fe0b2896700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac11a860 con 0x7fe0ac09e000 2023-10-03T10:16:27.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.145+0000 7fe0b2896700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac072420 con 0x7fe0ac09e410 2023-10-03T10:16:27.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.145+0000 7fe0b0e33700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe0ac09e410 0x7fe0ac1383a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:27.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.145+0000 7fe0ab7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe0ac09e000 0x7fe0ac137e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:27.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.145+0000 7fe0b0e33700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe0ac09e410 0x7fe0ac1383a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51498/0 (socket says 172.21.15.3:51498) 2023-10-03T10:16:27.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.145+0000 7fe0b0e33700 1 -- 172.21.15.3:0/3121244973 learned_addr learned my addr 172.21.15.3:0/3121244973 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:27.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.145+0000 7fe0ab7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe0ac09e000 0x7fe0ac137e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37818/0 (socket says 172.21.15.3:37818) 2023-10-03T10:16:27.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.145+0000 7fe0abfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0ac09d6f0 0x7fe0ac137920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:27.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.145+0000 7fe0b0e33700 1 -- 172.21.15.3:0/3121244973 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe0ac09e000 msgr2=0x7fe0ac137e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:27.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.145+0000 7fe0b0e33700 1 --2- 172.21.15.3:0/3121244973 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe0ac09e000 0x7fe0ac137e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.145+0000 7fe0b0e33700 1 -- 172.21.15.3:0/3121244973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0ac09d6f0 msgr2=0x7fe0ac137920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:27.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.145+0000 7fe0b0e33700 1 --2- 172.21.15.3:0/3121244973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0ac09d6f0 0x7fe0ac137920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.145+0000 7fe0b0e33700 1 -- 172.21.15.3:0/3121244973 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09c0094a0 con 0x7fe0ac09e410 2023-10-03T10:16:27.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.145+0000 7fe0ab7fe700 1 --2- 172.21.15.3:0/3121244973 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe0ac09e000 0x7fe0ac137e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:27.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.146+0000 7fe0b0e33700 1 --2- 172.21.15.3:0/3121244973 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe0ac09e410 0x7fe0ac1383a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4003f50 tx=0x7fe0a4004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:27.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.146+0000 7fe0a8ff9700 1 -- 172.21.15.3:0/3121244973 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4004450 con 0x7fe0ac09e410 2023-10-03T10:16:27.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.146+0000 7fe0a8ff9700 1 -- 172.21.15.3:0/3121244973 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a4007080 con 0x7fe0ac09e410 2023-10-03T10:16:27.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.146+0000 7fe0b2896700 1 -- 172.21.15.3:0/3121244973 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a00094a0 con 0x7fe0ac09e410 2023-10-03T10:16:27.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.146+0000 7fe0a8ff9700 1 -- 172.21.15.3:0/3121244973 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4004810 con 0x7fe0ac09e410 2023-10-03T10:16:27.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.146+0000 7fe0b2896700 1 -- 172.21.15.3:0/3121244973 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a4009b30 con 0x7fe0ac09e410 2023-10-03T10:16:27.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.147+0000 7fe0a8ff9700 1 -- 172.21.15.3:0/3121244973 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe0a401c410 con 0x7fe0ac09e410 2023-10-03T10:16:27.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.147+0000 7fe0b2896700 1 -- 172.21.15.3:0/3121244973 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0ac11a3e0 con 0x7fe0ac09e410 2023-10-03T10:16:27.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.147+0000 7fe0a8ff9700 1 -- 172.21.15.3:0/3121244973 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe0a40411b0 con 0x7fe0ac09e410 2023-10-03T10:16:27.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.152+0000 7fe0a8ff9700 1 -- 172.21.15.3:0/3121244973 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe0a40117a0 con 0x7fe0ac09e410 2023-10-03T10:16:27.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.172+0000 7f48a7987700 1 -- 172.21.15.3:0/3421329303 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f48a0074ea0 con 0x7f48a01338c0 2023-10-03T10:16:27.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.173+0000 7f4891ffb700 1 -- 172.21.15.3:0/3421329303 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4894017030 con 0x7f48a01338c0 2023-10-03T10:16:27.173 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:27.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.176+0000 7f48a7987700 1 -- 172.21.15.3:0/3421329303 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48a01338c0 msgr2=0x7f48a0119ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:27.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.176+0000 7f48a7987700 1 --2- 172.21.15.3:0/3421329303 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48a01338c0 0x7f48a0119ca0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f489400c700 tx=0x7f489400ca10 comp rx=0 tx=0).stop 2023-10-03T10:16:27.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.176+0000 7f48a7987700 1 -- 172.21.15.3:0/3421329303 shutdown_connections 2023-10-03T10:16:27.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.176+0000 7f48a7987700 1 --2- 172.21.15.3:0/3421329303 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48a0139150 0x7f48a011a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.176+0000 7f48a7987700 1 --2- 172.21.15.3:0/3421329303 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48a01338c0 0x7f48a0119ca0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.176+0000 7f48a7987700 1 --2- 172.21.15.3:0/3421329303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a0136830 0x7f48a011a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.176+0000 7f48a7987700 1 -- 172.21.15.3:0/3421329303 >> 172.21.15.3:0/3421329303 conn(0x7f48a00a2aa0 msgr2=0x7f48a009de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:27.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.176+0000 7f48a7987700 1 -- 172.21.15.3:0/3421329303 shutdown_connections 2023-10-03T10:16:27.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.176+0000 7f48a7987700 1 -- 172.21.15.3:0/3421329303 wait complete. 2023-10-03T10:16:27.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.184+0000 7f74e1c1f700 1 Processor -- start 2023-10-03T10:16:27.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.184+0000 7f74e1c1f700 1 -- start start 2023-10-03T10:16:27.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.185+0000 7f74e1c1f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74dc12cf60 0x7f74dc06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:27.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.185+0000 7f74e1c1f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74dc12f8b0 0x7f74dc12fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:27.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.185+0000 7f74e1c1f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74dc12cd90 con 0x7f74dc130260 2023-10-03T10:16:27.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.185+0000 7f74e1c1f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc12d330 con 0x7f74dc12cf60 2023-10-03T10:16:27.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.185+0000 7f74e1c1f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc1306d0 con 0x7f74dc12f8b0 2023-10-03T10:16:27.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.185+0000 7f74dbfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f74dc130260 0x7f74dc136c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35416/0 (socket says 172.21.15.3:35416) 2023-10-03T10:16:27.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.185+0000 7f74dbfff700 1 -- 172.21.15.3:0/975795372 learned_addr learned my addr 172.21.15.3:0/975795372 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:27.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.185+0000 7f74db7fe700 1 --2- 172.21.15.3:0/975795372 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74dc12cf60 0x7f74dc06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:27.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.185+0000 7f74daffd700 1 --2- 172.21.15.3:0/975795372 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74dc12f8b0 0x7f74dc12fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:27.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.185+0000 7f74daffd700 1 -- 172.21.15.3:0/975795372 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74dc12cf60 msgr2=0x7f74dc06f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:27.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.185+0000 7f74daffd700 1 --2- 172.21.15.3:0/975795372 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74dc12cf60 0x7f74dc06f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.185+0000 7f74daffd700 1 -- 172.21.15.3:0/975795372 >> v1:172.21.15.3:6789/0 conn(0x7f74dc130260 legacy=0x7f74dc136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:27.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.185+0000 7f74daffd700 1 -- 172.21.15.3:0/975795372 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74dc071780 con 0x7f74dc12f8b0 2023-10-03T10:16:27.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.185+0000 7f74da7fc700 1 -- 172.21.15.3:0/975795372 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2568785178 0 0) 0x7f74dc12cd90 con 0x7f74dc130260 2023-10-03T10:16:27.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.186+0000 7f74daffd700 1 --2- 172.21.15.3:0/975795372 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74dc12f8b0 0x7f74dc12fd20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f74d0009870 tx=0x7f74d0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17938b03d0c994fe server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:27.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.186+0000 7f74da7fc700 1 -- 172.21.15.3:0/975795372 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d0011830 con 0x7f74dc12f8b0 2023-10-03T10:16:27.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.186+0000 7f74da7fc700 1 -- 172.21.15.3:0/975795372 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d0011990 con 0x7f74dc12f8b0 2023-10-03T10:16:27.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.186+0000 7f74da7fc700 1 -- 172.21.15.3:0/975795372 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d0011d20 con 0x7f74dc12f8b0 2023-10-03T10:16:27.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.187+0000 7f74e1c1f700 1 -- 172.21.15.3:0/975795372 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74dc12f8b0 msgr2=0x7f74dc12fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:27.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.187+0000 7f74e1c1f700 1 --2- 172.21.15.3:0/975795372 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74dc12f8b0 0x7f74dc12fd20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f74d0009870 tx=0x7f74d0009b80 comp rx=0 tx=0).stop 2023-10-03T10:16:27.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.187+0000 7f74e1c1f700 1 -- 172.21.15.3:0/975795372 shutdown_connections 2023-10-03T10:16:27.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.187+0000 7f74e1c1f700 1 --2- 172.21.15.3:0/975795372 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74dc12cf60 0x7f74dc06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.187+0000 7f74e1c1f700 1 --2- 172.21.15.3:0/975795372 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74dc12f8b0 0x7f74dc12fd20 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.187+0000 7f74e1c1f700 1 -- 172.21.15.3:0/975795372 >> 172.21.15.3:0/975795372 conn(0x7f74dc0a2aa0 msgr2=0x7f74dc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:27.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.187+0000 7f74e1c1f700 1 -- 172.21.15.3:0/975795372 shutdown_connections 2023-10-03T10:16:27.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.187+0000 7f74e1c1f700 1 -- 172.21.15.3:0/975795372 wait complete. 2023-10-03T10:16:27.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.188+0000 7f74e1c1f700 1 Processor -- start 2023-10-03T10:16:27.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.188+0000 7f74e1c1f700 1 -- start start 2023-10-03T10:16:27.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.189+0000 7f74e1c1f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74dc12cf60 0x7f74dc09bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:27.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.189+0000 7f74e1c1f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74dc12f8b0 0x7f74dc09c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:27.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.189+0000 7f74e1c1f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74dc130260 0x7f74dc1c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.189+0000 7f74e1c1f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc129c90 con 0x7f74dc12cf60 2023-10-03T10:16:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.189+0000 7f74e1c1f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc09b770 con 0x7f74dc12f8b0 2023-10-03T10:16:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.189+0000 7f74e1c1f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc1299e0 con 0x7f74dc130260 2023-10-03T10:16:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.189+0000 7f74db7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74dc12cf60 0x7f74dc09bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.189+0000 7f74db7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74dc12cf60 0x7f74dc09bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42602/0 (socket says 172.21.15.3:42602) 2023-10-03T10:16:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.189+0000 7f74db7fe700 1 -- 172.21.15.3:0/359156022 learned_addr learned my addr 172.21.15.3:0/359156022 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.189+0000 7f74daffd700 1 --2- 172.21.15.3:0/359156022 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74dc12f8b0 0x7f74dc09c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.189+0000 7f74dbfff700 1 --2- 172.21.15.3:0/359156022 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74dc130260 0x7f74dc1c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.189+0000 7f74db7fe700 1 -- 172.21.15.3:0/359156022 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74dc130260 msgr2=0x7f74dc1c3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.189+0000 7f74db7fe700 1 --2- 172.21.15.3:0/359156022 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74dc130260 0x7f74dc1c3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.189+0000 7f74db7fe700 1 -- 172.21.15.3:0/359156022 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74dc12f8b0 msgr2=0x7f74dc09c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:27.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.189+0000 7f74db7fe700 1 --2- 172.21.15.3:0/359156022 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74dc12f8b0 0x7f74dc09c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.189+0000 7f74db7fe700 1 -- 172.21.15.3:0/359156022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d00094a0 con 0x7f74dc12cf60 2023-10-03T10:16:27.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.190+0000 7f74db7fe700 1 --2- 172.21.15.3:0/359156022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74dc12cf60 0x7f74dc09bbb0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f74cc007370 tx=0x7f74cc007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:27.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.190+0000 7f74c3fff700 1 -- 172.21.15.3:0/359156022 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc005c30 con 0x7f74dc12cf60 2023-10-03T10:16:27.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.190+0000 7f74c3fff700 1 -- 172.21.15.3:0/359156022 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74cc005d90 con 0x7f74dc12cf60 2023-10-03T10:16:27.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.190+0000 7f74e1c1f700 1 -- 172.21.15.3:0/359156022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74dc09ead0 con 0x7f74dc12cf60 2023-10-03T10:16:27.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.190+0000 7f74c3fff700 1 -- 172.21.15.3:0/359156022 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc006250 con 0x7f74dc12cf60 2023-10-03T10:16:27.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.190+0000 7f74e1c1f700 1 -- 172.21.15.3:0/359156022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74dc09d690 con 0x7f74dc12cf60 2023-10-03T10:16:27.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.191+0000 7f74c3fff700 1 -- 172.21.15.3:0/359156022 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f74cc0063b0 con 0x7f74dc12cf60 2023-10-03T10:16:27.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.191+0000 7f74c3fff700 1 -- 172.21.15.3:0/359156022 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f74cc03f870 con 0x7f74dc12cf60 2023-10-03T10:16:27.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.191+0000 7f74e1c1f700 1 -- 172.21.15.3:0/359156022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74dc074ea0 con 0x7f74dc12cf60 2023-10-03T10:16:27.197 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:27.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.197+0000 7f74c3fff700 1 -- 172.21.15.3:0/359156022 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f74cc012340 con 0x7f74dc12cf60 2023-10-03T10:16:27.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.322+0000 7f5e866d0700 1 -- 172.21.15.3:0/2025604083 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5e800a0000 con 0x7f5e80133420 2023-10-03T10:16:27.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.322+0000 7f5e7cff9700 1 -- 172.21.15.3:0/2025604083 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5e70017030 con 0x7f5e80133420 2023-10-03T10:16:27.323 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:27.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.324+0000 7f5e866d0700 1 -- 172.21.15.3:0/2025604083 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e80133420 msgr2=0x7f5e800709a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:27.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.324+0000 7f5e866d0700 1 --2- 172.21.15.3:0/2025604083 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e80133420 0x7f5e800709a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000c4e0 tx=0x7f5e7000c7f0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.325+0000 7f5e866d0700 1 -- 172.21.15.3:0/2025604083 shutdown_connections 2023-10-03T10:16:27.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.325+0000 7f5e866d0700 1 --2- 172.21.15.3:0/2025604083 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e80138dc0 0x7f5e80071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.325+0000 7f5e866d0700 1 --2- 172.21.15.3:0/2025604083 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e80133420 0x7f5e800709a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.325+0000 7f5e866d0700 1 --2- 172.21.15.3:0/2025604083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e801337f0 0x7f5e80070ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.325+0000 7f5e866d0700 1 -- 172.21.15.3:0/2025604083 >> 172.21.15.3:0/2025604083 conn(0x7f5e800a4210 msgr2=0x7f5e80131400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:27.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.325+0000 7f5e866d0700 1 -- 172.21.15.3:0/2025604083 shutdown_connections 2023-10-03T10:16:27.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.325+0000 7f5e866d0700 1 -- 172.21.15.3:0/2025604083 wait complete. 2023-10-03T10:16:27.339 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:27.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.372+0000 7fe0b2896700 1 -- 172.21.15.3:0/3121244973 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe0ac119640 con 0x7fe0ac09e410 2023-10-03T10:16:27.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.372+0000 7fe0a8ff9700 1 -- 172.21.15.3:0/3121244973 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe0a4011390 con 0x7fe0ac09e410 2023-10-03T10:16:27.373 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:27.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.375+0000 7fe0b2896700 1 -- 172.21.15.3:0/3121244973 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe0ac09e410 msgr2=0x7fe0ac1383a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:27.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.375+0000 7fe0b2896700 1 --2- 172.21.15.3:0/3121244973 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe0ac09e410 0x7fe0ac1383a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4003f50 tx=0x7fe0a4004ad0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.375+0000 7fe0b2896700 1 -- 172.21.15.3:0/3121244973 shutdown_connections 2023-10-03T10:16:27.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.375+0000 7fe0b2896700 1 --2- 172.21.15.3:0/3121244973 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe0ac09e410 0x7fe0ac1383a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.375+0000 7fe0b2896700 1 --2- 172.21.15.3:0/3121244973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0ac09d6f0 0x7fe0ac137920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.375+0000 7fe0b2896700 1 --2- 172.21.15.3:0/3121244973 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe0ac09e000 0x7fe0ac137e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.375+0000 7fe0b2896700 1 -- 172.21.15.3:0/3121244973 >> 172.21.15.3:0/3121244973 conn(0x7fe0ac066f50 msgr2=0x7fe0ac06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:27.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.375+0000 7fe0b2896700 1 -- 172.21.15.3:0/3121244973 shutdown_connections 2023-10-03T10:16:27.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.375+0000 7fe0b2896700 1 -- 172.21.15.3:0/3121244973 wait complete. 2023-10-03T10:16:27.388 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:27.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.406+0000 7f74e1c1f700 1 -- 172.21.15.3:0/359156022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f74dc0713a0 con 0x7f74dc12cf60 2023-10-03T10:16:27.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.407+0000 7f74c3fff700 1 -- 172.21.15.3:0/359156022 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f74cc01cb50 con 0x7f74dc12cf60 2023-10-03T10:16:27.407 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:27.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.409+0000 7f74e1c1f700 1 -- 172.21.15.3:0/359156022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74dc12cf60 msgr2=0x7f74dc09bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:27.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.409+0000 7f74e1c1f700 1 --2- 172.21.15.3:0/359156022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74dc12cf60 0x7f74dc09bbb0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f74cc007370 tx=0x7f74cc007680 comp rx=0 tx=0).stop 2023-10-03T10:16:27.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.409+0000 7f74e1c1f700 1 -- 172.21.15.3:0/359156022 shutdown_connections 2023-10-03T10:16:27.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.409+0000 7f74e1c1f700 1 --2- 172.21.15.3:0/359156022 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74dc130260 0x7f74dc1c3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.409+0000 7f74e1c1f700 1 --2- 172.21.15.3:0/359156022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74dc12cf60 0x7f74dc09bbb0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.409+0000 7f74e1c1f700 1 --2- 172.21.15.3:0/359156022 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74dc12f8b0 0x7f74dc09c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:27.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.409+0000 7f74e1c1f700 1 -- 172.21.15.3:0/359156022 >> 172.21.15.3:0/359156022 conn(0x7f74dc0a2aa0 msgr2=0x7f74dc12a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:27.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.409+0000 7f74e1c1f700 1 -- 172.21.15.3:0/359156022 shutdown_connections 2023-10-03T10:16:27.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:27.409+0000 7f74e1c1f700 1 -- 172.21.15.3:0/359156022 wait complete. 2023-10-03T10:16:27.423 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:28.199 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:28.340 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:28.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.345+0000 7f20acff3700 1 Processor -- start 2023-10-03T10:16:28.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.345+0000 7f20acff3700 1 -- start start 2023-10-03T10:16:28.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.346+0000 7f20acff3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f20a80a1510 0x7f20a8138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:28.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.346+0000 7f20acff3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f20a8139250 0x7f20a813b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:28.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.346+0000 7f20acff3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a809edd0 con 0x7f20a80a0a30 2023-10-03T10:16:28.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.346+0000 7f20acff3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a80a18e0 con 0x7f20a8139250 2023-10-03T10:16:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.346+0000 7f20acff3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20a809e1f0 con 0x7f20a80a1510 2023-10-03T10:16:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.346+0000 7f20a5d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f20a80a1510 0x7f20a8138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.346+0000 7f20a659c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f20a80a0a30 0x7f20a80a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35418/0 (socket says 172.21.15.3:35418) 2023-10-03T10:16:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.346+0000 7f20a6d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f20a8139250 0x7f20a813b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.346+0000 7f20a659c700 1 -- 172.21.15.3:0/3891813750 learned_addr learned my addr 172.21.15.3:0/3891813750 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.346+0000 7f20a559a700 1 -- 172.21.15.3:0/3891813750 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3337248560 0 0) 0x7f20a809edd0 con 0x7f20a80a0a30 2023-10-03T10:16:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.346+0000 7f20a559a700 1 -- 172.21.15.3:0/3891813750 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20900035b0 con 0x7f20a80a0a30 2023-10-03T10:16:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.346+0000 7f20a5d9b700 1 -- 172.21.15.3:0/3891813750 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f20a8139250 msgr2=0x7f20a813b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.346+0000 7f20a5d9b700 1 --2- 172.21.15.3:0/3891813750 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f20a8139250 0x7f20a813b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.346+0000 7f20a5d9b700 1 -- 172.21.15.3:0/3891813750 >> v1:172.21.15.3:6789/0 conn(0x7f20a80a0a30 legacy=0x7f20a80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.346+0000 7f20a5d9b700 1 -- 172.21.15.3:0/3891813750 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a809dc50 con 0x7f20a80a1510 2023-10-03T10:16:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.346+0000 7f20a6d9d700 1 --2- 172.21.15.3:0/3891813750 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f20a8139250 0x7f20a813b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:28.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.347+0000 7f20a5d9b700 1 --2- 172.21.15.3:0/3891813750 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f20a80a1510 0x7f20a8138d10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f209c009870 tx=0x7f209c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=255593ad954f223f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:28.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.347+0000 7f20a559a700 1 -- 172.21.15.3:0/3891813750 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c011830 con 0x7f20a80a1510 2023-10-03T10:16:28.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.347+0000 7f20a559a700 1 -- 172.21.15.3:0/3891813750 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209c011990 con 0x7f20a80a1510 2023-10-03T10:16:28.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.347+0000 7f20a559a700 1 -- 172.21.15.3:0/3891813750 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c011d20 con 0x7f20a80a1510 2023-10-03T10:16:28.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.347+0000 7f20acff3700 1 -- 172.21.15.3:0/3891813750 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f20a80a1510 msgr2=0x7f20a8138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:28.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.347+0000 7f20acff3700 1 --2- 172.21.15.3:0/3891813750 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f20a80a1510 0x7f20a8138d10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f209c009870 tx=0x7f209c009b80 comp rx=0 tx=0).stop 2023-10-03T10:16:28.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.348+0000 7f20acff3700 1 -- 172.21.15.3:0/3891813750 shutdown_connections 2023-10-03T10:16:28.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.348+0000 7f20acff3700 1 --2- 172.21.15.3:0/3891813750 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f20a8139250 0x7f20a813b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.348+0000 7f20acff3700 1 --2- 172.21.15.3:0/3891813750 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f20a80a1510 0x7f20a8138d10 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.348+0000 7f20acff3700 1 -- 172.21.15.3:0/3891813750 >> 172.21.15.3:0/3891813750 conn(0x7f20a80a6550 msgr2=0x7f20a80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:28.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.348+0000 7f20acff3700 1 -- 172.21.15.3:0/3891813750 shutdown_connections 2023-10-03T10:16:28.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.348+0000 7f20acff3700 1 -- 172.21.15.3:0/3891813750 wait complete. 2023-10-03T10:16:28.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.349+0000 7f20acff3700 1 Processor -- start 2023-10-03T10:16:28.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.349+0000 7f20acff3700 1 -- start start 2023-10-03T10:16:28.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.349+0000 7f20acff3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20a80a0a30 0x7f20a81c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:28.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.349+0000 7f20acff3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f20a80a1510 0x7f20a81c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:28.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.350+0000 7f20acff3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f20a8139250 0x7f20a81216a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:28.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.350+0000 7f20acff3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a8124f20 con 0x7f20a80a0a30 2023-10-03T10:16:28.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.350+0000 7f20acff3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a81be330 con 0x7f20a8139250 2023-10-03T10:16:28.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.350+0000 7f20acff3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20a8125170 con 0x7f20a80a1510 2023-10-03T10:16:28.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.350+0000 7f20a5d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f20a80a1510 0x7f20a81c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:28.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.350+0000 7f20a6d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f20a8139250 0x7f20a81216a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:28.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.350+0000 7f20a5d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f20a80a1510 0x7f20a81c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51524/0 (socket says 172.21.15.3:51524) 2023-10-03T10:16:28.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.350+0000 7f20a5d9b700 1 -- 172.21.15.3:0/3011041253 learned_addr learned my addr 172.21.15.3:0/3011041253 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:28.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.350+0000 7f20a659c700 1 --2- 172.21.15.3:0/3011041253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20a80a0a30 0x7f20a81c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:28.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.350+0000 7f20a5d9b700 1 -- 172.21.15.3:0/3011041253 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f20a8139250 msgr2=0x7f20a81216a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:28.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.350+0000 7f20a5d9b700 1 --2- 172.21.15.3:0/3011041253 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f20a8139250 0x7f20a81216a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.350+0000 7f20a5d9b700 1 -- 172.21.15.3:0/3011041253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20a80a0a30 msgr2=0x7f20a81c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:28.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.350+0000 7f20a5d9b700 1 --2- 172.21.15.3:0/3011041253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20a80a0a30 0x7f20a81c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.350+0000 7f20a5d9b700 1 -- 172.21.15.3:0/3011041253 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a0009a70 con 0x7f20a80a1510 2023-10-03T10:16:28.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.350+0000 7f20a5d9b700 1 --2- 172.21.15.3:0/3011041253 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f20a80a1510 0x7f20a81c8aa0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f209c000c00 tx=0x7f209c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:28.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.350+0000 7f2096ffd700 1 -- 172.21.15.3:0/3011041253 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c004450 con 0x7f20a80a1510 2023-10-03T10:16:28.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.350+0000 7f2096ffd700 1 -- 172.21.15.3:0/3011041253 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209c01f070 con 0x7f20a80a1510 2023-10-03T10:16:28.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.350+0000 7f20acff3700 1 -- 172.21.15.3:0/3011041253 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f209c0094a0 con 0x7f20a80a1510 2023-10-03T10:16:28.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.351+0000 7f2096ffd700 1 -- 172.21.15.3:0/3011041253 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c0038e0 con 0x7f20a80a1510 2023-10-03T10:16:28.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.351+0000 7f20acff3700 1 -- 172.21.15.3:0/3011041253 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a80a0220 con 0x7f20a80a1510 2023-10-03T10:16:28.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.351+0000 7f2096ffd700 1 -- 172.21.15.3:0/3011041253 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f209c003a40 con 0x7f20a80a1510 2023-10-03T10:16:28.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.352+0000 7f2096ffd700 1 -- 172.21.15.3:0/3011041253 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f209c021070 con 0x7f20a80a1510 2023-10-03T10:16:28.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.352+0000 7f20acff3700 1 -- 172.21.15.3:0/3011041253 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a8074ea0 con 0x7f20a80a1510 2023-10-03T10:16:28.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.357+0000 7f2096ffd700 1 -- 172.21.15.3:0/3011041253 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f209c01a430 con 0x7f20a80a1510 2023-10-03T10:16:28.389 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:28.423 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:28.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.483+0000 7ff4840af700 1 Processor -- start 2023-10-03T10:16:28.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.484+0000 7ff4840af700 1 -- start start 2023-10-03T10:16:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.484+0000 7ff4840af700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff47c09d6f0 0x7ff47c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.484+0000 7ff4840af700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff47c09e000 0x7ff47c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.484+0000 7ff4840af700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff47c09e410 0x7ff47c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.484+0000 7ff4840af700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff47c06b460 con 0x7ff47c09e000 2023-10-03T10:16:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.484+0000 7ff4840af700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c09b120 con 0x7ff47c09e410 2023-10-03T10:16:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.484+0000 7ff4840af700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c09a7e0 con 0x7ff47c09d6f0 2023-10-03T10:16:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.484+0000 7ff481e4b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff47c09d6f0 0x7ff47c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.485+0000 7ff481e4b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff47c09d6f0 0x7ff47c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51534/0 (socket says 172.21.15.3:51534) 2023-10-03T10:16:28.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.485+0000 7ff481e4b700 1 -- 172.21.15.3:0/1393881489 learned_addr learned my addr 172.21.15.3:0/1393881489 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:28.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.485+0000 7ff48164a700 1 --2- 172.21.15.3:0/1393881489 >> v2:172.21.15.3:3300/0 conn(0x7ff47c09e000 0x7ff47c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:28.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.485+0000 7ff48264c700 1 --2- 172.21.15.3:0/1393881489 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff47c09e410 0x7ff47c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:28.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.485+0000 7ff48164a700 1 -- 172.21.15.3:0/1393881489 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff47c09d6f0 msgr2=0x7ff47c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:28.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.485+0000 7ff48164a700 1 --2- 172.21.15.3:0/1393881489 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff47c09d6f0 0x7ff47c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.485+0000 7ff48164a700 1 -- 172.21.15.3:0/1393881489 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff47c09e410 msgr2=0x7ff47c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:28.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.485+0000 7ff48164a700 1 --2- 172.21.15.3:0/1393881489 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff47c09e410 0x7ff47c133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.485+0000 7ff48164a700 1 -- 172.21.15.3:0/1393881489 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47c09a500 con 0x7ff47c09e000 2023-10-03T10:16:28.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.485+0000 7ff481e4b700 1 --2- 172.21.15.3:0/1393881489 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff47c09d6f0 0x7ff47c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:28.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.485+0000 7ff48164a700 1 --2- 172.21.15.3:0/1393881489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff47c09e000 0x7ff47c1339f0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7ff46c009840 tx=0x7ff46c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b2c20b54f7fea6 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:28.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.485+0000 7ff480e49700 1 -- 172.21.15.3:0/1393881489 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c011830 con 0x7ff47c09e000 2023-10-03T10:16:28.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.485+0000 7ff480e49700 1 -- 172.21.15.3:0/1393881489 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46c011990 con 0x7ff47c09e000 2023-10-03T10:16:28.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.485+0000 7ff480e49700 1 -- 172.21.15.3:0/1393881489 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c011d20 con 0x7ff47c09e000 2023-10-03T10:16:28.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.486+0000 7ff4840af700 1 -- 172.21.15.3:0/1393881489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff47c09e000 msgr2=0x7ff47c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:28.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.486+0000 7ff4840af700 1 --2- 172.21.15.3:0/1393881489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff47c09e000 0x7ff47c1339f0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7ff46c009840 tx=0x7ff46c009b50 comp rx=0 tx=0).stop 2023-10-03T10:16:28.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.486+0000 7ff4840af700 1 -- 172.21.15.3:0/1393881489 shutdown_connections 2023-10-03T10:16:28.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.486+0000 7ff4840af700 1 --2- 172.21.15.3:0/1393881489 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff47c09e410 0x7ff47c133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.486+0000 7ff4840af700 1 --2- 172.21.15.3:0/1393881489 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff47c09d6f0 0x7ff47c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.486+0000 7ff4840af700 1 --2- 172.21.15.3:0/1393881489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff47c09e000 0x7ff47c1339f0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.486+0000 7ff4840af700 1 -- 172.21.15.3:0/1393881489 >> 172.21.15.3:0/1393881489 conn(0x7ff47c066f50 msgr2=0x7ff47c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:28.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.486+0000 7ff4840af700 1 -- 172.21.15.3:0/1393881489 shutdown_connections 2023-10-03T10:16:28.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.486+0000 7ff4840af700 1 -- 172.21.15.3:0/1393881489 wait complete. 2023-10-03T10:16:28.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.487+0000 7ff4840af700 1 Processor -- start 2023-10-03T10:16:28.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.487+0000 7ff4840af700 1 -- start start 2023-10-03T10:16:28.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.488+0000 7ff4840af700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff47c09d6f0 0x7ff47c1a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:28.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.488+0000 7ff4840af700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff47c09e000 0x7ff47c1a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:28.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.488+0000 7ff4840af700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff47c09e410 0x7ff47c1a6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:28.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.488+0000 7ff4840af700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c1a9870 con 0x7ff47c09e410 2023-10-03T10:16:28.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.488+0000 7ff4840af700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c1aa210 con 0x7ff47c09d6f0 2023-10-03T10:16:28.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.488+0000 7ff4840af700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c0a1f90 con 0x7ff47c09e000 2023-10-03T10:16:28.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.488+0000 7ff48164a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff47c09e000 0x7ff47c1a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:28.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.488+0000 7ff48164a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff47c09e000 0x7ff47c1a6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51544/0 (socket says 172.21.15.3:51544) 2023-10-03T10:16:28.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.488+0000 7ff48164a700 1 -- 172.21.15.3:0/613920738 learned_addr learned my addr 172.21.15.3:0/613920738 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:28.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.488+0000 7ff48264c700 1 --2- 172.21.15.3:0/613920738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff47c09e410 0x7ff47c1a6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:28.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.488+0000 7ff48164a700 1 -- 172.21.15.3:0/613920738 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff47c09d6f0 msgr2=0x7ff47c1a5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:28.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.488+0000 7ff48164a700 1 --2- 172.21.15.3:0/613920738 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff47c09d6f0 0x7ff47c1a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.488+0000 7ff48164a700 1 -- 172.21.15.3:0/613920738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff47c09e410 msgr2=0x7ff47c1a6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:28.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.488+0000 7ff48164a700 1 --2- 172.21.15.3:0/613920738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff47c09e410 0x7ff47c1a6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.488+0000 7ff48164a700 1 -- 172.21.15.3:0/613920738 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff478009550 con 0x7ff47c09e000 2023-10-03T10:16:28.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.489+0000 7ff48264c700 1 --2- 172.21.15.3:0/613920738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff47c09e410 0x7ff47c1a6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:28.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.489+0000 7ff48164a700 1 --2- 172.21.15.3:0/613920738 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff47c09e000 0x7ff47c1a6330 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff46c009fd0 tx=0x7ff46c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:28.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.489+0000 7ff4727fc700 1 -- 172.21.15.3:0/613920738 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c004510 con 0x7ff47c09e000 2023-10-03T10:16:28.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.489+0000 7ff4727fc700 1 -- 172.21.15.3:0/613920738 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46c01f070 con 0x7ff47c09e000 2023-10-03T10:16:28.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.489+0000 7ff4840af700 1 -- 172.21.15.3:0/613920738 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff46c0094a0 con 0x7ff47c09e000 2023-10-03T10:16:28.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.489+0000 7ff4727fc700 1 -- 172.21.15.3:0/613920738 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c0038e0 con 0x7ff47c09e000 2023-10-03T10:16:28.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.489+0000 7ff4840af700 1 -- 172.21.15.3:0/613920738 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff47c1a9050 con 0x7ff47c09e000 2023-10-03T10:16:28.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.490+0000 7ff4727fc700 1 -- 172.21.15.3:0/613920738 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff46c003a40 con 0x7ff47c09e000 2023-10-03T10:16:28.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.490+0000 7ff4727fc700 1 -- 172.21.15.3:0/613920738 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff46c021070 con 0x7ff47c09e000 2023-10-03T10:16:28.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.490+0000 7ff4840af700 1 -- 172.21.15.3:0/613920738 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff47c074ea0 con 0x7ff47c09e000 2023-10-03T10:16:28.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.500+0000 7ff4727fc700 1 -- 172.21.15.3:0/613920738 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff46c015020 con 0x7ff47c09e000 2023-10-03T10:16:28.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.536+0000 7f440ba11700 1 Processor -- start 2023-10-03T10:16:28.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.536+0000 7f440ba11700 1 -- start start 2023-10-03T10:16:28.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.536+0000 7f440ba11700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f440409d6f0 0x7f440409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:28.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.537+0000 7f440ba11700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f440409e410 0x7f4404130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:28.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.537+0000 7f440ba11700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f440409f130 con 0x7f440409e000 2023-10-03T10:16:28.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.537+0000 7f440ba11700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f440409f520 con 0x7f440409d6f0 2023-10-03T10:16:28.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.537+0000 7f440ba11700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f440409e930 con 0x7f440409e410 2023-10-03T10:16:28.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.537+0000 7f44097ad700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f440409d6f0 0x7f440409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:28.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.537+0000 7f4409fae700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f440409e410 0x7f4404130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:28.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.537+0000 7f44097ad700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f440409d6f0 0x7f440409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42656/0 (socket says 172.21.15.3:42656) 2023-10-03T10:16:28.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.537+0000 7f4409fae700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f440409e410 0x7f4404130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51550/0 (socket says 172.21.15.3:51550) 2023-10-03T10:16:28.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.537+0000 7f44097ad700 1 -- 172.21.15.3:0/3163058708 learned_addr learned my addr 172.21.15.3:0/3163058708 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:28.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.537+0000 7f4409fae700 1 -- 172.21.15.3:0/3163058708 >> v2:172.21.15.3:3300/0 conn(0x7f440409d6f0 msgr2=0x7f440409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:28.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.537+0000 7f4409fae700 1 --2- 172.21.15.3:0/3163058708 >> v2:172.21.15.3:3300/0 conn(0x7f440409d6f0 0x7f440409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.537+0000 7f4409fae700 1 -- 172.21.15.3:0/3163058708 >> v1:172.21.15.3:6789/0 conn(0x7f440409e000 legacy=0x7f4404130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:28.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.537+0000 7f4409fae700 1 -- 172.21.15.3:0/3163058708 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440409b2d0 con 0x7f440409e410 2023-10-03T10:16:28.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.537+0000 7f4409fae700 1 --2- 172.21.15.3:0/3163058708 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f440409e410 0x7f4404130d70 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009d90 tx=0x7f43fc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=945b9f34c4f6880b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:28.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.538+0000 7f43fbfff700 1 -- 172.21.15.3:0/3163058708 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc00e070 con 0x7f440409e410 2023-10-03T10:16:28.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.538+0000 7f43fbfff700 1 -- 172.21.15.3:0/3163058708 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43fc00bb20 con 0x7f440409e410 2023-10-03T10:16:28.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.538+0000 7f43fbfff700 1 -- 172.21.15.3:0/3163058708 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc012430 con 0x7f440409e410 2023-10-03T10:16:28.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.538+0000 7f440ba11700 1 -- 172.21.15.3:0/3163058708 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f440409e410 msgr2=0x7f4404130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:28.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.538+0000 7f440ba11700 1 --2- 172.21.15.3:0/3163058708 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f440409e410 0x7f4404130d70 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009d90 tx=0x7f43fc00b330 comp rx=0 tx=0).stop 2023-10-03T10:16:28.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.538+0000 7f440ba11700 1 -- 172.21.15.3:0/3163058708 shutdown_connections 2023-10-03T10:16:28.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.538+0000 7f440ba11700 1 --2- 172.21.15.3:0/3163058708 >> v2:172.21.15.3:3300/0 conn(0x7f440409d6f0 0x7f440409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.538+0000 7f440ba11700 1 --2- 172.21.15.3:0/3163058708 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f440409e410 0x7f4404130d70 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.539+0000 7f440ba11700 1 -- 172.21.15.3:0/3163058708 >> 172.21.15.3:0/3163058708 conn(0x7f4404066f50 msgr2=0x7f440409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:28.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.539+0000 7f440ba11700 1 -- 172.21.15.3:0/3163058708 shutdown_connections 2023-10-03T10:16:28.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.539+0000 7f440ba11700 1 -- 172.21.15.3:0/3163058708 wait complete. 2023-10-03T10:16:28.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.540+0000 7f440ba11700 1 Processor -- start 2023-10-03T10:16:28.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.540+0000 7f440ba11700 1 -- start start 2023-10-03T10:16:28.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.540+0000 7f440ba11700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f440409d6f0 0x7f4404131640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:28.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.540+0000 7f440ba11700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f440409e000 0x7f4404131b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:28.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.540+0000 7f440ba11700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f440409e410 0x7f44041320c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:28.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.540+0000 7f440ba11700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4404132bb0 con 0x7f440409e410 2023-10-03T10:16:28.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.540+0000 7f440ba11700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44041329c0 con 0x7f440409d6f0 2023-10-03T10:16:28.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.540+0000 7f440ba11700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f44041337c0 con 0x7f440409e000 2023-10-03T10:16:28.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.540+0000 7f4409fae700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f440409e410 0x7f44041320c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:28.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.540+0000 7f4409fae700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f440409e410 0x7f44041320c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42668/0 (socket says 172.21.15.3:42668) 2023-10-03T10:16:28.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.540+0000 7f4409fae700 1 -- 172.21.15.3:0/1913210952 learned_addr learned my addr 172.21.15.3:0/1913210952 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:28.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.541+0000 7f4408fac700 1 --2- 172.21.15.3:0/1913210952 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f440409e000 0x7f4404131b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:28.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.541+0000 7f4409fae700 1 -- 172.21.15.3:0/1913210952 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f440409e000 msgr2=0x7f4404131b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:28.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.541+0000 7f4409fae700 1 --2- 172.21.15.3:0/1913210952 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f440409e000 0x7f4404131b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.541+0000 7f4409fae700 1 -- 172.21.15.3:0/1913210952 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f440409d6f0 msgr2=0x7f4404131640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:28.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.541+0000 7f4409fae700 1 --2- 172.21.15.3:0/1913210952 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f440409d6f0 0x7f4404131640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.541+0000 7f4409fae700 1 -- 172.21.15.3:0/1913210952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43fc009a70 con 0x7f440409e410 2023-10-03T10:16:28.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.541+0000 7f4408fac700 1 --2- 172.21.15.3:0/1913210952 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f440409e000 0x7f4404131b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:16:28.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.541+0000 7f4409fae700 1 --2- 172.21.15.3:0/1913210952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f440409e410 0x7f44041320c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f440409ea70 tx=0x7f43fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:28.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.541+0000 7f43f9ffb700 1 -- 172.21.15.3:0/1913210952 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc012430 con 0x7f440409e410 2023-10-03T10:16:28.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.541+0000 7f43f9ffb700 1 -- 172.21.15.3:0/1913210952 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43fc020070 con 0x7f440409e410 2023-10-03T10:16:28.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.541+0000 7f43f9ffb700 1 -- 172.21.15.3:0/1913210952 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc003ea0 con 0x7f440409e410 2023-10-03T10:16:28.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.541+0000 7f440ba11700 1 -- 172.21.15.3:0/1913210952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4404072370 con 0x7f440409e410 2023-10-03T10:16:28.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.542+0000 7f440ba11700 1 -- 172.21.15.3:0/1913210952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4404133190 con 0x7f440409e410 2023-10-03T10:16:28.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.543+0000 7f43f9ffb700 1 -- 172.21.15.3:0/1913210952 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f43fc004280 con 0x7f440409e410 2023-10-03T10:16:28.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.543+0000 7f440ba11700 1 -- 172.21.15.3:0/1913210952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4404074ea0 con 0x7f440409e410 2023-10-03T10:16:28.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.543+0000 7f43f9ffb700 1 -- 172.21.15.3:0/1913210952 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f43fc022070 con 0x7f440409e410 2023-10-03T10:16:28.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.548+0000 7f43f9ffb700 1 -- 172.21.15.3:0/1913210952 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f43fc004530 con 0x7f440409e410 2023-10-03T10:16:28.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.579+0000 7f20acff3700 1 -- 172.21.15.3:0/3011041253 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f20a8137520 con 0x7f20a80a1510 2023-10-03T10:16:28.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.580+0000 7f2096ffd700 1 -- 172.21.15.3:0/3011041253 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f209c015030 con 0x7f20a80a1510 2023-10-03T10:16:28.580 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:28.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.582+0000 7f20acff3700 1 -- 172.21.15.3:0/3011041253 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f20a80a1510 msgr2=0x7f20a81c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:28.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.582+0000 7f20acff3700 1 --2- 172.21.15.3:0/3011041253 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f20a80a1510 0x7f20a81c8aa0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f209c000c00 tx=0x7f209c004750 comp rx=0 tx=0).stop 2023-10-03T10:16:28.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.582+0000 7f20acff3700 1 -- 172.21.15.3:0/3011041253 shutdown_connections 2023-10-03T10:16:28.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.582+0000 7f20acff3700 1 --2- 172.21.15.3:0/3011041253 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f20a8139250 0x7f20a81216a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.582+0000 7f20acff3700 1 --2- 172.21.15.3:0/3011041253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20a80a0a30 0x7f20a81c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.582+0000 7f20acff3700 1 --2- 172.21.15.3:0/3011041253 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f20a80a1510 0x7f20a81c8aa0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.582+0000 7f20acff3700 1 -- 172.21.15.3:0/3011041253 >> 172.21.15.3:0/3011041253 conn(0x7f20a80a6550 msgr2=0x7f20a8137b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:28.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.582+0000 7f20acff3700 1 -- 172.21.15.3:0/3011041253 shutdown_connections 2023-10-03T10:16:28.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.582+0000 7f20acff3700 1 -- 172.21.15.3:0/3011041253 wait complete. 2023-10-03T10:16:28.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.595+0000 7f5ddca4b700 1 Processor -- start 2023-10-03T10:16:28.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.595+0000 7f5ddca4b700 1 -- start start 2023-10-03T10:16:28.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.596+0000 7f5ddca4b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5dd809d6f0 0x7f5dd809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:28.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.596+0000 7f5ddca4b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5dd809e000 0x7f5dd81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:28.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.596+0000 7f5ddca4b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dd809b120 con 0x7f5dd809e410 2023-10-03T10:16:28.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.596+0000 7f5ddca4b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd806b460 con 0x7f5dd809e000 2023-10-03T10:16:28.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.596+0000 7f5ddca4b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd809a7e0 con 0x7f5dd809d6f0 2023-10-03T10:16:28.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.596+0000 7f5dd659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5dd809d6f0 0x7f5dd809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:28.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.596+0000 7f5dd659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5dd809d6f0 0x7f5dd809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37888/0 (socket says 172.21.15.3:37888) 2023-10-03T10:16:28.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.596+0000 7f5dd659c700 1 -- 172.21.15.3:0/4158496335 learned_addr learned my addr 172.21.15.3:0/4158496335 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:28.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.596+0000 7f5dd5d9b700 1 --2- 172.21.15.3:0/4158496335 >> v2:172.21.15.3:3300/0 conn(0x7f5dd809e000 0x7f5dd81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:28.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.596+0000 7f5dd659c700 1 -- 172.21.15.3:0/4158496335 >> v2:172.21.15.3:3300/0 conn(0x7f5dd809e000 msgr2=0x7f5dd81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:28.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.596+0000 7f5dd659c700 1 --2- 172.21.15.3:0/4158496335 >> v2:172.21.15.3:3300/0 conn(0x7f5dd809e000 0x7f5dd81339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.596+0000 7f5dd659c700 1 -- 172.21.15.3:0/4158496335 >> v1:172.21.15.3:6789/0 conn(0x7f5dd809e410 legacy=0x7f5dd8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:28.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.596+0000 7f5dd659c700 1 -- 172.21.15.3:0/4158496335 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd809a500 con 0x7f5dd809d6f0 2023-10-03T10:16:28.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.597+0000 7f5dd659c700 1 --2- 172.21.15.3:0/4158496335 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5dd809d6f0 0x7f5dd809dac0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8009920 tx=0x7f5dc8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d04014667cf9bf3b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:28.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.597+0000 7f5dd559a700 1 -- 172.21.15.3:0/4158496335 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8011830 con 0x7f5dd809d6f0 2023-10-03T10:16:28.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.597+0000 7f5dd559a700 1 -- 172.21.15.3:0/4158496335 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc8011990 con 0x7f5dd809d6f0 2023-10-03T10:16:28.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.597+0000 7f5dd559a700 1 -- 172.21.15.3:0/4158496335 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8011d20 con 0x7f5dd809d6f0 2023-10-03T10:16:28.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.598+0000 7f5ddca4b700 1 -- 172.21.15.3:0/4158496335 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5dd809d6f0 msgr2=0x7f5dd809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:28.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.598+0000 7f5ddca4b700 1 --2- 172.21.15.3:0/4158496335 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5dd809d6f0 0x7f5dd809dac0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8009920 tx=0x7f5dc8009c30 comp rx=0 tx=0).stop 2023-10-03T10:16:28.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.598+0000 7f5ddca4b700 1 -- 172.21.15.3:0/4158496335 shutdown_connections 2023-10-03T10:16:28.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.598+0000 7f5ddca4b700 1 --2- 172.21.15.3:0/4158496335 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5dd809d6f0 0x7f5dd809dac0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.598+0000 7f5ddca4b700 1 --2- 172.21.15.3:0/4158496335 >> v2:172.21.15.3:3300/0 conn(0x7f5dd809e000 0x7f5dd81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.598+0000 7f5ddca4b700 1 -- 172.21.15.3:0/4158496335 >> 172.21.15.3:0/4158496335 conn(0x7f5dd8066f50 msgr2=0x7f5dd809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:28.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.598+0000 7f5ddca4b700 1 -- 172.21.15.3:0/4158496335 shutdown_connections 2023-10-03T10:16:28.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.598+0000 7f5ddca4b700 1 -- 172.21.15.3:0/4158496335 wait complete. 2023-10-03T10:16:28.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.599+0000 7f5ddca4b700 1 Processor -- start 2023-10-03T10:16:28.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.599+0000 7f5ddca4b700 1 -- start start 2023-10-03T10:16:28.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.600+0000 7f5ddca4b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5dd809d6f0 0x7f5dd81a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:28.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.600+0000 7f5ddca4b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5dd809e000 0x7f5dd81a6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:28.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.600+0000 7f5ddca4b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dd809e410 0x7f5dd81a6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:28.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.600+0000 7f5ddca4b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd81a7cb0 con 0x7f5dd809e410 2023-10-03T10:16:28.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.600+0000 7f5ddca4b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd81a9820 con 0x7f5dd809e000 2023-10-03T10:16:28.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.600+0000 7f5ddca4b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd80a3380 con 0x7f5dd809d6f0 2023-10-03T10:16:28.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.600+0000 7f5dd659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5dd809d6f0 0x7f5dd81a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:28.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.600+0000 7f5dd659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5dd809d6f0 0x7f5dd81a60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51564/0 (socket says 172.21.15.3:51564) 2023-10-03T10:16:28.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.600+0000 7f5dd659c700 1 -- 172.21.15.3:0/1643474775 learned_addr learned my addr 172.21.15.3:0/1643474775 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:28.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.600+0000 7f5dd5d9b700 1 --2- 172.21.15.3:0/1643474775 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5dd809e000 0x7f5dd81a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:28.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.600+0000 7f5dd6d9d700 1 --2- 172.21.15.3:0/1643474775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dd809e410 0x7f5dd81a6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:28.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.600+0000 7f5dd659c700 1 -- 172.21.15.3:0/1643474775 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5dd809e000 msgr2=0x7f5dd81a6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:28.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.600+0000 7f5dd659c700 1 --2- 172.21.15.3:0/1643474775 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5dd809e000 0x7f5dd81a6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.600+0000 7f5dd659c700 1 -- 172.21.15.3:0/1643474775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dd809e410 msgr2=0x7f5dd81a6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:28.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.600+0000 7f5dd659c700 1 --2- 172.21.15.3:0/1643474775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dd809e410 0x7f5dd81a6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.600+0000 7f5dd659c700 1 -- 172.21.15.3:0/1643474775 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc8009550 con 0x7f5dd809d6f0 2023-10-03T10:16:28.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.600+0000 7f5dd6d9d700 1 --2- 172.21.15.3:0/1643474775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dd809e410 0x7f5dd81a6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:16:28.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.601+0000 7f5dd659c700 1 --2- 172.21.15.3:0/1643474775 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5dd809d6f0 0x7f5dd81a60f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8009c70 tx=0x7f5dc8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:28.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.601+0000 7f5dc6ffd700 1 -- 172.21.15.3:0/1643474775 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8004300 con 0x7f5dd809d6f0 2023-10-03T10:16:28.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.601+0000 7f5dc6ffd700 1 -- 172.21.15.3:0/1643474775 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc801f070 con 0x7f5dd809d6f0 2023-10-03T10:16:28.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.601+0000 7f5ddca4b700 1 -- 172.21.15.3:0/1643474775 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd809c710 con 0x7f5dd809d6f0 2023-10-03T10:16:28.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.601+0000 7f5dc6ffd700 1 -- 172.21.15.3:0/1643474775 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc80036a0 con 0x7f5dd809d6f0 2023-10-03T10:16:28.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.601+0000 7f5ddca4b700 1 -- 172.21.15.3:0/1643474775 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd81a9b00 con 0x7f5dd809d6f0 2023-10-03T10:16:28.604 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:28.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.602+0000 7f5dc6ffd700 1 -- 172.21.15.3:0/1643474775 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5dc8021030 con 0x7f5dd809d6f0 2023-10-03T10:16:28.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.602+0000 7f5ddca4b700 1 -- 172.21.15.3:0/1643474775 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd8074ea0 con 0x7f5dd809d6f0 2023-10-03T10:16:28.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.602+0000 7f5dc6ffd700 1 -- 172.21.15.3:0/1643474775 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5dc8040970 con 0x7f5dd809d6f0 2023-10-03T10:16:28.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.608+0000 7f5dc6ffd700 1 -- 172.21.15.3:0/1643474775 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5dc8019770 con 0x7f5dd809d6f0 2023-10-03T10:16:28.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.721+0000 7ff4840af700 1 -- 172.21.15.3:0/613920738 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff47c1a8bb0 con 0x7ff47c09e000 2023-10-03T10:16:28.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.721+0000 7ff4727fc700 1 -- 172.21.15.3:0/613920738 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff46c0253e0 con 0x7ff47c09e000 2023-10-03T10:16:28.721 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:28.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.723+0000 7ff4840af700 1 -- 172.21.15.3:0/613920738 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff47c09e000 msgr2=0x7ff47c1a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:28.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.723+0000 7ff4840af700 1 --2- 172.21.15.3:0/613920738 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff47c09e000 0x7ff47c1a6330 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff46c009fd0 tx=0x7ff46c004750 comp rx=0 tx=0).stop 2023-10-03T10:16:28.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.723+0000 7ff4840af700 1 -- 172.21.15.3:0/613920738 shutdown_connections 2023-10-03T10:16:28.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.723+0000 7ff4840af700 1 --2- 172.21.15.3:0/613920738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff47c09e410 0x7ff47c1a6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.723+0000 7ff4840af700 1 --2- 172.21.15.3:0/613920738 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff47c09d6f0 0x7ff47c1a5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.723+0000 7ff4840af700 1 --2- 172.21.15.3:0/613920738 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff47c09e000 0x7ff47c1a6330 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.723+0000 7ff4840af700 1 -- 172.21.15.3:0/613920738 >> 172.21.15.3:0/613920738 conn(0x7ff47c066f50 msgr2=0x7ff47c1bae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:28.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.723+0000 7ff4840af700 1 -- 172.21.15.3:0/613920738 shutdown_connections 2023-10-03T10:16:28.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.723+0000 7ff4840af700 1 -- 172.21.15.3:0/613920738 wait complete. 2023-10-03T10:16:28.737 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:28.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.779+0000 7f440ba11700 1 -- 172.21.15.3:0/1913210952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f440406c360 con 0x7f440409e410 2023-10-03T10:16:28.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.779+0000 7f43f9ffb700 1 -- 172.21.15.3:0/1913210952 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f43fc02c070 con 0x7f440409e410 2023-10-03T10:16:28.780 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:28.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.782+0000 7f440ba11700 1 -- 172.21.15.3:0/1913210952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f440409e410 msgr2=0x7f44041320c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:28.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.782+0000 7f440ba11700 1 --2- 172.21.15.3:0/1913210952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f440409e410 0x7f44041320c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f440409ea70 tx=0x7f43fc004ad0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.782+0000 7f440ba11700 1 -- 172.21.15.3:0/1913210952 shutdown_connections 2023-10-03T10:16:28.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.782+0000 7f440ba11700 1 --2- 172.21.15.3:0/1913210952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f440409e410 0x7f44041320c0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.782+0000 7f440ba11700 1 --2- 172.21.15.3:0/1913210952 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f440409d6f0 0x7f4404131640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.782+0000 7f440ba11700 1 --2- 172.21.15.3:0/1913210952 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f440409e000 0x7f4404131b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.782+0000 7f440ba11700 1 -- 172.21.15.3:0/1913210952 >> 172.21.15.3:0/1913210952 conn(0x7f4404066f50 msgr2=0x7f44040a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:28.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.782+0000 7f440ba11700 1 -- 172.21.15.3:0/1913210952 shutdown_connections 2023-10-03T10:16:28.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.782+0000 7f440ba11700 1 -- 172.21.15.3:0/1913210952 wait complete. 2023-10-03T10:16:28.795 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:28.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.817+0000 7f5ddca4b700 1 -- 172.21.15.3:0/1643474775 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5dd806c360 con 0x7f5dd809d6f0 2023-10-03T10:16:28.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.818+0000 7f5dc6ffd700 1 -- 172.21.15.3:0/1643474775 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5dc8015070 con 0x7f5dd809d6f0 2023-10-03T10:16:28.818 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:28.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.820+0000 7f5ddca4b700 1 -- 172.21.15.3:0/1643474775 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5dd809d6f0 msgr2=0x7f5dd81a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:28.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.820+0000 7f5ddca4b700 1 --2- 172.21.15.3:0/1643474775 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5dd809d6f0 0x7f5dd81a60f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8009c70 tx=0x7f5dc8004750 comp rx=0 tx=0).stop 2023-10-03T10:16:28.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.820+0000 7f5ddca4b700 1 -- 172.21.15.3:0/1643474775 shutdown_connections 2023-10-03T10:16:28.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.820+0000 7f5ddca4b700 1 --2- 172.21.15.3:0/1643474775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dd809e410 0x7f5dd81a6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.820+0000 7f5ddca4b700 1 --2- 172.21.15.3:0/1643474775 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5dd809d6f0 0x7f5dd81a60f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.820+0000 7f5ddca4b700 1 --2- 172.21.15.3:0/1643474775 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5dd809e000 0x7f5dd81a6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:28.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.820+0000 7f5ddca4b700 1 -- 172.21.15.3:0/1643474775 >> 172.21.15.3:0/1643474775 conn(0x7f5dd8066f50 msgr2=0x7f5dd81baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:28.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.820+0000 7f5ddca4b700 1 -- 172.21.15.3:0/1643474775 shutdown_connections 2023-10-03T10:16:28.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:28.820+0000 7f5ddca4b700 1 -- 172.21.15.3:0/1643474775 wait complete. 2023-10-03T10:16:28.834 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:29.604 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:29.738 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:29.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.751+0000 7f0850dc2700 1 Processor -- start 2023-10-03T10:16:29.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.751+0000 7f0850dc2700 1 -- start start 2023-10-03T10:16:29.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.752+0000 7f0850dc2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f084c09d6f0 0x7f084c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:29.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.752+0000 7f0850dc2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f084c09e000 0x7f084c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:29.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.752+0000 7f0850dc2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f084c09e410 0x7f084c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:29.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.752+0000 7f0850dc2700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f084c09f520 con 0x7f084c09e410 2023-10-03T10:16:29.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.752+0000 7f0850dc2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f084c09f130 con 0x7f084c09d6f0 2023-10-03T10:16:29.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.752+0000 7f0850dc2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f084c09e930 con 0x7f084c09e000 2023-10-03T10:16:29.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.752+0000 7f084ad9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f084c09e410 0x7f084c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:29.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.752+0000 7f084a59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f084c09d6f0 0x7f084c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:29.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.752+0000 7f0849d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f084c09e000 0x7f084c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:29.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.752+0000 7f084ad9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f084c09e410 0x7f084c130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42686/0 (socket says 172.21.15.3:42686) 2023-10-03T10:16:29.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.752+0000 7f084a59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f084c09d6f0 0x7f084c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37914/0 (socket says 172.21.15.3:37914) 2023-10-03T10:16:29.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.752+0000 7f084ad9d700 1 -- 172.21.15.3:0/685633387 learned_addr learned my addr 172.21.15.3:0/685633387 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:29.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.753+0000 7f084a59c700 1 -- 172.21.15.3:0/685633387 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f084c09e000 msgr2=0x7f084c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:29.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.753+0000 7f084a59c700 1 --2- 172.21.15.3:0/685633387 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f084c09e000 0x7f084c130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:29.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.753+0000 7f084a59c700 1 -- 172.21.15.3:0/685633387 >> v2:172.21.15.3:3300/0 conn(0x7f084c09e410 msgr2=0x7f084c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:29.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.753+0000 7f084a59c700 1 --2- 172.21.15.3:0/685633387 >> v2:172.21.15.3:3300/0 conn(0x7f084c09e410 0x7f084c130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:29.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.753+0000 7f084a59c700 1 -- 172.21.15.3:0/685633387 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f084c09b2d0 con 0x7f084c09d6f0 2023-10-03T10:16:29.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.753+0000 7f0849d9b700 1 --2- 172.21.15.3:0/685633387 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f084c09e000 0x7f084c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:29.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.753+0000 7f084ad9d700 1 --2- 172.21.15.3:0/685633387 >> v2:172.21.15.3:3300/0 conn(0x7f084c09e410 0x7f084c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:29.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.753+0000 7f084a59c700 1 --2- 172.21.15.3:0/685633387 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f084c09d6f0 0x7f084c09dac0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f083c009870 tx=0x7f083c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bac2090846b8c2b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:29.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.753+0000 7f084959a700 1 -- 172.21.15.3:0/685633387 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083c011830 con 0x7f084c09d6f0 2023-10-03T10:16:29.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.753+0000 7f084959a700 1 -- 172.21.15.3:0/685633387 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083c011990 con 0x7f084c09d6f0 2023-10-03T10:16:29.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.753+0000 7f084959a700 1 -- 172.21.15.3:0/685633387 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083c011d20 con 0x7f084c09d6f0 2023-10-03T10:16:29.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.754+0000 7f0850dc2700 1 -- 172.21.15.3:0/685633387 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f084c09d6f0 msgr2=0x7f084c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:29.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.754+0000 7f0850dc2700 1 --2- 172.21.15.3:0/685633387 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f084c09d6f0 0x7f084c09dac0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f083c009870 tx=0x7f083c009b80 comp rx=0 tx=0).stop 2023-10-03T10:16:29.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.754+0000 7f0850dc2700 1 -- 172.21.15.3:0/685633387 shutdown_connections 2023-10-03T10:16:29.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.754+0000 7f0850dc2700 1 --2- 172.21.15.3:0/685633387 >> v2:172.21.15.3:3300/0 conn(0x7f084c09e410 0x7f084c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:29.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.754+0000 7f0850dc2700 1 --2- 172.21.15.3:0/685633387 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f084c09d6f0 0x7f084c09dac0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:29.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.754+0000 7f0850dc2700 1 --2- 172.21.15.3:0/685633387 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f084c09e000 0x7f084c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:29.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.754+0000 7f0850dc2700 1 -- 172.21.15.3:0/685633387 >> 172.21.15.3:0/685633387 conn(0x7f084c066f50 msgr2=0x7f084c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:29.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.754+0000 7f0850dc2700 1 -- 172.21.15.3:0/685633387 shutdown_connections 2023-10-03T10:16:29.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.754+0000 7f0850dc2700 1 -- 172.21.15.3:0/685633387 wait complete. 2023-10-03T10:16:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.755+0000 7f0850dc2700 1 Processor -- start 2023-10-03T10:16:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.756+0000 7f0850dc2700 1 -- start start 2023-10-03T10:16:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.756+0000 7f0850dc2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f084c09d6f0 0x7f084c06e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.756+0000 7f0850dc2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f084c09e000 0x7f084c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.756+0000 7f0850dc2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f084c09e410 0x7f084c06f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.756+0000 7f0850dc2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084c125860 con 0x7f084c09e000 2023-10-03T10:16:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.756+0000 7f0850dc2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f084c06fcd0 con 0x7f084c09e410 2023-10-03T10:16:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.756+0000 7f0850dc2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f084c124750 con 0x7f084c09d6f0 2023-10-03T10:16:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.756+0000 7f084ad9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f084c09e410 0x7f084c06f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.756+0000 7f084a59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f084c09d6f0 0x7f084c06e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.756+0000 7f084ad9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f084c09e410 0x7f084c06f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37924/0 (socket says 172.21.15.3:37924) 2023-10-03T10:16:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.756+0000 7f084ad9d700 1 -- 172.21.15.3:0/3679353309 learned_addr learned my addr 172.21.15.3:0/3679353309 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.756+0000 7f0849d9b700 1 --2- 172.21.15.3:0/3679353309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f084c09e000 0x7f084c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.757+0000 7f084ad9d700 1 -- 172.21.15.3:0/3679353309 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f084c09d6f0 msgr2=0x7f084c06e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.757+0000 7f084ad9d700 1 --2- 172.21.15.3:0/3679353309 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f084c09d6f0 0x7f084c06e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.757+0000 7f084ad9d700 1 -- 172.21.15.3:0/3679353309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f084c09e000 msgr2=0x7f084c06edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.757+0000 7f084ad9d700 1 --2- 172.21.15.3:0/3679353309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f084c09e000 0x7f084c06edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.757+0000 7f084ad9d700 1 -- 172.21.15.3:0/3679353309 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08400094a0 con 0x7f084c09e410 2023-10-03T10:16:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.757+0000 7f084a59c700 1 --2- 172.21.15.3:0/3679353309 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f084c09d6f0 0x7f084c06e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.757+0000 7f084ad9d700 1 --2- 172.21.15.3:0/3679353309 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f084c09e410 0x7f084c06f2f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0844009f10 tx=0x7f084400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.757+0000 7f083affd700 1 -- 172.21.15.3:0/3679353309 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0844018070 con 0x7f084c09e410 2023-10-03T10:16:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.757+0000 7f083affd700 1 -- 172.21.15.3:0/3679353309 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f084400f040 con 0x7f084c09e410 2023-10-03T10:16:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.757+0000 7f0850dc2700 1 -- 172.21.15.3:0/3679353309 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0844009b30 con 0x7f084c09e410 2023-10-03T10:16:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.757+0000 7f083affd700 1 -- 172.21.15.3:0/3679353309 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0844003c40 con 0x7f084c09e410 2023-10-03T10:16:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.757+0000 7f0850dc2700 1 -- 172.21.15.3:0/3679353309 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f083c0094a0 con 0x7f084c09e410 2023-10-03T10:16:29.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.758+0000 7f0850dc2700 1 -- 172.21.15.3:0/3679353309 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f084c074ea0 con 0x7f084c09e410 2023-10-03T10:16:29.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.759+0000 7f083affd700 1 -- 172.21.15.3:0/3679353309 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f084400c940 con 0x7f084c09e410 2023-10-03T10:16:29.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.759+0000 7f083affd700 1 -- 172.21.15.3:0/3679353309 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0844040750 con 0x7f084c09e410 2023-10-03T10:16:29.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.763+0000 7f083affd700 1 -- 172.21.15.3:0/3679353309 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0844013bd0 con 0x7f084c09e410 2023-10-03T10:16:29.796 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:29.834 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:29.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.880+0000 7f5257627700 1 Processor -- start 2023-10-03T10:16:29.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.881+0000 7f5257627700 1 -- start start 2023-10-03T10:16:29.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.881+0000 7f5257627700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f525006e570 0x7f525006e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:29.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.881+0000 7f5257627700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f525009eca0 0x7f525006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.881+0000 7f5257627700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5250134610 0x7f5250134a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.881+0000 7f5257627700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5250070ee0 con 0x7f525006e570 2023-10-03T10:16:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.881+0000 7f5257627700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5250071480 con 0x7f525009eca0 2023-10-03T10:16:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.881+0000 7f5257627700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f525006f3c0 con 0x7f5250134610 2023-10-03T10:16:29.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.881+0000 7f5255bc4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5250134610 0x7f5250134a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:29.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.881+0000 7f5255bc4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5250134610 0x7f5250134a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51608/0 (socket says 172.21.15.3:51608) 2023-10-03T10:16:29.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.881+0000 7f5255bc4700 1 -- 172.21.15.3:0/4252606767 learned_addr learned my addr 172.21.15.3:0/4252606767 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:29.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.881+0000 7f5254bc2700 1 --2- 172.21.15.3:0/4252606767 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f525009eca0 0x7f525006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:29.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.882+0000 7f52553c3700 1 --2- 172.21.15.3:0/4252606767 >> v2:172.21.15.3:3300/0 conn(0x7f525006e570 0x7f525006e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:29.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.882+0000 7f5254bc2700 1 -- 172.21.15.3:0/4252606767 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5250134610 msgr2=0x7f5250134a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:29.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.882+0000 7f5254bc2700 1 --2- 172.21.15.3:0/4252606767 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5250134610 0x7f5250134a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:29.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.882+0000 7f5254bc2700 1 -- 172.21.15.3:0/4252606767 >> v2:172.21.15.3:3300/0 conn(0x7f525006e570 msgr2=0x7f525006e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:29.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.882+0000 7f5254bc2700 1 --2- 172.21.15.3:0/4252606767 >> v2:172.21.15.3:3300/0 conn(0x7f525006e570 0x7f525006e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:29.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.882+0000 7f5254bc2700 1 -- 172.21.15.3:0/4252606767 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5250071b90 con 0x7f525009eca0 2023-10-03T10:16:29.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.882+0000 7f5255bc4700 1 --2- 172.21.15.3:0/4252606767 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5250134610 0x7f5250134a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:29.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.882+0000 7f5254bc2700 1 --2- 172.21.15.3:0/4252606767 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f525009eca0 0x7f525006ee80 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5240009840 tx=0x7f5240009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=baa4950aabda82d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:29.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.882+0000 7f5247fff700 1 -- 172.21.15.3:0/4252606767 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5240011830 con 0x7f525009eca0 2023-10-03T10:16:29.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.882+0000 7f5247fff700 1 -- 172.21.15.3:0/4252606767 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5240011990 con 0x7f525009eca0 2023-10-03T10:16:29.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.882+0000 7f5247fff700 1 -- 172.21.15.3:0/4252606767 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5240011d20 con 0x7f525009eca0 2023-10-03T10:16:29.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.883+0000 7f5257627700 1 -- 172.21.15.3:0/4252606767 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f525009eca0 msgr2=0x7f525006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:29.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.883+0000 7f5257627700 1 --2- 172.21.15.3:0/4252606767 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f525009eca0 0x7f525006ee80 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5240009840 tx=0x7f5240009b50 comp rx=0 tx=0).stop 2023-10-03T10:16:29.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.883+0000 7f5257627700 1 -- 172.21.15.3:0/4252606767 shutdown_connections 2023-10-03T10:16:29.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.883+0000 7f5257627700 1 --2- 172.21.15.3:0/4252606767 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5250134610 0x7f5250134a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:29.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.883+0000 7f5257627700 1 --2- 172.21.15.3:0/4252606767 >> v2:172.21.15.3:3300/0 conn(0x7f525006e570 0x7f525006e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:29.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.883+0000 7f5257627700 1 --2- 172.21.15.3:0/4252606767 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f525009eca0 0x7f525006ee80 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:29.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.883+0000 7f5257627700 1 -- 172.21.15.3:0/4252606767 >> 172.21.15.3:0/4252606767 conn(0x7f52500a2aa0 msgr2=0x7f52500a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:29.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.883+0000 7f5257627700 1 -- 172.21.15.3:0/4252606767 shutdown_connections 2023-10-03T10:16:29.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.883+0000 7f5257627700 1 -- 172.21.15.3:0/4252606767 wait complete. 2023-10-03T10:16:29.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.884+0000 7f5257627700 1 Processor -- start 2023-10-03T10:16:29.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.884+0000 7f5257627700 1 -- start start 2023-10-03T10:16:29.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.885+0000 7f5257627700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f525006e570 0x7f52501276b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:29.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.885+0000 7f5257627700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5250134610 0x7f52501260d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:29.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.885+0000 7f5257627700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5250126610 0x7f5250126a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:29.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.885+0000 7f5257627700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525013b7e0 con 0x7f525006e570 2023-10-03T10:16:29.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.885+0000 7f5257627700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5250070610 con 0x7f5250126610 2023-10-03T10:16:29.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.885+0000 7f5257627700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f525013d150 con 0x7f5250134610 2023-10-03T10:16:29.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.885+0000 7f52553c3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f525006e570 0x7f52501276b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:29.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.885+0000 7f5254bc2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5250134610 0x7f52501260d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:29.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.885+0000 7f52553c3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f525006e570 0x7f52501276b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42706/0 (socket says 172.21.15.3:42706) 2023-10-03T10:16:29.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.885+0000 7f52553c3700 1 -- 172.21.15.3:0/77642387 learned_addr learned my addr 172.21.15.3:0/77642387 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:29.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.885+0000 7f5255bc4700 1 --2- 172.21.15.3:0/77642387 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5250126610 0x7f5250126a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:29.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.886+0000 7f52553c3700 1 -- 172.21.15.3:0/77642387 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5250134610 msgr2=0x7f52501260d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:29.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.886+0000 7f52553c3700 1 --2- 172.21.15.3:0/77642387 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5250134610 0x7f52501260d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:29.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.886+0000 7f52553c3700 1 -- 172.21.15.3:0/77642387 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5250126610 msgr2=0x7f5250126a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:29.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.886+0000 7f52553c3700 1 --2- 172.21.15.3:0/77642387 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5250126610 0x7f5250126a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:29.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.886+0000 7f52553c3700 1 -- 172.21.15.3:0/77642387 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5248009b30 con 0x7f525006e570 2023-10-03T10:16:29.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.886+0000 7f52553c3700 1 --2- 172.21.15.3:0/77642387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f525006e570 0x7f52501276b0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f524c00e810 tx=0x7f524c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:29.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.886+0000 7f5245ffb700 1 -- 172.21.15.3:0/77642387 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c00c070 con 0x7f525006e570 2023-10-03T10:16:29.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.886+0000 7f5245ffb700 1 -- 172.21.15.3:0/77642387 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524c013610 con 0x7f525006e570 2023-10-03T10:16:29.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.886+0000 7f5257627700 1 -- 172.21.15.3:0/77642387 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52400094a0 con 0x7f525006e570 2023-10-03T10:16:29.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.886+0000 7f5245ffb700 1 -- 172.21.15.3:0/77642387 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c0139c0 con 0x7f525006e570 2023-10-03T10:16:29.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.887+0000 7f5257627700 1 -- 172.21.15.3:0/77642387 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5250127bf0 con 0x7f525006e570 2023-10-03T10:16:29.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.888+0000 7f5245ffb700 1 -- 172.21.15.3:0/77642387 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f524c013b20 con 0x7f525006e570 2023-10-03T10:16:29.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.888+0000 7f5245ffb700 1 -- 172.21.15.3:0/77642387 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f524c03f3d0 con 0x7f525006e570 2023-10-03T10:16:29.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.888+0000 7f5257627700 1 -- 172.21.15.3:0/77642387 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f525013d630 con 0x7f525006e570 2023-10-03T10:16:29.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.893+0000 7f5245ffb700 1 -- 172.21.15.3:0/77642387 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f525013d630 con 0x7f525006e570 2023-10-03T10:16:29.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.951+0000 7f1858ccd700 1 Processor -- start 2023-10-03T10:16:29.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.951+0000 7f1858ccd700 1 -- start start 2023-10-03T10:16:29.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.952+0000 7f1858ccd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18540a0a30 0x7f18540a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:29.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.952+0000 7f1858ccd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f185409d630 0x7f18540a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:29.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.953+0000 7f1858ccd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18540a2a70 con 0x7f1854133930 2023-10-03T10:16:29.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.953+0000 7f1858ccd700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18540a01d0 con 0x7f185409d630 2023-10-03T10:16:29.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.953+0000 7f1858ccd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f185409e2b0 con 0x7f18540a0a30 2023-10-03T10:16:29.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.953+0000 7f185259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18540a0a30 0x7f18540a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:29.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.953+0000 7f185259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18540a0a30 0x7f18540a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51624/0 (socket says 172.21.15.3:51624) 2023-10-03T10:16:29.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.953+0000 7f185259c700 1 -- 172.21.15.3:0/3984814152 learned_addr learned my addr 172.21.15.3:0/3984814152 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:29.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.953+0000 7f1851d9b700 1 --2- 172.21.15.3:0/3984814152 >> v2:172.21.15.3:3300/0 conn(0x7f185409d630 0x7f18540a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:29.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.953+0000 7f185159a700 1 -- 172.21.15.3:0/3984814152 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3873383556 0 0) 0x7f18540a2a70 con 0x7f1854133930 2023-10-03T10:16:29.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.953+0000 7f185159a700 1 -- 172.21.15.3:0/3984814152 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f183c0035b0 con 0x7f1854133930 2023-10-03T10:16:29.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.953+0000 7f185259c700 1 -- 172.21.15.3:0/3984814152 >> v2:172.21.15.3:3300/0 conn(0x7f185409d630 msgr2=0x7f18540a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:29.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.953+0000 7f185259c700 1 --2- 172.21.15.3:0/3984814152 >> v2:172.21.15.3:3300/0 conn(0x7f185409d630 0x7f18540a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:29.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.953+0000 7f185259c700 1 -- 172.21.15.3:0/3984814152 >> v1:172.21.15.3:6789/0 conn(0x7f1854133930 legacy=0x7f1854133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:29.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.953+0000 7f185259c700 1 -- 172.21.15.3:0/3984814152 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185409dc50 con 0x7f18540a0a30 2023-10-03T10:16:29.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.953+0000 7f185259c700 1 --2- 172.21.15.3:0/3984814152 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18540a0a30 0x7f18540a0e00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f18440098f0 tx=0x7f1844009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=503520535ee76b82 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:29.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.954+0000 7f185159a700 1 -- 172.21.15.3:0/3984814152 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1844011830 con 0x7f18540a0a30 2023-10-03T10:16:29.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.954+0000 7f185159a700 1 -- 172.21.15.3:0/3984814152 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1844011990 con 0x7f18540a0a30 2023-10-03T10:16:29.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.954+0000 7f185159a700 1 -- 172.21.15.3:0/3984814152 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1844011d00 con 0x7f18540a0a30 2023-10-03T10:16:29.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.955+0000 7f1858ccd700 1 -- 172.21.15.3:0/3984814152 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18540a0a30 msgr2=0x7f18540a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:29.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.955+0000 7f1858ccd700 1 --2- 172.21.15.3:0/3984814152 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18540a0a30 0x7f18540a0e00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f18440098f0 tx=0x7f1844009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:29.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.955+0000 7f1858ccd700 1 -- 172.21.15.3:0/3984814152 shutdown_connections 2023-10-03T10:16:29.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.955+0000 7f1858ccd700 1 --2- 172.21.15.3:0/3984814152 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18540a0a30 0x7f18540a0e00 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:29.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.955+0000 7f1858ccd700 1 --2- 172.21.15.3:0/3984814152 >> v2:172.21.15.3:3300/0 conn(0x7f185409d630 0x7f18540a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:29.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.955+0000 7f1858ccd700 1 -- 172.21.15.3:0/3984814152 >> 172.21.15.3:0/3984814152 conn(0x7f18540a3050 msgr2=0x7f18540a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:29.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.955+0000 7f1858ccd700 1 -- 172.21.15.3:0/3984814152 shutdown_connections 2023-10-03T10:16:29.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.955+0000 7f1858ccd700 1 -- 172.21.15.3:0/3984814152 wait complete. 2023-10-03T10:16:29.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.957+0000 7f1858ccd700 1 Processor -- start 2023-10-03T10:16:29.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.957+0000 7f1858ccd700 1 -- start start 2023-10-03T10:16:29.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.957+0000 7f1858ccd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f185409d630 0x7f18541c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:29.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.957+0000 7f1858ccd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18540a0a30 0x7f18541c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:29.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.957+0000 7f1858ccd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1854133930 0x7f18541c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:29.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.958+0000 7f1858ccd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18541221c0 con 0x7f1854133930 2023-10-03T10:16:29.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.958+0000 7f1858ccd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1854133530 con 0x7f185409d630 2023-10-03T10:16:29.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.958+0000 7f1858ccd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1854121f00 con 0x7f18540a0a30 2023-10-03T10:16:29.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.958+0000 7f1851d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18540a0a30 0x7f18541c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:29.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.958+0000 7f1851d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18540a0a30 0x7f18541c84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51630/0 (socket says 172.21.15.3:51630) 2023-10-03T10:16:29.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.958+0000 7f1852d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1854133930 0x7f18541c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:29.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.958+0000 7f1851d9b700 1 -- 172.21.15.3:0/4171427169 learned_addr learned my addr 172.21.15.3:0/4171427169 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:29.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.958+0000 7f185259c700 1 --2- 172.21.15.3:0/4171427169 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f185409d630 0x7f18541c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:29.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.958+0000 7f1852d9d700 1 -- 172.21.15.3:0/4171427169 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18540a0a30 msgr2=0x7f18541c84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:29.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.958+0000 7f1852d9d700 1 --2- 172.21.15.3:0/4171427169 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18540a0a30 0x7f18541c84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:29.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.958+0000 7f1852d9d700 1 -- 172.21.15.3:0/4171427169 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f185409d630 msgr2=0x7f18541c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:29.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.958+0000 7f1852d9d700 1 --2- 172.21.15.3:0/4171427169 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f185409d630 0x7f18541c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:29.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.958+0000 7f1852d9d700 1 -- 172.21.15.3:0/4171427169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1844009550 con 0x7f1854133930 2023-10-03T10:16:29.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.958+0000 7f1851d9b700 1 --2- 172.21.15.3:0/4171427169 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18540a0a30 0x7f18541c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:29.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.958+0000 7f1852d9d700 1 --2- 172.21.15.3:0/4171427169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1854133930 0x7f18541c8a00 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f184c00cbc0 tx=0x7f184c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:29.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.959+0000 7f1842ffd700 1 -- 172.21.15.3:0/4171427169 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c003c00 con 0x7f1854133930 2023-10-03T10:16:29.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.959+0000 7f1842ffd700 1 -- 172.21.15.3:0/4171427169 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f184c003d60 con 0x7f1854133930 2023-10-03T10:16:29.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.959+0000 7f1858ccd700 1 -- 172.21.15.3:0/4171427169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1854120710 con 0x7f1854133930 2023-10-03T10:16:29.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.959+0000 7f1842ffd700 1 -- 172.21.15.3:0/4171427169 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c00aa80 con 0x7f1854133930 2023-10-03T10:16:29.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.959+0000 7f1858ccd700 1 -- 172.21.15.3:0/4171427169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1854122c30 con 0x7f1854133930 2023-10-03T10:16:29.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.960+0000 7f1858ccd700 1 -- 172.21.15.3:0/4171427169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18540747f0 con 0x7f1854133930 2023-10-03T10:16:29.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.960+0000 7f1842ffd700 1 -- 172.21.15.3:0/4171427169 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f184c00abe0 con 0x7f1854133930 2023-10-03T10:16:29.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.961+0000 7f1842ffd700 1 -- 172.21.15.3:0/4171427169 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f184c03f610 con 0x7f1854133930 2023-10-03T10:16:29.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.970+0000 7f1842ffd700 1 -- 172.21.15.3:0/4171427169 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f184c0196c0 con 0x7f1854133930 2023-10-03T10:16:29.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.985+0000 7f7f17ad1700 1 Processor -- start 2023-10-03T10:16:29.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.985+0000 7f7f17ad1700 1 -- start start 2023-10-03T10:16:29.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.986+0000 7f7f17ad1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7f1009d630 0x7f7f100a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:29.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.986+0000 7f7f17ad1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f10138d80 0x7f7f1013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:29.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.986+0000 7f7f17ad1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f100a2a70 con 0x7f7f100a0a30 2023-10-03T10:16:29.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.986+0000 7f7f17ad1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f100a01d0 con 0x7f7f1009d630 2023-10-03T10:16:29.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.986+0000 7f7f17ad1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1009e2b0 con 0x7f7f10138d80 2023-10-03T10:16:29.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.986+0000 7f7f1606e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f10138d80 0x7f7f1013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:29.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.986+0000 7f7f1506c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7f1009d630 0x7f7f100a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:29.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.986+0000 7f7f1586d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7f100a0a30 0x7f7f100a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35472/0 (socket says 172.21.15.3:35472) 2023-10-03T10:16:29.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.986+0000 7f7f1586d700 1 -- 172.21.15.3:0/2258412751 learned_addr learned my addr 172.21.15.3:0/2258412751 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:29.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.986+0000 7f7f1606e700 1 --2- 172.21.15.3:0/2258412751 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f10138d80 0x7f7f1013b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51646/0 (socket says 172.21.15.3:51646) 2023-10-03T10:16:29.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.986+0000 7f7f1606e700 1 -- 172.21.15.3:0/2258412751 >> v2:172.21.15.3:3300/0 conn(0x7f7f1009d630 msgr2=0x7f7f100a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:29.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.986+0000 7f7f1606e700 1 --2- 172.21.15.3:0/2258412751 >> v2:172.21.15.3:3300/0 conn(0x7f7f1009d630 0x7f7f100a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:29.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.986+0000 7f7f1606e700 1 -- 172.21.15.3:0/2258412751 >> v1:172.21.15.3:6789/0 conn(0x7f7f100a0a30 legacy=0x7f7f100a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:29.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.986+0000 7f7f1486b700 1 -- 172.21.15.3:0/2258412751 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2255344751 0 0) 0x7f7f100a2a70 con 0x7f7f100a0a30 2023-10-03T10:16:29.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.986+0000 7f7f1606e700 1 -- 172.21.15.3:0/2258412751 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1009dc50 con 0x7f7f10138d80 2023-10-03T10:16:29.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.986+0000 7f7f1506c700 1 --2- 172.21.15.3:0/2258412751 >> v2:172.21.15.3:3300/0 conn(0x7f7f1009d630 0x7f7f100a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:29.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.987+0000 7f7f1606e700 1 --2- 172.21.15.3:0/2258412751 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f10138d80 0x7f7f1013b160 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009d90 tx=0x7f7f0800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb7290ee4f4abde8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:29.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.987+0000 7f7f1486b700 1 -- 172.21.15.3:0/2258412751 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0800e070 con 0x7f7f10138d80 2023-10-03T10:16:29.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.987+0000 7f7f1486b700 1 -- 172.21.15.3:0/2258412751 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0800bb20 con 0x7f7f10138d80 2023-10-03T10:16:29.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.987+0000 7f7f1486b700 1 -- 172.21.15.3:0/2258412751 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f08012430 con 0x7f7f10138d80 2023-10-03T10:16:29.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.987+0000 7f7f17ad1700 1 -- 172.21.15.3:0/2258412751 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f10138d80 msgr2=0x7f7f1013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:29.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.987+0000 7f7f17ad1700 1 --2- 172.21.15.3:0/2258412751 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f10138d80 0x7f7f1013b160 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009d90 tx=0x7f7f0800b330 comp rx=0 tx=0).stop 2023-10-03T10:16:29.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.987+0000 7f7f17ad1700 1 -- 172.21.15.3:0/2258412751 shutdown_connections 2023-10-03T10:16:29.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.987+0000 7f7f17ad1700 1 --2- 172.21.15.3:0/2258412751 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f10138d80 0x7f7f1013b160 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:29.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.987+0000 7f7f17ad1700 1 --2- 172.21.15.3:0/2258412751 >> v2:172.21.15.3:3300/0 conn(0x7f7f1009d630 0x7f7f100a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:29.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.987+0000 7f7f17ad1700 1 -- 172.21.15.3:0/2258412751 >> 172.21.15.3:0/2258412751 conn(0x7f7f100a3050 msgr2=0x7f7f100a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:29.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.988+0000 7f7f17ad1700 1 -- 172.21.15.3:0/2258412751 shutdown_connections 2023-10-03T10:16:29.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.988+0000 7f7f17ad1700 1 -- 172.21.15.3:0/2258412751 wait complete. 2023-10-03T10:16:29.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.989+0000 7f7f17ad1700 1 Processor -- start 2023-10-03T10:16:29.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.989+0000 7f7f17ad1700 1 -- start start 2023-10-03T10:16:29.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.989+0000 7f7f17ad1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f1009d630 0x7f7f10133700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:29.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.989+0000 7f7f17ad1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f100a0a30 0x7f7f10133c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:29.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.989+0000 7f7f17ad1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f10138d80 0x7f7f10134180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:29.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.989+0000 7f7f17ad1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f10137180 con 0x7f7f10138d80 2023-10-03T10:16:29.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.989+0000 7f7f17ad1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f10136070 con 0x7f7f100a0a30 2023-10-03T10:16:29.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.989+0000 7f7f17ad1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f10136530 con 0x7f7f1009d630 2023-10-03T10:16:29.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.989+0000 7f7f1586d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f1009d630 0x7f7f10133700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:29.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.989+0000 7f7f1586d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f1009d630 0x7f7f10133700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51658/0 (socket says 172.21.15.3:51658) 2023-10-03T10:16:29.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.989+0000 7f7f1586d700 1 -- 172.21.15.3:0/4206952340 learned_addr learned my addr 172.21.15.3:0/4206952340 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:29.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.989+0000 7f7f1506c700 1 --2- 172.21.15.3:0/4206952340 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f100a0a30 0x7f7f10133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:29.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.990+0000 7f7f1606e700 1 --2- 172.21.15.3:0/4206952340 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f10138d80 0x7f7f10134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:29.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.990+0000 7f7f1586d700 1 -- 172.21.15.3:0/4206952340 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f100a0a30 msgr2=0x7f7f10133c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:29.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.990+0000 7f7f1586d700 1 --2- 172.21.15.3:0/4206952340 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f100a0a30 0x7f7f10133c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:29.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.990+0000 7f7f1586d700 1 -- 172.21.15.3:0/4206952340 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f10138d80 msgr2=0x7f7f10134180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:29.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.990+0000 7f7f1586d700 1 --2- 172.21.15.3:0/4206952340 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f10138d80 0x7f7f10134180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:29.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.990+0000 7f7f1586d700 1 -- 172.21.15.3:0/4206952340 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f000094a0 con 0x7f7f1009d630 2023-10-03T10:16:29.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.990+0000 7f7f1586d700 1 --2- 172.21.15.3:0/4206952340 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f1009d630 0x7f7f10133700 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00c520 tx=0x7f7f0c00c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:29.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.990+0000 7f7f067fc700 1 -- 172.21.15.3:0/4206952340 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c004c80 con 0x7f7f1009d630 2023-10-03T10:16:29.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.990+0000 7f7f067fc700 1 -- 172.21.15.3:0/4206952340 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0c004de0 con 0x7f7f1009d630 2023-10-03T10:16:29.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.990+0000 7f7f067fc700 1 -- 172.21.15.3:0/4206952340 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c007310 con 0x7f7f1009d630 2023-10-03T10:16:29.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.990+0000 7f7f17ad1700 1 -- 172.21.15.3:0/4206952340 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f08009a70 con 0x7f7f1009d630 2023-10-03T10:16:29.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.990+0000 7f7f17ad1700 1 -- 172.21.15.3:0/4206952340 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f10136be0 con 0x7f7f1009d630 2023-10-03T10:16:29.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.991+0000 7f7f067fc700 1 -- 172.21.15.3:0/4206952340 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7f0c007470 con 0x7f7f1009d630 2023-10-03T10:16:29.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.991+0000 7f7f067fc700 1 -- 172.21.15.3:0/4206952340 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7f0c03f470 con 0x7f7f1009d630 2023-10-03T10:16:29.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.991+0000 7f7f17ad1700 1 -- 172.21.15.3:0/4206952340 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f10137d60 con 0x7f7f1009d630 2023-10-03T10:16:29.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:29.997+0000 7f7f067fc700 1 -- 172.21.15.3:0/4206952340 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7f0c0123b0 con 0x7f7f1009d630 2023-10-03T10:16:30.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.008+0000 7f0850dc2700 1 -- 172.21.15.3:0/3679353309 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f084c06c360 con 0x7f084c09e410 2023-10-03T10:16:30.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.009+0000 7f083affd700 1 -- 172.21.15.3:0/3679353309 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0844013bd0 con 0x7f084c09e410 2023-10-03T10:16:30.009 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:30.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.012+0000 7f0850dc2700 1 -- 172.21.15.3:0/3679353309 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f084c09e410 msgr2=0x7f084c06f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:30.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.012+0000 7f0850dc2700 1 --2- 172.21.15.3:0/3679353309 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f084c09e410 0x7f084c06f2f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0844009f10 tx=0x7f084400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:16:30.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.012+0000 7f0850dc2700 1 -- 172.21.15.3:0/3679353309 shutdown_connections 2023-10-03T10:16:30.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.012+0000 7f0850dc2700 1 --2- 172.21.15.3:0/3679353309 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f084c09e410 0x7f084c06f2f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:30.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.012+0000 7f0850dc2700 1 --2- 172.21.15.3:0/3679353309 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f084c09d6f0 0x7f084c06e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:30.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.012+0000 7f0850dc2700 1 --2- 172.21.15.3:0/3679353309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f084c09e000 0x7f084c06edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:30.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.012+0000 7f0850dc2700 1 -- 172.21.15.3:0/3679353309 >> 172.21.15.3:0/3679353309 conn(0x7f084c066f50 msgr2=0x7f084c071330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:30.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.012+0000 7f0850dc2700 1 -- 172.21.15.3:0/3679353309 shutdown_connections 2023-10-03T10:16:30.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.012+0000 7f0850dc2700 1 -- 172.21.15.3:0/3679353309 wait complete. 2023-10-03T10:16:30.027 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:30.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.121+0000 7f5257627700 1 -- 172.21.15.3:0/77642387 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f525013d630 con 0x7f525006e570 2023-10-03T10:16:30.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.122+0000 7f5245ffb700 1 -- 172.21.15.3:0/77642387 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f525013d630 con 0x7f525006e570 2023-10-03T10:16:30.122 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:30.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.124+0000 7f5257627700 1 -- 172.21.15.3:0/77642387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f525006e570 msgr2=0x7f52501276b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:30.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.124+0000 7f5257627700 1 --2- 172.21.15.3:0/77642387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f525006e570 0x7f52501276b0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f524c00e810 tx=0x7f524c00eb20 comp rx=0 tx=0).stop 2023-10-03T10:16:30.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.124+0000 7f5257627700 1 -- 172.21.15.3:0/77642387 shutdown_connections 2023-10-03T10:16:30.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.124+0000 7f5257627700 1 --2- 172.21.15.3:0/77642387 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5250126610 0x7f5250126a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:30.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.124+0000 7f5257627700 1 --2- 172.21.15.3:0/77642387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f525006e570 0x7f52501276b0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:30.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.124+0000 7f5257627700 1 --2- 172.21.15.3:0/77642387 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5250134610 0x7f52501260d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:30.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.124+0000 7f5257627700 1 -- 172.21.15.3:0/77642387 >> 172.21.15.3:0/77642387 conn(0x7f52500a2aa0 msgr2=0x7f5250099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:30.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.124+0000 7f5257627700 1 -- 172.21.15.3:0/77642387 shutdown_connections 2023-10-03T10:16:30.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.124+0000 7f5257627700 1 -- 172.21.15.3:0/77642387 wait complete. 2023-10-03T10:16:30.138 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:30.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.196+0000 7f1858ccd700 1 -- 172.21.15.3:0/4171427169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1854072980 con 0x7f1854133930 2023-10-03T10:16:30.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.196+0000 7f1842ffd700 1 -- 172.21.15.3:0/4171427169 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f184c019300 con 0x7f1854133930 2023-10-03T10:16:30.196 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:30.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.198+0000 7f1858ccd700 1 -- 172.21.15.3:0/4171427169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1854133930 msgr2=0x7f18541c8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:30.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.198+0000 7f1858ccd700 1 --2- 172.21.15.3:0/4171427169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1854133930 0x7f18541c8a00 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f184c00cbc0 tx=0x7f184c00ced0 comp rx=0 tx=0).stop 2023-10-03T10:16:30.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.198+0000 7f1858ccd700 1 -- 172.21.15.3:0/4171427169 shutdown_connections 2023-10-03T10:16:30.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.198+0000 7f1858ccd700 1 --2- 172.21.15.3:0/4171427169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1854133930 0x7f18541c8a00 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:30.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.198+0000 7f1858ccd700 1 --2- 172.21.15.3:0/4171427169 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f185409d630 0x7f18541c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:30.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.198+0000 7f1858ccd700 1 --2- 172.21.15.3:0/4171427169 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18540a0a30 0x7f18541c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:30.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.198+0000 7f1858ccd700 1 -- 172.21.15.3:0/4171427169 >> 172.21.15.3:0/4171427169 conn(0x7f18540a3050 msgr2=0x7f18541bba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:30.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.198+0000 7f1858ccd700 1 -- 172.21.15.3:0/4171427169 shutdown_connections 2023-10-03T10:16:30.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.198+0000 7f1858ccd700 1 -- 172.21.15.3:0/4171427169 wait complete. 2023-10-03T10:16:30.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.205+0000 7f7f17ad1700 1 -- 172.21.15.3:0/4206952340 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7f100747f0 con 0x7f7f1009d630 2023-10-03T10:16:30.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.206+0000 7f7f067fc700 1 -- 172.21.15.3:0/4206952340 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7f0c011360 con 0x7f7f1009d630 2023-10-03T10:16:30.206 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:30.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.208+0000 7f7f17ad1700 1 -- 172.21.15.3:0/4206952340 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f1009d630 msgr2=0x7f7f10133700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:30.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.208+0000 7f7f17ad1700 1 --2- 172.21.15.3:0/4206952340 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f1009d630 0x7f7f10133700 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00c520 tx=0x7f7f0c00c830 comp rx=0 tx=0).stop 2023-10-03T10:16:30.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.208+0000 7f7f17ad1700 1 -- 172.21.15.3:0/4206952340 shutdown_connections 2023-10-03T10:16:30.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.208+0000 7f7f17ad1700 1 --2- 172.21.15.3:0/4206952340 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f10138d80 0x7f7f10134180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:30.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.208+0000 7f7f17ad1700 1 --2- 172.21.15.3:0/4206952340 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7f1009d630 0x7f7f10133700 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:30.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.208+0000 7f7f17ad1700 1 --2- 172.21.15.3:0/4206952340 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7f100a0a30 0x7f7f10133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:30.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.208+0000 7f7f17ad1700 1 -- 172.21.15.3:0/4206952340 >> 172.21.15.3:0/4206952340 conn(0x7f7f100a3050 msgr2=0x7f7f1009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:30.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.208+0000 7f7f17ad1700 1 -- 172.21.15.3:0/4206952340 shutdown_connections 2023-10-03T10:16:30.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:30.208+0000 7f7f17ad1700 1 -- 172.21.15.3:0/4206952340 wait complete. 2023-10-03T10:16:30.213 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:30.222 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:31.028 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:31.142 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:31.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.175+0000 7f01e0d39700 1 Processor -- start 2023-10-03T10:16:31.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.175+0000 7f01e0d39700 1 -- start start 2023-10-03T10:16:31.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.176+0000 7f01e0d39700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f01dc0a0a30 0x7f01dc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:31.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.176+0000 7f01e0d39700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f01dc09d630 0x7f01dc0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:31.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.176+0000 7f01e0d39700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f01dc133930 0x7f01dc133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:31.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.176+0000 7f01e0d39700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01dc0a2a70 con 0x7f01dc09d630 2023-10-03T10:16:31.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.176+0000 7f01e0d39700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01dc0a01d0 con 0x7f01dc133930 2023-10-03T10:16:31.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.176+0000 7f01e0d39700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f01dc09e2b0 con 0x7f01dc0a0a30 2023-10-03T10:16:31.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.176+0000 7f01da59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f01dc0a0a30 0x7f01dc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:31.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.176+0000 7f01da59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f01dc0a0a30 0x7f01dc0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51660/0 (socket says 172.21.15.3:51660) 2023-10-03T10:16:31.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.176+0000 7f01da59c700 1 -- 172.21.15.3:0/1444856835 learned_addr learned my addr 172.21.15.3:0/1444856835 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:31.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.176+0000 7f01d9d9b700 1 --2- 172.21.15.3:0/1444856835 >> v2:172.21.15.3:3300/0 conn(0x7f01dc09d630 0x7f01dc0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:31.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.176+0000 7f01dad9d700 1 --2- 172.21.15.3:0/1444856835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f01dc133930 0x7f01dc133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:31.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.176+0000 7f01da59c700 1 -- 172.21.15.3:0/1444856835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f01dc133930 msgr2=0x7f01dc133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:31.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.176+0000 7f01da59c700 1 --2- 172.21.15.3:0/1444856835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f01dc133930 0x7f01dc133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.176+0000 7f01da59c700 1 -- 172.21.15.3:0/1444856835 >> v2:172.21.15.3:3300/0 conn(0x7f01dc09d630 msgr2=0x7f01dc0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:31.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.176+0000 7f01da59c700 1 --2- 172.21.15.3:0/1444856835 >> v2:172.21.15.3:3300/0 conn(0x7f01dc09d630 0x7f01dc0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.176+0000 7f01da59c700 1 -- 172.21.15.3:0/1444856835 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01dc09dc50 con 0x7f01dc0a0a30 2023-10-03T10:16:31.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.176+0000 7f01dad9d700 1 --2- 172.21.15.3:0/1444856835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f01dc133930 0x7f01dc133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:31.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.177+0000 7f01da59c700 1 --2- 172.21.15.3:0/1444856835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f01dc0a0a30 0x7f01dc0a0e00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f01cc0098f0 tx=0x7f01cc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=467a133f57241c19 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:31.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.177+0000 7f01d959a700 1 -- 172.21.15.3:0/1444856835 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01cc011830 con 0x7f01dc0a0a30 2023-10-03T10:16:31.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.177+0000 7f01d959a700 1 -- 172.21.15.3:0/1444856835 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01cc011990 con 0x7f01dc0a0a30 2023-10-03T10:16:31.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.177+0000 7f01d959a700 1 -- 172.21.15.3:0/1444856835 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01cc011d20 con 0x7f01dc0a0a30 2023-10-03T10:16:31.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.178+0000 7f01e0d39700 1 -- 172.21.15.3:0/1444856835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f01dc0a0a30 msgr2=0x7f01dc0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:31.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.178+0000 7f01e0d39700 1 --2- 172.21.15.3:0/1444856835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f01dc0a0a30 0x7f01dc0a0e00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f01cc0098f0 tx=0x7f01cc009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:31.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.178+0000 7f01e0d39700 1 -- 172.21.15.3:0/1444856835 shutdown_connections 2023-10-03T10:16:31.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.178+0000 7f01e0d39700 1 --2- 172.21.15.3:0/1444856835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f01dc133930 0x7f01dc133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.178+0000 7f01e0d39700 1 --2- 172.21.15.3:0/1444856835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f01dc0a0a30 0x7f01dc0a0e00 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.178+0000 7f01e0d39700 1 --2- 172.21.15.3:0/1444856835 >> v2:172.21.15.3:3300/0 conn(0x7f01dc09d630 0x7f01dc0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.178+0000 7f01e0d39700 1 -- 172.21.15.3:0/1444856835 >> 172.21.15.3:0/1444856835 conn(0x7f01dc0a3050 msgr2=0x7f01dc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:31.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.178+0000 7f01e0d39700 1 -- 172.21.15.3:0/1444856835 shutdown_connections 2023-10-03T10:16:31.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.178+0000 7f01e0d39700 1 -- 172.21.15.3:0/1444856835 wait complete. 2023-10-03T10:16:31.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.179+0000 7f01e0d39700 1 Processor -- start 2023-10-03T10:16:31.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.180+0000 7f01e0d39700 1 -- start start 2023-10-03T10:16:31.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.180+0000 7f01e0d39700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01dc09d630 0x7f01dc11f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:31.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.180+0000 7f01e0d39700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f01dc0a0a30 0x7f01dc11f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:31.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.180+0000 7f01e0d39700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f01dc133930 0x7f01dc1c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:31.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.180+0000 7f01e0d39700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01dc120940 con 0x7f01dc09d630 2023-10-03T10:16:31.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.180+0000 7f01e0d39700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01dc120750 con 0x7f01dc0a0a30 2023-10-03T10:16:31.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.180+0000 7f01e0d39700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f01dc1334d0 con 0x7f01dc133930 2023-10-03T10:16:31.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.180+0000 7f01d9d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f01dc0a0a30 0x7f01dc11f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:31.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.180+0000 7f01dad9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f01dc133930 0x7f01dc1c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:31.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.181+0000 7f01da59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01dc09d630 0x7f01dc11f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:31.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.181+0000 7f01d9d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f01dc0a0a30 0x7f01dc11f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37972/0 (socket says 172.21.15.3:37972) 2023-10-03T10:16:31.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.181+0000 7f01d9d9b700 1 -- 172.21.15.3:0/3822129498 learned_addr learned my addr 172.21.15.3:0/3822129498 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:31.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.181+0000 7f01d9d9b700 1 -- 172.21.15.3:0/3822129498 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f01dc133930 msgr2=0x7f01dc1c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:31.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.181+0000 7f01d9d9b700 1 --2- 172.21.15.3:0/3822129498 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f01dc133930 0x7f01dc1c8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.181+0000 7f01d9d9b700 1 -- 172.21.15.3:0/3822129498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01dc09d630 msgr2=0x7f01dc11f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:31.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.181+0000 7f01d9d9b700 1 --2- 172.21.15.3:0/3822129498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01dc09d630 0x7f01dc11f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.181+0000 7f01d9d9b700 1 -- 172.21.15.3:0/3822129498 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d4009a70 con 0x7f01dc0a0a30 2023-10-03T10:16:31.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.181+0000 7f01da59c700 1 --2- 172.21.15.3:0/3822129498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01dc09d630 0x7f01dc11f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:31.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.181+0000 7f01d9d9b700 1 --2- 172.21.15.3:0/3822129498 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f01dc0a0a30 0x7f01dc11f9c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f01d000e980 tx=0x7f01d000ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:31.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.181+0000 7f01caffd700 1 -- 172.21.15.3:0/3822129498 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0009710 con 0x7f01dc0a0a30 2023-10-03T10:16:31.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.181+0000 7f01caffd700 1 -- 172.21.15.3:0/3822129498 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d000c640 con 0x7f01dc0a0a30 2023-10-03T10:16:31.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.181+0000 7f01e0d39700 1 -- 172.21.15.3:0/3822129498 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01cc009550 con 0x7f01dc0a0a30 2023-10-03T10:16:31.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.181+0000 7f01caffd700 1 -- 172.21.15.3:0/3822129498 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d000c9b0 con 0x7f01dc0a0a30 2023-10-03T10:16:31.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.181+0000 7f01e0d39700 1 -- 172.21.15.3:0/3822129498 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01dc122380 con 0x7f01dc0a0a30 2023-10-03T10:16:31.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.183+0000 7f01e0d39700 1 -- 172.21.15.3:0/3822129498 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01dc1221e0 con 0x7f01dc0a0a30 2023-10-03T10:16:31.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.183+0000 7f01caffd700 1 -- 172.21.15.3:0/3822129498 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f01d000cb30 con 0x7f01dc0a0a30 2023-10-03T10:16:31.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.184+0000 7f01caffd700 1 -- 172.21.15.3:0/3822129498 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f01d003fcf0 con 0x7f01dc0a0a30 2023-10-03T10:16:31.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.188+0000 7f01caffd700 1 -- 172.21.15.3:0/3822129498 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f01d000ce30 con 0x7f01dc0a0a30 2023-10-03T10:16:31.213 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:31.223 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:31.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.292+0000 7f79c8e2a700 1 Processor -- start 2023-10-03T10:16:31.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.292+0000 7f79c8e2a700 1 -- start start 2023-10-03T10:16:31.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.293+0000 7f79c8e2a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79c4133420 0x7f79c41337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:31.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.293+0000 7f79c8e2a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79c4133e30 0x7f79c412ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:31.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.293+0000 7f79c8e2a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79c406b990 con 0x7f79c4134260 2023-10-03T10:16:31.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.293+0000 7f79c8e2a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c40a0590 con 0x7f79c4133420 2023-10-03T10:16:31.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.293+0000 7f79c8e2a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79c409fb80 con 0x7f79c4133e30 2023-10-03T10:16:31.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.293+0000 7f79c259c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79c4133420 0x7f79c41337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:31.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.293+0000 7f79c259c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79c4133420 0x7f79c41337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:37988/0 (socket says 172.21.15.3:37988) 2023-10-03T10:16:31.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.293+0000 7f79c259c700 1 -- 172.21.15.3:0/3158015482 learned_addr learned my addr 172.21.15.3:0/3158015482 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:31.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.294+0000 7f79c1d9b700 1 --2- 172.21.15.3:0/3158015482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79c4133e30 0x7f79c412ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:31.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.294+0000 7f79c259c700 1 -- 172.21.15.3:0/3158015482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79c4133e30 msgr2=0x7f79c412ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:31.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.294+0000 7f79c259c700 1 --2- 172.21.15.3:0/3158015482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79c4133e30 0x7f79c412ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.294+0000 7f79c259c700 1 -- 172.21.15.3:0/3158015482 >> v1:172.21.15.3:6789/0 conn(0x7f79c4134260 legacy=0x7f79c41304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:31.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.294+0000 7f79c259c700 1 -- 172.21.15.3:0/3158015482 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c40a0030 con 0x7f79c4133420 2023-10-03T10:16:31.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.294+0000 7f79c259c700 1 --2- 172.21.15.3:0/3158015482 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79c4133420 0x7f79c41337f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f79b4009920 tx=0x7f79b4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=697d10c9fdbfecd7 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:31.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.294+0000 7f79c159a700 1 -- 172.21.15.3:0/3158015482 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b4004230 con 0x7f79c4133420 2023-10-03T10:16:31.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.294+0000 7f79c159a700 1 -- 172.21.15.3:0/3158015482 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b4004390 con 0x7f79c4133420 2023-10-03T10:16:31.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.294+0000 7f79c159a700 1 -- 172.21.15.3:0/3158015482 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b4009e70 con 0x7f79c4133420 2023-10-03T10:16:31.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.295+0000 7f79c8e2a700 1 -- 172.21.15.3:0/3158015482 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79c4133420 msgr2=0x7f79c41337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:31.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.295+0000 7f79c8e2a700 1 --2- 172.21.15.3:0/3158015482 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79c4133420 0x7f79c41337f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f79b4009920 tx=0x7f79b4009c30 comp rx=0 tx=0).stop 2023-10-03T10:16:31.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.296+0000 7f79c8e2a700 1 -- 172.21.15.3:0/3158015482 shutdown_connections 2023-10-03T10:16:31.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.296+0000 7f79c8e2a700 1 --2- 172.21.15.3:0/3158015482 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79c4133420 0x7f79c41337f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.296+0000 7f79c8e2a700 1 --2- 172.21.15.3:0/3158015482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79c4133e30 0x7f79c412ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.296+0000 7f79c8e2a700 1 -- 172.21.15.3:0/3158015482 >> 172.21.15.3:0/3158015482 conn(0x7f79c40a4210 msgr2=0x7f79c40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:31.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.296+0000 7f79c8e2a700 1 -- 172.21.15.3:0/3158015482 shutdown_connections 2023-10-03T10:16:31.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.296+0000 7f79c8e2a700 1 -- 172.21.15.3:0/3158015482 wait complete. 2023-10-03T10:16:31.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.297+0000 7f79c8e2a700 1 Processor -- start 2023-10-03T10:16:31.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.298+0000 7f79c8e2a700 1 -- start start 2023-10-03T10:16:31.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.298+0000 7f79c8e2a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79c4133420 0x7f79c409c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:31.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.298+0000 7f79c8e2a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79c4133e30 0x7f79c409ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:31.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.298+0000 7f79c8e2a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79c4134260 0x7f79c409d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:31.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.298+0000 7f79c8e2a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c4121290 con 0x7f79c4134260 2023-10-03T10:16:31.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.298+0000 7f79c8e2a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c412dbd0 con 0x7f79c4133420 2023-10-03T10:16:31.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.298+0000 7f79c8e2a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79c4120070 con 0x7f79c4133e30 2023-10-03T10:16:31.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.299+0000 7f79c259c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79c4133420 0x7f79c409c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:31.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.299+0000 7f79c259c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79c4133420 0x7f79c409c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38002/0 (socket says 172.21.15.3:38002) 2023-10-03T10:16:31.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.299+0000 7f79c259c700 1 -- 172.21.15.3:0/1088123932 learned_addr learned my addr 172.21.15.3:0/1088123932 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:31.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.299+0000 7f79c1d9b700 1 --2- 172.21.15.3:0/1088123932 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79c4133e30 0x7f79c409ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:31.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.299+0000 7f79c2d9d700 1 --2- 172.21.15.3:0/1088123932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79c4134260 0x7f79c409d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:31.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.299+0000 7f79c259c700 1 -- 172.21.15.3:0/1088123932 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79c4133e30 msgr2=0x7f79c409ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:31.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.299+0000 7f79c259c700 1 --2- 172.21.15.3:0/1088123932 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79c4133e30 0x7f79c409ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.299+0000 7f79c259c700 1 -- 172.21.15.3:0/1088123932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79c4134260 msgr2=0x7f79c409d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:31.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.299+0000 7f79c259c700 1 --2- 172.21.15.3:0/1088123932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79c4134260 0x7f79c409d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.299+0000 7f79c259c700 1 -- 172.21.15.3:0/1088123932 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b4009550 con 0x7f79c4133420 2023-10-03T10:16:31.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.299+0000 7f79c259c700 1 --2- 172.21.15.3:0/1088123932 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79c4133420 0x7f79c409c8c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f79c40a06d0 tx=0x7f79b4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:31.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.299+0000 7f79b2ffd700 1 -- 172.21.15.3:0/1088123932 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b400bd30 con 0x7f79c4133420 2023-10-03T10:16:31.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.300+0000 7f79b2ffd700 1 -- 172.21.15.3:0/1088123932 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b400be90 con 0x7f79c4133420 2023-10-03T10:16:31.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.300+0000 7f79b2ffd700 1 -- 172.21.15.3:0/1088123932 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b40038a0 con 0x7f79c4133420 2023-10-03T10:16:31.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.300+0000 7f79c8e2a700 1 -- 172.21.15.3:0/1088123932 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c411ec20 con 0x7f79c4133420 2023-10-03T10:16:31.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.300+0000 7f79c8e2a700 1 -- 172.21.15.3:0/1088123932 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c4120d10 con 0x7f79c4133420 2023-10-03T10:16:31.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.301+0000 7f79c8e2a700 1 -- 172.21.15.3:0/1088123932 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c41331c0 con 0x7f79c4133420 2023-10-03T10:16:31.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.301+0000 7f79b2ffd700 1 -- 172.21.15.3:0/1088123932 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f79b4021070 con 0x7f79c4133420 2023-10-03T10:16:31.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.301+0000 7f79b2ffd700 1 -- 172.21.15.3:0/1088123932 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f79b4040d60 con 0x7f79c4133420 2023-10-03T10:16:31.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.310+0000 7f79b2ffd700 1 -- 172.21.15.3:0/1088123932 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f79b402d990 con 0x7f79c4133420 2023-10-03T10:16:31.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.359+0000 7f84aa14f700 1 Processor -- start 2023-10-03T10:16:31.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.359+0000 7f84aa14f700 1 -- start start 2023-10-03T10:16:31.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.359+0000 7f84aa14f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f84a4133a80 0x7f84a4133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:31.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.359+0000 7f84aa14f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f84a4130a40 0x7f84a413b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:31.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.359+0000 7f84aa14f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84a406bc00 con 0x7f84a412ff20 2023-10-03T10:16:31.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.359+0000 7f84aa14f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a406c570 con 0x7f84a4133a80 2023-10-03T10:16:31.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.359+0000 7f84aa14f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f84a40a0040 con 0x7f84a4130a40 2023-10-03T10:16:31.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.360+0000 7f84a37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f84a4133a80 0x7f84a4133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:31.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.360+0000 7f84a37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f84a4133a80 0x7f84a4133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38010/0 (socket says 172.21.15.3:38010) 2023-10-03T10:16:31.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.360+0000 7f84a37fe700 1 -- 172.21.15.3:0/2501779335 learned_addr learned my addr 172.21.15.3:0/2501779335 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:31.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.360+0000 7f84a3fff700 1 --2- 172.21.15.3:0/2501779335 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f84a4130a40 0x7f84a413b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:31.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.360+0000 7f84a37fe700 1 -- 172.21.15.3:0/2501779335 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f84a4130a40 msgr2=0x7f84a413b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:31.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.360+0000 7f84a37fe700 1 --2- 172.21.15.3:0/2501779335 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f84a4130a40 0x7f84a413b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.360+0000 7f84a37fe700 1 -- 172.21.15.3:0/2501779335 >> v1:172.21.15.3:6789/0 conn(0x7f84a412ff20 legacy=0x7f84a4130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:31.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.360+0000 7f84a37fe700 1 -- 172.21.15.3:0/2501779335 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a4133860 con 0x7f84a4133a80 2023-10-03T10:16:31.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.360+0000 7f84a37fe700 1 --2- 172.21.15.3:0/2501779335 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f84a4133a80 0x7f84a4133e50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8494009920 tx=0x7f8494009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe933c6bee7e927c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:31.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.361+0000 7f84a27fc700 1 -- 172.21.15.3:0/2501779335 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8494011830 con 0x7f84a4133a80 2023-10-03T10:16:31.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.361+0000 7f84a27fc700 1 -- 172.21.15.3:0/2501779335 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8494011990 con 0x7f84a4133a80 2023-10-03T10:16:31.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.361+0000 7f84a27fc700 1 -- 172.21.15.3:0/2501779335 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8494011d20 con 0x7f84a4133a80 2023-10-03T10:16:31.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.362+0000 7f84aa14f700 1 -- 172.21.15.3:0/2501779335 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f84a4133a80 msgr2=0x7f84a4133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:31.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.362+0000 7f84aa14f700 1 --2- 172.21.15.3:0/2501779335 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f84a4133a80 0x7f84a4133e50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8494009920 tx=0x7f8494009c30 comp rx=0 tx=0).stop 2023-10-03T10:16:31.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.362+0000 7f84aa14f700 1 -- 172.21.15.3:0/2501779335 shutdown_connections 2023-10-03T10:16:31.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.362+0000 7f84aa14f700 1 --2- 172.21.15.3:0/2501779335 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f84a4133a80 0x7f84a4133e50 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.362+0000 7f84aa14f700 1 --2- 172.21.15.3:0/2501779335 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f84a4130a40 0x7f84a413b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.362+0000 7f84aa14f700 1 -- 172.21.15.3:0/2501779335 >> 172.21.15.3:0/2501779335 conn(0x7f84a40a4210 msgr2=0x7f84a40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:31.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.362+0000 7f84aa14f700 1 -- 172.21.15.3:0/2501779335 shutdown_connections 2023-10-03T10:16:31.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.362+0000 7f84aa14f700 1 -- 172.21.15.3:0/2501779335 wait complete. 2023-10-03T10:16:31.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.363+0000 7f84aa14f700 1 Processor -- start 2023-10-03T10:16:31.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.364+0000 7f84aa14f700 1 -- start start 2023-10-03T10:16:31.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.364+0000 7f84aa14f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f84a412ff20 0x7f84a409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:31.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.364+0000 7f84aa14f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f84a4130a40 0x7f84a409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:31.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.364+0000 7f84aa14f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84a4133a80 0x7f84a409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:31.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.364+0000 7f84aa14f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a4120f20 con 0x7f84a4133a80 2023-10-03T10:16:31.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.364+0000 7f84aa14f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a4120580 con 0x7f84a4130a40 2023-10-03T10:16:31.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.364+0000 7f84aa14f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f84a4120840 con 0x7f84a412ff20 2023-10-03T10:16:31.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.365+0000 7f84a37fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f84a412ff20 0x7f84a409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:31.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.365+0000 7f84a37fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f84a412ff20 0x7f84a409c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51718/0 (socket says 172.21.15.3:51718) 2023-10-03T10:16:31.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.365+0000 7f84a37fe700 1 -- 172.21.15.3:0/320986819 learned_addr learned my addr 172.21.15.3:0/320986819 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:31.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.365+0000 7f84a3fff700 1 --2- 172.21.15.3:0/320986819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84a4133a80 0x7f84a409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:31.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.365+0000 7f84a2ffd700 1 --2- 172.21.15.3:0/320986819 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f84a4130a40 0x7f84a409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:31.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.365+0000 7f84a3fff700 1 -- 172.21.15.3:0/320986819 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f84a412ff20 msgr2=0x7f84a409c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:31.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.365+0000 7f84a3fff700 1 --2- 172.21.15.3:0/320986819 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f84a412ff20 0x7f84a409c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.365+0000 7f84a3fff700 1 -- 172.21.15.3:0/320986819 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f84a4130a40 msgr2=0x7f84a409c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:31.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.365+0000 7f84a3fff700 1 --2- 172.21.15.3:0/320986819 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f84a4130a40 0x7f84a409c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.365+0000 7f84a3fff700 1 -- 172.21.15.3:0/320986819 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8494009550 con 0x7f84a4133a80 2023-10-03T10:16:31.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.365+0000 7f84a2ffd700 1 --2- 172.21.15.3:0/320986819 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f84a4130a40 0x7f84a409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:31.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.365+0000 7f84a3fff700 1 --2- 172.21.15.3:0/320986819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84a4133a80 0x7f84a409cf00 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f849c00efc0 tx=0x7f849c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:31.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.366+0000 7f848bfff700 1 -- 172.21.15.3:0/320986819 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c00cc30 con 0x7f84a4133a80 2023-10-03T10:16:31.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.366+0000 7f848bfff700 1 -- 172.21.15.3:0/320986819 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f849c00f040 con 0x7f84a4133a80 2023-10-03T10:16:31.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.366+0000 7f84aa14f700 1 -- 172.21.15.3:0/320986819 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a4120060 con 0x7f84a4133a80 2023-10-03T10:16:31.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.366+0000 7f848bfff700 1 -- 172.21.15.3:0/320986819 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c009e60 con 0x7f84a4133a80 2023-10-03T10:16:31.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.366+0000 7f84aa14f700 1 -- 172.21.15.3:0/320986819 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a411f9b0 con 0x7f84a4133a80 2023-10-03T10:16:31.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.367+0000 7f84aa14f700 1 -- 172.21.15.3:0/320986819 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84840052e0 con 0x7f84a4133a80 2023-10-03T10:16:31.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.367+0000 7f848bfff700 1 -- 172.21.15.3:0/320986819 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f849c014480 con 0x7f84a4133a80 2023-10-03T10:16:31.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.368+0000 7f848bfff700 1 -- 172.21.15.3:0/320986819 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f849c0401c0 con 0x7f84a4133a80 2023-10-03T10:16:31.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.377+0000 7f848bfff700 1 -- 172.21.15.3:0/320986819 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f849c007900 con 0x7f84a4133a80 2023-10-03T10:16:31.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.414+0000 7f01e0d39700 1 -- 172.21.15.3:0/3822129498 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f01dc1221e0 con 0x7f01dc0a0a30 2023-10-03T10:16:31.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.414+0000 7f01caffd700 1 -- 172.21.15.3:0/3822129498 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f01dc1221e0 con 0x7f01dc0a0a30 2023-10-03T10:16:31.414 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:31.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.414+0000 7f65d2621700 1 Processor -- start 2023-10-03T10:16:31.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.415+0000 7f65d2621700 1 -- start start 2023-10-03T10:16:31.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.415+0000 7f65d2621700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f65cc0a0a30 0x7f65cc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:31.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.415+0000 7f65d2621700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65cc138d80 0x7f65cc13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:31.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.415+0000 7f65d2621700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65cc0a2a70 con 0x7f65cc09d630 2023-10-03T10:16:31.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.415+0000 7f65d2621700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65cc0a01d0 con 0x7f65cc0a0a30 2023-10-03T10:16:31.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.415+0000 7f65d2621700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc09e2b0 con 0x7f65cc138d80 2023-10-03T10:16:31.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.416+0000 7f65d0bbe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65cc138d80 0x7f65cc13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:31.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.416+0000 7f65cb7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f65cc09d630 0x7f65cc0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35504/0 (socket says 172.21.15.3:35504) 2023-10-03T10:16:31.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.416+0000 7f65cb7fe700 1 -- 172.21.15.3:0/3691431518 learned_addr learned my addr 172.21.15.3:0/3691431518 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:31.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.416+0000 7f65cbfff700 1 --2- 172.21.15.3:0/3691431518 >> v2:172.21.15.3:3300/0 conn(0x7f65cc0a0a30 0x7f65cc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:31.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.416+0000 7f65d0bbe700 1 -- 172.21.15.3:0/3691431518 >> v2:172.21.15.3:3300/0 conn(0x7f65cc0a0a30 msgr2=0x7f65cc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:31.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.416+0000 7f65d0bbe700 1 --2- 172.21.15.3:0/3691431518 >> v2:172.21.15.3:3300/0 conn(0x7f65cc0a0a30 0x7f65cc0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.416+0000 7f65d0bbe700 1 -- 172.21.15.3:0/3691431518 >> v1:172.21.15.3:6789/0 conn(0x7f65cc09d630 legacy=0x7f65cc0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:31.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.416+0000 7f65d0bbe700 1 -- 172.21.15.3:0/3691431518 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65cc09dc50 con 0x7f65cc138d80 2023-10-03T10:16:31.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.416+0000 7f65caffd700 1 -- 172.21.15.3:0/3691431518 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3838622257 0 0) 0x7f65cc0a2a70 con 0x7f65cc09d630 2023-10-03T10:16:31.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.416+0000 7f65cbfff700 1 --2- 172.21.15.3:0/3691431518 >> v2:172.21.15.3:3300/0 conn(0x7f65cc0a0a30 0x7f65cc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:31.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.416+0000 7f65d0bbe700 1 --2- 172.21.15.3:0/3691431518 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65cc138d80 0x7f65cc13b160 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009d90 tx=0x7f65c400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30443739dbfb5993 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:31.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.417+0000 7f65caffd700 1 -- 172.21.15.3:0/3691431518 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c400e070 con 0x7f65cc138d80 2023-10-03T10:16:31.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.417+0000 7f65caffd700 1 -- 172.21.15.3:0/3691431518 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c400bb20 con 0x7f65cc138d80 2023-10-03T10:16:31.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.417+0000 7f65caffd700 1 -- 172.21.15.3:0/3691431518 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c4012430 con 0x7f65cc138d80 2023-10-03T10:16:31.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.417+0000 7f01e0d39700 1 -- 172.21.15.3:0/3822129498 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f01dc0a0a30 msgr2=0x7f01dc11f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:31.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.417+0000 7f01e0d39700 1 --2- 172.21.15.3:0/3822129498 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f01dc0a0a30 0x7f01dc11f9c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f01d000e980 tx=0x7f01d000ec90 comp rx=0 tx=0).stop 2023-10-03T10:16:31.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.417+0000 7f65d2621700 1 -- 172.21.15.3:0/3691431518 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65cc138d80 msgr2=0x7f65cc13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:31.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.417+0000 7f65d2621700 1 --2- 172.21.15.3:0/3691431518 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65cc138d80 0x7f65cc13b160 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009d90 tx=0x7f65c400b330 comp rx=0 tx=0).stop 2023-10-03T10:16:31.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.417+0000 7f65d2621700 1 -- 172.21.15.3:0/3691431518 shutdown_connections 2023-10-03T10:16:31.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.418+0000 7f65d2621700 1 --2- 172.21.15.3:0/3691431518 >> v2:172.21.15.3:3300/0 conn(0x7f65cc0a0a30 0x7f65cc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.418+0000 7f65d2621700 1 --2- 172.21.15.3:0/3691431518 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65cc138d80 0x7f65cc13b160 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.418+0000 7f65d2621700 1 -- 172.21.15.3:0/3691431518 >> 172.21.15.3:0/3691431518 conn(0x7f65cc0a3050 msgr2=0x7f65cc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:31.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.418+0000 7f01e0d39700 1 -- 172.21.15.3:0/3822129498 shutdown_connections 2023-10-03T10:16:31.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.418+0000 7f01e0d39700 1 --2- 172.21.15.3:0/3822129498 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f01dc133930 0x7f01dc1c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.418+0000 7f01e0d39700 1 --2- 172.21.15.3:0/3822129498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01dc09d630 0x7f01dc11f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.418+0000 7f01e0d39700 1 --2- 172.21.15.3:0/3822129498 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f01dc0a0a30 0x7f01dc11f9c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.418+0000 7f01e0d39700 1 -- 172.21.15.3:0/3822129498 >> 172.21.15.3:0/3822129498 conn(0x7f01dc0a3050 msgr2=0x7f01dc09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:31.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.418+0000 7f65d2621700 1 -- 172.21.15.3:0/3691431518 shutdown_connections 2023-10-03T10:16:31.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.418+0000 7f65d2621700 1 -- 172.21.15.3:0/3691431518 wait complete. 2023-10-03T10:16:31.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.418+0000 7f01e0d39700 1 -- 172.21.15.3:0/3822129498 shutdown_connections 2023-10-03T10:16:31.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.418+0000 7f01e0d39700 1 -- 172.21.15.3:0/3822129498 wait complete. 2023-10-03T10:16:31.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.419+0000 7f65d2621700 1 Processor -- start 2023-10-03T10:16:31.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.419+0000 7f65d2621700 1 -- start start 2023-10-03T10:16:31.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.420+0000 7f65d2621700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65cc09d630 0x7f65cc11c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:31.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.420+0000 7f65d2621700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65cc0a0a30 0x7f65cc11cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:31.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.420+0000 7f65d2621700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65cc138d80 0x7f65cc11d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:31.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.420+0000 7f65d2621700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc11e760 con 0x7f65cc138d80 2023-10-03T10:16:31.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.420+0000 7f65d2621700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc071600 con 0x7f65cc09d630 2023-10-03T10:16:31.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.420+0000 7f65d2621700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc13aba0 con 0x7f65cc0a0a30 2023-10-03T10:16:31.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.420+0000 7f65cbfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65cc09d630 0x7f65cc11c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:31.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.420+0000 7f65cbfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65cc09d630 0x7f65cc11c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38036/0 (socket says 172.21.15.3:38036) 2023-10-03T10:16:31.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.420+0000 7f65cbfff700 1 -- 172.21.15.3:0/2039168132 learned_addr learned my addr 172.21.15.3:0/2039168132 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:31.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.420+0000 7f65d0bbe700 1 --2- 172.21.15.3:0/2039168132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65cc138d80 0x7f65cc11d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:31.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.420+0000 7f65cb7fe700 1 --2- 172.21.15.3:0/2039168132 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65cc0a0a30 0x7f65cc11cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:31.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.420+0000 7f65cbfff700 1 -- 172.21.15.3:0/2039168132 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65cc0a0a30 msgr2=0x7f65cc11cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:31.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.420+0000 7f65cbfff700 1 --2- 172.21.15.3:0/2039168132 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65cc0a0a30 0x7f65cc11cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.420+0000 7f65cbfff700 1 -- 172.21.15.3:0/2039168132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65cc138d80 msgr2=0x7f65cc11d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:31.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.420+0000 7f65cbfff700 1 --2- 172.21.15.3:0/2039168132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65cc138d80 0x7f65cc11d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.420+0000 7f65cbfff700 1 -- 172.21.15.3:0/2039168132 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65bc0094a0 con 0x7f65cc09d630 2023-10-03T10:16:31.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.421+0000 7f65cbfff700 1 --2- 172.21.15.3:0/2039168132 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65cc09d630 0x7f65cc11c7c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00ec10 tx=0x7f65bc00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:31.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.421+0000 7f65c8ff9700 1 -- 172.21.15.3:0/2039168132 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc00c850 con 0x7f65cc09d630 2023-10-03T10:16:31.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.421+0000 7f65c8ff9700 1 -- 172.21.15.3:0/2039168132 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65bc00c9b0 con 0x7f65cc09d630 2023-10-03T10:16:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.421+0000 7f65c8ff9700 1 -- 172.21.15.3:0/2039168132 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc00cd20 con 0x7f65cc09d630 2023-10-03T10:16:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.421+0000 7f65d2621700 1 -- 172.21.15.3:0/2039168132 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c4009a70 con 0x7f65cc09d630 2023-10-03T10:16:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.421+0000 7f65d2621700 1 -- 172.21.15.3:0/2039168132 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65cc0a0410 con 0x7f65cc09d630 2023-10-03T10:16:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.422+0000 7f65d2621700 1 -- 172.21.15.3:0/2039168132 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65cc11ea40 con 0x7f65cc09d630 2023-10-03T10:16:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.423+0000 7f65c8ff9700 1 -- 172.21.15.3:0/2039168132 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f65bc004020 con 0x7f65cc09d630 2023-10-03T10:16:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.423+0000 7f65c8ff9700 1 -- 172.21.15.3:0/2039168132 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f65bc03fa40 con 0x7f65cc09d630 2023-10-03T10:16:31.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.428+0000 7f65c8ff9700 1 -- 172.21.15.3:0/2039168132 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f65bc0036a0 con 0x7f65cc09d630 2023-10-03T10:16:31.439 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:31.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.537+0000 7f79c8e2a700 1 -- 172.21.15.3:0/1088123932 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f79c4074ea0 con 0x7f79c4133420 2023-10-03T10:16:31.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.537+0000 7f79b2ffd700 1 -- 172.21.15.3:0/1088123932 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f79b4029070 con 0x7f79c4133420 2023-10-03T10:16:31.538 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:31.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.540+0000 7f79c8e2a700 1 -- 172.21.15.3:0/1088123932 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79c4133420 msgr2=0x7f79c409c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:31.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.540+0000 7f79c8e2a700 1 --2- 172.21.15.3:0/1088123932 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79c4133420 0x7f79c409c8c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f79c40a06d0 tx=0x7f79b4004750 comp rx=0 tx=0).stop 2023-10-03T10:16:31.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.540+0000 7f79c8e2a700 1 -- 172.21.15.3:0/1088123932 shutdown_connections 2023-10-03T10:16:31.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.540+0000 7f79c8e2a700 1 --2- 172.21.15.3:0/1088123932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79c4134260 0x7f79c409d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.540+0000 7f79c8e2a700 1 --2- 172.21.15.3:0/1088123932 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79c4133420 0x7f79c409c8c0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.540+0000 7f79c8e2a700 1 --2- 172.21.15.3:0/1088123932 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79c4133e30 0x7f79c409ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.540+0000 7f79c8e2a700 1 -- 172.21.15.3:0/1088123932 >> 172.21.15.3:0/1088123932 conn(0x7f79c40a4210 msgr2=0x7f79c412fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:31.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.540+0000 7f79c8e2a700 1 -- 172.21.15.3:0/1088123932 shutdown_connections 2023-10-03T10:16:31.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.540+0000 7f79c8e2a700 1 -- 172.21.15.3:0/1088123932 wait complete. 2023-10-03T10:16:31.554 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:31.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.599+0000 7f84aa14f700 1 -- 172.21.15.3:0/320986819 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8484005590 con 0x7f84a4133a80 2023-10-03T10:16:31.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.600+0000 7f848bfff700 1 -- 172.21.15.3:0/320986819 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f849c007330 con 0x7f84a4133a80 2023-10-03T10:16:31.600 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:31.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.602+0000 7f84aa14f700 1 -- 172.21.15.3:0/320986819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84a4133a80 msgr2=0x7f84a409cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:31.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.602+0000 7f84aa14f700 1 --2- 172.21.15.3:0/320986819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84a4133a80 0x7f84a409cf00 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f849c00efc0 tx=0x7f849c00c3a0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.602+0000 7f84aa14f700 1 -- 172.21.15.3:0/320986819 shutdown_connections 2023-10-03T10:16:31.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.602+0000 7f84aa14f700 1 --2- 172.21.15.3:0/320986819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84a4133a80 0x7f84a409cf00 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.602+0000 7f84aa14f700 1 --2- 172.21.15.3:0/320986819 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f84a412ff20 0x7f84a409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.602+0000 7f84aa14f700 1 --2- 172.21.15.3:0/320986819 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f84a4130a40 0x7f84a409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.602+0000 7f84aa14f700 1 -- 172.21.15.3:0/320986819 >> 172.21.15.3:0/320986819 conn(0x7f84a40a4210 msgr2=0x7f84a409e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:31.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.602+0000 7f84aa14f700 1 -- 172.21.15.3:0/320986819 shutdown_connections 2023-10-03T10:16:31.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.602+0000 7f84aa14f700 1 -- 172.21.15.3:0/320986819 wait complete. 2023-10-03T10:16:31.616 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:31.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.637+0000 7f65d2621700 1 -- 172.21.15.3:0/2039168132 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f65cc11ea40 con 0x7f65cc09d630 2023-10-03T10:16:31.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.639+0000 7f65c8ff9700 1 -- 172.21.15.3:0/2039168132 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f65cc11ea40 con 0x7f65cc09d630 2023-10-03T10:16:31.640 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:31.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.642+0000 7f65d2621700 1 -- 172.21.15.3:0/2039168132 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65cc09d630 msgr2=0x7f65cc11c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:31.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.642+0000 7f65d2621700 1 --2- 172.21.15.3:0/2039168132 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65cc09d630 0x7f65cc11c7c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00ec10 tx=0x7f65bc00ef20 comp rx=0 tx=0).stop 2023-10-03T10:16:31.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.642+0000 7f65d2621700 1 -- 172.21.15.3:0/2039168132 shutdown_connections 2023-10-03T10:16:31.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.642+0000 7f65d2621700 1 --2- 172.21.15.3:0/2039168132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65cc138d80 0x7f65cc11d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.642+0000 7f65d2621700 1 --2- 172.21.15.3:0/2039168132 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65cc09d630 0x7f65cc11c7c0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.642+0000 7f65d2621700 1 --2- 172.21.15.3:0/2039168132 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65cc0a0a30 0x7f65cc11cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:31.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.642+0000 7f65d2621700 1 -- 172.21.15.3:0/2039168132 >> 172.21.15.3:0/2039168132 conn(0x7f65cc0a3050 msgr2=0x7f65cc099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:31.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.642+0000 7f65d2621700 1 -- 172.21.15.3:0/2039168132 shutdown_connections 2023-10-03T10:16:31.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:31.642+0000 7f65d2621700 1 -- 172.21.15.3:0/2039168132 wait complete. 2023-10-03T10:16:31.656 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:32.440 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:32.555 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:32.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.586+0000 7f67204a3700 1 Processor -- start 2023-10-03T10:16:32.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.586+0000 7f67204a3700 1 -- start start 2023-10-03T10:16:32.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.587+0000 7f67204a3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f671809e930 0x7f671809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:32.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.587+0000 7f67204a3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f671809f650 0x7f6718131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:32.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.587+0000 7f67204a3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6718070940 con 0x7f671809f240 2023-10-03T10:16:32.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.587+0000 7f67204a3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f671806fd60 con 0x7f671809e930 2023-10-03T10:16:32.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.587+0000 7f67204a3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f671806fa80 con 0x7f671809f650 2023-10-03T10:16:32.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.587+0000 7f671ea40700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f671809f650 0x7f6718131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:32.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.587+0000 7f671e23f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f671809e930 0x7f671809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:32.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.587+0000 7f671ea40700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f671809f650 0x7f6718131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51742/0 (socket says 172.21.15.3:51742) 2023-10-03T10:16:32.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.587+0000 7f671ea40700 1 -- 172.21.15.3:0/183742087 learned_addr learned my addr 172.21.15.3:0/183742087 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:32.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.588+0000 7f671e23f700 1 -- 172.21.15.3:0/183742087 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f671809f650 msgr2=0x7f6718131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:32.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.588+0000 7f671e23f700 1 --2- 172.21.15.3:0/183742087 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f671809f650 0x7f6718131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.588+0000 7f671d23d700 1 -- 172.21.15.3:0/183742087 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3288186105 0 0) 0x7f6718070940 con 0x7f671809f240 2023-10-03T10:16:32.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.588+0000 7f671e23f700 1 -- 172.21.15.3:0/183742087 >> v1:172.21.15.3:6789/0 conn(0x7f671809f240 legacy=0x7f67181317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:32.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.588+0000 7f671e23f700 1 -- 172.21.15.3:0/183742087 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671809fb60 con 0x7f671809e930 2023-10-03T10:16:32.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.588+0000 7f671ea40700 1 --2- 172.21.15.3:0/183742087 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f671809f650 0x7f6718131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:32.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.588+0000 7f671e23f700 1 --2- 172.21.15.3:0/183742087 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f671809e930 0x7f671809ed00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6714009870 tx=0x7f6714009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ddcbca2b65660d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:32.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.588+0000 7f671d23d700 1 -- 172.21.15.3:0/183742087 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6714011830 con 0x7f671809e930 2023-10-03T10:16:32.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.588+0000 7f671d23d700 1 -- 172.21.15.3:0/183742087 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6714011990 con 0x7f671809e930 2023-10-03T10:16:32.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.588+0000 7f671d23d700 1 -- 172.21.15.3:0/183742087 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6714011d20 con 0x7f671809e930 2023-10-03T10:16:32.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.589+0000 7f67204a3700 1 -- 172.21.15.3:0/183742087 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f671809e930 msgr2=0x7f671809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:32.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.589+0000 7f67204a3700 1 --2- 172.21.15.3:0/183742087 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f671809e930 0x7f671809ed00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6714009870 tx=0x7f6714009b80 comp rx=0 tx=0).stop 2023-10-03T10:16:32.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.589+0000 7f67204a3700 1 -- 172.21.15.3:0/183742087 shutdown_connections 2023-10-03T10:16:32.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.589+0000 7f67204a3700 1 --2- 172.21.15.3:0/183742087 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f671809e930 0x7f671809ed00 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.589+0000 7f67204a3700 1 --2- 172.21.15.3:0/183742087 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f671809f650 0x7f6718131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.589+0000 7f67204a3700 1 -- 172.21.15.3:0/183742087 >> 172.21.15.3:0/183742087 conn(0x7f67180a4210 msgr2=0x7f67180a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:32.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.589+0000 7f67204a3700 1 -- 172.21.15.3:0/183742087 shutdown_connections 2023-10-03T10:16:32.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.589+0000 7f67204a3700 1 -- 172.21.15.3:0/183742087 wait complete. 2023-10-03T10:16:32.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.590+0000 7f67204a3700 1 Processor -- start 2023-10-03T10:16:32.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.590+0000 7f67204a3700 1 -- start start 2023-10-03T10:16:32.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.591+0000 7f67204a3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f671809e930 0x7f671806b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:32.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.591+0000 7f67204a3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f671809f240 0x7f671806bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:32.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.591+0000 7f67204a3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f671809f650 0x7f671806c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:32.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.591+0000 7f67204a3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67181a96a0 con 0x7f671809f650 2023-10-03T10:16:32.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.591+0000 7f67204a3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67181b7190 con 0x7f671809f240 2023-10-03T10:16:32.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.591+0000 7f67204a3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67181a8860 con 0x7f671809e930 2023-10-03T10:16:32.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.591+0000 7f671ea40700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f671809f650 0x7f671806c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:32.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.591+0000 7f671da3e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f671809f240 0x7f671806bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:32.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.591+0000 7f671ea40700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f671809f650 0x7f671806c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42794/0 (socket says 172.21.15.3:42794) 2023-10-03T10:16:32.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.591+0000 7f671ea40700 1 -- 172.21.15.3:0/2871641301 learned_addr learned my addr 172.21.15.3:0/2871641301 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:32.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.591+0000 7f671da3e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f671809f240 0x7f671806bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38058/0 (socket says 172.21.15.3:38058) 2023-10-03T10:16:32.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.591+0000 7f671e23f700 1 --2- 172.21.15.3:0/2871641301 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f671809e930 0x7f671806b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:32.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.591+0000 7f671e23f700 1 -- 172.21.15.3:0/2871641301 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f671809f240 msgr2=0x7f671806bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:32.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.591+0000 7f671e23f700 1 --2- 172.21.15.3:0/2871641301 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f671809f240 0x7f671806bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.591+0000 7f671e23f700 1 -- 172.21.15.3:0/2871641301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f671809f650 msgr2=0x7f671806c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:32.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.591+0000 7f671e23f700 1 --2- 172.21.15.3:0/2871641301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f671809f650 0x7f671806c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.591+0000 7f671e23f700 1 -- 172.21.15.3:0/2871641301 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6710009b30 con 0x7f671809e930 2023-10-03T10:16:32.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.591+0000 7f671da3e700 1 --2- 172.21.15.3:0/2871641301 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f671809f240 0x7f671806bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:32.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.591+0000 7f671e23f700 1 --2- 172.21.15.3:0/2871641301 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f671809e930 0x7f671806b8b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f67180a1bf0 tx=0x7f6714004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:32.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.591+0000 7f670effd700 1 -- 172.21.15.3:0/2871641301 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67140044d0 con 0x7f671809e930 2023-10-03T10:16:32.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.592+0000 7f670effd700 1 -- 172.21.15.3:0/2871641301 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671401f070 con 0x7f671809e930 2023-10-03T10:16:32.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.592+0000 7f67204a3700 1 -- 172.21.15.3:0/2871641301 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67140094a0 con 0x7f671809e930 2023-10-03T10:16:32.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.592+0000 7f670effd700 1 -- 172.21.15.3:0/2871641301 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67140038c0 con 0x7f671809e930 2023-10-03T10:16:32.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.592+0000 7f67204a3700 1 -- 172.21.15.3:0/2871641301 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67180a0190 con 0x7f671809e930 2023-10-03T10:16:32.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.592+0000 7f670effd700 1 -- 172.21.15.3:0/2871641301 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6714003a20 con 0x7f671809e930 2023-10-03T10:16:32.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.593+0000 7f670effd700 1 -- 172.21.15.3:0/2871641301 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6714021070 con 0x7f671809e930 2023-10-03T10:16:32.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.593+0000 7f67204a3700 1 -- 172.21.15.3:0/2871641301 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67181a0b30 con 0x7f671809e930 2023-10-03T10:16:32.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.598+0000 7f670effd700 1 -- 172.21.15.3:0/2871641301 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f671402b020 con 0x7f671809e930 2023-10-03T10:16:32.617 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:32.657 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:32.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.705+0000 7ff6658b1700 1 Processor -- start 2023-10-03T10:16:32.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.705+0000 7ff6658b1700 1 -- start start 2023-10-03T10:16:32.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.706+0000 7ff6658b1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff66009fa10 0x7ff660136950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:32.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.706+0000 7ff6658b1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff66009f2c0 0x7ff660138ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:32.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.706+0000 7ff6658b1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff66009b610 con 0x7ff66009e7e0 2023-10-03T10:16:32.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.706+0000 7ff6658b1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66009bbb0 con 0x7ff66009fa10 2023-10-03T10:16:32.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.706+0000 7ff6658b1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6600a0010 con 0x7ff66009f2c0 2023-10-03T10:16:32.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.706+0000 7ff65f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff66009f2c0 0x7ff660138ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:32.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.706+0000 7ff65f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff66009f2c0 0x7ff660138ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51760/0 (socket says 172.21.15.3:51760) 2023-10-03T10:16:32.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.706+0000 7ff65f7fe700 1 -- 172.21.15.3:0/2949269235 learned_addr learned my addr 172.21.15.3:0/2949269235 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:32.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.706+0000 7ff65dffb700 1 -- 172.21.15.3:0/2949269235 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1840232971 0 0) 0x7ff66009b610 con 0x7ff66009e7e0 2023-10-03T10:16:32.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.707+0000 7ff65dffb700 1 -- 172.21.15.3:0/2949269235 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6480035b0 con 0x7ff66009e7e0 2023-10-03T10:16:32.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.707+0000 7ff65e7fc700 1 --2- 172.21.15.3:0/2949269235 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff66009fa10 0x7ff660136950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:32.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.707+0000 7ff65f7fe700 1 -- 172.21.15.3:0/2949269235 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff66009fa10 msgr2=0x7ff660136950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:32.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.707+0000 7ff65f7fe700 1 --2- 172.21.15.3:0/2949269235 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff66009fa10 0x7ff660136950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.707+0000 7ff65f7fe700 1 -- 172.21.15.3:0/2949269235 >> v1:172.21.15.3:6789/0 conn(0x7ff66009e7e0 legacy=0x7ff66009ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:32.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.707+0000 7ff65f7fe700 1 -- 172.21.15.3:0/2949269235 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff66009b8d0 con 0x7ff66009f2c0 2023-10-03T10:16:32.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.707+0000 7ff65e7fc700 1 --2- 172.21.15.3:0/2949269235 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff66009fa10 0x7ff660136950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:16:32.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.707+0000 7ff65f7fe700 1 --2- 172.21.15.3:0/2949269235 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff66009f2c0 0x7ff660138ea0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff658009e80 tx=0x7ff65800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45a5765d4d671beb server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:32.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.707+0000 7ff65dffb700 1 -- 172.21.15.3:0/2949269235 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65800e070 con 0x7ff66009f2c0 2023-10-03T10:16:32.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.707+0000 7ff65dffb700 1 -- 172.21.15.3:0/2949269235 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65800b910 con 0x7ff66009f2c0 2023-10-03T10:16:32.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.707+0000 7ff65dffb700 1 -- 172.21.15.3:0/2949269235 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65800bcc0 con 0x7ff66009f2c0 2023-10-03T10:16:32.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.708+0000 7ff6658b1700 1 -- 172.21.15.3:0/2949269235 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff66009f2c0 msgr2=0x7ff660138ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:32.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.708+0000 7ff6658b1700 1 --2- 172.21.15.3:0/2949269235 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff66009f2c0 0x7ff660138ea0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff658009e80 tx=0x7ff65800b330 comp rx=0 tx=0).stop 2023-10-03T10:16:32.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.708+0000 7ff6658b1700 1 -- 172.21.15.3:0/2949269235 shutdown_connections 2023-10-03T10:16:32.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.708+0000 7ff6658b1700 1 --2- 172.21.15.3:0/2949269235 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff66009f2c0 0x7ff660138ea0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.708+0000 7ff6658b1700 1 --2- 172.21.15.3:0/2949269235 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff66009fa10 0x7ff660136950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.708+0000 7ff6658b1700 1 -- 172.21.15.3:0/2949269235 >> 172.21.15.3:0/2949269235 conn(0x7ff6600a4300 msgr2=0x7ff6600a4700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:32.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.708+0000 7ff6658b1700 1 -- 172.21.15.3:0/2949269235 shutdown_connections 2023-10-03T10:16:32.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.708+0000 7ff6658b1700 1 -- 172.21.15.3:0/2949269235 wait complete. 2023-10-03T10:16:32.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.709+0000 7ff6658b1700 1 Processor -- start 2023-10-03T10:16:32.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.710+0000 7ff6658b1700 1 -- start start 2023-10-03T10:16:32.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.710+0000 7ff6658b1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff66009e7e0 0x7ff66006b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:32.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.711+0000 7ff6658b1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff66009f2c0 0x7ff66006bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:32.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.711+0000 7ff6658b1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff66009fa10 0x7ff66006c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:32.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.711+0000 7ff6658b1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6600a2940 con 0x7ff66009f2c0 2023-10-03T10:16:32.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.711+0000 7ff6658b1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66011f5d0 con 0x7ff66009fa10 2023-10-03T10:16:32.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.711+0000 7ff6658b1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff66011eb10 con 0x7ff66009e7e0 2023-10-03T10:16:32.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.711+0000 7ff65f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff66009fa10 0x7ff66006c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:32.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.711+0000 7ff65effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff66009e7e0 0x7ff66006b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:32.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.711+0000 7ff65f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff66009fa10 0x7ff66006c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38074/0 (socket says 172.21.15.3:38074) 2023-10-03T10:16:32.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.711+0000 7ff65f7fe700 1 -- 172.21.15.3:0/2663772280 learned_addr learned my addr 172.21.15.3:0/2663772280 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:32.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.711+0000 7ff65e7fc700 1 --2- 172.21.15.3:0/2663772280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff66009f2c0 0x7ff66006bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:32.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.711+0000 7ff65f7fe700 1 -- 172.21.15.3:0/2663772280 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff66009e7e0 msgr2=0x7ff66006b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:32.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.711+0000 7ff65f7fe700 1 --2- 172.21.15.3:0/2663772280 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff66009e7e0 0x7ff66006b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.711+0000 7ff65f7fe700 1 -- 172.21.15.3:0/2663772280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff66009f2c0 msgr2=0x7ff66006bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:32.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.711+0000 7ff65f7fe700 1 --2- 172.21.15.3:0/2663772280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff66009f2c0 0x7ff66006bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.711+0000 7ff65f7fe700 1 -- 172.21.15.3:0/2663772280 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff658009b30 con 0x7ff66009fa10 2023-10-03T10:16:32.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.711+0000 7ff65f7fe700 1 --2- 172.21.15.3:0/2663772280 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff66009fa10 0x7ff66006c300 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff6600a0150 tx=0x7ff658004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:32.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.712+0000 7ff647fff700 1 -- 172.21.15.3:0/2663772280 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6580094d0 con 0x7ff66009fa10 2023-10-03T10:16:32.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.712+0000 7ff647fff700 1 -- 172.21.15.3:0/2663772280 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff658004250 con 0x7ff66009fa10 2023-10-03T10:16:32.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.712+0000 7ff6658b1700 1 -- 172.21.15.3:0/2663772280 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff66011f890 con 0x7ff66009fa10 2023-10-03T10:16:32.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.712+0000 7ff647fff700 1 -- 172.21.15.3:0/2663772280 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff658004600 con 0x7ff66009fa10 2023-10-03T10:16:32.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.712+0000 7ff6658b1700 1 -- 172.21.15.3:0/2663772280 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff66006d570 con 0x7ff66009fa10 2023-10-03T10:16:32.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.713+0000 7ff647fff700 1 -- 172.21.15.3:0/2663772280 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff658004760 con 0x7ff66009fa10 2023-10-03T10:16:32.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.713+0000 7ff6658b1700 1 -- 172.21.15.3:0/2663772280 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff66011fa50 con 0x7ff66009fa10 2023-10-03T10:16:32.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.713+0000 7ff647fff700 1 -- 172.21.15.3:0/2663772280 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff658040be0 con 0x7ff66009fa10 2023-10-03T10:16:32.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.718+0000 7ff647fff700 1 -- 172.21.15.3:0/2663772280 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff658019760 con 0x7ff66009fa10 2023-10-03T10:16:32.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.770+0000 7feb62228700 1 Processor -- start 2023-10-03T10:16:32.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.770+0000 7feb62228700 1 -- start start 2023-10-03T10:16:32.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.771+0000 7feb62228700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feb5c09ffb0 0x7feb5c09f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:32.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.771+0000 7feb62228700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feb5c138e30 0x7feb5c13b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:32.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.771+0000 7feb62228700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb5c0a06b0 con 0x7feb5c09e850 2023-10-03T10:16:32.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.771+0000 7feb62228700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb5c06b920 con 0x7feb5c09ffb0 2023-10-03T10:16:32.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.771+0000 7feb62228700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c09aa30 con 0x7feb5c138e30 2023-10-03T10:16:32.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.771+0000 7feb5b7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7feb5c09e850 0x7feb5c09ec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35524/0 (socket says 172.21.15.3:35524) 2023-10-03T10:16:32.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.771+0000 7feb5b7fe700 1 -- 172.21.15.3:0/1523751031 learned_addr learned my addr 172.21.15.3:0/1523751031 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:32.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.771+0000 7feb5bfff700 1 --2- 172.21.15.3:0/1523751031 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feb5c138e30 0x7feb5c13b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:32.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.771+0000 7feb5affd700 1 --2- 172.21.15.3:0/1523751031 >> v2:172.21.15.3:3300/0 conn(0x7feb5c09ffb0 0x7feb5c09f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:32.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.771+0000 7feb5affd700 1 -- 172.21.15.3:0/1523751031 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feb5c138e30 msgr2=0x7feb5c13b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:32.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.771+0000 7feb5affd700 1 --2- 172.21.15.3:0/1523751031 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feb5c138e30 0x7feb5c13b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.772+0000 7feb5affd700 1 -- 172.21.15.3:0/1523751031 >> v1:172.21.15.3:6789/0 conn(0x7feb5c09e850 legacy=0x7feb5c09ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:32.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.772+0000 7feb5a7fc700 1 -- 172.21.15.3:0/1523751031 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1267138294 0 0) 0x7feb5c0a06b0 con 0x7feb5c09e850 2023-10-03T10:16:32.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.772+0000 7feb5affd700 1 -- 172.21.15.3:0/1523751031 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb5c06caa0 con 0x7feb5c09ffb0 2023-10-03T10:16:32.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.772+0000 7feb5bfff700 1 --2- 172.21.15.3:0/1523751031 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feb5c138e30 0x7feb5c13b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:32.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.772+0000 7feb5affd700 1 --2- 172.21.15.3:0/1523751031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb5c09ffb0 0x7feb5c09f330 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7feb50009840 tx=0x7feb50009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914f2daa8543d361 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:32.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.772+0000 7feb5a7fc700 1 -- 172.21.15.3:0/1523751031 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb50011830 con 0x7feb5c09ffb0 2023-10-03T10:16:32.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.772+0000 7feb5a7fc700 1 -- 172.21.15.3:0/1523751031 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb50011990 con 0x7feb5c09ffb0 2023-10-03T10:16:32.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.772+0000 7feb5a7fc700 1 -- 172.21.15.3:0/1523751031 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb50011d20 con 0x7feb5c09ffb0 2023-10-03T10:16:32.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.773+0000 7feb62228700 1 -- 172.21.15.3:0/1523751031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb5c09ffb0 msgr2=0x7feb5c09f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:32.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.773+0000 7feb62228700 1 --2- 172.21.15.3:0/1523751031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb5c09ffb0 0x7feb5c09f330 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7feb50009840 tx=0x7feb50009b50 comp rx=0 tx=0).stop 2023-10-03T10:16:32.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.773+0000 7feb62228700 1 -- 172.21.15.3:0/1523751031 shutdown_connections 2023-10-03T10:16:32.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.773+0000 7feb62228700 1 --2- 172.21.15.3:0/1523751031 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feb5c138e30 0x7feb5c13b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.773+0000 7feb62228700 1 --2- 172.21.15.3:0/1523751031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb5c09ffb0 0x7feb5c09f330 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.773+0000 7feb62228700 1 -- 172.21.15.3:0/1523751031 >> 172.21.15.3:0/1523751031 conn(0x7feb5c0a3f70 msgr2=0x7feb5c0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:32.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.773+0000 7feb62228700 1 -- 172.21.15.3:0/1523751031 shutdown_connections 2023-10-03T10:16:32.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.773+0000 7feb62228700 1 -- 172.21.15.3:0/1523751031 wait complete. 2023-10-03T10:16:32.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.774+0000 7feb62228700 1 Processor -- start 2023-10-03T10:16:32.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.774+0000 7feb62228700 1 -- start start 2023-10-03T10:16:32.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.775+0000 7feb62228700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feb5c09e850 0x7feb5c1208d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:32.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.775+0000 7feb62228700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb5c09ffb0 0x7feb5c120e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:32.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.775+0000 7feb62228700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feb5c138e30 0x7feb5c121350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:32.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.775+0000 7feb62228700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c1243a0 con 0x7feb5c09ffb0 2023-10-03T10:16:32.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.775+0000 7feb62228700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c0a3960 con 0x7feb5c138e30 2023-10-03T10:16:32.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.775+0000 7feb62228700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c06ccc0 con 0x7feb5c09e850 2023-10-03T10:16:32.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.775+0000 7feb5affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb5c09ffb0 0x7feb5c120e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:32.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.775+0000 7feb5affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb5c09ffb0 0x7feb5c120e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42812/0 (socket says 172.21.15.3:42812) 2023-10-03T10:16:32.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.775+0000 7feb5affd700 1 -- 172.21.15.3:0/981299308 learned_addr learned my addr 172.21.15.3:0/981299308 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:32.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.775+0000 7feb5b7fe700 1 --2- 172.21.15.3:0/981299308 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feb5c09e850 0x7feb5c1208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:32.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.775+0000 7feb5bfff700 1 --2- 172.21.15.3:0/981299308 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feb5c138e30 0x7feb5c121350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:32.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.775+0000 7feb5affd700 1 -- 172.21.15.3:0/981299308 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feb5c09e850 msgr2=0x7feb5c1208d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:32.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.775+0000 7feb5affd700 1 --2- 172.21.15.3:0/981299308 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feb5c09e850 0x7feb5c1208d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.775+0000 7feb5affd700 1 -- 172.21.15.3:0/981299308 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feb5c138e30 msgr2=0x7feb5c121350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:32.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.775+0000 7feb5affd700 1 --2- 172.21.15.3:0/981299308 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feb5c138e30 0x7feb5c121350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.775+0000 7feb5affd700 1 -- 172.21.15.3:0/981299308 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb54009a70 con 0x7feb5c09ffb0 2023-10-03T10:16:32.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.776+0000 7feb5affd700 1 --2- 172.21.15.3:0/981299308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb5c09ffb0 0x7feb5c120e10 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7feb50009fd0 tx=0x7feb50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:32.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.776+0000 7feb43fff700 1 -- 172.21.15.3:0/981299308 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb500036a0 con 0x7feb5c09ffb0 2023-10-03T10:16:32.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.776+0000 7feb43fff700 1 -- 172.21.15.3:0/981299308 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb5001f070 con 0x7feb5c09ffb0 2023-10-03T10:16:32.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.776+0000 7feb62228700 1 -- 172.21.15.3:0/981299308 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb500094a0 con 0x7feb5c09ffb0 2023-10-03T10:16:32.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.776+0000 7feb43fff700 1 -- 172.21.15.3:0/981299308 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb50003a20 con 0x7feb5c09ffb0 2023-10-03T10:16:32.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.776+0000 7feb62228700 1 -- 172.21.15.3:0/981299308 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb5c123b30 con 0x7feb5c09ffb0 2023-10-03T10:16:32.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.777+0000 7feb43fff700 1 -- 172.21.15.3:0/981299308 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feb50003b80 con 0x7feb5c09ffb0 2023-10-03T10:16:32.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.777+0000 7feb62228700 1 -- 172.21.15.3:0/981299308 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb5c0747f0 con 0x7feb5c09ffb0 2023-10-03T10:16:32.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.777+0000 7feb43fff700 1 -- 172.21.15.3:0/981299308 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7feb50021070 con 0x7feb5c09ffb0 2023-10-03T10:16:32.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.787+0000 7feb43fff700 1 -- 172.21.15.3:0/981299308 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feb500253e0 con 0x7feb5c09ffb0 2023-10-03T10:16:32.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.820+0000 7f08d889f700 1 Processor -- start 2023-10-03T10:16:32.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.820+0000 7f08d889f700 1 -- start start 2023-10-03T10:16:32.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.821+0000 7f08d889f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08d40a0a30 0x7f08d40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:32.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.821+0000 7f08d889f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f08d409d630 0x7f08d40a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:32.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.821+0000 7f08d889f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08d4133930 0x7f08d4133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:32.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.821+0000 7f08d889f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08d40a2a70 con 0x7f08d409d630 2023-10-03T10:16:32.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.821+0000 7f08d889f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d40a01d0 con 0x7f08d4133930 2023-10-03T10:16:32.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.821+0000 7f08d889f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f08d409e2b0 con 0x7f08d40a0a30 2023-10-03T10:16:32.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.821+0000 7f08d1d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f08d409d630 0x7f08d40a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:32.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.821+0000 7f08d1d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f08d409d630 0x7f08d40a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42826/0 (socket says 172.21.15.3:42826) 2023-10-03T10:16:32.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.821+0000 7f08d1d9b700 1 -- 172.21.15.3:0/1472941443 learned_addr learned my addr 172.21.15.3:0/1472941443 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:32.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.821+0000 7f08d259c700 1 --2- 172.21.15.3:0/1472941443 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08d40a0a30 0x7f08d40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:32.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.821+0000 7f08d2d9d700 1 --2- 172.21.15.3:0/1472941443 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08d4133930 0x7f08d4133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:32.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.822+0000 7f08d1d9b700 1 -- 172.21.15.3:0/1472941443 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08d40a0a30 msgr2=0x7f08d40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:32.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.822+0000 7f08d1d9b700 1 --2- 172.21.15.3:0/1472941443 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08d40a0a30 0x7f08d40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.822+0000 7f08d1d9b700 1 -- 172.21.15.3:0/1472941443 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08d4133930 msgr2=0x7f08d4133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:32.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.822+0000 7f08d1d9b700 1 --2- 172.21.15.3:0/1472941443 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08d4133930 0x7f08d4133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.822+0000 7f08d1d9b700 1 -- 172.21.15.3:0/1472941443 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d409dc50 con 0x7f08d409d630 2023-10-03T10:16:32.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.822+0000 7f08d1d9b700 1 --2- 172.21.15.3:0/1472941443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d409d630 0x7f08d40a1340 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f08c80098f0 tx=0x7f08c8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0edf1ac477bb99 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:32.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.822+0000 7f08d159a700 1 -- 172.21.15.3:0/1472941443 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c8011830 con 0x7f08d409d630 2023-10-03T10:16:32.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.822+0000 7f08d159a700 1 -- 172.21.15.3:0/1472941443 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c8011990 con 0x7f08d409d630 2023-10-03T10:16:32.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.822+0000 7f08d159a700 1 -- 172.21.15.3:0/1472941443 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c8011d20 con 0x7f08d409d630 2023-10-03T10:16:32.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.823+0000 7f08d889f700 1 -- 172.21.15.3:0/1472941443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d409d630 msgr2=0x7f08d40a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:32.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.823+0000 7f08d889f700 1 --2- 172.21.15.3:0/1472941443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d409d630 0x7f08d40a1340 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f08c80098f0 tx=0x7f08c8009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:32.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.823+0000 7f08d889f700 1 -- 172.21.15.3:0/1472941443 shutdown_connections 2023-10-03T10:16:32.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.823+0000 7f08d889f700 1 --2- 172.21.15.3:0/1472941443 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08d4133930 0x7f08d4133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.823+0000 7f08d889f700 1 --2- 172.21.15.3:0/1472941443 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08d40a0a30 0x7f08d40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.823+0000 7f08d889f700 1 --2- 172.21.15.3:0/1472941443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d409d630 0x7f08d40a1340 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.823+0000 7f08d889f700 1 -- 172.21.15.3:0/1472941443 >> 172.21.15.3:0/1472941443 conn(0x7f08d40a3050 msgr2=0x7f08d40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:32.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.823+0000 7f08d889f700 1 -- 172.21.15.3:0/1472941443 shutdown_connections 2023-10-03T10:16:32.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.823+0000 7f08d889f700 1 -- 172.21.15.3:0/1472941443 wait complete. 2023-10-03T10:16:32.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.825+0000 7f08d889f700 1 Processor -- start 2023-10-03T10:16:32.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.825+0000 7f08d889f700 1 -- start start 2023-10-03T10:16:32.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.826+0000 7f08d889f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08d409d630 0x7f08d4132940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:32.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.826+0000 7f08d889f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08d40a0a30 0x7f08d4132e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:32.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.826+0000 7f08d889f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d4133930 0x7f08d41c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:32.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.826+0000 7f08d889f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d4122310 con 0x7f08d4133930 2023-10-03T10:16:32.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.826+0000 7f08d889f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d41333c0 con 0x7f08d40a0a30 2023-10-03T10:16:32.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.826+0000 7f08d889f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f08d4120650 con 0x7f08d409d630 2023-10-03T10:16:32.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.826+0000 7f08d259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08d409d630 0x7f08d4132940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:32.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.826+0000 7f08d259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08d409d630 0x7f08d4132940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51816/0 (socket says 172.21.15.3:51816) 2023-10-03T10:16:32.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.826+0000 7f08d259c700 1 -- 172.21.15.3:0/2316705871 learned_addr learned my addr 172.21.15.3:0/2316705871 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:32.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.826+0000 7f08d2d9d700 1 --2- 172.21.15.3:0/2316705871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d4133930 0x7f08d41c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:32.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.826+0000 7f08d1d9b700 1 --2- 172.21.15.3:0/2316705871 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08d40a0a30 0x7f08d4132e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:32.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.827+0000 7f08d259c700 1 -- 172.21.15.3:0/2316705871 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08d40a0a30 msgr2=0x7f08d4132e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:32.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.827+0000 7f08d259c700 1 --2- 172.21.15.3:0/2316705871 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08d40a0a30 0x7f08d4132e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.827+0000 7f08d259c700 1 -- 172.21.15.3:0/2316705871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d4133930 msgr2=0x7f08d41c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:32.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.827+0000 7f08d259c700 1 --2- 172.21.15.3:0/2316705871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d4133930 0x7f08d41c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.827+0000 7f08d259c700 1 -- 172.21.15.3:0/2316705871 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c8009550 con 0x7f08d409d630 2023-10-03T10:16:32.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.827+0000 7f08d259c700 1 --2- 172.21.15.3:0/2316705871 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08d409d630 0x7f08d4132940 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f08c400e9b0 tx=0x7f08c400ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:32.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.827+0000 7f08c2ffd700 1 -- 172.21.15.3:0/2316705871 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c4009710 con 0x7f08d409d630 2023-10-03T10:16:32.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.827+0000 7f08c2ffd700 1 -- 172.21.15.3:0/2316705871 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c400ee50 con 0x7f08d409d630 2023-10-03T10:16:32.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.827+0000 7f08d889f700 1 -- 172.21.15.3:0/2316705871 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d40a5040 con 0x7f08d409d630 2023-10-03T10:16:32.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.827+0000 7f08c2ffd700 1 -- 172.21.15.3:0/2316705871 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c400ca50 con 0x7f08d409d630 2023-10-03T10:16:32.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.827+0000 7f08d889f700 1 -- 172.21.15.3:0/2316705871 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d4120e40 con 0x7f08d409d630 2023-10-03T10:16:32.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.828+0000 7f08c2ffd700 1 -- 172.21.15.3:0/2316705871 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f08c400cbb0 con 0x7f08d409d630 2023-10-03T10:16:32.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.828+0000 7f08c2ffd700 1 -- 172.21.15.3:0/2316705871 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f08c403f790 con 0x7f08d409d630 2023-10-03T10:16:32.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.830+0000 7f08d889f700 1 -- 172.21.15.3:0/2316705871 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08b80052e0 con 0x7f08d409d630 2023-10-03T10:16:32.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.832+0000 7f67204a3700 1 -- 172.21.15.3:0/2871641301 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6718071d20 con 0x7f671809e930 2023-10-03T10:16:32.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.839+0000 7f670effd700 1 -- 172.21.15.3:0/2871641301 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6714015070 con 0x7f671809e930 2023-10-03T10:16:32.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.839+0000 7f08c2ffd700 1 -- 172.21.15.3:0/2316705871 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f08c4021020 con 0x7f08d409d630 2023-10-03T10:16:32.840 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.842+0000 7f67204a3700 1 -- 172.21.15.3:0/2871641301 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f671809e930 msgr2=0x7f671806b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.842+0000 7f67204a3700 1 --2- 172.21.15.3:0/2871641301 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f671809e930 0x7f671806b8b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f67180a1bf0 tx=0x7f6714004750 comp rx=0 tx=0).stop 2023-10-03T10:16:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.842+0000 7f67204a3700 1 -- 172.21.15.3:0/2871641301 shutdown_connections 2023-10-03T10:16:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.842+0000 7f67204a3700 1 --2- 172.21.15.3:0/2871641301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f671809f650 0x7f671806c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.842+0000 7f67204a3700 1 --2- 172.21.15.3:0/2871641301 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f671809e930 0x7f671806b8b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.842+0000 7f67204a3700 1 --2- 172.21.15.3:0/2871641301 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f671809f240 0x7f671806bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.842+0000 7f67204a3700 1 -- 172.21.15.3:0/2871641301 >> 172.21.15.3:0/2871641301 conn(0x7f67180a4210 msgr2=0x7f67181b8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:32.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.843+0000 7f67204a3700 1 -- 172.21.15.3:0/2871641301 shutdown_connections 2023-10-03T10:16:32.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.844+0000 7f67204a3700 1 -- 172.21.15.3:0/2871641301 wait complete. 2023-10-03T10:16:32.858 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:32.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.954+0000 7ff6658b1700 1 -- 172.21.15.3:0/2663772280 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff66004f2a0 con 0x7ff66009fa10 2023-10-03T10:16:32.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.955+0000 7ff647fff700 1 -- 172.21.15.3:0/2663772280 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff658013e80 con 0x7ff66009fa10 2023-10-03T10:16:32.955 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:32.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.957+0000 7ff6658b1700 1 -- 172.21.15.3:0/2663772280 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff66009fa10 msgr2=0x7ff66006c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:32.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.957+0000 7ff6658b1700 1 --2- 172.21.15.3:0/2663772280 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff66009fa10 0x7ff66006c300 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff6600a0150 tx=0x7ff658004ad0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.957+0000 7ff6658b1700 1 -- 172.21.15.3:0/2663772280 shutdown_connections 2023-10-03T10:16:32.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.957+0000 7ff6658b1700 1 --2- 172.21.15.3:0/2663772280 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff66009fa10 0x7ff66006c300 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.957+0000 7ff6658b1700 1 --2- 172.21.15.3:0/2663772280 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff66009e7e0 0x7ff66006b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.957+0000 7ff6658b1700 1 --2- 172.21.15.3:0/2663772280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff66009f2c0 0x7ff66006bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:32.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.957+0000 7ff6658b1700 1 -- 172.21.15.3:0/2663772280 >> 172.21.15.3:0/2663772280 conn(0x7ff6600a4300 msgr2=0x7ff66009c790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:32.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.957+0000 7ff6658b1700 1 -- 172.21.15.3:0/2663772280 shutdown_connections 2023-10-03T10:16:32.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:32.957+0000 7ff6658b1700 1 -- 172.21.15.3:0/2663772280 wait complete. 2023-10-03T10:16:32.971 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:33.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:33.003+0000 7feb62228700 1 -- 172.21.15.3:0/981299308 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7feb5c072820 con 0x7feb5c09ffb0 2023-10-03T10:16:33.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:33.003+0000 7feb43fff700 1 -- 172.21.15.3:0/981299308 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feb50010b70 con 0x7feb5c09ffb0 2023-10-03T10:16:33.003 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:33.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:33.005+0000 7feb62228700 1 -- 172.21.15.3:0/981299308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb5c09ffb0 msgr2=0x7feb5c120e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:33.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:33.005+0000 7feb62228700 1 --2- 172.21.15.3:0/981299308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb5c09ffb0 0x7feb5c120e10 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7feb50009fd0 tx=0x7feb50004750 comp rx=0 tx=0).stop 2023-10-03T10:16:33.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:33.005+0000 7feb62228700 1 -- 172.21.15.3:0/981299308 shutdown_connections 2023-10-03T10:16:33.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:33.005+0000 7feb62228700 1 --2- 172.21.15.3:0/981299308 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feb5c138e30 0x7feb5c121350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:33.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:33.005+0000 7feb62228700 1 --2- 172.21.15.3:0/981299308 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feb5c09e850 0x7feb5c1208d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:33.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:33.005+0000 7feb62228700 1 --2- 172.21.15.3:0/981299308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb5c09ffb0 0x7feb5c120e10 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:33.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:33.005+0000 7feb62228700 1 -- 172.21.15.3:0/981299308 >> 172.21.15.3:0/981299308 conn(0x7feb5c0a3f70 msgr2=0x7feb5c0a1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:33.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:33.006+0000 7feb62228700 1 -- 172.21.15.3:0/981299308 shutdown_connections 2023-10-03T10:16:33.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:33.006+0000 7feb62228700 1 -- 172.21.15.3:0/981299308 wait complete. 2023-10-03T10:16:33.019 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:33.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:33.070+0000 7f08d889f700 1 -- 172.21.15.3:0/2316705871 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f08b8005590 con 0x7f08d409d630 2023-10-03T10:16:33.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:33.070+0000 7f08c2ffd700 1 -- 172.21.15.3:0/2316705871 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f08c401be10 con 0x7f08d409d630 2023-10-03T10:16:33.070 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:33.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:33.072+0000 7f08d889f700 1 -- 172.21.15.3:0/2316705871 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08d409d630 msgr2=0x7f08d4132940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:33.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:33.072+0000 7f08d889f700 1 --2- 172.21.15.3:0/2316705871 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08d409d630 0x7f08d4132940 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f08c400e9b0 tx=0x7f08c400ecc0 comp rx=0 tx=0).stop 2023-10-03T10:16:33.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:33.072+0000 7f08d889f700 1 -- 172.21.15.3:0/2316705871 shutdown_connections 2023-10-03T10:16:33.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:33.072+0000 7f08d889f700 1 --2- 172.21.15.3:0/2316705871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d4133930 0x7f08d41c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:33.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:33.072+0000 7f08d889f700 1 --2- 172.21.15.3:0/2316705871 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08d409d630 0x7f08d4132940 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:33.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:33.072+0000 7f08d889f700 1 --2- 172.21.15.3:0/2316705871 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08d40a0a30 0x7f08d4132e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:33.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:33.072+0000 7f08d889f700 1 -- 172.21.15.3:0/2316705871 >> 172.21.15.3:0/2316705871 conn(0x7f08d40a3050 msgr2=0x7f08d41bbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:33.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:33.073+0000 7f08d889f700 1 -- 172.21.15.3:0/2316705871 shutdown_connections 2023-10-03T10:16:33.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:33.073+0000 7f08d889f700 1 -- 172.21.15.3:0/2316705871 wait complete. 2023-10-03T10:16:33.086 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:33.858 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:33.972 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:34.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.005+0000 7f9773377700 1 Processor -- start 2023-10-03T10:16:34.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.006+0000 7f9773377700 1 -- start start 2023-10-03T10:16:34.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.006+0000 7f9773377700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f976c09d6f0 0x7f976c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:34.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.006+0000 7f9773377700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f976c09e000 0x7f976c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:34.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.006+0000 7f9773377700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f976c09e410 0x7f976c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:34.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.006+0000 7f9773377700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f976c09f520 con 0x7f976c09d6f0 2023-10-03T10:16:34.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.006+0000 7f9773377700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f976c09f130 con 0x7f976c09e000 2023-10-03T10:16:34.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.006+0000 7f9773377700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f976c09e930 con 0x7f976c09e410 2023-10-03T10:16:34.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.007+0000 7f9771914700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f976c09e410 0x7f976c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:34.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.007+0000 7f9770912700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f976c09e000 0x7f976c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:34.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.007+0000 7f9771914700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f976c09e410 0x7f976c130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50370/0 (socket says 172.21.15.3:50370) 2023-10-03T10:16:34.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.007+0000 7f9770912700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f976c09e000 0x7f976c130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58570/0 (socket says 172.21.15.3:58570) 2023-10-03T10:16:34.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.007+0000 7f9771914700 1 -- 172.21.15.3:0/2995310856 learned_addr learned my addr 172.21.15.3:0/2995310856 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:34.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.007+0000 7f9771113700 1 --2- 172.21.15.3:0/2995310856 >> v2:172.21.15.3:3300/0 conn(0x7f976c09d6f0 0x7f976c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:34.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.007+0000 7f9770912700 1 -- 172.21.15.3:0/2995310856 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f976c09e410 msgr2=0x7f976c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:34.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.007+0000 7f9770912700 1 --2- 172.21.15.3:0/2995310856 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f976c09e410 0x7f976c130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.007+0000 7f9770912700 1 -- 172.21.15.3:0/2995310856 >> v2:172.21.15.3:3300/0 conn(0x7f976c09d6f0 msgr2=0x7f976c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:34.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.007+0000 7f9770912700 1 --2- 172.21.15.3:0/2995310856 >> v2:172.21.15.3:3300/0 conn(0x7f976c09d6f0 0x7f976c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.007+0000 7f9770912700 1 -- 172.21.15.3:0/2995310856 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976c09b2d0 con 0x7f976c09e000 2023-10-03T10:16:34.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.007+0000 7f9771914700 1 --2- 172.21.15.3:0/2995310856 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f976c09e410 0x7f976c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:34.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.007+0000 7f9771113700 1 --2- 172.21.15.3:0/2995310856 >> v2:172.21.15.3:3300/0 conn(0x7f976c09d6f0 0x7f976c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:34.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.007+0000 7f9770912700 1 --2- 172.21.15.3:0/2995310856 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f976c09e000 0x7f976c130660 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f975c009840 tx=0x7f975c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac344148848781e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:34.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.007+0000 7f9763fff700 1 -- 172.21.15.3:0/2995310856 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c011830 con 0x7f976c09e000 2023-10-03T10:16:34.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.007+0000 7f9763fff700 1 -- 172.21.15.3:0/2995310856 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975c011990 con 0x7f976c09e000 2023-10-03T10:16:34.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.007+0000 7f9763fff700 1 -- 172.21.15.3:0/2995310856 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c011d20 con 0x7f976c09e000 2023-10-03T10:16:34.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.008+0000 7f9773377700 1 -- 172.21.15.3:0/2995310856 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f976c09e000 msgr2=0x7f976c130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:34.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.008+0000 7f9773377700 1 --2- 172.21.15.3:0/2995310856 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f976c09e000 0x7f976c130660 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f975c009840 tx=0x7f975c009b50 comp rx=0 tx=0).stop 2023-10-03T10:16:34.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.008+0000 7f9773377700 1 -- 172.21.15.3:0/2995310856 shutdown_connections 2023-10-03T10:16:34.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.008+0000 7f9773377700 1 --2- 172.21.15.3:0/2995310856 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f976c09e410 0x7f976c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.009+0000 7f9773377700 1 --2- 172.21.15.3:0/2995310856 >> v2:172.21.15.3:3300/0 conn(0x7f976c09d6f0 0x7f976c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.009+0000 7f9773377700 1 --2- 172.21.15.3:0/2995310856 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f976c09e000 0x7f976c130660 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.009+0000 7f9773377700 1 -- 172.21.15.3:0/2995310856 >> 172.21.15.3:0/2995310856 conn(0x7f976c066f50 msgr2=0x7f976c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:34.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.009+0000 7f9773377700 1 -- 172.21.15.3:0/2995310856 shutdown_connections 2023-10-03T10:16:34.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.009+0000 7f9773377700 1 -- 172.21.15.3:0/2995310856 wait complete. 2023-10-03T10:16:34.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.010+0000 7f9773377700 1 Processor -- start 2023-10-03T10:16:34.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.010+0000 7f9773377700 1 -- start start 2023-10-03T10:16:34.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.010+0000 7f9773377700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f976c09d6f0 0x7f976c06e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:34.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.010+0000 7f9773377700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f976c09e000 0x7f976c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:34.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.010+0000 7f9773377700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f976c09e410 0x7f976c06f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:34.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.010+0000 7f9773377700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976c06fd40 con 0x7f976c09d6f0 2023-10-03T10:16:34.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.010+0000 7f9773377700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f976c125a80 con 0x7f976c09e410 2023-10-03T10:16:34.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.010+0000 7f9773377700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f976c09c700 con 0x7f976c09e000 2023-10-03T10:16:34.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.010+0000 7f9771113700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f976c09d6f0 0x7f976c06e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:34.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.010+0000 7f9770912700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f976c09e000 0x7f976c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:34.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.010+0000 7f9771113700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f976c09d6f0 0x7f976c06e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48828/0 (socket says 172.21.15.3:48828) 2023-10-03T10:16:34.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.010+0000 7f9771113700 1 -- 172.21.15.3:0/4055584815 learned_addr learned my addr 172.21.15.3:0/4055584815 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:34.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.010+0000 7f9770912700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f976c09e000 0x7f976c06edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50374/0 (socket says 172.21.15.3:50374) 2023-10-03T10:16:34.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.010+0000 7f9771914700 1 --2- 172.21.15.3:0/4055584815 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f976c09e410 0x7f976c06f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:34.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.011+0000 7f9771113700 1 -- 172.21.15.3:0/4055584815 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f976c09e000 msgr2=0x7f976c06edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:34.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.011+0000 7f9771113700 1 --2- 172.21.15.3:0/4055584815 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f976c09e000 0x7f976c06edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.011+0000 7f9771113700 1 -- 172.21.15.3:0/4055584815 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f976c09e410 msgr2=0x7f976c06f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:34.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.011+0000 7f9771113700 1 --2- 172.21.15.3:0/4055584815 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f976c09e410 0x7f976c06f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.011+0000 7f9771113700 1 -- 172.21.15.3:0/4055584815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97680094a0 con 0x7f976c09d6f0 2023-10-03T10:16:34.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.011+0000 7f9771113700 1 --2- 172.21.15.3:0/4055584815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f976c09d6f0 0x7f976c06e870 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f976800ebe0 tx=0x7f976800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:34.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.011+0000 7f9761ffb700 1 -- 172.21.15.3:0/4055584815 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9768009780 con 0x7f976c09d6f0 2023-10-03T10:16:34.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.011+0000 7f9761ffb700 1 -- 172.21.15.3:0/4055584815 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976800c850 con 0x7f976c09d6f0 2023-10-03T10:16:34.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.011+0000 7f9773377700 1 -- 172.21.15.3:0/4055584815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9764009b30 con 0x7f976c09d6f0 2023-10-03T10:16:34.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.011+0000 7f9761ffb700 1 -- 172.21.15.3:0/4055584815 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976800cbe0 con 0x7f976c09d6f0 2023-10-03T10:16:34.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.012+0000 7f9773377700 1 -- 172.21.15.3:0/4055584815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f975c0094a0 con 0x7f976c09d6f0 2023-10-03T10:16:34.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.012+0000 7f9761ffb700 1 -- 172.21.15.3:0/4055584815 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f976800cd40 con 0x7f976c09d6f0 2023-10-03T10:16:34.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.012+0000 7f9761ffb700 1 -- 172.21.15.3:0/4055584815 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f976803fab0 con 0x7f976c09d6f0 2023-10-03T10:16:34.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.013+0000 7f9773377700 1 -- 172.21.15.3:0/4055584815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f976c074ea0 con 0x7f976c09d6f0 2023-10-03T10:16:34.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.018+0000 7f9761ffb700 1 -- 172.21.15.3:0/4055584815 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f976801b3b0 con 0x7f976c09d6f0 2023-10-03T10:16:34.020 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:34.087 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:34.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.126+0000 7f3123d44700 1 Processor -- start 2023-10-03T10:16:34.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.127+0000 7f3123d44700 1 -- start start 2023-10-03T10:16:34.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.127+0000 7f3123d44700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f311c133420 0x7f311c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:34.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.128+0000 7f3123d44700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f311c134260 0x7f311c1306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:34.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.128+0000 7f3123d44700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f311c06b990 con 0x7f311c133e30 2023-10-03T10:16:34.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.128+0000 7f3123d44700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311c0a0590 con 0x7f311c134260 2023-10-03T10:16:34.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.128+0000 7f3123d44700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f311c09fb80 con 0x7f311c133420 2023-10-03T10:16:34.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.128+0000 7f31212df700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f311c133e30 0x7f311c12ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35786/0 (socket says 172.21.15.3:35786) 2023-10-03T10:16:34.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.128+0000 7f31212df700 1 -- 172.21.15.3:0/2823704967 learned_addr learned my addr 172.21.15.3:0/2823704967 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:34.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.128+0000 7f3121ae0700 1 --2- 172.21.15.3:0/2823704967 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f311c133420 0x7f311c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:34.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.128+0000 7f31222e1700 1 --2- 172.21.15.3:0/2823704967 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f311c134260 0x7f311c1306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:34.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.128+0000 7f3121ae0700 1 -- 172.21.15.3:0/2823704967 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f311c134260 msgr2=0x7f311c1306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:34.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.128+0000 7f3121ae0700 1 --2- 172.21.15.3:0/2823704967 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f311c134260 0x7f311c1306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.128+0000 7f3121ae0700 1 -- 172.21.15.3:0/2823704967 >> v1:172.21.15.3:6789/0 conn(0x7f311c133e30 legacy=0x7f311c12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:34.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.128+0000 7f3121ae0700 1 -- 172.21.15.3:0/2823704967 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311c0a0030 con 0x7f311c133420 2023-10-03T10:16:34.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.128+0000 7f3120ade700 1 -- 172.21.15.3:0/2823704967 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1514451633 0 0) 0x7f311c06b990 con 0x7f311c133e30 2023-10-03T10:16:34.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.128+0000 7f31222e1700 1 --2- 172.21.15.3:0/2823704967 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f311c134260 0x7f311c1306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:34.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.129+0000 7f3121ae0700 1 --2- 172.21.15.3:0/2823704967 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f311c133420 0x7f311c1337f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3118009870 tx=0x7f3118009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db16c18af9a18466 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:34.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.129+0000 7f3120ade700 1 -- 172.21.15.3:0/2823704967 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118011830 con 0x7f311c133420 2023-10-03T10:16:34.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.129+0000 7f3120ade700 1 -- 172.21.15.3:0/2823704967 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3118011990 con 0x7f311c133420 2023-10-03T10:16:34.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.129+0000 7f3120ade700 1 -- 172.21.15.3:0/2823704967 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118011d20 con 0x7f311c133420 2023-10-03T10:16:34.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.130+0000 7f3123d44700 1 -- 172.21.15.3:0/2823704967 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f311c133420 msgr2=0x7f311c1337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:34.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.130+0000 7f3123d44700 1 --2- 172.21.15.3:0/2823704967 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f311c133420 0x7f311c1337f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3118009870 tx=0x7f3118009b80 comp rx=0 tx=0).stop 2023-10-03T10:16:34.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.130+0000 7f3123d44700 1 -- 172.21.15.3:0/2823704967 shutdown_connections 2023-10-03T10:16:34.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.130+0000 7f3123d44700 1 --2- 172.21.15.3:0/2823704967 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f311c134260 0x7f311c1306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.130+0000 7f3123d44700 1 --2- 172.21.15.3:0/2823704967 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f311c133420 0x7f311c1337f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.130+0000 7f3123d44700 1 -- 172.21.15.3:0/2823704967 >> 172.21.15.3:0/2823704967 conn(0x7f311c0a4210 msgr2=0x7f311c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:34.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.130+0000 7f3123d44700 1 -- 172.21.15.3:0/2823704967 shutdown_connections 2023-10-03T10:16:34.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.130+0000 7f3123d44700 1 -- 172.21.15.3:0/2823704967 wait complete. 2023-10-03T10:16:34.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.132+0000 7f3123d44700 1 Processor -- start 2023-10-03T10:16:34.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.132+0000 7f3123d44700 1 -- start start 2023-10-03T10:16:34.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.132+0000 7f3123d44700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f311c133420 0x7f311c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:34.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.132+0000 7f3123d44700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f311c133e30 0x7f311c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:34.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.132+0000 7f3123d44700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f311c134260 0x7f311c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:34.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.132+0000 7f3123d44700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311c11f990 con 0x7f311c133e30 2023-10-03T10:16:34.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.132+0000 7f3123d44700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311c09d8e0 con 0x7f311c133420 2023-10-03T10:16:34.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.132+0000 7f3123d44700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f311c11fbd0 con 0x7f311c134260 2023-10-03T10:16:34.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.133+0000 7f3121ae0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f311c133420 0x7f311c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:34.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.133+0000 7f31222e1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f311c134260 0x7f311c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:34.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.133+0000 7f3121ae0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f311c133420 0x7f311c09c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58604/0 (socket says 172.21.15.3:58604) 2023-10-03T10:16:34.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.133+0000 7f3121ae0700 1 -- 172.21.15.3:0/133341823 learned_addr learned my addr 172.21.15.3:0/133341823 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:34.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.133+0000 7f31212df700 1 --2- 172.21.15.3:0/133341823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f311c133e30 0x7f311c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:34.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.133+0000 7f3121ae0700 1 -- 172.21.15.3:0/133341823 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f311c134260 msgr2=0x7f311c09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:34.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.133+0000 7f3121ae0700 1 --2- 172.21.15.3:0/133341823 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f311c134260 0x7f311c09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.133+0000 7f3121ae0700 1 -- 172.21.15.3:0/133341823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f311c133e30 msgr2=0x7f311c09c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:34.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.133+0000 7f3121ae0700 1 --2- 172.21.15.3:0/133341823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f311c133e30 0x7f311c09c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.133+0000 7f3121ae0700 1 -- 172.21.15.3:0/133341823 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3114009a70 con 0x7f311c133420 2023-10-03T10:16:34.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.133+0000 7f31212df700 1 --2- 172.21.15.3:0/133341823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f311c133e30 0x7f311c09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:34.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.133+0000 7f31222e1700 1 --2- 172.21.15.3:0/133341823 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f311c134260 0x7f311c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:34.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.133+0000 7f3121ae0700 1 --2- 172.21.15.3:0/133341823 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f311c133420 0x7f311c09c480 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3118000c00 tx=0x7f3118004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:34.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.134+0000 7f31127fc700 1 -- 172.21.15.3:0/133341823 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118004400 con 0x7f311c133420 2023-10-03T10:16:34.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.134+0000 7f31127fc700 1 -- 172.21.15.3:0/133341823 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311801f070 con 0x7f311c133420 2023-10-03T10:16:34.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.134+0000 7f3123d44700 1 -- 172.21.15.3:0/133341823 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31180094a0 con 0x7f311c133420 2023-10-03T10:16:34.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.134+0000 7f31127fc700 1 -- 172.21.15.3:0/133341823 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31180038c0 con 0x7f311c133420 2023-10-03T10:16:34.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.134+0000 7f3123d44700 1 -- 172.21.15.3:0/133341823 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f311c11ee60 con 0x7f311c133420 2023-10-03T10:16:34.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.135+0000 7f31127fc700 1 -- 172.21.15.3:0/133341823 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3118021030 con 0x7f311c133420 2023-10-03T10:16:34.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.135+0000 7f31127fc700 1 -- 172.21.15.3:0/133341823 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3118040970 con 0x7f311c133420 2023-10-03T10:16:34.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.135+0000 7f3123d44700 1 -- 172.21.15.3:0/133341823 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f311c1315b0 con 0x7f311c133420 2023-10-03T10:16:34.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.140+0000 7f31127fc700 1 -- 172.21.15.3:0/133341823 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f31180239c0 con 0x7f311c133420 2023-10-03T10:16:34.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.171+0000 7f5853973700 1 Processor -- start 2023-10-03T10:16:34.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.172+0000 7f5853973700 1 -- start start 2023-10-03T10:16:34.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.172+0000 7f5853973700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f584c09e930 0x7f584c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:34.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.172+0000 7f5853973700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f584c09f240 0x7f584c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:34.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.173+0000 7f5853973700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f584c070940 con 0x7f584c09f650 2023-10-03T10:16:34.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.173+0000 7f5853973700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584c06fd60 con 0x7f584c09e930 2023-10-03T10:16:34.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.173+0000 7f5853973700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f584c06fa80 con 0x7f584c09f240 2023-10-03T10:16:34.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.173+0000 7f5851f10700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f584c09f650 0x7f584c131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35788/0 (socket says 172.21.15.3:35788) 2023-10-03T10:16:34.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.173+0000 7f5851f10700 1 -- 172.21.15.3:0/436978403 learned_addr learned my addr 172.21.15.3:0/436978403 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:34.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.173+0000 7f585170f700 1 --2- 172.21.15.3:0/436978403 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f584c09e930 0x7f584c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:34.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.173+0000 7f5850f0e700 1 --2- 172.21.15.3:0/436978403 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f584c09f240 0x7f584c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:34.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.173+0000 7f5850f0e700 1 -- 172.21.15.3:0/436978403 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f584c09e930 msgr2=0x7f584c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:34.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.173+0000 7f5843fff700 1 -- 172.21.15.3:0/436978403 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 365911461 0 0) 0x7f584c070940 con 0x7f584c09f650 2023-10-03T10:16:34.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.173+0000 7f5850f0e700 1 --2- 172.21.15.3:0/436978403 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f584c09e930 0x7f584c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.173+0000 7f5850f0e700 1 -- 172.21.15.3:0/436978403 >> v1:172.21.15.3:6789/0 conn(0x7f584c09f650 legacy=0x7f584c131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:34.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.173+0000 7f5850f0e700 1 -- 172.21.15.3:0/436978403 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584c09fb60 con 0x7f584c09f240 2023-10-03T10:16:34.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.173+0000 7f5850f0e700 1 --2- 172.21.15.3:0/436978403 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f584c09f240 0x7f584c1317f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f583c009870 tx=0x7f583c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e50df9aedcd23507 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:34.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.174+0000 7f5843fff700 1 -- 172.21.15.3:0/436978403 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583c011830 con 0x7f584c09f240 2023-10-03T10:16:34.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.174+0000 7f5843fff700 1 -- 172.21.15.3:0/436978403 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583c011990 con 0x7f584c09f240 2023-10-03T10:16:34.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.174+0000 7f5843fff700 1 -- 172.21.15.3:0/436978403 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583c011d20 con 0x7f584c09f240 2023-10-03T10:16:34.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.175+0000 7f5853973700 1 -- 172.21.15.3:0/436978403 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f584c09f240 msgr2=0x7f584c1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:34.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.175+0000 7f5853973700 1 --2- 172.21.15.3:0/436978403 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f584c09f240 0x7f584c1317f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f583c009870 tx=0x7f583c009b80 comp rx=0 tx=0).stop 2023-10-03T10:16:34.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.175+0000 7f5853973700 1 -- 172.21.15.3:0/436978403 shutdown_connections 2023-10-03T10:16:34.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.175+0000 7f5853973700 1 --2- 172.21.15.3:0/436978403 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f584c09e930 0x7f584c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.175+0000 7f5853973700 1 --2- 172.21.15.3:0/436978403 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f584c09f240 0x7f584c1317f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.175+0000 7f5853973700 1 -- 172.21.15.3:0/436978403 >> 172.21.15.3:0/436978403 conn(0x7f584c0a4210 msgr2=0x7f584c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:34.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.175+0000 7f5853973700 1 -- 172.21.15.3:0/436978403 shutdown_connections 2023-10-03T10:16:34.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.175+0000 7f5853973700 1 -- 172.21.15.3:0/436978403 wait complete. 2023-10-03T10:16:34.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.176+0000 7f5853973700 1 Processor -- start 2023-10-03T10:16:34.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.176+0000 7f5853973700 1 -- start start 2023-10-03T10:16:34.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.176+0000 7f5853973700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f584c09e930 0x7f584c06b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:34.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.176+0000 7f5853973700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f584c09f240 0x7f584c06bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:34.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.176+0000 7f5853973700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f584c09f650 0x7f584c06c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:34.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.176+0000 7f5853973700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584c06d3e0 con 0x7f584c09f650 2023-10-03T10:16:34.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.176+0000 7f5853973700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584c0a3360 con 0x7f584c09f240 2023-10-03T10:16:34.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.176+0000 7f5853973700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f584c1a96a0 con 0x7f584c09e930 2023-10-03T10:16:34.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.177+0000 7f585170f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f584c09e930 0x7f584c06b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:34.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.177+0000 7f585170f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f584c09e930 0x7f584c06b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50398/0 (socket says 172.21.15.3:50398) 2023-10-03T10:16:34.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.177+0000 7f585170f700 1 -- 172.21.15.3:0/3081709591 learned_addr learned my addr 172.21.15.3:0/3081709591 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:34.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.177+0000 7f5851f10700 1 --2- 172.21.15.3:0/3081709591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f584c09f650 0x7f584c06c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:34.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.177+0000 7f5850f0e700 1 --2- 172.21.15.3:0/3081709591 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f584c09f240 0x7f584c06bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:34.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.177+0000 7f585170f700 1 -- 172.21.15.3:0/3081709591 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f584c09f240 msgr2=0x7f584c06bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:34.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.177+0000 7f585170f700 1 --2- 172.21.15.3:0/3081709591 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f584c09f240 0x7f584c06bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.177+0000 7f585170f700 1 -- 172.21.15.3:0/3081709591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f584c09f650 msgr2=0x7f584c06c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:34.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.177+0000 7f585170f700 1 --2- 172.21.15.3:0/3081709591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f584c09f650 0x7f584c06c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.177+0000 7f585170f700 1 -- 172.21.15.3:0/3081709591 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f583c0094a0 con 0x7f584c09e930 2023-10-03T10:16:34.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.177+0000 7f5851f10700 1 --2- 172.21.15.3:0/3081709591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f584c09f650 0x7f584c06c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:34.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.177+0000 7f5850f0e700 1 --2- 172.21.15.3:0/3081709591 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f584c09f240 0x7f584c06bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:34.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.177+0000 7f585170f700 1 --2- 172.21.15.3:0/3081709591 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f584c09e930 0x7f584c06b880 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f584800e9b0 tx=0x7f584800ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:34.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.178+0000 7f5841ffb700 1 -- 172.21.15.3:0/3081709591 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5848009710 con 0x7f584c09e930 2023-10-03T10:16:34.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.178+0000 7f5841ffb700 1 -- 172.21.15.3:0/3081709591 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584800ee50 con 0x7f584c09e930 2023-10-03T10:16:34.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.178+0000 7f5841ffb700 1 -- 172.21.15.3:0/3081709591 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584800ca50 con 0x7f584c09e930 2023-10-03T10:16:34.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.178+0000 7f5853973700 1 -- 172.21.15.3:0/3081709591 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f584c0a0190 con 0x7f584c09e930 2023-10-03T10:16:34.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.178+0000 7f5853973700 1 -- 172.21.15.3:0/3081709591 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f584c071d20 con 0x7f584c09e930 2023-10-03T10:16:34.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.179+0000 7f5841ffb700 1 -- 172.21.15.3:0/3081709591 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f584800cbb0 con 0x7f584c09e930 2023-10-03T10:16:34.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.179+0000 7f5841ffb700 1 -- 172.21.15.3:0/3081709591 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f584803fad0 con 0x7f584c09e930 2023-10-03T10:16:34.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.179+0000 7f5853973700 1 -- 172.21.15.3:0/3081709591 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f584c1a0b30 con 0x7f584c09e930 2023-10-03T10:16:34.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.184+0000 7f5841ffb700 1 -- 172.21.15.3:0/3081709591 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f584801b3b0 con 0x7f584c09e930 2023-10-03T10:16:34.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.238+0000 7f9773377700 1 -- 172.21.15.3:0/4055584815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f976c06c360 con 0x7f976c09d6f0 2023-10-03T10:16:34.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.239+0000 7f9761ffb700 1 -- 172.21.15.3:0/4055584815 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9768021030 con 0x7f976c09d6f0 2023-10-03T10:16:34.239 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:34.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.241+0000 7f9773377700 1 -- 172.21.15.3:0/4055584815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f976c09d6f0 msgr2=0x7f976c06e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:34.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.241+0000 7f9773377700 1 --2- 172.21.15.3:0/4055584815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f976c09d6f0 0x7f976c06e870 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f976800ebe0 tx=0x7f976800eef0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.241+0000 7f9773377700 1 -- 172.21.15.3:0/4055584815 shutdown_connections 2023-10-03T10:16:34.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.241+0000 7f9773377700 1 --2- 172.21.15.3:0/4055584815 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f976c09e410 0x7f976c06f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.241+0000 7f9773377700 1 --2- 172.21.15.3:0/4055584815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f976c09d6f0 0x7f976c06e870 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.241+0000 7f9773377700 1 --2- 172.21.15.3:0/4055584815 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f976c09e000 0x7f976c06edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.241+0000 7f9773377700 1 -- 172.21.15.3:0/4055584815 >> 172.21.15.3:0/4055584815 conn(0x7f976c066f50 msgr2=0x7f976c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:34.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.241+0000 7f9773377700 1 -- 172.21.15.3:0/4055584815 shutdown_connections 2023-10-03T10:16:34.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.241+0000 7f9773377700 1 -- 172.21.15.3:0/4055584815 wait complete. 2023-10-03T10:16:34.257 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:34.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.282+0000 7f66ca144700 1 Processor -- start 2023-10-03T10:16:34.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.282+0000 7f66ca144700 1 -- start start 2023-10-03T10:16:34.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.282+0000 7f66ca144700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66c409f410 0x7f66c4136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:34.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.283+0000 7f66ca144700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f66c4136fe0 0x7f66c41393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:34.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.283+0000 7f66ca144700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c4070940 con 0x7f66c409e930 2023-10-03T10:16:34.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.283+0000 7f66ca144700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66c406fd60 con 0x7f66c4136fe0 2023-10-03T10:16:34.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.283+0000 7f66ca144700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c406fa80 con 0x7f66c409f410 2023-10-03T10:16:34.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.283+0000 7f66c37fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f66c409e930 0x7f66c409ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35804/0 (socket says 172.21.15.3:35804) 2023-10-03T10:16:34.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.283+0000 7f66c37fe700 1 -- 172.21.15.3:0/3763254386 learned_addr learned my addr 172.21.15.3:0/3763254386 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:34.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.283+0000 7f66c3fff700 1 --2- 172.21.15.3:0/3763254386 >> v2:172.21.15.3:3300/0 conn(0x7f66c4136fe0 0x7f66c41393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:34.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.283+0000 7f66c2ffd700 1 --2- 172.21.15.3:0/3763254386 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66c409f410 0x7f66c4136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:34.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.283+0000 7f66c3fff700 1 -- 172.21.15.3:0/3763254386 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66c409f410 msgr2=0x7f66c4136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:34.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.283+0000 7f66c3fff700 1 --2- 172.21.15.3:0/3763254386 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66c409f410 0x7f66c4136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.283+0000 7f66c3fff700 1 -- 172.21.15.3:0/3763254386 >> v1:172.21.15.3:6789/0 conn(0x7f66c409e930 legacy=0x7f66c409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:34.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.283+0000 7f66c3fff700 1 -- 172.21.15.3:0/3763254386 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c409fb60 con 0x7f66c4136fe0 2023-10-03T10:16:34.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.283+0000 7f66c27fc700 1 -- 172.21.15.3:0/3763254386 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2449692700 0 0) 0x7f66c4070940 con 0x7f66c409e930 2023-10-03T10:16:34.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.283+0000 7f66c3fff700 1 --2- 172.21.15.3:0/3763254386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c4136fe0 0x7f66c41393c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f66bc009d90 tx=0x7f66bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=768cdb83d90dc201 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:34.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.284+0000 7f66c27fc700 1 -- 172.21.15.3:0/3763254386 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc00e070 con 0x7f66c4136fe0 2023-10-03T10:16:34.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.284+0000 7f66c27fc700 1 -- 172.21.15.3:0/3763254386 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66bc00bb20 con 0x7f66c4136fe0 2023-10-03T10:16:34.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.284+0000 7f66c27fc700 1 -- 172.21.15.3:0/3763254386 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc012430 con 0x7f66c4136fe0 2023-10-03T10:16:34.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.284+0000 7f66ca144700 1 -- 172.21.15.3:0/3763254386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c4136fe0 msgr2=0x7f66c41393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:34.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.284+0000 7f66ca144700 1 --2- 172.21.15.3:0/3763254386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c4136fe0 0x7f66c41393c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f66bc009d90 tx=0x7f66bc00b330 comp rx=0 tx=0).stop 2023-10-03T10:16:34.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.285+0000 7f66ca144700 1 -- 172.21.15.3:0/3763254386 shutdown_connections 2023-10-03T10:16:34.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.285+0000 7f66ca144700 1 --2- 172.21.15.3:0/3763254386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c4136fe0 0x7f66c41393c0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.285+0000 7f66ca144700 1 --2- 172.21.15.3:0/3763254386 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66c409f410 0x7f66c4136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.285+0000 7f66ca144700 1 -- 172.21.15.3:0/3763254386 >> 172.21.15.3:0/3763254386 conn(0x7f66c40a4210 msgr2=0x7f66c40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:34.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.285+0000 7f66ca144700 1 -- 172.21.15.3:0/3763254386 shutdown_connections 2023-10-03T10:16:34.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.285+0000 7f66ca144700 1 -- 172.21.15.3:0/3763254386 wait complete. 2023-10-03T10:16:34.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.286+0000 7f66ca144700 1 Processor -- start 2023-10-03T10:16:34.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.286+0000 7f66ca144700 1 -- start start 2023-10-03T10:16:34.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.286+0000 7f66ca144700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c409e930 0x7f66c411c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:34.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.286+0000 7f66ca144700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66c409f410 0x7f66c411cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:34.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.286+0000 7f66ca144700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66c4136fe0 0x7f66c411d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:34.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.286+0000 7f66ca144700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c4071d20 con 0x7f66c409e930 2023-10-03T10:16:34.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.286+0000 7f66ca144700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c411fe00 con 0x7f66c4136fe0 2023-10-03T10:16:34.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.286+0000 7f66ca144700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66c411e2e0 con 0x7f66c409f410 2023-10-03T10:16:34.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.287+0000 7f66c37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c409e930 0x7f66c411c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:34.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.287+0000 7f66c3fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66c4136fe0 0x7f66c411d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:34.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.287+0000 7f66c37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c409e930 0x7f66c411c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48872/0 (socket says 172.21.15.3:48872) 2023-10-03T10:16:34.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.287+0000 7f66c3fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66c4136fe0 0x7f66c411d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58642/0 (socket says 172.21.15.3:58642) 2023-10-03T10:16:34.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.287+0000 7f66c37fe700 1 -- 172.21.15.3:0/3368512730 learned_addr learned my addr 172.21.15.3:0/3368512730 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:34.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.287+0000 7f66c2ffd700 1 --2- 172.21.15.3:0/3368512730 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66c409f410 0x7f66c411cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:34.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.287+0000 7f66c37fe700 1 -- 172.21.15.3:0/3368512730 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66c409f410 msgr2=0x7f66c411cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:34.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.287+0000 7f66c37fe700 1 --2- 172.21.15.3:0/3368512730 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66c409f410 0x7f66c411cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.287+0000 7f66c37fe700 1 -- 172.21.15.3:0/3368512730 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66c4136fe0 msgr2=0x7f66c411d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:34.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.287+0000 7f66c37fe700 1 --2- 172.21.15.3:0/3368512730 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66c4136fe0 0x7f66c411d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.287+0000 7f66c37fe700 1 -- 172.21.15.3:0/3368512730 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66bc009a70 con 0x7f66c409e930 2023-10-03T10:16:34.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.287+0000 7f66c3fff700 1 --2- 172.21.15.3:0/3368512730 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66c4136fe0 0x7f66c411d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:34.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.287+0000 7f66c37fe700 1 --2- 172.21.15.3:0/3368512730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c409e930 0x7f66c411c5c0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f66b400c550 tx=0x7f66b400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:34.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.287+0000 7f66abfff700 1 -- 172.21.15.3:0/3368512730 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b4004c80 con 0x7f66c409e930 2023-10-03T10:16:34.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.287+0000 7f66abfff700 1 -- 172.21.15.3:0/3368512730 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b4004e00 con 0x7f66c409e930 2023-10-03T10:16:34.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.287+0000 7f66abfff700 1 -- 172.21.15.3:0/3368512730 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b4007310 con 0x7f66c409e930 2023-10-03T10:16:34.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.288+0000 7f66ca144700 1 -- 172.21.15.3:0/3368512730 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c409de30 con 0x7f66c409e930 2023-10-03T10:16:34.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.288+0000 7f66ca144700 1 -- 172.21.15.3:0/3368512730 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c4120870 con 0x7f66c409e930 2023-10-03T10:16:34.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.289+0000 7f66abfff700 1 -- 172.21.15.3:0/3368512730 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f66b4007470 con 0x7f66c409e930 2023-10-03T10:16:34.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.289+0000 7f66abfff700 1 -- 172.21.15.3:0/3368512730 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f66b403f2b0 con 0x7f66c409e930 2023-10-03T10:16:34.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.289+0000 7f66ca144700 1 -- 172.21.15.3:0/3368512730 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c411fa70 con 0x7f66c409e930 2023-10-03T10:16:34.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.298+0000 7f66abfff700 1 -- 172.21.15.3:0/3368512730 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f66b40123b0 con 0x7f66c409e930 2023-10-03T10:16:34.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.400+0000 7f5853973700 1 -- 172.21.15.3:0/3081709591 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f584c1a9aa0 con 0x7f584c09e930 2023-10-03T10:16:34.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.401+0000 7f5841ffb700 1 -- 172.21.15.3:0/3081709591 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5848017020 con 0x7f584c09e930 2023-10-03T10:16:34.401 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:34.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.403+0000 7f5853973700 1 -- 172.21.15.3:0/3081709591 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f584c09e930 msgr2=0x7f584c06b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:34.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.403+0000 7f5853973700 1 --2- 172.21.15.3:0/3081709591 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f584c09e930 0x7f584c06b880 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f584800e9b0 tx=0x7f584800ecc0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.403+0000 7f5853973700 1 -- 172.21.15.3:0/3081709591 shutdown_connections 2023-10-03T10:16:34.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.403+0000 7f5853973700 1 --2- 172.21.15.3:0/3081709591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f584c09f650 0x7f584c06c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.403+0000 7f5853973700 1 --2- 172.21.15.3:0/3081709591 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f584c09e930 0x7f584c06b880 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.403+0000 7f5853973700 1 --2- 172.21.15.3:0/3081709591 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f584c09f240 0x7f584c06bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.403+0000 7f5853973700 1 -- 172.21.15.3:0/3081709591 >> 172.21.15.3:0/3081709591 conn(0x7f584c0a4210 msgr2=0x7f584c1b8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:34.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.404+0000 7f5853973700 1 -- 172.21.15.3:0/3081709591 shutdown_connections 2023-10-03T10:16:34.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.404+0000 7f5853973700 1 -- 172.21.15.3:0/3081709591 wait complete. 2023-10-03T10:16:34.417 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.433+0000 7f3123d44700 1 -- 172.21.15.3:0/133341823 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f311c074ea0 con 0x7f311c133420 2023-10-03T10:16:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.433+0000 7f31127fc700 1 -- 172.21.15.3:0/133341823 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3118015020 con 0x7f311c133420 2023-10-03T10:16:34.433 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:34.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.435+0000 7f3123d44700 1 -- 172.21.15.3:0/133341823 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f311c133420 msgr2=0x7f311c09c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:34.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.435+0000 7f3123d44700 1 --2- 172.21.15.3:0/133341823 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f311c133420 0x7f311c09c480 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3118000c00 tx=0x7f3118004750 comp rx=0 tx=0).stop 2023-10-03T10:16:34.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.435+0000 7f3123d44700 1 -- 172.21.15.3:0/133341823 shutdown_connections 2023-10-03T10:16:34.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.435+0000 7f3123d44700 1 --2- 172.21.15.3:0/133341823 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f311c134260 0x7f311c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.435+0000 7f3123d44700 1 --2- 172.21.15.3:0/133341823 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f311c133420 0x7f311c09c480 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.435+0000 7f3123d44700 1 --2- 172.21.15.3:0/133341823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f311c133e30 0x7f311c09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.435+0000 7f3123d44700 1 -- 172.21.15.3:0/133341823 >> 172.21.15.3:0/133341823 conn(0x7f311c0a4210 msgr2=0x7f311c130480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:34.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.435+0000 7f3123d44700 1 -- 172.21.15.3:0/133341823 shutdown_connections 2023-10-03T10:16:34.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.435+0000 7f3123d44700 1 -- 172.21.15.3:0/133341823 wait complete. 2023-10-03T10:16:34.449 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:34.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.517+0000 7f66ca144700 1 -- 172.21.15.3:0/3368512730 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f66c40a13a0 con 0x7f66c409e930 2023-10-03T10:16:34.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.518+0000 7f66abfff700 1 -- 172.21.15.3:0/3368512730 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f66b4011360 con 0x7f66c409e930 2023-10-03T10:16:34.518 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:34.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.520+0000 7f66ca144700 1 -- 172.21.15.3:0/3368512730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c409e930 msgr2=0x7f66c411c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:34.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.520+0000 7f66ca144700 1 --2- 172.21.15.3:0/3368512730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c409e930 0x7f66c411c5c0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f66b400c550 tx=0x7f66b400c860 comp rx=0 tx=0).stop 2023-10-03T10:16:34.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.520+0000 7f66ca144700 1 -- 172.21.15.3:0/3368512730 shutdown_connections 2023-10-03T10:16:34.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.520+0000 7f66ca144700 1 --2- 172.21.15.3:0/3368512730 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66c4136fe0 0x7f66c411d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.520+0000 7f66ca144700 1 --2- 172.21.15.3:0/3368512730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c409e930 0x7f66c411c5c0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.520+0000 7f66ca144700 1 --2- 172.21.15.3:0/3368512730 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66c409f410 0x7f66c411cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:34.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.520+0000 7f66ca144700 1 -- 172.21.15.3:0/3368512730 >> 172.21.15.3:0/3368512730 conn(0x7f66c40a4210 msgr2=0x7f66c4134a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:34.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.520+0000 7f66ca144700 1 -- 172.21.15.3:0/3368512730 shutdown_connections 2023-10-03T10:16:34.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:34.520+0000 7f66ca144700 1 -- 172.21.15.3:0/3368512730 wait complete. 2023-10-03T10:16:34.534 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:35.257 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:35.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.403+0000 7fcfdf639700 1 Processor -- start 2023-10-03T10:16:35.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.403+0000 7fcfdf639700 1 -- start start 2023-10-03T10:16:35.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.403+0000 7fcfdf639700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd809e930 0x7fcfd809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:35.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.403+0000 7fcfdf639700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcfd809f240 0x7fcfd81317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:35.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.403+0000 7fcfdf639700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd809f650 0x7fcfd8131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:35.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.403+0000 7fcfdf639700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd8070940 con 0x7fcfd809f240 2023-10-03T10:16:35.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.403+0000 7fcfdf639700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd806fd60 con 0x7fcfd809f650 2023-10-03T10:16:35.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.403+0000 7fcfdf639700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd806fa80 con 0x7fcfd809e930 2023-10-03T10:16:35.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.403+0000 7fcfdd3d5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd809e930 0x7fcfd809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:35.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.404+0000 7fcfdcbd4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcfd809f240 0x7fcfd81317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:35.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.404+0000 7fcfddbd6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd809f650 0x7fcfd8131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:35.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.404+0000 7fcfdd3d5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd809e930 0x7fcfd809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50408/0 (socket says 172.21.15.3:50408) 2023-10-03T10:16:35.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.404+0000 7fcfdcbd4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcfd809f240 0x7fcfd81317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48884/0 (socket says 172.21.15.3:48884) 2023-10-03T10:16:35.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.404+0000 7fcfddbd6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd809f650 0x7fcfd8131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58654/0 (socket says 172.21.15.3:58654) 2023-10-03T10:16:35.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.404+0000 7fcfdcbd4700 1 -- 172.21.15.3:0/2459275709 learned_addr learned my addr 172.21.15.3:0/2459275709 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:35.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.404+0000 7fcfdcbd4700 1 -- 172.21.15.3:0/2459275709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd809e930 msgr2=0x7fcfd809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:35.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.404+0000 7fcfdcbd4700 1 --2- 172.21.15.3:0/2459275709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd809e930 0x7fcfd809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.404+0000 7fcfdcbd4700 1 -- 172.21.15.3:0/2459275709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd809f650 msgr2=0x7fcfd8131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:35.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.404+0000 7fcfdcbd4700 1 --2- 172.21.15.3:0/2459275709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd809f650 0x7fcfd8131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.404+0000 7fcfdcbd4700 1 -- 172.21.15.3:0/2459275709 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd809fb60 con 0x7fcfd809f240 2023-10-03T10:16:35.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.404+0000 7fcfdcbd4700 1 --2- 172.21.15.3:0/2459275709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfd809f240 0x7fcfd81317f0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80098f0 tx=0x7fcfc8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9cd04183e5ed66c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:35.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.404+0000 7fcfcffff700 1 -- 172.21.15.3:0/2459275709 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8011830 con 0x7fcfd809f240 2023-10-03T10:16:35.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.404+0000 7fcfcffff700 1 -- 172.21.15.3:0/2459275709 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc8011990 con 0x7fcfd809f240 2023-10-03T10:16:35.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.404+0000 7fcfcffff700 1 -- 172.21.15.3:0/2459275709 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8011d20 con 0x7fcfd809f240 2023-10-03T10:16:35.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.405+0000 7fcfdf639700 1 -- 172.21.15.3:0/2459275709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfd809f240 msgr2=0x7fcfd81317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:35.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.405+0000 7fcfdf639700 1 --2- 172.21.15.3:0/2459275709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfd809f240 0x7fcfd81317f0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80098f0 tx=0x7fcfc8009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:35.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.405+0000 7fcfdf639700 1 -- 172.21.15.3:0/2459275709 shutdown_connections 2023-10-03T10:16:35.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.405+0000 7fcfdf639700 1 --2- 172.21.15.3:0/2459275709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd809f650 0x7fcfd8131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.405+0000 7fcfdf639700 1 --2- 172.21.15.3:0/2459275709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd809e930 0x7fcfd809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.405+0000 7fcfdf639700 1 --2- 172.21.15.3:0/2459275709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfd809f240 0x7fcfd81317f0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.405+0000 7fcfdf639700 1 -- 172.21.15.3:0/2459275709 >> 172.21.15.3:0/2459275709 conn(0x7fcfd80a4210 msgr2=0x7fcfd80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:35.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.405+0000 7fcfdf639700 1 -- 172.21.15.3:0/2459275709 shutdown_connections 2023-10-03T10:16:35.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.405+0000 7fcfdf639700 1 -- 172.21.15.3:0/2459275709 wait complete. 2023-10-03T10:16:35.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.406+0000 7fcfdf639700 1 Processor -- start 2023-10-03T10:16:35.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.406+0000 7fcfdf639700 1 -- start start 2023-10-03T10:16:35.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.407+0000 7fcfdf639700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd809e930 0x7fcfd81c0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:35.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.407+0000 7fcfdf639700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfd809f240 0x7fcfd81c1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:35.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.407+0000 7fcfdf639700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd809f650 0x7fcfd81c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:35.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.407+0000 7fcfdf639700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd81a53d0 con 0x7fcfd809f240 2023-10-03T10:16:35.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.407+0000 7fcfdf639700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd81c1880 con 0x7fcfd809e930 2023-10-03T10:16:35.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.407+0000 7fcfdf639700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd8071d20 con 0x7fcfd809f650 2023-10-03T10:16:35.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.407+0000 7fcfdd3d5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd809e930 0x7fcfd81c0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:35.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.407+0000 7fcfdcbd4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfd809f240 0x7fcfd81c1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:35.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.407+0000 7fcfddbd6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd809f650 0x7fcfd81c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:35.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.407+0000 7fcfdd3d5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd809e930 0x7fcfd81c0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58658/0 (socket says 172.21.15.3:58658) 2023-10-03T10:16:35.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.407+0000 7fcfdcbd4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfd809f240 0x7fcfd81c1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48894/0 (socket says 172.21.15.3:48894) 2023-10-03T10:16:35.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.407+0000 7fcfdd3d5700 1 -- 172.21.15.3:0/2429413778 learned_addr learned my addr 172.21.15.3:0/2429413778 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:35.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.407+0000 7fcfddbd6700 1 -- 172.21.15.3:0/2429413778 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd809e930 msgr2=0x7fcfd81c0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:35.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.407+0000 7fcfddbd6700 1 --2- 172.21.15.3:0/2429413778 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd809e930 0x7fcfd81c0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.407+0000 7fcfddbd6700 1 -- 172.21.15.3:0/2429413778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfd809f240 msgr2=0x7fcfd81c1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:35.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.407+0000 7fcfddbd6700 1 --2- 172.21.15.3:0/2429413778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfd809f240 0x7fcfd81c1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.407+0000 7fcfddbd6700 1 -- 172.21.15.3:0/2429413778 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc8009550 con 0x7fcfd809f650 2023-10-03T10:16:35.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.407+0000 7fcfdd3d5700 1 --2- 172.21.15.3:0/2429413778 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd809e930 0x7fcfd81c0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:35.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.407+0000 7fcfdcbd4700 1 --2- 172.21.15.3:0/2429413778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfd809f240 0x7fcfd81c1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:35.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.407+0000 7fcfddbd6700 1 --2- 172.21.15.3:0/2429413778 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd809f650 0x7fcfd81c3e20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000efc0 tx=0x7fcfd000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:35.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.408+0000 7fcfcdffb700 1 -- 172.21.15.3:0/2429413778 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd000cc90 con 0x7fcfd809f650 2023-10-03T10:16:35.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.408+0000 7fcfcdffb700 1 -- 172.21.15.3:0/2429413778 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd000f040 con 0x7fcfd809f650 2023-10-03T10:16:35.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.408+0000 7fcfcdffb700 1 -- 172.21.15.3:0/2429413778 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0009e60 con 0x7fcfd809f650 2023-10-03T10:16:35.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.408+0000 7fcfdf639700 1 -- 172.21.15.3:0/2429413778 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd80a0190 con 0x7fcfd809f650 2023-10-03T10:16:35.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.408+0000 7fcfdf639700 1 -- 172.21.15.3:0/2429413778 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd81a36c0 con 0x7fcfd809f650 2023-10-03T10:16:35.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.409+0000 7fcfcdffb700 1 -- 172.21.15.3:0/2429413778 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcfd0014480 con 0x7fcfd809f650 2023-10-03T10:16:35.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.409+0000 7fcfcdffb700 1 -- 172.21.15.3:0/2429413778 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fcfd003fe40 con 0x7fcfd809f650 2023-10-03T10:16:35.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.409+0000 7fcfdf639700 1 -- 172.21.15.3:0/2429413778 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd80a35f0 con 0x7fcfd809f650 2023-10-03T10:16:35.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.414+0000 7fcfcdffb700 1 -- 172.21.15.3:0/2429413778 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcfd0018020 con 0x7fcfd809f650 2023-10-03T10:16:35.418 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:35.450 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:35.534 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:35.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.558+0000 7f5653231700 1 Processor -- start 2023-10-03T10:16:35.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.558+0000 7f5653231700 1 -- start start 2023-10-03T10:16:35.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.559+0000 7f5653231700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f564c131810 0x7f564c131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:35.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.559+0000 7f5653231700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f564c099a40 0x7f564c099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:35.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.559+0000 7f5653231700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f564c09a3b0 0x7f564c138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:35.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.559+0000 7f5653231700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f564c09b580 con 0x7f564c09a3b0 2023-10-03T10:16:35.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.559+0000 7f5653231700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f564c1313c0 con 0x7f564c131810 2023-10-03T10:16:35.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.559+0000 7f5653231700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f564c06e8c0 con 0x7f564c099a40 2023-10-03T10:16:35.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.559+0000 7f564bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f564c099a40 0x7f564c099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:35.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.559+0000 7f5650fcd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f564c131810 0x7f564c131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:35.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.559+0000 7f5650fcd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f564c131810 0x7f564c131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58670/0 (socket says 172.21.15.3:58670) 2023-10-03T10:16:35.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.559+0000 7f564bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f564c099a40 0x7f564c099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50430/0 (socket says 172.21.15.3:50430) 2023-10-03T10:16:35.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.559+0000 7f5650fcd700 1 -- 172.21.15.3:0/2900522826 learned_addr learned my addr 172.21.15.3:0/2900522826 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:35.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.559+0000 7f56517ce700 1 --2- 172.21.15.3:0/2900522826 >> v2:172.21.15.3:3300/0 conn(0x7f564c09a3b0 0x7f564c138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:35.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.559+0000 7f5650fcd700 1 -- 172.21.15.3:0/2900522826 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f564c099a40 msgr2=0x7f564c099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:35.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.560+0000 7f5650fcd700 1 --2- 172.21.15.3:0/2900522826 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f564c099a40 0x7f564c099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.560+0000 7f5650fcd700 1 -- 172.21.15.3:0/2900522826 >> v2:172.21.15.3:3300/0 conn(0x7f564c09a3b0 msgr2=0x7f564c138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:35.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.560+0000 7f5650fcd700 1 --2- 172.21.15.3:0/2900522826 >> v2:172.21.15.3:3300/0 conn(0x7f564c09a3b0 0x7f564c138650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.560+0000 7f5650fcd700 1 -- 172.21.15.3:0/2900522826 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f564c09a820 con 0x7f564c131810 2023-10-03T10:16:35.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.560+0000 7f5650fcd700 1 --2- 172.21.15.3:0/2900522826 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f564c131810 0x7f564c131be0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f56400098f0 tx=0x7f5640009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae8aee3c0c145192 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:35.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.560+0000 7f564b7fe700 1 -- 172.21.15.3:0/2900522826 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640011830 con 0x7f564c131810 2023-10-03T10:16:35.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.560+0000 7f564b7fe700 1 -- 172.21.15.3:0/2900522826 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5640011990 con 0x7f564c131810 2023-10-03T10:16:35.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.560+0000 7f564b7fe700 1 -- 172.21.15.3:0/2900522826 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640011d20 con 0x7f564c131810 2023-10-03T10:16:35.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.561+0000 7f5653231700 1 -- 172.21.15.3:0/2900522826 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f564c131810 msgr2=0x7f564c131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:35.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.561+0000 7f5653231700 1 --2- 172.21.15.3:0/2900522826 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f564c131810 0x7f564c131be0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f56400098f0 tx=0x7f5640009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:35.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.561+0000 7f5653231700 1 -- 172.21.15.3:0/2900522826 shutdown_connections 2023-10-03T10:16:35.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.561+0000 7f5653231700 1 --2- 172.21.15.3:0/2900522826 >> v2:172.21.15.3:3300/0 conn(0x7f564c09a3b0 0x7f564c138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.561+0000 7f5653231700 1 --2- 172.21.15.3:0/2900522826 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f564c131810 0x7f564c131be0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.561+0000 7f5653231700 1 --2- 172.21.15.3:0/2900522826 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f564c099a40 0x7f564c099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.561+0000 7f5653231700 1 -- 172.21.15.3:0/2900522826 >> 172.21.15.3:0/2900522826 conn(0x7f564c0a2aa0 msgr2=0x7f564c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:35.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.561+0000 7f5653231700 1 -- 172.21.15.3:0/2900522826 shutdown_connections 2023-10-03T10:16:35.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.561+0000 7f5653231700 1 -- 172.21.15.3:0/2900522826 wait complete. 2023-10-03T10:16:35.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.562+0000 7f5653231700 1 Processor -- start 2023-10-03T10:16:35.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.562+0000 7f5653231700 1 -- start start 2023-10-03T10:16:35.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.563+0000 7f5653231700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f564c099a40 0x7f564c1cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:35.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.563+0000 7f5653231700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f564c09a3b0 0x7f564c1ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:35.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.563+0000 7f5653231700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f564c1cd280 0x7f564c1c9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:35.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.563+0000 7f5653231700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564c127600 con 0x7f564c1cd280 2023-10-03T10:16:35.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.563+0000 7f5653231700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f564c1cd6d0 con 0x7f564c09a3b0 2023-10-03T10:16:35.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.563+0000 7f5653231700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f564c1277e0 con 0x7f564c099a40 2023-10-03T10:16:35.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.563+0000 7f564bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f564c09a3b0 0x7f564c1ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:35.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.563+0000 7f564bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f564c09a3b0 0x7f564c1ccd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58682/0 (socket says 172.21.15.3:58682) 2023-10-03T10:16:35.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.563+0000 7f564bfff700 1 -- 172.21.15.3:0/3136735486 learned_addr learned my addr 172.21.15.3:0/3136735486 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:35.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.563+0000 7f5650fcd700 1 --2- 172.21.15.3:0/3136735486 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f564c099a40 0x7f564c1cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:35.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.563+0000 7f56517ce700 1 --2- 172.21.15.3:0/3136735486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f564c1cd280 0x7f564c1c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:35.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.564+0000 7f564bfff700 1 -- 172.21.15.3:0/3136735486 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f564c099a40 msgr2=0x7f564c1cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:35.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.564+0000 7f564bfff700 1 --2- 172.21.15.3:0/3136735486 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f564c099a40 0x7f564c1cc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.564+0000 7f564bfff700 1 -- 172.21.15.3:0/3136735486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f564c1cd280 msgr2=0x7f564c1c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:35.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.564+0000 7f564bfff700 1 --2- 172.21.15.3:0/3136735486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f564c1cd280 0x7f564c1c9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.564+0000 7f564bfff700 1 -- 172.21.15.3:0/3136735486 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5640009550 con 0x7f564c09a3b0 2023-10-03T10:16:35.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.564+0000 7f56517ce700 1 --2- 172.21.15.3:0/3136735486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f564c1cd280 0x7f564c1c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:16:35.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.564+0000 7f5650fcd700 1 --2- 172.21.15.3:0/3136735486 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f564c099a40 0x7f564c1cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:35.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.564+0000 7f564bfff700 1 --2- 172.21.15.3:0/3136735486 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f564c09a3b0 0x7f564c1ccd40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f563800e980 tx=0x7f563800ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:35.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.564+0000 7f56497fa700 1 -- 172.21.15.3:0/3136735486 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5638009710 con 0x7f564c09a3b0 2023-10-03T10:16:35.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.564+0000 7f56497fa700 1 -- 172.21.15.3:0/3136735486 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563800eed0 con 0x7f564c09a3b0 2023-10-03T10:16:35.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.564+0000 7f5653231700 1 -- 172.21.15.3:0/3136735486 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f564c12e760 con 0x7f564c09a3b0 2023-10-03T10:16:35.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.564+0000 7f56497fa700 1 -- 172.21.15.3:0/3136735486 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563800ca70 con 0x7f564c09a3b0 2023-10-03T10:16:35.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.564+0000 7f5653231700 1 -- 172.21.15.3:0/3136735486 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f564c06eb50 con 0x7f564c09a3b0 2023-10-03T10:16:35.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.566+0000 7f56497fa700 1 -- 172.21.15.3:0/3136735486 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f563800cbd0 con 0x7f564c09a3b0 2023-10-03T10:16:35.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.566+0000 7f56497fa700 1 -- 172.21.15.3:0/3136735486 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f563803f760 con 0x7f564c09a3b0 2023-10-03T10:16:35.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.567+0000 7f5653231700 1 -- 172.21.15.3:0/3136735486 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f564c074ea0 con 0x7f564c09a3b0 2023-10-03T10:16:35.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.576+0000 7f56497fa700 1 -- 172.21.15.3:0/3136735486 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f56380232f0 con 0x7f564c09a3b0 2023-10-03T10:16:35.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.616+0000 7f527e26d700 1 Processor -- start 2023-10-03T10:16:35.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.616+0000 7f527e26d700 1 -- start start 2023-10-03T10:16:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.617+0000 7f527e26d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5278133420 0x7f52781337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.617+0000 7f527e26d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5278133e30 0x7f527812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:35.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.617+0000 7f527e26d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f527806b990 con 0x7f5278134260 2023-10-03T10:16:35.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.617+0000 7f527e26d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52780a0590 con 0x7f5278133420 2023-10-03T10:16:35.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.617+0000 7f527e26d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f527809fb80 con 0x7f5278133e30 2023-10-03T10:16:35.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.617+0000 7f5277fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5278133420 0x7f52781337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:35.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.617+0000 7f5277fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5278133420 0x7f52781337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58698/0 (socket says 172.21.15.3:58698) 2023-10-03T10:16:35.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.617+0000 7f5277fff700 1 -- 172.21.15.3:0/2749451652 learned_addr learned my addr 172.21.15.3:0/2749451652 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:35.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.617+0000 7f52777fe700 1 --2- 172.21.15.3:0/2749451652 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5278133e30 0x7f527812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:35.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.618+0000 7f5277fff700 1 -- 172.21.15.3:0/2749451652 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5278133e30 msgr2=0x7f527812ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:35.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.618+0000 7f5277fff700 1 --2- 172.21.15.3:0/2749451652 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5278133e30 0x7f527812ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.618+0000 7f5277fff700 1 -- 172.21.15.3:0/2749451652 >> v1:172.21.15.3:6789/0 conn(0x7f5278134260 legacy=0x7f52781304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:35.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.618+0000 7f5277fff700 1 -- 172.21.15.3:0/2749451652 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52780a0030 con 0x7f5278133420 2023-10-03T10:16:35.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.618+0000 7f5277fff700 1 --2- 172.21.15.3:0/2749451652 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5278133420 0x7f52781337f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5268009920 tx=0x7f5268009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecb7bad971bd1452 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:35.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.618+0000 7f5276ffd700 1 -- 172.21.15.3:0/2749451652 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5268011830 con 0x7f5278133420 2023-10-03T10:16:35.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.618+0000 7f5276ffd700 1 -- 172.21.15.3:0/2749451652 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5268011990 con 0x7f5278133420 2023-10-03T10:16:35.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.618+0000 7f5276ffd700 1 -- 172.21.15.3:0/2749451652 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5268011d20 con 0x7f5278133420 2023-10-03T10:16:35.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.619+0000 7f527e26d700 1 -- 172.21.15.3:0/2749451652 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5278133420 msgr2=0x7f52781337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:35.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.619+0000 7f527e26d700 1 --2- 172.21.15.3:0/2749451652 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5278133420 0x7f52781337f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5268009920 tx=0x7f5268009c30 comp rx=0 tx=0).stop 2023-10-03T10:16:35.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.619+0000 7f527e26d700 1 -- 172.21.15.3:0/2749451652 shutdown_connections 2023-10-03T10:16:35.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.619+0000 7f527e26d700 1 --2- 172.21.15.3:0/2749451652 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5278133420 0x7f52781337f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.619+0000 7f527e26d700 1 --2- 172.21.15.3:0/2749451652 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5278133e30 0x7f527812ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.619+0000 7f527e26d700 1 -- 172.21.15.3:0/2749451652 >> 172.21.15.3:0/2749451652 conn(0x7f52780a4210 msgr2=0x7f52780a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:35.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.619+0000 7f527e26d700 1 -- 172.21.15.3:0/2749451652 shutdown_connections 2023-10-03T10:16:35.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.619+0000 7f527e26d700 1 -- 172.21.15.3:0/2749451652 wait complete. 2023-10-03T10:16:35.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.620+0000 7f527e26d700 1 Processor -- start 2023-10-03T10:16:35.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.620+0000 7f527e26d700 1 -- start start 2023-10-03T10:16:35.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.621+0000 7f527e26d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5278133420 0x7f527809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:35.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.621+0000 7f527e26d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5278133e30 0x7f527809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:35.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.621+0000 7f527e26d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5278134260 0x7f527809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:35.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.621+0000 7f527e26d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527809def0 con 0x7f5278133e30 2023-10-03T10:16:35.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.621+0000 7f527e26d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f527811ffb0 con 0x7f5278134260 2023-10-03T10:16:35.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.621+0000 7f527e26d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f52781203e0 con 0x7f5278133420 2023-10-03T10:16:35.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.621+0000 7f527c80a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5278134260 0x7f527809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:35.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.621+0000 7f5277fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5278133420 0x7f527809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:35.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.621+0000 7f527c80a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5278134260 0x7f527809cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58704/0 (socket says 172.21.15.3:58704) 2023-10-03T10:16:35.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.621+0000 7f527c80a700 1 -- 172.21.15.3:0/2367193574 learned_addr learned my addr 172.21.15.3:0/2367193574 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:35.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.621+0000 7f52777fe700 1 --2- 172.21.15.3:0/2367193574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5278133e30 0x7f527809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:35.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.621+0000 7f5277fff700 1 -- 172.21.15.3:0/2367193574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5278134260 msgr2=0x7f527809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:35.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.621+0000 7f5277fff700 1 --2- 172.21.15.3:0/2367193574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5278134260 0x7f527809cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.621+0000 7f5277fff700 1 -- 172.21.15.3:0/2367193574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5278133e30 msgr2=0x7f527809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:35.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.621+0000 7f5277fff700 1 --2- 172.21.15.3:0/2367193574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5278133e30 0x7f527809c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.621+0000 7f5277fff700 1 -- 172.21.15.3:0/2367193574 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5268009550 con 0x7f5278133420 2023-10-03T10:16:35.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.622+0000 7f5277fff700 1 --2- 172.21.15.3:0/2367193574 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5278133420 0x7f527809c480 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5268016000 tx=0x7f5268004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:35.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.622+0000 7f5274ff9700 1 -- 172.21.15.3:0/2367193574 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5268004450 con 0x7f5278133420 2023-10-03T10:16:35.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.622+0000 7f5274ff9700 1 -- 172.21.15.3:0/2367193574 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52680045b0 con 0x7f5278133420 2023-10-03T10:16:35.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.622+0000 7f527e26d700 1 -- 172.21.15.3:0/2367193574 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f527811fcf0 con 0x7f5278133420 2023-10-03T10:16:35.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.622+0000 7f5274ff9700 1 -- 172.21.15.3:0/2367193574 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52680038c0 con 0x7f5278133420 2023-10-03T10:16:35.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.622+0000 7f527e26d700 1 -- 172.21.15.3:0/2367193574 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f527811ec70 con 0x7f5278133420 2023-10-03T10:16:35.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.623+0000 7f5274ff9700 1 -- 172.21.15.3:0/2367193574 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5268003a20 con 0x7f5278133420 2023-10-03T10:16:35.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.623+0000 7f5274ff9700 1 -- 172.21.15.3:0/2367193574 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5268021030 con 0x7f5278133420 2023-10-03T10:16:35.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.623+0000 7f527e26d700 1 -- 172.21.15.3:0/2367193574 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52781201f0 con 0x7f5278133420 2023-10-03T10:16:35.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.629+0000 7f5274ff9700 1 -- 172.21.15.3:0/2367193574 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f526801a430 con 0x7f5278133420 2023-10-03T10:16:35.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.638+0000 7fcfdf639700 1 -- 172.21.15.3:0/2429413778 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcfd8074ea0 con 0x7fcfd809f650 2023-10-03T10:16:35.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.638+0000 7fcfcdffb700 1 -- 172.21.15.3:0/2429413778 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcfd00118b0 con 0x7fcfd809f650 2023-10-03T10:16:35.638 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:35.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.640+0000 7fcfdf639700 1 -- 172.21.15.3:0/2429413778 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd809f650 msgr2=0x7fcfd81c3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:35.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.640+0000 7fcfdf639700 1 --2- 172.21.15.3:0/2429413778 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd809f650 0x7fcfd81c3e20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000efc0 tx=0x7fcfd000c3a0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.640+0000 7fcfdf639700 1 -- 172.21.15.3:0/2429413778 shutdown_connections 2023-10-03T10:16:35.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.640+0000 7fcfdf639700 1 --2- 172.21.15.3:0/2429413778 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd809f650 0x7fcfd81c3e20 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.640+0000 7fcfdf639700 1 --2- 172.21.15.3:0/2429413778 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd809e930 0x7fcfd81c0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.640+0000 7fcfdf639700 1 --2- 172.21.15.3:0/2429413778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfd809f240 0x7fcfd81c1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.640+0000 7fcfdf639700 1 -- 172.21.15.3:0/2429413778 >> 172.21.15.3:0/2429413778 conn(0x7fcfd80a4210 msgr2=0x7fcfd81b6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:35.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.641+0000 7fcfdf639700 1 -- 172.21.15.3:0/2429413778 shutdown_connections 2023-10-03T10:16:35.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.641+0000 7fcfdf639700 1 -- 172.21.15.3:0/2429413778 wait complete. 2023-10-03T10:16:35.656 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:35.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.720+0000 7ffa17dce700 1 Processor -- start 2023-10-03T10:16:35.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.720+0000 7ffa17dce700 1 -- start start 2023-10-03T10:16:35.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.721+0000 7ffa17dce700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffa1009f330 0x7ffa10138e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:35.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.721+0000 7ffa17dce700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa10139370 0x7ffa1013b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:35.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.721+0000 7ffa17dce700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa1006bc50 con 0x7ffa1009e850 2023-10-03T10:16:35.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.721+0000 7ffa17dce700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa1009ae60 con 0x7ffa1009f330 2023-10-03T10:16:35.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.721+0000 7ffa17dce700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1006b990 con 0x7ffa10139370 2023-10-03T10:16:35.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.721+0000 7ffa15369700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffa1009f330 0x7ffa10138e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:35.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.721+0000 7ffa1636b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa10139370 0x7ffa1013b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:35.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.721+0000 7ffa15369700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffa1009f330 0x7ffa10138e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48934/0 (socket says 172.21.15.3:48934) 2023-10-03T10:16:35.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.721+0000 7ffa15369700 1 -- 172.21.15.3:0/2695158084 learned_addr learned my addr 172.21.15.3:0/2695158084 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:35.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.721+0000 7ffa1636b700 1 -- 172.21.15.3:0/2695158084 >> v2:172.21.15.3:3300/0 conn(0x7ffa1009f330 msgr2=0x7ffa10138e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:35.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.721+0000 7ffa1636b700 1 --2- 172.21.15.3:0/2695158084 >> v2:172.21.15.3:3300/0 conn(0x7ffa1009f330 0x7ffa10138e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.721+0000 7ffa1636b700 1 -- 172.21.15.3:0/2695158084 >> v1:172.21.15.3:6789/0 conn(0x7ffa1009e850 legacy=0x7ffa1009ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:35.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.721+0000 7ffa1636b700 1 -- 172.21.15.3:0/2695158084 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1006cb10 con 0x7ffa10139370 2023-10-03T10:16:35.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.721+0000 7ffa15369700 1 --2- 172.21.15.3:0/2695158084 >> v2:172.21.15.3:3300/0 conn(0x7ffa1009f330 0x7ffa10138e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:35.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.722+0000 7ffa1636b700 1 --2- 172.21.15.3:0/2695158084 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa10139370 0x7ffa1013b750 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009dc0 tx=0x7ffa0800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2bae623cdcdf8a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.722+0000 7ffa14b68700 1 -- 172.21.15.3:0/2695158084 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0800e070 con 0x7ffa10139370 2023-10-03T10:16:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.722+0000 7ffa14b68700 1 -- 172.21.15.3:0/2695158084 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa0800bae0 con 0x7ffa10139370 2023-10-03T10:16:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.722+0000 7ffa14b68700 1 -- 172.21.15.3:0/2695158084 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0800be90 con 0x7ffa10139370 2023-10-03T10:16:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.722+0000 7ffa17dce700 1 -- 172.21.15.3:0/2695158084 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa10139370 msgr2=0x7ffa1013b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.722+0000 7ffa17dce700 1 --2- 172.21.15.3:0/2695158084 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa10139370 0x7ffa1013b750 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009dc0 tx=0x7ffa0800b330 comp rx=0 tx=0).stop 2023-10-03T10:16:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.722+0000 7ffa17dce700 1 -- 172.21.15.3:0/2695158084 shutdown_connections 2023-10-03T10:16:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.722+0000 7ffa17dce700 1 --2- 172.21.15.3:0/2695158084 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa10139370 0x7ffa1013b750 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.722+0000 7ffa17dce700 1 --2- 172.21.15.3:0/2695158084 >> v2:172.21.15.3:3300/0 conn(0x7ffa1009f330 0x7ffa10138e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.722+0000 7ffa17dce700 1 -- 172.21.15.3:0/2695158084 >> 172.21.15.3:0/2695158084 conn(0x7ffa100a3f70 msgr2=0x7ffa100a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.722+0000 7ffa17dce700 1 -- 172.21.15.3:0/2695158084 shutdown_connections 2023-10-03T10:16:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.722+0000 7ffa17dce700 1 -- 172.21.15.3:0/2695158084 wait complete. 2023-10-03T10:16:35.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.723+0000 7ffa17dce700 1 Processor -- start 2023-10-03T10:16:35.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.723+0000 7ffa17dce700 1 -- start start 2023-10-03T10:16:35.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.724+0000 7ffa17dce700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa1009e850 0x7ffa1011f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:35.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.724+0000 7ffa17dce700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa1009f330 0x7ffa1011fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.724+0000 7ffa17dce700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffa10139370 0x7ffa101c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.724+0000 7ffa17dce700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1011f4d0 con 0x7ffa1009f330 2023-10-03T10:16:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.724+0000 7ffa17dce700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa10122670 con 0x7ffa1009e850 2023-10-03T10:16:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.724+0000 7ffa17dce700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1006f810 con 0x7ffa10139370 2023-10-03T10:16:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.724+0000 7ffa15369700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa1009f330 0x7ffa1011fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.724+0000 7ffa15369700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa1009f330 0x7ffa1011fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48942/0 (socket says 172.21.15.3:48942) 2023-10-03T10:16:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.724+0000 7ffa15369700 1 -- 172.21.15.3:0/1822985810 learned_addr learned my addr 172.21.15.3:0/1822985810 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.724+0000 7ffa1636b700 1 --2- 172.21.15.3:0/1822985810 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffa10139370 0x7ffa101c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.724+0000 7ffa15b6a700 1 --2- 172.21.15.3:0/1822985810 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa1009e850 0x7ffa1011f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.724+0000 7ffa15369700 1 -- 172.21.15.3:0/1822985810 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffa10139370 msgr2=0x7ffa101c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.724+0000 7ffa15369700 1 --2- 172.21.15.3:0/1822985810 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffa10139370 0x7ffa101c8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.724+0000 7ffa15369700 1 -- 172.21.15.3:0/1822985810 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa1009e850 msgr2=0x7ffa1011f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:35.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.724+0000 7ffa15369700 1 --2- 172.21.15.3:0/1822985810 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa1009e850 0x7ffa1011f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.724+0000 7ffa15369700 1 -- 172.21.15.3:0/1822985810 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa00009550 con 0x7ffa1009f330 2023-10-03T10:16:35.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.725+0000 7ffa15b6a700 1 --2- 172.21.15.3:0/1822985810 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa1009e850 0x7ffa1011f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:16:35.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.725+0000 7ffa15369700 1 --2- 172.21.15.3:0/1822985810 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa1009f330 0x7ffa1011fd60 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7ffa0000ebe0 tx=0x7ffa0000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:35.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.725+0000 7ffa067fc700 1 -- 172.21.15.3:0/1822985810 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0000c850 con 0x7ffa1009f330 2023-10-03T10:16:35.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.725+0000 7ffa067fc700 1 -- 172.21.15.3:0/1822985810 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa0000c9b0 con 0x7ffa1009f330 2023-10-03T10:16:35.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.725+0000 7ffa17dce700 1 -- 172.21.15.3:0/1822985810 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa08009a70 con 0x7ffa1009f330 2023-10-03T10:16:35.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.725+0000 7ffa067fc700 1 -- 172.21.15.3:0/1822985810 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0000cd20 con 0x7ffa1009f330 2023-10-03T10:16:35.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.725+0000 7ffa17dce700 1 -- 172.21.15.3:0/1822985810 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa101202a0 con 0x7ffa1009f330 2023-10-03T10:16:35.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.726+0000 7ffa067fc700 1 -- 172.21.15.3:0/1822985810 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffa00013470 con 0x7ffa1009f330 2023-10-03T10:16:35.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.726+0000 7ffa067fc700 1 -- 172.21.15.3:0/1822985810 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ffa0003f9d0 con 0x7ffa1009f330 2023-10-03T10:16:35.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.727+0000 7ffa17dce700 1 -- 172.21.15.3:0/1822985810 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa1006c3f0 con 0x7ffa1009f330 2023-10-03T10:16:35.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.732+0000 7ffa067fc700 1 -- 172.21.15.3:0/1822985810 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffa00021080 con 0x7ffa1009f330 2023-10-03T10:16:35.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.788+0000 7f5653231700 1 -- 172.21.15.3:0/3136735486 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f564c127460 con 0x7f564c09a3b0 2023-10-03T10:16:35.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.789+0000 7f56497fa700 1 -- 172.21.15.3:0/3136735486 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f564c127460 con 0x7f564c09a3b0 2023-10-03T10:16:35.789 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:35.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.791+0000 7f5653231700 1 -- 172.21.15.3:0/3136735486 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f564c09a3b0 msgr2=0x7f564c1ccd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:35.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.791+0000 7f5653231700 1 --2- 172.21.15.3:0/3136735486 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f564c09a3b0 0x7f564c1ccd40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f563800e980 tx=0x7f563800ec90 comp rx=0 tx=0).stop 2023-10-03T10:16:35.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.791+0000 7f5653231700 1 -- 172.21.15.3:0/3136735486 shutdown_connections 2023-10-03T10:16:35.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.791+0000 7f5653231700 1 --2- 172.21.15.3:0/3136735486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f564c1cd280 0x7f564c1c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.791+0000 7f5653231700 1 --2- 172.21.15.3:0/3136735486 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f564c099a40 0x7f564c1cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.791+0000 7f5653231700 1 --2- 172.21.15.3:0/3136735486 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f564c09a3b0 0x7f564c1ccd40 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.791+0000 7f5653231700 1 -- 172.21.15.3:0/3136735486 >> 172.21.15.3:0/3136735486 conn(0x7f564c0a2aa0 msgr2=0x7f564c0a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:35.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.791+0000 7f5653231700 1 -- 172.21.15.3:0/3136735486 shutdown_connections 2023-10-03T10:16:35.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.791+0000 7f5653231700 1 -- 172.21.15.3:0/3136735486 wait complete. 2023-10-03T10:16:35.806 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:35.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.891+0000 7f527e26d700 1 -- 172.21.15.3:0/2367193574 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5278074ea0 con 0x7f5278133420 2023-10-03T10:16:35.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.891+0000 7f5274ff9700 1 -- 172.21.15.3:0/2367193574 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f526800bc10 con 0x7f5278133420 2023-10-03T10:16:35.891 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:35.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.893+0000 7f527e26d700 1 -- 172.21.15.3:0/2367193574 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5278133420 msgr2=0x7f527809c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:35.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.893+0000 7f527e26d700 1 --2- 172.21.15.3:0/2367193574 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5278133420 0x7f527809c480 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5268016000 tx=0x7f5268004750 comp rx=0 tx=0).stop 2023-10-03T10:16:35.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.893+0000 7f527e26d700 1 -- 172.21.15.3:0/2367193574 shutdown_connections 2023-10-03T10:16:35.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.893+0000 7f527e26d700 1 --2- 172.21.15.3:0/2367193574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5278134260 0x7f527809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.893+0000 7f527e26d700 1 --2- 172.21.15.3:0/2367193574 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5278133420 0x7f527809c480 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.893+0000 7f527e26d700 1 --2- 172.21.15.3:0/2367193574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5278133e30 0x7f527809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.893+0000 7f527e26d700 1 -- 172.21.15.3:0/2367193574 >> 172.21.15.3:0/2367193574 conn(0x7f52780a4210 msgr2=0x7f52780a3120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:35.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.894+0000 7f527e26d700 1 -- 172.21.15.3:0/2367193574 shutdown_connections 2023-10-03T10:16:35.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.894+0000 7f527e26d700 1 -- 172.21.15.3:0/2367193574 wait complete. 2023-10-03T10:16:35.908 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:35.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.947+0000 7ffa17dce700 1 -- 172.21.15.3:0/1822985810 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ffa101222a0 con 0x7ffa1009f330 2023-10-03T10:16:35.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.947+0000 7ffa067fc700 1 -- 172.21.15.3:0/1822985810 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffa00017020 con 0x7ffa1009f330 2023-10-03T10:16:35.947 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:35.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.950+0000 7ffa17dce700 1 -- 172.21.15.3:0/1822985810 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa1009f330 msgr2=0x7ffa1011fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:35.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.950+0000 7ffa17dce700 1 --2- 172.21.15.3:0/1822985810 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa1009f330 0x7ffa1011fd60 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7ffa0000ebe0 tx=0x7ffa0000eef0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.950+0000 7ffa17dce700 1 -- 172.21.15.3:0/1822985810 shutdown_connections 2023-10-03T10:16:35.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.950+0000 7ffa17dce700 1 --2- 172.21.15.3:0/1822985810 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffa10139370 0x7ffa101c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.950+0000 7ffa17dce700 1 --2- 172.21.15.3:0/1822985810 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffa1009e850 0x7ffa1011f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.950+0000 7ffa17dce700 1 --2- 172.21.15.3:0/1822985810 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa1009f330 0x7ffa1011fd60 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:35.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.950+0000 7ffa17dce700 1 -- 172.21.15.3:0/1822985810 >> 172.21.15.3:0/1822985810 conn(0x7ffa100a3f70 msgr2=0x7ffa1009d360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:35.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.950+0000 7ffa17dce700 1 -- 172.21.15.3:0/1822985810 shutdown_connections 2023-10-03T10:16:35.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:35.950+0000 7ffa17dce700 1 -- 172.21.15.3:0/1822985810 wait complete. 2023-10-03T10:16:35.963 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:36.657 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:36.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.803+0000 7f5667483700 1 Processor -- start 2023-10-03T10:16:36.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.803+0000 7f5667483700 1 -- start start 2023-10-03T10:16:36.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.803+0000 7f5667483700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f566009f410 0x7f5660136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:36.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.803+0000 7f5667483700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5660136fe0 0x7f56601393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:36.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.803+0000 7f5667483700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5660070940 con 0x7f566009e930 2023-10-03T10:16:36.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.804+0000 7f5667483700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f566006fd60 con 0x7f5660136fe0 2023-10-03T10:16:36.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.804+0000 7f5667483700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f566006fa80 con 0x7f566009f410 2023-10-03T10:16:36.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.804+0000 7f5665a20700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5660136fe0 0x7f56601393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:36.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.804+0000 7f5664a1e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f566009f410 0x7f5660136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:36.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.804+0000 7f5665a20700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5660136fe0 0x7f56601393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48956/0 (socket says 172.21.15.3:48956) 2023-10-03T10:16:36.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.804+0000 7f5664a1e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f566009f410 0x7f5660136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58724/0 (socket says 172.21.15.3:58724) 2023-10-03T10:16:36.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.804+0000 7f5665a20700 1 -- 172.21.15.3:0/3553878060 learned_addr learned my addr 172.21.15.3:0/3553878060 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:36.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.804+0000 7f5665a20700 1 -- 172.21.15.3:0/3553878060 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f566009f410 msgr2=0x7f5660136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:36.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.804+0000 7f5665a20700 1 --2- 172.21.15.3:0/3553878060 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f566009f410 0x7f5660136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:36.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.804+0000 7f5665a20700 1 -- 172.21.15.3:0/3553878060 >> v1:172.21.15.3:6789/0 conn(0x7f566009e930 legacy=0x7f566009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:36.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.804+0000 7f5665a20700 1 -- 172.21.15.3:0/3553878060 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f566009fb60 con 0x7f5660136fe0 2023-10-03T10:16:36.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.804+0000 7f5664a1e700 1 --2- 172.21.15.3:0/3553878060 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f566009f410 0x7f5660136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:36.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.804+0000 7f5665a20700 1 --2- 172.21.15.3:0/3553878060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5660136fe0 0x7f56601393c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f5658009e50 tx=0x7f565800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe25ed1b5f4f7347 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:36.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.804+0000 7f5657fff700 1 -- 172.21.15.3:0/3553878060 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565800e070 con 0x7f5660136fe0 2023-10-03T10:16:36.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.804+0000 7f5657fff700 1 -- 172.21.15.3:0/3553878060 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565800b910 con 0x7f5660136fe0 2023-10-03T10:16:36.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.804+0000 7f5657fff700 1 -- 172.21.15.3:0/3553878060 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565800bcc0 con 0x7f5660136fe0 2023-10-03T10:16:36.806 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:36.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.805+0000 7f5667483700 1 -- 172.21.15.3:0/3553878060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5660136fe0 msgr2=0x7f56601393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:36.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.805+0000 7f5667483700 1 --2- 172.21.15.3:0/3553878060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5660136fe0 0x7f56601393c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f5658009e50 tx=0x7f565800b330 comp rx=0 tx=0).stop 2023-10-03T10:16:36.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.805+0000 7f5667483700 1 -- 172.21.15.3:0/3553878060 shutdown_connections 2023-10-03T10:16:36.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.805+0000 7f5667483700 1 --2- 172.21.15.3:0/3553878060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5660136fe0 0x7f56601393c0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:36.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.805+0000 7f5667483700 1 --2- 172.21.15.3:0/3553878060 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f566009f410 0x7f5660136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:36.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.805+0000 7f5667483700 1 -- 172.21.15.3:0/3553878060 >> 172.21.15.3:0/3553878060 conn(0x7f56600a4210 msgr2=0x7f56600a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:36.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.805+0000 7f5667483700 1 -- 172.21.15.3:0/3553878060 shutdown_connections 2023-10-03T10:16:36.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.805+0000 7f5667483700 1 -- 172.21.15.3:0/3553878060 wait complete. 2023-10-03T10:16:36.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.806+0000 7f5667483700 1 Processor -- start 2023-10-03T10:16:36.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.806+0000 7f5667483700 1 -- start start 2023-10-03T10:16:36.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.807+0000 7f5667483700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f566009e930 0x7f56601c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:36.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.807+0000 7f5667483700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f566009f410 0x7f56601c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:36.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.807+0000 7f5667483700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5660136fe0 0x7f566006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:36.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.807+0000 7f5667483700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5660124000 con 0x7f5660136fe0 2023-10-03T10:16:36.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.807+0000 7f5667483700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56601bc0c0 con 0x7f566009f410 2023-10-03T10:16:36.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.807+0000 7f5667483700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5660134370 con 0x7f566009e930 2023-10-03T10:16:36.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.807+0000 7f5664a1e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f566009f410 0x7f56601c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:36.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.807+0000 7f5665a20700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5660136fe0 0x7f566006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:36.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.807+0000 7f566521f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f566009e930 0x7f56601c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:36.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.807+0000 7f5664a1e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f566009f410 0x7f56601c8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58726/0 (socket says 172.21.15.3:58726) 2023-10-03T10:16:36.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.807+0000 7f5664a1e700 1 -- 172.21.15.3:0/1027516118 learned_addr learned my addr 172.21.15.3:0/1027516118 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:36.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.807+0000 7f5664a1e700 1 -- 172.21.15.3:0/1027516118 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f566009e930 msgr2=0x7f56601c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:36.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.807+0000 7f5664a1e700 1 --2- 172.21.15.3:0/1027516118 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f566009e930 0x7f56601c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:36.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.807+0000 7f5664a1e700 1 -- 172.21.15.3:0/1027516118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5660136fe0 msgr2=0x7f566006b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:36.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.807+0000 7f5664a1e700 1 --2- 172.21.15.3:0/1027516118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5660136fe0 0x7f566006b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:36.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.807+0000 7f5664a1e700 1 -- 172.21.15.3:0/1027516118 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56500094a0 con 0x7f566009f410 2023-10-03T10:16:36.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.807+0000 7f5665a20700 1 --2- 172.21.15.3:0/1027516118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5660136fe0 0x7f566006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:36.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.807+0000 7f5664a1e700 1 --2- 172.21.15.3:0/1027516118 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f566009f410 0x7f56601c8a80 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f565000ebe0 tx=0x7f565000eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:36.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.808+0000 7f5655ffb700 1 -- 172.21.15.3:0/1027516118 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565000c7f0 con 0x7f566009f410 2023-10-03T10:16:36.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.808+0000 7f5655ffb700 1 -- 172.21.15.3:0/1027516118 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565000c950 con 0x7f566009f410 2023-10-03T10:16:36.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.808+0000 7f5667483700 1 -- 172.21.15.3:0/1027516118 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5658009b30 con 0x7f566009f410 2023-10-03T10:16:36.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.808+0000 7f5655ffb700 1 -- 172.21.15.3:0/1027516118 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565000cce0 con 0x7f566009f410 2023-10-03T10:16:36.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.808+0000 7f5667483700 1 -- 172.21.15.3:0/1027516118 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5660122bc0 con 0x7f566009f410 2023-10-03T10:16:36.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.809+0000 7f5655ffb700 1 -- 172.21.15.3:0/1027516118 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5650013470 con 0x7f566009f410 2023-10-03T10:16:36.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.809+0000 7f5655ffb700 1 -- 172.21.15.3:0/1027516118 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f565003f810 con 0x7f566009f410 2023-10-03T10:16:36.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.809+0000 7f5667483700 1 -- 172.21.15.3:0/1027516118 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5660124140 con 0x7f566009f410 2023-10-03T10:16:36.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.814+0000 7f5655ffb700 1 -- 172.21.15.3:0/1027516118 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5660124140 con 0x7f566009f410 2023-10-03T10:16:36.909 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:36.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.950+0000 7f4803361700 1 Processor -- start 2023-10-03T10:16:36.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.950+0000 7f4803361700 1 -- start start 2023-10-03T10:16:36.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.951+0000 7f4803361700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47fc130540 0x7f47fc130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:36.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.951+0000 7f4803361700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f47fc130e50 0x7f47fc06f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:36.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.951+0000 7f4803361700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47fc0716a0 con 0x7f47fc06fb20 2023-10-03T10:16:36.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.951+0000 7f4803361700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc09a390 con 0x7f47fc130e50 2023-10-03T10:16:36.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.951+0000 7f4803361700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc071290 con 0x7f47fc130540 2023-10-03T10:16:36.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.951+0000 7f48008fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f47fc130e50 0x7f47fc06f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:36.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.951+0000 7f48018fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f47fc06fb20 0x7f47fc137560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35828/0 (socket says 172.21.15.3:35828) 2023-10-03T10:16:36.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.951+0000 7f48018fe700 1 -- 172.21.15.3:0/2401539753 learned_addr learned my addr 172.21.15.3:0/2401539753 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:36.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.951+0000 7f48010fd700 1 --2- 172.21.15.3:0/2401539753 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47fc130540 0x7f47fc130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:36.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.952+0000 7f47f3fff700 1 -- 172.21.15.3:0/2401539753 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3101977530 0 0) 0x7f47fc0716a0 con 0x7f47fc06fb20 2023-10-03T10:16:36.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.952+0000 7f47f3fff700 1 -- 172.21.15.3:0/2401539753 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47e80035b0 con 0x7f47fc06fb20 2023-10-03T10:16:36.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.952+0000 7f48010fd700 1 -- 172.21.15.3:0/2401539753 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f47fc130e50 msgr2=0x7f47fc06f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:36.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.952+0000 7f48010fd700 1 --2- 172.21.15.3:0/2401539753 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f47fc130e50 0x7f47fc06f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:36.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.952+0000 7f48010fd700 1 -- 172.21.15.3:0/2401539753 >> v1:172.21.15.3:6789/0 conn(0x7f47fc06fb20 legacy=0x7f47fc137560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:36.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.952+0000 7f48010fd700 1 -- 172.21.15.3:0/2401539753 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47fc130340 con 0x7f47fc130540 2023-10-03T10:16:36.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.952+0000 7f48008fc700 1 --2- 172.21.15.3:0/2401539753 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f47fc130e50 0x7f47fc06f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:36.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.952+0000 7f48010fd700 1 --2- 172.21.15.3:0/2401539753 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47fc130540 0x7f47fc130910 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f47f80098a0 tx=0x7f47f8009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8f25f6abce6c4f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:36.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.952+0000 7f47f3fff700 1 -- 172.21.15.3:0/2401539753 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f8011830 con 0x7f47fc130540 2023-10-03T10:16:36.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.952+0000 7f47f3fff700 1 -- 172.21.15.3:0/2401539753 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f8011990 con 0x7f47fc130540 2023-10-03T10:16:36.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.952+0000 7f47f3fff700 1 -- 172.21.15.3:0/2401539753 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f8011d20 con 0x7f47fc130540 2023-10-03T10:16:36.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.953+0000 7f4803361700 1 -- 172.21.15.3:0/2401539753 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47fc130540 msgr2=0x7f47fc130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:36.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.953+0000 7f4803361700 1 --2- 172.21.15.3:0/2401539753 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47fc130540 0x7f47fc130910 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f47f80098a0 tx=0x7f47f8009bb0 comp rx=0 tx=0).stop 2023-10-03T10:16:36.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.953+0000 7f4803361700 1 -- 172.21.15.3:0/2401539753 shutdown_connections 2023-10-03T10:16:36.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.953+0000 7f4803361700 1 --2- 172.21.15.3:0/2401539753 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47fc130540 0x7f47fc130910 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:36.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.953+0000 7f4803361700 1 --2- 172.21.15.3:0/2401539753 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f47fc130e50 0x7f47fc06f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:36.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.953+0000 7f4803361700 1 -- 172.21.15.3:0/2401539753 >> 172.21.15.3:0/2401539753 conn(0x7f47fc0a2aa0 msgr2=0x7f47fc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:36.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.953+0000 7f4803361700 1 -- 172.21.15.3:0/2401539753 shutdown_connections 2023-10-03T10:16:36.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.953+0000 7f4803361700 1 -- 172.21.15.3:0/2401539753 wait complete. 2023-10-03T10:16:36.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.954+0000 7f4803361700 1 Processor -- start 2023-10-03T10:16:36.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.954+0000 7f4803361700 1 -- start start 2023-10-03T10:16:36.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.954+0000 7f4803361700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f47fc06fb20 0x7f47fc11eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:36.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.954+0000 7f4803361700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47fc130540 0x7f47fc11f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:36.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.954+0000 7f4803361700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47fc130e50 0x7f47fc11f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:36.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.954+0000 7f4803361700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc12ce50 con 0x7f47fc130e50 2023-10-03T10:16:36.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.954+0000 7f4803361700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc0a0810 con 0x7f47fc06fb20 2023-10-03T10:16:36.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.954+0000 7f4803361700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc120290 con 0x7f47fc130540 2023-10-03T10:16:36.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.955+0000 7f48018fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47fc130e50 0x7f47fc11f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:36.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.955+0000 7f48008fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47fc130540 0x7f47fc11f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:36.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.955+0000 7f48010fd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f47fc06fb20 0x7f47fc11eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:36.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.955+0000 7f48008fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47fc130540 0x7f47fc11f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50494/0 (socket says 172.21.15.3:50494) 2023-10-03T10:16:36.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.955+0000 7f48008fc700 1 -- 172.21.15.3:0/684048437 learned_addr learned my addr 172.21.15.3:0/684048437 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:36.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.955+0000 7f48018fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47fc130e50 0x7f47fc11f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48982/0 (socket says 172.21.15.3:48982) 2023-10-03T10:16:36.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.955+0000 7f48010fd700 1 --2- 172.21.15.3:0/684048437 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f47fc06fb20 0x7f47fc11eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58754/0 (socket says 172.21.15.3:58754) 2023-10-03T10:16:36.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.955+0000 7f48008fc700 1 -- 172.21.15.3:0/684048437 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f47fc06fb20 msgr2=0x7f47fc11eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.955+0000 7f48008fc700 1 --2- 172.21.15.3:0/684048437 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f47fc06fb20 0x7f47fc11eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.955+0000 7f48008fc700 1 -- 172.21.15.3:0/684048437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47fc130e50 msgr2=0x7f47fc11f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.955+0000 7f48008fc700 1 --2- 172.21.15.3:0/684048437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47fc130e50 0x7f47fc11f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.955+0000 7f48008fc700 1 -- 172.21.15.3:0/684048437 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ec00b740 con 0x7f47fc130540 2023-10-03T10:16:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.955+0000 7f48018fe700 1 --2- 172.21.15.3:0/684048437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47fc130e50 0x7f47fc11f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.955+0000 7f48008fc700 1 --2- 172.21.15.3:0/684048437 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47fc130540 0x7f47fc11f400 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00ecf0 tx=0x7f47ec00bf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.955+0000 7f47f1ffb700 1 -- 172.21.15.3:0/684048437 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec00ddc0 con 0x7f47fc130540 2023-10-03T10:16:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.955+0000 7f47f1ffb700 1 -- 172.21.15.3:0/684048437 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ec010440 con 0x7f47fc130540 2023-10-03T10:16:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.956+0000 7f4803361700 1 -- 172.21.15.3:0/684048437 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f80094a0 con 0x7f47fc130540 2023-10-03T10:16:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.956+0000 7f47f1ffb700 1 -- 172.21.15.3:0/684048437 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec0107b0 con 0x7f47fc130540 2023-10-03T10:16:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.956+0000 7f4803361700 1 -- 172.21.15.3:0/684048437 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47fc122150 con 0x7f47fc130540 2023-10-03T10:16:36.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.956+0000 7f47f1ffb700 1 -- 172.21.15.3:0/684048437 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f47ec010910 con 0x7f47fc130540 2023-10-03T10:16:36.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.957+0000 7f47f1ffb700 1 -- 172.21.15.3:0/684048437 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f47ec03fc10 con 0x7f47fc130540 2023-10-03T10:16:36.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.957+0000 7f4803361700 1 -- 172.21.15.3:0/684048437 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47fc131320 con 0x7f47fc130540 2023-10-03T10:16:36.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:36.962+0000 7f47f1ffb700 1 -- 172.21.15.3:0/684048437 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f47ec017770 con 0x7f47fc130540 2023-10-03T10:16:36.964 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:37.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.030+0000 7f5667483700 1 -- 172.21.15.3:0/1027516118 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5660124140 con 0x7f566009f410 2023-10-03T10:16:37.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.031+0000 7f5655ffb700 1 -- 172.21.15.3:0/1027516118 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5660124140 con 0x7f566009f410 2023-10-03T10:16:37.031 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:37.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.033+0000 7f5667483700 1 -- 172.21.15.3:0/1027516118 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f566009f410 msgr2=0x7f56601c8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:37.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.033+0000 7f5667483700 1 --2- 172.21.15.3:0/1027516118 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f566009f410 0x7f56601c8a80 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f565000ebe0 tx=0x7f565000eef0 comp rx=0 tx=0).stop 2023-10-03T10:16:37.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.033+0000 7f5667483700 1 -- 172.21.15.3:0/1027516118 shutdown_connections 2023-10-03T10:16:37.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.033+0000 7f5667483700 1 --2- 172.21.15.3:0/1027516118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5660136fe0 0x7f566006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:37.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.033+0000 7f5667483700 1 --2- 172.21.15.3:0/1027516118 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f566009e930 0x7f56601c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:37.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.033+0000 7f5667483700 1 --2- 172.21.15.3:0/1027516118 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f566009f410 0x7f56601c8a80 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:37.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.033+0000 7f5667483700 1 -- 172.21.15.3:0/1027516118 >> 172.21.15.3:0/1027516118 conn(0x7f56600a4210 msgr2=0x7f56600a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:37.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.033+0000 7f5667483700 1 -- 172.21.15.3:0/1027516118 shutdown_connections 2023-10-03T10:16:37.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.033+0000 7f5667483700 1 -- 172.21.15.3:0/1027516118 wait complete. 2023-10-03T10:16:37.049 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:37.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.058+0000 7f56aa48a700 1 Processor -- start 2023-10-03T10:16:37.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.058+0000 7f56aa48a700 1 -- start start 2023-10-03T10:16:37.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.059+0000 7f56aa48a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f56a406a770 0x7f56a406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.059+0000 7f56aa48a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56a406b080 0x7f56a4131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.059+0000 7f56aa48a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56a4131b40 0x7f56a4131fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.059+0000 7f56aa48a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56a409ece0 con 0x7f56a406a770 2023-10-03T10:16:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.059+0000 7f56aa48a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a409e760 con 0x7f56a406b080 2023-10-03T10:16:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.059+0000 7f56aa48a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56a409e040 con 0x7f56a4131b40 2023-10-03T10:16:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.059+0000 7f56a3fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f56a406a770 0x7f56a406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.059+0000 7f56a8a27700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56a4131b40 0x7f56a4131fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.059+0000 7f56a3fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f56a406a770 0x7f56a406ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48992/0 (socket says 172.21.15.3:48992) 2023-10-03T10:16:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.059+0000 7f56a8a27700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56a4131b40 0x7f56a4131fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50508/0 (socket says 172.21.15.3:50508) 2023-10-03T10:16:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.059+0000 7f56a3fff700 1 -- 172.21.15.3:0/4135396164 learned_addr learned my addr 172.21.15.3:0/4135396164 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.059+0000 7f56a37fe700 1 --2- 172.21.15.3:0/4135396164 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56a406b080 0x7f56a4131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.060+0000 7f56a3fff700 1 -- 172.21.15.3:0/4135396164 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56a4131b40 msgr2=0x7f56a4131fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:37.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.060+0000 7f56a3fff700 1 --2- 172.21.15.3:0/4135396164 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56a4131b40 0x7f56a4131fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:37.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.060+0000 7f56a3fff700 1 -- 172.21.15.3:0/4135396164 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56a406b080 msgr2=0x7f56a4131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:37.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.060+0000 7f56a3fff700 1 --2- 172.21.15.3:0/4135396164 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56a406b080 0x7f56a4131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:37.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.060+0000 7f56a3fff700 1 -- 172.21.15.3:0/4135396164 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a409f630 con 0x7f56a406a770 2023-10-03T10:16:37.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.060+0000 7f56a8a27700 1 --2- 172.21.15.3:0/4135396164 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56a4131b40 0x7f56a4131fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:37.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.060+0000 7f56a37fe700 1 --2- 172.21.15.3:0/4135396164 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56a406b080 0x7f56a4131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:16:37.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.060+0000 7f56a3fff700 1 --2- 172.21.15.3:0/4135396164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56a406a770 0x7f56a406ab40 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f56940098f0 tx=0x7f5694009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7933e8c619205fcd server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:37.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.060+0000 7f56a2ffd700 1 -- 172.21.15.3:0/4135396164 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5694011830 con 0x7f56a406a770 2023-10-03T10:16:37.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.060+0000 7f56a2ffd700 1 -- 172.21.15.3:0/4135396164 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5694011990 con 0x7f56a406a770 2023-10-03T10:16:37.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.060+0000 7f56a2ffd700 1 -- 172.21.15.3:0/4135396164 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5694011d20 con 0x7f56a406a770 2023-10-03T10:16:37.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.061+0000 7f56aa48a700 1 -- 172.21.15.3:0/4135396164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56a406a770 msgr2=0x7f56a406ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:37.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.061+0000 7f56aa48a700 1 --2- 172.21.15.3:0/4135396164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56a406a770 0x7f56a406ab40 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f56940098f0 tx=0x7f5694009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:37.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.061+0000 7f56aa48a700 1 -- 172.21.15.3:0/4135396164 shutdown_connections 2023-10-03T10:16:37.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.061+0000 7f56aa48a700 1 --2- 172.21.15.3:0/4135396164 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56a4131b40 0x7f56a4131fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:37.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.061+0000 7f56aa48a700 1 --2- 172.21.15.3:0/4135396164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56a406a770 0x7f56a406ab40 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:37.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.061+0000 7f56aa48a700 1 --2- 172.21.15.3:0/4135396164 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56a406b080 0x7f56a4131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:37.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.061+0000 7f56aa48a700 1 -- 172.21.15.3:0/4135396164 >> 172.21.15.3:0/4135396164 conn(0x7f56a4066f50 msgr2=0x7f56a409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:37.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.061+0000 7f56aa48a700 1 -- 172.21.15.3:0/4135396164 shutdown_connections 2023-10-03T10:16:37.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.061+0000 7f56aa48a700 1 -- 172.21.15.3:0/4135396164 wait complete. 2023-10-03T10:16:37.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.062+0000 7f56aa48a700 1 Processor -- start 2023-10-03T10:16:37.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.063+0000 7f56aa48a700 1 -- start start 2023-10-03T10:16:37.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.063+0000 7f56aa48a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56a406a770 0x7f56a4120fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:37.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.063+0000 7f56aa48a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56a406b080 0x7f56a41214f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:37.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.063+0000 7f56aa48a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56a4131b40 0x7f56a41c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:37.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.063+0000 7f56aa48a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a4122b00 con 0x7f56a4131b40 2023-10-03T10:16:37.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.063+0000 7f56aa48a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a4071e60 con 0x7f56a406a770 2023-10-03T10:16:37.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.063+0000 7f56aa48a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56a41222a0 con 0x7f56a406b080 2023-10-03T10:16:37.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.063+0000 7f56a37fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56a406b080 0x7f56a41214f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:37.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.063+0000 7f56a37fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56a406b080 0x7f56a41214f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50514/0 (socket says 172.21.15.3:50514) 2023-10-03T10:16:37.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.063+0000 7f56a37fe700 1 -- 172.21.15.3:0/4039208826 learned_addr learned my addr 172.21.15.3:0/4039208826 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:37.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.064+0000 7f56a8a27700 1 --2- 172.21.15.3:0/4039208826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56a4131b40 0x7f56a41c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:37.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.064+0000 7f56a3fff700 1 --2- 172.21.15.3:0/4039208826 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56a406a770 0x7f56a4120fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:37.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.064+0000 7f56a37fe700 1 -- 172.21.15.3:0/4039208826 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56a406a770 msgr2=0x7f56a4120fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:37.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.064+0000 7f56a37fe700 1 --2- 172.21.15.3:0/4039208826 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56a406a770 0x7f56a4120fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:37.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.064+0000 7f56a37fe700 1 -- 172.21.15.3:0/4039208826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56a4131b40 msgr2=0x7f56a41c5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:37.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.064+0000 7f56a37fe700 1 --2- 172.21.15.3:0/4039208826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56a4131b40 0x7f56a41c5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:37.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.064+0000 7f56a37fe700 1 -- 172.21.15.3:0/4039208826 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c009a70 con 0x7f56a406b080 2023-10-03T10:16:37.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.064+0000 7f56a37fe700 1 --2- 172.21.15.3:0/4039208826 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56a406b080 0x7f56a41214f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f569800e810 tx=0x7f569800eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:37.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.064+0000 7f56a0ff9700 1 -- 172.21.15.3:0/4039208826 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5698013610 con 0x7f56a406b080 2023-10-03T10:16:37.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.064+0000 7f56a0ff9700 1 -- 172.21.15.3:0/4039208826 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5698013770 con 0x7f56a406b080 2023-10-03T10:16:37.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.064+0000 7f56aa48a700 1 -- 172.21.15.3:0/4039208826 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5694009550 con 0x7f56a406b080 2023-10-03T10:16:37.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.064+0000 7f56a0ff9700 1 -- 172.21.15.3:0/4039208826 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5698013b20 con 0x7f56a406b080 2023-10-03T10:16:37.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.064+0000 7f56aa48a700 1 -- 172.21.15.3:0/4039208826 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a4122830 con 0x7f56a406b080 2023-10-03T10:16:37.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.065+0000 7f56a0ff9700 1 -- 172.21.15.3:0/4039208826 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5698013c80 con 0x7f56a406b080 2023-10-03T10:16:37.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.065+0000 7f56a0ff9700 1 -- 172.21.15.3:0/4039208826 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f569803f3b0 con 0x7f56a406b080 2023-10-03T10:16:37.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.066+0000 7f56aa48a700 1 -- 172.21.15.3:0/4039208826 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56a4074ea0 con 0x7f56a406b080 2023-10-03T10:16:37.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.071+0000 7f56a0ff9700 1 -- 172.21.15.3:0/4039208826 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5698012370 con 0x7f56a406b080 2023-10-03T10:16:37.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.110+0000 7fb04164f700 1 Processor -- start 2023-10-03T10:16:37.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.110+0000 7fb04164f700 1 -- start start 2023-10-03T10:16:37.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.111+0000 7fb04164f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb03c09e850 0x7fb03c09ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:37.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.111+0000 7fb04164f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb03c09f160 0x7fb03c133c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:37.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.111+0000 7fb04164f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb03c09f5b0 0x7fb03c1341a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:37.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.111+0000 7fb04164f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb03c06bec0 con 0x7fb03c09f160 2023-10-03T10:16:37.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.111+0000 7fb04164f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c0a0030 con 0x7fb03c09e850 2023-10-03T10:16:37.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.111+0000 7fb04164f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c0a0830 con 0x7fb03c09f5b0 2023-10-03T10:16:37.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.111+0000 7fb03affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb03c09e850 0x7fb03c09ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:37.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.111+0000 7fb03affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb03c09e850 0x7fb03c09ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58778/0 (socket says 172.21.15.3:58778) 2023-10-03T10:16:37.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.111+0000 7fb0327fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb03c09f160 0x7fb03c133c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:37.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.111+0000 7fb03affd700 1 -- 172.21.15.3:0/1079177373 learned_addr learned my addr 172.21.15.3:0/1079177373 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:37.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.111+0000 7fb03b7fe700 1 --2- 172.21.15.3:0/1079177373 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb03c09f5b0 0x7fb03c1341a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:37.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.111+0000 7fb0327fc700 1 -- 172.21.15.3:0/1079177373 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb03c09f5b0 msgr2=0x7fb03c1341a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:37.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.111+0000 7fb0327fc700 1 --2- 172.21.15.3:0/1079177373 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb03c09f5b0 0x7fb03c1341a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:37.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.111+0000 7fb0327fc700 1 -- 172.21.15.3:0/1079177373 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb03c09e850 msgr2=0x7fb03c09ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:37.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.111+0000 7fb0327fc700 1 --2- 172.21.15.3:0/1079177373 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb03c09e850 0x7fb03c09ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:37.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.111+0000 7fb0327fc700 1 -- 172.21.15.3:0/1079177373 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c06caa0 con 0x7fb03c09f160 2023-10-03T10:16:37.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.112+0000 7fb03b7fe700 1 --2- 172.21.15.3:0/1079177373 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb03c09f5b0 0x7fb03c1341a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:37.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.112+0000 7fb03affd700 1 --2- 172.21.15.3:0/1079177373 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb03c09e850 0x7fb03c09ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:37.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.112+0000 7fb0327fc700 1 --2- 172.21.15.3:0/1079177373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb03c09f160 0x7fb03c133c60 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fb024009840 tx=0x7fb024009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645b5d87c34a22b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:37.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.112+0000 7fb03a7fc700 1 -- 172.21.15.3:0/1079177373 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb024011830 con 0x7fb03c09f160 2023-10-03T10:16:37.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.112+0000 7fb03a7fc700 1 -- 172.21.15.3:0/1079177373 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb024011990 con 0x7fb03c09f160 2023-10-03T10:16:37.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.112+0000 7fb03a7fc700 1 -- 172.21.15.3:0/1079177373 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb024011d20 con 0x7fb03c09f160 2023-10-03T10:16:37.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.113+0000 7fb04164f700 1 -- 172.21.15.3:0/1079177373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb03c09f160 msgr2=0x7fb03c133c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:37.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.113+0000 7fb04164f700 1 --2- 172.21.15.3:0/1079177373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb03c09f160 0x7fb03c133c60 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fb024009840 tx=0x7fb024009b50 comp rx=0 tx=0).stop 2023-10-03T10:16:37.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.113+0000 7fb04164f700 1 -- 172.21.15.3:0/1079177373 shutdown_connections 2023-10-03T10:16:37.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.113+0000 7fb04164f700 1 --2- 172.21.15.3:0/1079177373 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb03c09f5b0 0x7fb03c1341a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:37.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.113+0000 7fb04164f700 1 --2- 172.21.15.3:0/1079177373 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb03c09e850 0x7fb03c09ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:37.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.113+0000 7fb04164f700 1 --2- 172.21.15.3:0/1079177373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb03c09f160 0x7fb03c133c60 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:37.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.113+0000 7fb04164f700 1 -- 172.21.15.3:0/1079177373 >> 172.21.15.3:0/1079177373 conn(0x7fb03c0a3f70 msgr2=0x7fb03c0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:37.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.113+0000 7fb04164f700 1 -- 172.21.15.3:0/1079177373 shutdown_connections 2023-10-03T10:16:37.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.113+0000 7fb04164f700 1 -- 172.21.15.3:0/1079177373 wait complete. 2023-10-03T10:16:37.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.114+0000 7fb04164f700 1 Processor -- start 2023-10-03T10:16:37.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.114+0000 7fb04164f700 1 -- start start 2023-10-03T10:16:37.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.114+0000 7fb04164f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb03c09e850 0x7fb03c1203f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:37.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.114+0000 7fb04164f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb03c09f160 0x7fb03c120930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:37.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.115+0000 7fb04164f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb03c09f5b0 0x7fb03c120e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:37.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.115+0000 7fb04164f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c123c70 con 0x7fb03c09f5b0 2023-10-03T10:16:37.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.115+0000 7fb04164f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c122d60 con 0x7fb03c09e850 2023-10-03T10:16:37.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.115+0000 7fb04164f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c122000 con 0x7fb03c09f160 2023-10-03T10:16:37.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.115+0000 7fb03affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb03c09e850 0x7fb03c1203f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:37.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.115+0000 7fb03affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb03c09e850 0x7fb03c1203f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58790/0 (socket says 172.21.15.3:58790) 2023-10-03T10:16:37.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.115+0000 7fb03affd700 1 -- 172.21.15.3:0/498625899 learned_addr learned my addr 172.21.15.3:0/498625899 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:37.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.115+0000 7fb03b7fe700 1 --2- 172.21.15.3:0/498625899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb03c09f5b0 0x7fb03c120e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:37.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.115+0000 7fb0327fc700 1 --2- 172.21.15.3:0/498625899 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb03c09f160 0x7fb03c120930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:37.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.115+0000 7fb03affd700 1 -- 172.21.15.3:0/498625899 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb03c09f160 msgr2=0x7fb03c120930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:37.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.115+0000 7fb03affd700 1 --2- 172.21.15.3:0/498625899 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb03c09f160 0x7fb03c120930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:37.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.115+0000 7fb03affd700 1 -- 172.21.15.3:0/498625899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb03c09f5b0 msgr2=0x7fb03c120e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:37.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.115+0000 7fb03affd700 1 --2- 172.21.15.3:0/498625899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb03c09f5b0 0x7fb03c120e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:37.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.115+0000 7fb03affd700 1 -- 172.21.15.3:0/498625899 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb02c009550 con 0x7fb03c09e850 2023-10-03T10:16:37.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.115+0000 7fb03b7fe700 1 --2- 172.21.15.3:0/498625899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb03c09f5b0 0x7fb03c120e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:37.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.115+0000 7fb03affd700 1 --2- 172.21.15.3:0/498625899 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb03c09e850 0x7fb03c1203f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00ebe0 tx=0x7fb02c00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:37.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.115+0000 7fb033fff700 1 -- 172.21.15.3:0/498625899 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c00c850 con 0x7fb03c09e850 2023-10-03T10:16:37.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.116+0000 7fb033fff700 1 -- 172.21.15.3:0/498625899 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02c00c9b0 con 0x7fb03c09e850 2023-10-03T10:16:37.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.116+0000 7fb033fff700 1 -- 172.21.15.3:0/498625899 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c00cd20 con 0x7fb03c09e850 2023-10-03T10:16:37.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.116+0000 7fb04164f700 1 -- 172.21.15.3:0/498625899 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb034009a70 con 0x7fb03c09e850 2023-10-03T10:16:37.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.116+0000 7fb04164f700 1 -- 172.21.15.3:0/498625899 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0240094a0 con 0x7fb03c09e850 2023-10-03T10:16:37.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.117+0000 7fb04164f700 1 -- 172.21.15.3:0/498625899 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb03c06cd20 con 0x7fb03c09e850 2023-10-03T10:16:37.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.117+0000 7fb033fff700 1 -- 172.21.15.3:0/498625899 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb02c013470 con 0x7fb03c09e850 2023-10-03T10:16:37.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.117+0000 7fb033fff700 1 -- 172.21.15.3:0/498625899 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb02c03f7e0 con 0x7fb03c09e850 2023-10-03T10:16:37.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.121+0000 7fb033fff700 1 -- 172.21.15.3:0/498625899 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb02c0232f0 con 0x7fb03c09e850 2023-10-03T10:16:37.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.199+0000 7f4803361700 1 -- 172.21.15.3:0/684048437 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f47fc074ea0 con 0x7f47fc130540 2023-10-03T10:16:37.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.199+0000 7f47f1ffb700 1 -- 172.21.15.3:0/684048437 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f47ec01d020 con 0x7f47fc130540 2023-10-03T10:16:37.200 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:37.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.201+0000 7f4803361700 1 -- 172.21.15.3:0/684048437 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47fc130540 msgr2=0x7f47fc11f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:37.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.201+0000 7f4803361700 1 --2- 172.21.15.3:0/684048437 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47fc130540 0x7f47fc11f400 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00ecf0 tx=0x7f47ec00bf70 comp rx=0 tx=0).stop 2023-10-03T10:16:37.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.202+0000 7f4803361700 1 -- 172.21.15.3:0/684048437 shutdown_connections 2023-10-03T10:16:37.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.202+0000 7f4803361700 1 --2- 172.21.15.3:0/684048437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47fc130e50 0x7f47fc11f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:37.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.202+0000 7f4803361700 1 --2- 172.21.15.3:0/684048437 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f47fc06fb20 0x7f47fc11eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:37.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.202+0000 7f4803361700 1 --2- 172.21.15.3:0/684048437 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f47fc130540 0x7f47fc11f400 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:37.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.202+0000 7f4803361700 1 -- 172.21.15.3:0/684048437 >> 172.21.15.3:0/684048437 conn(0x7f47fc0a2aa0 msgr2=0x7f47fc12d900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:37.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.202+0000 7f4803361700 1 -- 172.21.15.3:0/684048437 shutdown_connections 2023-10-03T10:16:37.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.202+0000 7f4803361700 1 -- 172.21.15.3:0/684048437 wait complete. 2023-10-03T10:16:37.216 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:37.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.300+0000 7f56aa48a700 1 -- 172.21.15.3:0/4039208826 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f56a41315d0 con 0x7f56a406b080 2023-10-03T10:16:37.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.300+0000 7f56a0ff9700 1 -- 172.21.15.3:0/4039208826 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f569802b090 con 0x7f56a406b080 2023-10-03T10:16:37.300 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:37.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.302+0000 7f56aa48a700 1 -- 172.21.15.3:0/4039208826 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56a406b080 msgr2=0x7f56a41214f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:37.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.302+0000 7f56aa48a700 1 --2- 172.21.15.3:0/4039208826 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56a406b080 0x7f56a41214f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f569800e810 tx=0x7f569800eb20 comp rx=0 tx=0).stop 2023-10-03T10:16:37.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.303+0000 7f56aa48a700 1 -- 172.21.15.3:0/4039208826 shutdown_connections 2023-10-03T10:16:37.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.303+0000 7f56aa48a700 1 --2- 172.21.15.3:0/4039208826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56a4131b40 0x7f56a41c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:37.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.303+0000 7f56aa48a700 1 --2- 172.21.15.3:0/4039208826 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56a406a770 0x7f56a4120fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:37.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.303+0000 7f56aa48a700 1 --2- 172.21.15.3:0/4039208826 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56a406b080 0x7f56a41214f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:37.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.303+0000 7f56aa48a700 1 -- 172.21.15.3:0/4039208826 >> 172.21.15.3:0/4039208826 conn(0x7f56a4066f50 msgr2=0x7f56a40700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:37.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.303+0000 7f56aa48a700 1 -- 172.21.15.3:0/4039208826 shutdown_connections 2023-10-03T10:16:37.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.303+0000 7f56aa48a700 1 -- 172.21.15.3:0/4039208826 wait complete. 2023-10-03T10:16:37.317 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:37.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.331+0000 7fb04164f700 1 -- 172.21.15.3:0/498625899 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb03c0747f0 con 0x7fb03c09e850 2023-10-03T10:16:37.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.331+0000 7fb033fff700 1 -- 172.21.15.3:0/498625899 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb02c021020 con 0x7fb03c09e850 2023-10-03T10:16:37.331 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:37.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.334+0000 7fb04164f700 1 -- 172.21.15.3:0/498625899 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb03c09e850 msgr2=0x7fb03c1203f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:37.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.334+0000 7fb04164f700 1 --2- 172.21.15.3:0/498625899 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb03c09e850 0x7fb03c1203f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00ebe0 tx=0x7fb02c00eef0 comp rx=0 tx=0).stop 2023-10-03T10:16:37.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.334+0000 7fb04164f700 1 -- 172.21.15.3:0/498625899 shutdown_connections 2023-10-03T10:16:37.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.334+0000 7fb04164f700 1 --2- 172.21.15.3:0/498625899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb03c09f5b0 0x7fb03c120e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:37.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.334+0000 7fb04164f700 1 --2- 172.21.15.3:0/498625899 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb03c09e850 0x7fb03c1203f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:37.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.334+0000 7fb04164f700 1 --2- 172.21.15.3:0/498625899 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb03c09f160 0x7fb03c120930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:37.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.334+0000 7fb04164f700 1 -- 172.21.15.3:0/498625899 >> 172.21.15.3:0/498625899 conn(0x7fb03c0a3f70 msgr2=0x7fb03c1b97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:37.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.334+0000 7fb04164f700 1 -- 172.21.15.3:0/498625899 shutdown_connections 2023-10-03T10:16:37.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:37.334+0000 7fb04164f700 1 -- 172.21.15.3:0/498625899 wait complete. 2023-10-03T10:16:37.348 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:38.050 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:38.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.195+0000 7f8cb2a2b700 1 Processor -- start 2023-10-03T10:16:38.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.196+0000 7f8cb2a2b700 1 -- start start 2023-10-03T10:16:38.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.196+0000 7f8cb2a2b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8cac09e930 0x7f8cac09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:38.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.196+0000 7f8cb2a2b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cac09f650 0x7f8cac131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:38.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.196+0000 7f8cb2a2b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cac070940 con 0x7f8cac09f240 2023-10-03T10:16:38.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.196+0000 7f8cb2a2b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cac06fd60 con 0x7f8cac09e930 2023-10-03T10:16:38.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.196+0000 7f8cb2a2b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac06fa80 con 0x7f8cac09f650 2023-10-03T10:16:38.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.196+0000 7f8cabfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8cac09e930 0x7f8cac09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:38.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.196+0000 7f8cb0fc8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cac09f650 0x7f8cac131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:38.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.196+0000 7f8cabfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8cac09e930 0x7f8cac09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49032/0 (socket says 172.21.15.3:49032) 2023-10-03T10:16:38.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.196+0000 7f8cb0fc8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cac09f650 0x7f8cac131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50558/0 (socket says 172.21.15.3:50558) 2023-10-03T10:16:38.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.196+0000 7f8cabfff700 1 -- 172.21.15.3:0/2761245804 learned_addr learned my addr 172.21.15.3:0/2761245804 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:38.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.197+0000 7f8cabfff700 1 -- 172.21.15.3:0/2761245804 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cac09f650 msgr2=0x7f8cac131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:38.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.197+0000 7f8cabfff700 1 --2- 172.21.15.3:0/2761245804 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cac09f650 0x7f8cac131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.197+0000 7f8cabfff700 1 -- 172.21.15.3:0/2761245804 >> v1:172.21.15.3:6789/0 conn(0x7f8cac09f240 legacy=0x7f8cac1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:38.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.197+0000 7f8cabfff700 1 -- 172.21.15.3:0/2761245804 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac09fb60 con 0x7f8cac09e930 2023-10-03T10:16:38.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.197+0000 7f8cabfff700 1 --2- 172.21.15.3:0/2761245804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cac09e930 0x7f8cac09ed00 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0098f0 tx=0x7f8c9c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4cfbc93d18fa990 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:38.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.197+0000 7f8caaffd700 1 -- 172.21.15.3:0/2761245804 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c011830 con 0x7f8cac09e930 2023-10-03T10:16:38.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.197+0000 7f8caaffd700 1 -- 172.21.15.3:0/2761245804 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9c011990 con 0x7f8cac09e930 2023-10-03T10:16:38.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.197+0000 7f8caaffd700 1 -- 172.21.15.3:0/2761245804 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c011d20 con 0x7f8cac09e930 2023-10-03T10:16:38.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.197+0000 7f8cb2a2b700 1 -- 172.21.15.3:0/2761245804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cac09e930 msgr2=0x7f8cac09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:38.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.197+0000 7f8cb2a2b700 1 --2- 172.21.15.3:0/2761245804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cac09e930 0x7f8cac09ed00 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0098f0 tx=0x7f8c9c009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:38.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.198+0000 7f8cb2a2b700 1 -- 172.21.15.3:0/2761245804 shutdown_connections 2023-10-03T10:16:38.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.198+0000 7f8cb2a2b700 1 --2- 172.21.15.3:0/2761245804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cac09e930 0x7f8cac09ed00 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.198+0000 7f8cb2a2b700 1 --2- 172.21.15.3:0/2761245804 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cac09f650 0x7f8cac131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.198+0000 7f8cb2a2b700 1 -- 172.21.15.3:0/2761245804 >> 172.21.15.3:0/2761245804 conn(0x7f8cac0a4210 msgr2=0x7f8cac0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:38.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.198+0000 7f8cb2a2b700 1 -- 172.21.15.3:0/2761245804 shutdown_connections 2023-10-03T10:16:38.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.198+0000 7f8cb2a2b700 1 -- 172.21.15.3:0/2761245804 wait complete. 2023-10-03T10:16:38.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.199+0000 7f8cb2a2b700 1 Processor -- start 2023-10-03T10:16:38.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.199+0000 7f8cb2a2b700 1 -- start start 2023-10-03T10:16:38.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.199+0000 7f8cb2a2b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cac09e930 0x7f8cac1a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:38.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.199+0000 7f8cb2a2b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8cac09f240 0x7f8cac1a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:38.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.199+0000 7f8cb2a2b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cac09f650 0x7f8cac1a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:38.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.199+0000 7f8cb2a2b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac1a5220 con 0x7f8cac09e930 2023-10-03T10:16:38.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.199+0000 7f8cb2a2b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac1a5980 con 0x7f8cac09f240 2023-10-03T10:16:38.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.199+0000 7f8cb2a2b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac1a6bd0 con 0x7f8cac09f650 2023-10-03T10:16:38.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.199+0000 7f8cabfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cac09e930 0x7f8cac1a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:38.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.199+0000 7f8cab7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8cac09f240 0x7f8cac1a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:38.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.199+0000 7f8cb0fc8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cac09f650 0x7f8cac1a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:38.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.199+0000 7f8cabfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cac09e930 0x7f8cac1a3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49036/0 (socket says 172.21.15.3:49036) 2023-10-03T10:16:38.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.199+0000 7f8cabfff700 1 -- 172.21.15.3:0/170606891 learned_addr learned my addr 172.21.15.3:0/170606891 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:38.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.200+0000 7f8cabfff700 1 -- 172.21.15.3:0/170606891 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cac09f650 msgr2=0x7f8cac1a47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:38.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.200+0000 7f8cabfff700 1 --2- 172.21.15.3:0/170606891 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cac09f650 0x7f8cac1a47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.200+0000 7f8cabfff700 1 -- 172.21.15.3:0/170606891 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8cac09f240 msgr2=0x7f8cac1a42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:38.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.200+0000 7f8cabfff700 1 --2- 172.21.15.3:0/170606891 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8cac09f240 0x7f8cac1a42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.200+0000 7f8cabfff700 1 -- 172.21.15.3:0/170606891 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9c009550 con 0x7f8cac09e930 2023-10-03T10:16:38.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.200+0000 7f8cabfff700 1 --2- 172.21.15.3:0/170606891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cac09e930 0x7f8cac1a3d70 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009c40 tx=0x7f8c9c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:38.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.200+0000 7f8ca8ff9700 1 -- 172.21.15.3:0/170606891 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c004400 con 0x7f8cac09e930 2023-10-03T10:16:38.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.200+0000 7f8ca8ff9700 1 -- 172.21.15.3:0/170606891 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9c01f070 con 0x7f8cac09e930 2023-10-03T10:16:38.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.200+0000 7f8cb2a2b700 1 -- 172.21.15.3:0/170606891 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cac0a0190 con 0x7f8cac09e930 2023-10-03T10:16:38.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.200+0000 7f8ca8ff9700 1 -- 172.21.15.3:0/170606891 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0038e0 con 0x7f8cac09e930 2023-10-03T10:16:38.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.200+0000 7f8cb2a2b700 1 -- 172.21.15.3:0/170606891 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cac0a3600 con 0x7f8cac09e930 2023-10-03T10:16:38.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.201+0000 7f8ca8ff9700 1 -- 172.21.15.3:0/170606891 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8c9c021030 con 0x7f8cac09e930 2023-10-03T10:16:38.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.201+0000 7f8ca8ff9700 1 -- 172.21.15.3:0/170606891 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8c9c040b40 con 0x7f8cac09e930 2023-10-03T10:16:38.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.201+0000 7f8cb2a2b700 1 -- 172.21.15.3:0/170606891 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cac071d20 con 0x7f8cac09e930 2023-10-03T10:16:38.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.207+0000 7f8ca8ff9700 1 -- 172.21.15.3:0/170606891 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8c9c004560 con 0x7f8cac09e930 2023-10-03T10:16:38.217 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:38.318 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:38.348 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:38.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.356+0000 7f4bcf11b700 1 Processor -- start 2023-10-03T10:16:38.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.356+0000 7f4bcf11b700 1 -- start start 2023-10-03T10:16:38.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.356+0000 7f4bcf11b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bc809d6f0 0x7f4bc809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:38.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.357+0000 7f4bcf11b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4bc809e410 0x7f4bc8134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:38.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.357+0000 7f4bcf11b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc809b120 con 0x7f4bc809e000 2023-10-03T10:16:38.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.357+0000 7f4bcf11b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc806b460 con 0x7f4bc809d6f0 2023-10-03T10:16:38.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.357+0000 7f4bcf11b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc809a7e0 con 0x7f4bc809e410 2023-10-03T10:16:38.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.357+0000 7f4bcceb7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bc809d6f0 0x7f4bc809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:38.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.357+0000 7f4bcceb7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bc809d6f0 0x7f4bc809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58800/0 (socket says 172.21.15.3:58800) 2023-10-03T10:16:38.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.357+0000 7f4bcceb7700 1 -- 172.21.15.3:0/580088096 learned_addr learned my addr 172.21.15.3:0/580088096 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:38.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.357+0000 7f4bcd6b8700 1 --2- 172.21.15.3:0/580088096 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4bc809e410 0x7f4bc8134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:38.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.357+0000 7f4bcceb7700 1 -- 172.21.15.3:0/580088096 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4bc809e410 msgr2=0x7f4bc8134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:38.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.357+0000 7f4bcceb7700 1 --2- 172.21.15.3:0/580088096 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4bc809e410 0x7f4bc8134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.357+0000 7f4bcceb7700 1 -- 172.21.15.3:0/580088096 >> v1:172.21.15.3:6789/0 conn(0x7f4bc809e000 legacy=0x7f4bc81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:38.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.357+0000 7f4bcceb7700 1 -- 172.21.15.3:0/580088096 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc809a500 con 0x7f4bc809d6f0 2023-10-03T10:16:38.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.357+0000 7f4bcceb7700 1 --2- 172.21.15.3:0/580088096 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bc809d6f0 0x7f4bc809dac0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc009920 tx=0x7f4bbc009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81c040a162a7622c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:38.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.358+0000 7f4bc77fe700 1 -- 172.21.15.3:0/580088096 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc004230 con 0x7f4bc809d6f0 2023-10-03T10:16:38.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.358+0000 7f4bc77fe700 1 -- 172.21.15.3:0/580088096 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bbc004390 con 0x7f4bc809d6f0 2023-10-03T10:16:38.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.358+0000 7f4bc77fe700 1 -- 172.21.15.3:0/580088096 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc0036a0 con 0x7f4bc809d6f0 2023-10-03T10:16:38.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.358+0000 7f4bcf11b700 1 -- 172.21.15.3:0/580088096 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bc809d6f0 msgr2=0x7f4bc809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:38.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.358+0000 7f4bcf11b700 1 --2- 172.21.15.3:0/580088096 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bc809d6f0 0x7f4bc809dac0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc009920 tx=0x7f4bbc009c30 comp rx=0 tx=0).stop 2023-10-03T10:16:38.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.359+0000 7f4bcf11b700 1 -- 172.21.15.3:0/580088096 shutdown_connections 2023-10-03T10:16:38.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.359+0000 7f4bcf11b700 1 --2- 172.21.15.3:0/580088096 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bc809d6f0 0x7f4bc809dac0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.359+0000 7f4bcf11b700 1 --2- 172.21.15.3:0/580088096 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4bc809e410 0x7f4bc8134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.359+0000 7f4bcf11b700 1 -- 172.21.15.3:0/580088096 >> 172.21.15.3:0/580088096 conn(0x7f4bc8066f50 msgr2=0x7f4bc809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:38.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.359+0000 7f4bcf11b700 1 -- 172.21.15.3:0/580088096 shutdown_connections 2023-10-03T10:16:38.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.359+0000 7f4bcf11b700 1 -- 172.21.15.3:0/580088096 wait complete. 2023-10-03T10:16:38.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.360+0000 7f4bcf11b700 1 Processor -- start 2023-10-03T10:16:38.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.360+0000 7f4bcf11b700 1 -- start start 2023-10-03T10:16:38.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.361+0000 7f4bcf11b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bc809d6f0 0x7f4bc8128e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:38.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.361+0000 7f4bcf11b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4bc809e000 0x7f4bc81cca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:38.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.361+0000 7f4bcf11b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bc809e410 0x7f4bc81ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:38.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.361+0000 7f4bcf11b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc8133fc0 con 0x7f4bc809d6f0 2023-10-03T10:16:38.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.361+0000 7f4bcf11b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc809c710 con 0x7f4bc809e410 2023-10-03T10:16:38.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.361+0000 7f4bcf11b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc812a630 con 0x7f4bc809e000 2023-10-03T10:16:38.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.361+0000 7f4bcceb7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bc809d6f0 0x7f4bc8128e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:38.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.361+0000 7f4bcceb7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bc809d6f0 0x7f4bc8128e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49046/0 (socket says 172.21.15.3:49046) 2023-10-03T10:16:38.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.361+0000 7f4bcceb7700 1 -- 172.21.15.3:0/2412762613 learned_addr learned my addr 172.21.15.3:0/2412762613 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:38.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.361+0000 7f4bcd6b8700 1 --2- 172.21.15.3:0/2412762613 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bc809e410 0x7f4bc81ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:38.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.361+0000 7f4bcceb7700 1 -- 172.21.15.3:0/2412762613 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4bc809e000 msgr2=0x7f4bc81cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:38.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.361+0000 7f4bcceb7700 1 --2- 172.21.15.3:0/2412762613 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4bc809e000 0x7f4bc81cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.361+0000 7f4bcceb7700 1 -- 172.21.15.3:0/2412762613 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bc809e410 msgr2=0x7f4bc81ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:38.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.361+0000 7f4bcceb7700 1 --2- 172.21.15.3:0/2412762613 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bc809e410 0x7f4bc81ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.361+0000 7f4bcceb7700 1 -- 172.21.15.3:0/2412762613 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bbc009550 con 0x7f4bc809d6f0 2023-10-03T10:16:38.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.362+0000 7f4bcceb7700 1 --2- 172.21.15.3:0/2412762613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bc809d6f0 0x7f4bc8128e60 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc0098f0 tx=0x7f4bbc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:38.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.362+0000 7f4bc57fa700 1 -- 172.21.15.3:0/2412762613 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc003e60 con 0x7f4bc809d6f0 2023-10-03T10:16:38.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.362+0000 7f4bc57fa700 1 -- 172.21.15.3:0/2412762613 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bbc011630 con 0x7f4bc809d6f0 2023-10-03T10:16:38.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.362+0000 7f4bc57fa700 1 -- 172.21.15.3:0/2412762613 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc0119a0 con 0x7f4bc809d6f0 2023-10-03T10:16:38.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.362+0000 7f4bcf11b700 1 -- 172.21.15.3:0/2412762613 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc8129480 con 0x7f4bc809d6f0 2023-10-03T10:16:38.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.362+0000 7f4bcf11b700 1 -- 172.21.15.3:0/2412762613 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc812b2d0 con 0x7f4bc809d6f0 2023-10-03T10:16:38.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.363+0000 7f4bc57fa700 1 -- 172.21.15.3:0/2412762613 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4bbc021070 con 0x7f4bc809d6f0 2023-10-03T10:16:38.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.363+0000 7f4bc57fa700 1 -- 172.21.15.3:0/2412762613 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4bbc040f00 con 0x7f4bc809d6f0 2023-10-03T10:16:38.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.363+0000 7f4bcf11b700 1 -- 172.21.15.3:0/2412762613 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc812a9d0 con 0x7f4bc809d6f0 2023-10-03T10:16:38.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.369+0000 7f4bc57fa700 1 -- 172.21.15.3:0/2412762613 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4bbc029080 con 0x7f4bc809d6f0 2023-10-03T10:16:38.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.422+0000 7f8cb2a2b700 1 -- 172.21.15.3:0/170606891 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8cac074ea0 con 0x7f8cac09e930 2023-10-03T10:16:38.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.422+0000 7f8ca8ff9700 1 -- 172.21.15.3:0/170606891 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8c9c015020 con 0x7f8cac09e930 2023-10-03T10:16:38.423 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:38.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.426+0000 7f8cb2a2b700 1 -- 172.21.15.3:0/170606891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cac09e930 msgr2=0x7f8cac1a3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:38.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.426+0000 7f8cb2a2b700 1 --2- 172.21.15.3:0/170606891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cac09e930 0x7f8cac1a3d70 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009c40 tx=0x7f8c9c004750 comp rx=0 tx=0).stop 2023-10-03T10:16:38.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.426+0000 7f8cb2a2b700 1 -- 172.21.15.3:0/170606891 shutdown_connections 2023-10-03T10:16:38.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.426+0000 7f8cb2a2b700 1 --2- 172.21.15.3:0/170606891 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cac09f650 0x7f8cac1a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.426+0000 7f8cb2a2b700 1 --2- 172.21.15.3:0/170606891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cac09e930 0x7f8cac1a3d70 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.426+0000 7f8cb2a2b700 1 --2- 172.21.15.3:0/170606891 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8cac09f240 0x7f8cac1a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.426+0000 7f8cb2a2b700 1 -- 172.21.15.3:0/170606891 >> 172.21.15.3:0/170606891 conn(0x7f8cac0a4210 msgr2=0x7f8cac1b8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:38.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.427+0000 7f8cb2a2b700 1 -- 172.21.15.3:0/170606891 shutdown_connections 2023-10-03T10:16:38.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.427+0000 7f8cb2a2b700 1 -- 172.21.15.3:0/170606891 wait complete. 2023-10-03T10:16:38.449 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:38.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.463+0000 7f9a9dc6c700 1 Processor -- start 2023-10-03T10:16:38.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.463+0000 7f9a9dc6c700 1 -- start start 2023-10-03T10:16:38.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.464+0000 7f9a9dc6c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a9809d6f0 0x7f9a9809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:38.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.464+0000 7f9a9dc6c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a9809e410 0x7f9a98134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:38.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.464+0000 7f9a9dc6c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a9809b120 con 0x7f9a9809e000 2023-10-03T10:16:38.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.464+0000 7f9a9dc6c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9806b460 con 0x7f9a9809d6f0 2023-10-03T10:16:38.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.464+0000 7f9a9dc6c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9809a7e0 con 0x7f9a9809e410 2023-10-03T10:16:38.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.464+0000 7f9a977fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a9809d6f0 0x7f9a9809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:38.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.464+0000 7f9a977fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a9809d6f0 0x7f9a9809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58822/0 (socket says 172.21.15.3:58822) 2023-10-03T10:16:38.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.464+0000 7f9a977fe700 1 -- 172.21.15.3:0/1817050482 learned_addr learned my addr 172.21.15.3:0/1817050482 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:38.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.464+0000 7f9a97fff700 1 --2- 172.21.15.3:0/1817050482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a9809e410 0x7f9a98134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:38.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.465+0000 7f9a977fe700 1 -- 172.21.15.3:0/1817050482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a9809e410 msgr2=0x7f9a98134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:38.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.465+0000 7f9a977fe700 1 --2- 172.21.15.3:0/1817050482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a9809e410 0x7f9a98134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.465+0000 7f9a977fe700 1 -- 172.21.15.3:0/1817050482 >> v1:172.21.15.3:6789/0 conn(0x7f9a9809e000 legacy=0x7f9a981339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:38.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.465+0000 7f9a977fe700 1 -- 172.21.15.3:0/1817050482 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9809a500 con 0x7f9a9809d6f0 2023-10-03T10:16:38.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.465+0000 7f9a97fff700 1 --2- 172.21.15.3:0/1817050482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a9809e410 0x7f9a98134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:38.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.465+0000 7f9a977fe700 1 --2- 172.21.15.3:0/1817050482 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a9809d6f0 0x7f9a9809dac0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9a88009920 tx=0x7f9a88009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7a4f80c80a1e945 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:38.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.465+0000 7f9a967fc700 1 -- 172.21.15.3:0/1817050482 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a88004230 con 0x7f9a9809d6f0 2023-10-03T10:16:38.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.465+0000 7f9a967fc700 1 -- 172.21.15.3:0/1817050482 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a88004390 con 0x7f9a9809d6f0 2023-10-03T10:16:38.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.465+0000 7f9a967fc700 1 -- 172.21.15.3:0/1817050482 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a880036a0 con 0x7f9a9809d6f0 2023-10-03T10:16:38.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.466+0000 7f9a9dc6c700 1 -- 172.21.15.3:0/1817050482 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a9809d6f0 msgr2=0x7f9a9809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:38.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.466+0000 7f9a9dc6c700 1 --2- 172.21.15.3:0/1817050482 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a9809d6f0 0x7f9a9809dac0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9a88009920 tx=0x7f9a88009c30 comp rx=0 tx=0).stop 2023-10-03T10:16:38.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.466+0000 7f9a9dc6c700 1 -- 172.21.15.3:0/1817050482 shutdown_connections 2023-10-03T10:16:38.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.466+0000 7f9a9dc6c700 1 --2- 172.21.15.3:0/1817050482 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a9809d6f0 0x7f9a9809dac0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.466+0000 7f9a9dc6c700 1 --2- 172.21.15.3:0/1817050482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a9809e410 0x7f9a98134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.466+0000 7f9a9dc6c700 1 -- 172.21.15.3:0/1817050482 >> 172.21.15.3:0/1817050482 conn(0x7f9a98066f50 msgr2=0x7f9a9809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:38.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.466+0000 7f9a9dc6c700 1 -- 172.21.15.3:0/1817050482 shutdown_connections 2023-10-03T10:16:38.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.466+0000 7f9a9dc6c700 1 -- 172.21.15.3:0/1817050482 wait complete. 2023-10-03T10:16:38.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.467+0000 7f9a9dc6c700 1 Processor -- start 2023-10-03T10:16:38.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.467+0000 7f9a9dc6c700 1 -- start start 2023-10-03T10:16:38.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.468+0000 7f9a9dc6c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a9809d6f0 0x7f9a981225c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:38.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.468+0000 7f9a9dc6c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a9809e000 0x7f9a98122b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:38.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.468+0000 7f9a9dc6c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a9809e410 0x7f9a98123040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:38.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.468+0000 7f9a9dc6c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a98125430 con 0x7f9a9809e410 2023-10-03T10:16:38.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.468+0000 7f9a9dc6c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a980a1e20 con 0x7f9a9809d6f0 2023-10-03T10:16:38.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.468+0000 7f9a9dc6c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a981246a0 con 0x7f9a9809e000 2023-10-03T10:16:38.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.468+0000 7f9a977fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a9809d6f0 0x7f9a981225c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:38.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.468+0000 7f9a977fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a9809d6f0 0x7f9a981225c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58824/0 (socket says 172.21.15.3:58824) 2023-10-03T10:16:38.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.468+0000 7f9a977fe700 1 -- 172.21.15.3:0/1952948267 learned_addr learned my addr 172.21.15.3:0/1952948267 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:38.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.468+0000 7f9a97fff700 1 --2- 172.21.15.3:0/1952948267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a9809e410 0x7f9a98123040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:38.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.468+0000 7f9a96ffd700 1 --2- 172.21.15.3:0/1952948267 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a9809e000 0x7f9a98122b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:38.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.469+0000 7f9a977fe700 1 -- 172.21.15.3:0/1952948267 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a9809e000 msgr2=0x7f9a98122b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:38.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.469+0000 7f9a977fe700 1 --2- 172.21.15.3:0/1952948267 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a9809e000 0x7f9a98122b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.469+0000 7f9a977fe700 1 -- 172.21.15.3:0/1952948267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a9809e410 msgr2=0x7f9a98123040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:38.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.469+0000 7f9a977fe700 1 --2- 172.21.15.3:0/1952948267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a9809e410 0x7f9a98123040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.469+0000 7f9a977fe700 1 -- 172.21.15.3:0/1952948267 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a90009a70 con 0x7f9a9809d6f0 2023-10-03T10:16:38.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.469+0000 7f9a97fff700 1 --2- 172.21.15.3:0/1952948267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a9809e410 0x7f9a98123040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:38.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.469+0000 7f9a977fe700 1 --2- 172.21.15.3:0/1952948267 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a9809d6f0 0x7f9a981225c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9a88009130 tx=0x7f9a88004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:38.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.469+0000 7f9a7ffff700 1 -- 172.21.15.3:0/1952948267 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a88003d20 con 0x7f9a9809d6f0 2023-10-03T10:16:38.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.469+0000 7f9a7ffff700 1 -- 172.21.15.3:0/1952948267 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a88003e80 con 0x7f9a9809d6f0 2023-10-03T10:16:38.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.469+0000 7f9a9dc6c700 1 -- 172.21.15.3:0/1952948267 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a88009550 con 0x7f9a9809d6f0 2023-10-03T10:16:38.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.469+0000 7f9a7ffff700 1 -- 172.21.15.3:0/1952948267 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a88011870 con 0x7f9a9809d6f0 2023-10-03T10:16:38.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.469+0000 7f9a9dc6c700 1 -- 172.21.15.3:0/1952948267 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a9809c760 con 0x7f9a9809d6f0 2023-10-03T10:16:38.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.470+0000 7f9a7ffff700 1 -- 172.21.15.3:0/1952948267 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9a88022070 con 0x7f9a9809d6f0 2023-10-03T10:16:38.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.470+0000 7f9a7ffff700 1 -- 172.21.15.3:0/1952948267 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9a88041170 con 0x7f9a9809d6f0 2023-10-03T10:16:38.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.470+0000 7f9a9dc6c700 1 -- 172.21.15.3:0/1952948267 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a98074ea0 con 0x7f9a9809d6f0 2023-10-03T10:16:38.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.480+0000 7f9a7ffff700 1 -- 172.21.15.3:0/1952948267 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9a8801b470 con 0x7f9a9809d6f0 2023-10-03T10:16:38.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.506+0000 7ff2060cd700 1 Processor -- start 2023-10-03T10:16:38.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.506+0000 7ff2060cd700 1 -- start start 2023-10-03T10:16:38.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.507+0000 7ff2060cd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2000a0a30 0x7ff2000a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:38.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.507+0000 7ff2060cd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2000a1340 0x7ff200133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:38.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.507+0000 7ff2060cd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff200133fa0 0x7ff200134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:38.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.507+0000 7ff2060cd700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff20009edd0 con 0x7ff200133fa0 2023-10-03T10:16:38.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.507+0000 7ff2060cd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff20009e1f0 con 0x7ff2000a0a30 2023-10-03T10:16:38.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.507+0000 7ff2060cd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff20009dc50 con 0x7ff2000a1340 2023-10-03T10:16:38.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.507+0000 7ff1ff7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2000a0a30 0x7ff2000a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:38.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.507+0000 7ff1feffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2000a1340 0x7ff200133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:38.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.507+0000 7ff1fffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff200133fa0 0x7ff200134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:38.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.507+0000 7ff1ff7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2000a0a30 0x7ff2000a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58840/0 (socket says 172.21.15.3:58840) 2023-10-03T10:16:38.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.507+0000 7ff1ff7fe700 1 -- 172.21.15.3:0/3284951664 learned_addr learned my addr 172.21.15.3:0/3284951664 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:38.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.507+0000 7ff1ff7fe700 1 -- 172.21.15.3:0/3284951664 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2000a1340 msgr2=0x7ff200133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:38.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.507+0000 7ff1ff7fe700 1 --2- 172.21.15.3:0/3284951664 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2000a1340 0x7ff200133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.507+0000 7ff1ff7fe700 1 -- 172.21.15.3:0/3284951664 >> v2:172.21.15.3:3300/0 conn(0x7ff200133fa0 msgr2=0x7ff200134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:38.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.507+0000 7ff1ff7fe700 1 --2- 172.21.15.3:0/3284951664 >> v2:172.21.15.3:3300/0 conn(0x7ff200133fa0 0x7ff200134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.507+0000 7ff1ff7fe700 1 -- 172.21.15.3:0/3284951664 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2000a1fe0 con 0x7ff2000a0a30 2023-10-03T10:16:38.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.507+0000 7ff1feffd700 1 --2- 172.21.15.3:0/3284951664 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2000a1340 0x7ff200133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:38.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.507+0000 7ff1ff7fe700 1 --2- 172.21.15.3:0/3284951664 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2000a0a30 0x7ff2000a0e00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff1f00098f0 tx=0x7ff1f0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54ff4169b043846d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:38.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.508+0000 7ff1fe7fc700 1 -- 172.21.15.3:0/3284951664 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0011830 con 0x7ff2000a0a30 2023-10-03T10:16:38.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.508+0000 7ff1fe7fc700 1 -- 172.21.15.3:0/3284951664 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f0011990 con 0x7ff2000a0a30 2023-10-03T10:16:38.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.508+0000 7ff1fe7fc700 1 -- 172.21.15.3:0/3284951664 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0011d20 con 0x7ff2000a0a30 2023-10-03T10:16:38.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.509+0000 7ff2060cd700 1 -- 172.21.15.3:0/3284951664 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2000a0a30 msgr2=0x7ff2000a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:38.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.509+0000 7ff2060cd700 1 --2- 172.21.15.3:0/3284951664 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2000a0a30 0x7ff2000a0e00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff1f00098f0 tx=0x7ff1f0009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:38.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.509+0000 7ff2060cd700 1 -- 172.21.15.3:0/3284951664 shutdown_connections 2023-10-03T10:16:38.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.509+0000 7ff2060cd700 1 --2- 172.21.15.3:0/3284951664 >> v2:172.21.15.3:3300/0 conn(0x7ff200133fa0 0x7ff200134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.509+0000 7ff2060cd700 1 --2- 172.21.15.3:0/3284951664 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2000a0a30 0x7ff2000a0e00 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.509+0000 7ff2060cd700 1 --2- 172.21.15.3:0/3284951664 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2000a1340 0x7ff200133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.509+0000 7ff2060cd700 1 -- 172.21.15.3:0/3284951664 >> 172.21.15.3:0/3284951664 conn(0x7ff2000a6550 msgr2=0x7ff2000a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:38.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.509+0000 7ff2060cd700 1 -- 172.21.15.3:0/3284951664 shutdown_connections 2023-10-03T10:16:38.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.509+0000 7ff2060cd700 1 -- 172.21.15.3:0/3284951664 wait complete. 2023-10-03T10:16:38.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.510+0000 7ff2060cd700 1 Processor -- start 2023-10-03T10:16:38.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.510+0000 7ff2060cd700 1 -- start start 2023-10-03T10:16:38.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.511+0000 7ff2060cd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2000a0a30 0x7ff2001201d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:38.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.511+0000 7ff2060cd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2000a1340 0x7ff200120710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:38.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.511+0000 7ff2060cd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff200133fa0 0x7ff200120c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:38.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.511+0000 7ff2060cd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2001245d0 con 0x7ff2000a1340 2023-10-03T10:16:38.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.511+0000 7ff2060cd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff200121da0 con 0x7ff2000a0a30 2023-10-03T10:16:38.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.511+0000 7ff2060cd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff200123930 con 0x7ff200133fa0 2023-10-03T10:16:38.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.511+0000 7ff1ff7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2000a0a30 0x7ff2001201d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:38.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.511+0000 7ff1ff7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2000a0a30 0x7ff2001201d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58848/0 (socket says 172.21.15.3:58848) 2023-10-03T10:16:38.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.511+0000 7ff1ff7fe700 1 -- 172.21.15.3:0/3142813594 learned_addr learned my addr 172.21.15.3:0/3142813594 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:38.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.511+0000 7ff1fffff700 1 --2- 172.21.15.3:0/3142813594 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff200133fa0 0x7ff200120c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:38.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.511+0000 7ff1feffd700 1 --2- 172.21.15.3:0/3142813594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2000a1340 0x7ff200120710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:38.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.512+0000 7ff1fffff700 1 -- 172.21.15.3:0/3142813594 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2000a0a30 msgr2=0x7ff2001201d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:38.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.512+0000 7ff1fffff700 1 --2- 172.21.15.3:0/3142813594 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2000a0a30 0x7ff2001201d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.512+0000 7ff1fffff700 1 -- 172.21.15.3:0/3142813594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2000a1340 msgr2=0x7ff200120710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:38.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.512+0000 7ff1fffff700 1 --2- 172.21.15.3:0/3142813594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2000a1340 0x7ff200120710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.512+0000 7ff1fffff700 1 -- 172.21.15.3:0/3142813594 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f40094a0 con 0x7ff200133fa0 2023-10-03T10:16:38.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.512+0000 7ff1ff7fe700 1 --2- 172.21.15.3:0/3142813594 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2000a0a30 0x7ff2001201d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:38.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.512+0000 7ff1feffd700 1 --2- 172.21.15.3:0/3142813594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2000a1340 0x7ff200120710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:16:38.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.512+0000 7ff1fffff700 1 --2- 172.21.15.3:0/3142813594 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff200133fa0 0x7ff200120c50 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8009d90 tx=0x7ff1f800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:38.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.512+0000 7ff1e7fff700 1 -- 172.21.15.3:0/3142813594 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8009c00 con 0x7ff200133fa0 2023-10-03T10:16:38.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.512+0000 7ff1e7fff700 1 -- 172.21.15.3:0/3142813594 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f800f040 con 0x7ff200133fa0 2023-10-03T10:16:38.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.512+0000 7ff2060cd700 1 -- 172.21.15.3:0/3142813594 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f0009550 con 0x7ff200133fa0 2023-10-03T10:16:38.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.512+0000 7ff1e7fff700 1 -- 172.21.15.3:0/3142813594 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8014480 con 0x7ff200133fa0 2023-10-03T10:16:38.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.512+0000 7ff2060cd700 1 -- 172.21.15.3:0/3142813594 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff200122f50 con 0x7ff200133fa0 2023-10-03T10:16:38.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.513+0000 7ff1e7fff700 1 -- 172.21.15.3:0/3142813594 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff1f80145e0 con 0x7ff200133fa0 2023-10-03T10:16:38.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.513+0000 7ff1e7fff700 1 -- 172.21.15.3:0/3142813594 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff1f80403a0 con 0x7ff200133fa0 2023-10-03T10:16:38.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.514+0000 7ff2060cd700 1 -- 172.21.15.3:0/3142813594 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff200074ea0 con 0x7ff200133fa0 2023-10-03T10:16:38.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.519+0000 7ff1e7fff700 1 -- 172.21.15.3:0/3142813594 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff1f8018070 con 0x7ff200133fa0 2023-10-03T10:16:38.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.593+0000 7f4bcf11b700 1 -- 172.21.15.3:0/2412762613 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4bc8074ea0 con 0x7f4bc809d6f0 2023-10-03T10:16:38.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.593+0000 7f4bc57fa700 1 -- 172.21.15.3:0/2412762613 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4bbc033090 con 0x7f4bc809d6f0 2023-10-03T10:16:38.594 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:38.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.596+0000 7f4bcf11b700 1 -- 172.21.15.3:0/2412762613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bc809d6f0 msgr2=0x7f4bc8128e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:38.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.596+0000 7f4bcf11b700 1 --2- 172.21.15.3:0/2412762613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bc809d6f0 0x7f4bc8128e60 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc0098f0 tx=0x7f4bbc004750 comp rx=0 tx=0).stop 2023-10-03T10:16:38.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.596+0000 7f4bcf11b700 1 -- 172.21.15.3:0/2412762613 shutdown_connections 2023-10-03T10:16:38.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.596+0000 7f4bcf11b700 1 --2- 172.21.15.3:0/2412762613 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bc809e410 0x7f4bc81ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.596+0000 7f4bcf11b700 1 --2- 172.21.15.3:0/2412762613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bc809d6f0 0x7f4bc8128e60 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.596+0000 7f4bcf11b700 1 --2- 172.21.15.3:0/2412762613 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4bc809e000 0x7f4bc81cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.596+0000 7f4bcf11b700 1 -- 172.21.15.3:0/2412762613 >> 172.21.15.3:0/2412762613 conn(0x7f4bc8066f50 msgr2=0x7f4bc80a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:38.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.596+0000 7f4bcf11b700 1 -- 172.21.15.3:0/2412762613 shutdown_connections 2023-10-03T10:16:38.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.596+0000 7f4bcf11b700 1 -- 172.21.15.3:0/2412762613 wait complete. 2023-10-03T10:16:38.614 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:38.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.692+0000 7f9a9dc6c700 1 -- 172.21.15.3:0/1952948267 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9a9806c360 con 0x7f9a9809d6f0 2023-10-03T10:16:38.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.693+0000 7f9a7ffff700 1 -- 172.21.15.3:0/1952948267 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9a88019ba0 con 0x7f9a9809d6f0 2023-10-03T10:16:38.693 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:38.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.695+0000 7f9a9dc6c700 1 -- 172.21.15.3:0/1952948267 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a9809d6f0 msgr2=0x7f9a981225c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:38.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.695+0000 7f9a9dc6c700 1 --2- 172.21.15.3:0/1952948267 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a9809d6f0 0x7f9a981225c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9a88009130 tx=0x7f9a88004750 comp rx=0 tx=0).stop 2023-10-03T10:16:38.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.695+0000 7f9a9dc6c700 1 -- 172.21.15.3:0/1952948267 shutdown_connections 2023-10-03T10:16:38.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.695+0000 7f9a9dc6c700 1 --2- 172.21.15.3:0/1952948267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a9809e410 0x7f9a98123040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.695+0000 7f9a9dc6c700 1 --2- 172.21.15.3:0/1952948267 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a9809d6f0 0x7f9a981225c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.695+0000 7f9a9dc6c700 1 --2- 172.21.15.3:0/1952948267 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a9809e000 0x7f9a98122b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.695+0000 7f9a9dc6c700 1 -- 172.21.15.3:0/1952948267 >> 172.21.15.3:0/1952948267 conn(0x7f9a98066f50 msgr2=0x7f9a9806f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:38.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.695+0000 7f9a9dc6c700 1 -- 172.21.15.3:0/1952948267 shutdown_connections 2023-10-03T10:16:38.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.695+0000 7f9a9dc6c700 1 -- 172.21.15.3:0/1952948267 wait complete. 2023-10-03T10:16:38.709 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:38.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.738+0000 7ff2060cd700 1 -- 172.21.15.3:0/3142813594 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff2000a28e0 con 0x7ff200133fa0 2023-10-03T10:16:38.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.738+0000 7ff1e7fff700 1 -- 172.21.15.3:0/3142813594 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff1f802c090 con 0x7ff200133fa0 2023-10-03T10:16:38.738 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.740+0000 7ff2060cd700 1 -- 172.21.15.3:0/3142813594 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff200133fa0 msgr2=0x7ff200120c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.740+0000 7ff2060cd700 1 --2- 172.21.15.3:0/3142813594 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff200133fa0 0x7ff200120c50 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8009d90 tx=0x7ff1f800c3a0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.741+0000 7ff2060cd700 1 -- 172.21.15.3:0/3142813594 shutdown_connections 2023-10-03T10:16:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.741+0000 7ff2060cd700 1 --2- 172.21.15.3:0/3142813594 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff200133fa0 0x7ff200120c50 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.741+0000 7ff2060cd700 1 --2- 172.21.15.3:0/3142813594 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff2000a0a30 0x7ff2001201d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.741+0000 7ff2060cd700 1 --2- 172.21.15.3:0/3142813594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2000a1340 0x7ff200120710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.741+0000 7ff2060cd700 1 -- 172.21.15.3:0/3142813594 >> 172.21.15.3:0/3142813594 conn(0x7ff2000a6550 msgr2=0x7ff2001bafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.741+0000 7ff2060cd700 1 -- 172.21.15.3:0/3142813594 shutdown_connections 2023-10-03T10:16:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:38.741+0000 7ff2060cd700 1 -- 172.21.15.3:0/3142813594 wait complete. 2023-10-03T10:16:38.754 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:39.450 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:39.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.595+0000 7fe82b187700 1 Processor -- start 2023-10-03T10:16:39.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.595+0000 7fe82b187700 1 -- start start 2023-10-03T10:16:39.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.595+0000 7fe82b187700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe82409d6f0 0x7fe82409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:39.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.595+0000 7fe82b187700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe82409e000 0x7fe824130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:39.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.595+0000 7fe82b187700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe82409f130 con 0x7fe82409e410 2023-10-03T10:16:39.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.595+0000 7fe82b187700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe82409f520 con 0x7fe82409e000 2023-10-03T10:16:39.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.595+0000 7fe82b187700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe82409e930 con 0x7fe82409d6f0 2023-10-03T10:16:39.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.596+0000 7fe828f23700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe82409d6f0 0x7fe82409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:39.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.596+0000 7fe828f23700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe82409d6f0 0x7fe82409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58854/0 (socket says 172.21.15.3:58854) 2023-10-03T10:16:39.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.596+0000 7fe828f23700 1 -- 172.21.15.3:0/255114254 learned_addr learned my addr 172.21.15.3:0/255114254 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:39.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.596+0000 7fe823fff700 1 --2- 172.21.15.3:0/255114254 >> v2:172.21.15.3:3300/0 conn(0x7fe82409e000 0x7fe824130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:39.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.596+0000 7fe828f23700 1 -- 172.21.15.3:0/255114254 >> v2:172.21.15.3:3300/0 conn(0x7fe82409e000 msgr2=0x7fe824130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:39.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.596+0000 7fe828f23700 1 --2- 172.21.15.3:0/255114254 >> v2:172.21.15.3:3300/0 conn(0x7fe82409e000 0x7fe824130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.596+0000 7fe828f23700 1 -- 172.21.15.3:0/255114254 >> v1:172.21.15.3:6789/0 conn(0x7fe82409e410 legacy=0x7fe824130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:39.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.596+0000 7fe828f23700 1 -- 172.21.15.3:0/255114254 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe82409b2d0 con 0x7fe82409d6f0 2023-10-03T10:16:39.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.596+0000 7fe823fff700 1 --2- 172.21.15.3:0/255114254 >> v2:172.21.15.3:3300/0 conn(0x7fe82409e000 0x7fe824130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:39.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.596+0000 7fe828f23700 1 --2- 172.21.15.3:0/255114254 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe82409d6f0 0x7fe82409dac0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe8180098f0 tx=0x7fe818009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=999bec0430daefe9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:39.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.596+0000 7fe8237fe700 1 -- 172.21.15.3:0/255114254 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe818011830 con 0x7fe82409d6f0 2023-10-03T10:16:39.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.596+0000 7fe8237fe700 1 -- 172.21.15.3:0/255114254 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe818011990 con 0x7fe82409d6f0 2023-10-03T10:16:39.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.596+0000 7fe8237fe700 1 -- 172.21.15.3:0/255114254 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe818011d20 con 0x7fe82409d6f0 2023-10-03T10:16:39.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.597+0000 7fe82b187700 1 -- 172.21.15.3:0/255114254 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe82409d6f0 msgr2=0x7fe82409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:39.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.597+0000 7fe82b187700 1 --2- 172.21.15.3:0/255114254 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe82409d6f0 0x7fe82409dac0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe8180098f0 tx=0x7fe818009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:39.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.597+0000 7fe82b187700 1 -- 172.21.15.3:0/255114254 shutdown_connections 2023-10-03T10:16:39.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.597+0000 7fe82b187700 1 --2- 172.21.15.3:0/255114254 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe82409d6f0 0x7fe82409dac0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.597+0000 7fe82b187700 1 --2- 172.21.15.3:0/255114254 >> v2:172.21.15.3:3300/0 conn(0x7fe82409e000 0x7fe824130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.597+0000 7fe82b187700 1 -- 172.21.15.3:0/255114254 >> 172.21.15.3:0/255114254 conn(0x7fe824066f50 msgr2=0x7fe82409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:39.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.597+0000 7fe82b187700 1 -- 172.21.15.3:0/255114254 shutdown_connections 2023-10-03T10:16:39.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.597+0000 7fe82b187700 1 -- 172.21.15.3:0/255114254 wait complete. 2023-10-03T10:16:39.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.598+0000 7fe82b187700 1 Processor -- start 2023-10-03T10:16:39.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.598+0000 7fe82b187700 1 -- start start 2023-10-03T10:16:39.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.599+0000 7fe82b187700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe82409d6f0 0x7fe82412e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:39.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.599+0000 7fe82b187700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe82409e000 0x7fe82412e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:39.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.599+0000 7fe82b187700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe82409e410 0x7fe82412eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:39.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.599+0000 7fe82b187700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82412ff50 con 0x7fe82409e410 2023-10-03T10:16:39.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.599+0000 7fe82b187700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8240a1ce0 con 0x7fe82409d6f0 2023-10-03T10:16:39.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.599+0000 7fe82b187700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe824131120 con 0x7fe82409e000 2023-10-03T10:16:39.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.599+0000 7fe828f23700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe82409d6f0 0x7fe82412e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:39.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.599+0000 7fe828f23700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe82409d6f0 0x7fe82412e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58860/0 (socket says 172.21.15.3:58860) 2023-10-03T10:16:39.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.599+0000 7fe828f23700 1 -- 172.21.15.3:0/3294588515 learned_addr learned my addr 172.21.15.3:0/3294588515 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:39.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.599+0000 7fe829724700 1 --2- 172.21.15.3:0/3294588515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe82409e410 0x7fe82412eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:39.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.599+0000 7fe823fff700 1 --2- 172.21.15.3:0/3294588515 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe82409e000 0x7fe82412e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:39.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.599+0000 7fe829724700 1 -- 172.21.15.3:0/3294588515 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe82409e000 msgr2=0x7fe82412e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:39.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.599+0000 7fe829724700 1 --2- 172.21.15.3:0/3294588515 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe82409e000 0x7fe82412e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.599+0000 7fe829724700 1 -- 172.21.15.3:0/3294588515 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe82409d6f0 msgr2=0x7fe82412e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:39.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.599+0000 7fe829724700 1 --2- 172.21.15.3:0/3294588515 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe82409d6f0 0x7fe82412e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.599+0000 7fe829724700 1 -- 172.21.15.3:0/3294588515 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8100094a0 con 0x7fe82409e410 2023-10-03T10:16:39.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.599+0000 7fe828f23700 1 --2- 172.21.15.3:0/3294588515 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe82409d6f0 0x7fe82412e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:39.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.599+0000 7fe829724700 1 --2- 172.21.15.3:0/3294588515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe82409e410 0x7fe82412eb20 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fe82406dff0 tx=0x7fe81c00ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:39.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.600+0000 7fe8217fa700 1 -- 172.21.15.3:0/3294588515 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c003c00 con 0x7fe82409e410 2023-10-03T10:16:39.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.600+0000 7fe8217fa700 1 -- 172.21.15.3:0/3294588515 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe81c003d60 con 0x7fe82409e410 2023-10-03T10:16:39.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.600+0000 7fe82b187700 1 -- 172.21.15.3:0/3294588515 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe818009550 con 0x7fe82409e410 2023-10-03T10:16:39.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.600+0000 7fe8217fa700 1 -- 172.21.15.3:0/3294588515 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c00a9c0 con 0x7fe82409e410 2023-10-03T10:16:39.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.600+0000 7fe82b187700 1 -- 172.21.15.3:0/3294588515 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe824130710 con 0x7fe82409e410 2023-10-03T10:16:39.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.601+0000 7fe8217fa700 1 -- 172.21.15.3:0/3294588515 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe81c00ab20 con 0x7fe82409e410 2023-10-03T10:16:39.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.601+0000 7fe82b187700 1 -- 172.21.15.3:0/3294588515 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe824074ea0 con 0x7fe82409e410 2023-10-03T10:16:39.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.601+0000 7fe8217fa700 1 -- 172.21.15.3:0/3294588515 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe81c03f5a0 con 0x7fe82409e410 2023-10-03T10:16:39.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.606+0000 7fe8217fa700 1 -- 172.21.15.3:0/3294588515 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe81c00add0 con 0x7fe82409e410 2023-10-03T10:16:39.615 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:39.710 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:39.755 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:39.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.757+0000 7f8177d0b700 1 Processor -- start 2023-10-03T10:16:39.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.757+0000 7f8177d0b700 1 -- start start 2023-10-03T10:16:39.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.757+0000 7f8177d0b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f817009d6f0 0x7f817009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:39.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.758+0000 7f8177d0b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f817009e410 0x7f8170130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:39.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.758+0000 7f8177d0b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f817009f130 con 0x7f817009e000 2023-10-03T10:16:39.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.758+0000 7f8177d0b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817009f520 con 0x7f817009d6f0 2023-10-03T10:16:39.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.758+0000 7f8177d0b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f817009e930 con 0x7f817009e410 2023-10-03T10:16:39.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.758+0000 7f81762a8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f817009e410 0x7f8170130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:39.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.758+0000 7f8175aa7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f817009d6f0 0x7f817009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:39.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.758+0000 7f81762a8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f817009e410 0x7f8170130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50626/0 (socket says 172.21.15.3:50626) 2023-10-03T10:16:39.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.758+0000 7f81762a8700 1 -- 172.21.15.3:0/3204104655 learned_addr learned my addr 172.21.15.3:0/3204104655 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:39.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.758+0000 7f81762a8700 1 -- 172.21.15.3:0/3204104655 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f817009d6f0 msgr2=0x7f817009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:39.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.758+0000 7f81762a8700 1 --2- 172.21.15.3:0/3204104655 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f817009d6f0 0x7f817009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.758+0000 7f81762a8700 1 -- 172.21.15.3:0/3204104655 >> v1:172.21.15.3:6789/0 conn(0x7f817009e000 legacy=0x7f8170130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:39.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.758+0000 7f81762a8700 1 -- 172.21.15.3:0/3204104655 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817009b2d0 con 0x7f817009e410 2023-10-03T10:16:39.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.758+0000 7f8175aa7700 1 --2- 172.21.15.3:0/3204104655 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f817009d6f0 0x7f817009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:39.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.758+0000 7f81762a8700 1 --2- 172.21.15.3:0/3204104655 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f817009e410 0x7f8170130d70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8168009e80 tx=0x7f816800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38b89d785df2ebd0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:39.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.759+0000 7f8174aa5700 1 -- 172.21.15.3:0/3204104655 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816800e070 con 0x7f817009e410 2023-10-03T10:16:39.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.759+0000 7f8174aa5700 1 -- 172.21.15.3:0/3204104655 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8168003a20 con 0x7f817009e410 2023-10-03T10:16:39.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.759+0000 7f8174aa5700 1 -- 172.21.15.3:0/3204104655 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8168003dd0 con 0x7f817009e410 2023-10-03T10:16:39.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.759+0000 7f8177d0b700 1 -- 172.21.15.3:0/3204104655 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f817009e410 msgr2=0x7f8170130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:39.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.759+0000 7f8177d0b700 1 --2- 172.21.15.3:0/3204104655 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f817009e410 0x7f8170130d70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8168009e80 tx=0x7f816800b330 comp rx=0 tx=0).stop 2023-10-03T10:16:39.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.760+0000 7f8177d0b700 1 -- 172.21.15.3:0/3204104655 shutdown_connections 2023-10-03T10:16:39.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.760+0000 7f8177d0b700 1 --2- 172.21.15.3:0/3204104655 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f817009d6f0 0x7f817009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.760+0000 7f8177d0b700 1 --2- 172.21.15.3:0/3204104655 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f817009e410 0x7f8170130d70 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.760+0000 7f8177d0b700 1 -- 172.21.15.3:0/3204104655 >> 172.21.15.3:0/3204104655 conn(0x7f8170066f50 msgr2=0x7f817009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:39.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.760+0000 7f8177d0b700 1 -- 172.21.15.3:0/3204104655 shutdown_connections 2023-10-03T10:16:39.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.760+0000 7f8177d0b700 1 -- 172.21.15.3:0/3204104655 wait complete. 2023-10-03T10:16:39.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.761+0000 7f8177d0b700 1 Processor -- start 2023-10-03T10:16:39.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.761+0000 7f8177d0b700 1 -- start start 2023-10-03T10:16:39.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.761+0000 7f8177d0b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f817009d6f0 0x7f817012ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:39.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.761+0000 7f8177d0b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f817009e000 0x7f817012e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:39.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.761+0000 7f8177d0b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f817009e410 0x7f817012e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:39.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.761+0000 7f8177d0b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8170072430 con 0x7f817009d6f0 2023-10-03T10:16:39.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.761+0000 7f8177d0b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817009c700 con 0x7f817009e410 2023-10-03T10:16:39.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.761+0000 7f8177d0b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8170131f80 con 0x7f817009e000 2023-10-03T10:16:39.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.762+0000 7f8175aa7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f817009d6f0 0x7f817012ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:39.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.762+0000 7f8175aa7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f817009d6f0 0x7f817012ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49094/0 (socket says 172.21.15.3:49094) 2023-10-03T10:16:39.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.762+0000 7f8175aa7700 1 -- 172.21.15.3:0/1881672931 learned_addr learned my addr 172.21.15.3:0/1881672931 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:39.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.762+0000 7f81762a8700 1 --2- 172.21.15.3:0/1881672931 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f817009e410 0x7f817012e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:39.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.762+0000 7f81752a6700 1 --2- 172.21.15.3:0/1881672931 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f817009e000 0x7f817012e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:39.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.762+0000 7f8175aa7700 1 -- 172.21.15.3:0/1881672931 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f817009e000 msgr2=0x7f817012e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:39.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.762+0000 7f8175aa7700 1 --2- 172.21.15.3:0/1881672931 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f817009e000 0x7f817012e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.762+0000 7f8175aa7700 1 -- 172.21.15.3:0/1881672931 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f817009e410 msgr2=0x7f817012e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:39.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.762+0000 7f8175aa7700 1 --2- 172.21.15.3:0/1881672931 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f817009e410 0x7f817012e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.762+0000 7f8175aa7700 1 -- 172.21.15.3:0/1881672931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816c0094a0 con 0x7f817009d6f0 2023-10-03T10:16:39.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.762+0000 7f8175aa7700 1 --2- 172.21.15.3:0/1881672931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f817009d6f0 0x7f817012ddc0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f816c00ebe0 tx=0x7f816c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:39.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.763+0000 7f81667fc700 1 -- 172.21.15.3:0/1881672931 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c009800 con 0x7f817009d6f0 2023-10-03T10:16:39.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.763+0000 7f81667fc700 1 -- 172.21.15.3:0/1881672931 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816c00c910 con 0x7f817009d6f0 2023-10-03T10:16:39.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.763+0000 7f8177d0b700 1 -- 172.21.15.3:0/1881672931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8168009b30 con 0x7f817009d6f0 2023-10-03T10:16:39.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.763+0000 7f81667fc700 1 -- 172.21.15.3:0/1881672931 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c00cc80 con 0x7f817009d6f0 2023-10-03T10:16:39.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.763+0000 7f8177d0b700 1 -- 172.21.15.3:0/1881672931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8170132290 con 0x7f817009d6f0 2023-10-03T10:16:39.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.764+0000 7f81667fc700 1 -- 172.21.15.3:0/1881672931 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f816c013470 con 0x7f817009d6f0 2023-10-03T10:16:39.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.764+0000 7f8177d0b700 1 -- 172.21.15.3:0/1881672931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8170131700 con 0x7f817009d6f0 2023-10-03T10:16:39.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.764+0000 7f81667fc700 1 -- 172.21.15.3:0/1881672931 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f816c03fbd0 con 0x7f817009d6f0 2023-10-03T10:16:39.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.773+0000 7f81667fc700 1 -- 172.21.15.3:0/1881672931 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f816c013720 con 0x7f817009d6f0 2023-10-03T10:16:39.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.825+0000 7fe82b187700 1 -- 172.21.15.3:0/3294588515 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe824131260 con 0x7fe82409e410 2023-10-03T10:16:39.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.825+0000 7fe8217fa700 1 -- 172.21.15.3:0/3294588515 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe81c011380 con 0x7fe82409e410 2023-10-03T10:16:39.826 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:39.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.828+0000 7fe82b187700 1 -- 172.21.15.3:0/3294588515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe82409e410 msgr2=0x7fe82412eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:39.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.828+0000 7fe82b187700 1 --2- 172.21.15.3:0/3294588515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe82409e410 0x7fe82412eb20 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fe82406dff0 tx=0x7fe81c00ce70 comp rx=0 tx=0).stop 2023-10-03T10:16:39.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.828+0000 7fe82b187700 1 -- 172.21.15.3:0/3294588515 shutdown_connections 2023-10-03T10:16:39.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.828+0000 7fe82b187700 1 --2- 172.21.15.3:0/3294588515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe82409e410 0x7fe82412eb20 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.828+0000 7fe82b187700 1 --2- 172.21.15.3:0/3294588515 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe82409d6f0 0x7fe82412e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.828+0000 7fe82b187700 1 --2- 172.21.15.3:0/3294588515 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe82409e000 0x7fe82412e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.828+0000 7fe82b187700 1 -- 172.21.15.3:0/3294588515 >> 172.21.15.3:0/3294588515 conn(0x7fe824066f50 msgr2=0x7fe8240a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:39.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.828+0000 7fe82b187700 1 -- 172.21.15.3:0/3294588515 shutdown_connections 2023-10-03T10:16:39.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.828+0000 7fe82b187700 1 -- 172.21.15.3:0/3294588515 wait complete. 2023-10-03T10:16:39.845 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:39.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.856+0000 7efecdda1700 1 Processor -- start 2023-10-03T10:16:39.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.856+0000 7efecdda1700 1 -- start start 2023-10-03T10:16:39.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.857+0000 7efecdda1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efec809e670 0x7efec809ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:39.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.857+0000 7efecdda1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efec809f8a0 0x7efec809ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:39.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.857+0000 7efecdda1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efec8133960 0x7efec8133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:39.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.857+0000 7efecdda1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efec809f4c0 con 0x7efec809e670 2023-10-03T10:16:39.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.857+0000 7efecdda1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec80a0240 con 0x7efec8133960 2023-10-03T10:16:39.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.857+0000 7efecdda1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efec806c9a0 con 0x7efec809f8a0 2023-10-03T10:16:39.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.857+0000 7efeccd9f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efec809e670 0x7efec809ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:39.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.857+0000 7efeccd9f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efec809e670 0x7efec809ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49098/0 (socket says 172.21.15.3:49098) 2023-10-03T10:16:39.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.857+0000 7efeccd9f700 1 -- 172.21.15.3:0/771958819 learned_addr learned my addr 172.21.15.3:0/771958819 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:39.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.857+0000 7efec7fff700 1 --2- 172.21.15.3:0/771958819 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efec809f8a0 0x7efec809ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:39.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.857+0000 7efecd5a0700 1 --2- 172.21.15.3:0/771958819 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efec8133960 0x7efec8133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:39.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.857+0000 7efeccd9f700 1 -- 172.21.15.3:0/771958819 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efec809f8a0 msgr2=0x7efec809ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:39.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.857+0000 7efeccd9f700 1 --2- 172.21.15.3:0/771958819 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efec809f8a0 0x7efec809ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.857+0000 7efeccd9f700 1 -- 172.21.15.3:0/771958819 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efec8133960 msgr2=0x7efec8133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:39.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.857+0000 7efeccd9f700 1 --2- 172.21.15.3:0/771958819 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efec8133960 0x7efec8133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.857+0000 7efeccd9f700 1 -- 172.21.15.3:0/771958819 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec806b880 con 0x7efec809e670 2023-10-03T10:16:39.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.857+0000 7efeccd9f700 1 --2- 172.21.15.3:0/771958819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efec809e670 0x7efec809ea40 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7efebc0098f0 tx=0x7efebc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d95aba7241bbb83 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:39.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.858+0000 7efec77fe700 1 -- 172.21.15.3:0/771958819 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc011830 con 0x7efec809e670 2023-10-03T10:16:39.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.858+0000 7efec77fe700 1 -- 172.21.15.3:0/771958819 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efebc011990 con 0x7efec809e670 2023-10-03T10:16:39.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.858+0000 7efec77fe700 1 -- 172.21.15.3:0/771958819 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc011d20 con 0x7efec809e670 2023-10-03T10:16:39.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.858+0000 7efecdda1700 1 -- 172.21.15.3:0/771958819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efec809e670 msgr2=0x7efec809ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:39.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.858+0000 7efecdda1700 1 --2- 172.21.15.3:0/771958819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efec809e670 0x7efec809ea40 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7efebc0098f0 tx=0x7efebc009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:39.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.858+0000 7efecdda1700 1 -- 172.21.15.3:0/771958819 shutdown_connections 2023-10-03T10:16:39.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.859+0000 7efecdda1700 1 --2- 172.21.15.3:0/771958819 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efec8133960 0x7efec8133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.859+0000 7efecdda1700 1 --2- 172.21.15.3:0/771958819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efec809e670 0x7efec809ea40 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.859+0000 7efecdda1700 1 --2- 172.21.15.3:0/771958819 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efec809f8a0 0x7efec809ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.859+0000 7efecdda1700 1 -- 172.21.15.3:0/771958819 >> 172.21.15.3:0/771958819 conn(0x7efec80a0c90 msgr2=0x7efec80a1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:39.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.859+0000 7efecdda1700 1 -- 172.21.15.3:0/771958819 shutdown_connections 2023-10-03T10:16:39.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.859+0000 7efecdda1700 1 -- 172.21.15.3:0/771958819 wait complete. 2023-10-03T10:16:39.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.860+0000 7efecdda1700 1 Processor -- start 2023-10-03T10:16:39.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.860+0000 7efecdda1700 1 -- start start 2023-10-03T10:16:39.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.861+0000 7efecdda1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efec809e670 0x7efec81c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:39.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.861+0000 7efecdda1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efec809f8a0 0x7efec81c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:39.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.861+0000 7efecdda1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efec8133960 0x7efec81c48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:39.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.861+0000 7efecdda1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec8123700 con 0x7efec8133960 2023-10-03T10:16:39.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.861+0000 7efecdda1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec8121330 con 0x7efec809e670 2023-10-03T10:16:39.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.861+0000 7efecdda1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efec8122820 con 0x7efec809f8a0 2023-10-03T10:16:39.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.861+0000 7efecd5a0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efec8133960 0x7efec81c48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:39.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.861+0000 7efecd5a0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efec8133960 0x7efec81c48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49112/0 (socket says 172.21.15.3:49112) 2023-10-03T10:16:39.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.861+0000 7efecd5a0700 1 -- 172.21.15.3:0/1804173598 learned_addr learned my addr 172.21.15.3:0/1804173598 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:39.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.861+0000 7efec7fff700 1 --2- 172.21.15.3:0/1804173598 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efec809f8a0 0x7efec81c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:39.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.861+0000 7efeccd9f700 1 --2- 172.21.15.3:0/1804173598 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efec809e670 0x7efec81c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:39.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.862+0000 7efecd5a0700 1 -- 172.21.15.3:0/1804173598 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efec809f8a0 msgr2=0x7efec81c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:39.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.862+0000 7efecd5a0700 1 --2- 172.21.15.3:0/1804173598 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efec809f8a0 0x7efec81c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.862+0000 7efecd5a0700 1 -- 172.21.15.3:0/1804173598 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efec809e670 msgr2=0x7efec81c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:39.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.862+0000 7efecd5a0700 1 --2- 172.21.15.3:0/1804173598 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efec809e670 0x7efec81c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.862+0000 7efecd5a0700 1 -- 172.21.15.3:0/1804173598 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efebc009550 con 0x7efec8133960 2023-10-03T10:16:39.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.862+0000 7efec7fff700 1 --2- 172.21.15.3:0/1804173598 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efec809f8a0 0x7efec81c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:39.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.862+0000 7efeccd9f700 1 --2- 172.21.15.3:0/1804173598 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efec809e670 0x7efec81c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:39.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.862+0000 7efecd5a0700 1 --2- 172.21.15.3:0/1804173598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efec8133960 0x7efec81c48e0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7efec0007e40 tx=0x7efec000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:39.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.862+0000 7efec57fa700 1 -- 172.21.15.3:0/1804173598 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec000ecd0 con 0x7efec8133960 2023-10-03T10:16:39.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.862+0000 7efec57fa700 1 -- 172.21.15.3:0/1804173598 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec000ee50 con 0x7efec8133960 2023-10-03T10:16:39.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.862+0000 7efecdda1700 1 -- 172.21.15.3:0/1804173598 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec8122c50 con 0x7efec8133960 2023-10-03T10:16:39.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.862+0000 7efec57fa700 1 -- 172.21.15.3:0/1804173598 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec0014660 con 0x7efec8133960 2023-10-03T10:16:39.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.862+0000 7efecdda1700 1 -- 172.21.15.3:0/1804173598 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec8124170 con 0x7efec8133960 2023-10-03T10:16:39.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.863+0000 7efec57fa700 1 -- 172.21.15.3:0/1804173598 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efec00147c0 con 0x7efec8133960 2023-10-03T10:16:39.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.863+0000 7efec57fa700 1 -- 172.21.15.3:0/1804173598 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7efec003fdf0 con 0x7efec8133960 2023-10-03T10:16:39.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.864+0000 7efecdda1700 1 -- 172.21.15.3:0/1804173598 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec8123940 con 0x7efec8133960 2023-10-03T10:16:39.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.869+0000 7efec57fa700 1 -- 172.21.15.3:0/1804173598 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efec001bd70 con 0x7efec8133960 2023-10-03T10:16:39.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.902+0000 7feaf568f700 1 Processor -- start 2023-10-03T10:16:39.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.902+0000 7feaf568f700 1 -- start start 2023-10-03T10:16:39.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.902+0000 7feaf568f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaf009c650 0x7feaf009ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:39.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.902+0000 7feaf568f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feaf009cf60 0x7feaf0132080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:39.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.902+0000 7feaf568f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf009ba70 con 0x7feaf00a0090 2023-10-03T10:16:39.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.902+0000 7feaf568f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf009e8f0 con 0x7feaf009cf60 2023-10-03T10:16:39.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.902+0000 7feaf568f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feaf00a0760 con 0x7feaf009c650 2023-10-03T10:16:39.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.903+0000 7feaee7fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7feaf00a0090 0x7feaf0131970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35898/0 (socket says 172.21.15.3:35898) 2023-10-03T10:16:39.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.903+0000 7feaef7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feaf009cf60 0x7feaf0132080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:39.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.903+0000 7feaee7fc700 1 -- 172.21.15.3:0/1727233989 learned_addr learned my addr 172.21.15.3:0/1727233989 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:39.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.903+0000 7feaeeffd700 1 --2- 172.21.15.3:0/1727233989 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaf009c650 0x7feaf009ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:39.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.903+0000 7feaeeffd700 1 -- 172.21.15.3:0/1727233989 >> v2:172.21.15.3:3300/0 conn(0x7feaf009cf60 msgr2=0x7feaf0132080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.903+0000 7feaeeffd700 1 --2- 172.21.15.3:0/1727233989 >> v2:172.21.15.3:3300/0 conn(0x7feaf009cf60 0x7feaf0132080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.903+0000 7feaeeffd700 1 -- 172.21.15.3:0/1727233989 >> v1:172.21.15.3:6789/0 conn(0x7feaf00a0090 legacy=0x7feaf0131970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.903+0000 7feaeeffd700 1 -- 172.21.15.3:0/1727233989 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf009a080 con 0x7feaf009c650 2023-10-03T10:16:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.903+0000 7feaef7fe700 1 --2- 172.21.15.3:0/1727233989 >> v2:172.21.15.3:3300/0 conn(0x7feaf009cf60 0x7feaf0132080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.903+0000 7feaeeffd700 1 --2- 172.21.15.3:0/1727233989 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaf009c650 0x7feaf009ca20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7feae000dc60 tx=0x7feae000df70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54944210d0ab2945 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.903+0000 7feaedffb700 1 -- 172.21.15.3:0/1727233989 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae0012070 con 0x7feaf009c650 2023-10-03T10:16:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.903+0000 7feaedffb700 1 -- 172.21.15.3:0/1727233989 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae000d2e0 con 0x7feaf009c650 2023-10-03T10:16:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.903+0000 7feaedffb700 1 -- 172.21.15.3:0/1727233989 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae000f900 con 0x7feaf009c650 2023-10-03T10:16:39.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.904+0000 7feaf568f700 1 -- 172.21.15.3:0/1727233989 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaf009c650 msgr2=0x7feaf009ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:39.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.904+0000 7feaf568f700 1 --2- 172.21.15.3:0/1727233989 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaf009c650 0x7feaf009ca20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7feae000dc60 tx=0x7feae000df70 comp rx=0 tx=0).stop 2023-10-03T10:16:39.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.904+0000 7feaf568f700 1 -- 172.21.15.3:0/1727233989 shutdown_connections 2023-10-03T10:16:39.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.904+0000 7feaf568f700 1 --2- 172.21.15.3:0/1727233989 >> v2:172.21.15.3:3300/0 conn(0x7feaf009cf60 0x7feaf0132080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.904+0000 7feaf568f700 1 --2- 172.21.15.3:0/1727233989 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaf009c650 0x7feaf009ca20 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.904+0000 7feaf568f700 1 -- 172.21.15.3:0/1727233989 >> 172.21.15.3:0/1727233989 conn(0x7feaf00a3f70 msgr2=0x7feaf00a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:39.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.904+0000 7feaf568f700 1 -- 172.21.15.3:0/1727233989 shutdown_connections 2023-10-03T10:16:39.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.904+0000 7feaf568f700 1 -- 172.21.15.3:0/1727233989 wait complete. 2023-10-03T10:16:39.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.905+0000 7feaf568f700 1 Processor -- start 2023-10-03T10:16:39.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.905+0000 7feaf568f700 1 -- start start 2023-10-03T10:16:39.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.906+0000 7feaf568f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf009c650 0x7feaf011c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:39.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.906+0000 7feaf568f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaf009cf60 0x7feaf011c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:39.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.906+0000 7feaf568f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaf00a0090 0x7feaf011caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:39.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.906+0000 7feaf568f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf012fe90 con 0x7feaf009c650 2023-10-03T10:16:39.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.906+0000 7feaf568f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf011e8f0 con 0x7feaf00a0090 2023-10-03T10:16:39.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.906+0000 7feaf568f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feaf009d980 con 0x7feaf009cf60 2023-10-03T10:16:39.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.906+0000 7feaeeffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf009c650 0x7feaf011c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:39.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.906+0000 7feaeeffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf009c650 0x7feaf011c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49140/0 (socket says 172.21.15.3:49140) 2023-10-03T10:16:39.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.906+0000 7feaeeffd700 1 -- 172.21.15.3:0/1068891760 learned_addr learned my addr 172.21.15.3:0/1068891760 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:39.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.906+0000 7feaee7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaf009cf60 0x7feaf011c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:39.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.906+0000 7feaef7fe700 1 --2- 172.21.15.3:0/1068891760 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaf00a0090 0x7feaf011caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:39.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.906+0000 7feaeeffd700 1 -- 172.21.15.3:0/1068891760 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaf009cf60 msgr2=0x7feaf011c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:39.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.906+0000 7feaeeffd700 1 --2- 172.21.15.3:0/1068891760 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaf009cf60 0x7feaf011c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.906+0000 7feaeeffd700 1 -- 172.21.15.3:0/1068891760 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaf00a0090 msgr2=0x7feaf011caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:39.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.906+0000 7feaeeffd700 1 --2- 172.21.15.3:0/1068891760 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaf00a0090 0x7feaf011caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.906+0000 7feaeeffd700 1 -- 172.21.15.3:0/1068891760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae8009a70 con 0x7feaf009c650 2023-10-03T10:16:39.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.907+0000 7feaeeffd700 1 --2- 172.21.15.3:0/1068891760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf009c650 0x7feaf011c020 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7feaf00a08a0 tx=0x7feae0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:39.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.907+0000 7fead77fe700 1 -- 172.21.15.3:0/1068891760 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae000d7c0 con 0x7feaf009c650 2023-10-03T10:16:39.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.907+0000 7fead77fe700 1 -- 172.21.15.3:0/1068891760 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae00036a0 con 0x7feaf009c650 2023-10-03T10:16:39.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.907+0000 7feaf568f700 1 -- 172.21.15.3:0/1068891760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae000d940 con 0x7feaf009c650 2023-10-03T10:16:39.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.907+0000 7fead77fe700 1 -- 172.21.15.3:0/1068891760 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae0003a10 con 0x7feaf009c650 2023-10-03T10:16:39.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.907+0000 7feaf568f700 1 -- 172.21.15.3:0/1068891760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf011e2f0 con 0x7feaf009c650 2023-10-03T10:16:39.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.908+0000 7feaf568f700 1 -- 172.21.15.3:0/1068891760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf0070fb0 con 0x7feaf009c650 2023-10-03T10:16:39.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.908+0000 7fead77fe700 1 -- 172.21.15.3:0/1068891760 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feae0003b70 con 0x7feaf009c650 2023-10-03T10:16:39.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.908+0000 7fead77fe700 1 -- 172.21.15.3:0/1068891760 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7feae0044910 con 0x7feaf009c650 2023-10-03T10:16:39.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.917+0000 7fead77fe700 1 -- 172.21.15.3:0/1068891760 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feae0015390 con 0x7feaf009c650 2023-10-03T10:16:39.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.985+0000 7f8177d0b700 1 -- 172.21.15.3:0/1881672931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8170074ea0 con 0x7f817009d6f0 2023-10-03T10:16:39.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.986+0000 7f81667fc700 1 -- 172.21.15.3:0/1881672931 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f816c01ab50 con 0x7f817009d6f0 2023-10-03T10:16:39.986 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:39.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.988+0000 7f8177d0b700 1 -- 172.21.15.3:0/1881672931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f817009d6f0 msgr2=0x7f817012ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:39.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.988+0000 7f8177d0b700 1 --2- 172.21.15.3:0/1881672931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f817009d6f0 0x7f817012ddc0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f816c00ebe0 tx=0x7f816c00eef0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.988+0000 7f8177d0b700 1 -- 172.21.15.3:0/1881672931 shutdown_connections 2023-10-03T10:16:39.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.988+0000 7f8177d0b700 1 --2- 172.21.15.3:0/1881672931 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f817009e410 0x7f817012e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.988+0000 7f8177d0b700 1 --2- 172.21.15.3:0/1881672931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f817009d6f0 0x7f817012ddc0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.988+0000 7f8177d0b700 1 --2- 172.21.15.3:0/1881672931 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f817009e000 0x7f817012e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:39.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.988+0000 7f8177d0b700 1 -- 172.21.15.3:0/1881672931 >> 172.21.15.3:0/1881672931 conn(0x7f8170066f50 msgr2=0x7f81700a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:39.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.988+0000 7f8177d0b700 1 -- 172.21.15.3:0/1881672931 shutdown_connections 2023-10-03T10:16:39.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:39.988+0000 7f8177d0b700 1 -- 172.21.15.3:0/1881672931 wait complete. 2023-10-03T10:16:40.002 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:40.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.082+0000 7efecdda1700 1 -- 172.21.15.3:0/1804173598 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7efec8123b10 con 0x7efec8133960 2023-10-03T10:16:40.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.083+0000 7efec57fa700 1 -- 172.21.15.3:0/1804173598 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efec001b310 con 0x7efec8133960 2023-10-03T10:16:40.083 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:40.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.086+0000 7efecdda1700 1 -- 172.21.15.3:0/1804173598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efec8133960 msgr2=0x7efec81c48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:40.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.086+0000 7efecdda1700 1 --2- 172.21.15.3:0/1804173598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efec8133960 0x7efec81c48e0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7efec0007e40 tx=0x7efec000e3c0 comp rx=0 tx=0).stop 2023-10-03T10:16:40.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.086+0000 7efecdda1700 1 -- 172.21.15.3:0/1804173598 shutdown_connections 2023-10-03T10:16:40.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.086+0000 7efecdda1700 1 --2- 172.21.15.3:0/1804173598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efec8133960 0x7efec81c48e0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:40.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.086+0000 7efecdda1700 1 --2- 172.21.15.3:0/1804173598 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efec809e670 0x7efec81c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:40.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.086+0000 7efecdda1700 1 --2- 172.21.15.3:0/1804173598 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efec809f8a0 0x7efec81c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:40.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.086+0000 7efecdda1700 1 -- 172.21.15.3:0/1804173598 >> 172.21.15.3:0/1804173598 conn(0x7efec80a0c90 msgr2=0x7efec81bad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:40.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.086+0000 7efecdda1700 1 -- 172.21.15.3:0/1804173598 shutdown_connections 2023-10-03T10:16:40.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.086+0000 7efecdda1700 1 -- 172.21.15.3:0/1804173598 wait complete. 2023-10-03T10:16:40.105 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:40.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.137+0000 7feaf568f700 1 -- 172.21.15.3:0/1068891760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7feaf00747f0 con 0x7feaf009c650 2023-10-03T10:16:40.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.137+0000 7fead77fe700 1 -- 172.21.15.3:0/1068891760 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feae001fb90 con 0x7feaf009c650 2023-10-03T10:16:40.138 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:40.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.139+0000 7feaf568f700 1 -- 172.21.15.3:0/1068891760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf009c650 msgr2=0x7feaf011c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:40.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.139+0000 7feaf568f700 1 --2- 172.21.15.3:0/1068891760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf009c650 0x7feaf011c020 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7feaf00a08a0 tx=0x7feae0004750 comp rx=0 tx=0).stop 2023-10-03T10:16:40.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.140+0000 7feaf568f700 1 -- 172.21.15.3:0/1068891760 shutdown_connections 2023-10-03T10:16:40.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.140+0000 7feaf568f700 1 --2- 172.21.15.3:0/1068891760 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaf00a0090 0x7feaf011caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:40.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.140+0000 7feaf568f700 1 --2- 172.21.15.3:0/1068891760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf009c650 0x7feaf011c020 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:40.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.140+0000 7feaf568f700 1 --2- 172.21.15.3:0/1068891760 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaf009cf60 0x7feaf011c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:40.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.140+0000 7feaf568f700 1 -- 172.21.15.3:0/1068891760 >> 172.21.15.3:0/1068891760 conn(0x7feaf00a3f70 msgr2=0x7feaf00a36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:40.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.140+0000 7feaf568f700 1 -- 172.21.15.3:0/1068891760 shutdown_connections 2023-10-03T10:16:40.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.140+0000 7feaf568f700 1 -- 172.21.15.3:0/1068891760 wait complete. 2023-10-03T10:16:40.153 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:40.846 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:40.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.993+0000 7f8f71a5b700 1 Processor -- start 2023-10-03T10:16:40.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.993+0000 7f8f71a5b700 1 -- start start 2023-10-03T10:16:40.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.994+0000 7f8f71a5b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8f6c09d630 0x7f8f6c0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:40.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.994+0000 7f8f71a5b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8f6c138d80 0x7f8f6c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:40.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.994+0000 7f8f71a5b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f6c0a2a70 con 0x7f8f6c0a0a30 2023-10-03T10:16:40.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.994+0000 7f8f71a5b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f6c0a01d0 con 0x7f8f6c09d630 2023-10-03T10:16:40.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.994+0000 7f8f71a5b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c09e2b0 con 0x7f8f6c138d80 2023-10-03T10:16:40.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.994+0000 7f8f6a7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8f6c09d630 0x7f8f6c0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:40.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.994+0000 7f8f6b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8f6c138d80 0x7f8f6c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:40.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.994+0000 7f8f6a7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8f6c09d630 0x7f8f6c0a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49142/0 (socket says 172.21.15.3:49142) 2023-10-03T10:16:40.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.994+0000 7f8f6a7fc700 1 -- 172.21.15.3:0/116483582 learned_addr learned my addr 172.21.15.3:0/116483582 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:40.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.994+0000 7f8f6b7fe700 1 -- 172.21.15.3:0/116483582 >> v2:172.21.15.3:3300/0 conn(0x7f8f6c09d630 msgr2=0x7f8f6c0a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:40.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.994+0000 7f8f6b7fe700 1 --2- 172.21.15.3:0/116483582 >> v2:172.21.15.3:3300/0 conn(0x7f8f6c09d630 0x7f8f6c0a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:40.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.994+0000 7f8f6b7fe700 1 -- 172.21.15.3:0/116483582 >> v1:172.21.15.3:6789/0 conn(0x7f8f6c0a0a30 legacy=0x7f8f6c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:40.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.994+0000 7f8f6b7fe700 1 -- 172.21.15.3:0/116483582 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6c09dc50 con 0x7f8f6c138d80 2023-10-03T10:16:40.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.994+0000 7f8f6a7fc700 1 --2- 172.21.15.3:0/116483582 >> v2:172.21.15.3:3300/0 conn(0x7f8f6c09d630 0x7f8f6c0a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:40.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.994+0000 7f8f6b7fe700 1 --2- 172.21.15.3:0/116483582 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8f6c138d80 0x7f8f6c13b160 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009d90 tx=0x7f8f6400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26df420d7b56ab93 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:40.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.995+0000 7f8f69ffb700 1 -- 172.21.15.3:0/116483582 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6400e070 con 0x7f8f6c138d80 2023-10-03T10:16:40.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.995+0000 7f8f69ffb700 1 -- 172.21.15.3:0/116483582 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6400bb20 con 0x7f8f6c138d80 2023-10-03T10:16:40.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.995+0000 7f8f69ffb700 1 -- 172.21.15.3:0/116483582 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f64012430 con 0x7f8f6c138d80 2023-10-03T10:16:40.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.995+0000 7f8f71a5b700 1 -- 172.21.15.3:0/116483582 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8f6c138d80 msgr2=0x7f8f6c13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:40.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.995+0000 7f8f71a5b700 1 --2- 172.21.15.3:0/116483582 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8f6c138d80 0x7f8f6c13b160 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009d90 tx=0x7f8f6400b330 comp rx=0 tx=0).stop 2023-10-03T10:16:40.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.995+0000 7f8f71a5b700 1 -- 172.21.15.3:0/116483582 shutdown_connections 2023-10-03T10:16:40.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.995+0000 7f8f71a5b700 1 --2- 172.21.15.3:0/116483582 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8f6c138d80 0x7f8f6c13b160 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:40.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.995+0000 7f8f71a5b700 1 --2- 172.21.15.3:0/116483582 >> v2:172.21.15.3:3300/0 conn(0x7f8f6c09d630 0x7f8f6c0a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:40.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.995+0000 7f8f71a5b700 1 -- 172.21.15.3:0/116483582 >> 172.21.15.3:0/116483582 conn(0x7f8f6c0a3050 msgr2=0x7f8f6c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:40.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.995+0000 7f8f71a5b700 1 -- 172.21.15.3:0/116483582 shutdown_connections 2023-10-03T10:16:40.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.995+0000 7f8f71a5b700 1 -- 172.21.15.3:0/116483582 wait complete. 2023-10-03T10:16:41.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.996+0000 7f8f71a5b700 1 Processor -- start 2023-10-03T10:16:41.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.997+0000 7f8f71a5b700 1 -- start start 2023-10-03T10:16:41.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.997+0000 7f8f71a5b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f6c09d630 0x7f8f6c120620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:41.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.997+0000 7f8f71a5b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8f6c0a0a30 0x7f8f6c120b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:41.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.997+0000 7f8f71a5b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f6c138d80 0x7f8f6c1210a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:41.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.997+0000 7f8f71a5b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c124140 con 0x7f8f6c138d80 2023-10-03T10:16:41.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.997+0000 7f8f71a5b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c1234f0 con 0x7f8f6c0a0a30 2023-10-03T10:16:41.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.997+0000 7f8f71a5b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c138b20 con 0x7f8f6c09d630 2023-10-03T10:16:41.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.997+0000 7f8f6a7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8f6c0a0a30 0x7f8f6c120b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:41.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.997+0000 7f8f6a7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8f6c0a0a30 0x7f8f6c120b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58920/0 (socket says 172.21.15.3:58920) 2023-10-03T10:16:41.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.997+0000 7f8f6a7fc700 1 -- 172.21.15.3:0/1614770808 learned_addr learned my addr 172.21.15.3:0/1614770808 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:41.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.997+0000 7f8f6b7fe700 1 --2- 172.21.15.3:0/1614770808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f6c138d80 0x7f8f6c1210a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:41.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.997+0000 7f8f6affd700 1 --2- 172.21.15.3:0/1614770808 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f6c09d630 0x7f8f6c120620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:41.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.997+0000 7f8f6a7fc700 1 -- 172.21.15.3:0/1614770808 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f6c09d630 msgr2=0x7f8f6c120620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:41.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.998+0000 7f8f6a7fc700 1 --2- 172.21.15.3:0/1614770808 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f6c09d630 0x7f8f6c120620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.998+0000 7f8f6a7fc700 1 -- 172.21.15.3:0/1614770808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f6c138d80 msgr2=0x7f8f6c1210a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:41.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.998+0000 7f8f6a7fc700 1 --2- 172.21.15.3:0/1614770808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f6c138d80 0x7f8f6c1210a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.998+0000 7f8f6a7fc700 1 -- 172.21.15.3:0/1614770808 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f60009550 con 0x7f8f6c0a0a30 2023-10-03T10:16:41.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.998+0000 7f8f6b7fe700 1 --2- 172.21.15.3:0/1614770808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f6c138d80 0x7f8f6c1210a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:41.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.998+0000 7f8f6a7fc700 1 --2- 172.21.15.3:0/1614770808 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8f6c0a0a30 0x7f8f6c120b60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000ebe0 tx=0x7f8f6000eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:41.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.998+0000 7f8f53fff700 1 -- 172.21.15.3:0/1614770808 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6000c850 con 0x7f8f6c0a0a30 2023-10-03T10:16:41.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.998+0000 7f8f53fff700 1 -- 172.21.15.3:0/1614770808 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6000c9b0 con 0x7f8f6c0a0a30 2023-10-03T10:16:41.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.998+0000 7f8f53fff700 1 -- 172.21.15.3:0/1614770808 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6000cd40 con 0x7f8f6c0a0a30 2023-10-03T10:16:41.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.998+0000 7f8f71a5b700 1 -- 172.21.15.3:0/1614770808 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f64009a70 con 0x7f8f6c0a0a30 2023-10-03T10:16:41.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.998+0000 7f8f71a5b700 1 -- 172.21.15.3:0/1614770808 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6c1219f0 con 0x7f8f6c0a0a30 2023-10-03T10:16:41.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.999+0000 7f8f53fff700 1 -- 172.21.15.3:0/1614770808 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8f60013470 con 0x7f8f6c0a0a30 2023-10-03T10:16:41.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.999+0000 7f8f53fff700 1 -- 172.21.15.3:0/1614770808 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8f6003fc40 con 0x7f8f6c0a0a30 2023-10-03T10:16:41.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:40.999+0000 7f8f71a5b700 1 -- 172.21.15.3:0/1614770808 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f6c124990 con 0x7f8f6c0a0a30 2023-10-03T10:16:41.003 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:41.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.004+0000 7f8f53fff700 1 -- 172.21.15.3:0/1614770808 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8f6001b3b0 con 0x7f8f6c0a0a30 2023-10-03T10:16:41.106 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:41.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.142+0000 7fdcb7f84700 1 Processor -- start 2023-10-03T10:16:41.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.142+0000 7fdcb7f84700 1 -- start start 2023-10-03T10:16:41.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.142+0000 7fdcb7f84700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdcb009e930 0x7fdcb009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:41.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.142+0000 7fdcb7f84700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdcb009f240 0x7fdcb01317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:41.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.142+0000 7fdcb7f84700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdcb009f650 0x7fdcb0131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:41.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.142+0000 7fdcb7f84700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcb0070940 con 0x7fdcb009f650 2023-10-03T10:16:41.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.142+0000 7fdcb7f84700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb006fd60 con 0x7fdcb009e930 2023-10-03T10:16:41.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.142+0000 7fdcb7f84700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb006fa80 con 0x7fdcb009f240 2023-10-03T10:16:41.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.143+0000 7fdcb5d20700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdcb009e930 0x7fdcb009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:41.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.143+0000 7fdcb551f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdcb009f240 0x7fdcb01317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:41.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.143+0000 7fdcb6521700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdcb009f650 0x7fdcb0131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:41.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.143+0000 7fdcb5d20700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdcb009e930 0x7fdcb009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58932/0 (socket says 172.21.15.3:58932) 2023-10-03T10:16:41.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.143+0000 7fdcb5d20700 1 -- 172.21.15.3:0/4120199479 learned_addr learned my addr 172.21.15.3:0/4120199479 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:41.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.143+0000 7fdcb5d20700 1 -- 172.21.15.3:0/4120199479 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdcb009f240 msgr2=0x7fdcb01317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:41.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.143+0000 7fdcb5d20700 1 --2- 172.21.15.3:0/4120199479 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdcb009f240 0x7fdcb01317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.143+0000 7fdcb5d20700 1 -- 172.21.15.3:0/4120199479 >> v2:172.21.15.3:3300/0 conn(0x7fdcb009f650 msgr2=0x7fdcb0131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:41.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.143+0000 7fdcb5d20700 1 --2- 172.21.15.3:0/4120199479 >> v2:172.21.15.3:3300/0 conn(0x7fdcb009f650 0x7fdcb0131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.143+0000 7fdcb5d20700 1 -- 172.21.15.3:0/4120199479 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb009fb60 con 0x7fdcb009e930 2023-10-03T10:16:41.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.143+0000 7fdcb5d20700 1 --2- 172.21.15.3:0/4120199479 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdcb009e930 0x7fdcb009ed00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fdcac0098f0 tx=0x7fdcac009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e059f5d862d250b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:41.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.143+0000 7fdcb4d1e700 1 -- 172.21.15.3:0/4120199479 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac011830 con 0x7fdcb009e930 2023-10-03T10:16:41.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.144+0000 7fdcb4d1e700 1 -- 172.21.15.3:0/4120199479 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcac011990 con 0x7fdcb009e930 2023-10-03T10:16:41.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.144+0000 7fdcb4d1e700 1 -- 172.21.15.3:0/4120199479 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac011d20 con 0x7fdcb009e930 2023-10-03T10:16:41.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.144+0000 7fdcb7f84700 1 -- 172.21.15.3:0/4120199479 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdcb009e930 msgr2=0x7fdcb009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:41.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.144+0000 7fdcb7f84700 1 --2- 172.21.15.3:0/4120199479 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdcb009e930 0x7fdcb009ed00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fdcac0098f0 tx=0x7fdcac009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:41.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.144+0000 7fdcb7f84700 1 -- 172.21.15.3:0/4120199479 shutdown_connections 2023-10-03T10:16:41.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.144+0000 7fdcb7f84700 1 --2- 172.21.15.3:0/4120199479 >> v2:172.21.15.3:3300/0 conn(0x7fdcb009f650 0x7fdcb0131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.144+0000 7fdcb7f84700 1 --2- 172.21.15.3:0/4120199479 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdcb009e930 0x7fdcb009ed00 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.144+0000 7fdcb7f84700 1 --2- 172.21.15.3:0/4120199479 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdcb009f240 0x7fdcb01317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.144+0000 7fdcb7f84700 1 -- 172.21.15.3:0/4120199479 >> 172.21.15.3:0/4120199479 conn(0x7fdcb00a4210 msgr2=0x7fdcb00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:41.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.144+0000 7fdcb7f84700 1 -- 172.21.15.3:0/4120199479 shutdown_connections 2023-10-03T10:16:41.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.144+0000 7fdcb7f84700 1 -- 172.21.15.3:0/4120199479 wait complete. 2023-10-03T10:16:41.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.145+0000 7fdcb7f84700 1 Processor -- start 2023-10-03T10:16:41.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.146+0000 7fdcb7f84700 1 -- start start 2023-10-03T10:16:41.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.146+0000 7fdcb7f84700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdcb009e930 0x7fdcb01a3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:41.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.146+0000 7fdcb7f84700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdcb009f240 0x7fdcb01a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:41.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.146+0000 7fdcb7f84700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcb009f650 0x7fdcb01a4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:41.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.146+0000 7fdcb7f84700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb01a77a0 con 0x7fdcb009f650 2023-10-03T10:16:41.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.146+0000 7fdcb7f84700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb01b72b0 con 0x7fdcb009e930 2023-10-03T10:16:41.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.146+0000 7fdcb7f84700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb01a6d20 con 0x7fdcb009f240 2023-10-03T10:16:41.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.146+0000 7fdcb5d20700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdcb009e930 0x7fdcb01a3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:41.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.146+0000 7fdcb5d20700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdcb009e930 0x7fdcb01a3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58940/0 (socket says 172.21.15.3:58940) 2023-10-03T10:16:41.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.146+0000 7fdcb5d20700 1 -- 172.21.15.3:0/48061488 learned_addr learned my addr 172.21.15.3:0/48061488 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:41.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.146+0000 7fdcb6521700 1 --2- 172.21.15.3:0/48061488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcb009f650 0x7fdcb01a4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:41.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.146+0000 7fdcb5d20700 1 -- 172.21.15.3:0/48061488 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdcb009f240 msgr2=0x7fdcb01a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:41.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.146+0000 7fdcb5d20700 1 --2- 172.21.15.3:0/48061488 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdcb009f240 0x7fdcb01a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.146+0000 7fdcb5d20700 1 -- 172.21.15.3:0/48061488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcb009f650 msgr2=0x7fdcb01a4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:41.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.146+0000 7fdcb5d20700 1 --2- 172.21.15.3:0/48061488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcb009f650 0x7fdcb01a4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.146+0000 7fdcb5d20700 1 -- 172.21.15.3:0/48061488 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcac009550 con 0x7fdcb009e930 2023-10-03T10:16:41.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.147+0000 7fdcb5d20700 1 --2- 172.21.15.3:0/48061488 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdcb009e930 0x7fdcb01a3f10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fdcac000c00 tx=0x7fdcac004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:41.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.147+0000 7fdca67fc700 1 -- 172.21.15.3:0/48061488 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac004240 con 0x7fdcb009e930 2023-10-03T10:16:41.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.147+0000 7fdca67fc700 1 -- 172.21.15.3:0/48061488 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcac01f070 con 0x7fdcb009e930 2023-10-03T10:16:41.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.147+0000 7fdcb7f84700 1 -- 172.21.15.3:0/48061488 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb00a0190 con 0x7fdcb009e930 2023-10-03T10:16:41.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.147+0000 7fdca67fc700 1 -- 172.21.15.3:0/48061488 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac0045a0 con 0x7fdcb009e930 2023-10-03T10:16:41.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.147+0000 7fdcb7f84700 1 -- 172.21.15.3:0/48061488 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb0071d20 con 0x7fdcb009e930 2023-10-03T10:16:41.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.148+0000 7fdca67fc700 1 -- 172.21.15.3:0/48061488 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdcac021030 con 0x7fdcb009e930 2023-10-03T10:16:41.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.148+0000 7fdca67fc700 1 -- 172.21.15.3:0/48061488 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fdcac0408b0 con 0x7fdcb009e930 2023-10-03T10:16:41.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.149+0000 7fdc93fff700 1 -- 172.21.15.3:0/48061488 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc8c0052e0 con 0x7fdcb009e930 2023-10-03T10:16:41.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.154+0000 7fdca67fc700 1 -- 172.21.15.3:0/48061488 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdcac015080 con 0x7fdcb009e930 2023-10-03T10:16:41.154 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:41.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.227+0000 7f8f71a5b700 1 -- 172.21.15.3:0/1614770808 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8f6c123dc0 con 0x7f8f6c0a0a30 2023-10-03T10:16:41.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.227+0000 7f8f53fff700 1 -- 172.21.15.3:0/1614770808 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8f6c123dc0 con 0x7f8f6c0a0a30 2023-10-03T10:16:41.228 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:41.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.231+0000 7f8f71a5b700 1 -- 172.21.15.3:0/1614770808 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8f6c0a0a30 msgr2=0x7f8f6c120b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:41.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.231+0000 7f8f71a5b700 1 --2- 172.21.15.3:0/1614770808 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8f6c0a0a30 0x7f8f6c120b60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000ebe0 tx=0x7f8f6000eef0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.231+0000 7f8f71a5b700 1 -- 172.21.15.3:0/1614770808 shutdown_connections 2023-10-03T10:16:41.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.231+0000 7f8f71a5b700 1 --2- 172.21.15.3:0/1614770808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f6c138d80 0x7f8f6c1210a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.231+0000 7f8f71a5b700 1 --2- 172.21.15.3:0/1614770808 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f6c09d630 0x7f8f6c120620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.231+0000 7f8f71a5b700 1 --2- 172.21.15.3:0/1614770808 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8f6c0a0a30 0x7f8f6c120b60 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.231+0000 7f8f71a5b700 1 -- 172.21.15.3:0/1614770808 >> 172.21.15.3:0/1614770808 conn(0x7f8f6c0a3050 msgr2=0x7f8f6c139150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:41.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.231+0000 7f8f71a5b700 1 -- 172.21.15.3:0/1614770808 shutdown_connections 2023-10-03T10:16:41.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.231+0000 7f8f71a5b700 1 -- 172.21.15.3:0/1614770808 wait complete. 2023-10-03T10:16:41.252 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:41.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.253+0000 7f0b49e09700 1 Processor -- start 2023-10-03T10:16:41.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.253+0000 7f0b49e09700 1 -- start start 2023-10-03T10:16:41.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.254+0000 7f0b49e09700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b4409d6f0 0x7f0b4409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:41.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.254+0000 7f0b49e09700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b4409e410 0x7f0b44130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:41.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.254+0000 7f0b49e09700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b4409f130 con 0x7f0b4409e000 2023-10-03T10:16:41.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.254+0000 7f0b49e09700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4409f520 con 0x7f0b4409d6f0 2023-10-03T10:16:41.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.254+0000 7f0b49e09700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4409e930 con 0x7f0b4409e410 2023-10-03T10:16:41.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.254+0000 7f0b43fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b4409e410 0x7f0b44130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:41.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.254+0000 7f0b437fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b4409d6f0 0x7f0b4409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:41.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.254+0000 7f0b43fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b4409e410 0x7f0b44130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50696/0 (socket says 172.21.15.3:50696) 2023-10-03T10:16:41.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.254+0000 7f0b43fff700 1 -- 172.21.15.3:0/3376299270 learned_addr learned my addr 172.21.15.3:0/3376299270 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:41.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.254+0000 7f0b437fe700 1 -- 172.21.15.3:0/3376299270 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b4409e410 msgr2=0x7f0b44130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:41.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.254+0000 7f0b437fe700 1 --2- 172.21.15.3:0/3376299270 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b4409e410 0x7f0b44130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.254+0000 7f0b437fe700 1 -- 172.21.15.3:0/3376299270 >> v1:172.21.15.3:6789/0 conn(0x7f0b4409e000 legacy=0x7f0b44130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:41.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.254+0000 7f0b437fe700 1 -- 172.21.15.3:0/3376299270 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4409b2d0 con 0x7f0b4409d6f0 2023-10-03T10:16:41.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.254+0000 7f0b43fff700 1 --2- 172.21.15.3:0/3376299270 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b4409e410 0x7f0b44130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:41.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.254+0000 7f0b437fe700 1 --2- 172.21.15.3:0/3376299270 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b4409d6f0 0x7f0b4409dac0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0b340098a0 tx=0x7f0b34009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=729a9d8b3f51aaa0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:41.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.255+0000 7f0b427fc700 1 -- 172.21.15.3:0/3376299270 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b34011830 con 0x7f0b4409d6f0 2023-10-03T10:16:41.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.255+0000 7f0b427fc700 1 -- 172.21.15.3:0/3376299270 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b34011990 con 0x7f0b4409d6f0 2023-10-03T10:16:41.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.255+0000 7f0b427fc700 1 -- 172.21.15.3:0/3376299270 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b34011d20 con 0x7f0b4409d6f0 2023-10-03T10:16:41.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.255+0000 7f0b49e09700 1 -- 172.21.15.3:0/3376299270 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b4409d6f0 msgr2=0x7f0b4409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:41.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.255+0000 7f0b49e09700 1 --2- 172.21.15.3:0/3376299270 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b4409d6f0 0x7f0b4409dac0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0b340098a0 tx=0x7f0b34009bb0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.255+0000 7f0b49e09700 1 -- 172.21.15.3:0/3376299270 shutdown_connections 2023-10-03T10:16:41.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.255+0000 7f0b49e09700 1 --2- 172.21.15.3:0/3376299270 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b4409d6f0 0x7f0b4409dac0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.255+0000 7f0b49e09700 1 --2- 172.21.15.3:0/3376299270 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b4409e410 0x7f0b44130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.255+0000 7f0b49e09700 1 -- 172.21.15.3:0/3376299270 >> 172.21.15.3:0/3376299270 conn(0x7f0b44066f50 msgr2=0x7f0b4409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:41.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.255+0000 7f0b49e09700 1 -- 172.21.15.3:0/3376299270 shutdown_connections 2023-10-03T10:16:41.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.256+0000 7f0b49e09700 1 -- 172.21.15.3:0/3376299270 wait complete. 2023-10-03T10:16:41.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.257+0000 7f0b49e09700 1 Processor -- start 2023-10-03T10:16:41.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.257+0000 7f0b49e09700 1 -- start start 2023-10-03T10:16:41.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.258+0000 7f0b49e09700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b4409d6f0 0x7f0b44139940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:41.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.258+0000 7f0b49e09700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b4409e000 0x7f0b44139e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:41.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.258+0000 7f0b49e09700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b4409e410 0x7f0b4413a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:41.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.258+0000 7f0b49e09700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b44136d00 con 0x7f0b4409d6f0 2023-10-03T10:16:41.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.258+0000 7f0b49e09700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b44139300 con 0x7f0b4409e410 2023-10-03T10:16:41.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.258+0000 7f0b49e09700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b44072590 con 0x7f0b4409e000 2023-10-03T10:16:41.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.258+0000 7f0b437fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b4409d6f0 0x7f0b44139940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:41.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.258+0000 7f0b437fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b4409d6f0 0x7f0b44139940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49184/0 (socket says 172.21.15.3:49184) 2023-10-03T10:16:41.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.258+0000 7f0b437fe700 1 -- 172.21.15.3:0/861410780 learned_addr learned my addr 172.21.15.3:0/861410780 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:41.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.258+0000 7f0b43fff700 1 --2- 172.21.15.3:0/861410780 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b4409e410 0x7f0b4413a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:41.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.258+0000 7f0b42ffd700 1 --2- 172.21.15.3:0/861410780 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b4409e000 0x7f0b44139e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:41.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.259+0000 7f0b437fe700 1 -- 172.21.15.3:0/861410780 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b4409e000 msgr2=0x7f0b44139e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:41.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.259+0000 7f0b437fe700 1 --2- 172.21.15.3:0/861410780 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b4409e000 0x7f0b44139e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.259+0000 7f0b437fe700 1 -- 172.21.15.3:0/861410780 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b4409e410 msgr2=0x7f0b4413a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:41.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.259+0000 7f0b437fe700 1 --2- 172.21.15.3:0/861410780 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b4409e410 0x7f0b4413a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.259+0000 7f0b437fe700 1 -- 172.21.15.3:0/861410780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c009b30 con 0x7f0b4409d6f0 2023-10-03T10:16:41.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.259+0000 7f0b43fff700 1 --2- 172.21.15.3:0/861410780 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b4409e410 0x7f0b4413a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:41.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.259+0000 7f0b437fe700 1 --2- 172.21.15.3:0/861410780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b4409d6f0 0x7f0b44139940 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f0b340090b0 tx=0x7f0b34004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:41.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.259+0000 7f0b2bfff700 1 -- 172.21.15.3:0/861410780 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b34004450 con 0x7f0b4409d6f0 2023-10-03T10:16:41.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.259+0000 7f0b2bfff700 1 -- 172.21.15.3:0/861410780 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3401f070 con 0x7f0b4409d6f0 2023-10-03T10:16:41.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.259+0000 7f0b49e09700 1 -- 172.21.15.3:0/861410780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b340094a0 con 0x7f0b4409d6f0 2023-10-03T10:16:41.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.259+0000 7f0b2bfff700 1 -- 172.21.15.3:0/861410780 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b340038e0 con 0x7f0b4409d6f0 2023-10-03T10:16:41.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.259+0000 7f0b49e09700 1 -- 172.21.15.3:0/861410780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b44136760 con 0x7f0b4409d6f0 2023-10-03T10:16:41.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.260+0000 7f0b2bfff700 1 -- 172.21.15.3:0/861410780 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0b34003a40 con 0x7f0b4409d6f0 2023-10-03T10:16:41.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.260+0000 7f0b2bfff700 1 -- 172.21.15.3:0/861410780 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0b34021070 con 0x7f0b4409d6f0 2023-10-03T10:16:41.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.260+0000 7f0b49e09700 1 -- 172.21.15.3:0/861410780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b44131290 con 0x7f0b4409d6f0 2023-10-03T10:16:41.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.266+0000 7f0b2bfff700 1 -- 172.21.15.3:0/861410780 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0b44131290 con 0x7f0b4409d6f0 2023-10-03T10:16:41.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.315+0000 7f7b86291700 1 Processor -- start 2023-10-03T10:16:41.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.315+0000 7f7b86291700 1 -- start start 2023-10-03T10:16:41.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.316+0000 7f7b86291700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b8009e930 0x7f7b8009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:41.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.316+0000 7f7b86291700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b8009f240 0x7f7b801317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:41.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.316+0000 7f7b86291700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b80070940 con 0x7f7b8009f650 2023-10-03T10:16:41.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.316+0000 7f7b86291700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8006fd60 con 0x7f7b8009f240 2023-10-03T10:16:41.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.316+0000 7f7b86291700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8006fa80 con 0x7f7b8009e930 2023-10-03T10:16:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.316+0000 7f7b7ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b8009e930 0x7f7b8009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.316+0000 7f7b7f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b8009f240 0x7f7b801317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.316+0000 7f7b7ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b8009e930 0x7f7b8009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50716/0 (socket says 172.21.15.3:50716) 2023-10-03T10:16:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.316+0000 7f7b7ffff700 1 -- 172.21.15.3:0/3798773422 learned_addr learned my addr 172.21.15.3:0/3798773422 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.316+0000 7f7b8482e700 1 --1- 172.21.15.3:0/3798773422 >> v1:172.21.15.3:6789/0 conn(0x7f7b8009f650 0x7f7b80131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35928/0 (socket says 172.21.15.3:35928) 2023-10-03T10:16:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.316+0000 7f7b7effd700 1 -- 172.21.15.3:0/3798773422 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3411011756 0 0) 0x7f7b80070940 con 0x7f7b8009f650 2023-10-03T10:16:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.316+0000 7f7b7f7fe700 1 -- 172.21.15.3:0/3798773422 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b8009e930 msgr2=0x7f7b8009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.316+0000 7f7b7f7fe700 1 --2- 172.21.15.3:0/3798773422 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b8009e930 0x7f7b8009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.316+0000 7f7b7f7fe700 1 -- 172.21.15.3:0/3798773422 >> v1:172.21.15.3:6789/0 conn(0x7f7b8009f650 legacy=0x7f7b80131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:41.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.316+0000 7f7b7f7fe700 1 -- 172.21.15.3:0/3798773422 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b8009fb60 con 0x7f7b8009f240 2023-10-03T10:16:41.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.317+0000 7f7b7ffff700 1 --2- 172.21.15.3:0/3798773422 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b8009e930 0x7f7b8009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:41.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.317+0000 7f7b7f7fe700 1 --2- 172.21.15.3:0/3798773422 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b8009f240 0x7f7b801317f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7b74009870 tx=0x7f7b74009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5487e69efa4d6444 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:41.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.317+0000 7f7b7effd700 1 -- 172.21.15.3:0/3798773422 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b74011830 con 0x7f7b8009f240 2023-10-03T10:16:41.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.317+0000 7f7b7effd700 1 -- 172.21.15.3:0/3798773422 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b74011990 con 0x7f7b8009f240 2023-10-03T10:16:41.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.317+0000 7f7b7effd700 1 -- 172.21.15.3:0/3798773422 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b74011d20 con 0x7f7b8009f240 2023-10-03T10:16:41.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.317+0000 7f7b86291700 1 -- 172.21.15.3:0/3798773422 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b8009f240 msgr2=0x7f7b801317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:41.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.317+0000 7f7b86291700 1 --2- 172.21.15.3:0/3798773422 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b8009f240 0x7f7b801317f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7b74009870 tx=0x7f7b74009b80 comp rx=0 tx=0).stop 2023-10-03T10:16:41.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.318+0000 7f7b86291700 1 -- 172.21.15.3:0/3798773422 shutdown_connections 2023-10-03T10:16:41.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.318+0000 7f7b86291700 1 --2- 172.21.15.3:0/3798773422 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b8009e930 0x7f7b8009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.318+0000 7f7b86291700 1 --2- 172.21.15.3:0/3798773422 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b8009f240 0x7f7b801317f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.318+0000 7f7b86291700 1 -- 172.21.15.3:0/3798773422 >> 172.21.15.3:0/3798773422 conn(0x7f7b800a4210 msgr2=0x7f7b800a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:41.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.318+0000 7f7b86291700 1 -- 172.21.15.3:0/3798773422 shutdown_connections 2023-10-03T10:16:41.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.318+0000 7f7b86291700 1 -- 172.21.15.3:0/3798773422 wait complete. 2023-10-03T10:16:41.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.319+0000 7f7b86291700 1 Processor -- start 2023-10-03T10:16:41.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.319+0000 7f7b86291700 1 -- start start 2023-10-03T10:16:41.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.320+0000 7f7b86291700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b8009e930 0x7f7b8006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:41.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.320+0000 7f7b86291700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b8009f240 0x7f7b8006bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:41.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.320+0000 7f7b86291700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b8009f650 0x7f7b8006c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.320+0000 7f7b86291700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8006cd60 con 0x7f7b8009f650 2023-10-03T10:16:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.320+0000 7f7b86291700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8006d430 con 0x7f7b8009e930 2023-10-03T10:16:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.320+0000 7f7b86291700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7b801a92c0 con 0x7f7b8009f240 2023-10-03T10:16:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.320+0000 7f7b7f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b8009f240 0x7f7b8006bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.320+0000 7f7b7f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b8009f240 0x7f7b8006bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50720/0 (socket says 172.21.15.3:50720) 2023-10-03T10:16:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.320+0000 7f7b7f7fe700 1 -- 172.21.15.3:0/1856159957 learned_addr learned my addr 172.21.15.3:0/1856159957 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.320+0000 7f7b7ffff700 1 --2- 172.21.15.3:0/1856159957 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b8009e930 0x7f7b8006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.320+0000 7f7b8482e700 1 --2- 172.21.15.3:0/1856159957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b8009f650 0x7f7b8006c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.321+0000 7f7b7f7fe700 1 -- 172.21.15.3:0/1856159957 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b8009e930 msgr2=0x7f7b8006b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.321+0000 7f7b7f7fe700 1 --2- 172.21.15.3:0/1856159957 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b8009e930 0x7f7b8006b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.321+0000 7f7b7f7fe700 1 -- 172.21.15.3:0/1856159957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b8009f650 msgr2=0x7f7b8006c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:41.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.321+0000 7f7b7f7fe700 1 --2- 172.21.15.3:0/1856159957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b8009f650 0x7f7b8006c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.321+0000 7f7b7f7fe700 1 -- 172.21.15.3:0/1856159957 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b70009550 con 0x7f7b8009f240 2023-10-03T10:16:41.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.321+0000 7f7b8482e700 1 --2- 172.21.15.3:0/1856159957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b8009f650 0x7f7b8006c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:16:41.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.321+0000 7f7b7f7fe700 1 --2- 172.21.15.3:0/1856159957 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b8009f240 0x7f7b8006bdf0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7b740090b0 tx=0x7f7b74004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:41.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.321+0000 7f7b7cff9700 1 -- 172.21.15.3:0/1856159957 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b740045d0 con 0x7f7b8009f240 2023-10-03T10:16:41.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.321+0000 7f7b7cff9700 1 -- 172.21.15.3:0/1856159957 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7401f070 con 0x7f7b8009f240 2023-10-03T10:16:41.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.321+0000 7f7b7cff9700 1 -- 172.21.15.3:0/1856159957 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b740038c0 con 0x7f7b8009f240 2023-10-03T10:16:41.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.321+0000 7f7b86291700 1 -- 172.21.15.3:0/1856159957 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b740094a0 con 0x7f7b8009f240 2023-10-03T10:16:41.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.321+0000 7f7b86291700 1 -- 172.21.15.3:0/1856159957 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b800a0190 con 0x7f7b8009f240 2023-10-03T10:16:41.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.322+0000 7f7b86291700 1 -- 172.21.15.3:0/1856159957 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b801a0b30 con 0x7f7b8009f240 2023-10-03T10:16:41.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.322+0000 7f7b7cff9700 1 -- 172.21.15.3:0/1856159957 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7b74003a20 con 0x7f7b8009f240 2023-10-03T10:16:41.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.323+0000 7f7b7cff9700 1 -- 172.21.15.3:0/1856159957 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7b74021070 con 0x7f7b8009f240 2023-10-03T10:16:41.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.327+0000 7f7b7cff9700 1 -- 172.21.15.3:0/1856159957 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7b7402f3a0 con 0x7f7b8009f240 2023-10-03T10:16:41.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.372+0000 7fdc93fff700 1 -- 172.21.15.3:0/48061488 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdc8c005590 con 0x7fdcb009e930 2023-10-03T10:16:41.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.373+0000 7fdca67fc700 1 -- 172.21.15.3:0/48061488 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdcac01a450 con 0x7fdcb009e930 2023-10-03T10:16:41.373 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:41.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.375+0000 7fdc93fff700 1 -- 172.21.15.3:0/48061488 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdcb009e930 msgr2=0x7fdcb01a3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:41.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.375+0000 7fdc93fff700 1 --2- 172.21.15.3:0/48061488 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdcb009e930 0x7fdcb01a3f10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fdcac000c00 tx=0x7fdcac004750 comp rx=0 tx=0).stop 2023-10-03T10:16:41.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.375+0000 7fdc93fff700 1 -- 172.21.15.3:0/48061488 shutdown_connections 2023-10-03T10:16:41.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.375+0000 7fdc93fff700 1 --2- 172.21.15.3:0/48061488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcb009f650 0x7fdcb01a4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.375+0000 7fdc93fff700 1 --2- 172.21.15.3:0/48061488 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdcb009e930 0x7fdcb01a3f10 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.375+0000 7fdc93fff700 1 --2- 172.21.15.3:0/48061488 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdcb009f240 0x7fdcb01a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.375+0000 7fdc93fff700 1 -- 172.21.15.3:0/48061488 >> 172.21.15.3:0/48061488 conn(0x7fdcb00a4210 msgr2=0x7fdcb01b8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:41.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.375+0000 7fdc93fff700 1 -- 172.21.15.3:0/48061488 shutdown_connections 2023-10-03T10:16:41.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.375+0000 7fdc93fff700 1 -- 172.21.15.3:0/48061488 wait complete. 2023-10-03T10:16:41.390 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:41.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.482+0000 7f0b49e09700 1 -- 172.21.15.3:0/861410780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0b44131290 con 0x7f0b4409d6f0 2023-10-03T10:16:41.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.482+0000 7f0b2bfff700 1 -- 172.21.15.3:0/861410780 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0b44131290 con 0x7f0b4409d6f0 2023-10-03T10:16:41.483 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:41.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.485+0000 7f0b49e09700 1 -- 172.21.15.3:0/861410780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b4409d6f0 msgr2=0x7f0b44139940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:41.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.485+0000 7f0b49e09700 1 --2- 172.21.15.3:0/861410780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b4409d6f0 0x7f0b44139940 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f0b340090b0 tx=0x7f0b34004750 comp rx=0 tx=0).stop 2023-10-03T10:16:41.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.485+0000 7f0b49e09700 1 -- 172.21.15.3:0/861410780 shutdown_connections 2023-10-03T10:16:41.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.485+0000 7f0b49e09700 1 --2- 172.21.15.3:0/861410780 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b4409e410 0x7f0b4413a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.485+0000 7f0b49e09700 1 --2- 172.21.15.3:0/861410780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b4409d6f0 0x7f0b44139940 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.485+0000 7f0b49e09700 1 --2- 172.21.15.3:0/861410780 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b4409e000 0x7f0b44139e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.485+0000 7f0b49e09700 1 -- 172.21.15.3:0/861410780 >> 172.21.15.3:0/861410780 conn(0x7f0b44066f50 msgr2=0x7f0b440a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:41.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.485+0000 7f0b49e09700 1 -- 172.21.15.3:0/861410780 shutdown_connections 2023-10-03T10:16:41.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.485+0000 7f0b49e09700 1 -- 172.21.15.3:0/861410780 wait complete. 2023-10-03T10:16:41.499 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:41.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.535+0000 7f7b86291700 1 -- 172.21.15.3:0/1856159957 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7b80071d20 con 0x7f7b8009f240 2023-10-03T10:16:41.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.536+0000 7f7b7cff9700 1 -- 172.21.15.3:0/1856159957 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7b74015070 con 0x7f7b8009f240 2023-10-03T10:16:41.536 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:41.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.538+0000 7f7b86291700 1 -- 172.21.15.3:0/1856159957 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b8009f240 msgr2=0x7f7b8006bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:41.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.538+0000 7f7b86291700 1 --2- 172.21.15.3:0/1856159957 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b8009f240 0x7f7b8006bdf0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7b740090b0 tx=0x7f7b74004750 comp rx=0 tx=0).stop 2023-10-03T10:16:41.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.538+0000 7f7b86291700 1 -- 172.21.15.3:0/1856159957 shutdown_connections 2023-10-03T10:16:41.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.538+0000 7f7b86291700 1 --2- 172.21.15.3:0/1856159957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b8009f650 0x7f7b8006c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.538+0000 7f7b86291700 1 --2- 172.21.15.3:0/1856159957 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b8009e930 0x7f7b8006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.538+0000 7f7b86291700 1 --2- 172.21.15.3:0/1856159957 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b8009f240 0x7f7b8006bdf0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:41.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.538+0000 7f7b86291700 1 -- 172.21.15.3:0/1856159957 >> 172.21.15.3:0/1856159957 conn(0x7f7b800a4210 msgr2=0x7f7b801b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:41.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.538+0000 7f7b86291700 1 -- 172.21.15.3:0/1856159957 shutdown_connections 2023-10-03T10:16:41.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:41.538+0000 7f7b86291700 1 -- 172.21.15.3:0/1856159957 wait complete. 2023-10-03T10:16:41.552 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:42.252 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:42.390 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:42.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.398+0000 7ff1d7797700 1 Processor -- start 2023-10-03T10:16:42.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.398+0000 7ff1d7797700 1 -- start start 2023-10-03T10:16:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.399+0000 7ff1d7797700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1d009e930 0x7ff1d009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.399+0000 7ff1d7797700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff1d009f240 0x7ff1d01317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.399+0000 7ff1d7797700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d0070940 con 0x7ff1d009f650 2023-10-03T10:16:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.399+0000 7ff1d7797700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1d006fd60 con 0x7ff1d009f240 2023-10-03T10:16:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.399+0000 7ff1d7797700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d006fa80 con 0x7ff1d009e930 2023-10-03T10:16:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.399+0000 7ff1d5533700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1d009e930 0x7ff1d009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.399+0000 7ff1d4d32700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff1d009f240 0x7ff1d01317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.399+0000 7ff1d5533700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1d009e930 0x7ff1d009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50734/0 (socket says 172.21.15.3:50734) 2023-10-03T10:16:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.399+0000 7ff1d4d32700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff1d009f240 0x7ff1d01317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49200/0 (socket says 172.21.15.3:49200) 2023-10-03T10:16:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.399+0000 7ff1d5533700 1 -- 172.21.15.3:0/516120462 learned_addr learned my addr 172.21.15.3:0/516120462 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.399+0000 7ff1d5533700 1 -- 172.21.15.3:0/516120462 >> v2:172.21.15.3:3300/0 conn(0x7ff1d009f240 msgr2=0x7ff1d01317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.399+0000 7ff1d5533700 1 --2- 172.21.15.3:0/516120462 >> v2:172.21.15.3:3300/0 conn(0x7ff1d009f240 0x7ff1d01317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.399+0000 7ff1d5533700 1 -- 172.21.15.3:0/516120462 >> v1:172.21.15.3:6789/0 conn(0x7ff1d009f650 legacy=0x7ff1d0131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.399+0000 7ff1d5533700 1 -- 172.21.15.3:0/516120462 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d009fb60 con 0x7ff1d009e930 2023-10-03T10:16:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.399+0000 7ff1d4d32700 1 --2- 172.21.15.3:0/516120462 >> v2:172.21.15.3:3300/0 conn(0x7ff1d009f240 0x7ff1d01317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.400+0000 7ff1d5533700 1 --2- 172.21.15.3:0/516120462 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1d009e930 0x7ff1d009ed00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0098f0 tx=0x7ff1cc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fc2950300806091 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.400+0000 7ff1c7fff700 1 -- 172.21.15.3:0/516120462 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc011830 con 0x7ff1d009e930 2023-10-03T10:16:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.400+0000 7ff1c7fff700 1 -- 172.21.15.3:0/516120462 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1cc011990 con 0x7ff1d009e930 2023-10-03T10:16:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.400+0000 7ff1c7fff700 1 -- 172.21.15.3:0/516120462 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc011d20 con 0x7ff1d009e930 2023-10-03T10:16:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.401+0000 7ff1d7797700 1 -- 172.21.15.3:0/516120462 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1d009e930 msgr2=0x7ff1d009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.401+0000 7ff1d7797700 1 --2- 172.21.15.3:0/516120462 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1d009e930 0x7ff1d009ed00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0098f0 tx=0x7ff1cc009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.401+0000 7ff1d7797700 1 -- 172.21.15.3:0/516120462 shutdown_connections 2023-10-03T10:16:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.401+0000 7ff1d7797700 1 --2- 172.21.15.3:0/516120462 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1d009e930 0x7ff1d009ed00 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.401+0000 7ff1d7797700 1 --2- 172.21.15.3:0/516120462 >> v2:172.21.15.3:3300/0 conn(0x7ff1d009f240 0x7ff1d01317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.401+0000 7ff1d7797700 1 -- 172.21.15.3:0/516120462 >> 172.21.15.3:0/516120462 conn(0x7ff1d00a4210 msgr2=0x7ff1d00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.401+0000 7ff1d7797700 1 -- 172.21.15.3:0/516120462 shutdown_connections 2023-10-03T10:16:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.401+0000 7ff1d7797700 1 -- 172.21.15.3:0/516120462 wait complete. 2023-10-03T10:16:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.402+0000 7ff1d7797700 1 Processor -- start 2023-10-03T10:16:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.402+0000 7ff1d7797700 1 -- start start 2023-10-03T10:16:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.403+0000 7ff1d7797700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1d009e930 0x7ff1d01a7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.403+0000 7ff1d7797700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1d009f240 0x7ff1d01a7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.403+0000 7ff1d7797700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1d009f650 0x7ff1d006b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.403+0000 7ff1d7797700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d01a5150 con 0x7ff1d009f240 2023-10-03T10:16:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.403+0000 7ff1d7797700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d01a5720 con 0x7ff1d009e930 2023-10-03T10:16:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.403+0000 7ff1d7797700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d01a5330 con 0x7ff1d009f650 2023-10-03T10:16:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.404+0000 7ff1d5533700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1d009e930 0x7ff1d01a7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.404+0000 7ff1d5533700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1d009e930 0x7ff1d01a7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58984/0 (socket says 172.21.15.3:58984) 2023-10-03T10:16:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.404+0000 7ff1d5d34700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1d009f650 0x7ff1d006b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.404+0000 7ff1d5533700 1 -- 172.21.15.3:0/3059962730 learned_addr learned my addr 172.21.15.3:0/3059962730 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.404+0000 7ff1d4d32700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1d009f240 0x7ff1d01a7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.404+0000 7ff1d5d34700 1 -- 172.21.15.3:0/3059962730 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1d009e930 msgr2=0x7ff1d01a7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:42.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.404+0000 7ff1d5d34700 1 --2- 172.21.15.3:0/3059962730 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1d009e930 0x7ff1d01a7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.404+0000 7ff1d5d34700 1 -- 172.21.15.3:0/3059962730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1d009f240 msgr2=0x7ff1d01a7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:42.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.404+0000 7ff1d5d34700 1 --2- 172.21.15.3:0/3059962730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1d009f240 0x7ff1d01a7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.404+0000 7ff1d5d34700 1 -- 172.21.15.3:0/3059962730 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c00094a0 con 0x7ff1d009f650 2023-10-03T10:16:42.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.404+0000 7ff1d5533700 1 --2- 172.21.15.3:0/3059962730 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1d009e930 0x7ff1d01a7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:42.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.404+0000 7ff1d4d32700 1 --2- 172.21.15.3:0/3059962730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1d009f240 0x7ff1d01a7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:42.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.404+0000 7ff1d5d34700 1 --2- 172.21.15.3:0/3059962730 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1d009f650 0x7ff1d006b880 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800cbc0 tx=0x7ff1c800ced0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:42.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.404+0000 7ff1c5ffb700 1 -- 172.21.15.3:0/3059962730 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8003c00 con 0x7ff1d009f650 2023-10-03T10:16:42.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.404+0000 7ff1c5ffb700 1 -- 172.21.15.3:0/3059962730 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c8003d60 con 0x7ff1d009f650 2023-10-03T10:16:42.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.404+0000 7ff1c5ffb700 1 -- 172.21.15.3:0/3059962730 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c800aa40 con 0x7ff1d009f650 2023-10-03T10:16:42.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.404+0000 7ff1d7797700 1 -- 172.21.15.3:0/3059962730 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1cc009550 con 0x7ff1d009f650 2023-10-03T10:16:42.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.405+0000 7ff1d7797700 1 -- 172.21.15.3:0/3059962730 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d00a0190 con 0x7ff1d009f650 2023-10-03T10:16:42.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.405+0000 7ff1c5ffb700 1 -- 172.21.15.3:0/3059962730 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff1c800aba0 con 0x7ff1d009f650 2023-10-03T10:16:42.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.406+0000 7ff1c5ffb700 1 -- 172.21.15.3:0/3059962730 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff1c803f640 con 0x7ff1d009f650 2023-10-03T10:16:42.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.406+0000 7ff1d7797700 1 -- 172.21.15.3:0/3059962730 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1d0074ea0 con 0x7ff1d009f650 2023-10-03T10:16:42.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.411+0000 7ff1c5ffb700 1 -- 172.21.15.3:0/3059962730 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff1c8012420 con 0x7ff1d009f650 2023-10-03T10:16:42.500 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:42.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.532+0000 7fd9cec76700 1 Processor -- start 2023-10-03T10:16:42.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.533+0000 7fd9cec76700 1 -- start start 2023-10-03T10:16:42.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.533+0000 7fd9cec76700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd9c809d6f0 0x7fd9c809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:42.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.533+0000 7fd9cec76700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9c809e000 0x7fd9c81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:42.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.533+0000 7fd9cec76700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9c809e410 0x7fd9c8133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:42.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.533+0000 7fd9cec76700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9c806b460 con 0x7fd9c809d6f0 2023-10-03T10:16:42.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.533+0000 7fd9cec76700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c809b120 con 0x7fd9c809e410 2023-10-03T10:16:42.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.533+0000 7fd9cec76700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c809a7e0 con 0x7fd9c809e000 2023-10-03T10:16:42.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.533+0000 7fd9cd213700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9c809e410 0x7fd9c8133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:42.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.533+0000 7fd9c7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9c809e000 0x7fd9c81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:42.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.533+0000 7fd9cd213700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9c809e410 0x7fd9c8133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:58996/0 (socket says 172.21.15.3:58996) 2023-10-03T10:16:42.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.533+0000 7fd9cd213700 1 -- 172.21.15.3:0/606401097 learned_addr learned my addr 172.21.15.3:0/606401097 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:42.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.533+0000 7fd9cca12700 1 --2- 172.21.15.3:0/606401097 >> v2:172.21.15.3:3300/0 conn(0x7fd9c809d6f0 0x7fd9c809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:42.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.534+0000 7fd9c7fff700 1 -- 172.21.15.3:0/606401097 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9c809e410 msgr2=0x7fd9c8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:42.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.534+0000 7fd9c7fff700 1 --2- 172.21.15.3:0/606401097 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9c809e410 0x7fd9c8133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.534+0000 7fd9c7fff700 1 -- 172.21.15.3:0/606401097 >> v2:172.21.15.3:3300/0 conn(0x7fd9c809d6f0 msgr2=0x7fd9c809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:42.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.534+0000 7fd9c7fff700 1 --2- 172.21.15.3:0/606401097 >> v2:172.21.15.3:3300/0 conn(0x7fd9c809d6f0 0x7fd9c809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.534+0000 7fd9c7fff700 1 -- 172.21.15.3:0/606401097 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c809a500 con 0x7fd9c809e000 2023-10-03T10:16:42.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.534+0000 7fd9cd213700 1 --2- 172.21.15.3:0/606401097 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9c809e410 0x7fd9c8133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:42.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.534+0000 7fd9c7fff700 1 --2- 172.21.15.3:0/606401097 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9c809e000 0x7fd9c81339f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009870 tx=0x7fd9b4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdc06ae1f053eede server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:42.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.534+0000 7fd9c77fe700 1 -- 172.21.15.3:0/606401097 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4011830 con 0x7fd9c809e000 2023-10-03T10:16:42.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.534+0000 7fd9c77fe700 1 -- 172.21.15.3:0/606401097 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b4011990 con 0x7fd9c809e000 2023-10-03T10:16:42.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.534+0000 7fd9c77fe700 1 -- 172.21.15.3:0/606401097 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4011d20 con 0x7fd9c809e000 2023-10-03T10:16:42.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.535+0000 7fd9cec76700 1 -- 172.21.15.3:0/606401097 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9c809e000 msgr2=0x7fd9c81339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:42.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.535+0000 7fd9cec76700 1 --2- 172.21.15.3:0/606401097 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9c809e000 0x7fd9c81339f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009870 tx=0x7fd9b4009b80 comp rx=0 tx=0).stop 2023-10-03T10:16:42.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.535+0000 7fd9cec76700 1 -- 172.21.15.3:0/606401097 shutdown_connections 2023-10-03T10:16:42.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.535+0000 7fd9cec76700 1 --2- 172.21.15.3:0/606401097 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9c809e410 0x7fd9c8133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.535+0000 7fd9cec76700 1 --2- 172.21.15.3:0/606401097 >> v2:172.21.15.3:3300/0 conn(0x7fd9c809d6f0 0x7fd9c809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.535+0000 7fd9cec76700 1 --2- 172.21.15.3:0/606401097 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9c809e000 0x7fd9c81339f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.535+0000 7fd9cec76700 1 -- 172.21.15.3:0/606401097 >> 172.21.15.3:0/606401097 conn(0x7fd9c8066f50 msgr2=0x7fd9c809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:42.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.535+0000 7fd9cec76700 1 -- 172.21.15.3:0/606401097 shutdown_connections 2023-10-03T10:16:42.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.535+0000 7fd9cec76700 1 -- 172.21.15.3:0/606401097 wait complete. 2023-10-03T10:16:42.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.536+0000 7fd9cec76700 1 Processor -- start 2023-10-03T10:16:42.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.537+0000 7fd9cec76700 1 -- start start 2023-10-03T10:16:42.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.537+0000 7fd9cec76700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9c809d6f0 0x7fd9c81a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:42.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.537+0000 7fd9cec76700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9c809e000 0x7fd9c81a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:42.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.537+0000 7fd9cec76700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9c809e410 0x7fd9c81a6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:42.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.537+0000 7fd9cec76700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c81a8790 con 0x7fd9c809d6f0 2023-10-03T10:16:42.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.537+0000 7fd9cec76700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c81a9940 con 0x7fd9c809e410 2023-10-03T10:16:42.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.537+0000 7fd9cec76700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c80a1f00 con 0x7fd9c809e000 2023-10-03T10:16:42.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.538+0000 7fd9cca12700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9c809d6f0 0x7fd9c81a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:42.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.538+0000 7fd9cd213700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9c809e410 0x7fd9c81a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:42.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.538+0000 7fd9cca12700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9c809d6f0 0x7fd9c81a5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49216/0 (socket says 172.21.15.3:49216) 2023-10-03T10:16:42.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.538+0000 7fd9cca12700 1 -- 172.21.15.3:0/1025764247 learned_addr learned my addr 172.21.15.3:0/1025764247 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:42.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.538+0000 7fd9c7fff700 1 --2- 172.21.15.3:0/1025764247 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9c809e000 0x7fd9c81a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:42.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.538+0000 7fd9cd213700 1 -- 172.21.15.3:0/1025764247 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9c809e000 msgr2=0x7fd9c81a6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:42.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.538+0000 7fd9cd213700 1 --2- 172.21.15.3:0/1025764247 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9c809e000 0x7fd9c81a6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.538+0000 7fd9cd213700 1 -- 172.21.15.3:0/1025764247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9c809d6f0 msgr2=0x7fd9c81a5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:42.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.538+0000 7fd9cd213700 1 --2- 172.21.15.3:0/1025764247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9c809d6f0 0x7fd9c81a5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.538+0000 7fd9cd213700 1 -- 172.21.15.3:0/1025764247 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c0009b30 con 0x7fd9c809e410 2023-10-03T10:16:42.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.538+0000 7fd9cca12700 1 --2- 172.21.15.3:0/1025764247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9c809d6f0 0x7fd9c81a5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:42.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.538+0000 7fd9cd213700 1 --2- 172.21.15.3:0/1025764247 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9c809e410 0x7fd9c81a6970 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0009e50 tx=0x7fd9c000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:42.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.538+0000 7fd9c57fa700 1 -- 172.21.15.3:0/1025764247 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0018070 con 0x7fd9c809e410 2023-10-03T10:16:42.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.539+0000 7fd9c57fa700 1 -- 172.21.15.3:0/1025764247 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c000f040 con 0x7fd9c809e410 2023-10-03T10:16:42.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.539+0000 7fd9c57fa700 1 -- 172.21.15.3:0/1025764247 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0003c40 con 0x7fd9c809e410 2023-10-03T10:16:42.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.539+0000 7fd9cec76700 1 -- 172.21.15.3:0/1025764247 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b40094a0 con 0x7fd9c809e410 2023-10-03T10:16:42.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.539+0000 7fd9cec76700 1 -- 172.21.15.3:0/1025764247 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c81a79c0 con 0x7fd9c809e410 2023-10-03T10:16:42.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.540+0000 7fd9cec76700 1 -- 172.21.15.3:0/1025764247 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c8074ea0 con 0x7fd9c809e410 2023-10-03T10:16:42.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.540+0000 7fd9c57fa700 1 -- 172.21.15.3:0/1025764247 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd9c000c880 con 0x7fd9c809e410 2023-10-03T10:16:42.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.540+0000 7fd9c57fa700 1 -- 172.21.15.3:0/1025764247 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd9c0040420 con 0x7fd9c809e410 2023-10-03T10:16:42.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.545+0000 7fd9c57fa700 1 -- 172.21.15.3:0/1025764247 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd9c000cb30 con 0x7fd9c809e410 2023-10-03T10:16:42.552 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:42.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.635+0000 7ff1d7797700 1 -- 172.21.15.3:0/3059962730 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff1d01a49e0 con 0x7ff1d009f650 2023-10-03T10:16:42.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.635+0000 7ff1c5ffb700 1 -- 172.21.15.3:0/3059962730 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff1c8011380 con 0x7ff1d009f650 2023-10-03T10:16:42.635 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:42.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.637+0000 7ff1d7797700 1 -- 172.21.15.3:0/3059962730 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1d009f650 msgr2=0x7ff1d006b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:42.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.637+0000 7ff1d7797700 1 --2- 172.21.15.3:0/3059962730 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1d009f650 0x7ff1d006b880 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800cbc0 tx=0x7ff1c800ced0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.637+0000 7ff1d7797700 1 -- 172.21.15.3:0/3059962730 shutdown_connections 2023-10-03T10:16:42.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.638+0000 7ff1d7797700 1 --2- 172.21.15.3:0/3059962730 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1d009f650 0x7ff1d006b880 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.638+0000 7ff1d7797700 1 --2- 172.21.15.3:0/3059962730 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1d009e930 0x7ff1d01a7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.638+0000 7ff1d7797700 1 --2- 172.21.15.3:0/3059962730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1d009f240 0x7ff1d01a7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.638+0000 7ff1d7797700 1 -- 172.21.15.3:0/3059962730 >> 172.21.15.3:0/3059962730 conn(0x7ff1d00a4210 msgr2=0x7ff1d01b8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:42.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.638+0000 7ff1d7797700 1 -- 172.21.15.3:0/3059962730 shutdown_connections 2023-10-03T10:16:42.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.638+0000 7ff1d7797700 1 -- 172.21.15.3:0/3059962730 wait complete. 2023-10-03T10:16:42.653 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:42.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.653+0000 7ff1bfb17700 1 Processor -- start 2023-10-03T10:16:42.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.653+0000 7ff1bfb17700 1 -- start start 2023-10-03T10:16:42.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.653+0000 7ff1bfb17700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1b8131810 0x7ff1b8131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:42.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.654+0000 7ff1bfb17700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff1b8099a40 0x7ff1b8099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:42.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.654+0000 7ff1bfb17700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1b809a3b0 0x7ff1b8138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:42.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.654+0000 7ff1bfb17700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b81313c0 con 0x7ff1b8099a40 2023-10-03T10:16:42.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.654+0000 7ff1bfb17700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b806e8c0 con 0x7ff1b809a3b0 2023-10-03T10:16:42.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.654+0000 7ff1bfb17700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b809a820 con 0x7ff1b8131810 2023-10-03T10:16:42.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.654+0000 7ff1bd0b2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff1b8099a40 0x7ff1b8099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:42.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.654+0000 7ff1bd0b2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff1b8099a40 0x7ff1b8099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49224/0 (socket says 172.21.15.3:49224) 2023-10-03T10:16:42.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.654+0000 7ff1bd0b2700 1 -- 172.21.15.3:0/3836710673 learned_addr learned my addr 172.21.15.3:0/3836710673 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:42.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.654+0000 7ff1bd8b3700 1 --2- 172.21.15.3:0/3836710673 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1b8131810 0x7ff1b8131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:42.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.654+0000 7ff1be0b4700 1 --2- 172.21.15.3:0/3836710673 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1b809a3b0 0x7ff1b8138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:42.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.654+0000 7ff1bd8b3700 1 -- 172.21.15.3:0/3836710673 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1b809a3b0 msgr2=0x7ff1b8138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:42.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.654+0000 7ff1bd8b3700 1 --2- 172.21.15.3:0/3836710673 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1b809a3b0 0x7ff1b8138650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.654+0000 7ff1bd8b3700 1 -- 172.21.15.3:0/3836710673 >> v2:172.21.15.3:3300/0 conn(0x7ff1b8099a40 msgr2=0x7ff1b8099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:42.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.654+0000 7ff1bd8b3700 1 --2- 172.21.15.3:0/3836710673 >> v2:172.21.15.3:3300/0 conn(0x7ff1b8099a40 0x7ff1b8099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.654+0000 7ff1bd8b3700 1 -- 172.21.15.3:0/3836710673 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b8071d20 con 0x7ff1b8131810 2023-10-03T10:16:42.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.654+0000 7ff1bd8b3700 1 --2- 172.21.15.3:0/3836710673 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1b8131810 0x7ff1b8131be0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4009840 tx=0x7ff1b4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0680c0d1c41db14 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:42.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.655+0000 7ff1bc8b1700 1 -- 172.21.15.3:0/3836710673 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4010a90 con 0x7ff1b8131810 2023-10-03T10:16:42.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.655+0000 7ff1bc8b1700 1 -- 172.21.15.3:0/3836710673 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b4010bf0 con 0x7ff1b8131810 2023-10-03T10:16:42.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.655+0000 7ff1bc8b1700 1 -- 172.21.15.3:0/3836710673 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4018440 con 0x7ff1b8131810 2023-10-03T10:16:42.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.655+0000 7ff1bfb17700 1 -- 172.21.15.3:0/3836710673 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1b8131810 msgr2=0x7ff1b8131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.655+0000 7ff1bfb17700 1 --2- 172.21.15.3:0/3836710673 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1b8131810 0x7ff1b8131be0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4009840 tx=0x7ff1b4009b50 comp rx=0 tx=0).stop 2023-10-03T10:16:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.655+0000 7ff1bfb17700 1 -- 172.21.15.3:0/3836710673 shutdown_connections 2023-10-03T10:16:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.655+0000 7ff1bfb17700 1 --2- 172.21.15.3:0/3836710673 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1b809a3b0 0x7ff1b8138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.655+0000 7ff1bfb17700 1 --2- 172.21.15.3:0/3836710673 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1b8131810 0x7ff1b8131be0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.656+0000 7ff1bfb17700 1 --2- 172.21.15.3:0/3836710673 >> v2:172.21.15.3:3300/0 conn(0x7ff1b8099a40 0x7ff1b8099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.656+0000 7ff1bfb17700 1 -- 172.21.15.3:0/3836710673 >> 172.21.15.3:0/3836710673 conn(0x7ff1b80a2aa0 msgr2=0x7ff1b80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.656+0000 7ff1bfb17700 1 -- 172.21.15.3:0/3836710673 shutdown_connections 2023-10-03T10:16:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.656+0000 7ff1bfb17700 1 -- 172.21.15.3:0/3836710673 wait complete. 2023-10-03T10:16:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.657+0000 7ff1bfb17700 1 Processor -- start 2023-10-03T10:16:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.657+0000 7ff1bfb17700 1 -- start start 2023-10-03T10:16:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.657+0000 7ff1bfb17700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1b8099a40 0x7ff1b8119700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:42.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.657+0000 7ff1bfb17700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1b809a3b0 0x7ff1b8119c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:42.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.657+0000 7ff1bfb17700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1b8131810 0x7ff1b811a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:42.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.657+0000 7ff1bfb17700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b811b310 con 0x7ff1b809a3b0 2023-10-03T10:16:42.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.657+0000 7ff1bfb17700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b811c560 con 0x7ff1b8099a40 2023-10-03T10:16:42.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.657+0000 7ff1bfb17700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b811b8f0 con 0x7ff1b8131810 2023-10-03T10:16:42.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.657+0000 7ff1be0b4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1b8131810 0x7ff1b811a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:42.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.657+0000 7ff1be0b4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1b8131810 0x7ff1b811a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50780/0 (socket says 172.21.15.3:50780) 2023-10-03T10:16:42.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.657+0000 7ff1be0b4700 1 -- 172.21.15.3:0/1519679344 learned_addr learned my addr 172.21.15.3:0/1519679344 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:42.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.657+0000 7ff1bd8b3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1b8099a40 0x7ff1b8119700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:42.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.657+0000 7ff1bd0b2700 1 --2- 172.21.15.3:0/1519679344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1b809a3b0 0x7ff1b8119c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:42.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.658+0000 7ff1be0b4700 1 -- 172.21.15.3:0/1519679344 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1b8099a40 msgr2=0x7ff1b8119700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:42.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.658+0000 7ff1be0b4700 1 --2- 172.21.15.3:0/1519679344 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1b8099a40 0x7ff1b8119700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.658+0000 7ff1be0b4700 1 -- 172.21.15.3:0/1519679344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1b809a3b0 msgr2=0x7ff1b8119c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:42.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.658+0000 7ff1be0b4700 1 --2- 172.21.15.3:0/1519679344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1b809a3b0 0x7ff1b8119c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.658+0000 7ff1be0b4700 1 -- 172.21.15.3:0/1519679344 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b40094a0 con 0x7ff1b8131810 2023-10-03T10:16:42.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.658+0000 7ff1bd8b3700 1 --2- 172.21.15.3:0/1519679344 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1b8099a40 0x7ff1b8119700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:42.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.658+0000 7ff1be0b4700 1 --2- 172.21.15.3:0/1519679344 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1b8131810 0x7ff1b811a180 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000efc0 tx=0x7ff1b000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:42.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.658+0000 7ff1ae7fc700 1 -- 172.21.15.3:0/1519679344 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0009ea0 con 0x7ff1b8131810 2023-10-03T10:16:42.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.658+0000 7ff1ae7fc700 1 -- 172.21.15.3:0/1519679344 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b000f040 con 0x7ff1b8131810 2023-10-03T10:16:42.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.658+0000 7ff1bfb17700 1 -- 172.21.15.3:0/1519679344 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b811cf40 con 0x7ff1b8131810 2023-10-03T10:16:42.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.658+0000 7ff1ae7fc700 1 -- 172.21.15.3:0/1519679344 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b000ced0 con 0x7ff1b8131810 2023-10-03T10:16:42.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.658+0000 7ff1bfb17700 1 -- 172.21.15.3:0/1519679344 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b806eb00 con 0x7ff1b8131810 2023-10-03T10:16:42.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.659+0000 7ff1bfb17700 1 -- 172.21.15.3:0/1519679344 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b8074ea0 con 0x7ff1b8131810 2023-10-03T10:16:42.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.660+0000 7ff1ae7fc700 1 -- 172.21.15.3:0/1519679344 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff1b0014480 con 0x7ff1b8131810 2023-10-03T10:16:42.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.660+0000 7ff1ae7fc700 1 -- 172.21.15.3:0/1519679344 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff1b0040880 con 0x7ff1b8131810 2023-10-03T10:16:42.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.669+0000 7ff1ae7fc700 1 -- 172.21.15.3:0/1519679344 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff1b0012930 con 0x7ff1b8131810 2023-10-03T10:16:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.697+0000 7f14d7b43700 1 Processor -- start 2023-10-03T10:16:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.697+0000 7f14d7b43700 1 -- start start 2023-10-03T10:16:42.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.698+0000 7f14d7b43700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f14d009d630 0x7f14d00a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:42.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.698+0000 7f14d7b43700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f14d0138d80 0x7f14d013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:42.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.698+0000 7f14d7b43700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d00a2a70 con 0x7f14d00a0a30 2023-10-03T10:16:42.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.698+0000 7f14d7b43700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14d00a01d0 con 0x7f14d009d630 2023-10-03T10:16:42.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.698+0000 7f14d7b43700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14d009e2b0 con 0x7f14d0138d80 2023-10-03T10:16:42.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.698+0000 7f14d50de700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f14d009d630 0x7f14d00a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:42.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.698+0000 7f14d50de700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f14d009d630 0x7f14d00a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49232/0 (socket says 172.21.15.3:49232) 2023-10-03T10:16:42.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.698+0000 7f14d60e0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f14d0138d80 0x7f14d013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:42.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.698+0000 7f14d50de700 1 -- 172.21.15.3:0/2070388664 learned_addr learned my addr 172.21.15.3:0/2070388664 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:42.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.699+0000 7f14d60e0700 1 -- 172.21.15.3:0/2070388664 >> v2:172.21.15.3:3300/0 conn(0x7f14d009d630 msgr2=0x7f14d00a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:42.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.699+0000 7f14d60e0700 1 --2- 172.21.15.3:0/2070388664 >> v2:172.21.15.3:3300/0 conn(0x7f14d009d630 0x7f14d00a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.699+0000 7f14d60e0700 1 -- 172.21.15.3:0/2070388664 >> v1:172.21.15.3:6789/0 conn(0x7f14d00a0a30 legacy=0x7f14d00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:42.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.699+0000 7f14d60e0700 1 -- 172.21.15.3:0/2070388664 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d009dc50 con 0x7f14d0138d80 2023-10-03T10:16:42.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.699+0000 7f14d50de700 1 --2- 172.21.15.3:0/2070388664 >> v2:172.21.15.3:3300/0 conn(0x7f14d009d630 0x7f14d00a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:42.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.699+0000 7f14d60e0700 1 --2- 172.21.15.3:0/2070388664 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f14d0138d80 0x7f14d013b160 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009d90 tx=0x7f14c800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8a92e29d3894eb0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:42.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.699+0000 7f14d48dd700 1 -- 172.21.15.3:0/2070388664 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c800e070 con 0x7f14d0138d80 2023-10-03T10:16:42.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.699+0000 7f14d48dd700 1 -- 172.21.15.3:0/2070388664 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c800bb20 con 0x7f14d0138d80 2023-10-03T10:16:42.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.699+0000 7f14d48dd700 1 -- 172.21.15.3:0/2070388664 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c8012430 con 0x7f14d0138d80 2023-10-03T10:16:42.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.700+0000 7f14d7b43700 1 -- 172.21.15.3:0/2070388664 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f14d0138d80 msgr2=0x7f14d013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:42.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.700+0000 7f14d7b43700 1 --2- 172.21.15.3:0/2070388664 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f14d0138d80 0x7f14d013b160 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009d90 tx=0x7f14c800b330 comp rx=0 tx=0).stop 2023-10-03T10:16:42.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.700+0000 7f14d7b43700 1 -- 172.21.15.3:0/2070388664 shutdown_connections 2023-10-03T10:16:42.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.700+0000 7f14d7b43700 1 --2- 172.21.15.3:0/2070388664 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f14d0138d80 0x7f14d013b160 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.700+0000 7f14d7b43700 1 --2- 172.21.15.3:0/2070388664 >> v2:172.21.15.3:3300/0 conn(0x7f14d009d630 0x7f14d00a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.700+0000 7f14d7b43700 1 -- 172.21.15.3:0/2070388664 >> 172.21.15.3:0/2070388664 conn(0x7f14d00a3050 msgr2=0x7f14d00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:42.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.700+0000 7f14d7b43700 1 -- 172.21.15.3:0/2070388664 shutdown_connections 2023-10-03T10:16:42.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.700+0000 7f14d7b43700 1 -- 172.21.15.3:0/2070388664 wait complete. 2023-10-03T10:16:42.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.701+0000 7f14d7b43700 1 Processor -- start 2023-10-03T10:16:42.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.701+0000 7f14d7b43700 1 -- start start 2023-10-03T10:16:42.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.702+0000 7f14d7b43700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f14d009d630 0x7f14d0120670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:42.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.702+0000 7f14d7b43700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f14d00a0a30 0x7f14d0120bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:42.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.702+0000 7f14d7b43700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14d0138d80 0x7f14d01210f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:42.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.702+0000 7f14d7b43700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d0124190 con 0x7f14d0138d80 2023-10-03T10:16:42.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.702+0000 7f14d7b43700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d0121ad0 con 0x7f14d009d630 2023-10-03T10:16:42.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.702+0000 7f14d7b43700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14d0123120 con 0x7f14d00a0a30 2023-10-03T10:16:42.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.702+0000 7f14d50de700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f14d00a0a30 0x7f14d0120bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:42.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.702+0000 7f14d50de700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f14d00a0a30 0x7f14d0120bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50798/0 (socket says 172.21.15.3:50798) 2023-10-03T10:16:42.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.702+0000 7f14d50de700 1 -- 172.21.15.3:0/470311475 learned_addr learned my addr 172.21.15.3:0/470311475 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:42.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.702+0000 7f14d60e0700 1 --2- 172.21.15.3:0/470311475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14d0138d80 0x7f14d01210f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:42.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.702+0000 7f14d50de700 1 -- 172.21.15.3:0/470311475 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f14d009d630 msgr2=0x7f14d0120670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:42.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.702+0000 7f14d50de700 1 --2- 172.21.15.3:0/470311475 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f14d009d630 0x7f14d0120670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.702+0000 7f14d50de700 1 -- 172.21.15.3:0/470311475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14d0138d80 msgr2=0x7f14d01210f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:42.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.702+0000 7f14d50de700 1 --2- 172.21.15.3:0/470311475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14d0138d80 0x7f14d01210f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.702+0000 7f14d50de700 1 -- 172.21.15.3:0/470311475 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c4009550 con 0x7f14d00a0a30 2023-10-03T10:16:42.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.703+0000 7f14d50de700 1 --2- 172.21.15.3:0/470311475 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f14d00a0a30 0x7f14d0120bb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f14c400ebe0 tx=0x7f14c400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:42.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.703+0000 7f14ce7fc700 1 -- 172.21.15.3:0/470311475 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c400c850 con 0x7f14d00a0a30 2023-10-03T10:16:42.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.703+0000 7f14ce7fc700 1 -- 172.21.15.3:0/470311475 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c400c9b0 con 0x7f14d00a0a30 2023-10-03T10:16:42.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.703+0000 7f14d7b43700 1 -- 172.21.15.3:0/470311475 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c8009a70 con 0x7f14d00a0a30 2023-10-03T10:16:42.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.703+0000 7f14ce7fc700 1 -- 172.21.15.3:0/470311475 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c400cd20 con 0x7f14d00a0a30 2023-10-03T10:16:42.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.703+0000 7f14d7b43700 1 -- 172.21.15.3:0/470311475 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d013afb0 con 0x7f14d00a0a30 2023-10-03T10:16:42.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.704+0000 7f14ce7fc700 1 -- 172.21.15.3:0/470311475 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f14c4013470 con 0x7f14d00a0a30 2023-10-03T10:16:42.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.704+0000 7f14ce7fc700 1 -- 172.21.15.3:0/470311475 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f14c403fc20 con 0x7f14d00a0a30 2023-10-03T10:16:42.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.704+0000 7f14d7b43700 1 -- 172.21.15.3:0/470311475 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d00747f0 con 0x7f14d00a0a30 2023-10-03T10:16:42.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.709+0000 7f14ce7fc700 1 -- 172.21.15.3:0/470311475 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f14c401b3b0 con 0x7f14d00a0a30 2023-10-03T10:16:42.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.780+0000 7fd9cec76700 1 -- 172.21.15.3:0/1025764247 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd9c81a9300 con 0x7fd9c809e410 2023-10-03T10:16:42.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.780+0000 7fd9c57fa700 1 -- 172.21.15.3:0/1025764247 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd9c81a9300 con 0x7fd9c809e410 2023-10-03T10:16:42.781 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:42.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.783+0000 7fd9cec76700 1 -- 172.21.15.3:0/1025764247 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9c809e410 msgr2=0x7fd9c81a6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:42.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.783+0000 7fd9cec76700 1 --2- 172.21.15.3:0/1025764247 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9c809e410 0x7fd9c81a6970 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0009e50 tx=0x7fd9c000c3a0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.783+0000 7fd9cec76700 1 -- 172.21.15.3:0/1025764247 shutdown_connections 2023-10-03T10:16:42.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.783+0000 7fd9cec76700 1 --2- 172.21.15.3:0/1025764247 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9c809e410 0x7fd9c81a6970 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.783+0000 7fd9cec76700 1 --2- 172.21.15.3:0/1025764247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9c809d6f0 0x7fd9c81a5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.783+0000 7fd9cec76700 1 --2- 172.21.15.3:0/1025764247 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9c809e000 0x7fd9c81a6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.783+0000 7fd9cec76700 1 -- 172.21.15.3:0/1025764247 >> 172.21.15.3:0/1025764247 conn(0x7fd9c8066f50 msgr2=0x7fd9c81badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:42.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.783+0000 7fd9cec76700 1 -- 172.21.15.3:0/1025764247 shutdown_connections 2023-10-03T10:16:42.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.783+0000 7fd9cec76700 1 -- 172.21.15.3:0/1025764247 wait complete. 2023-10-03T10:16:42.797 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:42.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.879+0000 7ff1bfb17700 1 -- 172.21.15.3:0/1519679344 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff1b806b9e0 con 0x7ff1b8131810 2023-10-03T10:16:42.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.880+0000 7ff1ae7fc700 1 -- 172.21.15.3:0/1519679344 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff1b002a020 con 0x7ff1b8131810 2023-10-03T10:16:42.880 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:42.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.882+0000 7ff1bfb17700 1 -- 172.21.15.3:0/1519679344 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1b8131810 msgr2=0x7ff1b811a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:42.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.882+0000 7ff1bfb17700 1 --2- 172.21.15.3:0/1519679344 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1b8131810 0x7ff1b811a180 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000efc0 tx=0x7ff1b000c3a0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.882+0000 7ff1bfb17700 1 -- 172.21.15.3:0/1519679344 shutdown_connections 2023-10-03T10:16:42.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.882+0000 7ff1bfb17700 1 --2- 172.21.15.3:0/1519679344 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1b8131810 0x7ff1b811a180 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.882+0000 7ff1bfb17700 1 --2- 172.21.15.3:0/1519679344 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1b8099a40 0x7ff1b8119700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.882+0000 7ff1bfb17700 1 --2- 172.21.15.3:0/1519679344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1b809a3b0 0x7ff1b8119c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.882+0000 7ff1bfb17700 1 -- 172.21.15.3:0/1519679344 >> 172.21.15.3:0/1519679344 conn(0x7ff1b80a2aa0 msgr2=0x7ff1b812e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:42.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.882+0000 7ff1bfb17700 1 -- 172.21.15.3:0/1519679344 shutdown_connections 2023-10-03T10:16:42.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.882+0000 7ff1bfb17700 1 -- 172.21.15.3:0/1519679344 wait complete. 2023-10-03T10:16:42.896 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:42.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.916+0000 7f14d7b43700 1 -- 172.21.15.3:0/470311475 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f14d0072980 con 0x7f14d00a0a30 2023-10-03T10:16:42.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.916+0000 7f14ce7fc700 1 -- 172.21.15.3:0/470311475 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f14c4017030 con 0x7f14d00a0a30 2023-10-03T10:16:42.916 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:42.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.918+0000 7f14d7b43700 1 -- 172.21.15.3:0/470311475 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f14d00a0a30 msgr2=0x7f14d0120bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:42.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.918+0000 7f14d7b43700 1 --2- 172.21.15.3:0/470311475 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f14d00a0a30 0x7f14d0120bb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f14c400ebe0 tx=0x7f14c400eef0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.918+0000 7f14d7b43700 1 -- 172.21.15.3:0/470311475 shutdown_connections 2023-10-03T10:16:42.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.918+0000 7f14d7b43700 1 --2- 172.21.15.3:0/470311475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14d0138d80 0x7f14d01210f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.918+0000 7f14d7b43700 1 --2- 172.21.15.3:0/470311475 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f14d009d630 0x7f14d0120670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.918+0000 7f14d7b43700 1 --2- 172.21.15.3:0/470311475 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f14d00a0a30 0x7f14d0120bb0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:42.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.918+0000 7f14d7b43700 1 -- 172.21.15.3:0/470311475 >> 172.21.15.3:0/470311475 conn(0x7f14d00a3050 msgr2=0x7f14d009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:42.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.919+0000 7f14d7b43700 1 -- 172.21.15.3:0/470311475 shutdown_connections 2023-10-03T10:16:42.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:42.919+0000 7f14d7b43700 1 -- 172.21.15.3:0/470311475 wait complete. 2023-10-03T10:16:42.932 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:43.654 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:43.798 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:43.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.799+0000 7fc92c00b700 1 Processor -- start 2023-10-03T10:16:43.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.799+0000 7fc92c00b700 1 -- start start 2023-10-03T10:16:43.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.800+0000 7fc92c00b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9240a0a30 0x7fc9240a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:43.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.800+0000 7fc92c00b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc9240a1340 0x7fc924133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:43.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.800+0000 7fc92c00b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc92409edd0 con 0x7fc924133fa0 2023-10-03T10:16:43.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.800+0000 7fc92c00b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc92409e1f0 con 0x7fc9240a1340 2023-10-03T10:16:43.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.800+0000 7fc92c00b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc92409dc50 con 0x7fc9240a0a30 2023-10-03T10:16:43.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.800+0000 7fc929da7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9240a0a30 0x7fc9240a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:43.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.800+0000 7fc9295a6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc9240a1340 0x7fc924133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:43.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.800+0000 7fc929da7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9240a0a30 0x7fc9240a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51520/0 (socket says 172.21.15.3:51520) 2023-10-03T10:16:43.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.800+0000 7fc929da7700 1 -- 172.21.15.3:0/4106881094 learned_addr learned my addr 172.21.15.3:0/4106881094 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:43.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.801+0000 7fc929da7700 1 -- 172.21.15.3:0/4106881094 >> v2:172.21.15.3:3300/0 conn(0x7fc9240a1340 msgr2=0x7fc924133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:43.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.801+0000 7fc929da7700 1 --2- 172.21.15.3:0/4106881094 >> v2:172.21.15.3:3300/0 conn(0x7fc9240a1340 0x7fc924133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:43.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.801+0000 7fc929da7700 1 -- 172.21.15.3:0/4106881094 >> v1:172.21.15.3:6789/0 conn(0x7fc924133fa0 legacy=0x7fc92413ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:43.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.801+0000 7fc929da7700 1 -- 172.21.15.3:0/4106881094 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9240a1fe0 con 0x7fc9240a0a30 2023-10-03T10:16:43.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.801+0000 7fc9295a6700 1 --2- 172.21.15.3:0/4106881094 >> v2:172.21.15.3:3300/0 conn(0x7fc9240a1340 0x7fc924133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:43.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.801+0000 7fc929da7700 1 --2- 172.21.15.3:0/4106881094 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9240a0a30 0x7fc9240a0e00 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc9200098f0 tx=0x7fc920009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dec4d1631c13cc3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:43.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.801+0000 7fc928da5700 1 -- 172.21.15.3:0/4106881094 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc920011830 con 0x7fc9240a0a30 2023-10-03T10:16:43.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.801+0000 7fc928da5700 1 -- 172.21.15.3:0/4106881094 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc920011990 con 0x7fc9240a0a30 2023-10-03T10:16:43.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.801+0000 7fc928da5700 1 -- 172.21.15.3:0/4106881094 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc920011d20 con 0x7fc9240a0a30 2023-10-03T10:16:43.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.802+0000 7fc92c00b700 1 -- 172.21.15.3:0/4106881094 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9240a0a30 msgr2=0x7fc9240a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:43.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.802+0000 7fc92c00b700 1 --2- 172.21.15.3:0/4106881094 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9240a0a30 0x7fc9240a0e00 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc9200098f0 tx=0x7fc920009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:43.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.802+0000 7fc92c00b700 1 -- 172.21.15.3:0/4106881094 shutdown_connections 2023-10-03T10:16:43.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.802+0000 7fc92c00b700 1 --2- 172.21.15.3:0/4106881094 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9240a0a30 0x7fc9240a0e00 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:43.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.802+0000 7fc92c00b700 1 --2- 172.21.15.3:0/4106881094 >> v2:172.21.15.3:3300/0 conn(0x7fc9240a1340 0x7fc924133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:43.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.802+0000 7fc92c00b700 1 -- 172.21.15.3:0/4106881094 >> 172.21.15.3:0/4106881094 conn(0x7fc9240a6550 msgr2=0x7fc9240a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:43.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.802+0000 7fc92c00b700 1 -- 172.21.15.3:0/4106881094 shutdown_connections 2023-10-03T10:16:43.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.802+0000 7fc92c00b700 1 -- 172.21.15.3:0/4106881094 wait complete. 2023-10-03T10:16:43.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.803+0000 7fc92c00b700 1 Processor -- start 2023-10-03T10:16:43.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.803+0000 7fc92c00b700 1 -- start start 2023-10-03T10:16:43.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.803+0000 7fc92c00b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9240a0a30 0x7fc92411e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:43.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.803+0000 7fc92c00b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9240a1340 0x7fc92411ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:43.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.803+0000 7fc92c00b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc924133fa0 0x7fc92411f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:43.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.803+0000 7fc92c00b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc924121f50 con 0x7fc9240a0a30 2023-10-03T10:16:43.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.803+0000 7fc92c00b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc924120430 con 0x7fc924133fa0 2023-10-03T10:16:43.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.803+0000 7fc92c00b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9240a37e0 con 0x7fc9240a1340 2023-10-03T10:16:43.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.803+0000 7fc9295a6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9240a1340 0x7fc92411ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:43.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.803+0000 7fc92a5a8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc924133fa0 0x7fc92411f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:43.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.803+0000 7fc9295a6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9240a1340 0x7fc92411ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51536/0 (socket says 172.21.15.3:51536) 2023-10-03T10:16:43.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.803+0000 7fc9295a6700 1 -- 172.21.15.3:0/2584964651 learned_addr learned my addr 172.21.15.3:0/2584964651 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:43.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.804+0000 7fc9295a6700 1 -- 172.21.15.3:0/2584964651 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc924133fa0 msgr2=0x7fc92411f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:43.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.804+0000 7fc9295a6700 1 --2- 172.21.15.3:0/2584964651 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc924133fa0 0x7fc92411f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:43.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.804+0000 7fc9295a6700 1 -- 172.21.15.3:0/2584964651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9240a0a30 msgr2=0x7fc92411e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:43.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.804+0000 7fc9295a6700 1 --2- 172.21.15.3:0/2584964651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9240a0a30 0x7fc92411e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:43.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.804+0000 7fc9295a6700 1 -- 172.21.15.3:0/2584964651 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9140094a0 con 0x7fc9240a1340 2023-10-03T10:16:43.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.804+0000 7fc9295a6700 1 --2- 172.21.15.3:0/2584964651 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9240a1340 0x7fc92411ece0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc91400ec10 tx=0x7fc91400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:43.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.804+0000 7fc91a7fc700 1 -- 172.21.15.3:0/2584964651 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91400c910 con 0x7fc9240a1340 2023-10-03T10:16:43.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.804+0000 7fc91a7fc700 1 -- 172.21.15.3:0/2584964651 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc91400ca70 con 0x7fc9240a1340 2023-10-03T10:16:43.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.804+0000 7fc92c00b700 1 -- 172.21.15.3:0/2584964651 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc920009550 con 0x7fc9240a1340 2023-10-03T10:16:43.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.804+0000 7fc91a7fc700 1 -- 172.21.15.3:0/2584964651 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91400cde0 con 0x7fc9240a1340 2023-10-03T10:16:43.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.804+0000 7fc92c00b700 1 -- 172.21.15.3:0/2584964651 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9240a0270 con 0x7fc9240a1340 2023-10-03T10:16:43.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.805+0000 7fc92c00b700 1 -- 172.21.15.3:0/2584964651 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc924122a80 con 0x7fc9240a1340 2023-10-03T10:16:43.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.806+0000 7fc91a7fc700 1 -- 172.21.15.3:0/2584964651 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc914013470 con 0x7fc9240a1340 2023-10-03T10:16:43.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.806+0000 7fc91a7fc700 1 -- 172.21.15.3:0/2584964651 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc914040050 con 0x7fc9240a1340 2023-10-03T10:16:43.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.811+0000 7fc91a7fc700 1 -- 172.21.15.3:0/2584964651 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc914024350 con 0x7fc9240a1340 2023-10-03T10:16:43.896 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:43.933 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:43.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.938+0000 7f5d45a0f700 1 Processor -- start 2023-10-03T10:16:43.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.938+0000 7f5d45a0f700 1 -- start start 2023-10-03T10:16:43.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.938+0000 7f5d45a0f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5d4009d630 0x7f5d400a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:43.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.938+0000 7f5d45a0f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d40138d80 0x7f5d4013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:43.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.938+0000 7f5d45a0f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d400a2a70 con 0x7f5d400a0a30 2023-10-03T10:16:43.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.938+0000 7f5d45a0f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d400a01d0 con 0x7f5d4009d630 2023-10-03T10:16:43.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.938+0000 7f5d45a0f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4009e2b0 con 0x7f5d40138d80 2023-10-03T10:16:43.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.939+0000 7f5d3e7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5d4009d630 0x7f5d400a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:43.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.939+0000 7f5d3e7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5d4009d630 0x7f5d400a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48308/0 (socket says 172.21.15.3:48308) 2023-10-03T10:16:43.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.939+0000 7f5d3f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d40138d80 0x7f5d4013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:43.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.939+0000 7f5d3e7fc700 1 -- 172.21.15.3:0/2652698418 learned_addr learned my addr 172.21.15.3:0/2652698418 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:43.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.939+0000 7f5d3f7fe700 1 -- 172.21.15.3:0/2652698418 >> v2:172.21.15.3:3300/0 conn(0x7f5d4009d630 msgr2=0x7f5d400a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:43.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.939+0000 7f5d3f7fe700 1 --2- 172.21.15.3:0/2652698418 >> v2:172.21.15.3:3300/0 conn(0x7f5d4009d630 0x7f5d400a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:43.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.939+0000 7f5d3f7fe700 1 -- 172.21.15.3:0/2652698418 >> v1:172.21.15.3:6789/0 conn(0x7f5d400a0a30 legacy=0x7f5d400a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:43.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.939+0000 7f5d3f7fe700 1 -- 172.21.15.3:0/2652698418 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4009dc50 con 0x7f5d40138d80 2023-10-03T10:16:43.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.939+0000 7f5d3f7fe700 1 --2- 172.21.15.3:0/2652698418 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d40138d80 0x7f5d4013b160 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5d38009d90 tx=0x7f5d3800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d570c16365913cb0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:43.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.939+0000 7f5d3dffb700 1 -- 172.21.15.3:0/2652698418 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3800e070 con 0x7f5d40138d80 2023-10-03T10:16:43.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.940+0000 7f5d3dffb700 1 -- 172.21.15.3:0/2652698418 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3800bb20 con 0x7f5d40138d80 2023-10-03T10:16:43.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.940+0000 7f5d3dffb700 1 -- 172.21.15.3:0/2652698418 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d38012430 con 0x7f5d40138d80 2023-10-03T10:16:43.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.940+0000 7f5d3e7fc700 1 --2- 172.21.15.3:0/2652698418 >> v2:172.21.15.3:3300/0 conn(0x7f5d4009d630 0x7f5d400a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:43.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.940+0000 7f5d45a0f700 1 -- 172.21.15.3:0/2652698418 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d40138d80 msgr2=0x7f5d4013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:43.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.940+0000 7f5d45a0f700 1 --2- 172.21.15.3:0/2652698418 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d40138d80 0x7f5d4013b160 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5d38009d90 tx=0x7f5d3800b330 comp rx=0 tx=0).stop 2023-10-03T10:16:43.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.940+0000 7f5d45a0f700 1 -- 172.21.15.3:0/2652698418 shutdown_connections 2023-10-03T10:16:43.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.940+0000 7f5d45a0f700 1 --2- 172.21.15.3:0/2652698418 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d40138d80 0x7f5d4013b160 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:43.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.940+0000 7f5d45a0f700 1 --2- 172.21.15.3:0/2652698418 >> v2:172.21.15.3:3300/0 conn(0x7f5d4009d630 0x7f5d400a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:43.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.940+0000 7f5d45a0f700 1 -- 172.21.15.3:0/2652698418 >> 172.21.15.3:0/2652698418 conn(0x7f5d400a3050 msgr2=0x7f5d400a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:43.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.941+0000 7f5d45a0f700 1 -- 172.21.15.3:0/2652698418 shutdown_connections 2023-10-03T10:16:43.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.941+0000 7f5d45a0f700 1 -- 172.21.15.3:0/2652698418 wait complete. 2023-10-03T10:16:43.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.942+0000 7f5d45a0f700 1 Processor -- start 2023-10-03T10:16:43.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.942+0000 7f5d45a0f700 1 -- start start 2023-10-03T10:16:43.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.943+0000 7f5d45a0f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d4009d630 0x7f5d40120660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:43.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.943+0000 7f5d45a0f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d400a0a30 0x7f5d40120ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:43.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.943+0000 7f5d45a0f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d40138d80 0x7f5d401210e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:43.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.943+0000 7f5d45a0f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d40123110 con 0x7f5d4009d630 2023-10-03T10:16:43.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.943+0000 7f5d45a0f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4013af60 con 0x7f5d400a0a30 2023-10-03T10:16:43.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.943+0000 7f5d45a0f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d400a0410 con 0x7f5d40138d80 2023-10-03T10:16:43.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.943+0000 7f5d3e7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d400a0a30 0x7f5d40120ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:43.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.943+0000 7f5d3e7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d400a0a30 0x7f5d40120ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52958/0 (socket says 172.21.15.3:52958) 2023-10-03T10:16:43.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.943+0000 7f5d3e7fc700 1 -- 172.21.15.3:0/749385612 learned_addr learned my addr 172.21.15.3:0/749385612 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:43.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.943+0000 7f5d3f7fe700 1 --2- 172.21.15.3:0/749385612 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d40138d80 0x7f5d401210e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:43.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.943+0000 7f5d3effd700 1 --2- 172.21.15.3:0/749385612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d4009d630 0x7f5d40120660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:43.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.943+0000 7f5d3e7fc700 1 -- 172.21.15.3:0/749385612 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d40138d80 msgr2=0x7f5d401210e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:43.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.943+0000 7f5d3e7fc700 1 --2- 172.21.15.3:0/749385612 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d40138d80 0x7f5d401210e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:43.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.943+0000 7f5d3e7fc700 1 -- 172.21.15.3:0/749385612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d4009d630 msgr2=0x7f5d40120660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:43.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.943+0000 7f5d3e7fc700 1 --2- 172.21.15.3:0/749385612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d4009d630 0x7f5d40120660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:43.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.943+0000 7f5d3e7fc700 1 -- 172.21.15.3:0/749385612 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d34009550 con 0x7f5d400a0a30 2023-10-03T10:16:43.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.944+0000 7f5d3e7fc700 1 --2- 172.21.15.3:0/749385612 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d400a0a30 0x7f5d40120ba0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400ebe0 tx=0x7f5d3400eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:43.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.944+0000 7f5d27fff700 1 -- 172.21.15.3:0/749385612 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3400c850 con 0x7f5d400a0a30 2023-10-03T10:16:43.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.944+0000 7f5d27fff700 1 -- 172.21.15.3:0/749385612 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3400c9b0 con 0x7f5d400a0a30 2023-10-03T10:16:43.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.944+0000 7f5d45a0f700 1 -- 172.21.15.3:0/749385612 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d38009a70 con 0x7f5d400a0a30 2023-10-03T10:16:43.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.944+0000 7f5d27fff700 1 -- 172.21.15.3:0/749385612 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3400cd40 con 0x7f5d400a0a30 2023-10-03T10:16:43.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.944+0000 7f5d45a0f700 1 -- 172.21.15.3:0/749385612 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d40123580 con 0x7f5d400a0a30 2023-10-03T10:16:43.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.946+0000 7f5d45a0f700 1 -- 172.21.15.3:0/749385612 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d401249d0 con 0x7f5d400a0a30 2023-10-03T10:16:43.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.946+0000 7f5d27fff700 1 -- 172.21.15.3:0/749385612 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5d34013470 con 0x7f5d400a0a30 2023-10-03T10:16:43.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.946+0000 7f5d27fff700 1 -- 172.21.15.3:0/749385612 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5d340483b0 con 0x7f5d400a0a30 2023-10-03T10:16:43.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:43.952+0000 7f5d27fff700 1 -- 172.21.15.3:0/749385612 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5d340102b0 con 0x7f5d400a0a30 2023-10-03T10:16:44.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.027+0000 7fc92c00b700 1 -- 172.21.15.3:0/2584964651 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc924074ea0 con 0x7fc9240a1340 2023-10-03T10:16:44.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.028+0000 7fc91a7fc700 1 -- 172.21.15.3:0/2584964651 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc91402a090 con 0x7fc9240a1340 2023-10-03T10:16:44.028 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:44.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.032+0000 7fc92c00b700 1 -- 172.21.15.3:0/2584964651 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9240a1340 msgr2=0x7fc92411ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:44.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.032+0000 7fc92c00b700 1 --2- 172.21.15.3:0/2584964651 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9240a1340 0x7fc92411ece0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc91400ec10 tx=0x7fc91400ef20 comp rx=0 tx=0).stop 2023-10-03T10:16:44.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.032+0000 7fc92c00b700 1 -- 172.21.15.3:0/2584964651 shutdown_connections 2023-10-03T10:16:44.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.032+0000 7fc92c00b700 1 --2- 172.21.15.3:0/2584964651 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc924133fa0 0x7fc92411f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:44.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.032+0000 7fc92c00b700 1 --2- 172.21.15.3:0/2584964651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9240a0a30 0x7fc92411e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:44.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.032+0000 7fc92c00b700 1 --2- 172.21.15.3:0/2584964651 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9240a1340 0x7fc92411ece0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:44.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.032+0000 7fc92c00b700 1 -- 172.21.15.3:0/2584964651 >> 172.21.15.3:0/2584964651 conn(0x7fc9240a6550 msgr2=0x7fc9241b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:44.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.032+0000 7fc92c00b700 1 -- 172.21.15.3:0/2584964651 shutdown_connections 2023-10-03T10:16:44.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.032+0000 7fc92c00b700 1 -- 172.21.15.3:0/2584964651 wait complete. 2023-10-03T10:16:44.048 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:44.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.071+0000 7fe5d43f6700 1 Processor -- start 2023-10-03T10:16:44.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.072+0000 7fe5d43f6700 1 -- start start 2023-10-03T10:16:44.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.072+0000 7fe5d43f6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5cc09d6f0 0x7fe5cc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:44.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.073+0000 7fe5d43f6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5cc09e000 0x7fe5cc130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:44.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.073+0000 7fe5d43f6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe5cc09e410 0x7fe5cc130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:44.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.073+0000 7fe5d43f6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5cc09f520 con 0x7fe5cc09e410 2023-10-03T10:16:44.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.073+0000 7fe5d43f6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc09f130 con 0x7fe5cc09e000 2023-10-03T10:16:44.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.073+0000 7fe5d43f6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc09e930 con 0x7fe5cc09d6f0 2023-10-03T10:16:44.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.073+0000 7fe5d2192700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5cc09d6f0 0x7fe5cc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:44.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.073+0000 7fe5d1991700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5cc09e000 0x7fe5cc130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:44.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.073+0000 7fe5d2192700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5cc09d6f0 0x7fe5cc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51572/0 (socket says 172.21.15.3:51572) 2023-10-03T10:16:44.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.073+0000 7fe5d2192700 1 -- 172.21.15.3:0/234024747 learned_addr learned my addr 172.21.15.3:0/234024747 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:44.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.073+0000 7fe5d2993700 1 --2- 172.21.15.3:0/234024747 >> v2:172.21.15.3:3300/0 conn(0x7fe5cc09e410 0x7fe5cc130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:44.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.073+0000 7fe5d1991700 1 -- 172.21.15.3:0/234024747 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5cc09d6f0 msgr2=0x7fe5cc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:44.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.073+0000 7fe5d1991700 1 --2- 172.21.15.3:0/234024747 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5cc09d6f0 0x7fe5cc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:44.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.073+0000 7fe5d1991700 1 -- 172.21.15.3:0/234024747 >> v2:172.21.15.3:3300/0 conn(0x7fe5cc09e410 msgr2=0x7fe5cc130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:44.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.073+0000 7fe5d1991700 1 --2- 172.21.15.3:0/234024747 >> v2:172.21.15.3:3300/0 conn(0x7fe5cc09e410 0x7fe5cc130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:44.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.073+0000 7fe5d1991700 1 -- 172.21.15.3:0/234024747 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5cc09b2d0 con 0x7fe5cc09e000 2023-10-03T10:16:44.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.073+0000 7fe5d2192700 1 --2- 172.21.15.3:0/234024747 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5cc09d6f0 0x7fe5cc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:44.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.073+0000 7fe5d1991700 1 --2- 172.21.15.3:0/234024747 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5cc09e000 0x7fe5cc130660 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009870 tx=0x7fe5bc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52eb3924563d8db4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:44.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.074+0000 7fe5d1190700 1 -- 172.21.15.3:0/234024747 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc011830 con 0x7fe5cc09e000 2023-10-03T10:16:44.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.074+0000 7fe5d1190700 1 -- 172.21.15.3:0/234024747 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5bc011990 con 0x7fe5cc09e000 2023-10-03T10:16:44.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.074+0000 7fe5d1190700 1 -- 172.21.15.3:0/234024747 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc011d20 con 0x7fe5cc09e000 2023-10-03T10:16:44.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.074+0000 7fe5d43f6700 1 -- 172.21.15.3:0/234024747 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5cc09e000 msgr2=0x7fe5cc130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:44.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.075+0000 7fe5d43f6700 1 --2- 172.21.15.3:0/234024747 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5cc09e000 0x7fe5cc130660 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009870 tx=0x7fe5bc009b80 comp rx=0 tx=0).stop 2023-10-03T10:16:44.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.075+0000 7fe5d43f6700 1 -- 172.21.15.3:0/234024747 shutdown_connections 2023-10-03T10:16:44.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.075+0000 7fe5d43f6700 1 --2- 172.21.15.3:0/234024747 >> v2:172.21.15.3:3300/0 conn(0x7fe5cc09e410 0x7fe5cc130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:44.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.075+0000 7fe5d43f6700 1 --2- 172.21.15.3:0/234024747 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5cc09d6f0 0x7fe5cc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:44.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.075+0000 7fe5d43f6700 1 --2- 172.21.15.3:0/234024747 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5cc09e000 0x7fe5cc130660 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:44.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.075+0000 7fe5d43f6700 1 -- 172.21.15.3:0/234024747 >> 172.21.15.3:0/234024747 conn(0x7fe5cc066f50 msgr2=0x7fe5cc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:44.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.075+0000 7fe5d43f6700 1 -- 172.21.15.3:0/234024747 shutdown_connections 2023-10-03T10:16:44.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.075+0000 7fe5d43f6700 1 -- 172.21.15.3:0/234024747 wait complete. 2023-10-03T10:16:44.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.076+0000 7fe5d43f6700 1 Processor -- start 2023-10-03T10:16:44.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.076+0000 7fe5d43f6700 1 -- start start 2023-10-03T10:16:44.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.077+0000 7fe5d43f6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5cc09d6f0 0x7fe5cc11dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:44.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.077+0000 7fe5d43f6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5cc09e000 0x7fe5cc11e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:44.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.077+0000 7fe5d43f6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5cc09e410 0x7fe5cc11ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:44.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.077+0000 7fe5d43f6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc121810 con 0x7fe5cc09e000 2023-10-03T10:16:44.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.077+0000 7fe5d43f6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc11fb40 con 0x7fe5cc09e410 2023-10-03T10:16:44.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.077+0000 7fe5d43f6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc120160 con 0x7fe5cc09d6f0 2023-10-03T10:16:44.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.077+0000 7fe5d2192700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5cc09d6f0 0x7fe5cc11dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:44.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.077+0000 7fe5d2192700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5cc09d6f0 0x7fe5cc11dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51576/0 (socket says 172.21.15.3:51576) 2023-10-03T10:16:44.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.077+0000 7fe5d2192700 1 -- 172.21.15.3:0/1294805540 learned_addr learned my addr 172.21.15.3:0/1294805540 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:44.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.077+0000 7fe5d1991700 1 --2- 172.21.15.3:0/1294805540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5cc09e000 0x7fe5cc11e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:44.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.077+0000 7fe5d2993700 1 --2- 172.21.15.3:0/1294805540 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5cc09e410 0x7fe5cc11ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:44.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.077+0000 7fe5d2192700 1 -- 172.21.15.3:0/1294805540 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5cc09e410 msgr2=0x7fe5cc11ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:44.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.077+0000 7fe5d2192700 1 --2- 172.21.15.3:0/1294805540 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5cc09e410 0x7fe5cc11ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:44.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.077+0000 7fe5d2192700 1 -- 172.21.15.3:0/1294805540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5cc09e000 msgr2=0x7fe5cc11e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:44.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.077+0000 7fe5d2192700 1 --2- 172.21.15.3:0/1294805540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5cc09e000 0x7fe5cc11e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:44.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.077+0000 7fe5d2192700 1 -- 172.21.15.3:0/1294805540 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c800b7f0 con 0x7fe5cc09d6f0 2023-10-03T10:16:44.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.077+0000 7fe5d2993700 1 --2- 172.21.15.3:0/1294805540 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5cc09e410 0x7fe5cc11ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:16:44.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.078+0000 7fe5d2192700 1 --2- 172.21.15.3:0/1294805540 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5cc09d6f0 0x7fe5cc11dfd0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8010f10 tx=0x7fe5c800e640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:44.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.078+0000 7fe5c2ffd700 1 -- 172.21.15.3:0/1294805540 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8004030 con 0x7fe5cc09d6f0 2023-10-03T10:16:44.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.078+0000 7fe5c2ffd700 1 -- 172.21.15.3:0/1294805540 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c8004190 con 0x7fe5cc09d6f0 2023-10-03T10:16:44.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.078+0000 7fe5d43f6700 1 -- 172.21.15.3:0/1294805540 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5bc0094a0 con 0x7fe5cc09d6f0 2023-10-03T10:16:44.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.078+0000 7fe5c2ffd700 1 -- 172.21.15.3:0/1294805540 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8004520 con 0x7fe5cc09d6f0 2023-10-03T10:16:44.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.078+0000 7fe5d43f6700 1 -- 172.21.15.3:0/1294805540 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5cc0a16b0 con 0x7fe5cc09d6f0 2023-10-03T10:16:44.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.079+0000 7fe5c2ffd700 1 -- 172.21.15.3:0/1294805540 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe5c80036a0 con 0x7fe5cc09d6f0 2023-10-03T10:16:44.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.079+0000 7fe5c2ffd700 1 -- 172.21.15.3:0/1294805540 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe5c803fd70 con 0x7fe5cc09d6f0 2023-10-03T10:16:44.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.079+0000 7fe5d43f6700 1 -- 172.21.15.3:0/1294805540 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5cc11ac20 con 0x7fe5cc09d6f0 2023-10-03T10:16:44.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.086+0000 7fa67cf81700 1 Processor -- start 2023-10-03T10:16:44.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.086+0000 7fa67cf81700 1 -- start start 2023-10-03T10:16:44.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.087+0000 7fa67cf81700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa67809d6f0 0x7fa67809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:44.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.087+0000 7fa67cf81700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa67809e410 0x7fa678130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:44.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.087+0000 7fa67cf81700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa67809f130 con 0x7fa67809e000 2023-10-03T10:16:44.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.087+0000 7fa67cf81700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa67809f520 con 0x7fa67809d6f0 2023-10-03T10:16:44.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.087+0000 7fa67cf81700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa67809e930 con 0x7fa67809e410 2023-10-03T10:16:44.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.087+0000 7fa675d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa67809e000 0x7fa678130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53130/0 (socket says 172.21.15.3:53130) 2023-10-03T10:16:44.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.087+0000 7fa675d9b700 1 -- 172.21.15.3:0/2075828313 learned_addr learned my addr 172.21.15.3:0/2075828313 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:44.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.087+0000 7fa676d9d700 1 --2- 172.21.15.3:0/2075828313 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa67809e410 0x7fa678130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:44.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.087+0000 7fa67659c700 1 --2- 172.21.15.3:0/2075828313 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa67809d6f0 0x7fa67809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:44.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.087+0000 7fe5c2ffd700 1 -- 172.21.15.3:0/1294805540 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe5c80218f0 con 0x7fe5cc09d6f0 2023-10-03T10:16:44.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.087+0000 7fa67659c700 1 -- 172.21.15.3:0/2075828313 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa67809e410 msgr2=0x7fa678130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:44.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.087+0000 7fa67559a700 1 -- 172.21.15.3:0/2075828313 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4032645974 0 0) 0x7fa67809f130 con 0x7fa67809e000 2023-10-03T10:16:44.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.087+0000 7fa67659c700 1 --2- 172.21.15.3:0/2075828313 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa67809e410 0x7fa678130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:44.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.087+0000 7fa67659c700 1 -- 172.21.15.3:0/2075828313 >> v1:172.21.15.3:6789/0 conn(0x7fa67809e000 legacy=0x7fa678130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:44.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.087+0000 7fa67659c700 1 -- 172.21.15.3:0/2075828313 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa67809b2d0 con 0x7fa67809d6f0 2023-10-03T10:16:44.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.087+0000 7fa676d9d700 1 --2- 172.21.15.3:0/2075828313 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa67809e410 0x7fa678130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:44.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.088+0000 7fa67659c700 1 --2- 172.21.15.3:0/2075828313 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa67809d6f0 0x7fa67809dac0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa668009870 tx=0x7fa668009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=badecd0407531c07 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:44.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.088+0000 7fa67559a700 1 -- 172.21.15.3:0/2075828313 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa668011830 con 0x7fa67809d6f0 2023-10-03T10:16:44.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.088+0000 7fa67559a700 1 -- 172.21.15.3:0/2075828313 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa668011990 con 0x7fa67809d6f0 2023-10-03T10:16:44.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.088+0000 7fa67559a700 1 -- 172.21.15.3:0/2075828313 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa668011d20 con 0x7fa67809d6f0 2023-10-03T10:16:44.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.088+0000 7fa67cf81700 1 -- 172.21.15.3:0/2075828313 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa67809d6f0 msgr2=0x7fa67809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:44.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.088+0000 7fa67cf81700 1 --2- 172.21.15.3:0/2075828313 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa67809d6f0 0x7fa67809dac0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa668009870 tx=0x7fa668009b80 comp rx=0 tx=0).stop 2023-10-03T10:16:44.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.089+0000 7fa67cf81700 1 -- 172.21.15.3:0/2075828313 shutdown_connections 2023-10-03T10:16:44.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.089+0000 7fa67cf81700 1 --2- 172.21.15.3:0/2075828313 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa67809d6f0 0x7fa67809dac0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:44.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.089+0000 7fa67cf81700 1 --2- 172.21.15.3:0/2075828313 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa67809e410 0x7fa678130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:44.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.089+0000 7fa67cf81700 1 -- 172.21.15.3:0/2075828313 >> 172.21.15.3:0/2075828313 conn(0x7fa678066f50 msgr2=0x7fa67809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:44.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.089+0000 7fa67cf81700 1 -- 172.21.15.3:0/2075828313 shutdown_connections 2023-10-03T10:16:44.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.089+0000 7fa67cf81700 1 -- 172.21.15.3:0/2075828313 wait complete. 2023-10-03T10:16:44.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.090+0000 7fa67cf81700 1 Processor -- start 2023-10-03T10:16:44.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.090+0000 7fa67cf81700 1 -- start start 2023-10-03T10:16:44.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.090+0000 7fa67cf81700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa67809d6f0 0x7fa678136ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:44.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.090+0000 7fa67cf81700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa67809e000 0x7fa678137000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:44.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.090+0000 7fa67cf81700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa67809e410 0x7fa678137540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:44.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.090+0000 7fa67cf81700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6781310f0 con 0x7fa67809e000 2023-10-03T10:16:44.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.090+0000 7fa67cf81700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa678139ff0 con 0x7fa67809d6f0 2023-10-03T10:16:44.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.090+0000 7fa67cf81700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa67809c700 con 0x7fa67809e410 2023-10-03T10:16:44.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.091+0000 7fa676d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa67809e410 0x7fa678137540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:44.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.091+0000 7fa676d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa67809e410 0x7fa678137540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51584/0 (socket says 172.21.15.3:51584) 2023-10-03T10:16:44.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.091+0000 7fa676d9d700 1 -- 172.21.15.3:0/2145596699 learned_addr learned my addr 172.21.15.3:0/2145596699 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:44.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.091+0000 7fa675d9b700 1 --2- 172.21.15.3:0/2145596699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa67809e000 0x7fa678137000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:44.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.091+0000 7fa67659c700 1 --2- 172.21.15.3:0/2145596699 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa67809d6f0 0x7fa678136ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:44.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.091+0000 7fa676d9d700 1 -- 172.21.15.3:0/2145596699 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa67809d6f0 msgr2=0x7fa678136ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:44.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.091+0000 7fa676d9d700 1 --2- 172.21.15.3:0/2145596699 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa67809d6f0 0x7fa678136ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:44.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.091+0000 7fa676d9d700 1 -- 172.21.15.3:0/2145596699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa67809e000 msgr2=0x7fa678137000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:44.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.091+0000 7fa676d9d700 1 --2- 172.21.15.3:0/2145596699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa67809e000 0x7fa678137000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:44.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.091+0000 7fa676d9d700 1 -- 172.21.15.3:0/2145596699 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa670009a70 con 0x7fa67809e410 2023-10-03T10:16:44.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.091+0000 7fa676d9d700 1 --2- 172.21.15.3:0/2145596699 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa67809e410 0x7fa678137540 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa67000efc0 tx=0x7fa67000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:44.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.091+0000 7fa666ffd700 1 -- 172.21.15.3:0/2145596699 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67000cb40 con 0x7fa67809e410 2023-10-03T10:16:44.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.091+0000 7fa666ffd700 1 -- 172.21.15.3:0/2145596699 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67000f040 con 0x7fa67809e410 2023-10-03T10:16:44.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.091+0000 7fa666ffd700 1 -- 172.21.15.3:0/2145596699 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670003a20 con 0x7fa67809e410 2023-10-03T10:16:44.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.092+0000 7fa67cf81700 1 -- 172.21.15.3:0/2145596699 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6680094a0 con 0x7fa67809e410 2023-10-03T10:16:44.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.092+0000 7fa67cf81700 1 -- 172.21.15.3:0/2145596699 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa67813b270 con 0x7fa67809e410 2023-10-03T10:16:44.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.093+0000 7fa666ffd700 1 -- 172.21.15.3:0/2145596699 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa670014480 con 0x7fa67809e410 2023-10-03T10:16:44.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.093+0000 7fa666ffd700 1 -- 172.21.15.3:0/2145596699 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa670040240 con 0x7fa67809e410 2023-10-03T10:16:44.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.093+0000 7fa67cf81700 1 -- 172.21.15.3:0/2145596699 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa67813a6b0 con 0x7fa67809e410 2023-10-03T10:16:44.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.098+0000 7fa666ffd700 1 -- 172.21.15.3:0/2145596699 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa670012930 con 0x7fa67809e410 2023-10-03T10:16:44.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.206+0000 7f5d45a0f700 1 -- 172.21.15.3:0/749385612 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5d400747f0 con 0x7f5d400a0a30 2023-10-03T10:16:44.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.207+0000 7f5d27fff700 1 -- 172.21.15.3:0/749385612 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5d34017070 con 0x7f5d400a0a30 2023-10-03T10:16:44.207 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:44.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.209+0000 7f5d45a0f700 1 -- 172.21.15.3:0/749385612 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d400a0a30 msgr2=0x7f5d40120ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:44.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.209+0000 7f5d45a0f700 1 --2- 172.21.15.3:0/749385612 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d400a0a30 0x7f5d40120ba0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400ebe0 tx=0x7f5d3400eef0 comp rx=0 tx=0).stop 2023-10-03T10:16:44.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.209+0000 7f5d45a0f700 1 -- 172.21.15.3:0/749385612 shutdown_connections 2023-10-03T10:16:44.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.209+0000 7f5d45a0f700 1 --2- 172.21.15.3:0/749385612 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d40138d80 0x7f5d401210e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:44.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.209+0000 7f5d45a0f700 1 --2- 172.21.15.3:0/749385612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d4009d630 0x7f5d40120660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:44.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.209+0000 7f5d45a0f700 1 --2- 172.21.15.3:0/749385612 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d400a0a30 0x7f5d40120ba0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:44.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.209+0000 7f5d45a0f700 1 -- 172.21.15.3:0/749385612 >> 172.21.15.3:0/749385612 conn(0x7f5d400a3050 msgr2=0x7f5d40139150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:44.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.209+0000 7f5d45a0f700 1 -- 172.21.15.3:0/749385612 shutdown_connections 2023-10-03T10:16:44.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.209+0000 7f5d45a0f700 1 -- 172.21.15.3:0/749385612 wait complete. 2023-10-03T10:16:44.223 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:44.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.315+0000 7fe5d43f6700 1 -- 172.21.15.3:0/1294805540 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe5cc09c760 con 0x7fe5cc09d6f0 2023-10-03T10:16:44.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.315+0000 7fe5c2ffd700 1 -- 172.21.15.3:0/1294805540 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe5c8022350 con 0x7fe5cc09d6f0 2023-10-03T10:16:44.316 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:44.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.318+0000 7fe5d43f6700 1 -- 172.21.15.3:0/1294805540 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5cc09d6f0 msgr2=0x7fe5cc11dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:44.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.318+0000 7fe5d43f6700 1 --2- 172.21.15.3:0/1294805540 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5cc09d6f0 0x7fe5cc11dfd0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8010f10 tx=0x7fe5c800e640 comp rx=0 tx=0).stop 2023-10-03T10:16:44.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.318+0000 7fe5d43f6700 1 -- 172.21.15.3:0/1294805540 shutdown_connections 2023-10-03T10:16:44.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.318+0000 7fe5d43f6700 1 --2- 172.21.15.3:0/1294805540 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5cc09e410 0x7fe5cc11ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:44.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.318+0000 7fe5d43f6700 1 --2- 172.21.15.3:0/1294805540 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5cc09d6f0 0x7fe5cc11dfd0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:44.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.318+0000 7fe5d43f6700 1 --2- 172.21.15.3:0/1294805540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5cc09e000 0x7fe5cc11e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:44.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.318+0000 7fe5d43f6700 1 -- 172.21.15.3:0/1294805540 >> 172.21.15.3:0/1294805540 conn(0x7fe5cc066f50 msgr2=0x7fe5cc130920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:44.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.318+0000 7fe5d43f6700 1 -- 172.21.15.3:0/1294805540 shutdown_connections 2023-10-03T10:16:44.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.318+0000 7fe5d43f6700 1 -- 172.21.15.3:0/1294805540 wait complete. 2023-10-03T10:16:44.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.320+0000 7fa67cf81700 1 -- 172.21.15.3:0/2145596699 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa67813a3d0 con 0x7fa67809e410 2023-10-03T10:16:44.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.320+0000 7fa666ffd700 1 -- 172.21.15.3:0/2145596699 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa67001c3b0 con 0x7fa67809e410 2023-10-03T10:16:44.321 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:44.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.323+0000 7fa67cf81700 1 -- 172.21.15.3:0/2145596699 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa67809e410 msgr2=0x7fa678137540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:44.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.323+0000 7fa67cf81700 1 --2- 172.21.15.3:0/2145596699 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa67809e410 0x7fa678137540 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa67000efc0 tx=0x7fa67000c3a0 comp rx=0 tx=0).stop 2023-10-03T10:16:44.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.323+0000 7fa67cf81700 1 -- 172.21.15.3:0/2145596699 shutdown_connections 2023-10-03T10:16:44.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.323+0000 7fa67cf81700 1 --2- 172.21.15.3:0/2145596699 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa67809e410 0x7fa678137540 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:44.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.323+0000 7fa67cf81700 1 --2- 172.21.15.3:0/2145596699 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa67809d6f0 0x7fa678136ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:44.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.323+0000 7fa67cf81700 1 --2- 172.21.15.3:0/2145596699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa67809e000 0x7fa678137000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:44.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.323+0000 7fa67cf81700 1 -- 172.21.15.3:0/2145596699 >> 172.21.15.3:0/2145596699 conn(0x7fa678066f50 msgr2=0x7fa678070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:44.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.323+0000 7fa67cf81700 1 -- 172.21.15.3:0/2145596699 shutdown_connections 2023-10-03T10:16:44.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:44.323+0000 7fa67cf81700 1 -- 172.21.15.3:0/2145596699 wait complete. 2023-10-03T10:16:44.334 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:44.338 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:45.049 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.195+0000 7fa975b82700 1 Processor -- start 2023-10-03T10:16:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.195+0000 7fa975b82700 1 -- start start 2023-10-03T10:16:45.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.196+0000 7fa975b82700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa97009f410 0x7fa970136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:45.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.196+0000 7fa975b82700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa970136fe0 0x7fa9701393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:45.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.196+0000 7fa975b82700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa970070940 con 0x7fa97009e930 2023-10-03T10:16:45.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.196+0000 7fa975b82700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa97006fd60 con 0x7fa97009f410 2023-10-03T10:16:45.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.196+0000 7fa975b82700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa97006fa80 con 0x7fa970136fe0 2023-10-03T10:16:45.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.196+0000 7fa96effd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa97009f410 0x7fa970136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:45.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.196+0000 7fa96effd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa97009f410 0x7fa970136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48344/0 (socket says 172.21.15.3:48344) 2023-10-03T10:16:45.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.196+0000 7fa96ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa970136fe0 0x7fa9701393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:45.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.196+0000 7fa96effd700 1 -- 172.21.15.3:0/669732850 learned_addr learned my addr 172.21.15.3:0/669732850 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:45.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.197+0000 7fa96ffff700 1 -- 172.21.15.3:0/669732850 >> v2:172.21.15.3:3300/0 conn(0x7fa97009f410 msgr2=0x7fa970136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:45.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.197+0000 7fa96ffff700 1 --2- 172.21.15.3:0/669732850 >> v2:172.21.15.3:3300/0 conn(0x7fa97009f410 0x7fa970136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.197+0000 7fa96ffff700 1 -- 172.21.15.3:0/669732850 >> v1:172.21.15.3:6789/0 conn(0x7fa97009e930 legacy=0x7fa97009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:45.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.197+0000 7fa96ffff700 1 -- 172.21.15.3:0/669732850 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa97009fb60 con 0x7fa970136fe0 2023-10-03T10:16:45.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.197+0000 7fa96effd700 1 --2- 172.21.15.3:0/669732850 >> v2:172.21.15.3:3300/0 conn(0x7fa97009f410 0x7fa970136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:45.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.197+0000 7fa96ffff700 1 --2- 172.21.15.3:0/669732850 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa970136fe0 0x7fa9701393c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa968009d90 tx=0x7fa96800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebe49905aa65d031 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:45.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.197+0000 7fa96e7fc700 1 -- 172.21.15.3:0/669732850 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96800e070 con 0x7fa970136fe0 2023-10-03T10:16:45.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.197+0000 7fa96e7fc700 1 -- 172.21.15.3:0/669732850 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa96800bb20 con 0x7fa970136fe0 2023-10-03T10:16:45.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.197+0000 7fa96e7fc700 1 -- 172.21.15.3:0/669732850 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa968012430 con 0x7fa970136fe0 2023-10-03T10:16:45.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.197+0000 7fa975b82700 1 -- 172.21.15.3:0/669732850 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa970136fe0 msgr2=0x7fa9701393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:45.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.197+0000 7fa975b82700 1 --2- 172.21.15.3:0/669732850 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa970136fe0 0x7fa9701393c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa968009d90 tx=0x7fa96800b330 comp rx=0 tx=0).stop 2023-10-03T10:16:45.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.198+0000 7fa975b82700 1 -- 172.21.15.3:0/669732850 shutdown_connections 2023-10-03T10:16:45.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.198+0000 7fa975b82700 1 --2- 172.21.15.3:0/669732850 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa970136fe0 0x7fa9701393c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.198+0000 7fa975b82700 1 --2- 172.21.15.3:0/669732850 >> v2:172.21.15.3:3300/0 conn(0x7fa97009f410 0x7fa970136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.198+0000 7fa975b82700 1 -- 172.21.15.3:0/669732850 >> 172.21.15.3:0/669732850 conn(0x7fa9700a4210 msgr2=0x7fa9700a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:45.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.198+0000 7fa975b82700 1 -- 172.21.15.3:0/669732850 shutdown_connections 2023-10-03T10:16:45.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.198+0000 7fa975b82700 1 -- 172.21.15.3:0/669732850 wait complete. 2023-10-03T10:16:45.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.199+0000 7fa975b82700 1 Processor -- start 2023-10-03T10:16:45.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.199+0000 7fa975b82700 1 -- start start 2023-10-03T10:16:45.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.199+0000 7fa975b82700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa97009e930 0x7fa97011c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:45.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.199+0000 7fa975b82700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa97009f410 0x7fa97011ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:45.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.199+0000 7fa975b82700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa970136fe0 0x7fa97011cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:45.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.199+0000 7fa975b82700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa970071db0 con 0x7fa970136fe0 2023-10-03T10:16:45.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.199+0000 7fa975b82700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9700a33f0 con 0x7fa97009e930 2023-10-03T10:16:45.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.199+0000 7fa975b82700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa97011f820 con 0x7fa97009f410 2023-10-03T10:16:45.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.199+0000 7fa96ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa970136fe0 0x7fa97011cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:45.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.199+0000 7fa96effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa97009f410 0x7fa97011ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:45.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.199+0000 7fa96f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa97009e930 0x7fa97011c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:45.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.199+0000 7fa96ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa970136fe0 0x7fa97011cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48346/0 (socket says 172.21.15.3:48346) 2023-10-03T10:16:45.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.200+0000 7fa96ffff700 1 -- 172.21.15.3:0/2476569504 learned_addr learned my addr 172.21.15.3:0/2476569504 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:45.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.200+0000 7fa96f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa97009e930 0x7fa97011c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53024/0 (socket says 172.21.15.3:53024) 2023-10-03T10:16:45.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.200+0000 7fa96effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa97009f410 0x7fa97011ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51602/0 (socket says 172.21.15.3:51602) 2023-10-03T10:16:45.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.200+0000 7fa96ffff700 1 -- 172.21.15.3:0/2476569504 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa97009f410 msgr2=0x7fa97011ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:45.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.200+0000 7fa96ffff700 1 --2- 172.21.15.3:0/2476569504 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa97009f410 0x7fa97011ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.200+0000 7fa96ffff700 1 -- 172.21.15.3:0/2476569504 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa97009e930 msgr2=0x7fa97011c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:45.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.200+0000 7fa96ffff700 1 --2- 172.21.15.3:0/2476569504 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa97009e930 0x7fa97011c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.200+0000 7fa96ffff700 1 -- 172.21.15.3:0/2476569504 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa958009550 con 0x7fa970136fe0 2023-10-03T10:16:45.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.200+0000 7fa96effd700 1 --2- 172.21.15.3:0/2476569504 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa97009f410 0x7fa97011ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:45.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.200+0000 7fa96ffff700 1 --2- 172.21.15.3:0/2476569504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa970136fe0 0x7fa97011cfd0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fa968014040 tx=0x7fa968004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:45.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.200+0000 7fa967fff700 1 -- 172.21.15.3:0/2476569504 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa968012430 con 0x7fa970136fe0 2023-10-03T10:16:45.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.200+0000 7fa967fff700 1 -- 172.21.15.3:0/2476569504 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa968020070 con 0x7fa970136fe0 2023-10-03T10:16:45.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.200+0000 7fa975b82700 1 -- 172.21.15.3:0/2476569504 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa968009a70 con 0x7fa970136fe0 2023-10-03T10:16:45.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.200+0000 7fa967fff700 1 -- 172.21.15.3:0/2476569504 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa968003ea0 con 0x7fa970136fe0 2023-10-03T10:16:45.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.200+0000 7fa975b82700 1 -- 172.21.15.3:0/2476569504 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa97011fd90 con 0x7fa970136fe0 2023-10-03T10:16:45.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.201+0000 7fa975b82700 1 -- 172.21.15.3:0/2476569504 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa970074ea0 con 0x7fa970136fe0 2023-10-03T10:16:45.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.202+0000 7fa967fff700 1 -- 172.21.15.3:0/2476569504 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa968004040 con 0x7fa970136fe0 2023-10-03T10:16:45.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.202+0000 7fa967fff700 1 -- 172.21.15.3:0/2476569504 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa968022070 con 0x7fa970136fe0 2023-10-03T10:16:45.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.207+0000 7fa967fff700 1 -- 172.21.15.3:0/2476569504 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa968011570 con 0x7fa970136fe0 2023-10-03T10:16:45.224 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:45.335 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:45.339 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:45.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.363+0000 7f19c3483700 1 Processor -- start 2023-10-03T10:16:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.363+0000 7f19c3483700 1 -- start start 2023-10-03T10:16:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.364+0000 7f19c3483700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19bc09d6f0 0x7f19bc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.364+0000 7f19c3483700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19bc09e000 0x7f19bc130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.364+0000 7f19c3483700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19bc09f130 con 0x7f19bc09e410 2023-10-03T10:16:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.364+0000 7f19c3483700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc09f520 con 0x7f19bc09e000 2023-10-03T10:16:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.364+0000 7f19c3483700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc09e930 con 0x7f19bc09d6f0 2023-10-03T10:16:45.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.364+0000 7f19c0a1e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19bc09e000 0x7f19bc130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:45.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.364+0000 7f19c121f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19bc09d6f0 0x7f19bc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:45.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.364+0000 7f19c0a1e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19bc09e000 0x7f19bc130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53032/0 (socket says 172.21.15.3:53032) 2023-10-03T10:16:45.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.364+0000 7f19c0a1e700 1 -- 172.21.15.3:0/3951594581 learned_addr learned my addr 172.21.15.3:0/3951594581 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:45.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.364+0000 7f19c0a1e700 1 -- 172.21.15.3:0/3951594581 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19bc09d6f0 msgr2=0x7f19bc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:45.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.364+0000 7f19c0a1e700 1 --2- 172.21.15.3:0/3951594581 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19bc09d6f0 0x7f19bc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.364+0000 7f19c0a1e700 1 -- 172.21.15.3:0/3951594581 >> v1:172.21.15.3:6789/0 conn(0x7f19bc09e410 legacy=0x7f19bc130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:45.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.365+0000 7f19c0a1e700 1 -- 172.21.15.3:0/3951594581 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19bc09b2d0 con 0x7f19bc09e000 2023-10-03T10:16:45.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.365+0000 7f19c0a1e700 1 --2- 172.21.15.3:0/3951594581 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19bc09e000 0x7f19bc130660 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f19ac009920 tx=0x7f19ac009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b605f8e3ac1e86d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:45.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.365+0000 7f19b3fff700 1 -- 172.21.15.3:0/3951594581 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac012b80 con 0x7f19bc09e000 2023-10-03T10:16:45.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.365+0000 7f19b3fff700 1 -- 172.21.15.3:0/3951594581 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ac012ce0 con 0x7f19bc09e000 2023-10-03T10:16:45.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.365+0000 7f19b3fff700 1 -- 172.21.15.3:0/3951594581 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac01b620 con 0x7f19bc09e000 2023-10-03T10:16:45.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.366+0000 7f19c3483700 1 -- 172.21.15.3:0/3951594581 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19bc09e000 msgr2=0x7f19bc130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.366+0000 7f19c3483700 1 --2- 172.21.15.3:0/3951594581 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19bc09e000 0x7f19bc130660 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f19ac009920 tx=0x7f19ac009c30 comp rx=0 tx=0).stop 2023-10-03T10:16:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.366+0000 7f19c3483700 1 -- 172.21.15.3:0/3951594581 shutdown_connections 2023-10-03T10:16:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.366+0000 7f19c3483700 1 --2- 172.21.15.3:0/3951594581 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19bc09d6f0 0x7f19bc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.366+0000 7f19c3483700 1 --2- 172.21.15.3:0/3951594581 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19bc09e000 0x7f19bc130660 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.366+0000 7f19c3483700 1 -- 172.21.15.3:0/3951594581 >> 172.21.15.3:0/3951594581 conn(0x7f19bc066f50 msgr2=0x7f19bc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.366+0000 7f19c3483700 1 -- 172.21.15.3:0/3951594581 shutdown_connections 2023-10-03T10:16:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.366+0000 7f19c3483700 1 -- 172.21.15.3:0/3951594581 wait complete. 2023-10-03T10:16:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.368+0000 7f19c3483700 1 Processor -- start 2023-10-03T10:16:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.368+0000 7f19c3483700 1 -- start start 2023-10-03T10:16:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.369+0000 7f19c3483700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19bc09d6f0 0x7f19bc11dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.369+0000 7f19c3483700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19bc09e000 0x7f19bc11e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.369+0000 7f19c3483700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19bc09e410 0x7f19bc11ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.369+0000 7f19c3483700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc11f480 con 0x7f19bc09d6f0 2023-10-03T10:16:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.369+0000 7f19c3483700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc130390 con 0x7f19bc09e000 2023-10-03T10:16:45.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.369+0000 7f19c3483700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc1201e0 con 0x7f19bc09e410 2023-10-03T10:16:45.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.369+0000 7f19c0a1e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19bc09e000 0x7f19bc11e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:45.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.369+0000 7f19c0a1e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19bc09e000 0x7f19bc11e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53048/0 (socket says 172.21.15.3:53048) 2023-10-03T10:16:45.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.369+0000 7f19c0a1e700 1 -- 172.21.15.3:0/15649482 learned_addr learned my addr 172.21.15.3:0/15649482 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:45.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.369+0000 7f19c121f700 1 --2- 172.21.15.3:0/15649482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19bc09d6f0 0x7f19bc11dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:45.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.369+0000 7f19c1a20700 1 --2- 172.21.15.3:0/15649482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19bc09e410 0x7f19bc11ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:45.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.369+0000 7f19c0a1e700 1 -- 172.21.15.3:0/15649482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19bc09e410 msgr2=0x7f19bc11ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:45.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.369+0000 7f19c0a1e700 1 --2- 172.21.15.3:0/15649482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19bc09e410 0x7f19bc11ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.369+0000 7f19c0a1e700 1 -- 172.21.15.3:0/15649482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19bc09d6f0 msgr2=0x7f19bc11dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:45.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.369+0000 7f19c0a1e700 1 --2- 172.21.15.3:0/15649482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19bc09d6f0 0x7f19bc11dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.369+0000 7f19c0a1e700 1 -- 172.21.15.3:0/15649482 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ac009550 con 0x7f19bc09e000 2023-10-03T10:16:45.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.370+0000 7f19c0a1e700 1 --2- 172.21.15.3:0/15649482 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19bc09e000 0x7f19bc11e4f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f19ac009c70 tx=0x7f19ac004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:45.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.370+0000 7f19b1ffb700 1 -- 172.21.15.3:0/15649482 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac020070 con 0x7f19bc09e000 2023-10-03T10:16:45.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.370+0000 7f19b1ffb700 1 -- 172.21.15.3:0/15649482 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ac0038b0 con 0x7f19bc09e000 2023-10-03T10:16:45.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.370+0000 7f19b1ffb700 1 -- 172.21.15.3:0/15649482 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac003c20 con 0x7f19bc09e000 2023-10-03T10:16:45.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.370+0000 7f19c3483700 1 -- 172.21.15.3:0/15649482 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19bc1217f0 con 0x7f19bc09e000 2023-10-03T10:16:45.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.370+0000 7f19c3483700 1 -- 172.21.15.3:0/15649482 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19bc09c700 con 0x7f19bc09e000 2023-10-03T10:16:45.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.371+0000 7f19b1ffb700 1 -- 172.21.15.3:0/15649482 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f19ac01b400 con 0x7f19bc09e000 2023-10-03T10:16:45.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.371+0000 7f19b1ffb700 1 -- 172.21.15.3:0/15649482 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f19ac044430 con 0x7f19bc09e000 2023-10-03T10:16:45.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.372+0000 7f19c3483700 1 -- 172.21.15.3:0/15649482 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19bc074ea0 con 0x7f19bc09e000 2023-10-03T10:16:45.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.377+0000 7f19b1ffb700 1 -- 172.21.15.3:0/15649482 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f19ac01b6b0 con 0x7f19bc09e000 2023-10-03T10:16:45.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.436+0000 7fa975b82700 1 -- 172.21.15.3:0/2476569504 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa97011fbd0 con 0x7fa970136fe0 2023-10-03T10:16:45.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.437+0000 7fa967fff700 1 -- 172.21.15.3:0/2476569504 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa968035090 con 0x7fa970136fe0 2023-10-03T10:16:45.437 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:45.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.439+0000 7fa975b82700 1 -- 172.21.15.3:0/2476569504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa970136fe0 msgr2=0x7fa97011cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:45.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.439+0000 7fa975b82700 1 --2- 172.21.15.3:0/2476569504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa970136fe0 0x7fa97011cfd0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fa968014040 tx=0x7fa968004ad0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.439+0000 7fa975b82700 1 -- 172.21.15.3:0/2476569504 shutdown_connections 2023-10-03T10:16:45.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.439+0000 7fa975b82700 1 --2- 172.21.15.3:0/2476569504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa970136fe0 0x7fa97011cfd0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.439+0000 7fa975b82700 1 --2- 172.21.15.3:0/2476569504 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa97009e930 0x7fa97011c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.439+0000 7fa975b82700 1 --2- 172.21.15.3:0/2476569504 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa97009f410 0x7fa97011ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.439+0000 7fa975b82700 1 -- 172.21.15.3:0/2476569504 >> 172.21.15.3:0/2476569504 conn(0x7fa9700a4210 msgr2=0x7fa970135c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:45.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.440+0000 7fa975b82700 1 -- 172.21.15.3:0/2476569504 shutdown_connections 2023-10-03T10:16:45.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.440+0000 7fa975b82700 1 -- 172.21.15.3:0/2476569504 wait complete. 2023-10-03T10:16:45.455 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:45.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.480+0000 7f462d061700 1 Processor -- start 2023-10-03T10:16:45.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.480+0000 7f462d061700 1 -- start start 2023-10-03T10:16:45.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.481+0000 7f462d061700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4628133490 0x7f4628135870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:45.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.481+0000 7f462d061700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4628135db0 0x7f4628138190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:45.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.481+0000 7f462d061700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f462809c190 con 0x7f4628130750 2023-10-03T10:16:45.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.481+0000 7f462d061700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462809b090 con 0x7f4628135db0 2023-10-03T10:16:45.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.481+0000 7f462d061700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f462809a6c0 con 0x7f4628133490 2023-10-03T10:16:45.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.481+0000 7f4626d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4628130750 0x7f4628130b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53148/0 (socket says 172.21.15.3:53148) 2023-10-03T10:16:45.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.481+0000 7f4626d9d700 1 -- 172.21.15.3:0/1886087282 learned_addr learned my addr 172.21.15.3:0/1886087282 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:45.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.481+0000 7f462659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4628133490 0x7f4628135870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:45.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.481+0000 7f462759e700 1 --2- 172.21.15.3:0/1886087282 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4628135db0 0x7f4628138190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:45.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.481+0000 7f462659c700 1 -- 172.21.15.3:0/1886087282 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4628135db0 msgr2=0x7f4628138190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:45.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.481+0000 7f462659c700 1 --2- 172.21.15.3:0/1886087282 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4628135db0 0x7f4628138190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.481+0000 7f462659c700 1 -- 172.21.15.3:0/1886087282 >> v1:172.21.15.3:6789/0 conn(0x7f4628130750 legacy=0x7f4628130b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.481+0000 7f4625d9b700 1 -- 172.21.15.3:0/1886087282 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2573713659 0 0) 0x7f462809c190 con 0x7f4628130750 2023-10-03T10:16:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.481+0000 7f462659c700 1 -- 172.21.15.3:0/1886087282 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4628130300 con 0x7f4628133490 2023-10-03T10:16:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.482+0000 7f462659c700 1 --2- 172.21.15.3:0/1886087282 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4628133490 0x7f4628135870 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f461c009870 tx=0x7f461c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c787c8a728f0d26e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.482+0000 7f4625d9b700 1 -- 172.21.15.3:0/1886087282 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c011830 con 0x7f4628133490 2023-10-03T10:16:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.482+0000 7f4625d9b700 1 -- 172.21.15.3:0/1886087282 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461c011990 con 0x7f4628133490 2023-10-03T10:16:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.482+0000 7f4625d9b700 1 -- 172.21.15.3:0/1886087282 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c011d20 con 0x7f4628133490 2023-10-03T10:16:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.483+0000 7f462d061700 1 -- 172.21.15.3:0/1886087282 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4628133490 msgr2=0x7f4628135870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.483+0000 7f462d061700 1 --2- 172.21.15.3:0/1886087282 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4628133490 0x7f4628135870 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f461c009870 tx=0x7f461c009b80 comp rx=0 tx=0).stop 2023-10-03T10:16:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.483+0000 7f462d061700 1 -- 172.21.15.3:0/1886087282 shutdown_connections 2023-10-03T10:16:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.483+0000 7f462d061700 1 --2- 172.21.15.3:0/1886087282 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4628135db0 0x7f4628138190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.483+0000 7f462d061700 1 --2- 172.21.15.3:0/1886087282 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4628133490 0x7f4628135870 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.483+0000 7f462d061700 1 -- 172.21.15.3:0/1886087282 >> 172.21.15.3:0/1886087282 conn(0x7f462809f470 msgr2=0x7f46280a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:45.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.483+0000 7f462d061700 1 -- 172.21.15.3:0/1886087282 shutdown_connections 2023-10-03T10:16:45.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.483+0000 7f462d061700 1 -- 172.21.15.3:0/1886087282 wait complete. 2023-10-03T10:16:45.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.484+0000 7f462d061700 1 Processor -- start 2023-10-03T10:16:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.484+0000 7f462d061700 1 -- start start 2023-10-03T10:16:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.485+0000 7f462d061700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4628130750 0x7f46281c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.485+0000 7f462d061700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4628133490 0x7f46281c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.485+0000 7f462d061700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4628135db0 0x7f46281c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.485+0000 7f462d061700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462812e100 con 0x7f4628130750 2023-10-03T10:16:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.485+0000 7f462d061700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462812d080 con 0x7f4628135db0 2023-10-03T10:16:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.485+0000 7f462d061700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f462809b2b0 con 0x7f4628133490 2023-10-03T10:16:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.485+0000 7f4626d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4628130750 0x7f46281c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.485+0000 7f462659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4628133490 0x7f46281c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.485+0000 7f4626d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4628130750 0x7f46281c83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48374/0 (socket says 172.21.15.3:48374) 2023-10-03T10:16:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.485+0000 7f4626d9d700 1 -- 172.21.15.3:0/245515884 learned_addr learned my addr 172.21.15.3:0/245515884 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:45.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.485+0000 7f462759e700 1 --2- 172.21.15.3:0/245515884 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4628135db0 0x7f46281c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:45.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.486+0000 7f462659c700 1 -- 172.21.15.3:0/245515884 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4628135db0 msgr2=0x7f46281c8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:45.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.486+0000 7f462659c700 1 --2- 172.21.15.3:0/245515884 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4628135db0 0x7f46281c8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.486+0000 7f462659c700 1 -- 172.21.15.3:0/245515884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4628130750 msgr2=0x7f46281c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:45.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.486+0000 7f462659c700 1 --2- 172.21.15.3:0/245515884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4628130750 0x7f46281c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.486+0000 7f462659c700 1 -- 172.21.15.3:0/245515884 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c0094a0 con 0x7f4628133490 2023-10-03T10:16:45.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.486+0000 7f462759e700 1 --2- 172.21.15.3:0/245515884 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4628135db0 0x7f46281c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:45.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.486+0000 7f4626d9d700 1 --2- 172.21.15.3:0/245515884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4628130750 0x7f46281c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:45.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.486+0000 7f462659c700 1 --2- 172.21.15.3:0/245515884 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4628133490 0x7f46281c8900 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f462806aeb0 tx=0x7f461c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:45.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.486+0000 7f460f7fe700 1 -- 172.21.15.3:0/245515884 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c004510 con 0x7f4628133490 2023-10-03T10:16:45.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.486+0000 7f460f7fe700 1 -- 172.21.15.3:0/245515884 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461c01f070 con 0x7f4628133490 2023-10-03T10:16:45.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.486+0000 7f462d061700 1 -- 172.21.15.3:0/245515884 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f462812fb50 con 0x7f4628133490 2023-10-03T10:16:45.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.486+0000 7f460f7fe700 1 -- 172.21.15.3:0/245515884 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c0038e0 con 0x7f4628133490 2023-10-03T10:16:45.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.486+0000 7f462d061700 1 -- 172.21.15.3:0/245515884 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462812f700 con 0x7f4628133490 2023-10-03T10:16:45.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.487+0000 7f460f7fe700 1 -- 172.21.15.3:0/245515884 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f461c003a40 con 0x7f4628133490 2023-10-03T10:16:45.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.487+0000 7f460f7fe700 1 -- 172.21.15.3:0/245515884 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f461c021070 con 0x7f4628133490 2023-10-03T10:16:45.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.487+0000 7f462d061700 1 -- 172.21.15.3:0/245515884 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46280a13a0 con 0x7f4628133490 2023-10-03T10:16:45.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.492+0000 7f460f7fe700 1 -- 172.21.15.3:0/245515884 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f461c015030 con 0x7f4628133490 2023-10-03T10:16:45.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.507+0000 7f5eac85b700 1 Processor -- start 2023-10-03T10:16:45.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.507+0000 7f5eac85b700 1 -- start start 2023-10-03T10:16:45.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.508+0000 7f5eac85b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5ea809d6f0 0x7f5ea809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:45.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.508+0000 7f5eac85b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ea809e410 0x7f5ea8130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:45.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.508+0000 7f5eac85b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea809f130 con 0x7f5ea809e000 2023-10-03T10:16:45.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.508+0000 7f5eac85b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea809f520 con 0x7f5ea809d6f0 2023-10-03T10:16:45.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.508+0000 7f5eac85b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea809e930 con 0x7f5ea809e410 2023-10-03T10:16:45.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.508+0000 7f5ea6d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ea809e410 0x7f5ea8130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:45.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.508+0000 7f5ea659c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5ea809d6f0 0x7f5ea809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:45.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.508+0000 7f5ea6d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ea809e410 0x7f5ea8130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51648/0 (socket says 172.21.15.3:51648) 2023-10-03T10:16:45.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.508+0000 7f5ea6d9d700 1 -- 172.21.15.3:0/2926572102 learned_addr learned my addr 172.21.15.3:0/2926572102 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:45.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.508+0000 7f5ea659c700 1 -- 172.21.15.3:0/2926572102 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ea809e410 msgr2=0x7f5ea8130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:45.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.508+0000 7f5ea659c700 1 --2- 172.21.15.3:0/2926572102 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ea809e410 0x7f5ea8130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.508+0000 7f5ea659c700 1 -- 172.21.15.3:0/2926572102 >> v1:172.21.15.3:6789/0 conn(0x7f5ea809e000 legacy=0x7f5ea8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:45.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.508+0000 7f5ea659c700 1 -- 172.21.15.3:0/2926572102 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea809b2d0 con 0x7f5ea809d6f0 2023-10-03T10:16:45.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.509+0000 7f5ea659c700 1 --2- 172.21.15.3:0/2926572102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ea809d6f0 0x7f5ea809dac0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009840 tx=0x7f5e98009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aba1ade158c0d4c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:45.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.509+0000 7f5ea559a700 1 -- 172.21.15.3:0/2926572102 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98011830 con 0x7f5ea809d6f0 2023-10-03T10:16:45.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.509+0000 7f5ea559a700 1 -- 172.21.15.3:0/2926572102 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e98011990 con 0x7f5ea809d6f0 2023-10-03T10:16:45.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.509+0000 7f5ea559a700 1 -- 172.21.15.3:0/2926572102 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98011d20 con 0x7f5ea809d6f0 2023-10-03T10:16:45.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.509+0000 7f5eac85b700 1 -- 172.21.15.3:0/2926572102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ea809d6f0 msgr2=0x7f5ea809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:45.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.509+0000 7f5eac85b700 1 --2- 172.21.15.3:0/2926572102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ea809d6f0 0x7f5ea809dac0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009840 tx=0x7f5e98009b50 comp rx=0 tx=0).stop 2023-10-03T10:16:45.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.510+0000 7f5eac85b700 1 -- 172.21.15.3:0/2926572102 shutdown_connections 2023-10-03T10:16:45.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.510+0000 7f5eac85b700 1 --2- 172.21.15.3:0/2926572102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ea809d6f0 0x7f5ea809dac0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.510+0000 7f5eac85b700 1 --2- 172.21.15.3:0/2926572102 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ea809e410 0x7f5ea8130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.510+0000 7f5eac85b700 1 -- 172.21.15.3:0/2926572102 >> 172.21.15.3:0/2926572102 conn(0x7f5ea8066f50 msgr2=0x7f5ea809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:45.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.510+0000 7f5eac85b700 1 -- 172.21.15.3:0/2926572102 shutdown_connections 2023-10-03T10:16:45.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.510+0000 7f5eac85b700 1 -- 172.21.15.3:0/2926572102 wait complete. 2023-10-03T10:16:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.511+0000 7f5eac85b700 1 Processor -- start 2023-10-03T10:16:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.511+0000 7f5eac85b700 1 -- start start 2023-10-03T10:16:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.511+0000 7f5eac85b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ea809d6f0 0x7f5ea812fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.511+0000 7f5eac85b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ea809e000 0x7f5ea8130070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.511+0000 7f5eac85b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ea809e410 0x7f5ea81305b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.511+0000 7f5eac85b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea8131cb0 con 0x7f5ea809d6f0 2023-10-03T10:16:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.511+0000 7f5eac85b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea8131000 con 0x7f5ea809e000 2023-10-03T10:16:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.511+0000 7f5eac85b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea8131ac0 con 0x7f5ea809e410 2023-10-03T10:16:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.511+0000 7f5ea5d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ea809e000 0x7f5ea8130070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.512+0000 7f5ea6d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ea809e410 0x7f5ea81305b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.512+0000 7f5ea5d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ea809e000 0x7f5ea8130070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53068/0 (socket says 172.21.15.3:53068) 2023-10-03T10:16:45.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.512+0000 7f5ea5d9b700 1 -- 172.21.15.3:0/1700570139 learned_addr learned my addr 172.21.15.3:0/1700570139 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:45.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.512+0000 7f5ea659c700 1 --2- 172.21.15.3:0/1700570139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ea809d6f0 0x7f5ea812fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:45.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.512+0000 7f5ea5d9b700 1 -- 172.21.15.3:0/1700570139 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ea809e410 msgr2=0x7f5ea81305b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:45.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.512+0000 7f5ea5d9b700 1 --2- 172.21.15.3:0/1700570139 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ea809e410 0x7f5ea81305b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.512+0000 7f5ea5d9b700 1 -- 172.21.15.3:0/1700570139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ea809d6f0 msgr2=0x7f5ea812fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:45.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.512+0000 7f5ea5d9b700 1 --2- 172.21.15.3:0/1700570139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ea809d6f0 0x7f5ea812fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.512+0000 7f5ea5d9b700 1 -- 172.21.15.3:0/1700570139 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e980094a0 con 0x7f5ea809e000 2023-10-03T10:16:45.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.512+0000 7f5ea6d9d700 1 --2- 172.21.15.3:0/1700570139 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ea809e410 0x7f5ea81305b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:45.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.512+0000 7f5ea5d9b700 1 --2- 172.21.15.3:0/1700570139 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ea809e000 0x7f5ea8130070 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00c4e0 tx=0x7f5e9c00c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:45.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.512+0000 7f5e96ffd700 1 -- 172.21.15.3:0/1700570139 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c0052d0 con 0x7f5ea809e000 2023-10-03T10:16:45.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.512+0000 7f5e96ffd700 1 -- 172.21.15.3:0/1700570139 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9c005430 con 0x7f5ea809e000 2023-10-03T10:16:45.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.512+0000 7f5eac85b700 1 -- 172.21.15.3:0/1700570139 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea8133a30 con 0x7f5ea809e000 2023-10-03T10:16:45.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.512+0000 7f5e96ffd700 1 -- 172.21.15.3:0/1700570139 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c0038c0 con 0x7f5ea809e000 2023-10-03T10:16:45.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.513+0000 7f5eac85b700 1 -- 172.21.15.3:0/1700570139 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea81336f0 con 0x7f5ea809e000 2023-10-03T10:16:45.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.513+0000 7f5e96ffd700 1 -- 172.21.15.3:0/1700570139 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5e9c003a20 con 0x7f5ea809e000 2023-10-03T10:16:45.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.514+0000 7f5e96ffd700 1 -- 172.21.15.3:0/1700570139 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5e9c03e9d0 con 0x7f5ea809e000 2023-10-03T10:16:45.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.514+0000 7f5eac85b700 1 -- 172.21.15.3:0/1700570139 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea8132310 con 0x7f5ea809e000 2023-10-03T10:16:45.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.518+0000 7f5e96ffd700 1 -- 172.21.15.3:0/1700570139 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5e9c017070 con 0x7f5ea809e000 2023-10-03T10:16:45.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.609+0000 7f19c3483700 1 -- 172.21.15.3:0/15649482 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f19bc06c360 con 0x7f19bc09e000 2023-10-03T10:16:45.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.610+0000 7f19b1ffb700 1 -- 172.21.15.3:0/15649482 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f19ac02a020 con 0x7f19bc09e000 2023-10-03T10:16:45.610 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:45.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.612+0000 7f19c3483700 1 -- 172.21.15.3:0/15649482 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19bc09e000 msgr2=0x7f19bc11e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:45.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.612+0000 7f19c3483700 1 --2- 172.21.15.3:0/15649482 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19bc09e000 0x7f19bc11e4f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f19ac009c70 tx=0x7f19ac004750 comp rx=0 tx=0).stop 2023-10-03T10:16:45.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.612+0000 7f19c3483700 1 -- 172.21.15.3:0/15649482 shutdown_connections 2023-10-03T10:16:45.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.612+0000 7f19c3483700 1 --2- 172.21.15.3:0/15649482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19bc09e410 0x7f19bc11ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.612+0000 7f19c3483700 1 --2- 172.21.15.3:0/15649482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19bc09d6f0 0x7f19bc11dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.612+0000 7f19c3483700 1 --2- 172.21.15.3:0/15649482 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19bc09e000 0x7f19bc11e4f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.612+0000 7f19c3483700 1 -- 172.21.15.3:0/15649482 >> 172.21.15.3:0/15649482 conn(0x7f19bc066f50 msgr2=0x7f19bc0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:45.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.612+0000 7f19c3483700 1 -- 172.21.15.3:0/15649482 shutdown_connections 2023-10-03T10:16:45.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.612+0000 7f19c3483700 1 -- 172.21.15.3:0/15649482 wait complete. 2023-10-03T10:16:45.626 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:45.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.702+0000 7f462d061700 1 -- 172.21.15.3:0/245515884 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4628130f70 con 0x7f4628133490 2023-10-03T10:16:45.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.703+0000 7f460f7fe700 1 -- 172.21.15.3:0/245515884 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f461c010b70 con 0x7f4628133490 2023-10-03T10:16:45.703 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:45.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.705+0000 7f462d061700 1 -- 172.21.15.3:0/245515884 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4628133490 msgr2=0x7f46281c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:45.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.705+0000 7f462d061700 1 --2- 172.21.15.3:0/245515884 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4628133490 0x7f46281c8900 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f462806aeb0 tx=0x7f461c004750 comp rx=0 tx=0).stop 2023-10-03T10:16:45.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.705+0000 7f462d061700 1 -- 172.21.15.3:0/245515884 shutdown_connections 2023-10-03T10:16:45.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.705+0000 7f462d061700 1 --2- 172.21.15.3:0/245515884 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4628135db0 0x7f46281c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.705+0000 7f462d061700 1 --2- 172.21.15.3:0/245515884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4628130750 0x7f46281c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.705+0000 7f462d061700 1 --2- 172.21.15.3:0/245515884 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4628133490 0x7f46281c8900 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.705+0000 7f462d061700 1 -- 172.21.15.3:0/245515884 >> 172.21.15.3:0/245515884 conn(0x7f462809f470 msgr2=0x7f462809c900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:45.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.705+0000 7f462d061700 1 -- 172.21.15.3:0/245515884 shutdown_connections 2023-10-03T10:16:45.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.705+0000 7f462d061700 1 -- 172.21.15.3:0/245515884 wait complete. 2023-10-03T10:16:45.719 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:45.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.726+0000 7f5eac85b700 1 -- 172.21.15.3:0/1700570139 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5ea8074ea0 con 0x7f5ea809e000 2023-10-03T10:16:45.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.726+0000 7f5e96ffd700 1 -- 172.21.15.3:0/1700570139 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5e9c017070 con 0x7f5ea809e000 2023-10-03T10:16:45.727 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:45.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.729+0000 7f5eac85b700 1 -- 172.21.15.3:0/1700570139 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ea809e000 msgr2=0x7f5ea8130070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:45.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.729+0000 7f5eac85b700 1 --2- 172.21.15.3:0/1700570139 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ea809e000 0x7f5ea8130070 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00c4e0 tx=0x7f5e9c00c7f0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.729+0000 7f5eac85b700 1 -- 172.21.15.3:0/1700570139 shutdown_connections 2023-10-03T10:16:45.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.729+0000 7f5eac85b700 1 --2- 172.21.15.3:0/1700570139 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ea809e410 0x7f5ea81305b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.729+0000 7f5eac85b700 1 --2- 172.21.15.3:0/1700570139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ea809d6f0 0x7f5ea812fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.729+0000 7f5eac85b700 1 --2- 172.21.15.3:0/1700570139 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ea809e000 0x7f5ea8130070 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:45.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.729+0000 7f5eac85b700 1 -- 172.21.15.3:0/1700570139 >> 172.21.15.3:0/1700570139 conn(0x7f5ea8066f50 msgr2=0x7f5ea80a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:45.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.729+0000 7f5eac85b700 1 -- 172.21.15.3:0/1700570139 shutdown_connections 2023-10-03T10:16:45.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:45.729+0000 7f5eac85b700 1 -- 172.21.15.3:0/1700570139 wait complete. 2023-10-03T10:16:45.743 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:46.456 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:46.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.602+0000 7f8feb11f700 1 Processor -- start 2023-10-03T10:16:46.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.602+0000 7f8feb11f700 1 -- start start 2023-10-03T10:16:46.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.602+0000 7f8feb11f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8fe409d6f0 0x7f8fe409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:46.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.602+0000 7f8feb11f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8fe409e000 0x7f8fe41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:46.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.602+0000 7f8feb11f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fe409b120 con 0x7f8fe409e410 2023-10-03T10:16:46.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.602+0000 7f8feb11f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fe406b460 con 0x7f8fe409d6f0 2023-10-03T10:16:46.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.602+0000 7f8feb11f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe409a7e0 con 0x7f8fe409e000 2023-10-03T10:16:46.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.602+0000 7f8fe8ebb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8fe409d6f0 0x7f8fe409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:46.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.602+0000 7f8fe3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8fe409e000 0x7f8fe41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:46.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.602+0000 7f8fe8ebb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8fe409d6f0 0x7f8fe409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48400/0 (socket says 172.21.15.3:48400) 2023-10-03T10:16:46.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.602+0000 7f8fe8ebb700 1 -- 172.21.15.3:0/2705965718 learned_addr learned my addr 172.21.15.3:0/2705965718 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:46.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.602+0000 7f8fe3fff700 1 --2- 172.21.15.3:0/2705965718 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8fe409e000 0x7f8fe41339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51664/0 (socket says 172.21.15.3:51664) 2023-10-03T10:16:46.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.603+0000 7f8fe8ebb700 1 -- 172.21.15.3:0/2705965718 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8fe409e000 msgr2=0x7f8fe41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:46.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.603+0000 7f8fe8ebb700 1 --2- 172.21.15.3:0/2705965718 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8fe409e000 0x7f8fe41339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:46.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.603+0000 7f8fe8ebb700 1 -- 172.21.15.3:0/2705965718 >> v1:172.21.15.3:6789/0 conn(0x7f8fe409e410 legacy=0x7f8fe4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:46.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.603+0000 7f8fe8ebb700 1 -- 172.21.15.3:0/2705965718 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe409a500 con 0x7f8fe409d6f0 2023-10-03T10:16:46.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.603+0000 7f8fe3fff700 1 --2- 172.21.15.3:0/2705965718 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8fe409e000 0x7f8fe41339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:46.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.603+0000 7f8fe8ebb700 1 --2- 172.21.15.3:0/2705965718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fe409d6f0 0x7f8fe409dac0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f8fd40098f0 tx=0x7f8fd4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee8f50ee4bf4dc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:46.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.603+0000 7f8fe37fe700 1 -- 172.21.15.3:0/2705965718 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4011830 con 0x7f8fe409d6f0 2023-10-03T10:16:46.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.603+0000 7f8fe37fe700 1 -- 172.21.15.3:0/2705965718 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd4011990 con 0x7f8fe409d6f0 2023-10-03T10:16:46.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.603+0000 7f8fe37fe700 1 -- 172.21.15.3:0/2705965718 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4011d20 con 0x7f8fe409d6f0 2023-10-03T10:16:46.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.604+0000 7f8feb11f700 1 -- 172.21.15.3:0/2705965718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fe409d6f0 msgr2=0x7f8fe409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:46.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.604+0000 7f8feb11f700 1 --2- 172.21.15.3:0/2705965718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fe409d6f0 0x7f8fe409dac0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f8fd40098f0 tx=0x7f8fd4009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:46.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.604+0000 7f8feb11f700 1 -- 172.21.15.3:0/2705965718 shutdown_connections 2023-10-03T10:16:46.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.604+0000 7f8feb11f700 1 --2- 172.21.15.3:0/2705965718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fe409d6f0 0x7f8fe409dac0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:46.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.604+0000 7f8feb11f700 1 --2- 172.21.15.3:0/2705965718 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8fe409e000 0x7f8fe41339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:46.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.604+0000 7f8feb11f700 1 -- 172.21.15.3:0/2705965718 >> 172.21.15.3:0/2705965718 conn(0x7f8fe4066f50 msgr2=0x7f8fe409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:46.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.604+0000 7f8feb11f700 1 -- 172.21.15.3:0/2705965718 shutdown_connections 2023-10-03T10:16:46.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.604+0000 7f8feb11f700 1 -- 172.21.15.3:0/2705965718 wait complete. 2023-10-03T10:16:46.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.605+0000 7f8feb11f700 1 Processor -- start 2023-10-03T10:16:46.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.605+0000 7f8feb11f700 1 -- start start 2023-10-03T10:16:46.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.605+0000 7f8feb11f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8fe409d6f0 0x7f8fe41a5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.606+0000 7f8feb11f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8fe409e000 0x7f8fe41a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.606+0000 7f8feb11f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fe409e410 0x7f8fe41a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.606+0000 7f8feb11f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe41a7410 con 0x7f8fe409e410 2023-10-03T10:16:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.606+0000 7f8feb11f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe41a7b20 con 0x7f8fe409d6f0 2023-10-03T10:16:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.606+0000 7f8feb11f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe41a8d70 con 0x7f8fe409e000 2023-10-03T10:16:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.606+0000 7f8fe3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8fe409e000 0x7f8fe41a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.606+0000 7f8fe96bc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fe409e410 0x7f8fe41a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.606+0000 7f8fe8ebb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8fe409d6f0 0x7f8fe41a5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.606+0000 7f8fe96bc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fe409e410 0x7f8fe41a69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48416/0 (socket says 172.21.15.3:48416) 2023-10-03T10:16:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.606+0000 7f8fe3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8fe409e000 0x7f8fe41a64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51674/0 (socket says 172.21.15.3:51674) 2023-10-03T10:16:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.606+0000 7f8fe96bc700 1 -- 172.21.15.3:0/1378794460 learned_addr learned my addr 172.21.15.3:0/1378794460 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:46.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.606+0000 7f8fe3fff700 1 -- 172.21.15.3:0/1378794460 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8fe409d6f0 msgr2=0x7f8fe41a5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:46.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.606+0000 7f8fe3fff700 1 --2- 172.21.15.3:0/1378794460 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8fe409d6f0 0x7f8fe41a5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:46.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.606+0000 7f8fe3fff700 1 -- 172.21.15.3:0/1378794460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fe409e410 msgr2=0x7f8fe41a69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:46.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.606+0000 7f8fe3fff700 1 --2- 172.21.15.3:0/1378794460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fe409e410 0x7f8fe41a69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:46.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.606+0000 7f8fe3fff700 1 -- 172.21.15.3:0/1378794460 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd80094a0 con 0x7f8fe409e000 2023-10-03T10:16:46.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.606+0000 7f8fe96bc700 1 --2- 172.21.15.3:0/1378794460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fe409e410 0x7f8fe41a69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:46.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.606+0000 7f8fe3fff700 1 --2- 172.21.15.3:0/1378794460 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8fe409e000 0x7f8fe41a64a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800e9e0 tx=0x7f8fd800ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:46.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.606+0000 7f8fe17fa700 1 -- 172.21.15.3:0/1378794460 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd800ee80 con 0x7f8fe409e000 2023-10-03T10:16:46.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.606+0000 7f8fe17fa700 1 -- 172.21.15.3:0/1378794460 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd800c850 con 0x7f8fe409e000 2023-10-03T10:16:46.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.606+0000 7f8feb11f700 1 -- 172.21.15.3:0/1378794460 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd4009550 con 0x7f8fe409e000 2023-10-03T10:16:46.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.606+0000 7f8fe17fa700 1 -- 172.21.15.3:0/1378794460 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd800cc00 con 0x7f8fe409e000 2023-10-03T10:16:46.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.606+0000 7f8feb11f700 1 -- 172.21.15.3:0/1378794460 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fe40a2260 con 0x7f8fe409e000 2023-10-03T10:16:46.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.608+0000 7f8fe17fa700 1 -- 172.21.15.3:0/1378794460 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8fd800cd60 con 0x7f8fe409e000 2023-10-03T10:16:46.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.608+0000 7f8feb11f700 1 -- 172.21.15.3:0/1378794460 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fe409c710 con 0x7f8fe409e000 2023-10-03T10:16:46.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.608+0000 7f8fe17fa700 1 -- 172.21.15.3:0/1378794460 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8fd803f6f0 con 0x7f8fe409e000 2023-10-03T10:16:46.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.613+0000 7f8fe17fa700 1 -- 172.21.15.3:0/1378794460 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8fd80232f0 con 0x7f8fe409e000 2023-10-03T10:16:46.627 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:46.720 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:46.744 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:46.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.767+0000 7f89eb100700 1 Processor -- start 2023-10-03T10:16:46.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.767+0000 7f89eb100700 1 -- start start 2023-10-03T10:16:46.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.768+0000 7f89eb100700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89e41306b0 0x7f89e4130a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:46.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.768+0000 7f89eb100700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f89e4099dc0 0x7f89e41385c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:46.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.768+0000 7f89eb100700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e409b720 con 0x7f89e40992c0 2023-10-03T10:16:46.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.768+0000 7f89eb100700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e409bc60 con 0x7f89e4099dc0 2023-10-03T10:16:46.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.768+0000 7f89eb100700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e41304b0 con 0x7f89e41306b0 2023-10-03T10:16:46.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.768+0000 7f89e8e9c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89e41306b0 0x7f89e4130a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:46.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.768+0000 7f89e8e9c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89e41306b0 0x7f89e4130a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53096/0 (socket says 172.21.15.3:53096) 2023-10-03T10:16:46.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.768+0000 7f89e8e9c700 1 -- 172.21.15.3:0/2486614408 learned_addr learned my addr 172.21.15.3:0/2486614408 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:46.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.768+0000 7f89e969d700 1 --2- 172.21.15.3:0/2486614408 >> v2:172.21.15.3:3300/0 conn(0x7f89e4099dc0 0x7f89e41385c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:46.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.768+0000 7f89e8e9c700 1 -- 172.21.15.3:0/2486614408 >> v2:172.21.15.3:3300/0 conn(0x7f89e4099dc0 msgr2=0x7f89e41385c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:46.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.768+0000 7f89e8e9c700 1 --2- 172.21.15.3:0/2486614408 >> v2:172.21.15.3:3300/0 conn(0x7f89e4099dc0 0x7f89e41385c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:46.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.768+0000 7f89e8e9c700 1 -- 172.21.15.3:0/2486614408 >> v1:172.21.15.3:6789/0 conn(0x7f89e40992c0 legacy=0x7f89e40996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:46.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.768+0000 7f89e8e9c700 1 -- 172.21.15.3:0/2486614408 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e4130280 con 0x7f89e41306b0 2023-10-03T10:16:46.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.769+0000 7f89e8e9c700 1 --2- 172.21.15.3:0/2486614408 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89e41306b0 0x7f89e4130a80 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f89d80098f0 tx=0x7f89d8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6c68d4bcfb6392d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:46.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.769+0000 7f89e37fe700 1 -- 172.21.15.3:0/2486614408 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d8011830 con 0x7f89e41306b0 2023-10-03T10:16:46.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.769+0000 7f89e37fe700 1 -- 172.21.15.3:0/2486614408 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d8011990 con 0x7f89e41306b0 2023-10-03T10:16:46.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.769+0000 7f89e37fe700 1 -- 172.21.15.3:0/2486614408 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d8011d20 con 0x7f89e41306b0 2023-10-03T10:16:46.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.770+0000 7f89eb100700 1 -- 172.21.15.3:0/2486614408 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89e41306b0 msgr2=0x7f89e4130a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:46.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.770+0000 7f89eb100700 1 --2- 172.21.15.3:0/2486614408 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89e41306b0 0x7f89e4130a80 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f89d80098f0 tx=0x7f89d8009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:46.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.770+0000 7f89eb100700 1 -- 172.21.15.3:0/2486614408 shutdown_connections 2023-10-03T10:16:46.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.770+0000 7f89eb100700 1 --2- 172.21.15.3:0/2486614408 >> v2:172.21.15.3:3300/0 conn(0x7f89e4099dc0 0x7f89e41385c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:46.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.770+0000 7f89eb100700 1 --2- 172.21.15.3:0/2486614408 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89e41306b0 0x7f89e4130a80 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:46.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.770+0000 7f89eb100700 1 -- 172.21.15.3:0/2486614408 >> 172.21.15.3:0/2486614408 conn(0x7f89e409e7e0 msgr2=0x7f89e409ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:46.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.770+0000 7f89eb100700 1 -- 172.21.15.3:0/2486614408 shutdown_connections 2023-10-03T10:16:46.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.770+0000 7f89eb100700 1 -- 172.21.15.3:0/2486614408 wait complete. 2023-10-03T10:16:46.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.771+0000 7f89eb100700 1 Processor -- start 2023-10-03T10:16:46.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.771+0000 7f89eb100700 1 -- start start 2023-10-03T10:16:46.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.772+0000 7f89eb100700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f89e40992c0 0x7f89e408c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:46.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.772+0000 7f89eb100700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89e4099dc0 0x7f89e408c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:46.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.772+0000 7f89eb100700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89e41306b0 0x7f89e408cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:46.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.772+0000 7f89eb100700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e408e1a0 con 0x7f89e4099dc0 2023-10-03T10:16:46.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.772+0000 7f89eb100700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e413b800 con 0x7f89e41306b0 2023-10-03T10:16:46.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.772+0000 7f89eb100700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89e408e880 con 0x7f89e40992c0 2023-10-03T10:16:46.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.772+0000 7f89e8e9c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f89e40992c0 0x7f89e408c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:46.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.772+0000 7f89e8e9c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f89e40992c0 0x7f89e408c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51682/0 (socket says 172.21.15.3:51682) 2023-10-03T10:16:46.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.772+0000 7f89e8e9c700 1 -- 172.21.15.3:0/3527058398 learned_addr learned my addr 172.21.15.3:0/3527058398 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:46.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.772+0000 7f89e8e9c700 1 -- 172.21.15.3:0/3527058398 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89e41306b0 msgr2=0x7f89e408cf10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:16:46.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.772+0000 7f89e8e9c700 1 --2- 172.21.15.3:0/3527058398 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89e41306b0 0x7f89e408cf10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:46.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.772+0000 7f89e8e9c700 1 -- 172.21.15.3:0/3527058398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89e4099dc0 msgr2=0x7f89e408c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:46.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.772+0000 7f89e8e9c700 1 --2- 172.21.15.3:0/3527058398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89e4099dc0 0x7f89e408c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:46.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.773+0000 7f89e8e9c700 1 -- 172.21.15.3:0/3527058398 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d8009550 con 0x7f89e40992c0 2023-10-03T10:16:46.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.773+0000 7f89e8e9c700 1 --2- 172.21.15.3:0/3527058398 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f89e40992c0 0x7f89e408c490 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f89d8000c00 tx=0x7f89d8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:46.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.773+0000 7f89e17fa700 1 -- 172.21.15.3:0/3527058398 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d80036a0 con 0x7f89e40992c0 2023-10-03T10:16:46.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.773+0000 7f89e17fa700 1 -- 172.21.15.3:0/3527058398 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d8003800 con 0x7f89e40992c0 2023-10-03T10:16:46.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.773+0000 7f89eb100700 1 -- 172.21.15.3:0/3527058398 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e408e5d0 con 0x7f89e40992c0 2023-10-03T10:16:46.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.773+0000 7f89e17fa700 1 -- 172.21.15.3:0/3527058398 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d8003b70 con 0x7f89e40992c0 2023-10-03T10:16:46.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.773+0000 7f89eb100700 1 -- 172.21.15.3:0/3527058398 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e408ec90 con 0x7f89e40992c0 2023-10-03T10:16:46.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.774+0000 7f89e17fa700 1 -- 172.21.15.3:0/3527058398 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f89d8021070 con 0x7f89e40992c0 2023-10-03T10:16:46.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.774+0000 7f89e17fa700 1 -- 172.21.15.3:0/3527058398 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f89d8040d70 con 0x7f89e40992c0 2023-10-03T10:16:46.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.774+0000 7f89eb100700 1 -- 172.21.15.3:0/3527058398 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e408f8f0 con 0x7f89e40992c0 2023-10-03T10:16:46.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.780+0000 7f89e17fa700 1 -- 172.21.15.3:0/3527058398 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f89e408f8f0 con 0x7f89e40992c0 2023-10-03T10:16:46.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.824+0000 7f8feb11f700 1 -- 172.21.15.3:0/1378794460 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8fe4074ea0 con 0x7f8fe409e000 2023-10-03T10:16:46.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.824+0000 7f8fe17fa700 1 -- 172.21.15.3:0/1378794460 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8fd8021030 con 0x7f8fe409e000 2023-10-03T10:16:46.825 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.827+0000 7f8feb11f700 1 -- 172.21.15.3:0/1378794460 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8fe409e000 msgr2=0x7f8fe41a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.827+0000 7f8feb11f700 1 --2- 172.21.15.3:0/1378794460 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8fe409e000 0x7f8fe41a64a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800e9e0 tx=0x7f8fd800ecf0 comp rx=0 tx=0).stop 2023-10-03T10:16:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.827+0000 7f8feb11f700 1 -- 172.21.15.3:0/1378794460 shutdown_connections 2023-10-03T10:16:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.827+0000 7f8feb11f700 1 --2- 172.21.15.3:0/1378794460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fe409e410 0x7f8fe41a69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.827+0000 7f8feb11f700 1 --2- 172.21.15.3:0/1378794460 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8fe409d6f0 0x7f8fe41a5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.827+0000 7f8feb11f700 1 --2- 172.21.15.3:0/1378794460 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8fe409e000 0x7f8fe41a64a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.827+0000 7f8feb11f700 1 -- 172.21.15.3:0/1378794460 >> 172.21.15.3:0/1378794460 conn(0x7f8fe4066f50 msgr2=0x7f8fe41baeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.827+0000 7f8feb11f700 1 -- 172.21.15.3:0/1378794460 shutdown_connections 2023-10-03T10:16:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.827+0000 7f8feb11f700 1 -- 172.21.15.3:0/1378794460 wait complete. 2023-10-03T10:16:46.843 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:46.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.866+0000 7f76f49d7700 1 Processor -- start 2023-10-03T10:16:46.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.866+0000 7f76f49d7700 1 -- start start 2023-10-03T10:16:46.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.867+0000 7f76f49d7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76f006a770 0x7f76f006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:46.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.867+0000 7f76f49d7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f76f0131d10 0x7f76f0139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:46.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.867+0000 7f76f49d7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76f009ece0 con 0x7f76f006b080 2023-10-03T10:16:46.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.867+0000 7f76f49d7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76f009e760 con 0x7f76f0131d10 2023-10-03T10:16:46.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.867+0000 7f76f49d7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76f009e040 con 0x7f76f006a770 2023-10-03T10:16:46.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.867+0000 7f76edd9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f76f006b080 0x7f76f0131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53184/0 (socket says 172.21.15.3:53184) 2023-10-03T10:16:46.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.867+0000 7f76edd9b700 1 -- 172.21.15.3:0/3554790760 learned_addr learned my addr 172.21.15.3:0/3554790760 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:46.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.867+0000 7f76eed9d700 1 --2- 172.21.15.3:0/3554790760 >> v2:172.21.15.3:3300/0 conn(0x7f76f0131d10 0x7f76f0139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:46.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.867+0000 7f76ee59c700 1 --2- 172.21.15.3:0/3554790760 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76f006a770 0x7f76f006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:46.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.868+0000 7f76ee59c700 1 -- 172.21.15.3:0/3554790760 >> v2:172.21.15.3:3300/0 conn(0x7f76f0131d10 msgr2=0x7f76f0139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:46.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.868+0000 7f76ee59c700 1 --2- 172.21.15.3:0/3554790760 >> v2:172.21.15.3:3300/0 conn(0x7f76f0131d10 0x7f76f0139860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:46.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.868+0000 7f76ee59c700 1 -- 172.21.15.3:0/3554790760 >> v1:172.21.15.3:6789/0 conn(0x7f76f006b080 legacy=0x7f76f0131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:46.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.868+0000 7f76ee59c700 1 -- 172.21.15.3:0/3554790760 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f009f630 con 0x7f76f006a770 2023-10-03T10:16:46.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.868+0000 7f76ed59a700 1 -- 172.21.15.3:0/3554790760 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4038259257 0 0) 0x7f76f009ece0 con 0x7f76f006b080 2023-10-03T10:16:46.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.868+0000 7f76eed9d700 1 --2- 172.21.15.3:0/3554790760 >> v2:172.21.15.3:3300/0 conn(0x7f76f0131d10 0x7f76f0139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:46.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.868+0000 7f76ee59c700 1 --2- 172.21.15.3:0/3554790760 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76f006a770 0x7f76f006ab40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009840 tx=0x7f76e0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a42805a3ee8dd41 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:46.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.868+0000 7f76ed59a700 1 -- 172.21.15.3:0/3554790760 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e0011830 con 0x7f76f006a770 2023-10-03T10:16:46.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.868+0000 7f76ed59a700 1 -- 172.21.15.3:0/3554790760 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e0011990 con 0x7f76f006a770 2023-10-03T10:16:46.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.868+0000 7f76ed59a700 1 -- 172.21.15.3:0/3554790760 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e0011d20 con 0x7f76f006a770 2023-10-03T10:16:46.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.869+0000 7f76f49d7700 1 -- 172.21.15.3:0/3554790760 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76f006a770 msgr2=0x7f76f006ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:46.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.869+0000 7f76f49d7700 1 --2- 172.21.15.3:0/3554790760 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76f006a770 0x7f76f006ab40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009840 tx=0x7f76e0009b50 comp rx=0 tx=0).stop 2023-10-03T10:16:46.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.869+0000 7f76f49d7700 1 -- 172.21.15.3:0/3554790760 shutdown_connections 2023-10-03T10:16:46.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.869+0000 7f76f49d7700 1 --2- 172.21.15.3:0/3554790760 >> v2:172.21.15.3:3300/0 conn(0x7f76f0131d10 0x7f76f0139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:46.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.869+0000 7f76f49d7700 1 --2- 172.21.15.3:0/3554790760 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76f006a770 0x7f76f006ab40 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:46.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.869+0000 7f76f49d7700 1 -- 172.21.15.3:0/3554790760 >> 172.21.15.3:0/3554790760 conn(0x7f76f0066f50 msgr2=0x7f76f009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:46.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.869+0000 7f76f49d7700 1 -- 172.21.15.3:0/3554790760 shutdown_connections 2023-10-03T10:16:46.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.869+0000 7f76f49d7700 1 -- 172.21.15.3:0/3554790760 wait complete. 2023-10-03T10:16:46.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.870+0000 7f76f49d7700 1 Processor -- start 2023-10-03T10:16:46.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.870+0000 7f76f49d7700 1 -- start start 2023-10-03T10:16:46.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.871+0000 7f76f49d7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76f006a770 0x7f76f0120560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:46.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.871+0000 7f76f49d7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76f006b080 0x7f76f0120aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:46.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.871+0000 7f76f49d7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76f0131d10 0x7f76f0120fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:46.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.871+0000 7f76f49d7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f0122120 con 0x7f76f006a770 2023-10-03T10:16:46.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.871+0000 7f76f49d7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f0123d50 con 0x7f76f006b080 2023-10-03T10:16:46.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.871+0000 7f76f49d7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76f0124270 con 0x7f76f0131d10 2023-10-03T10:16:46.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.871+0000 7f76ee59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76f006a770 0x7f76f0120560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:46.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.871+0000 7f76ee59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76f006a770 0x7f76f0120560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48444/0 (socket says 172.21.15.3:48444) 2023-10-03T10:16:46.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.871+0000 7f76ee59c700 1 -- 172.21.15.3:0/2125308342 learned_addr learned my addr 172.21.15.3:0/2125308342 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:46.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.871+0000 7f76edd9b700 1 --2- 172.21.15.3:0/2125308342 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76f006b080 0x7f76f0120aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:46.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.871+0000 7f76eed9d700 1 --2- 172.21.15.3:0/2125308342 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76f0131d10 0x7f76f0120fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:46.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.871+0000 7f76ee59c700 1 -- 172.21.15.3:0/2125308342 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76f0131d10 msgr2=0x7f76f0120fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:46.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.871+0000 7f76ee59c700 1 --2- 172.21.15.3:0/2125308342 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76f0131d10 0x7f76f0120fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:46.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.871+0000 7f76ee59c700 1 -- 172.21.15.3:0/2125308342 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76f006b080 msgr2=0x7f76f0120aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:46.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.871+0000 7f76ee59c700 1 --2- 172.21.15.3:0/2125308342 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76f006b080 0x7f76f0120aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:46.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.871+0000 7f76ee59c700 1 -- 172.21.15.3:0/2125308342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e8009a70 con 0x7f76f006a770 2023-10-03T10:16:46.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.871+0000 7f76edd9b700 1 --2- 172.21.15.3:0/2125308342 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76f006b080 0x7f76f0120aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:46.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.872+0000 7f76ee59c700 1 --2- 172.21.15.3:0/2125308342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76f006a770 0x7f76f0120560 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009ea0 tx=0x7f76e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:46.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.872+0000 7f76deffd700 1 -- 172.21.15.3:0/2125308342 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e001a720 con 0x7f76f006a770 2023-10-03T10:16:46.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.872+0000 7f76deffd700 1 -- 172.21.15.3:0/2125308342 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e001a880 con 0x7f76f006a770 2023-10-03T10:16:46.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.872+0000 7f76f49d7700 1 -- 172.21.15.3:0/2125308342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e00094a0 con 0x7f76f006a770 2023-10-03T10:16:46.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.872+0000 7f76deffd700 1 -- 172.21.15.3:0/2125308342 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e001ac30 con 0x7f76f006a770 2023-10-03T10:16:46.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.872+0000 7f76f49d7700 1 -- 172.21.15.3:0/2125308342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f0123ae0 con 0x7f76f006a770 2023-10-03T10:16:46.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.873+0000 7f76deffd700 1 -- 172.21.15.3:0/2125308342 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f76e0022070 con 0x7f76f006a770 2023-10-03T10:16:46.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.873+0000 7f76deffd700 1 -- 172.21.15.3:0/2125308342 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f76e00405d0 con 0x7f76f006a770 2023-10-03T10:16:46.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.873+0000 7f76f49d7700 1 -- 172.21.15.3:0/2125308342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f0074ea0 con 0x7f76f006a770 2023-10-03T10:16:46.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.879+0000 7f76deffd700 1 -- 172.21.15.3:0/2125308342 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f76e0029080 con 0x7f76f006a770 2023-10-03T10:16:46.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.912+0000 7f45469a0700 1 Processor -- start 2023-10-03T10:16:46.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.912+0000 7f45469a0700 1 -- start start 2023-10-03T10:16:46.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.913+0000 7f45469a0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f45400a0a30 0x7f45400a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:46.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.913+0000 7f45469a0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f454009d630 0x7f45400a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:46.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.913+0000 7f45469a0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4540133930 0x7f4540133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:46.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.913+0000 7f45469a0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45400a2a70 con 0x7f4540133930 2023-10-03T10:16:46.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.913+0000 7f45469a0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45400a01d0 con 0x7f45400a0a30 2023-10-03T10:16:46.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.913+0000 7f45469a0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f454009e2b0 con 0x7f454009d630 2023-10-03T10:16:46.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.913+0000 7f4544f3d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4540133930 0x7f4540133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:46.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.913+0000 7f453f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f454009d630 0x7f45400a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:46.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.913+0000 7f4544f3d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4540133930 0x7f4540133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48458/0 (socket says 172.21.15.3:48458) 2023-10-03T10:16:46.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.913+0000 7f4544f3d700 1 -- 172.21.15.3:0/3730484694 learned_addr learned my addr 172.21.15.3:0/3730484694 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:46.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.913+0000 7f453ffff700 1 --2- 172.21.15.3:0/3730484694 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f45400a0a30 0x7f45400a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:46.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.913+0000 7f4544f3d700 1 -- 172.21.15.3:0/3730484694 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f454009d630 msgr2=0x7f45400a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:46.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.913+0000 7f4544f3d700 1 --2- 172.21.15.3:0/3730484694 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f454009d630 0x7f45400a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:46.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.913+0000 7f4544f3d700 1 -- 172.21.15.3:0/3730484694 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f45400a0a30 msgr2=0x7f45400a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:46.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.913+0000 7f4544f3d700 1 --2- 172.21.15.3:0/3730484694 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f45400a0a30 0x7f45400a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:46.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.913+0000 7f4544f3d700 1 -- 172.21.15.3:0/3730484694 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454009dc50 con 0x7f4540133930 2023-10-03T10:16:46.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.914+0000 7f4544f3d700 1 --2- 172.21.15.3:0/3730484694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4540133930 0x7f4540133dd0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f4538009e50 tx=0x7f453800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec0a05c15bf2871 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:46.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.914+0000 7f453effd700 1 -- 172.21.15.3:0/3730484694 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453800e070 con 0x7f4540133930 2023-10-03T10:16:46.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.914+0000 7f453effd700 1 -- 172.21.15.3:0/3730484694 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453800b910 con 0x7f4540133930 2023-10-03T10:16:46.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.914+0000 7f453effd700 1 -- 172.21.15.3:0/3730484694 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453800bcc0 con 0x7f4540133930 2023-10-03T10:16:46.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.915+0000 7f45469a0700 1 -- 172.21.15.3:0/3730484694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4540133930 msgr2=0x7f4540133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:46.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.915+0000 7f45469a0700 1 --2- 172.21.15.3:0/3730484694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4540133930 0x7f4540133dd0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f4538009e50 tx=0x7f453800b330 comp rx=0 tx=0).stop 2023-10-03T10:16:46.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.915+0000 7f45469a0700 1 -- 172.21.15.3:0/3730484694 shutdown_connections 2023-10-03T10:16:46.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.915+0000 7f45469a0700 1 --2- 172.21.15.3:0/3730484694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4540133930 0x7f4540133dd0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:46.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.915+0000 7f45469a0700 1 --2- 172.21.15.3:0/3730484694 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f45400a0a30 0x7f45400a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:46.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.915+0000 7f45469a0700 1 --2- 172.21.15.3:0/3730484694 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f454009d630 0x7f45400a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:46.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.915+0000 7f45469a0700 1 -- 172.21.15.3:0/3730484694 >> 172.21.15.3:0/3730484694 conn(0x7f45400a3050 msgr2=0x7f45400a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:46.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.915+0000 7f45469a0700 1 -- 172.21.15.3:0/3730484694 shutdown_connections 2023-10-03T10:16:46.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.915+0000 7f45469a0700 1 -- 172.21.15.3:0/3730484694 wait complete. 2023-10-03T10:16:46.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.916+0000 7f45469a0700 1 Processor -- start 2023-10-03T10:16:46.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.917+0000 7f45469a0700 1 -- start start 2023-10-03T10:16:46.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.917+0000 7f45469a0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f454009d630 0x7f45401c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:46.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.917+0000 7f45469a0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f45400a0a30 0x7f45401c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:46.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.917+0000 7f45469a0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4540133930 0x7f45401c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:46.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.917+0000 7f45469a0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45400a0410 con 0x7f4540133930 2023-10-03T10:16:46.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.917+0000 7f45469a0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45401206f0 con 0x7f45400a0a30 2023-10-03T10:16:46.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.917+0000 7f45469a0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f45401334a0 con 0x7f454009d630 2023-10-03T10:16:46.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.918+0000 7f4544f3d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4540133930 0x7f45401c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:46.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.918+0000 7f4544f3d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4540133930 0x7f45401c8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48472/0 (socket says 172.21.15.3:48472) 2023-10-03T10:16:46.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.918+0000 7f4544f3d700 1 -- 172.21.15.3:0/185212036 learned_addr learned my addr 172.21.15.3:0/185212036 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:46.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.918+0000 7f453f7fe700 1 --2- 172.21.15.3:0/185212036 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f45400a0a30 0x7f45401c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:46.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.918+0000 7f453ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f454009d630 0x7f45401c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:46.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.918+0000 7f4544f3d700 1 -- 172.21.15.3:0/185212036 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f454009d630 msgr2=0x7f45401c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:46.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.918+0000 7f4544f3d700 1 --2- 172.21.15.3:0/185212036 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f454009d630 0x7f45401c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:46.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.918+0000 7f4544f3d700 1 -- 172.21.15.3:0/185212036 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f45400a0a30 msgr2=0x7f45401c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:46.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.918+0000 7f4544f3d700 1 --2- 172.21.15.3:0/185212036 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f45400a0a30 0x7f45401c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:46.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.918+0000 7f4544f3d700 1 -- 172.21.15.3:0/185212036 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4538009b30 con 0x7f4540133930 2023-10-03T10:16:46.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.918+0000 7f453f7fe700 1 --2- 172.21.15.3:0/185212036 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f45400a0a30 0x7f45401c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:46.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.918+0000 7f4544f3d700 1 --2- 172.21.15.3:0/185212036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4540133930 0x7f45401c8990 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f45400a2bb0 tx=0x7f4538004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:46.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.918+0000 7f453cff9700 1 -- 172.21.15.3:0/185212036 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538004450 con 0x7f4540133930 2023-10-03T10:16:46.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.919+0000 7f453cff9700 1 -- 172.21.15.3:0/185212036 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45380045f0 con 0x7f4540133930 2023-10-03T10:16:46.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.919+0000 7f453cff9700 1 -- 172.21.15.3:0/185212036 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45380092e0 con 0x7f4540133930 2023-10-03T10:16:46.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.919+0000 7f45469a0700 1 -- 172.21.15.3:0/185212036 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4540121ee0 con 0x7f4540133930 2023-10-03T10:16:46.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.919+0000 7f45469a0700 1 -- 172.21.15.3:0/185212036 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4540122c10 con 0x7f4540133930 2023-10-03T10:16:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.919+0000 7f453cff9700 1 -- 172.21.15.3:0/185212036 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f453801c410 con 0x7f4540133930 2023-10-03T10:16:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.920+0000 7f453cff9700 1 -- 172.21.15.3:0/185212036 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f45380412f0 con 0x7f4540133930 2023-10-03T10:16:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.920+0000 7f45469a0700 1 -- 172.21.15.3:0/185212036 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45400747f0 con 0x7f4540133930 2023-10-03T10:16:46.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.929+0000 7f453cff9700 1 -- 172.21.15.3:0/185212036 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f45380117a0 con 0x7f4540133930 2023-10-03T10:16:47.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:46.999+0000 7f89eb100700 1 -- 172.21.15.3:0/3527058398 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f89e408f8f0 con 0x7f89e40992c0 2023-10-03T10:16:47.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.000+0000 7f89e17fa700 1 -- 172.21.15.3:0/3527058398 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f89e408f8f0 con 0x7f89e40992c0 2023-10-03T10:16:47.071 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:47.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.003+0000 7f89eb100700 1 -- 172.21.15.3:0/3527058398 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f89e40992c0 msgr2=0x7f89e408c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:47.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.003+0000 7f89eb100700 1 --2- 172.21.15.3:0/3527058398 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f89e40992c0 0x7f89e408c490 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f89d8000c00 tx=0x7f89d8004750 comp rx=0 tx=0).stop 2023-10-03T10:16:47.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.003+0000 7f89eb100700 1 -- 172.21.15.3:0/3527058398 shutdown_connections 2023-10-03T10:16:47.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.003+0000 7f89eb100700 1 --2- 172.21.15.3:0/3527058398 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89e41306b0 0x7f89e408cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:47.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.003+0000 7f89eb100700 1 --2- 172.21.15.3:0/3527058398 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f89e40992c0 0x7f89e408c490 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:47.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.003+0000 7f89eb100700 1 --2- 172.21.15.3:0/3527058398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89e4099dc0 0x7f89e408c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:47.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.003+0000 7f89eb100700 1 -- 172.21.15.3:0/3527058398 >> 172.21.15.3:0/3527058398 conn(0x7f89e409e7e0 msgr2=0x7f89e409d330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:47.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.003+0000 7f89eb100700 1 -- 172.21.15.3:0/3527058398 shutdown_connections 2023-10-03T10:16:47.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.003+0000 7f89eb100700 1 -- 172.21.15.3:0/3527058398 wait complete. 2023-10-03T10:16:47.073 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:47.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.088+0000 7f76f49d7700 1 -- 172.21.15.3:0/2125308342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f76f0124820 con 0x7f76f006a770 2023-10-03T10:16:47.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.089+0000 7f76deffd700 1 -- 172.21.15.3:0/2125308342 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f76e0015020 con 0x7f76f006a770 2023-10-03T10:16:47.089 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:47.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.091+0000 7f76f49d7700 1 -- 172.21.15.3:0/2125308342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76f006a770 msgr2=0x7f76f0120560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:47.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.091+0000 7f76f49d7700 1 --2- 172.21.15.3:0/2125308342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76f006a770 0x7f76f0120560 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009ea0 tx=0x7f76e0004750 comp rx=0 tx=0).stop 2023-10-03T10:16:47.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.091+0000 7f76f49d7700 1 -- 172.21.15.3:0/2125308342 shutdown_connections 2023-10-03T10:16:47.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.091+0000 7f76f49d7700 1 --2- 172.21.15.3:0/2125308342 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76f0131d10 0x7f76f0120fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:47.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.091+0000 7f76f49d7700 1 --2- 172.21.15.3:0/2125308342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76f006a770 0x7f76f0120560 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:47.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.091+0000 7f76f49d7700 1 --2- 172.21.15.3:0/2125308342 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76f006b080 0x7f76f0120aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:47.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.091+0000 7f76f49d7700 1 -- 172.21.15.3:0/2125308342 >> 172.21.15.3:0/2125308342 conn(0x7f76f0066f50 msgr2=0x7f76f0070030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:47.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.091+0000 7f76f49d7700 1 -- 172.21.15.3:0/2125308342 shutdown_connections 2023-10-03T10:16:47.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.091+0000 7f76f49d7700 1 -- 172.21.15.3:0/2125308342 wait complete. 2023-10-03T10:16:47.105 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:47.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.144+0000 7f45469a0700 1 -- 172.21.15.3:0/185212036 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f45401223e0 con 0x7f4540133930 2023-10-03T10:16:47.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.145+0000 7f453cff9700 1 -- 172.21.15.3:0/185212036 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f453802a030 con 0x7f4540133930 2023-10-03T10:16:47.145 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:47.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.147+0000 7f45469a0700 1 -- 172.21.15.3:0/185212036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4540133930 msgr2=0x7f45401c8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:47.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.147+0000 7f45469a0700 1 --2- 172.21.15.3:0/185212036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4540133930 0x7f45401c8990 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f45400a2bb0 tx=0x7f4538004ad0 comp rx=0 tx=0).stop 2023-10-03T10:16:47.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.147+0000 7f45469a0700 1 -- 172.21.15.3:0/185212036 shutdown_connections 2023-10-03T10:16:47.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.147+0000 7f45469a0700 1 --2- 172.21.15.3:0/185212036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4540133930 0x7f45401c8990 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:47.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.147+0000 7f45469a0700 1 --2- 172.21.15.3:0/185212036 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f454009d630 0x7f45401c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:47.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.147+0000 7f45469a0700 1 --2- 172.21.15.3:0/185212036 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f45400a0a30 0x7f45401c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:47.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.147+0000 7f45469a0700 1 -- 172.21.15.3:0/185212036 >> 172.21.15.3:0/185212036 conn(0x7f45400a3050 msgr2=0x7f454009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:47.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.147+0000 7f45469a0700 1 -- 172.21.15.3:0/185212036 shutdown_connections 2023-10-03T10:16:47.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.147+0000 7f45469a0700 1 -- 172.21.15.3:0/185212036 wait complete. 2023-10-03T10:16:47.161 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:47.844 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:47.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.990+0000 7fc327ebb700 1 Processor -- start 2023-10-03T10:16:47.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.990+0000 7fc327ebb700 1 -- start start 2023-10-03T10:16:47.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.990+0000 7fc327ebb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc32009d6f0 0x7fc32009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:47.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.990+0000 7fc327ebb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc32009e410 0x7fc320130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:47.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.990+0000 7fc327ebb700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc32009f130 con 0x7fc32009e000 2023-10-03T10:16:47.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.990+0000 7fc327ebb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc32009f520 con 0x7fc32009d6f0 2023-10-03T10:16:47.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.990+0000 7fc327ebb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32009e930 con 0x7fc32009e410 2023-10-03T10:16:47.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.991+0000 7fc325c57700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc32009d6f0 0x7fc32009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:47.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.991+0000 7fc326458700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc32009e410 0x7fc320130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:47.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.991+0000 7fc325c57700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc32009d6f0 0x7fc32009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48488/0 (socket says 172.21.15.3:48488) 2023-10-03T10:16:47.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.991+0000 7fc325c57700 1 -- 172.21.15.3:0/2866210900 learned_addr learned my addr 172.21.15.3:0/2866210900 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:47.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.991+0000 7fc326458700 1 -- 172.21.15.3:0/2866210900 >> v2:172.21.15.3:3300/0 conn(0x7fc32009d6f0 msgr2=0x7fc32009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:47.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.991+0000 7fc326458700 1 --2- 172.21.15.3:0/2866210900 >> v2:172.21.15.3:3300/0 conn(0x7fc32009d6f0 0x7fc32009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:47.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.991+0000 7fc326458700 1 -- 172.21.15.3:0/2866210900 >> v1:172.21.15.3:6789/0 conn(0x7fc32009e000 legacy=0x7fc320130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:47.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.991+0000 7fc326458700 1 -- 172.21.15.3:0/2866210900 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32009b2d0 con 0x7fc32009e410 2023-10-03T10:16:47.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.991+0000 7fc325c57700 1 --2- 172.21.15.3:0/2866210900 >> v2:172.21.15.3:3300/0 conn(0x7fc32009d6f0 0x7fc32009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:47.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.991+0000 7fc324c55700 1 -- 172.21.15.3:0/2866210900 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3331681531 0 0) 0x7fc32009f130 con 0x7fc32009e000 2023-10-03T10:16:47.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.991+0000 7fc326458700 1 --2- 172.21.15.3:0/2866210900 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc32009e410 0x7fc320130d70 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc31800bfa0 tx=0x7fc31800d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b0c0060d5ab129e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:47.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.991+0000 7fc324c55700 1 -- 172.21.15.3:0/2866210900 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc318010070 con 0x7fc32009e410 2023-10-03T10:16:47.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.991+0000 7fc324c55700 1 -- 172.21.15.3:0/2866210900 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31800b2e0 con 0x7fc32009e410 2023-10-03T10:16:47.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.991+0000 7fc324c55700 1 -- 172.21.15.3:0/2866210900 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31800dc40 con 0x7fc32009e410 2023-10-03T10:16:47.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.992+0000 7fc327ebb700 1 -- 172.21.15.3:0/2866210900 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc32009e410 msgr2=0x7fc320130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:47.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.992+0000 7fc327ebb700 1 --2- 172.21.15.3:0/2866210900 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc32009e410 0x7fc320130d70 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc31800bfa0 tx=0x7fc31800d330 comp rx=0 tx=0).stop 2023-10-03T10:16:47.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.992+0000 7fc327ebb700 1 -- 172.21.15.3:0/2866210900 shutdown_connections 2023-10-03T10:16:47.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.992+0000 7fc327ebb700 1 --2- 172.21.15.3:0/2866210900 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc32009e410 0x7fc320130d70 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:47.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.992+0000 7fc327ebb700 1 --2- 172.21.15.3:0/2866210900 >> v2:172.21.15.3:3300/0 conn(0x7fc32009d6f0 0x7fc32009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:47.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.992+0000 7fc327ebb700 1 -- 172.21.15.3:0/2866210900 >> 172.21.15.3:0/2866210900 conn(0x7fc320066f50 msgr2=0x7fc32009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:47.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.992+0000 7fc327ebb700 1 -- 172.21.15.3:0/2866210900 shutdown_connections 2023-10-03T10:16:47.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.992+0000 7fc327ebb700 1 -- 172.21.15.3:0/2866210900 wait complete. 2023-10-03T10:16:47.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.993+0000 7fc327ebb700 1 Processor -- start 2023-10-03T10:16:47.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.993+0000 7fc327ebb700 1 -- start start 2023-10-03T10:16:47.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.993+0000 7fc327ebb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc32009d6f0 0x7fc32011dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:47.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.993+0000 7fc327ebb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc32009e000 0x7fc32011e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:47.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.993+0000 7fc327ebb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc32009e410 0x7fc32011e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:47.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.993+0000 7fc327ebb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32011ff10 con 0x7fc32009e000 2023-10-03T10:16:47.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.993+0000 7fc327ebb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc320120fd0 con 0x7fc32009e410 2023-10-03T10:16:47.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.993+0000 7fc327ebb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc32009c700 con 0x7fc32009d6f0 2023-10-03T10:16:47.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.994+0000 7fc326458700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc32009e410 0x7fc32011e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:47.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.994+0000 7fc325456700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc32009e000 0x7fc32011e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:47.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.994+0000 7fc325c57700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc32009d6f0 0x7fc32011dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:47.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.994+0000 7fc326458700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc32009e410 0x7fc32011e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53136/0 (socket says 172.21.15.3:53136) 2023-10-03T10:16:47.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.994+0000 7fc326458700 1 -- 172.21.15.3:0/2709603862 learned_addr learned my addr 172.21.15.3:0/2709603862 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:47.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.994+0000 7fc325c57700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc32009d6f0 0x7fc32011dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51700/0 (socket says 172.21.15.3:51700) 2023-10-03T10:16:47.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.994+0000 7fc326458700 1 -- 172.21.15.3:0/2709603862 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc32009d6f0 msgr2=0x7fc32011dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:47.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.994+0000 7fc326458700 1 --2- 172.21.15.3:0/2709603862 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc32009d6f0 0x7fc32011dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:47.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.994+0000 7fc326458700 1 -- 172.21.15.3:0/2709603862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc32009e000 msgr2=0x7fc32011e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:47.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.994+0000 7fc326458700 1 --2- 172.21.15.3:0/2709603862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc32009e000 0x7fc32011e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:47.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.994+0000 7fc326458700 1 -- 172.21.15.3:0/2709603862 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc31c009550 con 0x7fc32009e410 2023-10-03T10:16:47.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.994+0000 7fc325456700 1 --2- 172.21.15.3:0/2709603862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc32009e000 0x7fc32011e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:47.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.994+0000 7fc326458700 1 --2- 172.21.15.3:0/2709603862 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc32009e410 0x7fc32011e7a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc31800bfd0 tx=0x7fc318004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:47.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.994+0000 7fc3167fc700 1 -- 172.21.15.3:0/2709603862 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc318004450 con 0x7fc32009e410 2023-10-03T10:16:47.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.994+0000 7fc3167fc700 1 -- 172.21.15.3:0/2709603862 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31800b450 con 0x7fc32009e410 2023-10-03T10:16:47.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.994+0000 7fc327ebb700 1 -- 172.21.15.3:0/2709603862 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc31800bc80 con 0x7fc32009e410 2023-10-03T10:16:47.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.994+0000 7fc3167fc700 1 -- 172.21.15.3:0/2709603862 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3180047d0 con 0x7fc32009e410 2023-10-03T10:16:47.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.994+0000 7fc327ebb700 1 -- 172.21.15.3:0/2709603862 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc32011f930 con 0x7fc32009e410 2023-10-03T10:16:47.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.995+0000 7fc3167fc700 1 -- 172.21.15.3:0/2709603862 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc318009460 con 0x7fc32009e410 2023-10-03T10:16:48.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.995+0000 7fc327ebb700 1 -- 172.21.15.3:0/2709603862 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc32011ac20 con 0x7fc32009e410 2023-10-03T10:16:48.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:47.995+0000 7fc3167fc700 1 -- 172.21.15.3:0/2709603862 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc3180435d0 con 0x7fc32009e410 2023-10-03T10:16:48.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.000+0000 7fc3167fc700 1 -- 172.21.15.3:0/2709603862 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc31801dd80 con 0x7fc32009e410 2023-10-03T10:16:48.074 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:48.105 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:48.162 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:48.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.216+0000 7f1cbc611700 1 Processor -- start 2023-10-03T10:16:48.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.216+0000 7f1cbc611700 1 -- start start 2023-10-03T10:16:48.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.217+0000 7f1cbc611700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cb41338a0 0x7f1cb4133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:48.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.217+0000 7f1cbc611700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1cb409e930 0x7f1cb413b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:48.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.217+0000 7f1cbc611700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cb406ef30 con 0x7f1cb409de50 2023-10-03T10:16:48.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.217+0000 7f1cbc611700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cb406fef0 con 0x7f1cb409e930 2023-10-03T10:16:48.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.217+0000 7f1cbc611700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb41334b0 con 0x7f1cb41338a0 2023-10-03T10:16:48.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.217+0000 7f1cbabae700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1cb409e930 0x7f1cb413b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:48.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.217+0000 7f1cba3ad700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cb41338a0 0x7f1cb4133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:48.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.217+0000 7f1cbabae700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1cb409e930 0x7f1cb413b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48494/0 (socket says 172.21.15.3:48494) 2023-10-03T10:16:48.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.217+0000 7f1cbabae700 1 -- 172.21.15.3:0/2575402123 learned_addr learned my addr 172.21.15.3:0/2575402123 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:48.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.218+0000 7f1cba3ad700 1 -- 172.21.15.3:0/2575402123 >> v2:172.21.15.3:3300/0 conn(0x7f1cb409e930 msgr2=0x7f1cb413b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:48.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.218+0000 7f1cba3ad700 1 --2- 172.21.15.3:0/2575402123 >> v2:172.21.15.3:3300/0 conn(0x7f1cb409e930 0x7f1cb413b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:48.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.218+0000 7f1cba3ad700 1 -- 172.21.15.3:0/2575402123 >> v1:172.21.15.3:6789/0 conn(0x7f1cb409de50 legacy=0x7f1cb409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:48.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.218+0000 7f1cba3ad700 1 -- 172.21.15.3:0/2575402123 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb406fc30 con 0x7f1cb41338a0 2023-10-03T10:16:48.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.218+0000 7f1cbabae700 1 --2- 172.21.15.3:0/2575402123 >> v2:172.21.15.3:3300/0 conn(0x7f1cb409e930 0x7f1cb413b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:48.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.218+0000 7f1cba3ad700 1 --2- 172.21.15.3:0/2575402123 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cb41338a0 0x7f1cb4133c70 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009840 tx=0x7f1cb0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ca4f82552ab9c6f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:48.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.218+0000 7f1cb93ab700 1 -- 172.21.15.3:0/2575402123 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0011830 con 0x7f1cb41338a0 2023-10-03T10:16:48.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.218+0000 7f1cb93ab700 1 -- 172.21.15.3:0/2575402123 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb0011990 con 0x7f1cb41338a0 2023-10-03T10:16:48.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.218+0000 7f1cb93ab700 1 -- 172.21.15.3:0/2575402123 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0011d20 con 0x7f1cb41338a0 2023-10-03T10:16:48.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.219+0000 7f1cbc611700 1 -- 172.21.15.3:0/2575402123 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cb41338a0 msgr2=0x7f1cb4133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:48.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.219+0000 7f1cbc611700 1 --2- 172.21.15.3:0/2575402123 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cb41338a0 0x7f1cb4133c70 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009840 tx=0x7f1cb0009b50 comp rx=0 tx=0).stop 2023-10-03T10:16:48.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.220+0000 7f1cbc611700 1 -- 172.21.15.3:0/2575402123 shutdown_connections 2023-10-03T10:16:48.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.220+0000 7f1cbc611700 1 --2- 172.21.15.3:0/2575402123 >> v2:172.21.15.3:3300/0 conn(0x7f1cb409e930 0x7f1cb413b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:48.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.220+0000 7f1cbc611700 1 --2- 172.21.15.3:0/2575402123 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cb41338a0 0x7f1cb4133c70 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:48.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.220+0000 7f1cbc611700 1 -- 172.21.15.3:0/2575402123 >> 172.21.15.3:0/2575402123 conn(0x7f1cb40a2aa0 msgr2=0x7f1cb40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:48.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.220+0000 7f1cbc611700 1 -- 172.21.15.3:0/2575402123 shutdown_connections 2023-10-03T10:16:48.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.220+0000 7f1cbc611700 1 -- 172.21.15.3:0/2575402123 wait complete. 2023-10-03T10:16:48.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.221+0000 7f1cbc611700 1 Processor -- start 2023-10-03T10:16:48.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.221+0000 7f1cbc611700 1 -- start start 2023-10-03T10:16:48.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.222+0000 7f1cbc611700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cb409de50 0x7f1cb411d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:48.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.222+0000 7f1cbc611700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1cb409e930 0x7f1cb411dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:48.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.222+0000 7f1cbc611700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cb41338a0 0x7f1cb411e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:48.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.222+0000 7f1cbc611700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb411f350 con 0x7f1cb41338a0 2023-10-03T10:16:48.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.222+0000 7f1cbc611700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb406f640 con 0x7f1cb409de50 2023-10-03T10:16:48.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.222+0000 7f1cbc611700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb4120f80 con 0x7f1cb409e930 2023-10-03T10:16:48.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.222+0000 7fc327ebb700 1 -- 172.21.15.3:0/2709603862 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc320074ea0 con 0x7fc32009e410 2023-10-03T10:16:48.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.222+0000 7f1cb9bac700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1cb409e930 0x7f1cb411dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:48.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.222+0000 7f1cb9bac700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1cb409e930 0x7f1cb411dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51710/0 (socket says 172.21.15.3:51710) 2023-10-03T10:16:48.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.222+0000 7f1cb9bac700 1 -- 172.21.15.3:0/4050895081 learned_addr learned my addr 172.21.15.3:0/4050895081 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:48.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.222+0000 7f1cba3ad700 1 --2- 172.21.15.3:0/4050895081 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cb409de50 0x7f1cb411d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:48.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.222+0000 7f1cbabae700 1 --2- 172.21.15.3:0/4050895081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cb41338a0 0x7f1cb411e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:48.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.222+0000 7fc3167fc700 1 -- 172.21.15.3:0/2709603862 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc318035090 con 0x7fc32009e410 2023-10-03T10:16:48.226 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:48.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.223+0000 7f1cba3ad700 1 -- 172.21.15.3:0/4050895081 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1cb409e930 msgr2=0x7f1cb411dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:48.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.223+0000 7f1cba3ad700 1 --2- 172.21.15.3:0/4050895081 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1cb409e930 0x7f1cb411dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:48.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.223+0000 7f1cba3ad700 1 -- 172.21.15.3:0/4050895081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cb41338a0 msgr2=0x7f1cb411e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:48.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.223+0000 7f1cba3ad700 1 --2- 172.21.15.3:0/4050895081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cb41338a0 0x7f1cb411e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:48.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.223+0000 7f1cba3ad700 1 -- 172.21.15.3:0/4050895081 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cac009b30 con 0x7f1cb409de50 2023-10-03T10:16:48.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.223+0000 7f1cbabae700 1 --2- 172.21.15.3:0/4050895081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cb41338a0 0x7f1cb411e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:16:48.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.223+0000 7f1cba3ad700 1 --2- 172.21.15.3:0/4050895081 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cb409de50 0x7f1cb411d820 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0016000 tx=0x7f1cb0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:48.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.223+0000 7f1caaffd700 1 -- 172.21.15.3:0/4050895081 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb00036a0 con 0x7f1cb409de50 2023-10-03T10:16:48.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.223+0000 7f1caaffd700 1 -- 172.21.15.3:0/4050895081 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb001f070 con 0x7f1cb409de50 2023-10-03T10:16:48.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.223+0000 7f1cbc611700 1 -- 172.21.15.3:0/4050895081 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb00094a0 con 0x7f1cb409de50 2023-10-03T10:16:48.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.223+0000 7f1caaffd700 1 -- 172.21.15.3:0/4050895081 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0003a60 con 0x7f1cb409de50 2023-10-03T10:16:48.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.223+0000 7f1cbc611700 1 -- 172.21.15.3:0/4050895081 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb40a1050 con 0x7f1cb409de50 2023-10-03T10:16:48.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.224+0000 7f1caaffd700 1 -- 172.21.15.3:0/4050895081 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1cb0003bc0 con 0x7f1cb409de50 2023-10-03T10:16:48.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.224+0000 7f1caaffd700 1 -- 172.21.15.3:0/4050895081 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1cb0021070 con 0x7f1cb409de50 2023-10-03T10:16:48.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.225+0000 7f1cbc611700 1 -- 172.21.15.3:0/4050895081 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb411a470 con 0x7f1cb409de50 2023-10-03T10:16:48.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.227+0000 7fc327ebb700 1 -- 172.21.15.3:0/2709603862 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc32009e410 msgr2=0x7fc32011e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:48.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.227+0000 7fc327ebb700 1 --2- 172.21.15.3:0/2709603862 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc32009e410 0x7fc32011e7a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc31800bfd0 tx=0x7fc318004ad0 comp rx=0 tx=0).stop 2023-10-03T10:16:48.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.227+0000 7fc327ebb700 1 -- 172.21.15.3:0/2709603862 shutdown_connections 2023-10-03T10:16:48.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.227+0000 7fc327ebb700 1 --2- 172.21.15.3:0/2709603862 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc32009e410 0x7fc32011e7a0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:48.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.227+0000 7fc327ebb700 1 --2- 172.21.15.3:0/2709603862 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc32009d6f0 0x7fc32011dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:48.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.227+0000 7fc327ebb700 1 --2- 172.21.15.3:0/2709603862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc32009e000 0x7fc32011e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:48.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.227+0000 7fc327ebb700 1 -- 172.21.15.3:0/2709603862 >> 172.21.15.3:0/2709603862 conn(0x7fc320066f50 msgr2=0x7fc320130800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:48.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.228+0000 7fc327ebb700 1 -- 172.21.15.3:0/2709603862 shutdown_connections 2023-10-03T10:16:48.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.228+0000 7fc327ebb700 1 -- 172.21.15.3:0/2709603862 wait complete. 2023-10-03T10:16:48.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.234+0000 7f1caaffd700 1 -- 172.21.15.3:0/4050895081 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1cb002b020 con 0x7f1cb409de50 2023-10-03T10:16:48.252 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:48.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.267+0000 7fb50553b700 1 Processor -- start 2023-10-03T10:16:48.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.267+0000 7fb50553b700 1 -- start start 2023-10-03T10:16:48.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.268+0000 7fb50553b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb50009d6f0 0x7fb50009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:48.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.268+0000 7fb50553b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb50009e000 0x7fb500130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:48.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.268+0000 7fb50553b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb50009f130 con 0x7fb50009e410 2023-10-03T10:16:48.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.268+0000 7fb50553b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb50009f520 con 0x7fb50009e000 2023-10-03T10:16:48.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.268+0000 7fb50553b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50009e930 con 0x7fb50009d6f0 2023-10-03T10:16:48.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.268+0000 7fb4fe7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb50009e000 0x7fb500130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:48.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.268+0000 7fb4feffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb50009d6f0 0x7fb50009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:48.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.268+0000 7fb4fe7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb50009e000 0x7fb500130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48510/0 (socket says 172.21.15.3:48510) 2023-10-03T10:16:48.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.268+0000 7fb4fe7fc700 1 -- 172.21.15.3:0/1756912897 learned_addr learned my addr 172.21.15.3:0/1756912897 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:48.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.268+0000 7fb4feffd700 1 --2- 172.21.15.3:0/1756912897 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb50009d6f0 0x7fb50009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53164/0 (socket says 172.21.15.3:53164) 2023-10-03T10:16:48.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.268+0000 7fb4feffd700 1 -- 172.21.15.3:0/1756912897 >> v2:172.21.15.3:3300/0 conn(0x7fb50009e000 msgr2=0x7fb500130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:48.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.268+0000 7fb4feffd700 1 --2- 172.21.15.3:0/1756912897 >> v2:172.21.15.3:3300/0 conn(0x7fb50009e000 0x7fb500130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:48.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.268+0000 7fb4feffd700 1 -- 172.21.15.3:0/1756912897 >> v1:172.21.15.3:6789/0 conn(0x7fb50009e410 legacy=0x7fb500130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:48.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.268+0000 7fb4feffd700 1 -- 172.21.15.3:0/1756912897 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50009b2d0 con 0x7fb50009d6f0 2023-10-03T10:16:48.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.268+0000 7fb4fe7fc700 1 --2- 172.21.15.3:0/1756912897 >> v2:172.21.15.3:3300/0 conn(0x7fb50009e000 0x7fb500130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:48.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.269+0000 7fb4feffd700 1 --2- 172.21.15.3:0/1756912897 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb50009d6f0 0x7fb50009dac0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb4f000dc60 tx=0x7fb4f000df70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=754fdc98c6f9cfce server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:48.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.269+0000 7fb4fdffb700 1 -- 172.21.15.3:0/1756912897 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0012070 con 0x7fb50009d6f0 2023-10-03T10:16:48.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.269+0000 7fb4fdffb700 1 -- 172.21.15.3:0/1756912897 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f000d2e0 con 0x7fb50009d6f0 2023-10-03T10:16:48.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.269+0000 7fb4fdffb700 1 -- 172.21.15.3:0/1756912897 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f000f900 con 0x7fb50009d6f0 2023-10-03T10:16:48.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.270+0000 7fb50553b700 1 -- 172.21.15.3:0/1756912897 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb50009d6f0 msgr2=0x7fb50009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:48.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.270+0000 7fb50553b700 1 --2- 172.21.15.3:0/1756912897 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb50009d6f0 0x7fb50009dac0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb4f000dc60 tx=0x7fb4f000df70 comp rx=0 tx=0).stop 2023-10-03T10:16:48.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.270+0000 7fb50553b700 1 -- 172.21.15.3:0/1756912897 shutdown_connections 2023-10-03T10:16:48.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.270+0000 7fb50553b700 1 --2- 172.21.15.3:0/1756912897 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb50009d6f0 0x7fb50009dac0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:48.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.270+0000 7fb50553b700 1 --2- 172.21.15.3:0/1756912897 >> v2:172.21.15.3:3300/0 conn(0x7fb50009e000 0x7fb500130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:48.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.270+0000 7fb50553b700 1 -- 172.21.15.3:0/1756912897 >> 172.21.15.3:0/1756912897 conn(0x7fb500066f50 msgr2=0x7fb50009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:48.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.270+0000 7fb50553b700 1 -- 172.21.15.3:0/1756912897 shutdown_connections 2023-10-03T10:16:48.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.270+0000 7fb50553b700 1 -- 172.21.15.3:0/1756912897 wait complete. 2023-10-03T10:16:48.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.271+0000 7fb50553b700 1 Processor -- start 2023-10-03T10:16:48.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.271+0000 7fb50553b700 1 -- start start 2023-10-03T10:16:48.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.271+0000 7fb50553b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb50009d6f0 0x7fb50011e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:48.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.271+0000 7fb50553b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb50009e000 0x7fb50011e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:48.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.271+0000 7fb50553b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb50009e410 0x7fb50011eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:48.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.271+0000 7fb50553b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50011f4d0 con 0x7fb50009d6f0 2023-10-03T10:16:48.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.271+0000 7fb50553b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb500121af0 con 0x7fb50009e000 2023-10-03T10:16:48.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.271+0000 7fb50553b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb500120200 con 0x7fb50009e410 2023-10-03T10:16:48.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.272+0000 7fb4feffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb50009d6f0 0x7fb50011e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:48.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.272+0000 7fb4feffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb50009d6f0 0x7fb50011e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48526/0 (socket says 172.21.15.3:48526) 2023-10-03T10:16:48.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.272+0000 7fb4feffd700 1 -- 172.21.15.3:0/584575963 learned_addr learned my addr 172.21.15.3:0/584575963 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:48.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.272+0000 7fb4ff7fe700 1 --2- 172.21.15.3:0/584575963 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb50009e410 0x7fb50011eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:48.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.272+0000 7fb4fe7fc700 1 --2- 172.21.15.3:0/584575963 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb50009e000 0x7fb50011e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:48.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.272+0000 7fb4feffd700 1 -- 172.21.15.3:0/584575963 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb50009e410 msgr2=0x7fb50011eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:48.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.272+0000 7fb4feffd700 1 --2- 172.21.15.3:0/584575963 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb50009e410 0x7fb50011eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:48.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.272+0000 7fb4feffd700 1 -- 172.21.15.3:0/584575963 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb50009e000 msgr2=0x7fb50011e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:48.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.272+0000 7fb4feffd700 1 --2- 172.21.15.3:0/584575963 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb50009e000 0x7fb50011e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:48.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.272+0000 7fb4feffd700 1 -- 172.21.15.3:0/584575963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f4009550 con 0x7fb50009d6f0 2023-10-03T10:16:48.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.272+0000 7fb4feffd700 1 --2- 172.21.15.3:0/584575963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb50009d6f0 0x7fb50011e020 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fb4f000f330 tx=0x7fb4f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:48.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.272+0000 7fb4e77fe700 1 -- 172.21.15.3:0/584575963 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f00036a0 con 0x7fb50009d6f0 2023-10-03T10:16:48.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.272+0000 7fb4e77fe700 1 -- 172.21.15.3:0/584575963 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f0003820 con 0x7fb50009d6f0 2023-10-03T10:16:48.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.272+0000 7fb50553b700 1 -- 172.21.15.3:0/584575963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f000d940 con 0x7fb50009d6f0 2023-10-03T10:16:48.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.272+0000 7fb4e77fe700 1 -- 172.21.15.3:0/584575963 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0003bd0 con 0x7fb50009d6f0 2023-10-03T10:16:48.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.273+0000 7fb50553b700 1 -- 172.21.15.3:0/584575963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb500130400 con 0x7fb50009d6f0 2023-10-03T10:16:48.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.273+0000 7fb4e77fe700 1 -- 172.21.15.3:0/584575963 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb4f0003d30 con 0x7fb50009d6f0 2023-10-03T10:16:48.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.274+0000 7fb4e77fe700 1 -- 172.21.15.3:0/584575963 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb4f00444a0 con 0x7fb50009d6f0 2023-10-03T10:16:48.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.274+0000 7fb50553b700 1 -- 172.21.15.3:0/584575963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb500130400 con 0x7fb50009d6f0 2023-10-03T10:16:48.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.280+0000 7fb4e77fe700 1 -- 172.21.15.3:0/584575963 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb500130400 con 0x7fb50009d6f0 2023-10-03T10:16:48.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.318+0000 7fb7fb253700 1 Processor -- start 2023-10-03T10:16:48.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.319+0000 7fb7fb253700 1 -- start start 2023-10-03T10:16:48.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.319+0000 7fb7fb253700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7f409d630 0x7fb7f40a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:48.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.319+0000 7fb7fb253700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb7f4138d80 0x7fb7f413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:48.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.319+0000 7fb7fb253700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f40a2a70 con 0x7fb7f40a0a30 2023-10-03T10:16:48.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.319+0000 7fb7fb253700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f40a01d0 con 0x7fb7f4138d80 2023-10-03T10:16:48.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.319+0000 7fb7fb253700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f409e2b0 con 0x7fb7f409d630 2023-10-03T10:16:48.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.319+0000 7fb7f3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7f409d630 0x7fb7f40a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:48.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.319+0000 7fb7f3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7f409d630 0x7fb7f40a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51722/0 (socket says 172.21.15.3:51722) 2023-10-03T10:16:48.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.319+0000 7fb7f3fff700 1 -- 172.21.15.3:0/1853078422 learned_addr learned my addr 172.21.15.3:0/1853078422 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:48.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.319+0000 7fb7f97f0700 1 --2- 172.21.15.3:0/1853078422 >> v2:172.21.15.3:3300/0 conn(0x7fb7f4138d80 0x7fb7f413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:48.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.320+0000 7fb7f3fff700 1 -- 172.21.15.3:0/1853078422 >> v2:172.21.15.3:3300/0 conn(0x7fb7f4138d80 msgr2=0x7fb7f413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:48.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.320+0000 7fb7f3fff700 1 --2- 172.21.15.3:0/1853078422 >> v2:172.21.15.3:3300/0 conn(0x7fb7f4138d80 0x7fb7f413b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:48.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.320+0000 7fb7f3fff700 1 -- 172.21.15.3:0/1853078422 >> v1:172.21.15.3:6789/0 conn(0x7fb7f40a0a30 legacy=0x7fb7f40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:48.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.320+0000 7fb7f3fff700 1 -- 172.21.15.3:0/1853078422 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f409dc50 con 0x7fb7f409d630 2023-10-03T10:16:48.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.320+0000 7fb7f97f0700 1 --2- 172.21.15.3:0/1853078422 >> v2:172.21.15.3:3300/0 conn(0x7fb7f4138d80 0x7fb7f413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:48.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.320+0000 7fb7f3fff700 1 --2- 172.21.15.3:0/1853078422 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7f409d630 0x7fb7f40a1510 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb7e40098f0 tx=0x7fb7e4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fab693bc6663fd4b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:48.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.320+0000 7fb7f37fe700 1 -- 172.21.15.3:0/1853078422 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4011830 con 0x7fb7f409d630 2023-10-03T10:16:48.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.320+0000 7fb7f37fe700 1 -- 172.21.15.3:0/1853078422 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e4011990 con 0x7fb7f409d630 2023-10-03T10:16:48.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.320+0000 7fb7f37fe700 1 -- 172.21.15.3:0/1853078422 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4011d20 con 0x7fb7f409d630 2023-10-03T10:16:48.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.321+0000 7fb7fb253700 1 -- 172.21.15.3:0/1853078422 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7f409d630 msgr2=0x7fb7f40a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:48.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.321+0000 7fb7fb253700 1 --2- 172.21.15.3:0/1853078422 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7f409d630 0x7fb7f40a1510 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb7e40098f0 tx=0x7fb7e4009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:48.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.321+0000 7fb7fb253700 1 -- 172.21.15.3:0/1853078422 shutdown_connections 2023-10-03T10:16:48.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.321+0000 7fb7fb253700 1 --2- 172.21.15.3:0/1853078422 >> v2:172.21.15.3:3300/0 conn(0x7fb7f4138d80 0x7fb7f413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:48.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.321+0000 7fb7fb253700 1 --2- 172.21.15.3:0/1853078422 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7f409d630 0x7fb7f40a1510 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:48.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.321+0000 7fb7fb253700 1 -- 172.21.15.3:0/1853078422 >> 172.21.15.3:0/1853078422 conn(0x7fb7f40a3050 msgr2=0x7fb7f40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:48.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.321+0000 7fb7fb253700 1 -- 172.21.15.3:0/1853078422 shutdown_connections 2023-10-03T10:16:48.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.321+0000 7fb7fb253700 1 -- 172.21.15.3:0/1853078422 wait complete. 2023-10-03T10:16:48.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.323+0000 7fb7fb253700 1 Processor -- start 2023-10-03T10:16:48.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.323+0000 7fb7fb253700 1 -- start start 2023-10-03T10:16:48.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.324+0000 7fb7fb253700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7f409d630 0x7fb7f4133b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:48.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.324+0000 7fb7fb253700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7f40a0a30 0x7fb7f4134090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:48.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.324+0000 7fb7fb253700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7f4138d80 0x7fb7f41345d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:48.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.324+0000 7fb7fb253700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f4135010 con 0x7fb7f4138d80 2023-10-03T10:16:48.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.324+0000 7fb7fb253700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f413afc0 con 0x7fb7f409d630 2023-10-03T10:16:48.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.324+0000 7fb7fb253700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f4135c30 con 0x7fb7f40a0a30 2023-10-03T10:16:48.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.324+0000 7fb7f8fef700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7f409d630 0x7fb7f4133b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:48.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.324+0000 7fb7f8fef700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7f409d630 0x7fb7f4133b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53186/0 (socket says 172.21.15.3:53186) 2023-10-03T10:16:48.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.324+0000 7fb7f8fef700 1 -- 172.21.15.3:0/3332723956 learned_addr learned my addr 172.21.15.3:0/3332723956 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:48.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.324+0000 7fb7f97f0700 1 --2- 172.21.15.3:0/3332723956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7f4138d80 0x7fb7f41345d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:48.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.324+0000 7fb7f3fff700 1 --2- 172.21.15.3:0/3332723956 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7f40a0a30 0x7fb7f4134090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:48.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.324+0000 7fb7f3fff700 1 -- 172.21.15.3:0/3332723956 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7f409d630 msgr2=0x7fb7f4133b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:48.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.324+0000 7fb7f3fff700 1 --2- 172.21.15.3:0/3332723956 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7f409d630 0x7fb7f4133b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:48.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.324+0000 7fb7f3fff700 1 -- 172.21.15.3:0/3332723956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7f4138d80 msgr2=0x7fb7f41345d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:48.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.324+0000 7fb7f3fff700 1 --2- 172.21.15.3:0/3332723956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7f4138d80 0x7fb7f41345d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:48.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.324+0000 7fb7f3fff700 1 -- 172.21.15.3:0/3332723956 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ec009a70 con 0x7fb7f40a0a30 2023-10-03T10:16:48.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.324+0000 7fb7f3fff700 1 --2- 172.21.15.3:0/3332723956 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7f40a0a30 0x7fb7f4134090 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4016000 tx=0x7fb7e4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:48.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.325+0000 7fb7f17fa700 1 -- 172.21.15.3:0/3332723956 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e40045d0 con 0x7fb7f40a0a30 2023-10-03T10:16:48.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.325+0000 7fb7f17fa700 1 -- 172.21.15.3:0/3332723956 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e40036a0 con 0x7fb7f40a0a30 2023-10-03T10:16:48.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.325+0000 7fb7fb253700 1 -- 172.21.15.3:0/3332723956 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e4009550 con 0x7fb7f40a0a30 2023-10-03T10:16:48.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.325+0000 7fb7f17fa700 1 -- 172.21.15.3:0/3332723956 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4003a10 con 0x7fb7f40a0a30 2023-10-03T10:16:48.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.325+0000 7fb7fb253700 1 -- 172.21.15.3:0/3332723956 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f4137590 con 0x7fb7f40a0a30 2023-10-03T10:16:48.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.325+0000 7fb7f17fa700 1 -- 172.21.15.3:0/3332723956 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb7e4021070 con 0x7fb7f40a0a30 2023-10-03T10:16:48.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.326+0000 7fb7f17fa700 1 -- 172.21.15.3:0/3332723956 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb7e40409f0 con 0x7fb7f40a0a30 2023-10-03T10:16:48.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.326+0000 7fb7fb253700 1 -- 172.21.15.3:0/3332723956 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f40747f0 con 0x7fb7f40a0a30 2023-10-03T10:16:48.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.331+0000 7fb7f17fa700 1 -- 172.21.15.3:0/3332723956 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb7e401b7b0 con 0x7fb7f40a0a30 2023-10-03T10:16:48.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.453+0000 7f1cbc611700 1 -- 172.21.15.3:0/4050895081 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1cb4074ea0 con 0x7f1cb409de50 2023-10-03T10:16:48.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.454+0000 7f1caaffd700 1 -- 172.21.15.3:0/4050895081 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1cb0015070 con 0x7f1cb409de50 2023-10-03T10:16:48.454 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:48.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.457+0000 7f1cbc611700 1 -- 172.21.15.3:0/4050895081 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cb409de50 msgr2=0x7f1cb411d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.457+0000 7f1cbc611700 1 --2- 172.21.15.3:0/4050895081 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cb409de50 0x7f1cb411d820 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0016000 tx=0x7f1cb0004750 comp rx=0 tx=0).stop 2023-10-03T10:16:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.457+0000 7f1cbc611700 1 -- 172.21.15.3:0/4050895081 shutdown_connections 2023-10-03T10:16:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.457+0000 7f1cbc611700 1 --2- 172.21.15.3:0/4050895081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cb41338a0 0x7f1cb411e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.457+0000 7f1cbc611700 1 --2- 172.21.15.3:0/4050895081 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cb409de50 0x7f1cb411d820 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.457+0000 7f1cbc611700 1 --2- 172.21.15.3:0/4050895081 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1cb409e930 0x7f1cb411dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.457+0000 7f1cbc611700 1 -- 172.21.15.3:0/4050895081 >> 172.21.15.3:0/4050895081 conn(0x7f1cb40a2aa0 msgr2=0x7f1cb41badd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.457+0000 7f1cbc611700 1 -- 172.21.15.3:0/4050895081 shutdown_connections 2023-10-03T10:16:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.457+0000 7f1cbc611700 1 -- 172.21.15.3:0/4050895081 wait complete. 2023-10-03T10:16:48.472 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:48.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.496+0000 7fb50553b700 1 -- 172.21.15.3:0/584575963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb500130400 con 0x7fb50009d6f0 2023-10-03T10:16:48.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.496+0000 7fb4e77fe700 1 -- 172.21.15.3:0/584575963 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb500130400 con 0x7fb50009d6f0 2023-10-03T10:16:48.497 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:48.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.499+0000 7fb50553b700 1 -- 172.21.15.3:0/584575963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb50009d6f0 msgr2=0x7fb50011e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:48.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.499+0000 7fb50553b700 1 --2- 172.21.15.3:0/584575963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb50009d6f0 0x7fb50011e020 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fb4f000f330 tx=0x7fb4f0004750 comp rx=0 tx=0).stop 2023-10-03T10:16:48.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.499+0000 7fb50553b700 1 -- 172.21.15.3:0/584575963 shutdown_connections 2023-10-03T10:16:48.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.499+0000 7fb50553b700 1 --2- 172.21.15.3:0/584575963 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb50009e410 0x7fb50011eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:48.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.499+0000 7fb50553b700 1 --2- 172.21.15.3:0/584575963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb50009d6f0 0x7fb50011e020 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:48.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.499+0000 7fb50553b700 1 --2- 172.21.15.3:0/584575963 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb50009e000 0x7fb50011e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:48.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.499+0000 7fb50553b700 1 -- 172.21.15.3:0/584575963 >> 172.21.15.3:0/584575963 conn(0x7fb500066f50 msgr2=0x7fb5000710a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:48.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.499+0000 7fb50553b700 1 -- 172.21.15.3:0/584575963 shutdown_connections 2023-10-03T10:16:48.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.499+0000 7fb50553b700 1 -- 172.21.15.3:0/584575963 wait complete. 2023-10-03T10:16:48.513 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:48.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.537+0000 7fb7fb253700 1 -- 172.21.15.3:0/3332723956 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb7f4072980 con 0x7fb7f40a0a30 2023-10-03T10:16:48.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.537+0000 7fb7f17fa700 1 -- 172.21.15.3:0/3332723956 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb7e4015020 con 0x7fb7f40a0a30 2023-10-03T10:16:48.538 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:48.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.539+0000 7fb7fb253700 1 -- 172.21.15.3:0/3332723956 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7f40a0a30 msgr2=0x7fb7f4134090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:48.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.539+0000 7fb7fb253700 1 --2- 172.21.15.3:0/3332723956 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7f40a0a30 0x7fb7f4134090 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4016000 tx=0x7fb7e4004750 comp rx=0 tx=0).stop 2023-10-03T10:16:48.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.540+0000 7fb7fb253700 1 -- 172.21.15.3:0/3332723956 shutdown_connections 2023-10-03T10:16:48.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.540+0000 7fb7fb253700 1 --2- 172.21.15.3:0/3332723956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7f4138d80 0x7fb7f41345d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:48.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.540+0000 7fb7fb253700 1 --2- 172.21.15.3:0/3332723956 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb7f409d630 0x7fb7f4133b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:48.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.540+0000 7fb7fb253700 1 --2- 172.21.15.3:0/3332723956 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb7f40a0a30 0x7fb7f4134090 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:48.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.540+0000 7fb7fb253700 1 -- 172.21.15.3:0/3332723956 >> 172.21.15.3:0/3332723956 conn(0x7fb7f40a3050 msgr2=0x7fb7f409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:48.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.540+0000 7fb7fb253700 1 -- 172.21.15.3:0/3332723956 shutdown_connections 2023-10-03T10:16:48.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:48.540+0000 7fb7fb253700 1 -- 172.21.15.3:0/3332723956 wait complete. 2023-10-03T10:16:48.553 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:49.254 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:49.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.402+0000 7f2c29e36700 1 Processor -- start 2023-10-03T10:16:49.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.402+0000 7f2c29e36700 1 -- start start 2023-10-03T10:16:49.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.402+0000 7f2c29e36700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2c2409e7e0 0x7f2c2409ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:49.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.402+0000 7f2c29e36700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c241391d0 0x7f2c2413b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:49.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.402+0000 7f2c29e36700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c2406bd70 con 0x7f2c2409f0f0 2023-10-03T10:16:49.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.402+0000 7f2c29e36700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c2406cc30 con 0x7f2c2409e7e0 2023-10-03T10:16:49.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.402+0000 7f2c29e36700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2406d870 con 0x7f2c241391d0 2023-10-03T10:16:49.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.403+0000 7f2c23fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c241391d0 0x7f2c2413b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:49.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.403+0000 7f2c237fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2c2409e7e0 0x7f2c2409ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:49.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.403+0000 7f2c237fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2c2409e7e0 0x7f2c2409ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48552/0 (socket says 172.21.15.3:48552) 2023-10-03T10:16:49.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.403+0000 7f2c23fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c241391d0 0x7f2c2413b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51740/0 (socket says 172.21.15.3:51740) 2023-10-03T10:16:49.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.403+0000 7f2c237fe700 1 -- 172.21.15.3:0/592646799 learned_addr learned my addr 172.21.15.3:0/592646799 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:49.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.403+0000 7f2c237fe700 1 -- 172.21.15.3:0/592646799 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c241391d0 msgr2=0x7f2c2413b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:49.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.403+0000 7f2c237fe700 1 --2- 172.21.15.3:0/592646799 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c241391d0 0x7f2c2413b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.403+0000 7f2c237fe700 1 -- 172.21.15.3:0/592646799 >> v1:172.21.15.3:6789/0 conn(0x7f2c2409f0f0 legacy=0x7f2c24137ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:49.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.403+0000 7f2c237fe700 1 -- 172.21.15.3:0/592646799 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2406bab0 con 0x7f2c2409e7e0 2023-10-03T10:16:49.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.403+0000 7f2c23fff700 1 --2- 172.21.15.3:0/592646799 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c241391d0 0x7f2c2413b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:49.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.403+0000 7f2c237fe700 1 --2- 172.21.15.3:0/592646799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c2409e7e0 0x7f2c2409ebb0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f2c140098f0 tx=0x7f2c14009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c853da4dd0f620e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:49.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.403+0000 7f2c227fc700 1 -- 172.21.15.3:0/592646799 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c14011830 con 0x7f2c2409e7e0 2023-10-03T10:16:49.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.403+0000 7f2c227fc700 1 -- 172.21.15.3:0/592646799 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c14011990 con 0x7f2c2409e7e0 2023-10-03T10:16:49.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.403+0000 7f2c227fc700 1 -- 172.21.15.3:0/592646799 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c14011d20 con 0x7f2c2409e7e0 2023-10-03T10:16:49.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.404+0000 7f2c29e36700 1 -- 172.21.15.3:0/592646799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c2409e7e0 msgr2=0x7f2c2409ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:49.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.404+0000 7f2c29e36700 1 --2- 172.21.15.3:0/592646799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c2409e7e0 0x7f2c2409ebb0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f2c140098f0 tx=0x7f2c14009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:49.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.404+0000 7f2c29e36700 1 -- 172.21.15.3:0/592646799 shutdown_connections 2023-10-03T10:16:49.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.404+0000 7f2c29e36700 1 --2- 172.21.15.3:0/592646799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c2409e7e0 0x7f2c2409ebb0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.404+0000 7f2c29e36700 1 --2- 172.21.15.3:0/592646799 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c241391d0 0x7f2c2413b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.404+0000 7f2c29e36700 1 -- 172.21.15.3:0/592646799 >> 172.21.15.3:0/592646799 conn(0x7f2c240a4300 msgr2=0x7f2c240a4700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:49.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.404+0000 7f2c29e36700 1 -- 172.21.15.3:0/592646799 shutdown_connections 2023-10-03T10:16:49.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.404+0000 7f2c29e36700 1 -- 172.21.15.3:0/592646799 wait complete. 2023-10-03T10:16:49.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.405+0000 7f2c29e36700 1 Processor -- start 2023-10-03T10:16:49.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.405+0000 7f2c29e36700 1 -- start start 2023-10-03T10:16:49.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.406+0000 7f2c29e36700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c2409e7e0 0x7f2c2411e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:49.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.406+0000 7f2c29e36700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2c2409f0f0 0x7f2c2411ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:49.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.406+0000 7f2c29e36700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c241391d0 0x7f2c2411f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:49.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.406+0000 7f2c29e36700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c24134320 con 0x7f2c241391d0 2023-10-03T10:16:49.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.406+0000 7f2c29e36700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c24121980 con 0x7f2c2409f0f0 2023-10-03T10:16:49.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.406+0000 7f2c29e36700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c24120c00 con 0x7f2c2409e7e0 2023-10-03T10:16:49.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.406+0000 7f2c237fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c2409e7e0 0x7f2c2411e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:49.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.406+0000 7f2c23fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c241391d0 0x7f2c2411f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:49.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.406+0000 7f2c237fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c2409e7e0 0x7f2c2411e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51752/0 (socket says 172.21.15.3:51752) 2023-10-03T10:16:49.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.406+0000 7f2c237fe700 1 -- 172.21.15.3:0/2040532255 learned_addr learned my addr 172.21.15.3:0/2040532255 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:49.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.406+0000 7f2c22ffd700 1 --2- 172.21.15.3:0/2040532255 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2c2409f0f0 0x7f2c2411ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:49.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.406+0000 7f2c237fe700 1 -- 172.21.15.3:0/2040532255 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2c2409f0f0 msgr2=0x7f2c2411ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:49.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.406+0000 7f2c237fe700 1 --2- 172.21.15.3:0/2040532255 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2c2409f0f0 0x7f2c2411ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.406+0000 7f2c237fe700 1 -- 172.21.15.3:0/2040532255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c241391d0 msgr2=0x7f2c2411f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:49.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.406+0000 7f2c237fe700 1 --2- 172.21.15.3:0/2040532255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c241391d0 0x7f2c2411f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.406+0000 7f2c237fe700 1 -- 172.21.15.3:0/2040532255 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c1c009a70 con 0x7f2c2409e7e0 2023-10-03T10:16:49.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.406+0000 7f2c237fe700 1 --2- 172.21.15.3:0/2040532255 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c2409e7e0 0x7f2c2411e940 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2c14000c00 tx=0x7f2c14004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:49.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.406+0000 7f2c1bfff700 1 -- 172.21.15.3:0/2040532255 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c140045d0 con 0x7f2c2409e7e0 2023-10-03T10:16:49.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.407+0000 7f2c1bfff700 1 -- 172.21.15.3:0/2040532255 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c140036a0 con 0x7f2c2409e7e0 2023-10-03T10:16:49.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.407+0000 7f2c29e36700 1 -- 172.21.15.3:0/2040532255 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c14009550 con 0x7f2c2409e7e0 2023-10-03T10:16:49.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.407+0000 7f2c1bfff700 1 -- 172.21.15.3:0/2040532255 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c14003a50 con 0x7f2c2409e7e0 2023-10-03T10:16:49.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.407+0000 7f2c29e36700 1 -- 172.21.15.3:0/2040532255 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c240a3ab0 con 0x7f2c2409e7e0 2023-10-03T10:16:49.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.407+0000 7f2c1bfff700 1 -- 172.21.15.3:0/2040532255 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2c14021070 con 0x7f2c2409e7e0 2023-10-03T10:16:49.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.408+0000 7f2c29e36700 1 -- 172.21.15.3:0/2040532255 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c2404f2a0 con 0x7f2c2409e7e0 2023-10-03T10:16:49.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.408+0000 7f2c1bfff700 1 -- 172.21.15.3:0/2040532255 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2c14040b30 con 0x7f2c2409e7e0 2023-10-03T10:16:49.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.413+0000 7f2c1bfff700 1 -- 172.21.15.3:0/2040532255 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2c14015070 con 0x7f2c2409e7e0 2023-10-03T10:16:49.473 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:49.513 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:49.555 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:49.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.614+0000 7fe3ab675700 1 Processor -- start 2023-10-03T10:16:49.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.615+0000 7fe3ab675700 1 -- start start 2023-10-03T10:16:49.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.615+0000 7fe3ab675700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3a409e1d0 0x7fe3a4138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:49.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.615+0000 7fe3ab675700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a4139170 0x7fe3a413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:49.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.615+0000 7fe3ab675700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a409b120 con 0x7fe3a409d6f0 2023-10-03T10:16:49.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.615+0000 7fe3ab675700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a406b460 con 0x7fe3a4139170 2023-10-03T10:16:49.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.615+0000 7fe3ab675700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a409a7e0 con 0x7fe3a409e1d0 2023-10-03T10:16:49.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.615+0000 7fe3a8c10700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3a409e1d0 0x7fe3a4138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:49.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.615+0000 7fe3a9c12700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a4139170 0x7fe3a413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:49.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.615+0000 7fe3a8c10700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3a409e1d0 0x7fe3a4138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51764/0 (socket says 172.21.15.3:51764) 2023-10-03T10:16:49.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.615+0000 7fe3a9c12700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a4139170 0x7fe3a413b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53212/0 (socket says 172.21.15.3:53212) 2023-10-03T10:16:49.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.615+0000 7fe3a8c10700 1 -- 172.21.15.3:0/312050222 learned_addr learned my addr 172.21.15.3:0/312050222 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:49.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.615+0000 7fe3a9411700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe3a409d6f0 0x7fe3a409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53234/0 (socket says 172.21.15.3:53234) 2023-10-03T10:16:49.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.616+0000 7fe3a8c10700 1 -- 172.21.15.3:0/312050222 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a4139170 msgr2=0x7fe3a413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:49.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.616+0000 7fe3a8c10700 1 --2- 172.21.15.3:0/312050222 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a4139170 0x7fe3a413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.616+0000 7fe3a8c10700 1 -- 172.21.15.3:0/312050222 >> v1:172.21.15.3:6789/0 conn(0x7fe3a409d6f0 legacy=0x7fe3a409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:49.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.616+0000 7fe3a8c10700 1 -- 172.21.15.3:0/312050222 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a409a500 con 0x7fe3a409e1d0 2023-10-03T10:16:49.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.616+0000 7fe3a8c10700 1 --2- 172.21.15.3:0/312050222 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3a409e1d0 0x7fe3a4138c30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe394009950 tx=0x7fe394009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44a36cb577df99f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:49.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.616+0000 7fe39bfff700 1 -- 172.21.15.3:0/312050222 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe394011830 con 0x7fe3a409e1d0 2023-10-03T10:16:49.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.616+0000 7fe39bfff700 1 -- 172.21.15.3:0/312050222 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe394011990 con 0x7fe3a409e1d0 2023-10-03T10:16:49.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.616+0000 7fe39bfff700 1 -- 172.21.15.3:0/312050222 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe394011d40 con 0x7fe3a409e1d0 2023-10-03T10:16:49.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.617+0000 7fe3ab675700 1 -- 172.21.15.3:0/312050222 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3a409e1d0 msgr2=0x7fe3a4138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:49.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.617+0000 7fe3ab675700 1 --2- 172.21.15.3:0/312050222 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3a409e1d0 0x7fe3a4138c30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe394009950 tx=0x7fe394009c60 comp rx=0 tx=0).stop 2023-10-03T10:16:49.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.617+0000 7fe3ab675700 1 -- 172.21.15.3:0/312050222 shutdown_connections 2023-10-03T10:16:49.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.617+0000 7fe3ab675700 1 --2- 172.21.15.3:0/312050222 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a4139170 0x7fe3a413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.617+0000 7fe3ab675700 1 --2- 172.21.15.3:0/312050222 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3a409e1d0 0x7fe3a4138c30 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.617+0000 7fe3ab675700 1 -- 172.21.15.3:0/312050222 >> 172.21.15.3:0/312050222 conn(0x7fe3a4066f50 msgr2=0x7fe3a409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:49.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.618+0000 7fe3ab675700 1 -- 172.21.15.3:0/312050222 shutdown_connections 2023-10-03T10:16:49.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.618+0000 7fe3ab675700 1 -- 172.21.15.3:0/312050222 wait complete. 2023-10-03T10:16:49.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.619+0000 7fe3ab675700 1 Processor -- start 2023-10-03T10:16:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.619+0000 7fe3ab675700 1 -- start start 2023-10-03T10:16:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.619+0000 7fe3ab675700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a409d6f0 0x7fe3a411e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.619+0000 7fe3ab675700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3a409e1d0 0x7fe3a411edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.619+0000 7fe3ab675700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3a4139170 0x7fe3a411f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.619+0000 7fe3ab675700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a4121a90 con 0x7fe3a4139170 2023-10-03T10:16:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.619+0000 7fe3ab675700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a4122040 con 0x7fe3a409d6f0 2023-10-03T10:16:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.619+0000 7fe3ab675700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a4120410 con 0x7fe3a409e1d0 2023-10-03T10:16:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.619+0000 7fe3a9411700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a409d6f0 0x7fe3a411e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.619+0000 7fe3a9411700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a409d6f0 0x7fe3a411e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53224/0 (socket says 172.21.15.3:53224) 2023-10-03T10:16:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.619+0000 7fe3a9411700 1 -- 172.21.15.3:0/3677506614 learned_addr learned my addr 172.21.15.3:0/3677506614 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.619+0000 7fe3a9c12700 1 --2- 172.21.15.3:0/3677506614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3a4139170 0x7fe3a411f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:49.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.620+0000 7fe3a9c12700 1 -- 172.21.15.3:0/3677506614 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3a409e1d0 msgr2=0x7fe3a411edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:49.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.620+0000 7fe3a9c12700 1 --2- 172.21.15.3:0/3677506614 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3a409e1d0 0x7fe3a411edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.620+0000 7fe3a9c12700 1 -- 172.21.15.3:0/3677506614 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a409d6f0 msgr2=0x7fe3a411e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:49.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.620+0000 7fe3a9c12700 1 --2- 172.21.15.3:0/3677506614 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a409d6f0 0x7fe3a411e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.620+0000 7fe3a9c12700 1 -- 172.21.15.3:0/3677506614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe394009550 con 0x7fe3a4139170 2023-10-03T10:16:49.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.620+0000 7fe3a9c12700 1 --2- 172.21.15.3:0/3677506614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3a4139170 0x7fe3a411f310 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00efc0 tx=0x7fe39c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:49.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.620+0000 7fe399ffb700 1 -- 172.21.15.3:0/3677506614 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c00cd40 con 0x7fe3a4139170 2023-10-03T10:16:49.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.620+0000 7fe399ffb700 1 -- 172.21.15.3:0/3677506614 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39c00f040 con 0x7fe3a4139170 2023-10-03T10:16:49.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.620+0000 7fe3ab675700 1 -- 172.21.15.3:0/3677506614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a40a2120 con 0x7fe3a4139170 2023-10-03T10:16:49.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.620+0000 7fe399ffb700 1 -- 172.21.15.3:0/3677506614 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c00cea0 con 0x7fe3a4139170 2023-10-03T10:16:49.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.621+0000 7fe3ab675700 1 -- 172.21.15.3:0/3677506614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a4120bd0 con 0x7fe3a4139170 2023-10-03T10:16:49.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.622+0000 7fe399ffb700 1 -- 172.21.15.3:0/3677506614 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe39c0146c0 con 0x7fe3a4139170 2023-10-03T10:16:49.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.622+0000 7fe3ab675700 1 -- 172.21.15.3:0/3677506614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a4074ea0 con 0x7fe3a4139170 2023-10-03T10:16:49.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.622+0000 7fe399ffb700 1 -- 172.21.15.3:0/3677506614 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe39c0402a0 con 0x7fe3a4139170 2023-10-03T10:16:49.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.624+0000 7f2c29e36700 1 -- 172.21.15.3:0/2040532255 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2c24121e70 con 0x7f2c2409e7e0 2023-10-03T10:16:49.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.624+0000 7f2c1bfff700 1 -- 172.21.15.3:0/2040532255 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2c24121e70 con 0x7f2c2409e7e0 2023-10-03T10:16:49.624 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:49.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.626+0000 7f2c29e36700 1 -- 172.21.15.3:0/2040532255 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c2409e7e0 msgr2=0x7f2c2411e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:49.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.626+0000 7f2c29e36700 1 --2- 172.21.15.3:0/2040532255 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c2409e7e0 0x7f2c2411e940 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2c14000c00 tx=0x7f2c14004750 comp rx=0 tx=0).stop 2023-10-03T10:16:49.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.627+0000 7fe399ffb700 1 -- 172.21.15.3:0/3677506614 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe39c014a10 con 0x7fe3a4139170 2023-10-03T10:16:49.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.628+0000 7f2c29e36700 1 -- 172.21.15.3:0/2040532255 shutdown_connections 2023-10-03T10:16:49.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.628+0000 7f2c29e36700 1 --2- 172.21.15.3:0/2040532255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c241391d0 0x7f2c2411f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.628+0000 7f2c29e36700 1 --2- 172.21.15.3:0/2040532255 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2c2409e7e0 0x7f2c2411e940 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.628+0000 7f2c29e36700 1 --2- 172.21.15.3:0/2040532255 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2c2409f0f0 0x7f2c2411ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.628+0000 7f2c29e36700 1 -- 172.21.15.3:0/2040532255 >> 172.21.15.3:0/2040532255 conn(0x7f2c240a4300 msgr2=0x7f2c24136cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:49.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.628+0000 7f2c29e36700 1 -- 172.21.15.3:0/2040532255 shutdown_connections 2023-10-03T10:16:49.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.628+0000 7f2c29e36700 1 -- 172.21.15.3:0/2040532255 wait complete. 2023-10-03T10:16:49.649 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.663+0000 7f113e546700 1 Processor -- start 2023-10-03T10:16:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.663+0000 7f113e546700 1 -- start start 2023-10-03T10:16:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.664+0000 7f113e546700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1138130540 0x7f1138130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.664+0000 7f113e546700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1138130e50 0x7f113806f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.664+0000 7f113e546700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f113806fb20 0x7f113806ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.664+0000 7f113e546700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11380704b0 con 0x7f1138130e50 2023-10-03T10:16:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.664+0000 7f113e546700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f113809a390 con 0x7f1138130540 2023-10-03T10:16:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.664+0000 7f113e546700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11380716a0 con 0x7f113806fb20 2023-10-03T10:16:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.664+0000 7f1137fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1138130540 0x7f1138130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.664+0000 7f11377fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1138130e50 0x7f113806f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.664+0000 7f11377fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1138130e50 0x7f113806f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48588/0 (socket says 172.21.15.3:48588) 2023-10-03T10:16:49.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.664+0000 7f1137fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1138130540 0x7f1138130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53238/0 (socket says 172.21.15.3:53238) 2023-10-03T10:16:49.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.664+0000 7f113cae3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f113806fb20 0x7f113806ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:49.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.664+0000 7f11377fe700 1 -- 172.21.15.3:0/2861450080 learned_addr learned my addr 172.21.15.3:0/2861450080 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:49.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.664+0000 7f11377fe700 1 -- 172.21.15.3:0/2861450080 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f113806fb20 msgr2=0x7f113806ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:49.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.664+0000 7f11377fe700 1 --2- 172.21.15.3:0/2861450080 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f113806fb20 0x7f113806ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.664+0000 7f11377fe700 1 -- 172.21.15.3:0/2861450080 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1138130540 msgr2=0x7f1138130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:49.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.664+0000 7f11377fe700 1 --2- 172.21.15.3:0/2861450080 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1138130540 0x7f1138130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.664+0000 7f11377fe700 1 -- 172.21.15.3:0/2861450080 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1138071290 con 0x7f1138130e50 2023-10-03T10:16:49.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.664+0000 7f113cae3700 1 --2- 172.21.15.3:0/2861450080 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f113806fb20 0x7f113806ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:49.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.664+0000 7f1137fff700 1 --2- 172.21.15.3:0/2861450080 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1138130540 0x7f1138130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:49.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.665+0000 7f11377fe700 1 --2- 172.21.15.3:0/2861450080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1138130e50 0x7f113806f5e0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f112c013210 tx=0x7f112c013520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae1dd15b96e42b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:49.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.665+0000 7f1136ffd700 1 -- 172.21.15.3:0/2861450080 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c016070 con 0x7f1138130e50 2023-10-03T10:16:49.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.665+0000 7f1136ffd700 1 -- 172.21.15.3:0/2861450080 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112c013d70 con 0x7f1138130e50 2023-10-03T10:16:49.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.665+0000 7f1136ffd700 1 -- 172.21.15.3:0/2861450080 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c0112f0 con 0x7f1138130e50 2023-10-03T10:16:49.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.665+0000 7f113e546700 1 -- 172.21.15.3:0/2861450080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1138130e50 msgr2=0x7f113806f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:49.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.665+0000 7f113e546700 1 --2- 172.21.15.3:0/2861450080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1138130e50 0x7f113806f5e0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f112c013210 tx=0x7f112c013520 comp rx=0 tx=0).stop 2023-10-03T10:16:49.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.666+0000 7f113e546700 1 -- 172.21.15.3:0/2861450080 shutdown_connections 2023-10-03T10:16:49.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.666+0000 7f113e546700 1 --2- 172.21.15.3:0/2861450080 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f113806fb20 0x7f113806ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.666+0000 7f113e546700 1 --2- 172.21.15.3:0/2861450080 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1138130540 0x7f1138130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.666+0000 7f113e546700 1 --2- 172.21.15.3:0/2861450080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1138130e50 0x7f113806f5e0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.666+0000 7f113e546700 1 -- 172.21.15.3:0/2861450080 >> 172.21.15.3:0/2861450080 conn(0x7f11380a2aa0 msgr2=0x7f11380a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:49.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.666+0000 7f113e546700 1 -- 172.21.15.3:0/2861450080 shutdown_connections 2023-10-03T10:16:49.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.666+0000 7f113e546700 1 -- 172.21.15.3:0/2861450080 wait complete. 2023-10-03T10:16:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.667+0000 7f113e546700 1 Processor -- start 2023-10-03T10:16:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.667+0000 7f113e546700 1 -- start start 2023-10-03T10:16:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.667+0000 7f113e546700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f113806fb20 0x7f1138118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.667+0000 7f113e546700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1138130540 0x7f1138119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.667+0000 7f113e546700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1138130e50 0x7f1138119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.667+0000 7f113e546700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113811b9f0 con 0x7f113806fb20 2023-10-03T10:16:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.667+0000 7f113e546700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f113811a870 con 0x7f1138130e50 2023-10-03T10:16:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.667+0000 7f113e546700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f113811a190 con 0x7f1138130540 2023-10-03T10:16:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.668+0000 7f1137fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f113806fb20 0x7f1138118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.668+0000 7f1137fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f113806fb20 0x7f1138118cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48598/0 (socket says 172.21.15.3:48598) 2023-10-03T10:16:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.668+0000 7f1137fff700 1 -- 172.21.15.3:0/808715806 learned_addr learned my addr 172.21.15.3:0/808715806 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.668+0000 7f113cae3700 1 --2- 172.21.15.3:0/808715806 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1138130e50 0x7f1138119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.668+0000 7f11377fe700 1 --2- 172.21.15.3:0/808715806 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1138130540 0x7f1138119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.668+0000 7f1137fff700 1 -- 172.21.15.3:0/808715806 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1138130540 msgr2=0x7f1138119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.668+0000 7f1137fff700 1 --2- 172.21.15.3:0/808715806 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1138130540 0x7f1138119210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.668+0000 7f1137fff700 1 -- 172.21.15.3:0/808715806 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1138130e50 msgr2=0x7f1138119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.668+0000 7f1137fff700 1 --2- 172.21.15.3:0/808715806 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1138130e50 0x7f1138119750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.668+0000 7f1137fff700 1 -- 172.21.15.3:0/808715806 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1130009a70 con 0x7f113806fb20 2023-10-03T10:16:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.668+0000 7f113cae3700 1 --2- 172.21.15.3:0/808715806 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1138130e50 0x7f1138119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.668+0000 7f1137fff700 1 --2- 172.21.15.3:0/808715806 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f113806fb20 0x7f1138118cd0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f112800edf0 tx=0x7f112800c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.668+0000 7f1134ff9700 1 -- 172.21.15.3:0/808715806 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112800ee80 con 0x7f113806fb20 2023-10-03T10:16:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.668+0000 7f1134ff9700 1 -- 172.21.15.3:0/808715806 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112800cb50 con 0x7f113806fb20 2023-10-03T10:16:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.668+0000 7f1134ff9700 1 -- 172.21.15.3:0/808715806 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112800cec0 con 0x7f113806fb20 2023-10-03T10:16:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.668+0000 7f113e546700 1 -- 172.21.15.3:0/808715806 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11280096d0 con 0x7f113806fb20 2023-10-03T10:16:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.669+0000 7f113e546700 1 -- 172.21.15.3:0/808715806 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f112c011df0 con 0x7f113806fb20 2023-10-03T10:16:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.670+0000 7f113e546700 1 -- 172.21.15.3:0/808715806 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1138074ea0 con 0x7f113806fb20 2023-10-03T10:16:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.670+0000 7f1134ff9700 1 -- 172.21.15.3:0/808715806 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f11280070d0 con 0x7f113806fb20 2023-10-03T10:16:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.670+0000 7f1134ff9700 1 -- 172.21.15.3:0/808715806 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f112803ff40 con 0x7f113806fb20 2023-10-03T10:16:49.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.675+0000 7f1134ff9700 1 -- 172.21.15.3:0/808715806 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1128011460 con 0x7f113806fb20 2023-10-03T10:16:49.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.709+0000 7f3279c13700 1 Processor -- start 2023-10-03T10:16:49.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.709+0000 7f3279c13700 1 -- start start 2023-10-03T10:16:49.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.709+0000 7f3279c13700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f327409d630 0x7f32740a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:49.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.709+0000 7f3279c13700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3274138d80 0x7f327413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:49.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.710+0000 7f3279c13700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32740a2a70 con 0x7f32740a0a30 2023-10-03T10:16:49.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.710+0000 7f3279c13700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32740a01d0 con 0x7f3274138d80 2023-10-03T10:16:49.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.710+0000 7f3279c13700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f327409e2b0 con 0x7f327409d630 2023-10-03T10:16:49.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.710+0000 7f3272ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f327409d630 0x7f32740a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:49.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.710+0000 7f32737fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f32740a0a30 0x7f32740a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53244/0 (socket says 172.21.15.3:53244) 2023-10-03T10:16:49.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.710+0000 7f32737fe700 1 -- 172.21.15.3:0/389493537 learned_addr learned my addr 172.21.15.3:0/389493537 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:49.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.710+0000 7f3273fff700 1 --2- 172.21.15.3:0/389493537 >> v2:172.21.15.3:3300/0 conn(0x7f3274138d80 0x7f327413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:49.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.710+0000 7f3273fff700 1 -- 172.21.15.3:0/389493537 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f327409d630 msgr2=0x7f32740a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:49.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.710+0000 7f3273fff700 1 --2- 172.21.15.3:0/389493537 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f327409d630 0x7f32740a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.710+0000 7f3273fff700 1 -- 172.21.15.3:0/389493537 >> v1:172.21.15.3:6789/0 conn(0x7f32740a0a30 legacy=0x7f32740a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:49.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.710+0000 7f3273fff700 1 -- 172.21.15.3:0/389493537 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327409dc50 con 0x7f3274138d80 2023-10-03T10:16:49.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.710+0000 7f32727fc700 1 -- 172.21.15.3:0/389493537 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1821705287 0 0) 0x7f32740a2a70 con 0x7f32740a0a30 2023-10-03T10:16:49.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.710+0000 7f3272ffd700 1 --2- 172.21.15.3:0/389493537 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f327409d630 0x7f32740a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:49.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.710+0000 7f3273fff700 1 --2- 172.21.15.3:0/389493537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3274138d80 0x7f327413b160 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f326c009d90 tx=0x7f326c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2206144ff22733ad server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:49.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.711+0000 7f32727fc700 1 -- 172.21.15.3:0/389493537 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c00e070 con 0x7f3274138d80 2023-10-03T10:16:49.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.711+0000 7f32727fc700 1 -- 172.21.15.3:0/389493537 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326c00bb20 con 0x7f3274138d80 2023-10-03T10:16:49.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.711+0000 7f32727fc700 1 -- 172.21.15.3:0/389493537 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c012430 con 0x7f3274138d80 2023-10-03T10:16:49.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.711+0000 7f3279c13700 1 -- 172.21.15.3:0/389493537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3274138d80 msgr2=0x7f327413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:49.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.711+0000 7f3279c13700 1 --2- 172.21.15.3:0/389493537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3274138d80 0x7f327413b160 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f326c009d90 tx=0x7f326c00b330 comp rx=0 tx=0).stop 2023-10-03T10:16:49.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.711+0000 7f3279c13700 1 -- 172.21.15.3:0/389493537 shutdown_connections 2023-10-03T10:16:49.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.711+0000 7f3279c13700 1 --2- 172.21.15.3:0/389493537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3274138d80 0x7f327413b160 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.711+0000 7f3279c13700 1 --2- 172.21.15.3:0/389493537 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f327409d630 0x7f32740a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.711+0000 7f3279c13700 1 -- 172.21.15.3:0/389493537 >> 172.21.15.3:0/389493537 conn(0x7f32740a3050 msgr2=0x7f32740a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:49.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.711+0000 7f3279c13700 1 -- 172.21.15.3:0/389493537 shutdown_connections 2023-10-03T10:16:49.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.711+0000 7f3279c13700 1 -- 172.21.15.3:0/389493537 wait complete. 2023-10-03T10:16:49.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.713+0000 7f3279c13700 1 Processor -- start 2023-10-03T10:16:49.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.713+0000 7f3279c13700 1 -- start start 2023-10-03T10:16:49.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.714+0000 7f3279c13700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f327409d630 0x7f327412c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:49.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.714+0000 7f3279c13700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f32740a0a30 0x7f327412cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:49.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.714+0000 7f3279c13700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3274138d80 0x7f327412cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:49.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.714+0000 7f3279c13700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3274130190 con 0x7f327409d630 2023-10-03T10:16:49.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.714+0000 7f3279c13700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f327412fc10 con 0x7f32740a0a30 2023-10-03T10:16:49.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.714+0000 7f3279c13700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32740a0410 con 0x7f3274138d80 2023-10-03T10:16:49.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.714+0000 7f3272ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f32740a0a30 0x7f327412cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:49.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.714+0000 7f32737fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f327409d630 0x7f327412c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:49.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.714+0000 7f3272ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f32740a0a30 0x7f327412cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53254/0 (socket says 172.21.15.3:53254) 2023-10-03T10:16:49.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.714+0000 7f3272ffd700 1 -- 172.21.15.3:0/2161565673 learned_addr learned my addr 172.21.15.3:0/2161565673 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:49.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.714+0000 7f32737fe700 1 -- 172.21.15.3:0/2161565673 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3274138d80 msgr2=0x7f327412cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:49.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.714+0000 7f32737fe700 1 --2- 172.21.15.3:0/2161565673 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3274138d80 0x7f327412cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.714+0000 7f32737fe700 1 -- 172.21.15.3:0/2161565673 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f32740a0a30 msgr2=0x7f327412cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:49.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.714+0000 7f32737fe700 1 --2- 172.21.15.3:0/2161565673 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f32740a0a30 0x7f327412cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.714+0000 7f32737fe700 1 -- 172.21.15.3:0/2161565673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32680094a0 con 0x7f327409d630 2023-10-03T10:16:49.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.714+0000 7f3272ffd700 1 --2- 172.21.15.3:0/2161565673 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f32740a0a30 0x7f327412cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:49.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.714+0000 7f32737fe700 1 --2- 172.21.15.3:0/2161565673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f327409d630 0x7f327412c570 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f326400c470 tx=0x7f326400c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:49.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.715+0000 7f325bfff700 1 -- 172.21.15.3:0/2161565673 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3264004c80 con 0x7f327409d630 2023-10-03T10:16:49.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.715+0000 7f325bfff700 1 -- 172.21.15.3:0/2161565673 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3264004de0 con 0x7f327409d630 2023-10-03T10:16:49.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.715+0000 7f3279c13700 1 -- 172.21.15.3:0/2161565673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f326c009a70 con 0x7f327409d630 2023-10-03T10:16:49.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.715+0000 7f325bfff700 1 -- 172.21.15.3:0/2161565673 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3264007310 con 0x7f327409d630 2023-10-03T10:16:49.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.715+0000 7f3279c13700 1 -- 172.21.15.3:0/2161565673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f327412e5a0 con 0x7f327409d630 2023-10-03T10:16:49.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.716+0000 7f325bfff700 1 -- 172.21.15.3:0/2161565673 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3264007470 con 0x7f327409d630 2023-10-03T10:16:49.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.716+0000 7f325bfff700 1 -- 172.21.15.3:0/2161565673 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f326403f330 con 0x7f327409d630 2023-10-03T10:16:49.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.716+0000 7f3279c13700 1 -- 172.21.15.3:0/2161565673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32740a4a70 con 0x7f327409d630 2023-10-03T10:16:49.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.721+0000 7f325bfff700 1 -- 172.21.15.3:0/2161565673 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3264007720 con 0x7f327409d630 2023-10-03T10:16:49.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.848+0000 7fe3ab675700 1 -- 172.21.15.3:0/3677506614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe3a406c360 con 0x7fe3a4139170 2023-10-03T10:16:49.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.848+0000 7fe399ffb700 1 -- 172.21.15.3:0/3677506614 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe39c018020 con 0x7fe3a4139170 2023-10-03T10:16:49.849 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:49.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.850+0000 7fe3ab675700 1 -- 172.21.15.3:0/3677506614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3a4139170 msgr2=0x7fe3a411f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:49.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.850+0000 7fe3ab675700 1 --2- 172.21.15.3:0/3677506614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3a4139170 0x7fe3a411f310 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00efc0 tx=0x7fe39c00c3a0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.850+0000 7fe3ab675700 1 -- 172.21.15.3:0/3677506614 shutdown_connections 2023-10-03T10:16:49.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.850+0000 7fe3ab675700 1 --2- 172.21.15.3:0/3677506614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3a4139170 0x7fe3a411f310 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.850+0000 7fe3ab675700 1 --2- 172.21.15.3:0/3677506614 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3a409d6f0 0x7fe3a411e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.850+0000 7fe3ab675700 1 --2- 172.21.15.3:0/3677506614 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3a409e1d0 0x7fe3a411edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.850+0000 7fe3ab675700 1 -- 172.21.15.3:0/3677506614 >> 172.21.15.3:0/3677506614 conn(0x7fe3a4066f50 msgr2=0x7fe3a4138090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:49.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.851+0000 7fe3ab675700 1 -- 172.21.15.3:0/3677506614 shutdown_connections 2023-10-03T10:16:49.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.851+0000 7fe3ab675700 1 -- 172.21.15.3:0/3677506614 wait complete. 2023-10-03T10:16:49.864 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.883+0000 7f113e546700 1 -- 172.21.15.3:0/808715806 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f113811ae10 con 0x7f113806fb20 2023-10-03T10:16:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.884+0000 7f1134ff9700 1 -- 172.21.15.3:0/808715806 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1128019360 con 0x7f113806fb20 2023-10-03T10:16:49.884 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.886+0000 7f113e546700 1 -- 172.21.15.3:0/808715806 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f113806fb20 msgr2=0x7f1138118cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.886+0000 7f113e546700 1 --2- 172.21.15.3:0/808715806 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f113806fb20 0x7f1138118cd0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f112800edf0 tx=0x7f112800c540 comp rx=0 tx=0).stop 2023-10-03T10:16:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.886+0000 7f113e546700 1 -- 172.21.15.3:0/808715806 shutdown_connections 2023-10-03T10:16:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.886+0000 7f113e546700 1 --2- 172.21.15.3:0/808715806 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1138130e50 0x7f1138119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.886+0000 7f113e546700 1 --2- 172.21.15.3:0/808715806 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f113806fb20 0x7f1138118cd0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.886+0000 7f113e546700 1 --2- 172.21.15.3:0/808715806 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1138130540 0x7f1138119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.886+0000 7f113e546700 1 -- 172.21.15.3:0/808715806 >> 172.21.15.3:0/808715806 conn(0x7f11380a2aa0 msgr2=0x7f113812d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.886+0000 7f113e546700 1 -- 172.21.15.3:0/808715806 shutdown_connections 2023-10-03T10:16:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.886+0000 7f113e546700 1 -- 172.21.15.3:0/808715806 wait complete. 2023-10-03T10:16:49.900 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:49.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.927+0000 7f3279c13700 1 -- 172.21.15.3:0/2161565673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f32740747f0 con 0x7f327409d630 2023-10-03T10:16:49.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.927+0000 7f325bfff700 1 -- 172.21.15.3:0/2161565673 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3264011e80 con 0x7f327409d630 2023-10-03T10:16:49.928 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:49.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.930+0000 7f3279c13700 1 -- 172.21.15.3:0/2161565673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f327409d630 msgr2=0x7f327412c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:49.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.930+0000 7f3279c13700 1 --2- 172.21.15.3:0/2161565673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f327409d630 0x7f327412c570 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f326400c470 tx=0x7f326400c780 comp rx=0 tx=0).stop 2023-10-03T10:16:49.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.930+0000 7f3279c13700 1 -- 172.21.15.3:0/2161565673 shutdown_connections 2023-10-03T10:16:49.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.930+0000 7f3279c13700 1 --2- 172.21.15.3:0/2161565673 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3274138d80 0x7f327412cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.930+0000 7f3279c13700 1 --2- 172.21.15.3:0/2161565673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f327409d630 0x7f327412c570 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.930+0000 7f3279c13700 1 --2- 172.21.15.3:0/2161565673 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f32740a0a30 0x7f327412cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:49.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.930+0000 7f3279c13700 1 -- 172.21.15.3:0/2161565673 >> 172.21.15.3:0/2161565673 conn(0x7f32740a3050 msgr2=0x7f327409ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:49.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.930+0000 7f3279c13700 1 -- 172.21.15.3:0/2161565673 shutdown_connections 2023-10-03T10:16:49.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:49.930+0000 7f3279c13700 1 -- 172.21.15.3:0/2161565673 wait complete. 2023-10-03T10:16:49.944 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:50.650 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:50.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.796+0000 7fee1d9ca700 1 Processor -- start 2023-10-03T10:16:50.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.796+0000 7fee1d9ca700 1 -- start start 2023-10-03T10:16:50.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.796+0000 7fee1d9ca700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee1809f330 0x7fee18138f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:50.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.797+0000 7fee1d9ca700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fee18139450 0x7fee1813b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:50.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.797+0000 7fee1d9ca700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee1806bec0 con 0x7fee1809e850 2023-10-03T10:16:50.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.797+0000 7fee1d9ca700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee180a0030 con 0x7fee18139450 2023-10-03T10:16:50.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.797+0000 7fee1d9ca700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee180a0830 con 0x7fee1809f330 2023-10-03T10:16:50.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.797+0000 7fee16ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fee1809e850 0x7fee1809ec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53248/0 (socket says 172.21.15.3:53248) 2023-10-03T10:16:50.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.797+0000 7fee167fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee1809f330 0x7fee18138f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:50.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.797+0000 7fee16ffd700 1 -- 172.21.15.3:0/3058933097 learned_addr learned my addr 172.21.15.3:0/3058933097 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:50.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.797+0000 7fee177fe700 1 --2- 172.21.15.3:0/3058933097 >> v2:172.21.15.3:3300/0 conn(0x7fee18139450 0x7fee1813b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:50.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.797+0000 7fee167fc700 1 -- 172.21.15.3:0/3058933097 >> v2:172.21.15.3:3300/0 conn(0x7fee18139450 msgr2=0x7fee1813b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:50.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.797+0000 7fee167fc700 1 --2- 172.21.15.3:0/3058933097 >> v2:172.21.15.3:3300/0 conn(0x7fee18139450 0x7fee1813b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:50.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.797+0000 7fee167fc700 1 -- 172.21.15.3:0/3058933097 >> v1:172.21.15.3:6789/0 conn(0x7fee1809e850 legacy=0x7fee1809ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:50.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.797+0000 7fee167fc700 1 -- 172.21.15.3:0/3058933097 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1806caa0 con 0x7fee1809f330 2023-10-03T10:16:50.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.797+0000 7fee177fe700 1 --2- 172.21.15.3:0/3058933097 >> v2:172.21.15.3:3300/0 conn(0x7fee18139450 0x7fee1813b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:50.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.797+0000 7fee167fc700 1 --2- 172.21.15.3:0/3058933097 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee1809f330 0x7fee18138f10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fee00009840 tx=0x7fee00009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cdc34af669f6543 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:50.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.797+0000 7fee15ffb700 1 -- 172.21.15.3:0/3058933097 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee00011830 con 0x7fee1809f330 2023-10-03T10:16:50.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.797+0000 7fee15ffb700 1 -- 172.21.15.3:0/3058933097 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee00011990 con 0x7fee1809f330 2023-10-03T10:16:50.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.797+0000 7fee15ffb700 1 -- 172.21.15.3:0/3058933097 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee00011d20 con 0x7fee1809f330 2023-10-03T10:16:50.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.798+0000 7fee1d9ca700 1 -- 172.21.15.3:0/3058933097 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee1809f330 msgr2=0x7fee18138f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:50.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.798+0000 7fee1d9ca700 1 --2- 172.21.15.3:0/3058933097 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee1809f330 0x7fee18138f10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fee00009840 tx=0x7fee00009b50 comp rx=0 tx=0).stop 2023-10-03T10:16:50.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.798+0000 7fee1d9ca700 1 -- 172.21.15.3:0/3058933097 shutdown_connections 2023-10-03T10:16:50.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.798+0000 7fee1d9ca700 1 --2- 172.21.15.3:0/3058933097 >> v2:172.21.15.3:3300/0 conn(0x7fee18139450 0x7fee1813b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:50.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.798+0000 7fee1d9ca700 1 --2- 172.21.15.3:0/3058933097 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee1809f330 0x7fee18138f10 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:50.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.798+0000 7fee1d9ca700 1 -- 172.21.15.3:0/3058933097 >> 172.21.15.3:0/3058933097 conn(0x7fee180a3f70 msgr2=0x7fee180a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:50.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.798+0000 7fee1d9ca700 1 -- 172.21.15.3:0/3058933097 shutdown_connections 2023-10-03T10:16:50.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.798+0000 7fee1d9ca700 1 -- 172.21.15.3:0/3058933097 wait complete. 2023-10-03T10:16:50.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.799+0000 7fee1d9ca700 1 Processor -- start 2023-10-03T10:16:50.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.799+0000 7fee1d9ca700 1 -- start start 2023-10-03T10:16:50.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.800+0000 7fee1d9ca700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee1809e850 0x7fee1811eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:50.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.800+0000 7fee1d9ca700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee1809f330 0x7fee1811efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:50.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.800+0000 7fee1d9ca700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee18139450 0x7fee1811f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:50.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.800+0000 7fee1d9ca700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee181206b0 con 0x7fee1809e850 2023-10-03T10:16:50.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.800+0000 7fee1d9ca700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee1806ccc0 con 0x7fee18139450 2023-10-03T10:16:50.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.800+0000 7fee1d9ca700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee18122800 con 0x7fee1809f330 2023-10-03T10:16:50.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.800+0000 7fee16ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee1809e850 0x7fee1811eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:50.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.800+0000 7fee167fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee1809f330 0x7fee1811efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:50.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.800+0000 7fee177fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee18139450 0x7fee1811f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:50.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.800+0000 7fee16ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee1809e850 0x7fee1811eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48632/0 (socket says 172.21.15.3:48632) 2023-10-03T10:16:50.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.800+0000 7fee16ffd700 1 -- 172.21.15.3:0/2142449940 learned_addr learned my addr 172.21.15.3:0/2142449940 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:50.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.800+0000 7fee167fc700 1 -- 172.21.15.3:0/2142449940 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee18139450 msgr2=0x7fee1811f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:50.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.800+0000 7fee167fc700 1 --2- 172.21.15.3:0/2142449940 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee18139450 0x7fee1811f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:50.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.800+0000 7fee167fc700 1 -- 172.21.15.3:0/2142449940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee1809e850 msgr2=0x7fee1811eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:50.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.800+0000 7fee167fc700 1 --2- 172.21.15.3:0/2142449940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee1809e850 0x7fee1811eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:50.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.800+0000 7fee167fc700 1 -- 172.21.15.3:0/2142449940 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee10009a70 con 0x7fee1809f330 2023-10-03T10:16:50.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.800+0000 7fee16ffd700 1 --2- 172.21.15.3:0/2142449940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee1809e850 0x7fee1811eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:50.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.800+0000 7fee177fe700 1 --2- 172.21.15.3:0/2142449940 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee18139450 0x7fee1811f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:50.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.800+0000 7fee167fc700 1 --2- 172.21.15.3:0/2142449940 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee1809f330 0x7fee1811efe0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fee1809e610 tx=0x7fee00004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:50.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.801+0000 7fee0ffff700 1 -- 172.21.15.3:0/2142449940 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee000045d0 con 0x7fee1809f330 2023-10-03T10:16:50.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.801+0000 7fee0ffff700 1 -- 172.21.15.3:0/2142449940 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0001f070 con 0x7fee1809f330 2023-10-03T10:16:50.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.801+0000 7fee1d9ca700 1 -- 172.21.15.3:0/2142449940 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee000094a0 con 0x7fee1809f330 2023-10-03T10:16:50.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.801+0000 7fee0ffff700 1 -- 172.21.15.3:0/2142449940 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee000038e0 con 0x7fee1809f330 2023-10-03T10:16:50.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.801+0000 7fee1d9ca700 1 -- 172.21.15.3:0/2142449940 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee18121d50 con 0x7fee1809f330 2023-10-03T10:16:50.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.801+0000 7fee0ffff700 1 -- 172.21.15.3:0/2142449940 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fee00003a40 con 0x7fee1809f330 2023-10-03T10:16:50.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.802+0000 7fee0ffff700 1 -- 172.21.15.3:0/2142449940 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fee00021070 con 0x7fee1809f330 2023-10-03T10:16:50.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.802+0000 7fee1d9ca700 1 -- 172.21.15.3:0/2142449940 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee1806f7f0 con 0x7fee1809f330 2023-10-03T10:16:50.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:50.807+0000 7fee0ffff700 1 -- 172.21.15.3:0/2142449940 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fee00015070 con 0x7fee1809f330 2023-10-03T10:16:50.865 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:50.901 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:50.945 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:51.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.019+0000 7fc37347a700 1 Processor -- start 2023-10-03T10:16:51.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.019+0000 7fc37347a700 1 -- start start 2023-10-03T10:16:51.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.020+0000 7fc37347a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc36c136710 0x7fc36c138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:51.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.020+0000 7fc37347a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc36c139030 0x7fc36c13b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:51.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.020+0000 7fc37347a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc36c06ef30 con 0x7fc36c1338a0 2023-10-03T10:16:51.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.020+0000 7fc37347a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc36c06fef0 con 0x7fc36c136710 2023-10-03T10:16:51.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.020+0000 7fc37347a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c1334b0 con 0x7fc36c139030 2023-10-03T10:16:51.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.020+0000 7fc370a15700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc36c136710 0x7fc36c138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:51.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.020+0000 7fc370a15700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc36c136710 0x7fc36c138af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48640/0 (socket says 172.21.15.3:48640) 2023-10-03T10:16:51.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.020+0000 7fc370a15700 1 -- 172.21.15.3:0/304420002 learned_addr learned my addr 172.21.15.3:0/304420002 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:51.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.020+0000 7fc371a17700 1 --2- 172.21.15.3:0/304420002 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc36c139030 0x7fc36c13b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:51.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.020+0000 7fc370a15700 1 -- 172.21.15.3:0/304420002 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc36c139030 msgr2=0x7fc36c13b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:51.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.020+0000 7fc370a15700 1 --2- 172.21.15.3:0/304420002 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc36c139030 0x7fc36c13b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.020+0000 7fc370a15700 1 -- 172.21.15.3:0/304420002 >> v1:172.21.15.3:6789/0 conn(0x7fc36c1338a0 legacy=0x7fc36c133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:51.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.020+0000 7fc370a15700 1 -- 172.21.15.3:0/304420002 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc36c06fc30 con 0x7fc36c136710 2023-10-03T10:16:51.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.021+0000 7fc370a15700 1 --2- 172.21.15.3:0/304420002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc36c136710 0x7fc36c138af0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0098f0 tx=0x7fc35c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e49dbdc4cdc266e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:51.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.021+0000 7fc363fff700 1 -- 172.21.15.3:0/304420002 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c00b8a0 con 0x7fc36c136710 2023-10-03T10:16:51.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.021+0000 7fc363fff700 1 -- 172.21.15.3:0/304420002 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35c00ba00 con 0x7fc36c136710 2023-10-03T10:16:51.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.021+0000 7fc363fff700 1 -- 172.21.15.3:0/304420002 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c00bd90 con 0x7fc36c136710 2023-10-03T10:16:51.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.022+0000 7fc37347a700 1 -- 172.21.15.3:0/304420002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc36c136710 msgr2=0x7fc36c138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:51.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.022+0000 7fc37347a700 1 --2- 172.21.15.3:0/304420002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc36c136710 0x7fc36c138af0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0098f0 tx=0x7fc35c009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:51.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.022+0000 7fc37347a700 1 -- 172.21.15.3:0/304420002 shutdown_connections 2023-10-03T10:16:51.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.022+0000 7fc37347a700 1 --2- 172.21.15.3:0/304420002 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc36c139030 0x7fc36c13b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.022+0000 7fc37347a700 1 --2- 172.21.15.3:0/304420002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc36c136710 0x7fc36c138af0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.022+0000 7fc37347a700 1 -- 172.21.15.3:0/304420002 >> 172.21.15.3:0/304420002 conn(0x7fc36c0a2aa0 msgr2=0x7fc36c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:51.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.022+0000 7fc37347a700 1 -- 172.21.15.3:0/304420002 shutdown_connections 2023-10-03T10:16:51.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.022+0000 7fc37347a700 1 -- 172.21.15.3:0/304420002 wait complete. 2023-10-03T10:16:51.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.023+0000 7fee1d9ca700 1 -- 172.21.15.3:0/2142449940 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fee180747f0 con 0x7fee1809f330 2023-10-03T10:16:51.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.024+0000 7fc37347a700 1 Processor -- start 2023-10-03T10:16:51.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.024+0000 7fc37347a700 1 -- start start 2023-10-03T10:16:51.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.024+0000 7fc37347a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc36c1338a0 0x7fc36c11ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:51.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.024+0000 7fc37347a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc36c136710 0x7fc36c11b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:51.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.025+0000 7fc37347a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc36c139030 0x7fc36c11b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:51.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.025+0000 7fc37347a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c138b00 con 0x7fc36c1338a0 2023-10-03T10:16:51.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.025+0000 7fc37347a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c11cf60 con 0x7fc36c139030 2023-10-03T10:16:51.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.025+0000 7fc37347a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c11ca50 con 0x7fc36c136710 2023-10-03T10:16:51.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.025+0000 7fc370a15700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc36c136710 0x7fc36c11b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:51.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.025+0000 7fc371216700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc36c1338a0 0x7fc36c11ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:51.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.025+0000 7fc370a15700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc36c136710 0x7fc36c11b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51824/0 (socket says 172.21.15.3:51824) 2023-10-03T10:16:51.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.025+0000 7fc370a15700 1 -- 172.21.15.3:0/3034421363 learned_addr learned my addr 172.21.15.3:0/3034421363 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:51.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.025+0000 7fc371216700 1 -- 172.21.15.3:0/3034421363 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc36c136710 msgr2=0x7fc36c11b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:51.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.025+0000 7fc371216700 1 --2- 172.21.15.3:0/3034421363 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc36c136710 0x7fc36c11b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.025+0000 7fc371216700 1 -- 172.21.15.3:0/3034421363 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc36c139030 msgr2=0x7fc36c11b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:51.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.025+0000 7fc371216700 1 --2- 172.21.15.3:0/3034421363 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc36c139030 0x7fc36c11b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.025+0000 7fc371216700 1 -- 172.21.15.3:0/3034421363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35c009550 con 0x7fc36c1338a0 2023-10-03T10:16:51.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.025+0000 7fc370a15700 1 --2- 172.21.15.3:0/3034421363 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc36c136710 0x7fc36c11b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:51.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.025+0000 7fc371216700 1 --2- 172.21.15.3:0/3034421363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc36c1338a0 0x7fc36c11ae40 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fc368000f60 tx=0x7fc36800c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:51.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.025+0000 7fc361ffb700 1 -- 172.21.15.3:0/3034421363 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3680052d0 con 0x7fc36c1338a0 2023-10-03T10:16:51.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.025+0000 7fc361ffb700 1 -- 172.21.15.3:0/3034421363 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc368005430 con 0x7fc36c1338a0 2023-10-03T10:16:51.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.025+0000 7fc37347a700 1 -- 172.21.15.3:0/3034421363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc36c130c90 con 0x7fc36c1338a0 2023-10-03T10:16:51.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.026+0000 7fc361ffb700 1 -- 172.21.15.3:0/3034421363 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3680038c0 con 0x7fc36c1338a0 2023-10-03T10:16:51.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.026+0000 7fee0ffff700 1 -- 172.21.15.3:0/2142449940 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fee00010380 con 0x7fee1809f330 2023-10-03T10:16:51.028 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:51.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.026+0000 7fc37347a700 1 -- 172.21.15.3:0/3034421363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36c11e5e0 con 0x7fc36c1338a0 2023-10-03T10:16:51.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.027+0000 7fc37347a700 1 -- 172.21.15.3:0/3034421363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc36c11e200 con 0x7fc36c1338a0 2023-10-03T10:16:51.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.027+0000 7fc361ffb700 1 -- 172.21.15.3:0/3034421363 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc368003a20 con 0x7fc36c1338a0 2023-10-03T10:16:51.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.028+0000 7fee1d9ca700 1 -- 172.21.15.3:0/2142449940 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee1809f330 msgr2=0x7fee1811efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:51.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.028+0000 7fee1d9ca700 1 --2- 172.21.15.3:0/2142449940 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee1809f330 0x7fee1811efe0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fee1809e610 tx=0x7fee00004750 comp rx=0 tx=0).stop 2023-10-03T10:16:51.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.028+0000 7fee1d9ca700 1 -- 172.21.15.3:0/2142449940 shutdown_connections 2023-10-03T10:16:51.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.028+0000 7fee1d9ca700 1 --2- 172.21.15.3:0/2142449940 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee18139450 0x7fee1811f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.028+0000 7fee1d9ca700 1 --2- 172.21.15.3:0/2142449940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee1809e850 0x7fee1811eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.028+0000 7fee1d9ca700 1 --2- 172.21.15.3:0/2142449940 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee1809f330 0x7fee1811efe0 secure :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fee1809e610 tx=0x7fee00004750 comp rx=0 tx=0).stop 2023-10-03T10:16:51.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.028+0000 7fee1d9ca700 1 -- 172.21.15.3:0/2142449940 >> 172.21.15.3:0/2142449940 conn(0x7fee180a3f70 msgr2=0x7fee18136fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:51.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.029+0000 7fc361ffb700 1 -- 172.21.15.3:0/3034421363 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc36801ade0 con 0x7fc36c1338a0 2023-10-03T10:16:51.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.030+0000 7fee1d9ca700 1 -- 172.21.15.3:0/2142449940 shutdown_connections 2023-10-03T10:16:51.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.030+0000 7fee1d9ca700 1 -- 172.21.15.3:0/2142449940 wait complete. 2023-10-03T10:16:51.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.033+0000 7fc361ffb700 1 -- 172.21.15.3:0/3034421363 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc368044060 con 0x7fc36c1338a0 2023-10-03T10:16:51.052 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:51.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.061+0000 7f6732b28700 1 Processor -- start 2023-10-03T10:16:51.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.061+0000 7f6732b28700 1 -- start start 2023-10-03T10:16:51.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.062+0000 7f6732b28700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f672c06a770 0x7f672c06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:51.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.062+0000 7f6732b28700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f672c06b080 0x7f672c131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:51.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.062+0000 7f6732b28700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f672c131b40 0x7f672c131fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:51.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.062+0000 7f6732b28700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f672c09ece0 con 0x7f672c06b080 2023-10-03T10:16:51.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.062+0000 7f6732b28700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f672c09e760 con 0x7f672c06a770 2023-10-03T10:16:51.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.062+0000 7f6732b28700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f672c09e040 con 0x7f672c131b40 2023-10-03T10:16:51.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.062+0000 7f67308c4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f672c06a770 0x7f672c06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:51.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.062+0000 7f672bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f672c06b080 0x7f672c131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:51.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.062+0000 7f67308c4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f672c06a770 0x7f672c06ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53284/0 (socket says 172.21.15.3:53284) 2023-10-03T10:16:51.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.062+0000 7f672bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f672c06b080 0x7f672c131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48662/0 (socket says 172.21.15.3:48662) 2023-10-03T10:16:51.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.062+0000 7f67308c4700 1 -- 172.21.15.3:0/176274276 learned_addr learned my addr 172.21.15.3:0/176274276 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:51.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.062+0000 7f67310c5700 1 --2- 172.21.15.3:0/176274276 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f672c131b40 0x7f672c131fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:51.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.062+0000 7f672bfff700 1 -- 172.21.15.3:0/176274276 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f672c131b40 msgr2=0x7f672c131fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:51.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.062+0000 7f672bfff700 1 --2- 172.21.15.3:0/176274276 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f672c131b40 0x7f672c131fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.063+0000 7f672bfff700 1 -- 172.21.15.3:0/176274276 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f672c06a770 msgr2=0x7f672c06ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:51.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.063+0000 7f672bfff700 1 --2- 172.21.15.3:0/176274276 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f672c06a770 0x7f672c06ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.063+0000 7f672bfff700 1 -- 172.21.15.3:0/176274276 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672c09f630 con 0x7f672c06b080 2023-10-03T10:16:51.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.063+0000 7f67308c4700 1 --2- 172.21.15.3:0/176274276 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f672c06a770 0x7f672c06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:51.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.063+0000 7f672bfff700 1 --2- 172.21.15.3:0/176274276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f672c06b080 0x7f672c131600 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f6714009840 tx=0x7f6714009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9053f116acd9ea8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:51.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.063+0000 7f672b7fe700 1 -- 172.21.15.3:0/176274276 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6714011830 con 0x7f672c06b080 2023-10-03T10:16:51.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.063+0000 7f672b7fe700 1 -- 172.21.15.3:0/176274276 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6714011990 con 0x7f672c06b080 2023-10-03T10:16:51.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.063+0000 7f672b7fe700 1 -- 172.21.15.3:0/176274276 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6714011d20 con 0x7f672c06b080 2023-10-03T10:16:51.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.064+0000 7f6732b28700 1 -- 172.21.15.3:0/176274276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f672c06b080 msgr2=0x7f672c131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:51.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.064+0000 7f6732b28700 1 --2- 172.21.15.3:0/176274276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f672c06b080 0x7f672c131600 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f6714009840 tx=0x7f6714009b50 comp rx=0 tx=0).stop 2023-10-03T10:16:51.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.064+0000 7f6732b28700 1 -- 172.21.15.3:0/176274276 shutdown_connections 2023-10-03T10:16:51.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.064+0000 7f6732b28700 1 --2- 172.21.15.3:0/176274276 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f672c131b40 0x7f672c131fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.064+0000 7f6732b28700 1 --2- 172.21.15.3:0/176274276 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f672c06a770 0x7f672c06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.064+0000 7f6732b28700 1 --2- 172.21.15.3:0/176274276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f672c06b080 0x7f672c131600 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.064+0000 7f6732b28700 1 -- 172.21.15.3:0/176274276 >> 172.21.15.3:0/176274276 conn(0x7f672c066f50 msgr2=0x7f672c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:51.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.064+0000 7f6732b28700 1 -- 172.21.15.3:0/176274276 shutdown_connections 2023-10-03T10:16:51.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.064+0000 7f6732b28700 1 -- 172.21.15.3:0/176274276 wait complete. 2023-10-03T10:16:51.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.065+0000 7f6732b28700 1 Processor -- start 2023-10-03T10:16:51.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.065+0000 7f6732b28700 1 -- start start 2023-10-03T10:16:51.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.066+0000 7f6732b28700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f672c06a770 0x7f672c09a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:51.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.066+0000 7f6732b28700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f672c06b080 0x7f672c09a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.066+0000 7f6732b28700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f672c131b40 0x7f672c09ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.066+0000 7f6732b28700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c1255e0 con 0x7f672c06b080 2023-10-03T10:16:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.066+0000 7f6732b28700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f672c124830 con 0x7f672c131b40 2023-10-03T10:16:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.066+0000 7f6732b28700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f672c09b860 con 0x7f672c06a770 2023-10-03T10:16:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.066+0000 7f672bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f672c06b080 0x7f672c09a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.066+0000 7f67310c5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f672c131b40 0x7f672c09ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.066+0000 7f672bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f672c06b080 0x7f672c09a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48678/0 (socket says 172.21.15.3:48678) 2023-10-03T10:16:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.066+0000 7f672bfff700 1 -- 172.21.15.3:0/1145041941 learned_addr learned my addr 172.21.15.3:0/1145041941 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.066+0000 7f67308c4700 1 --2- 172.21.15.3:0/1145041941 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f672c06a770 0x7f672c09a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.066+0000 7f67310c5700 1 -- 172.21.15.3:0/1145041941 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f672c06a770 msgr2=0x7f672c09a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.066+0000 7f67310c5700 1 --2- 172.21.15.3:0/1145041941 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f672c06a770 0x7f672c09a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.066+0000 7f67310c5700 1 -- 172.21.15.3:0/1145041941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f672c06b080 msgr2=0x7f672c09a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:51.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.066+0000 7f67310c5700 1 --2- 172.21.15.3:0/1145041941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f672c06b080 0x7f672c09a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.066+0000 7f67310c5700 1 -- 172.21.15.3:0/1145041941 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671c009550 con 0x7f672c131b40 2023-10-03T10:16:51.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.067+0000 7f67310c5700 1 --2- 172.21.15.3:0/1145041941 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f672c131b40 0x7f672c09ae80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6724009d90 tx=0x7f672400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:51.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.067+0000 7f67297fa700 1 -- 172.21.15.3:0/1145041941 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6724009c00 con 0x7f672c131b40 2023-10-03T10:16:51.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.067+0000 7f67297fa700 1 -- 172.21.15.3:0/1145041941 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672400f040 con 0x7f672c131b40 2023-10-03T10:16:51.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.067+0000 7f67297fa700 1 -- 172.21.15.3:0/1145041941 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6724014480 con 0x7f672c131b40 2023-10-03T10:16:51.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.067+0000 7f6732b28700 1 -- 172.21.15.3:0/1145041941 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67140094a0 con 0x7f672c131b40 2023-10-03T10:16:51.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.067+0000 7f6732b28700 1 -- 172.21.15.3:0/1145041941 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f672c09c010 con 0x7f672c131b40 2023-10-03T10:16:51.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.068+0000 7f67297fa700 1 -- 172.21.15.3:0/1145041941 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f67240145e0 con 0x7f672c131b40 2023-10-03T10:16:51.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.068+0000 7f6732b28700 1 -- 172.21.15.3:0/1145041941 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f672c071e60 con 0x7f672c131b40 2023-10-03T10:16:51.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.068+0000 7f67297fa700 1 -- 172.21.15.3:0/1145041941 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6724040290 con 0x7f672c131b40 2023-10-03T10:16:51.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.073+0000 7f67297fa700 1 -- 172.21.15.3:0/1145041941 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6724007510 con 0x7f672c131b40 2023-10-03T10:16:51.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.099+0000 7f9b4c3cf700 1 Processor -- start 2023-10-03T10:16:51.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.099+0000 7f9b4c3cf700 1 -- start start 2023-10-03T10:16:51.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.100+0000 7f9b4c3cf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9b4409d6f0 0x7f9b4409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:51.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.100+0000 7f9b4c3cf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b4409e410 0x7f9b44130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:51.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.100+0000 7f9b4c3cf700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b4409f130 con 0x7f9b4409e000 2023-10-03T10:16:51.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.100+0000 7f9b4c3cf700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b4409f520 con 0x7f9b4409d6f0 2023-10-03T10:16:51.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.100+0000 7f9b4c3cf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4409e930 con 0x7f9b4409e410 2023-10-03T10:16:51.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.100+0000 7f9b4a96c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b4409e410 0x7f9b44130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:51.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.100+0000 7f9b4a16b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9b4409d6f0 0x7f9b4409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:51.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.100+0000 7f9b4996a700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9b4409e000 0x7f9b44130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53260/0 (socket says 172.21.15.3:53260) 2023-10-03T10:16:51.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.100+0000 7f9b4996a700 1 -- 172.21.15.3:0/456051802 learned_addr learned my addr 172.21.15.3:0/456051802 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:51.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.101+0000 7f9b49169700 1 -- 172.21.15.3:0/456051802 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3892963425 0 0) 0x7f9b4409f130 con 0x7f9b4409e000 2023-10-03T10:16:51.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.101+0000 7f9b49169700 1 -- 172.21.15.3:0/456051802 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b300035b0 con 0x7f9b4409e000 2023-10-03T10:16:51.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.101+0000 7f9b4a96c700 1 -- 172.21.15.3:0/456051802 >> v2:172.21.15.3:3300/0 conn(0x7f9b4409d6f0 msgr2=0x7f9b4409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:51.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.101+0000 7f9b4a96c700 1 --2- 172.21.15.3:0/456051802 >> v2:172.21.15.3:3300/0 conn(0x7f9b4409d6f0 0x7f9b4409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.101+0000 7f9b4a96c700 1 -- 172.21.15.3:0/456051802 >> v1:172.21.15.3:6789/0 conn(0x7f9b4409e000 legacy=0x7f9b44130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:51.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.101+0000 7f9b4a96c700 1 -- 172.21.15.3:0/456051802 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b4409b2d0 con 0x7f9b4409e410 2023-10-03T10:16:51.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.101+0000 7f9b4a96c700 1 --2- 172.21.15.3:0/456051802 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b4409e410 0x7f9b44130d70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c009d90 tx=0x7f9b3c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5216587d2a0b52 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:51.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.101+0000 7f9b49169700 1 -- 172.21.15.3:0/456051802 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c00e070 con 0x7f9b4409e410 2023-10-03T10:16:51.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.101+0000 7f9b49169700 1 -- 172.21.15.3:0/456051802 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b3c00bb20 con 0x7f9b4409e410 2023-10-03T10:16:51.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.101+0000 7f9b49169700 1 -- 172.21.15.3:0/456051802 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c012430 con 0x7f9b4409e410 2023-10-03T10:16:51.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.102+0000 7f9b4c3cf700 1 -- 172.21.15.3:0/456051802 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b4409e410 msgr2=0x7f9b44130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:51.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.102+0000 7f9b4c3cf700 1 --2- 172.21.15.3:0/456051802 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b4409e410 0x7f9b44130d70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c009d90 tx=0x7f9b3c00b330 comp rx=0 tx=0).stop 2023-10-03T10:16:51.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.102+0000 7f9b4c3cf700 1 -- 172.21.15.3:0/456051802 shutdown_connections 2023-10-03T10:16:51.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.102+0000 7f9b4c3cf700 1 --2- 172.21.15.3:0/456051802 >> v2:172.21.15.3:3300/0 conn(0x7f9b4409d6f0 0x7f9b4409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.102+0000 7f9b4c3cf700 1 --2- 172.21.15.3:0/456051802 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b4409e410 0x7f9b44130d70 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.102+0000 7f9b4c3cf700 1 -- 172.21.15.3:0/456051802 >> 172.21.15.3:0/456051802 conn(0x7f9b44066f50 msgr2=0x7f9b4409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.102+0000 7f9b4c3cf700 1 -- 172.21.15.3:0/456051802 shutdown_connections 2023-10-03T10:16:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.102+0000 7f9b4c3cf700 1 -- 172.21.15.3:0/456051802 wait complete. 2023-10-03T10:16:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.103+0000 7f9b4c3cf700 1 Processor -- start 2023-10-03T10:16:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.103+0000 7f9b4c3cf700 1 -- start start 2023-10-03T10:16:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.104+0000 7f9b4c3cf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b4409d6f0 0x7f9b44130c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.104+0000 7f9b4c3cf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9b4409e000 0x7f9b44136840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.104+0000 7f9b4c3cf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b4409e410 0x7f9b44136d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.104+0000 7f9b4c3cf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b440a38d0 con 0x7f9b4409e410 2023-10-03T10:16:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.104+0000 7f9b4c3cf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b44138a00 con 0x7f9b4409e000 2023-10-03T10:16:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.104+0000 7f9b4c3cf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4413b050 con 0x7f9b4409d6f0 2023-10-03T10:16:51.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.104+0000 7f9b4996a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9b4409e000 0x7f9b44136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:51.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.104+0000 7f9b4a16b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b4409d6f0 0x7f9b44130c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:51.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.104+0000 7f9b4996a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9b4409e000 0x7f9b44136840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53300/0 (socket says 172.21.15.3:53300) 2023-10-03T10:16:51.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.104+0000 7f9b4996a700 1 -- 172.21.15.3:0/2597082951 learned_addr learned my addr 172.21.15.3:0/2597082951 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:51.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.104+0000 7f9b4a96c700 1 --2- 172.21.15.3:0/2597082951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b4409e410 0x7f9b44136d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:51.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.104+0000 7f9b4a16b700 1 -- 172.21.15.3:0/2597082951 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9b4409e000 msgr2=0x7f9b44136840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:51.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.104+0000 7f9b4a16b700 1 --2- 172.21.15.3:0/2597082951 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9b4409e000 0x7f9b44136840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.104+0000 7f9b4a16b700 1 -- 172.21.15.3:0/2597082951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b4409e410 msgr2=0x7f9b44136d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:51.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.104+0000 7f9b4a16b700 1 --2- 172.21.15.3:0/2597082951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b4409e410 0x7f9b44136d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.104+0000 7f9b4a16b700 1 -- 172.21.15.3:0/2597082951 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3c009a70 con 0x7f9b4409d6f0 2023-10-03T10:16:51.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.104+0000 7f9b4a16b700 1 --2- 172.21.15.3:0/2597082951 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b4409d6f0 0x7f9b44130c50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9b4000e900 tx=0x7f9b4000ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:51.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.105+0000 7f9b3affd700 1 -- 172.21.15.3:0/2597082951 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b40013680 con 0x7f9b4409d6f0 2023-10-03T10:16:51.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.105+0000 7f9b3affd700 1 -- 172.21.15.3:0/2597082951 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b400137e0 con 0x7f9b4409d6f0 2023-10-03T10:16:51.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.105+0000 7f9b4c3cf700 1 -- 172.21.15.3:0/2597082951 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b4413a080 con 0x7f9b4409d6f0 2023-10-03T10:16:51.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.105+0000 7f9b3affd700 1 -- 172.21.15.3:0/2597082951 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b40013b50 con 0x7f9b4409d6f0 2023-10-03T10:16:51.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.105+0000 7f9b4c3cf700 1 -- 172.21.15.3:0/2597082951 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b4413aaf0 con 0x7f9b4409d6f0 2023-10-03T10:16:51.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.106+0000 7f9b3affd700 1 -- 172.21.15.3:0/2597082951 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9b40013cb0 con 0x7f9b4409d6f0 2023-10-03T10:16:51.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.106+0000 7f9b4c3cf700 1 -- 172.21.15.3:0/2597082951 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b44139ec0 con 0x7f9b4409d6f0 2023-10-03T10:16:51.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.106+0000 7f9b3affd700 1 -- 172.21.15.3:0/2597082951 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9b4003f860 con 0x7f9b4409d6f0 2023-10-03T10:16:51.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.111+0000 7f9b3affd700 1 -- 172.21.15.3:0/2597082951 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9b4001be10 con 0x7f9b4409d6f0 2023-10-03T10:16:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.243+0000 7fc37347a700 1 -- 172.21.15.3:0/3034421363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc36c06ea40 con 0x7fc36c1338a0 2023-10-03T10:16:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.243+0000 7fc361ffb700 1 -- 172.21.15.3:0/3034421363 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc368017070 con 0x7fc36c1338a0 2023-10-03T10:16:51.244 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.245+0000 7fc37347a700 1 -- 172.21.15.3:0/3034421363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc36c1338a0 msgr2=0x7fc36c11ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.245+0000 7fc37347a700 1 --2- 172.21.15.3:0/3034421363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc36c1338a0 0x7fc36c11ae40 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fc368000f60 tx=0x7fc36800c730 comp rx=0 tx=0).stop 2023-10-03T10:16:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.246+0000 7fc37347a700 1 -- 172.21.15.3:0/3034421363 shutdown_connections 2023-10-03T10:16:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.246+0000 7fc37347a700 1 --2- 172.21.15.3:0/3034421363 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc36c139030 0x7fc36c11b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.246+0000 7fc37347a700 1 --2- 172.21.15.3:0/3034421363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc36c1338a0 0x7fc36c11ae40 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.246+0000 7fc37347a700 1 --2- 172.21.15.3:0/3034421363 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc36c136710 0x7fc36c11b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.246+0000 7fc37347a700 1 -- 172.21.15.3:0/3034421363 >> 172.21.15.3:0/3034421363 conn(0x7fc36c0a2aa0 msgr2=0x7fc36c1344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:51.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.246+0000 7fc37347a700 1 -- 172.21.15.3:0/3034421363 shutdown_connections 2023-10-03T10:16:51.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.246+0000 7fc37347a700 1 -- 172.21.15.3:0/3034421363 wait complete. 2023-10-03T10:16:51.261 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:51.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.281+0000 7f6732b28700 1 -- 172.21.15.3:0/1145041941 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f672c074ea0 con 0x7f672c131b40 2023-10-03T10:16:51.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.281+0000 7f67297fa700 1 -- 172.21.15.3:0/1145041941 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6724007330 con 0x7f672c131b40 2023-10-03T10:16:51.282 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:51.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.284+0000 7f6732b28700 1 -- 172.21.15.3:0/1145041941 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f672c131b40 msgr2=0x7f672c09ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:51.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.284+0000 7f6732b28700 1 --2- 172.21.15.3:0/1145041941 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f672c131b40 0x7f672c09ae80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6724009d90 tx=0x7f672400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.284+0000 7f6732b28700 1 -- 172.21.15.3:0/1145041941 shutdown_connections 2023-10-03T10:16:51.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.284+0000 7f6732b28700 1 --2- 172.21.15.3:0/1145041941 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f672c131b40 0x7f672c09ae80 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.284+0000 7f6732b28700 1 --2- 172.21.15.3:0/1145041941 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f672c06a770 0x7f672c09a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.284+0000 7f6732b28700 1 --2- 172.21.15.3:0/1145041941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f672c06b080 0x7f672c09a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.284+0000 7f6732b28700 1 -- 172.21.15.3:0/1145041941 >> 172.21.15.3:0/1145041941 conn(0x7f672c066f50 msgr2=0x7f672c06fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:51.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.284+0000 7f6732b28700 1 -- 172.21.15.3:0/1145041941 shutdown_connections 2023-10-03T10:16:51.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.284+0000 7f6732b28700 1 -- 172.21.15.3:0/1145041941 wait complete. 2023-10-03T10:16:51.298 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:51.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.320+0000 7f9b4c3cf700 1 -- 172.21.15.3:0/2597082951 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9b44074ea0 con 0x7f9b4409d6f0 2023-10-03T10:16:51.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.320+0000 7f9b3affd700 1 -- 172.21.15.3:0/2597082951 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9b4001b3b0 con 0x7f9b4409d6f0 2023-10-03T10:16:51.320 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:51.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.322+0000 7f9b4c3cf700 1 -- 172.21.15.3:0/2597082951 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b4409d6f0 msgr2=0x7f9b44130c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:51.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.322+0000 7f9b4c3cf700 1 --2- 172.21.15.3:0/2597082951 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b4409d6f0 0x7f9b44130c50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9b4000e900 tx=0x7f9b4000ec10 comp rx=0 tx=0).stop 2023-10-03T10:16:51.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.322+0000 7f9b4c3cf700 1 -- 172.21.15.3:0/2597082951 shutdown_connections 2023-10-03T10:16:51.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.322+0000 7f9b4c3cf700 1 --2- 172.21.15.3:0/2597082951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b4409e410 0x7f9b44136d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.322+0000 7f9b4c3cf700 1 --2- 172.21.15.3:0/2597082951 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b4409d6f0 0x7f9b44130c50 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.322+0000 7f9b4c3cf700 1 --2- 172.21.15.3:0/2597082951 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9b4409e000 0x7f9b44136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:51.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.322+0000 7f9b4c3cf700 1 -- 172.21.15.3:0/2597082951 >> 172.21.15.3:0/2597082951 conn(0x7f9b44066f50 msgr2=0x7f9b44070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:51.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.322+0000 7f9b4c3cf700 1 -- 172.21.15.3:0/2597082951 shutdown_connections 2023-10-03T10:16:51.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:51.322+0000 7f9b4c3cf700 1 -- 172.21.15.3:0/2597082951 wait complete. 2023-10-03T10:16:51.336 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:52.053 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:52.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.199+0000 7f6632c5d700 1 Processor -- start 2023-10-03T10:16:52.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.200+0000 7f6632c5d700 1 -- start start 2023-10-03T10:16:52.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.200+0000 7f6632c5d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f662c0a09c0 0x7f662c06c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:52.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.200+0000 7f6632c5d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f662c0a0d90 0x7f662c0a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:52.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.200+0000 7f6632c5d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f662c0a2a20 con 0x7f662c1338a0 2023-10-03T10:16:52.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.200+0000 7f6632c5d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f662c09e690 con 0x7f662c0a0d90 2023-10-03T10:16:52.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.200+0000 7f6632c5d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f662c0a1cc0 con 0x7f662c0a09c0 2023-10-03T10:16:52.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.200+0000 7f66309f9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f662c0a09c0 0x7f662c06c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:52.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.200+0000 7f662bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f662c0a0d90 0x7f662c0a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:52.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.200+0000 7f66309f9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f662c0a09c0 0x7f662c06c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53318/0 (socket says 172.21.15.3:53318) 2023-10-03T10:16:52.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.200+0000 7f662bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f662c0a0d90 0x7f662c0a11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48694/0 (socket says 172.21.15.3:48694) 2023-10-03T10:16:52.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.200+0000 7f66309f9700 1 -- 172.21.15.3:0/3551582294 learned_addr learned my addr 172.21.15.3:0/3551582294 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:52.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.201+0000 7f66309f9700 1 -- 172.21.15.3:0/3551582294 >> v2:172.21.15.3:3300/0 conn(0x7f662c0a0d90 msgr2=0x7f662c0a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:52.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.201+0000 7f66309f9700 1 --2- 172.21.15.3:0/3551582294 >> v2:172.21.15.3:3300/0 conn(0x7f662c0a0d90 0x7f662c0a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.201+0000 7f66309f9700 1 -- 172.21.15.3:0/3551582294 >> v1:172.21.15.3:6789/0 conn(0x7f662c1338a0 legacy=0x7f662c133d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:52.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.201+0000 7f66309f9700 1 -- 172.21.15.3:0/3551582294 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f662c0a1f40 con 0x7f662c0a09c0 2023-10-03T10:16:52.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.201+0000 7f662bfff700 1 --2- 172.21.15.3:0/3551582294 >> v2:172.21.15.3:3300/0 conn(0x7f662c0a0d90 0x7f662c0a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:52.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.201+0000 7f66309f9700 1 --2- 172.21.15.3:0/3551582294 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f662c0a09c0 0x7f662c06c5d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f661c0098f0 tx=0x7f661c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f2ac77f2675b40b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:52.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.201+0000 7f662b7fe700 1 -- 172.21.15.3:0/3551582294 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c011830 con 0x7f662c0a09c0 2023-10-03T10:16:52.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.201+0000 7f662b7fe700 1 -- 172.21.15.3:0/3551582294 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661c011990 con 0x7f662c0a09c0 2023-10-03T10:16:52.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.201+0000 7f662b7fe700 1 -- 172.21.15.3:0/3551582294 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c011d20 con 0x7f662c0a09c0 2023-10-03T10:16:52.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.202+0000 7f6632c5d700 1 -- 172.21.15.3:0/3551582294 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f662c0a09c0 msgr2=0x7f662c06c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:52.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.202+0000 7f6632c5d700 1 --2- 172.21.15.3:0/3551582294 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f662c0a09c0 0x7f662c06c5d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f661c0098f0 tx=0x7f661c009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:52.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.202+0000 7f6632c5d700 1 -- 172.21.15.3:0/3551582294 shutdown_connections 2023-10-03T10:16:52.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.202+0000 7f6632c5d700 1 --2- 172.21.15.3:0/3551582294 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f662c0a09c0 0x7f662c06c5d0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.202+0000 7f6632c5d700 1 --2- 172.21.15.3:0/3551582294 >> v2:172.21.15.3:3300/0 conn(0x7f662c0a0d90 0x7f662c0a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.202+0000 7f6632c5d700 1 -- 172.21.15.3:0/3551582294 >> 172.21.15.3:0/3551582294 conn(0x7f662c0a60c0 msgr2=0x7f662c0a64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:52.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.202+0000 7f6632c5d700 1 -- 172.21.15.3:0/3551582294 shutdown_connections 2023-10-03T10:16:52.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.202+0000 7f6632c5d700 1 -- 172.21.15.3:0/3551582294 wait complete. 2023-10-03T10:16:52.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.203+0000 7f6632c5d700 1 Processor -- start 2023-10-03T10:16:52.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.203+0000 7f6632c5d700 1 -- start start 2023-10-03T10:16:52.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.203+0000 7f6632c5d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f662c0a09c0 0x7f662c121330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:52.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.203+0000 7f6632c5d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f662c0a0d90 0x7f662c121870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:52.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.203+0000 7f6632c5d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f662c1338a0 0x7f662c121db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:52.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.203+0000 7f6632c5d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662c132690 con 0x7f662c1338a0 2023-10-03T10:16:52.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.203+0000 7f6632c5d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f662c124dc0 con 0x7f662c0a09c0 2023-10-03T10:16:52.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.203+0000 7f6632c5d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f662c124820 con 0x7f662c0a0d90 2023-10-03T10:16:52.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.203+0000 7f66309f9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f662c0a09c0 0x7f662c121330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:52.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.203+0000 7f66309f9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f662c0a09c0 0x7f662c121330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53322/0 (socket says 172.21.15.3:53322) 2023-10-03T10:16:52.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.203+0000 7f66309f9700 1 -- 172.21.15.3:0/4055914170 learned_addr learned my addr 172.21.15.3:0/4055914170 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:52.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.203+0000 7f66311fa700 1 --2- 172.21.15.3:0/4055914170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f662c1338a0 0x7f662c121db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:52.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.203+0000 7f662bfff700 1 --2- 172.21.15.3:0/4055914170 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f662c0a0d90 0x7f662c121870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:52.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.204+0000 7f66309f9700 1 -- 172.21.15.3:0/4055914170 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f662c0a0d90 msgr2=0x7f662c121870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:52.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.204+0000 7f66309f9700 1 --2- 172.21.15.3:0/4055914170 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f662c0a0d90 0x7f662c121870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.204+0000 7f66309f9700 1 -- 172.21.15.3:0/4055914170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f662c1338a0 msgr2=0x7f662c121db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:52.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.204+0000 7f66309f9700 1 --2- 172.21.15.3:0/4055914170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f662c1338a0 0x7f662c121db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.204+0000 7f66309f9700 1 -- 172.21.15.3:0/4055914170 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66140094a0 con 0x7f662c0a09c0 2023-10-03T10:16:52.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.204+0000 7f66311fa700 1 --2- 172.21.15.3:0/4055914170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f662c1338a0 0x7f662c121db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:52.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.204+0000 7f66309f9700 1 --2- 172.21.15.3:0/4055914170 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f662c0a09c0 0x7f662c121330 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f661c000c00 tx=0x7f661c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:52.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.204+0000 7f66297fa700 1 -- 172.21.15.3:0/4055914170 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c0036a0 con 0x7f662c0a09c0 2023-10-03T10:16:52.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.204+0000 7f66297fa700 1 -- 172.21.15.3:0/4055914170 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661c003800 con 0x7f662c0a09c0 2023-10-03T10:16:52.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.204+0000 7f6632c5d700 1 -- 172.21.15.3:0/4055914170 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f661c009550 con 0x7f662c0a09c0 2023-10-03T10:16:52.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.204+0000 7f66297fa700 1 -- 172.21.15.3:0/4055914170 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c003bb0 con 0x7f662c0a09c0 2023-10-03T10:16:52.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.204+0000 7f6632c5d700 1 -- 172.21.15.3:0/4055914170 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f662c09dea0 con 0x7f662c0a09c0 2023-10-03T10:16:52.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.205+0000 7f66297fa700 1 -- 172.21.15.3:0/4055914170 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f661c021070 con 0x7f662c0a09c0 2023-10-03T10:16:52.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.205+0000 7f66297fa700 1 -- 172.21.15.3:0/4055914170 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f661c040ca0 con 0x7f662c0a09c0 2023-10-03T10:16:52.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.205+0000 7f6632c5d700 1 -- 172.21.15.3:0/4055914170 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f662c11ded0 con 0x7f662c0a09c0 2023-10-03T10:16:52.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.210+0000 7f66297fa700 1 -- 172.21.15.3:0/4055914170 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f661c029020 con 0x7f662c0a09c0 2023-10-03T10:16:52.262 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:52.298 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:52.336 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:52.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.406+0000 7f9ee5e81700 1 Processor -- start 2023-10-03T10:16:52.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.406+0000 7f9ee5e81700 1 -- start start 2023-10-03T10:16:52.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.407+0000 7f9ee5e81700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9ee009e1d0 0x7f9ee0138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:52.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.407+0000 7f9ee5e81700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ee0139170 0x7f9ee013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:52.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.407+0000 7f9ee5e81700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee009b120 con 0x7f9ee009d6f0 2023-10-03T10:16:52.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.407+0000 7f9ee5e81700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee006b460 con 0x7f9ee009e1d0 2023-10-03T10:16:52.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.407+0000 7f9ee5e81700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee009a7e0 con 0x7f9ee0139170 2023-10-03T10:16:52.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.407+0000 7f9edeffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9ee009e1d0 0x7f9ee0138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:52.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.407+0000 7f9edeffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9ee009e1d0 0x7f9ee0138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48710/0 (socket says 172.21.15.3:48710) 2023-10-03T10:16:52.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.407+0000 7f9edeffd700 1 -- 172.21.15.3:0/59306142 learned_addr learned my addr 172.21.15.3:0/59306142 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:52.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.407+0000 7f9edffff700 1 --2- 172.21.15.3:0/59306142 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ee0139170 0x7f9ee013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:52.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.407+0000 7f9edeffd700 1 -- 172.21.15.3:0/59306142 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ee0139170 msgr2=0x7f9ee013b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:52.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.407+0000 7f9edeffd700 1 --2- 172.21.15.3:0/59306142 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ee0139170 0x7f9ee013b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.407+0000 7f9edeffd700 1 -- 172.21.15.3:0/59306142 >> v1:172.21.15.3:6789/0 conn(0x7f9ee009d6f0 legacy=0x7f9ee009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:52.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.407+0000 7f9edeffd700 1 -- 172.21.15.3:0/59306142 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee009a500 con 0x7f9ee009e1d0 2023-10-03T10:16:52.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.407+0000 7f9edffff700 1 --2- 172.21.15.3:0/59306142 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ee0139170 0x7f9ee013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:52.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.408+0000 7f9edeffd700 1 --2- 172.21.15.3:0/59306142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee009e1d0 0x7f9ee0138c30 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009920 tx=0x7f9ed4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbee90c4e70d75e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:52.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.408+0000 7f9ede7fc700 1 -- 172.21.15.3:0/59306142 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4011830 con 0x7f9ee009e1d0 2023-10-03T10:16:52.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.408+0000 7f9ede7fc700 1 -- 172.21.15.3:0/59306142 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed4011990 con 0x7f9ee009e1d0 2023-10-03T10:16:52.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.408+0000 7f9ede7fc700 1 -- 172.21.15.3:0/59306142 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4011d20 con 0x7f9ee009e1d0 2023-10-03T10:16:52.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.409+0000 7f9ee5e81700 1 -- 172.21.15.3:0/59306142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee009e1d0 msgr2=0x7f9ee0138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:52.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.409+0000 7f9ee5e81700 1 --2- 172.21.15.3:0/59306142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee009e1d0 0x7f9ee0138c30 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009920 tx=0x7f9ed4009c30 comp rx=0 tx=0).stop 2023-10-03T10:16:52.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.409+0000 7f9ee5e81700 1 -- 172.21.15.3:0/59306142 shutdown_connections 2023-10-03T10:16:52.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.409+0000 7f9ee5e81700 1 --2- 172.21.15.3:0/59306142 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ee0139170 0x7f9ee013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.409+0000 7f9ee5e81700 1 --2- 172.21.15.3:0/59306142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee009e1d0 0x7f9ee0138c30 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.409+0000 7f9ee5e81700 1 -- 172.21.15.3:0/59306142 >> 172.21.15.3:0/59306142 conn(0x7f9ee0066f50 msgr2=0x7f9ee009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:52.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.409+0000 7f9ee5e81700 1 -- 172.21.15.3:0/59306142 shutdown_connections 2023-10-03T10:16:52.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.409+0000 7f9ee5e81700 1 -- 172.21.15.3:0/59306142 wait complete. 2023-10-03T10:16:52.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.410+0000 7f9ee5e81700 1 Processor -- start 2023-10-03T10:16:52.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.410+0000 7f9ee5e81700 1 -- start start 2023-10-03T10:16:52.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.411+0000 7f9ee5e81700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ee009d6f0 0x7f9ee01c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:52.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.411+0000 7f9ee5e81700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ee009e1d0 0x7f9ee01c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:52.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.411+0000 7f9ee5e81700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee0139170 0x7f9ee011e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:52.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.411+0000 7f9ee5e81700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee00a1a50 con 0x7f9ee0139170 2023-10-03T10:16:52.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.411+0000 7f9ee5e81700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee0121d20 con 0x7f9ee009e1d0 2023-10-03T10:16:52.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.411+0000 7f9ee5e81700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee009c710 con 0x7f9ee009d6f0 2023-10-03T10:16:52.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.411+0000 7f9edf7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ee009d6f0 0x7f9ee01c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:52.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.411+0000 7f9edf7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ee009d6f0 0x7f9ee01c84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51880/0 (socket says 172.21.15.3:51880) 2023-10-03T10:16:52.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.411+0000 7f9edf7fe700 1 -- 172.21.15.3:0/2100241523 learned_addr learned my addr 172.21.15.3:0/2100241523 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:52.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.411+0000 7f9edffff700 1 --2- 172.21.15.3:0/2100241523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee0139170 0x7f9ee011e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:52.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.411+0000 7f9edeffd700 1 --2- 172.21.15.3:0/2100241523 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ee009e1d0 0x7f9ee01c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:52.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.411+0000 7f9edf7fe700 1 -- 172.21.15.3:0/2100241523 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ee009e1d0 msgr2=0x7f9ee01c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:52.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.411+0000 7f9edf7fe700 1 --2- 172.21.15.3:0/2100241523 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ee009e1d0 0x7f9ee01c8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.411+0000 7f9edf7fe700 1 -- 172.21.15.3:0/2100241523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee0139170 msgr2=0x7f9ee011e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:52.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.411+0000 7f9edf7fe700 1 --2- 172.21.15.3:0/2100241523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee0139170 0x7f9ee011e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.411+0000 7f9edf7fe700 1 -- 172.21.15.3:0/2100241523 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed8009a70 con 0x7f9ee009d6f0 2023-10-03T10:16:52.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.412+0000 7f9edf7fe700 1 --2- 172.21.15.3:0/2100241523 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ee009d6f0 0x7f9ee01c84f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9ed000c540 tx=0x7f9ed000c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:52.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.412+0000 7f9ec7fff700 1 -- 172.21.15.3:0/2100241523 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed00052d0 con 0x7f9ee009d6f0 2023-10-03T10:16:52.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.412+0000 7f9ec7fff700 1 -- 172.21.15.3:0/2100241523 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed0005430 con 0x7f9ee009d6f0 2023-10-03T10:16:52.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.412+0000 7f9ec7fff700 1 -- 172.21.15.3:0/2100241523 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed00038a0 con 0x7f9ee009d6f0 2023-10-03T10:16:52.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.412+0000 7f9ee5e81700 1 -- 172.21.15.3:0/2100241523 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed4009550 con 0x7f9ee009d6f0 2023-10-03T10:16:52.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.412+0000 7f9ee5e81700 1 -- 172.21.15.3:0/2100241523 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee0120450 con 0x7f9ee009d6f0 2023-10-03T10:16:52.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.413+0000 7f9ec7fff700 1 -- 172.21.15.3:0/2100241523 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9ed0003a00 con 0x7f9ee009d6f0 2023-10-03T10:16:52.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.413+0000 7f9ee5e81700 1 -- 172.21.15.3:0/2100241523 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee0122b10 con 0x7f9ee009d6f0 2023-10-03T10:16:52.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.413+0000 7f9ec7fff700 1 -- 172.21.15.3:0/2100241523 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9ed003e9f0 con 0x7f9ee009d6f0 2023-10-03T10:16:52.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.419+0000 7f9ec7fff700 1 -- 172.21.15.3:0/2100241523 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9ed0003cb0 con 0x7f9ee009d6f0 2023-10-03T10:16:52.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.433+0000 7f6632c5d700 1 -- 172.21.15.3:0/4055914170 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f662c04f2a0 con 0x7f662c0a09c0 2023-10-03T10:16:52.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.433+0000 7f66297fa700 1 -- 172.21.15.3:0/4055914170 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f661c015070 con 0x7f662c0a09c0 2023-10-03T10:16:52.435 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:52.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.436+0000 7f6632c5d700 1 -- 172.21.15.3:0/4055914170 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f662c0a09c0 msgr2=0x7f662c121330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:52.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.436+0000 7f6632c5d700 1 --2- 172.21.15.3:0/4055914170 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f662c0a09c0 0x7f662c121330 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f661c000c00 tx=0x7f661c004750 comp rx=0 tx=0).stop 2023-10-03T10:16:52.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.436+0000 7f6632c5d700 1 -- 172.21.15.3:0/4055914170 shutdown_connections 2023-10-03T10:16:52.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.436+0000 7f6632c5d700 1 --2- 172.21.15.3:0/4055914170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f662c1338a0 0x7f662c121db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.436+0000 7f6632c5d700 1 --2- 172.21.15.3:0/4055914170 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f662c0a09c0 0x7f662c121330 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.436+0000 7f6632c5d700 1 --2- 172.21.15.3:0/4055914170 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f662c0a0d90 0x7f662c121870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.436+0000 7f6632c5d700 1 -- 172.21.15.3:0/4055914170 >> 172.21.15.3:0/4055914170 conn(0x7f662c0a60c0 msgr2=0x7f662c133180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:52.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.436+0000 7f6632c5d700 1 -- 172.21.15.3:0/4055914170 shutdown_connections 2023-10-03T10:16:52.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.436+0000 7f6632c5d700 1 -- 172.21.15.3:0/4055914170 wait complete. 2023-10-03T10:16:52.451 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:52.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.453+0000 7f56f1bb8700 1 Processor -- start 2023-10-03T10:16:52.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.453+0000 7f56f1bb8700 1 -- start start 2023-10-03T10:16:52.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.454+0000 7f56f1bb8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56ec133540 0x7f56ec135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:52.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.454+0000 7f56f1bb8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56ec135e60 0x7f56ec138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:52.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.454+0000 7f56f1bb8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56ec0716a0 con 0x7f56ec130540 2023-10-03T10:16:52.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.454+0000 7f56f1bb8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec071290 con 0x7f56ec133540 2023-10-03T10:16:52.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.454+0000 7f56f1bb8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec130340 con 0x7f56ec135e60 2023-10-03T10:16:52.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.454+0000 7f56eaffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56ec133540 0x7f56ec135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:52.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.454+0000 7f56eaffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56ec133540 0x7f56ec135920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53340/0 (socket says 172.21.15.3:53340) 2023-10-03T10:16:52.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.454+0000 7f56eaffd700 1 -- 172.21.15.3:0/3522175316 learned_addr learned my addr 172.21.15.3:0/3522175316 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:52.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.454+0000 7f56ebfff700 1 --2- 172.21.15.3:0/3522175316 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56ec135e60 0x7f56ec138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:52.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.454+0000 7f56ea7fc700 1 -- 172.21.15.3:0/3522175316 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1687531567 0 0) 0x7f56ec0716a0 con 0x7f56ec130540 2023-10-03T10:16:52.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.454+0000 7f56eaffd700 1 -- 172.21.15.3:0/3522175316 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56ec135e60 msgr2=0x7f56ec138240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:52.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.454+0000 7f56eaffd700 1 --2- 172.21.15.3:0/3522175316 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56ec135e60 0x7f56ec138240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.454+0000 7f56eaffd700 1 -- 172.21.15.3:0/3522175316 >> v1:172.21.15.3:6789/0 conn(0x7f56ec130540 legacy=0x7f56ec130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:52.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.454+0000 7f56eaffd700 1 -- 172.21.15.3:0/3522175316 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ec070b80 con 0x7f56ec133540 2023-10-03T10:16:52.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.455+0000 7f56eaffd700 1 --2- 172.21.15.3:0/3522175316 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56ec133540 0x7f56ec135920 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f56e000bbc0 tx=0x7f56e000bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d167c256a073d7ca server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:52.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.455+0000 7f56ea7fc700 1 -- 172.21.15.3:0/3522175316 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e000dc70 con 0x7f56ec133540 2023-10-03T10:16:52.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.455+0000 7f56ea7fc700 1 -- 172.21.15.3:0/3522175316 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e000ddd0 con 0x7f56ec133540 2023-10-03T10:16:52.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.455+0000 7f56ea7fc700 1 -- 172.21.15.3:0/3522175316 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e00106a0 con 0x7f56ec133540 2023-10-03T10:16:52.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.456+0000 7f56f1bb8700 1 -- 172.21.15.3:0/3522175316 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56ec133540 msgr2=0x7f56ec135920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:52.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.456+0000 7f56f1bb8700 1 --2- 172.21.15.3:0/3522175316 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56ec133540 0x7f56ec135920 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f56e000bbc0 tx=0x7f56e000bed0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.456+0000 7f56f1bb8700 1 -- 172.21.15.3:0/3522175316 shutdown_connections 2023-10-03T10:16:52.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.456+0000 7f56f1bb8700 1 --2- 172.21.15.3:0/3522175316 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56ec135e60 0x7f56ec138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.456+0000 7f56f1bb8700 1 --2- 172.21.15.3:0/3522175316 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56ec133540 0x7f56ec135920 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.456+0000 7f56f1bb8700 1 -- 172.21.15.3:0/3522175316 >> 172.21.15.3:0/3522175316 conn(0x7f56ec0a2aa0 msgr2=0x7f56ec0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:52.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.456+0000 7f56f1bb8700 1 -- 172.21.15.3:0/3522175316 shutdown_connections 2023-10-03T10:16:52.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.456+0000 7f56f1bb8700 1 -- 172.21.15.3:0/3522175316 wait complete. 2023-10-03T10:16:52.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.457+0000 7f56f1bb8700 1 Processor -- start 2023-10-03T10:16:52.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.457+0000 7f56f1bb8700 1 -- start start 2023-10-03T10:16:52.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.457+0000 7f56f1bb8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56ec130540 0x7f56ec09c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:52.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.457+0000 7f56f1bb8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56ec133540 0x7f56ec09ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:52.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.457+0000 7f56f1bb8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56ec135e60 0x7f56ec09cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:52.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.457+0000 7f56f1bb8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec09e360 con 0x7f56ec135e60 2023-10-03T10:16:52.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.457+0000 7f56f1bb8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec09e100 con 0x7f56ec133540 2023-10-03T10:16:52.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.457+0000 7f56f1bb8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec119a50 con 0x7f56ec130540 2023-10-03T10:16:52.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.458+0000 7f56eb7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56ec130540 0x7f56ec09c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:52.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.458+0000 7f56eaffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56ec133540 0x7f56ec09ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:52.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.458+0000 7f56eaffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56ec133540 0x7f56ec09ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:53344/0 (socket says 172.21.15.3:53344) 2023-10-03T10:16:52.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.458+0000 7f56eb7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56ec130540 0x7f56ec09c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51894/0 (socket says 172.21.15.3:51894) 2023-10-03T10:16:52.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.458+0000 7f56eaffd700 1 -- 172.21.15.3:0/299559242 learned_addr learned my addr 172.21.15.3:0/299559242 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:52.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.458+0000 7f56ebfff700 1 --2- 172.21.15.3:0/299559242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56ec135e60 0x7f56ec09cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:52.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.458+0000 7f56eb7fe700 1 -- 172.21.15.3:0/299559242 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56ec133540 msgr2=0x7f56ec09ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:52.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.458+0000 7f56eb7fe700 1 --2- 172.21.15.3:0/299559242 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56ec133540 0x7f56ec09ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.458+0000 7f56eb7fe700 1 -- 172.21.15.3:0/299559242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56ec135e60 msgr2=0x7f56ec09cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:52.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.458+0000 7f56eb7fe700 1 --2- 172.21.15.3:0/299559242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56ec135e60 0x7f56ec09cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.458+0000 7f56eb7fe700 1 -- 172.21.15.3:0/299559242 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e000b7f0 con 0x7f56ec130540 2023-10-03T10:16:52.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.458+0000 7f56eaffd700 1 --2- 172.21.15.3:0/299559242 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56ec133540 0x7f56ec09ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:52.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.458+0000 7f56eb7fe700 1 --2- 172.21.15.3:0/299559242 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56ec130540 0x7f56ec09c530 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f56dc00c3c0 tx=0x7f56dc00c6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:52.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.458+0000 7f56d3fff700 1 -- 172.21.15.3:0/299559242 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc0052d0 con 0x7f56ec130540 2023-10-03T10:16:52.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.459+0000 7f56d3fff700 1 -- 172.21.15.3:0/299559242 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56dc005450 con 0x7f56ec130540 2023-10-03T10:16:52.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.459+0000 7f56f1bb8700 1 -- 172.21.15.3:0/299559242 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56ec11a810 con 0x7f56ec130540 2023-10-03T10:16:52.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.459+0000 7f56d3fff700 1 -- 172.21.15.3:0/299559242 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc0038c0 con 0x7f56ec130540 2023-10-03T10:16:52.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.459+0000 7f56f1bb8700 1 -- 172.21.15.3:0/299559242 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ec11aea0 con 0x7f56ec130540 2023-10-03T10:16:52.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.460+0000 7f56d3fff700 1 -- 172.21.15.3:0/299559242 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f56dc003a20 con 0x7f56ec130540 2023-10-03T10:16:52.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.460+0000 7f56d3fff700 1 -- 172.21.15.3:0/299559242 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f56dc03e9d0 con 0x7f56ec130540 2023-10-03T10:16:52.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.460+0000 7f56f1bb8700 1 -- 172.21.15.3:0/299559242 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56ec074ea0 con 0x7f56ec130540 2023-10-03T10:16:52.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.465+0000 7f56d3fff700 1 -- 172.21.15.3:0/299559242 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f56dc017030 con 0x7f56ec130540 2023-10-03T10:16:52.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.497+0000 7f5a127de700 1 Processor -- start 2023-10-03T10:16:52.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.497+0000 7f5a127de700 1 -- start start 2023-10-03T10:16:52.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.498+0000 7f5a127de700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5a0c09d630 0x7f5a0c0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:52.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.498+0000 7f5a127de700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5a0c138d80 0x7f5a0c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:52.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.498+0000 7f5a127de700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a0c0a2a70 con 0x7f5a0c0a0a30 2023-10-03T10:16:52.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.498+0000 7f5a127de700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a0c0a01d0 con 0x7f5a0c138d80 2023-10-03T10:16:52.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.498+0000 7f5a127de700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c09e2b0 con 0x7f5a0c09d630 2023-10-03T10:16:52.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.498+0000 7f5a0b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5a0c09d630 0x7f5a0c0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:52.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.498+0000 7f5a0b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5a0c09d630 0x7f5a0c0a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:51896/0 (socket says 172.21.15.3:51896) 2023-10-03T10:16:52.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.498+0000 7f5a0b7fe700 1 -- 172.21.15.3:0/3986178575 learned_addr learned my addr 172.21.15.3:0/3986178575 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:52.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.498+0000 7f5a10d7b700 1 --2- 172.21.15.3:0/3986178575 >> v2:172.21.15.3:3300/0 conn(0x7f5a0c138d80 0x7f5a0c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:52.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.498+0000 7f5a0b7fe700 1 -- 172.21.15.3:0/3986178575 >> v2:172.21.15.3:3300/0 conn(0x7f5a0c138d80 msgr2=0x7f5a0c13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:52.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.498+0000 7f5a0b7fe700 1 --2- 172.21.15.3:0/3986178575 >> v2:172.21.15.3:3300/0 conn(0x7f5a0c138d80 0x7f5a0c13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.498+0000 7f5a0b7fe700 1 -- 172.21.15.3:0/3986178575 >> v1:172.21.15.3:6789/0 conn(0x7f5a0c0a0a30 legacy=0x7f5a0c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:52.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.498+0000 7f5a0affd700 1 -- 172.21.15.3:0/3986178575 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2673824686 0 0) 0x7f5a0c0a2a70 con 0x7f5a0c0a0a30 2023-10-03T10:16:52.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.498+0000 7f5a0b7fe700 1 -- 172.21.15.3:0/3986178575 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0c09dc50 con 0x7f5a0c09d630 2023-10-03T10:16:52.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.498+0000 7f5a0b7fe700 1 --2- 172.21.15.3:0/3986178575 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5a0c09d630 0x7f5a0c0a1510 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5a000098f0 tx=0x7f5a00009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2725d3c9ac7ad38 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:52.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.499+0000 7f5a0affd700 1 -- 172.21.15.3:0/3986178575 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a00011830 con 0x7f5a0c09d630 2023-10-03T10:16:52.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.499+0000 7f5a0affd700 1 -- 172.21.15.3:0/3986178575 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a00011990 con 0x7f5a0c09d630 2023-10-03T10:16:52.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.499+0000 7f5a0affd700 1 -- 172.21.15.3:0/3986178575 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a00011d20 con 0x7f5a0c09d630 2023-10-03T10:16:52.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.500+0000 7f5a127de700 1 -- 172.21.15.3:0/3986178575 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5a0c09d630 msgr2=0x7f5a0c0a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:52.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.500+0000 7f5a127de700 1 --2- 172.21.15.3:0/3986178575 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5a0c09d630 0x7f5a0c0a1510 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5a000098f0 tx=0x7f5a00009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:52.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.500+0000 7f5a127de700 1 -- 172.21.15.3:0/3986178575 shutdown_connections 2023-10-03T10:16:52.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.500+0000 7f5a127de700 1 --2- 172.21.15.3:0/3986178575 >> v2:172.21.15.3:3300/0 conn(0x7f5a0c138d80 0x7f5a0c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.500+0000 7f5a127de700 1 --2- 172.21.15.3:0/3986178575 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5a0c09d630 0x7f5a0c0a1510 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.500+0000 7f5a127de700 1 -- 172.21.15.3:0/3986178575 >> 172.21.15.3:0/3986178575 conn(0x7f5a0c0a3050 msgr2=0x7f5a0c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:52.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.500+0000 7f5a127de700 1 -- 172.21.15.3:0/3986178575 shutdown_connections 2023-10-03T10:16:52.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.500+0000 7f5a127de700 1 -- 172.21.15.3:0/3986178575 wait complete. 2023-10-03T10:16:52.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.501+0000 7f5a127de700 1 Processor -- start 2023-10-03T10:16:52.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.501+0000 7f5a127de700 1 -- start start 2023-10-03T10:16:52.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.502+0000 7f5a127de700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5a0c09d630 0x7f5a0c133740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:52.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.502+0000 7f5a127de700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5a0c0a0a30 0x7f5a0c133c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:52.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.502+0000 7f5a127de700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a0c138d80 0x7f5a0c1341c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:52.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.502+0000 7f5a127de700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c134ba0 con 0x7f5a0c138d80 2023-10-03T10:16:52.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.502+0000 7f5a127de700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c136580 con 0x7f5a0c09d630 2023-10-03T10:16:52.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.502+0000 7f5a127de700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c136ef0 con 0x7f5a0c0a0a30 2023-10-03T10:16:52.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.502+0000 7f5a10d7b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a0c138d80 0x7f5a0c1341c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:52.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.502+0000 7f5a10d7b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a0c138d80 0x7f5a0c1341c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48754/0 (socket says 172.21.15.3:48754) 2023-10-03T10:16:52.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.502+0000 7f5a10d7b700 1 -- 172.21.15.3:0/4200867760 learned_addr learned my addr 172.21.15.3:0/4200867760 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:52.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.502+0000 7f5a0b7fe700 1 --2- 172.21.15.3:0/4200867760 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5a0c0a0a30 0x7f5a0c133c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:52.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.502+0000 7f5a10d7b700 1 -- 172.21.15.3:0/4200867760 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5a0c0a0a30 msgr2=0x7f5a0c133c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:52.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.502+0000 7f5a10d7b700 1 --2- 172.21.15.3:0/4200867760 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5a0c0a0a30 0x7f5a0c133c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.502+0000 7f5a10d7b700 1 -- 172.21.15.3:0/4200867760 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5a0c09d630 msgr2=0x7f5a0c133740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:52.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.502+0000 7f5a10d7b700 1 --2- 172.21.15.3:0/4200867760 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5a0c09d630 0x7f5a0c133740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.502+0000 7f5a10d7b700 1 -- 172.21.15.3:0/4200867760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a00009550 con 0x7f5a0c138d80 2023-10-03T10:16:52.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.503+0000 7f5a10d7b700 1 --2- 172.21.15.3:0/4200867760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a0c138d80 0x7f5a0c1341c0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400efc0 tx=0x7f5a0400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:52.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.503+0000 7f5a08ff9700 1 -- 172.21.15.3:0/4200867760 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0400cd40 con 0x7f5a0c138d80 2023-10-03T10:16:52.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.503+0000 7f5a08ff9700 1 -- 172.21.15.3:0/4200867760 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0400f040 con 0x7f5a0c138d80 2023-10-03T10:16:52.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.503+0000 7f5a127de700 1 -- 172.21.15.3:0/4200867760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a0c13ac80 con 0x7f5a0c138d80 2023-10-03T10:16:52.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.503+0000 7f5a08ff9700 1 -- 172.21.15.3:0/4200867760 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a040146c0 con 0x7f5a0c138d80 2023-10-03T10:16:52.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.503+0000 7f5a127de700 1 -- 172.21.15.3:0/4200867760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a0c137960 con 0x7f5a0c138d80 2023-10-03T10:16:52.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.504+0000 7f5a127de700 1 -- 172.21.15.3:0/4200867760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a0c0a1810 con 0x7f5a0c138d80 2023-10-03T10:16:52.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.504+0000 7f5a08ff9700 1 -- 172.21.15.3:0/4200867760 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5a04014820 con 0x7f5a0c138d80 2023-10-03T10:16:52.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.505+0000 7f5a08ff9700 1 -- 172.21.15.3:0/4200867760 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5a04040320 con 0x7f5a0c138d80 2023-10-03T10:16:52.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.510+0000 7f5a08ff9700 1 -- 172.21.15.3:0/4200867760 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5a04007d90 con 0x7f5a0c138d80 2023-10-03T10:16:52.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.632+0000 7f9ee5e81700 1 -- 172.21.15.3:0/2100241523 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9ee009f420 con 0x7f9ee009d6f0 2023-10-03T10:16:52.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.632+0000 7f9ec7fff700 1 -- 172.21.15.3:0/2100241523 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9ed0017020 con 0x7f9ee009d6f0 2023-10-03T10:16:52.632 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:52.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.634+0000 7f9ee5e81700 1 -- 172.21.15.3:0/2100241523 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ee009d6f0 msgr2=0x7f9ee01c84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:52.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.634+0000 7f9ee5e81700 1 --2- 172.21.15.3:0/2100241523 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ee009d6f0 0x7f9ee01c84f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9ed000c540 tx=0x7f9ed000c850 comp rx=0 tx=0).stop 2023-10-03T10:16:52.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.635+0000 7f9ee5e81700 1 -- 172.21.15.3:0/2100241523 shutdown_connections 2023-10-03T10:16:52.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.635+0000 7f9ee5e81700 1 --2- 172.21.15.3:0/2100241523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee0139170 0x7f9ee011e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.635+0000 7f9ee5e81700 1 --2- 172.21.15.3:0/2100241523 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ee009d6f0 0x7f9ee01c84f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.635+0000 7f9ee5e81700 1 --2- 172.21.15.3:0/2100241523 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ee009e1d0 0x7f9ee01c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.635+0000 7f9ee5e81700 1 -- 172.21.15.3:0/2100241523 >> 172.21.15.3:0/2100241523 conn(0x7f9ee0066f50 msgr2=0x7f9ee009b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:52.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.635+0000 7f9ee5e81700 1 -- 172.21.15.3:0/2100241523 shutdown_connections 2023-10-03T10:16:52.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.635+0000 7f9ee5e81700 1 -- 172.21.15.3:0/2100241523 wait complete. 2023-10-03T10:16:52.649 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:52.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.673+0000 7f56f1bb8700 1 -- 172.21.15.3:0/299559242 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f56ec12de10 con 0x7f56ec130540 2023-10-03T10:16:52.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.673+0000 7f56d3fff700 1 -- 172.21.15.3:0/299559242 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f56dc019d20 con 0x7f56ec130540 2023-10-03T10:16:52.674 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:52.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.676+0000 7f56f1bb8700 1 -- 172.21.15.3:0/299559242 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56ec130540 msgr2=0x7f56ec09c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:52.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.676+0000 7f56f1bb8700 1 --2- 172.21.15.3:0/299559242 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56ec130540 0x7f56ec09c530 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f56dc00c3c0 tx=0x7f56dc00c6d0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.676+0000 7f56f1bb8700 1 -- 172.21.15.3:0/299559242 shutdown_connections 2023-10-03T10:16:52.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.676+0000 7f56f1bb8700 1 --2- 172.21.15.3:0/299559242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56ec135e60 0x7f56ec09cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.676+0000 7f56f1bb8700 1 --2- 172.21.15.3:0/299559242 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f56ec130540 0x7f56ec09c530 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.676+0000 7f56f1bb8700 1 --2- 172.21.15.3:0/299559242 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f56ec133540 0x7f56ec09ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.676+0000 7f56f1bb8700 1 -- 172.21.15.3:0/299559242 >> 172.21.15.3:0/299559242 conn(0x7f56ec0a2aa0 msgr2=0x7f56ec131300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:52.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.676+0000 7f56f1bb8700 1 -- 172.21.15.3:0/299559242 shutdown_connections 2023-10-03T10:16:52.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.676+0000 7f56f1bb8700 1 -- 172.21.15.3:0/299559242 wait complete. 2023-10-03T10:16:52.690 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:52.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.719+0000 7f5a127de700 1 -- 172.21.15.3:0/4200867760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5a0c0747f0 con 0x7f5a0c138d80 2023-10-03T10:16:52.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.720+0000 7f5a08ff9700 1 -- 172.21.15.3:0/4200867760 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5a04018020 con 0x7f5a0c138d80 2023-10-03T10:16:52.720 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:52.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.722+0000 7f5a127de700 1 -- 172.21.15.3:0/4200867760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a0c138d80 msgr2=0x7f5a0c1341c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:52.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.722+0000 7f5a127de700 1 --2- 172.21.15.3:0/4200867760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a0c138d80 0x7f5a0c1341c0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400efc0 tx=0x7f5a0400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.722+0000 7f5a127de700 1 -- 172.21.15.3:0/4200867760 shutdown_connections 2023-10-03T10:16:52.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.722+0000 7f5a127de700 1 --2- 172.21.15.3:0/4200867760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a0c138d80 0x7f5a0c1341c0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.722+0000 7f5a127de700 1 --2- 172.21.15.3:0/4200867760 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5a0c09d630 0x7f5a0c133740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.722+0000 7f5a127de700 1 --2- 172.21.15.3:0/4200867760 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5a0c0a0a30 0x7f5a0c133c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:52.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.722+0000 7f5a127de700 1 -- 172.21.15.3:0/4200867760 >> 172.21.15.3:0/4200867760 conn(0x7f5a0c0a3050 msgr2=0x7f5a0c0a6780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:52.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.722+0000 7f5a127de700 1 -- 172.21.15.3:0/4200867760 shutdown_connections 2023-10-03T10:16:52.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:52.722+0000 7f5a127de700 1 -- 172.21.15.3:0/4200867760 wait complete. 2023-10-03T10:16:52.736 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:53.453 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:53.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.598+0000 7f2ab1e47700 1 Processor -- start 2023-10-03T10:16:53.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.599+0000 7f2ab1e47700 1 -- start start 2023-10-03T10:16:53.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.599+0000 7f2ab1e47700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2aac1338a0 0x7f2aac133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:53.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.599+0000 7f2ab1e47700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2aac09e930 0x7f2aac13b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:53.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.599+0000 7f2ab1e47700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aac06ef30 con 0x7f2aac09de50 2023-10-03T10:16:53.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.599+0000 7f2ab1e47700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac06fef0 con 0x7f2aac1338a0 2023-10-03T10:16:53.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.599+0000 7f2ab1e47700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac1334b0 con 0x7f2aac09e930 2023-10-03T10:16:53.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.599+0000 7f2aab7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2aac1338a0 0x7f2aac133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:53.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.599+0000 7f2aaaffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2aac09de50 0x7f2aac09e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39834/0 (socket says 172.21.15.3:39834) 2023-10-03T10:16:53.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.599+0000 7f2aab7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2aac1338a0 0x7f2aac133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33148/0 (socket says 172.21.15.3:33148) 2023-10-03T10:16:53.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.599+0000 7f2aabfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2aac09e930 0x7f2aac13b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:53.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.599+0000 7f2aaaffd700 1 -- 172.21.15.3:0/3033647166 learned_addr learned my addr 172.21.15.3:0/3033647166 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:53.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.600+0000 7f2aaa7fc700 1 -- 172.21.15.3:0/3033647166 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2561244531 0 0) 0x7f2aac06ef30 con 0x7f2aac09de50 2023-10-03T10:16:53.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.600+0000 7f2aaa7fc700 1 -- 172.21.15.3:0/3033647166 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a940035b0 con 0x7f2aac09de50 2023-10-03T10:16:53.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.600+0000 7f2aab7fe700 1 -- 172.21.15.3:0/3033647166 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2aac09e930 msgr2=0x7f2aac13b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:53.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.600+0000 7f2aab7fe700 1 --2- 172.21.15.3:0/3033647166 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2aac09e930 0x7f2aac13b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:53.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.600+0000 7f2aab7fe700 1 -- 172.21.15.3:0/3033647166 >> v1:172.21.15.3:6789/0 conn(0x7f2aac09de50 legacy=0x7f2aac09e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:53.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.600+0000 7f2aab7fe700 1 -- 172.21.15.3:0/3033647166 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aac06fc30 con 0x7f2aac1338a0 2023-10-03T10:16:53.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.600+0000 7f2aabfff700 1 --2- 172.21.15.3:0/3033647166 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2aac09e930 0x7f2aac13b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:53.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.600+0000 7f2aab7fe700 1 --2- 172.21.15.3:0/3033647166 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2aac1338a0 0x7f2aac133c70 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c009870 tx=0x7f2a9c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e0ec654ffc39686 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:53.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.600+0000 7f2aaa7fc700 1 -- 172.21.15.3:0/3033647166 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c011830 con 0x7f2aac1338a0 2023-10-03T10:16:53.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.600+0000 7f2aaa7fc700 1 -- 172.21.15.3:0/3033647166 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9c011990 con 0x7f2aac1338a0 2023-10-03T10:16:53.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.600+0000 7f2aaa7fc700 1 -- 172.21.15.3:0/3033647166 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c011d20 con 0x7f2aac1338a0 2023-10-03T10:16:53.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.601+0000 7f2ab1e47700 1 -- 172.21.15.3:0/3033647166 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2aac1338a0 msgr2=0x7f2aac133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:53.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.601+0000 7f2ab1e47700 1 --2- 172.21.15.3:0/3033647166 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2aac1338a0 0x7f2aac133c70 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c009870 tx=0x7f2a9c009b80 comp rx=0 tx=0).stop 2023-10-03T10:16:53.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.601+0000 7f2ab1e47700 1 -- 172.21.15.3:0/3033647166 shutdown_connections 2023-10-03T10:16:53.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.601+0000 7f2ab1e47700 1 --2- 172.21.15.3:0/3033647166 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2aac1338a0 0x7f2aac133c70 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:53.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.601+0000 7f2ab1e47700 1 --2- 172.21.15.3:0/3033647166 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2aac09e930 0x7f2aac13b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:53.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.601+0000 7f2ab1e47700 1 -- 172.21.15.3:0/3033647166 >> 172.21.15.3:0/3033647166 conn(0x7f2aac0a2aa0 msgr2=0x7f2aac0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:53.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.601+0000 7f2ab1e47700 1 -- 172.21.15.3:0/3033647166 shutdown_connections 2023-10-03T10:16:53.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.601+0000 7f2ab1e47700 1 -- 172.21.15.3:0/3033647166 wait complete. 2023-10-03T10:16:53.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.602+0000 7f2ab1e47700 1 Processor -- start 2023-10-03T10:16:53.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.602+0000 7f2ab1e47700 1 -- start start 2023-10-03T10:16:53.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.602+0000 7f2ab1e47700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aac09de50 0x7f2aac11d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:53.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.602+0000 7f2ab1e47700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2aac09e930 0x7f2aac11de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:53.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.602+0000 7f2ab1e47700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2aac1338a0 0x7f2aac11e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:53.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.602+0000 7f2ab1e47700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac121100 con 0x7f2aac09de50 2023-10-03T10:16:53.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.602+0000 7f2ab1e47700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac130cb0 con 0x7f2aac1338a0 2023-10-03T10:16:53.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.602+0000 7f2ab1e47700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac0a0bc0 con 0x7f2aac09e930 2023-10-03T10:16:53.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.603+0000 7f2aab7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aac09de50 0x7f2aac11d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:53.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.603+0000 7f2aaaffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2aac09e930 0x7f2aac11de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:53.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.603+0000 7f2aabfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2aac1338a0 0x7f2aac11e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:53.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.603+0000 7f2aab7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aac09de50 0x7f2aac11d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48708/0 (socket says 172.21.15.3:48708) 2023-10-03T10:16:53.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.603+0000 7f2aab7fe700 1 -- 172.21.15.3:0/3823098160 learned_addr learned my addr 172.21.15.3:0/3823098160 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:53.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.603+0000 7f2aab7fe700 1 -- 172.21.15.3:0/3823098160 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2aac09e930 msgr2=0x7f2aac11de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:53.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.603+0000 7f2aab7fe700 1 --2- 172.21.15.3:0/3823098160 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2aac09e930 0x7f2aac11de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:53.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.603+0000 7f2aab7fe700 1 -- 172.21.15.3:0/3823098160 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2aac1338a0 msgr2=0x7f2aac11e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:53.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.603+0000 7f2aab7fe700 1 --2- 172.21.15.3:0/3823098160 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2aac1338a0 0x7f2aac11e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:53.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.603+0000 7f2aab7fe700 1 -- 172.21.15.3:0/3823098160 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa4009a70 con 0x7f2aac09de50 2023-10-03T10:16:53.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.603+0000 7f2aabfff700 1 --2- 172.21.15.3:0/3823098160 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2aac1338a0 0x7f2aac11e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:53.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.603+0000 7f2aab7fe700 1 --2- 172.21.15.3:0/3823098160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aac09de50 0x7f2aac11d8c0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c009bc0 tx=0x7f2a9c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:53.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.603+0000 7f2a93fff700 1 -- 172.21.15.3:0/3823098160 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c004450 con 0x7f2aac09de50 2023-10-03T10:16:53.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.603+0000 7f2a93fff700 1 -- 172.21.15.3:0/3823098160 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9c0045b0 con 0x7f2aac09de50 2023-10-03T10:16:53.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.603+0000 7f2ab1e47700 1 -- 172.21.15.3:0/3823098160 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a9c0094a0 con 0x7f2aac09de50 2023-10-03T10:16:53.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.603+0000 7f2a93fff700 1 -- 172.21.15.3:0/3823098160 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c0038e0 con 0x7f2aac09de50 2023-10-03T10:16:53.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.603+0000 7f2ab1e47700 1 -- 172.21.15.3:0/3823098160 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aac120e40 con 0x7f2aac09de50 2023-10-03T10:16:53.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.604+0000 7f2a93fff700 1 -- 172.21.15.3:0/3823098160 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2a9c003a40 con 0x7f2aac09de50 2023-10-03T10:16:53.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.604+0000 7f2a93fff700 1 -- 172.21.15.3:0/3823098160 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2a9c040840 con 0x7f2aac09de50 2023-10-03T10:16:53.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.604+0000 7f2ab1e47700 1 -- 172.21.15.3:0/3823098160 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aac072410 con 0x7f2aac09de50 2023-10-03T10:16:53.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.610+0000 7f2a93fff700 1 -- 172.21.15.3:0/3823098160 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2a9c015020 con 0x7f2aac09de50 2023-10-03T10:16:53.649 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:53.691 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:53.737 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:53.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.790+0000 7f3fff688700 1 Processor -- start 2023-10-03T10:16:53.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.790+0000 7f3fff688700 1 -- start start 2023-10-03T10:16:53.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.791+0000 7f3fff688700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ff806a770 0x7f3ff806ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:53.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.791+0000 7f3fff688700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3ff806b080 0x7f3ff8131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:53.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.791+0000 7f3fff688700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ff8131b40 0x7f3ff8131fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:53.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.791+0000 7f3fff688700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff809ece0 con 0x7f3ff806b080 2023-10-03T10:16:53.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.791+0000 7f3fff688700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff809e760 con 0x7f3ff8131b40 2023-10-03T10:16:53.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.791+0000 7f3fff688700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff809e040 con 0x7f3ff806a770 2023-10-03T10:16:53.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.791+0000 7f3ffcc23700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3ff806b080 0x7f3ff8131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:53.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.791+0000 7f3ffcc23700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3ff806b080 0x7f3ff8131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48718/0 (socket says 172.21.15.3:48718) 2023-10-03T10:16:53.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.791+0000 7f3ffcc23700 1 -- 172.21.15.3:0/349676846 learned_addr learned my addr 172.21.15.3:0/349676846 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:53.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.791+0000 7f3ffd424700 1 --2- 172.21.15.3:0/349676846 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ff806a770 0x7f3ff806ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:53.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.791+0000 7f3ffdc25700 1 --2- 172.21.15.3:0/349676846 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ff8131b40 0x7f3ff8131fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:53.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.791+0000 7f3ffdc25700 1 -- 172.21.15.3:0/349676846 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ff806a770 msgr2=0x7f3ff806ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:53.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.791+0000 7f3ffdc25700 1 --2- 172.21.15.3:0/349676846 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ff806a770 0x7f3ff806ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:53.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.791+0000 7f3ffdc25700 1 -- 172.21.15.3:0/349676846 >> v2:172.21.15.3:3300/0 conn(0x7f3ff806b080 msgr2=0x7f3ff8131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:53.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.791+0000 7f3ffdc25700 1 --2- 172.21.15.3:0/349676846 >> v2:172.21.15.3:3300/0 conn(0x7f3ff806b080 0x7f3ff8131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:53.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.791+0000 7f3ffdc25700 1 -- 172.21.15.3:0/349676846 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff809f630 con 0x7f3ff8131b40 2023-10-03T10:16:53.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.791+0000 7f3ffcc23700 1 --2- 172.21.15.3:0/349676846 >> v2:172.21.15.3:3300/0 conn(0x7f3ff806b080 0x7f3ff8131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:53.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.791+0000 7f3ffd424700 1 --2- 172.21.15.3:0/349676846 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ff806a770 0x7f3ff806ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:53.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.792+0000 7f3ffdc25700 1 --2- 172.21.15.3:0/349676846 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ff8131b40 0x7f3ff8131fc0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000f370 tx=0x7f3ff000f680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32bada7a47d08e3f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:53.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.792+0000 7f3feffff700 1 -- 172.21.15.3:0/349676846 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0012070 con 0x7f3ff8131b40 2023-10-03T10:16:53.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.792+0000 7f3feffff700 1 -- 172.21.15.3:0/349676846 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff000fe70 con 0x7f3ff8131b40 2023-10-03T10:16:53.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.792+0000 7f3feffff700 1 -- 172.21.15.3:0/349676846 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff000d2f0 con 0x7f3ff8131b40 2023-10-03T10:16:53.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.793+0000 7f3fff688700 1 -- 172.21.15.3:0/349676846 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ff8131b40 msgr2=0x7f3ff8131fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:53.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.793+0000 7f3fff688700 1 --2- 172.21.15.3:0/349676846 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ff8131b40 0x7f3ff8131fc0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000f370 tx=0x7f3ff000f680 comp rx=0 tx=0).stop 2023-10-03T10:16:53.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.793+0000 7f3fff688700 1 -- 172.21.15.3:0/349676846 shutdown_connections 2023-10-03T10:16:53.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.793+0000 7f3fff688700 1 --2- 172.21.15.3:0/349676846 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ff8131b40 0x7f3ff8131fc0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:53.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.793+0000 7f3fff688700 1 --2- 172.21.15.3:0/349676846 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ff806a770 0x7f3ff806ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:53.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.793+0000 7f3fff688700 1 --2- 172.21.15.3:0/349676846 >> v2:172.21.15.3:3300/0 conn(0x7f3ff806b080 0x7f3ff8131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:53.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.793+0000 7f3fff688700 1 -- 172.21.15.3:0/349676846 >> 172.21.15.3:0/349676846 conn(0x7f3ff8066f50 msgr2=0x7f3ff809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:53.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.793+0000 7f3fff688700 1 -- 172.21.15.3:0/349676846 shutdown_connections 2023-10-03T10:16:53.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.793+0000 7f3fff688700 1 -- 172.21.15.3:0/349676846 wait complete. 2023-10-03T10:16:53.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.794+0000 7f3fff688700 1 Processor -- start 2023-10-03T10:16:53.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.794+0000 7f3fff688700 1 -- start start 2023-10-03T10:16:53.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.795+0000 7f3fff688700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ff806a770 0x7f3ff811bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:53.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.795+0000 7f3fff688700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ff806b080 0x7f3ff811c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:53.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.795+0000 7f3fff688700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ff8131b40 0x7f3ff811c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:53.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.795+0000 7f3fff688700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff811f780 con 0x7f3ff806a770 2023-10-03T10:16:53.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.795+0000 7f3fff688700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff812f290 con 0x7f3ff806b080 2023-10-03T10:16:53.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.795+0000 7f3fff688700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff811ed00 con 0x7f3ff8131b40 2023-10-03T10:16:53.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.795+0000 7f3ffd424700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ff806a770 0x7f3ff811bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:53.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.795+0000 7f3ffd424700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ff806a770 0x7f3ff811bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48720/0 (socket says 172.21.15.3:48720) 2023-10-03T10:16:53.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.795+0000 7f3ffd424700 1 -- 172.21.15.3:0/2427446140 learned_addr learned my addr 172.21.15.3:0/2427446140 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:53.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.795+0000 7f3ffcc23700 1 --2- 172.21.15.3:0/2427446140 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ff806b080 0x7f3ff811c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:53.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.796+0000 7f3ffdc25700 1 --2- 172.21.15.3:0/2427446140 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ff8131b40 0x7f3ff811c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:53.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.796+0000 7f3ffd424700 1 -- 172.21.15.3:0/2427446140 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ff8131b40 msgr2=0x7f3ff811c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:53.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.796+0000 7f3ffd424700 1 --2- 172.21.15.3:0/2427446140 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ff8131b40 0x7f3ff811c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:53.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.796+0000 7f3ffd424700 1 -- 172.21.15.3:0/2427446140 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ff806b080 msgr2=0x7f3ff811c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:53.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.796+0000 7f3ffd424700 1 --2- 172.21.15.3:0/2427446140 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ff806b080 0x7f3ff811c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:53.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.796+0000 7f3ffd424700 1 -- 172.21.15.3:0/2427446140 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff40094a0 con 0x7f3ff806a770 2023-10-03T10:16:53.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.796+0000 7f3ffd424700 1 --2- 172.21.15.3:0/2427446140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ff806a770 0x7f3ff811bea0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400ec10 tx=0x7f3ff400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:53.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.796+0000 7f3fedffb700 1 -- 172.21.15.3:0/2427446140 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff400c910 con 0x7f3ff806a770 2023-10-03T10:16:53.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.796+0000 7f3fedffb700 1 -- 172.21.15.3:0/2427446140 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff400ca70 con 0x7f3ff806a770 2023-10-03T10:16:53.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.796+0000 7f3fff688700 1 -- 172.21.15.3:0/2427446140 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe8009550 con 0x7f3ff806a770 2023-10-03T10:16:53.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.796+0000 7f3fedffb700 1 -- 172.21.15.3:0/2427446140 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff400ce00 con 0x7f3ff806a770 2023-10-03T10:16:53.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.797+0000 7f3fedffb700 1 -- 172.21.15.3:0/2427446140 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3ff4013470 con 0x7f3ff806a770 2023-10-03T10:16:53.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.797+0000 7f3fff688700 1 -- 172.21.15.3:0/2427446140 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff000f050 con 0x7f3ff806a770 2023-10-03T10:16:53.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.798+0000 7f3fedffb700 1 -- 172.21.15.3:0/2427446140 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3ff4011e00 con 0x7f3ff806a770 2023-10-03T10:16:53.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.799+0000 7f3fff688700 1 -- 172.21.15.3:0/2427446140 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff8071e60 con 0x7f3ff806a770 2023-10-03T10:16:53.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.808+0000 7f3fedffb700 1 -- 172.21.15.3:0/2427446140 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3ff4017020 con 0x7f3ff806a770 2023-10-03T10:16:53.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.823+0000 7f2ab1e47700 1 -- 172.21.15.3:0/3823098160 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2aac074ea0 con 0x7f2aac09de50 2023-10-03T10:16:53.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.824+0000 7f2a93fff700 1 -- 172.21.15.3:0/3823098160 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2a9c02e970 con 0x7f2aac09de50 2023-10-03T10:16:53.824 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:53.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.827+0000 7f2ab1e47700 1 -- 172.21.15.3:0/3823098160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aac09de50 msgr2=0x7f2aac11d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:53.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.827+0000 7f2ab1e47700 1 --2- 172.21.15.3:0/3823098160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aac09de50 0x7f2aac11d8c0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c009bc0 tx=0x7f2a9c004750 comp rx=0 tx=0).stop 2023-10-03T10:16:53.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.827+0000 7f2ab1e47700 1 -- 172.21.15.3:0/3823098160 shutdown_connections 2023-10-03T10:16:53.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.827+0000 7f2ab1e47700 1 --2- 172.21.15.3:0/3823098160 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2aac1338a0 0x7f2aac11e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:53.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.827+0000 7f2ab1e47700 1 --2- 172.21.15.3:0/3823098160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aac09de50 0x7f2aac11d8c0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:53.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.827+0000 7f2ab1e47700 1 --2- 172.21.15.3:0/3823098160 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2aac09e930 0x7f2aac11de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:53.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.827+0000 7f2ab1e47700 1 -- 172.21.15.3:0/3823098160 >> 172.21.15.3:0/3823098160 conn(0x7f2aac0a2aa0 msgr2=0x7f2aac09ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:53.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.827+0000 7f2ab1e47700 1 -- 172.21.15.3:0/3823098160 shutdown_connections 2023-10-03T10:16:53.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.827+0000 7f2ab1e47700 1 -- 172.21.15.3:0/3823098160 wait complete. 2023-10-03T10:16:53.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.839+0000 7f093224f700 1 Processor -- start 2023-10-03T10:16:53.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.839+0000 7f093224f700 1 -- start start 2023-10-03T10:16:53.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.839+0000 7f093224f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f092c06e570 0x7f092c06e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:53.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.839+0000 7f093224f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f092c06ee80 0x7f092c134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:53.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.840+0000 7f093224f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f092c070ee0 con 0x7f092c09eca0 2023-10-03T10:16:53.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.840+0000 7f093224f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092c071480 con 0x7f092c06ee80 2023-10-03T10:16:53.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.840+0000 7f093224f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f092c071b90 con 0x7f092c06e570 2023-10-03T10:16:53.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.840+0000 7f092bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f092c06ee80 0x7f092c134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:53.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.840+0000 7f092b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f092c06e570 0x7f092c06e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:53.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.840+0000 7f092bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f092c06ee80 0x7f092c134d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33192/0 (socket says 172.21.15.3:33192) 2023-10-03T10:16:53.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.840+0000 7f092bfff700 1 -- 172.21.15.3:0/1659641899 learned_addr learned my addr 172.21.15.3:0/1659641899 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:53.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.840+0000 7f092bfff700 1 -- 172.21.15.3:0/1659641899 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f092c06e570 msgr2=0x7f092c06e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:53.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.840+0000 7f092bfff700 1 --2- 172.21.15.3:0/1659641899 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f092c06e570 0x7f092c06e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:53.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.840+0000 7f092bfff700 1 -- 172.21.15.3:0/1659641899 >> v1:172.21.15.3:6789/0 conn(0x7f092c09eca0 legacy=0x7f092c134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:53.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.840+0000 7f092bfff700 1 -- 172.21.15.3:0/1659641899 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c0711a0 con 0x7f092c06ee80 2023-10-03T10:16:53.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.840+0000 7f092bfff700 1 --2- 172.21.15.3:0/1659641899 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f092c06ee80 0x7f092c134d20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0924009e80 tx=0x7f092400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7c32aaf7d45f582 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:53.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.841+0000 7f092a7fc700 1 -- 172.21.15.3:0/1659641899 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092400e070 con 0x7f092c06ee80 2023-10-03T10:16:53.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.841+0000 7f092a7fc700 1 -- 172.21.15.3:0/1659641899 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092400b910 con 0x7f092c06ee80 2023-10-03T10:16:53.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.841+0000 7f092a7fc700 1 -- 172.21.15.3:0/1659641899 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092400bcc0 con 0x7f092c06ee80 2023-10-03T10:16:53.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.842+0000 7f093224f700 1 -- 172.21.15.3:0/1659641899 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f092c06ee80 msgr2=0x7f092c134d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:53.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.842+0000 7f093224f700 1 --2- 172.21.15.3:0/1659641899 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f092c06ee80 0x7f092c134d20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0924009e80 tx=0x7f092400b330 comp rx=0 tx=0).stop 2023-10-03T10:16:53.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.842+0000 7f093224f700 1 -- 172.21.15.3:0/1659641899 shutdown_connections 2023-10-03T10:16:53.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.842+0000 7f093224f700 1 --2- 172.21.15.3:0/1659641899 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f092c06ee80 0x7f092c134d20 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:53.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.842+0000 7f093224f700 1 --2- 172.21.15.3:0/1659641899 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f092c06e570 0x7f092c06e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:53.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.842+0000 7f093224f700 1 -- 172.21.15.3:0/1659641899 >> 172.21.15.3:0/1659641899 conn(0x7f092c0a2aa0 msgr2=0x7f092c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:53.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.842+0000 7f093224f700 1 -- 172.21.15.3:0/1659641899 shutdown_connections 2023-10-03T10:16:53.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.842+0000 7f093224f700 1 -- 172.21.15.3:0/1659641899 wait complete. 2023-10-03T10:16:53.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.843+0000 7f093224f700 1 Processor -- start 2023-10-03T10:16:53.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.843+0000 7f093224f700 1 -- start start 2023-10-03T10:16:53.844 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:53.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.844+0000 7f093224f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f092c06e570 0x7f092c1214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:53.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.844+0000 7f093224f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f092c06ee80 0x7f092c121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:53.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.844+0000 7f093224f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f092c121f60 0x7f092c1cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:53.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.844+0000 7f093224f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c124a60 con 0x7f092c121f60 2023-10-03T10:16:53.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.844+0000 7f093224f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092c070610 con 0x7f092c06e570 2023-10-03T10:16:53.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.844+0000 7f093224f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f092c122f40 con 0x7f092c06ee80 2023-10-03T10:16:53.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.844+0000 7f092b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f092c06e570 0x7f092c1214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:53.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.844+0000 7f092b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f092c06e570 0x7f092c1214e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33204/0 (socket says 172.21.15.3:33204) 2023-10-03T10:16:53.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.844+0000 7f092b7fe700 1 -- 172.21.15.3:0/1357655934 learned_addr learned my addr 172.21.15.3:0/1357655934 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:53.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.844+0000 7f092bfff700 1 --2- 172.21.15.3:0/1357655934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f092c121f60 0x7f092c1cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:53.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.844+0000 7f092affd700 1 --2- 172.21.15.3:0/1357655934 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f092c06ee80 0x7f092c121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:53.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.844+0000 7f092b7fe700 1 -- 172.21.15.3:0/1357655934 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f092c06ee80 msgr2=0x7f092c121a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:53.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.844+0000 7f092b7fe700 1 --2- 172.21.15.3:0/1357655934 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f092c06ee80 0x7f092c121a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:53.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.844+0000 7f092b7fe700 1 -- 172.21.15.3:0/1357655934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f092c121f60 msgr2=0x7f092c1cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:53.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.844+0000 7f092b7fe700 1 --2- 172.21.15.3:0/1357655934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f092c121f60 0x7f092c1cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:53.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.844+0000 7f092b7fe700 1 -- 172.21.15.3:0/1357655934 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0924009b30 con 0x7f092c06e570 2023-10-03T10:16:53.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.844+0000 7f092bfff700 1 --2- 172.21.15.3:0/1357655934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f092c121f60 0x7f092c1cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:53.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.845+0000 7f092b7fe700 1 --2- 172.21.15.3:0/1357655934 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f092c06e570 0x7f092c1214e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f091c00e9b0 tx=0x7f091c00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:53.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.845+0000 7f0913fff700 1 -- 172.21.15.3:0/1357655934 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c009710 con 0x7f092c06e570 2023-10-03T10:16:53.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.845+0000 7f0913fff700 1 -- 172.21.15.3:0/1357655934 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091c00ee50 con 0x7f092c06e570 2023-10-03T10:16:53.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.845+0000 7f093224f700 1 -- 172.21.15.3:0/1357655934 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f092c1359c0 con 0x7f092c06e570 2023-10-03T10:16:53.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.845+0000 7f0913fff700 1 -- 172.21.15.3:0/1357655934 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c00ca50 con 0x7f092c06e570 2023-10-03T10:16:53.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.845+0000 7f093224f700 1 -- 172.21.15.3:0/1357655934 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f092c1254d0 con 0x7f092c06e570 2023-10-03T10:16:53.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.846+0000 7f0913fff700 1 -- 172.21.15.3:0/1357655934 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f091c00cbb0 con 0x7f092c06e570 2023-10-03T10:16:53.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.846+0000 7f0913fff700 1 -- 172.21.15.3:0/1357655934 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f091c03f790 con 0x7f092c06e570 2023-10-03T10:16:53.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.846+0000 7f093224f700 1 -- 172.21.15.3:0/1357655934 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f092c074ea0 con 0x7f092c06e570 2023-10-03T10:16:53.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.851+0000 7f0913fff700 1 -- 172.21.15.3:0/1357655934 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f091c021020 con 0x7f092c06e570 2023-10-03T10:16:53.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.890+0000 7f108ebd6700 1 Processor -- start 2023-10-03T10:16:53.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.891+0000 7f108ebd6700 1 -- start start 2023-10-03T10:16:53.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.891+0000 7f108ebd6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f10880a0a30 0x7f10880a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:53.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.891+0000 7f108ebd6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f108809d630 0x7f10880a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:53.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.891+0000 7f108ebd6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1088133930 0x7f1088133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:53.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.891+0000 7f108ebd6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10880a2a70 con 0x7f108809d630 2023-10-03T10:16:53.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.891+0000 7f108ebd6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10880a01d0 con 0x7f10880a0a30 2023-10-03T10:16:53.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.891+0000 7f108ebd6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f108809e2b0 con 0x7f1088133930 2023-10-03T10:16:53.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.891+0000 7f108d173700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1088133930 0x7f1088133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:53.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.891+0000 7f108d173700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1088133930 0x7f1088133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:35730/0 (socket says 172.21.15.3:35730) 2023-10-03T10:16:53.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.891+0000 7f108d173700 1 -- 172.21.15.3:0/935104183 learned_addr learned my addr 172.21.15.3:0/935104183 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:53.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.891+0000 7f108c972700 1 --2- 172.21.15.3:0/935104183 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f10880a0a30 0x7f10880a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:53.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.891+0000 7f107ffff700 1 --2- 172.21.15.3:0/935104183 >> v2:172.21.15.3:3300/0 conn(0x7f108809d630 0x7f10880a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:53.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.892+0000 7f108d173700 1 -- 172.21.15.3:0/935104183 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f10880a0a30 msgr2=0x7f10880a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:53.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.892+0000 7f108d173700 1 --2- 172.21.15.3:0/935104183 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f10880a0a30 0x7f10880a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:53.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.892+0000 7f108d173700 1 -- 172.21.15.3:0/935104183 >> v2:172.21.15.3:3300/0 conn(0x7f108809d630 msgr2=0x7f10880a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:53.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.892+0000 7f108d173700 1 --2- 172.21.15.3:0/935104183 >> v2:172.21.15.3:3300/0 conn(0x7f108809d630 0x7f10880a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:53.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.892+0000 7f108d173700 1 -- 172.21.15.3:0/935104183 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108809dc50 con 0x7f1088133930 2023-10-03T10:16:53.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.892+0000 7f108d173700 1 --2- 172.21.15.3:0/935104183 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1088133930 0x7f1088133dd0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1080009e50 tx=0x7f108000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e2b41897b0c077f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:53.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.892+0000 7f107f7fe700 1 -- 172.21.15.3:0/935104183 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108000e070 con 0x7f1088133930 2023-10-03T10:16:53.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.892+0000 7f107f7fe700 1 -- 172.21.15.3:0/935104183 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108000b910 con 0x7f1088133930 2023-10-03T10:16:53.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.892+0000 7f107f7fe700 1 -- 172.21.15.3:0/935104183 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108000bcc0 con 0x7f1088133930 2023-10-03T10:16:53.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.893+0000 7f108ebd6700 1 -- 172.21.15.3:0/935104183 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1088133930 msgr2=0x7f1088133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:53.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.893+0000 7f108ebd6700 1 --2- 172.21.15.3:0/935104183 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1088133930 0x7f1088133dd0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1080009e50 tx=0x7f108000b330 comp rx=0 tx=0).stop 2023-10-03T10:16:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.893+0000 7f108ebd6700 1 -- 172.21.15.3:0/935104183 shutdown_connections 2023-10-03T10:16:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.893+0000 7f108ebd6700 1 --2- 172.21.15.3:0/935104183 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1088133930 0x7f1088133dd0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.893+0000 7f108ebd6700 1 --2- 172.21.15.3:0/935104183 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f10880a0a30 0x7f10880a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.893+0000 7f108ebd6700 1 --2- 172.21.15.3:0/935104183 >> v2:172.21.15.3:3300/0 conn(0x7f108809d630 0x7f10880a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.893+0000 7f108ebd6700 1 -- 172.21.15.3:0/935104183 >> 172.21.15.3:0/935104183 conn(0x7f10880a3050 msgr2=0x7f10880a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.893+0000 7f108ebd6700 1 -- 172.21.15.3:0/935104183 shutdown_connections 2023-10-03T10:16:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.893+0000 7f108ebd6700 1 -- 172.21.15.3:0/935104183 wait complete. 2023-10-03T10:16:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.894+0000 7f108ebd6700 1 Processor -- start 2023-10-03T10:16:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.894+0000 7f108ebd6700 1 -- start start 2023-10-03T10:16:53.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.895+0000 7f108ebd6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f108809d630 0x7f1088132b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:53.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.895+0000 7f108ebd6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f10880a0a30 0x7f1088133040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:53.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.895+0000 7f108ebd6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1088133930 0x7f10881c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:53.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.895+0000 7f108ebd6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10881225b0 con 0x7f1088133930 2023-10-03T10:16:53.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.895+0000 7f108ebd6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10880a0410 con 0x7f108809d630 2023-10-03T10:16:53.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.895+0000 7f108ebd6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f10881208a0 con 0x7f10880a0a30 2023-10-03T10:16:53.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.895+0000 7f108c972700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f108809d630 0x7f1088132b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:53.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.895+0000 7f108d173700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1088133930 0x7f10881c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:53.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.895+0000 7f108d173700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1088133930 0x7f10881c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48748/0 (socket says 172.21.15.3:48748) 2023-10-03T10:16:53.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.895+0000 7f108c972700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f108809d630 0x7f1088132b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33210/0 (socket says 172.21.15.3:33210) 2023-10-03T10:16:53.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.895+0000 7f108d173700 1 -- 172.21.15.3:0/4231461199 learned_addr learned my addr 172.21.15.3:0/4231461199 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:53.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.895+0000 7f107ffff700 1 --2- 172.21.15.3:0/4231461199 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f10880a0a30 0x7f1088133040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.895+0000 7f108c972700 1 -- 172.21.15.3:0/4231461199 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f10880a0a30 msgr2=0x7f1088133040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.895+0000 7f108c972700 1 --2- 172.21.15.3:0/4231461199 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f10880a0a30 0x7f1088133040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.895+0000 7f108c972700 1 -- 172.21.15.3:0/4231461199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1088133930 msgr2=0x7f10881c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.895+0000 7f108c972700 1 --2- 172.21.15.3:0/4231461199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1088133930 0x7f10881c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.895+0000 7f108c972700 1 -- 172.21.15.3:0/4231461199 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1080009b30 con 0x7f108809d630 2023-10-03T10:16:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.895+0000 7f108d173700 1 --2- 172.21.15.3:0/4231461199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1088133930 0x7f10881c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.896+0000 7f108c972700 1 --2- 172.21.15.3:0/4231461199 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f108809d630 0x7f1088132b00 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f108400e900 tx=0x7f108400ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.896+0000 7f107d7fa700 1 -- 172.21.15.3:0/4231461199 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1084013680 con 0x7f108809d630 2023-10-03T10:16:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.896+0000 7f107d7fa700 1 -- 172.21.15.3:0/4231461199 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10840137e0 con 0x7f108809d630 2023-10-03T10:16:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.896+0000 7f108ebd6700 1 -- 172.21.15.3:0/4231461199 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10880a4030 con 0x7f108809d630 2023-10-03T10:16:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.896+0000 7f107d7fa700 1 -- 172.21.15.3:0/4231461199 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1084013b50 con 0x7f108809d630 2023-10-03T10:16:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.896+0000 7f108ebd6700 1 -- 172.21.15.3:0/4231461199 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1088133580 con 0x7f108809d630 2023-10-03T10:16:53.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.897+0000 7f107d7fa700 1 -- 172.21.15.3:0/4231461199 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1084013cb0 con 0x7f108809d630 2023-10-03T10:16:53.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.897+0000 7f108ebd6700 1 -- 172.21.15.3:0/4231461199 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10880747f0 con 0x7f108809d630 2023-10-03T10:16:53.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.897+0000 7f107d7fa700 1 -- 172.21.15.3:0/4231461199 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f108403f8e0 con 0x7f108809d630 2023-10-03T10:16:53.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:53.902+0000 7f107d7fa700 1 -- 172.21.15.3:0/4231461199 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f108401b7c0 con 0x7f108809d630 2023-10-03T10:16:54.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.028+0000 7f3fff688700 1 -- 172.21.15.3:0/2427446140 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3ff8074ea0 con 0x7f3ff806a770 2023-10-03T10:16:54.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.028+0000 7f3fedffb700 1 -- 172.21.15.3:0/2427446140 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3ff4021070 con 0x7f3ff806a770 2023-10-03T10:16:54.028 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:54.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.030+0000 7f3fff688700 1 -- 172.21.15.3:0/2427446140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ff806a770 msgr2=0x7f3ff811bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:54.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.030+0000 7f3fff688700 1 --2- 172.21.15.3:0/2427446140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ff806a770 0x7f3ff811bea0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400ec10 tx=0x7f3ff400ef20 comp rx=0 tx=0).stop 2023-10-03T10:16:54.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.030+0000 7f3fff688700 1 -- 172.21.15.3:0/2427446140 shutdown_connections 2023-10-03T10:16:54.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.030+0000 7f3fff688700 1 --2- 172.21.15.3:0/2427446140 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ff8131b40 0x7f3ff811c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:54.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.030+0000 7f3fff688700 1 --2- 172.21.15.3:0/2427446140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ff806a770 0x7f3ff811bea0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:54.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.030+0000 7f3fff688700 1 --2- 172.21.15.3:0/2427446140 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ff806b080 0x7f3ff811c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:54.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.030+0000 7f3fff688700 1 -- 172.21.15.3:0/2427446140 >> 172.21.15.3:0/2427446140 conn(0x7f3ff8066f50 msgr2=0x7f3ff8130d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:54.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.031+0000 7f3fff688700 1 -- 172.21.15.3:0/2427446140 shutdown_connections 2023-10-03T10:16:54.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.031+0000 7f3fff688700 1 -- 172.21.15.3:0/2427446140 wait complete. 2023-10-03T10:16:54.045 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:54.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.060+0000 7f093224f700 1 -- 172.21.15.3:0/1357655934 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f092c06b9e0 con 0x7f092c06e570 2023-10-03T10:16:54.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.060+0000 7f0913fff700 1 -- 172.21.15.3:0/1357655934 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f091c01be10 con 0x7f092c06e570 2023-10-03T10:16:54.060 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:54.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.063+0000 7f093224f700 1 -- 172.21.15.3:0/1357655934 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f092c06e570 msgr2=0x7f092c1214e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:54.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.063+0000 7f093224f700 1 --2- 172.21.15.3:0/1357655934 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f092c06e570 0x7f092c1214e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f091c00e9b0 tx=0x7f091c00ecc0 comp rx=0 tx=0).stop 2023-10-03T10:16:54.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.063+0000 7f093224f700 1 -- 172.21.15.3:0/1357655934 shutdown_connections 2023-10-03T10:16:54.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.063+0000 7f093224f700 1 --2- 172.21.15.3:0/1357655934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f092c121f60 0x7f092c1cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:54.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.063+0000 7f093224f700 1 --2- 172.21.15.3:0/1357655934 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f092c06e570 0x7f092c1214e0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:54.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.063+0000 7f093224f700 1 --2- 172.21.15.3:0/1357655934 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f092c06ee80 0x7f092c121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:54.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.063+0000 7f093224f700 1 -- 172.21.15.3:0/1357655934 >> 172.21.15.3:0/1357655934 conn(0x7f092c0a2aa0 msgr2=0x7f092c0a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:54.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.063+0000 7f093224f700 1 -- 172.21.15.3:0/1357655934 shutdown_connections 2023-10-03T10:16:54.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.063+0000 7f093224f700 1 -- 172.21.15.3:0/1357655934 wait complete. 2023-10-03T10:16:54.077 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:54.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.109+0000 7f108ebd6700 1 -- 172.21.15.3:0/4231461199 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1088072980 con 0x7f108809d630 2023-10-03T10:16:54.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.110+0000 7f107d7fa700 1 -- 172.21.15.3:0/4231461199 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1084017030 con 0x7f108809d630 2023-10-03T10:16:54.110 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:54.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.112+0000 7f108ebd6700 1 -- 172.21.15.3:0/4231461199 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f108809d630 msgr2=0x7f1088132b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:54.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.112+0000 7f108ebd6700 1 --2- 172.21.15.3:0/4231461199 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f108809d630 0x7f1088132b00 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f108400e900 tx=0x7f108400ec10 comp rx=0 tx=0).stop 2023-10-03T10:16:54.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.112+0000 7f108ebd6700 1 -- 172.21.15.3:0/4231461199 shutdown_connections 2023-10-03T10:16:54.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.112+0000 7f108ebd6700 1 --2- 172.21.15.3:0/4231461199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1088133930 0x7f10881c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:54.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.112+0000 7f108ebd6700 1 --2- 172.21.15.3:0/4231461199 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f108809d630 0x7f1088132b00 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:54.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.112+0000 7f108ebd6700 1 --2- 172.21.15.3:0/4231461199 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f10880a0a30 0x7f1088133040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:54.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.112+0000 7f108ebd6700 1 -- 172.21.15.3:0/4231461199 >> 172.21.15.3:0/4231461199 conn(0x7f10880a3050 msgr2=0x7f108809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:54.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.112+0000 7f108ebd6700 1 -- 172.21.15.3:0/4231461199 shutdown_connections 2023-10-03T10:16:54.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.112+0000 7f108ebd6700 1 -- 172.21.15.3:0/4231461199 wait complete. 2023-10-03T10:16:54.126 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:54.845 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:54.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.992+0000 7f54b28d0700 1 Processor -- start 2023-10-03T10:16:54.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.992+0000 7f54b28d0700 1 -- start start 2023-10-03T10:16:54.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.993+0000 7f54b28d0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f54ac09d6f0 0x7f54ac09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:54.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.993+0000 7f54b28d0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f54ac09e000 0x7f54ac130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:54.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.993+0000 7f54b28d0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f54ac09e410 0x7f54ac130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:54.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.993+0000 7f54b28d0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54ac09f520 con 0x7f54ac09d6f0 2023-10-03T10:16:54.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.993+0000 7f54b28d0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac09f130 con 0x7f54ac09e410 2023-10-03T10:16:54.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.993+0000 7f54b28d0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac09e930 con 0x7f54ac09e000 2023-10-03T10:16:54.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.993+0000 7f54b0e6d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f54ac09e410 0x7f54ac130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:54.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.993+0000 7f54ab7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f54ac09e000 0x7f54ac130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:54.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.993+0000 7f54b0e6d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f54ac09e410 0x7f54ac130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33212/0 (socket says 172.21.15.3:33212) 2023-10-03T10:16:54.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.993+0000 7f54abfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f54ac09d6f0 0x7f54ac09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:54.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.993+0000 7f54ab7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f54ac09e000 0x7f54ac130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:35746/0 (socket says 172.21.15.3:35746) 2023-10-03T10:16:54.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.993+0000 7f54b0e6d700 1 -- 172.21.15.3:0/1912535215 learned_addr learned my addr 172.21.15.3:0/1912535215 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:54.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.993+0000 7f54abfff700 1 -- 172.21.15.3:0/1912535215 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f54ac09e000 msgr2=0x7f54ac130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:54.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.994+0000 7f54abfff700 1 --2- 172.21.15.3:0/1912535215 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f54ac09e000 0x7f54ac130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:54.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.994+0000 7f54abfff700 1 -- 172.21.15.3:0/1912535215 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f54ac09e410 msgr2=0x7f54ac130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:54.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.994+0000 7f54abfff700 1 --2- 172.21.15.3:0/1912535215 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f54ac09e410 0x7f54ac130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:54.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.994+0000 7f54abfff700 1 -- 172.21.15.3:0/1912535215 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ac09b2d0 con 0x7f54ac09d6f0 2023-10-03T10:16:54.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.994+0000 7f54b0e6d700 1 --2- 172.21.15.3:0/1912535215 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f54ac09e410 0x7f54ac130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:54.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.994+0000 7f54ab7fe700 1 --2- 172.21.15.3:0/1912535215 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f54ac09e000 0x7f54ac130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:54.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.994+0000 7f54abfff700 1 --2- 172.21.15.3:0/1912535215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54ac09d6f0 0x7f54ac09dac0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f549c009870 tx=0x7f549c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd5446fa1ae3a5a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:54.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.994+0000 7f54aaffd700 1 -- 172.21.15.3:0/1912535215 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c011830 con 0x7f54ac09d6f0 2023-10-03T10:16:54.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.994+0000 7f54aaffd700 1 -- 172.21.15.3:0/1912535215 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549c011990 con 0x7f54ac09d6f0 2023-10-03T10:16:54.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.994+0000 7f54aaffd700 1 -- 172.21.15.3:0/1912535215 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c011d20 con 0x7f54ac09d6f0 2023-10-03T10:16:54.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.994+0000 7f54b28d0700 1 -- 172.21.15.3:0/1912535215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54ac09d6f0 msgr2=0x7f54ac09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:54.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.994+0000 7f54b28d0700 1 --2- 172.21.15.3:0/1912535215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54ac09d6f0 0x7f54ac09dac0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f549c009870 tx=0x7f549c009b80 comp rx=0 tx=0).stop 2023-10-03T10:16:54.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.995+0000 7f54b28d0700 1 -- 172.21.15.3:0/1912535215 shutdown_connections 2023-10-03T10:16:54.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.995+0000 7f54b28d0700 1 --2- 172.21.15.3:0/1912535215 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f54ac09e410 0x7f54ac130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:54.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.995+0000 7f54b28d0700 1 --2- 172.21.15.3:0/1912535215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54ac09d6f0 0x7f54ac09dac0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:54.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.995+0000 7f54b28d0700 1 --2- 172.21.15.3:0/1912535215 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f54ac09e000 0x7f54ac130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:54.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.995+0000 7f54b28d0700 1 -- 172.21.15.3:0/1912535215 >> 172.21.15.3:0/1912535215 conn(0x7f54ac066f50 msgr2=0x7f54ac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:54.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.995+0000 7f54b28d0700 1 -- 172.21.15.3:0/1912535215 shutdown_connections 2023-10-03T10:16:54.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.995+0000 7f54b28d0700 1 -- 172.21.15.3:0/1912535215 wait complete. 2023-10-03T10:16:55.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.996+0000 7f54b28d0700 1 Processor -- start 2023-10-03T10:16:55.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.996+0000 7f54b28d0700 1 -- start start 2023-10-03T10:16:55.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.996+0000 7f54b28d0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54ac09d6f0 0x7f54ac11df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:55.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.996+0000 7f54b28d0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f54ac09e000 0x7f54ac11e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:55.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.996+0000 7f54b28d0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f54ac09e410 0x7f54ac11e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:55.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.996+0000 7f54b28d0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac130300 con 0x7f54ac09d6f0 2023-10-03T10:16:55.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.996+0000 7f54b28d0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac120150 con 0x7f54ac09e000 2023-10-03T10:16:55.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.996+0000 7f54b28d0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac11f380 con 0x7f54ac09e410 2023-10-03T10:16:55.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.996+0000 7f54abfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54ac09d6f0 0x7f54ac11df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:55.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.996+0000 7f54ab7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f54ac09e000 0x7f54ac11e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:55.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.996+0000 7f54b0e6d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f54ac09e410 0x7f54ac11e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:55.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.996+0000 7f54abfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54ac09d6f0 0x7f54ac11df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48770/0 (socket says 172.21.15.3:48770) 2023-10-03T10:16:55.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.996+0000 7f54abfff700 1 -- 172.21.15.3:0/3200390120 learned_addr learned my addr 172.21.15.3:0/3200390120 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:55.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.996+0000 7f54ab7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f54ac09e000 0x7f54ac11e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33222/0 (socket says 172.21.15.3:33222) 2023-10-03T10:16:55.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.997+0000 7f54abfff700 1 -- 172.21.15.3:0/3200390120 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f54ac09e410 msgr2=0x7f54ac11e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:55.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.997+0000 7f54abfff700 1 --2- 172.21.15.3:0/3200390120 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f54ac09e410 0x7f54ac11e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.997+0000 7f54abfff700 1 -- 172.21.15.3:0/3200390120 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f54ac09e000 msgr2=0x7f54ac11e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:55.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.997+0000 7f54abfff700 1 --2- 172.21.15.3:0/3200390120 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f54ac09e000 0x7f54ac11e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.997+0000 7f54abfff700 1 -- 172.21.15.3:0/3200390120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a000d940 con 0x7f54ac09d6f0 2023-10-03T10:16:55.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.997+0000 7f54ab7fe700 1 --2- 172.21.15.3:0/3200390120 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f54ac09e000 0x7f54ac11e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:55.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.997+0000 7f54abfff700 1 --2- 172.21.15.3:0/3200390120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54ac09d6f0 0x7f54ac11df20 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f549c000c00 tx=0x7f549c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:55.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.997+0000 7f54a8ff9700 1 -- 172.21.15.3:0/3200390120 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c004450 con 0x7f54ac09d6f0 2023-10-03T10:16:55.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.997+0000 7f54a8ff9700 1 -- 172.21.15.3:0/3200390120 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549c01f070 con 0x7f54ac09d6f0 2023-10-03T10:16:55.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.997+0000 7f54a8ff9700 1 -- 172.21.15.3:0/3200390120 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c0038c0 con 0x7f54ac09d6f0 2023-10-03T10:16:55.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.997+0000 7f54b28d0700 1 -- 172.21.15.3:0/3200390120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a400bb50 con 0x7f54ac09d6f0 2023-10-03T10:16:55.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.997+0000 7f54b28d0700 1 -- 172.21.15.3:0/3200390120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549c0094a0 con 0x7f54ac09d6f0 2023-10-03T10:16:55.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.998+0000 7f54a8ff9700 1 -- 172.21.15.3:0/3200390120 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f549c003a20 con 0x7f54ac09d6f0 2023-10-03T10:16:55.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.998+0000 7f54a8ff9700 1 -- 172.21.15.3:0/3200390120 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f549c021070 con 0x7f54ac09d6f0 2023-10-03T10:16:55.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:54.998+0000 7f54b28d0700 1 -- 172.21.15.3:0/3200390120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54ac0a1760 con 0x7f54ac09d6f0 2023-10-03T10:16:55.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.003+0000 7f54a8ff9700 1 -- 172.21.15.3:0/3200390120 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f549c02b080 con 0x7f54ac09d6f0 2023-10-03T10:16:55.046 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:55.077 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:55.127 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:55.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.190+0000 7fd1f1429700 1 Processor -- start 2023-10-03T10:16:55.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.190+0000 7fd1f1429700 1 -- start start 2023-10-03T10:16:55.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.190+0000 7fd1f1429700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd1ec06a770 0x7fd1ec06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:55.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.190+0000 7fd1f1429700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd1ec06b080 0x7fd1ec131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:55.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.190+0000 7fd1f1429700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ec09ece0 con 0x7fd1ec131b40 2023-10-03T10:16:55.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.190+0000 7fd1f1429700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ec09e760 con 0x7fd1ec06a770 2023-10-03T10:16:55.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.190+0000 7fd1f1429700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec09e040 con 0x7fd1ec06b080 2023-10-03T10:16:55.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.191+0000 7fd1eaffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd1ec06a770 0x7fd1ec06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:55.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.191+0000 7fd1eaffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd1ec06a770 0x7fd1ec06ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48774/0 (socket says 172.21.15.3:48774) 2023-10-03T10:16:55.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.191+0000 7fd1eaffd700 1 -- 172.21.15.3:0/4100190401 learned_addr learned my addr 172.21.15.3:0/4100190401 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:55.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.191+0000 7fd1ea7fc700 1 --2- 172.21.15.3:0/4100190401 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd1ec06b080 0x7fd1ec131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:55.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.191+0000 7fd1eaffd700 1 -- 172.21.15.3:0/4100190401 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd1ec06b080 msgr2=0x7fd1ec131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:55.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.191+0000 7fd1eaffd700 1 --2- 172.21.15.3:0/4100190401 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd1ec06b080 0x7fd1ec131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.191+0000 7fd1eaffd700 1 -- 172.21.15.3:0/4100190401 >> v1:172.21.15.3:6789/0 conn(0x7fd1ec131b40 legacy=0x7fd1ec138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:55.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.191+0000 7fd1eaffd700 1 -- 172.21.15.3:0/4100190401 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec09f630 con 0x7fd1ec06a770 2023-10-03T10:16:55.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.191+0000 7fd1eaffd700 1 --2- 172.21.15.3:0/4100190401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1ec06a770 0x7fd1ec06ab40 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0098f0 tx=0x7fd1dc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6444dd14b90b1485 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:55.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.192+0000 7fd1e9ffb700 1 -- 172.21.15.3:0/4100190401 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc011830 con 0x7fd1ec06a770 2023-10-03T10:16:55.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.192+0000 7fd1e9ffb700 1 -- 172.21.15.3:0/4100190401 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1dc011990 con 0x7fd1ec06a770 2023-10-03T10:16:55.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.192+0000 7fd1e9ffb700 1 -- 172.21.15.3:0/4100190401 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc011d20 con 0x7fd1ec06a770 2023-10-03T10:16:55.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.193+0000 7fd1f1429700 1 -- 172.21.15.3:0/4100190401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1ec06a770 msgr2=0x7fd1ec06ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:55.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.193+0000 7fd1f1429700 1 --2- 172.21.15.3:0/4100190401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1ec06a770 0x7fd1ec06ab40 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0098f0 tx=0x7fd1dc009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:55.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.193+0000 7fd1f1429700 1 -- 172.21.15.3:0/4100190401 shutdown_connections 2023-10-03T10:16:55.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.193+0000 7fd1f1429700 1 --2- 172.21.15.3:0/4100190401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1ec06a770 0x7fd1ec06ab40 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.193+0000 7fd1f1429700 1 --2- 172.21.15.3:0/4100190401 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd1ec06b080 0x7fd1ec131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.193+0000 7fd1f1429700 1 -- 172.21.15.3:0/4100190401 >> 172.21.15.3:0/4100190401 conn(0x7fd1ec066f50 msgr2=0x7fd1ec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:55.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.193+0000 7fd1f1429700 1 -- 172.21.15.3:0/4100190401 shutdown_connections 2023-10-03T10:16:55.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.193+0000 7fd1f1429700 1 -- 172.21.15.3:0/4100190401 wait complete. 2023-10-03T10:16:55.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.194+0000 7fd1f1429700 1 Processor -- start 2023-10-03T10:16:55.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.194+0000 7fd1f1429700 1 -- start start 2023-10-03T10:16:55.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.194+0000 7fd1f1429700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd1ec06a770 0x7fd1ec11c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:55.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.195+0000 7fd1f1429700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1ec06b080 0x7fd1ec11c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:55.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.195+0000 7fd1f1429700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1ec131b40 0x7fd1ec11cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:55.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.195+0000 7fd1f1429700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec071f10 con 0x7fd1ec06b080 2023-10-03T10:16:55.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.195+0000 7fd1f1429700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec131040 con 0x7fd1ec131b40 2023-10-03T10:16:55.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.195+0000 7fd1f1429700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec11fa10 con 0x7fd1ec06a770 2023-10-03T10:16:55.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.195+0000 7fd1eaffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd1ec06a770 0x7fd1ec11c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:55.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.195+0000 7fd1eaffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd1ec06a770 0x7fd1ec11c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:35782/0 (socket says 172.21.15.3:35782) 2023-10-03T10:16:55.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.195+0000 7fd1eaffd700 1 -- 172.21.15.3:0/4217110089 learned_addr learned my addr 172.21.15.3:0/4217110089 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:55.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.195+0000 7fd1ea7fc700 1 --2- 172.21.15.3:0/4217110089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1ec06b080 0x7fd1ec11c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:55.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.195+0000 7fd1eaffd700 1 -- 172.21.15.3:0/4217110089 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1ec131b40 msgr2=0x7fd1ec11cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:55.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.195+0000 7fd1eaffd700 1 --2- 172.21.15.3:0/4217110089 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1ec131b40 0x7fd1ec11cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.195+0000 7fd1eaffd700 1 -- 172.21.15.3:0/4217110089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1ec06b080 msgr2=0x7fd1ec11c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:55.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.195+0000 7fd1eaffd700 1 --2- 172.21.15.3:0/4217110089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1ec06b080 0x7fd1ec11c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.195+0000 7fd1eaffd700 1 -- 172.21.15.3:0/4217110089 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1dc009550 con 0x7fd1ec06a770 2023-10-03T10:16:55.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.195+0000 7fd1eaffd700 1 --2- 172.21.15.3:0/4217110089 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd1ec06a770 0x7fd1ec11c490 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009c40 tx=0x7fd1dc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:55.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.196+0000 7fd1d37fe700 1 -- 172.21.15.3:0/4217110089 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc004480 con 0x7fd1ec06a770 2023-10-03T10:16:55.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.196+0000 7fd1d37fe700 1 -- 172.21.15.3:0/4217110089 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1dc01f070 con 0x7fd1ec06a770 2023-10-03T10:16:55.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.196+0000 7fd1f1429700 1 -- 172.21.15.3:0/4217110089 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ec131510 con 0x7fd1ec06a770 2023-10-03T10:16:55.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.196+0000 7fd1d37fe700 1 -- 172.21.15.3:0/4217110089 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc0038c0 con 0x7fd1ec06a770 2023-10-03T10:16:55.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.197+0000 7fd1d37fe700 1 -- 172.21.15.3:0/4217110089 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd1dc021030 con 0x7fd1ec06a770 2023-10-03T10:16:55.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.197+0000 7fd1f1429700 1 -- 172.21.15.3:0/4217110089 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ec11e0a0 con 0x7fd1ec06a770 2023-10-03T10:16:55.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.197+0000 7fd1d37fe700 1 -- 172.21.15.3:0/4217110089 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd1dc019cb0 con 0x7fd1ec06a770 2023-10-03T10:16:55.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.198+0000 7fd1f1429700 1 -- 172.21.15.3:0/4217110089 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ec11fc30 con 0x7fd1ec06a770 2023-10-03T10:16:55.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.207+0000 7fd1d37fe700 1 -- 172.21.15.3:0/4217110089 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd1dc01a7d0 con 0x7fd1ec06a770 2023-10-03T10:16:55.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.227+0000 7f54b28d0700 1 -- 172.21.15.3:0/3200390120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f54ac09c700 con 0x7f54ac09d6f0 2023-10-03T10:16:55.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.227+0000 7f54a8ff9700 1 -- 172.21.15.3:0/3200390120 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f549c015020 con 0x7f54ac09d6f0 2023-10-03T10:16:55.228 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:55.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.230+0000 7f54b28d0700 1 -- 172.21.15.3:0/3200390120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54ac09d6f0 msgr2=0x7f54ac11df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:55.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.230+0000 7f54b28d0700 1 --2- 172.21.15.3:0/3200390120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54ac09d6f0 0x7f54ac11df20 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f549c000c00 tx=0x7f549c004750 comp rx=0 tx=0).stop 2023-10-03T10:16:55.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.230+0000 7f54b28d0700 1 -- 172.21.15.3:0/3200390120 shutdown_connections 2023-10-03T10:16:55.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.230+0000 7f54b28d0700 1 --2- 172.21.15.3:0/3200390120 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f54ac09e410 0x7f54ac11e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.230+0000 7f54b28d0700 1 --2- 172.21.15.3:0/3200390120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54ac09d6f0 0x7f54ac11df20 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.230+0000 7f54b28d0700 1 --2- 172.21.15.3:0/3200390120 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f54ac09e000 0x7f54ac11e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.230+0000 7f54b28d0700 1 -- 172.21.15.3:0/3200390120 >> 172.21.15.3:0/3200390120 conn(0x7f54ac066f50 msgr2=0x7f54ac06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:55.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.230+0000 7f54b28d0700 1 -- 172.21.15.3:0/3200390120 shutdown_connections 2023-10-03T10:16:55.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.230+0000 7f54b28d0700 1 -- 172.21.15.3:0/3200390120 wait complete. 2023-10-03T10:16:55.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.243+0000 7ff0011cc700 1 Processor -- start 2023-10-03T10:16:55.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.243+0000 7ff0011cc700 1 -- start start 2023-10-03T10:16:55.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.244+0000 7ff0011cc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feffc06b250 0x7feffc136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:55.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.244+0000 7ff0011cc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feffc136d80 0x7feffc139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:55.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.244+0000 7ff0011cc700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feffc09ece0 con 0x7feffc06a770 2023-10-03T10:16:55.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.244+0000 7ff0011cc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feffc09e760 con 0x7feffc06b250 2023-10-03T10:16:55.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.244+0000 7ff0011cc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feffc09e040 con 0x7feffc136d80 2023-10-03T10:16:55.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.244+0000 7feffb59e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feffc136d80 0x7feffc139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:55.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.244+0000 7feffb59e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feffc136d80 0x7feffc139160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:35790/0 (socket says 172.21.15.3:35790) 2023-10-03T10:16:55.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.244+0000 7feffb59e700 1 -- 172.21.15.3:0/672069044 learned_addr learned my addr 172.21.15.3:0/672069044 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:55.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.245+0000 7feffa59c700 1 --2- 172.21.15.3:0/672069044 >> v2:172.21.15.3:3300/0 conn(0x7feffc06b250 0x7feffc136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:55.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.245+0000 7feffb59e700 1 -- 172.21.15.3:0/672069044 >> v2:172.21.15.3:3300/0 conn(0x7feffc06b250 msgr2=0x7feffc136840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:55.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.245+0000 7feffb59e700 1 --2- 172.21.15.3:0/672069044 >> v2:172.21.15.3:3300/0 conn(0x7feffc06b250 0x7feffc136840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.245+0000 7feffb59e700 1 -- 172.21.15.3:0/672069044 >> v1:172.21.15.3:6789/0 conn(0x7feffc06a770 legacy=0x7feffc06ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:55.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.245+0000 7feffb59e700 1 -- 172.21.15.3:0/672069044 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feffc09f630 con 0x7feffc136d80 2023-10-03T10:16:55.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.245+0000 7feffb59e700 1 --2- 172.21.15.3:0/672069044 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feffc136d80 0x7feffc139160 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7feff4009e50 tx=0x7feff400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e453995447a40a9f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:55.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.245+0000 7feff9d9b700 1 -- 172.21.15.3:0/672069044 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff400e070 con 0x7feffc136d80 2023-10-03T10:16:55.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.245+0000 7feff9d9b700 1 -- 172.21.15.3:0/672069044 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff400b910 con 0x7feffc136d80 2023-10-03T10:16:55.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.245+0000 7feff9d9b700 1 -- 172.21.15.3:0/672069044 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff400bcc0 con 0x7feffc136d80 2023-10-03T10:16:55.247 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:55.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.246+0000 7ff0011cc700 1 -- 172.21.15.3:0/672069044 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feffc136d80 msgr2=0x7feffc139160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:55.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.246+0000 7ff0011cc700 1 --2- 172.21.15.3:0/672069044 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feffc136d80 0x7feffc139160 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7feff4009e50 tx=0x7feff400b330 comp rx=0 tx=0).stop 2023-10-03T10:16:55.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.246+0000 7ff0011cc700 1 -- 172.21.15.3:0/672069044 shutdown_connections 2023-10-03T10:16:55.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.246+0000 7ff0011cc700 1 --2- 172.21.15.3:0/672069044 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feffc136d80 0x7feffc139160 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.246+0000 7ff0011cc700 1 --2- 172.21.15.3:0/672069044 >> v2:172.21.15.3:3300/0 conn(0x7feffc06b250 0x7feffc136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.246+0000 7ff0011cc700 1 -- 172.21.15.3:0/672069044 >> 172.21.15.3:0/672069044 conn(0x7feffc066f50 msgr2=0x7feffc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:55.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.246+0000 7ff0011cc700 1 -- 172.21.15.3:0/672069044 shutdown_connections 2023-10-03T10:16:55.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.246+0000 7ff0011cc700 1 -- 172.21.15.3:0/672069044 wait complete. 2023-10-03T10:16:55.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.248+0000 7ff0011cc700 1 Processor -- start 2023-10-03T10:16:55.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.248+0000 7ff0011cc700 1 -- start start 2023-10-03T10:16:55.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.249+0000 7ff0011cc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feffc06a770 0x7feffc09ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:55.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.249+0000 7ff0011cc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feffc06b250 0x7feffc09b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:55.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.249+0000 7ff0011cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feffc136d80 0x7feffc1c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:55.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.249+0000 7ff0011cc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feffc09bef0 con 0x7feffc06b250 2023-10-03T10:16:55.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.249+0000 7ff0011cc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feffc122150 con 0x7feffc136d80 2023-10-03T10:16:55.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.249+0000 7ff0011cc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feffc1223c0 con 0x7feffc06a770 2023-10-03T10:16:55.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.249+0000 7feffa59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feffc06b250 0x7feffc09b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:55.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.249+0000 7feffa59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feffc06b250 0x7feffc09b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48784/0 (socket says 172.21.15.3:48784) 2023-10-03T10:16:55.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.249+0000 7feffa59c700 1 -- 172.21.15.3:0/3506919356 learned_addr learned my addr 172.21.15.3:0/3506919356 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:55.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.249+0000 7feffb59e700 1 --2- 172.21.15.3:0/3506919356 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feffc136d80 0x7feffc1c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:55.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.249+0000 7feffad9d700 1 --2- 172.21.15.3:0/3506919356 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feffc06a770 0x7feffc09ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:55.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.250+0000 7feffa59c700 1 -- 172.21.15.3:0/3506919356 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feffc06a770 msgr2=0x7feffc09ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:55.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.250+0000 7feffa59c700 1 --2- 172.21.15.3:0/3506919356 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feffc06a770 0x7feffc09ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.250+0000 7feffa59c700 1 -- 172.21.15.3:0/3506919356 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feffc136d80 msgr2=0x7feffc1c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:55.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.250+0000 7feffa59c700 1 --2- 172.21.15.3:0/3506919356 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feffc136d80 0x7feffc1c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.250+0000 7feffa59c700 1 -- 172.21.15.3:0/3506919356 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff4009b30 con 0x7feffc06b250 2023-10-03T10:16:55.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.250+0000 7feffad9d700 1 --2- 172.21.15.3:0/3506919356 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feffc06a770 0x7feffc09ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:55.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.250+0000 7feffb59e700 1 --2- 172.21.15.3:0/3506919356 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feffc136d80 0x7feffc1c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:55.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.250+0000 7feffa59c700 1 --2- 172.21.15.3:0/3506919356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feffc06b250 0x7feffc09b0c0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7feff000e9b0 tx=0x7feff000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:55.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.250+0000 7fefe37fe700 1 -- 172.21.15.3:0/3506919356 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff0009710 con 0x7feffc06b250 2023-10-03T10:16:55.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.250+0000 7fefe37fe700 1 -- 172.21.15.3:0/3506919356 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff000ee50 con 0x7feffc06b250 2023-10-03T10:16:55.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.250+0000 7fefe37fe700 1 -- 172.21.15.3:0/3506919356 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff000ca50 con 0x7feffc06b250 2023-10-03T10:16:55.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.250+0000 7ff0011cc700 1 -- 172.21.15.3:0/3506919356 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feffc06e8e0 con 0x7feffc06b250 2023-10-03T10:16:55.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.250+0000 7ff0011cc700 1 -- 172.21.15.3:0/3506919356 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feffc122b50 con 0x7feffc06b250 2023-10-03T10:16:55.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.251+0000 7fefe37fe700 1 -- 172.21.15.3:0/3506919356 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feff000cbb0 con 0x7feffc06b250 2023-10-03T10:16:55.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.252+0000 7fefe37fe700 1 -- 172.21.15.3:0/3506919356 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7feff003fad0 con 0x7feffc06b250 2023-10-03T10:16:55.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.252+0000 7ff0011cc700 1 -- 172.21.15.3:0/3506919356 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feffc074ea0 con 0x7feffc06b250 2023-10-03T10:16:55.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.261+0000 7fefe37fe700 1 -- 172.21.15.3:0/3506919356 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feff0017020 con 0x7feffc06b250 2023-10-03T10:16:55.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.284+0000 7fe989587700 1 Processor -- start 2023-10-03T10:16:55.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.284+0000 7fe989587700 1 -- start start 2023-10-03T10:16:55.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.284+0000 7fe989587700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe98409eca0 0x7fe98413cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:55.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.284+0000 7fe989587700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe98406f050 0x7fe98413f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:55.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.284+0000 7fe989587700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe984070ee0 con 0x7fe98406e570 2023-10-03T10:16:55.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.284+0000 7fe989587700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe984071980 con 0x7fe98406f050 2023-10-03T10:16:55.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.284+0000 7fe989587700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe984071480 con 0x7fe98409eca0 2023-10-03T10:16:55.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.285+0000 7fe9837fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe98406f050 0x7fe98413f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:55.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.285+0000 7fe982ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe98406e570 0x7fe98406e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39856/0 (socket says 172.21.15.3:39856) 2023-10-03T10:16:55.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.285+0000 7fe982ffd700 1 -- 172.21.15.3:0/4054016134 learned_addr learned my addr 172.21.15.3:0/4054016134 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:55.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.285+0000 7fe9827fc700 1 --2- 172.21.15.3:0/4054016134 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe98409eca0 0x7fe98413cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:55.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.285+0000 7fe9837fe700 1 -- 172.21.15.3:0/4054016134 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe98409eca0 msgr2=0x7fe98413cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:55.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.285+0000 7fe9837fe700 1 --2- 172.21.15.3:0/4054016134 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe98409eca0 0x7fe98413cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.285+0000 7fe9837fe700 1 -- 172.21.15.3:0/4054016134 >> v1:172.21.15.3:6789/0 conn(0x7fe98406e570 legacy=0x7fe98406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:55.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.285+0000 7fe9837fe700 1 -- 172.21.15.3:0/4054016134 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe984071b90 con 0x7fe98406f050 2023-10-03T10:16:55.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.285+0000 7fe981ffb700 1 -- 172.21.15.3:0/4054016134 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1728428788 0 0) 0x7fe984070ee0 con 0x7fe98406e570 2023-10-03T10:16:55.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.285+0000 7fe9827fc700 1 --2- 172.21.15.3:0/4054016134 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe98409eca0 0x7fe98413cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:55.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.285+0000 7fe9837fe700 1 --2- 172.21.15.3:0/4054016134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe98406f050 0x7fe98413f3f0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fe97c009d90 tx=0x7fe97c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27bf7793f789f8da server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:55.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.285+0000 7fe981ffb700 1 -- 172.21.15.3:0/4054016134 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97c00e070 con 0x7fe98406f050 2023-10-03T10:16:55.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.286+0000 7fe981ffb700 1 -- 172.21.15.3:0/4054016134 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe97c00bb20 con 0x7fe98406f050 2023-10-03T10:16:55.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.286+0000 7fe981ffb700 1 -- 172.21.15.3:0/4054016134 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97c012430 con 0x7fe98406f050 2023-10-03T10:16:55.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.286+0000 7fe989587700 1 -- 172.21.15.3:0/4054016134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe98406f050 msgr2=0x7fe98413f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:55.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.286+0000 7fe989587700 1 --2- 172.21.15.3:0/4054016134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe98406f050 0x7fe98413f3f0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fe97c009d90 tx=0x7fe97c00b330 comp rx=0 tx=0).stop 2023-10-03T10:16:55.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.286+0000 7fe989587700 1 -- 172.21.15.3:0/4054016134 shutdown_connections 2023-10-03T10:16:55.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.286+0000 7fe989587700 1 --2- 172.21.15.3:0/4054016134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe98406f050 0x7fe98413f3f0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.286+0000 7fe989587700 1 --2- 172.21.15.3:0/4054016134 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe98409eca0 0x7fe98413cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.286+0000 7fe989587700 1 -- 172.21.15.3:0/4054016134 >> 172.21.15.3:0/4054016134 conn(0x7fe9840a2aa0 msgr2=0x7fe9840a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:55.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.286+0000 7fe989587700 1 -- 172.21.15.3:0/4054016134 shutdown_connections 2023-10-03T10:16:55.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.286+0000 7fe989587700 1 -- 172.21.15.3:0/4054016134 wait complete. 2023-10-03T10:16:55.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.287+0000 7fe989587700 1 Processor -- start 2023-10-03T10:16:55.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.288+0000 7fe989587700 1 -- start start 2023-10-03T10:16:55.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.288+0000 7fe989587700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe98406e570 0x7fe9841363a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:55.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.288+0000 7fe989587700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe98406f050 0x7fe9841368e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:55.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.288+0000 7fe989587700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe98413a3c0 0x7fe984136e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:55.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.288+0000 7fe989587700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe984139200 con 0x7fe98406e570 2023-10-03T10:16:55.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.288+0000 7fe989587700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe984138130 con 0x7fe98413a3c0 2023-10-03T10:16:55.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.288+0000 7fe989587700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe984070610 con 0x7fe98406f050 2023-10-03T10:16:55.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.288+0000 7fe9827fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe98406f050 0x7fe9841368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:55.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.288+0000 7fe982ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe98406e570 0x7fe9841363a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:55.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.288+0000 7fe9827fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe98406f050 0x7fe9841368e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:35814/0 (socket says 172.21.15.3:35814) 2023-10-03T10:16:55.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.288+0000 7fe9827fc700 1 -- 172.21.15.3:0/2201702979 learned_addr learned my addr 172.21.15.3:0/2201702979 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:55.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.288+0000 7fe9837fe700 1 --2- 172.21.15.3:0/2201702979 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe98413a3c0 0x7fe984136e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:55.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.289+0000 7fe9827fc700 1 -- 172.21.15.3:0/2201702979 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe98413a3c0 msgr2=0x7fe984136e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:55.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.289+0000 7fe9827fc700 1 --2- 172.21.15.3:0/2201702979 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe98413a3c0 0x7fe984136e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.289+0000 7fe9827fc700 1 -- 172.21.15.3:0/2201702979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe98406e570 msgr2=0x7fe9841363a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:55.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.289+0000 7fe9827fc700 1 --2- 172.21.15.3:0/2201702979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe98406e570 0x7fe9841363a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.289+0000 7fe9827fc700 1 -- 172.21.15.3:0/2201702979 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9780094a0 con 0x7fe98406f050 2023-10-03T10:16:55.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.289+0000 7fe982ffd700 1 --2- 172.21.15.3:0/2201702979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe98406e570 0x7fe9841363a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:55.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.289+0000 7fe9837fe700 1 --2- 172.21.15.3:0/2201702979 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe98413a3c0 0x7fe984136e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:55.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.289+0000 7fe9827fc700 1 --2- 172.21.15.3:0/2201702979 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe98406f050 0x7fe9841368e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe97800ebb0 tx=0x7fe97800eec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:55.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.289+0000 7fe96b7fe700 1 -- 172.21.15.3:0/2201702979 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97800c850 con 0x7fe98406f050 2023-10-03T10:16:55.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.289+0000 7fe96b7fe700 1 -- 172.21.15.3:0/2201702979 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe97800c9b0 con 0x7fe98406f050 2023-10-03T10:16:55.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.289+0000 7fe96b7fe700 1 -- 172.21.15.3:0/2201702979 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97800cd40 con 0x7fe98406f050 2023-10-03T10:16:55.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.289+0000 7fe989587700 1 -- 172.21.15.3:0/2201702979 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe97c009a70 con 0x7fe98406f050 2023-10-03T10:16:55.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.289+0000 7fe989587700 1 -- 172.21.15.3:0/2201702979 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe984139920 con 0x7fe98406f050 2023-10-03T10:16:55.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.290+0000 7fe96b7fe700 1 -- 172.21.15.3:0/2201702979 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe978013470 con 0x7fe98406f050 2023-10-03T10:16:55.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.290+0000 7fe96b7fe700 1 -- 172.21.15.3:0/2201702979 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe97803fb30 con 0x7fe98406f050 2023-10-03T10:16:55.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.290+0000 7fe989587700 1 -- 172.21.15.3:0/2201702979 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe984138810 con 0x7fe98406f050 2023-10-03T10:16:55.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.296+0000 7fe96b7fe700 1 -- 172.21.15.3:0/2201702979 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe97801b3b0 con 0x7fe98406f050 2023-10-03T10:16:55.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.439+0000 7fd1f1429700 1 -- 172.21.15.3:0/4217110089 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd1ec11fe30 con 0x7fd1ec06a770 2023-10-03T10:16:55.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.439+0000 7fd1d37fe700 1 -- 172.21.15.3:0/4217110089 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd1dc00bbd0 con 0x7fd1ec06a770 2023-10-03T10:16:55.439 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:55.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.441+0000 7fd1f1429700 1 -- 172.21.15.3:0/4217110089 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd1ec06a770 msgr2=0x7fd1ec11c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:55.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.441+0000 7fd1f1429700 1 --2- 172.21.15.3:0/4217110089 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd1ec06a770 0x7fd1ec11c490 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009c40 tx=0x7fd1dc004750 comp rx=0 tx=0).stop 2023-10-03T10:16:55.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.441+0000 7fd1f1429700 1 -- 172.21.15.3:0/4217110089 shutdown_connections 2023-10-03T10:16:55.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.441+0000 7fd1f1429700 1 --2- 172.21.15.3:0/4217110089 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1ec131b40 0x7fd1ec11cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.441+0000 7fd1f1429700 1 --2- 172.21.15.3:0/4217110089 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd1ec06a770 0x7fd1ec11c490 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.441+0000 7fd1f1429700 1 --2- 172.21.15.3:0/4217110089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1ec06b080 0x7fd1ec11c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.441+0000 7fd1f1429700 1 -- 172.21.15.3:0/4217110089 >> 172.21.15.3:0/4217110089 conn(0x7fd1ec066f50 msgr2=0x7fd1ec0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:55.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.441+0000 7fd1f1429700 1 -- 172.21.15.3:0/4217110089 shutdown_connections 2023-10-03T10:16:55.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.441+0000 7fd1f1429700 1 -- 172.21.15.3:0/4217110089 wait complete. 2023-10-03T10:16:55.455 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:55.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.470+0000 7ff0011cc700 1 -- 172.21.15.3:0/3506919356 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7feffc121d00 con 0x7feffc06b250 2023-10-03T10:16:55.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.470+0000 7fefe37fe700 1 -- 172.21.15.3:0/3506919356 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feff001b3b0 con 0x7feffc06b250 2023-10-03T10:16:55.471 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:55.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.472+0000 7ff0011cc700 1 -- 172.21.15.3:0/3506919356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feffc06b250 msgr2=0x7feffc09b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:55.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.472+0000 7ff0011cc700 1 --2- 172.21.15.3:0/3506919356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feffc06b250 0x7feffc09b0c0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7feff000e9b0 tx=0x7feff000ecc0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.473+0000 7ff0011cc700 1 -- 172.21.15.3:0/3506919356 shutdown_connections 2023-10-03T10:16:55.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.473+0000 7ff0011cc700 1 --2- 172.21.15.3:0/3506919356 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feffc136d80 0x7feffc1c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.473+0000 7ff0011cc700 1 --2- 172.21.15.3:0/3506919356 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feffc06a770 0x7feffc09ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.473+0000 7ff0011cc700 1 --2- 172.21.15.3:0/3506919356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feffc06b250 0x7feffc09b0c0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.473+0000 7ff0011cc700 1 -- 172.21.15.3:0/3506919356 >> 172.21.15.3:0/3506919356 conn(0x7feffc066f50 msgr2=0x7feffc06f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:55.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.473+0000 7ff0011cc700 1 -- 172.21.15.3:0/3506919356 shutdown_connections 2023-10-03T10:16:55.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.473+0000 7ff0011cc700 1 -- 172.21.15.3:0/3506919356 wait complete. 2023-10-03T10:16:55.487 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:55.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.504+0000 7fe989587700 1 -- 172.21.15.3:0/2201702979 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe984074ea0 con 0x7fe98406f050 2023-10-03T10:16:55.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.504+0000 7fe96b7fe700 1 -- 172.21.15.3:0/2201702979 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe978017020 con 0x7fe98406f050 2023-10-03T10:16:55.505 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:55.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.507+0000 7fe989587700 1 -- 172.21.15.3:0/2201702979 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe98406f050 msgr2=0x7fe9841368e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:55.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.507+0000 7fe989587700 1 --2- 172.21.15.3:0/2201702979 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe98406f050 0x7fe9841368e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe97800ebb0 tx=0x7fe97800eec0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.507+0000 7fe989587700 1 -- 172.21.15.3:0/2201702979 shutdown_connections 2023-10-03T10:16:55.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.507+0000 7fe989587700 1 --2- 172.21.15.3:0/2201702979 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe98413a3c0 0x7fe984136e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.507+0000 7fe989587700 1 --2- 172.21.15.3:0/2201702979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe98406e570 0x7fe9841363a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.507+0000 7fe989587700 1 --2- 172.21.15.3:0/2201702979 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe98406f050 0x7fe9841368e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:55.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.507+0000 7fe989587700 1 -- 172.21.15.3:0/2201702979 >> 172.21.15.3:0/2201702979 conn(0x7fe9840a2aa0 msgr2=0x7fe98409ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:55.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.507+0000 7fe989587700 1 -- 172.21.15.3:0/2201702979 shutdown_connections 2023-10-03T10:16:55.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:55.507+0000 7fe989587700 1 -- 172.21.15.3:0/2201702979 wait complete. 2023-10-03T10:16:55.521 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:56.249 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:56.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.399+0000 7f75cded8700 1 Processor -- start 2023-10-03T10:16:56.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.400+0000 7f75cded8700 1 -- start start 2023-10-03T10:16:56.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.400+0000 7f75cded8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f75c809e1d0 0x7f75c8135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:56.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.400+0000 7f75cded8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f75c8135e50 0x7f75c8138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:56.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.400+0000 7f75cded8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75c809f130 con 0x7f75c809d6f0 2023-10-03T10:16:56.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.400+0000 7f75cded8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75c809f520 con 0x7f75c809e1d0 2023-10-03T10:16:56.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.400+0000 7f75cded8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f75c809e930 con 0x7f75c8135e50 2023-10-03T10:16:56.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.400+0000 7f75c6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f75c809e1d0 0x7f75c8135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:56.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.400+0000 7f75c6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f75c809e1d0 0x7f75c8135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48818/0 (socket says 172.21.15.3:48818) 2023-10-03T10:16:56.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.400+0000 7f75c7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f75c8135e50 0x7f75c8138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:56.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.400+0000 7f75c6ffd700 1 -- 172.21.15.3:0/570143918 learned_addr learned my addr 172.21.15.3:0/570143918 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:56.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.401+0000 7f75c6ffd700 1 -- 172.21.15.3:0/570143918 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f75c8135e50 msgr2=0x7f75c8138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:56.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.401+0000 7f75c6ffd700 1 --2- 172.21.15.3:0/570143918 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f75c8135e50 0x7f75c8138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.401+0000 7f75c6ffd700 1 -- 172.21.15.3:0/570143918 >> v1:172.21.15.3:6789/0 conn(0x7f75c809d6f0 legacy=0x7f75c809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:56.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.401+0000 7f75c6ffd700 1 -- 172.21.15.3:0/570143918 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c809b2d0 con 0x7f75c809e1d0 2023-10-03T10:16:56.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.401+0000 7f75c7fff700 1 --2- 172.21.15.3:0/570143918 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f75c8135e50 0x7f75c8138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:56.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.401+0000 7f75c6ffd700 1 --2- 172.21.15.3:0/570143918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75c809e1d0 0x7f75c8135910 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f75b00098f0 tx=0x7f75b0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbaa084a0167b533 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:56.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.401+0000 7f75c67fc700 1 -- 172.21.15.3:0/570143918 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b0011830 con 0x7f75c809e1d0 2023-10-03T10:16:56.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.401+0000 7f75c67fc700 1 -- 172.21.15.3:0/570143918 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b0011990 con 0x7f75c809e1d0 2023-10-03T10:16:56.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.401+0000 7f75c67fc700 1 -- 172.21.15.3:0/570143918 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b0011d20 con 0x7f75c809e1d0 2023-10-03T10:16:56.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.402+0000 7f75cded8700 1 -- 172.21.15.3:0/570143918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75c809e1d0 msgr2=0x7f75c8135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:56.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.402+0000 7f75cded8700 1 --2- 172.21.15.3:0/570143918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75c809e1d0 0x7f75c8135910 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f75b00098f0 tx=0x7f75b0009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:56.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.402+0000 7f75cded8700 1 -- 172.21.15.3:0/570143918 shutdown_connections 2023-10-03T10:16:56.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.402+0000 7f75cded8700 1 --2- 172.21.15.3:0/570143918 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f75c8135e50 0x7f75c8138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.402+0000 7f75cded8700 1 --2- 172.21.15.3:0/570143918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75c809e1d0 0x7f75c8135910 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.402+0000 7f75cded8700 1 -- 172.21.15.3:0/570143918 >> 172.21.15.3:0/570143918 conn(0x7f75c8066f50 msgr2=0x7f75c809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:56.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.402+0000 7f75cded8700 1 -- 172.21.15.3:0/570143918 shutdown_connections 2023-10-03T10:16:56.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.402+0000 7f75cded8700 1 -- 172.21.15.3:0/570143918 wait complete. 2023-10-03T10:16:56.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.403+0000 7f75cded8700 1 Processor -- start 2023-10-03T10:16:56.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.403+0000 7f75cded8700 1 -- start start 2023-10-03T10:16:56.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.403+0000 7f75cded8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f75c809d6f0 0x7f75c811c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:56.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.403+0000 7f75cded8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f75c809e1d0 0x7f75c811c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:56.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.403+0000 7f75cded8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75c8135e50 0x7f75c811cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:56.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.403+0000 7f75cded8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c809c700 con 0x7f75c8135e50 2023-10-03T10:16:56.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.403+0000 7f75cded8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c80a1f30 con 0x7f75c809e1d0 2023-10-03T10:16:56.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.403+0000 7f75cded8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f75c811f450 con 0x7f75c809d6f0 2023-10-03T10:16:56.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.403+0000 7f75c7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75c8135e50 0x7f75c811cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:56.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.403+0000 7f75c6ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f75c809e1d0 0x7f75c811c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:56.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.403+0000 7f75c7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75c8135e50 0x7f75c811cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48826/0 (socket says 172.21.15.3:48826) 2023-10-03T10:16:56.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.403+0000 7f75c6ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f75c809e1d0 0x7f75c811c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33266/0 (socket says 172.21.15.3:33266) 2023-10-03T10:16:56.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.403+0000 7f75c7fff700 1 -- 172.21.15.3:0/1173106031 learned_addr learned my addr 172.21.15.3:0/1173106031 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:56.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.404+0000 7f75c77fe700 1 --2- 172.21.15.3:0/1173106031 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f75c809d6f0 0x7f75c811c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:56.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.404+0000 7f75c7fff700 1 -- 172.21.15.3:0/1173106031 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f75c809d6f0 msgr2=0x7f75c811c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:56.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.404+0000 7f75c7fff700 1 --2- 172.21.15.3:0/1173106031 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f75c809d6f0 0x7f75c811c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.404+0000 7f75c7fff700 1 -- 172.21.15.3:0/1173106031 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f75c809e1d0 msgr2=0x7f75c811c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:56.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.404+0000 7f75c7fff700 1 --2- 172.21.15.3:0/1173106031 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f75c809e1d0 0x7f75c811c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.404+0000 7f75c7fff700 1 -- 172.21.15.3:0/1173106031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c0009a70 con 0x7f75c8135e50 2023-10-03T10:16:56.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.404+0000 7f75c6ffd700 1 --2- 172.21.15.3:0/1173106031 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f75c809e1d0 0x7f75c811c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:56.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.404+0000 7f75c7fff700 1 --2- 172.21.15.3:0/1173106031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75c8135e50 0x7f75c811cc20 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f75c000efb0 tx=0x7f75c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:56.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.404+0000 7f75bffff700 1 -- 172.21.15.3:0/1173106031 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c000cc90 con 0x7f75c8135e50 2023-10-03T10:16:56.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.404+0000 7f75bffff700 1 -- 172.21.15.3:0/1173106031 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c000f040 con 0x7f75c8135e50 2023-10-03T10:16:56.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.404+0000 7f75bffff700 1 -- 172.21.15.3:0/1173106031 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c0014480 con 0x7f75c8135e50 2023-10-03T10:16:56.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.404+0000 7f75cded8700 1 -- 172.21.15.3:0/1173106031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b0009550 con 0x7f75c8135e50 2023-10-03T10:16:56.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.404+0000 7f75cded8700 1 -- 172.21.15.3:0/1173106031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c811e5a0 con 0x7f75c8135e50 2023-10-03T10:16:56.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.405+0000 7f75bffff700 1 -- 172.21.15.3:0/1173106031 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f75c00145e0 con 0x7f75c8135e50 2023-10-03T10:16:56.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.405+0000 7f75bffff700 1 -- 172.21.15.3:0/1173106031 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f75c0040300 con 0x7f75c8135e50 2023-10-03T10:16:56.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.405+0000 7f75cded8700 1 -- 172.21.15.3:0/1173106031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75c8071b90 con 0x7f75c8135e50 2023-10-03T10:16:56.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.410+0000 7f75bffff700 1 -- 172.21.15.3:0/1173106031 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f75c00076f0 con 0x7f75c8135e50 2023-10-03T10:16:56.456 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:56.487 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:56.522 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:56.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.606+0000 7f4bebe9d700 1 Processor -- start 2023-10-03T10:16:56.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.606+0000 7f4bebe9d700 1 -- start start 2023-10-03T10:16:56.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.607+0000 7f4bebe9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4be409d6f0 0x7f4be409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:56.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.607+0000 7f4bebe9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4be409e000 0x7f4be4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:56.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.607+0000 7f4bebe9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4be409e410 0x7f4be4130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:56.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.607+0000 7f4bebe9d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be409f520 con 0x7f4be409d6f0 2023-10-03T10:16:56.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.607+0000 7f4bebe9d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be409f130 con 0x7f4be409e000 2023-10-03T10:16:56.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.607+0000 7f4bebe9d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4be409e930 con 0x7f4be409e410 2023-10-03T10:16:56.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.607+0000 7f4be9438700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4be409e000 0x7f4be4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:56.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.607+0000 7f4bea43a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4be409e410 0x7f4be4130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:56.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.607+0000 7f4be9438700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4be409e000 0x7f4be4130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33282/0 (socket says 172.21.15.3:33282) 2023-10-03T10:16:56.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.607+0000 7f4be9438700 1 -- 172.21.15.3:0/823946615 learned_addr learned my addr 172.21.15.3:0/823946615 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:56.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.607+0000 7f4be9c39700 1 --2- 172.21.15.3:0/823946615 >> v2:172.21.15.3:3300/0 conn(0x7f4be409d6f0 0x7f4be409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:56.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.608+0000 7f4be9438700 1 -- 172.21.15.3:0/823946615 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4be409e410 msgr2=0x7f4be4130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:56.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.608+0000 7f4be9438700 1 --2- 172.21.15.3:0/823946615 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4be409e410 0x7f4be4130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.608+0000 7f4be9438700 1 -- 172.21.15.3:0/823946615 >> v2:172.21.15.3:3300/0 conn(0x7f4be409d6f0 msgr2=0x7f4be409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:56.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.608+0000 7f4be9438700 1 --2- 172.21.15.3:0/823946615 >> v2:172.21.15.3:3300/0 conn(0x7f4be409d6f0 0x7f4be409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.608+0000 7f4be9438700 1 -- 172.21.15.3:0/823946615 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be409b2d0 con 0x7f4be409e000 2023-10-03T10:16:56.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.608+0000 7f4be9c39700 1 --2- 172.21.15.3:0/823946615 >> v2:172.21.15.3:3300/0 conn(0x7f4be409d6f0 0x7f4be409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:56.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.608+0000 7f4be9438700 1 --2- 172.21.15.3:0/823946615 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4be409e000 0x7f4be4130660 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40098f0 tx=0x7f4bd4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf3e8600e6a9ab0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:56.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.608+0000 7f4be8c37700 1 -- 172.21.15.3:0/823946615 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd4010a90 con 0x7f4be409e000 2023-10-03T10:16:56.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.608+0000 7f4be8c37700 1 -- 172.21.15.3:0/823946615 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd4010bf0 con 0x7f4be409e000 2023-10-03T10:16:56.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.608+0000 7f4be8c37700 1 -- 172.21.15.3:0/823946615 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd4018440 con 0x7f4be409e000 2023-10-03T10:16:56.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.609+0000 7f4bebe9d700 1 -- 172.21.15.3:0/823946615 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4be409e000 msgr2=0x7f4be4130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:56.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.609+0000 7f4bebe9d700 1 --2- 172.21.15.3:0/823946615 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4be409e000 0x7f4be4130660 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40098f0 tx=0x7f4bd4009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:56.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.609+0000 7f4bebe9d700 1 -- 172.21.15.3:0/823946615 shutdown_connections 2023-10-03T10:16:56.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.609+0000 7f4bebe9d700 1 --2- 172.21.15.3:0/823946615 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4be409e410 0x7f4be4130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.609+0000 7f4bebe9d700 1 --2- 172.21.15.3:0/823946615 >> v2:172.21.15.3:3300/0 conn(0x7f4be409d6f0 0x7f4be409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.609+0000 7f4bebe9d700 1 --2- 172.21.15.3:0/823946615 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4be409e000 0x7f4be4130660 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.609+0000 7f4bebe9d700 1 -- 172.21.15.3:0/823946615 >> 172.21.15.3:0/823946615 conn(0x7f4be4066f50 msgr2=0x7f4be409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:56.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.609+0000 7f4bebe9d700 1 -- 172.21.15.3:0/823946615 shutdown_connections 2023-10-03T10:16:56.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.609+0000 7f4bebe9d700 1 -- 172.21.15.3:0/823946615 wait complete. 2023-10-03T10:16:56.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.610+0000 7f4bebe9d700 1 Processor -- start 2023-10-03T10:16:56.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.610+0000 7f4bebe9d700 1 -- start start 2023-10-03T10:16:56.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.611+0000 7f4bebe9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4be409d6f0 0x7f4be41b6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:56.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.611+0000 7f4bebe9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4be409e000 0x7f4be41b6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:56.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.611+0000 7f4bebe9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4be409e410 0x7f4be412d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:56.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.611+0000 7f4bebe9d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be41b9cf0 con 0x7f4be409e410 2023-10-03T10:16:56.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.611+0000 7f4bebe9d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be41c6d50 con 0x7f4be409e000 2023-10-03T10:16:56.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.611+0000 7f4bebe9d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4be41b6680 con 0x7f4be409d6f0 2023-10-03T10:16:56.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.611+0000 7f4be9438700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4be409e000 0x7f4be41b6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:56.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.611+0000 7f4be9438700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4be409e000 0x7f4be41b6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33288/0 (socket says 172.21.15.3:33288) 2023-10-03T10:16:56.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.611+0000 7f4be9c39700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4be409d6f0 0x7f4be41b6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:56.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.611+0000 7f4be9438700 1 -- 172.21.15.3:0/2544385884 learned_addr learned my addr 172.21.15.3:0/2544385884 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:56.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.611+0000 7f4bea43a700 1 --2- 172.21.15.3:0/2544385884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4be409e410 0x7f4be412d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:56.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.611+0000 7f4bea43a700 1 -- 172.21.15.3:0/2544385884 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4be409d6f0 msgr2=0x7f4be41b6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:56.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.611+0000 7f4bea43a700 1 --2- 172.21.15.3:0/2544385884 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4be409d6f0 0x7f4be41b6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.611+0000 7f4bea43a700 1 -- 172.21.15.3:0/2544385884 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4be409e000 msgr2=0x7f4be41b6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:56.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.611+0000 7f4bea43a700 1 --2- 172.21.15.3:0/2544385884 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4be409e000 0x7f4be41b6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.611+0000 7f4bea43a700 1 -- 172.21.15.3:0/2544385884 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be00094a0 con 0x7f4be409e410 2023-10-03T10:16:56.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.611+0000 7f4be9c39700 1 --2- 172.21.15.3:0/2544385884 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4be409d6f0 0x7f4be41b6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:56.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.611+0000 7f4be9438700 1 --2- 172.21.15.3:0/2544385884 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4be409e000 0x7f4be41b6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:56.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.612+0000 7f4bea43a700 1 --2- 172.21.15.3:0/2544385884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4be409e410 0x7f4be412d1a0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc00efc0 tx=0x7f4bdc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:56.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.612+0000 7f4bda7fc700 1 -- 172.21.15.3:0/2544385884 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc00cc90 con 0x7f4be409e410 2023-10-03T10:16:56.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.612+0000 7f4bda7fc700 1 -- 172.21.15.3:0/2544385884 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bdc00f040 con 0x7f4be409e410 2023-10-03T10:16:56.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.612+0000 7f4bebe9d700 1 -- 172.21.15.3:0/2544385884 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd4009550 con 0x7f4be409e410 2023-10-03T10:16:56.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.612+0000 7f4bda7fc700 1 -- 172.21.15.3:0/2544385884 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc0146c0 con 0x7f4be409e410 2023-10-03T10:16:56.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.612+0000 7f4bebe9d700 1 -- 172.21.15.3:0/2544385884 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be40a2100 con 0x7f4be409e410 2023-10-03T10:16:56.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.613+0000 7f4bda7fc700 1 -- 172.21.15.3:0/2544385884 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4bdc014820 con 0x7f4be409e410 2023-10-03T10:16:56.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.613+0000 7f4bebe9d700 1 -- 172.21.15.3:0/2544385884 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be4074ea0 con 0x7f4be409e410 2023-10-03T10:16:56.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.613+0000 7f4bda7fc700 1 -- 172.21.15.3:0/2544385884 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4bdc040250 con 0x7f4be409e410 2023-10-03T10:16:56.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.622+0000 7f75cded8700 1 -- 172.21.15.3:0/1173106031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f75c8074ea0 con 0x7f75c8135e50 2023-10-03T10:16:56.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.622+0000 7f75bffff700 1 -- 172.21.15.3:0/1173106031 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f75c002c090 con 0x7f75c8135e50 2023-10-03T10:16:56.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.622+0000 7f4bda7fc700 1 -- 172.21.15.3:0/2544385884 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4bdc018030 con 0x7f4be409e410 2023-10-03T10:16:56.623 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:56.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.625+0000 7f75cded8700 1 -- 172.21.15.3:0/1173106031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75c8135e50 msgr2=0x7f75c811cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:56.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.625+0000 7f75cded8700 1 --2- 172.21.15.3:0/1173106031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75c8135e50 0x7f75c811cc20 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f75c000efb0 tx=0x7f75c000c3a0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.625+0000 7f75cded8700 1 -- 172.21.15.3:0/1173106031 shutdown_connections 2023-10-03T10:16:56.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.625+0000 7f75cded8700 1 --2- 172.21.15.3:0/1173106031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75c8135e50 0x7f75c811cc20 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.625+0000 7f75cded8700 1 --2- 172.21.15.3:0/1173106031 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f75c809d6f0 0x7f75c811c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.625+0000 7f75cded8700 1 --2- 172.21.15.3:0/1173106031 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f75c809e1d0 0x7f75c811c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.625+0000 7f75cded8700 1 -- 172.21.15.3:0/1173106031 >> 172.21.15.3:0/1173106031 conn(0x7f75c8066f50 msgr2=0x7f75c806ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:56.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.625+0000 7f75cded8700 1 -- 172.21.15.3:0/1173106031 shutdown_connections 2023-10-03T10:16:56.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.625+0000 7f75cded8700 1 -- 172.21.15.3:0/1173106031 wait complete. 2023-10-03T10:16:56.640 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.645+0000 7f61a70df700 1 Processor -- start 2023-10-03T10:16:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.645+0000 7f61a70df700 1 -- start start 2023-10-03T10:16:56.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.646+0000 7f61a70df700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f61a009d6f0 0x7f61a009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:56.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.646+0000 7f61a70df700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61a009e000 0x7f61a01339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:56.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.646+0000 7f61a70df700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61a009b120 con 0x7f61a009e410 2023-10-03T10:16:56.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.646+0000 7f61a70df700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a006b460 con 0x7f61a009e000 2023-10-03T10:16:56.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.646+0000 7f61a70df700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f61a009a7e0 con 0x7f61a009d6f0 2023-10-03T10:16:56.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.646+0000 7f61a567c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f61a009e410 0x7f61a0133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39866/0 (socket says 172.21.15.3:39866) 2023-10-03T10:16:56.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.646+0000 7f61a567c700 1 -- 172.21.15.3:0/1631271069 learned_addr learned my addr 172.21.15.3:0/1631271069 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:56.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.646+0000 7f61a4e7b700 1 --2- 172.21.15.3:0/1631271069 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f61a009d6f0 0x7f61a009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:56.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.646+0000 7f619ffff700 1 --2- 172.21.15.3:0/1631271069 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61a009e000 0x7f61a01339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:56.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.646+0000 7f619f7fe700 1 -- 172.21.15.3:0/1631271069 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 802676027 0 0) 0x7f61a009b120 con 0x7f61a009e410 2023-10-03T10:16:56.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.646+0000 7f61a4e7b700 1 -- 172.21.15.3:0/1631271069 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61a009e000 msgr2=0x7f61a01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:56.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.646+0000 7f61a4e7b700 1 --2- 172.21.15.3:0/1631271069 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61a009e000 0x7f61a01339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.646+0000 7f61a4e7b700 1 -- 172.21.15.3:0/1631271069 >> v1:172.21.15.3:6789/0 conn(0x7f61a009e410 legacy=0x7f61a0133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:56.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.646+0000 7f61a4e7b700 1 -- 172.21.15.3:0/1631271069 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a009a500 con 0x7f61a009d6f0 2023-10-03T10:16:56.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.646+0000 7f619ffff700 1 --2- 172.21.15.3:0/1631271069 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61a009e000 0x7f61a01339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:56.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.647+0000 7f61a4e7b700 1 --2- 172.21.15.3:0/1631271069 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f61a009d6f0 0x7f61a009dac0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f61940098a0 tx=0x7f6194009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d26249179bd38f95 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:56.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.647+0000 7f619f7fe700 1 -- 172.21.15.3:0/1631271069 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6194011830 con 0x7f61a009d6f0 2023-10-03T10:16:56.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.647+0000 7f619f7fe700 1 -- 172.21.15.3:0/1631271069 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6194011990 con 0x7f61a009d6f0 2023-10-03T10:16:56.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.647+0000 7f619f7fe700 1 -- 172.21.15.3:0/1631271069 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6194011d20 con 0x7f61a009d6f0 2023-10-03T10:16:56.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.647+0000 7f61a70df700 1 -- 172.21.15.3:0/1631271069 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f61a009d6f0 msgr2=0x7f61a009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:56.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.647+0000 7f61a70df700 1 --2- 172.21.15.3:0/1631271069 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f61a009d6f0 0x7f61a009dac0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f61940098a0 tx=0x7f6194009bb0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.647+0000 7f61a70df700 1 -- 172.21.15.3:0/1631271069 shutdown_connections 2023-10-03T10:16:56.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.647+0000 7f61a70df700 1 --2- 172.21.15.3:0/1631271069 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f61a009d6f0 0x7f61a009dac0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.647+0000 7f61a70df700 1 --2- 172.21.15.3:0/1631271069 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61a009e000 0x7f61a01339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.648+0000 7f61a70df700 1 -- 172.21.15.3:0/1631271069 >> 172.21.15.3:0/1631271069 conn(0x7f61a0066f50 msgr2=0x7f61a009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:56.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.648+0000 7f61a70df700 1 -- 172.21.15.3:0/1631271069 shutdown_connections 2023-10-03T10:16:56.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.648+0000 7f61a70df700 1 -- 172.21.15.3:0/1631271069 wait complete. 2023-10-03T10:16:56.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.649+0000 7f61a70df700 1 Processor -- start 2023-10-03T10:16:56.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.649+0000 7f61a70df700 1 -- start start 2023-10-03T10:16:56.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.649+0000 7f61a70df700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f61a009d6f0 0x7f61a01c5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:56.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.649+0000 7f61a70df700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61a009e000 0x7f61a01c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:56.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.649+0000 7f61a70df700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61a009e410 0x7f61a01c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:56.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.649+0000 7f61a70df700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a01a72e0 con 0x7f61a009e410 2023-10-03T10:16:56.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.649+0000 7f61a70df700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a01c5ad0 con 0x7f61a009e000 2023-10-03T10:16:56.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.649+0000 7f61a70df700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f61a01a9820 con 0x7f61a009d6f0 2023-10-03T10:16:56.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.649+0000 7f61a4e7b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f61a009d6f0 0x7f61a01c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:56.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.649+0000 7f61a4e7b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f61a009d6f0 0x7f61a01c5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:35858/0 (socket says 172.21.15.3:35858) 2023-10-03T10:16:56.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.649+0000 7f61a4e7b700 1 -- 172.21.15.3:0/2411899138 learned_addr learned my addr 172.21.15.3:0/2411899138 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:56.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.650+0000 7f619ffff700 1 --2- 172.21.15.3:0/2411899138 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61a009e000 0x7f61a01c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:56.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.650+0000 7f61a567c700 1 --2- 172.21.15.3:0/2411899138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61a009e410 0x7f61a01c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:56.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.650+0000 7f61a4e7b700 1 -- 172.21.15.3:0/2411899138 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61a009e000 msgr2=0x7f61a01c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:56.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.650+0000 7f61a4e7b700 1 --2- 172.21.15.3:0/2411899138 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61a009e000 0x7f61a01c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.650+0000 7f61a4e7b700 1 -- 172.21.15.3:0/2411899138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61a009e410 msgr2=0x7f61a01c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:56.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.650+0000 7f61a4e7b700 1 --2- 172.21.15.3:0/2411899138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61a009e410 0x7f61a01c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.650+0000 7f61a4e7b700 1 -- 172.21.15.3:0/2411899138 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618c0094a0 con 0x7f61a009d6f0 2023-10-03T10:16:56.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.650+0000 7f61a567c700 1 --2- 172.21.15.3:0/2411899138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61a009e410 0x7f61a01c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:56.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.650+0000 7f61a4e7b700 1 --2- 172.21.15.3:0/2411899138 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f61a009d6f0 0x7f61a01c5050 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f61940097c0 tx=0x7f6194004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:56.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.650+0000 7f619d7fa700 1 -- 172.21.15.3:0/2411899138 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6194004450 con 0x7f61a009d6f0 2023-10-03T10:16:56.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.650+0000 7f619d7fa700 1 -- 172.21.15.3:0/2411899138 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619401f070 con 0x7f61a009d6f0 2023-10-03T10:16:56.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.650+0000 7f61a70df700 1 -- 172.21.15.3:0/2411899138 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61940094a0 con 0x7f61a009d6f0 2023-10-03T10:16:56.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.650+0000 7f619d7fa700 1 -- 172.21.15.3:0/2411899138 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61940038e0 con 0x7f61a009d6f0 2023-10-03T10:16:56.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.650+0000 7f61a70df700 1 -- 172.21.15.3:0/2411899138 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a01a8b60 con 0x7f61a009d6f0 2023-10-03T10:16:56.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.651+0000 7f619d7fa700 1 -- 172.21.15.3:0/2411899138 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6194003a40 con 0x7f61a009d6f0 2023-10-03T10:16:56.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.652+0000 7f619d7fa700 1 -- 172.21.15.3:0/2411899138 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6194021070 con 0x7f61a009d6f0 2023-10-03T10:16:56.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.652+0000 7f61a70df700 1 -- 172.21.15.3:0/2411899138 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a01a79d0 con 0x7f61a009d6f0 2023-10-03T10:16:56.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.661+0000 7f619d7fa700 1 -- 172.21.15.3:0/2411899138 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f619402b020 con 0x7f61a009d6f0 2023-10-03T10:16:56.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.700+0000 7f5e78268700 1 Processor -- start 2023-10-03T10:16:56.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.700+0000 7f5e78268700 1 -- start start 2023-10-03T10:16:56.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.701+0000 7f5e78268700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e7009d630 0x7f5e700a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:56.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.701+0000 7f5e78268700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5e70138d80 0x7f5e7013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:56.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.701+0000 7f5e78268700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e700a2a70 con 0x7f5e700a0a30 2023-10-03T10:16:56.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.701+0000 7f5e78268700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e700a01d0 con 0x7f5e70138d80 2023-10-03T10:16:56.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.701+0000 7f5e78268700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7009e2b0 con 0x7f5e7009d630 2023-10-03T10:16:56.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.701+0000 7f5e75803700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e7009d630 0x7f5e700a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:56.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.701+0000 7f5e75803700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e7009d630 0x7f5e700a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33324/0 (socket says 172.21.15.3:33324) 2023-10-03T10:16:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.701+0000 7f5e75803700 1 -- 172.21.15.3:0/2479951197 learned_addr learned my addr 172.21.15.3:0/2479951197 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.701+0000 7f5e76805700 1 --2- 172.21.15.3:0/2479951197 >> v2:172.21.15.3:3300/0 conn(0x7f5e70138d80 0x7f5e7013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.702+0000 7f5e75803700 1 -- 172.21.15.3:0/2479951197 >> v2:172.21.15.3:3300/0 conn(0x7f5e70138d80 msgr2=0x7f5e7013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.702+0000 7f5e75803700 1 --2- 172.21.15.3:0/2479951197 >> v2:172.21.15.3:3300/0 conn(0x7f5e70138d80 0x7f5e7013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.702+0000 7f5e75803700 1 -- 172.21.15.3:0/2479951197 >> v1:172.21.15.3:6789/0 conn(0x7f5e700a0a30 legacy=0x7f5e700a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.702+0000 7f5e75803700 1 -- 172.21.15.3:0/2479951197 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e7009dc50 con 0x7f5e7009d630 2023-10-03T10:16:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.702+0000 7f5e75803700 1 --2- 172.21.15.3:0/2479951197 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e7009d630 0x7f5e700a1510 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5e600098f0 tx=0x7f5e60009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69af93b46557c454 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.702+0000 7f5e75002700 1 -- 172.21.15.3:0/2479951197 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e60011830 con 0x7f5e7009d630 2023-10-03T10:16:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.702+0000 7f5e75002700 1 -- 172.21.15.3:0/2479951197 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e60011990 con 0x7f5e7009d630 2023-10-03T10:16:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.702+0000 7f5e75002700 1 -- 172.21.15.3:0/2479951197 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e60011d20 con 0x7f5e7009d630 2023-10-03T10:16:56.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.703+0000 7f5e78268700 1 -- 172.21.15.3:0/2479951197 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e7009d630 msgr2=0x7f5e700a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:56.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.703+0000 7f5e78268700 1 --2- 172.21.15.3:0/2479951197 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e7009d630 0x7f5e700a1510 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5e600098f0 tx=0x7f5e60009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:56.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.703+0000 7f5e78268700 1 -- 172.21.15.3:0/2479951197 shutdown_connections 2023-10-03T10:16:56.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.703+0000 7f5e78268700 1 --2- 172.21.15.3:0/2479951197 >> v2:172.21.15.3:3300/0 conn(0x7f5e70138d80 0x7f5e7013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.703+0000 7f5e78268700 1 --2- 172.21.15.3:0/2479951197 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e7009d630 0x7f5e700a1510 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.703+0000 7f5e78268700 1 -- 172.21.15.3:0/2479951197 >> 172.21.15.3:0/2479951197 conn(0x7f5e700a3050 msgr2=0x7f5e700a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:56.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.703+0000 7f5e78268700 1 -- 172.21.15.3:0/2479951197 shutdown_connections 2023-10-03T10:16:56.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.703+0000 7f5e78268700 1 -- 172.21.15.3:0/2479951197 wait complete. 2023-10-03T10:16:56.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.704+0000 7f5e78268700 1 Processor -- start 2023-10-03T10:16:56.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.704+0000 7f5e78268700 1 -- start start 2023-10-03T10:16:56.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.705+0000 7f5e78268700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e7009d630 0x7f5e70120640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:56.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.705+0000 7f5e78268700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e700a0a30 0x7f5e70120b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:56.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.705+0000 7f5e78268700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e70138d80 0x7f5e701210c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:56.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.705+0000 7f5e78268700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e701230d0 con 0x7f5e700a0a30 2023-10-03T10:16:56.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.705+0000 7f5e78268700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e70123df0 con 0x7f5e7009d630 2023-10-03T10:16:56.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.705+0000 7f5e78268700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e70138a10 con 0x7f5e70138d80 2023-10-03T10:16:56.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.705+0000 7f5e75803700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e700a0a30 0x7f5e70120b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:56.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.705+0000 7f5e75803700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e700a0a30 0x7f5e70120b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48850/0 (socket says 172.21.15.3:48850) 2023-10-03T10:16:56.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.705+0000 7f5e75803700 1 -- 172.21.15.3:0/2472148131 learned_addr learned my addr 172.21.15.3:0/2472148131 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:56.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.705+0000 7f5e76805700 1 --2- 172.21.15.3:0/2472148131 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e70138d80 0x7f5e701210c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:56.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.705+0000 7f5e75803700 1 -- 172.21.15.3:0/2472148131 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e70138d80 msgr2=0x7f5e701210c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:56.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.705+0000 7f5e75803700 1 --2- 172.21.15.3:0/2472148131 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e70138d80 0x7f5e701210c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.705+0000 7f5e75803700 1 -- 172.21.15.3:0/2472148131 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e7009d630 msgr2=0x7f5e70120640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:56.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.705+0000 7f5e76004700 1 --2- 172.21.15.3:0/2472148131 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e7009d630 0x7f5e70120640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:56.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.705+0000 7f5e75803700 1 --2- 172.21.15.3:0/2472148131 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e7009d630 0x7f5e70120640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.705+0000 7f5e75803700 1 -- 172.21.15.3:0/2472148131 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e60009550 con 0x7f5e700a0a30 2023-10-03T10:16:56.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.705+0000 7f5e76004700 1 --2- 172.21.15.3:0/2472148131 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e7009d630 0x7f5e70120640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:16:56.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.705+0000 7f5e75803700 1 --2- 172.21.15.3:0/2472148131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e700a0a30 0x7f5e70120b80 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f5e60000c00 tx=0x7f5e60004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:56.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.706+0000 7f5e66ffd700 1 -- 172.21.15.3:0/2472148131 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e600036a0 con 0x7f5e700a0a30 2023-10-03T10:16:56.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.706+0000 7f5e66ffd700 1 -- 172.21.15.3:0/2472148131 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e60003800 con 0x7f5e700a0a30 2023-10-03T10:16:56.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.706+0000 7f5e78268700 1 -- 172.21.15.3:0/2472148131 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e7013ae90 con 0x7f5e700a0a30 2023-10-03T10:16:56.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.706+0000 7f5e66ffd700 1 -- 172.21.15.3:0/2472148131 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e60003bd0 con 0x7f5e700a0a30 2023-10-03T10:16:56.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.706+0000 7f5e78268700 1 -- 172.21.15.3:0/2472148131 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e70124860 con 0x7f5e700a0a30 2023-10-03T10:16:56.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.706+0000 7f5e66ffd700 1 -- 172.21.15.3:0/2472148131 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5e60021070 con 0x7f5e700a0a30 2023-10-03T10:16:56.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.707+0000 7f5e66ffd700 1 -- 172.21.15.3:0/2472148131 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5e60040c10 con 0x7f5e700a0a30 2023-10-03T10:16:56.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.707+0000 7f5e78268700 1 -- 172.21.15.3:0/2472148131 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e700747f0 con 0x7f5e700a0a30 2023-10-03T10:16:56.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.716+0000 7f5e66ffd700 1 -- 172.21.15.3:0/2472148131 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5e60015080 con 0x7f5e700a0a30 2023-10-03T10:16:56.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.833+0000 7f4bebe9d700 1 -- 172.21.15.3:0/2544385884 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4be41b7de0 con 0x7f4be409e410 2023-10-03T10:16:56.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.834+0000 7f4bda7fc700 1 -- 172.21.15.3:0/2544385884 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4bdc02b090 con 0x7f4be409e410 2023-10-03T10:16:56.834 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:56.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.836+0000 7f4bebe9d700 1 -- 172.21.15.3:0/2544385884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4be409e410 msgr2=0x7f4be412d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:56.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.836+0000 7f4bebe9d700 1 --2- 172.21.15.3:0/2544385884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4be409e410 0x7f4be412d1a0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc00efc0 tx=0x7f4bdc00c3a0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.836+0000 7f4bebe9d700 1 -- 172.21.15.3:0/2544385884 shutdown_connections 2023-10-03T10:16:56.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.836+0000 7f4bebe9d700 1 --2- 172.21.15.3:0/2544385884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4be409e410 0x7f4be412d1a0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.836+0000 7f4bebe9d700 1 --2- 172.21.15.3:0/2544385884 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4be409d6f0 0x7f4be41b6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.836+0000 7f4bebe9d700 1 --2- 172.21.15.3:0/2544385884 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4be409e000 0x7f4be41b6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.836+0000 7f4bebe9d700 1 -- 172.21.15.3:0/2544385884 >> 172.21.15.3:0/2544385884 conn(0x7f4be4066f50 msgr2=0x7f4be4070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:56.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.836+0000 7f4bebe9d700 1 -- 172.21.15.3:0/2544385884 shutdown_connections 2023-10-03T10:16:56.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.836+0000 7f4bebe9d700 1 -- 172.21.15.3:0/2544385884 wait complete. 2023-10-03T10:16:56.850 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:56.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.873+0000 7f61a70df700 1 -- 172.21.15.3:0/2411899138 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f61a0074ea0 con 0x7f61a009d6f0 2023-10-03T10:16:56.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.873+0000 7f619d7fa700 1 -- 172.21.15.3:0/2411899138 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6194015070 con 0x7f61a009d6f0 2023-10-03T10:16:56.873 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:56.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.876+0000 7f61a70df700 1 -- 172.21.15.3:0/2411899138 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f61a009d6f0 msgr2=0x7f61a01c5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:56.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.876+0000 7f61a70df700 1 --2- 172.21.15.3:0/2411899138 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f61a009d6f0 0x7f61a01c5050 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f61940097c0 tx=0x7f6194004750 comp rx=0 tx=0).stop 2023-10-03T10:16:56.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.876+0000 7f61a70df700 1 -- 172.21.15.3:0/2411899138 shutdown_connections 2023-10-03T10:16:56.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.876+0000 7f61a70df700 1 --2- 172.21.15.3:0/2411899138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61a009e410 0x7f61a01c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.876+0000 7f61a70df700 1 --2- 172.21.15.3:0/2411899138 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f61a009d6f0 0x7f61a01c5050 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.876+0000 7f61a70df700 1 --2- 172.21.15.3:0/2411899138 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61a009e000 0x7f61a01c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.876+0000 7f61a70df700 1 -- 172.21.15.3:0/2411899138 >> 172.21.15.3:0/2411899138 conn(0x7f61a0066f50 msgr2=0x7f61a01bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:56.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.876+0000 7f61a70df700 1 -- 172.21.15.3:0/2411899138 shutdown_connections 2023-10-03T10:16:56.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.876+0000 7f61a70df700 1 -- 172.21.15.3:0/2411899138 wait complete. 2023-10-03T10:16:56.889 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:56.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.922+0000 7f5e78268700 1 -- 172.21.15.3:0/2472148131 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5e70072980 con 0x7f5e700a0a30 2023-10-03T10:16:56.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.923+0000 7f5e66ffd700 1 -- 172.21.15.3:0/2472148131 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5e6000b3e0 con 0x7f5e700a0a30 2023-10-03T10:16:56.923 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:56.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.925+0000 7f5e78268700 1 -- 172.21.15.3:0/2472148131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e700a0a30 msgr2=0x7f5e70120b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:56.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.925+0000 7f5e78268700 1 --2- 172.21.15.3:0/2472148131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e700a0a30 0x7f5e70120b80 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f5e60000c00 tx=0x7f5e60004750 comp rx=0 tx=0).stop 2023-10-03T10:16:56.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.925+0000 7f5e78268700 1 -- 172.21.15.3:0/2472148131 shutdown_connections 2023-10-03T10:16:56.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.925+0000 7f5e78268700 1 --2- 172.21.15.3:0/2472148131 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e70138d80 0x7f5e701210c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.925+0000 7f5e78268700 1 --2- 172.21.15.3:0/2472148131 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e7009d630 0x7f5e70120640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.925+0000 7f5e78268700 1 --2- 172.21.15.3:0/2472148131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e700a0a30 0x7f5e70120b80 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:56.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.925+0000 7f5e78268700 1 -- 172.21.15.3:0/2472148131 >> 172.21.15.3:0/2472148131 conn(0x7f5e700a3050 msgr2=0x7f5e70139150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:56.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.925+0000 7f5e78268700 1 -- 172.21.15.3:0/2472148131 shutdown_connections 2023-10-03T10:16:56.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:56.925+0000 7f5e78268700 1 -- 172.21.15.3:0/2472148131 wait complete. 2023-10-03T10:16:56.939 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:57.641 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:57.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.787+0000 7ff3128a6700 1 Processor -- start 2023-10-03T10:16:57.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.787+0000 7ff3128a6700 1 -- start start 2023-10-03T10:16:57.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.788+0000 7ff3128a6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff30c09e1d0 0x7ff30c138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:57.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.788+0000 7ff3128a6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff30c139170 0x7ff30c13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:57.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.788+0000 7ff3128a6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff30c09b120 con 0x7ff30c09d6f0 2023-10-03T10:16:57.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.788+0000 7ff3128a6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff30c06b460 con 0x7ff30c09e1d0 2023-10-03T10:16:57.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.788+0000 7ff3128a6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c09a7e0 con 0x7ff30c139170 2023-10-03T10:16:57.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.788+0000 7ff310e43700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff30c139170 0x7ff30c13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:57.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.788+0000 7ff30b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff30c09e1d0 0x7ff30c138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:57.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.788+0000 7ff30b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff30c09e1d0 0x7ff30c138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48860/0 (socket says 172.21.15.3:48860) 2023-10-03T10:16:57.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.788+0000 7ff310e43700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff30c139170 0x7ff30c13b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33348/0 (socket says 172.21.15.3:33348) 2023-10-03T10:16:57.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.788+0000 7ff30b7fe700 1 -- 172.21.15.3:0/3093456523 learned_addr learned my addr 172.21.15.3:0/3093456523 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:57.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.788+0000 7ff30b7fe700 1 -- 172.21.15.3:0/3093456523 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff30c139170 msgr2=0x7ff30c13b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:57.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.788+0000 7ff30b7fe700 1 --2- 172.21.15.3:0/3093456523 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff30c139170 0x7ff30c13b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:57.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.788+0000 7ff30b7fe700 1 -- 172.21.15.3:0/3093456523 >> v1:172.21.15.3:6789/0 conn(0x7ff30c09d6f0 legacy=0x7ff30c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:57.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.788+0000 7ff30b7fe700 1 -- 172.21.15.3:0/3093456523 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30c09a500 con 0x7ff30c09e1d0 2023-10-03T10:16:57.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.788+0000 7ff310e43700 1 --2- 172.21.15.3:0/3093456523 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff30c139170 0x7ff30c13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:57.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.788+0000 7ff30b7fe700 1 --2- 172.21.15.3:0/3093456523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c09e1d0 0x7ff30c138c30 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009920 tx=0x7ff2f4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e0f4be2444a520 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:57.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.788+0000 7ff30affd700 1 -- 172.21.15.3:0/3093456523 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4011830 con 0x7ff30c09e1d0 2023-10-03T10:16:57.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.788+0000 7ff30affd700 1 -- 172.21.15.3:0/3093456523 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f4011990 con 0x7ff30c09e1d0 2023-10-03T10:16:57.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.788+0000 7ff30affd700 1 -- 172.21.15.3:0/3093456523 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4011d20 con 0x7ff30c09e1d0 2023-10-03T10:16:57.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.789+0000 7ff3128a6700 1 -- 172.21.15.3:0/3093456523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c09e1d0 msgr2=0x7ff30c138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:57.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.789+0000 7ff3128a6700 1 --2- 172.21.15.3:0/3093456523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c09e1d0 0x7ff30c138c30 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009920 tx=0x7ff2f4009c30 comp rx=0 tx=0).stop 2023-10-03T10:16:57.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.789+0000 7ff3128a6700 1 -- 172.21.15.3:0/3093456523 shutdown_connections 2023-10-03T10:16:57.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.789+0000 7ff3128a6700 1 --2- 172.21.15.3:0/3093456523 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff30c139170 0x7ff30c13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:57.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.789+0000 7ff3128a6700 1 --2- 172.21.15.3:0/3093456523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c09e1d0 0x7ff30c138c30 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:57.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.789+0000 7ff3128a6700 1 -- 172.21.15.3:0/3093456523 >> 172.21.15.3:0/3093456523 conn(0x7ff30c066f50 msgr2=0x7ff30c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:57.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.789+0000 7ff3128a6700 1 -- 172.21.15.3:0/3093456523 shutdown_connections 2023-10-03T10:16:57.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.789+0000 7ff3128a6700 1 -- 172.21.15.3:0/3093456523 wait complete. 2023-10-03T10:16:57.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.790+0000 7ff3128a6700 1 Processor -- start 2023-10-03T10:16:57.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.790+0000 7ff3128a6700 1 -- start start 2023-10-03T10:16:57.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.791+0000 7ff3128a6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff30c09d6f0 0x7ff30c11f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:57.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.791+0000 7ff3128a6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff30c09e1d0 0x7ff30c11fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:57.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.791+0000 7ff3128a6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c139170 0x7ff30c1c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:57.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.791+0000 7ff3128a6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c121f20 con 0x7ff30c139170 2023-10-03T10:16:57.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.791+0000 7ff3128a6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c0a1b40 con 0x7ff30c09e1d0 2023-10-03T10:16:57.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.791+0000 7ff3128a6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c09c710 con 0x7ff30c09d6f0 2023-10-03T10:16:57.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.791+0000 7ff310e43700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c139170 0x7ff30c1c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:57.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.791+0000 7ff30bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff30c09d6f0 0x7ff30c11f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:57.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.791+0000 7ff310e43700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c139170 0x7ff30c1c8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48862/0 (socket says 172.21.15.3:48862) 2023-10-03T10:16:57.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.791+0000 7ff30b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff30c09e1d0 0x7ff30c11fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:57.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.791+0000 7ff310e43700 1 -- 172.21.15.3:0/1530940328 learned_addr learned my addr 172.21.15.3:0/1530940328 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:57.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.791+0000 7ff30b7fe700 1 -- 172.21.15.3:0/1530940328 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff30c09d6f0 msgr2=0x7ff30c11f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:57.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.791+0000 7ff30b7fe700 1 --2- 172.21.15.3:0/1530940328 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff30c09d6f0 0x7ff30c11f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:57.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.791+0000 7ff30b7fe700 1 -- 172.21.15.3:0/1530940328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c139170 msgr2=0x7ff30c1c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:57.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.791+0000 7ff30b7fe700 1 --2- 172.21.15.3:0/1530940328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c139170 0x7ff30c1c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:57.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.791+0000 7ff30b7fe700 1 -- 172.21.15.3:0/1530940328 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff304009a70 con 0x7ff30c09e1d0 2023-10-03T10:16:57.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.791+0000 7ff30bfff700 1 --2- 172.21.15.3:0/1530940328 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff30c09d6f0 0x7ff30c11f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:57.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.791+0000 7ff310e43700 1 --2- 172.21.15.3:0/1530940328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c139170 0x7ff30c1c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:57.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.792+0000 7ff30b7fe700 1 --2- 172.21.15.3:0/1530940328 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff30c09e1d0 0x7ff30c11fa60 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4016000 tx=0x7ff2f4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:57.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.792+0000 7ff308ff9700 1 -- 172.21.15.3:0/1530940328 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4004510 con 0x7ff30c09e1d0 2023-10-03T10:16:57.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.792+0000 7ff308ff9700 1 -- 172.21.15.3:0/1530940328 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f40036a0 con 0x7ff30c09e1d0 2023-10-03T10:16:57.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.792+0000 7ff308ff9700 1 -- 172.21.15.3:0/1530940328 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4003a50 con 0x7ff30c09e1d0 2023-10-03T10:16:57.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.792+0000 7ff3128a6700 1 -- 172.21.15.3:0/1530940328 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f4009550 con 0x7ff30c09e1d0 2023-10-03T10:16:57.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.792+0000 7ff3128a6700 1 -- 172.21.15.3:0/1530940328 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff30c11ffa0 con 0x7ff30c09e1d0 2023-10-03T10:16:57.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.793+0000 7ff3128a6700 1 -- 172.21.15.3:0/1530940328 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff30c09c920 con 0x7ff30c09e1d0 2023-10-03T10:16:57.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.793+0000 7ff308ff9700 1 -- 172.21.15.3:0/1530940328 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff2f4003bb0 con 0x7ff30c09e1d0 2023-10-03T10:16:57.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.794+0000 7ff308ff9700 1 -- 172.21.15.3:0/1530940328 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff2f4021030 con 0x7ff30c09e1d0 2023-10-03T10:16:57.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.798+0000 7ff308ff9700 1 -- 172.21.15.3:0/1530940328 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff2f406c750 con 0x7ff30c09e1d0 2023-10-03T10:16:57.851 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:57.890 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:57.939 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:57.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.990+0000 7fa3e4042700 1 Processor -- start 2023-10-03T10:16:57.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.991+0000 7fa3e4042700 1 -- start start 2023-10-03T10:16:57.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.991+0000 7fa3e4042700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3dc1338a0 0x7fa3dc133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:57.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.991+0000 7fa3e4042700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3dc09e930 0x7fa3dc13b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:57.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.991+0000 7fa3e4042700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3dc06ef30 con 0x7fa3dc09de50 2023-10-03T10:16:57.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.991+0000 7fa3e4042700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3dc06fef0 con 0x7fa3dc09e930 2023-10-03T10:16:57.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.991+0000 7fa3e4042700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc1334b0 con 0x7fa3dc1338a0 2023-10-03T10:16:57.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.991+0000 7fa3e15dd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa3dc09de50 0x7fa3dc09e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39902/0 (socket says 172.21.15.3:39902) 2023-10-03T10:16:57.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.991+0000 7fa3e15dd700 1 -- 172.21.15.3:0/2406380053 learned_addr learned my addr 172.21.15.3:0/2406380053 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:57.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.991+0000 7fa3e25df700 1 --2- 172.21.15.3:0/2406380053 >> v2:172.21.15.3:3300/0 conn(0x7fa3dc09e930 0x7fa3dc13b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:57.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.992+0000 7fa3e25df700 1 -- 172.21.15.3:0/2406380053 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3dc1338a0 msgr2=0x7fa3dc133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:57.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.992+0000 7fa3e25df700 1 --2- 172.21.15.3:0/2406380053 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3dc1338a0 0x7fa3dc133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:57.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.992+0000 7fa3e25df700 1 -- 172.21.15.3:0/2406380053 >> v1:172.21.15.3:6789/0 conn(0x7fa3dc09de50 legacy=0x7fa3dc09e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:57.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.992+0000 7fa3e25df700 1 -- 172.21.15.3:0/2406380053 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc06fc30 con 0x7fa3dc09e930 2023-10-03T10:16:57.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.992+0000 7fa3e25df700 1 --2- 172.21.15.3:0/2406380053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3dc09e930 0x7fa3dc13b740 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4009d90 tx=0x7fa3d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25a48ce815e1f73 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:57.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.993+0000 7fa3e0ddc700 1 -- 172.21.15.3:0/2406380053 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d400e070 con 0x7fa3dc09e930 2023-10-03T10:16:57.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.993+0000 7fa3e0ddc700 1 -- 172.21.15.3:0/2406380053 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d400bbd0 con 0x7fa3dc09e930 2023-10-03T10:16:57.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.993+0000 7fa3e0ddc700 1 -- 172.21.15.3:0/2406380053 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4012430 con 0x7fa3dc09e930 2023-10-03T10:16:57.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.993+0000 7fa3e4042700 1 -- 172.21.15.3:0/2406380053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3dc09e930 msgr2=0x7fa3dc13b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:57.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.993+0000 7fa3e4042700 1 --2- 172.21.15.3:0/2406380053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3dc09e930 0x7fa3dc13b740 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4009d90 tx=0x7fa3d400b330 comp rx=0 tx=0).stop 2023-10-03T10:16:57.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.994+0000 7fa3e4042700 1 -- 172.21.15.3:0/2406380053 shutdown_connections 2023-10-03T10:16:57.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.994+0000 7fa3e4042700 1 --2- 172.21.15.3:0/2406380053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3dc09e930 0x7fa3dc13b740 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:57.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.994+0000 7fa3e4042700 1 --2- 172.21.15.3:0/2406380053 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3dc1338a0 0x7fa3dc133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:57.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.994+0000 7fa3e4042700 1 -- 172.21.15.3:0/2406380053 >> 172.21.15.3:0/2406380053 conn(0x7fa3dc0a2aa0 msgr2=0x7fa3dc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:57.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.994+0000 7fa3e4042700 1 -- 172.21.15.3:0/2406380053 shutdown_connections 2023-10-03T10:16:57.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.994+0000 7fa3e4042700 1 -- 172.21.15.3:0/2406380053 wait complete. 2023-10-03T10:16:57.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.995+0000 7fa3e4042700 1 Processor -- start 2023-10-03T10:16:57.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.995+0000 7fa3e4042700 1 -- start start 2023-10-03T10:16:57.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.995+0000 7fa3e4042700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3dc09de50 0x7fa3dc1ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:57.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.995+0000 7fa3e4042700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3dc09e930 0x7fa3dc1cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:57.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.995+0000 7fa3e4042700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3dc1338a0 0x7fa3dc099a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:57.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.995+0000 7fa3e4042700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc122e10 con 0x7fa3dc09e930 2023-10-03T10:16:57.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.995+0000 7fa3e4042700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc09c120 con 0x7fa3dc1338a0 2023-10-03T10:16:57.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.995+0000 7fa3e4042700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc06f640 con 0x7fa3dc09de50 2023-10-03T10:16:57.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.996+0000 7fa3e15dd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3dc09e930 0x7fa3dc1cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:57.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.996+0000 7fa3e15dd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3dc09e930 0x7fa3dc1cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48890/0 (socket says 172.21.15.3:48890) 2023-10-03T10:16:57.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.996+0000 7fa3e15dd700 1 -- 172.21.15.3:0/1631133007 learned_addr learned my addr 172.21.15.3:0/1631133007 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:57.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.996+0000 7fa3e25df700 1 --2- 172.21.15.3:0/1631133007 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3dc1338a0 0x7fa3dc099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:57.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.996+0000 7fa3e15dd700 1 -- 172.21.15.3:0/1631133007 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3dc09de50 msgr2=0x7fa3dc1ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:57.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.996+0000 7fa3e15dd700 1 --2- 172.21.15.3:0/1631133007 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3dc09de50 0x7fa3dc1ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:57.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.996+0000 7fa3e15dd700 1 -- 172.21.15.3:0/1631133007 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3dc1338a0 msgr2=0x7fa3dc099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:57.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.996+0000 7fa3e15dd700 1 --2- 172.21.15.3:0/1631133007 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3dc1338a0 0x7fa3dc099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:57.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.996+0000 7fa3e15dd700 1 -- 172.21.15.3:0/1631133007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d4009a70 con 0x7fa3dc09e930 2023-10-03T10:16:57.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.996+0000 7fa3e25df700 1 --2- 172.21.15.3:0/1631133007 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3dc1338a0 0x7fa3dc099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:57.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.996+0000 7fa3e15dd700 1 --2- 172.21.15.3:0/1631133007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3dc09e930 0x7fa3dc1cd0c0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00c450 tx=0x7fa3cc00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:57.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.996+0000 7fa3d27fc700 1 -- 172.21.15.3:0/1631133007 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc005d70 con 0x7fa3dc09e930 2023-10-03T10:16:57.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.997+0000 7fa3d27fc700 1 -- 172.21.15.3:0/1631133007 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3cc0070d0 con 0x7fa3dc09e930 2023-10-03T10:16:57.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.997+0000 7fa3d27fc700 1 -- 172.21.15.3:0/1631133007 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc007440 con 0x7fa3dc09e930 2023-10-03T10:16:57.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.997+0000 7fa3e4042700 1 -- 172.21.15.3:0/1631133007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3dc09d590 con 0x7fa3dc09e930 2023-10-03T10:16:57.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.997+0000 7fa3e4042700 1 -- 172.21.15.3:0/1631133007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3dc09d2b0 con 0x7fa3dc09e930 2023-10-03T10:16:57.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.998+0000 7fa3e4042700 1 -- 172.21.15.3:0/1631133007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3dc074ea0 con 0x7fa3dc09e930 2023-10-03T10:16:57.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.998+0000 7fa3d27fc700 1 -- 172.21.15.3:0/1631133007 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa3cc0075a0 con 0x7fa3dc09e930 2023-10-03T10:16:57.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:57.999+0000 7fa3d27fc700 1 -- 172.21.15.3:0/1631133007 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa3cc03ea40 con 0x7fa3dc09e930 2023-10-03T10:16:58.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.008+0000 7fa3d27fc700 1 -- 172.21.15.3:0/1631133007 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa3cc019d20 con 0x7fa3dc09e930 2023-10-03T10:16:58.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.013+0000 7ff3128a6700 1 -- 172.21.15.3:0/1530940328 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff30c09f420 con 0x7ff30c09e1d0 2023-10-03T10:16:58.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.013+0000 7ff308ff9700 1 -- 172.21.15.3:0/1530940328 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff2f4034090 con 0x7ff30c09e1d0 2023-10-03T10:16:58.014 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:58.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.017+0000 7ff3128a6700 1 -- 172.21.15.3:0/1530940328 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff30c09e1d0 msgr2=0x7ff30c11fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:58.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.017+0000 7ff3128a6700 1 --2- 172.21.15.3:0/1530940328 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff30c09e1d0 0x7ff30c11fa60 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4016000 tx=0x7ff2f4004750 comp rx=0 tx=0).stop 2023-10-03T10:16:58.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.018+0000 7ff3128a6700 1 -- 172.21.15.3:0/1530940328 shutdown_connections 2023-10-03T10:16:58.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.018+0000 7ff3128a6700 1 --2- 172.21.15.3:0/1530940328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c139170 0x7ff30c1c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:58.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.018+0000 7ff3128a6700 1 --2- 172.21.15.3:0/1530940328 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff30c09d6f0 0x7ff30c11f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:58.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.018+0000 7ff3128a6700 1 --2- 172.21.15.3:0/1530940328 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff30c09e1d0 0x7ff30c11fa60 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:58.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.018+0000 7ff3128a6700 1 -- 172.21.15.3:0/1530940328 >> 172.21.15.3:0/1530940328 conn(0x7ff30c066f50 msgr2=0x7ff30c136b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:58.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.018+0000 7ff3128a6700 1 -- 172.21.15.3:0/1530940328 shutdown_connections 2023-10-03T10:16:58.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.018+0000 7ff3128a6700 1 -- 172.21.15.3:0/1530940328 wait complete. 2023-10-03T10:16:58.033 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:58.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.040+0000 7f4a96d9f700 1 Processor -- start 2023-10-03T10:16:58.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.040+0000 7f4a96d9f700 1 -- start start 2023-10-03T10:16:58.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.041+0000 7f4a96d9f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4a900a0a30 0x7f4a900a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:58.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.041+0000 7f4a96d9f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a9009d630 0x7f4a900a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:58.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.041+0000 7f4a96d9f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a900a2a70 con 0x7f4a90133930 2023-10-03T10:16:58.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.041+0000 7f4a96d9f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a900a01d0 con 0x7f4a900a0a30 2023-10-03T10:16:58.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.041+0000 7f4a96d9f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9009e2b0 con 0x7f4a9009d630 2023-10-03T10:16:58.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.041+0000 7f4a94b3b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4a900a0a30 0x7f4a900a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:58.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.041+0000 7f4a8ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a9009d630 0x7f4a900a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:58.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.041+0000 7f4a94b3b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4a900a0a30 0x7f4a900a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48896/0 (socket says 172.21.15.3:48896) 2023-10-03T10:16:58.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.041+0000 7f4a94b3b700 1 -- 172.21.15.3:0/1220513023 learned_addr learned my addr 172.21.15.3:0/1220513023 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:58.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.041+0000 7f4a9533c700 1 --1- 172.21.15.3:0/1220513023 >> v1:172.21.15.3:6789/0 conn(0x7f4a90133930 0x7f4a90133d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39918/0 (socket says 172.21.15.3:39918) 2023-10-03T10:16:58.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.042+0000 7f4a94b3b700 1 -- 172.21.15.3:0/1220513023 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a9009d630 msgr2=0x7f4a900a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:58.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.042+0000 7f4a94b3b700 1 --2- 172.21.15.3:0/1220513023 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a9009d630 0x7f4a900a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:58.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.042+0000 7f4a94b3b700 1 -- 172.21.15.3:0/1220513023 >> v1:172.21.15.3:6789/0 conn(0x7f4a90133930 legacy=0x7f4a90133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:58.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.042+0000 7f4a94b3b700 1 -- 172.21.15.3:0/1220513023 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a9009dc50 con 0x7f4a900a0a30 2023-10-03T10:16:58.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.042+0000 7f4a8f7fe700 1 -- 172.21.15.3:0/1220513023 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 356010696 0 0) 0x7f4a900a2a70 con 0x7f4a90133930 2023-10-03T10:16:58.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.042+0000 7f4a8ffff700 1 --2- 172.21.15.3:0/1220513023 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a9009d630 0x7f4a900a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:58.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.042+0000 7f4a94b3b700 1 --2- 172.21.15.3:0/1220513023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a900a0a30 0x7f4a900a0e00 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f4a840098f0 tx=0x7f4a84009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52136185a4bfa29 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:58.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.042+0000 7f4a8f7fe700 1 -- 172.21.15.3:0/1220513023 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a84011830 con 0x7f4a900a0a30 2023-10-03T10:16:58.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.042+0000 7f4a8f7fe700 1 -- 172.21.15.3:0/1220513023 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a84011990 con 0x7f4a900a0a30 2023-10-03T10:16:58.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.042+0000 7f4a8f7fe700 1 -- 172.21.15.3:0/1220513023 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a84011d20 con 0x7f4a900a0a30 2023-10-03T10:16:58.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.043+0000 7f4a96d9f700 1 -- 172.21.15.3:0/1220513023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a900a0a30 msgr2=0x7f4a900a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:58.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.043+0000 7f4a96d9f700 1 --2- 172.21.15.3:0/1220513023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a900a0a30 0x7f4a900a0e00 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f4a840098f0 tx=0x7f4a84009c00 comp rx=0 tx=0).stop 2023-10-03T10:16:58.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.043+0000 7f4a96d9f700 1 -- 172.21.15.3:0/1220513023 shutdown_connections 2023-10-03T10:16:58.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.043+0000 7f4a96d9f700 1 --2- 172.21.15.3:0/1220513023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a900a0a30 0x7f4a900a0e00 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:58.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.043+0000 7f4a96d9f700 1 --2- 172.21.15.3:0/1220513023 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a9009d630 0x7f4a900a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:58.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.043+0000 7f4a96d9f700 1 -- 172.21.15.3:0/1220513023 >> 172.21.15.3:0/1220513023 conn(0x7f4a900a3050 msgr2=0x7f4a900a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:58.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.043+0000 7f4a96d9f700 1 -- 172.21.15.3:0/1220513023 shutdown_connections 2023-10-03T10:16:58.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.043+0000 7f4a96d9f700 1 -- 172.21.15.3:0/1220513023 wait complete. 2023-10-03T10:16:58.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.044+0000 7f4a96d9f700 1 Processor -- start 2023-10-03T10:16:58.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.044+0000 7f4a96d9f700 1 -- start start 2023-10-03T10:16:58.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.045+0000 7f4a96d9f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a9009d630 0x7f4a901c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:58.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.045+0000 7f4a96d9f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a900a0a30 0x7f4a901c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:58.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.045+0000 7f4a96d9f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a90133930 0x7f4a901c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:58.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.045+0000 7f4a96d9f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a90120620 con 0x7f4a900a0a30 2023-10-03T10:16:58.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.045+0000 7f4a96d9f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a900a42c0 con 0x7f4a9009d630 2023-10-03T10:16:58.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.045+0000 7f4a96d9f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a900a0410 con 0x7f4a90133930 2023-10-03T10:16:58.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.045+0000 7f4a94b3b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a9009d630 0x7f4a901c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:58.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.045+0000 7f4a94b3b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a9009d630 0x7f4a901c7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33390/0 (socket says 172.21.15.3:33390) 2023-10-03T10:16:58.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.045+0000 7f4a94b3b700 1 -- 172.21.15.3:0/449086912 learned_addr learned my addr 172.21.15.3:0/449086912 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:58.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.045+0000 7f4a9533c700 1 --2- 172.21.15.3:0/449086912 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a90133930 0x7f4a901c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:58.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.045+0000 7f4a8ffff700 1 --2- 172.21.15.3:0/449086912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a900a0a30 0x7f4a901c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:58.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.045+0000 7f4a94b3b700 1 -- 172.21.15.3:0/449086912 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a90133930 msgr2=0x7f4a901c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:58.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.045+0000 7f4a94b3b700 1 --2- 172.21.15.3:0/449086912 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a90133930 0x7f4a901c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:58.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.045+0000 7f4a94b3b700 1 -- 172.21.15.3:0/449086912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a900a0a30 msgr2=0x7f4a901c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:58.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.045+0000 7f4a94b3b700 1 --2- 172.21.15.3:0/449086912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a900a0a30 0x7f4a901c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:58.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.045+0000 7f4a94b3b700 1 -- 172.21.15.3:0/449086912 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a7c0094a0 con 0x7f4a9009d630 2023-10-03T10:16:58.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.046+0000 7f4a94b3b700 1 --2- 172.21.15.3:0/449086912 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a9009d630 0x7f4a901c7e80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4a900a2c00 tx=0x7f4a84004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:58.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.046+0000 7f4a8d7fa700 1 -- 172.21.15.3:0/449086912 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a84004510 con 0x7f4a9009d630 2023-10-03T10:16:58.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.046+0000 7f4a8d7fa700 1 -- 172.21.15.3:0/449086912 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a840036a0 con 0x7f4a9009d630 2023-10-03T10:16:58.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.046+0000 7f4a8d7fa700 1 -- 172.21.15.3:0/449086912 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a84003a50 con 0x7f4a9009d630 2023-10-03T10:16:58.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.046+0000 7f4a96d9f700 1 -- 172.21.15.3:0/449086912 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a84009550 con 0x7f4a9009d630 2023-10-03T10:16:58.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.047+0000 7f4a96d9f700 1 -- 172.21.15.3:0/449086912 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a901334c0 con 0x7f4a9009d630 2023-10-03T10:16:58.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.048+0000 7f4a96d9f700 1 -- 172.21.15.3:0/449086912 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a900747f0 con 0x7f4a9009d630 2023-10-03T10:16:58.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.048+0000 7f4a8d7fa700 1 -- 172.21.15.3:0/449086912 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4a84029070 con 0x7f4a9009d630 2023-10-03T10:16:58.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.048+0000 7f4a8d7fa700 1 -- 172.21.15.3:0/449086912 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4a84049650 con 0x7f4a9009d630 2023-10-03T10:16:58.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.053+0000 7f4a8d7fa700 1 -- 172.21.15.3:0/449086912 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4a84035760 con 0x7f4a9009d630 2023-10-03T10:16:58.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.085+0000 7f71da85d700 1 Processor -- start 2023-10-03T10:16:58.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.086+0000 7f71da85d700 1 -- start start 2023-10-03T10:16:58.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.086+0000 7f71da85d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f71d40a0a30 0x7f71d40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:58.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.086+0000 7f71da85d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71d40a1340 0x7f71d4133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:58.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.086+0000 7f71da85d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71d409edd0 con 0x7f71d4133fa0 2023-10-03T10:16:58.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.086+0000 7f71da85d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71d409e1f0 con 0x7f71d40a0a30 2023-10-03T10:16:58.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.086+0000 7f71da85d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71d409dc50 con 0x7f71d40a1340 2023-10-03T10:16:58.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.086+0000 7f71d3fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f71d40a0a30 0x7f71d40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:58.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.086+0000 7f71d8dfa700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f71d4133fa0 0x7f71d413ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39930/0 (socket says 172.21.15.3:39930) 2023-10-03T10:16:58.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.086+0000 7f71d37fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71d40a1340 0x7f71d4133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:58.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.086+0000 7f71d8dfa700 1 -- 172.21.15.3:0/344586 learned_addr learned my addr 172.21.15.3:0/344586 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:58.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.087+0000 7f71d3fff700 1 -- 172.21.15.3:0/344586 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71d40a1340 msgr2=0x7f71d4133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:58.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.087+0000 7f71d3fff700 1 --2- 172.21.15.3:0/344586 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71d40a1340 0x7f71d4133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:58.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.087+0000 7f71d3fff700 1 -- 172.21.15.3:0/344586 >> v1:172.21.15.3:6789/0 conn(0x7f71d4133fa0 legacy=0x7f71d413ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:58.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.087+0000 7f71d3fff700 1 -- 172.21.15.3:0/344586 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d40a1fe0 con 0x7f71d40a0a30 2023-10-03T10:16:58.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.087+0000 7f71d2ffd700 1 -- 172.21.15.3:0/344586 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2494277978 0 0) 0x7f71d409edd0 con 0x7f71d4133fa0 2023-10-03T10:16:58.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.087+0000 7f71d3fff700 1 --2- 172.21.15.3:0/344586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71d40a0a30 0x7f71d40a0e00 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009840 tx=0x7f71c4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e938322b2f957f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:58.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.087+0000 7f71d2ffd700 1 -- 172.21.15.3:0/344586 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4011830 con 0x7f71d40a0a30 2023-10-03T10:16:58.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.087+0000 7f71d2ffd700 1 -- 172.21.15.3:0/344586 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c4011990 con 0x7f71d40a0a30 2023-10-03T10:16:58.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.087+0000 7f71d2ffd700 1 -- 172.21.15.3:0/344586 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4011d20 con 0x7f71d40a0a30 2023-10-03T10:16:58.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.088+0000 7f71da85d700 1 -- 172.21.15.3:0/344586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71d40a0a30 msgr2=0x7f71d40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:58.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.088+0000 7f71da85d700 1 --2- 172.21.15.3:0/344586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71d40a0a30 0x7f71d40a0e00 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009840 tx=0x7f71c4009b50 comp rx=0 tx=0).stop 2023-10-03T10:16:58.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.088+0000 7f71da85d700 1 -- 172.21.15.3:0/344586 shutdown_connections 2023-10-03T10:16:58.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.088+0000 7f71da85d700 1 --2- 172.21.15.3:0/344586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71d40a0a30 0x7f71d40a0e00 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:58.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.088+0000 7f71da85d700 1 --2- 172.21.15.3:0/344586 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71d40a1340 0x7f71d4133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:58.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.088+0000 7f71da85d700 1 -- 172.21.15.3:0/344586 >> 172.21.15.3:0/344586 conn(0x7f71d40a6550 msgr2=0x7f71d40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:58.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.088+0000 7f71da85d700 1 -- 172.21.15.3:0/344586 shutdown_connections 2023-10-03T10:16:58.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.088+0000 7f71da85d700 1 -- 172.21.15.3:0/344586 wait complete. 2023-10-03T10:16:58.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.089+0000 7f71da85d700 1 Processor -- start 2023-10-03T10:16:58.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.089+0000 7f71da85d700 1 -- start start 2023-10-03T10:16:58.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.090+0000 7f71da85d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f71d40a0a30 0x7f71d4122890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:58.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.090+0000 7f71da85d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71d40a1340 0x7f71d4122dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:58.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.090+0000 7f71da85d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71d4133fa0 0x7f71d4123310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:58.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.090+0000 7f71da85d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d4124cb0 con 0x7f71d40a1340 2023-10-03T10:16:58.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.090+0000 7f71da85d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d4125d80 con 0x7f71d40a0a30 2023-10-03T10:16:58.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.090+0000 7f71da85d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71d4124520 con 0x7f71d4133fa0 2023-10-03T10:16:58.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.090+0000 7f71d37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71d40a1340 0x7f71d4122dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:58.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.090+0000 7f71d37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71d40a1340 0x7f71d4122dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48920/0 (socket says 172.21.15.3:48920) 2023-10-03T10:16:58.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.090+0000 7f71d37fe700 1 -- 172.21.15.3:0/1034919548 learned_addr learned my addr 172.21.15.3:0/1034919548 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:58.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.090+0000 7f71d8dfa700 1 --2- 172.21.15.3:0/1034919548 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71d4133fa0 0x7f71d4123310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:58.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.090+0000 7f71d3fff700 1 --2- 172.21.15.3:0/1034919548 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f71d40a0a30 0x7f71d4122890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:58.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.090+0000 7f71d8dfa700 1 -- 172.21.15.3:0/1034919548 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f71d40a0a30 msgr2=0x7f71d4122890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:58.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.090+0000 7f71d8dfa700 1 --2- 172.21.15.3:0/1034919548 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f71d40a0a30 0x7f71d4122890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:58.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.090+0000 7f71d8dfa700 1 -- 172.21.15.3:0/1034919548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71d40a1340 msgr2=0x7f71d4122dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:58.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.090+0000 7f71d8dfa700 1 --2- 172.21.15.3:0/1034919548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71d40a1340 0x7f71d4122dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:58.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.090+0000 7f71d8dfa700 1 -- 172.21.15.3:0/1034919548 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c40094a0 con 0x7f71d4133fa0 2023-10-03T10:16:58.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.090+0000 7f71d37fe700 1 --2- 172.21.15.3:0/1034919548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71d40a1340 0x7f71d4122dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:58.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.091+0000 7f71d8dfa700 1 --2- 172.21.15.3:0/1034919548 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71d4133fa0 0x7f71d4123310 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00cbe0 tx=0x7f71cc00cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:58.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.091+0000 7f71d0ff9700 1 -- 172.21.15.3:0/1034919548 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc00aa40 con 0x7f71d4133fa0 2023-10-03T10:16:58.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.091+0000 7f71d0ff9700 1 -- 172.21.15.3:0/1034919548 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71cc00abc0 con 0x7f71d4133fa0 2023-10-03T10:16:58.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.091+0000 7f71da85d700 1 -- 172.21.15.3:0/1034919548 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71d40a35a0 con 0x7f71d4133fa0 2023-10-03T10:16:58.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.091+0000 7f71d0ff9700 1 -- 172.21.15.3:0/1034919548 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc0070d0 con 0x7f71d4133fa0 2023-10-03T10:16:58.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.091+0000 7f71da85d700 1 -- 172.21.15.3:0/1034919548 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71d4133300 con 0x7f71d4133fa0 2023-10-03T10:16:58.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.092+0000 7f71d0ff9700 1 -- 172.21.15.3:0/1034919548 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f71cc007230 con 0x7f71d4133fa0 2023-10-03T10:16:58.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.092+0000 7f71d0ff9700 1 -- 172.21.15.3:0/1034919548 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f71cc03f080 con 0x7f71d4133fa0 2023-10-03T10:16:58.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.092+0000 7f71da85d700 1 -- 172.21.15.3:0/1034919548 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71d40a1d40 con 0x7f71d4133fa0 2023-10-03T10:16:58.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.097+0000 7f71d0ff9700 1 -- 172.21.15.3:0/1034919548 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f71cc018070 con 0x7f71d4133fa0 2023-10-03T10:16:58.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.222+0000 7fa3e4042700 1 -- 172.21.15.3:0/1631133007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa3dc09d080 con 0x7fa3dc09e930 2023-10-03T10:16:58.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.223+0000 7fa3d27fc700 1 -- 172.21.15.3:0/1631133007 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa3cc017020 con 0x7fa3dc09e930 2023-10-03T10:16:58.223 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:58.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.225+0000 7fa3e4042700 1 -- 172.21.15.3:0/1631133007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3dc09e930 msgr2=0x7fa3dc1cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:58.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.225+0000 7fa3e4042700 1 --2- 172.21.15.3:0/1631133007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3dc09e930 0x7fa3dc1cd0c0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00c450 tx=0x7fa3cc00c760 comp rx=0 tx=0).stop 2023-10-03T10:16:58.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.225+0000 7fa3e4042700 1 -- 172.21.15.3:0/1631133007 shutdown_connections 2023-10-03T10:16:58.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.225+0000 7fa3e4042700 1 --2- 172.21.15.3:0/1631133007 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3dc1338a0 0x7fa3dc099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:58.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.225+0000 7fa3e4042700 1 --2- 172.21.15.3:0/1631133007 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3dc09de50 0x7fa3dc1ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:58.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.225+0000 7fa3e4042700 1 --2- 172.21.15.3:0/1631133007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3dc09e930 0x7fa3dc1cd0c0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:58.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.225+0000 7fa3e4042700 1 -- 172.21.15.3:0/1631133007 >> 172.21.15.3:0/1631133007 conn(0x7fa3dc0a2aa0 msgr2=0x7fa3dc130730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:58.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.225+0000 7fa3e4042700 1 -- 172.21.15.3:0/1631133007 shutdown_connections 2023-10-03T10:16:58.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.226+0000 7fa3e4042700 1 -- 172.21.15.3:0/1631133007 wait complete. 2023-10-03T10:16:58.239 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:58.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.261+0000 7f4a96d9f700 1 -- 172.21.15.3:0/449086912 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4a90072980 con 0x7f4a9009d630 2023-10-03T10:16:58.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.262+0000 7f4a8d7fa700 1 -- 172.21.15.3:0/449086912 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4a84015030 con 0x7f4a9009d630 2023-10-03T10:16:58.262 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:58.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.265+0000 7f4a96d9f700 1 -- 172.21.15.3:0/449086912 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a9009d630 msgr2=0x7f4a901c7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:58.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.265+0000 7f4a96d9f700 1 --2- 172.21.15.3:0/449086912 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a9009d630 0x7f4a901c7e80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4a900a2c00 tx=0x7f4a84004750 comp rx=0 tx=0).stop 2023-10-03T10:16:58.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.265+0000 7f4a96d9f700 1 -- 172.21.15.3:0/449086912 shutdown_connections 2023-10-03T10:16:58.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.265+0000 7f4a96d9f700 1 --2- 172.21.15.3:0/449086912 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a90133930 0x7f4a901c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:58.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.265+0000 7f4a96d9f700 1 --2- 172.21.15.3:0/449086912 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a9009d630 0x7f4a901c7e80 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:58.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.265+0000 7f4a96d9f700 1 --2- 172.21.15.3:0/449086912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a900a0a30 0x7f4a901c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:58.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.265+0000 7f4a96d9f700 1 -- 172.21.15.3:0/449086912 >> 172.21.15.3:0/449086912 conn(0x7f4a900a3050 msgr2=0x7f4a901bb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:58.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.265+0000 7f4a96d9f700 1 -- 172.21.15.3:0/449086912 shutdown_connections 2023-10-03T10:16:58.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.265+0000 7f4a96d9f700 1 -- 172.21.15.3:0/449086912 wait complete. 2023-10-03T10:16:58.280 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:58.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.305+0000 7f71da85d700 1 -- 172.21.15.3:0/1034919548 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f71d4074ea0 con 0x7f71d4133fa0 2023-10-03T10:16:58.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.305+0000 7f71d0ff9700 1 -- 172.21.15.3:0/1034919548 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f71cc0074e0 con 0x7f71d4133fa0 2023-10-03T10:16:58.305 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:58.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.307+0000 7f71da85d700 1 -- 172.21.15.3:0/1034919548 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71d4133fa0 msgr2=0x7f71d4123310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:58.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.307+0000 7f71da85d700 1 --2- 172.21.15.3:0/1034919548 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71d4133fa0 0x7f71d4123310 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00cbe0 tx=0x7f71cc00cef0 comp rx=0 tx=0).stop 2023-10-03T10:16:58.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.307+0000 7f71da85d700 1 -- 172.21.15.3:0/1034919548 shutdown_connections 2023-10-03T10:16:58.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.307+0000 7f71da85d700 1 --2- 172.21.15.3:0/1034919548 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f71d4133fa0 0x7f71d4123310 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:58.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.307+0000 7f71da85d700 1 --2- 172.21.15.3:0/1034919548 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f71d40a0a30 0x7f71d4122890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:58.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.307+0000 7f71da85d700 1 --2- 172.21.15.3:0/1034919548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71d40a1340 0x7f71d4122dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:58.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.307+0000 7f71da85d700 1 -- 172.21.15.3:0/1034919548 >> 172.21.15.3:0/1034919548 conn(0x7f71d40a6550 msgr2=0x7f71d411c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:58.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.307+0000 7f71da85d700 1 -- 172.21.15.3:0/1034919548 shutdown_connections 2023-10-03T10:16:58.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:58.307+0000 7f71da85d700 1 -- 172.21.15.3:0/1034919548 wait complete. 2023-10-03T10:16:58.321 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:16:59.035 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:16:59.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.181+0000 7fe3f1b98700 1 Processor -- start 2023-10-03T10:16:59.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.181+0000 7fe3f1b98700 1 -- start start 2023-10-03T10:16:59.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.181+0000 7fe3f1b98700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3ec09d6f0 0x7fe3ec09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:59.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.181+0000 7fe3f1b98700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3ec09e000 0x7fe3ec130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:59.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.181+0000 7fe3f1b98700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe3ec09e410 0x7fe3ec130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:59.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.181+0000 7fe3f1b98700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3ec09f520 con 0x7fe3ec09e410 2023-10-03T10:16:59.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.181+0000 7fe3f1b98700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ec09f130 con 0x7fe3ec09e000 2023-10-03T10:16:59.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.181+0000 7fe3f1b98700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ec09e930 con 0x7fe3ec09d6f0 2023-10-03T10:16:59.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.182+0000 7fe3eb7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3ec09d6f0 0x7fe3ec09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:59.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.182+0000 7fe3eaffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3ec09e000 0x7fe3ec130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:59.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.182+0000 7fe3ebfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe3ec09e410 0x7fe3ec130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:59.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.182+0000 7fe3eaffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3ec09e000 0x7fe3ec130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33418/0 (socket says 172.21.15.3:33418) 2023-10-03T10:16:59.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.182+0000 7fe3eb7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3ec09d6f0 0x7fe3ec09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:35930/0 (socket says 172.21.15.3:35930) 2023-10-03T10:16:59.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.182+0000 7fe3eaffd700 1 -- 172.21.15.3:0/1358676809 learned_addr learned my addr 172.21.15.3:0/1358676809 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:59.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.182+0000 7fe3ebfff700 1 -- 172.21.15.3:0/1358676809 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3ec09d6f0 msgr2=0x7fe3ec09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:59.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.182+0000 7fe3ebfff700 1 --2- 172.21.15.3:0/1358676809 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3ec09d6f0 0x7fe3ec09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.182+0000 7fe3ebfff700 1 -- 172.21.15.3:0/1358676809 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3ec09e000 msgr2=0x7fe3ec130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:59.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.182+0000 7fe3ebfff700 1 --2- 172.21.15.3:0/1358676809 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3ec09e000 0x7fe3ec130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.182+0000 7fe3ebfff700 1 -- 172.21.15.3:0/1358676809 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ec09b2d0 con 0x7fe3ec09e410 2023-10-03T10:16:59.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.182+0000 7fe3eaffd700 1 --2- 172.21.15.3:0/1358676809 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3ec09e000 0x7fe3ec130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:59.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.182+0000 7fe3eb7fe700 1 --2- 172.21.15.3:0/1358676809 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3ec09d6f0 0x7fe3ec09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:59.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.182+0000 7fe3ebfff700 1 --2- 172.21.15.3:0/1358676809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3ec09e410 0x7fe3ec130ba0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4009d90 tx=0x7fe3e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41ecd29d3e69217 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:59.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.182+0000 7fe3ea7fc700 1 -- 172.21.15.3:0/1358676809 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e400e070 con 0x7fe3ec09e410 2023-10-03T10:16:59.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.182+0000 7fe3ea7fc700 1 -- 172.21.15.3:0/1358676809 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e400bb20 con 0x7fe3ec09e410 2023-10-03T10:16:59.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.182+0000 7fe3ea7fc700 1 -- 172.21.15.3:0/1358676809 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4012430 con 0x7fe3ec09e410 2023-10-03T10:16:59.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.183+0000 7fe3f1b98700 1 -- 172.21.15.3:0/1358676809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3ec09e410 msgr2=0x7fe3ec130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:59.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.183+0000 7fe3f1b98700 1 --2- 172.21.15.3:0/1358676809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3ec09e410 0x7fe3ec130ba0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4009d90 tx=0x7fe3e400b330 comp rx=0 tx=0).stop 2023-10-03T10:16:59.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.183+0000 7fe3f1b98700 1 -- 172.21.15.3:0/1358676809 shutdown_connections 2023-10-03T10:16:59.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.183+0000 7fe3f1b98700 1 --2- 172.21.15.3:0/1358676809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3ec09e410 0x7fe3ec130ba0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.183+0000 7fe3f1b98700 1 --2- 172.21.15.3:0/1358676809 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3ec09d6f0 0x7fe3ec09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.183+0000 7fe3f1b98700 1 --2- 172.21.15.3:0/1358676809 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3ec09e000 0x7fe3ec130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.183+0000 7fe3f1b98700 1 -- 172.21.15.3:0/1358676809 >> 172.21.15.3:0/1358676809 conn(0x7fe3ec066f50 msgr2=0x7fe3ec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:59.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.183+0000 7fe3f1b98700 1 -- 172.21.15.3:0/1358676809 shutdown_connections 2023-10-03T10:16:59.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.183+0000 7fe3f1b98700 1 -- 172.21.15.3:0/1358676809 wait complete. 2023-10-03T10:16:59.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.184+0000 7fe3f1b98700 1 Processor -- start 2023-10-03T10:16:59.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.184+0000 7fe3f1b98700 1 -- start start 2023-10-03T10:16:59.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.185+0000 7fe3f1b98700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3ec09d6f0 0x7fe3ec1225e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:59.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.185+0000 7fe3f1b98700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3ec09e000 0x7fe3ec122b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:59.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.185+0000 7fe3f1b98700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3ec09e410 0x7fe3ec123060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:59.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.185+0000 7fe3f1b98700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ec124300 con 0x7fe3ec09e000 2023-10-03T10:16:59.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.185+0000 7fe3f1b98700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ec09c700 con 0x7fe3ec09d6f0 2023-10-03T10:16:59.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.185+0000 7fe3f1b98700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ec125b60 con 0x7fe3ec09e410 2023-10-03T10:16:59.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.185+0000 7fe3eb7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3ec09d6f0 0x7fe3ec1225e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:59.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.185+0000 7fe3eaffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3ec09e000 0x7fe3ec122b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:59.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.185+0000 7fe3ebfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3ec09e410 0x7fe3ec123060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:59.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.185+0000 7fe3eb7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3ec09d6f0 0x7fe3ec1225e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33422/0 (socket says 172.21.15.3:33422) 2023-10-03T10:16:59.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.185+0000 7fe3ebfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3ec09e410 0x7fe3ec123060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:35942/0 (socket says 172.21.15.3:35942) 2023-10-03T10:16:59.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.185+0000 7fe3eb7fe700 1 -- 172.21.15.3:0/1528377172 learned_addr learned my addr 172.21.15.3:0/1528377172 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:59.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.185+0000 7fe3eaffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3ec09e000 0x7fe3ec122b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48930/0 (socket says 172.21.15.3:48930) 2023-10-03T10:16:59.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.185+0000 7fe3eb7fe700 1 -- 172.21.15.3:0/1528377172 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3ec09e410 msgr2=0x7fe3ec123060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:59.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.185+0000 7fe3eb7fe700 1 --2- 172.21.15.3:0/1528377172 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3ec09e410 0x7fe3ec123060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.185+0000 7fe3eb7fe700 1 -- 172.21.15.3:0/1528377172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3ec09e000 msgr2=0x7fe3ec122b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:59.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.185+0000 7fe3eb7fe700 1 --2- 172.21.15.3:0/1528377172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3ec09e000 0x7fe3ec122b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.185+0000 7fe3eb7fe700 1 -- 172.21.15.3:0/1528377172 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3dc0094a0 con 0x7fe3ec09d6f0 2023-10-03T10:16:59.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.185+0000 7fe3eaffd700 1 --2- 172.21.15.3:0/1528377172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3ec09e000 0x7fe3ec122b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:59.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.185+0000 7fe3eb7fe700 1 --2- 172.21.15.3:0/1528377172 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3ec09d6f0 0x7fe3ec1225e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc00ebe0 tx=0x7fe3dc00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:59.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.186+0000 7fe3d3fff700 1 -- 172.21.15.3:0/1528377172 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc009780 con 0x7fe3ec09d6f0 2023-10-03T10:16:59.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.186+0000 7fe3d3fff700 1 -- 172.21.15.3:0/1528377172 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3dc00c850 con 0x7fe3ec09d6f0 2023-10-03T10:16:59.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.186+0000 7fe3d3fff700 1 -- 172.21.15.3:0/1528377172 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc00cbe0 con 0x7fe3ec09d6f0 2023-10-03T10:16:59.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.186+0000 7fe3f1b98700 1 -- 172.21.15.3:0/1528377172 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e0009550 con 0x7fe3ec09d6f0 2023-10-03T10:16:59.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.186+0000 7fe3f1b98700 1 -- 172.21.15.3:0/1528377172 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e4009a70 con 0x7fe3ec09d6f0 2023-10-03T10:16:59.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.186+0000 7fe3d3fff700 1 -- 172.21.15.3:0/1528377172 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe3dc00cd40 con 0x7fe3ec09d6f0 2023-10-03T10:16:59.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.187+0000 7fe3d3fff700 1 -- 172.21.15.3:0/1528377172 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe3dc03f760 con 0x7fe3ec09d6f0 2023-10-03T10:16:59.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.187+0000 7fe3f1b98700 1 -- 172.21.15.3:0/1528377172 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3ec074ea0 con 0x7fe3ec09d6f0 2023-10-03T10:16:59.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.191+0000 7fe3d3fff700 1 -- 172.21.15.3:0/1528377172 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe3dc0232f0 con 0x7fe3ec09d6f0 2023-10-03T10:16:59.240 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:16:59.281 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:16:59.322 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:16:59.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.390+0000 7f8a57ccd700 1 Processor -- start 2023-10-03T10:16:59.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.391+0000 7f8a57ccd700 1 -- start start 2023-10-03T10:16:59.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.391+0000 7f8a57ccd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a5009e930 0x7f8a5009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:59.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.391+0000 7f8a57ccd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a5009f650 0x7f8a50131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:59.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.391+0000 7f8a57ccd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a50070940 con 0x7f8a5009f240 2023-10-03T10:16:59.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.391+0000 7f8a57ccd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5006fd60 con 0x7f8a5009e930 2023-10-03T10:16:59.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.391+0000 7f8a57ccd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5006fa80 con 0x7f8a5009f650 2023-10-03T10:16:59.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.392+0000 7f8a55a69700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a5009e930 0x7f8a5009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:59.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.392+0000 7f8a55a69700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a5009e930 0x7f8a5009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33426/0 (socket says 172.21.15.3:33426) 2023-10-03T10:16:59.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.392+0000 7f8a55a69700 1 -- 172.21.15.3:0/2787691732 learned_addr learned my addr 172.21.15.3:0/2787691732 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:59.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.392+0000 7f8a5626a700 1 --2- 172.21.15.3:0/2787691732 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a5009f650 0x7f8a50131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:59.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.392+0000 7f8a55a69700 1 -- 172.21.15.3:0/2787691732 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a5009f650 msgr2=0x7f8a50131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:59.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.392+0000 7f8a55a69700 1 --2- 172.21.15.3:0/2787691732 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a5009f650 0x7f8a50131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.392+0000 7f8a55a69700 1 -- 172.21.15.3:0/2787691732 >> v1:172.21.15.3:6789/0 conn(0x7f8a5009f240 legacy=0x7f8a501317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:59.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.392+0000 7f8a55a69700 1 -- 172.21.15.3:0/2787691732 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5009fb60 con 0x7f8a5009e930 2023-10-03T10:16:59.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.392+0000 7f8a55a69700 1 --2- 172.21.15.3:0/2787691732 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a5009e930 0x7f8a5009ed00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009920 tx=0x7f8a4c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9588e87c48b4490 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:59.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.393+0000 7f8a54a67700 1 -- 172.21.15.3:0/2787691732 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c011830 con 0x7f8a5009e930 2023-10-03T10:16:59.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.393+0000 7f8a54a67700 1 -- 172.21.15.3:0/2787691732 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4c011990 con 0x7f8a5009e930 2023-10-03T10:16:59.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.393+0000 7f8a54a67700 1 -- 172.21.15.3:0/2787691732 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c011d20 con 0x7f8a5009e930 2023-10-03T10:16:59.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.394+0000 7f8a57ccd700 1 -- 172.21.15.3:0/2787691732 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a5009e930 msgr2=0x7f8a5009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:59.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.394+0000 7f8a57ccd700 1 --2- 172.21.15.3:0/2787691732 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a5009e930 0x7f8a5009ed00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009920 tx=0x7f8a4c009c30 comp rx=0 tx=0).stop 2023-10-03T10:16:59.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.394+0000 7f8a57ccd700 1 -- 172.21.15.3:0/2787691732 shutdown_connections 2023-10-03T10:16:59.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.394+0000 7f8a57ccd700 1 --2- 172.21.15.3:0/2787691732 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a5009e930 0x7f8a5009ed00 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.394+0000 7f8a57ccd700 1 --2- 172.21.15.3:0/2787691732 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a5009f650 0x7f8a50131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.394+0000 7f8a57ccd700 1 -- 172.21.15.3:0/2787691732 >> 172.21.15.3:0/2787691732 conn(0x7f8a500a4210 msgr2=0x7f8a500a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:59.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.394+0000 7f8a57ccd700 1 -- 172.21.15.3:0/2787691732 shutdown_connections 2023-10-03T10:16:59.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.394+0000 7f8a57ccd700 1 -- 172.21.15.3:0/2787691732 wait complete. 2023-10-03T10:16:59.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.395+0000 7f8a57ccd700 1 Processor -- start 2023-10-03T10:16:59.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.396+0000 7f8a57ccd700 1 -- start start 2023-10-03T10:16:59.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.396+0000 7f8a57ccd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a5009e930 0x7f8a501a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:59.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.396+0000 7f8a57ccd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a5009f240 0x7f8a501a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:59.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.396+0000 7f8a57ccd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a5009f650 0x7f8a501a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:59.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.396+0000 7f8a57ccd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a501a4840 con 0x7f8a5009f240 2023-10-03T10:16:59.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.396+0000 7f8a57ccd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a50071d20 con 0x7f8a5009f650 2023-10-03T10:16:59.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.396+0000 7f8a57ccd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a501a6470 con 0x7f8a5009e930 2023-10-03T10:16:59.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.396+0000 7f8a55a69700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a5009e930 0x7f8a501a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:59.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.396+0000 7f8a5626a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a5009f650 0x7f8a501a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:59.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.397+0000 7f8a55a69700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a5009e930 0x7f8a501a2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:35964/0 (socket says 172.21.15.3:35964) 2023-10-03T10:16:59.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.397+0000 7f8a55a69700 1 -- 172.21.15.3:0/3333425735 learned_addr learned my addr 172.21.15.3:0/3333425735 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:59.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.397+0000 7f8a55268700 1 --2- 172.21.15.3:0/3333425735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a5009f240 0x7f8a501a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:59.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.397+0000 7f8a55a69700 1 -- 172.21.15.3:0/3333425735 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a5009f650 msgr2=0x7f8a501a36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:59.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.397+0000 7f8a55a69700 1 --2- 172.21.15.3:0/3333425735 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a5009f650 0x7f8a501a36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.397+0000 7f8a55a69700 1 -- 172.21.15.3:0/3333425735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a5009f240 msgr2=0x7f8a501a31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:59.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.397+0000 7f8a55a69700 1 --2- 172.21.15.3:0/3333425735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a5009f240 0x7f8a501a31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.397+0000 7f8a55a69700 1 -- 172.21.15.3:0/3333425735 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4c009550 con 0x7f8a5009e930 2023-10-03T10:16:59.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.397+0000 7f8a5626a700 1 --2- 172.21.15.3:0/3333425735 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a5009f650 0x7f8a501a36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:59.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.397+0000 7f8a55268700 1 --2- 172.21.15.3:0/3333425735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a5009f240 0x7f8a501a31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:59.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.397+0000 7f8a55a69700 1 --2- 172.21.15.3:0/3333425735 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a5009e930 0x7f8a501a2c70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c000c00 tx=0x7f8a4c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:59.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.397+0000 7f8a467fc700 1 -- 172.21.15.3:0/3333425735 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c004450 con 0x7f8a5009e930 2023-10-03T10:16:59.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.397+0000 7f8a467fc700 1 -- 172.21.15.3:0/3333425735 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4c0045b0 con 0x7f8a5009e930 2023-10-03T10:16:59.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.397+0000 7f8a57ccd700 1 -- 172.21.15.3:0/3333425735 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a500a0190 con 0x7f8a5009e930 2023-10-03T10:16:59.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.398+0000 7f8a467fc700 1 -- 172.21.15.3:0/3333425735 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c027a70 con 0x7f8a5009e930 2023-10-03T10:16:59.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.398+0000 7f8a57ccd700 1 -- 172.21.15.3:0/3333425735 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a501a5a90 con 0x7f8a5009e930 2023-10-03T10:16:59.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.399+0000 7f8a467fc700 1 -- 172.21.15.3:0/3333425735 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8a4c027bd0 con 0x7f8a5009e930 2023-10-03T10:16:59.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.399+0000 7f8a57ccd700 1 -- 172.21.15.3:0/3333425735 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a50074ea0 con 0x7f8a5009e930 2023-10-03T10:16:59.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.399+0000 7f8a467fc700 1 -- 172.21.15.3:0/3333425735 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8a4c048e80 con 0x7f8a5009e930 2023-10-03T10:16:59.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.408+0000 7f8a467fc700 1 -- 172.21.15.3:0/3333425735 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8a4c010380 con 0x7f8a5009e930 2023-10-03T10:16:59.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.425+0000 7fe3f1b98700 1 -- 172.21.15.3:0/1528377172 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe3ec06c360 con 0x7fe3ec09d6f0 2023-10-03T10:16:59.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.425+0000 7fe3d3fff700 1 -- 172.21.15.3:0/1528377172 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe3dc021030 con 0x7fe3ec09d6f0 2023-10-03T10:16:59.425 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:16:59.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.427+0000 7fe3f1b98700 1 -- 172.21.15.3:0/1528377172 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3ec09d6f0 msgr2=0x7fe3ec1225e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:59.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.427+0000 7fe3f1b98700 1 --2- 172.21.15.3:0/1528377172 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3ec09d6f0 0x7fe3ec1225e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc00ebe0 tx=0x7fe3dc00eef0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.428+0000 7fe3f1b98700 1 -- 172.21.15.3:0/1528377172 shutdown_connections 2023-10-03T10:16:59.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.428+0000 7fe3f1b98700 1 --2- 172.21.15.3:0/1528377172 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe3ec09e410 0x7fe3ec123060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.428+0000 7fe3f1b98700 1 --2- 172.21.15.3:0/1528377172 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe3ec09d6f0 0x7fe3ec1225e0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.428+0000 7fe3f1b98700 1 --2- 172.21.15.3:0/1528377172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3ec09e000 0x7fe3ec122b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.428+0000 7fe3f1b98700 1 -- 172.21.15.3:0/1528377172 >> 172.21.15.3:0/1528377172 conn(0x7fe3ec066f50 msgr2=0x7fe3ec0a1590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:59.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.428+0000 7fe3f1b98700 1 -- 172.21.15.3:0/1528377172 shutdown_connections 2023-10-03T10:16:59.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.428+0000 7fe3f1b98700 1 -- 172.21.15.3:0/1528377172 wait complete. 2023-10-03T10:16:59.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.440+0000 7f30e970a700 1 Processor -- start 2023-10-03T10:16:59.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.440+0000 7f30e970a700 1 -- start start 2023-10-03T10:16:59.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.441+0000 7f30e970a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30e412cf60 0x7f30e406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:59.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.441+0000 7f30e970a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f30e412f8b0 0x7f30e412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:59.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.441+0000 7f30e970a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30e412cd90 con 0x7f30e4130260 2023-10-03T10:16:59.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.441+0000 7f30e970a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e412d330 con 0x7f30e412f8b0 2023-10-03T10:16:59.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.441+0000 7f30e970a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f30e41306e0 con 0x7f30e412cf60 2023-10-03T10:16:59.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.441+0000 7f30e2ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30e412cf60 0x7f30e406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:59.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.441+0000 7f30e2ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30e412cf60 0x7f30e406f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:35980/0 (socket says 172.21.15.3:35980) 2023-10-03T10:16:59.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.441+0000 7f30e2ffd700 1 -- 172.21.15.3:0/1754937157 learned_addr learned my addr 172.21.15.3:0/1754937157 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:59.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.441+0000 7f30e27fc700 1 --2- 172.21.15.3:0/1754937157 >> v2:172.21.15.3:3300/0 conn(0x7f30e412f8b0 0x7f30e412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:59.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.442+0000 7f30e2ffd700 1 -- 172.21.15.3:0/1754937157 >> v2:172.21.15.3:3300/0 conn(0x7f30e412f8b0 msgr2=0x7f30e412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:59.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.442+0000 7f30e2ffd700 1 --2- 172.21.15.3:0/1754937157 >> v2:172.21.15.3:3300/0 conn(0x7f30e412f8b0 0x7f30e412fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.442+0000 7f30e2ffd700 1 -- 172.21.15.3:0/1754937157 >> v1:172.21.15.3:6789/0 conn(0x7f30e4130260 legacy=0x7f30e4136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:59.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.442+0000 7f30e2ffd700 1 -- 172.21.15.3:0/1754937157 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e4071780 con 0x7f30e412cf60 2023-10-03T10:16:59.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.442+0000 7f30e2ffd700 1 --2- 172.21.15.3:0/1754937157 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30e412cf60 0x7f30e406f5d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f30d400bb90 tx=0x7f30d400bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8bb7a0455a56413 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:59.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.442+0000 7f30e1ffb700 1 -- 172.21.15.3:0/1754937157 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d400dc20 con 0x7f30e412cf60 2023-10-03T10:16:59.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.442+0000 7f30e1ffb700 1 -- 172.21.15.3:0/1754937157 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d400dd80 con 0x7f30e412cf60 2023-10-03T10:16:59.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.442+0000 7f30e1ffb700 1 -- 172.21.15.3:0/1754937157 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d4017640 con 0x7f30e412cf60 2023-10-03T10:16:59.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.443+0000 7f30e970a700 1 -- 172.21.15.3:0/1754937157 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30e412cf60 msgr2=0x7f30e406f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:59.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.443+0000 7f30e970a700 1 --2- 172.21.15.3:0/1754937157 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30e412cf60 0x7f30e406f5d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f30d400bb90 tx=0x7f30d400bea0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.443+0000 7f30e970a700 1 -- 172.21.15.3:0/1754937157 shutdown_connections 2023-10-03T10:16:59.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.443+0000 7f30e970a700 1 --2- 172.21.15.3:0/1754937157 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30e412cf60 0x7f30e406f5d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.443+0000 7f30e970a700 1 --2- 172.21.15.3:0/1754937157 >> v2:172.21.15.3:3300/0 conn(0x7f30e412f8b0 0x7f30e412fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.443+0000 7f30e970a700 1 -- 172.21.15.3:0/1754937157 >> 172.21.15.3:0/1754937157 conn(0x7f30e40a2aa0 msgr2=0x7f30e40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:59.445 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:16:59.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.443+0000 7f30e970a700 1 -- 172.21.15.3:0/1754937157 shutdown_connections 2023-10-03T10:16:59.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.443+0000 7f30e970a700 1 -- 172.21.15.3:0/1754937157 wait complete. 2023-10-03T10:16:59.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.444+0000 7f30e970a700 1 Processor -- start 2023-10-03T10:16:59.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.445+0000 7f30e970a700 1 -- start start 2023-10-03T10:16:59.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.445+0000 7f30e970a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30e412cf60 0x7f30e41c4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:59.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.445+0000 7f30e970a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30e412f8b0 0x7f30e41c4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:59.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.445+0000 7f30e970a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f30e4130260 0x7f30e41c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:59.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.445+0000 7f30e970a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e409cc10 con 0x7f30e412cf60 2023-10-03T10:16:59.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.445+0000 7f30e970a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e412f440 con 0x7f30e4130260 2023-10-03T10:16:59.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.445+0000 7f30e970a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f30e4129fa0 con 0x7f30e412f8b0 2023-10-03T10:16:59.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.445+0000 7f30e27fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30e412f8b0 0x7f30e41c4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:59.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.445+0000 7f30e2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30e412cf60 0x7f30e41c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:59.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.445+0000 7f30e27fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30e412f8b0 0x7f30e41c4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:35992/0 (socket says 172.21.15.3:35992) 2023-10-03T10:16:59.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.445+0000 7f30e27fc700 1 -- 172.21.15.3:0/4083057516 learned_addr learned my addr 172.21.15.3:0/4083057516 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:59.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.445+0000 7f30e37fe700 1 --2- 172.21.15.3:0/4083057516 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f30e4130260 0x7f30e41c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:59.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.446+0000 7f30e2ffd700 1 -- 172.21.15.3:0/4083057516 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30e412f8b0 msgr2=0x7f30e41c4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:59.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.446+0000 7f30e2ffd700 1 --2- 172.21.15.3:0/4083057516 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30e412f8b0 0x7f30e41c4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.446+0000 7f30e2ffd700 1 -- 172.21.15.3:0/4083057516 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f30e4130260 msgr2=0x7f30e41c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:59.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.446+0000 7f30e2ffd700 1 --2- 172.21.15.3:0/4083057516 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f30e4130260 0x7f30e41c4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.446+0000 7f30e2ffd700 1 -- 172.21.15.3:0/4083057516 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d400b7f0 con 0x7f30e412cf60 2023-10-03T10:16:59.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.446+0000 7f30e27fc700 1 --2- 172.21.15.3:0/4083057516 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30e412f8b0 0x7f30e41c4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:59.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.446+0000 7f30e37fe700 1 --2- 172.21.15.3:0/4083057516 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f30e4130260 0x7f30e41c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:16:59.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.446+0000 7f30e2ffd700 1 --2- 172.21.15.3:0/4083057516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30e412cf60 0x7f30e41c4030 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f30d4019000 tx=0x7f30d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:59.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.446+0000 7f30cb7fe700 1 -- 172.21.15.3:0/4083057516 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d40036a0 con 0x7f30e412cf60 2023-10-03T10:16:59.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.446+0000 7f30cb7fe700 1 -- 172.21.15.3:0/4083057516 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d4003820 con 0x7f30e412cf60 2023-10-03T10:16:59.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.446+0000 7f30e970a700 1 -- 172.21.15.3:0/4083057516 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e409e6c0 con 0x7f30e412cf60 2023-10-03T10:16:59.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.446+0000 7f30cb7fe700 1 -- 172.21.15.3:0/4083057516 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d4003bd0 con 0x7f30e412cf60 2023-10-03T10:16:59.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.446+0000 7f30e970a700 1 -- 172.21.15.3:0/4083057516 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e4129860 con 0x7f30e412cf60 2023-10-03T10:16:59.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.447+0000 7f30cb7fe700 1 -- 172.21.15.3:0/4083057516 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f30d4015070 con 0x7f30e412cf60 2023-10-03T10:16:59.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.447+0000 7f30cb7fe700 1 -- 172.21.15.3:0/4083057516 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f30d40444c0 con 0x7f30e412cf60 2023-10-03T10:16:59.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.448+0000 7f30e970a700 1 -- 172.21.15.3:0/4083057516 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e4074ea0 con 0x7f30e412cf60 2023-10-03T10:16:59.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.453+0000 7f30cb7fe700 1 -- 172.21.15.3:0/4083057516 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f30d401c030 con 0x7f30e412cf60 2023-10-03T10:16:59.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.492+0000 7fbf16e83700 1 Processor -- start 2023-10-03T10:16:59.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.492+0000 7fbf16e83700 1 -- start start 2023-10-03T10:16:59.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.492+0000 7fbf16e83700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbf100a0a30 0x7fbf100a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:59.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.492+0000 7fbf16e83700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf1009d630 0x7fbf100a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:59.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.492+0000 7fbf16e83700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf100a2a70 con 0x7fbf10133930 2023-10-03T10:16:59.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.492+0000 7fbf16e83700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf100a01d0 con 0x7fbf100a0a30 2023-10-03T10:16:59.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.492+0000 7fbf16e83700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1009e2b0 con 0x7fbf1009d630 2023-10-03T10:16:59.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.493+0000 7fbf0ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf1009d630 0x7fbf100a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:59.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.493+0000 7fbf15420700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbf10133930 0x7fbf10133d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39960/0 (socket says 172.21.15.3:39960) 2023-10-03T10:16:59.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.493+0000 7fbf15420700 1 -- 172.21.15.3:0/3669113705 learned_addr learned my addr 172.21.15.3:0/3669113705 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:16:59.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.493+0000 7fbf14c1f700 1 --2- 172.21.15.3:0/3669113705 >> v2:172.21.15.3:3300/0 conn(0x7fbf100a0a30 0x7fbf100a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:59.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.493+0000 7fbf0f7fe700 1 -- 172.21.15.3:0/3669113705 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 647017927 0 0) 0x7fbf100a2a70 con 0x7fbf10133930 2023-10-03T10:16:59.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.493+0000 7fbf0f7fe700 1 -- 172.21.15.3:0/3669113705 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf000035b0 con 0x7fbf10133930 2023-10-03T10:16:59.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.493+0000 7fbf14c1f700 1 -- 172.21.15.3:0/3669113705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf1009d630 msgr2=0x7fbf100a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:16:59.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.493+0000 7fbf14c1f700 1 --2- 172.21.15.3:0/3669113705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf1009d630 0x7fbf100a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.493+0000 7fbf14c1f700 1 -- 172.21.15.3:0/3669113705 >> v1:172.21.15.3:6789/0 conn(0x7fbf10133930 legacy=0x7fbf10133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:59.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.493+0000 7fbf14c1f700 1 -- 172.21.15.3:0/3669113705 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1009dc50 con 0x7fbf100a0a30 2023-10-03T10:16:59.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.493+0000 7fbf14c1f700 1 --2- 172.21.15.3:0/3669113705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf100a0a30 0x7fbf100a0e00 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009840 tx=0x7fbf04009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569a3824f3508920 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:59.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.493+0000 7fbf0f7fe700 1 -- 172.21.15.3:0/3669113705 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf04011830 con 0x7fbf100a0a30 2023-10-03T10:16:59.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.493+0000 7fbf0f7fe700 1 -- 172.21.15.3:0/3669113705 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf04011990 con 0x7fbf100a0a30 2023-10-03T10:16:59.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.493+0000 7fbf0f7fe700 1 -- 172.21.15.3:0/3669113705 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf04011d20 con 0x7fbf100a0a30 2023-10-03T10:16:59.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.494+0000 7fbf16e83700 1 -- 172.21.15.3:0/3669113705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf100a0a30 msgr2=0x7fbf100a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:59.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.494+0000 7fbf16e83700 1 --2- 172.21.15.3:0/3669113705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf100a0a30 0x7fbf100a0e00 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009840 tx=0x7fbf04009b50 comp rx=0 tx=0).stop 2023-10-03T10:16:59.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.494+0000 7fbf16e83700 1 -- 172.21.15.3:0/3669113705 shutdown_connections 2023-10-03T10:16:59.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.494+0000 7fbf16e83700 1 --2- 172.21.15.3:0/3669113705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf100a0a30 0x7fbf100a0e00 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.494+0000 7fbf16e83700 1 --2- 172.21.15.3:0/3669113705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf1009d630 0x7fbf100a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.494+0000 7fbf16e83700 1 -- 172.21.15.3:0/3669113705 >> 172.21.15.3:0/3669113705 conn(0x7fbf100a3050 msgr2=0x7fbf100a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:59.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.494+0000 7fbf16e83700 1 -- 172.21.15.3:0/3669113705 shutdown_connections 2023-10-03T10:16:59.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.494+0000 7fbf16e83700 1 -- 172.21.15.3:0/3669113705 wait complete. 2023-10-03T10:16:59.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.495+0000 7fbf16e83700 1 Processor -- start 2023-10-03T10:16:59.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.495+0000 7fbf16e83700 1 -- start start 2023-10-03T10:16:59.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.496+0000 7fbf16e83700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf1009d630 0x7fbf101c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:59.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.496+0000 7fbf16e83700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf100a0a30 0x7fbf101c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:59.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.496+0000 7fbf16e83700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf10133930 0x7fbf1011eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:16:59.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.496+0000 7fbf16e83700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf100a43b0 con 0x7fbf1009d630 2023-10-03T10:16:59.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.496+0000 7fbf16e83700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf10133530 con 0x7fbf10133930 2023-10-03T10:16:59.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.496+0000 7fbf16e83700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf101bcd60 con 0x7fbf100a0a30 2023-10-03T10:16:59.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.496+0000 7fbf0ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf100a0a30 0x7fbf101c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:59.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.496+0000 7fbf0ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf100a0a30 0x7fbf101c85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36010/0 (socket says 172.21.15.3:36010) 2023-10-03T10:16:59.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.496+0000 7fbf0ffff700 1 -- 172.21.15.3:0/4085925376 learned_addr learned my addr 172.21.15.3:0/4085925376 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:16:59.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.496+0000 7fbf15420700 1 --2- 172.21.15.3:0/4085925376 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf10133930 0x7fbf1011eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:59.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.496+0000 7fbf14c1f700 1 --2- 172.21.15.3:0/4085925376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf1009d630 0x7fbf101c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:16:59.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.497+0000 7fbf0ffff700 1 -- 172.21.15.3:0/4085925376 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf10133930 msgr2=0x7fbf1011eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:59.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.497+0000 7fbf0ffff700 1 --2- 172.21.15.3:0/4085925376 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf10133930 0x7fbf1011eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.497+0000 7fbf0ffff700 1 -- 172.21.15.3:0/4085925376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf1009d630 msgr2=0x7fbf101c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:59.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.497+0000 7fbf0ffff700 1 --2- 172.21.15.3:0/4085925376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf1009d630 0x7fbf101c80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.497+0000 7fbf0ffff700 1 -- 172.21.15.3:0/4085925376 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf040094a0 con 0x7fbf100a0a30 2023-10-03T10:16:59.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.497+0000 7fbf0ffff700 1 --2- 172.21.15.3:0/4085925376 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf100a0a30 0x7fbf101c85e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00e9b0 tx=0x7fbefc00ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:16:59.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.497+0000 7fbf0d7fa700 1 -- 172.21.15.3:0/4085925376 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbefc009710 con 0x7fbf100a0a30 2023-10-03T10:16:59.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.497+0000 7fbf0d7fa700 1 -- 172.21.15.3:0/4085925376 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbefc00ee50 con 0x7fbf100a0a30 2023-10-03T10:16:59.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.497+0000 7fbf16e83700 1 -- 172.21.15.3:0/4085925376 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf101221a0 con 0x7fbf100a0a30 2023-10-03T10:16:59.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.497+0000 7fbf0d7fa700 1 -- 172.21.15.3:0/4085925376 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbefc00ca50 con 0x7fbf100a0a30 2023-10-03T10:16:59.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.497+0000 7fbf16e83700 1 -- 172.21.15.3:0/4085925376 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf10121220 con 0x7fbf100a0a30 2023-10-03T10:16:59.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.498+0000 7fbf0d7fa700 1 -- 172.21.15.3:0/4085925376 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbefc00cbb0 con 0x7fbf100a0a30 2023-10-03T10:16:59.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.498+0000 7fbf0d7fa700 1 -- 172.21.15.3:0/4085925376 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fbefc03f5c0 con 0x7fbf100a0a30 2023-10-03T10:16:59.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.499+0000 7fbf16e83700 1 -- 172.21.15.3:0/4085925376 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf10121220 con 0x7fbf100a0a30 2023-10-03T10:16:59.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.504+0000 7fbf0d7fa700 1 -- 172.21.15.3:0/4085925376 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbf10121220 con 0x7fbf100a0a30 2023-10-03T10:16:59.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.622+0000 7f8a57ccd700 1 -- 172.21.15.3:0/3333425735 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8a5006b250 con 0x7f8a5009e930 2023-10-03T10:16:59.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.622+0000 7f8a467fc700 1 -- 172.21.15.3:0/3333425735 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8a4c02b020 con 0x7f8a5009e930 2023-10-03T10:16:59.623 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:16:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.625+0000 7f8a57ccd700 1 -- 172.21.15.3:0/3333425735 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a5009e930 msgr2=0x7f8a501a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.625+0000 7f8a57ccd700 1 --2- 172.21.15.3:0/3333425735 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a5009e930 0x7f8a501a2c70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c000c00 tx=0x7f8a4c004750 comp rx=0 tx=0).stop 2023-10-03T10:16:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.625+0000 7f8a57ccd700 1 -- 172.21.15.3:0/3333425735 shutdown_connections 2023-10-03T10:16:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.625+0000 7f8a57ccd700 1 --2- 172.21.15.3:0/3333425735 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8a5009f650 0x7f8a501a36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.625+0000 7f8a57ccd700 1 --2- 172.21.15.3:0/3333425735 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8a5009e930 0x7f8a501a2c70 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.625+0000 7f8a57ccd700 1 --2- 172.21.15.3:0/3333425735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a5009f240 0x7f8a501a31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.625+0000 7f8a57ccd700 1 -- 172.21.15.3:0/3333425735 >> 172.21.15.3:0/3333425735 conn(0x7f8a500a4210 msgr2=0x7f8a501b7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.625+0000 7f8a57ccd700 1 -- 172.21.15.3:0/3333425735 shutdown_connections 2023-10-03T10:16:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.625+0000 7f8a57ccd700 1 -- 172.21.15.3:0/3333425735 wait complete. 2023-10-03T10:16:59.639 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:16:59.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.706+0000 7f30e970a700 1 -- 172.21.15.3:0/4083057516 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f30e40713a0 con 0x7f30e412cf60 2023-10-03T10:16:59.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.707+0000 7f30cb7fe700 1 -- 172.21.15.3:0/4083057516 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f30d4038090 con 0x7f30e412cf60 2023-10-03T10:16:59.707 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:16:59.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.709+0000 7f30e970a700 1 -- 172.21.15.3:0/4083057516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30e412cf60 msgr2=0x7f30e41c4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:59.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.709+0000 7f30e970a700 1 --2- 172.21.15.3:0/4083057516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30e412cf60 0x7f30e41c4030 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f30d4019000 tx=0x7f30d4004750 comp rx=0 tx=0).stop 2023-10-03T10:16:59.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.709+0000 7f30e970a700 1 -- 172.21.15.3:0/4083057516 shutdown_connections 2023-10-03T10:16:59.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.709+0000 7f30e970a700 1 --2- 172.21.15.3:0/4083057516 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f30e4130260 0x7f30e41c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.709+0000 7f30e970a700 1 --2- 172.21.15.3:0/4083057516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30e412cf60 0x7f30e41c4030 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.709+0000 7f30e970a700 1 --2- 172.21.15.3:0/4083057516 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30e412f8b0 0x7f30e41c4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.709+0000 7f30e970a700 1 -- 172.21.15.3:0/4083057516 >> 172.21.15.3:0/4083057516 conn(0x7f30e40a2aa0 msgr2=0x7f30e406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:59.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.709+0000 7f30e970a700 1 -- 172.21.15.3:0/4083057516 shutdown_connections 2023-10-03T10:16:59.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.709+0000 7f30e970a700 1 -- 172.21.15.3:0/4083057516 wait complete. 2023-10-03T10:16:59.723 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:16:59.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.739+0000 7fbf16e83700 1 -- 172.21.15.3:0/4085925376 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbf10121220 con 0x7fbf100a0a30 2023-10-03T10:16:59.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.739+0000 7fbf0d7fa700 1 -- 172.21.15.3:0/4085925376 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbf10121220 con 0x7fbf100a0a30 2023-10-03T10:16:59.739 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:16:59.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.741+0000 7fbf16e83700 1 -- 172.21.15.3:0/4085925376 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf100a0a30 msgr2=0x7fbf101c85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:16:59.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.741+0000 7fbf16e83700 1 --2- 172.21.15.3:0/4085925376 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf100a0a30 0x7fbf101c85e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00e9b0 tx=0x7fbefc00ecc0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.742+0000 7fbf16e83700 1 -- 172.21.15.3:0/4085925376 shutdown_connections 2023-10-03T10:16:59.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.742+0000 7fbf16e83700 1 --2- 172.21.15.3:0/4085925376 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf10133930 0x7fbf1011eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.742+0000 7fbf16e83700 1 --2- 172.21.15.3:0/4085925376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf1009d630 0x7fbf101c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.742+0000 7fbf16e83700 1 --2- 172.21.15.3:0/4085925376 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf100a0a30 0x7fbf101c85e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:16:59.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.742+0000 7fbf16e83700 1 -- 172.21.15.3:0/4085925376 >> 172.21.15.3:0/4085925376 conn(0x7fbf100a3050 msgr2=0x7fbf1009f620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:16:59.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.742+0000 7fbf16e83700 1 -- 172.21.15.3:0/4085925376 shutdown_connections 2023-10-03T10:16:59.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:16:59.742+0000 7fbf16e83700 1 -- 172.21.15.3:0/4085925376 wait complete. 2023-10-03T10:16:59.755 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:00.446 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:00.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.593+0000 7f3501522700 1 Processor -- start 2023-10-03T10:17:00.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.593+0000 7f3501522700 1 -- start start 2023-10-03T10:17:00.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.594+0000 7f3501522700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f34fc09d6f0 0x7f34fc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:00.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.594+0000 7f3501522700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f34fc09e410 0x7f34fc130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:00.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.594+0000 7f3501522700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34fc09f130 con 0x7f34fc09e000 2023-10-03T10:17:00.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.594+0000 7f3501522700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34fc09f520 con 0x7f34fc09d6f0 2023-10-03T10:17:00.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.594+0000 7f3501522700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc09e930 con 0x7f34fc09e410 2023-10-03T10:17:00.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.594+0000 7f34faffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f34fc09d6f0 0x7f34fc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:00.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.594+0000 7f34fb7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f34fc09e410 0x7f34fc130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:00.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.594+0000 7f34faffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f34fc09d6f0 0x7f34fc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48976/0 (socket says 172.21.15.3:48976) 2023-10-03T10:17:00.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.594+0000 7f34faffd700 1 -- 172.21.15.3:0/3260027520 learned_addr learned my addr 172.21.15.3:0/3260027520 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:00.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.595+0000 7f34faffd700 1 -- 172.21.15.3:0/3260027520 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f34fc09e410 msgr2=0x7f34fc130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:00.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.595+0000 7f34faffd700 1 --2- 172.21.15.3:0/3260027520 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f34fc09e410 0x7f34fc130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:00.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.595+0000 7f34faffd700 1 -- 172.21.15.3:0/3260027520 >> v1:172.21.15.3:6789/0 conn(0x7f34fc09e000 legacy=0x7f34fc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:00.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.595+0000 7f34faffd700 1 -- 172.21.15.3:0/3260027520 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34fc09b2d0 con 0x7f34fc09d6f0 2023-10-03T10:17:00.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.595+0000 7f34f9ffb700 1 -- 172.21.15.3:0/3260027520 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4010740996 0 0) 0x7f34fc09f130 con 0x7f34fc09e000 2023-10-03T10:17:00.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.595+0000 7f34fb7fe700 1 --2- 172.21.15.3:0/3260027520 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f34fc09e410 0x7f34fc130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:00.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.595+0000 7f34faffd700 1 --2- 172.21.15.3:0/3260027520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34fc09d6f0 0x7f34fc09dac0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f34ec00bb90 tx=0x7f34ec00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf3b479dfcff9ef server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:00.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.595+0000 7f34f9ffb700 1 -- 172.21.15.3:0/3260027520 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ec00dd30 con 0x7f34fc09d6f0 2023-10-03T10:17:00.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.595+0000 7f34f9ffb700 1 -- 172.21.15.3:0/3260027520 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34ec00de90 con 0x7f34fc09d6f0 2023-10-03T10:17:00.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.595+0000 7f34f9ffb700 1 -- 172.21.15.3:0/3260027520 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ec010690 con 0x7f34fc09d6f0 2023-10-03T10:17:00.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.596+0000 7f3501522700 1 -- 172.21.15.3:0/3260027520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34fc09d6f0 msgr2=0x7f34fc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:00.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.596+0000 7f3501522700 1 --2- 172.21.15.3:0/3260027520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34fc09d6f0 0x7f34fc09dac0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f34ec00bb90 tx=0x7f34ec00bea0 comp rx=0 tx=0).stop 2023-10-03T10:17:00.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.596+0000 7f3501522700 1 -- 172.21.15.3:0/3260027520 shutdown_connections 2023-10-03T10:17:00.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.596+0000 7f3501522700 1 --2- 172.21.15.3:0/3260027520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34fc09d6f0 0x7f34fc09dac0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:00.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.596+0000 7f3501522700 1 --2- 172.21.15.3:0/3260027520 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f34fc09e410 0x7f34fc130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:00.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.596+0000 7f3501522700 1 -- 172.21.15.3:0/3260027520 >> 172.21.15.3:0/3260027520 conn(0x7f34fc066f50 msgr2=0x7f34fc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:00.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.596+0000 7f3501522700 1 -- 172.21.15.3:0/3260027520 shutdown_connections 2023-10-03T10:17:00.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.596+0000 7f3501522700 1 -- 172.21.15.3:0/3260027520 wait complete. 2023-10-03T10:17:00.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.597+0000 7f3501522700 1 Processor -- start 2023-10-03T10:17:00.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.597+0000 7f3501522700 1 -- start start 2023-10-03T10:17:00.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.597+0000 7f3501522700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34fc09d6f0 0x7f34fc11df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:00.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.597+0000 7f3501522700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f34fc09e000 0x7f34fc11e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:00.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.597+0000 7f3501522700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f34fc09e410 0x7f34fc11e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:00.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.597+0000 7f3501522700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc1302d0 con 0x7f34fc09d6f0 2023-10-03T10:17:00.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.598+0000 7f3501522700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc120d50 con 0x7f34fc09e000 2023-10-03T10:17:00.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.598+0000 7f3501522700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc11f3a0 con 0x7f34fc09e410 2023-10-03T10:17:00.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.598+0000 7f34faffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34fc09d6f0 0x7f34fc11df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:00.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.598+0000 7f34faffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34fc09d6f0 0x7f34fc11df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48988/0 (socket says 172.21.15.3:48988) 2023-10-03T10:17:00.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.598+0000 7f34fb7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f34fc09e410 0x7f34fc11e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:00.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.598+0000 7f34faffd700 1 -- 172.21.15.3:0/1560225655 learned_addr learned my addr 172.21.15.3:0/1560225655 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:00.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.598+0000 7f34fa7fc700 1 --2- 172.21.15.3:0/1560225655 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f34fc09e000 0x7f34fc11e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:00.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.598+0000 7f34fb7fe700 1 -- 172.21.15.3:0/1560225655 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f34fc09e000 msgr2=0x7f34fc11e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:00.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.598+0000 7f34fb7fe700 1 --2- 172.21.15.3:0/1560225655 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f34fc09e000 0x7f34fc11e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:00.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.598+0000 7f34fb7fe700 1 -- 172.21.15.3:0/1560225655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34fc09d6f0 msgr2=0x7f34fc11df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:00.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.598+0000 7f34fb7fe700 1 --2- 172.21.15.3:0/1560225655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34fc09d6f0 0x7f34fc11df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:00.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.598+0000 7f34fb7fe700 1 -- 172.21.15.3:0/1560225655 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f4009a70 con 0x7f34fc09e410 2023-10-03T10:17:00.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.598+0000 7f34faffd700 1 --2- 172.21.15.3:0/1560225655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34fc09d6f0 0x7f34fc11df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:00.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.598+0000 7f34fb7fe700 1 --2- 172.21.15.3:0/1560225655 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f34fc09e410 0x7f34fc11e9c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f34f400efc0 tx=0x7f34f400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:00.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.598+0000 7f34f37fe700 1 -- 172.21.15.3:0/1560225655 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f400ca90 con 0x7f34fc09e410 2023-10-03T10:17:00.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.598+0000 7f34f37fe700 1 -- 172.21.15.3:0/1560225655 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f400f040 con 0x7f34fc09e410 2023-10-03T10:17:00.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.598+0000 7f3501522700 1 -- 172.21.15.3:0/1560225655 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34ec00b7f0 con 0x7f34fc09e410 2023-10-03T10:17:00.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.598+0000 7f34f37fe700 1 -- 172.21.15.3:0/1560225655 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f400ce30 con 0x7f34fc09e410 2023-10-03T10:17:00.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.599+0000 7f3501522700 1 -- 172.21.15.3:0/1560225655 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34fc1211a0 con 0x7f34fc09e410 2023-10-03T10:17:00.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.599+0000 7f34f37fe700 1 -- 172.21.15.3:0/1560225655 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f34f4014480 con 0x7f34fc09e410 2023-10-03T10:17:00.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.599+0000 7f34f37fe700 1 -- 172.21.15.3:0/1560225655 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f34f403fca0 con 0x7f34fc09e410 2023-10-03T10:17:00.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.599+0000 7f3501522700 1 -- 172.21.15.3:0/1560225655 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34fc09c700 con 0x7f34fc09e410 2023-10-03T10:17:00.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.604+0000 7f34f37fe700 1 -- 172.21.15.3:0/1560225655 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f34f4012930 con 0x7f34fc09e410 2023-10-03T10:17:00.640 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:00.724 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:00.756 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:00.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.788+0000 7fbd0b2b4700 1 Processor -- start 2023-10-03T10:17:00.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.789+0000 7fbd0b2b4700 1 -- start start 2023-10-03T10:17:00.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.789+0000 7fbd0b2b4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbd0406a770 0x7fbd0406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:00.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.789+0000 7fbd0b2b4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd04131d10 0x7fbd04139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:00.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.789+0000 7fbd0b2b4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd0409ece0 con 0x7fbd0406b080 2023-10-03T10:17:00.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.789+0000 7fbd0b2b4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd0409e760 con 0x7fbd0406a770 2023-10-03T10:17:00.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.789+0000 7fbd0b2b4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0409e040 con 0x7fbd04131d10 2023-10-03T10:17:00.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.790+0000 7fbd09050700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbd0406a770 0x7fbd0406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:00.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.790+0000 7fbd09050700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbd0406a770 0x7fbd0406ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48994/0 (socket says 172.21.15.3:48994) 2023-10-03T10:17:00.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.790+0000 7fbd09050700 1 -- 172.21.15.3:0/1841033214 learned_addr learned my addr 172.21.15.3:0/1841033214 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:00.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.790+0000 7fbd0884f700 1 --1- 172.21.15.3:0/1841033214 >> v1:172.21.15.3:6789/0 conn(0x7fbd0406b080 0x7fbd04131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39974/0 (socket says 172.21.15.3:39974) 2023-10-03T10:17:00.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.790+0000 7fbd09851700 1 --2- 172.21.15.3:0/1841033214 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd04131d10 0x7fbd04139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:00.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.790+0000 7fbd09851700 1 -- 172.21.15.3:0/1841033214 >> v2:172.21.15.3:3300/0 conn(0x7fbd0406a770 msgr2=0x7fbd0406ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:00.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.790+0000 7fbd09851700 1 --2- 172.21.15.3:0/1841033214 >> v2:172.21.15.3:3300/0 conn(0x7fbd0406a770 0x7fbd0406ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:00.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.790+0000 7fbd09851700 1 -- 172.21.15.3:0/1841033214 >> v1:172.21.15.3:6789/0 conn(0x7fbd0406b080 legacy=0x7fbd04131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:00.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.790+0000 7fbd09851700 1 -- 172.21.15.3:0/1841033214 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0409f630 con 0x7fbd04131d10 2023-10-03T10:17:00.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.790+0000 7fbd09050700 1 --2- 172.21.15.3:0/1841033214 >> v2:172.21.15.3:3300/0 conn(0x7fbd0406a770 0x7fbd0406ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:00.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.790+0000 7fbd09851700 1 --2- 172.21.15.3:0/1841033214 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd04131d10 0x7fbd04139860 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc009d90 tx=0x7fbcfc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb752bec255c8bef server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:00.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.790+0000 7fbcfbfff700 1 -- 172.21.15.3:0/1841033214 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc00e070 con 0x7fbd04131d10 2023-10-03T10:17:00.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.790+0000 7fbcfbfff700 1 -- 172.21.15.3:0/1841033214 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcfc00bb20 con 0x7fbd04131d10 2023-10-03T10:17:00.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.790+0000 7fbcfbfff700 1 -- 172.21.15.3:0/1841033214 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc012430 con 0x7fbd04131d10 2023-10-03T10:17:00.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.791+0000 7fbd0b2b4700 1 -- 172.21.15.3:0/1841033214 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd04131d10 msgr2=0x7fbd04139860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:00.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.791+0000 7fbd0b2b4700 1 --2- 172.21.15.3:0/1841033214 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd04131d10 0x7fbd04139860 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc009d90 tx=0x7fbcfc00b330 comp rx=0 tx=0).stop 2023-10-03T10:17:00.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.791+0000 7fbd0b2b4700 1 -- 172.21.15.3:0/1841033214 shutdown_connections 2023-10-03T10:17:00.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.791+0000 7fbd0b2b4700 1 --2- 172.21.15.3:0/1841033214 >> v2:172.21.15.3:3300/0 conn(0x7fbd0406a770 0x7fbd0406ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:00.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.791+0000 7fbd0b2b4700 1 --2- 172.21.15.3:0/1841033214 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd04131d10 0x7fbd04139860 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:00.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.791+0000 7fbd0b2b4700 1 -- 172.21.15.3:0/1841033214 >> 172.21.15.3:0/1841033214 conn(0x7fbd04066f50 msgr2=0x7fbd0409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:00.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.791+0000 7fbd0b2b4700 1 -- 172.21.15.3:0/1841033214 shutdown_connections 2023-10-03T10:17:00.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.791+0000 7fbd0b2b4700 1 -- 172.21.15.3:0/1841033214 wait complete. 2023-10-03T10:17:00.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.793+0000 7fbd0b2b4700 1 Processor -- start 2023-10-03T10:17:00.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.793+0000 7fbd0b2b4700 1 -- start start 2023-10-03T10:17:00.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.793+0000 7fbd0b2b4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd0406a770 0x7fbd041c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:00.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.793+0000 7fbd0b2b4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd0406b080 0x7fbd041c8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:00.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.793+0000 7fbd0b2b4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd04131d10 0x7fbd041c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:00.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.793+0000 7fbd0b2b4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd04122300 con 0x7fbd0406b080 2023-10-03T10:17:00.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.793+0000 7fbd0b2b4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd041212e0 con 0x7fbd0406a770 2023-10-03T10:17:00.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.793+0000 7fbd0b2b4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0409a0c0 con 0x7fbd04131d10 2023-10-03T10:17:00.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.793+0000 7fbd09050700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd0406a770 0x7fbd041c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:00.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.793+0000 7fbd09050700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd0406a770 0x7fbd041c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33460/0 (socket says 172.21.15.3:33460) 2023-10-03T10:17:00.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.794+0000 7fbd09050700 1 -- 172.21.15.3:0/2845452163 learned_addr learned my addr 172.21.15.3:0/2845452163 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:00.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.794+0000 7fbd0884f700 1 --2- 172.21.15.3:0/2845452163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd0406b080 0x7fbd041c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:00.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.794+0000 7fbd09851700 1 --2- 172.21.15.3:0/2845452163 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd04131d10 0x7fbd041c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:00.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.794+0000 7fbd09851700 1 -- 172.21.15.3:0/2845452163 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd0406a770 msgr2=0x7fbd041c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:00.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.794+0000 7fbd09851700 1 --2- 172.21.15.3:0/2845452163 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd0406a770 0x7fbd041c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:00.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.794+0000 7fbd09851700 1 -- 172.21.15.3:0/2845452163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd0406b080 msgr2=0x7fbd041c8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:00.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.794+0000 7fbd09851700 1 --2- 172.21.15.3:0/2845452163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd0406b080 0x7fbd041c8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:00.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.794+0000 7fbd09851700 1 -- 172.21.15.3:0/2845452163 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd00009550 con 0x7fbd04131d10 2023-10-03T10:17:00.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.794+0000 7fbd09851700 1 --2- 172.21.15.3:0/2845452163 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd04131d10 0x7fbd041c8eb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc016000 tx=0x7fbcfc004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:00.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.794+0000 7fbcf9ffb700 1 -- 172.21.15.3:0/2845452163 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc012430 con 0x7fbd04131d10 2023-10-03T10:17:00.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.794+0000 7fbcf9ffb700 1 -- 172.21.15.3:0/2845452163 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcfc020070 con 0x7fbd04131d10 2023-10-03T10:17:00.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.794+0000 7fbd0b2b4700 1 -- 172.21.15.3:0/2845452163 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcfc009a70 con 0x7fbd04131d10 2023-10-03T10:17:00.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.794+0000 7fbcf9ffb700 1 -- 172.21.15.3:0/2845452163 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc004040 con 0x7fbd04131d10 2023-10-03T10:17:00.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.795+0000 7fbd0b2b4700 1 -- 172.21.15.3:0/2845452163 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd04122a90 con 0x7fbd04131d10 2023-10-03T10:17:00.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.796+0000 7fbd0b2b4700 1 -- 172.21.15.3:0/2845452163 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd04074ea0 con 0x7fbd04131d10 2023-10-03T10:17:00.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.796+0000 7fbcf9ffb700 1 -- 172.21.15.3:0/2845452163 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbcfc0041a0 con 0x7fbd04131d10 2023-10-03T10:17:00.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.797+0000 7fbcf9ffb700 1 -- 172.21.15.3:0/2845452163 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fbcfc022070 con 0x7fbd04131d10 2023-10-03T10:17:00.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.801+0000 7fbcf9ffb700 1 -- 172.21.15.3:0/2845452163 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbcfc004450 con 0x7fbd04131d10 2023-10-03T10:17:00.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.816+0000 7f3501522700 1 -- 172.21.15.3:0/1560225655 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f34fc074ea0 con 0x7f34fc09e410 2023-10-03T10:17:00.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.816+0000 7f34f37fe700 1 -- 172.21.15.3:0/1560225655 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f34f4012340 con 0x7f34fc09e410 2023-10-03T10:17:00.817 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:00.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.819+0000 7f3501522700 1 -- 172.21.15.3:0/1560225655 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f34fc09e410 msgr2=0x7f34fc11e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:00.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.819+0000 7f3501522700 1 --2- 172.21.15.3:0/1560225655 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f34fc09e410 0x7f34fc11e9c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f34f400efc0 tx=0x7f34f400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:17:00.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.819+0000 7f3501522700 1 -- 172.21.15.3:0/1560225655 shutdown_connections 2023-10-03T10:17:00.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.819+0000 7f3501522700 1 --2- 172.21.15.3:0/1560225655 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f34fc09e410 0x7f34fc11e9c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:00.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.819+0000 7f3501522700 1 --2- 172.21.15.3:0/1560225655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34fc09d6f0 0x7f34fc11df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:00.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.819+0000 7f3501522700 1 --2- 172.21.15.3:0/1560225655 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f34fc09e000 0x7f34fc11e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:00.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.819+0000 7f3501522700 1 -- 172.21.15.3:0/1560225655 >> 172.21.15.3:0/1560225655 conn(0x7f34fc066f50 msgr2=0x7f34fc070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:00.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.820+0000 7f3501522700 1 -- 172.21.15.3:0/1560225655 shutdown_connections 2023-10-03T10:17:00.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.820+0000 7f3501522700 1 -- 172.21.15.3:0/1560225655 wait complete. 2023-10-03T10:17:00.835 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:00.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.880+0000 7f4c40e12700 1 Processor -- start 2023-10-03T10:17:00.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.880+0000 7f4c40e12700 1 -- start start 2023-10-03T10:17:00.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.881+0000 7f4c40e12700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c3c1337f0 0x7f4c3c138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:00.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.881+0000 7f4c40e12700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4c3c138dc0 0x7f4c3c13b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:00.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.881+0000 7f4c40e12700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c3c06b990 con 0x7f4c3c133420 2023-10-03T10:17:00.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.881+0000 7f4c40e12700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c3c09b0e0 con 0x7f4c3c138dc0 2023-10-03T10:17:00.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.881+0000 7f4c40e12700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c133c20 con 0x7f4c3c1337f0 2023-10-03T10:17:00.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.881+0000 7f4c3a59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4c3c133420 0x7f4c3c133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39986/0 (socket says 172.21.15.3:39986) 2023-10-03T10:17:00.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.881+0000 7f4c3a59c700 1 -- 172.21.15.3:0/1346026884 learned_addr learned my addr 172.21.15.3:0/1346026884 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:00.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.881+0000 7f4c3ad9d700 1 --2- 172.21.15.3:0/1346026884 >> v2:172.21.15.3:3300/0 conn(0x7f4c3c138dc0 0x7f4c3c13b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:00.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.881+0000 7f4c39d9b700 1 --2- 172.21.15.3:0/1346026884 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c3c1337f0 0x7f4c3c138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:00.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.882+0000 7f4c3959a700 1 -- 172.21.15.3:0/1346026884 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2229086242 0 0) 0x7f4c3c06b990 con 0x7f4c3c133420 2023-10-03T10:17:00.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.882+0000 7f4c3959a700 1 -- 172.21.15.3:0/1346026884 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c240035b0 con 0x7f4c3c133420 2023-10-03T10:17:00.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.882+0000 7f4c3ad9d700 1 -- 172.21.15.3:0/1346026884 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c3c1337f0 msgr2=0x7f4c3c138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:00.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.882+0000 7f4c3ad9d700 1 --2- 172.21.15.3:0/1346026884 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c3c1337f0 0x7f4c3c138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:00.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.882+0000 7f4c3ad9d700 1 -- 172.21.15.3:0/1346026884 >> v1:172.21.15.3:6789/0 conn(0x7f4c3c133420 legacy=0x7f4c3c133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:00.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.882+0000 7f4c3ad9d700 1 -- 172.21.15.3:0/1346026884 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c0a0590 con 0x7f4c3c138dc0 2023-10-03T10:17:00.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.882+0000 7f4c39d9b700 1 --2- 172.21.15.3:0/1346026884 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c3c1337f0 0x7f4c3c138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:00.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.882+0000 7f4c3ad9d700 1 --2- 172.21.15.3:0/1346026884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3c138dc0 0x7f4c3c13b1a0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f4c34009d90 tx=0x7f4c3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79139e010987f3ba server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:00.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.882+0000 7f4c3959a700 1 -- 172.21.15.3:0/1346026884 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3400e070 con 0x7f4c3c138dc0 2023-10-03T10:17:00.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.882+0000 7f4c3959a700 1 -- 172.21.15.3:0/1346026884 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3400bb20 con 0x7f4c3c138dc0 2023-10-03T10:17:00.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.882+0000 7f4c3959a700 1 -- 172.21.15.3:0/1346026884 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c34012430 con 0x7f4c3c138dc0 2023-10-03T10:17:00.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.883+0000 7f4c40e12700 1 -- 172.21.15.3:0/1346026884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3c138dc0 msgr2=0x7f4c3c13b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:00.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.883+0000 7f4c40e12700 1 --2- 172.21.15.3:0/1346026884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3c138dc0 0x7f4c3c13b1a0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f4c34009d90 tx=0x7f4c3400b330 comp rx=0 tx=0).stop 2023-10-03T10:17:00.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.883+0000 7f4c40e12700 1 -- 172.21.15.3:0/1346026884 shutdown_connections 2023-10-03T10:17:00.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.883+0000 7f4c40e12700 1 --2- 172.21.15.3:0/1346026884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3c138dc0 0x7f4c3c13b1a0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:00.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.883+0000 7f4c40e12700 1 --2- 172.21.15.3:0/1346026884 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c3c1337f0 0x7f4c3c138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:00.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.883+0000 7f4c40e12700 1 -- 172.21.15.3:0/1346026884 >> 172.21.15.3:0/1346026884 conn(0x7f4c3c0a4210 msgr2=0x7f4c3c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:00.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.883+0000 7f4c40e12700 1 -- 172.21.15.3:0/1346026884 shutdown_connections 2023-10-03T10:17:00.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.883+0000 7f4c40e12700 1 -- 172.21.15.3:0/1346026884 wait complete. 2023-10-03T10:17:00.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.884+0000 7f4c40e12700 1 Processor -- start 2023-10-03T10:17:00.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.884+0000 7f4c40e12700 1 -- start start 2023-10-03T10:17:00.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.885+0000 7f4c40e12700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3c133420 0x7f4c3c09c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:00.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.885+0000 7f4c40e12700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4c3c1337f0 0x7f4c3c09c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:00.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.885+0000 7f4c40e12700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c3c138dc0 0x7f4c3c09cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:00.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.885+0000 7f4c40e12700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c12f440 con 0x7f4c3c133420 2023-10-03T10:17:00.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.885+0000 7f4c40e12700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c130dc0 con 0x7f4c3c138dc0 2023-10-03T10:17:00.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.885+0000 7f4c40e12700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c1389c0 con 0x7f4c3c1337f0 2023-10-03T10:17:00.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.885+0000 7f4c3a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3c133420 0x7f4c3c09c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:00.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.885+0000 7f4c3a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3c133420 0x7f4c3c09c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49030/0 (socket says 172.21.15.3:49030) 2023-10-03T10:17:00.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.885+0000 7f4c3a59c700 1 -- 172.21.15.3:0/1839933875 learned_addr learned my addr 172.21.15.3:0/1839933875 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:00.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.885+0000 7f4c3ad9d700 1 --2- 172.21.15.3:0/1839933875 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c3c138dc0 0x7f4c3c09cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:00.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.885+0000 7f4c39d9b700 1 --2- 172.21.15.3:0/1839933875 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4c3c1337f0 0x7f4c3c09c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:00.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.885+0000 7f4c3a59c700 1 -- 172.21.15.3:0/1839933875 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4c3c1337f0 msgr2=0x7f4c3c09c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:00.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.885+0000 7f4c3a59c700 1 --2- 172.21.15.3:0/1839933875 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4c3c1337f0 0x7f4c3c09c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:00.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.885+0000 7f4c3a59c700 1 -- 172.21.15.3:0/1839933875 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c3c138dc0 msgr2=0x7f4c3c09cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:00.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.885+0000 7f4c3a59c700 1 --2- 172.21.15.3:0/1839933875 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c3c138dc0 0x7f4c3c09cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:00.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.885+0000 7f4c3a59c700 1 -- 172.21.15.3:0/1839933875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c300094a0 con 0x7f4c3c133420 2023-10-03T10:17:00.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.886+0000 7f4c3a59c700 1 --2- 172.21.15.3:0/1839933875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3c133420 0x7f4c3c09c2e0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00c730 tx=0x7f4c2c00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:00.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.886+0000 7f4c2affd700 1 -- 172.21.15.3:0/1839933875 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c005d70 con 0x7f4c3c133420 2023-10-03T10:17:00.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.886+0000 7f4c2affd700 1 -- 172.21.15.3:0/1839933875 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2c0070d0 con 0x7f4c3c133420 2023-10-03T10:17:00.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.886+0000 7f4c40e12700 1 -- 172.21.15.3:0/1839933875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c34009a70 con 0x7f4c3c133420 2023-10-03T10:17:00.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.886+0000 7f4c2affd700 1 -- 172.21.15.3:0/1839933875 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c007440 con 0x7f4c3c133420 2023-10-03T10:17:00.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.886+0000 7f4c40e12700 1 -- 172.21.15.3:0/1839933875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3c12e960 con 0x7f4c3c133420 2023-10-03T10:17:00.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.887+0000 7f4c2affd700 1 -- 172.21.15.3:0/1839933875 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4c2c0075a0 con 0x7f4c3c133420 2023-10-03T10:17:00.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.887+0000 7f4c2affd700 1 -- 172.21.15.3:0/1839933875 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4c2c03eae0 con 0x7f4c3c133420 2023-10-03T10:17:00.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.887+0000 7f4c40e12700 1 -- 172.21.15.3:0/1839933875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c3c12f610 con 0x7f4c3c133420 2023-10-03T10:17:00.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.897+0000 7f4c2affd700 1 -- 172.21.15.3:0/1839933875 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4c2c017020 con 0x7f4c3c133420 2023-10-03T10:17:00.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.913+0000 7f4220f19700 1 Processor -- start 2023-10-03T10:17:00.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.913+0000 7f4220f19700 1 -- start start 2023-10-03T10:17:00.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.914+0000 7f4220f19700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f421c09d6f0 0x7f421c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:00.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.914+0000 7f4220f19700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f421c09e440 0x7f421c134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:00.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.914+0000 7f4220f19700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f421c09b120 con 0x7f421c09e000 2023-10-03T10:17:00.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.914+0000 7f4220f19700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f421c06b460 con 0x7f421c09d6f0 2023-10-03T10:17:00.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.914+0000 7f4220f19700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f421c09a7e0 con 0x7f421c09e440 2023-10-03T10:17:00.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.914+0000 7f421ad9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f421c09e440 0x7f421c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:00.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.914+0000 7f421a59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f421c09d6f0 0x7f421c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:00.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.914+0000 7f421ad9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f421c09e440 0x7f421c134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36074/0 (socket says 172.21.15.3:36074) 2023-10-03T10:17:00.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.914+0000 7f421ad9d700 1 -- 172.21.15.3:0/1133039556 learned_addr learned my addr 172.21.15.3:0/1133039556 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:00.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.914+0000 7f421a59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f421c09d6f0 0x7f421c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49042/0 (socket says 172.21.15.3:49042) 2023-10-03T10:17:00.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.914+0000 7f4219d9b700 1 --1- 172.21.15.3:0/1133039556 >> v1:172.21.15.3:6789/0 conn(0x7f421c09e000 0x7f421c1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39998/0 (socket says 172.21.15.3:39998) 2023-10-03T10:17:00.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.914+0000 7f421ad9d700 1 -- 172.21.15.3:0/1133039556 >> v2:172.21.15.3:3300/0 conn(0x7f421c09d6f0 msgr2=0x7f421c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:00.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.914+0000 7f421ad9d700 1 --2- 172.21.15.3:0/1133039556 >> v2:172.21.15.3:3300/0 conn(0x7f421c09d6f0 0x7f421c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:00.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.914+0000 7f421ad9d700 1 -- 172.21.15.3:0/1133039556 >> v1:172.21.15.3:6789/0 conn(0x7f421c09e000 legacy=0x7f421c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:00.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.914+0000 7f421959a700 1 -- 172.21.15.3:0/1133039556 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2376143320 0 0) 0x7f421c09b120 con 0x7f421c09e000 2023-10-03T10:17:00.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.914+0000 7f421ad9d700 1 -- 172.21.15.3:0/1133039556 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421c09a500 con 0x7f421c09e440 2023-10-03T10:17:00.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.914+0000 7f421a59c700 1 --2- 172.21.15.3:0/1133039556 >> v2:172.21.15.3:3300/0 conn(0x7f421c09d6f0 0x7f421c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:00.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.915+0000 7f421ad9d700 1 --2- 172.21.15.3:0/1133039556 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f421c09e440 0x7f421c134100 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4214009e80 tx=0x7f421400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14b45e87c8834234 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:00.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.915+0000 7f421959a700 1 -- 172.21.15.3:0/1133039556 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421400e070 con 0x7f421c09e440 2023-10-03T10:17:00.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.915+0000 7f421959a700 1 -- 172.21.15.3:0/1133039556 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421400b8d0 con 0x7f421c09e440 2023-10-03T10:17:00.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.915+0000 7f421959a700 1 -- 172.21.15.3:0/1133039556 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421400bc80 con 0x7f421c09e440 2023-10-03T10:17:00.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.916+0000 7f4220f19700 1 -- 172.21.15.3:0/1133039556 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f421c09e440 msgr2=0x7f421c134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:00.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.916+0000 7f4220f19700 1 --2- 172.21.15.3:0/1133039556 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f421c09e440 0x7f421c134100 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4214009e80 tx=0x7f421400b330 comp rx=0 tx=0).stop 2023-10-03T10:17:00.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.916+0000 7f4220f19700 1 -- 172.21.15.3:0/1133039556 shutdown_connections 2023-10-03T10:17:00.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.916+0000 7f4220f19700 1 --2- 172.21.15.3:0/1133039556 >> v2:172.21.15.3:3300/0 conn(0x7f421c09d6f0 0x7f421c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:00.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.916+0000 7f4220f19700 1 --2- 172.21.15.3:0/1133039556 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f421c09e440 0x7f421c134100 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:00.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.916+0000 7f4220f19700 1 -- 172.21.15.3:0/1133039556 >> 172.21.15.3:0/1133039556 conn(0x7f421c066f50 msgr2=0x7f421c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:00.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.916+0000 7f4220f19700 1 -- 172.21.15.3:0/1133039556 shutdown_connections 2023-10-03T10:17:00.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.916+0000 7f4220f19700 1 -- 172.21.15.3:0/1133039556 wait complete. 2023-10-03T10:17:00.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.917+0000 7f4220f19700 1 Processor -- start 2023-10-03T10:17:00.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.917+0000 7f4220f19700 1 -- start start 2023-10-03T10:17:00.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.917+0000 7f4220f19700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f421c09d6f0 0x7f421c1a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:00.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.917+0000 7f4220f19700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f421c09e000 0x7f421c1a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:00.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.917+0000 7f4220f19700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f421c09e440 0x7f421c1c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:00.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.917+0000 7f4220f19700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421c1a7d80 con 0x7f421c09e000 2023-10-03T10:17:00.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.917+0000 7f4220f19700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f421c0a1d00 con 0x7f421c09e440 2023-10-03T10:17:00.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.917+0000 7f4220f19700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f421c1a6c70 con 0x7f421c09d6f0 2023-10-03T10:17:00.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.918+0000 7f4219d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f421c09e000 0x7f421c1a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:00.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.918+0000 7f421a59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f421c09d6f0 0x7f421c1a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:00.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.918+0000 7f421ad9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f421c09e440 0x7f421c1c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:00.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.918+0000 7f421a59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f421c09d6f0 0x7f421c1a5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36086/0 (socket says 172.21.15.3:36086) 2023-10-03T10:17:00.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.918+0000 7f4219d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f421c09e000 0x7f421c1a57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49048/0 (socket says 172.21.15.3:49048) 2023-10-03T10:17:00.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.918+0000 7f421a59c700 1 -- 172.21.15.3:0/461803854 learned_addr learned my addr 172.21.15.3:0/461803854 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:00.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.918+0000 7f421a59c700 1 -- 172.21.15.3:0/461803854 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f421c09e440 msgr2=0x7f421c1c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:00.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.918+0000 7f421a59c700 1 --2- 172.21.15.3:0/461803854 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f421c09e440 0x7f421c1c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:00.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.918+0000 7f421a59c700 1 -- 172.21.15.3:0/461803854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f421c09e000 msgr2=0x7f421c1a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:00.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.918+0000 7f421a59c700 1 --2- 172.21.15.3:0/461803854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f421c09e000 0x7f421c1a57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:00.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.918+0000 7f421a59c700 1 -- 172.21.15.3:0/461803854 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420c0094a0 con 0x7f421c09d6f0 2023-10-03T10:17:00.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.918+0000 7f4219d9b700 1 --2- 172.21.15.3:0/461803854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f421c09e000 0x7f421c1a57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:00.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.918+0000 7f421ad9d700 1 --2- 172.21.15.3:0/461803854 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f421c09e440 0x7f421c1c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:17:00.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.918+0000 7f421a59c700 1 --2- 172.21.15.3:0/461803854 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f421c09d6f0 0x7f421c1a5280 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f420c00ebe0 tx=0x7f420c00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:00.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.918+0000 7f420affd700 1 -- 172.21.15.3:0/461803854 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420c009800 con 0x7f421c09d6f0 2023-10-03T10:17:00.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.918+0000 7f420affd700 1 -- 172.21.15.3:0/461803854 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420c00c910 con 0x7f421c09d6f0 2023-10-03T10:17:00.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.918+0000 7f4220f19700 1 -- 172.21.15.3:0/461803854 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4214009b30 con 0x7f421c09d6f0 2023-10-03T10:17:00.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.919+0000 7f420affd700 1 -- 172.21.15.3:0/461803854 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420c00cc80 con 0x7f421c09d6f0 2023-10-03T10:17:00.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.919+0000 7f4220f19700 1 -- 172.21.15.3:0/461803854 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f421c1a4d20 con 0x7f421c09d6f0 2023-10-03T10:17:00.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.920+0000 7f4220f19700 1 -- 172.21.15.3:0/461803854 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f421c1b91f0 con 0x7f421c09d6f0 2023-10-03T10:17:00.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.920+0000 7f420affd700 1 -- 172.21.15.3:0/461803854 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f420c013470 con 0x7f421c09d6f0 2023-10-03T10:17:00.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.920+0000 7f420affd700 1 -- 172.21.15.3:0/461803854 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f420c03f880 con 0x7f421c09d6f0 2023-10-03T10:17:00.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:00.929+0000 7f420affd700 1 -- 172.21.15.3:0/461803854 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f420c013720 con 0x7f421c09d6f0 2023-10-03T10:17:01.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.033+0000 7fbd0b2b4700 1 -- 172.21.15.3:0/2845452163 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbd0406ed90 con 0x7fbd04131d10 2023-10-03T10:17:01.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.034+0000 7fbcf9ffb700 1 -- 172.21.15.3:0/2845452163 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbcfc02c070 con 0x7fbd04131d10 2023-10-03T10:17:01.034 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:01.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.036+0000 7fbd0b2b4700 1 -- 172.21.15.3:0/2845452163 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd04131d10 msgr2=0x7fbd041c8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:01.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.036+0000 7fbd0b2b4700 1 --2- 172.21.15.3:0/2845452163 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd04131d10 0x7fbd041c8eb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc016000 tx=0x7fbcfc004ad0 comp rx=0 tx=0).stop 2023-10-03T10:17:01.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.036+0000 7fbd0b2b4700 1 -- 172.21.15.3:0/2845452163 shutdown_connections 2023-10-03T10:17:01.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.036+0000 7fbd0b2b4700 1 --2- 172.21.15.3:0/2845452163 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd04131d10 0x7fbd041c8eb0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:01.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.036+0000 7fbd0b2b4700 1 --2- 172.21.15.3:0/2845452163 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd0406a770 0x7fbd041c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:01.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.036+0000 7fbd0b2b4700 1 --2- 172.21.15.3:0/2845452163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd0406b080 0x7fbd041c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:01.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.036+0000 7fbd0b2b4700 1 -- 172.21.15.3:0/2845452163 >> 172.21.15.3:0/2845452163 conn(0x7fbd04066f50 msgr2=0x7fbd040700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:01.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.036+0000 7fbd0b2b4700 1 -- 172.21.15.3:0/2845452163 shutdown_connections 2023-10-03T10:17:01.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.037+0000 7fbd0b2b4700 1 -- 172.21.15.3:0/2845452163 wait complete. 2023-10-03T10:17:01.051 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:01.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.111+0000 7f4c40e12700 1 -- 172.21.15.3:0/1839933875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4c3c131fe0 con 0x7f4c3c133420 2023-10-03T10:17:01.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.112+0000 7f4c2affd700 1 -- 172.21.15.3:0/1839933875 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4c2c019b40 con 0x7f4c3c133420 2023-10-03T10:17:01.112 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:01.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.114+0000 7f4c40e12700 1 -- 172.21.15.3:0/1839933875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3c133420 msgr2=0x7f4c3c09c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:01.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.114+0000 7f4c40e12700 1 --2- 172.21.15.3:0/1839933875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3c133420 0x7f4c3c09c2e0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00c730 tx=0x7f4c2c00ca40 comp rx=0 tx=0).stop 2023-10-03T10:17:01.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.114+0000 7f4c40e12700 1 -- 172.21.15.3:0/1839933875 shutdown_connections 2023-10-03T10:17:01.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.114+0000 7f4c40e12700 1 --2- 172.21.15.3:0/1839933875 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c3c138dc0 0x7f4c3c09cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:01.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.114+0000 7f4c40e12700 1 --2- 172.21.15.3:0/1839933875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3c133420 0x7f4c3c09c2e0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:01.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.114+0000 7f4c40e12700 1 --2- 172.21.15.3:0/1839933875 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4c3c1337f0 0x7f4c3c09c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:01.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.114+0000 7f4c40e12700 1 -- 172.21.15.3:0/1839933875 >> 172.21.15.3:0/1839933875 conn(0x7f4c3c0a4210 msgr2=0x7f4c3c0a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:01.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.114+0000 7f4c40e12700 1 -- 172.21.15.3:0/1839933875 shutdown_connections 2023-10-03T10:17:01.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.114+0000 7f4c40e12700 1 -- 172.21.15.3:0/1839933875 wait complete. 2023-10-03T10:17:01.128 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:01.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.152+0000 7f4220f19700 1 -- 172.21.15.3:0/461803854 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f421c09f420 con 0x7f421c09d6f0 2023-10-03T10:17:01.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.152+0000 7f420affd700 1 -- 172.21.15.3:0/461803854 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f420c021070 con 0x7f421c09d6f0 2023-10-03T10:17:01.153 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:01.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.154+0000 7f4220f19700 1 -- 172.21.15.3:0/461803854 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f421c09d6f0 msgr2=0x7f421c1a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:01.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.154+0000 7f4220f19700 1 --2- 172.21.15.3:0/461803854 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f421c09d6f0 0x7f421c1a5280 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f420c00ebe0 tx=0x7f420c00eef0 comp rx=0 tx=0).stop 2023-10-03T10:17:01.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.155+0000 7f4220f19700 1 -- 172.21.15.3:0/461803854 shutdown_connections 2023-10-03T10:17:01.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.155+0000 7f4220f19700 1 --2- 172.21.15.3:0/461803854 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f421c09e440 0x7f421c1c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:01.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.155+0000 7f4220f19700 1 --2- 172.21.15.3:0/461803854 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f421c09d6f0 0x7f421c1a5280 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:01.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.155+0000 7f4220f19700 1 --2- 172.21.15.3:0/461803854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f421c09e000 0x7f421c1a57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:01.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.155+0000 7f4220f19700 1 -- 172.21.15.3:0/461803854 >> 172.21.15.3:0/461803854 conn(0x7f421c066f50 msgr2=0x7f421c1bbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:01.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.155+0000 7f4220f19700 1 -- 172.21.15.3:0/461803854 shutdown_connections 2023-10-03T10:17:01.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.155+0000 7f4220f19700 1 -- 172.21.15.3:0/461803854 wait complete. 2023-10-03T10:17:01.168 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:01.836 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:01.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.982+0000 7f4e9aa60700 1 Processor -- start 2023-10-03T10:17:01.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.982+0000 7f4e9aa60700 1 -- start start 2023-10-03T10:17:01.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.983+0000 7f4e9aa60700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4e940a0a30 0x7f4e940a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:01.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.983+0000 7f4e9aa60700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4e940a1340 0x7f4e94133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:01.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.983+0000 7f4e9aa60700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4e94133fa0 0x7f4e94134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.983+0000 7f4e9aa60700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e9409edd0 con 0x7f4e940a0a30 2023-10-03T10:17:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.983+0000 7f4e9aa60700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9409e1f0 con 0x7f4e940a1340 2023-10-03T10:17:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.983+0000 7f4e9aa60700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9409dc50 con 0x7f4e94133fa0 2023-10-03T10:17:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.983+0000 7f4e93fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4e940a0a30 0x7f4e940a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.983+0000 7f4e937fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4e940a1340 0x7f4e94133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.983+0000 7f4e98ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4e94133fa0 0x7f4e94134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.983+0000 7f4e93fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4e940a0a30 0x7f4e940a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49060/0 (socket says 172.21.15.3:49060) 2023-10-03T10:17:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.983+0000 7f4e93fff700 1 -- 172.21.15.3:0/1921481899 learned_addr learned my addr 172.21.15.3:0/1921481899 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.983+0000 7f4e98ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4e94133fa0 0x7f4e94134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36088/0 (socket says 172.21.15.3:36088) 2023-10-03T10:17:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.983+0000 7f4e937fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4e940a1340 0x7f4e94133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33494/0 (socket says 172.21.15.3:33494) 2023-10-03T10:17:01.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.983+0000 7f4e93fff700 1 -- 172.21.15.3:0/1921481899 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4e94133fa0 msgr2=0x7f4e94134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.983+0000 7f4e93fff700 1 --2- 172.21.15.3:0/1921481899 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4e94133fa0 0x7f4e94134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.983+0000 7f4e93fff700 1 -- 172.21.15.3:0/1921481899 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4e940a1340 msgr2=0x7f4e94133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.983+0000 7f4e93fff700 1 --2- 172.21.15.3:0/1921481899 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4e940a1340 0x7f4e94133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.983+0000 7f4e93fff700 1 -- 172.21.15.3:0/1921481899 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e940a1fe0 con 0x7f4e940a0a30 2023-10-03T10:17:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.983+0000 7f4e98ffd700 1 --2- 172.21.15.3:0/1921481899 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4e94133fa0 0x7f4e94134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.983+0000 7f4e937fe700 1 --2- 172.21.15.3:0/1921481899 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4e940a1340 0x7f4e94133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.984+0000 7f4e93fff700 1 --2- 172.21.15.3:0/1921481899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e940a0a30 0x7f4e940a0e00 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f4e840098f0 tx=0x7f4e84009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2513966bbda4f28 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.984+0000 7f4e92ffd700 1 -- 172.21.15.3:0/1921481899 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e84011830 con 0x7f4e940a0a30 2023-10-03T10:17:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.984+0000 7f4e92ffd700 1 -- 172.21.15.3:0/1921481899 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e84011990 con 0x7f4e940a0a30 2023-10-03T10:17:01.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.984+0000 7f4e92ffd700 1 -- 172.21.15.3:0/1921481899 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e84011d20 con 0x7f4e940a0a30 2023-10-03T10:17:01.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.984+0000 7f4e9aa60700 1 -- 172.21.15.3:0/1921481899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e940a0a30 msgr2=0x7f4e940a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:01.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.984+0000 7f4e9aa60700 1 --2- 172.21.15.3:0/1921481899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e940a0a30 0x7f4e940a0e00 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f4e840098f0 tx=0x7f4e84009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:01.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.984+0000 7f4e9aa60700 1 -- 172.21.15.3:0/1921481899 shutdown_connections 2023-10-03T10:17:01.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.984+0000 7f4e9aa60700 1 --2- 172.21.15.3:0/1921481899 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4e94133fa0 0x7f4e94134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:01.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.984+0000 7f4e9aa60700 1 --2- 172.21.15.3:0/1921481899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e940a0a30 0x7f4e940a0e00 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:01.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.984+0000 7f4e9aa60700 1 --2- 172.21.15.3:0/1921481899 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4e940a1340 0x7f4e94133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:01.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.984+0000 7f4e9aa60700 1 -- 172.21.15.3:0/1921481899 >> 172.21.15.3:0/1921481899 conn(0x7f4e940a6550 msgr2=0x7f4e940a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:01.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.985+0000 7f4e9aa60700 1 -- 172.21.15.3:0/1921481899 shutdown_connections 2023-10-03T10:17:01.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.985+0000 7f4e9aa60700 1 -- 172.21.15.3:0/1921481899 wait complete. 2023-10-03T10:17:01.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.986+0000 7f4e9aa60700 1 Processor -- start 2023-10-03T10:17:01.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.986+0000 7f4e9aa60700 1 -- start start 2023-10-03T10:17:01.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.986+0000 7f4e9aa60700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e940a0a30 0x7f4e9411e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:01.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.986+0000 7f4e9aa60700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4e940a1340 0x7f4e9411e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:01.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.986+0000 7f4e9aa60700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4e94133fa0 0x7f4e9411eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:01.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.986+0000 7f4e9aa60700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9411fcc0 con 0x7f4e940a0a30 2023-10-03T10:17:01.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.986+0000 7f4e9aa60700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e94120f10 con 0x7f4e94133fa0 2023-10-03T10:17:01.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.986+0000 7f4e9aa60700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4e941202a0 con 0x7f4e940a1340 2023-10-03T10:17:01.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.986+0000 7f4e93fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e940a0a30 0x7f4e9411e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:01.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.986+0000 7f4e937fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4e940a1340 0x7f4e9411e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:01.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.986+0000 7f4e98ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4e94133fa0 0x7f4e9411eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:01.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.986+0000 7f4e93fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e940a0a30 0x7f4e9411e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49072/0 (socket says 172.21.15.3:49072) 2023-10-03T10:17:01.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.986+0000 7f4e93fff700 1 -- 172.21.15.3:0/3929392724 learned_addr learned my addr 172.21.15.3:0/3929392724 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:01.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.987+0000 7f4e937fe700 1 -- 172.21.15.3:0/3929392724 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4e94133fa0 msgr2=0x7f4e9411eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:01.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.987+0000 7f4e937fe700 1 --2- 172.21.15.3:0/3929392724 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4e94133fa0 0x7f4e9411eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:01.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.987+0000 7f4e937fe700 1 -- 172.21.15.3:0/3929392724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e940a0a30 msgr2=0x7f4e9411e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:01.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.987+0000 7f4e937fe700 1 --2- 172.21.15.3:0/3929392724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e940a0a30 0x7f4e9411e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:01.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.987+0000 7f4e937fe700 1 -- 172.21.15.3:0/3929392724 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7c0094a0 con 0x7f4e940a1340 2023-10-03T10:17:01.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.987+0000 7f4e93fff700 1 --2- 172.21.15.3:0/3929392724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e940a0a30 0x7f4e9411e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:01.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.987+0000 7f4e98ffd700 1 --2- 172.21.15.3:0/3929392724 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4e94133fa0 0x7f4e9411eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:01.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.987+0000 7f4e937fe700 1 --2- 172.21.15.3:0/3929392724 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4e940a1340 0x7f4e9411e5f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00e9b0 tx=0x7f4e7c00ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:01.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.987+0000 7f4e90ff9700 1 -- 172.21.15.3:0/3929392724 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c00ee40 con 0x7f4e940a1340 2023-10-03T10:17:01.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.987+0000 7f4e90ff9700 1 -- 172.21.15.3:0/3929392724 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7c00c850 con 0x7f4e940a1340 2023-10-03T10:17:01.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.987+0000 7f4e9aa60700 1 -- 172.21.15.3:0/3929392724 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e8c009a70 con 0x7f4e940a1340 2023-10-03T10:17:01.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.987+0000 7f4e90ff9700 1 -- 172.21.15.3:0/3929392724 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c00cc00 con 0x7f4e940a1340 2023-10-03T10:17:01.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.987+0000 7f4e9aa60700 1 -- 172.21.15.3:0/3929392724 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e84009550 con 0x7f4e940a1340 2023-10-03T10:17:01.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.988+0000 7f4e90ff9700 1 -- 172.21.15.3:0/3929392724 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4e7c013470 con 0x7f4e940a1340 2023-10-03T10:17:01.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.988+0000 7f4e90ff9700 1 -- 172.21.15.3:0/3929392724 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4e7c03f6f0 con 0x7f4e940a1340 2023-10-03T10:17:01.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.988+0000 7f4e9aa60700 1 -- 172.21.15.3:0/3929392724 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e9409b1e0 con 0x7f4e940a1340 2023-10-03T10:17:01.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:01.993+0000 7f4e90ff9700 1 -- 172.21.15.3:0/3929392724 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4e7c017030 con 0x7f4e940a1340 2023-10-03T10:17:02.051 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:02.129 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:02.169 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:02.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.200+0000 7f791e82e700 1 Processor -- start 2023-10-03T10:17:02.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.200+0000 7f791e82e700 1 -- start start 2023-10-03T10:17:02.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.201+0000 7f791e82e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f791809e1d0 0x7f7918135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:02.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.201+0000 7f791e82e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7918135e50 0x7f7918138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:02.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.201+0000 7f791e82e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f791809f130 con 0x7f791809d6f0 2023-10-03T10:17:02.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.201+0000 7f791e82e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f791809f520 con 0x7f7918135e50 2023-10-03T10:17:02.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.201+0000 7f791e82e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f791809e930 con 0x7f791809e1d0 2023-10-03T10:17:02.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.201+0000 7f7917fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f791809d6f0 0x7f791809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40004/0 (socket says 172.21.15.3:40004) 2023-10-03T10:17:02.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.201+0000 7f7917fff700 1 -- 172.21.15.3:0/4138941769 learned_addr learned my addr 172.21.15.3:0/4138941769 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:02.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.201+0000 7f791cdcb700 1 --2- 172.21.15.3:0/4138941769 >> v2:172.21.15.3:3300/0 conn(0x7f7918135e50 0x7f7918138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:02.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.201+0000 7f79177fe700 1 --2- 172.21.15.3:0/4138941769 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f791809e1d0 0x7f7918135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:02.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.202+0000 7f791cdcb700 1 -- 172.21.15.3:0/4138941769 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f791809e1d0 msgr2=0x7f7918135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:02.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.202+0000 7f791cdcb700 1 --2- 172.21.15.3:0/4138941769 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f791809e1d0 0x7f7918135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.202+0000 7f791cdcb700 1 -- 172.21.15.3:0/4138941769 >> v1:172.21.15.3:6789/0 conn(0x7f791809d6f0 legacy=0x7f791809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:02.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.202+0000 7f791cdcb700 1 -- 172.21.15.3:0/4138941769 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791809b2d0 con 0x7f7918135e50 2023-10-03T10:17:02.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.202+0000 7f791cdcb700 1 --2- 172.21.15.3:0/4138941769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7918135e50 0x7f7918138230 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f791000bfd0 tx=0x7f791000d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe79c9d66180f2ab server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:02.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.202+0000 7f7916ffd700 1 -- 172.21.15.3:0/4138941769 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7910010070 con 0x7f7918135e50 2023-10-03T10:17:02.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.202+0000 7f7916ffd700 1 -- 172.21.15.3:0/4138941769 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f791000b2e0 con 0x7f7918135e50 2023-10-03T10:17:02.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.202+0000 7f7916ffd700 1 -- 172.21.15.3:0/4138941769 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7910003c40 con 0x7f7918135e50 2023-10-03T10:17:02.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.203+0000 7f791e82e700 1 -- 172.21.15.3:0/4138941769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7918135e50 msgr2=0x7f7918138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:02.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.203+0000 7f791e82e700 1 --2- 172.21.15.3:0/4138941769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7918135e50 0x7f7918138230 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f791000bfd0 tx=0x7f791000d330 comp rx=0 tx=0).stop 2023-10-03T10:17:02.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.203+0000 7f791e82e700 1 -- 172.21.15.3:0/4138941769 shutdown_connections 2023-10-03T10:17:02.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.203+0000 7f791e82e700 1 --2- 172.21.15.3:0/4138941769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7918135e50 0x7f7918138230 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.203+0000 7f791e82e700 1 --2- 172.21.15.3:0/4138941769 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f791809e1d0 0x7f7918135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.203+0000 7f791e82e700 1 -- 172.21.15.3:0/4138941769 >> 172.21.15.3:0/4138941769 conn(0x7f7918066f50 msgr2=0x7f791809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:02.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.204+0000 7f791e82e700 1 -- 172.21.15.3:0/4138941769 shutdown_connections 2023-10-03T10:17:02.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.204+0000 7f791e82e700 1 -- 172.21.15.3:0/4138941769 wait complete. 2023-10-03T10:17:02.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.205+0000 7f791e82e700 1 Processor -- start 2023-10-03T10:17:02.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.205+0000 7f791e82e700 1 -- start start 2023-10-03T10:17:02.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.206+0000 7f791e82e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f791809d6f0 0x7f79181c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:02.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.206+0000 7f791e82e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f791809e1d0 0x7f79181c8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:02.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.206+0000 7f791e82e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7918135e50 0x7f79181c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:02.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.206+0000 7f791e82e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7918120cc0 con 0x7f791809e1d0 2023-10-03T10:17:02.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.206+0000 7f791e82e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791811f880 con 0x7f7918135e50 2023-10-03T10:17:02.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.206+0000 7f791e82e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79180a1d00 con 0x7f791809d6f0 2023-10-03T10:17:02.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.206+0000 7f791cdcb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7918135e50 0x7f79181c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:02.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.206+0000 7f791cdcb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7918135e50 0x7f79181c8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:33518/0 (socket says 172.21.15.3:33518) 2023-10-03T10:17:02.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.206+0000 7f79177fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f791809e1d0 0x7f79181c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:02.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.206+0000 7f791cdcb700 1 -- 172.21.15.3:0/3912802468 learned_addr learned my addr 172.21.15.3:0/3912802468 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:02.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.206+0000 7f79177fe700 1 -- 172.21.15.3:0/3912802468 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f791809d6f0 msgr2=0x7f79181c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:02.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.206+0000 7f79177fe700 1 --2- 172.21.15.3:0/3912802468 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f791809d6f0 0x7f79181c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.206+0000 7f79177fe700 1 -- 172.21.15.3:0/3912802468 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7918135e50 msgr2=0x7f79181c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:02.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.206+0000 7f79177fe700 1 --2- 172.21.15.3:0/3912802468 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7918135e50 0x7f79181c8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.207+0000 7f79177fe700 1 -- 172.21.15.3:0/3912802468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791000bc80 con 0x7f791809e1d0 2023-10-03T10:17:02.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.207+0000 7f791cdcb700 1 --2- 172.21.15.3:0/3912802468 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7918135e50 0x7f79181c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:02.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.207+0000 7f79177fe700 1 --2- 172.21.15.3:0/3912802468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f791809e1d0 0x7f79181c8720 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f790c00e900 tx=0x7f790c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:02.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.207+0000 7f7914ff9700 1 -- 172.21.15.3:0/3912802468 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c00ee50 con 0x7f791809e1d0 2023-10-03T10:17:02.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.207+0000 7f7914ff9700 1 -- 172.21.15.3:0/3912802468 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790c00c850 con 0x7f791809e1d0 2023-10-03T10:17:02.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.207+0000 7f7914ff9700 1 -- 172.21.15.3:0/3912802468 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c00cbc0 con 0x7f791809e1d0 2023-10-03T10:17:02.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.207+0000 7f791e82e700 1 -- 172.21.15.3:0/3912802468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f791811de30 con 0x7f791809e1d0 2023-10-03T10:17:02.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.207+0000 7f791e82e700 1 -- 172.21.15.3:0/3912802468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f791811dac0 con 0x7f791809e1d0 2023-10-03T10:17:02.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.208+0000 7f791e82e700 1 -- 172.21.15.3:0/3912802468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7918074ea0 con 0x7f791809e1d0 2023-10-03T10:17:02.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.208+0000 7f7914ff9700 1 -- 172.21.15.3:0/3912802468 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f790c00cd20 con 0x7f791809e1d0 2023-10-03T10:17:02.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.209+0000 7f7914ff9700 1 -- 172.21.15.3:0/3912802468 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f790c03f730 con 0x7f791809e1d0 2023-10-03T10:17:02.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.215+0000 7f4e9aa60700 1 -- 172.21.15.3:0/3929392724 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4e940a3530 con 0x7f4e940a1340 2023-10-03T10:17:02.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.216+0000 7f4e90ff9700 1 -- 172.21.15.3:0/3929392724 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4e7c02a090 con 0x7f4e940a1340 2023-10-03T10:17:02.217 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:02.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.218+0000 7f7914ff9700 1 -- 172.21.15.3:0/3912802468 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f790c01b3b0 con 0x7f791809e1d0 2023-10-03T10:17:02.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.219+0000 7f4e9aa60700 1 -- 172.21.15.3:0/3929392724 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4e940a1340 msgr2=0x7f4e9411e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:02.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.219+0000 7f4e9aa60700 1 --2- 172.21.15.3:0/3929392724 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4e940a1340 0x7f4e9411e5f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00e9b0 tx=0x7f4e7c00ecc0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.219+0000 7f4e9aa60700 1 -- 172.21.15.3:0/3929392724 shutdown_connections 2023-10-03T10:17:02.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.219+0000 7f4e9aa60700 1 --2- 172.21.15.3:0/3929392724 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4e94133fa0 0x7f4e9411eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.219+0000 7f4e9aa60700 1 --2- 172.21.15.3:0/3929392724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e940a0a30 0x7f4e9411e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.219+0000 7f4e9aa60700 1 --2- 172.21.15.3:0/3929392724 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4e940a1340 0x7f4e9411e5f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.219+0000 7f4e9aa60700 1 -- 172.21.15.3:0/3929392724 >> 172.21.15.3:0/3929392724 conn(0x7f4e940a6550 msgr2=0x7f4e941330f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:02.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.219+0000 7f4e9aa60700 1 -- 172.21.15.3:0/3929392724 shutdown_connections 2023-10-03T10:17:02.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.219+0000 7f4e9aa60700 1 -- 172.21.15.3:0/3929392724 wait complete. 2023-10-03T10:17:02.236 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:02.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.284+0000 7ff76c06e700 1 Processor -- start 2023-10-03T10:17:02.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.285+0000 7ff76c06e700 1 -- start start 2023-10-03T10:17:02.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.285+0000 7ff76c06e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7641338a0 0x7ff764133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:02.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.285+0000 7ff76c06e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff76409de50 0x7ff76409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:02.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.285+0000 7ff76c06e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff76409e760 0x7ff76413a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:02.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.285+0000 7ff76c06e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff76406fef0 con 0x7ff76409e760 2023-10-03T10:17:02.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.285+0000 7ff76c06e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff76406ef30 con 0x7ff7641338a0 2023-10-03T10:17:02.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.285+0000 7ff76c06e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff7641334b0 con 0x7ff76409de50 2023-10-03T10:17:02.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.285+0000 7ff769609700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff76409de50 0x7ff76409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:02.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.285+0000 7ff76a60b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff76409e760 0x7ff76413a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:02.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.286+0000 7ff769609700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff76409de50 0x7ff76409e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36136/0 (socket says 172.21.15.3:36136) 2023-10-03T10:17:02.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.286+0000 7ff76a60b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff76409e760 0x7ff76413a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49096/0 (socket says 172.21.15.3:49096) 2023-10-03T10:17:02.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.286+0000 7ff769609700 1 -- 172.21.15.3:0/937946106 learned_addr learned my addr 172.21.15.3:0/937946106 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.286+0000 7ff769e0a700 1 --2- 172.21.15.3:0/937946106 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7641338a0 0x7ff764133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.286+0000 7ff76a60b700 1 -- 172.21.15.3:0/937946106 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff76409de50 msgr2=0x7ff76409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.286+0000 7ff76a60b700 1 --2- 172.21.15.3:0/937946106 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff76409de50 0x7ff76409e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.286+0000 7ff76a60b700 1 -- 172.21.15.3:0/937946106 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7641338a0 msgr2=0x7ff764133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.286+0000 7ff76a60b700 1 --2- 172.21.15.3:0/937946106 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7641338a0 0x7ff764133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.286+0000 7ff76a60b700 1 -- 172.21.15.3:0/937946106 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76406fc30 con 0x7ff76409e760 2023-10-03T10:17:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.286+0000 7ff769609700 1 --2- 172.21.15.3:0/937946106 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff76409de50 0x7ff76409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.286+0000 7ff76a60b700 1 --2- 172.21.15.3:0/937946106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff76409e760 0x7ff76413a730 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009d90 tx=0x7ff75c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2caa5277916fae28 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.286+0000 7ff768e08700 1 -- 172.21.15.3:0/937946106 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c00e070 con 0x7ff76409e760 2023-10-03T10:17:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.286+0000 7ff768e08700 1 -- 172.21.15.3:0/937946106 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75c00bb20 con 0x7ff76409e760 2023-10-03T10:17:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.286+0000 7ff768e08700 1 -- 172.21.15.3:0/937946106 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c012430 con 0x7ff76409e760 2023-10-03T10:17:02.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.287+0000 7ff76c06e700 1 -- 172.21.15.3:0/937946106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff76409e760 msgr2=0x7ff76413a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:02.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.287+0000 7ff76c06e700 1 --2- 172.21.15.3:0/937946106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff76409e760 0x7ff76413a730 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009d90 tx=0x7ff75c00b330 comp rx=0 tx=0).stop 2023-10-03T10:17:02.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.287+0000 7ff76c06e700 1 -- 172.21.15.3:0/937946106 shutdown_connections 2023-10-03T10:17:02.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.287+0000 7ff76c06e700 1 --2- 172.21.15.3:0/937946106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff76409e760 0x7ff76413a730 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.287+0000 7ff76c06e700 1 --2- 172.21.15.3:0/937946106 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff7641338a0 0x7ff764133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.287+0000 7ff76c06e700 1 --2- 172.21.15.3:0/937946106 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff76409de50 0x7ff76409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.287+0000 7ff76c06e700 1 -- 172.21.15.3:0/937946106 >> 172.21.15.3:0/937946106 conn(0x7ff7640a2aa0 msgr2=0x7ff7640a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:02.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.287+0000 7ff76c06e700 1 -- 172.21.15.3:0/937946106 shutdown_connections 2023-10-03T10:17:02.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.287+0000 7ff76c06e700 1 -- 172.21.15.3:0/937946106 wait complete. 2023-10-03T10:17:02.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.288+0000 7ff76c06e700 1 Processor -- start 2023-10-03T10:17:02.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.288+0000 7ff76c06e700 1 -- start start 2023-10-03T10:17:02.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.289+0000 7ff76c06e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff76409de50 0x7ff76411b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:02.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.289+0000 7ff76c06e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff76409e760 0x7ff76411bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:02.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.289+0000 7ff76c06e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff7641338a0 0x7ff76411c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:02.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.289+0000 7ff76c06e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76411f2c0 con 0x7ff76409de50 2023-10-03T10:17:02.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.289+0000 7ff76c06e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff76411e690 con 0x7ff76409e760 2023-10-03T10:17:02.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.289+0000 7ff76c06e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff76411d990 con 0x7ff7641338a0 2023-10-03T10:17:02.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.289+0000 7ff769e0a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff76409de50 0x7ff76411b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:02.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.289+0000 7ff769e0a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff76409de50 0x7ff76411b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49106/0 (socket says 172.21.15.3:49106) 2023-10-03T10:17:02.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.289+0000 7ff769e0a700 1 -- 172.21.15.3:0/2297063548 learned_addr learned my addr 172.21.15.3:0/2297063548 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:02.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.289+0000 7ff769609700 1 --2- 172.21.15.3:0/2297063548 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff76409e760 0x7ff76411bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:02.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.289+0000 7ff76a60b700 1 --2- 172.21.15.3:0/2297063548 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff7641338a0 0x7ff76411c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:02.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.289+0000 7ff769e0a700 1 -- 172.21.15.3:0/2297063548 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff7641338a0 msgr2=0x7ff76411c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:02.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.289+0000 7ff769e0a700 1 --2- 172.21.15.3:0/2297063548 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff7641338a0 0x7ff76411c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.289+0000 7ff769e0a700 1 -- 172.21.15.3:0/2297063548 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff76409e760 msgr2=0x7ff76411bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:02.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.289+0000 7ff769e0a700 1 --2- 172.21.15.3:0/2297063548 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff76409e760 0x7ff76411bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.289+0000 7ff769e0a700 1 -- 172.21.15.3:0/2297063548 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff754009550 con 0x7ff76409de50 2023-10-03T10:17:02.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.289+0000 7ff76a60b700 1 --2- 172.21.15.3:0/2297063548 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff7641338a0 0x7ff76411c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:17:02.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.290+0000 7ff769e0a700 1 --2- 172.21.15.3:0/2297063548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff76409de50 0x7ff76411b7a0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7ff76000e9e0 tx=0x7ff76000ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:02.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.290+0000 7ff75a7fc700 1 -- 172.21.15.3:0/2297063548 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff760009710 con 0x7ff76409de50 2023-10-03T10:17:02.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.290+0000 7ff75a7fc700 1 -- 172.21.15.3:0/2297063548 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76000c850 con 0x7ff76409de50 2023-10-03T10:17:02.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.290+0000 7ff76c06e700 1 -- 172.21.15.3:0/2297063548 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75c009a70 con 0x7ff76409de50 2023-10-03T10:17:02.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.290+0000 7ff75a7fc700 1 -- 172.21.15.3:0/2297063548 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76000cbc0 con 0x7ff76409de50 2023-10-03T10:17:02.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.290+0000 7ff76c06e700 1 -- 172.21.15.3:0/2297063548 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7640a0f10 con 0x7ff76409de50 2023-10-03T10:17:02.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.291+0000 7ff76c06e700 1 -- 172.21.15.3:0/2297063548 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff764074ea0 con 0x7ff76409de50 2023-10-03T10:17:02.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.291+0000 7ff75a7fc700 1 -- 172.21.15.3:0/2297063548 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff76000cd20 con 0x7ff76409de50 2023-10-03T10:17:02.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.292+0000 7ff75a7fc700 1 -- 172.21.15.3:0/2297063548 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff76003faf0 con 0x7ff76409de50 2023-10-03T10:17:02.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.297+0000 7ff75a7fc700 1 -- 172.21.15.3:0/2297063548 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff76001adc0 con 0x7ff76409de50 2023-10-03T10:17:02.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.323+0000 7f0dd67c4700 1 Processor -- start 2023-10-03T10:17:02.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.323+0000 7f0dd67c4700 1 -- start start 2023-10-03T10:17:02.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.323+0000 7f0dd67c4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dd009e850 0x7f0dd009ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:02.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.323+0000 7f0dd67c4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0dd009f160 0x7f0dd0133b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:02.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.323+0000 7f0dd67c4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd006bc50 con 0x7f0dd01340c0 2023-10-03T10:17:02.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.323+0000 7f0dd67c4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd009ae60 con 0x7f0dd009f160 2023-10-03T10:17:02.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.323+0000 7f0dd67c4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd006b990 con 0x7f0dd009e850 2023-10-03T10:17:02.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.323+0000 7f0dcffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dd009e850 0x7f0dd009ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:02.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.323+0000 7f0dcffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dd009e850 0x7f0dd009ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36150/0 (socket says 172.21.15.3:36150) 2023-10-03T10:17:02.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.323+0000 7f0dcffff700 1 -- 172.21.15.3:0/399633811 learned_addr learned my addr 172.21.15.3:0/399633811 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:02.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.324+0000 7f0dcf7fe700 1 --2- 172.21.15.3:0/399633811 >> v2:172.21.15.3:3300/0 conn(0x7f0dd009f160 0x7f0dd0133b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:02.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.324+0000 7f0dcffff700 1 -- 172.21.15.3:0/399633811 >> v2:172.21.15.3:3300/0 conn(0x7f0dd009f160 msgr2=0x7f0dd0133b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:02.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.324+0000 7f0dceffd700 1 -- 172.21.15.3:0/399633811 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 434655925 0 0) 0x7f0dd006bc50 con 0x7f0dd01340c0 2023-10-03T10:17:02.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.324+0000 7f0dcffff700 1 --2- 172.21.15.3:0/399633811 >> v2:172.21.15.3:3300/0 conn(0x7f0dd009f160 0x7f0dd0133b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.324+0000 7f0dcffff700 1 -- 172.21.15.3:0/399633811 >> v1:172.21.15.3:6789/0 conn(0x7f0dd01340c0 legacy=0x7f0dd013ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:02.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.324+0000 7f0dcffff700 1 -- 172.21.15.3:0/399633811 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd006cb10 con 0x7f0dd009e850 2023-10-03T10:17:02.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.324+0000 7f0dcf7fe700 1 --2- 172.21.15.3:0/399633811 >> v2:172.21.15.3:3300/0 conn(0x7f0dd009f160 0x7f0dd0133b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:02.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.324+0000 7f0dcffff700 1 --2- 172.21.15.3:0/399633811 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dd009e850 0x7f0dd009ec20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00098f0 tx=0x7f0dc0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99a94ca48d2abe36 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:02.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.324+0000 7f0dceffd700 1 -- 172.21.15.3:0/399633811 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0011830 con 0x7f0dd009e850 2023-10-03T10:17:02.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.324+0000 7f0dceffd700 1 -- 172.21.15.3:0/399633811 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc0011990 con 0x7f0dd009e850 2023-10-03T10:17:02.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.324+0000 7f0dceffd700 1 -- 172.21.15.3:0/399633811 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0011d20 con 0x7f0dd009e850 2023-10-03T10:17:02.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.325+0000 7f0dd67c4700 1 -- 172.21.15.3:0/399633811 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dd009e850 msgr2=0x7f0dd009ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:02.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.325+0000 7f0dd67c4700 1 --2- 172.21.15.3:0/399633811 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dd009e850 0x7f0dd009ec20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00098f0 tx=0x7f0dc0009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:02.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.325+0000 7f0dd67c4700 1 -- 172.21.15.3:0/399633811 shutdown_connections 2023-10-03T10:17:02.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.325+0000 7f0dd67c4700 1 --2- 172.21.15.3:0/399633811 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dd009e850 0x7f0dd009ec20 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.325+0000 7f0dd67c4700 1 --2- 172.21.15.3:0/399633811 >> v2:172.21.15.3:3300/0 conn(0x7f0dd009f160 0x7f0dd0133b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.325+0000 7f0dd67c4700 1 -- 172.21.15.3:0/399633811 >> 172.21.15.3:0/399633811 conn(0x7f0dd00a3f70 msgr2=0x7f0dd00a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:02.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.325+0000 7f0dd67c4700 1 -- 172.21.15.3:0/399633811 shutdown_connections 2023-10-03T10:17:02.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.325+0000 7f0dd67c4700 1 -- 172.21.15.3:0/399633811 wait complete. 2023-10-03T10:17:02.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.327+0000 7f0dd67c4700 1 Processor -- start 2023-10-03T10:17:02.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.327+0000 7f0dd67c4700 1 -- start start 2023-10-03T10:17:02.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.328+0000 7f0dd67c4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dd009e850 0x7f0dd0123420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:02.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.328+0000 7f0dd67c4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dd009f160 0x7f0dd0123960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:02.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.328+0000 7f0dd67c4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dd01340c0 0x7f0dd01c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:02.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.328+0000 7f0dd67c4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd0126720 con 0x7f0dd01340c0 2023-10-03T10:17:02.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.328+0000 7f0dd67c4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd00a1750 con 0x7f0dd009e850 2023-10-03T10:17:02.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.328+0000 7f0dd67c4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd0122fa0 con 0x7f0dd009f160 2023-10-03T10:17:02.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.328+0000 7f0dcf7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dd009f160 0x7f0dd0123960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:02.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.328+0000 7f0dcf7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dd009f160 0x7f0dd0123960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36156/0 (socket says 172.21.15.3:36156) 2023-10-03T10:17:02.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.328+0000 7f0dcf7fe700 1 -- 172.21.15.3:0/1498545935 learned_addr learned my addr 172.21.15.3:0/1498545935 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:02.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.328+0000 7f0dcffff700 1 --2- 172.21.15.3:0/1498545935 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dd009e850 0x7f0dd0123420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:02.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.328+0000 7f0dd4d61700 1 --2- 172.21.15.3:0/1498545935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dd01340c0 0x7f0dd01c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:02.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.328+0000 7f0dcf7fe700 1 -- 172.21.15.3:0/1498545935 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dd009e850 msgr2=0x7f0dd0123420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:02.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.328+0000 7f0dcf7fe700 1 --2- 172.21.15.3:0/1498545935 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dd009e850 0x7f0dd0123420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.328+0000 7f0dcf7fe700 1 -- 172.21.15.3:0/1498545935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dd01340c0 msgr2=0x7f0dd01c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:02.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.328+0000 7f0dcf7fe700 1 --2- 172.21.15.3:0/1498545935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dd01340c0 0x7f0dd01c8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.328+0000 7f0dcf7fe700 1 -- 172.21.15.3:0/1498545935 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc40094a0 con 0x7f0dd009f160 2023-10-03T10:17:02.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.328+0000 7f0dcffff700 1 --2- 172.21.15.3:0/1498545935 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dd009e850 0x7f0dd0123420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:02.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.328+0000 7f0dcf7fe700 1 --2- 172.21.15.3:0/1498545935 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dd009f160 0x7f0dd0123960 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400ebe0 tx=0x7f0dc400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:02.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.329+0000 7f0dccff9700 1 -- 172.21.15.3:0/1498545935 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc400c850 con 0x7f0dd009f160 2023-10-03T10:17:02.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.329+0000 7f0dccff9700 1 -- 172.21.15.3:0/1498545935 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc400c9b0 con 0x7f0dd009f160 2023-10-03T10:17:02.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.329+0000 7f0dd67c4700 1 -- 172.21.15.3:0/1498545935 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc0009550 con 0x7f0dd009f160 2023-10-03T10:17:02.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.329+0000 7f0dccff9700 1 -- 172.21.15.3:0/1498545935 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc400cd40 con 0x7f0dd009f160 2023-10-03T10:17:02.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.329+0000 7f0dd67c4700 1 -- 172.21.15.3:0/1498545935 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd0126db0 con 0x7f0dd009f160 2023-10-03T10:17:02.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.330+0000 7f0dd67c4700 1 -- 172.21.15.3:0/1498545935 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd00747f0 con 0x7f0dd009f160 2023-10-03T10:17:02.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.330+0000 7f0dccff9700 1 -- 172.21.15.3:0/1498545935 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0dc4013470 con 0x7f0dd009f160 2023-10-03T10:17:02.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.330+0000 7f0dccff9700 1 -- 172.21.15.3:0/1498545935 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0dc403ffa0 con 0x7f0dd009f160 2023-10-03T10:17:02.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.335+0000 7f0dccff9700 1 -- 172.21.15.3:0/1498545935 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0dc4013720 con 0x7f0dd009f160 2023-10-03T10:17:02.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.430+0000 7f791e82e700 1 -- 172.21.15.3:0/3912802468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f791806c360 con 0x7f791809e1d0 2023-10-03T10:17:02.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.430+0000 7f7914ff9700 1 -- 172.21.15.3:0/3912802468 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f790c021070 con 0x7f791809e1d0 2023-10-03T10:17:02.431 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:02.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.432+0000 7f791e82e700 1 -- 172.21.15.3:0/3912802468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f791809e1d0 msgr2=0x7f79181c8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:02.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.432+0000 7f791e82e700 1 --2- 172.21.15.3:0/3912802468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f791809e1d0 0x7f79181c8720 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f790c00e900 tx=0x7f790c00ec10 comp rx=0 tx=0).stop 2023-10-03T10:17:02.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.433+0000 7f791e82e700 1 -- 172.21.15.3:0/3912802468 shutdown_connections 2023-10-03T10:17:02.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.433+0000 7f791e82e700 1 --2- 172.21.15.3:0/3912802468 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7918135e50 0x7f79181c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.433+0000 7f791e82e700 1 --2- 172.21.15.3:0/3912802468 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f791809d6f0 0x7f79181c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.433+0000 7f791e82e700 1 --2- 172.21.15.3:0/3912802468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f791809e1d0 0x7f79181c8720 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.433+0000 7f791e82e700 1 -- 172.21.15.3:0/3912802468 >> 172.21.15.3:0/3912802468 conn(0x7f7918066f50 msgr2=0x7f79180a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:02.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.433+0000 7f791e82e700 1 -- 172.21.15.3:0/3912802468 shutdown_connections 2023-10-03T10:17:02.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.433+0000 7f791e82e700 1 -- 172.21.15.3:0/3912802468 wait complete. 2023-10-03T10:17:02.447 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:02.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.506+0000 7ff76c06e700 1 -- 172.21.15.3:0/2297063548 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff76412eb90 con 0x7ff76409de50 2023-10-03T10:17:02.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.506+0000 7ff75a7fc700 1 -- 172.21.15.3:0/2297063548 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff76003fe60 con 0x7ff76409de50 2023-10-03T10:17:02.507 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:02.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.509+0000 7ff76c06e700 1 -- 172.21.15.3:0/2297063548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff76409de50 msgr2=0x7ff76411b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:02.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.509+0000 7ff76c06e700 1 --2- 172.21.15.3:0/2297063548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff76409de50 0x7ff76411b7a0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7ff76000e9e0 tx=0x7ff76000ecf0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.509+0000 7ff76c06e700 1 -- 172.21.15.3:0/2297063548 shutdown_connections 2023-10-03T10:17:02.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.509+0000 7ff76c06e700 1 --2- 172.21.15.3:0/2297063548 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff7641338a0 0x7ff76411c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.509+0000 7ff76c06e700 1 --2- 172.21.15.3:0/2297063548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff76409de50 0x7ff76411b7a0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.509+0000 7ff76c06e700 1 --2- 172.21.15.3:0/2297063548 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff76409e760 0x7ff76411bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.509+0000 7ff76c06e700 1 -- 172.21.15.3:0/2297063548 >> 172.21.15.3:0/2297063548 conn(0x7ff7640a2aa0 msgr2=0x7ff7641305a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:02.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.509+0000 7ff76c06e700 1 -- 172.21.15.3:0/2297063548 shutdown_connections 2023-10-03T10:17:02.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.509+0000 7ff76c06e700 1 -- 172.21.15.3:0/2297063548 wait complete. 2023-10-03T10:17:02.523 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:02.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.543+0000 7f0dd67c4700 1 -- 172.21.15.3:0/1498545935 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0dd0126580 con 0x7f0dd009f160 2023-10-03T10:17:02.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.543+0000 7f0dccff9700 1 -- 172.21.15.3:0/1498545935 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0dd0126580 con 0x7f0dd009f160 2023-10-03T10:17:02.544 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:02.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.546+0000 7f0dd67c4700 1 -- 172.21.15.3:0/1498545935 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dd009f160 msgr2=0x7f0dd0123960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:02.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.546+0000 7f0dd67c4700 1 --2- 172.21.15.3:0/1498545935 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dd009f160 0x7f0dd0123960 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400ebe0 tx=0x7f0dc400eef0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.546+0000 7f0dd67c4700 1 -- 172.21.15.3:0/1498545935 shutdown_connections 2023-10-03T10:17:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.546+0000 7f0dd67c4700 1 --2- 172.21.15.3:0/1498545935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dd01340c0 0x7f0dd01c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.546+0000 7f0dd67c4700 1 --2- 172.21.15.3:0/1498545935 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0dd009e850 0x7f0dd0123420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.546+0000 7f0dd67c4700 1 --2- 172.21.15.3:0/1498545935 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0dd009f160 0x7f0dd0123960 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.546+0000 7f0dd67c4700 1 -- 172.21.15.3:0/1498545935 >> 172.21.15.3:0/1498545935 conn(0x7f0dd00a3f70 msgr2=0x7f0dd0070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.546+0000 7f0dd67c4700 1 -- 172.21.15.3:0/1498545935 shutdown_connections 2023-10-03T10:17:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:02.546+0000 7f0dd67c4700 1 -- 172.21.15.3:0/1498545935 wait complete. 2023-10-03T10:17:02.560 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:03.236 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:03.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.384+0000 7fd399a95700 1 Processor -- start 2023-10-03T10:17:03.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.384+0000 7fd399a95700 1 -- start start 2023-10-03T10:17:03.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.385+0000 7fd399a95700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd39409e930 0x7fd39409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.385+0000 7fd399a95700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd39409f240 0x7fd3941317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.385+0000 7fd399a95700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd39409f650 0x7fd394131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.385+0000 7fd399a95700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd394070940 con 0x7fd39409e930 2023-10-03T10:17:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.385+0000 7fd399a95700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39406fd60 con 0x7fd39409f650 2023-10-03T10:17:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.385+0000 7fd399a95700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd39406fa80 con 0x7fd39409f240 2023-10-03T10:17:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.385+0000 7fd3937fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd39409e930 0x7fd39409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.385+0000 7fd393fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd39409f650 0x7fd394131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.385+0000 7fd392ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd39409f240 0x7fd3941317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.385+0000 7fd393fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd39409f650 0x7fd394131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50174/0 (socket says 172.21.15.3:50174) 2023-10-03T10:17:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.385+0000 7fd392ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd39409f240 0x7fd3941317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:49724/0 (socket says 172.21.15.3:49724) 2023-10-03T10:17:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.385+0000 7fd3937fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd39409e930 0x7fd39409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36818/0 (socket says 172.21.15.3:36818) 2023-10-03T10:17:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.385+0000 7fd393fff700 1 -- 172.21.15.3:0/1999208766 learned_addr learned my addr 172.21.15.3:0/1999208766 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:03.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.385+0000 7fd393fff700 1 -- 172.21.15.3:0/1999208766 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd39409f240 msgr2=0x7fd3941317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:03.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.385+0000 7fd393fff700 1 --2- 172.21.15.3:0/1999208766 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd39409f240 0x7fd3941317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.385+0000 7fd393fff700 1 -- 172.21.15.3:0/1999208766 >> v2:172.21.15.3:3300/0 conn(0x7fd39409e930 msgr2=0x7fd39409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:03.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.385+0000 7fd393fff700 1 --2- 172.21.15.3:0/1999208766 >> v2:172.21.15.3:3300/0 conn(0x7fd39409e930 0x7fd39409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.385+0000 7fd393fff700 1 -- 172.21.15.3:0/1999208766 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd39409fb60 con 0x7fd39409f650 2023-10-03T10:17:03.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.385+0000 7fd392ffd700 1 --2- 172.21.15.3:0/1999208766 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd39409f240 0x7fd3941317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:03.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.385+0000 7fd3937fe700 1 --2- 172.21.15.3:0/1999208766 >> v2:172.21.15.3:3300/0 conn(0x7fd39409e930 0x7fd39409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:03.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.385+0000 7fd393fff700 1 --2- 172.21.15.3:0/1999208766 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd39409f650 0x7fd394131d30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd38c009e50 tx=0x7fd38c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad1275777974d9c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:03.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.385+0000 7fd3927fc700 1 -- 172.21.15.3:0/1999208766 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38c00e070 con 0x7fd39409f650 2023-10-03T10:17:03.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.385+0000 7fd3927fc700 1 -- 172.21.15.3:0/1999208766 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38c00b910 con 0x7fd39409f650 2023-10-03T10:17:03.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.386+0000 7fd3927fc700 1 -- 172.21.15.3:0/1999208766 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38c00bcc0 con 0x7fd39409f650 2023-10-03T10:17:03.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.386+0000 7fd399a95700 1 -- 172.21.15.3:0/1999208766 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd39409f650 msgr2=0x7fd394131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:03.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.386+0000 7fd399a95700 1 --2- 172.21.15.3:0/1999208766 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd39409f650 0x7fd394131d30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd38c009e50 tx=0x7fd38c00b330 comp rx=0 tx=0).stop 2023-10-03T10:17:03.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.386+0000 7fd399a95700 1 -- 172.21.15.3:0/1999208766 shutdown_connections 2023-10-03T10:17:03.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.386+0000 7fd399a95700 1 --2- 172.21.15.3:0/1999208766 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd39409f650 0x7fd394131d30 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.386+0000 7fd399a95700 1 --2- 172.21.15.3:0/1999208766 >> v2:172.21.15.3:3300/0 conn(0x7fd39409e930 0x7fd39409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.386+0000 7fd399a95700 1 --2- 172.21.15.3:0/1999208766 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd39409f240 0x7fd3941317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.386+0000 7fd399a95700 1 -- 172.21.15.3:0/1999208766 >> 172.21.15.3:0/1999208766 conn(0x7fd3940a4210 msgr2=0x7fd3940a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:03.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.386+0000 7fd399a95700 1 -- 172.21.15.3:0/1999208766 shutdown_connections 2023-10-03T10:17:03.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.386+0000 7fd399a95700 1 -- 172.21.15.3:0/1999208766 wait complete. 2023-10-03T10:17:03.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.387+0000 7fd399a95700 1 Processor -- start 2023-10-03T10:17:03.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.387+0000 7fd399a95700 1 -- start start 2023-10-03T10:17:03.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.388+0000 7fd399a95700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd39409e930 0x7fd39419f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:03.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.388+0000 7fd399a95700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd39409f240 0x7fd39419f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:03.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.388+0000 7fd399a95700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd39409f650 0x7fd39419fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:03.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.388+0000 7fd399a95700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3941ad260 con 0x7fd39409e930 2023-10-03T10:17:03.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.388+0000 7fd399a95700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3940a31f0 con 0x7fd39409f240 2023-10-03T10:17:03.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.388+0000 7fd399a95700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd394071d20 con 0x7fd39409f650 2023-10-03T10:17:03.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.388+0000 7fd3937fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd39409e930 0x7fd39419f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:03.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.388+0000 7fd392ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd39409f240 0x7fd39419f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:03.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.388+0000 7fd393fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd39409f650 0x7fd39419fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:03.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.388+0000 7fd3937fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd39409e930 0x7fd39419f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36832/0 (socket says 172.21.15.3:36832) 2023-10-03T10:17:03.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.388+0000 7fd393fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd39409f650 0x7fd39419fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:49738/0 (socket says 172.21.15.3:49738) 2023-10-03T10:17:03.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.388+0000 7fd392ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd39409f240 0x7fd39419f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50182/0 (socket says 172.21.15.3:50182) 2023-10-03T10:17:03.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.388+0000 7fd3937fe700 1 -- 172.21.15.3:0/1029348042 learned_addr learned my addr 172.21.15.3:0/1029348042 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:03.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.388+0000 7fd3937fe700 1 -- 172.21.15.3:0/1029348042 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd39409f650 msgr2=0x7fd39419fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:03.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.388+0000 7fd3937fe700 1 --2- 172.21.15.3:0/1029348042 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd39409f650 0x7fd39419fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.388+0000 7fd3937fe700 1 -- 172.21.15.3:0/1029348042 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd39409f240 msgr2=0x7fd39419f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:03.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.388+0000 7fd3937fe700 1 --2- 172.21.15.3:0/1029348042 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd39409f240 0x7fd39419f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.388+0000 7fd3937fe700 1 -- 172.21.15.3:0/1029348042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3840094a0 con 0x7fd39409e930 2023-10-03T10:17:03.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.389+0000 7fd3937fe700 1 --2- 172.21.15.3:0/1029348042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd39409e930 0x7fd39419f3c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fd38400ebe0 tx=0x7fd38400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:03.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.389+0000 7fd38bfff700 1 -- 172.21.15.3:0/1029348042 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd384009780 con 0x7fd39409e930 2023-10-03T10:17:03.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.389+0000 7fd38bfff700 1 -- 172.21.15.3:0/1029348042 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38400c850 con 0x7fd39409e930 2023-10-03T10:17:03.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.389+0000 7fd38bfff700 1 -- 172.21.15.3:0/1029348042 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38400cbe0 con 0x7fd39409e930 2023-10-03T10:17:03.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.389+0000 7fd399a95700 1 -- 172.21.15.3:0/1029348042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd37c0094a0 con 0x7fd39409e930 2023-10-03T10:17:03.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.389+0000 7fd399a95700 1 -- 172.21.15.3:0/1029348042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38c009b30 con 0x7fd39409e930 2023-10-03T10:17:03.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.389+0000 7fd38bfff700 1 -- 172.21.15.3:0/1029348042 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd38400cd40 con 0x7fd39409e930 2023-10-03T10:17:03.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.390+0000 7fd38bfff700 1 -- 172.21.15.3:0/1029348042 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd38403f750 con 0x7fd39409e930 2023-10-03T10:17:03.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.390+0000 7fd399a95700 1 -- 172.21.15.3:0/1029348042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd394074ea0 con 0x7fd39409e930 2023-10-03T10:17:03.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.395+0000 7fd38bfff700 1 -- 172.21.15.3:0/1029348042 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd3840232f0 con 0x7fd39409e930 2023-10-03T10:17:03.447 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:03.523 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:03.561 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:03.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.590+0000 7f1a6ecd5700 1 Processor -- start 2023-10-03T10:17:03.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.591+0000 7f1a6ecd5700 1 -- start start 2023-10-03T10:17:03.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.591+0000 7f1a6ecd5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a6809d9f0 0x7f1a68136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:03.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.591+0000 7f1a6ecd5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1a68136a70 0x7f1a68138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:03.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.591+0000 7f1a6ecd5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a6809e130 con 0x7f1a681316c0 2023-10-03T10:17:03.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.591+0000 7f1a6ecd5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a681321a0 con 0x7f1a68136a70 2023-10-03T10:17:03.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.591+0000 7f1a6ecd5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a681312b0 con 0x7f1a6809d9f0 2023-10-03T10:17:03.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.592+0000 7f1a6d272700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1a68136a70 0x7f1a68138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:03.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.592+0000 7f1a6d272700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1a68136a70 0x7f1a68138e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36834/0 (socket says 172.21.15.3:36834) 2023-10-03T10:17:03.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.592+0000 7f1a6d272700 1 -- 172.21.15.3:0/2095237157 learned_addr learned my addr 172.21.15.3:0/2095237157 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:03.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.592+0000 7f1a67fff700 1 --2- 172.21.15.3:0/2095237157 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a6809d9f0 0x7f1a68136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:03.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.592+0000 7f1a67fff700 1 -- 172.21.15.3:0/2095237157 >> v2:172.21.15.3:3300/0 conn(0x7f1a68136a70 msgr2=0x7f1a68138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:03.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.592+0000 7f1a67fff700 1 --2- 172.21.15.3:0/2095237157 >> v2:172.21.15.3:3300/0 conn(0x7f1a68136a70 0x7f1a68138e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.592+0000 7f1a67fff700 1 -- 172.21.15.3:0/2095237157 >> v1:172.21.15.3:6789/0 conn(0x7f1a681316c0 legacy=0x7f1a68131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:03.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.592+0000 7f1a677fe700 1 -- 172.21.15.3:0/2095237157 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3192613388 0 0) 0x7f1a5c001110 con 0x7f1a681316c0 2023-10-03T10:17:03.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.592+0000 7f1a67fff700 1 -- 172.21.15.3:0/2095237157 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a6809f6e0 con 0x7f1a6809d9f0 2023-10-03T10:17:03.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.592+0000 7f1a67fff700 1 --2- 172.21.15.3:0/2095237157 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a6809d9f0 0x7f1a68136530 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1a54009840 tx=0x7f1a54009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ac52e5b23baa340 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:03.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.593+0000 7f1a677fe700 1 -- 172.21.15.3:0/2095237157 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a54011830 con 0x7f1a6809d9f0 2023-10-03T10:17:03.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.593+0000 7f1a677fe700 1 -- 172.21.15.3:0/2095237157 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a54011990 con 0x7f1a6809d9f0 2023-10-03T10:17:03.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.593+0000 7f1a677fe700 1 -- 172.21.15.3:0/2095237157 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a54011d20 con 0x7f1a6809d9f0 2023-10-03T10:17:03.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.594+0000 7f1a6ecd5700 1 -- 172.21.15.3:0/2095237157 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a6809d9f0 msgr2=0x7f1a68136530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:03.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.594+0000 7f1a6ecd5700 1 --2- 172.21.15.3:0/2095237157 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a6809d9f0 0x7f1a68136530 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1a54009840 tx=0x7f1a54009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:03.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.594+0000 7f1a6ecd5700 1 -- 172.21.15.3:0/2095237157 shutdown_connections 2023-10-03T10:17:03.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.594+0000 7f1a6ecd5700 1 --2- 172.21.15.3:0/2095237157 >> v2:172.21.15.3:3300/0 conn(0x7f1a68136a70 0x7f1a68138e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.594+0000 7f1a6ecd5700 1 --2- 172.21.15.3:0/2095237157 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a6809d9f0 0x7f1a68136530 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.594+0000 7f1a6ecd5700 1 -- 172.21.15.3:0/2095237157 >> 172.21.15.3:0/2095237157 conn(0x7f1a68066f50 msgr2=0x7f1a6809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:03.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.594+0000 7f1a6ecd5700 1 -- 172.21.15.3:0/2095237157 shutdown_connections 2023-10-03T10:17:03.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.594+0000 7f1a6ecd5700 1 -- 172.21.15.3:0/2095237157 wait complete. 2023-10-03T10:17:03.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.595+0000 7f1a6ecd5700 1 Processor -- start 2023-10-03T10:17:03.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.595+0000 7f1a6ecd5700 1 -- start start 2023-10-03T10:17:03.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.596+0000 7f1a6ecd5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a6809d9f0 0x7f1a68070f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:03.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.596+0000 7f1a6ecd5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a681316c0 0x7f1a68071480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:03.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.596+0000 7f1a6ecd5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a68136a70 0x7f1a680719c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:03.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.596+0000 7f1a6ecd5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6812b900 con 0x7f1a681316c0 2023-10-03T10:17:03.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.596+0000 7f1a6ecd5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6806fdc0 con 0x7f1a68136a70 2023-10-03T10:17:03.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.596+0000 7f1a6ecd5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a68135f00 con 0x7f1a6809d9f0 2023-10-03T10:17:03.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.597+0000 7f1a6d272700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a68136a70 0x7f1a680719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:03.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.597+0000 7f1a6d272700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a68136a70 0x7f1a680719c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50184/0 (socket says 172.21.15.3:50184) 2023-10-03T10:17:03.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.597+0000 7f1a6d272700 1 -- 172.21.15.3:0/3005352164 learned_addr learned my addr 172.21.15.3:0/3005352164 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:03.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.597+0000 7f1a6ca71700 1 --2- 172.21.15.3:0/3005352164 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a6809d9f0 0x7f1a68070f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:03.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.597+0000 7f1a67fff700 1 --2- 172.21.15.3:0/3005352164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a681316c0 0x7f1a68071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:03.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.597+0000 7f1a6d272700 1 -- 172.21.15.3:0/3005352164 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a6809d9f0 msgr2=0x7f1a68070f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:03.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.597+0000 7f1a6d272700 1 --2- 172.21.15.3:0/3005352164 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a6809d9f0 0x7f1a68070f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.597+0000 7f1a6d272700 1 -- 172.21.15.3:0/3005352164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a681316c0 msgr2=0x7f1a68071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:03.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.597+0000 7f1a6d272700 1 --2- 172.21.15.3:0/3005352164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a681316c0 0x7f1a68071480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.597+0000 7f1a6d272700 1 -- 172.21.15.3:0/3005352164 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a540094a0 con 0x7f1a68136a70 2023-10-03T10:17:03.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.597+0000 7f1a6d272700 1 --2- 172.21.15.3:0/3005352164 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a68136a70 0x7f1a680719c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000efc0 tx=0x7f1a6000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:03.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.598+0000 7f1a657fa700 1 -- 172.21.15.3:0/3005352164 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60003c20 con 0x7f1a68136a70 2023-10-03T10:17:03.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.598+0000 7f1a657fa700 1 -- 172.21.15.3:0/3005352164 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a6000f040 con 0x7f1a68136a70 2023-10-03T10:17:03.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.598+0000 7f1a6ecd5700 1 -- 172.21.15.3:0/3005352164 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a6806fab0 con 0x7f1a68136a70 2023-10-03T10:17:03.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.598+0000 7f1a6ecd5700 1 -- 172.21.15.3:0/3005352164 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a68071f00 con 0x7f1a68136a70 2023-10-03T10:17:03.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.598+0000 7f1a657fa700 1 -- 172.21.15.3:0/3005352164 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60009e30 con 0x7f1a68136a70 2023-10-03T10:17:03.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.599+0000 7f1a6ecd5700 1 -- 172.21.15.3:0/3005352164 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a480052e0 con 0x7f1a68136a70 2023-10-03T10:17:03.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.600+0000 7f1a657fa700 1 -- 172.21.15.3:0/3005352164 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1a6000c930 con 0x7f1a68136a70 2023-10-03T10:17:03.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.600+0000 7f1a657fa700 1 -- 172.21.15.3:0/3005352164 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1a600407f0 con 0x7f1a68136a70 2023-10-03T10:17:03.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.605+0000 7f1a657fa700 1 -- 172.21.15.3:0/3005352164 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1a60026bd0 con 0x7f1a68136a70 2023-10-03T10:17:03.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.608+0000 7fd399a95700 1 -- 172.21.15.3:0/1029348042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd39406b250 con 0x7fd39409e930 2023-10-03T10:17:03.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.608+0000 7fd38bfff700 1 -- 172.21.15.3:0/1029348042 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd384021020 con 0x7fd39409e930 2023-10-03T10:17:03.609 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:03.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.612+0000 7fd399a95700 1 -- 172.21.15.3:0/1029348042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd39409e930 msgr2=0x7fd39419f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:03.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.612+0000 7fd399a95700 1 --2- 172.21.15.3:0/1029348042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd39409e930 0x7fd39419f3c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fd38400ebe0 tx=0x7fd38400eef0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.612+0000 7fd399a95700 1 -- 172.21.15.3:0/1029348042 shutdown_connections 2023-10-03T10:17:03.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.612+0000 7fd399a95700 1 --2- 172.21.15.3:0/1029348042 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd39409f650 0x7fd39419fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.612+0000 7fd399a95700 1 --2- 172.21.15.3:0/1029348042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd39409e930 0x7fd39419f3c0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.612+0000 7fd399a95700 1 --2- 172.21.15.3:0/1029348042 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd39409f240 0x7fd39419f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.612+0000 7fd399a95700 1 -- 172.21.15.3:0/1029348042 >> 172.21.15.3:0/1029348042 conn(0x7fd3940a4210 msgr2=0x7fd39406c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:03.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.612+0000 7fd399a95700 1 -- 172.21.15.3:0/1029348042 shutdown_connections 2023-10-03T10:17:03.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.612+0000 7fd399a95700 1 -- 172.21.15.3:0/1029348042 wait complete. 2023-10-03T10:17:03.628 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:03.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.674+0000 7fe2a4794700 1 Processor -- start 2023-10-03T10:17:03.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.674+0000 7fe2a4794700 1 -- start start 2023-10-03T10:17:03.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.675+0000 7fe2a4794700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe29c0a0a30 0x7fe29c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:03.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.675+0000 7fe2a4794700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe29c09d630 0x7fe29c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:03.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.675+0000 7fe2a4794700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe29c0a2a70 con 0x7fe29c133930 2023-10-03T10:17:03.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.675+0000 7fe2a4794700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c0a01d0 con 0x7fe29c09d630 2023-10-03T10:17:03.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.675+0000 7fe2a4794700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c09e2b0 con 0x7fe29c0a0a30 2023-10-03T10:17:03.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.675+0000 7fe2a2530700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe29c0a0a30 0x7fe29c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:03.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.675+0000 7fe2a2530700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe29c0a0a30 0x7fe29c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:49764/0 (socket says 172.21.15.3:49764) 2023-10-03T10:17:03.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.675+0000 7fe2a2530700 1 -- 172.21.15.3:0/1988524169 learned_addr learned my addr 172.21.15.3:0/1988524169 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:03.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.675+0000 7fe2a2d31700 1 --1- 172.21.15.3:0/1988524169 >> v1:172.21.15.3:6789/0 conn(0x7fe29c133930 0x7fe29c133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45896/0 (socket says 172.21.15.3:45896) 2023-10-03T10:17:03.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.675+0000 7fe2a1d2f700 1 --2- 172.21.15.3:0/1988524169 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe29c09d630 0x7fe29c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:03.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.676+0000 7fe2a152e700 1 -- 172.21.15.3:0/1988524169 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1991742305 0 0) 0x7fe29c0a2a70 con 0x7fe29c133930 2023-10-03T10:17:03.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.676+0000 7fe2a152e700 1 -- 172.21.15.3:0/1988524169 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2880035b0 con 0x7fe29c133930 2023-10-03T10:17:03.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.676+0000 7fe2a2530700 1 -- 172.21.15.3:0/1988524169 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe29c09d630 msgr2=0x7fe29c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:03.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.676+0000 7fe2a2530700 1 --2- 172.21.15.3:0/1988524169 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe29c09d630 0x7fe29c0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.676+0000 7fe2a2530700 1 -- 172.21.15.3:0/1988524169 >> v1:172.21.15.3:6789/0 conn(0x7fe29c133930 legacy=0x7fe29c133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:03.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.676+0000 7fe2a2530700 1 -- 172.21.15.3:0/1988524169 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29c09dc50 con 0x7fe29c0a0a30 2023-10-03T10:17:03.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.676+0000 7fe2a1d2f700 1 --2- 172.21.15.3:0/1988524169 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe29c09d630 0x7fe29c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:03.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.676+0000 7fe2a2530700 1 --2- 172.21.15.3:0/1988524169 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe29c0a0a30 0x7fe29c0a0e00 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe298009e80 tx=0x7fe29800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f57b27d0fb14996 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:03.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.676+0000 7fe2a152e700 1 -- 172.21.15.3:0/1988524169 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29800e070 con 0x7fe29c0a0a30 2023-10-03T10:17:03.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.676+0000 7fe2a152e700 1 -- 172.21.15.3:0/1988524169 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29800b910 con 0x7fe29c0a0a30 2023-10-03T10:17:03.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.676+0000 7fe2a152e700 1 -- 172.21.15.3:0/1988524169 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29800bcc0 con 0x7fe29c0a0a30 2023-10-03T10:17:03.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.677+0000 7fe2a4794700 1 -- 172.21.15.3:0/1988524169 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe29c0a0a30 msgr2=0x7fe29c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:03.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.677+0000 7fe2a4794700 1 --2- 172.21.15.3:0/1988524169 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe29c0a0a30 0x7fe29c0a0e00 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe298009e80 tx=0x7fe29800b330 comp rx=0 tx=0).stop 2023-10-03T10:17:03.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.677+0000 7fe2a4794700 1 -- 172.21.15.3:0/1988524169 shutdown_connections 2023-10-03T10:17:03.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.677+0000 7fe2a4794700 1 --2- 172.21.15.3:0/1988524169 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe29c0a0a30 0x7fe29c0a0e00 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.677+0000 7fe2a4794700 1 --2- 172.21.15.3:0/1988524169 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe29c09d630 0x7fe29c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.677+0000 7fe2a4794700 1 -- 172.21.15.3:0/1988524169 >> 172.21.15.3:0/1988524169 conn(0x7fe29c0a3050 msgr2=0x7fe29c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:03.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.677+0000 7fe2a4794700 1 -- 172.21.15.3:0/1988524169 shutdown_connections 2023-10-03T10:17:03.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.677+0000 7fe2a4794700 1 -- 172.21.15.3:0/1988524169 wait complete. 2023-10-03T10:17:03.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.678+0000 7fe2a4794700 1 Processor -- start 2023-10-03T10:17:03.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.678+0000 7fe2a4794700 1 -- start start 2023-10-03T10:17:03.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.679+0000 7fe2a4794700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe29c09d630 0x7fe29c1c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:03.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.679+0000 7fe2a4794700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe29c0a0a30 0x7fe29c1c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:03.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.679+0000 7fe2a4794700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe29c133930 0x7fe29c1c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:03.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.679+0000 7fe2a4794700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c1208e0 con 0x7fe29c133930 2023-10-03T10:17:03.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.679+0000 7fe2a4794700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c1223c0 con 0x7fe29c09d630 2023-10-03T10:17:03.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.679+0000 7fe2a4794700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c1206b0 con 0x7fe29c0a0a30 2023-10-03T10:17:03.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.679+0000 7fe2a1d2f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe29c0a0a30 0x7fe29c1c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:03.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.679+0000 7fe2a2530700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe29c09d630 0x7fe29c1c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:03.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.679+0000 7fe2a1d2f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe29c0a0a30 0x7fe29c1c8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:49766/0 (socket says 172.21.15.3:49766) 2023-10-03T10:17:03.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.679+0000 7fe2a2530700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe29c09d630 0x7fe29c1c80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50204/0 (socket says 172.21.15.3:50204) 2023-10-03T10:17:03.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.679+0000 7fe2a1d2f700 1 -- 172.21.15.3:0/1881676233 learned_addr learned my addr 172.21.15.3:0/1881676233 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:03.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.679+0000 7fe2a2d31700 1 --2- 172.21.15.3:0/1881676233 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe29c133930 0x7fe29c1c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:03.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.679+0000 7fe2a1d2f700 1 -- 172.21.15.3:0/1881676233 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe29c09d630 msgr2=0x7fe29c1c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:03.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.679+0000 7fe2a1d2f700 1 --2- 172.21.15.3:0/1881676233 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe29c09d630 0x7fe29c1c80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.679+0000 7fe2a1d2f700 1 -- 172.21.15.3:0/1881676233 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe29c133930 msgr2=0x7fe29c1c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:03.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.679+0000 7fe2a1d2f700 1 --2- 172.21.15.3:0/1881676233 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe29c133930 0x7fe29c1c8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.679+0000 7fe2a1d2f700 1 -- 172.21.15.3:0/1881676233 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28c0094a0 con 0x7fe29c0a0a30 2023-10-03T10:17:03.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.680+0000 7fe2a1d2f700 1 --2- 172.21.15.3:0/1881676233 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe29c0a0a30 0x7fe29c1c8600 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe28c00ebe0 tx=0x7fe28c00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:03.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.680+0000 7fe292ffd700 1 -- 172.21.15.3:0/1881676233 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c00c850 con 0x7fe29c0a0a30 2023-10-03T10:17:03.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.680+0000 7fe292ffd700 1 -- 172.21.15.3:0/1881676233 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28c00c9b0 con 0x7fe29c0a0a30 2023-10-03T10:17:03.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.680+0000 7fe2a4794700 1 -- 172.21.15.3:0/1881676233 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe298009b30 con 0x7fe29c0a0a30 2023-10-03T10:17:03.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.680+0000 7fe292ffd700 1 -- 172.21.15.3:0/1881676233 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c00cd40 con 0x7fe29c0a0a30 2023-10-03T10:17:03.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.680+0000 7fe2a4794700 1 -- 172.21.15.3:0/1881676233 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29c120ea0 con 0x7fe29c0a0a30 2023-10-03T10:17:03.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.681+0000 7fe292ffd700 1 -- 172.21.15.3:0/1881676233 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe28c013470 con 0x7fe29c0a0a30 2023-10-03T10:17:03.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.681+0000 7fe2a4794700 1 -- 172.21.15.3:0/1881676233 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe29c133610 con 0x7fe29c0a0a30 2023-10-03T10:17:03.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.681+0000 7fe292ffd700 1 -- 172.21.15.3:0/1881676233 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe28c03f7c0 con 0x7fe29c0a0a30 2023-10-03T10:17:03.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.686+0000 7fe292ffd700 1 -- 172.21.15.3:0/1881676233 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe29c133610 con 0x7fe29c0a0a30 2023-10-03T10:17:03.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.710+0000 7fc347d2c700 1 Processor -- start 2023-10-03T10:17:03.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.711+0000 7fc347d2c700 1 -- start start 2023-10-03T10:17:03.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.712+0000 7fc347d2c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc340133420 0x7fc3401337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:03.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.712+0000 7fc347d2c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc340133e30 0x7fc34012ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:03.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.712+0000 7fc347d2c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc34006b990 con 0x7fc340134260 2023-10-03T10:17:03.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.712+0000 7fc347d2c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3400a0590 con 0x7fc340133420 2023-10-03T10:17:03.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.712+0000 7fc347d2c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc34009fb80 con 0x7fc340133e30 2023-10-03T10:17:03.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.712+0000 7fc345ac8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc340133420 0x7fc3401337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:03.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.712+0000 7fc345ac8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc340133420 0x7fc3401337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36864/0 (socket says 172.21.15.3:36864) 2023-10-03T10:17:03.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.712+0000 7fc345ac8700 1 -- 172.21.15.3:0/3446775804 learned_addr learned my addr 172.21.15.3:0/3446775804 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:03.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.712+0000 7fc3452c7700 1 --2- 172.21.15.3:0/3446775804 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc340133e30 0x7fc34012ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:03.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.712+0000 7fc345ac8700 1 -- 172.21.15.3:0/3446775804 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc340133e30 msgr2=0x7fc34012ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:03.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.712+0000 7fc345ac8700 1 --2- 172.21.15.3:0/3446775804 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc340133e30 0x7fc34012ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.712+0000 7fc345ac8700 1 -- 172.21.15.3:0/3446775804 >> v1:172.21.15.3:6789/0 conn(0x7fc340134260 legacy=0x7fc3401304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:03.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.712+0000 7fc345ac8700 1 -- 172.21.15.3:0/3446775804 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3400a0030 con 0x7fc340133420 2023-10-03T10:17:03.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.713+0000 7fc345ac8700 1 --2- 172.21.15.3:0/3446775804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc340133420 0x7fc3401337f0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fc33c0098f0 tx=0x7fc33c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6365e1bbde2711 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:03.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.713+0000 7fc344ac6700 1 -- 172.21.15.3:0/3446775804 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c011830 con 0x7fc340133420 2023-10-03T10:17:03.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.713+0000 7fc344ac6700 1 -- 172.21.15.3:0/3446775804 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33c011990 con 0x7fc340133420 2023-10-03T10:17:03.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.713+0000 7fc344ac6700 1 -- 172.21.15.3:0/3446775804 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c011d20 con 0x7fc340133420 2023-10-03T10:17:03.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.714+0000 7fc347d2c700 1 -- 172.21.15.3:0/3446775804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc340133420 msgr2=0x7fc3401337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:03.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.714+0000 7fc347d2c700 1 --2- 172.21.15.3:0/3446775804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc340133420 0x7fc3401337f0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fc33c0098f0 tx=0x7fc33c009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:03.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.714+0000 7fc347d2c700 1 -- 172.21.15.3:0/3446775804 shutdown_connections 2023-10-03T10:17:03.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.714+0000 7fc347d2c700 1 --2- 172.21.15.3:0/3446775804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc340133420 0x7fc3401337f0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.714+0000 7fc347d2c700 1 --2- 172.21.15.3:0/3446775804 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc340133e30 0x7fc34012ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.714+0000 7fc347d2c700 1 -- 172.21.15.3:0/3446775804 >> 172.21.15.3:0/3446775804 conn(0x7fc3400a4210 msgr2=0x7fc3400a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:03.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.714+0000 7fc347d2c700 1 -- 172.21.15.3:0/3446775804 shutdown_connections 2023-10-03T10:17:03.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.714+0000 7fc347d2c700 1 -- 172.21.15.3:0/3446775804 wait complete. 2023-10-03T10:17:03.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.715+0000 7fc347d2c700 1 Processor -- start 2023-10-03T10:17:03.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.715+0000 7fc347d2c700 1 -- start start 2023-10-03T10:17:03.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.716+0000 7fc347d2c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc340133420 0x7fc34009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:03.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.716+0000 7fc347d2c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc340133e30 0x7fc34009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:03.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.716+0000 7fc347d2c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc340134260 0x7fc34009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:03.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.716+0000 7fc347d2c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc340120710 con 0x7fc340134260 2023-10-03T10:17:03.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.716+0000 7fc347d2c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc34011fbb0 con 0x7fc340133e30 2023-10-03T10:17:03.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.716+0000 7fc347d2c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc34011f8e0 con 0x7fc340133420 2023-10-03T10:17:03.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.716+0000 7fc345ac8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc340133420 0x7fc34009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:03.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.716+0000 7fc3452c7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc340133e30 0x7fc34009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:03.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.716+0000 7fc345ac8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc340133420 0x7fc34009c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:49780/0 (socket says 172.21.15.3:49780) 2023-10-03T10:17:03.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.716+0000 7fc345ac8700 1 -- 172.21.15.3:0/1248514492 learned_addr learned my addr 172.21.15.3:0/1248514492 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:03.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.716+0000 7fc3462c9700 1 --2- 172.21.15.3:0/1248514492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc340134260 0x7fc34009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:03.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.717+0000 7fc345ac8700 1 -- 172.21.15.3:0/1248514492 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc340133e30 msgr2=0x7fc34009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:03.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.717+0000 7fc345ac8700 1 --2- 172.21.15.3:0/1248514492 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc340133e30 0x7fc34009c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.717+0000 7fc345ac8700 1 -- 172.21.15.3:0/1248514492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc340134260 msgr2=0x7fc34009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:03.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.717+0000 7fc345ac8700 1 --2- 172.21.15.3:0/1248514492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc340134260 0x7fc34009cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.717+0000 7fc345ac8700 1 -- 172.21.15.3:0/1248514492 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc33c009550 con 0x7fc340133420 2023-10-03T10:17:03.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.717+0000 7fc345ac8700 1 --2- 172.21.15.3:0/1248514492 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc340133420 0x7fc34009c480 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc33c000c00 tx=0x7fc33c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:03.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.717+0000 7fc3367fc700 1 -- 172.21.15.3:0/1248514492 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c0036a0 con 0x7fc340133420 2023-10-03T10:17:03.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.717+0000 7fc3367fc700 1 -- 172.21.15.3:0/1248514492 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33c003800 con 0x7fc340133420 2023-10-03T10:17:03.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.717+0000 7fc347d2c700 1 -- 172.21.15.3:0/1248514492 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc340120230 con 0x7fc340133420 2023-10-03T10:17:03.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.717+0000 7fc3367fc700 1 -- 172.21.15.3:0/1248514492 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c003bb0 con 0x7fc340133420 2023-10-03T10:17:03.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.717+0000 7fc347d2c700 1 -- 172.21.15.3:0/1248514492 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc34009dfc0 con 0x7fc340133420 2023-10-03T10:17:03.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.718+0000 7fc3367fc700 1 -- 172.21.15.3:0/1248514492 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc33c021070 con 0x7fc340133420 2023-10-03T10:17:03.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.718+0000 7fc3367fc700 1 -- 172.21.15.3:0/1248514492 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc33c040ab0 con 0x7fc340133420 2023-10-03T10:17:03.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.718+0000 7fc347d2c700 1 -- 172.21.15.3:0/1248514492 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc340074ea0 con 0x7fc340133420 2023-10-03T10:17:03.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.723+0000 7fc3367fc700 1 -- 172.21.15.3:0/1248514492 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc33c015020 con 0x7fc340133420 2023-10-03T10:17:03.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.818+0000 7f1a6ecd5700 1 -- 172.21.15.3:0/3005352164 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1a48005150 con 0x7f1a68136a70 2023-10-03T10:17:03.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.818+0000 7f1a657fa700 1 -- 172.21.15.3:0/3005352164 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1a6002d090 con 0x7f1a68136a70 2023-10-03T10:17:03.819 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:03.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.821+0000 7f1a6ecd5700 1 -- 172.21.15.3:0/3005352164 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a68136a70 msgr2=0x7f1a680719c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:03.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.821+0000 7f1a6ecd5700 1 --2- 172.21.15.3:0/3005352164 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a68136a70 0x7f1a680719c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000efc0 tx=0x7f1a6000c3a0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.821+0000 7f1a6ecd5700 1 -- 172.21.15.3:0/3005352164 shutdown_connections 2023-10-03T10:17:03.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.821+0000 7f1a6ecd5700 1 --2- 172.21.15.3:0/3005352164 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a68136a70 0x7f1a680719c0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.821+0000 7f1a6ecd5700 1 --2- 172.21.15.3:0/3005352164 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a6809d9f0 0x7f1a68070f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.821+0000 7f1a6ecd5700 1 --2- 172.21.15.3:0/3005352164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a681316c0 0x7f1a68071480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.821+0000 7f1a6ecd5700 1 -- 172.21.15.3:0/3005352164 >> 172.21.15.3:0/3005352164 conn(0x7f1a68066f50 msgr2=0x7f1a68136e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:03.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.821+0000 7f1a6ecd5700 1 -- 172.21.15.3:0/3005352164 shutdown_connections 2023-10-03T10:17:03.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.821+0000 7f1a6ecd5700 1 -- 172.21.15.3:0/3005352164 wait complete. 2023-10-03T10:17:03.835 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.898+0000 7fe2a4794700 1 -- 172.21.15.3:0/1881676233 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe29c133610 con 0x7fe29c0a0a30 2023-10-03T10:17:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.898+0000 7fe292ffd700 1 -- 172.21.15.3:0/1881676233 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe29c133610 con 0x7fe29c0a0a30 2023-10-03T10:17:03.899 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:03.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.901+0000 7fe2a4794700 1 -- 172.21.15.3:0/1881676233 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe29c0a0a30 msgr2=0x7fe29c1c8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:03.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.901+0000 7fe2a4794700 1 --2- 172.21.15.3:0/1881676233 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe29c0a0a30 0x7fe29c1c8600 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe28c00ebe0 tx=0x7fe28c00eef0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.901+0000 7fe2a4794700 1 -- 172.21.15.3:0/1881676233 shutdown_connections 2023-10-03T10:17:03.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.901+0000 7fe2a4794700 1 --2- 172.21.15.3:0/1881676233 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe29c133930 0x7fe29c1c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.901+0000 7fe2a4794700 1 --2- 172.21.15.3:0/1881676233 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe29c09d630 0x7fe29c1c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.901+0000 7fe2a4794700 1 --2- 172.21.15.3:0/1881676233 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe29c0a0a30 0x7fe29c1c8600 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.901+0000 7fe2a4794700 1 -- 172.21.15.3:0/1881676233 >> 172.21.15.3:0/1881676233 conn(0x7fe29c0a3050 msgr2=0x7fe29c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:03.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.901+0000 7fe2a4794700 1 -- 172.21.15.3:0/1881676233 shutdown_connections 2023-10-03T10:17:03.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.901+0000 7fe2a4794700 1 -- 172.21.15.3:0/1881676233 wait complete. 2023-10-03T10:17:03.915 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:03.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.930+0000 7fc347d2c700 1 -- 172.21.15.3:0/1248514492 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc34006b250 con 0x7fc340133420 2023-10-03T10:17:03.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.930+0000 7fc3367fc700 1 -- 172.21.15.3:0/1248514492 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc33c015020 con 0x7fc340133420 2023-10-03T10:17:03.931 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:03.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.932+0000 7fc347d2c700 1 -- 172.21.15.3:0/1248514492 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc340133420 msgr2=0x7fc34009c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:03.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.932+0000 7fc347d2c700 1 --2- 172.21.15.3:0/1248514492 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc340133420 0x7fc34009c480 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc33c000c00 tx=0x7fc33c004750 comp rx=0 tx=0).stop 2023-10-03T10:17:03.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.932+0000 7fc347d2c700 1 -- 172.21.15.3:0/1248514492 shutdown_connections 2023-10-03T10:17:03.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.932+0000 7fc347d2c700 1 --2- 172.21.15.3:0/1248514492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc340134260 0x7fc34009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.932+0000 7fc347d2c700 1 --2- 172.21.15.3:0/1248514492 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc340133420 0x7fc34009c480 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.932+0000 7fc347d2c700 1 --2- 172.21.15.3:0/1248514492 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc340133e30 0x7fc34009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:03.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.932+0000 7fc347d2c700 1 -- 172.21.15.3:0/1248514492 >> 172.21.15.3:0/1248514492 conn(0x7fc3400a4210 msgr2=0x7fc34012de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:03.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.933+0000 7fc347d2c700 1 -- 172.21.15.3:0/1248514492 shutdown_connections 2023-10-03T10:17:03.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:03.933+0000 7fc347d2c700 1 -- 172.21.15.3:0/1248514492 wait complete. 2023-10-03T10:17:03.946 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:04.628 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:04.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.775+0000 7ff286bc0700 1 Processor -- start 2023-10-03T10:17:04.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.775+0000 7ff286bc0700 1 -- start start 2023-10-03T10:17:04.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.776+0000 7ff286bc0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff28009f410 0x7ff280136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:04.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.776+0000 7ff286bc0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff280136fe0 0x7ff2801393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:04.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.776+0000 7ff286bc0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff280070940 con 0x7ff28009e930 2023-10-03T10:17:04.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.776+0000 7ff286bc0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff28006fd60 con 0x7ff28009f410 2023-10-03T10:17:04.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.776+0000 7ff286bc0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28006fa80 con 0x7ff280136fe0 2023-10-03T10:17:04.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.776+0000 7ff28495c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff28009e930 0x7ff28009ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45914/0 (socket says 172.21.15.3:45914) 2023-10-03T10:17:04.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.776+0000 7ff28515d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff280136fe0 0x7ff2801393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:04.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.776+0000 7ff28495c700 1 -- 172.21.15.3:0/1578901876 learned_addr learned my addr 172.21.15.3:0/1578901876 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:04.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.776+0000 7ff27ffff700 1 --2- 172.21.15.3:0/1578901876 >> v2:172.21.15.3:3300/0 conn(0x7ff28009f410 0x7ff280136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:04.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.777+0000 7ff27ffff700 1 -- 172.21.15.3:0/1578901876 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff280136fe0 msgr2=0x7ff2801393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:04.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.777+0000 7ff27ffff700 1 --2- 172.21.15.3:0/1578901876 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff280136fe0 0x7ff2801393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:04.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.777+0000 7ff27ffff700 1 -- 172.21.15.3:0/1578901876 >> v1:172.21.15.3:6789/0 conn(0x7ff28009e930 legacy=0x7ff28009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:04.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.777+0000 7ff27ffff700 1 -- 172.21.15.3:0/1578901876 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28009fb60 con 0x7ff28009f410 2023-10-03T10:17:04.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.777+0000 7ff27f7fe700 1 -- 172.21.15.3:0/1578901876 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1817407212 0 0) 0x7ff280070940 con 0x7ff28009e930 2023-10-03T10:17:04.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.777+0000 7ff28515d700 1 --2- 172.21.15.3:0/1578901876 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff280136fe0 0x7ff2801393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:04.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.777+0000 7ff27ffff700 1 --2- 172.21.15.3:0/1578901876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff28009f410 0x7ff280136aa0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7ff268009840 tx=0x7ff268009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3246e194cded2af1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:04.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.777+0000 7ff27f7fe700 1 -- 172.21.15.3:0/1578901876 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff268011830 con 0x7ff28009f410 2023-10-03T10:17:04.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.777+0000 7ff27f7fe700 1 -- 172.21.15.3:0/1578901876 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff268011990 con 0x7ff28009f410 2023-10-03T10:17:04.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.777+0000 7ff27f7fe700 1 -- 172.21.15.3:0/1578901876 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff268011d20 con 0x7ff28009f410 2023-10-03T10:17:04.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.778+0000 7ff286bc0700 1 -- 172.21.15.3:0/1578901876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff28009f410 msgr2=0x7ff280136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:04.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.778+0000 7ff286bc0700 1 --2- 172.21.15.3:0/1578901876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff28009f410 0x7ff280136aa0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7ff268009840 tx=0x7ff268009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:04.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.778+0000 7ff286bc0700 1 -- 172.21.15.3:0/1578901876 shutdown_connections 2023-10-03T10:17:04.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.778+0000 7ff286bc0700 1 --2- 172.21.15.3:0/1578901876 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff280136fe0 0x7ff2801393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:04.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.778+0000 7ff286bc0700 1 --2- 172.21.15.3:0/1578901876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff28009f410 0x7ff280136aa0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:04.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.778+0000 7ff286bc0700 1 -- 172.21.15.3:0/1578901876 >> 172.21.15.3:0/1578901876 conn(0x7ff2800a4210 msgr2=0x7ff2800a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:04.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.778+0000 7ff286bc0700 1 -- 172.21.15.3:0/1578901876 shutdown_connections 2023-10-03T10:17:04.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.778+0000 7ff286bc0700 1 -- 172.21.15.3:0/1578901876 wait complete. 2023-10-03T10:17:04.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.779+0000 7ff286bc0700 1 Processor -- start 2023-10-03T10:17:04.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.779+0000 7ff286bc0700 1 -- start start 2023-10-03T10:17:04.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.779+0000 7ff286bc0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff28009e930 0x7ff280122a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:04.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.779+0000 7ff286bc0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff28009f410 0x7ff280122fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:04.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.779+0000 7ff286bc0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff280136fe0 0x7ff280123510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:04.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.779+0000 7ff286bc0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff280071d90 con 0x7ff28009e930 2023-10-03T10:17:04.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.779+0000 7ff286bc0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2800a31b0 con 0x7ff280136fe0 2023-10-03T10:17:04.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.779+0000 7ff286bc0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff280124f00 con 0x7ff28009f410 2023-10-03T10:17:04.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.779+0000 7ff28495c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff28009e930 0x7ff280122a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:04.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.780+0000 7ff28515d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff280136fe0 0x7ff280123510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:04.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.780+0000 7ff28495c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff28009e930 0x7ff280122a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36898/0 (socket says 172.21.15.3:36898) 2023-10-03T10:17:04.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.780+0000 7ff28495c700 1 -- 172.21.15.3:0/2519576384 learned_addr learned my addr 172.21.15.3:0/2519576384 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:04.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.780+0000 7ff27ffff700 1 --2- 172.21.15.3:0/2519576384 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff28009f410 0x7ff280122fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:04.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.780+0000 7ff28495c700 1 -- 172.21.15.3:0/2519576384 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff28009f410 msgr2=0x7ff280122fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:04.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.780+0000 7ff28495c700 1 --2- 172.21.15.3:0/2519576384 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff28009f410 0x7ff280122fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:04.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.780+0000 7ff28495c700 1 -- 172.21.15.3:0/2519576384 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff280136fe0 msgr2=0x7ff280123510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:04.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.780+0000 7ff28495c700 1 --2- 172.21.15.3:0/2519576384 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff280136fe0 0x7ff280123510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:04.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.780+0000 7ff28495c700 1 -- 172.21.15.3:0/2519576384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff278009a70 con 0x7ff28009e930 2023-10-03T10:17:04.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.780+0000 7ff28515d700 1 --2- 172.21.15.3:0/2519576384 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff280136fe0 0x7ff280123510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:04.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.780+0000 7ff27ffff700 1 --2- 172.21.15.3:0/2519576384 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff28009f410 0x7ff280122fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:04.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.780+0000 7ff28495c700 1 --2- 172.21.15.3:0/2519576384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff28009e930 0x7ff280122a90 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7ff27000c520 tx=0x7ff27000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:04.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.780+0000 7ff27d7fa700 1 -- 172.21.15.3:0/2519576384 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff270004c80 con 0x7ff28009e930 2023-10-03T10:17:04.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.780+0000 7ff27d7fa700 1 -- 172.21.15.3:0/2519576384 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff270004e00 con 0x7ff28009e930 2023-10-03T10:17:04.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.780+0000 7ff286bc0700 1 -- 172.21.15.3:0/2519576384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2680094a0 con 0x7ff28009e930 2023-10-03T10:17:04.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.780+0000 7ff27d7fa700 1 -- 172.21.15.3:0/2519576384 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff270007310 con 0x7ff28009e930 2023-10-03T10:17:04.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.780+0000 7ff286bc0700 1 -- 172.21.15.3:0/2519576384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff280124c10 con 0x7ff28009e930 2023-10-03T10:17:04.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.781+0000 7ff27d7fa700 1 -- 172.21.15.3:0/2519576384 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff270007470 con 0x7ff28009e930 2023-10-03T10:17:04.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.781+0000 7ff27d7fa700 1 -- 172.21.15.3:0/2519576384 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff27003f340 con 0x7ff28009e930 2023-10-03T10:17:04.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.781+0000 7ff286bc0700 1 -- 172.21.15.3:0/2519576384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff280074ea0 con 0x7ff28009e930 2023-10-03T10:17:04.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.786+0000 7ff27d7fa700 1 -- 172.21.15.3:0/2519576384 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff2700123b0 con 0x7ff28009e930 2023-10-03T10:17:04.836 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:04.916 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:04.947 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:04.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.975+0000 7efd0d650700 1 Processor -- start 2023-10-03T10:17:04.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.976+0000 7efd0d650700 1 -- start start 2023-10-03T10:17:04.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.976+0000 7efd0d650700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efd08136710 0x7efd08138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:04.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.976+0000 7efd0d650700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd08139030 0x7efd0813b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:04.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.976+0000 7efd0d650700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd0806ef30 con 0x7efd081338a0 2023-10-03T10:17:04.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.976+0000 7efd0d650700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd0806fef0 con 0x7efd08136710 2023-10-03T10:17:04.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.977+0000 7efd0d650700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efd081334b0 con 0x7efd08139030 2023-10-03T10:17:04.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.977+0000 7efd077fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd08139030 0x7efd0813b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:04.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.977+0000 7efd067fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efd08136710 0x7efd08138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:04.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.977+0000 7efd077fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd08139030 0x7efd0813b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:49792/0 (socket says 172.21.15.3:49792) 2023-10-03T10:17:04.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.977+0000 7efd077fe700 1 -- 172.21.15.3:0/4217374709 learned_addr learned my addr 172.21.15.3:0/4217374709 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:04.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.977+0000 7efd067fc700 1 -- 172.21.15.3:0/4217374709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd08139030 msgr2=0x7efd0813b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:04.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.977+0000 7efd067fc700 1 --2- 172.21.15.3:0/4217374709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd08139030 0x7efd0813b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:04.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.977+0000 7efd067fc700 1 -- 172.21.15.3:0/4217374709 >> v1:172.21.15.3:6789/0 conn(0x7efd081338a0 legacy=0x7efd08133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:04.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.977+0000 7efd067fc700 1 -- 172.21.15.3:0/4217374709 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd0806fc30 con 0x7efd08136710 2023-10-03T10:17:04.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.977+0000 7efd077fe700 1 --2- 172.21.15.3:0/4217374709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd08139030 0x7efd0813b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:04.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.977+0000 7efd067fc700 1 --2- 172.21.15.3:0/4217374709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd08136710 0x7efd08138af0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7efcfc009840 tx=0x7efcfc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5106b669db735a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:04.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.978+0000 7efd05ffb700 1 -- 172.21.15.3:0/4217374709 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc011830 con 0x7efd08136710 2023-10-03T10:17:04.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.978+0000 7efd05ffb700 1 -- 172.21.15.3:0/4217374709 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcfc011990 con 0x7efd08136710 2023-10-03T10:17:04.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.978+0000 7efd05ffb700 1 -- 172.21.15.3:0/4217374709 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc011d20 con 0x7efd08136710 2023-10-03T10:17:04.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.979+0000 7efd0d650700 1 -- 172.21.15.3:0/4217374709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd08136710 msgr2=0x7efd08138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:04.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.979+0000 7efd0d650700 1 --2- 172.21.15.3:0/4217374709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd08136710 0x7efd08138af0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7efcfc009840 tx=0x7efcfc009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:04.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.979+0000 7efd0d650700 1 -- 172.21.15.3:0/4217374709 shutdown_connections 2023-10-03T10:17:04.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.979+0000 7efd0d650700 1 --2- 172.21.15.3:0/4217374709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd08139030 0x7efd0813b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:04.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.979+0000 7efd0d650700 1 --2- 172.21.15.3:0/4217374709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd08136710 0x7efd08138af0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:04.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.979+0000 7efd0d650700 1 -- 172.21.15.3:0/4217374709 >> 172.21.15.3:0/4217374709 conn(0x7efd080a2aa0 msgr2=0x7efd080a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:04.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.979+0000 7efd0d650700 1 -- 172.21.15.3:0/4217374709 shutdown_connections 2023-10-03T10:17:04.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.979+0000 7efd0d650700 1 -- 172.21.15.3:0/4217374709 wait complete. 2023-10-03T10:17:04.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.981+0000 7efd0d650700 1 Processor -- start 2023-10-03T10:17:04.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.981+0000 7efd0d650700 1 -- start start 2023-10-03T10:17:04.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.982+0000 7efd0d650700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd081338a0 0x7efd081214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:04.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.982+0000 7efd0d650700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd08136710 0x7efd0809e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:04.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.982+0000 7efd0d650700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd08139030 0x7efd0809ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:04.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.982+0000 7efd0d650700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd08122c30 con 0x7efd08139030 2023-10-03T10:17:04.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.982+0000 7efd0d650700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd08130d50 con 0x7efd081338a0 2023-10-03T10:17:04.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.982+0000 7efd0d650700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efd08123150 con 0x7efd08136710 2023-10-03T10:17:04.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.982+0000 7efd06ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd081338a0 0x7efd081214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:04.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.982+0000 7efd06ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd081338a0 0x7efd081214e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50242/0 (socket says 172.21.15.3:50242) 2023-10-03T10:17:04.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.982+0000 7efd06ffd700 1 -- 172.21.15.3:0/1645588644 learned_addr learned my addr 172.21.15.3:0/1645588644 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:04.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.982+0000 7efd067fc700 1 --2- 172.21.15.3:0/1645588644 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd08136710 0x7efd0809e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:04.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.982+0000 7efd077fe700 1 --2- 172.21.15.3:0/1645588644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd08139030 0x7efd0809ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:04.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.982+0000 7efd067fc700 1 -- 172.21.15.3:0/1645588644 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd081338a0 msgr2=0x7efd081214e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:04.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.982+0000 7efd067fc700 1 --2- 172.21.15.3:0/1645588644 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd081338a0 0x7efd081214e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:04.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.982+0000 7efd067fc700 1 -- 172.21.15.3:0/1645588644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd08139030 msgr2=0x7efd0809ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:04.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.982+0000 7efd067fc700 1 --2- 172.21.15.3:0/1645588644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd08139030 0x7efd0809ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:04.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.982+0000 7efd067fc700 1 -- 172.21.15.3:0/1645588644 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd00009b30 con 0x7efd08136710 2023-10-03T10:17:04.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.982+0000 7efd077fe700 1 --2- 172.21.15.3:0/1645588644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd08139030 0x7efd0809ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:04.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.983+0000 7efd067fc700 1 --2- 172.21.15.3:0/1645588644 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd08136710 0x7efd0809e690 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7efcfc016000 tx=0x7efcfc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:04.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.983+0000 7efcef7fe700 1 -- 172.21.15.3:0/1645588644 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc0036a0 con 0x7efd08136710 2023-10-03T10:17:04.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.983+0000 7efcef7fe700 1 -- 172.21.15.3:0/1645588644 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcfc01f070 con 0x7efd08136710 2023-10-03T10:17:04.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.983+0000 7efcef7fe700 1 -- 172.21.15.3:0/1645588644 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc003a60 con 0x7efd08136710 2023-10-03T10:17:04.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.984+0000 7efd0d650700 1 -- 172.21.15.3:0/1645588644 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcfc0094a0 con 0x7efd08136710 2023-10-03T10:17:04.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.984+0000 7efd0d650700 1 -- 172.21.15.3:0/1645588644 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd08122700 con 0x7efd08136710 2023-10-03T10:17:04.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.985+0000 7efcef7fe700 1 -- 172.21.15.3:0/1645588644 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efcfc003bc0 con 0x7efd08136710 2023-10-03T10:17:04.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.986+0000 7efd0d650700 1 -- 172.21.15.3:0/1645588644 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd0804ea50 con 0x7efd08136710 2023-10-03T10:17:04.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.986+0000 7efcef7fe700 1 -- 172.21.15.3:0/1645588644 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7efcfc021070 con 0x7efd08136710 2023-10-03T10:17:04.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.993+0000 7efcef7fe700 1 -- 172.21.15.3:0/1645588644 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efcfc02f3a0 con 0x7efd08136710 2023-10-03T10:17:04.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.997+0000 7ff286bc0700 1 -- 172.21.15.3:0/2519576384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff280125f70 con 0x7ff28009e930 2023-10-03T10:17:04.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:04.997+0000 7ff27d7fa700 1 -- 172.21.15.3:0/2519576384 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff27000ce60 con 0x7ff28009e930 2023-10-03T10:17:04.998 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:05.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.001+0000 7ff286bc0700 1 -- 172.21.15.3:0/2519576384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff28009e930 msgr2=0x7ff280122a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:05.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.001+0000 7ff286bc0700 1 --2- 172.21.15.3:0/2519576384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff28009e930 0x7ff280122a90 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7ff27000c520 tx=0x7ff27000c830 comp rx=0 tx=0).stop 2023-10-03T10:17:05.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.001+0000 7ff286bc0700 1 -- 172.21.15.3:0/2519576384 shutdown_connections 2023-10-03T10:17:05.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.001+0000 7ff286bc0700 1 --2- 172.21.15.3:0/2519576384 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff280136fe0 0x7ff280123510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:05.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.001+0000 7ff286bc0700 1 --2- 172.21.15.3:0/2519576384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff28009e930 0x7ff280122a90 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:05.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.001+0000 7ff286bc0700 1 --2- 172.21.15.3:0/2519576384 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff28009f410 0x7ff280122fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:05.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.001+0000 7ff286bc0700 1 -- 172.21.15.3:0/2519576384 >> 172.21.15.3:0/2519576384 conn(0x7ff2800a4210 msgr2=0x7ff28006bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:05.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.001+0000 7ff286bc0700 1 -- 172.21.15.3:0/2519576384 shutdown_connections 2023-10-03T10:17:05.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.001+0000 7ff286bc0700 1 -- 172.21.15.3:0/2519576384 wait complete. 2023-10-03T10:17:05.016 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:05.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.061+0000 7f3fa9645700 1 Processor -- start 2023-10-03T10:17:05.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.062+0000 7f3fa9645700 1 -- start start 2023-10-03T10:17:05.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.062+0000 7f3fa9645700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3fa40a09c0 0x7f3fa40a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:05.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.062+0000 7f3fa9645700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3fa40a12d0 0x7f3fa4134030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:05.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.062+0000 7f3fa9645700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fa409ea10 con 0x7f3fa409d5c0 2023-10-03T10:17:05.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.062+0000 7f3fa9645700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa409e210 con 0x7f3fa40a12d0 2023-10-03T10:17:05.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.062+0000 7f3fa9645700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa40a26f0 con 0x7f3fa40a09c0 2023-10-03T10:17:05.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.063+0000 7f3fa2ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3fa40a09c0 0x7f3fa40a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:05.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.063+0000 7f3fa27fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3fa409d5c0 0x7f3fa4133920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45942/0 (socket says 172.21.15.3:45942) 2023-10-03T10:17:05.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.063+0000 7f3fa27fc700 1 -- 172.21.15.3:0/4140426570 learned_addr learned my addr 172.21.15.3:0/4140426570 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:05.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.063+0000 7f3fa37fe700 1 --2- 172.21.15.3:0/4140426570 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3fa40a12d0 0x7f3fa4134030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:05.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.063+0000 7f3fa1ffb700 1 -- 172.21.15.3:0/4140426570 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 893325708 0 0) 0x7f3fa409ea10 con 0x7f3fa409d5c0 2023-10-03T10:17:05.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.063+0000 7f3fa1ffb700 1 -- 172.21.15.3:0/4140426570 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f8c0035b0 con 0x7f3fa409d5c0 2023-10-03T10:17:05.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.063+0000 7f3fa2ffd700 1 -- 172.21.15.3:0/4140426570 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3fa40a12d0 msgr2=0x7f3fa4134030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:05.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.063+0000 7f3fa2ffd700 1 --2- 172.21.15.3:0/4140426570 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3fa40a12d0 0x7f3fa4134030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:05.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.063+0000 7f3fa2ffd700 1 -- 172.21.15.3:0/4140426570 >> v1:172.21.15.3:6789/0 conn(0x7f3fa409d5c0 legacy=0x7f3fa4133920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:05.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.063+0000 7f3fa2ffd700 1 -- 172.21.15.3:0/4140426570 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa409db70 con 0x7f3fa40a09c0 2023-10-03T10:17:05.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.063+0000 7f3fa37fe700 1 --2- 172.21.15.3:0/4140426570 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3fa40a12d0 0x7f3fa4134030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:05.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.063+0000 7f3fa2ffd700 1 --2- 172.21.15.3:0/4140426570 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3fa40a09c0 0x7f3fa40a0d90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009dc0 tx=0x7f3f9400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53f9f16b2ccae6aa server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:05.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.064+0000 7f3fa1ffb700 1 -- 172.21.15.3:0/4140426570 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9400e070 con 0x7f3fa40a09c0 2023-10-03T10:17:05.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.064+0000 7f3fa1ffb700 1 -- 172.21.15.3:0/4140426570 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f9400bb20 con 0x7f3fa40a09c0 2023-10-03T10:17:05.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.064+0000 7f3fa1ffb700 1 -- 172.21.15.3:0/4140426570 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9400be90 con 0x7f3fa40a09c0 2023-10-03T10:17:05.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.064+0000 7f3fa9645700 1 -- 172.21.15.3:0/4140426570 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3fa40a09c0 msgr2=0x7f3fa40a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:05.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.064+0000 7f3fa9645700 1 --2- 172.21.15.3:0/4140426570 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3fa40a09c0 0x7f3fa40a0d90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009dc0 tx=0x7f3f9400b330 comp rx=0 tx=0).stop 2023-10-03T10:17:05.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.064+0000 7f3fa9645700 1 -- 172.21.15.3:0/4140426570 shutdown_connections 2023-10-03T10:17:05.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.064+0000 7f3fa9645700 1 --2- 172.21.15.3:0/4140426570 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3fa40a12d0 0x7f3fa4134030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:05.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.064+0000 7f3fa9645700 1 --2- 172.21.15.3:0/4140426570 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3fa40a09c0 0x7f3fa40a0d90 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:05.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.064+0000 7f3fa9645700 1 -- 172.21.15.3:0/4140426570 >> 172.21.15.3:0/4140426570 conn(0x7f3fa40a2fe0 msgr2=0x7f3fa40a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:05.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.064+0000 7f3fa9645700 1 -- 172.21.15.3:0/4140426570 shutdown_connections 2023-10-03T10:17:05.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.064+0000 7f3fa9645700 1 -- 172.21.15.3:0/4140426570 wait complete. 2023-10-03T10:17:05.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.066+0000 7f3fa9645700 1 Processor -- start 2023-10-03T10:17:05.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.066+0000 7f3fa9645700 1 -- start start 2023-10-03T10:17:05.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.066+0000 7f3fa9645700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3fa409d5c0 0x7f3fa4120410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:05.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.066+0000 7f3fa9645700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3fa40a09c0 0x7f3fa4120950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:05.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.066+0000 7f3fa9645700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fa40a12d0 0x7f3fa41c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:05.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.066+0000 7f3fa9645700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa4121620 con 0x7f3fa40a12d0 2023-10-03T10:17:05.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.066+0000 7f3fa9645700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa4122f90 con 0x7f3fa409d5c0 2023-10-03T10:17:05.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.066+0000 7f3fa9645700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa409c3d0 con 0x7f3fa40a09c0 2023-10-03T10:17:05.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.066+0000 7f3fa27fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3fa40a09c0 0x7f3fa4120950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:05.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.066+0000 7f3fa27fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3fa40a09c0 0x7f3fa4120950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:49832/0 (socket says 172.21.15.3:49832) 2023-10-03T10:17:05.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.066+0000 7f3fa2ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3fa409d5c0 0x7f3fa4120410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:05.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.066+0000 7f3fa27fc700 1 -- 172.21.15.3:0/2443981901 learned_addr learned my addr 172.21.15.3:0/2443981901 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:05.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.066+0000 7f3fa37fe700 1 --2- 172.21.15.3:0/2443981901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fa40a12d0 0x7f3fa41c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:05.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.067+0000 7f3fa2ffd700 1 -- 172.21.15.3:0/2443981901 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3fa40a09c0 msgr2=0x7f3fa4120950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:05.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.067+0000 7f3fa2ffd700 1 --2- 172.21.15.3:0/2443981901 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3fa40a09c0 0x7f3fa4120950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:05.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.067+0000 7f3fa2ffd700 1 -- 172.21.15.3:0/2443981901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fa40a12d0 msgr2=0x7f3fa41c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:05.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.067+0000 7f3fa2ffd700 1 --2- 172.21.15.3:0/2443981901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fa40a12d0 0x7f3fa41c7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:05.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.067+0000 7f3fa2ffd700 1 -- 172.21.15.3:0/2443981901 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f9c0094a0 con 0x7f3fa409d5c0 2023-10-03T10:17:05.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.067+0000 7f3fa37fe700 1 --2- 172.21.15.3:0/2443981901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fa40a12d0 0x7f3fa41c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:05.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.067+0000 7f3fa27fc700 1 --2- 172.21.15.3:0/2443981901 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3fa40a09c0 0x7f3fa4120950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:05.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.067+0000 7f3fa2ffd700 1 --2- 172.21.15.3:0/2443981901 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3fa409d5c0 0x7f3fa4120410 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3f94016000 tx=0x7f3f94004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:05.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.067+0000 7f3f8b7fe700 1 -- 172.21.15.3:0/2443981901 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f94004930 con 0x7f3fa409d5c0 2023-10-03T10:17:05.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.067+0000 7f3f8b7fe700 1 -- 172.21.15.3:0/2443981901 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f94020070 con 0x7f3fa409d5c0 2023-10-03T10:17:05.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.067+0000 7f3fa9645700 1 -- 172.21.15.3:0/2443981901 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f94009a70 con 0x7f3fa409d5c0 2023-10-03T10:17:05.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.067+0000 7f3f8b7fe700 1 -- 172.21.15.3:0/2443981901 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f940276c0 con 0x7f3fa409d5c0 2023-10-03T10:17:05.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.067+0000 7f3fa9645700 1 -- 172.21.15.3:0/2443981901 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa4120e90 con 0x7f3fa409d5c0 2023-10-03T10:17:05.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.068+0000 7f3f8b7fe700 1 -- 172.21.15.3:0/2443981901 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3f94012430 con 0x7f3fa409d5c0 2023-10-03T10:17:05.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.068+0000 7f3fa9645700 1 -- 172.21.15.3:0/2443981901 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa41231b0 con 0x7f3fa409d5c0 2023-10-03T10:17:05.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.068+0000 7f3f8b7fe700 1 -- 172.21.15.3:0/2443981901 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3f94022070 con 0x7f3fa409d5c0 2023-10-03T10:17:05.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.073+0000 7f3f8b7fe700 1 -- 172.21.15.3:0/2443981901 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3f940126e0 con 0x7f3fa409d5c0 2023-10-03T10:17:05.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.154+0000 7f549595c700 1 Processor -- start 2023-10-03T10:17:05.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.154+0000 7f549595c700 1 -- start start 2023-10-03T10:17:05.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.154+0000 7f549595c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5490133a80 0x7f5490133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:05.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.154+0000 7f549595c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5490130a70 0x7f549013b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:05.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.154+0000 7f549595c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549006bc00 con 0x7f549012ff20 2023-10-03T10:17:05.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.154+0000 7f549595c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f549006c570 con 0x7f5490133a80 2023-10-03T10:17:05.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.154+0000 7f549595c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54900a0040 con 0x7f5490130a70 2023-10-03T10:17:05.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.154+0000 7f548f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5490130a70 0x7f549013b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:05.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.154+0000 7f548e7fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f549012ff20 0x7f5490130360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45952/0 (socket says 172.21.15.3:45952) 2023-10-03T10:17:05.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.154+0000 7f548e7fc700 1 -- 172.21.15.3:0/3434944883 learned_addr learned my addr 172.21.15.3:0/3434944883 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:05.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.154+0000 7f548f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5490130a70 0x7f549013b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50278/0 (socket says 172.21.15.3:50278) 2023-10-03T10:17:05.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.155+0000 7f548effd700 1 --2- 172.21.15.3:0/3434944883 >> v2:172.21.15.3:3300/0 conn(0x7f5490133a80 0x7f5490133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:05.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.155+0000 7f548effd700 1 -- 172.21.15.3:0/3434944883 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5490130a70 msgr2=0x7f549013b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:05.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.155+0000 7f548effd700 1 --2- 172.21.15.3:0/3434944883 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5490130a70 0x7f549013b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:05.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.155+0000 7f548effd700 1 -- 172.21.15.3:0/3434944883 >> v1:172.21.15.3:6789/0 conn(0x7f549012ff20 legacy=0x7f5490130360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:05.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.155+0000 7f548effd700 1 -- 172.21.15.3:0/3434944883 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5490133860 con 0x7f5490133a80 2023-10-03T10:17:05.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.155+0000 7f548dffb700 1 -- 172.21.15.3:0/3434944883 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1114147163 0 0) 0x7f549006bc00 con 0x7f549012ff20 2023-10-03T10:17:05.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.155+0000 7f548f7fe700 1 --2- 172.21.15.3:0/3434944883 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5490130a70 0x7f549013b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:05.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.155+0000 7f548effd700 1 --2- 172.21.15.3:0/3434944883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5490133a80 0x7f5490133e50 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f5480009840 tx=0x7f5480009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c237f56946c5030f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:05.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.155+0000 7f548dffb700 1 -- 172.21.15.3:0/3434944883 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5480011830 con 0x7f5490133a80 2023-10-03T10:17:05.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.155+0000 7f548dffb700 1 -- 172.21.15.3:0/3434944883 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5480011990 con 0x7f5490133a80 2023-10-03T10:17:05.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.155+0000 7f548dffb700 1 -- 172.21.15.3:0/3434944883 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5480011d20 con 0x7f5490133a80 2023-10-03T10:17:05.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.156+0000 7f549595c700 1 -- 172.21.15.3:0/3434944883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5490133a80 msgr2=0x7f5490133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:05.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.156+0000 7f549595c700 1 --2- 172.21.15.3:0/3434944883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5490133a80 0x7f5490133e50 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f5480009840 tx=0x7f5480009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:05.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.156+0000 7f549595c700 1 -- 172.21.15.3:0/3434944883 shutdown_connections 2023-10-03T10:17:05.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.156+0000 7f549595c700 1 --2- 172.21.15.3:0/3434944883 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5490130a70 0x7f549013b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:05.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.156+0000 7f549595c700 1 --2- 172.21.15.3:0/3434944883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5490133a80 0x7f5490133e50 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:05.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.156+0000 7f549595c700 1 -- 172.21.15.3:0/3434944883 >> 172.21.15.3:0/3434944883 conn(0x7f54900a4210 msgr2=0x7f54900a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:05.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.156+0000 7f549595c700 1 -- 172.21.15.3:0/3434944883 shutdown_connections 2023-10-03T10:17:05.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.156+0000 7f549595c700 1 -- 172.21.15.3:0/3434944883 wait complete. 2023-10-03T10:17:05.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.157+0000 7f549595c700 1 Processor -- start 2023-10-03T10:17:05.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.158+0000 7f549595c700 1 -- start start 2023-10-03T10:17:05.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.158+0000 7f549595c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f549012ff20 0x7f549009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:05.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.158+0000 7f549595c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5490130a70 0x7f549009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:05.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.158+0000 7f549595c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5490133a80 0x7f549009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:05.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.158+0000 7f549595c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549011f180 con 0x7f5490130a70 2023-10-03T10:17:05.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.158+0000 7f549595c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549012da40 con 0x7f5490133a80 2023-10-03T10:17:05.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.158+0000 7f549595c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f549012dc30 con 0x7f549012ff20 2023-10-03T10:17:05.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.159+0000 7f548e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5490130a70 0x7f549009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:05.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.159+0000 7f548e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5490130a70 0x7f549009c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36944/0 (socket says 172.21.15.3:36944) 2023-10-03T10:17:05.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.159+0000 7f548e7fc700 1 -- 172.21.15.3:0/4109086162 learned_addr learned my addr 172.21.15.3:0/4109086162 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:05.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.159+0000 7f548effd700 1 --2- 172.21.15.3:0/4109086162 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f549012ff20 0x7f549009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:05.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.159+0000 7f548f7fe700 1 --2- 172.21.15.3:0/4109086162 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5490133a80 0x7f549009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:05.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.159+0000 7f548effd700 1 -- 172.21.15.3:0/4109086162 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5490133a80 msgr2=0x7f549009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:05.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.159+0000 7f548effd700 1 --2- 172.21.15.3:0/4109086162 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5490133a80 0x7f549009cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:05.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.159+0000 7f548effd700 1 -- 172.21.15.3:0/4109086162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5490130a70 msgr2=0x7f549009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:05.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.159+0000 7f548effd700 1 --2- 172.21.15.3:0/4109086162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5490130a70 0x7f549009c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:05.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.159+0000 7f548effd700 1 -- 172.21.15.3:0/4109086162 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5488009a70 con 0x7f549012ff20 2023-10-03T10:17:05.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.159+0000 7f548effd700 1 --2- 172.21.15.3:0/4109086162 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f549012ff20 0x7f549009c480 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f5480016000 tx=0x7f5480004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:05.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.159+0000 7f5477fff700 1 -- 172.21.15.3:0/4109086162 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54800045d0 con 0x7f549012ff20 2023-10-03T10:17:05.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.159+0000 7f5477fff700 1 -- 172.21.15.3:0/4109086162 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548001f070 con 0x7f549012ff20 2023-10-03T10:17:05.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.159+0000 7f549595c700 1 -- 172.21.15.3:0/4109086162 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54800094a0 con 0x7f549012ff20 2023-10-03T10:17:05.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.160+0000 7f5477fff700 1 -- 172.21.15.3:0/4109086162 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54800038a0 con 0x7f549012ff20 2023-10-03T10:17:05.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.160+0000 7f549595c700 1 -- 172.21.15.3:0/4109086162 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549011fbf0 con 0x7f549012ff20 2023-10-03T10:17:05.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.161+0000 7f5477fff700 1 -- 172.21.15.3:0/4109086162 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5480003a00 con 0x7f549012ff20 2023-10-03T10:17:05.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.161+0000 7f549595c700 1 -- 172.21.15.3:0/4109086162 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5490074ea0 con 0x7f549012ff20 2023-10-03T10:17:05.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.161+0000 7f5477fff700 1 -- 172.21.15.3:0/4109086162 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5480021070 con 0x7f549012ff20 2023-10-03T10:17:05.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.166+0000 7f5477fff700 1 -- 172.21.15.3:0/4109086162 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5480010380 con 0x7f549012ff20 2023-10-03T10:17:05.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.270+0000 7efd0d650700 1 -- 172.21.15.3:0/1645588644 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7efd08122700 con 0x7efd08136710 2023-10-03T10:17:05.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.270+0000 7efcef7fe700 1 -- 172.21.15.3:0/1645588644 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efd08122700 con 0x7efd08136710 2023-10-03T10:17:05.270 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:05.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.274+0000 7efd0d650700 1 -- 172.21.15.3:0/1645588644 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd08136710 msgr2=0x7efd0809e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:05.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.274+0000 7efd0d650700 1 --2- 172.21.15.3:0/1645588644 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd08136710 0x7efd0809e690 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7efcfc016000 tx=0x7efcfc004750 comp rx=0 tx=0).stop 2023-10-03T10:17:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.274+0000 7efd0d650700 1 -- 172.21.15.3:0/1645588644 shutdown_connections 2023-10-03T10:17:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.274+0000 7efd0d650700 1 --2- 172.21.15.3:0/1645588644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd08139030 0x7efd0809ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.274+0000 7efd0d650700 1 --2- 172.21.15.3:0/1645588644 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd081338a0 0x7efd081214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.274+0000 7efd0d650700 1 --2- 172.21.15.3:0/1645588644 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efd08136710 0x7efd0809e690 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.274+0000 7efd0d650700 1 -- 172.21.15.3:0/1645588644 >> 172.21.15.3:0/1645588644 conn(0x7efd080a2aa0 msgr2=0x7efd081372d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.274+0000 7efd0d650700 1 -- 172.21.15.3:0/1645588644 shutdown_connections 2023-10-03T10:17:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.274+0000 7efd0d650700 1 -- 172.21.15.3:0/1645588644 wait complete. 2023-10-03T10:17:05.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.284+0000 7f3fa9645700 1 -- 172.21.15.3:0/2443981901 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3fa40747f0 con 0x7f3fa409d5c0 2023-10-03T10:17:05.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.285+0000 7f3f8b7fe700 1 -- 172.21.15.3:0/2443981901 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3f940303e0 con 0x7f3fa409d5c0 2023-10-03T10:17:05.285 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:05.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.287+0000 7f3fa9645700 1 -- 172.21.15.3:0/2443981901 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3fa409d5c0 msgr2=0x7f3fa4120410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:05.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.287+0000 7f3fa9645700 1 --2- 172.21.15.3:0/2443981901 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3fa409d5c0 0x7f3fa4120410 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3f94016000 tx=0x7f3f94004ad0 comp rx=0 tx=0).stop 2023-10-03T10:17:05.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.287+0000 7f3fa9645700 1 -- 172.21.15.3:0/2443981901 shutdown_connections 2023-10-03T10:17:05.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.287+0000 7f3fa9645700 1 --2- 172.21.15.3:0/2443981901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fa40a12d0 0x7f3fa41c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:05.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.287+0000 7f3fa9645700 1 --2- 172.21.15.3:0/2443981901 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3fa409d5c0 0x7f3fa4120410 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:05.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.287+0000 7f3fa9645700 1 --2- 172.21.15.3:0/2443981901 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3fa40a09c0 0x7f3fa4120950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:05.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.287+0000 7f3fa9645700 1 -- 172.21.15.3:0/2443981901 >> 172.21.15.3:0/2443981901 conn(0x7f3fa40a2fe0 msgr2=0x7f3fa41bba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:05.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.287+0000 7f3fa9645700 1 -- 172.21.15.3:0/2443981901 shutdown_connections 2023-10-03T10:17:05.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.287+0000 7f3fa9645700 1 -- 172.21.15.3:0/2443981901 wait complete. 2023-10-03T10:17:05.289 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:05.303 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:05.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.377+0000 7f549595c700 1 -- 172.21.15.3:0/4109086162 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f549006b250 con 0x7f549012ff20 2023-10-03T10:17:05.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.378+0000 7f5477fff700 1 -- 172.21.15.3:0/4109086162 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5480040e60 con 0x7f549012ff20 2023-10-03T10:17:05.378 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.380+0000 7f549595c700 1 -- 172.21.15.3:0/4109086162 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f549012ff20 msgr2=0x7f549009c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.380+0000 7f549595c700 1 --2- 172.21.15.3:0/4109086162 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f549012ff20 0x7f549009c480 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f5480016000 tx=0x7f5480004750 comp rx=0 tx=0).stop 2023-10-03T10:17:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.380+0000 7f549595c700 1 -- 172.21.15.3:0/4109086162 shutdown_connections 2023-10-03T10:17:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.380+0000 7f549595c700 1 --2- 172.21.15.3:0/4109086162 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5490133a80 0x7f549009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.380+0000 7f549595c700 1 --2- 172.21.15.3:0/4109086162 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f549012ff20 0x7f549009c480 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.380+0000 7f549595c700 1 --2- 172.21.15.3:0/4109086162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5490130a70 0x7f549009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.380+0000 7f549595c700 1 -- 172.21.15.3:0/4109086162 >> 172.21.15.3:0/4109086162 conn(0x7f54900a4210 msgr2=0x7f549009e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.380+0000 7f549595c700 1 -- 172.21.15.3:0/4109086162 shutdown_connections 2023-10-03T10:17:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:05.380+0000 7f549595c700 1 -- 172.21.15.3:0/4109086162 wait complete. 2023-10-03T10:17:05.394 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:06.017 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:06.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.163+0000 7f94938e5700 1 Processor -- start 2023-10-03T10:17:06.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.163+0000 7f94938e5700 1 -- start start 2023-10-03T10:17:06.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.164+0000 7f94938e5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f948c09d6f0 0x7f948c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:06.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.164+0000 7f94938e5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f948c134100 0x7f948c13bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:06.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.164+0000 7f94938e5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f948c09d2f0 con 0x7f948c09e000 2023-10-03T10:17:06.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.164+0000 7f94938e5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f948c09a810 con 0x7f948c134100 2023-10-03T10:17:06.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.164+0000 7f94938e5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f948c09a470 con 0x7f948c09d6f0 2023-10-03T10:17:06.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.164+0000 7f9491681700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f948c09d6f0 0x7f948c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:06.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.164+0000 7f9491e82700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f948c134100 0x7f948c13bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:06.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.164+0000 7f9491681700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f948c09d6f0 0x7f948c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:49850/0 (socket says 172.21.15.3:49850) 2023-10-03T10:17:06.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.164+0000 7f9491681700 1 -- 172.21.15.3:0/1384058448 learned_addr learned my addr 172.21.15.3:0/1384058448 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:06.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.164+0000 7f9491681700 1 -- 172.21.15.3:0/1384058448 >> v2:172.21.15.3:3300/0 conn(0x7f948c134100 msgr2=0x7f948c13bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:06.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.164+0000 7f9491681700 1 --2- 172.21.15.3:0/1384058448 >> v2:172.21.15.3:3300/0 conn(0x7f948c134100 0x7f948c13bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.164+0000 7f9491681700 1 -- 172.21.15.3:0/1384058448 >> v1:172.21.15.3:6789/0 conn(0x7f948c09e000 legacy=0x7f948c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:06.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.164+0000 7f9491681700 1 -- 172.21.15.3:0/1384058448 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948c09ec40 con 0x7f948c09d6f0 2023-10-03T10:17:06.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.164+0000 7f9491e82700 1 --2- 172.21.15.3:0/1384058448 >> v2:172.21.15.3:3300/0 conn(0x7f948c134100 0x7f948c13bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:06.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.165+0000 7f9491681700 1 --2- 172.21.15.3:0/1384058448 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f948c09d6f0 0x7f948c09dac0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f94880098f0 tx=0x7f9488009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a656eb720f9e1cc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:06.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.165+0000 7f9483fff700 1 -- 172.21.15.3:0/1384058448 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9488011830 con 0x7f948c09d6f0 2023-10-03T10:17:06.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.165+0000 7f9483fff700 1 -- 172.21.15.3:0/1384058448 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9488011990 con 0x7f948c09d6f0 2023-10-03T10:17:06.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.165+0000 7f9483fff700 1 -- 172.21.15.3:0/1384058448 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9488011d20 con 0x7f948c09d6f0 2023-10-03T10:17:06.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.165+0000 7f94938e5700 1 -- 172.21.15.3:0/1384058448 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f948c09d6f0 msgr2=0x7f948c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:06.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.165+0000 7f94938e5700 1 --2- 172.21.15.3:0/1384058448 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f948c09d6f0 0x7f948c09dac0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f94880098f0 tx=0x7f9488009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:06.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.165+0000 7f94938e5700 1 -- 172.21.15.3:0/1384058448 shutdown_connections 2023-10-03T10:17:06.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.165+0000 7f94938e5700 1 --2- 172.21.15.3:0/1384058448 >> v2:172.21.15.3:3300/0 conn(0x7f948c134100 0x7f948c13bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.165+0000 7f94938e5700 1 --2- 172.21.15.3:0/1384058448 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f948c09d6f0 0x7f948c09dac0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.165+0000 7f94938e5700 1 -- 172.21.15.3:0/1384058448 >> 172.21.15.3:0/1384058448 conn(0x7f948c066f50 msgr2=0x7f948c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:06.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.166+0000 7f94938e5700 1 -- 172.21.15.3:0/1384058448 shutdown_connections 2023-10-03T10:17:06.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.166+0000 7f94938e5700 1 -- 172.21.15.3:0/1384058448 wait complete. 2023-10-03T10:17:06.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.166+0000 7f94938e5700 1 Processor -- start 2023-10-03T10:17:06.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.167+0000 7f94938e5700 1 -- start start 2023-10-03T10:17:06.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.167+0000 7f94938e5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f948c09d6f0 0x7f948c1227d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:06.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.167+0000 7f94938e5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f948c09e000 0x7f948c122d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:06.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.167+0000 7f94938e5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f948c134100 0x7f948c123250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:06.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.167+0000 7f94938e5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c1339d0 con 0x7f948c09d6f0 2023-10-03T10:17:06.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.167+0000 7f94938e5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948c1244f0 con 0x7f948c134100 2023-10-03T10:17:06.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.167+0000 7f94938e5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f948c125d50 con 0x7f948c09e000 2023-10-03T10:17:06.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.167+0000 7f9490e80700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f948c09e000 0x7f948c122d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:06.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.167+0000 7f9491681700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f948c09d6f0 0x7f948c1227d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:06.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.167+0000 7f9491e82700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f948c134100 0x7f948c123250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:06.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.167+0000 7f9490e80700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f948c09e000 0x7f948c122d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:49864/0 (socket says 172.21.15.3:49864) 2023-10-03T10:17:06.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.167+0000 7f9490e80700 1 -- 172.21.15.3:0/2729412366 learned_addr learned my addr 172.21.15.3:0/2729412366 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:06.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.167+0000 7f9491681700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f948c09d6f0 0x7f948c1227d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36948/0 (socket says 172.21.15.3:36948) 2023-10-03T10:17:06.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.167+0000 7f9490e80700 1 -- 172.21.15.3:0/2729412366 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f948c134100 msgr2=0x7f948c123250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:06.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.167+0000 7f9490e80700 1 --2- 172.21.15.3:0/2729412366 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f948c134100 0x7f948c123250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.168+0000 7f9490e80700 1 -- 172.21.15.3:0/2729412366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f948c09d6f0 msgr2=0x7f948c1227d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:06.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.168+0000 7f9490e80700 1 --2- 172.21.15.3:0/2729412366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f948c09d6f0 0x7f948c1227d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.168+0000 7f9490e80700 1 -- 172.21.15.3:0/2729412366 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9484009a70 con 0x7f948c09e000 2023-10-03T10:17:06.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.168+0000 7f9491681700 1 --2- 172.21.15.3:0/2729412366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f948c09d6f0 0x7f948c1227d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:06.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.168+0000 7f9490e80700 1 --2- 172.21.15.3:0/2729412366 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f948c09e000 0x7f948c122d10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f947c00c4b0 tx=0x7f947c00c7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:06.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.168+0000 7f9481ffb700 1 -- 172.21.15.3:0/2729412366 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c0052d0 con 0x7f948c09e000 2023-10-03T10:17:06.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.168+0000 7f9481ffb700 1 -- 172.21.15.3:0/2729412366 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947c005430 con 0x7f948c09e000 2023-10-03T10:17:06.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.168+0000 7f94938e5700 1 -- 172.21.15.3:0/2729412366 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9488009550 con 0x7f948c09e000 2023-10-03T10:17:06.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.168+0000 7f9481ffb700 1 -- 172.21.15.3:0/2729412366 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c0038c0 con 0x7f948c09e000 2023-10-03T10:17:06.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.168+0000 7f94938e5700 1 -- 172.21.15.3:0/2729412366 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948c126a80 con 0x7f948c09e000 2023-10-03T10:17:06.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.169+0000 7f9481ffb700 1 -- 172.21.15.3:0/2729412366 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f947c003a20 con 0x7f948c09e000 2023-10-03T10:17:06.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.169+0000 7f9481ffb700 1 -- 172.21.15.3:0/2729412366 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f947c03ead0 con 0x7f948c09e000 2023-10-03T10:17:06.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.169+0000 7f94938e5700 1 -- 172.21.15.3:0/2729412366 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f948c074ea0 con 0x7f948c09e000 2023-10-03T10:17:06.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.174+0000 7f9481ffb700 1 -- 172.21.15.3:0/2729412366 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f947c017020 con 0x7f948c09e000 2023-10-03T10:17:06.290 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:06.304 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:06.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.380+0000 7f94938e5700 1 -- 172.21.15.3:0/2729412366 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f948c06c360 con 0x7f948c09e000 2023-10-03T10:17:06.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.380+0000 7f9481ffb700 1 -- 172.21.15.3:0/2729412366 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f947c018260 con 0x7f948c09e000 2023-10-03T10:17:06.381 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:06.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.383+0000 7f94938e5700 1 -- 172.21.15.3:0/2729412366 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f948c09e000 msgr2=0x7f948c122d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:06.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.383+0000 7f94938e5700 1 --2- 172.21.15.3:0/2729412366 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f948c09e000 0x7f948c122d10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f947c00c4b0 tx=0x7f947c00c7c0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.383+0000 7f94938e5700 1 -- 172.21.15.3:0/2729412366 shutdown_connections 2023-10-03T10:17:06.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.383+0000 7f94938e5700 1 --2- 172.21.15.3:0/2729412366 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f948c134100 0x7f948c123250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.383+0000 7f94938e5700 1 --2- 172.21.15.3:0/2729412366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f948c09d6f0 0x7f948c1227d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.383+0000 7f94938e5700 1 --2- 172.21.15.3:0/2729412366 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f948c09e000 0x7f948c122d10 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.383+0000 7f94938e5700 1 -- 172.21.15.3:0/2729412366 >> 172.21.15.3:0/2729412366 conn(0x7f948c066f50 msgr2=0x7f948c070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:06.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.383+0000 7f94938e5700 1 -- 172.21.15.3:0/2729412366 shutdown_connections 2023-10-03T10:17:06.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.383+0000 7f94938e5700 1 -- 172.21.15.3:0/2729412366 wait complete. 2023-10-03T10:17:06.394 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:06.398 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:06.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.433+0000 7facf1d81700 1 Processor -- start 2023-10-03T10:17:06.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.433+0000 7facf1d81700 1 -- start start 2023-10-03T10:17:06.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.434+0000 7facf1d81700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7facec09e1d0 0x7facec138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:06.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.434+0000 7facf1d81700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7facec139170 0x7facec13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:06.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.434+0000 7facf1d81700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facec09b120 con 0x7facec09d6f0 2023-10-03T10:17:06.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.434+0000 7facf1d81700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facec06b460 con 0x7facec09e1d0 2023-10-03T10:17:06.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.434+0000 7facf1d81700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7facec09a7e0 con 0x7facec139170 2023-10-03T10:17:06.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.434+0000 7faceb7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7facec09d6f0 0x7facec09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45972/0 (socket says 172.21.15.3:45972) 2023-10-03T10:17:06.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.434+0000 7faceb7fe700 1 -- 172.21.15.3:0/3192966970 learned_addr learned my addr 172.21.15.3:0/3192966970 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:06.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.434+0000 7faceaffd700 1 --2- 172.21.15.3:0/3192966970 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7facec09e1d0 0x7facec138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:06.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.434+0000 7facebfff700 1 --2- 172.21.15.3:0/3192966970 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7facec139170 0x7facec13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:06.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.435+0000 7facea7fc700 1 -- 172.21.15.3:0/3192966970 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 279336889 0 0) 0x7facec09b120 con 0x7facec09d6f0 2023-10-03T10:17:06.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.435+0000 7facea7fc700 1 -- 172.21.15.3:0/3192966970 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facd40035b0 con 0x7facec09d6f0 2023-10-03T10:17:06.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.435+0000 7faceaffd700 1 -- 172.21.15.3:0/3192966970 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7facec139170 msgr2=0x7facec13b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:06.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.435+0000 7faceaffd700 1 --2- 172.21.15.3:0/3192966970 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7facec139170 0x7facec13b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.435+0000 7faceaffd700 1 -- 172.21.15.3:0/3192966970 >> v1:172.21.15.3:6789/0 conn(0x7facec09d6f0 legacy=0x7facec09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:06.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.435+0000 7faceaffd700 1 -- 172.21.15.3:0/3192966970 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facec09a500 con 0x7facec09e1d0 2023-10-03T10:17:06.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.435+0000 7facebfff700 1 --2- 172.21.15.3:0/3192966970 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7facec139170 0x7facec13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:06.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.435+0000 7faceaffd700 1 --2- 172.21.15.3:0/3192966970 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7facec09e1d0 0x7facec138c30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7face0009870 tx=0x7face0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae76031251b21afe server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:06.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.435+0000 7facea7fc700 1 -- 172.21.15.3:0/3192966970 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0011830 con 0x7facec09e1d0 2023-10-03T10:17:06.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.435+0000 7facea7fc700 1 -- 172.21.15.3:0/3192966970 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face0011990 con 0x7facec09e1d0 2023-10-03T10:17:06.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.435+0000 7facea7fc700 1 -- 172.21.15.3:0/3192966970 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0011d20 con 0x7facec09e1d0 2023-10-03T10:17:06.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.436+0000 7facf1d81700 1 -- 172.21.15.3:0/3192966970 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7facec09e1d0 msgr2=0x7facec138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:06.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.436+0000 7facf1d81700 1 --2- 172.21.15.3:0/3192966970 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7facec09e1d0 0x7facec138c30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7face0009870 tx=0x7face0009b80 comp rx=0 tx=0).stop 2023-10-03T10:17:06.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.436+0000 7facf1d81700 1 -- 172.21.15.3:0/3192966970 shutdown_connections 2023-10-03T10:17:06.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.436+0000 7facf1d81700 1 --2- 172.21.15.3:0/3192966970 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7facec139170 0x7facec13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.436+0000 7facf1d81700 1 --2- 172.21.15.3:0/3192966970 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7facec09e1d0 0x7facec138c30 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.436+0000 7facf1d81700 1 -- 172.21.15.3:0/3192966970 >> 172.21.15.3:0/3192966970 conn(0x7facec066f50 msgr2=0x7facec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:06.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.436+0000 7facf1d81700 1 -- 172.21.15.3:0/3192966970 shutdown_connections 2023-10-03T10:17:06.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.436+0000 7facf1d81700 1 -- 172.21.15.3:0/3192966970 wait complete. 2023-10-03T10:17:06.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.438+0000 7facf1d81700 1 Processor -- start 2023-10-03T10:17:06.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.438+0000 7facf1d81700 1 -- start start 2023-10-03T10:17:06.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.439+0000 7facf1d81700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7facec09d6f0 0x7facec1216a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:06.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.439+0000 7facf1d81700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facec09e1d0 0x7facec121be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:06.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.439+0000 7facf1d81700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7facec139170 0x7facec122120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:06.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.439+0000 7facf1d81700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec1251c0 con 0x7facec09e1d0 2023-10-03T10:17:06.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.439+0000 7facf1d81700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facec1240b0 con 0x7facec09d6f0 2023-10-03T10:17:06.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.439+0000 7facf1d81700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7facec0a1780 con 0x7facec139170 2023-10-03T10:17:06.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.439+0000 7faceb7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7facec09d6f0 0x7facec1216a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:06.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.439+0000 7faceb7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7facec09d6f0 0x7facec1216a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50320/0 (socket says 172.21.15.3:50320) 2023-10-03T10:17:06.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.439+0000 7faceb7fe700 1 -- 172.21.15.3:0/1269843717 learned_addr learned my addr 172.21.15.3:0/1269843717 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:06.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.439+0000 7faceaffd700 1 --2- 172.21.15.3:0/1269843717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facec09e1d0 0x7facec121be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:06.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.439+0000 7facebfff700 1 --2- 172.21.15.3:0/1269843717 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7facec139170 0x7facec122120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:06.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.439+0000 7faceaffd700 1 -- 172.21.15.3:0/1269843717 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7facec139170 msgr2=0x7facec122120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:06.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.439+0000 7faceaffd700 1 --2- 172.21.15.3:0/1269843717 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7facec139170 0x7facec122120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.439+0000 7faceaffd700 1 -- 172.21.15.3:0/1269843717 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7facec09d6f0 msgr2=0x7facec1216a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:06.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.439+0000 7faceaffd700 1 --2- 172.21.15.3:0/1269843717 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7facec09d6f0 0x7facec1216a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.439+0000 7faceaffd700 1 -- 172.21.15.3:0/1269843717 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face4009a70 con 0x7facec09e1d0 2023-10-03T10:17:06.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.439+0000 7faceaffd700 1 --2- 172.21.15.3:0/1269843717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facec09e1d0 0x7facec121be0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7face0009870 tx=0x7face0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:06.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.440+0000 7facd3fff700 1 -- 172.21.15.3:0/1269843717 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0004450 con 0x7facec09e1d0 2023-10-03T10:17:06.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.440+0000 7facd3fff700 1 -- 172.21.15.3:0/1269843717 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face001f070 con 0x7facec09e1d0 2023-10-03T10:17:06.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.440+0000 7facf1d81700 1 -- 172.21.15.3:0/1269843717 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face00094a0 con 0x7facec09e1d0 2023-10-03T10:17:06.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.440+0000 7facd3fff700 1 -- 172.21.15.3:0/1269843717 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face00038e0 con 0x7facec09e1d0 2023-10-03T10:17:06.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.440+0000 7facf1d81700 1 -- 172.21.15.3:0/1269843717 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facec123350 con 0x7facec09e1d0 2023-10-03T10:17:06.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.441+0000 7facd3fff700 1 -- 172.21.15.3:0/1269843717 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7face0003a40 con 0x7facec09e1d0 2023-10-03T10:17:06.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.441+0000 7facf1d81700 1 -- 172.21.15.3:0/1269843717 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facec074ea0 con 0x7facec09e1d0 2023-10-03T10:17:06.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.441+0000 7facd3fff700 1 -- 172.21.15.3:0/1269843717 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7face0021070 con 0x7facec09e1d0 2023-10-03T10:17:06.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.446+0000 7facd3fff700 1 -- 172.21.15.3:0/1269843717 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7face0015030 con 0x7facec09e1d0 2023-10-03T10:17:06.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.477+0000 7fe91da13700 1 Processor -- start 2023-10-03T10:17:06.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.477+0000 7fe91da13700 1 -- start start 2023-10-03T10:17:06.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.478+0000 7fe91da13700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe91809d5c0 0x7fe918138d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:06.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.478+0000 7fe91da13700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9180a14a0 0x7fe91813b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:06.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.478+0000 7fe91da13700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe91809ea10 con 0x7fe9180a09c0 2023-10-03T10:17:06.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.478+0000 7fe91da13700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe91809e210 con 0x7fe91809d5c0 2023-10-03T10:17:06.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.478+0000 7fe91da13700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9180a26f0 con 0x7fe9180a14a0 2023-10-03T10:17:06.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.478+0000 7fe9177fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9180a14a0 0x7fe91813b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:06.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.478+0000 7fe90ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe91809d5c0 0x7fe918138d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:06.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.478+0000 7fe9177fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9180a14a0 0x7fe91813b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:49890/0 (socket says 172.21.15.3:49890) 2023-10-03T10:17:06.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.478+0000 7fe9177fe700 1 -- 172.21.15.3:0/2032561380 learned_addr learned my addr 172.21.15.3:0/2032561380 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:06.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.478+0000 7fe916ffd700 1 --1- 172.21.15.3:0/2032561380 >> v1:172.21.15.3:6789/0 conn(0x7fe9180a09c0 0x7fe9180a0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45984/0 (socket says 172.21.15.3:45984) 2023-10-03T10:17:06.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.478+0000 7fe90ffff700 1 -- 172.21.15.3:0/2032561380 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9180a14a0 msgr2=0x7fe91813b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:06.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.478+0000 7fe90ffff700 1 --2- 172.21.15.3:0/2032561380 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9180a14a0 0x7fe91813b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.478+0000 7fe90ffff700 1 -- 172.21.15.3:0/2032561380 >> v1:172.21.15.3:6789/0 conn(0x7fe9180a09c0 legacy=0x7fe9180a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:06.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.478+0000 7fe90ffff700 1 -- 172.21.15.3:0/2032561380 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe91809db70 con 0x7fe91809d5c0 2023-10-03T10:17:06.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.478+0000 7fe9167fc700 1 -- 172.21.15.3:0/2032561380 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3089081769 0 0) 0x7fe91809ea10 con 0x7fe9180a09c0 2023-10-03T10:17:06.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.479+0000 7fe90ffff700 1 --2- 172.21.15.3:0/2032561380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe91809d5c0 0x7fe918138d70 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fe900009840 tx=0x7fe900009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=469822bd41c210a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:06.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.479+0000 7fe9167fc700 1 -- 172.21.15.3:0/2032561380 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe900011830 con 0x7fe91809d5c0 2023-10-03T10:17:06.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.479+0000 7fe9167fc700 1 -- 172.21.15.3:0/2032561380 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe900011990 con 0x7fe91809d5c0 2023-10-03T10:17:06.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.479+0000 7fe9167fc700 1 -- 172.21.15.3:0/2032561380 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe900011d20 con 0x7fe91809d5c0 2023-10-03T10:17:06.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.480+0000 7fe91da13700 1 -- 172.21.15.3:0/2032561380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe91809d5c0 msgr2=0x7fe918138d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:06.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.480+0000 7fe91da13700 1 --2- 172.21.15.3:0/2032561380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe91809d5c0 0x7fe918138d70 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fe900009840 tx=0x7fe900009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:06.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.480+0000 7fe91da13700 1 -- 172.21.15.3:0/2032561380 shutdown_connections 2023-10-03T10:17:06.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.480+0000 7fe91da13700 1 --2- 172.21.15.3:0/2032561380 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9180a14a0 0x7fe91813b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.480+0000 7fe91da13700 1 --2- 172.21.15.3:0/2032561380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe91809d5c0 0x7fe918138d70 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.480+0000 7fe91da13700 1 -- 172.21.15.3:0/2032561380 >> 172.21.15.3:0/2032561380 conn(0x7fe9180a2fe0 msgr2=0x7fe9180a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:06.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.480+0000 7fe91da13700 1 -- 172.21.15.3:0/2032561380 shutdown_connections 2023-10-03T10:17:06.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.480+0000 7fe91da13700 1 -- 172.21.15.3:0/2032561380 wait complete. 2023-10-03T10:17:06.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.481+0000 7fe91da13700 1 Processor -- start 2023-10-03T10:17:06.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.481+0000 7fe91da13700 1 -- start start 2023-10-03T10:17:06.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.481+0000 7fe91da13700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe91809d5c0 0x7fe91811c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:06.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.481+0000 7fe91da13700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9180a09c0 0x7fe91811ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:06.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.481+0000 7fe91da13700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9180a14a0 0x7fe91811d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:06.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.481+0000 7fe91da13700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91811fb90 con 0x7fe91809d5c0 2023-10-03T10:17:06.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.481+0000 7fe91da13700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe91811ed20 con 0x7fe9180a09c0 2023-10-03T10:17:06.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.481+0000 7fe91da13700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe91809c340 con 0x7fe9180a14a0 2023-10-03T10:17:06.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.481+0000 7fe9177fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9180a14a0 0x7fe91811d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:06.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.481+0000 7fe916ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe91809d5c0 0x7fe91811c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:06.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.481+0000 7fe9177fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9180a14a0 0x7fe91811d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:49900/0 (socket says 172.21.15.3:49900) 2023-10-03T10:17:06.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.481+0000 7fe9177fe700 1 -- 172.21.15.3:0/1943195078 learned_addr learned my addr 172.21.15.3:0/1943195078 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:06.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.482+0000 7fe90ffff700 1 --2- 172.21.15.3:0/1943195078 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9180a09c0 0x7fe91811ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:06.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.482+0000 7fe916ffd700 1 -- 172.21.15.3:0/1943195078 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9180a14a0 msgr2=0x7fe91811d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:06.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.482+0000 7fe916ffd700 1 --2- 172.21.15.3:0/1943195078 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9180a14a0 0x7fe91811d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.482+0000 7fe916ffd700 1 -- 172.21.15.3:0/1943195078 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9180a09c0 msgr2=0x7fe91811ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:06.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.482+0000 7fe916ffd700 1 --2- 172.21.15.3:0/1943195078 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9180a09c0 0x7fe91811ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.482+0000 7fe916ffd700 1 -- 172.21.15.3:0/1943195078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9000094a0 con 0x7fe91809d5c0 2023-10-03T10:17:06.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.482+0000 7fe916ffd700 1 --2- 172.21.15.3:0/1943195078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe91809d5c0 0x7fe91811c7b0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fe90800c470 tx=0x7fe90800c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:06.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.482+0000 7fe914ff9700 1 -- 172.21.15.3:0/1943195078 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe908004c80 con 0x7fe91809d5c0 2023-10-03T10:17:06.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.482+0000 7fe914ff9700 1 -- 172.21.15.3:0/1943195078 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe908004de0 con 0x7fe91809d5c0 2023-10-03T10:17:06.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.482+0000 7fe91da13700 1 -- 172.21.15.3:0/1943195078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe91809cec0 con 0x7fe91809d5c0 2023-10-03T10:17:06.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.482+0000 7fe914ff9700 1 -- 172.21.15.3:0/1943195078 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe908007310 con 0x7fe91809d5c0 2023-10-03T10:17:06.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.482+0000 7fe91da13700 1 -- 172.21.15.3:0/1943195078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9181346d0 con 0x7fe91809d5c0 2023-10-03T10:17:06.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.483+0000 7fe914ff9700 1 -- 172.21.15.3:0/1943195078 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe908007470 con 0x7fe91809d5c0 2023-10-03T10:17:06.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.483+0000 7fe914ff9700 1 -- 172.21.15.3:0/1943195078 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe90803f340 con 0x7fe91809d5c0 2023-10-03T10:17:06.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.484+0000 7fe91da13700 1 -- 172.21.15.3:0/1943195078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9180747f0 con 0x7fe91809d5c0 2023-10-03T10:17:06.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.489+0000 7fe914ff9700 1 -- 172.21.15.3:0/1943195078 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe908007720 con 0x7fe91809d5c0 2023-10-03T10:17:06.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.541+0000 7f7633308700 1 Processor -- start 2023-10-03T10:17:06.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.541+0000 7f7633308700 1 -- start start 2023-10-03T10:17:06.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.542+0000 7f7633308700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f762c09d6f0 0x7f762c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:06.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.542+0000 7f7633308700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f762c09e410 0x7f762c134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:06.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.542+0000 7f7633308700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f762c09b120 con 0x7f762c09e000 2023-10-03T10:17:06.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.542+0000 7f7633308700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f762c06b460 con 0x7f762c09e410 2023-10-03T10:17:06.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.542+0000 7f7633308700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f762c09a7e0 con 0x7f762c09d6f0 2023-10-03T10:17:06.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.542+0000 7f76310a4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f762c09d6f0 0x7f762c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:06.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.542+0000 7f76310a4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f762c09d6f0 0x7f762c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50328/0 (socket says 172.21.15.3:50328) 2023-10-03T10:17:06.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.542+0000 7f76310a4700 1 -- 172.21.15.3:0/2202853221 learned_addr learned my addr 172.21.15.3:0/2202853221 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:06.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.542+0000 7f76318a5700 1 --2- 172.21.15.3:0/2202853221 >> v2:172.21.15.3:3300/0 conn(0x7f762c09e410 0x7f762c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:06.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.542+0000 7f76310a4700 1 -- 172.21.15.3:0/2202853221 >> v2:172.21.15.3:3300/0 conn(0x7f762c09e410 msgr2=0x7f762c134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:06.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.542+0000 7f76310a4700 1 --2- 172.21.15.3:0/2202853221 >> v2:172.21.15.3:3300/0 conn(0x7f762c09e410 0x7f762c134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.542+0000 7f7623fff700 1 -- 172.21.15.3:0/2202853221 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4087286850 0 0) 0x7f762c09b120 con 0x7f762c09e000 2023-10-03T10:17:06.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.542+0000 7f76310a4700 1 -- 172.21.15.3:0/2202853221 >> v1:172.21.15.3:6789/0 conn(0x7f762c09e000 legacy=0x7f762c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:06.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.542+0000 7f76310a4700 1 -- 172.21.15.3:0/2202853221 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f762c09a500 con 0x7f762c09d6f0 2023-10-03T10:17:06.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.543+0000 7f76310a4700 1 --2- 172.21.15.3:0/2202853221 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f762c09d6f0 0x7f762c09dac0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f7628009920 tx=0x7f7628009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fad0bc1ad09c0437 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:06.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.543+0000 7f7623fff700 1 -- 172.21.15.3:0/2202853221 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7628011830 con 0x7f762c09d6f0 2023-10-03T10:17:06.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.543+0000 7f7623fff700 1 -- 172.21.15.3:0/2202853221 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7628011990 con 0x7f762c09d6f0 2023-10-03T10:17:06.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.543+0000 7f7623fff700 1 -- 172.21.15.3:0/2202853221 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7628011d20 con 0x7f762c09d6f0 2023-10-03T10:17:06.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.543+0000 7f7633308700 1 -- 172.21.15.3:0/2202853221 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f762c09d6f0 msgr2=0x7f762c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:06.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.543+0000 7f7633308700 1 --2- 172.21.15.3:0/2202853221 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f762c09d6f0 0x7f762c09dac0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f7628009920 tx=0x7f7628009c30 comp rx=0 tx=0).stop 2023-10-03T10:17:06.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.543+0000 7f7633308700 1 -- 172.21.15.3:0/2202853221 shutdown_connections 2023-10-03T10:17:06.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.543+0000 7f7633308700 1 --2- 172.21.15.3:0/2202853221 >> v2:172.21.15.3:3300/0 conn(0x7f762c09e410 0x7f762c134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.543+0000 7f7633308700 1 --2- 172.21.15.3:0/2202853221 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f762c09d6f0 0x7f762c09dac0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.544+0000 7f7633308700 1 -- 172.21.15.3:0/2202853221 >> 172.21.15.3:0/2202853221 conn(0x7f762c066f50 msgr2=0x7f762c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:06.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.544+0000 7f7633308700 1 -- 172.21.15.3:0/2202853221 shutdown_connections 2023-10-03T10:17:06.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.544+0000 7f7633308700 1 -- 172.21.15.3:0/2202853221 wait complete. 2023-10-03T10:17:06.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.545+0000 7f7633308700 1 Processor -- start 2023-10-03T10:17:06.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.545+0000 7f7633308700 1 -- start start 2023-10-03T10:17:06.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.545+0000 7f7633308700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f762c09d6f0 0x7f762c1a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:06.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.545+0000 7f7633308700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f762c09e000 0x7f762c1a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:06.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.545+0000 7f7633308700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f762c09e410 0x7f762c1a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:06.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.545+0000 7f7633308700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762c1aa220 con 0x7f762c09e410 2023-10-03T10:17:06.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.545+0000 7f7633308700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f762c1a7950 con 0x7f762c09e000 2023-10-03T10:17:06.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.545+0000 7f7633308700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f762c1a7f70 con 0x7f762c09d6f0 2023-10-03T10:17:06.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.545+0000 7f76308a3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f762c09e000 0x7f762c1a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:06.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.545+0000 7f76308a3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f762c09e000 0x7f762c1a6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50338/0 (socket says 172.21.15.3:50338) 2023-10-03T10:17:06.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.545+0000 7f76318a5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f762c09e410 0x7f762c1a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:06.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.545+0000 7f76308a3700 1 -- 172.21.15.3:0/528320742 learned_addr learned my addr 172.21.15.3:0/528320742 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:06.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.545+0000 7f76310a4700 1 --2- 172.21.15.3:0/528320742 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f762c09d6f0 0x7f762c1a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:06.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.546+0000 7f76308a3700 1 -- 172.21.15.3:0/528320742 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f762c09d6f0 msgr2=0x7f762c1a5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:06.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.546+0000 7f76308a3700 1 --2- 172.21.15.3:0/528320742 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f762c09d6f0 0x7f762c1a5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.546+0000 7f76308a3700 1 -- 172.21.15.3:0/528320742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f762c09e410 msgr2=0x7f762c1a6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:06.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.546+0000 7f76308a3700 1 --2- 172.21.15.3:0/528320742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f762c09e410 0x7f762c1a6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.546+0000 7f76308a3700 1 -- 172.21.15.3:0/528320742 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7628009550 con 0x7f762c09e000 2023-10-03T10:17:06.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.546+0000 7f76318a5700 1 --2- 172.21.15.3:0/528320742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f762c09e410 0x7f762c1a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:06.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.546+0000 7f76308a3700 1 --2- 172.21.15.3:0/528320742 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f762c09e000 0x7f762c1a6320 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f761c00c580 tx=0x7f761c00c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:06.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.546+0000 7f7621ffb700 1 -- 172.21.15.3:0/528320742 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761c0054e0 con 0x7f762c09e000 2023-10-03T10:17:06.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.546+0000 7f7621ffb700 1 -- 172.21.15.3:0/528320742 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761c003680 con 0x7f762c09e000 2023-10-03T10:17:06.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.546+0000 7f7633308700 1 -- 172.21.15.3:0/528320742 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f762c1a8ba0 con 0x7f762c09e000 2023-10-03T10:17:06.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.546+0000 7f7621ffb700 1 -- 172.21.15.3:0/528320742 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761c003a30 con 0x7f762c09e000 2023-10-03T10:17:06.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.547+0000 7f7633308700 1 -- 172.21.15.3:0/528320742 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f762c0a3090 con 0x7f762c09e000 2023-10-03T10:17:06.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.547+0000 7f7621ffb700 1 -- 172.21.15.3:0/528320742 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f761c003b90 con 0x7f762c09e000 2023-10-03T10:17:06.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.547+0000 7f7633308700 1 -- 172.21.15.3:0/528320742 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f762c09c710 con 0x7f762c09e000 2023-10-03T10:17:06.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.548+0000 7f7621ffb700 1 -- 172.21.15.3:0/528320742 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f761c03eaa0 con 0x7f762c09e000 2023-10-03T10:17:06.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.557+0000 7f7621ffb700 1 -- 172.21.15.3:0/528320742 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f761c003e40 con 0x7f762c09e000 2023-10-03T10:17:06.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.674+0000 7facf1d81700 1 -- 172.21.15.3:0/1269843717 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7facec124570 con 0x7facec09e1d0 2023-10-03T10:17:06.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.675+0000 7facd3fff700 1 -- 172.21.15.3:0/1269843717 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7face0026480 con 0x7facec09e1d0 2023-10-03T10:17:06.675 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:06.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.677+0000 7facf1d81700 1 -- 172.21.15.3:0/1269843717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facec09e1d0 msgr2=0x7facec121be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:06.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.677+0000 7facf1d81700 1 --2- 172.21.15.3:0/1269843717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facec09e1d0 0x7facec121be0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7face0009870 tx=0x7face0004750 comp rx=0 tx=0).stop 2023-10-03T10:17:06.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.678+0000 7facf1d81700 1 -- 172.21.15.3:0/1269843717 shutdown_connections 2023-10-03T10:17:06.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.678+0000 7facf1d81700 1 --2- 172.21.15.3:0/1269843717 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7facec139170 0x7facec122120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.678+0000 7facf1d81700 1 --2- 172.21.15.3:0/1269843717 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7facec09d6f0 0x7facec1216a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.678+0000 7facf1d81700 1 --2- 172.21.15.3:0/1269843717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facec09e1d0 0x7facec121be0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.678+0000 7facf1d81700 1 -- 172.21.15.3:0/1269843717 >> 172.21.15.3:0/1269843717 conn(0x7facec066f50 msgr2=0x7facec09b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:06.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.678+0000 7facf1d81700 1 -- 172.21.15.3:0/1269843717 shutdown_connections 2023-10-03T10:17:06.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.678+0000 7facf1d81700 1 -- 172.21.15.3:0/1269843717 wait complete. 2023-10-03T10:17:06.692 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.716+0000 7fe91da13700 1 -- 172.21.15.3:0/1943195078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe918072980 con 0x7fe91809d5c0 2023-10-03T10:17:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.717+0000 7fe914ff9700 1 -- 172.21.15.3:0/1943195078 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe908011360 con 0x7fe91809d5c0 2023-10-03T10:17:06.717 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.719+0000 7fe91da13700 1 -- 172.21.15.3:0/1943195078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe91809d5c0 msgr2=0x7fe91811c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.719+0000 7fe91da13700 1 --2- 172.21.15.3:0/1943195078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe91809d5c0 0x7fe91811c7b0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fe90800c470 tx=0x7fe90800c780 comp rx=0 tx=0).stop 2023-10-03T10:17:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.719+0000 7fe91da13700 1 -- 172.21.15.3:0/1943195078 shutdown_connections 2023-10-03T10:17:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.719+0000 7fe91da13700 1 --2- 172.21.15.3:0/1943195078 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9180a14a0 0x7fe91811d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.719+0000 7fe91da13700 1 --2- 172.21.15.3:0/1943195078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe91809d5c0 0x7fe91811c7b0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.719+0000 7fe91da13700 1 --2- 172.21.15.3:0/1943195078 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9180a09c0 0x7fe91811ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.719+0000 7fe91da13700 1 -- 172.21.15.3:0/1943195078 >> 172.21.15.3:0/1943195078 conn(0x7fe9180a2fe0 msgr2=0x7fe918136d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.719+0000 7fe91da13700 1 -- 172.21.15.3:0/1943195078 shutdown_connections 2023-10-03T10:17:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.719+0000 7fe91da13700 1 -- 172.21.15.3:0/1943195078 wait complete. 2023-10-03T10:17:06.733 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:06.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.764+0000 7f7633308700 1 -- 172.21.15.3:0/528320742 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f762c074ea0 con 0x7f762c09e000 2023-10-03T10:17:06.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.765+0000 7f7621ffb700 1 -- 172.21.15.3:0/528320742 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f761c017030 con 0x7f762c09e000 2023-10-03T10:17:06.765 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:06.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.767+0000 7f7633308700 1 -- 172.21.15.3:0/528320742 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f762c09e000 msgr2=0x7f762c1a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:06.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.767+0000 7f7633308700 1 --2- 172.21.15.3:0/528320742 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f762c09e000 0x7f762c1a6320 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f761c00c580 tx=0x7f761c00c890 comp rx=0 tx=0).stop 2023-10-03T10:17:06.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.767+0000 7f7633308700 1 -- 172.21.15.3:0/528320742 shutdown_connections 2023-10-03T10:17:06.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.767+0000 7f7633308700 1 --2- 172.21.15.3:0/528320742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f762c09e410 0x7f762c1a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.767+0000 7f7633308700 1 --2- 172.21.15.3:0/528320742 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f762c09d6f0 0x7f762c1a5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.767+0000 7f7633308700 1 --2- 172.21.15.3:0/528320742 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f762c09e000 0x7f762c1a6320 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:06.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.767+0000 7f7633308700 1 -- 172.21.15.3:0/528320742 >> 172.21.15.3:0/528320742 conn(0x7f762c066f50 msgr2=0x7f762c1badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:06.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.767+0000 7f7633308700 1 -- 172.21.15.3:0/528320742 shutdown_connections 2023-10-03T10:17:06.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:06.767+0000 7f7633308700 1 -- 172.21.15.3:0/528320742 wait complete. 2023-10-03T10:17:06.781 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:07.399 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:07.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.545+0000 7fe1bb02f700 1 Processor -- start 2023-10-03T10:17:07.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.545+0000 7fe1bb02f700 1 -- start start 2023-10-03T10:17:07.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.546+0000 7fe1bb02f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1b406b250 0x7fe1b4136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:07.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.546+0000 7fe1bb02f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1b4136d80 0x7fe1b4139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:07.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.546+0000 7fe1bb02f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b409ece0 con 0x7fe1b406a770 2023-10-03T10:17:07.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.546+0000 7fe1bb02f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b406b640 con 0x7fe1b4136d80 2023-10-03T10:17:07.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.546+0000 7fe1bb02f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b409e760 con 0x7fe1b406b250 2023-10-03T10:17:07.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.546+0000 7fe1b95cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1b4136d80 0x7fe1b4139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:07.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.546+0000 7fe1b3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1b406b250 0x7fe1b4136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:07.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.546+0000 7fe1b95cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1b4136d80 0x7fe1b4139160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50354/0 (socket says 172.21.15.3:50354) 2023-10-03T10:17:07.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.546+0000 7fe1b3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1b406b250 0x7fe1b4136840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:49926/0 (socket says 172.21.15.3:49926) 2023-10-03T10:17:07.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.546+0000 7fe1b95cc700 1 -- 172.21.15.3:0/2424658650 learned_addr learned my addr 172.21.15.3:0/2424658650 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:07.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.546+0000 7fe1b95cc700 1 -- 172.21.15.3:0/2424658650 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1b406b250 msgr2=0x7fe1b4136840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:07.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.546+0000 7fe1b95cc700 1 --2- 172.21.15.3:0/2424658650 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1b406b250 0x7fe1b4136840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:07.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.546+0000 7fe1b37fe700 1 -- 172.21.15.3:0/2424658650 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1416968171 0 0) 0x7fe1b409ece0 con 0x7fe1b406a770 2023-10-03T10:17:07.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.546+0000 7fe1b95cc700 1 -- 172.21.15.3:0/2424658650 >> v1:172.21.15.3:6789/0 conn(0x7fe1b406a770 legacy=0x7fe1b406ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:07.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.546+0000 7fe1b95cc700 1 -- 172.21.15.3:0/2424658650 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b409e040 con 0x7fe1b4136d80 2023-10-03T10:17:07.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.546+0000 7fe1b3fff700 1 --2- 172.21.15.3:0/2424658650 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1b406b250 0x7fe1b4136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:07.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.546+0000 7fe1b95cc700 1 --2- 172.21.15.3:0/2424658650 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1b4136d80 0x7fe1b4139160 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009e80 tx=0x7fe1ac00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=deabd10149a1a3d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:07.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.547+0000 7fe1b37fe700 1 -- 172.21.15.3:0/2424658650 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac00e070 con 0x7fe1b4136d80 2023-10-03T10:17:07.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.547+0000 7fe1b37fe700 1 -- 172.21.15.3:0/2424658650 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ac00b910 con 0x7fe1b4136d80 2023-10-03T10:17:07.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.547+0000 7fe1b37fe700 1 -- 172.21.15.3:0/2424658650 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac00bcc0 con 0x7fe1b4136d80 2023-10-03T10:17:07.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.547+0000 7fe1bb02f700 1 -- 172.21.15.3:0/2424658650 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1b4136d80 msgr2=0x7fe1b4139160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:07.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.547+0000 7fe1bb02f700 1 --2- 172.21.15.3:0/2424658650 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1b4136d80 0x7fe1b4139160 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009e80 tx=0x7fe1ac00b330 comp rx=0 tx=0).stop 2023-10-03T10:17:07.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.547+0000 7fe1bb02f700 1 -- 172.21.15.3:0/2424658650 shutdown_connections 2023-10-03T10:17:07.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.547+0000 7fe1bb02f700 1 --2- 172.21.15.3:0/2424658650 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1b4136d80 0x7fe1b4139160 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:07.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.547+0000 7fe1bb02f700 1 --2- 172.21.15.3:0/2424658650 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1b406b250 0x7fe1b4136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:07.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.547+0000 7fe1bb02f700 1 -- 172.21.15.3:0/2424658650 >> 172.21.15.3:0/2424658650 conn(0x7fe1b4066f50 msgr2=0x7fe1b409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:07.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.547+0000 7fe1bb02f700 1 -- 172.21.15.3:0/2424658650 shutdown_connections 2023-10-03T10:17:07.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.547+0000 7fe1bb02f700 1 -- 172.21.15.3:0/2424658650 wait complete. 2023-10-03T10:17:07.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.549+0000 7fe1bb02f700 1 Processor -- start 2023-10-03T10:17:07.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.549+0000 7fe1bb02f700 1 -- start start 2023-10-03T10:17:07.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.550+0000 7fe1bb02f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1b406a770 0x7fe1b41c6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:07.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.550+0000 7fe1bb02f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1b406b250 0x7fe1b41c6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:07.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.550+0000 7fe1bb02f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1b4136d80 0x7fe1b411c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:07.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.550+0000 7fe1bb02f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b41bbf00 con 0x7fe1b406a770 2023-10-03T10:17:07.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.550+0000 7fe1bb02f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b4132380 con 0x7fe1b4136d80 2023-10-03T10:17:07.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.550+0000 7fe1bb02f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b40718d0 con 0x7fe1b406b250 2023-10-03T10:17:07.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.550+0000 7fe1b8dcb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1b406a770 0x7fe1b41c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:07.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.550+0000 7fe1b95cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1b4136d80 0x7fe1b411c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:07.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.550+0000 7fe1b95cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1b4136d80 0x7fe1b411c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50362/0 (socket says 172.21.15.3:50362) 2023-10-03T10:17:07.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.550+0000 7fe1b8dcb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1b406a770 0x7fe1b41c6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36984/0 (socket says 172.21.15.3:36984) 2023-10-03T10:17:07.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.550+0000 7fe1b95cc700 1 -- 172.21.15.3:0/2436461413 learned_addr learned my addr 172.21.15.3:0/2436461413 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:07.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.550+0000 7fe1b3fff700 1 --2- 172.21.15.3:0/2436461413 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1b406b250 0x7fe1b41c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:07.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.550+0000 7fe1b95cc700 1 -- 172.21.15.3:0/2436461413 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1b406b250 msgr2=0x7fe1b41c6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:07.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.550+0000 7fe1b95cc700 1 --2- 172.21.15.3:0/2436461413 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1b406b250 0x7fe1b41c6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:07.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.550+0000 7fe1b95cc700 1 -- 172.21.15.3:0/2436461413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1b406a770 msgr2=0x7fe1b41c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:07.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.550+0000 7fe1b95cc700 1 --2- 172.21.15.3:0/2436461413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1b406a770 0x7fe1b41c6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:07.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.550+0000 7fe1b95cc700 1 -- 172.21.15.3:0/2436461413 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe19c0094a0 con 0x7fe1b4136d80 2023-10-03T10:17:07.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.550+0000 7fe1b3fff700 1 --2- 172.21.15.3:0/2436461413 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1b406b250 0x7fe1b41c6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:07.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.550+0000 7fe1b8dcb700 1 --2- 172.21.15.3:0/2436461413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1b406a770 0x7fe1b41c6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:07.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.550+0000 7fe1b95cc700 1 --2- 172.21.15.3:0/2436461413 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1b4136d80 0x7fe1b411c4b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac00b680 tx=0x7fe1ac004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:07.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.551+0000 7fe1b17fa700 1 -- 172.21.15.3:0/2436461413 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac004240 con 0x7fe1b4136d80 2023-10-03T10:17:07.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.551+0000 7fe1b17fa700 1 -- 172.21.15.3:0/2436461413 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ac0043a0 con 0x7fe1b4136d80 2023-10-03T10:17:07.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.551+0000 7fe1bb02f700 1 -- 172.21.15.3:0/2436461413 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1ac009b30 con 0x7fe1b4136d80 2023-10-03T10:17:07.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.551+0000 7fe1b17fa700 1 -- 172.21.15.3:0/2436461413 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac004730 con 0x7fe1b4136d80 2023-10-03T10:17:07.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.551+0000 7fe1bb02f700 1 -- 172.21.15.3:0/2436461413 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b41c6d90 con 0x7fe1b4136d80 2023-10-03T10:17:07.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.551+0000 7fe1b17fa700 1 -- 172.21.15.3:0/2436461413 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe1ac01c410 con 0x7fe1b4136d80 2023-10-03T10:17:07.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.552+0000 7fe1b17fa700 1 -- 172.21.15.3:0/2436461413 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe1ac041000 con 0x7fe1b4136d80 2023-10-03T10:17:07.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.552+0000 7fe1bb02f700 1 -- 172.21.15.3:0/2436461413 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b4074ea0 con 0x7fe1b4136d80 2023-10-03T10:17:07.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.557+0000 7fe1b17fa700 1 -- 172.21.15.3:0/2436461413 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe1ac0245a0 con 0x7fe1b4136d80 2023-10-03T10:17:07.692 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:07.734 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:07.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.764+0000 7fe1bb02f700 1 -- 172.21.15.3:0/2436461413 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe1b409f6b0 con 0x7fe1b4136d80 2023-10-03T10:17:07.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.764+0000 7fe1b17fa700 1 -- 172.21.15.3:0/2436461413 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe1ac011390 con 0x7fe1b4136d80 2023-10-03T10:17:07.765 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:07.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.767+0000 7fe1bb02f700 1 -- 172.21.15.3:0/2436461413 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1b4136d80 msgr2=0x7fe1b411c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:07.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.767+0000 7fe1bb02f700 1 --2- 172.21.15.3:0/2436461413 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1b4136d80 0x7fe1b411c4b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac00b680 tx=0x7fe1ac004ad0 comp rx=0 tx=0).stop 2023-10-03T10:17:07.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.767+0000 7fe1bb02f700 1 -- 172.21.15.3:0/2436461413 shutdown_connections 2023-10-03T10:17:07.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.767+0000 7fe1bb02f700 1 --2- 172.21.15.3:0/2436461413 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1b4136d80 0x7fe1b411c4b0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:07.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.767+0000 7fe1bb02f700 1 --2- 172.21.15.3:0/2436461413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1b406a770 0x7fe1b41c6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:07.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.767+0000 7fe1bb02f700 1 --2- 172.21.15.3:0/2436461413 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1b406b250 0x7fe1b41c6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:07.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.767+0000 7fe1bb02f700 1 -- 172.21.15.3:0/2436461413 >> 172.21.15.3:0/2436461413 conn(0x7fe1b4066f50 msgr2=0x7fe1b40a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:07.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.767+0000 7fe1bb02f700 1 -- 172.21.15.3:0/2436461413 shutdown_connections 2023-10-03T10:17:07.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.767+0000 7fe1bb02f700 1 -- 172.21.15.3:0/2436461413 wait complete. 2023-10-03T10:17:07.782 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:07.782 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:07.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.834+0000 7f73b1ac6700 1 Processor -- start 2023-10-03T10:17:07.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.834+0000 7f73b1ac6700 1 -- start start 2023-10-03T10:17:07.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.835+0000 7f73b1ac6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73ac136830 0x7f73ac138c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:07.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.835+0000 7f73b1ac6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73ac139150 0x7f73ac13b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:07.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.835+0000 7f73b1ac6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73ac06f720 con 0x7f73ac1338c0 2023-10-03T10:17:07.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.835+0000 7f73b1ac6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac070c80 con 0x7f73ac139150 2023-10-03T10:17:07.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.835+0000 7f73b1ac6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac06fce0 con 0x7f73ac136830 2023-10-03T10:17:07.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.835+0000 7f73abfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73ac139150 0x7f73ac13b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:07.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.835+0000 7f73abfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73ac139150 0x7f73ac13b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50366/0 (socket says 172.21.15.3:50366) 2023-10-03T10:17:07.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.835+0000 7f73abfff700 1 -- 172.21.15.3:0/667912185 learned_addr learned my addr 172.21.15.3:0/667912185 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:07.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.835+0000 7f73aaffd700 1 --2- 172.21.15.3:0/667912185 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73ac136830 0x7f73ac138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:07.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.835+0000 7f73abfff700 1 -- 172.21.15.3:0/667912185 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73ac136830 msgr2=0x7f73ac138c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:07.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.835+0000 7f73abfff700 1 --2- 172.21.15.3:0/667912185 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73ac136830 0x7f73ac138c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:07.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.835+0000 7f73abfff700 1 -- 172.21.15.3:0/667912185 >> v1:172.21.15.3:6789/0 conn(0x7f73ac1338c0 legacy=0x7f73ac133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:07.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.835+0000 7f73abfff700 1 -- 172.21.15.3:0/667912185 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ac1336a0 con 0x7f73ac139150 2023-10-03T10:17:07.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.836+0000 7f73abfff700 1 --2- 172.21.15.3:0/667912185 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73ac139150 0x7f73ac13b530 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009e80 tx=0x7f73a400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43d02a129fb4ed01 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:07.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.836+0000 7f73aa7fc700 1 -- 172.21.15.3:0/667912185 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a400e070 con 0x7f73ac139150 2023-10-03T10:17:07.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.836+0000 7f73aa7fc700 1 -- 172.21.15.3:0/667912185 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a400b910 con 0x7f73ac139150 2023-10-03T10:17:07.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.836+0000 7f73aa7fc700 1 -- 172.21.15.3:0/667912185 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a400bcc0 con 0x7f73ac139150 2023-10-03T10:17:07.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.837+0000 7f73b1ac6700 1 -- 172.21.15.3:0/667912185 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73ac139150 msgr2=0x7f73ac13b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:07.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.837+0000 7f73b1ac6700 1 --2- 172.21.15.3:0/667912185 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73ac139150 0x7f73ac13b530 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009e80 tx=0x7f73a400b330 comp rx=0 tx=0).stop 2023-10-03T10:17:07.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.837+0000 7f73b1ac6700 1 -- 172.21.15.3:0/667912185 shutdown_connections 2023-10-03T10:17:07.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.837+0000 7f73b1ac6700 1 --2- 172.21.15.3:0/667912185 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73ac139150 0x7f73ac13b530 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:07.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.837+0000 7f73b1ac6700 1 --2- 172.21.15.3:0/667912185 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73ac136830 0x7f73ac138c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:07.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.837+0000 7f73b1ac6700 1 -- 172.21.15.3:0/667912185 >> 172.21.15.3:0/667912185 conn(0x7f73ac0a2aa0 msgr2=0x7f73ac0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:07.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.837+0000 7f73b1ac6700 1 -- 172.21.15.3:0/667912185 shutdown_connections 2023-10-03T10:17:07.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.837+0000 7f73b1ac6700 1 -- 172.21.15.3:0/667912185 wait complete. 2023-10-03T10:17:07.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.838+0000 7f73b1ac6700 1 Processor -- start 2023-10-03T10:17:07.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.838+0000 7f73b1ac6700 1 -- start start 2023-10-03T10:17:07.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.839+0000 7f73b1ac6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73ac1338c0 0x7f73ac119e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:07.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.839+0000 7f73b1ac6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73ac136830 0x7f73ac11a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:07.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.839+0000 7f73b1ac6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73ac139150 0x7f73ac11a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:07.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.839+0000 7f73b1ac6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac136110 con 0x7f73ac1338c0 2023-10-03T10:17:07.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.839+0000 7f73b1ac6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac11bb80 con 0x7f73ac139150 2023-10-03T10:17:07.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.839+0000 7f73b1ac6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac11d3e0 con 0x7f73ac136830 2023-10-03T10:17:07.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.839+0000 7f73aaffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73ac136830 0x7f73ac11a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:07.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.839+0000 7f73ab7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73ac1338c0 0x7f73ac119e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:07.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.839+0000 7f73aaffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73ac136830 0x7f73ac11a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:49956/0 (socket says 172.21.15.3:49956) 2023-10-03T10:17:07.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.839+0000 7f73aaffd700 1 -- 172.21.15.3:0/2553058307 learned_addr learned my addr 172.21.15.3:0/2553058307 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:07.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.839+0000 7f73abfff700 1 --2- 172.21.15.3:0/2553058307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73ac139150 0x7f73ac11a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:07.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.840+0000 7f73aaffd700 1 -- 172.21.15.3:0/2553058307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73ac139150 msgr2=0x7f73ac11a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:07.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.840+0000 7f73aaffd700 1 --2- 172.21.15.3:0/2553058307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73ac139150 0x7f73ac11a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:07.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.840+0000 7f73aaffd700 1 -- 172.21.15.3:0/2553058307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73ac1338c0 msgr2=0x7f73ac119e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:07.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.840+0000 7f73aaffd700 1 --2- 172.21.15.3:0/2553058307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73ac1338c0 0x7f73ac119e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:07.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.840+0000 7f73aaffd700 1 -- 172.21.15.3:0/2553058307 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a4009b30 con 0x7f73ac136830 2023-10-03T10:17:07.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.840+0000 7f73ab7fe700 1 --2- 172.21.15.3:0/2553058307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73ac1338c0 0x7f73ac119e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:07.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.840+0000 7f73abfff700 1 --2- 172.21.15.3:0/2553058307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73ac139150 0x7f73ac11a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:07.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.840+0000 7f73aaffd700 1 --2- 172.21.15.3:0/2553058307 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73ac136830 0x7f73ac11a3a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f73a000e980 tx=0x7f73a000ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:07.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.840+0000 7f7393fff700 1 -- 172.21.15.3:0/2553058307 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a0009710 con 0x7f73ac136830 2023-10-03T10:17:07.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.840+0000 7f7393fff700 1 -- 172.21.15.3:0/2553058307 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a000c690 con 0x7f73ac136830 2023-10-03T10:17:07.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.840+0000 7f73b1ac6700 1 -- 172.21.15.3:0/2553058307 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73ac11d6a0 con 0x7f73ac136830 2023-10-03T10:17:07.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.840+0000 7f7393fff700 1 -- 172.21.15.3:0/2553058307 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a000ca40 con 0x7f73ac136830 2023-10-03T10:17:07.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.840+0000 7f73b1ac6700 1 -- 172.21.15.3:0/2553058307 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73ac11bfc0 con 0x7f73ac136830 2023-10-03T10:17:07.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.841+0000 7f7393fff700 1 -- 172.21.15.3:0/2553058307 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f73a000cba0 con 0x7f73ac136830 2023-10-03T10:17:07.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.841+0000 7f73b1ac6700 1 -- 172.21.15.3:0/2553058307 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73ac074ea0 con 0x7f73ac136830 2023-10-03T10:17:07.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.841+0000 7f7393fff700 1 -- 172.21.15.3:0/2553058307 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f73a003f6f0 con 0x7f73ac136830 2023-10-03T10:17:07.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.847+0000 7f7393fff700 1 -- 172.21.15.3:0/2553058307 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f73a0021080 con 0x7f73ac136830 2023-10-03T10:17:07.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.878+0000 7fcd7a39c700 1 Processor -- start 2023-10-03T10:17:07.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.878+0000 7fcd7a39c700 1 -- start start 2023-10-03T10:17:07.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.878+0000 7fcd7a39c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcd74133420 0x7fcd741337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:07.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.878+0000 7fcd7a39c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd74133e30 0x7fcd7412ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:07.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.878+0000 7fcd7a39c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd7406b990 con 0x7fcd74134260 2023-10-03T10:17:07.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.878+0000 7fcd7a39c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd740a0590 con 0x7fcd74133420 2023-10-03T10:17:07.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.878+0000 7fcd7a39c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7409fb80 con 0x7fcd74133e30 2023-10-03T10:17:07.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.878+0000 7fcd73fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcd74133420 0x7fcd741337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:07.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.878+0000 7fcd73fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcd74133420 0x7fcd741337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36994/0 (socket says 172.21.15.3:36994) 2023-10-03T10:17:07.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.878+0000 7fcd73fff700 1 -- 172.21.15.3:0/1323896552 learned_addr learned my addr 172.21.15.3:0/1323896552 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:07.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.878+0000 7fcd737fe700 1 --2- 172.21.15.3:0/1323896552 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd74133e30 0x7fcd7412ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:07.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.879+0000 7fcd737fe700 1 -- 172.21.15.3:0/1323896552 >> v2:172.21.15.3:3300/0 conn(0x7fcd74133420 msgr2=0x7fcd741337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:07.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.879+0000 7fcd737fe700 1 --2- 172.21.15.3:0/1323896552 >> v2:172.21.15.3:3300/0 conn(0x7fcd74133420 0x7fcd741337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:07.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.879+0000 7fcd737fe700 1 -- 172.21.15.3:0/1323896552 >> v1:172.21.15.3:6789/0 conn(0x7fcd74134260 legacy=0x7fcd741304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:07.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.879+0000 7fcd737fe700 1 -- 172.21.15.3:0/1323896552 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd740a0030 con 0x7fcd74133e30 2023-10-03T10:17:07.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.879+0000 7fcd73fff700 1 --2- 172.21.15.3:0/1323896552 >> v2:172.21.15.3:3300/0 conn(0x7fcd74133420 0x7fcd741337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:07.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.879+0000 7fcd737fe700 1 --2- 172.21.15.3:0/1323896552 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd74133e30 0x7fcd7412ff90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009840 tx=0x7fcd68009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a6177b1c06d6ea3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:07.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.879+0000 7fcd72ffd700 1 -- 172.21.15.3:0/1323896552 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68011830 con 0x7fcd74133e30 2023-10-03T10:17:07.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.879+0000 7fcd72ffd700 1 -- 172.21.15.3:0/1323896552 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd68011990 con 0x7fcd74133e30 2023-10-03T10:17:07.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.879+0000 7fcd72ffd700 1 -- 172.21.15.3:0/1323896552 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68011d20 con 0x7fcd74133e30 2023-10-03T10:17:07.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.880+0000 7fcd7a39c700 1 -- 172.21.15.3:0/1323896552 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd74133e30 msgr2=0x7fcd7412ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:07.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.880+0000 7fcd7a39c700 1 --2- 172.21.15.3:0/1323896552 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd74133e30 0x7fcd7412ff90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009840 tx=0x7fcd68009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:07.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.880+0000 7fcd7a39c700 1 -- 172.21.15.3:0/1323896552 shutdown_connections 2023-10-03T10:17:07.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.880+0000 7fcd7a39c700 1 --2- 172.21.15.3:0/1323896552 >> v2:172.21.15.3:3300/0 conn(0x7fcd74133420 0x7fcd741337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:07.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.880+0000 7fcd7a39c700 1 --2- 172.21.15.3:0/1323896552 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd74133e30 0x7fcd7412ff90 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:07.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.880+0000 7fcd7a39c700 1 -- 172.21.15.3:0/1323896552 >> 172.21.15.3:0/1323896552 conn(0x7fcd740a4210 msgr2=0x7fcd740a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:07.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.880+0000 7fcd7a39c700 1 -- 172.21.15.3:0/1323896552 shutdown_connections 2023-10-03T10:17:07.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.880+0000 7fcd7a39c700 1 -- 172.21.15.3:0/1323896552 wait complete. 2023-10-03T10:17:07.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.881+0000 7fcd7a39c700 1 Processor -- start 2023-10-03T10:17:07.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.881+0000 7fcd7a39c700 1 -- start start 2023-10-03T10:17:07.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.882+0000 7fcd7a39c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd74133420 0x7fcd7409c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:07.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.882+0000 7fcd7a39c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd74133e30 0x7fcd7409cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:07.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.882+0000 7fcd7a39c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd74134260 0x7fcd7409d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:07.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.882+0000 7fcd7a39c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7411ffb0 con 0x7fcd74133e30 2023-10-03T10:17:07.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.882+0000 7fcd7a39c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd741203e0 con 0x7fcd74134260 2023-10-03T10:17:07.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.882+0000 7fcd7a39c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7409dbb0 con 0x7fcd74133420 2023-10-03T10:17:07.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.882+0000 7fcd73fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd74133420 0x7fcd7409c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:07.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.882+0000 7fcd78939700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd74134260 0x7fcd7409d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:07.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.882+0000 7fcd73fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd74133420 0x7fcd7409c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:49978/0 (socket says 172.21.15.3:49978) 2023-10-03T10:17:07.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.882+0000 7fcd73fff700 1 -- 172.21.15.3:0/2037590370 learned_addr learned my addr 172.21.15.3:0/2037590370 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:07.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.882+0000 7fcd737fe700 1 --2- 172.21.15.3:0/2037590370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd74133e30 0x7fcd7409cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:07.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.882+0000 7fcd73fff700 1 -- 172.21.15.3:0/2037590370 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd74134260 msgr2=0x7fcd7409d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:07.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.882+0000 7fcd73fff700 1 --2- 172.21.15.3:0/2037590370 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd74134260 0x7fcd7409d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:07.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.882+0000 7fcd73fff700 1 -- 172.21.15.3:0/2037590370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd74133e30 msgr2=0x7fcd7409cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:07.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.882+0000 7fcd73fff700 1 --2- 172.21.15.3:0/2037590370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd74133e30 0x7fcd7409cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:07.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.882+0000 7fcd73fff700 1 -- 172.21.15.3:0/2037590370 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd64009550 con 0x7fcd74133420 2023-10-03T10:17:07.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.883+0000 7fcd73fff700 1 --2- 172.21.15.3:0/2037590370 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd74133420 0x7fcd7409c6e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fcd6400ebe0 tx=0x7fcd6400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:07.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.883+0000 7fcd70ff9700 1 -- 172.21.15.3:0/2037590370 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6400c850 con 0x7fcd74133420 2023-10-03T10:17:07.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.883+0000 7fcd70ff9700 1 -- 172.21.15.3:0/2037590370 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6400c9b0 con 0x7fcd74133420 2023-10-03T10:17:07.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.883+0000 7fcd7a39c700 1 -- 172.21.15.3:0/2037590370 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd680094a0 con 0x7fcd74133420 2023-10-03T10:17:07.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.883+0000 7fcd70ff9700 1 -- 172.21.15.3:0/2037590370 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6400cd20 con 0x7fcd74133420 2023-10-03T10:17:07.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.883+0000 7fcd7a39c700 1 -- 172.21.15.3:0/2037590370 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7412dba0 con 0x7fcd74133420 2023-10-03T10:17:07.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.884+0000 7fcd70ff9700 1 -- 172.21.15.3:0/2037590370 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcd64013470 con 0x7fcd74133420 2023-10-03T10:17:07.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.884+0000 7fcd7a39c700 1 -- 172.21.15.3:0/2037590370 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd74074ea0 con 0x7fcd74133420 2023-10-03T10:17:07.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.884+0000 7fcd70ff9700 1 -- 172.21.15.3:0/2037590370 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fcd6403fbf0 con 0x7fcd74133420 2023-10-03T10:17:07.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.889+0000 7fcd70ff9700 1 -- 172.21.15.3:0/2037590370 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcd6401b3b0 con 0x7fcd74133420 2023-10-03T10:17:07.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.966+0000 7f2ed1988700 1 Processor -- start 2023-10-03T10:17:07.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.966+0000 7f2ed1988700 1 -- start start 2023-10-03T10:17:07.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.966+0000 7f2ed1988700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2ecc0a09c0 0x7f2ecc0a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.966+0000 7f2ed1988700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ecc09d5c0 0x7f2ecc0a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.966+0000 7f2ed1988700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ecc0a1810 con 0x7f2ecc133920 2023-10-03T10:17:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.966+0000 7f2ed1988700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ecc09ea10 con 0x7f2ecc0a09c0 2023-10-03T10:17:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.966+0000 7f2ed1988700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc09e210 con 0x7f2ecc09d5c0 2023-10-03T10:17:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.966+0000 7f2ec3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ecc09d5c0 0x7f2ecc0a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.966+0000 7f2ecaffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2ecc0a09c0 0x7f2ecc0a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.966+0000 7f2ecaffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2ecc0a09c0 0x7f2ecc0a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37000/0 (socket says 172.21.15.3:37000) 2023-10-03T10:17:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.966+0000 7f2ec3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ecc09d5c0 0x7f2ecc0a12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:49986/0 (socket says 172.21.15.3:49986) 2023-10-03T10:17:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.966+0000 7f2ecaffd700 1 -- 172.21.15.3:0/2908185882 learned_addr learned my addr 172.21.15.3:0/2908185882 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.967+0000 7f2ecaffd700 1 -- 172.21.15.3:0/2908185882 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ecc09d5c0 msgr2=0x7f2ecc0a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.967+0000 7f2ecaffd700 1 --2- 172.21.15.3:0/2908185882 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ecc09d5c0 0x7f2ecc0a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:07.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.967+0000 7f2ecaffd700 1 -- 172.21.15.3:0/2908185882 >> v1:172.21.15.3:6789/0 conn(0x7f2ecc133920 legacy=0x7f2ecc133d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:07.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.967+0000 7f2ecaffd700 1 -- 172.21.15.3:0/2908185882 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ecc0a26f0 con 0x7f2ecc0a09c0 2023-10-03T10:17:07.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.967+0000 7f2ec3fff700 1 --2- 172.21.15.3:0/2908185882 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ecc09d5c0 0x7f2ecc0a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:07.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.967+0000 7f2ecaffd700 1 --2- 172.21.15.3:0/2908185882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ecc0a09c0 0x7f2ecc0a0d90 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0098f0 tx=0x7f2ebc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d19b9867387830 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:07.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.967+0000 7f2eca7fc700 1 -- 172.21.15.3:0/2908185882 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc011830 con 0x7f2ecc0a09c0 2023-10-03T10:17:07.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.967+0000 7f2eca7fc700 1 -- 172.21.15.3:0/2908185882 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ebc011990 con 0x7f2ecc0a09c0 2023-10-03T10:17:07.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.967+0000 7f2eca7fc700 1 -- 172.21.15.3:0/2908185882 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc011d20 con 0x7f2ecc0a09c0 2023-10-03T10:17:07.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.968+0000 7f2ed1988700 1 -- 172.21.15.3:0/2908185882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ecc0a09c0 msgr2=0x7f2ecc0a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:07.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.968+0000 7f2ed1988700 1 --2- 172.21.15.3:0/2908185882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ecc0a09c0 0x7f2ecc0a0d90 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0098f0 tx=0x7f2ebc009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:07.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.968+0000 7f2ed1988700 1 -- 172.21.15.3:0/2908185882 shutdown_connections 2023-10-03T10:17:07.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.968+0000 7f2ed1988700 1 --2- 172.21.15.3:0/2908185882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ecc0a09c0 0x7f2ecc0a0d90 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:07.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.968+0000 7f2ed1988700 1 --2- 172.21.15.3:0/2908185882 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ecc09d5c0 0x7f2ecc0a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:07.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.968+0000 7f2ed1988700 1 -- 172.21.15.3:0/2908185882 >> 172.21.15.3:0/2908185882 conn(0x7f2ecc0a2fe0 msgr2=0x7f2ecc0a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:07.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.968+0000 7f2ed1988700 1 -- 172.21.15.3:0/2908185882 shutdown_connections 2023-10-03T10:17:07.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.968+0000 7f2ed1988700 1 -- 172.21.15.3:0/2908185882 wait complete. 2023-10-03T10:17:07.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.969+0000 7f2ed1988700 1 Processor -- start 2023-10-03T10:17:07.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.969+0000 7f2ed1988700 1 -- start start 2023-10-03T10:17:07.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.970+0000 7f2ed1988700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ecc09d5c0 0x7f2ecc121150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.970+0000 7f2ed1988700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ecc0a09c0 0x7f2ecc121690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.970+0000 7f2ed1988700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ecc133920 0x7f2ecc121bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.970+0000 7f2ed1988700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc123b20 con 0x7f2ecc133920 2023-10-03T10:17:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.970+0000 7f2ed1988700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc122e00 con 0x7f2ecc0a09c0 2023-10-03T10:17:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.970+0000 7f2ed1988700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc1245f0 con 0x7f2ecc09d5c0 2023-10-03T10:17:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.970+0000 7f2ecaffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ecc09d5c0 0x7f2ecc121150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.970+0000 7f2ecaffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ecc09d5c0 0x7f2ecc121150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:49998/0 (socket says 172.21.15.3:49998) 2023-10-03T10:17:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.970+0000 7f2ecaffd700 1 -- 172.21.15.3:0/2276329098 learned_addr learned my addr 172.21.15.3:0/2276329098 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.970+0000 7f2ecb7fe700 1 --2- 172.21.15.3:0/2276329098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ecc133920 0x7f2ecc121bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.970+0000 7f2ec3fff700 1 --2- 172.21.15.3:0/2276329098 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ecc0a09c0 0x7f2ecc121690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.970+0000 7f2ecaffd700 1 -- 172.21.15.3:0/2276329098 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ecc0a09c0 msgr2=0x7f2ecc121690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.970+0000 7f2ecaffd700 1 --2- 172.21.15.3:0/2276329098 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ecc0a09c0 0x7f2ecc121690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.970+0000 7f2ecaffd700 1 -- 172.21.15.3:0/2276329098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ecc133920 msgr2=0x7f2ecc121bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.970+0000 7f2ecaffd700 1 --2- 172.21.15.3:0/2276329098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ecc133920 0x7f2ecc121bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.970+0000 7f2ecaffd700 1 -- 172.21.15.3:0/2276329098 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb40094a0 con 0x7f2ecc09d5c0 2023-10-03T10:17:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.970+0000 7f2ecaffd700 1 --2- 172.21.15.3:0/2276329098 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ecc09d5c0 0x7f2ecc121150 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc000c00 tx=0x7f2ebc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.971+0000 7f2ec8ff9700 1 -- 172.21.15.3:0/2276329098 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc0036a0 con 0x7f2ecc09d5c0 2023-10-03T10:17:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.971+0000 7f2ec8ff9700 1 -- 172.21.15.3:0/2276329098 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ebc003800 con 0x7f2ecc09d5c0 2023-10-03T10:17:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.971+0000 7f2ec8ff9700 1 -- 172.21.15.3:0/2276329098 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc003bd0 con 0x7f2ecc09d5c0 2023-10-03T10:17:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.971+0000 7f2ed1988700 1 -- 172.21.15.3:0/2276329098 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ebc009550 con 0x7f2ecc09d5c0 2023-10-03T10:17:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.971+0000 7f2ed1988700 1 -- 172.21.15.3:0/2276329098 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ecc09cec0 con 0x7f2ecc09d5c0 2023-10-03T10:17:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.972+0000 7f2ed1988700 1 -- 172.21.15.3:0/2276329098 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ecc11df50 con 0x7f2ecc09d5c0 2023-10-03T10:17:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.972+0000 7f2ec8ff9700 1 -- 172.21.15.3:0/2276329098 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2ebc021070 con 0x7f2ecc09d5c0 2023-10-03T10:17:07.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.972+0000 7f2ec8ff9700 1 -- 172.21.15.3:0/2276329098 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2ebc040c60 con 0x7f2ecc09d5c0 2023-10-03T10:17:07.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:07.977+0000 7f2ec8ff9700 1 -- 172.21.15.3:0/2276329098 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2ebc00bc60 con 0x7f2ecc09d5c0 2023-10-03T10:17:08.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.057+0000 7f73b1ac6700 1 -- 172.21.15.3:0/2553058307 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f73ac06b9e0 con 0x7f73ac136830 2023-10-03T10:17:08.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.057+0000 7f7393fff700 1 -- 172.21.15.3:0/2553058307 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f73a0017020 con 0x7f73ac136830 2023-10-03T10:17:08.058 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:08.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.061+0000 7f73b1ac6700 1 -- 172.21.15.3:0/2553058307 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73ac136830 msgr2=0x7f73ac11a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:08.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.061+0000 7f73b1ac6700 1 --2- 172.21.15.3:0/2553058307 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73ac136830 0x7f73ac11a3a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f73a000e980 tx=0x7f73a000ec90 comp rx=0 tx=0).stop 2023-10-03T10:17:08.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.061+0000 7f73b1ac6700 1 -- 172.21.15.3:0/2553058307 shutdown_connections 2023-10-03T10:17:08.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.061+0000 7f73b1ac6700 1 --2- 172.21.15.3:0/2553058307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f73ac139150 0x7f73ac11a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:08.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.061+0000 7f73b1ac6700 1 --2- 172.21.15.3:0/2553058307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73ac1338c0 0x7f73ac119e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:08.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.061+0000 7f73b1ac6700 1 --2- 172.21.15.3:0/2553058307 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f73ac136830 0x7f73ac11a3a0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:08.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.061+0000 7f73b1ac6700 1 -- 172.21.15.3:0/2553058307 >> 172.21.15.3:0/2553058307 conn(0x7f73ac0a2aa0 msgr2=0x7f73ac134610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:08.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.061+0000 7f73b1ac6700 1 -- 172.21.15.3:0/2553058307 shutdown_connections 2023-10-03T10:17:08.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.061+0000 7f73b1ac6700 1 -- 172.21.15.3:0/2553058307 wait complete. 2023-10-03T10:17:08.075 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:08.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.113+0000 7fcd7a39c700 1 -- 172.21.15.3:0/2037590370 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fcd741201d0 con 0x7fcd74133420 2023-10-03T10:17:08.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.113+0000 7fcd70ff9700 1 -- 172.21.15.3:0/2037590370 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcd64011b00 con 0x7fcd74133420 2023-10-03T10:17:08.113 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:08.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.115+0000 7fcd7a39c700 1 -- 172.21.15.3:0/2037590370 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd74133420 msgr2=0x7fcd7409c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:08.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.115+0000 7fcd7a39c700 1 --2- 172.21.15.3:0/2037590370 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd74133420 0x7fcd7409c6e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fcd6400ebe0 tx=0x7fcd6400eef0 comp rx=0 tx=0).stop 2023-10-03T10:17:08.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.115+0000 7fcd7a39c700 1 -- 172.21.15.3:0/2037590370 shutdown_connections 2023-10-03T10:17:08.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.115+0000 7fcd7a39c700 1 --2- 172.21.15.3:0/2037590370 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd74134260 0x7fcd7409d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:08.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.115+0000 7fcd7a39c700 1 --2- 172.21.15.3:0/2037590370 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd74133420 0x7fcd7409c6e0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:08.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.115+0000 7fcd7a39c700 1 --2- 172.21.15.3:0/2037590370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd74133e30 0x7fcd7409cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:08.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.115+0000 7fcd7a39c700 1 -- 172.21.15.3:0/2037590370 >> 172.21.15.3:0/2037590370 conn(0x7fcd740a4210 msgr2=0x7fcd740a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:08.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.116+0000 7fcd7a39c700 1 -- 172.21.15.3:0/2037590370 shutdown_connections 2023-10-03T10:17:08.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.116+0000 7fcd7a39c700 1 -- 172.21.15.3:0/2037590370 wait complete. 2023-10-03T10:17:08.129 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:08.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.200+0000 7f2ed1988700 1 -- 172.21.15.3:0/2276329098 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2ecc124cb0 con 0x7f2ecc09d5c0 2023-10-03T10:17:08.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.200+0000 7f2ec8ff9700 1 -- 172.21.15.3:0/2276329098 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2ebc029030 con 0x7f2ecc09d5c0 2023-10-03T10:17:08.200 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:08.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.202+0000 7f2ed1988700 1 -- 172.21.15.3:0/2276329098 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ecc09d5c0 msgr2=0x7f2ecc121150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:08.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.202+0000 7f2ed1988700 1 --2- 172.21.15.3:0/2276329098 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ecc09d5c0 0x7f2ecc121150 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc000c00 tx=0x7f2ebc004750 comp rx=0 tx=0).stop 2023-10-03T10:17:08.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.202+0000 7f2ed1988700 1 -- 172.21.15.3:0/2276329098 shutdown_connections 2023-10-03T10:17:08.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.202+0000 7f2ed1988700 1 --2- 172.21.15.3:0/2276329098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ecc133920 0x7f2ecc121bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:08.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.202+0000 7f2ed1988700 1 --2- 172.21.15.3:0/2276329098 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ecc09d5c0 0x7f2ecc121150 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:08.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.202+0000 7f2ed1988700 1 --2- 172.21.15.3:0/2276329098 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ecc0a09c0 0x7f2ecc121690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:08.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.202+0000 7f2ed1988700 1 -- 172.21.15.3:0/2276329098 >> 172.21.15.3:0/2276329098 conn(0x7f2ecc0a2fe0 msgr2=0x7f2ecc133200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:08.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.203+0000 7f2ed1988700 1 -- 172.21.15.3:0/2276329098 shutdown_connections 2023-10-03T10:17:08.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.203+0000 7f2ed1988700 1 -- 172.21.15.3:0/2276329098 wait complete. 2023-10-03T10:17:08.216 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:08.783 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:08.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.929+0000 7fa22eb5b700 1 Processor -- start 2023-10-03T10:17:08.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.929+0000 7fa22eb5b700 1 -- start start 2023-10-03T10:17:08.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.929+0000 7fa22eb5b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa22809e930 0x7fa22809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:08.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.929+0000 7fa22eb5b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa22809f240 0x7fa228133be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:08.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.929+0000 7fa22eb5b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa22806c9f0 con 0x7fa228134120 2023-10-03T10:17:08.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.929+0000 7fa22eb5b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa22806c670 con 0x7fa22809f240 2023-10-03T10:17:08.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.929+0000 7fa22eb5b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa22806bc90 con 0x7fa22809e930 2023-10-03T10:17:08.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.929+0000 7fa22c8f7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa22809e930 0x7fa22809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:08.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.929+0000 7fa227fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa22809f240 0x7fa228133be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:08.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.929+0000 7fa22c8f7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa22809e930 0x7fa22809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50392/0 (socket says 172.21.15.3:50392) 2023-10-03T10:17:08.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.929+0000 7fa227fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa22809f240 0x7fa228133be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37030/0 (socket says 172.21.15.3:37030) 2023-10-03T10:17:08.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.929+0000 7fa22c8f7700 1 -- 172.21.15.3:0/3411113863 learned_addr learned my addr 172.21.15.3:0/3411113863 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:08.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.929+0000 7fa22d0f8700 1 --1- 172.21.15.3:0/3411113863 >> v1:172.21.15.3:6789/0 conn(0x7fa228134120 0x7fa22813ae30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:46030/0 (socket says 172.21.15.3:46030) 2023-10-03T10:17:08.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.930+0000 7fa227fff700 1 -- 172.21.15.3:0/3411113863 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa22809e930 msgr2=0x7fa22809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:08.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.930+0000 7fa227fff700 1 --2- 172.21.15.3:0/3411113863 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa22809e930 0x7fa22809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:08.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.930+0000 7fa227fff700 1 -- 172.21.15.3:0/3411113863 >> v1:172.21.15.3:6789/0 conn(0x7fa228134120 legacy=0x7fa22813ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:08.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.930+0000 7fa227fff700 1 -- 172.21.15.3:0/3411113863 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22806d860 con 0x7fa22809f240 2023-10-03T10:17:08.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.930+0000 7fa22c8f7700 1 --2- 172.21.15.3:0/3411113863 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa22809e930 0x7fa22809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:08.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.930+0000 7fa2277fe700 1 -- 172.21.15.3:0/3411113863 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4115356913 0 0) 0x7fa220001490 con 0x7fa228134120 2023-10-03T10:17:08.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.930+0000 7fa227fff700 1 --2- 172.21.15.3:0/3411113863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa22809f240 0x7fa228133be0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fa21c009840 tx=0x7fa21c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d38b3cef67c4db server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:08.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.930+0000 7fa2277fe700 1 -- 172.21.15.3:0/3411113863 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c011830 con 0x7fa22809f240 2023-10-03T10:17:08.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.930+0000 7fa2277fe700 1 -- 172.21.15.3:0/3411113863 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21c011990 con 0x7fa22809f240 2023-10-03T10:17:08.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.930+0000 7fa2277fe700 1 -- 172.21.15.3:0/3411113863 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c011d20 con 0x7fa22809f240 2023-10-03T10:17:08.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.932+0000 7fa22eb5b700 1 -- 172.21.15.3:0/3411113863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa22809f240 msgr2=0x7fa228133be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:08.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.932+0000 7fa22eb5b700 1 --2- 172.21.15.3:0/3411113863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa22809f240 0x7fa228133be0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fa21c009840 tx=0x7fa21c009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:08.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.932+0000 7fa22eb5b700 1 -- 172.21.15.3:0/3411113863 shutdown_connections 2023-10-03T10:17:08.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.932+0000 7fa22eb5b700 1 --2- 172.21.15.3:0/3411113863 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa22809e930 0x7fa22809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:08.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.932+0000 7fa22eb5b700 1 --2- 172.21.15.3:0/3411113863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa22809f240 0x7fa228133be0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:08.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.932+0000 7fa22eb5b700 1 -- 172.21.15.3:0/3411113863 >> 172.21.15.3:0/3411113863 conn(0x7fa2280a4210 msgr2=0x7fa2280a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:08.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.932+0000 7fa22eb5b700 1 -- 172.21.15.3:0/3411113863 shutdown_connections 2023-10-03T10:17:08.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.932+0000 7fa22eb5b700 1 -- 172.21.15.3:0/3411113863 wait complete. 2023-10-03T10:17:08.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.933+0000 7fa22eb5b700 1 Processor -- start 2023-10-03T10:17:08.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.933+0000 7fa22eb5b700 1 -- start start 2023-10-03T10:17:08.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.934+0000 7fa22eb5b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa22809e930 0x7fa228122960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:08.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.934+0000 7fa22eb5b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa22809f240 0x7fa228122ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:08.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.934+0000 7fa22eb5b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa228134120 0x7fa2281233e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:08.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.934+0000 7fa22eb5b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa228126480 con 0x7fa22809e930 2023-10-03T10:17:08.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.934+0000 7fa22eb5b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa228125ee0 con 0x7fa228134120 2023-10-03T10:17:08.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.934+0000 7fa22eb5b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa228124610 con 0x7fa22809f240 2023-10-03T10:17:08.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.934+0000 7fa22d0f8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa228134120 0x7fa2281233e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:08.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.934+0000 7fa22c8f7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa22809e930 0x7fa228122960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:08.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.934+0000 7fa22d0f8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa228134120 0x7fa2281233e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50406/0 (socket says 172.21.15.3:50406) 2023-10-03T10:17:08.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.934+0000 7fa22c8f7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa22809e930 0x7fa228122960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37042/0 (socket says 172.21.15.3:37042) 2023-10-03T10:17:08.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.934+0000 7fa22d0f8700 1 -- 172.21.15.3:0/2639780240 learned_addr learned my addr 172.21.15.3:0/2639780240 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:08.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.934+0000 7fa227fff700 1 --2- 172.21.15.3:0/2639780240 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa22809f240 0x7fa228122ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:08.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.934+0000 7fa22c8f7700 1 -- 172.21.15.3:0/2639780240 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa22809f240 msgr2=0x7fa228122ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:08.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.934+0000 7fa22c8f7700 1 --2- 172.21.15.3:0/2639780240 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa22809f240 0x7fa228122ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:08.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.934+0000 7fa22c8f7700 1 -- 172.21.15.3:0/2639780240 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa228134120 msgr2=0x7fa2281233e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:08.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.934+0000 7fa22c8f7700 1 --2- 172.21.15.3:0/2639780240 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa228134120 0x7fa2281233e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:08.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.934+0000 7fa22c8f7700 1 -- 172.21.15.3:0/2639780240 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa218009550 con 0x7fa22809e930 2023-10-03T10:17:08.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.934+0000 7fa22d0f8700 1 --2- 172.21.15.3:0/2639780240 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa228134120 0x7fa2281233e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:08.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.934+0000 7fa22c8f7700 1 --2- 172.21.15.3:0/2639780240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa22809e930 0x7fa228122960 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fa21800e9a0 tx=0x7fa21800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:08.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.934+0000 7fa2257fa700 1 -- 172.21.15.3:0/2639780240 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa218004030 con 0x7fa22809e930 2023-10-03T10:17:08.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.934+0000 7fa2257fa700 1 -- 172.21.15.3:0/2639780240 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa218004190 con 0x7fa22809e930 2023-10-03T10:17:08.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.934+0000 7fa22eb5b700 1 -- 172.21.15.3:0/2639780240 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21c0094a0 con 0x7fa22809e930 2023-10-03T10:17:08.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.934+0000 7fa2257fa700 1 -- 172.21.15.3:0/2639780240 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa218004500 con 0x7fa22809e930 2023-10-03T10:17:08.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.934+0000 7fa22eb5b700 1 -- 172.21.15.3:0/2639780240 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22806cbb0 con 0x7fa22809e930 2023-10-03T10:17:08.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.935+0000 7fa2257fa700 1 -- 172.21.15.3:0/2639780240 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa2180036a0 con 0x7fa22809e930 2023-10-03T10:17:08.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.935+0000 7fa2257fa700 1 -- 172.21.15.3:0/2639780240 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa21803f590 con 0x7fa22809e930 2023-10-03T10:17:08.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.935+0000 7fa22eb5b700 1 -- 172.21.15.3:0/2639780240 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa228074ea0 con 0x7fa22809e930 2023-10-03T10:17:08.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:08.941+0000 7fa2257fa700 1 -- 172.21.15.3:0/2639780240 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa218022d50 con 0x7fa22809e930 2023-10-03T10:17:09.076 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:09.130 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:09.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.152+0000 7fa22eb5b700 1 -- 172.21.15.3:0/2639780240 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa228126c70 con 0x7fa22809e930 2023-10-03T10:17:09.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.152+0000 7fa2257fa700 1 -- 172.21.15.3:0/2639780240 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa2180222f0 con 0x7fa22809e930 2023-10-03T10:17:09.152 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:09.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.155+0000 7fa22eb5b700 1 -- 172.21.15.3:0/2639780240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa22809e930 msgr2=0x7fa228122960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:09.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.155+0000 7fa22eb5b700 1 --2- 172.21.15.3:0/2639780240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa22809e930 0x7fa228122960 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fa21800e9a0 tx=0x7fa21800ecb0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.155+0000 7fa22eb5b700 1 -- 172.21.15.3:0/2639780240 shutdown_connections 2023-10-03T10:17:09.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.155+0000 7fa22eb5b700 1 --2- 172.21.15.3:0/2639780240 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa228134120 0x7fa2281233e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.155+0000 7fa22eb5b700 1 --2- 172.21.15.3:0/2639780240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa22809e930 0x7fa228122960 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.155+0000 7fa22eb5b700 1 --2- 172.21.15.3:0/2639780240 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa22809f240 0x7fa228122ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.155+0000 7fa22eb5b700 1 -- 172.21.15.3:0/2639780240 >> 172.21.15.3:0/2639780240 conn(0x7fa2280a4210 msgr2=0x7fa228070f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:09.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.155+0000 7fa22eb5b700 1 -- 172.21.15.3:0/2639780240 shutdown_connections 2023-10-03T10:17:09.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.155+0000 7fa22eb5b700 1 -- 172.21.15.3:0/2639780240 wait complete. 2023-10-03T10:17:09.170 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:09.217 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:09.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.219+0000 7fda784f4700 1 Processor -- start 2023-10-03T10:17:09.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.219+0000 7fda784f4700 1 -- start start 2023-10-03T10:17:09.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.219+0000 7fda784f4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fda7009d630 0x7fda700a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:09.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.219+0000 7fda784f4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fda70138d80 0x7fda7013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:09.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.219+0000 7fda784f4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda700a2a70 con 0x7fda700a0a30 2023-10-03T10:17:09.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.219+0000 7fda784f4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda700a01d0 con 0x7fda70138d80 2023-10-03T10:17:09.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.219+0000 7fda784f4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fda7009e2b0 con 0x7fda7009d630 2023-10-03T10:17:09.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.220+0000 7fda75a8f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fda7009d630 0x7fda700a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:09.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.220+0000 7fda76a91700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fda70138d80 0x7fda7013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:09.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.220+0000 7fda75a8f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fda7009d630 0x7fda700a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50012/0 (socket says 172.21.15.3:50012) 2023-10-03T10:17:09.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.220+0000 7fda75a8f700 1 -- 172.21.15.3:0/1549825101 learned_addr learned my addr 172.21.15.3:0/1549825101 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:09.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.220+0000 7fda75a8f700 1 -- 172.21.15.3:0/1549825101 >> v2:172.21.15.3:3300/0 conn(0x7fda70138d80 msgr2=0x7fda7013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:09.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.220+0000 7fda75a8f700 1 --2- 172.21.15.3:0/1549825101 >> v2:172.21.15.3:3300/0 conn(0x7fda70138d80 0x7fda7013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.220+0000 7fda75a8f700 1 -- 172.21.15.3:0/1549825101 >> v1:172.21.15.3:6789/0 conn(0x7fda700a0a30 legacy=0x7fda700a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:09.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.220+0000 7fda75a8f700 1 -- 172.21.15.3:0/1549825101 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda7009dc50 con 0x7fda7009d630 2023-10-03T10:17:09.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.220+0000 7fda75a8f700 1 --2- 172.21.15.3:0/1549825101 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fda7009d630 0x7fda700a1510 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fda600098f0 tx=0x7fda60009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81246d88f896d2f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:09.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.220+0000 7fda7528e700 1 -- 172.21.15.3:0/1549825101 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda60011830 con 0x7fda7009d630 2023-10-03T10:17:09.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.220+0000 7fda7528e700 1 -- 172.21.15.3:0/1549825101 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda60011990 con 0x7fda7009d630 2023-10-03T10:17:09.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.220+0000 7fda7528e700 1 -- 172.21.15.3:0/1549825101 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda60011d20 con 0x7fda7009d630 2023-10-03T10:17:09.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.221+0000 7fda784f4700 1 -- 172.21.15.3:0/1549825101 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fda7009d630 msgr2=0x7fda700a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:09.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.221+0000 7fda784f4700 1 --2- 172.21.15.3:0/1549825101 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fda7009d630 0x7fda700a1510 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fda600098f0 tx=0x7fda60009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:09.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.221+0000 7fda784f4700 1 -- 172.21.15.3:0/1549825101 shutdown_connections 2023-10-03T10:17:09.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.221+0000 7fda784f4700 1 --2- 172.21.15.3:0/1549825101 >> v2:172.21.15.3:3300/0 conn(0x7fda70138d80 0x7fda7013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.221+0000 7fda784f4700 1 --2- 172.21.15.3:0/1549825101 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fda7009d630 0x7fda700a1510 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.221+0000 7fda784f4700 1 -- 172.21.15.3:0/1549825101 >> 172.21.15.3:0/1549825101 conn(0x7fda700a3050 msgr2=0x7fda700a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:09.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.221+0000 7fda784f4700 1 -- 172.21.15.3:0/1549825101 shutdown_connections 2023-10-03T10:17:09.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.221+0000 7fda784f4700 1 -- 172.21.15.3:0/1549825101 wait complete. 2023-10-03T10:17:09.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.222+0000 7fda784f4700 1 Processor -- start 2023-10-03T10:17:09.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.223+0000 7fda784f4700 1 -- start start 2023-10-03T10:17:09.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.223+0000 7fda784f4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda7009d630 0x7fda701c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:09.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.223+0000 7fda784f4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda700a0a30 0x7fda701c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:09.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.223+0000 7fda784f4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fda70138d80 0x7fda701c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:09.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.223+0000 7fda784f4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda701381b0 con 0x7fda700a0a30 2023-10-03T10:17:09.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.223+0000 7fda784f4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda701361d0 con 0x7fda7009d630 2023-10-03T10:17:09.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.223+0000 7fda784f4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fda7013b030 con 0x7fda70138d80 2023-10-03T10:17:09.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.223+0000 7fda75a8f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda700a0a30 0x7fda701c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:09.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.223+0000 7fda75a8f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda700a0a30 0x7fda701c8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37060/0 (socket says 172.21.15.3:37060) 2023-10-03T10:17:09.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.223+0000 7fda75a8f700 1 -- 172.21.15.3:0/156905502 learned_addr learned my addr 172.21.15.3:0/156905502 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:09.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.223+0000 7fda76290700 1 --2- 172.21.15.3:0/156905502 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda7009d630 0x7fda701c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:09.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.224+0000 7fda76290700 1 -- 172.21.15.3:0/156905502 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fda70138d80 msgr2=0x7fda701c9080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:17:09.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.224+0000 7fda76290700 1 --2- 172.21.15.3:0/156905502 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fda70138d80 0x7fda701c9080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.224+0000 7fda76290700 1 -- 172.21.15.3:0/156905502 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda700a0a30 msgr2=0x7fda701c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:09.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.224+0000 7fda76290700 1 --2- 172.21.15.3:0/156905502 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda700a0a30 0x7fda701c8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.224+0000 7fda76290700 1 -- 172.21.15.3:0/156905502 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda60009550 con 0x7fda7009d630 2023-10-03T10:17:09.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.224+0000 7fda75a8f700 1 --2- 172.21.15.3:0/156905502 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda700a0a30 0x7fda701c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:09.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.224+0000 7fda76290700 1 --2- 172.21.15.3:0/156905502 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda7009d630 0x7fda701c8600 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fda6c000f60 tx=0x7fda6c00c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:09.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.224+0000 7fda66ffd700 1 -- 172.21.15.3:0/156905502 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c0052d0 con 0x7fda7009d630 2023-10-03T10:17:09.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.224+0000 7fda66ffd700 1 -- 172.21.15.3:0/156905502 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6c005450 con 0x7fda7009d630 2023-10-03T10:17:09.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.224+0000 7fda784f4700 1 -- 172.21.15.3:0/156905502 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda700a0410 con 0x7fda7009d630 2023-10-03T10:17:09.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.224+0000 7fda66ffd700 1 -- 172.21.15.3:0/156905502 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c0038c0 con 0x7fda7009d630 2023-10-03T10:17:09.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.224+0000 7fda784f4700 1 -- 172.21.15.3:0/156905502 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda70136c40 con 0x7fda7009d630 2023-10-03T10:17:09.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.225+0000 7fda66ffd700 1 -- 172.21.15.3:0/156905502 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fda6c003a20 con 0x7fda7009d630 2023-10-03T10:17:09.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.225+0000 7fda784f4700 1 -- 172.21.15.3:0/156905502 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda70136010 con 0x7fda7009d630 2023-10-03T10:17:09.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.225+0000 7fda66ffd700 1 -- 172.21.15.3:0/156905502 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fda6c03ea80 con 0x7fda7009d630 2023-10-03T10:17:09.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.230+0000 7fda66ffd700 1 -- 172.21.15.3:0/156905502 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fda6c017020 con 0x7fda7009d630 2023-10-03T10:17:09.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.277+0000 7fb822e82700 1 Processor -- start 2023-10-03T10:17:09.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.277+0000 7fb822e82700 1 -- start start 2023-10-03T10:17:09.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.278+0000 7fb822e82700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb81c12c2f0 0x7fb81c130040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:09.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.278+0000 7fb822e82700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb81c12cff0 0x7fb81c137cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:09.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.278+0000 7fb822e82700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb81c09a510 con 0x7fb81c12cc20 2023-10-03T10:17:09.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.278+0000 7fb822e82700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb81c12c7d0 con 0x7fb81c12cff0 2023-10-03T10:17:09.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.278+0000 7fb822e82700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c09c190 con 0x7fb81c12c2f0 2023-10-03T10:17:09.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.278+0000 7fb82141f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb81c12cff0 0x7fb81c137cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:09.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.278+0000 7fb82141f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb81c12cff0 0x7fb81c137cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37074/0 (socket says 172.21.15.3:37074) 2023-10-03T10:17:09.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.278+0000 7fb82141f700 1 -- 172.21.15.3:0/3993089396 learned_addr learned my addr 172.21.15.3:0/3993089396 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:09.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.278+0000 7fb81bfff700 1 --2- 172.21.15.3:0/3993089396 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb81c12c2f0 0x7fb81c130040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:09.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.279+0000 7fb82141f700 1 -- 172.21.15.3:0/3993089396 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb81c12c2f0 msgr2=0x7fb81c130040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:09.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.279+0000 7fb82141f700 1 --2- 172.21.15.3:0/3993089396 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb81c12c2f0 0x7fb81c130040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.279+0000 7fb82141f700 1 -- 172.21.15.3:0/3993089396 >> v1:172.21.15.3:6789/0 conn(0x7fb81c12cc20 legacy=0x7fb81c12f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:09.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.279+0000 7fb82141f700 1 -- 172.21.15.3:0/3993089396 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c09b540 con 0x7fb81c12cff0 2023-10-03T10:17:09.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.279+0000 7fb82141f700 1 --2- 172.21.15.3:0/3993089396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb81c12cff0 0x7fb81c137cd0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fb81400d150 tx=0x7fb81400d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d0143b06836887 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:09.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.279+0000 7fb81b7fe700 1 -- 172.21.15.3:0/3993089396 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb814010070 con 0x7fb81c12cff0 2023-10-03T10:17:09.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.279+0000 7fb81b7fe700 1 -- 172.21.15.3:0/3993089396 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81400db00 con 0x7fb81c12cff0 2023-10-03T10:17:09.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.279+0000 7fb81b7fe700 1 -- 172.21.15.3:0/3993089396 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb814016400 con 0x7fb81c12cff0 2023-10-03T10:17:09.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.280+0000 7fb822e82700 1 -- 172.21.15.3:0/3993089396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb81c12cff0 msgr2=0x7fb81c137cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:09.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.280+0000 7fb822e82700 1 --2- 172.21.15.3:0/3993089396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb81c12cff0 0x7fb81c137cd0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fb81400d150 tx=0x7fb81400d460 comp rx=0 tx=0).stop 2023-10-03T10:17:09.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.280+0000 7fb822e82700 1 -- 172.21.15.3:0/3993089396 shutdown_connections 2023-10-03T10:17:09.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.280+0000 7fb822e82700 1 --2- 172.21.15.3:0/3993089396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb81c12cff0 0x7fb81c137cd0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.280+0000 7fb822e82700 1 --2- 172.21.15.3:0/3993089396 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb81c12c2f0 0x7fb81c130040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.280+0000 7fb822e82700 1 -- 172.21.15.3:0/3993089396 >> 172.21.15.3:0/3993089396 conn(0x7fb81c09f470 msgr2=0x7fb81c0a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:09.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.280+0000 7fb822e82700 1 -- 172.21.15.3:0/3993089396 shutdown_connections 2023-10-03T10:17:09.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.280+0000 7fb822e82700 1 -- 172.21.15.3:0/3993089396 wait complete. 2023-10-03T10:17:09.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.281+0000 7fb822e82700 1 Processor -- start 2023-10-03T10:17:09.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.281+0000 7fb822e82700 1 -- start start 2023-10-03T10:17:09.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.282+0000 7fb822e82700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb81c12c2f0 0x7fb81c06eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:09.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.282+0000 7fb822e82700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb81c12cc20 0x7fb81c06f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:09.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.282+0000 7fb822e82700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb81c12cff0 0x7fb81c06f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:09.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.282+0000 7fb822e82700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c12b6b0 con 0x7fb81c12cff0 2023-10-03T10:17:09.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.282+0000 7fb822e82700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c070860 con 0x7fb81c12c2f0 2023-10-03T10:17:09.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.282+0000 7fb822e82700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c0715c0 con 0x7fb81c12cc20 2023-10-03T10:17:09.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.283+0000 7fb81bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb81c12cc20 0x7fb81c06f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:09.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.283+0000 7fb81bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb81c12cc20 0x7fb81c06f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50026/0 (socket says 172.21.15.3:50026) 2023-10-03T10:17:09.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.283+0000 7fb81bfff700 1 -- 172.21.15.3:0/4261916444 learned_addr learned my addr 172.21.15.3:0/4261916444 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:09.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.283+0000 7fb82141f700 1 --2- 172.21.15.3:0/4261916444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb81c12cff0 0x7fb81c06f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:09.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.283+0000 7fb82141f700 1 -- 172.21.15.3:0/4261916444 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb81c12cc20 msgr2=0x7fb81c06f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:09.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.283+0000 7fb82141f700 1 --2- 172.21.15.3:0/4261916444 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb81c12cc20 0x7fb81c06f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.283+0000 7fb82141f700 1 -- 172.21.15.3:0/4261916444 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb81c12c2f0 msgr2=0x7fb81c06eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:09.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.283+0000 7fb82141f700 1 --2- 172.21.15.3:0/4261916444 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb81c12c2f0 0x7fb81c06eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.283+0000 7fb82141f700 1 -- 172.21.15.3:0/4261916444 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81400bd40 con 0x7fb81c12cff0 2023-10-03T10:17:09.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.283+0000 7fb82141f700 1 --2- 172.21.15.3:0/4261916444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb81c12cff0 0x7fb81c06f820 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fb814019000 tx=0x7fb814004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:09.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.285+0000 7fb8197fa700 1 -- 172.21.15.3:0/4261916444 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb814004140 con 0x7fb81c12cff0 2023-10-03T10:17:09.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.285+0000 7fb8197fa700 1 -- 172.21.15.3:0/4261916444 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8140042c0 con 0x7fb81c12cff0 2023-10-03T10:17:09.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.285+0000 7fb8197fa700 1 -- 172.21.15.3:0/4261916444 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb814004670 con 0x7fb81c12cff0 2023-10-03T10:17:09.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.286+0000 7fb822e82700 1 -- 172.21.15.3:0/4261916444 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb81c070fb0 con 0x7fb81c12cff0 2023-10-03T10:17:09.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.286+0000 7fb822e82700 1 -- 172.21.15.3:0/4261916444 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb81c072460 con 0x7fb81c12cff0 2023-10-03T10:17:09.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.286+0000 7fb8197fa700 1 -- 172.21.15.3:0/4261916444 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb8140047d0 con 0x7fb81c12cff0 2023-10-03T10:17:09.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.287+0000 7fb8197fa700 1 -- 172.21.15.3:0/4261916444 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb814040dc0 con 0x7fb81c12cff0 2023-10-03T10:17:09.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.287+0000 7fb822e82700 1 -- 172.21.15.3:0/4261916444 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb81c1285a0 con 0x7fb81c12cff0 2023-10-03T10:17:09.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.292+0000 7fb8197fa700 1 -- 172.21.15.3:0/4261916444 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb814023020 con 0x7fb81c12cff0 2023-10-03T10:17:09.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.359+0000 7f803ece3700 1 Processor -- start 2023-10-03T10:17:09.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.359+0000 7f803ece3700 1 -- start start 2023-10-03T10:17:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.360+0000 7f803ece3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f803809e850 0x7f803809ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.360+0000 7f803ece3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f803809f160 0x7f8038133b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.360+0000 7f803ece3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f80381340c0 0x7f8038134560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:09.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.360+0000 7f803ece3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f803809aa30 con 0x7f803809f160 2023-10-03T10:17:09.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.360+0000 7f803ece3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80380a0050 con 0x7f803809e850 2023-10-03T10:17:09.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.360+0000 7f803ece3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f803806bc50 con 0x7f80381340c0 2023-10-03T10:17:09.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.360+0000 7f8037fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f803809f160 0x7f8038133b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:09.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.360+0000 7f8037fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f803809f160 0x7f8038133b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37096/0 (socket says 172.21.15.3:37096) 2023-10-03T10:17:09.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.360+0000 7f8037fff700 1 -- 172.21.15.3:0/4191931801 learned_addr learned my addr 172.21.15.3:0/4191931801 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:09.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.360+0000 7f803d280700 1 --2- 172.21.15.3:0/4191931801 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f80381340c0 0x7f8038134560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:09.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.360+0000 7f803ca7f700 1 --2- 172.21.15.3:0/4191931801 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f803809e850 0x7f803809ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:09.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.361+0000 7f803d280700 1 -- 172.21.15.3:0/4191931801 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f803809e850 msgr2=0x7f803809ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:09.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.361+0000 7f803d280700 1 --2- 172.21.15.3:0/4191931801 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f803809e850 0x7f803809ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.361+0000 7f803d280700 1 -- 172.21.15.3:0/4191931801 >> v2:172.21.15.3:3300/0 conn(0x7f803809f160 msgr2=0x7f8038133b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:09.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.361+0000 7f803d280700 1 --2- 172.21.15.3:0/4191931801 >> v2:172.21.15.3:3300/0 conn(0x7f803809f160 0x7f8038133b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.361+0000 7f803d280700 1 -- 172.21.15.3:0/4191931801 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803806cb10 con 0x7f80381340c0 2023-10-03T10:17:09.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.361+0000 7f8037fff700 1 --2- 172.21.15.3:0/4191931801 >> v2:172.21.15.3:3300/0 conn(0x7f803809f160 0x7f8038133b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:09.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.361+0000 7f803d280700 1 --2- 172.21.15.3:0/4191931801 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f80381340c0 0x7f8038134560 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8030009d90 tx=0x7f803000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb050316400a8718 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:09.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.361+0000 7f80377fe700 1 -- 172.21.15.3:0/4191931801 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803000e070 con 0x7f80381340c0 2023-10-03T10:17:09.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.361+0000 7f80377fe700 1 -- 172.21.15.3:0/4191931801 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803000bb20 con 0x7f80381340c0 2023-10-03T10:17:09.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.361+0000 7f80377fe700 1 -- 172.21.15.3:0/4191931801 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030012430 con 0x7f80381340c0 2023-10-03T10:17:09.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.362+0000 7f803ece3700 1 -- 172.21.15.3:0/4191931801 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f80381340c0 msgr2=0x7f8038134560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:09.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.362+0000 7f803ece3700 1 --2- 172.21.15.3:0/4191931801 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f80381340c0 0x7f8038134560 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8030009d90 tx=0x7f803000b330 comp rx=0 tx=0).stop 2023-10-03T10:17:09.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.362+0000 7f803ece3700 1 -- 172.21.15.3:0/4191931801 shutdown_connections 2023-10-03T10:17:09.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.362+0000 7f803ece3700 1 --2- 172.21.15.3:0/4191931801 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f80381340c0 0x7f8038134560 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.362+0000 7f803ece3700 1 --2- 172.21.15.3:0/4191931801 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f803809e850 0x7f803809ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.362+0000 7f803ece3700 1 --2- 172.21.15.3:0/4191931801 >> v2:172.21.15.3:3300/0 conn(0x7f803809f160 0x7f8038133b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.362+0000 7f803ece3700 1 -- 172.21.15.3:0/4191931801 >> 172.21.15.3:0/4191931801 conn(0x7f80380a3f70 msgr2=0x7f80380a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:09.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.362+0000 7f803ece3700 1 -- 172.21.15.3:0/4191931801 shutdown_connections 2023-10-03T10:17:09.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.362+0000 7f803ece3700 1 -- 172.21.15.3:0/4191931801 wait complete. 2023-10-03T10:17:09.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.363+0000 7f803ece3700 1 Processor -- start 2023-10-03T10:17:09.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.363+0000 7f803ece3700 1 -- start start 2023-10-03T10:17:09.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.363+0000 7f803ece3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f803809e850 0x7f8038122410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:09.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.363+0000 7f803ece3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f803809f160 0x7f8038122950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:09.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.363+0000 7f803ece3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80381340c0 0x7f8038122e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:09.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.363+0000 7f803ece3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8038125f30 con 0x7f80381340c0 2023-10-03T10:17:09.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.363+0000 7f803ece3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8038125990 con 0x7f803809f160 2023-10-03T10:17:09.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.363+0000 7f803ece3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f80381338a0 con 0x7f803809e850 2023-10-03T10:17:09.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.364+0000 7f803d280700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80381340c0 0x7f8038122e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:09.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.364+0000 7f8037fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f803809f160 0x7f8038122950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:09.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.364+0000 7f803ca7f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f803809e850 0x7f8038122410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:09.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.364+0000 7f803d280700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80381340c0 0x7f8038122e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37108/0 (socket says 172.21.15.3:37108) 2023-10-03T10:17:09.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.364+0000 7f803d280700 1 -- 172.21.15.3:0/1105338413 learned_addr learned my addr 172.21.15.3:0/1105338413 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:09.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.364+0000 7f803ca7f700 1 -- 172.21.15.3:0/1105338413 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f803809f160 msgr2=0x7f8038122950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:09.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.364+0000 7f803ca7f700 1 --2- 172.21.15.3:0/1105338413 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f803809f160 0x7f8038122950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.364+0000 7f803ca7f700 1 -- 172.21.15.3:0/1105338413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80381340c0 msgr2=0x7f8038122e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:09.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.364+0000 7f803ca7f700 1 --2- 172.21.15.3:0/1105338413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80381340c0 0x7f8038122e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.364+0000 7f803ca7f700 1 -- 172.21.15.3:0/1105338413 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8024009550 con 0x7f803809e850 2023-10-03T10:17:09.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.364+0000 7f803d280700 1 --2- 172.21.15.3:0/1105338413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80381340c0 0x7f8038122e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:09.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.364+0000 7f803ca7f700 1 --2- 172.21.15.3:0/1105338413 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f803809e850 0x7f8038122410 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f802c00e870 tx=0x7f802c00eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:09.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.364+0000 7f80357fa700 1 -- 172.21.15.3:0/1105338413 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c013680 con 0x7f803809e850 2023-10-03T10:17:09.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.364+0000 7f80357fa700 1 -- 172.21.15.3:0/1105338413 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802c0137e0 con 0x7f803809e850 2023-10-03T10:17:09.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.364+0000 7f803ece3700 1 -- 172.21.15.3:0/1105338413 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8030009a70 con 0x7f803809e850 2023-10-03T10:17:09.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.364+0000 7f80357fa700 1 -- 172.21.15.3:0/1105338413 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c013b90 con 0x7f803809e850 2023-10-03T10:17:09.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.365+0000 7f803ece3700 1 -- 172.21.15.3:0/1105338413 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f803806cd80 con 0x7f803809e850 2023-10-03T10:17:09.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.365+0000 7f80357fa700 1 -- 172.21.15.3:0/1105338413 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f802c013cf0 con 0x7f803809e850 2023-10-03T10:17:09.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.366+0000 7f80357fa700 1 -- 172.21.15.3:0/1105338413 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f802c03f560 con 0x7f803809e850 2023-10-03T10:17:09.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.366+0000 7f803ece3700 1 -- 172.21.15.3:0/1105338413 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80380747f0 con 0x7f803809e850 2023-10-03T10:17:09.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.375+0000 7f80357fa700 1 -- 172.21.15.3:0/1105338413 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f802c01bbe0 con 0x7f803809e850 2023-10-03T10:17:09.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.442+0000 7fda784f4700 1 -- 172.21.15.3:0/156905502 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fda700747f0 con 0x7fda7009d630 2023-10-03T10:17:09.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.443+0000 7fda66ffd700 1 -- 172.21.15.3:0/156905502 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fda6c019c00 con 0x7fda7009d630 2023-10-03T10:17:09.442 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:09.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.447+0000 7fda784f4700 1 -- 172.21.15.3:0/156905502 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda7009d630 msgr2=0x7fda701c8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:09.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.447+0000 7fda784f4700 1 --2- 172.21.15.3:0/156905502 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda7009d630 0x7fda701c8600 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fda6c000f60 tx=0x7fda6c00c730 comp rx=0 tx=0).stop 2023-10-03T10:17:09.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.447+0000 7fda784f4700 1 -- 172.21.15.3:0/156905502 shutdown_connections 2023-10-03T10:17:09.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.447+0000 7fda784f4700 1 --2- 172.21.15.3:0/156905502 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fda70138d80 0x7fda701c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.447+0000 7fda784f4700 1 --2- 172.21.15.3:0/156905502 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda7009d630 0x7fda701c8600 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.447+0000 7fda784f4700 1 --2- 172.21.15.3:0/156905502 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda700a0a30 0x7fda701c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.447+0000 7fda784f4700 1 -- 172.21.15.3:0/156905502 >> 172.21.15.3:0/156905502 conn(0x7fda700a3050 msgr2=0x7fda7009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:09.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.447+0000 7fda784f4700 1 -- 172.21.15.3:0/156905502 shutdown_connections 2023-10-03T10:17:09.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.447+0000 7fda784f4700 1 -- 172.21.15.3:0/156905502 wait complete. 2023-10-03T10:17:09.460 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:09.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.504+0000 7fb822e82700 1 -- 172.21.15.3:0/4261916444 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb81c0722a0 con 0x7fb81c12cff0 2023-10-03T10:17:09.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.504+0000 7fb8197fa700 1 -- 172.21.15.3:0/4261916444 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb81401d760 con 0x7fb81c12cff0 2023-10-03T10:17:09.504 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:09.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.506+0000 7fb822e82700 1 -- 172.21.15.3:0/4261916444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb81c12cff0 msgr2=0x7fb81c06f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:09.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.506+0000 7fb822e82700 1 --2- 172.21.15.3:0/4261916444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb81c12cff0 0x7fb81c06f820 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fb814019000 tx=0x7fb814004ad0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.506+0000 7fb822e82700 1 -- 172.21.15.3:0/4261916444 shutdown_connections 2023-10-03T10:17:09.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.506+0000 7fb822e82700 1 --2- 172.21.15.3:0/4261916444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb81c12cff0 0x7fb81c06f820 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.506+0000 7fb822e82700 1 --2- 172.21.15.3:0/4261916444 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb81c12c2f0 0x7fb81c06eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.506+0000 7fb822e82700 1 --2- 172.21.15.3:0/4261916444 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb81c12cc20 0x7fb81c06f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.506+0000 7fb822e82700 1 -- 172.21.15.3:0/4261916444 >> 172.21.15.3:0/4261916444 conn(0x7fb81c09f470 msgr2=0x7fb81c12fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:09.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.507+0000 7fb822e82700 1 -- 172.21.15.3:0/4261916444 shutdown_connections 2023-10-03T10:17:09.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.507+0000 7fb822e82700 1 -- 172.21.15.3:0/4261916444 wait complete. 2023-10-03T10:17:09.519 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:09.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.585+0000 7f803ece3700 1 -- 172.21.15.3:0/1105338413 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8038072840 con 0x7f803809e850 2023-10-03T10:17:09.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.586+0000 7f80357fa700 1 -- 172.21.15.3:0/1105338413 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f802c017020 con 0x7f803809e850 2023-10-03T10:17:09.585 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:09.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.588+0000 7f803ece3700 1 -- 172.21.15.3:0/1105338413 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f803809e850 msgr2=0x7f8038122410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:09.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.588+0000 7f803ece3700 1 --2- 172.21.15.3:0/1105338413 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f803809e850 0x7f8038122410 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f802c00e870 tx=0x7f802c00eb80 comp rx=0 tx=0).stop 2023-10-03T10:17:09.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.588+0000 7f803ece3700 1 -- 172.21.15.3:0/1105338413 shutdown_connections 2023-10-03T10:17:09.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.588+0000 7f803ece3700 1 --2- 172.21.15.3:0/1105338413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80381340c0 0x7f8038122e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.588+0000 7f803ece3700 1 --2- 172.21.15.3:0/1105338413 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f803809e850 0x7f8038122410 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.588+0000 7f803ece3700 1 --2- 172.21.15.3:0/1105338413 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f803809f160 0x7f8038122950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:09.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.588+0000 7f803ece3700 1 -- 172.21.15.3:0/1105338413 >> 172.21.15.3:0/1105338413 conn(0x7f80380a3f70 msgr2=0x7f8038071f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:09.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.588+0000 7f803ece3700 1 -- 172.21.15.3:0/1105338413 shutdown_connections 2023-10-03T10:17:09.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:09.588+0000 7f803ece3700 1 -- 172.21.15.3:0/1105338413 wait complete. 2023-10-03T10:17:09.601 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:10.171 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:10.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.319+0000 7fe901c3a700 1 Processor -- start 2023-10-03T10:17:10.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.319+0000 7fe901c3a700 1 -- start start 2023-10-03T10:17:10.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.320+0000 7fe901c3a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe8fc09e930 0x7fe8fc09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:10.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.320+0000 7fe901c3a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe8fc09f650 0x7fe8fc131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:10.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.320+0000 7fe901c3a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8fc070940 con 0x7fe8fc09f240 2023-10-03T10:17:10.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.320+0000 7fe901c3a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8fc06fd60 con 0x7fe8fc09e930 2023-10-03T10:17:10.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.320+0000 7fe901c3a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc06fa80 con 0x7fe8fc09f650 2023-10-03T10:17:10.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.320+0000 7fe8fb7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe8fc09e930 0x7fe8fc09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:10.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.320+0000 7fe8fbfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe8fc09f650 0x7fe8fc131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:10.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.320+0000 7fe8fb7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe8fc09e930 0x7fe8fc09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37110/0 (socket says 172.21.15.3:37110) 2023-10-03T10:17:10.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.320+0000 7fe8fb7fe700 1 -- 172.21.15.3:0/3711108788 learned_addr learned my addr 172.21.15.3:0/3711108788 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:10.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.320+0000 7fe8fbfff700 1 -- 172.21.15.3:0/3711108788 >> v2:172.21.15.3:3300/0 conn(0x7fe8fc09e930 msgr2=0x7fe8fc09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:10.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.320+0000 7fe8fbfff700 1 --2- 172.21.15.3:0/3711108788 >> v2:172.21.15.3:3300/0 conn(0x7fe8fc09e930 0x7fe8fc09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.320+0000 7fe8fbfff700 1 -- 172.21.15.3:0/3711108788 >> v1:172.21.15.3:6789/0 conn(0x7fe8fc09f240 legacy=0x7fe8fc1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:10.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.320+0000 7fe8fbfff700 1 -- 172.21.15.3:0/3711108788 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8fc09fb60 con 0x7fe8fc09f650 2023-10-03T10:17:10.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.320+0000 7fe8fb7fe700 1 --2- 172.21.15.3:0/3711108788 >> v2:172.21.15.3:3300/0 conn(0x7fe8fc09e930 0x7fe8fc09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:10.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.321+0000 7fe8fbfff700 1 --2- 172.21.15.3:0/3711108788 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe8fc09f650 0x7fe8fc131f00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4009d90 tx=0x7fe8f400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ef84f58a461ec8f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:10.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.321+0000 7fe8fa7fc700 1 -- 172.21.15.3:0/3711108788 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f400e070 con 0x7fe8fc09f650 2023-10-03T10:17:10.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.321+0000 7fe8fa7fc700 1 -- 172.21.15.3:0/3711108788 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f400bb20 con 0x7fe8fc09f650 2023-10-03T10:17:10.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.321+0000 7fe8fa7fc700 1 -- 172.21.15.3:0/3711108788 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4012430 con 0x7fe8fc09f650 2023-10-03T10:17:10.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.321+0000 7fe901c3a700 1 -- 172.21.15.3:0/3711108788 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe8fc09f650 msgr2=0x7fe8fc131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:10.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.321+0000 7fe901c3a700 1 --2- 172.21.15.3:0/3711108788 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe8fc09f650 0x7fe8fc131f00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4009d90 tx=0x7fe8f400b330 comp rx=0 tx=0).stop 2023-10-03T10:17:10.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.321+0000 7fe901c3a700 1 -- 172.21.15.3:0/3711108788 shutdown_connections 2023-10-03T10:17:10.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.321+0000 7fe901c3a700 1 --2- 172.21.15.3:0/3711108788 >> v2:172.21.15.3:3300/0 conn(0x7fe8fc09e930 0x7fe8fc09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.321+0000 7fe901c3a700 1 --2- 172.21.15.3:0/3711108788 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe8fc09f650 0x7fe8fc131f00 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.321+0000 7fe901c3a700 1 -- 172.21.15.3:0/3711108788 >> 172.21.15.3:0/3711108788 conn(0x7fe8fc0a4210 msgr2=0x7fe8fc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:10.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.321+0000 7fe901c3a700 1 -- 172.21.15.3:0/3711108788 shutdown_connections 2023-10-03T10:17:10.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.322+0000 7fe901c3a700 1 -- 172.21.15.3:0/3711108788 wait complete. 2023-10-03T10:17:10.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.323+0000 7fe901c3a700 1 Processor -- start 2023-10-03T10:17:10.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.323+0000 7fe901c3a700 1 -- start start 2023-10-03T10:17:10.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.324+0000 7fe901c3a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe8fc09e930 0x7fe8fc1c2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:10.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.324+0000 7fe901c3a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8fc09f240 0x7fe8fc1c3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:10.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.324+0000 7fe901c3a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe8fc09f650 0x7fe8fc1c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:10.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.324+0000 7fe901c3a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc1a7420 con 0x7fe8fc09f240 2023-10-03T10:17:10.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.324+0000 7fe901c3a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc1a6960 con 0x7fe8fc09e930 2023-10-03T10:17:10.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.324+0000 7fe901c3a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc1c3980 con 0x7fe8fc09f650 2023-10-03T10:17:10.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.324+0000 7fe8fb7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe8fc09e930 0x7fe8fc1c2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:10.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.324+0000 7fe8faffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8fc09f240 0x7fe8fc1c3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:10.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.324+0000 7fe8fb7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe8fc09e930 0x7fe8fc1c2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50452/0 (socket says 172.21.15.3:50452) 2023-10-03T10:17:10.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.324+0000 7fe8fb7fe700 1 -- 172.21.15.3:0/4083453131 learned_addr learned my addr 172.21.15.3:0/4083453131 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:10.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.324+0000 7fe8fbfff700 1 --2- 172.21.15.3:0/4083453131 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe8fc09f650 0x7fe8fc1c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:10.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.324+0000 7fe8fb7fe700 1 -- 172.21.15.3:0/4083453131 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe8fc09f650 msgr2=0x7fe8fc1c5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:10.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.324+0000 7fe8fb7fe700 1 --2- 172.21.15.3:0/4083453131 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe8fc09f650 0x7fe8fc1c5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.324+0000 7fe8fb7fe700 1 -- 172.21.15.3:0/4083453131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8fc09f240 msgr2=0x7fe8fc1c3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:10.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.324+0000 7fe8fb7fe700 1 --2- 172.21.15.3:0/4083453131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8fc09f240 0x7fe8fc1c3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.324+0000 7fe8fb7fe700 1 -- 172.21.15.3:0/4083453131 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ec009550 con 0x7fe8fc09e930 2023-10-03T10:17:10.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.324+0000 7fe8faffd700 1 --2- 172.21.15.3:0/4083453131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8fc09f240 0x7fe8fc1c3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:10.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.325+0000 7fe8fb7fe700 1 --2- 172.21.15.3:0/4083453131 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe8fc09e930 0x7fe8fc1c2f00 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00ebe0 tx=0x7fe8ec00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:10.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.325+0000 7fe8e3fff700 1 -- 172.21.15.3:0/4083453131 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec00c850 con 0x7fe8fc09e930 2023-10-03T10:17:10.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.325+0000 7fe8e3fff700 1 -- 172.21.15.3:0/4083453131 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8ec00c9b0 con 0x7fe8fc09e930 2023-10-03T10:17:10.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.325+0000 7fe901c3a700 1 -- 172.21.15.3:0/4083453131 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f4009a70 con 0x7fe8fc09e930 2023-10-03T10:17:10.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.325+0000 7fe8e3fff700 1 -- 172.21.15.3:0/4083453131 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec00cd40 con 0x7fe8fc09e930 2023-10-03T10:17:10.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.325+0000 7fe901c3a700 1 -- 172.21.15.3:0/4083453131 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8fc0a0190 con 0x7fe8fc09e930 2023-10-03T10:17:10.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.326+0000 7fe901c3a700 1 -- 172.21.15.3:0/4083453131 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8fc0a3630 con 0x7fe8fc09e930 2023-10-03T10:17:10.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.326+0000 7fe8e3fff700 1 -- 172.21.15.3:0/4083453131 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe8ec013470 con 0x7fe8fc09e930 2023-10-03T10:17:10.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.327+0000 7fe8e3fff700 1 -- 172.21.15.3:0/4083453131 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe8ec040060 con 0x7fe8fc09e930 2023-10-03T10:17:10.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.331+0000 7fe8e3fff700 1 -- 172.21.15.3:0/4083453131 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe8ec011760 con 0x7fe8fc09e930 2023-10-03T10:17:10.461 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:10.521 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:10.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.538+0000 7fe901c3a700 1 -- 172.21.15.3:0/4083453131 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe8fc074ea0 con 0x7fe8fc09e930 2023-10-03T10:17:10.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.539+0000 7fe8e3fff700 1 -- 172.21.15.3:0/4083453131 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe8ec01a360 con 0x7fe8fc09e930 2023-10-03T10:17:10.538 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:10.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.541+0000 7fe901c3a700 1 -- 172.21.15.3:0/4083453131 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe8fc09e930 msgr2=0x7fe8fc1c2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:10.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.541+0000 7fe901c3a700 1 --2- 172.21.15.3:0/4083453131 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe8fc09e930 0x7fe8fc1c2f00 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00ebe0 tx=0x7fe8ec00eef0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.541+0000 7fe901c3a700 1 -- 172.21.15.3:0/4083453131 shutdown_connections 2023-10-03T10:17:10.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.541+0000 7fe901c3a700 1 --2- 172.21.15.3:0/4083453131 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe8fc09f650 0x7fe8fc1c5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.541+0000 7fe901c3a700 1 --2- 172.21.15.3:0/4083453131 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe8fc09e930 0x7fe8fc1c2f00 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.541+0000 7fe901c3a700 1 --2- 172.21.15.3:0/4083453131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8fc09f240 0x7fe8fc1c3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.541+0000 7fe901c3a700 1 -- 172.21.15.3:0/4083453131 >> 172.21.15.3:0/4083453131 conn(0x7fe8fc0a4210 msgr2=0x7fe8fc1b8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:10.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.542+0000 7fe901c3a700 1 -- 172.21.15.3:0/4083453131 shutdown_connections 2023-10-03T10:17:10.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.542+0000 7fe901c3a700 1 -- 172.21.15.3:0/4083453131 wait complete. 2023-10-03T10:17:10.555 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:10.602 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:10.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.604+0000 7f8706216700 1 Processor -- start 2023-10-03T10:17:10.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.604+0000 7f8706216700 1 -- start start 2023-10-03T10:17:10.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.605+0000 7f8706216700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f870009e930 0x7f870009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:10.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.605+0000 7f8706216700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f870009f240 0x7f87001317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:10.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.605+0000 7f8706216700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8700070940 con 0x7f870009f650 2023-10-03T10:17:10.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.605+0000 7f8706216700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f870006fd60 con 0x7f870009e930 2023-10-03T10:17:10.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.605+0000 7f8706216700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f870006fa80 con 0x7f870009f240 2023-10-03T10:17:10.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.605+0000 7f86ff7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f870009e930 0x7f870009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:10.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.605+0000 7f86ff7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f870009e930 0x7f870009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50454/0 (socket says 172.21.15.3:50454) 2023-10-03T10:17:10.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.605+0000 7f86ff7fe700 1 -- 172.21.15.3:0/1871420616 learned_addr learned my addr 172.21.15.3:0/1871420616 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:10.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.605+0000 7f86feffd700 1 --2- 172.21.15.3:0/1871420616 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f870009f240 0x7f87001317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:10.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.605+0000 7f86ff7fe700 1 -- 172.21.15.3:0/1871420616 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f870009f240 msgr2=0x7f87001317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:10.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.605+0000 7f86ff7fe700 1 --2- 172.21.15.3:0/1871420616 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f870009f240 0x7f87001317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.605+0000 7f86ff7fe700 1 -- 172.21.15.3:0/1871420616 >> v1:172.21.15.3:6789/0 conn(0x7f870009f650 legacy=0x7f8700131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:10.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.605+0000 7f86ff7fe700 1 -- 172.21.15.3:0/1871420616 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870009fb60 con 0x7f870009e930 2023-10-03T10:17:10.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.605+0000 7f86ff7fe700 1 --2- 172.21.15.3:0/1871420616 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f870009e930 0x7f870009ed00 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009920 tx=0x7f86f0009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2046e8939db140a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:10.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.606+0000 7f86fe7fc700 1 -- 172.21.15.3:0/1871420616 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f0011830 con 0x7f870009e930 2023-10-03T10:17:10.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.606+0000 7f86fe7fc700 1 -- 172.21.15.3:0/1871420616 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f0011990 con 0x7f870009e930 2023-10-03T10:17:10.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.606+0000 7f86fe7fc700 1 -- 172.21.15.3:0/1871420616 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f0011d20 con 0x7f870009e930 2023-10-03T10:17:10.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.606+0000 7f8706216700 1 -- 172.21.15.3:0/1871420616 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f870009e930 msgr2=0x7f870009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:10.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.606+0000 7f8706216700 1 --2- 172.21.15.3:0/1871420616 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f870009e930 0x7f870009ed00 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009920 tx=0x7f86f0009c30 comp rx=0 tx=0).stop 2023-10-03T10:17:10.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.607+0000 7f8706216700 1 -- 172.21.15.3:0/1871420616 shutdown_connections 2023-10-03T10:17:10.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.607+0000 7f8706216700 1 --2- 172.21.15.3:0/1871420616 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f870009e930 0x7f870009ed00 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.607+0000 7f8706216700 1 --2- 172.21.15.3:0/1871420616 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f870009f240 0x7f87001317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.607+0000 7f8706216700 1 -- 172.21.15.3:0/1871420616 >> 172.21.15.3:0/1871420616 conn(0x7f87000a4210 msgr2=0x7f87000a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:10.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.607+0000 7f8706216700 1 -- 172.21.15.3:0/1871420616 shutdown_connections 2023-10-03T10:17:10.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.607+0000 7f8706216700 1 -- 172.21.15.3:0/1871420616 wait complete. 2023-10-03T10:17:10.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.608+0000 7f8706216700 1 Processor -- start 2023-10-03T10:17:10.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.608+0000 7f8706216700 1 -- start start 2023-10-03T10:17:10.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.608+0000 7f8706216700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f870009e930 0x7f87001a2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:10.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.608+0000 7f8706216700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f870009f240 0x7f87001a3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:10.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.608+0000 7f8706216700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f870009f650 0x7f87001a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:10.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.608+0000 7f8706216700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87001a4980 con 0x7f870009e930 2023-10-03T10:17:10.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.608+0000 7f8706216700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87001a6510 con 0x7f870009f650 2023-10-03T10:17:10.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.608+0000 7f8706216700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f87000a37f0 con 0x7f870009f240 2023-10-03T10:17:10.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.609+0000 7f86ff7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f870009e930 0x7f87001a2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:10.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.609+0000 7f86ff7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f870009e930 0x7f87001a2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37122/0 (socket says 172.21.15.3:37122) 2023-10-03T10:17:10.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.609+0000 7f86ff7fe700 1 -- 172.21.15.3:0/2536237724 learned_addr learned my addr 172.21.15.3:0/2536237724 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:10.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.609+0000 7f86fffff700 1 --2- 172.21.15.3:0/2536237724 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f870009f650 0x7f87001a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:10.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.609+0000 7f86ff7fe700 1 -- 172.21.15.3:0/2536237724 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f870009f240 msgr2=0x7f87001a3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:10.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.609+0000 7f86ff7fe700 1 --2- 172.21.15.3:0/2536237724 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f870009f240 0x7f87001a3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.609+0000 7f86ff7fe700 1 -- 172.21.15.3:0/2536237724 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f870009f650 msgr2=0x7f87001a3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:10.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.609+0000 7f86ff7fe700 1 --2- 172.21.15.3:0/2536237724 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f870009f650 0x7f87001a3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.609+0000 7f86ff7fe700 1 -- 172.21.15.3:0/2536237724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f0009550 con 0x7f870009e930 2023-10-03T10:17:10.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.609+0000 7f86fffff700 1 --2- 172.21.15.3:0/2536237724 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f870009f650 0x7f87001a3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:10.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.609+0000 7f86ff7fe700 1 --2- 172.21.15.3:0/2536237724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f870009e930 0x7f87001a2cd0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f86f0000c00 tx=0x7f86f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:10.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.609+0000 7f86e7fff700 1 -- 172.21.15.3:0/2536237724 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f0004510 con 0x7f870009e930 2023-10-03T10:17:10.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.609+0000 7f86e7fff700 1 -- 172.21.15.3:0/2536237724 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f00036a0 con 0x7f870009e930 2023-10-03T10:17:10.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.609+0000 7f86e7fff700 1 -- 172.21.15.3:0/2536237724 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f0003a10 con 0x7f870009e930 2023-10-03T10:17:10.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.609+0000 7f8706216700 1 -- 172.21.15.3:0/2536237724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87000a0190 con 0x7f870009e930 2023-10-03T10:17:10.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.610+0000 7f8706216700 1 -- 172.21.15.3:0/2536237724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87001a5b30 con 0x7f870009e930 2023-10-03T10:17:10.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.610+0000 7f86e7fff700 1 -- 172.21.15.3:0/2536237724 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f86f0003b70 con 0x7f870009e930 2023-10-03T10:17:10.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.611+0000 7f8706216700 1 -- 172.21.15.3:0/2536237724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87001a6190 con 0x7f870009e930 2023-10-03T10:17:10.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.611+0000 7f86e7fff700 1 -- 172.21.15.3:0/2536237724 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f86f0021030 con 0x7f870009e930 2023-10-03T10:17:10.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.616+0000 7f86e7fff700 1 -- 172.21.15.3:0/2536237724 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f87001a6190 con 0x7f870009e930 2023-10-03T10:17:10.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.667+0000 7f6fdc41a700 1 Processor -- start 2023-10-03T10:17:10.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.667+0000 7f6fdc41a700 1 -- start start 2023-10-03T10:17:10.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.668+0000 7f6fdc41a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6fd409e850 0x7f6fd409ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:10.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.668+0000 7f6fdc41a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6fd409ffb0 0x7f6fd409f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:10.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.668+0000 7f6fdc41a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd40a06b0 con 0x7f6fd4133b80 2023-10-03T10:17:10.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.668+0000 7f6fdc41a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd406b920 con 0x7f6fd409e850 2023-10-03T10:17:10.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.668+0000 7f6fdc41a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd409aa30 con 0x7f6fd409ffb0 2023-10-03T10:17:10.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.668+0000 7f6fd99b5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6fd409ffb0 0x7f6fd409f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:10.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.668+0000 7f6fda1b6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6fd409e850 0x7f6fd409ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:10.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.668+0000 7f6fd99b5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6fd409ffb0 0x7f6fd409f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50486/0 (socket says 172.21.15.3:50486) 2023-10-03T10:17:10.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.668+0000 7f6fd99b5700 1 -- 172.21.15.3:0/3502911516 learned_addr learned my addr 172.21.15.3:0/3502911516 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:10.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.669+0000 7f6fd99b5700 1 -- 172.21.15.3:0/3502911516 >> v2:172.21.15.3:3300/0 conn(0x7f6fd409e850 msgr2=0x7f6fd409ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:10.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.669+0000 7f6fd99b5700 1 --2- 172.21.15.3:0/3502911516 >> v2:172.21.15.3:3300/0 conn(0x7f6fd409e850 0x7f6fd409ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.669+0000 7f6fd99b5700 1 -- 172.21.15.3:0/3502911516 >> v1:172.21.15.3:6789/0 conn(0x7f6fd4133b80 legacy=0x7f6fd4133fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:10.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.669+0000 7f6fd99b5700 1 -- 172.21.15.3:0/3502911516 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd406caa0 con 0x7f6fd409ffb0 2023-10-03T10:17:10.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.669+0000 7f6fd91b4700 1 -- 172.21.15.3:0/3502911516 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 868222338 0 0) 0x7f6fd40a06b0 con 0x7f6fd4133b80 2023-10-03T10:17:10.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.669+0000 7f6fda1b6700 1 --2- 172.21.15.3:0/3502911516 >> v2:172.21.15.3:3300/0 conn(0x7f6fd409e850 0x7f6fd409ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:10.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.669+0000 7f6fd99b5700 1 --2- 172.21.15.3:0/3502911516 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6fd409ffb0 0x7f6fd409f160 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6fc40098f0 tx=0x7f6fc4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4c9b9ce5c66d5c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:10.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.669+0000 7f6fd91b4700 1 -- 172.21.15.3:0/3502911516 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4011830 con 0x7f6fd409ffb0 2023-10-03T10:17:10.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.669+0000 7f6fd91b4700 1 -- 172.21.15.3:0/3502911516 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc4011990 con 0x7f6fd409ffb0 2023-10-03T10:17:10.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.669+0000 7f6fd91b4700 1 -- 172.21.15.3:0/3502911516 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4011d20 con 0x7f6fd409ffb0 2023-10-03T10:17:10.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.670+0000 7f6fdc41a700 1 -- 172.21.15.3:0/3502911516 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6fd409ffb0 msgr2=0x7f6fd409f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:10.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.670+0000 7f6fdc41a700 1 --2- 172.21.15.3:0/3502911516 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6fd409ffb0 0x7f6fd409f160 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6fc40098f0 tx=0x7f6fc4009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:10.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.670+0000 7f6fdc41a700 1 -- 172.21.15.3:0/3502911516 shutdown_connections 2023-10-03T10:17:10.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.670+0000 7f6fdc41a700 1 --2- 172.21.15.3:0/3502911516 >> v2:172.21.15.3:3300/0 conn(0x7f6fd409e850 0x7f6fd409ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.670+0000 7f6fdc41a700 1 --2- 172.21.15.3:0/3502911516 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6fd409ffb0 0x7f6fd409f160 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.670+0000 7f6fdc41a700 1 -- 172.21.15.3:0/3502911516 >> 172.21.15.3:0/3502911516 conn(0x7f6fd40a3f70 msgr2=0x7f6fd40a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:10.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.670+0000 7f6fdc41a700 1 -- 172.21.15.3:0/3502911516 shutdown_connections 2023-10-03T10:17:10.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.671+0000 7f6fdc41a700 1 -- 172.21.15.3:0/3502911516 wait complete. 2023-10-03T10:17:10.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.672+0000 7f6fdc41a700 1 Processor -- start 2023-10-03T10:17:10.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.672+0000 7f6fdc41a700 1 -- start start 2023-10-03T10:17:10.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.673+0000 7f6fdc41a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6fd409e850 0x7f6fd4120520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:10.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.673+0000 7f6fdc41a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6fd409ffb0 0x7f6fd4120a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:10.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.673+0000 7f6fdc41a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6fd4133b80 0x7f6fd4120fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:10.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.673+0000 7f6fdc41a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd4121980 con 0x7f6fd409e850 2023-10-03T10:17:10.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.673+0000 7f6fdc41a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd4122710 con 0x7f6fd4133b80 2023-10-03T10:17:10.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.673+0000 7f6fdc41a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd41237d0 con 0x7f6fd409ffb0 2023-10-03T10:17:10.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.673+0000 7f6fd99b5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6fd409ffb0 0x7f6fd4120a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:10.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.673+0000 7f6fd99b5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6fd409ffb0 0x7f6fd4120a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50086/0 (socket says 172.21.15.3:50086) 2023-10-03T10:17:10.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.673+0000 7f6fd99b5700 1 -- 172.21.15.3:0/2960596205 learned_addr learned my addr 172.21.15.3:0/2960596205 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:10.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.673+0000 7f6fda9b7700 1 --2- 172.21.15.3:0/2960596205 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6fd4133b80 0x7f6fd4120fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:10.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.673+0000 7f6fda1b6700 1 --2- 172.21.15.3:0/2960596205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6fd409e850 0x7f6fd4120520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:10.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.673+0000 7f6fda9b7700 1 -- 172.21.15.3:0/2960596205 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6fd409ffb0 msgr2=0x7f6fd4120a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:10.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.673+0000 7f6fda9b7700 1 --2- 172.21.15.3:0/2960596205 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6fd409ffb0 0x7f6fd4120a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.673+0000 7f6fda9b7700 1 -- 172.21.15.3:0/2960596205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6fd409e850 msgr2=0x7f6fd4120520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:10.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.673+0000 7f6fda9b7700 1 --2- 172.21.15.3:0/2960596205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6fd409e850 0x7f6fd4120520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.673+0000 7f6fda9b7700 1 -- 172.21.15.3:0/2960596205 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd00094a0 con 0x7f6fd4133b80 2023-10-03T10:17:10.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.674+0000 7f6fda9b7700 1 --2- 172.21.15.3:0/2960596205 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6fd4133b80 0x7f6fd4120fa0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00c7c0 tx=0x7f6fcc00cad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:10.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.674+0000 7f6fcaffd700 1 -- 172.21.15.3:0/2960596205 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc003e30 con 0x7f6fd4133b80 2023-10-03T10:17:10.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.674+0000 7f6fcaffd700 1 -- 172.21.15.3:0/2960596205 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fcc005070 con 0x7f6fd4133b80 2023-10-03T10:17:10.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.674+0000 7f6fdc41a700 1 -- 172.21.15.3:0/2960596205 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc4009550 con 0x7f6fd4133b80 2023-10-03T10:17:10.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.674+0000 7f6fcaffd700 1 -- 172.21.15.3:0/2960596205 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc005400 con 0x7f6fd4133b80 2023-10-03T10:17:10.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.674+0000 7f6fdc41a700 1 -- 172.21.15.3:0/2960596205 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd40a34a0 con 0x7f6fd4133b80 2023-10-03T10:17:10.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.675+0000 7f6fcaffd700 1 -- 172.21.15.3:0/2960596205 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6fcc005560 con 0x7f6fd4133b80 2023-10-03T10:17:10.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.675+0000 7f6fcaffd700 1 -- 172.21.15.3:0/2960596205 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6fcc03ed30 con 0x7f6fd4133b80 2023-10-03T10:17:10.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.675+0000 7f6fdc41a700 1 -- 172.21.15.3:0/2960596205 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd4123a00 con 0x7f6fd4133b80 2023-10-03T10:17:10.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.680+0000 7f6fcaffd700 1 -- 172.21.15.3:0/2960596205 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6fd4123a00 con 0x7f6fd4133b80 2023-10-03T10:17:10.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.753+0000 7f7b90edc700 1 Processor -- start 2023-10-03T10:17:10.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.753+0000 7f7b90edc700 1 -- start start 2023-10-03T10:17:10.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.754+0000 7f7b90edc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b8c09d6f0 0x7f7b8c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:10.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.754+0000 7f7b90edc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7b8c09e000 0x7f7b8c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:10.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.754+0000 7f7b90edc700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b8c09f130 con 0x7f7b8c09e410 2023-10-03T10:17:10.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.754+0000 7f7b90edc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b8c09f520 con 0x7f7b8c09e000 2023-10-03T10:17:10.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.754+0000 7f7b90edc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8c09e930 con 0x7f7b8c09d6f0 2023-10-03T10:17:10.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.754+0000 7f7b8a59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b8c09d6f0 0x7f7b8c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:10.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.754+0000 7f7b89d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7b8c09e000 0x7f7b8c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:10.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.754+0000 7f7b8a59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b8c09d6f0 0x7f7b8c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50506/0 (socket says 172.21.15.3:50506) 2023-10-03T10:17:10.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.754+0000 7f7b8a59c700 1 -- 172.21.15.3:0/3610173348 learned_addr learned my addr 172.21.15.3:0/3610173348 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:10.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.754+0000 7f7b8a59c700 1 -- 172.21.15.3:0/3610173348 >> v2:172.21.15.3:3300/0 conn(0x7f7b8c09e000 msgr2=0x7f7b8c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:10.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.755+0000 7f7b8a59c700 1 --2- 172.21.15.3:0/3610173348 >> v2:172.21.15.3:3300/0 conn(0x7f7b8c09e000 0x7f7b8c130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.755+0000 7f7b8a59c700 1 -- 172.21.15.3:0/3610173348 >> v1:172.21.15.3:6789/0 conn(0x7f7b8c09e410 legacy=0x7f7b8c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:10.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.755+0000 7f7b8a59c700 1 -- 172.21.15.3:0/3610173348 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b8c09b2d0 con 0x7f7b8c09d6f0 2023-10-03T10:17:10.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.755+0000 7f7b8a59c700 1 --2- 172.21.15.3:0/3610173348 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b8c09d6f0 0x7f7b8c09dac0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c0098f0 tx=0x7f7b7c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7353989a9d42eab3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:10.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.755+0000 7f7b8959a700 1 -- 172.21.15.3:0/3610173348 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c011830 con 0x7f7b8c09d6f0 2023-10-03T10:17:10.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.755+0000 7f7b8959a700 1 -- 172.21.15.3:0/3610173348 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7c011990 con 0x7f7b8c09d6f0 2023-10-03T10:17:10.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.755+0000 7f7b8959a700 1 -- 172.21.15.3:0/3610173348 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c011d20 con 0x7f7b8c09d6f0 2023-10-03T10:17:10.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.755+0000 7f7b90edc700 1 -- 172.21.15.3:0/3610173348 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b8c09d6f0 msgr2=0x7f7b8c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:10.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.755+0000 7f7b90edc700 1 --2- 172.21.15.3:0/3610173348 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b8c09d6f0 0x7f7b8c09dac0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c0098f0 tx=0x7f7b7c009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:10.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.756+0000 7f7b90edc700 1 -- 172.21.15.3:0/3610173348 shutdown_connections 2023-10-03T10:17:10.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.756+0000 7f7b90edc700 1 --2- 172.21.15.3:0/3610173348 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b8c09d6f0 0x7f7b8c09dac0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.756+0000 7f7b90edc700 1 --2- 172.21.15.3:0/3610173348 >> v2:172.21.15.3:3300/0 conn(0x7f7b8c09e000 0x7f7b8c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.756+0000 7f7b90edc700 1 -- 172.21.15.3:0/3610173348 >> 172.21.15.3:0/3610173348 conn(0x7f7b8c066f50 msgr2=0x7f7b8c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:10.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.756+0000 7f7b90edc700 1 -- 172.21.15.3:0/3610173348 shutdown_connections 2023-10-03T10:17:10.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.756+0000 7f7b90edc700 1 -- 172.21.15.3:0/3610173348 wait complete. 2023-10-03T10:17:10.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.757+0000 7f7b90edc700 1 Processor -- start 2023-10-03T10:17:10.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.757+0000 7f7b90edc700 1 -- start start 2023-10-03T10:17:10.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.757+0000 7f7b90edc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b8c09d6f0 0x7f7b8c131670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:10.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.757+0000 7f7b90edc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b8c09e000 0x7f7b8c131bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:10.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.757+0000 7f7b90edc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b8c09e410 0x7f7b8c1320f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:10.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.757+0000 7f7b90edc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8c133c60 con 0x7f7b8c09e410 2023-10-03T10:17:10.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.757+0000 7f7b90edc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8c130980 con 0x7f7b8c09e000 2023-10-03T10:17:10.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.757+0000 7f7b90edc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8c133030 con 0x7f7b8c09d6f0 2023-10-03T10:17:10.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.758+0000 7f7b8a59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b8c09d6f0 0x7f7b8c131670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:10.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.758+0000 7f7b8ad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b8c09e410 0x7f7b8c1320f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:10.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.758+0000 7f7b8a59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b8c09d6f0 0x7f7b8c131670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50096/0 (socket says 172.21.15.3:50096) 2023-10-03T10:17:10.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.758+0000 7f7b8a59c700 1 -- 172.21.15.3:0/2661609691 learned_addr learned my addr 172.21.15.3:0/2661609691 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:10.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.758+0000 7f7b89d9b700 1 --2- 172.21.15.3:0/2661609691 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b8c09e000 0x7f7b8c131bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:10.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.758+0000 7f7b8a59c700 1 -- 172.21.15.3:0/2661609691 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b8c09e000 msgr2=0x7f7b8c131bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:10.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.758+0000 7f7b8a59c700 1 --2- 172.21.15.3:0/2661609691 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b8c09e000 0x7f7b8c131bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.758+0000 7f7b8a59c700 1 -- 172.21.15.3:0/2661609691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b8c09e410 msgr2=0x7f7b8c1320f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:10.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.758+0000 7f7b8a59c700 1 --2- 172.21.15.3:0/2661609691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b8c09e410 0x7f7b8c1320f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.758+0000 7f7b8a59c700 1 -- 172.21.15.3:0/2661609691 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7c009550 con 0x7f7b8c09d6f0 2023-10-03T10:17:10.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.758+0000 7f7b8ad9d700 1 --2- 172.21.15.3:0/2661609691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b8c09e410 0x7f7b8c1320f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:10.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.758+0000 7f7b8a59c700 1 --2- 172.21.15.3:0/2661609691 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b8c09d6f0 0x7f7b8c131670 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c009c40 tx=0x7f7b7c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:10.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.758+0000 7f7b7affd700 1 -- 172.21.15.3:0/2661609691 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c004510 con 0x7f7b8c09d6f0 2023-10-03T10:17:10.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.758+0000 7f7b7affd700 1 -- 172.21.15.3:0/2661609691 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7c01e070 con 0x7f7b8c09d6f0 2023-10-03T10:17:10.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.758+0000 7f7b90edc700 1 -- 172.21.15.3:0/2661609691 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b8c0725e0 con 0x7f7b8c09d6f0 2023-10-03T10:17:10.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.758+0000 7f7b7affd700 1 -- 172.21.15.3:0/2661609691 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c0038e0 con 0x7f7b8c09d6f0 2023-10-03T10:17:10.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.759+0000 7f7b90edc700 1 -- 172.21.15.3:0/2661609691 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b8c1336c0 con 0x7f7b8c09d6f0 2023-10-03T10:17:10.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.759+0000 7f7b7affd700 1 -- 172.21.15.3:0/2661609691 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7b7c003a40 con 0x7f7b8c09d6f0 2023-10-03T10:17:10.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.760+0000 7f7b7affd700 1 -- 172.21.15.3:0/2661609691 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7b7c040480 con 0x7f7b8c09d6f0 2023-10-03T10:17:10.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.760+0000 7f7b90edc700 1 -- 172.21.15.3:0/2661609691 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b8c074ea0 con 0x7f7b8c09d6f0 2023-10-03T10:17:10.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.769+0000 7f7b7affd700 1 -- 172.21.15.3:0/2661609691 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7b7c015070 con 0x7f7b8c09d6f0 2023-10-03T10:17:10.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.872+0000 7f8706216700 1 -- 172.21.15.3:0/2536237724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f87001a6190 con 0x7f870009e930 2023-10-03T10:17:10.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.873+0000 7f86e7fff700 1 -- 172.21.15.3:0/2536237724 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f87001a6190 con 0x7f870009e930 2023-10-03T10:17:10.873 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:10.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.875+0000 7f8706216700 1 -- 172.21.15.3:0/2536237724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f870009e930 msgr2=0x7f87001a2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:10.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.875+0000 7f8706216700 1 --2- 172.21.15.3:0/2536237724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f870009e930 0x7f87001a2cd0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f86f0000c00 tx=0x7f86f0004750 comp rx=0 tx=0).stop 2023-10-03T10:17:10.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.875+0000 7f8706216700 1 -- 172.21.15.3:0/2536237724 shutdown_connections 2023-10-03T10:17:10.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.875+0000 7f8706216700 1 --2- 172.21.15.3:0/2536237724 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f870009f650 0x7f87001a3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.875+0000 7f8706216700 1 --2- 172.21.15.3:0/2536237724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f870009e930 0x7f87001a2cd0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.875+0000 7f8706216700 1 --2- 172.21.15.3:0/2536237724 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f870009f240 0x7f87001a3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.875+0000 7f8706216700 1 -- 172.21.15.3:0/2536237724 >> 172.21.15.3:0/2536237724 conn(0x7f87000a4210 msgr2=0x7f87001b8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:10.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.875+0000 7f8706216700 1 -- 172.21.15.3:0/2536237724 shutdown_connections 2023-10-03T10:17:10.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.875+0000 7f8706216700 1 -- 172.21.15.3:0/2536237724 wait complete. 2023-10-03T10:17:10.889 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:10.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.909+0000 7f6fdc41a700 1 -- 172.21.15.3:0/2960596205 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6fd4123a00 con 0x7f6fd4133b80 2023-10-03T10:17:10.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.909+0000 7f6fcaffd700 1 -- 172.21.15.3:0/2960596205 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6fd4123a00 con 0x7f6fd4133b80 2023-10-03T10:17:10.910 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:10.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.912+0000 7f6fdc41a700 1 -- 172.21.15.3:0/2960596205 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6fd4133b80 msgr2=0x7f6fd4120fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:10.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.912+0000 7f6fdc41a700 1 --2- 172.21.15.3:0/2960596205 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6fd4133b80 0x7f6fd4120fa0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00c7c0 tx=0x7f6fcc00cad0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.912+0000 7f6fdc41a700 1 -- 172.21.15.3:0/2960596205 shutdown_connections 2023-10-03T10:17:10.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.912+0000 7f6fdc41a700 1 --2- 172.21.15.3:0/2960596205 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6fd4133b80 0x7f6fd4120fa0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.912+0000 7f6fdc41a700 1 --2- 172.21.15.3:0/2960596205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6fd409e850 0x7f6fd4120520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.912+0000 7f6fdc41a700 1 --2- 172.21.15.3:0/2960596205 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6fd409ffb0 0x7f6fd4120a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.912+0000 7f6fdc41a700 1 -- 172.21.15.3:0/2960596205 >> 172.21.15.3:0/2960596205 conn(0x7f6fd40a3f70 msgr2=0x7f6fd41baed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:10.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.912+0000 7f6fdc41a700 1 -- 172.21.15.3:0/2960596205 shutdown_connections 2023-10-03T10:17:10.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.912+0000 7f6fdc41a700 1 -- 172.21.15.3:0/2960596205 wait complete. 2023-10-03T10:17:10.926 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:10.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.979+0000 7f7b90edc700 1 -- 172.21.15.3:0/2661609691 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7b8c06c360 con 0x7f7b8c09d6f0 2023-10-03T10:17:10.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.979+0000 7f7b7affd700 1 -- 172.21.15.3:0/2661609691 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7b7c015070 con 0x7f7b8c09d6f0 2023-10-03T10:17:10.980 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:10.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.982+0000 7f7b90edc700 1 -- 172.21.15.3:0/2661609691 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b8c09d6f0 msgr2=0x7f7b8c131670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:10.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.982+0000 7f7b90edc700 1 --2- 172.21.15.3:0/2661609691 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b8c09d6f0 0x7f7b8c131670 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c009c40 tx=0x7f7b7c004750 comp rx=0 tx=0).stop 2023-10-03T10:17:10.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.982+0000 7f7b90edc700 1 -- 172.21.15.3:0/2661609691 shutdown_connections 2023-10-03T10:17:10.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.982+0000 7f7b90edc700 1 --2- 172.21.15.3:0/2661609691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b8c09e410 0x7f7b8c1320f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.982+0000 7f7b90edc700 1 --2- 172.21.15.3:0/2661609691 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b8c09d6f0 0x7f7b8c131670 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.982+0000 7f7b90edc700 1 --2- 172.21.15.3:0/2661609691 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b8c09e000 0x7f7b8c131bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:10.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.982+0000 7f7b90edc700 1 -- 172.21.15.3:0/2661609691 >> 172.21.15.3:0/2661609691 conn(0x7f7b8c066f50 msgr2=0x7f7b8c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:10.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.982+0000 7f7b90edc700 1 -- 172.21.15.3:0/2661609691 shutdown_connections 2023-10-03T10:17:10.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:10.982+0000 7f7b90edc700 1 -- 172.21.15.3:0/2661609691 wait complete. 2023-10-03T10:17:10.995 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:11.556 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:11.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.702+0000 7fd1bf39d700 1 Processor -- start 2023-10-03T10:17:11.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.702+0000 7fd1bf39d700 1 -- start start 2023-10-03T10:17:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.702+0000 7fd1bf39d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1b809f410 0x7fd1b8136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.702+0000 7fd1bf39d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd1b8136fe0 0x7fd1b81393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.702+0000 7fd1bf39d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1b8070940 con 0x7fd1b809e930 2023-10-03T10:17:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.702+0000 7fd1bf39d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1b806fd60 con 0x7fd1b8136fe0 2023-10-03T10:17:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.702+0000 7fd1bf39d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b806fa80 con 0x7fd1b809f410 2023-10-03T10:17:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.703+0000 7fd1bd139700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd1b809e930 0x7fd1b809ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:46098/0 (socket says 172.21.15.3:46098) 2023-10-03T10:17:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.703+0000 7fd1bd139700 1 -- 172.21.15.3:0/2675614265 learned_addr learned my addr 172.21.15.3:0/2675614265 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.703+0000 7fd1bc938700 1 --2- 172.21.15.3:0/2675614265 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1b809f410 0x7fd1b8136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.703+0000 7fd1bd93a700 1 --2- 172.21.15.3:0/2675614265 >> v2:172.21.15.3:3300/0 conn(0x7fd1b8136fe0 0x7fd1b81393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.703+0000 7fd1bc938700 1 -- 172.21.15.3:0/2675614265 >> v2:172.21.15.3:3300/0 conn(0x7fd1b8136fe0 msgr2=0x7fd1b81393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.703+0000 7fd1bc938700 1 --2- 172.21.15.3:0/2675614265 >> v2:172.21.15.3:3300/0 conn(0x7fd1b8136fe0 0x7fd1b81393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.703+0000 7fd1bc938700 1 -- 172.21.15.3:0/2675614265 >> v1:172.21.15.3:6789/0 conn(0x7fd1b809e930 legacy=0x7fd1b809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.703+0000 7fd1bc938700 1 -- 172.21.15.3:0/2675614265 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b809fb60 con 0x7fd1b809f410 2023-10-03T10:17:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.703+0000 7fd1bd93a700 1 --2- 172.21.15.3:0/2675614265 >> v2:172.21.15.3:3300/0 conn(0x7fd1b8136fe0 0x7fd1b81393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.703+0000 7fd1bc938700 1 --2- 172.21.15.3:0/2675614265 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1b809f410 0x7fd1b8136aa0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009840 tx=0x7fd1b4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80f4a4da771584d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.703+0000 7fd1abfff700 1 -- 172.21.15.3:0/2675614265 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4011830 con 0x7fd1b809f410 2023-10-03T10:17:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.703+0000 7fd1abfff700 1 -- 172.21.15.3:0/2675614265 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b4011990 con 0x7fd1b809f410 2023-10-03T10:17:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.703+0000 7fd1abfff700 1 -- 172.21.15.3:0/2675614265 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4011d20 con 0x7fd1b809f410 2023-10-03T10:17:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.704+0000 7fd1bf39d700 1 -- 172.21.15.3:0/2675614265 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1b809f410 msgr2=0x7fd1b8136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.704+0000 7fd1bf39d700 1 --2- 172.21.15.3:0/2675614265 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1b809f410 0x7fd1b8136aa0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009840 tx=0x7fd1b4009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.704+0000 7fd1bf39d700 1 -- 172.21.15.3:0/2675614265 shutdown_connections 2023-10-03T10:17:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.704+0000 7fd1bf39d700 1 --2- 172.21.15.3:0/2675614265 >> v2:172.21.15.3:3300/0 conn(0x7fd1b8136fe0 0x7fd1b81393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.704+0000 7fd1bf39d700 1 --2- 172.21.15.3:0/2675614265 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1b809f410 0x7fd1b8136aa0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.704+0000 7fd1bf39d700 1 -- 172.21.15.3:0/2675614265 >> 172.21.15.3:0/2675614265 conn(0x7fd1b80a4210 msgr2=0x7fd1b80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.704+0000 7fd1bf39d700 1 -- 172.21.15.3:0/2675614265 shutdown_connections 2023-10-03T10:17:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.704+0000 7fd1bf39d700 1 -- 172.21.15.3:0/2675614265 wait complete. 2023-10-03T10:17:11.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.705+0000 7fd1bf39d700 1 Processor -- start 2023-10-03T10:17:11.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.705+0000 7fd1bf39d700 1 -- start start 2023-10-03T10:17:11.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.706+0000 7fd1bf39d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd1b809e930 0x7fd1b811c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:11.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.706+0000 7fd1bf39d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1b809f410 0x7fd1b811cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:11.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.706+0000 7fd1bf39d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1b8136fe0 0x7fd1b811d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:11.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.706+0000 7fd1bf39d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b811fe50 con 0x7fd1b809f410 2023-10-03T10:17:11.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.706+0000 7fd1bf39d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b80a2f40 con 0x7fd1b8136fe0 2023-10-03T10:17:11.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.706+0000 7fd1bf39d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b811f8a0 con 0x7fd1b809e930 2023-10-03T10:17:11.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.706+0000 7fd1bc938700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1b809f410 0x7fd1b811cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:11.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.706+0000 7fd1bd93a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1b8136fe0 0x7fd1b811d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:11.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.706+0000 7fd1bc938700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1b809f410 0x7fd1b811cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37170/0 (socket says 172.21.15.3:37170) 2023-10-03T10:17:11.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.706+0000 7fd1bd93a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1b8136fe0 0x7fd1b811d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50542/0 (socket says 172.21.15.3:50542) 2023-10-03T10:17:11.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.706+0000 7fd1bc938700 1 -- 172.21.15.3:0/1043064513 learned_addr learned my addr 172.21.15.3:0/1043064513 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:11.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.706+0000 7fd1bd139700 1 --2- 172.21.15.3:0/1043064513 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd1b809e930 0x7fd1b811c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:11.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.706+0000 7fd1bc938700 1 -- 172.21.15.3:0/1043064513 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd1b809e930 msgr2=0x7fd1b811c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:11.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.706+0000 7fd1bc938700 1 --2- 172.21.15.3:0/1043064513 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd1b809e930 0x7fd1b811c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:11.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.706+0000 7fd1bc938700 1 -- 172.21.15.3:0/1043064513 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1b8136fe0 msgr2=0x7fd1b811d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:11.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.706+0000 7fd1bc938700 1 --2- 172.21.15.3:0/1043064513 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1b8136fe0 0x7fd1b811d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:11.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.706+0000 7fd1bc938700 1 -- 172.21.15.3:0/1043064513 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b0009a70 con 0x7fd1b809f410 2023-10-03T10:17:11.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.706+0000 7fd1bc938700 1 --2- 172.21.15.3:0/1043064513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1b809f410 0x7fd1b811cba0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009fd0 tx=0x7fd1b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:11.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.706+0000 7fd1a9ffb700 1 -- 172.21.15.3:0/1043064513 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b40036a0 con 0x7fd1b809f410 2023-10-03T10:17:11.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.706+0000 7fd1a9ffb700 1 -- 172.21.15.3:0/1043064513 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b401f070 con 0x7fd1b809f410 2023-10-03T10:17:11.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.706+0000 7fd1bf39d700 1 -- 172.21.15.3:0/1043064513 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b40094a0 con 0x7fd1b809f410 2023-10-03T10:17:11.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.706+0000 7fd1a9ffb700 1 -- 172.21.15.3:0/1043064513 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4003a60 con 0x7fd1b809f410 2023-10-03T10:17:11.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.707+0000 7fd1bf39d700 1 -- 172.21.15.3:0/1043064513 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b811e330 con 0x7fd1b809f410 2023-10-03T10:17:11.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.707+0000 7fd1a9ffb700 1 -- 172.21.15.3:0/1043064513 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd1b4003bc0 con 0x7fd1b809f410 2023-10-03T10:17:11.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.708+0000 7fd1a9ffb700 1 -- 172.21.15.3:0/1043064513 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd1b4021070 con 0x7fd1b809f410 2023-10-03T10:17:11.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.708+0000 7fd1bf39d700 1 -- 172.21.15.3:0/1043064513 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1b811ea90 con 0x7fd1b809f410 2023-10-03T10:17:11.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.713+0000 7fd1a9ffb700 1 -- 172.21.15.3:0/1043064513 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd1b402b020 con 0x7fd1b809f410 2023-10-03T10:17:11.890 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:11.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.916+0000 7fd1bf39d700 1 -- 172.21.15.3:0/1043064513 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd1b8074ea0 con 0x7fd1b809f410 2023-10-03T10:17:11.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.916+0000 7fd1a9ffb700 1 -- 172.21.15.3:0/1043064513 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd1b4034070 con 0x7fd1b809f410 2023-10-03T10:17:11.917 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:11.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.919+0000 7fd1bf39d700 1 -- 172.21.15.3:0/1043064513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1b809f410 msgr2=0x7fd1b811cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:11.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.919+0000 7fd1bf39d700 1 --2- 172.21.15.3:0/1043064513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1b809f410 0x7fd1b811cba0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009fd0 tx=0x7fd1b4004750 comp rx=0 tx=0).stop 2023-10-03T10:17:11.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.919+0000 7fd1bf39d700 1 -- 172.21.15.3:0/1043064513 shutdown_connections 2023-10-03T10:17:11.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.919+0000 7fd1bf39d700 1 --2- 172.21.15.3:0/1043064513 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1b8136fe0 0x7fd1b811d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:11.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.919+0000 7fd1bf39d700 1 --2- 172.21.15.3:0/1043064513 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd1b809e930 0x7fd1b811c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:11.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.919+0000 7fd1bf39d700 1 --2- 172.21.15.3:0/1043064513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1b809f410 0x7fd1b811cba0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:11.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.919+0000 7fd1bf39d700 1 -- 172.21.15.3:0/1043064513 >> 172.21.15.3:0/1043064513 conn(0x7fd1b80a4210 msgr2=0x7fd1b80a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:11.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.919+0000 7fd1bf39d700 1 -- 172.21.15.3:0/1043064513 shutdown_connections 2023-10-03T10:17:11.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:11.919+0000 7fd1bf39d700 1 -- 172.21.15.3:0/1043064513 wait complete. 2023-10-03T10:17:11.926 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:11.933 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:11.996 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:12.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.032+0000 7fd8dd3c5700 1 Processor -- start 2023-10-03T10:17:12.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.032+0000 7fd8dd3c5700 1 -- start start 2023-10-03T10:17:12.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.033+0000 7fd8dd3c5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd8d8133420 0x7fd8d81337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:12.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.033+0000 7fd8dd3c5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8d8133e30 0x7fd8d812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:12.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.033+0000 7fd8dd3c5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd8d8134260 0x7fd8d81304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:12.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.033+0000 7fd8dd3c5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8d806b990 con 0x7fd8d8133420 2023-10-03T10:17:12.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.033+0000 7fd8dd3c5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d80a0590 con 0x7fd8d8133e30 2023-10-03T10:17:12.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.033+0000 7fd8dd3c5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d809fb80 con 0x7fd8d8134260 2023-10-03T10:17:12.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.033+0000 7fd8d6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd8d8133420 0x7fd8d81337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:12.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.033+0000 7fd8d6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd8d8133420 0x7fd8d81337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37180/0 (socket says 172.21.15.3:37180) 2023-10-03T10:17:12.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.033+0000 7fd8d6ffd700 1 -- 172.21.15.3:0/1898244847 learned_addr learned my addr 172.21.15.3:0/1898244847 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:12.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.033+0000 7fd8ce7fc700 1 --2- 172.21.15.3:0/1898244847 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8d8133e30 0x7fd8d812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:12.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.033+0000 7fd8d77fe700 1 --2- 172.21.15.3:0/1898244847 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd8d8134260 0x7fd8d81304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:12.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.033+0000 7fd8d6ffd700 1 -- 172.21.15.3:0/1898244847 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd8d8134260 msgr2=0x7fd8d81304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:12.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.033+0000 7fd8d6ffd700 1 --2- 172.21.15.3:0/1898244847 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd8d8134260 0x7fd8d81304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:12.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.033+0000 7fd8d6ffd700 1 -- 172.21.15.3:0/1898244847 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8d8133e30 msgr2=0x7fd8d812ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:12.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.033+0000 7fd8d6ffd700 1 --2- 172.21.15.3:0/1898244847 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8d8133e30 0x7fd8d812ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:12.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.033+0000 7fd8d6ffd700 1 -- 172.21.15.3:0/1898244847 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d80a0030 con 0x7fd8d8133420 2023-10-03T10:17:12.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.033+0000 7fd8ce7fc700 1 --2- 172.21.15.3:0/1898244847 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8d8133e30 0x7fd8d812ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:12.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.033+0000 7fd8d6ffd700 1 --2- 172.21.15.3:0/1898244847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8d8133420 0x7fd8d81337f0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80098f0 tx=0x7fd8c8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a84663b4aaf2c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:12.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.034+0000 7fd8d67fc700 1 -- 172.21.15.3:0/1898244847 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8011830 con 0x7fd8d8133420 2023-10-03T10:17:12.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.034+0000 7fd8d67fc700 1 -- 172.21.15.3:0/1898244847 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c8011990 con 0x7fd8d8133420 2023-10-03T10:17:12.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.034+0000 7fd8d67fc700 1 -- 172.21.15.3:0/1898244847 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8011d20 con 0x7fd8d8133420 2023-10-03T10:17:12.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.034+0000 7fd8dd3c5700 1 -- 172.21.15.3:0/1898244847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8d8133420 msgr2=0x7fd8d81337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:12.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.034+0000 7fd8dd3c5700 1 --2- 172.21.15.3:0/1898244847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8d8133420 0x7fd8d81337f0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80098f0 tx=0x7fd8c8009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:12.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.034+0000 7fd8dd3c5700 1 -- 172.21.15.3:0/1898244847 shutdown_connections 2023-10-03T10:17:12.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.034+0000 7fd8dd3c5700 1 --2- 172.21.15.3:0/1898244847 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd8d8134260 0x7fd8d81304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:12.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.034+0000 7fd8dd3c5700 1 --2- 172.21.15.3:0/1898244847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8d8133420 0x7fd8d81337f0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:12.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.034+0000 7fd8dd3c5700 1 --2- 172.21.15.3:0/1898244847 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8d8133e30 0x7fd8d812ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:12.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.034+0000 7fd8dd3c5700 1 -- 172.21.15.3:0/1898244847 >> 172.21.15.3:0/1898244847 conn(0x7fd8d80a4210 msgr2=0x7fd8d80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:12.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.035+0000 7fd8dd3c5700 1 -- 172.21.15.3:0/1898244847 shutdown_connections 2023-10-03T10:17:12.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.035+0000 7fd8dd3c5700 1 -- 172.21.15.3:0/1898244847 wait complete. 2023-10-03T10:17:12.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.036+0000 7fd8dd3c5700 1 Processor -- start 2023-10-03T10:17:12.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.036+0000 7fd8dd3c5700 1 -- start start 2023-10-03T10:17:12.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.036+0000 7fd8dd3c5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8d8133420 0x7fd8d809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:12.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.036+0000 7fd8dd3c5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd8d8133e30 0x7fd8d809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:12.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.036+0000 7fd8dd3c5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8d8134260 0x7fd8d809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:12.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.036+0000 7fd8dd3c5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d8131400 con 0x7fd8d8133420 2023-10-03T10:17:12.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.036+0000 7fd8dd3c5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d809dca0 con 0x7fd8d8134260 2023-10-03T10:17:12.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.036+0000 7fd8dd3c5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d811fee0 con 0x7fd8d8133e30 2023-10-03T10:17:12.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.036+0000 7fd8d6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8d8133420 0x7fd8d809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:12.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.036+0000 7fd8ce7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd8d8133e30 0x7fd8d809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:12.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.036+0000 7fd8d77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8d8134260 0x7fd8d809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:12.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.037+0000 7fd8d6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8d8133420 0x7fd8d809c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37186/0 (socket says 172.21.15.3:37186) 2023-10-03T10:17:12.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.037+0000 7fd8ce7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd8d8133e30 0x7fd8d809c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50126/0 (socket says 172.21.15.3:50126) 2023-10-03T10:17:12.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.037+0000 7fd8d6ffd700 1 -- 172.21.15.3:0/445895406 learned_addr learned my addr 172.21.15.3:0/445895406 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:12.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.037+0000 7fd8d77fe700 1 -- 172.21.15.3:0/445895406 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd8d8133e30 msgr2=0x7fd8d809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:12.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.037+0000 7fd8d77fe700 1 --2- 172.21.15.3:0/445895406 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd8d8133e30 0x7fd8d809c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:12.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.037+0000 7fd8d77fe700 1 -- 172.21.15.3:0/445895406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8d8133420 msgr2=0x7fd8d809c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:12.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.037+0000 7fd8d77fe700 1 --2- 172.21.15.3:0/445895406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8d8133420 0x7fd8d809c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:12.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.037+0000 7fd8d77fe700 1 -- 172.21.15.3:0/445895406 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c00094a0 con 0x7fd8d8134260 2023-10-03T10:17:12.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.037+0000 7fd8ce7fc700 1 --2- 172.21.15.3:0/445895406 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd8d8133e30 0x7fd8d809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:12.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.037+0000 7fd8d6ffd700 1 --2- 172.21.15.3:0/445895406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8d8133420 0x7fd8d809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:12.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.037+0000 7fd8d77fe700 1 --2- 172.21.15.3:0/445895406 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8d8134260 0x7fd8d809cf00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd8d000ef90 tx=0x7fd8d000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:12.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.037+0000 7fd8cffff700 1 -- 172.21.15.3:0/445895406 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d000cc90 con 0x7fd8d8134260 2023-10-03T10:17:12.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.037+0000 7fd8cffff700 1 -- 172.21.15.3:0/445895406 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d000f040 con 0x7fd8d8134260 2023-10-03T10:17:12.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.037+0000 7fd8dd3c5700 1 -- 172.21.15.3:0/445895406 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c8009550 con 0x7fd8d8134260 2023-10-03T10:17:12.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.037+0000 7fd8cffff700 1 -- 172.21.15.3:0/445895406 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d00146c0 con 0x7fd8d8134260 2023-10-03T10:17:12.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.037+0000 7fd8dd3c5700 1 -- 172.21.15.3:0/445895406 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d811fbb0 con 0x7fd8d8134260 2023-10-03T10:17:12.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.038+0000 7fd8cffff700 1 -- 172.21.15.3:0/445895406 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd8d0014820 con 0x7fd8d8134260 2023-10-03T10:17:12.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.039+0000 7fd8cffff700 1 -- 172.21.15.3:0/445895406 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd8d00400f0 con 0x7fd8d8134260 2023-10-03T10:17:12.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.039+0000 7fd8dd3c5700 1 -- 172.21.15.3:0/445895406 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8d8074ea0 con 0x7fd8d8134260 2023-10-03T10:17:12.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.044+0000 7fd8cffff700 1 -- 172.21.15.3:0/445895406 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd8d0014ad0 con 0x7fd8d8134260 2023-10-03T10:17:12.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.070+0000 7fb3f0543700 1 Processor -- start 2023-10-03T10:17:12.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.070+0000 7fb3f0543700 1 -- start start 2023-10-03T10:17:12.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.071+0000 7fb3f0543700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb3e809d630 0x7fb3e80a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:12.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.071+0000 7fb3f0543700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb3e8138d80 0x7fb3e813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:12.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.071+0000 7fb3f0543700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e80a2a70 con 0x7fb3e80a0a30 2023-10-03T10:17:12.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.071+0000 7fb3f0543700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e80a01d0 con 0x7fb3e809d630 2023-10-03T10:17:12.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.071+0000 7fb3f0543700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e809e2b0 con 0x7fb3e8138d80 2023-10-03T10:17:12.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.071+0000 7fb3edade700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb3e809d630 0x7fb3e80a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:12.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.071+0000 7fb3edade700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb3e809d630 0x7fb3e80a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37194/0 (socket says 172.21.15.3:37194) 2023-10-03T10:17:12.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.071+0000 7fb3edade700 1 -- 172.21.15.3:0/2957719988 learned_addr learned my addr 172.21.15.3:0/2957719988 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:12.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.071+0000 7fb3eeae0700 1 --2- 172.21.15.3:0/2957719988 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb3e8138d80 0x7fb3e813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:12.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.071+0000 7fb3edade700 1 -- 172.21.15.3:0/2957719988 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb3e8138d80 msgr2=0x7fb3e813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:12.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.071+0000 7fb3edade700 1 --2- 172.21.15.3:0/2957719988 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb3e8138d80 0x7fb3e813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:12.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.071+0000 7fb3edade700 1 -- 172.21.15.3:0/2957719988 >> v1:172.21.15.3:6789/0 conn(0x7fb3e80a0a30 legacy=0x7fb3e80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:12.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.072+0000 7fb3edade700 1 -- 172.21.15.3:0/2957719988 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e809dc50 con 0x7fb3e809d630 2023-10-03T10:17:12.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.072+0000 7fb3edade700 1 --2- 172.21.15.3:0/2957719988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3e809d630 0x7fb3e80a1510 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fb3d80098f0 tx=0x7fb3d8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a607a4f9e9523 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:12.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.072+0000 7fb3ed2dd700 1 -- 172.21.15.3:0/2957719988 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8011830 con 0x7fb3e809d630 2023-10-03T10:17:12.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.072+0000 7fb3ed2dd700 1 -- 172.21.15.3:0/2957719988 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d8011990 con 0x7fb3e809d630 2023-10-03T10:17:12.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.072+0000 7fb3ed2dd700 1 -- 172.21.15.3:0/2957719988 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8011d20 con 0x7fb3e809d630 2023-10-03T10:17:12.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.073+0000 7fb3f0543700 1 -- 172.21.15.3:0/2957719988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3e809d630 msgr2=0x7fb3e80a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:12.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.073+0000 7fb3f0543700 1 --2- 172.21.15.3:0/2957719988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3e809d630 0x7fb3e80a1510 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fb3d80098f0 tx=0x7fb3d8009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:12.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.073+0000 7fb3f0543700 1 -- 172.21.15.3:0/2957719988 shutdown_connections 2023-10-03T10:17:12.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.073+0000 7fb3f0543700 1 --2- 172.21.15.3:0/2957719988 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb3e8138d80 0x7fb3e813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:12.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.073+0000 7fb3f0543700 1 --2- 172.21.15.3:0/2957719988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3e809d630 0x7fb3e80a1510 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:12.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.073+0000 7fb3f0543700 1 -- 172.21.15.3:0/2957719988 >> 172.21.15.3:0/2957719988 conn(0x7fb3e80a3050 msgr2=0x7fb3e80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:12.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.073+0000 7fb3f0543700 1 -- 172.21.15.3:0/2957719988 shutdown_connections 2023-10-03T10:17:12.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.073+0000 7fb3f0543700 1 -- 172.21.15.3:0/2957719988 wait complete. 2023-10-03T10:17:12.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.074+0000 7fb3f0543700 1 Processor -- start 2023-10-03T10:17:12.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.074+0000 7fb3f0543700 1 -- start start 2023-10-03T10:17:12.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.074+0000 7fb3f0543700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb3e809d630 0x7fb3e8133840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:12.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.075+0000 7fb3f0543700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3e80a0a30 0x7fb3e8133d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:12.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.075+0000 7fb3f0543700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb3e8138d80 0x7fb3e81342c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:12.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.075+0000 7fb3f0543700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e8134d10 con 0x7fb3e80a0a30 2023-10-03T10:17:12.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.075+0000 7fb3f0543700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e8138020 con 0x7fb3e8138d80 2023-10-03T10:17:12.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.075+0000 7fb3f0543700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e8137470 con 0x7fb3e809d630 2023-10-03T10:17:12.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.075+0000 7fb3edade700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3e80a0a30 0x7fb3e8133d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:12.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.075+0000 7fb3ee2df700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb3e809d630 0x7fb3e8133840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:12.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.075+0000 7fb3edade700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3e80a0a30 0x7fb3e8133d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37196/0 (socket says 172.21.15.3:37196) 2023-10-03T10:17:12.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.075+0000 7fb3edade700 1 -- 172.21.15.3:0/1130743854 learned_addr learned my addr 172.21.15.3:0/1130743854 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:12.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.075+0000 7fb3edade700 1 -- 172.21.15.3:0/1130743854 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb3e809d630 msgr2=0x7fb3e8133840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:12.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.075+0000 7fb3edade700 1 --2- 172.21.15.3:0/1130743854 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb3e809d630 0x7fb3e8133840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:12.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.075+0000 7fb3edade700 1 -- 172.21.15.3:0/1130743854 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb3e8138d80 msgr2=0x7fb3e81342c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:12.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.075+0000 7fb3edade700 1 --2- 172.21.15.3:0/1130743854 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb3e8138d80 0x7fb3e81342c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:12.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.075+0000 7fb3edade700 1 -- 172.21.15.3:0/1130743854 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d8009550 con 0x7fb3e80a0a30 2023-10-03T10:17:12.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.075+0000 7fb3edade700 1 --2- 172.21.15.3:0/1130743854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3e80a0a30 0x7fb3e8133d80 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8009c40 tx=0x7fb3d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:12.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.075+0000 7fb3deffd700 1 -- 172.21.15.3:0/1130743854 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d80036a0 con 0x7fb3e80a0a30 2023-10-03T10:17:12.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.076+0000 7fb3deffd700 1 -- 172.21.15.3:0/1130743854 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d8003800 con 0x7fb3e80a0a30 2023-10-03T10:17:12.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.076+0000 7fb3f0543700 1 -- 172.21.15.3:0/1130743854 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e80a0410 con 0x7fb3e80a0a30 2023-10-03T10:17:12.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.076+0000 7fb3deffd700 1 -- 172.21.15.3:0/1130743854 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8003b90 con 0x7fb3e80a0a30 2023-10-03T10:17:12.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.076+0000 7fb3f0543700 1 -- 172.21.15.3:0/1130743854 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e8137080 con 0x7fb3e80a0a30 2023-10-03T10:17:12.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.077+0000 7fb3f0543700 1 -- 172.21.15.3:0/1130743854 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e80747f0 con 0x7fb3e80a0a30 2023-10-03T10:17:12.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.077+0000 7fb3deffd700 1 -- 172.21.15.3:0/1130743854 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb3d8021070 con 0x7fb3e80a0a30 2023-10-03T10:17:12.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.077+0000 7fb3deffd700 1 -- 172.21.15.3:0/1130743854 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb3d8040a80 con 0x7fb3e80a0a30 2023-10-03T10:17:12.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.082+0000 7fb3deffd700 1 -- 172.21.15.3:0/1130743854 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb3d800b3e0 con 0x7fb3e80a0a30 2023-10-03T10:17:12.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.140+0000 7fbf7d8aa700 1 Processor -- start 2023-10-03T10:17:12.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.140+0000 7fbf7d8aa700 1 -- start start 2023-10-03T10:17:12.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.141+0000 7fbf7d8aa700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf780a0a30 0x7fbf780a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:12.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.141+0000 7fbf7d8aa700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf7809d630 0x7fbf780a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:12.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.141+0000 7fbf7d8aa700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf780a2a70 con 0x7fbf78133930 2023-10-03T10:17:12.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.141+0000 7fbf7d8aa700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf780a01d0 con 0x7fbf7809d630 2023-10-03T10:17:12.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.141+0000 7fbf7d8aa700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7809e2b0 con 0x7fbf780a0a30 2023-10-03T10:17:12.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.141+0000 7fbf777fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbf78133930 0x7fbf78133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:46122/0 (socket says 172.21.15.3:46122) 2023-10-03T10:17:12.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.141+0000 7fbf76ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf780a0a30 0x7fbf780a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:12.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.141+0000 7fbf777fe700 1 -- 172.21.15.3:0/920883100 learned_addr learned my addr 172.21.15.3:0/920883100 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:12.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.141+0000 7fbf767fc700 1 --2- 172.21.15.3:0/920883100 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf7809d630 0x7fbf780a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:12.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.142+0000 7fbf75ffb700 1 -- 172.21.15.3:0/920883100 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2665817057 0 0) 0x7fbf780a2a70 con 0x7fbf78133930 2023-10-03T10:17:12.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.142+0000 7fbf75ffb700 1 -- 172.21.15.3:0/920883100 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf600035b0 con 0x7fbf78133930 2023-10-03T10:17:12.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.142+0000 7fbf76ffd700 1 -- 172.21.15.3:0/920883100 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf7809d630 msgr2=0x7fbf780a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:12.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.142+0000 7fbf76ffd700 1 --2- 172.21.15.3:0/920883100 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf7809d630 0x7fbf780a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:12.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.142+0000 7fbf76ffd700 1 -- 172.21.15.3:0/920883100 >> v1:172.21.15.3:6789/0 conn(0x7fbf78133930 legacy=0x7fbf78133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:12.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.142+0000 7fbf76ffd700 1 -- 172.21.15.3:0/920883100 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7809dc50 con 0x7fbf780a0a30 2023-10-03T10:17:12.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.142+0000 7fbf767fc700 1 --2- 172.21.15.3:0/920883100 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf7809d630 0x7fbf780a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:12.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.142+0000 7fbf76ffd700 1 --2- 172.21.15.3:0/920883100 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf780a0a30 0x7fbf780a0e00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009870 tx=0x7fbf68009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7d15a31e3af05e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:12.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.142+0000 7fbf75ffb700 1 -- 172.21.15.3:0/920883100 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf68004230 con 0x7fbf780a0a30 2023-10-03T10:17:12.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.142+0000 7fbf75ffb700 1 -- 172.21.15.3:0/920883100 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf68004390 con 0x7fbf780a0a30 2023-10-03T10:17:12.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.142+0000 7fbf75ffb700 1 -- 172.21.15.3:0/920883100 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf680036a0 con 0x7fbf780a0a30 2023-10-03T10:17:12.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.143+0000 7fbf7d8aa700 1 -- 172.21.15.3:0/920883100 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf780a0a30 msgr2=0x7fbf780a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:12.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.143+0000 7fbf7d8aa700 1 --2- 172.21.15.3:0/920883100 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf780a0a30 0x7fbf780a0e00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009870 tx=0x7fbf68009b80 comp rx=0 tx=0).stop 2023-10-03T10:17:12.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.143+0000 7fbf7d8aa700 1 -- 172.21.15.3:0/920883100 shutdown_connections 2023-10-03T10:17:12.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.143+0000 7fbf7d8aa700 1 --2- 172.21.15.3:0/920883100 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf780a0a30 0x7fbf780a0e00 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:12.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.143+0000 7fbf7d8aa700 1 --2- 172.21.15.3:0/920883100 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf7809d630 0x7fbf780a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:12.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.143+0000 7fbf7d8aa700 1 -- 172.21.15.3:0/920883100 >> 172.21.15.3:0/920883100 conn(0x7fbf780a3050 msgr2=0x7fbf780a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:12.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.143+0000 7fbf7d8aa700 1 -- 172.21.15.3:0/920883100 shutdown_connections 2023-10-03T10:17:12.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.143+0000 7fbf7d8aa700 1 -- 172.21.15.3:0/920883100 wait complete. 2023-10-03T10:17:12.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.144+0000 7fbf7d8aa700 1 Processor -- start 2023-10-03T10:17:12.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.144+0000 7fbf7d8aa700 1 -- start start 2023-10-03T10:17:12.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.145+0000 7fbf7d8aa700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf7809d630 0x7fbf7811f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:12.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.145+0000 7fbf7d8aa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf780a0a30 0x7fbf7811f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:12.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.145+0000 7fbf7d8aa700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf78133930 0x7fbf781c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:12.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.145+0000 7fbf7d8aa700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf78122340 con 0x7fbf780a0a30 2023-10-03T10:17:12.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.145+0000 7fbf7d8aa700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf781333b0 con 0x7fbf78133930 2023-10-03T10:17:12.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.145+0000 7fbf7d8aa700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7811ef00 con 0x7fbf7809d630 2023-10-03T10:17:12.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.145+0000 7fbf76ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf7809d630 0x7fbf7811f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:12.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.145+0000 7fbf767fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf780a0a30 0x7fbf7811f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:12.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.145+0000 7fbf76ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf7809d630 0x7fbf7811f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50174/0 (socket says 172.21.15.3:50174) 2023-10-03T10:17:12.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.145+0000 7fbf76ffd700 1 -- 172.21.15.3:0/1983003120 learned_addr learned my addr 172.21.15.3:0/1983003120 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:12.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.145+0000 7fbf777fe700 1 --2- 172.21.15.3:0/1983003120 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf78133930 0x7fbf781c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:12.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.145+0000 7fbf76ffd700 1 -- 172.21.15.3:0/1983003120 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf78133930 msgr2=0x7fbf781c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:12.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.145+0000 7fbf76ffd700 1 --2- 172.21.15.3:0/1983003120 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf78133930 0x7fbf781c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:12.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.145+0000 7fbf76ffd700 1 -- 172.21.15.3:0/1983003120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf780a0a30 msgr2=0x7fbf7811f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:12.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.145+0000 7fbf76ffd700 1 --2- 172.21.15.3:0/1983003120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf780a0a30 0x7fbf7811f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:12.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.145+0000 7fbf76ffd700 1 -- 172.21.15.3:0/1983003120 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf6c0094a0 con 0x7fbf7809d630 2023-10-03T10:17:12.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.145+0000 7fbf767fc700 1 --2- 172.21.15.3:0/1983003120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf780a0a30 0x7fbf7811f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:12.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.145+0000 7fbf76ffd700 1 --2- 172.21.15.3:0/1983003120 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf7809d630 0x7fbf7811f480 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fbf68000c00 tx=0x7fbf68004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:12.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.146+0000 7fbf5ffff700 1 -- 172.21.15.3:0/1983003120 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf68003cf0 con 0x7fbf7809d630 2023-10-03T10:17:12.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.146+0000 7fbf5ffff700 1 -- 172.21.15.3:0/1983003120 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf6801e070 con 0x7fbf7809d630 2023-10-03T10:17:12.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.146+0000 7fbf7d8aa700 1 -- 172.21.15.3:0/1983003120 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf680094a0 con 0x7fbf7809d630 2023-10-03T10:17:12.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.146+0000 7fbf5ffff700 1 -- 172.21.15.3:0/1983003120 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf68011870 con 0x7fbf7809d630 2023-10-03T10:17:12.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.146+0000 7fbf7d8aa700 1 -- 172.21.15.3:0/1983003120 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf781225a0 con 0x7fbf7809d630 2023-10-03T10:17:12.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.147+0000 7fbf5ffff700 1 -- 172.21.15.3:0/1983003120 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbf680119d0 con 0x7fbf7809d630 2023-10-03T10:17:12.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.147+0000 7fbf7d8aa700 1 -- 172.21.15.3:0/1983003120 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf780a0410 con 0x7fbf7809d630 2023-10-03T10:17:12.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.147+0000 7fbf5ffff700 1 -- 172.21.15.3:0/1983003120 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fbf68020070 con 0x7fbf7809d630 2023-10-03T10:17:12.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.152+0000 7fbf5ffff700 1 -- 172.21.15.3:0/1983003120 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbf680243e0 con 0x7fbf7809d630 2023-10-03T10:17:12.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.254+0000 7fd8dd3c5700 1 -- 172.21.15.3:0/445895406 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd8d81209a0 con 0x7fd8d8134260 2023-10-03T10:17:12.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.254+0000 7fd8cffff700 1 -- 172.21.15.3:0/445895406 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd8d0012340 con 0x7fd8d8134260 2023-10-03T10:17:12.254 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:12.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.256+0000 7fd8dd3c5700 1 -- 172.21.15.3:0/445895406 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8d8134260 msgr2=0x7fd8d809cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:12.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.256+0000 7fd8dd3c5700 1 --2- 172.21.15.3:0/445895406 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8d8134260 0x7fd8d809cf00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd8d000ef90 tx=0x7fd8d000c3a0 comp rx=0 tx=0).stop 2023-10-03T10:17:12.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.256+0000 7fd8dd3c5700 1 -- 172.21.15.3:0/445895406 shutdown_connections 2023-10-03T10:17:12.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.256+0000 7fd8dd3c5700 1 --2- 172.21.15.3:0/445895406 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8d8134260 0x7fd8d809cf00 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:12.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.256+0000 7fd8dd3c5700 1 --2- 172.21.15.3:0/445895406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8d8133420 0x7fd8d809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:12.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.256+0000 7fd8dd3c5700 1 --2- 172.21.15.3:0/445895406 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd8d8133e30 0x7fd8d809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:12.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.256+0000 7fd8dd3c5700 1 -- 172.21.15.3:0/445895406 >> 172.21.15.3:0/445895406 conn(0x7fd8d80a4210 msgr2=0x7fd8d81302a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:12.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.256+0000 7fd8dd3c5700 1 -- 172.21.15.3:0/445895406 shutdown_connections 2023-10-03T10:17:12.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.256+0000 7fd8dd3c5700 1 -- 172.21.15.3:0/445895406 wait complete. 2023-10-03T10:17:12.270 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:12.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.295+0000 7fb3f0543700 1 -- 172.21.15.3:0/1130743854 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb3e8135c40 con 0x7fb3e80a0a30 2023-10-03T10:17:12.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.295+0000 7fb3deffd700 1 -- 172.21.15.3:0/1130743854 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb3d8015030 con 0x7fb3e80a0a30 2023-10-03T10:17:12.298 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:12.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.298+0000 7fb3f0543700 1 -- 172.21.15.3:0/1130743854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3e80a0a30 msgr2=0x7fb3e8133d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:12.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.298+0000 7fb3f0543700 1 --2- 172.21.15.3:0/1130743854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3e80a0a30 0x7fb3e8133d80 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8009c40 tx=0x7fb3d8004750 comp rx=0 tx=0).stop 2023-10-03T10:17:12.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.298+0000 7fb3f0543700 1 -- 172.21.15.3:0/1130743854 shutdown_connections 2023-10-03T10:17:12.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.298+0000 7fb3f0543700 1 --2- 172.21.15.3:0/1130743854 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb3e8138d80 0x7fb3e81342c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:12.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.298+0000 7fb3f0543700 1 --2- 172.21.15.3:0/1130743854 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb3e809d630 0x7fb3e8133840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:12.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.298+0000 7fb3f0543700 1 --2- 172.21.15.3:0/1130743854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3e80a0a30 0x7fb3e8133d80 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:12.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.298+0000 7fb3f0543700 1 -- 172.21.15.3:0/1130743854 >> 172.21.15.3:0/1130743854 conn(0x7fb3e80a3050 msgr2=0x7fb3e809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:12.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.298+0000 7fb3f0543700 1 -- 172.21.15.3:0/1130743854 shutdown_connections 2023-10-03T10:17:12.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.298+0000 7fb3f0543700 1 -- 172.21.15.3:0/1130743854 wait complete. 2023-10-03T10:17:12.313 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:12.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.367+0000 7fbf7d8aa700 1 -- 172.21.15.3:0/1983003120 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbf780747f0 con 0x7fbf7809d630 2023-10-03T10:17:12.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.367+0000 7fbf5ffff700 1 -- 172.21.15.3:0/1983003120 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbf6802a030 con 0x7fbf7809d630 2023-10-03T10:17:12.368 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:12.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.370+0000 7fbf7d8aa700 1 -- 172.21.15.3:0/1983003120 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf7809d630 msgr2=0x7fbf7811f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:12.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.370+0000 7fbf7d8aa700 1 --2- 172.21.15.3:0/1983003120 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf7809d630 0x7fbf7811f480 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fbf68000c00 tx=0x7fbf68004750 comp rx=0 tx=0).stop 2023-10-03T10:17:12.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.370+0000 7fbf7d8aa700 1 -- 172.21.15.3:0/1983003120 shutdown_connections 2023-10-03T10:17:12.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.370+0000 7fbf7d8aa700 1 --2- 172.21.15.3:0/1983003120 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf78133930 0x7fbf781c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:12.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.370+0000 7fbf7d8aa700 1 --2- 172.21.15.3:0/1983003120 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf7809d630 0x7fbf7811f480 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:12.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.370+0000 7fbf7d8aa700 1 --2- 172.21.15.3:0/1983003120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf780a0a30 0x7fbf7811f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:12.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.370+0000 7fbf7d8aa700 1 -- 172.21.15.3:0/1983003120 >> 172.21.15.3:0/1983003120 conn(0x7fbf780a3050 msgr2=0x7fbf7809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:12.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.370+0000 7fbf7d8aa700 1 -- 172.21.15.3:0/1983003120 shutdown_connections 2023-10-03T10:17:12.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:12.370+0000 7fbf7d8aa700 1 -- 172.21.15.3:0/1983003120 wait complete. 2023-10-03T10:17:12.383 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:12.934 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:13.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.080+0000 7fb90d3b6700 1 Processor -- start 2023-10-03T10:17:13.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.080+0000 7fb90d3b6700 1 -- start start 2023-10-03T10:17:13.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.080+0000 7fb90d3b6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb90809e850 0x7fb90809ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:13.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.081+0000 7fb90d3b6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb90809f550 0x7fb908134290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:13.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.081+0000 7fb90d3b6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb90806bec0 con 0x7fb90809f160 2023-10-03T10:17:13.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.081+0000 7fb90d3b6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb90806bce0 con 0x7fb90809f550 2023-10-03T10:17:13.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.081+0000 7fb90d3b6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb90806caa0 con 0x7fb90809e850 2023-10-03T10:17:13.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.081+0000 7fb9077fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb90809f550 0x7fb908134290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:13.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.081+0000 7fb906ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb90809e850 0x7fb90809ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:13.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.081+0000 7fb9077fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb90809f550 0x7fb908134290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37210/0 (socket says 172.21.15.3:37210) 2023-10-03T10:17:13.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.081+0000 7fb906ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb90809e850 0x7fb90809ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50182/0 (socket says 172.21.15.3:50182) 2023-10-03T10:17:13.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.081+0000 7fb9077fe700 1 -- 172.21.15.3:0/1251934901 learned_addr learned my addr 172.21.15.3:0/1251934901 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:13.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.081+0000 7fb9077fe700 1 -- 172.21.15.3:0/1251934901 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb90809e850 msgr2=0x7fb90809ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:13.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.081+0000 7fb9077fe700 1 --2- 172.21.15.3:0/1251934901 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb90809e850 0x7fb90809ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.081+0000 7fb9077fe700 1 -- 172.21.15.3:0/1251934901 >> v1:172.21.15.3:6789/0 conn(0x7fb90809f160 legacy=0x7fb908133b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:13.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.081+0000 7fb905ffb700 1 -- 172.21.15.3:0/1251934901 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1122475372 0 0) 0x7fb90806bec0 con 0x7fb90809f160 2023-10-03T10:17:13.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.081+0000 7fb9077fe700 1 -- 172.21.15.3:0/1251934901 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9080a00a0 con 0x7fb90809f550 2023-10-03T10:17:13.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.081+0000 7fb906ffd700 1 --2- 172.21.15.3:0/1251934901 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb90809e850 0x7fb90809ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:13.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.081+0000 7fb9077fe700 1 --2- 172.21.15.3:0/1251934901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb90809f550 0x7fb908134290 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fb900009e50 tx=0x7fb90000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576bb8bee4ae847b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:13.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.081+0000 7fb905ffb700 1 -- 172.21.15.3:0/1251934901 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90000e070 con 0x7fb90809f550 2023-10-03T10:17:13.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.082+0000 7fb905ffb700 1 -- 172.21.15.3:0/1251934901 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb90000b910 con 0x7fb90809f550 2023-10-03T10:17:13.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.082+0000 7fb905ffb700 1 -- 172.21.15.3:0/1251934901 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90000bcc0 con 0x7fb90809f550 2023-10-03T10:17:13.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.082+0000 7fb90d3b6700 1 -- 172.21.15.3:0/1251934901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb90809f550 msgr2=0x7fb908134290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:13.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.082+0000 7fb90d3b6700 1 --2- 172.21.15.3:0/1251934901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb90809f550 0x7fb908134290 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fb900009e50 tx=0x7fb90000b330 comp rx=0 tx=0).stop 2023-10-03T10:17:13.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.082+0000 7fb90d3b6700 1 -- 172.21.15.3:0/1251934901 shutdown_connections 2023-10-03T10:17:13.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.082+0000 7fb90d3b6700 1 --2- 172.21.15.3:0/1251934901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb90809f550 0x7fb908134290 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.082+0000 7fb90d3b6700 1 --2- 172.21.15.3:0/1251934901 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb90809e850 0x7fb90809ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.082+0000 7fb90d3b6700 1 -- 172.21.15.3:0/1251934901 >> 172.21.15.3:0/1251934901 conn(0x7fb9080a3f70 msgr2=0x7fb9080a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:13.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.082+0000 7fb90d3b6700 1 -- 172.21.15.3:0/1251934901 shutdown_connections 2023-10-03T10:17:13.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.082+0000 7fb90d3b6700 1 -- 172.21.15.3:0/1251934901 wait complete. 2023-10-03T10:17:13.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.083+0000 7fb90d3b6700 1 Processor -- start 2023-10-03T10:17:13.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.083+0000 7fb90d3b6700 1 -- start start 2023-10-03T10:17:13.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.084+0000 7fb90d3b6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb90809e850 0x7fb908122830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:13.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.084+0000 7fb90d3b6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb90809f160 0x7fb908122d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:13.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.084+0000 7fb90d3b6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb90809f550 0x7fb9081232b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:13.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.084+0000 7fb90d3b6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb908124440 con 0x7fb90809e850 2023-10-03T10:17:13.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.084+0000 7fb90d3b6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb908125d10 con 0x7fb90809f550 2023-10-03T10:17:13.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.084+0000 7fb90d3b6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb90806ccc0 con 0x7fb90809f160 2023-10-03T10:17:13.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.084+0000 7fb906ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb90809e850 0x7fb908122830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:13.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.084+0000 7fb9067fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb90809f160 0x7fb908122d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:13.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.084+0000 7fb906ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb90809e850 0x7fb908122830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37222/0 (socket says 172.21.15.3:37222) 2023-10-03T10:17:13.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.084+0000 7fb9067fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb90809f160 0x7fb908122d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:50184/0 (socket says 172.21.15.3:50184) 2023-10-03T10:17:13.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.084+0000 7fb906ffd700 1 -- 172.21.15.3:0/3610475791 learned_addr learned my addr 172.21.15.3:0/3610475791 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:13.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.084+0000 7fb9077fe700 1 --2- 172.21.15.3:0/3610475791 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb90809f550 0x7fb9081232b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:13.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.084+0000 7fb906ffd700 1 -- 172.21.15.3:0/3610475791 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb90809f160 msgr2=0x7fb908122d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:13.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.084+0000 7fb906ffd700 1 --2- 172.21.15.3:0/3610475791 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb90809f160 0x7fb908122d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.084+0000 7fb906ffd700 1 -- 172.21.15.3:0/3610475791 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb90809f550 msgr2=0x7fb9081232b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:13.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.084+0000 7fb906ffd700 1 --2- 172.21.15.3:0/3610475791 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb90809f550 0x7fb9081232b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.084+0000 7fb906ffd700 1 -- 172.21.15.3:0/3610475791 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f80094a0 con 0x7fb90809e850 2023-10-03T10:17:13.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.084+0000 7fb9067fc700 1 --2- 172.21.15.3:0/3610475791 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb90809f160 0x7fb908122d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:13.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.085+0000 7fb906ffd700 1 --2- 172.21.15.3:0/3610475791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb90809e850 0x7fb908122830 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800ebe0 tx=0x7fb8f800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:13.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.085+0000 7fb8ff7fe700 1 -- 172.21.15.3:0/3610475791 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f800c850 con 0x7fb90809e850 2023-10-03T10:17:13.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.085+0000 7fb8ff7fe700 1 -- 172.21.15.3:0/3610475791 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f800c9b0 con 0x7fb90809e850 2023-10-03T10:17:13.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.085+0000 7fb90d3b6700 1 -- 172.21.15.3:0/3610475791 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb900009b30 con 0x7fb90809e850 2023-10-03T10:17:13.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.085+0000 7fb8ff7fe700 1 -- 172.21.15.3:0/3610475791 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f800cd40 con 0x7fb90809e850 2023-10-03T10:17:13.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.085+0000 7fb90d3b6700 1 -- 172.21.15.3:0/3610475791 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb908133290 con 0x7fb90809e850 2023-10-03T10:17:13.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.086+0000 7fb8ff7fe700 1 -- 172.21.15.3:0/3610475791 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb8f8013470 con 0x7fb90809e850 2023-10-03T10:17:13.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.086+0000 7fb8ff7fe700 1 -- 172.21.15.3:0/3610475791 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb8f803f7c0 con 0x7fb90809e850 2023-10-03T10:17:13.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.086+0000 7fb90d3b6700 1 -- 172.21.15.3:0/3610475791 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb908126b00 con 0x7fb90809e850 2023-10-03T10:17:13.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.091+0000 7fb8ff7fe700 1 -- 172.21.15.3:0/3610475791 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb8f8017080 con 0x7fb90809e850 2023-10-03T10:17:13.271 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:13.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.300+0000 7fb90d3b6700 1 -- 172.21.15.3:0/3610475791 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb9080747f0 con 0x7fb90809e850 2023-10-03T10:17:13.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.300+0000 7fb8ff7fe700 1 -- 172.21.15.3:0/3610475791 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb8f8021020 con 0x7fb90809e850 2023-10-03T10:17:13.301 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:13.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.303+0000 7fb90d3b6700 1 -- 172.21.15.3:0/3610475791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb90809e850 msgr2=0x7fb908122830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:13.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.303+0000 7fb90d3b6700 1 --2- 172.21.15.3:0/3610475791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb90809e850 0x7fb908122830 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800ebe0 tx=0x7fb8f800eef0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.303+0000 7fb90d3b6700 1 -- 172.21.15.3:0/3610475791 shutdown_connections 2023-10-03T10:17:13.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.303+0000 7fb90d3b6700 1 --2- 172.21.15.3:0/3610475791 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb90809f550 0x7fb9081232b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.303+0000 7fb90d3b6700 1 --2- 172.21.15.3:0/3610475791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb90809e850 0x7fb908122830 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.303+0000 7fb90d3b6700 1 --2- 172.21.15.3:0/3610475791 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb90809f160 0x7fb908122d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.303+0000 7fb90d3b6700 1 -- 172.21.15.3:0/3610475791 >> 172.21.15.3:0/3610475791 conn(0x7fb9080a3f70 msgr2=0x7fb908070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:13.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.303+0000 7fb90d3b6700 1 -- 172.21.15.3:0/3610475791 shutdown_connections 2023-10-03T10:17:13.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.303+0000 7fb90d3b6700 1 -- 172.21.15.3:0/3610475791 wait complete. 2023-10-03T10:17:13.314 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:13.319 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:13.384 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:13.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.417+0000 7f0fd80ff700 1 Processor -- start 2023-10-03T10:17:13.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.418+0000 7f0fd80ff700 1 -- start start 2023-10-03T10:17:13.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.418+0000 7f0fd80ff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0fd009d6f0 0x7f0fd009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:13.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.418+0000 7f0fd80ff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0fd009e440 0x7f0fd0134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:13.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.418+0000 7f0fd80ff700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd009b120 con 0x7f0fd009e000 2023-10-03T10:17:13.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.418+0000 7f0fd80ff700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd006b460 con 0x7f0fd009d6f0 2023-10-03T10:17:13.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.418+0000 7f0fd80ff700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd009a7e0 con 0x7f0fd009e440 2023-10-03T10:17:13.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.419+0000 7f0fd5e9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0fd009d6f0 0x7f0fd009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:13.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.419+0000 7f0fd5e9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0fd009d6f0 0x7f0fd009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41310/0 (socket says 172.21.15.3:41310) 2023-10-03T10:17:13.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.419+0000 7f0fd5e9b700 1 -- 172.21.15.3:0/1705793 learned_addr learned my addr 172.21.15.3:0/1705793 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:13.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.419+0000 7f0fd669c700 1 --2- 172.21.15.3:0/1705793 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0fd009e440 0x7f0fd0134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:13.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.419+0000 7f0fd669c700 1 -- 172.21.15.3:0/1705793 >> v2:172.21.15.3:3300/0 conn(0x7f0fd009d6f0 msgr2=0x7f0fd009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:13.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.419+0000 7f0fd669c700 1 --2- 172.21.15.3:0/1705793 >> v2:172.21.15.3:3300/0 conn(0x7f0fd009d6f0 0x7f0fd009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.419+0000 7f0fd669c700 1 -- 172.21.15.3:0/1705793 >> v1:172.21.15.3:6789/0 conn(0x7f0fd009e000 legacy=0x7f0fd01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:13.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.419+0000 7f0fd669c700 1 -- 172.21.15.3:0/1705793 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd009a500 con 0x7f0fd009e440 2023-10-03T10:17:13.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.419+0000 7f0fd5e9b700 1 --2- 172.21.15.3:0/1705793 >> v2:172.21.15.3:3300/0 conn(0x7f0fd009d6f0 0x7f0fd009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:13.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.419+0000 7f0fd669c700 1 --2- 172.21.15.3:0/1705793 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0fd009e440 0x7f0fd0134100 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8009dc0 tx=0x7f0fc800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e8a8bfbf9af2e4b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:13.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.419+0000 7f0fd4e99700 1 -- 172.21.15.3:0/1705793 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc800e070 con 0x7f0fd009e440 2023-10-03T10:17:13.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.420+0000 7f0fd4e99700 1 -- 172.21.15.3:0/1705793 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc800bae0 con 0x7f0fd009e440 2023-10-03T10:17:13.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.420+0000 7f0fd4e99700 1 -- 172.21.15.3:0/1705793 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc800be90 con 0x7f0fd009e440 2023-10-03T10:17:13.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.421+0000 7f0fd80ff700 1 -- 172.21.15.3:0/1705793 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0fd009e440 msgr2=0x7f0fd0134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:13.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.421+0000 7f0fd80ff700 1 --2- 172.21.15.3:0/1705793 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0fd009e440 0x7f0fd0134100 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8009dc0 tx=0x7f0fc800b330 comp rx=0 tx=0).stop 2023-10-03T10:17:13.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.421+0000 7f0fd80ff700 1 -- 172.21.15.3:0/1705793 shutdown_connections 2023-10-03T10:17:13.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.421+0000 7f0fd80ff700 1 --2- 172.21.15.3:0/1705793 >> v2:172.21.15.3:3300/0 conn(0x7f0fd009d6f0 0x7f0fd009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.421+0000 7f0fd80ff700 1 --2- 172.21.15.3:0/1705793 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0fd009e440 0x7f0fd0134100 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.421+0000 7f0fd80ff700 1 -- 172.21.15.3:0/1705793 >> 172.21.15.3:0/1705793 conn(0x7f0fd0066f50 msgr2=0x7f0fd009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:13.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.421+0000 7f0fd80ff700 1 -- 172.21.15.3:0/1705793 shutdown_connections 2023-10-03T10:17:13.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.421+0000 7f0fd80ff700 1 -- 172.21.15.3:0/1705793 wait complete. 2023-10-03T10:17:13.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.422+0000 7f0fd80ff700 1 Processor -- start 2023-10-03T10:17:13.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.422+0000 7f0fd80ff700 1 -- start start 2023-10-03T10:17:13.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.423+0000 7f0fd80ff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0fd009d6f0 0x7f0fd01c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:13.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.423+0000 7f0fd80ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fd009e000 0x7f0fd01c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:13.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.423+0000 7f0fd80ff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0fd009e440 0x7f0fd01c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:13.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.423+0000 7f0fd80ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd01a80b0 con 0x7f0fd009e000 2023-10-03T10:17:13.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.423+0000 7f0fd80ff700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd00a1f90 con 0x7f0fd009d6f0 2023-10-03T10:17:13.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.423+0000 7f0fd80ff700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd01b9550 con 0x7f0fd009e440 2023-10-03T10:17:13.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.423+0000 7f0fd5e9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0fd009d6f0 0x7f0fd01c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:13.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.423+0000 7f0fd669c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0fd009e440 0x7f0fd01c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:13.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.423+0000 7f0fd5e9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0fd009d6f0 0x7f0fd01c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50788/0 (socket says 172.21.15.3:50788) 2023-10-03T10:17:13.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.423+0000 7f0fd5e9b700 1 -- 172.21.15.3:0/3679304815 learned_addr learned my addr 172.21.15.3:0/3679304815 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:13.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.423+0000 7f0fd569a700 1 --2- 172.21.15.3:0/3679304815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fd009e000 0x7f0fd01c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:13.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.424+0000 7f0fd669c700 1 -- 172.21.15.3:0/3679304815 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0fd009d6f0 msgr2=0x7f0fd01c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:13.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.424+0000 7f0fd669c700 1 --2- 172.21.15.3:0/3679304815 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0fd009d6f0 0x7f0fd01c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.424+0000 7f0fd669c700 1 -- 172.21.15.3:0/3679304815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fd009e000 msgr2=0x7f0fd01c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:13.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.424+0000 7f0fd669c700 1 --2- 172.21.15.3:0/3679304815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fd009e000 0x7f0fd01c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.424+0000 7f0fd669c700 1 -- 172.21.15.3:0/3679304815 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fcc009550 con 0x7f0fd009e440 2023-10-03T10:17:13.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.424+0000 7f0fd5e9b700 1 --2- 172.21.15.3:0/3679304815 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0fd009d6f0 0x7f0fd01c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:13.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.424+0000 7f0fd669c700 1 --2- 172.21.15.3:0/3679304815 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0fd009e440 0x7f0fd01c49d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8016000 tx=0x7f0fc8004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:13.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.424+0000 7f0fc67fc700 1 -- 172.21.15.3:0/3679304815 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc80045d0 con 0x7f0fd009e440 2023-10-03T10:17:13.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.424+0000 7f0fc67fc700 1 -- 172.21.15.3:0/3679304815 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc8020070 con 0x7f0fd009e440 2023-10-03T10:17:13.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.424+0000 7f0fd80ff700 1 -- 172.21.15.3:0/3679304815 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc8009a70 con 0x7f0fd009e440 2023-10-03T10:17:13.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.424+0000 7f0fc67fc700 1 -- 172.21.15.3:0/3679304815 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc8027480 con 0x7f0fd009e440 2023-10-03T10:17:13.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.424+0000 7f0fd80ff700 1 -- 172.21.15.3:0/3679304815 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd01a77e0 con 0x7f0fd009e440 2023-10-03T10:17:13.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.425+0000 7f0fc67fc700 1 -- 172.21.15.3:0/3679304815 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0fc8012430 con 0x7f0fd009e440 2023-10-03T10:17:13.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.425+0000 7f0fc67fc700 1 -- 172.21.15.3:0/3679304815 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0fc8022070 con 0x7f0fd009e440 2023-10-03T10:17:13.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.426+0000 7f0fd80ff700 1 -- 172.21.15.3:0/3679304815 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd01a79d0 con 0x7f0fd009e440 2023-10-03T10:17:13.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.431+0000 7f0fc67fc700 1 -- 172.21.15.3:0/3679304815 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0fc801c400 con 0x7f0fd009e440 2023-10-03T10:17:13.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.456+0000 7fb447eef700 1 Processor -- start 2023-10-03T10:17:13.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.456+0000 7fb447eef700 1 -- start start 2023-10-03T10:17:13.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.457+0000 7fb447eef700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb4400a0a30 0x7fb4400a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:13.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.457+0000 7fb447eef700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb440138d80 0x7fb44013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:13.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.457+0000 7fb447eef700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4400a2a70 con 0x7fb44009d630 2023-10-03T10:17:13.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.457+0000 7fb447eef700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4400a01d0 con 0x7fb440138d80 2023-10-03T10:17:13.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.457+0000 7fb447eef700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb44009e2b0 con 0x7fb4400a0a30 2023-10-03T10:17:13.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.457+0000 7fb445c8b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb4400a0a30 0x7fb4400a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:13.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.457+0000 7fb44648c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb440138d80 0x7fb44013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:13.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.457+0000 7fb445c8b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb4400a0a30 0x7fb4400a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53224/0 (socket says 172.21.15.3:53224) 2023-10-03T10:17:13.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.457+0000 7fb44548a700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb44009d630 0x7fb4400a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55194/0 (socket says 172.21.15.3:55194) 2023-10-03T10:17:13.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.457+0000 7fb445c8b700 1 -- 172.21.15.3:0/2872374368 learned_addr learned my addr 172.21.15.3:0/2872374368 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:13.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.457+0000 7fb445c8b700 1 -- 172.21.15.3:0/2872374368 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb440138d80 msgr2=0x7fb44013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:13.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.457+0000 7fb445c8b700 1 --2- 172.21.15.3:0/2872374368 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb440138d80 0x7fb44013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.457+0000 7fb445c8b700 1 -- 172.21.15.3:0/2872374368 >> v1:172.21.15.3:6789/0 conn(0x7fb44009d630 legacy=0x7fb4400a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:13.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.457+0000 7fb444c89700 1 -- 172.21.15.3:0/2872374368 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3944289350 0 0) 0x7fb4400a2a70 con 0x7fb44009d630 2023-10-03T10:17:13.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.457+0000 7fb445c8b700 1 -- 172.21.15.3:0/2872374368 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44009dc50 con 0x7fb4400a0a30 2023-10-03T10:17:13.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.457+0000 7fb44648c700 1 --2- 172.21.15.3:0/2872374368 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb440138d80 0x7fb44013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:13.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.458+0000 7fb445c8b700 1 --2- 172.21.15.3:0/2872374368 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb4400a0a30 0x7fb4400a0e00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb43c009920 tx=0x7fb43c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bfbe4753cb8eb4b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:13.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.458+0000 7fb444c89700 1 -- 172.21.15.3:0/2872374368 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c011830 con 0x7fb4400a0a30 2023-10-03T10:17:13.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.458+0000 7fb444c89700 1 -- 172.21.15.3:0/2872374368 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb43c011990 con 0x7fb4400a0a30 2023-10-03T10:17:13.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.458+0000 7fb444c89700 1 -- 172.21.15.3:0/2872374368 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c011d20 con 0x7fb4400a0a30 2023-10-03T10:17:13.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.458+0000 7fb447eef700 1 -- 172.21.15.3:0/2872374368 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb4400a0a30 msgr2=0x7fb4400a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:13.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.458+0000 7fb447eef700 1 --2- 172.21.15.3:0/2872374368 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb4400a0a30 0x7fb4400a0e00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb43c009920 tx=0x7fb43c009c30 comp rx=0 tx=0).stop 2023-10-03T10:17:13.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.459+0000 7fb447eef700 1 -- 172.21.15.3:0/2872374368 shutdown_connections 2023-10-03T10:17:13.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.459+0000 7fb447eef700 1 --2- 172.21.15.3:0/2872374368 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb440138d80 0x7fb44013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.459+0000 7fb447eef700 1 --2- 172.21.15.3:0/2872374368 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb4400a0a30 0x7fb4400a0e00 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.459+0000 7fb447eef700 1 -- 172.21.15.3:0/2872374368 >> 172.21.15.3:0/2872374368 conn(0x7fb4400a3050 msgr2=0x7fb4400a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:13.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.459+0000 7fb447eef700 1 -- 172.21.15.3:0/2872374368 shutdown_connections 2023-10-03T10:17:13.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.459+0000 7fb447eef700 1 -- 172.21.15.3:0/2872374368 wait complete. 2023-10-03T10:17:13.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.460+0000 7fb447eef700 1 Processor -- start 2023-10-03T10:17:13.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.460+0000 7fb447eef700 1 -- start start 2023-10-03T10:17:13.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.461+0000 7fb447eef700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb44009d630 0x7fb4401337d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:13.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.461+0000 7fb447eef700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4400a0a30 0x7fb440133d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:13.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.461+0000 7fb447eef700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb440138d80 0x7fb440134250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:13.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.461+0000 7fb447eef700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4400a0410 con 0x7fb4400a0a30 2023-10-03T10:17:13.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.461+0000 7fb447eef700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb440136cc0 con 0x7fb44009d630 2023-10-03T10:17:13.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.461+0000 7fb447eef700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb4400a4b00 con 0x7fb440138d80 2023-10-03T10:17:13.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.461+0000 7fb445c8b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb44009d630 0x7fb4401337d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:13.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.461+0000 7fb445c8b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb44009d630 0x7fb4401337d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50806/0 (socket says 172.21.15.3:50806) 2023-10-03T10:17:13.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.461+0000 7fb445c8b700 1 -- 172.21.15.3:0/3238743972 learned_addr learned my addr 172.21.15.3:0/3238743972 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:13.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.461+0000 7fb44548a700 1 --2- 172.21.15.3:0/3238743972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4400a0a30 0x7fb440133d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:13.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.461+0000 7fb44648c700 1 --2- 172.21.15.3:0/3238743972 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb440138d80 0x7fb440134250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:13.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.462+0000 7fb445c8b700 1 -- 172.21.15.3:0/3238743972 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb440138d80 msgr2=0x7fb440134250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:13.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.462+0000 7fb445c8b700 1 --2- 172.21.15.3:0/3238743972 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb440138d80 0x7fb440134250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.462+0000 7fb445c8b700 1 -- 172.21.15.3:0/3238743972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4400a0a30 msgr2=0x7fb440133d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:13.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.462+0000 7fb445c8b700 1 --2- 172.21.15.3:0/3238743972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4400a0a30 0x7fb440133d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.462+0000 7fb445c8b700 1 -- 172.21.15.3:0/3238743972 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb438009a70 con 0x7fb44009d630 2023-10-03T10:17:13.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.462+0000 7fb44548a700 1 --2- 172.21.15.3:0/3238743972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4400a0a30 0x7fb440133d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:13.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.462+0000 7fb445c8b700 1 --2- 172.21.15.3:0/3238743972 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb44009d630 0x7fb4401337d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb43c00a010 tx=0x7fb43c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:13.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.462+0000 7fb4367fc700 1 -- 172.21.15.3:0/3238743972 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c004040 con 0x7fb44009d630 2023-10-03T10:17:13.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.462+0000 7fb4367fc700 1 -- 172.21.15.3:0/3238743972 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb43c0041a0 con 0x7fb44009d630 2023-10-03T10:17:13.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.462+0000 7fb4367fc700 1 -- 172.21.15.3:0/3238743972 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c004550 con 0x7fb44009d630 2023-10-03T10:17:13.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.462+0000 7fb447eef700 1 -- 172.21.15.3:0/3238743972 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb43c009550 con 0x7fb44009d630 2023-10-03T10:17:13.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.462+0000 7fb447eef700 1 -- 172.21.15.3:0/3238743972 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb440135af0 con 0x7fb44009d630 2023-10-03T10:17:13.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.463+0000 7fb4367fc700 1 -- 172.21.15.3:0/3238743972 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb43c0036a0 con 0x7fb44009d630 2023-10-03T10:17:13.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.463+0000 7fb4367fc700 1 -- 172.21.15.3:0/3238743972 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb43c048810 con 0x7fb44009d630 2023-10-03T10:17:13.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.463+0000 7fb447eef700 1 -- 172.21.15.3:0/3238743972 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4400747f0 con 0x7fb44009d630 2023-10-03T10:17:13.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.473+0000 7fb4367fc700 1 -- 172.21.15.3:0/3238743972 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb43c02f020 con 0x7fb44009d630 2023-10-03T10:17:13.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.529+0000 7f087eeff700 1 Processor -- start 2023-10-03T10:17:13.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.529+0000 7f087eeff700 1 -- start start 2023-10-03T10:17:13.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.530+0000 7f087eeff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f087809d6f0 0x7f087809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:13.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.530+0000 7f087eeff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f087809e440 0x7f0878134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:13.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.530+0000 7f087eeff700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f087809b120 con 0x7f087809e000 2023-10-03T10:17:13.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.530+0000 7f087eeff700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f087806b460 con 0x7f087809d6f0 2023-10-03T10:17:13.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.530+0000 7f087eeff700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f087809a7e0 con 0x7f087809e440 2023-10-03T10:17:13.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.530+0000 7f087d49c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f087809e440 0x7f0878134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:13.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.530+0000 7f087cc9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f087809d6f0 0x7f087809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:13.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.530+0000 7f087cc9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f087809d6f0 0x7f087809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41344/0 (socket says 172.21.15.3:41344) 2023-10-03T10:17:13.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.530+0000 7f087d49c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f087809e440 0x7f0878134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53238/0 (socket says 172.21.15.3:53238) 2023-10-03T10:17:13.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.530+0000 7f087cc9b700 1 -- 172.21.15.3:0/1132657457 learned_addr learned my addr 172.21.15.3:0/1132657457 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:13.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.530+0000 7f087d49c700 1 -- 172.21.15.3:0/1132657457 >> v2:172.21.15.3:3300/0 conn(0x7f087809d6f0 msgr2=0x7f087809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:13.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.530+0000 7f087d49c700 1 --2- 172.21.15.3:0/1132657457 >> v2:172.21.15.3:3300/0 conn(0x7f087809d6f0 0x7f087809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.530+0000 7f087d49c700 1 -- 172.21.15.3:0/1132657457 >> v1:172.21.15.3:6789/0 conn(0x7f087809e000 legacy=0x7f08781339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:13.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.530+0000 7f087d49c700 1 -- 172.21.15.3:0/1132657457 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087809a500 con 0x7f087809e440 2023-10-03T10:17:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.530+0000 7f087cc9b700 1 --2- 172.21.15.3:0/1132657457 >> v2:172.21.15.3:3300/0 conn(0x7f087809d6f0 0x7f087809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.530+0000 7f087d49c700 1 --2- 172.21.15.3:0/1132657457 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f087809e440 0x7f0878134100 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0870009dc0 tx=0x7f087000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e51d037c133495a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.531+0000 7f08777fe700 1 -- 172.21.15.3:0/1132657457 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087000e070 con 0x7f087809e440 2023-10-03T10:17:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.531+0000 7f08777fe700 1 -- 172.21.15.3:0/1132657457 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087000bae0 con 0x7f087809e440 2023-10-03T10:17:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.531+0000 7f08777fe700 1 -- 172.21.15.3:0/1132657457 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087000be90 con 0x7f087809e440 2023-10-03T10:17:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.531+0000 7f087eeff700 1 -- 172.21.15.3:0/1132657457 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f087809e440 msgr2=0x7f0878134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.531+0000 7f087eeff700 1 --2- 172.21.15.3:0/1132657457 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f087809e440 0x7f0878134100 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0870009dc0 tx=0x7f087000b330 comp rx=0 tx=0).stop 2023-10-03T10:17:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.531+0000 7f087eeff700 1 -- 172.21.15.3:0/1132657457 shutdown_connections 2023-10-03T10:17:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.531+0000 7f087eeff700 1 --2- 172.21.15.3:0/1132657457 >> v2:172.21.15.3:3300/0 conn(0x7f087809d6f0 0x7f087809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.531+0000 7f087eeff700 1 --2- 172.21.15.3:0/1132657457 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f087809e440 0x7f0878134100 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.531+0000 7f087eeff700 1 -- 172.21.15.3:0/1132657457 >> 172.21.15.3:0/1132657457 conn(0x7f0878066f50 msgr2=0x7f087809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.531+0000 7f087eeff700 1 -- 172.21.15.3:0/1132657457 shutdown_connections 2023-10-03T10:17:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.531+0000 7f087eeff700 1 -- 172.21.15.3:0/1132657457 wait complete. 2023-10-03T10:17:13.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.532+0000 7f087eeff700 1 Processor -- start 2023-10-03T10:17:13.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.533+0000 7f087eeff700 1 -- start start 2023-10-03T10:17:13.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.533+0000 7f087eeff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f087809d6f0 0x7f08781a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:13.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.533+0000 7f087eeff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f087809e000 0x7f08781a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:13.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.533+0000 7f087eeff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f087809e440 0x7f08781a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:13.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.533+0000 7f087eeff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08781a81f0 con 0x7f087809e000 2023-10-03T10:17:13.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.533+0000 7f087eeff700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08781a6b10 con 0x7f087809e440 2023-10-03T10:17:13.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.533+0000 7f087eeff700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f08781a86a0 con 0x7f087809d6f0 2023-10-03T10:17:13.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.533+0000 7f0877fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f087809e000 0x7f08781a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:13.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.533+0000 7f087cc9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f087809d6f0 0x7f08781a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:13.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.533+0000 7f0877fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f087809e000 0x7f08781a53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41354/0 (socket says 172.21.15.3:41354) 2023-10-03T10:17:13.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.533+0000 7f087cc9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f087809d6f0 0x7f08781a4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53252/0 (socket says 172.21.15.3:53252) 2023-10-03T10:17:13.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.533+0000 7f0877fff700 1 -- 172.21.15.3:0/2951090860 learned_addr learned my addr 172.21.15.3:0/2951090860 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:13.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.534+0000 7f087d49c700 1 --2- 172.21.15.3:0/2951090860 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f087809e440 0x7f08781a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:13.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.534+0000 7f0877fff700 1 -- 172.21.15.3:0/2951090860 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f087809d6f0 msgr2=0x7f08781a4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:13.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.534+0000 7f0877fff700 1 --2- 172.21.15.3:0/2951090860 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f087809d6f0 0x7f08781a4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.534+0000 7f0877fff700 1 -- 172.21.15.3:0/2951090860 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f087809e440 msgr2=0x7f08781a58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:13.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.534+0000 7f0877fff700 1 --2- 172.21.15.3:0/2951090860 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f087809e440 0x7f08781a58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.534+0000 7f0877fff700 1 -- 172.21.15.3:0/2951090860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f086c009550 con 0x7f087809e000 2023-10-03T10:17:13.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.534+0000 7f087cc9b700 1 --2- 172.21.15.3:0/2951090860 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f087809d6f0 0x7f08781a4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:13.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.534+0000 7f0877fff700 1 --2- 172.21.15.3:0/2951090860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f087809e000 0x7f08781a53a0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f086400c4e0 tx=0x7f086400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:13.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.534+0000 7f08757fa700 1 -- 172.21.15.3:0/2951090860 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08640052d0 con 0x7f087809e000 2023-10-03T10:17:13.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.534+0000 7f08757fa700 1 -- 172.21.15.3:0/2951090860 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0864005430 con 0x7f087809e000 2023-10-03T10:17:13.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.534+0000 7f087eeff700 1 -- 172.21.15.3:0/2951090860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0870009a70 con 0x7f087809e000 2023-10-03T10:17:13.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.534+0000 7f08757fa700 1 -- 172.21.15.3:0/2951090860 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08640038c0 con 0x7f087809e000 2023-10-03T10:17:13.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.534+0000 7f087eeff700 1 -- 172.21.15.3:0/2951090860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08781a7cc0 con 0x7f087809e000 2023-10-03T10:17:13.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.535+0000 7f08757fa700 1 -- 172.21.15.3:0/2951090860 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0864003a20 con 0x7f087809e000 2023-10-03T10:17:13.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.535+0000 7f08757fa700 1 -- 172.21.15.3:0/2951090860 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f086403ec30 con 0x7f087809e000 2023-10-03T10:17:13.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.536+0000 7f087eeff700 1 -- 172.21.15.3:0/2951090860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0878074ea0 con 0x7f087809e000 2023-10-03T10:17:13.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.541+0000 7f08757fa700 1 -- 172.21.15.3:0/2951090860 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0864017080 con 0x7f087809e000 2023-10-03T10:17:13.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.648+0000 7f0fd80ff700 1 -- 172.21.15.3:0/3679304815 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0fd009f420 con 0x7f0fd009e440 2023-10-03T10:17:13.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.649+0000 7f0fc67fc700 1 -- 172.21.15.3:0/3679304815 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0fc802c070 con 0x7f0fd009e440 2023-10-03T10:17:13.649 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:13.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.652+0000 7f0fd80ff700 1 -- 172.21.15.3:0/3679304815 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0fd009e440 msgr2=0x7f0fd01c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:13.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.652+0000 7f0fd80ff700 1 --2- 172.21.15.3:0/3679304815 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0fd009e440 0x7f0fd01c49d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8016000 tx=0x7f0fc8004ad0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.652+0000 7f0fd80ff700 1 -- 172.21.15.3:0/3679304815 shutdown_connections 2023-10-03T10:17:13.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.652+0000 7f0fd80ff700 1 --2- 172.21.15.3:0/3679304815 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0fd009e440 0x7f0fd01c49d0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.652+0000 7f0fd80ff700 1 --2- 172.21.15.3:0/3679304815 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0fd009d6f0 0x7f0fd01c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.652+0000 7f0fd80ff700 1 --2- 172.21.15.3:0/3679304815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fd009e000 0x7f0fd01c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.652+0000 7f0fd80ff700 1 -- 172.21.15.3:0/3679304815 >> 172.21.15.3:0/3679304815 conn(0x7f0fd0066f50 msgr2=0x7f0fd01bad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:13.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.652+0000 7f0fd80ff700 1 -- 172.21.15.3:0/3679304815 shutdown_connections 2023-10-03T10:17:13.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.652+0000 7f0fd80ff700 1 -- 172.21.15.3:0/3679304815 wait complete. 2023-10-03T10:17:13.671 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:13.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.691+0000 7fb447eef700 1 -- 172.21.15.3:0/3238743972 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb440072980 con 0x7fb44009d630 2023-10-03T10:17:13.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.691+0000 7fb4367fc700 1 -- 172.21.15.3:0/3238743972 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb43c015070 con 0x7fb44009d630 2023-10-03T10:17:13.691 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:13.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.693+0000 7fb447eef700 1 -- 172.21.15.3:0/3238743972 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb44009d630 msgr2=0x7fb4401337d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:13.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.693+0000 7fb447eef700 1 --2- 172.21.15.3:0/3238743972 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb44009d630 0x7fb4401337d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb43c00a010 tx=0x7fb43c004750 comp rx=0 tx=0).stop 2023-10-03T10:17:13.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.693+0000 7fb447eef700 1 -- 172.21.15.3:0/3238743972 shutdown_connections 2023-10-03T10:17:13.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.693+0000 7fb447eef700 1 --2- 172.21.15.3:0/3238743972 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb440138d80 0x7fb440134250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.693+0000 7fb447eef700 1 --2- 172.21.15.3:0/3238743972 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb44009d630 0x7fb4401337d0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.693+0000 7fb447eef700 1 --2- 172.21.15.3:0/3238743972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4400a0a30 0x7fb440133d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.693+0000 7fb447eef700 1 -- 172.21.15.3:0/3238743972 >> 172.21.15.3:0/3238743972 conn(0x7fb4400a3050 msgr2=0x7fb44009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:13.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.693+0000 7fb447eef700 1 -- 172.21.15.3:0/3238743972 shutdown_connections 2023-10-03T10:17:13.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.693+0000 7fb447eef700 1 -- 172.21.15.3:0/3238743972 wait complete. 2023-10-03T10:17:13.707 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.747+0000 7f087eeff700 1 -- 172.21.15.3:0/2951090860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f087806c360 con 0x7f087809e000 2023-10-03T10:17:13.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.748+0000 7f08757fa700 1 -- 172.21.15.3:0/2951090860 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0864018260 con 0x7f087809e000 2023-10-03T10:17:13.748 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:13.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.750+0000 7f087eeff700 1 -- 172.21.15.3:0/2951090860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f087809e000 msgr2=0x7f08781a53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:13.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.750+0000 7f087eeff700 1 --2- 172.21.15.3:0/2951090860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f087809e000 0x7f08781a53a0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f086400c4e0 tx=0x7f086400c7f0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.750+0000 7f087eeff700 1 -- 172.21.15.3:0/2951090860 shutdown_connections 2023-10-03T10:17:13.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.750+0000 7f087eeff700 1 --2- 172.21.15.3:0/2951090860 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f087809e440 0x7f08781a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.750+0000 7f087eeff700 1 --2- 172.21.15.3:0/2951090860 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f087809d6f0 0x7f08781a4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.750+0000 7f087eeff700 1 --2- 172.21.15.3:0/2951090860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f087809e000 0x7f08781a53a0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:13.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.750+0000 7f087eeff700 1 -- 172.21.15.3:0/2951090860 >> 172.21.15.3:0/2951090860 conn(0x7f0878066f50 msgr2=0x7f08781b9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:13.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.750+0000 7f087eeff700 1 -- 172.21.15.3:0/2951090860 shutdown_connections 2023-10-03T10:17:13.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:13.750+0000 7f087eeff700 1 -- 172.21.15.3:0/2951090860 wait complete. 2023-10-03T10:17:13.764 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:14.320 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:14.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.466+0000 7fa361ca8700 1 Processor -- start 2023-10-03T10:17:14.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.466+0000 7fa361ca8700 1 -- start start 2023-10-03T10:17:14.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.467+0000 7fa361ca8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa35c09d6f0 0x7fa35c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:14.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.467+0000 7fa361ca8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa35c09e410 0x7fa35c134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:14.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.467+0000 7fa361ca8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa35c09b120 con 0x7fa35c09e000 2023-10-03T10:17:14.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.467+0000 7fa361ca8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa35c06b460 con 0x7fa35c09e410 2023-10-03T10:17:14.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.467+0000 7fa361ca8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c09a7e0 con 0x7fa35c09d6f0 2023-10-03T10:17:14.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.467+0000 7fa35b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa35c09d6f0 0x7fa35c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:14.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.467+0000 7fa35bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa35c09e410 0x7fa35c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:14.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.467+0000 7fa35bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa35c09e410 0x7fa35c134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41356/0 (socket says 172.21.15.3:41356) 2023-10-03T10:17:14.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.467+0000 7fa35b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa35c09d6f0 0x7fa35c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50820/0 (socket says 172.21.15.3:50820) 2023-10-03T10:17:14.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.467+0000 7fa35bfff700 1 -- 172.21.15.3:0/1556923381 learned_addr learned my addr 172.21.15.3:0/1556923381 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:14.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.467+0000 7fa35bfff700 1 -- 172.21.15.3:0/1556923381 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa35c09d6f0 msgr2=0x7fa35c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:14.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.467+0000 7fa35bfff700 1 --2- 172.21.15.3:0/1556923381 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa35c09d6f0 0x7fa35c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:14.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.467+0000 7fa35bfff700 1 -- 172.21.15.3:0/1556923381 >> v1:172.21.15.3:6789/0 conn(0x7fa35c09e000 legacy=0x7fa35c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:14.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.467+0000 7fa35bfff700 1 -- 172.21.15.3:0/1556923381 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c09a500 con 0x7fa35c09e410 2023-10-03T10:17:14.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.468+0000 7fa35bfff700 1 --2- 172.21.15.3:0/1556923381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa35c09e410 0x7fa35c134100 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fa354009e50 tx=0x7fa35400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa98326051c9c494 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:14.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.468+0000 7fa35a7fc700 1 -- 172.21.15.3:0/1556923381 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35400e070 con 0x7fa35c09e410 2023-10-03T10:17:14.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.468+0000 7fa35a7fc700 1 -- 172.21.15.3:0/1556923381 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35400b910 con 0x7fa35c09e410 2023-10-03T10:17:14.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.468+0000 7fa35a7fc700 1 -- 172.21.15.3:0/1556923381 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35400bcc0 con 0x7fa35c09e410 2023-10-03T10:17:14.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.468+0000 7fa361ca8700 1 -- 172.21.15.3:0/1556923381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa35c09e410 msgr2=0x7fa35c134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:14.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.468+0000 7fa361ca8700 1 --2- 172.21.15.3:0/1556923381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa35c09e410 0x7fa35c134100 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fa354009e50 tx=0x7fa35400b330 comp rx=0 tx=0).stop 2023-10-03T10:17:14.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.468+0000 7fa361ca8700 1 -- 172.21.15.3:0/1556923381 shutdown_connections 2023-10-03T10:17:14.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.468+0000 7fa361ca8700 1 --2- 172.21.15.3:0/1556923381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa35c09e410 0x7fa35c134100 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.468+0000 7fa361ca8700 1 --2- 172.21.15.3:0/1556923381 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa35c09d6f0 0x7fa35c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.468+0000 7fa361ca8700 1 -- 172.21.15.3:0/1556923381 >> 172.21.15.3:0/1556923381 conn(0x7fa35c066f50 msgr2=0x7fa35c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.469+0000 7fa361ca8700 1 -- 172.21.15.3:0/1556923381 shutdown_connections 2023-10-03T10:17:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.469+0000 7fa361ca8700 1 -- 172.21.15.3:0/1556923381 wait complete. 2023-10-03T10:17:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.470+0000 7fa361ca8700 1 Processor -- start 2023-10-03T10:17:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.470+0000 7fa361ca8700 1 -- start start 2023-10-03T10:17:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.470+0000 7fa361ca8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa35c09d6f0 0x7fa35c1a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.470+0000 7fa361ca8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa35c09e000 0x7fa35c1a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.470+0000 7fa361ca8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa35c09e410 0x7fa35c1a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.470+0000 7fa361ca8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c1a9870 con 0x7fa35c09d6f0 2023-10-03T10:17:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.470+0000 7fa361ca8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c1a7b90 con 0x7fa35c09e000 2023-10-03T10:17:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.470+0000 7fa361ca8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c1a9a70 con 0x7fa35c09e410 2023-10-03T10:17:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.470+0000 7fa35b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa35c09d6f0 0x7fa35c1a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.470+0000 7fa35bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa35c09e410 0x7fa35c1a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.470+0000 7fa35b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa35c09d6f0 0x7fa35c1a60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41370/0 (socket says 172.21.15.3:41370) 2023-10-03T10:17:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.470+0000 7fa35b7fe700 1 -- 172.21.15.3:0/1383512520 learned_addr learned my addr 172.21.15.3:0/1383512520 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.470+0000 7fa35affd700 1 --2- 172.21.15.3:0/1383512520 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa35c09e000 0x7fa35c1a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.471+0000 7fa35bfff700 1 -- 172.21.15.3:0/1383512520 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa35c09e000 msgr2=0x7fa35c1a65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.471+0000 7fa35bfff700 1 --2- 172.21.15.3:0/1383512520 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa35c09e000 0x7fa35c1a65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.471+0000 7fa35bfff700 1 -- 172.21.15.3:0/1383512520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa35c09d6f0 msgr2=0x7fa35c1a60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:14.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.471+0000 7fa35bfff700 1 --2- 172.21.15.3:0/1383512520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa35c09d6f0 0x7fa35c1a60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:14.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.471+0000 7fa35bfff700 1 -- 172.21.15.3:0/1383512520 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa354009b30 con 0x7fa35c09e410 2023-10-03T10:17:14.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.471+0000 7fa35affd700 1 --2- 172.21.15.3:0/1383512520 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa35c09e000 0x7fa35c1a65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:14.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.471+0000 7fa35b7fe700 1 --2- 172.21.15.3:0/1383512520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa35c09d6f0 0x7fa35c1a60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:14.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.471+0000 7fa35bfff700 1 --2- 172.21.15.3:0/1383512520 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa35c09e410 0x7fa35c1a6b20 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa35c06e470 tx=0x7fa354004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:14.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.471+0000 7fa353fff700 1 -- 172.21.15.3:0/1383512520 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa354004840 con 0x7fa35c09e410 2023-10-03T10:17:14.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.471+0000 7fa353fff700 1 -- 172.21.15.3:0/1383512520 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3540092e0 con 0x7fa35c09e410 2023-10-03T10:17:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.471+0000 7fa361ca8700 1 -- 172.21.15.3:0/1383512520 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa35c0a2f40 con 0x7fa35c09e410 2023-10-03T10:17:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.471+0000 7fa353fff700 1 -- 172.21.15.3:0/1383512520 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35401c660 con 0x7fa35c09e410 2023-10-03T10:17:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.471+0000 7fa361ca8700 1 -- 172.21.15.3:0/1383512520 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa35c09c760 con 0x7fa35c09e410 2023-10-03T10:17:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.472+0000 7fa353fff700 1 -- 172.21.15.3:0/1383512520 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa35401c7c0 con 0x7fa35c09e410 2023-10-03T10:17:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.472+0000 7fa353fff700 1 -- 172.21.15.3:0/1383512520 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa354040f20 con 0x7fa35c09e410 2023-10-03T10:17:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.472+0000 7fa361ca8700 1 -- 172.21.15.3:0/1383512520 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa35c1aa400 con 0x7fa35c09e410 2023-10-03T10:17:14.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.477+0000 7fa353fff700 1 -- 172.21.15.3:0/1383512520 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa3540123f0 con 0x7fa35c09e410 2023-10-03T10:17:14.672 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:14.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.682+0000 7fa361ca8700 1 -- 172.21.15.3:0/1383512520 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa35c074ea0 con 0x7fa35c09e410 2023-10-03T10:17:14.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.683+0000 7fa353fff700 1 -- 172.21.15.3:0/1383512520 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa3540243b0 con 0x7fa35c09e410 2023-10-03T10:17:14.683 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:14.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.685+0000 7fa361ca8700 1 -- 172.21.15.3:0/1383512520 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa35c09e410 msgr2=0x7fa35c1a6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:14.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.685+0000 7fa361ca8700 1 --2- 172.21.15.3:0/1383512520 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa35c09e410 0x7fa35c1a6b20 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa35c06e470 tx=0x7fa354004ad0 comp rx=0 tx=0).stop 2023-10-03T10:17:14.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.685+0000 7fa361ca8700 1 -- 172.21.15.3:0/1383512520 shutdown_connections 2023-10-03T10:17:14.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.685+0000 7fa361ca8700 1 --2- 172.21.15.3:0/1383512520 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa35c09e410 0x7fa35c1a6b20 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:14.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.685+0000 7fa361ca8700 1 --2- 172.21.15.3:0/1383512520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa35c09d6f0 0x7fa35c1a60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:14.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.685+0000 7fa361ca8700 1 --2- 172.21.15.3:0/1383512520 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa35c09e000 0x7fa35c1a65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:14.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.685+0000 7fa361ca8700 1 -- 172.21.15.3:0/1383512520 >> 172.21.15.3:0/1383512520 conn(0x7fa35c066f50 msgr2=0x7fa35c1baef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:14.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.685+0000 7fa361ca8700 1 -- 172.21.15.3:0/1383512520 shutdown_connections 2023-10-03T10:17:14.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.686+0000 7fa361ca8700 1 -- 172.21.15.3:0/1383512520 wait complete. 2023-10-03T10:17:14.700 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:14.707 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:14.765 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:14.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.811+0000 7f0191c69700 1 Processor -- start 2023-10-03T10:17:14.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.811+0000 7f0191c69700 1 -- start start 2023-10-03T10:17:14.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.812+0000 7f0191c69700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f018c133510 0x7f018c1358f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:14.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.812+0000 7f0191c69700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f018c135e30 0x7f018c138210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:14.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.812+0000 7f0191c69700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f018c130260 con 0x7f018c130670 2023-10-03T10:17:14.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.812+0000 7f0191c69700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f018c09a350 con 0x7f018c135e30 2023-10-03T10:17:14.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.812+0000 7f0191c69700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f018c071ff0 con 0x7f018c133510 2023-10-03T10:17:14.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.812+0000 7f018b7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f018c130670 0x7f018c130a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55224/0 (socket says 172.21.15.3:55224) 2023-10-03T10:17:14.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.812+0000 7f018b7fe700 1 -- 172.21.15.3:0/3130415822 learned_addr learned my addr 172.21.15.3:0/3130415822 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:14.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.812+0000 7f018bfff700 1 --2- 172.21.15.3:0/3130415822 >> v2:172.21.15.3:3300/0 conn(0x7f018c135e30 0x7f018c138210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:14.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.812+0000 7f018affd700 1 --2- 172.21.15.3:0/3130415822 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f018c133510 0x7f018c1358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.812+0000 7f018bfff700 1 -- 172.21.15.3:0/3130415822 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f018c133510 msgr2=0x7f018c1358f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.812+0000 7f018bfff700 1 --2- 172.21.15.3:0/3130415822 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f018c133510 0x7f018c1358f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.812+0000 7f018bfff700 1 -- 172.21.15.3:0/3130415822 >> v1:172.21.15.3:6789/0 conn(0x7f018c130670 legacy=0x7f018c130a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.812+0000 7f018a7fc700 1 -- 172.21.15.3:0/3130415822 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2519889765 0 0) 0x7f018c130260 con 0x7f018c130670 2023-10-03T10:17:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.812+0000 7f018bfff700 1 -- 172.21.15.3:0/3130415822 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f018c09ecf0 con 0x7f018c135e30 2023-10-03T10:17:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.812+0000 7f018affd700 1 --2- 172.21.15.3:0/3130415822 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f018c133510 0x7f018c1358f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.813+0000 7f018bfff700 1 --2- 172.21.15.3:0/3130415822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f018c135e30 0x7f018c138210 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f0184009d90 tx=0x7f018400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5f54b1d1e80cc8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.813+0000 7f018a7fc700 1 -- 172.21.15.3:0/3130415822 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018400e070 con 0x7f018c135e30 2023-10-03T10:17:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.813+0000 7f018a7fc700 1 -- 172.21.15.3:0/3130415822 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f018400bb20 con 0x7f018c135e30 2023-10-03T10:17:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.813+0000 7f018a7fc700 1 -- 172.21.15.3:0/3130415822 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0184012430 con 0x7f018c135e30 2023-10-03T10:17:14.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.814+0000 7f0191c69700 1 -- 172.21.15.3:0/3130415822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f018c135e30 msgr2=0x7f018c138210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:14.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.814+0000 7f0191c69700 1 --2- 172.21.15.3:0/3130415822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f018c135e30 0x7f018c138210 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f0184009d90 tx=0x7f018400b330 comp rx=0 tx=0).stop 2023-10-03T10:17:14.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.814+0000 7f0191c69700 1 -- 172.21.15.3:0/3130415822 shutdown_connections 2023-10-03T10:17:14.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.814+0000 7f0191c69700 1 --2- 172.21.15.3:0/3130415822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f018c135e30 0x7f018c138210 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:14.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.814+0000 7f0191c69700 1 --2- 172.21.15.3:0/3130415822 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f018c133510 0x7f018c1358f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:14.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.814+0000 7f0191c69700 1 -- 172.21.15.3:0/3130415822 >> 172.21.15.3:0/3130415822 conn(0x7f018c0a2aa0 msgr2=0x7f018c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:14.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.814+0000 7f0191c69700 1 -- 172.21.15.3:0/3130415822 shutdown_connections 2023-10-03T10:17:14.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.814+0000 7f0191c69700 1 -- 172.21.15.3:0/3130415822 wait complete. 2023-10-03T10:17:14.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.815+0000 7f0191c69700 1 Processor -- start 2023-10-03T10:17:14.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.815+0000 7f0191c69700 1 -- start start 2023-10-03T10:17:14.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.815+0000 7f0191c69700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f018c130670 0x7f018c1c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:14.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.815+0000 7f0191c69700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f018c133510 0x7f018c1c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:14.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.815+0000 7f0191c69700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f018c135e30 0x7f018c1c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:14.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.815+0000 7f0191c69700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018c11dd30 con 0x7f018c133510 2023-10-03T10:17:14.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.815+0000 7f0191c69700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f018c09e540 con 0x7f018c135e30 2023-10-03T10:17:14.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.815+0000 7f0191c69700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f018c12fea0 con 0x7f018c130670 2023-10-03T10:17:14.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.816+0000 7f018b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f018c130670 0x7f018c1c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:14.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.816+0000 7f018affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f018c133510 0x7f018c1c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:14.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.816+0000 7f018affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f018c133510 0x7f018c1c8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41386/0 (socket says 172.21.15.3:41386) 2023-10-03T10:17:14.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.816+0000 7f018b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f018c130670 0x7f018c1c82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53286/0 (socket says 172.21.15.3:53286) 2023-10-03T10:17:14.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.816+0000 7f018affd700 1 -- 172.21.15.3:0/543544973 learned_addr learned my addr 172.21.15.3:0/543544973 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:14.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.816+0000 7f018bfff700 1 --2- 172.21.15.3:0/543544973 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f018c135e30 0x7f018c1c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:14.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.816+0000 7f018affd700 1 -- 172.21.15.3:0/543544973 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f018c130670 msgr2=0x7f018c1c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:14.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.816+0000 7f018affd700 1 --2- 172.21.15.3:0/543544973 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f018c130670 0x7f018c1c82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:14.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.816+0000 7f018affd700 1 -- 172.21.15.3:0/543544973 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f018c135e30 msgr2=0x7f018c1c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:14.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.816+0000 7f018affd700 1 --2- 172.21.15.3:0/543544973 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f018c135e30 0x7f018c1c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:14.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.816+0000 7f018affd700 1 -- 172.21.15.3:0/543544973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f018000b740 con 0x7f018c133510 2023-10-03T10:17:14.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.816+0000 7f018b7fe700 1 --2- 172.21.15.3:0/543544973 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f018c130670 0x7f018c1c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:14.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.816+0000 7f018affd700 1 --2- 172.21.15.3:0/543544973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f018c133510 0x7f018c1c8810 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f018000ed40 tx=0x7f018000d630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:14.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.816+0000 7f0173fff700 1 -- 172.21.15.3:0/543544973 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018000dd90 con 0x7f018c133510 2023-10-03T10:17:14.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.817+0000 7f0173fff700 1 -- 172.21.15.3:0/543544973 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0180010440 con 0x7f018c133510 2023-10-03T10:17:14.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.817+0000 7f0173fff700 1 -- 172.21.15.3:0/543544973 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01800107b0 con 0x7f018c133510 2023-10-03T10:17:14.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.817+0000 7f0191c69700 1 -- 172.21.15.3:0/543544973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0184009a70 con 0x7f018c133510 2023-10-03T10:17:14.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.817+0000 7f0191c69700 1 -- 172.21.15.3:0/543544973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f018c11ea70 con 0x7f018c133510 2023-10-03T10:17:14.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.818+0000 7f0173fff700 1 -- 172.21.15.3:0/543544973 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0180010910 con 0x7f018c133510 2023-10-03T10:17:14.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.818+0000 7f0191c69700 1 -- 172.21.15.3:0/543544973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f018c070e70 con 0x7f018c133510 2023-10-03T10:17:14.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.818+0000 7f0173fff700 1 -- 172.21.15.3:0/543544973 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f018003fe30 con 0x7f018c133510 2023-10-03T10:17:14.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.823+0000 7f0173fff700 1 -- 172.21.15.3:0/543544973 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0180043030 con 0x7f018c133510 2023-10-03T10:17:14.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.900+0000 7fdba35c0700 1 Processor -- start 2023-10-03T10:17:14.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.900+0000 7fdba35c0700 1 -- start start 2023-10-03T10:17:14.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.900+0000 7fdba35c0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb9c0a0a30 0x7fdb9c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:14.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.900+0000 7fdba35c0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb9c138d80 0x7fdb9c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:14.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.900+0000 7fdba35c0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb9c0a2a70 con 0x7fdb9c09d630 2023-10-03T10:17:14.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.900+0000 7fdba35c0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c0a01d0 con 0x7fdb9c0a0a30 2023-10-03T10:17:14.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.900+0000 7fdba35c0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c09e2b0 con 0x7fdb9c138d80 2023-10-03T10:17:14.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.901+0000 7fdba0b5b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdb9c09d630 0x7fdb9c0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55230/0 (socket says 172.21.15.3:55230) 2023-10-03T10:17:14.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.901+0000 7fdba0b5b700 1 -- 172.21.15.3:0/2033262824 learned_addr learned my addr 172.21.15.3:0/2033262824 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:14.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.901+0000 7fdba1b5d700 1 --2- 172.21.15.3:0/2033262824 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb9c138d80 0x7fdb9c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:14.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.901+0000 7fdba135c700 1 --2- 172.21.15.3:0/2033262824 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb9c0a0a30 0x7fdb9c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:14.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.901+0000 7fdb93fff700 1 -- 172.21.15.3:0/2033262824 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2400133174 0 0) 0x7fdb9c0a2a70 con 0x7fdb9c09d630 2023-10-03T10:17:14.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.901+0000 7fdb93fff700 1 -- 172.21.15.3:0/2033262824 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb880035b0 con 0x7fdb9c09d630 2023-10-03T10:17:14.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.901+0000 7fdba135c700 1 -- 172.21.15.3:0/2033262824 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb9c138d80 msgr2=0x7fdb9c13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:14.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.901+0000 7fdba135c700 1 --2- 172.21.15.3:0/2033262824 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb9c138d80 0x7fdb9c13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:14.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.901+0000 7fdba135c700 1 -- 172.21.15.3:0/2033262824 >> v1:172.21.15.3:6789/0 conn(0x7fdb9c09d630 legacy=0x7fdb9c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:14.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.901+0000 7fdba135c700 1 -- 172.21.15.3:0/2033262824 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9c09dc50 con 0x7fdb9c0a0a30 2023-10-03T10:17:14.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.901+0000 7fdba1b5d700 1 --2- 172.21.15.3:0/2033262824 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb9c138d80 0x7fdb9c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:14.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.901+0000 7fdba135c700 1 --2- 172.21.15.3:0/2033262824 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb9c0a0a30 0x7fdb9c0a0e00 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009870 tx=0x7fdb98009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b9442b336fe34dd server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:14.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.902+0000 7fdb93fff700 1 -- 172.21.15.3:0/2033262824 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98004230 con 0x7fdb9c0a0a30 2023-10-03T10:17:14.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.902+0000 7fdb93fff700 1 -- 172.21.15.3:0/2033262824 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb98004390 con 0x7fdb9c0a0a30 2023-10-03T10:17:14.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.902+0000 7fdb93fff700 1 -- 172.21.15.3:0/2033262824 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb980036a0 con 0x7fdb9c0a0a30 2023-10-03T10:17:14.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.902+0000 7fdba35c0700 1 -- 172.21.15.3:0/2033262824 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb9c0a0a30 msgr2=0x7fdb9c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:14.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.902+0000 7fdba35c0700 1 --2- 172.21.15.3:0/2033262824 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb9c0a0a30 0x7fdb9c0a0e00 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009870 tx=0x7fdb98009b80 comp rx=0 tx=0).stop 2023-10-03T10:17:14.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.902+0000 7fdba35c0700 1 -- 172.21.15.3:0/2033262824 shutdown_connections 2023-10-03T10:17:14.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.902+0000 7fdba35c0700 1 --2- 172.21.15.3:0/2033262824 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb9c0a0a30 0x7fdb9c0a0e00 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:14.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.902+0000 7fdba35c0700 1 --2- 172.21.15.3:0/2033262824 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb9c138d80 0x7fdb9c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:14.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.902+0000 7fdba35c0700 1 -- 172.21.15.3:0/2033262824 >> 172.21.15.3:0/2033262824 conn(0x7fdb9c0a3050 msgr2=0x7fdb9c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:14.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.902+0000 7fdba35c0700 1 -- 172.21.15.3:0/2033262824 shutdown_connections 2023-10-03T10:17:14.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.902+0000 7fdba35c0700 1 -- 172.21.15.3:0/2033262824 wait complete. 2023-10-03T10:17:14.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.903+0000 7fdba35c0700 1 Processor -- start 2023-10-03T10:17:14.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.903+0000 7fdba35c0700 1 -- start start 2023-10-03T10:17:14.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.904+0000 7fdba35c0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb9c09d630 0x7fdb9c133900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:14.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.904+0000 7fdba35c0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb9c0a0a30 0x7fdb9c133e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:14.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.904+0000 7fdba35c0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb9c138d80 0x7fdb9c134380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:14.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.904+0000 7fdba35c0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c137340 con 0x7fdb9c0a0a30 2023-10-03T10:17:14.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.904+0000 7fdba35c0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c135910 con 0x7fdb9c09d630 2023-10-03T10:17:14.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.904+0000 7fdba35c0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c137fc0 con 0x7fdb9c138d80 2023-10-03T10:17:14.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.904+0000 7fdba135c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb9c09d630 0x7fdb9c133900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:14.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.904+0000 7fdba135c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb9c09d630 0x7fdb9c133900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50876/0 (socket says 172.21.15.3:50876) 2023-10-03T10:17:14.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.904+0000 7fdba135c700 1 -- 172.21.15.3:0/1468528957 learned_addr learned my addr 172.21.15.3:0/1468528957 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:14.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.904+0000 7fdba0b5b700 1 --2- 172.21.15.3:0/1468528957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb9c0a0a30 0x7fdb9c133e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:14.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.904+0000 7fdba1b5d700 1 --2- 172.21.15.3:0/1468528957 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb9c138d80 0x7fdb9c134380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:14.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.904+0000 7fdba135c700 1 -- 172.21.15.3:0/1468528957 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb9c138d80 msgr2=0x7fdb9c134380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:14.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.904+0000 7fdba135c700 1 --2- 172.21.15.3:0/1468528957 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb9c138d80 0x7fdb9c134380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:14.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.904+0000 7fdba135c700 1 -- 172.21.15.3:0/1468528957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb9c0a0a30 msgr2=0x7fdb9c133e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:14.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.904+0000 7fdba135c700 1 --2- 172.21.15.3:0/1468528957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb9c0a0a30 0x7fdb9c133e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:14.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.904+0000 7fdba135c700 1 -- 172.21.15.3:0/1468528957 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb94009a70 con 0x7fdb9c09d630 2023-10-03T10:17:14.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.904+0000 7fdba1b5d700 1 --2- 172.21.15.3:0/1468528957 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb9c138d80 0x7fdb9c134380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:17:14.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.905+0000 7fdba135c700 1 --2- 172.21.15.3:0/1468528957 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb9c09d630 0x7fdb9c133900 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009840 tx=0x7fdb98004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:14.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.905+0000 7fdb91ffb700 1 -- 172.21.15.3:0/1468528957 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98003da0 con 0x7fdb9c09d630 2023-10-03T10:17:14.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.905+0000 7fdb91ffb700 1 -- 172.21.15.3:0/1468528957 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9801e070 con 0x7fdb9c09d630 2023-10-03T10:17:14.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.905+0000 7fdba35c0700 1 -- 172.21.15.3:0/1468528957 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb980094a0 con 0x7fdb9c09d630 2023-10-03T10:17:14.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.905+0000 7fdb91ffb700 1 -- 172.21.15.3:0/1468528957 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98011870 con 0x7fdb9c09d630 2023-10-03T10:17:14.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.905+0000 7fdba35c0700 1 -- 172.21.15.3:0/1468528957 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb9c138720 con 0x7fdb9c09d630 2023-10-03T10:17:14.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.906+0000 7fdba35c0700 1 -- 172.21.15.3:0/1468528957 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb9c0747f0 con 0x7fdb9c09d630 2023-10-03T10:17:14.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.907+0000 7fdb91ffb700 1 -- 172.21.15.3:0/1468528957 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdb980119d0 con 0x7fdb9c09d630 2023-10-03T10:17:14.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.907+0000 7fdb91ffb700 1 -- 172.21.15.3:0/1468528957 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fdb98040b30 con 0x7fdb9c09d630 2023-10-03T10:17:14.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.911+0000 7fdb91ffb700 1 -- 172.21.15.3:0/1468528957 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdb98018350 con 0x7fdb9c09d630 2023-10-03T10:17:14.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.939+0000 7fda009e3700 1 Processor -- start 2023-10-03T10:17:14.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.940+0000 7fda009e3700 1 -- start start 2023-10-03T10:17:14.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.940+0000 7fda009e3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9fc09d630 0x7fd9fc0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.940+0000 7fda009e3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9fc138d80 0x7fd9fc13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.940+0000 7fda009e3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9fc0a2a70 con 0x7fd9fc0a0a30 2023-10-03T10:17:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.940+0000 7fda009e3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc0a01d0 con 0x7fd9fc09d630 2023-10-03T10:17:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.940+0000 7fda009e3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc09e2b0 con 0x7fd9fc138d80 2023-10-03T10:17:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.940+0000 7fd9f1d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9fc09d630 0x7fd9fc0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.940+0000 7fd9fa59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd9fc0a0a30 0x7fd9fc0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55246/0 (socket says 172.21.15.3:55246) 2023-10-03T10:17:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.940+0000 7fd9fa59c700 1 -- 172.21.15.3:0/1948255211 learned_addr learned my addr 172.21.15.3:0/1948255211 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.941+0000 7fd9f9d9b700 1 -- 172.21.15.3:0/1948255211 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3711092338 0 0) 0x7fd9fc0a2a70 con 0x7fd9fc0a0a30 2023-10-03T10:17:14.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.941+0000 7fd9f1d9b700 1 -- 172.21.15.3:0/1948255211 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9fc138d80 msgr2=0x7fd9fc13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:14.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.941+0000 7fd9f1d9b700 1 --2- 172.21.15.3:0/1948255211 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9fc138d80 0x7fd9fc13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:14.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.941+0000 7fd9f1d9b700 1 -- 172.21.15.3:0/1948255211 >> v1:172.21.15.3:6789/0 conn(0x7fd9fc0a0a30 legacy=0x7fd9fc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:14.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.941+0000 7fd9f1d9b700 1 -- 172.21.15.3:0/1948255211 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9fc09dc50 con 0x7fd9fc09d630 2023-10-03T10:17:14.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.941+0000 7fd9f1d9b700 1 --2- 172.21.15.3:0/1948255211 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9fc09d630 0x7fd9fc0a1510 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009870 tx=0x7fd9e4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bab865645547d22 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:14.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.941+0000 7fd9f9d9b700 1 -- 172.21.15.3:0/1948255211 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4011830 con 0x7fd9fc09d630 2023-10-03T10:17:14.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.941+0000 7fd9f9d9b700 1 -- 172.21.15.3:0/1948255211 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e4011990 con 0x7fd9fc09d630 2023-10-03T10:17:14.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.941+0000 7fd9f9d9b700 1 -- 172.21.15.3:0/1948255211 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4011d20 con 0x7fd9fc09d630 2023-10-03T10:17:14.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.942+0000 7fda009e3700 1 -- 172.21.15.3:0/1948255211 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9fc09d630 msgr2=0x7fd9fc0a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:14.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.942+0000 7fda009e3700 1 --2- 172.21.15.3:0/1948255211 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9fc09d630 0x7fd9fc0a1510 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009870 tx=0x7fd9e4009b80 comp rx=0 tx=0).stop 2023-10-03T10:17:14.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.942+0000 7fda009e3700 1 -- 172.21.15.3:0/1948255211 shutdown_connections 2023-10-03T10:17:14.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.942+0000 7fda009e3700 1 --2- 172.21.15.3:0/1948255211 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9fc138d80 0x7fd9fc13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:14.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.942+0000 7fda009e3700 1 --2- 172.21.15.3:0/1948255211 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9fc09d630 0x7fd9fc0a1510 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:14.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.942+0000 7fda009e3700 1 -- 172.21.15.3:0/1948255211 >> 172.21.15.3:0/1948255211 conn(0x7fd9fc0a3050 msgr2=0x7fd9fc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:14.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.942+0000 7fda009e3700 1 -- 172.21.15.3:0/1948255211 shutdown_connections 2023-10-03T10:17:14.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.942+0000 7fda009e3700 1 -- 172.21.15.3:0/1948255211 wait complete. 2023-10-03T10:17:14.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.944+0000 7fda009e3700 1 Processor -- start 2023-10-03T10:17:14.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.944+0000 7fda009e3700 1 -- start start 2023-10-03T10:17:14.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.944+0000 7fda009e3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9fc09d630 0x7fd9fc133790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:14.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.944+0000 7fda009e3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9fc0a0a30 0x7fd9fc133cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:14.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.944+0000 7fda009e3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9fc138d80 0x7fd9fc134210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:14.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.944+0000 7fda009e3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc138bd0 con 0x7fd9fc0a0a30 2023-10-03T10:17:14.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.944+0000 7fda009e3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc136fd0 con 0x7fd9fc138d80 2023-10-03T10:17:14.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.944+0000 7fda009e3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc136660 con 0x7fd9fc09d630 2023-10-03T10:17:14.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.944+0000 7fd9f1d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9fc0a0a30 0x7fd9fc133cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:14.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.944+0000 7fd9f1d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9fc0a0a30 0x7fd9fc133cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41392/0 (socket says 172.21.15.3:41392) 2023-10-03T10:17:14.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.944+0000 7fd9f1d9b700 1 -- 172.21.15.3:0/4063415977 learned_addr learned my addr 172.21.15.3:0/4063415977 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:14.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.944+0000 7fd9fad9d700 1 --2- 172.21.15.3:0/4063415977 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9fc138d80 0x7fd9fc134210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:14.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.944+0000 7fd9fa59c700 1 --2- 172.21.15.3:0/4063415977 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9fc09d630 0x7fd9fc133790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:14.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.945+0000 7fd9f1d9b700 1 -- 172.21.15.3:0/4063415977 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9fc09d630 msgr2=0x7fd9fc133790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:14.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.945+0000 7fd9f1d9b700 1 --2- 172.21.15.3:0/4063415977 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9fc09d630 0x7fd9fc133790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:14.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.945+0000 7fd9f1d9b700 1 -- 172.21.15.3:0/4063415977 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9fc138d80 msgr2=0x7fd9fc134210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:14.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.945+0000 7fd9f1d9b700 1 --2- 172.21.15.3:0/4063415977 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9fc138d80 0x7fd9fc134210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:14.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.945+0000 7fd9f1d9b700 1 -- 172.21.15.3:0/4063415977 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e40094a0 con 0x7fd9fc0a0a30 2023-10-03T10:17:14.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.945+0000 7fd9f1d9b700 1 --2- 172.21.15.3:0/4063415977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9fc0a0a30 0x7fd9fc133cd0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4000c00 tx=0x7fd9e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:14.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.945+0000 7fd9f37fe700 1 -- 172.21.15.3:0/4063415977 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4004480 con 0x7fd9fc0a0a30 2023-10-03T10:17:14.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.945+0000 7fd9f37fe700 1 -- 172.21.15.3:0/4063415977 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e401f070 con 0x7fd9fc0a0a30 2023-10-03T10:17:14.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.945+0000 7fd9f37fe700 1 -- 172.21.15.3:0/4063415977 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e40038e0 con 0x7fd9fc0a0a30 2023-10-03T10:17:14.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.945+0000 7fda009e3700 1 -- 172.21.15.3:0/4063415977 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9fc0a4be0 con 0x7fd9fc0a0a30 2023-10-03T10:17:14.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.945+0000 7fda009e3700 1 -- 172.21.15.3:0/4063415977 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9fc137a40 con 0x7fd9fc0a0a30 2023-10-03T10:17:14.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.946+0000 7fd9f37fe700 1 -- 172.21.15.3:0/4063415977 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd9e4021030 con 0x7fd9fc0a0a30 2023-10-03T10:17:14.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.946+0000 7fd9f37fe700 1 -- 172.21.15.3:0/4063415977 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd9e40408c0 con 0x7fd9fc0a0a30 2023-10-03T10:17:14.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.946+0000 7fda009e3700 1 -- 172.21.15.3:0/4063415977 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9fc0747f0 con 0x7fd9fc0a0a30 2023-10-03T10:17:14.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:14.951+0000 7fd9f37fe700 1 -- 172.21.15.3:0/4063415977 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd9e4003a40 con 0x7fd9fc0a0a30 2023-10-03T10:17:15.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.037+0000 7f0191c69700 1 -- 172.21.15.3:0/543544973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f018c074ea0 con 0x7f018c133510 2023-10-03T10:17:15.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.038+0000 7f0173fff700 1 -- 172.21.15.3:0/543544973 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0180017930 con 0x7f018c133510 2023-10-03T10:17:15.038 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:15.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.040+0000 7f0191c69700 1 -- 172.21.15.3:0/543544973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f018c133510 msgr2=0x7f018c1c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:15.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.040+0000 7f0191c69700 1 --2- 172.21.15.3:0/543544973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f018c133510 0x7f018c1c8810 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f018000ed40 tx=0x7f018000d630 comp rx=0 tx=0).stop 2023-10-03T10:17:15.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.040+0000 7f0191c69700 1 -- 172.21.15.3:0/543544973 shutdown_connections 2023-10-03T10:17:15.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.040+0000 7f0191c69700 1 --2- 172.21.15.3:0/543544973 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f018c135e30 0x7f018c1c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:15.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.040+0000 7f0191c69700 1 --2- 172.21.15.3:0/543544973 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f018c130670 0x7f018c1c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:15.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.040+0000 7f0191c69700 1 --2- 172.21.15.3:0/543544973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f018c133510 0x7f018c1c8810 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:15.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.040+0000 7f0191c69700 1 -- 172.21.15.3:0/543544973 >> 172.21.15.3:0/543544973 conn(0x7f018c0a2aa0 msgr2=0x7f018c132390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:15.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.040+0000 7f0191c69700 1 -- 172.21.15.3:0/543544973 shutdown_connections 2023-10-03T10:17:15.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.040+0000 7f0191c69700 1 -- 172.21.15.3:0/543544973 wait complete. 2023-10-03T10:17:15.054 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:15.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.120+0000 7fdba35c0700 1 -- 172.21.15.3:0/1468528957 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdb9c0a4640 con 0x7fdb9c09d630 2023-10-03T10:17:15.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.120+0000 7fdb91ffb700 1 -- 172.21.15.3:0/1468528957 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdb98010b70 con 0x7fdb9c09d630 2023-10-03T10:17:15.121 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:15.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.123+0000 7fdba35c0700 1 -- 172.21.15.3:0/1468528957 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb9c09d630 msgr2=0x7fdb9c133900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:15.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.123+0000 7fdba35c0700 1 --2- 172.21.15.3:0/1468528957 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb9c09d630 0x7fdb9c133900 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009840 tx=0x7fdb98004750 comp rx=0 tx=0).stop 2023-10-03T10:17:15.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.123+0000 7fdba35c0700 1 -- 172.21.15.3:0/1468528957 shutdown_connections 2023-10-03T10:17:15.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.123+0000 7fdba35c0700 1 --2- 172.21.15.3:0/1468528957 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdb9c138d80 0x7fdb9c134380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:15.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.123+0000 7fdba35c0700 1 --2- 172.21.15.3:0/1468528957 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdb9c09d630 0x7fdb9c133900 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:15.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.123+0000 7fdba35c0700 1 --2- 172.21.15.3:0/1468528957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb9c0a0a30 0x7fdb9c133e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:15.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.123+0000 7fdba35c0700 1 -- 172.21.15.3:0/1468528957 >> 172.21.15.3:0/1468528957 conn(0x7fdb9c0a3050 msgr2=0x7fdb9c13ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:15.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.123+0000 7fdba35c0700 1 -- 172.21.15.3:0/1468528957 shutdown_connections 2023-10-03T10:17:15.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.123+0000 7fdba35c0700 1 -- 172.21.15.3:0/1468528957 wait complete. 2023-10-03T10:17:15.136 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:15.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.159+0000 7fda009e3700 1 -- 172.21.15.3:0/4063415977 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd9fc072980 con 0x7fd9fc0a0a30 2023-10-03T10:17:15.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.159+0000 7fd9f37fe700 1 -- 172.21.15.3:0/4063415977 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd9e40193b0 con 0x7fd9fc0a0a30 2023-10-03T10:17:15.160 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:15.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.162+0000 7fda009e3700 1 -- 172.21.15.3:0/4063415977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9fc0a0a30 msgr2=0x7fd9fc133cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:15.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.162+0000 7fda009e3700 1 --2- 172.21.15.3:0/4063415977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9fc0a0a30 0x7fd9fc133cd0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4000c00 tx=0x7fd9e4004750 comp rx=0 tx=0).stop 2023-10-03T10:17:15.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.162+0000 7fda009e3700 1 -- 172.21.15.3:0/4063415977 shutdown_connections 2023-10-03T10:17:15.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.162+0000 7fda009e3700 1 --2- 172.21.15.3:0/4063415977 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9fc138d80 0x7fd9fc134210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:15.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.162+0000 7fda009e3700 1 --2- 172.21.15.3:0/4063415977 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9fc09d630 0x7fd9fc133790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:15.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.162+0000 7fda009e3700 1 --2- 172.21.15.3:0/4063415977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9fc0a0a30 0x7fd9fc133cd0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:15.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.162+0000 7fda009e3700 1 -- 172.21.15.3:0/4063415977 >> 172.21.15.3:0/4063415977 conn(0x7fd9fc0a3050 msgr2=0x7fd9fc13af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:15.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.162+0000 7fda009e3700 1 -- 172.21.15.3:0/4063415977 shutdown_connections 2023-10-03T10:17:15.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.162+0000 7fda009e3700 1 -- 172.21.15.3:0/4063415977 wait complete. 2023-10-03T10:17:15.176 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:15.701 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:15.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.847+0000 7fd663410700 1 Processor -- start 2023-10-03T10:17:15.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.847+0000 7fd663410700 1 -- start start 2023-10-03T10:17:15.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.847+0000 7fd663410700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd65c09f410 0x7fd65c136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:15.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.847+0000 7fd663410700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd65c136fe0 0x7fd65c1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:15.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.847+0000 7fd663410700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65c070940 con 0x7fd65c09e930 2023-10-03T10:17:15.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.847+0000 7fd663410700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c06fd60 con 0x7fd65c136fe0 2023-10-03T10:17:15.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.847+0000 7fd663410700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c06fa80 con 0x7fd65c09f410 2023-10-03T10:17:15.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.848+0000 7fd6611ac700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd65c09e930 0x7fd65c09ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55262/0 (socket says 172.21.15.3:55262) 2023-10-03T10:17:15.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.848+0000 7fd6619ad700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd65c136fe0 0x7fd65c1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:15.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.848+0000 7fd6609ab700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd65c09f410 0x7fd65c136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:15.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.848+0000 7fd6611ac700 1 -- 172.21.15.3:0/4078656876 learned_addr learned my addr 172.21.15.3:0/4078656876 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:15.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.848+0000 7fd653fff700 1 -- 172.21.15.3:0/4078656876 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 589155730 0 0) 0x7fd65c070940 con 0x7fd65c09e930 2023-10-03T10:17:15.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.848+0000 7fd653fff700 1 -- 172.21.15.3:0/4078656876 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6480035b0 con 0x7fd65c09e930 2023-10-03T10:17:15.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.848+0000 7fd6619ad700 1 -- 172.21.15.3:0/4078656876 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd65c09f410 msgr2=0x7fd65c136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:15.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.848+0000 7fd6619ad700 1 --2- 172.21.15.3:0/4078656876 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd65c09f410 0x7fd65c136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:15.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.848+0000 7fd6619ad700 1 -- 172.21.15.3:0/4078656876 >> v1:172.21.15.3:6789/0 conn(0x7fd65c09e930 legacy=0x7fd65c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:15.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.848+0000 7fd6619ad700 1 -- 172.21.15.3:0/4078656876 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c09fb60 con 0x7fd65c136fe0 2023-10-03T10:17:15.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.848+0000 7fd6609ab700 1 --2- 172.21.15.3:0/4078656876 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd65c09f410 0x7fd65c136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:15.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.848+0000 7fd6619ad700 1 --2- 172.21.15.3:0/4078656876 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd65c136fe0 0x7fd65c1393c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd654009dc0 tx=0x7fd65400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0fc28cbb1c6b00e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:15.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.848+0000 7fd653fff700 1 -- 172.21.15.3:0/4078656876 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65400e070 con 0x7fd65c136fe0 2023-10-03T10:17:15.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.848+0000 7fd653fff700 1 -- 172.21.15.3:0/4078656876 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65400bb20 con 0x7fd65c136fe0 2023-10-03T10:17:15.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.848+0000 7fd653fff700 1 -- 172.21.15.3:0/4078656876 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654012430 con 0x7fd65c136fe0 2023-10-03T10:17:15.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.849+0000 7fd663410700 1 -- 172.21.15.3:0/4078656876 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd65c136fe0 msgr2=0x7fd65c1393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:15.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.849+0000 7fd663410700 1 --2- 172.21.15.3:0/4078656876 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd65c136fe0 0x7fd65c1393c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd654009dc0 tx=0x7fd65400b330 comp rx=0 tx=0).stop 2023-10-03T10:17:15.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.849+0000 7fd663410700 1 -- 172.21.15.3:0/4078656876 shutdown_connections 2023-10-03T10:17:15.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.849+0000 7fd663410700 1 --2- 172.21.15.3:0/4078656876 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd65c136fe0 0x7fd65c1393c0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:15.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.849+0000 7fd663410700 1 --2- 172.21.15.3:0/4078656876 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd65c09f410 0x7fd65c136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:15.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.849+0000 7fd663410700 1 -- 172.21.15.3:0/4078656876 >> 172.21.15.3:0/4078656876 conn(0x7fd65c0a4210 msgr2=0x7fd65c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:15.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.849+0000 7fd663410700 1 -- 172.21.15.3:0/4078656876 shutdown_connections 2023-10-03T10:17:15.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.849+0000 7fd663410700 1 -- 172.21.15.3:0/4078656876 wait complete. 2023-10-03T10:17:15.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.850+0000 7fd663410700 1 Processor -- start 2023-10-03T10:17:15.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.851+0000 7fd663410700 1 -- start start 2023-10-03T10:17:15.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.851+0000 7fd663410700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd65c09e930 0x7fd65c11e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:15.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.851+0000 7fd663410700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd65c09f410 0x7fd65c11ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:15.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.851+0000 7fd663410700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd65c136fe0 0x7fd65c11f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:15.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.851+0000 7fd663410700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c134540 con 0x7fd65c09f410 2023-10-03T10:17:15.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.851+0000 7fd663410700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c0a3310 con 0x7fd65c136fe0 2023-10-03T10:17:15.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.851+0000 7fd663410700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c071d20 con 0x7fd65c09e930 2023-10-03T10:17:15.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.851+0000 7fd6611ac700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd65c09e930 0x7fd65c11e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:15.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.851+0000 7fd6609ab700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd65c09f410 0x7fd65c11ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:15.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.851+0000 7fd6619ad700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd65c136fe0 0x7fd65c11f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:15.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.851+0000 7fd6611ac700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd65c09e930 0x7fd65c11e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53356/0 (socket says 172.21.15.3:53356) 2023-10-03T10:17:15.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.851+0000 7fd6609ab700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd65c09f410 0x7fd65c11ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41400/0 (socket says 172.21.15.3:41400) 2023-10-03T10:17:15.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.851+0000 7fd6619ad700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd65c136fe0 0x7fd65c11f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50908/0 (socket says 172.21.15.3:50908) 2023-10-03T10:17:15.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.851+0000 7fd6611ac700 1 -- 172.21.15.3:0/1246328063 learned_addr learned my addr 172.21.15.3:0/1246328063 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:15.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.852+0000 7fd6611ac700 1 -- 172.21.15.3:0/1246328063 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd65c136fe0 msgr2=0x7fd65c11f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:15.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.852+0000 7fd6611ac700 1 --2- 172.21.15.3:0/1246328063 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd65c136fe0 0x7fd65c11f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:15.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.852+0000 7fd6611ac700 1 -- 172.21.15.3:0/1246328063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd65c09f410 msgr2=0x7fd65c11ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:15.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.852+0000 7fd6611ac700 1 --2- 172.21.15.3:0/1246328063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd65c09f410 0x7fd65c11ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:15.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.852+0000 7fd6611ac700 1 -- 172.21.15.3:0/1246328063 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64c0094a0 con 0x7fd65c09e930 2023-10-03T10:17:15.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.852+0000 7fd6611ac700 1 --2- 172.21.15.3:0/1246328063 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd65c09e930 0x7fd65c11e770 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd65800c730 tx=0x7fd65800ca40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:15.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.852+0000 7fd651ffb700 1 -- 172.21.15.3:0/1246328063 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd658005d70 con 0x7fd65c09e930 2023-10-03T10:17:15.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.852+0000 7fd651ffb700 1 -- 172.21.15.3:0/1246328063 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6580070d0 con 0x7fd65c09e930 2023-10-03T10:17:15.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.852+0000 7fd651ffb700 1 -- 172.21.15.3:0/1246328063 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd658007440 con 0x7fd65c09e930 2023-10-03T10:17:15.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.852+0000 7fd663410700 1 -- 172.21.15.3:0/1246328063 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd654009a70 con 0x7fd65c09e930 2023-10-03T10:17:15.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.852+0000 7fd663410700 1 -- 172.21.15.3:0/1246328063 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65c120370 con 0x7fd65c09e930 2023-10-03T10:17:15.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.853+0000 7fd651ffb700 1 -- 172.21.15.3:0/1246328063 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd6580075a0 con 0x7fd65c09e930 2023-10-03T10:17:15.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.853+0000 7fd651ffb700 1 -- 172.21.15.3:0/1246328063 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd65803eae0 con 0x7fd65c09e930 2023-10-03T10:17:15.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.853+0000 7fd663410700 1 -- 172.21.15.3:0/1246328063 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd65c121b30 con 0x7fd65c09e930 2023-10-03T10:17:15.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:15.858+0000 7fd651ffb700 1 -- 172.21.15.3:0/1246328063 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd658019b40 con 0x7fd65c09e930 2023-10-03T10:17:16.055 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:16.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.062+0000 7fd663410700 1 -- 172.21.15.3:0/1246328063 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd65c074ea0 con 0x7fd65c09e930 2023-10-03T10:17:16.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.062+0000 7fd651ffb700 1 -- 172.21.15.3:0/1246328063 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd658017020 con 0x7fd65c09e930 2023-10-03T10:17:16.062 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.064+0000 7fd663410700 1 -- 172.21.15.3:0/1246328063 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd65c09e930 msgr2=0x7fd65c11e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.064+0000 7fd663410700 1 --2- 172.21.15.3:0/1246328063 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd65c09e930 0x7fd65c11e770 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd65800c730 tx=0x7fd65800ca40 comp rx=0 tx=0).stop 2023-10-03T10:17:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.064+0000 7fd663410700 1 -- 172.21.15.3:0/1246328063 shutdown_connections 2023-10-03T10:17:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.065+0000 7fd663410700 1 --2- 172.21.15.3:0/1246328063 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd65c136fe0 0x7fd65c11f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.065+0000 7fd663410700 1 --2- 172.21.15.3:0/1246328063 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd65c09e930 0x7fd65c11e770 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.065+0000 7fd663410700 1 --2- 172.21.15.3:0/1246328063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd65c09f410 0x7fd65c11ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.065+0000 7fd663410700 1 -- 172.21.15.3:0/1246328063 >> 172.21.15.3:0/1246328063 conn(0x7fd65c0a4210 msgr2=0x7fd65c134900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:16.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.065+0000 7fd663410700 1 -- 172.21.15.3:0/1246328063 shutdown_connections 2023-10-03T10:17:16.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.065+0000 7fd663410700 1 -- 172.21.15.3:0/1246328063 wait complete. 2023-10-03T10:17:16.080 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:16.137 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:16.176 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:16.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.196+0000 7f53a1be6700 1 Processor -- start 2023-10-03T10:17:16.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.196+0000 7f53a1be6700 1 -- start start 2023-10-03T10:17:16.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.197+0000 7f53a1be6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f539c1338a0 0x7f539c133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:16.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.197+0000 7f53a1be6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f539c09de50 0x7f539c09e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:16.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.197+0000 7f53a1be6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f539c09e760 0x7f539c13a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:16.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.197+0000 7f53a1be6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f539c06fef0 con 0x7f539c09e760 2023-10-03T10:17:16.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.197+0000 7f53a1be6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f539c06ef30 con 0x7f539c1338a0 2023-10-03T10:17:16.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.197+0000 7f53a1be6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f539c1334b0 con 0x7f539c09de50 2023-10-03T10:17:16.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.197+0000 7f539bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f539c09e760 0x7f539c13a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:16.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.197+0000 7f539b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f539c1338a0 0x7f539c133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:16.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.197+0000 7f539bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f539c09e760 0x7f539c13a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41406/0 (socket says 172.21.15.3:41406) 2023-10-03T10:17:16.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.197+0000 7f539affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f539c09de50 0x7f539c09e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:16.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.197+0000 7f539bfff700 1 -- 172.21.15.3:0/2080611997 learned_addr learned my addr 172.21.15.3:0/2080611997 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:16.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.197+0000 7f539bfff700 1 -- 172.21.15.3:0/2080611997 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f539c09de50 msgr2=0x7f539c09e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:16.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.197+0000 7f539bfff700 1 --2- 172.21.15.3:0/2080611997 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f539c09de50 0x7f539c09e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.197+0000 7f539bfff700 1 -- 172.21.15.3:0/2080611997 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f539c1338a0 msgr2=0x7f539c133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:16.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.197+0000 7f539bfff700 1 --2- 172.21.15.3:0/2080611997 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f539c1338a0 0x7f539c133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.197+0000 7f539bfff700 1 -- 172.21.15.3:0/2080611997 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f539c06fc30 con 0x7f539c09e760 2023-10-03T10:17:16.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.198+0000 7f539bfff700 1 --2- 172.21.15.3:0/2080611997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f539c09e760 0x7f539c13a730 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f5394009e50 tx=0x7f539400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a06d625258f841a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:16.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.198+0000 7f539a7fc700 1 -- 172.21.15.3:0/2080611997 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539400e070 con 0x7f539c09e760 2023-10-03T10:17:16.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.198+0000 7f539a7fc700 1 -- 172.21.15.3:0/2080611997 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539400b910 con 0x7f539c09e760 2023-10-03T10:17:16.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.198+0000 7f539a7fc700 1 -- 172.21.15.3:0/2080611997 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539400bcc0 con 0x7f539c09e760 2023-10-03T10:17:16.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.199+0000 7f53a1be6700 1 -- 172.21.15.3:0/2080611997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f539c09e760 msgr2=0x7f539c13a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:16.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.199+0000 7f53a1be6700 1 --2- 172.21.15.3:0/2080611997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f539c09e760 0x7f539c13a730 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f5394009e50 tx=0x7f539400b330 comp rx=0 tx=0).stop 2023-10-03T10:17:16.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.199+0000 7f53a1be6700 1 -- 172.21.15.3:0/2080611997 shutdown_connections 2023-10-03T10:17:16.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.199+0000 7f53a1be6700 1 --2- 172.21.15.3:0/2080611997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f539c09e760 0x7f539c13a730 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.199+0000 7f53a1be6700 1 --2- 172.21.15.3:0/2080611997 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f539c1338a0 0x7f539c133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.199+0000 7f53a1be6700 1 --2- 172.21.15.3:0/2080611997 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f539c09de50 0x7f539c09e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.199+0000 7f53a1be6700 1 -- 172.21.15.3:0/2080611997 >> 172.21.15.3:0/2080611997 conn(0x7f539c0a2aa0 msgr2=0x7f539c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:16.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.200+0000 7f53a1be6700 1 -- 172.21.15.3:0/2080611997 shutdown_connections 2023-10-03T10:17:16.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.200+0000 7f53a1be6700 1 -- 172.21.15.3:0/2080611997 wait complete. 2023-10-03T10:17:16.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.201+0000 7f53a1be6700 1 Processor -- start 2023-10-03T10:17:16.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.201+0000 7f53a1be6700 1 -- start start 2023-10-03T10:17:16.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.201+0000 7f53a1be6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f539c09de50 0x7f539c11b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:16.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.201+0000 7f53a1be6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f539c09e760 0x7f539c11baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:16.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.201+0000 7f53a1be6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f539c1338a0 0x7f539c11c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:16.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.201+0000 7f53a1be6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539c11d7a0 con 0x7f539c09de50 2023-10-03T10:17:16.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.201+0000 7f53a1be6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f539c11e860 con 0x7f539c09e760 2023-10-03T10:17:16.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.201+0000 7f53a1be6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f539c06f640 con 0x7f539c1338a0 2023-10-03T10:17:16.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.202+0000 7f539b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f539c09de50 0x7f539c11b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:16.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.202+0000 7f539affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f539c09e760 0x7f539c11baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:16.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.202+0000 7f539b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f539c09de50 0x7f539c11b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41422/0 (socket says 172.21.15.3:41422) 2023-10-03T10:17:16.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.202+0000 7f539b7fe700 1 -- 172.21.15.3:0/1717951027 learned_addr learned my addr 172.21.15.3:0/1717951027 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:16.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.202+0000 7f539bfff700 1 --2- 172.21.15.3:0/1717951027 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f539c1338a0 0x7f539c11c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:16.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.202+0000 7f539affd700 1 -- 172.21.15.3:0/1717951027 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f539c1338a0 msgr2=0x7f539c11c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:16.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.202+0000 7f539affd700 1 --2- 172.21.15.3:0/1717951027 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f539c1338a0 0x7f539c11c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.202+0000 7f539affd700 1 -- 172.21.15.3:0/1717951027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f539c09de50 msgr2=0x7f539c11b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:16.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.202+0000 7f539affd700 1 --2- 172.21.15.3:0/1717951027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f539c09de50 0x7f539c11b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.202+0000 7f539affd700 1 -- 172.21.15.3:0/1717951027 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5394009b30 con 0x7f539c09e760 2023-10-03T10:17:16.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.202+0000 7f539b7fe700 1 --2- 172.21.15.3:0/1717951027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f539c09de50 0x7f539c11b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:16.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.202+0000 7f539affd700 1 --2- 172.21.15.3:0/1717951027 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f539c09e760 0x7f539c11baf0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f539000e840 tx=0x7f539000eb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:16.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.203+0000 7f5383fff700 1 -- 172.21.15.3:0/1717951027 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5390013680 con 0x7f539c09e760 2023-10-03T10:17:16.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.203+0000 7f5383fff700 1 -- 172.21.15.3:0/1717951027 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53900137e0 con 0x7f539c09e760 2023-10-03T10:17:16.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.203+0000 7f53a1be6700 1 -- 172.21.15.3:0/1717951027 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f539c0a0f90 con 0x7f539c09e760 2023-10-03T10:17:16.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.203+0000 7f5383fff700 1 -- 172.21.15.3:0/1717951027 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5390013b50 con 0x7f539c09e760 2023-10-03T10:17:16.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.203+0000 7f53a1be6700 1 -- 172.21.15.3:0/1717951027 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f539c11e410 con 0x7f539c09e760 2023-10-03T10:17:16.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.204+0000 7f5383fff700 1 -- 172.21.15.3:0/1717951027 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5390013cb0 con 0x7f539c09e760 2023-10-03T10:17:16.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.204+0000 7f5383fff700 1 -- 172.21.15.3:0/1717951027 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f539003f4e0 con 0x7f539c09e760 2023-10-03T10:17:16.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.204+0000 7f53a1be6700 1 -- 172.21.15.3:0/1717951027 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f539c074ea0 con 0x7f539c09e760 2023-10-03T10:17:16.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.209+0000 7f5383fff700 1 -- 172.21.15.3:0/1717951027 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f539001be10 con 0x7f539c09e760 2023-10-03T10:17:16.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.279+0000 7f236f4b7700 1 Processor -- start 2023-10-03T10:17:16.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.279+0000 7f236f4b7700 1 -- start start 2023-10-03T10:17:16.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.280+0000 7f236f4b7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f236809c650 0x7f236809ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:16.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.280+0000 7f236f4b7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f23680a0090 0x7f236809cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:16.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.280+0000 7f236f4b7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2368131970 0x7f2368131e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:16.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.280+0000 7f236f4b7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f236809ba70 con 0x7f23680a0090 2023-10-03T10:17:16.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.280+0000 7f236f4b7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236809d4a0 con 0x7f236809c650 2023-10-03T10:17:16.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.280+0000 7f236f4b7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f236809e8f0 con 0x7f2368131970 2023-10-03T10:17:16.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.280+0000 7f236d253700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f236809c650 0x7f236809ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:16.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.280+0000 7f236ca52700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f23680a0090 0x7f236809cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:16.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.280+0000 7f236da54700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2368131970 0x7f2368131e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:16.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.280+0000 7f236d253700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f236809c650 0x7f236809ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50946/0 (socket says 172.21.15.3:50946) 2023-10-03T10:17:16.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.280+0000 7f236d253700 1 -- 172.21.15.3:0/1523555489 learned_addr learned my addr 172.21.15.3:0/1523555489 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:16.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.281+0000 7f236ca52700 1 -- 172.21.15.3:0/1523555489 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2368131970 msgr2=0x7f2368131e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:16.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.281+0000 7f236ca52700 1 --2- 172.21.15.3:0/1523555489 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2368131970 0x7f2368131e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.281+0000 7f236ca52700 1 -- 172.21.15.3:0/1523555489 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f236809c650 msgr2=0x7f236809ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:16.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.281+0000 7f236ca52700 1 --2- 172.21.15.3:0/1523555489 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f236809c650 0x7f236809ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.281+0000 7f236ca52700 1 -- 172.21.15.3:0/1523555489 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23680a0760 con 0x7f23680a0090 2023-10-03T10:17:16.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.281+0000 7f236d253700 1 --2- 172.21.15.3:0/1523555489 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f236809c650 0x7f236809ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:16.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.281+0000 7f236da54700 1 --2- 172.21.15.3:0/1523555489 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2368131970 0x7f2368131e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:16.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.281+0000 7f236ca52700 1 --2- 172.21.15.3:0/1523555489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23680a0090 0x7f236809cf60 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f2358009840 tx=0x7f2358009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4bfe8905390583 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:16.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.281+0000 7f235ffff700 1 -- 172.21.15.3:0/1523555489 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2358011830 con 0x7f23680a0090 2023-10-03T10:17:16.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.281+0000 7f235ffff700 1 -- 172.21.15.3:0/1523555489 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2358011990 con 0x7f23680a0090 2023-10-03T10:17:16.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.281+0000 7f235ffff700 1 -- 172.21.15.3:0/1523555489 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2358011d20 con 0x7f23680a0090 2023-10-03T10:17:16.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.282+0000 7f236f4b7700 1 -- 172.21.15.3:0/1523555489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23680a0090 msgr2=0x7f236809cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:16.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.282+0000 7f236f4b7700 1 --2- 172.21.15.3:0/1523555489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23680a0090 0x7f236809cf60 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f2358009840 tx=0x7f2358009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:16.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.282+0000 7f236f4b7700 1 -- 172.21.15.3:0/1523555489 shutdown_connections 2023-10-03T10:17:16.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.282+0000 7f236f4b7700 1 --2- 172.21.15.3:0/1523555489 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2368131970 0x7f2368131e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.282+0000 7f236f4b7700 1 --2- 172.21.15.3:0/1523555489 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f236809c650 0x7f236809ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.282+0000 7f236f4b7700 1 --2- 172.21.15.3:0/1523555489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23680a0090 0x7f236809cf60 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.282+0000 7f236f4b7700 1 -- 172.21.15.3:0/1523555489 >> 172.21.15.3:0/1523555489 conn(0x7f23680a3f70 msgr2=0x7f23680a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:16.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.282+0000 7f236f4b7700 1 -- 172.21.15.3:0/1523555489 shutdown_connections 2023-10-03T10:17:16.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.282+0000 7f236f4b7700 1 -- 172.21.15.3:0/1523555489 wait complete. 2023-10-03T10:17:16.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.283+0000 7f236f4b7700 1 Processor -- start 2023-10-03T10:17:16.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.283+0000 7f236f4b7700 1 -- start start 2023-10-03T10:17:16.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.284+0000 7f236f4b7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f236809c650 0x7f23680711b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:16.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.284+0000 7f236f4b7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f23680a0090 0x7f23680716f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:16.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.284+0000 7f236f4b7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2368131970 0x7f2368071c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:16.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.284+0000 7f236f4b7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236811ca60 con 0x7f2368131970 2023-10-03T10:17:16.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.284+0000 7f236f4b7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236811c1f0 con 0x7f236809c650 2023-10-03T10:17:16.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.284+0000 7f236f4b7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f236811bda0 con 0x7f23680a0090 2023-10-03T10:17:16.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.284+0000 7f236ca52700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f23680a0090 0x7f23680716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:16.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.284+0000 7f236da54700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2368131970 0x7f2368071c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:16.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.284+0000 7f236ca52700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f23680a0090 0x7f23680716f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53390/0 (socket says 172.21.15.3:53390) 2023-10-03T10:17:16.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.284+0000 7f236ca52700 1 -- 172.21.15.3:0/3979592351 learned_addr learned my addr 172.21.15.3:0/3979592351 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:16.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.284+0000 7f236d253700 1 --2- 172.21.15.3:0/3979592351 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f236809c650 0x7f23680711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:16.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.284+0000 7f236ca52700 1 -- 172.21.15.3:0/3979592351 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f236809c650 msgr2=0x7f23680711b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:16.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.284+0000 7f236ca52700 1 --2- 172.21.15.3:0/3979592351 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f236809c650 0x7f23680711b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.284+0000 7f236ca52700 1 -- 172.21.15.3:0/3979592351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2368131970 msgr2=0x7f2368071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:16.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.284+0000 7f236ca52700 1 --2- 172.21.15.3:0/3979592351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2368131970 0x7f2368071c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.284+0000 7f236ca52700 1 -- 172.21.15.3:0/3979592351 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2364009550 con 0x7f23680a0090 2023-10-03T10:17:16.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.285+0000 7f236ca52700 1 --2- 172.21.15.3:0/3979592351 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f23680a0090 0x7f23680716f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f23580090b0 tx=0x7f2358004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:16.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.285+0000 7f235dffb700 1 -- 172.21.15.3:0/3979592351 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2358004450 con 0x7f23680a0090 2023-10-03T10:17:16.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.285+0000 7f235dffb700 1 -- 172.21.15.3:0/3979592351 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235801f070 con 0x7f23680a0090 2023-10-03T10:17:16.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.285+0000 7f236f4b7700 1 -- 172.21.15.3:0/3979592351 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2360009a70 con 0x7f23680a0090 2023-10-03T10:17:16.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.285+0000 7f235dffb700 1 -- 172.21.15.3:0/3979592351 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23580038e0 con 0x7f23680a0090 2023-10-03T10:17:16.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.285+0000 7f236f4b7700 1 -- 172.21.15.3:0/3979592351 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23580094a0 con 0x7f23680a0090 2023-10-03T10:17:16.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.286+0000 7f236f4b7700 1 -- 172.21.15.3:0/3979592351 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23680747f0 con 0x7f23680a0090 2023-10-03T10:17:16.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.286+0000 7f235dffb700 1 -- 172.21.15.3:0/3979592351 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2358003a40 con 0x7f23680a0090 2023-10-03T10:17:16.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.287+0000 7f235dffb700 1 -- 172.21.15.3:0/3979592351 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2358021070 con 0x7f23680a0090 2023-10-03T10:17:16.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.292+0000 7f235dffb700 1 -- 172.21.15.3:0/3979592351 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f23580253e0 con 0x7f23680a0090 2023-10-03T10:17:16.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.323+0000 7fb272748700 1 Processor -- start 2023-10-03T10:17:16.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.323+0000 7fb272748700 1 -- start start 2023-10-03T10:17:16.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.323+0000 7fb272748700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb26c09e1d0 0x7fb26c135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:16.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.323+0000 7fb272748700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb26c135e50 0x7fb26c138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:16.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.323+0000 7fb272748700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb26c09f130 con 0x7fb26c09d6f0 2023-10-03T10:17:16.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.323+0000 7fb272748700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c09f520 con 0x7fb26c09e1d0 2023-10-03T10:17:16.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.323+0000 7fb272748700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c09e930 con 0x7fb26c135e50 2023-10-03T10:17:16.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.323+0000 7fb26b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb26c09e1d0 0x7fb26c135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:16.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.323+0000 7fb26b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb26c09e1d0 0x7fb26c135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50974/0 (socket says 172.21.15.3:50974) 2023-10-03T10:17:16.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.323+0000 7fb26b7fe700 1 -- 172.21.15.3:0/1292602415 learned_addr learned my addr 172.21.15.3:0/1292602415 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:16.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.324+0000 7fb270ce5700 1 --2- 172.21.15.3:0/1292602415 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb26c135e50 0x7fb26c138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:16.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.324+0000 7fb26b7fe700 1 -- 172.21.15.3:0/1292602415 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb26c135e50 msgr2=0x7fb26c138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:16.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.324+0000 7fb26b7fe700 1 --2- 172.21.15.3:0/1292602415 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb26c135e50 0x7fb26c138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.324+0000 7fb26b7fe700 1 -- 172.21.15.3:0/1292602415 >> v1:172.21.15.3:6789/0 conn(0x7fb26c09d6f0 legacy=0x7fb26c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:16.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.324+0000 7fb26b7fe700 1 -- 172.21.15.3:0/1292602415 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26c09b2d0 con 0x7fb26c09e1d0 2023-10-03T10:17:16.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.324+0000 7fb26affd700 1 -- 172.21.15.3:0/1292602415 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 19062809 0 0) 0x7fb26c09f130 con 0x7fb26c09d6f0 2023-10-03T10:17:16.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.324+0000 7fb270ce5700 1 --2- 172.21.15.3:0/1292602415 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb26c135e50 0x7fb26c138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:17:16.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.324+0000 7fb26b7fe700 1 --2- 172.21.15.3:0/1292602415 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb26c09e1d0 0x7fb26c135910 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb26000bbc0 tx=0x7fb26000bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c91373b993dabedd server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:16.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.324+0000 7fb26affd700 1 -- 172.21.15.3:0/1292602415 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26000dd90 con 0x7fb26c09e1d0 2023-10-03T10:17:16.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.324+0000 7fb26affd700 1 -- 172.21.15.3:0/1292602415 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb260010470 con 0x7fb26c09e1d0 2023-10-03T10:17:16.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.324+0000 7fb26affd700 1 -- 172.21.15.3:0/1292602415 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb260010800 con 0x7fb26c09e1d0 2023-10-03T10:17:16.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.325+0000 7fb272748700 1 -- 172.21.15.3:0/1292602415 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb26c09e1d0 msgr2=0x7fb26c135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:16.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.325+0000 7fb272748700 1 --2- 172.21.15.3:0/1292602415 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb26c09e1d0 0x7fb26c135910 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb26000bbc0 tx=0x7fb26000bed0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.325+0000 7fb272748700 1 -- 172.21.15.3:0/1292602415 shutdown_connections 2023-10-03T10:17:16.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.325+0000 7fb272748700 1 --2- 172.21.15.3:0/1292602415 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb26c135e50 0x7fb26c138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.325+0000 7fb272748700 1 --2- 172.21.15.3:0/1292602415 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb26c09e1d0 0x7fb26c135910 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.325+0000 7fb272748700 1 -- 172.21.15.3:0/1292602415 >> 172.21.15.3:0/1292602415 conn(0x7fb26c066f50 msgr2=0x7fb26c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:16.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.325+0000 7fb272748700 1 -- 172.21.15.3:0/1292602415 shutdown_connections 2023-10-03T10:17:16.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.325+0000 7fb272748700 1 -- 172.21.15.3:0/1292602415 wait complete. 2023-10-03T10:17:16.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.326+0000 7fb272748700 1 Processor -- start 2023-10-03T10:17:16.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.326+0000 7fb272748700 1 -- start start 2023-10-03T10:17:16.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.327+0000 7fb272748700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb26c09d6f0 0x7fb26c119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:16.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.327+0000 7fb272748700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb26c09e1d0 0x7fb26c1199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:16.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.327+0000 7fb272748700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb26c135e50 0x7fb26c119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:16.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.327+0000 7fb272748700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c130090 con 0x7fb26c09e1d0 2023-10-03T10:17:16.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.327+0000 7fb272748700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c11c850 con 0x7fb26c135e50 2023-10-03T10:17:16.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.327+0000 7fb272748700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c09c700 con 0x7fb26c09d6f0 2023-10-03T10:17:16.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.327+0000 7fb26bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb26c09d6f0 0x7fb26c119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:16.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.327+0000 7fb26bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb26c09d6f0 0x7fb26c119480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53414/0 (socket says 172.21.15.3:53414) 2023-10-03T10:17:16.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.327+0000 7fb26bfff700 1 -- 172.21.15.3:0/4272068864 learned_addr learned my addr 172.21.15.3:0/4272068864 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:16.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.327+0000 7fb270ce5700 1 --2- 172.21.15.3:0/4272068864 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb26c135e50 0x7fb26c119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:16.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.327+0000 7fb26b7fe700 1 --2- 172.21.15.3:0/4272068864 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb26c09e1d0 0x7fb26c1199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:16.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.327+0000 7fb26bfff700 1 -- 172.21.15.3:0/4272068864 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb26c135e50 msgr2=0x7fb26c119f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:16.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.327+0000 7fb26bfff700 1 --2- 172.21.15.3:0/4272068864 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb26c135e50 0x7fb26c119f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.327+0000 7fb26bfff700 1 -- 172.21.15.3:0/4272068864 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb26c09e1d0 msgr2=0x7fb26c1199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:16.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.327+0000 7fb26bfff700 1 --2- 172.21.15.3:0/4272068864 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb26c09e1d0 0x7fb26c1199c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.327+0000 7fb26bfff700 1 -- 172.21.15.3:0/4272068864 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26000b7f0 con 0x7fb26c09d6f0 2023-10-03T10:17:16.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.327+0000 7fb26b7fe700 1 --2- 172.21.15.3:0/4272068864 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb26c09e1d0 0x7fb26c1199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:17:16.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.328+0000 7fb26bfff700 1 --2- 172.21.15.3:0/4272068864 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb26c09d6f0 0x7fb26c119480 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb25c00c450 tx=0x7fb25c00c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:16.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.328+0000 7fb268ff9700 1 -- 172.21.15.3:0/4272068864 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c0054e0 con 0x7fb26c09d6f0 2023-10-03T10:17:16.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.328+0000 7fb268ff9700 1 -- 172.21.15.3:0/4272068864 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25c003680 con 0x7fb26c09d6f0 2023-10-03T10:17:16.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.328+0000 7fb272748700 1 -- 172.21.15.3:0/4272068864 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb26c11af80 con 0x7fb26c09d6f0 2023-10-03T10:17:16.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.328+0000 7fb268ff9700 1 -- 172.21.15.3:0/4272068864 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c003a30 con 0x7fb26c09d6f0 2023-10-03T10:17:16.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.328+0000 7fb272748700 1 -- 172.21.15.3:0/4272068864 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26c11cdf0 con 0x7fb26c09d6f0 2023-10-03T10:17:16.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.329+0000 7fb268ff9700 1 -- 172.21.15.3:0/4272068864 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb25c003b90 con 0x7fb26c09d6f0 2023-10-03T10:17:16.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.329+0000 7fb268ff9700 1 -- 172.21.15.3:0/4272068864 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb25c03ec20 con 0x7fb26c09d6f0 2023-10-03T10:17:16.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.329+0000 7fb272748700 1 -- 172.21.15.3:0/4272068864 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb26c11cc10 con 0x7fb26c09d6f0 2023-10-03T10:17:16.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.334+0000 7fb268ff9700 1 -- 172.21.15.3:0/4272068864 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb25c017020 con 0x7fb26c09d6f0 2023-10-03T10:17:16.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.420+0000 7f53a1be6700 1 -- 172.21.15.3:0/1717951027 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f539c06b9e0 con 0x7f539c09e760 2023-10-03T10:17:16.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.420+0000 7f5383fff700 1 -- 172.21.15.3:0/1717951027 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5390017020 con 0x7f539c09e760 2023-10-03T10:17:16.420 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:16.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.423+0000 7f53a1be6700 1 -- 172.21.15.3:0/1717951027 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f539c09e760 msgr2=0x7f539c11baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:16.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.423+0000 7f53a1be6700 1 --2- 172.21.15.3:0/1717951027 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f539c09e760 0x7f539c11baf0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f539000e840 tx=0x7f539000eb50 comp rx=0 tx=0).stop 2023-10-03T10:17:16.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.423+0000 7f53a1be6700 1 -- 172.21.15.3:0/1717951027 shutdown_connections 2023-10-03T10:17:16.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.423+0000 7f53a1be6700 1 --2- 172.21.15.3:0/1717951027 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f539c1338a0 0x7f539c11c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.423+0000 7f53a1be6700 1 --2- 172.21.15.3:0/1717951027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f539c09de50 0x7f539c11b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.423+0000 7f53a1be6700 1 --2- 172.21.15.3:0/1717951027 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f539c09e760 0x7f539c11baf0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.423+0000 7f53a1be6700 1 -- 172.21.15.3:0/1717951027 >> 172.21.15.3:0/1717951027 conn(0x7f539c0a2aa0 msgr2=0x7f539c130630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:16.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.423+0000 7f53a1be6700 1 -- 172.21.15.3:0/1717951027 shutdown_connections 2023-10-03T10:17:16.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.423+0000 7f53a1be6700 1 -- 172.21.15.3:0/1717951027 wait complete. 2023-10-03T10:17:16.437 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:16.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.501+0000 7f236f4b7700 1 -- 172.21.15.3:0/3979592351 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f236811ab90 con 0x7f23680a0090 2023-10-03T10:17:16.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.502+0000 7f235dffb700 1 -- 172.21.15.3:0/3979592351 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2358010b70 con 0x7f23680a0090 2023-10-03T10:17:16.502 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:16.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.504+0000 7f236f4b7700 1 -- 172.21.15.3:0/3979592351 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f23680a0090 msgr2=0x7f23680716f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:16.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.504+0000 7f236f4b7700 1 --2- 172.21.15.3:0/3979592351 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f23680a0090 0x7f23680716f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f23580090b0 tx=0x7f2358004750 comp rx=0 tx=0).stop 2023-10-03T10:17:16.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.504+0000 7f236f4b7700 1 -- 172.21.15.3:0/3979592351 shutdown_connections 2023-10-03T10:17:16.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.504+0000 7f236f4b7700 1 --2- 172.21.15.3:0/3979592351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2368131970 0x7f2368071c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.504+0000 7f236f4b7700 1 --2- 172.21.15.3:0/3979592351 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f236809c650 0x7f23680711b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.504+0000 7f236f4b7700 1 --2- 172.21.15.3:0/3979592351 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f23680a0090 0x7f23680716f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.504+0000 7f236f4b7700 1 -- 172.21.15.3:0/3979592351 >> 172.21.15.3:0/3979592351 conn(0x7f23680a3f70 msgr2=0x7f236812dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:16.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.504+0000 7f236f4b7700 1 -- 172.21.15.3:0/3979592351 shutdown_connections 2023-10-03T10:17:16.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.504+0000 7f236f4b7700 1 -- 172.21.15.3:0/3979592351 wait complete. 2023-10-03T10:17:16.518 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:16.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.543+0000 7fb272748700 1 -- 172.21.15.3:0/4272068864 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb26c074ea0 con 0x7fb26c09d6f0 2023-10-03T10:17:16.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.543+0000 7fb268ff9700 1 -- 172.21.15.3:0/4272068864 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb25c003e40 con 0x7fb26c09d6f0 2023-10-03T10:17:16.543 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:16.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.545+0000 7fb272748700 1 -- 172.21.15.3:0/4272068864 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb26c09d6f0 msgr2=0x7fb26c119480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:16.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.545+0000 7fb272748700 1 --2- 172.21.15.3:0/4272068864 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb26c09d6f0 0x7fb26c119480 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb25c00c450 tx=0x7fb25c00c760 comp rx=0 tx=0).stop 2023-10-03T10:17:16.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.545+0000 7fb272748700 1 -- 172.21.15.3:0/4272068864 shutdown_connections 2023-10-03T10:17:16.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.545+0000 7fb272748700 1 --2- 172.21.15.3:0/4272068864 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb26c135e50 0x7fb26c119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.545+0000 7fb272748700 1 --2- 172.21.15.3:0/4272068864 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb26c09d6f0 0x7fb26c119480 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.545+0000 7fb272748700 1 --2- 172.21.15.3:0/4272068864 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb26c09e1d0 0x7fb26c1199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:16.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.545+0000 7fb272748700 1 -- 172.21.15.3:0/4272068864 >> 172.21.15.3:0/4272068864 conn(0x7fb26c066f50 msgr2=0x7fb26c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:16.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.545+0000 7fb272748700 1 -- 172.21.15.3:0/4272068864 shutdown_connections 2023-10-03T10:17:16.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:16.545+0000 7fb272748700 1 -- 172.21.15.3:0/4272068864 wait complete. 2023-10-03T10:17:16.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:17.081 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:17.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.227+0000 7f173978a700 1 Processor -- start 2023-10-03T10:17:17.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.227+0000 7f173978a700 1 -- start start 2023-10-03T10:17:17.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.228+0000 7f173978a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1734130750 0x7f1734130b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:17.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.228+0000 7f173978a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f173409d6f0 0x7f173409db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:17.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.228+0000 7f173978a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f173409c190 con 0x7f173409e080 2023-10-03T10:17:17.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.228+0000 7f173978a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f173409e520 con 0x7f173409d6f0 2023-10-03T10:17:17.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.228+0000 7f173978a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f173409b090 con 0x7f1734130750 2023-10-03T10:17:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.228+0000 7f1732ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1734130750 0x7f1734130b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.228+0000 7f17327fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f173409d6f0 0x7f173409db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.228+0000 7f1732ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1734130750 0x7f1734130b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53422/0 (socket says 172.21.15.3:53422) 2023-10-03T10:17:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.228+0000 7f17327fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f173409d6f0 0x7f173409db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41438/0 (socket says 172.21.15.3:41438) 2023-10-03T10:17:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.228+0000 7f1732ffd700 1 -- 172.21.15.3:0/1149828951 learned_addr learned my addr 172.21.15.3:0/1149828951 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.228+0000 7f17337fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f173409e080 0x7f17341374b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55270/0 (socket says 172.21.15.3:55270) 2023-10-03T10:17:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.229+0000 7f17327fc700 1 -- 172.21.15.3:0/1149828951 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1734130750 msgr2=0x7f1734130b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.229+0000 7f17327fc700 1 --2- 172.21.15.3:0/1149828951 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1734130750 0x7f1734130b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.229+0000 7f17327fc700 1 -- 172.21.15.3:0/1149828951 >> v1:172.21.15.3:6789/0 conn(0x7f173409e080 legacy=0x7f17341374b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.229+0000 7f17327fc700 1 -- 172.21.15.3:0/1149828951 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f173409a6c0 con 0x7f173409d6f0 2023-10-03T10:17:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.229+0000 7f1731ffb700 1 -- 172.21.15.3:0/1149828951 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1165209015 0 0) 0x7f173409c190 con 0x7f173409e080 2023-10-03T10:17:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.229+0000 7f17327fc700 1 --2- 172.21.15.3:0/1149828951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f173409d6f0 0x7f173409db40 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f1728009840 tx=0x7f1728009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc64ca94c8d040a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:17.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.229+0000 7f1731ffb700 1 -- 172.21.15.3:0/1149828951 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1728011830 con 0x7f173409d6f0 2023-10-03T10:17:17.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.229+0000 7f1731ffb700 1 -- 172.21.15.3:0/1149828951 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1728011990 con 0x7f173409d6f0 2023-10-03T10:17:17.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.229+0000 7f1731ffb700 1 -- 172.21.15.3:0/1149828951 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1728011d20 con 0x7f173409d6f0 2023-10-03T10:17:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.230+0000 7f173978a700 1 -- 172.21.15.3:0/1149828951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f173409d6f0 msgr2=0x7f173409db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.230+0000 7f173978a700 1 --2- 172.21.15.3:0/1149828951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f173409d6f0 0x7f173409db40 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f1728009840 tx=0x7f1728009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.230+0000 7f173978a700 1 -- 172.21.15.3:0/1149828951 shutdown_connections 2023-10-03T10:17:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.230+0000 7f173978a700 1 --2- 172.21.15.3:0/1149828951 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1734130750 0x7f1734130b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.230+0000 7f173978a700 1 --2- 172.21.15.3:0/1149828951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f173409d6f0 0x7f173409db40 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.230+0000 7f173978a700 1 -- 172.21.15.3:0/1149828951 >> 172.21.15.3:0/1149828951 conn(0x7f173409f470 msgr2=0x7f17340a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:17.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.230+0000 7f173978a700 1 -- 172.21.15.3:0/1149828951 shutdown_connections 2023-10-03T10:17:17.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.230+0000 7f173978a700 1 -- 172.21.15.3:0/1149828951 wait complete. 2023-10-03T10:17:17.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.231+0000 7f173978a700 1 Processor -- start 2023-10-03T10:17:17.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.231+0000 7f173978a700 1 -- start start 2023-10-03T10:17:17.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.231+0000 7f173978a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f173409d6f0 0x7f17341b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:17.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.231+0000 7f173978a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f173409e080 0x7f17341b6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:17.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.231+0000 7f173978a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1734130750 0x7f17341b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:17.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.231+0000 7f173978a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173409b320 con 0x7f173409d6f0 2023-10-03T10:17:17.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.231+0000 7f173978a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17341b9bc0 con 0x7f173409e080 2023-10-03T10:17:17.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.231+0000 7f173978a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f17341ba7a0 con 0x7f1734130750 2023-10-03T10:17:17.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.231+0000 7f1732ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f173409d6f0 0x7f17341b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:17.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.231+0000 7f17337fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1734130750 0x7f17341b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:17.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.231+0000 7f1732ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f173409d6f0 0x7f17341b61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41454/0 (socket says 172.21.15.3:41454) 2023-10-03T10:17:17.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.231+0000 7f1732ffd700 1 -- 172.21.15.3:0/2710255718 learned_addr learned my addr 172.21.15.3:0/2710255718 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:17.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.231+0000 7f17327fc700 1 --2- 172.21.15.3:0/2710255718 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f173409e080 0x7f17341b6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:17.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.232+0000 7f1732ffd700 1 -- 172.21.15.3:0/2710255718 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1734130750 msgr2=0x7f17341b6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:17.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.232+0000 7f1732ffd700 1 --2- 172.21.15.3:0/2710255718 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1734130750 0x7f17341b6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.232+0000 7f1732ffd700 1 -- 172.21.15.3:0/2710255718 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f173409e080 msgr2=0x7f17341b6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:17.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.232+0000 7f1732ffd700 1 --2- 172.21.15.3:0/2710255718 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f173409e080 0x7f17341b6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.232+0000 7f1732ffd700 1 -- 172.21.15.3:0/2710255718 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17280094a0 con 0x7f173409d6f0 2023-10-03T10:17:17.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.232+0000 7f1732ffd700 1 --2- 172.21.15.3:0/2710255718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f173409d6f0 0x7f17341b61c0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f172400e940 tx=0x7f172400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:17.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.232+0000 7f171b7fe700 1 -- 172.21.15.3:0/2710255718 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17240096a0 con 0x7f173409d6f0 2023-10-03T10:17:17.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.232+0000 7f171b7fe700 1 -- 172.21.15.3:0/2710255718 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1724013680 con 0x7f173409d6f0 2023-10-03T10:17:17.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.232+0000 7f173978a700 1 -- 172.21.15.3:0/2710255718 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17341310e0 con 0x7f173409d6f0 2023-10-03T10:17:17.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.232+0000 7f171b7fe700 1 -- 172.21.15.3:0/2710255718 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724013a30 con 0x7f173409d6f0 2023-10-03T10:17:17.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.232+0000 7f173978a700 1 -- 172.21.15.3:0/2710255718 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17341b98e0 con 0x7f173409d6f0 2023-10-03T10:17:17.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.233+0000 7f171b7fe700 1 -- 172.21.15.3:0/2710255718 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1724013b90 con 0x7f173409d6f0 2023-10-03T10:17:17.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.233+0000 7f171b7fe700 1 -- 172.21.15.3:0/2710255718 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f172403f7e0 con 0x7f173409d6f0 2023-10-03T10:17:17.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.233+0000 7f173978a700 1 -- 172.21.15.3:0/2710255718 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17341b7600 con 0x7f173409d6f0 2023-10-03T10:17:17.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.238+0000 7f171b7fe700 1 -- 172.21.15.3:0/2710255718 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1724017080 con 0x7f173409d6f0 2023-10-03T10:17:17.437 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:17.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.443+0000 7f173978a700 1 -- 172.21.15.3:0/2710255718 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f17340747f0 con 0x7f173409d6f0 2023-10-03T10:17:17.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.443+0000 7f171b7fe700 1 -- 172.21.15.3:0/2710255718 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f172401ba00 con 0x7f173409d6f0 2023-10-03T10:17:17.443 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:17.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.445+0000 7f173978a700 1 -- 172.21.15.3:0/2710255718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f173409d6f0 msgr2=0x7f17341b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:17.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.445+0000 7f173978a700 1 --2- 172.21.15.3:0/2710255718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f173409d6f0 0x7f17341b61c0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f172400e940 tx=0x7f172400ec50 comp rx=0 tx=0).stop 2023-10-03T10:17:17.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.445+0000 7f173978a700 1 -- 172.21.15.3:0/2710255718 shutdown_connections 2023-10-03T10:17:17.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.445+0000 7f173978a700 1 --2- 172.21.15.3:0/2710255718 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1734130750 0x7f17341b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.445+0000 7f173978a700 1 --2- 172.21.15.3:0/2710255718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f173409d6f0 0x7f17341b61c0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.445+0000 7f173978a700 1 --2- 172.21.15.3:0/2710255718 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f173409e080 0x7f17341b6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.445+0000 7f173978a700 1 -- 172.21.15.3:0/2710255718 >> 172.21.15.3:0/2710255718 conn(0x7f173409f470 msgr2=0x7f17340992c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:17.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.446+0000 7f173978a700 1 -- 172.21.15.3:0/2710255718 shutdown_connections 2023-10-03T10:17:17.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.446+0000 7f173978a700 1 -- 172.21.15.3:0/2710255718 wait complete. 2023-10-03T10:17:17.461 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:17.518 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:17.560 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:17.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.578+0000 7f635d455700 1 Processor -- start 2023-10-03T10:17:17.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.578+0000 7f635d455700 1 -- start start 2023-10-03T10:17:17.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.579+0000 7f635d455700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63580954c0 0x7f6358095890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:17.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.579+0000 7f635d455700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6358099680 0x7f6358099ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:17.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.579+0000 7f635d455700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f635809c280 con 0x7f6358099ff0 2023-10-03T10:17:17.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.579+0000 7f635d455700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635809aaf0 con 0x7f6358099680 2023-10-03T10:17:17.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.579+0000 7f635d455700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f635809b960 con 0x7f63580954c0 2023-10-03T10:17:17.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.579+0000 7f6356ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63580954c0 0x7f6358095890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:17.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.579+0000 7f6356ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63580954c0 0x7f6358095890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53450/0 (socket says 172.21.15.3:53450) 2023-10-03T10:17:17.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.579+0000 7f6356ffd700 1 -- 172.21.15.3:0/2251148103 learned_addr learned my addr 172.21.15.3:0/2251148103 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:17.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.579+0000 7f63567fc700 1 --2- 172.21.15.3:0/2251148103 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6358099680 0x7f6358099ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:17.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.579+0000 7f6356ffd700 1 -- 172.21.15.3:0/2251148103 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6358099680 msgr2=0x7f6358099ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:17.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.579+0000 7f6356ffd700 1 --2- 172.21.15.3:0/2251148103 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6358099680 0x7f6358099ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.579+0000 7f6356ffd700 1 -- 172.21.15.3:0/2251148103 >> v1:172.21.15.3:6789/0 conn(0x7f6358099ff0 legacy=0x7f6358137590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:17.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.579+0000 7f6356ffd700 1 -- 172.21.15.3:0/2251148103 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f635809a7b0 con 0x7f63580954c0 2023-10-03T10:17:17.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.579+0000 7f6356ffd700 1 --2- 172.21.15.3:0/2251148103 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63580954c0 0x7f6358095890 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6348009920 tx=0x7f6348009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=271ee0161b98b3c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:17.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.580+0000 7f6355ffb700 1 -- 172.21.15.3:0/2251148103 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634800ba60 con 0x7f63580954c0 2023-10-03T10:17:17.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.580+0000 7f6355ffb700 1 -- 172.21.15.3:0/2251148103 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634800bbc0 con 0x7f63580954c0 2023-10-03T10:17:17.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.580+0000 7f6355ffb700 1 -- 172.21.15.3:0/2251148103 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6348009e70 con 0x7f63580954c0 2023-10-03T10:17:17.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.580+0000 7f635d455700 1 -- 172.21.15.3:0/2251148103 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63580954c0 msgr2=0x7f6358095890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:17.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.580+0000 7f635d455700 1 --2- 172.21.15.3:0/2251148103 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63580954c0 0x7f6358095890 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6348009920 tx=0x7f6348009c30 comp rx=0 tx=0).stop 2023-10-03T10:17:17.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.580+0000 7f635d455700 1 -- 172.21.15.3:0/2251148103 shutdown_connections 2023-10-03T10:17:17.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.580+0000 7f635d455700 1 --2- 172.21.15.3:0/2251148103 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f63580954c0 0x7f6358095890 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.581+0000 7f635d455700 1 --2- 172.21.15.3:0/2251148103 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6358099680 0x7f6358099ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.581+0000 7f635d455700 1 -- 172.21.15.3:0/2251148103 >> 172.21.15.3:0/2251148103 conn(0x7f63580a2360 msgr2=0x7f63580a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:17.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.581+0000 7f635d455700 1 -- 172.21.15.3:0/2251148103 shutdown_connections 2023-10-03T10:17:17.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.581+0000 7f635d455700 1 -- 172.21.15.3:0/2251148103 wait complete. 2023-10-03T10:17:17.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.582+0000 7f635d455700 1 Processor -- start 2023-10-03T10:17:17.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.582+0000 7f635d455700 1 -- start start 2023-10-03T10:17:17.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.582+0000 7f635d455700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63580954c0 0x7f63580935d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:17.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.582+0000 7f635d455700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6358099680 0x7f6358093b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:17.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.582+0000 7f635d455700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6358099ff0 0x7f6358094050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:17.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.582+0000 7f635d455700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635808f010 con 0x7f63580954c0 2023-10-03T10:17:17.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.582+0000 7f635d455700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6358092450 con 0x7f6358099ff0 2023-10-03T10:17:17.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.582+0000 7f635d455700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6358090a70 con 0x7f6358099680 2023-10-03T10:17:17.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.582+0000 7f6356ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63580954c0 0x7f63580935d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:17.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.582+0000 7f63567fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6358099680 0x7f6358093b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:17.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.582+0000 7f6356ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63580954c0 0x7f63580935d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41468/0 (socket says 172.21.15.3:41468) 2023-10-03T10:17:17.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.582+0000 7f63567fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6358099680 0x7f6358093b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53456/0 (socket says 172.21.15.3:53456) 2023-10-03T10:17:17.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.582+0000 7f6356ffd700 1 -- 172.21.15.3:0/305846582 learned_addr learned my addr 172.21.15.3:0/305846582 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:17.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.583+0000 7f63577fe700 1 --2- 172.21.15.3:0/305846582 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6358099ff0 0x7f6358094050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:17.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.583+0000 7f6356ffd700 1 -- 172.21.15.3:0/305846582 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6358099680 msgr2=0x7f6358093b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:17.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.583+0000 7f6356ffd700 1 --2- 172.21.15.3:0/305846582 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6358099680 0x7f6358093b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.583+0000 7f6356ffd700 1 -- 172.21.15.3:0/305846582 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6358099ff0 msgr2=0x7f6358094050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:17.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.583+0000 7f6356ffd700 1 --2- 172.21.15.3:0/305846582 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6358099ff0 0x7f6358094050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.583+0000 7f6356ffd700 1 -- 172.21.15.3:0/305846582 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6348009550 con 0x7f63580954c0 2023-10-03T10:17:17.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.583+0000 7f63577fe700 1 --2- 172.21.15.3:0/305846582 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6358099ff0 0x7f6358094050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:17.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.583+0000 7f6356ffd700 1 --2- 172.21.15.3:0/305846582 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63580954c0 0x7f63580935d0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f6348000c00 tx=0x7f6348004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:17.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.583+0000 7f633f7fe700 1 -- 172.21.15.3:0/305846582 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6348004510 con 0x7f63580954c0 2023-10-03T10:17:17.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.583+0000 7f633f7fe700 1 -- 172.21.15.3:0/305846582 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634801d7c0 con 0x7f63580954c0 2023-10-03T10:17:17.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.583+0000 7f635d455700 1 -- 172.21.15.3:0/305846582 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63580a0d10 con 0x7f63580954c0 2023-10-03T10:17:17.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.583+0000 7f633f7fe700 1 -- 172.21.15.3:0/305846582 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634801db70 con 0x7f63580954c0 2023-10-03T10:17:17.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.583+0000 7f635d455700 1 -- 172.21.15.3:0/305846582 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6358093180 con 0x7f63580954c0 2023-10-03T10:17:17.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.584+0000 7f633f7fe700 1 -- 172.21.15.3:0/305846582 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f634801dcd0 con 0x7f63580954c0 2023-10-03T10:17:17.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.585+0000 7f633f7fe700 1 -- 172.21.15.3:0/305846582 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f63480449e0 con 0x7f63580954c0 2023-10-03T10:17:17.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.585+0000 7f635d455700 1 -- 172.21.15.3:0/305846582 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6358074ea0 con 0x7f63580954c0 2023-10-03T10:17:17.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.594+0000 7f633f7fe700 1 -- 172.21.15.3:0/305846582 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6348015080 con 0x7f63580954c0 2023-10-03T10:17:17.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.663+0000 7ff8ee4ae700 1 Processor -- start 2023-10-03T10:17:17.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.663+0000 7ff8ee4ae700 1 -- start start 2023-10-03T10:17:17.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.664+0000 7ff8ee4ae700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8e812cc20 0x7ff8e812f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:17.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.664+0000 7ff8ee4ae700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff8e812c2f0 0x7ff8e812fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:17.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.664+0000 7ff8ee4ae700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e809a510 con 0x7ff8e812cff0 2023-10-03T10:17:17.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.664+0000 7ff8ee4ae700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e812c7d0 con 0x7ff8e812c2f0 2023-10-03T10:17:17.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.664+0000 7ff8ee4ae700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e809c190 con 0x7ff8e812cc20 2023-10-03T10:17:17.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.664+0000 7ff8e7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8e812cc20 0x7ff8e812f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:17.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.664+0000 7ff8e7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8e812cc20 0x7ff8e812f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53460/0 (socket says 172.21.15.3:53460) 2023-10-03T10:17:17.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.664+0000 7ff8e7fff700 1 -- 172.21.15.3:0/2575706108 learned_addr learned my addr 172.21.15.3:0/2575706108 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:17.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.664+0000 7ff8e77fe700 1 --2- 172.21.15.3:0/2575706108 >> v2:172.21.15.3:3300/0 conn(0x7ff8e812c2f0 0x7ff8e812fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:17.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.664+0000 7ff8e7fff700 1 -- 172.21.15.3:0/2575706108 >> v2:172.21.15.3:3300/0 conn(0x7ff8e812c2f0 msgr2=0x7ff8e812fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:17.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.664+0000 7ff8e7fff700 1 --2- 172.21.15.3:0/2575706108 >> v2:172.21.15.3:3300/0 conn(0x7ff8e812c2f0 0x7ff8e812fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.664+0000 7ff8e7fff700 1 -- 172.21.15.3:0/2575706108 >> v1:172.21.15.3:6789/0 conn(0x7ff8e812cff0 legacy=0x7ff8e8136cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:17.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.664+0000 7ff8e7fff700 1 -- 172.21.15.3:0/2575706108 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e809b540 con 0x7ff8e812cc20 2023-10-03T10:17:17.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.665+0000 7ff8e7fff700 1 --2- 172.21.15.3:0/2575706108 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8e812cc20 0x7ff8e812f930 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80098f0 tx=0x7ff8d8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=831da0fa0ff2c84d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:17.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.665+0000 7ff8e6ffd700 1 -- 172.21.15.3:0/2575706108 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8011830 con 0x7ff8e812cc20 2023-10-03T10:17:17.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.665+0000 7ff8e6ffd700 1 -- 172.21.15.3:0/2575706108 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d8011990 con 0x7ff8e812cc20 2023-10-03T10:17:17.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.665+0000 7ff8e6ffd700 1 -- 172.21.15.3:0/2575706108 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8011d20 con 0x7ff8e812cc20 2023-10-03T10:17:17.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.665+0000 7ff8ee4ae700 1 -- 172.21.15.3:0/2575706108 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8e812cc20 msgr2=0x7ff8e812f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:17.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.665+0000 7ff8ee4ae700 1 --2- 172.21.15.3:0/2575706108 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8e812cc20 0x7ff8e812f930 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80098f0 tx=0x7ff8d8009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:17.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.666+0000 7ff8ee4ae700 1 -- 172.21.15.3:0/2575706108 shutdown_connections 2023-10-03T10:17:17.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.666+0000 7ff8ee4ae700 1 --2- 172.21.15.3:0/2575706108 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8e812cc20 0x7ff8e812f930 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.666+0000 7ff8ee4ae700 1 --2- 172.21.15.3:0/2575706108 >> v2:172.21.15.3:3300/0 conn(0x7ff8e812c2f0 0x7ff8e812fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.666+0000 7ff8ee4ae700 1 -- 172.21.15.3:0/2575706108 >> 172.21.15.3:0/2575706108 conn(0x7ff8e809f470 msgr2=0x7ff8e80a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:17.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.666+0000 7ff8ee4ae700 1 -- 172.21.15.3:0/2575706108 shutdown_connections 2023-10-03T10:17:17.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.666+0000 7ff8ee4ae700 1 -- 172.21.15.3:0/2575706108 wait complete. 2023-10-03T10:17:17.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.667+0000 7ff8ee4ae700 1 Processor -- start 2023-10-03T10:17:17.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.667+0000 7ff8ee4ae700 1 -- start start 2023-10-03T10:17:17.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.667+0000 7ff8ee4ae700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8e812c2f0 0x7ff8e8070840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:17.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.667+0000 7ff8ee4ae700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8e812cc20 0x7ff8e8070d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:17.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.667+0000 7ff8ee4ae700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8e812cff0 0x7ff8e80712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:17.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.667+0000 7ff8ee4ae700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e8139d90 con 0x7ff8e812c2f0 2023-10-03T10:17:17.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.667+0000 7ff8ee4ae700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e8131d20 con 0x7ff8e812cc20 2023-10-03T10:17:17.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.667+0000 7ff8ee4ae700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e80723d0 con 0x7ff8e812cff0 2023-10-03T10:17:17.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.667+0000 7ff8e7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8e812c2f0 0x7ff8e8070840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:17.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.668+0000 7ff8e7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8e812c2f0 0x7ff8e8070840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41478/0 (socket says 172.21.15.3:41478) 2023-10-03T10:17:17.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.668+0000 7ff8e7fff700 1 -- 172.21.15.3:0/603389695 learned_addr learned my addr 172.21.15.3:0/603389695 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:17.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.668+0000 7ff8eca4b700 1 --2- 172.21.15.3:0/603389695 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8e812cff0 0x7ff8e80712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:17.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.668+0000 7ff8e77fe700 1 --2- 172.21.15.3:0/603389695 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8e812cc20 0x7ff8e8070d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:17.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.668+0000 7ff8e7fff700 1 -- 172.21.15.3:0/603389695 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8e812cff0 msgr2=0x7ff8e80712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:17.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.668+0000 7ff8e7fff700 1 --2- 172.21.15.3:0/603389695 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8e812cff0 0x7ff8e80712c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.668+0000 7ff8e7fff700 1 -- 172.21.15.3:0/603389695 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8e812cc20 msgr2=0x7ff8e8070d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:17.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.668+0000 7ff8e7fff700 1 --2- 172.21.15.3:0/603389695 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8e812cc20 0x7ff8e8070d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.668+0000 7ff8e7fff700 1 -- 172.21.15.3:0/603389695 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d8009550 con 0x7ff8e812c2f0 2023-10-03T10:17:17.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.668+0000 7ff8e7fff700 1 --2- 172.21.15.3:0/603389695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8e812c2f0 0x7ff8e8070840 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8000c00 tx=0x7ff8d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:17.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.668+0000 7ff8e4ff9700 1 -- 172.21.15.3:0/603389695 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8004300 con 0x7ff8e812c2f0 2023-10-03T10:17:17.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.668+0000 7ff8e4ff9700 1 -- 172.21.15.3:0/603389695 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d801f070 con 0x7ff8e812c2f0 2023-10-03T10:17:17.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.668+0000 7ff8e4ff9700 1 -- 172.21.15.3:0/603389695 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d80036a0 con 0x7ff8e812c2f0 2023-10-03T10:17:17.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.668+0000 7ff8ee4ae700 1 -- 172.21.15.3:0/603389695 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e8139700 con 0x7ff8e812c2f0 2023-10-03T10:17:17.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.669+0000 7ff8ee4ae700 1 -- 172.21.15.3:0/603389695 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e8129f60 con 0x7ff8e812c2f0 2023-10-03T10:17:17.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.669+0000 7ff8e4ff9700 1 -- 172.21.15.3:0/603389695 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff8d8021030 con 0x7ff8e812c2f0 2023-10-03T10:17:17.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.670+0000 7ff8e4ff9700 1 -- 172.21.15.3:0/603389695 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff8d80407c0 con 0x7ff8e812c2f0 2023-10-03T10:17:17.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.670+0000 7ff8ee4ae700 1 -- 172.21.15.3:0/603389695 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e8130330 con 0x7ff8e812c2f0 2023-10-03T10:17:17.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.679+0000 7ff8e4ff9700 1 -- 172.21.15.3:0/603389695 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff8d8015080 con 0x7ff8e812c2f0 2023-10-03T10:17:17.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.708+0000 7f36e26a6700 1 Processor -- start 2023-10-03T10:17:17.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.708+0000 7f36e26a6700 1 -- start start 2023-10-03T10:17:17.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.709+0000 7f36e26a6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f36dc09d6f0 0x7f36dc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:17.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.709+0000 7f36e26a6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f36dc134100 0x7f36dc13bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:17.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.709+0000 7f36e26a6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36dc09d2f0 con 0x7f36dc09e000 2023-10-03T10:17:17.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.709+0000 7f36e26a6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36dc09a810 con 0x7f36dc134100 2023-10-03T10:17:17.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.709+0000 7f36e26a6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc09a470 con 0x7f36dc09d6f0 2023-10-03T10:17:17.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.709+0000 7f36dbfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f36dc09d6f0 0x7f36dc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:17.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.709+0000 7f36e0c43700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f36dc134100 0x7f36dc13bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:17.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.709+0000 7f36dbfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f36dc09d6f0 0x7f36dc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53482/0 (socket says 172.21.15.3:53482) 2023-10-03T10:17:17.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.709+0000 7f36e0c43700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f36dc134100 0x7f36dc13bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41494/0 (socket says 172.21.15.3:41494) 2023-10-03T10:17:17.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.709+0000 7f36dbfff700 1 -- 172.21.15.3:0/972753566 learned_addr learned my addr 172.21.15.3:0/972753566 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:17.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.709+0000 7f36db7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f36dc09e000 0x7f36dc1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55288/0 (socket says 172.21.15.3:55288) 2023-10-03T10:17:17.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.709+0000 7f36dbfff700 1 -- 172.21.15.3:0/972753566 >> v2:172.21.15.3:3300/0 conn(0x7f36dc134100 msgr2=0x7f36dc13bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:17.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.709+0000 7f36dbfff700 1 --2- 172.21.15.3:0/972753566 >> v2:172.21.15.3:3300/0 conn(0x7f36dc134100 0x7f36dc13bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.709+0000 7f36dbfff700 1 -- 172.21.15.3:0/972753566 >> v1:172.21.15.3:6789/0 conn(0x7f36dc09e000 legacy=0x7f36dc1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:17.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.709+0000 7f36daffd700 1 -- 172.21.15.3:0/972753566 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3558539018 0 0) 0x7f36dc09d2f0 con 0x7f36dc09e000 2023-10-03T10:17:17.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.709+0000 7f36dbfff700 1 -- 172.21.15.3:0/972753566 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36dc09ec40 con 0x7f36dc09d6f0 2023-10-03T10:17:17.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.709+0000 7f36e0c43700 1 --2- 172.21.15.3:0/972753566 >> v2:172.21.15.3:3300/0 conn(0x7f36dc134100 0x7f36dc13bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:17.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.710+0000 7f36dbfff700 1 --2- 172.21.15.3:0/972753566 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f36dc09d6f0 0x7f36dc09dac0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f36cc0098f0 tx=0x7f36cc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21c37335c67453e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:17.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.710+0000 7f36daffd700 1 -- 172.21.15.3:0/972753566 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc011830 con 0x7f36dc09d6f0 2023-10-03T10:17:17.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.710+0000 7f36daffd700 1 -- 172.21.15.3:0/972753566 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36cc011990 con 0x7f36dc09d6f0 2023-10-03T10:17:17.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.710+0000 7f36daffd700 1 -- 172.21.15.3:0/972753566 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc011d20 con 0x7f36dc09d6f0 2023-10-03T10:17:17.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.710+0000 7f36e26a6700 1 -- 172.21.15.3:0/972753566 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f36dc09d6f0 msgr2=0x7f36dc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:17.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.710+0000 7f36e26a6700 1 --2- 172.21.15.3:0/972753566 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f36dc09d6f0 0x7f36dc09dac0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f36cc0098f0 tx=0x7f36cc009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:17.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.710+0000 7f36e26a6700 1 -- 172.21.15.3:0/972753566 shutdown_connections 2023-10-03T10:17:17.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.710+0000 7f36e26a6700 1 --2- 172.21.15.3:0/972753566 >> v2:172.21.15.3:3300/0 conn(0x7f36dc134100 0x7f36dc13bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.710+0000 7f36e26a6700 1 --2- 172.21.15.3:0/972753566 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f36dc09d6f0 0x7f36dc09dac0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.710+0000 7f36e26a6700 1 -- 172.21.15.3:0/972753566 >> 172.21.15.3:0/972753566 conn(0x7f36dc066f50 msgr2=0x7f36dc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:17.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.710+0000 7f36e26a6700 1 -- 172.21.15.3:0/972753566 shutdown_connections 2023-10-03T10:17:17.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.711+0000 7f36e26a6700 1 -- 172.21.15.3:0/972753566 wait complete. 2023-10-03T10:17:17.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.711+0000 7f36e26a6700 1 Processor -- start 2023-10-03T10:17:17.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.712+0000 7f36e26a6700 1 -- start start 2023-10-03T10:17:17.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.712+0000 7f36e26a6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f36dc09d6f0 0x7f36dc122560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:17.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.712+0000 7f36e26a6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f36dc09e000 0x7f36dc122aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:17.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.712+0000 7f36e26a6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36dc134100 0x7f36dc122fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:17.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.712+0000 7f36e26a6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc133060 con 0x7f36dc134100 2023-10-03T10:17:17.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.712+0000 7f36e26a6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc124210 con 0x7f36dc09d6f0 2023-10-03T10:17:17.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.712+0000 7f36e26a6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc0a2f20 con 0x7f36dc09e000 2023-10-03T10:17:17.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.712+0000 7f36db7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f36dc09e000 0x7f36dc122aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:17.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.712+0000 7f36db7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f36dc09e000 0x7f36dc122aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53494/0 (socket says 172.21.15.3:53494) 2023-10-03T10:17:17.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.712+0000 7f36db7fe700 1 -- 172.21.15.3:0/2309238587 learned_addr learned my addr 172.21.15.3:0/2309238587 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:17.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.712+0000 7f36e0c43700 1 --2- 172.21.15.3:0/2309238587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36dc134100 0x7f36dc122fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:17.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.712+0000 7f36dbfff700 1 --2- 172.21.15.3:0/2309238587 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f36dc09d6f0 0x7f36dc122560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:17.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.713+0000 7f36db7fe700 1 -- 172.21.15.3:0/2309238587 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f36dc09d6f0 msgr2=0x7f36dc122560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:17.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.713+0000 7f36db7fe700 1 --2- 172.21.15.3:0/2309238587 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f36dc09d6f0 0x7f36dc122560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.713+0000 7f36db7fe700 1 -- 172.21.15.3:0/2309238587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36dc134100 msgr2=0x7f36dc122fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:17.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.713+0000 7f36db7fe700 1 --2- 172.21.15.3:0/2309238587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36dc134100 0x7f36dc122fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.713+0000 7f36db7fe700 1 -- 172.21.15.3:0/2309238587 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d4009a70 con 0x7f36dc09e000 2023-10-03T10:17:17.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.713+0000 7f36db7fe700 1 --2- 172.21.15.3:0/2309238587 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f36dc09e000 0x7f36dc122aa0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f36d0001eb0 tx=0x7f36d000c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:17.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.713+0000 7f36d8ff9700 1 -- 172.21.15.3:0/2309238587 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d0004c80 con 0x7f36dc09e000 2023-10-03T10:17:17.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.713+0000 7f36d8ff9700 1 -- 172.21.15.3:0/2309238587 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d0004de0 con 0x7f36dc09e000 2023-10-03T10:17:17.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.713+0000 7f36e26a6700 1 -- 172.21.15.3:0/2309238587 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36cc009550 con 0x7f36dc09e000 2023-10-03T10:17:17.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.713+0000 7f36d8ff9700 1 -- 172.21.15.3:0/2309238587 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d0007310 con 0x7f36dc09e000 2023-10-03T10:17:17.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.713+0000 7f36e26a6700 1 -- 172.21.15.3:0/2309238587 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36dc125ae0 con 0x7f36dc09e000 2023-10-03T10:17:17.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.714+0000 7f36d8ff9700 1 -- 172.21.15.3:0/2309238587 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f36d0007470 con 0x7f36dc09e000 2023-10-03T10:17:17.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.714+0000 7f36e26a6700 1 -- 172.21.15.3:0/2309238587 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36dc125cd0 con 0x7f36dc09e000 2023-10-03T10:17:17.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.714+0000 7f36d8ff9700 1 -- 172.21.15.3:0/2309238587 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f36d003f3b0 con 0x7f36dc09e000 2023-10-03T10:17:17.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.719+0000 7f36d8ff9700 1 -- 172.21.15.3:0/2309238587 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f36d00133c0 con 0x7f36dc09e000 2023-10-03T10:17:17.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.807+0000 7f635d455700 1 -- 172.21.15.3:0/305846582 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f635806b700 con 0x7f63580954c0 2023-10-03T10:17:17.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.807+0000 7f633f7fe700 1 -- 172.21.15.3:0/305846582 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6348030450 con 0x7f63580954c0 2023-10-03T10:17:17.807 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:17.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.810+0000 7f635d455700 1 -- 172.21.15.3:0/305846582 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63580954c0 msgr2=0x7f63580935d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:17.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.810+0000 7f635d455700 1 --2- 172.21.15.3:0/305846582 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63580954c0 0x7f63580935d0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f6348000c00 tx=0x7f6348004750 comp rx=0 tx=0).stop 2023-10-03T10:17:17.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.810+0000 7f635d455700 1 -- 172.21.15.3:0/305846582 shutdown_connections 2023-10-03T10:17:17.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.810+0000 7f635d455700 1 --2- 172.21.15.3:0/305846582 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6358099ff0 0x7f6358094050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.810+0000 7f635d455700 1 --2- 172.21.15.3:0/305846582 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63580954c0 0x7f63580935d0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.811+0000 7f635d455700 1 --2- 172.21.15.3:0/305846582 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6358099680 0x7f6358093b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.811+0000 7f635d455700 1 -- 172.21.15.3:0/305846582 >> 172.21.15.3:0/305846582 conn(0x7f63580a2360 msgr2=0x7f635809d650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:17.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.811+0000 7f635d455700 1 -- 172.21.15.3:0/305846582 shutdown_connections 2023-10-03T10:17:17.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.811+0000 7f635d455700 1 -- 172.21.15.3:0/305846582 wait complete. 2023-10-03T10:17:17.829 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:17.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.901+0000 7ff8ee4ae700 1 -- 172.21.15.3:0/603389695 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff8e80747f0 con 0x7ff8e812c2f0 2023-10-03T10:17:17.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.901+0000 7ff8e4ff9700 1 -- 172.21.15.3:0/603389695 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff8d80199a0 con 0x7ff8e812c2f0 2023-10-03T10:17:17.902 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:17.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.904+0000 7ff8ee4ae700 1 -- 172.21.15.3:0/603389695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8e812c2f0 msgr2=0x7ff8e8070840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:17.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.904+0000 7ff8ee4ae700 1 --2- 172.21.15.3:0/603389695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8e812c2f0 0x7ff8e8070840 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8000c00 tx=0x7ff8d8004750 comp rx=0 tx=0).stop 2023-10-03T10:17:17.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.904+0000 7ff8ee4ae700 1 -- 172.21.15.3:0/603389695 shutdown_connections 2023-10-03T10:17:17.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.904+0000 7ff8ee4ae700 1 --2- 172.21.15.3:0/603389695 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8e812cff0 0x7ff8e80712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.904+0000 7ff8ee4ae700 1 --2- 172.21.15.3:0/603389695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8e812c2f0 0x7ff8e8070840 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.904+0000 7ff8ee4ae700 1 --2- 172.21.15.3:0/603389695 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8e812cc20 0x7ff8e8070d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.904+0000 7ff8ee4ae700 1 -- 172.21.15.3:0/603389695 >> 172.21.15.3:0/603389695 conn(0x7ff8e809f470 msgr2=0x7ff8e8128e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:17.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.904+0000 7ff8ee4ae700 1 -- 172.21.15.3:0/603389695 shutdown_connections 2023-10-03T10:17:17.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.904+0000 7ff8ee4ae700 1 -- 172.21.15.3:0/603389695 wait complete. 2023-10-03T10:17:17.918 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:17.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.927+0000 7f36e26a6700 1 -- 172.21.15.3:0/2309238587 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f36dc1268d0 con 0x7f36dc09e000 2023-10-03T10:17:17.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.927+0000 7f36d8ff9700 1 -- 172.21.15.3:0/2309238587 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f36d00102c0 con 0x7f36dc09e000 2023-10-03T10:17:17.928 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:17.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.930+0000 7f36e26a6700 1 -- 172.21.15.3:0/2309238587 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f36dc09e000 msgr2=0x7f36dc122aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:17.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.930+0000 7f36e26a6700 1 --2- 172.21.15.3:0/2309238587 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f36dc09e000 0x7f36dc122aa0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f36d0001eb0 tx=0x7f36d000c800 comp rx=0 tx=0).stop 2023-10-03T10:17:17.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.930+0000 7f36e26a6700 1 -- 172.21.15.3:0/2309238587 shutdown_connections 2023-10-03T10:17:17.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.930+0000 7f36e26a6700 1 --2- 172.21.15.3:0/2309238587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36dc134100 0x7f36dc122fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.930+0000 7f36e26a6700 1 --2- 172.21.15.3:0/2309238587 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f36dc09d6f0 0x7f36dc122560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.930+0000 7f36e26a6700 1 --2- 172.21.15.3:0/2309238587 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f36dc09e000 0x7f36dc122aa0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:17.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.930+0000 7f36e26a6700 1 -- 172.21.15.3:0/2309238587 >> 172.21.15.3:0/2309238587 conn(0x7f36dc066f50 msgr2=0x7f36dc070c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:17.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.930+0000 7f36e26a6700 1 -- 172.21.15.3:0/2309238587 shutdown_connections 2023-10-03T10:17:17.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:17.930+0000 7f36e26a6700 1 -- 172.21.15.3:0/2309238587 wait complete. 2023-10-03T10:17:17.944 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:18.461 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:18.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.607+0000 7f653bd3a700 1 Processor -- start 2023-10-03T10:17:18.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.607+0000 7f653bd3a700 1 -- start start 2023-10-03T10:17:18.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.607+0000 7f653bd3a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f653409d6f0 0x7f653409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:18.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.607+0000 7f653bd3a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f653409e000 0x7f6534130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:18.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.607+0000 7f653bd3a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f653409f130 con 0x7f653409e410 2023-10-03T10:17:18.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.607+0000 7f653bd3a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f653409f520 con 0x7f653409e000 2023-10-03T10:17:18.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.607+0000 7f653bd3a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653409e930 con 0x7f653409d6f0 2023-10-03T10:17:18.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.608+0000 7f6539ad6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f653409d6f0 0x7f653409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:18.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.608+0000 7f65392d5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f653409e000 0x7f6534130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:18.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.608+0000 7f6539ad6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f653409d6f0 0x7f653409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51038/0 (socket says 172.21.15.3:51038) 2023-10-03T10:17:18.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.608+0000 7f6539ad6700 1 -- 172.21.15.3:0/2943703483 learned_addr learned my addr 172.21.15.3:0/2943703483 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:18.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.608+0000 7f65392d5700 1 -- 172.21.15.3:0/2943703483 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f653409d6f0 msgr2=0x7f653409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:18.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.608+0000 7f65392d5700 1 --2- 172.21.15.3:0/2943703483 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f653409d6f0 0x7f653409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:18.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.608+0000 7f65392d5700 1 -- 172.21.15.3:0/2943703483 >> v1:172.21.15.3:6789/0 conn(0x7f653409e410 legacy=0x7f6534130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:18.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.608+0000 7f65392d5700 1 -- 172.21.15.3:0/2943703483 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f653409b2d0 con 0x7f653409e000 2023-10-03T10:17:18.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.608+0000 7f6539ad6700 1 --2- 172.21.15.3:0/2943703483 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f653409d6f0 0x7f653409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:18.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.608+0000 7f65392d5700 1 --2- 172.21.15.3:0/2943703483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f653409e000 0x7f6534130660 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f6530009840 tx=0x7f6530009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32df8e4b1aa9ab7 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:18.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.608+0000 7f6538ad4700 1 -- 172.21.15.3:0/2943703483 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6530011830 con 0x7f653409e000 2023-10-03T10:17:18.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.608+0000 7f6538ad4700 1 -- 172.21.15.3:0/2943703483 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6530011990 con 0x7f653409e000 2023-10-03T10:17:18.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.608+0000 7f6538ad4700 1 -- 172.21.15.3:0/2943703483 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6530011d20 con 0x7f653409e000 2023-10-03T10:17:18.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.609+0000 7f653bd3a700 1 -- 172.21.15.3:0/2943703483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f653409e000 msgr2=0x7f6534130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:18.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.609+0000 7f653bd3a700 1 --2- 172.21.15.3:0/2943703483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f653409e000 0x7f6534130660 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f6530009840 tx=0x7f6530009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:18.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.609+0000 7f653bd3a700 1 -- 172.21.15.3:0/2943703483 shutdown_connections 2023-10-03T10:17:18.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.609+0000 7f653bd3a700 1 --2- 172.21.15.3:0/2943703483 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f653409d6f0 0x7f653409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:18.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.609+0000 7f653bd3a700 1 --2- 172.21.15.3:0/2943703483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f653409e000 0x7f6534130660 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:18.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.609+0000 7f653bd3a700 1 -- 172.21.15.3:0/2943703483 >> 172.21.15.3:0/2943703483 conn(0x7f6534066f50 msgr2=0x7f653409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:18.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.609+0000 7f653bd3a700 1 -- 172.21.15.3:0/2943703483 shutdown_connections 2023-10-03T10:17:18.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.609+0000 7f653bd3a700 1 -- 172.21.15.3:0/2943703483 wait complete. 2023-10-03T10:17:18.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.610+0000 7f653bd3a700 1 Processor -- start 2023-10-03T10:17:18.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.611+0000 7f653bd3a700 1 -- start start 2023-10-03T10:17:18.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.611+0000 7f653bd3a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f653409d6f0 0x7f653412faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:18.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.611+0000 7f653bd3a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f653409e000 0x7f6534130030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:18.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.611+0000 7f653bd3a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f653409e410 0x7f6534130570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:18.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.611+0000 7f653bd3a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65341336f0 con 0x7f653409d6f0 2023-10-03T10:17:18.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.611+0000 7f653bd3a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6534130f50 con 0x7f653409e000 2023-10-03T10:17:18.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.611+0000 7f653bd3a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f653409c700 con 0x7f653409e410 2023-10-03T10:17:18.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.611+0000 7f6539ad6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f653409d6f0 0x7f653412faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:18.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.611+0000 7f6539ad6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f653409d6f0 0x7f653412faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41522/0 (socket says 172.21.15.3:41522) 2023-10-03T10:17:18.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.611+0000 7f65392d5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f653409e000 0x7f6534130030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:18.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.611+0000 7f6539ad6700 1 -- 172.21.15.3:0/445916120 learned_addr learned my addr 172.21.15.3:0/445916120 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:18.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.611+0000 7f653a2d7700 1 --2- 172.21.15.3:0/445916120 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f653409e410 0x7f6534130570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:18.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.612+0000 7f6539ad6700 1 -- 172.21.15.3:0/445916120 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f653409e410 msgr2=0x7f6534130570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:18.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.612+0000 7f6539ad6700 1 --2- 172.21.15.3:0/445916120 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f653409e410 0x7f6534130570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:18.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.612+0000 7f6539ad6700 1 -- 172.21.15.3:0/445916120 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f653409e000 msgr2=0x7f6534130030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:18.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.612+0000 7f6539ad6700 1 --2- 172.21.15.3:0/445916120 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f653409e000 0x7f6534130030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:18.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.612+0000 7f6539ad6700 1 -- 172.21.15.3:0/445916120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6524009550 con 0x7f653409d6f0 2023-10-03T10:17:18.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.612+0000 7f65392d5700 1 --2- 172.21.15.3:0/445916120 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f653409e000 0x7f6534130030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:18.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.612+0000 7f6539ad6700 1 --2- 172.21.15.3:0/445916120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f653409d6f0 0x7f653412faf0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f652400e9b0 tx=0x7f652400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:18.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.612+0000 7f652a7fc700 1 -- 172.21.15.3:0/445916120 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652400ee50 con 0x7f653409d6f0 2023-10-03T10:17:18.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.612+0000 7f652a7fc700 1 -- 172.21.15.3:0/445916120 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652400c850 con 0x7f653409d6f0 2023-10-03T10:17:18.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.612+0000 7f653bd3a700 1 -- 172.21.15.3:0/445916120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65300094a0 con 0x7f653409d6f0 2023-10-03T10:17:18.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.612+0000 7f652a7fc700 1 -- 172.21.15.3:0/445916120 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652400cbc0 con 0x7f653409d6f0 2023-10-03T10:17:18.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.612+0000 7f653bd3a700 1 -- 172.21.15.3:0/445916120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6534133490 con 0x7f653409d6f0 2023-10-03T10:17:18.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.613+0000 7f652a7fc700 1 -- 172.21.15.3:0/445916120 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f652400cd20 con 0x7f653409d6f0 2023-10-03T10:17:18.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.613+0000 7f652a7fc700 1 -- 172.21.15.3:0/445916120 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f652403faa0 con 0x7f653409d6f0 2023-10-03T10:17:18.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.613+0000 7f653bd3a700 1 -- 172.21.15.3:0/445916120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6534074ea0 con 0x7f653409d6f0 2023-10-03T10:17:18.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.618+0000 7f652a7fc700 1 -- 172.21.15.3:0/445916120 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f652401b3b0 con 0x7f653409d6f0 2023-10-03T10:17:18.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.820+0000 7f653bd3a700 1 -- 172.21.15.3:0/445916120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6534133ed0 con 0x7f653409d6f0 2023-10-03T10:17:18.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.820+0000 7f652a7fc700 1 -- 172.21.15.3:0/445916120 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f652401ab50 con 0x7f653409d6f0 2023-10-03T10:17:18.821 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:18.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.822+0000 7f653bd3a700 1 -- 172.21.15.3:0/445916120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f653409d6f0 msgr2=0x7f653412faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:18.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.822+0000 7f653bd3a700 1 --2- 172.21.15.3:0/445916120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f653409d6f0 0x7f653412faf0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f652400e9b0 tx=0x7f652400ecc0 comp rx=0 tx=0).stop 2023-10-03T10:17:18.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.822+0000 7f653bd3a700 1 -- 172.21.15.3:0/445916120 shutdown_connections 2023-10-03T10:17:18.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.822+0000 7f653bd3a700 1 --2- 172.21.15.3:0/445916120 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f653409e410 0x7f6534130570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:18.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.822+0000 7f653bd3a700 1 --2- 172.21.15.3:0/445916120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f653409d6f0 0x7f653412faf0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:18.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.822+0000 7f653bd3a700 1 --2- 172.21.15.3:0/445916120 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f653409e000 0x7f6534130030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:18.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.822+0000 7f653bd3a700 1 -- 172.21.15.3:0/445916120 >> 172.21.15.3:0/445916120 conn(0x7f6534066f50 msgr2=0x7f65340a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:18.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.823+0000 7f653bd3a700 1 -- 172.21.15.3:0/445916120 shutdown_connections 2023-10-03T10:17:18.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.823+0000 7f653bd3a700 1 -- 172.21.15.3:0/445916120 wait complete. 2023-10-03T10:17:18.830 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:18.836 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:18.919 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:18.945 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:18.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.968+0000 7ff620c8a700 1 Processor -- start 2023-10-03T10:17:18.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.968+0000 7ff620c8a700 1 -- start start 2023-10-03T10:17:18.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.969+0000 7ff620c8a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff61c1338c0 0x7ff61c133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:18.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.969+0000 7ff620c8a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff61c1341d0 0x7ff61c09de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:18.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.969+0000 7ff620c8a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff61c09e390 0x7ff61c09e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:18.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.969+0000 7ff620c8a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff61c070c80 con 0x7ff61c1338c0 2023-10-03T10:17:18.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.969+0000 7ff620c8a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c09ed20 con 0x7ff61c09e390 2023-10-03T10:17:18.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.969+0000 7ff620c8a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c06f720 con 0x7ff61c1341d0 2023-10-03T10:17:18.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.969+0000 7ff619d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff61c1341d0 0x7ff61c09de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:18.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.969+0000 7ff619d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff61c1341d0 0x7ff61c09de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53518/0 (socket says 172.21.15.3:53518) 2023-10-03T10:17:18.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.969+0000 7ff619d9b700 1 -- 172.21.15.3:0/2291825686 learned_addr learned my addr 172.21.15.3:0/2291825686 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:18.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.969+0000 7ff61a59c700 1 --2- 172.21.15.3:0/2291825686 >> v2:172.21.15.3:3300/0 conn(0x7ff61c1338c0 0x7ff61c133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:18.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.969+0000 7ff61ad9d700 1 --2- 172.21.15.3:0/2291825686 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff61c09e390 0x7ff61c09e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:18.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.969+0000 7ff619d9b700 1 -- 172.21.15.3:0/2291825686 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff61c09e390 msgr2=0x7ff61c09e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:18.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.969+0000 7ff619d9b700 1 --2- 172.21.15.3:0/2291825686 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff61c09e390 0x7ff61c09e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:18.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.969+0000 7ff619d9b700 1 -- 172.21.15.3:0/2291825686 >> v2:172.21.15.3:3300/0 conn(0x7ff61c1338c0 msgr2=0x7ff61c133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:18.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.969+0000 7ff619d9b700 1 --2- 172.21.15.3:0/2291825686 >> v2:172.21.15.3:3300/0 conn(0x7ff61c1338c0 0x7ff61c133c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:18.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.969+0000 7ff619d9b700 1 -- 172.21.15.3:0/2291825686 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61c06fce0 con 0x7ff61c1341d0 2023-10-03T10:17:18.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.969+0000 7ff61a59c700 1 --2- 172.21.15.3:0/2291825686 >> v2:172.21.15.3:3300/0 conn(0x7ff61c1338c0 0x7ff61c133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:18.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.969+0000 7ff619d9b700 1 --2- 172.21.15.3:0/2291825686 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff61c1341d0 0x7ff61c09de50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff610009920 tx=0x7ff610009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a4a2763eb9a2a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:18.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.970+0000 7ff61959a700 1 -- 172.21.15.3:0/2291825686 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610011830 con 0x7ff61c1341d0 2023-10-03T10:17:18.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.970+0000 7ff61959a700 1 -- 172.21.15.3:0/2291825686 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff610011990 con 0x7ff61c1341d0 2023-10-03T10:17:18.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.970+0000 7ff61959a700 1 -- 172.21.15.3:0/2291825686 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610011d20 con 0x7ff61c1341d0 2023-10-03T10:17:18.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.970+0000 7ff620c8a700 1 -- 172.21.15.3:0/2291825686 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff61c1341d0 msgr2=0x7ff61c09de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:18.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.970+0000 7ff620c8a700 1 --2- 172.21.15.3:0/2291825686 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff61c1341d0 0x7ff61c09de50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff610009920 tx=0x7ff610009c30 comp rx=0 tx=0).stop 2023-10-03T10:17:18.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.970+0000 7ff620c8a700 1 -- 172.21.15.3:0/2291825686 shutdown_connections 2023-10-03T10:17:18.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.970+0000 7ff620c8a700 1 --2- 172.21.15.3:0/2291825686 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff61c09e390 0x7ff61c09e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:18.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.970+0000 7ff620c8a700 1 --2- 172.21.15.3:0/2291825686 >> v2:172.21.15.3:3300/0 conn(0x7ff61c1338c0 0x7ff61c133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:18.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.970+0000 7ff620c8a700 1 --2- 172.21.15.3:0/2291825686 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff61c1341d0 0x7ff61c09de50 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:18.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.971+0000 7ff620c8a700 1 -- 172.21.15.3:0/2291825686 >> 172.21.15.3:0/2291825686 conn(0x7ff61c0a2aa0 msgr2=0x7ff61c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:18.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.971+0000 7ff620c8a700 1 -- 172.21.15.3:0/2291825686 shutdown_connections 2023-10-03T10:17:18.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.971+0000 7ff620c8a700 1 -- 172.21.15.3:0/2291825686 wait complete. 2023-10-03T10:17:18.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.972+0000 7ff620c8a700 1 Processor -- start 2023-10-03T10:17:18.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.972+0000 7ff620c8a700 1 -- start start 2023-10-03T10:17:18.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.972+0000 7ff620c8a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff61c09e390 0x7ff61c11eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:18.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.972+0000 7ff620c8a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff61c1338c0 0x7ff61c11f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:18.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.972+0000 7ff620c8a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff61c1341d0 0x7ff61c11f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:18.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.972+0000 7ff620c8a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c1222f0 con 0x7ff61c1341d0 2023-10-03T10:17:18.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.972+0000 7ff620c8a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c1207d0 con 0x7ff61c1338c0 2023-10-03T10:17:18.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.972+0000 7ff620c8a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c130c50 con 0x7ff61c09e390 2023-10-03T10:17:18.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.973+0000 7ff619d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff61c1338c0 0x7ff61c11f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:18.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.973+0000 7ff619d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff61c1338c0 0x7ff61c11f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51046/0 (socket says 172.21.15.3:51046) 2023-10-03T10:17:18.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.973+0000 7ff619d9b700 1 -- 172.21.15.3:0/2780851992 learned_addr learned my addr 172.21.15.3:0/2780851992 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:18.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.973+0000 7ff61a59c700 1 --2- 172.21.15.3:0/2780851992 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff61c09e390 0x7ff61c11eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:18.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.973+0000 7ff61ad9d700 1 --2- 172.21.15.3:0/2780851992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff61c1341d0 0x7ff61c11f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:18.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.973+0000 7ff619d9b700 1 -- 172.21.15.3:0/2780851992 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff61c09e390 msgr2=0x7ff61c11eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:18.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.973+0000 7ff619d9b700 1 --2- 172.21.15.3:0/2780851992 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff61c09e390 0x7ff61c11eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:18.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.973+0000 7ff619d9b700 1 -- 172.21.15.3:0/2780851992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff61c1341d0 msgr2=0x7ff61c11f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:18.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.973+0000 7ff619d9b700 1 --2- 172.21.15.3:0/2780851992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff61c1341d0 0x7ff61c11f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:18.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.973+0000 7ff619d9b700 1 -- 172.21.15.3:0/2780851992 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60c0094a0 con 0x7ff61c1338c0 2023-10-03T10:17:18.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.973+0000 7ff619d9b700 1 --2- 172.21.15.3:0/2780851992 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff61c1338c0 0x7ff61c11f090 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff610000c00 tx=0x7ff610004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:18.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.973+0000 7ff60affd700 1 -- 172.21.15.3:0/2780851992 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610004450 con 0x7ff61c1338c0 2023-10-03T10:17:18.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.974+0000 7ff60affd700 1 -- 172.21.15.3:0/2780851992 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6100045b0 con 0x7ff61c1338c0 2023-10-03T10:17:18.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.974+0000 7ff620c8a700 1 -- 172.21.15.3:0/2780851992 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff610009550 con 0x7ff61c1338c0 2023-10-03T10:17:18.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.974+0000 7ff60affd700 1 -- 172.21.15.3:0/2780851992 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6100038e0 con 0x7ff61c1338c0 2023-10-03T10:17:18.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.974+0000 7ff620c8a700 1 -- 172.21.15.3:0/2780851992 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61c12f5d0 con 0x7ff61c1338c0 2023-10-03T10:17:18.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.975+0000 7ff60affd700 1 -- 172.21.15.3:0/2780851992 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff610003a40 con 0x7ff61c1338c0 2023-10-03T10:17:18.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.975+0000 7ff60affd700 1 -- 172.21.15.3:0/2780851992 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff610040620 con 0x7ff61c1338c0 2023-10-03T10:17:18.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.975+0000 7ff620c8a700 1 -- 172.21.15.3:0/2780851992 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61c074ea0 con 0x7ff61c1338c0 2023-10-03T10:17:18.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:18.980+0000 7ff60affd700 1 -- 172.21.15.3:0/2780851992 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff61001a420 con 0x7ff61c1338c0 2023-10-03T10:17:19.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.061+0000 7f3cc5054700 1 Processor -- start 2023-10-03T10:17:19.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.061+0000 7f3cc5054700 1 -- start start 2023-10-03T10:17:19.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.062+0000 7f3cc5054700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3cc006a770 0x7f3cc006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:19.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.062+0000 7f3cc5054700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3cc0131d10 0x7f3cc0139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:19.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.062+0000 7f3cc5054700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cc009ece0 con 0x7f3cc006b080 2023-10-03T10:17:19.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.062+0000 7f3cc5054700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc009e760 con 0x7f3cc006a770 2023-10-03T10:17:19.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.062+0000 7f3cc5054700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc009e040 con 0x7f3cc0131d10 2023-10-03T10:17:19.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.062+0000 7f3cbf59e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3cc0131d10 0x7f3cc0139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:19.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.062+0000 7f3cbe59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3cc006b080 0x7f3cc0131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55302/0 (socket says 172.21.15.3:55302) 2023-10-03T10:17:19.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.062+0000 7f3cbe59c700 1 -- 172.21.15.3:0/3210887208 learned_addr learned my addr 172.21.15.3:0/3210887208 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:19.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.062+0000 7f3cbed9d700 1 --2- 172.21.15.3:0/3210887208 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3cc006a770 0x7f3cc006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:19.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.062+0000 7f3cbf59e700 1 -- 172.21.15.3:0/3210887208 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3cc006a770 msgr2=0x7f3cc006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:19.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.062+0000 7f3cbf59e700 1 --2- 172.21.15.3:0/3210887208 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3cc006a770 0x7f3cc006ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:19.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.062+0000 7f3cbf59e700 1 -- 172.21.15.3:0/3210887208 >> v1:172.21.15.3:6789/0 conn(0x7f3cc006b080 legacy=0x7f3cc0131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:19.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.062+0000 7f3cbf59e700 1 -- 172.21.15.3:0/3210887208 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc009f630 con 0x7f3cc0131d10 2023-10-03T10:17:19.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.062+0000 7f3cbed9d700 1 --2- 172.21.15.3:0/3210887208 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3cc006a770 0x7f3cc006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:19.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.063+0000 7f3cbf59e700 1 --2- 172.21.15.3:0/3210887208 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3cc0131d10 0x7f3cc0139860 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009dc0 tx=0x7f3cb800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f951abbfdc82b760 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:19.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.063+0000 7f3cbdd9b700 1 -- 172.21.15.3:0/3210887208 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb800e070 con 0x7f3cc0131d10 2023-10-03T10:17:19.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.063+0000 7f3cbdd9b700 1 -- 172.21.15.3:0/3210887208 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb800bb20 con 0x7f3cc0131d10 2023-10-03T10:17:19.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.063+0000 7f3cbdd9b700 1 -- 172.21.15.3:0/3210887208 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8012430 con 0x7f3cc0131d10 2023-10-03T10:17:19.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.063+0000 7f3cc5054700 1 -- 172.21.15.3:0/3210887208 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3cc0131d10 msgr2=0x7f3cc0139860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:19.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.063+0000 7f3cc5054700 1 --2- 172.21.15.3:0/3210887208 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3cc0131d10 0x7f3cc0139860 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009dc0 tx=0x7f3cb800b330 comp rx=0 tx=0).stop 2023-10-03T10:17:19.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.063+0000 7f3cc5054700 1 -- 172.21.15.3:0/3210887208 shutdown_connections 2023-10-03T10:17:19.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.063+0000 7f3cc5054700 1 --2- 172.21.15.3:0/3210887208 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3cc006a770 0x7f3cc006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:19.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.064+0000 7f3cc5054700 1 --2- 172.21.15.3:0/3210887208 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3cc0131d10 0x7f3cc0139860 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:19.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.064+0000 7f3cc5054700 1 -- 172.21.15.3:0/3210887208 >> 172.21.15.3:0/3210887208 conn(0x7f3cc0066f50 msgr2=0x7f3cc009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:19.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.064+0000 7f3cc5054700 1 -- 172.21.15.3:0/3210887208 shutdown_connections 2023-10-03T10:17:19.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.064+0000 7f3cc5054700 1 -- 172.21.15.3:0/3210887208 wait complete. 2023-10-03T10:17:19.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.066+0000 7f3cc5054700 1 Processor -- start 2023-10-03T10:17:19.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.066+0000 7f3cc5054700 1 -- start start 2023-10-03T10:17:19.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.066+0000 7f3cc5054700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3cc006a770 0x7f3cc01c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:19.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.066+0000 7f3cc5054700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3cc006b080 0x7f3cc01c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:19.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.066+0000 7f3cc5054700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cc0131d10 0x7f3cc0130b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:19.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.066+0000 7f3cc5054700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc0119910 con 0x7f3cc0131d10 2023-10-03T10:17:19.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.066+0000 7f3cc5054700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc01381e0 con 0x7f3cc006b080 2023-10-03T10:17:19.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.066+0000 7f3cc5054700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc01c9230 con 0x7f3cc006a770 2023-10-03T10:17:19.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.067+0000 7f3cbf59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cc0131d10 0x7f3cc0130b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:19.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.067+0000 7f3cbed9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3cc006a770 0x7f3cc01c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:19.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.067+0000 7f3cbf59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cc0131d10 0x7f3cc0130b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41558/0 (socket says 172.21.15.3:41558) 2023-10-03T10:17:19.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.067+0000 7f3cbf59e700 1 -- 172.21.15.3:0/2144629307 learned_addr learned my addr 172.21.15.3:0/2144629307 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:19.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.067+0000 7f3cbe59c700 1 --2- 172.21.15.3:0/2144629307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3cc006b080 0x7f3cc01c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:19.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.067+0000 7f3cbf59e700 1 -- 172.21.15.3:0/2144629307 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3cc006a770 msgr2=0x7f3cc01c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:19.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.067+0000 7f3cbf59e700 1 --2- 172.21.15.3:0/2144629307 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3cc006a770 0x7f3cc01c87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:19.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.067+0000 7f3cbf59e700 1 -- 172.21.15.3:0/2144629307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3cc006b080 msgr2=0x7f3cc01c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:19.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.067+0000 7f3cbf59e700 1 --2- 172.21.15.3:0/2144629307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3cc006b080 0x7f3cc01c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:19.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.067+0000 7f3cbf59e700 1 -- 172.21.15.3:0/2144629307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb00094a0 con 0x7f3cc0131d10 2023-10-03T10:17:19.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.067+0000 7f3cbed9d700 1 --2- 172.21.15.3:0/2144629307 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3cc006a770 0x7f3cc01c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:19.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.067+0000 7f3cbf59e700 1 --2- 172.21.15.3:0/2144629307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cc0131d10 0x7f3cc0130b50 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f3cb800b370 tx=0x7f3cb8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:19.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.067+0000 7f3ca77fe700 1 -- 172.21.15.3:0/2144629307 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8004800 con 0x7f3cc0131d10 2023-10-03T10:17:19.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.068+0000 7f3ca77fe700 1 -- 172.21.15.3:0/2144629307 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb8020070 con 0x7f3cc0131d10 2023-10-03T10:17:19.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.068+0000 7f3cc5054700 1 -- 172.21.15.3:0/2144629307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb8009a70 con 0x7f3cc0131d10 2023-10-03T10:17:19.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.068+0000 7f3ca77fe700 1 -- 172.21.15.3:0/2144629307 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb80276c0 con 0x7f3cc0131d10 2023-10-03T10:17:19.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.068+0000 7f3cc5054700 1 -- 172.21.15.3:0/2144629307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc01390e0 con 0x7f3cc0131d10 2023-10-03T10:17:19.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.068+0000 7f3ca77fe700 1 -- 172.21.15.3:0/2144629307 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3cb8012430 con 0x7f3cc0131d10 2023-10-03T10:17:19.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.069+0000 7f3ca77fe700 1 -- 172.21.15.3:0/2144629307 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3cb8022070 con 0x7f3cc0131d10 2023-10-03T10:17:19.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.069+0000 7f3cc5054700 1 -- 172.21.15.3:0/2144629307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc0074ea0 con 0x7f3cc0131d10 2023-10-03T10:17:19.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.074+0000 7f3ca77fe700 1 -- 172.21.15.3:0/2144629307 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3cb80126e0 con 0x7f3cc0131d10 2023-10-03T10:17:19.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.104+0000 7f89d034d700 1 Processor -- start 2023-10-03T10:17:19.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.104+0000 7f89d034d700 1 -- start start 2023-10-03T10:17:19.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.105+0000 7f89d034d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89c80a0a30 0x7f89c80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:19.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.105+0000 7f89d034d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f89c80a1340 0x7f89c8133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:19.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.105+0000 7f89d034d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f89c8133fa0 0x7f89c8134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:19.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.105+0000 7f89d034d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c809edd0 con 0x7f89c80a1340 2023-10-03T10:17:19.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.105+0000 7f89d034d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c809e1f0 con 0x7f89c80a0a30 2023-10-03T10:17:19.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.105+0000 7f89d034d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89c809dc50 con 0x7f89c8133fa0 2023-10-03T10:17:19.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.105+0000 7f89cd8e8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f89c80a1340 0x7f89c8133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:19.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.105+0000 7f89ce0e9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89c80a0a30 0x7f89c80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:19.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.105+0000 7f89cd8e8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f89c80a1340 0x7f89c8133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41564/0 (socket says 172.21.15.3:41564) 2023-10-03T10:17:19.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.105+0000 7f89cd8e8700 1 -- 172.21.15.3:0/3996750495 learned_addr learned my addr 172.21.15.3:0/3996750495 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:19.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.105+0000 7f89ce8ea700 1 --2- 172.21.15.3:0/3996750495 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f89c8133fa0 0x7f89c8134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:19.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.105+0000 7f89cd8e8700 1 -- 172.21.15.3:0/3996750495 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f89c8133fa0 msgr2=0x7f89c8134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:19.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.105+0000 7f89cd8e8700 1 --2- 172.21.15.3:0/3996750495 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f89c8133fa0 0x7f89c8134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:19.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.105+0000 7f89cd8e8700 1 -- 172.21.15.3:0/3996750495 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89c80a0a30 msgr2=0x7f89c80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:19.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.105+0000 7f89cd8e8700 1 --2- 172.21.15.3:0/3996750495 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89c80a0a30 0x7f89c80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:19.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.105+0000 7f89cd8e8700 1 -- 172.21.15.3:0/3996750495 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c80a1fe0 con 0x7f89c80a1340 2023-10-03T10:17:19.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.105+0000 7f89ce8ea700 1 --2- 172.21.15.3:0/3996750495 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f89c8133fa0 0x7f89c8134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:19.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.106+0000 7f89cd8e8700 1 --2- 172.21.15.3:0/3996750495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89c80a1340 0x7f89c8133a60 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f89b80098f0 tx=0x7f89b8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7b2b558ec9a0ce server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:19.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.106+0000 7f89cd0e7700 1 -- 172.21.15.3:0/3996750495 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b8011830 con 0x7f89c80a1340 2023-10-03T10:17:19.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.106+0000 7f89cd0e7700 1 -- 172.21.15.3:0/3996750495 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b8011990 con 0x7f89c80a1340 2023-10-03T10:17:19.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.106+0000 7f89cd0e7700 1 -- 172.21.15.3:0/3996750495 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b8011d20 con 0x7f89c80a1340 2023-10-03T10:17:19.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.106+0000 7f89d034d700 1 -- 172.21.15.3:0/3996750495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89c80a1340 msgr2=0x7f89c8133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:19.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.106+0000 7f89d034d700 1 --2- 172.21.15.3:0/3996750495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89c80a1340 0x7f89c8133a60 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f89b80098f0 tx=0x7f89b8009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:19.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.107+0000 7f89d034d700 1 -- 172.21.15.3:0/3996750495 shutdown_connections 2023-10-03T10:17:19.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.107+0000 7f89d034d700 1 --2- 172.21.15.3:0/3996750495 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f89c8133fa0 0x7f89c8134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:19.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.107+0000 7f89d034d700 1 --2- 172.21.15.3:0/3996750495 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89c80a0a30 0x7f89c80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:19.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.107+0000 7f89d034d700 1 --2- 172.21.15.3:0/3996750495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89c80a1340 0x7f89c8133a60 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:19.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.107+0000 7f89d034d700 1 -- 172.21.15.3:0/3996750495 >> 172.21.15.3:0/3996750495 conn(0x7f89c80a6550 msgr2=0x7f89c80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:19.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.107+0000 7f89d034d700 1 -- 172.21.15.3:0/3996750495 shutdown_connections 2023-10-03T10:17:19.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.107+0000 7f89d034d700 1 -- 172.21.15.3:0/3996750495 wait complete. 2023-10-03T10:17:19.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.108+0000 7f89d034d700 1 Processor -- start 2023-10-03T10:17:19.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.108+0000 7f89d034d700 1 -- start start 2023-10-03T10:17:19.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.108+0000 7f89d034d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89c80a0a30 0x7f89c81205e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:19.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.108+0000 7f89d034d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f89c80a1340 0x7f89c8120b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:19.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.108+0000 7f89d034d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89c8133fa0 0x7f89c8121060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:19.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.108+0000 7f89d034d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c8123e20 con 0x7f89c80a0a30 2023-10-03T10:17:19.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.108+0000 7f89d034d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c80a3510 con 0x7f89c8133fa0 2023-10-03T10:17:19.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.108+0000 7f89d034d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89c8122300 con 0x7f89c80a1340 2023-10-03T10:17:19.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.108+0000 7f89ce0e9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89c80a0a30 0x7f89c81205e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:19.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.108+0000 7f89cd8e8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f89c80a1340 0x7f89c8120b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:19.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.108+0000 7f89ce0e9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89c80a0a30 0x7f89c81205e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41580/0 (socket says 172.21.15.3:41580) 2023-10-03T10:17:19.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.108+0000 7f89cd8e8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f89c80a1340 0x7f89c8120b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53560/0 (socket says 172.21.15.3:53560) 2023-10-03T10:17:19.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.108+0000 7f89ce0e9700 1 -- 172.21.15.3:0/897537327 learned_addr learned my addr 172.21.15.3:0/897537327 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:19.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.108+0000 7f89ce8ea700 1 --2- 172.21.15.3:0/897537327 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89c8133fa0 0x7f89c8121060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:19.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.109+0000 7f89ce0e9700 1 -- 172.21.15.3:0/897537327 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f89c80a1340 msgr2=0x7f89c8120b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:19.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.109+0000 7f89ce0e9700 1 --2- 172.21.15.3:0/897537327 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f89c80a1340 0x7f89c8120b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:19.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.109+0000 7f89ce0e9700 1 -- 172.21.15.3:0/897537327 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89c8133fa0 msgr2=0x7f89c8121060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:19.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.109+0000 7f89ce0e9700 1 --2- 172.21.15.3:0/897537327 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89c8133fa0 0x7f89c8121060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:19.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.109+0000 7f89ce0e9700 1 -- 172.21.15.3:0/897537327 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c0009a70 con 0x7f89c80a0a30 2023-10-03T10:17:19.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.109+0000 7f89cd8e8700 1 --2- 172.21.15.3:0/897537327 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f89c80a1340 0x7f89c8120b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:19.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.109+0000 7f89ce0e9700 1 --2- 172.21.15.3:0/897537327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89c80a0a30 0x7f89c81205e0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f89c400e9b0 tx=0x7f89c400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:19.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.109+0000 7f89beffd700 1 -- 172.21.15.3:0/897537327 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c4009710 con 0x7f89c80a0a30 2023-10-03T10:17:19.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.109+0000 7f89beffd700 1 -- 172.21.15.3:0/897537327 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c400ee40 con 0x7f89c80a0a30 2023-10-03T10:17:19.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.109+0000 7f89d034d700 1 -- 172.21.15.3:0/897537327 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b8009550 con 0x7f89c80a0a30 2023-10-03T10:17:19.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.109+0000 7f89beffd700 1 -- 172.21.15.3:0/897537327 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c400ca50 con 0x7f89c80a0a30 2023-10-03T10:17:19.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.109+0000 7f89d034d700 1 -- 172.21.15.3:0/897537327 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c8124890 con 0x7f89c80a0a30 2023-10-03T10:17:19.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.110+0000 7f89beffd700 1 -- 172.21.15.3:0/897537327 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f89c400cbb0 con 0x7f89c80a0a30 2023-10-03T10:17:19.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.110+0000 7f89beffd700 1 -- 172.21.15.3:0/897537327 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f89c403f710 con 0x7f89c80a0a30 2023-10-03T10:17:19.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.110+0000 7f89d034d700 1 -- 172.21.15.3:0/897537327 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89c8123a90 con 0x7f89c80a0a30 2023-10-03T10:17:19.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.120+0000 7f89beffd700 1 -- 172.21.15.3:0/897537327 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f89c4021020 con 0x7f89c80a0a30 2023-10-03T10:17:19.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.191+0000 7ff620c8a700 1 -- 172.21.15.3:0/2780851992 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff61c122dc0 con 0x7ff61c1338c0 2023-10-03T10:17:19.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.191+0000 7ff60affd700 1 -- 172.21.15.3:0/2780851992 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff610028070 con 0x7ff61c1338c0 2023-10-03T10:17:19.192 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:19.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.194+0000 7ff620c8a700 1 -- 172.21.15.3:0/2780851992 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff61c1338c0 msgr2=0x7ff61c11f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:19.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.194+0000 7ff620c8a700 1 --2- 172.21.15.3:0/2780851992 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff61c1338c0 0x7ff61c11f090 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff610000c00 tx=0x7ff610004750 comp rx=0 tx=0).stop 2023-10-03T10:17:19.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.194+0000 7ff620c8a700 1 -- 172.21.15.3:0/2780851992 shutdown_connections 2023-10-03T10:17:19.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.194+0000 7ff620c8a700 1 --2- 172.21.15.3:0/2780851992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff61c1341d0 0x7ff61c11f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:19.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.194+0000 7ff620c8a700 1 --2- 172.21.15.3:0/2780851992 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff61c09e390 0x7ff61c11eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:19.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.194+0000 7ff620c8a700 1 --2- 172.21.15.3:0/2780851992 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff61c1338c0 0x7ff61c11f090 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:19.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.194+0000 7ff620c8a700 1 -- 172.21.15.3:0/2780851992 >> 172.21.15.3:0/2780851992 conn(0x7ff61c0a2aa0 msgr2=0x7ff61c130f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:19.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.194+0000 7ff620c8a700 1 -- 172.21.15.3:0/2780851992 shutdown_connections 2023-10-03T10:17:19.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.194+0000 7ff620c8a700 1 -- 172.21.15.3:0/2780851992 wait complete. 2023-10-03T10:17:19.208 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:19.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.291+0000 7f3cc5054700 1 -- 172.21.15.3:0/2144629307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3cc00008d0 con 0x7f3cc0131d10 2023-10-03T10:17:19.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.291+0000 7f3ca77fe700 1 -- 172.21.15.3:0/2144629307 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3cb801a380 con 0x7f3cc0131d10 2023-10-03T10:17:19.291 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:19.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.293+0000 7f3cc5054700 1 -- 172.21.15.3:0/2144629307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cc0131d10 msgr2=0x7f3cc0130b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:19.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.293+0000 7f3cc5054700 1 --2- 172.21.15.3:0/2144629307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cc0131d10 0x7f3cc0130b50 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f3cb800b370 tx=0x7f3cb8004ad0 comp rx=0 tx=0).stop 2023-10-03T10:17:19.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.293+0000 7f3cc5054700 1 -- 172.21.15.3:0/2144629307 shutdown_connections 2023-10-03T10:17:19.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.293+0000 7f3cc5054700 1 --2- 172.21.15.3:0/2144629307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3cc0131d10 0x7f3cc0130b50 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:19.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.293+0000 7f3cc5054700 1 --2- 172.21.15.3:0/2144629307 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3cc006a770 0x7f3cc01c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:19.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.293+0000 7f3cc5054700 1 --2- 172.21.15.3:0/2144629307 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3cc006b080 0x7f3cc01c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:19.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.293+0000 7f3cc5054700 1 -- 172.21.15.3:0/2144629307 >> 172.21.15.3:0/2144629307 conn(0x7f3cc0066f50 msgr2=0x7f3cc00a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:19.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.294+0000 7f3cc5054700 1 -- 172.21.15.3:0/2144629307 shutdown_connections 2023-10-03T10:17:19.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.294+0000 7f3cc5054700 1 -- 172.21.15.3:0/2144629307 wait complete. 2023-10-03T10:17:19.308 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:19.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.330+0000 7f89d034d700 1 -- 172.21.15.3:0/897537327 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f89c8074ea0 con 0x7f89c80a0a30 2023-10-03T10:17:19.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.330+0000 7f89beffd700 1 -- 172.21.15.3:0/897537327 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f89c4017070 con 0x7f89c80a0a30 2023-10-03T10:17:19.330 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:19.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.333+0000 7f89d034d700 1 -- 172.21.15.3:0/897537327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89c80a0a30 msgr2=0x7f89c81205e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:19.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.333+0000 7f89d034d700 1 --2- 172.21.15.3:0/897537327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89c80a0a30 0x7f89c81205e0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f89c400e9b0 tx=0x7f89c400ecc0 comp rx=0 tx=0).stop 2023-10-03T10:17:19.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.333+0000 7f89d034d700 1 -- 172.21.15.3:0/897537327 shutdown_connections 2023-10-03T10:17:19.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.333+0000 7f89d034d700 1 --2- 172.21.15.3:0/897537327 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89c8133fa0 0x7f89c8121060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:19.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.333+0000 7f89d034d700 1 --2- 172.21.15.3:0/897537327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89c80a0a30 0x7f89c81205e0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:19.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.333+0000 7f89d034d700 1 --2- 172.21.15.3:0/897537327 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f89c80a1340 0x7f89c8120b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:19.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.333+0000 7f89d034d700 1 -- 172.21.15.3:0/897537327 >> 172.21.15.3:0/897537327 conn(0x7f89c80a6550 msgr2=0x7f89c81bafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:19.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.333+0000 7f89d034d700 1 -- 172.21.15.3:0/897537327 shutdown_connections 2023-10-03T10:17:19.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.333+0000 7f89d034d700 1 -- 172.21.15.3:0/897537327 wait complete. 2023-10-03T10:17:19.346 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:19.838 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:19.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.985+0000 7ff0b3b05700 1 Processor -- start 2023-10-03T10:17:19.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.985+0000 7ff0b3b05700 1 -- start start 2023-10-03T10:17:19.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.986+0000 7ff0b3b05700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff0ac1337f0 0x7ff0ac138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:19.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.986+0000 7ff0b3b05700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff0ac138dc0 0x7ff0ac13b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:19.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.986+0000 7ff0b3b05700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0ac06b990 con 0x7ff0ac133420 2023-10-03T10:17:19.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.986+0000 7ff0b3b05700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0ac09b0e0 con 0x7ff0ac138dc0 2023-10-03T10:17:19.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.986+0000 7ff0b3b05700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac133c20 con 0x7ff0ac1337f0 2023-10-03T10:17:19.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.986+0000 7ff0b20a2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff0ac138dc0 0x7ff0ac13b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:19.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.986+0000 7ff0b10a0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff0ac1337f0 0x7ff0ac138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:19.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.986+0000 7ff0b20a2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff0ac138dc0 0x7ff0ac13b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41588/0 (socket says 172.21.15.3:41588) 2023-10-03T10:17:19.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.986+0000 7ff0b20a2700 1 -- 172.21.15.3:0/888157348 learned_addr learned my addr 172.21.15.3:0/888157348 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:19.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.986+0000 7ff0b10a0700 1 -- 172.21.15.3:0/888157348 >> v2:172.21.15.3:3300/0 conn(0x7ff0ac138dc0 msgr2=0x7ff0ac13b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:19.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.986+0000 7ff0b10a0700 1 --2- 172.21.15.3:0/888157348 >> v2:172.21.15.3:3300/0 conn(0x7ff0ac138dc0 0x7ff0ac13b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:19.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.986+0000 7ff0b10a0700 1 -- 172.21.15.3:0/888157348 >> v1:172.21.15.3:6789/0 conn(0x7ff0ac133420 legacy=0x7ff0ac133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:19.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.986+0000 7ff0b10a0700 1 -- 172.21.15.3:0/888157348 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ac0a0590 con 0x7ff0ac1337f0 2023-10-03T10:17:19.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.986+0000 7ff0b20a2700 1 --2- 172.21.15.3:0/888157348 >> v2:172.21.15.3:3300/0 conn(0x7ff0ac138dc0 0x7ff0ac13b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:19.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.986+0000 7ff0b10a0700 1 --2- 172.21.15.3:0/888157348 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff0ac1337f0 0x7ff0ac138880 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff09c009840 tx=0x7ff09c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3471ec4174fe554 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:19.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.986+0000 7ff0b089f700 1 -- 172.21.15.3:0/888157348 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c011830 con 0x7ff0ac1337f0 2023-10-03T10:17:19.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.986+0000 7ff0b089f700 1 -- 172.21.15.3:0/888157348 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09c011990 con 0x7ff0ac1337f0 2023-10-03T10:17:19.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.986+0000 7ff0b089f700 1 -- 172.21.15.3:0/888157348 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c011d20 con 0x7ff0ac1337f0 2023-10-03T10:17:19.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.987+0000 7ff0b3b05700 1 -- 172.21.15.3:0/888157348 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff0ac1337f0 msgr2=0x7ff0ac138880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:19.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.987+0000 7ff0b3b05700 1 --2- 172.21.15.3:0/888157348 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff0ac1337f0 0x7ff0ac138880 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff09c009840 tx=0x7ff09c009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:19.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.987+0000 7ff0b3b05700 1 -- 172.21.15.3:0/888157348 shutdown_connections 2023-10-03T10:17:19.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.987+0000 7ff0b3b05700 1 --2- 172.21.15.3:0/888157348 >> v2:172.21.15.3:3300/0 conn(0x7ff0ac138dc0 0x7ff0ac13b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:19.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.987+0000 7ff0b3b05700 1 --2- 172.21.15.3:0/888157348 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff0ac1337f0 0x7ff0ac138880 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:19.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.987+0000 7ff0b3b05700 1 -- 172.21.15.3:0/888157348 >> 172.21.15.3:0/888157348 conn(0x7ff0ac0a4210 msgr2=0x7ff0ac0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:19.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.987+0000 7ff0b3b05700 1 -- 172.21.15.3:0/888157348 shutdown_connections 2023-10-03T10:17:19.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.987+0000 7ff0b3b05700 1 -- 172.21.15.3:0/888157348 wait complete. 2023-10-03T10:17:19.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.988+0000 7ff0b3b05700 1 Processor -- start 2023-10-03T10:17:19.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.988+0000 7ff0b3b05700 1 -- start start 2023-10-03T10:17:19.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.989+0000 7ff0b3b05700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0ac133420 0x7ff0ac0709a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:19.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.989+0000 7ff0b3b05700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff0ac1337f0 0x7ff0ac070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:19.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.989+0000 7ff0b3b05700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff0ac138dc0 0x7ff0ac071420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:19.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.989+0000 7ff0b3b05700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac09ca90 con 0x7ff0ac133420 2023-10-03T10:17:19.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.989+0000 7ff0b3b05700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac09d640 con 0x7ff0ac1337f0 2023-10-03T10:17:19.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.989+0000 7ff0b3b05700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac09c7d0 con 0x7ff0ac138dc0 2023-10-03T10:17:19.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.989+0000 7ff0b18a1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0ac133420 0x7ff0ac0709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:19.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.989+0000 7ff0b20a2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff0ac138dc0 0x7ff0ac071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:19.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.989+0000 7ff0b10a0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff0ac1337f0 0x7ff0ac070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:19.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.989+0000 7ff0b18a1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0ac133420 0x7ff0ac0709a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41590/0 (socket says 172.21.15.3:41590) 2023-10-03T10:17:19.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.989+0000 7ff0b18a1700 1 -- 172.21.15.3:0/3402077117 learned_addr learned my addr 172.21.15.3:0/3402077117 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:19.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.989+0000 7ff0b20a2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff0ac138dc0 0x7ff0ac071420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53572/0 (socket says 172.21.15.3:53572) 2023-10-03T10:17:19.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.989+0000 7ff0b18a1700 1 -- 172.21.15.3:0/3402077117 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff0ac138dc0 msgr2=0x7ff0ac071420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:19.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.989+0000 7ff0b18a1700 1 --2- 172.21.15.3:0/3402077117 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff0ac138dc0 0x7ff0ac071420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:19.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.989+0000 7ff0b18a1700 1 -- 172.21.15.3:0/3402077117 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff0ac1337f0 msgr2=0x7ff0ac070ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:19.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.989+0000 7ff0b18a1700 1 --2- 172.21.15.3:0/3402077117 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff0ac1337f0 0x7ff0ac070ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:19.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.989+0000 7ff0b18a1700 1 -- 172.21.15.3:0/3402077117 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a4009b30 con 0x7ff0ac133420 2023-10-03T10:17:19.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.989+0000 7ff0b20a2700 1 --2- 172.21.15.3:0/3402077117 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff0ac138dc0 0x7ff0ac071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.989+0000 7ff0b10a0700 1 --2- 172.21.15.3:0/3402077117 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff0ac1337f0 0x7ff0ac070ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.989+0000 7ff0b18a1700 1 --2- 172.21.15.3:0/3402077117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0ac133420 0x7ff0ac0709a0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7ff0a800c4e0 tx=0x7ff0a800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.989+0000 7ff0a27fc700 1 -- 172.21.15.3:0/3402077117 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a80052d0 con 0x7ff0ac133420 2023-10-03T10:17:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.990+0000 7ff0a27fc700 1 -- 172.21.15.3:0/3402077117 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a8005430 con 0x7ff0ac133420 2023-10-03T10:17:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.990+0000 7ff0b3b05700 1 -- 172.21.15.3:0/3402077117 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff09c0094a0 con 0x7ff0ac133420 2023-10-03T10:17:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.990+0000 7ff0a27fc700 1 -- 172.21.15.3:0/3402077117 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a80038c0 con 0x7ff0ac133420 2023-10-03T10:17:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.990+0000 7ff0b3b05700 1 -- 172.21.15.3:0/3402077117 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0ac0724f0 con 0x7ff0ac133420 2023-10-03T10:17:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.990+0000 7ff0a27fc700 1 -- 172.21.15.3:0/3402077117 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff0a8003a20 con 0x7ff0ac133420 2023-10-03T10:17:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.991+0000 7ff0a27fc700 1 -- 172.21.15.3:0/3402077117 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff0a803eab0 con 0x7ff0ac133420 2023-10-03T10:17:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.991+0000 7ff0b3b05700 1 -- 172.21.15.3:0/3402077117 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0ac09cbe0 con 0x7ff0ac133420 2023-10-03T10:17:19.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:19.996+0000 7ff0a27fc700 1 -- 172.21.15.3:0/3402077117 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff0a8018260 con 0x7ff0ac133420 2023-10-03T10:17:20.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.200+0000 7ff0b3b05700 1 -- 172.21.15.3:0/3402077117 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff0ac074ea0 con 0x7ff0ac133420 2023-10-03T10:17:20.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.200+0000 7ff0a27fc700 1 -- 172.21.15.3:0/3402077117 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff0a8017020 con 0x7ff0ac133420 2023-10-03T10:17:20.200 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:20.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.202+0000 7ff0b3b05700 1 -- 172.21.15.3:0/3402077117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0ac133420 msgr2=0x7ff0ac0709a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:20.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.202+0000 7ff0b3b05700 1 --2- 172.21.15.3:0/3402077117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0ac133420 0x7ff0ac0709a0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7ff0a800c4e0 tx=0x7ff0a800c7f0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.202+0000 7ff0b3b05700 1 -- 172.21.15.3:0/3402077117 shutdown_connections 2023-10-03T10:17:20.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.202+0000 7ff0b3b05700 1 --2- 172.21.15.3:0/3402077117 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff0ac138dc0 0x7ff0ac071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.202+0000 7ff0b3b05700 1 --2- 172.21.15.3:0/3402077117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0ac133420 0x7ff0ac0709a0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.202+0000 7ff0b3b05700 1 --2- 172.21.15.3:0/3402077117 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff0ac1337f0 0x7ff0ac070ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.202+0000 7ff0b3b05700 1 -- 172.21.15.3:0/3402077117 >> 172.21.15.3:0/3402077117 conn(0x7ff0ac0a4210 msgr2=0x7ff0ac131400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:20.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.202+0000 7ff0b3b05700 1 -- 172.21.15.3:0/3402077117 shutdown_connections 2023-10-03T10:17:20.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.202+0000 7ff0b3b05700 1 -- 172.21.15.3:0/3402077117 wait complete. 2023-10-03T10:17:20.209 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:20.216 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:20.309 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:20.347 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:20.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.350+0000 7f0c3bde3700 1 Processor -- start 2023-10-03T10:17:20.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.350+0000 7f0c3bde3700 1 -- start start 2023-10-03T10:17:20.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.351+0000 7f0c3bde3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c340a1510 0x7f0c34138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:20.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.351+0000 7f0c3bde3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c34139250 0x7f0c3413b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:20.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.351+0000 7f0c3bde3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c3409edd0 con 0x7f0c340a0a30 2023-10-03T10:17:20.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.351+0000 7f0c3bde3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c340a18e0 con 0x7f0c34139250 2023-10-03T10:17:20.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.351+0000 7f0c3bde3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3409e1f0 con 0x7f0c340a1510 2023-10-03T10:17:20.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.351+0000 7f0c3a380700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c34139250 0x7f0c3413b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:20.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.351+0000 7f0c3937e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c340a1510 0x7f0c34138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:20.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.351+0000 7f0c3a380700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c34139250 0x7f0c3413b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51084/0 (socket says 172.21.15.3:51084) 2023-10-03T10:17:20.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.351+0000 7f0c3a380700 1 -- 172.21.15.3:0/2327602125 learned_addr learned my addr 172.21.15.3:0/2327602125 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:20.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.351+0000 7f0c39b7f700 1 --1- 172.21.15.3:0/2327602125 >> v1:172.21.15.3:6789/0 conn(0x7f0c340a0a30 0x7f0c340a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55328/0 (socket says 172.21.15.3:55328) 2023-10-03T10:17:20.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.352+0000 7f0c3a380700 1 -- 172.21.15.3:0/2327602125 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c340a1510 msgr2=0x7f0c34138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:20.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.352+0000 7f0c3a380700 1 --2- 172.21.15.3:0/2327602125 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c340a1510 0x7f0c34138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.352+0000 7f0c3a380700 1 -- 172.21.15.3:0/2327602125 >> v1:172.21.15.3:6789/0 conn(0x7f0c340a0a30 legacy=0x7f0c340a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:20.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.352+0000 7f0c38b7d700 1 -- 172.21.15.3:0/2327602125 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2896145951 0 0) 0x7f0c3409edd0 con 0x7f0c340a0a30 2023-10-03T10:17:20.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.352+0000 7f0c3a380700 1 -- 172.21.15.3:0/2327602125 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3409dc50 con 0x7f0c34139250 2023-10-03T10:17:20.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.352+0000 7f0c3937e700 1 --2- 172.21.15.3:0/2327602125 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c340a1510 0x7f0c34138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:20.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.352+0000 7f0c3a380700 1 --2- 172.21.15.3:0/2327602125 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c34139250 0x7f0c3413b630 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009e80 tx=0x7f0c2c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=807be9680c43ff66 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:20.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.352+0000 7f0c38b7d700 1 -- 172.21.15.3:0/2327602125 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c00e070 con 0x7f0c34139250 2023-10-03T10:17:20.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.352+0000 7f0c38b7d700 1 -- 172.21.15.3:0/2327602125 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2c00b910 con 0x7f0c34139250 2023-10-03T10:17:20.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.352+0000 7f0c38b7d700 1 -- 172.21.15.3:0/2327602125 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c00bcc0 con 0x7f0c34139250 2023-10-03T10:17:20.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.353+0000 7f0c3bde3700 1 -- 172.21.15.3:0/2327602125 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c34139250 msgr2=0x7f0c3413b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:20.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.353+0000 7f0c3bde3700 1 --2- 172.21.15.3:0/2327602125 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c34139250 0x7f0c3413b630 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009e80 tx=0x7f0c2c00b330 comp rx=0 tx=0).stop 2023-10-03T10:17:20.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.353+0000 7f0c3bde3700 1 -- 172.21.15.3:0/2327602125 shutdown_connections 2023-10-03T10:17:20.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.353+0000 7f0c3bde3700 1 --2- 172.21.15.3:0/2327602125 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c34139250 0x7f0c3413b630 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.353+0000 7f0c3bde3700 1 --2- 172.21.15.3:0/2327602125 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c340a1510 0x7f0c34138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.353+0000 7f0c3bde3700 1 -- 172.21.15.3:0/2327602125 >> 172.21.15.3:0/2327602125 conn(0x7f0c340a6550 msgr2=0x7f0c340a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:20.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.353+0000 7f0c3bde3700 1 -- 172.21.15.3:0/2327602125 shutdown_connections 2023-10-03T10:17:20.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.353+0000 7f0c3bde3700 1 -- 172.21.15.3:0/2327602125 wait complete. 2023-10-03T10:17:20.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.354+0000 7f0c3bde3700 1 Processor -- start 2023-10-03T10:17:20.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.354+0000 7f0c3bde3700 1 -- start start 2023-10-03T10:17:20.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.355+0000 7f0c3bde3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c340a0a30 0x7f0c3411e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:20.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.355+0000 7f0c3bde3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c340a1510 0x7f0c3411ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:20.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.355+0000 7f0c3bde3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c34139250 0x7f0c3411f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:20.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.355+0000 7f0c3bde3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c340a0220 con 0x7f0c340a0a30 2023-10-03T10:17:20.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.355+0000 7f0c3bde3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c341204b0 con 0x7f0c34139250 2023-10-03T10:17:20.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.355+0000 7f0c3bde3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3409c9e0 con 0x7f0c340a1510 2023-10-03T10:17:20.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.355+0000 7f0c3937e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c340a1510 0x7f0c3411ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:20.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.355+0000 7f0c3937e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c340a1510 0x7f0c3411ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53590/0 (socket says 172.21.15.3:53590) 2023-10-03T10:17:20.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.355+0000 7f0c3937e700 1 -- 172.21.15.3:0/2295310134 learned_addr learned my addr 172.21.15.3:0/2295310134 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:20.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.355+0000 7f0c3a380700 1 --2- 172.21.15.3:0/2295310134 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c34139250 0x7f0c3411f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:20.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.355+0000 7f0c39b7f700 1 --2- 172.21.15.3:0/2295310134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c340a0a30 0x7f0c3411e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:20.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.355+0000 7f0c3937e700 1 -- 172.21.15.3:0/2295310134 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c34139250 msgr2=0x7f0c3411f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:20.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.355+0000 7f0c3937e700 1 --2- 172.21.15.3:0/2295310134 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c34139250 0x7f0c3411f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.355+0000 7f0c3937e700 1 -- 172.21.15.3:0/2295310134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c340a0a30 msgr2=0x7f0c3411e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:20.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.355+0000 7f0c3937e700 1 --2- 172.21.15.3:0/2295310134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c340a0a30 0x7f0c3411e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.355+0000 7f0c3937e700 1 -- 172.21.15.3:0/2295310134 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c240094a0 con 0x7f0c340a1510 2023-10-03T10:17:20.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.355+0000 7f0c39b7f700 1 --2- 172.21.15.3:0/2295310134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c340a0a30 0x7f0c3411e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:17:20.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.355+0000 7f0c3a380700 1 --2- 172.21.15.3:0/2295310134 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c34139250 0x7f0c3411f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:20.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.356+0000 7f0c3937e700 1 --2- 172.21.15.3:0/2295310134 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c340a1510 0x7f0c3411ecd0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0c2400ebe0 tx=0x7f0c2400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:20.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.356+0000 7f0c2a7fc700 1 -- 172.21.15.3:0/2295310134 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c24009800 con 0x7f0c340a1510 2023-10-03T10:17:20.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.356+0000 7f0c2a7fc700 1 -- 172.21.15.3:0/2295310134 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2400c910 con 0x7f0c340a1510 2023-10-03T10:17:20.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.356+0000 7f0c3bde3700 1 -- 172.21.15.3:0/2295310134 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c2c009b30 con 0x7f0c340a1510 2023-10-03T10:17:20.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.356+0000 7f0c2a7fc700 1 -- 172.21.15.3:0/2295310134 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2400cc80 con 0x7f0c340a1510 2023-10-03T10:17:20.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.356+0000 7f0c3bde3700 1 -- 172.21.15.3:0/2295310134 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c34122a40 con 0x7f0c340a1510 2023-10-03T10:17:20.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.357+0000 7f0c2a7fc700 1 -- 172.21.15.3:0/2295310134 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0c24013470 con 0x7f0c340a1510 2023-10-03T10:17:20.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.357+0000 7f0c3bde3700 1 -- 172.21.15.3:0/2295310134 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c34074ea0 con 0x7f0c340a1510 2023-10-03T10:17:20.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.357+0000 7f0c2a7fc700 1 -- 172.21.15.3:0/2295310134 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0c2403fc40 con 0x7f0c340a1510 2023-10-03T10:17:20.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.362+0000 7f0c2a7fc700 1 -- 172.21.15.3:0/2295310134 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0c24013720 con 0x7f0c340a1510 2023-10-03T10:17:20.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.451+0000 7f668ea4d700 1 Processor -- start 2023-10-03T10:17:20.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.451+0000 7f668ea4d700 1 -- start start 2023-10-03T10:17:20.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.452+0000 7f668ea4d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f66880a0a30 0x7f66880a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:20.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.452+0000 7f668ea4d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6688138d80 0x7f668813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:20.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.452+0000 7f668ea4d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66880a2a70 con 0x7f668809d630 2023-10-03T10:17:20.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.452+0000 7f668ea4d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66880a01d0 con 0x7f66880a0a30 2023-10-03T10:17:20.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.452+0000 7f668ea4d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f668809e2b0 con 0x7f6688138d80 2023-10-03T10:17:20.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.452+0000 7f6687fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f66880a0a30 0x7f66880a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:20.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.452+0000 7f6687fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f66880a0a30 0x7f66880a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41596/0 (socket says 172.21.15.3:41596) 2023-10-03T10:17:20.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.452+0000 7f6687fff700 1 -- 172.21.15.3:0/2614209934 learned_addr learned my addr 172.21.15.3:0/2614209934 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:20.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.452+0000 7f668cfea700 1 --2- 172.21.15.3:0/2614209934 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6688138d80 0x7f668813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:20.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.453+0000 7f668cfea700 1 -- 172.21.15.3:0/2614209934 >> v2:172.21.15.3:3300/0 conn(0x7f66880a0a30 msgr2=0x7f66880a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.453+0000 7f668cfea700 1 --2- 172.21.15.3:0/2614209934 >> v2:172.21.15.3:3300/0 conn(0x7f66880a0a30 0x7f66880a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.453+0000 7f668cfea700 1 -- 172.21.15.3:0/2614209934 >> v1:172.21.15.3:6789/0 conn(0x7f668809d630 legacy=0x7f66880a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.453+0000 7f668cfea700 1 -- 172.21.15.3:0/2614209934 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668809dc50 con 0x7f6688138d80 2023-10-03T10:17:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.453+0000 7f6687fff700 1 --2- 172.21.15.3:0/2614209934 >> v2:172.21.15.3:3300/0 conn(0x7f66880a0a30 0x7f66880a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.453+0000 7f668cfea700 1 --2- 172.21.15.3:0/2614209934 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6688138d80 0x7f668813b160 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6680009d90 tx=0x7f668000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2f073252470f29d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.453+0000 7f6686ffd700 1 -- 172.21.15.3:0/2614209934 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668000e070 con 0x7f6688138d80 2023-10-03T10:17:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.453+0000 7f6686ffd700 1 -- 172.21.15.3:0/2614209934 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668000bb20 con 0x7f6688138d80 2023-10-03T10:17:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.453+0000 7f6686ffd700 1 -- 172.21.15.3:0/2614209934 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6680012430 con 0x7f6688138d80 2023-10-03T10:17:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.454+0000 7f668ea4d700 1 -- 172.21.15.3:0/2614209934 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6688138d80 msgr2=0x7f668813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.454+0000 7f668ea4d700 1 --2- 172.21.15.3:0/2614209934 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6688138d80 0x7f668813b160 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6680009d90 tx=0x7f668000b330 comp rx=0 tx=0).stop 2023-10-03T10:17:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.454+0000 7f668ea4d700 1 -- 172.21.15.3:0/2614209934 shutdown_connections 2023-10-03T10:17:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.454+0000 7f668ea4d700 1 --2- 172.21.15.3:0/2614209934 >> v2:172.21.15.3:3300/0 conn(0x7f66880a0a30 0x7f66880a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.454+0000 7f668ea4d700 1 --2- 172.21.15.3:0/2614209934 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6688138d80 0x7f668813b160 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.454+0000 7f668ea4d700 1 -- 172.21.15.3:0/2614209934 >> 172.21.15.3:0/2614209934 conn(0x7f66880a3050 msgr2=0x7f66880a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:20.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.454+0000 7f668ea4d700 1 -- 172.21.15.3:0/2614209934 shutdown_connections 2023-10-03T10:17:20.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.454+0000 7f668ea4d700 1 -- 172.21.15.3:0/2614209934 wait complete. 2023-10-03T10:17:20.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.456+0000 7f668ea4d700 1 Processor -- start 2023-10-03T10:17:20.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.456+0000 7f668ea4d700 1 -- start start 2023-10-03T10:17:20.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.456+0000 7f668ea4d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f668809d630 0x7f66881339e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:20.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.456+0000 7f668ea4d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66880a0a30 0x7f6688133f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:20.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.456+0000 7f668ea4d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6688138d80 0x7f6688134460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:20.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.457+0000 7f668ea4d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6688136ed0 con 0x7f668809d630 2023-10-03T10:17:20.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.457+0000 7f668ea4d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6688138850 con 0x7f66880a0a30 2023-10-03T10:17:20.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.457+0000 7f668ea4d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6688138160 con 0x7f6688138d80 2023-10-03T10:17:20.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.457+0000 7f6687fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f668809d630 0x7f66881339e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:20.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.457+0000 7f6687fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f668809d630 0x7f66881339e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41606/0 (socket says 172.21.15.3:41606) 2023-10-03T10:17:20.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.457+0000 7f6687fff700 1 -- 172.21.15.3:0/282279684 learned_addr learned my addr 172.21.15.3:0/282279684 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:20.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.457+0000 7f668cfea700 1 --2- 172.21.15.3:0/282279684 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6688138d80 0x7f6688134460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:20.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.457+0000 7f66877fe700 1 --2- 172.21.15.3:0/282279684 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66880a0a30 0x7f6688133f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:20.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.457+0000 7f668cfea700 1 -- 172.21.15.3:0/282279684 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66880a0a30 msgr2=0x7f6688133f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:20.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.457+0000 7f668cfea700 1 --2- 172.21.15.3:0/282279684 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66880a0a30 0x7f6688133f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.457+0000 7f668cfea700 1 -- 172.21.15.3:0/282279684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f668809d630 msgr2=0x7f66881339e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:20.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.457+0000 7f668cfea700 1 --2- 172.21.15.3:0/282279684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f668809d630 0x7f66881339e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.457+0000 7f668cfea700 1 -- 172.21.15.3:0/282279684 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6678009550 con 0x7f6688138d80 2023-10-03T10:17:20.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.457+0000 7f6687fff700 1 --2- 172.21.15.3:0/282279684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f668809d630 0x7f66881339e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:20.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.457+0000 7f668cfea700 1 --2- 172.21.15.3:0/282279684 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6688138d80 0x7f6688134460 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6680016000 tx=0x7f6680004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:20.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.458+0000 7f6684ff9700 1 -- 172.21.15.3:0/282279684 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6680012430 con 0x7f6688138d80 2023-10-03T10:17:20.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.458+0000 7f6684ff9700 1 -- 172.21.15.3:0/282279684 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6680020070 con 0x7f6688138d80 2023-10-03T10:17:20.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.458+0000 7f668ea4d700 1 -- 172.21.15.3:0/282279684 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6680009a70 con 0x7f6688138d80 2023-10-03T10:17:20.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.458+0000 7f6684ff9700 1 -- 172.21.15.3:0/282279684 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6680004280 con 0x7f6688138d80 2023-10-03T10:17:20.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.458+0000 7f668ea4d700 1 -- 172.21.15.3:0/282279684 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6688138390 con 0x7f6688138d80 2023-10-03T10:17:20.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.458+0000 7f6684ff9700 1 -- 172.21.15.3:0/282279684 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f66800043e0 con 0x7f6688138d80 2023-10-03T10:17:20.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.459+0000 7f6684ff9700 1 -- 172.21.15.3:0/282279684 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6680022070 con 0x7f6688138d80 2023-10-03T10:17:20.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.459+0000 7f668ea4d700 1 -- 172.21.15.3:0/282279684 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66880a26f0 con 0x7f6688138d80 2023-10-03T10:17:20.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.464+0000 7f6684ff9700 1 -- 172.21.15.3:0/282279684 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f668001c400 con 0x7f6688138d80 2023-10-03T10:17:20.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.492+0000 7fc25490a700 1 Processor -- start 2023-10-03T10:17:20.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.492+0000 7fc25490a700 1 -- start start 2023-10-03T10:17:20.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.493+0000 7fc25490a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc2501316c0 0x7fc250131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:20.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.493+0000 7fc25490a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc25006ae80 0x7fc250139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:20.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.493+0000 7fc25490a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc25009e130 con 0x7fc25009d9f0 2023-10-03T10:17:20.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.493+0000 7fc25490a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2501312b0 con 0x7fc25006ae80 2023-10-03T10:17:20.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.493+0000 7fc25490a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc25009f6e0 con 0x7fc2501316c0 2023-10-03T10:17:20.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.494+0000 7fc24ed9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc25006ae80 0x7fc250139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:20.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.494+0000 7fc24e59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc2501316c0 0x7fc250131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:20.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.494+0000 7fc24ed9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc25006ae80 0x7fc250139550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51104/0 (socket says 172.21.15.3:51104) 2023-10-03T10:17:20.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.494+0000 7fc24e59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc2501316c0 0x7fc250131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53620/0 (socket says 172.21.15.3:53620) 2023-10-03T10:17:20.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.494+0000 7fc24ed9d700 1 -- 172.21.15.3:0/4020415552 learned_addr learned my addr 172.21.15.3:0/4020415552 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:20.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.494+0000 7fc24ed9d700 1 -- 172.21.15.3:0/4020415552 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc2501316c0 msgr2=0x7fc250131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:20.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.494+0000 7fc24ed9d700 1 --2- 172.21.15.3:0/4020415552 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc2501316c0 0x7fc250131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.494+0000 7fc24ed9d700 1 -- 172.21.15.3:0/4020415552 >> v1:172.21.15.3:6789/0 conn(0x7fc25009d9f0 legacy=0x7fc25006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:20.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.494+0000 7fc24ed9d700 1 -- 172.21.15.3:0/4020415552 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc25009e760 con 0x7fc25006ae80 2023-10-03T10:17:20.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.494+0000 7fc24e59c700 1 --2- 172.21.15.3:0/4020415552 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc2501316c0 0x7fc250131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:17:20.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.494+0000 7fc24ed9d700 1 --2- 172.21.15.3:0/4020415552 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc25006ae80 0x7fc250139550 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc248009e80 tx=0x7fc24800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32b58bdc8f0cfa8d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:20.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.494+0000 7fc24d59a700 1 -- 172.21.15.3:0/4020415552 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24800e070 con 0x7fc25006ae80 2023-10-03T10:17:20.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.494+0000 7fc24d59a700 1 -- 172.21.15.3:0/4020415552 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24800b9c0 con 0x7fc25006ae80 2023-10-03T10:17:20.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.494+0000 7fc24d59a700 1 -- 172.21.15.3:0/4020415552 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24800bd70 con 0x7fc25006ae80 2023-10-03T10:17:20.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.495+0000 7fc25490a700 1 -- 172.21.15.3:0/4020415552 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc25006ae80 msgr2=0x7fc250139550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:20.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.495+0000 7fc25490a700 1 --2- 172.21.15.3:0/4020415552 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc25006ae80 0x7fc250139550 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc248009e80 tx=0x7fc24800b330 comp rx=0 tx=0).stop 2023-10-03T10:17:20.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.495+0000 7fc25490a700 1 -- 172.21.15.3:0/4020415552 shutdown_connections 2023-10-03T10:17:20.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.495+0000 7fc25490a700 1 --2- 172.21.15.3:0/4020415552 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc25006ae80 0x7fc250139550 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.495+0000 7fc25490a700 1 --2- 172.21.15.3:0/4020415552 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc2501316c0 0x7fc250131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.495+0000 7fc25490a700 1 -- 172.21.15.3:0/4020415552 >> 172.21.15.3:0/4020415552 conn(0x7fc250066f50 msgr2=0x7fc25009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:20.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.495+0000 7fc25490a700 1 -- 172.21.15.3:0/4020415552 shutdown_connections 2023-10-03T10:17:20.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.495+0000 7fc25490a700 1 -- 172.21.15.3:0/4020415552 wait complete. 2023-10-03T10:17:20.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.496+0000 7fc25490a700 1 Processor -- start 2023-10-03T10:17:20.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.496+0000 7fc25490a700 1 -- start start 2023-10-03T10:17:20.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.497+0000 7fc25490a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc25006ae80 0x7fc250070d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:20.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.497+0000 7fc25490a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc25009d9f0 0x7fc2500712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:20.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.497+0000 7fc25490a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc2501316c0 0x7fc250071800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:20.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.497+0000 7fc25490a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25011bbe0 con 0x7fc25009d9f0 2023-10-03T10:17:20.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.497+0000 7fc25490a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc250072720 con 0x7fc2501316c0 2023-10-03T10:17:20.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.497+0000 7fc25490a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc25011ad90 con 0x7fc25006ae80 2023-10-03T10:17:20.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.497+0000 7fc24e59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc25006ae80 0x7fc250070d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:20.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.497+0000 7fc24e59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc25006ae80 0x7fc250070d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53622/0 (socket says 172.21.15.3:53622) 2023-10-03T10:17:20.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.497+0000 7fc24ed9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc2501316c0 0x7fc250071800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:20.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.497+0000 7fc24e59c700 1 -- 172.21.15.3:0/3230805570 learned_addr learned my addr 172.21.15.3:0/3230805570 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:20.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.497+0000 7fc24dd9b700 1 --2- 172.21.15.3:0/3230805570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc25009d9f0 0x7fc2500712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:20.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.497+0000 7fc24ed9d700 1 -- 172.21.15.3:0/3230805570 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc25006ae80 msgr2=0x7fc250070d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:20.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.497+0000 7fc24ed9d700 1 --2- 172.21.15.3:0/3230805570 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc25006ae80 0x7fc250070d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.497+0000 7fc24ed9d700 1 -- 172.21.15.3:0/3230805570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc25009d9f0 msgr2=0x7fc2500712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:20.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.497+0000 7fc24ed9d700 1 --2- 172.21.15.3:0/3230805570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc25009d9f0 0x7fc2500712c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.497+0000 7fc24ed9d700 1 -- 172.21.15.3:0/3230805570 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc248009b30 con 0x7fc2501316c0 2023-10-03T10:17:20.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.498+0000 7fc24e59c700 1 --2- 172.21.15.3:0/3230805570 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc25006ae80 0x7fc250070d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:20.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.498+0000 7fc24ed9d700 1 --2- 172.21.15.3:0/3230805570 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc2501316c0 0x7fc250071800 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc248015000 tx=0x7fc248004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:20.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.498+0000 7fc23effd700 1 -- 172.21.15.3:0/3230805570 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc248004250 con 0x7fc2501316c0 2023-10-03T10:17:20.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.498+0000 7fc23effd700 1 -- 172.21.15.3:0/3230805570 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2480043f0 con 0x7fc2501316c0 2023-10-03T10:17:20.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.498+0000 7fc23effd700 1 -- 172.21.15.3:0/3230805570 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2480047a0 con 0x7fc2501316c0 2023-10-03T10:17:20.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.498+0000 7fc25490a700 1 -- 172.21.15.3:0/3230805570 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc250119630 con 0x7fc2501316c0 2023-10-03T10:17:20.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.498+0000 7fc25490a700 1 -- 172.21.15.3:0/3230805570 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2500a37c0 con 0x7fc2501316c0 2023-10-03T10:17:20.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.499+0000 7fc23effd700 1 -- 172.21.15.3:0/3230805570 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc24801c440 con 0x7fc2501316c0 2023-10-03T10:17:20.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.499+0000 7fc23effd700 1 -- 172.21.15.3:0/3230805570 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc248040af0 con 0x7fc2501316c0 2023-10-03T10:17:20.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.499+0000 7fc25490a700 1 -- 172.21.15.3:0/3230805570 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc25011b850 con 0x7fc2501316c0 2023-10-03T10:17:20.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.504+0000 7fc23effd700 1 -- 172.21.15.3:0/3230805570 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc248004900 con 0x7fc2501316c0 2023-10-03T10:17:20.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.593+0000 7f0c3bde3700 1 -- 172.21.15.3:0/2295310134 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0c340a28e0 con 0x7f0c340a1510 2023-10-03T10:17:20.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.593+0000 7f0c2a7fc700 1 -- 172.21.15.3:0/2295310134 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0c24017020 con 0x7f0c340a1510 2023-10-03T10:17:20.594 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:20.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.596+0000 7f0c3bde3700 1 -- 172.21.15.3:0/2295310134 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c340a1510 msgr2=0x7f0c3411ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:20.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.596+0000 7f0c3bde3700 1 --2- 172.21.15.3:0/2295310134 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c340a1510 0x7f0c3411ecd0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0c2400ebe0 tx=0x7f0c2400eef0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.596+0000 7f0c3bde3700 1 -- 172.21.15.3:0/2295310134 shutdown_connections 2023-10-03T10:17:20.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.596+0000 7f0c3bde3700 1 --2- 172.21.15.3:0/2295310134 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c34139250 0x7f0c3411f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.596+0000 7f0c3bde3700 1 --2- 172.21.15.3:0/2295310134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c340a0a30 0x7f0c3411e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.596+0000 7f0c3bde3700 1 --2- 172.21.15.3:0/2295310134 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c340a1510 0x7f0c3411ecd0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.596+0000 7f0c3bde3700 1 -- 172.21.15.3:0/2295310134 >> 172.21.15.3:0/2295310134 conn(0x7f0c340a6550 msgr2=0x7f0c340a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:20.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.596+0000 7f0c3bde3700 1 -- 172.21.15.3:0/2295310134 shutdown_connections 2023-10-03T10:17:20.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.596+0000 7f0c3bde3700 1 -- 172.21.15.3:0/2295310134 wait complete. 2023-10-03T10:17:20.610 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:20.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.671+0000 7f668ea4d700 1 -- 172.21.15.3:0/282279684 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6688137040 con 0x7f6688138d80 2023-10-03T10:17:20.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.672+0000 7f6684ff9700 1 -- 172.21.15.3:0/282279684 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6680026760 con 0x7f6688138d80 2023-10-03T10:17:20.672 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:20.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.674+0000 7f668ea4d700 1 -- 172.21.15.3:0/282279684 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6688138d80 msgr2=0x7f6688134460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:20.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.674+0000 7f668ea4d700 1 --2- 172.21.15.3:0/282279684 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6688138d80 0x7f6688134460 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6680016000 tx=0x7f6680004ad0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.674+0000 7f668ea4d700 1 -- 172.21.15.3:0/282279684 shutdown_connections 2023-10-03T10:17:20.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.674+0000 7f668ea4d700 1 --2- 172.21.15.3:0/282279684 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6688138d80 0x7f6688134460 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.674+0000 7f668ea4d700 1 --2- 172.21.15.3:0/282279684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f668809d630 0x7f66881339e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.674+0000 7f668ea4d700 1 --2- 172.21.15.3:0/282279684 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66880a0a30 0x7f6688133f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.674+0000 7f668ea4d700 1 -- 172.21.15.3:0/282279684 >> 172.21.15.3:0/282279684 conn(0x7f66880a3050 msgr2=0x7f668809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:20.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.674+0000 7f668ea4d700 1 -- 172.21.15.3:0/282279684 shutdown_connections 2023-10-03T10:17:20.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.674+0000 7f668ea4d700 1 -- 172.21.15.3:0/282279684 wait complete. 2023-10-03T10:17:20.688 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:20.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.727+0000 7fc25490a700 1 -- 172.21.15.3:0/3230805570 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc250074ea0 con 0x7fc2501316c0 2023-10-03T10:17:20.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.728+0000 7fc23effd700 1 -- 172.21.15.3:0/3230805570 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc248020020 con 0x7fc2501316c0 2023-10-03T10:17:20.728 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:20.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.730+0000 7fc25490a700 1 -- 172.21.15.3:0/3230805570 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc2501316c0 msgr2=0x7fc250071800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:20.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.730+0000 7fc25490a700 1 --2- 172.21.15.3:0/3230805570 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc2501316c0 0x7fc250071800 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc248015000 tx=0x7fc248004ad0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.730+0000 7fc25490a700 1 -- 172.21.15.3:0/3230805570 shutdown_connections 2023-10-03T10:17:20.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.730+0000 7fc25490a700 1 --2- 172.21.15.3:0/3230805570 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc2501316c0 0x7fc250071800 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.730+0000 7fc25490a700 1 --2- 172.21.15.3:0/3230805570 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc25006ae80 0x7fc250070d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.730+0000 7fc25490a700 1 --2- 172.21.15.3:0/3230805570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc25009d9f0 0x7fc2500712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:20.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.730+0000 7fc25490a700 1 -- 172.21.15.3:0/3230805570 >> 172.21.15.3:0/3230805570 conn(0x7fc250066f50 msgr2=0x7fc2500a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:20.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.730+0000 7fc25490a700 1 -- 172.21.15.3:0/3230805570 shutdown_connections 2023-10-03T10:17:20.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:20.730+0000 7fc25490a700 1 -- 172.21.15.3:0/3230805570 wait complete. 2023-10-03T10:17:20.744 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:21.217 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:21.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.368+0000 7ff0b6d1e700 1 Processor -- start 2023-10-03T10:17:21.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.368+0000 7ff0b6d1e700 1 -- start start 2023-10-03T10:17:21.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.369+0000 7ff0b6d1e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff0b01316c0 0x7ff0b0131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:21.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.369+0000 7ff0b6d1e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff0b009d9f0 0x7ff0b006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:21.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.369+0000 7ff0b6d1e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0b009e130 con 0x7ff0b006acb0 2023-10-03T10:17:21.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.369+0000 7ff0b6d1e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b01312b0 con 0x7ff0b009d9f0 2023-10-03T10:17:21.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.369+0000 7ff0b6d1e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b009f6e0 con 0x7ff0b01316c0 2023-10-03T10:17:21.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.369+0000 7ff0b4aba700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff0b01316c0 0x7ff0b0131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:21.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.369+0000 7ff0b4aba700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff0b01316c0 0x7ff0b0131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53636/0 (socket says 172.21.15.3:53636) 2023-10-03T10:17:21.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.369+0000 7ff0b4aba700 1 -- 172.21.15.3:0/3817123759 learned_addr learned my addr 172.21.15.3:0/3817123759 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:21.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.369+0000 7ff0affff700 1 --2- 172.21.15.3:0/3817123759 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff0b009d9f0 0x7ff0b006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:21.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.370+0000 7ff0b4aba700 1 -- 172.21.15.3:0/3817123759 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff0b009d9f0 msgr2=0x7ff0b006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:21.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.370+0000 7ff0b4aba700 1 --2- 172.21.15.3:0/3817123759 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff0b009d9f0 0x7ff0b006a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.370+0000 7ff0b4aba700 1 -- 172.21.15.3:0/3817123759 >> v1:172.21.15.3:6789/0 conn(0x7ff0b006acb0 legacy=0x7ff0b0138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:21.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.370+0000 7ff0b4aba700 1 -- 172.21.15.3:0/3817123759 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b009e760 con 0x7ff0b01316c0 2023-10-03T10:17:21.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.370+0000 7ff0b4aba700 1 --2- 172.21.15.3:0/3817123759 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff0b01316c0 0x7ff0b0131a90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0009920 tx=0x7ff0a0009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd74d83fce72b626 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:21.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.370+0000 7ff0af7fe700 1 -- 172.21.15.3:0/3817123759 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0011830 con 0x7ff0b01316c0 2023-10-03T10:17:21.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.370+0000 7ff0af7fe700 1 -- 172.21.15.3:0/3817123759 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a0011990 con 0x7ff0b01316c0 2023-10-03T10:17:21.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.370+0000 7ff0af7fe700 1 -- 172.21.15.3:0/3817123759 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0011d20 con 0x7ff0b01316c0 2023-10-03T10:17:21.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.371+0000 7ff0b6d1e700 1 -- 172.21.15.3:0/3817123759 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff0b01316c0 msgr2=0x7ff0b0131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:21.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.371+0000 7ff0b6d1e700 1 --2- 172.21.15.3:0/3817123759 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff0b01316c0 0x7ff0b0131a90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0009920 tx=0x7ff0a0009c30 comp rx=0 tx=0).stop 2023-10-03T10:17:21.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.371+0000 7ff0b6d1e700 1 -- 172.21.15.3:0/3817123759 shutdown_connections 2023-10-03T10:17:21.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.371+0000 7ff0b6d1e700 1 --2- 172.21.15.3:0/3817123759 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff0b01316c0 0x7ff0b0131a90 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.371+0000 7ff0b6d1e700 1 --2- 172.21.15.3:0/3817123759 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff0b009d9f0 0x7ff0b006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.371+0000 7ff0b6d1e700 1 -- 172.21.15.3:0/3817123759 >> 172.21.15.3:0/3817123759 conn(0x7ff0b0066f50 msgr2=0x7ff0b009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:21.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.371+0000 7ff0b6d1e700 1 -- 172.21.15.3:0/3817123759 shutdown_connections 2023-10-03T10:17:21.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.371+0000 7ff0b6d1e700 1 -- 172.21.15.3:0/3817123759 wait complete. 2023-10-03T10:17:21.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.372+0000 7ff0b6d1e700 1 Processor -- start 2023-10-03T10:17:21.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.372+0000 7ff0b6d1e700 1 -- start start 2023-10-03T10:17:21.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.372+0000 7ff0b6d1e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0b006acb0 0x7ff0b009a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:21.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.372+0000 7ff0b6d1e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff0b009d9f0 0x7ff0b009a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:21.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.372+0000 7ff0b6d1e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff0b01316c0 0x7ff0b009ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:21.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.372+0000 7ff0b6d1e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b012b900 con 0x7ff0b006acb0 2023-10-03T10:17:21.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.372+0000 7ff0b6d1e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b0120040 con 0x7ff0b01316c0 2023-10-03T10:17:21.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.372+0000 7ff0b6d1e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b012b0f0 con 0x7ff0b009d9f0 2023-10-03T10:17:21.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.373+0000 7ff0b4aba700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0b006acb0 0x7ff0b009a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:21.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.373+0000 7ff0affff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff0b009d9f0 0x7ff0b009a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:21.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.373+0000 7ff0b52bb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff0b01316c0 0x7ff0b009ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:21.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.373+0000 7ff0b4aba700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0b006acb0 0x7ff0b009a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41636/0 (socket says 172.21.15.3:41636) 2023-10-03T10:17:21.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.373+0000 7ff0b4aba700 1 -- 172.21.15.3:0/3624579184 learned_addr learned my addr 172.21.15.3:0/3624579184 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:21.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.373+0000 7ff0affff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff0b009d9f0 0x7ff0b009a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53648/0 (socket says 172.21.15.3:53648) 2023-10-03T10:17:21.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.373+0000 7ff0b52bb700 1 -- 172.21.15.3:0/3624579184 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff0b009d9f0 msgr2=0x7ff0b009a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:21.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.373+0000 7ff0b52bb700 1 --2- 172.21.15.3:0/3624579184 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff0b009d9f0 0x7ff0b009a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.373+0000 7ff0b52bb700 1 -- 172.21.15.3:0/3624579184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0b006acb0 msgr2=0x7ff0b009a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:21.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.373+0000 7ff0b52bb700 1 --2- 172.21.15.3:0/3624579184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0b006acb0 0x7ff0b009a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.373+0000 7ff0b52bb700 1 -- 172.21.15.3:0/3624579184 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a0009550 con 0x7ff0b01316c0 2023-10-03T10:17:21.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.373+0000 7ff0affff700 1 --2- 172.21.15.3:0/3624579184 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff0b009d9f0 0x7ff0b009a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:21.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.373+0000 7ff0b4aba700 1 --2- 172.21.15.3:0/3624579184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0b006acb0 0x7ff0b009a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:21.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.373+0000 7ff0b52bb700 1 --2- 172.21.15.3:0/3624579184 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff0b01316c0 0x7ff0b009ae50 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff0a800cb90 tx=0x7ff0a800cea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:21.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.373+0000 7ff0ad7fa700 1 -- 172.21.15.3:0/3624579184 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8003c00 con 0x7ff0b01316c0 2023-10-03T10:17:21.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.373+0000 7ff0ad7fa700 1 -- 172.21.15.3:0/3624579184 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a8003d60 con 0x7ff0b01316c0 2023-10-03T10:17:21.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.373+0000 7ff0b6d1e700 1 -- 172.21.15.3:0/3624579184 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b009bf30 con 0x7ff0b01316c0 2023-10-03T10:17:21.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.373+0000 7ff0ad7fa700 1 -- 172.21.15.3:0/3624579184 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a800aa80 con 0x7ff0b01316c0 2023-10-03T10:17:21.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.373+0000 7ff0b6d1e700 1 -- 172.21.15.3:0/3624579184 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b01207d0 con 0x7ff0b01316c0 2023-10-03T10:17:21.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.374+0000 7ff0ad7fa700 1 -- 172.21.15.3:0/3624579184 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff0a800abe0 con 0x7ff0b01316c0 2023-10-03T10:17:21.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.374+0000 7ff0ad7fa700 1 -- 172.21.15.3:0/3624579184 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff0a803f450 con 0x7ff0b01316c0 2023-10-03T10:17:21.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.374+0000 7ff0b6d1e700 1 -- 172.21.15.3:0/3624579184 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b009e130 con 0x7ff0b01316c0 2023-10-03T10:17:21.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.379+0000 7ff0ad7fa700 1 -- 172.21.15.3:0/3624579184 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff0b009e130 con 0x7ff0b01316c0 2023-10-03T10:17:21.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.585+0000 7ff0b6d1e700 1 -- 172.21.15.3:0/3624579184 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff0b009e130 con 0x7ff0b01316c0 2023-10-03T10:17:21.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.585+0000 7ff0ad7fa700 1 -- 172.21.15.3:0/3624579184 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff0b009e130 con 0x7ff0b01316c0 2023-10-03T10:17:21.586 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:21.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.587+0000 7ff0b6d1e700 1 -- 172.21.15.3:0/3624579184 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff0b01316c0 msgr2=0x7ff0b009ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:21.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.588+0000 7ff0b6d1e700 1 --2- 172.21.15.3:0/3624579184 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff0b01316c0 0x7ff0b009ae50 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff0a800cb90 tx=0x7ff0a800cea0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.588+0000 7ff0b6d1e700 1 -- 172.21.15.3:0/3624579184 shutdown_connections 2023-10-03T10:17:21.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.588+0000 7ff0b6d1e700 1 --2- 172.21.15.3:0/3624579184 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff0b01316c0 0x7ff0b009ae50 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.588+0000 7ff0b6d1e700 1 --2- 172.21.15.3:0/3624579184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0b006acb0 0x7ff0b009a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.588+0000 7ff0b6d1e700 1 --2- 172.21.15.3:0/3624579184 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff0b009d9f0 0x7ff0b009a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.588+0000 7ff0b6d1e700 1 -- 172.21.15.3:0/3624579184 >> 172.21.15.3:0/3624579184 conn(0x7ff0b0066f50 msgr2=0x7ff0b00a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:21.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.588+0000 7ff0b6d1e700 1 -- 172.21.15.3:0/3624579184 shutdown_connections 2023-10-03T10:17:21.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.588+0000 7ff0b6d1e700 1 -- 172.21.15.3:0/3624579184 wait complete. 2023-10-03T10:17:21.602 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:21.610 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:21.689 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:21.744 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:21.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.752+0000 7f76c0bbc700 1 Processor -- start 2023-10-03T10:17:21.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.752+0000 7f76c0bbc700 1 -- start start 2023-10-03T10:17:21.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.753+0000 7f76c0bbc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76bc09d6f0 0x7f76bc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:21.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.753+0000 7f76c0bbc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f76bc09e000 0x7f76bc130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:21.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.753+0000 7f76c0bbc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76bc09e410 0x7f76bc130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:21.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.753+0000 7f76c0bbc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76bc09f520 con 0x7f76bc09e000 2023-10-03T10:17:21.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.753+0000 7f76c0bbc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc09f130 con 0x7f76bc09e410 2023-10-03T10:17:21.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.753+0000 7f76c0bbc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc09e930 con 0x7f76bc09d6f0 2023-10-03T10:17:21.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.753+0000 7f76ba59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76bc09d6f0 0x7f76bc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:21.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.753+0000 7f76bad9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76bc09e410 0x7f76bc130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:21.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.753+0000 7f76ba59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76bc09d6f0 0x7f76bc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53650/0 (socket says 172.21.15.3:53650) 2023-10-03T10:17:21.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.753+0000 7f76ba59c700 1 -- 172.21.15.3:0/1384827482 learned_addr learned my addr 172.21.15.3:0/1384827482 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:21.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.754+0000 7f76b9d9b700 1 --2- 172.21.15.3:0/1384827482 >> v2:172.21.15.3:3300/0 conn(0x7f76bc09e000 0x7f76bc130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:21.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.754+0000 7f76bad9d700 1 -- 172.21.15.3:0/1384827482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76bc09d6f0 msgr2=0x7f76bc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:21.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.754+0000 7f76bad9d700 1 --2- 172.21.15.3:0/1384827482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76bc09d6f0 0x7f76bc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.754+0000 7f76bad9d700 1 -- 172.21.15.3:0/1384827482 >> v2:172.21.15.3:3300/0 conn(0x7f76bc09e000 msgr2=0x7f76bc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:21.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.754+0000 7f76bad9d700 1 --2- 172.21.15.3:0/1384827482 >> v2:172.21.15.3:3300/0 conn(0x7f76bc09e000 0x7f76bc130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.754+0000 7f76bad9d700 1 -- 172.21.15.3:0/1384827482 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76bc09b2d0 con 0x7f76bc09e410 2023-10-03T10:17:21.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.754+0000 7f76ba59c700 1 --2- 172.21.15.3:0/1384827482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76bc09d6f0 0x7f76bc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:21.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.754+0000 7f76bad9d700 1 --2- 172.21.15.3:0/1384827482 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76bc09e410 0x7f76bc130ba0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009d90 tx=0x7f76b400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25a7396074b3a151 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:21.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.754+0000 7f76b959a700 1 -- 172.21.15.3:0/1384827482 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b400e070 con 0x7f76bc09e410 2023-10-03T10:17:21.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.754+0000 7f76b959a700 1 -- 172.21.15.3:0/1384827482 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b400bb20 con 0x7f76bc09e410 2023-10-03T10:17:21.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.754+0000 7f76b959a700 1 -- 172.21.15.3:0/1384827482 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b4012430 con 0x7f76bc09e410 2023-10-03T10:17:21.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.755+0000 7f76c0bbc700 1 -- 172.21.15.3:0/1384827482 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76bc09e410 msgr2=0x7f76bc130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:21.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.755+0000 7f76c0bbc700 1 --2- 172.21.15.3:0/1384827482 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76bc09e410 0x7f76bc130ba0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009d90 tx=0x7f76b400b330 comp rx=0 tx=0).stop 2023-10-03T10:17:21.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.755+0000 7f76c0bbc700 1 -- 172.21.15.3:0/1384827482 shutdown_connections 2023-10-03T10:17:21.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.755+0000 7f76c0bbc700 1 --2- 172.21.15.3:0/1384827482 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76bc09e410 0x7f76bc130ba0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.755+0000 7f76c0bbc700 1 --2- 172.21.15.3:0/1384827482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76bc09d6f0 0x7f76bc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.755+0000 7f76c0bbc700 1 --2- 172.21.15.3:0/1384827482 >> v2:172.21.15.3:3300/0 conn(0x7f76bc09e000 0x7f76bc130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.755+0000 7f76c0bbc700 1 -- 172.21.15.3:0/1384827482 >> 172.21.15.3:0/1384827482 conn(0x7f76bc066f50 msgr2=0x7f76bc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:21.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.755+0000 7f76c0bbc700 1 -- 172.21.15.3:0/1384827482 shutdown_connections 2023-10-03T10:17:21.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.755+0000 7f76c0bbc700 1 -- 172.21.15.3:0/1384827482 wait complete. 2023-10-03T10:17:21.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.757+0000 7f76c0bbc700 1 Processor -- start 2023-10-03T10:17:21.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.757+0000 7f76c0bbc700 1 -- start start 2023-10-03T10:17:21.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.758+0000 7f76c0bbc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76bc09d6f0 0x7f76bc11f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:21.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.758+0000 7f76c0bbc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76bc09e000 0x7f76bc11f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:21.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.758+0000 7f76c0bbc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76bc09e410 0x7f76bc11fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:21.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.758+0000 7f76c0bbc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc122ac0 con 0x7f76bc09e410 2023-10-03T10:17:21.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.758+0000 7f76c0bbc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc122ca0 con 0x7f76bc09d6f0 2023-10-03T10:17:21.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.758+0000 7f76c0bbc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc0a2a90 con 0x7f76bc09e000 2023-10-03T10:17:21.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.758+0000 7f76ba59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76bc09d6f0 0x7f76bc11f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:21.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.758+0000 7f76ba59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76bc09d6f0 0x7f76bc11f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51156/0 (socket says 172.21.15.3:51156) 2023-10-03T10:17:21.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.758+0000 7f76ba59c700 1 -- 172.21.15.3:0/2026187411 learned_addr learned my addr 172.21.15.3:0/2026187411 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:21.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.758+0000 7f76bad9d700 1 --2- 172.21.15.3:0/2026187411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76bc09e410 0x7f76bc11fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:21.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.759+0000 7f76b9d9b700 1 --2- 172.21.15.3:0/2026187411 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76bc09e000 0x7f76bc11f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:21.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.759+0000 7f76ba59c700 1 -- 172.21.15.3:0/2026187411 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76bc09e000 msgr2=0x7f76bc11f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:21.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.759+0000 7f76ba59c700 1 --2- 172.21.15.3:0/2026187411 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76bc09e000 0x7f76bc11f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.759+0000 7f76ba59c700 1 -- 172.21.15.3:0/2026187411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76bc09e410 msgr2=0x7f76bc11fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:21.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.759+0000 7f76ba59c700 1 --2- 172.21.15.3:0/2026187411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76bc09e410 0x7f76bc11fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.759+0000 7f76ba59c700 1 -- 172.21.15.3:0/2026187411 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ac009550 con 0x7f76bc09d6f0 2023-10-03T10:17:21.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.759+0000 7f76bad9d700 1 --2- 172.21.15.3:0/2026187411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76bc09e410 0x7f76bc11fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:21.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.759+0000 7f76b9d9b700 1 --2- 172.21.15.3:0/2026187411 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76bc09e000 0x7f76bc11f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:17:21.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.759+0000 7f76ba59c700 1 --2- 172.21.15.3:0/2026187411 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76bc09d6f0 0x7f76bc11f250 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00ebb0 tx=0x7f76ac00eec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:21.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.759+0000 7f76aaffd700 1 -- 172.21.15.3:0/2026187411 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac004030 con 0x7f76bc09d6f0 2023-10-03T10:17:21.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.759+0000 7f76aaffd700 1 -- 172.21.15.3:0/2026187411 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76ac004190 con 0x7f76bc09d6f0 2023-10-03T10:17:21.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.759+0000 7f76c0bbc700 1 -- 172.21.15.3:0/2026187411 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b4009a70 con 0x7f76bc09d6f0 2023-10-03T10:17:21.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.759+0000 7f76aaffd700 1 -- 172.21.15.3:0/2026187411 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac004520 con 0x7f76bc09d6f0 2023-10-03T10:17:21.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.759+0000 7f76c0bbc700 1 -- 172.21.15.3:0/2026187411 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76bc122700 con 0x7f76bc09d6f0 2023-10-03T10:17:21.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.760+0000 7f76aaffd700 1 -- 172.21.15.3:0/2026187411 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f76ac00c850 con 0x7f76bc09d6f0 2023-10-03T10:17:21.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.760+0000 7f76aaffd700 1 -- 172.21.15.3:0/2026187411 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f76ac03f720 con 0x7f76bc09d6f0 2023-10-03T10:17:21.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.761+0000 7f76c0bbc700 1 -- 172.21.15.3:0/2026187411 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76bc09c700 con 0x7f76bc09d6f0 2023-10-03T10:17:21.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.766+0000 7f76aaffd700 1 -- 172.21.15.3:0/2026187411 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f76ac0036a0 con 0x7f76bc09d6f0 2023-10-03T10:17:21.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.837+0000 7f08de08d700 1 Processor -- start 2023-10-03T10:17:21.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.837+0000 7f08de08d700 1 -- start start 2023-10-03T10:17:21.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.837+0000 7f08de08d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f08d809f410 0x7f08d8136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:21.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.837+0000 7f08de08d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08d8136fe0 0x7f08d81393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:21.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.837+0000 7f08de08d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d8070940 con 0x7f08d809e930 2023-10-03T10:17:21.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.837+0000 7f08de08d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08d806fd60 con 0x7f08d809f410 2023-10-03T10:17:21.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.837+0000 7f08de08d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f08d806fa80 con 0x7f08d8136fe0 2023-10-03T10:17:21.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.837+0000 7f08d6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f08d809f410 0x7f08d8136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:21.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.837+0000 7f08d6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f08d809f410 0x7f08d8136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41668/0 (socket says 172.21.15.3:41668) 2023-10-03T10:17:21.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.837+0000 7f08d6ffd700 1 -- 172.21.15.3:0/1685572782 learned_addr learned my addr 172.21.15.3:0/1685572782 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:21.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.838+0000 7f08d7fff700 1 --2- 172.21.15.3:0/1685572782 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08d8136fe0 0x7f08d81393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:21.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.838+0000 7f08d6ffd700 1 -- 172.21.15.3:0/1685572782 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08d8136fe0 msgr2=0x7f08d81393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:21.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.838+0000 7f08d6ffd700 1 --2- 172.21.15.3:0/1685572782 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08d8136fe0 0x7f08d81393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.838+0000 7f08d6ffd700 1 -- 172.21.15.3:0/1685572782 >> v1:172.21.15.3:6789/0 conn(0x7f08d809e930 legacy=0x7f08d809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:21.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.838+0000 7f08d6ffd700 1 -- 172.21.15.3:0/1685572782 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d809fb60 con 0x7f08d809f410 2023-10-03T10:17:21.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.838+0000 7f08d6ffd700 1 --2- 172.21.15.3:0/1685572782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d809f410 0x7f08d8136aa0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0098f0 tx=0x7f08cc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c6ec10c24cdf467 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:21.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.838+0000 7f08d67fc700 1 -- 172.21.15.3:0/1685572782 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc011830 con 0x7f08d809f410 2023-10-03T10:17:21.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.838+0000 7f08d67fc700 1 -- 172.21.15.3:0/1685572782 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08cc011990 con 0x7f08d809f410 2023-10-03T10:17:21.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.838+0000 7f08d67fc700 1 -- 172.21.15.3:0/1685572782 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc011d20 con 0x7f08d809f410 2023-10-03T10:17:21.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.839+0000 7f08de08d700 1 -- 172.21.15.3:0/1685572782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d809f410 msgr2=0x7f08d8136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:21.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.839+0000 7f08de08d700 1 --2- 172.21.15.3:0/1685572782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d809f410 0x7f08d8136aa0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0098f0 tx=0x7f08cc009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:21.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.839+0000 7f08de08d700 1 -- 172.21.15.3:0/1685572782 shutdown_connections 2023-10-03T10:17:21.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.839+0000 7f08de08d700 1 --2- 172.21.15.3:0/1685572782 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08d8136fe0 0x7f08d81393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.839+0000 7f08de08d700 1 --2- 172.21.15.3:0/1685572782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d809f410 0x7f08d8136aa0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.839+0000 7f08de08d700 1 -- 172.21.15.3:0/1685572782 >> 172.21.15.3:0/1685572782 conn(0x7f08d80a4210 msgr2=0x7f08d80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:21.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.839+0000 7f08de08d700 1 -- 172.21.15.3:0/1685572782 shutdown_connections 2023-10-03T10:17:21.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.839+0000 7f08de08d700 1 -- 172.21.15.3:0/1685572782 wait complete. 2023-10-03T10:17:21.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.840+0000 7f08de08d700 1 Processor -- start 2023-10-03T10:17:21.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.841+0000 7f08de08d700 1 -- start start 2023-10-03T10:17:21.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.841+0000 7f08de08d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d809e930 0x7f08d8121ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:21.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.841+0000 7f08de08d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08d809f410 0x7f08d8122000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:21.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.841+0000 7f08de08d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08d8136fe0 0x7f08d8122540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:21.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.841+0000 7f08de08d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d806d810 con 0x7f08d809e930 2023-10-03T10:17:21.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.841+0000 7f08de08d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d8125270 con 0x7f08d809f410 2023-10-03T10:17:21.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.841+0000 7f08de08d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f08d81236e0 con 0x7f08d8136fe0 2023-10-03T10:17:21.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.841+0000 7f08d7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08d8136fe0 0x7f08d8122540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:21.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.841+0000 7f08d77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d809e930 0x7f08d8121ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:21.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.841+0000 7f08d7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08d8136fe0 0x7f08d8122540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53682/0 (socket says 172.21.15.3:53682) 2023-10-03T10:17:21.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.841+0000 7f08d7fff700 1 -- 172.21.15.3:0/3157306232 learned_addr learned my addr 172.21.15.3:0/3157306232 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:21.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.841+0000 7f08d77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d809e930 0x7f08d8121ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41676/0 (socket says 172.21.15.3:41676) 2023-10-03T10:17:21.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.841+0000 7f08d6ffd700 1 --2- 172.21.15.3:0/3157306232 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08d809f410 0x7f08d8122000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:21.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.842+0000 7f08d7fff700 1 -- 172.21.15.3:0/3157306232 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08d809f410 msgr2=0x7f08d8122000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:21.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.842+0000 7f08d7fff700 1 --2- 172.21.15.3:0/3157306232 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08d809f410 0x7f08d8122000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.842+0000 7f08d7fff700 1 -- 172.21.15.3:0/3157306232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d809e930 msgr2=0x7f08d8121ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:21.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.842+0000 7f08d7fff700 1 --2- 172.21.15.3:0/3157306232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d809e930 0x7f08d8121ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.842+0000 7f08d7fff700 1 -- 172.21.15.3:0/3157306232 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08cc009550 con 0x7f08d8136fe0 2023-10-03T10:17:21.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.842+0000 7f08d77fe700 1 --2- 172.21.15.3:0/3157306232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d809e930 0x7f08d8121ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:21.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.842+0000 7f08d7fff700 1 --2- 172.21.15.3:0/3157306232 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08d8136fe0 0x7f08d8122540 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f08d0009dd0 tx=0x7f08d000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:21.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.842+0000 7f08bffff700 1 -- 172.21.15.3:0/3157306232 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d0009c40 con 0x7f08d8136fe0 2023-10-03T10:17:21.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.842+0000 7f08bffff700 1 -- 172.21.15.3:0/3157306232 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d000f040 con 0x7f08d8136fe0 2023-10-03T10:17:21.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.842+0000 7f08bffff700 1 -- 172.21.15.3:0/3157306232 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d000ceb0 con 0x7f08d8136fe0 2023-10-03T10:17:21.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.842+0000 7f08de08d700 1 -- 172.21.15.3:0/3157306232 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d8123e30 con 0x7f08d8136fe0 2023-10-03T10:17:21.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.842+0000 7f08de08d700 1 -- 172.21.15.3:0/3157306232 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d81369c0 con 0x7f08d8136fe0 2023-10-03T10:17:21.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.843+0000 7f08bffff700 1 -- 172.21.15.3:0/3157306232 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f08d0014480 con 0x7f08d8136fe0 2023-10-03T10:17:21.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.843+0000 7f08bffff700 1 -- 172.21.15.3:0/3157306232 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f08d003fe40 con 0x7f08d8136fe0 2023-10-03T10:17:21.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.844+0000 7f08de08d700 1 -- 172.21.15.3:0/3157306232 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d81253b0 con 0x7f08d8136fe0 2023-10-03T10:17:21.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.849+0000 7f08bffff700 1 -- 172.21.15.3:0/3157306232 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f08d81253b0 con 0x7f08d8136fe0 2023-10-03T10:17:21.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.887+0000 7f788430f700 1 Processor -- start 2023-10-03T10:17:21.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.887+0000 7f788430f700 1 -- start start 2023-10-03T10:17:21.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.888+0000 7f788430f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f787c0a0a30 0x7f787c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:21.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.888+0000 7f788430f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f787c09d630 0x7f787c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:21.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.888+0000 7f788430f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f787c133930 0x7f787c133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:21.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.888+0000 7f788430f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f787c0a2a70 con 0x7f787c09d630 2023-10-03T10:17:21.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.888+0000 7f788430f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787c0a01d0 con 0x7f787c133930 2023-10-03T10:17:21.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.888+0000 7f788430f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f787c09e2b0 con 0x7f787c0a0a30 2023-10-03T10:17:21.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.888+0000 7f78818aa700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f787c09d630 0x7f787c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:21.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.888+0000 7f78820ab700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f787c0a0a30 0x7f787c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:21.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.888+0000 7f78828ac700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f787c133930 0x7f787c133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:21.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.888+0000 7f78818aa700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f787c09d630 0x7f787c0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41684/0 (socket says 172.21.15.3:41684) 2023-10-03T10:17:21.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.888+0000 7f78820ab700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f787c0a0a30 0x7f787c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53694/0 (socket says 172.21.15.3:53694) 2023-10-03T10:17:21.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.888+0000 7f78818aa700 1 -- 172.21.15.3:0/1502048028 learned_addr learned my addr 172.21.15.3:0/1502048028 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:21.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.889+0000 7f78828ac700 1 -- 172.21.15.3:0/1502048028 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f787c0a0a30 msgr2=0x7f787c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:21.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.889+0000 7f78828ac700 1 --2- 172.21.15.3:0/1502048028 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f787c0a0a30 0x7f787c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.889+0000 7f78828ac700 1 -- 172.21.15.3:0/1502048028 >> v2:172.21.15.3:3300/0 conn(0x7f787c09d630 msgr2=0x7f787c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:21.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.889+0000 7f78828ac700 1 --2- 172.21.15.3:0/1502048028 >> v2:172.21.15.3:3300/0 conn(0x7f787c09d630 0x7f787c0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.889+0000 7f78828ac700 1 -- 172.21.15.3:0/1502048028 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787c09dc50 con 0x7f787c133930 2023-10-03T10:17:21.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.889+0000 7f78820ab700 1 --2- 172.21.15.3:0/1502048028 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f787c0a0a30 0x7f787c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:21.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.889+0000 7f78818aa700 1 --2- 172.21.15.3:0/1502048028 >> v2:172.21.15.3:3300/0 conn(0x7f787c09d630 0x7f787c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:21.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.889+0000 7f78828ac700 1 --2- 172.21.15.3:0/1502048028 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f787c133930 0x7f787c133dd0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7874009d90 tx=0x7f787400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb8acad2bbec6e26 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:21.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.889+0000 7f78810a9700 1 -- 172.21.15.3:0/1502048028 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787400e070 con 0x7f787c133930 2023-10-03T10:17:21.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.889+0000 7f78810a9700 1 -- 172.21.15.3:0/1502048028 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787400bb20 con 0x7f787c133930 2023-10-03T10:17:21.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.889+0000 7f78810a9700 1 -- 172.21.15.3:0/1502048028 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7874012430 con 0x7f787c133930 2023-10-03T10:17:21.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.890+0000 7f788430f700 1 -- 172.21.15.3:0/1502048028 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f787c133930 msgr2=0x7f787c133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:21.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.890+0000 7f788430f700 1 --2- 172.21.15.3:0/1502048028 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f787c133930 0x7f787c133dd0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7874009d90 tx=0x7f787400b330 comp rx=0 tx=0).stop 2023-10-03T10:17:21.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.890+0000 7f788430f700 1 -- 172.21.15.3:0/1502048028 shutdown_connections 2023-10-03T10:17:21.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.890+0000 7f788430f700 1 --2- 172.21.15.3:0/1502048028 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f787c133930 0x7f787c133dd0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.890+0000 7f788430f700 1 --2- 172.21.15.3:0/1502048028 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f787c0a0a30 0x7f787c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.890+0000 7f788430f700 1 --2- 172.21.15.3:0/1502048028 >> v2:172.21.15.3:3300/0 conn(0x7f787c09d630 0x7f787c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.890+0000 7f788430f700 1 -- 172.21.15.3:0/1502048028 >> 172.21.15.3:0/1502048028 conn(0x7f787c0a3050 msgr2=0x7f787c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:21.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.890+0000 7f788430f700 1 -- 172.21.15.3:0/1502048028 shutdown_connections 2023-10-03T10:17:21.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.890+0000 7f788430f700 1 -- 172.21.15.3:0/1502048028 wait complete. 2023-10-03T10:17:21.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.892+0000 7f788430f700 1 Processor -- start 2023-10-03T10:17:21.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.892+0000 7f788430f700 1 -- start start 2023-10-03T10:17:21.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.892+0000 7f788430f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f787c09d630 0x7f787c1c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:21.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.892+0000 7f788430f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f787c0a0a30 0x7f787c1c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:21.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.892+0000 7f788430f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f787c133930 0x7f787c1c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:21.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.892+0000 7f788430f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c0a0410 con 0x7f787c133930 2023-10-03T10:17:21.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.892+0000 7f788430f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787c120630 con 0x7f787c0a0a30 2023-10-03T10:17:21.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.892+0000 7f788430f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f787c0a4030 con 0x7f787c09d630 2023-10-03T10:17:21.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.893+0000 7f78820ab700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f787c09d630 0x7f787c1c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:21.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.893+0000 7f78820ab700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f787c09d630 0x7f787c1c7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53700/0 (socket says 172.21.15.3:53700) 2023-10-03T10:17:21.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.893+0000 7f78820ab700 1 -- 172.21.15.3:0/2896754425 learned_addr learned my addr 172.21.15.3:0/2896754425 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:21.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.893+0000 7f78828ac700 1 --2- 172.21.15.3:0/2896754425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f787c133930 0x7f787c1c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:21.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.893+0000 7f78818aa700 1 --2- 172.21.15.3:0/2896754425 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f787c0a0a30 0x7f787c1c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:21.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.893+0000 7f78820ab700 1 -- 172.21.15.3:0/2896754425 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f787c0a0a30 msgr2=0x7f787c1c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:21.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.893+0000 7f78820ab700 1 --2- 172.21.15.3:0/2896754425 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f787c0a0a30 0x7f787c1c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.893+0000 7f78820ab700 1 -- 172.21.15.3:0/2896754425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f787c133930 msgr2=0x7f787c1c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:21.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.893+0000 7f78820ab700 1 --2- 172.21.15.3:0/2896754425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f787c133930 0x7f787c1c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.893+0000 7f78820ab700 1 -- 172.21.15.3:0/2896754425 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c009550 con 0x7f787c09d630 2023-10-03T10:17:21.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.893+0000 7f78820ab700 1 --2- 172.21.15.3:0/2896754425 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f787c09d630 0x7f787c1c7e80 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f787800ebe0 tx=0x7f787800eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:21.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.893+0000 7f7872ffd700 1 -- 172.21.15.3:0/2896754425 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787800c850 con 0x7f787c09d630 2023-10-03T10:17:21.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.893+0000 7f7872ffd700 1 -- 172.21.15.3:0/2896754425 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787800c9b0 con 0x7f787c09d630 2023-10-03T10:17:21.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.893+0000 7f7872ffd700 1 -- 172.21.15.3:0/2896754425 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787800cd40 con 0x7f787c09d630 2023-10-03T10:17:21.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.893+0000 7f788430f700 1 -- 172.21.15.3:0/2896754425 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78780094a0 con 0x7f787c09d630 2023-10-03T10:17:21.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.894+0000 7f788430f700 1 -- 172.21.15.3:0/2896754425 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7874009a70 con 0x7f787c09d630 2023-10-03T10:17:21.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.895+0000 7f7872ffd700 1 -- 172.21.15.3:0/2896754425 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7878013470 con 0x7f787c09d630 2023-10-03T10:17:21.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.895+0000 7f788430f700 1 -- 172.21.15.3:0/2896754425 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f787c0747f0 con 0x7f787c09d630 2023-10-03T10:17:21.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.895+0000 7f7872ffd700 1 -- 172.21.15.3:0/2896754425 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f787803f780 con 0x7f787c09d630 2023-10-03T10:17:21.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.900+0000 7f7872ffd700 1 -- 172.21.15.3:0/2896754425 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f787800cea0 con 0x7f787c09d630 2023-10-03T10:17:21.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.982+0000 7f76c0bbc700 1 -- 172.21.15.3:0/2026187411 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f76bc074ea0 con 0x7f76bc09d6f0 2023-10-03T10:17:21.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.982+0000 7f76aaffd700 1 -- 172.21.15.3:0/2026187411 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f76ac013e60 con 0x7f76bc09d6f0 2023-10-03T10:17:21.982 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:21.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.985+0000 7f76c0bbc700 1 -- 172.21.15.3:0/2026187411 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76bc09d6f0 msgr2=0x7f76bc11f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:21.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.985+0000 7f76c0bbc700 1 --2- 172.21.15.3:0/2026187411 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76bc09d6f0 0x7f76bc11f250 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00ebb0 tx=0x7f76ac00eec0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.985+0000 7f76c0bbc700 1 -- 172.21.15.3:0/2026187411 shutdown_connections 2023-10-03T10:17:21.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.985+0000 7f76c0bbc700 1 --2- 172.21.15.3:0/2026187411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76bc09e410 0x7f76bc11fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.985+0000 7f76c0bbc700 1 --2- 172.21.15.3:0/2026187411 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76bc09d6f0 0x7f76bc11f250 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.985+0000 7f76c0bbc700 1 --2- 172.21.15.3:0/2026187411 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76bc09e000 0x7f76bc11f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:21.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.985+0000 7f76c0bbc700 1 -- 172.21.15.3:0/2026187411 >> 172.21.15.3:0/2026187411 conn(0x7f76bc066f50 msgr2=0x7f76bc119480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:21.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.985+0000 7f76c0bbc700 1 -- 172.21.15.3:0/2026187411 shutdown_connections 2023-10-03T10:17:21.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:21.985+0000 7f76c0bbc700 1 -- 172.21.15.3:0/2026187411 wait complete. 2023-10-03T10:17:21.999 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:22.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.058+0000 7f08de08d700 1 -- 172.21.15.3:0/3157306232 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f08d81253b0 con 0x7f08d8136fe0 2023-10-03T10:17:22.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.058+0000 7f08bffff700 1 -- 172.21.15.3:0/3157306232 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f08d81253b0 con 0x7f08d8136fe0 2023-10-03T10:17:22.059 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:22.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.061+0000 7f08de08d700 1 -- 172.21.15.3:0/3157306232 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08d8136fe0 msgr2=0x7f08d8122540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:22.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.061+0000 7f08de08d700 1 --2- 172.21.15.3:0/3157306232 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08d8136fe0 0x7f08d8122540 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f08d0009dd0 tx=0x7f08d000c3a0 comp rx=0 tx=0).stop 2023-10-03T10:17:22.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.061+0000 7f08de08d700 1 -- 172.21.15.3:0/3157306232 shutdown_connections 2023-10-03T10:17:22.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.061+0000 7f08de08d700 1 --2- 172.21.15.3:0/3157306232 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f08d8136fe0 0x7f08d8122540 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:22.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.061+0000 7f08de08d700 1 --2- 172.21.15.3:0/3157306232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d809e930 0x7f08d8121ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:22.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.061+0000 7f08de08d700 1 --2- 172.21.15.3:0/3157306232 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f08d809f410 0x7f08d8122000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:22.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.061+0000 7f08de08d700 1 -- 172.21.15.3:0/3157306232 >> 172.21.15.3:0/3157306232 conn(0x7f08d80a4210 msgr2=0x7f08d80a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:22.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.061+0000 7f08de08d700 1 -- 172.21.15.3:0/3157306232 shutdown_connections 2023-10-03T10:17:22.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.061+0000 7f08de08d700 1 -- 172.21.15.3:0/3157306232 wait complete. 2023-10-03T10:17:22.075 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:22.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.109+0000 7f788430f700 1 -- 172.21.15.3:0/2896754425 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f787c121f40 con 0x7f787c09d630 2023-10-03T10:17:22.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.109+0000 7f7872ffd700 1 -- 172.21.15.3:0/2896754425 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f787c121f40 con 0x7f787c09d630 2023-10-03T10:17:22.109 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:22.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.111+0000 7f788430f700 1 -- 172.21.15.3:0/2896754425 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f787c09d630 msgr2=0x7f787c1c7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:22.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.111+0000 7f788430f700 1 --2- 172.21.15.3:0/2896754425 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f787c09d630 0x7f787c1c7e80 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f787800ebe0 tx=0x7f787800eef0 comp rx=0 tx=0).stop 2023-10-03T10:17:22.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.111+0000 7f788430f700 1 -- 172.21.15.3:0/2896754425 shutdown_connections 2023-10-03T10:17:22.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.111+0000 7f788430f700 1 --2- 172.21.15.3:0/2896754425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f787c133930 0x7f787c1c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:22.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.111+0000 7f788430f700 1 --2- 172.21.15.3:0/2896754425 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f787c09d630 0x7f787c1c7e80 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:22.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.111+0000 7f788430f700 1 --2- 172.21.15.3:0/2896754425 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f787c0a0a30 0x7f787c1c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:22.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.111+0000 7f788430f700 1 -- 172.21.15.3:0/2896754425 >> 172.21.15.3:0/2896754425 conn(0x7f787c0a3050 msgr2=0x7f787c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:22.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.111+0000 7f788430f700 1 -- 172.21.15.3:0/2896754425 shutdown_connections 2023-10-03T10:17:22.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.111+0000 7f788430f700 1 -- 172.21.15.3:0/2896754425 wait complete. 2023-10-03T10:17:22.125 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:22.602 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:22.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.748+0000 7f346c300700 1 Processor -- start 2023-10-03T10:17:22.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.748+0000 7f346c300700 1 -- start start 2023-10-03T10:17:22.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.749+0000 7f346c300700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3464133420 0x7f34641337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:22.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.749+0000 7f346c300700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3464134260 0x7f34641306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:22.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.749+0000 7f346c300700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f346406b990 con 0x7f3464133e30 2023-10-03T10:17:22.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.749+0000 7f346c300700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34640a0590 con 0x7f3464133420 2023-10-03T10:17:22.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.749+0000 7f346c300700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346409fb80 con 0x7f3464134260 2023-10-03T10:17:22.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.749+0000 7f346a89d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3464134260 0x7f34641306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:22.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.749+0000 7f346a09c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3464133420 0x7f34641337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:22.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.749+0000 7f346a89d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3464134260 0x7f34641306a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51202/0 (socket says 172.21.15.3:51202) 2023-10-03T10:17:22.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.749+0000 7f346a89d700 1 -- 172.21.15.3:0/3763540202 learned_addr learned my addr 172.21.15.3:0/3763540202 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:22.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.749+0000 7f346a09c700 1 --2- 172.21.15.3:0/3763540202 >> v2:172.21.15.3:3300/0 conn(0x7f3464133420 0x7f34641337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41710/0 (socket says 172.21.15.3:41710) 2023-10-03T10:17:22.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.749+0000 7f346a89d700 1 -- 172.21.15.3:0/3763540202 >> v2:172.21.15.3:3300/0 conn(0x7f3464133420 msgr2=0x7f34641337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:22.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.749+0000 7f346a89d700 1 --2- 172.21.15.3:0/3763540202 >> v2:172.21.15.3:3300/0 conn(0x7f3464133420 0x7f34641337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:22.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.749+0000 7f346a89d700 1 -- 172.21.15.3:0/3763540202 >> v1:172.21.15.3:6789/0 conn(0x7f3464133e30 legacy=0x7f346412ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:22.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.749+0000 7f346a89d700 1 -- 172.21.15.3:0/3763540202 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34640a0030 con 0x7f3464134260 2023-10-03T10:17:22.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.749+0000 7f346a09c700 1 --2- 172.21.15.3:0/3763540202 >> v2:172.21.15.3:3300/0 conn(0x7f3464133420 0x7f34641337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:22.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.750+0000 7f346a89d700 1 --2- 172.21.15.3:0/3763540202 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3464134260 0x7f34641306a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f345c009e50 tx=0x7f345c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b14be9ab98871120 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:22.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.750+0000 7f346909a700 1 -- 172.21.15.3:0/3763540202 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345c00e070 con 0x7f3464134260 2023-10-03T10:17:22.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.750+0000 7f346909a700 1 -- 172.21.15.3:0/3763540202 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f345c00b910 con 0x7f3464134260 2023-10-03T10:17:22.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.750+0000 7f346909a700 1 -- 172.21.15.3:0/3763540202 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345c00bcc0 con 0x7f3464134260 2023-10-03T10:17:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.750+0000 7f346c300700 1 -- 172.21.15.3:0/3763540202 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3464134260 msgr2=0x7f34641306a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.750+0000 7f346c300700 1 --2- 172.21.15.3:0/3763540202 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3464134260 0x7f34641306a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f345c009e50 tx=0x7f345c00b330 comp rx=0 tx=0).stop 2023-10-03T10:17:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.750+0000 7f346c300700 1 -- 172.21.15.3:0/3763540202 shutdown_connections 2023-10-03T10:17:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.750+0000 7f346c300700 1 --2- 172.21.15.3:0/3763540202 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3464134260 0x7f34641306a0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.750+0000 7f346c300700 1 --2- 172.21.15.3:0/3763540202 >> v2:172.21.15.3:3300/0 conn(0x7f3464133420 0x7f34641337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.750+0000 7f346c300700 1 -- 172.21.15.3:0/3763540202 >> 172.21.15.3:0/3763540202 conn(0x7f34640a4210 msgr2=0x7f34640a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.750+0000 7f346c300700 1 -- 172.21.15.3:0/3763540202 shutdown_connections 2023-10-03T10:17:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.750+0000 7f346c300700 1 -- 172.21.15.3:0/3763540202 wait complete. 2023-10-03T10:17:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.751+0000 7f346c300700 1 Processor -- start 2023-10-03T10:17:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.751+0000 7f346c300700 1 -- start start 2023-10-03T10:17:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.752+0000 7f346c300700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3464133420 0x7f346409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.752+0000 7f346c300700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3464133e30 0x7f346409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:22.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.752+0000 7f346c300700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3464134260 0x7f346409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:22.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.752+0000 7f346c300700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346411fc20 con 0x7f3464133e30 2023-10-03T10:17:22.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.752+0000 7f346c300700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346411fee0 con 0x7f3464133420 2023-10-03T10:17:22.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.752+0000 7f346c300700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f346411f910 con 0x7f3464134260 2023-10-03T10:17:22.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.752+0000 7f346a09c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3464133420 0x7f346409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:22.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.752+0000 7f346989b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3464133e30 0x7f346409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:22.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.752+0000 7f346a09c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3464133420 0x7f346409c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51218/0 (socket says 172.21.15.3:51218) 2023-10-03T10:17:22.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.752+0000 7f346989b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3464133e30 0x7f346409c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41718/0 (socket says 172.21.15.3:41718) 2023-10-03T10:17:22.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.752+0000 7f346a09c700 1 -- 172.21.15.3:0/1844487813 learned_addr learned my addr 172.21.15.3:0/1844487813 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:22.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.752+0000 7f346a89d700 1 --2- 172.21.15.3:0/1844487813 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3464134260 0x7f346409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:22.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.752+0000 7f346989b700 1 -- 172.21.15.3:0/1844487813 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3464134260 msgr2=0x7f346409cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:22.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.752+0000 7f346989b700 1 --2- 172.21.15.3:0/1844487813 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3464134260 0x7f346409cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:22.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.752+0000 7f346989b700 1 -- 172.21.15.3:0/1844487813 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3464133420 msgr2=0x7f346409c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:22.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.752+0000 7f346989b700 1 --2- 172.21.15.3:0/1844487813 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3464133420 0x7f346409c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:22.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.752+0000 7f346989b700 1 -- 172.21.15.3:0/1844487813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34600094a0 con 0x7f3464133e30 2023-10-03T10:17:22.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.752+0000 7f346a09c700 1 --2- 172.21.15.3:0/1844487813 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3464133420 0x7f346409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:22.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.753+0000 7f346989b700 1 --2- 172.21.15.3:0/1844487813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3464133e30 0x7f346409c9c0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f3458000f60 tx=0x7f345800c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:22.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.753+0000 7f3456ffd700 1 -- 172.21.15.3:0/1844487813 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34580052d0 con 0x7f3464133e30 2023-10-03T10:17:22.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.753+0000 7f3456ffd700 1 -- 172.21.15.3:0/1844487813 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3458005430 con 0x7f3464133e30 2023-10-03T10:17:22.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.753+0000 7f346c300700 1 -- 172.21.15.3:0/1844487813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f345c009b30 con 0x7f3464133e30 2023-10-03T10:17:22.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.753+0000 7f3456ffd700 1 -- 172.21.15.3:0/1844487813 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34580038c0 con 0x7f3464133e30 2023-10-03T10:17:22.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.753+0000 7f346c300700 1 -- 172.21.15.3:0/1844487813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34641201c0 con 0x7f3464133e30 2023-10-03T10:17:22.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.754+0000 7f3456ffd700 1 -- 172.21.15.3:0/1844487813 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3458003a20 con 0x7f3464133e30 2023-10-03T10:17:22.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.754+0000 7f3456ffd700 1 -- 172.21.15.3:0/1844487813 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f345803e9d0 con 0x7f3464133e30 2023-10-03T10:17:22.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.754+0000 7f346c300700 1 -- 172.21.15.3:0/1844487813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3464074ea0 con 0x7f3464133e30 2023-10-03T10:17:22.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.759+0000 7f3456ffd700 1 -- 172.21.15.3:0/1844487813 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3458017070 con 0x7f3464133e30 2023-10-03T10:17:22.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.961+0000 7f346c300700 1 -- 172.21.15.3:0/1844487813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f346406b250 con 0x7f3464133e30 2023-10-03T10:17:22.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.962+0000 7f3456ffd700 1 -- 172.21.15.3:0/1844487813 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3458011590 con 0x7f3464133e30 2023-10-03T10:17:22.962 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:22.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.964+0000 7f346c300700 1 -- 172.21.15.3:0/1844487813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3464133e30 msgr2=0x7f346409c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:22.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.964+0000 7f346c300700 1 --2- 172.21.15.3:0/1844487813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3464133e30 0x7f346409c9c0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f3458000f60 tx=0x7f345800c730 comp rx=0 tx=0).stop 2023-10-03T10:17:22.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.964+0000 7f346c300700 1 -- 172.21.15.3:0/1844487813 shutdown_connections 2023-10-03T10:17:22.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.964+0000 7f346c300700 1 --2- 172.21.15.3:0/1844487813 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3464134260 0x7f346409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:22.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.964+0000 7f346c300700 1 --2- 172.21.15.3:0/1844487813 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3464133420 0x7f346409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:22.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.964+0000 7f346c300700 1 --2- 172.21.15.3:0/1844487813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3464133e30 0x7f346409c9c0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:22.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.964+0000 7f346c300700 1 -- 172.21.15.3:0/1844487813 >> 172.21.15.3:0/1844487813 conn(0x7f34640a4210 msgr2=0x7f34640a0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:22.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.964+0000 7f346c300700 1 -- 172.21.15.3:0/1844487813 shutdown_connections 2023-10-03T10:17:22.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:22.964+0000 7f346c300700 1 -- 172.21.15.3:0/1844487813 wait complete. 2023-10-03T10:17:22.977 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:23.000 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:23.075 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:23.126 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:23.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.139+0000 7f9f321bd700 1 Processor -- start 2023-10-03T10:17:23.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.139+0000 7f9f321bd700 1 -- start start 2023-10-03T10:17:23.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.140+0000 7f9f321bd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9f2c09e850 0x7f9f2c09ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:23.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.140+0000 7f9f321bd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9f2c131ea0 0x7f9f2c139c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:23.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.140+0000 7f9f321bd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f2c09b560 con 0x7f9f2c09f160 2023-10-03T10:17:23.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.140+0000 7f9f321bd700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f2c0a0040 con 0x7f9f2c131ea0 2023-10-03T10:17:23.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.140+0000 7f9f321bd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c0a02e0 con 0x7f9f2c09e850 2023-10-03T10:17:23.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.140+0000 7f9f2affd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9f2c09f160 0x7f9f2c131790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55396/0 (socket says 172.21.15.3:55396) 2023-10-03T10:17:23.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.140+0000 7f9f2affd700 1 -- 172.21.15.3:0/3979942606 learned_addr learned my addr 172.21.15.3:0/3979942606 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:23.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.140+0000 7f9f2b7fe700 1 --2- 172.21.15.3:0/3979942606 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9f2c09e850 0x7f9f2c09ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:23.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.140+0000 7f9f2bfff700 1 --2- 172.21.15.3:0/3979942606 >> v2:172.21.15.3:3300/0 conn(0x7f9f2c131ea0 0x7f9f2c139c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:23.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.141+0000 7f9f2b7fe700 1 -- 172.21.15.3:0/3979942606 >> v2:172.21.15.3:3300/0 conn(0x7f9f2c131ea0 msgr2=0x7f9f2c139c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:23.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.141+0000 7f9f2b7fe700 1 --2- 172.21.15.3:0/3979942606 >> v2:172.21.15.3:3300/0 conn(0x7f9f2c131ea0 0x7f9f2c139c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:23.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.141+0000 7f9f2b7fe700 1 -- 172.21.15.3:0/3979942606 >> v1:172.21.15.3:6789/0 conn(0x7f9f2c09f160 legacy=0x7f9f2c131790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:23.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.141+0000 7f9f2a7fc700 1 -- 172.21.15.3:0/3979942606 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 918718824 0 0) 0x7f9f2c09b560 con 0x7f9f2c09f160 2023-10-03T10:17:23.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.141+0000 7f9f2b7fe700 1 -- 172.21.15.3:0/3979942606 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2c09fab0 con 0x7f9f2c09e850 2023-10-03T10:17:23.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.141+0000 7f9f2bfff700 1 --2- 172.21.15.3:0/3979942606 >> v2:172.21.15.3:3300/0 conn(0x7f9f2c131ea0 0x7f9f2c139c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:17:23.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.141+0000 7f9f2b7fe700 1 --2- 172.21.15.3:0/3979942606 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9f2c09e850 0x7f9f2c09ec20 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009840 tx=0x7f9f1c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3b58f6b1ae29fa server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:23.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.141+0000 7f9f2a7fc700 1 -- 172.21.15.3:0/3979942606 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c011830 con 0x7f9f2c09e850 2023-10-03T10:17:23.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.141+0000 7f9f2a7fc700 1 -- 172.21.15.3:0/3979942606 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1c011990 con 0x7f9f2c09e850 2023-10-03T10:17:23.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.141+0000 7f9f2a7fc700 1 -- 172.21.15.3:0/3979942606 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c011d20 con 0x7f9f2c09e850 2023-10-03T10:17:23.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.142+0000 7f9f321bd700 1 -- 172.21.15.3:0/3979942606 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9f2c09e850 msgr2=0x7f9f2c09ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:23.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.142+0000 7f9f321bd700 1 --2- 172.21.15.3:0/3979942606 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9f2c09e850 0x7f9f2c09ec20 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009840 tx=0x7f9f1c009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:23.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.142+0000 7f9f321bd700 1 -- 172.21.15.3:0/3979942606 shutdown_connections 2023-10-03T10:17:23.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.142+0000 7f9f321bd700 1 --2- 172.21.15.3:0/3979942606 >> v2:172.21.15.3:3300/0 conn(0x7f9f2c131ea0 0x7f9f2c139c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:23.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.142+0000 7f9f321bd700 1 --2- 172.21.15.3:0/3979942606 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9f2c09e850 0x7f9f2c09ec20 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:23.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.142+0000 7f9f321bd700 1 -- 172.21.15.3:0/3979942606 >> 172.21.15.3:0/3979942606 conn(0x7f9f2c0a3f70 msgr2=0x7f9f2c0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:23.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.142+0000 7f9f321bd700 1 -- 172.21.15.3:0/3979942606 shutdown_connections 2023-10-03T10:17:23.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.142+0000 7f9f321bd700 1 -- 172.21.15.3:0/3979942606 wait complete. 2023-10-03T10:17:23.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.143+0000 7f9f321bd700 1 Processor -- start 2023-10-03T10:17:23.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.144+0000 7f9f321bd700 1 -- start start 2023-10-03T10:17:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.144+0000 7f9f321bd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9f2c09e850 0x7f9f2c1200d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.144+0000 7f9f321bd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c09f160 0x7f9f2c120610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.144+0000 7f9f321bd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9f2c131ea0 0x7f9f2c120b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.144+0000 7f9f321bd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c121530 con 0x7f9f2c09f160 2023-10-03T10:17:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.144+0000 7f9f321bd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c1240a0 con 0x7f9f2c09e850 2023-10-03T10:17:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.144+0000 7f9f321bd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c123600 con 0x7f9f2c131ea0 2023-10-03T10:17:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.145+0000 7f9f2affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c09f160 0x7f9f2c120610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.145+0000 7f9f2b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9f2c09e850 0x7f9f2c1200d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:23.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.145+0000 7f9f2affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c09f160 0x7f9f2c120610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41740/0 (socket says 172.21.15.3:41740) 2023-10-03T10:17:23.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.145+0000 7f9f2bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9f2c131ea0 0x7f9f2c120b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:23.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.145+0000 7f9f2affd700 1 -- 172.21.15.3:0/170784240 learned_addr learned my addr 172.21.15.3:0/170784240 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:23.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.145+0000 7f9f2bfff700 1 -- 172.21.15.3:0/170784240 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9f2c09e850 msgr2=0x7f9f2c1200d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:23.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.145+0000 7f9f2bfff700 1 --2- 172.21.15.3:0/170784240 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9f2c09e850 0x7f9f2c1200d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:23.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.145+0000 7f9f2bfff700 1 -- 172.21.15.3:0/170784240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c09f160 msgr2=0x7f9f2c120610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:23.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.145+0000 7f9f2bfff700 1 --2- 172.21.15.3:0/170784240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c09f160 0x7f9f2c120610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:23.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.145+0000 7f9f2bfff700 1 -- 172.21.15.3:0/170784240 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1c0094a0 con 0x7f9f2c131ea0 2023-10-03T10:17:23.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.145+0000 7f9f2affd700 1 --2- 172.21.15.3:0/170784240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c09f160 0x7f9f2c120610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:23.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.145+0000 7f9f2bfff700 1 --2- 172.21.15.3:0/170784240 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9f2c131ea0 0x7f9f2c120b50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9f24007f20 tx=0x7f9f2400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:23.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.146+0000 7f9f13fff700 1 -- 172.21.15.3:0/170784240 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2400dd80 con 0x7f9f2c131ea0 2023-10-03T10:17:23.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.146+0000 7f9f13fff700 1 -- 172.21.15.3:0/170784240 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f24016070 con 0x7f9f2c131ea0 2023-10-03T10:17:23.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.146+0000 7f9f321bd700 1 -- 172.21.15.3:0/170784240 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2c122f50 con 0x7f9f2c131ea0 2023-10-03T10:17:23.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.146+0000 7f9f13fff700 1 -- 172.21.15.3:0/170784240 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2400dee0 con 0x7f9f2c131ea0 2023-10-03T10:17:23.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.146+0000 7f9f321bd700 1 -- 172.21.15.3:0/170784240 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2c121c90 con 0x7f9f2c131ea0 2023-10-03T10:17:23.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.147+0000 7f9f13fff700 1 -- 172.21.15.3:0/170784240 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9f24003c60 con 0x7f9f2c131ea0 2023-10-03T10:17:23.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.147+0000 7f9f13fff700 1 -- 172.21.15.3:0/170784240 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9f24018070 con 0x7f9f2c131ea0 2023-10-03T10:17:23.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.147+0000 7f9f321bd700 1 -- 172.21.15.3:0/170784240 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f2c11d4e0 con 0x7f9f2c131ea0 2023-10-03T10:17:23.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.152+0000 7f9f13fff700 1 -- 172.21.15.3:0/170784240 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9f2401bbc0 con 0x7f9f2c131ea0 2023-10-03T10:17:23.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.218+0000 7fbb95ba7700 1 Processor -- start 2023-10-03T10:17:23.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.218+0000 7fbb95ba7700 1 -- start start 2023-10-03T10:17:23.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.219+0000 7fbb95ba7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbb901305f0 0x7fbb901309c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:23.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.219+0000 7fbb95ba7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb9006f660 0x7fbb9006fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:23.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.219+0000 7fbb95ba7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb9006ff90 0x7fbb90137530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:23.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.219+0000 7fbb95ba7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb901301c0 con 0x7fbb901305f0 2023-10-03T10:17:23.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.219+0000 7fbb95ba7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb90071ff0 con 0x7fbb9006f660 2023-10-03T10:17:23.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.219+0000 7fbb95ba7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb90070e70 con 0x7fbb9006ff90 2023-10-03T10:17:23.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.219+0000 7fbb8effd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb9006f660 0x7fbb9006fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:23.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.219+0000 7fbb8f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbb901305f0 0x7fbb901309c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:23.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.219+0000 7fbb8effd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb9006f660 0x7fbb9006fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:35810/0 (socket says 172.21.15.3:35810) 2023-10-03T10:17:23.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.219+0000 7fbb8effd700 1 -- 172.21.15.3:0/4053622364 learned_addr learned my addr 172.21.15.3:0/4053622364 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:23.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.219+0000 7fbb8ffff700 1 --2- 172.21.15.3:0/4053622364 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb9006ff90 0x7fbb90137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:23.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.220+0000 7fbb8f7fe700 1 -- 172.21.15.3:0/4053622364 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb9006ff90 msgr2=0x7fbb90137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:23.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.220+0000 7fbb8f7fe700 1 --2- 172.21.15.3:0/4053622364 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb9006ff90 0x7fbb90137530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:23.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.220+0000 7fbb8f7fe700 1 -- 172.21.15.3:0/4053622364 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb9006f660 msgr2=0x7fbb9006fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:23.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.220+0000 7fbb8f7fe700 1 --2- 172.21.15.3:0/4053622364 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb9006f660 0x7fbb9006fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:23.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.220+0000 7fbb8f7fe700 1 -- 172.21.15.3:0/4053622364 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb900722b0 con 0x7fbb901305f0 2023-10-03T10:17:23.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.220+0000 7fbb8effd700 1 --2- 172.21.15.3:0/4053622364 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb9006f660 0x7fbb9006fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:23.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.220+0000 7fbb8ffff700 1 --2- 172.21.15.3:0/4053622364 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb9006ff90 0x7fbb90137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:23.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.220+0000 7fbb8f7fe700 1 --2- 172.21.15.3:0/4053622364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb901305f0 0x7fbb901309c0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000dc60 tx=0x7fbb8000df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e2ae49bcb7af86 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:23.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.220+0000 7fbb8e7fc700 1 -- 172.21.15.3:0/4053622364 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb80012070 con 0x7fbb901305f0 2023-10-03T10:17:23.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.220+0000 7fbb8e7fc700 1 -- 172.21.15.3:0/4053622364 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8000d2e0 con 0x7fbb901305f0 2023-10-03T10:17:23.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.220+0000 7fbb8e7fc700 1 -- 172.21.15.3:0/4053622364 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8000f900 con 0x7fbb901305f0 2023-10-03T10:17:23.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.221+0000 7fbb95ba7700 1 -- 172.21.15.3:0/4053622364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb901305f0 msgr2=0x7fbb901309c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:23.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.221+0000 7fbb95ba7700 1 --2- 172.21.15.3:0/4053622364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb901305f0 0x7fbb901309c0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000dc60 tx=0x7fbb8000df70 comp rx=0 tx=0).stop 2023-10-03T10:17:23.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.221+0000 7fbb95ba7700 1 -- 172.21.15.3:0/4053622364 shutdown_connections 2023-10-03T10:17:23.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.221+0000 7fbb95ba7700 1 --2- 172.21.15.3:0/4053622364 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb9006ff90 0x7fbb90137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:23.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.221+0000 7fbb95ba7700 1 --2- 172.21.15.3:0/4053622364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb901305f0 0x7fbb901309c0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:23.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.221+0000 7fbb95ba7700 1 --2- 172.21.15.3:0/4053622364 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb9006f660 0x7fbb9006fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:23.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.221+0000 7fbb95ba7700 1 -- 172.21.15.3:0/4053622364 >> 172.21.15.3:0/4053622364 conn(0x7fbb900a2aa0 msgr2=0x7fbb900a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:23.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.221+0000 7fbb95ba7700 1 -- 172.21.15.3:0/4053622364 shutdown_connections 2023-10-03T10:17:23.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.221+0000 7fbb95ba7700 1 -- 172.21.15.3:0/4053622364 wait complete. 2023-10-03T10:17:23.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.223+0000 7fbb95ba7700 1 Processor -- start 2023-10-03T10:17:23.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.223+0000 7fbb95ba7700 1 -- start start 2023-10-03T10:17:23.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.224+0000 7fbb95ba7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb9006f660 0x7fbb9011b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:23.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.224+0000 7fbb95ba7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb9006ff90 0x7fbb9011be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:23.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.224+0000 7fbb95ba7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb901305f0 0x7fbb9011c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:23.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.224+0000 7fbb95ba7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9011f420 con 0x7fbb9006f660 2023-10-03T10:17:23.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.224+0000 7fbb95ba7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9012ecf0 con 0x7fbb9006ff90 2023-10-03T10:17:23.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.224+0000 7fbb95ba7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb90071820 con 0x7fbb901305f0 2023-10-03T10:17:23.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.224+0000 7fbb8effd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb9006ff90 0x7fbb9011be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:23.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.224+0000 7fbb8f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb9006f660 0x7fbb9011b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:23.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.224+0000 7fbb8effd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb9006ff90 0x7fbb9011be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:35820/0 (socket says 172.21.15.3:35820) 2023-10-03T10:17:23.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.224+0000 7fbb8effd700 1 -- 172.21.15.3:0/301498745 learned_addr learned my addr 172.21.15.3:0/301498745 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:23.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.224+0000 7fbb8f7fe700 1 --2- 172.21.15.3:0/301498745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb9006f660 0x7fbb9011b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40162/0 (socket says 172.21.15.3:40162) 2023-10-03T10:17:23.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.224+0000 7fbb8ffff700 1 --2- 172.21.15.3:0/301498745 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb901305f0 0x7fbb9011c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:23.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.224+0000 7fbb8effd700 1 -- 172.21.15.3:0/301498745 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb901305f0 msgr2=0x7fbb9011c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:23.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.224+0000 7fbb8effd700 1 --2- 172.21.15.3:0/301498745 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb901305f0 0x7fbb9011c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:23.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.224+0000 7fbb8effd700 1 -- 172.21.15.3:0/301498745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb9006f660 msgr2=0x7fbb9011b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:23.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.224+0000 7fbb8effd700 1 --2- 172.21.15.3:0/301498745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb9006f660 0x7fbb9011b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:23.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.224+0000 7fbb8effd700 1 -- 172.21.15.3:0/301498745 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8400b600 con 0x7fbb9006ff90 2023-10-03T10:17:23.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.224+0000 7fbb8f7fe700 1 --2- 172.21.15.3:0/301498745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb9006f660 0x7fbb9011b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:23.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.224+0000 7fbb8effd700 1 --2- 172.21.15.3:0/301498745 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb9006ff90 0x7fbb9011be40 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400fde0 tx=0x7fbb8400d630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:23.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.225+0000 7fbb77fff700 1 -- 172.21.15.3:0/301498745 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8400de40 con 0x7fbb9006ff90 2023-10-03T10:17:23.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.225+0000 7fbb77fff700 1 -- 172.21.15.3:0/301498745 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8400be70 con 0x7fbb9006ff90 2023-10-03T10:17:23.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.225+0000 7fbb77fff700 1 -- 172.21.15.3:0/301498745 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84018640 con 0x7fbb9006ff90 2023-10-03T10:17:23.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.225+0000 7fbb95ba7700 1 -- 172.21.15.3:0/301498745 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb88009a70 con 0x7fbb9006ff90 2023-10-03T10:17:23.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.225+0000 7fbb95ba7700 1 -- 172.21.15.3:0/301498745 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb8000d940 con 0x7fbb9006ff90 2023-10-03T10:17:23.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.226+0000 7fbb77fff700 1 -- 172.21.15.3:0/301498745 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbb840187a0 con 0x7fbb9006ff90 2023-10-03T10:17:23.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.226+0000 7fbb95ba7700 1 -- 172.21.15.3:0/301498745 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb9011e7f0 con 0x7fbb9006ff90 2023-10-03T10:17:23.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.226+0000 7fbb77fff700 1 -- 172.21.15.3:0/301498745 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fbb8403fd90 con 0x7fbb9006ff90 2023-10-03T10:17:23.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.231+0000 7fbb77fff700 1 -- 172.21.15.3:0/301498745 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbb84022440 con 0x7fbb9006ff90 2023-10-03T10:17:23.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.268+0000 7fa9b4f52700 1 Processor -- start 2023-10-03T10:17:23.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.269+0000 7fa9b4f52700 1 -- start start 2023-10-03T10:17:23.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.269+0000 7fa9b4f52700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa9b00a0a30 0x7fa9b00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:23.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.269+0000 7fa9b4f52700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9b009d630 0x7fa9b00a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:23.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.269+0000 7fa9b4f52700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9b00a2a70 con 0x7fa9b0133930 2023-10-03T10:17:23.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.269+0000 7fa9b4f52700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9b00a01d0 con 0x7fa9b00a0a30 2023-10-03T10:17:23.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.269+0000 7fa9b4f52700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b009e2b0 con 0x7fa9b009d630 2023-10-03T10:17:23.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.269+0000 7fa9a7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9b009d630 0x7fa9b00a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:23.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.269+0000 7fa9a7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9b009d630 0x7fa9b00a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:39726/0 (socket says 172.21.15.3:39726) 2023-10-03T10:17:23.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.269+0000 7fa9a7fff700 1 -- 172.21.15.3:0/2297360936 learned_addr learned my addr 172.21.15.3:0/2297360936 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:23.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.269+0000 7fa9ae59c700 1 --2- 172.21.15.3:0/2297360936 >> v2:172.21.15.3:3300/0 conn(0x7fa9b00a0a30 0x7fa9b00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:23.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.270+0000 7fa9a7fff700 1 -- 172.21.15.3:0/2297360936 >> v2:172.21.15.3:3300/0 conn(0x7fa9b00a0a30 msgr2=0x7fa9b00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:23.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.270+0000 7fa9a7fff700 1 --2- 172.21.15.3:0/2297360936 >> v2:172.21.15.3:3300/0 conn(0x7fa9b00a0a30 0x7fa9b00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.270+0000 7fa9a7fff700 1 -- 172.21.15.3:0/2297360936 >> v1:172.21.15.3:6789/0 conn(0x7fa9b0133930 legacy=0x7fa9b0133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.270+0000 7fa9a7fff700 1 -- 172.21.15.3:0/2297360936 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b009dc50 con 0x7fa9b009d630 2023-10-03T10:17:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.270+0000 7fa9a7fff700 1 --2- 172.21.15.3:0/2297360936 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9b009d630 0x7fa9b00a1340 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa9980098f0 tx=0x7fa998009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecc49188074e6e50 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.270+0000 7fa9add9b700 1 -- 172.21.15.3:0/2297360936 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa998011830 con 0x7fa9b009d630 2023-10-03T10:17:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.270+0000 7fa9add9b700 1 -- 172.21.15.3:0/2297360936 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa998011990 con 0x7fa9b009d630 2023-10-03T10:17:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.270+0000 7fa9add9b700 1 -- 172.21.15.3:0/2297360936 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa998011d20 con 0x7fa9b009d630 2023-10-03T10:17:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.271+0000 7fa9b4f52700 1 -- 172.21.15.3:0/2297360936 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9b009d630 msgr2=0x7fa9b00a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.271+0000 7fa9b4f52700 1 --2- 172.21.15.3:0/2297360936 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9b009d630 0x7fa9b00a1340 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa9980098f0 tx=0x7fa998009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.271+0000 7fa9b4f52700 1 -- 172.21.15.3:0/2297360936 shutdown_connections 2023-10-03T10:17:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.271+0000 7fa9b4f52700 1 --2- 172.21.15.3:0/2297360936 >> v2:172.21.15.3:3300/0 conn(0x7fa9b00a0a30 0x7fa9b00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.271+0000 7fa9b4f52700 1 --2- 172.21.15.3:0/2297360936 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9b009d630 0x7fa9b00a1340 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.271+0000 7fa9b4f52700 1 -- 172.21.15.3:0/2297360936 >> 172.21.15.3:0/2297360936 conn(0x7fa9b00a3050 msgr2=0x7fa9b00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:23.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.271+0000 7fa9b4f52700 1 -- 172.21.15.3:0/2297360936 shutdown_connections 2023-10-03T10:17:23.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.271+0000 7fa9b4f52700 1 -- 172.21.15.3:0/2297360936 wait complete. 2023-10-03T10:17:23.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.272+0000 7fa9b4f52700 1 Processor -- start 2023-10-03T10:17:23.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.272+0000 7fa9b4f52700 1 -- start start 2023-10-03T10:17:23.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.273+0000 7fa9b4f52700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9b009d630 0x7fa9b011f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:23.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.273+0000 7fa9b4f52700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b00a0a30 0x7fa9b011f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:23.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.273+0000 7fa9b4f52700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9b0133930 0x7fa9b01c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:23.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.273+0000 7fa9b4f52700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b0121fb0 con 0x7fa9b00a0a30 2023-10-03T10:17:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.273+0000 7fa9b4f52700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b011eef0 con 0x7fa9b009d630 2023-10-03T10:17:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.273+0000 7fa9b4f52700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b0120640 con 0x7fa9b0133930 2023-10-03T10:17:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.273+0000 7fa9a7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b00a0a30 0x7fa9b011f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.273+0000 7fa9a7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b00a0a30 0x7fa9b011f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40182/0 (socket says 172.21.15.3:40182) 2023-10-03T10:17:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.273+0000 7fa9a7fff700 1 -- 172.21.15.3:0/3674196725 learned_addr learned my addr 172.21.15.3:0/3674196725 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.273+0000 7fa9aed9d700 1 --2- 172.21.15.3:0/3674196725 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9b0133930 0x7fa9b01c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.273+0000 7fa9a7fff700 1 -- 172.21.15.3:0/3674196725 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9b0133930 msgr2=0x7fa9b01c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.273+0000 7fa9a7fff700 1 --2- 172.21.15.3:0/3674196725 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9b0133930 0x7fa9b01c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.273+0000 7fa9a7fff700 1 -- 172.21.15.3:0/3674196725 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9b009d630 msgr2=0x7fa9b011f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.273+0000 7fa9a7fff700 1 --2- 172.21.15.3:0/3674196725 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9b009d630 0x7fa9b011f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.273+0000 7fa9a7fff700 1 -- 172.21.15.3:0/3674196725 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa998009550 con 0x7fa9b00a0a30 2023-10-03T10:17:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.273+0000 7fa9a7fff700 1 --2- 172.21.15.3:0/3674196725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b00a0a30 0x7fa9b011f9d0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fa998000c00 tx=0x7fa998004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.274+0000 7fa9a6ffd700 1 -- 172.21.15.3:0/3674196725 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9980036a0 con 0x7fa9b00a0a30 2023-10-03T10:17:23.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.274+0000 7fa9b4f52700 1 -- 172.21.15.3:0/3674196725 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9b00a5040 con 0x7fa9b00a0a30 2023-10-03T10:17:23.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.274+0000 7fa9b4f52700 1 -- 172.21.15.3:0/3674196725 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b01334e0 con 0x7fa9b00a0a30 2023-10-03T10:17:23.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.274+0000 7fa9a6ffd700 1 -- 172.21.15.3:0/3674196725 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa998003800 con 0x7fa9b00a0a30 2023-10-03T10:17:23.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.274+0000 7fa9a6ffd700 1 -- 172.21.15.3:0/3674196725 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa998003bb0 con 0x7fa9b00a0a30 2023-10-03T10:17:23.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.275+0000 7fa9a6ffd700 1 -- 172.21.15.3:0/3674196725 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa998021070 con 0x7fa9b00a0a30 2023-10-03T10:17:23.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.275+0000 7fa9a4ff9700 1 -- 172.21.15.3:0/3674196725 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9b011ff10 con 0x7fa9b00a0a30 2023-10-03T10:17:23.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.275+0000 7fa9a6ffd700 1 -- 172.21.15.3:0/3674196725 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa9980410c0 con 0x7fa9b00a0a30 2023-10-03T10:17:23.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.280+0000 7fa9a6ffd700 1 -- 172.21.15.3:0/3674196725 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa998010380 con 0x7fa9b00a0a30 2023-10-03T10:17:23.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.367+0000 7f9f321bd700 1 -- 172.21.15.3:0/170784240 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9f2c123b00 con 0x7f9f2c131ea0 2023-10-03T10:17:23.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.367+0000 7f9f13fff700 1 -- 172.21.15.3:0/170784240 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9f2401b990 con 0x7f9f2c131ea0 2023-10-03T10:17:23.367 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:23.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.370+0000 7f9f321bd700 1 -- 172.21.15.3:0/170784240 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9f2c131ea0 msgr2=0x7f9f2c120b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:23.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.370+0000 7f9f321bd700 1 --2- 172.21.15.3:0/170784240 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9f2c131ea0 0x7f9f2c120b50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9f24007f20 tx=0x7f9f2400d3b0 comp rx=0 tx=0).stop 2023-10-03T10:17:23.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.370+0000 7f9f321bd700 1 -- 172.21.15.3:0/170784240 shutdown_connections 2023-10-03T10:17:23.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.370+0000 7f9f321bd700 1 --2- 172.21.15.3:0/170784240 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9f2c131ea0 0x7f9f2c120b50 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:23.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.370+0000 7f9f321bd700 1 --2- 172.21.15.3:0/170784240 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9f2c09e850 0x7f9f2c1200d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:23.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.370+0000 7f9f321bd700 1 --2- 172.21.15.3:0/170784240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c09f160 0x7f9f2c120610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:23.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.370+0000 7f9f321bd700 1 -- 172.21.15.3:0/170784240 >> 172.21.15.3:0/170784240 conn(0x7f9f2c0a3f70 msgr2=0x7f9f2c11a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:23.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.370+0000 7f9f321bd700 1 -- 172.21.15.3:0/170784240 shutdown_connections 2023-10-03T10:17:23.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.370+0000 7f9f321bd700 1 -- 172.21.15.3:0/170784240 wait complete. 2023-10-03T10:17:23.384 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:23.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.444+0000 7fbb95ba7700 1 -- 172.21.15.3:0/301498745 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbb9011ec70 con 0x7fbb9006ff90 2023-10-03T10:17:23.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.444+0000 7fbb77fff700 1 -- 172.21.15.3:0/301498745 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbb84016b30 con 0x7fbb9006ff90 2023-10-03T10:17:23.445 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:23.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.447+0000 7fbb95ba7700 1 -- 172.21.15.3:0/301498745 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb9006ff90 msgr2=0x7fbb9011be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:23.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.447+0000 7fbb95ba7700 1 --2- 172.21.15.3:0/301498745 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb9006ff90 0x7fbb9011be40 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400fde0 tx=0x7fbb8400d630 comp rx=0 tx=0).stop 2023-10-03T10:17:23.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.447+0000 7fbb95ba7700 1 -- 172.21.15.3:0/301498745 shutdown_connections 2023-10-03T10:17:23.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.447+0000 7fbb95ba7700 1 --2- 172.21.15.3:0/301498745 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbb901305f0 0x7fbb9011c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:23.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.447+0000 7fbb95ba7700 1 --2- 172.21.15.3:0/301498745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb9006f660 0x7fbb9011b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:23.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.447+0000 7fbb95ba7700 1 --2- 172.21.15.3:0/301498745 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbb9006ff90 0x7fbb9011be40 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:23.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.447+0000 7fbb95ba7700 1 -- 172.21.15.3:0/301498745 >> 172.21.15.3:0/301498745 conn(0x7fbb900a2aa0 msgr2=0x7fbb9006e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:23.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.447+0000 7fbb95ba7700 1 -- 172.21.15.3:0/301498745 shutdown_connections 2023-10-03T10:17:23.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.447+0000 7fbb95ba7700 1 -- 172.21.15.3:0/301498745 wait complete. 2023-10-03T10:17:23.460 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:23.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.491+0000 7fa9a4ff9700 1 -- 172.21.15.3:0/3674196725 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa9b00747f0 con 0x7fa9b00a0a30 2023-10-03T10:17:23.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.491+0000 7fa9a6ffd700 1 -- 172.21.15.3:0/3674196725 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa998015030 con 0x7fa9b00a0a30 2023-10-03T10:17:23.491 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:23.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.493+0000 7fa9a4ff9700 1 -- 172.21.15.3:0/3674196725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b00a0a30 msgr2=0x7fa9b011f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:23.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.493+0000 7fa9a4ff9700 1 --2- 172.21.15.3:0/3674196725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b00a0a30 0x7fa9b011f9d0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fa998000c00 tx=0x7fa998004750 comp rx=0 tx=0).stop 2023-10-03T10:17:23.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.493+0000 7fa9a4ff9700 1 -- 172.21.15.3:0/3674196725 shutdown_connections 2023-10-03T10:17:23.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.493+0000 7fa9a4ff9700 1 --2- 172.21.15.3:0/3674196725 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9b0133930 0x7fa9b01c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:23.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.493+0000 7fa9a4ff9700 1 --2- 172.21.15.3:0/3674196725 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9b009d630 0x7fa9b011f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:23.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.493+0000 7fa9a4ff9700 1 --2- 172.21.15.3:0/3674196725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b00a0a30 0x7fa9b011f9d0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:23.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.493+0000 7fa9a4ff9700 1 -- 172.21.15.3:0/3674196725 >> 172.21.15.3:0/3674196725 conn(0x7fa9b00a3050 msgr2=0x7fa9b01bbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:23.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.493+0000 7fa9a4ff9700 1 -- 172.21.15.3:0/3674196725 shutdown_connections 2023-10-03T10:17:23.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:23.493+0000 7fa9a4ff9700 1 -- 172.21.15.3:0/3674196725 wait complete. 2023-10-03T10:17:23.507 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:23.978 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:24.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.124+0000 7f70c4328700 1 Processor -- start 2023-10-03T10:17:24.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.124+0000 7f70c4328700 1 -- start start 2023-10-03T10:17:24.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.125+0000 7f70c4328700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70bc133a80 0x7f70bc133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:24.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.125+0000 7f70c4328700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f70bc12ff20 0x7f70bc130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:24.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.125+0000 7f70c4328700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70bc06bc00 con 0x7f70bc130870 2023-10-03T10:17:24.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.125+0000 7f70c4328700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70bc06c570 con 0x7f70bc12ff20 2023-10-03T10:17:24.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.125+0000 7f70c4328700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc0a0040 con 0x7f70bc133a80 2023-10-03T10:17:24.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.125+0000 7f70c18c3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f70bc12ff20 0x7f70bc130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:24.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.125+0000 7f70c20c4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70bc133a80 0x7f70bc133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:24.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.125+0000 7f70c18c3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f70bc12ff20 0x7f70bc130330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40196/0 (socket says 172.21.15.3:40196) 2023-10-03T10:17:24.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.125+0000 7f70c18c3700 1 -- 172.21.15.3:0/678528848 learned_addr learned my addr 172.21.15.3:0/678528848 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:24.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.125+0000 7f70c28c5700 1 --1- 172.21.15.3:0/678528848 >> v1:172.21.15.3:6789/0 conn(0x7f70bc130870 0x7f70bc13a840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45368/0 (socket says 172.21.15.3:45368) 2023-10-03T10:17:24.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.126+0000 7f70c18c3700 1 -- 172.21.15.3:0/678528848 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70bc133a80 msgr2=0x7f70bc133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:24.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.126+0000 7f70c18c3700 1 --2- 172.21.15.3:0/678528848 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70bc133a80 0x7f70bc133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.126+0000 7f70c18c3700 1 -- 172.21.15.3:0/678528848 >> v1:172.21.15.3:6789/0 conn(0x7f70bc130870 legacy=0x7f70bc13a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:24.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.126+0000 7f70c18c3700 1 -- 172.21.15.3:0/678528848 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70bc133860 con 0x7f70bc12ff20 2023-10-03T10:17:24.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.126+0000 7f70c20c4700 1 --2- 172.21.15.3:0/678528848 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70bc133a80 0x7f70bc133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:24.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.126+0000 7f70c18c3700 1 --2- 172.21.15.3:0/678528848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70bc12ff20 0x7f70bc130330 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f70ac0098f0 tx=0x7f70ac009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6f095b3fe2e4be server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:24.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.126+0000 7f70c10c2700 1 -- 172.21.15.3:0/678528848 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac011830 con 0x7f70bc12ff20 2023-10-03T10:17:24.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.126+0000 7f70c10c2700 1 -- 172.21.15.3:0/678528848 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ac011990 con 0x7f70bc12ff20 2023-10-03T10:17:24.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.126+0000 7f70c10c2700 1 -- 172.21.15.3:0/678528848 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac011d20 con 0x7f70bc12ff20 2023-10-03T10:17:24.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.127+0000 7f70c4328700 1 -- 172.21.15.3:0/678528848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70bc12ff20 msgr2=0x7f70bc130330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:24.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.127+0000 7f70c4328700 1 --2- 172.21.15.3:0/678528848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70bc12ff20 0x7f70bc130330 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f70ac0098f0 tx=0x7f70ac009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:24.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.127+0000 7f70c4328700 1 -- 172.21.15.3:0/678528848 shutdown_connections 2023-10-03T10:17:24.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.127+0000 7f70c4328700 1 --2- 172.21.15.3:0/678528848 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70bc133a80 0x7f70bc133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.127+0000 7f70c4328700 1 --2- 172.21.15.3:0/678528848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70bc12ff20 0x7f70bc130330 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.127+0000 7f70c4328700 1 -- 172.21.15.3:0/678528848 >> 172.21.15.3:0/678528848 conn(0x7f70bc0a4210 msgr2=0x7f70bc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:24.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.127+0000 7f70c4328700 1 -- 172.21.15.3:0/678528848 shutdown_connections 2023-10-03T10:17:24.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.127+0000 7f70c4328700 1 -- 172.21.15.3:0/678528848 wait complete. 2023-10-03T10:17:24.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.128+0000 7f70c4328700 1 Processor -- start 2023-10-03T10:17:24.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.128+0000 7f70c4328700 1 -- start start 2023-10-03T10:17:24.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.128+0000 7f70c4328700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70bc12ff20 0x7f70bc09c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:24.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.128+0000 7f70c4328700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70bc130870 0x7f70bc09ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:24.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.128+0000 7f70c4328700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70bc133a80 0x7f70bc09cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:24.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.128+0000 7f70c4328700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc0a1fa0 con 0x7f70bc130870 2023-10-03T10:17:24.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.128+0000 7f70c4328700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc09e0a0 con 0x7f70bc12ff20 2023-10-03T10:17:24.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.128+0000 7f70c4328700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc120040 con 0x7f70bc133a80 2023-10-03T10:17:24.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.128+0000 7f70c20c4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70bc12ff20 0x7f70bc09c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:24.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.128+0000 7f70c18c3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70bc130870 0x7f70bc09ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:24.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.128+0000 7f70c18c3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70bc130870 0x7f70bc09ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40210/0 (socket says 172.21.15.3:40210) 2023-10-03T10:17:24.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.128+0000 7f70c18c3700 1 -- 172.21.15.3:0/1965602699 learned_addr learned my addr 172.21.15.3:0/1965602699 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:24.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.128+0000 7f70c28c5700 1 --2- 172.21.15.3:0/1965602699 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70bc133a80 0x7f70bc09cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:24.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.129+0000 7f70c20c4700 1 -- 172.21.15.3:0/1965602699 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70bc133a80 msgr2=0x7f70bc09cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:24.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.129+0000 7f70c20c4700 1 --2- 172.21.15.3:0/1965602699 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70bc133a80 0x7f70bc09cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.129+0000 7f70c20c4700 1 -- 172.21.15.3:0/1965602699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70bc130870 msgr2=0x7f70bc09ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:24.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.129+0000 7f70c20c4700 1 --2- 172.21.15.3:0/1965602699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70bc130870 0x7f70bc09ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.129+0000 7f70c20c4700 1 -- 172.21.15.3:0/1965602699 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b80094a0 con 0x7f70bc12ff20 2023-10-03T10:17:24.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.129+0000 7f70c18c3700 1 --2- 172.21.15.3:0/1965602699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70bc130870 0x7f70bc09ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:24.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.129+0000 7f70c20c4700 1 --2- 172.21.15.3:0/1965602699 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70bc12ff20 0x7f70bc09c550 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f70b800e9e0 tx=0x7f70b800ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:24.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.129+0000 7f70b2ffd700 1 -- 172.21.15.3:0/1965602699 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b800ee80 con 0x7f70bc12ff20 2023-10-03T10:17:24.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.129+0000 7f70b2ffd700 1 -- 172.21.15.3:0/1965602699 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b800c850 con 0x7f70bc12ff20 2023-10-03T10:17:24.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.129+0000 7f70c4328700 1 -- 172.21.15.3:0/1965602699 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70ac009550 con 0x7f70bc12ff20 2023-10-03T10:17:24.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.129+0000 7f70b2ffd700 1 -- 172.21.15.3:0/1965602699 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b800cc00 con 0x7f70bc12ff20 2023-10-03T10:17:24.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.129+0000 7f70c4328700 1 -- 172.21.15.3:0/1965602699 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70bc11fd80 con 0x7f70bc12ff20 2023-10-03T10:17:24.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.130+0000 7f70b2ffd700 1 -- 172.21.15.3:0/1965602699 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f70b800cd60 con 0x7f70bc12ff20 2023-10-03T10:17:24.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.130+0000 7f70c4328700 1 -- 172.21.15.3:0/1965602699 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70bc12d7b0 con 0x7f70bc12ff20 2023-10-03T10:17:24.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.130+0000 7f70b2ffd700 1 -- 172.21.15.3:0/1965602699 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f70b803fad0 con 0x7f70bc12ff20 2023-10-03T10:17:24.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.135+0000 7f70b2ffd700 1 -- 172.21.15.3:0/1965602699 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f70bc12d7b0 con 0x7f70bc12ff20 2023-10-03T10:17:24.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.342+0000 7f70c4328700 1 -- 172.21.15.3:0/1965602699 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f70bc12d7b0 con 0x7f70bc12ff20 2023-10-03T10:17:24.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.342+0000 7f70b2ffd700 1 -- 172.21.15.3:0/1965602699 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f70bc12d7b0 con 0x7f70bc12ff20 2023-10-03T10:17:24.343 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:24.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.344+0000 7f70c4328700 1 -- 172.21.15.3:0/1965602699 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70bc12ff20 msgr2=0x7f70bc09c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:24.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.344+0000 7f70c4328700 1 --2- 172.21.15.3:0/1965602699 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70bc12ff20 0x7f70bc09c550 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f70b800e9e0 tx=0x7f70b800ecf0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.344+0000 7f70c4328700 1 -- 172.21.15.3:0/1965602699 shutdown_connections 2023-10-03T10:17:24.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.344+0000 7f70c4328700 1 --2- 172.21.15.3:0/1965602699 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f70bc133a80 0x7f70bc09cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.344+0000 7f70c4328700 1 --2- 172.21.15.3:0/1965602699 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f70bc12ff20 0x7f70bc09c550 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.344+0000 7f70c4328700 1 --2- 172.21.15.3:0/1965602699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70bc130870 0x7f70bc09ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.345+0000 7f70c4328700 1 -- 172.21.15.3:0/1965602699 >> 172.21.15.3:0/1965602699 conn(0x7f70bc0a4210 msgr2=0x7f70bc09e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:24.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.345+0000 7f70c4328700 1 -- 172.21.15.3:0/1965602699 shutdown_connections 2023-10-03T10:17:24.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.345+0000 7f70c4328700 1 -- 172.21.15.3:0/1965602699 wait complete. 2023-10-03T10:17:24.359 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:24.385 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:24.462 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:24.508 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:24.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.526+0000 7fd55617b700 1 Processor -- start 2023-10-03T10:17:24.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.526+0000 7fd55617b700 1 -- start start 2023-10-03T10:17:24.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.527+0000 7fd55617b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd5501338a0 0x7fd550133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:24.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.527+0000 7fd55617b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd55009de50 0x7fd55009e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:24.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.527+0000 7fd55617b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd55009e760 0x7fd55013a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:24.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.527+0000 7fd55617b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd55006fef0 con 0x7fd55009e760 2023-10-03T10:17:24.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.527+0000 7fd55617b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd55006ef30 con 0x7fd55009de50 2023-10-03T10:17:24.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.527+0000 7fd55617b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5501334b0 con 0x7fd5501338a0 2023-10-03T10:17:24.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.527+0000 7fd54effd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd55009de50 0x7fd55009e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:24.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.527+0000 7fd54f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd5501338a0 0x7fd550133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:24.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.527+0000 7fd54f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd5501338a0 0x7fd550133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:39754/0 (socket says 172.21.15.3:39754) 2023-10-03T10:17:24.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.527+0000 7fd54effd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd55009de50 0x7fd55009e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:35866/0 (socket says 172.21.15.3:35866) 2023-10-03T10:17:24.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.527+0000 7fd54f7fe700 1 -- 172.21.15.3:0/3669503330 learned_addr learned my addr 172.21.15.3:0/3669503330 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:24.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.527+0000 7fd54ffff700 1 --2- 172.21.15.3:0/3669503330 >> v2:172.21.15.3:3300/0 conn(0x7fd55009e760 0x7fd55013a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:24.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.527+0000 7fd54effd700 1 -- 172.21.15.3:0/3669503330 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd5501338a0 msgr2=0x7fd550133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:24.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.527+0000 7fd54effd700 1 --2- 172.21.15.3:0/3669503330 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd5501338a0 0x7fd550133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.527+0000 7fd54effd700 1 -- 172.21.15.3:0/3669503330 >> v2:172.21.15.3:3300/0 conn(0x7fd55009e760 msgr2=0x7fd55013a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:24.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.527+0000 7fd54effd700 1 --2- 172.21.15.3:0/3669503330 >> v2:172.21.15.3:3300/0 conn(0x7fd55009e760 0x7fd55013a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.527+0000 7fd54effd700 1 -- 172.21.15.3:0/3669503330 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55006fc30 con 0x7fd55009de50 2023-10-03T10:17:24.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.527+0000 7fd54ffff700 1 --2- 172.21.15.3:0/3669503330 >> v2:172.21.15.3:3300/0 conn(0x7fd55009e760 0x7fd55013a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:17:24.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.528+0000 7fd54effd700 1 --2- 172.21.15.3:0/3669503330 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd55009de50 0x7fd55009e220 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fd544009870 tx=0x7fd544009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55bb45961cbf3de5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:24.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.528+0000 7fd54e7fc700 1 -- 172.21.15.3:0/3669503330 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd544011830 con 0x7fd55009de50 2023-10-03T10:17:24.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.528+0000 7fd54e7fc700 1 -- 172.21.15.3:0/3669503330 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd544011990 con 0x7fd55009de50 2023-10-03T10:17:24.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.528+0000 7fd54e7fc700 1 -- 172.21.15.3:0/3669503330 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd544011d20 con 0x7fd55009de50 2023-10-03T10:17:24.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.529+0000 7fd55617b700 1 -- 172.21.15.3:0/3669503330 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd55009de50 msgr2=0x7fd55009e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:24.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.529+0000 7fd55617b700 1 --2- 172.21.15.3:0/3669503330 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd55009de50 0x7fd55009e220 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fd544009870 tx=0x7fd544009b80 comp rx=0 tx=0).stop 2023-10-03T10:17:24.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.529+0000 7fd55617b700 1 -- 172.21.15.3:0/3669503330 shutdown_connections 2023-10-03T10:17:24.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.529+0000 7fd55617b700 1 --2- 172.21.15.3:0/3669503330 >> v2:172.21.15.3:3300/0 conn(0x7fd55009e760 0x7fd55013a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.529+0000 7fd55617b700 1 --2- 172.21.15.3:0/3669503330 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd5501338a0 0x7fd550133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.529+0000 7fd55617b700 1 --2- 172.21.15.3:0/3669503330 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd55009de50 0x7fd55009e220 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.529+0000 7fd55617b700 1 -- 172.21.15.3:0/3669503330 >> 172.21.15.3:0/3669503330 conn(0x7fd5500a2aa0 msgr2=0x7fd5500a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:24.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.529+0000 7fd55617b700 1 -- 172.21.15.3:0/3669503330 shutdown_connections 2023-10-03T10:17:24.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.529+0000 7fd55617b700 1 -- 172.21.15.3:0/3669503330 wait complete. 2023-10-03T10:17:24.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.530+0000 7fd55617b700 1 Processor -- start 2023-10-03T10:17:24.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.530+0000 7fd55617b700 1 -- start start 2023-10-03T10:17:24.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.530+0000 7fd55617b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd55009de50 0x7fd55011e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:24.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.531+0000 7fd55617b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd55009e760 0x7fd55011eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:24.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.531+0000 7fd55617b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd5501338a0 0x7fd55011f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:24.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.531+0000 7fd55617b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5500a1020 con 0x7fd55009e760 2023-10-03T10:17:24.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.531+0000 7fd55617b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd55006f640 con 0x7fd55009de50 2023-10-03T10:17:24.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.531+0000 7fd55617b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5501213b0 con 0x7fd5501338a0 2023-10-03T10:17:24.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.531+0000 7fd54effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd55009e760 0x7fd55011eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:24.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.531+0000 7fd54ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd5501338a0 0x7fd55011f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:24.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.531+0000 7fd54effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd55009e760 0x7fd55011eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40220/0 (socket says 172.21.15.3:40220) 2023-10-03T10:17:24.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.531+0000 7fd54effd700 1 -- 172.21.15.3:0/871123939 learned_addr learned my addr 172.21.15.3:0/871123939 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.531+0000 7fd54f7fe700 1 --2- 172.21.15.3:0/871123939 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd55009de50 0x7fd55011e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.531+0000 7fd54ffff700 1 -- 172.21.15.3:0/871123939 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd55009de50 msgr2=0x7fd55011e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.531+0000 7fd54ffff700 1 --2- 172.21.15.3:0/871123939 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd55009de50 0x7fd55011e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.531+0000 7fd54ffff700 1 -- 172.21.15.3:0/871123939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd55009e760 msgr2=0x7fd55011eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.531+0000 7fd54ffff700 1 --2- 172.21.15.3:0/871123939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd55009e760 0x7fd55011eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.531+0000 7fd54ffff700 1 -- 172.21.15.3:0/871123939 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5440094a0 con 0x7fd5501338a0 2023-10-03T10:17:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.531+0000 7fd54effd700 1 --2- 172.21.15.3:0/871123939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd55009e760 0x7fd55011eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.531+0000 7fd54ffff700 1 --2- 172.21.15.3:0/871123939 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd5501338a0 0x7fd55011f400 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd54800cdf0 tx=0x7fd54800ce20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.532+0000 7fd537fff700 1 -- 172.21.15.3:0/871123939 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54800ac80 con 0x7fd5501338a0 2023-10-03T10:17:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.532+0000 7fd537fff700 1 -- 172.21.15.3:0/871123939 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54800ade0 con 0x7fd5501338a0 2023-10-03T10:17:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.532+0000 7fd537fff700 1 -- 172.21.15.3:0/871123939 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548015680 con 0x7fd5501338a0 2023-10-03T10:17:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.532+0000 7fd55617b700 1 -- 172.21.15.3:0/871123939 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd550121c30 con 0x7fd5501338a0 2023-10-03T10:17:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.532+0000 7fd55617b700 1 -- 172.21.15.3:0/871123939 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd550122c30 con 0x7fd5501338a0 2023-10-03T10:17:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.533+0000 7fd55617b700 1 -- 172.21.15.3:0/871123939 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd550074ea0 con 0x7fd5501338a0 2023-10-03T10:17:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.533+0000 7fd537fff700 1 -- 172.21.15.3:0/871123939 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd54800d430 con 0x7fd5501338a0 2023-10-03T10:17:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.533+0000 7fd537fff700 1 -- 172.21.15.3:0/871123939 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd54803fff0 con 0x7fd5501338a0 2023-10-03T10:17:24.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.542+0000 7fd537fff700 1 -- 172.21.15.3:0/871123939 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd548008910 con 0x7fd5501338a0 2023-10-03T10:17:24.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.607+0000 7f2eee2ef700 1 Processor -- start 2023-10-03T10:17:24.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.607+0000 7f2eee2ef700 1 -- start start 2023-10-03T10:17:24.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.608+0000 7f2eee2ef700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ee809e1d0 0x7f2ee8138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:24.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.608+0000 7f2eee2ef700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ee8139170 0x7f2ee813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:24.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.608+0000 7f2eee2ef700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee809b120 con 0x7f2ee809d6f0 2023-10-03T10:17:24.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.608+0000 7f2eee2ef700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee806b460 con 0x7f2ee809e1d0 2023-10-03T10:17:24.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.608+0000 7f2eee2ef700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee809a7e0 con 0x7f2ee8139170 2023-10-03T10:17:24.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.608+0000 7f2eec88c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ee8139170 0x7f2ee813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:24.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.608+0000 7f2ee77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ee809e1d0 0x7f2ee8138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:24.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.608+0000 7f2eec88c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ee8139170 0x7f2ee813b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:39772/0 (socket says 172.21.15.3:39772) 2023-10-03T10:17:24.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.608+0000 7f2eec88c700 1 -- 172.21.15.3:0/2517403638 learned_addr learned my addr 172.21.15.3:0/2517403638 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:24.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.608+0000 7f2ee77fe700 1 -- 172.21.15.3:0/2517403638 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ee8139170 msgr2=0x7f2ee813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:24.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.608+0000 7f2ee77fe700 1 --2- 172.21.15.3:0/2517403638 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ee8139170 0x7f2ee813b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.608+0000 7f2ee6ffd700 1 -- 172.21.15.3:0/2517403638 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3553177982 0 0) 0x7f2ee809b120 con 0x7f2ee809d6f0 2023-10-03T10:17:24.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.608+0000 7f2ee77fe700 1 -- 172.21.15.3:0/2517403638 >> v1:172.21.15.3:6789/0 conn(0x7f2ee809d6f0 legacy=0x7f2ee809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:24.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.608+0000 7f2ee77fe700 1 -- 172.21.15.3:0/2517403638 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee809a500 con 0x7f2ee809e1d0 2023-10-03T10:17:24.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.608+0000 7f2eec88c700 1 --2- 172.21.15.3:0/2517403638 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ee8139170 0x7f2ee813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:24.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.609+0000 7f2ee77fe700 1 --2- 172.21.15.3:0/2517403638 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ee809e1d0 0x7f2ee8138c30 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009870 tx=0x7f2edc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b45814dbb42561c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:24.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.609+0000 7f2ee6ffd700 1 -- 172.21.15.3:0/2517403638 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc011830 con 0x7f2ee809e1d0 2023-10-03T10:17:24.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.609+0000 7f2ee6ffd700 1 -- 172.21.15.3:0/2517403638 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2edc011990 con 0x7f2ee809e1d0 2023-10-03T10:17:24.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.609+0000 7f2ee6ffd700 1 -- 172.21.15.3:0/2517403638 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc011d20 con 0x7f2ee809e1d0 2023-10-03T10:17:24.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.609+0000 7f2eee2ef700 1 -- 172.21.15.3:0/2517403638 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ee809e1d0 msgr2=0x7f2ee8138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:24.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.609+0000 7f2eee2ef700 1 --2- 172.21.15.3:0/2517403638 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ee809e1d0 0x7f2ee8138c30 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009870 tx=0x7f2edc009b80 comp rx=0 tx=0).stop 2023-10-03T10:17:24.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.610+0000 7f2eee2ef700 1 -- 172.21.15.3:0/2517403638 shutdown_connections 2023-10-03T10:17:24.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.610+0000 7f2eee2ef700 1 --2- 172.21.15.3:0/2517403638 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ee8139170 0x7f2ee813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.610+0000 7f2eee2ef700 1 --2- 172.21.15.3:0/2517403638 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ee809e1d0 0x7f2ee8138c30 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.610+0000 7f2eee2ef700 1 -- 172.21.15.3:0/2517403638 >> 172.21.15.3:0/2517403638 conn(0x7f2ee8066f50 msgr2=0x7f2ee809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:24.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.610+0000 7f2eee2ef700 1 -- 172.21.15.3:0/2517403638 shutdown_connections 2023-10-03T10:17:24.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.610+0000 7f2eee2ef700 1 -- 172.21.15.3:0/2517403638 wait complete. 2023-10-03T10:17:24.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.611+0000 7f2eee2ef700 1 Processor -- start 2023-10-03T10:17:24.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.611+0000 7f2eee2ef700 1 -- start start 2023-10-03T10:17:24.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.611+0000 7f2eee2ef700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ee809d6f0 0x7f2ee8120770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:24.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.611+0000 7f2eee2ef700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ee809e1d0 0x7f2ee8120cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:24.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.611+0000 7f2eee2ef700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ee8139170 0x7f2ee81211f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:24.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.611+0000 7f2eee2ef700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee8122ba0 con 0x7f2ee8139170 2023-10-03T10:17:24.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.611+0000 7f2eee2ef700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee8136610 con 0x7f2ee809e1d0 2023-10-03T10:17:24.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.611+0000 7f2eee2ef700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee809c710 con 0x7f2ee809d6f0 2023-10-03T10:17:24.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.612+0000 7f2ee77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ee809e1d0 0x7f2ee8120cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:24.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.612+0000 7f2ee77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ee809e1d0 0x7f2ee8120cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:35898/0 (socket says 172.21.15.3:35898) 2023-10-03T10:17:24.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.612+0000 7f2ee77fe700 1 -- 172.21.15.3:0/971321592 learned_addr learned my addr 172.21.15.3:0/971321592 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:24.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.612+0000 7f2ee7fff700 1 --2- 172.21.15.3:0/971321592 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ee809d6f0 0x7f2ee8120770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:24.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.612+0000 7f2eec88c700 1 --2- 172.21.15.3:0/971321592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ee8139170 0x7f2ee81211f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:24.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.612+0000 7f2ee77fe700 1 -- 172.21.15.3:0/971321592 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ee809d6f0 msgr2=0x7f2ee8120770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:24.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.612+0000 7f2ee77fe700 1 --2- 172.21.15.3:0/971321592 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ee809d6f0 0x7f2ee8120770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.612+0000 7f2ee77fe700 1 -- 172.21.15.3:0/971321592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ee8139170 msgr2=0x7f2ee81211f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:24.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.612+0000 7f2ee77fe700 1 --2- 172.21.15.3:0/971321592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ee8139170 0x7f2ee81211f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.612+0000 7f2ee77fe700 1 -- 172.21.15.3:0/971321592 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee0009b30 con 0x7f2ee809e1d0 2023-10-03T10:17:24.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.612+0000 7f2ee77fe700 1 --2- 172.21.15.3:0/971321592 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ee809e1d0 0x7f2ee8120cb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f2edc000c00 tx=0x7f2edc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:24.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.612+0000 7f2ee4ff9700 1 -- 172.21.15.3:0/971321592 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc024620 con 0x7f2ee809e1d0 2023-10-03T10:17:24.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.612+0000 7f2ee4ff9700 1 -- 172.21.15.3:0/971321592 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2edc024780 con 0x7f2ee809e1d0 2023-10-03T10:17:24.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.612+0000 7f2ee4ff9700 1 -- 172.21.15.3:0/971321592 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc024b30 con 0x7f2ee809e1d0 2023-10-03T10:17:24.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.612+0000 7f2eee2ef700 1 -- 172.21.15.3:0/971321592 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2edc0094a0 con 0x7f2ee809e1d0 2023-10-03T10:17:24.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.613+0000 7f2eee2ef700 1 -- 172.21.15.3:0/971321592 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee8072150 con 0x7f2ee809e1d0 2023-10-03T10:17:24.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.613+0000 7f2ee4ff9700 1 -- 172.21.15.3:0/971321592 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2edc022070 con 0x7f2ee809e1d0 2023-10-03T10:17:24.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.614+0000 7f2eee2ef700 1 -- 172.21.15.3:0/971321592 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee8123e80 con 0x7f2ee809e1d0 2023-10-03T10:17:24.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.614+0000 7f2ee4ff9700 1 -- 172.21.15.3:0/971321592 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2edc040c10 con 0x7f2ee809e1d0 2023-10-03T10:17:24.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.623+0000 7f2ee4ff9700 1 -- 172.21.15.3:0/971321592 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2edc022320 con 0x7f2ee809e1d0 2023-10-03T10:17:24.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.650+0000 7f8ab2880700 1 Processor -- start 2023-10-03T10:17:24.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.651+0000 7f8ab2880700 1 -- start start 2023-10-03T10:17:24.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.651+0000 7f8ab2880700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8aac06a770 0x7f8aac06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:24.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.651+0000 7f8ab2880700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8aac131d10 0x7f8aac139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:24.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.651+0000 7f8ab2880700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aac09ece0 con 0x7f8aac06b080 2023-10-03T10:17:24.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.651+0000 7f8ab2880700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aac09e760 con 0x7f8aac06a770 2023-10-03T10:17:24.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.651+0000 7f8ab2880700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac09e040 con 0x7f8aac131d10 2023-10-03T10:17:24.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.651+0000 7f8ab0e1d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8aac131d10 0x7f8aac139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:24.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.651+0000 7f8aabfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8aac06a770 0x7f8aac06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:24.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.651+0000 7f8aabfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8aac06a770 0x7f8aac06ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40236/0 (socket says 172.21.15.3:40236) 2023-10-03T10:17:24.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.651+0000 7f8aab7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8aac06b080 0x7f8aac131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45394/0 (socket says 172.21.15.3:45394) 2023-10-03T10:17:24.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.651+0000 7f8ab0e1d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8aac131d10 0x7f8aac139860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:39796/0 (socket says 172.21.15.3:39796) 2023-10-03T10:17:24.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.651+0000 7f8aabfff700 1 -- 172.21.15.3:0/3839592729 learned_addr learned my addr 172.21.15.3:0/3839592729 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:24.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.652+0000 7f8ab0e1d700 1 -- 172.21.15.3:0/3839592729 >> v2:172.21.15.3:3300/0 conn(0x7f8aac06a770 msgr2=0x7f8aac06ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:24.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.652+0000 7f8ab0e1d700 1 --2- 172.21.15.3:0/3839592729 >> v2:172.21.15.3:3300/0 conn(0x7f8aac06a770 0x7f8aac06ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.652+0000 7f8ab0e1d700 1 -- 172.21.15.3:0/3839592729 >> v1:172.21.15.3:6789/0 conn(0x7f8aac06b080 legacy=0x7f8aac131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:24.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.652+0000 7f8ab0e1d700 1 -- 172.21.15.3:0/3839592729 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aac09f630 con 0x7f8aac131d10 2023-10-03T10:17:24.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.652+0000 7f8aabfff700 1 --2- 172.21.15.3:0/3839592729 >> v2:172.21.15.3:3300/0 conn(0x7f8aac06a770 0x7f8aac06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:24.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.652+0000 7f8ab0e1d700 1 --2- 172.21.15.3:0/3839592729 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8aac131d10 0x7f8aac139860 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4009d90 tx=0x7f8aa400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6ff8e9e2bdd1e38 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:24.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.652+0000 7f8aaaffd700 1 -- 172.21.15.3:0/3839592729 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa400e070 con 0x7f8aac131d10 2023-10-03T10:17:24.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.652+0000 7f8aaaffd700 1 -- 172.21.15.3:0/3839592729 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa400bb20 con 0x7f8aac131d10 2023-10-03T10:17:24.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.652+0000 7f8aaaffd700 1 -- 172.21.15.3:0/3839592729 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4012430 con 0x7f8aac131d10 2023-10-03T10:17:24.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.653+0000 7f8ab2880700 1 -- 172.21.15.3:0/3839592729 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8aac131d10 msgr2=0x7f8aac139860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:24.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.653+0000 7f8ab2880700 1 --2- 172.21.15.3:0/3839592729 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8aac131d10 0x7f8aac139860 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4009d90 tx=0x7f8aa400b330 comp rx=0 tx=0).stop 2023-10-03T10:17:24.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.653+0000 7f8ab2880700 1 -- 172.21.15.3:0/3839592729 shutdown_connections 2023-10-03T10:17:24.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.653+0000 7f8ab2880700 1 --2- 172.21.15.3:0/3839592729 >> v2:172.21.15.3:3300/0 conn(0x7f8aac06a770 0x7f8aac06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.653+0000 7f8ab2880700 1 --2- 172.21.15.3:0/3839592729 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8aac131d10 0x7f8aac139860 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.653+0000 7f8ab2880700 1 -- 172.21.15.3:0/3839592729 >> 172.21.15.3:0/3839592729 conn(0x7f8aac066f50 msgr2=0x7f8aac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:24.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.653+0000 7f8ab2880700 1 -- 172.21.15.3:0/3839592729 shutdown_connections 2023-10-03T10:17:24.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.653+0000 7f8ab2880700 1 -- 172.21.15.3:0/3839592729 wait complete. 2023-10-03T10:17:24.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.654+0000 7f8ab2880700 1 Processor -- start 2023-10-03T10:17:24.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.654+0000 7f8ab2880700 1 -- start start 2023-10-03T10:17:24.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.655+0000 7f8ab2880700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8aac06a770 0x7f8aac1c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:24.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.655+0000 7f8ab2880700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8aac06b080 0x7f8aac1c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:24.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.655+0000 7f8ab2880700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8aac131d10 0x7f8aac1c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:24.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.655+0000 7f8ab2880700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac11e1c0 con 0x7f8aac06b080 2023-10-03T10:17:24.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.655+0000 7f8ab2880700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac11e7e0 con 0x7f8aac131d10 2023-10-03T10:17:24.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.655+0000 7f8ab2880700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac071e60 con 0x7f8aac06a770 2023-10-03T10:17:24.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.655+0000 7f8aabfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8aac06a770 0x7f8aac1c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:24.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.655+0000 7f8aab7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8aac06b080 0x7f8aac1c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:24.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.655+0000 7f8aabfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8aac06a770 0x7f8aac1c6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:39800/0 (socket says 172.21.15.3:39800) 2023-10-03T10:17:24.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.655+0000 7f8aabfff700 1 -- 172.21.15.3:0/1727682043 learned_addr learned my addr 172.21.15.3:0/1727682043 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:24.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.655+0000 7f8ab0e1d700 1 --2- 172.21.15.3:0/1727682043 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8aac131d10 0x7f8aac1c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:24.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.655+0000 7f8aab7fe700 1 -- 172.21.15.3:0/1727682043 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8aac06a770 msgr2=0x7f8aac1c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:24.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.655+0000 7f8aab7fe700 1 --2- 172.21.15.3:0/1727682043 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8aac06a770 0x7f8aac1c6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.655+0000 7f8aab7fe700 1 -- 172.21.15.3:0/1727682043 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8aac131d10 msgr2=0x7f8aac1c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:24.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.655+0000 7f8aab7fe700 1 --2- 172.21.15.3:0/1727682043 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8aac131d10 0x7f8aac1c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.655+0000 7f8aab7fe700 1 -- 172.21.15.3:0/1727682043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9c009550 con 0x7f8aac06b080 2023-10-03T10:17:24.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.655+0000 7f8aabfff700 1 --2- 172.21.15.3:0/1727682043 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8aac06a770 0x7f8aac1c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:24.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.655+0000 7f8aab7fe700 1 --2- 172.21.15.3:0/1727682043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8aac06b080 0x7f8aac1c66d0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000c450 tx=0x7f8aa000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:24.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.656+0000 7f8aa8ff9700 1 -- 172.21.15.3:0/1727682043 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0005cb0 con 0x7f8aac06b080 2023-10-03T10:17:24.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.656+0000 7f8aa8ff9700 1 -- 172.21.15.3:0/1727682043 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa0005e10 con 0x7f8aac06b080 2023-10-03T10:17:24.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.656+0000 7f8aa8ff9700 1 -- 172.21.15.3:0/1727682043 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa00072d0 con 0x7f8aac06b080 2023-10-03T10:17:24.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.656+0000 7f8ab2880700 1 -- 172.21.15.3:0/1727682043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa4009a70 con 0x7f8aac06b080 2023-10-03T10:17:24.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.656+0000 7f8ab2880700 1 -- 172.21.15.3:0/1727682043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aac11fdf0 con 0x7f8aac06b080 2023-10-03T10:17:24.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.657+0000 7f8aa8ff9700 1 -- 172.21.15.3:0/1727682043 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8aa0007430 con 0x7f8aac06b080 2023-10-03T10:17:24.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.657+0000 7f8aa8ff9700 1 -- 172.21.15.3:0/1727682043 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8aa003e9d0 con 0x7f8aac06b080 2023-10-03T10:17:24.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.657+0000 7f8ab2880700 1 -- 172.21.15.3:0/1727682043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aac11ffe0 con 0x7f8aac06b080 2023-10-03T10:17:24.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.662+0000 7f8aa8ff9700 1 -- 172.21.15.3:0/1727682043 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8aa0017080 con 0x7f8aac06b080 2023-10-03T10:17:24.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.758+0000 7fd55617b700 1 -- 172.21.15.3:0/871123939 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd55006b9e0 con 0x7fd5501338a0 2023-10-03T10:17:24.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.759+0000 7fd537fff700 1 -- 172.21.15.3:0/871123939 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd54801a070 con 0x7fd5501338a0 2023-10-03T10:17:24.759 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:24.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.762+0000 7fd55617b700 1 -- 172.21.15.3:0/871123939 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd5501338a0 msgr2=0x7fd55011f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:24.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.762+0000 7fd55617b700 1 --2- 172.21.15.3:0/871123939 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd5501338a0 0x7fd55011f400 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd54800cdf0 tx=0x7fd54800ce20 comp rx=0 tx=0).stop 2023-10-03T10:17:24.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.762+0000 7fd55617b700 1 -- 172.21.15.3:0/871123939 shutdown_connections 2023-10-03T10:17:24.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.762+0000 7fd55617b700 1 --2- 172.21.15.3:0/871123939 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd5501338a0 0x7fd55011f400 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.762+0000 7fd55617b700 1 --2- 172.21.15.3:0/871123939 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd55009de50 0x7fd55011e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.762+0000 7fd55617b700 1 --2- 172.21.15.3:0/871123939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd55009e760 0x7fd55011eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.762+0000 7fd55617b700 1 -- 172.21.15.3:0/871123939 >> 172.21.15.3:0/871123939 conn(0x7fd5500a2aa0 msgr2=0x7fd55009ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:24.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.762+0000 7fd55617b700 1 -- 172.21.15.3:0/871123939 shutdown_connections 2023-10-03T10:17:24.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.762+0000 7fd55617b700 1 -- 172.21.15.3:0/871123939 wait complete. 2023-10-03T10:17:24.781 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:24.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.831+0000 7f2eee2ef700 1 -- 172.21.15.3:0/971321592 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2ee8074ea0 con 0x7f2ee809e1d0 2023-10-03T10:17:24.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.831+0000 7f2ee4ff9700 1 -- 172.21.15.3:0/971321592 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2edc029070 con 0x7f2ee809e1d0 2023-10-03T10:17:24.831 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:24.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.834+0000 7f2eee2ef700 1 -- 172.21.15.3:0/971321592 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ee809e1d0 msgr2=0x7f2ee8120cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:24.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.834+0000 7f2eee2ef700 1 --2- 172.21.15.3:0/971321592 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ee809e1d0 0x7f2ee8120cb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f2edc000c00 tx=0x7f2edc004750 comp rx=0 tx=0).stop 2023-10-03T10:17:24.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.834+0000 7f2eee2ef700 1 -- 172.21.15.3:0/971321592 shutdown_connections 2023-10-03T10:17:24.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.834+0000 7f2eee2ef700 1 --2- 172.21.15.3:0/971321592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ee8139170 0x7f2ee81211f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.834+0000 7f2eee2ef700 1 --2- 172.21.15.3:0/971321592 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2ee809d6f0 0x7f2ee8120770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.834+0000 7f2eee2ef700 1 --2- 172.21.15.3:0/971321592 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2ee809e1d0 0x7f2ee8120cb0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.834+0000 7f2eee2ef700 1 -- 172.21.15.3:0/971321592 >> 172.21.15.3:0/971321592 conn(0x7f2ee8066f50 msgr2=0x7f2ee8136b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:24.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.834+0000 7f2eee2ef700 1 -- 172.21.15.3:0/971321592 shutdown_connections 2023-10-03T10:17:24.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.834+0000 7f2eee2ef700 1 -- 172.21.15.3:0/971321592 wait complete. 2023-10-03T10:17:24.848 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:24.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.879+0000 7f8ab2880700 1 -- 172.21.15.3:0/1727682043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8aac074ea0 con 0x7f8aac06b080 2023-10-03T10:17:24.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.879+0000 7f8aa8ff9700 1 -- 172.21.15.3:0/1727682043 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8aa00076e0 con 0x7f8aac06b080 2023-10-03T10:17:24.880 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:24.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.882+0000 7f8ab2880700 1 -- 172.21.15.3:0/1727682043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8aac06b080 msgr2=0x7f8aac1c66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:24.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.882+0000 7f8ab2880700 1 --2- 172.21.15.3:0/1727682043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8aac06b080 0x7f8aac1c66d0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000c450 tx=0x7f8aa000c760 comp rx=0 tx=0).stop 2023-10-03T10:17:24.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.882+0000 7f8ab2880700 1 -- 172.21.15.3:0/1727682043 shutdown_connections 2023-10-03T10:17:24.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.882+0000 7f8ab2880700 1 --2- 172.21.15.3:0/1727682043 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8aac131d10 0x7f8aac1c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.882+0000 7f8ab2880700 1 --2- 172.21.15.3:0/1727682043 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8aac06a770 0x7f8aac1c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.882+0000 7f8ab2880700 1 --2- 172.21.15.3:0/1727682043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8aac06b080 0x7f8aac1c66d0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:24.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.882+0000 7f8ab2880700 1 -- 172.21.15.3:0/1727682043 >> 172.21.15.3:0/1727682043 conn(0x7f8aac066f50 msgr2=0x7f8aac0700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:24.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.882+0000 7f8ab2880700 1 -- 172.21.15.3:0/1727682043 shutdown_connections 2023-10-03T10:17:24.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:24.882+0000 7f8ab2880700 1 -- 172.21.15.3:0/1727682043 wait complete. 2023-10-03T10:17:24.895 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:25.359 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:25.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.506+0000 7f430aed7700 1 Processor -- start 2023-10-03T10:17:25.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.506+0000 7f430aed7700 1 -- start start 2023-10-03T10:17:25.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.507+0000 7f430aed7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f43041316c0 0x7f4304131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:25.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.507+0000 7f430aed7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f430409d9f0 0x7f430406a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:25.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.507+0000 7f430aed7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f430406acb0 0x7f430406b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:25.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.507+0000 7f430aed7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f430409e130 con 0x7f43041316c0 2023-10-03T10:17:25.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.507+0000 7f430aed7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43041312b0 con 0x7f430406acb0 2023-10-03T10:17:25.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.507+0000 7f430aed7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f430409f6e0 con 0x7f430409d9f0 2023-10-03T10:17:25.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.507+0000 7f4308c73700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f43041316c0 0x7f4304131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:25.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.507+0000 7f4309474700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f430406acb0 0x7f430406b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:25.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.507+0000 7f4303fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f430409d9f0 0x7f430406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:25.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.507+0000 7f4308c73700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f43041316c0 0x7f4304131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40260/0 (socket says 172.21.15.3:40260) 2023-10-03T10:17:25.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.507+0000 7f4309474700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f430406acb0 0x7f430406b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:35924/0 (socket says 172.21.15.3:35924) 2023-10-03T10:17:25.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.507+0000 7f4303fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f430409d9f0 0x7f430406a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:39806/0 (socket says 172.21.15.3:39806) 2023-10-03T10:17:25.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.507+0000 7f4309474700 1 -- 172.21.15.3:0/650264664 learned_addr learned my addr 172.21.15.3:0/650264664 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:25.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.507+0000 7f4303fff700 1 -- 172.21.15.3:0/650264664 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f430406acb0 msgr2=0x7f430406b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:25.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.507+0000 7f4303fff700 1 --2- 172.21.15.3:0/650264664 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f430406acb0 0x7f430406b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:25.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.507+0000 7f4303fff700 1 -- 172.21.15.3:0/650264664 >> v2:172.21.15.3:3300/0 conn(0x7f43041316c0 msgr2=0x7f4304131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:25.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.507+0000 7f4303fff700 1 --2- 172.21.15.3:0/650264664 >> v2:172.21.15.3:3300/0 conn(0x7f43041316c0 0x7f4304131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:25.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.507+0000 7f4303fff700 1 -- 172.21.15.3:0/650264664 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430409e760 con 0x7f430409d9f0 2023-10-03T10:17:25.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.507+0000 7f4308c73700 1 --2- 172.21.15.3:0/650264664 >> v2:172.21.15.3:3300/0 conn(0x7f43041316c0 0x7f4304131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:25.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.507+0000 7f4309474700 1 --2- 172.21.15.3:0/650264664 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f430406acb0 0x7f430406b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:25.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.507+0000 7f4303fff700 1 --2- 172.21.15.3:0/650264664 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f430409d9f0 0x7f430406a770 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009840 tx=0x7f42ec009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8ccb1fbca4283a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:25.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.508+0000 7f43037fe700 1 -- 172.21.15.3:0/650264664 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec011830 con 0x7f430409d9f0 2023-10-03T10:17:25.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.508+0000 7f43037fe700 1 -- 172.21.15.3:0/650264664 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ec011990 con 0x7f430409d9f0 2023-10-03T10:17:25.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.508+0000 7f43037fe700 1 -- 172.21.15.3:0/650264664 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec011d20 con 0x7f430409d9f0 2023-10-03T10:17:25.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.508+0000 7f430aed7700 1 -- 172.21.15.3:0/650264664 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f430409d9f0 msgr2=0x7f430406a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:25.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.508+0000 7f430aed7700 1 --2- 172.21.15.3:0/650264664 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f430409d9f0 0x7f430406a770 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009840 tx=0x7f42ec009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:25.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.508+0000 7f430aed7700 1 -- 172.21.15.3:0/650264664 shutdown_connections 2023-10-03T10:17:25.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.508+0000 7f430aed7700 1 --2- 172.21.15.3:0/650264664 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f430406acb0 0x7f430406b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:25.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.508+0000 7f430aed7700 1 --2- 172.21.15.3:0/650264664 >> v2:172.21.15.3:3300/0 conn(0x7f43041316c0 0x7f4304131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:25.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.508+0000 7f430aed7700 1 --2- 172.21.15.3:0/650264664 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f430409d9f0 0x7f430406a770 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:25.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.508+0000 7f430aed7700 1 -- 172.21.15.3:0/650264664 >> 172.21.15.3:0/650264664 conn(0x7f4304066f50 msgr2=0x7f430409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:25.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.508+0000 7f430aed7700 1 -- 172.21.15.3:0/650264664 shutdown_connections 2023-10-03T10:17:25.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.508+0000 7f430aed7700 1 -- 172.21.15.3:0/650264664 wait complete. 2023-10-03T10:17:25.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.509+0000 7f430aed7700 1 Processor -- start 2023-10-03T10:17:25.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.509+0000 7f430aed7700 1 -- start start 2023-10-03T10:17:25.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.510+0000 7f430aed7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f430406acb0 0x7f4304070740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:25.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.510+0000 7f430aed7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f430409d9f0 0x7f4304070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:25.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.510+0000 7f430aed7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43041316c0 0x7f43040711c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:25.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.510+0000 7f430aed7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43041284e0 con 0x7f43041316c0 2023-10-03T10:17:25.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.510+0000 7f430aed7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43040a3760 con 0x7f430406acb0 2023-10-03T10:17:25.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.510+0000 7f430aed7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f430412b900 con 0x7f430409d9f0 2023-10-03T10:17:25.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.510+0000 7f4303fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f430409d9f0 0x7f4304070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:25.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.510+0000 7f4309474700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43041316c0 0x7f43040711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:25.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.510+0000 7f4308c73700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f430406acb0 0x7f4304070740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:25.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.510+0000 7f4303fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f430409d9f0 0x7f4304070c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:39822/0 (socket says 172.21.15.3:39822) 2023-10-03T10:17:25.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.510+0000 7f4309474700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43041316c0 0x7f43040711c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40276/0 (socket says 172.21.15.3:40276) 2023-10-03T10:17:25.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.510+0000 7f4303fff700 1 -- 172.21.15.3:0/854079593 learned_addr learned my addr 172.21.15.3:0/854079593 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:25.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.510+0000 7f4303fff700 1 -- 172.21.15.3:0/854079593 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f430406acb0 msgr2=0x7f4304070740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:25.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.510+0000 7f4303fff700 1 --2- 172.21.15.3:0/854079593 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f430406acb0 0x7f4304070740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:25.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.510+0000 7f4303fff700 1 -- 172.21.15.3:0/854079593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43041316c0 msgr2=0x7f43040711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:25.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.510+0000 7f4303fff700 1 --2- 172.21.15.3:0/854079593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43041316c0 0x7f43040711c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:25.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.510+0000 7f4303fff700 1 -- 172.21.15.3:0/854079593 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42fc009b30 con 0x7f430409d9f0 2023-10-03T10:17:25.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.510+0000 7f4308c73700 1 --2- 172.21.15.3:0/854079593 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f430406acb0 0x7f4304070740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:25.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.510+0000 7f4309474700 1 --2- 172.21.15.3:0/854079593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43041316c0 0x7f43040711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:25.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.511+0000 7f4303fff700 1 --2- 172.21.15.3:0/854079593 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f430409d9f0 0x7f4304070c80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f42ec000c00 tx=0x7f42ec004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:25.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.511+0000 7f43017fa700 1 -- 172.21.15.3:0/854079593 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec004450 con 0x7f430409d9f0 2023-10-03T10:17:25.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.511+0000 7f43017fa700 1 -- 172.21.15.3:0/854079593 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ec01f070 con 0x7f430409d9f0 2023-10-03T10:17:25.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.511+0000 7f430aed7700 1 -- 172.21.15.3:0/854079593 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f40094a0 con 0x7f430409d9f0 2023-10-03T10:17:25.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.511+0000 7f43017fa700 1 -- 172.21.15.3:0/854079593 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec0038e0 con 0x7f430409d9f0 2023-10-03T10:17:25.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.511+0000 7f430aed7700 1 -- 172.21.15.3:0/854079593 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42ec0094a0 con 0x7f430409d9f0 2023-10-03T10:17:25.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.512+0000 7f43017fa700 1 -- 172.21.15.3:0/854079593 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f42ec003a40 con 0x7f430409d9f0 2023-10-03T10:17:25.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.512+0000 7f43017fa700 1 -- 172.21.15.3:0/854079593 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f42ec021070 con 0x7f430409d9f0 2023-10-03T10:17:25.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.512+0000 7f430aed7700 1 -- 172.21.15.3:0/854079593 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4304074ea0 con 0x7f430409d9f0 2023-10-03T10:17:25.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.517+0000 7f43017fa700 1 -- 172.21.15.3:0/854079593 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f42ec015080 con 0x7f430409d9f0 2023-10-03T10:17:25.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.721+0000 7f430aed7700 1 -- 172.21.15.3:0/854079593 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f430409e980 con 0x7f430409d9f0 2023-10-03T10:17:25.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.721+0000 7f43017fa700 1 -- 172.21.15.3:0/854079593 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f42ec024380 con 0x7f430409d9f0 2023-10-03T10:17:25.722 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:25.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.724+0000 7f430aed7700 1 -- 172.21.15.3:0/854079593 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f430409d9f0 msgr2=0x7f4304070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:25.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.724+0000 7f430aed7700 1 --2- 172.21.15.3:0/854079593 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f430409d9f0 0x7f4304070c80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f42ec000c00 tx=0x7f42ec004750 comp rx=0 tx=0).stop 2023-10-03T10:17:25.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.724+0000 7f430aed7700 1 -- 172.21.15.3:0/854079593 shutdown_connections 2023-10-03T10:17:25.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.724+0000 7f430aed7700 1 --2- 172.21.15.3:0/854079593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43041316c0 0x7f43040711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:25.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.724+0000 7f430aed7700 1 --2- 172.21.15.3:0/854079593 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f430406acb0 0x7f4304070740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:25.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.724+0000 7f430aed7700 1 --2- 172.21.15.3:0/854079593 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f430409d9f0 0x7f4304070c80 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:25.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.724+0000 7f430aed7700 1 -- 172.21.15.3:0/854079593 >> 172.21.15.3:0/854079593 conn(0x7f4304066f50 msgr2=0x7f43040a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:25.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.724+0000 7f430aed7700 1 -- 172.21.15.3:0/854079593 shutdown_connections 2023-10-03T10:17:25.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.724+0000 7f430aed7700 1 -- 172.21.15.3:0/854079593 wait complete. 2023-10-03T10:17:25.738 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:25.782 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:25.849 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:25.896 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:25.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.925+0000 7f5c7d434700 1 Processor -- start 2023-10-03T10:17:25.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.925+0000 7f5c7d434700 1 -- start start 2023-10-03T10:17:25.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.926+0000 7f5c7d434700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c781338a0 0x7f5c78133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:25.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.926+0000 7f5c7d434700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c7809de50 0x7f5c7809e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:25.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.926+0000 7f5c7d434700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c7809e760 0x7f5c7813a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:25.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.926+0000 7f5c7d434700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c7806fef0 con 0x7f5c781338a0 2023-10-03T10:17:25.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.926+0000 7f5c7d434700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7806ef30 con 0x7f5c7809e760 2023-10-03T10:17:25.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.926+0000 7f5c7d434700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c781334b0 con 0x7f5c7809de50 2023-10-03T10:17:25.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.926+0000 7f5c76ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c781338a0 0x7f5c78133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:25.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.926+0000 7f5c777fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c7809e760 0x7f5c7813a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:25.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.926+0000 7f5c76ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c781338a0 0x7f5c78133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40292/0 (socket says 172.21.15.3:40292) 2023-10-03T10:17:25.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.926+0000 7f5c777fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c7809e760 0x7f5c7813a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:35942/0 (socket says 172.21.15.3:35942) 2023-10-03T10:17:25.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.926+0000 7f5c76ffd700 1 -- 172.21.15.3:0/1742489164 learned_addr learned my addr 172.21.15.3:0/1742489164 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:25.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.926+0000 7f5c767fc700 1 --2- 172.21.15.3:0/1742489164 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c7809de50 0x7f5c7809e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:25.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.927+0000 7f5c777fe700 1 -- 172.21.15.3:0/1742489164 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c7809de50 msgr2=0x7f5c7809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:25.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.927+0000 7f5c777fe700 1 --2- 172.21.15.3:0/1742489164 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c7809de50 0x7f5c7809e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:25.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.927+0000 7f5c777fe700 1 -- 172.21.15.3:0/1742489164 >> v2:172.21.15.3:3300/0 conn(0x7f5c781338a0 msgr2=0x7f5c78133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:25.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.927+0000 7f5c777fe700 1 --2- 172.21.15.3:0/1742489164 >> v2:172.21.15.3:3300/0 conn(0x7f5c781338a0 0x7f5c78133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:25.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.927+0000 7f5c777fe700 1 -- 172.21.15.3:0/1742489164 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7806fc30 con 0x7f5c7809e760 2023-10-03T10:17:25.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.927+0000 7f5c76ffd700 1 --2- 172.21.15.3:0/1742489164 >> v2:172.21.15.3:3300/0 conn(0x7f5c781338a0 0x7f5c78133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:25.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.927+0000 7f5c777fe700 1 --2- 172.21.15.3:0/1742489164 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c7809e760 0x7f5c7813a730 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5c70009d90 tx=0x7f5c7000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7db441a593ecdf server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:25.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.927+0000 7f5c75ffb700 1 -- 172.21.15.3:0/1742489164 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7000e070 con 0x7f5c7809e760 2023-10-03T10:17:25.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.927+0000 7f5c75ffb700 1 -- 172.21.15.3:0/1742489164 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7000bb20 con 0x7f5c7809e760 2023-10-03T10:17:25.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.927+0000 7f5c75ffb700 1 -- 172.21.15.3:0/1742489164 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70012430 con 0x7f5c7809e760 2023-10-03T10:17:25.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.928+0000 7f5c7d434700 1 -- 172.21.15.3:0/1742489164 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c7809e760 msgr2=0x7f5c7813a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:25.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.928+0000 7f5c7d434700 1 --2- 172.21.15.3:0/1742489164 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c7809e760 0x7f5c7813a730 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5c70009d90 tx=0x7f5c7000b330 comp rx=0 tx=0).stop 2023-10-03T10:17:25.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.928+0000 7f5c7d434700 1 -- 172.21.15.3:0/1742489164 shutdown_connections 2023-10-03T10:17:25.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.928+0000 7f5c7d434700 1 --2- 172.21.15.3:0/1742489164 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c7809e760 0x7f5c7813a730 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:25.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.928+0000 7f5c7d434700 1 --2- 172.21.15.3:0/1742489164 >> v2:172.21.15.3:3300/0 conn(0x7f5c781338a0 0x7f5c78133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:25.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.928+0000 7f5c7d434700 1 --2- 172.21.15.3:0/1742489164 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c7809de50 0x7f5c7809e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:25.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.928+0000 7f5c7d434700 1 -- 172.21.15.3:0/1742489164 >> 172.21.15.3:0/1742489164 conn(0x7f5c780a2aa0 msgr2=0x7f5c780a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:25.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.928+0000 7f5c7d434700 1 -- 172.21.15.3:0/1742489164 shutdown_connections 2023-10-03T10:17:25.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.928+0000 7f5c7d434700 1 -- 172.21.15.3:0/1742489164 wait complete. 2023-10-03T10:17:25.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.929+0000 7f5c7d434700 1 Processor -- start 2023-10-03T10:17:25.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.929+0000 7f5c7d434700 1 -- start start 2023-10-03T10:17:25.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.930+0000 7f5c7d434700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c7809de50 0x7f5c7811d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:25.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.930+0000 7f5c7d434700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c7809e760 0x7f5c7811dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:25.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.930+0000 7f5c7d434700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c781338a0 0x7f5c7811e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:25.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.930+0000 7f5c7d434700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c78130ba0 con 0x7f5c7809de50 2023-10-03T10:17:25.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.930+0000 7f5c7d434700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c78120610 con 0x7f5c7809e760 2023-10-03T10:17:25.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.930+0000 7f5c7d434700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7811f9a0 con 0x7f5c781338a0 2023-10-03T10:17:25.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.930+0000 7f5c76ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c7809de50 0x7f5c7811d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:25.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.930+0000 7f5c76ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c7809de50 0x7f5c7811d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40304/0 (socket says 172.21.15.3:40304) 2023-10-03T10:17:25.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.930+0000 7f5c76ffd700 1 -- 172.21.15.3:0/1448184493 learned_addr learned my addr 172.21.15.3:0/1448184493 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:25.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.930+0000 7f5c767fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c7809e760 0x7f5c7811dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:25.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.930+0000 7f5c777fe700 1 --2- 172.21.15.3:0/1448184493 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c781338a0 0x7f5c7811e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:25.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.930+0000 7f5c76ffd700 1 -- 172.21.15.3:0/1448184493 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c781338a0 msgr2=0x7f5c7811e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:25.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.930+0000 7f5c76ffd700 1 --2- 172.21.15.3:0/1448184493 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c781338a0 0x7f5c7811e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:25.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.930+0000 7f5c76ffd700 1 -- 172.21.15.3:0/1448184493 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c7809e760 msgr2=0x7f5c7811dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:25.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.930+0000 7f5c76ffd700 1 --2- 172.21.15.3:0/1448184493 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c7809e760 0x7f5c7811dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:25.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.930+0000 7f5c76ffd700 1 -- 172.21.15.3:0/1448184493 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c68009550 con 0x7f5c7809de50 2023-10-03T10:17:25.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.930+0000 7f5c777fe700 1 --2- 172.21.15.3:0/1448184493 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c781338a0 0x7f5c7811e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:17:25.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.930+0000 7f5c767fc700 1 --2- 172.21.15.3:0/1448184493 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c7809e760 0x7f5c7811dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:25.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.930+0000 7f5c76ffd700 1 --2- 172.21.15.3:0/1448184493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c7809de50 0x7f5c7811d840 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800ec10 tx=0x7f5c6800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:25.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.931+0000 7f5c5f7fe700 1 -- 172.21.15.3:0/1448184493 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6800c850 con 0x7f5c7809de50 2023-10-03T10:17:25.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.931+0000 7f5c5f7fe700 1 -- 172.21.15.3:0/1448184493 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6800c9b0 con 0x7f5c7809de50 2023-10-03T10:17:25.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.931+0000 7f5c7d434700 1 -- 172.21.15.3:0/1448184493 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c70009a70 con 0x7f5c7809de50 2023-10-03T10:17:25.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.931+0000 7f5c5f7fe700 1 -- 172.21.15.3:0/1448184493 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6800cd20 con 0x7f5c7809de50 2023-10-03T10:17:25.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.931+0000 7f5c7d434700 1 -- 172.21.15.3:0/1448184493 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c780a0d70 con 0x7f5c7809de50 2023-10-03T10:17:25.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.932+0000 7f5c7d434700 1 -- 172.21.15.3:0/1448184493 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c78074ea0 con 0x7f5c7809de50 2023-10-03T10:17:25.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.932+0000 7f5c5f7fe700 1 -- 172.21.15.3:0/1448184493 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5c68013470 con 0x7f5c7809de50 2023-10-03T10:17:25.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.932+0000 7f5c5f7fe700 1 -- 172.21.15.3:0/1448184493 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5c6803fba0 con 0x7f5c7809de50 2023-10-03T10:17:25.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.941+0000 7f5c5f7fe700 1 -- 172.21.15.3:0/1448184493 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5c68013720 con 0x7f5c7809de50 2023-10-03T10:17:25.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.995+0000 7f044de72700 1 Processor -- start 2023-10-03T10:17:25.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.995+0000 7f044de72700 1 -- start start 2023-10-03T10:17:25.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.996+0000 7f044de72700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f04480a0a30 0x7f04480a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:25.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.996+0000 7f044de72700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f044809d630 0x7f04480a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:25.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.996+0000 7f044de72700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0448133930 0x7f0448133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:25.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.996+0000 7f044de72700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04480a2a70 con 0x7f0448133930 2023-10-03T10:17:25.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.996+0000 7f044de72700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04480a01d0 con 0x7f044809d630 2023-10-03T10:17:25.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.996+0000 7f044de72700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f044809e2b0 con 0x7f04480a0a30 2023-10-03T10:17:25.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.996+0000 7f0446ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f044809d630 0x7f04480a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:25.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.996+0000 7f04477fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f04480a0a30 0x7f04480a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:25.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.996+0000 7f0446ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f044809d630 0x7f04480a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:35950/0 (socket says 172.21.15.3:35950) 2023-10-03T10:17:25.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.996+0000 7f0446ffd700 1 -- 172.21.15.3:0/3749395194 learned_addr learned my addr 172.21.15.3:0/3749395194 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:25.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.996+0000 7f0447fff700 1 --2- 172.21.15.3:0/3749395194 >> v2:172.21.15.3:3300/0 conn(0x7f0448133930 0x7f0448133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:25.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.997+0000 7f04477fe700 1 -- 172.21.15.3:0/3749395194 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f044809d630 msgr2=0x7f04480a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:25.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.997+0000 7f04477fe700 1 --2- 172.21.15.3:0/3749395194 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f044809d630 0x7f04480a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:25.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.997+0000 7f04477fe700 1 -- 172.21.15.3:0/3749395194 >> v2:172.21.15.3:3300/0 conn(0x7f0448133930 msgr2=0x7f0448133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:25.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.997+0000 7f04477fe700 1 --2- 172.21.15.3:0/3749395194 >> v2:172.21.15.3:3300/0 conn(0x7f0448133930 0x7f0448133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:25.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.997+0000 7f04477fe700 1 -- 172.21.15.3:0/3749395194 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f044809dc50 con 0x7f04480a0a30 2023-10-03T10:17:25.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.997+0000 7f0446ffd700 1 --2- 172.21.15.3:0/3749395194 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f044809d630 0x7f04480a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:25.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.997+0000 7f0447fff700 1 --2- 172.21.15.3:0/3749395194 >> v2:172.21.15.3:3300/0 conn(0x7f0448133930 0x7f0448133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:25.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.997+0000 7f04477fe700 1 --2- 172.21.15.3:0/3749395194 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f04480a0a30 0x7f04480a0e00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0438009840 tx=0x7f0438009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c209b09f0c39f74 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:26.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.997+0000 7f04467fc700 1 -- 172.21.15.3:0/3749395194 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0438011830 con 0x7f04480a0a30 2023-10-03T10:17:26.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.997+0000 7f04467fc700 1 -- 172.21.15.3:0/3749395194 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0438011990 con 0x7f04480a0a30 2023-10-03T10:17:26.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.997+0000 7f04467fc700 1 -- 172.21.15.3:0/3749395194 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0438011d20 con 0x7f04480a0a30 2023-10-03T10:17:26.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.998+0000 7f044de72700 1 -- 172.21.15.3:0/3749395194 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f04480a0a30 msgr2=0x7f04480a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:26.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.998+0000 7f044de72700 1 --2- 172.21.15.3:0/3749395194 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f04480a0a30 0x7f04480a0e00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0438009840 tx=0x7f0438009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:26.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.998+0000 7f044de72700 1 -- 172.21.15.3:0/3749395194 shutdown_connections 2023-10-03T10:17:26.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.998+0000 7f044de72700 1 --2- 172.21.15.3:0/3749395194 >> v2:172.21.15.3:3300/0 conn(0x7f0448133930 0x7f0448133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:26.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.998+0000 7f044de72700 1 --2- 172.21.15.3:0/3749395194 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f04480a0a30 0x7f04480a0e00 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:26.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.998+0000 7f044de72700 1 --2- 172.21.15.3:0/3749395194 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f044809d630 0x7f04480a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:26.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.998+0000 7f044de72700 1 -- 172.21.15.3:0/3749395194 >> 172.21.15.3:0/3749395194 conn(0x7f04480a3050 msgr2=0x7f04480a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:26.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.998+0000 7f044de72700 1 -- 172.21.15.3:0/3749395194 shutdown_connections 2023-10-03T10:17:26.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.998+0000 7f044de72700 1 -- 172.21.15.3:0/3749395194 wait complete. 2023-10-03T10:17:26.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:25.999+0000 7f044de72700 1 Processor -- start 2023-10-03T10:17:26.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.000+0000 7f044de72700 1 -- start start 2023-10-03T10:17:26.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.000+0000 7f044de72700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f044809d630 0x7f04481c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:26.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.000+0000 7f044de72700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04480a0a30 0x7f04481c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:26.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.000+0000 7f044de72700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0448133930 0x7f04481c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:26.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.000+0000 7f044de72700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0448122620 con 0x7f04480a0a30 2023-10-03T10:17:26.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.000+0000 7f044de72700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04481208b0 con 0x7f044809d630 2023-10-03T10:17:26.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.000+0000 7f044de72700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0448120680 con 0x7f0448133930 2023-10-03T10:17:26.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.000+0000 7f04477fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f044809d630 0x7f04481c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:26.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.000+0000 7f0446ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04480a0a30 0x7f04481c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:26.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.000+0000 7f04477fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f044809d630 0x7f04481c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:35954/0 (socket says 172.21.15.3:35954) 2023-10-03T10:17:26.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.000+0000 7f04477fe700 1 -- 172.21.15.3:0/2335750264 learned_addr learned my addr 172.21.15.3:0/2335750264 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:26.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.000+0000 7f0447fff700 1 --2- 172.21.15.3:0/2335750264 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0448133930 0x7f04481c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:26.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.001+0000 7f0446ffd700 1 -- 172.21.15.3:0/2335750264 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0448133930 msgr2=0x7f04481c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:26.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.001+0000 7f0446ffd700 1 --2- 172.21.15.3:0/2335750264 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0448133930 0x7f04481c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:26.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.001+0000 7f0446ffd700 1 -- 172.21.15.3:0/2335750264 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f044809d630 msgr2=0x7f04481c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:26.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.001+0000 7f0446ffd700 1 --2- 172.21.15.3:0/2335750264 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f044809d630 0x7f04481c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:26.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.001+0000 7f0446ffd700 1 -- 172.21.15.3:0/2335750264 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f043c009550 con 0x7f04480a0a30 2023-10-03T10:17:26.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.001+0000 7f0446ffd700 1 --2- 172.21.15.3:0/2335750264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04480a0a30 0x7f04481c85d0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f043c00e9a0 tx=0x7f043c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:26.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.001+0000 7f042ffff700 1 -- 172.21.15.3:0/2335750264 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043c004030 con 0x7f04480a0a30 2023-10-03T10:17:26.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.001+0000 7f042ffff700 1 -- 172.21.15.3:0/2335750264 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f043c004190 con 0x7f04480a0a30 2023-10-03T10:17:26.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.001+0000 7f042ffff700 1 -- 172.21.15.3:0/2335750264 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043c004520 con 0x7f04480a0a30 2023-10-03T10:17:26.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.001+0000 7f044de72700 1 -- 172.21.15.3:0/2335750264 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0440009a70 con 0x7f04480a0a30 2023-10-03T10:17:26.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.001+0000 7f044de72700 1 -- 172.21.15.3:0/2335750264 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04380094a0 con 0x7f04480a0a30 2023-10-03T10:17:26.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.002+0000 7f042ffff700 1 -- 172.21.15.3:0/2335750264 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f043c0036a0 con 0x7f04480a0a30 2023-10-03T10:17:26.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.002+0000 7f044de72700 1 -- 172.21.15.3:0/2335750264 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04480a0470 con 0x7f04480a0a30 2023-10-03T10:17:26.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.003+0000 7f042ffff700 1 -- 172.21.15.3:0/2335750264 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f043c03f830 con 0x7f04480a0a30 2023-10-03T10:17:26.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.012+0000 7f042ffff700 1 -- 172.21.15.3:0/2335750264 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f043c01bbe0 con 0x7f04480a0a30 2023-10-03T10:17:26.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.039+0000 7f70813df700 1 Processor -- start 2023-10-03T10:17:26.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.039+0000 7f70813df700 1 -- start start 2023-10-03T10:17:26.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.039+0000 7f70813df700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f707c06a770 0x7f707c06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:26.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.039+0000 7f70813df700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f707c131d10 0x7f707c139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:26.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.039+0000 7f70813df700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f707c09ece0 con 0x7f707c06b080 2023-10-03T10:17:26.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.039+0000 7f70813df700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f707c09e760 con 0x7f707c131d10 2023-10-03T10:17:26.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.039+0000 7f70813df700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f707c09e040 con 0x7f707c06a770 2023-10-03T10:17:26.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.039+0000 7f707affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f707c06a770 0x7f707c06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:26.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.040+0000 7f707b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f707c131d10 0x7f707c139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:26.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.040+0000 7f707affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f707c06a770 0x7f707c06ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:35970/0 (socket says 172.21.15.3:35970) 2023-10-03T10:17:26.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.040+0000 7f707affd700 1 -- 172.21.15.3:0/1526830809 learned_addr learned my addr 172.21.15.3:0/1526830809 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:26.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.040+0000 7f707b7fe700 1 -- 172.21.15.3:0/1526830809 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f707c06a770 msgr2=0x7f707c06ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:26.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.040+0000 7f707b7fe700 1 --2- 172.21.15.3:0/1526830809 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f707c06a770 0x7f707c06ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:26.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.040+0000 7f707b7fe700 1 -- 172.21.15.3:0/1526830809 >> v1:172.21.15.3:6789/0 conn(0x7f707c06b080 legacy=0x7f707c131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:26.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.040+0000 7f707b7fe700 1 -- 172.21.15.3:0/1526830809 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c09f630 con 0x7f707c131d10 2023-10-03T10:17:26.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.040+0000 7f7079ffb700 1 -- 172.21.15.3:0/1526830809 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2885954301 0 0) 0x7f707c09ece0 con 0x7f707c06b080 2023-10-03T10:17:26.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.040+0000 7f707affd700 1 --2- 172.21.15.3:0/1526830809 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f707c06a770 0x7f707c06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:26.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.040+0000 7f707b7fe700 1 --2- 172.21.15.3:0/1526830809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f707c131d10 0x7f707c139860 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f7074009d90 tx=0x7f707400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75834c83a81d294b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:26.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.040+0000 7f7079ffb700 1 -- 172.21.15.3:0/1526830809 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707400e070 con 0x7f707c131d10 2023-10-03T10:17:26.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.040+0000 7f7079ffb700 1 -- 172.21.15.3:0/1526830809 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707400bb20 con 0x7f707c131d10 2023-10-03T10:17:26.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.040+0000 7f7079ffb700 1 -- 172.21.15.3:0/1526830809 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7074012430 con 0x7f707c131d10 2023-10-03T10:17:26.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.041+0000 7f70813df700 1 -- 172.21.15.3:0/1526830809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f707c131d10 msgr2=0x7f707c139860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:26.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.041+0000 7f70813df700 1 --2- 172.21.15.3:0/1526830809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f707c131d10 0x7f707c139860 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f7074009d90 tx=0x7f707400b330 comp rx=0 tx=0).stop 2023-10-03T10:17:26.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.041+0000 7f70813df700 1 -- 172.21.15.3:0/1526830809 shutdown_connections 2023-10-03T10:17:26.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.041+0000 7f70813df700 1 --2- 172.21.15.3:0/1526830809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f707c131d10 0x7f707c139860 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:26.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.041+0000 7f70813df700 1 --2- 172.21.15.3:0/1526830809 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f707c06a770 0x7f707c06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:26.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.041+0000 7f70813df700 1 -- 172.21.15.3:0/1526830809 >> 172.21.15.3:0/1526830809 conn(0x7f707c066f50 msgr2=0x7f707c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:26.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.041+0000 7f70813df700 1 -- 172.21.15.3:0/1526830809 shutdown_connections 2023-10-03T10:17:26.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.041+0000 7f70813df700 1 -- 172.21.15.3:0/1526830809 wait complete. 2023-10-03T10:17:26.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.042+0000 7f70813df700 1 Processor -- start 2023-10-03T10:17:26.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.042+0000 7f70813df700 1 -- start start 2023-10-03T10:17:26.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.043+0000 7f70813df700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f707c06a770 0x7f707c1204f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:26.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.043+0000 7f70813df700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f707c06b080 0x7f707c120a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:26.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.043+0000 7f70813df700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f707c131d10 0x7f707c120f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:26.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.043+0000 7f70813df700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c123b00 con 0x7f707c131d10 2023-10-03T10:17:26.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.043+0000 7f70813df700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f707c124bf0 con 0x7f707c06a770 2023-10-03T10:17:26.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.043+0000 7f70813df700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f707c1226e0 con 0x7f707c06b080 2023-10-03T10:17:26.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.043+0000 7f707affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f707c06a770 0x7f707c1204f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:26.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.043+0000 7f707affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f707c06a770 0x7f707c1204f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:35974/0 (socket says 172.21.15.3:35974) 2023-10-03T10:17:26.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.043+0000 7f707affd700 1 -- 172.21.15.3:0/2276525297 learned_addr learned my addr 172.21.15.3:0/2276525297 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:26.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.043+0000 7f707b7fe700 1 --2- 172.21.15.3:0/2276525297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f707c131d10 0x7f707c120f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:26.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.043+0000 7f707affd700 1 -- 172.21.15.3:0/2276525297 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f707c06b080 msgr2=0x7f707c120a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:26.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.043+0000 7f707affd700 1 --2- 172.21.15.3:0/2276525297 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f707c06b080 0x7f707c120a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:26.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.043+0000 7f707affd700 1 -- 172.21.15.3:0/2276525297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f707c131d10 msgr2=0x7f707c120f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:26.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.043+0000 7f707affd700 1 --2- 172.21.15.3:0/2276525297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f707c131d10 0x7f707c120f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:26.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.043+0000 7f707affd700 1 -- 172.21.15.3:0/2276525297 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706c009550 con 0x7f707c06a770 2023-10-03T10:17:26.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.043+0000 7f707b7fe700 1 --2- 172.21.15.3:0/2276525297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f707c131d10 0x7f707c120f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:26.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.044+0000 7f707affd700 1 --2- 172.21.15.3:0/2276525297 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f707c06a770 0x7f707c1204f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f706c00ec10 tx=0x7f706c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:26.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.044+0000 7f70637fe700 1 -- 172.21.15.3:0/2276525297 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c00c850 con 0x7f707c06a770 2023-10-03T10:17:26.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.044+0000 7f70637fe700 1 -- 172.21.15.3:0/2276525297 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706c00c9b0 con 0x7f707c06a770 2023-10-03T10:17:26.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.044+0000 7f70813df700 1 -- 172.21.15.3:0/2276525297 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7074009a70 con 0x7f707c06a770 2023-10-03T10:17:26.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.044+0000 7f70637fe700 1 -- 172.21.15.3:0/2276525297 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c00cd20 con 0x7f707c06a770 2023-10-03T10:17:26.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.044+0000 7f70813df700 1 -- 172.21.15.3:0/2276525297 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f707c123d80 con 0x7f707c06a770 2023-10-03T10:17:26.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.045+0000 7f70637fe700 1 -- 172.21.15.3:0/2276525297 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f706c013470 con 0x7f707c06a770 2023-10-03T10:17:26.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.045+0000 7f70637fe700 1 -- 172.21.15.3:0/2276525297 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f706c03fa30 con 0x7f707c06a770 2023-10-03T10:17:26.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.045+0000 7f70813df700 1 -- 172.21.15.3:0/2276525297 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f707c123d80 con 0x7f707c06a770 2023-10-03T10:17:26.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.050+0000 7f70637fe700 1 -- 172.21.15.3:0/2276525297 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f707c123d80 con 0x7f707c06a770 2023-10-03T10:17:26.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.158+0000 7f5c7d434700 1 -- 172.21.15.3:0/1448184493 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5c78120c90 con 0x7f5c7809de50 2023-10-03T10:17:26.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.158+0000 7f5c5f7fe700 1 -- 172.21.15.3:0/1448184493 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5c68011b40 con 0x7f5c7809de50 2023-10-03T10:17:26.159 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:26.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.161+0000 7f5c7d434700 1 -- 172.21.15.3:0/1448184493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c7809de50 msgr2=0x7f5c7811d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:26.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.161+0000 7f5c7d434700 1 --2- 172.21.15.3:0/1448184493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c7809de50 0x7f5c7811d840 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800ec10 tx=0x7f5c6800ef20 comp rx=0 tx=0).stop 2023-10-03T10:17:26.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.161+0000 7f5c7d434700 1 -- 172.21.15.3:0/1448184493 shutdown_connections 2023-10-03T10:17:26.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.161+0000 7f5c7d434700 1 --2- 172.21.15.3:0/1448184493 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5c781338a0 0x7f5c7811e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:26.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.161+0000 7f5c7d434700 1 --2- 172.21.15.3:0/1448184493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c7809de50 0x7f5c7811d840 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:26.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.161+0000 7f5c7d434700 1 --2- 172.21.15.3:0/1448184493 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5c7809e760 0x7f5c7811dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:26.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.161+0000 7f5c7d434700 1 -- 172.21.15.3:0/1448184493 >> 172.21.15.3:0/1448184493 conn(0x7f5c780a2aa0 msgr2=0x7f5c781b9410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:26.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.161+0000 7f5c7d434700 1 -- 172.21.15.3:0/1448184493 shutdown_connections 2023-10-03T10:17:26.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.161+0000 7f5c7d434700 1 -- 172.21.15.3:0/1448184493 wait complete. 2023-10-03T10:17:26.175 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:26.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.221+0000 7f044de72700 1 -- 172.21.15.3:0/2335750264 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f04480747f0 con 0x7f04480a0a30 2023-10-03T10:17:26.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.221+0000 7f042ffff700 1 -- 172.21.15.3:0/2335750264 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f043c01b3b0 con 0x7f04480a0a30 2023-10-03T10:17:26.221 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:26.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.224+0000 7f044de72700 1 -- 172.21.15.3:0/2335750264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04480a0a30 msgr2=0x7f04481c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:26.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.224+0000 7f044de72700 1 --2- 172.21.15.3:0/2335750264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04480a0a30 0x7f04481c85d0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f043c00e9a0 tx=0x7f043c00ecb0 comp rx=0 tx=0).stop 2023-10-03T10:17:26.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.224+0000 7f044de72700 1 -- 172.21.15.3:0/2335750264 shutdown_connections 2023-10-03T10:17:26.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.224+0000 7f044de72700 1 --2- 172.21.15.3:0/2335750264 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0448133930 0x7f04481c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:26.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.224+0000 7f044de72700 1 --2- 172.21.15.3:0/2335750264 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f044809d630 0x7f04481c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:26.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.224+0000 7f044de72700 1 --2- 172.21.15.3:0/2335750264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04480a0a30 0x7f04481c85d0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:26.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.224+0000 7f044de72700 1 -- 172.21.15.3:0/2335750264 >> 172.21.15.3:0/2335750264 conn(0x7f04480a3050 msgr2=0x7f044809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:26.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.224+0000 7f044de72700 1 -- 172.21.15.3:0/2335750264 shutdown_connections 2023-10-03T10:17:26.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.224+0000 7f044de72700 1 -- 172.21.15.3:0/2335750264 wait complete. 2023-10-03T10:17:26.237 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:26.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.258+0000 7f70813df700 1 -- 172.21.15.3:0/2276525297 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f707c123d80 con 0x7f707c06a770 2023-10-03T10:17:26.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.258+0000 7f70637fe700 1 -- 172.21.15.3:0/2276525297 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f707c123d80 con 0x7f707c06a770 2023-10-03T10:17:26.258 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:26.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.260+0000 7f70813df700 1 -- 172.21.15.3:0/2276525297 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f707c06a770 msgr2=0x7f707c1204f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:26.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.260+0000 7f70813df700 1 --2- 172.21.15.3:0/2276525297 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f707c06a770 0x7f707c1204f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f706c00ec10 tx=0x7f706c00ef20 comp rx=0 tx=0).stop 2023-10-03T10:17:26.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.260+0000 7f70813df700 1 -- 172.21.15.3:0/2276525297 shutdown_connections 2023-10-03T10:17:26.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.260+0000 7f70813df700 1 --2- 172.21.15.3:0/2276525297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f707c131d10 0x7f707c120f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:26.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.260+0000 7f70813df700 1 --2- 172.21.15.3:0/2276525297 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f707c06a770 0x7f707c1204f0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:26.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.260+0000 7f70813df700 1 --2- 172.21.15.3:0/2276525297 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f707c06b080 0x7f707c120a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:26.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.260+0000 7f70813df700 1 -- 172.21.15.3:0/2276525297 >> 172.21.15.3:0/2276525297 conn(0x7f707c066f50 msgr2=0x7f707c0700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:26.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.261+0000 7f70813df700 1 -- 172.21.15.3:0/2276525297 shutdown_connections 2023-10-03T10:17:26.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.261+0000 7f70813df700 1 -- 172.21.15.3:0/2276525297 wait complete. 2023-10-03T10:17:26.274 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:26.739 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:26.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.886+0000 7f31caafc700 1 Processor -- start 2023-10-03T10:17:26.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.886+0000 7f31caafc700 1 -- start start 2023-10-03T10:17:26.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.886+0000 7f31caafc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f31c409d6f0 0x7f31c409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:26.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.886+0000 7f31caafc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31c409e000 0x7f31c4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:26.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.886+0000 7f31caafc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31c409e410 0x7f31c4130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:26.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.886+0000 7f31caafc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31c409f520 con 0x7f31c409d6f0 2023-10-03T10:17:26.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.886+0000 7f31caafc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31c409f130 con 0x7f31c409e410 2023-10-03T10:17:26.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.886+0000 7f31caafc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31c409e930 con 0x7f31c409e000 2023-10-03T10:17:26.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.887+0000 7f31c9099700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31c409e410 0x7f31c4130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:26.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.887+0000 7f31c3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31c409e000 0x7f31c4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:26.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.887+0000 7f31c3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31c409e000 0x7f31c4130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:39884/0 (socket says 172.21.15.3:39884) 2023-10-03T10:17:26.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.887+0000 7f31c9099700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31c409e410 0x7f31c4130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:35984/0 (socket says 172.21.15.3:35984) 2023-10-03T10:17:26.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.887+0000 7f31c3fff700 1 -- 172.21.15.3:0/2212670870 learned_addr learned my addr 172.21.15.3:0/2212670870 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:26.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.887+0000 7f31c8898700 1 --2- 172.21.15.3:0/2212670870 >> v2:172.21.15.3:3300/0 conn(0x7f31c409d6f0 0x7f31c409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:26.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.887+0000 7f31c3fff700 1 -- 172.21.15.3:0/2212670870 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31c409e410 msgr2=0x7f31c4130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:26.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.887+0000 7f31c3fff700 1 --2- 172.21.15.3:0/2212670870 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31c409e410 0x7f31c4130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:26.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.887+0000 7f31c3fff700 1 -- 172.21.15.3:0/2212670870 >> v2:172.21.15.3:3300/0 conn(0x7f31c409d6f0 msgr2=0x7f31c409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:26.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.887+0000 7f31c3fff700 1 --2- 172.21.15.3:0/2212670870 >> v2:172.21.15.3:3300/0 conn(0x7f31c409d6f0 0x7f31c409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:26.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.887+0000 7f31c3fff700 1 -- 172.21.15.3:0/2212670870 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c409b2d0 con 0x7f31c409e000 2023-10-03T10:17:26.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.887+0000 7f31c8898700 1 --2- 172.21.15.3:0/2212670870 >> v2:172.21.15.3:3300/0 conn(0x7f31c409d6f0 0x7f31c409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:26.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.887+0000 7f31c9099700 1 --2- 172.21.15.3:0/2212670870 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31c409e410 0x7f31c4130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:26.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.887+0000 7f31c3fff700 1 --2- 172.21.15.3:0/2212670870 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31c409e000 0x7f31c4130660 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f31b8009920 tx=0x7f31b8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2deea1ba63bde1d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:26.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.887+0000 7f31c37fe700 1 -- 172.21.15.3:0/2212670870 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b8011830 con 0x7f31c409e000 2023-10-03T10:17:26.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.887+0000 7f31c37fe700 1 -- 172.21.15.3:0/2212670870 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b8011990 con 0x7f31c409e000 2023-10-03T10:17:26.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.887+0000 7f31c37fe700 1 -- 172.21.15.3:0/2212670870 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b8011d20 con 0x7f31c409e000 2023-10-03T10:17:26.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.888+0000 7f31caafc700 1 -- 172.21.15.3:0/2212670870 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31c409e000 msgr2=0x7f31c4130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:26.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.888+0000 7f31caafc700 1 --2- 172.21.15.3:0/2212670870 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31c409e000 0x7f31c4130660 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f31b8009920 tx=0x7f31b8009c30 comp rx=0 tx=0).stop 2023-10-03T10:17:26.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.888+0000 7f31caafc700 1 -- 172.21.15.3:0/2212670870 shutdown_connections 2023-10-03T10:17:26.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.888+0000 7f31caafc700 1 --2- 172.21.15.3:0/2212670870 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31c409e410 0x7f31c4130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:26.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.888+0000 7f31caafc700 1 --2- 172.21.15.3:0/2212670870 >> v2:172.21.15.3:3300/0 conn(0x7f31c409d6f0 0x7f31c409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:26.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.888+0000 7f31caafc700 1 --2- 172.21.15.3:0/2212670870 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31c409e000 0x7f31c4130660 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:26.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.888+0000 7f31caafc700 1 -- 172.21.15.3:0/2212670870 >> 172.21.15.3:0/2212670870 conn(0x7f31c4066f50 msgr2=0x7f31c409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:26.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.888+0000 7f31caafc700 1 -- 172.21.15.3:0/2212670870 shutdown_connections 2023-10-03T10:17:26.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.888+0000 7f31caafc700 1 -- 172.21.15.3:0/2212670870 wait complete. 2023-10-03T10:17:26.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.889+0000 7f31caafc700 1 Processor -- start 2023-10-03T10:17:26.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.889+0000 7f31caafc700 1 -- start start 2023-10-03T10:17:26.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.890+0000 7f31caafc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31c409d6f0 0x7f31c4120240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:26.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.890+0000 7f31caafc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31c409e000 0x7f31c4120780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:26.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.890+0000 7f31caafc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31c409e410 0x7f31c4120cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:26.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.890+0000 7f31caafc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c409c700 con 0x7f31c409d6f0 2023-10-03T10:17:26.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.890+0000 7f31caafc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31c4130780 con 0x7f31c409e000 2023-10-03T10:17:26.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.890+0000 7f31caafc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31c41234f0 con 0x7f31c409e410 2023-10-03T10:17:26.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.890+0000 7f31c8898700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31c409d6f0 0x7f31c4120240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:26.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.890+0000 7f31c9099700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31c409e410 0x7f31c4120cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:26.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.890+0000 7f31c9099700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31c409e410 0x7f31c4120cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:39892/0 (socket says 172.21.15.3:39892) 2023-10-03T10:17:26.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.890+0000 7f31c8898700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31c409d6f0 0x7f31c4120240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40344/0 (socket says 172.21.15.3:40344) 2023-10-03T10:17:26.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.890+0000 7f31c9099700 1 -- 172.21.15.3:0/1388877635 learned_addr learned my addr 172.21.15.3:0/1388877635 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:26.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.890+0000 7f31c3fff700 1 --2- 172.21.15.3:0/1388877635 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31c409e000 0x7f31c4120780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:26.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.890+0000 7f31c9099700 1 -- 172.21.15.3:0/1388877635 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31c409e000 msgr2=0x7f31c4120780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:26.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.890+0000 7f31c9099700 1 --2- 172.21.15.3:0/1388877635 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31c409e000 0x7f31c4120780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:26.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.890+0000 7f31c9099700 1 -- 172.21.15.3:0/1388877635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31c409d6f0 msgr2=0x7f31c4120240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:26.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.890+0000 7f31c9099700 1 --2- 172.21.15.3:0/1388877635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31c409d6f0 0x7f31c4120240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:26.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.890+0000 7f31c9099700 1 -- 172.21.15.3:0/1388877635 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b40094a0 con 0x7f31c409e410 2023-10-03T10:17:26.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.890+0000 7f31c9099700 1 --2- 172.21.15.3:0/1388877635 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31c409e410 0x7f31c4120cc0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00efc0 tx=0x7f31bc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:26.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.890+0000 7f31c17fa700 1 -- 172.21.15.3:0/1388877635 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31bc00ca90 con 0x7f31c409e410 2023-10-03T10:17:26.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.890+0000 7f31c17fa700 1 -- 172.21.15.3:0/1388877635 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31bc00f040 con 0x7f31c409e410 2023-10-03T10:17:26.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.891+0000 7f31caafc700 1 -- 172.21.15.3:0/1388877635 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31bc009a70 con 0x7f31c409e410 2023-10-03T10:17:26.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.891+0000 7f31c17fa700 1 -- 172.21.15.3:0/1388877635 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31bc00ce10 con 0x7f31c409e410 2023-10-03T10:17:26.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.891+0000 7f31caafc700 1 -- 172.21.15.3:0/1388877635 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b8009550 con 0x7f31c409e410 2023-10-03T10:17:26.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.891+0000 7f31c17fa700 1 -- 172.21.15.3:0/1388877635 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f31bc014480 con 0x7f31c409e410 2023-10-03T10:17:26.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.892+0000 7f31c17fa700 1 -- 172.21.15.3:0/1388877635 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f31bc03fe10 con 0x7f31c409e410 2023-10-03T10:17:26.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.892+0000 7f31caafc700 1 -- 172.21.15.3:0/1388877635 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31c4074ea0 con 0x7f31c409e410 2023-10-03T10:17:26.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:26.897+0000 7f31c17fa700 1 -- 172.21.15.3:0/1388877635 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f31bc012930 con 0x7f31c409e410 2023-10-03T10:17:27.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.099+0000 7f31caafc700 1 -- 172.21.15.3:0/1388877635 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f31c4122640 con 0x7f31c409e410 2023-10-03T10:17:27.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.100+0000 7f31c17fa700 1 -- 172.21.15.3:0/1388877635 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f31bc02a090 con 0x7f31c409e410 2023-10-03T10:17:27.100 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:27.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.102+0000 7f31caafc700 1 -- 172.21.15.3:0/1388877635 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31c409e410 msgr2=0x7f31c4120cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:27.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.102+0000 7f31caafc700 1 --2- 172.21.15.3:0/1388877635 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31c409e410 0x7f31c4120cc0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00efc0 tx=0x7f31bc00c3a0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.102+0000 7f31caafc700 1 -- 172.21.15.3:0/1388877635 shutdown_connections 2023-10-03T10:17:27.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.102+0000 7f31caafc700 1 --2- 172.21.15.3:0/1388877635 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31c409e410 0x7f31c4120cc0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.102+0000 7f31caafc700 1 --2- 172.21.15.3:0/1388877635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31c409d6f0 0x7f31c4120240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.102+0000 7f31caafc700 1 --2- 172.21.15.3:0/1388877635 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31c409e000 0x7f31c4120780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.102+0000 7f31caafc700 1 -- 172.21.15.3:0/1388877635 >> 172.21.15.3:0/1388877635 conn(0x7f31c4066f50 msgr2=0x7f31c40713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.102+0000 7f31caafc700 1 -- 172.21.15.3:0/1388877635 shutdown_connections 2023-10-03T10:17:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.102+0000 7f31caafc700 1 -- 172.21.15.3:0/1388877635 wait complete. 2023-10-03T10:17:27.116 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:27.175 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:27.238 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:27.275 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:27.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.323+0000 7f1e5d1a2700 1 Processor -- start 2023-10-03T10:17:27.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.323+0000 7f1e5d1a2700 1 -- start start 2023-10-03T10:17:27.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.324+0000 7f1e5d1a2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e5809dba0 0x7f1e5809df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:27.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.324+0000 7f1e5d1a2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e5812f8c0 0x7f1e5812fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:27.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.324+0000 7f1e5d1a2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1e58130210 0x7f1e58136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:27.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.324+0000 7f1e5d1a2700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e5809d670 con 0x7f1e58130210 2023-10-03T10:17:27.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.324+0000 7f1e5d1a2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e58099d80 con 0x7f1e5812f8c0 2023-10-03T10:17:27.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.324+0000 7f1e5d1a2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5809b020 con 0x7f1e5809dba0 2023-10-03T10:17:27.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.324+0000 7f1e577fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e5812f8c0 0x7f1e5812fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:27.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.324+0000 7f1e57fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e5809dba0 0x7f1e5809df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:27.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.324+0000 7f1e577fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e5812f8c0 0x7f1e5812fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36000/0 (socket says 172.21.15.3:36000) 2023-10-03T10:17:27.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.324+0000 7f1e57fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e5809dba0 0x7f1e5809df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:39904/0 (socket says 172.21.15.3:39904) 2023-10-03T10:17:27.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.324+0000 7f1e577fe700 1 -- 172.21.15.3:0/234793634 learned_addr learned my addr 172.21.15.3:0/234793634 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:27.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.325+0000 7f1e5c9a1700 1 --2- 172.21.15.3:0/234793634 >> v2:172.21.15.3:3300/0 conn(0x7f1e58130210 0x7f1e58136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:27.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.325+0000 7f1e577fe700 1 -- 172.21.15.3:0/234793634 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e5809dba0 msgr2=0x7f1e5809df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:27.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.325+0000 7f1e577fe700 1 --2- 172.21.15.3:0/234793634 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e5809dba0 0x7f1e5809df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.325+0000 7f1e577fe700 1 -- 172.21.15.3:0/234793634 >> v2:172.21.15.3:3300/0 conn(0x7f1e58130210 msgr2=0x7f1e58136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:27.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.325+0000 7f1e577fe700 1 --2- 172.21.15.3:0/234793634 >> v2:172.21.15.3:3300/0 conn(0x7f1e58130210 0x7f1e58136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.325+0000 7f1e577fe700 1 -- 172.21.15.3:0/234793634 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5809a440 con 0x7f1e5812f8c0 2023-10-03T10:17:27.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.325+0000 7f1e57fff700 1 --2- 172.21.15.3:0/234793634 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e5809dba0 0x7f1e5809df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:27.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.325+0000 7f1e577fe700 1 --2- 172.21.15.3:0/234793634 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e5812f8c0 0x7f1e5812fcd0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c0098f0 tx=0x7f1e4c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53de95c9a4fec55f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:27.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.325+0000 7f1e56ffd700 1 -- 172.21.15.3:0/234793634 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c011830 con 0x7f1e5812f8c0 2023-10-03T10:17:27.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.325+0000 7f1e56ffd700 1 -- 172.21.15.3:0/234793634 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4c011990 con 0x7f1e5812f8c0 2023-10-03T10:17:27.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.325+0000 7f1e56ffd700 1 -- 172.21.15.3:0/234793634 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c011d20 con 0x7f1e5812f8c0 2023-10-03T10:17:27.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.326+0000 7f1e5d1a2700 1 -- 172.21.15.3:0/234793634 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e5812f8c0 msgr2=0x7f1e5812fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:27.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.326+0000 7f1e5d1a2700 1 --2- 172.21.15.3:0/234793634 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e5812f8c0 0x7f1e5812fcd0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c0098f0 tx=0x7f1e4c009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:27.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.326+0000 7f1e5d1a2700 1 -- 172.21.15.3:0/234793634 shutdown_connections 2023-10-03T10:17:27.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.326+0000 7f1e5d1a2700 1 --2- 172.21.15.3:0/234793634 >> v2:172.21.15.3:3300/0 conn(0x7f1e58130210 0x7f1e58136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.326+0000 7f1e5d1a2700 1 --2- 172.21.15.3:0/234793634 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e5809dba0 0x7f1e5809df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.326+0000 7f1e5d1a2700 1 --2- 172.21.15.3:0/234793634 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e5812f8c0 0x7f1e5812fcd0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.326+0000 7f1e5d1a2700 1 -- 172.21.15.3:0/234793634 >> 172.21.15.3:0/234793634 conn(0x7f1e580a2540 msgr2=0x7f1e580a2940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:27.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.326+0000 7f1e5d1a2700 1 -- 172.21.15.3:0/234793634 shutdown_connections 2023-10-03T10:17:27.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.327+0000 7f1e5d1a2700 1 -- 172.21.15.3:0/234793634 wait complete. 2023-10-03T10:17:27.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.328+0000 7f1e5d1a2700 1 Processor -- start 2023-10-03T10:17:27.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.328+0000 7f1e5d1a2700 1 -- start start 2023-10-03T10:17:27.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.329+0000 7f1e5d1a2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e5809dba0 0x7f1e5812c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:27.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.329+0000 7f1e5d1a2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e5812f8c0 0x7f1e5812c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:27.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.329+0000 7f1e5d1a2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e58130210 0x7f1e581c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:27.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.329+0000 7f1e5d1a2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5808c1b0 con 0x7f1e58130210 2023-10-03T10:17:27.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.329+0000 7f1e5d1a2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5809a8f0 con 0x7f1e5812f8c0 2023-10-03T10:17:27.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.329+0000 7f1e5d1a2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5812bdd0 con 0x7f1e5809dba0 2023-10-03T10:17:27.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.329+0000 7f1e577fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e5812f8c0 0x7f1e5812c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:27.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.329+0000 7f1e577fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e5812f8c0 0x7f1e5812c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36004/0 (socket says 172.21.15.3:36004) 2023-10-03T10:17:27.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.329+0000 7f1e577fe700 1 -- 172.21.15.3:0/2610001663 learned_addr learned my addr 172.21.15.3:0/2610001663 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:27.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.329+0000 7f1e5c9a1700 1 --2- 172.21.15.3:0/2610001663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e58130210 0x7f1e581c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:27.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.329+0000 7f1e57fff700 1 --2- 172.21.15.3:0/2610001663 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e5809dba0 0x7f1e5812c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:27.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.329+0000 7f1e577fe700 1 -- 172.21.15.3:0/2610001663 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e5809dba0 msgr2=0x7f1e5812c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:27.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.329+0000 7f1e577fe700 1 --2- 172.21.15.3:0/2610001663 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e5809dba0 0x7f1e5812c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.329+0000 7f1e577fe700 1 -- 172.21.15.3:0/2610001663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e58130210 msgr2=0x7f1e581c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:27.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.329+0000 7f1e577fe700 1 --2- 172.21.15.3:0/2610001663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e58130210 0x7f1e581c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.329+0000 7f1e577fe700 1 -- 172.21.15.3:0/2610001663 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4800b6d0 con 0x7f1e5812f8c0 2023-10-03T10:17:27.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.329+0000 7f1e5c9a1700 1 --2- 172.21.15.3:0/2610001663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e58130210 0x7f1e581c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:27.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.330+0000 7f1e577fe700 1 --2- 172.21.15.3:0/2610001663 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e5812f8c0 0x7f1e5812c760 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c000c00 tx=0x7f1e4c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:27.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.330+0000 7f1e54ff9700 1 -- 172.21.15.3:0/2610001663 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c004450 con 0x7f1e5812f8c0 2023-10-03T10:17:27.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.330+0000 7f1e54ff9700 1 -- 172.21.15.3:0/2610001663 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4c0045b0 con 0x7f1e5812f8c0 2023-10-03T10:17:27.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.330+0000 7f1e54ff9700 1 -- 172.21.15.3:0/2610001663 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c0038e0 con 0x7f1e5812f8c0 2023-10-03T10:17:27.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.330+0000 7f1e5d1a2700 1 -- 172.21.15.3:0/2610001663 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e4c009550 con 0x7f1e5812f8c0 2023-10-03T10:17:27.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.330+0000 7f1e5d1a2700 1 -- 172.21.15.3:0/2610001663 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e5808bb10 con 0x7f1e5812f8c0 2023-10-03T10:17:27.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.331+0000 7f1e5d1a2700 1 -- 172.21.15.3:0/2610001663 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e5804f2a0 con 0x7f1e5812f8c0 2023-10-03T10:17:27.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.332+0000 7f1e54ff9700 1 -- 172.21.15.3:0/2610001663 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1e4c021070 con 0x7f1e5812f8c0 2023-10-03T10:17:27.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.332+0000 7f1e54ff9700 1 -- 172.21.15.3:0/2610001663 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1e4c040c20 con 0x7f1e5812f8c0 2023-10-03T10:17:27.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.341+0000 7f1e54ff9700 1 -- 172.21.15.3:0/2610001663 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1e4c003a40 con 0x7f1e5812f8c0 2023-10-03T10:17:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.387+0000 7f7ea77b6700 1 Processor -- start 2023-10-03T10:17:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.388+0000 7f7ea77b6700 1 -- start start 2023-10-03T10:17:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.388+0000 7f7ea77b6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ea00a0a30 0x7f7ea00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:27.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.388+0000 7f7ea77b6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ea0134170 0x7f7ea013bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:27.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.388+0000 7f7ea77b6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ea009edd0 con 0x7f7ea00a1340 2023-10-03T10:17:27.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.388+0000 7f7ea77b6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea009e1f0 con 0x7f7ea0134170 2023-10-03T10:17:27.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.388+0000 7f7ea77b6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea009dc50 con 0x7f7ea00a0a30 2023-10-03T10:17:27.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.389+0000 7f7ea4d51700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7ea00a1340 0x7f7ea0133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45408/0 (socket says 172.21.15.3:45408) 2023-10-03T10:17:27.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.389+0000 7f7ea4d51700 1 -- 172.21.15.3:0/1378774869 learned_addr learned my addr 172.21.15.3:0/1378774869 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:27.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.389+0000 7f7ea5552700 1 --2- 172.21.15.3:0/1378774869 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ea00a0a30 0x7f7ea00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:27.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.389+0000 7f7ea5d53700 1 --2- 172.21.15.3:0/1378774869 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ea0134170 0x7f7ea013bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:27.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.389+0000 7f7ea5552700 1 -- 172.21.15.3:0/1378774869 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ea0134170 msgr2=0x7f7ea013bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:27.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.389+0000 7f7ea5552700 1 --2- 172.21.15.3:0/1378774869 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ea0134170 0x7f7ea013bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.389+0000 7f7ea5552700 1 -- 172.21.15.3:0/1378774869 >> v1:172.21.15.3:6789/0 conn(0x7f7ea00a1340 legacy=0x7f7ea0133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:27.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.389+0000 7f7ea5552700 1 -- 172.21.15.3:0/1378774869 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea00a1fe0 con 0x7f7ea00a0a30 2023-10-03T10:17:27.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.389+0000 7f7ea5d53700 1 --2- 172.21.15.3:0/1378774869 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ea0134170 0x7f7ea013bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:27.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.389+0000 7f7ea5552700 1 --2- 172.21.15.3:0/1378774869 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ea00a0a30 0x7f7ea00a0e00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c009870 tx=0x7f7e9c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e9b9567366de9f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:27.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.389+0000 7f7e97fff700 1 -- 172.21.15.3:0/1378774869 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c011830 con 0x7f7ea00a0a30 2023-10-03T10:17:27.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.389+0000 7f7e97fff700 1 -- 172.21.15.3:0/1378774869 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9c011990 con 0x7f7ea00a0a30 2023-10-03T10:17:27.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.389+0000 7f7e97fff700 1 -- 172.21.15.3:0/1378774869 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c011d20 con 0x7f7ea00a0a30 2023-10-03T10:17:27.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.390+0000 7f7ea77b6700 1 -- 172.21.15.3:0/1378774869 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ea00a0a30 msgr2=0x7f7ea00a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:27.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.390+0000 7f7ea77b6700 1 --2- 172.21.15.3:0/1378774869 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ea00a0a30 0x7f7ea00a0e00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c009870 tx=0x7f7e9c009b80 comp rx=0 tx=0).stop 2023-10-03T10:17:27.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.390+0000 7f7ea77b6700 1 -- 172.21.15.3:0/1378774869 shutdown_connections 2023-10-03T10:17:27.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.390+0000 7f7ea77b6700 1 --2- 172.21.15.3:0/1378774869 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ea0134170 0x7f7ea013bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.390+0000 7f7ea77b6700 1 --2- 172.21.15.3:0/1378774869 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ea00a0a30 0x7f7ea00a0e00 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.390+0000 7f7ea77b6700 1 -- 172.21.15.3:0/1378774869 >> 172.21.15.3:0/1378774869 conn(0x7f7ea00a6550 msgr2=0x7f7ea00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:27.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.390+0000 7f7ea77b6700 1 -- 172.21.15.3:0/1378774869 shutdown_connections 2023-10-03T10:17:27.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.390+0000 7f7ea77b6700 1 -- 172.21.15.3:0/1378774869 wait complete. 2023-10-03T10:17:27.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.392+0000 7f7ea77b6700 1 Processor -- start 2023-10-03T10:17:27.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.392+0000 7f7ea77b6700 1 -- start start 2023-10-03T10:17:27.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.393+0000 7f7ea77b6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea00a0a30 0x7f7ea011e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:27.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.393+0000 7f7ea77b6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ea00a1340 0x7f7ea011ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:27.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.393+0000 7f7ea77b6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ea0134170 0x7f7ea011f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:27.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.393+0000 7f7ea77b6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea01207b0 con 0x7f7ea00a0a30 2023-10-03T10:17:27.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.393+0000 7f7ea77b6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea01222a0 con 0x7f7ea0134170 2023-10-03T10:17:27.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.393+0000 7f7ea77b6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea009bf90 con 0x7f7ea00a1340 2023-10-03T10:17:27.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.393+0000 7f7ea4d51700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ea00a1340 0x7f7ea011ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:27.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.393+0000 7f7ea5d53700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ea0134170 0x7f7ea011f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:27.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.393+0000 7f7ea5d53700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ea0134170 0x7f7ea011f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36018/0 (socket says 172.21.15.3:36018) 2023-10-03T10:17:27.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.393+0000 7f7ea4d51700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ea00a1340 0x7f7ea011ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:39936/0 (socket says 172.21.15.3:39936) 2023-10-03T10:17:27.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.393+0000 7f7ea5d53700 1 -- 172.21.15.3:0/1270296282 learned_addr learned my addr 172.21.15.3:0/1270296282 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:27.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.393+0000 7f7ea5552700 1 --2- 172.21.15.3:0/1270296282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea00a0a30 0x7f7ea011e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:27.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.393+0000 7f7ea5d53700 1 -- 172.21.15.3:0/1270296282 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ea00a1340 msgr2=0x7f7ea011ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:27.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.393+0000 7f7ea5d53700 1 --2- 172.21.15.3:0/1270296282 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ea00a1340 0x7f7ea011ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.393+0000 7f7ea5d53700 1 -- 172.21.15.3:0/1270296282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea00a0a30 msgr2=0x7f7ea011e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:27.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.393+0000 7f7ea5d53700 1 --2- 172.21.15.3:0/1270296282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea00a0a30 0x7f7ea011e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.393+0000 7f7ea5d53700 1 -- 172.21.15.3:0/1270296282 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e98009a70 con 0x7f7ea0134170 2023-10-03T10:17:27.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.393+0000 7f7ea4d51700 1 --2- 172.21.15.3:0/1270296282 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ea00a1340 0x7f7ea011ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:27.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.393+0000 7f7ea5d53700 1 --2- 172.21.15.3:0/1270296282 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ea0134170 0x7f7ea011f2a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800efc0 tx=0x7f7e9800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:27.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.394+0000 7f7e95ffb700 1 -- 172.21.15.3:0/1270296282 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9800ca90 con 0x7f7ea0134170 2023-10-03T10:17:27.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.394+0000 7f7e95ffb700 1 -- 172.21.15.3:0/1270296282 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9800f040 con 0x7f7ea0134170 2023-10-03T10:17:27.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.394+0000 7f7ea77b6700 1 -- 172.21.15.3:0/1270296282 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e9c0094a0 con 0x7f7ea0134170 2023-10-03T10:17:27.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.394+0000 7f7e95ffb700 1 -- 172.21.15.3:0/1270296282 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9800ce10 con 0x7f7ea0134170 2023-10-03T10:17:27.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.394+0000 7f7ea77b6700 1 -- 172.21.15.3:0/1270296282 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea00a25e0 con 0x7f7ea0134170 2023-10-03T10:17:27.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.395+0000 7f7e95ffb700 1 -- 172.21.15.3:0/1270296282 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7e98014480 con 0x7f7ea0134170 2023-10-03T10:17:27.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.395+0000 7f7e95ffb700 1 -- 172.21.15.3:0/1270296282 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7e9803fd20 con 0x7f7ea0134170 2023-10-03T10:17:27.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.396+0000 7f7ea77b6700 1 -- 172.21.15.3:0/1270296282 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea0074ea0 con 0x7f7ea0134170 2023-10-03T10:17:27.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.405+0000 7f7e95ffb700 1 -- 172.21.15.3:0/1270296282 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7e98012930 con 0x7f7ea0134170 2023-10-03T10:17:27.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.429+0000 7fa1a6019700 1 Processor -- start 2023-10-03T10:17:27.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.429+0000 7fa1a6019700 1 -- start start 2023-10-03T10:17:27.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.430+0000 7fa1a6019700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa1a00a0a30 0x7fa1a00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:27.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.430+0000 7fa1a6019700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa1a009d630 0x7fa1a00a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:27.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.430+0000 7fa1a6019700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1a00a2a70 con 0x7fa1a0133930 2023-10-03T10:17:27.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.430+0000 7fa1a6019700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1a00a01d0 con 0x7fa1a00a0a30 2023-10-03T10:17:27.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.430+0000 7fa1a6019700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a009e2b0 con 0x7fa1a009d630 2023-10-03T10:17:27.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.430+0000 7fa19f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa1a00a0a30 0x7fa1a00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:27.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.430+0000 7fa19effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa1a009d630 0x7fa1a00a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:27.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.430+0000 7fa19effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa1a009d630 0x7fa1a00a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:39940/0 (socket says 172.21.15.3:39940) 2023-10-03T10:17:27.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.430+0000 7fa19f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa1a00a0a30 0x7fa1a00a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40386/0 (socket says 172.21.15.3:40386) 2023-10-03T10:17:27.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.430+0000 7fa19effd700 1 -- 172.21.15.3:0/2967375455 learned_addr learned my addr 172.21.15.3:0/2967375455 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:27.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.431+0000 7fa19effd700 1 -- 172.21.15.3:0/2967375455 >> v2:172.21.15.3:3300/0 conn(0x7fa1a00a0a30 msgr2=0x7fa1a00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:27.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.431+0000 7fa19effd700 1 --2- 172.21.15.3:0/2967375455 >> v2:172.21.15.3:3300/0 conn(0x7fa1a00a0a30 0x7fa1a00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.431+0000 7fa19effd700 1 -- 172.21.15.3:0/2967375455 >> v1:172.21.15.3:6789/0 conn(0x7fa1a0133930 legacy=0x7fa1a0133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:27.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.431+0000 7fa19effd700 1 -- 172.21.15.3:0/2967375455 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a009dc50 con 0x7fa1a009d630 2023-10-03T10:17:27.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.431+0000 7fa19effd700 1 --2- 172.21.15.3:0/2967375455 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa1a009d630 0x7fa1a00a1340 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa1940098f0 tx=0x7fa194009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a30dbe8d2a7df033 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:27.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.431+0000 7fa19e7fc700 1 -- 172.21.15.3:0/2967375455 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa194011830 con 0x7fa1a009d630 2023-10-03T10:17:27.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.431+0000 7fa19e7fc700 1 -- 172.21.15.3:0/2967375455 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa194011990 con 0x7fa1a009d630 2023-10-03T10:17:27.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.431+0000 7fa19e7fc700 1 -- 172.21.15.3:0/2967375455 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa194011d20 con 0x7fa1a009d630 2023-10-03T10:17:27.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.432+0000 7fa1a6019700 1 -- 172.21.15.3:0/2967375455 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa1a009d630 msgr2=0x7fa1a00a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:27.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.432+0000 7fa1a6019700 1 --2- 172.21.15.3:0/2967375455 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa1a009d630 0x7fa1a00a1340 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa1940098f0 tx=0x7fa194009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:27.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.432+0000 7fa1a6019700 1 -- 172.21.15.3:0/2967375455 shutdown_connections 2023-10-03T10:17:27.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.432+0000 7fa1a6019700 1 --2- 172.21.15.3:0/2967375455 >> v2:172.21.15.3:3300/0 conn(0x7fa1a00a0a30 0x7fa1a00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.432+0000 7fa1a6019700 1 --2- 172.21.15.3:0/2967375455 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa1a009d630 0x7fa1a00a1340 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.432+0000 7fa1a6019700 1 -- 172.21.15.3:0/2967375455 >> 172.21.15.3:0/2967375455 conn(0x7fa1a00a3050 msgr2=0x7fa1a00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:27.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.432+0000 7fa1a6019700 1 -- 172.21.15.3:0/2967375455 shutdown_connections 2023-10-03T10:17:27.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.432+0000 7fa1a6019700 1 -- 172.21.15.3:0/2967375455 wait complete. 2023-10-03T10:17:27.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.433+0000 7fa1a6019700 1 Processor -- start 2023-10-03T10:17:27.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.433+0000 7fa1a6019700 1 -- start start 2023-10-03T10:17:27.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.434+0000 7fa1a6019700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1a009d630 0x7fa1a011f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:27.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.434+0000 7fa1a6019700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1a00a0a30 0x7fa1a011f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:27.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.434+0000 7fa1a6019700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa1a0133930 0x7fa1a01c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:27.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.434+0000 7fa1a6019700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a0121fb0 con 0x7fa1a00a0a30 2023-10-03T10:17:27.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.434+0000 7fa1a6019700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a011eef0 con 0x7fa1a009d630 2023-10-03T10:17:27.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.434+0000 7fa1a6019700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a0120640 con 0x7fa1a0133930 2023-10-03T10:17:27.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.434+0000 7fa19f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1a009d630 0x7fa1a011f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:27.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.434+0000 7fa19f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1a009d630 0x7fa1a011f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36028/0 (socket says 172.21.15.3:36028) 2023-10-03T10:17:27.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.434+0000 7fa19f7fe700 1 -- 172.21.15.3:0/3182769111 learned_addr learned my addr 172.21.15.3:0/3182769111 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:27.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.434+0000 7fa19effd700 1 --2- 172.21.15.3:0/3182769111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1a00a0a30 0x7fa1a011f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:27.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.434+0000 7fa19ffff700 1 --2- 172.21.15.3:0/3182769111 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa1a0133930 0x7fa1a01c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:27.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.434+0000 7fa19f7fe700 1 -- 172.21.15.3:0/3182769111 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa1a0133930 msgr2=0x7fa1a01c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:27.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.434+0000 7fa19f7fe700 1 --2- 172.21.15.3:0/3182769111 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa1a0133930 0x7fa1a01c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.434+0000 7fa19f7fe700 1 -- 172.21.15.3:0/3182769111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1a00a0a30 msgr2=0x7fa1a011f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:27.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.434+0000 7fa19f7fe700 1 --2- 172.21.15.3:0/3182769111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1a00a0a30 0x7fa1a011f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.434+0000 7fa19f7fe700 1 -- 172.21.15.3:0/3182769111 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa194009550 con 0x7fa1a009d630 2023-10-03T10:17:27.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.435+0000 7fa19f7fe700 1 --2- 172.21.15.3:0/3182769111 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1a009d630 0x7fa1a011f490 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa19000e9b0 tx=0x7fa19000ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:27.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.435+0000 7fa187fff700 1 -- 172.21.15.3:0/3182769111 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa190009710 con 0x7fa1a009d630 2023-10-03T10:17:27.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.435+0000 7fa187fff700 1 -- 172.21.15.3:0/3182769111 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19000ee50 con 0x7fa1a009d630 2023-10-03T10:17:27.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.435+0000 7fa1a6019700 1 -- 172.21.15.3:0/3182769111 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a00a5040 con 0x7fa1a009d630 2023-10-03T10:17:27.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.435+0000 7fa187fff700 1 -- 172.21.15.3:0/3182769111 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19000ca50 con 0x7fa1a009d630 2023-10-03T10:17:27.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.435+0000 7fa1a6019700 1 -- 172.21.15.3:0/3182769111 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a01334e0 con 0x7fa1a009d630 2023-10-03T10:17:27.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.436+0000 7fa187fff700 1 -- 172.21.15.3:0/3182769111 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa19000cbb0 con 0x7fa1a009d630 2023-10-03T10:17:27.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.436+0000 7fa187fff700 1 -- 172.21.15.3:0/3182769111 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa19003faa0 con 0x7fa1a009d630 2023-10-03T10:17:27.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.436+0000 7fa1a6019700 1 -- 172.21.15.3:0/3182769111 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1a011ff10 con 0x7fa1a009d630 2023-10-03T10:17:27.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.446+0000 7fa187fff700 1 -- 172.21.15.3:0/3182769111 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa190017020 con 0x7fa1a009d630 2023-10-03T10:17:27.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.560+0000 7f1e5d1a2700 1 -- 172.21.15.3:0/2610001663 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1e5806b7c0 con 0x7f1e5812f8c0 2023-10-03T10:17:27.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.560+0000 7f1e54ff9700 1 -- 172.21.15.3:0/2610001663 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1e4c015020 con 0x7f1e5812f8c0 2023-10-03T10:17:27.561 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:27.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.564+0000 7f1e5d1a2700 1 -- 172.21.15.3:0/2610001663 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e5812f8c0 msgr2=0x7f1e5812c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:27.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.564+0000 7f1e5d1a2700 1 --2- 172.21.15.3:0/2610001663 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e5812f8c0 0x7f1e5812c760 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c000c00 tx=0x7f1e4c004750 comp rx=0 tx=0).stop 2023-10-03T10:17:27.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.564+0000 7f1e5d1a2700 1 -- 172.21.15.3:0/2610001663 shutdown_connections 2023-10-03T10:17:27.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.564+0000 7f1e5d1a2700 1 --2- 172.21.15.3:0/2610001663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e58130210 0x7f1e581c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.564+0000 7f1e5d1a2700 1 --2- 172.21.15.3:0/2610001663 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e5809dba0 0x7f1e5812c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.564+0000 7f1e5d1a2700 1 --2- 172.21.15.3:0/2610001663 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e5812f8c0 0x7f1e5812c760 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.564+0000 7f1e5d1a2700 1 -- 172.21.15.3:0/2610001663 >> 172.21.15.3:0/2610001663 conn(0x7f1e580a2540 msgr2=0x7f1e581b7910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:27.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.564+0000 7f1e5d1a2700 1 -- 172.21.15.3:0/2610001663 shutdown_connections 2023-10-03T10:17:27.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.564+0000 7f1e5d1a2700 1 -- 172.21.15.3:0/2610001663 wait complete. 2023-10-03T10:17:27.578 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:27.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.619+0000 7f7ea77b6700 1 -- 172.21.15.3:0/1270296282 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7ea00a28e0 con 0x7f7ea0134170 2023-10-03T10:17:27.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.619+0000 7f7e95ffb700 1 -- 172.21.15.3:0/1270296282 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7e9802a090 con 0x7f7ea0134170 2023-10-03T10:17:27.619 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:27.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.621+0000 7f7ea77b6700 1 -- 172.21.15.3:0/1270296282 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ea0134170 msgr2=0x7f7ea011f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:27.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.621+0000 7f7ea77b6700 1 --2- 172.21.15.3:0/1270296282 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ea0134170 0x7f7ea011f2a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800efc0 tx=0x7f7e9800c3a0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.621+0000 7f7ea77b6700 1 -- 172.21.15.3:0/1270296282 shutdown_connections 2023-10-03T10:17:27.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.621+0000 7f7ea77b6700 1 --2- 172.21.15.3:0/1270296282 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ea0134170 0x7f7ea011f2a0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.621+0000 7f7ea77b6700 1 --2- 172.21.15.3:0/1270296282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea00a0a30 0x7f7ea011e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.621+0000 7f7ea77b6700 1 --2- 172.21.15.3:0/1270296282 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ea00a1340 0x7f7ea011ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.621+0000 7f7ea77b6700 1 -- 172.21.15.3:0/1270296282 >> 172.21.15.3:0/1270296282 conn(0x7f7ea00a6550 msgr2=0x7f7ea01b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:27.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.621+0000 7f7ea77b6700 1 -- 172.21.15.3:0/1270296282 shutdown_connections 2023-10-03T10:17:27.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.621+0000 7f7ea77b6700 1 -- 172.21.15.3:0/1270296282 wait complete. 2023-10-03T10:17:27.635 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:27.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.654+0000 7fa1a6019700 1 -- 172.21.15.3:0/3182769111 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa1a00747f0 con 0x7fa1a009d630 2023-10-03T10:17:27.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.654+0000 7fa187fff700 1 -- 172.21.15.3:0/3182769111 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa19002b090 con 0x7fa1a009d630 2023-10-03T10:17:27.654 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:27.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.656+0000 7fa1a6019700 1 -- 172.21.15.3:0/3182769111 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1a009d630 msgr2=0x7fa1a011f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:27.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.656+0000 7fa1a6019700 1 --2- 172.21.15.3:0/3182769111 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1a009d630 0x7fa1a011f490 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa19000e9b0 tx=0x7fa19000ecc0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.657+0000 7fa1a6019700 1 -- 172.21.15.3:0/3182769111 shutdown_connections 2023-10-03T10:17:27.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.657+0000 7fa1a6019700 1 --2- 172.21.15.3:0/3182769111 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa1a0133930 0x7fa1a01c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.657+0000 7fa1a6019700 1 --2- 172.21.15.3:0/3182769111 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1a009d630 0x7fa1a011f490 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.657+0000 7fa1a6019700 1 --2- 172.21.15.3:0/3182769111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1a00a0a30 0x7fa1a011f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:27.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.657+0000 7fa1a6019700 1 -- 172.21.15.3:0/3182769111 >> 172.21.15.3:0/3182769111 conn(0x7fa1a00a3050 msgr2=0x7fa1a01bbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:27.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.657+0000 7fa1a6019700 1 -- 172.21.15.3:0/3182769111 shutdown_connections 2023-10-03T10:17:27.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:27.657+0000 7fa1a6019700 1 -- 172.21.15.3:0/3182769111 wait complete. 2023-10-03T10:17:27.670 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:28.116 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:28.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.262+0000 7f216b704700 1 Processor -- start 2023-10-03T10:17:28.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.263+0000 7f216b704700 1 -- start start 2023-10-03T10:17:28.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.263+0000 7f216b704700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2164136820 0x7f2164138c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:28.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.263+0000 7f216b704700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2164139140 0x7f216413b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:28.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.263+0000 7f216b704700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f216406bc00 con 0x7f2164133a80 2023-10-03T10:17:28.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.263+0000 7f216b704700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f216406c570 con 0x7f2164139140 2023-10-03T10:17:28.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.263+0000 7f216b704700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21640a0040 con 0x7f2164136820 2023-10-03T10:17:28.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.263+0000 7f2169ca1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2164139140 0x7f216413b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:28.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.263+0000 7f2168c9f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2164136820 0x7f2164138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:28.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.263+0000 7f2169ca1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2164139140 0x7f216413b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40398/0 (socket says 172.21.15.3:40398) 2023-10-03T10:17:28.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.263+0000 7f2168c9f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2164136820 0x7f2164138c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36042/0 (socket says 172.21.15.3:36042) 2023-10-03T10:17:28.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.263+0000 7f2169ca1700 1 -- 172.21.15.3:0/2521615887 learned_addr learned my addr 172.21.15.3:0/2521615887 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:28.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.264+0000 7f2169ca1700 1 -- 172.21.15.3:0/2521615887 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2164136820 msgr2=0x7f2164138c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:28.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.264+0000 7f2169ca1700 1 --2- 172.21.15.3:0/2521615887 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2164136820 0x7f2164138c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.264+0000 7f2169ca1700 1 -- 172.21.15.3:0/2521615887 >> v1:172.21.15.3:6789/0 conn(0x7f2164133a80 legacy=0x7f2164133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:28.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.264+0000 7f2169ca1700 1 -- 172.21.15.3:0/2521615887 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2164133860 con 0x7f2164139140 2023-10-03T10:17:28.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.264+0000 7f2168c9f700 1 --2- 172.21.15.3:0/2521615887 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2164136820 0x7f2164138c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:28.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.264+0000 7f2169ca1700 1 --2- 172.21.15.3:0/2521615887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2164139140 0x7f216413b520 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f215c009e50 tx=0x7f215c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5702a1ff7fad47b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:28.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.264+0000 7f215bfff700 1 -- 172.21.15.3:0/2521615887 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c00e070 con 0x7f2164139140 2023-10-03T10:17:28.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.264+0000 7f215bfff700 1 -- 172.21.15.3:0/2521615887 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215c00b910 con 0x7f2164139140 2023-10-03T10:17:28.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.264+0000 7f215bfff700 1 -- 172.21.15.3:0/2521615887 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c00bcc0 con 0x7f2164139140 2023-10-03T10:17:28.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.265+0000 7f216b704700 1 -- 172.21.15.3:0/2521615887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2164139140 msgr2=0x7f216413b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:28.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.265+0000 7f216b704700 1 --2- 172.21.15.3:0/2521615887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2164139140 0x7f216413b520 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f215c009e50 tx=0x7f215c00b330 comp rx=0 tx=0).stop 2023-10-03T10:17:28.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.265+0000 7f216b704700 1 -- 172.21.15.3:0/2521615887 shutdown_connections 2023-10-03T10:17:28.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.265+0000 7f216b704700 1 --2- 172.21.15.3:0/2521615887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2164139140 0x7f216413b520 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.265+0000 7f216b704700 1 --2- 172.21.15.3:0/2521615887 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2164136820 0x7f2164138c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.265+0000 7f216b704700 1 -- 172.21.15.3:0/2521615887 >> 172.21.15.3:0/2521615887 conn(0x7f21640a4210 msgr2=0x7f21640a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:28.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.265+0000 7f216b704700 1 -- 172.21.15.3:0/2521615887 shutdown_connections 2023-10-03T10:17:28.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.265+0000 7f216b704700 1 -- 172.21.15.3:0/2521615887 wait complete. 2023-10-03T10:17:28.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.266+0000 7f216b704700 1 Processor -- start 2023-10-03T10:17:28.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.266+0000 7f216b704700 1 -- start start 2023-10-03T10:17:28.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.266+0000 7f216b704700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2164133a80 0x7f216409d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:28.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.266+0000 7f216b704700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2164136820 0x7f216409d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:28.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.266+0000 7f216b704700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2164139140 0x7f21641c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:28.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.266+0000 7f216b704700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2164130b90 con 0x7f2164136820 2023-10-03T10:17:28.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.266+0000 7f216b704700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216412f0c0 con 0x7f2164133a80 2023-10-03T10:17:28.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.266+0000 7f216b704700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2164138c10 con 0x7f2164139140 2023-10-03T10:17:28.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.266+0000 7f2168c9f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2164136820 0x7f216409d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:28.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.266+0000 7f2168c9f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2164136820 0x7f216409d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40414/0 (socket says 172.21.15.3:40414) 2023-10-03T10:17:28.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.266+0000 7f2169ca1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2164139140 0x7f21641c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:28.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.266+0000 7f2168c9f700 1 -- 172.21.15.3:0/461961468 learned_addr learned my addr 172.21.15.3:0/461961468 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:28.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.266+0000 7f21694a0700 1 --2- 172.21.15.3:0/461961468 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2164133a80 0x7f216409d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:28.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.267+0000 7f2168c9f700 1 -- 172.21.15.3:0/461961468 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2164139140 msgr2=0x7f21641c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:28.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.267+0000 7f2168c9f700 1 --2- 172.21.15.3:0/461961468 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2164139140 0x7f21641c83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.267+0000 7f2168c9f700 1 -- 172.21.15.3:0/461961468 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2164133a80 msgr2=0x7f216409d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:28.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.267+0000 7f2168c9f700 1 --2- 172.21.15.3:0/461961468 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2164133a80 0x7f216409d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.267+0000 7f2168c9f700 1 -- 172.21.15.3:0/461961468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21540094a0 con 0x7f2164136820 2023-10-03T10:17:28.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.267+0000 7f2169ca1700 1 --2- 172.21.15.3:0/461961468 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2164139140 0x7f21641c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:28.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.267+0000 7f2168c9f700 1 --2- 172.21.15.3:0/461961468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2164136820 0x7f216409d8b0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f215400ebe0 tx=0x7f215400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:28.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.267+0000 7f2159ffb700 1 -- 172.21.15.3:0/461961468 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215400c850 con 0x7f2164136820 2023-10-03T10:17:28.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.267+0000 7f2159ffb700 1 -- 172.21.15.3:0/461961468 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215400c9b0 con 0x7f2164136820 2023-10-03T10:17:28.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.267+0000 7f216b704700 1 -- 172.21.15.3:0/461961468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f215c009b30 con 0x7f2164136820 2023-10-03T10:17:28.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.267+0000 7f2159ffb700 1 -- 172.21.15.3:0/461961468 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215400cd40 con 0x7f2164136820 2023-10-03T10:17:28.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.267+0000 7f216b704700 1 -- 172.21.15.3:0/461961468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2164072730 con 0x7f2164136820 2023-10-03T10:17:28.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.268+0000 7f2159ffb700 1 -- 172.21.15.3:0/461961468 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2154013470 con 0x7f2164136820 2023-10-03T10:17:28.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.268+0000 7f216b704700 1 -- 172.21.15.3:0/461961468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21641309d0 con 0x7f2164136820 2023-10-03T10:17:28.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.269+0000 7f2159ffb700 1 -- 172.21.15.3:0/461961468 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f215403f7d0 con 0x7f2164136820 2023-10-03T10:17:28.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.274+0000 7f2159ffb700 1 -- 172.21.15.3:0/461961468 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f21641309d0 con 0x7f2164136820 2023-10-03T10:17:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.482+0000 7f216b704700 1 -- 172.21.15.3:0/461961468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2164074ea0 con 0x7f2164136820 2023-10-03T10:17:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.482+0000 7f2159ffb700 1 -- 172.21.15.3:0/461961468 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f215400cea0 con 0x7f2164136820 2023-10-03T10:17:28.482 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.484+0000 7f216b704700 1 -- 172.21.15.3:0/461961468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2164136820 msgr2=0x7f216409d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.484+0000 7f216b704700 1 --2- 172.21.15.3:0/461961468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2164136820 0x7f216409d8b0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f215400ebe0 tx=0x7f215400eef0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.484+0000 7f216b704700 1 -- 172.21.15.3:0/461961468 shutdown_connections 2023-10-03T10:17:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.484+0000 7f216b704700 1 --2- 172.21.15.3:0/461961468 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2164139140 0x7f21641c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.484+0000 7f216b704700 1 --2- 172.21.15.3:0/461961468 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2164133a80 0x7f216409d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.484+0000 7f216b704700 1 --2- 172.21.15.3:0/461961468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2164136820 0x7f216409d8b0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.484+0000 7f216b704700 1 -- 172.21.15.3:0/461961468 >> 172.21.15.3:0/461961468 conn(0x7f21640a4210 msgr2=0x7f21640a0890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.484+0000 7f216b704700 1 -- 172.21.15.3:0/461961468 shutdown_connections 2023-10-03T10:17:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.484+0000 7f216b704700 1 -- 172.21.15.3:0/461961468 wait complete. 2023-10-03T10:17:28.498 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:28.579 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:28.637 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:28.671 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:28.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.728+0000 7f21ee586700 1 Processor -- start 2023-10-03T10:17:28.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.728+0000 7f21ee586700 1 -- start start 2023-10-03T10:17:28.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.729+0000 7f21ee586700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f21e809d6f0 0x7f21e809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:28.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.729+0000 7f21ee586700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21e809e440 0x7f21e8134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:28.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.729+0000 7f21ee586700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e809b120 con 0x7f21e809e000 2023-10-03T10:17:28.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.729+0000 7f21ee586700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e806b460 con 0x7f21e809d6f0 2023-10-03T10:17:28.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.729+0000 7f21ee586700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f21e809a7e0 con 0x7f21e809e440 2023-10-03T10:17:28.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.729+0000 7f21ecb23700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21e809e440 0x7f21e8134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:28.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.729+0000 7f21e7fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f21e809d6f0 0x7f21e809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:28.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.729+0000 7f21ecb23700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21e809e440 0x7f21e8134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:39968/0 (socket says 172.21.15.3:39968) 2023-10-03T10:17:28.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.729+0000 7f21ecb23700 1 -- 172.21.15.3:0/2619855125 learned_addr learned my addr 172.21.15.3:0/2619855125 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:28.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.729+0000 7f21ecb23700 1 -- 172.21.15.3:0/2619855125 >> v2:172.21.15.3:3300/0 conn(0x7f21e809d6f0 msgr2=0x7f21e809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:28.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.729+0000 7f21ecb23700 1 --2- 172.21.15.3:0/2619855125 >> v2:172.21.15.3:3300/0 conn(0x7f21e809d6f0 0x7f21e809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.729+0000 7f21ecb23700 1 -- 172.21.15.3:0/2619855125 >> v1:172.21.15.3:6789/0 conn(0x7f21e809e000 legacy=0x7f21e81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:28.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.730+0000 7f21ecb23700 1 -- 172.21.15.3:0/2619855125 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e809a500 con 0x7f21e809e440 2023-10-03T10:17:28.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.730+0000 7f21e7fff700 1 --2- 172.21.15.3:0/2619855125 >> v2:172.21.15.3:3300/0 conn(0x7f21e809d6f0 0x7f21e809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:28.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.730+0000 7f21ecb23700 1 --2- 172.21.15.3:0/2619855125 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21e809e440 0x7f21e8134100 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009e80 tx=0x7f21e000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5594bbfe5932f173 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:28.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.730+0000 7f21e6ffd700 1 -- 172.21.15.3:0/2619855125 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e000e070 con 0x7f21e809e440 2023-10-03T10:17:28.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.730+0000 7f21e6ffd700 1 -- 172.21.15.3:0/2619855125 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e000b8d0 con 0x7f21e809e440 2023-10-03T10:17:28.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.730+0000 7f21e6ffd700 1 -- 172.21.15.3:0/2619855125 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e000bc80 con 0x7f21e809e440 2023-10-03T10:17:28.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.731+0000 7f21ee586700 1 -- 172.21.15.3:0/2619855125 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21e809e440 msgr2=0x7f21e8134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:28.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.731+0000 7f21ee586700 1 --2- 172.21.15.3:0/2619855125 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21e809e440 0x7f21e8134100 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009e80 tx=0x7f21e000b330 comp rx=0 tx=0).stop 2023-10-03T10:17:28.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.731+0000 7f21ee586700 1 -- 172.21.15.3:0/2619855125 shutdown_connections 2023-10-03T10:17:28.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.731+0000 7f21ee586700 1 --2- 172.21.15.3:0/2619855125 >> v2:172.21.15.3:3300/0 conn(0x7f21e809d6f0 0x7f21e809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.731+0000 7f21ee586700 1 --2- 172.21.15.3:0/2619855125 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21e809e440 0x7f21e8134100 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.731+0000 7f21ee586700 1 -- 172.21.15.3:0/2619855125 >> 172.21.15.3:0/2619855125 conn(0x7f21e8066f50 msgr2=0x7f21e809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:28.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.731+0000 7f21ee586700 1 -- 172.21.15.3:0/2619855125 shutdown_connections 2023-10-03T10:17:28.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.731+0000 7f21ee586700 1 -- 172.21.15.3:0/2619855125 wait complete. 2023-10-03T10:17:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.732+0000 7f21ee586700 1 Processor -- start 2023-10-03T10:17:28.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.732+0000 7f21ee586700 1 -- start start 2023-10-03T10:17:28.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.733+0000 7f21ee586700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21e809d6f0 0x7f21e81a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:28.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.733+0000 7f21ee586700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21e809e000 0x7f21e81a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:28.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.733+0000 7f21ee586700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21e809e440 0x7f21e81a6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:28.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.733+0000 7f21ee586700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e81aa360 con 0x7f21e809d6f0 2023-10-03T10:17:28.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.733+0000 7f21ee586700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e81a99a0 con 0x7f21e809e000 2023-10-03T10:17:28.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.733+0000 7f21ee586700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f21e81a8ce0 con 0x7f21e809e440 2023-10-03T10:17:28.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.733+0000 7f21e7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21e809d6f0 0x7f21e81a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:28.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.733+0000 7f21e77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21e809e000 0x7f21e81a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:28.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.733+0000 7f21e7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21e809d6f0 0x7f21e81a5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40436/0 (socket says 172.21.15.3:40436) 2023-10-03T10:17:28.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.733+0000 7f21e7fff700 1 -- 172.21.15.3:0/684453315 learned_addr learned my addr 172.21.15.3:0/684453315 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:28.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.733+0000 7f21ecb23700 1 --2- 172.21.15.3:0/684453315 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21e809e440 0x7f21e81a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:28.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.733+0000 7f21ecb23700 1 -- 172.21.15.3:0/684453315 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21e809e000 msgr2=0x7f21e81a6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:28.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.733+0000 7f21ecb23700 1 --2- 172.21.15.3:0/684453315 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21e809e000 0x7f21e81a6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.733+0000 7f21ecb23700 1 -- 172.21.15.3:0/684453315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21e809d6f0 msgr2=0x7f21e81a5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:28.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.733+0000 7f21ecb23700 1 --2- 172.21.15.3:0/684453315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21e809d6f0 0x7f21e81a5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.733+0000 7f21ecb23700 1 -- 172.21.15.3:0/684453315 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d80094a0 con 0x7f21e809e440 2023-10-03T10:17:28.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.733+0000 7f21e77fe700 1 --2- 172.21.15.3:0/684453315 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21e809e000 0x7f21e81a6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:28.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.734+0000 7f21ecb23700 1 --2- 172.21.15.3:0/684453315 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21e809e440 0x7f21e81a6950 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009e50 tx=0x7f21e0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:28.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.734+0000 7f21e4ff9700 1 -- 172.21.15.3:0/684453315 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e0004800 con 0x7f21e809e440 2023-10-03T10:17:28.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.734+0000 7f21e4ff9700 1 -- 172.21.15.3:0/684453315 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e0004960 con 0x7f21e809e440 2023-10-03T10:17:28.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.734+0000 7f21e4ff9700 1 -- 172.21.15.3:0/684453315 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e001c680 con 0x7f21e809e440 2023-10-03T10:17:28.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.734+0000 7f21ee586700 1 -- 172.21.15.3:0/684453315 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e0009b30 con 0x7f21e809e440 2023-10-03T10:17:28.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.734+0000 7f21ee586700 1 -- 172.21.15.3:0/684453315 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e80a2050 con 0x7f21e809e440 2023-10-03T10:17:28.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.735+0000 7f21e4ff9700 1 -- 172.21.15.3:0/684453315 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f21e001c7e0 con 0x7f21e809e440 2023-10-03T10:17:28.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.735+0000 7f21ee586700 1 -- 172.21.15.3:0/684453315 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e81a9540 con 0x7f21e809e440 2023-10-03T10:17:28.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.735+0000 7f21e4ff9700 1 -- 172.21.15.3:0/684453315 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f21e0041110 con 0x7f21e809e440 2023-10-03T10:17:28.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.740+0000 7f21e4ff9700 1 -- 172.21.15.3:0/684453315 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f21e001a330 con 0x7f21e809e440 2023-10-03T10:17:28.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.782+0000 7f488023a700 1 Processor -- start 2023-10-03T10:17:28.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.782+0000 7f488023a700 1 -- start start 2023-10-03T10:17:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.783+0000 7f488023a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4878130750 0x7f4878130b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.783+0000 7f488023a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f487809d6f0 0x7f487809db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.783+0000 7f488023a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f487809e080 0x7f48781374b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.783+0000 7f488023a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f487809c190 con 0x7f487809d6f0 2023-10-03T10:17:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.783+0000 7f488023a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487809e520 con 0x7f4878130750 2023-10-03T10:17:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.783+0000 7f488023a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f487809b090 con 0x7f487809e080 2023-10-03T10:17:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.783+0000 7f487e7d7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f487809e080 0x7f48781374b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.783+0000 7f487d7d5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f487809d6f0 0x7f487809db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.783+0000 7f487e7d7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f487809e080 0x7f48781374b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:39982/0 (socket says 172.21.15.3:39982) 2023-10-03T10:17:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.783+0000 7f487e7d7700 1 -- 172.21.15.3:0/629247932 learned_addr learned my addr 172.21.15.3:0/629247932 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.783+0000 7f487dfd6700 1 --2- 172.21.15.3:0/629247932 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4878130750 0x7f4878130b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.783+0000 7f487e7d7700 1 -- 172.21.15.3:0/629247932 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4878130750 msgr2=0x7f4878130b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.783+0000 7f487e7d7700 1 --2- 172.21.15.3:0/629247932 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4878130750 0x7f4878130b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.783+0000 7f487e7d7700 1 -- 172.21.15.3:0/629247932 >> v2:172.21.15.3:3300/0 conn(0x7f487809d6f0 msgr2=0x7f487809db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.783+0000 7f487e7d7700 1 --2- 172.21.15.3:0/629247932 >> v2:172.21.15.3:3300/0 conn(0x7f487809d6f0 0x7f487809db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.783+0000 7f487e7d7700 1 -- 172.21.15.3:0/629247932 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487809a6c0 con 0x7f487809e080 2023-10-03T10:17:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.783+0000 7f487e7d7700 1 --2- 172.21.15.3:0/629247932 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f487809e080 0x7f48781374b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4870009e50 tx=0x7f487000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=758c0c91a4ebd760 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.784+0000 7f487cfd4700 1 -- 172.21.15.3:0/629247932 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487000e070 con 0x7f487809e080 2023-10-03T10:17:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.784+0000 7f487cfd4700 1 -- 172.21.15.3:0/629247932 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f487000b910 con 0x7f487809e080 2023-10-03T10:17:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.784+0000 7f487cfd4700 1 -- 172.21.15.3:0/629247932 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487000bcc0 con 0x7f487809e080 2023-10-03T10:17:28.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.784+0000 7f488023a700 1 -- 172.21.15.3:0/629247932 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f487809e080 msgr2=0x7f48781374b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:28.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.784+0000 7f488023a700 1 --2- 172.21.15.3:0/629247932 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f487809e080 0x7f48781374b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4870009e50 tx=0x7f487000b330 comp rx=0 tx=0).stop 2023-10-03T10:17:28.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.784+0000 7f488023a700 1 -- 172.21.15.3:0/629247932 shutdown_connections 2023-10-03T10:17:28.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.784+0000 7f488023a700 1 --2- 172.21.15.3:0/629247932 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f487809e080 0x7f48781374b0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.784+0000 7f488023a700 1 --2- 172.21.15.3:0/629247932 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4878130750 0x7f4878130b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.784+0000 7f488023a700 1 --2- 172.21.15.3:0/629247932 >> v2:172.21.15.3:3300/0 conn(0x7f487809d6f0 0x7f487809db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.784+0000 7f488023a700 1 -- 172.21.15.3:0/629247932 >> 172.21.15.3:0/629247932 conn(0x7f487809f470 msgr2=0x7f48780a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:28.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.785+0000 7f488023a700 1 -- 172.21.15.3:0/629247932 shutdown_connections 2023-10-03T10:17:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.785+0000 7f488023a700 1 -- 172.21.15.3:0/629247932 wait complete. 2023-10-03T10:17:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.786+0000 7f488023a700 1 Processor -- start 2023-10-03T10:17:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.786+0000 7f488023a700 1 -- start start 2023-10-03T10:17:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.786+0000 7f488023a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f487809d6f0 0x7f4878135a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.786+0000 7f488023a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f487809e080 0x7f4878135f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.786+0000 7f488023a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4878130750 0x7f48781364c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.786+0000 7f488023a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487809b2b0 con 0x7f4878130750 2023-10-03T10:17:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.786+0000 7f488023a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487812bb40 con 0x7f487809e080 2023-10-03T10:17:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.787+0000 7f488023a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4878139840 con 0x7f487809d6f0 2023-10-03T10:17:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.787+0000 7f487dfd6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f487809d6f0 0x7f4878135a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.787+0000 7f487dfd6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f487809d6f0 0x7f4878135a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:39996/0 (socket says 172.21.15.3:39996) 2023-10-03T10:17:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.787+0000 7f487dfd6700 1 -- 172.21.15.3:0/3165505335 learned_addr learned my addr 172.21.15.3:0/3165505335 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.787+0000 7f487e7d7700 1 --2- 172.21.15.3:0/3165505335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4878130750 0x7f48781364c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.787+0000 7f487d7d5700 1 --2- 172.21.15.3:0/3165505335 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f487809e080 0x7f4878135f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.787+0000 7f487e7d7700 1 -- 172.21.15.3:0/3165505335 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f487809d6f0 msgr2=0x7f4878135a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.787+0000 7f487e7d7700 1 --2- 172.21.15.3:0/3165505335 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f487809d6f0 0x7f4878135a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.787+0000 7f487e7d7700 1 -- 172.21.15.3:0/3165505335 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f487809e080 msgr2=0x7f4878135f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.787+0000 7f487e7d7700 1 --2- 172.21.15.3:0/3165505335 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f487809e080 0x7f4878135f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.787+0000 7f487e7d7700 1 -- 172.21.15.3:0/3165505335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4870009b30 con 0x7f4878130750 2023-10-03T10:17:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.787+0000 7f487dfd6700 1 --2- 172.21.15.3:0/3165505335 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f487809d6f0 0x7f4878135a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.787+0000 7f487e7d7700 1 --2- 172.21.15.3:0/3165505335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4878130750 0x7f48781364c0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f487804f490 tx=0x7f4870004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.787+0000 7f486e7fc700 1 -- 172.21.15.3:0/3165505335 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48700092e0 con 0x7f4878130750 2023-10-03T10:17:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.788+0000 7f486e7fc700 1 -- 172.21.15.3:0/3165505335 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f487000ce80 con 0x7f4878130750 2023-10-03T10:17:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.788+0000 7f488023a700 1 -- 172.21.15.3:0/3165505335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4878137cb0 con 0x7f4878130750 2023-10-03T10:17:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.788+0000 7f486e7fc700 1 -- 172.21.15.3:0/3165505335 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48700044b0 con 0x7f4878130750 2023-10-03T10:17:28.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.788+0000 7f488023a700 1 -- 172.21.15.3:0/3165505335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4878138400 con 0x7f4878130750 2023-10-03T10:17:28.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.789+0000 7f486e7fc700 1 -- 172.21.15.3:0/3165505335 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4870004610 con 0x7f4878130750 2023-10-03T10:17:28.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.789+0000 7f488023a700 1 -- 172.21.15.3:0/3165505335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4878132a10 con 0x7f4878130750 2023-10-03T10:17:28.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.789+0000 7f486e7fc700 1 -- 172.21.15.3:0/3165505335 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f48700409d0 con 0x7f4878130750 2023-10-03T10:17:28.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.794+0000 7f486e7fc700 1 -- 172.21.15.3:0/3165505335 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4870011c10 con 0x7f4878130750 2023-10-03T10:17:28.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.830+0000 7fba7b2bf700 1 Processor -- start 2023-10-03T10:17:28.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.830+0000 7fba7b2bf700 1 -- start start 2023-10-03T10:17:28.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.831+0000 7fba7b2bf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fba7409d6f0 0x7fba7409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:28.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.831+0000 7fba7b2bf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fba7409e410 0x7fba74134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:28.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.831+0000 7fba7b2bf700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba7409b120 con 0x7fba7409e000 2023-10-03T10:17:28.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.831+0000 7fba7b2bf700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba7406b460 con 0x7fba7409e410 2023-10-03T10:17:28.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.831+0000 7fba7b2bf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba7409a7e0 con 0x7fba7409d6f0 2023-10-03T10:17:28.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.831+0000 7fba7985c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fba7409e410 0x7fba74134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:28.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.831+0000 7fba7905b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fba7409d6f0 0x7fba7409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:28.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.831+0000 7fba7985c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fba7409e410 0x7fba74134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40464/0 (socket says 172.21.15.3:40464) 2023-10-03T10:17:28.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.831+0000 7fba7985c700 1 -- 172.21.15.3:0/1457840345 learned_addr learned my addr 172.21.15.3:0/1457840345 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:28.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.831+0000 7fba7905b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fba7409d6f0 0x7fba7409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:40004/0 (socket says 172.21.15.3:40004) 2023-10-03T10:17:28.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.831+0000 7fba7985c700 1 -- 172.21.15.3:0/1457840345 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fba7409d6f0 msgr2=0x7fba7409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:28.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.831+0000 7fba7985c700 1 --2- 172.21.15.3:0/1457840345 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fba7409d6f0 0x7fba7409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.831+0000 7fba7985c700 1 -- 172.21.15.3:0/1457840345 >> v1:172.21.15.3:6789/0 conn(0x7fba7409e000 legacy=0x7fba741339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:28.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.831+0000 7fba7985c700 1 -- 172.21.15.3:0/1457840345 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7409a500 con 0x7fba7409e410 2023-10-03T10:17:28.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.831+0000 7fba7905b700 1 --2- 172.21.15.3:0/1457840345 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fba7409d6f0 0x7fba7409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:28.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.832+0000 7fba7985c700 1 --2- 172.21.15.3:0/1457840345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba7409e410 0x7fba74134100 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009e50 tx=0x7fba6c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a7706fc8f02423 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:28.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.832+0000 7fba6bfff700 1 -- 172.21.15.3:0/1457840345 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c00e070 con 0x7fba7409e410 2023-10-03T10:17:28.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.832+0000 7fba6bfff700 1 -- 172.21.15.3:0/1457840345 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6c00b910 con 0x7fba7409e410 2023-10-03T10:17:28.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.832+0000 7fba6bfff700 1 -- 172.21.15.3:0/1457840345 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c00bcc0 con 0x7fba7409e410 2023-10-03T10:17:28.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.832+0000 7fba7b2bf700 1 -- 172.21.15.3:0/1457840345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba7409e410 msgr2=0x7fba74134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:28.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.832+0000 7fba7b2bf700 1 --2- 172.21.15.3:0/1457840345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba7409e410 0x7fba74134100 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009e50 tx=0x7fba6c00b330 comp rx=0 tx=0).stop 2023-10-03T10:17:28.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.832+0000 7fba7b2bf700 1 -- 172.21.15.3:0/1457840345 shutdown_connections 2023-10-03T10:17:28.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.832+0000 7fba7b2bf700 1 --2- 172.21.15.3:0/1457840345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba7409e410 0x7fba74134100 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.832+0000 7fba7b2bf700 1 --2- 172.21.15.3:0/1457840345 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fba7409d6f0 0x7fba7409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.832+0000 7fba7b2bf700 1 -- 172.21.15.3:0/1457840345 >> 172.21.15.3:0/1457840345 conn(0x7fba74066f50 msgr2=0x7fba7409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:28.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.833+0000 7fba7b2bf700 1 -- 172.21.15.3:0/1457840345 shutdown_connections 2023-10-03T10:17:28.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.833+0000 7fba7b2bf700 1 -- 172.21.15.3:0/1457840345 wait complete. 2023-10-03T10:17:28.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.834+0000 7fba7b2bf700 1 Processor -- start 2023-10-03T10:17:28.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.834+0000 7fba7b2bf700 1 -- start start 2023-10-03T10:17:28.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.834+0000 7fba7b2bf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba7409d6f0 0x7fba741a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:28.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.834+0000 7fba7b2bf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fba7409e000 0x7fba741a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:28.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.834+0000 7fba7b2bf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fba7409e410 0x7fba741a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:28.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.834+0000 7fba7b2bf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba741a9870 con 0x7fba7409d6f0 2023-10-03T10:17:28.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.834+0000 7fba7b2bf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba741a7b90 con 0x7fba7409e410 2023-10-03T10:17:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.834+0000 7fba7b2bf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba741a9a70 con 0x7fba7409e000 2023-10-03T10:17:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.834+0000 7fba7905b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba7409d6f0 0x7fba741a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.834+0000 7fba7885a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fba7409e000 0x7fba741a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.834+0000 7fba7905b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba7409d6f0 0x7fba741a60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40476/0 (socket says 172.21.15.3:40476) 2023-10-03T10:17:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.834+0000 7fba7905b700 1 -- 172.21.15.3:0/174875519 learned_addr learned my addr 172.21.15.3:0/174875519 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.834+0000 7fba7985c700 1 --2- 172.21.15.3:0/174875519 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fba7409e410 0x7fba741a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.835+0000 7fba7905b700 1 -- 172.21.15.3:0/174875519 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fba7409e000 msgr2=0x7fba741a65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.835+0000 7fba7905b700 1 --2- 172.21.15.3:0/174875519 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fba7409e000 0x7fba741a65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.835+0000 7fba7905b700 1 -- 172.21.15.3:0/174875519 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fba7409e410 msgr2=0x7fba741a6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.835+0000 7fba7905b700 1 --2- 172.21.15.3:0/174875519 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fba7409e410 0x7fba741a6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.835+0000 7fba7905b700 1 -- 172.21.15.3:0/174875519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba700094a0 con 0x7fba7409d6f0 2023-10-03T10:17:28.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.835+0000 7fba7885a700 1 --2- 172.21.15.3:0/174875519 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fba7409e000 0x7fba741a65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:28.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.835+0000 7fba7905b700 1 --2- 172.21.15.3:0/174875519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba7409d6f0 0x7fba741a60a0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fba7000ebe0 tx=0x7fba7000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:28.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.835+0000 7fba69ffb700 1 -- 172.21.15.3:0/174875519 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7000c850 con 0x7fba7409d6f0 2023-10-03T10:17:28.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.835+0000 7fba69ffb700 1 -- 172.21.15.3:0/174875519 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7000c9b0 con 0x7fba7409d6f0 2023-10-03T10:17:28.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.835+0000 7fba7b2bf700 1 -- 172.21.15.3:0/174875519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba6c009b30 con 0x7fba7409d6f0 2023-10-03T10:17:28.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.835+0000 7fba69ffb700 1 -- 172.21.15.3:0/174875519 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7000cd40 con 0x7fba7409d6f0 2023-10-03T10:17:28.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.835+0000 7fba7b2bf700 1 -- 172.21.15.3:0/174875519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba741a88f0 con 0x7fba7409d6f0 2023-10-03T10:17:28.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.836+0000 7fba69ffb700 1 -- 172.21.15.3:0/174875519 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fba70013470 con 0x7fba7409d6f0 2023-10-03T10:17:28.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.836+0000 7fba7b2bf700 1 -- 172.21.15.3:0/174875519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba74074ea0 con 0x7fba7409d6f0 2023-10-03T10:17:28.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.837+0000 7fba69ffb700 1 -- 172.21.15.3:0/174875519 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fba7003f7d0 con 0x7fba7409d6f0 2023-10-03T10:17:28.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.841+0000 7fba69ffb700 1 -- 172.21.15.3:0/174875519 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fba70013720 con 0x7fba7409d6f0 2023-10-03T10:17:28.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.949+0000 7f21ee586700 1 -- 172.21.15.3:0/684453315 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f21e8074ea0 con 0x7f21e809e440 2023-10-03T10:17:28.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.950+0000 7f21e4ff9700 1 -- 172.21.15.3:0/684453315 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f21e0024ba0 con 0x7f21e809e440 2023-10-03T10:17:28.950 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:28.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.952+0000 7f21ee586700 1 -- 172.21.15.3:0/684453315 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21e809e440 msgr2=0x7f21e81a6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:28.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.952+0000 7f21ee586700 1 --2- 172.21.15.3:0/684453315 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21e809e440 0x7f21e81a6950 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009e50 tx=0x7f21e0004ad0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.952+0000 7f21ee586700 1 -- 172.21.15.3:0/684453315 shutdown_connections 2023-10-03T10:17:28.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.952+0000 7f21ee586700 1 --2- 172.21.15.3:0/684453315 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21e809e440 0x7f21e81a6950 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.952+0000 7f21ee586700 1 --2- 172.21.15.3:0/684453315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21e809d6f0 0x7f21e81a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.952+0000 7f21ee586700 1 --2- 172.21.15.3:0/684453315 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21e809e000 0x7f21e81a6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:28.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.952+0000 7f21ee586700 1 -- 172.21.15.3:0/684453315 >> 172.21.15.3:0/684453315 conn(0x7f21e8066f50 msgr2=0x7f21e81baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:28.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.952+0000 7f21ee586700 1 -- 172.21.15.3:0/684453315 shutdown_connections 2023-10-03T10:17:28.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:28.952+0000 7f21ee586700 1 -- 172.21.15.3:0/684453315 wait complete. 2023-10-03T10:17:28.967 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:29.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.006+0000 7f488023a700 1 -- 172.21.15.3:0/3165505335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f48780747f0 con 0x7f4878130750 2023-10-03T10:17:29.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.007+0000 7f486e7fc700 1 -- 172.21.15.3:0/3165505335 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4870011800 con 0x7f4878130750 2023-10-03T10:17:29.007 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:29.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.009+0000 7f488023a700 1 -- 172.21.15.3:0/3165505335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4878130750 msgr2=0x7f48781364c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:29.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.009+0000 7f488023a700 1 --2- 172.21.15.3:0/3165505335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4878130750 0x7f48781364c0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f487804f490 tx=0x7f4870004ad0 comp rx=0 tx=0).stop 2023-10-03T10:17:29.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.009+0000 7f488023a700 1 -- 172.21.15.3:0/3165505335 shutdown_connections 2023-10-03T10:17:29.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.009+0000 7f488023a700 1 --2- 172.21.15.3:0/3165505335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4878130750 0x7f48781364c0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:29.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.009+0000 7f488023a700 1 --2- 172.21.15.3:0/3165505335 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f487809d6f0 0x7f4878135a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:29.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.009+0000 7f488023a700 1 --2- 172.21.15.3:0/3165505335 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f487809e080 0x7f4878135f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:29.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.009+0000 7f488023a700 1 -- 172.21.15.3:0/3165505335 >> 172.21.15.3:0/3165505335 conn(0x7f487809f470 msgr2=0x7f4878071450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:29.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.009+0000 7f488023a700 1 -- 172.21.15.3:0/3165505335 shutdown_connections 2023-10-03T10:17:29.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.009+0000 7f488023a700 1 -- 172.21.15.3:0/3165505335 wait complete. 2023-10-03T10:17:29.023 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:29.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.051+0000 7fba7b2bf700 1 -- 172.21.15.3:0/174875519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fba741aa3a0 con 0x7fba7409d6f0 2023-10-03T10:17:29.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.052+0000 7fba69ffb700 1 -- 172.21.15.3:0/174875519 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fba7001b3b0 con 0x7fba7409d6f0 2023-10-03T10:17:29.052 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:29.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.054+0000 7fba7b2bf700 1 -- 172.21.15.3:0/174875519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba7409d6f0 msgr2=0x7fba741a60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:29.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.054+0000 7fba7b2bf700 1 --2- 172.21.15.3:0/174875519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba7409d6f0 0x7fba741a60a0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fba7000ebe0 tx=0x7fba7000eef0 comp rx=0 tx=0).stop 2023-10-03T10:17:29.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.054+0000 7fba7b2bf700 1 -- 172.21.15.3:0/174875519 shutdown_connections 2023-10-03T10:17:29.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.054+0000 7fba7b2bf700 1 --2- 172.21.15.3:0/174875519 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fba7409e410 0x7fba741a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:29.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.054+0000 7fba7b2bf700 1 --2- 172.21.15.3:0/174875519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba7409d6f0 0x7fba741a60a0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:29.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.054+0000 7fba7b2bf700 1 --2- 172.21.15.3:0/174875519 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fba7409e000 0x7fba741a65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:29.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.054+0000 7fba7b2bf700 1 -- 172.21.15.3:0/174875519 >> 172.21.15.3:0/174875519 conn(0x7fba74066f50 msgr2=0x7fba741baef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:29.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.054+0000 7fba7b2bf700 1 -- 172.21.15.3:0/174875519 shutdown_connections 2023-10-03T10:17:29.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.054+0000 7fba7b2bf700 1 -- 172.21.15.3:0/174875519 wait complete. 2023-10-03T10:17:29.068 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:29.499 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:29.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.644+0000 7fc73e70c700 1 Processor -- start 2023-10-03T10:17:29.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.644+0000 7fc73e70c700 1 -- start start 2023-10-03T10:17:29.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.644+0000 7fc73e70c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc73809d6f0 0x7fc73809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:29.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.644+0000 7fc73e70c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc73809e000 0x7fc738130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:29.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.644+0000 7fc73e70c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc73809e410 0x7fc738130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:29.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.644+0000 7fc73e70c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc73809f520 con 0x7fc73809e410 2023-10-03T10:17:29.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.644+0000 7fc73e70c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73809f130 con 0x7fc73809e000 2023-10-03T10:17:29.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.644+0000 7fc73e70c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc73809e930 con 0x7fc73809d6f0 2023-10-03T10:17:29.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.645+0000 7fc7377fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc73809e000 0x7fc738130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.645+0000 7fc737fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc73809d6f0 0x7fc73809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.645+0000 7fc73cca9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc73809e410 0x7fc738130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.645+0000 7fc737fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc73809d6f0 0x7fc73809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:40030/0 (socket says 172.21.15.3:40030) 2023-10-03T10:17:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.645+0000 7fc7377fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc73809e000 0x7fc738130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36108/0 (socket says 172.21.15.3:36108) 2023-10-03T10:17:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.645+0000 7fc737fff700 1 -- 172.21.15.3:0/2836214882 learned_addr learned my addr 172.21.15.3:0/2836214882 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.645+0000 7fc73cca9700 1 -- 172.21.15.3:0/2836214882 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc73809d6f0 msgr2=0x7fc73809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.645+0000 7fc73cca9700 1 --2- 172.21.15.3:0/2836214882 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc73809d6f0 0x7fc73809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.645+0000 7fc73cca9700 1 -- 172.21.15.3:0/2836214882 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc73809e000 msgr2=0x7fc738130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.645+0000 7fc73cca9700 1 --2- 172.21.15.3:0/2836214882 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc73809e000 0x7fc738130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.645+0000 7fc73cca9700 1 -- 172.21.15.3:0/2836214882 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73809b2d0 con 0x7fc73809e410 2023-10-03T10:17:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.645+0000 7fc737fff700 1 --2- 172.21.15.3:0/2836214882 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc73809d6f0 0x7fc73809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.645+0000 7fc7377fe700 1 --2- 172.21.15.3:0/2836214882 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc73809e000 0x7fc738130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:29.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.645+0000 7fc73cca9700 1 --2- 172.21.15.3:0/2836214882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc73809e410 0x7fc738130ba0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fc730009d90 tx=0x7fc73000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a2676f51e2bb35 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:29.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.645+0000 7fc736ffd700 1 -- 172.21.15.3:0/2836214882 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73000e070 con 0x7fc73809e410 2023-10-03T10:17:29.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.645+0000 7fc736ffd700 1 -- 172.21.15.3:0/2836214882 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73000bb20 con 0x7fc73809e410 2023-10-03T10:17:29.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.645+0000 7fc736ffd700 1 -- 172.21.15.3:0/2836214882 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc730012430 con 0x7fc73809e410 2023-10-03T10:17:29.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.646+0000 7fc73e70c700 1 -- 172.21.15.3:0/2836214882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc73809e410 msgr2=0x7fc738130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.646+0000 7fc73e70c700 1 --2- 172.21.15.3:0/2836214882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc73809e410 0x7fc738130ba0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fc730009d90 tx=0x7fc73000b330 comp rx=0 tx=0).stop 2023-10-03T10:17:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.646+0000 7fc73e70c700 1 -- 172.21.15.3:0/2836214882 shutdown_connections 2023-10-03T10:17:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.646+0000 7fc73e70c700 1 --2- 172.21.15.3:0/2836214882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc73809e410 0x7fc738130ba0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.646+0000 7fc73e70c700 1 --2- 172.21.15.3:0/2836214882 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc73809d6f0 0x7fc73809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.646+0000 7fc73e70c700 1 --2- 172.21.15.3:0/2836214882 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc73809e000 0x7fc738130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.646+0000 7fc73e70c700 1 -- 172.21.15.3:0/2836214882 >> 172.21.15.3:0/2836214882 conn(0x7fc738066f50 msgr2=0x7fc73809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.646+0000 7fc73e70c700 1 -- 172.21.15.3:0/2836214882 shutdown_connections 2023-10-03T10:17:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.646+0000 7fc73e70c700 1 -- 172.21.15.3:0/2836214882 wait complete. 2023-10-03T10:17:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.647+0000 7fc73e70c700 1 Processor -- start 2023-10-03T10:17:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.647+0000 7fc73e70c700 1 -- start start 2023-10-03T10:17:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.648+0000 7fc73e70c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc73809d6f0 0x7fc738133190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.648+0000 7fc73e70c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc73809e000 0x7fc7381336d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.648+0000 7fc73e70c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc73809e410 0x7fc738133c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.648+0000 7fc73e70c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc738072480 con 0x7fc73809d6f0 2023-10-03T10:17:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.648+0000 7fc73e70c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc738135180 con 0x7fc73809e410 2023-10-03T10:17:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.648+0000 7fc73e70c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc73812d3b0 con 0x7fc73809e000 2023-10-03T10:17:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.648+0000 7fc737fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc73809d6f0 0x7fc738133190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.648+0000 7fc7377fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc73809e000 0x7fc7381336d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.648+0000 7fc737fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc73809d6f0 0x7fc738133190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40490/0 (socket says 172.21.15.3:40490) 2023-10-03T10:17:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.648+0000 7fc73cca9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc73809e410 0x7fc738133c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.648+0000 7fc737fff700 1 -- 172.21.15.3:0/2564796017 learned_addr learned my addr 172.21.15.3:0/2564796017 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.648+0000 7fc7377fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc73809e000 0x7fc7381336d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:40036/0 (socket says 172.21.15.3:40036) 2023-10-03T10:17:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.648+0000 7fc7377fe700 1 -- 172.21.15.3:0/2564796017 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc73809e410 msgr2=0x7fc738133c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.648+0000 7fc7377fe700 1 --2- 172.21.15.3:0/2564796017 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc73809e410 0x7fc738133c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.648+0000 7fc7377fe700 1 -- 172.21.15.3:0/2564796017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc73809d6f0 msgr2=0x7fc738133190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.648+0000 7fc7377fe700 1 --2- 172.21.15.3:0/2564796017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc73809d6f0 0x7fc738133190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.648+0000 7fc7377fe700 1 -- 172.21.15.3:0/2564796017 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc720009550 con 0x7fc73809e000 2023-10-03T10:17:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.648+0000 7fc737fff700 1 --2- 172.21.15.3:0/2564796017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc73809d6f0 0x7fc738133190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.648+0000 7fc7377fe700 1 --2- 172.21.15.3:0/2564796017 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc73809e000 0x7fc7381336d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc728010c90 tx=0x7fc72800b140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.649+0000 7fc734ff9700 1 -- 172.21.15.3:0/2564796017 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc728015070 con 0x7fc73809e000 2023-10-03T10:17:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.649+0000 7fc734ff9700 1 -- 172.21.15.3:0/2564796017 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72800b970 con 0x7fc73809e000 2023-10-03T10:17:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.649+0000 7fc734ff9700 1 -- 172.21.15.3:0/2564796017 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72800bd00 con 0x7fc73809e000 2023-10-03T10:17:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.649+0000 7fc73e70c700 1 -- 172.21.15.3:0/2564796017 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7280094a0 con 0x7fc73809e000 2023-10-03T10:17:29.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.649+0000 7fc73e70c700 1 -- 172.21.15.3:0/2564796017 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc730009a70 con 0x7fc73809e000 2023-10-03T10:17:29.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.649+0000 7fc734ff9700 1 -- 172.21.15.3:0/2564796017 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc728019410 con 0x7fc73809e000 2023-10-03T10:17:29.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.650+0000 7fc734ff9700 1 -- 172.21.15.3:0/2564796017 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc728042fe0 con 0x7fc73809e000 2023-10-03T10:17:29.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.650+0000 7fc73e70c700 1 -- 172.21.15.3:0/2564796017 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc738074ea0 con 0x7fc73809e000 2023-10-03T10:17:29.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.655+0000 7fc734ff9700 1 -- 172.21.15.3:0/2564796017 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc72801e070 con 0x7fc73809e000 2023-10-03T10:17:29.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.868+0000 7fc73e70c700 1 -- 172.21.15.3:0/2564796017 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc7381353c0 con 0x7fc73809e000 2023-10-03T10:17:29.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.868+0000 7fc734ff9700 1 -- 172.21.15.3:0/2564796017 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc72800be60 con 0x7fc73809e000 2023-10-03T10:17:29.869 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:29.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.871+0000 7fc73e70c700 1 -- 172.21.15.3:0/2564796017 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc73809e000 msgr2=0x7fc7381336d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:29.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.871+0000 7fc73e70c700 1 --2- 172.21.15.3:0/2564796017 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc73809e000 0x7fc7381336d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc728010c90 tx=0x7fc72800b140 comp rx=0 tx=0).stop 2023-10-03T10:17:29.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.871+0000 7fc73e70c700 1 -- 172.21.15.3:0/2564796017 shutdown_connections 2023-10-03T10:17:29.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.871+0000 7fc73e70c700 1 --2- 172.21.15.3:0/2564796017 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc73809e410 0x7fc738133c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:29.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.871+0000 7fc73e70c700 1 --2- 172.21.15.3:0/2564796017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc73809d6f0 0x7fc738133190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:29.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.871+0000 7fc73e70c700 1 --2- 172.21.15.3:0/2564796017 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc73809e000 0x7fc7381336d0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:29.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.871+0000 7fc73e70c700 1 -- 172.21.15.3:0/2564796017 >> 172.21.15.3:0/2564796017 conn(0x7fc738066f50 msgr2=0x7fc73806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:29.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.871+0000 7fc73e70c700 1 -- 172.21.15.3:0/2564796017 shutdown_connections 2023-10-03T10:17:29.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:29.871+0000 7fc73e70c700 1 -- 172.21.15.3:0/2564796017 wait complete. 2023-10-03T10:17:29.885 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:29.968 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:30.024 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:30.068 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:30.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.113+0000 7fe328c5e700 1 Processor -- start 2023-10-03T10:17:30.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.113+0000 7fe328c5e700 1 -- start start 2023-10-03T10:17:30.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.113+0000 7fe328c5e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe32409d1f0 0x7fe32409d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:30.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.113+0000 7fe328c5e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe32409e420 0x7fe32409db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:30.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.113+0000 7fe328c5e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe32409cdc0 con 0x7fe324133580 2023-10-03T10:17:30.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.113+0000 7fe328c5e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe32409e040 con 0x7fe32409d1f0 2023-10-03T10:17:30.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.113+0000 7fe328c5e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe324099e50 con 0x7fe32409e420 2023-10-03T10:17:30.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.114+0000 7fe321d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe32409e420 0x7fe32409db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:30.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.114+0000 7fe32259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe32409d1f0 0x7fe32409d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:30.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.114+0000 7fe321d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe32409e420 0x7fe32409db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:40040/0 (socket says 172.21.15.3:40040) 2023-10-03T10:17:30.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.114+0000 7fe32259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe32409d1f0 0x7fe32409d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40500/0 (socket says 172.21.15.3:40500) 2023-10-03T10:17:30.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.114+0000 7fe321d9b700 1 -- 172.21.15.3:0/1338825995 learned_addr learned my addr 172.21.15.3:0/1338825995 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:30.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.114+0000 7fe321d9b700 1 -- 172.21.15.3:0/1338825995 >> v2:172.21.15.3:3300/0 conn(0x7fe32409d1f0 msgr2=0x7fe32409d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:30.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.114+0000 7fe321d9b700 1 --2- 172.21.15.3:0/1338825995 >> v2:172.21.15.3:3300/0 conn(0x7fe32409d1f0 0x7fe32409d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:30.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.114+0000 7fe321d9b700 1 -- 172.21.15.3:0/1338825995 >> v1:172.21.15.3:6789/0 conn(0x7fe324133580 legacy=0x7fe3241339c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:30.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.114+0000 7fe321d9b700 1 -- 172.21.15.3:0/1338825995 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe32409e250 con 0x7fe32409e420 2023-10-03T10:17:30.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.114+0000 7fe321d9b700 1 --2- 172.21.15.3:0/1338825995 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe32409e420 0x7fe32409db00 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe3180098f0 tx=0x7fe318009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d8ad04b12b94578 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:30.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.114+0000 7fe32159a700 1 -- 172.21.15.3:0/1338825995 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe318011830 con 0x7fe32409e420 2023-10-03T10:17:30.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.114+0000 7fe32159a700 1 -- 172.21.15.3:0/1338825995 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe318011990 con 0x7fe32409e420 2023-10-03T10:17:30.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.115+0000 7fe32159a700 1 -- 172.21.15.3:0/1338825995 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe318011d20 con 0x7fe32409e420 2023-10-03T10:17:30.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.115+0000 7fe328c5e700 1 -- 172.21.15.3:0/1338825995 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe32409e420 msgr2=0x7fe32409db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:30.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.115+0000 7fe328c5e700 1 --2- 172.21.15.3:0/1338825995 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe32409e420 0x7fe32409db00 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe3180098f0 tx=0x7fe318009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:30.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.115+0000 7fe328c5e700 1 -- 172.21.15.3:0/1338825995 shutdown_connections 2023-10-03T10:17:30.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.115+0000 7fe328c5e700 1 --2- 172.21.15.3:0/1338825995 >> v2:172.21.15.3:3300/0 conn(0x7fe32409d1f0 0x7fe32409d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:30.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.115+0000 7fe328c5e700 1 --2- 172.21.15.3:0/1338825995 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe32409e420 0x7fe32409db00 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:30.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.115+0000 7fe328c5e700 1 -- 172.21.15.3:0/1338825995 >> 172.21.15.3:0/1338825995 conn(0x7fe3240a2360 msgr2=0x7fe3240a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:30.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.115+0000 7fe328c5e700 1 -- 172.21.15.3:0/1338825995 shutdown_connections 2023-10-03T10:17:30.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.115+0000 7fe328c5e700 1 -- 172.21.15.3:0/1338825995 wait complete. 2023-10-03T10:17:30.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.117+0000 7fe328c5e700 1 Processor -- start 2023-10-03T10:17:30.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.117+0000 7fe328c5e700 1 -- start start 2023-10-03T10:17:30.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.117+0000 7fe328c5e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe32409d1f0 0x7fe324129560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:30.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.117+0000 7fe328c5e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe32409e420 0x7fe324129aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:30.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.117+0000 7fe328c5e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe324133580 0x7fe324129fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:30.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.117+0000 7fe328c5e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32412b1c0 con 0x7fe32409e420 2023-10-03T10:17:30.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.117+0000 7fe328c5e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3240a0c80 con 0x7fe324133580 2023-10-03T10:17:30.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.117+0000 7fe328c5e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe32409c160 con 0x7fe32409d1f0 2023-10-03T10:17:30.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.117+0000 7fe322d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe324133580 0x7fe324129fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:30.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.117+0000 7fe32259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe32409d1f0 0x7fe324129560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:30.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.117+0000 7fe32259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe32409d1f0 0x7fe324129560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:40052/0 (socket says 172.21.15.3:40052) 2023-10-03T10:17:30.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.117+0000 7fe322d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe324133580 0x7fe324129fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36140/0 (socket says 172.21.15.3:36140) 2023-10-03T10:17:30.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.117+0000 7fe32259c700 1 -- 172.21.15.3:0/1678106028 learned_addr learned my addr 172.21.15.3:0/1678106028 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:30.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.118+0000 7fe321d9b700 1 --2- 172.21.15.3:0/1678106028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe32409e420 0x7fe324129aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:30.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.118+0000 7fe322d9d700 1 -- 172.21.15.3:0/1678106028 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe32409d1f0 msgr2=0x7fe324129560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:30.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.118+0000 7fe322d9d700 1 --2- 172.21.15.3:0/1678106028 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe32409d1f0 0x7fe324129560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:30.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.118+0000 7fe322d9d700 1 -- 172.21.15.3:0/1678106028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe32409e420 msgr2=0x7fe324129aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:30.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.118+0000 7fe322d9d700 1 --2- 172.21.15.3:0/1678106028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe32409e420 0x7fe324129aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:30.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.118+0000 7fe322d9d700 1 -- 172.21.15.3:0/1678106028 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe318009550 con 0x7fe324133580 2023-10-03T10:17:30.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.118+0000 7fe322d9d700 1 --2- 172.21.15.3:0/1678106028 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe324133580 0x7fe324129fe0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00cb90 tx=0x7fe31c00cea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:30.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.118+0000 7fe312ffd700 1 -- 172.21.15.3:0/1678106028 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c003c00 con 0x7fe324133580 2023-10-03T10:17:30.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.118+0000 7fe312ffd700 1 -- 172.21.15.3:0/1678106028 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe31c003d60 con 0x7fe324133580 2023-10-03T10:17:30.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.118+0000 7fe312ffd700 1 -- 172.21.15.3:0/1678106028 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c00aa80 con 0x7fe324133580 2023-10-03T10:17:30.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.118+0000 7fe328c5e700 1 -- 172.21.15.3:0/1678106028 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe32412c9d0 con 0x7fe324133580 2023-10-03T10:17:30.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.119+0000 7fe328c5e700 1 -- 172.21.15.3:0/1678106028 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe32412cf70 con 0x7fe324133580 2023-10-03T10:17:30.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.119+0000 7fe312ffd700 1 -- 172.21.15.3:0/1678106028 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe31c00abe0 con 0x7fe324133580 2023-10-03T10:17:30.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.120+0000 7fe312ffd700 1 -- 172.21.15.3:0/1678106028 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe31c03f570 con 0x7fe324133580 2023-10-03T10:17:30.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.120+0000 7fe328c5e700 1 -- 172.21.15.3:0/1678106028 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe32412cbc0 con 0x7fe324133580 2023-10-03T10:17:30.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.125+0000 7fe312ffd700 1 -- 172.21.15.3:0/1678106028 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe31c0198f0 con 0x7fe324133580 2023-10-03T10:17:30.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.167+0000 7f02ee065700 1 Processor -- start 2023-10-03T10:17:30.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.167+0000 7f02ee065700 1 -- start start 2023-10-03T10:17:30.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.167+0000 7f02ee065700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f02e80a0a30 0x7f02e80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:30.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.167+0000 7f02ee065700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f02e80a1340 0x7f02e8133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:30.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.167+0000 7f02ee065700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e809edd0 con 0x7f02e8133fa0 2023-10-03T10:17:30.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.167+0000 7f02ee065700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e809e1f0 con 0x7f02e80a1340 2023-10-03T10:17:30.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.168+0000 7f02ee065700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f02e809dc50 con 0x7f02e80a0a30 2023-10-03T10:17:30.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.168+0000 7f02e77fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f02e80a0a30 0x7f02e80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:30.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.168+0000 7f02e77fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f02e80a0a30 0x7f02e80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:40064/0 (socket says 172.21.15.3:40064) 2023-10-03T10:17:30.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.168+0000 7f02e77fe700 1 -- 172.21.15.3:0/1183470518 learned_addr learned my addr 172.21.15.3:0/1183470518 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:30.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.168+0000 7f02e6ffd700 1 --2- 172.21.15.3:0/1183470518 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f02e80a1340 0x7f02e8133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:30.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.168+0000 7f02e67fc700 1 -- 172.21.15.3:0/1183470518 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3377583002 0 0) 0x7f02e809edd0 con 0x7f02e8133fa0 2023-10-03T10:17:30.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.168+0000 7f02e6ffd700 1 -- 172.21.15.3:0/1183470518 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f02e80a0a30 msgr2=0x7f02e80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:30.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.168+0000 7f02e6ffd700 1 --2- 172.21.15.3:0/1183470518 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f02e80a0a30 0x7f02e80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:30.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.168+0000 7f02e6ffd700 1 -- 172.21.15.3:0/1183470518 >> v1:172.21.15.3:6789/0 conn(0x7f02e8133fa0 legacy=0x7f02e813ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:30.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.168+0000 7f02e6ffd700 1 -- 172.21.15.3:0/1183470518 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e80a1fe0 con 0x7f02e80a1340 2023-10-03T10:17:30.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.168+0000 7f02e77fe700 1 --2- 172.21.15.3:0/1183470518 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f02e80a0a30 0x7f02e80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:30.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.168+0000 7f02e6ffd700 1 --2- 172.21.15.3:0/1183470518 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f02e80a1340 0x7f02e8133a60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f02dc009870 tx=0x7f02dc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b6e02db39448a55 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:30.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.169+0000 7f02e67fc700 1 -- 172.21.15.3:0/1183470518 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc011830 con 0x7f02e80a1340 2023-10-03T10:17:30.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.169+0000 7f02e67fc700 1 -- 172.21.15.3:0/1183470518 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02dc011990 con 0x7f02e80a1340 2023-10-03T10:17:30.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.169+0000 7f02e67fc700 1 -- 172.21.15.3:0/1183470518 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc011d20 con 0x7f02e80a1340 2023-10-03T10:17:30.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.169+0000 7f02ee065700 1 -- 172.21.15.3:0/1183470518 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f02e80a1340 msgr2=0x7f02e8133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:30.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.169+0000 7f02ee065700 1 --2- 172.21.15.3:0/1183470518 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f02e80a1340 0x7f02e8133a60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f02dc009870 tx=0x7f02dc009b80 comp rx=0 tx=0).stop 2023-10-03T10:17:30.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.169+0000 7f02ee065700 1 -- 172.21.15.3:0/1183470518 shutdown_connections 2023-10-03T10:17:30.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.169+0000 7f02ee065700 1 --2- 172.21.15.3:0/1183470518 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f02e80a0a30 0x7f02e80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:30.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.169+0000 7f02ee065700 1 --2- 172.21.15.3:0/1183470518 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f02e80a1340 0x7f02e8133a60 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:30.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.169+0000 7f02ee065700 1 -- 172.21.15.3:0/1183470518 >> 172.21.15.3:0/1183470518 conn(0x7f02e80a6550 msgr2=0x7f02e80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:30.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.169+0000 7f02ee065700 1 -- 172.21.15.3:0/1183470518 shutdown_connections 2023-10-03T10:17:30.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.169+0000 7f02ee065700 1 -- 172.21.15.3:0/1183470518 wait complete. 2023-10-03T10:17:30.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.170+0000 7f02ee065700 1 Processor -- start 2023-10-03T10:17:30.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.171+0000 7f02ee065700 1 -- start start 2023-10-03T10:17:30.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.171+0000 7f02ee065700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f02e80a0a30 0x7f02e81c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:30.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.171+0000 7f02ee065700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f02e80a1340 0x7f02e81c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:30.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.171+0000 7f02ee065700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02e8133fa0 0x7f02e81c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:30.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.171+0000 7f02ee065700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e80a0220 con 0x7f02e8133fa0 2023-10-03T10:17:30.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.171+0000 7f02ee065700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e8125170 con 0x7f02e80a0a30 2023-10-03T10:17:30.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.171+0000 7f02ee065700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f02e8124ab0 con 0x7f02e80a1340 2023-10-03T10:17:30.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.171+0000 7f02e77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f02e80a0a30 0x7f02e81c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:30.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.171+0000 7f02e6ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f02e80a1340 0x7f02e81c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:30.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.171+0000 7f02e77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f02e80a0a30 0x7f02e81c8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36150/0 (socket says 172.21.15.3:36150) 2023-10-03T10:17:30.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.171+0000 7f02e77fe700 1 -- 172.21.15.3:0/227496930 learned_addr learned my addr 172.21.15.3:0/227496930 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:30.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.171+0000 7f02e7fff700 1 --2- 172.21.15.3:0/227496930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02e8133fa0 0x7f02e81c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:30.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.172+0000 7f02e77fe700 1 -- 172.21.15.3:0/227496930 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f02e80a1340 msgr2=0x7f02e81c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:30.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.172+0000 7f02e77fe700 1 --2- 172.21.15.3:0/227496930 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f02e80a1340 0x7f02e81c87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:30.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.172+0000 7f02e77fe700 1 -- 172.21.15.3:0/227496930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02e8133fa0 msgr2=0x7f02e81c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:30.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.172+0000 7f02e77fe700 1 --2- 172.21.15.3:0/227496930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02e8133fa0 0x7f02e81c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:30.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.172+0000 7f02e77fe700 1 -- 172.21.15.3:0/227496930 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d8009550 con 0x7f02e80a0a30 2023-10-03T10:17:30.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.172+0000 7f02e77fe700 1 --2- 172.21.15.3:0/227496930 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f02e80a0a30 0x7f02e81c8280 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f02d800ebe0 tx=0x7f02d800eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:30.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.172+0000 7f02cffff700 1 -- 172.21.15.3:0/227496930 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d800c850 con 0x7f02e80a0a30 2023-10-03T10:17:30.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.172+0000 7f02cffff700 1 -- 172.21.15.3:0/227496930 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d800c9b0 con 0x7f02e80a0a30 2023-10-03T10:17:30.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.172+0000 7f02ee065700 1 -- 172.21.15.3:0/227496930 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02dc0094a0 con 0x7f02e80a0a30 2023-10-03T10:17:30.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.172+0000 7f02cffff700 1 -- 172.21.15.3:0/227496930 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d800cd40 con 0x7f02e80a0a30 2023-10-03T10:17:30.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.172+0000 7f02ee065700 1 -- 172.21.15.3:0/227496930 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e80a25e0 con 0x7f02e80a0a30 2023-10-03T10:17:30.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.173+0000 7f02cffff700 1 -- 172.21.15.3:0/227496930 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f02d8013470 con 0x7f02e80a0a30 2023-10-03T10:17:30.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.173+0000 7f02ee065700 1 -- 172.21.15.3:0/227496930 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02e8125dd0 con 0x7f02e80a0a30 2023-10-03T10:17:30.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.173+0000 7f02cffff700 1 -- 172.21.15.3:0/227496930 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f02d803f8a0 con 0x7f02e80a0a30 2023-10-03T10:17:30.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.182+0000 7f02cffff700 1 -- 172.21.15.3:0/227496930 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f02d80232f0 con 0x7f02e80a0a30 2023-10-03T10:17:30.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.214+0000 7f8b38908700 1 Processor -- start 2023-10-03T10:17:30.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.214+0000 7f8b38908700 1 -- start start 2023-10-03T10:17:30.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.214+0000 7f8b38908700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8b340a0a30 0x7f8b340a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:30.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.214+0000 7f8b38908700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b3409d630 0x7f8b340a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:30.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.214+0000 7f8b38908700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b34133930 0x7f8b34133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:30.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.214+0000 7f8b38908700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b340a2a70 con 0x7f8b340a0a30 2023-10-03T10:17:30.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.214+0000 7f8b38908700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b340a01d0 con 0x7f8b34133930 2023-10-03T10:17:30.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.214+0000 7f8b38908700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3409e2b0 con 0x7f8b3409d630 2023-10-03T10:17:30.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.215+0000 7f8b3259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8b340a0a30 0x7f8b340a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:30.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.215+0000 7f8b3259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8b340a0a30 0x7f8b340a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40528/0 (socket says 172.21.15.3:40528) 2023-10-03T10:17:30.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.215+0000 7f8b3259c700 1 -- 172.21.15.3:0/202337259 learned_addr learned my addr 172.21.15.3:0/202337259 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:30.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.215+0000 7f8b32d9d700 1 --2- 172.21.15.3:0/202337259 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b34133930 0x7f8b34133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:30.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.215+0000 7f8b31d9b700 1 --2- 172.21.15.3:0/202337259 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b3409d630 0x7f8b340a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:30.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.215+0000 7f8b32d9d700 1 -- 172.21.15.3:0/202337259 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b3409d630 msgr2=0x7f8b340a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:30.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.215+0000 7f8b32d9d700 1 --2- 172.21.15.3:0/202337259 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b3409d630 0x7f8b340a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:30.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.215+0000 7f8b32d9d700 1 -- 172.21.15.3:0/202337259 >> v2:172.21.15.3:3300/0 conn(0x7f8b340a0a30 msgr2=0x7f8b340a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:30.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.215+0000 7f8b32d9d700 1 --2- 172.21.15.3:0/202337259 >> v2:172.21.15.3:3300/0 conn(0x7f8b340a0a30 0x7f8b340a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:30.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.215+0000 7f8b32d9d700 1 -- 172.21.15.3:0/202337259 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3409dc50 con 0x7f8b34133930 2023-10-03T10:17:30.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.215+0000 7f8b3259c700 1 --2- 172.21.15.3:0/202337259 >> v2:172.21.15.3:3300/0 conn(0x7f8b340a0a30 0x7f8b340a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:30.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.215+0000 7f8b31d9b700 1 --2- 172.21.15.3:0/202337259 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b3409d630 0x7f8b340a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:17:30.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.215+0000 7f8b32d9d700 1 --2- 172.21.15.3:0/202337259 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b34133930 0x7f8b34133db0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c009d90 tx=0x7f8b2c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aa0ee4cec78a300 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:30.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.215+0000 7f8b3159a700 1 -- 172.21.15.3:0/202337259 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c00e070 con 0x7f8b34133930 2023-10-03T10:17:30.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.216+0000 7f8b3159a700 1 -- 172.21.15.3:0/202337259 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2c00bb20 con 0x7f8b34133930 2023-10-03T10:17:30.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.216+0000 7f8b3159a700 1 -- 172.21.15.3:0/202337259 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c012430 con 0x7f8b34133930 2023-10-03T10:17:30.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.216+0000 7f8b38908700 1 -- 172.21.15.3:0/202337259 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b34133930 msgr2=0x7f8b34133db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:30.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.216+0000 7f8b38908700 1 --2- 172.21.15.3:0/202337259 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b34133930 0x7f8b34133db0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c009d90 tx=0x7f8b2c00b330 comp rx=0 tx=0).stop 2023-10-03T10:17:30.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.216+0000 7f8b38908700 1 -- 172.21.15.3:0/202337259 shutdown_connections 2023-10-03T10:17:30.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.216+0000 7f8b38908700 1 --2- 172.21.15.3:0/202337259 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b34133930 0x7f8b34133db0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:30.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.216+0000 7f8b38908700 1 --2- 172.21.15.3:0/202337259 >> v2:172.21.15.3:3300/0 conn(0x7f8b340a0a30 0x7f8b340a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:30.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.216+0000 7f8b38908700 1 --2- 172.21.15.3:0/202337259 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b3409d630 0x7f8b340a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.216+0000 7f8b38908700 1 -- 172.21.15.3:0/202337259 >> 172.21.15.3:0/202337259 conn(0x7f8b340a3050 msgr2=0x7f8b340a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.216+0000 7f8b38908700 1 -- 172.21.15.3:0/202337259 shutdown_connections 2023-10-03T10:17:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.216+0000 7f8b38908700 1 -- 172.21.15.3:0/202337259 wait complete. 2023-10-03T10:17:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.218+0000 7f8b38908700 1 Processor -- start 2023-10-03T10:17:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.218+0000 7f8b38908700 1 -- start start 2023-10-03T10:17:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.218+0000 7f8b38908700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b3409d630 0x7f8b3411f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.218+0000 7f8b38908700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b340a0a30 0x7f8b3411f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.218+0000 7f8b38908700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b34133930 0x7f8b341c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.218+0000 7f8b38908700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3411ede0 con 0x7f8b34133930 2023-10-03T10:17:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.218+0000 7f8b38908700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b34133290 con 0x7f8b340a0a30 2023-10-03T10:17:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.218+0000 7f8b38908700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b34122140 con 0x7f8b3409d630 2023-10-03T10:17:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.218+0000 7f8b32d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b34133930 0x7f8b341c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:30.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.218+0000 7f8b31d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b340a0a30 0x7f8b3411f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:30.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.218+0000 7f8b32d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b34133930 0x7f8b341c8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40540/0 (socket says 172.21.15.3:40540) 2023-10-03T10:17:30.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.218+0000 7f8b32d9d700 1 -- 172.21.15.3:0/1563965 learned_addr learned my addr 172.21.15.3:0/1563965 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:30.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.218+0000 7f8b3259c700 1 --2- 172.21.15.3:0/1563965 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b3409d630 0x7f8b3411f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:30.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.219+0000 7f8b31d9b700 1 -- 172.21.15.3:0/1563965 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b3409d630 msgr2=0x7f8b3411f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:30.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.219+0000 7f8b31d9b700 1 --2- 172.21.15.3:0/1563965 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b3409d630 0x7f8b3411f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:30.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.219+0000 7f8b31d9b700 1 -- 172.21.15.3:0/1563965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b34133930 msgr2=0x7f8b341c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:30.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.219+0000 7f8b31d9b700 1 --2- 172.21.15.3:0/1563965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b34133930 0x7f8b341c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:30.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.219+0000 7f8b31d9b700 1 -- 172.21.15.3:0/1563965 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b24009550 con 0x7f8b340a0a30 2023-10-03T10:17:30.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.219+0000 7f8b3259c700 1 --2- 172.21.15.3:0/1563965 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b3409d630 0x7f8b3411f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:17:30.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.219+0000 7f8b32d9d700 1 --2- 172.21.15.3:0/1563965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b34133930 0x7f8b341c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:30.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.219+0000 7f8b31d9b700 1 --2- 172.21.15.3:0/1563965 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b340a0a30 0x7f8b3411f6d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800c820 tx=0x7f8b2800cb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:30.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.219+0000 7f8b22ffd700 1 -- 172.21.15.3:0/1563965 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b28011800 con 0x7f8b340a0a30 2023-10-03T10:17:30.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.219+0000 7f8b22ffd700 1 -- 172.21.15.3:0/1563965 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b28011960 con 0x7f8b340a0a30 2023-10-03T10:17:30.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.219+0000 7f8b22ffd700 1 -- 172.21.15.3:0/1563965 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b28011cd0 con 0x7f8b340a0a30 2023-10-03T10:17:30.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.219+0000 7f8b38908700 1 -- 172.21.15.3:0/1563965 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b2c009a70 con 0x7f8b340a0a30 2023-10-03T10:17:30.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.219+0000 7f8b38908700 1 -- 172.21.15.3:0/1563965 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b34120a90 con 0x7f8b340a0a30 2023-10-03T10:17:30.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.220+0000 7f8b22ffd700 1 -- 172.21.15.3:0/1563965 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8b28022350 con 0x7f8b340a0a30 2023-10-03T10:17:30.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.220+0000 7f8b22ffd700 1 -- 172.21.15.3:0/1563965 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8b2803f660 con 0x7f8b340a0a30 2023-10-03T10:17:30.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.221+0000 7f8b38908700 1 -- 172.21.15.3:0/1563965 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b340747f0 con 0x7f8b340a0a30 2023-10-03T10:17:30.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.230+0000 7f8b22ffd700 1 -- 172.21.15.3:0/1563965 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8b28022600 con 0x7f8b340a0a30 2023-10-03T10:17:30.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.341+0000 7fe328c5e700 1 -- 172.21.15.3:0/1678106028 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe324074ea0 con 0x7fe324133580 2023-10-03T10:17:30.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.342+0000 7fe312ffd700 1 -- 172.21.15.3:0/1678106028 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe31c019300 con 0x7fe324133580 2023-10-03T10:17:30.342 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:30.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.344+0000 7fe328c5e700 1 -- 172.21.15.3:0/1678106028 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe324133580 msgr2=0x7fe324129fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:30.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.344+0000 7fe328c5e700 1 --2- 172.21.15.3:0/1678106028 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe324133580 0x7fe324129fe0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00cb90 tx=0x7fe31c00cea0 comp rx=0 tx=0).stop 2023-10-03T10:17:30.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.344+0000 7fe328c5e700 1 -- 172.21.15.3:0/1678106028 shutdown_connections 2023-10-03T10:17:30.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.344+0000 7fe328c5e700 1 --2- 172.21.15.3:0/1678106028 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe324133580 0x7fe324129fe0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:30.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.344+0000 7fe328c5e700 1 --2- 172.21.15.3:0/1678106028 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe32409d1f0 0x7fe324129560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:30.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.344+0000 7fe328c5e700 1 --2- 172.21.15.3:0/1678106028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe32409e420 0x7fe324129aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:30.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.344+0000 7fe328c5e700 1 -- 172.21.15.3:0/1678106028 >> 172.21.15.3:0/1678106028 conn(0x7fe3240a2360 msgr2=0x7fe32409ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:30.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.344+0000 7fe328c5e700 1 -- 172.21.15.3:0/1678106028 shutdown_connections 2023-10-03T10:17:30.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.344+0000 7fe328c5e700 1 -- 172.21.15.3:0/1678106028 wait complete. 2023-10-03T10:17:30.358 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:30.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.395+0000 7f02ee065700 1 -- 172.21.15.3:0/227496930 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f02e8074ea0 con 0x7f02e80a0a30 2023-10-03T10:17:30.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.395+0000 7f02cffff700 1 -- 172.21.15.3:0/227496930 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f02d8017070 con 0x7f02e80a0a30 2023-10-03T10:17:30.395 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:30.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.397+0000 7f02ee065700 1 -- 172.21.15.3:0/227496930 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f02e80a0a30 msgr2=0x7f02e81c8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:30.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.397+0000 7f02ee065700 1 --2- 172.21.15.3:0/227496930 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f02e80a0a30 0x7f02e81c8280 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f02d800ebe0 tx=0x7f02d800eef0 comp rx=0 tx=0).stop 2023-10-03T10:17:30.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.397+0000 7f02ee065700 1 -- 172.21.15.3:0/227496930 shutdown_connections 2023-10-03T10:17:30.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.397+0000 7f02ee065700 1 --2- 172.21.15.3:0/227496930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02e8133fa0 0x7f02e81c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:30.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.397+0000 7f02ee065700 1 --2- 172.21.15.3:0/227496930 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f02e80a0a30 0x7f02e81c8280 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:30.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.397+0000 7f02ee065700 1 --2- 172.21.15.3:0/227496930 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f02e80a1340 0x7f02e81c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:30.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.397+0000 7f02ee065700 1 -- 172.21.15.3:0/227496930 >> 172.21.15.3:0/227496930 conn(0x7f02e80a6550 msgr2=0x7f02e809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:30.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.398+0000 7f02ee065700 1 -- 172.21.15.3:0/227496930 shutdown_connections 2023-10-03T10:17:30.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.398+0000 7f02ee065700 1 -- 172.21.15.3:0/227496930 wait complete. 2023-10-03T10:17:30.412 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:30.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.440+0000 7f8b38908700 1 -- 172.21.15.3:0/1563965 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8b34121de0 con 0x7f8b340a0a30 2023-10-03T10:17:30.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.440+0000 7f8b22ffd700 1 -- 172.21.15.3:0/1563965 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8b34121de0 con 0x7f8b340a0a30 2023-10-03T10:17:30.441 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:30.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.443+0000 7f8b38908700 1 -- 172.21.15.3:0/1563965 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b340a0a30 msgr2=0x7f8b3411f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:30.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.443+0000 7f8b38908700 1 --2- 172.21.15.3:0/1563965 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b340a0a30 0x7f8b3411f6d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800c820 tx=0x7f8b2800cb30 comp rx=0 tx=0).stop 2023-10-03T10:17:30.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.443+0000 7f8b38908700 1 -- 172.21.15.3:0/1563965 shutdown_connections 2023-10-03T10:17:30.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.443+0000 7f8b38908700 1 --2- 172.21.15.3:0/1563965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b34133930 0x7f8b341c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:30.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.443+0000 7f8b38908700 1 --2- 172.21.15.3:0/1563965 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8b3409d630 0x7f8b3411f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:30.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.443+0000 7f8b38908700 1 --2- 172.21.15.3:0/1563965 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8b340a0a30 0x7f8b3411f6d0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:30.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.443+0000 7f8b38908700 1 -- 172.21.15.3:0/1563965 >> 172.21.15.3:0/1563965 conn(0x7f8b340a3050 msgr2=0x7f8b341bbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:30.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.443+0000 7f8b38908700 1 -- 172.21.15.3:0/1563965 shutdown_connections 2023-10-03T10:17:30.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:30.443+0000 7f8b38908700 1 -- 172.21.15.3:0/1563965 wait complete. 2023-10-03T10:17:30.456 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:30.886 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:31.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.032+0000 7f51e4788700 1 Processor -- start 2023-10-03T10:17:31.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.032+0000 7f51e4788700 1 -- start start 2023-10-03T10:17:31.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.033+0000 7f51e4788700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f51dc133420 0x7f51dc1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.033+0000 7f51e4788700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f51dc133e30 0x7f51dc12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.033+0000 7f51e4788700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f51dc134260 0x7f51dc1304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.033+0000 7f51e4788700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51dc06b990 con 0x7f51dc133e30 2023-10-03T10:17:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.033+0000 7f51e4788700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc0a0590 con 0x7f51dc133420 2023-10-03T10:17:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.033+0000 7f51e4788700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc09fb80 con 0x7f51dc134260 2023-10-03T10:17:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.033+0000 7f51e2524700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f51dc133420 0x7f51dc1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.033+0000 7f51e1d23700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f51dc133e30 0x7f51dc12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.033+0000 7f51e2d25700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f51dc134260 0x7f51dc1304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.033+0000 7f51e1d23700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f51dc133e30 0x7f51dc12ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40554/0 (socket says 172.21.15.3:40554) 2023-10-03T10:17:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.033+0000 7f51e2524700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f51dc133420 0x7f51dc1337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36178/0 (socket says 172.21.15.3:36178) 2023-10-03T10:17:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.033+0000 7f51e1d23700 1 -- 172.21.15.3:0/3909895976 learned_addr learned my addr 172.21.15.3:0/3909895976 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.033+0000 7f51e2d25700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f51dc134260 0x7f51dc1304d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:40096/0 (socket says 172.21.15.3:40096) 2023-10-03T10:17:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.033+0000 7f51e2524700 1 -- 172.21.15.3:0/3909895976 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f51dc134260 msgr2=0x7f51dc1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:31.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.033+0000 7f51e2524700 1 --2- 172.21.15.3:0/3909895976 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f51dc134260 0x7f51dc1304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.033+0000 7f51e2524700 1 -- 172.21.15.3:0/3909895976 >> v2:172.21.15.3:3300/0 conn(0x7f51dc133e30 msgr2=0x7f51dc12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:31.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.033+0000 7f51e2524700 1 --2- 172.21.15.3:0/3909895976 >> v2:172.21.15.3:3300/0 conn(0x7f51dc133e30 0x7f51dc12ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.033+0000 7f51e2524700 1 -- 172.21.15.3:0/3909895976 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc0a0030 con 0x7f51dc133420 2023-10-03T10:17:31.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.033+0000 7f51e1d23700 1 --2- 172.21.15.3:0/3909895976 >> v2:172.21.15.3:3300/0 conn(0x7f51dc133e30 0x7f51dc12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:31.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.033+0000 7f51e2d25700 1 --2- 172.21.15.3:0/3909895976 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f51dc134260 0x7f51dc1304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:31.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.033+0000 7f51e2524700 1 --2- 172.21.15.3:0/3909895976 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f51dc133420 0x7f51dc1337f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009840 tx=0x7f51d8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6926edefd10409d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:31.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.034+0000 7f51e1522700 1 -- 172.21.15.3:0/3909895976 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8011830 con 0x7f51dc133420 2023-10-03T10:17:31.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.034+0000 7f51e1522700 1 -- 172.21.15.3:0/3909895976 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d8011990 con 0x7f51dc133420 2023-10-03T10:17:31.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.034+0000 7f51e1522700 1 -- 172.21.15.3:0/3909895976 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8011d20 con 0x7f51dc133420 2023-10-03T10:17:31.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.034+0000 7f51e4788700 1 -- 172.21.15.3:0/3909895976 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f51dc133420 msgr2=0x7f51dc1337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:31.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.034+0000 7f51e4788700 1 --2- 172.21.15.3:0/3909895976 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f51dc133420 0x7f51dc1337f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009840 tx=0x7f51d8009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:31.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.034+0000 7f51e4788700 1 -- 172.21.15.3:0/3909895976 shutdown_connections 2023-10-03T10:17:31.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.034+0000 7f51e4788700 1 --2- 172.21.15.3:0/3909895976 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f51dc134260 0x7f51dc1304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.034+0000 7f51e4788700 1 --2- 172.21.15.3:0/3909895976 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f51dc133420 0x7f51dc1337f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.034+0000 7f51e4788700 1 --2- 172.21.15.3:0/3909895976 >> v2:172.21.15.3:3300/0 conn(0x7f51dc133e30 0x7f51dc12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.034+0000 7f51e4788700 1 -- 172.21.15.3:0/3909895976 >> 172.21.15.3:0/3909895976 conn(0x7f51dc0a4210 msgr2=0x7f51dc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:31.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.034+0000 7f51e4788700 1 -- 172.21.15.3:0/3909895976 shutdown_connections 2023-10-03T10:17:31.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.034+0000 7f51e4788700 1 -- 172.21.15.3:0/3909895976 wait complete. 2023-10-03T10:17:31.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.035+0000 7f51e4788700 1 Processor -- start 2023-10-03T10:17:31.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.035+0000 7f51e4788700 1 -- start start 2023-10-03T10:17:31.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.036+0000 7f51e4788700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f51dc133420 0x7f51dc09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:31.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.036+0000 7f51e4788700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51dc133e30 0x7f51dc09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:31.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.036+0000 7f51e4788700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f51dc134260 0x7f51dc09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:31.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.036+0000 7f51e4788700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc12da80 con 0x7f51dc133e30 2023-10-03T10:17:31.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.036+0000 7f51e4788700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc12dc70 con 0x7f51dc134260 2023-10-03T10:17:31.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.036+0000 7f51e4788700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc11fc30 con 0x7f51dc133420 2023-10-03T10:17:31.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.036+0000 7f51e2524700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f51dc133420 0x7f51dc09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:31.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.036+0000 7f51e2d25700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f51dc134260 0x7f51dc09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:31.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.036+0000 7f51e2524700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f51dc133420 0x7f51dc09c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:40108/0 (socket says 172.21.15.3:40108) 2023-10-03T10:17:31.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.036+0000 7f51e2d25700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f51dc134260 0x7f51dc09cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36190/0 (socket says 172.21.15.3:36190) 2023-10-03T10:17:31.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.036+0000 7f51e2524700 1 -- 172.21.15.3:0/3446407926 learned_addr learned my addr 172.21.15.3:0/3446407926 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:31.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.036+0000 7f51e1d23700 1 --2- 172.21.15.3:0/3446407926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51dc133e30 0x7f51dc09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:31.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.036+0000 7f51e2524700 1 -- 172.21.15.3:0/3446407926 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f51dc134260 msgr2=0x7f51dc09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:31.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.036+0000 7f51e2524700 1 --2- 172.21.15.3:0/3446407926 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f51dc134260 0x7f51dc09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.036+0000 7f51e2524700 1 -- 172.21.15.3:0/3446407926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51dc133e30 msgr2=0x7f51dc09c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:31.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.036+0000 7f51e2524700 1 --2- 172.21.15.3:0/3446407926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51dc133e30 0x7f51dc09c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.036+0000 7f51e2524700 1 -- 172.21.15.3:0/3446407926 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d0009550 con 0x7f51dc133420 2023-10-03T10:17:31.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.036+0000 7f51e2d25700 1 --2- 172.21.15.3:0/3446407926 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f51dc134260 0x7f51dc09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:31.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.036+0000 7f51e1d23700 1 --2- 172.21.15.3:0/3446407926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51dc133e30 0x7f51dc09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:31.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.036+0000 7f51e2524700 1 --2- 172.21.15.3:0/3446407926 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f51dc133420 0x7f51dc09c480 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f51d8000c00 tx=0x7f51d8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:31.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.037+0000 7f51ceffd700 1 -- 172.21.15.3:0/3446407926 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8004450 con 0x7f51dc133420 2023-10-03T10:17:31.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.037+0000 7f51ceffd700 1 -- 172.21.15.3:0/3446407926 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d801f070 con 0x7f51dc133420 2023-10-03T10:17:31.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.037+0000 7f51e4788700 1 -- 172.21.15.3:0/3446407926 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d4009a70 con 0x7f51dc133420 2023-10-03T10:17:31.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.037+0000 7f51ceffd700 1 -- 172.21.15.3:0/3446407926 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d80038e0 con 0x7f51dc133420 2023-10-03T10:17:31.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.037+0000 7f51e4788700 1 -- 172.21.15.3:0/3446407926 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d80094a0 con 0x7f51dc133420 2023-10-03T10:17:31.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.037+0000 7f51ceffd700 1 -- 172.21.15.3:0/3446407926 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f51d8003a40 con 0x7f51dc133420 2023-10-03T10:17:31.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.038+0000 7f51ceffd700 1 -- 172.21.15.3:0/3446407926 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f51d8021070 con 0x7f51dc133420 2023-10-03T10:17:31.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.038+0000 7f51e4788700 1 -- 172.21.15.3:0/3446407926 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51dc11f540 con 0x7f51dc133420 2023-10-03T10:17:31.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.043+0000 7f51ceffd700 1 -- 172.21.15.3:0/3446407926 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f51d801a430 con 0x7f51dc133420 2023-10-03T10:17:31.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.248+0000 7f51e4788700 1 -- 172.21.15.3:0/3446407926 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f51dc074ea0 con 0x7f51dc133420 2023-10-03T10:17:31.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.248+0000 7f51ceffd700 1 -- 172.21.15.3:0/3446407926 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f51d8015020 con 0x7f51dc133420 2023-10-03T10:17:31.249 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:31.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.251+0000 7f51e4788700 1 -- 172.21.15.3:0/3446407926 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f51dc133420 msgr2=0x7f51dc09c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:31.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.251+0000 7f51e4788700 1 --2- 172.21.15.3:0/3446407926 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f51dc133420 0x7f51dc09c480 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f51d8000c00 tx=0x7f51d8004750 comp rx=0 tx=0).stop 2023-10-03T10:17:31.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.251+0000 7f51e4788700 1 -- 172.21.15.3:0/3446407926 shutdown_connections 2023-10-03T10:17:31.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.251+0000 7f51e4788700 1 --2- 172.21.15.3:0/3446407926 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f51dc134260 0x7f51dc09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.251+0000 7f51e4788700 1 --2- 172.21.15.3:0/3446407926 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f51dc133420 0x7f51dc09c480 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.251+0000 7f51e4788700 1 --2- 172.21.15.3:0/3446407926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51dc133e30 0x7f51dc09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.251+0000 7f51e4788700 1 -- 172.21.15.3:0/3446407926 >> 172.21.15.3:0/3446407926 conn(0x7f51dc0a4210 msgr2=0x7f51dc0a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:31.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.251+0000 7f51e4788700 1 -- 172.21.15.3:0/3446407926 shutdown_connections 2023-10-03T10:17:31.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.251+0000 7f51e4788700 1 -- 172.21.15.3:0/3446407926 wait complete. 2023-10-03T10:17:31.265 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:31.359 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:31.413 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:31.457 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:31.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.508+0000 7f12d7d98700 1 Processor -- start 2023-10-03T10:17:31.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.508+0000 7f12d7d98700 1 -- start start 2023-10-03T10:17:31.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.509+0000 7f12d7d98700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12d009e1d0 0x7f12d0138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:31.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.509+0000 7f12d7d98700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f12d0139170 0x7f12d013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:31.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.509+0000 7f12d7d98700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d009b120 con 0x7f12d009d6f0 2023-10-03T10:17:31.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.509+0000 7f12d7d98700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d006b460 con 0x7f12d0139170 2023-10-03T10:17:31.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.509+0000 7f12d7d98700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12d009a7e0 con 0x7f12d009e1d0 2023-10-03T10:17:31.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.509+0000 7f12d5b34700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f12d009d6f0 0x7f12d009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45462/0 (socket says 172.21.15.3:45462) 2023-10-03T10:17:31.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.509+0000 7f12d5333700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12d009e1d0 0x7f12d0138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:31.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.509+0000 7f12d5b34700 1 -- 172.21.15.3:0/1352991122 learned_addr learned my addr 172.21.15.3:0/1352991122 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:31.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.509+0000 7f12d6335700 1 --2- 172.21.15.3:0/1352991122 >> v2:172.21.15.3:3300/0 conn(0x7f12d0139170 0x7f12d013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:31.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.510+0000 7f12d5333700 1 -- 172.21.15.3:0/1352991122 >> v2:172.21.15.3:3300/0 conn(0x7f12d0139170 msgr2=0x7f12d013b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:31.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.510+0000 7f12d5333700 1 --2- 172.21.15.3:0/1352991122 >> v2:172.21.15.3:3300/0 conn(0x7f12d0139170 0x7f12d013b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.510+0000 7f12d5333700 1 -- 172.21.15.3:0/1352991122 >> v1:172.21.15.3:6789/0 conn(0x7f12d009d6f0 legacy=0x7f12d009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:31.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.510+0000 7f12d4b32700 1 -- 172.21.15.3:0/1352991122 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2515229669 0 0) 0x7f12d009b120 con 0x7f12d009d6f0 2023-10-03T10:17:31.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.510+0000 7f12d5333700 1 -- 172.21.15.3:0/1352991122 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d009a500 con 0x7f12d009e1d0 2023-10-03T10:17:31.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.510+0000 7f12d5333700 1 --2- 172.21.15.3:0/1352991122 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12d009e1d0 0x7f12d0138c30 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f12c0009870 tx=0x7f12c0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9755b8509c43e75 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:31.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.510+0000 7f12d4b32700 1 -- 172.21.15.3:0/1352991122 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0011830 con 0x7f12d009e1d0 2023-10-03T10:17:31.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.510+0000 7f12d4b32700 1 -- 172.21.15.3:0/1352991122 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c0011990 con 0x7f12d009e1d0 2023-10-03T10:17:31.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.510+0000 7f12d4b32700 1 -- 172.21.15.3:0/1352991122 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0011d20 con 0x7f12d009e1d0 2023-10-03T10:17:31.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.511+0000 7f12d7d98700 1 -- 172.21.15.3:0/1352991122 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12d009e1d0 msgr2=0x7f12d0138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:31.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.511+0000 7f12d7d98700 1 --2- 172.21.15.3:0/1352991122 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12d009e1d0 0x7f12d0138c30 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f12c0009870 tx=0x7f12c0009b80 comp rx=0 tx=0).stop 2023-10-03T10:17:31.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.511+0000 7f12d7d98700 1 -- 172.21.15.3:0/1352991122 shutdown_connections 2023-10-03T10:17:31.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.511+0000 7f12d7d98700 1 --2- 172.21.15.3:0/1352991122 >> v2:172.21.15.3:3300/0 conn(0x7f12d0139170 0x7f12d013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.511+0000 7f12d7d98700 1 --2- 172.21.15.3:0/1352991122 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12d009e1d0 0x7f12d0138c30 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.511+0000 7f12d7d98700 1 -- 172.21.15.3:0/1352991122 >> 172.21.15.3:0/1352991122 conn(0x7f12d0066f50 msgr2=0x7f12d009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:31.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.511+0000 7f12d7d98700 1 -- 172.21.15.3:0/1352991122 shutdown_connections 2023-10-03T10:17:31.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.511+0000 7f12d7d98700 1 -- 172.21.15.3:0/1352991122 wait complete. 2023-10-03T10:17:31.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.512+0000 7f12d7d98700 1 Processor -- start 2023-10-03T10:17:31.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.512+0000 7f12d7d98700 1 -- start start 2023-10-03T10:17:31.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.512+0000 7f12d7d98700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12d009d6f0 0x7f12d0121760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:31.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.512+0000 7f12d7d98700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12d009e1d0 0x7f12d0121ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:31.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.513+0000 7f12d7d98700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12d0139170 0x7f12d01221e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:31.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.513+0000 7f12d7d98700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d0124280 con 0x7f12d0139170 2023-10-03T10:17:31.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.513+0000 7f12d7d98700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d009c710 con 0x7f12d009e1d0 2023-10-03T10:17:31.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.513+0000 7f12d7d98700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12d0124fa0 con 0x7f12d009d6f0 2023-10-03T10:17:31.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.513+0000 7f12d5b34700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12d009d6f0 0x7f12d0121760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:31.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.513+0000 7f12d5333700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12d009e1d0 0x7f12d0121ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:31.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.513+0000 7f12d5b34700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12d009d6f0 0x7f12d0121760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:40120/0 (socket says 172.21.15.3:40120) 2023-10-03T10:17:31.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.513+0000 7f12d5b34700 1 -- 172.21.15.3:0/2908060005 learned_addr learned my addr 172.21.15.3:0/2908060005 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:31.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.513+0000 7f12d6335700 1 --2- 172.21.15.3:0/2908060005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12d0139170 0x7f12d01221e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:31.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.513+0000 7f12d5b34700 1 -- 172.21.15.3:0/2908060005 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12d009e1d0 msgr2=0x7f12d0121ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:31.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.513+0000 7f12d5b34700 1 --2- 172.21.15.3:0/2908060005 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12d009e1d0 0x7f12d0121ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.513+0000 7f12d5b34700 1 -- 172.21.15.3:0/2908060005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12d0139170 msgr2=0x7f12d01221e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:31.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.513+0000 7f12d5b34700 1 --2- 172.21.15.3:0/2908060005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12d0139170 0x7f12d01221e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.513+0000 7f12d5b34700 1 -- 172.21.15.3:0/2908060005 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c00094a0 con 0x7f12d009d6f0 2023-10-03T10:17:31.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.513+0000 7f12d5333700 1 --2- 172.21.15.3:0/2908060005 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12d009e1d0 0x7f12d0121ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:31.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.513+0000 7f12d5b34700 1 --2- 172.21.15.3:0/2908060005 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12d009d6f0 0x7f12d0121760 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f12cc00c550 tx=0x7f12cc00c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:31.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.513+0000 7f12c67fc700 1 -- 172.21.15.3:0/2908060005 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc004c80 con 0x7f12d009d6f0 2023-10-03T10:17:31.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.514+0000 7f12c67fc700 1 -- 172.21.15.3:0/2908060005 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12cc004e00 con 0x7f12d009d6f0 2023-10-03T10:17:31.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.514+0000 7f12c67fc700 1 -- 172.21.15.3:0/2908060005 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc007310 con 0x7f12d009d6f0 2023-10-03T10:17:31.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.514+0000 7f12d7d98700 1 -- 172.21.15.3:0/2908060005 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d0071f40 con 0x7f12d009d6f0 2023-10-03T10:17:31.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.514+0000 7f12d7d98700 1 -- 172.21.15.3:0/2908060005 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d0125a10 con 0x7f12d009d6f0 2023-10-03T10:17:31.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.515+0000 7f12c67fc700 1 -- 172.21.15.3:0/2908060005 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f12cc007470 con 0x7f12d009d6f0 2023-10-03T10:17:31.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.515+0000 7f12d7d98700 1 -- 172.21.15.3:0/2908060005 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d0074ea0 con 0x7f12d009d6f0 2023-10-03T10:17:31.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.515+0000 7f12c67fc700 1 -- 172.21.15.3:0/2908060005 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f12cc03f300 con 0x7f12d009d6f0 2023-10-03T10:17:31.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.520+0000 7f12c67fc700 1 -- 172.21.15.3:0/2908060005 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f12cc0133c0 con 0x7f12d009d6f0 2023-10-03T10:17:31.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.558+0000 7f0f60644700 1 Processor -- start 2023-10-03T10:17:31.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.559+0000 7f0f60644700 1 -- start start 2023-10-03T10:17:31.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.560+0000 7f0f60644700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f580a0a30 0x7f0f580a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:31.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.560+0000 7f0f60644700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0f58138d80 0x7f0f5813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:31.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.560+0000 7f0f60644700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f580a2a70 con 0x7f0f5809d630 2023-10-03T10:17:31.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.560+0000 7f0f60644700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f580a01d0 con 0x7f0f58138d80 2023-10-03T10:17:31.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.560+0000 7f0f60644700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5809e2b0 con 0x7f0f580a0a30 2023-10-03T10:17:31.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.560+0000 7f0f5e3e0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f580a0a30 0x7f0f580a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:31.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.560+0000 7f0f5e3e0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f580a0a30 0x7f0f580a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:40132/0 (socket says 172.21.15.3:40132) 2023-10-03T10:17:31.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.560+0000 7f0f5ebe1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0f58138d80 0x7f0f5813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:31.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.560+0000 7f0f5e3e0700 1 -- 172.21.15.3:0/3107825121 learned_addr learned my addr 172.21.15.3:0/3107825121 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:31.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.560+0000 7f0f5ebe1700 1 -- 172.21.15.3:0/3107825121 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f580a0a30 msgr2=0x7f0f580a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:31.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.560+0000 7f0f5ebe1700 1 --2- 172.21.15.3:0/3107825121 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f580a0a30 0x7f0f580a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.560+0000 7f0f5ebe1700 1 -- 172.21.15.3:0/3107825121 >> v1:172.21.15.3:6789/0 conn(0x7f0f5809d630 legacy=0x7f0f580a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:31.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.560+0000 7f0f5ebe1700 1 -- 172.21.15.3:0/3107825121 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5809dc50 con 0x7f0f58138d80 2023-10-03T10:17:31.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.560+0000 7f0f5e3e0700 1 --2- 172.21.15.3:0/3107825121 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f580a0a30 0x7f0f580a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:17:31.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.561+0000 7f0f5ebe1700 1 --2- 172.21.15.3:0/3107825121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f58138d80 0x7f0f5813b160 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009d90 tx=0x7f0f5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940ee3d26ae4f5c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:31.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.561+0000 7f0f5d3de700 1 -- 172.21.15.3:0/3107825121 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5000e070 con 0x7f0f58138d80 2023-10-03T10:17:31.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.561+0000 7f0f5d3de700 1 -- 172.21.15.3:0/3107825121 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5000bbd0 con 0x7f0f58138d80 2023-10-03T10:17:31.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.561+0000 7f0f5d3de700 1 -- 172.21.15.3:0/3107825121 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f50012430 con 0x7f0f58138d80 2023-10-03T10:17:31.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.561+0000 7f0f60644700 1 -- 172.21.15.3:0/3107825121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f58138d80 msgr2=0x7f0f5813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:31.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.561+0000 7f0f60644700 1 --2- 172.21.15.3:0/3107825121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f58138d80 0x7f0f5813b160 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009d90 tx=0x7f0f5000b330 comp rx=0 tx=0).stop 2023-10-03T10:17:31.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.562+0000 7f0f60644700 1 -- 172.21.15.3:0/3107825121 shutdown_connections 2023-10-03T10:17:31.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.562+0000 7f0f60644700 1 --2- 172.21.15.3:0/3107825121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f58138d80 0x7f0f5813b160 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.562+0000 7f0f60644700 1 --2- 172.21.15.3:0/3107825121 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f580a0a30 0x7f0f580a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.562+0000 7f0f60644700 1 -- 172.21.15.3:0/3107825121 >> 172.21.15.3:0/3107825121 conn(0x7f0f580a3050 msgr2=0x7f0f580a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:31.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.562+0000 7f0f60644700 1 -- 172.21.15.3:0/3107825121 shutdown_connections 2023-10-03T10:17:31.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.562+0000 7f0f60644700 1 -- 172.21.15.3:0/3107825121 wait complete. 2023-10-03T10:17:31.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.563+0000 7f0f60644700 1 Processor -- start 2023-10-03T10:17:31.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.563+0000 7f0f60644700 1 -- start start 2023-10-03T10:17:31.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.563+0000 7f0f60644700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f5809d630 0x7f0f58133930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:31.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.563+0000 7f0f60644700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f580a0a30 0x7f0f58133e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:31.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.563+0000 7f0f60644700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f58138d80 0x7f0f581343b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:31.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.563+0000 7f0f60644700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f58136ef0 con 0x7f0f580a0a30 2023-10-03T10:17:31.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.563+0000 7f0f60644700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f581380a0 con 0x7f0f58138d80 2023-10-03T10:17:31.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.563+0000 7f0f60644700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f581370d0 con 0x7f0f5809d630 2023-10-03T10:17:31.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.563+0000 7f0f5dbdf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f580a0a30 0x7f0f58133e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:31.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.563+0000 7f0f5e3e0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f5809d630 0x7f0f58133930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:31.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.563+0000 7f0f5dbdf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f580a0a30 0x7f0f58133e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40600/0 (socket says 172.21.15.3:40600) 2023-10-03T10:17:31.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.563+0000 7f0f5e3e0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f5809d630 0x7f0f58133930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:40138/0 (socket says 172.21.15.3:40138) 2023-10-03T10:17:31.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.564+0000 7f0f5dbdf700 1 -- 172.21.15.3:0/741721941 learned_addr learned my addr 172.21.15.3:0/741721941 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:31.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.564+0000 7f0f5ebe1700 1 --2- 172.21.15.3:0/741721941 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f58138d80 0x7f0f581343b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:31.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.564+0000 7f0f5dbdf700 1 -- 172.21.15.3:0/741721941 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f5809d630 msgr2=0x7f0f58133930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:31.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.564+0000 7f0f5dbdf700 1 --2- 172.21.15.3:0/741721941 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f5809d630 0x7f0f58133930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.564+0000 7f0f5dbdf700 1 -- 172.21.15.3:0/741721941 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f58138d80 msgr2=0x7f0f581343b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:31.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.564+0000 7f0f5dbdf700 1 --2- 172.21.15.3:0/741721941 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f58138d80 0x7f0f581343b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.564+0000 7f0f5dbdf700 1 -- 172.21.15.3:0/741721941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f50009a70 con 0x7f0f580a0a30 2023-10-03T10:17:31.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.564+0000 7f0f5e3e0700 1 --2- 172.21.15.3:0/741721941 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f5809d630 0x7f0f58133930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:31.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.564+0000 7f0f5dbdf700 1 --2- 172.21.15.3:0/741721941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f580a0a30 0x7f0f58133e70 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800c4e0 tx=0x7f0f4800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:31.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.564+0000 7f0f4effd700 1 -- 172.21.15.3:0/741721941 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f480052d0 con 0x7f0f580a0a30 2023-10-03T10:17:31.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.564+0000 7f0f4effd700 1 -- 172.21.15.3:0/741721941 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f48005430 con 0x7f0f580a0a30 2023-10-03T10:17:31.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.564+0000 7f0f60644700 1 -- 172.21.15.3:0/741721941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f581362a0 con 0x7f0f580a0a30 2023-10-03T10:17:31.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.564+0000 7f0f4effd700 1 -- 172.21.15.3:0/741721941 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f480038c0 con 0x7f0f580a0a30 2023-10-03T10:17:31.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.565+0000 7f0f4effd700 1 -- 172.21.15.3:0/741721941 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0f48003a20 con 0x7f0f580a0a30 2023-10-03T10:17:31.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.565+0000 7f0f60644700 1 -- 172.21.15.3:0/741721941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f58137b40 con 0x7f0f580a0a30 2023-10-03T10:17:31.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.566+0000 7f0f4effd700 1 -- 172.21.15.3:0/741721941 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0f48040a60 con 0x7f0f580a0a30 2023-10-03T10:17:31.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.567+0000 7f0f60644700 1 -- 172.21.15.3:0/741721941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f580747f0 con 0x7f0f580a0a30 2023-10-03T10:17:31.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.572+0000 7f0f4effd700 1 -- 172.21.15.3:0/741721941 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0f48017080 con 0x7f0f580a0a30 2023-10-03T10:17:31.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.614+0000 7f5b183ba700 1 Processor -- start 2023-10-03T10:17:31.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.614+0000 7f5b183ba700 1 -- start start 2023-10-03T10:17:31.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.615+0000 7f5b183ba700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b10133420 0x7f5b101337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:31.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.615+0000 7f5b183ba700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5b10134260 0x7f5b101306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:31.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.615+0000 7f5b183ba700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b1006b990 con 0x7f5b10133e30 2023-10-03T10:17:31.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.615+0000 7f5b183ba700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b100a0590 con 0x7f5b10134260 2023-10-03T10:17:31.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.615+0000 7f5b183ba700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1009fb80 con 0x7f5b10133420 2023-10-03T10:17:31.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.615+0000 7f5b16156700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b10133420 0x7f5b101337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:31.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.615+0000 7f5b16957700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5b10134260 0x7f5b101306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:31.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.615+0000 7f5b15955700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5b10133e30 0x7f5b1012ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45472/0 (socket says 172.21.15.3:45472) 2023-10-03T10:17:31.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.615+0000 7f5b15955700 1 -- 172.21.15.3:0/3249281194 learned_addr learned my addr 172.21.15.3:0/3249281194 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:31.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.616+0000 7f5b16957700 1 -- 172.21.15.3:0/3249281194 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b10133420 msgr2=0x7f5b101337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:31.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.616+0000 7f5b16957700 1 --2- 172.21.15.3:0/3249281194 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b10133420 0x7f5b101337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.616+0000 7f5b16957700 1 -- 172.21.15.3:0/3249281194 >> v1:172.21.15.3:6789/0 conn(0x7f5b10133e30 legacy=0x7f5b1012ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:31.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.616+0000 7f5b16957700 1 -- 172.21.15.3:0/3249281194 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b100a0030 con 0x7f5b10134260 2023-10-03T10:17:31.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.616+0000 7f5b16957700 1 --2- 172.21.15.3:0/3249281194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b10134260 0x7f5b101306a0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009d90 tx=0x7f5b0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411fe063a30b10a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:31.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.616+0000 7f5b15154700 1 -- 172.21.15.3:0/3249281194 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0800e070 con 0x7f5b10134260 2023-10-03T10:17:31.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.616+0000 7f5b15154700 1 -- 172.21.15.3:0/3249281194 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0800bb20 con 0x7f5b10134260 2023-10-03T10:17:31.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.616+0000 7f5b15154700 1 -- 172.21.15.3:0/3249281194 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b08012430 con 0x7f5b10134260 2023-10-03T10:17:31.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.616+0000 7f5b16156700 1 --2- 172.21.15.3:0/3249281194 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b10133420 0x7f5b101337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:31.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.617+0000 7f5b183ba700 1 -- 172.21.15.3:0/3249281194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b10134260 msgr2=0x7f5b101306a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:31.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.617+0000 7f5b183ba700 1 --2- 172.21.15.3:0/3249281194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b10134260 0x7f5b101306a0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009d90 tx=0x7f5b0800b330 comp rx=0 tx=0).stop 2023-10-03T10:17:31.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.617+0000 7f5b183ba700 1 -- 172.21.15.3:0/3249281194 shutdown_connections 2023-10-03T10:17:31.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.617+0000 7f5b183ba700 1 --2- 172.21.15.3:0/3249281194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b10134260 0x7f5b101306a0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.617+0000 7f5b183ba700 1 --2- 172.21.15.3:0/3249281194 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b10133420 0x7f5b101337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.617+0000 7f5b183ba700 1 -- 172.21.15.3:0/3249281194 >> 172.21.15.3:0/3249281194 conn(0x7f5b100a4210 msgr2=0x7f5b100a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:31.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.617+0000 7f5b183ba700 1 -- 172.21.15.3:0/3249281194 shutdown_connections 2023-10-03T10:17:31.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.617+0000 7f5b183ba700 1 -- 172.21.15.3:0/3249281194 wait complete. 2023-10-03T10:17:31.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.618+0000 7f5b183ba700 1 Processor -- start 2023-10-03T10:17:31.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.618+0000 7f5b183ba700 1 -- start start 2023-10-03T10:17:31.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.619+0000 7f5b183ba700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b10133420 0x7f5b1009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:31.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.619+0000 7f5b183ba700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b10133e30 0x7f5b1009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:31.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.619+0000 7f5b183ba700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5b10134260 0x7f5b1009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:31.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.619+0000 7f5b183ba700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1009d850 con 0x7f5b10133e30 2023-10-03T10:17:31.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.619+0000 7f5b183ba700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1009ddf0 con 0x7f5b10133420 2023-10-03T10:17:31.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.619+0000 7f5b183ba700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b101202d0 con 0x7f5b10134260 2023-10-03T10:17:31.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.619+0000 7f5b15955700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b10133e30 0x7f5b1009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:31.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.619+0000 7f5b16156700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b10133420 0x7f5b1009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:31.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.619+0000 7f5b15955700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b10133e30 0x7f5b1009c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40612/0 (socket says 172.21.15.3:40612) 2023-10-03T10:17:31.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.619+0000 7f5b15955700 1 -- 172.21.15.3:0/392933088 learned_addr learned my addr 172.21.15.3:0/392933088 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:31.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.619+0000 7f5b16957700 1 --2- 172.21.15.3:0/392933088 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5b10134260 0x7f5b1009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:31.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.619+0000 7f5b16957700 1 -- 172.21.15.3:0/392933088 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b10133420 msgr2=0x7f5b1009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:31.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.619+0000 7f5b16957700 1 --2- 172.21.15.3:0/392933088 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b10133420 0x7f5b1009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.619+0000 7f5b16957700 1 -- 172.21.15.3:0/392933088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b10133e30 msgr2=0x7f5b1009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:31.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.619+0000 7f5b16957700 1 --2- 172.21.15.3:0/392933088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b10133e30 0x7f5b1009c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.619+0000 7f5b16957700 1 -- 172.21.15.3:0/392933088 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0c0094a0 con 0x7f5b10134260 2023-10-03T10:17:31.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.619+0000 7f5b15955700 1 --2- 172.21.15.3:0/392933088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b10133e30 0x7f5b1009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:31.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.619+0000 7f5b16156700 1 --2- 172.21.15.3:0/392933088 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b10133420 0x7f5b1009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:31.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.619+0000 7f5b16957700 1 --2- 172.21.15.3:0/392933088 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5b10134260 0x7f5b1009cf00 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5b08016000 tx=0x7f5b08004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:31.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.620+0000 7f5b06ffd700 1 -- 172.21.15.3:0/392933088 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b08012430 con 0x7f5b10134260 2023-10-03T10:17:31.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.620+0000 7f5b06ffd700 1 -- 172.21.15.3:0/392933088 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b08020070 con 0x7f5b10134260 2023-10-03T10:17:31.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.620+0000 7f5b183ba700 1 -- 172.21.15.3:0/392933088 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b08009a70 con 0x7f5b10134260 2023-10-03T10:17:31.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.620+0000 7f5b06ffd700 1 -- 172.21.15.3:0/392933088 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b08012e90 con 0x7f5b10134260 2023-10-03T10:17:31.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.620+0000 7f5b183ba700 1 -- 172.21.15.3:0/392933088 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b1012d190 con 0x7f5b10134260 2023-10-03T10:17:31.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.621+0000 7f5b183ba700 1 -- 172.21.15.3:0/392933088 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b1009e3c0 con 0x7f5b10134260 2023-10-03T10:17:31.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.621+0000 7f5b06ffd700 1 -- 172.21.15.3:0/392933088 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5b08004040 con 0x7f5b10134260 2023-10-03T10:17:31.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.621+0000 7f5b06ffd700 1 -- 172.21.15.3:0/392933088 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5b08022070 con 0x7f5b10134260 2023-10-03T10:17:31.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.630+0000 7f5b06ffd700 1 -- 172.21.15.3:0/392933088 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5b080042f0 con 0x7f5b10134260 2023-10-03T10:17:31.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.746+0000 7f12d7d98700 1 -- 172.21.15.3:0/2908060005 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f12d009f420 con 0x7f12d009d6f0 2023-10-03T10:17:31.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.746+0000 7f12c67fc700 1 -- 172.21.15.3:0/2908060005 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f12cc011360 con 0x7f12d009d6f0 2023-10-03T10:17:31.747 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:31.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.748+0000 7f12d7d98700 1 -- 172.21.15.3:0/2908060005 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12d009d6f0 msgr2=0x7f12d0121760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:31.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.748+0000 7f12d7d98700 1 --2- 172.21.15.3:0/2908060005 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12d009d6f0 0x7f12d0121760 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f12cc00c550 tx=0x7f12cc00c860 comp rx=0 tx=0).stop 2023-10-03T10:17:31.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.749+0000 7f12d7d98700 1 -- 172.21.15.3:0/2908060005 shutdown_connections 2023-10-03T10:17:31.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.749+0000 7f12d7d98700 1 --2- 172.21.15.3:0/2908060005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12d0139170 0x7f12d01221e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.749+0000 7f12d7d98700 1 --2- 172.21.15.3:0/2908060005 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12d009d6f0 0x7f12d0121760 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.749+0000 7f12d7d98700 1 --2- 172.21.15.3:0/2908060005 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12d009e1d0 0x7f12d0121ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.749+0000 7f12d7d98700 1 -- 172.21.15.3:0/2908060005 >> 172.21.15.3:0/2908060005 conn(0x7f12d0066f50 msgr2=0x7f12d0070d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:31.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.749+0000 7f12d7d98700 1 -- 172.21.15.3:0/2908060005 shutdown_connections 2023-10-03T10:17:31.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.749+0000 7f12d7d98700 1 -- 172.21.15.3:0/2908060005 wait complete. 2023-10-03T10:17:31.763 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:31.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.787+0000 7f0f60644700 1 -- 172.21.15.3:0/741721941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0f58072980 con 0x7f0f580a0a30 2023-10-03T10:17:31.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.787+0000 7f0f4effd700 1 -- 172.21.15.3:0/741721941 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0f48044060 con 0x7f0f580a0a30 2023-10-03T10:17:31.787 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:31.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.789+0000 7f0f60644700 1 -- 172.21.15.3:0/741721941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f580a0a30 msgr2=0x7f0f58133e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:31.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.789+0000 7f0f60644700 1 --2- 172.21.15.3:0/741721941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f580a0a30 0x7f0f58133e70 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800c4e0 tx=0x7f0f4800c7f0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.789+0000 7f0f60644700 1 -- 172.21.15.3:0/741721941 shutdown_connections 2023-10-03T10:17:31.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.789+0000 7f0f60644700 1 --2- 172.21.15.3:0/741721941 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f58138d80 0x7f0f581343b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.789+0000 7f0f60644700 1 --2- 172.21.15.3:0/741721941 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f5809d630 0x7f0f58133930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.789+0000 7f0f60644700 1 --2- 172.21.15.3:0/741721941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f580a0a30 0x7f0f58133e70 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.789+0000 7f0f60644700 1 -- 172.21.15.3:0/741721941 >> 172.21.15.3:0/741721941 conn(0x7f0f580a3050 msgr2=0x7f0f5809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:31.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.789+0000 7f0f60644700 1 -- 172.21.15.3:0/741721941 shutdown_connections 2023-10-03T10:17:31.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.789+0000 7f0f60644700 1 -- 172.21.15.3:0/741721941 wait complete. 2023-10-03T10:17:31.803 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.837+0000 7f5b183ba700 1 -- 172.21.15.3:0/392933088 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5b10074ea0 con 0x7f5b10134260 2023-10-03T10:17:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.838+0000 7f5b06ffd700 1 -- 172.21.15.3:0/392933088 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5b08030390 con 0x7f5b10134260 2023-10-03T10:17:31.838 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:31.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.840+0000 7f5b183ba700 1 -- 172.21.15.3:0/392933088 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5b10134260 msgr2=0x7f5b1009cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:31.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.840+0000 7f5b183ba700 1 --2- 172.21.15.3:0/392933088 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5b10134260 0x7f5b1009cf00 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5b08016000 tx=0x7f5b08004ad0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.840+0000 7f5b183ba700 1 -- 172.21.15.3:0/392933088 shutdown_connections 2023-10-03T10:17:31.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.840+0000 7f5b183ba700 1 --2- 172.21.15.3:0/392933088 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5b10134260 0x7f5b1009cf00 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.840+0000 7f5b183ba700 1 --2- 172.21.15.3:0/392933088 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b10133420 0x7f5b1009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.840+0000 7f5b183ba700 1 --2- 172.21.15.3:0/392933088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b10133e30 0x7f5b1009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:31.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.840+0000 7f5b183ba700 1 -- 172.21.15.3:0/392933088 >> 172.21.15.3:0/392933088 conn(0x7f5b100a4210 msgr2=0x7f5b100a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:31.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.840+0000 7f5b183ba700 1 -- 172.21.15.3:0/392933088 shutdown_connections 2023-10-03T10:17:31.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:31.840+0000 7f5b183ba700 1 -- 172.21.15.3:0/392933088 wait complete. 2023-10-03T10:17:31.853 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:32.265 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:32.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.412+0000 7f1b7b034700 1 Processor -- start 2023-10-03T10:17:32.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.412+0000 7f1b7b034700 1 -- start start 2023-10-03T10:17:32.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.412+0000 7f1b7b034700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b7409e1d0 0x7f1b74135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:32.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.412+0000 7f1b7b034700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1b74135e50 0x7f1b74138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:32.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.412+0000 7f1b7b034700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b7409f130 con 0x7f1b7409d6f0 2023-10-03T10:17:32.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.412+0000 7f1b7b034700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b7409f520 con 0x7f1b74135e50 2023-10-03T10:17:32.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.412+0000 7f1b7b034700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7409e930 con 0x7f1b7409e1d0 2023-10-03T10:17:32.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.412+0000 7f1b73fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b7409e1d0 0x7f1b74135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:32.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.412+0000 7f1b795d1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1b74135e50 0x7f1b74138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:32.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.412+0000 7f1b73fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b7409e1d0 0x7f1b74135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:40154/0 (socket says 172.21.15.3:40154) 2023-10-03T10:17:32.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.412+0000 7f1b795d1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1b74135e50 0x7f1b74138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40624/0 (socket says 172.21.15.3:40624) 2023-10-03T10:17:32.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.412+0000 7f1b73fff700 1 -- 172.21.15.3:0/1667968339 learned_addr learned my addr 172.21.15.3:0/1667968339 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:32.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.413+0000 7f1b73fff700 1 -- 172.21.15.3:0/1667968339 >> v2:172.21.15.3:3300/0 conn(0x7f1b74135e50 msgr2=0x7f1b74138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:32.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.413+0000 7f1b73fff700 1 --2- 172.21.15.3:0/1667968339 >> v2:172.21.15.3:3300/0 conn(0x7f1b74135e50 0x7f1b74138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:32.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.413+0000 7f1b73fff700 1 -- 172.21.15.3:0/1667968339 >> v1:172.21.15.3:6789/0 conn(0x7f1b7409d6f0 legacy=0x7f1b7409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:32.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.413+0000 7f1b73fff700 1 -- 172.21.15.3:0/1667968339 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b7409b2d0 con 0x7f1b7409e1d0 2023-10-03T10:17:32.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.413+0000 7f1b73fff700 1 --2- 172.21.15.3:0/1667968339 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b7409e1d0 0x7f1b74135910 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1b60009920 tx=0x7f1b60009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a33a78649a84ef2c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:32.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.413+0000 7f1b737fe700 1 -- 172.21.15.3:0/1667968339 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b60011830 con 0x7f1b7409e1d0 2023-10-03T10:17:32.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.413+0000 7f1b737fe700 1 -- 172.21.15.3:0/1667968339 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b60011990 con 0x7f1b7409e1d0 2023-10-03T10:17:32.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.413+0000 7f1b737fe700 1 -- 172.21.15.3:0/1667968339 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b60011d20 con 0x7f1b7409e1d0 2023-10-03T10:17:32.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.414+0000 7f1b7b034700 1 -- 172.21.15.3:0/1667968339 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b7409e1d0 msgr2=0x7f1b74135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:32.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.414+0000 7f1b7b034700 1 --2- 172.21.15.3:0/1667968339 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b7409e1d0 0x7f1b74135910 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1b60009920 tx=0x7f1b60009c30 comp rx=0 tx=0).stop 2023-10-03T10:17:32.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.414+0000 7f1b7b034700 1 -- 172.21.15.3:0/1667968339 shutdown_connections 2023-10-03T10:17:32.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.414+0000 7f1b7b034700 1 --2- 172.21.15.3:0/1667968339 >> v2:172.21.15.3:3300/0 conn(0x7f1b74135e50 0x7f1b74138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:32.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.414+0000 7f1b7b034700 1 --2- 172.21.15.3:0/1667968339 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b7409e1d0 0x7f1b74135910 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:32.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.414+0000 7f1b7b034700 1 -- 172.21.15.3:0/1667968339 >> 172.21.15.3:0/1667968339 conn(0x7f1b74066f50 msgr2=0x7f1b7409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:32.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.414+0000 7f1b7b034700 1 -- 172.21.15.3:0/1667968339 shutdown_connections 2023-10-03T10:17:32.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.414+0000 7f1b7b034700 1 -- 172.21.15.3:0/1667968339 wait complete. 2023-10-03T10:17:32.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.415+0000 7f1b7b034700 1 Processor -- start 2023-10-03T10:17:32.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.415+0000 7f1b7b034700 1 -- start start 2023-10-03T10:17:32.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.415+0000 7f1b7b034700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b7409d6f0 0x7f1b7411f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:32.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.415+0000 7f1b7b034700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b7409e1d0 0x7f1b7411f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:32.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.415+0000 7f1b7b034700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b74135e50 0x7f1b7411fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:32.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.415+0000 7f1b7b034700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b741344e0 con 0x7f1b7409e1d0 2023-10-03T10:17:32.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.415+0000 7f1b7b034700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b74121120 con 0x7f1b74135e50 2023-10-03T10:17:32.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.415+0000 7f1b7b034700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1b740a1f30 con 0x7f1b7409d6f0 2023-10-03T10:17:32.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.416+0000 7f1b73fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b7409e1d0 0x7f1b7411f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:32.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.416+0000 7f1b78dd0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b7409d6f0 0x7f1b7411f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:32.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.416+0000 7f1b73fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b7409e1d0 0x7f1b7411f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40632/0 (socket says 172.21.15.3:40632) 2023-10-03T10:17:32.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.416+0000 7f1b78dd0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b7409d6f0 0x7f1b7411f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:40164/0 (socket says 172.21.15.3:40164) 2023-10-03T10:17:32.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.416+0000 7f1b795d1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b74135e50 0x7f1b7411fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:32.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.416+0000 7f1b73fff700 1 -- 172.21.15.3:0/2006594637 learned_addr learned my addr 172.21.15.3:0/2006594637 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:32.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.416+0000 7f1b78dd0700 1 -- 172.21.15.3:0/2006594637 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b74135e50 msgr2=0x7f1b7411fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:32.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.416+0000 7f1b78dd0700 1 --2- 172.21.15.3:0/2006594637 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b74135e50 0x7f1b7411fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:32.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.416+0000 7f1b78dd0700 1 -- 172.21.15.3:0/2006594637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b7409e1d0 msgr2=0x7f1b7411f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:32.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.416+0000 7f1b78dd0700 1 --2- 172.21.15.3:0/2006594637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b7409e1d0 0x7f1b7411f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:32.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.416+0000 7f1b78dd0700 1 -- 172.21.15.3:0/2006594637 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b60009550 con 0x7f1b7409d6f0 2023-10-03T10:17:32.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.416+0000 7f1b78dd0700 1 --2- 172.21.15.3:0/2006594637 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b7409d6f0 0x7f1b7411f470 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800c450 tx=0x7f1b6800c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:32.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.416+0000 7f1b717fa700 1 -- 172.21.15.3:0/2006594637 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b680052d0 con 0x7f1b7409d6f0 2023-10-03T10:17:32.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.416+0000 7f1b717fa700 1 -- 172.21.15.3:0/2006594637 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b68005430 con 0x7f1b7409d6f0 2023-10-03T10:17:32.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.416+0000 7f1b717fa700 1 -- 172.21.15.3:0/2006594637 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b680038c0 con 0x7f1b7409d6f0 2023-10-03T10:17:32.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.416+0000 7f1b7b034700 1 -- 172.21.15.3:0/2006594637 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b7409c760 con 0x7f1b7409d6f0 2023-10-03T10:17:32.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.416+0000 7f1b7b034700 1 -- 172.21.15.3:0/2006594637 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b74123720 con 0x7f1b7409d6f0 2023-10-03T10:17:32.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.417+0000 7f1b717fa700 1 -- 172.21.15.3:0/2006594637 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1b68003a20 con 0x7f1b7409d6f0 2023-10-03T10:17:32.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.417+0000 7f1b717fa700 1 -- 172.21.15.3:0/2006594637 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1b6803e9d0 con 0x7f1b7409d6f0 2023-10-03T10:17:32.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.417+0000 7f1b7b034700 1 -- 172.21.15.3:0/2006594637 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b74074ea0 con 0x7f1b7409d6f0 2023-10-03T10:17:32.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.422+0000 7f1b717fa700 1 -- 172.21.15.3:0/2006594637 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1b68017070 con 0x7f1b7409d6f0 2023-10-03T10:17:32.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.628+0000 7f1b7b034700 1 -- 172.21.15.3:0/2006594637 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1b7406c360 con 0x7f1b7409d6f0 2023-10-03T10:17:32.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.629+0000 7f1b717fa700 1 -- 172.21.15.3:0/2006594637 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1b68011590 con 0x7f1b7409d6f0 2023-10-03T10:17:32.629 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:32.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.631+0000 7f1b7b034700 1 -- 172.21.15.3:0/2006594637 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b7409d6f0 msgr2=0x7f1b7411f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:32.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.631+0000 7f1b7b034700 1 --2- 172.21.15.3:0/2006594637 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b7409d6f0 0x7f1b7411f470 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800c450 tx=0x7f1b6800c760 comp rx=0 tx=0).stop 2023-10-03T10:17:32.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.631+0000 7f1b7b034700 1 -- 172.21.15.3:0/2006594637 shutdown_connections 2023-10-03T10:17:32.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.631+0000 7f1b7b034700 1 --2- 172.21.15.3:0/2006594637 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b74135e50 0x7f1b7411fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:32.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.631+0000 7f1b7b034700 1 --2- 172.21.15.3:0/2006594637 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b7409d6f0 0x7f1b7411f470 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:32.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.631+0000 7f1b7b034700 1 --2- 172.21.15.3:0/2006594637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b7409e1d0 0x7f1b7411f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:32.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.631+0000 7f1b7b034700 1 -- 172.21.15.3:0/2006594637 >> 172.21.15.3:0/2006594637 conn(0x7f1b74066f50 msgr2=0x7f1b740712a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:32.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.631+0000 7f1b7b034700 1 -- 172.21.15.3:0/2006594637 shutdown_connections 2023-10-03T10:17:32.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.631+0000 7f1b7b034700 1 -- 172.21.15.3:0/2006594637 wait complete. 2023-10-03T10:17:32.645 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:32.764 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:32.804 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:32.854 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:32.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.916+0000 7fc26b9be700 1 Processor -- start 2023-10-03T10:17:32.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.917+0000 7fc26b9be700 1 -- start start 2023-10-03T10:17:32.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.918+0000 7fc26b9be700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc26409d6f0 0x7fc26409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:32.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.918+0000 7fc26b9be700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc26409e000 0x7fc264130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:32.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.918+0000 7fc26b9be700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc26409f130 con 0x7fc26409e410 2023-10-03T10:17:32.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.918+0000 7fc26b9be700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc26409f520 con 0x7fc26409d6f0 2023-10-03T10:17:32.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.918+0000 7fc26b9be700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc26409e930 con 0x7fc26409e000 2023-10-03T10:17:32.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.918+0000 7fc26975a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc26409d6f0 0x7fc26409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:32.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.918+0000 7fc268f59700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc26409e000 0x7fc264130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:32.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.918+0000 7fc26975a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc26409d6f0 0x7fc26409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40644/0 (socket says 172.21.15.3:40644) 2023-10-03T10:17:32.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.918+0000 7fc26975a700 1 -- 172.21.15.3:0/2148359331 learned_addr learned my addr 172.21.15.3:0/2148359331 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:32.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.918+0000 7fc268f59700 1 -- 172.21.15.3:0/2148359331 >> v2:172.21.15.3:3300/0 conn(0x7fc26409d6f0 msgr2=0x7fc26409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:32.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.918+0000 7fc268f59700 1 --2- 172.21.15.3:0/2148359331 >> v2:172.21.15.3:3300/0 conn(0x7fc26409d6f0 0x7fc26409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:32.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.918+0000 7fc268f59700 1 -- 172.21.15.3:0/2148359331 >> v1:172.21.15.3:6789/0 conn(0x7fc26409e410 legacy=0x7fc264130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:32.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.918+0000 7fc268f59700 1 -- 172.21.15.3:0/2148359331 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc26409b2d0 con 0x7fc26409e000 2023-10-03T10:17:32.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.918+0000 7fc26975a700 1 --2- 172.21.15.3:0/2148359331 >> v2:172.21.15.3:3300/0 conn(0x7fc26409d6f0 0x7fc26409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:32.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.918+0000 7fc268f59700 1 --2- 172.21.15.3:0/2148359331 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc26409e000 0x7fc264130660 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc254009840 tx=0x7fc254009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce079cd2f376b1d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:32.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.919+0000 7fc25bfff700 1 -- 172.21.15.3:0/2148359331 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc254011830 con 0x7fc26409e000 2023-10-03T10:17:32.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.919+0000 7fc25bfff700 1 -- 172.21.15.3:0/2148359331 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc254011990 con 0x7fc26409e000 2023-10-03T10:17:32.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.919+0000 7fc25bfff700 1 -- 172.21.15.3:0/2148359331 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc254011d20 con 0x7fc26409e000 2023-10-03T10:17:32.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.920+0000 7fc26b9be700 1 -- 172.21.15.3:0/2148359331 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc26409e000 msgr2=0x7fc264130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:32.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.920+0000 7fc26b9be700 1 --2- 172.21.15.3:0/2148359331 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc26409e000 0x7fc264130660 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc254009840 tx=0x7fc254009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:32.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.920+0000 7fc26b9be700 1 -- 172.21.15.3:0/2148359331 shutdown_connections 2023-10-03T10:17:32.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.920+0000 7fc26b9be700 1 --2- 172.21.15.3:0/2148359331 >> v2:172.21.15.3:3300/0 conn(0x7fc26409d6f0 0x7fc26409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:32.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.920+0000 7fc26b9be700 1 --2- 172.21.15.3:0/2148359331 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc26409e000 0x7fc264130660 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:32.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.920+0000 7fc26b9be700 1 -- 172.21.15.3:0/2148359331 >> 172.21.15.3:0/2148359331 conn(0x7fc264066f50 msgr2=0x7fc26409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:32.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.920+0000 7fc26b9be700 1 -- 172.21.15.3:0/2148359331 shutdown_connections 2023-10-03T10:17:32.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.920+0000 7fc26b9be700 1 -- 172.21.15.3:0/2148359331 wait complete. 2023-10-03T10:17:32.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.921+0000 7fc26b9be700 1 Processor -- start 2023-10-03T10:17:32.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.921+0000 7fc26b9be700 1 -- start start 2023-10-03T10:17:32.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.921+0000 7fc26b9be700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc26409d6f0 0x7fc26411f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:32.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.921+0000 7fc26b9be700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc26409e000 0x7fc26411f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:32.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.921+0000 7fc26b9be700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc26409e410 0x7fc26411fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:32.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.921+0000 7fc26b9be700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc264122ce0 con 0x7fc26409d6f0 2023-10-03T10:17:32.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.921+0000 7fc26b9be700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc264120dd0 con 0x7fc26409e410 2023-10-03T10:17:32.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.921+0000 7fc26b9be700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc26406e8d0 con 0x7fc26409e000 2023-10-03T10:17:32.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.922+0000 7fc269f5b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc26409e410 0x7fc26411fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:32.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.922+0000 7fc269f5b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc26409e410 0x7fc26411fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:36232/0 (socket says 172.21.15.3:36232) 2023-10-03T10:17:32.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.922+0000 7fc269f5b700 1 -- 172.21.15.3:0/2463645482 learned_addr learned my addr 172.21.15.3:0/2463645482 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:32.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.922+0000 7fc26975a700 1 --2- 172.21.15.3:0/2463645482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc26409d6f0 0x7fc26411f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:32.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.922+0000 7fc268f59700 1 --2- 172.21.15.3:0/2463645482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc26409e000 0x7fc26411f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:32.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.922+0000 7fc269f5b700 1 -- 172.21.15.3:0/2463645482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc26409e000 msgr2=0x7fc26411f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:32.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.922+0000 7fc269f5b700 1 --2- 172.21.15.3:0/2463645482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc26409e000 0x7fc26411f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:32.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.922+0000 7fc269f5b700 1 -- 172.21.15.3:0/2463645482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc26409d6f0 msgr2=0x7fc26411f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:32.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.922+0000 7fc269f5b700 1 --2- 172.21.15.3:0/2463645482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc26409d6f0 0x7fc26411f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:32.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.922+0000 7fc269f5b700 1 -- 172.21.15.3:0/2463645482 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc260009550 con 0x7fc26409e410 2023-10-03T10:17:32.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.922+0000 7fc269f5b700 1 --2- 172.21.15.3:0/2463645482 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc26409e410 0x7fc26411fc40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc25c0021c0 tx=0x7fc25c0021f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:32.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.922+0000 7fc259ffb700 1 -- 172.21.15.3:0/2463645482 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c003a00 con 0x7fc26409e410 2023-10-03T10:17:32.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.923+0000 7fc259ffb700 1 -- 172.21.15.3:0/2463645482 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25c003b60 con 0x7fc26409e410 2023-10-03T10:17:32.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.923+0000 7fc26b9be700 1 -- 172.21.15.3:0/2463645482 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2540094a0 con 0x7fc26409e410 2023-10-03T10:17:32.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.923+0000 7fc259ffb700 1 -- 172.21.15.3:0/2463645482 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c0050d0 con 0x7fc26409e410 2023-10-03T10:17:32.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.923+0000 7fc26b9be700 1 -- 172.21.15.3:0/2463645482 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc26412fce0 con 0x7fc26409e410 2023-10-03T10:17:32.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.924+0000 7fc259ffb700 1 -- 172.21.15.3:0/2463645482 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc25c005230 con 0x7fc26409e410 2023-10-03T10:17:32.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.924+0000 7fc259ffb700 1 -- 172.21.15.3:0/2463645482 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc25c041e50 con 0x7fc26409e410 2023-10-03T10:17:32.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.924+0000 7fc26b9be700 1 -- 172.21.15.3:0/2463645482 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc26409dcd0 con 0x7fc26409e410 2023-10-03T10:17:32.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.929+0000 7fc259ffb700 1 -- 172.21.15.3:0/2463645482 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc25c011470 con 0x7fc26409e410 2023-10-03T10:17:32.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.953+0000 7f44f9547700 1 Processor -- start 2023-10-03T10:17:32.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.954+0000 7f44f9547700 1 -- start start 2023-10-03T10:17:32.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.954+0000 7f44f9547700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44f409e930 0x7f44f409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:32.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.954+0000 7f44f9547700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44f409f650 0x7f44f4131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:32.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.954+0000 7f44f9547700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44f4070940 con 0x7f44f409f240 2023-10-03T10:17:32.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.954+0000 7f44f9547700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44f406fd60 con 0x7f44f409e930 2023-10-03T10:17:32.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.954+0000 7f44f9547700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f44f406fa80 con 0x7f44f409f650 2023-10-03T10:17:32.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.954+0000 7f44f2ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44f409e930 0x7f44f409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:32.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.954+0000 7f44f37fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44f409f650 0x7f44f4131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:32.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.954+0000 7f44f2ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44f409e930 0x7f44f409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40670/0 (socket says 172.21.15.3:40670) 2023-10-03T10:17:32.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.954+0000 7f44f27fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f44f409f240 0x7f44f41317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45492/0 (socket says 172.21.15.3:45492) 2023-10-03T10:17:32.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.954+0000 7f44f2ffd700 1 -- 172.21.15.3:0/1998664613 learned_addr learned my addr 172.21.15.3:0/1998664613 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:32.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.954+0000 7f44f37fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44f409f650 0x7f44f4131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:40194/0 (socket says 172.21.15.3:40194) 2023-10-03T10:17:32.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.955+0000 7f44f37fe700 1 -- 172.21.15.3:0/1998664613 >> v2:172.21.15.3:3300/0 conn(0x7f44f409e930 msgr2=0x7f44f409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:32.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.955+0000 7f44f37fe700 1 --2- 172.21.15.3:0/1998664613 >> v2:172.21.15.3:3300/0 conn(0x7f44f409e930 0x7f44f409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:32.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.955+0000 7f44f37fe700 1 -- 172.21.15.3:0/1998664613 >> v1:172.21.15.3:6789/0 conn(0x7f44f409f240 legacy=0x7f44f41317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:32.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.955+0000 7f44f37fe700 1 -- 172.21.15.3:0/1998664613 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f409fb60 con 0x7f44f409f650 2023-10-03T10:17:32.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.955+0000 7f44f2ffd700 1 --2- 172.21.15.3:0/1998664613 >> v2:172.21.15.3:3300/0 conn(0x7f44f409e930 0x7f44f409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:32.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.955+0000 7f44f37fe700 1 --2- 172.21.15.3:0/1998664613 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44f409f650 0x7f44f4131f00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009d90 tx=0x7f44ec00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=622834147246f6b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:32.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.955+0000 7f44f1ffb700 1 -- 172.21.15.3:0/1998664613 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec00e070 con 0x7f44f409f650 2023-10-03T10:17:32.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.955+0000 7f44f1ffb700 1 -- 172.21.15.3:0/1998664613 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ec00bb20 con 0x7f44f409f650 2023-10-03T10:17:32.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.955+0000 7f44f1ffb700 1 -- 172.21.15.3:0/1998664613 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec012430 con 0x7f44f409f650 2023-10-03T10:17:32.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.956+0000 7f44f9547700 1 -- 172.21.15.3:0/1998664613 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44f409f650 msgr2=0x7f44f4131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:32.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.956+0000 7f44f9547700 1 --2- 172.21.15.3:0/1998664613 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44f409f650 0x7f44f4131f00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009d90 tx=0x7f44ec00b330 comp rx=0 tx=0).stop 2023-10-03T10:17:32.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.956+0000 7f44f9547700 1 -- 172.21.15.3:0/1998664613 shutdown_connections 2023-10-03T10:17:32.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.956+0000 7f44f9547700 1 --2- 172.21.15.3:0/1998664613 >> v2:172.21.15.3:3300/0 conn(0x7f44f409e930 0x7f44f409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:32.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.956+0000 7f44f9547700 1 --2- 172.21.15.3:0/1998664613 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44f409f650 0x7f44f4131f00 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:32.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.956+0000 7f44f9547700 1 -- 172.21.15.3:0/1998664613 >> 172.21.15.3:0/1998664613 conn(0x7f44f40a4210 msgr2=0x7f44f40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:32.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.956+0000 7f44f9547700 1 -- 172.21.15.3:0/1998664613 shutdown_connections 2023-10-03T10:17:32.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.956+0000 7f44f9547700 1 -- 172.21.15.3:0/1998664613 wait complete. 2023-10-03T10:17:32.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.957+0000 7f44f9547700 1 Processor -- start 2023-10-03T10:17:32.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.957+0000 7f44f9547700 1 -- start start 2023-10-03T10:17:32.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.958+0000 7f44f9547700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f409e930 0x7f44f41c2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:32.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.958+0000 7f44f9547700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44f409f240 0x7f44f41c3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:32.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.958+0000 7f44f9547700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44f409f650 0x7f44f41c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:32.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.958+0000 7f44f9547700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f41a7420 con 0x7f44f409e930 2023-10-03T10:17:32.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.958+0000 7f44f9547700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f41a6960 con 0x7f44f409f240 2023-10-03T10:17:32.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.958+0000 7f44f9547700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f44f41c3980 con 0x7f44f409f650 2023-10-03T10:17:32.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.958+0000 7f44f37fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44f409f650 0x7f44f41c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:32.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.958+0000 7f44f2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f409e930 0x7f44f41c2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:32.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.958+0000 7f44f37fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44f409f650 0x7f44f41c5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:40206/0 (socket says 172.21.15.3:40206) 2023-10-03T10:17:32.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.958+0000 7f44f2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f409e930 0x7f44f41c2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40684/0 (socket says 172.21.15.3:40684) 2023-10-03T10:17:32.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.958+0000 7f44f37fe700 1 -- 172.21.15.3:0/761209405 learned_addr learned my addr 172.21.15.3:0/761209405 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:32.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.958+0000 7f44f37fe700 1 -- 172.21.15.3:0/761209405 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44f409f240 msgr2=0x7f44f41c3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:32.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.958+0000 7f44f37fe700 1 --2- 172.21.15.3:0/761209405 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44f409f240 0x7f44f41c3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:32.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.958+0000 7f44f37fe700 1 -- 172.21.15.3:0/761209405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f409e930 msgr2=0x7f44f41c2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:32.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.958+0000 7f44f37fe700 1 --2- 172.21.15.3:0/761209405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f409e930 0x7f44f41c2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:32.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.958+0000 7f44f37fe700 1 -- 172.21.15.3:0/761209405 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e4009550 con 0x7f44f409f650 2023-10-03T10:17:32.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.958+0000 7f44f2ffd700 1 --2- 172.21.15.3:0/761209405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f409e930 0x7f44f41c2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:32.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.959+0000 7f44f37fe700 1 --2- 172.21.15.3:0/761209405 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44f409f650 0x7f44f41c5f20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f44ec014040 tx=0x7f44ec004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:32.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.959+0000 7f44db7fe700 1 -- 172.21.15.3:0/761209405 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec012430 con 0x7f44f409f650 2023-10-03T10:17:32.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.959+0000 7f44db7fe700 1 -- 172.21.15.3:0/761209405 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ec020070 con 0x7f44f409f650 2023-10-03T10:17:32.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.959+0000 7f44f9547700 1 -- 172.21.15.3:0/761209405 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44ec009a70 con 0x7f44f409f650 2023-10-03T10:17:32.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.959+0000 7f44db7fe700 1 -- 172.21.15.3:0/761209405 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec003ea0 con 0x7f44f409f650 2023-10-03T10:17:32.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.959+0000 7f44f9547700 1 -- 172.21.15.3:0/761209405 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f40a0190 con 0x7f44f409f650 2023-10-03T10:17:32.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.960+0000 7f44db7fe700 1 -- 172.21.15.3:0/761209405 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f44ec004280 con 0x7f44f409f650 2023-10-03T10:17:32.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.960+0000 7f44f9547700 1 -- 172.21.15.3:0/761209405 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44f40a3630 con 0x7f44f409f650 2023-10-03T10:17:32.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.960+0000 7f44db7fe700 1 -- 172.21.15.3:0/761209405 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f44ec022070 con 0x7f44f409f650 2023-10-03T10:17:32.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.965+0000 7f44db7fe700 1 -- 172.21.15.3:0/761209405 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f44ec01c400 con 0x7f44f409f650 2023-10-03T10:17:32.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.999+0000 7f9c657d9700 1 Processor -- start 2023-10-03T10:17:32.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.999+0000 7f9c657d9700 1 -- start start 2023-10-03T10:17:33.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.999+0000 7f9c657d9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c60133960 0x7f9c60133d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:33.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.999+0000 7f9c657d9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c6009de50 0x7f9c6009e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:33.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.999+0000 7f9c657d9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c60133550 con 0x7f9c6009e7c0 2023-10-03T10:17:33.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.999+0000 7f9c657d9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6006fab0 con 0x7f9c6009de50 2023-10-03T10:17:33.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.999+0000 7f9c657d9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c60071e20 con 0x7f9c60133960 2023-10-03T10:17:33.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.999+0000 7f9c5e7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c6009de50 0x7f9c6009e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:33.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:32.999+0000 7f9c5f7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9c6009e7c0 0x7f9c6013a7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45500/0 (socket says 172.21.15.3:45500) 2023-10-03T10:17:33.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.000+0000 7f9c5f7fe700 1 -- 172.21.15.3:0/3175056337 learned_addr learned my addr 172.21.15.3:0/3175056337 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:33.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.000+0000 7f9c5effd700 1 --2- 172.21.15.3:0/3175056337 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c60133960 0x7f9c60133d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:33.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.000+0000 7f9c5dffb700 1 -- 172.21.15.3:0/3175056337 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 318284643 0 0) 0x7f9c60133550 con 0x7f9c6009e7c0 2023-10-03T10:17:33.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.000+0000 7f9c5effd700 1 -- 172.21.15.3:0/3175056337 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c6009de50 msgr2=0x7f9c6009e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:33.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.000+0000 7f9c5effd700 1 --2- 172.21.15.3:0/3175056337 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c6009de50 0x7f9c6009e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:33.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.000+0000 7f9c5effd700 1 -- 172.21.15.3:0/3175056337 >> v1:172.21.15.3:6789/0 conn(0x7f9c6009e7c0 legacy=0x7f9c6013a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:33.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.000+0000 7f9c5effd700 1 -- 172.21.15.3:0/3175056337 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6009ec60 con 0x7f9c60133960 2023-10-03T10:17:33.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.000+0000 7f9c5e7fc700 1 --2- 172.21.15.3:0/3175056337 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c6009de50 0x7f9c6009e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:33.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.000+0000 7f9c5effd700 1 --2- 172.21.15.3:0/3175056337 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c60133960 0x7f9c60133d30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9c500098d0 tx=0x7f9c50009be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f605c10e0f77c68a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:33.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.000+0000 7f9c5dffb700 1 -- 172.21.15.3:0/3175056337 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c50011830 con 0x7f9c60133960 2023-10-03T10:17:33.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.000+0000 7f9c5dffb700 1 -- 172.21.15.3:0/3175056337 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c50011990 con 0x7f9c60133960 2023-10-03T10:17:33.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.000+0000 7f9c5dffb700 1 -- 172.21.15.3:0/3175056337 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c50011d40 con 0x7f9c60133960 2023-10-03T10:17:33.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.001+0000 7f9c657d9700 1 -- 172.21.15.3:0/3175056337 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c60133960 msgr2=0x7f9c60133d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:33.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.001+0000 7f9c657d9700 1 --2- 172.21.15.3:0/3175056337 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c60133960 0x7f9c60133d30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9c500098d0 tx=0x7f9c50009be0 comp rx=0 tx=0).stop 2023-10-03T10:17:33.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.001+0000 7f9c657d9700 1 -- 172.21.15.3:0/3175056337 shutdown_connections 2023-10-03T10:17:33.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.001+0000 7f9c657d9700 1 --2- 172.21.15.3:0/3175056337 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c60133960 0x7f9c60133d30 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:33.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.001+0000 7f9c657d9700 1 --2- 172.21.15.3:0/3175056337 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c6009de50 0x7f9c6009e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:33.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.001+0000 7f9c657d9700 1 -- 172.21.15.3:0/3175056337 >> 172.21.15.3:0/3175056337 conn(0x7f9c600a2610 msgr2=0x7f9c600a2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:33.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.001+0000 7f9c657d9700 1 -- 172.21.15.3:0/3175056337 shutdown_connections 2023-10-03T10:17:33.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.001+0000 7f9c657d9700 1 -- 172.21.15.3:0/3175056337 wait complete. 2023-10-03T10:17:33.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.002+0000 7f9c657d9700 1 Processor -- start 2023-10-03T10:17:33.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.002+0000 7f9c657d9700 1 -- start start 2023-10-03T10:17:33.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.003+0000 7f9c657d9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c6009de50 0x7f9c6011b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:33.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.003+0000 7f9c657d9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c6009e7c0 0x7f9c6011ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:33.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.003+0000 7f9c657d9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c60133960 0x7f9c6011bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:33.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.003+0000 7f9c657d9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6011c9d0 con 0x7f9c6009e7c0 2023-10-03T10:17:33.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.003+0000 7f9c657d9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6011d130 con 0x7f9c60133960 2023-10-03T10:17:33.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.003+0000 7f9c657d9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6011e380 con 0x7f9c6009de50 2023-10-03T10:17:33.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.003+0000 7f9c5effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c6009de50 0x7f9c6011b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:33.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.003+0000 7f9c5effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c6009de50 0x7f9c6011b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:40218/0 (socket says 172.21.15.3:40218) 2023-10-03T10:17:33.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.003+0000 7f9c5effd700 1 -- 172.21.15.3:0/2136867262 learned_addr learned my addr 172.21.15.3:0/2136867262 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:33.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.003+0000 7f9c5e7fc700 1 --2- 172.21.15.3:0/2136867262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c6009e7c0 0x7f9c6011ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:33.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.003+0000 7f9c5f7fe700 1 --2- 172.21.15.3:0/2136867262 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c60133960 0x7f9c6011bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:33.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.003+0000 7f9c5effd700 1 -- 172.21.15.3:0/2136867262 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c60133960 msgr2=0x7f9c6011bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:33.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.003+0000 7f9c5effd700 1 --2- 172.21.15.3:0/2136867262 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c60133960 0x7f9c6011bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:33.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.003+0000 7f9c5effd700 1 -- 172.21.15.3:0/2136867262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c6009e7c0 msgr2=0x7f9c6011ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:33.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.003+0000 7f9c5effd700 1 --2- 172.21.15.3:0/2136867262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c6009e7c0 0x7f9c6011ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:33.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.003+0000 7f9c5effd700 1 -- 172.21.15.3:0/2136867262 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c540094a0 con 0x7f9c6009de50 2023-10-03T10:17:33.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.003+0000 7f9c5f7fe700 1 --2- 172.21.15.3:0/2136867262 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c60133960 0x7f9c6011bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:17:33.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.004+0000 7f9c5effd700 1 --2- 172.21.15.3:0/2136867262 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c6009de50 0x7f9c6011b520 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9c500090b0 tx=0x7f9c50004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:33.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.004+0000 7f9c477fe700 1 -- 172.21.15.3:0/2136867262 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c50004450 con 0x7f9c6009de50 2023-10-03T10:17:33.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.004+0000 7f9c477fe700 1 -- 172.21.15.3:0/2136867262 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c500045b0 con 0x7f9c6009de50 2023-10-03T10:17:33.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.004+0000 7f9c657d9700 1 -- 172.21.15.3:0/2136867262 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c500094a0 con 0x7f9c6009de50 2023-10-03T10:17:33.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.004+0000 7f9c477fe700 1 -- 172.21.15.3:0/2136867262 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c500038e0 con 0x7f9c6009de50 2023-10-03T10:17:33.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.004+0000 7f9c657d9700 1 -- 172.21.15.3:0/2136867262 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c60131cd0 con 0x7f9c6009de50 2023-10-03T10:17:33.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.005+0000 7f9c477fe700 1 -- 172.21.15.3:0/2136867262 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9c50003a40 con 0x7f9c6009de50 2023-10-03T10:17:33.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.005+0000 7f9c477fe700 1 -- 172.21.15.3:0/2136867262 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9c50040a30 con 0x7f9c6009de50 2023-10-03T10:17:33.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.005+0000 7f9c657d9700 1 -- 172.21.15.3:0/2136867262 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c600a1cb0 con 0x7f9c6009de50 2023-10-03T10:17:33.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.010+0000 7f9c477fe700 1 -- 172.21.15.3:0/2136867262 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9c50015080 con 0x7f9c6009de50 2023-10-03T10:17:33.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.168+0000 7fc26b9be700 1 -- 172.21.15.3:0/2463645482 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc264074ea0 con 0x7fc26409e410 2023-10-03T10:17:33.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.168+0000 7fc259ffb700 1 -- 172.21.15.3:0/2463645482 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc25c00fe60 con 0x7fc26409e410 2023-10-03T10:17:33.169 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:33.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.171+0000 7fc26b9be700 1 -- 172.21.15.3:0/2463645482 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc26409e410 msgr2=0x7fc26411fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:33.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.171+0000 7fc26b9be700 1 --2- 172.21.15.3:0/2463645482 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc26409e410 0x7fc26411fc40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc25c0021c0 tx=0x7fc25c0021f0 comp rx=0 tx=0).stop 2023-10-03T10:17:33.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.171+0000 7fc26b9be700 1 -- 172.21.15.3:0/2463645482 shutdown_connections 2023-10-03T10:17:33.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.171+0000 7fc26b9be700 1 --2- 172.21.15.3:0/2463645482 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc26409e410 0x7fc26411fc40 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:33.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.171+0000 7fc26b9be700 1 --2- 172.21.15.3:0/2463645482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc26409d6f0 0x7fc26411f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:33.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.171+0000 7fc26b9be700 1 --2- 172.21.15.3:0/2463645482 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc26409e000 0x7fc26411f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:33.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.171+0000 7fc26b9be700 1 -- 172.21.15.3:0/2463645482 >> 172.21.15.3:0/2463645482 conn(0x7fc264066f50 msgr2=0x7fc264070120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:33.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.171+0000 7fc26b9be700 1 -- 172.21.15.3:0/2463645482 shutdown_connections 2023-10-03T10:17:33.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.171+0000 7fc26b9be700 1 -- 172.21.15.3:0/2463645482 wait complete. 2023-10-03T10:17:33.185 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:33.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.202+0000 7f44f9547700 1 -- 172.21.15.3:0/761209405 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f44f4074ea0 con 0x7f44f409f650 2023-10-03T10:17:33.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.202+0000 7f44db7fe700 1 -- 172.21.15.3:0/761209405 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f44ec02c030 con 0x7f44f409f650 2023-10-03T10:17:33.202 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:33.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.204+0000 7f44f9547700 1 -- 172.21.15.3:0/761209405 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44f409f650 msgr2=0x7f44f41c5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:33.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.204+0000 7f44f9547700 1 --2- 172.21.15.3:0/761209405 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44f409f650 0x7f44f41c5f20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f44ec014040 tx=0x7f44ec004ad0 comp rx=0 tx=0).stop 2023-10-03T10:17:33.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.204+0000 7f44f9547700 1 -- 172.21.15.3:0/761209405 shutdown_connections 2023-10-03T10:17:33.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.204+0000 7f44f9547700 1 --2- 172.21.15.3:0/761209405 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f44f409f650 0x7f44f41c5f20 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:33.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.204+0000 7f44f9547700 1 --2- 172.21.15.3:0/761209405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f409e930 0x7f44f41c2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:33.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.204+0000 7f44f9547700 1 --2- 172.21.15.3:0/761209405 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f44f409f240 0x7f44f41c3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:33.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.204+0000 7f44f9547700 1 -- 172.21.15.3:0/761209405 >> 172.21.15.3:0/761209405 conn(0x7f44f40a4210 msgr2=0x7f44f41b8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:33.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.204+0000 7f44f9547700 1 -- 172.21.15.3:0/761209405 shutdown_connections 2023-10-03T10:17:33.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.204+0000 7f44f9547700 1 -- 172.21.15.3:0/761209405 wait complete. 2023-10-03T10:17:33.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.216+0000 7f9c657d9700 1 -- 172.21.15.3:0/2136867262 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9c6006f420 con 0x7f9c6009de50 2023-10-03T10:17:33.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.217+0000 7f9c477fe700 1 -- 172.21.15.3:0/2136867262 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9c5002e970 con 0x7f9c6009de50 2023-10-03T10:17:33.217 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:33.218 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:33.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.219+0000 7f9c657d9700 1 -- 172.21.15.3:0/2136867262 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c6009de50 msgr2=0x7f9c6011b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:33.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.219+0000 7f9c657d9700 1 --2- 172.21.15.3:0/2136867262 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c6009de50 0x7f9c6011b520 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9c500090b0 tx=0x7f9c50004750 comp rx=0 tx=0).stop 2023-10-03T10:17:33.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.219+0000 7f9c657d9700 1 -- 172.21.15.3:0/2136867262 shutdown_connections 2023-10-03T10:17:33.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.219+0000 7f9c657d9700 1 --2- 172.21.15.3:0/2136867262 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c60133960 0x7f9c6011bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:33.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.219+0000 7f9c657d9700 1 --2- 172.21.15.3:0/2136867262 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c6009de50 0x7f9c6011b520 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:33.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.219+0000 7f9c657d9700 1 --2- 172.21.15.3:0/2136867262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c6009e7c0 0x7f9c6011ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:33.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.219+0000 7f9c657d9700 1 -- 172.21.15.3:0/2136867262 >> 172.21.15.3:0/2136867262 conn(0x7f9c600a2610 msgr2=0x7f9c60130530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:33.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.220+0000 7f9c657d9700 1 -- 172.21.15.3:0/2136867262 shutdown_connections 2023-10-03T10:17:33.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.220+0000 7f9c657d9700 1 -- 172.21.15.3:0/2136867262 wait complete. 2023-10-03T10:17:33.233 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:33.646 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:33.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.792+0000 7fbd78cc0700 1 Processor -- start 2023-10-03T10:17:33.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.792+0000 7fbd78cc0700 1 -- start start 2023-10-03T10:17:33.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.792+0000 7fbd78cc0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbd7409d6f0 0x7fbd7409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:33.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.792+0000 7fbd78cc0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409e000 0x7fbd74130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:33.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.792+0000 7fbd78cc0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7409e410 0x7fbd74130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:33.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.792+0000 7fbd78cc0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd7409f520 con 0x7fbd7409d6f0 2023-10-03T10:17:33.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.792+0000 7fbd78cc0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7409f130 con 0x7fbd7409e410 2023-10-03T10:17:33.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.792+0000 7fbd78cc0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7409e930 con 0x7fbd7409e000 2023-10-03T10:17:33.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.793+0000 7fbd7259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbd7409d6f0 0x7fbd7409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:33.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.793+0000 7fbd72d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7409e410 0x7fbd74130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:33.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.793+0000 7fbd71d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409e000 0x7fbd74130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:33.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.793+0000 7fbd71d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409e000 0x7fbd74130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53798/0 (socket says 172.21.15.3:53798) 2023-10-03T10:17:33.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.793+0000 7fbd72d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7409e410 0x7fbd74130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38614/0 (socket says 172.21.15.3:38614) 2023-10-03T10:17:33.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.793+0000 7fbd7259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbd7409d6f0 0x7fbd7409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39556/0 (socket says 172.21.15.3:39556) 2023-10-03T10:17:33.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.793+0000 7fbd71d9b700 1 -- 172.21.15.3:0/3434793323 learned_addr learned my addr 172.21.15.3:0/3434793323 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:33.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.793+0000 7fbd71d9b700 1 -- 172.21.15.3:0/3434793323 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7409e410 msgr2=0x7fbd74130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:33.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.793+0000 7fbd71d9b700 1 --2- 172.21.15.3:0/3434793323 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7409e410 0x7fbd74130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:33.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.793+0000 7fbd71d9b700 1 -- 172.21.15.3:0/3434793323 >> v2:172.21.15.3:3300/0 conn(0x7fbd7409d6f0 msgr2=0x7fbd7409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:33.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.793+0000 7fbd71d9b700 1 --2- 172.21.15.3:0/3434793323 >> v2:172.21.15.3:3300/0 conn(0x7fbd7409d6f0 0x7fbd7409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:33.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.793+0000 7fbd71d9b700 1 -- 172.21.15.3:0/3434793323 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7409b2d0 con 0x7fbd7409e000 2023-10-03T10:17:33.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.793+0000 7fbd72d9d700 1 --2- 172.21.15.3:0/3434793323 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7409e410 0x7fbd74130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:33.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.793+0000 7fbd71d9b700 1 --2- 172.21.15.3:0/3434793323 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409e000 0x7fbd74130660 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009920 tx=0x7fbd68009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=708936483e728623 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:33.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.793+0000 7fbd7159a700 1 -- 172.21.15.3:0/3434793323 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd68011830 con 0x7fbd7409e000 2023-10-03T10:17:33.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.793+0000 7fbd7159a700 1 -- 172.21.15.3:0/3434793323 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd68011990 con 0x7fbd7409e000 2023-10-03T10:17:33.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.793+0000 7fbd7159a700 1 -- 172.21.15.3:0/3434793323 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd68011d20 con 0x7fbd7409e000 2023-10-03T10:17:33.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.794+0000 7fbd78cc0700 1 -- 172.21.15.3:0/3434793323 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409e000 msgr2=0x7fbd74130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:33.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.794+0000 7fbd78cc0700 1 --2- 172.21.15.3:0/3434793323 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409e000 0x7fbd74130660 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009920 tx=0x7fbd68009c30 comp rx=0 tx=0).stop 2023-10-03T10:17:33.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.794+0000 7fbd78cc0700 1 -- 172.21.15.3:0/3434793323 shutdown_connections 2023-10-03T10:17:33.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.794+0000 7fbd78cc0700 1 --2- 172.21.15.3:0/3434793323 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7409e410 0x7fbd74130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:33.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.794+0000 7fbd78cc0700 1 --2- 172.21.15.3:0/3434793323 >> v2:172.21.15.3:3300/0 conn(0x7fbd7409d6f0 0x7fbd7409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:33.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.794+0000 7fbd78cc0700 1 --2- 172.21.15.3:0/3434793323 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409e000 0x7fbd74130660 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:33.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.794+0000 7fbd78cc0700 1 -- 172.21.15.3:0/3434793323 >> 172.21.15.3:0/3434793323 conn(0x7fbd74066f50 msgr2=0x7fbd7409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:33.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.794+0000 7fbd78cc0700 1 -- 172.21.15.3:0/3434793323 shutdown_connections 2023-10-03T10:17:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.794+0000 7fbd78cc0700 1 -- 172.21.15.3:0/3434793323 wait complete. 2023-10-03T10:17:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.795+0000 7fbd78cc0700 1 Processor -- start 2023-10-03T10:17:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.795+0000 7fbd78cc0700 1 -- start start 2023-10-03T10:17:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.796+0000 7fbd78cc0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7409d6f0 0x7fbd7406e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.796+0000 7fbd78cc0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409e000 0x7fbd7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.796+0000 7fbd78cc0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd7409e410 0x7fbd7406f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.796+0000 7fbd78cc0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7406fcd0 con 0x7fbd7409e410 2023-10-03T10:17:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.796+0000 7fbd78cc0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd74125640 con 0x7fbd7409d6f0 2023-10-03T10:17:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.796+0000 7fbd78cc0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd74125cf0 con 0x7fbd7409e000 2023-10-03T10:17:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.796+0000 7fbd7259c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7409d6f0 0x7fbd7406e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.796+0000 7fbd72d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd7409e410 0x7fbd7406f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.796+0000 7fbd71d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409e000 0x7fbd7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.796+0000 7fbd7259c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7409d6f0 0x7fbd7406e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38628/0 (socket says 172.21.15.3:38628) 2023-10-03T10:17:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.796+0000 7fbd71d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409e000 0x7fbd7406edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53802/0 (socket says 172.21.15.3:53802) 2023-10-03T10:17:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.796+0000 7fbd7259c700 1 -- 172.21.15.3:0/449954912 learned_addr learned my addr 172.21.15.3:0/449954912 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.796+0000 7fbd71d9b700 1 -- 172.21.15.3:0/449954912 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7409d6f0 msgr2=0x7fbd7406e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.796+0000 7fbd71d9b700 1 --2- 172.21.15.3:0/449954912 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7409d6f0 0x7fbd7406e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.796+0000 7fbd71d9b700 1 -- 172.21.15.3:0/449954912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd7409e410 msgr2=0x7fbd7406f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.796+0000 7fbd71d9b700 1 --2- 172.21.15.3:0/449954912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd7409e410 0x7fbd7406f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.796+0000 7fbd71d9b700 1 -- 172.21.15.3:0/449954912 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6c009a70 con 0x7fbd7409e000 2023-10-03T10:17:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.796+0000 7fbd72d9d700 1 --2- 172.21.15.3:0/449954912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd7409e410 0x7fbd7406f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.796+0000 7fbd71d9b700 1 --2- 172.21.15.3:0/449954912 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409e000 0x7fbd7406edb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbd68016000 tx=0x7fbd68004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.797+0000 7fbd62ffd700 1 -- 172.21.15.3:0/449954912 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd68004240 con 0x7fbd7409e000 2023-10-03T10:17:33.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.797+0000 7fbd62ffd700 1 -- 172.21.15.3:0/449954912 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6801f070 con 0x7fbd7409e000 2023-10-03T10:17:33.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.797+0000 7fbd62ffd700 1 -- 172.21.15.3:0/449954912 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd680045e0 con 0x7fbd7409e000 2023-10-03T10:17:33.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.797+0000 7fbd78cc0700 1 -- 172.21.15.3:0/449954912 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd68009550 con 0x7fbd7409e000 2023-10-03T10:17:33.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.797+0000 7fbd78cc0700 1 -- 172.21.15.3:0/449954912 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd741259c0 con 0x7fbd7409e000 2023-10-03T10:17:33.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.797+0000 7fbd62ffd700 1 -- 172.21.15.3:0/449954912 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbd68021030 con 0x7fbd7409e000 2023-10-03T10:17:33.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.798+0000 7fbd62ffd700 1 -- 172.21.15.3:0/449954912 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fbd68040680 con 0x7fbd7409e000 2023-10-03T10:17:33.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.798+0000 7fbd78cc0700 1 -- 172.21.15.3:0/449954912 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd741303d0 con 0x7fbd7409e000 2023-10-03T10:17:33.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:33.803+0000 7fbd62ffd700 1 -- 172.21.15.3:0/449954912 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbd680093b0 con 0x7fbd7409e000 2023-10-03T10:17:34.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.022+0000 7fbd78cc0700 1 -- 172.21.15.3:0/449954912 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbd74130780 con 0x7fbd7409e000 2023-10-03T10:17:34.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.023+0000 7fbd62ffd700 1 -- 172.21.15.3:0/449954912 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbd68015020 con 0x7fbd7409e000 2023-10-03T10:17:34.023 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:34.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.025+0000 7fbd78cc0700 1 -- 172.21.15.3:0/449954912 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409e000 msgr2=0x7fbd7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:34.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.025+0000 7fbd78cc0700 1 --2- 172.21.15.3:0/449954912 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409e000 0x7fbd7406edb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbd68016000 tx=0x7fbd68004750 comp rx=0 tx=0).stop 2023-10-03T10:17:34.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.025+0000 7fbd78cc0700 1 -- 172.21.15.3:0/449954912 shutdown_connections 2023-10-03T10:17:34.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.025+0000 7fbd78cc0700 1 --2- 172.21.15.3:0/449954912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd7409e410 0x7fbd7406f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.025+0000 7fbd78cc0700 1 --2- 172.21.15.3:0/449954912 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd7409d6f0 0x7fbd7406e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.025+0000 7fbd78cc0700 1 --2- 172.21.15.3:0/449954912 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409e000 0x7fbd7406edb0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.025+0000 7fbd78cc0700 1 -- 172.21.15.3:0/449954912 >> 172.21.15.3:0/449954912 conn(0x7fbd74066f50 msgr2=0x7fbd740713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:34.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.025+0000 7fbd78cc0700 1 -- 172.21.15.3:0/449954912 shutdown_connections 2023-10-03T10:17:34.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.025+0000 7fbd78cc0700 1 -- 172.21.15.3:0/449954912 wait complete. 2023-10-03T10:17:34.039 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:34.186 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:34.219 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:34.234 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:34.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.337+0000 7f9cccc54700 1 Processor -- start 2023-10-03T10:17:34.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.337+0000 7f9cccc54700 1 -- start start 2023-10-03T10:17:34.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.338+0000 7f9cccc54700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9cc8130750 0x7f9cc8130b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:34.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.338+0000 7f9cccc54700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9cc809e250 0x7f9cc81384c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:34.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.338+0000 7f9cccc54700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cc809c190 con 0x7f9cc809d6f0 2023-10-03T10:17:34.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.338+0000 7f9cccc54700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc809b090 con 0x7f9cc8130750 2023-10-03T10:17:34.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.338+0000 7f9cccc54700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc809a6c0 con 0x7f9cc809e250 2023-10-03T10:17:34.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.338+0000 7f9cc659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9cc8130750 0x7f9cc8130b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:34.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.338+0000 7f9cc5d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9cc809d6f0 0x7f9cc809db40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:32956/0 (socket says 172.21.15.3:32956) 2023-10-03T10:17:34.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.338+0000 7f9cc5d9b700 1 -- 172.21.15.3:0/408867561 learned_addr learned my addr 172.21.15.3:0/408867561 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:34.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.338+0000 7f9cc6d9d700 1 --2- 172.21.15.3:0/408867561 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9cc809e250 0x7f9cc81384c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:34.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.339+0000 7f9cc559a700 1 -- 172.21.15.3:0/408867561 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2833264752 0 0) 0x7f9cc809c190 con 0x7f9cc809d6f0 2023-10-03T10:17:34.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.339+0000 7f9cc559a700 1 -- 172.21.15.3:0/408867561 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cb00035b0 con 0x7f9cc809d6f0 2023-10-03T10:17:34.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.339+0000 7f9cc659c700 1 -- 172.21.15.3:0/408867561 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9cc809e250 msgr2=0x7f9cc81384c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:34.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.339+0000 7f9cc659c700 1 --2- 172.21.15.3:0/408867561 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9cc809e250 0x7f9cc81384c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.339+0000 7f9cc659c700 1 -- 172.21.15.3:0/408867561 >> v1:172.21.15.3:6789/0 conn(0x7f9cc809d6f0 legacy=0x7f9cc809db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:34.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.339+0000 7f9cc659c700 1 -- 172.21.15.3:0/408867561 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc8131060 con 0x7f9cc8130750 2023-10-03T10:17:34.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.339+0000 7f9cc659c700 1 --2- 172.21.15.3:0/408867561 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9cc8130750 0x7f9cc8130b20 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8009870 tx=0x7f9cb8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30f2cd346531e42f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:34.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.339+0000 7f9cc559a700 1 -- 172.21.15.3:0/408867561 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8011830 con 0x7f9cc8130750 2023-10-03T10:17:34.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.339+0000 7f9cc559a700 1 -- 172.21.15.3:0/408867561 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb8011990 con 0x7f9cc8130750 2023-10-03T10:17:34.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.339+0000 7f9cc559a700 1 -- 172.21.15.3:0/408867561 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8011d20 con 0x7f9cc8130750 2023-10-03T10:17:34.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.340+0000 7f9cccc54700 1 -- 172.21.15.3:0/408867561 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9cc8130750 msgr2=0x7f9cc8130b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:34.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.340+0000 7f9cccc54700 1 --2- 172.21.15.3:0/408867561 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9cc8130750 0x7f9cc8130b20 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8009870 tx=0x7f9cb8009b80 comp rx=0 tx=0).stop 2023-10-03T10:17:34.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.340+0000 7f9cccc54700 1 -- 172.21.15.3:0/408867561 shutdown_connections 2023-10-03T10:17:34.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.340+0000 7f9cccc54700 1 --2- 172.21.15.3:0/408867561 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9cc8130750 0x7f9cc8130b20 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.340+0000 7f9cccc54700 1 --2- 172.21.15.3:0/408867561 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9cc809e250 0x7f9cc81384c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.340+0000 7f9cccc54700 1 -- 172.21.15.3:0/408867561 >> 172.21.15.3:0/408867561 conn(0x7f9cc809f470 msgr2=0x7f9cc80a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:34.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.340+0000 7f9cccc54700 1 -- 172.21.15.3:0/408867561 shutdown_connections 2023-10-03T10:17:34.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.340+0000 7f9cccc54700 1 -- 172.21.15.3:0/408867561 wait complete. 2023-10-03T10:17:34.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.341+0000 7f9cccc54700 1 Processor -- start 2023-10-03T10:17:34.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.341+0000 7f9cccc54700 1 -- start start 2023-10-03T10:17:34.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.342+0000 7f9cccc54700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9cc809d6f0 0x7f9cc812cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:34.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.342+0000 7f9cccc54700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9cc809e250 0x7f9cc812d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:34.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.342+0000 7f9cccc54700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cc8130750 0x7f9cc812d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:34.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.342+0000 7f9cccc54700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc808b150 con 0x7f9cc8130750 2023-10-03T10:17:34.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.342+0000 7f9cccc54700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc809b320 con 0x7f9cc809e250 2023-10-03T10:17:34.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.342+0000 7f9cccc54700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc808b640 con 0x7f9cc809d6f0 2023-10-03T10:17:34.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.342+0000 7f9cc5d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9cc809e250 0x7f9cc812d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:34.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.342+0000 7f9cc5d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9cc809e250 0x7f9cc812d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38648/0 (socket says 172.21.15.3:38648) 2023-10-03T10:17:34.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.342+0000 7f9cc5d9b700 1 -- 172.21.15.3:0/3746968473 learned_addr learned my addr 172.21.15.3:0/3746968473 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:34.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.342+0000 7f9cc6d9d700 1 --2- 172.21.15.3:0/3746968473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cc8130750 0x7f9cc812d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:34.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.342+0000 7f9cc659c700 1 --2- 172.21.15.3:0/3746968473 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9cc809d6f0 0x7f9cc812cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:34.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.342+0000 7f9cc5d9b700 1 -- 172.21.15.3:0/3746968473 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9cc809d6f0 msgr2=0x7f9cc812cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:34.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.342+0000 7f9cc5d9b700 1 --2- 172.21.15.3:0/3746968473 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9cc809d6f0 0x7f9cc812cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.342+0000 7f9cc5d9b700 1 -- 172.21.15.3:0/3746968473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cc8130750 msgr2=0x7f9cc812d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:34.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.342+0000 7f9cc5d9b700 1 --2- 172.21.15.3:0/3746968473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cc8130750 0x7f9cc812d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.342+0000 7f9cc5d9b700 1 -- 172.21.15.3:0/3746968473 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb80094a0 con 0x7f9cc809e250 2023-10-03T10:17:34.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.343+0000 7f9cc5d9b700 1 --2- 172.21.15.3:0/3746968473 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9cc809e250 0x7f9cc812d1a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc005910 tx=0x7f9cbc00c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:34.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.343+0000 7f9cb6ffd700 1 -- 172.21.15.3:0/3746968473 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc005d70 con 0x7f9cc809e250 2023-10-03T10:17:34.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.343+0000 7f9cb6ffd700 1 -- 172.21.15.3:0/3746968473 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cbc00ce80 con 0x7f9cc809e250 2023-10-03T10:17:34.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.343+0000 7f9cb6ffd700 1 -- 172.21.15.3:0/3746968473 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc0072f0 con 0x7f9cc809e250 2023-10-03T10:17:34.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.343+0000 7f9cccc54700 1 -- 172.21.15.3:0/3746968473 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc80a0170 con 0x7f9cc809e250 2023-10-03T10:17:34.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.343+0000 7f9cccc54700 1 -- 172.21.15.3:0/3746968473 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc812efc0 con 0x7f9cc809e250 2023-10-03T10:17:34.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.344+0000 7f9cccc54700 1 -- 172.21.15.3:0/3746968473 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cc80747f0 con 0x7f9cc809e250 2023-10-03T10:17:34.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.344+0000 7f9cb6ffd700 1 -- 172.21.15.3:0/3746968473 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9cbc007450 con 0x7f9cc809e250 2023-10-03T10:17:34.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.344+0000 7f9cb6ffd700 1 -- 172.21.15.3:0/3746968473 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9cbc03eb10 con 0x7f9cc809e250 2023-10-03T10:17:34.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.349+0000 7f9cb6ffd700 1 -- 172.21.15.3:0/3746968473 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9cbc019b20 con 0x7f9cc809e250 2023-10-03T10:17:34.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.376+0000 7ff6a8376700 1 Processor -- start 2023-10-03T10:17:34.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.376+0000 7ff6a8376700 1 -- start start 2023-10-03T10:17:34.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.376+0000 7ff6a8376700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff6a00a0a30 0x7ff6a00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:34.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.376+0000 7ff6a8376700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff6a0138d80 0x7ff6a013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:34.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.377+0000 7ff6a8376700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a00a2a70 con 0x7ff6a009d630 2023-10-03T10:17:34.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.377+0000 7ff6a8376700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a00a01d0 con 0x7ff6a00a0a30 2023-10-03T10:17:34.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.377+0000 7ff6a8376700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a009e2b0 con 0x7ff6a0138d80 2023-10-03T10:17:34.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.377+0000 7ff6a6112700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff6a00a0a30 0x7ff6a00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:34.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.377+0000 7ff6a6112700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff6a00a0a30 0x7ff6a00a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39576/0 (socket says 172.21.15.3:39576) 2023-10-03T10:17:34.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.377+0000 7ff6a6112700 1 -- 172.21.15.3:0/1252982899 learned_addr learned my addr 172.21.15.3:0/1252982899 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:34.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.377+0000 7ff6a6913700 1 --2- 172.21.15.3:0/1252982899 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff6a0138d80 0x7ff6a013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:34.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.377+0000 7ff6a6112700 1 -- 172.21.15.3:0/1252982899 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff6a0138d80 msgr2=0x7ff6a013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:34.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.377+0000 7ff6a6112700 1 --2- 172.21.15.3:0/1252982899 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff6a0138d80 0x7ff6a013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.377+0000 7ff6a6112700 1 -- 172.21.15.3:0/1252982899 >> v1:172.21.15.3:6789/0 conn(0x7ff6a009d630 legacy=0x7ff6a00a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:34.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.377+0000 7ff6a6112700 1 -- 172.21.15.3:0/1252982899 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a009dc50 con 0x7ff6a00a0a30 2023-10-03T10:17:34.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.377+0000 7ff6a6913700 1 --2- 172.21.15.3:0/1252982899 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff6a0138d80 0x7ff6a013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:34.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.377+0000 7ff6a6112700 1 --2- 172.21.15.3:0/1252982899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a00a0a30 0x7ff6a00a0e00 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7ff69c0098f0 tx=0x7ff69c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103827b7ec8ef57a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:34.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.378+0000 7ff6a5110700 1 -- 172.21.15.3:0/1252982899 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c011830 con 0x7ff6a00a0a30 2023-10-03T10:17:34.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.378+0000 7ff6a5110700 1 -- 172.21.15.3:0/1252982899 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69c011990 con 0x7ff6a00a0a30 2023-10-03T10:17:34.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.378+0000 7ff6a5110700 1 -- 172.21.15.3:0/1252982899 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c011d20 con 0x7ff6a00a0a30 2023-10-03T10:17:34.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.378+0000 7ff6a8376700 1 -- 172.21.15.3:0/1252982899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a00a0a30 msgr2=0x7ff6a00a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:34.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.378+0000 7ff6a8376700 1 --2- 172.21.15.3:0/1252982899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a00a0a30 0x7ff6a00a0e00 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7ff69c0098f0 tx=0x7ff69c009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:34.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.378+0000 7ff6a8376700 1 -- 172.21.15.3:0/1252982899 shutdown_connections 2023-10-03T10:17:34.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.378+0000 7ff6a8376700 1 --2- 172.21.15.3:0/1252982899 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff6a0138d80 0x7ff6a013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.378+0000 7ff6a8376700 1 --2- 172.21.15.3:0/1252982899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a00a0a30 0x7ff6a00a0e00 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.378+0000 7ff6a8376700 1 -- 172.21.15.3:0/1252982899 >> 172.21.15.3:0/1252982899 conn(0x7ff6a00a3050 msgr2=0x7ff6a00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:34.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.378+0000 7ff6a8376700 1 -- 172.21.15.3:0/1252982899 shutdown_connections 2023-10-03T10:17:34.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.379+0000 7ff6a8376700 1 -- 172.21.15.3:0/1252982899 wait complete. 2023-10-03T10:17:34.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.380+0000 7ff6a8376700 1 Processor -- start 2023-10-03T10:17:34.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.380+0000 7ff6a8376700 1 -- start start 2023-10-03T10:17:34.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.380+0000 7ff6a8376700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff6a009d630 0x7ff6a0120700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:34.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.380+0000 7ff6a8376700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a00a0a30 0x7ff6a0120c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:34.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.380+0000 7ff6a8376700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff6a0138d80 0x7ff6a0121180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:34.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.380+0000 7ff6a8376700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a0123110 con 0x7ff6a00a0a30 2023-10-03T10:17:34.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.380+0000 7ff6a8376700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a0122be0 con 0x7ff6a009d630 2023-10-03T10:17:34.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.380+0000 7ff6a8376700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a0122310 con 0x7ff6a0138d80 2023-10-03T10:17:34.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.380+0000 7ff6a5911700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a00a0a30 0x7ff6a0120c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:34.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.380+0000 7ff6a5911700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a00a0a30 0x7ff6a0120c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39582/0 (socket says 172.21.15.3:39582) 2023-10-03T10:17:34.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.380+0000 7ff6a5911700 1 -- 172.21.15.3:0/1201273532 learned_addr learned my addr 172.21.15.3:0/1201273532 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:34.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.380+0000 7ff6a6112700 1 --2- 172.21.15.3:0/1201273532 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff6a009d630 0x7ff6a0120700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:34.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.380+0000 7ff6a6913700 1 --2- 172.21.15.3:0/1201273532 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff6a0138d80 0x7ff6a0121180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:34.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.381+0000 7ff6a5911700 1 -- 172.21.15.3:0/1201273532 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff6a0138d80 msgr2=0x7ff6a0121180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:34.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.381+0000 7ff6a5911700 1 --2- 172.21.15.3:0/1201273532 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff6a0138d80 0x7ff6a0121180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.381+0000 7ff6a5911700 1 -- 172.21.15.3:0/1201273532 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff6a009d630 msgr2=0x7ff6a0120700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:34.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.381+0000 7ff6a5911700 1 --2- 172.21.15.3:0/1201273532 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff6a009d630 0x7ff6a0120700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.381+0000 7ff6a5911700 1 -- 172.21.15.3:0/1201273532 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff698009a70 con 0x7ff6a00a0a30 2023-10-03T10:17:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.381+0000 7ff6a5911700 1 --2- 172.21.15.3:0/1201273532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a00a0a30 0x7ff6a0120c40 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7ff69000c510 tx=0x7ff69000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.381+0000 7ff696ffd700 1 -- 172.21.15.3:0/1201273532 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6900052d0 con 0x7ff6a00a0a30 2023-10-03T10:17:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.381+0000 7ff696ffd700 1 -- 172.21.15.3:0/1201273532 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff690005430 con 0x7ff6a00a0a30 2023-10-03T10:17:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.381+0000 7ff6a8376700 1 -- 172.21.15.3:0/1201273532 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff69c009550 con 0x7ff6a00a0a30 2023-10-03T10:17:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.381+0000 7ff696ffd700 1 -- 172.21.15.3:0/1201273532 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6900038c0 con 0x7ff6a00a0a30 2023-10-03T10:17:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.381+0000 7ff6a8376700 1 -- 172.21.15.3:0/1201273532 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a00a4ec0 con 0x7ff6a00a0a30 2023-10-03T10:17:34.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.382+0000 7ff696ffd700 1 -- 172.21.15.3:0/1201273532 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff690003a20 con 0x7ff6a00a0a30 2023-10-03T10:17:34.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.382+0000 7ff696ffd700 1 -- 172.21.15.3:0/1201273532 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff69003ead0 con 0x7ff6a00a0a30 2023-10-03T10:17:34.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.382+0000 7ff6a8376700 1 -- 172.21.15.3:0/1201273532 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a00a0470 con 0x7ff6a00a0a30 2023-10-03T10:17:34.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.387+0000 7ff696ffd700 1 -- 172.21.15.3:0/1201273532 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff690017080 con 0x7ff6a00a0a30 2023-10-03T10:17:34.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.419+0000 7f14a0679700 1 Processor -- start 2023-10-03T10:17:34.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.419+0000 7f14a0679700 1 -- start start 2023-10-03T10:17:34.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.420+0000 7f14a0679700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f14980a0a30 0x7f14980a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:34.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.420+0000 7f14a0679700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1498138d80 0x7f149813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:34.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.420+0000 7f14a0679700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14980a2a70 con 0x7f149809d630 2023-10-03T10:17:34.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.420+0000 7f14a0679700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14980a01d0 con 0x7f1498138d80 2023-10-03T10:17:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.420+0000 7f14a0679700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f149809e2b0 con 0x7f14980a0a30 2023-10-03T10:17:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.420+0000 7f149e415700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f14980a0a30 0x7f14980a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.420+0000 7f149e415700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f14980a0a30 0x7f14980a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38678/0 (socket says 172.21.15.3:38678) 2023-10-03T10:17:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.420+0000 7f149e415700 1 -- 172.21.15.3:0/3818718274 learned_addr learned my addr 172.21.15.3:0/3818718274 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.420+0000 7f149ec16700 1 --2- 172.21.15.3:0/3818718274 >> v2:172.21.15.3:3300/0 conn(0x7f1498138d80 0x7f149813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.421+0000 7f149e415700 1 -- 172.21.15.3:0/3818718274 >> v2:172.21.15.3:3300/0 conn(0x7f1498138d80 msgr2=0x7f149813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.421+0000 7f149e415700 1 --2- 172.21.15.3:0/3818718274 >> v2:172.21.15.3:3300/0 conn(0x7f1498138d80 0x7f149813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.421+0000 7f149e415700 1 -- 172.21.15.3:0/3818718274 >> v1:172.21.15.3:6789/0 conn(0x7f149809d630 legacy=0x7f14980a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.421+0000 7f149e415700 1 -- 172.21.15.3:0/3818718274 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149809dc50 con 0x7f14980a0a30 2023-10-03T10:17:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.421+0000 7f149e415700 1 --2- 172.21.15.3:0/3818718274 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f14980a0a30 0x7f14980a0e00 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f14940098f0 tx=0x7f1494009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=378a86eef1180a5b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:34.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.421+0000 7f149d413700 1 -- 172.21.15.3:0/3818718274 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1494011830 con 0x7f14980a0a30 2023-10-03T10:17:34.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.421+0000 7f149d413700 1 -- 172.21.15.3:0/3818718274 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1494011990 con 0x7f14980a0a30 2023-10-03T10:17:34.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.421+0000 7f149d413700 1 -- 172.21.15.3:0/3818718274 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1494011d20 con 0x7f14980a0a30 2023-10-03T10:17:34.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.422+0000 7f14a0679700 1 -- 172.21.15.3:0/3818718274 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f14980a0a30 msgr2=0x7f14980a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:34.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.422+0000 7f14a0679700 1 --2- 172.21.15.3:0/3818718274 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f14980a0a30 0x7f14980a0e00 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f14940098f0 tx=0x7f1494009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:34.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.422+0000 7f14a0679700 1 -- 172.21.15.3:0/3818718274 shutdown_connections 2023-10-03T10:17:34.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.422+0000 7f14a0679700 1 --2- 172.21.15.3:0/3818718274 >> v2:172.21.15.3:3300/0 conn(0x7f1498138d80 0x7f149813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.422+0000 7f14a0679700 1 --2- 172.21.15.3:0/3818718274 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f14980a0a30 0x7f14980a0e00 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.422+0000 7f14a0679700 1 -- 172.21.15.3:0/3818718274 >> 172.21.15.3:0/3818718274 conn(0x7f14980a3050 msgr2=0x7f14980a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:34.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.422+0000 7f14a0679700 1 -- 172.21.15.3:0/3818718274 shutdown_connections 2023-10-03T10:17:34.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.422+0000 7f14a0679700 1 -- 172.21.15.3:0/3818718274 wait complete. 2023-10-03T10:17:34.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.423+0000 7f14a0679700 1 Processor -- start 2023-10-03T10:17:34.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.423+0000 7f14a0679700 1 -- start start 2023-10-03T10:17:34.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.424+0000 7f14a0679700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f149809d630 0x7f149811d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:34.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.424+0000 7f14a0679700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14980a0a30 0x7f149811d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:34.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.424+0000 7f14a0679700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1498138d80 0x7f149811def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:34.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.424+0000 7f14a0679700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1498120a90 con 0x7f14980a0a30 2023-10-03T10:17:34.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.424+0000 7f14a0679700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14980a0410 con 0x7f1498138d80 2023-10-03T10:17:34.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.424+0000 7f14a0679700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f149811f5e0 con 0x7f149809d630 2023-10-03T10:17:34.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.424+0000 7f149dc14700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14980a0a30 0x7f149811d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:34.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.424+0000 7f149dc14700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14980a0a30 0x7f149811d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39602/0 (socket says 172.21.15.3:39602) 2023-10-03T10:17:34.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.424+0000 7f149dc14700 1 -- 172.21.15.3:0/3079292834 learned_addr learned my addr 172.21.15.3:0/3079292834 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:34.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.424+0000 7f149e415700 1 --2- 172.21.15.3:0/3079292834 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f149809d630 0x7f149811d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:34.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.424+0000 7f149ec16700 1 --2- 172.21.15.3:0/3079292834 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1498138d80 0x7f149811def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:34.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.424+0000 7f149dc14700 1 -- 172.21.15.3:0/3079292834 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f149809d630 msgr2=0x7f149811d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:34.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.424+0000 7f149dc14700 1 --2- 172.21.15.3:0/3079292834 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f149809d630 0x7f149811d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.424+0000 7f149dc14700 1 -- 172.21.15.3:0/3079292834 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1498138d80 msgr2=0x7f149811def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:34.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.424+0000 7f149dc14700 1 --2- 172.21.15.3:0/3079292834 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1498138d80 0x7f149811def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.424+0000 7f149dc14700 1 -- 172.21.15.3:0/3079292834 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1494009550 con 0x7f14980a0a30 2023-10-03T10:17:34.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.424+0000 7f149ec16700 1 --2- 172.21.15.3:0/3079292834 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1498138d80 0x7f149811def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:34.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.424+0000 7f149e415700 1 --2- 172.21.15.3:0/3079292834 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f149809d630 0x7f149811d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:34.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.425+0000 7f149dc14700 1 --2- 172.21.15.3:0/3079292834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14980a0a30 0x7f149811d9b0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f148800c4b0 tx=0x7f148800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:34.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.425+0000 7f148effd700 1 -- 172.21.15.3:0/3079292834 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14880052d0 con 0x7f14980a0a30 2023-10-03T10:17:34.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.425+0000 7f148effd700 1 -- 172.21.15.3:0/3079292834 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1488005430 con 0x7f14980a0a30 2023-10-03T10:17:34.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.425+0000 7f14a0679700 1 -- 172.21.15.3:0/3079292834 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f149813b010 con 0x7f14980a0a30 2023-10-03T10:17:34.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.425+0000 7f148effd700 1 -- 172.21.15.3:0/3079292834 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14880038c0 con 0x7f14980a0a30 2023-10-03T10:17:34.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.425+0000 7f14a0679700 1 -- 172.21.15.3:0/3079292834 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1498120c70 con 0x7f14980a0a30 2023-10-03T10:17:34.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.426+0000 7f14a0679700 1 -- 172.21.15.3:0/3079292834 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14980747f0 con 0x7f14980a0a30 2023-10-03T10:17:34.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.426+0000 7f148effd700 1 -- 172.21.15.3:0/3079292834 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1488003a20 con 0x7f14980a0a30 2023-10-03T10:17:34.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.426+0000 7f148effd700 1 -- 172.21.15.3:0/3079292834 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f148803ea20 con 0x7f14980a0a30 2023-10-03T10:17:34.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.431+0000 7f148effd700 1 -- 172.21.15.3:0/3079292834 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1488018260 con 0x7f14980a0a30 2023-10-03T10:17:34.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.561+0000 7f9cccc54700 1 -- 172.21.15.3:0/3746968473 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9cc806b250 con 0x7f9cc809e250 2023-10-03T10:17:34.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.562+0000 7f9cb6ffd700 1 -- 172.21.15.3:0/3746968473 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9cbc017070 con 0x7f9cc809e250 2023-10-03T10:17:34.562 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:34.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.564+0000 7f9cccc54700 1 -- 172.21.15.3:0/3746968473 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9cc809e250 msgr2=0x7f9cc812d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:34.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.564+0000 7f9cccc54700 1 --2- 172.21.15.3:0/3746968473 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9cc809e250 0x7f9cc812d1a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc005910 tx=0x7f9cbc00c7b0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.564+0000 7f9cccc54700 1 -- 172.21.15.3:0/3746968473 shutdown_connections 2023-10-03T10:17:34.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.564+0000 7f9cccc54700 1 --2- 172.21.15.3:0/3746968473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cc8130750 0x7f9cc812d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.564+0000 7f9cccc54700 1 --2- 172.21.15.3:0/3746968473 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9cc809d6f0 0x7f9cc812cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.564+0000 7f9cccc54700 1 --2- 172.21.15.3:0/3746968473 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9cc809e250 0x7f9cc812d1a0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.564+0000 7f9cccc54700 1 -- 172.21.15.3:0/3746968473 >> 172.21.15.3:0/3746968473 conn(0x7f9cc809f470 msgr2=0x7f9cc8098700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:34.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.564+0000 7f9cccc54700 1 -- 172.21.15.3:0/3746968473 shutdown_connections 2023-10-03T10:17:34.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.564+0000 7f9cccc54700 1 -- 172.21.15.3:0/3746968473 wait complete. 2023-10-03T10:17:34.579 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:34.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.597+0000 7ff6a8376700 1 -- 172.21.15.3:0/1201273532 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff6a00747f0 con 0x7ff6a00a0a30 2023-10-03T10:17:34.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.598+0000 7ff696ffd700 1 -- 172.21.15.3:0/1201273532 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff690018260 con 0x7ff6a00a0a30 2023-10-03T10:17:34.598 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:34.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.600+0000 7ff6a8376700 1 -- 172.21.15.3:0/1201273532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a00a0a30 msgr2=0x7ff6a0120c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:34.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.600+0000 7ff6a8376700 1 --2- 172.21.15.3:0/1201273532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a00a0a30 0x7ff6a0120c40 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7ff69000c510 tx=0x7ff69000c820 comp rx=0 tx=0).stop 2023-10-03T10:17:34.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.600+0000 7ff6a8376700 1 -- 172.21.15.3:0/1201273532 shutdown_connections 2023-10-03T10:17:34.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.600+0000 7ff6a8376700 1 --2- 172.21.15.3:0/1201273532 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff6a0138d80 0x7ff6a0121180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.600+0000 7ff6a8376700 1 --2- 172.21.15.3:0/1201273532 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff6a009d630 0x7ff6a0120700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.600+0000 7ff6a8376700 1 --2- 172.21.15.3:0/1201273532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a00a0a30 0x7ff6a0120c40 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.600+0000 7ff6a8376700 1 -- 172.21.15.3:0/1201273532 >> 172.21.15.3:0/1201273532 conn(0x7ff6a00a3050 msgr2=0x7ff6a009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:34.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.600+0000 7ff6a8376700 1 -- 172.21.15.3:0/1201273532 shutdown_connections 2023-10-03T10:17:34.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.600+0000 7ff6a8376700 1 -- 172.21.15.3:0/1201273532 wait complete. 2023-10-03T10:17:34.614 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:34.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.638+0000 7f14a0679700 1 -- 172.21.15.3:0/3079292834 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f149811f830 con 0x7f14980a0a30 2023-10-03T10:17:34.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.638+0000 7f148effd700 1 -- 172.21.15.3:0/3079292834 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1488017070 con 0x7f14980a0a30 2023-10-03T10:17:34.639 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:34.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.641+0000 7f14a0679700 1 -- 172.21.15.3:0/3079292834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14980a0a30 msgr2=0x7f149811d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:34.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.641+0000 7f14a0679700 1 --2- 172.21.15.3:0/3079292834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14980a0a30 0x7f149811d9b0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f148800c4b0 tx=0x7f148800c7c0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.641+0000 7f14a0679700 1 -- 172.21.15.3:0/3079292834 shutdown_connections 2023-10-03T10:17:34.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.641+0000 7f14a0679700 1 --2- 172.21.15.3:0/3079292834 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1498138d80 0x7f149811def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.641+0000 7f14a0679700 1 --2- 172.21.15.3:0/3079292834 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f149809d630 0x7f149811d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.641+0000 7f14a0679700 1 --2- 172.21.15.3:0/3079292834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14980a0a30 0x7f149811d9b0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:34.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.641+0000 7f14a0679700 1 -- 172.21.15.3:0/3079292834 >> 172.21.15.3:0/3079292834 conn(0x7f14980a3050 msgr2=0x7f149809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:34.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.641+0000 7f14a0679700 1 -- 172.21.15.3:0/3079292834 shutdown_connections 2023-10-03T10:17:34.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:34.641+0000 7f14a0679700 1 -- 172.21.15.3:0/3079292834 wait complete. 2023-10-03T10:17:34.654 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:35.040 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:35.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.186+0000 7f8628432700 1 Processor -- start 2023-10-03T10:17:35.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.186+0000 7f8628432700 1 -- start start 2023-10-03T10:17:35.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.187+0000 7f8628432700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f862009f410 0x7f8620136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:35.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.187+0000 7f8628432700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8620136fe0 0x7f86201393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:35.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.187+0000 7f8628432700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8620070940 con 0x7f862009e930 2023-10-03T10:17:35.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.187+0000 7f8628432700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862006fd60 con 0x7f862009f410 2023-10-03T10:17:35.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.187+0000 7f8628432700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f862006fa80 con 0x7f8620136fe0 2023-10-03T10:17:35.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.187+0000 7f86261ce700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f862009e930 0x7f862009ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:32990/0 (socket says 172.21.15.3:32990) 2023-10-03T10:17:35.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.187+0000 7f86261ce700 1 -- 172.21.15.3:0/59315447 learned_addr learned my addr 172.21.15.3:0/59315447 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:35.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.187+0000 7f86259cd700 1 --2- 172.21.15.3:0/59315447 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f862009f410 0x7f8620136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:35.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.187+0000 7f86269cf700 1 --2- 172.21.15.3:0/59315447 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8620136fe0 0x7f86201393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:35.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.187+0000 7f86251cc700 1 -- 172.21.15.3:0/59315447 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1663653218 0 0) 0x7f8620070940 con 0x7f862009e930 2023-10-03T10:17:35.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.187+0000 7f86251cc700 1 -- 172.21.15.3:0/59315447 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f860c0035b0 con 0x7f862009e930 2023-10-03T10:17:35.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.187+0000 7f86259cd700 1 -- 172.21.15.3:0/59315447 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8620136fe0 msgr2=0x7f86201393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:35.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.187+0000 7f86259cd700 1 --2- 172.21.15.3:0/59315447 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8620136fe0 0x7f86201393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:35.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.187+0000 7f86259cd700 1 -- 172.21.15.3:0/59315447 >> v1:172.21.15.3:6789/0 conn(0x7f862009e930 legacy=0x7f862009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:35.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.187+0000 7f86259cd700 1 -- 172.21.15.3:0/59315447 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862009fb60 con 0x7f862009f410 2023-10-03T10:17:35.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.187+0000 7f86269cf700 1 --2- 172.21.15.3:0/59315447 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8620136fe0 0x7f86201393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:35.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.187+0000 7f86259cd700 1 --2- 172.21.15.3:0/59315447 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f862009f410 0x7f8620136aa0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f8610009870 tx=0x7f8610009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfb8d6dea1655d60 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:35.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.188+0000 7f86251cc700 1 -- 172.21.15.3:0/59315447 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8610011830 con 0x7f862009f410 2023-10-03T10:17:35.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.188+0000 7f86251cc700 1 -- 172.21.15.3:0/59315447 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8610011990 con 0x7f862009f410 2023-10-03T10:17:35.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.188+0000 7f86251cc700 1 -- 172.21.15.3:0/59315447 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8610011d20 con 0x7f862009f410 2023-10-03T10:17:35.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.188+0000 7f8628432700 1 -- 172.21.15.3:0/59315447 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f862009f410 msgr2=0x7f8620136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:35.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.188+0000 7f8628432700 1 --2- 172.21.15.3:0/59315447 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f862009f410 0x7f8620136aa0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f8610009870 tx=0x7f8610009b80 comp rx=0 tx=0).stop 2023-10-03T10:17:35.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.188+0000 7f8628432700 1 -- 172.21.15.3:0/59315447 shutdown_connections 2023-10-03T10:17:35.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.188+0000 7f8628432700 1 --2- 172.21.15.3:0/59315447 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8620136fe0 0x7f86201393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:35.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.188+0000 7f8628432700 1 --2- 172.21.15.3:0/59315447 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f862009f410 0x7f8620136aa0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:35.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.188+0000 7f8628432700 1 -- 172.21.15.3:0/59315447 >> 172.21.15.3:0/59315447 conn(0x7f86200a4210 msgr2=0x7f86200a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:35.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.188+0000 7f8628432700 1 -- 172.21.15.3:0/59315447 shutdown_connections 2023-10-03T10:17:35.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.188+0000 7f8628432700 1 -- 172.21.15.3:0/59315447 wait complete. 2023-10-03T10:17:35.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.189+0000 7f8628432700 1 Processor -- start 2023-10-03T10:17:35.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.189+0000 7f8628432700 1 -- start start 2023-10-03T10:17:35.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.190+0000 7f8628432700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f862009e930 0x7f86201c6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:35.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.190+0000 7f8628432700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f862009f410 0x7f86201c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:35.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.190+0000 7f8628432700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8620136fe0 0x7f862011c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:35.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.190+0000 7f8628432700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862011e1d0 con 0x7f862009f410 2023-10-03T10:17:35.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.190+0000 7f8628432700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862011ec90 con 0x7f862009e930 2023-10-03T10:17:35.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.190+0000 7f8628432700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f86201c6ff0 con 0x7f8620136fe0 2023-10-03T10:17:35.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.190+0000 7f86261ce700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f862009e930 0x7f86201c6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:35.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.190+0000 7f86269cf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8620136fe0 0x7f862011c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:35.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.190+0000 7f86259cd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f862009f410 0x7f86201c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:35.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.190+0000 7f86269cf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8620136fe0 0x7f862011c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53850/0 (socket says 172.21.15.3:53850) 2023-10-03T10:17:35.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.190+0000 7f86269cf700 1 -- 172.21.15.3:0/1376815164 learned_addr learned my addr 172.21.15.3:0/1376815164 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:35.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.190+0000 7f86261ce700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f862009e930 0x7f86201c6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38708/0 (socket says 172.21.15.3:38708) 2023-10-03T10:17:35.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.190+0000 7f86259cd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f862009f410 0x7f86201c6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39604/0 (socket says 172.21.15.3:39604) 2023-10-03T10:17:35.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.190+0000 7f86269cf700 1 -- 172.21.15.3:0/1376815164 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f862009e930 msgr2=0x7f86201c6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:35.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.190+0000 7f86269cf700 1 --2- 172.21.15.3:0/1376815164 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f862009e930 0x7f86201c6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:35.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.190+0000 7f86269cf700 1 -- 172.21.15.3:0/1376815164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f862009f410 msgr2=0x7f86201c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:35.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.190+0000 7f86269cf700 1 --2- 172.21.15.3:0/1376815164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f862009f410 0x7f86201c6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:35.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.190+0000 7f86269cf700 1 -- 172.21.15.3:0/1376815164 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8618009a70 con 0x7f8620136fe0 2023-10-03T10:17:35.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.190+0000 7f86261ce700 1 --2- 172.21.15.3:0/1376815164 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f862009e930 0x7f86201c6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:35.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.190+0000 7f86269cf700 1 --2- 172.21.15.3:0/1376815164 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8620136fe0 0x7f862011c660 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f861800efc0 tx=0x7f861800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:35.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.191+0000 7f8616ffd700 1 -- 172.21.15.3:0/1376815164 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861800cc90 con 0x7f8620136fe0 2023-10-03T10:17:35.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.191+0000 7f8616ffd700 1 -- 172.21.15.3:0/1376815164 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861800f040 con 0x7f8620136fe0 2023-10-03T10:17:35.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.191+0000 7f8628432700 1 -- 172.21.15.3:0/1376815164 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86100094a0 con 0x7f8620136fe0 2023-10-03T10:17:35.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.191+0000 7f8616ffd700 1 -- 172.21.15.3:0/1376815164 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86180146c0 con 0x7f8620136fe0 2023-10-03T10:17:35.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.191+0000 7f8628432700 1 -- 172.21.15.3:0/1376815164 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86201324b0 con 0x7f8620136fe0 2023-10-03T10:17:35.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.191+0000 7f8616ffd700 1 -- 172.21.15.3:0/1376815164 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8618014820 con 0x7f8620136fe0 2023-10-03T10:17:35.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.192+0000 7f8616ffd700 1 -- 172.21.15.3:0/1376815164 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8618040320 con 0x7f8620136fe0 2023-10-03T10:17:35.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.192+0000 7f8628432700 1 -- 172.21.15.3:0/1376815164 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f862011fd40 con 0x7f8620136fe0 2023-10-03T10:17:35.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.197+0000 7f8616ffd700 1 -- 172.21.15.3:0/1376815164 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8618018080 con 0x7f8620136fe0 2023-10-03T10:17:35.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.406+0000 7f8628432700 1 -- 172.21.15.3:0/1376815164 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8620074ea0 con 0x7f8620136fe0 2023-10-03T10:17:35.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.406+0000 7f8616ffd700 1 -- 172.21.15.3:0/1376815164 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8618007510 con 0x7f8620136fe0 2023-10-03T10:17:35.407 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:35.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.409+0000 7f8628432700 1 -- 172.21.15.3:0/1376815164 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8620136fe0 msgr2=0x7f862011c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:35.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.409+0000 7f8628432700 1 --2- 172.21.15.3:0/1376815164 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8620136fe0 0x7f862011c660 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f861800efc0 tx=0x7f861800c3a0 comp rx=0 tx=0).stop 2023-10-03T10:17:35.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.409+0000 7f8628432700 1 -- 172.21.15.3:0/1376815164 shutdown_connections 2023-10-03T10:17:35.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.409+0000 7f8628432700 1 --2- 172.21.15.3:0/1376815164 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8620136fe0 0x7f862011c660 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:35.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.409+0000 7f8628432700 1 --2- 172.21.15.3:0/1376815164 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f862009e930 0x7f86201c6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:35.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.409+0000 7f8628432700 1 --2- 172.21.15.3:0/1376815164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f862009f410 0x7f86201c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:35.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.409+0000 7f8628432700 1 -- 172.21.15.3:0/1376815164 >> 172.21.15.3:0/1376815164 conn(0x7f86200a4210 msgr2=0x7f8620134ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:35.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.409+0000 7f8628432700 1 -- 172.21.15.3:0/1376815164 shutdown_connections 2023-10-03T10:17:35.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.409+0000 7f8628432700 1 -- 172.21.15.3:0/1376815164 wait complete. 2023-10-03T10:17:35.423 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:35.579 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:35.615 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:35.655 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:35.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.729+0000 7f5bae057700 1 Processor -- start 2023-10-03T10:17:35.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.730+0000 7f5bae057700 1 -- start start 2023-10-03T10:17:35.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.730+0000 7f5bae057700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5ba80a1510 0x7f5ba8138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:35.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.730+0000 7f5bae057700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ba8139250 0x7f5ba813b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:35.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.730+0000 7f5bae057700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ba809edd0 con 0x7f5ba80a0a30 2023-10-03T10:17:35.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.730+0000 7f5bae057700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ba80a18e0 con 0x7f5ba80a1510 2023-10-03T10:17:35.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.730+0000 7f5bae057700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba809e1f0 con 0x7f5ba8139250 2023-10-03T10:17:35.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.731+0000 7f5ba6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5ba80a1510 0x7f5ba8138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:35.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.731+0000 7f5ba7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ba8139250 0x7f5ba813b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:35.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.731+0000 7f5ba6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5ba80a1510 0x7f5ba8138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39616/0 (socket says 172.21.15.3:39616) 2023-10-03T10:17:35.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.731+0000 7f5ba7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ba8139250 0x7f5ba813b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38722/0 (socket says 172.21.15.3:38722) 2023-10-03T10:17:35.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.731+0000 7f5ba6ffd700 1 -- 172.21.15.3:0/4152502285 learned_addr learned my addr 172.21.15.3:0/4152502285 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:35.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.731+0000 7f5ba7fff700 1 -- 172.21.15.3:0/4152502285 >> v2:172.21.15.3:3300/0 conn(0x7f5ba80a1510 msgr2=0x7f5ba8138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:35.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.731+0000 7f5ba7fff700 1 --2- 172.21.15.3:0/4152502285 >> v2:172.21.15.3:3300/0 conn(0x7f5ba80a1510 0x7f5ba8138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:35.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.731+0000 7f5ba7fff700 1 -- 172.21.15.3:0/4152502285 >> v1:172.21.15.3:6789/0 conn(0x7f5ba80a0a30 legacy=0x7f5ba80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:35.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.731+0000 7f5ba7fff700 1 -- 172.21.15.3:0/4152502285 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba809dc50 con 0x7f5ba8139250 2023-10-03T10:17:35.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.731+0000 7f5ba6ffd700 1 --2- 172.21.15.3:0/4152502285 >> v2:172.21.15.3:3300/0 conn(0x7f5ba80a1510 0x7f5ba8138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:35.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.731+0000 7f5ba7fff700 1 --2- 172.21.15.3:0/4152502285 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ba8139250 0x7f5ba813b630 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009d90 tx=0x7f5ba000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5084e271464ae280 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:35.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.731+0000 7f5ba67fc700 1 -- 172.21.15.3:0/4152502285 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba000e070 con 0x7f5ba8139250 2023-10-03T10:17:35.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.731+0000 7f5ba67fc700 1 -- 172.21.15.3:0/4152502285 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba000bb20 con 0x7f5ba8139250 2023-10-03T10:17:35.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.731+0000 7f5ba67fc700 1 -- 172.21.15.3:0/4152502285 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0012430 con 0x7f5ba8139250 2023-10-03T10:17:35.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.733+0000 7f5bae057700 1 -- 172.21.15.3:0/4152502285 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ba8139250 msgr2=0x7f5ba813b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:35.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.733+0000 7f5bae057700 1 --2- 172.21.15.3:0/4152502285 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ba8139250 0x7f5ba813b630 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009d90 tx=0x7f5ba000b330 comp rx=0 tx=0).stop 2023-10-03T10:17:35.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.733+0000 7f5bae057700 1 -- 172.21.15.3:0/4152502285 shutdown_connections 2023-10-03T10:17:35.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.733+0000 7f5bae057700 1 --2- 172.21.15.3:0/4152502285 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ba8139250 0x7f5ba813b630 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:35.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.733+0000 7f5bae057700 1 --2- 172.21.15.3:0/4152502285 >> v2:172.21.15.3:3300/0 conn(0x7f5ba80a1510 0x7f5ba8138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:35.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.733+0000 7f5bae057700 1 -- 172.21.15.3:0/4152502285 >> 172.21.15.3:0/4152502285 conn(0x7f5ba80a6550 msgr2=0x7f5ba80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:35.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.733+0000 7f5bae057700 1 -- 172.21.15.3:0/4152502285 shutdown_connections 2023-10-03T10:17:35.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.733+0000 7f5bae057700 1 -- 172.21.15.3:0/4152502285 wait complete. 2023-10-03T10:17:35.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.734+0000 7f5bae057700 1 Processor -- start 2023-10-03T10:17:35.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.734+0000 7f5bae057700 1 -- start start 2023-10-03T10:17:35.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.735+0000 7f5bae057700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ba80a0a30 0x7f5ba81c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:35.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.735+0000 7f5bae057700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ba80a1510 0x7f5ba81c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:35.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.735+0000 7f5bae057700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ba8139250 0x7f5ba81c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:35.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.735+0000 7f5bae057700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba8124820 con 0x7f5ba8139250 2023-10-03T10:17:35.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.735+0000 7f5bae057700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba8125d40 con 0x7f5ba80a1510 2023-10-03T10:17:35.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.735+0000 7f5bae057700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba81253a0 con 0x7f5ba80a0a30 2023-10-03T10:17:35.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.735+0000 7f5ba77fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ba80a0a30 0x7f5ba81c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:35.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.735+0000 7f5ba6ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ba80a1510 0x7f5ba81c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:35.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.735+0000 7f5ba77fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ba80a0a30 0x7f5ba81c8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53862/0 (socket says 172.21.15.3:53862) 2023-10-03T10:17:35.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.735+0000 7f5ba77fe700 1 -- 172.21.15.3:0/2041431037 learned_addr learned my addr 172.21.15.3:0/2041431037 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:35.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.735+0000 7f5ba7fff700 1 --2- 172.21.15.3:0/2041431037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ba8139250 0x7f5ba81c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.736+0000 7f5ba7fff700 1 -- 172.21.15.3:0/2041431037 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ba80a0a30 msgr2=0x7f5ba81c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.736+0000 7f5ba7fff700 1 --2- 172.21.15.3:0/2041431037 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ba80a0a30 0x7f5ba81c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.736+0000 7f5ba7fff700 1 -- 172.21.15.3:0/2041431037 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ba80a1510 msgr2=0x7f5ba81c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.736+0000 7f5ba7fff700 1 --2- 172.21.15.3:0/2041431037 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ba80a1510 0x7f5ba81c8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.736+0000 7f5ba7fff700 1 -- 172.21.15.3:0/2041431037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9c009550 con 0x7f5ba8139250 2023-10-03T10:17:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.736+0000 7f5ba6ffd700 1 --2- 172.21.15.3:0/2041431037 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ba80a1510 0x7f5ba81c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.736+0000 7f5ba7fff700 1 --2- 172.21.15.3:0/2041431037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ba8139250 0x7f5ba81c8fc0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0016000 tx=0x7f5ba0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.736+0000 7f5b8ffff700 1 -- 172.21.15.3:0/2041431037 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0012430 con 0x7f5ba8139250 2023-10-03T10:17:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.736+0000 7f5b8ffff700 1 -- 172.21.15.3:0/2041431037 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba0020070 con 0x7f5ba8139250 2023-10-03T10:17:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.736+0000 7f5bae057700 1 -- 172.21.15.3:0/2041431037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba0009a70 con 0x7f5ba8139250 2023-10-03T10:17:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.736+0000 7f5b8ffff700 1 -- 172.21.15.3:0/2041431037 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0004040 con 0x7f5ba8139250 2023-10-03T10:17:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.736+0000 7f5bae057700 1 -- 172.21.15.3:0/2041431037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba81262d0 con 0x7f5ba8139250 2023-10-03T10:17:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.737+0000 7f5b8ffff700 1 -- 172.21.15.3:0/2041431037 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5ba00041a0 con 0x7f5ba8139250 2023-10-03T10:17:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.737+0000 7f5b8ffff700 1 -- 172.21.15.3:0/2041431037 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5ba0022070 con 0x7f5ba8139250 2023-10-03T10:17:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.737+0000 7f5bae057700 1 -- 172.21.15.3:0/2041431037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba81264c0 con 0x7f5ba8139250 2023-10-03T10:17:35.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.743+0000 7f5b8ffff700 1 -- 172.21.15.3:0/2041431037 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5ba0004450 con 0x7f5ba8139250 2023-10-03T10:17:35.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.777+0000 7f80c2a4e700 1 Processor -- start 2023-10-03T10:17:35.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.777+0000 7f80c2a4e700 1 -- start start 2023-10-03T10:17:35.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.778+0000 7f80c2a4e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f80bc131630 0x7f80bc131a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:35.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.778+0000 7f80c2a4e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f80bc06eba0 0x7f80bc139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:35.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.778+0000 7f80c2a4e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80bc071a00 con 0x7f80bc131f40 2023-10-03T10:17:35.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.778+0000 7f80c2a4e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80bc131430 con 0x7f80bc06eba0 2023-10-03T10:17:35.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.778+0000 7f80c2a4e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80bc09a510 con 0x7f80bc131630 2023-10-03T10:17:35.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.779+0000 7f80bb7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f80bc131f40 0x7f80bc06e490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33014/0 (socket says 172.21.15.3:33014) 2023-10-03T10:17:35.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.779+0000 7f80bbfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f80bc131630 0x7f80bc131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:35.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.779+0000 7f80bb7fe700 1 -- 172.21.15.3:0/530429109 learned_addr learned my addr 172.21.15.3:0/530429109 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:35.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.779+0000 7f80c0feb700 1 --2- 172.21.15.3:0/530429109 >> v2:172.21.15.3:3300/0 conn(0x7f80bc06eba0 0x7f80bc139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.779+0000 7f80baffd700 1 -- 172.21.15.3:0/530429109 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3559632472 0 0) 0x7f80bc071a00 con 0x7f80bc131f40 2023-10-03T10:17:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.779+0000 7f80baffd700 1 -- 172.21.15.3:0/530429109 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80a40035b0 con 0x7f80bc131f40 2023-10-03T10:17:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.779+0000 7f80c0feb700 1 -- 172.21.15.3:0/530429109 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f80bc131630 msgr2=0x7f80bc131a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.779+0000 7f80c0feb700 1 --2- 172.21.15.3:0/530429109 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f80bc131630 0x7f80bc131a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.779+0000 7f80c0feb700 1 -- 172.21.15.3:0/530429109 >> v1:172.21.15.3:6789/0 conn(0x7f80bc131f40 legacy=0x7f80bc06e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.779+0000 7f80c0feb700 1 -- 172.21.15.3:0/530429109 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80bc0721c0 con 0x7f80bc06eba0 2023-10-03T10:17:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.779+0000 7f80bbfff700 1 --2- 172.21.15.3:0/530429109 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f80bc131630 0x7f80bc131a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.779+0000 7f80c0feb700 1 --2- 172.21.15.3:0/530429109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80bc06eba0 0x7f80bc139660 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f80b4009d90 tx=0x7f80b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46440d632286d64a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.779+0000 7f80baffd700 1 -- 172.21.15.3:0/530429109 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b400e070 con 0x7f80bc06eba0 2023-10-03T10:17:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.780+0000 7f80baffd700 1 -- 172.21.15.3:0/530429109 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b400bb20 con 0x7f80bc06eba0 2023-10-03T10:17:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.780+0000 7f80baffd700 1 -- 172.21.15.3:0/530429109 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b4012430 con 0x7f80bc06eba0 2023-10-03T10:17:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.780+0000 7f80c2a4e700 1 -- 172.21.15.3:0/530429109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80bc06eba0 msgr2=0x7f80bc139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.780+0000 7f80c2a4e700 1 --2- 172.21.15.3:0/530429109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80bc06eba0 0x7f80bc139660 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f80b4009d90 tx=0x7f80b400b330 comp rx=0 tx=0).stop 2023-10-03T10:17:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.780+0000 7f80c2a4e700 1 -- 172.21.15.3:0/530429109 shutdown_connections 2023-10-03T10:17:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.780+0000 7f80c2a4e700 1 --2- 172.21.15.3:0/530429109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80bc06eba0 0x7f80bc139660 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.780+0000 7f80c2a4e700 1 --2- 172.21.15.3:0/530429109 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f80bc131630 0x7f80bc131a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.780+0000 7f80c2a4e700 1 -- 172.21.15.3:0/530429109 >> 172.21.15.3:0/530429109 conn(0x7f80bc0a2aa0 msgr2=0x7f80bc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.780+0000 7f80c2a4e700 1 -- 172.21.15.3:0/530429109 shutdown_connections 2023-10-03T10:17:35.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.780+0000 7f80c2a4e700 1 -- 172.21.15.3:0/530429109 wait complete. 2023-10-03T10:17:35.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.782+0000 7f80c2a4e700 1 Processor -- start 2023-10-03T10:17:35.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.782+0000 7f80c2a4e700 1 -- start start 2023-10-03T10:17:35.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.783+0000 7f80c2a4e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80bc06eba0 0x7f80bc1194c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:35.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.783+0000 7f80c2a4e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f80bc131630 0x7f80bc119a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:35.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.783+0000 7f80c2a4e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f80bc131f40 0x7f80bc119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:35.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.783+0000 7f80c2a4e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80bc11b720 con 0x7f80bc06eba0 2023-10-03T10:17:35.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.783+0000 7f80c2a4e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80bc11d9a0 con 0x7f80bc131630 2023-10-03T10:17:35.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.783+0000 7f80c2a4e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f80bc11cfe0 con 0x7f80bc131f40 2023-10-03T10:17:35.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.783+0000 7f80bb7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f80bc131630 0x7f80bc119a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:35.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.783+0000 7f80bbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80bc06eba0 0x7f80bc1194c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:35.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.783+0000 7f80bbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80bc06eba0 0x7f80bc1194c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39636/0 (socket says 172.21.15.3:39636) 2023-10-03T10:17:35.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.783+0000 7f80bb7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f80bc131630 0x7f80bc119a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38746/0 (socket says 172.21.15.3:38746) 2023-10-03T10:17:35.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.783+0000 7f80bbfff700 1 -- 172.21.15.3:0/2536498047 learned_addr learned my addr 172.21.15.3:0/2536498047 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:35.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.783+0000 7f80c0feb700 1 --2- 172.21.15.3:0/2536498047 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f80bc131f40 0x7f80bc119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:35.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.783+0000 7f80bb7fe700 1 -- 172.21.15.3:0/2536498047 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f80bc131f40 msgr2=0x7f80bc119f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:35.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.783+0000 7f80bb7fe700 1 --2- 172.21.15.3:0/2536498047 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f80bc131f40 0x7f80bc119f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:35.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.783+0000 7f80bb7fe700 1 -- 172.21.15.3:0/2536498047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80bc06eba0 msgr2=0x7f80bc1194c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:35.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.783+0000 7f80bb7fe700 1 --2- 172.21.15.3:0/2536498047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80bc06eba0 0x7f80bc1194c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:35.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.783+0000 7f80bb7fe700 1 -- 172.21.15.3:0/2536498047 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80ac0094a0 con 0x7f80bc131630 2023-10-03T10:17:35.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.784+0000 7f80bb7fe700 1 --2- 172.21.15.3:0/2536498047 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f80bc131630 0x7f80bc119a00 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f80b000c490 tx=0x7f80b000c7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:35.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.784+0000 7f80b8ff9700 1 -- 172.21.15.3:0/2536498047 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b0005d70 con 0x7f80bc131630 2023-10-03T10:17:35.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.784+0000 7f80b8ff9700 1 -- 172.21.15.3:0/2536498047 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b00070d0 con 0x7f80bc131630 2023-10-03T10:17:35.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.784+0000 7f80b8ff9700 1 -- 172.21.15.3:0/2536498047 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b0007440 con 0x7f80bc131630 2023-10-03T10:17:35.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.784+0000 7f80c2a4e700 1 -- 172.21.15.3:0/2536498047 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b4009a70 con 0x7f80bc131630 2023-10-03T10:17:35.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.784+0000 7f80c2a4e700 1 -- 172.21.15.3:0/2536498047 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80bc09f7d0 con 0x7f80bc131630 2023-10-03T10:17:35.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.785+0000 7f80b8ff9700 1 -- 172.21.15.3:0/2536498047 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f80b0017020 con 0x7f80bc131630 2023-10-03T10:17:35.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.785+0000 7f80b8ff9700 1 -- 172.21.15.3:0/2536498047 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f80b00426c0 con 0x7f80bc131630 2023-10-03T10:17:35.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.786+0000 7f80c2a4e700 1 -- 172.21.15.3:0/2536498047 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80bc11c9a0 con 0x7f80bc131630 2023-10-03T10:17:35.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.796+0000 7f80b8ff9700 1 -- 172.21.15.3:0/2536498047 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f80bc11c9a0 con 0x7f80bc131630 2023-10-03T10:17:35.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.823+0000 7f33617ce700 1 Processor -- start 2023-10-03T10:17:35.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.824+0000 7f33617ce700 1 -- start start 2023-10-03T10:17:35.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.824+0000 7f33617ce700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f335c1338a0 0x7f335c133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:35.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.824+0000 7f33617ce700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f335c09e930 0x7f335c13b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:35.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.824+0000 7f33617ce700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f335c06ef30 con 0x7f335c09de50 2023-10-03T10:17:35.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.824+0000 7f33617ce700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f335c06fef0 con 0x7f335c1338a0 2023-10-03T10:17:35.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.824+0000 7f33617ce700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f335c1334b0 con 0x7f335c09e930 2023-10-03T10:17:35.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.824+0000 7f335affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f335c1338a0 0x7f335c133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:35.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.824+0000 7f335a7fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f335c09de50 0x7f335c09e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33024/0 (socket says 172.21.15.3:33024) 2023-10-03T10:17:35.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.824+0000 7f335affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f335c1338a0 0x7f335c133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38760/0 (socket says 172.21.15.3:38760) 2023-10-03T10:17:35.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.824+0000 7f335a7fc700 1 -- 172.21.15.3:0/3653868915 learned_addr learned my addr 172.21.15.3:0/3653868915 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:35.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.824+0000 7f335b7fe700 1 --2- 172.21.15.3:0/3653868915 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f335c09e930 0x7f335c13b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:35.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.825+0000 7f335affd700 1 -- 172.21.15.3:0/3653868915 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f335c09e930 msgr2=0x7f335c13b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:35.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.825+0000 7f335affd700 1 --2- 172.21.15.3:0/3653868915 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f335c09e930 0x7f335c13b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:35.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.825+0000 7f335affd700 1 -- 172.21.15.3:0/3653868915 >> v1:172.21.15.3:6789/0 conn(0x7f335c09de50 legacy=0x7f335c09e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:35.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.825+0000 7f335affd700 1 -- 172.21.15.3:0/3653868915 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335c06fc30 con 0x7f335c1338a0 2023-10-03T10:17:35.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.825+0000 7f335b7fe700 1 --2- 172.21.15.3:0/3653868915 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f335c09e930 0x7f335c13b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:35.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.825+0000 7f335affd700 1 --2- 172.21.15.3:0/3653868915 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f335c1338a0 0x7f335c133c70 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f334c009870 tx=0x7f334c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89b4ba37498d23e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:35.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.825+0000 7f3359ffb700 1 -- 172.21.15.3:0/3653868915 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c011830 con 0x7f335c1338a0 2023-10-03T10:17:35.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.825+0000 7f3359ffb700 1 -- 172.21.15.3:0/3653868915 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f334c011990 con 0x7f335c1338a0 2023-10-03T10:17:35.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.825+0000 7f3359ffb700 1 -- 172.21.15.3:0/3653868915 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c011d20 con 0x7f335c1338a0 2023-10-03T10:17:35.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.826+0000 7f33617ce700 1 -- 172.21.15.3:0/3653868915 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f335c1338a0 msgr2=0x7f335c133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:35.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.826+0000 7f33617ce700 1 --2- 172.21.15.3:0/3653868915 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f335c1338a0 0x7f335c133c70 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f334c009870 tx=0x7f334c009b80 comp rx=0 tx=0).stop 2023-10-03T10:17:35.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.826+0000 7f33617ce700 1 -- 172.21.15.3:0/3653868915 shutdown_connections 2023-10-03T10:17:35.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.826+0000 7f33617ce700 1 --2- 172.21.15.3:0/3653868915 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f335c1338a0 0x7f335c133c70 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:35.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.826+0000 7f33617ce700 1 --2- 172.21.15.3:0/3653868915 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f335c09e930 0x7f335c13b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:35.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.826+0000 7f33617ce700 1 -- 172.21.15.3:0/3653868915 >> 172.21.15.3:0/3653868915 conn(0x7f335c0a2aa0 msgr2=0x7f335c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:35.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.826+0000 7f33617ce700 1 -- 172.21.15.3:0/3653868915 shutdown_connections 2023-10-03T10:17:35.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.826+0000 7f33617ce700 1 -- 172.21.15.3:0/3653868915 wait complete. 2023-10-03T10:17:35.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.827+0000 7f33617ce700 1 Processor -- start 2023-10-03T10:17:35.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.827+0000 7f33617ce700 1 -- start start 2023-10-03T10:17:35.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.828+0000 7f33617ce700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f335c09de50 0x7f335c11d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:35.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.828+0000 7f33617ce700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f335c09e930 0x7f335c11de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:35.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.828+0000 7f33617ce700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f335c1338a0 0x7f335c11e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:35.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.828+0000 7f33617ce700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335c121110 con 0x7f335c1338a0 2023-10-03T10:17:35.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.828+0000 7f33617ce700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f335c06f640 con 0x7f335c09e930 2023-10-03T10:17:35.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.828+0000 7f33617ce700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f335c11fd40 con 0x7f335c09de50 2023-10-03T10:17:35.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.828+0000 7f335a7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f335c09e930 0x7f335c11de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:35.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.828+0000 7f335b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f335c1338a0 0x7f335c11e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:35.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.828+0000 7f335a7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f335c09e930 0x7f335c11de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38768/0 (socket says 172.21.15.3:38768) 2023-10-03T10:17:35.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.828+0000 7f335a7fc700 1 -- 172.21.15.3:0/3973511644 learned_addr learned my addr 172.21.15.3:0/3973511644 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:35.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.828+0000 7f335affd700 1 --2- 172.21.15.3:0/3973511644 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f335c09de50 0x7f335c11d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:35.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.828+0000 7f335b7fe700 1 -- 172.21.15.3:0/3973511644 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f335c09de50 msgr2=0x7f335c11d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:35.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.828+0000 7f335b7fe700 1 --2- 172.21.15.3:0/3973511644 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f335c09de50 0x7f335c11d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:35.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.828+0000 7f335b7fe700 1 -- 172.21.15.3:0/3973511644 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f335c09e930 msgr2=0x7f335c11de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:35.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.828+0000 7f335b7fe700 1 --2- 172.21.15.3:0/3973511644 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f335c09e930 0x7f335c11de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:35.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.828+0000 7f335b7fe700 1 -- 172.21.15.3:0/3973511644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3354009a70 con 0x7f335c1338a0 2023-10-03T10:17:35.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.828+0000 7f335a7fc700 1 --2- 172.21.15.3:0/3973511644 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f335c09e930 0x7f335c11de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:17:35.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.828+0000 7f335b7fe700 1 --2- 172.21.15.3:0/3973511644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f335c1338a0 0x7f335c11e350 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f3354009e50 tx=0x7f335400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:35.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.829+0000 7f33437fe700 1 -- 172.21.15.3:0/3973511644 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335400cb40 con 0x7f335c1338a0 2023-10-03T10:17:35.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.829+0000 7f33437fe700 1 -- 172.21.15.3:0/3973511644 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f335400f040 con 0x7f335c1338a0 2023-10-03T10:17:35.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.829+0000 7f33437fe700 1 -- 172.21.15.3:0/3973511644 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3354003a20 con 0x7f335c1338a0 2023-10-03T10:17:35.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.829+0000 7f33617ce700 1 -- 172.21.15.3:0/3973511644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f334c0094a0 con 0x7f335c1338a0 2023-10-03T10:17:35.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.829+0000 7f33617ce700 1 -- 172.21.15.3:0/3973511644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f335c121450 con 0x7f335c1338a0 2023-10-03T10:17:35.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.830+0000 7f33437fe700 1 -- 172.21.15.3:0/3973511644 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3354014480 con 0x7f335c1338a0 2023-10-03T10:17:35.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.830+0000 7f33617ce700 1 -- 172.21.15.3:0/3973511644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f335c120ca0 con 0x7f335c1338a0 2023-10-03T10:17:35.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.830+0000 7f33437fe700 1 -- 172.21.15.3:0/3973511644 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f335403fe40 con 0x7f335c1338a0 2023-10-03T10:17:35.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.839+0000 7f33437fe700 1 -- 172.21.15.3:0/3973511644 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3354012750 con 0x7f335c1338a0 2023-10-03T10:17:35.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.957+0000 7f5bae057700 1 -- 172.21.15.3:0/2041431037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5ba8074ea0 con 0x7f5ba8139250 2023-10-03T10:17:35.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.958+0000 7f5b8ffff700 1 -- 172.21.15.3:0/2041431037 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5ba0011390 con 0x7f5ba8139250 2023-10-03T10:17:35.958 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:35.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.960+0000 7f5bae057700 1 -- 172.21.15.3:0/2041431037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ba8139250 msgr2=0x7f5ba81c8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:35.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.960+0000 7f5bae057700 1 --2- 172.21.15.3:0/2041431037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ba8139250 0x7f5ba81c8fc0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0016000 tx=0x7f5ba0004ad0 comp rx=0 tx=0).stop 2023-10-03T10:17:35.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.960+0000 7f5bae057700 1 -- 172.21.15.3:0/2041431037 shutdown_connections 2023-10-03T10:17:35.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.960+0000 7f5bae057700 1 --2- 172.21.15.3:0/2041431037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ba8139250 0x7f5ba81c8fc0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:35.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.960+0000 7f5bae057700 1 --2- 172.21.15.3:0/2041431037 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ba80a0a30 0x7f5ba81c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:35.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.960+0000 7f5bae057700 1 --2- 172.21.15.3:0/2041431037 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ba80a1510 0x7f5ba81c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:35.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.960+0000 7f5bae057700 1 -- 172.21.15.3:0/2041431037 >> 172.21.15.3:0/2041431037 conn(0x7f5ba80a6550 msgr2=0x7f5ba80a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:35.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.960+0000 7f5bae057700 1 -- 172.21.15.3:0/2041431037 shutdown_connections 2023-10-03T10:17:35.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:35.960+0000 7f5bae057700 1 -- 172.21.15.3:0/2041431037 wait complete. 2023-10-03T10:17:35.975 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:36.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.013+0000 7f80c2a4e700 1 -- 172.21.15.3:0/2536498047 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f80bc11c9a0 con 0x7f80bc131630 2023-10-03T10:17:36.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.014+0000 7f80b8ff9700 1 -- 172.21.15.3:0/2536498047 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f80bc11c9a0 con 0x7f80bc131630 2023-10-03T10:17:36.014 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:36.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.016+0000 7f80c2a4e700 1 -- 172.21.15.3:0/2536498047 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f80bc131630 msgr2=0x7f80bc119a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:36.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.016+0000 7f80c2a4e700 1 --2- 172.21.15.3:0/2536498047 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f80bc131630 0x7f80bc119a00 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f80b000c490 tx=0x7f80b000c7a0 comp rx=0 tx=0).stop 2023-10-03T10:17:36.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.016+0000 7f80c2a4e700 1 -- 172.21.15.3:0/2536498047 shutdown_connections 2023-10-03T10:17:36.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.016+0000 7f80c2a4e700 1 --2- 172.21.15.3:0/2536498047 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f80bc131f40 0x7f80bc119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:36.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.016+0000 7f80c2a4e700 1 --2- 172.21.15.3:0/2536498047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80bc06eba0 0x7f80bc1194c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:36.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.016+0000 7f80c2a4e700 1 --2- 172.21.15.3:0/2536498047 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f80bc131630 0x7f80bc119a00 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:36.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.016+0000 7f80c2a4e700 1 -- 172.21.15.3:0/2536498047 >> 172.21.15.3:0/2536498047 conn(0x7f80bc0a2aa0 msgr2=0x7f80bc12e560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:36.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.016+0000 7f80c2a4e700 1 -- 172.21.15.3:0/2536498047 shutdown_connections 2023-10-03T10:17:36.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.016+0000 7f80c2a4e700 1 -- 172.21.15.3:0/2536498047 wait complete. 2023-10-03T10:17:36.030 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:36.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.045+0000 7f33617ce700 1 -- 172.21.15.3:0/3973511644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f335c121450 con 0x7f335c1338a0 2023-10-03T10:17:36.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.046+0000 7f33437fe700 1 -- 172.21.15.3:0/3973511644 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f335c121450 con 0x7f335c1338a0 2023-10-03T10:17:36.046 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:36.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.048+0000 7f33617ce700 1 -- 172.21.15.3:0/3973511644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f335c1338a0 msgr2=0x7f335c11e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:36.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.048+0000 7f33617ce700 1 --2- 172.21.15.3:0/3973511644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f335c1338a0 0x7f335c11e350 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f3354009e50 tx=0x7f335400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:17:36.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.048+0000 7f33617ce700 1 -- 172.21.15.3:0/3973511644 shutdown_connections 2023-10-03T10:17:36.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.048+0000 7f33617ce700 1 --2- 172.21.15.3:0/3973511644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f335c1338a0 0x7f335c11e350 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:36.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.048+0000 7f33617ce700 1 --2- 172.21.15.3:0/3973511644 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f335c09de50 0x7f335c11d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:36.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.048+0000 7f33617ce700 1 --2- 172.21.15.3:0/3973511644 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f335c09e930 0x7f335c11de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:36.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.048+0000 7f33617ce700 1 -- 172.21.15.3:0/3973511644 >> 172.21.15.3:0/3973511644 conn(0x7f335c0a2aa0 msgr2=0x7f335c09fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:36.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.048+0000 7f33617ce700 1 -- 172.21.15.3:0/3973511644 shutdown_connections 2023-10-03T10:17:36.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.048+0000 7f33617ce700 1 -- 172.21.15.3:0/3973511644 wait complete. 2023-10-03T10:17:36.062 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:36.423 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:36.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.570+0000 7f46ec57b700 1 Processor -- start 2023-10-03T10:17:36.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.570+0000 7f46ec57b700 1 -- start start 2023-10-03T10:17:36.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.571+0000 7f46ec57b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46e409e1d0 0x7f46e4135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:36.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.571+0000 7f46ec57b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f46e4135e50 0x7f46e4138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:36.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.571+0000 7f46ec57b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46e409f130 con 0x7f46e409d6f0 2023-10-03T10:17:36.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.571+0000 7f46ec57b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e409f520 con 0x7f46e4135e50 2023-10-03T10:17:36.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.571+0000 7f46ec57b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e409e930 con 0x7f46e409e1d0 2023-10-03T10:17:36.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.571+0000 7f46ea317700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f46e409d6f0 0x7f46e409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33038/0 (socket says 172.21.15.3:33038) 2023-10-03T10:17:36.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.571+0000 7f46ea317700 1 -- 172.21.15.3:0/1522466746 learned_addr learned my addr 172.21.15.3:0/1522466746 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:36.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.571+0000 7f46eab18700 1 --2- 172.21.15.3:0/1522466746 >> v2:172.21.15.3:3300/0 conn(0x7f46e4135e50 0x7f46e4138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:36.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.571+0000 7f46e9b16700 1 --2- 172.21.15.3:0/1522466746 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46e409e1d0 0x7f46e4135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:36.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.572+0000 7f46e9b16700 1 -- 172.21.15.3:0/1522466746 >> v2:172.21.15.3:3300/0 conn(0x7f46e4135e50 msgr2=0x7f46e4138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:36.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.572+0000 7f46e9b16700 1 --2- 172.21.15.3:0/1522466746 >> v2:172.21.15.3:3300/0 conn(0x7f46e4135e50 0x7f46e4138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:36.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.572+0000 7f46e9b16700 1 -- 172.21.15.3:0/1522466746 >> v1:172.21.15.3:6789/0 conn(0x7f46e409d6f0 legacy=0x7f46e409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:36.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.572+0000 7f46e9b16700 1 -- 172.21.15.3:0/1522466746 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e409b2d0 con 0x7f46e409e1d0 2023-10-03T10:17:36.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.572+0000 7f46eab18700 1 --2- 172.21.15.3:0/1522466746 >> v2:172.21.15.3:3300/0 conn(0x7f46e4135e50 0x7f46e4138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:36.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.572+0000 7f46e9b16700 1 --2- 172.21.15.3:0/1522466746 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46e409e1d0 0x7f46e4135910 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f46d4009870 tx=0x7f46d4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c6b48e51d51e83d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:36.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.572+0000 7f46e9315700 1 -- 172.21.15.3:0/1522466746 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d4011830 con 0x7f46e409e1d0 2023-10-03T10:17:36.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.572+0000 7f46e9315700 1 -- 172.21.15.3:0/1522466746 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d4011990 con 0x7f46e409e1d0 2023-10-03T10:17:36.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.572+0000 7f46e9315700 1 -- 172.21.15.3:0/1522466746 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d4011d20 con 0x7f46e409e1d0 2023-10-03T10:17:36.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.572+0000 7f46ec57b700 1 -- 172.21.15.3:0/1522466746 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46e409e1d0 msgr2=0x7f46e4135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:36.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.572+0000 7f46ec57b700 1 --2- 172.21.15.3:0/1522466746 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46e409e1d0 0x7f46e4135910 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f46d4009870 tx=0x7f46d4009b80 comp rx=0 tx=0).stop 2023-10-03T10:17:36.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.573+0000 7f46ec57b700 1 -- 172.21.15.3:0/1522466746 shutdown_connections 2023-10-03T10:17:36.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.573+0000 7f46ec57b700 1 --2- 172.21.15.3:0/1522466746 >> v2:172.21.15.3:3300/0 conn(0x7f46e4135e50 0x7f46e4138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:36.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.573+0000 7f46ec57b700 1 --2- 172.21.15.3:0/1522466746 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46e409e1d0 0x7f46e4135910 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:36.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.573+0000 7f46ec57b700 1 -- 172.21.15.3:0/1522466746 >> 172.21.15.3:0/1522466746 conn(0x7f46e4066f50 msgr2=0x7f46e409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:36.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.573+0000 7f46ec57b700 1 -- 172.21.15.3:0/1522466746 shutdown_connections 2023-10-03T10:17:36.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.573+0000 7f46ec57b700 1 -- 172.21.15.3:0/1522466746 wait complete. 2023-10-03T10:17:36.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.574+0000 7f46ec57b700 1 Processor -- start 2023-10-03T10:17:36.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.574+0000 7f46ec57b700 1 -- start start 2023-10-03T10:17:36.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.574+0000 7f46ec57b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46e409d6f0 0x7f46e41c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:36.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.574+0000 7f46ec57b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46e409e1d0 0x7f46e41c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:36.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.574+0000 7f46ec57b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46e4135e50 0x7f46e41c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:36.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.574+0000 7f46ec57b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e406b420 con 0x7f46e409d6f0 2023-10-03T10:17:36.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.574+0000 7f46ec57b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e406b230 con 0x7f46e4135e50 2023-10-03T10:17:36.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.574+0000 7f46ec57b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f46e411cad0 con 0x7f46e409e1d0 2023-10-03T10:17:36.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.574+0000 7f46ea317700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46e409d6f0 0x7f46e41c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:36.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.574+0000 7f46ea317700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46e409d6f0 0x7f46e41c8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39666/0 (socket says 172.21.15.3:39666) 2023-10-03T10:17:36.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.574+0000 7f46eab18700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46e4135e50 0x7f46e41c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:36.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.574+0000 7f46ea317700 1 -- 172.21.15.3:0/3931817668 learned_addr learned my addr 172.21.15.3:0/3931817668 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:36.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.574+0000 7f46e9b16700 1 --2- 172.21.15.3:0/3931817668 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46e409e1d0 0x7f46e41c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:36.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.575+0000 7f46ea317700 1 -- 172.21.15.3:0/3931817668 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46e409e1d0 msgr2=0x7f46e41c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:36.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.575+0000 7f46ea317700 1 --2- 172.21.15.3:0/3931817668 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46e409e1d0 0x7f46e41c87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:36.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.575+0000 7f46ea317700 1 -- 172.21.15.3:0/3931817668 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46e4135e50 msgr2=0x7f46e41c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:36.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.575+0000 7f46ea317700 1 --2- 172.21.15.3:0/3931817668 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46e4135e50 0x7f46e41c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:36.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.575+0000 7f46ea317700 1 -- 172.21.15.3:0/3931817668 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46dc009a70 con 0x7f46e409d6f0 2023-10-03T10:17:36.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.575+0000 7f46eab18700 1 --2- 172.21.15.3:0/3931817668 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46e4135e50 0x7f46e41c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:36.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.575+0000 7f46ea317700 1 --2- 172.21.15.3:0/3931817668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46e409d6f0 0x7f46e41c8280 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f46e000c550 tx=0x7f46e000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:36.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.575+0000 7f46daffd700 1 -- 172.21.15.3:0/3931817668 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e0005d70 con 0x7f46e409d6f0 2023-10-03T10:17:36.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.575+0000 7f46daffd700 1 -- 172.21.15.3:0/3931817668 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e00070d0 con 0x7f46e409d6f0 2023-10-03T10:17:36.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.575+0000 7f46ec57b700 1 -- 172.21.15.3:0/3931817668 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d40094a0 con 0x7f46e409d6f0 2023-10-03T10:17:36.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.575+0000 7f46daffd700 1 -- 172.21.15.3:0/3931817668 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e0007440 con 0x7f46e409d6f0 2023-10-03T10:17:36.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.575+0000 7f46ec57b700 1 -- 172.21.15.3:0/3931817668 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e4121b10 con 0x7f46e409d6f0 2023-10-03T10:17:36.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.576+0000 7f46daffd700 1 -- 172.21.15.3:0/3931817668 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f46e00075a0 con 0x7f46e409d6f0 2023-10-03T10:17:36.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.576+0000 7f46ec57b700 1 -- 172.21.15.3:0/3931817668 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e4074ea0 con 0x7f46e409d6f0 2023-10-03T10:17:36.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.576+0000 7f46daffd700 1 -- 172.21.15.3:0/3931817668 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f46e003eaa0 con 0x7f46e409d6f0 2023-10-03T10:17:36.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.581+0000 7f46daffd700 1 -- 172.21.15.3:0/3931817668 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f46e0017080 con 0x7f46e409d6f0 2023-10-03T10:17:36.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.787+0000 7f46ec57b700 1 -- 172.21.15.3:0/3931817668 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f46e406c360 con 0x7f46e409d6f0 2023-10-03T10:17:36.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.788+0000 7f46daffd700 1 -- 172.21.15.3:0/3931817668 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f46e0019d20 con 0x7f46e409d6f0 2023-10-03T10:17:36.788 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:36.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.790+0000 7f46ec57b700 1 -- 172.21.15.3:0/3931817668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46e409d6f0 msgr2=0x7f46e41c8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:36.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.790+0000 7f46ec57b700 1 --2- 172.21.15.3:0/3931817668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46e409d6f0 0x7f46e41c8280 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f46e000c550 tx=0x7f46e000c860 comp rx=0 tx=0).stop 2023-10-03T10:17:36.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.790+0000 7f46ec57b700 1 -- 172.21.15.3:0/3931817668 shutdown_connections 2023-10-03T10:17:36.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.790+0000 7f46ec57b700 1 --2- 172.21.15.3:0/3931817668 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46e4135e50 0x7f46e41c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:36.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.790+0000 7f46ec57b700 1 --2- 172.21.15.3:0/3931817668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46e409d6f0 0x7f46e41c8280 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:36.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.790+0000 7f46ec57b700 1 --2- 172.21.15.3:0/3931817668 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46e409e1d0 0x7f46e41c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:36.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.790+0000 7f46ec57b700 1 -- 172.21.15.3:0/3931817668 >> 172.21.15.3:0/3931817668 conn(0x7f46e4066f50 msgr2=0x7f46e41339c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:36.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.790+0000 7f46ec57b700 1 -- 172.21.15.3:0/3931817668 shutdown_connections 2023-10-03T10:17:36.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:36.790+0000 7f46ec57b700 1 -- 172.21.15.3:0/3931817668 wait complete. 2023-10-03T10:17:36.804 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:36.975 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:37.031 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:37.063 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:37.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.124+0000 7f12f6fd8700 1 Processor -- start 2023-10-03T10:17:37.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.124+0000 7f12f6fd8700 1 -- start start 2023-10-03T10:17:37.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.125+0000 7f12f6fd8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12f0133420 0x7f12f01337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:37.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.125+0000 7f12f6fd8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12f0134260 0x7f12f01306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:37.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.125+0000 7f12f6fd8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12f006b990 con 0x7f12f0133e30 2023-10-03T10:17:37.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.125+0000 7f12f6fd8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12f00a0590 con 0x7f12f0133420 2023-10-03T10:17:37.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.125+0000 7f12f6fd8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12f009fb80 con 0x7f12f0134260 2023-10-03T10:17:37.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.125+0000 7f12f5575700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12f0134260 0x7f12f01306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:37.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.125+0000 7f12effff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f12f0133e30 0x7f12f012ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33052/0 (socket says 172.21.15.3:33052) 2023-10-03T10:17:37.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.125+0000 7f12f4d74700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12f0133420 0x7f12f01337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:37.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.125+0000 7f12effff700 1 -- 172.21.15.3:0/2537533985 learned_addr learned my addr 172.21.15.3:0/2537533985 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:37.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.125+0000 7f12f4d74700 1 -- 172.21.15.3:0/2537533985 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12f0134260 msgr2=0x7f12f01306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:37.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.125+0000 7f12f4d74700 1 --2- 172.21.15.3:0/2537533985 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12f0134260 0x7f12f01306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.125+0000 7f12f4d74700 1 -- 172.21.15.3:0/2537533985 >> v1:172.21.15.3:6789/0 conn(0x7f12f0133e30 legacy=0x7f12f012ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:37.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.125+0000 7f12ef7fe700 1 -- 172.21.15.3:0/2537533985 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 756561712 0 0) 0x7f12f006b990 con 0x7f12f0133e30 2023-10-03T10:17:37.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.126+0000 7f12f4d74700 1 -- 172.21.15.3:0/2537533985 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f00a0030 con 0x7f12f0133420 2023-10-03T10:17:37.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.126+0000 7f12f5575700 1 --2- 172.21.15.3:0/2537533985 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12f0134260 0x7f12f01306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:37.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.126+0000 7f12f4d74700 1 --2- 172.21.15.3:0/2537533985 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12f0133420 0x7f12f01337f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f12e4009870 tx=0x7f12e4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7256712648e52628 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:37.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.126+0000 7f12ef7fe700 1 -- 172.21.15.3:0/2537533985 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e4011830 con 0x7f12f0133420 2023-10-03T10:17:37.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.126+0000 7f12ef7fe700 1 -- 172.21.15.3:0/2537533985 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e4011990 con 0x7f12f0133420 2023-10-03T10:17:37.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.126+0000 7f12ef7fe700 1 -- 172.21.15.3:0/2537533985 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e4011d20 con 0x7f12f0133420 2023-10-03T10:17:37.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.127+0000 7f12f6fd8700 1 -- 172.21.15.3:0/2537533985 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12f0133420 msgr2=0x7f12f01337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:37.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.127+0000 7f12f6fd8700 1 --2- 172.21.15.3:0/2537533985 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12f0133420 0x7f12f01337f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f12e4009870 tx=0x7f12e4009b80 comp rx=0 tx=0).stop 2023-10-03T10:17:37.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.127+0000 7f12f6fd8700 1 -- 172.21.15.3:0/2537533985 shutdown_connections 2023-10-03T10:17:37.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.127+0000 7f12f6fd8700 1 --2- 172.21.15.3:0/2537533985 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12f0133420 0x7f12f01337f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.127+0000 7f12f6fd8700 1 --2- 172.21.15.3:0/2537533985 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12f0134260 0x7f12f01306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.127+0000 7f12f6fd8700 1 -- 172.21.15.3:0/2537533985 >> 172.21.15.3:0/2537533985 conn(0x7f12f00a4210 msgr2=0x7f12f00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:37.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.127+0000 7f12f6fd8700 1 -- 172.21.15.3:0/2537533985 shutdown_connections 2023-10-03T10:17:37.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.127+0000 7f12f6fd8700 1 -- 172.21.15.3:0/2537533985 wait complete. 2023-10-03T10:17:37.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.128+0000 7f12f6fd8700 1 Processor -- start 2023-10-03T10:17:37.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.128+0000 7f12f6fd8700 1 -- start start 2023-10-03T10:17:37.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.129+0000 7f12f6fd8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12f0133420 0x7f12f009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:37.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.129+0000 7f12f6fd8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12f0133e30 0x7f12f009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:37.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.129+0000 7f12f6fd8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12f0134260 0x7f12f009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:37.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.129+0000 7f12f6fd8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f0120340 con 0x7f12f0133420 2023-10-03T10:17:37.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.129+0000 7f12f6fd8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12f0120e70 con 0x7f12f0133e30 2023-10-03T10:17:37.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.129+0000 7f12f6fd8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12f011fc50 con 0x7f12f0134260 2023-10-03T10:17:37.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.129+0000 7f12effff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12f0133e30 0x7f12f009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:37.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.129+0000 7f12effff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12f0133e30 0x7f12f009c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38808/0 (socket says 172.21.15.3:38808) 2023-10-03T10:17:37.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.129+0000 7f12effff700 1 -- 172.21.15.3:0/107278946 learned_addr learned my addr 172.21.15.3:0/107278946 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:37.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.129+0000 7f12f5575700 1 --2- 172.21.15.3:0/107278946 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12f0134260 0x7f12f009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:37.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.129+0000 7f12f4d74700 1 --2- 172.21.15.3:0/107278946 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12f0133420 0x7f12f009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:37.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.129+0000 7f12f5575700 1 -- 172.21.15.3:0/107278946 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12f0133e30 msgr2=0x7f12f009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:37.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.129+0000 7f12f5575700 1 --2- 172.21.15.3:0/107278946 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12f0133e30 0x7f12f009c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.129+0000 7f12f5575700 1 -- 172.21.15.3:0/107278946 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12f0133420 msgr2=0x7f12f009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:37.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.129+0000 7f12f5575700 1 --2- 172.21.15.3:0/107278946 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12f0133420 0x7f12f009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.129+0000 7f12f5575700 1 -- 172.21.15.3:0/107278946 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e8009a70 con 0x7f12f0134260 2023-10-03T10:17:37.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.129+0000 7f12effff700 1 --2- 172.21.15.3:0/107278946 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12f0133e30 0x7f12f009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:37.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.129+0000 7f12f4d74700 1 --2- 172.21.15.3:0/107278946 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12f0133420 0x7f12f009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:37.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.130+0000 7f12f5575700 1 --2- 172.21.15.3:0/107278946 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12f0134260 0x7f12f009cf00 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f12e800ef90 tx=0x7f12e800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:37.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.130+0000 7f12ed7fa700 1 -- 172.21.15.3:0/107278946 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e800ca90 con 0x7f12f0134260 2023-10-03T10:17:37.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.130+0000 7f12ed7fa700 1 -- 172.21.15.3:0/107278946 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e800f040 con 0x7f12f0134260 2023-10-03T10:17:37.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.130+0000 7f12f6fd8700 1 -- 172.21.15.3:0/107278946 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e40094a0 con 0x7f12f0134260 2023-10-03T10:17:37.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.130+0000 7f12ed7fa700 1 -- 172.21.15.3:0/107278946 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e800ce30 con 0x7f12f0134260 2023-10-03T10:17:37.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.130+0000 7f12f6fd8700 1 -- 172.21.15.3:0/107278946 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12f012dbd0 con 0x7f12f0134260 2023-10-03T10:17:37.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.131+0000 7f12ed7fa700 1 -- 172.21.15.3:0/107278946 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f12e8014480 con 0x7f12f0134260 2023-10-03T10:17:37.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.131+0000 7f12f6fd8700 1 -- 172.21.15.3:0/107278946 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12f0120150 con 0x7f12f0134260 2023-10-03T10:17:37.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.131+0000 7f12ed7fa700 1 -- 172.21.15.3:0/107278946 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f12e803fd70 con 0x7f12f0134260 2023-10-03T10:17:37.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.136+0000 7f12ed7fa700 1 -- 172.21.15.3:0/107278946 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f12e8012910 con 0x7f12f0134260 2023-10-03T10:17:37.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.177+0000 7f5df467a700 1 Processor -- start 2023-10-03T10:17:37.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.177+0000 7f5df467a700 1 -- start start 2023-10-03T10:17:37.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.178+0000 7f5df467a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5dec0a1510 0x7f5dec138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.178+0000 7f5df467a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5dec139250 0x7f5dec13b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.178+0000 7f5df467a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dec09edd0 con 0x7f5dec0a0a30 2023-10-03T10:17:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.178+0000 7f5df467a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dec0a18e0 con 0x7f5dec139250 2023-10-03T10:17:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.178+0000 7f5df467a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec09e1f0 con 0x7f5dec0a1510 2023-10-03T10:17:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.178+0000 7f5df1c15700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5dec0a1510 0x7f5dec138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.178+0000 7f5df2c17700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5dec139250 0x7f5dec13b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.178+0000 7f5df2c17700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5dec139250 0x7f5dec13b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39684/0 (socket says 172.21.15.3:39684) 2023-10-03T10:17:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.178+0000 7f5df1c15700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5dec0a1510 0x7f5dec138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38816/0 (socket says 172.21.15.3:38816) 2023-10-03T10:17:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.178+0000 7f5df2c17700 1 -- 172.21.15.3:0/2798756257 learned_addr learned my addr 172.21.15.3:0/2798756257 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:37.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.179+0000 7f5df1c15700 1 -- 172.21.15.3:0/2798756257 >> v2:172.21.15.3:3300/0 conn(0x7f5dec139250 msgr2=0x7f5dec13b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.179+0000 7f5df1c15700 1 --2- 172.21.15.3:0/2798756257 >> v2:172.21.15.3:3300/0 conn(0x7f5dec139250 0x7f5dec13b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.179+0000 7f5df1c15700 1 -- 172.21.15.3:0/2798756257 >> v1:172.21.15.3:6789/0 conn(0x7f5dec0a0a30 legacy=0x7f5dec0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.179+0000 7f5df1c15700 1 -- 172.21.15.3:0/2798756257 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dec09dc50 con 0x7f5dec0a1510 2023-10-03T10:17:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.179+0000 7f5df2c17700 1 --2- 172.21.15.3:0/2798756257 >> v2:172.21.15.3:3300/0 conn(0x7f5dec139250 0x7f5dec13b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.179+0000 7f5df1c15700 1 --2- 172.21.15.3:0/2798756257 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5dec0a1510 0x7f5dec138d10 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009870 tx=0x7f5ddc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc4720ac724b1964 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.179+0000 7f5df1414700 1 -- 172.21.15.3:0/2798756257 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc011830 con 0x7f5dec0a1510 2023-10-03T10:17:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.179+0000 7f5df1414700 1 -- 172.21.15.3:0/2798756257 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ddc011990 con 0x7f5dec0a1510 2023-10-03T10:17:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.179+0000 7f5df1414700 1 -- 172.21.15.3:0/2798756257 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc011d20 con 0x7f5dec0a1510 2023-10-03T10:17:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.180+0000 7f5df467a700 1 -- 172.21.15.3:0/2798756257 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5dec0a1510 msgr2=0x7f5dec138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.180+0000 7f5df467a700 1 --2- 172.21.15.3:0/2798756257 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5dec0a1510 0x7f5dec138d10 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009870 tx=0x7f5ddc009b80 comp rx=0 tx=0).stop 2023-10-03T10:17:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.180+0000 7f5df467a700 1 -- 172.21.15.3:0/2798756257 shutdown_connections 2023-10-03T10:17:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.180+0000 7f5df467a700 1 --2- 172.21.15.3:0/2798756257 >> v2:172.21.15.3:3300/0 conn(0x7f5dec139250 0x7f5dec13b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.180+0000 7f5df467a700 1 --2- 172.21.15.3:0/2798756257 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5dec0a1510 0x7f5dec138d10 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.180+0000 7f5df467a700 1 -- 172.21.15.3:0/2798756257 >> 172.21.15.3:0/2798756257 conn(0x7f5dec0a6550 msgr2=0x7f5dec0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:37.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.180+0000 7f5df467a700 1 -- 172.21.15.3:0/2798756257 shutdown_connections 2023-10-03T10:17:37.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.180+0000 7f5df467a700 1 -- 172.21.15.3:0/2798756257 wait complete. 2023-10-03T10:17:37.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.181+0000 7f5df467a700 1 Processor -- start 2023-10-03T10:17:37.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.181+0000 7f5df467a700 1 -- start start 2023-10-03T10:17:37.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.182+0000 7f5df467a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5dec0a0a30 0x7f5dec1c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:37.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.182+0000 7f5df467a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5dec0a1510 0x7f5dec1c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:37.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.182+0000 7f5df467a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dec139250 0x7f5dec122a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:37.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.182+0000 7f5df467a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec1389b0 con 0x7f5dec139250 2023-10-03T10:17:37.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.182+0000 7f5df467a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec125cd0 con 0x7f5dec0a0a30 2023-10-03T10:17:37.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.182+0000 7f5df467a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec126710 con 0x7f5dec0a1510 2023-10-03T10:17:37.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.182+0000 7f5df2416700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5dec0a0a30 0x7f5dec1c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:37.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.182+0000 7f5df2416700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5dec0a0a30 0x7f5dec1c8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38822/0 (socket says 172.21.15.3:38822) 2023-10-03T10:17:37.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.182+0000 7f5df2416700 1 -- 172.21.15.3:0/3779135289 learned_addr learned my addr 172.21.15.3:0/3779135289 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:37.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.182+0000 7f5df2c17700 1 --2- 172.21.15.3:0/3779135289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dec139250 0x7f5dec122a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:37.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.182+0000 7f5df1c15700 1 --2- 172.21.15.3:0/3779135289 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5dec0a1510 0x7f5dec1c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:37.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.182+0000 7f5df2416700 1 -- 172.21.15.3:0/3779135289 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5dec0a1510 msgr2=0x7f5dec1c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:37.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.182+0000 7f5df2416700 1 --2- 172.21.15.3:0/3779135289 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5dec0a1510 0x7f5dec1c8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.182+0000 7f5df2416700 1 -- 172.21.15.3:0/3779135289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dec139250 msgr2=0x7f5dec122a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:37.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.182+0000 7f5df2416700 1 --2- 172.21.15.3:0/3779135289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dec139250 0x7f5dec122a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.182+0000 7f5df2416700 1 -- 172.21.15.3:0/3779135289 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de4009b30 con 0x7f5dec0a0a30 2023-10-03T10:17:37.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.182+0000 7f5df2c17700 1 --2- 172.21.15.3:0/3779135289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dec139250 0x7f5dec122a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:37.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.183+0000 7f5df2416700 1 --2- 172.21.15.3:0/3779135289 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5dec0a0a30 0x7f5dec1c8800 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5de800c4e0 tx=0x7f5de800c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:37.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.183+0000 7f5de2ffd700 1 -- 172.21.15.3:0/3779135289 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de80052d0 con 0x7f5dec0a0a30 2023-10-03T10:17:37.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.183+0000 7f5de2ffd700 1 -- 172.21.15.3:0/3779135289 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de8005430 con 0x7f5dec0a0a30 2023-10-03T10:17:37.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.183+0000 7f5df467a700 1 -- 172.21.15.3:0/3779135289 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ddc0094a0 con 0x7f5dec0a0a30 2023-10-03T10:17:37.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.183+0000 7f5de2ffd700 1 -- 172.21.15.3:0/3779135289 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de80038c0 con 0x7f5dec0a0a30 2023-10-03T10:17:37.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.183+0000 7f5df467a700 1 -- 172.21.15.3:0/3779135289 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dec125fa0 con 0x7f5dec0a0a30 2023-10-03T10:17:37.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.184+0000 7f5df467a700 1 -- 172.21.15.3:0/3779135289 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dec1c92e0 con 0x7f5dec0a0a30 2023-10-03T10:17:37.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.184+0000 7f5de2ffd700 1 -- 172.21.15.3:0/3779135289 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5de8003a20 con 0x7f5dec0a0a30 2023-10-03T10:17:37.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.185+0000 7f5de2ffd700 1 -- 172.21.15.3:0/3779135289 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5de803ef10 con 0x7f5dec0a0a30 2023-10-03T10:17:37.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.193+0000 7f5de2ffd700 1 -- 172.21.15.3:0/3779135289 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5de8018260 con 0x7f5dec0a0a30 2023-10-03T10:17:37.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.217+0000 7f6eeb6fc700 1 Processor -- start 2023-10-03T10:17:37.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.218+0000 7f6eeb6fc700 1 -- start start 2023-10-03T10:17:37.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.218+0000 7f6eeb6fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6ee40a0a30 0x7f6ee40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:37.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.218+0000 7f6eeb6fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ee4138d80 0x7f6ee413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:37.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.218+0000 7f6eeb6fc700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee40a2a70 con 0x7f6ee409d630 2023-10-03T10:17:37.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.218+0000 7f6eeb6fc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ee40a01d0 con 0x7f6ee40a0a30 2023-10-03T10:17:37.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.218+0000 7f6eeb6fc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee409e2b0 con 0x7f6ee4138d80 2023-10-03T10:17:37.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.218+0000 7f6ee9498700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6ee40a0a30 0x7f6ee40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:37.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.218+0000 7f6ee9498700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6ee40a0a30 0x7f6ee40a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39700/0 (socket says 172.21.15.3:39700) 2023-10-03T10:17:37.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.218+0000 7f6ee9c99700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ee4138d80 0x7f6ee413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:37.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.218+0000 7f6ee9498700 1 -- 172.21.15.3:0/1356190978 learned_addr learned my addr 172.21.15.3:0/1356190978 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:37.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.219+0000 7f6ee9c99700 1 -- 172.21.15.3:0/1356190978 >> v2:172.21.15.3:3300/0 conn(0x7f6ee40a0a30 msgr2=0x7f6ee40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:37.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.219+0000 7f6ee9c99700 1 --2- 172.21.15.3:0/1356190978 >> v2:172.21.15.3:3300/0 conn(0x7f6ee40a0a30 0x7f6ee40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.219+0000 7f6ee9c99700 1 -- 172.21.15.3:0/1356190978 >> v1:172.21.15.3:6789/0 conn(0x7f6ee409d630 legacy=0x7f6ee40a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:37.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.219+0000 7f6ee9c99700 1 -- 172.21.15.3:0/1356190978 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee409dc50 con 0x7f6ee4138d80 2023-10-03T10:17:37.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.219+0000 7f6ee9498700 1 --2- 172.21.15.3:0/1356190978 >> v2:172.21.15.3:3300/0 conn(0x7f6ee40a0a30 0x7f6ee40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:37.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.219+0000 7f6ee9c99700 1 --2- 172.21.15.3:0/1356190978 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ee4138d80 0x7f6ee413b160 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6edc009d90 tx=0x7f6edc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b025e4514ecc75ef server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:37.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.219+0000 7f6edbfff700 1 -- 172.21.15.3:0/1356190978 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc00e070 con 0x7f6ee4138d80 2023-10-03T10:17:37.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.219+0000 7f6edbfff700 1 -- 172.21.15.3:0/1356190978 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6edc00bb20 con 0x7f6ee4138d80 2023-10-03T10:17:37.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.219+0000 7f6edbfff700 1 -- 172.21.15.3:0/1356190978 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc012430 con 0x7f6ee4138d80 2023-10-03T10:17:37.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.220+0000 7f6eeb6fc700 1 -- 172.21.15.3:0/1356190978 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ee4138d80 msgr2=0x7f6ee413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:37.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.220+0000 7f6eeb6fc700 1 --2- 172.21.15.3:0/1356190978 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ee4138d80 0x7f6ee413b160 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6edc009d90 tx=0x7f6edc00b330 comp rx=0 tx=0).stop 2023-10-03T10:17:37.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.220+0000 7f6eeb6fc700 1 -- 172.21.15.3:0/1356190978 shutdown_connections 2023-10-03T10:17:37.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.220+0000 7f6eeb6fc700 1 --2- 172.21.15.3:0/1356190978 >> v2:172.21.15.3:3300/0 conn(0x7f6ee40a0a30 0x7f6ee40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.220+0000 7f6eeb6fc700 1 --2- 172.21.15.3:0/1356190978 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ee4138d80 0x7f6ee413b160 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.220+0000 7f6eeb6fc700 1 -- 172.21.15.3:0/1356190978 >> 172.21.15.3:0/1356190978 conn(0x7f6ee40a3050 msgr2=0x7f6ee40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:37.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.220+0000 7f6eeb6fc700 1 -- 172.21.15.3:0/1356190978 shutdown_connections 2023-10-03T10:17:37.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.220+0000 7f6eeb6fc700 1 -- 172.21.15.3:0/1356190978 wait complete. 2023-10-03T10:17:37.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.221+0000 7f6eeb6fc700 1 Processor -- start 2023-10-03T10:17:37.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.221+0000 7f6eeb6fc700 1 -- start start 2023-10-03T10:17:37.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.222+0000 7f6eeb6fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ee409d630 0x7f6ee41339e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:37.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.222+0000 7f6eeb6fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ee40a0a30 0x7f6ee4133f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:37.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.222+0000 7f6eeb6fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ee4138d80 0x7f6ee4134460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:37.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.222+0000 7f6eeb6fc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee4136ed0 con 0x7f6ee40a0a30 2023-10-03T10:17:37.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.222+0000 7f6eeb6fc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee4138850 con 0x7f6ee4138d80 2023-10-03T10:17:37.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.222+0000 7f6eeb6fc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee4138160 con 0x7f6ee409d630 2023-10-03T10:17:37.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.222+0000 7f6ee8c97700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ee40a0a30 0x7f6ee4133f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:37.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.222+0000 7f6ee8c97700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ee40a0a30 0x7f6ee4133f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39702/0 (socket says 172.21.15.3:39702) 2023-10-03T10:17:37.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.222+0000 7f6ee8c97700 1 -- 172.21.15.3:0/191639993 learned_addr learned my addr 172.21.15.3:0/191639993 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:37.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.222+0000 7f6ee9498700 1 --2- 172.21.15.3:0/191639993 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ee409d630 0x7f6ee41339e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:37.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.222+0000 7f6ee9c99700 1 --2- 172.21.15.3:0/191639993 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ee4138d80 0x7f6ee4134460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:37.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.222+0000 7f6ee8c97700 1 -- 172.21.15.3:0/191639993 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ee409d630 msgr2=0x7f6ee41339e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:37.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.222+0000 7f6ee8c97700 1 --2- 172.21.15.3:0/191639993 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ee409d630 0x7f6ee41339e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.222+0000 7f6ee8c97700 1 -- 172.21.15.3:0/191639993 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ee4138d80 msgr2=0x7f6ee4134460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:37.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.222+0000 7f6ee8c97700 1 --2- 172.21.15.3:0/191639993 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ee4138d80 0x7f6ee4134460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.222+0000 7f6ee8c97700 1 -- 172.21.15.3:0/191639993 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee0009550 con 0x7f6ee40a0a30 2023-10-03T10:17:37.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.222+0000 7f6ee8c97700 1 --2- 172.21.15.3:0/191639993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ee40a0a30 0x7f6ee4133f20 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400c4e0 tx=0x7f6ed400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:37.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.223+0000 7f6ed9ffb700 1 -- 172.21.15.3:0/191639993 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed40052d0 con 0x7f6ee40a0a30 2023-10-03T10:17:37.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.223+0000 7f6ed9ffb700 1 -- 172.21.15.3:0/191639993 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed4005430 con 0x7f6ee40a0a30 2023-10-03T10:17:37.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.223+0000 7f6eeb6fc700 1 -- 172.21.15.3:0/191639993 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6edc009a70 con 0x7f6ee40a0a30 2023-10-03T10:17:37.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.223+0000 7f6ed9ffb700 1 -- 172.21.15.3:0/191639993 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed40038c0 con 0x7f6ee40a0a30 2023-10-03T10:17:37.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.223+0000 7f6eeb6fc700 1 -- 172.21.15.3:0/191639993 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee4138390 con 0x7f6ee40a0a30 2023-10-03T10:17:37.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.224+0000 7f6ed9ffb700 1 -- 172.21.15.3:0/191639993 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6ed4003a20 con 0x7f6ee40a0a30 2023-10-03T10:17:37.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.224+0000 7f6ed9ffb700 1 -- 172.21.15.3:0/191639993 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6ed403ea50 con 0x7f6ee40a0a30 2023-10-03T10:17:37.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.224+0000 7f6eeb6fc700 1 -- 172.21.15.3:0/191639993 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ee40a26f0 con 0x7f6ee40a0a30 2023-10-03T10:17:37.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.229+0000 7f6ed9ffb700 1 -- 172.21.15.3:0/191639993 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6ed4017080 con 0x7f6ee40a0a30 2023-10-03T10:17:37.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.368+0000 7f12f6fd8700 1 -- 172.21.15.3:0/107278946 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f12f0074ea0 con 0x7f12f0134260 2023-10-03T10:17:37.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.368+0000 7f12ed7fa700 1 -- 172.21.15.3:0/107278946 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f12e8009cc0 con 0x7f12f0134260 2023-10-03T10:17:37.369 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:37.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.371+0000 7f12f6fd8700 1 -- 172.21.15.3:0/107278946 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12f0134260 msgr2=0x7f12f009cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:37.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.371+0000 7f12f6fd8700 1 --2- 172.21.15.3:0/107278946 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12f0134260 0x7f12f009cf00 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f12e800ef90 tx=0x7f12e800c3a0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.371+0000 7f12f6fd8700 1 -- 172.21.15.3:0/107278946 shutdown_connections 2023-10-03T10:17:37.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.371+0000 7f12f6fd8700 1 --2- 172.21.15.3:0/107278946 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12f0134260 0x7f12f009cf00 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.371+0000 7f12f6fd8700 1 --2- 172.21.15.3:0/107278946 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12f0133420 0x7f12f009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.371+0000 7f12f6fd8700 1 --2- 172.21.15.3:0/107278946 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12f0133e30 0x7f12f009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.371+0000 7f12f6fd8700 1 -- 172.21.15.3:0/107278946 >> 172.21.15.3:0/107278946 conn(0x7f12f00a4210 msgr2=0x7f12f00a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:37.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.371+0000 7f12f6fd8700 1 -- 172.21.15.3:0/107278946 shutdown_connections 2023-10-03T10:17:37.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.371+0000 7f12f6fd8700 1 -- 172.21.15.3:0/107278946 wait complete. 2023-10-03T10:17:37.385 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:37.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.409+0000 7f5df467a700 1 -- 172.21.15.3:0/3779135289 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5dec126190 con 0x7f5dec0a0a30 2023-10-03T10:17:37.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.409+0000 7f5de2ffd700 1 -- 172.21.15.3:0/3779135289 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5de8017030 con 0x7f5dec0a0a30 2023-10-03T10:17:37.409 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:37.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.411+0000 7f5df467a700 1 -- 172.21.15.3:0/3779135289 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5dec0a0a30 msgr2=0x7f5dec1c8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:37.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.411+0000 7f5df467a700 1 --2- 172.21.15.3:0/3779135289 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5dec0a0a30 0x7f5dec1c8800 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5de800c4e0 tx=0x7f5de800c7f0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.411+0000 7f5df467a700 1 -- 172.21.15.3:0/3779135289 shutdown_connections 2023-10-03T10:17:37.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.411+0000 7f5df467a700 1 --2- 172.21.15.3:0/3779135289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dec139250 0x7f5dec122a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.411+0000 7f5df467a700 1 --2- 172.21.15.3:0/3779135289 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5dec0a0a30 0x7f5dec1c8800 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.411+0000 7f5df467a700 1 --2- 172.21.15.3:0/3779135289 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5dec0a1510 0x7f5dec1c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.411+0000 7f5df467a700 1 -- 172.21.15.3:0/3779135289 >> 172.21.15.3:0/3779135289 conn(0x7f5dec0a6550 msgr2=0x7f5dec0a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:37.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.411+0000 7f5df467a700 1 -- 172.21.15.3:0/3779135289 shutdown_connections 2023-10-03T10:17:37.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.411+0000 7f5df467a700 1 -- 172.21.15.3:0/3779135289 wait complete. 2023-10-03T10:17:37.426 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:37.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.451+0000 7f6eeb6fc700 1 -- 172.21.15.3:0/191639993 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6ee4137080 con 0x7f6ee40a0a30 2023-10-03T10:17:37.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.451+0000 7f6ed9ffb700 1 -- 172.21.15.3:0/191639993 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6ee4137080 con 0x7f6ee40a0a30 2023-10-03T10:17:37.452 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:37.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.453+0000 7f6eeb6fc700 1 -- 172.21.15.3:0/191639993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ee40a0a30 msgr2=0x7f6ee4133f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:37.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.453+0000 7f6eeb6fc700 1 --2- 172.21.15.3:0/191639993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ee40a0a30 0x7f6ee4133f20 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400c4e0 tx=0x7f6ed400c7f0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.454+0000 7f6eeb6fc700 1 -- 172.21.15.3:0/191639993 shutdown_connections 2023-10-03T10:17:37.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.454+0000 7f6eeb6fc700 1 --2- 172.21.15.3:0/191639993 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ee4138d80 0x7f6ee4134460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.454+0000 7f6eeb6fc700 1 --2- 172.21.15.3:0/191639993 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ee409d630 0x7f6ee41339e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.454+0000 7f6eeb6fc700 1 --2- 172.21.15.3:0/191639993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ee40a0a30 0x7f6ee4133f20 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.454+0000 7f6eeb6fc700 1 -- 172.21.15.3:0/191639993 >> 172.21.15.3:0/191639993 conn(0x7f6ee40a3050 msgr2=0x7f6ee409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:37.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.454+0000 7f6eeb6fc700 1 -- 172.21.15.3:0/191639993 shutdown_connections 2023-10-03T10:17:37.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.454+0000 7f6eeb6fc700 1 -- 172.21.15.3:0/191639993 wait complete. 2023-10-03T10:17:37.467 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:37.805 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:37.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.952+0000 7f5878441700 1 Processor -- start 2023-10-03T10:17:37.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.952+0000 7f5878441700 1 -- start start 2023-10-03T10:17:37.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.952+0000 7f5878441700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f587009e930 0x7f587009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:37.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.952+0000 7f5878441700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f587009f240 0x7f58701317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:37.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.952+0000 7f5878441700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5870070940 con 0x7f587009f650 2023-10-03T10:17:37.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.952+0000 7f5878441700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f587006fd60 con 0x7f587009e930 2023-10-03T10:17:37.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.952+0000 7f5878441700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f587006fa80 con 0x7f587009f240 2023-10-03T10:17:37.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.952+0000 7f58759dc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f587009f240 0x7f58701317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:37.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.952+0000 7f58769de700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f587009f650 0x7f5870131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33080/0 (socket says 172.21.15.3:33080) 2023-10-03T10:17:37.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.952+0000 7f58769de700 1 -- 172.21.15.3:0/1050181543 learned_addr learned my addr 172.21.15.3:0/1050181543 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:37.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.952+0000 7f58761dd700 1 --2- 172.21.15.3:0/1050181543 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f587009e930 0x7f587009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:37.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.953+0000 7f58751db700 1 -- 172.21.15.3:0/1050181543 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 541172636 0 0) 0x7f5870070940 con 0x7f587009f650 2023-10-03T10:17:37.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.953+0000 7f58761dd700 1 -- 172.21.15.3:0/1050181543 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f587009f240 msgr2=0x7f58701317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:37.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.953+0000 7f58761dd700 1 --2- 172.21.15.3:0/1050181543 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f587009f240 0x7f58701317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.953+0000 7f58761dd700 1 -- 172.21.15.3:0/1050181543 >> v1:172.21.15.3:6789/0 conn(0x7f587009f650 legacy=0x7f5870131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:37.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.953+0000 7f58761dd700 1 -- 172.21.15.3:0/1050181543 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587009fb60 con 0x7f587009e930 2023-10-03T10:17:37.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.953+0000 7f58759dc700 1 --2- 172.21.15.3:0/1050181543 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f587009f240 0x7f58701317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:37.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.953+0000 7f58761dd700 1 --2- 172.21.15.3:0/1050181543 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f587009e930 0x7f587009ed00 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f586c009870 tx=0x7f586c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=656eba84fb75eb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:37.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.953+0000 7f58751db700 1 -- 172.21.15.3:0/1050181543 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c011830 con 0x7f587009e930 2023-10-03T10:17:37.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.953+0000 7f58751db700 1 -- 172.21.15.3:0/1050181543 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f586c011990 con 0x7f587009e930 2023-10-03T10:17:37.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.953+0000 7f58751db700 1 -- 172.21.15.3:0/1050181543 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c011d20 con 0x7f587009e930 2023-10-03T10:17:37.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.954+0000 7f5878441700 1 -- 172.21.15.3:0/1050181543 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f587009e930 msgr2=0x7f587009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:37.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.954+0000 7f5878441700 1 --2- 172.21.15.3:0/1050181543 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f587009e930 0x7f587009ed00 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f586c009870 tx=0x7f586c009b80 comp rx=0 tx=0).stop 2023-10-03T10:17:37.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.954+0000 7f5878441700 1 -- 172.21.15.3:0/1050181543 shutdown_connections 2023-10-03T10:17:37.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.954+0000 7f5878441700 1 --2- 172.21.15.3:0/1050181543 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f587009e930 0x7f587009ed00 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.954+0000 7f5878441700 1 --2- 172.21.15.3:0/1050181543 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f587009f240 0x7f58701317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.954+0000 7f5878441700 1 -- 172.21.15.3:0/1050181543 >> 172.21.15.3:0/1050181543 conn(0x7f58700a4210 msgr2=0x7f58700a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:37.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.954+0000 7f5878441700 1 -- 172.21.15.3:0/1050181543 shutdown_connections 2023-10-03T10:17:37.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.954+0000 7f5878441700 1 -- 172.21.15.3:0/1050181543 wait complete. 2023-10-03T10:17:37.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.955+0000 7f5878441700 1 Processor -- start 2023-10-03T10:17:37.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.955+0000 7f5878441700 1 -- start start 2023-10-03T10:17:37.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.955+0000 7f5878441700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587009e930 0x7f58701a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:37.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.955+0000 7f5878441700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f587009f240 0x7f58701a4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:37.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.955+0000 7f5878441700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f587009f650 0x7f58701a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:37.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.956+0000 7f5878441700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58701a7650 con 0x7f587009e930 2023-10-03T10:17:37.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.956+0000 7f5878441700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58701a6a20 con 0x7f587009f650 2023-10-03T10:17:37.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.956+0000 7f5878441700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f58700a3410 con 0x7f587009f240 2023-10-03T10:17:37.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.956+0000 7f58761dd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587009e930 0x7f58701a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:37.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.956+0000 7f58769de700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f587009f650 0x7f58701a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:37.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.956+0000 7f58759dc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f587009f240 0x7f58701a4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:37.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.956+0000 7f58761dd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587009e930 0x7f58701a3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39716/0 (socket says 172.21.15.3:39716) 2023-10-03T10:17:37.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.956+0000 7f58761dd700 1 -- 172.21.15.3:0/2963151042 learned_addr learned my addr 172.21.15.3:0/2963151042 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:37.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.956+0000 7f58761dd700 1 -- 172.21.15.3:0/2963151042 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f587009f240 msgr2=0x7f58701a4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:37.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.956+0000 7f58761dd700 1 --2- 172.21.15.3:0/2963151042 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f587009f240 0x7f58701a4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.956+0000 7f58761dd700 1 -- 172.21.15.3:0/2963151042 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f587009f650 msgr2=0x7f58701a4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:37.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.956+0000 7f58761dd700 1 --2- 172.21.15.3:0/2963151042 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f587009f650 0x7f58701a4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:37.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.956+0000 7f58761dd700 1 -- 172.21.15.3:0/2963151042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58600094a0 con 0x7f587009e930 2023-10-03T10:17:37.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.956+0000 7f58759dc700 1 --2- 172.21.15.3:0/2963151042 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f587009f240 0x7f58701a4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:37.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.956+0000 7f58769de700 1 --2- 172.21.15.3:0/2963151042 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f587009f650 0x7f58701a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:37.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.956+0000 7f58761dd700 1 --2- 172.21.15.3:0/2963151042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587009e930 0x7f58701a3bd0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f586c0090b0 tx=0x7f586c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:37.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.956+0000 7f5866ffd700 1 -- 172.21.15.3:0/2963151042 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c004510 con 0x7f587009e930 2023-10-03T10:17:37.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.956+0000 7f5866ffd700 1 -- 172.21.15.3:0/2963151042 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f586c01f070 con 0x7f587009e930 2023-10-03T10:17:37.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.956+0000 7f5878441700 1 -- 172.21.15.3:0/2963151042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f586c0094a0 con 0x7f587009e930 2023-10-03T10:17:37.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.956+0000 7f5866ffd700 1 -- 172.21.15.3:0/2963151042 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c0038c0 con 0x7f587009e930 2023-10-03T10:17:37.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.957+0000 7f5878441700 1 -- 172.21.15.3:0/2963151042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58700a0190 con 0x7f587009e930 2023-10-03T10:17:37.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.957+0000 7f5866ffd700 1 -- 172.21.15.3:0/2963151042 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f586c003a20 con 0x7f587009e930 2023-10-03T10:17:37.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.957+0000 7f5866ffd700 1 -- 172.21.15.3:0/2963151042 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f586c021070 con 0x7f587009e930 2023-10-03T10:17:37.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.958+0000 7f5878441700 1 -- 172.21.15.3:0/2963151042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58701a0b30 con 0x7f587009e930 2023-10-03T10:17:37.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:37.963+0000 7f5866ffd700 1 -- 172.21.15.3:0/2963151042 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f586c015020 con 0x7f587009e930 2023-10-03T10:17:38.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.167+0000 7f5878441700 1 -- 172.21.15.3:0/2963151042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5870071d20 con 0x7f587009e930 2023-10-03T10:17:38.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.168+0000 7f5866ffd700 1 -- 172.21.15.3:0/2963151042 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f586c01a430 con 0x7f587009e930 2023-10-03T10:17:38.168 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:38.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.170+0000 7f5878441700 1 -- 172.21.15.3:0/2963151042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587009e930 msgr2=0x7f58701a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:38.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.170+0000 7f5878441700 1 --2- 172.21.15.3:0/2963151042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587009e930 0x7f58701a3bd0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f586c0090b0 tx=0x7f586c004750 comp rx=0 tx=0).stop 2023-10-03T10:17:38.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.170+0000 7f5878441700 1 -- 172.21.15.3:0/2963151042 shutdown_connections 2023-10-03T10:17:38.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.170+0000 7f5878441700 1 --2- 172.21.15.3:0/2963151042 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f587009f650 0x7f58701a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.170+0000 7f5878441700 1 --2- 172.21.15.3:0/2963151042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587009e930 0x7f58701a3bd0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.170+0000 7f5878441700 1 --2- 172.21.15.3:0/2963151042 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f587009f240 0x7f58701a4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.170+0000 7f5878441700 1 -- 172.21.15.3:0/2963151042 >> 172.21.15.3:0/2963151042 conn(0x7f58700a4210 msgr2=0x7f58701b8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:38.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.170+0000 7f5878441700 1 -- 172.21.15.3:0/2963151042 shutdown_connections 2023-10-03T10:17:38.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.170+0000 7f5878441700 1 -- 172.21.15.3:0/2963151042 wait complete. 2023-10-03T10:17:38.184 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:38.386 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:38.427 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:38.468 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:38.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.537+0000 7f3705d66700 1 Processor -- start 2023-10-03T10:17:38.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.537+0000 7f3705d66700 1 -- start start 2023-10-03T10:17:38.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.538+0000 7f3705d66700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3700131630 0x7f3700131a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:38.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.538+0000 7f3705d66700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f370006eba0 0x7f3700139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:38.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.538+0000 7f3705d66700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3700071a00 con 0x7f3700131f40 2023-10-03T10:17:38.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.538+0000 7f3705d66700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3700131430 con 0x7f3700131630 2023-10-03T10:17:38.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.538+0000 7f3705d66700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f370009a510 con 0x7f370006eba0 2023-10-03T10:17:38.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.538+0000 7f36ff7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3700131630 0x7f3700131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:38.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.538+0000 7f36fffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f370006eba0 0x7f3700139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:38.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.538+0000 7f36ff7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3700131630 0x7f3700131a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39732/0 (socket says 172.21.15.3:39732) 2023-10-03T10:17:38.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.538+0000 7f36ff7fe700 1 -- 172.21.15.3:0/1654374969 learned_addr learned my addr 172.21.15.3:0/1654374969 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:38.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.538+0000 7f36ff7fe700 1 -- 172.21.15.3:0/1654374969 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f370006eba0 msgr2=0x7f3700139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:38.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.538+0000 7f36ff7fe700 1 --2- 172.21.15.3:0/1654374969 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f370006eba0 0x7f3700139660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.538+0000 7f36ff7fe700 1 -- 172.21.15.3:0/1654374969 >> v1:172.21.15.3:6789/0 conn(0x7f3700131f40 legacy=0x7f370006e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:38.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.538+0000 7f36ff7fe700 1 -- 172.21.15.3:0/1654374969 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37000721c0 con 0x7f3700131630 2023-10-03T10:17:38.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.538+0000 7f36fffff700 1 --2- 172.21.15.3:0/1654374969 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f370006eba0 0x7f3700139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:38.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.539+0000 7f36ff7fe700 1 --2- 172.21.15.3:0/1654374969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3700131630 0x7f3700131a00 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f36f00098f0 tx=0x7f36f0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f9b9ed26565766 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:38.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.539+0000 7f36fe7fc700 1 -- 172.21.15.3:0/1654374969 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f0011830 con 0x7f3700131630 2023-10-03T10:17:38.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.539+0000 7f36fe7fc700 1 -- 172.21.15.3:0/1654374969 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f0011990 con 0x7f3700131630 2023-10-03T10:17:38.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.539+0000 7f36fe7fc700 1 -- 172.21.15.3:0/1654374969 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f0011d20 con 0x7f3700131630 2023-10-03T10:17:38.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.540+0000 7f3705d66700 1 -- 172.21.15.3:0/1654374969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3700131630 msgr2=0x7f3700131a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:38.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.540+0000 7f3705d66700 1 --2- 172.21.15.3:0/1654374969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3700131630 0x7f3700131a00 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f36f00098f0 tx=0x7f36f0009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:38.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.540+0000 7f3705d66700 1 -- 172.21.15.3:0/1654374969 shutdown_connections 2023-10-03T10:17:38.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.540+0000 7f3705d66700 1 --2- 172.21.15.3:0/1654374969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3700131630 0x7f3700131a00 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.540+0000 7f3705d66700 1 --2- 172.21.15.3:0/1654374969 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f370006eba0 0x7f3700139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.540+0000 7f3705d66700 1 -- 172.21.15.3:0/1654374969 >> 172.21.15.3:0/1654374969 conn(0x7f37000a2aa0 msgr2=0x7f37000a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:38.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.540+0000 7f3705d66700 1 -- 172.21.15.3:0/1654374969 shutdown_connections 2023-10-03T10:17:38.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.540+0000 7f3705d66700 1 -- 172.21.15.3:0/1654374969 wait complete. 2023-10-03T10:17:38.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.541+0000 7f3705d66700 1 Processor -- start 2023-10-03T10:17:38.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.541+0000 7f3705d66700 1 -- start start 2023-10-03T10:17:38.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.541+0000 7f3705d66700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f370006eba0 0x7f370011b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:38.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.541+0000 7f3705d66700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3700131630 0x7f370011ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:38.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.541+0000 7f3705d66700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3700131f40 0x7f370011bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:38.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.541+0000 7f3705d66700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370011fa10 con 0x7f3700131f40 2023-10-03T10:17:38.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.541+0000 7f3705d66700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370011f050 con 0x7f370006eba0 2023-10-03T10:17:38.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.542+0000 7f3705d66700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f370011e390 con 0x7f3700131630 2023-10-03T10:17:38.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.542+0000 7f36feffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3700131630 0x7f370011ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:38.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.542+0000 7f36ff7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f370006eba0 0x7f370011b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:38.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.542+0000 7f36ff7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f370006eba0 0x7f370011b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38856/0 (socket says 172.21.15.3:38856) 2023-10-03T10:17:38.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.542+0000 7f36ff7fe700 1 -- 172.21.15.3:0/1465003066 learned_addr learned my addr 172.21.15.3:0/1465003066 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:38.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.542+0000 7f36fffff700 1 --2- 172.21.15.3:0/1465003066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3700131f40 0x7f370011bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.542+0000 7f36feffd700 1 -- 172.21.15.3:0/1465003066 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f370006eba0 msgr2=0x7f370011b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.542+0000 7f36feffd700 1 --2- 172.21.15.3:0/1465003066 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f370006eba0 0x7f370011b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.542+0000 7f36feffd700 1 -- 172.21.15.3:0/1465003066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3700131f40 msgr2=0x7f370011bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.542+0000 7f36feffd700 1 --2- 172.21.15.3:0/1465003066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3700131f40 0x7f370011bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.542+0000 7f36feffd700 1 -- 172.21.15.3:0/1465003066 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f800f050 con 0x7f3700131630 2023-10-03T10:17:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.542+0000 7f36ff7fe700 1 --2- 172.21.15.3:0/1465003066 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f370006eba0 0x7f370011b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.542+0000 7f36feffd700 1 --2- 172.21.15.3:0/1465003066 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3700131630 0x7f370011ba70 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f36f400c450 tx=0x7f36f400c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.542+0000 7f36e7fff700 1 -- 172.21.15.3:0/1465003066 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f40052d0 con 0x7f3700131630 2023-10-03T10:17:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.543+0000 7f36e7fff700 1 -- 172.21.15.3:0/1465003066 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f4005430 con 0x7f3700131630 2023-10-03T10:17:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.543+0000 7f3705d66700 1 -- 172.21.15.3:0/1465003066 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36f0009550 con 0x7f3700131630 2023-10-03T10:17:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.543+0000 7f36e7fff700 1 -- 172.21.15.3:0/1465003066 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f40038c0 con 0x7f3700131630 2023-10-03T10:17:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.543+0000 7f3705d66700 1 -- 172.21.15.3:0/1465003066 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f370011d140 con 0x7f3700131630 2023-10-03T10:17:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.544+0000 7f3705d66700 1 -- 172.21.15.3:0/1465003066 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3700074ea0 con 0x7f3700131630 2023-10-03T10:17:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.544+0000 7f36e7fff700 1 -- 172.21.15.3:0/1465003066 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f36f4003a20 con 0x7f3700131630 2023-10-03T10:17:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.545+0000 7f36e7fff700 1 -- 172.21.15.3:0/1465003066 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f36f403ec10 con 0x7f3700131630 2023-10-03T10:17:38.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.550+0000 7f36e7fff700 1 -- 172.21.15.3:0/1465003066 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f36f4011590 con 0x7f3700131630 2023-10-03T10:17:38.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.576+0000 7f406eddf700 1 Processor -- start 2023-10-03T10:17:38.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.576+0000 7f406eddf700 1 -- start start 2023-10-03T10:17:38.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.576+0000 7f406eddf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f406809d6f0 0x7f406809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:38.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.576+0000 7f406eddf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f406809e000 0x7f4068130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:38.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.576+0000 7f406eddf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f406809e410 0x7f4068130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:38.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.576+0000 7f406eddf700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f406809f520 con 0x7f406809d6f0 2023-10-03T10:17:38.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.576+0000 7f406eddf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f406809f130 con 0x7f406809e000 2023-10-03T10:17:38.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.576+0000 7f406eddf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f406809e930 con 0x7f406809e410 2023-10-03T10:17:38.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.576+0000 7f406d37c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f406809e410 0x7f4068130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:38.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.576+0000 7f406d37c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f406809e410 0x7f4068130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:53988/0 (socket says 172.21.15.3:53988) 2023-10-03T10:17:38.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.576+0000 7f406d37c700 1 -- 172.21.15.3:0/4078353163 learned_addr learned my addr 172.21.15.3:0/4078353163 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:38.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.576+0000 7f4067fff700 1 --2- 172.21.15.3:0/4078353163 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f406809e000 0x7f4068130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:38.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.577+0000 7f406cb7b700 1 --2- 172.21.15.3:0/4078353163 >> v2:172.21.15.3:3300/0 conn(0x7f406809d6f0 0x7f406809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:38.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.577+0000 7f406d37c700 1 -- 172.21.15.3:0/4078353163 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f406809e000 msgr2=0x7f4068130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:38.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.577+0000 7f406d37c700 1 --2- 172.21.15.3:0/4078353163 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f406809e000 0x7f4068130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.577+0000 7f406d37c700 1 -- 172.21.15.3:0/4078353163 >> v2:172.21.15.3:3300/0 conn(0x7f406809d6f0 msgr2=0x7f406809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:38.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.577+0000 7f406d37c700 1 --2- 172.21.15.3:0/4078353163 >> v2:172.21.15.3:3300/0 conn(0x7f406809d6f0 0x7f406809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.577+0000 7f406d37c700 1 -- 172.21.15.3:0/4078353163 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406809b2d0 con 0x7f406809e410 2023-10-03T10:17:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.577+0000 7f4067fff700 1 --2- 172.21.15.3:0/4078353163 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f406809e000 0x7f4068130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.577+0000 7f406d37c700 1 --2- 172.21.15.3:0/4078353163 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f406809e410 0x7f4068130ba0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4060009e80 tx=0x7f406000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0a0d2aaba4cd6bd server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.577+0000 7f40677fe700 1 -- 172.21.15.3:0/4078353163 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406000e070 con 0x7f406809e410 2023-10-03T10:17:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.577+0000 7f40677fe700 1 -- 172.21.15.3:0/4078353163 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f406000b8d0 con 0x7f406809e410 2023-10-03T10:17:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.577+0000 7f40677fe700 1 -- 172.21.15.3:0/4078353163 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406000bc80 con 0x7f406809e410 2023-10-03T10:17:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.578+0000 7f406eddf700 1 -- 172.21.15.3:0/4078353163 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f406809e410 msgr2=0x7f4068130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.578+0000 7f406eddf700 1 --2- 172.21.15.3:0/4078353163 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f406809e410 0x7f4068130ba0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4060009e80 tx=0x7f406000b330 comp rx=0 tx=0).stop 2023-10-03T10:17:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.578+0000 7f406eddf700 1 -- 172.21.15.3:0/4078353163 shutdown_connections 2023-10-03T10:17:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.578+0000 7f406eddf700 1 --2- 172.21.15.3:0/4078353163 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f406809e410 0x7f4068130ba0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.578+0000 7f406eddf700 1 --2- 172.21.15.3:0/4078353163 >> v2:172.21.15.3:3300/0 conn(0x7f406809d6f0 0x7f406809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.578+0000 7f406eddf700 1 --2- 172.21.15.3:0/4078353163 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f406809e000 0x7f4068130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.578+0000 7f406eddf700 1 -- 172.21.15.3:0/4078353163 >> 172.21.15.3:0/4078353163 conn(0x7f4068066f50 msgr2=0x7f406809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:38.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.578+0000 7f406eddf700 1 -- 172.21.15.3:0/4078353163 shutdown_connections 2023-10-03T10:17:38.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.578+0000 7f406eddf700 1 -- 172.21.15.3:0/4078353163 wait complete. 2023-10-03T10:17:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.580+0000 7f406eddf700 1 Processor -- start 2023-10-03T10:17:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.580+0000 7f406eddf700 1 -- start start 2023-10-03T10:17:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.581+0000 7f406eddf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f406809d6f0 0x7f40681221a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.581+0000 7f406eddf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f406809e000 0x7f40681226e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.581+0000 7f406eddf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f406809e410 0x7f4068122c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.581+0000 7f406eddf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4068123db0 con 0x7f406809e000 2023-10-03T10:17:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.581+0000 7f406eddf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40681243d0 con 0x7f406809e410 2023-10-03T10:17:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.581+0000 7f406eddf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40681259e0 con 0x7f406809d6f0 2023-10-03T10:17:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.581+0000 7f4067fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f406809e000 0x7f40681226e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.581+0000 7f406cb7b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f406809d6f0 0x7f40681221a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:38.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.581+0000 7f4067fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f406809e000 0x7f40681226e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39778/0 (socket says 172.21.15.3:39778) 2023-10-03T10:17:38.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.581+0000 7f4067fff700 1 -- 172.21.15.3:0/3717995514 learned_addr learned my addr 172.21.15.3:0/3717995514 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:38.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.581+0000 7f406d37c700 1 --2- 172.21.15.3:0/3717995514 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f406809e410 0x7f4068122c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:38.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.581+0000 7f4067fff700 1 -- 172.21.15.3:0/3717995514 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f406809d6f0 msgr2=0x7f40681221a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:38.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.581+0000 7f4067fff700 1 --2- 172.21.15.3:0/3717995514 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f406809d6f0 0x7f40681221a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.581+0000 7f4067fff700 1 -- 172.21.15.3:0/3717995514 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f406809e410 msgr2=0x7f4068122c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:38.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.581+0000 7f4067fff700 1 --2- 172.21.15.3:0/3717995514 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f406809e410 0x7f4068122c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.581+0000 7f4067fff700 1 -- 172.21.15.3:0/3717995514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40540094a0 con 0x7f406809e000 2023-10-03T10:17:38.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.581+0000 7f406cb7b700 1 --2- 172.21.15.3:0/3717995514 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f406809d6f0 0x7f40681221a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:38.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.582+0000 7f4067fff700 1 --2- 172.21.15.3:0/3717995514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f406809e000 0x7f40681226e0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f405400ebe0 tx=0x7f405400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:38.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.582+0000 7f40657fa700 1 -- 172.21.15.3:0/3717995514 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405400c850 con 0x7f406809e000 2023-10-03T10:17:38.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.582+0000 7f40657fa700 1 -- 172.21.15.3:0/3717995514 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f405400c9b0 con 0x7f406809e000 2023-10-03T10:17:38.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.582+0000 7f406eddf700 1 -- 172.21.15.3:0/3717995514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4060009b30 con 0x7f406809e000 2023-10-03T10:17:38.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.582+0000 7f40657fa700 1 -- 172.21.15.3:0/3717995514 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405400cd40 con 0x7f406809e000 2023-10-03T10:17:38.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.582+0000 7f406eddf700 1 -- 172.21.15.3:0/3717995514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40681245a0 con 0x7f406809e000 2023-10-03T10:17:38.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.583+0000 7f40657fa700 1 -- 172.21.15.3:0/3717995514 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4054013470 con 0x7f406809e000 2023-10-03T10:17:38.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.583+0000 7f40657fa700 1 -- 172.21.15.3:0/3717995514 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f405403fc80 con 0x7f406809e000 2023-10-03T10:17:38.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.583+0000 7f406eddf700 1 -- 172.21.15.3:0/3717995514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f406811ac20 con 0x7f406809e000 2023-10-03T10:17:38.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.588+0000 7f40657fa700 1 -- 172.21.15.3:0/3717995514 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4054021080 con 0x7f406809e000 2023-10-03T10:17:38.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.623+0000 7feea2c38700 1 Processor -- start 2023-10-03T10:17:38.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.623+0000 7feea2c38700 1 -- start start 2023-10-03T10:17:38.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.624+0000 7feea2c38700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fee9c09f330 0x7fee9c138ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:38.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.624+0000 7feea2c38700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee9c1393e0 0x7fee9c13b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:38.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.624+0000 7feea2c38700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee9c09f960 con 0x7fee9c09e850 2023-10-03T10:17:38.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.624+0000 7feea2c38700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee9c09f720 con 0x7fee9c09f330 2023-10-03T10:17:38.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.624+0000 7feea2c38700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c09ae60 con 0x7fee9c1393e0 2023-10-03T10:17:38.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.624+0000 7fee9bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fee9c09f330 0x7fee9c138ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:38.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.624+0000 7feea11d5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee9c1393e0 0x7fee9c13b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:38.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.624+0000 7fee9bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fee9c09f330 0x7fee9c138ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39786/0 (socket says 172.21.15.3:39786) 2023-10-03T10:17:38.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.624+0000 7feea09d4700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fee9c09e850 0x7fee9c09ec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33100/0 (socket says 172.21.15.3:33100) 2023-10-03T10:17:38.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.624+0000 7fee9bfff700 1 -- 172.21.15.3:0/3892484323 learned_addr learned my addr 172.21.15.3:0/3892484323 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:38.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.624+0000 7feea11d5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee9c1393e0 0x7fee9c13b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38894/0 (socket says 172.21.15.3:38894) 2023-10-03T10:17:38.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.625+0000 7fee9bfff700 1 -- 172.21.15.3:0/3892484323 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee9c1393e0 msgr2=0x7fee9c13b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:38.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.625+0000 7fee9bfff700 1 --2- 172.21.15.3:0/3892484323 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee9c1393e0 0x7fee9c13b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.625+0000 7fee9bfff700 1 -- 172.21.15.3:0/3892484323 >> v1:172.21.15.3:6789/0 conn(0x7fee9c09e850 legacy=0x7fee9c09ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:38.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.625+0000 7fee9bfff700 1 -- 172.21.15.3:0/3892484323 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9c06cb10 con 0x7fee9c09f330 2023-10-03T10:17:38.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.625+0000 7fee9bfff700 1 --2- 172.21.15.3:0/3892484323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee9c09f330 0x7fee9c138ea0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fee880098f0 tx=0x7fee88009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a9d055deb291b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:38.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.625+0000 7fee9b7fe700 1 -- 172.21.15.3:0/3892484323 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee88011830 con 0x7fee9c09f330 2023-10-03T10:17:38.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.625+0000 7fee9b7fe700 1 -- 172.21.15.3:0/3892484323 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee88011990 con 0x7fee9c09f330 2023-10-03T10:17:38.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.625+0000 7fee9b7fe700 1 -- 172.21.15.3:0/3892484323 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee88011d20 con 0x7fee9c09f330 2023-10-03T10:17:38.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.626+0000 7feea2c38700 1 -- 172.21.15.3:0/3892484323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee9c09f330 msgr2=0x7fee9c138ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:38.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.626+0000 7feea2c38700 1 --2- 172.21.15.3:0/3892484323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee9c09f330 0x7fee9c138ea0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fee880098f0 tx=0x7fee88009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:38.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.626+0000 7feea2c38700 1 -- 172.21.15.3:0/3892484323 shutdown_connections 2023-10-03T10:17:38.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.626+0000 7feea2c38700 1 --2- 172.21.15.3:0/3892484323 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee9c1393e0 0x7fee9c13b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.626+0000 7feea2c38700 1 --2- 172.21.15.3:0/3892484323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee9c09f330 0x7fee9c138ea0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.626+0000 7feea2c38700 1 -- 172.21.15.3:0/3892484323 >> 172.21.15.3:0/3892484323 conn(0x7fee9c0a3f70 msgr2=0x7fee9c0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:38.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.626+0000 7feea2c38700 1 -- 172.21.15.3:0/3892484323 shutdown_connections 2023-10-03T10:17:38.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.626+0000 7feea2c38700 1 -- 172.21.15.3:0/3892484323 wait complete. 2023-10-03T10:17:38.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.628+0000 7feea2c38700 1 Processor -- start 2023-10-03T10:17:38.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.628+0000 7feea2c38700 1 -- start start 2023-10-03T10:17:38.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.629+0000 7feea2c38700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee9c09e850 0x7fee9c1328a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:38.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.629+0000 7feea2c38700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee9c09f330 0x7fee9c132de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:38.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.629+0000 7feea2c38700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee9c1393e0 0x7fee9c11ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:38.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.629+0000 7feea2c38700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c1216f0 con 0x7fee9c09e850 2023-10-03T10:17:38.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.629+0000 7feea2c38700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c06f810 con 0x7fee9c09f330 2023-10-03T10:17:38.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.629+0000 7feea2c38700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c09e270 con 0x7fee9c1393e0 2023-10-03T10:17:38.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.629+0000 7fee9bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee9c09f330 0x7fee9c132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:38.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.629+0000 7fee9bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee9c09f330 0x7fee9c132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38904/0 (socket says 172.21.15.3:38904) 2023-10-03T10:17:38.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.629+0000 7fee9bfff700 1 -- 172.21.15.3:0/3763926423 learned_addr learned my addr 172.21.15.3:0/3763926423 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:38.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.629+0000 7feea09d4700 1 --2- 172.21.15.3:0/3763926423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee9c09e850 0x7fee9c1328a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:38.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.629+0000 7feea11d5700 1 --2- 172.21.15.3:0/3763926423 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee9c1393e0 0x7fee9c11ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:38.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.629+0000 7fee9bfff700 1 -- 172.21.15.3:0/3763926423 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee9c1393e0 msgr2=0x7fee9c11ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:38.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.629+0000 7fee9bfff700 1 --2- 172.21.15.3:0/3763926423 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee9c1393e0 0x7fee9c11ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.629+0000 7fee9bfff700 1 -- 172.21.15.3:0/3763926423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee9c09e850 msgr2=0x7fee9c1328a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:38.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.629+0000 7fee9bfff700 1 --2- 172.21.15.3:0/3763926423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee9c09e850 0x7fee9c1328a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.629+0000 7fee9bfff700 1 -- 172.21.15.3:0/3763926423 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee88009550 con 0x7fee9c09f330 2023-10-03T10:17:38.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.630+0000 7fee9bfff700 1 --2- 172.21.15.3:0/3763926423 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee9c09f330 0x7fee9c132de0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fee88000c00 tx=0x7fee88004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:38.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.630+0000 7fee997fa700 1 -- 172.21.15.3:0/3763926423 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee880036a0 con 0x7fee9c09f330 2023-10-03T10:17:38.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.630+0000 7fee997fa700 1 -- 172.21.15.3:0/3763926423 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee88003800 con 0x7fee9c09f330 2023-10-03T10:17:38.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.630+0000 7feea2c38700 1 -- 172.21.15.3:0/3763926423 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee9c11fc70 con 0x7fee9c09f330 2023-10-03T10:17:38.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.630+0000 7fee997fa700 1 -- 172.21.15.3:0/3763926423 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee88003bb0 con 0x7fee9c09f330 2023-10-03T10:17:38.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.630+0000 7feea2c38700 1 -- 172.21.15.3:0/3763926423 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee9c133320 con 0x7fee9c09f330 2023-10-03T10:17:38.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.631+0000 7fee997fa700 1 -- 172.21.15.3:0/3763926423 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fee88021070 con 0x7fee9c09f330 2023-10-03T10:17:38.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.631+0000 7fee997fa700 1 -- 172.21.15.3:0/3763926423 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fee88040c90 con 0x7fee9c09f330 2023-10-03T10:17:38.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.631+0000 7feea2c38700 1 -- 172.21.15.3:0/3763926423 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee9c0747f0 con 0x7fee9c09f330 2023-10-03T10:17:38.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.641+0000 7fee997fa700 1 -- 172.21.15.3:0/3763926423 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fee88015020 con 0x7fee9c09f330 2023-10-03T10:17:38.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.761+0000 7f3705d66700 1 -- 172.21.15.3:0/1465003066 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f370009ed00 con 0x7f3700131630 2023-10-03T10:17:38.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.761+0000 7f36e7fff700 1 -- 172.21.15.3:0/1465003066 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f36f4017070 con 0x7f3700131630 2023-10-03T10:17:38.762 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:38.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.764+0000 7f3705d66700 1 -- 172.21.15.3:0/1465003066 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3700131630 msgr2=0x7f370011ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:38.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.764+0000 7f3705d66700 1 --2- 172.21.15.3:0/1465003066 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3700131630 0x7f370011ba70 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f36f400c450 tx=0x7f36f400c760 comp rx=0 tx=0).stop 2023-10-03T10:17:38.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.764+0000 7f3705d66700 1 -- 172.21.15.3:0/1465003066 shutdown_connections 2023-10-03T10:17:38.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.764+0000 7f3705d66700 1 --2- 172.21.15.3:0/1465003066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3700131f40 0x7f370011bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.764+0000 7f3705d66700 1 --2- 172.21.15.3:0/1465003066 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f370006eba0 0x7f370011b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.764+0000 7f3705d66700 1 --2- 172.21.15.3:0/1465003066 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3700131630 0x7f370011ba70 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.764+0000 7f3705d66700 1 -- 172.21.15.3:0/1465003066 >> 172.21.15.3:0/1465003066 conn(0x7f37000a2aa0 msgr2=0x7f3700070f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:38.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.764+0000 7f3705d66700 1 -- 172.21.15.3:0/1465003066 shutdown_connections 2023-10-03T10:17:38.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.764+0000 7f3705d66700 1 -- 172.21.15.3:0/1465003066 wait complete. 2023-10-03T10:17:38.778 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:38.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.813+0000 7f406eddf700 1 -- 172.21.15.3:0/3717995514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4068074ea0 con 0x7f406809e000 2023-10-03T10:17:38.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.814+0000 7f40657fa700 1 -- 172.21.15.3:0/3717995514 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4054017020 con 0x7f406809e000 2023-10-03T10:17:38.814 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:38.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.816+0000 7f406eddf700 1 -- 172.21.15.3:0/3717995514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f406809e000 msgr2=0x7f40681226e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:38.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.816+0000 7f406eddf700 1 --2- 172.21.15.3:0/3717995514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f406809e000 0x7f40681226e0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f405400ebe0 tx=0x7f405400eef0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.816+0000 7f406eddf700 1 -- 172.21.15.3:0/3717995514 shutdown_connections 2023-10-03T10:17:38.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.816+0000 7f406eddf700 1 --2- 172.21.15.3:0/3717995514 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f406809e410 0x7f4068122c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.816+0000 7f406eddf700 1 --2- 172.21.15.3:0/3717995514 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f406809d6f0 0x7f40681221a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.816+0000 7f406eddf700 1 --2- 172.21.15.3:0/3717995514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f406809e000 0x7f40681226e0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.816+0000 7f406eddf700 1 -- 172.21.15.3:0/3717995514 >> 172.21.15.3:0/3717995514 conn(0x7f4068066f50 msgr2=0x7f4068130970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:38.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.816+0000 7f406eddf700 1 -- 172.21.15.3:0/3717995514 shutdown_connections 2023-10-03T10:17:38.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.816+0000 7f406eddf700 1 -- 172.21.15.3:0/3717995514 wait complete. 2023-10-03T10:17:38.830 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:38.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.846+0000 7feea2c38700 1 -- 172.21.15.3:0/3763926423 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fee9c1213b0 con 0x7fee9c09f330 2023-10-03T10:17:38.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.847+0000 7fee997fa700 1 -- 172.21.15.3:0/3763926423 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fee88029070 con 0x7fee9c09f330 2023-10-03T10:17:38.847 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:38.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.849+0000 7feea2c38700 1 -- 172.21.15.3:0/3763926423 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee9c09f330 msgr2=0x7fee9c132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:38.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.849+0000 7feea2c38700 1 --2- 172.21.15.3:0/3763926423 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee9c09f330 0x7fee9c132de0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fee88000c00 tx=0x7fee88004750 comp rx=0 tx=0).stop 2023-10-03T10:17:38.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.849+0000 7feea2c38700 1 -- 172.21.15.3:0/3763926423 shutdown_connections 2023-10-03T10:17:38.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.849+0000 7feea2c38700 1 --2- 172.21.15.3:0/3763926423 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee9c1393e0 0x7fee9c11ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.849+0000 7feea2c38700 1 --2- 172.21.15.3:0/3763926423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee9c09e850 0x7fee9c1328a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.849+0000 7feea2c38700 1 --2- 172.21.15.3:0/3763926423 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee9c09f330 0x7fee9c132de0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:38.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.849+0000 7feea2c38700 1 -- 172.21.15.3:0/3763926423 >> 172.21.15.3:0/3763926423 conn(0x7fee9c0a3f70 msgr2=0x7fee9c138540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:38.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.849+0000 7feea2c38700 1 -- 172.21.15.3:0/3763926423 shutdown_connections 2023-10-03T10:17:38.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:38.849+0000 7feea2c38700 1 -- 172.21.15.3:0/3763926423 wait complete. 2023-10-03T10:17:38.863 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:39.185 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:39.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.331+0000 7fb3896eb700 1 Processor -- start 2023-10-03T10:17:39.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.331+0000 7fb3896eb700 1 -- start start 2023-10-03T10:17:39.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.332+0000 7fb3896eb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb38409d6f0 0x7fb38409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:39.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.332+0000 7fb3896eb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb38409e000 0x7fb3841339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:39.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.332+0000 7fb3896eb700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb38409b120 con 0x7fb38409e410 2023-10-03T10:17:39.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.332+0000 7fb3896eb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb38406b460 con 0x7fb38409e000 2023-10-03T10:17:39.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.332+0000 7fb3896eb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38409a7e0 con 0x7fb38409d6f0 2023-10-03T10:17:39.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.332+0000 7fb382ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb38409d6f0 0x7fb38409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:39.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.332+0000 7fb382ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb38409d6f0 0x7fb38409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38910/0 (socket says 172.21.15.3:38910) 2023-10-03T10:17:39.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.332+0000 7fb382ffd700 1 -- 172.21.15.3:0/3238458624 learned_addr learned my addr 172.21.15.3:0/3238458624 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:39.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.332+0000 7fb3827fc700 1 --2- 172.21.15.3:0/3238458624 >> v2:172.21.15.3:3300/0 conn(0x7fb38409e000 0x7fb3841339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:39.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.332+0000 7fb382ffd700 1 -- 172.21.15.3:0/3238458624 >> v2:172.21.15.3:3300/0 conn(0x7fb38409e000 msgr2=0x7fb3841339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:39.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.332+0000 7fb382ffd700 1 --2- 172.21.15.3:0/3238458624 >> v2:172.21.15.3:3300/0 conn(0x7fb38409e000 0x7fb3841339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:39.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.332+0000 7fb382ffd700 1 -- 172.21.15.3:0/3238458624 >> v1:172.21.15.3:6789/0 conn(0x7fb38409e410 legacy=0x7fb384133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:39.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.332+0000 7fb382ffd700 1 -- 172.21.15.3:0/3238458624 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38409a500 con 0x7fb38409d6f0 2023-10-03T10:17:39.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.332+0000 7fb3827fc700 1 --2- 172.21.15.3:0/3238458624 >> v2:172.21.15.3:3300/0 conn(0x7fb38409e000 0x7fb3841339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:39.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.333+0000 7fb382ffd700 1 --2- 172.21.15.3:0/3238458624 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb38409d6f0 0x7fb38409dac0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb374009920 tx=0x7fb374009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba869ccfcd2a90a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:39.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.333+0000 7fb381ffb700 1 -- 172.21.15.3:0/3238458624 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb374011830 con 0x7fb38409d6f0 2023-10-03T10:17:39.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.333+0000 7fb381ffb700 1 -- 172.21.15.3:0/3238458624 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb374011990 con 0x7fb38409d6f0 2023-10-03T10:17:39.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.333+0000 7fb381ffb700 1 -- 172.21.15.3:0/3238458624 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb374011d20 con 0x7fb38409d6f0 2023-10-03T10:17:39.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.333+0000 7fb3896eb700 1 -- 172.21.15.3:0/3238458624 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb38409d6f0 msgr2=0x7fb38409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:39.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.333+0000 7fb3896eb700 1 --2- 172.21.15.3:0/3238458624 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb38409d6f0 0x7fb38409dac0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb374009920 tx=0x7fb374009c30 comp rx=0 tx=0).stop 2023-10-03T10:17:39.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.333+0000 7fb3896eb700 1 -- 172.21.15.3:0/3238458624 shutdown_connections 2023-10-03T10:17:39.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.333+0000 7fb3896eb700 1 --2- 172.21.15.3:0/3238458624 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb38409d6f0 0x7fb38409dac0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:39.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.333+0000 7fb3896eb700 1 --2- 172.21.15.3:0/3238458624 >> v2:172.21.15.3:3300/0 conn(0x7fb38409e000 0x7fb3841339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:39.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.333+0000 7fb3896eb700 1 -- 172.21.15.3:0/3238458624 >> 172.21.15.3:0/3238458624 conn(0x7fb384066f50 msgr2=0x7fb38409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:39.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.333+0000 7fb3896eb700 1 -- 172.21.15.3:0/3238458624 shutdown_connections 2023-10-03T10:17:39.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.333+0000 7fb3896eb700 1 -- 172.21.15.3:0/3238458624 wait complete. 2023-10-03T10:17:39.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.335+0000 7fb3896eb700 1 Processor -- start 2023-10-03T10:17:39.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.335+0000 7fb3896eb700 1 -- start start 2023-10-03T10:17:39.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.335+0000 7fb3896eb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb38409d6f0 0x7fb3841a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:39.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.335+0000 7fb3896eb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb38409e000 0x7fb3841a6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:39.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.335+0000 7fb3896eb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb38409e410 0x7fb3841a6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:39.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.335+0000 7fb3896eb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3841a7cb0 con 0x7fb38409e000 2023-10-03T10:17:39.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.335+0000 7fb3896eb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3841a9820 con 0x7fb38409e410 2023-10-03T10:17:39.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.335+0000 7fb3896eb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb3840a3380 con 0x7fb38409d6f0 2023-10-03T10:17:39.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.335+0000 7fb3827fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb38409e000 0x7fb3841a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:39.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.335+0000 7fb382ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb38409d6f0 0x7fb3841a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:39.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.335+0000 7fb3827fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb38409e000 0x7fb3841a6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39818/0 (socket says 172.21.15.3:39818) 2023-10-03T10:17:39.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.335+0000 7fb3827fc700 1 -- 172.21.15.3:0/3631707147 learned_addr learned my addr 172.21.15.3:0/3631707147 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:39.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.335+0000 7fb382ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb38409d6f0 0x7fb3841a60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54022/0 (socket says 172.21.15.3:54022) 2023-10-03T10:17:39.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.335+0000 7fb3837fe700 1 --2- 172.21.15.3:0/3631707147 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb38409e410 0x7fb3841a6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:39.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.336+0000 7fb3827fc700 1 -- 172.21.15.3:0/3631707147 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb38409d6f0 msgr2=0x7fb3841a60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:39.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.336+0000 7fb3827fc700 1 --2- 172.21.15.3:0/3631707147 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb38409d6f0 0x7fb3841a60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:39.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.336+0000 7fb3827fc700 1 -- 172.21.15.3:0/3631707147 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb38409e410 msgr2=0x7fb3841a6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:39.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.336+0000 7fb3827fc700 1 --2- 172.21.15.3:0/3631707147 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb38409e410 0x7fb3841a6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:39.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.336+0000 7fb3827fc700 1 -- 172.21.15.3:0/3631707147 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3780094a0 con 0x7fb38409e000 2023-10-03T10:17:39.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.336+0000 7fb3837fe700 1 --2- 172.21.15.3:0/3631707147 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb38409e410 0x7fb3841a6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:39.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.336+0000 7fb382ffd700 1 --2- 172.21.15.3:0/3631707147 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb38409d6f0 0x7fb3841a60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:39.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.336+0000 7fb3827fc700 1 --2- 172.21.15.3:0/3631707147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb38409e000 0x7fb3841a6630 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fb37800ebb0 tx=0x7fb37800eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:39.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.336+0000 7fb36b7fe700 1 -- 172.21.15.3:0/3631707147 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37800c850 con 0x7fb38409e000 2023-10-03T10:17:39.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.336+0000 7fb36b7fe700 1 -- 172.21.15.3:0/3631707147 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37800c9b0 con 0x7fb38409e000 2023-10-03T10:17:39.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.336+0000 7fb3896eb700 1 -- 172.21.15.3:0/3631707147 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb374009550 con 0x7fb38409e000 2023-10-03T10:17:39.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.336+0000 7fb36b7fe700 1 -- 172.21.15.3:0/3631707147 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37800cd40 con 0x7fb38409e000 2023-10-03T10:17:39.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.336+0000 7fb3896eb700 1 -- 172.21.15.3:0/3631707147 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb38409c760 con 0x7fb38409e000 2023-10-03T10:17:39.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.337+0000 7fb36b7fe700 1 -- 172.21.15.3:0/3631707147 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb378013470 con 0x7fb38409e000 2023-10-03T10:17:39.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.337+0000 7fb36b7fe700 1 -- 172.21.15.3:0/3631707147 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb37803fb30 con 0x7fb38409e000 2023-10-03T10:17:39.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.337+0000 7fb3896eb700 1 -- 172.21.15.3:0/3631707147 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb384074ea0 con 0x7fb38409e000 2023-10-03T10:17:39.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.342+0000 7fb36b7fe700 1 -- 172.21.15.3:0/3631707147 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb37801b3b0 con 0x7fb38409e000 2023-10-03T10:17:39.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.549+0000 7fb3896eb700 1 -- 172.21.15.3:0/3631707147 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb3841a9bc0 con 0x7fb38409e000 2023-10-03T10:17:39.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.549+0000 7fb36b7fe700 1 -- 172.21.15.3:0/3631707147 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb378017020 con 0x7fb38409e000 2023-10-03T10:17:39.550 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:39.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.552+0000 7fb3896eb700 1 -- 172.21.15.3:0/3631707147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb38409e000 msgr2=0x7fb3841a6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:39.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.552+0000 7fb3896eb700 1 --2- 172.21.15.3:0/3631707147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb38409e000 0x7fb3841a6630 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fb37800ebb0 tx=0x7fb37800eec0 comp rx=0 tx=0).stop 2023-10-03T10:17:39.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.552+0000 7fb3896eb700 1 -- 172.21.15.3:0/3631707147 shutdown_connections 2023-10-03T10:17:39.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.552+0000 7fb3896eb700 1 --2- 172.21.15.3:0/3631707147 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb38409e410 0x7fb3841a6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:39.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.552+0000 7fb3896eb700 1 --2- 172.21.15.3:0/3631707147 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb38409d6f0 0x7fb3841a60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:39.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.552+0000 7fb3896eb700 1 --2- 172.21.15.3:0/3631707147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb38409e000 0x7fb3841a6630 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:39.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.552+0000 7fb3896eb700 1 -- 172.21.15.3:0/3631707147 >> 172.21.15.3:0/3631707147 conn(0x7fb384066f50 msgr2=0x7fb3841baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:39.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.552+0000 7fb3896eb700 1 -- 172.21.15.3:0/3631707147 shutdown_connections 2023-10-03T10:17:39.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.552+0000 7fb3896eb700 1 -- 172.21.15.3:0/3631707147 wait complete. 2023-10-03T10:17:39.566 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:39.778 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:39.830 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:39.864 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:39.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.927+0000 7fb1fe545700 1 Processor -- start 2023-10-03T10:17:39.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.927+0000 7fb1fe545700 1 -- start start 2023-10-03T10:17:39.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.927+0000 7fb1fe545700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1f809e930 0x7fb1f809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:39.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.927+0000 7fb1fe545700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb1f809f240 0x7fb1f81317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:39.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.927+0000 7fb1fe545700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb1f809f650 0x7fb1f8131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:39.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.927+0000 7fb1fe545700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1f8070940 con 0x7fb1f809f240 2023-10-03T10:17:39.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.927+0000 7fb1fe545700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f806fd60 con 0x7fb1f809e930 2023-10-03T10:17:39.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.927+0000 7fb1fe545700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f806fa80 con 0x7fb1f809f650 2023-10-03T10:17:39.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.927+0000 7fb1f7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1f809e930 0x7fb1f809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:39.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.928+0000 7fb1f7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1f809e930 0x7fb1f809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38930/0 (socket says 172.21.15.3:38930) 2023-10-03T10:17:39.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.928+0000 7fb1f7fff700 1 -- 172.21.15.3:0/4262743001 learned_addr learned my addr 172.21.15.3:0/4262743001 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:39.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.928+0000 7fb1f77fe700 1 --2- 172.21.15.3:0/4262743001 >> v2:172.21.15.3:3300/0 conn(0x7fb1f809f240 0x7fb1f81317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:39.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.928+0000 7fb1fcae2700 1 --2- 172.21.15.3:0/4262743001 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb1f809f650 0x7fb1f8131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:39.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.928+0000 7fb1f77fe700 1 -- 172.21.15.3:0/4262743001 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb1f809f650 msgr2=0x7fb1f8131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:39.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.928+0000 7fb1f77fe700 1 --2- 172.21.15.3:0/4262743001 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb1f809f650 0x7fb1f8131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:39.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.928+0000 7fb1f77fe700 1 -- 172.21.15.3:0/4262743001 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1f809e930 msgr2=0x7fb1f809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:39.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.928+0000 7fb1f77fe700 1 --2- 172.21.15.3:0/4262743001 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1f809e930 0x7fb1f809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:39.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.928+0000 7fb1f77fe700 1 -- 172.21.15.3:0/4262743001 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f809fb60 con 0x7fb1f809f240 2023-10-03T10:17:39.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.928+0000 7fb1f7fff700 1 --2- 172.21.15.3:0/4262743001 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1f809e930 0x7fb1f809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:39.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.928+0000 7fb1fcae2700 1 --2- 172.21.15.3:0/4262743001 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb1f809f650 0x7fb1f8131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:17:39.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.928+0000 7fb1f77fe700 1 --2- 172.21.15.3:0/4262743001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1f809f240 0x7fb1f81317f0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec009840 tx=0x7fb1ec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f44d62fa3bd91e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:39.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.928+0000 7fb1f6ffd700 1 -- 172.21.15.3:0/4262743001 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec011830 con 0x7fb1f809f240 2023-10-03T10:17:39.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.928+0000 7fb1f6ffd700 1 -- 172.21.15.3:0/4262743001 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ec011990 con 0x7fb1f809f240 2023-10-03T10:17:39.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.928+0000 7fb1f6ffd700 1 -- 172.21.15.3:0/4262743001 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec011d20 con 0x7fb1f809f240 2023-10-03T10:17:39.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.929+0000 7fb1fe545700 1 -- 172.21.15.3:0/4262743001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1f809f240 msgr2=0x7fb1f81317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:39.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.929+0000 7fb1fe545700 1 --2- 172.21.15.3:0/4262743001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1f809f240 0x7fb1f81317f0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec009840 tx=0x7fb1ec009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:39.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.929+0000 7fb1fe545700 1 -- 172.21.15.3:0/4262743001 shutdown_connections 2023-10-03T10:17:39.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.929+0000 7fb1fe545700 1 --2- 172.21.15.3:0/4262743001 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb1f809f650 0x7fb1f8131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:39.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.929+0000 7fb1fe545700 1 --2- 172.21.15.3:0/4262743001 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1f809e930 0x7fb1f809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:39.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.929+0000 7fb1fe545700 1 --2- 172.21.15.3:0/4262743001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1f809f240 0x7fb1f81317f0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:39.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.929+0000 7fb1fe545700 1 -- 172.21.15.3:0/4262743001 >> 172.21.15.3:0/4262743001 conn(0x7fb1f80a4210 msgr2=0x7fb1f80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:39.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.929+0000 7fb1fe545700 1 -- 172.21.15.3:0/4262743001 shutdown_connections 2023-10-03T10:17:39.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.929+0000 7fb1fe545700 1 -- 172.21.15.3:0/4262743001 wait complete. 2023-10-03T10:17:39.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.931+0000 7fb1fe545700 1 Processor -- start 2023-10-03T10:17:39.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.931+0000 7fb1fe545700 1 -- start start 2023-10-03T10:17:39.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.932+0000 7fb1fe545700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb1f809e930 0x7fb1f81a3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:39.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.932+0000 7fb1fe545700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1f809f240 0x7fb1f81a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:39.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.932+0000 7fb1fe545700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1f809f650 0x7fb1f81a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:39.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.932+0000 7fb1fe545700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f81a5160 con 0x7fb1f809f240 2023-10-03T10:17:39.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.932+0000 7fb1fe545700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f81a5870 con 0x7fb1f809f650 2023-10-03T10:17:39.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.932+0000 7fb1fe545700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f81a6ac0 con 0x7fb1f809e930 2023-10-03T10:17:39.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.932+0000 7fb1f7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb1f809e930 0x7fb1f81a3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:39.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.932+0000 7fb1fcae2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1f809f650 0x7fb1f81a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:39.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.932+0000 7fb1fcae2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1f809f650 0x7fb1f81a4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38932/0 (socket says 172.21.15.3:38932) 2023-10-03T10:17:39.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.932+0000 7fb1f7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb1f809e930 0x7fb1f81a3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54054/0 (socket says 172.21.15.3:54054) 2023-10-03T10:17:39.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.932+0000 7fb1fcae2700 1 -- 172.21.15.3:0/4115768164 learned_addr learned my addr 172.21.15.3:0/4115768164 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:39.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.932+0000 7fb1f77fe700 1 --2- 172.21.15.3:0/4115768164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1f809f240 0x7fb1f81a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:39.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.932+0000 7fb1fcae2700 1 -- 172.21.15.3:0/4115768164 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb1f809e930 msgr2=0x7fb1f81a3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:39.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.932+0000 7fb1fcae2700 1 --2- 172.21.15.3:0/4115768164 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb1f809e930 0x7fb1f81a3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:39.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.932+0000 7fb1fcae2700 1 -- 172.21.15.3:0/4115768164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1f809f240 msgr2=0x7fb1f81a41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:39.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.932+0000 7fb1fcae2700 1 --2- 172.21.15.3:0/4115768164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1f809f240 0x7fb1f81a41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:39.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.932+0000 7fb1fcae2700 1 -- 172.21.15.3:0/4115768164 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e8009550 con 0x7fb1f809f650 2023-10-03T10:17:39.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.932+0000 7fb1f7fff700 1 --2- 172.21.15.3:0/4115768164 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb1f809e930 0x7fb1f81a3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:39.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.932+0000 7fb1f77fe700 1 --2- 172.21.15.3:0/4115768164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1f809f240 0x7fb1f81a41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:39.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.932+0000 7fb1fcae2700 1 --2- 172.21.15.3:0/4115768164 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1f809f650 0x7fb1f81a4730 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0007e70 tx=0x7fb1f000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:39.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.933+0000 7fb1f4ff9700 1 -- 172.21.15.3:0/4115768164 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f000ddd0 con 0x7fb1f809f650 2023-10-03T10:17:39.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.933+0000 7fb1f4ff9700 1 -- 172.21.15.3:0/4115768164 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f000f040 con 0x7fb1f809f650 2023-10-03T10:17:39.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.933+0000 7fb1f4ff9700 1 -- 172.21.15.3:0/4115768164 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0015640 con 0x7fb1f809f650 2023-10-03T10:17:39.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.933+0000 7fb1fe545700 1 -- 172.21.15.3:0/4115768164 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1ec0094a0 con 0x7fb1f809f650 2023-10-03T10:17:39.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.933+0000 7fb1fe545700 1 -- 172.21.15.3:0/4115768164 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f80a0190 con 0x7fb1f809f650 2023-10-03T10:17:39.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.933+0000 7fb1f4ff9700 1 -- 172.21.15.3:0/4115768164 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb1f001e3b0 con 0x7fb1f809f650 2023-10-03T10:17:39.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.934+0000 7fb1f4ff9700 1 -- 172.21.15.3:0/4115768164 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb1f003f770 con 0x7fb1f809f650 2023-10-03T10:17:39.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.934+0000 7fb1fe545700 1 -- 172.21.15.3:0/4115768164 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1f81a5e50 con 0x7fb1f809f650 2023-10-03T10:17:39.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.939+0000 7fb1f4ff9700 1 -- 172.21.15.3:0/4115768164 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb1f00143a0 con 0x7fb1f809f650 2023-10-03T10:17:39.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.976+0000 7feab260a700 1 Processor -- start 2023-10-03T10:17:39.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.976+0000 7feab260a700 1 -- start start 2023-10-03T10:17:39.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.977+0000 7feab260a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaac0a0a30 0x7feaac0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:39.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.977+0000 7feab260a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaac0a1340 0x7feaac133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:39.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.977+0000 7feab260a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaac09edd0 con 0x7feaac133fa0 2023-10-03T10:17:39.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.977+0000 7feab260a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaac09e1f0 con 0x7feaac0a0a30 2023-10-03T10:17:39.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.977+0000 7feab260a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feaac09dc50 con 0x7feaac0a1340 2023-10-03T10:17:39.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.977+0000 7feaabfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaac0a0a30 0x7feaac0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:39.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.977+0000 7feaab7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaac0a1340 0x7feaac133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:39.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.977+0000 7feaabfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaac0a0a30 0x7feaac0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38944/0 (socket says 172.21.15.3:38944) 2023-10-03T10:17:39.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.977+0000 7feaab7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaac0a1340 0x7feaac133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54066/0 (socket says 172.21.15.3:54066) 2023-10-03T10:17:39.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.977+0000 7feaabfff700 1 -- 172.21.15.3:0/3821404678 learned_addr learned my addr 172.21.15.3:0/3821404678 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:39.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.977+0000 7feaabfff700 1 -- 172.21.15.3:0/3821404678 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaac0a1340 msgr2=0x7feaac133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:39.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.978+0000 7feaabfff700 1 --2- 172.21.15.3:0/3821404678 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaac0a1340 0x7feaac133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:39.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.978+0000 7feaabfff700 1 -- 172.21.15.3:0/3821404678 >> v1:172.21.15.3:6789/0 conn(0x7feaac133fa0 legacy=0x7feaac13ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:39.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.978+0000 7feaabfff700 1 -- 172.21.15.3:0/3821404678 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaac0a1fe0 con 0x7feaac0a0a30 2023-10-03T10:17:39.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.978+0000 7feaab7fe700 1 --2- 172.21.15.3:0/3821404678 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaac0a1340 0x7feaac133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:39.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.978+0000 7feaabfff700 1 --2- 172.21.15.3:0/3821404678 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaac0a0a30 0x7feaac0a0e00 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fea9c009920 tx=0x7fea9c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c22fd1289a00a8c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:39.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.978+0000 7feaaaffd700 1 -- 172.21.15.3:0/3821404678 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9c004230 con 0x7feaac0a0a30 2023-10-03T10:17:39.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.978+0000 7feaaaffd700 1 -- 172.21.15.3:0/3821404678 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea9c004390 con 0x7feaac0a0a30 2023-10-03T10:17:39.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.978+0000 7feaaaffd700 1 -- 172.21.15.3:0/3821404678 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9c0036a0 con 0x7feaac0a0a30 2023-10-03T10:17:39.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.979+0000 7feab260a700 1 -- 172.21.15.3:0/3821404678 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaac0a0a30 msgr2=0x7feaac0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:39.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.979+0000 7feab260a700 1 --2- 172.21.15.3:0/3821404678 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaac0a0a30 0x7feaac0a0e00 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fea9c009920 tx=0x7fea9c009c30 comp rx=0 tx=0).stop 2023-10-03T10:17:39.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.979+0000 7feab260a700 1 -- 172.21.15.3:0/3821404678 shutdown_connections 2023-10-03T10:17:39.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.979+0000 7feab260a700 1 --2- 172.21.15.3:0/3821404678 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaac0a0a30 0x7feaac0a0e00 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:39.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.979+0000 7feab260a700 1 --2- 172.21.15.3:0/3821404678 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaac0a1340 0x7feaac133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:39.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.979+0000 7feab260a700 1 -- 172.21.15.3:0/3821404678 >> 172.21.15.3:0/3821404678 conn(0x7feaac0a6550 msgr2=0x7feaac0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:39.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.979+0000 7feab260a700 1 -- 172.21.15.3:0/3821404678 shutdown_connections 2023-10-03T10:17:39.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.979+0000 7feab260a700 1 -- 172.21.15.3:0/3821404678 wait complete. 2023-10-03T10:17:39.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.980+0000 7feab260a700 1 Processor -- start 2023-10-03T10:17:39.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.980+0000 7feab260a700 1 -- start start 2023-10-03T10:17:39.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.981+0000 7feab260a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaac0a0a30 0x7feaac1c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:39.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.981+0000 7feab260a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaac0a1340 0x7feaac1c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:39.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.981+0000 7feab260a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaac133fa0 0x7feaac1c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:39.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.981+0000 7feab260a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaac11eeb0 con 0x7feaac133fa0 2023-10-03T10:17:39.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.981+0000 7feab260a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaac122280 con 0x7feaac0a1340 2023-10-03T10:17:39.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.981+0000 7feab260a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feaac120570 con 0x7feaac0a0a30 2023-10-03T10:17:39.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.981+0000 7feaab7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaac0a1340 0x7feaac1c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:39.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.981+0000 7feaab7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaac0a1340 0x7feaac1c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38946/0 (socket says 172.21.15.3:38946) 2023-10-03T10:17:39.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.981+0000 7feab0ba7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaac133fa0 0x7feaac1c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:39.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.981+0000 7feaab7fe700 1 -- 172.21.15.3:0/3583205211 learned_addr learned my addr 172.21.15.3:0/3583205211 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:39.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.981+0000 7feaabfff700 1 --2- 172.21.15.3:0/3583205211 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaac0a0a30 0x7feaac1c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:39.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.981+0000 7feaab7fe700 1 -- 172.21.15.3:0/3583205211 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaac0a0a30 msgr2=0x7feaac1c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:39.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.981+0000 7feaab7fe700 1 --2- 172.21.15.3:0/3583205211 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaac0a0a30 0x7feaac1c8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:39.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.981+0000 7feaab7fe700 1 -- 172.21.15.3:0/3583205211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaac133fa0 msgr2=0x7feaac1c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:39.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.981+0000 7feaab7fe700 1 --2- 172.21.15.3:0/3583205211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaac133fa0 0x7feaac1c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:39.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.981+0000 7feaab7fe700 1 -- 172.21.15.3:0/3583205211 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa00094a0 con 0x7feaac0a1340 2023-10-03T10:17:39.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.981+0000 7feaabfff700 1 --2- 172.21.15.3:0/3583205211 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaac0a0a30 0x7feaac1c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:39.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.981+0000 7feab0ba7700 1 --2- 172.21.15.3:0/3583205211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaac133fa0 0x7feaac1c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:39.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.982+0000 7feaab7fe700 1 --2- 172.21.15.3:0/3583205211 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaac0a1340 0x7feaac1c8a40 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7feaa000ec10 tx=0x7feaa000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:39.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.982+0000 7feaa8ff9700 1 -- 172.21.15.3:0/3583205211 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa000c850 con 0x7feaac0a1340 2023-10-03T10:17:39.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.982+0000 7feaa8ff9700 1 -- 172.21.15.3:0/3583205211 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa000c9b0 con 0x7feaac0a1340 2023-10-03T10:17:39.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.982+0000 7feab260a700 1 -- 172.21.15.3:0/3583205211 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea9c009550 con 0x7feaac0a1340 2023-10-03T10:17:39.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.982+0000 7feaa8ff9700 1 -- 172.21.15.3:0/3583205211 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa000cd20 con 0x7feaac0a1340 2023-10-03T10:17:39.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.982+0000 7feab260a700 1 -- 172.21.15.3:0/3583205211 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaac0a25e0 con 0x7feaac0a1340 2023-10-03T10:17:39.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.983+0000 7feaa8ff9700 1 -- 172.21.15.3:0/3583205211 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feaa0013470 con 0x7feaac0a1340 2023-10-03T10:17:39.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.983+0000 7feaa8ff9700 1 -- 172.21.15.3:0/3583205211 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7feaa003f830 con 0x7feaac0a1340 2023-10-03T10:17:39.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.983+0000 7feab260a700 1 -- 172.21.15.3:0/3583205211 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaac0a53c0 con 0x7feaac0a1340 2023-10-03T10:17:39.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:39.988+0000 7feaa8ff9700 1 -- 172.21.15.3:0/3583205211 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feaa0013720 con 0x7feaac0a1340 2023-10-03T10:17:40.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.018+0000 7fbed3d5c700 1 Processor -- start 2023-10-03T10:17:40.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.019+0000 7fbed3d5c700 1 -- start start 2023-10-03T10:17:40.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.019+0000 7fbed3d5c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbecc06a770 0x7fbecc09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:40.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.019+0000 7fbed3d5c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbecc13d1f0 0x7fbecc13f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:40.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.019+0000 7fbed3d5c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbecc09a810 con 0x7fbecc09dc30 2023-10-03T10:17:40.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.019+0000 7fbed3d5c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbecc06ab40 con 0x7fbecc06a770 2023-10-03T10:17:40.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.019+0000 7fbed3d5c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc09ebc0 con 0x7fbecc13d1f0 2023-10-03T10:17:40.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.019+0000 7fbed1af8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbecc06a770 0x7fbecc09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:40.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.019+0000 7fbed1af8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbecc06a770 0x7fbecc09d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39868/0 (socket says 172.21.15.3:39868) 2023-10-03T10:17:40.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.019+0000 7fbed22f9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbecc13d1f0 0x7fbecc13f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:40.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.019+0000 7fbed1af8700 1 -- 172.21.15.3:0/1127784156 learned_addr learned my addr 172.21.15.3:0/1127784156 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:40.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.019+0000 7fbed12f7700 1 --1- 172.21.15.3:0/1127784156 >> v1:172.21.15.3:6789/0 conn(0x7fbecc09dc30 0x7fbecc09e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33108/0 (socket says 172.21.15.3:33108) 2023-10-03T10:17:40.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.020+0000 7fbed22f9700 1 -- 172.21.15.3:0/1127784156 >> v2:172.21.15.3:3300/0 conn(0x7fbecc06a770 msgr2=0x7fbecc09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:40.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.020+0000 7fbed22f9700 1 --2- 172.21.15.3:0/1127784156 >> v2:172.21.15.3:3300/0 conn(0x7fbecc06a770 0x7fbecc09d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.020+0000 7fbed22f9700 1 -- 172.21.15.3:0/1127784156 >> v1:172.21.15.3:6789/0 conn(0x7fbecc09dc30 legacy=0x7fbecc09e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.020+0000 7fbed22f9700 1 -- 172.21.15.3:0/1127784156 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc09e920 con 0x7fbecc13d1f0 2023-10-03T10:17:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.020+0000 7fbed0af6700 1 -- 172.21.15.3:0/1127784156 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1457716878 0 0) 0x7fbecc09a810 con 0x7fbecc09dc30 2023-10-03T10:17:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.020+0000 7fbed1af8700 1 --2- 172.21.15.3:0/1127784156 >> v2:172.21.15.3:3300/0 conn(0x7fbecc06a770 0x7fbecc09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.020+0000 7fbed22f9700 1 --2- 172.21.15.3:0/1127784156 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbecc13d1f0 0x7fbecc13f5d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fbec4009d90 tx=0x7fbec400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a8ad67d37e495a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.020+0000 7fbed0af6700 1 -- 172.21.15.3:0/1127784156 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec400e070 con 0x7fbecc13d1f0 2023-10-03T10:17:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.020+0000 7fbed0af6700 1 -- 172.21.15.3:0/1127784156 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec400bb20 con 0x7fbecc13d1f0 2023-10-03T10:17:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.020+0000 7fbed0af6700 1 -- 172.21.15.3:0/1127784156 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec4012430 con 0x7fbecc13d1f0 2023-10-03T10:17:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.021+0000 7fbed3d5c700 1 -- 172.21.15.3:0/1127784156 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbecc13d1f0 msgr2=0x7fbecc13f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.021+0000 7fbed3d5c700 1 --2- 172.21.15.3:0/1127784156 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbecc13d1f0 0x7fbecc13f5d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fbec4009d90 tx=0x7fbec400b330 comp rx=0 tx=0).stop 2023-10-03T10:17:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.021+0000 7fbed3d5c700 1 -- 172.21.15.3:0/1127784156 shutdown_connections 2023-10-03T10:17:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.021+0000 7fbed3d5c700 1 --2- 172.21.15.3:0/1127784156 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbecc13d1f0 0x7fbecc13f5d0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:40.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.021+0000 7fbed3d5c700 1 --2- 172.21.15.3:0/1127784156 >> v2:172.21.15.3:3300/0 conn(0x7fbecc06a770 0x7fbecc09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:40.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.021+0000 7fbed3d5c700 1 -- 172.21.15.3:0/1127784156 >> 172.21.15.3:0/1127784156 conn(0x7fbecc066f50 msgr2=0x7fbecc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:40.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.021+0000 7fbed3d5c700 1 -- 172.21.15.3:0/1127784156 shutdown_connections 2023-10-03T10:17:40.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.021+0000 7fbed3d5c700 1 -- 172.21.15.3:0/1127784156 wait complete. 2023-10-03T10:17:40.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.023+0000 7fbed3d5c700 1 Processor -- start 2023-10-03T10:17:40.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.023+0000 7fbed3d5c700 1 -- start start 2023-10-03T10:17:40.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.023+0000 7fbed3d5c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbecc06a770 0x7fbecc11c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:40.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.023+0000 7fbed3d5c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbecc09dc30 0x7fbecc13b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:40.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.024+0000 7fbed3d5c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbecc13d1f0 0x7fbecc13bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:40.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.024+0000 7fbed3d5c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc135cd0 con 0x7fbecc06a770 2023-10-03T10:17:40.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.024+0000 7fbed3d5c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc11ef00 con 0x7fbecc09dc30 2023-10-03T10:17:40.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.024+0000 7fbed3d5c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc09f1a0 con 0x7fbecc13d1f0 2023-10-03T10:17:40.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.024+0000 7fbed12f7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbecc09dc30 0x7fbecc13b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:40.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.024+0000 7fbed12f7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbecc09dc30 0x7fbecc13b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:38966/0 (socket says 172.21.15.3:38966) 2023-10-03T10:17:40.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.024+0000 7fbed12f7700 1 -- 172.21.15.3:0/2131874595 learned_addr learned my addr 172.21.15.3:0/2131874595 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:40.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.024+0000 7fbed1af8700 1 --2- 172.21.15.3:0/2131874595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbecc06a770 0x7fbecc11c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:40.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.024+0000 7fbed22f9700 1 --2- 172.21.15.3:0/2131874595 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbecc13d1f0 0x7fbecc13bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:40.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.024+0000 7fbed12f7700 1 -- 172.21.15.3:0/2131874595 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbecc13d1f0 msgr2=0x7fbecc13bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:40.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.024+0000 7fbed12f7700 1 --2- 172.21.15.3:0/2131874595 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbecc13d1f0 0x7fbecc13bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:40.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.024+0000 7fbed12f7700 1 -- 172.21.15.3:0/2131874595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbecc06a770 msgr2=0x7fbecc11c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:40.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.024+0000 7fbed12f7700 1 --2- 172.21.15.3:0/2131874595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbecc06a770 0x7fbecc11c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:40.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.024+0000 7fbed12f7700 1 -- 172.21.15.3:0/2131874595 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec8009550 con 0x7fbecc09dc30 2023-10-03T10:17:40.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.024+0000 7fbed1af8700 1 --2- 172.21.15.3:0/2131874595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbecc06a770 0x7fbecc11c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:40.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.024+0000 7fbed12f7700 1 --2- 172.21.15.3:0/2131874595 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbecc09dc30 0x7fbecc13b600 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00c520 tx=0x7fbebc00c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:40.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.024+0000 7fbec27fc700 1 -- 172.21.15.3:0/2131874595 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc004c80 con 0x7fbecc09dc30 2023-10-03T10:17:40.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.025+0000 7fbec27fc700 1 -- 172.21.15.3:0/2131874595 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbebc004e00 con 0x7fbecc09dc30 2023-10-03T10:17:40.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.025+0000 7fbed3d5c700 1 -- 172.21.15.3:0/2131874595 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec4009a70 con 0x7fbecc09dc30 2023-10-03T10:17:40.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.025+0000 7fbec27fc700 1 -- 172.21.15.3:0/2131874595 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc007310 con 0x7fbecc09dc30 2023-10-03T10:17:40.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.025+0000 7fbed3d5c700 1 -- 172.21.15.3:0/2131874595 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbecc13ce50 con 0x7fbecc09dc30 2023-10-03T10:17:40.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.026+0000 7fbec27fc700 1 -- 172.21.15.3:0/2131874595 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbebc007470 con 0x7fbecc09dc30 2023-10-03T10:17:40.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.026+0000 7fbed3d5c700 1 -- 172.21.15.3:0/2131874595 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbecc074ea0 con 0x7fbecc09dc30 2023-10-03T10:17:40.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.026+0000 7fbec27fc700 1 -- 172.21.15.3:0/2131874595 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fbebc03f340 con 0x7fbecc09dc30 2023-10-03T10:17:40.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.031+0000 7fbec27fc700 1 -- 172.21.15.3:0/2131874595 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbebc0133c0 con 0x7fbecc09dc30 2023-10-03T10:17:40.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.147+0000 7fb1fe545700 1 -- 172.21.15.3:0/4115768164 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb1f80a34d0 con 0x7fb1f809f650 2023-10-03T10:17:40.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.148+0000 7fb1f4ff9700 1 -- 172.21.15.3:0/4115768164 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb1f001a020 con 0x7fb1f809f650 2023-10-03T10:17:40.148 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:40.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.150+0000 7fb1fe545700 1 -- 172.21.15.3:0/4115768164 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1f809f650 msgr2=0x7fb1f81a4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:40.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.150+0000 7fb1fe545700 1 --2- 172.21.15.3:0/4115768164 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1f809f650 0x7fb1f81a4730 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0007e70 tx=0x7fb1f000d3b0 comp rx=0 tx=0).stop 2023-10-03T10:17:40.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.150+0000 7fb1fe545700 1 -- 172.21.15.3:0/4115768164 shutdown_connections 2023-10-03T10:17:40.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.150+0000 7fb1fe545700 1 --2- 172.21.15.3:0/4115768164 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1f809f650 0x7fb1f81a4730 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:40.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.150+0000 7fb1fe545700 1 --2- 172.21.15.3:0/4115768164 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb1f809e930 0x7fb1f81a3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:40.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.150+0000 7fb1fe545700 1 --2- 172.21.15.3:0/4115768164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1f809f240 0x7fb1f81a41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:40.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.150+0000 7fb1fe545700 1 -- 172.21.15.3:0/4115768164 >> 172.21.15.3:0/4115768164 conn(0x7fb1f80a4210 msgr2=0x7fb1f81b8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:40.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.150+0000 7fb1fe545700 1 -- 172.21.15.3:0/4115768164 shutdown_connections 2023-10-03T10:17:40.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.150+0000 7fb1fe545700 1 -- 172.21.15.3:0/4115768164 wait complete. 2023-10-03T10:17:40.164 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:40.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.195+0000 7feab260a700 1 -- 172.21.15.3:0/3583205211 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7feaac074ea0 con 0x7feaac0a1340 2023-10-03T10:17:40.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.196+0000 7feaa8ff9700 1 -- 172.21.15.3:0/3583205211 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feaa0021020 con 0x7feaac0a1340 2023-10-03T10:17:40.196 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:40.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.198+0000 7feab260a700 1 -- 172.21.15.3:0/3583205211 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaac0a1340 msgr2=0x7feaac1c8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:40.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.198+0000 7feab260a700 1 --2- 172.21.15.3:0/3583205211 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaac0a1340 0x7feaac1c8a40 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7feaa000ec10 tx=0x7feaa000ef20 comp rx=0 tx=0).stop 2023-10-03T10:17:40.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.198+0000 7feab260a700 1 -- 172.21.15.3:0/3583205211 shutdown_connections 2023-10-03T10:17:40.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.198+0000 7feab260a700 1 --2- 172.21.15.3:0/3583205211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaac133fa0 0x7feaac1c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:40.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.198+0000 7feab260a700 1 --2- 172.21.15.3:0/3583205211 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feaac0a0a30 0x7feaac1c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:40.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.198+0000 7feab260a700 1 --2- 172.21.15.3:0/3583205211 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feaac0a1340 0x7feaac1c8a40 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:40.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.198+0000 7feab260a700 1 -- 172.21.15.3:0/3583205211 >> 172.21.15.3:0/3583205211 conn(0x7feaac0a6550 msgr2=0x7feaac1b9120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:40.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.198+0000 7feab260a700 1 -- 172.21.15.3:0/3583205211 shutdown_connections 2023-10-03T10:17:40.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.198+0000 7feab260a700 1 -- 172.21.15.3:0/3583205211 wait complete. 2023-10-03T10:17:40.217 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:40.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.246+0000 7fbed3d5c700 1 -- 172.21.15.3:0/2131874595 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbecc09a3d0 con 0x7fbecc09dc30 2023-10-03T10:17:40.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.247+0000 7fbec27fc700 1 -- 172.21.15.3:0/2131874595 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbebc011360 con 0x7fbecc09dc30 2023-10-03T10:17:40.247 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:40.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.249+0000 7fbed3d5c700 1 -- 172.21.15.3:0/2131874595 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbecc09dc30 msgr2=0x7fbecc13b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:40.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.249+0000 7fbed3d5c700 1 --2- 172.21.15.3:0/2131874595 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbecc09dc30 0x7fbecc13b600 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00c520 tx=0x7fbebc00c830 comp rx=0 tx=0).stop 2023-10-03T10:17:40.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.249+0000 7fbed3d5c700 1 -- 172.21.15.3:0/2131874595 shutdown_connections 2023-10-03T10:17:40.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.249+0000 7fbed3d5c700 1 --2- 172.21.15.3:0/2131874595 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbecc13d1f0 0x7fbecc13bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:40.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.249+0000 7fbed3d5c700 1 --2- 172.21.15.3:0/2131874595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbecc06a770 0x7fbecc11c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:40.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.249+0000 7fbed3d5c700 1 --2- 172.21.15.3:0/2131874595 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbecc09dc30 0x7fbecc13b600 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:40.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.249+0000 7fbed3d5c700 1 -- 172.21.15.3:0/2131874595 >> 172.21.15.3:0/2131874595 conn(0x7fbecc066f50 msgr2=0x7fbecc09b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:40.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.249+0000 7fbed3d5c700 1 -- 172.21.15.3:0/2131874595 shutdown_connections 2023-10-03T10:17:40.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.249+0000 7fbed3d5c700 1 -- 172.21.15.3:0/2131874595 wait complete. 2023-10-03T10:17:40.262 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:40.567 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:40.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.713+0000 7ff65e120700 1 Processor -- start 2023-10-03T10:17:40.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.713+0000 7ff65e120700 1 -- start start 2023-10-03T10:17:40.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.713+0000 7ff65e120700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff65809d6f0 0x7ff65809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:40.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.713+0000 7ff65e120700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff65809e000 0x7ff6581339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:40.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.713+0000 7ff65e120700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff65809b120 con 0x7ff65809e410 2023-10-03T10:17:40.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.713+0000 7ff65e120700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff65806b460 con 0x7ff65809d6f0 2023-10-03T10:17:40.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.713+0000 7ff65e120700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff65809a7e0 con 0x7ff65809e000 2023-10-03T10:17:40.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.714+0000 7ff656ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff65809e000 0x7ff6581339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:40.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.714+0000 7ff6577fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff65809d6f0 0x7ff65809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:40.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.714+0000 7ff656ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff65809e000 0x7ff6581339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54092/0 (socket says 172.21.15.3:54092) 2023-10-03T10:17:40.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.714+0000 7ff6577fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff65809d6f0 0x7ff65809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39880/0 (socket says 172.21.15.3:39880) 2023-10-03T10:17:40.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.714+0000 7ff656ffd700 1 -- 172.21.15.3:0/646035512 learned_addr learned my addr 172.21.15.3:0/646035512 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:40.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.714+0000 7ff656ffd700 1 -- 172.21.15.3:0/646035512 >> v2:172.21.15.3:3300/0 conn(0x7ff65809d6f0 msgr2=0x7ff65809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:40.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.714+0000 7ff656ffd700 1 --2- 172.21.15.3:0/646035512 >> v2:172.21.15.3:3300/0 conn(0x7ff65809d6f0 0x7ff65809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:40.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.714+0000 7ff656ffd700 1 -- 172.21.15.3:0/646035512 >> v1:172.21.15.3:6789/0 conn(0x7ff65809e410 legacy=0x7ff658133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:40.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.714+0000 7ff656ffd700 1 -- 172.21.15.3:0/646035512 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65809a500 con 0x7ff65809e000 2023-10-03T10:17:40.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.714+0000 7ff6577fe700 1 --2- 172.21.15.3:0/646035512 >> v2:172.21.15.3:3300/0 conn(0x7ff65809d6f0 0x7ff65809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:40.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.714+0000 7ff656ffd700 1 --2- 172.21.15.3:0/646035512 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff65809e000 0x7ff6581339f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff640009920 tx=0x7ff640009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=671a75add7df7ea7 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:40.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.714+0000 7ff6567fc700 1 -- 172.21.15.3:0/646035512 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff640011830 con 0x7ff65809e000 2023-10-03T10:17:40.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.714+0000 7ff6567fc700 1 -- 172.21.15.3:0/646035512 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff640011990 con 0x7ff65809e000 2023-10-03T10:17:40.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.714+0000 7ff6567fc700 1 -- 172.21.15.3:0/646035512 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff640011d20 con 0x7ff65809e000 2023-10-03T10:17:40.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.715+0000 7ff65e120700 1 -- 172.21.15.3:0/646035512 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff65809e000 msgr2=0x7ff6581339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:40.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.715+0000 7ff65e120700 1 --2- 172.21.15.3:0/646035512 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff65809e000 0x7ff6581339f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff640009920 tx=0x7ff640009c30 comp rx=0 tx=0).stop 2023-10-03T10:17:40.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.715+0000 7ff65e120700 1 -- 172.21.15.3:0/646035512 shutdown_connections 2023-10-03T10:17:40.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.715+0000 7ff65e120700 1 --2- 172.21.15.3:0/646035512 >> v2:172.21.15.3:3300/0 conn(0x7ff65809d6f0 0x7ff65809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:40.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.715+0000 7ff65e120700 1 --2- 172.21.15.3:0/646035512 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff65809e000 0x7ff6581339f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:40.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.715+0000 7ff65e120700 1 -- 172.21.15.3:0/646035512 >> 172.21.15.3:0/646035512 conn(0x7ff658066f50 msgr2=0x7ff65809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:40.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.715+0000 7ff65e120700 1 -- 172.21.15.3:0/646035512 shutdown_connections 2023-10-03T10:17:40.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.715+0000 7ff65e120700 1 -- 172.21.15.3:0/646035512 wait complete. 2023-10-03T10:17:40.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.716+0000 7ff65e120700 1 Processor -- start 2023-10-03T10:17:40.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.716+0000 7ff65e120700 1 -- start start 2023-10-03T10:17:40.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.717+0000 7ff65e120700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff65809d6f0 0x7ff6581a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:40.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.717+0000 7ff65e120700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff65809e000 0x7ff6581a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:40.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.717+0000 7ff65e120700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff65809e410 0x7ff6581a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:40.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.717+0000 7ff65e120700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6581b8420 con 0x7ff65809e000 2023-10-03T10:17:40.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.717+0000 7ff65e120700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6581a7260 con 0x7ff65809e410 2023-10-03T10:17:40.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.717+0000 7ff65e120700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6581a7e90 con 0x7ff65809d6f0 2023-10-03T10:17:40.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.717+0000 7ff6577fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff65809d6f0 0x7ff6581a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:40.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.717+0000 7ff656ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff65809e000 0x7ff6581a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:40.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.717+0000 7ff6577fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff65809d6f0 0x7ff6581a5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54106/0 (socket says 172.21.15.3:54106) 2023-10-03T10:17:40.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.717+0000 7ff6577fe700 1 -- 172.21.15.3:0/845561717 learned_addr learned my addr 172.21.15.3:0/845561717 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:40.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.717+0000 7ff657fff700 1 --2- 172.21.15.3:0/845561717 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff65809e410 0x7ff6581a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:40.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.717+0000 7ff6577fe700 1 -- 172.21.15.3:0/845561717 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff65809e410 msgr2=0x7ff6581a5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:40.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.717+0000 7ff6577fe700 1 --2- 172.21.15.3:0/845561717 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff65809e410 0x7ff6581a5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:40.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.717+0000 7ff6577fe700 1 -- 172.21.15.3:0/845561717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff65809e000 msgr2=0x7ff6581a55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:40.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.717+0000 7ff6577fe700 1 --2- 172.21.15.3:0/845561717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff65809e000 0x7ff6581a55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:40.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.717+0000 7ff6577fe700 1 -- 172.21.15.3:0/845561717 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6480094a0 con 0x7ff65809d6f0 2023-10-03T10:17:40.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.717+0000 7ff6577fe700 1 --2- 172.21.15.3:0/845561717 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff65809d6f0 0x7ff6581a5080 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff64800ebe0 tx=0x7ff64800eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:40.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.718+0000 7ff64ffff700 1 -- 172.21.15.3:0/845561717 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff648009760 con 0x7ff65809d6f0 2023-10-03T10:17:40.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.718+0000 7ff64ffff700 1 -- 172.21.15.3:0/845561717 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff64800c850 con 0x7ff65809d6f0 2023-10-03T10:17:40.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.718+0000 7ff65e120700 1 -- 172.21.15.3:0/845561717 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff640009550 con 0x7ff65809d6f0 2023-10-03T10:17:40.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.718+0000 7ff64ffff700 1 -- 172.21.15.3:0/845561717 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64800cbe0 con 0x7ff65809d6f0 2023-10-03T10:17:40.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.718+0000 7ff65e120700 1 -- 172.21.15.3:0/845561717 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6581a82e0 con 0x7ff65809d6f0 2023-10-03T10:17:40.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.718+0000 7ff64ffff700 1 -- 172.21.15.3:0/845561717 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff64800cd40 con 0x7ff65809d6f0 2023-10-03T10:17:40.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.719+0000 7ff64ffff700 1 -- 172.21.15.3:0/845561717 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff64803f750 con 0x7ff65809d6f0 2023-10-03T10:17:40.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.719+0000 7ff65e120700 1 -- 172.21.15.3:0/845561717 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff65809c710 con 0x7ff65809d6f0 2023-10-03T10:17:40.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.724+0000 7ff64ffff700 1 -- 172.21.15.3:0/845561717 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff6480232f0 con 0x7ff65809d6f0 2023-10-03T10:17:40.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.927+0000 7ff65e120700 1 -- 172.21.15.3:0/845561717 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff65809acc0 con 0x7ff65809d6f0 2023-10-03T10:17:40.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.927+0000 7ff64ffff700 1 -- 172.21.15.3:0/845561717 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff65809acc0 con 0x7ff65809d6f0 2023-10-03T10:17:40.927 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:40.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.929+0000 7ff65e120700 1 -- 172.21.15.3:0/845561717 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff65809d6f0 msgr2=0x7ff6581a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:40.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.929+0000 7ff65e120700 1 --2- 172.21.15.3:0/845561717 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff65809d6f0 0x7ff6581a5080 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff64800ebe0 tx=0x7ff64800eef0 comp rx=0 tx=0).stop 2023-10-03T10:17:40.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.929+0000 7ff65e120700 1 -- 172.21.15.3:0/845561717 shutdown_connections 2023-10-03T10:17:40.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.929+0000 7ff65e120700 1 --2- 172.21.15.3:0/845561717 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff65809e410 0x7ff6581a5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:40.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.929+0000 7ff65e120700 1 --2- 172.21.15.3:0/845561717 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff65809d6f0 0x7ff6581a5080 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:40.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.929+0000 7ff65e120700 1 --2- 172.21.15.3:0/845561717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff65809e000 0x7ff6581a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:40.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.929+0000 7ff65e120700 1 -- 172.21.15.3:0/845561717 >> 172.21.15.3:0/845561717 conn(0x7ff658066f50 msgr2=0x7ff6581b9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:40.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.929+0000 7ff65e120700 1 -- 172.21.15.3:0/845561717 shutdown_connections 2023-10-03T10:17:40.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:40.929+0000 7ff65e120700 1 -- 172.21.15.3:0/845561717 wait complete. 2023-10-03T10:17:40.943 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:41.166 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:41.218 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:41.263 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:41.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.316+0000 7fadf1f8f700 1 Processor -- start 2023-10-03T10:17:41.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.316+0000 7fadf1f8f700 1 -- start start 2023-10-03T10:17:41.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.316+0000 7fadf1f8f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fadec1338a0 0x7fadec133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:41.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.317+0000 7fadf1f8f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fadec09e930 0x7fadec13b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:41.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.317+0000 7fadf1f8f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadec06ef30 con 0x7fadec09de50 2023-10-03T10:17:41.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.317+0000 7fadf1f8f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadec06fef0 con 0x7fadec09e930 2023-10-03T10:17:41.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.317+0000 7fadf1f8f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadec1334b0 con 0x7fadec1338a0 2023-10-03T10:17:41.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.317+0000 7fadebfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fadec09e930 0x7fadec13b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:41.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.317+0000 7fadebfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fadec09e930 0x7fadec13b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39898/0 (socket says 172.21.15.3:39898) 2023-10-03T10:17:41.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.317+0000 7fadebfff700 1 -- 172.21.15.3:0/3479180917 learned_addr learned my addr 172.21.15.3:0/3479180917 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:41.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.317+0000 7fadeb7fe700 1 --2- 172.21.15.3:0/3479180917 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fadec1338a0 0x7fadec133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:41.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.317+0000 7fadebfff700 1 -- 172.21.15.3:0/3479180917 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fadec1338a0 msgr2=0x7fadec133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.317+0000 7fadebfff700 1 --2- 172.21.15.3:0/3479180917 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fadec1338a0 0x7fadec133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.317+0000 7fadebfff700 1 -- 172.21.15.3:0/3479180917 >> v1:172.21.15.3:6789/0 conn(0x7fadec09de50 legacy=0x7fadec09e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.317+0000 7fadebfff700 1 -- 172.21.15.3:0/3479180917 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadec06fc30 con 0x7fadec09e930 2023-10-03T10:17:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.317+0000 7fadeb7fe700 1 --2- 172.21.15.3:0/3479180917 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fadec1338a0 0x7fadec133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.317+0000 7fadebfff700 1 --2- 172.21.15.3:0/3479180917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadec09e930 0x7fadec13b740 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fade4009e50 tx=0x7fade400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9bcf36dc133280b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.317+0000 7fadea7fc700 1 -- 172.21.15.3:0/3479180917 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade400e070 con 0x7fadec09e930 2023-10-03T10:17:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.318+0000 7fadea7fc700 1 -- 172.21.15.3:0/3479180917 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade400b910 con 0x7fadec09e930 2023-10-03T10:17:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.318+0000 7fadea7fc700 1 -- 172.21.15.3:0/3479180917 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade400bcc0 con 0x7fadec09e930 2023-10-03T10:17:41.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.318+0000 7fadf1f8f700 1 -- 172.21.15.3:0/3479180917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadec09e930 msgr2=0x7fadec13b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:41.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.318+0000 7fadf1f8f700 1 --2- 172.21.15.3:0/3479180917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadec09e930 0x7fadec13b740 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fade4009e50 tx=0x7fade400b330 comp rx=0 tx=0).stop 2023-10-03T10:17:41.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.318+0000 7fadf1f8f700 1 -- 172.21.15.3:0/3479180917 shutdown_connections 2023-10-03T10:17:41.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.318+0000 7fadf1f8f700 1 --2- 172.21.15.3:0/3479180917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadec09e930 0x7fadec13b740 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:41.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.318+0000 7fadf1f8f700 1 --2- 172.21.15.3:0/3479180917 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fadec1338a0 0x7fadec133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:41.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.318+0000 7fadf1f8f700 1 -- 172.21.15.3:0/3479180917 >> 172.21.15.3:0/3479180917 conn(0x7fadec0a2aa0 msgr2=0x7fadec0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:41.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.318+0000 7fadf1f8f700 1 -- 172.21.15.3:0/3479180917 shutdown_connections 2023-10-03T10:17:41.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.318+0000 7fadf1f8f700 1 -- 172.21.15.3:0/3479180917 wait complete. 2023-10-03T10:17:41.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.319+0000 7fadf1f8f700 1 Processor -- start 2023-10-03T10:17:41.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.320+0000 7fadf1f8f700 1 -- start start 2023-10-03T10:17:41.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.320+0000 7fadf1f8f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fadec09de50 0x7fadec11d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:41.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.320+0000 7fadf1f8f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fadec09e930 0x7fadec11dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:41.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.320+0000 7fadf1f8f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadec1338a0 0x7fadec11e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:41.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.320+0000 7fadf1f8f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec1211a0 con 0x7fadec1338a0 2023-10-03T10:17:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.320+0000 7fadf1f8f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadec11f330 con 0x7fadec09de50 2023-10-03T10:17:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.320+0000 7fadf1f8f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fadec0a0fe0 con 0x7fadec09e930 2023-10-03T10:17:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.320+0000 7fadeaffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fadec09e930 0x7fadec11dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.320+0000 7fadeb7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fadec09de50 0x7fadec11d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.320+0000 7fadeb7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fadec09de50 0x7fadec11d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39004/0 (socket says 172.21.15.3:39004) 2023-10-03T10:17:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.320+0000 7fadeaffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fadec09e930 0x7fadec11dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54120/0 (socket says 172.21.15.3:54120) 2023-10-03T10:17:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.320+0000 7fadeb7fe700 1 -- 172.21.15.3:0/924526766 learned_addr learned my addr 172.21.15.3:0/924526766 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.320+0000 7fadebfff700 1 --2- 172.21.15.3:0/924526766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadec1338a0 0x7fadec11e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:41.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.321+0000 7fadeb7fe700 1 -- 172.21.15.3:0/924526766 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fadec09e930 msgr2=0x7fadec11dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:41.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.321+0000 7fadeb7fe700 1 --2- 172.21.15.3:0/924526766 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fadec09e930 0x7fadec11dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:41.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.321+0000 7fadeb7fe700 1 -- 172.21.15.3:0/924526766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadec1338a0 msgr2=0x7fadec11e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.321+0000 7fadeb7fe700 1 --2- 172.21.15.3:0/924526766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadec1338a0 0x7fadec11e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.321+0000 7fadeb7fe700 1 -- 172.21.15.3:0/924526766 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faddc0094a0 con 0x7fadec09de50 2023-10-03T10:17:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.321+0000 7fadebfff700 1 --2- 172.21.15.3:0/924526766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadec1338a0 0x7fadec11e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.321+0000 7fadeaffd700 1 --2- 172.21.15.3:0/924526766 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fadec09e930 0x7fadec11dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.321+0000 7fadeb7fe700 1 --2- 172.21.15.3:0/924526766 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fadec09de50 0x7fadec11d6d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7faddc00ebe0 tx=0x7faddc00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.321+0000 7fadd3fff700 1 -- 172.21.15.3:0/924526766 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc00c850 con 0x7fadec09de50 2023-10-03T10:17:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.321+0000 7fadd3fff700 1 -- 172.21.15.3:0/924526766 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faddc00c9b0 con 0x7fadec09de50 2023-10-03T10:17:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.321+0000 7fadd3fff700 1 -- 172.21.15.3:0/924526766 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc00cd40 con 0x7fadec09de50 2023-10-03T10:17:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.321+0000 7fadf1f8f700 1 -- 172.21.15.3:0/924526766 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade4009b30 con 0x7fadec09de50 2023-10-03T10:17:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.321+0000 7fadf1f8f700 1 -- 172.21.15.3:0/924526766 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadec120090 con 0x7fadec09de50 2023-10-03T10:17:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.322+0000 7fadd3fff700 1 -- 172.21.15.3:0/924526766 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faddc013470 con 0x7fadec09de50 2023-10-03T10:17:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.322+0000 7fadd3fff700 1 -- 172.21.15.3:0/924526766 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7faddc03fb70 con 0x7fadec09de50 2023-10-03T10:17:41.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.323+0000 7fadf1f8f700 1 -- 172.21.15.3:0/924526766 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadec11a470 con 0x7fadec09de50 2023-10-03T10:17:41.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.328+0000 7fadd3fff700 1 -- 172.21.15.3:0/924526766 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faddc013720 con 0x7fadec09de50 2023-10-03T10:17:41.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.365+0000 7fab0a4aa700 1 Processor -- start 2023-10-03T10:17:41.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.365+0000 7fab0a4aa700 1 -- start start 2023-10-03T10:17:41.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.365+0000 7fab0a4aa700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab0409e850 0x7fab0409ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:41.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.365+0000 7fab0a4aa700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fab0409ffb0 0x7fab0409f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:41.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.365+0000 7fab0a4aa700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab040a06b0 con 0x7fab04133b80 2023-10-03T10:17:41.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.365+0000 7fab0a4aa700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab0406b920 con 0x7fab0409ffb0 2023-10-03T10:17:41.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.365+0000 7fab0a4aa700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab0409aa30 con 0x7fab0409e850 2023-10-03T10:17:41.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.365+0000 7fab03fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab0409e850 0x7fab0409ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:41.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.365+0000 7fab037fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fab0409ffb0 0x7fab0409f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:41.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.365+0000 7fab037fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fab0409ffb0 0x7fab0409f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39918/0 (socket says 172.21.15.3:39918) 2023-10-03T10:17:41.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.365+0000 7fab03fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab0409e850 0x7fab0409ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39016/0 (socket says 172.21.15.3:39016) 2023-10-03T10:17:41.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.365+0000 7fab037fe700 1 -- 172.21.15.3:0/1715759582 learned_addr learned my addr 172.21.15.3:0/1715759582 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:41.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.366+0000 7fab03fff700 1 -- 172.21.15.3:0/1715759582 >> v2:172.21.15.3:3300/0 conn(0x7fab0409ffb0 msgr2=0x7fab0409f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:41.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.366+0000 7fab03fff700 1 --2- 172.21.15.3:0/1715759582 >> v2:172.21.15.3:3300/0 conn(0x7fab0409ffb0 0x7fab0409f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:41.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.366+0000 7fab03fff700 1 -- 172.21.15.3:0/1715759582 >> v1:172.21.15.3:6789/0 conn(0x7fab04133b80 legacy=0x7fab04133f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:41.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.366+0000 7fab03fff700 1 -- 172.21.15.3:0/1715759582 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab0406caa0 con 0x7fab0409e850 2023-10-03T10:17:41.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.366+0000 7fab037fe700 1 --2- 172.21.15.3:0/1715759582 >> v2:172.21.15.3:3300/0 conn(0x7fab0409ffb0 0x7fab0409f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:41.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.366+0000 7fab03fff700 1 --2- 172.21.15.3:0/1715759582 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab0409e850 0x7fab0409ec20 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7faaf4009840 tx=0x7faaf4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbb64f644d56d269 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:41.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.366+0000 7fab02ffd700 1 -- 172.21.15.3:0/1715759582 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf4011830 con 0x7fab0409e850 2023-10-03T10:17:41.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.366+0000 7fab02ffd700 1 -- 172.21.15.3:0/1715759582 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf4011990 con 0x7fab0409e850 2023-10-03T10:17:41.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.366+0000 7fab02ffd700 1 -- 172.21.15.3:0/1715759582 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf4011d20 con 0x7fab0409e850 2023-10-03T10:17:41.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.367+0000 7fab0a4aa700 1 -- 172.21.15.3:0/1715759582 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab0409e850 msgr2=0x7fab0409ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:41.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.367+0000 7fab0a4aa700 1 --2- 172.21.15.3:0/1715759582 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab0409e850 0x7fab0409ec20 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7faaf4009840 tx=0x7faaf4009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:41.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.367+0000 7fab0a4aa700 1 -- 172.21.15.3:0/1715759582 shutdown_connections 2023-10-03T10:17:41.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.367+0000 7fab0a4aa700 1 --2- 172.21.15.3:0/1715759582 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab0409e850 0x7fab0409ec20 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:41.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.367+0000 7fab0a4aa700 1 --2- 172.21.15.3:0/1715759582 >> v2:172.21.15.3:3300/0 conn(0x7fab0409ffb0 0x7fab0409f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:41.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.367+0000 7fab0a4aa700 1 -- 172.21.15.3:0/1715759582 >> 172.21.15.3:0/1715759582 conn(0x7fab040a3f70 msgr2=0x7fab040a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:41.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.367+0000 7fab0a4aa700 1 -- 172.21.15.3:0/1715759582 shutdown_connections 2023-10-03T10:17:41.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.367+0000 7fab0a4aa700 1 -- 172.21.15.3:0/1715759582 wait complete. 2023-10-03T10:17:41.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.368+0000 7fab0a4aa700 1 Processor -- start 2023-10-03T10:17:41.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.368+0000 7fab0a4aa700 1 -- start start 2023-10-03T10:17:41.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.369+0000 7fab0a4aa700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab0409e850 0x7fab041228a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:41.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.369+0000 7fab0a4aa700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab0409ffb0 0x7fab04122de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:41.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.369+0000 7fab0a4aa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab04133b80 0x7fab04123320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:41.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.369+0000 7fab0a4aa700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab04125e20 con 0x7fab04133b80 2023-10-03T10:17:41.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.369+0000 7fab0a4aa700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab041244b0 con 0x7fab0409ffb0 2023-10-03T10:17:41.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.369+0000 7fab0a4aa700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fab04123d00 con 0x7fab0409e850 2023-10-03T10:17:41.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.369+0000 7fab037fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab0409ffb0 0x7fab04122de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:41.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.369+0000 7fab037fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab0409ffb0 0x7fab04122de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39018/0 (socket says 172.21.15.3:39018) 2023-10-03T10:17:41.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.369+0000 7fab037fe700 1 -- 172.21.15.3:0/229473454 learned_addr learned my addr 172.21.15.3:0/229473454 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:41.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.369+0000 7fab08a47700 1 --2- 172.21.15.3:0/229473454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab04133b80 0x7fab04123320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:41.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.369+0000 7fab08a47700 1 -- 172.21.15.3:0/229473454 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab0409e850 msgr2=0x7fab041228a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:41.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.369+0000 7fab08a47700 1 --2- 172.21.15.3:0/229473454 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab0409e850 0x7fab041228a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:41.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.369+0000 7fab08a47700 1 -- 172.21.15.3:0/229473454 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab0409ffb0 msgr2=0x7fab04122de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:41.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.369+0000 7fab08a47700 1 --2- 172.21.15.3:0/229473454 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab0409ffb0 0x7fab04122de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:41.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.369+0000 7fab08a47700 1 -- 172.21.15.3:0/229473454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf8009550 con 0x7fab04133b80 2023-10-03T10:17:41.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.369+0000 7fab037fe700 1 --2- 172.21.15.3:0/229473454 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab0409ffb0 0x7fab04122de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:41.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.370+0000 7fab08a47700 1 --2- 172.21.15.3:0/229473454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab04133b80 0x7fab04123320 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7faafc00cb90 tx=0x7faafc00cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:41.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.370+0000 7fab00ff9700 1 -- 172.21.15.3:0/229473454 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc003c00 con 0x7fab04133b80 2023-10-03T10:17:41.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.370+0000 7fab00ff9700 1 -- 172.21.15.3:0/229473454 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faafc003d60 con 0x7fab04133b80 2023-10-03T10:17:41.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.370+0000 7fab0a4aa700 1 -- 172.21.15.3:0/229473454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaf40094a0 con 0x7fab04133b80 2023-10-03T10:17:41.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.370+0000 7fab00ff9700 1 -- 172.21.15.3:0/229473454 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc00aaa0 con 0x7fab04133b80 2023-10-03T10:17:41.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.370+0000 7fab0a4aa700 1 -- 172.21.15.3:0/229473454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab0406ccc0 con 0x7fab04133b80 2023-10-03T10:17:41.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.371+0000 7fab00ff9700 1 -- 172.21.15.3:0/229473454 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faafc00ac00 con 0x7fab04133b80 2023-10-03T10:17:41.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.371+0000 7fab0a4aa700 1 -- 172.21.15.3:0/229473454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab0409f620 con 0x7fab04133b80 2023-10-03T10:17:41.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.371+0000 7fab00ff9700 1 -- 172.21.15.3:0/229473454 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7faafc03f630 con 0x7fab04133b80 2023-10-03T10:17:41.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.376+0000 7fab00ff9700 1 -- 172.21.15.3:0/229473454 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faafc0196c0 con 0x7fab04133b80 2023-10-03T10:17:41.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.423+0000 7f48a302e700 1 Processor -- start 2023-10-03T10:17:41.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.423+0000 7f48a302e700 1 -- start start 2023-10-03T10:17:41.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.423+0000 7f48a302e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f489c0a0a30 0x7f489c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:41.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.423+0000 7f48a302e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f489c138d80 0x7f489c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:41.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.423+0000 7f48a302e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f489c0a2a70 con 0x7f489c09d630 2023-10-03T10:17:41.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.423+0000 7f48a302e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f489c0a01d0 con 0x7f489c138d80 2023-10-03T10:17:41.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.423+0000 7f48a302e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f489c09e2b0 con 0x7f489c0a0a30 2023-10-03T10:17:41.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.424+0000 7f489bfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f489c09d630 0x7f489c0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33136/0 (socket says 172.21.15.3:33136) 2023-10-03T10:17:41.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.424+0000 7f489bfff700 1 -- 172.21.15.3:0/3975112521 learned_addr learned my addr 172.21.15.3:0/3975112521 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:41.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.424+0000 7f48a15cb700 1 --2- 172.21.15.3:0/3975112521 >> v2:172.21.15.3:3300/0 conn(0x7f489c138d80 0x7f489c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:41.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.424+0000 7f48a0dca700 1 --2- 172.21.15.3:0/3975112521 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f489c0a0a30 0x7f489c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:41.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.424+0000 7f48a15cb700 1 -- 172.21.15.3:0/3975112521 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f489c0a0a30 msgr2=0x7f489c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:41.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.424+0000 7f48a15cb700 1 --2- 172.21.15.3:0/3975112521 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f489c0a0a30 0x7f489c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:41.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.424+0000 7f48a15cb700 1 -- 172.21.15.3:0/3975112521 >> v1:172.21.15.3:6789/0 conn(0x7f489c09d630 legacy=0x7f489c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:41.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.424+0000 7f48a15cb700 1 -- 172.21.15.3:0/3975112521 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489c09dc50 con 0x7f489c138d80 2023-10-03T10:17:41.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.424+0000 7f489b7fe700 1 -- 172.21.15.3:0/3975112521 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213013960 0 0) 0x7f489c0a2a70 con 0x7f489c09d630 2023-10-03T10:17:41.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.424+0000 7f48a0dca700 1 --2- 172.21.15.3:0/3975112521 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f489c0a0a30 0x7f489c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:41.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.424+0000 7f48a15cb700 1 --2- 172.21.15.3:0/3975112521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f489c138d80 0x7f489c13b160 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f4894009840 tx=0x7f4894009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beaaa8e5f1c55540 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:41.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.424+0000 7f489b7fe700 1 -- 172.21.15.3:0/3975112521 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4894011830 con 0x7f489c138d80 2023-10-03T10:17:41.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.425+0000 7f489b7fe700 1 -- 172.21.15.3:0/3975112521 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4894011990 con 0x7f489c138d80 2023-10-03T10:17:41.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.425+0000 7f489b7fe700 1 -- 172.21.15.3:0/3975112521 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4894011d20 con 0x7f489c138d80 2023-10-03T10:17:41.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.425+0000 7f48a302e700 1 -- 172.21.15.3:0/3975112521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f489c138d80 msgr2=0x7f489c13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:41.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.425+0000 7f48a302e700 1 --2- 172.21.15.3:0/3975112521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f489c138d80 0x7f489c13b160 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f4894009840 tx=0x7f4894009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:41.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.425+0000 7f48a302e700 1 -- 172.21.15.3:0/3975112521 shutdown_connections 2023-10-03T10:17:41.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.425+0000 7f48a302e700 1 --2- 172.21.15.3:0/3975112521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f489c138d80 0x7f489c13b160 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:41.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.425+0000 7f48a302e700 1 --2- 172.21.15.3:0/3975112521 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f489c0a0a30 0x7f489c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:41.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.425+0000 7f48a302e700 1 -- 172.21.15.3:0/3975112521 >> 172.21.15.3:0/3975112521 conn(0x7f489c0a3050 msgr2=0x7f489c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:41.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.425+0000 7f48a302e700 1 -- 172.21.15.3:0/3975112521 shutdown_connections 2023-10-03T10:17:41.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.425+0000 7f48a302e700 1 -- 172.21.15.3:0/3975112521 wait complete. 2023-10-03T10:17:41.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.426+0000 7f48a302e700 1 Processor -- start 2023-10-03T10:17:41.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.426+0000 7f48a302e700 1 -- start start 2023-10-03T10:17:41.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.427+0000 7f48a302e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f489c09d630 0x7f489c133690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:41.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.427+0000 7f48a302e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f489c0a0a30 0x7f489c133bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:41.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.427+0000 7f48a302e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f489c138d80 0x7f489c134110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:41.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.427+0000 7f48a302e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489c1361b0 con 0x7f489c0a0a30 2023-10-03T10:17:41.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.427+0000 7f48a302e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489c136ed0 con 0x7f489c09d630 2023-10-03T10:17:41.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.427+0000 7f48a302e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f489c0a4b60 con 0x7f489c138d80 2023-10-03T10:17:41.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.427+0000 7f489bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f489c0a0a30 0x7f489c133bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:41.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.427+0000 7f489bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f489c0a0a30 0x7f489c133bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39946/0 (socket says 172.21.15.3:39946) 2023-10-03T10:17:41.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.427+0000 7f489bfff700 1 -- 172.21.15.3:0/1841759197 learned_addr learned my addr 172.21.15.3:0/1841759197 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:41.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.427+0000 7f48a0dca700 1 --2- 172.21.15.3:0/1841759197 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f489c09d630 0x7f489c133690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:41.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.427+0000 7f48a15cb700 1 --2- 172.21.15.3:0/1841759197 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f489c138d80 0x7f489c134110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:41.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.428+0000 7f489bfff700 1 -- 172.21.15.3:0/1841759197 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f489c138d80 msgr2=0x7f489c134110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:41.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.428+0000 7f489bfff700 1 --2- 172.21.15.3:0/1841759197 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f489c138d80 0x7f489c134110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:41.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.428+0000 7f489bfff700 1 -- 172.21.15.3:0/1841759197 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f489c09d630 msgr2=0x7f489c133690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:41.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.428+0000 7f489bfff700 1 --2- 172.21.15.3:0/1841759197 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f489c09d630 0x7f489c133690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:41.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.428+0000 7f489bfff700 1 -- 172.21.15.3:0/1841759197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488c009a70 con 0x7f489c0a0a30 2023-10-03T10:17:41.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.428+0000 7f489bfff700 1 --2- 172.21.15.3:0/1841759197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f489c0a0a30 0x7f489c133bd0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f489000c520 tx=0x7f489000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:41.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.428+0000 7f48997fa700 1 -- 172.21.15.3:0/1841759197 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4890004c80 con 0x7f489c0a0a30 2023-10-03T10:17:41.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.428+0000 7f48997fa700 1 -- 172.21.15.3:0/1841759197 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4890004de0 con 0x7f489c0a0a30 2023-10-03T10:17:41.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.428+0000 7f48a302e700 1 -- 172.21.15.3:0/1841759197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48940094a0 con 0x7f489c0a0a30 2023-10-03T10:17:41.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.428+0000 7f48997fa700 1 -- 172.21.15.3:0/1841759197 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4890007310 con 0x7f489c0a0a30 2023-10-03T10:17:41.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.428+0000 7f48a302e700 1 -- 172.21.15.3:0/1841759197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f489c136560 con 0x7f489c0a0a30 2023-10-03T10:17:41.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.429+0000 7f48997fa700 1 -- 172.21.15.3:0/1841759197 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4890007470 con 0x7f489c0a0a30 2023-10-03T10:17:41.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.429+0000 7f48997fa700 1 -- 172.21.15.3:0/1841759197 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f489003f4b0 con 0x7f489c0a0a30 2023-10-03T10:17:41.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.429+0000 7f48a302e700 1 -- 172.21.15.3:0/1841759197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f489c137a00 con 0x7f489c0a0a30 2023-10-03T10:17:41.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.434+0000 7f48997fa700 1 -- 172.21.15.3:0/1841759197 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f48900123b0 con 0x7f489c0a0a30 2023-10-03T10:17:41.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.559+0000 7fadf1f8f700 1 -- 172.21.15.3:0/924526766 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fadec074ea0 con 0x7fadec09de50 2023-10-03T10:17:41.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.559+0000 7fadd3fff700 1 -- 172.21.15.3:0/924526766 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faddc017020 con 0x7fadec09de50 2023-10-03T10:17:41.560 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:41.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.562+0000 7fadf1f8f700 1 -- 172.21.15.3:0/924526766 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fadec09de50 msgr2=0x7fadec11d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:41.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.562+0000 7fadf1f8f700 1 --2- 172.21.15.3:0/924526766 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fadec09de50 0x7fadec11d6d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7faddc00ebe0 tx=0x7faddc00eef0 comp rx=0 tx=0).stop 2023-10-03T10:17:41.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.562+0000 7fadf1f8f700 1 -- 172.21.15.3:0/924526766 shutdown_connections 2023-10-03T10:17:41.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.562+0000 7fadf1f8f700 1 --2- 172.21.15.3:0/924526766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadec1338a0 0x7fadec11e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:41.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.562+0000 7fadf1f8f700 1 --2- 172.21.15.3:0/924526766 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fadec09de50 0x7fadec11d6d0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:41.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.562+0000 7fadf1f8f700 1 --2- 172.21.15.3:0/924526766 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fadec09e930 0x7fadec11dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:41.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.562+0000 7fadf1f8f700 1 -- 172.21.15.3:0/924526766 >> 172.21.15.3:0/924526766 conn(0x7fadec0a2aa0 msgr2=0x7fadec1baea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:41.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.562+0000 7fadf1f8f700 1 -- 172.21.15.3:0/924526766 shutdown_connections 2023-10-03T10:17:41.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.562+0000 7fadf1f8f700 1 -- 172.21.15.3:0/924526766 wait complete. 2023-10-03T10:17:41.576 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:41.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.588+0000 7fab0a4aa700 1 -- 172.21.15.3:0/229473454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fab040747f0 con 0x7fab04133b80 2023-10-03T10:17:41.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.588+0000 7fab00ff9700 1 -- 172.21.15.3:0/229473454 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faafc019300 con 0x7fab04133b80 2023-10-03T10:17:41.589 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:41.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.591+0000 7fab0a4aa700 1 -- 172.21.15.3:0/229473454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab04133b80 msgr2=0x7fab04123320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:41.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.591+0000 7fab0a4aa700 1 --2- 172.21.15.3:0/229473454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab04133b80 0x7fab04123320 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7faafc00cb90 tx=0x7faafc00cea0 comp rx=0 tx=0).stop 2023-10-03T10:17:41.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.591+0000 7fab0a4aa700 1 -- 172.21.15.3:0/229473454 shutdown_connections 2023-10-03T10:17:41.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.591+0000 7fab0a4aa700 1 --2- 172.21.15.3:0/229473454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab04133b80 0x7fab04123320 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:41.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.591+0000 7fab0a4aa700 1 --2- 172.21.15.3:0/229473454 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab0409e850 0x7fab041228a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:41.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.591+0000 7fab0a4aa700 1 --2- 172.21.15.3:0/229473454 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab0409ffb0 0x7fab04122de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:41.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.591+0000 7fab0a4aa700 1 -- 172.21.15.3:0/229473454 >> 172.21.15.3:0/229473454 conn(0x7fab040a3f70 msgr2=0x7fab0409d460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:41.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.591+0000 7fab0a4aa700 1 -- 172.21.15.3:0/229473454 shutdown_connections 2023-10-03T10:17:41.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.591+0000 7fab0a4aa700 1 -- 172.21.15.3:0/229473454 wait complete. 2023-10-03T10:17:41.605 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:41.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.638+0000 7f48a302e700 1 -- 172.21.15.3:0/1841759197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f489c0747f0 con 0x7f489c0a0a30 2023-10-03T10:17:41.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.638+0000 7f48997fa700 1 -- 172.21.15.3:0/1841759197 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4890011360 con 0x7f489c0a0a30 2023-10-03T10:17:41.639 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:41.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.641+0000 7f48a302e700 1 -- 172.21.15.3:0/1841759197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f489c0a0a30 msgr2=0x7f489c133bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:41.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.641+0000 7f48a302e700 1 --2- 172.21.15.3:0/1841759197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f489c0a0a30 0x7f489c133bd0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f489000c520 tx=0x7f489000c830 comp rx=0 tx=0).stop 2023-10-03T10:17:41.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.641+0000 7f48a302e700 1 -- 172.21.15.3:0/1841759197 shutdown_connections 2023-10-03T10:17:41.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.641+0000 7f48a302e700 1 --2- 172.21.15.3:0/1841759197 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f489c138d80 0x7f489c134110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:41.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.641+0000 7f48a302e700 1 --2- 172.21.15.3:0/1841759197 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f489c09d630 0x7f489c133690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:41.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.641+0000 7f48a302e700 1 --2- 172.21.15.3:0/1841759197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f489c0a0a30 0x7f489c133bd0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:41.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.641+0000 7f48a302e700 1 -- 172.21.15.3:0/1841759197 >> 172.21.15.3:0/1841759197 conn(0x7f489c0a3050 msgr2=0x7f489c13af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:41.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.641+0000 7f48a302e700 1 -- 172.21.15.3:0/1841759197 shutdown_connections 2023-10-03T10:17:41.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:41.641+0000 7f48a302e700 1 -- 172.21.15.3:0/1841759197 wait complete. 2023-10-03T10:17:41.654 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:41.945 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:42.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.091+0000 7f6a3fdca700 1 Processor -- start 2023-10-03T10:17:42.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.091+0000 7f6a3fdca700 1 -- start start 2023-10-03T10:17:42.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.092+0000 7f6a3fdca700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6a3809e1d0 0x7f6a38135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:42.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.092+0000 7f6a3fdca700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a38135e50 0x7f6a38138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:42.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.092+0000 7f6a3fdca700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3809f130 con 0x7f6a3809d6f0 2023-10-03T10:17:42.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.092+0000 7f6a3fdca700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3809f520 con 0x7f6a38135e50 2023-10-03T10:17:42.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.092+0000 7f6a3fdca700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3809e930 con 0x7f6a3809e1d0 2023-10-03T10:17:42.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.092+0000 7f6a3d365700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6a3809e1d0 0x7f6a38135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:42.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.092+0000 7f6a3e367700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a38135e50 0x7f6a38138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:42.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.092+0000 7f6a3db66700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6a3809d6f0 0x7f6a3809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33144/0 (socket says 172.21.15.3:33144) 2023-10-03T10:17:42.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.092+0000 7f6a3db66700 1 -- 172.21.15.3:0/2159261572 learned_addr learned my addr 172.21.15.3:0/2159261572 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:42.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.092+0000 7f6a3cb64700 1 -- 172.21.15.3:0/2159261572 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 410903404 0 0) 0x7f6a3809f130 con 0x7f6a3809d6f0 2023-10-03T10:17:42.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.092+0000 7f6a3d365700 1 -- 172.21.15.3:0/2159261572 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a38135e50 msgr2=0x7f6a38138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:42.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.092+0000 7f6a3d365700 1 --2- 172.21.15.3:0/2159261572 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a38135e50 0x7f6a38138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.092+0000 7f6a3d365700 1 -- 172.21.15.3:0/2159261572 >> v1:172.21.15.3:6789/0 conn(0x7f6a3809d6f0 legacy=0x7f6a3809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:42.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.092+0000 7f6a3d365700 1 -- 172.21.15.3:0/2159261572 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3809b2d0 con 0x7f6a3809e1d0 2023-10-03T10:17:42.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.092+0000 7f6a3e367700 1 --2- 172.21.15.3:0/2159261572 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a38135e50 0x7f6a38138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:42.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.093+0000 7f6a3d365700 1 --2- 172.21.15.3:0/2159261572 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6a3809e1d0 0x7f6a38135910 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6a28009870 tx=0x7f6a28009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b04160406731894 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:42.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.093+0000 7f6a3cb64700 1 -- 172.21.15.3:0/2159261572 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a28011830 con 0x7f6a3809e1d0 2023-10-03T10:17:42.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.093+0000 7f6a3cb64700 1 -- 172.21.15.3:0/2159261572 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a28011990 con 0x7f6a3809e1d0 2023-10-03T10:17:42.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.093+0000 7f6a3cb64700 1 -- 172.21.15.3:0/2159261572 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a28011d20 con 0x7f6a3809e1d0 2023-10-03T10:17:42.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.093+0000 7f6a3fdca700 1 -- 172.21.15.3:0/2159261572 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6a3809e1d0 msgr2=0x7f6a38135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:42.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.093+0000 7f6a3fdca700 1 --2- 172.21.15.3:0/2159261572 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6a3809e1d0 0x7f6a38135910 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6a28009870 tx=0x7f6a28009b80 comp rx=0 tx=0).stop 2023-10-03T10:17:42.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.093+0000 7f6a3fdca700 1 -- 172.21.15.3:0/2159261572 shutdown_connections 2023-10-03T10:17:42.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.093+0000 7f6a3fdca700 1 --2- 172.21.15.3:0/2159261572 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a38135e50 0x7f6a38138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.093+0000 7f6a3fdca700 1 --2- 172.21.15.3:0/2159261572 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6a3809e1d0 0x7f6a38135910 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.093+0000 7f6a3fdca700 1 -- 172.21.15.3:0/2159261572 >> 172.21.15.3:0/2159261572 conn(0x7f6a38066f50 msgr2=0x7f6a3809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:42.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.093+0000 7f6a3fdca700 1 -- 172.21.15.3:0/2159261572 shutdown_connections 2023-10-03T10:17:42.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.094+0000 7f6a3fdca700 1 -- 172.21.15.3:0/2159261572 wait complete. 2023-10-03T10:17:42.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.094+0000 7f6a3fdca700 1 Processor -- start 2023-10-03T10:17:42.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.095+0000 7f6a3fdca700 1 -- start start 2023-10-03T10:17:42.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.095+0000 7f6a3fdca700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a3809d6f0 0x7f6a381c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:42.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.095+0000 7f6a3fdca700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a3809e1d0 0x7f6a381c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:42.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.095+0000 7f6a3fdca700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6a38135e50 0x7f6a381c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:42.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.095+0000 7f6a3fdca700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a38121ff0 con 0x7f6a3809e1d0 2023-10-03T10:17:42.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.095+0000 7f6a3fdca700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a381221d0 con 0x7f6a3809d6f0 2023-10-03T10:17:42.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.095+0000 7f6a3fdca700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3811b350 con 0x7f6a38135e50 2023-10-03T10:17:42.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.095+0000 7f6a3db66700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a3809d6f0 0x7f6a381c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:42.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.095+0000 7f6a3db66700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a3809d6f0 0x7f6a381c8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39046/0 (socket says 172.21.15.3:39046) 2023-10-03T10:17:42.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.095+0000 7f6a3db66700 1 -- 172.21.15.3:0/3275291985 learned_addr learned my addr 172.21.15.3:0/3275291985 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:42.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.095+0000 7f6a3d365700 1 --2- 172.21.15.3:0/3275291985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a3809e1d0 0x7f6a381c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:42.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.095+0000 7f6a3e367700 1 --2- 172.21.15.3:0/3275291985 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6a38135e50 0x7f6a381c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:42.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.096+0000 7f6a3db66700 1 -- 172.21.15.3:0/3275291985 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6a38135e50 msgr2=0x7f6a381c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:42.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.096+0000 7f6a3db66700 1 --2- 172.21.15.3:0/3275291985 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6a38135e50 0x7f6a381c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.096+0000 7f6a3db66700 1 -- 172.21.15.3:0/3275291985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a3809e1d0 msgr2=0x7f6a381c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:42.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.096+0000 7f6a3db66700 1 --2- 172.21.15.3:0/3275291985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a3809e1d0 0x7f6a381c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.096+0000 7f6a3db66700 1 -- 172.21.15.3:0/3275291985 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a30009a70 con 0x7f6a3809d6f0 2023-10-03T10:17:42.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.096+0000 7f6a3d365700 1 --2- 172.21.15.3:0/3275291985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a3809e1d0 0x7f6a381c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:42.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.096+0000 7f6a3db66700 1 --2- 172.21.15.3:0/3275291985 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a3809d6f0 0x7f6a381c8170 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6a3400c580 tx=0x7f6a3400c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:42.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.096+0000 7f6a2e7fc700 1 -- 172.21.15.3:0/3275291985 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a34004c80 con 0x7f6a3809d6f0 2023-10-03T10:17:42.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.096+0000 7f6a2e7fc700 1 -- 172.21.15.3:0/3275291985 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a34004e00 con 0x7f6a3809d6f0 2023-10-03T10:17:42.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.096+0000 7f6a3fdca700 1 -- 172.21.15.3:0/3275291985 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a280094a0 con 0x7f6a3809d6f0 2023-10-03T10:17:42.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.096+0000 7f6a2e7fc700 1 -- 172.21.15.3:0/3275291985 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a34007310 con 0x7f6a3809d6f0 2023-10-03T10:17:42.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.096+0000 7f6a3fdca700 1 -- 172.21.15.3:0/3275291985 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a3809c700 con 0x7f6a3809d6f0 2023-10-03T10:17:42.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.097+0000 7f6a2e7fc700 1 -- 172.21.15.3:0/3275291985 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6a34007470 con 0x7f6a3809d6f0 2023-10-03T10:17:42.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.097+0000 7f6a2e7fc700 1 -- 172.21.15.3:0/3275291985 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6a3403f310 con 0x7f6a3809d6f0 2023-10-03T10:17:42.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.097+0000 7f6a3fdca700 1 -- 172.21.15.3:0/3275291985 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a38074ea0 con 0x7f6a3809d6f0 2023-10-03T10:17:42.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.102+0000 7f6a2e7fc700 1 -- 172.21.15.3:0/3275291985 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6a340123b0 con 0x7f6a3809d6f0 2023-10-03T10:17:42.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.307+0000 7f6a3fdca700 1 -- 172.21.15.3:0/3275291985 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6a3806b280 con 0x7f6a3809d6f0 2023-10-03T10:17:42.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.308+0000 7f6a2e7fc700 1 -- 172.21.15.3:0/3275291985 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6a34011360 con 0x7f6a3809d6f0 2023-10-03T10:17:42.308 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:42.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.310+0000 7f6a3fdca700 1 -- 172.21.15.3:0/3275291985 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a3809d6f0 msgr2=0x7f6a381c8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:42.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.310+0000 7f6a3fdca700 1 --2- 172.21.15.3:0/3275291985 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a3809d6f0 0x7f6a381c8170 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6a3400c580 tx=0x7f6a3400c890 comp rx=0 tx=0).stop 2023-10-03T10:17:42.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.310+0000 7f6a3fdca700 1 -- 172.21.15.3:0/3275291985 shutdown_connections 2023-10-03T10:17:42.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.310+0000 7f6a3fdca700 1 --2- 172.21.15.3:0/3275291985 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6a38135e50 0x7f6a381c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.310+0000 7f6a3fdca700 1 --2- 172.21.15.3:0/3275291985 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6a3809d6f0 0x7f6a381c8170 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.310+0000 7f6a3fdca700 1 --2- 172.21.15.3:0/3275291985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a3809e1d0 0x7f6a381c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.310+0000 7f6a3fdca700 1 -- 172.21.15.3:0/3275291985 >> 172.21.15.3:0/3275291985 conn(0x7f6a38066f50 msgr2=0x7f6a38134e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:42.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.310+0000 7f6a3fdca700 1 -- 172.21.15.3:0/3275291985 shutdown_connections 2023-10-03T10:17:42.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.310+0000 7f6a3fdca700 1 -- 172.21.15.3:0/3275291985 wait complete. 2023-10-03T10:17:42.324 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:42.578 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:42.605 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:42.656 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:42.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.729+0000 7f4406e77700 1 Processor -- start 2023-10-03T10:17:42.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.730+0000 7f4406e77700 1 -- start start 2023-10-03T10:17:42.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.730+0000 7f4406e77700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f440009e930 0x7f440009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:42.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.730+0000 7f4406e77700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f440009f240 0x7f44001317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:42.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.730+0000 7f4406e77700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f440009f650 0x7f4400131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:42.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.730+0000 7f4406e77700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4400070940 con 0x7f440009f240 2023-10-03T10:17:42.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.730+0000 7f4406e77700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440006fd60 con 0x7f440009e930 2023-10-03T10:17:42.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.730+0000 7f4406e77700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f440006fa80 con 0x7f440009f650 2023-10-03T10:17:42.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.730+0000 7f4404c13700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f440009e930 0x7f440009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:42.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.730+0000 7f4404c13700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f440009e930 0x7f440009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39052/0 (socket says 172.21.15.3:39052) 2023-10-03T10:17:42.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.730+0000 7f4404c13700 1 -- 172.21.15.3:0/3678032233 learned_addr learned my addr 172.21.15.3:0/3678032233 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:42.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.730+0000 7f43fffff700 1 --2- 172.21.15.3:0/3678032233 >> v2:172.21.15.3:3300/0 conn(0x7f440009f240 0x7f44001317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:42.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.730+0000 7f4405414700 1 --2- 172.21.15.3:0/3678032233 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f440009f650 0x7f4400131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:42.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.731+0000 7f4404c13700 1 -- 172.21.15.3:0/3678032233 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f440009f650 msgr2=0x7f4400131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:42.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.731+0000 7f4404c13700 1 --2- 172.21.15.3:0/3678032233 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f440009f650 0x7f4400131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.731+0000 7f4404c13700 1 -- 172.21.15.3:0/3678032233 >> v2:172.21.15.3:3300/0 conn(0x7f440009f240 msgr2=0x7f44001317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:42.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.731+0000 7f4404c13700 1 --2- 172.21.15.3:0/3678032233 >> v2:172.21.15.3:3300/0 conn(0x7f440009f240 0x7f44001317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.731+0000 7f4404c13700 1 -- 172.21.15.3:0/3678032233 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440009fb60 con 0x7f440009e930 2023-10-03T10:17:42.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.731+0000 7f43fffff700 1 --2- 172.21.15.3:0/3678032233 >> v2:172.21.15.3:3300/0 conn(0x7f440009f240 0x7f44001317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:42.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.731+0000 7f4404c13700 1 --2- 172.21.15.3:0/3678032233 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f440009e930 0x7f440009ed00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f43f40098f0 tx=0x7f43f4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d1ca2ba718dcf1b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:42.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.731+0000 7f43ff7fe700 1 -- 172.21.15.3:0/3678032233 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f4011830 con 0x7f440009e930 2023-10-03T10:17:42.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.731+0000 7f43ff7fe700 1 -- 172.21.15.3:0/3678032233 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f4011990 con 0x7f440009e930 2023-10-03T10:17:42.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.731+0000 7f43ff7fe700 1 -- 172.21.15.3:0/3678032233 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f4011d20 con 0x7f440009e930 2023-10-03T10:17:42.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.732+0000 7f4406e77700 1 -- 172.21.15.3:0/3678032233 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f440009e930 msgr2=0x7f440009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:42.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.732+0000 7f4406e77700 1 --2- 172.21.15.3:0/3678032233 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f440009e930 0x7f440009ed00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f43f40098f0 tx=0x7f43f4009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:42.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.732+0000 7f4406e77700 1 -- 172.21.15.3:0/3678032233 shutdown_connections 2023-10-03T10:17:42.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.732+0000 7f4406e77700 1 --2- 172.21.15.3:0/3678032233 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f440009f650 0x7f4400131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.732+0000 7f4406e77700 1 --2- 172.21.15.3:0/3678032233 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f440009e930 0x7f440009ed00 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.732+0000 7f4406e77700 1 --2- 172.21.15.3:0/3678032233 >> v2:172.21.15.3:3300/0 conn(0x7f440009f240 0x7f44001317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.732+0000 7f4406e77700 1 -- 172.21.15.3:0/3678032233 >> 172.21.15.3:0/3678032233 conn(0x7f44000a4210 msgr2=0x7f44000a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:42.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.732+0000 7f4406e77700 1 -- 172.21.15.3:0/3678032233 shutdown_connections 2023-10-03T10:17:42.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.732+0000 7f4406e77700 1 -- 172.21.15.3:0/3678032233 wait complete. 2023-10-03T10:17:42.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.733+0000 7f4406e77700 1 Processor -- start 2023-10-03T10:17:42.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.733+0000 7f4406e77700 1 -- start start 2023-10-03T10:17:42.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.734+0000 7f4406e77700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f440009e930 0x7f44001a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:42.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.734+0000 7f4406e77700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f440009f240 0x7f44001a35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:42.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.734+0000 7f4406e77700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f440009f650 0x7f44001c3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:42.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.734+0000 7f4406e77700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44001a44c0 con 0x7f440009f240 2023-10-03T10:17:42.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.734+0000 7f4406e77700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44001a42d0 con 0x7f440009f650 2023-10-03T10:17:42.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.734+0000 7f4406e77700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f44001b7050 con 0x7f440009e930 2023-10-03T10:17:42.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.734+0000 7f4404c13700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f440009e930 0x7f44001a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:42.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.734+0000 7f4405414700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f440009f650 0x7f44001c3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:42.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.734+0000 7f4404c13700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f440009e930 0x7f44001a3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54180/0 (socket says 172.21.15.3:54180) 2023-10-03T10:17:42.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.734+0000 7f4404c13700 1 -- 172.21.15.3:0/2513512073 learned_addr learned my addr 172.21.15.3:0/2513512073 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:42.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.734+0000 7f43fffff700 1 --2- 172.21.15.3:0/2513512073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f440009f240 0x7f44001a35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:42.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.734+0000 7f4404c13700 1 -- 172.21.15.3:0/2513512073 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f440009f650 msgr2=0x7f44001c3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:42.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.734+0000 7f4404c13700 1 --2- 172.21.15.3:0/2513512073 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f440009f650 0x7f44001c3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.734+0000 7f4404c13700 1 -- 172.21.15.3:0/2513512073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f440009f240 msgr2=0x7f44001a35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:42.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.734+0000 7f4404c13700 1 --2- 172.21.15.3:0/2513512073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f440009f240 0x7f44001a35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.734+0000 7f4404c13700 1 -- 172.21.15.3:0/2513512073 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ec0094a0 con 0x7f440009e930 2023-10-03T10:17:42.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.735+0000 7f4404c13700 1 --2- 172.21.15.3:0/2513512073 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f440009e930 0x7f44001a3060 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f43f4000c00 tx=0x7f43f4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:42.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.735+0000 7f43fd7fa700 1 -- 172.21.15.3:0/2513512073 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f4004300 con 0x7f440009e930 2023-10-03T10:17:42.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.735+0000 7f43fd7fa700 1 -- 172.21.15.3:0/2513512073 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f401f070 con 0x7f440009e930 2023-10-03T10:17:42.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.735+0000 7f4406e77700 1 -- 172.21.15.3:0/2513512073 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f4009550 con 0x7f440009e930 2023-10-03T10:17:42.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.735+0000 7f43fd7fa700 1 -- 172.21.15.3:0/2513512073 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f40036a0 con 0x7f440009e930 2023-10-03T10:17:42.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.735+0000 7f4406e77700 1 -- 172.21.15.3:0/2513512073 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44000a0190 con 0x7f440009e930 2023-10-03T10:17:42.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.736+0000 7f43fd7fa700 1 -- 172.21.15.3:0/2513512073 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f43f4021030 con 0x7f440009e930 2023-10-03T10:17:42.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.736+0000 7f43fd7fa700 1 -- 172.21.15.3:0/2513512073 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f43f4040800 con 0x7f440009e930 2023-10-03T10:17:42.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.737+0000 7f4406e77700 1 -- 172.21.15.3:0/2513512073 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4400074ea0 con 0x7f440009e930 2023-10-03T10:17:42.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.742+0000 7f43fd7fa700 1 -- 172.21.15.3:0/2513512073 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f43f4015080 con 0x7f440009e930 2023-10-03T10:17:42.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.780+0000 7f80517e5700 1 Processor -- start 2023-10-03T10:17:42.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.780+0000 7f80517e5700 1 -- start start 2023-10-03T10:17:42.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.781+0000 7f80517e5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f804c0983f0 0x7f804c0987c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:42.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.781+0000 7f80517e5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f804c09d6f0 0x7f804c09dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:42.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.781+0000 7f80517e5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f804c09e020 0x7f804c1385c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:42.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.781+0000 7f80517e5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f804c099c50 con 0x7f804c0983f0 2023-10-03T10:17:42.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.781+0000 7f80517e5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804c09c1a0 con 0x7f804c09e020 2023-10-03T10:17:42.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.781+0000 7f80517e5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f804c09bcc0 con 0x7f804c09d6f0 2023-10-03T10:17:42.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.781+0000 7f80427fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f804c09d6f0 0x7f804c09dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:42.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.781+0000 7f80427fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f804c09d6f0 0x7f804c09dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54194/0 (socket says 172.21.15.3:54194) 2023-10-03T10:17:42.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.781+0000 7f80427fc700 1 -- 172.21.15.3:0/1336555026 learned_addr learned my addr 172.21.15.3:0/1336555026 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:42.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.781+0000 7f804b7fe700 1 --2- 172.21.15.3:0/1336555026 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f804c09e020 0x7f804c1385c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:42.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.781+0000 7f804affd700 1 --2- 172.21.15.3:0/1336555026 >> v2:172.21.15.3:3300/0 conn(0x7f804c0983f0 0x7f804c0987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:42.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.781+0000 7f80427fc700 1 -- 172.21.15.3:0/1336555026 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f804c09e020 msgr2=0x7f804c1385c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:42.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.781+0000 7f80427fc700 1 --2- 172.21.15.3:0/1336555026 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f804c09e020 0x7f804c1385c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.781+0000 7f80427fc700 1 -- 172.21.15.3:0/1336555026 >> v2:172.21.15.3:3300/0 conn(0x7f804c0983f0 msgr2=0x7f804c0987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:42.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.781+0000 7f80427fc700 1 --2- 172.21.15.3:0/1336555026 >> v2:172.21.15.3:3300/0 conn(0x7f804c0983f0 0x7f804c0987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.781+0000 7f80427fc700 1 -- 172.21.15.3:0/1336555026 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804c09a9c0 con 0x7f804c09d6f0 2023-10-03T10:17:42.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.782+0000 7f80427fc700 1 --2- 172.21.15.3:0/1336555026 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f804c09d6f0 0x7f804c09dae0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f80340098f0 tx=0x7f8034009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebe6f43bdfc11175 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:42.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.782+0000 7f804a7fc700 1 -- 172.21.15.3:0/1336555026 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034010a90 con 0x7f804c09d6f0 2023-10-03T10:17:42.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.782+0000 7f804a7fc700 1 -- 172.21.15.3:0/1336555026 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8034010bf0 con 0x7f804c09d6f0 2023-10-03T10:17:42.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.782+0000 7f804a7fc700 1 -- 172.21.15.3:0/1336555026 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034018440 con 0x7f804c09d6f0 2023-10-03T10:17:42.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.783+0000 7f80517e5700 1 -- 172.21.15.3:0/1336555026 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f804c09d6f0 msgr2=0x7f804c09dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:42.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.783+0000 7f80517e5700 1 --2- 172.21.15.3:0/1336555026 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f804c09d6f0 0x7f804c09dae0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f80340098f0 tx=0x7f8034009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:42.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.783+0000 7f80517e5700 1 -- 172.21.15.3:0/1336555026 shutdown_connections 2023-10-03T10:17:42.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.783+0000 7f80517e5700 1 --2- 172.21.15.3:0/1336555026 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f804c09e020 0x7f804c1385c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.783+0000 7f80517e5700 1 --2- 172.21.15.3:0/1336555026 >> v2:172.21.15.3:3300/0 conn(0x7f804c0983f0 0x7f804c0987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.783+0000 7f80517e5700 1 --2- 172.21.15.3:0/1336555026 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f804c09d6f0 0x7f804c09dae0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.783+0000 7f80517e5700 1 -- 172.21.15.3:0/1336555026 >> 172.21.15.3:0/1336555026 conn(0x7f804c09f470 msgr2=0x7f804c0a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:42.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.783+0000 7f80517e5700 1 -- 172.21.15.3:0/1336555026 shutdown_connections 2023-10-03T10:17:42.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.783+0000 7f80517e5700 1 -- 172.21.15.3:0/1336555026 wait complete. 2023-10-03T10:17:42.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.784+0000 7f80517e5700 1 Processor -- start 2023-10-03T10:17:42.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.784+0000 7f80517e5700 1 -- start start 2023-10-03T10:17:42.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.785+0000 7f80517e5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f804c0983f0 0x7f804c12d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:42.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.785+0000 7f80517e5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f804c09d6f0 0x7f804c12d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:42.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.785+0000 7f80517e5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f804c09e020 0x7f804c12ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:42.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.785+0000 7f80517e5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804c12c250 con 0x7f804c09e020 2023-10-03T10:17:42.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.785+0000 7f80517e5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804c0a1450 con 0x7f804c09d6f0 2023-10-03T10:17:42.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.785+0000 7f80517e5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f804c12e3f0 con 0x7f804c0983f0 2023-10-03T10:17:42.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.785+0000 7f804affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f804c0983f0 0x7f804c12d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:42.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.785+0000 7f804affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f804c0983f0 0x7f804c12d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54206/0 (socket says 172.21.15.3:54206) 2023-10-03T10:17:42.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.785+0000 7f804affd700 1 -- 172.21.15.3:0/1065297472 learned_addr learned my addr 172.21.15.3:0/1065297472 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:42.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.785+0000 7f804b7fe700 1 --2- 172.21.15.3:0/1065297472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f804c09e020 0x7f804c12ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:42.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.785+0000 7f804affd700 1 -- 172.21.15.3:0/1065297472 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f804c09d6f0 msgr2=0x7f804c12d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:42.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.785+0000 7f804affd700 1 --2- 172.21.15.3:0/1065297472 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f804c09d6f0 0x7f804c12d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.785+0000 7f804affd700 1 -- 172.21.15.3:0/1065297472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f804c09e020 msgr2=0x7f804c12ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:42.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.785+0000 7f804affd700 1 --2- 172.21.15.3:0/1065297472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f804c09e020 0x7f804c12ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.785+0000 7f804affd700 1 -- 172.21.15.3:0/1065297472 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8034009550 con 0x7f804c0983f0 2023-10-03T10:17:42.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.785+0000 7f804affd700 1 --2- 172.21.15.3:0/1065297472 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f804c0983f0 0x7f804c12d340 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f803c00e810 tx=0x7f803c00eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:42.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.786+0000 7f8043fff700 1 -- 172.21.15.3:0/1065297472 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c013610 con 0x7f804c0983f0 2023-10-03T10:17:42.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.786+0000 7f80517e5700 1 -- 172.21.15.3:0/1065297472 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f804c12ae00 con 0x7f804c0983f0 2023-10-03T10:17:42.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.786+0000 7f8043fff700 1 -- 172.21.15.3:0/1065297472 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803c013770 con 0x7f804c0983f0 2023-10-03T10:17:42.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.786+0000 7f8043fff700 1 -- 172.21.15.3:0/1065297472 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c013b20 con 0x7f804c0983f0 2023-10-03T10:17:42.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.786+0000 7f80517e5700 1 -- 172.21.15.3:0/1065297472 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f804c09a2e0 con 0x7f804c0983f0 2023-10-03T10:17:42.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.787+0000 7f80517e5700 1 -- 172.21.15.3:0/1065297472 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f804c0747f0 con 0x7f804c0983f0 2023-10-03T10:17:42.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.787+0000 7f8043fff700 1 -- 172.21.15.3:0/1065297472 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f803c013c80 con 0x7f804c0983f0 2023-10-03T10:17:42.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.787+0000 7f8043fff700 1 -- 172.21.15.3:0/1065297472 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f803c03f450 con 0x7f804c0983f0 2023-10-03T10:17:42.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.797+0000 7f8043fff700 1 -- 172.21.15.3:0/1065297472 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f803c012370 con 0x7f804c0983f0 2023-10-03T10:17:42.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.811+0000 7f917290a700 1 Processor -- start 2023-10-03T10:17:42.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.811+0000 7f917290a700 1 -- start start 2023-10-03T10:17:42.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.812+0000 7f917290a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f916c09e930 0x7f916c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:42.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.812+0000 7f917290a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f916c09f240 0x7f916c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:42.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.812+0000 7f917290a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f916c09f650 0x7f916c131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:42.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.812+0000 7f917290a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f916c070940 con 0x7f916c09e930 2023-10-03T10:17:42.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.812+0000 7f917290a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c06fd60 con 0x7f916c09f240 2023-10-03T10:17:42.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.812+0000 7f917290a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f916c06fa80 con 0x7f916c09f650 2023-10-03T10:17:42.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.812+0000 7f9170ea7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f916c09f650 0x7f916c131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:42.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.812+0000 7f9170ea7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f916c09f650 0x7f916c131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54216/0 (socket says 172.21.15.3:54216) 2023-10-03T10:17:42.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.812+0000 7f9170ea7700 1 -- 172.21.15.3:0/239449245 learned_addr learned my addr 172.21.15.3:0/239449245 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:42.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.812+0000 7f916b7fe700 1 --2- 172.21.15.3:0/239449245 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f916c09f240 0x7f916c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:42.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.812+0000 7f916bfff700 1 --2- 172.21.15.3:0/239449245 >> v2:172.21.15.3:3300/0 conn(0x7f916c09e930 0x7f916c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:42.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.813+0000 7f9170ea7700 1 -- 172.21.15.3:0/239449245 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f916c09f240 msgr2=0x7f916c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:42.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.813+0000 7f9170ea7700 1 --2- 172.21.15.3:0/239449245 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f916c09f240 0x7f916c1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.813+0000 7f9170ea7700 1 -- 172.21.15.3:0/239449245 >> v2:172.21.15.3:3300/0 conn(0x7f916c09e930 msgr2=0x7f916c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:42.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.813+0000 7f9170ea7700 1 --2- 172.21.15.3:0/239449245 >> v2:172.21.15.3:3300/0 conn(0x7f916c09e930 0x7f916c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.813+0000 7f9170ea7700 1 -- 172.21.15.3:0/239449245 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c09fb60 con 0x7f916c09f650 2023-10-03T10:17:42.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.813+0000 7f9170ea7700 1 --2- 172.21.15.3:0/239449245 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f916c09f650 0x7f916c131d30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9164009e50 tx=0x7f916400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54a2984faaf5686 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:42.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.813+0000 7f916affd700 1 -- 172.21.15.3:0/239449245 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916400e070 con 0x7f916c09f650 2023-10-03T10:17:42.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.813+0000 7f916affd700 1 -- 172.21.15.3:0/239449245 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916400b910 con 0x7f916c09f650 2023-10-03T10:17:42.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.813+0000 7f916affd700 1 -- 172.21.15.3:0/239449245 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916400bcc0 con 0x7f916c09f650 2023-10-03T10:17:42.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.814+0000 7f917290a700 1 -- 172.21.15.3:0/239449245 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f916c09f650 msgr2=0x7f916c131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:42.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.814+0000 7f917290a700 1 --2- 172.21.15.3:0/239449245 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f916c09f650 0x7f916c131d30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9164009e50 tx=0x7f916400b330 comp rx=0 tx=0).stop 2023-10-03T10:17:42.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.814+0000 7f917290a700 1 -- 172.21.15.3:0/239449245 shutdown_connections 2023-10-03T10:17:42.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.814+0000 7f917290a700 1 --2- 172.21.15.3:0/239449245 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f916c09f650 0x7f916c131d30 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.814+0000 7f917290a700 1 --2- 172.21.15.3:0/239449245 >> v2:172.21.15.3:3300/0 conn(0x7f916c09e930 0x7f916c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.814+0000 7f917290a700 1 --2- 172.21.15.3:0/239449245 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f916c09f240 0x7f916c1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.814+0000 7f917290a700 1 -- 172.21.15.3:0/239449245 >> 172.21.15.3:0/239449245 conn(0x7f916c0a4210 msgr2=0x7f916c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:42.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.815+0000 7f917290a700 1 -- 172.21.15.3:0/239449245 shutdown_connections 2023-10-03T10:17:42.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.815+0000 7f917290a700 1 -- 172.21.15.3:0/239449245 wait complete. 2023-10-03T10:17:42.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.816+0000 7f917290a700 1 Processor -- start 2023-10-03T10:17:42.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.816+0000 7f917290a700 1 -- start start 2023-10-03T10:17:42.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.816+0000 7f917290a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f916c09e930 0x7f916c1a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:42.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.816+0000 7f917290a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f916c09f240 0x7f916c1a4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:42.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.816+0000 7f917290a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f916c09f650 0x7f916c1a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:42.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.816+0000 7f917290a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c1a6f70 con 0x7f916c09e930 2023-10-03T10:17:42.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.816+0000 7f917290a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c1a5890 con 0x7f916c09f240 2023-10-03T10:17:42.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.816+0000 7f917290a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f916c1a7420 con 0x7f916c09f650 2023-10-03T10:17:42.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.816+0000 7f916b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f916c09f240 0x7f916c1a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:42.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.816+0000 7f916b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f916c09f240 0x7f916c1a4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39118/0 (socket says 172.21.15.3:39118) 2023-10-03T10:17:42.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.816+0000 7f916b7fe700 1 -- 172.21.15.3:0/3433075559 learned_addr learned my addr 172.21.15.3:0/3433075559 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:42.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.817+0000 7f9170ea7700 1 --2- 172.21.15.3:0/3433075559 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f916c09f650 0x7f916c1a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:42.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.817+0000 7f9170ea7700 1 -- 172.21.15.3:0/3433075559 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f916c09f240 msgr2=0x7f916c1a4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:42.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.817+0000 7f9170ea7700 1 --2- 172.21.15.3:0/3433075559 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f916c09f240 0x7f916c1a4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.817+0000 7f9170ea7700 1 -- 172.21.15.3:0/3433075559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f916c09e930 msgr2=0x7f916c1a3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:42.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.817+0000 7f9170ea7700 1 --2- 172.21.15.3:0/3433075559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f916c09e930 0x7f916c1a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.817+0000 7f9170ea7700 1 -- 172.21.15.3:0/3433075559 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9164009b30 con 0x7f916c09f650 2023-10-03T10:17:42.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.817+0000 7f916b7fe700 1 --2- 172.21.15.3:0/3433075559 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f916c09f240 0x7f916c1a4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:42.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.817+0000 7f9170ea7700 1 --2- 172.21.15.3:0/3433075559 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f916c09f650 0x7f916c1a4660 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f916c06f570 tx=0x7f9164004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:42.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.817+0000 7f9168ff9700 1 -- 172.21.15.3:0/3433075559 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9164004450 con 0x7f916c09f650 2023-10-03T10:17:42.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.817+0000 7f9168ff9700 1 -- 172.21.15.3:0/3433075559 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91640045d0 con 0x7f916c09f650 2023-10-03T10:17:42.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.817+0000 7f9168ff9700 1 -- 172.21.15.3:0/3433075559 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916400ce80 con 0x7f916c09f650 2023-10-03T10:17:42.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.817+0000 7f917290a700 1 -- 172.21.15.3:0/3433075559 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f916c0a0190 con 0x7f916c09f650 2023-10-03T10:17:42.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.818+0000 7f917290a700 1 -- 172.21.15.3:0/3433075559 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916c071d20 con 0x7f916c09f650 2023-10-03T10:17:42.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.819+0000 7f9168ff9700 1 -- 172.21.15.3:0/3433075559 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f916401c410 con 0x7f916c09f650 2023-10-03T10:17:42.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.819+0000 7f917290a700 1 -- 172.21.15.3:0/3433075559 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f916c1a0b30 con 0x7f916c09f650 2023-10-03T10:17:42.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.819+0000 7f9168ff9700 1 -- 172.21.15.3:0/3433075559 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f91640410f0 con 0x7f916c09f650 2023-10-03T10:17:42.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.824+0000 7f9168ff9700 1 -- 172.21.15.3:0/3433075559 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9164011570 con 0x7f916c09f650 2023-10-03T10:17:42.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.972+0000 7f4406e77700 1 -- 172.21.15.3:0/2513512073 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f44001a4ac0 con 0x7f440009e930 2023-10-03T10:17:42.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.972+0000 7f43fd7fa700 1 -- 172.21.15.3:0/2513512073 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f43f40199a0 con 0x7f440009e930 2023-10-03T10:17:42.972 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:42.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.974+0000 7f4406e77700 1 -- 172.21.15.3:0/2513512073 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f440009e930 msgr2=0x7f44001a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:42.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.974+0000 7f4406e77700 1 --2- 172.21.15.3:0/2513512073 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f440009e930 0x7f44001a3060 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f43f4000c00 tx=0x7f43f4004750 comp rx=0 tx=0).stop 2023-10-03T10:17:42.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.974+0000 7f4406e77700 1 -- 172.21.15.3:0/2513512073 shutdown_connections 2023-10-03T10:17:42.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.974+0000 7f4406e77700 1 --2- 172.21.15.3:0/2513512073 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f440009f650 0x7f44001c3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.974+0000 7f4406e77700 1 --2- 172.21.15.3:0/2513512073 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f440009e930 0x7f44001a3060 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.974+0000 7f4406e77700 1 --2- 172.21.15.3:0/2513512073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f440009f240 0x7f44001a35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:42.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.975+0000 7f4406e77700 1 -- 172.21.15.3:0/2513512073 >> 172.21.15.3:0/2513512073 conn(0x7f44000a4210 msgr2=0x7f44001b7790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:42.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.975+0000 7f4406e77700 1 -- 172.21.15.3:0/2513512073 shutdown_connections 2023-10-03T10:17:42.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:42.975+0000 7f4406e77700 1 -- 172.21.15.3:0/2513512073 wait complete. 2023-10-03T10:17:42.988 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.004+0000 7f80517e5700 1 -- 172.21.15.3:0/1065297472 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f804c12fa00 con 0x7f804c0983f0 2023-10-03T10:17:43.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.005+0000 7f8043fff700 1 -- 172.21.15.3:0/1065297472 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f803c021070 con 0x7f804c0983f0 2023-10-03T10:17:43.005 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:43.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.007+0000 7f80517e5700 1 -- 172.21.15.3:0/1065297472 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f804c0983f0 msgr2=0x7f804c12d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:43.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.007+0000 7f80517e5700 1 --2- 172.21.15.3:0/1065297472 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f804c0983f0 0x7f804c12d340 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f803c00e810 tx=0x7f803c00eb20 comp rx=0 tx=0).stop 2023-10-03T10:17:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.007+0000 7f80517e5700 1 -- 172.21.15.3:0/1065297472 shutdown_connections 2023-10-03T10:17:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.007+0000 7f80517e5700 1 --2- 172.21.15.3:0/1065297472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f804c09e020 0x7f804c12ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.007+0000 7f80517e5700 1 --2- 172.21.15.3:0/1065297472 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f804c0983f0 0x7f804c12d340 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.007+0000 7f80517e5700 1 --2- 172.21.15.3:0/1065297472 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f804c09d6f0 0x7f804c12d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.007+0000 7f80517e5700 1 -- 172.21.15.3:0/1065297472 >> 172.21.15.3:0/1065297472 conn(0x7f804c09f470 msgr2=0x7f804c1305c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.007+0000 7f80517e5700 1 -- 172.21.15.3:0/1065297472 shutdown_connections 2023-10-03T10:17:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.007+0000 7f80517e5700 1 -- 172.21.15.3:0/1065297472 wait complete. 2023-10-03T10:17:43.021 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:43.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.033+0000 7f917290a700 1 -- 172.21.15.3:0/3433075559 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f916c1a6aa0 con 0x7f916c09f650 2023-10-03T10:17:43.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.033+0000 7f9168ff9700 1 -- 172.21.15.3:0/3433075559 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f916402a030 con 0x7f916c09f650 2023-10-03T10:17:43.034 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:43.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.036+0000 7f917290a700 1 -- 172.21.15.3:0/3433075559 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f916c09f650 msgr2=0x7f916c1a4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:43.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.036+0000 7f917290a700 1 --2- 172.21.15.3:0/3433075559 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f916c09f650 0x7f916c1a4660 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f916c06f570 tx=0x7f9164004ad0 comp rx=0 tx=0).stop 2023-10-03T10:17:43.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.036+0000 7f917290a700 1 -- 172.21.15.3:0/3433075559 shutdown_connections 2023-10-03T10:17:43.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.036+0000 7f917290a700 1 --2- 172.21.15.3:0/3433075559 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f916c09f650 0x7f916c1a4660 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:43.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.036+0000 7f917290a700 1 --2- 172.21.15.3:0/3433075559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f916c09e930 0x7f916c1a3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:43.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.036+0000 7f917290a700 1 --2- 172.21.15.3:0/3433075559 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f916c09f240 0x7f916c1a4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:43.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.036+0000 7f917290a700 1 -- 172.21.15.3:0/3433075559 >> 172.21.15.3:0/3433075559 conn(0x7f916c0a4210 msgr2=0x7f916c1b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:43.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.036+0000 7f917290a700 1 -- 172.21.15.3:0/3433075559 shutdown_connections 2023-10-03T10:17:43.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.036+0000 7f917290a700 1 -- 172.21.15.3:0/3433075559 wait complete. 2023-10-03T10:17:43.049 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:43.325 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:43.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.472+0000 7fa190873700 1 Processor -- start 2023-10-03T10:17:43.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.472+0000 7fa190873700 1 -- start start 2023-10-03T10:17:43.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.473+0000 7fa190873700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa18c131810 0x7fa18c131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:43.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.473+0000 7fa190873700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa18c099a40 0x7fa18c099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:43.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.473+0000 7fa190873700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa18c09a3b0 0x7fa18c138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:43.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.473+0000 7fa190873700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18c1313c0 con 0x7fa18c099a40 2023-10-03T10:17:43.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.473+0000 7fa190873700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c06e8c0 con 0x7fa18c131810 2023-10-03T10:17:43.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.473+0000 7fa190873700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c09a820 con 0x7fa18c09a3b0 2023-10-03T10:17:43.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.473+0000 7fa18a59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa18c131810 0x7fa18c131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:43.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.473+0000 7fa189d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa18c099a40 0x7fa18c099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:43.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.473+0000 7fa18ad9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa18c09a3b0 0x7fa18c138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:43.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.473+0000 7fa18ad9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa18c09a3b0 0x7fa18c138650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43056/0 (socket says 172.21.15.3:43056) 2023-10-03T10:17:43.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.473+0000 7fa189d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa18c099a40 0x7fa18c099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58860/0 (socket says 172.21.15.3:58860) 2023-10-03T10:17:43.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.473+0000 7fa18a59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa18c131810 0x7fa18c131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60232/0 (socket says 172.21.15.3:60232) 2023-10-03T10:17:43.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.473+0000 7fa189d9b700 1 -- 172.21.15.3:0/492606946 learned_addr learned my addr 172.21.15.3:0/492606946 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:43.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.473+0000 7fa18ad9d700 1 -- 172.21.15.3:0/492606946 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa18c131810 msgr2=0x7fa18c131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:43.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.473+0000 7fa18ad9d700 1 --2- 172.21.15.3:0/492606946 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa18c131810 0x7fa18c131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:43.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.473+0000 7fa18ad9d700 1 -- 172.21.15.3:0/492606946 >> v2:172.21.15.3:3300/0 conn(0x7fa18c099a40 msgr2=0x7fa18c099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:43.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.473+0000 7fa18ad9d700 1 --2- 172.21.15.3:0/492606946 >> v2:172.21.15.3:3300/0 conn(0x7fa18c099a40 0x7fa18c099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:43.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.473+0000 7fa18ad9d700 1 -- 172.21.15.3:0/492606946 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18c071d20 con 0x7fa18c09a3b0 2023-10-03T10:17:43.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.473+0000 7fa189d9b700 1 --2- 172.21.15.3:0/492606946 >> v2:172.21.15.3:3300/0 conn(0x7fa18c099a40 0x7fa18c099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:43.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.474+0000 7fa18ad9d700 1 --2- 172.21.15.3:0/492606946 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa18c09a3b0 0x7fa18c138650 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa184009d90 tx=0x7fa18400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc2e6421a2aa2ca4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:43.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.474+0000 7fa18959a700 1 -- 172.21.15.3:0/492606946 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18400e070 con 0x7fa18c09a3b0 2023-10-03T10:17:43.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.474+0000 7fa18959a700 1 -- 172.21.15.3:0/492606946 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa18400bb20 con 0x7fa18c09a3b0 2023-10-03T10:17:43.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.474+0000 7fa18959a700 1 -- 172.21.15.3:0/492606946 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa184012430 con 0x7fa18c09a3b0 2023-10-03T10:17:43.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.474+0000 7fa190873700 1 -- 172.21.15.3:0/492606946 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa18c09a3b0 msgr2=0x7fa18c138650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:43.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.474+0000 7fa190873700 1 --2- 172.21.15.3:0/492606946 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa18c09a3b0 0x7fa18c138650 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa184009d90 tx=0x7fa18400b330 comp rx=0 tx=0).stop 2023-10-03T10:17:43.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.474+0000 7fa190873700 1 -- 172.21.15.3:0/492606946 shutdown_connections 2023-10-03T10:17:43.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.474+0000 7fa190873700 1 --2- 172.21.15.3:0/492606946 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa18c09a3b0 0x7fa18c138650 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:43.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.474+0000 7fa190873700 1 --2- 172.21.15.3:0/492606946 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa18c131810 0x7fa18c131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:43.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.474+0000 7fa190873700 1 --2- 172.21.15.3:0/492606946 >> v2:172.21.15.3:3300/0 conn(0x7fa18c099a40 0x7fa18c099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:43.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.475+0000 7fa190873700 1 -- 172.21.15.3:0/492606946 >> 172.21.15.3:0/492606946 conn(0x7fa18c0a2aa0 msgr2=0x7fa18c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:43.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.475+0000 7fa190873700 1 -- 172.21.15.3:0/492606946 shutdown_connections 2023-10-03T10:17:43.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.475+0000 7fa190873700 1 -- 172.21.15.3:0/492606946 wait complete. 2023-10-03T10:17:43.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.476+0000 7fa190873700 1 Processor -- start 2023-10-03T10:17:43.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.476+0000 7fa190873700 1 -- start start 2023-10-03T10:17:43.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.476+0000 7fa190873700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa18c099a40 0x7fa18c06f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:43.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.476+0000 7fa190873700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa18c09a3b0 0x7fa18c06fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:43.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.476+0000 7fa190873700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa18c131810 0x7fa18c070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:43.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.476+0000 7fa190873700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c121310 con 0x7fa18c099a40 2023-10-03T10:17:43.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.476+0000 7fa190873700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c1202b0 con 0x7fa18c09a3b0 2023-10-03T10:17:43.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.476+0000 7fa190873700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c0a1b60 con 0x7fa18c131810 2023-10-03T10:17:43.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.476+0000 7fa18a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa18c099a40 0x7fa18c06f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:43.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.476+0000 7fa18a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa18c099a40 0x7fa18c06f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58866/0 (socket says 172.21.15.3:58866) 2023-10-03T10:17:43.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.476+0000 7fa18a59c700 1 -- 172.21.15.3:0/1222705952 learned_addr learned my addr 172.21.15.3:0/1222705952 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:43.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.476+0000 7fa18ad9d700 1 --2- 172.21.15.3:0/1222705952 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa18c131810 0x7fa18c070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:43.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.476+0000 7fa189d9b700 1 --2- 172.21.15.3:0/1222705952 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa18c09a3b0 0x7fa18c06fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:43.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.477+0000 7fa18a59c700 1 -- 172.21.15.3:0/1222705952 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa18c131810 msgr2=0x7fa18c070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:43.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.477+0000 7fa18a59c700 1 --2- 172.21.15.3:0/1222705952 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa18c131810 0x7fa18c070090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:43.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.477+0000 7fa18a59c700 1 -- 172.21.15.3:0/1222705952 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa18c09a3b0 msgr2=0x7fa18c06fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:43.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.477+0000 7fa18a59c700 1 --2- 172.21.15.3:0/1222705952 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa18c09a3b0 0x7fa18c06fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:43.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.477+0000 7fa18a59c700 1 -- 172.21.15.3:0/1222705952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa174009550 con 0x7fa18c099a40 2023-10-03T10:17:43.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.477+0000 7fa18ad9d700 1 --2- 172.21.15.3:0/1222705952 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa18c131810 0x7fa18c070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:43.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.477+0000 7fa189d9b700 1 --2- 172.21.15.3:0/1222705952 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa18c09a3b0 0x7fa18c06fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:43.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.477+0000 7fa18a59c700 1 --2- 172.21.15.3:0/1222705952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa18c099a40 0x7fa18c06f610 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00e9b0 tx=0x7fa17c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:43.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.477+0000 7fa182ffd700 1 -- 172.21.15.3:0/1222705952 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c009710 con 0x7fa18c099a40 2023-10-03T10:17:43.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.477+0000 7fa182ffd700 1 -- 172.21.15.3:0/1222705952 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17c00ee50 con 0x7fa18c099a40 2023-10-03T10:17:43.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.477+0000 7fa190873700 1 -- 172.21.15.3:0/1222705952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa184009a70 con 0x7fa18c099a40 2023-10-03T10:17:43.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.477+0000 7fa182ffd700 1 -- 172.21.15.3:0/1222705952 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c00ca50 con 0x7fa18c099a40 2023-10-03T10:17:43.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.477+0000 7fa190873700 1 -- 172.21.15.3:0/1222705952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa18c120af0 con 0x7fa18c099a40 2023-10-03T10:17:43.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.478+0000 7fa182ffd700 1 -- 172.21.15.3:0/1222705952 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa17c00cbb0 con 0x7fa18c099a40 2023-10-03T10:17:43.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.478+0000 7fa182ffd700 1 -- 172.21.15.3:0/1222705952 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa17c03fa30 con 0x7fa18c099a40 2023-10-03T10:17:43.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.478+0000 7fa190873700 1 -- 172.21.15.3:0/1222705952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa18c074ea0 con 0x7fa18c099a40 2023-10-03T10:17:43.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.484+0000 7fa182ffd700 1 -- 172.21.15.3:0/1222705952 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa17c021020 con 0x7fa18c099a40 2023-10-03T10:17:43.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.687+0000 7fa190873700 1 -- 172.21.15.3:0/1222705952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa18c12ebc0 con 0x7fa18c099a40 2023-10-03T10:17:43.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.687+0000 7fa182ffd700 1 -- 172.21.15.3:0/1222705952 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa17c01b3b0 con 0x7fa18c099a40 2023-10-03T10:17:43.687 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:43.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.689+0000 7fa190873700 1 -- 172.21.15.3:0/1222705952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa18c099a40 msgr2=0x7fa18c06f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:43.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.689+0000 7fa190873700 1 --2- 172.21.15.3:0/1222705952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa18c099a40 0x7fa18c06f610 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00e9b0 tx=0x7fa17c00ecc0 comp rx=0 tx=0).stop 2023-10-03T10:17:43.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.689+0000 7fa190873700 1 -- 172.21.15.3:0/1222705952 shutdown_connections 2023-10-03T10:17:43.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.689+0000 7fa190873700 1 --2- 172.21.15.3:0/1222705952 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa18c131810 0x7fa18c070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:43.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.689+0000 7fa190873700 1 --2- 172.21.15.3:0/1222705952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa18c099a40 0x7fa18c06f610 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:43.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.689+0000 7fa190873700 1 --2- 172.21.15.3:0/1222705952 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa18c09a3b0 0x7fa18c06fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:43.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.689+0000 7fa190873700 1 -- 172.21.15.3:0/1222705952 >> 172.21.15.3:0/1222705952 conn(0x7fa18c0a2aa0 msgr2=0x7fa18c1b8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:43.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.689+0000 7fa190873700 1 -- 172.21.15.3:0/1222705952 shutdown_connections 2023-10-03T10:17:43.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:43.689+0000 7fa190873700 1 -- 172.21.15.3:0/1222705952 wait complete. 2023-10-03T10:17:43.703 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:43.990 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:44.021 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:44.051 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:44.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.146+0000 7f97b88da700 1 Processor -- start 2023-10-03T10:17:44.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.146+0000 7f97b88da700 1 -- start start 2023-10-03T10:17:44.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.147+0000 7f97b88da700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f97b409e930 0x7f97b409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:44.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.147+0000 7f97b88da700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f97b409f240 0x7f97b41317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:44.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.147+0000 7f97b88da700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f97b409f650 0x7f97b4131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:44.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.147+0000 7f97b88da700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b4070940 con 0x7f97b409f650 2023-10-03T10:17:44.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.147+0000 7f97b88da700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b406fd60 con 0x7f97b409f240 2023-10-03T10:17:44.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.147+0000 7f97b88da700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97b406fa80 con 0x7f97b409e930 2023-10-03T10:17:44.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.147+0000 7f97b259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f97b409e930 0x7f97b409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:44.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.147+0000 7f97b1d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f97b409f240 0x7f97b41317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:44.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.147+0000 7f97b2d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f97b409f650 0x7f97b4131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:44.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.147+0000 7f97b259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f97b409e930 0x7f97b409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43076/0 (socket says 172.21.15.3:43076) 2023-10-03T10:17:44.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.147+0000 7f97b1d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f97b409f240 0x7f97b41317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60252/0 (socket says 172.21.15.3:60252) 2023-10-03T10:17:44.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.147+0000 7f97b259c700 1 -- 172.21.15.3:0/1993535771 learned_addr learned my addr 172.21.15.3:0/1993535771 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:44.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.148+0000 7f97b1d9b700 1 -- 172.21.15.3:0/1993535771 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f97b409e930 msgr2=0x7f97b409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:44.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.148+0000 7f97b1d9b700 1 --2- 172.21.15.3:0/1993535771 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f97b409e930 0x7f97b409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.148+0000 7f97b1d9b700 1 -- 172.21.15.3:0/1993535771 >> v2:172.21.15.3:3300/0 conn(0x7f97b409f650 msgr2=0x7f97b4131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:44.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.148+0000 7f97b1d9b700 1 --2- 172.21.15.3:0/1993535771 >> v2:172.21.15.3:3300/0 conn(0x7f97b409f650 0x7f97b4131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.148+0000 7f97b1d9b700 1 -- 172.21.15.3:0/1993535771 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b409fb60 con 0x7f97b409f240 2023-10-03T10:17:44.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.148+0000 7f97b259c700 1 --2- 172.21.15.3:0/1993535771 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f97b409e930 0x7f97b409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:44.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.148+0000 7f97b2d9d700 1 --2- 172.21.15.3:0/1993535771 >> v2:172.21.15.3:3300/0 conn(0x7f97b409f650 0x7f97b4131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:44.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.148+0000 7f97b1d9b700 1 --2- 172.21.15.3:0/1993535771 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f97b409f240 0x7f97b41317f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f97a8009840 tx=0x7f97a8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6338a2c029a8f985 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:44.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.148+0000 7f97b159a700 1 -- 172.21.15.3:0/1993535771 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a8011830 con 0x7f97b409f240 2023-10-03T10:17:44.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.148+0000 7f97b159a700 1 -- 172.21.15.3:0/1993535771 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a8011990 con 0x7f97b409f240 2023-10-03T10:17:44.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.148+0000 7f97b159a700 1 -- 172.21.15.3:0/1993535771 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a8011d20 con 0x7f97b409f240 2023-10-03T10:17:44.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.149+0000 7f97b88da700 1 -- 172.21.15.3:0/1993535771 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f97b409f240 msgr2=0x7f97b41317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:44.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.149+0000 7f97b88da700 1 --2- 172.21.15.3:0/1993535771 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f97b409f240 0x7f97b41317f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f97a8009840 tx=0x7f97a8009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:44.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.149+0000 7f97b88da700 1 -- 172.21.15.3:0/1993535771 shutdown_connections 2023-10-03T10:17:44.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.149+0000 7f97b88da700 1 --2- 172.21.15.3:0/1993535771 >> v2:172.21.15.3:3300/0 conn(0x7f97b409f650 0x7f97b4131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.149+0000 7f97b88da700 1 --2- 172.21.15.3:0/1993535771 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f97b409e930 0x7f97b409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.149+0000 7f97b88da700 1 --2- 172.21.15.3:0/1993535771 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f97b409f240 0x7f97b41317f0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.149+0000 7f97b88da700 1 -- 172.21.15.3:0/1993535771 >> 172.21.15.3:0/1993535771 conn(0x7f97b40a4210 msgr2=0x7f97b40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:44.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.149+0000 7f97b88da700 1 -- 172.21.15.3:0/1993535771 shutdown_connections 2023-10-03T10:17:44.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.149+0000 7f97b88da700 1 -- 172.21.15.3:0/1993535771 wait complete. 2023-10-03T10:17:44.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.151+0000 7f97b88da700 1 Processor -- start 2023-10-03T10:17:44.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.151+0000 7f97b88da700 1 -- start start 2023-10-03T10:17:44.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.152+0000 7f97b88da700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f97b409e930 0x7f97b406b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:44.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.152+0000 7f97b88da700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f97b409f240 0x7f97b406bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:44.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.152+0000 7f97b88da700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97b409f650 0x7f97b406c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:44.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.152+0000 7f97b88da700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b406cd80 con 0x7f97b409f650 2023-10-03T10:17:44.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.152+0000 7f97b88da700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b41b7240 con 0x7f97b409e930 2023-10-03T10:17:44.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.152+0000 7f97b88da700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97b41a93f0 con 0x7f97b409f240 2023-10-03T10:17:44.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.152+0000 7f97b1d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f97b409f240 0x7f97b406bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:44.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.152+0000 7f97b1d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f97b409f240 0x7f97b406bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43082/0 (socket says 172.21.15.3:43082) 2023-10-03T10:17:44.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.152+0000 7f97b1d9b700 1 -- 172.21.15.3:0/2142599746 learned_addr learned my addr 172.21.15.3:0/2142599746 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:44.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.152+0000 7f97b259c700 1 --2- 172.21.15.3:0/2142599746 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f97b409e930 0x7f97b406b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:44.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.152+0000 7f97b2d9d700 1 --2- 172.21.15.3:0/2142599746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97b409f650 0x7f97b406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:44.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.152+0000 7f97b1d9b700 1 -- 172.21.15.3:0/2142599746 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f97b409e930 msgr2=0x7f97b406b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:44.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.152+0000 7f97b1d9b700 1 --2- 172.21.15.3:0/2142599746 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f97b409e930 0x7f97b406b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.152+0000 7f97b1d9b700 1 -- 172.21.15.3:0/2142599746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97b409f650 msgr2=0x7f97b406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:44.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.152+0000 7f97b1d9b700 1 --2- 172.21.15.3:0/2142599746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97b409f650 0x7f97b406c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.152+0000 7f97b1d9b700 1 -- 172.21.15.3:0/2142599746 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a4009550 con 0x7f97b409f240 2023-10-03T10:17:44.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.153+0000 7f97b1d9b700 1 --2- 172.21.15.3:0/2142599746 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f97b409f240 0x7f97b406bdf0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f97a8000c00 tx=0x7f97a8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:44.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.153+0000 7f97a2ffd700 1 -- 172.21.15.3:0/2142599746 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a8004450 con 0x7f97b409f240 2023-10-03T10:17:44.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.153+0000 7f97a2ffd700 1 -- 172.21.15.3:0/2142599746 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a801f070 con 0x7f97b409f240 2023-10-03T10:17:44.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.153+0000 7f97a2ffd700 1 -- 172.21.15.3:0/2142599746 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a80038e0 con 0x7f97b409f240 2023-10-03T10:17:44.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.153+0000 7f97b88da700 1 -- 172.21.15.3:0/2142599746 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ac009a70 con 0x7f97b409f240 2023-10-03T10:17:44.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.153+0000 7f97b88da700 1 -- 172.21.15.3:0/2142599746 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a80094a0 con 0x7f97b409f240 2023-10-03T10:17:44.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.154+0000 7f97a2ffd700 1 -- 172.21.15.3:0/2142599746 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f97a8003a40 con 0x7f97b409f240 2023-10-03T10:17:44.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.154+0000 7f97a2ffd700 1 -- 172.21.15.3:0/2142599746 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f97a8021070 con 0x7f97b409f240 2023-10-03T10:17:44.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.154+0000 7f97b88da700 1 -- 172.21.15.3:0/2142599746 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b41a0b30 con 0x7f97b409f240 2023-10-03T10:17:44.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.164+0000 7f97a2ffd700 1 -- 172.21.15.3:0/2142599746 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f97a801a430 con 0x7f97b409f240 2023-10-03T10:17:44.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.197+0000 7f94615ba700 1 Processor -- start 2023-10-03T10:17:44.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.197+0000 7f94615ba700 1 -- start start 2023-10-03T10:17:44.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.198+0000 7f94615ba700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f945c1338c0 0x7f945c133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:44.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.198+0000 7f94615ba700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f945c1341d0 0x7f945c09de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:44.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.198+0000 7f94615ba700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f945c06f720 con 0x7f945c09e390 2023-10-03T10:17:44.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.198+0000 7f94615ba700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f945c070c80 con 0x7f945c1341d0 2023-10-03T10:17:44.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.198+0000 7f94615ba700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f945c06fce0 con 0x7f945c1338c0 2023-10-03T10:17:44.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.198+0000 7f945a7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f945c1341d0 0x7f945c09de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:44.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.198+0000 7f945affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f945c1338c0 0x7f945c133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:44.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.198+0000 7f945affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f945c1338c0 0x7f945c133c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43092/0 (socket says 172.21.15.3:43092) 2023-10-03T10:17:44.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.198+0000 7f945a7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f945c1341d0 0x7f945c09de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58882/0 (socket says 172.21.15.3:58882) 2023-10-03T10:17:44.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.198+0000 7f945affd700 1 -- 172.21.15.3:0/2096625445 learned_addr learned my addr 172.21.15.3:0/2096625445 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:44.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.198+0000 7f945a7fc700 1 -- 172.21.15.3:0/2096625445 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f945c1338c0 msgr2=0x7f945c133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:44.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.198+0000 7f945a7fc700 1 --2- 172.21.15.3:0/2096625445 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f945c1338c0 0x7f945c133c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.198+0000 7f945a7fc700 1 -- 172.21.15.3:0/2096625445 >> v1:172.21.15.3:6789/0 conn(0x7f945c09e390 legacy=0x7f945c13a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:44.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.198+0000 7f945a7fc700 1 -- 172.21.15.3:0/2096625445 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945c1336a0 con 0x7f945c1341d0 2023-10-03T10:17:44.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.198+0000 7f9459ffb700 1 -- 172.21.15.3:0/2096625445 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3468305690 0 0) 0x7f945c06f720 con 0x7f945c09e390 2023-10-03T10:17:44.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.198+0000 7f945affd700 1 --2- 172.21.15.3:0/2096625445 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f945c1338c0 0x7f945c133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:44.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.199+0000 7f945a7fc700 1 --2- 172.21.15.3:0/2096625445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f945c1341d0 0x7f945c09de50 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f9450009840 tx=0x7f9450009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02ed952a6c36f61 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:44.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.199+0000 7f9459ffb700 1 -- 172.21.15.3:0/2096625445 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9450011830 con 0x7f945c1341d0 2023-10-03T10:17:44.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.199+0000 7f9459ffb700 1 -- 172.21.15.3:0/2096625445 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9450011990 con 0x7f945c1341d0 2023-10-03T10:17:44.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.199+0000 7f9459ffb700 1 -- 172.21.15.3:0/2096625445 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9450011d20 con 0x7f945c1341d0 2023-10-03T10:17:44.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.199+0000 7f94615ba700 1 -- 172.21.15.3:0/2096625445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f945c1341d0 msgr2=0x7f945c09de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:44.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.199+0000 7f94615ba700 1 --2- 172.21.15.3:0/2096625445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f945c1341d0 0x7f945c09de50 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f9450009840 tx=0x7f9450009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:44.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.200+0000 7f94615ba700 1 -- 172.21.15.3:0/2096625445 shutdown_connections 2023-10-03T10:17:44.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.200+0000 7f94615ba700 1 --2- 172.21.15.3:0/2096625445 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f945c1338c0 0x7f945c133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.200+0000 7f94615ba700 1 --2- 172.21.15.3:0/2096625445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f945c1341d0 0x7f945c09de50 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.200+0000 7f94615ba700 1 -- 172.21.15.3:0/2096625445 >> 172.21.15.3:0/2096625445 conn(0x7f945c0a2aa0 msgr2=0x7f945c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:44.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.200+0000 7f94615ba700 1 -- 172.21.15.3:0/2096625445 shutdown_connections 2023-10-03T10:17:44.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.200+0000 7f94615ba700 1 -- 172.21.15.3:0/2096625445 wait complete. 2023-10-03T10:17:44.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.201+0000 7f94615ba700 1 Processor -- start 2023-10-03T10:17:44.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.201+0000 7f94615ba700 1 -- start start 2023-10-03T10:17:44.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.202+0000 7f94615ba700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f945c09e390 0x7f945c11a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:44.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.202+0000 7f94615ba700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f945c1338c0 0x7f945c11aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:44.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.202+0000 7f94615ba700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f945c1341d0 0x7f945c11b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:44.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.202+0000 7f94615ba700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945c12feb0 con 0x7f945c1338c0 2023-10-03T10:17:44.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.202+0000 7f94615ba700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945c11db40 con 0x7f945c1341d0 2023-10-03T10:17:44.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.202+0000 7f94615ba700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f945c11c2e0 con 0x7f945c09e390 2023-10-03T10:17:44.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.202+0000 7f945a7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f945c1338c0 0x7f945c11aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:44.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.202+0000 7f945a7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f945c1338c0 0x7f945c11aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58896/0 (socket says 172.21.15.3:58896) 2023-10-03T10:17:44.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.202+0000 7f945a7fc700 1 -- 172.21.15.3:0/264749285 learned_addr learned my addr 172.21.15.3:0/264749285 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:44.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.202+0000 7f945b7fe700 1 --2- 172.21.15.3:0/264749285 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f945c1341d0 0x7f945c11b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:44.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.202+0000 7f945affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f945c09e390 0x7f945c11a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:44.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.203+0000 7f945a7fc700 1 -- 172.21.15.3:0/264749285 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f945c09e390 msgr2=0x7f945c11a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:44.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.203+0000 7f945a7fc700 1 --2- 172.21.15.3:0/264749285 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f945c09e390 0x7f945c11a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.203+0000 7f945a7fc700 1 -- 172.21.15.3:0/264749285 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f945c1341d0 msgr2=0x7f945c11b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:44.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.203+0000 7f945a7fc700 1 --2- 172.21.15.3:0/264749285 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f945c1341d0 0x7f945c11b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.203+0000 7f945a7fc700 1 -- 172.21.15.3:0/264749285 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f944c009550 con 0x7f945c1338c0 2023-10-03T10:17:44.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.203+0000 7f945b7fe700 1 --2- 172.21.15.3:0/264749285 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f945c1341d0 0x7f945c11b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:44.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.203+0000 7f945a7fc700 1 --2- 172.21.15.3:0/264749285 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f945c1338c0 0x7f945c11aba0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f9450009fd0 tx=0x7f9450004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:44.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.203+0000 7f94437fe700 1 -- 172.21.15.3:0/264749285 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94500036a0 con 0x7f945c1338c0 2023-10-03T10:17:44.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.203+0000 7f94437fe700 1 -- 172.21.15.3:0/264749285 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945001f070 con 0x7f945c1338c0 2023-10-03T10:17:44.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.203+0000 7f94615ba700 1 -- 172.21.15.3:0/264749285 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94500094a0 con 0x7f945c1338c0 2023-10-03T10:17:44.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.203+0000 7f94437fe700 1 -- 172.21.15.3:0/264749285 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9450003a60 con 0x7f945c1338c0 2023-10-03T10:17:44.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.203+0000 7f94615ba700 1 -- 172.21.15.3:0/264749285 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f945c11cfd0 con 0x7f945c1338c0 2023-10-03T10:17:44.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.204+0000 7f94615ba700 1 -- 172.21.15.3:0/264749285 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f945c11dc80 con 0x7f945c1338c0 2023-10-03T10:17:44.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.205+0000 7f94437fe700 1 -- 172.21.15.3:0/264749285 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9450003bc0 con 0x7f945c1338c0 2023-10-03T10:17:44.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.205+0000 7f94437fe700 1 -- 172.21.15.3:0/264749285 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9450021070 con 0x7f945c1338c0 2023-10-03T10:17:44.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.214+0000 7f94437fe700 1 -- 172.21.15.3:0/264749285 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9450025d70 con 0x7f945c1338c0 2023-10-03T10:17:44.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.227+0000 7f372838e700 1 Processor -- start 2023-10-03T10:17:44.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.227+0000 7f372838e700 1 -- start start 2023-10-03T10:17:44.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.228+0000 7f372838e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f372009d6f0 0x7f372009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:44.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.228+0000 7f372838e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f372009e000 0x7f37201339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:44.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.228+0000 7f372838e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f372009b120 con 0x7f372009e410 2023-10-03T10:17:44.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.228+0000 7f372838e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f372006b460 con 0x7f372009d6f0 2023-10-03T10:17:44.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.228+0000 7f372838e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f372009a7e0 con 0x7f372009e000 2023-10-03T10:17:44.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.228+0000 7f3725929700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f372009e000 0x7f37201339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:44.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.228+0000 7f372692b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f372009e410 0x7f3720133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42418/0 (socket says 172.21.15.3:42418) 2023-10-03T10:17:44.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.228+0000 7f3725929700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f372009e000 0x7f37201339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60292/0 (socket says 172.21.15.3:60292) 2023-10-03T10:17:44.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.228+0000 7f372692b700 1 -- 172.21.15.3:0/2717577804 learned_addr learned my addr 172.21.15.3:0/2717577804 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:44.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.228+0000 7f372612a700 1 --2- 172.21.15.3:0/2717577804 >> v2:172.21.15.3:3300/0 conn(0x7f372009d6f0 0x7f372009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:44.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.228+0000 7f3725128700 1 -- 172.21.15.3:0/2717577804 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1086903136 0 0) 0x7f372009b120 con 0x7f372009e410 2023-10-03T10:17:44.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.228+0000 7f3725128700 1 -- 172.21.15.3:0/2717577804 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f370c0035b0 con 0x7f372009e410 2023-10-03T10:17:44.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.228+0000 7f3725929700 1 -- 172.21.15.3:0/2717577804 >> v2:172.21.15.3:3300/0 conn(0x7f372009d6f0 msgr2=0x7f372009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:44.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.228+0000 7f3725929700 1 --2- 172.21.15.3:0/2717577804 >> v2:172.21.15.3:3300/0 conn(0x7f372009d6f0 0x7f372009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.228+0000 7f3725929700 1 -- 172.21.15.3:0/2717577804 >> v1:172.21.15.3:6789/0 conn(0x7f372009e410 legacy=0x7f3720133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:44.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.228+0000 7f3725929700 1 -- 172.21.15.3:0/2717577804 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f372009a500 con 0x7f372009e000 2023-10-03T10:17:44.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.228+0000 7f372612a700 1 --2- 172.21.15.3:0/2717577804 >> v2:172.21.15.3:3300/0 conn(0x7f372009d6f0 0x7f372009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:44.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.228+0000 7f3725929700 1 --2- 172.21.15.3:0/2717577804 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f372009e000 0x7f37201339f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3710009870 tx=0x7f3710009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57bd3ec633d5197a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:44.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.229+0000 7f3725128700 1 -- 172.21.15.3:0/2717577804 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3710011830 con 0x7f372009e000 2023-10-03T10:17:44.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.229+0000 7f3725128700 1 -- 172.21.15.3:0/2717577804 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3710011990 con 0x7f372009e000 2023-10-03T10:17:44.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.229+0000 7f3725128700 1 -- 172.21.15.3:0/2717577804 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3710011d20 con 0x7f372009e000 2023-10-03T10:17:44.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.229+0000 7f372838e700 1 -- 172.21.15.3:0/2717577804 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f372009e000 msgr2=0x7f37201339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:44.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.229+0000 7f372838e700 1 --2- 172.21.15.3:0/2717577804 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f372009e000 0x7f37201339f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3710009870 tx=0x7f3710009b80 comp rx=0 tx=0).stop 2023-10-03T10:17:44.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.229+0000 7f372838e700 1 -- 172.21.15.3:0/2717577804 shutdown_connections 2023-10-03T10:17:44.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.229+0000 7f372838e700 1 --2- 172.21.15.3:0/2717577804 >> v2:172.21.15.3:3300/0 conn(0x7f372009d6f0 0x7f372009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.229+0000 7f372838e700 1 --2- 172.21.15.3:0/2717577804 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f372009e000 0x7f37201339f0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.229+0000 7f372838e700 1 -- 172.21.15.3:0/2717577804 >> 172.21.15.3:0/2717577804 conn(0x7f3720066f50 msgr2=0x7f372009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:44.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.230+0000 7f372838e700 1 -- 172.21.15.3:0/2717577804 shutdown_connections 2023-10-03T10:17:44.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.230+0000 7f372838e700 1 -- 172.21.15.3:0/2717577804 wait complete. 2023-10-03T10:17:44.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.231+0000 7f372838e700 1 Processor -- start 2023-10-03T10:17:44.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.231+0000 7f372838e700 1 -- start start 2023-10-03T10:17:44.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.231+0000 7f372838e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f372009d6f0 0x7f37201c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:44.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.231+0000 7f372838e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f372009e000 0x7f37201c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:44.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.231+0000 7f372838e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f372009e410 0x7f37201c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:44.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.231+0000 7f372838e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37201a8160 con 0x7f372009e000 2023-10-03T10:17:44.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.231+0000 7f372838e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37201a6450 con 0x7f372009e410 2023-10-03T10:17:44.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.231+0000 7f372838e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f37200a2cc0 con 0x7f372009d6f0 2023-10-03T10:17:44.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.231+0000 7f372692b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f372009e410 0x7f37201c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:44.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.231+0000 7f372612a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f372009d6f0 0x7f37201c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:44.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.231+0000 7f372692b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f372009e410 0x7f37201c8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60302/0 (socket says 172.21.15.3:60302) 2023-10-03T10:17:44.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.231+0000 7f372692b700 1 -- 172.21.15.3:0/1867950682 learned_addr learned my addr 172.21.15.3:0/1867950682 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:44.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.231+0000 7f372612a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f372009d6f0 0x7f37201c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43124/0 (socket says 172.21.15.3:43124) 2023-10-03T10:17:44.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.231+0000 7f3725929700 1 --2- 172.21.15.3:0/1867950682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f372009e000 0x7f37201c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:44.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.232+0000 7f372692b700 1 -- 172.21.15.3:0/1867950682 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f372009d6f0 msgr2=0x7f37201c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:44.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.232+0000 7f372692b700 1 --2- 172.21.15.3:0/1867950682 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f372009d6f0 0x7f37201c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.232+0000 7f372692b700 1 -- 172.21.15.3:0/1867950682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f372009e000 msgr2=0x7f37201c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:44.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.232+0000 7f372692b700 1 --2- 172.21.15.3:0/1867950682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f372009e000 0x7f37201c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.232+0000 7f372692b700 1 -- 172.21.15.3:0/1867950682 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371c0094a0 con 0x7f372009e410 2023-10-03T10:17:44.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.232+0000 7f372692b700 1 --2- 172.21.15.3:0/1867950682 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f372009e410 0x7f37201c8ac0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f371800cb70 tx=0x7f371800ce80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:44.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.232+0000 7f3716ffd700 1 -- 172.21.15.3:0/1867950682 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3718015e30 con 0x7f372009e410 2023-10-03T10:17:44.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.232+0000 7f3716ffd700 1 -- 172.21.15.3:0/1867950682 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371800a590 con 0x7f372009e410 2023-10-03T10:17:44.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.232+0000 7f372838e700 1 -- 172.21.15.3:0/1867950682 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37100094a0 con 0x7f372009e410 2023-10-03T10:17:44.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.232+0000 7f3716ffd700 1 -- 172.21.15.3:0/1867950682 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371800a920 con 0x7f372009e410 2023-10-03T10:17:44.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.232+0000 7f372838e700 1 -- 172.21.15.3:0/1867950682 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37201a8360 con 0x7f372009e410 2023-10-03T10:17:44.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.233+0000 7f3716ffd700 1 -- 172.21.15.3:0/1867950682 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f371801b070 con 0x7f372009e410 2023-10-03T10:17:44.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.233+0000 7f372838e700 1 -- 172.21.15.3:0/1867950682 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3720074ea0 con 0x7f372009e410 2023-10-03T10:17:44.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.233+0000 7f3716ffd700 1 -- 172.21.15.3:0/1867950682 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3718047ba0 con 0x7f372009e410 2023-10-03T10:17:44.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.243+0000 7f3716ffd700 1 -- 172.21.15.3:0/1867950682 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3718026940 con 0x7f372009e410 2023-10-03T10:17:44.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.393+0000 7f97b88da700 1 -- 172.21.15.3:0/2142599746 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f97b40a3670 con 0x7f97b409f240 2023-10-03T10:17:44.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.393+0000 7f97a2ffd700 1 -- 172.21.15.3:0/2142599746 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f97a8015030 con 0x7f97b409f240 2023-10-03T10:17:44.394 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:44.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.396+0000 7f97b88da700 1 -- 172.21.15.3:0/2142599746 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f97b409f240 msgr2=0x7f97b406bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:44.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.396+0000 7f97b88da700 1 --2- 172.21.15.3:0/2142599746 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f97b409f240 0x7f97b406bdf0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f97a8000c00 tx=0x7f97a8004750 comp rx=0 tx=0).stop 2023-10-03T10:17:44.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.396+0000 7f97b88da700 1 -- 172.21.15.3:0/2142599746 shutdown_connections 2023-10-03T10:17:44.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.396+0000 7f97b88da700 1 --2- 172.21.15.3:0/2142599746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97b409f650 0x7f97b406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.396+0000 7f97b88da700 1 --2- 172.21.15.3:0/2142599746 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f97b409e930 0x7f97b406b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.396+0000 7f97b88da700 1 --2- 172.21.15.3:0/2142599746 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f97b409f240 0x7f97b406bdf0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.396+0000 7f97b88da700 1 -- 172.21.15.3:0/2142599746 >> 172.21.15.3:0/2142599746 conn(0x7f97b40a4210 msgr2=0x7f97b41b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:44.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.396+0000 7f97b88da700 1 -- 172.21.15.3:0/2142599746 shutdown_connections 2023-10-03T10:17:44.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.396+0000 7f97b88da700 1 -- 172.21.15.3:0/2142599746 wait complete. 2023-10-03T10:17:44.410 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:44.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.436+0000 7f94615ba700 1 -- 172.21.15.3:0/264749285 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f945c074ea0 con 0x7f945c1338c0 2023-10-03T10:17:44.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.437+0000 7f94437fe700 1 -- 172.21.15.3:0/264749285 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9450025b90 con 0x7f945c1338c0 2023-10-03T10:17:44.437 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:44.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.440+0000 7f94615ba700 1 -- 172.21.15.3:0/264749285 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f945c1338c0 msgr2=0x7f945c11aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:44.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.440+0000 7f94615ba700 1 --2- 172.21.15.3:0/264749285 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f945c1338c0 0x7f945c11aba0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f9450009fd0 tx=0x7f9450004750 comp rx=0 tx=0).stop 2023-10-03T10:17:44.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.440+0000 7f94615ba700 1 -- 172.21.15.3:0/264749285 shutdown_connections 2023-10-03T10:17:44.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.440+0000 7f94615ba700 1 --2- 172.21.15.3:0/264749285 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f945c1341d0 0x7f945c11b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.440+0000 7f94615ba700 1 --2- 172.21.15.3:0/264749285 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f945c09e390 0x7f945c11a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.440+0000 7f94615ba700 1 --2- 172.21.15.3:0/264749285 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f945c1338c0 0x7f945c11aba0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.440+0000 7f94615ba700 1 -- 172.21.15.3:0/264749285 >> 172.21.15.3:0/264749285 conn(0x7f945c0a2aa0 msgr2=0x7f945c12f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:44.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.440+0000 7f94615ba700 1 -- 172.21.15.3:0/264749285 shutdown_connections 2023-10-03T10:17:44.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.440+0000 7f94615ba700 1 -- 172.21.15.3:0/264749285 wait complete. 2023-10-03T10:17:44.459 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:44.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.467+0000 7f372838e700 1 -- 172.21.15.3:0/1867950682 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f37201a7f50 con 0x7f372009e410 2023-10-03T10:17:44.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.467+0000 7f3716ffd700 1 -- 172.21.15.3:0/1867950682 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3718023020 con 0x7f372009e410 2023-10-03T10:17:44.467 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.469+0000 7f372838e700 1 -- 172.21.15.3:0/1867950682 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f372009e410 msgr2=0x7f37201c8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.469+0000 7f372838e700 1 --2- 172.21.15.3:0/1867950682 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f372009e410 0x7f37201c8ac0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f371800cb70 tx=0x7f371800ce80 comp rx=0 tx=0).stop 2023-10-03T10:17:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.469+0000 7f372838e700 1 -- 172.21.15.3:0/1867950682 shutdown_connections 2023-10-03T10:17:44.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.470+0000 7f372838e700 1 --2- 172.21.15.3:0/1867950682 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f372009e410 0x7f37201c8ac0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.470+0000 7f372838e700 1 --2- 172.21.15.3:0/1867950682 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f372009d6f0 0x7f37201c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.470+0000 7f372838e700 1 --2- 172.21.15.3:0/1867950682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f372009e000 0x7f37201c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.470+0000 7f372838e700 1 -- 172.21.15.3:0/1867950682 >> 172.21.15.3:0/1867950682 conn(0x7f3720066f50 msgr2=0x7f37201bbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:44.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.470+0000 7f372838e700 1 -- 172.21.15.3:0/1867950682 shutdown_connections 2023-10-03T10:17:44.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.470+0000 7f372838e700 1 -- 172.21.15.3:0/1867950682 wait complete. 2023-10-03T10:17:44.483 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:44.704 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:44.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.850+0000 7f067b086700 1 Processor -- start 2023-10-03T10:17:44.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.850+0000 7f067b086700 1 -- start start 2023-10-03T10:17:44.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.851+0000 7f067b086700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f067409d6f0 0x7f067409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:44.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.851+0000 7f067b086700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f067409e000 0x7f06741339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:44.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.851+0000 7f067b086700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f067409e410 0x7f0674133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:44.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.851+0000 7f067b086700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f067406b460 con 0x7f067409d6f0 2023-10-03T10:17:44.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.851+0000 7f067b086700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f067409b120 con 0x7f067409e000 2023-10-03T10:17:44.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.851+0000 7f067b086700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f067409a7e0 con 0x7f067409e410 2023-10-03T10:17:44.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.851+0000 7f0678e22700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f067409d6f0 0x7f067409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:44.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.851+0000 7f0673fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f067409e000 0x7f06741339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:44.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.851+0000 7f0679623700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f067409e410 0x7f0674133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:44.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.851+0000 7f0673fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f067409e000 0x7f06741339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60314/0 (socket says 172.21.15.3:60314) 2023-10-03T10:17:44.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.851+0000 7f0678e22700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f067409d6f0 0x7f067409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58914/0 (socket says 172.21.15.3:58914) 2023-10-03T10:17:44.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.851+0000 7f0673fff700 1 -- 172.21.15.3:0/340496643 learned_addr learned my addr 172.21.15.3:0/340496643 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:44.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.852+0000 7f0673fff700 1 -- 172.21.15.3:0/340496643 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f067409e410 msgr2=0x7f0674133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:44.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.852+0000 7f0673fff700 1 --2- 172.21.15.3:0/340496643 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f067409e410 0x7f0674133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.852+0000 7f0673fff700 1 -- 172.21.15.3:0/340496643 >> v2:172.21.15.3:3300/0 conn(0x7f067409d6f0 msgr2=0x7f067409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:44.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.852+0000 7f0673fff700 1 --2- 172.21.15.3:0/340496643 >> v2:172.21.15.3:3300/0 conn(0x7f067409d6f0 0x7f067409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.852+0000 7f0673fff700 1 -- 172.21.15.3:0/340496643 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067409a500 con 0x7f067409e000 2023-10-03T10:17:44.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.852+0000 7f0679623700 1 --2- 172.21.15.3:0/340496643 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f067409e410 0x7f0674133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:44.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.852+0000 7f0673fff700 1 --2- 172.21.15.3:0/340496643 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f067409e000 0x7f06741339f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f065c009920 tx=0x7f065c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86e138a7bd1a6cd0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:44.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.852+0000 7f06737fe700 1 -- 172.21.15.3:0/340496643 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c011830 con 0x7f067409e000 2023-10-03T10:17:44.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.852+0000 7f06737fe700 1 -- 172.21.15.3:0/340496643 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065c011990 con 0x7f067409e000 2023-10-03T10:17:44.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.852+0000 7f06737fe700 1 -- 172.21.15.3:0/340496643 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c011d20 con 0x7f067409e000 2023-10-03T10:17:44.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.853+0000 7f067b086700 1 -- 172.21.15.3:0/340496643 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f067409e000 msgr2=0x7f06741339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:44.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.853+0000 7f067b086700 1 --2- 172.21.15.3:0/340496643 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f067409e000 0x7f06741339f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f065c009920 tx=0x7f065c009c30 comp rx=0 tx=0).stop 2023-10-03T10:17:44.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.853+0000 7f067b086700 1 -- 172.21.15.3:0/340496643 shutdown_connections 2023-10-03T10:17:44.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.853+0000 7f067b086700 1 --2- 172.21.15.3:0/340496643 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f067409e410 0x7f0674133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.853+0000 7f067b086700 1 --2- 172.21.15.3:0/340496643 >> v2:172.21.15.3:3300/0 conn(0x7f067409d6f0 0x7f067409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.853+0000 7f067b086700 1 --2- 172.21.15.3:0/340496643 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f067409e000 0x7f06741339f0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.853+0000 7f067b086700 1 -- 172.21.15.3:0/340496643 >> 172.21.15.3:0/340496643 conn(0x7f0674066f50 msgr2=0x7f067409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:44.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.853+0000 7f067b086700 1 -- 172.21.15.3:0/340496643 shutdown_connections 2023-10-03T10:17:44.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.853+0000 7f067b086700 1 -- 172.21.15.3:0/340496643 wait complete. 2023-10-03T10:17:44.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.854+0000 7f067b086700 1 Processor -- start 2023-10-03T10:17:44.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.854+0000 7f067b086700 1 -- start start 2023-10-03T10:17:44.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.855+0000 7f067b086700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f067409d6f0 0x7f06741c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:44.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.855+0000 7f067b086700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f067409e000 0x7f06741c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:44.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.855+0000 7f067b086700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f067409e410 0x7f06741c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:44.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.855+0000 7f067b086700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06741a9920 con 0x7f067409e410 2023-10-03T10:17:44.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.855+0000 7f067b086700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06741a7910 con 0x7f067409e000 2023-10-03T10:17:44.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.855+0000 7f067b086700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f067409c710 con 0x7f067409d6f0 2023-10-03T10:17:44.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.855+0000 7f0678e22700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f067409d6f0 0x7f06741c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:44.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.855+0000 7f0673fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f067409e000 0x7f06741c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:44.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.855+0000 7f0678e22700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f067409d6f0 0x7f06741c8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43138/0 (socket says 172.21.15.3:43138) 2023-10-03T10:17:44.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.855+0000 7f0678e22700 1 -- 172.21.15.3:0/1755509948 learned_addr learned my addr 172.21.15.3:0/1755509948 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:44.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.855+0000 7f0673fff700 1 --2- 172.21.15.3:0/1755509948 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f067409e000 0x7f06741c8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60324/0 (socket says 172.21.15.3:60324) 2023-10-03T10:17:44.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.855+0000 7f0679623700 1 --2- 172.21.15.3:0/1755509948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f067409e410 0x7f06741c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:44.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.855+0000 7f0673fff700 1 -- 172.21.15.3:0/1755509948 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f067409d6f0 msgr2=0x7f06741c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:44.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.855+0000 7f0673fff700 1 --2- 172.21.15.3:0/1755509948 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f067409d6f0 0x7f06741c8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.855+0000 7f0673fff700 1 -- 172.21.15.3:0/1755509948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f067409e410 msgr2=0x7f06741c4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:44.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.855+0000 7f0673fff700 1 --2- 172.21.15.3:0/1755509948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f067409e410 0x7f06741c4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:44.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.855+0000 7f0673fff700 1 -- 172.21.15.3:0/1755509948 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066c009a70 con 0x7f067409e000 2023-10-03T10:17:44.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.855+0000 7f0678e22700 1 --2- 172.21.15.3:0/1755509948 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f067409d6f0 0x7f06741c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:44.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.855+0000 7f0673fff700 1 --2- 172.21.15.3:0/1755509948 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f067409e000 0x7f06741c8a60 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f065c016000 tx=0x7f065c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:44.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.856+0000 7f06717fa700 1 -- 172.21.15.3:0/1755509948 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c004450 con 0x7f067409e000 2023-10-03T10:17:44.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.856+0000 7f06717fa700 1 -- 172.21.15.3:0/1755509948 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065c0045d0 con 0x7f067409e000 2023-10-03T10:17:44.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.856+0000 7f067b086700 1 -- 172.21.15.3:0/1755509948 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f065c009550 con 0x7f067409e000 2023-10-03T10:17:44.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.856+0000 7f06717fa700 1 -- 172.21.15.3:0/1755509948 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c0038e0 con 0x7f067409e000 2023-10-03T10:17:44.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.856+0000 7f067b086700 1 -- 172.21.15.3:0/1755509948 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06741a9280 con 0x7f067409e000 2023-10-03T10:17:44.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.856+0000 7f06717fa700 1 -- 172.21.15.3:0/1755509948 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f065c003a40 con 0x7f067409e000 2023-10-03T10:17:44.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.857+0000 7f06717fa700 1 -- 172.21.15.3:0/1755509948 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f065c021030 con 0x7f067409e000 2023-10-03T10:17:44.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.857+0000 7f067b086700 1 -- 172.21.15.3:0/1755509948 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06741a87d0 con 0x7f067409e000 2023-10-03T10:17:44.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:44.862+0000 7f06717fa700 1 -- 172.21.15.3:0/1755509948 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f06741a87d0 con 0x7f067409e000 2023-10-03T10:17:45.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.065+0000 7f067b086700 1 -- 172.21.15.3:0/1755509948 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f06741a87d0 con 0x7f067409e000 2023-10-03T10:17:45.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.066+0000 7f06717fa700 1 -- 172.21.15.3:0/1755509948 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f06741a87d0 con 0x7f067409e000 2023-10-03T10:17:45.066 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:45.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.068+0000 7f067b086700 1 -- 172.21.15.3:0/1755509948 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f067409e000 msgr2=0x7f06741c8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:45.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.068+0000 7f067b086700 1 --2- 172.21.15.3:0/1755509948 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f067409e000 0x7f06741c8a60 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f065c016000 tx=0x7f065c004750 comp rx=0 tx=0).stop 2023-10-03T10:17:45.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.068+0000 7f067b086700 1 -- 172.21.15.3:0/1755509948 shutdown_connections 2023-10-03T10:17:45.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.068+0000 7f067b086700 1 --2- 172.21.15.3:0/1755509948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f067409e410 0x7f06741c4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.068+0000 7f067b086700 1 --2- 172.21.15.3:0/1755509948 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f067409d6f0 0x7f06741c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.068+0000 7f067b086700 1 --2- 172.21.15.3:0/1755509948 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f067409e000 0x7f06741c8a60 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.068+0000 7f067b086700 1 -- 172.21.15.3:0/1755509948 >> 172.21.15.3:0/1755509948 conn(0x7f0674066f50 msgr2=0x7f06741bad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:45.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.068+0000 7f067b086700 1 -- 172.21.15.3:0/1755509948 shutdown_connections 2023-10-03T10:17:45.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.068+0000 7f067b086700 1 -- 172.21.15.3:0/1755509948 wait complete. 2023-10-03T10:17:45.082 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:45.411 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:45.459 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:45.484 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:45.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.573+0000 7f1a44e18700 1 Processor -- start 2023-10-03T10:17:45.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.573+0000 7f1a44e18700 1 -- start start 2023-10-03T10:17:45.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.574+0000 7f1a44e18700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1a4009d6f0 0x7f1a4009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:45.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.574+0000 7f1a44e18700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a4009e000 0x7f1a401339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:45.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.574+0000 7f1a44e18700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a4009e410 0x7f1a40133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:45.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.574+0000 7f1a44e18700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a4006b460 con 0x7f1a4009d6f0 2023-10-03T10:17:45.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.574+0000 7f1a44e18700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4009b120 con 0x7f1a4009e410 2023-10-03T10:17:45.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.574+0000 7f1a44e18700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4009a7e0 con 0x7f1a4009e000 2023-10-03T10:17:45.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.574+0000 7f1a3e59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1a4009d6f0 0x7f1a4009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:45.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.574+0000 7f1a3ed9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a4009e410 0x7f1a40133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:45.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.574+0000 7f1a3ed9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a4009e410 0x7f1a40133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60334/0 (socket says 172.21.15.3:60334) 2023-10-03T10:17:45.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.574+0000 7f1a3e59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1a4009d6f0 0x7f1a4009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58932/0 (socket says 172.21.15.3:58932) 2023-10-03T10:17:45.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.574+0000 7f1a3ed9d700 1 -- 172.21.15.3:0/383711930 learned_addr learned my addr 172.21.15.3:0/383711930 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:45.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.574+0000 7f1a3dd9b700 1 --2- 172.21.15.3:0/383711930 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a4009e000 0x7f1a401339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:45.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.574+0000 7f1a3dd9b700 1 -- 172.21.15.3:0/383711930 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a4009e410 msgr2=0x7f1a40133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:45.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.574+0000 7f1a3dd9b700 1 --2- 172.21.15.3:0/383711930 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a4009e410 0x7f1a40133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.574+0000 7f1a3dd9b700 1 -- 172.21.15.3:0/383711930 >> v2:172.21.15.3:3300/0 conn(0x7f1a4009d6f0 msgr2=0x7f1a4009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:45.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.574+0000 7f1a3dd9b700 1 --2- 172.21.15.3:0/383711930 >> v2:172.21.15.3:3300/0 conn(0x7f1a4009d6f0 0x7f1a4009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.574+0000 7f1a3dd9b700 1 -- 172.21.15.3:0/383711930 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a4009a500 con 0x7f1a4009e000 2023-10-03T10:17:45.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.574+0000 7f1a3ed9d700 1 --2- 172.21.15.3:0/383711930 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a4009e410 0x7f1a40133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:45.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.575+0000 7f1a3e59c700 1 --2- 172.21.15.3:0/383711930 >> v2:172.21.15.3:3300/0 conn(0x7f1a4009d6f0 0x7f1a4009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:45.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.575+0000 7f1a3dd9b700 1 --2- 172.21.15.3:0/383711930 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a4009e000 0x7f1a401339f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1a34009870 tx=0x7f1a34009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62db8ca26c6f3a79 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:45.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.575+0000 7f1a3d59a700 1 -- 172.21.15.3:0/383711930 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a34011830 con 0x7f1a4009e000 2023-10-03T10:17:45.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.575+0000 7f1a3d59a700 1 -- 172.21.15.3:0/383711930 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a34011990 con 0x7f1a4009e000 2023-10-03T10:17:45.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.575+0000 7f1a3d59a700 1 -- 172.21.15.3:0/383711930 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a34011d20 con 0x7f1a4009e000 2023-10-03T10:17:45.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.576+0000 7f1a44e18700 1 -- 172.21.15.3:0/383711930 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a4009e000 msgr2=0x7f1a401339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:45.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.576+0000 7f1a44e18700 1 --2- 172.21.15.3:0/383711930 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a4009e000 0x7f1a401339f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1a34009870 tx=0x7f1a34009b80 comp rx=0 tx=0).stop 2023-10-03T10:17:45.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.576+0000 7f1a44e18700 1 -- 172.21.15.3:0/383711930 shutdown_connections 2023-10-03T10:17:45.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.576+0000 7f1a44e18700 1 --2- 172.21.15.3:0/383711930 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a4009e410 0x7f1a40133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.576+0000 7f1a44e18700 1 --2- 172.21.15.3:0/383711930 >> v2:172.21.15.3:3300/0 conn(0x7f1a4009d6f0 0x7f1a4009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.576+0000 7f1a44e18700 1 --2- 172.21.15.3:0/383711930 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a4009e000 0x7f1a401339f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.576+0000 7f1a44e18700 1 -- 172.21.15.3:0/383711930 >> 172.21.15.3:0/383711930 conn(0x7f1a40066f50 msgr2=0x7f1a4009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:45.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.576+0000 7f1a44e18700 1 -- 172.21.15.3:0/383711930 shutdown_connections 2023-10-03T10:17:45.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.576+0000 7f1a44e18700 1 -- 172.21.15.3:0/383711930 wait complete. 2023-10-03T10:17:45.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.577+0000 7f1a44e18700 1 Processor -- start 2023-10-03T10:17:45.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.577+0000 7f1a44e18700 1 -- start start 2023-10-03T10:17:45.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.578+0000 7f1a44e18700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a4009d6f0 0x7f1a401a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:45.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.578+0000 7f1a44e18700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a4009e000 0x7f1a401a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:45.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.578+0000 7f1a44e18700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a4009e410 0x7f1a401a6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:45.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.578+0000 7f1a44e18700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a401a8790 con 0x7f1a4009e000 2023-10-03T10:17:45.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.578+0000 7f1a44e18700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a401a9940 con 0x7f1a4009d6f0 2023-10-03T10:17:45.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.578+0000 7f1a44e18700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a400a1f00 con 0x7f1a4009e410 2023-10-03T10:17:45.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.578+0000 7f1a3dd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a4009e000 0x7f1a401a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:45.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.578+0000 7f1a3e59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a4009d6f0 0x7f1a401a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:45.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.578+0000 7f1a3dd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a4009e000 0x7f1a401a6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58934/0 (socket says 172.21.15.3:58934) 2023-10-03T10:17:45.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.578+0000 7f1a3dd9b700 1 -- 172.21.15.3:0/1974081924 learned_addr learned my addr 172.21.15.3:0/1974081924 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:45.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.578+0000 7f1a3ed9d700 1 --2- 172.21.15.3:0/1974081924 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a4009e410 0x7f1a401a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:45.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.578+0000 7f1a3dd9b700 1 -- 172.21.15.3:0/1974081924 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a4009e410 msgr2=0x7f1a401a6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:45.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.578+0000 7f1a3dd9b700 1 --2- 172.21.15.3:0/1974081924 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a4009e410 0x7f1a401a6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.578+0000 7f1a3dd9b700 1 -- 172.21.15.3:0/1974081924 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a4009d6f0 msgr2=0x7f1a401a5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:45.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.578+0000 7f1a3dd9b700 1 --2- 172.21.15.3:0/1974081924 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a4009d6f0 0x7f1a401a5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.578+0000 7f1a3dd9b700 1 -- 172.21.15.3:0/1974081924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a300094a0 con 0x7f1a4009e000 2023-10-03T10:17:45.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.578+0000 7f1a3e59c700 1 --2- 172.21.15.3:0/1974081924 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a4009d6f0 0x7f1a401a5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:45.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.579+0000 7f1a3dd9b700 1 --2- 172.21.15.3:0/1974081924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a4009e000 0x7f1a401a6430 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f1a34000c00 tx=0x7f1a34004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:45.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.579+0000 7f1a2effd700 1 -- 172.21.15.3:0/1974081924 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a34004450 con 0x7f1a4009e000 2023-10-03T10:17:45.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.579+0000 7f1a2effd700 1 -- 172.21.15.3:0/1974081924 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a3401f070 con 0x7f1a4009e000 2023-10-03T10:17:45.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.579+0000 7f1a44e18700 1 -- 172.21.15.3:0/1974081924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a38009b30 con 0x7f1a4009e000 2023-10-03T10:17:45.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.579+0000 7f1a2effd700 1 -- 172.21.15.3:0/1974081924 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a340038e0 con 0x7f1a4009e000 2023-10-03T10:17:45.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.579+0000 7f1a44e18700 1 -- 172.21.15.3:0/1974081924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a340094a0 con 0x7f1a4009e000 2023-10-03T10:17:45.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.580+0000 7f1a2effd700 1 -- 172.21.15.3:0/1974081924 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1a34003a40 con 0x7f1a4009e000 2023-10-03T10:17:45.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.580+0000 7f1a2effd700 1 -- 172.21.15.3:0/1974081924 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1a34021070 con 0x7f1a4009e000 2023-10-03T10:17:45.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.581+0000 7f1a44e18700 1 -- 172.21.15.3:0/1974081924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a40074ea0 con 0x7f1a4009e000 2023-10-03T10:17:45.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.590+0000 7f1a2effd700 1 -- 172.21.15.3:0/1974081924 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1a3402b020 con 0x7f1a4009e000 2023-10-03T10:17:45.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.619+0000 7fd9c4531700 1 Processor -- start 2023-10-03T10:17:45.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.619+0000 7fd9c4531700 1 -- start start 2023-10-03T10:17:45.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.620+0000 7fd9c4531700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9bc133420 0x7fd9bc1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:45.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.620+0000 7fd9c4531700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9bc134260 0x7fd9bc1306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:45.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.620+0000 7fd9c4531700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9bc06b990 con 0x7fd9bc133e30 2023-10-03T10:17:45.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.620+0000 7fd9c4531700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc0a0590 con 0x7fd9bc133420 2023-10-03T10:17:45.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.620+0000 7fd9c4531700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc09fb80 con 0x7fd9bc134260 2023-10-03T10:17:45.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.620+0000 7fd9c22cd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9bc133420 0x7fd9bc1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:45.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.620+0000 7fd9c2ace700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9bc134260 0x7fd9bc1306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:45.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.620+0000 7fd9c22cd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9bc133420 0x7fd9bc1337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60344/0 (socket says 172.21.15.3:60344) 2023-10-03T10:17:45.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.620+0000 7fd9c22cd700 1 -- 172.21.15.3:0/1567364555 learned_addr learned my addr 172.21.15.3:0/1567364555 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:45.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.621+0000 7fd9c2ace700 1 -- 172.21.15.3:0/1567364555 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9bc133420 msgr2=0x7fd9bc1337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:45.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.621+0000 7fd9c2ace700 1 --2- 172.21.15.3:0/1567364555 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9bc133420 0x7fd9bc1337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.621+0000 7fd9c2ace700 1 -- 172.21.15.3:0/1567364555 >> v1:172.21.15.3:6789/0 conn(0x7fd9bc133e30 legacy=0x7fd9bc12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:45.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.621+0000 7fd9c2ace700 1 -- 172.21.15.3:0/1567364555 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9bc0a0030 con 0x7fd9bc134260 2023-10-03T10:17:45.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.621+0000 7fd9c12cb700 1 -- 172.21.15.3:0/1567364555 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1087594710 0 0) 0x7fd9bc06b990 con 0x7fd9bc133e30 2023-10-03T10:17:45.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.621+0000 7fd9c22cd700 1 --2- 172.21.15.3:0/1567364555 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9bc133420 0x7fd9bc1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:45.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.621+0000 7fd9c2ace700 1 --2- 172.21.15.3:0/1567364555 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9bc134260 0x7fd9bc1306a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009dc0 tx=0x7fd9b400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4099a3ef4b89fab server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:45.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.621+0000 7fd9c12cb700 1 -- 172.21.15.3:0/1567364555 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b400e070 con 0x7fd9bc134260 2023-10-03T10:17:45.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.621+0000 7fd9c12cb700 1 -- 172.21.15.3:0/1567364555 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b400bb20 con 0x7fd9bc134260 2023-10-03T10:17:45.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.621+0000 7fd9c12cb700 1 -- 172.21.15.3:0/1567364555 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4012430 con 0x7fd9bc134260 2023-10-03T10:17:45.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.622+0000 7fd9c4531700 1 -- 172.21.15.3:0/1567364555 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9bc134260 msgr2=0x7fd9bc1306a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:45.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.622+0000 7fd9c4531700 1 --2- 172.21.15.3:0/1567364555 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9bc134260 0x7fd9bc1306a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009dc0 tx=0x7fd9b400b330 comp rx=0 tx=0).stop 2023-10-03T10:17:45.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.622+0000 7fd9c4531700 1 -- 172.21.15.3:0/1567364555 shutdown_connections 2023-10-03T10:17:45.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.622+0000 7fd9c4531700 1 --2- 172.21.15.3:0/1567364555 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9bc133420 0x7fd9bc1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.622+0000 7fd9c4531700 1 --2- 172.21.15.3:0/1567364555 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9bc134260 0x7fd9bc1306a0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.622+0000 7fd9c4531700 1 -- 172.21.15.3:0/1567364555 >> 172.21.15.3:0/1567364555 conn(0x7fd9bc0a4210 msgr2=0x7fd9bc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:45.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.622+0000 7fd9c4531700 1 -- 172.21.15.3:0/1567364555 shutdown_connections 2023-10-03T10:17:45.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.622+0000 7fd9c4531700 1 -- 172.21.15.3:0/1567364555 wait complete. 2023-10-03T10:17:45.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.624+0000 7fd9c4531700 1 Processor -- start 2023-10-03T10:17:45.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.624+0000 7fd9c4531700 1 -- start start 2023-10-03T10:17:45.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.624+0000 7fd9c4531700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9bc133420 0x7fd9bc09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:45.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.624+0000 7fd9c4531700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9bc133e30 0x7fd9bc09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:45.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.624+0000 7fd9c4531700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9bc134260 0x7fd9bc09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:45.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.624+0000 7fd9c4531700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc09d850 con 0x7fd9bc133420 2023-10-03T10:17:45.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.624+0000 7fd9c4531700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc120430 con 0x7fd9bc133e30 2023-10-03T10:17:45.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.624+0000 7fd9c4531700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc121010 con 0x7fd9bc134260 2023-10-03T10:17:45.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.625+0000 7fd9c22cd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9bc133420 0x7fd9bc09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:45.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.625+0000 7fd9c22cd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9bc133420 0x7fd9bc09c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58950/0 (socket says 172.21.15.3:58950) 2023-10-03T10:17:45.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.625+0000 7fd9c22cd700 1 -- 172.21.15.3:0/4293625912 learned_addr learned my addr 172.21.15.3:0/4293625912 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:45.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.625+0000 7fd9c2ace700 1 --2- 172.21.15.3:0/4293625912 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9bc134260 0x7fd9bc09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:45.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.625+0000 7fd9c1acc700 1 --2- 172.21.15.3:0/4293625912 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9bc133e30 0x7fd9bc09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:45.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.625+0000 7fd9c22cd700 1 -- 172.21.15.3:0/4293625912 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9bc134260 msgr2=0x7fd9bc09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:45.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.625+0000 7fd9c22cd700 1 --2- 172.21.15.3:0/4293625912 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9bc134260 0x7fd9bc09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.625+0000 7fd9c22cd700 1 -- 172.21.15.3:0/4293625912 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9bc133e30 msgr2=0x7fd9bc09c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:45.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.625+0000 7fd9c22cd700 1 --2- 172.21.15.3:0/4293625912 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9bc133e30 0x7fd9bc09c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.625+0000 7fd9c22cd700 1 -- 172.21.15.3:0/4293625912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b8009550 con 0x7fd9bc133420 2023-10-03T10:17:45.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.625+0000 7fd9c22cd700 1 --2- 172.21.15.3:0/4293625912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9bc133420 0x7fd9bc09c480 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800ec10 tx=0x7fd9b800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:45.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.625+0000 7fd9b2ffd700 1 -- 172.21.15.3:0/4293625912 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b800c850 con 0x7fd9bc133420 2023-10-03T10:17:45.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.626+0000 7fd9b2ffd700 1 -- 172.21.15.3:0/4293625912 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b800c9b0 con 0x7fd9bc133420 2023-10-03T10:17:45.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.626+0000 7fd9b2ffd700 1 -- 172.21.15.3:0/4293625912 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b800cd40 con 0x7fd9bc133420 2023-10-03T10:17:45.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.626+0000 7fd9c4531700 1 -- 172.21.15.3:0/4293625912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b4009a70 con 0x7fd9bc133420 2023-10-03T10:17:45.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.626+0000 7fd9c4531700 1 -- 172.21.15.3:0/4293625912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9bc09df80 con 0x7fd9bc133420 2023-10-03T10:17:45.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.627+0000 7fd9b2ffd700 1 -- 172.21.15.3:0/4293625912 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd9b8013470 con 0x7fd9bc133420 2023-10-03T10:17:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.627+0000 7fd9c4531700 1 -- 172.21.15.3:0/4293625912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9bc12dc40 con 0x7fd9bc133420 2023-10-03T10:17:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.627+0000 7fd9b2ffd700 1 -- 172.21.15.3:0/4293625912 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd9b803f900 con 0x7fd9bc133420 2023-10-03T10:17:45.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.636+0000 7fd9b2ffd700 1 -- 172.21.15.3:0/4293625912 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd9b8013720 con 0x7fd9bc133420 2023-10-03T10:17:45.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.648+0000 7ff162b73700 1 Processor -- start 2023-10-03T10:17:45.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.648+0000 7ff162b73700 1 -- start start 2023-10-03T10:17:45.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.649+0000 7ff162b73700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff15c09d630 0x7ff15c0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:45.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.649+0000 7ff162b73700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff15c138d80 0x7ff15c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:45.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.649+0000 7ff162b73700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff15c0a2a70 con 0x7ff15c0a0a30 2023-10-03T10:17:45.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.649+0000 7ff162b73700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c0a01d0 con 0x7ff15c09d630 2023-10-03T10:17:45.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.649+0000 7ff162b73700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c09e2b0 con 0x7ff15c138d80 2023-10-03T10:17:45.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.649+0000 7ff15bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff15c09d630 0x7ff15c0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:45.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.649+0000 7ff15bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff15c09d630 0x7ff15c0a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60360/0 (socket says 172.21.15.3:60360) 2023-10-03T10:17:45.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.649+0000 7ff15bfff700 1 -- 172.21.15.3:0/2201114618 learned_addr learned my addr 172.21.15.3:0/2201114618 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:45.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.649+0000 7ff161110700 1 --2- 172.21.15.3:0/2201114618 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff15c138d80 0x7ff15c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:45.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.649+0000 7ff15b7fe700 1 -- 172.21.15.3:0/2201114618 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3337726365 0 0) 0x7ff15c0a2a70 con 0x7ff15c0a0a30 2023-10-03T10:17:45.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.649+0000 7ff15b7fe700 1 -- 172.21.15.3:0/2201114618 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1440035b0 con 0x7ff15c0a0a30 2023-10-03T10:17:45.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.649+0000 7ff15bfff700 1 -- 172.21.15.3:0/2201114618 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff15c138d80 msgr2=0x7ff15c13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:45.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.649+0000 7ff15bfff700 1 --2- 172.21.15.3:0/2201114618 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff15c138d80 0x7ff15c13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.649+0000 7ff15bfff700 1 -- 172.21.15.3:0/2201114618 >> v1:172.21.15.3:6789/0 conn(0x7ff15c0a0a30 legacy=0x7ff15c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:45.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.650+0000 7ff15bfff700 1 -- 172.21.15.3:0/2201114618 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15c09dc50 con 0x7ff15c09d630 2023-10-03T10:17:45.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.650+0000 7ff15bfff700 1 --2- 172.21.15.3:0/2201114618 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff15c09d630 0x7ff15c0a1510 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff150009920 tx=0x7ff150009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14487facf64ac577 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:45.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.650+0000 7ff15b7fe700 1 -- 172.21.15.3:0/2201114618 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff150011830 con 0x7ff15c09d630 2023-10-03T10:17:45.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.650+0000 7ff15b7fe700 1 -- 172.21.15.3:0/2201114618 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff150011990 con 0x7ff15c09d630 2023-10-03T10:17:45.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.650+0000 7ff15b7fe700 1 -- 172.21.15.3:0/2201114618 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff150011d20 con 0x7ff15c09d630 2023-10-03T10:17:45.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.651+0000 7ff162b73700 1 -- 172.21.15.3:0/2201114618 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff15c09d630 msgr2=0x7ff15c0a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:45.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.651+0000 7ff162b73700 1 --2- 172.21.15.3:0/2201114618 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff15c09d630 0x7ff15c0a1510 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff150009920 tx=0x7ff150009c30 comp rx=0 tx=0).stop 2023-10-03T10:17:45.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.651+0000 7ff162b73700 1 -- 172.21.15.3:0/2201114618 shutdown_connections 2023-10-03T10:17:45.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.651+0000 7ff162b73700 1 --2- 172.21.15.3:0/2201114618 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff15c138d80 0x7ff15c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.651+0000 7ff162b73700 1 --2- 172.21.15.3:0/2201114618 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff15c09d630 0x7ff15c0a1510 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.651+0000 7ff162b73700 1 -- 172.21.15.3:0/2201114618 >> 172.21.15.3:0/2201114618 conn(0x7ff15c0a3050 msgr2=0x7ff15c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:45.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.651+0000 7ff162b73700 1 -- 172.21.15.3:0/2201114618 shutdown_connections 2023-10-03T10:17:45.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.651+0000 7ff162b73700 1 -- 172.21.15.3:0/2201114618 wait complete. 2023-10-03T10:17:45.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.652+0000 7ff162b73700 1 Processor -- start 2023-10-03T10:17:45.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.652+0000 7ff162b73700 1 -- start start 2023-10-03T10:17:45.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.653+0000 7ff162b73700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff15c09d630 0x7ff15c1337e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.653+0000 7ff162b73700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff15c0a0a30 0x7ff15c133d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.653+0000 7ff162b73700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff15c138d80 0x7ff15c134260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.653+0000 7ff162b73700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c134cb0 con 0x7ff15c0a0a30 2023-10-03T10:17:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.653+0000 7ff162b73700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c136f40 con 0x7ff15c138d80 2023-10-03T10:17:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.653+0000 7ff162b73700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c136220 con 0x7ff15c09d630 2023-10-03T10:17:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.653+0000 7ff16090f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff15c09d630 0x7ff15c1337e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.653+0000 7ff161110700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff15c138d80 0x7ff15c134260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.653+0000 7ff16090f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff15c09d630 0x7ff15c1337e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43188/0 (socket says 172.21.15.3:43188) 2023-10-03T10:17:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.653+0000 7ff16090f700 1 -- 172.21.15.3:0/856765555 learned_addr learned my addr 172.21.15.3:0/856765555 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.653+0000 7ff15bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff15c0a0a30 0x7ff15c133d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.653+0000 7ff161110700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff15c138d80 0x7ff15c134260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60374/0 (socket says 172.21.15.3:60374) 2023-10-03T10:17:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.653+0000 7ff16090f700 1 -- 172.21.15.3:0/856765555 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff15c138d80 msgr2=0x7ff15c134260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.653+0000 7ff16090f700 1 --2- 172.21.15.3:0/856765555 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff15c138d80 0x7ff15c134260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.653+0000 7ff16090f700 1 -- 172.21.15.3:0/856765555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff15c0a0a30 msgr2=0x7ff15c133d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.653+0000 7ff16090f700 1 --2- 172.21.15.3:0/856765555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff15c0a0a30 0x7ff15c133d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.653+0000 7ff16090f700 1 -- 172.21.15.3:0/856765555 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff150009550 con 0x7ff15c09d630 2023-10-03T10:17:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.653+0000 7ff15bfff700 1 --2- 172.21.15.3:0/856765555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff15c0a0a30 0x7ff15c133d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.653+0000 7ff16090f700 1 --2- 172.21.15.3:0/856765555 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff15c09d630 0x7ff15c1337e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00c520 tx=0x7ff14c00c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.654+0000 7ff1597fa700 1 -- 172.21.15.3:0/856765555 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c0052d0 con 0x7ff15c09d630 2023-10-03T10:17:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.654+0000 7ff1597fa700 1 -- 172.21.15.3:0/856765555 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff14c005430 con 0x7ff15c09d630 2023-10-03T10:17:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.654+0000 7ff162b73700 1 -- 172.21.15.3:0/856765555 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff15c1365d0 con 0x7ff15c09d630 2023-10-03T10:17:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.654+0000 7ff1597fa700 1 -- 172.21.15.3:0/856765555 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c0038c0 con 0x7ff15c09d630 2023-10-03T10:17:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.654+0000 7ff162b73700 1 -- 172.21.15.3:0/856765555 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff15c0a4c60 con 0x7ff15c09d630 2023-10-03T10:17:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.655+0000 7ff1597fa700 1 -- 172.21.15.3:0/856765555 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff14c003a20 con 0x7ff15c09d630 2023-10-03T10:17:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.655+0000 7ff162b73700 1 -- 172.21.15.3:0/856765555 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff15c0747f0 con 0x7ff15c09d630 2023-10-03T10:17:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.655+0000 7ff1597fa700 1 -- 172.21.15.3:0/856765555 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff14c03eb40 con 0x7ff15c09d630 2023-10-03T10:17:45.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.660+0000 7ff1597fa700 1 -- 172.21.15.3:0/856765555 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff14c00f260 con 0x7ff15c09d630 2023-10-03T10:17:45.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.808+0000 7f1a44e18700 1 -- 172.21.15.3:0/1974081924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1a4006c360 con 0x7f1a4009e000 2023-10-03T10:17:45.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.808+0000 7f1a2effd700 1 -- 172.21.15.3:0/1974081924 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1a34034070 con 0x7f1a4009e000 2023-10-03T10:17:45.809 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:45.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.811+0000 7f1a44e18700 1 -- 172.21.15.3:0/1974081924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a4009e000 msgr2=0x7f1a401a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:45.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.811+0000 7f1a44e18700 1 --2- 172.21.15.3:0/1974081924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a4009e000 0x7f1a401a6430 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f1a34000c00 tx=0x7f1a34004750 comp rx=0 tx=0).stop 2023-10-03T10:17:45.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.811+0000 7f1a44e18700 1 -- 172.21.15.3:0/1974081924 shutdown_connections 2023-10-03T10:17:45.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.811+0000 7f1a44e18700 1 --2- 172.21.15.3:0/1974081924 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1a4009e410 0x7f1a401a6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.811+0000 7f1a44e18700 1 --2- 172.21.15.3:0/1974081924 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a4009d6f0 0x7f1a401a5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.811+0000 7f1a44e18700 1 --2- 172.21.15.3:0/1974081924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a4009e000 0x7f1a401a6430 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.811+0000 7f1a44e18700 1 -- 172.21.15.3:0/1974081924 >> 172.21.15.3:0/1974081924 conn(0x7f1a40066f50 msgr2=0x7f1a401badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:45.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.811+0000 7f1a44e18700 1 -- 172.21.15.3:0/1974081924 shutdown_connections 2023-10-03T10:17:45.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.811+0000 7f1a44e18700 1 -- 172.21.15.3:0/1974081924 wait complete. 2023-10-03T10:17:45.825 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:45.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.849+0000 7fd9c4531700 1 -- 172.21.15.3:0/4293625912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd9bc120070 con 0x7fd9bc133420 2023-10-03T10:17:45.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.849+0000 7fd9b2ffd700 1 -- 172.21.15.3:0/4293625912 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd9b801bb60 con 0x7fd9bc133420 2023-10-03T10:17:45.849 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:45.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.851+0000 7fd9c4531700 1 -- 172.21.15.3:0/4293625912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9bc133420 msgr2=0x7fd9bc09c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:45.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.851+0000 7fd9c4531700 1 --2- 172.21.15.3:0/4293625912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9bc133420 0x7fd9bc09c480 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800ec10 tx=0x7fd9b800ef20 comp rx=0 tx=0).stop 2023-10-03T10:17:45.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.851+0000 7fd9c4531700 1 -- 172.21.15.3:0/4293625912 shutdown_connections 2023-10-03T10:17:45.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.851+0000 7fd9c4531700 1 --2- 172.21.15.3:0/4293625912 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd9bc134260 0x7fd9bc09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.851+0000 7fd9c4531700 1 --2- 172.21.15.3:0/4293625912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9bc133420 0x7fd9bc09c480 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.851+0000 7fd9c4531700 1 --2- 172.21.15.3:0/4293625912 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9bc133e30 0x7fd9bc09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.851+0000 7fd9c4531700 1 -- 172.21.15.3:0/4293625912 >> 172.21.15.3:0/4293625912 conn(0x7fd9bc0a4210 msgr2=0x7fd9bc12de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:45.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.851+0000 7fd9c4531700 1 -- 172.21.15.3:0/4293625912 shutdown_connections 2023-10-03T10:17:45.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.851+0000 7fd9c4531700 1 -- 172.21.15.3:0/4293625912 wait complete. 2023-10-03T10:17:45.866 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:45.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.869+0000 7ff162b73700 1 -- 172.21.15.3:0/856765555 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff15c137a10 con 0x7ff15c09d630 2023-10-03T10:17:45.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.870+0000 7ff1597fa700 1 -- 172.21.15.3:0/856765555 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff14c017020 con 0x7ff15c09d630 2023-10-03T10:17:45.870 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:45.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.872+0000 7ff162b73700 1 -- 172.21.15.3:0/856765555 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff15c09d630 msgr2=0x7ff15c1337e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:45.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.872+0000 7ff162b73700 1 --2- 172.21.15.3:0/856765555 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff15c09d630 0x7ff15c1337e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00c520 tx=0x7ff14c00c830 comp rx=0 tx=0).stop 2023-10-03T10:17:45.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.872+0000 7ff162b73700 1 -- 172.21.15.3:0/856765555 shutdown_connections 2023-10-03T10:17:45.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.872+0000 7ff162b73700 1 --2- 172.21.15.3:0/856765555 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff15c138d80 0x7ff15c134260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.872+0000 7ff162b73700 1 --2- 172.21.15.3:0/856765555 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff15c09d630 0x7ff15c1337e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.872+0000 7ff162b73700 1 --2- 172.21.15.3:0/856765555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff15c0a0a30 0x7ff15c133d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:45.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.872+0000 7ff162b73700 1 -- 172.21.15.3:0/856765555 >> 172.21.15.3:0/856765555 conn(0x7ff15c0a3050 msgr2=0x7ff15c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:45.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.872+0000 7ff162b73700 1 -- 172.21.15.3:0/856765555 shutdown_connections 2023-10-03T10:17:45.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:45.872+0000 7ff162b73700 1 -- 172.21.15.3:0/856765555 wait complete. 2023-10-03T10:17:45.886 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:46.083 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:46.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.226+0000 7fa26b10c700 1 Processor -- start 2023-10-03T10:17:46.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.227+0000 7fa26b10c700 1 -- start start 2023-10-03T10:17:46.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.227+0000 7fa26b10c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa26406a770 0x7fa26406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:46.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.227+0000 7fa26b10c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa26406b080 0x7fa264131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:46.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.227+0000 7fa26b10c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa264131b40 0x7fa264131fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:46.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.227+0000 7fa26b10c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa26409ece0 con 0x7fa26406a770 2023-10-03T10:17:46.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.227+0000 7fa26b10c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa26409e760 con 0x7fa264131b40 2023-10-03T10:17:46.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.227+0000 7fa26b10c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa26409e040 con 0x7fa26406b080 2023-10-03T10:17:46.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.227+0000 7fa268ea8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa26406a770 0x7fa26406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:46.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.227+0000 7fa2696a9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa264131b40 0x7fa264131fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:46.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.227+0000 7fa268ea8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa26406a770 0x7fa26406ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58970/0 (socket says 172.21.15.3:58970) 2023-10-03T10:17:46.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.227+0000 7fa268ea8700 1 -- 172.21.15.3:0/1534309154 learned_addr learned my addr 172.21.15.3:0/1534309154 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:46.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.227+0000 7fa263fff700 1 --2- 172.21.15.3:0/1534309154 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa26406b080 0x7fa264131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:46.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.228+0000 7fa268ea8700 1 -- 172.21.15.3:0/1534309154 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa26406b080 msgr2=0x7fa264131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:46.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.228+0000 7fa268ea8700 1 --2- 172.21.15.3:0/1534309154 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa26406b080 0x7fa264131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:46.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.228+0000 7fa268ea8700 1 -- 172.21.15.3:0/1534309154 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa264131b40 msgr2=0x7fa264131fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:46.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.228+0000 7fa268ea8700 1 --2- 172.21.15.3:0/1534309154 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa264131b40 0x7fa264131fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:46.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.228+0000 7fa268ea8700 1 -- 172.21.15.3:0/1534309154 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa26409f630 con 0x7fa26406a770 2023-10-03T10:17:46.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.228+0000 7fa2696a9700 1 --2- 172.21.15.3:0/1534309154 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa264131b40 0x7fa264131fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:46.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.228+0000 7fa263fff700 1 --2- 172.21.15.3:0/1534309154 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa26406b080 0x7fa264131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:46.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.228+0000 7fa268ea8700 1 --2- 172.21.15.3:0/1534309154 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa26406a770 0x7fa26406ab40 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fa2540098f0 tx=0x7fa254009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44689f1dcfab63cb server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:46.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.228+0000 7fa2637fe700 1 -- 172.21.15.3:0/1534309154 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254011830 con 0x7fa26406a770 2023-10-03T10:17:46.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.228+0000 7fa2637fe700 1 -- 172.21.15.3:0/1534309154 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa254011990 con 0x7fa26406a770 2023-10-03T10:17:46.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.228+0000 7fa2637fe700 1 -- 172.21.15.3:0/1534309154 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254011d20 con 0x7fa26406a770 2023-10-03T10:17:46.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.229+0000 7fa26b10c700 1 -- 172.21.15.3:0/1534309154 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa26406a770 msgr2=0x7fa26406ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:46.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.229+0000 7fa26b10c700 1 --2- 172.21.15.3:0/1534309154 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa26406a770 0x7fa26406ab40 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fa2540098f0 tx=0x7fa254009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:46.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.229+0000 7fa26b10c700 1 -- 172.21.15.3:0/1534309154 shutdown_connections 2023-10-03T10:17:46.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.229+0000 7fa26b10c700 1 --2- 172.21.15.3:0/1534309154 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa264131b40 0x7fa264131fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:46.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.229+0000 7fa26b10c700 1 --2- 172.21.15.3:0/1534309154 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa26406a770 0x7fa26406ab40 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:46.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.229+0000 7fa26b10c700 1 --2- 172.21.15.3:0/1534309154 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa26406b080 0x7fa264131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:46.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.229+0000 7fa26b10c700 1 -- 172.21.15.3:0/1534309154 >> 172.21.15.3:0/1534309154 conn(0x7fa264066f50 msgr2=0x7fa26409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:46.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.229+0000 7fa26b10c700 1 -- 172.21.15.3:0/1534309154 shutdown_connections 2023-10-03T10:17:46.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.229+0000 7fa26b10c700 1 -- 172.21.15.3:0/1534309154 wait complete. 2023-10-03T10:17:46.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.230+0000 7fa26b10c700 1 Processor -- start 2023-10-03T10:17:46.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.230+0000 7fa26b10c700 1 -- start start 2023-10-03T10:17:46.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.230+0000 7fa26b10c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa26406a770 0x7fa26411c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:46.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.230+0000 7fa26b10c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa26406b080 0x7fa26411c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:46.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.231+0000 7fa26b10c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa264131b40 0x7fa26411cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:46.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.231+0000 7fa26b10c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26411fdb0 con 0x7fa26406b080 2023-10-03T10:17:46.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.231+0000 7fa26b10c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa264131530 con 0x7fa264131b40 2023-10-03T10:17:46.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.231+0000 7fa26b10c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa26411e650 con 0x7fa26406a770 2023-10-03T10:17:46.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.231+0000 7fa268ea8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa26406a770 0x7fa26411c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:46.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.231+0000 7fa2696a9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa264131b40 0x7fa26411cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:46.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.231+0000 7fa268ea8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa26406a770 0x7fa26411c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43198/0 (socket says 172.21.15.3:43198) 2023-10-03T10:17:46.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.231+0000 7fa263fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa26406b080 0x7fa26411c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:46.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.231+0000 7fa268ea8700 1 -- 172.21.15.3:0/3981523136 learned_addr learned my addr 172.21.15.3:0/3981523136 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:46.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.231+0000 7fa268ea8700 1 -- 172.21.15.3:0/3981523136 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa264131b40 msgr2=0x7fa26411cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:46.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.231+0000 7fa268ea8700 1 --2- 172.21.15.3:0/3981523136 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa264131b40 0x7fa26411cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:46.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.231+0000 7fa268ea8700 1 -- 172.21.15.3:0/3981523136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa26406b080 msgr2=0x7fa26411c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:46.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.231+0000 7fa268ea8700 1 --2- 172.21.15.3:0/3981523136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa26406b080 0x7fa26411c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:46.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.231+0000 7fa268ea8700 1 -- 172.21.15.3:0/3981523136 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2580094a0 con 0x7fa26406a770 2023-10-03T10:17:46.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.231+0000 7fa2696a9700 1 --2- 172.21.15.3:0/3981523136 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa264131b40 0x7fa26411cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:46.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.231+0000 7fa263fff700 1 --2- 172.21.15.3:0/3981523136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa26406b080 0x7fa26411c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:46.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.231+0000 7fa268ea8700 1 --2- 172.21.15.3:0/3981523136 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa26406a770 0x7fa26411c490 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa254000c00 tx=0x7fa254004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:46.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.231+0000 7fa2617fa700 1 -- 172.21.15.3:0/3981523136 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254004240 con 0x7fa26406a770 2023-10-03T10:17:46.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.231+0000 7fa2617fa700 1 -- 172.21.15.3:0/3981523136 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25401f070 con 0x7fa26406a770 2023-10-03T10:17:46.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.231+0000 7fa26b10c700 1 -- 172.21.15.3:0/3981523136 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa25c009a70 con 0x7fa26406a770 2023-10-03T10:17:46.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.231+0000 7fa2617fa700 1 -- 172.21.15.3:0/3981523136 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2540045c0 con 0x7fa26406a770 2023-10-03T10:17:46.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.232+0000 7fa26b10c700 1 -- 172.21.15.3:0/3981523136 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa254009550 con 0x7fa26406a770 2023-10-03T10:17:46.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.232+0000 7fa2617fa700 1 -- 172.21.15.3:0/3981523136 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa254021030 con 0x7fa26406a770 2023-10-03T10:17:46.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.232+0000 7fa26b10c700 1 -- 172.21.15.3:0/3981523136 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa264074ea0 con 0x7fa26406a770 2023-10-03T10:17:46.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.232+0000 7fa2617fa700 1 -- 172.21.15.3:0/3981523136 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa2540408f0 con 0x7fa26406a770 2023-10-03T10:17:46.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.237+0000 7fa2617fa700 1 -- 172.21.15.3:0/3981523136 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa254015070 con 0x7fa26406a770 2023-10-03T10:17:46.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.443+0000 7fa26b10c700 1 -- 172.21.15.3:0/3981523136 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa26406c360 con 0x7fa26406a770 2023-10-03T10:17:46.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.443+0000 7fa2617fa700 1 -- 172.21.15.3:0/3981523136 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa254015070 con 0x7fa26406a770 2023-10-03T10:17:46.443 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:46.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.445+0000 7fa26b10c700 1 -- 172.21.15.3:0/3981523136 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa26406a770 msgr2=0x7fa26411c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:46.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.445+0000 7fa26b10c700 1 --2- 172.21.15.3:0/3981523136 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa26406a770 0x7fa26411c490 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa254000c00 tx=0x7fa254004750 comp rx=0 tx=0).stop 2023-10-03T10:17:46.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.445+0000 7fa26b10c700 1 -- 172.21.15.3:0/3981523136 shutdown_connections 2023-10-03T10:17:46.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.445+0000 7fa26b10c700 1 --2- 172.21.15.3:0/3981523136 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa264131b40 0x7fa26411cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:46.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.445+0000 7fa26b10c700 1 --2- 172.21.15.3:0/3981523136 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa26406a770 0x7fa26411c490 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:46.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.445+0000 7fa26b10c700 1 --2- 172.21.15.3:0/3981523136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa26406b080 0x7fa26411c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:46.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.445+0000 7fa26b10c700 1 -- 172.21.15.3:0/3981523136 >> 172.21.15.3:0/3981523136 conn(0x7fa264066f50 msgr2=0x7fa264070110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:46.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.445+0000 7fa26b10c700 1 -- 172.21.15.3:0/3981523136 shutdown_connections 2023-10-03T10:17:46.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.446+0000 7fa26b10c700 1 -- 172.21.15.3:0/3981523136 wait complete. 2023-10-03T10:17:46.459 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:46.826 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:46.867 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:46.887 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:46.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.975+0000 7f2108072700 1 Processor -- start 2023-10-03T10:17:46.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.975+0000 7f2108072700 1 -- start start 2023-10-03T10:17:46.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.976+0000 7f2108072700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210006a770 0x7f210006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:46.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.976+0000 7f2108072700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2100131d10 0x7f2100139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:46.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.976+0000 7f2108072700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f210009ece0 con 0x7f210006b080 2023-10-03T10:17:46.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.976+0000 7f2108072700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210009e760 con 0x7f210006a770 2023-10-03T10:17:46.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.976+0000 7f2108072700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f210009e040 con 0x7f2100131d10 2023-10-03T10:17:46.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.976+0000 7f2105e0e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210006a770 0x7f210006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:46.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.976+0000 7f2105e0e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210006a770 0x7f210006ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60396/0 (socket says 172.21.15.3:60396) 2023-10-03T10:17:46.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.976+0000 7f210560d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f210006b080 0x7f2100131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42446/0 (socket says 172.21.15.3:42446) 2023-10-03T10:17:46.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.976+0000 7f2105e0e700 1 -- 172.21.15.3:0/3849862733 learned_addr learned my addr 172.21.15.3:0/3849862733 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:46.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.976+0000 7f210660f700 1 --2- 172.21.15.3:0/3849862733 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2100131d10 0x7f2100139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:46.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.977+0000 7f210660f700 1 -- 172.21.15.3:0/3849862733 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210006a770 msgr2=0x7f210006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:46.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.977+0000 7f210660f700 1 --2- 172.21.15.3:0/3849862733 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210006a770 0x7f210006ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:46.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.977+0000 7f210660f700 1 -- 172.21.15.3:0/3849862733 >> v1:172.21.15.3:6789/0 conn(0x7f210006b080 legacy=0x7f2100131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:46.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.977+0000 7f2104e0c700 1 -- 172.21.15.3:0/3849862733 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1426511293 0 0) 0x7f210009ece0 con 0x7f210006b080 2023-10-03T10:17:46.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.977+0000 7f210660f700 1 -- 172.21.15.3:0/3849862733 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210009f630 con 0x7f2100131d10 2023-10-03T10:17:46.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.977+0000 7f2105e0e700 1 --2- 172.21.15.3:0/3849862733 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210006a770 0x7f210006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:46.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.977+0000 7f210660f700 1 --2- 172.21.15.3:0/3849862733 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2100131d10 0x7f2100139860 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009dc0 tx=0x7f20f800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d4147595c62d2a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:46.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.977+0000 7f2104e0c700 1 -- 172.21.15.3:0/3849862733 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f800e070 con 0x7f2100131d10 2023-10-03T10:17:46.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.977+0000 7f2104e0c700 1 -- 172.21.15.3:0/3849862733 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f800bb20 con 0x7f2100131d10 2023-10-03T10:17:46.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.977+0000 7f2104e0c700 1 -- 172.21.15.3:0/3849862733 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f8012430 con 0x7f2100131d10 2023-10-03T10:17:46.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.978+0000 7f2108072700 1 -- 172.21.15.3:0/3849862733 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2100131d10 msgr2=0x7f2100139860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:46.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.978+0000 7f2108072700 1 --2- 172.21.15.3:0/3849862733 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2100131d10 0x7f2100139860 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009dc0 tx=0x7f20f800b330 comp rx=0 tx=0).stop 2023-10-03T10:17:46.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.978+0000 7f2108072700 1 -- 172.21.15.3:0/3849862733 shutdown_connections 2023-10-03T10:17:46.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.978+0000 7f2108072700 1 --2- 172.21.15.3:0/3849862733 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210006a770 0x7f210006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:46.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.978+0000 7f2108072700 1 --2- 172.21.15.3:0/3849862733 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2100131d10 0x7f2100139860 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:46.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.978+0000 7f2108072700 1 -- 172.21.15.3:0/3849862733 >> 172.21.15.3:0/3849862733 conn(0x7f2100066f50 msgr2=0x7f210009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:46.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.978+0000 7f2108072700 1 -- 172.21.15.3:0/3849862733 shutdown_connections 2023-10-03T10:17:46.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.978+0000 7f2108072700 1 -- 172.21.15.3:0/3849862733 wait complete. 2023-10-03T10:17:46.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.979+0000 7f2108072700 1 Processor -- start 2023-10-03T10:17:46.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.979+0000 7f2108072700 1 -- start start 2023-10-03T10:17:46.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.980+0000 7f2108072700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210006a770 0x7f21001203f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:46.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.980+0000 7f2108072700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210006b080 0x7f2100120930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:46.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.980+0000 7f2108072700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2100131d10 0x7f2100120e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:46.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.980+0000 7f2108072700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2100123cc0 con 0x7f2100131d10 2023-10-03T10:17:46.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.980+0000 7f2108072700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2100123ec0 con 0x7f210006a770 2023-10-03T10:17:46.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.980+0000 7f2108072700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2100130ea0 con 0x7f210006b080 2023-10-03T10:17:46.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.980+0000 7f2105e0e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210006a770 0x7f21001203f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:46.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.980+0000 7f2105e0e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210006a770 0x7f21001203f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60404/0 (socket says 172.21.15.3:60404) 2023-10-03T10:17:46.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.980+0000 7f2105e0e700 1 -- 172.21.15.3:0/995267706 learned_addr learned my addr 172.21.15.3:0/995267706 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:46.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.980+0000 7f210560d700 1 --2- 172.21.15.3:0/995267706 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210006b080 0x7f2100120930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:46.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.980+0000 7f210660f700 1 --2- 172.21.15.3:0/995267706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2100131d10 0x7f2100120e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:46.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.981+0000 7f2105e0e700 1 -- 172.21.15.3:0/995267706 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210006b080 msgr2=0x7f2100120930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:46.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.981+0000 7f2105e0e700 1 --2- 172.21.15.3:0/995267706 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210006b080 0x7f2100120930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:46.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.981+0000 7f2105e0e700 1 -- 172.21.15.3:0/995267706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2100131d10 msgr2=0x7f2100120e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:46.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.981+0000 7f2105e0e700 1 --2- 172.21.15.3:0/995267706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2100131d10 0x7f2100120e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:46.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.981+0000 7f2105e0e700 1 -- 172.21.15.3:0/995267706 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20fc009550 con 0x7f210006a770 2023-10-03T10:17:46.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.981+0000 7f2105e0e700 1 --2- 172.21.15.3:0/995267706 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210006a770 0x7f21001203f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00ebe0 tx=0x7f20fc00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:46.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.981+0000 7f20f67fc700 1 -- 172.21.15.3:0/995267706 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc00c850 con 0x7f210006a770 2023-10-03T10:17:46.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.981+0000 7f20f67fc700 1 -- 172.21.15.3:0/995267706 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20fc00c9b0 con 0x7f210006a770 2023-10-03T10:17:46.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.981+0000 7f20f67fc700 1 -- 172.21.15.3:0/995267706 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc00cd20 con 0x7f210006a770 2023-10-03T10:17:46.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.981+0000 7f2108072700 1 -- 172.21.15.3:0/995267706 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f8009a70 con 0x7f210006a770 2023-10-03T10:17:46.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.981+0000 7f2108072700 1 -- 172.21.15.3:0/995267706 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f210006ed20 con 0x7f210006a770 2023-10-03T10:17:46.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.982+0000 7f20f67fc700 1 -- 172.21.15.3:0/995267706 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f20fc013470 con 0x7f210006a770 2023-10-03T10:17:46.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.982+0000 7f2108072700 1 -- 172.21.15.3:0/995267706 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2100074ea0 con 0x7f210006a770 2023-10-03T10:17:46.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.983+0000 7f20f67fc700 1 -- 172.21.15.3:0/995267706 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f20fc03f820 con 0x7f210006a770 2023-10-03T10:17:46.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:46.992+0000 7f20f67fc700 1 -- 172.21.15.3:0/995267706 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f20fc0232f0 con 0x7f210006a770 2023-10-03T10:17:47.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.013+0000 7f19ebba9700 1 Processor -- start 2023-10-03T10:17:47.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.014+0000 7f19ebba9700 1 -- start start 2023-10-03T10:17:47.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.014+0000 7f19ebba9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19e409de00 0x7f19e4138f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:47.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.014+0000 7f19ebba9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19e409e210 0x7f19e413b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:47.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.014+0000 7f19ebba9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e4099ef0 con 0x7f19e406c2f0 2023-10-03T10:17:47.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.014+0000 7f19ebba9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e406c6c0 con 0x7f19e409e210 2023-10-03T10:17:47.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.014+0000 7f19ebba9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f19e409bb60 con 0x7f19e409de00 2023-10-03T10:17:47.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.014+0000 7f19e9144700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19e409de00 0x7f19e4138f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:47.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.015+0000 7f19ea146700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19e409e210 0x7f19e413b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:47.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.015+0000 7f19e9945700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f19e406c2f0 0x7f19e409d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42462/0 (socket says 172.21.15.3:42462) 2023-10-03T10:17:47.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.015+0000 7f19e9945700 1 -- 172.21.15.3:0/3175253299 learned_addr learned my addr 172.21.15.3:0/3175253299 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.015+0000 7f19e8943700 1 -- 172.21.15.3:0/3175253299 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1819760776 0 0) 0x7f19e4099ef0 con 0x7f19e406c2f0 2023-10-03T10:17:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.015+0000 7f19e8943700 1 -- 172.21.15.3:0/3175253299 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19d00035b0 con 0x7f19e406c2f0 2023-10-03T10:17:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.015+0000 7f19e9144700 1 -- 172.21.15.3:0/3175253299 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19e409e210 msgr2=0x7f19e413b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.015+0000 7f19e9144700 1 --2- 172.21.15.3:0/3175253299 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19e409e210 0x7f19e413b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.015+0000 7f19e9144700 1 -- 172.21.15.3:0/3175253299 >> v1:172.21.15.3:6789/0 conn(0x7f19e406c2f0 legacy=0x7f19e409d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.015+0000 7f19e9144700 1 -- 172.21.15.3:0/3175253299 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e409aca0 con 0x7f19e409de00 2023-10-03T10:17:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.015+0000 7f19e9144700 1 --2- 172.21.15.3:0/3175253299 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19e409de00 0x7f19e4138f00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009870 tx=0x7f19d4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4016fbccc97fb6f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.015+0000 7f19e8943700 1 -- 172.21.15.3:0/3175253299 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d4011830 con 0x7f19e409de00 2023-10-03T10:17:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.015+0000 7f19e8943700 1 -- 172.21.15.3:0/3175253299 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d4011990 con 0x7f19e409de00 2023-10-03T10:17:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.015+0000 7f19e8943700 1 -- 172.21.15.3:0/3175253299 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d4011d20 con 0x7f19e409de00 2023-10-03T10:17:47.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.016+0000 7f19ebba9700 1 -- 172.21.15.3:0/3175253299 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19e409de00 msgr2=0x7f19e4138f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:47.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.016+0000 7f19ebba9700 1 --2- 172.21.15.3:0/3175253299 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19e409de00 0x7f19e4138f00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009870 tx=0x7f19d4009b80 comp rx=0 tx=0).stop 2023-10-03T10:17:47.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.016+0000 7f19ebba9700 1 -- 172.21.15.3:0/3175253299 shutdown_connections 2023-10-03T10:17:47.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.016+0000 7f19ebba9700 1 --2- 172.21.15.3:0/3175253299 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19e409e210 0x7f19e413b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.016+0000 7f19ebba9700 1 --2- 172.21.15.3:0/3175253299 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19e409de00 0x7f19e4138f00 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.016+0000 7f19ebba9700 1 -- 172.21.15.3:0/3175253299 >> 172.21.15.3:0/3175253299 conn(0x7f19e409f470 msgr2=0x7f19e40a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:47.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.016+0000 7f19ebba9700 1 -- 172.21.15.3:0/3175253299 shutdown_connections 2023-10-03T10:17:47.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.016+0000 7f19ebba9700 1 -- 172.21.15.3:0/3175253299 wait complete. 2023-10-03T10:17:47.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.018+0000 7f19ebba9700 1 Processor -- start 2023-10-03T10:17:47.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.018+0000 7f19ebba9700 1 -- start start 2023-10-03T10:17:47.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.019+0000 7f19ebba9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19e406c2f0 0x7f19e41c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:47.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.019+0000 7f19ebba9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19e409de00 0x7f19e41c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:47.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.019+0000 7f19ebba9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19e409e210 0x7f19e41c48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:47.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.019+0000 7f19ebba9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e4097c30 con 0x7f19e409de00 2023-10-03T10:17:47.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.019+0000 7f19ebba9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e409a0c0 con 0x7f19e406c2f0 2023-10-03T10:17:47.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.019+0000 7f19ebba9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f19e4097110 con 0x7f19e409e210 2023-10-03T10:17:47.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.019+0000 7f19e9144700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19e409de00 0x7f19e41c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:47.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.019+0000 7f19e9945700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19e406c2f0 0x7f19e41c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:47.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.019+0000 7f19e9144700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19e409de00 0x7f19e41c8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58986/0 (socket says 172.21.15.3:58986) 2023-10-03T10:17:47.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.019+0000 7f19e9144700 1 -- 172.21.15.3:0/4191999797 learned_addr learned my addr 172.21.15.3:0/4191999797 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:47.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.019+0000 7f19ea146700 1 --2- 172.21.15.3:0/4191999797 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19e409e210 0x7f19e41c48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:47.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.019+0000 7f19e9144700 1 -- 172.21.15.3:0/4191999797 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19e409e210 msgr2=0x7f19e41c48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:47.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.019+0000 7f19e9144700 1 --2- 172.21.15.3:0/4191999797 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19e409e210 0x7f19e41c48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.019+0000 7f19e9144700 1 -- 172.21.15.3:0/4191999797 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19e406c2f0 msgr2=0x7f19e41c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:47.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.019+0000 7f19e9144700 1 --2- 172.21.15.3:0/4191999797 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19e406c2f0 0x7f19e41c8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.019+0000 7f19e9144700 1 -- 172.21.15.3:0/4191999797 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d40094a0 con 0x7f19e409de00 2023-10-03T10:17:47.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.020+0000 7f19e9144700 1 --2- 172.21.15.3:0/4191999797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19e409de00 0x7f19e41c8950 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f19d40090b0 tx=0x7f19d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:47.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.020+0000 7f19da7fc700 1 -- 172.21.15.3:0/4191999797 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d4004450 con 0x7f19e409de00 2023-10-03T10:17:47.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.020+0000 7f19ebba9700 1 -- 172.21.15.3:0/4191999797 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e40962c0 con 0x7f19e409de00 2023-10-03T10:17:47.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.020+0000 7f19da7fc700 1 -- 172.21.15.3:0/4191999797 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d401f070 con 0x7f19e409de00 2023-10-03T10:17:47.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.020+0000 7f19da7fc700 1 -- 172.21.15.3:0/4191999797 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d40038e0 con 0x7f19e409de00 2023-10-03T10:17:47.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.020+0000 7f19ebba9700 1 -- 172.21.15.3:0/4191999797 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e41abf50 con 0x7f19e409de00 2023-10-03T10:17:47.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.021+0000 7f19da7fc700 1 -- 172.21.15.3:0/4191999797 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f19d4003a40 con 0x7f19e409de00 2023-10-03T10:17:47.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.021+0000 7f19ebba9700 1 -- 172.21.15.3:0/4191999797 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e40747f0 con 0x7f19e409de00 2023-10-03T10:17:47.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.021+0000 7f19da7fc700 1 -- 172.21.15.3:0/4191999797 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f19d4021070 con 0x7f19e409de00 2023-10-03T10:17:47.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.030+0000 7f19da7fc700 1 -- 172.21.15.3:0/4191999797 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f19d40253e0 con 0x7f19e409de00 2023-10-03T10:17:47.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.063+0000 7fc95042b700 1 Processor -- start 2023-10-03T10:17:47.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.063+0000 7fc95042b700 1 -- start start 2023-10-03T10:17:47.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.064+0000 7fc95042b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc9480a0a30 0x7fc9480a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:47.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.064+0000 7fc95042b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9480a1340 0x7fc948133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:47.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.064+0000 7fc95042b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc94809edd0 con 0x7fc948133fa0 2023-10-03T10:17:47.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.064+0000 7fc95042b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc94809e1f0 con 0x7fc9480a0a30 2023-10-03T10:17:47.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.064+0000 7fc95042b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc94809dc50 con 0x7fc9480a1340 2023-10-03T10:17:47.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.064+0000 7fc94d9c6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9480a1340 0x7fc948133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:47.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.064+0000 7fc94e9c8700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc948133fa0 0x7fc94813ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42472/0 (socket says 172.21.15.3:42472) 2023-10-03T10:17:47.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.064+0000 7fc94e9c8700 1 -- 172.21.15.3:0/2534504262 learned_addr learned my addr 172.21.15.3:0/2534504262 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:47.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.064+0000 7fc94e1c7700 1 --2- 172.21.15.3:0/2534504262 >> v2:172.21.15.3:3300/0 conn(0x7fc9480a0a30 0x7fc9480a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:47.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.064+0000 7fc94d1c5700 1 -- 172.21.15.3:0/2534504262 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3538739522 0 0) 0x7fc94809edd0 con 0x7fc948133fa0 2023-10-03T10:17:47.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.064+0000 7fc94d1c5700 1 -- 172.21.15.3:0/2534504262 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9340035b0 con 0x7fc948133fa0 2023-10-03T10:17:47.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.064+0000 7fc94d9c6700 1 -- 172.21.15.3:0/2534504262 >> v2:172.21.15.3:3300/0 conn(0x7fc9480a0a30 msgr2=0x7fc9480a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:47.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.064+0000 7fc94d9c6700 1 --2- 172.21.15.3:0/2534504262 >> v2:172.21.15.3:3300/0 conn(0x7fc9480a0a30 0x7fc9480a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.064+0000 7fc94d9c6700 1 -- 172.21.15.3:0/2534504262 >> v1:172.21.15.3:6789/0 conn(0x7fc948133fa0 legacy=0x7fc94813ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:47.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.064+0000 7fc94d9c6700 1 -- 172.21.15.3:0/2534504262 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9480a1fe0 con 0x7fc9480a1340 2023-10-03T10:17:47.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.065+0000 7fc94e1c7700 1 --2- 172.21.15.3:0/2534504262 >> v2:172.21.15.3:3300/0 conn(0x7fc9480a0a30 0x7fc9480a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:47.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.065+0000 7fc94d9c6700 1 --2- 172.21.15.3:0/2534504262 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9480a1340 0x7fc948133a60 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc938009840 tx=0x7fc938009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2368865638f57e7a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:47.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.065+0000 7fc94d1c5700 1 -- 172.21.15.3:0/2534504262 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc938011830 con 0x7fc9480a1340 2023-10-03T10:17:47.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.065+0000 7fc94d1c5700 1 -- 172.21.15.3:0/2534504262 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc938011990 con 0x7fc9480a1340 2023-10-03T10:17:47.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.065+0000 7fc94d1c5700 1 -- 172.21.15.3:0/2534504262 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc938011d20 con 0x7fc9480a1340 2023-10-03T10:17:47.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.065+0000 7fc95042b700 1 -- 172.21.15.3:0/2534504262 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9480a1340 msgr2=0x7fc948133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:47.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.065+0000 7fc95042b700 1 --2- 172.21.15.3:0/2534504262 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9480a1340 0x7fc948133a60 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc938009840 tx=0x7fc938009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:47.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.066+0000 7fc95042b700 1 -- 172.21.15.3:0/2534504262 shutdown_connections 2023-10-03T10:17:47.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.066+0000 7fc95042b700 1 --2- 172.21.15.3:0/2534504262 >> v2:172.21.15.3:3300/0 conn(0x7fc9480a0a30 0x7fc9480a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.066+0000 7fc95042b700 1 --2- 172.21.15.3:0/2534504262 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9480a1340 0x7fc948133a60 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.066+0000 7fc95042b700 1 -- 172.21.15.3:0/2534504262 >> 172.21.15.3:0/2534504262 conn(0x7fc9480a6550 msgr2=0x7fc9480a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:47.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.066+0000 7fc95042b700 1 -- 172.21.15.3:0/2534504262 shutdown_connections 2023-10-03T10:17:47.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.066+0000 7fc95042b700 1 -- 172.21.15.3:0/2534504262 wait complete. 2023-10-03T10:17:47.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.067+0000 7fc95042b700 1 Processor -- start 2023-10-03T10:17:47.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.067+0000 7fc95042b700 1 -- start start 2023-10-03T10:17:47.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.067+0000 7fc95042b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9480a0a30 0x7fc948122950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:47.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.067+0000 7fc95042b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9480a1340 0x7fc948122e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:47.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.067+0000 7fc95042b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc948133fa0 0x7fc9481233d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:47.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.067+0000 7fc95042b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc948124610 con 0x7fc948133fa0 2023-10-03T10:17:47.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.067+0000 7fc95042b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc948124420 con 0x7fc9480a0a30 2023-10-03T10:17:47.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.067+0000 7fc95042b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9481263a0 con 0x7fc9480a1340 2023-10-03T10:17:47.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.067+0000 7fc94d9c6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9480a1340 0x7fc948122e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:47.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.067+0000 7fc94e1c7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9480a0a30 0x7fc948122950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:47.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.067+0000 7fc94e9c8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc948133fa0 0x7fc9481233d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:47.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.067+0000 7fc94e1c7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9480a0a30 0x7fc948122950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60434/0 (socket says 172.21.15.3:60434) 2023-10-03T10:17:47.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.067+0000 7fc94d9c6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9480a1340 0x7fc948122e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43242/0 (socket says 172.21.15.3:43242) 2023-10-03T10:17:47.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.067+0000 7fc94e1c7700 1 -- 172.21.15.3:0/2012496452 learned_addr learned my addr 172.21.15.3:0/2012496452 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:47.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.068+0000 7fc94e1c7700 1 -- 172.21.15.3:0/2012496452 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9480a1340 msgr2=0x7fc948122e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:47.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.068+0000 7fc94e1c7700 1 --2- 172.21.15.3:0/2012496452 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9480a1340 0x7fc948122e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.068+0000 7fc94e1c7700 1 -- 172.21.15.3:0/2012496452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc948133fa0 msgr2=0x7fc9481233d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:47.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.068+0000 7fc94e1c7700 1 --2- 172.21.15.3:0/2012496452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc948133fa0 0x7fc9481233d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.068+0000 7fc94e1c7700 1 -- 172.21.15.3:0/2012496452 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9440094a0 con 0x7fc9480a0a30 2023-10-03T10:17:47.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.068+0000 7fc94e9c8700 1 --2- 172.21.15.3:0/2012496452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc948133fa0 0x7fc9481233d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:47.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.068+0000 7fc94d9c6700 1 --2- 172.21.15.3:0/2012496452 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9480a1340 0x7fc948122e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:47.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.068+0000 7fc94e1c7700 1 --2- 172.21.15.3:0/2012496452 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9480a0a30 0x7fc948122950 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fc94400ebe0 tx=0x7fc94400eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:47.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.068+0000 7fc93effd700 1 -- 172.21.15.3:0/2012496452 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94400c850 con 0x7fc9480a0a30 2023-10-03T10:17:47.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.068+0000 7fc93effd700 1 -- 172.21.15.3:0/2012496452 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94400c9b0 con 0x7fc9480a0a30 2023-10-03T10:17:47.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.068+0000 7fc95042b700 1 -- 172.21.15.3:0/2012496452 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9380094a0 con 0x7fc9480a0a30 2023-10-03T10:17:47.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.068+0000 7fc93effd700 1 -- 172.21.15.3:0/2012496452 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94400cd40 con 0x7fc9480a0a30 2023-10-03T10:17:47.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.068+0000 7fc95042b700 1 -- 172.21.15.3:0/2012496452 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc948125e00 con 0x7fc9480a0a30 2023-10-03T10:17:47.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.069+0000 7fc93effd700 1 -- 172.21.15.3:0/2012496452 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc944013470 con 0x7fc9480a0a30 2023-10-03T10:17:47.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.069+0000 7fc95042b700 1 -- 172.21.15.3:0/2012496452 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc948133430 con 0x7fc9480a0a30 2023-10-03T10:17:47.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.070+0000 7fc93effd700 1 -- 172.21.15.3:0/2012496452 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc94403f820 con 0x7fc9480a0a30 2023-10-03T10:17:47.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.074+0000 7fc93effd700 1 -- 172.21.15.3:0/2012496452 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc944013720 con 0x7fc9480a0a30 2023-10-03T10:17:47.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.202+0000 7f2108072700 1 -- 172.21.15.3:0/995267706 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f210006c360 con 0x7f210006a770 2023-10-03T10:17:47.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.202+0000 7f20f67fc700 1 -- 172.21.15.3:0/995267706 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f20fc017070 con 0x7f210006a770 2023-10-03T10:17:47.203 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:47.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.205+0000 7f2108072700 1 -- 172.21.15.3:0/995267706 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210006a770 msgr2=0x7f21001203f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:47.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.205+0000 7f2108072700 1 --2- 172.21.15.3:0/995267706 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210006a770 0x7f21001203f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00ebe0 tx=0x7f20fc00eef0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.205+0000 7f2108072700 1 -- 172.21.15.3:0/995267706 shutdown_connections 2023-10-03T10:17:47.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.205+0000 7f2108072700 1 --2- 172.21.15.3:0/995267706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2100131d10 0x7f2100120e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.205+0000 7f2108072700 1 --2- 172.21.15.3:0/995267706 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210006a770 0x7f21001203f0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.205+0000 7f2108072700 1 --2- 172.21.15.3:0/995267706 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210006b080 0x7f2100120930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.205+0000 7f2108072700 1 -- 172.21.15.3:0/995267706 >> 172.21.15.3:0/995267706 conn(0x7f2100066f50 msgr2=0x7f2100070070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:47.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.205+0000 7f2108072700 1 -- 172.21.15.3:0/995267706 shutdown_connections 2023-10-03T10:17:47.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.205+0000 7f2108072700 1 -- 172.21.15.3:0/995267706 wait complete. 2023-10-03T10:17:47.219 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:47.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.239+0000 7f19ebba9700 1 -- 172.21.15.3:0/4191999797 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f19e406b250 con 0x7f19e409de00 2023-10-03T10:17:47.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.239+0000 7f19da7fc700 1 -- 172.21.15.3:0/4191999797 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f19d4015070 con 0x7f19e409de00 2023-10-03T10:17:47.239 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:47.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.241+0000 7f19ebba9700 1 -- 172.21.15.3:0/4191999797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19e409de00 msgr2=0x7f19e41c8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:47.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.241+0000 7f19ebba9700 1 --2- 172.21.15.3:0/4191999797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19e409de00 0x7f19e41c8950 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f19d40090b0 tx=0x7f19d4004750 comp rx=0 tx=0).stop 2023-10-03T10:17:47.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.241+0000 7f19ebba9700 1 -- 172.21.15.3:0/4191999797 shutdown_connections 2023-10-03T10:17:47.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.241+0000 7f19ebba9700 1 --2- 172.21.15.3:0/4191999797 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f19e409e210 0x7f19e41c48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.241+0000 7f19ebba9700 1 --2- 172.21.15.3:0/4191999797 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f19e406c2f0 0x7f19e41c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.241+0000 7f19ebba9700 1 --2- 172.21.15.3:0/4191999797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19e409de00 0x7f19e41c8950 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.241+0000 7f19ebba9700 1 -- 172.21.15.3:0/4191999797 >> 172.21.15.3:0/4191999797 conn(0x7f19e409f470 msgr2=0x7f19e4070a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:47.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.241+0000 7f19ebba9700 1 -- 172.21.15.3:0/4191999797 shutdown_connections 2023-10-03T10:17:47.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.241+0000 7f19ebba9700 1 -- 172.21.15.3:0/4191999797 wait complete. 2023-10-03T10:17:47.255 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.281+0000 7fc95042b700 1 -- 172.21.15.3:0/2012496452 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc9480a1d40 con 0x7fc9480a0a30 2023-10-03T10:17:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.282+0000 7fc93effd700 1 -- 172.21.15.3:0/2012496452 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc94401b9c0 con 0x7fc9480a0a30 2023-10-03T10:17:47.282 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:47.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.284+0000 7fc95042b700 1 -- 172.21.15.3:0/2012496452 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9480a0a30 msgr2=0x7fc948122950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:47.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.284+0000 7fc95042b700 1 --2- 172.21.15.3:0/2012496452 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9480a0a30 0x7fc948122950 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fc94400ebe0 tx=0x7fc94400eef0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.284+0000 7fc95042b700 1 -- 172.21.15.3:0/2012496452 shutdown_connections 2023-10-03T10:17:47.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.284+0000 7fc95042b700 1 --2- 172.21.15.3:0/2012496452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc948133fa0 0x7fc9481233d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.284+0000 7fc95042b700 1 --2- 172.21.15.3:0/2012496452 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9480a0a30 0x7fc948122950 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.284+0000 7fc95042b700 1 --2- 172.21.15.3:0/2012496452 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9480a1340 0x7fc948122e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.284+0000 7fc95042b700 1 -- 172.21.15.3:0/2012496452 >> 172.21.15.3:0/2012496452 conn(0x7fc9480a6550 msgr2=0x7fc94811c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:47.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.284+0000 7fc95042b700 1 -- 172.21.15.3:0/2012496452 shutdown_connections 2023-10-03T10:17:47.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.284+0000 7fc95042b700 1 -- 172.21.15.3:0/2012496452 wait complete. 2023-10-03T10:17:47.298 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:47.461 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:47.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.606+0000 7f64e63e1700 1 Processor -- start 2023-10-03T10:17:47.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.606+0000 7f64e63e1700 1 -- start start 2023-10-03T10:17:47.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.607+0000 7f64e63e1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64e009d6f0 0x7f64e009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:47.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.607+0000 7f64e63e1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f64e009e000 0x7f64e0130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:47.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.607+0000 7f64e63e1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64e009e410 0x7f64e0130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:47.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.607+0000 7f64e63e1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64e009f520 con 0x7f64e009e000 2023-10-03T10:17:47.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.607+0000 7f64e63e1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e009f130 con 0x7f64e009e410 2023-10-03T10:17:47.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.607+0000 7f64e63e1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f64e009e930 con 0x7f64e009d6f0 2023-10-03T10:17:47.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.607+0000 7f64dffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64e009d6f0 0x7f64e009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:47.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.607+0000 7f64df7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f64e009e000 0x7f64e0130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:47.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.607+0000 7f64e497e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64e009e410 0x7f64e0130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:47.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.607+0000 7f64df7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f64e009e000 0x7f64e0130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59008/0 (socket says 172.21.15.3:59008) 2023-10-03T10:17:47.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.607+0000 7f64dffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64e009d6f0 0x7f64e009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43250/0 (socket says 172.21.15.3:43250) 2023-10-03T10:17:47.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.607+0000 7f64e497e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64e009e410 0x7f64e0130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60444/0 (socket says 172.21.15.3:60444) 2023-10-03T10:17:47.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.607+0000 7f64df7fe700 1 -- 172.21.15.3:0/3013552258 learned_addr learned my addr 172.21.15.3:0/3013552258 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:47.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.607+0000 7f64dffff700 1 -- 172.21.15.3:0/3013552258 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64e009e410 msgr2=0x7f64e0130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:47.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.607+0000 7f64dffff700 1 --2- 172.21.15.3:0/3013552258 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64e009e410 0x7f64e0130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.607+0000 7f64dffff700 1 -- 172.21.15.3:0/3013552258 >> v2:172.21.15.3:3300/0 conn(0x7f64e009e000 msgr2=0x7f64e0130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:47.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.607+0000 7f64dffff700 1 --2- 172.21.15.3:0/3013552258 >> v2:172.21.15.3:3300/0 conn(0x7f64e009e000 0x7f64e0130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.607+0000 7f64dffff700 1 -- 172.21.15.3:0/3013552258 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e009b2d0 con 0x7f64e009d6f0 2023-10-03T10:17:47.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.607+0000 7f64df7fe700 1 --2- 172.21.15.3:0/3013552258 >> v2:172.21.15.3:3300/0 conn(0x7f64e009e000 0x7f64e0130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:47.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.608+0000 7f64dffff700 1 --2- 172.21.15.3:0/3013552258 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64e009d6f0 0x7f64e009dac0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f64c800bb10 tx=0x7f64c800be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51729762ae98841d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:47.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.608+0000 7f64deffd700 1 -- 172.21.15.3:0/3013552258 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c800dc30 con 0x7f64e009d6f0 2023-10-03T10:17:47.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.608+0000 7f64deffd700 1 -- 172.21.15.3:0/3013552258 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c800dd90 con 0x7f64e009d6f0 2023-10-03T10:17:47.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.608+0000 7f64deffd700 1 -- 172.21.15.3:0/3013552258 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8010690 con 0x7f64e009d6f0 2023-10-03T10:17:47.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.608+0000 7f64e63e1700 1 -- 172.21.15.3:0/3013552258 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64e009d6f0 msgr2=0x7f64e009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:47.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.608+0000 7f64e63e1700 1 --2- 172.21.15.3:0/3013552258 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64e009d6f0 0x7f64e009dac0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f64c800bb10 tx=0x7f64c800be20 comp rx=0 tx=0).stop 2023-10-03T10:17:47.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.608+0000 7f64e63e1700 1 -- 172.21.15.3:0/3013552258 shutdown_connections 2023-10-03T10:17:47.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.608+0000 7f64e63e1700 1 --2- 172.21.15.3:0/3013552258 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64e009e410 0x7f64e0130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.609+0000 7f64e63e1700 1 --2- 172.21.15.3:0/3013552258 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64e009d6f0 0x7f64e009dac0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.609+0000 7f64e63e1700 1 --2- 172.21.15.3:0/3013552258 >> v2:172.21.15.3:3300/0 conn(0x7f64e009e000 0x7f64e0130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.609+0000 7f64e63e1700 1 -- 172.21.15.3:0/3013552258 >> 172.21.15.3:0/3013552258 conn(0x7f64e0066f50 msgr2=0x7f64e009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:47.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.609+0000 7f64e63e1700 1 -- 172.21.15.3:0/3013552258 shutdown_connections 2023-10-03T10:17:47.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.609+0000 7f64e63e1700 1 -- 172.21.15.3:0/3013552258 wait complete. 2023-10-03T10:17:47.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.610+0000 7f64e63e1700 1 Processor -- start 2023-10-03T10:17:47.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.610+0000 7f64e63e1700 1 -- start start 2023-10-03T10:17:47.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.611+0000 7f64e63e1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64e009d6f0 0x7f64e011e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:47.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.611+0000 7f64e63e1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64e009e000 0x7f64e011e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:47.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.611+0000 7f64e63e1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64e009e410 0x7f64e011ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:47.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.611+0000 7f64e63e1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e01201f0 con 0x7f64e009e000 2023-10-03T10:17:47.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.611+0000 7f64e63e1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e0120650 con 0x7f64e009e410 2023-10-03T10:17:47.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.611+0000 7f64e63e1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f64e0121f80 con 0x7f64e009d6f0 2023-10-03T10:17:47.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.611+0000 7f64df7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64e009e000 0x7f64e011e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:47.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.611+0000 7f64df7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64e009e000 0x7f64e011e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59014/0 (socket says 172.21.15.3:59014) 2023-10-03T10:17:47.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.611+0000 7f64df7fe700 1 -- 172.21.15.3:0/2891477765 learned_addr learned my addr 172.21.15.3:0/2891477765 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:47.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.611+0000 7f64dffff700 1 --2- 172.21.15.3:0/2891477765 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64e009d6f0 0x7f64e011e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:47.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.611+0000 7f64e497e700 1 --2- 172.21.15.3:0/2891477765 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64e009e410 0x7f64e011ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:47.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.611+0000 7f64df7fe700 1 -- 172.21.15.3:0/2891477765 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64e009d6f0 msgr2=0x7f64e011e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:47.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.611+0000 7f64df7fe700 1 --2- 172.21.15.3:0/2891477765 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64e009d6f0 0x7f64e011e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.611+0000 7f64df7fe700 1 -- 172.21.15.3:0/2891477765 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64e009e410 msgr2=0x7f64e011ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:47.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.611+0000 7f64df7fe700 1 --2- 172.21.15.3:0/2891477765 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64e009e410 0x7f64e011ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.611+0000 7f64df7fe700 1 -- 172.21.15.3:0/2891477765 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d0009550 con 0x7f64e009e000 2023-10-03T10:17:47.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.611+0000 7f64df7fe700 1 --2- 172.21.15.3:0/2891477765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64e009e000 0x7f64e011e9f0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f64d000ebe0 tx=0x7f64d000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:47.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.612+0000 7f64dcff9700 1 -- 172.21.15.3:0/2891477765 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d0004240 con 0x7f64e009e000 2023-10-03T10:17:47.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.612+0000 7f64dcff9700 1 -- 172.21.15.3:0/2891477765 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d00043a0 con 0x7f64e009e000 2023-10-03T10:17:47.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.612+0000 7f64dcff9700 1 -- 172.21.15.3:0/2891477765 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d00036a0 con 0x7f64e009e000 2023-10-03T10:17:47.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.612+0000 7f64e63e1700 1 -- 172.21.15.3:0/2891477765 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c800b740 con 0x7f64e009e000 2023-10-03T10:17:47.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.612+0000 7f64e63e1700 1 -- 172.21.15.3:0/2891477765 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e009c700 con 0x7f64e009e000 2023-10-03T10:17:47.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.612+0000 7f64dcff9700 1 -- 172.21.15.3:0/2891477765 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f64d0003800 con 0x7f64e009e000 2023-10-03T10:17:47.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.613+0000 7f64dcff9700 1 -- 172.21.15.3:0/2891477765 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f64d003f8b0 con 0x7f64e009e000 2023-10-03T10:17:47.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.613+0000 7f64e63e1700 1 -- 172.21.15.3:0/2891477765 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e01303c0 con 0x7f64e009e000 2023-10-03T10:17:47.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.618+0000 7f64dcff9700 1 -- 172.21.15.3:0/2891477765 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f64d000c640 con 0x7f64e009e000 2023-10-03T10:17:47.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.822+0000 7f64e63e1700 1 -- 172.21.15.3:0/2891477765 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f64e00a29f0 con 0x7f64e009e000 2023-10-03T10:17:47.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.823+0000 7f64dcff9700 1 -- 172.21.15.3:0/2891477765 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f64d001bb60 con 0x7f64e009e000 2023-10-03T10:17:47.823 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:47.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.825+0000 7f64e63e1700 1 -- 172.21.15.3:0/2891477765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64e009e000 msgr2=0x7f64e011e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:47.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.825+0000 7f64e63e1700 1 --2- 172.21.15.3:0/2891477765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64e009e000 0x7f64e011e9f0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f64d000ebe0 tx=0x7f64d000eef0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.825+0000 7f64e63e1700 1 -- 172.21.15.3:0/2891477765 shutdown_connections 2023-10-03T10:17:47.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.825+0000 7f64e63e1700 1 --2- 172.21.15.3:0/2891477765 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64e009e410 0x7f64e011ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.825+0000 7f64e63e1700 1 --2- 172.21.15.3:0/2891477765 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64e009d6f0 0x7f64e011e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.825+0000 7f64e63e1700 1 --2- 172.21.15.3:0/2891477765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64e009e000 0x7f64e011e9f0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:47.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.825+0000 7f64e63e1700 1 -- 172.21.15.3:0/2891477765 >> 172.21.15.3:0/2891477765 conn(0x7f64e0066f50 msgr2=0x7f64e006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:47.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.825+0000 7f64e63e1700 1 -- 172.21.15.3:0/2891477765 shutdown_connections 2023-10-03T10:17:47.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:47.825+0000 7f64e63e1700 1 -- 172.21.15.3:0/2891477765 wait complete. 2023-10-03T10:17:47.839 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:48.220 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:48.256 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:48.299 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:48.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.371+0000 7f5b1af44700 1 Processor -- start 2023-10-03T10:17:48.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.371+0000 7f5b1af44700 1 -- start start 2023-10-03T10:17:48.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.371+0000 7f5b1af44700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b1409d6f0 0x7f5b1409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:48.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.371+0000 7f5b1af44700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5b1409e000 0x7f5b141339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:48.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.372+0000 7f5b1af44700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5b1409e410 0x7f5b14133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:48.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.372+0000 7f5b1af44700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b1406b460 con 0x7f5b1409e410 2023-10-03T10:17:48.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.372+0000 7f5b1af44700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1409b120 con 0x7f5b1409d6f0 2023-10-03T10:17:48.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.372+0000 7f5b1af44700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1409a7e0 con 0x7f5b1409e000 2023-10-03T10:17:48.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.372+0000 7f5b18ce0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b1409d6f0 0x7f5b1409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:48.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.372+0000 7f5b13fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5b1409e000 0x7f5b141339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:48.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.372+0000 7f5b194e1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5b1409e410 0x7f5b14133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:48.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.372+0000 7f5b194e1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5b1409e410 0x7f5b14133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59030/0 (socket says 172.21.15.3:59030) 2023-10-03T10:17:48.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.372+0000 7f5b13fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5b1409e000 0x7f5b141339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43254/0 (socket says 172.21.15.3:43254) 2023-10-03T10:17:48.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.372+0000 7f5b18ce0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b1409d6f0 0x7f5b1409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60454/0 (socket says 172.21.15.3:60454) 2023-10-03T10:17:48.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.372+0000 7f5b194e1700 1 -- 172.21.15.3:0/3310196828 learned_addr learned my addr 172.21.15.3:0/3310196828 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:48.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.372+0000 7f5b194e1700 1 -- 172.21.15.3:0/3310196828 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5b1409e000 msgr2=0x7f5b141339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:48.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.372+0000 7f5b194e1700 1 --2- 172.21.15.3:0/3310196828 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5b1409e000 0x7f5b141339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.372+0000 7f5b194e1700 1 -- 172.21.15.3:0/3310196828 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b1409d6f0 msgr2=0x7f5b1409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:48.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.372+0000 7f5b194e1700 1 --2- 172.21.15.3:0/3310196828 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b1409d6f0 0x7f5b1409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.372+0000 7f5b194e1700 1 -- 172.21.15.3:0/3310196828 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1409a500 con 0x7f5b1409e410 2023-10-03T10:17:48.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.372+0000 7f5b13fff700 1 --2- 172.21.15.3:0/3310196828 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5b1409e000 0x7f5b141339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:48.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.372+0000 7f5b194e1700 1 --2- 172.21.15.3:0/3310196828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b1409e410 0x7f5b14133f30 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c009e50 tx=0x7f5b0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d2bdbbb2047fee server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:48.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.373+0000 7f5b137fe700 1 -- 172.21.15.3:0/3310196828 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c00e070 con 0x7f5b1409e410 2023-10-03T10:17:48.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.373+0000 7f5b137fe700 1 -- 172.21.15.3:0/3310196828 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0c00b910 con 0x7f5b1409e410 2023-10-03T10:17:48.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.373+0000 7f5b137fe700 1 -- 172.21.15.3:0/3310196828 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c00bcc0 con 0x7f5b1409e410 2023-10-03T10:17:48.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.373+0000 7f5b1af44700 1 -- 172.21.15.3:0/3310196828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b1409e410 msgr2=0x7f5b14133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:48.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.373+0000 7f5b1af44700 1 --2- 172.21.15.3:0/3310196828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b1409e410 0x7f5b14133f30 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c009e50 tx=0x7f5b0c00b330 comp rx=0 tx=0).stop 2023-10-03T10:17:48.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.373+0000 7f5b1af44700 1 -- 172.21.15.3:0/3310196828 shutdown_connections 2023-10-03T10:17:48.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.373+0000 7f5b1af44700 1 --2- 172.21.15.3:0/3310196828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b1409e410 0x7f5b14133f30 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.373+0000 7f5b1af44700 1 --2- 172.21.15.3:0/3310196828 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b1409d6f0 0x7f5b1409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.373+0000 7f5b1af44700 1 --2- 172.21.15.3:0/3310196828 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5b1409e000 0x7f5b141339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.373+0000 7f5b1af44700 1 -- 172.21.15.3:0/3310196828 >> 172.21.15.3:0/3310196828 conn(0x7f5b14066f50 msgr2=0x7f5b1409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:48.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.373+0000 7f5b1af44700 1 -- 172.21.15.3:0/3310196828 shutdown_connections 2023-10-03T10:17:48.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.373+0000 7f5b1af44700 1 -- 172.21.15.3:0/3310196828 wait complete. 2023-10-03T10:17:48.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.375+0000 7f5b1af44700 1 Processor -- start 2023-10-03T10:17:48.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.375+0000 7f5b1af44700 1 -- start start 2023-10-03T10:17:48.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.376+0000 7f5b1af44700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b1409d6f0 0x7f5b141c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:48.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.376+0000 7f5b1af44700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5b1409e000 0x7f5b141c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:48.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.376+0000 7f5b1af44700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b1409e410 0x7f5b141c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:48.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.376+0000 7f5b1af44700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b141a67c0 con 0x7f5b1409e410 2023-10-03T10:17:48.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.376+0000 7f5b1af44700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b141a65d0 con 0x7f5b1409d6f0 2023-10-03T10:17:48.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.376+0000 7f5b1af44700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b141a6ea0 con 0x7f5b1409e000 2023-10-03T10:17:48.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.376+0000 7f5b13fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5b1409e000 0x7f5b141c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:48.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.376+0000 7f5b18ce0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b1409d6f0 0x7f5b141c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:48.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.376+0000 7f5b13fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5b1409e000 0x7f5b141c8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43262/0 (socket says 172.21.15.3:43262) 2023-10-03T10:17:48.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.376+0000 7f5b194e1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b1409e410 0x7f5b141c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:48.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.376+0000 7f5b13fff700 1 -- 172.21.15.3:0/175981101 learned_addr learned my addr 172.21.15.3:0/175981101 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:48.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.376+0000 7f5b18ce0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b1409d6f0 0x7f5b141c8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60458/0 (socket says 172.21.15.3:60458) 2023-10-03T10:17:48.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.376+0000 7f5b13fff700 1 -- 172.21.15.3:0/175981101 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b1409d6f0 msgr2=0x7f5b141c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:48.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.376+0000 7f5b13fff700 1 --2- 172.21.15.3:0/175981101 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b1409d6f0 0x7f5b141c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.377+0000 7f5b13fff700 1 -- 172.21.15.3:0/175981101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b1409e410 msgr2=0x7f5b141c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:48.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.377+0000 7f5b13fff700 1 --2- 172.21.15.3:0/175981101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b1409e410 0x7f5b141c8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.377+0000 7f5b13fff700 1 -- 172.21.15.3:0/175981101 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b000094a0 con 0x7f5b1409e000 2023-10-03T10:17:48.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.377+0000 7f5b194e1700 1 --2- 172.21.15.3:0/175981101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b1409e410 0x7f5b141c8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:48.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.377+0000 7f5b18ce0700 1 --2- 172.21.15.3:0/175981101 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b1409d6f0 0x7f5b141c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:48.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.377+0000 7f5b13fff700 1 --2- 172.21.15.3:0/175981101 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5b1409e000 0x7f5b141c8770 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000ebb0 tx=0x7f5b0000eec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:48.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.377+0000 7f5b117fa700 1 -- 172.21.15.3:0/175981101 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0000c850 con 0x7f5b1409e000 2023-10-03T10:17:48.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.377+0000 7f5b117fa700 1 -- 172.21.15.3:0/175981101 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0000c9b0 con 0x7f5b1409e000 2023-10-03T10:17:48.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.377+0000 7f5b1af44700 1 -- 172.21.15.3:0/175981101 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b0c009b30 con 0x7f5b1409e000 2023-10-03T10:17:48.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.377+0000 7f5b117fa700 1 -- 172.21.15.3:0/175981101 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0000cd40 con 0x7f5b1409e000 2023-10-03T10:17:48.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.377+0000 7f5b1af44700 1 -- 172.21.15.3:0/175981101 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b1409c710 con 0x7f5b1409e000 2023-10-03T10:17:48.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.378+0000 7f5b1af44700 1 -- 172.21.15.3:0/175981101 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b141a7ea0 con 0x7f5b1409e000 2023-10-03T10:17:48.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.379+0000 7f5b117fa700 1 -- 172.21.15.3:0/175981101 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5b00013470 con 0x7f5b1409e000 2023-10-03T10:17:48.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.379+0000 7f5b117fa700 1 -- 172.21.15.3:0/175981101 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5b0003fdc0 con 0x7f5b1409e000 2023-10-03T10:17:48.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.384+0000 7f5b117fa700 1 -- 172.21.15.3:0/175981101 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5b00013720 con 0x7f5b1409e000 2023-10-03T10:17:48.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.410+0000 7fe6eff3a700 1 Processor -- start 2023-10-03T10:17:48.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.411+0000 7fe6eff3a700 1 -- start start 2023-10-03T10:17:48.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.411+0000 7fe6eff3a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe6e80a0a30 0x7fe6e80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:48.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.411+0000 7fe6eff3a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6e80a1340 0x7fe6e8133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:48.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.411+0000 7fe6eff3a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe6e8133fa0 0x7fe6e8134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:48.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.411+0000 7fe6eff3a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6e809edd0 con 0x7fe6e8133fa0 2023-10-03T10:17:48.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.411+0000 7fe6eff3a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e809e1f0 con 0x7fe6e80a1340 2023-10-03T10:17:48.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.411+0000 7fe6eff3a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e809dc50 con 0x7fe6e80a0a30 2023-10-03T10:17:48.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.411+0000 7fe6ed4d5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6e80a1340 0x7fe6e8133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:48.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.411+0000 7fe6ed4d5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6e80a1340 0x7fe6e8133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60464/0 (socket says 172.21.15.3:60464) 2023-10-03T10:17:48.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.411+0000 7fe6ed4d5700 1 -- 172.21.15.3:0/4072934758 learned_addr learned my addr 172.21.15.3:0/4072934758 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:48.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.411+0000 7fe6ee4d7700 1 --2- 172.21.15.3:0/4072934758 >> v2:172.21.15.3:3300/0 conn(0x7fe6e8133fa0 0x7fe6e8134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:48.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.412+0000 7fe6edcd6700 1 --2- 172.21.15.3:0/4072934758 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe6e80a0a30 0x7fe6e80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:48.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.412+0000 7fe6ed4d5700 1 -- 172.21.15.3:0/4072934758 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe6e80a0a30 msgr2=0x7fe6e80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:48.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.412+0000 7fe6ed4d5700 1 --2- 172.21.15.3:0/4072934758 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe6e80a0a30 0x7fe6e80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.412+0000 7fe6ed4d5700 1 -- 172.21.15.3:0/4072934758 >> v2:172.21.15.3:3300/0 conn(0x7fe6e8133fa0 msgr2=0x7fe6e8134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:48.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.412+0000 7fe6ed4d5700 1 --2- 172.21.15.3:0/4072934758 >> v2:172.21.15.3:3300/0 conn(0x7fe6e8133fa0 0x7fe6e8134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.412+0000 7fe6ed4d5700 1 -- 172.21.15.3:0/4072934758 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e80a1fe0 con 0x7fe6e80a1340 2023-10-03T10:17:48.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.412+0000 7fe6ee4d7700 1 --2- 172.21.15.3:0/4072934758 >> v2:172.21.15.3:3300/0 conn(0x7fe6e8133fa0 0x7fe6e8134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:48.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.412+0000 7fe6ed4d5700 1 --2- 172.21.15.3:0/4072934758 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6e80a1340 0x7fe6e8133a60 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80098f0 tx=0x7fe6d8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db26d368b76b48a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:48.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.412+0000 7fe6eccd4700 1 -- 172.21.15.3:0/4072934758 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8011830 con 0x7fe6e80a1340 2023-10-03T10:17:48.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.412+0000 7fe6eccd4700 1 -- 172.21.15.3:0/4072934758 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6d8011990 con 0x7fe6e80a1340 2023-10-03T10:17:48.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.412+0000 7fe6eccd4700 1 -- 172.21.15.3:0/4072934758 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8011d20 con 0x7fe6e80a1340 2023-10-03T10:17:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.413+0000 7fe6eff3a700 1 -- 172.21.15.3:0/4072934758 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6e80a1340 msgr2=0x7fe6e8133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.413+0000 7fe6eff3a700 1 --2- 172.21.15.3:0/4072934758 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6e80a1340 0x7fe6e8133a60 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80098f0 tx=0x7fe6d8009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.413+0000 7fe6eff3a700 1 -- 172.21.15.3:0/4072934758 shutdown_connections 2023-10-03T10:17:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.413+0000 7fe6eff3a700 1 --2- 172.21.15.3:0/4072934758 >> v2:172.21.15.3:3300/0 conn(0x7fe6e8133fa0 0x7fe6e8134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.413+0000 7fe6eff3a700 1 --2- 172.21.15.3:0/4072934758 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe6e80a0a30 0x7fe6e80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.413+0000 7fe6eff3a700 1 --2- 172.21.15.3:0/4072934758 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6e80a1340 0x7fe6e8133a60 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.413+0000 7fe6eff3a700 1 -- 172.21.15.3:0/4072934758 >> 172.21.15.3:0/4072934758 conn(0x7fe6e80a6550 msgr2=0x7fe6e80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.413+0000 7fe6eff3a700 1 -- 172.21.15.3:0/4072934758 shutdown_connections 2023-10-03T10:17:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.414+0000 7fe6eff3a700 1 -- 172.21.15.3:0/4072934758 wait complete. 2023-10-03T10:17:48.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.414+0000 7fe6eff3a700 1 Processor -- start 2023-10-03T10:17:48.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.415+0000 7fe6eff3a700 1 -- start start 2023-10-03T10:17:48.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.415+0000 7fe6eff3a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6e80a0a30 0x7fe6e8120130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:48.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.415+0000 7fe6eff3a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6e80a1340 0x7fe6e8120670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:48.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.415+0000 7fe6eff3a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe6e8133fa0 0x7fe6e8120bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:48.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.415+0000 7fe6eff3a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e8123c50 con 0x7fe6e80a1340 2023-10-03T10:17:48.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.415+0000 7fe6eff3a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e80a3530 con 0x7fe6e80a0a30 2023-10-03T10:17:48.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.415+0000 7fe6eff3a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e81233e0 con 0x7fe6e8133fa0 2023-10-03T10:17:48.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.415+0000 7fe6edcd6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6e80a0a30 0x7fe6e8120130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:48.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.415+0000 7fe6edcd6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6e80a0a30 0x7fe6e8120130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60470/0 (socket says 172.21.15.3:60470) 2023-10-03T10:17:48.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.415+0000 7fe6edcd6700 1 -- 172.21.15.3:0/1784824602 learned_addr learned my addr 172.21.15.3:0/1784824602 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:48.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.415+0000 7fe6ee4d7700 1 --2- 172.21.15.3:0/1784824602 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe6e8133fa0 0x7fe6e8120bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:48.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.415+0000 7fe6ed4d5700 1 --2- 172.21.15.3:0/1784824602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6e80a1340 0x7fe6e8120670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:48.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.416+0000 7fe6edcd6700 1 -- 172.21.15.3:0/1784824602 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe6e8133fa0 msgr2=0x7fe6e8120bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:48.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.416+0000 7fe6edcd6700 1 --2- 172.21.15.3:0/1784824602 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe6e8133fa0 0x7fe6e8120bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.416+0000 7fe6edcd6700 1 -- 172.21.15.3:0/1784824602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6e80a1340 msgr2=0x7fe6e8120670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:48.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.416+0000 7fe6edcd6700 1 --2- 172.21.15.3:0/1784824602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6e80a1340 0x7fe6e8120670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.416+0000 7fe6edcd6700 1 -- 172.21.15.3:0/1784824602 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e0009a70 con 0x7fe6e80a0a30 2023-10-03T10:17:48.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.416+0000 7fe6edcd6700 1 --2- 172.21.15.3:0/1784824602 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6e80a0a30 0x7fe6e8120130 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400e9b0 tx=0x7fe6e400ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:48.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.416+0000 7fe6de7fc700 1 -- 172.21.15.3:0/1784824602 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4009710 con 0x7fe6e80a0a30 2023-10-03T10:17:48.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.416+0000 7fe6de7fc700 1 -- 172.21.15.3:0/1784824602 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e400ee40 con 0x7fe6e80a0a30 2023-10-03T10:17:48.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.416+0000 7fe6de7fc700 1 -- 172.21.15.3:0/1784824602 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e400ca50 con 0x7fe6e80a0a30 2023-10-03T10:17:48.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.416+0000 7fe6eff3a700 1 -- 172.21.15.3:0/1784824602 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6d8009550 con 0x7fe6e80a0a30 2023-10-03T10:17:48.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.416+0000 7fe6eff3a700 1 -- 172.21.15.3:0/1784824602 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e8121d40 con 0x7fe6e80a0a30 2023-10-03T10:17:48.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.417+0000 7fe6de7fc700 1 -- 172.21.15.3:0/1784824602 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe6e400cbb0 con 0x7fe6e80a0a30 2023-10-03T10:17:48.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.417+0000 7fe6de7fc700 1 -- 172.21.15.3:0/1784824602 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe6e403fad0 con 0x7fe6e80a0a30 2023-10-03T10:17:48.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.417+0000 7fe6eff3a700 1 -- 172.21.15.3:0/1784824602 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6e81237d0 con 0x7fe6e80a0a30 2023-10-03T10:17:48.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.422+0000 7fe6de7fc700 1 -- 172.21.15.3:0/1784824602 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe6e81237d0 con 0x7fe6e80a0a30 2023-10-03T10:17:48.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.451+0000 7fc206412700 1 Processor -- start 2023-10-03T10:17:48.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.451+0000 7fc206412700 1 -- start start 2023-10-03T10:17:48.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.452+0000 7fc206412700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc2000a1510 0x7fc200138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:48.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.452+0000 7fc206412700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc200139250 0x7fc20013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:48.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.452+0000 7fc206412700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc20009edd0 con 0x7fc2000a0a30 2023-10-03T10:17:48.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.452+0000 7fc206412700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2000a18e0 con 0x7fc200139250 2023-10-03T10:17:48.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.452+0000 7fc206412700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc20009e1f0 con 0x7fc2000a1510 2023-10-03T10:17:48.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.452+0000 7fc1ff7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc2000a1510 0x7fc200138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:48.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.452+0000 7fc1fffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc2000a0a30 0x7fc2000a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42486/0 (socket says 172.21.15.3:42486) 2023-10-03T10:17:48.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.452+0000 7fc1fffff700 1 -- 172.21.15.3:0/659310083 learned_addr learned my addr 172.21.15.3:0/659310083 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:48.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.452+0000 7fc2049af700 1 --2- 172.21.15.3:0/659310083 >> v2:172.21.15.3:3300/0 conn(0x7fc200139250 0x7fc20013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:48.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.453+0000 7fc2049af700 1 -- 172.21.15.3:0/659310083 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc2000a1510 msgr2=0x7fc200138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:48.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.453+0000 7fc2049af700 1 --2- 172.21.15.3:0/659310083 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc2000a1510 0x7fc200138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.453+0000 7fc2049af700 1 -- 172.21.15.3:0/659310083 >> v1:172.21.15.3:6789/0 conn(0x7fc2000a0a30 legacy=0x7fc2000a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:48.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.453+0000 7fc2049af700 1 -- 172.21.15.3:0/659310083 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20009dc50 con 0x7fc200139250 2023-10-03T10:17:48.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.453+0000 7fc2049af700 1 --2- 172.21.15.3:0/659310083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc200139250 0x7fc20013b630 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8009dc0 tx=0x7fc1f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3436b7291a2457 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:48.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.453+0000 7fc1feffd700 1 -- 172.21.15.3:0/659310083 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f800e070 con 0x7fc200139250 2023-10-03T10:17:48.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.453+0000 7fc1feffd700 1 -- 172.21.15.3:0/659310083 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f800bae0 con 0x7fc200139250 2023-10-03T10:17:48.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.453+0000 7fc1feffd700 1 -- 172.21.15.3:0/659310083 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f800be90 con 0x7fc200139250 2023-10-03T10:17:48.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.454+0000 7fc206412700 1 -- 172.21.15.3:0/659310083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc200139250 msgr2=0x7fc20013b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:48.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.454+0000 7fc206412700 1 --2- 172.21.15.3:0/659310083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc200139250 0x7fc20013b630 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8009dc0 tx=0x7fc1f800b330 comp rx=0 tx=0).stop 2023-10-03T10:17:48.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.454+0000 7fc206412700 1 -- 172.21.15.3:0/659310083 shutdown_connections 2023-10-03T10:17:48.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.454+0000 7fc206412700 1 --2- 172.21.15.3:0/659310083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc200139250 0x7fc20013b630 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.454+0000 7fc206412700 1 --2- 172.21.15.3:0/659310083 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc2000a1510 0x7fc200138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.454+0000 7fc206412700 1 -- 172.21.15.3:0/659310083 >> 172.21.15.3:0/659310083 conn(0x7fc2000a6550 msgr2=0x7fc2000a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:48.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.454+0000 7fc206412700 1 -- 172.21.15.3:0/659310083 shutdown_connections 2023-10-03T10:17:48.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.454+0000 7fc206412700 1 -- 172.21.15.3:0/659310083 wait complete. 2023-10-03T10:17:48.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.455+0000 7fc206412700 1 Processor -- start 2023-10-03T10:17:48.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.456+0000 7fc206412700 1 -- start start 2023-10-03T10:17:48.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.456+0000 7fc206412700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc2000a0a30 0x7fc2001c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:48.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.456+0000 7fc206412700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc2000a1510 0x7fc2001c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:48.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.456+0000 7fc206412700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc200139250 0x7fc2001227d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:48.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.456+0000 7fc206412700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2001243e0 con 0x7fc200139250 2023-10-03T10:17:48.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.456+0000 7fc206412700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2001263d0 con 0x7fc2000a0a30 2023-10-03T10:17:48.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.456+0000 7fc206412700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2000a0220 con 0x7fc2000a1510 2023-10-03T10:17:48.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.456+0000 7fc1ff7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc2000a1510 0x7fc2001c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:48.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.456+0000 7fc2049af700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc200139250 0x7fc2001227d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:48.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.456+0000 7fc1fffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc2000a0a30 0x7fc2001c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:48.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.456+0000 7fc1ff7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc2000a1510 0x7fc2001c8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43296/0 (socket says 172.21.15.3:43296) 2023-10-03T10:17:48.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.456+0000 7fc2049af700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc200139250 0x7fc2001227d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59076/0 (socket says 172.21.15.3:59076) 2023-10-03T10:17:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.457+0000 7fc1ff7fe700 1 -- 172.21.15.3:0/3418630563 learned_addr learned my addr 172.21.15.3:0/3418630563 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.457+0000 7fc1ff7fe700 1 -- 172.21.15.3:0/3418630563 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc2000a0a30 msgr2=0x7fc2001c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.457+0000 7fc1ff7fe700 1 --2- 172.21.15.3:0/3418630563 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc2000a0a30 0x7fc2001c8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.457+0000 7fc1ff7fe700 1 -- 172.21.15.3:0/3418630563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc200139250 msgr2=0x7fc2001227d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.457+0000 7fc1ff7fe700 1 --2- 172.21.15.3:0/3418630563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc200139250 0x7fc2001227d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.457+0000 7fc1ff7fe700 1 -- 172.21.15.3:0/3418630563 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f8009a70 con 0x7fc2000a1510 2023-10-03T10:17:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.457+0000 7fc2049af700 1 --2- 172.21.15.3:0/3418630563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc200139250 0x7fc2001227d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.457+0000 7fc1ff7fe700 1 --2- 172.21.15.3:0/3418630563 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc2000a1510 0x7fc2001c8b60 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400e9b0 tx=0x7fc1f400ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.457+0000 7fc1fcff9700 1 -- 172.21.15.3:0/3418630563 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4009710 con 0x7fc2000a1510 2023-10-03T10:17:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.457+0000 7fc1fcff9700 1 -- 172.21.15.3:0/3418630563 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f400ee50 con 0x7fc2000a1510 2023-10-03T10:17:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.457+0000 7fc206412700 1 -- 172.21.15.3:0/3418630563 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2001386a0 con 0x7fc2000a1510 2023-10-03T10:17:48.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.457+0000 7fc206412700 1 -- 172.21.15.3:0/3418630563 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2001389e0 con 0x7fc2000a1510 2023-10-03T10:17:48.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.458+0000 7fc1fcff9700 1 -- 172.21.15.3:0/3418630563 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f400ca50 con 0x7fc2000a1510 2023-10-03T10:17:48.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.458+0000 7fc1fcff9700 1 -- 172.21.15.3:0/3418630563 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc1f400cbb0 con 0x7fc2000a1510 2023-10-03T10:17:48.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.459+0000 7fc1fcff9700 1 -- 172.21.15.3:0/3418630563 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc1f403f7e0 con 0x7fc2000a1510 2023-10-03T10:17:48.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.459+0000 7fc206412700 1 -- 172.21.15.3:0/3418630563 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1e00052e0 con 0x7fc2000a1510 2023-10-03T10:17:48.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.468+0000 7fc1fcff9700 1 -- 172.21.15.3:0/3418630563 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc1f4021080 con 0x7fc2000a1510 2023-10-03T10:17:48.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.597+0000 7f5b1af44700 1 -- 172.21.15.3:0/175981101 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5b141a7ea0 con 0x7f5b1409e000 2023-10-03T10:17:48.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.598+0000 7f5b117fa700 1 -- 172.21.15.3:0/175981101 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5b141a7ea0 con 0x7f5b1409e000 2023-10-03T10:17:48.598 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:48.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.600+0000 7f5b1af44700 1 -- 172.21.15.3:0/175981101 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5b1409e000 msgr2=0x7f5b141c8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:48.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.600+0000 7f5b1af44700 1 --2- 172.21.15.3:0/175981101 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5b1409e000 0x7f5b141c8770 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000ebb0 tx=0x7f5b0000eec0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.600+0000 7f5b1af44700 1 -- 172.21.15.3:0/175981101 shutdown_connections 2023-10-03T10:17:48.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.600+0000 7f5b1af44700 1 --2- 172.21.15.3:0/175981101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b1409e410 0x7f5b141c8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.600+0000 7f5b1af44700 1 --2- 172.21.15.3:0/175981101 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b1409d6f0 0x7f5b141c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.600+0000 7f5b1af44700 1 --2- 172.21.15.3:0/175981101 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5b1409e000 0x7f5b141c8770 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.600+0000 7f5b1af44700 1 -- 172.21.15.3:0/175981101 >> 172.21.15.3:0/175981101 conn(0x7f5b14066f50 msgr2=0x7f5b141bbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:48.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.600+0000 7f5b1af44700 1 -- 172.21.15.3:0/175981101 shutdown_connections 2023-10-03T10:17:48.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.600+0000 7f5b1af44700 1 -- 172.21.15.3:0/175981101 wait complete. 2023-10-03T10:17:48.615 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:48.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.640+0000 7fe6eff3a700 1 -- 172.21.15.3:0/1784824602 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe6e81237d0 con 0x7fe6e80a0a30 2023-10-03T10:17:48.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.641+0000 7fe6de7fc700 1 -- 172.21.15.3:0/1784824602 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe6e81237d0 con 0x7fe6e80a0a30 2023-10-03T10:17:48.641 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:48.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.643+0000 7fe6eff3a700 1 -- 172.21.15.3:0/1784824602 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6e80a0a30 msgr2=0x7fe6e8120130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:48.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.643+0000 7fe6eff3a700 1 --2- 172.21.15.3:0/1784824602 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6e80a0a30 0x7fe6e8120130 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400e9b0 tx=0x7fe6e400ecc0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.643+0000 7fe6eff3a700 1 -- 172.21.15.3:0/1784824602 shutdown_connections 2023-10-03T10:17:48.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.643+0000 7fe6eff3a700 1 --2- 172.21.15.3:0/1784824602 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe6e8133fa0 0x7fe6e8120bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.643+0000 7fe6eff3a700 1 --2- 172.21.15.3:0/1784824602 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe6e80a0a30 0x7fe6e8120130 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.643+0000 7fe6eff3a700 1 --2- 172.21.15.3:0/1784824602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6e80a1340 0x7fe6e8120670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.643+0000 7fe6eff3a700 1 -- 172.21.15.3:0/1784824602 >> 172.21.15.3:0/1784824602 conn(0x7fe6e80a6550 msgr2=0x7fe6e81bb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:48.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.643+0000 7fe6eff3a700 1 -- 172.21.15.3:0/1784824602 shutdown_connections 2023-10-03T10:17:48.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.643+0000 7fe6eff3a700 1 -- 172.21.15.3:0/1784824602 wait complete. 2023-10-03T10:17:48.657 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:48.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.675+0000 7fc206412700 1 -- 172.21.15.3:0/3418630563 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc1e0005590 con 0x7fc2000a1510 2023-10-03T10:17:48.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.675+0000 7fc1fcff9700 1 -- 172.21.15.3:0/3418630563 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc1f4017020 con 0x7fc2000a1510 2023-10-03T10:17:48.675 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:48.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.677+0000 7fc206412700 1 -- 172.21.15.3:0/3418630563 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc2000a1510 msgr2=0x7fc2001c8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:48.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.677+0000 7fc206412700 1 --2- 172.21.15.3:0/3418630563 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc2000a1510 0x7fc2001c8b60 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400e9b0 tx=0x7fc1f400ecc0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.677+0000 7fc206412700 1 -- 172.21.15.3:0/3418630563 shutdown_connections 2023-10-03T10:17:48.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.677+0000 7fc206412700 1 --2- 172.21.15.3:0/3418630563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc200139250 0x7fc2001227d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.677+0000 7fc206412700 1 --2- 172.21.15.3:0/3418630563 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc2000a0a30 0x7fc2001c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.677+0000 7fc206412700 1 --2- 172.21.15.3:0/3418630563 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc2000a1510 0x7fc2001c8b60 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.677+0000 7fc206412700 1 -- 172.21.15.3:0/3418630563 >> 172.21.15.3:0/3418630563 conn(0x7fc2000a6550 msgr2=0x7fc2000a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.678+0000 7fc206412700 1 -- 172.21.15.3:0/3418630563 shutdown_connections 2023-10-03T10:17:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.678+0000 7fc206412700 1 -- 172.21.15.3:0/3418630563 wait complete. 2023-10-03T10:17:48.691 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:48.840 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:48.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.984+0000 7f88dcfa7700 1 Processor -- start 2023-10-03T10:17:48.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.984+0000 7f88dcfa7700 1 -- start start 2023-10-03T10:17:48.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.985+0000 7f88dcfa7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88d8136830 0x7f88d8138c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:48.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.985+0000 7f88dcfa7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f88d8139150 0x7f88d813b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:48.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.985+0000 7f88dcfa7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d806f720 con 0x7f88d81338c0 2023-10-03T10:17:48.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.985+0000 7f88dcfa7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88d8070c80 con 0x7f88d8139150 2023-10-03T10:17:48.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.985+0000 7f88dcfa7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f88d806fce0 con 0x7f88d8136830 2023-10-03T10:17:48.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.985+0000 7f88d5d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88d8136830 0x7f88d8138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:48.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.985+0000 7f88d659c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f88d81338c0 0x7f88d8133c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42492/0 (socket says 172.21.15.3:42492) 2023-10-03T10:17:48.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.985+0000 7f88d659c700 1 -- 172.21.15.3:0/3275705367 learned_addr learned my addr 172.21.15.3:0/3275705367 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:48.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.985+0000 7f88d6d9d700 1 --2- 172.21.15.3:0/3275705367 >> v2:172.21.15.3:3300/0 conn(0x7f88d8139150 0x7f88d813b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:48.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.985+0000 7f88d5d9b700 1 -- 172.21.15.3:0/3275705367 >> v2:172.21.15.3:3300/0 conn(0x7f88d8139150 msgr2=0x7f88d813b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:48.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.986+0000 7f88d5d9b700 1 --2- 172.21.15.3:0/3275705367 >> v2:172.21.15.3:3300/0 conn(0x7f88d8139150 0x7f88d813b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.986+0000 7f88d5d9b700 1 -- 172.21.15.3:0/3275705367 >> v1:172.21.15.3:6789/0 conn(0x7f88d81338c0 legacy=0x7f88d8133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:48.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.986+0000 7f88d5d9b700 1 -- 172.21.15.3:0/3275705367 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d81336a0 con 0x7f88d8136830 2023-10-03T10:17:48.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.986+0000 7f88d559a700 1 -- 172.21.15.3:0/3275705367 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3153952201 0 0) 0x7f88d806f720 con 0x7f88d81338c0 2023-10-03T10:17:48.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.986+0000 7f88d6d9d700 1 --2- 172.21.15.3:0/3275705367 >> v2:172.21.15.3:3300/0 conn(0x7f88d8139150 0x7f88d813b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:48.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.986+0000 7f88d5d9b700 1 --2- 172.21.15.3:0/3275705367 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88d8136830 0x7f88d8138c10 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f88c0009840 tx=0x7f88c0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3541b026025e20 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:48.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.986+0000 7f88d559a700 1 -- 172.21.15.3:0/3275705367 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c0011830 con 0x7f88d8136830 2023-10-03T10:17:48.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.986+0000 7f88d559a700 1 -- 172.21.15.3:0/3275705367 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c0011990 con 0x7f88d8136830 2023-10-03T10:17:48.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.986+0000 7f88d559a700 1 -- 172.21.15.3:0/3275705367 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c0011d20 con 0x7f88d8136830 2023-10-03T10:17:48.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.986+0000 7f88dcfa7700 1 -- 172.21.15.3:0/3275705367 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88d8136830 msgr2=0x7f88d8138c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:48.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.986+0000 7f88dcfa7700 1 --2- 172.21.15.3:0/3275705367 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88d8136830 0x7f88d8138c10 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f88c0009840 tx=0x7f88c0009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:48.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.987+0000 7f88dcfa7700 1 -- 172.21.15.3:0/3275705367 shutdown_connections 2023-10-03T10:17:48.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.987+0000 7f88dcfa7700 1 --2- 172.21.15.3:0/3275705367 >> v2:172.21.15.3:3300/0 conn(0x7f88d8139150 0x7f88d813b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.987+0000 7f88dcfa7700 1 --2- 172.21.15.3:0/3275705367 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88d8136830 0x7f88d8138c10 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.987+0000 7f88dcfa7700 1 -- 172.21.15.3:0/3275705367 >> 172.21.15.3:0/3275705367 conn(0x7f88d80a2aa0 msgr2=0x7f88d80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:48.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.987+0000 7f88dcfa7700 1 -- 172.21.15.3:0/3275705367 shutdown_connections 2023-10-03T10:17:48.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.987+0000 7f88dcfa7700 1 -- 172.21.15.3:0/3275705367 wait complete. 2023-10-03T10:17:48.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.988+0000 7f88dcfa7700 1 Processor -- start 2023-10-03T10:17:48.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.988+0000 7f88dcfa7700 1 -- start start 2023-10-03T10:17:48.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.988+0000 7f88dcfa7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88d81338c0 0x7f88d81c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:48.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.988+0000 7f88dcfa7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88d8136830 0x7f88d81c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:48.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.988+0000 7f88dcfa7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88d8139150 0x7f88d8119eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:48.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.988+0000 7f88dcfa7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d811dc10 con 0x7f88d8139150 2023-10-03T10:17:48.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.988+0000 7f88dcfa7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d81be2d0 con 0x7f88d81338c0 2023-10-03T10:17:48.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.988+0000 7f88dcfa7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f88d81c91e0 con 0x7f88d8136830 2023-10-03T10:17:48.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.989+0000 7f88d5d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88d8136830 0x7f88d81c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:48.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.989+0000 7f88d6d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88d8139150 0x7f88d8119eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:48.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.989+0000 7f88d5d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88d8136830 0x7f88d81c8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43314/0 (socket says 172.21.15.3:43314) 2023-10-03T10:17:48.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.989+0000 7f88d5d9b700 1 -- 172.21.15.3:0/3771416127 learned_addr learned my addr 172.21.15.3:0/3771416127 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:48.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.989+0000 7f88d659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88d81338c0 0x7f88d81c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:48.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.989+0000 7f88d6d9d700 1 -- 172.21.15.3:0/3771416127 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88d8136830 msgr2=0x7f88d81c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:48.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.989+0000 7f88d6d9d700 1 --2- 172.21.15.3:0/3771416127 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88d8136830 0x7f88d81c8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.989+0000 7f88d6d9d700 1 -- 172.21.15.3:0/3771416127 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88d81338c0 msgr2=0x7f88d81c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:48.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.989+0000 7f88d6d9d700 1 --2- 172.21.15.3:0/3771416127 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88d81338c0 0x7f88d81c8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:48.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.989+0000 7f88d6d9d700 1 -- 172.21.15.3:0/3771416127 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d0009a70 con 0x7f88d8139150 2023-10-03T10:17:48.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.989+0000 7f88d6d9d700 1 --2- 172.21.15.3:0/3771416127 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88d8139150 0x7f88d8119eb0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f88d806eb30 tx=0x7f88d000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:48.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.989+0000 7f88ceffd700 1 -- 172.21.15.3:0/3771416127 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d000cc20 con 0x7f88d8139150 2023-10-03T10:17:48.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.989+0000 7f88ceffd700 1 -- 172.21.15.3:0/3771416127 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d000f040 con 0x7f88d8139150 2023-10-03T10:17:48.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.989+0000 7f88dcfa7700 1 -- 172.21.15.3:0/3771416127 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c00094a0 con 0x7f88d8139150 2023-10-03T10:17:48.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.989+0000 7f88ceffd700 1 -- 172.21.15.3:0/3771416127 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d0014480 con 0x7f88d8139150 2023-10-03T10:17:48.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.989+0000 7f88dcfa7700 1 -- 172.21.15.3:0/3771416127 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d811d430 con 0x7f88d8139150 2023-10-03T10:17:48.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.990+0000 7f88ceffd700 1 -- 172.21.15.3:0/3771416127 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f88d0014600 con 0x7f88d8139150 2023-10-03T10:17:48.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.990+0000 7f88ceffd700 1 -- 172.21.15.3:0/3771416127 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f88d003fef0 con 0x7f88d8139150 2023-10-03T10:17:48.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.990+0000 7f88dcfa7700 1 -- 172.21.15.3:0/3771416127 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d8072000 con 0x7f88d8139150 2023-10-03T10:17:48.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:48.996+0000 7f88ceffd700 1 -- 172.21.15.3:0/3771416127 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f88d0018080 con 0x7f88d8139150 2023-10-03T10:17:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.201+0000 7f88dcfa7700 1 -- 172.21.15.3:0/3771416127 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f88d8074ea0 con 0x7f88d8139150 2023-10-03T10:17:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.201+0000 7f88ceffd700 1 -- 172.21.15.3:0/3771416127 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f88d00114a0 con 0x7f88d8139150 2023-10-03T10:17:49.202 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:49.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.203+0000 7f88dcfa7700 1 -- 172.21.15.3:0/3771416127 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88d8139150 msgr2=0x7f88d8119eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:49.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.203+0000 7f88dcfa7700 1 --2- 172.21.15.3:0/3771416127 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88d8139150 0x7f88d8119eb0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f88d806eb30 tx=0x7f88d000ef90 comp rx=0 tx=0).stop 2023-10-03T10:17:49.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.203+0000 7f88dcfa7700 1 -- 172.21.15.3:0/3771416127 shutdown_connections 2023-10-03T10:17:49.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.203+0000 7f88dcfa7700 1 --2- 172.21.15.3:0/3771416127 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88d8139150 0x7f88d8119eb0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:49.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.203+0000 7f88dcfa7700 1 --2- 172.21.15.3:0/3771416127 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88d81338c0 0x7f88d81c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:49.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.203+0000 7f88dcfa7700 1 --2- 172.21.15.3:0/3771416127 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f88d8136830 0x7f88d81c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:49.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.203+0000 7f88dcfa7700 1 -- 172.21.15.3:0/3771416127 >> 172.21.15.3:0/3771416127 conn(0x7f88d80a2aa0 msgr2=0x7f88d809de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:49.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.203+0000 7f88dcfa7700 1 -- 172.21.15.3:0/3771416127 shutdown_connections 2023-10-03T10:17:49.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.203+0000 7f88dcfa7700 1 -- 172.21.15.3:0/3771416127 wait complete. 2023-10-03T10:17:49.217 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:49.615 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:49.657 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:49.692 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:49.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.770+0000 7f40eb375700 1 Processor -- start 2023-10-03T10:17:49.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.770+0000 7f40eb375700 1 -- start start 2023-10-03T10:17:49.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.771+0000 7f40eb375700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f40e409d6f0 0x7f40e409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:49.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.771+0000 7f40eb375700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40e409e410 0x7f40e4130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:49.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.771+0000 7f40eb375700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40e409f130 con 0x7f40e409e000 2023-10-03T10:17:49.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.771+0000 7f40eb375700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40e409f520 con 0x7f40e409d6f0 2023-10-03T10:17:49.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.771+0000 7f40eb375700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e409e930 con 0x7f40e409e410 2023-10-03T10:17:49.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.771+0000 7f40e9111700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f40e409d6f0 0x7f40e409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:49.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.771+0000 7f40e9912700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40e409e410 0x7f40e4130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:49.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.771+0000 7f40e9111700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f40e409d6f0 0x7f40e409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59104/0 (socket says 172.21.15.3:59104) 2023-10-03T10:17:49.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.771+0000 7f40e9111700 1 -- 172.21.15.3:0/1964351439 learned_addr learned my addr 172.21.15.3:0/1964351439 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:49.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.772+0000 7f40e9111700 1 -- 172.21.15.3:0/1964351439 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40e409e410 msgr2=0x7f40e4130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:49.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.772+0000 7f40e9111700 1 --2- 172.21.15.3:0/1964351439 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40e409e410 0x7f40e4130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:49.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.772+0000 7f40e9111700 1 -- 172.21.15.3:0/1964351439 >> v1:172.21.15.3:6789/0 conn(0x7f40e409e000 legacy=0x7f40e4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:49.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.772+0000 7f40e9111700 1 -- 172.21.15.3:0/1964351439 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e409b2d0 con 0x7f40e409d6f0 2023-10-03T10:17:49.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.772+0000 7f40e9912700 1 --2- 172.21.15.3:0/1964351439 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40e409e410 0x7f40e4130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:49.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.772+0000 7f40e9111700 1 --2- 172.21.15.3:0/1964351439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e409d6f0 0x7f40e409dac0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f40e00098f0 tx=0x7f40e0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5f797ce4c6d21c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:49.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.772+0000 7f40dbfff700 1 -- 172.21.15.3:0/1964351439 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e0011830 con 0x7f40e409d6f0 2023-10-03T10:17:49.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.772+0000 7f40dbfff700 1 -- 172.21.15.3:0/1964351439 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e0011990 con 0x7f40e409d6f0 2023-10-03T10:17:49.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.772+0000 7f40dbfff700 1 -- 172.21.15.3:0/1964351439 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e0011d20 con 0x7f40e409d6f0 2023-10-03T10:17:49.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.773+0000 7f40eb375700 1 -- 172.21.15.3:0/1964351439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e409d6f0 msgr2=0x7f40e409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:49.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.773+0000 7f40eb375700 1 --2- 172.21.15.3:0/1964351439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e409d6f0 0x7f40e409dac0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f40e00098f0 tx=0x7f40e0009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:49.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.773+0000 7f40eb375700 1 -- 172.21.15.3:0/1964351439 shutdown_connections 2023-10-03T10:17:49.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.773+0000 7f40eb375700 1 --2- 172.21.15.3:0/1964351439 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40e409e410 0x7f40e4130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:49.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.773+0000 7f40eb375700 1 --2- 172.21.15.3:0/1964351439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e409d6f0 0x7f40e409dac0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:49.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.773+0000 7f40eb375700 1 -- 172.21.15.3:0/1964351439 >> 172.21.15.3:0/1964351439 conn(0x7f40e4066f50 msgr2=0x7f40e409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:49.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.773+0000 7f40eb375700 1 -- 172.21.15.3:0/1964351439 shutdown_connections 2023-10-03T10:17:49.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.773+0000 7f40eb375700 1 -- 172.21.15.3:0/1964351439 wait complete. 2023-10-03T10:17:49.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.774+0000 7f40eb375700 1 Processor -- start 2023-10-03T10:17:49.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.774+0000 7f40eb375700 1 -- start start 2023-10-03T10:17:49.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.774+0000 7f40eb375700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e409d6f0 0x7f40e41229c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:49.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.774+0000 7f40eb375700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40e409e000 0x7f40e4122f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:49.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.774+0000 7f40eb375700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40e409e410 0x7f40e4123440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:49.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.774+0000 7f40eb375700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e409a0c0 con 0x7f40e409d6f0 2023-10-03T10:17:49.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.774+0000 7f40eb375700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e4124ac0 con 0x7f40e409e410 2023-10-03T10:17:49.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.774+0000 7f40eb375700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40e4126ff0 con 0x7f40e409e000 2023-10-03T10:17:49.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.775+0000 7f40e9111700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e409d6f0 0x7f40e41229c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:49.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.775+0000 7f40e9912700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40e409e410 0x7f40e4123440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:49.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.775+0000 7f40e9912700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40e409e410 0x7f40e4123440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60500/0 (socket says 172.21.15.3:60500) 2023-10-03T10:17:49.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.775+0000 7f40e9111700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e409d6f0 0x7f40e41229c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59116/0 (socket says 172.21.15.3:59116) 2023-10-03T10:17:49.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.775+0000 7f40e9912700 1 -- 172.21.15.3:0/3512837272 learned_addr learned my addr 172.21.15.3:0/3512837272 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:49.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.775+0000 7f40e9912700 1 -- 172.21.15.3:0/3512837272 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40e409e000 msgr2=0x7f40e4122f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:49.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.775+0000 7f40e9912700 1 --2- 172.21.15.3:0/3512837272 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40e409e000 0x7f40e4122f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:49.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.775+0000 7f40e9912700 1 -- 172.21.15.3:0/3512837272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e409d6f0 msgr2=0x7f40e41229c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:49.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.775+0000 7f40e9912700 1 --2- 172.21.15.3:0/3512837272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e409d6f0 0x7f40e41229c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:49.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.775+0000 7f40e9912700 1 -- 172.21.15.3:0/3512837272 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40dc009a70 con 0x7f40e409e410 2023-10-03T10:17:49.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.775+0000 7f40e9111700 1 --2- 172.21.15.3:0/3512837272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e409d6f0 0x7f40e41229c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:49.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.775+0000 7f40e9912700 1 --2- 172.21.15.3:0/3512837272 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40e409e410 0x7f40e4123440 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f40dc00efc0 tx=0x7f40dc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:49.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.775+0000 7f40d9ffb700 1 -- 172.21.15.3:0/3512837272 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40dc00cb40 con 0x7f40e409e410 2023-10-03T10:17:49.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.775+0000 7f40d9ffb700 1 -- 172.21.15.3:0/3512837272 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40dc00f040 con 0x7f40e409e410 2023-10-03T10:17:49.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.775+0000 7f40eb375700 1 -- 172.21.15.3:0/3512837272 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40e0009550 con 0x7f40e409e410 2023-10-03T10:17:49.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.775+0000 7f40d9ffb700 1 -- 172.21.15.3:0/3512837272 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40dc003a20 con 0x7f40e409e410 2023-10-03T10:17:49.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.776+0000 7f40eb375700 1 -- 172.21.15.3:0/3512837272 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e4126130 con 0x7f40e409e410 2023-10-03T10:17:49.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.777+0000 7f40d9ffb700 1 -- 172.21.15.3:0/3512837272 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f40dc014480 con 0x7f40e409e410 2023-10-03T10:17:49.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.777+0000 7f40eb375700 1 -- 172.21.15.3:0/3512837272 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40e4124d50 con 0x7f40e409e410 2023-10-03T10:17:49.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.777+0000 7f40d9ffb700 1 -- 172.21.15.3:0/3512837272 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f40dc040130 con 0x7f40e409e410 2023-10-03T10:17:49.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.782+0000 7f40d9ffb700 1 -- 172.21.15.3:0/3512837272 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f40dc012930 con 0x7f40e409e410 2023-10-03T10:17:49.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.807+0000 7f1f5c9e4700 1 Processor -- start 2023-10-03T10:17:49.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.807+0000 7f1f5c9e4700 1 -- start start 2023-10-03T10:17:49.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.808+0000 7f1f5c9e4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1f580a0a30 0x7f1f580a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.808+0000 7f1f5c9e4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1f58134170 0x7f1f5813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.808+0000 7f1f5c9e4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f5809edd0 con 0x7f1f580a1340 2023-10-03T10:17:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.808+0000 7f1f5c9e4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5809e1f0 con 0x7f1f580a0a30 2023-10-03T10:17:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.808+0000 7f1f5c9e4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5809dc50 con 0x7f1f58134170 2023-10-03T10:17:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.808+0000 7f1f56d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1f58134170 0x7f1f5813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.808+0000 7f1f56d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1f58134170 0x7f1f5813bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43330/0 (socket says 172.21.15.3:43330) 2023-10-03T10:17:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.808+0000 7f1f56d9d700 1 -- 172.21.15.3:0/3002082871 learned_addr learned my addr 172.21.15.3:0/3002082871 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.808+0000 7f1f5659c700 1 --2- 172.21.15.3:0/3002082871 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1f580a0a30 0x7f1f580a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:49.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.809+0000 7f1f56d9d700 1 -- 172.21.15.3:0/3002082871 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1f580a0a30 msgr2=0x7f1f580a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:49.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.809+0000 7f1f56d9d700 1 --2- 172.21.15.3:0/3002082871 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1f580a0a30 0x7f1f580a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:49.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.809+0000 7f1f56d9d700 1 -- 172.21.15.3:0/3002082871 >> v1:172.21.15.3:6789/0 conn(0x7f1f580a1340 legacy=0x7f1f58133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:49.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.809+0000 7f1f56d9d700 1 -- 172.21.15.3:0/3002082871 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f580a1fe0 con 0x7f1f58134170 2023-10-03T10:17:49.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.809+0000 7f1f5559a700 1 -- 172.21.15.3:0/3002082871 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4006712086 0 0) 0x7f1f5809edd0 con 0x7f1f580a1340 2023-10-03T10:17:49.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.809+0000 7f1f56d9d700 1 --2- 172.21.15.3:0/3002082871 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1f58134170 0x7f1f5813bd30 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1f50009e80 tx=0x7f1f5000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36956462b98e68e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:49.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.809+0000 7f1f5559a700 1 -- 172.21.15.3:0/3002082871 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5000e070 con 0x7f1f58134170 2023-10-03T10:17:49.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.809+0000 7f1f5559a700 1 -- 172.21.15.3:0/3002082871 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5000b910 con 0x7f1f58134170 2023-10-03T10:17:49.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.809+0000 7f1f5559a700 1 -- 172.21.15.3:0/3002082871 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5000bcc0 con 0x7f1f58134170 2023-10-03T10:17:49.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.810+0000 7f1f5c9e4700 1 -- 172.21.15.3:0/3002082871 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1f58134170 msgr2=0x7f1f5813bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:49.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.810+0000 7f1f5c9e4700 1 --2- 172.21.15.3:0/3002082871 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1f58134170 0x7f1f5813bd30 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1f50009e80 tx=0x7f1f5000b330 comp rx=0 tx=0).stop 2023-10-03T10:17:49.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.810+0000 7f1f5c9e4700 1 -- 172.21.15.3:0/3002082871 shutdown_connections 2023-10-03T10:17:49.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.810+0000 7f1f5c9e4700 1 --2- 172.21.15.3:0/3002082871 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1f580a0a30 0x7f1f580a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:49.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.810+0000 7f1f5c9e4700 1 --2- 172.21.15.3:0/3002082871 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1f58134170 0x7f1f5813bd30 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:49.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.810+0000 7f1f5c9e4700 1 -- 172.21.15.3:0/3002082871 >> 172.21.15.3:0/3002082871 conn(0x7f1f580a6550 msgr2=0x7f1f580a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:49.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.810+0000 7f1f5c9e4700 1 -- 172.21.15.3:0/3002082871 shutdown_connections 2023-10-03T10:17:49.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.810+0000 7f1f5c9e4700 1 -- 172.21.15.3:0/3002082871 wait complete. 2023-10-03T10:17:49.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.812+0000 7f1f5c9e4700 1 Processor -- start 2023-10-03T10:17:49.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.812+0000 7f1f5c9e4700 1 -- start start 2023-10-03T10:17:49.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.813+0000 7f1f5c9e4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1f580a0a30 0x7f1f58122830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:49.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.813+0000 7f1f5c9e4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f580a1340 0x7f1f58122d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:49.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.813+0000 7f1f5c9e4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1f58134170 0x7f1f581232b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:49.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.813+0000 7f1f5c9e4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f580a4d10 con 0x7f1f580a1340 2023-10-03T10:17:49.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.813+0000 7f1f5c9e4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f581244c0 con 0x7f1f58134170 2023-10-03T10:17:49.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.813+0000 7f1f5c9e4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f580a0220 con 0x7f1f580a0a30 2023-10-03T10:17:49.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.813+0000 7f1f56d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1f58134170 0x7f1f581232b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:49.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.813+0000 7f1f55d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f580a1340 0x7f1f58122d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:49.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.813+0000 7f1f56d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1f58134170 0x7f1f581232b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60504/0 (socket says 172.21.15.3:60504) 2023-10-03T10:17:49.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.813+0000 7f1f56d9d700 1 -- 172.21.15.3:0/3048719962 learned_addr learned my addr 172.21.15.3:0/3048719962 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:49.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.813+0000 7f1f5659c700 1 --2- 172.21.15.3:0/3048719962 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1f580a0a30 0x7f1f58122830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:49.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.813+0000 7f1f55d9b700 1 -- 172.21.15.3:0/3048719962 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1f580a0a30 msgr2=0x7f1f58122830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:49.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.813+0000 7f1f55d9b700 1 --2- 172.21.15.3:0/3048719962 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1f580a0a30 0x7f1f58122830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:49.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.813+0000 7f1f55d9b700 1 -- 172.21.15.3:0/3048719962 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1f58134170 msgr2=0x7f1f581232b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:49.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.813+0000 7f1f55d9b700 1 --2- 172.21.15.3:0/3048719962 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1f58134170 0x7f1f581232b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:49.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.813+0000 7f1f55d9b700 1 -- 172.21.15.3:0/3048719962 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f50009b30 con 0x7f1f580a1340 2023-10-03T10:17:49.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.813+0000 7f1f56d9d700 1 --2- 172.21.15.3:0/3048719962 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1f58134170 0x7f1f581232b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:49.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.813+0000 7f1f5659c700 1 --2- 172.21.15.3:0/3048719962 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1f580a0a30 0x7f1f58122830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:49.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.814+0000 7f1f55d9b700 1 --2- 172.21.15.3:0/3048719962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f580a1340 0x7f1f58122d70 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00c430 tx=0x7f1f4c00c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:49.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.814+0000 7f1f46ffd700 1 -- 172.21.15.3:0/3048719962 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c0054e0 con 0x7f1f580a1340 2023-10-03T10:17:49.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.814+0000 7f1f46ffd700 1 -- 172.21.15.3:0/3048719962 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4c00ce40 con 0x7f1f580a1340 2023-10-03T10:17:49.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.814+0000 7f1f5c9e4700 1 -- 172.21.15.3:0/3048719962 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f580a25e0 con 0x7f1f580a1340 2023-10-03T10:17:49.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.814+0000 7f1f46ffd700 1 -- 172.21.15.3:0/3048719962 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c0038c0 con 0x7f1f580a1340 2023-10-03T10:17:49.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.814+0000 7f1f5c9e4700 1 -- 172.21.15.3:0/3048719962 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f581262c0 con 0x7f1f580a1340 2023-10-03T10:17:49.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.815+0000 7f1f46ffd700 1 -- 172.21.15.3:0/3048719962 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1f4c003a20 con 0x7f1f580a1340 2023-10-03T10:17:49.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.815+0000 7f1f5c9e4700 1 -- 172.21.15.3:0/3048719962 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f3c0052e0 con 0x7f1f580a1340 2023-10-03T10:17:49.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.815+0000 7f1f46ffd700 1 -- 172.21.15.3:0/3048719962 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1f4c03ea00 con 0x7f1f580a1340 2023-10-03T10:17:49.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.820+0000 7f1f46ffd700 1 -- 172.21.15.3:0/3048719962 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1f4c019b40 con 0x7f1f580a1340 2023-10-03T10:17:49.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.850+0000 7f13d2a07700 1 Processor -- start 2023-10-03T10:17:49.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.850+0000 7f13d2a07700 1 -- start start 2023-10-03T10:17:49.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.851+0000 7f13d2a07700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13cc09d630 0x7f13cc0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:49.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.851+0000 7f13d2a07700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f13cc138d80 0x7f13cc13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:49.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.851+0000 7f13d2a07700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13cc0a2a70 con 0x7f13cc0a0a30 2023-10-03T10:17:49.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.851+0000 7f13d2a07700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc0a01d0 con 0x7f13cc09d630 2023-10-03T10:17:49.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.851+0000 7f13d2a07700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc09e2b0 con 0x7f13cc138d80 2023-10-03T10:17:49.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.851+0000 7f13d0fa4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f13cc138d80 0x7f13cc13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:49.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.851+0000 7f13cb7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13cc09d630 0x7f13cc0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:49.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.851+0000 7f13cbfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f13cc0a0a30 0x7f13cc0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42518/0 (socket says 172.21.15.3:42518) 2023-10-03T10:17:49.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.851+0000 7f13cbfff700 1 -- 172.21.15.3:0/3354135513 learned_addr learned my addr 172.21.15.3:0/3354135513 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:49.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.851+0000 7f13d0fa4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f13cc138d80 0x7f13cc13b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43336/0 (socket says 172.21.15.3:43336) 2023-10-03T10:17:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.851+0000 7f13caffd700 1 -- 172.21.15.3:0/3354135513 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1019873379 0 0) 0x7f13cc0a2a70 con 0x7f13cc0a0a30 2023-10-03T10:17:49.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.852+0000 7f13cb7fe700 1 -- 172.21.15.3:0/3354135513 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f13cc138d80 msgr2=0x7f13cc13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:49.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.852+0000 7f13cb7fe700 1 --2- 172.21.15.3:0/3354135513 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f13cc138d80 0x7f13cc13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:49.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.852+0000 7f13cb7fe700 1 -- 172.21.15.3:0/3354135513 >> v1:172.21.15.3:6789/0 conn(0x7f13cc0a0a30 legacy=0x7f13cc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:49.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.852+0000 7f13cb7fe700 1 -- 172.21.15.3:0/3354135513 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13cc09dc50 con 0x7f13cc09d630 2023-10-03T10:17:49.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.852+0000 7f13d0fa4700 1 --2- 172.21.15.3:0/3354135513 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f13cc138d80 0x7f13cc13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:49.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.852+0000 7f13cb7fe700 1 --2- 172.21.15.3:0/3354135513 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13cc09d630 0x7f13cc0a1510 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009870 tx=0x7f13c0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed39d2a12983d00 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:49.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.852+0000 7f13caffd700 1 -- 172.21.15.3:0/3354135513 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c0011830 con 0x7f13cc09d630 2023-10-03T10:17:49.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.852+0000 7f13caffd700 1 -- 172.21.15.3:0/3354135513 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c0011990 con 0x7f13cc09d630 2023-10-03T10:17:49.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.852+0000 7f13caffd700 1 -- 172.21.15.3:0/3354135513 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c0011d20 con 0x7f13cc09d630 2023-10-03T10:17:49.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.853+0000 7f13d2a07700 1 -- 172.21.15.3:0/3354135513 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13cc09d630 msgr2=0x7f13cc0a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:49.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.853+0000 7f13d2a07700 1 --2- 172.21.15.3:0/3354135513 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13cc09d630 0x7f13cc0a1510 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009870 tx=0x7f13c0009b80 comp rx=0 tx=0).stop 2023-10-03T10:17:49.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.853+0000 7f13d2a07700 1 -- 172.21.15.3:0/3354135513 shutdown_connections 2023-10-03T10:17:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.853+0000 7f13d2a07700 1 --2- 172.21.15.3:0/3354135513 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f13cc138d80 0x7f13cc13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.853+0000 7f13d2a07700 1 --2- 172.21.15.3:0/3354135513 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13cc09d630 0x7f13cc0a1510 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.853+0000 7f13d2a07700 1 -- 172.21.15.3:0/3354135513 >> 172.21.15.3:0/3354135513 conn(0x7f13cc0a3050 msgr2=0x7f13cc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.853+0000 7f13d2a07700 1 -- 172.21.15.3:0/3354135513 shutdown_connections 2023-10-03T10:17:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.853+0000 7f13d2a07700 1 -- 172.21.15.3:0/3354135513 wait complete. 2023-10-03T10:17:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.854+0000 7f13d2a07700 1 Processor -- start 2023-10-03T10:17:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.854+0000 7f13d2a07700 1 -- start start 2023-10-03T10:17:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.855+0000 7f13d2a07700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f13cc09d630 0x7f13cc1338d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.855+0000 7f13d2a07700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13cc0a0a30 0x7f13cc133e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.855+0000 7f13d2a07700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13cc138d80 0x7f13cc134350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.855+0000 7f13d2a07700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc136dc0 con 0x7f13cc138d80 2023-10-03T10:17:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.855+0000 7f13d2a07700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc137080 con 0x7f13cc0a0a30 2023-10-03T10:17:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.855+0000 7f13d2a07700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc13ad70 con 0x7f13cc09d630 2023-10-03T10:17:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.855+0000 7f13cbfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f13cc09d630 0x7f13cc1338d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.855+0000 7f13cbfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f13cc09d630 0x7f13cc1338d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43352/0 (socket says 172.21.15.3:43352) 2023-10-03T10:17:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.855+0000 7f13cbfff700 1 -- 172.21.15.3:0/2524475350 learned_addr learned my addr 172.21.15.3:0/2524475350 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.855+0000 7f13d0fa4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13cc138d80 0x7f13cc134350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.855+0000 7f13cb7fe700 1 --2- 172.21.15.3:0/2524475350 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13cc0a0a30 0x7f13cc133e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.855+0000 7f13cbfff700 1 -- 172.21.15.3:0/2524475350 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13cc0a0a30 msgr2=0x7f13cc133e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.855+0000 7f13cbfff700 1 --2- 172.21.15.3:0/2524475350 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13cc0a0a30 0x7f13cc133e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.855+0000 7f13cbfff700 1 -- 172.21.15.3:0/2524475350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13cc138d80 msgr2=0x7f13cc134350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.855+0000 7f13cbfff700 1 --2- 172.21.15.3:0/2524475350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13cc138d80 0x7f13cc134350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.855+0000 7f13cbfff700 1 -- 172.21.15.3:0/2524475350 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c4009a70 con 0x7f13cc09d630 2023-10-03T10:17:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.855+0000 7f13cbfff700 1 --2- 172.21.15.3:0/2524475350 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f13cc09d630 0x7f13cc1338d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f13bc00c520 tx=0x7f13bc00c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.855+0000 7f13c8ff9700 1 -- 172.21.15.3:0/2524475350 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc004c80 con 0x7f13cc09d630 2023-10-03T10:17:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.856+0000 7f13c8ff9700 1 -- 172.21.15.3:0/2524475350 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13bc004e00 con 0x7f13cc09d630 2023-10-03T10:17:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.856+0000 7f13d2a07700 1 -- 172.21.15.3:0/2524475350 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c00094a0 con 0x7f13cc09d630 2023-10-03T10:17:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.856+0000 7f13c8ff9700 1 -- 172.21.15.3:0/2524475350 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc007310 con 0x7f13cc09d630 2023-10-03T10:17:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.856+0000 7f13d2a07700 1 -- 172.21.15.3:0/2524475350 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13cc137af0 con 0x7f13cc09d630 2023-10-03T10:17:49.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.857+0000 7f13d2a07700 1 -- 172.21.15.3:0/2524475350 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13cc136730 con 0x7f13cc09d630 2023-10-03T10:17:49.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.857+0000 7f13c8ff9700 1 -- 172.21.15.3:0/2524475350 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f13bc007470 con 0x7f13cc09d630 2023-10-03T10:17:49.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.857+0000 7f13c8ff9700 1 -- 172.21.15.3:0/2524475350 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f13bc013db0 con 0x7f13cc09d630 2023-10-03T10:17:49.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:49.862+0000 7f13c8ff9700 1 -- 172.21.15.3:0/2524475350 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f13bc044060 con 0x7f13cc09d630 2023-10-03T10:17:50.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.005+0000 7f40eb375700 1 -- 172.21.15.3:0/3512837272 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f40e4074ea0 con 0x7f40e409e410 2023-10-03T10:17:50.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.005+0000 7f40d9ffb700 1 -- 172.21.15.3:0/3512837272 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f40dc01c3b0 con 0x7f40e409e410 2023-10-03T10:17:50.006 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:50.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.008+0000 7f40eb375700 1 -- 172.21.15.3:0/3512837272 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40e409e410 msgr2=0x7f40e4123440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:50.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.008+0000 7f40eb375700 1 --2- 172.21.15.3:0/3512837272 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40e409e410 0x7f40e4123440 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f40dc00efc0 tx=0x7f40dc00c3a0 comp rx=0 tx=0).stop 2023-10-03T10:17:50.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.008+0000 7f40eb375700 1 -- 172.21.15.3:0/3512837272 shutdown_connections 2023-10-03T10:17:50.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.008+0000 7f40eb375700 1 --2- 172.21.15.3:0/3512837272 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f40e409e410 0x7f40e4123440 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:50.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.008+0000 7f40eb375700 1 --2- 172.21.15.3:0/3512837272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e409d6f0 0x7f40e41229c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:50.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.008+0000 7f40eb375700 1 --2- 172.21.15.3:0/3512837272 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f40e409e000 0x7f40e4122f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:50.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.008+0000 7f40eb375700 1 -- 172.21.15.3:0/3512837272 >> 172.21.15.3:0/3512837272 conn(0x7f40e4066f50 msgr2=0x7f40e40711a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:50.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.008+0000 7f40eb375700 1 -- 172.21.15.3:0/3512837272 shutdown_connections 2023-10-03T10:17:50.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.008+0000 7f40eb375700 1 -- 172.21.15.3:0/3512837272 wait complete. 2023-10-03T10:17:50.022 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:50.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.028+0000 7f1f5c9e4700 1 -- 172.21.15.3:0/3048719962 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1f3c005590 con 0x7f1f580a1340 2023-10-03T10:17:50.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.029+0000 7f1f46ffd700 1 -- 172.21.15.3:0/3048719962 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1f4c011e60 con 0x7f1f580a1340 2023-10-03T10:17:50.029 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:50.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.031+0000 7f1f5c9e4700 1 -- 172.21.15.3:0/3048719962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f580a1340 msgr2=0x7f1f58122d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:50.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.031+0000 7f1f5c9e4700 1 --2- 172.21.15.3:0/3048719962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f580a1340 0x7f1f58122d70 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00c430 tx=0x7f1f4c00c740 comp rx=0 tx=0).stop 2023-10-03T10:17:50.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.031+0000 7f1f5c9e4700 1 -- 172.21.15.3:0/3048719962 shutdown_connections 2023-10-03T10:17:50.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.031+0000 7f1f5c9e4700 1 --2- 172.21.15.3:0/3048719962 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1f58134170 0x7f1f581232b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:50.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.031+0000 7f1f5c9e4700 1 --2- 172.21.15.3:0/3048719962 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1f580a0a30 0x7f1f58122830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:50.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.031+0000 7f1f5c9e4700 1 --2- 172.21.15.3:0/3048719962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f580a1340 0x7f1f58122d70 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:50.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.031+0000 7f1f5c9e4700 1 -- 172.21.15.3:0/3048719962 >> 172.21.15.3:0/3048719962 conn(0x7f1f580a6550 msgr2=0x7f1f5809edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:50.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.031+0000 7f1f5c9e4700 1 -- 172.21.15.3:0/3048719962 shutdown_connections 2023-10-03T10:17:50.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.031+0000 7f1f5c9e4700 1 -- 172.21.15.3:0/3048719962 wait complete. 2023-10-03T10:17:50.045 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:50.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.082+0000 7f13d2a07700 1 -- 172.21.15.3:0/2524475350 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f13cc0747f0 con 0x7f13cc09d630 2023-10-03T10:17:50.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.083+0000 7f13c8ff9700 1 -- 172.21.15.3:0/2524475350 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f13bc044070 con 0x7f13cc09d630 2023-10-03T10:17:50.083 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:50.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.085+0000 7f13d2a07700 1 -- 172.21.15.3:0/2524475350 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f13cc09d630 msgr2=0x7f13cc1338d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:50.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.085+0000 7f13d2a07700 1 --2- 172.21.15.3:0/2524475350 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f13cc09d630 0x7f13cc1338d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f13bc00c520 tx=0x7f13bc00c830 comp rx=0 tx=0).stop 2023-10-03T10:17:50.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.085+0000 7f13d2a07700 1 -- 172.21.15.3:0/2524475350 shutdown_connections 2023-10-03T10:17:50.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.085+0000 7f13d2a07700 1 --2- 172.21.15.3:0/2524475350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13cc138d80 0x7f13cc134350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:50.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.085+0000 7f13d2a07700 1 --2- 172.21.15.3:0/2524475350 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f13cc09d630 0x7f13cc1338d0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:50.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.085+0000 7f13d2a07700 1 --2- 172.21.15.3:0/2524475350 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13cc0a0a30 0x7f13cc133e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:50.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.085+0000 7f13d2a07700 1 -- 172.21.15.3:0/2524475350 >> 172.21.15.3:0/2524475350 conn(0x7f13cc0a3050 msgr2=0x7f13cc09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:50.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.085+0000 7f13d2a07700 1 -- 172.21.15.3:0/2524475350 shutdown_connections 2023-10-03T10:17:50.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.085+0000 7f13d2a07700 1 -- 172.21.15.3:0/2524475350 wait complete. 2023-10-03T10:17:50.099 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:50.219 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:50.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.362+0000 7fb049128700 1 Processor -- start 2023-10-03T10:17:50.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.362+0000 7fb049128700 1 -- start start 2023-10-03T10:17:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.363+0000 7fb049128700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb04409e1d0 0x7fb044135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.363+0000 7fb049128700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb044135e50 0x7fb044138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.363+0000 7fb049128700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb04409f130 con 0x7fb04409d6f0 2023-10-03T10:17:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.363+0000 7fb049128700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb04409f520 con 0x7fb04409e1d0 2023-10-03T10:17:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.363+0000 7fb049128700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb04409e930 con 0x7fb044135e50 2023-10-03T10:17:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.363+0000 7fb042d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb04409d6f0 0x7fb04409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42522/0 (socket says 172.21.15.3:42522) 2023-10-03T10:17:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.363+0000 7fb042d9d700 1 -- 172.21.15.3:0/3131854652 learned_addr learned my addr 172.21.15.3:0/3131854652 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.363+0000 7fb04259c700 1 --2- 172.21.15.3:0/3131854652 >> v2:172.21.15.3:3300/0 conn(0x7fb04409e1d0 0x7fb044135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.363+0000 7fb04359e700 1 --2- 172.21.15.3:0/3131854652 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb044135e50 0x7fb044138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:50.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.363+0000 7fb04259c700 1 -- 172.21.15.3:0/3131854652 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb044135e50 msgr2=0x7fb044138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:50.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.363+0000 7fb04259c700 1 --2- 172.21.15.3:0/3131854652 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb044135e50 0x7fb044138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:50.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.363+0000 7fb04259c700 1 -- 172.21.15.3:0/3131854652 >> v1:172.21.15.3:6789/0 conn(0x7fb04409d6f0 legacy=0x7fb04409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:50.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.363+0000 7fb04259c700 1 -- 172.21.15.3:0/3131854652 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04409b2d0 con 0x7fb04409e1d0 2023-10-03T10:17:50.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.363+0000 7fb041d9b700 1 -- 172.21.15.3:0/3131854652 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2058479663 0 0) 0x7fb04409f130 con 0x7fb04409d6f0 2023-10-03T10:17:50.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.363+0000 7fb04359e700 1 --2- 172.21.15.3:0/3131854652 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb044135e50 0x7fb044138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:50.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.364+0000 7fb04259c700 1 --2- 172.21.15.3:0/3131854652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb04409e1d0 0x7fb044135910 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fb038009840 tx=0x7fb038009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3f8c49bb28d861 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:50.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.364+0000 7fb041d9b700 1 -- 172.21.15.3:0/3131854652 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb038011830 con 0x7fb04409e1d0 2023-10-03T10:17:50.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.364+0000 7fb041d9b700 1 -- 172.21.15.3:0/3131854652 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb038011990 con 0x7fb04409e1d0 2023-10-03T10:17:50.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.364+0000 7fb041d9b700 1 -- 172.21.15.3:0/3131854652 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb038011d20 con 0x7fb04409e1d0 2023-10-03T10:17:50.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.364+0000 7fb049128700 1 -- 172.21.15.3:0/3131854652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb04409e1d0 msgr2=0x7fb044135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:50.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.364+0000 7fb049128700 1 --2- 172.21.15.3:0/3131854652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb04409e1d0 0x7fb044135910 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fb038009840 tx=0x7fb038009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:50.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.364+0000 7fb049128700 1 -- 172.21.15.3:0/3131854652 shutdown_connections 2023-10-03T10:17:50.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.364+0000 7fb049128700 1 --2- 172.21.15.3:0/3131854652 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb044135e50 0x7fb044138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:50.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.364+0000 7fb049128700 1 --2- 172.21.15.3:0/3131854652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb04409e1d0 0x7fb044135910 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:50.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.364+0000 7fb049128700 1 -- 172.21.15.3:0/3131854652 >> 172.21.15.3:0/3131854652 conn(0x7fb044066f50 msgr2=0x7fb04409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:50.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.364+0000 7fb049128700 1 -- 172.21.15.3:0/3131854652 shutdown_connections 2023-10-03T10:17:50.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.364+0000 7fb049128700 1 -- 172.21.15.3:0/3131854652 wait complete. 2023-10-03T10:17:50.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.366+0000 7fb049128700 1 Processor -- start 2023-10-03T10:17:50.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.366+0000 7fb049128700 1 -- start start 2023-10-03T10:17:50.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.366+0000 7fb049128700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb04409d6f0 0x7fb044136250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:50.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.366+0000 7fb049128700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb04409e1d0 0x7fb044136790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:50.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.366+0000 7fb049128700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb044135e50 0x7fb044136cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:50.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.366+0000 7fb049128700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb044121c10 con 0x7fb044135e50 2023-10-03T10:17:50.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.366+0000 7fb049128700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04409c700 con 0x7fb04409d6f0 2023-10-03T10:17:50.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.366+0000 7fb049128700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb044137f70 con 0x7fb04409e1d0 2023-10-03T10:17:50.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.366+0000 7fb042d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb04409d6f0 0x7fb044136250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:50.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.366+0000 7fb04359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb044135e50 0x7fb044136cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:50.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.366+0000 7fb042d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb04409d6f0 0x7fb044136250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60528/0 (socket says 172.21.15.3:60528) 2023-10-03T10:17:50.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.366+0000 7fb04359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb044135e50 0x7fb044136cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59156/0 (socket says 172.21.15.3:59156) 2023-10-03T10:17:50.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.366+0000 7fb042d9d700 1 -- 172.21.15.3:0/2894015724 learned_addr learned my addr 172.21.15.3:0/2894015724 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:50.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.366+0000 7fb04259c700 1 --2- 172.21.15.3:0/2894015724 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb04409e1d0 0x7fb044136790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:50.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.367+0000 7fb04359e700 1 -- 172.21.15.3:0/2894015724 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb04409e1d0 msgr2=0x7fb044136790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:50.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.367+0000 7fb04359e700 1 --2- 172.21.15.3:0/2894015724 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb04409e1d0 0x7fb044136790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:50.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.367+0000 7fb04359e700 1 -- 172.21.15.3:0/2894015724 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb04409d6f0 msgr2=0x7fb044136250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.367+0000 7fb04359e700 1 --2- 172.21.15.3:0/2894015724 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb04409d6f0 0x7fb044136250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.367+0000 7fb04359e700 1 -- 172.21.15.3:0/2894015724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c009a70 con 0x7fb044135e50 2023-10-03T10:17:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.367+0000 7fb042d9d700 1 --2- 172.21.15.3:0/2894015724 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb04409d6f0 0x7fb044136250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.367+0000 7fb04359e700 1 --2- 172.21.15.3:0/2894015724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb044135e50 0x7fb044136cd0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00ef90 tx=0x7fb03c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.367+0000 7fb02b7fe700 1 -- 172.21.15.3:0/2894015724 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c00ca90 con 0x7fb044135e50 2023-10-03T10:17:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.367+0000 7fb02b7fe700 1 -- 172.21.15.3:0/2894015724 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03c00f040 con 0x7fb044135e50 2023-10-03T10:17:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.367+0000 7fb049128700 1 -- 172.21.15.3:0/2894015724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0380094a0 con 0x7fb044135e50 2023-10-03T10:17:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.367+0000 7fb02b7fe700 1 -- 172.21.15.3:0/2894015724 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c00ce30 con 0x7fb044135e50 2023-10-03T10:17:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.367+0000 7fb049128700 1 -- 172.21.15.3:0/2894015724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb044122680 con 0x7fb044135e50 2023-10-03T10:17:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.368+0000 7fb049128700 1 -- 172.21.15.3:0/2894015724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb04406b0a0 con 0x7fb044135e50 2023-10-03T10:17:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.369+0000 7fb02b7fe700 1 -- 172.21.15.3:0/2894015724 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb03c003a20 con 0x7fb044135e50 2023-10-03T10:17:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.369+0000 7fb02b7fe700 1 -- 172.21.15.3:0/2894015724 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb03c040650 con 0x7fb044135e50 2023-10-03T10:17:50.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.374+0000 7fb02b7fe700 1 -- 172.21.15.3:0/2894015724 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb03c01c7a0 con 0x7fb044135e50 2023-10-03T10:17:50.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.579+0000 7fb049128700 1 -- 172.21.15.3:0/2894015724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb044074ea0 con 0x7fb044135e50 2023-10-03T10:17:50.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.579+0000 7fb02b7fe700 1 -- 172.21.15.3:0/2894015724 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb03c02b090 con 0x7fb044135e50 2023-10-03T10:17:50.580 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:50.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.581+0000 7fb049128700 1 -- 172.21.15.3:0/2894015724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb044135e50 msgr2=0x7fb044136cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:50.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.581+0000 7fb049128700 1 --2- 172.21.15.3:0/2894015724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb044135e50 0x7fb044136cd0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00ef90 tx=0x7fb03c00c3a0 comp rx=0 tx=0).stop 2023-10-03T10:17:50.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.581+0000 7fb049128700 1 -- 172.21.15.3:0/2894015724 shutdown_connections 2023-10-03T10:17:50.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.582+0000 7fb049128700 1 --2- 172.21.15.3:0/2894015724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb044135e50 0x7fb044136cd0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:50.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.582+0000 7fb049128700 1 --2- 172.21.15.3:0/2894015724 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb04409d6f0 0x7fb044136250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:50.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.582+0000 7fb049128700 1 --2- 172.21.15.3:0/2894015724 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb04409e1d0 0x7fb044136790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:50.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.582+0000 7fb049128700 1 -- 172.21.15.3:0/2894015724 >> 172.21.15.3:0/2894015724 conn(0x7fb044066f50 msgr2=0x7fb04406ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:50.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.582+0000 7fb049128700 1 -- 172.21.15.3:0/2894015724 shutdown_connections 2023-10-03T10:17:50.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:50.582+0000 7fb049128700 1 -- 172.21.15.3:0/2894015724 wait complete. 2023-10-03T10:17:50.596 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:51.023 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:51.046 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:51.101 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:51.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.173+0000 7f3eb7219700 1 Processor -- start 2023-10-03T10:17:51.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.173+0000 7f3eb7219700 1 -- start start 2023-10-03T10:17:51.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.173+0000 7f3eb7219700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3eb0130540 0x7f3eb0130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:51.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.173+0000 7f3eb7219700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3eb0130e50 0x7f3eb006f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:51.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.173+0000 7f3eb7219700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eb00716a0 con 0x7f3eb006fb20 2023-10-03T10:17:51.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.173+0000 7f3eb7219700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb009a390 con 0x7f3eb0130540 2023-10-03T10:17:51.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.173+0000 7f3eb7219700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb0071290 con 0x7f3eb0130e50 2023-10-03T10:17:51.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.174+0000 7f3eaffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3eb0130e50 0x7f3eb006f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:51.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.174+0000 7f3eb4fb5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3eb0130540 0x7f3eb0130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:51.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.174+0000 7f3eb57b6700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3eb006fb20 0x7f3eb0137560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42532/0 (socket says 172.21.15.3:42532) 2023-10-03T10:17:51.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.174+0000 7f3eb57b6700 1 -- 172.21.15.3:0/3802796643 learned_addr learned my addr 172.21.15.3:0/3802796643 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:51.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.174+0000 7f3eaf7fe700 1 -- 172.21.15.3:0/3802796643 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390244775 0 0) 0x7f3eb00716a0 con 0x7f3eb006fb20 2023-10-03T10:17:51.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.174+0000 7f3eb4fb5700 1 -- 172.21.15.3:0/3802796643 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3eb0130e50 msgr2=0x7f3eb006f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:51.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.174+0000 7f3eb4fb5700 1 --2- 172.21.15.3:0/3802796643 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3eb0130e50 0x7f3eb006f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.174+0000 7f3eb4fb5700 1 -- 172.21.15.3:0/3802796643 >> v1:172.21.15.3:6789/0 conn(0x7f3eb006fb20 legacy=0x7f3eb0137560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:51.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.174+0000 7f3eb4fb5700 1 -- 172.21.15.3:0/3802796643 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb0130340 con 0x7f3eb0130540 2023-10-03T10:17:51.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.174+0000 7f3eaffff700 1 --2- 172.21.15.3:0/3802796643 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3eb0130e50 0x7f3eb006f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:51.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.174+0000 7f3eb4fb5700 1 --2- 172.21.15.3:0/3802796643 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3eb0130540 0x7f3eb0130910 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40098a0 tx=0x7f3ea4009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76b8543aa4640a6e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:51.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.174+0000 7f3eaf7fe700 1 -- 172.21.15.3:0/3802796643 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4011830 con 0x7f3eb0130540 2023-10-03T10:17:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.175+0000 7f3eaf7fe700 1 -- 172.21.15.3:0/3802796643 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea4011990 con 0x7f3eb0130540 2023-10-03T10:17:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.175+0000 7f3eaf7fe700 1 -- 172.21.15.3:0/3802796643 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4011d20 con 0x7f3eb0130540 2023-10-03T10:17:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.175+0000 7f3eb7219700 1 -- 172.21.15.3:0/3802796643 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3eb0130540 msgr2=0x7f3eb0130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.175+0000 7f3eb7219700 1 --2- 172.21.15.3:0/3802796643 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3eb0130540 0x7f3eb0130910 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40098a0 tx=0x7f3ea4009bb0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.175+0000 7f3eb7219700 1 -- 172.21.15.3:0/3802796643 shutdown_connections 2023-10-03T10:17:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.175+0000 7f3eb7219700 1 --2- 172.21.15.3:0/3802796643 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3eb0130540 0x7f3eb0130910 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.175+0000 7f3eb7219700 1 --2- 172.21.15.3:0/3802796643 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3eb0130e50 0x7f3eb006f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.175+0000 7f3eb7219700 1 -- 172.21.15.3:0/3802796643 >> 172.21.15.3:0/3802796643 conn(0x7f3eb00a2aa0 msgr2=0x7f3eb00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.175+0000 7f3eb7219700 1 -- 172.21.15.3:0/3802796643 shutdown_connections 2023-10-03T10:17:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.175+0000 7f3eb7219700 1 -- 172.21.15.3:0/3802796643 wait complete. 2023-10-03T10:17:51.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.176+0000 7f3eb7219700 1 Processor -- start 2023-10-03T10:17:51.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.176+0000 7f3eb7219700 1 -- start start 2023-10-03T10:17:51.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.177+0000 7f3eb7219700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3eb006fb20 0x7f3eb012acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:51.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.177+0000 7f3eb7219700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3eb0130540 0x7f3eb012b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:51.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.177+0000 7f3eb7219700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3eb0130e50 0x7f3eb01c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:51.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.177+0000 7f3eb7219700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb012d9d0 con 0x7f3eb0130e50 2023-10-03T10:17:51.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.177+0000 7f3eb7219700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb012a8a0 con 0x7f3eb006fb20 2023-10-03T10:17:51.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.177+0000 7f3eb7219700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb0138f60 con 0x7f3eb0130540 2023-10-03T10:17:51.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.177+0000 7f3eb4fb5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3eb006fb20 0x7f3eb012acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:51.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.177+0000 7f3eb4fb5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3eb006fb20 0x7f3eb012acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60550/0 (socket says 172.21.15.3:60550) 2023-10-03T10:17:51.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.177+0000 7f3eb4fb5700 1 -- 172.21.15.3:0/2930342808 learned_addr learned my addr 172.21.15.3:0/2930342808 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:51.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.177+0000 7f3eb57b6700 1 --2- 172.21.15.3:0/2930342808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3eb0130e50 0x7f3eb01c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:51.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.177+0000 7f3eaffff700 1 --2- 172.21.15.3:0/2930342808 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3eb0130540 0x7f3eb012b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:51.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.178+0000 7f3eb4fb5700 1 -- 172.21.15.3:0/2930342808 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3eb0130540 msgr2=0x7f3eb012b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:51.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.178+0000 7f3eb4fb5700 1 --2- 172.21.15.3:0/2930342808 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3eb0130540 0x7f3eb012b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.178+0000 7f3eb4fb5700 1 -- 172.21.15.3:0/2930342808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3eb0130e50 msgr2=0x7f3eb01c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:51.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.178+0000 7f3eb4fb5700 1 --2- 172.21.15.3:0/2930342808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3eb0130e50 0x7f3eb01c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.178+0000 7f3eb4fb5700 1 -- 172.21.15.3:0/2930342808 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9c0094a0 con 0x7f3eb006fb20 2023-10-03T10:17:51.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.178+0000 7f3eb4fb5700 1 --2- 172.21.15.3:0/2930342808 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3eb006fb20 0x7f3eb012acd0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4009840 tx=0x7f3ea4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:51.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.178+0000 7f3ead7fa700 1 -- 172.21.15.3:0/2930342808 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4004240 con 0x7f3eb006fb20 2023-10-03T10:17:51.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.178+0000 7f3ead7fa700 1 -- 172.21.15.3:0/2930342808 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea401f070 con 0x7f3eb006fb20 2023-10-03T10:17:51.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.178+0000 7f3eb7219700 1 -- 172.21.15.3:0/2930342808 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea40094a0 con 0x7f3eb006fb20 2023-10-03T10:17:51.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.178+0000 7f3ead7fa700 1 -- 172.21.15.3:0/2930342808 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea40045e0 con 0x7f3eb006fb20 2023-10-03T10:17:51.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.178+0000 7f3eb7219700 1 -- 172.21.15.3:0/2930342808 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb0131740 con 0x7f3eb006fb20 2023-10-03T10:17:51.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.179+0000 7f3ead7fa700 1 -- 172.21.15.3:0/2930342808 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3ea4021030 con 0x7f3eb006fb20 2023-10-03T10:17:51.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.179+0000 7f3eb7219700 1 -- 172.21.15.3:0/2930342808 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eb0074ea0 con 0x7f3eb006fb20 2023-10-03T10:17:51.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.179+0000 7f3ead7fa700 1 -- 172.21.15.3:0/2930342808 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3ea4040870 con 0x7f3eb006fb20 2023-10-03T10:17:51.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.184+0000 7f3ead7fa700 1 -- 172.21.15.3:0/2930342808 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3ea40197f0 con 0x7f3eb006fb20 2023-10-03T10:17:51.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.211+0000 7f6d24e83700 1 Processor -- start 2023-10-03T10:17:51.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.211+0000 7f6d24e83700 1 -- start start 2023-10-03T10:17:51.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.212+0000 7f6d24e83700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6d2009d6f0 0x7f6d2009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:51.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.212+0000 7f6d24e83700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d2009e410 0x7f6d20130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:51.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.212+0000 7f6d24e83700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d2009f130 con 0x7f6d2009e000 2023-10-03T10:17:51.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.212+0000 7f6d24e83700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d2009f520 con 0x7f6d2009d6f0 2023-10-03T10:17:51.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.212+0000 7f6d24e83700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2009e930 con 0x7f6d2009e410 2023-10-03T10:17:51.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.212+0000 7f6d1e59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6d2009d6f0 0x7f6d2009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:51.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.212+0000 7f6d1ed9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d2009e410 0x7f6d20130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:51.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.212+0000 7f6d1dd9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6d2009e000 0x7f6d20130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42538/0 (socket says 172.21.15.3:42538) 2023-10-03T10:17:51.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.212+0000 7f6d1ed9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d2009e410 0x7f6d20130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60556/0 (socket says 172.21.15.3:60556) 2023-10-03T10:17:51.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.212+0000 7f6d1dd9b700 1 -- 172.21.15.3:0/1638641499 learned_addr learned my addr 172.21.15.3:0/1638641499 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:51.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.212+0000 7f6d1e59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6d2009d6f0 0x7f6d2009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59166/0 (socket says 172.21.15.3:59166) 2023-10-03T10:17:51.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.213+0000 7f6d1e59c700 1 -- 172.21.15.3:0/1638641499 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d2009e410 msgr2=0x7f6d20130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:51.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.213+0000 7f6d1e59c700 1 --2- 172.21.15.3:0/1638641499 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d2009e410 0x7f6d20130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.213+0000 7f6d1e59c700 1 -- 172.21.15.3:0/1638641499 >> v1:172.21.15.3:6789/0 conn(0x7f6d2009e000 legacy=0x7f6d20130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:51.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.213+0000 7f6d1d59a700 1 -- 172.21.15.3:0/1638641499 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1892711189 0 0) 0x7f6d2009f130 con 0x7f6d2009e000 2023-10-03T10:17:51.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.213+0000 7f6d1e59c700 1 -- 172.21.15.3:0/1638641499 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d2009b2d0 con 0x7f6d2009d6f0 2023-10-03T10:17:51.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.213+0000 7f6d1ed9d700 1 --2- 172.21.15.3:0/1638641499 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d2009e410 0x7f6d20130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:51.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.213+0000 7f6d1e59c700 1 --2- 172.21.15.3:0/1638641499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d2009d6f0 0x7f6d2009dac0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f6d10009840 tx=0x7f6d10009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b95f6785694857 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:51.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.213+0000 7f6d1d59a700 1 -- 172.21.15.3:0/1638641499 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d10011830 con 0x7f6d2009d6f0 2023-10-03T10:17:51.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.213+0000 7f6d1d59a700 1 -- 172.21.15.3:0/1638641499 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d10011990 con 0x7f6d2009d6f0 2023-10-03T10:17:51.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.213+0000 7f6d1d59a700 1 -- 172.21.15.3:0/1638641499 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d10011d20 con 0x7f6d2009d6f0 2023-10-03T10:17:51.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.214+0000 7f6d24e83700 1 -- 172.21.15.3:0/1638641499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d2009d6f0 msgr2=0x7f6d2009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:51.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.214+0000 7f6d24e83700 1 --2- 172.21.15.3:0/1638641499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d2009d6f0 0x7f6d2009dac0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f6d10009840 tx=0x7f6d10009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:51.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.214+0000 7f6d24e83700 1 -- 172.21.15.3:0/1638641499 shutdown_connections 2023-10-03T10:17:51.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.214+0000 7f6d24e83700 1 --2- 172.21.15.3:0/1638641499 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d2009e410 0x7f6d20130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.214+0000 7f6d24e83700 1 --2- 172.21.15.3:0/1638641499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d2009d6f0 0x7f6d2009dac0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.214+0000 7f6d24e83700 1 -- 172.21.15.3:0/1638641499 >> 172.21.15.3:0/1638641499 conn(0x7f6d20066f50 msgr2=0x7f6d2009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:51.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.214+0000 7f6d24e83700 1 -- 172.21.15.3:0/1638641499 shutdown_connections 2023-10-03T10:17:51.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.214+0000 7f6d24e83700 1 -- 172.21.15.3:0/1638641499 wait complete. 2023-10-03T10:17:51.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.216+0000 7f6d24e83700 1 Processor -- start 2023-10-03T10:17:51.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.216+0000 7f6d24e83700 1 -- start start 2023-10-03T10:17:51.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.216+0000 7f6d24e83700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d2009d6f0 0x7f6d201c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:51.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.216+0000 7f6d24e83700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d2009e000 0x7f6d201c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:51.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.216+0000 7f6d24e83700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6d2009e410 0x7f6d201c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:51.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.216+0000 7f6d24e83700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d200a1d00 con 0x7f6d2009d6f0 2023-10-03T10:17:51.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.216+0000 7f6d24e83700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d201320a0 con 0x7f6d2009e000 2023-10-03T10:17:51.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.216+0000 7f6d24e83700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6d201322d0 con 0x7f6d2009e410 2023-10-03T10:17:51.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.217+0000 7f6d1dd9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d2009e000 0x7f6d201c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:51.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.217+0000 7f6d1e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d2009d6f0 0x7f6d201c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:51.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.217+0000 7f6d1dd9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d2009e000 0x7f6d201c89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60568/0 (socket says 172.21.15.3:60568) 2023-10-03T10:17:51.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.217+0000 7f6d1e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d2009d6f0 0x7f6d201c8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59182/0 (socket says 172.21.15.3:59182) 2023-10-03T10:17:51.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.217+0000 7f6d1dd9b700 1 -- 172.21.15.3:0/554037893 learned_addr learned my addr 172.21.15.3:0/554037893 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:51.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.217+0000 7f6d1ed9d700 1 --2- 172.21.15.3:0/554037893 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6d2009e410 0x7f6d201c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:51.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.217+0000 7f6d1dd9b700 1 -- 172.21.15.3:0/554037893 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6d2009e410 msgr2=0x7f6d201c4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:51.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.217+0000 7f6d1dd9b700 1 --2- 172.21.15.3:0/554037893 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6d2009e410 0x7f6d201c4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.217+0000 7f6d1dd9b700 1 -- 172.21.15.3:0/554037893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d2009d6f0 msgr2=0x7f6d201c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:51.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.217+0000 7f6d1dd9b700 1 --2- 172.21.15.3:0/554037893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d2009d6f0 0x7f6d201c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.217+0000 7f6d1dd9b700 1 -- 172.21.15.3:0/554037893 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d18009a70 con 0x7f6d2009e000 2023-10-03T10:17:51.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.217+0000 7f6d1e59c700 1 --2- 172.21.15.3:0/554037893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d2009d6f0 0x7f6d201c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:51.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.217+0000 7f6d1dd9b700 1 --2- 172.21.15.3:0/554037893 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d2009e000 0x7f6d201c89b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400c550 tx=0x7f6d1400c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:51.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.217+0000 7f6d0effd700 1 -- 172.21.15.3:0/554037893 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d14004c80 con 0x7f6d2009e000 2023-10-03T10:17:51.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.218+0000 7f6d0effd700 1 -- 172.21.15.3:0/554037893 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d14004e00 con 0x7f6d2009e000 2023-10-03T10:17:51.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.218+0000 7f6d0effd700 1 -- 172.21.15.3:0/554037893 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d14007310 con 0x7f6d2009e000 2023-10-03T10:17:51.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.218+0000 7f6d24e83700 1 -- 172.21.15.3:0/554037893 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d100094a0 con 0x7f6d2009e000 2023-10-03T10:17:51.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.218+0000 7f6d24e83700 1 -- 172.21.15.3:0/554037893 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d201c8100 con 0x7f6d2009e000 2023-10-03T10:17:51.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.219+0000 7f6d0effd700 1 -- 172.21.15.3:0/554037893 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6d14007470 con 0x7f6d2009e000 2023-10-03T10:17:51.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.219+0000 7f6d0effd700 1 -- 172.21.15.3:0/554037893 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6d1403f3b0 con 0x7f6d2009e000 2023-10-03T10:17:51.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.219+0000 7f6d24e83700 1 -- 172.21.15.3:0/554037893 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d20074ea0 con 0x7f6d2009e000 2023-10-03T10:17:51.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.224+0000 7f6d0effd700 1 -- 172.21.15.3:0/554037893 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6d140123b0 con 0x7f6d2009e000 2023-10-03T10:17:51.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.254+0000 7f4193cfe700 1 Processor -- start 2023-10-03T10:17:51.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.254+0000 7f4193cfe700 1 -- start start 2023-10-03T10:17:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.254+0000 7f4193cfe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f418c09e930 0x7f418c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.254+0000 7f4193cfe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f418c09f240 0x7f418c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.255+0000 7f4193cfe700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f418c070940 con 0x7f418c09f650 2023-10-03T10:17:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.255+0000 7f4193cfe700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f418c06fd60 con 0x7f418c09e930 2023-10-03T10:17:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.255+0000 7f4193cfe700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f418c06fa80 con 0x7f418c09f240 2023-10-03T10:17:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.255+0000 7f4191299700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f418c09f240 0x7f418c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.255+0000 7f4191a9a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f418c09e930 0x7f418c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.255+0000 7f4191299700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f418c09f240 0x7f418c1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60582/0 (socket says 172.21.15.3:60582) 2023-10-03T10:17:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.255+0000 7f4191299700 1 -- 172.21.15.3:0/1298986098 learned_addr learned my addr 172.21.15.3:0/1298986098 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.255+0000 7f4191a9a700 1 -- 172.21.15.3:0/1298986098 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f418c09f240 msgr2=0x7f418c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.255+0000 7f4191a9a700 1 --2- 172.21.15.3:0/1298986098 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f418c09f240 0x7f418c1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.255+0000 7f4191a9a700 1 -- 172.21.15.3:0/1298986098 >> v1:172.21.15.3:6789/0 conn(0x7f418c09f650 legacy=0x7f418c131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:51.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.255+0000 7f4191a9a700 1 -- 172.21.15.3:0/1298986098 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f418c09fb60 con 0x7f418c09e930 2023-10-03T10:17:51.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.255+0000 7f4191299700 1 --2- 172.21.15.3:0/1298986098 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f418c09f240 0x7f418c1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:51.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.255+0000 7f4191a9a700 1 --2- 172.21.15.3:0/1298986098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f418c09e930 0x7f418c09ed00 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f4188009840 tx=0x7f4188009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8996c33f6b804788 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:51.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.255+0000 7f4190a98700 1 -- 172.21.15.3:0/1298986098 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4188011830 con 0x7f418c09e930 2023-10-03T10:17:51.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.256+0000 7f4190a98700 1 -- 172.21.15.3:0/1298986098 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4188011990 con 0x7f418c09e930 2023-10-03T10:17:51.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.256+0000 7f4190a98700 1 -- 172.21.15.3:0/1298986098 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4188011d20 con 0x7f418c09e930 2023-10-03T10:17:51.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.256+0000 7f4193cfe700 1 -- 172.21.15.3:0/1298986098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f418c09e930 msgr2=0x7f418c09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:51.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.256+0000 7f4193cfe700 1 --2- 172.21.15.3:0/1298986098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f418c09e930 0x7f418c09ed00 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f4188009840 tx=0x7f4188009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:51.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.256+0000 7f4193cfe700 1 -- 172.21.15.3:0/1298986098 shutdown_connections 2023-10-03T10:17:51.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.256+0000 7f4193cfe700 1 --2- 172.21.15.3:0/1298986098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f418c09e930 0x7f418c09ed00 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.256+0000 7f4193cfe700 1 --2- 172.21.15.3:0/1298986098 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f418c09f240 0x7f418c1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.256+0000 7f4193cfe700 1 -- 172.21.15.3:0/1298986098 >> 172.21.15.3:0/1298986098 conn(0x7f418c0a4210 msgr2=0x7f418c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:51.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.256+0000 7f4193cfe700 1 -- 172.21.15.3:0/1298986098 shutdown_connections 2023-10-03T10:17:51.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.256+0000 7f4193cfe700 1 -- 172.21.15.3:0/1298986098 wait complete. 2023-10-03T10:17:51.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.257+0000 7f4193cfe700 1 Processor -- start 2023-10-03T10:17:51.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.257+0000 7f4193cfe700 1 -- start start 2023-10-03T10:17:51.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.258+0000 7f4193cfe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f418c09e930 0x7f418c1a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:51.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.258+0000 7f4193cfe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f418c09f240 0x7f418c1a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:51.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.258+0000 7f4193cfe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f418c09f650 0x7f418c1a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:51.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.258+0000 7f4193cfe700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418c1a5120 con 0x7f418c09f240 2023-10-03T10:17:51.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.258+0000 7f4193cfe700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f418c1a5880 con 0x7f418c09f650 2023-10-03T10:17:51.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.258+0000 7f4193cfe700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f418c1a6ad0 con 0x7f418c09e930 2023-10-03T10:17:51.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.258+0000 7f4191299700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f418c09f240 0x7f418c1a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:51.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.258+0000 7f4191299700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f418c09f240 0x7f418c1a41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59198/0 (socket says 172.21.15.3:59198) 2023-10-03T10:17:51.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.258+0000 7f4191299700 1 -- 172.21.15.3:0/1160931692 learned_addr learned my addr 172.21.15.3:0/1160931692 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:51.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.258+0000 7f419229b700 1 --2- 172.21.15.3:0/1160931692 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f418c09f650 0x7f418c1a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:51.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.258+0000 7f4191299700 1 -- 172.21.15.3:0/1160931692 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f418c09e930 msgr2=0x7f418c1a3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:51.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.258+0000 7f4191299700 1 --2- 172.21.15.3:0/1160931692 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f418c09e930 0x7f418c1a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.258+0000 7f4191299700 1 -- 172.21.15.3:0/1160931692 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f418c09f650 msgr2=0x7f418c1a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:51.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.258+0000 7f4191299700 1 --2- 172.21.15.3:0/1160931692 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f418c09f650 0x7f418c1a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.258+0000 7f4191299700 1 -- 172.21.15.3:0/1160931692 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f417c009550 con 0x7f418c09f240 2023-10-03T10:17:51.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.258+0000 7f419229b700 1 --2- 172.21.15.3:0/1160931692 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f418c09f650 0x7f418c1a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:17:51.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.259+0000 7f4191299700 1 --2- 172.21.15.3:0/1160931692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f418c09f240 0x7f418c1a41b0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f417c00ec10 tx=0x7f417c009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:51.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.259+0000 7f41827fc700 1 -- 172.21.15.3:0/1160931692 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c00c8e0 con 0x7f418c09f240 2023-10-03T10:17:51.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.259+0000 7f41827fc700 1 -- 172.21.15.3:0/1160931692 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417c00ca40 con 0x7f418c09f240 2023-10-03T10:17:51.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.259+0000 7f4193cfe700 1 -- 172.21.15.3:0/1160931692 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41880094a0 con 0x7f418c09f240 2023-10-03T10:17:51.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.259+0000 7f41827fc700 1 -- 172.21.15.3:0/1160931692 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c00cdb0 con 0x7f418c09f240 2023-10-03T10:17:51.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.259+0000 7f4193cfe700 1 -- 172.21.15.3:0/1160931692 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f418c0a0190 con 0x7f418c09f240 2023-10-03T10:17:51.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.260+0000 7f4193cfe700 1 -- 172.21.15.3:0/1160931692 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f418c0a3510 con 0x7f418c09f240 2023-10-03T10:17:51.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.260+0000 7f41827fc700 1 -- 172.21.15.3:0/1160931692 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f417c013470 con 0x7f418c09f240 2023-10-03T10:17:51.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.260+0000 7f41827fc700 1 -- 172.21.15.3:0/1160931692 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f417c03f8f0 con 0x7f418c09f240 2023-10-03T10:17:51.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.265+0000 7f41827fc700 1 -- 172.21.15.3:0/1160931692 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f417c0232f0 con 0x7f418c09f240 2023-10-03T10:17:51.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.396+0000 7f3eb7219700 1 -- 172.21.15.3:0/2930342808 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3eb0139180 con 0x7f3eb006fb20 2023-10-03T10:17:51.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.397+0000 7f3ead7fa700 1 -- 172.21.15.3:0/2930342808 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3ea4015030 con 0x7f3eb006fb20 2023-10-03T10:17:51.397 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:51.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.399+0000 7f3eb7219700 1 -- 172.21.15.3:0/2930342808 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3eb006fb20 msgr2=0x7f3eb012acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:51.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.399+0000 7f3eb7219700 1 --2- 172.21.15.3:0/2930342808 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3eb006fb20 0x7f3eb012acd0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4009840 tx=0x7f3ea4004750 comp rx=0 tx=0).stop 2023-10-03T10:17:51.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.399+0000 7f3eb7219700 1 -- 172.21.15.3:0/2930342808 shutdown_connections 2023-10-03T10:17:51.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.399+0000 7f3eb7219700 1 --2- 172.21.15.3:0/2930342808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3eb0130e50 0x7f3eb01c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.399+0000 7f3eb7219700 1 --2- 172.21.15.3:0/2930342808 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3eb006fb20 0x7f3eb012acd0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.399+0000 7f3eb7219700 1 --2- 172.21.15.3:0/2930342808 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3eb0130540 0x7f3eb012b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.399+0000 7f3eb7219700 1 -- 172.21.15.3:0/2930342808 >> 172.21.15.3:0/2930342808 conn(0x7f3eb00a2aa0 msgr2=0x7f3eb006e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:51.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.399+0000 7f3eb7219700 1 -- 172.21.15.3:0/2930342808 shutdown_connections 2023-10-03T10:17:51.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.399+0000 7f3eb7219700 1 -- 172.21.15.3:0/2930342808 wait complete. 2023-10-03T10:17:51.414 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:51.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.434+0000 7f6d24e83700 1 -- 172.21.15.3:0/554037893 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6d2006c360 con 0x7f6d2009e000 2023-10-03T10:17:51.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.435+0000 7f6d0effd700 1 -- 172.21.15.3:0/554037893 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6d14011360 con 0x7f6d2009e000 2023-10-03T10:17:51.435 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:51.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.437+0000 7f6d24e83700 1 -- 172.21.15.3:0/554037893 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d2009e000 msgr2=0x7f6d201c89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.437+0000 7f6d24e83700 1 --2- 172.21.15.3:0/554037893 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d2009e000 0x7f6d201c89b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400c550 tx=0x7f6d1400c860 comp rx=0 tx=0).stop 2023-10-03T10:17:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.437+0000 7f6d24e83700 1 -- 172.21.15.3:0/554037893 shutdown_connections 2023-10-03T10:17:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.437+0000 7f6d24e83700 1 --2- 172.21.15.3:0/554037893 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6d2009e410 0x7f6d201c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.437+0000 7f6d24e83700 1 --2- 172.21.15.3:0/554037893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d2009d6f0 0x7f6d201c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.437+0000 7f6d24e83700 1 --2- 172.21.15.3:0/554037893 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d2009e000 0x7f6d201c89b0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.437+0000 7f6d24e83700 1 -- 172.21.15.3:0/554037893 >> 172.21.15.3:0/554037893 conn(0x7f6d20066f50 msgr2=0x7f6d200a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.437+0000 7f6d24e83700 1 -- 172.21.15.3:0/554037893 shutdown_connections 2023-10-03T10:17:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.437+0000 7f6d24e83700 1 -- 172.21.15.3:0/554037893 wait complete. 2023-10-03T10:17:51.451 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:51.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.472+0000 7f4193cfe700 1 -- 172.21.15.3:0/1160931692 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f418c071d20 con 0x7f418c09f240 2023-10-03T10:17:51.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.473+0000 7f41827fc700 1 -- 172.21.15.3:0/1160931692 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f417c021070 con 0x7f418c09f240 2023-10-03T10:17:51.473 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:51.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.475+0000 7f4193cfe700 1 -- 172.21.15.3:0/1160931692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f418c09f240 msgr2=0x7f418c1a41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:51.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.475+0000 7f4193cfe700 1 --2- 172.21.15.3:0/1160931692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f418c09f240 0x7f418c1a41b0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f417c00ec10 tx=0x7f417c009e70 comp rx=0 tx=0).stop 2023-10-03T10:17:51.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.475+0000 7f4193cfe700 1 -- 172.21.15.3:0/1160931692 shutdown_connections 2023-10-03T10:17:51.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.475+0000 7f4193cfe700 1 --2- 172.21.15.3:0/1160931692 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f418c09f650 0x7f418c1a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.475+0000 7f4193cfe700 1 --2- 172.21.15.3:0/1160931692 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f418c09e930 0x7f418c1a3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.475+0000 7f4193cfe700 1 --2- 172.21.15.3:0/1160931692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f418c09f240 0x7f418c1a41b0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.475+0000 7f4193cfe700 1 -- 172.21.15.3:0/1160931692 >> 172.21.15.3:0/1160931692 conn(0x7f418c0a4210 msgr2=0x7f418c1b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:51.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.475+0000 7f4193cfe700 1 -- 172.21.15.3:0/1160931692 shutdown_connections 2023-10-03T10:17:51.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.475+0000 7f4193cfe700 1 -- 172.21.15.3:0/1160931692 wait complete. 2023-10-03T10:17:51.489 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:51.596 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:51.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.739+0000 7f66a802a700 1 Processor -- start 2023-10-03T10:17:51.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.739+0000 7f66a802a700 1 -- start start 2023-10-03T10:17:51.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.740+0000 7f66a802a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f66a009d6f0 0x7f66a009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:51.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.740+0000 7f66a802a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66a009e000 0x7f66a0130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:51.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.740+0000 7f66a802a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66a009e410 0x7f66a0130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:51.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.740+0000 7f66a802a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66a009f520 con 0x7f66a009d6f0 2023-10-03T10:17:51.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.740+0000 7f66a802a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a009f130 con 0x7f66a009e410 2023-10-03T10:17:51.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.740+0000 7f66a802a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66a009e930 con 0x7f66a009e000 2023-10-03T10:17:51.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.740+0000 7f66a55c5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66a009e000 0x7f66a0130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:51.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.740+0000 7f66a5dc6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f66a009d6f0 0x7f66a009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:51.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.740+0000 7f66a65c7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66a009e410 0x7f66a0130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:51.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.740+0000 7f66a5dc6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f66a009d6f0 0x7f66a009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59210/0 (socket says 172.21.15.3:59210) 2023-10-03T10:17:51.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.740+0000 7f66a55c5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66a009e000 0x7f66a0130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43424/0 (socket says 172.21.15.3:43424) 2023-10-03T10:17:51.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.740+0000 7f66a5dc6700 1 -- 172.21.15.3:0/1718325635 learned_addr learned my addr 172.21.15.3:0/1718325635 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:51.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.740+0000 7f66a65c7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66a009e410 0x7f66a0130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60592/0 (socket says 172.21.15.3:60592) 2023-10-03T10:17:51.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.740+0000 7f66a5dc6700 1 -- 172.21.15.3:0/1718325635 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66a009e000 msgr2=0x7f66a0130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:51.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.740+0000 7f66a5dc6700 1 --2- 172.21.15.3:0/1718325635 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66a009e000 0x7f66a0130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.740+0000 7f66a5dc6700 1 -- 172.21.15.3:0/1718325635 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66a009e410 msgr2=0x7f66a0130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:51.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.740+0000 7f66a5dc6700 1 --2- 172.21.15.3:0/1718325635 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66a009e410 0x7f66a0130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.740+0000 7f66a5dc6700 1 -- 172.21.15.3:0/1718325635 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a009b2d0 con 0x7f66a009d6f0 2023-10-03T10:17:51.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.740+0000 7f66a65c7700 1 --2- 172.21.15.3:0/1718325635 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66a009e410 0x7f66a0130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:51.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.741+0000 7f66a5dc6700 1 --2- 172.21.15.3:0/1718325635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66a009d6f0 0x7f66a009dac0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f6694009920 tx=0x7f6694009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb09220f1d15488f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:51.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.741+0000 7f66a4dc4700 1 -- 172.21.15.3:0/1718325635 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6694011830 con 0x7f66a009d6f0 2023-10-03T10:17:51.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.741+0000 7f66a4dc4700 1 -- 172.21.15.3:0/1718325635 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6694011990 con 0x7f66a009d6f0 2023-10-03T10:17:51.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.741+0000 7f66a4dc4700 1 -- 172.21.15.3:0/1718325635 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6694011d20 con 0x7f66a009d6f0 2023-10-03T10:17:51.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.741+0000 7f66a802a700 1 -- 172.21.15.3:0/1718325635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66a009d6f0 msgr2=0x7f66a009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:51.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.741+0000 7f66a802a700 1 --2- 172.21.15.3:0/1718325635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66a009d6f0 0x7f66a009dac0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f6694009920 tx=0x7f6694009c30 comp rx=0 tx=0).stop 2023-10-03T10:17:51.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.741+0000 7f66a802a700 1 -- 172.21.15.3:0/1718325635 shutdown_connections 2023-10-03T10:17:51.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.741+0000 7f66a802a700 1 --2- 172.21.15.3:0/1718325635 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66a009e410 0x7f66a0130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.741+0000 7f66a802a700 1 --2- 172.21.15.3:0/1718325635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66a009d6f0 0x7f66a009dac0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.741+0000 7f66a802a700 1 --2- 172.21.15.3:0/1718325635 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66a009e000 0x7f66a0130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.741+0000 7f66a802a700 1 -- 172.21.15.3:0/1718325635 >> 172.21.15.3:0/1718325635 conn(0x7f66a0066f50 msgr2=0x7f66a009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:51.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.742+0000 7f66a802a700 1 -- 172.21.15.3:0/1718325635 shutdown_connections 2023-10-03T10:17:51.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.742+0000 7f66a802a700 1 -- 172.21.15.3:0/1718325635 wait complete. 2023-10-03T10:17:51.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.743+0000 7f66a802a700 1 Processor -- start 2023-10-03T10:17:51.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.743+0000 7f66a802a700 1 -- start start 2023-10-03T10:17:51.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.744+0000 7f66a802a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66a009d6f0 0x7f66a01211a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:51.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.744+0000 7f66a802a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66a009e000 0x7f66a01216e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:51.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.744+0000 7f66a802a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66a009e410 0x7f66a0121c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:51.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.744+0000 7f66a802a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a0122380 con 0x7f66a009d6f0 2023-10-03T10:17:51.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.744+0000 7f66a802a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a009c700 con 0x7f66a009e000 2023-10-03T10:17:51.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.744+0000 7f66a802a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66a01245a0 con 0x7f66a009e410 2023-10-03T10:17:51.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.744+0000 7f66a5dc6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66a009d6f0 0x7f66a01211a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:51.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.744+0000 7f66a5dc6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66a009d6f0 0x7f66a01211a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59222/0 (socket says 172.21.15.3:59222) 2023-10-03T10:17:51.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.744+0000 7f66a55c5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66a009e000 0x7f66a01216e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:51.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.744+0000 7f66a5dc6700 1 -- 172.21.15.3:0/3299669040 learned_addr learned my addr 172.21.15.3:0/3299669040 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:51.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.744+0000 7f66a65c7700 1 --2- 172.21.15.3:0/3299669040 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66a009e410 0x7f66a0121c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:51.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.744+0000 7f66a5dc6700 1 -- 172.21.15.3:0/3299669040 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66a009e410 msgr2=0x7f66a0121c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:51.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.744+0000 7f66a5dc6700 1 --2- 172.21.15.3:0/3299669040 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66a009e410 0x7f66a0121c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.744+0000 7f66a5dc6700 1 -- 172.21.15.3:0/3299669040 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66a009e000 msgr2=0x7f66a01216e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:51.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.744+0000 7f66a5dc6700 1 --2- 172.21.15.3:0/3299669040 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66a009e000 0x7f66a01216e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.744+0000 7f66a5dc6700 1 -- 172.21.15.3:0/3299669040 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6698009a70 con 0x7f66a009d6f0 2023-10-03T10:17:51.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.744+0000 7f66a5dc6700 1 --2- 172.21.15.3:0/3299669040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66a009d6f0 0x7f66a01211a0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f66a009d470 tx=0x7f6694004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:51.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.745+0000 7f66927fc700 1 -- 172.21.15.3:0/3299669040 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6694004240 con 0x7f66a009d6f0 2023-10-03T10:17:51.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.745+0000 7f66927fc700 1 -- 172.21.15.3:0/3299669040 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f669401f070 con 0x7f66a009d6f0 2023-10-03T10:17:51.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.745+0000 7f66927fc700 1 -- 172.21.15.3:0/3299669040 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66940045c0 con 0x7f66a009d6f0 2023-10-03T10:17:51.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.745+0000 7f66a802a700 1 -- 172.21.15.3:0/3299669040 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6694009550 con 0x7f66a009d6f0 2023-10-03T10:17:51.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.745+0000 7f66a802a700 1 -- 172.21.15.3:0/3299669040 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a0123cd0 con 0x7f66a009d6f0 2023-10-03T10:17:51.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.746+0000 7f66927fc700 1 -- 172.21.15.3:0/3299669040 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6694021030 con 0x7f66a009d6f0 2023-10-03T10:17:51.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.746+0000 7f66927fc700 1 -- 172.21.15.3:0/3299669040 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6694040630 con 0x7f66a009d6f0 2023-10-03T10:17:51.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.746+0000 7f66a802a700 1 -- 172.21.15.3:0/3299669040 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66a0074ea0 con 0x7f66a009d6f0 2023-10-03T10:17:51.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.751+0000 7f66927fc700 1 -- 172.21.15.3:0/3299669040 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6694015070 con 0x7f66a009d6f0 2023-10-03T10:17:51.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.955+0000 7f66a802a700 1 -- 172.21.15.3:0/3299669040 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f66a0122510 con 0x7f66a009d6f0 2023-10-03T10:17:51.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.956+0000 7f66927fc700 1 -- 172.21.15.3:0/3299669040 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f669401a450 con 0x7f66a009d6f0 2023-10-03T10:17:51.956 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:51.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.958+0000 7f66a802a700 1 -- 172.21.15.3:0/3299669040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66a009d6f0 msgr2=0x7f66a01211a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:51.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.958+0000 7f66a802a700 1 --2- 172.21.15.3:0/3299669040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66a009d6f0 0x7f66a01211a0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f66a009d470 tx=0x7f6694004750 comp rx=0 tx=0).stop 2023-10-03T10:17:51.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.958+0000 7f66a802a700 1 -- 172.21.15.3:0/3299669040 shutdown_connections 2023-10-03T10:17:51.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.958+0000 7f66a802a700 1 --2- 172.21.15.3:0/3299669040 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f66a009e410 0x7f66a0121c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.958+0000 7f66a802a700 1 --2- 172.21.15.3:0/3299669040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66a009d6f0 0x7f66a01211a0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.958+0000 7f66a802a700 1 --2- 172.21.15.3:0/3299669040 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f66a009e000 0x7f66a01216e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:51.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.958+0000 7f66a802a700 1 -- 172.21.15.3:0/3299669040 >> 172.21.15.3:0/3299669040 conn(0x7f66a0066f50 msgr2=0x7f66a0119480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:51.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.958+0000 7f66a802a700 1 -- 172.21.15.3:0/3299669040 shutdown_connections 2023-10-03T10:17:51.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:51.958+0000 7f66a802a700 1 -- 172.21.15.3:0/3299669040 wait complete. 2023-10-03T10:17:51.972 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:52.414 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:52.452 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:52.490 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:52.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.567+0000 7fe27f3ce700 1 Processor -- start 2023-10-03T10:17:52.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.567+0000 7fe27f3ce700 1 -- start start 2023-10-03T10:17:52.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.567+0000 7fe27f3ce700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe27809d6f0 0x7fe27809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:52.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.567+0000 7fe27f3ce700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe27809e410 0x7fe278134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:52.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.567+0000 7fe27f3ce700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe27809b120 con 0x7fe27809e000 2023-10-03T10:17:52.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.567+0000 7fe27f3ce700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27806b460 con 0x7fe27809d6f0 2023-10-03T10:17:52.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.567+0000 7fe27f3ce700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe27809a7e0 con 0x7fe27809e410 2023-10-03T10:17:52.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.568+0000 7fe27d16a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe27809d6f0 0x7fe27809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:52.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.568+0000 7fe27c969700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe27809e000 0x7fe2781339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42568/0 (socket says 172.21.15.3:42568) 2023-10-03T10:17:52.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.568+0000 7fe27c969700 1 -- 172.21.15.3:0/32030021 learned_addr learned my addr 172.21.15.3:0/32030021 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:52.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.568+0000 7fe27d96b700 1 --2- 172.21.15.3:0/32030021 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe27809e410 0x7fe278134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:52.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.568+0000 7fe26ffff700 1 -- 172.21.15.3:0/32030021 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3602659793 0 0) 0x7fe27809b120 con 0x7fe27809e000 2023-10-03T10:17:52.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.568+0000 7fe26ffff700 1 -- 172.21.15.3:0/32030021 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2640035b0 con 0x7fe27809e000 2023-10-03T10:17:52.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.568+0000 7fe27d96b700 1 -- 172.21.15.3:0/32030021 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe27809d6f0 msgr2=0x7fe27809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:52.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.568+0000 7fe27d96b700 1 --2- 172.21.15.3:0/32030021 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe27809d6f0 0x7fe27809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:52.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.568+0000 7fe27d96b700 1 -- 172.21.15.3:0/32030021 >> v1:172.21.15.3:6789/0 conn(0x7fe27809e000 legacy=0x7fe2781339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:52.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.568+0000 7fe27d96b700 1 -- 172.21.15.3:0/32030021 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27809a500 con 0x7fe27809e410 2023-10-03T10:17:52.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.568+0000 7fe27d96b700 1 --2- 172.21.15.3:0/32030021 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe27809e410 0x7fe278134100 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe270009df0 tx=0x7fe27000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab9f13c6031cc677 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:52.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.569+0000 7fe26ffff700 1 -- 172.21.15.3:0/32030021 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27000e070 con 0x7fe27809e410 2023-10-03T10:17:52.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.569+0000 7fe26ffff700 1 -- 172.21.15.3:0/32030021 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27000bae0 con 0x7fe27809e410 2023-10-03T10:17:52.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.569+0000 7fe26ffff700 1 -- 172.21.15.3:0/32030021 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27000be90 con 0x7fe27809e410 2023-10-03T10:17:52.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.569+0000 7fe27f3ce700 1 -- 172.21.15.3:0/32030021 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe27809e410 msgr2=0x7fe278134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:52.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.569+0000 7fe27f3ce700 1 --2- 172.21.15.3:0/32030021 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe27809e410 0x7fe278134100 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe270009df0 tx=0x7fe27000b330 comp rx=0 tx=0).stop 2023-10-03T10:17:52.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.569+0000 7fe27f3ce700 1 -- 172.21.15.3:0/32030021 shutdown_connections 2023-10-03T10:17:52.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.569+0000 7fe27f3ce700 1 --2- 172.21.15.3:0/32030021 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe27809d6f0 0x7fe27809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:52.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.569+0000 7fe27f3ce700 1 --2- 172.21.15.3:0/32030021 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe27809e410 0x7fe278134100 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:52.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.569+0000 7fe27f3ce700 1 -- 172.21.15.3:0/32030021 >> 172.21.15.3:0/32030021 conn(0x7fe278066f50 msgr2=0x7fe27809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:52.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.570+0000 7fe27f3ce700 1 -- 172.21.15.3:0/32030021 shutdown_connections 2023-10-03T10:17:52.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.570+0000 7fe27f3ce700 1 -- 172.21.15.3:0/32030021 wait complete. 2023-10-03T10:17:52.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.571+0000 7fe27f3ce700 1 Processor -- start 2023-10-03T10:17:52.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.571+0000 7fe27f3ce700 1 -- start start 2023-10-03T10:17:52.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.572+0000 7fe27f3ce700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe27809d6f0 0x7fe278122260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:52.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.572+0000 7fe27f3ce700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe27809e000 0x7fe2781227a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:52.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.572+0000 7fe27f3ce700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe27809e410 0x7fe278122ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:52.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.572+0000 7fe27f3ce700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe278123d80 con 0x7fe27809e410 2023-10-03T10:17:52.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.572+0000 7fe27f3ce700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2781243d0 con 0x7fe27809d6f0 2023-10-03T10:17:52.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.572+0000 7fe27f3ce700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe278123630 con 0x7fe27809e000 2023-10-03T10:17:52.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.572+0000 7fe27d16a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe27809d6f0 0x7fe278122260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:52.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.572+0000 7fe27d16a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe27809d6f0 0x7fe278122260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60632/0 (socket says 172.21.15.3:60632) 2023-10-03T10:17:52.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.572+0000 7fe27d16a700 1 -- 172.21.15.3:0/738052425 learned_addr learned my addr 172.21.15.3:0/738052425 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:52.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.572+0000 7fe27c969700 1 --2- 172.21.15.3:0/738052425 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe27809e000 0x7fe2781227a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:52.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.572+0000 7fe27d96b700 1 --2- 172.21.15.3:0/738052425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe27809e410 0x7fe278122ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:52.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.572+0000 7fe27d16a700 1 -- 172.21.15.3:0/738052425 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe27809e000 msgr2=0x7fe2781227a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:52.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.572+0000 7fe27d16a700 1 --2- 172.21.15.3:0/738052425 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe27809e000 0x7fe2781227a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:52.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.572+0000 7fe27d16a700 1 -- 172.21.15.3:0/738052425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe27809e410 msgr2=0x7fe278122ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:52.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.572+0000 7fe27d16a700 1 --2- 172.21.15.3:0/738052425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe27809e410 0x7fe278122ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:52.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.572+0000 7fe27d16a700 1 -- 172.21.15.3:0/738052425 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe270009a70 con 0x7fe27809d6f0 2023-10-03T10:17:52.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.573+0000 7fe27d16a700 1 --2- 172.21.15.3:0/738052425 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe27809d6f0 0x7fe278122260 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe27400e9b0 tx=0x7fe27400ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:52.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.573+0000 7fe26dffb700 1 -- 172.21.15.3:0/738052425 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe274009710 con 0x7fe27809d6f0 2023-10-03T10:17:52.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.573+0000 7fe26dffb700 1 -- 172.21.15.3:0/738052425 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27400ee50 con 0x7fe27809d6f0 2023-10-03T10:17:52.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.573+0000 7fe27f3ce700 1 -- 172.21.15.3:0/738052425 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2780a3af0 con 0x7fe27809d6f0 2023-10-03T10:17:52.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.573+0000 7fe26dffb700 1 -- 172.21.15.3:0/738052425 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27400ca50 con 0x7fe27809d6f0 2023-10-03T10:17:52.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.573+0000 7fe27f3ce700 1 -- 172.21.15.3:0/738052425 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe278125760 con 0x7fe27809d6f0 2023-10-03T10:17:52.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.574+0000 7fe26dffb700 1 -- 172.21.15.3:0/738052425 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe27400cbb0 con 0x7fe27809d6f0 2023-10-03T10:17:52.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.574+0000 7fe26dffb700 1 -- 172.21.15.3:0/738052425 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe27403f720 con 0x7fe27809d6f0 2023-10-03T10:17:52.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.575+0000 7fe27f3ce700 1 -- 172.21.15.3:0/738052425 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe27811f030 con 0x7fe27809d6f0 2023-10-03T10:17:52.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.580+0000 7fe26dffb700 1 -- 172.21.15.3:0/738052425 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe2740232f0 con 0x7fe27809d6f0 2023-10-03T10:17:52.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.606+0000 7fee206b9700 1 Processor -- start 2023-10-03T10:17:52.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.606+0000 7fee206b9700 1 -- start start 2023-10-03T10:17:52.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.606+0000 7fee206b9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee180a0a30 0x7fee180a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:52.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.606+0000 7fee206b9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee1809d630 0x7fee180a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:52.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.606+0000 7fee206b9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee180a2a70 con 0x7fee18133930 2023-10-03T10:17:52.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.606+0000 7fee206b9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee180a01d0 con 0x7fee1809d630 2023-10-03T10:17:52.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.606+0000 7fee206b9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee1809e2b0 con 0x7fee180a0a30 2023-10-03T10:17:52.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.607+0000 7fee1ec56700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fee18133930 0x7fee18133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42570/0 (socket says 172.21.15.3:42570) 2023-10-03T10:17:52.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.607+0000 7fee1e455700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee180a0a30 0x7fee180a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:52.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.607+0000 7fee1ec56700 1 -- 172.21.15.3:0/1366249594 learned_addr learned my addr 172.21.15.3:0/1366249594 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:52.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.607+0000 7fee1dc54700 1 --2- 172.21.15.3:0/1366249594 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee1809d630 0x7fee180a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:52.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.607+0000 7fee1e455700 1 -- 172.21.15.3:0/1366249594 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee1809d630 msgr2=0x7fee180a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:52.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.607+0000 7fee1e455700 1 --2- 172.21.15.3:0/1366249594 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee1809d630 0x7fee180a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:52.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.607+0000 7fee1d453700 1 -- 172.21.15.3:0/1366249594 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112979705 0 0) 0x7fee180a2a70 con 0x7fee18133930 2023-10-03T10:17:52.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.607+0000 7fee1e455700 1 -- 172.21.15.3:0/1366249594 >> v1:172.21.15.3:6789/0 conn(0x7fee18133930 legacy=0x7fee18133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:52.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.607+0000 7fee1e455700 1 -- 172.21.15.3:0/1366249594 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1809dc50 con 0x7fee180a0a30 2023-10-03T10:17:52.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.607+0000 7fee1dc54700 1 --2- 172.21.15.3:0/1366249594 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee1809d630 0x7fee180a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:52.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.607+0000 7fee1e455700 1 --2- 172.21.15.3:0/1366249594 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee180a0a30 0x7fee180a0e00 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fee14009870 tx=0x7fee14009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6410b14177897052 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:52.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.607+0000 7fee1d453700 1 -- 172.21.15.3:0/1366249594 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee14011830 con 0x7fee180a0a30 2023-10-03T10:17:52.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.607+0000 7fee1d453700 1 -- 172.21.15.3:0/1366249594 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee14011990 con 0x7fee180a0a30 2023-10-03T10:17:52.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.607+0000 7fee1d453700 1 -- 172.21.15.3:0/1366249594 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee14011d20 con 0x7fee180a0a30 2023-10-03T10:17:52.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.608+0000 7fee206b9700 1 -- 172.21.15.3:0/1366249594 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee180a0a30 msgr2=0x7fee180a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:52.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.608+0000 7fee206b9700 1 --2- 172.21.15.3:0/1366249594 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee180a0a30 0x7fee180a0e00 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fee14009870 tx=0x7fee14009b80 comp rx=0 tx=0).stop 2023-10-03T10:17:52.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.608+0000 7fee206b9700 1 -- 172.21.15.3:0/1366249594 shutdown_connections 2023-10-03T10:17:52.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.608+0000 7fee206b9700 1 --2- 172.21.15.3:0/1366249594 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee180a0a30 0x7fee180a0e00 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:52.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.608+0000 7fee206b9700 1 --2- 172.21.15.3:0/1366249594 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee1809d630 0x7fee180a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:52.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.608+0000 7fee206b9700 1 -- 172.21.15.3:0/1366249594 >> 172.21.15.3:0/1366249594 conn(0x7fee180a3050 msgr2=0x7fee180a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:52.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.608+0000 7fee206b9700 1 -- 172.21.15.3:0/1366249594 shutdown_connections 2023-10-03T10:17:52.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.608+0000 7fee206b9700 1 -- 172.21.15.3:0/1366249594 wait complete. 2023-10-03T10:17:52.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.609+0000 7fee206b9700 1 Processor -- start 2023-10-03T10:17:52.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.609+0000 7fee206b9700 1 -- start start 2023-10-03T10:17:52.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.610+0000 7fee206b9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee1809d630 0x7fee181c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.610+0000 7fee206b9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee180a0a30 0x7fee181c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.610+0000 7fee206b9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee18133930 0x7fee181c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.610+0000 7fee206b9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee181221b0 con 0x7fee180a0a30 2023-10-03T10:17:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.610+0000 7fee206b9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee18133520 con 0x7fee1809d630 2023-10-03T10:17:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.610+0000 7fee206b9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee18120700 con 0x7fee18133930 2023-10-03T10:17:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.610+0000 7fee1e455700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee1809d630 0x7fee181c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.610+0000 7fee1e455700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee1809d630 0x7fee181c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60650/0 (socket says 172.21.15.3:60650) 2023-10-03T10:17:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.610+0000 7fee1e455700 1 -- 172.21.15.3:0/4118010382 learned_addr learned my addr 172.21.15.3:0/4118010382 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.610+0000 7fee1dc54700 1 --2- 172.21.15.3:0/4118010382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee180a0a30 0x7fee181c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.610+0000 7fee1ec56700 1 --2- 172.21.15.3:0/4118010382 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee18133930 0x7fee181c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.610+0000 7fee1e455700 1 -- 172.21.15.3:0/4118010382 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee18133930 msgr2=0x7fee181c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:52.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.610+0000 7fee1e455700 1 --2- 172.21.15.3:0/4118010382 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee18133930 0x7fee181c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.610+0000 7fee1e455700 1 -- 172.21.15.3:0/4118010382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee180a0a30 msgr2=0x7fee181c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.610+0000 7fee1e455700 1 --2- 172.21.15.3:0/4118010382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee180a0a30 0x7fee181c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.610+0000 7fee1e455700 1 -- 172.21.15.3:0/4118010382 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee080094a0 con 0x7fee1809d630 2023-10-03T10:17:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.611+0000 7fee1e455700 1 --2- 172.21.15.3:0/4118010382 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee1809d630 0x7fee181c7f10 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fee140090b0 tx=0x7fee14004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.611+0000 7fee0effd700 1 -- 172.21.15.3:0/4118010382 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee14004550 con 0x7fee1809d630 2023-10-03T10:17:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.611+0000 7fee0effd700 1 -- 172.21.15.3:0/4118010382 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1401f070 con 0x7fee1809d630 2023-10-03T10:17:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.611+0000 7fee206b9700 1 -- 172.21.15.3:0/4118010382 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee140094a0 con 0x7fee1809d630 2023-10-03T10:17:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.611+0000 7fee0effd700 1 -- 172.21.15.3:0/4118010382 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee140038c0 con 0x7fee1809d630 2023-10-03T10:17:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.611+0000 7fee206b9700 1 -- 172.21.15.3:0/4118010382 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee18122c20 con 0x7fee1809d630 2023-10-03T10:17:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.612+0000 7fee206b9700 1 -- 172.21.15.3:0/4118010382 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee180747f0 con 0x7fee1809d630 2023-10-03T10:17:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.612+0000 7fee0effd700 1 -- 172.21.15.3:0/4118010382 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fee14003a20 con 0x7fee1809d630 2023-10-03T10:17:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.612+0000 7fee0effd700 1 -- 172.21.15.3:0/4118010382 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fee14021070 con 0x7fee1809d630 2023-10-03T10:17:52.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.617+0000 7fee0effd700 1 -- 172.21.15.3:0/4118010382 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fee14010380 con 0x7fee1809d630 2023-10-03T10:17:52.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.647+0000 7f17a8e63700 1 Processor -- start 2023-10-03T10:17:52.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.647+0000 7f17a8e63700 1 -- start start 2023-10-03T10:17:52.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.648+0000 7f17a8e63700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f17a40a0a30 0x7f17a40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:52.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.648+0000 7f17a8e63700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17a409d630 0x7f17a40a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:52.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.648+0000 7f17a8e63700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17a40a2a70 con 0x7f17a4133930 2023-10-03T10:17:52.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.648+0000 7f17a8e63700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17a40a01d0 con 0x7f17a40a0a30 2023-10-03T10:17:52.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.648+0000 7f17a8e63700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a409e2b0 con 0x7f17a409d630 2023-10-03T10:17:52.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.648+0000 7f1799d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17a409d630 0x7f17a40a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:52.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.648+0000 7f1799d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17a409d630 0x7f17a40a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60656/0 (socket says 172.21.15.3:60656) 2023-10-03T10:17:52.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.648+0000 7f1799d9b700 1 -- 172.21.15.3:0/1712661950 learned_addr learned my addr 172.21.15.3:0/1712661950 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:52.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.648+0000 7f17a259c700 1 --2- 172.21.15.3:0/1712661950 >> v2:172.21.15.3:3300/0 conn(0x7f17a40a0a30 0x7f17a40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:52.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.648+0000 7f1799d9b700 1 -- 172.21.15.3:0/1712661950 >> v2:172.21.15.3:3300/0 conn(0x7f17a40a0a30 msgr2=0x7f17a40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:52.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.648+0000 7f1799d9b700 1 --2- 172.21.15.3:0/1712661950 >> v2:172.21.15.3:3300/0 conn(0x7f17a40a0a30 0x7f17a40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:52.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.648+0000 7f1799d9b700 1 -- 172.21.15.3:0/1712661950 >> v1:172.21.15.3:6789/0 conn(0x7f17a4133930 legacy=0x7f17a4133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:52.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.648+0000 7f1799d9b700 1 -- 172.21.15.3:0/1712661950 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a409dc50 con 0x7f17a409d630 2023-10-03T10:17:52.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.649+0000 7f17a259c700 1 --2- 172.21.15.3:0/1712661950 >> v2:172.21.15.3:3300/0 conn(0x7f17a40a0a30 0x7f17a40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:52.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.649+0000 7f1799d9b700 1 --2- 172.21.15.3:0/1712661950 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17a409d630 0x7f17a40a1340 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f178c0098f0 tx=0x7f178c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41d460bcb4e5b61c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:52.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.649+0000 7f17a1d9b700 1 -- 172.21.15.3:0/1712661950 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c011830 con 0x7f17a409d630 2023-10-03T10:17:52.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.649+0000 7f17a1d9b700 1 -- 172.21.15.3:0/1712661950 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178c011990 con 0x7f17a409d630 2023-10-03T10:17:52.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.649+0000 7f17a1d9b700 1 -- 172.21.15.3:0/1712661950 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c011d20 con 0x7f17a409d630 2023-10-03T10:17:52.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.650+0000 7f17a8e63700 1 -- 172.21.15.3:0/1712661950 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17a409d630 msgr2=0x7f17a40a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:52.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.650+0000 7f17a8e63700 1 --2- 172.21.15.3:0/1712661950 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17a409d630 0x7f17a40a1340 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f178c0098f0 tx=0x7f178c009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:52.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.650+0000 7f17a8e63700 1 -- 172.21.15.3:0/1712661950 shutdown_connections 2023-10-03T10:17:52.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.650+0000 7f17a8e63700 1 --2- 172.21.15.3:0/1712661950 >> v2:172.21.15.3:3300/0 conn(0x7f17a40a0a30 0x7f17a40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:52.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.650+0000 7f17a8e63700 1 --2- 172.21.15.3:0/1712661950 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17a409d630 0x7f17a40a1340 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:52.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.650+0000 7f17a8e63700 1 -- 172.21.15.3:0/1712661950 >> 172.21.15.3:0/1712661950 conn(0x7f17a40a3050 msgr2=0x7f17a40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:52.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.650+0000 7f17a8e63700 1 -- 172.21.15.3:0/1712661950 shutdown_connections 2023-10-03T10:17:52.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.650+0000 7f17a8e63700 1 -- 172.21.15.3:0/1712661950 wait complete. 2023-10-03T10:17:52.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.652+0000 7f17a8e63700 1 Processor -- start 2023-10-03T10:17:52.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.652+0000 7f17a8e63700 1 -- start start 2023-10-03T10:17:52.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.653+0000 7f17a8e63700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f17a409d630 0x7f17a41c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:52.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.653+0000 7f17a8e63700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17a40a0a30 0x7f17a41c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:52.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.653+0000 7f17a8e63700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17a4133930 0x7f17a41c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:52.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.653+0000 7f17a8e63700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a4120780 con 0x7f17a4133930 2023-10-03T10:17:52.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.653+0000 7f17a8e63700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a4120590 con 0x7f17a40a0a30 2023-10-03T10:17:52.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.653+0000 7f17a8e63700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f17a4122230 con 0x7f17a409d630 2023-10-03T10:17:52.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.653+0000 7f1799d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17a40a0a30 0x7f17a41c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:52.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.653+0000 7f1799d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17a40a0a30 0x7f17a41c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60672/0 (socket says 172.21.15.3:60672) 2023-10-03T10:17:52.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.653+0000 7f1799d9b700 1 -- 172.21.15.3:0/2066844308 learned_addr learned my addr 172.21.15.3:0/2066844308 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:52.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.653+0000 7f17a2d9d700 1 --2- 172.21.15.3:0/2066844308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17a4133930 0x7f17a41c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:52.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.653+0000 7f17a259c700 1 --2- 172.21.15.3:0/2066844308 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f17a409d630 0x7f17a41c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:52.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.653+0000 7f1799d9b700 1 -- 172.21.15.3:0/2066844308 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f17a409d630 msgr2=0x7f17a41c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:52.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.653+0000 7f1799d9b700 1 --2- 172.21.15.3:0/2066844308 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f17a409d630 0x7f17a41c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:52.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.653+0000 7f1799d9b700 1 -- 172.21.15.3:0/2066844308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17a4133930 msgr2=0x7f17a41c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:52.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.653+0000 7f1799d9b700 1 --2- 172.21.15.3:0/2066844308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17a4133930 0x7f17a41c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:52.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.653+0000 7f1799d9b700 1 -- 172.21.15.3:0/2066844308 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17940094a0 con 0x7f17a40a0a30 2023-10-03T10:17:52.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.654+0000 7f1799d9b700 1 --2- 172.21.15.3:0/2066844308 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17a40a0a30 0x7f17a41c86b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f17a409e3f0 tx=0x7f178c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:52.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.654+0000 7f179b7fe700 1 -- 172.21.15.3:0/2066844308 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c004550 con 0x7f17a40a0a30 2023-10-03T10:17:52.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.654+0000 7f179b7fe700 1 -- 172.21.15.3:0/2066844308 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178c0036a0 con 0x7f17a40a0a30 2023-10-03T10:17:52.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.654+0000 7f17a8e63700 1 -- 172.21.15.3:0/2066844308 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f178c009550 con 0x7f17a40a0a30 2023-10-03T10:17:52.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.654+0000 7f179b7fe700 1 -- 172.21.15.3:0/2066844308 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c003a30 con 0x7f17a40a0a30 2023-10-03T10:17:52.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.654+0000 7f17a8e63700 1 -- 172.21.15.3:0/2066844308 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a4120d80 con 0x7f17a40a0a30 2023-10-03T10:17:52.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.655+0000 7f179b7fe700 1 -- 172.21.15.3:0/2066844308 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f178c021070 con 0x7f17a40a0a30 2023-10-03T10:17:52.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.655+0000 7f179b7fe700 1 -- 172.21.15.3:0/2066844308 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f178c040b60 con 0x7f17a40a0a30 2023-10-03T10:17:52.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.655+0000 7f17a8e63700 1 -- 172.21.15.3:0/2066844308 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17a4121f50 con 0x7f17a40a0a30 2023-10-03T10:17:52.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.660+0000 7f179b7fe700 1 -- 172.21.15.3:0/2066844308 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f178c015080 con 0x7f17a40a0a30 2023-10-03T10:17:52.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.792+0000 7fe27f3ce700 1 -- 172.21.15.3:0/738052425 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe278074ea0 con 0x7fe27809d6f0 2023-10-03T10:17:52.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.792+0000 7fe26dffb700 1 -- 172.21.15.3:0/738052425 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe27401ae80 con 0x7fe27809d6f0 2023-10-03T10:17:52.792 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:52.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.794+0000 7fe27f3ce700 1 -- 172.21.15.3:0/738052425 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe27809d6f0 msgr2=0x7fe278122260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:52.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.794+0000 7fe27f3ce700 1 --2- 172.21.15.3:0/738052425 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe27809d6f0 0x7fe278122260 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe27400e9b0 tx=0x7fe27400ecc0 comp rx=0 tx=0).stop 2023-10-03T10:17:52.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.794+0000 7fe27f3ce700 1 -- 172.21.15.3:0/738052425 shutdown_connections 2023-10-03T10:17:52.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.794+0000 7fe27f3ce700 1 --2- 172.21.15.3:0/738052425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe27809e410 0x7fe278122ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:52.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.794+0000 7fe27f3ce700 1 --2- 172.21.15.3:0/738052425 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe27809d6f0 0x7fe278122260 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:52.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.794+0000 7fe27f3ce700 1 --2- 172.21.15.3:0/738052425 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe27809e000 0x7fe2781227a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:52.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.794+0000 7fe27f3ce700 1 -- 172.21.15.3:0/738052425 >> 172.21.15.3:0/738052425 conn(0x7fe278066f50 msgr2=0x7fe278133f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:52.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.794+0000 7fe27f3ce700 1 -- 172.21.15.3:0/738052425 shutdown_connections 2023-10-03T10:17:52.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.794+0000 7fe27f3ce700 1 -- 172.21.15.3:0/738052425 wait complete. 2023-10-03T10:17:52.809 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:52.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.853+0000 7fee206b9700 1 -- 172.21.15.3:0/4118010382 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fee181223f0 con 0x7fee1809d630 2023-10-03T10:17:52.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.853+0000 7fee0effd700 1 -- 172.21.15.3:0/4118010382 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fee14015070 con 0x7fee1809d630 2023-10-03T10:17:52.854 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:52.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.855+0000 7fee206b9700 1 -- 172.21.15.3:0/4118010382 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee1809d630 msgr2=0x7fee181c7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:52.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.855+0000 7fee206b9700 1 --2- 172.21.15.3:0/4118010382 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee1809d630 0x7fee181c7f10 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fee140090b0 tx=0x7fee14004750 comp rx=0 tx=0).stop 2023-10-03T10:17:52.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.855+0000 7fee206b9700 1 -- 172.21.15.3:0/4118010382 shutdown_connections 2023-10-03T10:17:52.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.855+0000 7fee206b9700 1 --2- 172.21.15.3:0/4118010382 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fee18133930 0x7fee181c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:52.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.855+0000 7fee206b9700 1 --2- 172.21.15.3:0/4118010382 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fee1809d630 0x7fee181c7f10 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:52.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.855+0000 7fee206b9700 1 --2- 172.21.15.3:0/4118010382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee180a0a30 0x7fee181c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:52.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.855+0000 7fee206b9700 1 -- 172.21.15.3:0/4118010382 >> 172.21.15.3:0/4118010382 conn(0x7fee180a3050 msgr2=0x7fee181bb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:52.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.856+0000 7fee206b9700 1 -- 172.21.15.3:0/4118010382 shutdown_connections 2023-10-03T10:17:52.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.856+0000 7fee206b9700 1 -- 172.21.15.3:0/4118010382 wait complete. 2023-10-03T10:17:52.869 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:52.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.896+0000 7f17a8e63700 1 -- 172.21.15.3:0/2066844308 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f17a40747f0 con 0x7f17a40a0a30 2023-10-03T10:17:52.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.897+0000 7f179b7fe700 1 -- 172.21.15.3:0/2066844308 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f178c03ee60 con 0x7f17a40a0a30 2023-10-03T10:17:52.897 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:52.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.899+0000 7f17a8e63700 1 -- 172.21.15.3:0/2066844308 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17a40a0a30 msgr2=0x7f17a41c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:52.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.899+0000 7f17a8e63700 1 --2- 172.21.15.3:0/2066844308 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17a40a0a30 0x7f17a41c86b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f17a409e3f0 tx=0x7f178c004750 comp rx=0 tx=0).stop 2023-10-03T10:17:52.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.899+0000 7f17a8e63700 1 -- 172.21.15.3:0/2066844308 shutdown_connections 2023-10-03T10:17:52.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.899+0000 7f17a8e63700 1 --2- 172.21.15.3:0/2066844308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17a4133930 0x7f17a41c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:52.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.899+0000 7f17a8e63700 1 --2- 172.21.15.3:0/2066844308 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f17a409d630 0x7f17a41c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:52.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.899+0000 7f17a8e63700 1 --2- 172.21.15.3:0/2066844308 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17a40a0a30 0x7f17a41c86b0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:52.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.899+0000 7f17a8e63700 1 -- 172.21.15.3:0/2066844308 >> 172.21.15.3:0/2066844308 conn(0x7f17a40a3050 msgr2=0x7f17a40a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:52.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.899+0000 7f17a8e63700 1 -- 172.21.15.3:0/2066844308 shutdown_connections 2023-10-03T10:17:52.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:52.899+0000 7f17a8e63700 1 -- 172.21.15.3:0/2066844308 wait complete. 2023-10-03T10:17:52.913 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:52.973 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:53.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.113+0000 7fcb08d9f700 1 Processor -- start 2023-10-03T10:17:53.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.113+0000 7fcb08d9f700 1 -- start start 2023-10-03T10:17:53.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.113+0000 7fcb08d9f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb0409d9f0 0x7fcb04136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:53.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.113+0000 7fcb08d9f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb04136a70 0x7fcb04138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:53.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.113+0000 7fcb08d9f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb0409e130 con 0x7fcb041316c0 2023-10-03T10:17:53.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.113+0000 7fcb08d9f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb041321a0 con 0x7fcb0409d9f0 2023-10-03T10:17:53.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.114+0000 7fcb08d9f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb041312b0 con 0x7fcb04136a70 2023-10-03T10:17:53.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.114+0000 7fcb0259c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcb041316c0 0x7fcb04131a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42582/0 (socket says 172.21.15.3:42582) 2023-10-03T10:17:53.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.114+0000 7fcb0259c700 1 -- 172.21.15.3:0/2100007452 learned_addr learned my addr 172.21.15.3:0/2100007452 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:53.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.114+0000 7fcb02d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb04136a70 0x7fcb04138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:53.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.114+0000 7fcb01d9b700 1 --2- 172.21.15.3:0/2100007452 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb0409d9f0 0x7fcb04136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:53.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.114+0000 7fcb0159a700 1 -- 172.21.15.3:0/2100007452 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3360599227 0 0) 0x7fcaf4001110 con 0x7fcb041316c0 2023-10-03T10:17:53.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.114+0000 7fcb0159a700 1 -- 172.21.15.3:0/2100007452 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaec0035b0 con 0x7fcb041316c0 2023-10-03T10:17:53.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.114+0000 7fcb02d9d700 1 -- 172.21.15.3:0/2100007452 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb0409d9f0 msgr2=0x7fcb04136530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:53.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.114+0000 7fcb02d9d700 1 --2- 172.21.15.3:0/2100007452 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb0409d9f0 0x7fcb04136530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:53.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.114+0000 7fcb02d9d700 1 -- 172.21.15.3:0/2100007452 >> v1:172.21.15.3:6789/0 conn(0x7fcb041316c0 legacy=0x7fcb04131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:53.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.114+0000 7fcb02d9d700 1 -- 172.21.15.3:0/2100007452 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0409f6e0 con 0x7fcb04136a70 2023-10-03T10:17:53.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.114+0000 7fcb01d9b700 1 --2- 172.21.15.3:0/2100007452 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb0409d9f0 0x7fcb04136530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:53.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.114+0000 7fcb02d9d700 1 --2- 172.21.15.3:0/2100007452 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb04136a70 0x7fcb04138e50 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009df0 tx=0x7fcafc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d1b2e0132682b92 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:53.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.114+0000 7fcb0159a700 1 -- 172.21.15.3:0/2100007452 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc00e070 con 0x7fcb04136a70 2023-10-03T10:17:53.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.115+0000 7fcb0159a700 1 -- 172.21.15.3:0/2100007452 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcafc00bae0 con 0x7fcb04136a70 2023-10-03T10:17:53.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.115+0000 7fcb0159a700 1 -- 172.21.15.3:0/2100007452 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc00be90 con 0x7fcb04136a70 2023-10-03T10:17:53.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.115+0000 7fcb08d9f700 1 -- 172.21.15.3:0/2100007452 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb04136a70 msgr2=0x7fcb04138e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:53.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.115+0000 7fcb08d9f700 1 --2- 172.21.15.3:0/2100007452 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb04136a70 0x7fcb04138e50 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009df0 tx=0x7fcafc00b330 comp rx=0 tx=0).stop 2023-10-03T10:17:53.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.115+0000 7fcb08d9f700 1 -- 172.21.15.3:0/2100007452 shutdown_connections 2023-10-03T10:17:53.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.115+0000 7fcb08d9f700 1 --2- 172.21.15.3:0/2100007452 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb04136a70 0x7fcb04138e50 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:53.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.115+0000 7fcb08d9f700 1 --2- 172.21.15.3:0/2100007452 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb0409d9f0 0x7fcb04136530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:53.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.115+0000 7fcb08d9f700 1 -- 172.21.15.3:0/2100007452 >> 172.21.15.3:0/2100007452 conn(0x7fcb04066f50 msgr2=0x7fcb0409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:53.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.115+0000 7fcb08d9f700 1 -- 172.21.15.3:0/2100007452 shutdown_connections 2023-10-03T10:17:53.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.115+0000 7fcb08d9f700 1 -- 172.21.15.3:0/2100007452 wait complete. 2023-10-03T10:17:53.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.116+0000 7fcb08d9f700 1 Processor -- start 2023-10-03T10:17:53.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.116+0000 7fcb08d9f700 1 -- start start 2023-10-03T10:17:53.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.117+0000 7fcb08d9f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb0409d9f0 0x7fcb0412b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:53.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.117+0000 7fcb08d9f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb041316c0 0x7fcb04134310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:53.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.117+0000 7fcb08d9f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb04136a70 0x7fcb04134850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:53.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.117+0000 7fcb08d9f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0412b900 con 0x7fcb0409d9f0 2023-10-03T10:17:53.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.117+0000 7fcb08d9f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0409bfc0 con 0x7fcb041316c0 2023-10-03T10:17:53.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.117+0000 7fcb08d9f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb04138e60 con 0x7fcb04136a70 2023-10-03T10:17:53.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.117+0000 7fcb0259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb0409d9f0 0x7fcb0412b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:53.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.117+0000 7fcb01d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb041316c0 0x7fcb04134310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:53.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.117+0000 7fcb01d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb041316c0 0x7fcb04134310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60700/0 (socket says 172.21.15.3:60700) 2023-10-03T10:17:53.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.117+0000 7fcb01d9b700 1 -- 172.21.15.3:0/3965970315 learned_addr learned my addr 172.21.15.3:0/3965970315 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:53.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.117+0000 7fcb0259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb0409d9f0 0x7fcb0412b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59274/0 (socket says 172.21.15.3:59274) 2023-10-03T10:17:53.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.117+0000 7fcb02d9d700 1 --2- 172.21.15.3:0/3965970315 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb04136a70 0x7fcb04134850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:53.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.117+0000 7fcb0259c700 1 -- 172.21.15.3:0/3965970315 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb04136a70 msgr2=0x7fcb04134850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:53.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.117+0000 7fcb0259c700 1 --2- 172.21.15.3:0/3965970315 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb04136a70 0x7fcb04134850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:53.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.117+0000 7fcb0259c700 1 -- 172.21.15.3:0/3965970315 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb041316c0 msgr2=0x7fcb04134310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:53.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.117+0000 7fcb0259c700 1 --2- 172.21.15.3:0/3965970315 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb041316c0 0x7fcb04134310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:53.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.117+0000 7fcb0259c700 1 -- 172.21.15.3:0/3965970315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf80094a0 con 0x7fcb0409d9f0 2023-10-03T10:17:53.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.117+0000 7fcb01d9b700 1 --2- 172.21.15.3:0/3965970315 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb041316c0 0x7fcb04134310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:53.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.118+0000 7fcb0259c700 1 --2- 172.21.15.3:0/3965970315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb0409d9f0 0x7fcb0412b120 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400c730 tx=0x7fcaf400ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:53.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.118+0000 7fcaf2ffd700 1 -- 172.21.15.3:0/3965970315 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4005d70 con 0x7fcb0409d9f0 2023-10-03T10:17:53.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.118+0000 7fcaf2ffd700 1 -- 172.21.15.3:0/3965970315 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf40070d0 con 0x7fcb0409d9f0 2023-10-03T10:17:53.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.118+0000 7fcaf2ffd700 1 -- 172.21.15.3:0/3965970315 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4007460 con 0x7fcb0409d9f0 2023-10-03T10:17:53.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.118+0000 7fcb08d9f700 1 -- 172.21.15.3:0/3965970315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcafc009a70 con 0x7fcb0409d9f0 2023-10-03T10:17:53.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.118+0000 7fcb08d9f700 1 -- 172.21.15.3:0/3965970315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb0412b660 con 0x7fcb0409d9f0 2023-10-03T10:17:53.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.119+0000 7fcaf2ffd700 1 -- 172.21.15.3:0/3965970315 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcaf40075c0 con 0x7fcb0409d9f0 2023-10-03T10:17:53.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.119+0000 7fcaf2ffd700 1 -- 172.21.15.3:0/3965970315 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fcaf403e660 con 0x7fcb0409d9f0 2023-10-03T10:17:53.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.119+0000 7fcb08d9f700 1 -- 172.21.15.3:0/3965970315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb0409e130 con 0x7fcb0409d9f0 2023-10-03T10:17:53.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.124+0000 7fcaf2ffd700 1 -- 172.21.15.3:0/3965970315 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcb0409e130 con 0x7fcb0409d9f0 2023-10-03T10:17:53.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.332+0000 7fcb08d9f700 1 -- 172.21.15.3:0/3965970315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcb0409e130 con 0x7fcb0409d9f0 2023-10-03T10:17:53.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.332+0000 7fcaf2ffd700 1 -- 172.21.15.3:0/3965970315 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcb0409e130 con 0x7fcb0409d9f0 2023-10-03T10:17:53.333 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:53.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.334+0000 7fcb08d9f700 1 -- 172.21.15.3:0/3965970315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb0409d9f0 msgr2=0x7fcb0412b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:53.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.334+0000 7fcb08d9f700 1 --2- 172.21.15.3:0/3965970315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb0409d9f0 0x7fcb0412b120 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400c730 tx=0x7fcaf400ca40 comp rx=0 tx=0).stop 2023-10-03T10:17:53.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.334+0000 7fcb08d9f700 1 -- 172.21.15.3:0/3965970315 shutdown_connections 2023-10-03T10:17:53.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.334+0000 7fcb08d9f700 1 --2- 172.21.15.3:0/3965970315 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb04136a70 0x7fcb04134850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:53.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.334+0000 7fcb08d9f700 1 --2- 172.21.15.3:0/3965970315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb0409d9f0 0x7fcb0412b120 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:53.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.334+0000 7fcb08d9f700 1 --2- 172.21.15.3:0/3965970315 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb041316c0 0x7fcb04134310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:53.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.334+0000 7fcb08d9f700 1 -- 172.21.15.3:0/3965970315 >> 172.21.15.3:0/3965970315 conn(0x7fcb04066f50 msgr2=0x7fcb04130440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:53.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.335+0000 7fcb08d9f700 1 -- 172.21.15.3:0/3965970315 shutdown_connections 2023-10-03T10:17:53.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.335+0000 7fcb08d9f700 1 -- 172.21.15.3:0/3965970315 wait complete. 2023-10-03T10:17:53.349 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:53.810 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:53.870 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:53.914 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:53.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.964+0000 7f9d37cd9700 1 Processor -- start 2023-10-03T10:17:53.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.964+0000 7f9d37cd9700 1 -- start start 2023-10-03T10:17:53.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.965+0000 7f9d37cd9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d3009e930 0x7f9d3009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:53.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.965+0000 7f9d37cd9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9d3009f240 0x7f9d301317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:53.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.965+0000 7f9d37cd9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d30070940 con 0x7f9d3009f650 2023-10-03T10:17:53.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.965+0000 7f9d37cd9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d3006fd60 con 0x7f9d3009f240 2023-10-03T10:17:53.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.965+0000 7f9d37cd9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3006fa80 con 0x7f9d3009e930 2023-10-03T10:17:53.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.965+0000 7f9d35a75700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d3009e930 0x7f9d3009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:53.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.965+0000 7f9d35274700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9d3009f240 0x7f9d301317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:53.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.965+0000 7f9d35a75700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d3009e930 0x7f9d3009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59076/0 (socket says 172.21.15.3:59076) 2023-10-03T10:17:53.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.965+0000 7f9d35a75700 1 -- 172.21.15.3:0/4247739659 learned_addr learned my addr 172.21.15.3:0/4247739659 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:53.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.965+0000 7f9d35274700 1 -- 172.21.15.3:0/4247739659 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d3009e930 msgr2=0x7f9d3009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:53.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.965+0000 7f9d35274700 1 --2- 172.21.15.3:0/4247739659 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d3009e930 0x7f9d3009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:53.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.965+0000 7f9d35274700 1 -- 172.21.15.3:0/4247739659 >> v1:172.21.15.3:6789/0 conn(0x7f9d3009f650 legacy=0x7f9d30131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:53.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.965+0000 7f9d35274700 1 -- 172.21.15.3:0/4247739659 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3009fb60 con 0x7f9d3009f240 2023-10-03T10:17:53.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.966+0000 7f9d35a75700 1 --2- 172.21.15.3:0/4247739659 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d3009e930 0x7f9d3009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:53.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.966+0000 7f9d35274700 1 --2- 172.21.15.3:0/4247739659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d3009f240 0x7f9d301317f0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f9d20009840 tx=0x7f9d20009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798609a9b7b28e0a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:53.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.966+0000 7f9d34a73700 1 -- 172.21.15.3:0/4247739659 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d20011830 con 0x7f9d3009f240 2023-10-03T10:17:53.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.966+0000 7f9d34a73700 1 -- 172.21.15.3:0/4247739659 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d20011990 con 0x7f9d3009f240 2023-10-03T10:17:53.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.966+0000 7f9d34a73700 1 -- 172.21.15.3:0/4247739659 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d20011d20 con 0x7f9d3009f240 2023-10-03T10:17:53.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.967+0000 7f9d37cd9700 1 -- 172.21.15.3:0/4247739659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d3009f240 msgr2=0x7f9d301317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:53.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.967+0000 7f9d37cd9700 1 --2- 172.21.15.3:0/4247739659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d3009f240 0x7f9d301317f0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f9d20009840 tx=0x7f9d20009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:53.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.967+0000 7f9d37cd9700 1 -- 172.21.15.3:0/4247739659 shutdown_connections 2023-10-03T10:17:53.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.967+0000 7f9d37cd9700 1 --2- 172.21.15.3:0/4247739659 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d3009e930 0x7f9d3009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:53.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.967+0000 7f9d37cd9700 1 --2- 172.21.15.3:0/4247739659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d3009f240 0x7f9d301317f0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:53.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.967+0000 7f9d37cd9700 1 -- 172.21.15.3:0/4247739659 >> 172.21.15.3:0/4247739659 conn(0x7f9d300a4210 msgr2=0x7f9d300a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:53.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.967+0000 7f9d37cd9700 1 -- 172.21.15.3:0/4247739659 shutdown_connections 2023-10-03T10:17:53.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.967+0000 7f9d37cd9700 1 -- 172.21.15.3:0/4247739659 wait complete. 2023-10-03T10:17:53.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.968+0000 7f9d37cd9700 1 Processor -- start 2023-10-03T10:17:53.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.968+0000 7f9d37cd9700 1 -- start start 2023-10-03T10:17:53.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.968+0000 7f9d37cd9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d3009e930 0x7f9d3006b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:53.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.968+0000 7f9d37cd9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d3009f240 0x7f9d3006bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:53.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.968+0000 7f9d37cd9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d3009f650 0x7f9d3006c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:53.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.968+0000 7f9d37cd9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d301a99e0 con 0x7f9d3009f650 2023-10-03T10:17:53.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.968+0000 7f9d37cd9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d300a3560 con 0x7f9d3009e930 2023-10-03T10:17:53.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.968+0000 7f9d37cd9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d30071d20 con 0x7f9d3009f240 2023-10-03T10:17:53.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.969+0000 7f9d35a75700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d3009e930 0x7f9d3006b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:53.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.969+0000 7f9d35a75700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d3009e930 0x7f9d3006b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59086/0 (socket says 172.21.15.3:59086) 2023-10-03T10:17:53.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.969+0000 7f9d35a75700 1 -- 172.21.15.3:0/2899271122 learned_addr learned my addr 172.21.15.3:0/2899271122 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:53.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.969+0000 7f9d36276700 1 --2- 172.21.15.3:0/2899271122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d3009f650 0x7f9d3006c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:53.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.969+0000 7f9d35274700 1 --2- 172.21.15.3:0/2899271122 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d3009f240 0x7f9d3006bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:53.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.969+0000 7f9d35a75700 1 -- 172.21.15.3:0/2899271122 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d3009f240 msgr2=0x7f9d3006bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:53.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.969+0000 7f9d35a75700 1 --2- 172.21.15.3:0/2899271122 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d3009f240 0x7f9d3006bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:53.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.969+0000 7f9d35a75700 1 -- 172.21.15.3:0/2899271122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d3009f650 msgr2=0x7f9d3006c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:53.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.969+0000 7f9d35a75700 1 --2- 172.21.15.3:0/2899271122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d3009f650 0x7f9d3006c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:53.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.969+0000 7f9d35a75700 1 -- 172.21.15.3:0/2899271122 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c009550 con 0x7f9d3009e930 2023-10-03T10:17:53.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.969+0000 7f9d36276700 1 --2- 172.21.15.3:0/2899271122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d3009f650 0x7f9d3006c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:53.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.969+0000 7f9d35a75700 1 --2- 172.21.15.3:0/2899271122 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d3009e930 0x7f9d3006b880 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00ebe0 tx=0x7f9d2c00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:53.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.969+0000 7f9d267fc700 1 -- 172.21.15.3:0/2899271122 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c00c850 con 0x7f9d3009e930 2023-10-03T10:17:53.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.969+0000 7f9d267fc700 1 -- 172.21.15.3:0/2899271122 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2c00c9b0 con 0x7f9d3009e930 2023-10-03T10:17:53.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.969+0000 7f9d37cd9700 1 -- 172.21.15.3:0/2899271122 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d200094a0 con 0x7f9d3009e930 2023-10-03T10:17:53.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.969+0000 7f9d267fc700 1 -- 172.21.15.3:0/2899271122 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c00cd40 con 0x7f9d3009e930 2023-10-03T10:17:53.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.970+0000 7f9d37cd9700 1 -- 172.21.15.3:0/2899271122 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d300a0190 con 0x7f9d3009e930 2023-10-03T10:17:53.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.970+0000 7f9d267fc700 1 -- 172.21.15.3:0/2899271122 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9d2c013470 con 0x7f9d3009e930 2023-10-03T10:17:53.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.971+0000 7f9d37cd9700 1 -- 172.21.15.3:0/2899271122 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d300a3180 con 0x7f9d3009e930 2023-10-03T10:17:53.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.971+0000 7f9d267fc700 1 -- 172.21.15.3:0/2899271122 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9d2c03f8a0 con 0x7f9d3009e930 2023-10-03T10:17:53.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:53.976+0000 7f9d267fc700 1 -- 172.21.15.3:0/2899271122 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9d2c0232f0 con 0x7f9d3009e930 2023-10-03T10:17:54.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.015+0000 7f3b49dca700 1 Processor -- start 2023-10-03T10:17:54.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.015+0000 7f3b49dca700 1 -- start start 2023-10-03T10:17:54.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.015+0000 7f3b49dca700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3b4409e930 0x7f3b4409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:54.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.015+0000 7f3b49dca700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b4409f650 0x7f3b44131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:54.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.015+0000 7f3b49dca700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b44070940 con 0x7f3b4409f240 2023-10-03T10:17:54.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.015+0000 7f3b49dca700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b4406fd60 con 0x7f3b4409e930 2023-10-03T10:17:54.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.015+0000 7f3b49dca700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4406fa80 con 0x7f3b4409f650 2023-10-03T10:17:54.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.016+0000 7f3b437fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3b4409e930 0x7f3b4409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:54.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.016+0000 7f3b43fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b4409f650 0x7f3b44131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:54.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.016+0000 7f3b437fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3b4409e930 0x7f3b4409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44520/0 (socket says 172.21.15.3:44520) 2023-10-03T10:17:54.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.016+0000 7f3b437fe700 1 -- 172.21.15.3:0/3061036893 learned_addr learned my addr 172.21.15.3:0/3061036893 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:54.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.016+0000 7f3b437fe700 1 -- 172.21.15.3:0/3061036893 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b4409f650 msgr2=0x7f3b44131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:54.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.016+0000 7f3b437fe700 1 --2- 172.21.15.3:0/3061036893 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b4409f650 0x7f3b44131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.016+0000 7f3b437fe700 1 -- 172.21.15.3:0/3061036893 >> v1:172.21.15.3:6789/0 conn(0x7f3b4409f240 legacy=0x7f3b441317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:54.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.016+0000 7f3b437fe700 1 -- 172.21.15.3:0/3061036893 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b4409fb60 con 0x7f3b4409e930 2023-10-03T10:17:54.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.016+0000 7f3b43fff700 1 --2- 172.21.15.3:0/3061036893 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b4409f650 0x7f3b44131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:54.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.016+0000 7f3b437fe700 1 --2- 172.21.15.3:0/3061036893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b4409e930 0x7f3b4409ed00 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f3b340098f0 tx=0x7f3b34009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7503854620391881 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:54.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.016+0000 7f3b427fc700 1 -- 172.21.15.3:0/3061036893 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b34011830 con 0x7f3b4409e930 2023-10-03T10:17:54.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.017+0000 7f3b427fc700 1 -- 172.21.15.3:0/3061036893 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b34011990 con 0x7f3b4409e930 2023-10-03T10:17:54.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.017+0000 7f3b427fc700 1 -- 172.21.15.3:0/3061036893 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b34011d20 con 0x7f3b4409e930 2023-10-03T10:17:54.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.017+0000 7f3b49dca700 1 -- 172.21.15.3:0/3061036893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b4409e930 msgr2=0x7f3b4409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:54.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.017+0000 7f3b49dca700 1 --2- 172.21.15.3:0/3061036893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b4409e930 0x7f3b4409ed00 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f3b340098f0 tx=0x7f3b34009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:54.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.017+0000 7f3b49dca700 1 -- 172.21.15.3:0/3061036893 shutdown_connections 2023-10-03T10:17:54.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.017+0000 7f3b49dca700 1 --2- 172.21.15.3:0/3061036893 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b4409f650 0x7f3b44131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.017+0000 7f3b49dca700 1 --2- 172.21.15.3:0/3061036893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b4409e930 0x7f3b4409ed00 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.017+0000 7f3b49dca700 1 -- 172.21.15.3:0/3061036893 >> 172.21.15.3:0/3061036893 conn(0x7f3b440a4210 msgr2=0x7f3b440a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:54.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.017+0000 7f3b49dca700 1 -- 172.21.15.3:0/3061036893 shutdown_connections 2023-10-03T10:17:54.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.017+0000 7f3b49dca700 1 -- 172.21.15.3:0/3061036893 wait complete. 2023-10-03T10:17:54.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.018+0000 7f3b49dca700 1 Processor -- start 2023-10-03T10:17:54.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.019+0000 7f3b49dca700 1 -- start start 2023-10-03T10:17:54.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.019+0000 7f3b49dca700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b4409e930 0x7f3b441a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:54.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.019+0000 7f3b49dca700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b4409f240 0x7f3b441a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:54.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.019+0000 7f3b49dca700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b4409f650 0x7f3b441a2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:54.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.019+0000 7f3b49dca700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b441a3160 con 0x7f3b4409e930 2023-10-03T10:17:54.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.019+0000 7f3b49dca700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b441a38c0 con 0x7f3b4409f650 2023-10-03T10:17:54.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.019+0000 7f3b49dca700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3b441a4b10 con 0x7f3b4409f240 2023-10-03T10:17:54.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.019+0000 7f3b42ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b4409f240 0x7f3b441a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:54.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.019+0000 7f3b42ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b4409f240 0x7f3b441a21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47724/0 (socket says 172.21.15.3:47724) 2023-10-03T10:17:54.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.019+0000 7f3b42ffd700 1 -- 172.21.15.3:0/445976395 learned_addr learned my addr 172.21.15.3:0/445976395 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:54.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.019+0000 7f3b437fe700 1 --2- 172.21.15.3:0/445976395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b4409e930 0x7f3b441a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:54.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.019+0000 7f3b43fff700 1 --2- 172.21.15.3:0/445976395 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b4409f650 0x7f3b441a2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:54.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.020+0000 7f3b42ffd700 1 -- 172.21.15.3:0/445976395 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b4409f650 msgr2=0x7f3b441a2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:54.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.020+0000 7f3b42ffd700 1 --2- 172.21.15.3:0/445976395 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b4409f650 0x7f3b441a2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.020+0000 7f3b42ffd700 1 -- 172.21.15.3:0/445976395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b4409e930 msgr2=0x7f3b441a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:54.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.020+0000 7f3b42ffd700 1 --2- 172.21.15.3:0/445976395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b4409e930 0x7f3b441a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.020+0000 7f3b42ffd700 1 -- 172.21.15.3:0/445976395 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3c009a70 con 0x7f3b4409f240 2023-10-03T10:17:54.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.020+0000 7f3b437fe700 1 --2- 172.21.15.3:0/445976395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b4409e930 0x7f3b441a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:54.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.020+0000 7f3b42ffd700 1 --2- 172.21.15.3:0/445976395 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b4409f240 0x7f3b441a21f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800c510 tx=0x7f3b3800c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:54.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.020+0000 7f3b2bfff700 1 -- 172.21.15.3:0/445976395 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b380052d0 con 0x7f3b4409f240 2023-10-03T10:17:54.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.020+0000 7f3b2bfff700 1 -- 172.21.15.3:0/445976395 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b38005430 con 0x7f3b4409f240 2023-10-03T10:17:54.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.020+0000 7f3b49dca700 1 -- 172.21.15.3:0/445976395 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b34009550 con 0x7f3b4409f240 2023-10-03T10:17:54.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.020+0000 7f3b2bfff700 1 -- 172.21.15.3:0/445976395 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b380038c0 con 0x7f3b4409f240 2023-10-03T10:17:54.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.020+0000 7f3b49dca700 1 -- 172.21.15.3:0/445976395 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b440a0190 con 0x7f3b4409f240 2023-10-03T10:17:54.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.021+0000 7f3b2bfff700 1 -- 172.21.15.3:0/445976395 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3b38003a20 con 0x7f3b4409f240 2023-10-03T10:17:54.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.021+0000 7f3b2bfff700 1 -- 172.21.15.3:0/445976395 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3b3803e9e0 con 0x7f3b4409f240 2023-10-03T10:17:54.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.022+0000 7f3b49dca700 1 -- 172.21.15.3:0/445976395 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b440a35c0 con 0x7f3b4409f240 2023-10-03T10:17:54.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.027+0000 7f3b2bfff700 1 -- 172.21.15.3:0/445976395 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3b38017080 con 0x7f3b4409f240 2023-10-03T10:17:54.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.059+0000 7f6167000700 1 Processor -- start 2023-10-03T10:17:54.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.059+0000 7f6167000700 1 -- start start 2023-10-03T10:17:54.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.060+0000 7f6167000700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61600a0a30 0x7f61600a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:54.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.060+0000 7f6167000700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f616009d630 0x7f61600a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:54.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.060+0000 7f6167000700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6160133930 0x7f6160133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:54.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.060+0000 7f6167000700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61600a2a70 con 0x7f616009d630 2023-10-03T10:17:54.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.060+0000 7f6167000700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61600a01d0 con 0x7f61600a0a30 2023-10-03T10:17:54.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.060+0000 7f6167000700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f616009e2b0 con 0x7f6160133930 2023-10-03T10:17:54.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.060+0000 7f616559d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6160133930 0x7f6160133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:54.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.060+0000 7f616559d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6160133930 0x7f6160133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47732/0 (socket says 172.21.15.3:47732) 2023-10-03T10:17:54.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.060+0000 7f616559d700 1 -- 172.21.15.3:0/2702799754 learned_addr learned my addr 172.21.15.3:0/2702799754 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:54.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.060+0000 7f6164d9c700 1 --2- 172.21.15.3:0/2702799754 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61600a0a30 0x7f61600a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:54.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.060+0000 7f615ffff700 1 --2- 172.21.15.3:0/2702799754 >> v2:172.21.15.3:3300/0 conn(0x7f616009d630 0x7f61600a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:54.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.060+0000 7f616559d700 1 -- 172.21.15.3:0/2702799754 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61600a0a30 msgr2=0x7f61600a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:54.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.060+0000 7f616559d700 1 --2- 172.21.15.3:0/2702799754 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61600a0a30 0x7f61600a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.060+0000 7f616559d700 1 -- 172.21.15.3:0/2702799754 >> v2:172.21.15.3:3300/0 conn(0x7f616009d630 msgr2=0x7f61600a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:54.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.060+0000 7f616559d700 1 --2- 172.21.15.3:0/2702799754 >> v2:172.21.15.3:3300/0 conn(0x7f616009d630 0x7f61600a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.061+0000 7f616559d700 1 -- 172.21.15.3:0/2702799754 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616009dc50 con 0x7f6160133930 2023-10-03T10:17:54.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.061+0000 7f615ffff700 1 --2- 172.21.15.3:0/2702799754 >> v2:172.21.15.3:3300/0 conn(0x7f616009d630 0x7f61600a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:54.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.061+0000 7f6164d9c700 1 --2- 172.21.15.3:0/2702799754 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61600a0a30 0x7f61600a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:54.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.061+0000 7f616559d700 1 --2- 172.21.15.3:0/2702799754 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6160133930 0x7f6160133dd0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6158009e50 tx=0x7f615800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7099c2e53b6c07a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:54.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.061+0000 7f615f7fe700 1 -- 172.21.15.3:0/2702799754 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615800e070 con 0x7f6160133930 2023-10-03T10:17:54.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.061+0000 7f615f7fe700 1 -- 172.21.15.3:0/2702799754 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615800b910 con 0x7f6160133930 2023-10-03T10:17:54.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.061+0000 7f615f7fe700 1 -- 172.21.15.3:0/2702799754 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615800bcc0 con 0x7f6160133930 2023-10-03T10:17:54.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.061+0000 7f6167000700 1 -- 172.21.15.3:0/2702799754 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6160133930 msgr2=0x7f6160133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:54.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.061+0000 7f6167000700 1 --2- 172.21.15.3:0/2702799754 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6160133930 0x7f6160133dd0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6158009e50 tx=0x7f615800b330 comp rx=0 tx=0).stop 2023-10-03T10:17:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.062+0000 7f6167000700 1 -- 172.21.15.3:0/2702799754 shutdown_connections 2023-10-03T10:17:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.062+0000 7f6167000700 1 --2- 172.21.15.3:0/2702799754 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6160133930 0x7f6160133dd0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.062+0000 7f6167000700 1 --2- 172.21.15.3:0/2702799754 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61600a0a30 0x7f61600a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.062+0000 7f6167000700 1 --2- 172.21.15.3:0/2702799754 >> v2:172.21.15.3:3300/0 conn(0x7f616009d630 0x7f61600a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.062+0000 7f6167000700 1 -- 172.21.15.3:0/2702799754 >> 172.21.15.3:0/2702799754 conn(0x7f61600a3050 msgr2=0x7f61600a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.062+0000 7f6167000700 1 -- 172.21.15.3:0/2702799754 shutdown_connections 2023-10-03T10:17:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.062+0000 7f6167000700 1 -- 172.21.15.3:0/2702799754 wait complete. 2023-10-03T10:17:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.063+0000 7f6167000700 1 Processor -- start 2023-10-03T10:17:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.063+0000 7f6167000700 1 -- start start 2023-10-03T10:17:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.064+0000 7f6167000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f616009d630 0x7f616011f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.064+0000 7f6167000700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61600a0a30 0x7f616011f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.064+0000 7f6167000700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6160133930 0x7f61601c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.064+0000 7f6167000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6160121240 con 0x7f616009d630 2023-10-03T10:17:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.064+0000 7f6167000700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6160121f20 con 0x7f61600a0a30 2023-10-03T10:17:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.064+0000 7f6167000700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f61601332a0 con 0x7f6160133930 2023-10-03T10:17:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.064+0000 7f615ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61600a0a30 0x7f616011f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.064+0000 7f615ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61600a0a30 0x7f616011f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59142/0 (socket says 172.21.15.3:59142) 2023-10-03T10:17:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.064+0000 7f615ffff700 1 -- 172.21.15.3:0/765429647 learned_addr learned my addr 172.21.15.3:0/765429647 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.064+0000 7f616559d700 1 --2- 172.21.15.3:0/765429647 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6160133930 0x7f61601c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.064+0000 7f6164d9c700 1 --2- 172.21.15.3:0/765429647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f616009d630 0x7f616011f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.064+0000 7f615ffff700 1 -- 172.21.15.3:0/765429647 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6160133930 msgr2=0x7f61601c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.064+0000 7f615ffff700 1 --2- 172.21.15.3:0/765429647 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6160133930 0x7f61601c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.064+0000 7f615ffff700 1 -- 172.21.15.3:0/765429647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f616009d630 msgr2=0x7f616011f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.064+0000 7f615ffff700 1 --2- 172.21.15.3:0/765429647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f616009d630 0x7f616011f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.064+0000 7f615ffff700 1 -- 172.21.15.3:0/765429647 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f614c0094a0 con 0x7f61600a0a30 2023-10-03T10:17:54.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.064+0000 7f615ffff700 1 --2- 172.21.15.3:0/765429647 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61600a0a30 0x7f616011f9e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f614c00ebb0 tx=0x7f614c00eec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:54.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.065+0000 7f615d7fa700 1 -- 172.21.15.3:0/765429647 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c00c850 con 0x7f61600a0a30 2023-10-03T10:17:54.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.065+0000 7f615d7fa700 1 -- 172.21.15.3:0/765429647 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614c00c9b0 con 0x7f61600a0a30 2023-10-03T10:17:54.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.065+0000 7f6167000700 1 -- 172.21.15.3:0/765429647 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61540094a0 con 0x7f61600a0a30 2023-10-03T10:17:54.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.065+0000 7f615d7fa700 1 -- 172.21.15.3:0/765429647 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c00cd20 con 0x7f61600a0a30 2023-10-03T10:17:54.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.065+0000 7f6167000700 1 -- 172.21.15.3:0/765429647 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6158009b30 con 0x7f61600a0a30 2023-10-03T10:17:54.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.066+0000 7f615d7fa700 1 -- 172.21.15.3:0/765429647 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f614c013470 con 0x7f61600a0a30 2023-10-03T10:17:54.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.066+0000 7f6167000700 1 -- 172.21.15.3:0/765429647 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61600a4150 con 0x7f61600a0a30 2023-10-03T10:17:54.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.066+0000 7f615d7fa700 1 -- 172.21.15.3:0/765429647 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f614c03fb30 con 0x7f61600a0a30 2023-10-03T10:17:54.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.075+0000 7f615d7fa700 1 -- 172.21.15.3:0/765429647 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f614c01b3b0 con 0x7f61600a0a30 2023-10-03T10:17:54.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.190+0000 7f9d37cd9700 1 -- 172.21.15.3:0/2899271122 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9d30074ea0 con 0x7f9d3009e930 2023-10-03T10:17:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.190+0000 7f9d267fc700 1 -- 172.21.15.3:0/2899271122 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9d2c021070 con 0x7f9d3009e930 2023-10-03T10:17:54.191 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:54.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.193+0000 7f9d37cd9700 1 -- 172.21.15.3:0/2899271122 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d3009e930 msgr2=0x7f9d3006b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:54.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.193+0000 7f9d37cd9700 1 --2- 172.21.15.3:0/2899271122 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d3009e930 0x7f9d3006b880 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00ebe0 tx=0x7f9d2c00eef0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.193+0000 7f9d37cd9700 1 -- 172.21.15.3:0/2899271122 shutdown_connections 2023-10-03T10:17:54.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.193+0000 7f9d37cd9700 1 --2- 172.21.15.3:0/2899271122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d3009f650 0x7f9d3006c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.193+0000 7f9d37cd9700 1 --2- 172.21.15.3:0/2899271122 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9d3009e930 0x7f9d3006b880 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.193+0000 7f9d37cd9700 1 --2- 172.21.15.3:0/2899271122 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9d3009f240 0x7f9d3006bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.193+0000 7f9d37cd9700 1 -- 172.21.15.3:0/2899271122 >> 172.21.15.3:0/2899271122 conn(0x7f9d300a4210 msgr2=0x7f9d301b8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:54.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.193+0000 7f9d37cd9700 1 -- 172.21.15.3:0/2899271122 shutdown_connections 2023-10-03T10:17:54.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.193+0000 7f9d37cd9700 1 -- 172.21.15.3:0/2899271122 wait complete. 2023-10-03T10:17:54.207 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:54.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.239+0000 7f3b49dca700 1 -- 172.21.15.3:0/445976395 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3b44071d20 con 0x7f3b4409f240 2023-10-03T10:17:54.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.240+0000 7f3b2bfff700 1 -- 172.21.15.3:0/445976395 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3b38011590 con 0x7f3b4409f240 2023-10-03T10:17:54.240 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:54.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.242+0000 7f3b49dca700 1 -- 172.21.15.3:0/445976395 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b4409f240 msgr2=0x7f3b441a21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:54.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.242+0000 7f3b49dca700 1 --2- 172.21.15.3:0/445976395 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b4409f240 0x7f3b441a21f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800c510 tx=0x7f3b3800c820 comp rx=0 tx=0).stop 2023-10-03T10:17:54.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.242+0000 7f3b49dca700 1 -- 172.21.15.3:0/445976395 shutdown_connections 2023-10-03T10:17:54.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.242+0000 7f3b49dca700 1 --2- 172.21.15.3:0/445976395 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3b4409f650 0x7f3b441a2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.242+0000 7f3b49dca700 1 --2- 172.21.15.3:0/445976395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b4409e930 0x7f3b441a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.242+0000 7f3b49dca700 1 --2- 172.21.15.3:0/445976395 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3b4409f240 0x7f3b441a21f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.242+0000 7f3b49dca700 1 -- 172.21.15.3:0/445976395 >> 172.21.15.3:0/445976395 conn(0x7f3b440a4210 msgr2=0x7f3b441b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:54.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.242+0000 7f3b49dca700 1 -- 172.21.15.3:0/445976395 shutdown_connections 2023-10-03T10:17:54.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.243+0000 7f3b49dca700 1 -- 172.21.15.3:0/445976395 wait complete. 2023-10-03T10:17:54.256 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:54.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.283+0000 7f6167000700 1 -- 172.21.15.3:0/765429647 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f61600747f0 con 0x7f61600a0a30 2023-10-03T10:17:54.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.284+0000 7f615d7fa700 1 -- 172.21.15.3:0/765429647 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f614c017030 con 0x7f61600a0a30 2023-10-03T10:17:54.284 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:54.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.286+0000 7f6167000700 1 -- 172.21.15.3:0/765429647 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61600a0a30 msgr2=0x7f616011f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:54.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.286+0000 7f6167000700 1 --2- 172.21.15.3:0/765429647 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61600a0a30 0x7f616011f9e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f614c00ebb0 tx=0x7f614c00eec0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.286+0000 7f6167000700 1 -- 172.21.15.3:0/765429647 shutdown_connections 2023-10-03T10:17:54.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.286+0000 7f6167000700 1 --2- 172.21.15.3:0/765429647 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6160133930 0x7f61601c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.286+0000 7f6167000700 1 --2- 172.21.15.3:0/765429647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f616009d630 0x7f616011f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.286+0000 7f6167000700 1 --2- 172.21.15.3:0/765429647 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f61600a0a30 0x7f616011f9e0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.286+0000 7f6167000700 1 -- 172.21.15.3:0/765429647 >> 172.21.15.3:0/765429647 conn(0x7f61600a3050 msgr2=0x7f61601bbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:54.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.286+0000 7f6167000700 1 -- 172.21.15.3:0/765429647 shutdown_connections 2023-10-03T10:17:54.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.286+0000 7f6167000700 1 -- 172.21.15.3:0/765429647 wait complete. 2023-10-03T10:17:54.300 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:54.350 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:54.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.491+0000 7f3f2f407700 1 Processor -- start 2023-10-03T10:17:54.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.491+0000 7f3f2f407700 1 -- start start 2023-10-03T10:17:54.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.492+0000 7f3f2f407700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3f2809e1d0 0x7f3f28138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:54.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.492+0000 7f3f2f407700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3f28139170 0x7f3f2813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:54.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.492+0000 7f3f2f407700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f2809b120 con 0x7f3f2809d6f0 2023-10-03T10:17:54.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.492+0000 7f3f2f407700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f2806b460 con 0x7f3f28139170 2023-10-03T10:17:54.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.492+0000 7f3f2f407700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2809a7e0 con 0x7f3f2809e1d0 2023-10-03T10:17:54.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.492+0000 7f3f2c9a2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3f2809e1d0 0x7f3f28138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:54.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.492+0000 7f3f2d1a3700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3f2809d6f0 0x7f3f2809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45058/0 (socket says 172.21.15.3:45058) 2023-10-03T10:17:54.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.492+0000 7f3f2d1a3700 1 -- 172.21.15.3:0/480360740 learned_addr learned my addr 172.21.15.3:0/480360740 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:54.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.492+0000 7f3f2d9a4700 1 --2- 172.21.15.3:0/480360740 >> v2:172.21.15.3:3300/0 conn(0x7f3f28139170 0x7f3f2813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:54.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.492+0000 7f3f2d9a4700 1 -- 172.21.15.3:0/480360740 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3f2809e1d0 msgr2=0x7f3f28138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:54.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.492+0000 7f3f2d9a4700 1 --2- 172.21.15.3:0/480360740 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3f2809e1d0 0x7f3f28138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.492+0000 7f3f2d9a4700 1 -- 172.21.15.3:0/480360740 >> v1:172.21.15.3:6789/0 conn(0x7f3f2809d6f0 legacy=0x7f3f2809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:54.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.492+0000 7f3f2d9a4700 1 -- 172.21.15.3:0/480360740 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f2809a500 con 0x7f3f28139170 2023-10-03T10:17:54.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.492+0000 7f3f2c9a2700 1 --2- 172.21.15.3:0/480360740 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3f2809e1d0 0x7f3f28138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:54.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.493+0000 7f3f2d9a4700 1 --2- 172.21.15.3:0/480360740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f28139170 0x7f3f2813b550 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f3f20009d90 tx=0x7f3f2000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18b631e0605b391 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:54.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.493+0000 7f3f1ffff700 1 -- 172.21.15.3:0/480360740 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2000e070 con 0x7f3f28139170 2023-10-03T10:17:54.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.493+0000 7f3f1ffff700 1 -- 172.21.15.3:0/480360740 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f2000bb20 con 0x7f3f28139170 2023-10-03T10:17:54.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.493+0000 7f3f1ffff700 1 -- 172.21.15.3:0/480360740 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f20012430 con 0x7f3f28139170 2023-10-03T10:17:54.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.493+0000 7f3f2f407700 1 -- 172.21.15.3:0/480360740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f28139170 msgr2=0x7f3f2813b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:54.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.493+0000 7f3f2f407700 1 --2- 172.21.15.3:0/480360740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f28139170 0x7f3f2813b550 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f3f20009d90 tx=0x7f3f2000b330 comp rx=0 tx=0).stop 2023-10-03T10:17:54.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.493+0000 7f3f2f407700 1 -- 172.21.15.3:0/480360740 shutdown_connections 2023-10-03T10:17:54.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.493+0000 7f3f2f407700 1 --2- 172.21.15.3:0/480360740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f28139170 0x7f3f2813b550 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.493+0000 7f3f2f407700 1 --2- 172.21.15.3:0/480360740 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3f2809e1d0 0x7f3f28138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.493+0000 7f3f2f407700 1 -- 172.21.15.3:0/480360740 >> 172.21.15.3:0/480360740 conn(0x7f3f28066f50 msgr2=0x7f3f2809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:54.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.493+0000 7f3f2f407700 1 -- 172.21.15.3:0/480360740 shutdown_connections 2023-10-03T10:17:54.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.493+0000 7f3f2f407700 1 -- 172.21.15.3:0/480360740 wait complete. 2023-10-03T10:17:54.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.495+0000 7f3f2f407700 1 Processor -- start 2023-10-03T10:17:54.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.495+0000 7f3f2f407700 1 -- start start 2023-10-03T10:17:54.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.496+0000 7f3f2f407700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f2809d6f0 0x7f3f281c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:54.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.496+0000 7f3f2f407700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3f2809e1d0 0x7f3f281c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:54.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.496+0000 7f3f2f407700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f28139170 0x7f3f2811e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:54.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.496+0000 7f3f2f407700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f28120560 con 0x7f3f2809d6f0 2023-10-03T10:17:54.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.496+0000 7f3f2f407700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f28120370 con 0x7f3f2809e1d0 2023-10-03T10:17:54.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.496+0000 7f3f2f407700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3f28121ad0 con 0x7f3f28139170 2023-10-03T10:17:54.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.496+0000 7f3f2d1a3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f2809d6f0 0x7f3f281c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:54.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.496+0000 7f3f2d1a3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f2809d6f0 0x7f3f281c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44570/0 (socket says 172.21.15.3:44570) 2023-10-03T10:17:54.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.496+0000 7f3f2d1a3700 1 -- 172.21.15.3:0/3115415997 learned_addr learned my addr 172.21.15.3:0/3115415997 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:54.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.496+0000 7f3f2d9a4700 1 --2- 172.21.15.3:0/3115415997 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f28139170 0x7f3f2811e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:54.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.496+0000 7f3f2c9a2700 1 --2- 172.21.15.3:0/3115415997 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3f2809e1d0 0x7f3f281c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:54.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.496+0000 7f3f2d1a3700 1 -- 172.21.15.3:0/3115415997 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f28139170 msgr2=0x7f3f2811e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:54.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.496+0000 7f3f2d1a3700 1 --2- 172.21.15.3:0/3115415997 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f28139170 0x7f3f2811e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.496+0000 7f3f2d1a3700 1 -- 172.21.15.3:0/3115415997 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3f2809e1d0 msgr2=0x7f3f281c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:54.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.496+0000 7f3f2d1a3700 1 --2- 172.21.15.3:0/3115415997 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3f2809e1d0 0x7f3f281c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.496+0000 7f3f2d1a3700 1 -- 172.21.15.3:0/3115415997 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f240094a0 con 0x7f3f2809d6f0 2023-10-03T10:17:54.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.496+0000 7f3f2d9a4700 1 --2- 172.21.15.3:0/3115415997 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f28139170 0x7f3f2811e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:54.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.496+0000 7f3f2d1a3700 1 --2- 172.21.15.3:0/3115415997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f2809d6f0 0x7f3f281c8560 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f3f1800c520 tx=0x7f3f1800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:54.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.496+0000 7f3f1dffb700 1 -- 172.21.15.3:0/3115415997 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f18005d70 con 0x7f3f2809d6f0 2023-10-03T10:17:54.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.496+0000 7f3f1dffb700 1 -- 172.21.15.3:0/3115415997 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f180070d0 con 0x7f3f2809d6f0 2023-10-03T10:17:54.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.496+0000 7f3f2f407700 1 -- 172.21.15.3:0/3115415997 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f20009a70 con 0x7f3f2809d6f0 2023-10-03T10:17:54.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.496+0000 7f3f1dffb700 1 -- 172.21.15.3:0/3115415997 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f18007440 con 0x7f3f2809d6f0 2023-10-03T10:17:54.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.497+0000 7f3f2f407700 1 -- 172.21.15.3:0/3115415997 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f28121fa0 con 0x7f3f2809d6f0 2023-10-03T10:17:54.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.497+0000 7f3f1dffb700 1 -- 172.21.15.3:0/3115415997 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3f180075a0 con 0x7f3f2809d6f0 2023-10-03T10:17:54.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.498+0000 7f3f1dffb700 1 -- 172.21.15.3:0/3115415997 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3f1803ea40 con 0x7f3f2809d6f0 2023-10-03T10:17:54.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.498+0000 7f3f2f407700 1 -- 172.21.15.3:0/3115415997 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f28074ea0 con 0x7f3f2809d6f0 2023-10-03T10:17:54.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.503+0000 7f3f1dffb700 1 -- 172.21.15.3:0/3115415997 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3f18019b40 con 0x7f3f2809d6f0 2023-10-03T10:17:54.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.711+0000 7f3f2f407700 1 -- 172.21.15.3:0/3115415997 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3f28120b60 con 0x7f3f2809d6f0 2023-10-03T10:17:54.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.711+0000 7f3f1dffb700 1 -- 172.21.15.3:0/3115415997 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3f18017030 con 0x7f3f2809d6f0 2023-10-03T10:17:54.712 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:54.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.714+0000 7f3f2f407700 1 -- 172.21.15.3:0/3115415997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f2809d6f0 msgr2=0x7f3f281c8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:54.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.714+0000 7f3f2f407700 1 --2- 172.21.15.3:0/3115415997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f2809d6f0 0x7f3f281c8560 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f3f1800c520 tx=0x7f3f1800c830 comp rx=0 tx=0).stop 2023-10-03T10:17:54.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.714+0000 7f3f2f407700 1 -- 172.21.15.3:0/3115415997 shutdown_connections 2023-10-03T10:17:54.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.714+0000 7f3f2f407700 1 --2- 172.21.15.3:0/3115415997 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3f28139170 0x7f3f2811e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.714+0000 7f3f2f407700 1 --2- 172.21.15.3:0/3115415997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f2809d6f0 0x7f3f281c8560 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.714+0000 7f3f2f407700 1 --2- 172.21.15.3:0/3115415997 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3f2809e1d0 0x7f3f281c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:54.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.714+0000 7f3f2f407700 1 -- 172.21.15.3:0/3115415997 >> 172.21.15.3:0/3115415997 conn(0x7f3f28066f50 msgr2=0x7f3f280a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:54.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.714+0000 7f3f2f407700 1 -- 172.21.15.3:0/3115415997 shutdown_connections 2023-10-03T10:17:54.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:54.714+0000 7f3f2f407700 1 -- 172.21.15.3:0/3115415997 wait complete. 2023-10-03T10:17:54.728 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:55.208 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:55.256 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:55.300 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:55.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.366+0000 7f3e89890700 1 Processor -- start 2023-10-03T10:17:55.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.366+0000 7f3e89890700 1 -- start start 2023-10-03T10:17:55.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.367+0000 7f3e89890700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3e8409d6f0 0x7f3e8409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:55.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.367+0000 7f3e89890700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3e8409e000 0x7f3e841339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:55.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.367+0000 7f3e89890700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e8409b120 con 0x7f3e8409e410 2023-10-03T10:17:55.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.367+0000 7f3e89890700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e8406b460 con 0x7f3e8409d6f0 2023-10-03T10:17:55.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.367+0000 7f3e89890700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8409a7e0 con 0x7f3e8409e000 2023-10-03T10:17:55.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.367+0000 7f3e82ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3e8409d6f0 0x7f3e8409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:55.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.367+0000 7f3e82ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3e8409d6f0 0x7f3e8409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44576/0 (socket says 172.21.15.3:44576) 2023-10-03T10:17:55.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.367+0000 7f3e82ffd700 1 -- 172.21.15.3:0/1209243137 learned_addr learned my addr 172.21.15.3:0/1209243137 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:55.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.367+0000 7f3e827fc700 1 --2- 172.21.15.3:0/1209243137 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3e8409e000 0x7f3e841339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:55.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.368+0000 7f3e82ffd700 1 -- 172.21.15.3:0/1209243137 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3e8409e000 msgr2=0x7f3e841339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:55.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.368+0000 7f3e82ffd700 1 --2- 172.21.15.3:0/1209243137 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3e8409e000 0x7f3e841339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.368+0000 7f3e82ffd700 1 -- 172.21.15.3:0/1209243137 >> v1:172.21.15.3:6789/0 conn(0x7f3e8409e410 legacy=0x7f3e84133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:55.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.368+0000 7f3e82ffd700 1 -- 172.21.15.3:0/1209243137 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8409a500 con 0x7f3e8409d6f0 2023-10-03T10:17:55.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.368+0000 7f3e827fc700 1 --2- 172.21.15.3:0/1209243137 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3e8409e000 0x7f3e841339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:55.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.368+0000 7f3e82ffd700 1 --2- 172.21.15.3:0/1209243137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e8409d6f0 0x7f3e8409dac0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f3e740098f0 tx=0x7f3e74009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c330684a333bd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:55.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.368+0000 7f3e81ffb700 1 -- 172.21.15.3:0/1209243137 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e74011830 con 0x7f3e8409d6f0 2023-10-03T10:17:55.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.368+0000 7f3e81ffb700 1 -- 172.21.15.3:0/1209243137 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e74011990 con 0x7f3e8409d6f0 2023-10-03T10:17:55.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.368+0000 7f3e81ffb700 1 -- 172.21.15.3:0/1209243137 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e74011d20 con 0x7f3e8409d6f0 2023-10-03T10:17:55.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.369+0000 7f3e89890700 1 -- 172.21.15.3:0/1209243137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e8409d6f0 msgr2=0x7f3e8409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:55.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.369+0000 7f3e89890700 1 --2- 172.21.15.3:0/1209243137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e8409d6f0 0x7f3e8409dac0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f3e740098f0 tx=0x7f3e74009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:55.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.369+0000 7f3e89890700 1 -- 172.21.15.3:0/1209243137 shutdown_connections 2023-10-03T10:17:55.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.369+0000 7f3e89890700 1 --2- 172.21.15.3:0/1209243137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e8409d6f0 0x7f3e8409dac0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.369+0000 7f3e89890700 1 --2- 172.21.15.3:0/1209243137 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3e8409e000 0x7f3e841339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.369+0000 7f3e89890700 1 -- 172.21.15.3:0/1209243137 >> 172.21.15.3:0/1209243137 conn(0x7f3e84066f50 msgr2=0x7f3e8409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:55.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.369+0000 7f3e89890700 1 -- 172.21.15.3:0/1209243137 shutdown_connections 2023-10-03T10:17:55.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.369+0000 7f3e89890700 1 -- 172.21.15.3:0/1209243137 wait complete. 2023-10-03T10:17:55.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.370+0000 7f3e89890700 1 Processor -- start 2023-10-03T10:17:55.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.370+0000 7f3e89890700 1 -- start start 2023-10-03T10:17:55.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.371+0000 7f3e89890700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3e8409d6f0 0x7f3e841a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:55.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.371+0000 7f3e89890700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e8409e000 0x7f3e841a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:55.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.371+0000 7f3e89890700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3e8409e410 0x7f3e841a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:55.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.371+0000 7f3e89890700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e841a7340 con 0x7f3e8409e000 2023-10-03T10:17:55.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.371+0000 7f3e89890700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e841a7ac0 con 0x7f3e8409e410 2023-10-03T10:17:55.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.371+0000 7f3e89890700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3e841a80e0 con 0x7f3e8409d6f0 2023-10-03T10:17:55.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.371+0000 7f3e82ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3e8409d6f0 0x7f3e841a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:55.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.371+0000 7f3e837fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3e8409e410 0x7f3e841a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:55.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.371+0000 7f3e82ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3e8409d6f0 0x7f3e841a5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47762/0 (socket says 172.21.15.3:47762) 2023-10-03T10:17:55.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.371+0000 7f3e82ffd700 1 -- 172.21.15.3:0/836256735 learned_addr learned my addr 172.21.15.3:0/836256735 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:55.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.371+0000 7f3e827fc700 1 --2- 172.21.15.3:0/836256735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e8409e000 0x7f3e841a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:55.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.371+0000 7f3e82ffd700 1 -- 172.21.15.3:0/836256735 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3e8409e410 msgr2=0x7f3e841a6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:55.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.371+0000 7f3e82ffd700 1 --2- 172.21.15.3:0/836256735 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3e8409e410 0x7f3e841a6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.371+0000 7f3e82ffd700 1 -- 172.21.15.3:0/836256735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e8409e000 msgr2=0x7f3e841a63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:55.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.371+0000 7f3e82ffd700 1 --2- 172.21.15.3:0/836256735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e8409e000 0x7f3e841a63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.372+0000 7f3e82ffd700 1 -- 172.21.15.3:0/836256735 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e780094a0 con 0x7f3e8409d6f0 2023-10-03T10:17:55.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.372+0000 7f3e837fe700 1 --2- 172.21.15.3:0/836256735 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3e8409e410 0x7f3e841a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:55.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.372+0000 7f3e82ffd700 1 --2- 172.21.15.3:0/836256735 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3e8409d6f0 0x7f3e841a5eb0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3e74009130 tx=0x7f3e74004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:55.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.372+0000 7f3e6bfff700 1 -- 172.21.15.3:0/836256735 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e74004510 con 0x7f3e8409d6f0 2023-10-03T10:17:55.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.372+0000 7f3e6bfff700 1 -- 172.21.15.3:0/836256735 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7401e070 con 0x7f3e8409d6f0 2023-10-03T10:17:55.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.372+0000 7f3e89890700 1 -- 172.21.15.3:0/836256735 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e74009550 con 0x7f3e8409d6f0 2023-10-03T10:17:55.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.372+0000 7f3e6bfff700 1 -- 172.21.15.3:0/836256735 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e740038c0 con 0x7f3e8409d6f0 2023-10-03T10:17:55.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.372+0000 7f3e89890700 1 -- 172.21.15.3:0/836256735 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e840a3220 con 0x7f3e8409d6f0 2023-10-03T10:17:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.373+0000 7f3e89890700 1 -- 172.21.15.3:0/836256735 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e8409c710 con 0x7f3e8409d6f0 2023-10-03T10:17:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.373+0000 7f3e6bfff700 1 -- 172.21.15.3:0/836256735 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3e74003a20 con 0x7f3e8409d6f0 2023-10-03T10:17:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.374+0000 7f3e6bfff700 1 -- 172.21.15.3:0/836256735 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3e74040580 con 0x7f3e8409d6f0 2023-10-03T10:17:55.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.379+0000 7f3e6bfff700 1 -- 172.21.15.3:0/836256735 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3e74003cd0 con 0x7f3e8409d6f0 2023-10-03T10:17:55.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.423+0000 7f568c874700 1 Processor -- start 2023-10-03T10:17:55.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.424+0000 7f568c874700 1 -- start start 2023-10-03T10:17:55.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.424+0000 7f568c874700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f568809d6f0 0x7f568809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:55.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.424+0000 7f568c874700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f568809e410 0x7f5688134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:55.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.424+0000 7f568c874700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f568809b120 con 0x7f568809e000 2023-10-03T10:17:55.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.424+0000 7f568c874700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f568806b460 con 0x7f568809e410 2023-10-03T10:17:55.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.424+0000 7f568c874700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f568809a7e0 con 0x7f568809d6f0 2023-10-03T10:17:55.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.424+0000 7f568659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f568809d6f0 0x7f568809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:55.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.425+0000 7f568659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f568809d6f0 0x7f568809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47768/0 (socket says 172.21.15.3:47768) 2023-10-03T10:17:55.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.425+0000 7f568659c700 1 -- 172.21.15.3:0/3077058588 learned_addr learned my addr 172.21.15.3:0/3077058588 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:55.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.425+0000 7f5686d9d700 1 --2- 172.21.15.3:0/3077058588 >> v2:172.21.15.3:3300/0 conn(0x7f568809e410 0x7f5688134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:55.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.425+0000 7f568659c700 1 -- 172.21.15.3:0/3077058588 >> v2:172.21.15.3:3300/0 conn(0x7f568809e410 msgr2=0x7f5688134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:55.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.425+0000 7f568659c700 1 --2- 172.21.15.3:0/3077058588 >> v2:172.21.15.3:3300/0 conn(0x7f568809e410 0x7f5688134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.425+0000 7f568659c700 1 -- 172.21.15.3:0/3077058588 >> v1:172.21.15.3:6789/0 conn(0x7f568809e000 legacy=0x7f56881339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:55.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.425+0000 7f568659c700 1 -- 172.21.15.3:0/3077058588 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568809a500 con 0x7f568809d6f0 2023-10-03T10:17:55.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.425+0000 7f5686d9d700 1 --2- 172.21.15.3:0/3077058588 >> v2:172.21.15.3:3300/0 conn(0x7f568809e410 0x7f5688134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:55.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.425+0000 7f568659c700 1 --2- 172.21.15.3:0/3077058588 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f568809d6f0 0x7f568809dac0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5678009920 tx=0x7f5678009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=433e67f7df3eff9d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:55.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.425+0000 7f568559a700 1 -- 172.21.15.3:0/3077058588 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5678011830 con 0x7f568809d6f0 2023-10-03T10:17:55.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.425+0000 7f568559a700 1 -- 172.21.15.3:0/3077058588 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5678011990 con 0x7f568809d6f0 2023-10-03T10:17:55.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.425+0000 7f568559a700 1 -- 172.21.15.3:0/3077058588 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5678011d20 con 0x7f568809d6f0 2023-10-03T10:17:55.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.426+0000 7f568c874700 1 -- 172.21.15.3:0/3077058588 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f568809d6f0 msgr2=0x7f568809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:55.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.426+0000 7f568c874700 1 --2- 172.21.15.3:0/3077058588 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f568809d6f0 0x7f568809dac0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5678009920 tx=0x7f5678009c30 comp rx=0 tx=0).stop 2023-10-03T10:17:55.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.426+0000 7f568c874700 1 -- 172.21.15.3:0/3077058588 shutdown_connections 2023-10-03T10:17:55.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.426+0000 7f568c874700 1 --2- 172.21.15.3:0/3077058588 >> v2:172.21.15.3:3300/0 conn(0x7f568809e410 0x7f5688134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.426+0000 7f568c874700 1 --2- 172.21.15.3:0/3077058588 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f568809d6f0 0x7f568809dac0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.426+0000 7f568c874700 1 -- 172.21.15.3:0/3077058588 >> 172.21.15.3:0/3077058588 conn(0x7f5688066f50 msgr2=0x7f568809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:55.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.426+0000 7f568c874700 1 -- 172.21.15.3:0/3077058588 shutdown_connections 2023-10-03T10:17:55.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.426+0000 7f568c874700 1 -- 172.21.15.3:0/3077058588 wait complete. 2023-10-03T10:17:55.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.427+0000 7f568c874700 1 Processor -- start 2023-10-03T10:17:55.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.428+0000 7f568c874700 1 -- start start 2023-10-03T10:17:55.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.428+0000 7f568c874700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f568809d6f0 0x7f56881a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:55.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.428+0000 7f568c874700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f568809e000 0x7f56881a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:55.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.428+0000 7f568c874700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f568809e410 0x7f56881a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:55.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.428+0000 7f568c874700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56880a3480 con 0x7f568809e410 2023-10-03T10:17:55.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.428+0000 7f568c874700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56881a6b90 con 0x7f568809e000 2023-10-03T10:17:55.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.428+0000 7f568c874700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56881a7d40 con 0x7f568809d6f0 2023-10-03T10:17:55.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.428+0000 7f568659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f568809d6f0 0x7f56881a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:55.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.428+0000 7f5685d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f568809e000 0x7f56881a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:55.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.428+0000 7f568659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f568809d6f0 0x7f56881a4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47778/0 (socket says 172.21.15.3:47778) 2023-10-03T10:17:55.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.428+0000 7f568659c700 1 -- 172.21.15.3:0/473003659 learned_addr learned my addr 172.21.15.3:0/473003659 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:55.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.428+0000 7f5686d9d700 1 --2- 172.21.15.3:0/473003659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f568809e410 0x7f56881a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:55.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.429+0000 7f5685d9b700 1 -- 172.21.15.3:0/473003659 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f568809d6f0 msgr2=0x7f56881a4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:55.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.429+0000 7f5685d9b700 1 --2- 172.21.15.3:0/473003659 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f568809d6f0 0x7f56881a4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.429+0000 7f5685d9b700 1 -- 172.21.15.3:0/473003659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f568809e410 msgr2=0x7f56881a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:55.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.429+0000 7f5685d9b700 1 --2- 172.21.15.3:0/473003659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f568809e410 0x7f56881a5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.429+0000 7f5685d9b700 1 -- 172.21.15.3:0/473003659 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5680009a70 con 0x7f568809e000 2023-10-03T10:17:55.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.429+0000 7f5685d9b700 1 --2- 172.21.15.3:0/473003659 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f568809e000 0x7f56881a5440 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f567c000f60 tx=0x7f567c00c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:55.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.429+0000 7f5676ffd700 1 -- 172.21.15.3:0/473003659 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c0052d0 con 0x7f568809e000 2023-10-03T10:17:55.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.429+0000 7f5676ffd700 1 -- 172.21.15.3:0/473003659 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567c005430 con 0x7f568809e000 2023-10-03T10:17:55.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.429+0000 7f5676ffd700 1 -- 172.21.15.3:0/473003659 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c0038c0 con 0x7f568809e000 2023-10-03T10:17:55.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.429+0000 7f568c874700 1 -- 172.21.15.3:0/473003659 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5678009550 con 0x7f568809e000 2023-10-03T10:17:55.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.429+0000 7f568c874700 1 -- 172.21.15.3:0/473003659 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56881a8990 con 0x7f568809e000 2023-10-03T10:17:55.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.430+0000 7f5676ffd700 1 -- 172.21.15.3:0/473003659 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f567c003a20 con 0x7f568809e000 2023-10-03T10:17:55.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.430+0000 7f5676ffd700 1 -- 172.21.15.3:0/473003659 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f567c03e9c0 con 0x7f568809e000 2023-10-03T10:17:55.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.430+0000 7f568c874700 1 -- 172.21.15.3:0/473003659 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56881a8520 con 0x7f568809e000 2023-10-03T10:17:55.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.439+0000 7f5676ffd700 1 -- 172.21.15.3:0/473003659 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f567c019b40 con 0x7f568809e000 2023-10-03T10:17:55.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.447+0000 7f9b344d1700 1 Processor -- start 2023-10-03T10:17:55.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.447+0000 7f9b344d1700 1 -- start start 2023-10-03T10:17:55.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.448+0000 7f9b344d1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9b2c09e930 0x7f9b2c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:55.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.448+0000 7f9b344d1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9b2c09f650 0x7f9b2c131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:55.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.448+0000 7f9b344d1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b2c070940 con 0x7f9b2c09f240 2023-10-03T10:17:55.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.448+0000 7f9b344d1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b2c06fd60 con 0x7f9b2c09f650 2023-10-03T10:17:55.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.448+0000 7f9b344d1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c06fa80 con 0x7f9b2c09e930 2023-10-03T10:17:55.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.448+0000 7f9b31a6c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9b2c09f240 0x7f9b2c1317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45084/0 (socket says 172.21.15.3:45084) 2023-10-03T10:17:55.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.448+0000 7f9b31a6c700 1 -- 172.21.15.3:0/3502288203 learned_addr learned my addr 172.21.15.3:0/3502288203 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:55.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.448+0000 7f9b32a6e700 1 --2- 172.21.15.3:0/3502288203 >> v2:172.21.15.3:3300/0 conn(0x7f9b2c09f650 0x7f9b2c131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:55.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.448+0000 7f9b3226d700 1 --2- 172.21.15.3:0/3502288203 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9b2c09e930 0x7f9b2c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:55.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.448+0000 7f9b3126b700 1 -- 172.21.15.3:0/3502288203 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 51116764 0 0) 0x7f9b2c070940 con 0x7f9b2c09f240 2023-10-03T10:17:55.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.448+0000 7f9b3126b700 1 -- 172.21.15.3:0/3502288203 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b180035b0 con 0x7f9b2c09f240 2023-10-03T10:17:55.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.449+0000 7f9b32a6e700 1 -- 172.21.15.3:0/3502288203 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9b2c09e930 msgr2=0x7f9b2c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:55.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.449+0000 7f9b32a6e700 1 --2- 172.21.15.3:0/3502288203 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9b2c09e930 0x7f9b2c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.449+0000 7f9b32a6e700 1 -- 172.21.15.3:0/3502288203 >> v1:172.21.15.3:6789/0 conn(0x7f9b2c09f240 legacy=0x7f9b2c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:55.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.449+0000 7f9b32a6e700 1 -- 172.21.15.3:0/3502288203 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b2c09fb60 con 0x7f9b2c09f650 2023-10-03T10:17:55.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.449+0000 7f9b3226d700 1 --2- 172.21.15.3:0/3502288203 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9b2c09e930 0x7f9b2c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:55.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.449+0000 7f9b32a6e700 1 --2- 172.21.15.3:0/3502288203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b2c09f650 0x7f9b2c131f00 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009d90 tx=0x7f9b2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d725c8a086ac457 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:55.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.449+0000 7f9b3126b700 1 -- 172.21.15.3:0/3502288203 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2400e070 con 0x7f9b2c09f650 2023-10-03T10:17:55.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.449+0000 7f9b3126b700 1 -- 172.21.15.3:0/3502288203 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b2400bb20 con 0x7f9b2c09f650 2023-10-03T10:17:55.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.449+0000 7f9b3126b700 1 -- 172.21.15.3:0/3502288203 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2400be90 con 0x7f9b2c09f650 2023-10-03T10:17:55.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.450+0000 7f9b344d1700 1 -- 172.21.15.3:0/3502288203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b2c09f650 msgr2=0x7f9b2c131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:55.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.450+0000 7f9b344d1700 1 --2- 172.21.15.3:0/3502288203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b2c09f650 0x7f9b2c131f00 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009d90 tx=0x7f9b2400b330 comp rx=0 tx=0).stop 2023-10-03T10:17:55.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.450+0000 7f9b344d1700 1 -- 172.21.15.3:0/3502288203 shutdown_connections 2023-10-03T10:17:55.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.450+0000 7f9b344d1700 1 --2- 172.21.15.3:0/3502288203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b2c09f650 0x7f9b2c131f00 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.450+0000 7f9b344d1700 1 --2- 172.21.15.3:0/3502288203 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9b2c09e930 0x7f9b2c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.450+0000 7f9b344d1700 1 -- 172.21.15.3:0/3502288203 >> 172.21.15.3:0/3502288203 conn(0x7f9b2c0a4210 msgr2=0x7f9b2c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:55.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.450+0000 7f9b344d1700 1 -- 172.21.15.3:0/3502288203 shutdown_connections 2023-10-03T10:17:55.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.450+0000 7f9b344d1700 1 -- 172.21.15.3:0/3502288203 wait complete. 2023-10-03T10:17:55.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.451+0000 7f9b344d1700 1 Processor -- start 2023-10-03T10:17:55.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.451+0000 7f9b344d1700 1 -- start start 2023-10-03T10:17:55.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.452+0000 7f9b344d1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9b2c09e930 0x7f9b2c1a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:55.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.452+0000 7f9b344d1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b2c09f240 0x7f9b2c1a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:55.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.452+0000 7f9b344d1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b2c09f650 0x7f9b2c1a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:55.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.452+0000 7f9b344d1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c1a5180 con 0x7f9b2c09f240 2023-10-03T10:17:55.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.452+0000 7f9b344d1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c1a58e0 con 0x7f9b2c09e930 2023-10-03T10:17:55.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.452+0000 7f9b344d1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c1a5f00 con 0x7f9b2c09f650 2023-10-03T10:17:55.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.452+0000 7f9b31a6c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b2c09f240 0x7f9b2c1a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:55.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.452+0000 7f9b31a6c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b2c09f240 0x7f9b2c1a4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44640/0 (socket says 172.21.15.3:44640) 2023-10-03T10:17:55.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.452+0000 7f9b32a6e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b2c09f650 0x7f9b2c1a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:55.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.452+0000 7f9b31a6c700 1 -- 172.21.15.3:0/3824374109 learned_addr learned my addr 172.21.15.3:0/3824374109 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:55.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.452+0000 7f9b32a6e700 1 -- 172.21.15.3:0/3824374109 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9b2c09e930 msgr2=0x7f9b2c1a3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:55.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.452+0000 7f9b32a6e700 1 --2- 172.21.15.3:0/3824374109 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9b2c09e930 0x7f9b2c1a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.452+0000 7f9b32a6e700 1 -- 172.21.15.3:0/3824374109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b2c09f240 msgr2=0x7f9b2c1a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:55.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.452+0000 7f9b32a6e700 1 --2- 172.21.15.3:0/3824374109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b2c09f240 0x7f9b2c1a4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.452+0000 7f9b32a6e700 1 -- 172.21.15.3:0/3824374109 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b280094a0 con 0x7f9b2c09f650 2023-10-03T10:17:55.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.452+0000 7f9b31a6c700 1 --2- 172.21.15.3:0/3824374109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b2c09f240 0x7f9b2c1a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:55.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.452+0000 7f9b32a6e700 1 --2- 172.21.15.3:0/3824374109 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b2c09f650 0x7f9b2c1a4750 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9b24016000 tx=0x7f9b24004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:55.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.452+0000 7f9b22ffd700 1 -- 172.21.15.3:0/3824374109 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b24022070 con 0x7f9b2c09f650 2023-10-03T10:17:55.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.453+0000 7f9b22ffd700 1 -- 172.21.15.3:0/3824374109 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b24003e30 con 0x7f9b2c09f650 2023-10-03T10:17:55.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.453+0000 7f9b22ffd700 1 -- 172.21.15.3:0/3824374109 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2401dd80 con 0x7f9b2c09f650 2023-10-03T10:17:55.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.453+0000 7f9b344d1700 1 -- 172.21.15.3:0/3824374109 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b24009a70 con 0x7f9b2c09f650 2023-10-03T10:17:55.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.453+0000 7f9b344d1700 1 -- 172.21.15.3:0/3824374109 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b2c0a0190 con 0x7f9b2c09f650 2023-10-03T10:17:55.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.454+0000 7f9b22ffd700 1 -- 172.21.15.3:0/3824374109 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9b24026070 con 0x7f9b2c09f650 2023-10-03T10:17:55.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.454+0000 7f9b344d1700 1 -- 172.21.15.3:0/3824374109 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b2c1a0b30 con 0x7f9b2c09f650 2023-10-03T10:17:55.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.454+0000 7f9b22ffd700 1 -- 172.21.15.3:0/3824374109 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9b240449b0 con 0x7f9b2c09f650 2023-10-03T10:17:55.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.463+0000 7f9b22ffd700 1 -- 172.21.15.3:0/3824374109 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9b24012430 con 0x7f9b2c09f650 2023-10-03T10:17:55.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.606+0000 7f3e89890700 1 -- 172.21.15.3:0/836256735 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3e84074ea0 con 0x7f3e8409d6f0 2023-10-03T10:17:55.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.606+0000 7f3e6bfff700 1 -- 172.21.15.3:0/836256735 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3e74015030 con 0x7f3e8409d6f0 2023-10-03T10:17:55.606 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:55.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.609+0000 7f3e89890700 1 -- 172.21.15.3:0/836256735 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3e8409d6f0 msgr2=0x7f3e841a5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:55.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.609+0000 7f3e89890700 1 --2- 172.21.15.3:0/836256735 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3e8409d6f0 0x7f3e841a5eb0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3e74009130 tx=0x7f3e74004750 comp rx=0 tx=0).stop 2023-10-03T10:17:55.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.609+0000 7f3e89890700 1 -- 172.21.15.3:0/836256735 shutdown_connections 2023-10-03T10:17:55.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.609+0000 7f3e89890700 1 --2- 172.21.15.3:0/836256735 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3e8409e410 0x7f3e841a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.609+0000 7f3e89890700 1 --2- 172.21.15.3:0/836256735 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3e8409d6f0 0x7f3e841a5eb0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.609+0000 7f3e89890700 1 --2- 172.21.15.3:0/836256735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e8409e000 0x7f3e841a63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.609+0000 7f3e89890700 1 -- 172.21.15.3:0/836256735 >> 172.21.15.3:0/836256735 conn(0x7f3e84066f50 msgr2=0x7f3e841badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:55.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.609+0000 7f3e89890700 1 -- 172.21.15.3:0/836256735 shutdown_connections 2023-10-03T10:17:55.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.609+0000 7f3e89890700 1 -- 172.21.15.3:0/836256735 wait complete. 2023-10-03T10:17:55.623 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:55.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.654+0000 7f568c874700 1 -- 172.21.15.3:0/473003659 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5688074ea0 con 0x7f568809e000 2023-10-03T10:17:55.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.655+0000 7f5676ffd700 1 -- 172.21.15.3:0/473003659 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f567c017020 con 0x7f568809e000 2023-10-03T10:17:55.655 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:55.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.657+0000 7f568c874700 1 -- 172.21.15.3:0/473003659 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f568809e000 msgr2=0x7f56881a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:55.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.657+0000 7f568c874700 1 --2- 172.21.15.3:0/473003659 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f568809e000 0x7f56881a5440 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f567c000f60 tx=0x7f567c00c730 comp rx=0 tx=0).stop 2023-10-03T10:17:55.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.657+0000 7f568c874700 1 -- 172.21.15.3:0/473003659 shutdown_connections 2023-10-03T10:17:55.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.657+0000 7f568c874700 1 --2- 172.21.15.3:0/473003659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f568809e410 0x7f56881a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.657+0000 7f568c874700 1 --2- 172.21.15.3:0/473003659 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f568809d6f0 0x7f56881a4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.657+0000 7f568c874700 1 --2- 172.21.15.3:0/473003659 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f568809e000 0x7f56881a5440 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.657+0000 7f568c874700 1 -- 172.21.15.3:0/473003659 >> 172.21.15.3:0/473003659 conn(0x7f5688066f50 msgr2=0x7f56881bb020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:55.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.657+0000 7f568c874700 1 -- 172.21.15.3:0/473003659 shutdown_connections 2023-10-03T10:17:55.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.657+0000 7f568c874700 1 -- 172.21.15.3:0/473003659 wait complete. 2023-10-03T10:17:55.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.670+0000 7f9b344d1700 1 -- 172.21.15.3:0/3824374109 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9b2c0a3310 con 0x7f9b2c09f650 2023-10-03T10:17:55.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.670+0000 7f9b22ffd700 1 -- 172.21.15.3:0/3824374109 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9b2402d070 con 0x7f9b2c09f650 2023-10-03T10:17:55.671 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:55.671 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:55.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.673+0000 7f9b344d1700 1 -- 172.21.15.3:0/3824374109 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b2c09f650 msgr2=0x7f9b2c1a4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:55.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.673+0000 7f9b344d1700 1 --2- 172.21.15.3:0/3824374109 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b2c09f650 0x7f9b2c1a4750 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9b24016000 tx=0x7f9b24004ad0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.673+0000 7f9b344d1700 1 -- 172.21.15.3:0/3824374109 shutdown_connections 2023-10-03T10:17:55.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.673+0000 7f9b344d1700 1 --2- 172.21.15.3:0/3824374109 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9b2c09f650 0x7f9b2c1a4750 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.673+0000 7f9b344d1700 1 --2- 172.21.15.3:0/3824374109 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9b2c09e930 0x7f9b2c1a3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.673+0000 7f9b344d1700 1 --2- 172.21.15.3:0/3824374109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b2c09f240 0x7f9b2c1a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.673+0000 7f9b344d1700 1 -- 172.21.15.3:0/3824374109 >> 172.21.15.3:0/3824374109 conn(0x7f9b2c0a4210 msgr2=0x7f9b2c1b8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:55.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.673+0000 7f9b344d1700 1 -- 172.21.15.3:0/3824374109 shutdown_connections 2023-10-03T10:17:55.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.673+0000 7f9b344d1700 1 -- 172.21.15.3:0/3824374109 wait complete. 2023-10-03T10:17:55.686 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:55.729 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:55.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.868+0000 7f49c23ca700 1 Processor -- start 2023-10-03T10:17:55.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.868+0000 7f49c23ca700 1 -- start start 2023-10-03T10:17:55.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.869+0000 7f49c23ca700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f49bc09e930 0x7f49bc09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:55.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.869+0000 7f49c23ca700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f49bc09f240 0x7f49bc1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:55.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.869+0000 7f49c23ca700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f49bc09f650 0x7f49bc131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:55.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.869+0000 7f49c23ca700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49bc070940 con 0x7f49bc09f240 2023-10-03T10:17:55.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.869+0000 7f49c23ca700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc06fd60 con 0x7f49bc09f650 2023-10-03T10:17:55.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.869+0000 7f49c23ca700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc06fa80 con 0x7f49bc09e930 2023-10-03T10:17:55.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.869+0000 7f49bbfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f49bc09e930 0x7f49bc09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:55.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.869+0000 7f49bb7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f49bc09f240 0x7f49bc1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:55.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.869+0000 7f49c0967700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f49bc09f650 0x7f49bc131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:55.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.869+0000 7f49bbfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f49bc09e930 0x7f49bc09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47806/0 (socket says 172.21.15.3:47806) 2023-10-03T10:17:55.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.869+0000 7f49bb7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f49bc09f240 0x7f49bc1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44654/0 (socket says 172.21.15.3:44654) 2023-10-03T10:17:55.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.869+0000 7f49bbfff700 1 -- 172.21.15.3:0/1200666213 learned_addr learned my addr 172.21.15.3:0/1200666213 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:55.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.869+0000 7f49c0967700 1 -- 172.21.15.3:0/1200666213 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f49bc09e930 msgr2=0x7f49bc09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:55.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.869+0000 7f49c0967700 1 --2- 172.21.15.3:0/1200666213 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f49bc09e930 0x7f49bc09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.869+0000 7f49c0967700 1 -- 172.21.15.3:0/1200666213 >> v2:172.21.15.3:3300/0 conn(0x7f49bc09f240 msgr2=0x7f49bc1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:55.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.869+0000 7f49c0967700 1 --2- 172.21.15.3:0/1200666213 >> v2:172.21.15.3:3300/0 conn(0x7f49bc09f240 0x7f49bc1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.869+0000 7f49c0967700 1 -- 172.21.15.3:0/1200666213 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc09fb60 con 0x7f49bc09f650 2023-10-03T10:17:55.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.869+0000 7f49bb7fe700 1 --2- 172.21.15.3:0/1200666213 >> v2:172.21.15.3:3300/0 conn(0x7f49bc09f240 0x7f49bc1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:55.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.869+0000 7f49c0967700 1 --2- 172.21.15.3:0/1200666213 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f49bc09f650 0x7f49bc131d30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f49b4009d90 tx=0x7f49b400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcdfd444a2568078 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:55.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.869+0000 7f49baffd700 1 -- 172.21.15.3:0/1200666213 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b400e070 con 0x7f49bc09f650 2023-10-03T10:17:55.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.870+0000 7f49baffd700 1 -- 172.21.15.3:0/1200666213 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b400bb20 con 0x7f49bc09f650 2023-10-03T10:17:55.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.870+0000 7f49baffd700 1 -- 172.21.15.3:0/1200666213 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4012430 con 0x7f49bc09f650 2023-10-03T10:17:55.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.870+0000 7f49c23ca700 1 -- 172.21.15.3:0/1200666213 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f49bc09f650 msgr2=0x7f49bc131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:55.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.870+0000 7f49c23ca700 1 --2- 172.21.15.3:0/1200666213 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f49bc09f650 0x7f49bc131d30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f49b4009d90 tx=0x7f49b400b330 comp rx=0 tx=0).stop 2023-10-03T10:17:55.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.870+0000 7f49c23ca700 1 -- 172.21.15.3:0/1200666213 shutdown_connections 2023-10-03T10:17:55.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.870+0000 7f49c23ca700 1 --2- 172.21.15.3:0/1200666213 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f49bc09f650 0x7f49bc131d30 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.870+0000 7f49c23ca700 1 --2- 172.21.15.3:0/1200666213 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f49bc09e930 0x7f49bc09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.870+0000 7f49c23ca700 1 --2- 172.21.15.3:0/1200666213 >> v2:172.21.15.3:3300/0 conn(0x7f49bc09f240 0x7f49bc1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.870+0000 7f49c23ca700 1 -- 172.21.15.3:0/1200666213 >> 172.21.15.3:0/1200666213 conn(0x7f49bc0a4210 msgr2=0x7f49bc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:55.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.870+0000 7f49c23ca700 1 -- 172.21.15.3:0/1200666213 shutdown_connections 2023-10-03T10:17:55.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.870+0000 7f49c23ca700 1 -- 172.21.15.3:0/1200666213 wait complete. 2023-10-03T10:17:55.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.871+0000 7f49c23ca700 1 Processor -- start 2023-10-03T10:17:55.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.872+0000 7f49c23ca700 1 -- start start 2023-10-03T10:17:55.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.872+0000 7f49c23ca700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49bc09e930 0x7f49bc1203a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:55.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.872+0000 7f49c23ca700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f49bc09f240 0x7f49bc1208e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:55.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.872+0000 7f49c23ca700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f49bc09f650 0x7f49bc120e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:55.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.872+0000 7f49c23ca700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc123e70 con 0x7f49bc09e930 2023-10-03T10:17:55.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.872+0000 7f49c23ca700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc071d20 con 0x7f49bc09f240 2023-10-03T10:17:55.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.872+0000 7f49c23ca700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc0a2fd0 con 0x7f49bc09f650 2023-10-03T10:17:55.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.872+0000 7f49bbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49bc09e930 0x7f49bc1203a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:55.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.872+0000 7f49bbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49bc09e930 0x7f49bc1203a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44666/0 (socket says 172.21.15.3:44666) 2023-10-03T10:17:55.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.872+0000 7f49bb7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f49bc09f240 0x7f49bc1208e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:55.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.872+0000 7f49bbfff700 1 -- 172.21.15.3:0/3293511558 learned_addr learned my addr 172.21.15.3:0/3293511558 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:55.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.872+0000 7f49c0967700 1 --2- 172.21.15.3:0/3293511558 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f49bc09f650 0x7f49bc120e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:55.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.872+0000 7f49bb7fe700 1 -- 172.21.15.3:0/3293511558 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f49bc09f650 msgr2=0x7f49bc120e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:55.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.872+0000 7f49bb7fe700 1 --2- 172.21.15.3:0/3293511558 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f49bc09f650 0x7f49bc120e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.872+0000 7f49bb7fe700 1 -- 172.21.15.3:0/3293511558 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49bc09e930 msgr2=0x7f49bc1203a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:55.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.872+0000 7f49bb7fe700 1 --2- 172.21.15.3:0/3293511558 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49bc09e930 0x7f49bc1203a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:55.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.872+0000 7f49bb7fe700 1 -- 172.21.15.3:0/3293511558 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a40094a0 con 0x7f49bc09f240 2023-10-03T10:17:55.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.873+0000 7f49bb7fe700 1 --2- 172.21.15.3:0/3293511558 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f49bc09f240 0x7f49bc1208e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f49a400e980 tx=0x7f49a400ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:55.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.873+0000 7f49b8ff9700 1 -- 172.21.15.3:0/3293511558 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a400c690 con 0x7f49bc09f240 2023-10-03T10:17:55.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.873+0000 7f49b8ff9700 1 -- 172.21.15.3:0/3293511558 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a400c7f0 con 0x7f49bc09f240 2023-10-03T10:17:55.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.873+0000 7f49c23ca700 1 -- 172.21.15.3:0/3293511558 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b4009a70 con 0x7f49bc09f240 2023-10-03T10:17:55.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.873+0000 7f49b8ff9700 1 -- 172.21.15.3:0/3293511558 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a400cb80 con 0x7f49bc09f240 2023-10-03T10:17:55.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.873+0000 7f49c23ca700 1 -- 172.21.15.3:0/3293511558 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49bc0a0190 con 0x7f49bc09f240 2023-10-03T10:17:55.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.874+0000 7f49b8ff9700 1 -- 172.21.15.3:0/3293511558 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f49a400cce0 con 0x7f49bc09f240 2023-10-03T10:17:55.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.874+0000 7f49b8ff9700 1 -- 172.21.15.3:0/3293511558 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f49a403f800 con 0x7f49bc09f240 2023-10-03T10:17:55.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.874+0000 7f49c23ca700 1 -- 172.21.15.3:0/3293511558 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49bc11d540 con 0x7f49bc09f240 2023-10-03T10:17:55.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:55.879+0000 7f49b8ff9700 1 -- 172.21.15.3:0/3293511558 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f49a4021080 con 0x7f49bc09f240 2023-10-03T10:17:56.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.083+0000 7f49c23ca700 1 -- 172.21.15.3:0/3293511558 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f49bc123a10 con 0x7f49bc09f240 2023-10-03T10:17:56.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.083+0000 7f49b8ff9700 1 -- 172.21.15.3:0/3293511558 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f49a401bb50 con 0x7f49bc09f240 2023-10-03T10:17:56.084 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:56.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.086+0000 7f49c23ca700 1 -- 172.21.15.3:0/3293511558 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f49bc09f240 msgr2=0x7f49bc1208e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:56.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.086+0000 7f49c23ca700 1 --2- 172.21.15.3:0/3293511558 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f49bc09f240 0x7f49bc1208e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f49a400e980 tx=0x7f49a400ec90 comp rx=0 tx=0).stop 2023-10-03T10:17:56.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.086+0000 7f49c23ca700 1 -- 172.21.15.3:0/3293511558 shutdown_connections 2023-10-03T10:17:56.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.086+0000 7f49c23ca700 1 --2- 172.21.15.3:0/3293511558 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f49bc09f650 0x7f49bc120e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:56.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.086+0000 7f49c23ca700 1 --2- 172.21.15.3:0/3293511558 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49bc09e930 0x7f49bc1203a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:56.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.086+0000 7f49c23ca700 1 --2- 172.21.15.3:0/3293511558 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f49bc09f240 0x7f49bc1208e0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:56.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.086+0000 7f49c23ca700 1 -- 172.21.15.3:0/3293511558 >> 172.21.15.3:0/3293511558 conn(0x7f49bc0a4210 msgr2=0x7f49bc11a680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:56.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.086+0000 7f49c23ca700 1 -- 172.21.15.3:0/3293511558 shutdown_connections 2023-10-03T10:17:56.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.086+0000 7f49c23ca700 1 -- 172.21.15.3:0/3293511558 wait complete. 2023-10-03T10:17:56.100 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:56.624 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:56.672 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:56.687 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:56.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.782+0000 7f17e6ece700 1 Processor -- start 2023-10-03T10:17:56.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.782+0000 7f17e6ece700 1 -- start start 2023-10-03T10:17:56.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.782+0000 7f17e6ece700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17e0133a80 0x7f17e0133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:56.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.782+0000 7f17e6ece700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f17e0130a40 0x7f17e013b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:56.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.782+0000 7f17e6ece700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17e006bc00 con 0x7f17e012ff20 2023-10-03T10:17:56.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.782+0000 7f17e6ece700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17e006c570 con 0x7f17e0130a40 2023-10-03T10:17:56.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.782+0000 7f17e6ece700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e00a0040 con 0x7f17e0133a80 2023-10-03T10:17:56.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.783+0000 7f17e546b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f17e0130a40 0x7f17e013b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:56.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.783+0000 7f17e546b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f17e0130a40 0x7f17e013b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44668/0 (socket says 172.21.15.3:44668) 2023-10-03T10:17:56.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.783+0000 7f17e546b700 1 -- 172.21.15.3:0/104452383 learned_addr learned my addr 172.21.15.3:0/104452383 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:56.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.783+0000 7f17e4c6a700 1 --2- 172.21.15.3:0/104452383 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17e0133a80 0x7f17e0133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:56.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.783+0000 7f17e546b700 1 -- 172.21.15.3:0/104452383 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17e0133a80 msgr2=0x7f17e0133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:56.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.783+0000 7f17e546b700 1 --2- 172.21.15.3:0/104452383 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17e0133a80 0x7f17e0133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:56.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.783+0000 7f17e546b700 1 -- 172.21.15.3:0/104452383 >> v1:172.21.15.3:6789/0 conn(0x7f17e012ff20 legacy=0x7f17e0130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:56.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.783+0000 7f17e546b700 1 -- 172.21.15.3:0/104452383 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e0133860 con 0x7f17e0130a40 2023-10-03T10:17:56.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.783+0000 7f17df7fe700 1 -- 172.21.15.3:0/104452383 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3454609648 0 0) 0x7f17e006bc00 con 0x7f17e012ff20 2023-10-03T10:17:56.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.783+0000 7f17e546b700 1 --2- 172.21.15.3:0/104452383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17e0130a40 0x7f17e013b850 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f17d8009e50 tx=0x7f17d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22179cb5f004d33b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:56.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.783+0000 7f17df7fe700 1 -- 172.21.15.3:0/104452383 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d800e070 con 0x7f17e0130a40 2023-10-03T10:17:56.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.783+0000 7f17df7fe700 1 -- 172.21.15.3:0/104452383 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d800b910 con 0x7f17e0130a40 2023-10-03T10:17:56.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.783+0000 7f17df7fe700 1 -- 172.21.15.3:0/104452383 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d800bcc0 con 0x7f17e0130a40 2023-10-03T10:17:56.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.784+0000 7f17e6ece700 1 -- 172.21.15.3:0/104452383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17e0130a40 msgr2=0x7f17e013b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:56.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.784+0000 7f17e6ece700 1 --2- 172.21.15.3:0/104452383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17e0130a40 0x7f17e013b850 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f17d8009e50 tx=0x7f17d800b330 comp rx=0 tx=0).stop 2023-10-03T10:17:56.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.784+0000 7f17e6ece700 1 -- 172.21.15.3:0/104452383 shutdown_connections 2023-10-03T10:17:56.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.784+0000 7f17e6ece700 1 --2- 172.21.15.3:0/104452383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17e0130a40 0x7f17e013b850 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:56.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.784+0000 7f17e6ece700 1 --2- 172.21.15.3:0/104452383 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17e0133a80 0x7f17e0133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:56.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.784+0000 7f17e6ece700 1 -- 172.21.15.3:0/104452383 >> 172.21.15.3:0/104452383 conn(0x7f17e00a4210 msgr2=0x7f17e00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:56.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.784+0000 7f17e6ece700 1 -- 172.21.15.3:0/104452383 shutdown_connections 2023-10-03T10:17:56.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.784+0000 7f17e6ece700 1 -- 172.21.15.3:0/104452383 wait complete. 2023-10-03T10:17:56.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.785+0000 7f17e6ece700 1 Processor -- start 2023-10-03T10:17:56.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.785+0000 7f17e6ece700 1 -- start start 2023-10-03T10:17:56.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.786+0000 7f17e6ece700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17e012ff20 0x7f17e009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:56.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.786+0000 7f17e6ece700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f17e0130a40 0x7f17e009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:56.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.786+0000 7f17e6ece700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17e0133a80 0x7f17e009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:56.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.786+0000 7f17e6ece700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e009d850 con 0x7f17e012ff20 2023-10-03T10:17:56.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.786+0000 7f17e6ece700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e012d940 con 0x7f17e0133a80 2023-10-03T10:17:56.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.786+0000 7f17e6ece700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f17e011ff20 con 0x7f17e0130a40 2023-10-03T10:17:56.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.786+0000 7f17e546b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17e0133a80 0x7f17e009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:56.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.786+0000 7f17e546b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17e0133a80 0x7f17e009cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59240/0 (socket says 172.21.15.3:59240) 2023-10-03T10:17:56.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.786+0000 7f17e546b700 1 -- 172.21.15.3:0/157381572 learned_addr learned my addr 172.21.15.3:0/157381572 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:56.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.786+0000 7f17e4c6a700 1 --2- 172.21.15.3:0/157381572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17e012ff20 0x7f17e009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:56.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.787+0000 7f17e546b700 1 -- 172.21.15.3:0/157381572 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f17e0130a40 msgr2=0x7f17e009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:56.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.787+0000 7f17e546b700 1 --2- 172.21.15.3:0/157381572 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f17e0130a40 0x7f17e009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:56.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.787+0000 7f17e546b700 1 -- 172.21.15.3:0/157381572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17e012ff20 msgr2=0x7f17e009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:56.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.787+0000 7f17e546b700 1 --2- 172.21.15.3:0/157381572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17e012ff20 0x7f17e009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:56.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.787+0000 7f17e546b700 1 -- 172.21.15.3:0/157381572 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d8009b30 con 0x7f17e0133a80 2023-10-03T10:17:56.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.787+0000 7f17e546b700 1 --2- 172.21.15.3:0/157381572 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17e0133a80 0x7f17e009cf00 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f17d80042f0 tx=0x7f17d8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:56.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.787+0000 7f17dd7fa700 1 -- 172.21.15.3:0/157381572 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d8003ea0 con 0x7f17e0133a80 2023-10-03T10:17:56.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.787+0000 7f17dd7fa700 1 -- 172.21.15.3:0/157381572 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d80099b0 con 0x7f17e0133a80 2023-10-03T10:17:56.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.787+0000 7f17e6ece700 1 -- 172.21.15.3:0/157381572 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e009de80 con 0x7f17e0133a80 2023-10-03T10:17:56.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.787+0000 7f17dd7fa700 1 -- 172.21.15.3:0/157381572 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d801c660 con 0x7f17e0133a80 2023-10-03T10:17:56.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.787+0000 7f17e6ece700 1 -- 172.21.15.3:0/157381572 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e011fbd0 con 0x7f17e0133a80 2023-10-03T10:17:56.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.788+0000 7f17dd7fa700 1 -- 172.21.15.3:0/157381572 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f17d801c7c0 con 0x7f17e0133a80 2023-10-03T10:17:56.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.788+0000 7f17e6ece700 1 -- 172.21.15.3:0/157381572 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17e0074ea0 con 0x7f17e0133a80 2023-10-03T10:17:56.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.789+0000 7f17dd7fa700 1 -- 172.21.15.3:0/157381572 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f17d8041440 con 0x7f17e0133a80 2023-10-03T10:17:56.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.797+0000 7f17dd7fa700 1 -- 172.21.15.3:0/157381572 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f17d8025450 con 0x7f17e0133a80 2023-10-03T10:17:56.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.821+0000 7f55c8398700 1 Processor -- start 2023-10-03T10:17:56.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.821+0000 7f55c8398700 1 -- start start 2023-10-03T10:17:56.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.822+0000 7f55c8398700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f55c009e930 0x7f55c009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:56.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.822+0000 7f55c8398700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f55c009f650 0x7f55c0131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:56.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.822+0000 7f55c8398700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55c0070940 con 0x7f55c009f240 2023-10-03T10:17:56.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.822+0000 7f55c8398700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55c006fd60 con 0x7f55c009e930 2023-10-03T10:17:56.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.822+0000 7f55c8398700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f55c006fa80 con 0x7f55c009f650 2023-10-03T10:17:56.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.822+0000 7f55c6134700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f55c009e930 0x7f55c009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:56.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.822+0000 7f55c6935700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f55c009f650 0x7f55c0131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:56.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.822+0000 7f55c6134700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f55c009e930 0x7f55c009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44686/0 (socket says 172.21.15.3:44686) 2023-10-03T10:17:56.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.822+0000 7f55c6134700 1 -- 172.21.15.3:0/2439823131 learned_addr learned my addr 172.21.15.3:0/2439823131 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:56.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.822+0000 7f55c6134700 1 -- 172.21.15.3:0/2439823131 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f55c009f650 msgr2=0x7f55c0131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:56.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.822+0000 7f55c6134700 1 --2- 172.21.15.3:0/2439823131 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f55c009f650 0x7f55c0131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:56.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.822+0000 7f55c6134700 1 -- 172.21.15.3:0/2439823131 >> v1:172.21.15.3:6789/0 conn(0x7f55c009f240 legacy=0x7f55c01317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:56.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.822+0000 7f55c6134700 1 -- 172.21.15.3:0/2439823131 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c009fb60 con 0x7f55c009e930 2023-10-03T10:17:56.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.822+0000 7f55c6935700 1 --2- 172.21.15.3:0/2439823131 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f55c009f650 0x7f55c0131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:56.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.823+0000 7f55c6134700 1 --2- 172.21.15.3:0/2439823131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55c009e930 0x7f55c009ed00 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009e50 tx=0x7f55b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75cf9e5c4c0edd54 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:56.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.823+0000 7f55c5132700 1 -- 172.21.15.3:0/2439823131 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b000e070 con 0x7f55c009e930 2023-10-03T10:17:56.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.823+0000 7f55c5132700 1 -- 172.21.15.3:0/2439823131 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b000b910 con 0x7f55c009e930 2023-10-03T10:17:56.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.823+0000 7f55c5132700 1 -- 172.21.15.3:0/2439823131 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b000bcc0 con 0x7f55c009e930 2023-10-03T10:17:56.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.823+0000 7f55c8398700 1 -- 172.21.15.3:0/2439823131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55c009e930 msgr2=0x7f55c009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:56.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.823+0000 7f55c8398700 1 --2- 172.21.15.3:0/2439823131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55c009e930 0x7f55c009ed00 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009e50 tx=0x7f55b000b330 comp rx=0 tx=0).stop 2023-10-03T10:17:56.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.824+0000 7f55c8398700 1 -- 172.21.15.3:0/2439823131 shutdown_connections 2023-10-03T10:17:56.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.824+0000 7f55c8398700 1 --2- 172.21.15.3:0/2439823131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55c009e930 0x7f55c009ed00 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:56.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.824+0000 7f55c8398700 1 --2- 172.21.15.3:0/2439823131 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f55c009f650 0x7f55c0131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:56.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.824+0000 7f55c8398700 1 -- 172.21.15.3:0/2439823131 >> 172.21.15.3:0/2439823131 conn(0x7f55c00a4210 msgr2=0x7f55c00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:56.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.824+0000 7f55c8398700 1 -- 172.21.15.3:0/2439823131 shutdown_connections 2023-10-03T10:17:56.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.824+0000 7f55c8398700 1 -- 172.21.15.3:0/2439823131 wait complete. 2023-10-03T10:17:56.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.825+0000 7f55c8398700 1 Processor -- start 2023-10-03T10:17:56.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.825+0000 7f55c8398700 1 -- start start 2023-10-03T10:17:56.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.825+0000 7f55c8398700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f55c009e930 0x7f55c006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:56.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.825+0000 7f55c8398700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f55c009f240 0x7f55c006bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:56.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.825+0000 7f55c8398700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55c009f650 0x7f55c006c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:56.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.825+0000 7f55c8398700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c01aa9f0 con 0x7f55c009f650 2023-10-03T10:17:56.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.825+0000 7f55c8398700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c01aba30 con 0x7f55c009f240 2023-10-03T10:17:56.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.825+0000 7f55c8398700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f55c00a3530 con 0x7f55c009e930 2023-10-03T10:17:56.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.825+0000 7f55c6134700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f55c009e930 0x7f55c006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:56.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.825+0000 7f55c6134700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f55c009e930 0x7f55c006b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47826/0 (socket says 172.21.15.3:47826) 2023-10-03T10:17:56.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.825+0000 7f55c6134700 1 -- 172.21.15.3:0/901245751 learned_addr learned my addr 172.21.15.3:0/901245751 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:56.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.826+0000 7f55c6935700 1 --2- 172.21.15.3:0/901245751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55c009f650 0x7f55c006c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:56.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.826+0000 7f55c5933700 1 --2- 172.21.15.3:0/901245751 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f55c009f240 0x7f55c006bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:56.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.826+0000 7f55c6134700 1 -- 172.21.15.3:0/901245751 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f55c009f240 msgr2=0x7f55c006bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:56.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.826+0000 7f55c6134700 1 --2- 172.21.15.3:0/901245751 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f55c009f240 0x7f55c006bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:56.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.826+0000 7f55c6134700 1 -- 172.21.15.3:0/901245751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55c009f650 msgr2=0x7f55c006c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:56.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.826+0000 7f55c6134700 1 --2- 172.21.15.3:0/901245751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55c009f650 0x7f55c006c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:56.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.826+0000 7f55c6134700 1 -- 172.21.15.3:0/901245751 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b80094a0 con 0x7f55c009e930 2023-10-03T10:17:56.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.826+0000 7f55c6134700 1 --2- 172.21.15.3:0/901245751 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f55c009e930 0x7f55c006b8b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f55b000b370 tx=0x7f55b0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:56.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.826+0000 7f55beffd700 1 -- 172.21.15.3:0/901245751 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b00048b0 con 0x7f55c009e930 2023-10-03T10:17:56.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.826+0000 7f55beffd700 1 -- 172.21.15.3:0/901245751 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b00092e0 con 0x7f55c009e930 2023-10-03T10:17:56.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.826+0000 7f55beffd700 1 -- 172.21.15.3:0/901245751 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b001c680 con 0x7f55c009e930 2023-10-03T10:17:56.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.826+0000 7f55c8398700 1 -- 172.21.15.3:0/901245751 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b0009b30 con 0x7f55c009e930 2023-10-03T10:17:56.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.826+0000 7f55c8398700 1 -- 172.21.15.3:0/901245751 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c00a0190 con 0x7f55c009e930 2023-10-03T10:17:56.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.827+0000 7f55beffd700 1 -- 172.21.15.3:0/901245751 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f55b001c7e0 con 0x7f55c009e930 2023-10-03T10:17:56.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.828+0000 7f55beffd700 1 -- 172.21.15.3:0/901245751 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f55b00412b0 con 0x7f55c009e930 2023-10-03T10:17:56.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.828+0000 7f55c8398700 1 -- 172.21.15.3:0/901245751 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c01a2b40 con 0x7f55c009e930 2023-10-03T10:17:56.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.833+0000 7f55beffd700 1 -- 172.21.15.3:0/901245751 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f55b00257d0 con 0x7f55c009e930 2023-10-03T10:17:56.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.880+0000 7f1afd18e700 1 Processor -- start 2023-10-03T10:17:56.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.880+0000 7f1afd18e700 1 -- start start 2023-10-03T10:17:56.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.881+0000 7f1afd18e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1af80a0a30 0x7f1af80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:56.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.881+0000 7f1afd18e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1af80a1340 0x7f1af8133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:56.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.881+0000 7f1afd18e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af809edd0 con 0x7f1af8133fa0 2023-10-03T10:17:56.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.881+0000 7f1afd18e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1af809e1f0 con 0x7f1af80a1340 2023-10-03T10:17:56.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.881+0000 7f1afd18e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1af809dc50 con 0x7f1af80a0a30 2023-10-03T10:17:56.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.881+0000 7f1af6d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1af80a0a30 0x7f1af80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:56.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.881+0000 7f1af659c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1af80a1340 0x7f1af8133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:56.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.881+0000 7f1af6d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1af80a0a30 0x7f1af80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47830/0 (socket says 172.21.15.3:47830) 2023-10-03T10:17:56.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.881+0000 7f1af659c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1af80a1340 0x7f1af8133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44700/0 (socket says 172.21.15.3:44700) 2023-10-03T10:17:56.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.881+0000 7f1af6d9d700 1 -- 172.21.15.3:0/2647068012 learned_addr learned my addr 172.21.15.3:0/2647068012 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:56.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.881+0000 7f1af659c700 1 -- 172.21.15.3:0/2647068012 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1af80a0a30 msgr2=0x7f1af80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:56.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.881+0000 7f1af659c700 1 --2- 172.21.15.3:0/2647068012 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1af80a0a30 0x7f1af80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:56.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.881+0000 7f1af659c700 1 -- 172.21.15.3:0/2647068012 >> v1:172.21.15.3:6789/0 conn(0x7f1af8133fa0 legacy=0x7f1af813ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:56.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.881+0000 7f1af659c700 1 -- 172.21.15.3:0/2647068012 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af80a1fe0 con 0x7f1af80a1340 2023-10-03T10:17:56.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.881+0000 7f1af5d9b700 1 -- 172.21.15.3:0/2647068012 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 440198304 0 0) 0x7f1af809edd0 con 0x7f1af8133fa0 2023-10-03T10:17:56.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.882+0000 7f1af6d9d700 1 --2- 172.21.15.3:0/2647068012 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1af80a0a30 0x7f1af80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:56.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.882+0000 7f1af659c700 1 --2- 172.21.15.3:0/2647068012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af80a1340 0x7f1af8133a60 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f1aec009840 tx=0x7f1aec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2008c011be7c9a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:56.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.882+0000 7f1af5d9b700 1 -- 172.21.15.3:0/2647068012 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec011830 con 0x7f1af80a1340 2023-10-03T10:17:56.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.882+0000 7f1af5d9b700 1 -- 172.21.15.3:0/2647068012 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aec011990 con 0x7f1af80a1340 2023-10-03T10:17:56.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.882+0000 7f1af5d9b700 1 -- 172.21.15.3:0/2647068012 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec011d20 con 0x7f1af80a1340 2023-10-03T10:17:56.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.882+0000 7f1afd18e700 1 -- 172.21.15.3:0/2647068012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af80a1340 msgr2=0x7f1af8133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:56.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.882+0000 7f1afd18e700 1 --2- 172.21.15.3:0/2647068012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af80a1340 0x7f1af8133a60 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f1aec009840 tx=0x7f1aec009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:56.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.882+0000 7f1afd18e700 1 -- 172.21.15.3:0/2647068012 shutdown_connections 2023-10-03T10:17:56.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.883+0000 7f1afd18e700 1 --2- 172.21.15.3:0/2647068012 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1af80a0a30 0x7f1af80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:56.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.883+0000 7f1afd18e700 1 --2- 172.21.15.3:0/2647068012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af80a1340 0x7f1af8133a60 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:56.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.883+0000 7f1afd18e700 1 -- 172.21.15.3:0/2647068012 >> 172.21.15.3:0/2647068012 conn(0x7f1af80a6550 msgr2=0x7f1af80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:56.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.883+0000 7f1afd18e700 1 -- 172.21.15.3:0/2647068012 shutdown_connections 2023-10-03T10:17:56.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.883+0000 7f1afd18e700 1 -- 172.21.15.3:0/2647068012 wait complete. 2023-10-03T10:17:56.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.884+0000 7f1afd18e700 1 Processor -- start 2023-10-03T10:17:56.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.884+0000 7f1afd18e700 1 -- start start 2023-10-03T10:17:56.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.884+0000 7f1afd18e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af80a0a30 0x7f1af8122820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:56.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.884+0000 7f1afd18e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1af80a1340 0x7f1af8122d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:56.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.884+0000 7f1afd18e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af8133fa0 0x7f1af81232a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:56.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.885+0000 7f1afd18e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af8126f80 con 0x7f1af8133fa0 2023-10-03T10:17:56.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.885+0000 7f1afd18e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af8125d60 con 0x7f1af80a0a30 2023-10-03T10:17:56.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.885+0000 7f1afd18e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1af80a3580 con 0x7f1af80a1340 2023-10-03T10:17:56.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.885+0000 7f1af6d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af80a0a30 0x7f1af8122820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:56.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.885+0000 7f1af659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1af80a1340 0x7f1af8122d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:56.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.885+0000 7f1af659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1af80a1340 0x7f1af8122d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47832/0 (socket says 172.21.15.3:47832) 2023-10-03T10:17:56.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.885+0000 7f1af6d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af80a0a30 0x7f1af8122820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59262/0 (socket says 172.21.15.3:59262) 2023-10-03T10:17:56.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.885+0000 7f1af659c700 1 -- 172.21.15.3:0/1745066934 learned_addr learned my addr 172.21.15.3:0/1745066934 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:56.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.885+0000 7f1af759e700 1 --2- 172.21.15.3:0/1745066934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af8133fa0 0x7f1af81232a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:56.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.885+0000 7f1af659c700 1 -- 172.21.15.3:0/1745066934 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af80a0a30 msgr2=0x7f1af8122820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:56.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.885+0000 7f1af659c700 1 --2- 172.21.15.3:0/1745066934 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af80a0a30 0x7f1af8122820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:56.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.885+0000 7f1af659c700 1 -- 172.21.15.3:0/1745066934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af8133fa0 msgr2=0x7f1af81232a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:56.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.885+0000 7f1af659c700 1 --2- 172.21.15.3:0/1745066934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af8133fa0 0x7f1af81232a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:56.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.885+0000 7f1af659c700 1 -- 172.21.15.3:0/1745066934 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae8009550 con 0x7f1af80a1340 2023-10-03T10:17:56.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.885+0000 7f1af659c700 1 --2- 172.21.15.3:0/1745066934 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1af80a1340 0x7f1af8122d60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1aec0090b0 tx=0x7f1aec004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:56.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.885+0000 7f1adf7fe700 1 -- 172.21.15.3:0/1745066934 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec0036a0 con 0x7f1af80a1340 2023-10-03T10:17:56.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.886+0000 7f1adf7fe700 1 -- 172.21.15.3:0/1745066934 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aec01f070 con 0x7f1af80a1340 2023-10-03T10:17:56.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.886+0000 7f1adf7fe700 1 -- 172.21.15.3:0/1745066934 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec003a60 con 0x7f1af80a1340 2023-10-03T10:17:56.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.886+0000 7f1afd18e700 1 -- 172.21.15.3:0/1745066934 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aec0094a0 con 0x7f1af80a1340 2023-10-03T10:17:56.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.886+0000 7f1afd18e700 1 -- 172.21.15.3:0/1745066934 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af8124bf0 con 0x7f1af80a1340 2023-10-03T10:17:56.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.886+0000 7f1adf7fe700 1 -- 172.21.15.3:0/1745066934 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1aec003bc0 con 0x7f1af80a1340 2023-10-03T10:17:56.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.887+0000 7f1adf7fe700 1 -- 172.21.15.3:0/1745066934 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1aec021070 con 0x7f1af80a1340 2023-10-03T10:17:56.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.887+0000 7f1afd18e700 1 -- 172.21.15.3:0/1745066934 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af8125ed0 con 0x7f1af80a1340 2023-10-03T10:17:56.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:56.892+0000 7f1adf7fe700 1 -- 172.21.15.3:0/1745066934 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1aec02b080 con 0x7f1af80a1340 2023-10-03T10:17:57.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.009+0000 7f17e6ece700 1 -- 172.21.15.3:0/157381572 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f17e0134370 con 0x7f17e0133a80 2023-10-03T10:17:57.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.009+0000 7f17dd7fa700 1 -- 172.21.15.3:0/157381572 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f17e0134370 con 0x7f17e0133a80 2023-10-03T10:17:57.010 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:57.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.013+0000 7f17e6ece700 1 -- 172.21.15.3:0/157381572 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17e0133a80 msgr2=0x7f17e009cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:57.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.013+0000 7f17e6ece700 1 --2- 172.21.15.3:0/157381572 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17e0133a80 0x7f17e009cf00 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f17d80042f0 tx=0x7f17d8004ad0 comp rx=0 tx=0).stop 2023-10-03T10:17:57.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.013+0000 7f17e6ece700 1 -- 172.21.15.3:0/157381572 shutdown_connections 2023-10-03T10:17:57.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.013+0000 7f17e6ece700 1 --2- 172.21.15.3:0/157381572 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f17e0133a80 0x7f17e009cf00 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:57.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.013+0000 7f17e6ece700 1 --2- 172.21.15.3:0/157381572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17e012ff20 0x7f17e009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:57.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.013+0000 7f17e6ece700 1 --2- 172.21.15.3:0/157381572 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f17e0130a40 0x7f17e009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:57.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.013+0000 7f17e6ece700 1 -- 172.21.15.3:0/157381572 >> 172.21.15.3:0/157381572 conn(0x7f17e00a4210 msgr2=0x7f17e009e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:57.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.013+0000 7f17e6ece700 1 -- 172.21.15.3:0/157381572 shutdown_connections 2023-10-03T10:17:57.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.013+0000 7f17e6ece700 1 -- 172.21.15.3:0/157381572 wait complete. 2023-10-03T10:17:57.032 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:57.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.058+0000 7f55c8398700 1 -- 172.21.15.3:0/901245751 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f55c0074ea0 con 0x7f55c009e930 2023-10-03T10:17:57.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.058+0000 7f55beffd700 1 -- 172.21.15.3:0/901245751 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f55b0034090 con 0x7f55c009e930 2023-10-03T10:17:57.058 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:57.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.060+0000 7f55c8398700 1 -- 172.21.15.3:0/901245751 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f55c009e930 msgr2=0x7f55c006b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:57.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.060+0000 7f55c8398700 1 --2- 172.21.15.3:0/901245751 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f55c009e930 0x7f55c006b8b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f55b000b370 tx=0x7f55b0004ad0 comp rx=0 tx=0).stop 2023-10-03T10:17:57.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.060+0000 7f55c8398700 1 -- 172.21.15.3:0/901245751 shutdown_connections 2023-10-03T10:17:57.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.060+0000 7f55c8398700 1 --2- 172.21.15.3:0/901245751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55c009f650 0x7f55c006c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:57.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.060+0000 7f55c8398700 1 --2- 172.21.15.3:0/901245751 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f55c009e930 0x7f55c006b8b0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:57.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.060+0000 7f55c8398700 1 --2- 172.21.15.3:0/901245751 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f55c009f240 0x7f55c006bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:57.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.060+0000 7f55c8398700 1 -- 172.21.15.3:0/901245751 >> 172.21.15.3:0/901245751 conn(0x7f55c00a4210 msgr2=0x7f55c009dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:57.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.060+0000 7f55c8398700 1 -- 172.21.15.3:0/901245751 shutdown_connections 2023-10-03T10:17:57.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.060+0000 7f55c8398700 1 -- 172.21.15.3:0/901245751 wait complete. 2023-10-03T10:17:57.074 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:57.100 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:57.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.106+0000 7f1afd18e700 1 -- 172.21.15.3:0/1745066934 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1af8074ea0 con 0x7f1af80a1340 2023-10-03T10:17:57.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.106+0000 7f1adf7fe700 1 -- 172.21.15.3:0/1745066934 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1aec015020 con 0x7f1af80a1340 2023-10-03T10:17:57.107 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:57.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.109+0000 7f1afd18e700 1 -- 172.21.15.3:0/1745066934 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1af80a1340 msgr2=0x7f1af8122d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:57.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.109+0000 7f1afd18e700 1 --2- 172.21.15.3:0/1745066934 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1af80a1340 0x7f1af8122d60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1aec0090b0 tx=0x7f1aec004750 comp rx=0 tx=0).stop 2023-10-03T10:17:57.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.109+0000 7f1afd18e700 1 -- 172.21.15.3:0/1745066934 shutdown_connections 2023-10-03T10:17:57.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.109+0000 7f1afd18e700 1 --2- 172.21.15.3:0/1745066934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af8133fa0 0x7f1af81232a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:57.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.109+0000 7f1afd18e700 1 --2- 172.21.15.3:0/1745066934 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af80a0a30 0x7f1af8122820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:57.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.109+0000 7f1afd18e700 1 --2- 172.21.15.3:0/1745066934 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1af80a1340 0x7f1af8122d60 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:57.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.109+0000 7f1afd18e700 1 -- 172.21.15.3:0/1745066934 >> 172.21.15.3:0/1745066934 conn(0x7f1af80a6550 msgr2=0x7f1af811c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:57.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.109+0000 7f1afd18e700 1 -- 172.21.15.3:0/1745066934 shutdown_connections 2023-10-03T10:17:57.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.109+0000 7f1afd18e700 1 -- 172.21.15.3:0/1745066934 wait complete. 2023-10-03T10:17:57.123 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:57.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.243+0000 7fcff4f2b700 1 Processor -- start 2023-10-03T10:17:57.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.243+0000 7fcff4f2b700 1 -- start start 2023-10-03T10:17:57.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.244+0000 7fcff4f2b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcff006a770 0x7fcff006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:57.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.244+0000 7fcff4f2b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcff0131d10 0x7fcff0139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:57.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.244+0000 7fcff4f2b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcff009ece0 con 0x7fcff006b080 2023-10-03T10:17:57.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.244+0000 7fcff4f2b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff009e760 con 0x7fcff0131d10 2023-10-03T10:17:57.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.244+0000 7fcff4f2b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcff009e040 con 0x7fcff006a770 2023-10-03T10:17:57.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.244+0000 7fcfeed9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcff0131d10 0x7fcff0139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:57.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.244+0000 7fcfee59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcff006a770 0x7fcff006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:57.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.244+0000 7fcfee59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcff006a770 0x7fcff006ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47842/0 (socket says 172.21.15.3:47842) 2023-10-03T10:17:57.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.244+0000 7fcfeed9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcff0131d10 0x7fcff0139860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59270/0 (socket says 172.21.15.3:59270) 2023-10-03T10:17:57.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.244+0000 7fcfee59c700 1 -- 172.21.15.3:0/1628097803 learned_addr learned my addr 172.21.15.3:0/1628097803 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:57.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.244+0000 7fcfed59a700 1 -- 172.21.15.3:0/1628097803 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3298288904 0 0) 0x7fcff009ece0 con 0x7fcff006b080 2023-10-03T10:17:57.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.244+0000 7fcfee59c700 1 -- 172.21.15.3:0/1628097803 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcff0131d10 msgr2=0x7fcff0139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:57.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.244+0000 7fcfee59c700 1 --2- 172.21.15.3:0/1628097803 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcff0131d10 0x7fcff0139860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:57.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.244+0000 7fcfee59c700 1 -- 172.21.15.3:0/1628097803 >> v1:172.21.15.3:6789/0 conn(0x7fcff006b080 legacy=0x7fcff0131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:57.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.244+0000 7fcfee59c700 1 -- 172.21.15.3:0/1628097803 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff009f630 con 0x7fcff006a770 2023-10-03T10:17:57.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.244+0000 7fcfeed9d700 1 --2- 172.21.15.3:0/1628097803 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcff0131d10 0x7fcff0139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:57.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.244+0000 7fcfee59c700 1 --2- 172.21.15.3:0/1628097803 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcff006a770 0x7fcff006ab40 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0009920 tx=0x7fcfe0009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=879956dda9c8cb29 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:57.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.244+0000 7fcfed59a700 1 -- 172.21.15.3:0/1628097803 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe0011830 con 0x7fcff006a770 2023-10-03T10:17:57.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.245+0000 7fcfed59a700 1 -- 172.21.15.3:0/1628097803 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe0011990 con 0x7fcff006a770 2023-10-03T10:17:57.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.245+0000 7fcfed59a700 1 -- 172.21.15.3:0/1628097803 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe0011d20 con 0x7fcff006a770 2023-10-03T10:17:57.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.245+0000 7fcff4f2b700 1 -- 172.21.15.3:0/1628097803 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcff006a770 msgr2=0x7fcff006ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:57.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.245+0000 7fcff4f2b700 1 --2- 172.21.15.3:0/1628097803 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcff006a770 0x7fcff006ab40 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0009920 tx=0x7fcfe0009c30 comp rx=0 tx=0).stop 2023-10-03T10:17:57.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.245+0000 7fcff4f2b700 1 -- 172.21.15.3:0/1628097803 shutdown_connections 2023-10-03T10:17:57.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.245+0000 7fcff4f2b700 1 --2- 172.21.15.3:0/1628097803 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcff0131d10 0x7fcff0139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:57.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.245+0000 7fcff4f2b700 1 --2- 172.21.15.3:0/1628097803 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcff006a770 0x7fcff006ab40 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:57.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.245+0000 7fcff4f2b700 1 -- 172.21.15.3:0/1628097803 >> 172.21.15.3:0/1628097803 conn(0x7fcff0066f50 msgr2=0x7fcff009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:57.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.245+0000 7fcff4f2b700 1 -- 172.21.15.3:0/1628097803 shutdown_connections 2023-10-03T10:17:57.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.245+0000 7fcff4f2b700 1 -- 172.21.15.3:0/1628097803 wait complete. 2023-10-03T10:17:57.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.246+0000 7fcff4f2b700 1 Processor -- start 2023-10-03T10:17:57.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.246+0000 7fcff4f2b700 1 -- start start 2023-10-03T10:17:57.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.247+0000 7fcff4f2b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcff006a770 0x7fcff009ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:57.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.247+0000 7fcff4f2b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcff006b080 0x7fcff009b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:57.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.247+0000 7fcff4f2b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcff0131d10 0x7fcff01c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:57.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.247+0000 7fcff4f2b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff009b8a0 con 0x7fcff006a770 2023-10-03T10:17:57.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.247+0000 7fcff4f2b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff0071e60 con 0x7fcff006b080 2023-10-03T10:17:57.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.247+0000 7fcff4f2b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcff009a9b0 con 0x7fcff0131d10 2023-10-03T10:17:57.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.247+0000 7fcfedd9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcff006b080 0x7fcff009b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:57.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.247+0000 7fcfee59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcff006a770 0x7fcff009ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:57.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.247+0000 7fcfeed9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcff0131d10 0x7fcff01c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:57.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.247+0000 7fcfedd9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcff006b080 0x7fcff009b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59280/0 (socket says 172.21.15.3:59280) 2023-10-03T10:17:57.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.247+0000 7fcfee59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcff006a770 0x7fcff009ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44722/0 (socket says 172.21.15.3:44722) 2023-10-03T10:17:57.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.247+0000 7fcfedd9b700 1 -- 172.21.15.3:0/3102657563 learned_addr learned my addr 172.21.15.3:0/3102657563 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:57.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.247+0000 7fcfeed9d700 1 -- 172.21.15.3:0/3102657563 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcff006b080 msgr2=0x7fcff009b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:57.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.247+0000 7fcfeed9d700 1 --2- 172.21.15.3:0/3102657563 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcff006b080 0x7fcff009b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:57.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.247+0000 7fcfeed9d700 1 -- 172.21.15.3:0/3102657563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcff006a770 msgr2=0x7fcff009ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:57.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.247+0000 7fcfeed9d700 1 --2- 172.21.15.3:0/3102657563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcff006a770 0x7fcff009ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:57.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.247+0000 7fcfeed9d700 1 -- 172.21.15.3:0/3102657563 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe8009a70 con 0x7fcff0131d10 2023-10-03T10:17:57.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.248+0000 7fcfeed9d700 1 --2- 172.21.15.3:0/3102657563 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcff0131d10 0x7fcff01c8120 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009e50 tx=0x7fcfe800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:57.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.248+0000 7fcfe6ffd700 1 -- 172.21.15.3:0/3102657563 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe800ca90 con 0x7fcff0131d10 2023-10-03T10:17:57.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.248+0000 7fcfe6ffd700 1 -- 172.21.15.3:0/3102657563 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe800f040 con 0x7fcff0131d10 2023-10-03T10:17:57.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.248+0000 7fcff4f2b700 1 -- 172.21.15.3:0/3102657563 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe0009550 con 0x7fcff0131d10 2023-10-03T10:17:57.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.248+0000 7fcfe6ffd700 1 -- 172.21.15.3:0/3102657563 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe800ce50 con 0x7fcff0131d10 2023-10-03T10:17:57.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.248+0000 7fcff4f2b700 1 -- 172.21.15.3:0/3102657563 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff01269b0 con 0x7fcff0131d10 2023-10-03T10:17:57.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.249+0000 7fcfe6ffd700 1 -- 172.21.15.3:0/3102657563 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcfe8014480 con 0x7fcff0131d10 2023-10-03T10:17:57.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.249+0000 7fcfe6ffd700 1 -- 172.21.15.3:0/3102657563 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fcfe803fc30 con 0x7fcff0131d10 2023-10-03T10:17:57.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.249+0000 7fcff4f2b700 1 -- 172.21.15.3:0/3102657563 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff0131b30 con 0x7fcff0131d10 2023-10-03T10:17:57.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.254+0000 7fcfe6ffd700 1 -- 172.21.15.3:0/3102657563 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcfe8012910 con 0x7fcff0131d10 2023-10-03T10:17:57.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.461+0000 7fcff4f2b700 1 -- 172.21.15.3:0/3102657563 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcff0074ea0 con 0x7fcff0131d10 2023-10-03T10:17:57.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.462+0000 7fcfe6ffd700 1 -- 172.21.15.3:0/3102657563 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcfe8012340 con 0x7fcff0131d10 2023-10-03T10:17:57.462 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:57.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.464+0000 7fcff4f2b700 1 -- 172.21.15.3:0/3102657563 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcff0131d10 msgr2=0x7fcff01c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:57.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.464+0000 7fcff4f2b700 1 --2- 172.21.15.3:0/3102657563 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcff0131d10 0x7fcff01c8120 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009e50 tx=0x7fcfe800c3a0 comp rx=0 tx=0).stop 2023-10-03T10:17:57.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.464+0000 7fcff4f2b700 1 -- 172.21.15.3:0/3102657563 shutdown_connections 2023-10-03T10:17:57.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.464+0000 7fcff4f2b700 1 --2- 172.21.15.3:0/3102657563 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcff0131d10 0x7fcff01c8120 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:57.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.464+0000 7fcff4f2b700 1 --2- 172.21.15.3:0/3102657563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcff006a770 0x7fcff009ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:57.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.464+0000 7fcff4f2b700 1 --2- 172.21.15.3:0/3102657563 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcff006b080 0x7fcff009b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:57.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.464+0000 7fcff4f2b700 1 -- 172.21.15.3:0/3102657563 >> 172.21.15.3:0/3102657563 conn(0x7fcff0066f50 msgr2=0x7fcff00700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:57.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.464+0000 7fcff4f2b700 1 -- 172.21.15.3:0/3102657563 shutdown_connections 2023-10-03T10:17:57.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:57.464+0000 7fcff4f2b700 1 -- 172.21.15.3:0/3102657563 wait complete. 2023-10-03T10:17:57.478 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:58.033 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:58.074 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:58.124 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:58.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.185+0000 7fd1c8e2b700 1 Processor -- start 2023-10-03T10:17:58.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.185+0000 7fd1c8e2b700 1 -- start start 2023-10-03T10:17:58.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.186+0000 7fd1c8e2b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd1c406f5d0 0x7fd1c412ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:58.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.186+0000 7fd1c8e2b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1c4135c20 0x7fd1c4138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:58.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.186+0000 7fd1c8e2b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1c406fa40 con 0x7fd1c412cf60 2023-10-03T10:17:58.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.186+0000 7fd1c8e2b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1c412cd90 con 0x7fd1c406f5d0 2023-10-03T10:17:58.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.186+0000 7fd1c8e2b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c4071780 con 0x7fd1c4135c20 2023-10-03T10:17:58.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.186+0000 7fd1c1d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd1c406f5d0 0x7fd1c412ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:58.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.186+0000 7fd1c1d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd1c406f5d0 0x7fd1c412ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44730/0 (socket says 172.21.15.3:44730) 2023-10-03T10:17:58.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.186+0000 7fd1c1d9b700 1 -- 172.21.15.3:0/2823185504 learned_addr learned my addr 172.21.15.3:0/2823185504 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:58.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.186+0000 7fd1c2d9d700 1 --2- 172.21.15.3:0/2823185504 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1c4135c20 0x7fd1c4138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:58.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.187+0000 7fd1c2d9d700 1 -- 172.21.15.3:0/2823185504 >> v2:172.21.15.3:3300/0 conn(0x7fd1c406f5d0 msgr2=0x7fd1c412ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:58.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.187+0000 7fd1c2d9d700 1 --2- 172.21.15.3:0/2823185504 >> v2:172.21.15.3:3300/0 conn(0x7fd1c406f5d0 0x7fd1c412ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.187+0000 7fd1c2d9d700 1 -- 172.21.15.3:0/2823185504 >> v1:172.21.15.3:6789/0 conn(0x7fd1c412cf60 legacy=0x7fd1c412f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:58.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.187+0000 7fd1c2d9d700 1 -- 172.21.15.3:0/2823185504 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c4071a40 con 0x7fd1c4135c20 2023-10-03T10:17:58.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.187+0000 7fd1c1d9b700 1 --2- 172.21.15.3:0/2823185504 >> v2:172.21.15.3:3300/0 conn(0x7fd1c406f5d0 0x7fd1c412ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:58.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.187+0000 7fd1c2d9d700 1 --2- 172.21.15.3:0/2823185504 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1c4135c20 0x7fd1c4138000 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc009d90 tx=0x7fd1bc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9bee459e00be254 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:58.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.187+0000 7fd1c159a700 1 -- 172.21.15.3:0/2823185504 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc00e070 con 0x7fd1c4135c20 2023-10-03T10:17:58.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.187+0000 7fd1c159a700 1 -- 172.21.15.3:0/2823185504 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1bc00bb20 con 0x7fd1c4135c20 2023-10-03T10:17:58.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.187+0000 7fd1c159a700 1 -- 172.21.15.3:0/2823185504 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc012430 con 0x7fd1c4135c20 2023-10-03T10:17:58.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.188+0000 7fd1c8e2b700 1 -- 172.21.15.3:0/2823185504 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1c4135c20 msgr2=0x7fd1c4138000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:58.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.188+0000 7fd1c8e2b700 1 --2- 172.21.15.3:0/2823185504 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1c4135c20 0x7fd1c4138000 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc009d90 tx=0x7fd1bc00b330 comp rx=0 tx=0).stop 2023-10-03T10:17:58.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.188+0000 7fd1c8e2b700 1 -- 172.21.15.3:0/2823185504 shutdown_connections 2023-10-03T10:17:58.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.188+0000 7fd1c8e2b700 1 --2- 172.21.15.3:0/2823185504 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1c4135c20 0x7fd1c4138000 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.188+0000 7fd1c8e2b700 1 --2- 172.21.15.3:0/2823185504 >> v2:172.21.15.3:3300/0 conn(0x7fd1c406f5d0 0x7fd1c412ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.188+0000 7fd1c8e2b700 1 -- 172.21.15.3:0/2823185504 >> 172.21.15.3:0/2823185504 conn(0x7fd1c40a2aa0 msgr2=0x7fd1c40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:58.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.188+0000 7fd1c8e2b700 1 -- 172.21.15.3:0/2823185504 shutdown_connections 2023-10-03T10:17:58.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.188+0000 7fd1c8e2b700 1 -- 172.21.15.3:0/2823185504 wait complete. 2023-10-03T10:17:58.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.189+0000 7fd1c8e2b700 1 Processor -- start 2023-10-03T10:17:58.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.189+0000 7fd1c8e2b700 1 -- start start 2023-10-03T10:17:58.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.190+0000 7fd1c8e2b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd1c406f5d0 0x7fd1c4099d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:58.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.190+0000 7fd1c8e2b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1c412cf60 0x7fd1c409a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:58.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.190+0000 7fd1c8e2b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1c4135c20 0x7fd1c409a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:58.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.190+0000 7fd1c8e2b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c409b9e0 con 0x7fd1c4135c20 2023-10-03T10:17:58.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.190+0000 7fd1c8e2b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c409d1d0 con 0x7fd1c412cf60 2023-10-03T10:17:58.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.190+0000 7fd1c8e2b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c40a1040 con 0x7fd1c406f5d0 2023-10-03T10:17:58.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.190+0000 7fd1c1d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1c412cf60 0x7fd1c409a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:58.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.190+0000 7fd1c1d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1c412cf60 0x7fd1c409a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59300/0 (socket says 172.21.15.3:59300) 2023-10-03T10:17:58.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.190+0000 7fd1c1d9b700 1 -- 172.21.15.3:0/165420694 learned_addr learned my addr 172.21.15.3:0/165420694 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:58.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.190+0000 7fd1c2d9d700 1 --2- 172.21.15.3:0/165420694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1c4135c20 0x7fd1c409a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:58.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.190+0000 7fd1c1d9b700 1 -- 172.21.15.3:0/165420694 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd1c406f5d0 msgr2=0x7fd1c4099d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:58.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.190+0000 7fd1c1d9b700 1 --2- 172.21.15.3:0/165420694 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd1c406f5d0 0x7fd1c4099d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.190+0000 7fd1c1d9b700 1 -- 172.21.15.3:0/165420694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1c4135c20 msgr2=0x7fd1c409a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:58.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.190+0000 7fd1c1d9b700 1 --2- 172.21.15.3:0/165420694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1c4135c20 0x7fd1c409a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.190+0000 7fd1c1d9b700 1 -- 172.21.15.3:0/165420694 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b8009550 con 0x7fd1c412cf60 2023-10-03T10:17:58.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.190+0000 7fd1c2d9d700 1 --2- 172.21.15.3:0/165420694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1c4135c20 0x7fd1c409a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:17:58.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.190+0000 7fd1c1d9b700 1 --2- 172.21.15.3:0/165420694 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1c412cf60 0x7fd1c409a290 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd1b800ebe0 tx=0x7fd1b8009e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:58.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.191+0000 7fd1b2ffd700 1 -- 172.21.15.3:0/165420694 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b800c8b0 con 0x7fd1c412cf60 2023-10-03T10:17:58.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.191+0000 7fd1b2ffd700 1 -- 172.21.15.3:0/165420694 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b800ca10 con 0x7fd1c412cf60 2023-10-03T10:17:58.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.191+0000 7fd1c8e2b700 1 -- 172.21.15.3:0/165420694 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1bc009a70 con 0x7fd1c412cf60 2023-10-03T10:17:58.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.191+0000 7fd1b2ffd700 1 -- 172.21.15.3:0/165420694 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b800cd80 con 0x7fd1c412cf60 2023-10-03T10:17:58.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.191+0000 7fd1c8e2b700 1 -- 172.21.15.3:0/165420694 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c409df00 con 0x7fd1c412cf60 2023-10-03T10:17:58.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.192+0000 7fd1c8e2b700 1 -- 172.21.15.3:0/165420694 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c4074ea0 con 0x7fd1c412cf60 2023-10-03T10:17:58.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.192+0000 7fd1b2ffd700 1 -- 172.21.15.3:0/165420694 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd1b8013470 con 0x7fd1c412cf60 2023-10-03T10:17:58.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.192+0000 7fd1b2ffd700 1 -- 172.21.15.3:0/165420694 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd1b803f990 con 0x7fd1c412cf60 2023-10-03T10:17:58.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.197+0000 7fd1b2ffd700 1 -- 172.21.15.3:0/165420694 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd1b80232f0 con 0x7fd1c412cf60 2023-10-03T10:17:58.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.221+0000 7fe49cd95700 1 Processor -- start 2023-10-03T10:17:58.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.221+0000 7fe49cd95700 1 -- start start 2023-10-03T10:17:58.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.222+0000 7fe49cd95700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe498136830 0x7fe498138c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:58.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.222+0000 7fe49cd95700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe498139150 0x7fe49813b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:58.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.222+0000 7fe49cd95700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe49806f720 con 0x7fe4981338c0 2023-10-03T10:17:58.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.222+0000 7fe49cd95700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe498070c80 con 0x7fe498139150 2023-10-03T10:17:58.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.222+0000 7fe49cd95700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe49806fce0 con 0x7fe498136830 2023-10-03T10:17:58.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.222+0000 7fe495d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe498136830 0x7fe498138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:58.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.222+0000 7fe49659c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe4981338c0 0x7fe498133c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45144/0 (socket says 172.21.15.3:45144) 2023-10-03T10:17:58.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.222+0000 7fe49659c700 1 -- 172.21.15.3:0/40999238 learned_addr learned my addr 172.21.15.3:0/40999238 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:58.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.222+0000 7fe496d9d700 1 --2- 172.21.15.3:0/40999238 >> v2:172.21.15.3:3300/0 conn(0x7fe498139150 0x7fe49813b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:58.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.223+0000 7fe495d9b700 1 -- 172.21.15.3:0/40999238 >> v2:172.21.15.3:3300/0 conn(0x7fe498139150 msgr2=0x7fe49813b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:58.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.223+0000 7fe495d9b700 1 --2- 172.21.15.3:0/40999238 >> v2:172.21.15.3:3300/0 conn(0x7fe498139150 0x7fe49813b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.223+0000 7fe495d9b700 1 -- 172.21.15.3:0/40999238 >> v1:172.21.15.3:6789/0 conn(0x7fe4981338c0 legacy=0x7fe498133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:58.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.223+0000 7fe495d9b700 1 -- 172.21.15.3:0/40999238 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4981336a0 con 0x7fe498136830 2023-10-03T10:17:58.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.223+0000 7fe496d9d700 1 --2- 172.21.15.3:0/40999238 >> v2:172.21.15.3:3300/0 conn(0x7fe498139150 0x7fe49813b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:58.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.223+0000 7fe495d9b700 1 --2- 172.21.15.3:0/40999238 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe498136830 0x7fe498138c10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009840 tx=0x7fe48c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b50994ad3bdc898b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:58.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.223+0000 7fe49559a700 1 -- 172.21.15.3:0/40999238 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c011830 con 0x7fe498136830 2023-10-03T10:17:58.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.223+0000 7fe49559a700 1 -- 172.21.15.3:0/40999238 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe48c011990 con 0x7fe498136830 2023-10-03T10:17:58.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.223+0000 7fe49559a700 1 -- 172.21.15.3:0/40999238 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c011d20 con 0x7fe498136830 2023-10-03T10:17:58.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.224+0000 7fe49cd95700 1 -- 172.21.15.3:0/40999238 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe498136830 msgr2=0x7fe498138c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:58.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.224+0000 7fe49cd95700 1 --2- 172.21.15.3:0/40999238 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe498136830 0x7fe498138c10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009840 tx=0x7fe48c009b50 comp rx=0 tx=0).stop 2023-10-03T10:17:58.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.224+0000 7fe49cd95700 1 -- 172.21.15.3:0/40999238 shutdown_connections 2023-10-03T10:17:58.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.224+0000 7fe49cd95700 1 --2- 172.21.15.3:0/40999238 >> v2:172.21.15.3:3300/0 conn(0x7fe498139150 0x7fe49813b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.224+0000 7fe49cd95700 1 --2- 172.21.15.3:0/40999238 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe498136830 0x7fe498138c10 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.224+0000 7fe49cd95700 1 -- 172.21.15.3:0/40999238 >> 172.21.15.3:0/40999238 conn(0x7fe4980a2aa0 msgr2=0x7fe4980a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:58.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.224+0000 7fe49cd95700 1 -- 172.21.15.3:0/40999238 shutdown_connections 2023-10-03T10:17:58.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.224+0000 7fe49cd95700 1 -- 172.21.15.3:0/40999238 wait complete. 2023-10-03T10:17:58.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.226+0000 7fe49cd95700 1 Processor -- start 2023-10-03T10:17:58.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.226+0000 7fe49cd95700 1 -- start start 2023-10-03T10:17:58.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.227+0000 7fe49cd95700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4981338c0 0x7fe4981c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:58.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.227+0000 7fe49cd95700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe498136830 0x7fe4981c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:58.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.227+0000 7fe49cd95700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe498139150 0x7fe498118f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:58.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.227+0000 7fe49cd95700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4981c91f0 con 0x7fe4981338c0 2023-10-03T10:17:58.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.227+0000 7fe49cd95700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49811a930 con 0x7fe498139150 2023-10-03T10:17:58.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.227+0000 7fe49cd95700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe49806f9d0 con 0x7fe498136830 2023-10-03T10:17:58.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.227+0000 7fe49659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4981338c0 0x7fe4981c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:58.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.227+0000 7fe495d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe498136830 0x7fe4981c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:58.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.227+0000 7fe495d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe498136830 0x7fe4981c8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47894/0 (socket says 172.21.15.3:47894) 2023-10-03T10:17:58.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.227+0000 7fe495d9b700 1 -- 172.21.15.3:0/4043040905 learned_addr learned my addr 172.21.15.3:0/4043040905 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:58.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.227+0000 7fe496d9d700 1 --2- 172.21.15.3:0/4043040905 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe498139150 0x7fe498118f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:58.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.227+0000 7fe49659c700 1 -- 172.21.15.3:0/4043040905 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe498136830 msgr2=0x7fe4981c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:58.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.227+0000 7fe49659c700 1 --2- 172.21.15.3:0/4043040905 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe498136830 0x7fe4981c8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.227+0000 7fe49659c700 1 -- 172.21.15.3:0/4043040905 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe498139150 msgr2=0x7fe498118f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:58.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.227+0000 7fe49659c700 1 --2- 172.21.15.3:0/4043040905 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe498139150 0x7fe498118f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.227+0000 7fe49659c700 1 -- 172.21.15.3:0/4043040905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe490009a70 con 0x7fe4981338c0 2023-10-03T10:17:58.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.227+0000 7fe496d9d700 1 --2- 172.21.15.3:0/4043040905 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe498139150 0x7fe498118f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:58.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.227+0000 7fe495d9b700 1 --2- 172.21.15.3:0/4043040905 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe498136830 0x7fe4981c8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:58.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.228+0000 7fe49659c700 1 --2- 172.21.15.3:0/4043040905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4981338c0 0x7fe4981c8770 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fe48800c530 tx=0x7fe48800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:58.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.228+0000 7fe486ffd700 1 -- 172.21.15.3:0/4043040905 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe488005d70 con 0x7fe4981338c0 2023-10-03T10:17:58.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.228+0000 7fe486ffd700 1 -- 172.21.15.3:0/4043040905 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe48800ce40 con 0x7fe4981338c0 2023-10-03T10:17:58.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.228+0000 7fe486ffd700 1 -- 172.21.15.3:0/4043040905 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4880072d0 con 0x7fe4981338c0 2023-10-03T10:17:58.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.228+0000 7fe49cd95700 1 -- 172.21.15.3:0/4043040905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe48c0094a0 con 0x7fe4981338c0 2023-10-03T10:17:58.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.228+0000 7fe49cd95700 1 -- 172.21.15.3:0/4043040905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4981360d0 con 0x7fe4981338c0 2023-10-03T10:17:58.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.229+0000 7fe486ffd700 1 -- 172.21.15.3:0/4043040905 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe488007430 con 0x7fe4981338c0 2023-10-03T10:17:58.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.229+0000 7fe486ffd700 1 -- 172.21.15.3:0/4043040905 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe48803ea70 con 0x7fe4981338c0 2023-10-03T10:17:58.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.229+0000 7fe49cd95700 1 -- 172.21.15.3:0/4043040905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe49811af50 con 0x7fe4981338c0 2023-10-03T10:17:58.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.234+0000 7fe486ffd700 1 -- 172.21.15.3:0/4043040905 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe488017080 con 0x7fe4981338c0 2023-10-03T10:17:58.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.270+0000 7f64ecdc0700 1 Processor -- start 2023-10-03T10:17:58.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.270+0000 7f64ecdc0700 1 -- start start 2023-10-03T10:17:58.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.270+0000 7f64ecdc0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64e80a1510 0x7f64e8138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:58.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.270+0000 7f64ecdc0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f64e8139250 0x7f64e813b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:58.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.270+0000 7f64ecdc0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64e809edd0 con 0x7f64e80a0a30 2023-10-03T10:17:58.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.270+0000 7f64ecdc0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64e80a18e0 con 0x7f64e8139250 2023-10-03T10:17:58.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.270+0000 7f64ecdc0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f64e809e1f0 con 0x7f64e80a1510 2023-10-03T10:17:58.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.270+0000 7f64e6d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f64e8139250 0x7f64e813b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:58.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.270+0000 7f64e659c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f64e80a0a30 0x7f64e80a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45150/0 (socket says 172.21.15.3:45150) 2023-10-03T10:17:58.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.270+0000 7f64e659c700 1 -- 172.21.15.3:0/2166536929 learned_addr learned my addr 172.21.15.3:0/2166536929 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:58.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.270+0000 7f64ddd9b700 1 --2- 172.21.15.3:0/2166536929 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64e80a1510 0x7f64e8138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:58.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.271+0000 7f64e6d9d700 1 -- 172.21.15.3:0/2166536929 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64e80a1510 msgr2=0x7f64e8138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:58.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.271+0000 7f64e6d9d700 1 --2- 172.21.15.3:0/2166536929 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64e80a1510 0x7f64e8138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.271+0000 7f64e6d9d700 1 -- 172.21.15.3:0/2166536929 >> v1:172.21.15.3:6789/0 conn(0x7f64e80a0a30 legacy=0x7f64e80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:58.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.271+0000 7f64e6d9d700 1 -- 172.21.15.3:0/2166536929 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e809dc50 con 0x7f64e8139250 2023-10-03T10:17:58.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.271+0000 7f64ddd9b700 1 --2- 172.21.15.3:0/2166536929 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64e80a1510 0x7f64e8138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:58.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.271+0000 7f64e6d9d700 1 --2- 172.21.15.3:0/2166536929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64e8139250 0x7f64e813b630 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009dc0 tx=0x7f64e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e4fe132a37d58a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:58.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.271+0000 7f64e5d9b700 1 -- 172.21.15.3:0/2166536929 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e000e070 con 0x7f64e8139250 2023-10-03T10:17:58.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.271+0000 7f64e5d9b700 1 -- 172.21.15.3:0/2166536929 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e000bae0 con 0x7f64e8139250 2023-10-03T10:17:58.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.271+0000 7f64e5d9b700 1 -- 172.21.15.3:0/2166536929 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e000be90 con 0x7f64e8139250 2023-10-03T10:17:58.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.272+0000 7f64ecdc0700 1 -- 172.21.15.3:0/2166536929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64e8139250 msgr2=0x7f64e813b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:58.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.272+0000 7f64ecdc0700 1 --2- 172.21.15.3:0/2166536929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64e8139250 0x7f64e813b630 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009dc0 tx=0x7f64e000b330 comp rx=0 tx=0).stop 2023-10-03T10:17:58.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.272+0000 7f64ecdc0700 1 -- 172.21.15.3:0/2166536929 shutdown_connections 2023-10-03T10:17:58.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.272+0000 7f64ecdc0700 1 --2- 172.21.15.3:0/2166536929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64e8139250 0x7f64e813b630 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.272+0000 7f64ecdc0700 1 --2- 172.21.15.3:0/2166536929 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64e80a1510 0x7f64e8138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.272+0000 7f64ecdc0700 1 -- 172.21.15.3:0/2166536929 >> 172.21.15.3:0/2166536929 conn(0x7f64e80a6550 msgr2=0x7f64e80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:58.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.272+0000 7f64ecdc0700 1 -- 172.21.15.3:0/2166536929 shutdown_connections 2023-10-03T10:17:58.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.272+0000 7f64ecdc0700 1 -- 172.21.15.3:0/2166536929 wait complete. 2023-10-03T10:17:58.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.274+0000 7f64ecdc0700 1 Processor -- start 2023-10-03T10:17:58.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.274+0000 7f64ecdc0700 1 -- start start 2023-10-03T10:17:58.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.274+0000 7f64ecdc0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64e80a0a30 0x7f64e81c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:58.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.274+0000 7f64ecdc0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64e80a1510 0x7f64e81c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:58.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.274+0000 7f64ecdc0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64e8139250 0x7f64e81227d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:58.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.274+0000 7f64ecdc0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e81243e0 con 0x7f64e8139250 2023-10-03T10:17:58.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.274+0000 7f64ecdc0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e81263d0 con 0x7f64e80a0a30 2023-10-03T10:17:58.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.274+0000 7f64ecdc0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f64e80a0220 con 0x7f64e80a1510 2023-10-03T10:17:58.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.274+0000 7f64ddd9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64e80a1510 0x7f64e81c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:58.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.275+0000 7f64ddd9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64e80a1510 0x7f64e81c8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47912/0 (socket says 172.21.15.3:47912) 2023-10-03T10:17:58.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.275+0000 7f64e659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64e80a0a30 0x7f64e81c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:58.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.275+0000 7f64ddd9b700 1 -- 172.21.15.3:0/2185422951 learned_addr learned my addr 172.21.15.3:0/2185422951 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:58.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.275+0000 7f64e6d9d700 1 --2- 172.21.15.3:0/2185422951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64e8139250 0x7f64e81227d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:58.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.275+0000 7f64ddd9b700 1 -- 172.21.15.3:0/2185422951 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64e80a0a30 msgr2=0x7f64e81c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:58.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.275+0000 7f64ddd9b700 1 --2- 172.21.15.3:0/2185422951 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64e80a0a30 0x7f64e81c8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.275+0000 7f64ddd9b700 1 -- 172.21.15.3:0/2185422951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64e8139250 msgr2=0x7f64e81227d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:58.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.275+0000 7f64ddd9b700 1 --2- 172.21.15.3:0/2185422951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64e8139250 0x7f64e81227d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.275+0000 7f64ddd9b700 1 -- 172.21.15.3:0/2185422951 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d00094a0 con 0x7f64e80a1510 2023-10-03T10:17:58.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.275+0000 7f64ddd9b700 1 --2- 172.21.15.3:0/2185422951 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64e80a1510 0x7f64e81c8b60 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f64d000ebe0 tx=0x7f64d000eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:58.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.275+0000 7f64df7fe700 1 -- 172.21.15.3:0/2185422951 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d0009800 con 0x7f64e80a1510 2023-10-03T10:17:58.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.275+0000 7f64df7fe700 1 -- 172.21.15.3:0/2185422951 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d000c910 con 0x7f64e80a1510 2023-10-03T10:17:58.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.275+0000 7f64df7fe700 1 -- 172.21.15.3:0/2185422951 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d000cc80 con 0x7f64e80a1510 2023-10-03T10:17:58.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.275+0000 7f64ecdc0700 1 -- 172.21.15.3:0/2185422951 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e0009a70 con 0x7f64e80a1510 2023-10-03T10:17:58.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.275+0000 7f64ecdc0700 1 -- 172.21.15.3:0/2185422951 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e8125d50 con 0x7f64e80a1510 2023-10-03T10:17:58.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.276+0000 7f64ecdc0700 1 -- 172.21.15.3:0/2185422951 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e8126230 con 0x7f64e80a1510 2023-10-03T10:17:58.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.277+0000 7f64df7fe700 1 -- 172.21.15.3:0/2185422951 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f64d0013470 con 0x7f64e80a1510 2023-10-03T10:17:58.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.277+0000 7f64df7fe700 1 -- 172.21.15.3:0/2185422951 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f64d003fee0 con 0x7f64e80a1510 2023-10-03T10:17:58.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.282+0000 7f64df7fe700 1 -- 172.21.15.3:0/2185422951 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f64d000cde0 con 0x7f64e80a1510 2023-10-03T10:17:58.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.409+0000 7fd1c8e2b700 1 -- 172.21.15.3:0/165420694 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd1c40713a0 con 0x7fd1c412cf60 2023-10-03T10:17:58.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.409+0000 7fd1b2ffd700 1 -- 172.21.15.3:0/165420694 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd1b801ae80 con 0x7fd1c412cf60 2023-10-03T10:17:58.410 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:58.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.412+0000 7fd1c8e2b700 1 -- 172.21.15.3:0/165420694 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1c412cf60 msgr2=0x7fd1c409a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:58.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.412+0000 7fd1c8e2b700 1 --2- 172.21.15.3:0/165420694 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1c412cf60 0x7fd1c409a290 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd1b800ebe0 tx=0x7fd1b8009e70 comp rx=0 tx=0).stop 2023-10-03T10:17:58.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.412+0000 7fd1c8e2b700 1 -- 172.21.15.3:0/165420694 shutdown_connections 2023-10-03T10:17:58.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.412+0000 7fd1c8e2b700 1 --2- 172.21.15.3:0/165420694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1c4135c20 0x7fd1c409a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.412+0000 7fd1c8e2b700 1 --2- 172.21.15.3:0/165420694 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd1c406f5d0 0x7fd1c4099d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.412+0000 7fd1c8e2b700 1 --2- 172.21.15.3:0/165420694 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1c412cf60 0x7fd1c409a290 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.412+0000 7fd1c8e2b700 1 -- 172.21.15.3:0/165420694 >> 172.21.15.3:0/165420694 conn(0x7fd1c40a2aa0 msgr2=0x7fd1c412f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:58.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.412+0000 7fd1c8e2b700 1 -- 172.21.15.3:0/165420694 shutdown_connections 2023-10-03T10:17:58.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.412+0000 7fd1c8e2b700 1 -- 172.21.15.3:0/165420694 wait complete. 2023-10-03T10:17:58.426 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:58.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.444+0000 7fe49cd95700 1 -- 172.21.15.3:0/4043040905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe498074ea0 con 0x7fe4981338c0 2023-10-03T10:17:58.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.445+0000 7fe486ffd700 1 -- 172.21.15.3:0/4043040905 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe48801b3b0 con 0x7fe4981338c0 2023-10-03T10:17:58.445 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:58.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.447+0000 7fe49cd95700 1 -- 172.21.15.3:0/4043040905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4981338c0 msgr2=0x7fe4981c8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:58.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.447+0000 7fe49cd95700 1 --2- 172.21.15.3:0/4043040905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4981338c0 0x7fe4981c8770 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fe48800c530 tx=0x7fe48800c840 comp rx=0 tx=0).stop 2023-10-03T10:17:58.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.447+0000 7fe49cd95700 1 -- 172.21.15.3:0/4043040905 shutdown_connections 2023-10-03T10:17:58.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.447+0000 7fe49cd95700 1 --2- 172.21.15.3:0/4043040905 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe498139150 0x7fe498118f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.447+0000 7fe49cd95700 1 --2- 172.21.15.3:0/4043040905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4981338c0 0x7fe4981c8770 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.447+0000 7fe49cd95700 1 --2- 172.21.15.3:0/4043040905 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe498136830 0x7fe4981c8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.447+0000 7fe49cd95700 1 -- 172.21.15.3:0/4043040905 >> 172.21.15.3:0/4043040905 conn(0x7fe4980a2aa0 msgr2=0x7fe49812f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:58.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.447+0000 7fe49cd95700 1 -- 172.21.15.3:0/4043040905 shutdown_connections 2023-10-03T10:17:58.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.447+0000 7fe49cd95700 1 -- 172.21.15.3:0/4043040905 wait complete. 2023-10-03T10:17:58.461 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:58.479 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:58.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.492+0000 7f64ecdc0700 1 -- 172.21.15.3:0/2185422951 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f64e81261d0 con 0x7f64e80a1510 2023-10-03T10:17:58.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.493+0000 7f64df7fe700 1 -- 172.21.15.3:0/2185422951 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f64d0021020 con 0x7f64e80a1510 2023-10-03T10:17:58.493 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:58.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.495+0000 7f64ecdc0700 1 -- 172.21.15.3:0/2185422951 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64e80a1510 msgr2=0x7f64e81c8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:58.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.495+0000 7f64ecdc0700 1 --2- 172.21.15.3:0/2185422951 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64e80a1510 0x7f64e81c8b60 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f64d000ebe0 tx=0x7f64d000eef0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.495+0000 7f64ecdc0700 1 -- 172.21.15.3:0/2185422951 shutdown_connections 2023-10-03T10:17:58.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.495+0000 7f64ecdc0700 1 --2- 172.21.15.3:0/2185422951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64e8139250 0x7f64e81227d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.495+0000 7f64ecdc0700 1 --2- 172.21.15.3:0/2185422951 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f64e80a0a30 0x7f64e81c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.495+0000 7f64ecdc0700 1 --2- 172.21.15.3:0/2185422951 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f64e80a1510 0x7f64e81c8b60 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.495+0000 7f64ecdc0700 1 -- 172.21.15.3:0/2185422951 >> 172.21.15.3:0/2185422951 conn(0x7f64e80a6550 msgr2=0x7f64e80a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:58.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.495+0000 7f64ecdc0700 1 -- 172.21.15.3:0/2185422951 shutdown_connections 2023-10-03T10:17:58.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.495+0000 7f64ecdc0700 1 -- 172.21.15.3:0/2185422951 wait complete. 2023-10-03T10:17:58.509 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:17:58.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.628+0000 7fc0e585b700 1 Processor -- start 2023-10-03T10:17:58.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.628+0000 7fc0e585b700 1 -- start start 2023-10-03T10:17:58.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.629+0000 7fc0e585b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0e00a0a30 0x7fc0e00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:58.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.629+0000 7fc0e585b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc0e0134170 0x7fc0e013bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:58.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.629+0000 7fc0e585b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0e009edd0 con 0x7fc0e00a1340 2023-10-03T10:17:58.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.629+0000 7fc0e585b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0e009e1f0 con 0x7fc0e0134170 2023-10-03T10:17:58.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.629+0000 7fc0e585b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e009dc50 con 0x7fc0e00a0a30 2023-10-03T10:17:58.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.629+0000 7fc0deffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0e00a0a30 0x7fc0e00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:58.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.629+0000 7fc0df7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc0e0134170 0x7fc0e013bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:58.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.629+0000 7fc0deffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0e00a0a30 0x7fc0e00a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47928/0 (socket says 172.21.15.3:47928) 2023-10-03T10:17:58.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.629+0000 7fc0deffd700 1 -- 172.21.15.3:0/3694982773 learned_addr learned my addr 172.21.15.3:0/3694982773 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:58.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.629+0000 7fc0df7fe700 1 -- 172.21.15.3:0/3694982773 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0e00a0a30 msgr2=0x7fc0e00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:58.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.629+0000 7fc0df7fe700 1 --2- 172.21.15.3:0/3694982773 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0e00a0a30 0x7fc0e00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.629+0000 7fc0df7fe700 1 -- 172.21.15.3:0/3694982773 >> v1:172.21.15.3:6789/0 conn(0x7fc0e00a1340 legacy=0x7fc0e0133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:58.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.629+0000 7fc0df7fe700 1 -- 172.21.15.3:0/3694982773 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e00a1fe0 con 0x7fc0e0134170 2023-10-03T10:17:58.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.629+0000 7fc0deffd700 1 --2- 172.21.15.3:0/3694982773 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0e00a0a30 0x7fc0e00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:58.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.630+0000 7fc0df7fe700 1 --2- 172.21.15.3:0/3694982773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0e0134170 0x7fc0e013bd30 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8009d90 tx=0x7fc0d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e25ae7dbf2fad78 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:58.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.630+0000 7fc0ddffb700 1 -- 172.21.15.3:0/3694982773 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d800e070 con 0x7fc0e0134170 2023-10-03T10:17:58.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.630+0000 7fc0ddffb700 1 -- 172.21.15.3:0/3694982773 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d800bb20 con 0x7fc0e0134170 2023-10-03T10:17:58.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.630+0000 7fc0ddffb700 1 -- 172.21.15.3:0/3694982773 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d8012430 con 0x7fc0e0134170 2023-10-03T10:17:58.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.630+0000 7fc0e585b700 1 -- 172.21.15.3:0/3694982773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0e0134170 msgr2=0x7fc0e013bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:58.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.630+0000 7fc0e585b700 1 --2- 172.21.15.3:0/3694982773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0e0134170 0x7fc0e013bd30 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8009d90 tx=0x7fc0d800b330 comp rx=0 tx=0).stop 2023-10-03T10:17:58.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.630+0000 7fc0e585b700 1 -- 172.21.15.3:0/3694982773 shutdown_connections 2023-10-03T10:17:58.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.630+0000 7fc0e585b700 1 --2- 172.21.15.3:0/3694982773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0e0134170 0x7fc0e013bd30 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.630+0000 7fc0e585b700 1 --2- 172.21.15.3:0/3694982773 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0e00a0a30 0x7fc0e00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.630+0000 7fc0e585b700 1 -- 172.21.15.3:0/3694982773 >> 172.21.15.3:0/3694982773 conn(0x7fc0e00a6550 msgr2=0x7fc0e00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:58.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.630+0000 7fc0e585b700 1 -- 172.21.15.3:0/3694982773 shutdown_connections 2023-10-03T10:17:58.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.631+0000 7fc0e585b700 1 -- 172.21.15.3:0/3694982773 wait complete. 2023-10-03T10:17:58.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.632+0000 7fc0e585b700 1 Processor -- start 2023-10-03T10:17:58.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.632+0000 7fc0e585b700 1 -- start start 2023-10-03T10:17:58.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.633+0000 7fc0e585b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0e00a0a30 0x7fc0e011e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:58.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.633+0000 7fc0e585b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0e00a1340 0x7fc0e011ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:58.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.633+0000 7fc0e585b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc0e0134170 0x7fc0e011f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:58.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.633+0000 7fc0e585b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e0122190 con 0x7fc0e00a0a30 2023-10-03T10:17:58.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.633+0000 7fc0e585b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e0121bf0 con 0x7fc0e0134170 2023-10-03T10:17:58.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.633+0000 7fc0e585b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e0120280 con 0x7fc0e00a1340 2023-10-03T10:17:58.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.633+0000 7fc0deffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0e00a0a30 0x7fc0e011e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:58.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.633+0000 7fc0deffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0e00a0a30 0x7fc0e011e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44806/0 (socket says 172.21.15.3:44806) 2023-10-03T10:17:58.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.633+0000 7fc0deffd700 1 -- 172.21.15.3:0/1713745740 learned_addr learned my addr 172.21.15.3:0/1713745740 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:58.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.633+0000 7fc0de7fc700 1 --2- 172.21.15.3:0/1713745740 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0e00a1340 0x7fc0e011ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:58.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.633+0000 7fc0df7fe700 1 --2- 172.21.15.3:0/1713745740 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc0e0134170 0x7fc0e011f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:58.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.633+0000 7fc0deffd700 1 -- 172.21.15.3:0/1713745740 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0e00a1340 msgr2=0x7fc0e011ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:58.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.633+0000 7fc0deffd700 1 --2- 172.21.15.3:0/1713745740 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0e00a1340 0x7fc0e011ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.633+0000 7fc0deffd700 1 -- 172.21.15.3:0/1713745740 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc0e0134170 msgr2=0x7fc0e011f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:58.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.633+0000 7fc0deffd700 1 --2- 172.21.15.3:0/1713745740 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc0e0134170 0x7fc0e011f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.633+0000 7fc0deffd700 1 -- 172.21.15.3:0/1713745740 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d0009550 con 0x7fc0e00a0a30 2023-10-03T10:17:58.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.633+0000 7fc0deffd700 1 --2- 172.21.15.3:0/1713745740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0e00a0a30 0x7fc0e011e700 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fc0d000ebe0 tx=0x7fc0d000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:58.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.634+0000 7fc0c7fff700 1 -- 172.21.15.3:0/1713745740 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d000c850 con 0x7fc0e00a0a30 2023-10-03T10:17:58.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.634+0000 7fc0c7fff700 1 -- 172.21.15.3:0/1713745740 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d000c9b0 con 0x7fc0e00a0a30 2023-10-03T10:17:58.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.634+0000 7fc0e585b700 1 -- 172.21.15.3:0/1713745740 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d8009a70 con 0x7fc0e00a0a30 2023-10-03T10:17:58.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.634+0000 7fc0c7fff700 1 -- 172.21.15.3:0/1713745740 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d000cd40 con 0x7fc0e00a0a30 2023-10-03T10:17:58.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.634+0000 7fc0e585b700 1 -- 172.21.15.3:0/1713745740 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e0120a30 con 0x7fc0e00a0a30 2023-10-03T10:17:58.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.634+0000 7fc0c7fff700 1 -- 172.21.15.3:0/1713745740 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc0d0013470 con 0x7fc0e00a0a30 2023-10-03T10:17:58.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.635+0000 7fc0c7fff700 1 -- 172.21.15.3:0/1713745740 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc0d003fc40 con 0x7fc0e00a0a30 2023-10-03T10:17:58.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.635+0000 7fc0e585b700 1 -- 172.21.15.3:0/1713745740 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0e0074ea0 con 0x7fc0e00a0a30 2023-10-03T10:17:58.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.640+0000 7fc0c7fff700 1 -- 172.21.15.3:0/1713745740 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc0d001b3b0 con 0x7fc0e00a0a30 2023-10-03T10:17:58.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.845+0000 7fc0e585b700 1 -- 172.21.15.3:0/1713745740 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc0e00a28e0 con 0x7fc0e00a0a30 2023-10-03T10:17:58.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.845+0000 7fc0c7fff700 1 -- 172.21.15.3:0/1713745740 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc0d0024350 con 0x7fc0e00a0a30 2023-10-03T10:17:58.846 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:17:58.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.847+0000 7fc0e585b700 1 -- 172.21.15.3:0/1713745740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0e00a0a30 msgr2=0x7fc0e011e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:58.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.847+0000 7fc0e585b700 1 --2- 172.21.15.3:0/1713745740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0e00a0a30 0x7fc0e011e700 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fc0d000ebe0 tx=0x7fc0d000eef0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.848+0000 7fc0e585b700 1 -- 172.21.15.3:0/1713745740 shutdown_connections 2023-10-03T10:17:58.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.848+0000 7fc0e585b700 1 --2- 172.21.15.3:0/1713745740 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc0e0134170 0x7fc0e011f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.848+0000 7fc0e585b700 1 --2- 172.21.15.3:0/1713745740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0e00a0a30 0x7fc0e011e700 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.848+0000 7fc0e585b700 1 --2- 172.21.15.3:0/1713745740 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0e00a1340 0x7fc0e011ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:58.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.848+0000 7fc0e585b700 1 -- 172.21.15.3:0/1713745740 >> 172.21.15.3:0/1713745740 conn(0x7fc0e00a6550 msgr2=0x7fc0e009c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:58.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.848+0000 7fc0e585b700 1 -- 172.21.15.3:0/1713745740 shutdown_connections 2023-10-03T10:17:58.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:58.848+0000 7fc0e585b700 1 -- 172.21.15.3:0/1713745740 wait complete. 2023-10-03T10:17:58.862 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:17:59.426 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:17:59.462 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:17:59.510 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:17:59.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.580+0000 7f76c94ec700 1 Processor -- start 2023-10-03T10:17:59.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.580+0000 7f76c94ec700 1 -- start start 2023-10-03T10:17:59.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.581+0000 7f76c94ec700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76c40a09c0 0x7f76c40a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:59.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.581+0000 7f76c94ec700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76c409d5c0 0x7f76c40a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:59.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.581+0000 7f76c94ec700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76c40a1810 con 0x7f76c4133920 2023-10-03T10:17:59.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.581+0000 7f76c94ec700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c409ea10 con 0x7f76c40a09c0 2023-10-03T10:17:59.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.581+0000 7f76c94ec700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76c409e210 con 0x7f76c409d5c0 2023-10-03T10:17:59.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.581+0000 7f76c37fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f76c4133920 0x7f76c4133d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45168/0 (socket says 172.21.15.3:45168) 2023-10-03T10:17:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.581+0000 7f76c37fe700 1 -- 172.21.15.3:0/3655095275 learned_addr learned my addr 172.21.15.3:0/3655095275 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:17:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.581+0000 7f76c2ffd700 1 --2- 172.21.15.3:0/3655095275 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76c40a09c0 0x7f76c40a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.581+0000 7f76c27fc700 1 --2- 172.21.15.3:0/3655095275 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76c409d5c0 0x7f76c40a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.581+0000 7f76c2ffd700 1 -- 172.21.15.3:0/3655095275 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76c409d5c0 msgr2=0x7f76c40a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.581+0000 7f76c1ffb700 1 -- 172.21.15.3:0/3655095275 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1525151548 0 0) 0x7f76c40a1810 con 0x7f76c4133920 2023-10-03T10:17:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.581+0000 7f76c2ffd700 1 --2- 172.21.15.3:0/3655095275 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76c409d5c0 0x7f76c40a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.581+0000 7f76c2ffd700 1 -- 172.21.15.3:0/3655095275 >> v1:172.21.15.3:6789/0 conn(0x7f76c4133920 legacy=0x7f76c4133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.581+0000 7f76c2ffd700 1 -- 172.21.15.3:0/3655095275 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c40a26f0 con 0x7f76c40a09c0 2023-10-03T10:17:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.581+0000 7f76c27fc700 1 --2- 172.21.15.3:0/3655095275 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76c409d5c0 0x7f76c40a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.582+0000 7f76c2ffd700 1 --2- 172.21.15.3:0/3655095275 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76c40a09c0 0x7f76c40a0d90 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009dc0 tx=0x7f76b400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82ac380de6796dfb server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.582+0000 7f76c1ffb700 1 -- 172.21.15.3:0/3655095275 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b400e070 con 0x7f76c40a09c0 2023-10-03T10:17:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.582+0000 7f76c1ffb700 1 -- 172.21.15.3:0/3655095275 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b400bb20 con 0x7f76c40a09c0 2023-10-03T10:17:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.582+0000 7f76c1ffb700 1 -- 172.21.15.3:0/3655095275 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b4012430 con 0x7f76c40a09c0 2023-10-03T10:17:59.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.582+0000 7f76c94ec700 1 -- 172.21.15.3:0/3655095275 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76c40a09c0 msgr2=0x7f76c40a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:59.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.582+0000 7f76c94ec700 1 --2- 172.21.15.3:0/3655095275 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76c40a09c0 0x7f76c40a0d90 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009dc0 tx=0x7f76b400b330 comp rx=0 tx=0).stop 2023-10-03T10:17:59.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.583+0000 7f76c94ec700 1 -- 172.21.15.3:0/3655095275 shutdown_connections 2023-10-03T10:17:59.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.583+0000 7f76c94ec700 1 --2- 172.21.15.3:0/3655095275 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76c40a09c0 0x7f76c40a0d90 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:59.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.583+0000 7f76c94ec700 1 --2- 172.21.15.3:0/3655095275 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76c409d5c0 0x7f76c40a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:59.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.583+0000 7f76c94ec700 1 -- 172.21.15.3:0/3655095275 >> 172.21.15.3:0/3655095275 conn(0x7f76c40a2fe0 msgr2=0x7f76c40a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:59.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.583+0000 7f76c94ec700 1 -- 172.21.15.3:0/3655095275 shutdown_connections 2023-10-03T10:17:59.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.583+0000 7f76c94ec700 1 -- 172.21.15.3:0/3655095275 wait complete. 2023-10-03T10:17:59.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.584+0000 7f76c94ec700 1 Processor -- start 2023-10-03T10:17:59.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.584+0000 7f76c94ec700 1 -- start start 2023-10-03T10:17:59.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.584+0000 7f76c94ec700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76c409d5c0 0x7f76c411f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:59.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.584+0000 7f76c94ec700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76c40a09c0 0x7f76c411f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:59.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.584+0000 7f76c94ec700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76c4133920 0x7f76c41c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:59.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.584+0000 7f76c94ec700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c411f030 con 0x7f76c40a09c0 2023-10-03T10:17:59.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.584+0000 7f76c94ec700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c4133690 con 0x7f76c409d5c0 2023-10-03T10:17:59.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.584+0000 7f76c94ec700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76c4133450 con 0x7f76c4133920 2023-10-03T10:17:59.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.585+0000 7f76c2ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76c409d5c0 0x7f76c411f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:59.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.585+0000 7f76c2ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76c409d5c0 0x7f76c411f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59334/0 (socket says 172.21.15.3:59334) 2023-10-03T10:17:59.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.585+0000 7f76c2ffd700 1 -- 172.21.15.3:0/1773012150 learned_addr learned my addr 172.21.15.3:0/1773012150 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:59.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.585+0000 7f76c27fc700 1 --2- 172.21.15.3:0/1773012150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76c40a09c0 0x7f76c411f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:59.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.585+0000 7f76c37fe700 1 --2- 172.21.15.3:0/1773012150 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76c4133920 0x7f76c41c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:59.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.585+0000 7f76c2ffd700 1 -- 172.21.15.3:0/1773012150 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76c4133920 msgr2=0x7f76c41c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:59.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.585+0000 7f76c2ffd700 1 --2- 172.21.15.3:0/1773012150 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76c4133920 0x7f76c41c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:59.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.585+0000 7f76c2ffd700 1 -- 172.21.15.3:0/1773012150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76c40a09c0 msgr2=0x7f76c411f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:59.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.585+0000 7f76c2ffd700 1 --2- 172.21.15.3:0/1773012150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76c40a09c0 0x7f76c411f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:59.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.585+0000 7f76c2ffd700 1 -- 172.21.15.3:0/1773012150 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b80094a0 con 0x7f76c409d5c0 2023-10-03T10:17:59.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.585+0000 7f76c2ffd700 1 --2- 172.21.15.3:0/1773012150 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76c409d5c0 0x7f76c411f470 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f76c40a19a0 tx=0x7f76b4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:59.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.585+0000 7f76ab7fe700 1 -- 172.21.15.3:0/1773012150 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b4012430 con 0x7f76c409d5c0 2023-10-03T10:17:59.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.586+0000 7f76ab7fe700 1 -- 172.21.15.3:0/1773012150 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b4020070 con 0x7f76c409d5c0 2023-10-03T10:17:59.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.586+0000 7f76c94ec700 1 -- 172.21.15.3:0/1773012150 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b4009a70 con 0x7f76c409d5c0 2023-10-03T10:17:59.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.586+0000 7f76ab7fe700 1 -- 172.21.15.3:0/1773012150 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b4004280 con 0x7f76c409d5c0 2023-10-03T10:17:59.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.586+0000 7f76c94ec700 1 -- 172.21.15.3:0/1773012150 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c40a5950 con 0x7f76c409d5c0 2023-10-03T10:17:59.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.586+0000 7f76ab7fe700 1 -- 172.21.15.3:0/1773012150 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f76b40043e0 con 0x7f76c409d5c0 2023-10-03T10:17:59.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.587+0000 7f76ab7fe700 1 -- 172.21.15.3:0/1773012150 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f76b4022070 con 0x7f76c409d5c0 2023-10-03T10:17:59.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.587+0000 7f76c94ec700 1 -- 172.21.15.3:0/1773012150 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76c4120c80 con 0x7f76c409d5c0 2023-10-03T10:17:59.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.592+0000 7f76ab7fe700 1 -- 172.21.15.3:0/1773012150 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f76b401b400 con 0x7f76c409d5c0 2023-10-03T10:17:59.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.618+0000 7faaa9c52700 1 Processor -- start 2023-10-03T10:17:59.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.618+0000 7faaa9c52700 1 -- start start 2023-10-03T10:17:59.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.618+0000 7faaa9c52700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7faaa40a0a30 0x7faaa40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:59.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.618+0000 7faaa9c52700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faaa4138d80 0x7faaa413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:59.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.618+0000 7faaa9c52700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa40a2a70 con 0x7faaa409d630 2023-10-03T10:17:59.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.619+0000 7faaa9c52700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaa40a01d0 con 0x7faaa40a0a30 2023-10-03T10:17:59.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.619+0000 7faaa9c52700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa409e2b0 con 0x7faaa4138d80 2023-10-03T10:17:59.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.619+0000 7faaa37fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7faaa40a0a30 0x7faaa40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:59.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.619+0000 7faaa37fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7faaa40a0a30 0x7faaa40a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44826/0 (socket says 172.21.15.3:44826) 2023-10-03T10:17:59.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.619+0000 7faaa37fe700 1 -- 172.21.15.3:0/1580486784 learned_addr learned my addr 172.21.15.3:0/1580486784 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:59.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.619+0000 7faaa3fff700 1 --2- 172.21.15.3:0/1580486784 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faaa4138d80 0x7faaa413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:59.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.619+0000 7faaa37fe700 1 -- 172.21.15.3:0/1580486784 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faaa4138d80 msgr2=0x7faaa413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:59.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.619+0000 7faaa37fe700 1 --2- 172.21.15.3:0/1580486784 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faaa4138d80 0x7faaa413b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:59.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.619+0000 7faaa37fe700 1 -- 172.21.15.3:0/1580486784 >> v1:172.21.15.3:6789/0 conn(0x7faaa409d630 legacy=0x7faaa40a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:59.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.619+0000 7faaa37fe700 1 -- 172.21.15.3:0/1580486784 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa409dc50 con 0x7faaa40a0a30 2023-10-03T10:17:59.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.619+0000 7faaa3fff700 1 --2- 172.21.15.3:0/1580486784 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faaa4138d80 0x7faaa413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:59.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.619+0000 7faaa37fe700 1 --2- 172.21.15.3:0/1580486784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa40a0a30 0x7faaa40a0e00 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7faa94009e50 tx=0x7faa9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a987f809953d4a55 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:59.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.619+0000 7faaa27fc700 1 -- 172.21.15.3:0/1580486784 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9400e070 con 0x7faaa40a0a30 2023-10-03T10:17:59.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.620+0000 7faaa27fc700 1 -- 172.21.15.3:0/1580486784 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa9400b910 con 0x7faaa40a0a30 2023-10-03T10:17:59.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.620+0000 7faaa27fc700 1 -- 172.21.15.3:0/1580486784 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9400bcc0 con 0x7faaa40a0a30 2023-10-03T10:17:59.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.620+0000 7faaa9c52700 1 -- 172.21.15.3:0/1580486784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa40a0a30 msgr2=0x7faaa40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:59.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.620+0000 7faaa9c52700 1 --2- 172.21.15.3:0/1580486784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa40a0a30 0x7faaa40a0e00 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7faa94009e50 tx=0x7faa9400b330 comp rx=0 tx=0).stop 2023-10-03T10:17:59.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.620+0000 7faaa9c52700 1 -- 172.21.15.3:0/1580486784 shutdown_connections 2023-10-03T10:17:59.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.620+0000 7faaa9c52700 1 --2- 172.21.15.3:0/1580486784 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faaa4138d80 0x7faaa413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.620+0000 7faaa9c52700 1 --2- 172.21.15.3:0/1580486784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa40a0a30 0x7faaa40a0e00 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.620+0000 7faaa9c52700 1 -- 172.21.15.3:0/1580486784 >> 172.21.15.3:0/1580486784 conn(0x7faaa40a3050 msgr2=0x7faaa40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.620+0000 7faaa9c52700 1 -- 172.21.15.3:0/1580486784 shutdown_connections 2023-10-03T10:17:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.620+0000 7faaa9c52700 1 -- 172.21.15.3:0/1580486784 wait complete. 2023-10-03T10:17:59.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.621+0000 7faaa9c52700 1 Processor -- start 2023-10-03T10:17:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.622+0000 7faaa9c52700 1 -- start start 2023-10-03T10:17:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.622+0000 7faaa9c52700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faaa409d630 0x7faaa4120700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.622+0000 7faaa9c52700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7faaa40a0a30 0x7faaa4120c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.622+0000 7faaa9c52700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa4138d80 0x7faaa4121180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.622+0000 7faaa9c52700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa4123110 con 0x7faaa4138d80 2023-10-03T10:17:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.622+0000 7faaa9c52700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa4122be0 con 0x7faaa409d630 2023-10-03T10:17:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.622+0000 7faaa9c52700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faaa4122310 con 0x7faaa40a0a30 2023-10-03T10:17:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.622+0000 7faaa37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faaa409d630 0x7faaa4120700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.622+0000 7faaa37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faaa409d630 0x7faaa4120700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59348/0 (socket says 172.21.15.3:59348) 2023-10-03T10:17:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.622+0000 7faaa37fe700 1 -- 172.21.15.3:0/1774772189 learned_addr learned my addr 172.21.15.3:0/1774772189 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.622+0000 7faaa2ffd700 1 --2- 172.21.15.3:0/1774772189 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7faaa40a0a30 0x7faaa4120c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.622+0000 7faaa3fff700 1 --2- 172.21.15.3:0/1774772189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa4138d80 0x7faaa4121180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.623+0000 7faaa37fe700 1 -- 172.21.15.3:0/1774772189 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7faaa40a0a30 msgr2=0x7faaa4120c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.623+0000 7faaa37fe700 1 --2- 172.21.15.3:0/1774772189 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7faaa40a0a30 0x7faaa4120c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.623+0000 7faaa37fe700 1 -- 172.21.15.3:0/1774772189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa4138d80 msgr2=0x7faaa4121180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.623+0000 7faaa37fe700 1 --2- 172.21.15.3:0/1774772189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa4138d80 0x7faaa4121180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.623+0000 7faaa37fe700 1 -- 172.21.15.3:0/1774772189 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa9c0094a0 con 0x7faaa409d630 2023-10-03T10:17:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.623+0000 7faaa37fe700 1 --2- 172.21.15.3:0/1774772189 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faaa409d630 0x7faaa4120700 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7faaa40a2c00 tx=0x7faa94004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.623+0000 7faa8bfff700 1 -- 172.21.15.3:0/1774772189 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa94004800 con 0x7faaa409d630 2023-10-03T10:17:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.623+0000 7faa8bfff700 1 -- 172.21.15.3:0/1774772189 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa940092e0 con 0x7faaa409d630 2023-10-03T10:17:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.623+0000 7faaa9c52700 1 -- 172.21.15.3:0/1774772189 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa94009b30 con 0x7faaa409d630 2023-10-03T10:17:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.623+0000 7faa8bfff700 1 -- 172.21.15.3:0/1774772189 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9401c660 con 0x7faaa409d630 2023-10-03T10:17:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.623+0000 7faaa9c52700 1 -- 172.21.15.3:0/1774772189 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa40a4ec0 con 0x7faaa409d630 2023-10-03T10:17:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.624+0000 7faa8bfff700 1 -- 172.21.15.3:0/1774772189 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faa9401c7c0 con 0x7faaa409d630 2023-10-03T10:17:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.624+0000 7faa8bfff700 1 -- 172.21.15.3:0/1774772189 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7faa94041280 con 0x7faaa409d630 2023-10-03T10:17:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.624+0000 7faaa9c52700 1 -- 172.21.15.3:0/1774772189 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa40a0410 con 0x7faaa409d630 2023-10-03T10:17:59.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.629+0000 7faa8bfff700 1 -- 172.21.15.3:0/1774772189 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faa940257d0 con 0x7faaa409d630 2023-10-03T10:17:59.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.669+0000 7fc473eed700 1 Processor -- start 2023-10-03T10:17:59.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.669+0000 7fc473eed700 1 -- start start 2023-10-03T10:17:59.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.670+0000 7fc473eed700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc46c133420 0x7fc46c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:59.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.670+0000 7fc473eed700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc46c133e30 0x7fc46c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:59.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.670+0000 7fc473eed700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc46c06b990 con 0x7fc46c134260 2023-10-03T10:17:59.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.670+0000 7fc473eed700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc46c0a0590 con 0x7fc46c133e30 2023-10-03T10:17:59.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.670+0000 7fc473eed700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c09fb80 con 0x7fc46c133420 2023-10-03T10:17:59.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.670+0000 7fc471c89700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc46c133420 0x7fc46c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:59.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.670+0000 7fc471c89700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc46c133420 0x7fc46c1337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47970/0 (socket says 172.21.15.3:47970) 2023-10-03T10:17:59.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.670+0000 7fc471c89700 1 -- 172.21.15.3:0/2687449692 learned_addr learned my addr 172.21.15.3:0/2687449692 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:59.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.670+0000 7fc471488700 1 --2- 172.21.15.3:0/2687449692 >> v2:172.21.15.3:3300/0 conn(0x7fc46c133e30 0x7fc46c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:59.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.670+0000 7fc471c89700 1 -- 172.21.15.3:0/2687449692 >> v2:172.21.15.3:3300/0 conn(0x7fc46c133e30 msgr2=0x7fc46c12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:17:59.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.670+0000 7fc471c89700 1 --2- 172.21.15.3:0/2687449692 >> v2:172.21.15.3:3300/0 conn(0x7fc46c133e30 0x7fc46c12ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:59.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.670+0000 7fc471c89700 1 -- 172.21.15.3:0/2687449692 >> v1:172.21.15.3:6789/0 conn(0x7fc46c134260 legacy=0x7fc46c1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:59.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.670+0000 7fc471c89700 1 -- 172.21.15.3:0/2687449692 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46c0a0030 con 0x7fc46c133420 2023-10-03T10:17:59.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.670+0000 7fc471488700 1 --2- 172.21.15.3:0/2687449692 >> v2:172.21.15.3:3300/0 conn(0x7fc46c133e30 0x7fc46c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:17:59.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.671+0000 7fc471c89700 1 --2- 172.21.15.3:0/2687449692 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc46c133420 0x7fc46c1337f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc4680098f0 tx=0x7fc468009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e18ba58428141da5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:59.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.671+0000 7fc470c87700 1 -- 172.21.15.3:0/2687449692 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468011830 con 0x7fc46c133420 2023-10-03T10:17:59.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.671+0000 7fc470c87700 1 -- 172.21.15.3:0/2687449692 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc468011990 con 0x7fc46c133420 2023-10-03T10:17:59.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.671+0000 7fc470c87700 1 -- 172.21.15.3:0/2687449692 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468011d20 con 0x7fc46c133420 2023-10-03T10:17:59.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.672+0000 7fc473eed700 1 -- 172.21.15.3:0/2687449692 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc46c133420 msgr2=0x7fc46c1337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:59.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.672+0000 7fc473eed700 1 --2- 172.21.15.3:0/2687449692 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc46c133420 0x7fc46c1337f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc4680098f0 tx=0x7fc468009c00 comp rx=0 tx=0).stop 2023-10-03T10:17:59.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.672+0000 7fc473eed700 1 -- 172.21.15.3:0/2687449692 shutdown_connections 2023-10-03T10:17:59.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.672+0000 7fc473eed700 1 --2- 172.21.15.3:0/2687449692 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc46c133420 0x7fc46c1337f0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:59.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.672+0000 7fc473eed700 1 --2- 172.21.15.3:0/2687449692 >> v2:172.21.15.3:3300/0 conn(0x7fc46c133e30 0x7fc46c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:59.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.672+0000 7fc473eed700 1 -- 172.21.15.3:0/2687449692 >> 172.21.15.3:0/2687449692 conn(0x7fc46c0a4210 msgr2=0x7fc46c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:59.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.672+0000 7fc473eed700 1 -- 172.21.15.3:0/2687449692 shutdown_connections 2023-10-03T10:17:59.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.672+0000 7fc473eed700 1 -- 172.21.15.3:0/2687449692 wait complete. 2023-10-03T10:17:59.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.674+0000 7fc473eed700 1 Processor -- start 2023-10-03T10:17:59.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.674+0000 7fc473eed700 1 -- start start 2023-10-03T10:17:59.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.674+0000 7fc473eed700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc46c133420 0x7fc46c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:59.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.674+0000 7fc473eed700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc46c133e30 0x7fc46c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:59.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.674+0000 7fc473eed700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc46c134260 0x7fc46c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:17:59.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.674+0000 7fc473eed700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c09dcd0 con 0x7fc46c133420 2023-10-03T10:17:59.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.674+0000 7fc473eed700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c11fe90 con 0x7fc46c133e30 2023-10-03T10:17:59.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.674+0000 7fc473eed700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c09df30 con 0x7fc46c134260 2023-10-03T10:17:59.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.674+0000 7fc471c89700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc46c133420 0x7fc46c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:59.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.675+0000 7fc471c89700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc46c133420 0x7fc46c09c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44870/0 (socket says 172.21.15.3:44870) 2023-10-03T10:17:59.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.675+0000 7fc471c89700 1 -- 172.21.15.3:0/203891262 learned_addr learned my addr 172.21.15.3:0/203891262 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:17:59.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.675+0000 7fc471488700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc46c133e30 0x7fc46c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:59.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.675+0000 7fc47248a700 1 --2- 172.21.15.3:0/203891262 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc46c134260 0x7fc46c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:17:59.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.675+0000 7fc471488700 1 -- 172.21.15.3:0/203891262 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc46c134260 msgr2=0x7fc46c09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:59.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.675+0000 7fc471488700 1 --2- 172.21.15.3:0/203891262 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc46c134260 0x7fc46c09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:59.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.675+0000 7fc471488700 1 -- 172.21.15.3:0/203891262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc46c133420 msgr2=0x7fc46c09c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:59.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.675+0000 7fc471488700 1 --2- 172.21.15.3:0/203891262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc46c133420 0x7fc46c09c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:59.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.675+0000 7fc471488700 1 -- 172.21.15.3:0/203891262 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45c0094a0 con 0x7fc46c133e30 2023-10-03T10:17:59.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.675+0000 7fc471488700 1 --2- 172.21.15.3:0/203891262 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc46c133e30 0x7fc46c09c9c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00e9b0 tx=0x7fc45c00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:17:59.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.675+0000 7fc4627fc700 1 -- 172.21.15.3:0/203891262 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c00ee50 con 0x7fc46c133e30 2023-10-03T10:17:59.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.675+0000 7fc4627fc700 1 -- 172.21.15.3:0/203891262 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45c00c850 con 0x7fc46c133e30 2023-10-03T10:17:59.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.675+0000 7fc473eed700 1 -- 172.21.15.3:0/203891262 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc468009550 con 0x7fc46c133e30 2023-10-03T10:17:59.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.675+0000 7fc4627fc700 1 -- 172.21.15.3:0/203891262 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c00cbe0 con 0x7fc46c133e30 2023-10-03T10:17:59.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.676+0000 7fc473eed700 1 -- 172.21.15.3:0/203891262 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc46c11fb60 con 0x7fc46c133e30 2023-10-03T10:17:59.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.676+0000 7fc4627fc700 1 -- 172.21.15.3:0/203891262 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc45c00cd40 con 0x7fc46c133e30 2023-10-03T10:17:59.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.677+0000 7fc4627fc700 1 -- 172.21.15.3:0/203891262 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc45c013990 con 0x7fc46c133e30 2023-10-03T10:17:59.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.677+0000 7fc473eed700 1 -- 172.21.15.3:0/203891262 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc46c131990 con 0x7fc46c133e30 2023-10-03T10:17:59.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.682+0000 7fc4627fc700 1 -- 172.21.15.3:0/203891262 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc45c0102b0 con 0x7fc46c133e30 2023-10-03T10:17:59.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.818+0000 7f76c94ec700 1 -- 172.21.15.3:0/1773012150 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f76c40747f0 con 0x7f76c409d5c0 2023-10-03T10:17:59.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.818+0000 7f76ab7fe700 1 -- 172.21.15.3:0/1773012150 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f76b401ab70 con 0x7f76c409d5c0 2023-10-03T10:17:59.819 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:17:59.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.821+0000 7f76c94ec700 1 -- 172.21.15.3:0/1773012150 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76c409d5c0 msgr2=0x7f76c411f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:59.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.821+0000 7f76c94ec700 1 --2- 172.21.15.3:0/1773012150 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76c409d5c0 0x7f76c411f470 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f76c40a19a0 tx=0x7f76b4004ad0 comp rx=0 tx=0).stop 2023-10-03T10:17:59.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.821+0000 7f76c94ec700 1 -- 172.21.15.3:0/1773012150 shutdown_connections 2023-10-03T10:17:59.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.821+0000 7f76c94ec700 1 --2- 172.21.15.3:0/1773012150 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76c4133920 0x7f76c41c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:59.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.821+0000 7f76c94ec700 1 --2- 172.21.15.3:0/1773012150 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76c409d5c0 0x7f76c411f470 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:59.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.821+0000 7f76c94ec700 1 --2- 172.21.15.3:0/1773012150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76c40a09c0 0x7f76c411f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:59.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.821+0000 7f76c94ec700 1 -- 172.21.15.3:0/1773012150 >> 172.21.15.3:0/1773012150 conn(0x7f76c40a2fe0 msgr2=0x7f76c41bbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:59.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.821+0000 7f76c94ec700 1 -- 172.21.15.3:0/1773012150 shutdown_connections 2023-10-03T10:17:59.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.821+0000 7f76c94ec700 1 -- 172.21.15.3:0/1773012150 wait complete. 2023-10-03T10:17:59.835 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:17:59.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.842+0000 7faaa9c52700 1 -- 172.21.15.3:0/1774772189 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7faaa40747f0 con 0x7faaa409d630 2023-10-03T10:17:59.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.842+0000 7faa8bfff700 1 -- 172.21.15.3:0/1774772189 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faa94033070 con 0x7faaa409d630 2023-10-03T10:17:59.843 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:17:59.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.845+0000 7faaa9c52700 1 -- 172.21.15.3:0/1774772189 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faaa409d630 msgr2=0x7faaa4120700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:59.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.845+0000 7faaa9c52700 1 --2- 172.21.15.3:0/1774772189 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faaa409d630 0x7faaa4120700 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7faaa40a2c00 tx=0x7faa94004ad0 comp rx=0 tx=0).stop 2023-10-03T10:17:59.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.845+0000 7faaa9c52700 1 -- 172.21.15.3:0/1774772189 shutdown_connections 2023-10-03T10:17:59.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.845+0000 7faaa9c52700 1 --2- 172.21.15.3:0/1774772189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa4138d80 0x7faaa4121180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:59.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.845+0000 7faaa9c52700 1 --2- 172.21.15.3:0/1774772189 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faaa409d630 0x7faaa4120700 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:59.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.845+0000 7faaa9c52700 1 --2- 172.21.15.3:0/1774772189 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7faaa40a0a30 0x7faaa4120c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:59.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.845+0000 7faaa9c52700 1 -- 172.21.15.3:0/1774772189 >> 172.21.15.3:0/1774772189 conn(0x7faaa40a3050 msgr2=0x7faaa409ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:59.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.845+0000 7faaa9c52700 1 -- 172.21.15.3:0/1774772189 shutdown_connections 2023-10-03T10:17:59.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.845+0000 7faaa9c52700 1 -- 172.21.15.3:0/1774772189 wait complete. 2023-10-03T10:17:59.859 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:17:59.862 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:17:59.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.887+0000 7fc473eed700 1 -- 172.21.15.3:0/203891262 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc46c074ea0 con 0x7fc46c133e30 2023-10-03T10:17:59.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.887+0000 7fc4627fc700 1 -- 172.21.15.3:0/203891262 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc45c017020 con 0x7fc46c133e30 2023-10-03T10:17:59.888 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:17:59.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.889+0000 7fc473eed700 1 -- 172.21.15.3:0/203891262 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc46c133e30 msgr2=0x7fc46c09c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:17:59.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.889+0000 7fc473eed700 1 --2- 172.21.15.3:0/203891262 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc46c133e30 0x7fc46c09c9c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00e9b0 tx=0x7fc45c00ecc0 comp rx=0 tx=0).stop 2023-10-03T10:17:59.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.889+0000 7fc473eed700 1 -- 172.21.15.3:0/203891262 shutdown_connections 2023-10-03T10:17:59.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.889+0000 7fc473eed700 1 --2- 172.21.15.3:0/203891262 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc46c134260 0x7fc46c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:59.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.889+0000 7fc473eed700 1 --2- 172.21.15.3:0/203891262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc46c133420 0x7fc46c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:59.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.889+0000 7fc473eed700 1 --2- 172.21.15.3:0/203891262 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc46c133e30 0x7fc46c09c9c0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:17:59.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.889+0000 7fc473eed700 1 -- 172.21.15.3:0/203891262 >> 172.21.15.3:0/203891262 conn(0x7fc46c0a4210 msgr2=0x7fc46c0a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:17:59.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.889+0000 7fc473eed700 1 -- 172.21.15.3:0/203891262 shutdown_connections 2023-10-03T10:17:59.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:17:59.889+0000 7fc473eed700 1 -- 172.21.15.3:0/203891262 wait complete. 2023-10-03T10:17:59.905 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:00.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.023+0000 7f09bbb1f700 1 Processor -- start 2023-10-03T10:18:00.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.023+0000 7f09bbb1f700 1 -- start start 2023-10-03T10:18:00.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.024+0000 7f09bbb1f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09b409d6f0 0x7f09b409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:00.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.024+0000 7f09bbb1f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f09b409e000 0x7f09b4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:00.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.024+0000 7f09bbb1f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b409f130 con 0x7f09b409e410 2023-10-03T10:18:00.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.024+0000 7f09bbb1f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09b409f520 con 0x7f09b409e000 2023-10-03T10:18:00.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.024+0000 7f09bbb1f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b409e930 con 0x7f09b409d6f0 2023-10-03T10:18:00.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.024+0000 7f09b98bb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09b409d6f0 0x7f09b409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:00.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.024+0000 7f09b90ba700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f09b409e000 0x7f09b4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:00.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.024+0000 7f09b98bb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09b409d6f0 0x7f09b409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59360/0 (socket says 172.21.15.3:59360) 2023-10-03T10:18:00.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.024+0000 7f09b90ba700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f09b409e000 0x7f09b4130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44878/0 (socket says 172.21.15.3:44878) 2023-10-03T10:18:00.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.024+0000 7f09b98bb700 1 -- 172.21.15.3:0/3843347398 learned_addr learned my addr 172.21.15.3:0/3843347398 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:00.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.024+0000 7f09b98bb700 1 -- 172.21.15.3:0/3843347398 >> v2:172.21.15.3:3300/0 conn(0x7f09b409e000 msgr2=0x7f09b4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:00.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.024+0000 7f09b98bb700 1 --2- 172.21.15.3:0/3843347398 >> v2:172.21.15.3:3300/0 conn(0x7f09b409e000 0x7f09b4130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:00.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.024+0000 7f09b98bb700 1 -- 172.21.15.3:0/3843347398 >> v1:172.21.15.3:6789/0 conn(0x7f09b409e410 legacy=0x7f09b4130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:00.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.024+0000 7f09b98bb700 1 -- 172.21.15.3:0/3843347398 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b409b2d0 con 0x7f09b409d6f0 2023-10-03T10:18:00.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.024+0000 7f09b88b9700 1 -- 172.21.15.3:0/3843347398 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2697526435 0 0) 0x7f09b409f130 con 0x7f09b409e410 2023-10-03T10:18:00.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.024+0000 7f09b90ba700 1 --2- 172.21.15.3:0/3843347398 >> v2:172.21.15.3:3300/0 conn(0x7f09b409e000 0x7f09b4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:00.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.025+0000 7f09b98bb700 1 --2- 172.21.15.3:0/3843347398 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09b409d6f0 0x7f09b409dac0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f09b00098f0 tx=0x7f09b0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ad4bf2aff5396c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:00.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.025+0000 7f09b88b9700 1 -- 172.21.15.3:0/3843347398 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b0011830 con 0x7f09b409d6f0 2023-10-03T10:18:00.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.025+0000 7f09b88b9700 1 -- 172.21.15.3:0/3843347398 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b0011990 con 0x7f09b409d6f0 2023-10-03T10:18:00.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.025+0000 7f09b88b9700 1 -- 172.21.15.3:0/3843347398 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b0011d20 con 0x7f09b409d6f0 2023-10-03T10:18:00.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.025+0000 7f09bbb1f700 1 -- 172.21.15.3:0/3843347398 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09b409d6f0 msgr2=0x7f09b409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:00.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.025+0000 7f09bbb1f700 1 --2- 172.21.15.3:0/3843347398 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09b409d6f0 0x7f09b409dac0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f09b00098f0 tx=0x7f09b0009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:00.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.025+0000 7f09bbb1f700 1 -- 172.21.15.3:0/3843347398 shutdown_connections 2023-10-03T10:18:00.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.025+0000 7f09bbb1f700 1 --2- 172.21.15.3:0/3843347398 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09b409d6f0 0x7f09b409dac0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:00.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.025+0000 7f09bbb1f700 1 --2- 172.21.15.3:0/3843347398 >> v2:172.21.15.3:3300/0 conn(0x7f09b409e000 0x7f09b4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:00.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.025+0000 7f09bbb1f700 1 -- 172.21.15.3:0/3843347398 >> 172.21.15.3:0/3843347398 conn(0x7f09b4066f50 msgr2=0x7f09b409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:00.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.026+0000 7f09bbb1f700 1 -- 172.21.15.3:0/3843347398 shutdown_connections 2023-10-03T10:18:00.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.026+0000 7f09bbb1f700 1 -- 172.21.15.3:0/3843347398 wait complete. 2023-10-03T10:18:00.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.027+0000 7f09bbb1f700 1 Processor -- start 2023-10-03T10:18:00.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.027+0000 7f09bbb1f700 1 -- start start 2023-10-03T10:18:00.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.027+0000 7f09bbb1f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f09b409d6f0 0x7f09b411ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:00.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.027+0000 7f09bbb1f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09b409e000 0x7f09b41204f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:00.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.027+0000 7f09bbb1f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09b409e410 0x7f09b4120a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:00.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.027+0000 7f09bbb1f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b4130370 con 0x7f09b409e000 2023-10-03T10:18:00.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.027+0000 7f09bbb1f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b4123530 con 0x7f09b409e410 2023-10-03T10:18:00.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.027+0000 7f09bbb1f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f09b41237f0 con 0x7f09b409d6f0 2023-10-03T10:18:00.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.027+0000 7f09b98bb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f09b409d6f0 0x7f09b411ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:00.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.027+0000 7f09b90ba700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09b409e000 0x7f09b41204f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:00.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.027+0000 7f09b98bb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f09b409d6f0 0x7f09b411ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47988/0 (socket says 172.21.15.3:47988) 2023-10-03T10:18:00.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.027+0000 7f09b90ba700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09b409e000 0x7f09b41204f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44892/0 (socket says 172.21.15.3:44892) 2023-10-03T10:18:00.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.027+0000 7f09b98bb700 1 -- 172.21.15.3:0/3139668252 learned_addr learned my addr 172.21.15.3:0/3139668252 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:00.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.027+0000 7f09ba0bc700 1 --2- 172.21.15.3:0/3139668252 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09b409e410 0x7f09b4120a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:00.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.028+0000 7f09b90ba700 1 -- 172.21.15.3:0/3139668252 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f09b409d6f0 msgr2=0x7f09b411ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:00.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.028+0000 7f09b90ba700 1 --2- 172.21.15.3:0/3139668252 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f09b409d6f0 0x7f09b411ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:00.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.028+0000 7f09b90ba700 1 -- 172.21.15.3:0/3139668252 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09b409e410 msgr2=0x7f09b4120a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:00.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.028+0000 7f09b90ba700 1 --2- 172.21.15.3:0/3139668252 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09b409e410 0x7f09b4120a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:00.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.028+0000 7f09b90ba700 1 -- 172.21.15.3:0/3139668252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a80094a0 con 0x7f09b409e000 2023-10-03T10:18:00.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.028+0000 7f09b90ba700 1 --2- 172.21.15.3:0/3139668252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09b409e000 0x7f09b41204f0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f09a800e980 tx=0x7f09a800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:00.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.028+0000 7f09a67fc700 1 -- 172.21.15.3:0/3139668252 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a800c690 con 0x7f09b409e000 2023-10-03T10:18:00.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.028+0000 7f09a67fc700 1 -- 172.21.15.3:0/3139668252 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a800c7f0 con 0x7f09b409e000 2023-10-03T10:18:00.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.028+0000 7f09bbb1f700 1 -- 172.21.15.3:0/3139668252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b0009550 con 0x7f09b409e000 2023-10-03T10:18:00.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.028+0000 7f09a67fc700 1 -- 172.21.15.3:0/3139668252 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a800cb80 con 0x7f09b409e000 2023-10-03T10:18:00.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.028+0000 7f09bbb1f700 1 -- 172.21.15.3:0/3139668252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b4123260 con 0x7f09b409e000 2023-10-03T10:18:00.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.029+0000 7f09a67fc700 1 -- 172.21.15.3:0/3139668252 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f09a800cce0 con 0x7f09b409e000 2023-10-03T10:18:00.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.029+0000 7f09a67fc700 1 -- 172.21.15.3:0/3139668252 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f09a803fb50 con 0x7f09b409e000 2023-10-03T10:18:00.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.029+0000 7f09bbb1f700 1 -- 172.21.15.3:0/3139668252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b4074ea0 con 0x7f09b409e000 2023-10-03T10:18:00.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.034+0000 7f09a67fc700 1 -- 172.21.15.3:0/3139668252 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f09a8017020 con 0x7f09b409e000 2023-10-03T10:18:00.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.239+0000 7f09bbb1f700 1 -- 172.21.15.3:0/3139668252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f09b41223c0 con 0x7f09b409e000 2023-10-03T10:18:00.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.239+0000 7f09a67fc700 1 -- 172.21.15.3:0/3139668252 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f09a801b3b0 con 0x7f09b409e000 2023-10-03T10:18:00.240 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:00.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.242+0000 7f09bbb1f700 1 -- 172.21.15.3:0/3139668252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09b409e000 msgr2=0x7f09b41204f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:00.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.242+0000 7f09bbb1f700 1 --2- 172.21.15.3:0/3139668252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09b409e000 0x7f09b41204f0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f09a800e980 tx=0x7f09a800ec90 comp rx=0 tx=0).stop 2023-10-03T10:18:00.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.242+0000 7f09bbb1f700 1 -- 172.21.15.3:0/3139668252 shutdown_connections 2023-10-03T10:18:00.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.242+0000 7f09bbb1f700 1 --2- 172.21.15.3:0/3139668252 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09b409e410 0x7f09b4120a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:00.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.242+0000 7f09bbb1f700 1 --2- 172.21.15.3:0/3139668252 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f09b409d6f0 0x7f09b411ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:00.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.242+0000 7f09bbb1f700 1 --2- 172.21.15.3:0/3139668252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09b409e000 0x7f09b41204f0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:00.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.242+0000 7f09bbb1f700 1 -- 172.21.15.3:0/3139668252 >> 172.21.15.3:0/3139668252 conn(0x7f09b4066f50 msgr2=0x7f09b4071110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:00.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.242+0000 7f09bbb1f700 1 -- 172.21.15.3:0/3139668252 shutdown_connections 2023-10-03T10:18:00.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.242+0000 7f09bbb1f700 1 -- 172.21.15.3:0/3139668252 wait complete. 2023-10-03T10:18:00.256 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:00.836 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:00.861 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:00.906 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:00.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.990+0000 7ff453513700 1 Processor -- start 2023-10-03T10:18:00.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.990+0000 7ff453513700 1 -- start start 2023-10-03T10:18:00.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.990+0000 7ff453513700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff44c0a0a30 0x7ff44c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.990+0000 7ff453513700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff44c138d80 0x7ff44c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.990+0000 7ff453513700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff44c0a2a70 con 0x7ff44c09d630 2023-10-03T10:18:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.991+0000 7ff453513700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff44c0a01d0 con 0x7ff44c0a0a30 2023-10-03T10:18:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.991+0000 7ff453513700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c09e2b0 con 0x7ff44c138d80 2023-10-03T10:18:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.991+0000 7ff4512af700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff44c0a0a30 0x7ff44c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.991+0000 7ff451ab0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff44c138d80 0x7ff44c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.991+0000 7ff4512af700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff44c0a0a30 0x7ff44c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44894/0 (socket says 172.21.15.3:44894) 2023-10-03T10:18:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.991+0000 7ff4512af700 1 -- 172.21.15.3:0/1468326177 learned_addr learned my addr 172.21.15.3:0/1468326177 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.991+0000 7ff4512af700 1 -- 172.21.15.3:0/1468326177 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff44c138d80 msgr2=0x7ff44c13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.991+0000 7ff4512af700 1 --2- 172.21.15.3:0/1468326177 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff44c138d80 0x7ff44c13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.991+0000 7ff4512af700 1 -- 172.21.15.3:0/1468326177 >> v1:172.21.15.3:6789/0 conn(0x7ff44c09d630 legacy=0x7ff44c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:00.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.991+0000 7ff4512af700 1 -- 172.21.15.3:0/1468326177 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44c09dc50 con 0x7ff44c0a0a30 2023-10-03T10:18:00.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.991+0000 7ff4512af700 1 --2- 172.21.15.3:0/1468326177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff44c0a0a30 0x7ff44c0a0e00 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7ff4480098f0 tx=0x7ff448009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d633d49470ec0bb server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:00.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.991+0000 7ff443fff700 1 -- 172.21.15.3:0/1468326177 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff448011830 con 0x7ff44c0a0a30 2023-10-03T10:18:00.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.992+0000 7ff443fff700 1 -- 172.21.15.3:0/1468326177 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff448011990 con 0x7ff44c0a0a30 2023-10-03T10:18:00.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.992+0000 7ff443fff700 1 -- 172.21.15.3:0/1468326177 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff448011d20 con 0x7ff44c0a0a30 2023-10-03T10:18:00.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.992+0000 7ff453513700 1 -- 172.21.15.3:0/1468326177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff44c0a0a30 msgr2=0x7ff44c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:00.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.992+0000 7ff453513700 1 --2- 172.21.15.3:0/1468326177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff44c0a0a30 0x7ff44c0a0e00 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7ff4480098f0 tx=0x7ff448009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:00.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.992+0000 7ff453513700 1 -- 172.21.15.3:0/1468326177 shutdown_connections 2023-10-03T10:18:00.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.992+0000 7ff453513700 1 --2- 172.21.15.3:0/1468326177 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff44c138d80 0x7ff44c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:00.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.992+0000 7ff453513700 1 --2- 172.21.15.3:0/1468326177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff44c0a0a30 0x7ff44c0a0e00 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:00.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.992+0000 7ff453513700 1 -- 172.21.15.3:0/1468326177 >> 172.21.15.3:0/1468326177 conn(0x7ff44c0a3050 msgr2=0x7ff44c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:00.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.992+0000 7ff453513700 1 -- 172.21.15.3:0/1468326177 shutdown_connections 2023-10-03T10:18:00.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.992+0000 7ff453513700 1 -- 172.21.15.3:0/1468326177 wait complete. 2023-10-03T10:18:00.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.993+0000 7ff453513700 1 Processor -- start 2023-10-03T10:18:00.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.993+0000 7ff453513700 1 -- start start 2023-10-03T10:18:00.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.994+0000 7ff453513700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff44c09d630 0x7ff44c120830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:00.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.994+0000 7ff453513700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff44c0a0a30 0x7ff44c120d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:00.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.994+0000 7ff453513700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff44c138d80 0x7ff44c1212b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:00.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.994+0000 7ff453513700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c0a4ea0 con 0x7ff44c138d80 2023-10-03T10:18:00.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.994+0000 7ff453513700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c0a0410 con 0x7ff44c0a0a30 2023-10-03T10:18:00.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.994+0000 7ff453513700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c123ae0 con 0x7ff44c09d630 2023-10-03T10:18:00.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.994+0000 7ff451ab0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff44c138d80 0x7ff44c1212b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:00.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.994+0000 7ff451ab0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff44c138d80 0x7ff44c1212b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44908/0 (socket says 172.21.15.3:44908) 2023-10-03T10:18:00.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.994+0000 7ff4512af700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff44c09d630 0x7ff44c120830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:00.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.994+0000 7ff451ab0700 1 -- 172.21.15.3:0/1654097425 learned_addr learned my addr 172.21.15.3:0/1654097425 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:00.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.994+0000 7ff450aae700 1 --2- 172.21.15.3:0/1654097425 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff44c0a0a30 0x7ff44c120d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:00.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.994+0000 7ff451ab0700 1 -- 172.21.15.3:0/1654097425 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff44c09d630 msgr2=0x7ff44c120830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:00.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.994+0000 7ff451ab0700 1 --2- 172.21.15.3:0/1654097425 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff44c09d630 0x7ff44c120830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:00.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.994+0000 7ff451ab0700 1 -- 172.21.15.3:0/1654097425 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff44c0a0a30 msgr2=0x7ff44c120d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:00.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.994+0000 7ff451ab0700 1 --2- 172.21.15.3:0/1654097425 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff44c0a0a30 0x7ff44c120d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:00.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.995+0000 7ff451ab0700 1 -- 172.21.15.3:0/1654097425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff448009550 con 0x7ff44c138d80 2023-10-03T10:18:00.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.995+0000 7ff4512af700 1 --2- 172.21.15.3:0/1654097425 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff44c09d630 0x7ff44c120830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:00.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.995+0000 7ff450aae700 1 --2- 172.21.15.3:0/1654097425 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff44c0a0a30 0x7ff44c120d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:00.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.995+0000 7ff451ab0700 1 --2- 172.21.15.3:0/1654097425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff44c138d80 0x7ff44c1212b0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7ff44400efc0 tx=0x7ff44400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:00.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.995+0000 7ff441ffb700 1 -- 172.21.15.3:0/1654097425 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44400cc90 con 0x7ff44c138d80 2023-10-03T10:18:00.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.995+0000 7ff441ffb700 1 -- 172.21.15.3:0/1654097425 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44400f040 con 0x7ff44c138d80 2023-10-03T10:18:00.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.995+0000 7ff453513700 1 -- 172.21.15.3:0/1654097425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff44c124070 con 0x7ff44c138d80 2023-10-03T10:18:00.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.995+0000 7ff441ffb700 1 -- 172.21.15.3:0/1654097425 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4440146c0 con 0x7ff44c138d80 2023-10-03T10:18:00.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.995+0000 7ff453513700 1 -- 172.21.15.3:0/1654097425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff44c124ae0 con 0x7ff44c138d80 2023-10-03T10:18:00.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.996+0000 7ff441ffb700 1 -- 172.21.15.3:0/1654097425 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff444014820 con 0x7ff44c138d80 2023-10-03T10:18:00.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.996+0000 7ff453513700 1 -- 172.21.15.3:0/1654097425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff44c0747f0 con 0x7ff44c138d80 2023-10-03T10:18:00.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:00.996+0000 7ff441ffb700 1 -- 172.21.15.3:0/1654097425 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff444040290 con 0x7ff44c138d80 2023-10-03T10:18:01.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.001+0000 7ff441ffb700 1 -- 172.21.15.3:0/1654097425 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff444007920 con 0x7ff44c138d80 2023-10-03T10:18:01.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.026+0000 7f6ce6580700 1 Processor -- start 2023-10-03T10:18:01.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.026+0000 7f6ce6580700 1 -- start start 2023-10-03T10:18:01.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.027+0000 7f6ce6580700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6ce009d630 0x7f6ce00a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:01.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.027+0000 7f6ce6580700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ce0138d80 0x7f6ce013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:01.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.027+0000 7f6ce6580700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ce00a2a70 con 0x7f6ce00a0a30 2023-10-03T10:18:01.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.027+0000 7f6ce6580700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ce00a01d0 con 0x7f6ce009d630 2023-10-03T10:18:01.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.027+0000 7f6ce6580700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce009e2b0 con 0x7f6ce0138d80 2023-10-03T10:18:01.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.027+0000 7f6ce4b1d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ce0138d80 0x7f6ce013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:01.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.027+0000 7f6cdf7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6ce009d630 0x7f6ce00a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:01.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.027+0000 7f6cdffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6ce00a0a30 0x7f6ce00a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45206/0 (socket says 172.21.15.3:45206) 2023-10-03T10:18:01.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.027+0000 7f6cdffff700 1 -- 172.21.15.3:0/1187555596 learned_addr learned my addr 172.21.15.3:0/1187555596 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:01.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.027+0000 7f6cdeffd700 1 -- 172.21.15.3:0/1187555596 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3653979541 0 0) 0x7f6ce00a2a70 con 0x7f6ce00a0a30 2023-10-03T10:18:01.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.027+0000 7f6cdeffd700 1 -- 172.21.15.3:0/1187555596 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cc80035b0 con 0x7f6ce00a0a30 2023-10-03T10:18:01.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.028+0000 7f6ce4b1d700 1 -- 172.21.15.3:0/1187555596 >> v2:172.21.15.3:3300/0 conn(0x7f6ce009d630 msgr2=0x7f6ce00a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:01.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.028+0000 7f6ce4b1d700 1 --2- 172.21.15.3:0/1187555596 >> v2:172.21.15.3:3300/0 conn(0x7f6ce009d630 0x7f6ce00a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.028+0000 7f6ce4b1d700 1 -- 172.21.15.3:0/1187555596 >> v1:172.21.15.3:6789/0 conn(0x7f6ce00a0a30 legacy=0x7f6ce00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:01.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.028+0000 7f6ce4b1d700 1 -- 172.21.15.3:0/1187555596 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce009dc50 con 0x7f6ce0138d80 2023-10-03T10:18:01.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.028+0000 7f6cdf7fe700 1 --2- 172.21.15.3:0/1187555596 >> v2:172.21.15.3:3300/0 conn(0x7f6ce009d630 0x7f6ce00a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:01.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.028+0000 7f6ce4b1d700 1 --2- 172.21.15.3:0/1187555596 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ce0138d80 0x7f6ce013b160 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8009d90 tx=0x7f6cd800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52e68dd5713a9bd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:01.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.028+0000 7f6cdeffd700 1 -- 172.21.15.3:0/1187555596 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd800e070 con 0x7f6ce0138d80 2023-10-03T10:18:01.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.028+0000 7f6cdeffd700 1 -- 172.21.15.3:0/1187555596 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd800bb20 con 0x7f6ce0138d80 2023-10-03T10:18:01.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.028+0000 7f6cdeffd700 1 -- 172.21.15.3:0/1187555596 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8012430 con 0x7f6ce0138d80 2023-10-03T10:18:01.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.029+0000 7f6ce6580700 1 -- 172.21.15.3:0/1187555596 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ce0138d80 msgr2=0x7f6ce013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:01.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.029+0000 7f6ce6580700 1 --2- 172.21.15.3:0/1187555596 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ce0138d80 0x7f6ce013b160 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8009d90 tx=0x7f6cd800b330 comp rx=0 tx=0).stop 2023-10-03T10:18:01.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.029+0000 7f6ce6580700 1 -- 172.21.15.3:0/1187555596 shutdown_connections 2023-10-03T10:18:01.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.029+0000 7f6ce6580700 1 --2- 172.21.15.3:0/1187555596 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ce0138d80 0x7f6ce013b160 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.029+0000 7f6ce6580700 1 --2- 172.21.15.3:0/1187555596 >> v2:172.21.15.3:3300/0 conn(0x7f6ce009d630 0x7f6ce00a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.029+0000 7f6ce6580700 1 -- 172.21.15.3:0/1187555596 >> 172.21.15.3:0/1187555596 conn(0x7f6ce00a3050 msgr2=0x7f6ce00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:01.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.029+0000 7f6ce6580700 1 -- 172.21.15.3:0/1187555596 shutdown_connections 2023-10-03T10:18:01.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.029+0000 7f6ce6580700 1 -- 172.21.15.3:0/1187555596 wait complete. 2023-10-03T10:18:01.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.030+0000 7f6ce6580700 1 Processor -- start 2023-10-03T10:18:01.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.030+0000 7f6ce6580700 1 -- start start 2023-10-03T10:18:01.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.031+0000 7f6ce6580700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ce009d630 0x7f6ce012c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:01.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.031+0000 7f6ce6580700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ce00a0a30 0x7f6ce012cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:01.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.031+0000 7f6ce6580700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ce0138d80 0x7f6ce012cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:01.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.031+0000 7f6ce6580700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce012d940 con 0x7f6ce009d630 2023-10-03T10:18:01.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.031+0000 7f6ce6580700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce013abc0 con 0x7f6ce00a0a30 2023-10-03T10:18:01.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.031+0000 7f6ce6580700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce00a0410 con 0x7f6ce0138d80 2023-10-03T10:18:01.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.031+0000 7f6cdf7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ce00a0a30 0x7f6ce012cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:01.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.031+0000 7f6cdf7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ce00a0a30 0x7f6ce012cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59392/0 (socket says 172.21.15.3:59392) 2023-10-03T10:18:01.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.031+0000 7f6cdf7fe700 1 -- 172.21.15.3:0/395226105 learned_addr learned my addr 172.21.15.3:0/395226105 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:01.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.031+0000 7f6cdffff700 1 --2- 172.21.15.3:0/395226105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ce009d630 0x7f6ce012c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:01.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.031+0000 7f6ce4b1d700 1 --2- 172.21.15.3:0/395226105 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ce0138d80 0x7f6ce012cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:01.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.031+0000 7f6cdffff700 1 -- 172.21.15.3:0/395226105 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ce0138d80 msgr2=0x7f6ce012cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:01.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.031+0000 7f6cdffff700 1 --2- 172.21.15.3:0/395226105 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ce0138d80 0x7f6ce012cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.031+0000 7f6cdffff700 1 -- 172.21.15.3:0/395226105 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ce00a0a30 msgr2=0x7f6ce012cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:01.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.031+0000 7f6cdffff700 1 --2- 172.21.15.3:0/395226105 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ce00a0a30 0x7f6ce012cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.031+0000 7f6cdffff700 1 -- 172.21.15.3:0/395226105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd40094a0 con 0x7f6ce009d630 2023-10-03T10:18:01.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.031+0000 7f6cdf7fe700 1 --2- 172.21.15.3:0/395226105 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ce00a0a30 0x7f6ce012cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:01.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.031+0000 7f6cdffff700 1 --2- 172.21.15.3:0/395226105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ce009d630 0x7f6ce012c570 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f6cd000c490 tx=0x7f6cd000c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:01.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.032+0000 7f6cdcff9700 1 -- 172.21.15.3:0/395226105 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0005d70 con 0x7f6ce009d630 2023-10-03T10:18:01.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.032+0000 7f6cdcff9700 1 -- 172.21.15.3:0/395226105 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd00070d0 con 0x7f6ce009d630 2023-10-03T10:18:01.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.032+0000 7f6ce6580700 1 -- 172.21.15.3:0/395226105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd8009a70 con 0x7f6ce009d630 2023-10-03T10:18:01.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.032+0000 7f6cdcff9700 1 -- 172.21.15.3:0/395226105 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0007440 con 0x7f6ce009d630 2023-10-03T10:18:01.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.032+0000 7f6ce6580700 1 -- 172.21.15.3:0/395226105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce012fe60 con 0x7f6ce009d630 2023-10-03T10:18:01.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.033+0000 7f6cdcff9700 1 -- 172.21.15.3:0/395226105 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6cd00075a0 con 0x7f6ce009d630 2023-10-03T10:18:01.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.033+0000 7f6cdcff9700 1 -- 172.21.15.3:0/395226105 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6cd003eb20 con 0x7f6ce009d630 2023-10-03T10:18:01.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.033+0000 7f6ce6580700 1 -- 172.21.15.3:0/395226105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce012fca0 con 0x7f6ce009d630 2023-10-03T10:18:01.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.043+0000 7f6cdcff9700 1 -- 172.21.15.3:0/395226105 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6cd0017020 con 0x7f6ce009d630 2023-10-03T10:18:01.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.070+0000 7fbe7c6e3700 1 Processor -- start 2023-10-03T10:18:01.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.070+0000 7fbe7c6e3700 1 -- start start 2023-10-03T10:18:01.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.070+0000 7fbe7c6e3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe7409d630 0x7fbe740a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:01.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.070+0000 7fbe7c6e3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbe74138d80 0x7fbe7413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:01.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.070+0000 7fbe7c6e3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe740a2a70 con 0x7fbe740a0a30 2023-10-03T10:18:01.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.070+0000 7fbe7c6e3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe740a01d0 con 0x7fbe74138d80 2023-10-03T10:18:01.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.070+0000 7fbe7c6e3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7409e2b0 con 0x7fbe7409d630 2023-10-03T10:18:01.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.071+0000 7fbe7ac80700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbe74138d80 0x7fbe7413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:01.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.071+0000 7fbe7ac80700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbe74138d80 0x7fbe7413b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44936/0 (socket says 172.21.15.3:44936) 2023-10-03T10:18:01.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.071+0000 7fbe7a47f700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbe740a0a30 0x7fbe740a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45222/0 (socket says 172.21.15.3:45222) 2023-10-03T10:18:01.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.071+0000 7fbe7ac80700 1 -- 172.21.15.3:0/2406274950 learned_addr learned my addr 172.21.15.3:0/2406274950 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:01.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.071+0000 7fbe79c7e700 1 --2- 172.21.15.3:0/2406274950 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe7409d630 0x7fbe740a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:01.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.071+0000 7fbe79c7e700 1 -- 172.21.15.3:0/2406274950 >> v2:172.21.15.3:3300/0 conn(0x7fbe74138d80 msgr2=0x7fbe7413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:01.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.071+0000 7fbe79c7e700 1 --2- 172.21.15.3:0/2406274950 >> v2:172.21.15.3:3300/0 conn(0x7fbe74138d80 0x7fbe7413b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.071+0000 7fbe79c7e700 1 -- 172.21.15.3:0/2406274950 >> v1:172.21.15.3:6789/0 conn(0x7fbe740a0a30 legacy=0x7fbe740a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:01.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.071+0000 7fbe79c7e700 1 -- 172.21.15.3:0/2406274950 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe7409dc50 con 0x7fbe7409d630 2023-10-03T10:18:01.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.071+0000 7fbe79c7e700 1 --2- 172.21.15.3:0/2406274950 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe7409d630 0x7fbe740a1510 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009840 tx=0x7fbe64009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2323f33a8ddbe5c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:01.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.071+0000 7fbe7947d700 1 -- 172.21.15.3:0/2406274950 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe64011830 con 0x7fbe7409d630 2023-10-03T10:18:01.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.072+0000 7fbe7947d700 1 -- 172.21.15.3:0/2406274950 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe64011990 con 0x7fbe7409d630 2023-10-03T10:18:01.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.072+0000 7fbe7947d700 1 -- 172.21.15.3:0/2406274950 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe64011d20 con 0x7fbe7409d630 2023-10-03T10:18:01.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.072+0000 7fbe7c6e3700 1 -- 172.21.15.3:0/2406274950 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe7409d630 msgr2=0x7fbe740a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:01.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.072+0000 7fbe7c6e3700 1 --2- 172.21.15.3:0/2406274950 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe7409d630 0x7fbe740a1510 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009840 tx=0x7fbe64009b50 comp rx=0 tx=0).stop 2023-10-03T10:18:01.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.072+0000 7fbe7c6e3700 1 -- 172.21.15.3:0/2406274950 shutdown_connections 2023-10-03T10:18:01.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.072+0000 7fbe7c6e3700 1 --2- 172.21.15.3:0/2406274950 >> v2:172.21.15.3:3300/0 conn(0x7fbe74138d80 0x7fbe7413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.072+0000 7fbe7c6e3700 1 --2- 172.21.15.3:0/2406274950 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe7409d630 0x7fbe740a1510 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.072+0000 7fbe7c6e3700 1 -- 172.21.15.3:0/2406274950 >> 172.21.15.3:0/2406274950 conn(0x7fbe740a3050 msgr2=0x7fbe740a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:01.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.072+0000 7fbe7c6e3700 1 -- 172.21.15.3:0/2406274950 shutdown_connections 2023-10-03T10:18:01.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.072+0000 7fbe7c6e3700 1 -- 172.21.15.3:0/2406274950 wait complete. 2023-10-03T10:18:01.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.073+0000 7fbe7c6e3700 1 Processor -- start 2023-10-03T10:18:01.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.073+0000 7fbe7c6e3700 1 -- start start 2023-10-03T10:18:01.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.074+0000 7fbe7c6e3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe7409d630 0x7fbe741c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:01.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.074+0000 7fbe7c6e3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe740a0a30 0x7fbe741c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:01.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.074+0000 7fbe7c6e3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe74138d80 0x7fbe741c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:01.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.074+0000 7fbe7c6e3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe741381b0 con 0x7fbe740a0a30 2023-10-03T10:18:01.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.074+0000 7fbe7c6e3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe741361d0 con 0x7fbe74138d80 2023-10-03T10:18:01.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.074+0000 7fbe7c6e3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7413b030 con 0x7fbe7409d630 2023-10-03T10:18:01.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.074+0000 7fbe7a47f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe7409d630 0x7fbe741c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:01.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.074+0000 7fbe79c7e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe740a0a30 0x7fbe741c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:01.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.074+0000 7fbe7ac80700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe74138d80 0x7fbe741c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:01.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.074+0000 7fbe7a47f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe7409d630 0x7fbe741c8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48016/0 (socket says 172.21.15.3:48016) 2023-10-03T10:18:01.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.074+0000 7fbe7a47f700 1 -- 172.21.15.3:0/1182206159 learned_addr learned my addr 172.21.15.3:0/1182206159 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:01.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.074+0000 7fbe7ac80700 1 -- 172.21.15.3:0/1182206159 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe7409d630 msgr2=0x7fbe741c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:01.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.074+0000 7fbe7ac80700 1 --2- 172.21.15.3:0/1182206159 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe7409d630 0x7fbe741c8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.074+0000 7fbe7ac80700 1 -- 172.21.15.3:0/1182206159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe740a0a30 msgr2=0x7fbe741c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:01.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.075+0000 7fbe7ac80700 1 --2- 172.21.15.3:0/1182206159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe740a0a30 0x7fbe741c8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.075+0000 7fbe7ac80700 1 -- 172.21.15.3:0/1182206159 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe640094a0 con 0x7fbe74138d80 2023-10-03T10:18:01.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.075+0000 7fbe7ac80700 1 --2- 172.21.15.3:0/1182206159 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe74138d80 0x7fbe741c9080 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c00ef90 tx=0x7fbe6c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:01.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.075+0000 7fbe6affd700 1 -- 172.21.15.3:0/1182206159 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c003c20 con 0x7fbe74138d80 2023-10-03T10:18:01.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.075+0000 7fbe6affd700 1 -- 172.21.15.3:0/1182206159 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe6c00f040 con 0x7fbe74138d80 2023-10-03T10:18:01.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.075+0000 7fbe6affd700 1 -- 172.21.15.3:0/1182206159 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c009e60 con 0x7fbe74138d80 2023-10-03T10:18:01.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.075+0000 7fbe7c6e3700 1 -- 172.21.15.3:0/1182206159 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe740a0410 con 0x7fbe74138d80 2023-10-03T10:18:01.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.075+0000 7fbe7c6e3700 1 -- 172.21.15.3:0/1182206159 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe74136c40 con 0x7fbe74138d80 2023-10-03T10:18:01.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.076+0000 7fbe6affd700 1 -- 172.21.15.3:0/1182206159 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbe6c00c880 con 0x7fbe74138d80 2023-10-03T10:18:01.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.076+0000 7fbe6affd700 1 -- 172.21.15.3:0/1182206159 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fbe6c03f930 con 0x7fbe74138d80 2023-10-03T10:18:01.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.076+0000 7fbe7c6e3700 1 -- 172.21.15.3:0/1182206159 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe74136010 con 0x7fbe74138d80 2023-10-03T10:18:01.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.081+0000 7fbe6affd700 1 -- 172.21.15.3:0/1182206159 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbe6c01c360 con 0x7fbe74138d80 2023-10-03T10:18:01.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.215+0000 7ff453513700 1 -- 172.21.15.3:0/1654097425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff44c123690 con 0x7ff44c138d80 2023-10-03T10:18:01.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.215+0000 7ff441ffb700 1 -- 172.21.15.3:0/1654097425 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff444018020 con 0x7ff44c138d80 2023-10-03T10:18:01.216 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:01.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.218+0000 7ff453513700 1 -- 172.21.15.3:0/1654097425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff44c138d80 msgr2=0x7ff44c1212b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:01.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.218+0000 7ff453513700 1 --2- 172.21.15.3:0/1654097425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff44c138d80 0x7ff44c1212b0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7ff44400efc0 tx=0x7ff44400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.218+0000 7ff453513700 1 -- 172.21.15.3:0/1654097425 shutdown_connections 2023-10-03T10:18:01.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.218+0000 7ff453513700 1 --2- 172.21.15.3:0/1654097425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff44c138d80 0x7ff44c1212b0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.218+0000 7ff453513700 1 --2- 172.21.15.3:0/1654097425 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff44c09d630 0x7ff44c120830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.218+0000 7ff453513700 1 --2- 172.21.15.3:0/1654097425 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff44c0a0a30 0x7ff44c120d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.218+0000 7ff453513700 1 -- 172.21.15.3:0/1654097425 >> 172.21.15.3:0/1654097425 conn(0x7ff44c0a3050 msgr2=0x7ff44c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:01.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.218+0000 7ff453513700 1 -- 172.21.15.3:0/1654097425 shutdown_connections 2023-10-03T10:18:01.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.218+0000 7ff453513700 1 -- 172.21.15.3:0/1654097425 wait complete. 2023-10-03T10:18:01.232 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:01.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.250+0000 7f6ce6580700 1 -- 172.21.15.3:0/395226105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6ce00747f0 con 0x7f6ce009d630 2023-10-03T10:18:01.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.251+0000 7f6cdcff9700 1 -- 172.21.15.3:0/395226105 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6cd001bb10 con 0x7f6ce009d630 2023-10-03T10:18:01.251 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:01.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.253+0000 7f6ce6580700 1 -- 172.21.15.3:0/395226105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ce009d630 msgr2=0x7f6ce012c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:01.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.253+0000 7f6ce6580700 1 --2- 172.21.15.3:0/395226105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ce009d630 0x7f6ce012c570 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f6cd000c490 tx=0x7f6cd000c7a0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.253+0000 7f6ce6580700 1 -- 172.21.15.3:0/395226105 shutdown_connections 2023-10-03T10:18:01.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.253+0000 7f6ce6580700 1 --2- 172.21.15.3:0/395226105 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6ce0138d80 0x7f6ce012cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.253+0000 7f6ce6580700 1 --2- 172.21.15.3:0/395226105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ce009d630 0x7f6ce012c570 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.253+0000 7f6ce6580700 1 --2- 172.21.15.3:0/395226105 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6ce00a0a30 0x7f6ce012cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.253+0000 7f6ce6580700 1 -- 172.21.15.3:0/395226105 >> 172.21.15.3:0/395226105 conn(0x7f6ce00a3050 msgr2=0x7f6ce012adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:01.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.253+0000 7f6ce6580700 1 -- 172.21.15.3:0/395226105 shutdown_connections 2023-10-03T10:18:01.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.253+0000 7f6ce6580700 1 -- 172.21.15.3:0/395226105 wait complete. 2023-10-03T10:18:01.257 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:01.268 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:01.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.291+0000 7fbe7c6e3700 1 -- 172.21.15.3:0/1182206159 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbe740747f0 con 0x7fbe74138d80 2023-10-03T10:18:01.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.292+0000 7fbe6affd700 1 -- 172.21.15.3:0/1182206159 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbe6c014e00 con 0x7fbe74138d80 2023-10-03T10:18:01.292 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:01.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.294+0000 7fbe7c6e3700 1 -- 172.21.15.3:0/1182206159 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe74138d80 msgr2=0x7fbe741c9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:01.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.294+0000 7fbe7c6e3700 1 --2- 172.21.15.3:0/1182206159 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe74138d80 0x7fbe741c9080 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c00ef90 tx=0x7fbe6c00c3a0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.294+0000 7fbe7c6e3700 1 -- 172.21.15.3:0/1182206159 shutdown_connections 2023-10-03T10:18:01.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.294+0000 7fbe7c6e3700 1 --2- 172.21.15.3:0/1182206159 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe74138d80 0x7fbe741c9080 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.294+0000 7fbe7c6e3700 1 --2- 172.21.15.3:0/1182206159 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe7409d630 0x7fbe741c8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.294+0000 7fbe7c6e3700 1 --2- 172.21.15.3:0/1182206159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe740a0a30 0x7fbe741c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.294+0000 7fbe7c6e3700 1 -- 172.21.15.3:0/1182206159 >> 172.21.15.3:0/1182206159 conn(0x7fbe740a3050 msgr2=0x7fbe7409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:01.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.294+0000 7fbe7c6e3700 1 -- 172.21.15.3:0/1182206159 shutdown_connections 2023-10-03T10:18:01.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.294+0000 7fbe7c6e3700 1 -- 172.21.15.3:0/1182206159 wait complete. 2023-10-03T10:18:01.311 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:01.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.398+0000 7f4cf7d61700 1 Processor -- start 2023-10-03T10:18:01.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.398+0000 7f4cf7d61700 1 -- start start 2023-10-03T10:18:01.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.398+0000 7f4cf7d61700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4cf009e930 0x7f4cf009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:01.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.398+0000 7f4cf7d61700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cf009f240 0x7f4cf01317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:01.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.398+0000 7f4cf7d61700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf0070940 con 0x7f4cf009f650 2023-10-03T10:18:01.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.398+0000 7f4cf7d61700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cf006fd60 con 0x7f4cf009e930 2023-10-03T10:18:01.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.398+0000 7f4cf7d61700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf006fa80 con 0x7f4cf009f240 2023-10-03T10:18:01.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.399+0000 7f4cf52fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cf009f240 0x7f4cf01317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:01.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.399+0000 7f4cf5afd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4cf009e930 0x7f4cf009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:01.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.399+0000 7f4cf5afd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4cf009e930 0x7f4cf009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44944/0 (socket says 172.21.15.3:44944) 2023-10-03T10:18:01.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.399+0000 7f4cf52fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cf009f240 0x7f4cf01317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59408/0 (socket says 172.21.15.3:59408) 2023-10-03T10:18:01.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.399+0000 7f4cf5afd700 1 -- 172.21.15.3:0/803826551 learned_addr learned my addr 172.21.15.3:0/803826551 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:01.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.399+0000 7f4cf52fc700 1 -- 172.21.15.3:0/803826551 >> v2:172.21.15.3:3300/0 conn(0x7f4cf009e930 msgr2=0x7f4cf009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:01.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.399+0000 7f4cf52fc700 1 --2- 172.21.15.3:0/803826551 >> v2:172.21.15.3:3300/0 conn(0x7f4cf009e930 0x7f4cf009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.399+0000 7f4cf52fc700 1 -- 172.21.15.3:0/803826551 >> v1:172.21.15.3:6789/0 conn(0x7f4cf009f650 legacy=0x7f4cf0131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:01.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.399+0000 7f4cf52fc700 1 -- 172.21.15.3:0/803826551 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf009fb60 con 0x7f4cf009f240 2023-10-03T10:18:01.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.399+0000 7f4cf5afd700 1 --2- 172.21.15.3:0/803826551 >> v2:172.21.15.3:3300/0 conn(0x7f4cf009e930 0x7f4cf009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:01.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.399+0000 7f4cf52fc700 1 --2- 172.21.15.3:0/803826551 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cf009f240 0x7f4cf01317f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009840 tx=0x7f4ce0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e8eed54b8c4e3a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:01.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.399+0000 7f4cf4afb700 1 -- 172.21.15.3:0/803826551 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0011830 con 0x7f4cf009f240 2023-10-03T10:18:01.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.399+0000 7f4cf4afb700 1 -- 172.21.15.3:0/803826551 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce0011990 con 0x7f4cf009f240 2023-10-03T10:18:01.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.399+0000 7f4cf4afb700 1 -- 172.21.15.3:0/803826551 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0011d20 con 0x7f4cf009f240 2023-10-03T10:18:01.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.400+0000 7f4cf7d61700 1 -- 172.21.15.3:0/803826551 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cf009f240 msgr2=0x7f4cf01317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:01.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.400+0000 7f4cf7d61700 1 --2- 172.21.15.3:0/803826551 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cf009f240 0x7f4cf01317f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009840 tx=0x7f4ce0009b50 comp rx=0 tx=0).stop 2023-10-03T10:18:01.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.400+0000 7f4cf7d61700 1 -- 172.21.15.3:0/803826551 shutdown_connections 2023-10-03T10:18:01.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.400+0000 7f4cf7d61700 1 --2- 172.21.15.3:0/803826551 >> v2:172.21.15.3:3300/0 conn(0x7f4cf009e930 0x7f4cf009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.400+0000 7f4cf7d61700 1 --2- 172.21.15.3:0/803826551 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cf009f240 0x7f4cf01317f0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.400+0000 7f4cf7d61700 1 -- 172.21.15.3:0/803826551 >> 172.21.15.3:0/803826551 conn(0x7f4cf00a4210 msgr2=0x7f4cf00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:01.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.400+0000 7f4cf7d61700 1 -- 172.21.15.3:0/803826551 shutdown_connections 2023-10-03T10:18:01.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.400+0000 7f4cf7d61700 1 -- 172.21.15.3:0/803826551 wait complete. 2023-10-03T10:18:01.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.401+0000 7f4cf7d61700 1 Processor -- start 2023-10-03T10:18:01.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.401+0000 7f4cf7d61700 1 -- start start 2023-10-03T10:18:01.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.401+0000 7f4cf7d61700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cf009e930 0x7f4cf01a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:01.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.402+0000 7f4cf7d61700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cf009f240 0x7f4cf01a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:01.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.402+0000 7f4cf7d61700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cf009f650 0x7f4cf01a6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:01.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.402+0000 7f4cf7d61700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf01a71a0 con 0x7f4cf009f240 2023-10-03T10:18:01.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.402+0000 7f4cf7d61700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf01a78b0 con 0x7f4cf009f650 2023-10-03T10:18:01.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.402+0000 7f4cf7d61700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf01a7ed0 con 0x7f4cf009e930 2023-10-03T10:18:01.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.402+0000 7f4cf52fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cf009f240 0x7f4cf01a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:01.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.402+0000 7f4cf62fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cf009f650 0x7f4cf01a6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:01.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.402+0000 7f4cf5afd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cf009e930 0x7f4cf01a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:01.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.402+0000 7f4cf52fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cf009f240 0x7f4cf01a6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44954/0 (socket says 172.21.15.3:44954) 2023-10-03T10:18:01.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.402+0000 7f4cf52fc700 1 -- 172.21.15.3:0/503387609 learned_addr learned my addr 172.21.15.3:0/503387609 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:01.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.402+0000 7f4cf52fc700 1 -- 172.21.15.3:0/503387609 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cf009e930 msgr2=0x7f4cf01a5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:01.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.402+0000 7f4cf52fc700 1 --2- 172.21.15.3:0/503387609 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cf009e930 0x7f4cf01a5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.402+0000 7f4cf52fc700 1 -- 172.21.15.3:0/503387609 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cf009f650 msgr2=0x7f4cf01a6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:01.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.402+0000 7f4cf52fc700 1 --2- 172.21.15.3:0/503387609 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cf009f650 0x7f4cf01a6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.402+0000 7f4cf52fc700 1 -- 172.21.15.3:0/503387609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cec009550 con 0x7f4cf009f240 2023-10-03T10:18:01.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.402+0000 7f4cf62fe700 1 --2- 172.21.15.3:0/503387609 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cf009f650 0x7f4cf01a6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:01.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.402+0000 7f4cf52fc700 1 --2- 172.21.15.3:0/503387609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cf009f240 0x7f4cf01a6230 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00090b0 tx=0x7f4ce0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:01.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.402+0000 7f4ce67fc700 1 -- 172.21.15.3:0/503387609 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce00036a0 con 0x7f4cf009f240 2023-10-03T10:18:01.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.402+0000 7f4ce67fc700 1 -- 172.21.15.3:0/503387609 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce001f070 con 0x7f4cf009f240 2023-10-03T10:18:01.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.402+0000 7f4cf7d61700 1 -- 172.21.15.3:0/503387609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce00094a0 con 0x7f4cf009f240 2023-10-03T10:18:01.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.402+0000 7f4ce67fc700 1 -- 172.21.15.3:0/503387609 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0003a60 con 0x7f4cf009f240 2023-10-03T10:18:01.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.402+0000 7f4cf7d61700 1 -- 172.21.15.3:0/503387609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf00a0190 con 0x7f4cf009f240 2023-10-03T10:18:01.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.403+0000 7f4ce67fc700 1 -- 172.21.15.3:0/503387609 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4ce0003bc0 con 0x7f4cf009f240 2023-10-03T10:18:01.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.403+0000 7f4ce67fc700 1 -- 172.21.15.3:0/503387609 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4ce0021070 con 0x7f4cf009f240 2023-10-03T10:18:01.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.403+0000 7f4cf7d61700 1 -- 172.21.15.3:0/503387609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf01a2b40 con 0x7f4cf009f240 2023-10-03T10:18:01.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.408+0000 7f4ce67fc700 1 -- 172.21.15.3:0/503387609 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4ce0015080 con 0x7f4cf009f240 2023-10-03T10:18:01.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.615+0000 7f4cf7d61700 1 -- 172.21.15.3:0/503387609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4cf00a34f0 con 0x7f4cf009f240 2023-10-03T10:18:01.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.615+0000 7f4ce67fc700 1 -- 172.21.15.3:0/503387609 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4ce0035090 con 0x7f4cf009f240 2023-10-03T10:18:01.616 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.618+0000 7f4cf7d61700 1 -- 172.21.15.3:0/503387609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cf009f240 msgr2=0x7f4cf01a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.618+0000 7f4cf7d61700 1 --2- 172.21.15.3:0/503387609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cf009f240 0x7f4cf01a6230 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00090b0 tx=0x7f4ce0004750 comp rx=0 tx=0).stop 2023-10-03T10:18:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.618+0000 7f4cf7d61700 1 -- 172.21.15.3:0/503387609 shutdown_connections 2023-10-03T10:18:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.618+0000 7f4cf7d61700 1 --2- 172.21.15.3:0/503387609 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cf009f650 0x7f4cf01a6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.618+0000 7f4cf7d61700 1 --2- 172.21.15.3:0/503387609 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cf009e930 0x7f4cf01a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.618+0000 7f4cf7d61700 1 --2- 172.21.15.3:0/503387609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cf009f240 0x7f4cf01a6230 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:01.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.618+0000 7f4cf7d61700 1 -- 172.21.15.3:0/503387609 >> 172.21.15.3:0/503387609 conn(0x7f4cf00a4210 msgr2=0x7f4cf009dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:01.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.618+0000 7f4cf7d61700 1 -- 172.21.15.3:0/503387609 shutdown_connections 2023-10-03T10:18:01.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:01.618+0000 7f4cf7d61700 1 -- 172.21.15.3:0/503387609 wait complete. 2023-10-03T10:18:01.631 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:02.233 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:02.269 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:02.312 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:02.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.384+0000 7f9699fa6700 1 Processor -- start 2023-10-03T10:18:02.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.384+0000 7f9699fa6700 1 -- start start 2023-10-03T10:18:02.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.385+0000 7f9699fa6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f969409d6f0 0x7f969409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:02.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.385+0000 7f9699fa6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f969409e000 0x7f96941339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:02.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.385+0000 7f9699fa6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f969409e410 0x7f9694133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:02.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.385+0000 7f9699fa6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f969406b460 con 0x7f969409d6f0 2023-10-03T10:18:02.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.385+0000 7f9699fa6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969409b120 con 0x7f969409e000 2023-10-03T10:18:02.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.385+0000 7f9699fa6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f969409a7e0 con 0x7f969409e410 2023-10-03T10:18:02.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.385+0000 7f9692ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f969409e000 0x7f96941339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:02.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.385+0000 7f96937fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f969409d6f0 0x7f969409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:02.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.385+0000 7f9692ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f969409e000 0x7f96941339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59430/0 (socket says 172.21.15.3:59430) 2023-10-03T10:18:02.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.385+0000 7f96937fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f969409d6f0 0x7f969409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44958/0 (socket says 172.21.15.3:44958) 2023-10-03T10:18:02.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.385+0000 7f9692ffd700 1 -- 172.21.15.3:0/1890114503 learned_addr learned my addr 172.21.15.3:0/1890114503 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:02.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.385+0000 7f9693fff700 1 --2- 172.21.15.3:0/1890114503 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f969409e410 0x7f9694133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:02.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.385+0000 7f96937fe700 1 -- 172.21.15.3:0/1890114503 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f969409e410 msgr2=0x7f9694133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:02.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.385+0000 7f96937fe700 1 --2- 172.21.15.3:0/1890114503 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f969409e410 0x7f9694133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.385+0000 7f96937fe700 1 -- 172.21.15.3:0/1890114503 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f969409e000 msgr2=0x7f96941339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:02.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.385+0000 7f96937fe700 1 --2- 172.21.15.3:0/1890114503 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f969409e000 0x7f96941339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.385+0000 7f96937fe700 1 -- 172.21.15.3:0/1890114503 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969409a500 con 0x7f969409d6f0 2023-10-03T10:18:02.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.385+0000 7f9692ffd700 1 --2- 172.21.15.3:0/1890114503 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f969409e000 0x7f96941339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:02.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.386+0000 7f96937fe700 1 --2- 172.21.15.3:0/1890114503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f969409d6f0 0x7f969409dac0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f9684009840 tx=0x7f9684009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34801eeb3cdc2bba server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:02.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.386+0000 7f96927fc700 1 -- 172.21.15.3:0/1890114503 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9684011830 con 0x7f969409d6f0 2023-10-03T10:18:02.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.386+0000 7f96927fc700 1 -- 172.21.15.3:0/1890114503 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9684011990 con 0x7f969409d6f0 2023-10-03T10:18:02.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.386+0000 7f96927fc700 1 -- 172.21.15.3:0/1890114503 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9684011d20 con 0x7f969409d6f0 2023-10-03T10:18:02.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.387+0000 7f9699fa6700 1 -- 172.21.15.3:0/1890114503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f969409d6f0 msgr2=0x7f969409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:02.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.387+0000 7f9699fa6700 1 --2- 172.21.15.3:0/1890114503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f969409d6f0 0x7f969409dac0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f9684009840 tx=0x7f9684009b50 comp rx=0 tx=0).stop 2023-10-03T10:18:02.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.387+0000 7f9699fa6700 1 -- 172.21.15.3:0/1890114503 shutdown_connections 2023-10-03T10:18:02.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.387+0000 7f9699fa6700 1 --2- 172.21.15.3:0/1890114503 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f969409e410 0x7f9694133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.387+0000 7f9699fa6700 1 --2- 172.21.15.3:0/1890114503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f969409d6f0 0x7f969409dac0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.387+0000 7f9699fa6700 1 --2- 172.21.15.3:0/1890114503 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f969409e000 0x7f96941339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.387+0000 7f9699fa6700 1 -- 172.21.15.3:0/1890114503 >> 172.21.15.3:0/1890114503 conn(0x7f9694066f50 msgr2=0x7f969409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:02.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.387+0000 7f9699fa6700 1 -- 172.21.15.3:0/1890114503 shutdown_connections 2023-10-03T10:18:02.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.387+0000 7f9699fa6700 1 -- 172.21.15.3:0/1890114503 wait complete. 2023-10-03T10:18:02.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.388+0000 7f9699fa6700 1 Processor -- start 2023-10-03T10:18:02.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.389+0000 7f9699fa6700 1 -- start start 2023-10-03T10:18:02.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.389+0000 7f9699fa6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f969409d6f0 0x7f96941a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:02.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.389+0000 7f9699fa6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f969409e000 0x7f96941a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:02.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.389+0000 7f9699fa6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f969409e410 0x7f96941c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:02.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.389+0000 7f9699fa6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96940a1f20 con 0x7f969409d6f0 2023-10-03T10:18:02.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.389+0000 7f9699fa6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96941a80a0 con 0x7f969409e410 2023-10-03T10:18:02.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.389+0000 7f9699fa6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f96941b91f0 con 0x7f969409e000 2023-10-03T10:18:02.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.390+0000 7f96937fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f969409d6f0 0x7f96941a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:02.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.390+0000 7f96937fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f969409d6f0 0x7f96941a5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44974/0 (socket says 172.21.15.3:44974) 2023-10-03T10:18:02.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.390+0000 7f9693fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f969409e410 0x7f96941c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:02.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.390+0000 7f96937fe700 1 -- 172.21.15.3:0/1531502776 learned_addr learned my addr 172.21.15.3:0/1531502776 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:02.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.390+0000 7f9692ffd700 1 --2- 172.21.15.3:0/1531502776 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f969409e000 0x7f96941a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:02.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.390+0000 7f96937fe700 1 -- 172.21.15.3:0/1531502776 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f969409e000 msgr2=0x7f96941a5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:02.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.390+0000 7f96937fe700 1 --2- 172.21.15.3:0/1531502776 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f969409e000 0x7f96941a5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.390+0000 7f96937fe700 1 -- 172.21.15.3:0/1531502776 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f969409e410 msgr2=0x7f96941c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:02.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.390+0000 7f96937fe700 1 --2- 172.21.15.3:0/1531502776 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f969409e410 0x7f96941c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.390+0000 7f96937fe700 1 -- 172.21.15.3:0/1531502776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9688009550 con 0x7f969409d6f0 2023-10-03T10:18:02.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.390+0000 7f96937fe700 1 --2- 172.21.15.3:0/1531502776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f969409d6f0 0x7f96941a5130 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f9684009ea0 tx=0x7f9684004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:02.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.390+0000 7f967bfff700 1 -- 172.21.15.3:0/1531502776 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9684004450 con 0x7f969409d6f0 2023-10-03T10:18:02.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.391+0000 7f967bfff700 1 -- 172.21.15.3:0/1531502776 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968401f070 con 0x7f969409d6f0 2023-10-03T10:18:02.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.391+0000 7f9699fa6700 1 -- 172.21.15.3:0/1531502776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96840094a0 con 0x7f969409d6f0 2023-10-03T10:18:02.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.391+0000 7f967bfff700 1 -- 172.21.15.3:0/1531502776 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96840038e0 con 0x7f969409d6f0 2023-10-03T10:18:02.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.391+0000 7f9699fa6700 1 -- 172.21.15.3:0/1531502776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96941a6c60 con 0x7f969409d6f0 2023-10-03T10:18:02.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.392+0000 7f967bfff700 1 -- 172.21.15.3:0/1531502776 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9684003a40 con 0x7f969409d6f0 2023-10-03T10:18:02.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.392+0000 7f967bfff700 1 -- 172.21.15.3:0/1531502776 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9684021070 con 0x7f969409d6f0 2023-10-03T10:18:02.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.392+0000 7f9699fa6700 1 -- 172.21.15.3:0/1531502776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96941a81e0 con 0x7f969409d6f0 2023-10-03T10:18:02.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.397+0000 7f967bfff700 1 -- 172.21.15.3:0/1531502776 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f96941a81e0 con 0x7f969409d6f0 2023-10-03T10:18:02.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.424+0000 7fcd82c18700 1 Processor -- start 2023-10-03T10:18:02.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.424+0000 7fcd82c18700 1 -- start start 2023-10-03T10:18:02.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.425+0000 7fcd82c18700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd7c09d6f0 0x7fcd7c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:02.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.425+0000 7fcd82c18700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd7c09e000 0x7fcd7c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:02.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.425+0000 7fcd82c18700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd7c09b120 con 0x7fcd7c09e410 2023-10-03T10:18:02.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.425+0000 7fcd82c18700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c06b460 con 0x7fcd7c09d6f0 2023-10-03T10:18:02.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.425+0000 7fcd82c18700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c09a7e0 con 0x7fcd7c09e000 2023-10-03T10:18:02.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.425+0000 7fcd809b4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd7c09d6f0 0x7fcd7c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:02.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.425+0000 7fcd809b4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd7c09d6f0 0x7fcd7c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59450/0 (socket says 172.21.15.3:59450) 2023-10-03T10:18:02.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.425+0000 7fcd809b4700 1 -- 172.21.15.3:0/3878503740 learned_addr learned my addr 172.21.15.3:0/3878503740 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:02.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.425+0000 7fcd7bfff700 1 --2- 172.21.15.3:0/3878503740 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd7c09e000 0x7fcd7c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:02.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.425+0000 7fcd809b4700 1 -- 172.21.15.3:0/3878503740 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd7c09e000 msgr2=0x7fcd7c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:02.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.425+0000 7fcd7b7fe700 1 -- 172.21.15.3:0/3878503740 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 772364992 0 0) 0x7fcd7c09b120 con 0x7fcd7c09e410 2023-10-03T10:18:02.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.425+0000 7fcd809b4700 1 --2- 172.21.15.3:0/3878503740 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd7c09e000 0x7fcd7c1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.425+0000 7fcd809b4700 1 -- 172.21.15.3:0/3878503740 >> v1:172.21.15.3:6789/0 conn(0x7fcd7c09e410 legacy=0x7fcd7c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:02.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.425+0000 7fcd809b4700 1 -- 172.21.15.3:0/3878503740 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c09a500 con 0x7fcd7c09d6f0 2023-10-03T10:18:02.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.425+0000 7fcd7bfff700 1 --2- 172.21.15.3:0/3878503740 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd7c09e000 0x7fcd7c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:02.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.426+0000 7fcd809b4700 1 --2- 172.21.15.3:0/3878503740 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd7c09d6f0 0x7fcd7c09dac0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fcd70009920 tx=0x7fcd70009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83b78a367eca29e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:02.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.426+0000 7fcd7b7fe700 1 -- 172.21.15.3:0/3878503740 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd70011830 con 0x7fcd7c09d6f0 2023-10-03T10:18:02.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.426+0000 7fcd7b7fe700 1 -- 172.21.15.3:0/3878503740 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd70011990 con 0x7fcd7c09d6f0 2023-10-03T10:18:02.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.426+0000 7fcd7b7fe700 1 -- 172.21.15.3:0/3878503740 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd70011d20 con 0x7fcd7c09d6f0 2023-10-03T10:18:02.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.427+0000 7fcd82c18700 1 -- 172.21.15.3:0/3878503740 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd7c09d6f0 msgr2=0x7fcd7c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:02.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.427+0000 7fcd82c18700 1 --2- 172.21.15.3:0/3878503740 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd7c09d6f0 0x7fcd7c09dac0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fcd70009920 tx=0x7fcd70009c30 comp rx=0 tx=0).stop 2023-10-03T10:18:02.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.427+0000 7fcd82c18700 1 -- 172.21.15.3:0/3878503740 shutdown_connections 2023-10-03T10:18:02.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.427+0000 7fcd82c18700 1 --2- 172.21.15.3:0/3878503740 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd7c09d6f0 0x7fcd7c09dac0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.427+0000 7fcd82c18700 1 --2- 172.21.15.3:0/3878503740 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd7c09e000 0x7fcd7c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.427+0000 7fcd82c18700 1 -- 172.21.15.3:0/3878503740 >> 172.21.15.3:0/3878503740 conn(0x7fcd7c066f50 msgr2=0x7fcd7c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:02.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.427+0000 7fcd82c18700 1 -- 172.21.15.3:0/3878503740 shutdown_connections 2023-10-03T10:18:02.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.427+0000 7fcd82c18700 1 -- 172.21.15.3:0/3878503740 wait complete. 2023-10-03T10:18:02.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.428+0000 7fcd82c18700 1 Processor -- start 2023-10-03T10:18:02.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.429+0000 7fcd82c18700 1 -- start start 2023-10-03T10:18:02.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.429+0000 7fcd82c18700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd7c09d6f0 0x7fcd7c1c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:02.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.429+0000 7fcd82c18700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd7c09e000 0x7fcd7c1c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:02.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.429+0000 7fcd82c18700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd7c09e410 0x7fcd7c1c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:02.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.430+0000 7fcd82c18700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c1a3d80 con 0x7fcd7c09e000 2023-10-03T10:18:02.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.430+0000 7fcd82c18700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c1a7110 con 0x7fcd7c09e410 2023-10-03T10:18:02.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.430+0000 7fcd82c18700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c1a5450 con 0x7fcd7c09d6f0 2023-10-03T10:18:02.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.430+0000 7fcd7bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd7c09e000 0x7fcd7c1c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:02.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.430+0000 7fcd809b4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd7c09d6f0 0x7fcd7c1c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:02.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.430+0000 7fcd7bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd7c09e000 0x7fcd7c1c8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44980/0 (socket says 172.21.15.3:44980) 2023-10-03T10:18:02.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.430+0000 7fcd7bfff700 1 -- 172.21.15.3:0/3133950449 learned_addr learned my addr 172.21.15.3:0/3133950449 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:02.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.430+0000 7fcd811b5700 1 --2- 172.21.15.3:0/3133950449 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd7c09e410 0x7fcd7c1c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:02.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.430+0000 7fcd7bfff700 1 -- 172.21.15.3:0/3133950449 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd7c09d6f0 msgr2=0x7fcd7c1c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:02.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.430+0000 7fcd7bfff700 1 --2- 172.21.15.3:0/3133950449 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd7c09d6f0 0x7fcd7c1c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.430+0000 7fcd7bfff700 1 -- 172.21.15.3:0/3133950449 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd7c09e410 msgr2=0x7fcd7c1c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:02.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.430+0000 7fcd7bfff700 1 --2- 172.21.15.3:0/3133950449 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd7c09e410 0x7fcd7c1c8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.430+0000 7fcd7bfff700 1 -- 172.21.15.3:0/3133950449 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd680094a0 con 0x7fcd7c09e000 2023-10-03T10:18:02.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.430+0000 7fcd809b4700 1 --2- 172.21.15.3:0/3133950449 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd7c09d6f0 0x7fcd7c1c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:02.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.430+0000 7fcd811b5700 1 --2- 172.21.15.3:0/3133950449 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd7c09e410 0x7fcd7c1c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:02.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.430+0000 7fcd7bfff700 1 --2- 172.21.15.3:0/3133950449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd7c09e000 0x7fcd7c1c8740 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fcd6800ebb0 tx=0x7fcd6800eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:02.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.431+0000 7fcd797fa700 1 -- 172.21.15.3:0/3133950449 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6800c850 con 0x7fcd7c09e000 2023-10-03T10:18:02.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.431+0000 7fcd797fa700 1 -- 172.21.15.3:0/3133950449 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6800c9b0 con 0x7fcd7c09e000 2023-10-03T10:18:02.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.431+0000 7fcd797fa700 1 -- 172.21.15.3:0/3133950449 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6800cd40 con 0x7fcd7c09e000 2023-10-03T10:18:02.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.431+0000 7fcd82c18700 1 -- 172.21.15.3:0/3133950449 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd70009550 con 0x7fcd7c09e000 2023-10-03T10:18:02.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.431+0000 7fcd82c18700 1 -- 172.21.15.3:0/3133950449 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7c1a5c40 con 0x7fcd7c09e000 2023-10-03T10:18:02.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.432+0000 7fcd82c18700 1 -- 172.21.15.3:0/3133950449 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd7c074ea0 con 0x7fcd7c09e000 2023-10-03T10:18:02.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.432+0000 7fcd797fa700 1 -- 172.21.15.3:0/3133950449 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcd68013470 con 0x7fcd7c09e000 2023-10-03T10:18:02.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.432+0000 7fcd797fa700 1 -- 172.21.15.3:0/3133950449 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fcd6803f790 con 0x7fcd7c09e000 2023-10-03T10:18:02.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.441+0000 7fcd797fa700 1 -- 172.21.15.3:0/3133950449 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcd680232f0 con 0x7fcd7c09e000 2023-10-03T10:18:02.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.465+0000 7f21569d0700 1 Processor -- start 2023-10-03T10:18:02.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.465+0000 7f21569d0700 1 -- start start 2023-10-03T10:18:02.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.466+0000 7f21569d0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f21500a14a0 0x7f215006c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:02.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.466+0000 7f21569d0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2150138cf0 0x7f215013b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:02.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.466+0000 7f21569d0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21500a2a20 con 0x7f21500a09c0 2023-10-03T10:18:02.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.466+0000 7f21569d0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f215009e690 con 0x7f21500a14a0 2023-10-03T10:18:02.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.466+0000 7f21569d0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21500a1cc0 con 0x7f2150138cf0 2023-10-03T10:18:02.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.466+0000 7f214f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f21500a14a0 0x7f215006c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:02.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.466+0000 7f214f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f21500a14a0 0x7f215006c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44984/0 (socket says 172.21.15.3:44984) 2023-10-03T10:18:02.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.466+0000 7f214f7fe700 1 -- 172.21.15.3:0/2410587836 learned_addr learned my addr 172.21.15.3:0/2410587836 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:02.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.466+0000 7f2154f6d700 1 --2- 172.21.15.3:0/2410587836 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2150138cf0 0x7f215013b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:02.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.466+0000 7f214f7fe700 1 -- 172.21.15.3:0/2410587836 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2150138cf0 msgr2=0x7f215013b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:02.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.466+0000 7f214f7fe700 1 --2- 172.21.15.3:0/2410587836 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2150138cf0 0x7f215013b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.466+0000 7f214f7fe700 1 -- 172.21.15.3:0/2410587836 >> v1:172.21.15.3:6789/0 conn(0x7f21500a09c0 legacy=0x7f21500a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:02.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.466+0000 7f214effd700 1 -- 172.21.15.3:0/2410587836 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 434150508 0 0) 0x7f2140001110 con 0x7f21500a09c0 2023-10-03T10:18:02.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.466+0000 7f214f7fe700 1 -- 172.21.15.3:0/2410587836 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21500a1f40 con 0x7f21500a14a0 2023-10-03T10:18:02.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.467+0000 7f214f7fe700 1 --2- 172.21.15.3:0/2410587836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21500a14a0 0x7f215006c5d0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f21440098f0 tx=0x7f2144009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7258caef7d84f544 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:02.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.467+0000 7f214effd700 1 -- 172.21.15.3:0/2410587836 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2144011830 con 0x7f21500a14a0 2023-10-03T10:18:02.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.467+0000 7f214effd700 1 -- 172.21.15.3:0/2410587836 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2144011990 con 0x7f21500a14a0 2023-10-03T10:18:02.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.467+0000 7f214effd700 1 -- 172.21.15.3:0/2410587836 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2144011d20 con 0x7f21500a14a0 2023-10-03T10:18:02.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.468+0000 7f21569d0700 1 -- 172.21.15.3:0/2410587836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21500a14a0 msgr2=0x7f215006c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:02.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.468+0000 7f21569d0700 1 --2- 172.21.15.3:0/2410587836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21500a14a0 0x7f215006c5d0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f21440098f0 tx=0x7f2144009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:02.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.468+0000 7f21569d0700 1 -- 172.21.15.3:0/2410587836 shutdown_connections 2023-10-03T10:18:02.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.468+0000 7f21569d0700 1 --2- 172.21.15.3:0/2410587836 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2150138cf0 0x7f215013b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.468+0000 7f21569d0700 1 --2- 172.21.15.3:0/2410587836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21500a14a0 0x7f215006c5d0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.468+0000 7f21569d0700 1 -- 172.21.15.3:0/2410587836 >> 172.21.15.3:0/2410587836 conn(0x7f21500a60c0 msgr2=0x7f21500a64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:02.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.468+0000 7f21569d0700 1 -- 172.21.15.3:0/2410587836 shutdown_connections 2023-10-03T10:18:02.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.468+0000 7f21569d0700 1 -- 172.21.15.3:0/2410587836 wait complete. 2023-10-03T10:18:02.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.469+0000 7f21569d0700 1 Processor -- start 2023-10-03T10:18:02.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.469+0000 7f21569d0700 1 -- start start 2023-10-03T10:18:02.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.469+0000 7f21569d0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21500a09c0 0x7f2150133840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:02.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.469+0000 7f21569d0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21500a14a0 0x7f2150133d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:02.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.469+0000 7f21569d0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2150138cf0 0x7f21501342c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:02.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.469+0000 7f21569d0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2150135d20 con 0x7f21500a09c0 2023-10-03T10:18:02.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.469+0000 7f21569d0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2150137360 con 0x7f21500a14a0 2023-10-03T10:18:02.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.469+0000 7f21569d0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f21500a3630 con 0x7f2150138cf0 2023-10-03T10:18:02.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.470+0000 7f2154f6d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2150138cf0 0x7f21501342c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:02.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.470+0000 7f2154f6d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2150138cf0 0x7f21501342c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48062/0 (socket says 172.21.15.3:48062) 2023-10-03T10:18:02.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.470+0000 7f2154f6d700 1 -- 172.21.15.3:0/1282964443 learned_addr learned my addr 172.21.15.3:0/1282964443 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:02.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.470+0000 7f214f7fe700 1 --2- 172.21.15.3:0/1282964443 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21500a14a0 0x7f2150133d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:02.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.470+0000 7f214ffff700 1 --2- 172.21.15.3:0/1282964443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21500a09c0 0x7f2150133840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:02.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.470+0000 7f2154f6d700 1 -- 172.21.15.3:0/1282964443 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21500a14a0 msgr2=0x7f2150133d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:02.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.470+0000 7f2154f6d700 1 --2- 172.21.15.3:0/1282964443 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21500a14a0 0x7f2150133d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.470+0000 7f2154f6d700 1 -- 172.21.15.3:0/1282964443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21500a09c0 msgr2=0x7f2150133840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:02.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.470+0000 7f2154f6d700 1 --2- 172.21.15.3:0/1282964443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21500a09c0 0x7f2150133840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.470+0000 7f2154f6d700 1 -- 172.21.15.3:0/1282964443 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2144009550 con 0x7f2150138cf0 2023-10-03T10:18:02.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.470+0000 7f2154f6d700 1 --2- 172.21.15.3:0/1282964443 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2150138cf0 0x7f21501342c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f214800efc0 tx=0x7f214800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:02.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.470+0000 7f214cff9700 1 -- 172.21.15.3:0/1282964443 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214800cc90 con 0x7f2150138cf0 2023-10-03T10:18:02.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.471+0000 7f214cff9700 1 -- 172.21.15.3:0/1282964443 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214800f040 con 0x7f2150138cf0 2023-10-03T10:18:02.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.471+0000 7f21569d0700 1 -- 172.21.15.3:0/1282964443 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2150136dc0 con 0x7f2150138cf0 2023-10-03T10:18:02.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.471+0000 7f214cff9700 1 -- 172.21.15.3:0/1282964443 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2148014480 con 0x7f2150138cf0 2023-10-03T10:18:02.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.471+0000 7f21569d0700 1 -- 172.21.15.3:0/1282964443 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f215009cea0 con 0x7f2150138cf0 2023-10-03T10:18:02.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.472+0000 7f21569d0700 1 -- 172.21.15.3:0/1282964443 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f215004f2a0 con 0x7f2150138cf0 2023-10-03T10:18:02.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.472+0000 7f214cff9700 1 -- 172.21.15.3:0/1282964443 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2148014600 con 0x7f2150138cf0 2023-10-03T10:18:02.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.472+0000 7f214cff9700 1 -- 172.21.15.3:0/1282964443 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f21480406f0 con 0x7f2150138cf0 2023-10-03T10:18:02.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.477+0000 7f214cff9700 1 -- 172.21.15.3:0/1282964443 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2148012930 con 0x7f2150138cf0 2023-10-03T10:18:02.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.615+0000 7f9699fa6700 1 -- 172.21.15.3:0/1531502776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f96941a81e0 con 0x7f969409d6f0 2023-10-03T10:18:02.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.615+0000 7f967bfff700 1 -- 172.21.15.3:0/1531502776 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f96941a81e0 con 0x7f969409d6f0 2023-10-03T10:18:02.616 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:02.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.618+0000 7f9699fa6700 1 -- 172.21.15.3:0/1531502776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f969409d6f0 msgr2=0x7f96941a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:02.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.618+0000 7f9699fa6700 1 --2- 172.21.15.3:0/1531502776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f969409d6f0 0x7f96941a5130 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f9684009ea0 tx=0x7f9684004750 comp rx=0 tx=0).stop 2023-10-03T10:18:02.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.618+0000 7f9699fa6700 1 -- 172.21.15.3:0/1531502776 shutdown_connections 2023-10-03T10:18:02.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.618+0000 7f9699fa6700 1 --2- 172.21.15.3:0/1531502776 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f969409e410 0x7f96941c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.618+0000 7f9699fa6700 1 --2- 172.21.15.3:0/1531502776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f969409d6f0 0x7f96941a5130 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.618+0000 7f9699fa6700 1 --2- 172.21.15.3:0/1531502776 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f969409e000 0x7f96941a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.618+0000 7f9699fa6700 1 -- 172.21.15.3:0/1531502776 >> 172.21.15.3:0/1531502776 conn(0x7f9694066f50 msgr2=0x7f96941bbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:02.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.618+0000 7f9699fa6700 1 -- 172.21.15.3:0/1531502776 shutdown_connections 2023-10-03T10:18:02.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.618+0000 7f9699fa6700 1 -- 172.21.15.3:0/1531502776 wait complete. 2023-10-03T10:18:02.632 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:02.633 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:02.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.652+0000 7fcd82c18700 1 -- 172.21.15.3:0/3133950449 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fcd7c1a6f70 con 0x7fcd7c09e000 2023-10-03T10:18:02.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.653+0000 7fcd797fa700 1 -- 172.21.15.3:0/3133950449 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcd7c1a6f70 con 0x7fcd7c09e000 2023-10-03T10:18:02.653 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:02.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.655+0000 7fcd82c18700 1 -- 172.21.15.3:0/3133950449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd7c09e000 msgr2=0x7fcd7c1c8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:02.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.655+0000 7fcd82c18700 1 --2- 172.21.15.3:0/3133950449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd7c09e000 0x7fcd7c1c8740 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fcd6800ebb0 tx=0x7fcd6800eec0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.655+0000 7fcd82c18700 1 -- 172.21.15.3:0/3133950449 shutdown_connections 2023-10-03T10:18:02.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.655+0000 7fcd82c18700 1 --2- 172.21.15.3:0/3133950449 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcd7c09e410 0x7fcd7c1c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.655+0000 7fcd82c18700 1 --2- 172.21.15.3:0/3133950449 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcd7c09d6f0 0x7fcd7c1c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.655+0000 7fcd82c18700 1 --2- 172.21.15.3:0/3133950449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd7c09e000 0x7fcd7c1c8740 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.655+0000 7fcd82c18700 1 -- 172.21.15.3:0/3133950449 >> 172.21.15.3:0/3133950449 conn(0x7fcd7c066f50 msgr2=0x7fcd7c1bbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:02.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.656+0000 7fcd82c18700 1 -- 172.21.15.3:0/3133950449 shutdown_connections 2023-10-03T10:18:02.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.656+0000 7fcd82c18700 1 -- 172.21.15.3:0/3133950449 wait complete. 2023-10-03T10:18:02.670 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:02.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.684+0000 7f21569d0700 1 -- 172.21.15.3:0/1282964443 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f215009dd50 con 0x7f2150138cf0 2023-10-03T10:18:02.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.685+0000 7f214cff9700 1 -- 172.21.15.3:0/1282964443 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2148012340 con 0x7f2150138cf0 2023-10-03T10:18:02.685 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:02.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.687+0000 7f21569d0700 1 -- 172.21.15.3:0/1282964443 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2150138cf0 msgr2=0x7f21501342c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:02.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.687+0000 7f21569d0700 1 --2- 172.21.15.3:0/1282964443 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2150138cf0 0x7f21501342c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f214800efc0 tx=0x7f214800c3a0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.687+0000 7f21569d0700 1 -- 172.21.15.3:0/1282964443 shutdown_connections 2023-10-03T10:18:02.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.687+0000 7f21569d0700 1 --2- 172.21.15.3:0/1282964443 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2150138cf0 0x7f21501342c0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.687+0000 7f21569d0700 1 --2- 172.21.15.3:0/1282964443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21500a09c0 0x7f2150133840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.687+0000 7f21569d0700 1 --2- 172.21.15.3:0/1282964443 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21500a14a0 0x7f2150133d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.687+0000 7f21569d0700 1 -- 172.21.15.3:0/1282964443 >> 172.21.15.3:0/1282964443 conn(0x7f21500a60c0 msgr2=0x7f215013ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:02.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.688+0000 7f21569d0700 1 -- 172.21.15.3:0/1282964443 shutdown_connections 2023-10-03T10:18:02.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.688+0000 7f21569d0700 1 -- 172.21.15.3:0/1282964443 wait complete. 2023-10-03T10:18:02.702 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:02.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.773+0000 7f38ba48b700 1 Processor -- start 2023-10-03T10:18:02.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.773+0000 7f38ba48b700 1 -- start start 2023-10-03T10:18:02.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.773+0000 7f38ba48b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f38b409e1d0 0x7f38b4135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:02.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.773+0000 7f38ba48b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38b4135e50 0x7f38b4138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:02.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.773+0000 7f38ba48b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b409f130 con 0x7f38b409d6f0 2023-10-03T10:18:02.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.773+0000 7f38ba48b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b409f520 con 0x7f38b409e1d0 2023-10-03T10:18:02.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.773+0000 7f38ba48b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38b409e930 con 0x7f38b4135e50 2023-10-03T10:18:02.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.774+0000 7f38b37fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f38b409e1d0 0x7f38b4135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:02.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.774+0000 7f38b8a28700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38b4135e50 0x7f38b4138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:02.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.774+0000 7f38b37fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f38b409e1d0 0x7f38b4135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45006/0 (socket says 172.21.15.3:45006) 2023-10-03T10:18:02.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.774+0000 7f38b8a28700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38b4135e50 0x7f38b4138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48066/0 (socket says 172.21.15.3:48066) 2023-10-03T10:18:02.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.774+0000 7f38b37fe700 1 -- 172.21.15.3:0/2539034881 learned_addr learned my addr 172.21.15.3:0/2539034881 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:02.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.774+0000 7f38b8a28700 1 -- 172.21.15.3:0/2539034881 >> v2:172.21.15.3:3300/0 conn(0x7f38b409e1d0 msgr2=0x7f38b4135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:02.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.774+0000 7f38b8a28700 1 --2- 172.21.15.3:0/2539034881 >> v2:172.21.15.3:3300/0 conn(0x7f38b409e1d0 0x7f38b4135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.774+0000 7f38b8a28700 1 -- 172.21.15.3:0/2539034881 >> v1:172.21.15.3:6789/0 conn(0x7f38b409d6f0 legacy=0x7f38b409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:02.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.774+0000 7f38b8a28700 1 -- 172.21.15.3:0/2539034881 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b409b2d0 con 0x7f38b4135e50 2023-10-03T10:18:02.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.774+0000 7f38b2ffd700 1 -- 172.21.15.3:0/2539034881 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2320127023 0 0) 0x7f38b409f130 con 0x7f38b409d6f0 2023-10-03T10:18:02.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.774+0000 7f38b37fe700 1 --2- 172.21.15.3:0/2539034881 >> v2:172.21.15.3:3300/0 conn(0x7f38b409e1d0 0x7f38b4135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:02.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.774+0000 7f38b8a28700 1 --2- 172.21.15.3:0/2539034881 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38b4135e50 0x7f38b4138230 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f38ac009d90 tx=0x7f38ac00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90da70b9a6d76c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:02.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.774+0000 7f38b2ffd700 1 -- 172.21.15.3:0/2539034881 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac00e070 con 0x7f38b4135e50 2023-10-03T10:18:02.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.774+0000 7f38b2ffd700 1 -- 172.21.15.3:0/2539034881 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ac00bb20 con 0x7f38b4135e50 2023-10-03T10:18:02.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.774+0000 7f38b2ffd700 1 -- 172.21.15.3:0/2539034881 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac012430 con 0x7f38b4135e50 2023-10-03T10:18:02.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.775+0000 7f38ba48b700 1 -- 172.21.15.3:0/2539034881 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38b4135e50 msgr2=0x7f38b4138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:02.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.775+0000 7f38ba48b700 1 --2- 172.21.15.3:0/2539034881 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38b4135e50 0x7f38b4138230 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f38ac009d90 tx=0x7f38ac00b330 comp rx=0 tx=0).stop 2023-10-03T10:18:02.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.775+0000 7f38ba48b700 1 -- 172.21.15.3:0/2539034881 shutdown_connections 2023-10-03T10:18:02.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.775+0000 7f38ba48b700 1 --2- 172.21.15.3:0/2539034881 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38b4135e50 0x7f38b4138230 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.775+0000 7f38ba48b700 1 --2- 172.21.15.3:0/2539034881 >> v2:172.21.15.3:3300/0 conn(0x7f38b409e1d0 0x7f38b4135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.775+0000 7f38ba48b700 1 -- 172.21.15.3:0/2539034881 >> 172.21.15.3:0/2539034881 conn(0x7f38b4066f50 msgr2=0x7f38b409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:02.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.775+0000 7f38ba48b700 1 -- 172.21.15.3:0/2539034881 shutdown_connections 2023-10-03T10:18:02.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.775+0000 7f38ba48b700 1 -- 172.21.15.3:0/2539034881 wait complete. 2023-10-03T10:18:02.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.776+0000 7f38ba48b700 1 Processor -- start 2023-10-03T10:18:02.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.777+0000 7f38ba48b700 1 -- start start 2023-10-03T10:18:02.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.777+0000 7f38ba48b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38b409d6f0 0x7f38b41c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:02.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.777+0000 7f38ba48b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38b409e1d0 0x7f38b41c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:02.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.777+0000 7f38ba48b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38b4135e50 0x7f38b4120600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:02.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.777+0000 7f38ba48b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b4132f90 con 0x7f38b409e1d0 2023-10-03T10:18:02.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.777+0000 7f38ba48b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b4122210 con 0x7f38b4135e50 2023-10-03T10:18:02.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.777+0000 7f38ba48b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38b41331b0 con 0x7f38b409d6f0 2023-10-03T10:18:02.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.777+0000 7f38b3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38b409d6f0 0x7f38b41c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:02.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.777+0000 7f38b8a28700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38b4135e50 0x7f38b4120600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:02.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.777+0000 7f38b3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38b409d6f0 0x7f38b41c8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48082/0 (socket says 172.21.15.3:48082) 2023-10-03T10:18:02.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.777+0000 7f38b3fff700 1 -- 172.21.15.3:0/541160012 learned_addr learned my addr 172.21.15.3:0/541160012 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:02.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.777+0000 7f38b37fe700 1 --2- 172.21.15.3:0/541160012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38b409e1d0 0x7f38b41c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:02.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.778+0000 7f38b3fff700 1 -- 172.21.15.3:0/541160012 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38b4135e50 msgr2=0x7f38b4120600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:02.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.778+0000 7f38b3fff700 1 --2- 172.21.15.3:0/541160012 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38b4135e50 0x7f38b4120600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.778+0000 7f38b3fff700 1 -- 172.21.15.3:0/541160012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38b409e1d0 msgr2=0x7f38b41c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:02.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.778+0000 7f38b3fff700 1 --2- 172.21.15.3:0/541160012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38b409e1d0 0x7f38b41c8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.778+0000 7f38b3fff700 1 -- 172.21.15.3:0/541160012 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a400b7f0 con 0x7f38b409d6f0 2023-10-03T10:18:02.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.778+0000 7f38b3fff700 1 --2- 172.21.15.3:0/541160012 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38b409d6f0 0x7f38b41c8410 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f38a8001eb0 tx=0x7f38a800c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:02.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.778+0000 7f38b0ff9700 1 -- 172.21.15.3:0/541160012 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a80054e0 con 0x7f38b409d6f0 2023-10-03T10:18:02.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.778+0000 7f38b0ff9700 1 -- 172.21.15.3:0/541160012 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a8003680 con 0x7f38b409d6f0 2023-10-03T10:18:02.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.778+0000 7f38ba48b700 1 -- 172.21.15.3:0/541160012 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ac009a70 con 0x7f38b409d6f0 2023-10-03T10:18:02.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.778+0000 7f38b0ff9700 1 -- 172.21.15.3:0/541160012 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a80039f0 con 0x7f38b409d6f0 2023-10-03T10:18:02.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.778+0000 7f38ba48b700 1 -- 172.21.15.3:0/541160012 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b41c8e90 con 0x7f38b409d6f0 2023-10-03T10:18:02.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.779+0000 7f38b0ff9700 1 -- 172.21.15.3:0/541160012 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f38a8003b50 con 0x7f38b409d6f0 2023-10-03T10:18:02.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.779+0000 7f38b0ff9700 1 -- 172.21.15.3:0/541160012 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f38a803ea70 con 0x7f38b409d6f0 2023-10-03T10:18:02.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.779+0000 7f38ba48b700 1 -- 172.21.15.3:0/541160012 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b4123c40 con 0x7f38b409d6f0 2023-10-03T10:18:02.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.784+0000 7f38b0ff9700 1 -- 172.21.15.3:0/541160012 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f38a8017020 con 0x7f38b409d6f0 2023-10-03T10:18:02.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.987+0000 7f38ba48b700 1 -- 172.21.15.3:0/541160012 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f38b4074ea0 con 0x7f38b409d6f0 2023-10-03T10:18:02.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.987+0000 7f38b0ff9700 1 -- 172.21.15.3:0/541160012 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f38a801b3b0 con 0x7f38b409d6f0 2023-10-03T10:18:02.988 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:02.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.989+0000 7f38ba48b700 1 -- 172.21.15.3:0/541160012 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38b409d6f0 msgr2=0x7f38b41c8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:02.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.989+0000 7f38ba48b700 1 --2- 172.21.15.3:0/541160012 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38b409d6f0 0x7f38b41c8410 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f38a8001eb0 tx=0x7f38a800c800 comp rx=0 tx=0).stop 2023-10-03T10:18:02.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.990+0000 7f38ba48b700 1 -- 172.21.15.3:0/541160012 shutdown_connections 2023-10-03T10:18:02.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.990+0000 7f38ba48b700 1 --2- 172.21.15.3:0/541160012 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38b4135e50 0x7f38b4120600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.990+0000 7f38ba48b700 1 --2- 172.21.15.3:0/541160012 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38b409d6f0 0x7f38b41c8410 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.990+0000 7f38ba48b700 1 --2- 172.21.15.3:0/541160012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38b409e1d0 0x7f38b41c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:02.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.990+0000 7f38ba48b700 1 -- 172.21.15.3:0/541160012 >> 172.21.15.3:0/541160012 conn(0x7f38b4066f50 msgr2=0x7f38b406edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:02.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.990+0000 7f38ba48b700 1 -- 172.21.15.3:0/541160012 shutdown_connections 2023-10-03T10:18:02.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:02.990+0000 7f38ba48b700 1 -- 172.21.15.3:0/541160012 wait complete. 2023-10-03T10:18:03.004 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:03.633 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:03.672 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:03.703 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:03.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.791+0000 7f285a679700 1 Processor -- start 2023-10-03T10:18:03.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.791+0000 7f285a679700 1 -- start start 2023-10-03T10:18:03.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.792+0000 7f285a679700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285409d6f0 0x7f285409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:03.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.792+0000 7f285a679700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f285409e410 0x7f2854134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:03.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.792+0000 7f285a679700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f285409b120 con 0x7f285409e000 2023-10-03T10:18:03.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.792+0000 7f285a679700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f285406b460 con 0x7f285409e410 2023-10-03T10:18:03.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.792+0000 7f285a679700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f285409a7e0 con 0x7f285409d6f0 2023-10-03T10:18:03.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.792+0000 7f2853fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285409d6f0 0x7f285409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:03.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.792+0000 7f28537fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f285409e000 0x7f28541339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50500/0 (socket says 172.21.15.3:50500) 2023-10-03T10:18:03.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.792+0000 7f28537fe700 1 -- 172.21.15.3:0/2848017917 learned_addr learned my addr 172.21.15.3:0/2848017917 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:03.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.792+0000 7f2858c16700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f285409e410 0x7f2854134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:03.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.792+0000 7f2853fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285409d6f0 0x7f285409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41770/0 (socket says 172.21.15.3:41770) 2023-10-03T10:18:03.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.793+0000 7f2858c16700 1 -- 172.21.15.3:0/2848017917 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285409d6f0 msgr2=0x7f285409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:03.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.793+0000 7f2858c16700 1 --2- 172.21.15.3:0/2848017917 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285409d6f0 0x7f285409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:03.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.793+0000 7f2858c16700 1 -- 172.21.15.3:0/2848017917 >> v1:172.21.15.3:6789/0 conn(0x7f285409e000 legacy=0x7f28541339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:03.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.793+0000 7f2852ffd700 1 -- 172.21.15.3:0/2848017917 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2621625241 0 0) 0x7f285409b120 con 0x7f285409e000 2023-10-03T10:18:03.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.793+0000 7f2858c16700 1 -- 172.21.15.3:0/2848017917 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f285409a500 con 0x7f285409e410 2023-10-03T10:18:03.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.793+0000 7f2858c16700 1 --2- 172.21.15.3:0/2848017917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285409e410 0x7f2854134100 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f284c009d90 tx=0x7f284c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a53f0a41069e540 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:03.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.793+0000 7f2852ffd700 1 -- 172.21.15.3:0/2848017917 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c00e070 con 0x7f285409e410 2023-10-03T10:18:03.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.793+0000 7f2852ffd700 1 -- 172.21.15.3:0/2848017917 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284c00bb20 con 0x7f285409e410 2023-10-03T10:18:03.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.793+0000 7f2852ffd700 1 -- 172.21.15.3:0/2848017917 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c012430 con 0x7f285409e410 2023-10-03T10:18:03.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.794+0000 7f285a679700 1 -- 172.21.15.3:0/2848017917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285409e410 msgr2=0x7f2854134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:03.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.794+0000 7f285a679700 1 --2- 172.21.15.3:0/2848017917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285409e410 0x7f2854134100 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f284c009d90 tx=0x7f284c00b330 comp rx=0 tx=0).stop 2023-10-03T10:18:03.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.794+0000 7f285a679700 1 -- 172.21.15.3:0/2848017917 shutdown_connections 2023-10-03T10:18:03.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.794+0000 7f285a679700 1 --2- 172.21.15.3:0/2848017917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285409e410 0x7f2854134100 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:03.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.794+0000 7f285a679700 1 --2- 172.21.15.3:0/2848017917 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285409d6f0 0x7f285409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:03.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.794+0000 7f285a679700 1 -- 172.21.15.3:0/2848017917 >> 172.21.15.3:0/2848017917 conn(0x7f2854066f50 msgr2=0x7f285409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:03.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.794+0000 7f285a679700 1 -- 172.21.15.3:0/2848017917 shutdown_connections 2023-10-03T10:18:03.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.794+0000 7f285a679700 1 -- 172.21.15.3:0/2848017917 wait complete. 2023-10-03T10:18:03.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.796+0000 7f285a679700 1 Processor -- start 2023-10-03T10:18:03.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.796+0000 7f285a679700 1 -- start start 2023-10-03T10:18:03.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.797+0000 7f285a679700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285409d6f0 0x7f28541a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:03.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.797+0000 7f285a679700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285409e000 0x7f28541a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:03.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.797+0000 7f285a679700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f285409e410 0x7f28541c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:03.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.797+0000 7f285a679700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28541a8120 con 0x7f285409d6f0 2023-10-03T10:18:03.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.797+0000 7f285a679700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28541a8300 con 0x7f285409e000 2023-10-03T10:18:03.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.797+0000 7f285a679700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28541b9170 con 0x7f285409e410 2023-10-03T10:18:03.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.797+0000 7f28537fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285409e000 0x7f28541a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:03.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.797+0000 7f2853fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285409d6f0 0x7f28541a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:03.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.797+0000 7f28537fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285409e000 0x7f28541a5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41786/0 (socket says 172.21.15.3:41786) 2023-10-03T10:18:03.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.797+0000 7f2853fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285409d6f0 0x7f28541a5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46538/0 (socket says 172.21.15.3:46538) 2023-10-03T10:18:03.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.797+0000 7f28537fe700 1 -- 172.21.15.3:0/3227185051 learned_addr learned my addr 172.21.15.3:0/3227185051 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:03.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.797+0000 7f2858c16700 1 --2- 172.21.15.3:0/3227185051 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f285409e410 0x7f28541c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:03.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.798+0000 7f2853fff700 1 -- 172.21.15.3:0/3227185051 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f285409e410 msgr2=0x7f28541c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:03.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.798+0000 7f2853fff700 1 --2- 172.21.15.3:0/3227185051 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f285409e410 0x7f28541c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:03.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.798+0000 7f2853fff700 1 -- 172.21.15.3:0/3227185051 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285409e000 msgr2=0x7f28541a5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:03.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.798+0000 7f2853fff700 1 --2- 172.21.15.3:0/3227185051 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285409e000 0x7f28541a5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:03.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.798+0000 7f2853fff700 1 -- 172.21.15.3:0/3227185051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f284c009a70 con 0x7f285409d6f0 2023-10-03T10:18:03.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.798+0000 7f28537fe700 1 --2- 172.21.15.3:0/3227185051 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285409e000 0x7f28541a5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:03.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.798+0000 7f2858c16700 1 --2- 172.21.15.3:0/3227185051 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f285409e410 0x7f28541c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:03.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.798+0000 7f2853fff700 1 --2- 172.21.15.3:0/3227185051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285409d6f0 0x7f28541a5240 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f284400e930 tx=0x7f284400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:03.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.798+0000 7f2850ff9700 1 -- 172.21.15.3:0/3227185051 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2844013680 con 0x7f285409d6f0 2023-10-03T10:18:03.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.798+0000 7f2850ff9700 1 -- 172.21.15.3:0/3227185051 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28440137e0 con 0x7f285409d6f0 2023-10-03T10:18:03.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.798+0000 7f285a679700 1 -- 172.21.15.3:0/3227185051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28541a4ca0 con 0x7f285409d6f0 2023-10-03T10:18:03.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.798+0000 7f2850ff9700 1 -- 172.21.15.3:0/3227185051 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2844013b50 con 0x7f285409d6f0 2023-10-03T10:18:03.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.798+0000 7f285a679700 1 -- 172.21.15.3:0/3227185051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28541a7b00 con 0x7f285409d6f0 2023-10-03T10:18:03.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.799+0000 7f2850ff9700 1 -- 172.21.15.3:0/3227185051 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2844013cb0 con 0x7f285409d6f0 2023-10-03T10:18:03.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.799+0000 7f2850ff9700 1 -- 172.21.15.3:0/3227185051 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f284403f7f0 con 0x7f285409d6f0 2023-10-03T10:18:03.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.799+0000 7f285a679700 1 -- 172.21.15.3:0/3227185051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28541a5cc0 con 0x7f285409d6f0 2023-10-03T10:18:03.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.804+0000 7f2850ff9700 1 -- 172.21.15.3:0/3227185051 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f284401be10 con 0x7f285409d6f0 2023-10-03T10:18:03.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.834+0000 7febb0aec700 1 Processor -- start 2023-10-03T10:18:03.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.834+0000 7febb0aec700 1 -- start start 2023-10-03T10:18:03.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.835+0000 7febb0aec700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7febac09f410 0x7febac136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:03.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.835+0000 7febb0aec700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febac136fe0 0x7febac1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:03.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.835+0000 7febb0aec700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febac070940 con 0x7febac09e930 2023-10-03T10:18:03.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.835+0000 7febb0aec700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febac06fd60 con 0x7febac09f410 2023-10-03T10:18:03.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.835+0000 7febb0aec700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7febac06fa80 con 0x7febac136fe0 2023-10-03T10:18:03.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.835+0000 7feba9d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7febac09f410 0x7febac136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:03.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.835+0000 7feba9d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7febac09f410 0x7febac136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46542/0 (socket says 172.21.15.3:46542) 2023-10-03T10:18:03.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.835+0000 7febaa59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7febac09e930 0x7febac09ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50512/0 (socket says 172.21.15.3:50512) 2023-10-03T10:18:03.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.835+0000 7feba9d9b700 1 -- 172.21.15.3:0/4196334517 learned_addr learned my addr 172.21.15.3:0/4196334517 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:03.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.835+0000 7febaad9d700 1 --2- 172.21.15.3:0/4196334517 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febac136fe0 0x7febac1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:03.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.835+0000 7feba9d9b700 1 -- 172.21.15.3:0/4196334517 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febac136fe0 msgr2=0x7febac1393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:03.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.835+0000 7feba9d9b700 1 --2- 172.21.15.3:0/4196334517 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febac136fe0 0x7febac1393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:03.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.835+0000 7feba9d9b700 1 -- 172.21.15.3:0/4196334517 >> v1:172.21.15.3:6789/0 conn(0x7febac09e930 legacy=0x7febac09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:03.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.835+0000 7feba9d9b700 1 -- 172.21.15.3:0/4196334517 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febac09fb60 con 0x7febac09f410 2023-10-03T10:18:03.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.835+0000 7feba959a700 1 -- 172.21.15.3:0/4196334517 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1146799497 0 0) 0x7febac070940 con 0x7febac09e930 2023-10-03T10:18:03.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.836+0000 7feba9d9b700 1 --2- 172.21.15.3:0/4196334517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febac09f410 0x7febac136aa0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7feba00098f0 tx=0x7feba0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337006ab31cc267d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:03.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.836+0000 7feba959a700 1 -- 172.21.15.3:0/4196334517 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba0011830 con 0x7febac09f410 2023-10-03T10:18:03.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.836+0000 7feba959a700 1 -- 172.21.15.3:0/4196334517 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba0011990 con 0x7febac09f410 2023-10-03T10:18:03.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.836+0000 7feba959a700 1 -- 172.21.15.3:0/4196334517 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba0011d20 con 0x7febac09f410 2023-10-03T10:18:03.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.837+0000 7febb0aec700 1 -- 172.21.15.3:0/4196334517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febac09f410 msgr2=0x7febac136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:03.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.837+0000 7febb0aec700 1 --2- 172.21.15.3:0/4196334517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febac09f410 0x7febac136aa0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7feba00098f0 tx=0x7feba0009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:03.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.837+0000 7febb0aec700 1 -- 172.21.15.3:0/4196334517 shutdown_connections 2023-10-03T10:18:03.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.837+0000 7febb0aec700 1 --2- 172.21.15.3:0/4196334517 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febac136fe0 0x7febac1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:03.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.837+0000 7febb0aec700 1 --2- 172.21.15.3:0/4196334517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febac09f410 0x7febac136aa0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:03.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.837+0000 7febb0aec700 1 -- 172.21.15.3:0/4196334517 >> 172.21.15.3:0/4196334517 conn(0x7febac0a4210 msgr2=0x7febac0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:03.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.837+0000 7febb0aec700 1 -- 172.21.15.3:0/4196334517 shutdown_connections 2023-10-03T10:18:03.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.837+0000 7febb0aec700 1 -- 172.21.15.3:0/4196334517 wait complete. 2023-10-03T10:18:03.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.838+0000 7febb0aec700 1 Processor -- start 2023-10-03T10:18:03.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.839+0000 7febb0aec700 1 -- start start 2023-10-03T10:18:03.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.839+0000 7febb0aec700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febac09e930 0x7febac1c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:03.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.839+0000 7febb0aec700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7febac09f410 0x7febac1c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:03.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.839+0000 7febb0aec700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febac136fe0 0x7febac06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:03.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.839+0000 7febb0aec700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac124c00 con 0x7febac09e930 2023-10-03T10:18:03.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.839+0000 7febb0aec700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febac135250 con 0x7febac09f410 2023-10-03T10:18:03.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.839+0000 7febb0aec700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7febac06d640 con 0x7febac136fe0 2023-10-03T10:18:03.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.840+0000 7feba9d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7febac09f410 0x7febac1c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:03.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.840+0000 7febaa59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febac09e930 0x7febac1c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:03.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.840+0000 7feba9d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7febac09f410 0x7febac1c8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41792/0 (socket says 172.21.15.3:41792) 2023-10-03T10:18:03.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.840+0000 7feba9d9b700 1 -- 172.21.15.3:0/1002202453 learned_addr learned my addr 172.21.15.3:0/1002202453 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:03.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.840+0000 7febaad9d700 1 --2- 172.21.15.3:0/1002202453 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febac136fe0 0x7febac06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:03.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.840+0000 7feba9d9b700 1 -- 172.21.15.3:0/1002202453 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febac136fe0 msgr2=0x7febac06b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:03.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.840+0000 7feba9d9b700 1 --2- 172.21.15.3:0/1002202453 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febac136fe0 0x7febac06b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:03.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.840+0000 7feba9d9b700 1 -- 172.21.15.3:0/1002202453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febac09e930 msgr2=0x7febac1c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:03.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.840+0000 7feba9d9b700 1 --2- 172.21.15.3:0/1002202453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febac09e930 0x7febac1c8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:03.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.840+0000 7feba9d9b700 1 -- 172.21.15.3:0/1002202453 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba0009550 con 0x7febac09f410 2023-10-03T10:18:03.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.840+0000 7febaa59c700 1 --2- 172.21.15.3:0/1002202453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febac09e930 0x7febac1c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:03.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.840+0000 7feba9d9b700 1 --2- 172.21.15.3:0/1002202453 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7febac09f410 0x7febac1c8a60 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7febac06fea0 tx=0x7feba0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:03.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.840+0000 7feb9affd700 1 -- 172.21.15.3:0/1002202453 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba0004300 con 0x7febac09f410 2023-10-03T10:18:03.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.840+0000 7feb9affd700 1 -- 172.21.15.3:0/1002202453 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba001f070 con 0x7febac09f410 2023-10-03T10:18:03.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.840+0000 7feb9affd700 1 -- 172.21.15.3:0/1002202453 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba00036a0 con 0x7febac09f410 2023-10-03T10:18:03.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.840+0000 7febb0aec700 1 -- 172.21.15.3:0/1002202453 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febac09dec0 con 0x7febac09f410 2023-10-03T10:18:03.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.841+0000 7febb0aec700 1 -- 172.21.15.3:0/1002202453 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febac071d20 con 0x7febac09f410 2023-10-03T10:18:03.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.841+0000 7febb0aec700 1 -- 172.21.15.3:0/1002202453 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febac074ea0 con 0x7febac09f410 2023-10-03T10:18:03.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.842+0000 7feb9affd700 1 -- 172.21.15.3:0/1002202453 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feba0021030 con 0x7febac09f410 2023-10-03T10:18:03.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.842+0000 7feb9affd700 1 -- 172.21.15.3:0/1002202453 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7feba0040a70 con 0x7febac09f410 2023-10-03T10:18:03.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.851+0000 7feb9affd700 1 -- 172.21.15.3:0/1002202453 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feba0042e60 con 0x7febac09f410 2023-10-03T10:18:03.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.865+0000 7fd1baa1d700 1 Processor -- start 2023-10-03T10:18:03.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.865+0000 7fd1baa1d700 1 -- start start 2023-10-03T10:18:03.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.866+0000 7fd1baa1d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1b409ac90 0x7fd1b413cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:03.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.866+0000 7fd1baa1d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd1b413d240 0x7fd1b413f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:03.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.866+0000 7fd1baa1d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1b4134980 con 0x7fd1b406c0e0 2023-10-03T10:18:03.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.866+0000 7fd1baa1d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1b406c4b0 con 0x7fd1b413d240 2023-10-03T10:18:03.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.866+0000 7fd1baa1d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b409b0e0 con 0x7fd1b409ac90 2023-10-03T10:18:03.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.866+0000 7fd1b8fba700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd1b413d240 0x7fd1b413f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.866+0000 7fd1b8fba700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd1b413d240 0x7fd1b413f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46548/0 (socket says 172.21.15.3:46548) 2023-10-03T10:18:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.866+0000 7fd1b3fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd1b406c0e0 0x7fd1b4134ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50526/0 (socket says 172.21.15.3:50526) 2023-10-03T10:18:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.866+0000 7fd1b8fba700 1 -- 172.21.15.3:0/1023910382 learned_addr learned my addr 172.21.15.3:0/1023910382 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.866+0000 7fd1b37fe700 1 --2- 172.21.15.3:0/1023910382 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1b409ac90 0x7fd1b413cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.866+0000 7fd1b37fe700 1 -- 172.21.15.3:0/1023910382 >> v2:172.21.15.3:3300/0 conn(0x7fd1b413d240 msgr2=0x7fd1b413f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.866+0000 7fd1b37fe700 1 --2- 172.21.15.3:0/1023910382 >> v2:172.21.15.3:3300/0 conn(0x7fd1b413d240 0x7fd1b413f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.866+0000 7fd1b37fe700 1 -- 172.21.15.3:0/1023910382 >> v1:172.21.15.3:6789/0 conn(0x7fd1b406c0e0 legacy=0x7fd1b4134ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.866+0000 7fd1b37fe700 1 -- 172.21.15.3:0/1023910382 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b40a0600 con 0x7fd1b409ac90 2023-10-03T10:18:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.866+0000 7fd1b2ffd700 1 -- 172.21.15.3:0/1023910382 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 624324596 0 0) 0x7fd1b4134980 con 0x7fd1b406c0e0 2023-10-03T10:18:03.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.866+0000 7fd1b8fba700 1 --2- 172.21.15.3:0/1023910382 >> v2:172.21.15.3:3300/0 conn(0x7fd1b413d240 0x7fd1b413f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:03.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.866+0000 7fd1b37fe700 1 --2- 172.21.15.3:0/1023910382 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1b409ac90 0x7fd1b413cd00 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009840 tx=0x7fd1a8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3609542064bfad9f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:03.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.867+0000 7fd1b2ffd700 1 -- 172.21.15.3:0/1023910382 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8011830 con 0x7fd1b409ac90 2023-10-03T10:18:03.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.867+0000 7fd1b2ffd700 1 -- 172.21.15.3:0/1023910382 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a8011990 con 0x7fd1b409ac90 2023-10-03T10:18:03.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.867+0000 7fd1b2ffd700 1 -- 172.21.15.3:0/1023910382 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8011d20 con 0x7fd1b409ac90 2023-10-03T10:18:03.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.867+0000 7fd1baa1d700 1 -- 172.21.15.3:0/1023910382 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1b409ac90 msgr2=0x7fd1b413cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:03.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.867+0000 7fd1baa1d700 1 --2- 172.21.15.3:0/1023910382 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1b409ac90 0x7fd1b413cd00 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009840 tx=0x7fd1a8009b50 comp rx=0 tx=0).stop 2023-10-03T10:18:03.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.867+0000 7fd1baa1d700 1 -- 172.21.15.3:0/1023910382 shutdown_connections 2023-10-03T10:18:03.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.867+0000 7fd1baa1d700 1 --2- 172.21.15.3:0/1023910382 >> v2:172.21.15.3:3300/0 conn(0x7fd1b413d240 0x7fd1b413f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:03.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.867+0000 7fd1baa1d700 1 --2- 172.21.15.3:0/1023910382 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1b409ac90 0x7fd1b413cd00 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:03.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.867+0000 7fd1baa1d700 1 -- 172.21.15.3:0/1023910382 >> 172.21.15.3:0/1023910382 conn(0x7fd1b40a4210 msgr2=0x7fd1b40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:03.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.867+0000 7fd1baa1d700 1 -- 172.21.15.3:0/1023910382 shutdown_connections 2023-10-03T10:18:03.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.868+0000 7fd1baa1d700 1 -- 172.21.15.3:0/1023910382 wait complete. 2023-10-03T10:18:03.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.868+0000 7fd1baa1d700 1 Processor -- start 2023-10-03T10:18:03.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.869+0000 7fd1baa1d700 1 -- start start 2023-10-03T10:18:03.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.869+0000 7fd1baa1d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd1b406c0e0 0x7fd1b409b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:03.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.869+0000 7fd1baa1d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1b409ac90 0x7fd1b409bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:03.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.869+0000 7fd1baa1d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1b413d240 0x7fd1b409c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:03.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.869+0000 7fd1baa1d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b413cfb0 con 0x7fd1b413d240 2023-10-03T10:18:03.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.869+0000 7fd1baa1d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b409d580 con 0x7fd1b409ac90 2023-10-03T10:18:03.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.869+0000 7fd1baa1d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b411b1a0 con 0x7fd1b406c0e0 2023-10-03T10:18:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.869+0000 7fd1b37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1b409ac90 0x7fd1b409bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.869+0000 7fd1b37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1b409ac90 0x7fd1b409bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41816/0 (socket says 172.21.15.3:41816) 2023-10-03T10:18:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.869+0000 7fd1b37fe700 1 -- 172.21.15.3:0/2769183718 learned_addr learned my addr 172.21.15.3:0/2769183718 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.869+0000 7fd1b3fff700 1 --2- 172.21.15.3:0/2769183718 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd1b406c0e0 0x7fd1b409b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.869+0000 7fd1b8fba700 1 --2- 172.21.15.3:0/2769183718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1b413d240 0x7fd1b409c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.870+0000 7fd1b37fe700 1 -- 172.21.15.3:0/2769183718 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd1b406c0e0 msgr2=0x7fd1b409b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.870+0000 7fd1b37fe700 1 --2- 172.21.15.3:0/2769183718 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd1b406c0e0 0x7fd1b409b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.870+0000 7fd1b37fe700 1 -- 172.21.15.3:0/2769183718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1b413d240 msgr2=0x7fd1b409c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.870+0000 7fd1b37fe700 1 --2- 172.21.15.3:0/2769183718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1b413d240 0x7fd1b409c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.870+0000 7fd1b37fe700 1 -- 172.21.15.3:0/2769183718 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ac009b30 con 0x7fd1b409ac90 2023-10-03T10:18:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.870+0000 7fd1b37fe700 1 --2- 172.21.15.3:0/2769183718 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1b409ac90 0x7fd1b409bdf0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8000c00 tx=0x7fd1a8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.870+0000 7fd1b0ff9700 1 -- 172.21.15.3:0/2769183718 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8004550 con 0x7fd1b409ac90 2023-10-03T10:18:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.870+0000 7fd1b0ff9700 1 -- 172.21.15.3:0/2769183718 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a801e070 con 0x7fd1b409ac90 2023-10-03T10:18:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.870+0000 7fd1b0ff9700 1 -- 172.21.15.3:0/2769183718 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a80038e0 con 0x7fd1b409ac90 2023-10-03T10:18:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.870+0000 7fd1baa1d700 1 -- 172.21.15.3:0/2769183718 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a80094a0 con 0x7fd1b409ac90 2023-10-03T10:18:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.870+0000 7fd1baa1d700 1 -- 172.21.15.3:0/2769183718 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b409dc60 con 0x7fd1b409ac90 2023-10-03T10:18:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.871+0000 7fd1b0ff9700 1 -- 172.21.15.3:0/2769183718 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd1a8003a40 con 0x7fd1b409ac90 2023-10-03T10:18:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.871+0000 7fd1baa1d700 1 -- 172.21.15.3:0/2769183718 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1b40a03f0 con 0x7fd1b409ac90 2023-10-03T10:18:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.872+0000 7fd1b0ff9700 1 -- 172.21.15.3:0/2769183718 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd1a8020070 con 0x7fd1b409ac90 2023-10-03T10:18:03.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:03.881+0000 7fd1b0ff9700 1 -- 172.21.15.3:0/2769183718 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd1a8024c00 con 0x7fd1b409ac90 2023-10-03T10:18:04.004 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:04.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.026+0000 7f285a679700 1 -- 172.21.15.3:0/3227185051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2854074ea0 con 0x7f285409d6f0 2023-10-03T10:18:04.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.026+0000 7f2850ff9700 1 -- 172.21.15.3:0/3227185051 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f284402a090 con 0x7f285409d6f0 2023-10-03T10:18:04.026 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:04.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.028+0000 7f285a679700 1 -- 172.21.15.3:0/3227185051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285409d6f0 msgr2=0x7f28541a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:04.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.028+0000 7f285a679700 1 --2- 172.21.15.3:0/3227185051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285409d6f0 0x7f28541a5240 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f284400e930 tx=0x7f284400ec40 comp rx=0 tx=0).stop 2023-10-03T10:18:04.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.029+0000 7f285a679700 1 -- 172.21.15.3:0/3227185051 shutdown_connections 2023-10-03T10:18:04.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.029+0000 7f285a679700 1 --2- 172.21.15.3:0/3227185051 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f285409e410 0x7f28541c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:04.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.029+0000 7f285a679700 1 --2- 172.21.15.3:0/3227185051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285409d6f0 0x7f28541a5240 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:04.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.029+0000 7f285a679700 1 --2- 172.21.15.3:0/3227185051 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f285409e000 0x7f28541a5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:04.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.029+0000 7f285a679700 1 -- 172.21.15.3:0/3227185051 >> 172.21.15.3:0/3227185051 conn(0x7f2854066f50 msgr2=0x7f285406f070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:04.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.029+0000 7f285a679700 1 -- 172.21.15.3:0/3227185051 shutdown_connections 2023-10-03T10:18:04.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.029+0000 7f285a679700 1 -- 172.21.15.3:0/3227185051 wait complete. 2023-10-03T10:18:04.044 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:04.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.065+0000 7febb0aec700 1 -- 172.21.15.3:0/1002202453 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7febac123aa0 con 0x7febac09f410 2023-10-03T10:18:04.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.065+0000 7feb9affd700 1 -- 172.21.15.3:0/1002202453 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feba00199a0 con 0x7febac09f410 2023-10-03T10:18:04.066 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:04.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.068+0000 7febb0aec700 1 -- 172.21.15.3:0/1002202453 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7febac09f410 msgr2=0x7febac1c8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:04.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.068+0000 7febb0aec700 1 --2- 172.21.15.3:0/1002202453 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7febac09f410 0x7febac1c8a60 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7febac06fea0 tx=0x7feba0004750 comp rx=0 tx=0).stop 2023-10-03T10:18:04.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.068+0000 7febb0aec700 1 -- 172.21.15.3:0/1002202453 shutdown_connections 2023-10-03T10:18:04.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.068+0000 7febb0aec700 1 --2- 172.21.15.3:0/1002202453 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febac136fe0 0x7febac06b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:04.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.068+0000 7febb0aec700 1 --2- 172.21.15.3:0/1002202453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febac09e930 0x7febac1c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:04.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.068+0000 7febb0aec700 1 --2- 172.21.15.3:0/1002202453 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7febac09f410 0x7febac1c8a60 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:04.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.068+0000 7febb0aec700 1 -- 172.21.15.3:0/1002202453 >> 172.21.15.3:0/1002202453 conn(0x7febac0a4210 msgr2=0x7febac09c820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:04.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.068+0000 7febb0aec700 1 -- 172.21.15.3:0/1002202453 shutdown_connections 2023-10-03T10:18:04.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.068+0000 7febb0aec700 1 -- 172.21.15.3:0/1002202453 wait complete. 2023-10-03T10:18:04.082 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:04.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.093+0000 7fd1baa1d700 1 -- 172.21.15.3:0/2769183718 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd1b4074ea0 con 0x7fd1b409ac90 2023-10-03T10:18:04.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.093+0000 7fd1b0ff9700 1 -- 172.21.15.3:0/2769183718 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd1a802a070 con 0x7fd1b409ac90 2023-10-03T10:18:04.093 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:04.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.095+0000 7fd1baa1d700 1 -- 172.21.15.3:0/2769183718 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1b409ac90 msgr2=0x7fd1b409bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:04.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.095+0000 7fd1baa1d700 1 --2- 172.21.15.3:0/2769183718 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1b409ac90 0x7fd1b409bdf0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8000c00 tx=0x7fd1a8004750 comp rx=0 tx=0).stop 2023-10-03T10:18:04.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.095+0000 7fd1baa1d700 1 -- 172.21.15.3:0/2769183718 shutdown_connections 2023-10-03T10:18:04.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.095+0000 7fd1baa1d700 1 --2- 172.21.15.3:0/2769183718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1b413d240 0x7fd1b409c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:04.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.095+0000 7fd1baa1d700 1 --2- 172.21.15.3:0/2769183718 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd1b406c0e0 0x7fd1b409b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:04.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.095+0000 7fd1baa1d700 1 --2- 172.21.15.3:0/2769183718 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd1b409ac90 0x7fd1b409bdf0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:04.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.095+0000 7fd1baa1d700 1 -- 172.21.15.3:0/2769183718 >> 172.21.15.3:0/2769183718 conn(0x7fd1b40a4210 msgr2=0x7fd1b40a1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:04.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.096+0000 7fd1baa1d700 1 -- 172.21.15.3:0/2769183718 shutdown_connections 2023-10-03T10:18:04.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.096+0000 7fd1baa1d700 1 -- 172.21.15.3:0/2769183718 wait complete. 2023-10-03T10:18:04.109 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:04.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.148+0000 7f8565c5c700 1 Processor -- start 2023-10-03T10:18:04.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.148+0000 7f8565c5c700 1 -- start start 2023-10-03T10:18:04.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.149+0000 7f8565c5c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85600a0a30 0x7f85600a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:04.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.149+0000 7f8565c5c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85600a1340 0x7f8560133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:04.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.149+0000 7f8565c5c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f856009edd0 con 0x7f8560133fa0 2023-10-03T10:18:04.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.149+0000 7f8565c5c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f856009e1f0 con 0x7f85600a0a30 2023-10-03T10:18:04.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.149+0000 7f8565c5c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f856009dc50 con 0x7f85600a1340 2023-10-03T10:18:04.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.149+0000 7f855f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85600a0a30 0x7f85600a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:04.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.149+0000 7f855effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85600a1340 0x7f8560133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:04.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.149+0000 7f855f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85600a0a30 0x7f85600a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41828/0 (socket says 172.21.15.3:41828) 2023-10-03T10:18:04.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.149+0000 7f855f7fe700 1 -- 172.21.15.3:0/434213383 learned_addr learned my addr 172.21.15.3:0/434213383 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:04.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.149+0000 7f855f7fe700 1 -- 172.21.15.3:0/434213383 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85600a1340 msgr2=0x7f8560133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:04.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.149+0000 7f855e7fc700 1 -- 172.21.15.3:0/434213383 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3719675462 0 0) 0x7f856009edd0 con 0x7f8560133fa0 2023-10-03T10:18:04.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.149+0000 7f855f7fe700 1 --2- 172.21.15.3:0/434213383 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85600a1340 0x7f8560133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:04.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.149+0000 7f855f7fe700 1 -- 172.21.15.3:0/434213383 >> v1:172.21.15.3:6789/0 conn(0x7f8560133fa0 legacy=0x7f856013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:04.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.149+0000 7f855f7fe700 1 -- 172.21.15.3:0/434213383 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85600a1fe0 con 0x7f85600a0a30 2023-10-03T10:18:04.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.149+0000 7f855effd700 1 --2- 172.21.15.3:0/434213383 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85600a1340 0x7f8560133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:04.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.149+0000 7f855f7fe700 1 --2- 172.21.15.3:0/434213383 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85600a0a30 0x7f85600a0e00 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8550009920 tx=0x7f8550009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41d38fc2c9b95fd7 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:04.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.149+0000 7f855e7fc700 1 -- 172.21.15.3:0/434213383 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8550011830 con 0x7f85600a0a30 2023-10-03T10:18:04.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.150+0000 7f855e7fc700 1 -- 172.21.15.3:0/434213383 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8550011990 con 0x7f85600a0a30 2023-10-03T10:18:04.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.150+0000 7f855e7fc700 1 -- 172.21.15.3:0/434213383 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8550011d20 con 0x7f85600a0a30 2023-10-03T10:18:04.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.150+0000 7f8565c5c700 1 -- 172.21.15.3:0/434213383 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85600a0a30 msgr2=0x7f85600a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:04.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.150+0000 7f8565c5c700 1 --2- 172.21.15.3:0/434213383 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85600a0a30 0x7f85600a0e00 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8550009920 tx=0x7f8550009c30 comp rx=0 tx=0).stop 2023-10-03T10:18:04.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.150+0000 7f8565c5c700 1 -- 172.21.15.3:0/434213383 shutdown_connections 2023-10-03T10:18:04.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.150+0000 7f8565c5c700 1 --2- 172.21.15.3:0/434213383 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85600a0a30 0x7f85600a0e00 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:04.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.150+0000 7f8565c5c700 1 --2- 172.21.15.3:0/434213383 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85600a1340 0x7f8560133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:04.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.150+0000 7f8565c5c700 1 -- 172.21.15.3:0/434213383 >> 172.21.15.3:0/434213383 conn(0x7f85600a6550 msgr2=0x7f85600a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:04.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.150+0000 7f8565c5c700 1 -- 172.21.15.3:0/434213383 shutdown_connections 2023-10-03T10:18:04.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.150+0000 7f8565c5c700 1 -- 172.21.15.3:0/434213383 wait complete. 2023-10-03T10:18:04.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.151+0000 7f8565c5c700 1 Processor -- start 2023-10-03T10:18:04.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.151+0000 7f8565c5c700 1 -- start start 2023-10-03T10:18:04.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.152+0000 7f8565c5c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85600a0a30 0x7f85601c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:04.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.152+0000 7f8565c5c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85600a1340 0x7f85601c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:04.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.152+0000 7f8565c5c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8560133fa0 0x7f85601c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:04.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.152+0000 7f8565c5c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85600a0220 con 0x7f85600a1340 2023-10-03T10:18:04.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.152+0000 7f8565c5c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8560125170 con 0x7f85600a0a30 2023-10-03T10:18:04.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.152+0000 7f8565c5c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8560124ab0 con 0x7f8560133fa0 2023-10-03T10:18:04.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.152+0000 7f855effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85600a1340 0x7f85601c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:04.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.152+0000 7f855f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85600a0a30 0x7f85601c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:04.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.152+0000 7f855ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8560133fa0 0x7f85601c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:04.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.152+0000 7f855effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85600a1340 0x7f85601c87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46570/0 (socket says 172.21.15.3:46570) 2023-10-03T10:18:04.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.152+0000 7f855f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85600a0a30 0x7f85601c8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41838/0 (socket says 172.21.15.3:41838) 2023-10-03T10:18:04.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.152+0000 7f855effd700 1 -- 172.21.15.3:0/3098320031 learned_addr learned my addr 172.21.15.3:0/3098320031 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:04.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.152+0000 7f855effd700 1 -- 172.21.15.3:0/3098320031 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8560133fa0 msgr2=0x7f85601c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:04.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.152+0000 7f855effd700 1 --2- 172.21.15.3:0/3098320031 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8560133fa0 0x7f85601c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:04.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.152+0000 7f855effd700 1 -- 172.21.15.3:0/3098320031 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85600a0a30 msgr2=0x7f85601c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:04.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.152+0000 7f855effd700 1 --2- 172.21.15.3:0/3098320031 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85600a0a30 0x7f85601c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:04.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.152+0000 7f855effd700 1 -- 172.21.15.3:0/3098320031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85480094a0 con 0x7f85600a1340 2023-10-03T10:18:04.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.152+0000 7f855f7fe700 1 --2- 172.21.15.3:0/3098320031 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85600a0a30 0x7f85601c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:04.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.153+0000 7f855effd700 1 --2- 172.21.15.3:0/3098320031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85600a1340 0x7f85601c87c0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f854800ebe0 tx=0x7f854800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:04.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.153+0000 7f8557fff700 1 -- 172.21.15.3:0/3098320031 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854800c850 con 0x7f85600a1340 2023-10-03T10:18:04.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.153+0000 7f8557fff700 1 -- 172.21.15.3:0/3098320031 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f854800c9b0 con 0x7f85600a1340 2023-10-03T10:18:04.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.153+0000 7f8565c5c700 1 -- 172.21.15.3:0/3098320031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8550009550 con 0x7f85600a1340 2023-10-03T10:18:04.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.153+0000 7f8557fff700 1 -- 172.21.15.3:0/3098320031 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854800cd40 con 0x7f85600a1340 2023-10-03T10:18:04.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.153+0000 7f8565c5c700 1 -- 172.21.15.3:0/3098320031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85600a25e0 con 0x7f85600a1340 2023-10-03T10:18:04.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.154+0000 7f8565c5c700 1 -- 172.21.15.3:0/3098320031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8560125dd0 con 0x7f85600a1340 2023-10-03T10:18:04.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.154+0000 7f8557fff700 1 -- 172.21.15.3:0/3098320031 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8548013470 con 0x7f85600a1340 2023-10-03T10:18:04.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.154+0000 7f8557fff700 1 -- 172.21.15.3:0/3098320031 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8548040100 con 0x7f85600a1340 2023-10-03T10:18:04.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.159+0000 7f8557fff700 1 -- 172.21.15.3:0/3098320031 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8548013720 con 0x7f85600a1340 2023-10-03T10:18:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.365+0000 7f8565c5c700 1 -- 172.21.15.3:0/3098320031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8560074ea0 con 0x7f85600a1340 2023-10-03T10:18:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.366+0000 7f8557fff700 1 -- 172.21.15.3:0/3098320031 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8548017030 con 0x7f85600a1340 2023-10-03T10:18:04.366 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:04.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.368+0000 7f8565c5c700 1 -- 172.21.15.3:0/3098320031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85600a1340 msgr2=0x7f85601c87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:04.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.368+0000 7f8565c5c700 1 --2- 172.21.15.3:0/3098320031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85600a1340 0x7f85601c87c0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f854800ebe0 tx=0x7f854800eef0 comp rx=0 tx=0).stop 2023-10-03T10:18:04.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.368+0000 7f8565c5c700 1 -- 172.21.15.3:0/3098320031 shutdown_connections 2023-10-03T10:18:04.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.368+0000 7f8565c5c700 1 --2- 172.21.15.3:0/3098320031 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8560133fa0 0x7f85601c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:04.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.368+0000 7f8565c5c700 1 --2- 172.21.15.3:0/3098320031 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85600a0a30 0x7f85601c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:04.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.368+0000 7f8565c5c700 1 --2- 172.21.15.3:0/3098320031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85600a1340 0x7f85601c87c0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:04.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.368+0000 7f8565c5c700 1 -- 172.21.15.3:0/3098320031 >> 172.21.15.3:0/3098320031 conn(0x7f85600a6550 msgr2=0x7f856011c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:04.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.368+0000 7f8565c5c700 1 -- 172.21.15.3:0/3098320031 shutdown_connections 2023-10-03T10:18:04.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:04.368+0000 7f8565c5c700 1 -- 172.21.15.3:0/3098320031 wait complete. 2023-10-03T10:18:04.382 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:05.045 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:05.083 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:05.111 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:05.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.196+0000 7f58301da700 1 Processor -- start 2023-10-03T10:18:05.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.196+0000 7f58301da700 1 -- start start 2023-10-03T10:18:05.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.197+0000 7f58301da700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5828133420 0x7f58281337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:05.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.197+0000 7f58301da700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5828133e30 0x7f582812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:05.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.197+0000 7f58301da700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5828134260 0x7f58281304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:05.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.197+0000 7f58301da700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f582806b990 con 0x7f5828134260 2023-10-03T10:18:05.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.197+0000 7f58301da700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58280a0590 con 0x7f5828133e30 2023-10-03T10:18:05.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.197+0000 7f58301da700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f582809fb80 con 0x7f5828133420 2023-10-03T10:18:05.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.197+0000 7f582d775700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5828133e30 0x7f582812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:05.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.197+0000 7f582d775700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5828133e30 0x7f582812ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41848/0 (socket says 172.21.15.3:41848) 2023-10-03T10:18:05.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.197+0000 7f582d775700 1 -- 172.21.15.3:0/2131549304 learned_addr learned my addr 172.21.15.3:0/2131549304 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:05.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.197+0000 7f582e777700 1 --2- 172.21.15.3:0/2131549304 >> v2:172.21.15.3:3300/0 conn(0x7f5828134260 0x7f58281304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:05.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.197+0000 7f582df76700 1 --2- 172.21.15.3:0/2131549304 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5828133420 0x7f58281337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:05.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.197+0000 7f582d775700 1 -- 172.21.15.3:0/2131549304 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5828133420 msgr2=0x7f58281337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:05.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.197+0000 7f582d775700 1 --2- 172.21.15.3:0/2131549304 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5828133420 0x7f58281337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.197+0000 7f582d775700 1 -- 172.21.15.3:0/2131549304 >> v2:172.21.15.3:3300/0 conn(0x7f5828134260 msgr2=0x7f58281304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:05.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.197+0000 7f582d775700 1 --2- 172.21.15.3:0/2131549304 >> v2:172.21.15.3:3300/0 conn(0x7f5828134260 0x7f58281304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.197+0000 7f582d775700 1 -- 172.21.15.3:0/2131549304 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58280a0030 con 0x7f5828133e30 2023-10-03T10:18:05.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.197+0000 7f582e777700 1 --2- 172.21.15.3:0/2131549304 >> v2:172.21.15.3:3300/0 conn(0x7f5828134260 0x7f58281304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:05.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.198+0000 7f582d775700 1 --2- 172.21.15.3:0/2131549304 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5828133e30 0x7f582812ff90 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f58180098f0 tx=0x7f5818009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20bb4271c921ee2e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:05.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.198+0000 7f582cf74700 1 -- 172.21.15.3:0/2131549304 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5818011830 con 0x7f5828133e30 2023-10-03T10:18:05.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.198+0000 7f582cf74700 1 -- 172.21.15.3:0/2131549304 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5818011990 con 0x7f5828133e30 2023-10-03T10:18:05.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.198+0000 7f582cf74700 1 -- 172.21.15.3:0/2131549304 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5818011d20 con 0x7f5828133e30 2023-10-03T10:18:05.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.199+0000 7f58301da700 1 -- 172.21.15.3:0/2131549304 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5828133e30 msgr2=0x7f582812ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:05.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.199+0000 7f58301da700 1 --2- 172.21.15.3:0/2131549304 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5828133e30 0x7f582812ff90 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f58180098f0 tx=0x7f5818009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:05.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.199+0000 7f58301da700 1 -- 172.21.15.3:0/2131549304 shutdown_connections 2023-10-03T10:18:05.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.199+0000 7f58301da700 1 --2- 172.21.15.3:0/2131549304 >> v2:172.21.15.3:3300/0 conn(0x7f5828134260 0x7f58281304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.199+0000 7f58301da700 1 --2- 172.21.15.3:0/2131549304 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5828133420 0x7f58281337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.199+0000 7f58301da700 1 --2- 172.21.15.3:0/2131549304 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5828133e30 0x7f582812ff90 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.199+0000 7f58301da700 1 -- 172.21.15.3:0/2131549304 >> 172.21.15.3:0/2131549304 conn(0x7f58280a4210 msgr2=0x7f58280a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:05.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.199+0000 7f58301da700 1 -- 172.21.15.3:0/2131549304 shutdown_connections 2023-10-03T10:18:05.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.199+0000 7f58301da700 1 -- 172.21.15.3:0/2131549304 wait complete. 2023-10-03T10:18:05.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.200+0000 7f58301da700 1 Processor -- start 2023-10-03T10:18:05.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.200+0000 7f58301da700 1 -- start start 2023-10-03T10:18:05.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.200+0000 7f58301da700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5828133420 0x7f582809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:05.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.200+0000 7f58301da700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5828133e30 0x7f582809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:05.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.200+0000 7f58301da700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5828134260 0x7f582809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:05.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.200+0000 7f58301da700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582812d9b0 con 0x7f5828134260 2023-10-03T10:18:05.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.200+0000 7f58301da700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582811feb0 con 0x7f5828133e30 2023-10-03T10:18:05.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.200+0000 7f58301da700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5828120100 con 0x7f5828133420 2023-10-03T10:18:05.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.200+0000 7f582d775700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5828133e30 0x7f582809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:05.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.200+0000 7f582d775700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5828133e30 0x7f582809c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41860/0 (socket says 172.21.15.3:41860) 2023-10-03T10:18:05.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.200+0000 7f582d775700 1 -- 172.21.15.3:0/2388663649 learned_addr learned my addr 172.21.15.3:0/2388663649 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:05.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.201+0000 7f582d775700 1 -- 172.21.15.3:0/2388663649 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5828133420 msgr2=0x7f582809c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:05.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.201+0000 7f582d775700 1 --2- 172.21.15.3:0/2388663649 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5828133420 0x7f582809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.201+0000 7f582d775700 1 -- 172.21.15.3:0/2388663649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5828134260 msgr2=0x7f582809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:05.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.201+0000 7f582d775700 1 --2- 172.21.15.3:0/2388663649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5828134260 0x7f582809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.201+0000 7f582d775700 1 -- 172.21.15.3:0/2388663649 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5820009a70 con 0x7f5828133e30 2023-10-03T10:18:05.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.201+0000 7f582d775700 1 --2- 172.21.15.3:0/2388663649 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5828133e30 0x7f582809c9c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5818009c40 tx=0x7f5818004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:05.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.201+0000 7f581e7fc700 1 -- 172.21.15.3:0/2388663649 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5818004450 con 0x7f5828133e30 2023-10-03T10:18:05.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.201+0000 7f581e7fc700 1 -- 172.21.15.3:0/2388663649 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58180045b0 con 0x7f5828133e30 2023-10-03T10:18:05.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.201+0000 7f58301da700 1 -- 172.21.15.3:0/2388663649 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5818009550 con 0x7f5828133e30 2023-10-03T10:18:05.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.201+0000 7f581e7fc700 1 -- 172.21.15.3:0/2388663649 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58180038e0 con 0x7f5828133e30 2023-10-03T10:18:05.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.201+0000 7f58301da700 1 -- 172.21.15.3:0/2388663649 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5828131520 con 0x7f5828133e30 2023-10-03T10:18:05.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.202+0000 7f581e7fc700 1 -- 172.21.15.3:0/2388663649 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5818021070 con 0x7f5828133e30 2023-10-03T10:18:05.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.202+0000 7f581e7fc700 1 -- 172.21.15.3:0/2388663649 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5818040a70 con 0x7f5828133e30 2023-10-03T10:18:05.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.202+0000 7f58301da700 1 -- 172.21.15.3:0/2388663649 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5828074ea0 con 0x7f5828133e30 2023-10-03T10:18:05.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.211+0000 7f581e7fc700 1 -- 172.21.15.3:0/2388663649 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5818029080 con 0x7f5828133e30 2023-10-03T10:18:05.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.240+0000 7f7ca53cf700 1 Processor -- start 2023-10-03T10:18:05.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.240+0000 7f7ca53cf700 1 -- start start 2023-10-03T10:18:05.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.241+0000 7f7ca53cf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ca0133420 0x7f7ca01337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:05.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.241+0000 7f7ca53cf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7ca0134260 0x7f7ca01306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:05.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.241+0000 7f7ca53cf700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ca006b990 con 0x7f7ca0133e30 2023-10-03T10:18:05.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.241+0000 7f7ca53cf700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca00a0590 con 0x7f7ca0134260 2023-10-03T10:18:05.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.241+0000 7f7ca53cf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca009fb80 con 0x7f7ca0133420 2023-10-03T10:18:05.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.241+0000 7f7c9effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ca0133420 0x7f7ca01337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:05.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.241+0000 7f7c9effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ca0133420 0x7f7ca01337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47702/0 (socket says 172.21.15.3:47702) 2023-10-03T10:18:05.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.241+0000 7f7c9effd700 1 -- 172.21.15.3:0/4225932910 learned_addr learned my addr 172.21.15.3:0/4225932910 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:05.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.241+0000 7f7c9f7fe700 1 --2- 172.21.15.3:0/4225932910 >> v2:172.21.15.3:3300/0 conn(0x7f7ca0134260 0x7f7ca01306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:05.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.242+0000 7f7c9effd700 1 -- 172.21.15.3:0/4225932910 >> v2:172.21.15.3:3300/0 conn(0x7f7ca0134260 msgr2=0x7f7ca01306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:05.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.242+0000 7f7c9effd700 1 --2- 172.21.15.3:0/4225932910 >> v2:172.21.15.3:3300/0 conn(0x7f7ca0134260 0x7f7ca01306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.242+0000 7f7c9effd700 1 -- 172.21.15.3:0/4225932910 >> v1:172.21.15.3:6789/0 conn(0x7f7ca0133e30 legacy=0x7f7ca012ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:05.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.242+0000 7f7c9effd700 1 -- 172.21.15.3:0/4225932910 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca00a0030 con 0x7f7ca0133420 2023-10-03T10:18:05.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.242+0000 7f7c9effd700 1 --2- 172.21.15.3:0/4225932910 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ca0133420 0x7f7ca01337f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7c90009e50 tx=0x7f7c9000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e79616a4beff8c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:05.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.242+0000 7f7c9dffb700 1 -- 172.21.15.3:0/4225932910 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9000e070 con 0x7f7ca0133420 2023-10-03T10:18:05.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.242+0000 7f7c9dffb700 1 -- 172.21.15.3:0/4225932910 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9000b910 con 0x7f7ca0133420 2023-10-03T10:18:05.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.242+0000 7f7c9dffb700 1 -- 172.21.15.3:0/4225932910 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9000bca0 con 0x7f7ca0133420 2023-10-03T10:18:05.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.243+0000 7f7ca53cf700 1 -- 172.21.15.3:0/4225932910 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ca0133420 msgr2=0x7f7ca01337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:05.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.243+0000 7f7ca53cf700 1 --2- 172.21.15.3:0/4225932910 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ca0133420 0x7f7ca01337f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7c90009e50 tx=0x7f7c9000b330 comp rx=0 tx=0).stop 2023-10-03T10:18:05.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.243+0000 7f7ca53cf700 1 -- 172.21.15.3:0/4225932910 shutdown_connections 2023-10-03T10:18:05.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.243+0000 7f7ca53cf700 1 --2- 172.21.15.3:0/4225932910 >> v2:172.21.15.3:3300/0 conn(0x7f7ca0134260 0x7f7ca01306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.243+0000 7f7ca53cf700 1 --2- 172.21.15.3:0/4225932910 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ca0133420 0x7f7ca01337f0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.243+0000 7f7ca53cf700 1 -- 172.21.15.3:0/4225932910 >> 172.21.15.3:0/4225932910 conn(0x7f7ca00a4210 msgr2=0x7f7ca00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:05.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.243+0000 7f7ca53cf700 1 -- 172.21.15.3:0/4225932910 shutdown_connections 2023-10-03T10:18:05.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.243+0000 7f7ca53cf700 1 -- 172.21.15.3:0/4225932910 wait complete. 2023-10-03T10:18:05.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.244+0000 7f7ca53cf700 1 Processor -- start 2023-10-03T10:18:05.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.245+0000 7f7ca53cf700 1 -- start start 2023-10-03T10:18:05.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.245+0000 7f7ca53cf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ca0133420 0x7f7ca009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:05.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.245+0000 7f7ca53cf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ca0133e30 0x7f7ca009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:05.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.245+0000 7f7ca53cf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ca0134260 0x7f7ca009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.245+0000 7f7ca53cf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca012d260 con 0x7f7ca0134260 2023-10-03T10:18:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.245+0000 7f7ca53cf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca009df00 con 0x7f7ca0133e30 2023-10-03T10:18:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.245+0000 7f7ca53cf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca011fcc0 con 0x7f7ca0133420 2023-10-03T10:18:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.245+0000 7f7c9effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ca0133420 0x7f7ca009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.245+0000 7f7c9effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ca0133420 0x7f7ca009c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47710/0 (socket says 172.21.15.3:47710) 2023-10-03T10:18:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.245+0000 7f7c9effd700 1 -- 172.21.15.3:0/2030358173 learned_addr learned my addr 172.21.15.3:0/2030358173 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.245+0000 7f7c9e7fc700 1 --2- 172.21.15.3:0/2030358173 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ca0133e30 0x7f7ca009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.246+0000 7f7c9f7fe700 1 --2- 172.21.15.3:0/2030358173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ca0134260 0x7f7ca009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.246+0000 7f7c9e7fc700 1 -- 172.21.15.3:0/2030358173 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ca0133420 msgr2=0x7f7ca009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.246+0000 7f7c9e7fc700 1 --2- 172.21.15.3:0/2030358173 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ca0133420 0x7f7ca009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.246+0000 7f7c9e7fc700 1 -- 172.21.15.3:0/2030358173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ca0134260 msgr2=0x7f7ca009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.246+0000 7f7c9e7fc700 1 --2- 172.21.15.3:0/2030358173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ca0134260 0x7f7ca009cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.246+0000 7f7c9e7fc700 1 -- 172.21.15.3:0/2030358173 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c90009b30 con 0x7f7ca0133e30 2023-10-03T10:18:05.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.246+0000 7f7c9f7fe700 1 --2- 172.21.15.3:0/2030358173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ca0134260 0x7f7ca009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:05.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.246+0000 7f7c9effd700 1 --2- 172.21.15.3:0/2030358173 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ca0133420 0x7f7ca009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:05.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.246+0000 7f7c9e7fc700 1 --2- 172.21.15.3:0/2030358173 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ca0133e30 0x7f7ca009c9c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7c88000f60 tx=0x7f7c8800c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:05.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.246+0000 7f7c977fe700 1 -- 172.21.15.3:0/2030358173 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c880050d0 con 0x7f7ca0133e30 2023-10-03T10:18:05.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.246+0000 7f7c977fe700 1 -- 172.21.15.3:0/2030358173 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c88005230 con 0x7f7ca0133e30 2023-10-03T10:18:05.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.246+0000 7f7ca53cf700 1 -- 172.21.15.3:0/2030358173 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca01209f0 con 0x7f7ca0133e30 2023-10-03T10:18:05.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.246+0000 7f7c977fe700 1 -- 172.21.15.3:0/2030358173 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c88003680 con 0x7f7ca0133e30 2023-10-03T10:18:05.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.246+0000 7f7ca53cf700 1 -- 172.21.15.3:0/2030358173 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca012dac0 con 0x7f7ca0133e30 2023-10-03T10:18:05.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.247+0000 7f7c977fe700 1 -- 172.21.15.3:0/2030358173 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7c880037e0 con 0x7f7ca0133e30 2023-10-03T10:18:05.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.248+0000 7f7ca53cf700 1 -- 172.21.15.3:0/2030358173 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca0074ea0 con 0x7f7ca0133e30 2023-10-03T10:18:05.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.248+0000 7f7c977fe700 1 -- 172.21.15.3:0/2030358173 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7c8803e9e0 con 0x7f7ca0133e30 2023-10-03T10:18:05.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.257+0000 7f7c977fe700 1 -- 172.21.15.3:0/2030358173 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7c88011590 con 0x7f7ca0133e30 2023-10-03T10:18:05.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.276+0000 7f25ecca9700 1 Processor -- start 2023-10-03T10:18:05.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.276+0000 7f25ecca9700 1 -- start start 2023-10-03T10:18:05.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.277+0000 7f25ecca9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25e809d6f0 0x7f25e809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:05.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.277+0000 7f25ecca9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25e809e000 0x7f25e81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:05.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.277+0000 7f25ecca9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f25e809e410 0x7f25e8133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:05.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.277+0000 7f25ecca9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e806b460 con 0x7f25e809e410 2023-10-03T10:18:05.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.277+0000 7f25ecca9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e809b120 con 0x7f25e809e000 2023-10-03T10:18:05.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.277+0000 7f25ecca9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25e809a7e0 con 0x7f25e809d6f0 2023-10-03T10:18:05.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.277+0000 7f25e659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25e809d6f0 0x7f25e809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:05.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.277+0000 7f25e6d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f25e809e410 0x7f25e8133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:05.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.277+0000 7f25e5d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25e809e000 0x7f25e81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:05.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.277+0000 7f25e659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25e809d6f0 0x7f25e809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47718/0 (socket says 172.21.15.3:47718) 2023-10-03T10:18:05.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.277+0000 7f25e659c700 1 -- 172.21.15.3:0/4247088039 learned_addr learned my addr 172.21.15.3:0/4247088039 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:05.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.278+0000 7f25e659c700 1 -- 172.21.15.3:0/4247088039 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25e809e000 msgr2=0x7f25e81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:05.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.278+0000 7f25e659c700 1 --2- 172.21.15.3:0/4247088039 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25e809e000 0x7f25e81339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.278+0000 7f25e659c700 1 -- 172.21.15.3:0/4247088039 >> v2:172.21.15.3:3300/0 conn(0x7f25e809e410 msgr2=0x7f25e8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:05.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.278+0000 7f25e659c700 1 --2- 172.21.15.3:0/4247088039 >> v2:172.21.15.3:3300/0 conn(0x7f25e809e410 0x7f25e8133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.278+0000 7f25e659c700 1 -- 172.21.15.3:0/4247088039 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e809a500 con 0x7f25e809d6f0 2023-10-03T10:18:05.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.278+0000 7f25e5d9b700 1 --2- 172.21.15.3:0/4247088039 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25e809e000 0x7f25e81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:05.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.278+0000 7f25e6d9d700 1 --2- 172.21.15.3:0/4247088039 >> v2:172.21.15.3:3300/0 conn(0x7f25e809e410 0x7f25e8133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:05.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.278+0000 7f25e659c700 1 --2- 172.21.15.3:0/4247088039 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25e809d6f0 0x7f25e809dac0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f25d8009920 tx=0x7f25d8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49d39e7921450c4d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:05.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.278+0000 7f25e559a700 1 -- 172.21.15.3:0/4247088039 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8011830 con 0x7f25e809d6f0 2023-10-03T10:18:05.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.278+0000 7f25e559a700 1 -- 172.21.15.3:0/4247088039 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d8011990 con 0x7f25e809d6f0 2023-10-03T10:18:05.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.278+0000 7f25e559a700 1 -- 172.21.15.3:0/4247088039 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8011d20 con 0x7f25e809d6f0 2023-10-03T10:18:05.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.279+0000 7f25ecca9700 1 -- 172.21.15.3:0/4247088039 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25e809d6f0 msgr2=0x7f25e809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:05.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.279+0000 7f25ecca9700 1 --2- 172.21.15.3:0/4247088039 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25e809d6f0 0x7f25e809dac0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f25d8009920 tx=0x7f25d8009c30 comp rx=0 tx=0).stop 2023-10-03T10:18:05.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.279+0000 7f25ecca9700 1 -- 172.21.15.3:0/4247088039 shutdown_connections 2023-10-03T10:18:05.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.279+0000 7f25ecca9700 1 --2- 172.21.15.3:0/4247088039 >> v2:172.21.15.3:3300/0 conn(0x7f25e809e410 0x7f25e8133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.279+0000 7f25ecca9700 1 --2- 172.21.15.3:0/4247088039 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25e809d6f0 0x7f25e809dac0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.279+0000 7f25ecca9700 1 --2- 172.21.15.3:0/4247088039 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25e809e000 0x7f25e81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.279+0000 7f25ecca9700 1 -- 172.21.15.3:0/4247088039 >> 172.21.15.3:0/4247088039 conn(0x7f25e8066f50 msgr2=0x7f25e809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:05.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.279+0000 7f25ecca9700 1 -- 172.21.15.3:0/4247088039 shutdown_connections 2023-10-03T10:18:05.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.279+0000 7f25ecca9700 1 -- 172.21.15.3:0/4247088039 wait complete. 2023-10-03T10:18:05.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.280+0000 7f25ecca9700 1 Processor -- start 2023-10-03T10:18:05.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.280+0000 7f25ecca9700 1 -- start start 2023-10-03T10:18:05.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.281+0000 7f25ecca9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25e809d6f0 0x7f25e81a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:05.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.281+0000 7f25ecca9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25e809e000 0x7f25e81a6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:05.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.281+0000 7f25ecca9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25e809e410 0x7f25e81a6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:05.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.281+0000 7f25ecca9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e81a9200 con 0x7f25e809e410 2023-10-03T10:18:05.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.281+0000 7f25ecca9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e81a7b20 con 0x7f25e809e000 2023-10-03T10:18:05.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.281+0000 7f25ecca9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25e81a96b0 con 0x7f25e809d6f0 2023-10-03T10:18:05.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.281+0000 7f25e659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25e809d6f0 0x7f25e81a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:05.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.281+0000 7f25e659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25e809d6f0 0x7f25e81a5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47720/0 (socket says 172.21.15.3:47720) 2023-10-03T10:18:05.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.281+0000 7f25e659c700 1 -- 172.21.15.3:0/2276378648 learned_addr learned my addr 172.21.15.3:0/2276378648 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:05.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.281+0000 7f25e5d9b700 1 --2- 172.21.15.3:0/2276378648 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25e809e000 0x7f25e81a6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:05.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.281+0000 7f25e6d9d700 1 --2- 172.21.15.3:0/2276378648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25e809e410 0x7f25e81a6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:05.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.281+0000 7f25e5d9b700 1 -- 172.21.15.3:0/2276378648 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25e809d6f0 msgr2=0x7f25e81a5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:05.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.281+0000 7f25e5d9b700 1 --2- 172.21.15.3:0/2276378648 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25e809d6f0 0x7f25e81a5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.281+0000 7f25e5d9b700 1 -- 172.21.15.3:0/2276378648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25e809e410 msgr2=0x7f25e81a6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:05.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.281+0000 7f25e5d9b700 1 --2- 172.21.15.3:0/2276378648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25e809e410 0x7f25e81a6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.281+0000 7f25e5d9b700 1 -- 172.21.15.3:0/2276378648 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25dc0094a0 con 0x7f25e809e000 2023-10-03T10:18:05.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.281+0000 7f25e6d9d700 1 --2- 172.21.15.3:0/2276378648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25e809e410 0x7f25e81a6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:05.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.282+0000 7f25e5d9b700 1 --2- 172.21.15.3:0/2276378648 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25e809e000 0x7f25e81a6440 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f25dc00e9b0 tx=0x7f25dc00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:05.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.282+0000 7f25d6ffd700 1 -- 172.21.15.3:0/2276378648 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25dc00ee50 con 0x7f25e809e000 2023-10-03T10:18:05.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.282+0000 7f25d6ffd700 1 -- 172.21.15.3:0/2276378648 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25dc00c850 con 0x7f25e809e000 2023-10-03T10:18:05.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.282+0000 7f25ecca9700 1 -- 172.21.15.3:0/2276378648 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e0009a70 con 0x7f25e809e000 2023-10-03T10:18:05.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.282+0000 7f25d6ffd700 1 -- 172.21.15.3:0/2276378648 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25dc00cbe0 con 0x7f25e809e000 2023-10-03T10:18:05.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.282+0000 7f25ecca9700 1 -- 172.21.15.3:0/2276378648 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d8009550 con 0x7f25e809e000 2023-10-03T10:18:05.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.283+0000 7f25d6ffd700 1 -- 172.21.15.3:0/2276378648 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f25dc00cd40 con 0x7f25e809e000 2023-10-03T10:18:05.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.283+0000 7f25ecca9700 1 -- 172.21.15.3:0/2276378648 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e8074ea0 con 0x7f25e809e000 2023-10-03T10:18:05.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.283+0000 7f25d6ffd700 1 -- 172.21.15.3:0/2276378648 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f25dc03faa0 con 0x7f25e809e000 2023-10-03T10:18:05.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.292+0000 7f25d6ffd700 1 -- 172.21.15.3:0/2276378648 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f25dc01b3b0 con 0x7f25e809e000 2023-10-03T10:18:05.383 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:05.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.429+0000 7f58301da700 1 -- 172.21.15.3:0/2388663649 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f582806b250 con 0x7f5828133e30 2023-10-03T10:18:05.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.430+0000 7f581e7fc700 1 -- 172.21.15.3:0/2388663649 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5818015020 con 0x7f5828133e30 2023-10-03T10:18:05.430 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:05.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.433+0000 7f58301da700 1 -- 172.21.15.3:0/2388663649 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5828133e30 msgr2=0x7f582809c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:05.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.433+0000 7f58301da700 1 --2- 172.21.15.3:0/2388663649 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5828133e30 0x7f582809c9c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5818009c40 tx=0x7f5818004750 comp rx=0 tx=0).stop 2023-10-03T10:18:05.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.433+0000 7f58301da700 1 -- 172.21.15.3:0/2388663649 shutdown_connections 2023-10-03T10:18:05.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.433+0000 7f58301da700 1 --2- 172.21.15.3:0/2388663649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5828134260 0x7f582809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.433+0000 7f58301da700 1 --2- 172.21.15.3:0/2388663649 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5828133420 0x7f582809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.433+0000 7f58301da700 1 --2- 172.21.15.3:0/2388663649 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5828133e30 0x7f582809c9c0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.433+0000 7f58301da700 1 -- 172.21.15.3:0/2388663649 >> 172.21.15.3:0/2388663649 conn(0x7f58280a4210 msgr2=0x7f582812de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:05.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.434+0000 7f58301da700 1 -- 172.21.15.3:0/2388663649 shutdown_connections 2023-10-03T10:18:05.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.434+0000 7f58301da700 1 -- 172.21.15.3:0/2388663649 wait complete. 2023-10-03T10:18:05.454 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:05.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.488+0000 7f7ca53cf700 1 -- 172.21.15.3:0/2030358173 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7ca006b250 con 0x7f7ca0133e30 2023-10-03T10:18:05.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.489+0000 7f7c977fe700 1 -- 172.21.15.3:0/2030358173 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7c880192c0 con 0x7f7ca0133e30 2023-10-03T10:18:05.489 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:05.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.491+0000 7f7ca53cf700 1 -- 172.21.15.3:0/2030358173 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ca0133e30 msgr2=0x7f7ca009c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:05.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.491+0000 7f7ca53cf700 1 --2- 172.21.15.3:0/2030358173 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ca0133e30 0x7f7ca009c9c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7c88000f60 tx=0x7f7c8800c730 comp rx=0 tx=0).stop 2023-10-03T10:18:05.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.491+0000 7f7ca53cf700 1 -- 172.21.15.3:0/2030358173 shutdown_connections 2023-10-03T10:18:05.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.491+0000 7f7ca53cf700 1 --2- 172.21.15.3:0/2030358173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ca0134260 0x7f7ca009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.491+0000 7f7ca53cf700 1 --2- 172.21.15.3:0/2030358173 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ca0133420 0x7f7ca009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.491+0000 7f7ca53cf700 1 --2- 172.21.15.3:0/2030358173 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ca0133e30 0x7f7ca009c9c0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.491+0000 7f7ca53cf700 1 -- 172.21.15.3:0/2030358173 >> 172.21.15.3:0/2030358173 conn(0x7f7ca00a4210 msgr2=0x7f7ca00a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:05.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.491+0000 7f7ca53cf700 1 -- 172.21.15.3:0/2030358173 shutdown_connections 2023-10-03T10:18:05.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.491+0000 7f7ca53cf700 1 -- 172.21.15.3:0/2030358173 wait complete. 2023-10-03T10:18:05.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.501+0000 7f25ecca9700 1 -- 172.21.15.3:0/2276378648 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f25e81a8cd0 con 0x7f25e809e000 2023-10-03T10:18:05.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.501+0000 7f25d6ffd700 1 -- 172.21.15.3:0/2276378648 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f25dc011b40 con 0x7f25e809e000 2023-10-03T10:18:05.501 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:05.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.503+0000 7f25ecca9700 1 -- 172.21.15.3:0/2276378648 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25e809e000 msgr2=0x7f25e81a6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:05.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.503+0000 7f25ecca9700 1 --2- 172.21.15.3:0/2276378648 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25e809e000 0x7f25e81a6440 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f25dc00e9b0 tx=0x7f25dc00ecc0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.503+0000 7f25ecca9700 1 -- 172.21.15.3:0/2276378648 shutdown_connections 2023-10-03T10:18:05.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.503+0000 7f25ecca9700 1 --2- 172.21.15.3:0/2276378648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25e809e410 0x7f25e81a6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.503+0000 7f25ecca9700 1 --2- 172.21.15.3:0/2276378648 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25e809d6f0 0x7f25e81a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.503+0000 7f25ecca9700 1 --2- 172.21.15.3:0/2276378648 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25e809e000 0x7f25e81a6440 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.503+0000 7f25ecca9700 1 -- 172.21.15.3:0/2276378648 >> 172.21.15.3:0/2276378648 conn(0x7f25e8066f50 msgr2=0x7f25e81baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:05.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.504+0000 7f25ecca9700 1 -- 172.21.15.3:0/2276378648 shutdown_connections 2023-10-03T10:18:05.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.504+0000 7f25ecca9700 1 -- 172.21.15.3:0/2276378648 wait complete. 2023-10-03T10:18:05.505 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:05.517 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:05.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.526+0000 7f459465b700 1 Processor -- start 2023-10-03T10:18:05.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.526+0000 7f459465b700 1 -- start start 2023-10-03T10:18:05.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.526+0000 7f459465b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f458c0a0a30 0x7f458c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:05.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.526+0000 7f459465b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458c0a1340 0x7f458c133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:05.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.526+0000 7f459465b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f458c09edd0 con 0x7f458c133fa0 2023-10-03T10:18:05.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.526+0000 7f459465b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f458c09e1f0 con 0x7f458c0a0a30 2023-10-03T10:18:05.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.526+0000 7f459465b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458c09dc50 con 0x7f458c0a1340 2023-10-03T10:18:05.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.526+0000 7f4591bf6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458c0a1340 0x7f458c133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:05.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.526+0000 7f45923f7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f458c0a0a30 0x7f458c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:05.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.527+0000 7f4592bf8700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f458c133fa0 0x7f458c13ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50546/0 (socket says 172.21.15.3:50546) 2023-10-03T10:18:05.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.527+0000 7f4592bf8700 1 -- 172.21.15.3:0/1091326244 learned_addr learned my addr 172.21.15.3:0/1091326244 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:05.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.527+0000 7f4591bf6700 1 -- 172.21.15.3:0/1091326244 >> v2:172.21.15.3:3300/0 conn(0x7f458c0a0a30 msgr2=0x7f458c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:05.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.527+0000 7f4591bf6700 1 --2- 172.21.15.3:0/1091326244 >> v2:172.21.15.3:3300/0 conn(0x7f458c0a0a30 0x7f458c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.527+0000 7f4591bf6700 1 -- 172.21.15.3:0/1091326244 >> v1:172.21.15.3:6789/0 conn(0x7f458c133fa0 legacy=0x7f458c13ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:05.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.527+0000 7f4591bf6700 1 -- 172.21.15.3:0/1091326244 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458c0a1fe0 con 0x7f458c0a1340 2023-10-03T10:18:05.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.527+0000 7f45923f7700 1 --2- 172.21.15.3:0/1091326244 >> v2:172.21.15.3:3300/0 conn(0x7f458c0a0a30 0x7f458c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:05.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.527+0000 7f4591bf6700 1 --2- 172.21.15.3:0/1091326244 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458c0a1340 0x7f458c133a60 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f457c009840 tx=0x7f457c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c00d9eba7c53e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:05.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.527+0000 7f45913f5700 1 -- 172.21.15.3:0/1091326244 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c011830 con 0x7f458c0a1340 2023-10-03T10:18:05.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.527+0000 7f45913f5700 1 -- 172.21.15.3:0/1091326244 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f457c011990 con 0x7f458c0a1340 2023-10-03T10:18:05.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.527+0000 7f45913f5700 1 -- 172.21.15.3:0/1091326244 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c011d20 con 0x7f458c0a1340 2023-10-03T10:18:05.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.528+0000 7f459465b700 1 -- 172.21.15.3:0/1091326244 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458c0a1340 msgr2=0x7f458c133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:05.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.528+0000 7f459465b700 1 --2- 172.21.15.3:0/1091326244 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458c0a1340 0x7f458c133a60 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f457c009840 tx=0x7f457c009b50 comp rx=0 tx=0).stop 2023-10-03T10:18:05.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.528+0000 7f459465b700 1 -- 172.21.15.3:0/1091326244 shutdown_connections 2023-10-03T10:18:05.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.528+0000 7f459465b700 1 --2- 172.21.15.3:0/1091326244 >> v2:172.21.15.3:3300/0 conn(0x7f458c0a0a30 0x7f458c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.528+0000 7f459465b700 1 --2- 172.21.15.3:0/1091326244 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458c0a1340 0x7f458c133a60 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.528+0000 7f459465b700 1 -- 172.21.15.3:0/1091326244 >> 172.21.15.3:0/1091326244 conn(0x7f458c0a6550 msgr2=0x7f458c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:05.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.528+0000 7f459465b700 1 -- 172.21.15.3:0/1091326244 shutdown_connections 2023-10-03T10:18:05.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.528+0000 7f459465b700 1 -- 172.21.15.3:0/1091326244 wait complete. 2023-10-03T10:18:05.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.529+0000 7f459465b700 1 Processor -- start 2023-10-03T10:18:05.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.530+0000 7f459465b700 1 -- start start 2023-10-03T10:18:05.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.530+0000 7f459465b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f458c0a0a30 0x7f458c11e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:05.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.530+0000 7f459465b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f458c0a1340 0x7f458c11ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:05.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.530+0000 7f459465b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458c133fa0 0x7f458c11f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:05.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.530+0000 7f459465b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c120bc0 con 0x7f458c0a1340 2023-10-03T10:18:05.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.530+0000 7f459465b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458c0a3760 con 0x7f458c133fa0 2023-10-03T10:18:05.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.530+0000 7f459465b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f458c0a0220 con 0x7f458c0a0a30 2023-10-03T10:18:05.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.530+0000 7f4591bf6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f458c0a1340 0x7f458c11ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:05.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.530+0000 7f4592bf8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458c133fa0 0x7f458c11f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:05.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.530+0000 7f4591bf6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f458c0a1340 0x7f458c11ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46634/0 (socket says 172.21.15.3:46634) 2023-10-03T10:18:05.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.530+0000 7f4591bf6700 1 -- 172.21.15.3:0/3664261663 learned_addr learned my addr 172.21.15.3:0/3664261663 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:05.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.530+0000 7f45923f7700 1 --2- 172.21.15.3:0/3664261663 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f458c0a0a30 0x7f458c11e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:05.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.531+0000 7f4592bf8700 1 -- 172.21.15.3:0/3664261663 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f458c0a0a30 msgr2=0x7f458c11e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:05.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.531+0000 7f4592bf8700 1 --2- 172.21.15.3:0/3664261663 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f458c0a0a30 0x7f458c11e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.531+0000 7f4592bf8700 1 -- 172.21.15.3:0/3664261663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f458c0a1340 msgr2=0x7f458c11ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:05.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.531+0000 7f4592bf8700 1 --2- 172.21.15.3:0/3664261663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f458c0a1340 0x7f458c11ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.531+0000 7f4592bf8700 1 -- 172.21.15.3:0/3664261663 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45880094a0 con 0x7f458c133fa0 2023-10-03T10:18:05.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.531+0000 7f45923f7700 1 --2- 172.21.15.3:0/3664261663 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f458c0a0a30 0x7f458c11e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:05.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.531+0000 7f4591bf6700 1 --2- 172.21.15.3:0/3664261663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f458c0a1340 0x7f458c11ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:05.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.531+0000 7f4592bf8700 1 --2- 172.21.15.3:0/3664261663 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458c133fa0 0x7f458c11f1d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f458400caa0 tx=0x7f458400cdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:05.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.531+0000 7f4582ffd700 1 -- 172.21.15.3:0/3664261663 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458400a9a0 con 0x7f458c133fa0 2023-10-03T10:18:05.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.531+0000 7f4582ffd700 1 -- 172.21.15.3:0/3664261663 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458400ab00 con 0x7f458c133fa0 2023-10-03T10:18:05.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.531+0000 7f459465b700 1 -- 172.21.15.3:0/3664261663 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f457c0094a0 con 0x7f458c133fa0 2023-10-03T10:18:05.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.531+0000 7f4582ffd700 1 -- 172.21.15.3:0/3664261663 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458400ae70 con 0x7f458c133fa0 2023-10-03T10:18:05.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.531+0000 7f459465b700 1 -- 172.21.15.3:0/3664261663 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458c122a00 con 0x7f458c133fa0 2023-10-03T10:18:05.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.532+0000 7f459465b700 1 -- 172.21.15.3:0/3664261663 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f458c074ea0 con 0x7f458c133fa0 2023-10-03T10:18:05.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.532+0000 7f4582ffd700 1 -- 172.21.15.3:0/3664261663 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f458401b3b0 con 0x7f458c133fa0 2023-10-03T10:18:05.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.533+0000 7f4582ffd700 1 -- 172.21.15.3:0/3664261663 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f458403f710 con 0x7f458c133fa0 2023-10-03T10:18:05.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.537+0000 7f4582ffd700 1 -- 172.21.15.3:0/3664261663 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f45840192c0 con 0x7f458c133fa0 2023-10-03T10:18:05.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.744+0000 7f459465b700 1 -- 172.21.15.3:0/3664261663 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f458c121e40 con 0x7f458c133fa0 2023-10-03T10:18:05.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.744+0000 7f4582ffd700 1 -- 172.21.15.3:0/3664261663 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4584017030 con 0x7f458c133fa0 2023-10-03T10:18:05.745 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:05.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.747+0000 7f459465b700 1 -- 172.21.15.3:0/3664261663 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458c133fa0 msgr2=0x7f458c11f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:05.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.747+0000 7f459465b700 1 --2- 172.21.15.3:0/3664261663 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458c133fa0 0x7f458c11f1d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f458400caa0 tx=0x7f458400cdb0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.747+0000 7f459465b700 1 -- 172.21.15.3:0/3664261663 shutdown_connections 2023-10-03T10:18:05.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.747+0000 7f459465b700 1 --2- 172.21.15.3:0/3664261663 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f458c133fa0 0x7f458c11f1d0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.747+0000 7f459465b700 1 --2- 172.21.15.3:0/3664261663 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f458c0a0a30 0x7f458c11e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.747+0000 7f459465b700 1 --2- 172.21.15.3:0/3664261663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f458c0a1340 0x7f458c11ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:05.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.747+0000 7f459465b700 1 -- 172.21.15.3:0/3664261663 >> 172.21.15.3:0/3664261663 conn(0x7f458c0a6550 msgr2=0x7f458c0a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:05.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.747+0000 7f459465b700 1 -- 172.21.15.3:0/3664261663 shutdown_connections 2023-10-03T10:18:05.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:05.747+0000 7f459465b700 1 -- 172.21.15.3:0/3664261663 wait complete. 2023-10-03T10:18:05.761 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:06.454 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:06.506 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:06.518 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:06.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.603+0000 7f5b88808700 1 Processor -- start 2023-10-03T10:18:06.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.603+0000 7f5b88808700 1 -- start start 2023-10-03T10:18:06.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.604+0000 7f5b88808700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5b84134630 0x7f5b84136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:06.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.604+0000 7f5b88808700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b84136f50 0x7f5b84139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:06.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.604+0000 7f5b88808700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b8409b4f0 con 0x7f5b84131810 2023-10-03T10:18:06.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.604+0000 7f5b88808700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b841313c0 con 0x7f5b84134630 2023-10-03T10:18:06.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.604+0000 7f5b88808700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8406e8c0 con 0x7f5b84136f50 2023-10-03T10:18:06.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.604+0000 7f5b81d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5b84134630 0x7f5b84136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:06.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.604+0000 7f5b82d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b84136f50 0x7f5b84139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:06.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.604+0000 7f5b81d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5b84134630 0x7f5b84136a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46650/0 (socket says 172.21.15.3:46650) 2023-10-03T10:18:06.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.604+0000 7f5b81d9b700 1 -- 172.21.15.3:0/78721355 learned_addr learned my addr 172.21.15.3:0/78721355 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:06.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.605+0000 7f5b81d9b700 1 -- 172.21.15.3:0/78721355 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b84136f50 msgr2=0x7f5b84139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:06.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.605+0000 7f5b81d9b700 1 --2- 172.21.15.3:0/78721355 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b84136f50 0x7f5b84139330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.605+0000 7f5b81d9b700 1 -- 172.21.15.3:0/78721355 >> v1:172.21.15.3:6789/0 conn(0x7f5b84131810 legacy=0x7f5b84131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:06.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.605+0000 7f5b81d9b700 1 -- 172.21.15.3:0/78721355 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b84071d20 con 0x7f5b84134630 2023-10-03T10:18:06.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.605+0000 7f5b8159a700 1 -- 172.21.15.3:0/78721355 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2435213095 0 0) 0x7f5b8409b4f0 con 0x7f5b84131810 2023-10-03T10:18:06.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.605+0000 7f5b82d9d700 1 --2- 172.21.15.3:0/78721355 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b84136f50 0x7f5b84139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:06.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.605+0000 7f5b81d9b700 1 --2- 172.21.15.3:0/78721355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b84134630 0x7f5b84136a10 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f5b780098f0 tx=0x7f5b78009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dbb39b006a46f15 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.605+0000 7f5b8159a700 1 -- 172.21.15.3:0/78721355 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b78011830 con 0x7f5b84134630 2023-10-03T10:18:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.605+0000 7f5b8159a700 1 -- 172.21.15.3:0/78721355 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b78011990 con 0x7f5b84134630 2023-10-03T10:18:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.605+0000 7f5b8159a700 1 -- 172.21.15.3:0/78721355 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b78011d20 con 0x7f5b84134630 2023-10-03T10:18:06.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.606+0000 7f5b88808700 1 -- 172.21.15.3:0/78721355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b84134630 msgr2=0x7f5b84136a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:06.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.606+0000 7f5b88808700 1 --2- 172.21.15.3:0/78721355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b84134630 0x7f5b84136a10 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f5b780098f0 tx=0x7f5b78009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:06.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.606+0000 7f5b88808700 1 -- 172.21.15.3:0/78721355 shutdown_connections 2023-10-03T10:18:06.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.606+0000 7f5b88808700 1 --2- 172.21.15.3:0/78721355 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b84136f50 0x7f5b84139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.606+0000 7f5b88808700 1 --2- 172.21.15.3:0/78721355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b84134630 0x7f5b84136a10 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.606+0000 7f5b88808700 1 -- 172.21.15.3:0/78721355 >> 172.21.15.3:0/78721355 conn(0x7f5b840a2aa0 msgr2=0x7f5b840a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:06.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.606+0000 7f5b88808700 1 -- 172.21.15.3:0/78721355 shutdown_connections 2023-10-03T10:18:06.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.606+0000 7f5b88808700 1 -- 172.21.15.3:0/78721355 wait complete. 2023-10-03T10:18:06.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.608+0000 7f5b88808700 1 Processor -- start 2023-10-03T10:18:06.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.608+0000 7f5b88808700 1 -- start start 2023-10-03T10:18:06.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.609+0000 7f5b88808700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b84131810 0x7f5b841c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:06.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.609+0000 7f5b88808700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5b84134630 0x7f5b841c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:06.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.609+0000 7f5b88808700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b84136f50 0x7f5b84118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:06.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.609+0000 7f5b88808700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8411b0c0 con 0x7f5b84131810 2023-10-03T10:18:06.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.609+0000 7f5b88808700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8411a9e0 con 0x7f5b84136f50 2023-10-03T10:18:06.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.609+0000 7f5b88808700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8411c700 con 0x7f5b84134630 2023-10-03T10:18:06.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.609+0000 7f5b8259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b84131810 0x7f5b841c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:06.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.609+0000 7f5b8259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b84131810 0x7f5b841c6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46664/0 (socket says 172.21.15.3:46664) 2023-10-03T10:18:06.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.609+0000 7f5b8259c700 1 -- 172.21.15.3:0/590069788 learned_addr learned my addr 172.21.15.3:0/590069788 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:06.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.609+0000 7f5b82d9d700 1 --2- 172.21.15.3:0/590069788 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b84136f50 0x7f5b84118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:06.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.609+0000 7f5b81d9b700 1 --2- 172.21.15.3:0/590069788 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5b84134630 0x7f5b841c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:06.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.609+0000 7f5b8259c700 1 -- 172.21.15.3:0/590069788 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5b84134630 msgr2=0x7f5b841c68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:06.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.609+0000 7f5b8259c700 1 --2- 172.21.15.3:0/590069788 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5b84134630 0x7f5b841c68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.609+0000 7f5b8259c700 1 -- 172.21.15.3:0/590069788 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b84136f50 msgr2=0x7f5b84118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:06.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.609+0000 7f5b8259c700 1 --2- 172.21.15.3:0/590069788 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b84136f50 0x7f5b84118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.609+0000 7f5b8259c700 1 -- 172.21.15.3:0/590069788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7c009a70 con 0x7f5b84131810 2023-10-03T10:18:06.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.609+0000 7f5b82d9d700 1 --2- 172.21.15.3:0/590069788 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b84136f50 0x7f5b84118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:06.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.610+0000 7f5b8259c700 1 --2- 172.21.15.3:0/590069788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b84131810 0x7f5b841c6370 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400c480 tx=0x7f5b7400c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:06.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.610+0000 7f5b72ffd700 1 -- 172.21.15.3:0/590069788 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b740054e0 con 0x7f5b84131810 2023-10-03T10:18:06.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.610+0000 7f5b72ffd700 1 -- 172.21.15.3:0/590069788 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7400ce40 con 0x7f5b84131810 2023-10-03T10:18:06.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.610+0000 7f5b88808700 1 -- 172.21.15.3:0/590069788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b78009550 con 0x7f5b84131810 2023-10-03T10:18:06.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.610+0000 7f5b72ffd700 1 -- 172.21.15.3:0/590069788 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b740038c0 con 0x7f5b84131810 2023-10-03T10:18:06.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.610+0000 7f5b88808700 1 -- 172.21.15.3:0/590069788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b8406eb00 con 0x7f5b84131810 2023-10-03T10:18:06.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.611+0000 7f5b72ffd700 1 -- 172.21.15.3:0/590069788 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5b74003a20 con 0x7f5b84131810 2023-10-03T10:18:06.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.611+0000 7f5b72ffd700 1 -- 172.21.15.3:0/590069788 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5b7403eaa0 con 0x7f5b84131810 2023-10-03T10:18:06.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.611+0000 7f5b88808700 1 -- 172.21.15.3:0/590069788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b8411c040 con 0x7f5b84131810 2023-10-03T10:18:06.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.616+0000 7f5b72ffd700 1 -- 172.21.15.3:0/590069788 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5b74017080 con 0x7f5b84131810 2023-10-03T10:18:06.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.651+0000 7ff90260a700 1 Processor -- start 2023-10-03T10:18:06.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.651+0000 7ff90260a700 1 -- start start 2023-10-03T10:18:06.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.652+0000 7ff90260a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8fc0a0a30 0x7ff8fc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:06.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.652+0000 7ff90260a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff8fc0a1340 0x7ff8fc133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:06.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.652+0000 7ff90260a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8fc133fa0 0x7ff8fc134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:06.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.652+0000 7ff90260a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8fc09edd0 con 0x7ff8fc0a1340 2023-10-03T10:18:06.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.652+0000 7ff90260a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc09e1f0 con 0x7ff8fc0a0a30 2023-10-03T10:18:06.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.652+0000 7ff90260a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc09dc50 con 0x7ff8fc133fa0 2023-10-03T10:18:06.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.652+0000 7ff8fbfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8fc0a0a30 0x7ff8fc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:06.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.652+0000 7ff8fb7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff8fc0a1340 0x7ff8fc133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:06.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.652+0000 7ff8fbfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8fc0a0a30 0x7ff8fc0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41942/0 (socket says 172.21.15.3:41942) 2023-10-03T10:18:06.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.652+0000 7ff8fbfff700 1 -- 172.21.15.3:0/522705020 learned_addr learned my addr 172.21.15.3:0/522705020 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:06.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.652+0000 7ff900ba7700 1 --2- 172.21.15.3:0/522705020 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8fc133fa0 0x7ff8fc134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:06.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.653+0000 7ff8fb7fe700 1 -- 172.21.15.3:0/522705020 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8fc133fa0 msgr2=0x7ff8fc134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:06.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.653+0000 7ff8fb7fe700 1 --2- 172.21.15.3:0/522705020 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8fc133fa0 0x7ff8fc134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.653+0000 7ff8fb7fe700 1 -- 172.21.15.3:0/522705020 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8fc0a0a30 msgr2=0x7ff8fc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:06.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.653+0000 7ff8fb7fe700 1 --2- 172.21.15.3:0/522705020 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8fc0a0a30 0x7ff8fc0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.653+0000 7ff8fb7fe700 1 -- 172.21.15.3:0/522705020 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8fc0a1fe0 con 0x7ff8fc0a1340 2023-10-03T10:18:06.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.653+0000 7ff900ba7700 1 --2- 172.21.15.3:0/522705020 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8fc133fa0 0x7ff8fc134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:06.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.653+0000 7ff8fb7fe700 1 --2- 172.21.15.3:0/522705020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8fc0a1340 0x7ff8fc133a60 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009840 tx=0x7ff8f0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64c2bb84c672714 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:06.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.653+0000 7ff8faffd700 1 -- 172.21.15.3:0/522705020 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0011830 con 0x7ff8fc0a1340 2023-10-03T10:18:06.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.653+0000 7ff8faffd700 1 -- 172.21.15.3:0/522705020 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f0011990 con 0x7ff8fc0a1340 2023-10-03T10:18:06.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.653+0000 7ff8faffd700 1 -- 172.21.15.3:0/522705020 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0011d20 con 0x7ff8fc0a1340 2023-10-03T10:18:06.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.654+0000 7ff90260a700 1 -- 172.21.15.3:0/522705020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8fc0a1340 msgr2=0x7ff8fc133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:06.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.654+0000 7ff90260a700 1 --2- 172.21.15.3:0/522705020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8fc0a1340 0x7ff8fc133a60 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009840 tx=0x7ff8f0009b50 comp rx=0 tx=0).stop 2023-10-03T10:18:06.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.654+0000 7ff90260a700 1 -- 172.21.15.3:0/522705020 shutdown_connections 2023-10-03T10:18:06.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.654+0000 7ff90260a700 1 --2- 172.21.15.3:0/522705020 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8fc133fa0 0x7ff8fc134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.654+0000 7ff90260a700 1 --2- 172.21.15.3:0/522705020 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8fc0a0a30 0x7ff8fc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.654+0000 7ff90260a700 1 --2- 172.21.15.3:0/522705020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8fc0a1340 0x7ff8fc133a60 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.654+0000 7ff90260a700 1 -- 172.21.15.3:0/522705020 >> 172.21.15.3:0/522705020 conn(0x7ff8fc0a6550 msgr2=0x7ff8fc0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:06.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.654+0000 7ff90260a700 1 -- 172.21.15.3:0/522705020 shutdown_connections 2023-10-03T10:18:06.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.654+0000 7ff90260a700 1 -- 172.21.15.3:0/522705020 wait complete. 2023-10-03T10:18:06.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.655+0000 7ff90260a700 1 Processor -- start 2023-10-03T10:18:06.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.655+0000 7ff90260a700 1 -- start start 2023-10-03T10:18:06.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.655+0000 7ff90260a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8fc0a0a30 0x7ff8fc1205e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:06.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.656+0000 7ff90260a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8fc0a1340 0x7ff8fc120b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:06.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.656+0000 7ff90260a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8fc133fa0 0x7ff8fc121060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:06.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.656+0000 7ff90260a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc123e20 con 0x7ff8fc133fa0 2023-10-03T10:18:06.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.656+0000 7ff90260a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc0a3510 con 0x7ff8fc0a1340 2023-10-03T10:18:06.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.656+0000 7ff90260a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc122300 con 0x7ff8fc0a0a30 2023-10-03T10:18:06.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.656+0000 7ff8fbfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8fc0a0a30 0x7ff8fc1205e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:06.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.656+0000 7ff900ba7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8fc133fa0 0x7ff8fc121060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:06.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.656+0000 7ff8fbfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8fc0a0a30 0x7ff8fc1205e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47754/0 (socket says 172.21.15.3:47754) 2023-10-03T10:18:06.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.656+0000 7ff8fbfff700 1 -- 172.21.15.3:0/1060874255 learned_addr learned my addr 172.21.15.3:0/1060874255 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:06.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.656+0000 7ff8fb7fe700 1 --2- 172.21.15.3:0/1060874255 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8fc0a1340 0x7ff8fc120b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:06.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.656+0000 7ff900ba7700 1 -- 172.21.15.3:0/1060874255 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8fc0a0a30 msgr2=0x7ff8fc1205e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:06.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.656+0000 7ff900ba7700 1 --2- 172.21.15.3:0/1060874255 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8fc0a0a30 0x7ff8fc1205e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.656+0000 7ff900ba7700 1 -- 172.21.15.3:0/1060874255 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8fc0a1340 msgr2=0x7ff8fc120b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:06.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.656+0000 7ff900ba7700 1 --2- 172.21.15.3:0/1060874255 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8fc0a1340 0x7ff8fc120b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.656+0000 7ff900ba7700 1 -- 172.21.15.3:0/1060874255 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f4009a70 con 0x7ff8fc133fa0 2023-10-03T10:18:06.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.656+0000 7ff900ba7700 1 --2- 172.21.15.3:0/1060874255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8fc133fa0 0x7ff8fc121060 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7ff8f400efc0 tx=0x7ff8f400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:06.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.656+0000 7ff8f8ff9700 1 -- 172.21.15.3:0/1060874255 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f400ca90 con 0x7ff8fc133fa0 2023-10-03T10:18:06.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.657+0000 7ff8f8ff9700 1 -- 172.21.15.3:0/1060874255 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f400f040 con 0x7ff8fc133fa0 2023-10-03T10:18:06.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.657+0000 7ff8f8ff9700 1 -- 172.21.15.3:0/1060874255 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f400ce30 con 0x7ff8fc133fa0 2023-10-03T10:18:06.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.657+0000 7ff90260a700 1 -- 172.21.15.3:0/1060874255 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f00094a0 con 0x7ff8fc133fa0 2023-10-03T10:18:06.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.657+0000 7ff90260a700 1 -- 172.21.15.3:0/1060874255 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8fc124890 con 0x7ff8fc133fa0 2023-10-03T10:18:06.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.657+0000 7ff8f8ff9700 1 -- 172.21.15.3:0/1060874255 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff8f4014480 con 0x7ff8fc133fa0 2023-10-03T10:18:06.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.658+0000 7ff8f8ff9700 1 -- 172.21.15.3:0/1060874255 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff8f403fbc0 con 0x7ff8fc133fa0 2023-10-03T10:18:06.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.658+0000 7ff90260a700 1 -- 172.21.15.3:0/1060874255 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8fc074ea0 con 0x7ff8fc133fa0 2023-10-03T10:18:06.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.663+0000 7ff8f8ff9700 1 -- 172.21.15.3:0/1060874255 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff8f4012930 con 0x7ff8fc133fa0 2023-10-03T10:18:06.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.719+0000 7f9937fae700 1 Processor -- start 2023-10-03T10:18:06.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.720+0000 7f9937fae700 1 -- start start 2023-10-03T10:18:06.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.720+0000 7f9937fae700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f99300a0090 0x7f9930136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:06.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.720+0000 7f9937fae700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f993009d130 0x7f9930139170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:06.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.720+0000 7f9937fae700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f993009ba70 con 0x7f993009c650 2023-10-03T10:18:06.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.720+0000 7f9937fae700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993009e8f0 con 0x7f993009d130 2023-10-03T10:18:06.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.720+0000 7f9937fae700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99300a0760 con 0x7f99300a0090 2023-10-03T10:18:06.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.720+0000 7f9935d4a700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f993009c650 0x7f993009ca20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50566/0 (socket says 172.21.15.3:50566) 2023-10-03T10:18:06.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.720+0000 7f9935d4a700 1 -- 172.21.15.3:0/147611330 learned_addr learned my addr 172.21.15.3:0/147611330 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:06.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.720+0000 7f9935549700 1 --2- 172.21.15.3:0/147611330 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f99300a0090 0x7f9930136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:06.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.720+0000 7f993654b700 1 --2- 172.21.15.3:0/147611330 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f993009d130 0x7f9930139170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:06.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.721+0000 7f9934d48700 1 -- 172.21.15.3:0/147611330 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4228172053 0 0) 0x7f993009ba70 con 0x7f993009c650 2023-10-03T10:18:06.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.721+0000 7f9934d48700 1 -- 172.21.15.3:0/147611330 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f991c0035b0 con 0x7f993009c650 2023-10-03T10:18:06.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.721+0000 7f9935549700 1 -- 172.21.15.3:0/147611330 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f993009d130 msgr2=0x7f9930139170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:06.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.721+0000 7f9935549700 1 --2- 172.21.15.3:0/147611330 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f993009d130 0x7f9930139170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.721+0000 7f9935549700 1 -- 172.21.15.3:0/147611330 >> v1:172.21.15.3:6789/0 conn(0x7f993009c650 legacy=0x7f993009ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:06.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.721+0000 7f9935549700 1 -- 172.21.15.3:0/147611330 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993009a080 con 0x7f99300a0090 2023-10-03T10:18:06.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.721+0000 7f993654b700 1 --2- 172.21.15.3:0/147611330 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f993009d130 0x7f9930139170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:06.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.721+0000 7f9935549700 1 --2- 172.21.15.3:0/147611330 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f99300a0090 0x7f9930136c20 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9920009870 tx=0x7f9920009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91e125a2e159ab4c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:06.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.721+0000 7f9934d48700 1 -- 172.21.15.3:0/147611330 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9920011830 con 0x7f99300a0090 2023-10-03T10:18:06.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.721+0000 7f9934d48700 1 -- 172.21.15.3:0/147611330 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9920011990 con 0x7f99300a0090 2023-10-03T10:18:06.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.721+0000 7f9934d48700 1 -- 172.21.15.3:0/147611330 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9920011d20 con 0x7f99300a0090 2023-10-03T10:18:06.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.722+0000 7f9937fae700 1 -- 172.21.15.3:0/147611330 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f99300a0090 msgr2=0x7f9930136c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:06.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.722+0000 7f9937fae700 1 --2- 172.21.15.3:0/147611330 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f99300a0090 0x7f9930136c20 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9920009870 tx=0x7f9920009b80 comp rx=0 tx=0).stop 2023-10-03T10:18:06.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.722+0000 7f9937fae700 1 -- 172.21.15.3:0/147611330 shutdown_connections 2023-10-03T10:18:06.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.722+0000 7f9937fae700 1 --2- 172.21.15.3:0/147611330 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f993009d130 0x7f9930139170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.722+0000 7f9937fae700 1 --2- 172.21.15.3:0/147611330 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f99300a0090 0x7f9930136c20 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.722+0000 7f9937fae700 1 -- 172.21.15.3:0/147611330 >> 172.21.15.3:0/147611330 conn(0x7f99300a3f70 msgr2=0x7f99300a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:06.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.722+0000 7f9937fae700 1 -- 172.21.15.3:0/147611330 shutdown_connections 2023-10-03T10:18:06.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.722+0000 7f9937fae700 1 -- 172.21.15.3:0/147611330 wait complete. 2023-10-03T10:18:06.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.724+0000 7f9937fae700 1 Processor -- start 2023-10-03T10:18:06.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.724+0000 7f9937fae700 1 -- start start 2023-10-03T10:18:06.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.724+0000 7f9937fae700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f993009c650 0x7f9930135210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:06.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.724+0000 7f9937fae700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f993009d130 0x7f9930135750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:06.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.724+0000 7f9937fae700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f99300a0090 0x7f9930135c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:06.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.724+0000 7f9937fae700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993009d9f0 con 0x7f993009c650 2023-10-03T10:18:06.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.724+0000 7f9937fae700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9930137cf0 con 0x7f993009d130 2023-10-03T10:18:06.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.725+0000 7f9937fae700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99300a3760 con 0x7f99300a0090 2023-10-03T10:18:06.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.725+0000 7f9935549700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f993009d130 0x7f9930135750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:06.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.725+0000 7f9935549700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f993009d130 0x7f9930135750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41962/0 (socket says 172.21.15.3:41962) 2023-10-03T10:18:06.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.725+0000 7f9935549700 1 -- 172.21.15.3:0/1784558634 learned_addr learned my addr 172.21.15.3:0/1784558634 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:06.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.725+0000 7f9935d4a700 1 --2- 172.21.15.3:0/1784558634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f993009c650 0x7f9930135210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:06.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.725+0000 7f993654b700 1 --2- 172.21.15.3:0/1784558634 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f99300a0090 0x7f9930135c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:06.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.725+0000 7f9935549700 1 -- 172.21.15.3:0/1784558634 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f99300a0090 msgr2=0x7f9930135c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:06.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.725+0000 7f9935549700 1 --2- 172.21.15.3:0/1784558634 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f99300a0090 0x7f9930135c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.725+0000 7f9935549700 1 -- 172.21.15.3:0/1784558634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f993009c650 msgr2=0x7f9930135210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:06.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.725+0000 7f9935549700 1 --2- 172.21.15.3:0/1784558634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f993009c650 0x7f9930135210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.725+0000 7f9935549700 1 -- 172.21.15.3:0/1784558634 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9928009a70 con 0x7f993009d130 2023-10-03T10:18:06.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.725+0000 7f993654b700 1 --2- 172.21.15.3:0/1784558634 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f99300a0090 0x7f9930135c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:06.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.725+0000 7f9935549700 1 --2- 172.21.15.3:0/1784558634 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f993009d130 0x7f9930135750 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9920009bc0 tx=0x7f9920004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:06.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.726+0000 7f99267fc700 1 -- 172.21.15.3:0/1784558634 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9920004450 con 0x7f993009d130 2023-10-03T10:18:06.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.726+0000 7f99267fc700 1 -- 172.21.15.3:0/1784558634 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99200045b0 con 0x7f993009d130 2023-10-03T10:18:06.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.726+0000 7f9937fae700 1 -- 172.21.15.3:0/1784558634 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99200094a0 con 0x7f993009d130 2023-10-03T10:18:06.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.726+0000 7f99267fc700 1 -- 172.21.15.3:0/1784558634 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99200038e0 con 0x7f993009d130 2023-10-03T10:18:06.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.726+0000 7f9937fae700 1 -- 172.21.15.3:0/1784558634 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993011a750 con 0x7f993009d130 2023-10-03T10:18:06.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.726+0000 7f99267fc700 1 -- 172.21.15.3:0/1784558634 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9920003a40 con 0x7f993009d130 2023-10-03T10:18:06.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.727+0000 7f99267fc700 1 -- 172.21.15.3:0/1784558634 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f99200407a0 con 0x7f993009d130 2023-10-03T10:18:06.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.727+0000 7f9937fae700 1 -- 172.21.15.3:0/1784558634 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99300747f0 con 0x7f993009d130 2023-10-03T10:18:06.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.732+0000 7f99267fc700 1 -- 172.21.15.3:0/1784558634 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9920015080 con 0x7f993009d130 2023-10-03T10:18:06.761 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:06.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.865+0000 7f5b88808700 1 -- 172.21.15.3:0/590069788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5b84074ea0 con 0x7f5b84131810 2023-10-03T10:18:06.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.865+0000 7f5b72ffd700 1 -- 172.21.15.3:0/590069788 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5b74019cd0 con 0x7f5b84131810 2023-10-03T10:18:06.865 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:06.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.867+0000 7f5b88808700 1 -- 172.21.15.3:0/590069788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b84131810 msgr2=0x7f5b841c6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:06.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.867+0000 7f5b88808700 1 --2- 172.21.15.3:0/590069788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b84131810 0x7f5b841c6370 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400c480 tx=0x7f5b7400c790 comp rx=0 tx=0).stop 2023-10-03T10:18:06.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.867+0000 7f5b88808700 1 -- 172.21.15.3:0/590069788 shutdown_connections 2023-10-03T10:18:06.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.867+0000 7f5b88808700 1 --2- 172.21.15.3:0/590069788 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b84136f50 0x7f5b84118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.867+0000 7f5b88808700 1 --2- 172.21.15.3:0/590069788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b84131810 0x7f5b841c6370 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.867+0000 7f5b88808700 1 --2- 172.21.15.3:0/590069788 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5b84134630 0x7f5b841c68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.867+0000 7f5b88808700 1 -- 172.21.15.3:0/590069788 >> 172.21.15.3:0/590069788 conn(0x7f5b840a2aa0 msgr2=0x7f5b84130060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:06.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.867+0000 7f5b88808700 1 -- 172.21.15.3:0/590069788 shutdown_connections 2023-10-03T10:18:06.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.867+0000 7f5b88808700 1 -- 172.21.15.3:0/590069788 wait complete. 2023-10-03T10:18:06.882 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:06.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.902+0000 7fa424d6b700 1 Processor -- start 2023-10-03T10:18:06.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.902+0000 7fa424d6b700 1 -- start start 2023-10-03T10:18:06.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.903+0000 7fa424d6b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa420130540 0x7fa420130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:06.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.903+0000 7fa424d6b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa42006fcf0 0x7fa420138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:06.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.903+0000 7fa424d6b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4200716a0 con 0x7fa420130e50 2023-10-03T10:18:06.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.903+0000 7fa424d6b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa42009a390 con 0x7fa42006fcf0 2023-10-03T10:18:06.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.903+0000 7fa424d6b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa420071290 con 0x7fa420130540 2023-10-03T10:18:06.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.904+0000 7fa41e59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa420130540 0x7fa420130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:06.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.904+0000 7fa41e59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa420130540 0x7fa420130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47772/0 (socket says 172.21.15.3:47772) 2023-10-03T10:18:06.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.904+0000 7fa41e59c700 1 -- 172.21.15.3:0/1105503811 learned_addr learned my addr 172.21.15.3:0/1105503811 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:06.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.904+0000 7fa41dd9b700 1 --1- 172.21.15.3:0/1105503811 >> v1:172.21.15.3:6789/0 conn(0x7fa420130e50 0x7fa42006f5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50570/0 (socket says 172.21.15.3:50570) 2023-10-03T10:18:06.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.904+0000 7fa41ed9d700 1 --2- 172.21.15.3:0/1105503811 >> v2:172.21.15.3:3300/0 conn(0x7fa42006fcf0 0x7fa420138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:06.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.904+0000 7fa41e59c700 1 -- 172.21.15.3:0/1105503811 >> v2:172.21.15.3:3300/0 conn(0x7fa42006fcf0 msgr2=0x7fa420138570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:06.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.904+0000 7fa41e59c700 1 --2- 172.21.15.3:0/1105503811 >> v2:172.21.15.3:3300/0 conn(0x7fa42006fcf0 0x7fa420138570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.904+0000 7fa41e59c700 1 -- 172.21.15.3:0/1105503811 >> v1:172.21.15.3:6789/0 conn(0x7fa420130e50 legacy=0x7fa42006f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:06.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.904+0000 7fa41e59c700 1 -- 172.21.15.3:0/1105503811 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa420130340 con 0x7fa420130540 2023-10-03T10:18:06.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.904+0000 7fa41ed9d700 1 --2- 172.21.15.3:0/1105503811 >> v2:172.21.15.3:3300/0 conn(0x7fa42006fcf0 0x7fa420138570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:06.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.904+0000 7fa41e59c700 1 --2- 172.21.15.3:0/1105503811 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa420130540 0x7fa420130910 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa418009920 tx=0x7fa418009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43b6873679f5454f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:06.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.904+0000 7fa41d59a700 1 -- 172.21.15.3:0/1105503811 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa418011830 con 0x7fa420130540 2023-10-03T10:18:06.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.904+0000 7fa41d59a700 1 -- 172.21.15.3:0/1105503811 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa418011990 con 0x7fa420130540 2023-10-03T10:18:06.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.904+0000 7fa41d59a700 1 -- 172.21.15.3:0/1105503811 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa418011d20 con 0x7fa420130540 2023-10-03T10:18:06.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.905+0000 7fa424d6b700 1 -- 172.21.15.3:0/1105503811 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa420130540 msgr2=0x7fa420130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:06.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.905+0000 7fa424d6b700 1 --2- 172.21.15.3:0/1105503811 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa420130540 0x7fa420130910 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa418009920 tx=0x7fa418009c30 comp rx=0 tx=0).stop 2023-10-03T10:18:06.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.905+0000 7fa424d6b700 1 -- 172.21.15.3:0/1105503811 shutdown_connections 2023-10-03T10:18:06.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.905+0000 7fa424d6b700 1 --2- 172.21.15.3:0/1105503811 >> v2:172.21.15.3:3300/0 conn(0x7fa42006fcf0 0x7fa420138570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.905+0000 7fa424d6b700 1 --2- 172.21.15.3:0/1105503811 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa420130540 0x7fa420130910 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.905+0000 7fa424d6b700 1 -- 172.21.15.3:0/1105503811 >> 172.21.15.3:0/1105503811 conn(0x7fa4200a2aa0 msgr2=0x7fa4200a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:06.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.905+0000 7fa424d6b700 1 -- 172.21.15.3:0/1105503811 shutdown_connections 2023-10-03T10:18:06.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.905+0000 7fa424d6b700 1 -- 172.21.15.3:0/1105503811 wait complete. 2023-10-03T10:18:06.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.906+0000 7fa424d6b700 1 Processor -- start 2023-10-03T10:18:06.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.906+0000 7fa424d6b700 1 -- start start 2023-10-03T10:18:06.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.907+0000 7fa424d6b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa42006fcf0 0x7fa42011fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:06.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.907+0000 7fa424d6b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa420130540 0x7fa4201200e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:06.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.907+0000 7fa424d6b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa420130e50 0x7fa420120620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:06.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.907+0000 7fa424d6b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa420122e40 con 0x7fa42006fcf0 2023-10-03T10:18:06.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.907+0000 7fa424d6b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42012daa0 con 0x7fa420130540 2023-10-03T10:18:06.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.907+0000 7fa424d6b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa4200718e0 con 0x7fa420130e50 2023-10-03T10:18:06.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.907+0000 7fa41ed9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa420130e50 0x7fa420120620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:06.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.907+0000 7fa41dd9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa420130540 0x7fa4201200e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:06.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.907+0000 7fa41ed9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa420130e50 0x7fa420120620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47782/0 (socket says 172.21.15.3:47782) 2023-10-03T10:18:06.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.907+0000 7fa41ed9d700 1 -- 172.21.15.3:0/3693709700 learned_addr learned my addr 172.21.15.3:0/3693709700 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:06.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.907+0000 7fa41e59c700 1 --2- 172.21.15.3:0/3693709700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa42006fcf0 0x7fa42011fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:06.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.908+0000 7fa41ed9d700 1 -- 172.21.15.3:0/3693709700 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa420130540 msgr2=0x7fa4201200e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:06.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.908+0000 7fa41ed9d700 1 --2- 172.21.15.3:0/3693709700 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa420130540 0x7fa4201200e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.908+0000 7fa41ed9d700 1 -- 172.21.15.3:0/3693709700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa42006fcf0 msgr2=0x7fa42011fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:06.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.908+0000 7fa41ed9d700 1 --2- 172.21.15.3:0/3693709700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa42006fcf0 0x7fa42011fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.908+0000 7fa41ed9d700 1 -- 172.21.15.3:0/3693709700 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41000bc80 con 0x7fa420130e50 2023-10-03T10:18:06.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.908+0000 7fa41dd9b700 1 --2- 172.21.15.3:0/3693709700 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa420130540 0x7fa4201200e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:06.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.908+0000 7fa41ed9d700 1 --2- 172.21.15.3:0/3693709700 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa420130e50 0x7fa420120620 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa410007fc0 tx=0x7fa41000e3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:06.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.908+0000 7fa416ffd700 1 -- 172.21.15.3:0/3693709700 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa410010040 con 0x7fa420130e50 2023-10-03T10:18:06.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.908+0000 7fa416ffd700 1 -- 172.21.15.3:0/3693709700 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa410003a20 con 0x7fa420130e50 2023-10-03T10:18:06.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.908+0000 7fa424d6b700 1 -- 172.21.15.3:0/3693709700 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa418009550 con 0x7fa420130e50 2023-10-03T10:18:06.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.908+0000 7fa416ffd700 1 -- 172.21.15.3:0/3693709700 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa410003d90 con 0x7fa420130e50 2023-10-03T10:18:06.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.908+0000 7fa424d6b700 1 -- 172.21.15.3:0/3693709700 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa420123630 con 0x7fa420130e50 2023-10-03T10:18:06.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.909+0000 7fa416ffd700 1 -- 172.21.15.3:0/3693709700 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa41000e9a0 con 0x7fa420130e50 2023-10-03T10:18:06.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.909+0000 7fa416ffd700 1 -- 172.21.15.3:0/3693709700 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa41003f930 con 0x7fa420130e50 2023-10-03T10:18:06.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.909+0000 7fa424d6b700 1 -- 172.21.15.3:0/3693709700 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa420123270 con 0x7fa420130e50 2023-10-03T10:18:06.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.914+0000 7fa416ffd700 1 -- 172.21.15.3:0/3693709700 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa41001d770 con 0x7fa420130e50 2023-10-03T10:18:06.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.992+0000 7ff90260a700 1 -- 172.21.15.3:0/1060874255 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff8fc0a28e0 con 0x7ff8fc133fa0 2023-10-03T10:18:06.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.992+0000 7ff8f8ff9700 1 -- 172.21.15.3:0/1060874255 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff8f4012340 con 0x7ff8fc133fa0 2023-10-03T10:18:06.993 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:06.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.994+0000 7ff90260a700 1 -- 172.21.15.3:0/1060874255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8fc133fa0 msgr2=0x7ff8fc121060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:06.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.994+0000 7ff90260a700 1 --2- 172.21.15.3:0/1060874255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8fc133fa0 0x7ff8fc121060 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7ff8f400efc0 tx=0x7ff8f400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.994+0000 7ff90260a700 1 -- 172.21.15.3:0/1060874255 shutdown_connections 2023-10-03T10:18:06.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.994+0000 7ff90260a700 1 --2- 172.21.15.3:0/1060874255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8fc133fa0 0x7ff8fc121060 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.994+0000 7ff90260a700 1 --2- 172.21.15.3:0/1060874255 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff8fc0a0a30 0x7ff8fc1205e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.994+0000 7ff90260a700 1 --2- 172.21.15.3:0/1060874255 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff8fc0a1340 0x7ff8fc120b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:06.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.995+0000 7ff90260a700 1 -- 172.21.15.3:0/1060874255 >> 172.21.15.3:0/1060874255 conn(0x7ff8fc0a6550 msgr2=0x7ff8fc1bafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:06.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.995+0000 7ff90260a700 1 -- 172.21.15.3:0/1060874255 shutdown_connections 2023-10-03T10:18:06.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:06.995+0000 7ff90260a700 1 -- 172.21.15.3:0/1060874255 wait complete. 2023-10-03T10:18:07.013 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:07.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:07.059+0000 7f9937fae700 1 -- 172.21.15.3:0/1784558634 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9930072840 con 0x7f993009d130 2023-10-03T10:18:07.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:07.059+0000 7f99267fc700 1 -- 172.21.15.3:0/1784558634 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f992002e970 con 0x7f993009d130 2023-10-03T10:18:07.059 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:07.061+0000 7f9937fae700 1 -- 172.21.15.3:0/1784558634 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f993009d130 msgr2=0x7f9930135750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:07.061+0000 7f9937fae700 1 --2- 172.21.15.3:0/1784558634 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f993009d130 0x7f9930135750 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9920009bc0 tx=0x7f9920004750 comp rx=0 tx=0).stop 2023-10-03T10:18:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:07.061+0000 7f9937fae700 1 -- 172.21.15.3:0/1784558634 shutdown_connections 2023-10-03T10:18:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:07.061+0000 7f9937fae700 1 --2- 172.21.15.3:0/1784558634 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f99300a0090 0x7f9930135c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:07.061+0000 7f9937fae700 1 --2- 172.21.15.3:0/1784558634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f993009c650 0x7f9930135210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:07.061+0000 7f9937fae700 1 --2- 172.21.15.3:0/1784558634 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f993009d130 0x7f9930135750 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:07.061+0000 7f9937fae700 1 -- 172.21.15.3:0/1784558634 >> 172.21.15.3:0/1784558634 conn(0x7f99300a3f70 msgr2=0x7f9930071820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:07.061+0000 7f9937fae700 1 -- 172.21.15.3:0/1784558634 shutdown_connections 2023-10-03T10:18:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:07.061+0000 7f9937fae700 1 -- 172.21.15.3:0/1784558634 wait complete. 2023-10-03T10:18:07.079 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:07.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:07.139+0000 7fa424d6b700 1 -- 172.21.15.3:0/3693709700 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa4201216e0 con 0x7fa420130e50 2023-10-03T10:18:07.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:07.139+0000 7fa416ffd700 1 -- 172.21.15.3:0/3693709700 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa4100143e0 con 0x7fa420130e50 2023-10-03T10:18:07.140 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:07.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:07.142+0000 7fa424d6b700 1 -- 172.21.15.3:0/3693709700 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa420130e50 msgr2=0x7fa420120620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:07.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:07.142+0000 7fa424d6b700 1 --2- 172.21.15.3:0/3693709700 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa420130e50 0x7fa420120620 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa410007fc0 tx=0x7fa41000e3a0 comp rx=0 tx=0).stop 2023-10-03T10:18:07.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:07.142+0000 7fa424d6b700 1 -- 172.21.15.3:0/3693709700 shutdown_connections 2023-10-03T10:18:07.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:07.142+0000 7fa424d6b700 1 --2- 172.21.15.3:0/3693709700 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa420130e50 0x7fa420120620 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:07.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:07.142+0000 7fa424d6b700 1 --2- 172.21.15.3:0/3693709700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa42006fcf0 0x7fa42011fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:07.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:07.142+0000 7fa424d6b700 1 --2- 172.21.15.3:0/3693709700 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa420130540 0x7fa4201200e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:07.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:07.142+0000 7fa424d6b700 1 -- 172.21.15.3:0/3693709700 >> 172.21.15.3:0/3693709700 conn(0x7fa4200a2aa0 msgr2=0x7fa42012e130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:07.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:07.142+0000 7fa424d6b700 1 -- 172.21.15.3:0/3693709700 shutdown_connections 2023-10-03T10:18:07.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:07.142+0000 7fa424d6b700 1 -- 172.21.15.3:0/3693709700 wait complete. 2023-10-03T10:18:07.156 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:07.883 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:08.015 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:08.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.029+0000 7f8ae09e1700 1 Processor -- start 2023-10-03T10:18:08.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.029+0000 7f8ae09e1700 1 -- start start 2023-10-03T10:18:08.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.030+0000 7f8ae09e1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8adc09d6f0 0x7f8adc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:08.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.030+0000 7f8ae09e1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8adc09e000 0x7f8adc130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:08.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.030+0000 7f8ae09e1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8adc09e410 0x7f8adc130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:08.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.030+0000 7f8ae09e1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8adc09f520 con 0x7f8adc09e410 2023-10-03T10:18:08.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.030+0000 7f8ae09e1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc09f130 con 0x7f8adc09d6f0 2023-10-03T10:18:08.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.030+0000 7f8ae09e1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc09e930 con 0x7f8adc09e000 2023-10-03T10:18:08.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.030+0000 7f8ada59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8adc09d6f0 0x7f8adc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:08.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.030+0000 7f8ad9d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8adc09e000 0x7f8adc130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:08.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.030+0000 7f8ada59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8adc09d6f0 0x7f8adc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41984/0 (socket says 172.21.15.3:41984) 2023-10-03T10:18:08.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.030+0000 7f8ada59c700 1 -- 172.21.15.3:0/1390403095 learned_addr learned my addr 172.21.15.3:0/1390403095 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:08.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.030+0000 7f8adad9d700 1 --2- 172.21.15.3:0/1390403095 >> v2:172.21.15.3:3300/0 conn(0x7f8adc09e410 0x7f8adc130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:08.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.031+0000 7f8ada59c700 1 -- 172.21.15.3:0/1390403095 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8adc09e000 msgr2=0x7f8adc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:08.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.031+0000 7f8ada59c700 1 --2- 172.21.15.3:0/1390403095 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8adc09e000 0x7f8adc130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.031+0000 7f8ada59c700 1 -- 172.21.15.3:0/1390403095 >> v2:172.21.15.3:3300/0 conn(0x7f8adc09e410 msgr2=0x7f8adc130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:08.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.031+0000 7f8ada59c700 1 --2- 172.21.15.3:0/1390403095 >> v2:172.21.15.3:3300/0 conn(0x7f8adc09e410 0x7f8adc130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.031+0000 7f8ada59c700 1 -- 172.21.15.3:0/1390403095 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8adc09b2d0 con 0x7f8adc09d6f0 2023-10-03T10:18:08.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.031+0000 7f8ad9d9b700 1 --2- 172.21.15.3:0/1390403095 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8adc09e000 0x7f8adc130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:08.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.031+0000 7f8ada59c700 1 --2- 172.21.15.3:0/1390403095 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8adc09d6f0 0x7f8adc09dac0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009e80 tx=0x7f8acc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87cdf427257adf1e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:08.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.031+0000 7f8ad959a700 1 -- 172.21.15.3:0/1390403095 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc00e070 con 0x7f8adc09d6f0 2023-10-03T10:18:08.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.031+0000 7f8ad959a700 1 -- 172.21.15.3:0/1390403095 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8acc00b8d0 con 0x7f8adc09d6f0 2023-10-03T10:18:08.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.031+0000 7f8ad959a700 1 -- 172.21.15.3:0/1390403095 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc00bc80 con 0x7f8adc09d6f0 2023-10-03T10:18:08.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.032+0000 7f8ae09e1700 1 -- 172.21.15.3:0/1390403095 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8adc09d6f0 msgr2=0x7f8adc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:08.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.032+0000 7f8ae09e1700 1 --2- 172.21.15.3:0/1390403095 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8adc09d6f0 0x7f8adc09dac0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009e80 tx=0x7f8acc00b330 comp rx=0 tx=0).stop 2023-10-03T10:18:08.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.032+0000 7f8ae09e1700 1 -- 172.21.15.3:0/1390403095 shutdown_connections 2023-10-03T10:18:08.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.032+0000 7f8ae09e1700 1 --2- 172.21.15.3:0/1390403095 >> v2:172.21.15.3:3300/0 conn(0x7f8adc09e410 0x7f8adc130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.032+0000 7f8ae09e1700 1 --2- 172.21.15.3:0/1390403095 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8adc09d6f0 0x7f8adc09dac0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.032+0000 7f8ae09e1700 1 --2- 172.21.15.3:0/1390403095 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8adc09e000 0x7f8adc130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.032+0000 7f8ae09e1700 1 -- 172.21.15.3:0/1390403095 >> 172.21.15.3:0/1390403095 conn(0x7f8adc066f50 msgr2=0x7f8adc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:08.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.032+0000 7f8ae09e1700 1 -- 172.21.15.3:0/1390403095 shutdown_connections 2023-10-03T10:18:08.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.032+0000 7f8ae09e1700 1 -- 172.21.15.3:0/1390403095 wait complete. 2023-10-03T10:18:08.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.033+0000 7f8ae09e1700 1 Processor -- start 2023-10-03T10:18:08.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.033+0000 7f8ae09e1700 1 -- start start 2023-10-03T10:18:08.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.034+0000 7f8ae09e1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8adc09d6f0 0x7f8adc1330a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:08.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.034+0000 7f8ae09e1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8adc09e000 0x7f8adc1335e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:08.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.034+0000 7f8ae09e1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8adc09e410 0x7f8adc133b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:08.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.034+0000 7f8ae09e1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc137540 con 0x7f8adc09e410 2023-10-03T10:18:08.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.034+0000 7f8ae09e1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc072580 con 0x7f8adc09d6f0 2023-10-03T10:18:08.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.034+0000 7f8ae09e1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc09c700 con 0x7f8adc09e000 2023-10-03T10:18:08.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.034+0000 7f8ada59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8adc09d6f0 0x7f8adc1330a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:08.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.034+0000 7f8ad9d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8adc09e000 0x7f8adc1335e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:08.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.034+0000 7f8ada59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8adc09d6f0 0x7f8adc1330a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41988/0 (socket says 172.21.15.3:41988) 2023-10-03T10:18:08.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.034+0000 7f8ada59c700 1 -- 172.21.15.3:0/2195572618 learned_addr learned my addr 172.21.15.3:0/2195572618 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:08.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.034+0000 7f8adad9d700 1 --2- 172.21.15.3:0/2195572618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8adc09e410 0x7f8adc133b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:08.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.034+0000 7f8ad9d9b700 1 -- 172.21.15.3:0/2195572618 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8adc09d6f0 msgr2=0x7f8adc1330a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:08.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.034+0000 7f8ad9d9b700 1 --2- 172.21.15.3:0/2195572618 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8adc09d6f0 0x7f8adc1330a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.034+0000 7f8ad9d9b700 1 -- 172.21.15.3:0/2195572618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8adc09e410 msgr2=0x7f8adc133b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:08.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.035+0000 7f8ad9d9b700 1 --2- 172.21.15.3:0/2195572618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8adc09e410 0x7f8adc133b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.035+0000 7f8ad9d9b700 1 -- 172.21.15.3:0/2195572618 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad00094a0 con 0x7f8adc09e000 2023-10-03T10:18:08.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.035+0000 7f8adad9d700 1 --2- 172.21.15.3:0/2195572618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8adc09e410 0x7f8adc133b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:08.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.035+0000 7f8ada59c700 1 --2- 172.21.15.3:0/2195572618 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8adc09d6f0 0x7f8adc1330a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:08.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.035+0000 7f8ad9d9b700 1 --2- 172.21.15.3:0/2195572618 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8adc09e000 0x7f8adc1335e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000e980 tx=0x7f8ad000ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:08.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.035+0000 7f8acaffd700 1 -- 172.21.15.3:0/2195572618 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0004240 con 0x7f8adc09e000 2023-10-03T10:18:08.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.035+0000 7f8acaffd700 1 -- 172.21.15.3:0/2195572618 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad00043a0 con 0x7f8adc09e000 2023-10-03T10:18:08.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.035+0000 7f8acaffd700 1 -- 172.21.15.3:0/2195572618 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad00036a0 con 0x7f8adc09e000 2023-10-03T10:18:08.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.035+0000 7f8ae09e1700 1 -- 172.21.15.3:0/2195572618 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8acc009b30 con 0x7f8adc09e000 2023-10-03T10:18:08.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.035+0000 7f8ae09e1700 1 -- 172.21.15.3:0/2195572618 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8adc136c20 con 0x7f8adc09e000 2023-10-03T10:18:08.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.036+0000 7f8acaffd700 1 -- 172.21.15.3:0/2195572618 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8ad0003800 con 0x7f8adc09e000 2023-10-03T10:18:08.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.036+0000 7f8acaffd700 1 -- 172.21.15.3:0/2195572618 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8ad003f4e0 con 0x7f8adc09e000 2023-10-03T10:18:08.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.037+0000 7f8ae09e1700 1 -- 172.21.15.3:0/2195572618 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8adc1364f0 con 0x7f8adc09e000 2023-10-03T10:18:08.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.042+0000 7f8acaffd700 1 -- 172.21.15.3:0/2195572618 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8ad0013470 con 0x7f8adc09e000 2023-10-03T10:18:08.080 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:08.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.155+0000 7fbe48768700 1 Processor -- start 2023-10-03T10:18:08.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.155+0000 7fbe48768700 1 -- start start 2023-10-03T10:18:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.156+0000 7fbe48768700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe4006a770 0x7fbe4006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.156+0000 7fbe48768700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbe4006b080 0x7fbe40131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.156+0000 7fbe48768700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe4009ece0 con 0x7fbe40131b40 2023-10-03T10:18:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.156+0000 7fbe48768700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe4009e760 con 0x7fbe4006b080 2023-10-03T10:18:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.156+0000 7fbe48768700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4009e040 con 0x7fbe4006a770 2023-10-03T10:18:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.156+0000 7fbe45d03700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbe4006b080 0x7fbe40131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.156+0000 7fbe46504700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe4006a770 0x7fbe4006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:08.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.156+0000 7fbe45d03700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbe4006b080 0x7fbe40131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46744/0 (socket says 172.21.15.3:46744) 2023-10-03T10:18:08.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.156+0000 7fbe45d03700 1 -- 172.21.15.3:0/427057577 learned_addr learned my addr 172.21.15.3:0/427057577 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:08.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.156+0000 7fbe46504700 1 --2- 172.21.15.3:0/427057577 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe4006a770 0x7fbe4006ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41994/0 (socket says 172.21.15.3:41994) 2023-10-03T10:18:08.157 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:08.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.156+0000 7fbe46504700 1 -- 172.21.15.3:0/427057577 >> v2:172.21.15.3:3300/0 conn(0x7fbe4006b080 msgr2=0x7fbe40131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:08.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.156+0000 7fbe46504700 1 --2- 172.21.15.3:0/427057577 >> v2:172.21.15.3:3300/0 conn(0x7fbe4006b080 0x7fbe40131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.156+0000 7fbe46504700 1 -- 172.21.15.3:0/427057577 >> v1:172.21.15.3:6789/0 conn(0x7fbe40131b40 legacy=0x7fbe40138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:08.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.156+0000 7fbe46504700 1 -- 172.21.15.3:0/427057577 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4009f630 con 0x7fbe4006a770 2023-10-03T10:18:08.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.156+0000 7fbe45d03700 1 --2- 172.21.15.3:0/427057577 >> v2:172.21.15.3:3300/0 conn(0x7fbe4006b080 0x7fbe40131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:08.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.157+0000 7fbe46504700 1 --2- 172.21.15.3:0/427057577 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe4006a770 0x7fbe4006ab40 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c009840 tx=0x7fbe3c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9aeceb4304e1b4d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:08.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.157+0000 7fbe45502700 1 -- 172.21.15.3:0/427057577 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c011830 con 0x7fbe4006a770 2023-10-03T10:18:08.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.157+0000 7fbe45502700 1 -- 172.21.15.3:0/427057577 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3c011990 con 0x7fbe4006a770 2023-10-03T10:18:08.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.157+0000 7fbe45502700 1 -- 172.21.15.3:0/427057577 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c011d20 con 0x7fbe4006a770 2023-10-03T10:18:08.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.157+0000 7fbe48768700 1 -- 172.21.15.3:0/427057577 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe4006a770 msgr2=0x7fbe4006ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:08.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.157+0000 7fbe48768700 1 --2- 172.21.15.3:0/427057577 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe4006a770 0x7fbe4006ab40 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c009840 tx=0x7fbe3c009b50 comp rx=0 tx=0).stop 2023-10-03T10:18:08.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.158+0000 7fbe48768700 1 -- 172.21.15.3:0/427057577 shutdown_connections 2023-10-03T10:18:08.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.158+0000 7fbe48768700 1 --2- 172.21.15.3:0/427057577 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe4006a770 0x7fbe4006ab40 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.158+0000 7fbe48768700 1 --2- 172.21.15.3:0/427057577 >> v2:172.21.15.3:3300/0 conn(0x7fbe4006b080 0x7fbe40131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.158+0000 7fbe48768700 1 -- 172.21.15.3:0/427057577 >> 172.21.15.3:0/427057577 conn(0x7fbe40066f50 msgr2=0x7fbe4009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:08.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.158+0000 7fbe48768700 1 -- 172.21.15.3:0/427057577 shutdown_connections 2023-10-03T10:18:08.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.158+0000 7fbe48768700 1 -- 172.21.15.3:0/427057577 wait complete. 2023-10-03T10:18:08.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.159+0000 7fbe48768700 1 Processor -- start 2023-10-03T10:18:08.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.159+0000 7fbe48768700 1 -- start start 2023-10-03T10:18:08.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.159+0000 7fbe48768700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe4006a770 0x7fbe4011c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:08.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.159+0000 7fbe48768700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe4006b080 0x7fbe4011ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:08.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.159+0000 7fbe48768700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe40131b40 0x7fbe4011cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:08.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.159+0000 7fbe48768700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe401315d0 con 0x7fbe4006a770 2023-10-03T10:18:08.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.159+0000 7fbe48768700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe401200e0 con 0x7fbe4006b080 2023-10-03T10:18:08.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.159+0000 7fbe48768700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4011fa60 con 0x7fbe40131b40 2023-10-03T10:18:08.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.159+0000 7fbe45d03700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe4006b080 0x7fbe4011ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:08.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.159+0000 7fbe45d03700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe4006b080 0x7fbe4011ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42008/0 (socket says 172.21.15.3:42008) 2023-10-03T10:18:08.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.159+0000 7fbe45d03700 1 -- 172.21.15.3:0/3384928746 learned_addr learned my addr 172.21.15.3:0/3384928746 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:08.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.159+0000 7fbe46504700 1 --2- 172.21.15.3:0/3384928746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe4006a770 0x7fbe4011c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:08.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.160+0000 7fbe46d05700 1 --2- 172.21.15.3:0/3384928746 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe40131b40 0x7fbe4011cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:08.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.160+0000 7fbe46504700 1 -- 172.21.15.3:0/3384928746 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe40131b40 msgr2=0x7fbe4011cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:08.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.160+0000 7fbe46504700 1 --2- 172.21.15.3:0/3384928746 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe40131b40 0x7fbe4011cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.160+0000 7fbe46504700 1 -- 172.21.15.3:0/3384928746 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe4006b080 msgr2=0x7fbe4011ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:08.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.160+0000 7fbe46504700 1 --2- 172.21.15.3:0/3384928746 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe4006b080 0x7fbe4011ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.160+0000 7fbe46504700 1 -- 172.21.15.3:0/3384928746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe30009550 con 0x7fbe4006a770 2023-10-03T10:18:08.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.160+0000 7fbe46504700 1 --2- 172.21.15.3:0/3384928746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe4006a770 0x7fbe4011c4e0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c016000 tx=0x7fbe3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:08.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.160+0000 7fbe36ffd700 1 -- 172.21.15.3:0/3384928746 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c0036a0 con 0x7fbe4006a770 2023-10-03T10:18:08.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.160+0000 7fbe36ffd700 1 -- 172.21.15.3:0/3384928746 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3c01f070 con 0x7fbe4006a770 2023-10-03T10:18:08.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.160+0000 7fbe48768700 1 -- 172.21.15.3:0/3384928746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3c0094a0 con 0x7fbe4006a770 2023-10-03T10:18:08.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.160+0000 7fbe36ffd700 1 -- 172.21.15.3:0/3384928746 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c003a60 con 0x7fbe4006a770 2023-10-03T10:18:08.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.160+0000 7fbe48768700 1 -- 172.21.15.3:0/3384928746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe40120790 con 0x7fbe4006a770 2023-10-03T10:18:08.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.161+0000 7fbe36ffd700 1 -- 172.21.15.3:0/3384928746 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbe3c003bc0 con 0x7fbe4006a770 2023-10-03T10:18:08.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.161+0000 7fbe36ffd700 1 -- 172.21.15.3:0/3384928746 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fbe3c021070 con 0x7fbe4006a770 2023-10-03T10:18:08.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.162+0000 7fbe48768700 1 -- 172.21.15.3:0/3384928746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe40074ea0 con 0x7fbe4006a770 2023-10-03T10:18:08.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.167+0000 7fbe36ffd700 1 -- 172.21.15.3:0/3384928746 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbe3c02b020 con 0x7fbe4006a770 2023-10-03T10:18:08.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.221+0000 7fdbb8826700 1 Processor -- start 2023-10-03T10:18:08.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.221+0000 7fdbb8826700 1 -- start start 2023-10-03T10:18:08.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.222+0000 7fdbb8826700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdbb409d6f0 0x7fdbb409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:08.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.222+0000 7fdbb8826700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdbb409e000 0x7fdbb41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:08.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.222+0000 7fdbb8826700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdbb409e410 0x7fdbb4133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:08.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.222+0000 7fdbb8826700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbb406b460 con 0x7fdbb409d6f0 2023-10-03T10:18:08.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.222+0000 7fdbb8826700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb409b120 con 0x7fdbb409e410 2023-10-03T10:18:08.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.222+0000 7fdbb8826700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb409a7e0 con 0x7fdbb409e000 2023-10-03T10:18:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.222+0000 7fdbb1d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdbb409e000 0x7fdbb41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.222+0000 7fdbb2d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdbb409e410 0x7fdbb4133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.222+0000 7fdbb1d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdbb409e000 0x7fdbb41339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47814/0 (socket says 172.21.15.3:47814) 2023-10-03T10:18:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.222+0000 7fdbb2d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdbb409e410 0x7fdbb4133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42010/0 (socket says 172.21.15.3:42010) 2023-10-03T10:18:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.222+0000 7fdbb1d9b700 1 -- 172.21.15.3:0/1240936587 learned_addr learned my addr 172.21.15.3:0/1240936587 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.223+0000 7fdbb2d9d700 1 -- 172.21.15.3:0/1240936587 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdbb409e000 msgr2=0x7fdbb41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.223+0000 7fdbb2d9d700 1 --2- 172.21.15.3:0/1240936587 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdbb409e000 0x7fdbb41339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.223+0000 7fdbb2d9d700 1 -- 172.21.15.3:0/1240936587 >> v2:172.21.15.3:3300/0 conn(0x7fdbb409d6f0 msgr2=0x7fdbb409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.223+0000 7fdbb2d9d700 1 --2- 172.21.15.3:0/1240936587 >> v2:172.21.15.3:3300/0 conn(0x7fdbb409d6f0 0x7fdbb409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.223+0000 7fdbb2d9d700 1 -- 172.21.15.3:0/1240936587 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb409a500 con 0x7fdbb409e410 2023-10-03T10:18:08.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.223+0000 7fdbb1d9b700 1 --2- 172.21.15.3:0/1240936587 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdbb409e000 0x7fdbb41339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:08.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.223+0000 7fdbb2d9d700 1 --2- 172.21.15.3:0/1240936587 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdbb409e410 0x7fdbb4133f30 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009dc0 tx=0x7fdbac00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77d70c98af9db266 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:08.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.223+0000 7fdbb159a700 1 -- 172.21.15.3:0/1240936587 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac00e070 con 0x7fdbb409e410 2023-10-03T10:18:08.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.223+0000 7fdbb159a700 1 -- 172.21.15.3:0/1240936587 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbac00bae0 con 0x7fdbb409e410 2023-10-03T10:18:08.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.223+0000 7fdbb159a700 1 -- 172.21.15.3:0/1240936587 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac00be90 con 0x7fdbb409e410 2023-10-03T10:18:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.224+0000 7fdbb8826700 1 -- 172.21.15.3:0/1240936587 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdbb409e410 msgr2=0x7fdbb4133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.224+0000 7fdbb8826700 1 --2- 172.21.15.3:0/1240936587 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdbb409e410 0x7fdbb4133f30 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009dc0 tx=0x7fdbac00b330 comp rx=0 tx=0).stop 2023-10-03T10:18:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.224+0000 7fdbb8826700 1 -- 172.21.15.3:0/1240936587 shutdown_connections 2023-10-03T10:18:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.224+0000 7fdbb8826700 1 --2- 172.21.15.3:0/1240936587 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdbb409e410 0x7fdbb4133f30 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.224+0000 7fdbb8826700 1 --2- 172.21.15.3:0/1240936587 >> v2:172.21.15.3:3300/0 conn(0x7fdbb409d6f0 0x7fdbb409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.224+0000 7fdbb8826700 1 --2- 172.21.15.3:0/1240936587 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdbb409e000 0x7fdbb41339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.224+0000 7fdbb8826700 1 -- 172.21.15.3:0/1240936587 >> 172.21.15.3:0/1240936587 conn(0x7fdbb4066f50 msgr2=0x7fdbb409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.225+0000 7fdbb8826700 1 -- 172.21.15.3:0/1240936587 shutdown_connections 2023-10-03T10:18:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.225+0000 7fdbb8826700 1 -- 172.21.15.3:0/1240936587 wait complete. 2023-10-03T10:18:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.226+0000 7fdbb8826700 1 Processor -- start 2023-10-03T10:18:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.226+0000 7fdbb8826700 1 -- start start 2023-10-03T10:18:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.227+0000 7fdbb8826700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb409d6f0 0x7fdbb41a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.227+0000 7fdbb8826700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdbb409e000 0x7fdbb41a6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.227+0000 7fdbb8826700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdbb409e410 0x7fdbb41a6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.227+0000 7fdbb8826700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb41a9af0 con 0x7fdbb409d6f0 2023-10-03T10:18:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.227+0000 7fdbb8826700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb41a8200 con 0x7fdbb409e000 2023-10-03T10:18:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.227+0000 7fdbb8826700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb41a9280 con 0x7fdbb409e410 2023-10-03T10:18:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.227+0000 7fdbb259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb409d6f0 0x7fdbb41a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:08.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.227+0000 7fdbb259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb409d6f0 0x7fdbb41a5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46774/0 (socket says 172.21.15.3:46774) 2023-10-03T10:18:08.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.227+0000 7fdbb259c700 1 -- 172.21.15.3:0/188305670 learned_addr learned my addr 172.21.15.3:0/188305670 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:08.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.227+0000 7fdbb1d9b700 1 --2- 172.21.15.3:0/188305670 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdbb409e000 0x7fdbb41a6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:08.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.228+0000 7fdbb2d9d700 1 --2- 172.21.15.3:0/188305670 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdbb409e410 0x7fdbb41a6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:08.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.228+0000 7fdbb259c700 1 -- 172.21.15.3:0/188305670 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdbb409e410 msgr2=0x7fdbb41a6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:08.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.228+0000 7fdbb259c700 1 --2- 172.21.15.3:0/188305670 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdbb409e410 0x7fdbb41a6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.228+0000 7fdbb259c700 1 -- 172.21.15.3:0/188305670 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdbb409e000 msgr2=0x7fdbb41a6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:08.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.228+0000 7fdbb259c700 1 --2- 172.21.15.3:0/188305670 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdbb409e000 0x7fdbb41a6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.228+0000 7fdbb259c700 1 -- 172.21.15.3:0/188305670 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba8009550 con 0x7fdbb409d6f0 2023-10-03T10:18:08.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.228+0000 7fdbb259c700 1 --2- 172.21.15.3:0/188305670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb409d6f0 0x7fdbb41a5fd0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fdba400b530 tx=0x7fdba400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:08.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.228+0000 7fdba2ffd700 1 -- 172.21.15.3:0/188305670 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba4011830 con 0x7fdbb409d6f0 2023-10-03T10:18:08.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.228+0000 7fdba2ffd700 1 -- 172.21.15.3:0/188305670 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba40119b0 con 0x7fdbb409d6f0 2023-10-03T10:18:08.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.228+0000 7fdbb8826700 1 -- 172.21.15.3:0/188305670 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbac009a70 con 0x7fdbb409d6f0 2023-10-03T10:18:08.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.228+0000 7fdba2ffd700 1 -- 172.21.15.3:0/188305670 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba4011d60 con 0x7fdbb409d6f0 2023-10-03T10:18:08.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.228+0000 7fdbb8826700 1 -- 172.21.15.3:0/188305670 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb409c760 con 0x7fdbb409d6f0 2023-10-03T10:18:08.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.230+0000 7fdbb8826700 1 -- 172.21.15.3:0/188305670 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb41a9670 con 0x7fdbb409d6f0 2023-10-03T10:18:08.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.230+0000 7fdba2ffd700 1 -- 172.21.15.3:0/188305670 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdba401a400 con 0x7fdbb409d6f0 2023-10-03T10:18:08.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.232+0000 7fdba2ffd700 1 -- 172.21.15.3:0/188305670 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fdba400f6e0 con 0x7fdbb409d6f0 2023-10-03T10:18:08.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.240+0000 7fdba2ffd700 1 -- 172.21.15.3:0/188305670 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdbb41a9670 con 0x7fdbb409d6f0 2023-10-03T10:18:08.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.279+0000 7f8ae09e1700 1 -- 172.21.15.3:0/2195572618 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8adc074ea0 con 0x7f8adc09e000 2023-10-03T10:18:08.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.279+0000 7f8acaffd700 1 -- 172.21.15.3:0/2195572618 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8ad0011b40 con 0x7f8adc09e000 2023-10-03T10:18:08.280 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:08.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.283+0000 7f8ae09e1700 1 -- 172.21.15.3:0/2195572618 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8adc09e000 msgr2=0x7f8adc1335e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:08.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.283+0000 7f8ae09e1700 1 --2- 172.21.15.3:0/2195572618 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8adc09e000 0x7f8adc1335e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000e980 tx=0x7f8ad000ec90 comp rx=0 tx=0).stop 2023-10-03T10:18:08.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.283+0000 7f8ae09e1700 1 -- 172.21.15.3:0/2195572618 shutdown_connections 2023-10-03T10:18:08.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.283+0000 7f8ae09e1700 1 --2- 172.21.15.3:0/2195572618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8adc09e410 0x7f8adc133b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.283+0000 7f8ae09e1700 1 --2- 172.21.15.3:0/2195572618 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8adc09d6f0 0x7f8adc1330a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.283+0000 7f8ae09e1700 1 --2- 172.21.15.3:0/2195572618 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8adc09e000 0x7f8adc1335e0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.283+0000 7f8ae09e1700 1 -- 172.21.15.3:0/2195572618 >> 172.21.15.3:0/2195572618 conn(0x7f8adc066f50 msgr2=0x7f8adc06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:08.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.283+0000 7f8ae09e1700 1 -- 172.21.15.3:0/2195572618 shutdown_connections 2023-10-03T10:18:08.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.283+0000 7f8ae09e1700 1 -- 172.21.15.3:0/2195572618 wait complete. 2023-10-03T10:18:08.303 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:08.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.315+0000 7f4d755a6700 1 Processor -- start 2023-10-03T10:18:08.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.315+0000 7f4d755a6700 1 -- start start 2023-10-03T10:18:08.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.316+0000 7f4d755a6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d700a0a30 0x7f4d700a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:08.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.316+0000 7f4d755a6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d7009d630 0x7f4d700a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:08.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.316+0000 7f4d755a6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d700a2a70 con 0x7f4d70133930 2023-10-03T10:18:08.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.316+0000 7f4d755a6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d700a01d0 con 0x7f4d700a0a30 2023-10-03T10:18:08.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.316+0000 7f4d755a6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7009e2b0 con 0x7f4d7009d630 2023-10-03T10:18:08.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.316+0000 7f4d6effd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d700a0a30 0x7f4d700a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:08.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.316+0000 7f4d6effd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d700a0a30 0x7f4d700a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46780/0 (socket says 172.21.15.3:46780) 2023-10-03T10:18:08.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.316+0000 7f4d6effd700 1 -- 172.21.15.3:0/1150865990 learned_addr learned my addr 172.21.15.3:0/1150865990 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:08.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.316+0000 7f4d6f7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4d70133930 0x7f4d70133d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50596/0 (socket says 172.21.15.3:50596) 2023-10-03T10:18:08.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.316+0000 7f4d6effd700 1 -- 172.21.15.3:0/1150865990 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d7009d630 msgr2=0x7f4d700a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:08.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.316+0000 7f4d6effd700 1 --2- 172.21.15.3:0/1150865990 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d7009d630 0x7f4d700a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.316+0000 7f4d6effd700 1 -- 172.21.15.3:0/1150865990 >> v1:172.21.15.3:6789/0 conn(0x7f4d70133930 legacy=0x7f4d70133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:08.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.316+0000 7f4d6dffb700 1 -- 172.21.15.3:0/1150865990 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2276416808 0 0) 0x7f4d700a2a70 con 0x7f4d70133930 2023-10-03T10:18:08.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.316+0000 7f4d6effd700 1 -- 172.21.15.3:0/1150865990 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7009dc50 con 0x7f4d700a0a30 2023-10-03T10:18:08.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.316+0000 7f4d6effd700 1 --2- 172.21.15.3:0/1150865990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d700a0a30 0x7f4d700a0e00 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f4d600098f0 tx=0x7f4d60009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84347ccb930a887a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:08.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.317+0000 7f4d6dffb700 1 -- 172.21.15.3:0/1150865990 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6000b8a0 con 0x7f4d700a0a30 2023-10-03T10:18:08.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.317+0000 7f4d6dffb700 1 -- 172.21.15.3:0/1150865990 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6000ba00 con 0x7f4d700a0a30 2023-10-03T10:18:08.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.317+0000 7f4d6dffb700 1 -- 172.21.15.3:0/1150865990 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6000bd90 con 0x7f4d700a0a30 2023-10-03T10:18:08.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.317+0000 7f4d755a6700 1 -- 172.21.15.3:0/1150865990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d700a0a30 msgr2=0x7f4d700a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:08.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.317+0000 7f4d755a6700 1 --2- 172.21.15.3:0/1150865990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d700a0a30 0x7f4d700a0e00 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f4d600098f0 tx=0x7f4d60009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:08.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.317+0000 7f4d755a6700 1 -- 172.21.15.3:0/1150865990 shutdown_connections 2023-10-03T10:18:08.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.317+0000 7f4d755a6700 1 --2- 172.21.15.3:0/1150865990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d700a0a30 0x7f4d700a0e00 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.317+0000 7f4d755a6700 1 --2- 172.21.15.3:0/1150865990 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d7009d630 0x7f4d700a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.317+0000 7f4d755a6700 1 -- 172.21.15.3:0/1150865990 >> 172.21.15.3:0/1150865990 conn(0x7f4d700a3050 msgr2=0x7f4d700a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:08.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.317+0000 7f4d755a6700 1 -- 172.21.15.3:0/1150865990 shutdown_connections 2023-10-03T10:18:08.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.317+0000 7f4d755a6700 1 -- 172.21.15.3:0/1150865990 wait complete. 2023-10-03T10:18:08.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.318+0000 7f4d755a6700 1 Processor -- start 2023-10-03T10:18:08.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.319+0000 7f4d755a6700 1 -- start start 2023-10-03T10:18:08.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.319+0000 7f4d755a6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d7009d630 0x7f4d7011f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:08.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.319+0000 7f4d755a6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d700a0a30 0x7f4d7011f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:08.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.319+0000 7f4d755a6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d70133930 0x7f4d701c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:08.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.319+0000 7f4d755a6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7011eeb0 con 0x7f4d700a0a30 2023-10-03T10:18:08.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.319+0000 7f4d755a6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d701211f0 con 0x7f4d7009d630 2023-10-03T10:18:08.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.319+0000 7f4d755a6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d70122140 con 0x7f4d70133930 2023-10-03T10:18:08.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.319+0000 7f4d6e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d700a0a30 0x7f4d7011f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:08.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.319+0000 7f4d6e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d700a0a30 0x7f4d7011f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46790/0 (socket says 172.21.15.3:46790) 2023-10-03T10:18:08.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.319+0000 7f4d6e7fc700 1 -- 172.21.15.3:0/3188030346 learned_addr learned my addr 172.21.15.3:0/3188030346 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:08.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.319+0000 7f4d6e7fc700 1 -- 172.21.15.3:0/3188030346 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d70133930 msgr2=0x7f4d701c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:08.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.319+0000 7f4d6e7fc700 1 --2- 172.21.15.3:0/3188030346 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d70133930 0x7f4d701c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.320+0000 7f4d6e7fc700 1 -- 172.21.15.3:0/3188030346 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d7009d630 msgr2=0x7f4d7011f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:08.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.320+0000 7f4d6e7fc700 1 --2- 172.21.15.3:0/3188030346 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d7009d630 0x7f4d7011f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.320+0000 7f4d6e7fc700 1 -- 172.21.15.3:0/3188030346 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d60009550 con 0x7f4d700a0a30 2023-10-03T10:18:08.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.320+0000 7f4d6e7fc700 1 --2- 172.21.15.3:0/3188030346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d700a0a30 0x7f4d7011f820 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f4d6400b530 tx=0x7f4d6400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:08.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.320+0000 7f4d577fe700 1 -- 172.21.15.3:0/3188030346 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d64011830 con 0x7f4d700a0a30 2023-10-03T10:18:08.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.320+0000 7f4d577fe700 1 -- 172.21.15.3:0/3188030346 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d640119b0 con 0x7f4d700a0a30 2023-10-03T10:18:08.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.320+0000 7f4d755a6700 1 -- 172.21.15.3:0/3188030346 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d70120620 con 0x7f4d700a0a30 2023-10-03T10:18:08.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.320+0000 7f4d577fe700 1 -- 172.21.15.3:0/3188030346 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d64011d60 con 0x7f4d700a0a30 2023-10-03T10:18:08.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.320+0000 7f4d755a6700 1 -- 172.21.15.3:0/3188030346 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d70122920 con 0x7f4d700a0a30 2023-10-03T10:18:08.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.321+0000 7f4d577fe700 1 -- 172.21.15.3:0/3188030346 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4d6401a400 con 0x7f4d700a0a30 2023-10-03T10:18:08.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.321+0000 7f4d755a6700 1 -- 172.21.15.3:0/3188030346 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d700747f0 con 0x7f4d700a0a30 2023-10-03T10:18:08.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.321+0000 7f4d577fe700 1 -- 172.21.15.3:0/3188030346 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4d6403f6e0 con 0x7f4d700a0a30 2023-10-03T10:18:08.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.331+0000 7f4d577fe700 1 -- 172.21.15.3:0/3188030346 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4d6401ae60 con 0x7f4d700a0a30 2023-10-03T10:18:08.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.394+0000 7fbe48768700 1 -- 172.21.15.3:0/3384928746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbe4006c360 con 0x7fbe4006a770 2023-10-03T10:18:08.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.395+0000 7fbe36ffd700 1 -- 172.21.15.3:0/3384928746 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbe3c015070 con 0x7fbe4006a770 2023-10-03T10:18:08.395 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:08.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.397+0000 7fbe48768700 1 -- 172.21.15.3:0/3384928746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe4006a770 msgr2=0x7fbe4011c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:08.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.397+0000 7fbe48768700 1 --2- 172.21.15.3:0/3384928746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe4006a770 0x7fbe4011c4e0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c016000 tx=0x7fbe3c004750 comp rx=0 tx=0).stop 2023-10-03T10:18:08.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.397+0000 7fbe48768700 1 -- 172.21.15.3:0/3384928746 shutdown_connections 2023-10-03T10:18:08.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.397+0000 7fbe48768700 1 --2- 172.21.15.3:0/3384928746 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbe40131b40 0x7fbe4011cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.397+0000 7fbe48768700 1 --2- 172.21.15.3:0/3384928746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe4006a770 0x7fbe4011c4e0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.397+0000 7fbe48768700 1 --2- 172.21.15.3:0/3384928746 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbe4006b080 0x7fbe4011ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.397+0000 7fbe48768700 1 -- 172.21.15.3:0/3384928746 >> 172.21.15.3:0/3384928746 conn(0x7fbe40066f50 msgr2=0x7fbe400a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:08.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.397+0000 7fbe48768700 1 -- 172.21.15.3:0/3384928746 shutdown_connections 2023-10-03T10:18:08.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.397+0000 7fbe48768700 1 -- 172.21.15.3:0/3384928746 wait complete. 2023-10-03T10:18:08.411 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:08.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.512+0000 7fdbb8826700 1 -- 172.21.15.3:0/188305670 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fdbb41a9670 con 0x7fdbb409d6f0 2023-10-03T10:18:08.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.512+0000 7fdba2ffd700 1 -- 172.21.15.3:0/188305670 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdbb41a9670 con 0x7fdbb409d6f0 2023-10-03T10:18:08.513 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:08.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.515+0000 7fdbb8826700 1 -- 172.21.15.3:0/188305670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb409d6f0 msgr2=0x7fdbb41a5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:08.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.515+0000 7fdbb8826700 1 --2- 172.21.15.3:0/188305670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb409d6f0 0x7fdbb41a5fd0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fdba400b530 tx=0x7fdba400b8f0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.515+0000 7fdbb8826700 1 -- 172.21.15.3:0/188305670 shutdown_connections 2023-10-03T10:18:08.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.515+0000 7fdbb8826700 1 --2- 172.21.15.3:0/188305670 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdbb409e410 0x7fdbb41a6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.515+0000 7fdbb8826700 1 --2- 172.21.15.3:0/188305670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb409d6f0 0x7fdbb41a5fd0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.515+0000 7fdbb8826700 1 --2- 172.21.15.3:0/188305670 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdbb409e000 0x7fdbb41a6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.515+0000 7fdbb8826700 1 -- 172.21.15.3:0/188305670 >> 172.21.15.3:0/188305670 conn(0x7fdbb4066f50 msgr2=0x7fdbb41bac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:08.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.515+0000 7fdbb8826700 1 -- 172.21.15.3:0/188305670 shutdown_connections 2023-10-03T10:18:08.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.515+0000 7fdbb8826700 1 -- 172.21.15.3:0/188305670 wait complete. 2023-10-03T10:18:08.529 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:08.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.575+0000 7f4d755a6700 1 -- 172.21.15.3:0/3188030346 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4d70072980 con 0x7f4d700a0a30 2023-10-03T10:18:08.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.576+0000 7f4d577fe700 1 -- 172.21.15.3:0/3188030346 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4d64015020 con 0x7f4d700a0a30 2023-10-03T10:18:08.576 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:08.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.578+0000 7f4d755a6700 1 -- 172.21.15.3:0/3188030346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d700a0a30 msgr2=0x7f4d7011f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:08.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.578+0000 7f4d755a6700 1 --2- 172.21.15.3:0/3188030346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d700a0a30 0x7f4d7011f820 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f4d6400b530 tx=0x7f4d6400b8f0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.578+0000 7f4d755a6700 1 -- 172.21.15.3:0/3188030346 shutdown_connections 2023-10-03T10:18:08.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.578+0000 7f4d755a6700 1 --2- 172.21.15.3:0/3188030346 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d70133930 0x7f4d701c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.578+0000 7f4d755a6700 1 --2- 172.21.15.3:0/3188030346 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d7009d630 0x7f4d7011f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.578+0000 7f4d755a6700 1 --2- 172.21.15.3:0/3188030346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d700a0a30 0x7f4d7011f820 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:08.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.578+0000 7f4d755a6700 1 -- 172.21.15.3:0/3188030346 >> 172.21.15.3:0/3188030346 conn(0x7f4d700a3050 msgr2=0x7f4d701bbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:08.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.578+0000 7f4d755a6700 1 -- 172.21.15.3:0/3188030346 shutdown_connections 2023-10-03T10:18:08.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:08.578+0000 7f4d755a6700 1 -- 172.21.15.3:0/3188030346 wait complete. 2023-10-03T10:18:08.592 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:09.303 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:09.412 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:09.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.455+0000 7f292f1be700 1 Processor -- start 2023-10-03T10:18:09.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.455+0000 7f292f1be700 1 -- start start 2023-10-03T10:18:09.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.455+0000 7f292f1be700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29280a0a30 0x7f29280a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:09.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.455+0000 7f292f1be700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29280a1340 0x7f2928133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:09.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.455+0000 7f292f1be700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2928133fa0 0x7f2928134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:09.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.455+0000 7f292f1be700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f292809edd0 con 0x7f2928133fa0 2023-10-03T10:18:09.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.455+0000 7f292f1be700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f292809e1f0 con 0x7f29280a0a30 2023-10-03T10:18:09.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.455+0000 7f292f1be700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f292809dc50 con 0x7f29280a1340 2023-10-03T10:18:09.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.455+0000 7f292cf5a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29280a0a30 0x7f29280a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:09.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.455+0000 7f292cf5a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29280a0a30 0x7f29280a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42050/0 (socket says 172.21.15.3:42050) 2023-10-03T10:18:09.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.455+0000 7f2927fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29280a1340 0x7f2928133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:09.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.455+0000 7f292cf5a700 1 -- 172.21.15.3:0/1463813179 learned_addr learned my addr 172.21.15.3:0/1463813179 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:09.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.455+0000 7f292d75b700 1 --2- 172.21.15.3:0/1463813179 >> v2:172.21.15.3:3300/0 conn(0x7f2928133fa0 0x7f2928134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:09.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.456+0000 7f292cf5a700 1 -- 172.21.15.3:0/1463813179 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29280a1340 msgr2=0x7f2928133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:09.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.456+0000 7f292cf5a700 1 --2- 172.21.15.3:0/1463813179 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29280a1340 0x7f2928133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.456+0000 7f292cf5a700 1 -- 172.21.15.3:0/1463813179 >> v2:172.21.15.3:3300/0 conn(0x7f2928133fa0 msgr2=0x7f2928134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:09.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.456+0000 7f292cf5a700 1 --2- 172.21.15.3:0/1463813179 >> v2:172.21.15.3:3300/0 conn(0x7f2928133fa0 0x7f2928134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.456+0000 7f292cf5a700 1 -- 172.21.15.3:0/1463813179 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29280a1fe0 con 0x7f29280a0a30 2023-10-03T10:18:09.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.456+0000 7f2927fff700 1 --2- 172.21.15.3:0/1463813179 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29280a1340 0x7f2928133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:09.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.456+0000 7f292cf5a700 1 --2- 172.21.15.3:0/1463813179 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29280a0a30 0x7f29280a0e00 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f29180098f0 tx=0x7f2918009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d3f357e4ca69850 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:09.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.456+0000 7f29277fe700 1 -- 172.21.15.3:0/1463813179 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2918011830 con 0x7f29280a0a30 2023-10-03T10:18:09.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.456+0000 7f29277fe700 1 -- 172.21.15.3:0/1463813179 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2918011990 con 0x7f29280a0a30 2023-10-03T10:18:09.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.456+0000 7f29277fe700 1 -- 172.21.15.3:0/1463813179 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2918011d20 con 0x7f29280a0a30 2023-10-03T10:18:09.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.457+0000 7f292f1be700 1 -- 172.21.15.3:0/1463813179 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29280a0a30 msgr2=0x7f29280a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:09.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.457+0000 7f292f1be700 1 --2- 172.21.15.3:0/1463813179 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29280a0a30 0x7f29280a0e00 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f29180098f0 tx=0x7f2918009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:09.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.457+0000 7f292f1be700 1 -- 172.21.15.3:0/1463813179 shutdown_connections 2023-10-03T10:18:09.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.457+0000 7f292f1be700 1 --2- 172.21.15.3:0/1463813179 >> v2:172.21.15.3:3300/0 conn(0x7f2928133fa0 0x7f2928134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.457+0000 7f292f1be700 1 --2- 172.21.15.3:0/1463813179 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29280a0a30 0x7f29280a0e00 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.457+0000 7f292f1be700 1 --2- 172.21.15.3:0/1463813179 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29280a1340 0x7f2928133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.457+0000 7f292f1be700 1 -- 172.21.15.3:0/1463813179 >> 172.21.15.3:0/1463813179 conn(0x7f29280a6550 msgr2=0x7f29280a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:09.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.457+0000 7f292f1be700 1 -- 172.21.15.3:0/1463813179 shutdown_connections 2023-10-03T10:18:09.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.457+0000 7f292f1be700 1 -- 172.21.15.3:0/1463813179 wait complete. 2023-10-03T10:18:09.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.458+0000 7f292f1be700 1 Processor -- start 2023-10-03T10:18:09.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.458+0000 7f292f1be700 1 -- start start 2023-10-03T10:18:09.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.459+0000 7f292f1be700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29280a0a30 0x7f29281201d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:09.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.459+0000 7f292f1be700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29280a1340 0x7f2928120710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:09.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.459+0000 7f292f1be700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2928133fa0 0x7f2928120c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:09.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.459+0000 7f292f1be700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29281245d0 con 0x7f29280a1340 2023-10-03T10:18:09.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.459+0000 7f292f1be700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2928121da0 con 0x7f29280a0a30 2023-10-03T10:18:09.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.459+0000 7f292f1be700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2928123930 con 0x7f2928133fa0 2023-10-03T10:18:09.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.459+0000 7f2927fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29280a1340 0x7f2928120710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:09.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.459+0000 7f292d75b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2928133fa0 0x7f2928120c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:09.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.459+0000 7f2927fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29280a1340 0x7f2928120710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46802/0 (socket says 172.21.15.3:46802) 2023-10-03T10:18:09.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.459+0000 7f2927fff700 1 -- 172.21.15.3:0/896683986 learned_addr learned my addr 172.21.15.3:0/896683986 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:09.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.459+0000 7f292cf5a700 1 --2- 172.21.15.3:0/896683986 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29280a0a30 0x7f29281201d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:09.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.459+0000 7f292cf5a700 1 -- 172.21.15.3:0/896683986 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2928133fa0 msgr2=0x7f2928120c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:09.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.459+0000 7f292cf5a700 1 --2- 172.21.15.3:0/896683986 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2928133fa0 0x7f2928120c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.459+0000 7f292cf5a700 1 -- 172.21.15.3:0/896683986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29280a1340 msgr2=0x7f2928120710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:09.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.459+0000 7f292cf5a700 1 --2- 172.21.15.3:0/896683986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29280a1340 0x7f2928120710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.459+0000 7f292cf5a700 1 -- 172.21.15.3:0/896683986 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f291c0094a0 con 0x7f29280a0a30 2023-10-03T10:18:09.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.459+0000 7f2927fff700 1 --2- 172.21.15.3:0/896683986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29280a1340 0x7f2928120710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:09.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.459+0000 7f292cf5a700 1 --2- 172.21.15.3:0/896683986 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29280a0a30 0x7f29281201d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f292809d2f0 tx=0x7f2918004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:09.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.460+0000 7f29257fa700 1 -- 172.21.15.3:0/896683986 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2918004240 con 0x7f29280a0a30 2023-10-03T10:18:09.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.460+0000 7f29257fa700 1 -- 172.21.15.3:0/896683986 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291801f070 con 0x7f29280a0a30 2023-10-03T10:18:09.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.460+0000 7f292f1be700 1 -- 172.21.15.3:0/896683986 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2918009550 con 0x7f29280a0a30 2023-10-03T10:18:09.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.460+0000 7f29257fa700 1 -- 172.21.15.3:0/896683986 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29180045e0 con 0x7f29280a0a30 2023-10-03T10:18:09.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.460+0000 7f292f1be700 1 -- 172.21.15.3:0/896683986 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2928122f50 con 0x7f29280a0a30 2023-10-03T10:18:09.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.460+0000 7f29257fa700 1 -- 172.21.15.3:0/896683986 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2918021030 con 0x7f29280a0a30 2023-10-03T10:18:09.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.461+0000 7f29257fa700 1 -- 172.21.15.3:0/896683986 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2918040700 con 0x7f29280a0a30 2023-10-03T10:18:09.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.461+0000 7f292f1be700 1 -- 172.21.15.3:0/896683986 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2928074ea0 con 0x7f29280a0a30 2023-10-03T10:18:09.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.466+0000 7f29257fa700 1 -- 172.21.15.3:0/896683986 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f291801a450 con 0x7f29280a0a30 2023-10-03T10:18:09.529 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:09.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.571+0000 7f26398fa700 1 Processor -- start 2023-10-03T10:18:09.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.571+0000 7f26398fa700 1 -- start start 2023-10-03T10:18:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.572+0000 7f26398fa700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f263409e1d0 0x7f2634135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.572+0000 7f26398fa700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2634135e50 0x7f2634138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.572+0000 7f26398fa700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f263409f130 con 0x7f263409d6f0 2023-10-03T10:18:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.572+0000 7f26398fa700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f263409f520 con 0x7f263409e1d0 2023-10-03T10:18:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.572+0000 7f26398fa700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f263409e930 con 0x7f2634135e50 2023-10-03T10:18:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.572+0000 7f26337fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2634135e50 0x7f2634138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.572+0000 7f262a7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f263409e1d0 0x7f2634135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.572+0000 7f262a7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f263409e1d0 0x7f2634135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42064/0 (socket says 172.21.15.3:42064) 2023-10-03T10:18:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.572+0000 7f26337fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2634135e50 0x7f2634138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47864/0 (socket says 172.21.15.3:47864) 2023-10-03T10:18:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.572+0000 7f262a7fc700 1 -- 172.21.15.3:0/2525369076 learned_addr learned my addr 172.21.15.3:0/2525369076 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.572+0000 7f26337fe700 1 -- 172.21.15.3:0/2525369076 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f263409e1d0 msgr2=0x7f2634135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.573+0000 7f26337fe700 1 --2- 172.21.15.3:0/2525369076 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f263409e1d0 0x7f2634135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.573+0000 7f26337fe700 1 -- 172.21.15.3:0/2525369076 >> v1:172.21.15.3:6789/0 conn(0x7f263409d6f0 legacy=0x7f263409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.573+0000 7f26337fe700 1 -- 172.21.15.3:0/2525369076 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263409b2d0 con 0x7f2634135e50 2023-10-03T10:18:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.573+0000 7f262a7fc700 1 --2- 172.21.15.3:0/2525369076 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f263409e1d0 0x7f2634135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.573+0000 7f26337fe700 1 --2- 172.21.15.3:0/2525369076 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2634135e50 0x7f2634138230 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f262c009dc0 tx=0x7f262c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9c55a2950326a4e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.573+0000 7f26327fc700 1 -- 172.21.15.3:0/2525369076 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c00e070 con 0x7f2634135e50 2023-10-03T10:18:09.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.573+0000 7f26327fc700 1 -- 172.21.15.3:0/2525369076 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262c00bb20 con 0x7f2634135e50 2023-10-03T10:18:09.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.573+0000 7f26327fc700 1 -- 172.21.15.3:0/2525369076 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c012430 con 0x7f2634135e50 2023-10-03T10:18:09.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.574+0000 7f26398fa700 1 -- 172.21.15.3:0/2525369076 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2634135e50 msgr2=0x7f2634138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:09.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.574+0000 7f26398fa700 1 --2- 172.21.15.3:0/2525369076 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2634135e50 0x7f2634138230 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f262c009dc0 tx=0x7f262c00b330 comp rx=0 tx=0).stop 2023-10-03T10:18:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.574+0000 7f26398fa700 1 -- 172.21.15.3:0/2525369076 shutdown_connections 2023-10-03T10:18:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.574+0000 7f26398fa700 1 --2- 172.21.15.3:0/2525369076 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2634135e50 0x7f2634138230 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.574+0000 7f26398fa700 1 --2- 172.21.15.3:0/2525369076 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f263409e1d0 0x7f2634135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.574+0000 7f26398fa700 1 -- 172.21.15.3:0/2525369076 >> 172.21.15.3:0/2525369076 conn(0x7f2634066f50 msgr2=0x7f263409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.574+0000 7f26398fa700 1 -- 172.21.15.3:0/2525369076 shutdown_connections 2023-10-03T10:18:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.574+0000 7f26398fa700 1 -- 172.21.15.3:0/2525369076 wait complete. 2023-10-03T10:18:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.575+0000 7f26398fa700 1 Processor -- start 2023-10-03T10:18:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.575+0000 7f26398fa700 1 -- start start 2023-10-03T10:18:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.576+0000 7f26398fa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f263409d6f0 0x7f2634070b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.576+0000 7f26398fa700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f263409e1d0 0x7f2634071090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.576+0000 7f26398fa700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2634135e50 0x7f26341c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:09.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.576+0000 7f26398fa700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26340715d0 con 0x7f263409d6f0 2023-10-03T10:18:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.576+0000 7f26398fa700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2634132440 con 0x7f2634135e50 2023-10-03T10:18:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.576+0000 7f26398fa700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2634122e20 con 0x7f263409e1d0 2023-10-03T10:18:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.576+0000 7f26337fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2634135e50 0x7f26341c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.576+0000 7f262a7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f263409e1d0 0x7f2634071090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.576+0000 7f2632ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f263409d6f0 0x7f2634070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.576+0000 7f262a7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f263409e1d0 0x7f2634071090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47874/0 (socket says 172.21.15.3:47874) 2023-10-03T10:18:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.576+0000 7f26337fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2634135e50 0x7f26341c8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42070/0 (socket says 172.21.15.3:42070) 2023-10-03T10:18:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.576+0000 7f262a7fc700 1 -- 172.21.15.3:0/2240074772 learned_addr learned my addr 172.21.15.3:0/2240074772 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.576+0000 7f262a7fc700 1 -- 172.21.15.3:0/2240074772 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2634135e50 msgr2=0x7f26341c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.576+0000 7f262a7fc700 1 --2- 172.21.15.3:0/2240074772 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2634135e50 0x7f26341c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.576+0000 7f262a7fc700 1 -- 172.21.15.3:0/2240074772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f263409d6f0 msgr2=0x7f2634070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.576+0000 7f262a7fc700 1 --2- 172.21.15.3:0/2240074772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f263409d6f0 0x7f2634070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.576+0000 7f262a7fc700 1 -- 172.21.15.3:0/2240074772 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261c009550 con 0x7f263409e1d0 2023-10-03T10:18:09.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.576+0000 7f2632ffd700 1 --2- 172.21.15.3:0/2240074772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f263409d6f0 0x7f2634070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:09.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.576+0000 7f262a7fc700 1 --2- 172.21.15.3:0/2240074772 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f263409e1d0 0x7f2634071090 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f261c00ebe0 tx=0x7f261c00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:09.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.577+0000 7f2630ff9700 1 -- 172.21.15.3:0/2240074772 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c00c850 con 0x7f263409e1d0 2023-10-03T10:18:09.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.577+0000 7f2630ff9700 1 -- 172.21.15.3:0/2240074772 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261c00c9b0 con 0x7f263409e1d0 2023-10-03T10:18:09.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.577+0000 7f26398fa700 1 -- 172.21.15.3:0/2240074772 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f262c009a70 con 0x7f263409e1d0 2023-10-03T10:18:09.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.577+0000 7f2630ff9700 1 -- 172.21.15.3:0/2240074772 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c00cd40 con 0x7f263409e1d0 2023-10-03T10:18:09.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.577+0000 7f26398fa700 1 -- 172.21.15.3:0/2240074772 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2634070700 con 0x7f263409e1d0 2023-10-03T10:18:09.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.578+0000 7f2630ff9700 1 -- 172.21.15.3:0/2240074772 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f261c013470 con 0x7f263409e1d0 2023-10-03T10:18:09.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.578+0000 7f2630ff9700 1 -- 172.21.15.3:0/2240074772 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f261c03fc40 con 0x7f263409e1d0 2023-10-03T10:18:09.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.578+0000 7f26398fa700 1 -- 172.21.15.3:0/2240074772 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2634074ea0 con 0x7f263409e1d0 2023-10-03T10:18:09.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.587+0000 7f2630ff9700 1 -- 172.21.15.3:0/2240074772 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f261c01b3b0 con 0x7f263409e1d0 2023-10-03T10:18:09.593 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:09.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.684+0000 7efdf1d0b700 1 Processor -- start 2023-10-03T10:18:09.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.684+0000 7efdf1d0b700 1 -- start start 2023-10-03T10:18:09.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.685+0000 7efdf1d0b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efdec0a0a30 0x7efdec0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:09.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.685+0000 7efdf1d0b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efdec13d820 0x7efdec13fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:09.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.685+0000 7efdf1d0b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdec09d920 con 0x7efdec0a1340 2023-10-03T10:18:09.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.685+0000 7efdf1d0b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdec09d020 con 0x7efdec13d820 2023-10-03T10:18:09.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.685+0000 7efdf1d0b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdec0a2010 con 0x7efdec0a0a30 2023-10-03T10:18:09.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.685+0000 7efdebfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efdec13d820 0x7efdec13fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:09.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.685+0000 7efdeb7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efdec0a0a30 0x7efdec0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:09.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.685+0000 7efdebfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efdec13d820 0x7efdec13fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46830/0 (socket says 172.21.15.3:46830) 2023-10-03T10:18:09.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.685+0000 7efdeb7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efdec0a0a30 0x7efdec0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42086/0 (socket says 172.21.15.3:42086) 2023-10-03T10:18:09.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.685+0000 7efdebfff700 1 -- 172.21.15.3:0/3546586601 learned_addr learned my addr 172.21.15.3:0/3546586601 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:09.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.685+0000 7efdeaffd700 1 --1- 172.21.15.3:0/3546586601 >> v1:172.21.15.3:6789/0 conn(0x7efdec0a1340 0x7efdec13c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50606/0 (socket says 172.21.15.3:50606) 2023-10-03T10:18:09.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.686+0000 7efdeb7fe700 1 -- 172.21.15.3:0/3546586601 >> v2:172.21.15.3:3300/0 conn(0x7efdec13d820 msgr2=0x7efdec13fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:09.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.686+0000 7efdeb7fe700 1 --2- 172.21.15.3:0/3546586601 >> v2:172.21.15.3:3300/0 conn(0x7efdec13d820 0x7efdec13fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.686+0000 7efdeb7fe700 1 -- 172.21.15.3:0/3546586601 >> v1:172.21.15.3:6789/0 conn(0x7efdec0a1340 legacy=0x7efdec13c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:09.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.686+0000 7efdeb7fe700 1 -- 172.21.15.3:0/3546586601 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec09dbe0 con 0x7efdec0a0a30 2023-10-03T10:18:09.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.686+0000 7efdebfff700 1 --2- 172.21.15.3:0/3546586601 >> v2:172.21.15.3:3300/0 conn(0x7efdec13d820 0x7efdec13fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:09.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.686+0000 7efdeb7fe700 1 --2- 172.21.15.3:0/3546586601 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efdec0a0a30 0x7efdec0a0e00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7efddc009840 tx=0x7efddc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34780c0898dfe29 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:09.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.686+0000 7efdea7fc700 1 -- 172.21.15.3:0/3546586601 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc011830 con 0x7efdec0a0a30 2023-10-03T10:18:09.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.686+0000 7efdea7fc700 1 -- 172.21.15.3:0/3546586601 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efddc011990 con 0x7efdec0a0a30 2023-10-03T10:18:09.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.686+0000 7efdea7fc700 1 -- 172.21.15.3:0/3546586601 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc011d20 con 0x7efdec0a0a30 2023-10-03T10:18:09.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.687+0000 7efdf1d0b700 1 -- 172.21.15.3:0/3546586601 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efdec0a0a30 msgr2=0x7efdec0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:09.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.687+0000 7efdf1d0b700 1 --2- 172.21.15.3:0/3546586601 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efdec0a0a30 0x7efdec0a0e00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7efddc009840 tx=0x7efddc009b50 comp rx=0 tx=0).stop 2023-10-03T10:18:09.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.687+0000 7efdf1d0b700 1 -- 172.21.15.3:0/3546586601 shutdown_connections 2023-10-03T10:18:09.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.687+0000 7efdf1d0b700 1 --2- 172.21.15.3:0/3546586601 >> v2:172.21.15.3:3300/0 conn(0x7efdec13d820 0x7efdec13fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.687+0000 7efdf1d0b700 1 --2- 172.21.15.3:0/3546586601 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efdec0a0a30 0x7efdec0a0e00 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.687+0000 7efdf1d0b700 1 -- 172.21.15.3:0/3546586601 >> 172.21.15.3:0/3546586601 conn(0x7efdec0a6550 msgr2=0x7efdec0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:09.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.687+0000 7efdf1d0b700 1 -- 172.21.15.3:0/3546586601 shutdown_connections 2023-10-03T10:18:09.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.688+0000 7efdf1d0b700 1 -- 172.21.15.3:0/3546586601 wait complete. 2023-10-03T10:18:09.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.689+0000 7efdf1d0b700 1 Processor -- start 2023-10-03T10:18:09.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.689+0000 7efdf1d0b700 1 -- start start 2023-10-03T10:18:09.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.690+0000 7efdf1d0b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efdec0a0a30 0x7efdec125dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:09.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.690+0000 7efdf1d0b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efdec0a1340 0x7efdec126310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:09.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.690+0000 7efdf1d0b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdec13d820 0x7efdec126850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:09.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.690+0000 7efdf1d0b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdec0a0020 con 0x7efdec13d820 2023-10-03T10:18:09.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.690+0000 7efdf1d0b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdec127af0 con 0x7efdec0a0a30 2023-10-03T10:18:09.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.690+0000 7efdf1d0b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efdec129610 con 0x7efdec0a1340 2023-10-03T10:18:09.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.690+0000 7efdeb7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efdec0a0a30 0x7efdec125dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:09.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.690+0000 7efdeb7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efdec0a0a30 0x7efdec125dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42092/0 (socket says 172.21.15.3:42092) 2023-10-03T10:18:09.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.690+0000 7efdeb7fe700 1 -- 172.21.15.3:0/3578301994 learned_addr learned my addr 172.21.15.3:0/3578301994 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:09.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.690+0000 7efdebfff700 1 --2- 172.21.15.3:0/3578301994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdec13d820 0x7efdec126850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:09.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.691+0000 7efdeaffd700 1 --2- 172.21.15.3:0/3578301994 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efdec0a1340 0x7efdec126310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:09.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.691+0000 7efdebfff700 1 -- 172.21.15.3:0/3578301994 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efdec0a1340 msgr2=0x7efdec126310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:09.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.691+0000 7efdebfff700 1 --2- 172.21.15.3:0/3578301994 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efdec0a1340 0x7efdec126310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.691+0000 7efdebfff700 1 -- 172.21.15.3:0/3578301994 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efdec0a0a30 msgr2=0x7efdec125dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:09.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.691+0000 7efdebfff700 1 --2- 172.21.15.3:0/3578301994 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efdec0a0a30 0x7efdec125dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.691+0000 7efdebfff700 1 -- 172.21.15.3:0/3578301994 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde4009b30 con 0x7efdec13d820 2023-10-03T10:18:09.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.691+0000 7efdeaffd700 1 --2- 172.21.15.3:0/3578301994 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efdec0a1340 0x7efdec126310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:18:09.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.691+0000 7efdeb7fe700 1 --2- 172.21.15.3:0/3578301994 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efdec0a0a30 0x7efdec125dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:09.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.691+0000 7efdebfff700 1 --2- 172.21.15.3:0/3578301994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdec13d820 0x7efdec126850 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7efde400ef90 tx=0x7efde400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:09.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.691+0000 7efdd3fff700 1 -- 172.21.15.3:0/3578301994 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde400cb30 con 0x7efdec13d820 2023-10-03T10:18:09.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.691+0000 7efdd3fff700 1 -- 172.21.15.3:0/3578301994 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde400f040 con 0x7efdec13d820 2023-10-03T10:18:09.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.691+0000 7efdf1d0b700 1 -- 172.21.15.3:0/3578301994 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efddc0094a0 con 0x7efdec13d820 2023-10-03T10:18:09.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.691+0000 7efdd3fff700 1 -- 172.21.15.3:0/3578301994 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde400ced0 con 0x7efdec13d820 2023-10-03T10:18:09.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.691+0000 7efdf1d0b700 1 -- 172.21.15.3:0/3578301994 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdec1281d0 con 0x7efdec13d820 2023-10-03T10:18:09.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.693+0000 7efdd3fff700 1 -- 172.21.15.3:0/3578301994 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efde4014480 con 0x7efdec13d820 2023-10-03T10:18:09.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.693+0000 7efdf1d0b700 1 -- 172.21.15.3:0/3578301994 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdec071e20 con 0x7efdec13d820 2023-10-03T10:18:09.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.693+0000 7efdd3fff700 1 -- 172.21.15.3:0/3578301994 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7efde403fb90 con 0x7efdec13d820 2023-10-03T10:18:09.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.702+0000 7efdd3fff700 1 -- 172.21.15.3:0/3578301994 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efde4012520 con 0x7efdec13d820 2023-10-03T10:18:09.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.707+0000 7f292f1be700 1 -- 172.21.15.3:0/896683986 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f29280a28e0 con 0x7f29280a0a30 2023-10-03T10:18:09.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.707+0000 7f29257fa700 1 -- 172.21.15.3:0/896683986 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f29180233d0 con 0x7f29280a0a30 2023-10-03T10:18:09.707 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:09.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.710+0000 7f292f1be700 1 -- 172.21.15.3:0/896683986 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29280a0a30 msgr2=0x7f29281201d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:09.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.710+0000 7f292f1be700 1 --2- 172.21.15.3:0/896683986 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29280a0a30 0x7f29281201d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f292809d2f0 tx=0x7f2918004750 comp rx=0 tx=0).stop 2023-10-03T10:18:09.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.711+0000 7f292f1be700 1 -- 172.21.15.3:0/896683986 shutdown_connections 2023-10-03T10:18:09.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.711+0000 7f292f1be700 1 --2- 172.21.15.3:0/896683986 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2928133fa0 0x7f2928120c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.711+0000 7f292f1be700 1 --2- 172.21.15.3:0/896683986 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29280a0a30 0x7f29281201d0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.711+0000 7f292f1be700 1 --2- 172.21.15.3:0/896683986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29280a1340 0x7f2928120710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.711+0000 7f292f1be700 1 -- 172.21.15.3:0/896683986 >> 172.21.15.3:0/896683986 conn(0x7f29280a6550 msgr2=0x7f29281bafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:09.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.711+0000 7f292f1be700 1 -- 172.21.15.3:0/896683986 shutdown_connections 2023-10-03T10:18:09.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.711+0000 7f292f1be700 1 -- 172.21.15.3:0/896683986 wait complete. 2023-10-03T10:18:09.731 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:09.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.746+0000 7fd2b9ca7700 1 Processor -- start 2023-10-03T10:18:09.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.746+0000 7fd2b9ca7700 1 -- start start 2023-10-03T10:18:09.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.747+0000 7fd2b9ca7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd2b409d6f0 0x7fd2b409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:09.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.747+0000 7fd2b9ca7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd2b409e000 0x7fd2b41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:09.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.747+0000 7fd2b9ca7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2b409b120 con 0x7fd2b409e410 2023-10-03T10:18:09.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.747+0000 7fd2b9ca7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b406b460 con 0x7fd2b409d6f0 2023-10-03T10:18:09.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.747+0000 7fd2b9ca7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b409a7e0 con 0x7fd2b409e000 2023-10-03T10:18:09.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.747+0000 7fd2b3fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd2b409e410 0x7fd2b4133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50614/0 (socket says 172.21.15.3:50614) 2023-10-03T10:18:09.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.747+0000 7fd2b2ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd2b409e000 0x7fd2b41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:09.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.747+0000 7fd2b3fff700 1 -- 172.21.15.3:0/3382779590 learned_addr learned my addr 172.21.15.3:0/3382779590 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:09.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.747+0000 7fd2b37fe700 1 --2- 172.21.15.3:0/3382779590 >> v2:172.21.15.3:3300/0 conn(0x7fd2b409d6f0 0x7fd2b409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:09.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.747+0000 7fd2b2ffd700 1 -- 172.21.15.3:0/3382779590 >> v2:172.21.15.3:3300/0 conn(0x7fd2b409d6f0 msgr2=0x7fd2b409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:09.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.747+0000 7fd2b2ffd700 1 --2- 172.21.15.3:0/3382779590 >> v2:172.21.15.3:3300/0 conn(0x7fd2b409d6f0 0x7fd2b409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.747+0000 7fd2b2ffd700 1 -- 172.21.15.3:0/3382779590 >> v1:172.21.15.3:6789/0 conn(0x7fd2b409e410 legacy=0x7fd2b4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:09.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.747+0000 7fd2b27fc700 1 -- 172.21.15.3:0/3382779590 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 842726319 0 0) 0x7fd2b409b120 con 0x7fd2b409e410 2023-10-03T10:18:09.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.747+0000 7fd2b2ffd700 1 -- 172.21.15.3:0/3382779590 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b409a500 con 0x7fd2b409e000 2023-10-03T10:18:09.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.747+0000 7fd2b37fe700 1 --2- 172.21.15.3:0/3382779590 >> v2:172.21.15.3:3300/0 conn(0x7fd2b409d6f0 0x7fd2b409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:09.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.748+0000 7fd2b2ffd700 1 --2- 172.21.15.3:0/3382779590 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd2b409e000 0x7fd2b41339f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009870 tx=0x7fd2a8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c0ebb86263866d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:09.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.748+0000 7fd2b27fc700 1 -- 172.21.15.3:0/3382779590 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8011830 con 0x7fd2b409e000 2023-10-03T10:18:09.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.748+0000 7fd2b27fc700 1 -- 172.21.15.3:0/3382779590 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a8011990 con 0x7fd2b409e000 2023-10-03T10:18:09.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.748+0000 7fd2b27fc700 1 -- 172.21.15.3:0/3382779590 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8011d20 con 0x7fd2b409e000 2023-10-03T10:18:09.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.748+0000 7fd2b9ca7700 1 -- 172.21.15.3:0/3382779590 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd2b409e000 msgr2=0x7fd2b41339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:09.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.748+0000 7fd2b9ca7700 1 --2- 172.21.15.3:0/3382779590 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd2b409e000 0x7fd2b41339f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009870 tx=0x7fd2a8009b80 comp rx=0 tx=0).stop 2023-10-03T10:18:09.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.748+0000 7fd2b9ca7700 1 -- 172.21.15.3:0/3382779590 shutdown_connections 2023-10-03T10:18:09.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.748+0000 7fd2b9ca7700 1 --2- 172.21.15.3:0/3382779590 >> v2:172.21.15.3:3300/0 conn(0x7fd2b409d6f0 0x7fd2b409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.748+0000 7fd2b9ca7700 1 --2- 172.21.15.3:0/3382779590 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd2b409e000 0x7fd2b41339f0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.748+0000 7fd2b9ca7700 1 -- 172.21.15.3:0/3382779590 >> 172.21.15.3:0/3382779590 conn(0x7fd2b4066f50 msgr2=0x7fd2b409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:09.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.748+0000 7fd2b9ca7700 1 -- 172.21.15.3:0/3382779590 shutdown_connections 2023-10-03T10:18:09.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.749+0000 7fd2b9ca7700 1 -- 172.21.15.3:0/3382779590 wait complete. 2023-10-03T10:18:09.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.749+0000 7fd2b9ca7700 1 Processor -- start 2023-10-03T10:18:09.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.750+0000 7fd2b9ca7700 1 -- start start 2023-10-03T10:18:09.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.750+0000 7fd2b9ca7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd2b409d6f0 0x7fd2b41c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:09.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.750+0000 7fd2b9ca7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd2b409e000 0x7fd2b41c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:09.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.750+0000 7fd2b9ca7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2b409e410 0x7fd2b41c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:09.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.750+0000 7fd2b9ca7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b41a65f0 con 0x7fd2b409e410 2023-10-03T10:18:09.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.750+0000 7fd2b9ca7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b41a63c0 con 0x7fd2b409d6f0 2023-10-03T10:18:09.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.750+0000 7fd2b9ca7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b41a7030 con 0x7fd2b409e000 2023-10-03T10:18:09.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.750+0000 7fd2b2ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd2b409e000 0x7fd2b41c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:09.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.750+0000 7fd2b3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2b409e410 0x7fd2b41c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:09.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.750+0000 7fd2b2ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd2b409e000 0x7fd2b41c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47890/0 (socket says 172.21.15.3:47890) 2023-10-03T10:18:09.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.750+0000 7fd2b3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2b409e410 0x7fd2b41c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46842/0 (socket says 172.21.15.3:46842) 2023-10-03T10:18:09.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.750+0000 7fd2b37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd2b409d6f0 0x7fd2b41c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:09.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.750+0000 7fd2b2ffd700 1 -- 172.21.15.3:0/1990818813 learned_addr learned my addr 172.21.15.3:0/1990818813 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:09.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.751+0000 7fd2b2ffd700 1 -- 172.21.15.3:0/1990818813 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd2b409d6f0 msgr2=0x7fd2b41c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:09.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.751+0000 7fd2b2ffd700 1 --2- 172.21.15.3:0/1990818813 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd2b409d6f0 0x7fd2b41c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.751+0000 7fd2b2ffd700 1 -- 172.21.15.3:0/1990818813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2b409e410 msgr2=0x7fd2b41c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:09.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.751+0000 7fd2b2ffd700 1 --2- 172.21.15.3:0/1990818813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2b409e410 0x7fd2b41c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.751+0000 7fd2b2ffd700 1 -- 172.21.15.3:0/1990818813 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a40094a0 con 0x7fd2b409e000 2023-10-03T10:18:09.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.751+0000 7fd2b3fff700 1 --2- 172.21.15.3:0/1990818813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2b409e410 0x7fd2b41c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:09.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.751+0000 7fd2b2ffd700 1 --2- 172.21.15.3:0/1990818813 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd2b409e000 0x7fd2b41c8560 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8000c00 tx=0x7fd2a8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:09.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.751+0000 7fd29bfff700 1 -- 172.21.15.3:0/1990818813 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a801a8b0 con 0x7fd2b409e000 2023-10-03T10:18:09.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.751+0000 7fd29bfff700 1 -- 172.21.15.3:0/1990818813 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a801aa10 con 0x7fd2b409e000 2023-10-03T10:18:09.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.751+0000 7fd2b9ca7700 1 -- 172.21.15.3:0/1990818813 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a80094a0 con 0x7fd2b409e000 2023-10-03T10:18:09.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.751+0000 7fd29bfff700 1 -- 172.21.15.3:0/1990818813 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a801ada0 con 0x7fd2b409e000 2023-10-03T10:18:09.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.751+0000 7fd2b9ca7700 1 -- 172.21.15.3:0/1990818813 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b409c710 con 0x7fd2b409e000 2023-10-03T10:18:09.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.752+0000 7fd29bfff700 1 -- 172.21.15.3:0/1990818813 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd2a80243f0 con 0x7fd2b409e000 2023-10-03T10:18:09.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.752+0000 7fd2b9ca7700 1 -- 172.21.15.3:0/1990818813 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b41a7f50 con 0x7fd2b409e000 2023-10-03T10:18:09.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.753+0000 7fd29bfff700 1 -- 172.21.15.3:0/1990818813 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd2a8040b20 con 0x7fd2b409e000 2023-10-03T10:18:09.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.758+0000 7fd29bfff700 1 -- 172.21.15.3:0/1990818813 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd2a800b360 con 0x7fd2b409e000 2023-10-03T10:18:09.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.822+0000 7f26398fa700 1 -- 172.21.15.3:0/2240074772 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f26341220b0 con 0x7f263409e1d0 2023-10-03T10:18:09.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.822+0000 7f2630ff9700 1 -- 172.21.15.3:0/2240074772 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f261c024350 con 0x7f263409e1d0 2023-10-03T10:18:09.823 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:09.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.825+0000 7f26398fa700 1 -- 172.21.15.3:0/2240074772 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f263409e1d0 msgr2=0x7f2634071090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:09.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.825+0000 7f26398fa700 1 --2- 172.21.15.3:0/2240074772 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f263409e1d0 0x7f2634071090 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f261c00ebe0 tx=0x7f261c00eef0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.825+0000 7f26398fa700 1 -- 172.21.15.3:0/2240074772 shutdown_connections 2023-10-03T10:18:09.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.825+0000 7f26398fa700 1 --2- 172.21.15.3:0/2240074772 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2634135e50 0x7f26341c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.825+0000 7f26398fa700 1 --2- 172.21.15.3:0/2240074772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f263409d6f0 0x7f2634070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.825+0000 7f26398fa700 1 --2- 172.21.15.3:0/2240074772 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f263409e1d0 0x7f2634071090 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.825+0000 7f26398fa700 1 -- 172.21.15.3:0/2240074772 >> 172.21.15.3:0/2240074772 conn(0x7f2634066f50 msgr2=0x7f263406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:09.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.825+0000 7f26398fa700 1 -- 172.21.15.3:0/2240074772 shutdown_connections 2023-10-03T10:18:09.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.825+0000 7f26398fa700 1 -- 172.21.15.3:0/2240074772 wait complete. 2023-10-03T10:18:09.839 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:09.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.920+0000 7efdf1d0b700 1 -- 172.21.15.3:0/3578301994 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7efdec1292b0 con 0x7efdec13d820 2023-10-03T10:18:09.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.920+0000 7efdd3fff700 1 -- 172.21.15.3:0/3578301994 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efdec1292b0 con 0x7efdec13d820 2023-10-03T10:18:09.921 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:09.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.923+0000 7efdf1d0b700 1 -- 172.21.15.3:0/3578301994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdec13d820 msgr2=0x7efdec126850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:09.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.923+0000 7efdf1d0b700 1 --2- 172.21.15.3:0/3578301994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdec13d820 0x7efdec126850 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7efde400ef90 tx=0x7efde400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.923+0000 7efdf1d0b700 1 -- 172.21.15.3:0/3578301994 shutdown_connections 2023-10-03T10:18:09.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.923+0000 7efdf1d0b700 1 --2- 172.21.15.3:0/3578301994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdec13d820 0x7efdec126850 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.923+0000 7efdf1d0b700 1 --2- 172.21.15.3:0/3578301994 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efdec0a0a30 0x7efdec125dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.923+0000 7efdf1d0b700 1 --2- 172.21.15.3:0/3578301994 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7efdec0a1340 0x7efdec126310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.923+0000 7efdf1d0b700 1 -- 172.21.15.3:0/3578301994 >> 172.21.15.3:0/3578301994 conn(0x7efdec0a6550 msgr2=0x7efdec0a2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:09.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.923+0000 7efdf1d0b700 1 -- 172.21.15.3:0/3578301994 shutdown_connections 2023-10-03T10:18:09.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.923+0000 7efdf1d0b700 1 -- 172.21.15.3:0/3578301994 wait complete. 2023-10-03T10:18:09.936 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:09.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.965+0000 7fd2b9ca7700 1 -- 172.21.15.3:0/1990818813 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd2b4074ea0 con 0x7fd2b409e000 2023-10-03T10:18:09.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.965+0000 7fd29bfff700 1 -- 172.21.15.3:0/1990818813 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd2a8015020 con 0x7fd2b409e000 2023-10-03T10:18:09.965 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:09.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.967+0000 7fd2b9ca7700 1 -- 172.21.15.3:0/1990818813 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd2b409e000 msgr2=0x7fd2b41c8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:09.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.967+0000 7fd2b9ca7700 1 --2- 172.21.15.3:0/1990818813 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd2b409e000 0x7fd2b41c8560 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8000c00 tx=0x7fd2a8004750 comp rx=0 tx=0).stop 2023-10-03T10:18:09.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.967+0000 7fd2b9ca7700 1 -- 172.21.15.3:0/1990818813 shutdown_connections 2023-10-03T10:18:09.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.967+0000 7fd2b9ca7700 1 --2- 172.21.15.3:0/1990818813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2b409e410 0x7fd2b41c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.967+0000 7fd2b9ca7700 1 --2- 172.21.15.3:0/1990818813 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd2b409d6f0 0x7fd2b41c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.967+0000 7fd2b9ca7700 1 --2- 172.21.15.3:0/1990818813 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd2b409e000 0x7fd2b41c8560 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:09.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.967+0000 7fd2b9ca7700 1 -- 172.21.15.3:0/1990818813 >> 172.21.15.3:0/1990818813 conn(0x7fd2b4066f50 msgr2=0x7fd2b409b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:09.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.968+0000 7fd2b9ca7700 1 -- 172.21.15.3:0/1990818813 shutdown_connections 2023-10-03T10:18:09.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:09.968+0000 7fd2b9ca7700 1 -- 172.21.15.3:0/1990818813 wait complete. 2023-10-03T10:18:09.981 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:10.732 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:10.840 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:10.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.878+0000 7fd4490ff700 1 Processor -- start 2023-10-03T10:18:10.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.878+0000 7fd4490ff700 1 -- start start 2023-10-03T10:18:10.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.879+0000 7fd4490ff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd44409d6f0 0x7fd44409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:10.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.879+0000 7fd4490ff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd44409e000 0x7fd444130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:10.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.879+0000 7fd4490ff700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd44409f130 con 0x7fd44409e410 2023-10-03T10:18:10.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.879+0000 7fd4490ff700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd44409f520 con 0x7fd44409e000 2023-10-03T10:18:10.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.879+0000 7fd4490ff700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd44409e930 con 0x7fd44409d6f0 2023-10-03T10:18:10.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.879+0000 7fd43a59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd44409e000 0x7fd444130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:10.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.879+0000 7fd442d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd44409d6f0 0x7fd44409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:10.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.879+0000 7fd442d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd44409d6f0 0x7fd44409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42096/0 (socket says 172.21.15.3:42096) 2023-10-03T10:18:10.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.879+0000 7fd43a59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd44409e000 0x7fd444130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46856/0 (socket says 172.21.15.3:46856) 2023-10-03T10:18:10.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.879+0000 7fd442d9d700 1 -- 172.21.15.3:0/104128104 learned_addr learned my addr 172.21.15.3:0/104128104 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:10.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.879+0000 7fd442d9d700 1 -- 172.21.15.3:0/104128104 >> v2:172.21.15.3:3300/0 conn(0x7fd44409e000 msgr2=0x7fd444130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:10.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.879+0000 7fd442d9d700 1 --2- 172.21.15.3:0/104128104 >> v2:172.21.15.3:3300/0 conn(0x7fd44409e000 0x7fd444130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:10.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.879+0000 7fd442d9d700 1 -- 172.21.15.3:0/104128104 >> v1:172.21.15.3:6789/0 conn(0x7fd44409e410 legacy=0x7fd444130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:10.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.879+0000 7fd442d9d700 1 -- 172.21.15.3:0/104128104 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44409b2d0 con 0x7fd44409d6f0 2023-10-03T10:18:10.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.880+0000 7fd43a59c700 1 --2- 172.21.15.3:0/104128104 >> v2:172.21.15.3:3300/0 conn(0x7fd44409e000 0x7fd444130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:10.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.880+0000 7fd442d9d700 1 --2- 172.21.15.3:0/104128104 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd44409d6f0 0x7fd44409dac0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd43400bb90 tx=0x7fd43400bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b9315f65cc33358 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:10.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.880+0000 7fd44259c700 1 -- 172.21.15.3:0/104128104 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43400dd30 con 0x7fd44409d6f0 2023-10-03T10:18:10.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.880+0000 7fd44259c700 1 -- 172.21.15.3:0/104128104 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43400de90 con 0x7fd44409d6f0 2023-10-03T10:18:10.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.880+0000 7fd44259c700 1 -- 172.21.15.3:0/104128104 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd434010690 con 0x7fd44409d6f0 2023-10-03T10:18:10.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.880+0000 7fd4490ff700 1 -- 172.21.15.3:0/104128104 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd44409d6f0 msgr2=0x7fd44409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:10.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.880+0000 7fd4490ff700 1 --2- 172.21.15.3:0/104128104 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd44409d6f0 0x7fd44409dac0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd43400bb90 tx=0x7fd43400bea0 comp rx=0 tx=0).stop 2023-10-03T10:18:10.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.880+0000 7fd4490ff700 1 -- 172.21.15.3:0/104128104 shutdown_connections 2023-10-03T10:18:10.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.880+0000 7fd4490ff700 1 --2- 172.21.15.3:0/104128104 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd44409d6f0 0x7fd44409dac0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:10.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.880+0000 7fd4490ff700 1 --2- 172.21.15.3:0/104128104 >> v2:172.21.15.3:3300/0 conn(0x7fd44409e000 0x7fd444130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:10.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.880+0000 7fd4490ff700 1 -- 172.21.15.3:0/104128104 >> 172.21.15.3:0/104128104 conn(0x7fd444066f50 msgr2=0x7fd44409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:10.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.881+0000 7fd4490ff700 1 -- 172.21.15.3:0/104128104 shutdown_connections 2023-10-03T10:18:10.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.881+0000 7fd4490ff700 1 -- 172.21.15.3:0/104128104 wait complete. 2023-10-03T10:18:10.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.882+0000 7fd4490ff700 1 Processor -- start 2023-10-03T10:18:10.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.882+0000 7fd4490ff700 1 -- start start 2023-10-03T10:18:10.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.883+0000 7fd4490ff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd44409d6f0 0x7fd44411e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:10.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.883+0000 7fd4490ff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd44409e000 0x7fd44411e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:10.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.883+0000 7fd4490ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd44409e410 0x7fd44411eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:10.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.883+0000 7fd4490ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44411f4d0 con 0x7fd44409e410 2023-10-03T10:18:10.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.883+0000 7fd4490ff700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd444121af0 con 0x7fd44409e000 2023-10-03T10:18:10.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.883+0000 7fd4490ff700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd444120200 con 0x7fd44409d6f0 2023-10-03T10:18:10.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.883+0000 7fd43a59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd44409e000 0x7fd44411e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:10.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.883+0000 7fd44359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd44409e410 0x7fd44411eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:10.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.883+0000 7fd43a59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd44409e000 0x7fd44411e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42108/0 (socket says 172.21.15.3:42108) 2023-10-03T10:18:10.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.883+0000 7fd44359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd44409e410 0x7fd44411eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46864/0 (socket says 172.21.15.3:46864) 2023-10-03T10:18:10.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.883+0000 7fd43a59c700 1 -- 172.21.15.3:0/1754220570 learned_addr learned my addr 172.21.15.3:0/1754220570 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:10.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.883+0000 7fd442d9d700 1 --2- 172.21.15.3:0/1754220570 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd44409d6f0 0x7fd44411e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:10.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.884+0000 7fd44359e700 1 -- 172.21.15.3:0/1754220570 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd44409d6f0 msgr2=0x7fd44411e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:10.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.884+0000 7fd44359e700 1 --2- 172.21.15.3:0/1754220570 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd44409d6f0 0x7fd44411e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:10.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.884+0000 7fd44359e700 1 -- 172.21.15.3:0/1754220570 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd44409e000 msgr2=0x7fd44411e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:10.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.884+0000 7fd44359e700 1 --2- 172.21.15.3:0/1754220570 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd44409e000 0x7fd44411e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:10.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.884+0000 7fd44359e700 1 -- 172.21.15.3:0/1754220570 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd42c0094a0 con 0x7fd44409e410 2023-10-03T10:18:10.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.884+0000 7fd43a59c700 1 --2- 172.21.15.3:0/1754220570 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd44409e000 0x7fd44411e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:10.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.884+0000 7fd44359e700 1 --2- 172.21.15.3:0/1754220570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd44409e410 0x7fd44411eaa0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fd43c00cb90 tx=0x7fd43c00cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:10.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.884+0000 7fd43bfff700 1 -- 172.21.15.3:0/1754220570 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c003c00 con 0x7fd44409e410 2023-10-03T10:18:10.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.884+0000 7fd43bfff700 1 -- 172.21.15.3:0/1754220570 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43c003d60 con 0x7fd44409e410 2023-10-03T10:18:10.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.884+0000 7fd4490ff700 1 -- 172.21.15.3:0/1754220570 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd43400b7f0 con 0x7fd44409e410 2023-10-03T10:18:10.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.884+0000 7fd43bfff700 1 -- 172.21.15.3:0/1754220570 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c00aa40 con 0x7fd44409e410 2023-10-03T10:18:10.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.884+0000 7fd4490ff700 1 -- 172.21.15.3:0/1754220570 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4440a1d90 con 0x7fd44409e410 2023-10-03T10:18:10.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.885+0000 7fd43bfff700 1 -- 172.21.15.3:0/1754220570 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd43c00aba0 con 0x7fd44409e410 2023-10-03T10:18:10.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.885+0000 7fd43bfff700 1 -- 172.21.15.3:0/1754220570 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd43c03f5b0 con 0x7fd44409e410 2023-10-03T10:18:10.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.885+0000 7fd4490ff700 1 -- 172.21.15.3:0/1754220570 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd44409c700 con 0x7fd44409e410 2023-10-03T10:18:10.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.890+0000 7fd43bfff700 1 -- 172.21.15.3:0/1754220570 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd43c012420 con 0x7fd44409e410 2023-10-03T10:18:10.937 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:10.982 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:10.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.989+0000 7fa486c43700 1 Processor -- start 2023-10-03T10:18:10.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.989+0000 7fa486c43700 1 -- start start 2023-10-03T10:18:10.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.989+0000 7fa486c43700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa48009e1d0 0x7fa480135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:10.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.989+0000 7fa486c43700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa480135e50 0x7fa480138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:10.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.989+0000 7fa486c43700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa48009f130 con 0x7fa48009d6f0 2023-10-03T10:18:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.989+0000 7fa486c43700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa48009f520 con 0x7fa48009e1d0 2023-10-03T10:18:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.989+0000 7fa486c43700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa48009e930 con 0x7fa480135e50 2023-10-03T10:18:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.989+0000 7fa4849df700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa48009d6f0 0x7fa48009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50624/0 (socket says 172.21.15.3:50624) 2023-10-03T10:18:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.989+0000 7fa4849df700 1 -- 172.21.15.3:0/3880213701 learned_addr learned my addr 172.21.15.3:0/3880213701 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.989+0000 7fa4851e0700 1 --2- 172.21.15.3:0/3880213701 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa480135e50 0x7fa480138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.990+0000 7fa47ffff700 1 --2- 172.21.15.3:0/3880213701 >> v2:172.21.15.3:3300/0 conn(0x7fa48009e1d0 0x7fa480135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.990+0000 7fa4851e0700 1 -- 172.21.15.3:0/3880213701 >> v2:172.21.15.3:3300/0 conn(0x7fa48009e1d0 msgr2=0x7fa480135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.990+0000 7fa4851e0700 1 --2- 172.21.15.3:0/3880213701 >> v2:172.21.15.3:3300/0 conn(0x7fa48009e1d0 0x7fa480135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.990+0000 7fa4851e0700 1 -- 172.21.15.3:0/3880213701 >> v1:172.21.15.3:6789/0 conn(0x7fa48009d6f0 legacy=0x7fa48009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.990+0000 7fa4851e0700 1 -- 172.21.15.3:0/3880213701 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48009b2d0 con 0x7fa480135e50 2023-10-03T10:18:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.990+0000 7fa47f7fe700 1 -- 172.21.15.3:0/3880213701 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2546538663 0 0) 0x7fa48009f130 con 0x7fa48009d6f0 2023-10-03T10:18:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.990+0000 7fa4851e0700 1 --2- 172.21.15.3:0/3880213701 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa480135e50 0x7fa480138230 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa47800be30 tx=0x7fa478010730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c6589ebeaef6a10 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.990+0000 7fa47f7fe700 1 -- 172.21.15.3:0/3880213701 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47800b2d0 con 0x7fa480135e50 2023-10-03T10:18:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.990+0000 7fa47f7fe700 1 -- 172.21.15.3:0/3880213701 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa478009540 con 0x7fa480135e50 2023-10-03T10:18:10.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.990+0000 7fa47f7fe700 1 -- 172.21.15.3:0/3880213701 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4780098f0 con 0x7fa480135e50 2023-10-03T10:18:10.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.991+0000 7fa486c43700 1 -- 172.21.15.3:0/3880213701 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa480135e50 msgr2=0x7fa480138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:10.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.991+0000 7fa486c43700 1 --2- 172.21.15.3:0/3880213701 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa480135e50 0x7fa480138230 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa47800be30 tx=0x7fa478010730 comp rx=0 tx=0).stop 2023-10-03T10:18:10.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.991+0000 7fa486c43700 1 -- 172.21.15.3:0/3880213701 shutdown_connections 2023-10-03T10:18:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.991+0000 7fa486c43700 1 --2- 172.21.15.3:0/3880213701 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa480135e50 0x7fa480138230 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.991+0000 7fa486c43700 1 --2- 172.21.15.3:0/3880213701 >> v2:172.21.15.3:3300/0 conn(0x7fa48009e1d0 0x7fa480135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.991+0000 7fa486c43700 1 -- 172.21.15.3:0/3880213701 >> 172.21.15.3:0/3880213701 conn(0x7fa480066f50 msgr2=0x7fa48009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.991+0000 7fa486c43700 1 -- 172.21.15.3:0/3880213701 shutdown_connections 2023-10-03T10:18:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.991+0000 7fa486c43700 1 -- 172.21.15.3:0/3880213701 wait complete. 2023-10-03T10:18:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.993+0000 7fa486c43700 1 Processor -- start 2023-10-03T10:18:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.993+0000 7fa486c43700 1 -- start start 2023-10-03T10:18:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.994+0000 7fa486c43700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa48009d6f0 0x7fa480119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.994+0000 7fa486c43700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa48009e1d0 0x7fa4801199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.994+0000 7fa486c43700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa480135e50 0x7fa480119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.994+0000 7fa486c43700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48011b440 con 0x7fa480135e50 2023-10-03T10:18:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.994+0000 7fa486c43700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa48011cd70 con 0x7fa48009e1d0 2023-10-03T10:18:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.994+0000 7fa486c43700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa480072630 con 0x7fa48009d6f0 2023-10-03T10:18:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.994+0000 7fa4849df700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa48009d6f0 0x7fa480119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.994+0000 7fa4849df700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa48009d6f0 0x7fa480119480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47906/0 (socket says 172.21.15.3:47906) 2023-10-03T10:18:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.994+0000 7fa4849df700 1 -- 172.21.15.3:0/849560627 learned_addr learned my addr 172.21.15.3:0/849560627 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.994+0000 7fa47ffff700 1 --2- 172.21.15.3:0/849560627 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa48009e1d0 0x7fa4801199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.994+0000 7fa4851e0700 1 --2- 172.21.15.3:0/849560627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa480135e50 0x7fa480119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:10.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.995+0000 7fa47ffff700 1 -- 172.21.15.3:0/849560627 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa48009d6f0 msgr2=0x7fa480119480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:10.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.995+0000 7fa47ffff700 1 --2- 172.21.15.3:0/849560627 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa48009d6f0 0x7fa480119480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:10.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.995+0000 7fa47ffff700 1 -- 172.21.15.3:0/849560627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa480135e50 msgr2=0x7fa480119f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:10.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.995+0000 7fa47ffff700 1 --2- 172.21.15.3:0/849560627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa480135e50 0x7fa480119f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:10.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.995+0000 7fa47ffff700 1 -- 172.21.15.3:0/849560627 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47800ba90 con 0x7fa48009e1d0 2023-10-03T10:18:10.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.995+0000 7fa4851e0700 1 --2- 172.21.15.3:0/849560627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa480135e50 0x7fa480119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.995+0000 7fa47ffff700 1 --2- 172.21.15.3:0/849560627 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa48009e1d0 0x7fa4801199c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00e870 tx=0x7fa46c00eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.995+0000 7fa47d7fa700 1 -- 172.21.15.3:0/849560627 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c013680 con 0x7fa48009e1d0 2023-10-03T10:18:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.995+0000 7fa47d7fa700 1 -- 172.21.15.3:0/849560627 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46c0137e0 con 0x7fa48009e1d0 2023-10-03T10:18:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.995+0000 7fa486c43700 1 -- 172.21.15.3:0/849560627 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa48009c700 con 0x7fa48009e1d0 2023-10-03T10:18:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.995+0000 7fa486c43700 1 -- 172.21.15.3:0/849560627 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa48011b650 con 0x7fa48009e1d0 2023-10-03T10:18:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.996+0000 7fa47d7fa700 1 -- 172.21.15.3:0/849560627 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c013b90 con 0x7fa48009e1d0 2023-10-03T10:18:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.996+0000 7fa47d7fa700 1 -- 172.21.15.3:0/849560627 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa46c013cf0 con 0x7fa48009e1d0 2023-10-03T10:18:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.997+0000 7fa47d7fa700 1 -- 172.21.15.3:0/849560627 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa46c03f470 con 0x7fa48009e1d0 2023-10-03T10:18:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:10.998+0000 7fa486c43700 1 -- 172.21.15.3:0/849560627 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4600052e0 con 0x7fa48009e1d0 2023-10-03T10:18:11.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.003+0000 7fa47d7fa700 1 -- 172.21.15.3:0/849560627 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa46c01be10 con 0x7fa48009e1d0 2023-10-03T10:18:11.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.092+0000 7f4b6be4c700 1 Processor -- start 2023-10-03T10:18:11.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.092+0000 7f4b6be4c700 1 -- start start 2023-10-03T10:18:11.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.093+0000 7f4b6be4c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4b640a1510 0x7f4b64138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:11.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.093+0000 7f4b6be4c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4b64139250 0x7f4b6413b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:11.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.093+0000 7f4b6be4c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b6409edd0 con 0x7f4b640a0a30 2023-10-03T10:18:11.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.093+0000 7f4b6be4c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b640a18e0 con 0x7f4b64139250 2023-10-03T10:18:11.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.093+0000 7f4b6be4c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6409e1f0 con 0x7f4b640a1510 2023-10-03T10:18:11.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.093+0000 7f4b693e7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4b640a1510 0x7f4b64138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:11.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.093+0000 7f4b69be8700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4b640a0a30 0x7f4b640a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50640/0 (socket says 172.21.15.3:50640) 2023-10-03T10:18:11.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.093+0000 7f4b69be8700 1 -- 172.21.15.3:0/1712443643 learned_addr learned my addr 172.21.15.3:0/1712443643 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:11.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.093+0000 7f4b6a3e9700 1 --2- 172.21.15.3:0/1712443643 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4b64139250 0x7f4b6413b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:11.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.094+0000 7f4b6a3e9700 1 -- 172.21.15.3:0/1712443643 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4b640a1510 msgr2=0x7f4b64138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:11.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.094+0000 7f4b6a3e9700 1 --2- 172.21.15.3:0/1712443643 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4b640a1510 0x7f4b64138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:11.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.094+0000 7f4b68be6700 1 -- 172.21.15.3:0/1712443643 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3186936206 0 0) 0x7f4b6409edd0 con 0x7f4b640a0a30 2023-10-03T10:18:11.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.094+0000 7f4b6a3e9700 1 -- 172.21.15.3:0/1712443643 >> v1:172.21.15.3:6789/0 conn(0x7f4b640a0a30 legacy=0x7f4b640a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:11.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.094+0000 7f4b6a3e9700 1 -- 172.21.15.3:0/1712443643 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b6409dc50 con 0x7f4b64139250 2023-10-03T10:18:11.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.094+0000 7f4b693e7700 1 --2- 172.21.15.3:0/1712443643 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4b640a1510 0x7f4b64138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:11.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.094+0000 7f4b6a3e9700 1 --2- 172.21.15.3:0/1712443643 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4b64139250 0x7f4b6413b630 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009dc0 tx=0x7f4b5c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=befe1782bb7da5f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:11.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.094+0000 7f4b68be6700 1 -- 172.21.15.3:0/1712443643 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c00e070 con 0x7f4b64139250 2023-10-03T10:18:11.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.094+0000 7f4b68be6700 1 -- 172.21.15.3:0/1712443643 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5c00bb20 con 0x7f4b64139250 2023-10-03T10:18:11.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.094+0000 7f4b68be6700 1 -- 172.21.15.3:0/1712443643 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c012430 con 0x7f4b64139250 2023-10-03T10:18:11.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.095+0000 7f4b6be4c700 1 -- 172.21.15.3:0/1712443643 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4b64139250 msgr2=0x7f4b6413b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:11.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.095+0000 7f4b6be4c700 1 --2- 172.21.15.3:0/1712443643 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4b64139250 0x7f4b6413b630 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009dc0 tx=0x7f4b5c00b330 comp rx=0 tx=0).stop 2023-10-03T10:18:11.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.095+0000 7f4b6be4c700 1 -- 172.21.15.3:0/1712443643 shutdown_connections 2023-10-03T10:18:11.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.095+0000 7f4b6be4c700 1 --2- 172.21.15.3:0/1712443643 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4b64139250 0x7f4b6413b630 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:11.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.095+0000 7f4b6be4c700 1 --2- 172.21.15.3:0/1712443643 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4b640a1510 0x7f4b64138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:11.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.095+0000 7f4b6be4c700 1 -- 172.21.15.3:0/1712443643 >> 172.21.15.3:0/1712443643 conn(0x7f4b640a6550 msgr2=0x7f4b640a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:11.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.096+0000 7f4b6be4c700 1 -- 172.21.15.3:0/1712443643 shutdown_connections 2023-10-03T10:18:11.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.096+0000 7f4b6be4c700 1 -- 172.21.15.3:0/1712443643 wait complete. 2023-10-03T10:18:11.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.097+0000 7f4b6be4c700 1 Processor -- start 2023-10-03T10:18:11.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.097+0000 7f4b6be4c700 1 -- start start 2023-10-03T10:18:11.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.097+0000 7f4b6be4c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4b640a0a30 0x7f4b64122720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:11.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.097+0000 7f4b6be4c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b640a1510 0x7f4b64122c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:11.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.097+0000 7f4b6be4c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4b64139250 0x7f4b641231a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:11.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.097+0000 7f4b6be4c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b64138950 con 0x7f4b640a1510 2023-10-03T10:18:11.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.097+0000 7f4b6be4c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b64124bd0 con 0x7f4b64139250 2023-10-03T10:18:11.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.097+0000 7f4b6be4c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4b64126240 con 0x7f4b640a0a30 2023-10-03T10:18:11.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.097+0000 7f4b693e7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b640a1510 0x7f4b64122c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:11.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.097+0000 7f4b693e7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b640a1510 0x7f4b64122c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46890/0 (socket says 172.21.15.3:46890) 2023-10-03T10:18:11.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.097+0000 7f4b693e7700 1 -- 172.21.15.3:0/115530833 learned_addr learned my addr 172.21.15.3:0/115530833 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:11.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.097+0000 7f4b6a3e9700 1 --2- 172.21.15.3:0/115530833 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4b64139250 0x7f4b641231a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:11.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.098+0000 7f4b69be8700 1 --2- 172.21.15.3:0/115530833 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4b640a0a30 0x7f4b64122720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:11.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.098+0000 7f4b693e7700 1 -- 172.21.15.3:0/115530833 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4b640a0a30 msgr2=0x7f4b64122720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:11.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.098+0000 7f4b693e7700 1 --2- 172.21.15.3:0/115530833 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4b640a0a30 0x7f4b64122720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:11.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.098+0000 7f4b693e7700 1 -- 172.21.15.3:0/115530833 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4b64139250 msgr2=0x7f4b641231a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:11.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.098+0000 7f4b693e7700 1 --2- 172.21.15.3:0/115530833 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4b64139250 0x7f4b641231a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:11.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.098+0000 7f4b693e7700 1 -- 172.21.15.3:0/115530833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b540094a0 con 0x7f4b640a1510 2023-10-03T10:18:11.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.098+0000 7f4b693e7700 1 --2- 172.21.15.3:0/115530833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b640a1510 0x7f4b64122c60 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400ebe0 tx=0x7f4b5400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:11.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.098+0000 7f4b5a7fc700 1 -- 172.21.15.3:0/115530833 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b54009800 con 0x7f4b640a1510 2023-10-03T10:18:11.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.098+0000 7f4b5a7fc700 1 -- 172.21.15.3:0/115530833 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5400c910 con 0x7f4b640a1510 2023-10-03T10:18:11.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.098+0000 7f4b6be4c700 1 -- 172.21.15.3:0/115530833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b5c009a70 con 0x7f4b640a1510 2023-10-03T10:18:11.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.098+0000 7f4b5a7fc700 1 -- 172.21.15.3:0/115530833 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5400cc80 con 0x7f4b640a1510 2023-10-03T10:18:11.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.099+0000 7f4b6be4c700 1 -- 172.21.15.3:0/115530833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b640a0220 con 0x7f4b640a1510 2023-10-03T10:18:11.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.100+0000 7f4b5a7fc700 1 -- 172.21.15.3:0/115530833 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4b54013470 con 0x7f4b640a1510 2023-10-03T10:18:11.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.100+0000 7f4b6be4c700 1 -- 172.21.15.3:0/115530833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b64074ea0 con 0x7f4b640a1510 2023-10-03T10:18:11.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.100+0000 7f4b5a7fc700 1 -- 172.21.15.3:0/115530833 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4b5403fbd0 con 0x7f4b640a1510 2023-10-03T10:18:11.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.105+0000 7f4b5a7fc700 1 -- 172.21.15.3:0/115530833 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4b54013720 con 0x7f4b640a1510 2023-10-03T10:18:11.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.119+0000 7fd4490ff700 1 -- 172.21.15.3:0/1754220570 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd444074ea0 con 0x7fd44409e410 2023-10-03T10:18:11.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.119+0000 7fd43bfff700 1 -- 172.21.15.3:0/1754220570 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd43c011380 con 0x7fd44409e410 2023-10-03T10:18:11.120 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:11.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.122+0000 7fd4490ff700 1 -- 172.21.15.3:0/1754220570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd44409e410 msgr2=0x7fd44411eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:11.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.122+0000 7fd4490ff700 1 --2- 172.21.15.3:0/1754220570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd44409e410 0x7fd44411eaa0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fd43c00cb90 tx=0x7fd43c00cea0 comp rx=0 tx=0).stop 2023-10-03T10:18:11.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.122+0000 7fd4490ff700 1 -- 172.21.15.3:0/1754220570 shutdown_connections 2023-10-03T10:18:11.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.122+0000 7fd4490ff700 1 --2- 172.21.15.3:0/1754220570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd44409e410 0x7fd44411eaa0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:11.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.122+0000 7fd4490ff700 1 --2- 172.21.15.3:0/1754220570 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd44409d6f0 0x7fd44411e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:11.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.122+0000 7fd4490ff700 1 --2- 172.21.15.3:0/1754220570 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd44409e000 0x7fd44411e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:11.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.122+0000 7fd4490ff700 1 -- 172.21.15.3:0/1754220570 >> 172.21.15.3:0/1754220570 conn(0x7fd444066f50 msgr2=0x7fd4440710a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:11.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.122+0000 7fd4490ff700 1 -- 172.21.15.3:0/1754220570 shutdown_connections 2023-10-03T10:18:11.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.122+0000 7fd4490ff700 1 -- 172.21.15.3:0/1754220570 wait complete. 2023-10-03T10:18:11.137 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:11.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.138+0000 7fb5eb279700 1 Processor -- start 2023-10-03T10:18:11.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.138+0000 7fb5eb279700 1 -- start start 2023-10-03T10:18:11.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.139+0000 7fb5eb279700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5e409e1d0 0x7fb5e4138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:11.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.139+0000 7fb5eb279700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5e4139170 0x7fb5e413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:11.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.139+0000 7fb5eb279700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5e409b120 con 0x7fb5e409d6f0 2023-10-03T10:18:11.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.139+0000 7fb5eb279700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e406b460 con 0x7fb5e4139170 2023-10-03T10:18:11.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.139+0000 7fb5eb279700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e409a7e0 con 0x7fb5e409e1d0 2023-10-03T10:18:11.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.139+0000 7fb5e9816700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5e4139170 0x7fb5e413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:11.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.139+0000 7fb5e9015700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb5e409d6f0 0x7fb5e409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50654/0 (socket says 172.21.15.3:50654) 2023-10-03T10:18:11.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.139+0000 7fb5e9015700 1 -- 172.21.15.3:0/200185442 learned_addr learned my addr 172.21.15.3:0/200185442 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:11.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.139+0000 7fb5e8814700 1 --2- 172.21.15.3:0/200185442 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5e409e1d0 0x7fb5e4138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:11.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.140+0000 7fb5e9816700 1 -- 172.21.15.3:0/200185442 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5e409e1d0 msgr2=0x7fb5e4138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:11.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.140+0000 7fb5e9816700 1 --2- 172.21.15.3:0/200185442 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5e409e1d0 0x7fb5e4138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:11.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.140+0000 7fb5dbfff700 1 -- 172.21.15.3:0/200185442 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1518513733 0 0) 0x7fb5e409b120 con 0x7fb5e409d6f0 2023-10-03T10:18:11.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.140+0000 7fb5e9816700 1 -- 172.21.15.3:0/200185442 >> v1:172.21.15.3:6789/0 conn(0x7fb5e409d6f0 legacy=0x7fb5e409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:11.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.140+0000 7fb5e9816700 1 -- 172.21.15.3:0/200185442 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e409a500 con 0x7fb5e4139170 2023-10-03T10:18:11.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.140+0000 7fb5e9816700 1 --2- 172.21.15.3:0/200185442 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5e4139170 0x7fb5e413b550 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc009dc0 tx=0x7fb5dc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=918892bef4948909 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:11.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.140+0000 7fb5dbfff700 1 -- 172.21.15.3:0/200185442 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc00e070 con 0x7fb5e4139170 2023-10-03T10:18:11.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.140+0000 7fb5dbfff700 1 -- 172.21.15.3:0/200185442 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5dc00bb20 con 0x7fb5e4139170 2023-10-03T10:18:11.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.140+0000 7fb5dbfff700 1 -- 172.21.15.3:0/200185442 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc012430 con 0x7fb5e4139170 2023-10-03T10:18:11.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.141+0000 7fb5eb279700 1 -- 172.21.15.3:0/200185442 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5e4139170 msgr2=0x7fb5e413b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:11.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.141+0000 7fb5eb279700 1 --2- 172.21.15.3:0/200185442 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5e4139170 0x7fb5e413b550 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc009dc0 tx=0x7fb5dc00b330 comp rx=0 tx=0).stop 2023-10-03T10:18:11.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.141+0000 7fb5eb279700 1 -- 172.21.15.3:0/200185442 shutdown_connections 2023-10-03T10:18:11.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.141+0000 7fb5eb279700 1 --2- 172.21.15.3:0/200185442 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5e4139170 0x7fb5e413b550 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:11.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.141+0000 7fb5eb279700 1 --2- 172.21.15.3:0/200185442 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5e409e1d0 0x7fb5e4138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:11.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.141+0000 7fb5eb279700 1 -- 172.21.15.3:0/200185442 >> 172.21.15.3:0/200185442 conn(0x7fb5e4066f50 msgr2=0x7fb5e409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:11.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.141+0000 7fb5eb279700 1 -- 172.21.15.3:0/200185442 shutdown_connections 2023-10-03T10:18:11.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.141+0000 7fb5eb279700 1 -- 172.21.15.3:0/200185442 wait complete. 2023-10-03T10:18:11.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.142+0000 7fb5eb279700 1 Processor -- start 2023-10-03T10:18:11.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.142+0000 7fb5eb279700 1 -- start start 2023-10-03T10:18:11.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.143+0000 7fb5eb279700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5e409d6f0 0x7fb5e41205f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:11.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.143+0000 7fb5eb279700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5e409e1d0 0x7fb5e4120b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:11.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.143+0000 7fb5eb279700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5e4139170 0x7fb5e4121070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:11.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.143+0000 7fb5eb279700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e4123f30 con 0x7fb5e409d6f0 2023-10-03T10:18:11.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.143+0000 7fb5eb279700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e4124110 con 0x7fb5e4139170 2023-10-03T10:18:11.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.143+0000 7fb5eb279700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e41230a0 con 0x7fb5e409e1d0 2023-10-03T10:18:11.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.143+0000 7fb5e9015700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5e409d6f0 0x7fb5e41205f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:11.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.143+0000 7fb5e9015700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5e409d6f0 0x7fb5e41205f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46894/0 (socket says 172.21.15.3:46894) 2023-10-03T10:18:11.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.143+0000 7fb5e9015700 1 -- 172.21.15.3:0/4058646381 learned_addr learned my addr 172.21.15.3:0/4058646381 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:11.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.143+0000 7fb5e9816700 1 --2- 172.21.15.3:0/4058646381 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5e4139170 0x7fb5e4121070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:11.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.143+0000 7fb5e8814700 1 --2- 172.21.15.3:0/4058646381 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5e409e1d0 0x7fb5e4120b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:11.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.144+0000 7fb5e9816700 1 -- 172.21.15.3:0/4058646381 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5e409e1d0 msgr2=0x7fb5e4120b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:11.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.144+0000 7fb5e9816700 1 --2- 172.21.15.3:0/4058646381 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5e409e1d0 0x7fb5e4120b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:11.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.144+0000 7fb5e9816700 1 -- 172.21.15.3:0/4058646381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5e409d6f0 msgr2=0x7fb5e41205f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:11.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.144+0000 7fb5e9816700 1 --2- 172.21.15.3:0/4058646381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5e409d6f0 0x7fb5e41205f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:11.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.144+0000 7fb5e9816700 1 -- 172.21.15.3:0/4058646381 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5dc009a70 con 0x7fb5e4139170 2023-10-03T10:18:11.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.144+0000 7fb5e9816700 1 --2- 172.21.15.3:0/4058646381 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5e4139170 0x7fb5e4121070 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc009dc0 tx=0x7fb5dc004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:11.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.144+0000 7fb5d9ffb700 1 -- 172.21.15.3:0/4058646381 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc01b810 con 0x7fb5e4139170 2023-10-03T10:18:11.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.144+0000 7fb5d9ffb700 1 -- 172.21.15.3:0/4058646381 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5dc020070 con 0x7fb5e4139170 2023-10-03T10:18:11.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.144+0000 7fb5eb279700 1 -- 172.21.15.3:0/4058646381 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e4072150 con 0x7fb5e4139170 2023-10-03T10:18:11.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.144+0000 7fb5d9ffb700 1 -- 172.21.15.3:0/4058646381 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc01bbd0 con 0x7fb5e4139170 2023-10-03T10:18:11.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.144+0000 7fb5eb279700 1 -- 172.21.15.3:0/4058646381 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e4136820 con 0x7fb5e4139170 2023-10-03T10:18:11.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.146+0000 7fb5eb279700 1 -- 172.21.15.3:0/4058646381 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e4074ea0 con 0x7fb5e4139170 2023-10-03T10:18:11.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.146+0000 7fb5d9ffb700 1 -- 172.21.15.3:0/4058646381 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb5dc01bd30 con 0x7fb5e4139170 2023-10-03T10:18:11.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.147+0000 7fb5d9ffb700 1 -- 172.21.15.3:0/4058646381 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb5dc022070 con 0x7fb5e4139170 2023-10-03T10:18:11.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.151+0000 7fb5d9ffb700 1 -- 172.21.15.3:0/4058646381 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb5dc025790 con 0x7fb5e4139170 2023-10-03T10:18:11.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.229+0000 7fa486c43700 1 -- 172.21.15.3:0/849560627 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa460005590 con 0x7fa48009e1d0 2023-10-03T10:18:11.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.230+0000 7fa47d7fa700 1 -- 172.21.15.3:0/849560627 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa46c021030 con 0x7fa48009e1d0 2023-10-03T10:18:11.230 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:11.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.232+0000 7fa486c43700 1 -- 172.21.15.3:0/849560627 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa48009e1d0 msgr2=0x7fa4801199c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:11.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.232+0000 7fa486c43700 1 --2- 172.21.15.3:0/849560627 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa48009e1d0 0x7fa4801199c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00e870 tx=0x7fa46c00eb80 comp rx=0 tx=0).stop 2023-10-03T10:18:11.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.232+0000 7fa486c43700 1 -- 172.21.15.3:0/849560627 shutdown_connections 2023-10-03T10:18:11.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.232+0000 7fa486c43700 1 --2- 172.21.15.3:0/849560627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa480135e50 0x7fa480119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:11.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.232+0000 7fa486c43700 1 --2- 172.21.15.3:0/849560627 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa48009d6f0 0x7fa480119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:11.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.232+0000 7fa486c43700 1 --2- 172.21.15.3:0/849560627 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa48009e1d0 0x7fa4801199c0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:11.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.232+0000 7fa486c43700 1 -- 172.21.15.3:0/849560627 >> 172.21.15.3:0/849560627 conn(0x7fa480066f50 msgr2=0x7fa4800a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:11.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.232+0000 7fa486c43700 1 -- 172.21.15.3:0/849560627 shutdown_connections 2023-10-03T10:18:11.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.232+0000 7fa486c43700 1 -- 172.21.15.3:0/849560627 wait complete. 2023-10-03T10:18:11.246 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:11.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.330+0000 7f4b6be4c700 1 -- 172.21.15.3:0/115530833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4b640a28e0 con 0x7f4b640a1510 2023-10-03T10:18:11.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.331+0000 7f4b5a7fc700 1 -- 172.21.15.3:0/115530833 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4b54021070 con 0x7f4b640a1510 2023-10-03T10:18:11.331 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:11.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.333+0000 7f4b6be4c700 1 -- 172.21.15.3:0/115530833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b640a1510 msgr2=0x7f4b64122c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:11.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.333+0000 7f4b6be4c700 1 --2- 172.21.15.3:0/115530833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b640a1510 0x7f4b64122c60 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400ebe0 tx=0x7f4b5400eef0 comp rx=0 tx=0).stop 2023-10-03T10:18:11.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.333+0000 7f4b6be4c700 1 -- 172.21.15.3:0/115530833 shutdown_connections 2023-10-03T10:18:11.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.333+0000 7f4b6be4c700 1 --2- 172.21.15.3:0/115530833 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4b64139250 0x7f4b641231a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:11.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.333+0000 7f4b6be4c700 1 --2- 172.21.15.3:0/115530833 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4b640a0a30 0x7f4b64122720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:11.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.333+0000 7f4b6be4c700 1 --2- 172.21.15.3:0/115530833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b640a1510 0x7f4b64122c60 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:11.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.333+0000 7f4b6be4c700 1 -- 172.21.15.3:0/115530833 >> 172.21.15.3:0/115530833 conn(0x7f4b640a6550 msgr2=0x7f4b6411c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:11.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.333+0000 7f4b6be4c700 1 -- 172.21.15.3:0/115530833 shutdown_connections 2023-10-03T10:18:11.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.333+0000 7f4b6be4c700 1 -- 172.21.15.3:0/115530833 wait complete. 2023-10-03T10:18:11.347 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:11.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.360+0000 7fb5eb279700 1 -- 172.21.15.3:0/4058646381 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb5e406c360 con 0x7fb5e4139170 2023-10-03T10:18:11.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.360+0000 7fb5d9ffb700 1 -- 172.21.15.3:0/4058646381 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb5dc035020 con 0x7fb5e4139170 2023-10-03T10:18:11.361 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:11.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.363+0000 7fb5eb279700 1 -- 172.21.15.3:0/4058646381 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5e4139170 msgr2=0x7fb5e4121070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:11.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.363+0000 7fb5eb279700 1 --2- 172.21.15.3:0/4058646381 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5e4139170 0x7fb5e4121070 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc009dc0 tx=0x7fb5dc004ad0 comp rx=0 tx=0).stop 2023-10-03T10:18:11.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.363+0000 7fb5eb279700 1 -- 172.21.15.3:0/4058646381 shutdown_connections 2023-10-03T10:18:11.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.363+0000 7fb5eb279700 1 --2- 172.21.15.3:0/4058646381 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5e4139170 0x7fb5e4121070 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:11.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.363+0000 7fb5eb279700 1 --2- 172.21.15.3:0/4058646381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5e409d6f0 0x7fb5e41205f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:11.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.363+0000 7fb5eb279700 1 --2- 172.21.15.3:0/4058646381 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5e409e1d0 0x7fb5e4120b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:11.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.363+0000 7fb5eb279700 1 -- 172.21.15.3:0/4058646381 >> 172.21.15.3:0/4058646381 conn(0x7fb5e4066f50 msgr2=0x7fb5e4070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:11.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.363+0000 7fb5eb279700 1 -- 172.21.15.3:0/4058646381 shutdown_connections 2023-10-03T10:18:11.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:11.363+0000 7fb5eb279700 1 -- 172.21.15.3:0/4058646381 wait complete. 2023-10-03T10:18:11.378 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:12.137 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:12.247 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:12.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.286+0000 7fbd9a821700 1 Processor -- start 2023-10-03T10:18:12.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.286+0000 7fbd9a821700 1 -- start start 2023-10-03T10:18:12.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.286+0000 7fbd9a821700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbd940a0a30 0x7fbd940a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:12.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.286+0000 7fbd9a821700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd9409d630 0x7fbd940a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:12.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.286+0000 7fbd9a821700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd940a2a70 con 0x7fbd94133930 2023-10-03T10:18:12.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.286+0000 7fbd9a821700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd940a01d0 con 0x7fbd940a0a30 2023-10-03T10:18:12.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.286+0000 7fbd9a821700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9409e2b0 con 0x7fbd9409d630 2023-10-03T10:18:12.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.286+0000 7fbd937fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd9409d630 0x7fbd940a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:12.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.286+0000 7fbd93fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbd940a0a30 0x7fbd940a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:12.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.286+0000 7fbd937fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd9409d630 0x7fbd940a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42170/0 (socket says 172.21.15.3:42170) 2023-10-03T10:18:12.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.286+0000 7fbd937fe700 1 -- 172.21.15.3:0/3226208351 learned_addr learned my addr 172.21.15.3:0/3226208351 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:12.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.287+0000 7fbd937fe700 1 -- 172.21.15.3:0/3226208351 >> v2:172.21.15.3:3300/0 conn(0x7fbd940a0a30 msgr2=0x7fbd940a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:12.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.287+0000 7fbd937fe700 1 --2- 172.21.15.3:0/3226208351 >> v2:172.21.15.3:3300/0 conn(0x7fbd940a0a30 0x7fbd940a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.287+0000 7fbd937fe700 1 -- 172.21.15.3:0/3226208351 >> v1:172.21.15.3:6789/0 conn(0x7fbd94133930 legacy=0x7fbd94133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:12.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.287+0000 7fbd937fe700 1 -- 172.21.15.3:0/3226208351 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9409dc50 con 0x7fbd9409d630 2023-10-03T10:18:12.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.287+0000 7fbd93fff700 1 --2- 172.21.15.3:0/3226208351 >> v2:172.21.15.3:3300/0 conn(0x7fbd940a0a30 0x7fbd940a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:12.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.287+0000 7fbd937fe700 1 --2- 172.21.15.3:0/3226208351 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd9409d630 0x7fbd940a1340 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fbd880098f0 tx=0x7fbd88009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1062fec9e4ff56da server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:12.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.287+0000 7fbd92ffd700 1 -- 172.21.15.3:0/3226208351 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd88011830 con 0x7fbd9409d630 2023-10-03T10:18:12.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.287+0000 7fbd92ffd700 1 -- 172.21.15.3:0/3226208351 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd88011990 con 0x7fbd9409d630 2023-10-03T10:18:12.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.287+0000 7fbd92ffd700 1 -- 172.21.15.3:0/3226208351 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd88011d20 con 0x7fbd9409d630 2023-10-03T10:18:12.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.288+0000 7fbd9a821700 1 -- 172.21.15.3:0/3226208351 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd9409d630 msgr2=0x7fbd940a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:12.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.288+0000 7fbd9a821700 1 --2- 172.21.15.3:0/3226208351 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd9409d630 0x7fbd940a1340 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fbd880098f0 tx=0x7fbd88009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:12.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.288+0000 7fbd9a821700 1 -- 172.21.15.3:0/3226208351 shutdown_connections 2023-10-03T10:18:12.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.288+0000 7fbd9a821700 1 --2- 172.21.15.3:0/3226208351 >> v2:172.21.15.3:3300/0 conn(0x7fbd940a0a30 0x7fbd940a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.288+0000 7fbd9a821700 1 --2- 172.21.15.3:0/3226208351 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd9409d630 0x7fbd940a1340 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.288+0000 7fbd9a821700 1 -- 172.21.15.3:0/3226208351 >> 172.21.15.3:0/3226208351 conn(0x7fbd940a3050 msgr2=0x7fbd940a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:12.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.288+0000 7fbd9a821700 1 -- 172.21.15.3:0/3226208351 shutdown_connections 2023-10-03T10:18:12.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.288+0000 7fbd9a821700 1 -- 172.21.15.3:0/3226208351 wait complete. 2023-10-03T10:18:12.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.289+0000 7fbd9a821700 1 Processor -- start 2023-10-03T10:18:12.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.289+0000 7fbd9a821700 1 -- start start 2023-10-03T10:18:12.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.290+0000 7fbd9a821700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd9409d630 0x7fbd94121100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:12.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.290+0000 7fbd9a821700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd940a0a30 0x7fbd94121640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:12.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.290+0000 7fbd9a821700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd94133930 0x7fbd94121b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:12.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.290+0000 7fbd9a821700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd941225b0 con 0x7fbd9409d630 2023-10-03T10:18:12.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.290+0000 7fbd9a821700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd94124bd0 con 0x7fbd940a0a30 2023-10-03T10:18:12.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.290+0000 7fbd9a821700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd94124630 con 0x7fbd94133930 2023-10-03T10:18:12.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.290+0000 7fbd93fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd9409d630 0x7fbd94121100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:12.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.290+0000 7fbd937fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd940a0a30 0x7fbd94121640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:12.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.290+0000 7fbd93fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd9409d630 0x7fbd94121100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46914/0 (socket says 172.21.15.3:46914) 2023-10-03T10:18:12.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.290+0000 7fbd93fff700 1 -- 172.21.15.3:0/459568597 learned_addr learned my addr 172.21.15.3:0/459568597 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:12.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.290+0000 7fbd98dbe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd94133930 0x7fbd94121b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:12.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.290+0000 7fbd93fff700 1 -- 172.21.15.3:0/459568597 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd94133930 msgr2=0x7fbd94121b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:12.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.290+0000 7fbd93fff700 1 --2- 172.21.15.3:0/459568597 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd94133930 0x7fbd94121b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.290+0000 7fbd93fff700 1 -- 172.21.15.3:0/459568597 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd940a0a30 msgr2=0x7fbd94121640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:12.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.290+0000 7fbd93fff700 1 --2- 172.21.15.3:0/459568597 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd940a0a30 0x7fbd94121640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.290+0000 7fbd93fff700 1 -- 172.21.15.3:0/459568597 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd840094a0 con 0x7fbd9409d630 2023-10-03T10:18:12.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.290+0000 7fbd937fe700 1 --2- 172.21.15.3:0/459568597 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd940a0a30 0x7fbd94121640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:12.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.290+0000 7fbd93fff700 1 --2- 172.21.15.3:0/459568597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd9409d630 0x7fbd94121100 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400ebe0 tx=0x7fbd8400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:12.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.291+0000 7fbd90ff9700 1 -- 172.21.15.3:0/459568597 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8400c850 con 0x7fbd9409d630 2023-10-03T10:18:12.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.291+0000 7fbd9a821700 1 -- 172.21.15.3:0/459568597 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd88009550 con 0x7fbd9409d630 2023-10-03T10:18:12.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.291+0000 7fbd9a821700 1 -- 172.21.15.3:0/459568597 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd940a0410 con 0x7fbd9409d630 2023-10-03T10:18:12.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.291+0000 7fbd90ff9700 1 -- 172.21.15.3:0/459568597 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8400c9b0 con 0x7fbd9409d630 2023-10-03T10:18:12.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.291+0000 7fbd90ff9700 1 -- 172.21.15.3:0/459568597 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8400cd20 con 0x7fbd9409d630 2023-10-03T10:18:12.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.291+0000 7fbd90ff9700 1 -- 172.21.15.3:0/459568597 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbd84013470 con 0x7fbd9409d630 2023-10-03T10:18:12.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.292+0000 7fbd90ff9700 1 -- 172.21.15.3:0/459568597 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fbd8403fb50 con 0x7fbd9409d630 2023-10-03T10:18:12.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.292+0000 7fbd9a821700 1 -- 172.21.15.3:0/459568597 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd740052e0 con 0x7fbd9409d630 2023-10-03T10:18:12.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.297+0000 7fbd90ff9700 1 -- 172.21.15.3:0/459568597 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbd84021020 con 0x7fbd9409d630 2023-10-03T10:18:12.348 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:12.379 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:12.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.387+0000 7fa9b9568700 1 Processor -- start 2023-10-03T10:18:12.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.387+0000 7fa9b9568700 1 -- start start 2023-10-03T10:18:12.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.388+0000 7fa9b9568700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa9b409d6f0 0x7fa9b409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:12.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.388+0000 7fa9b9568700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9b409e000 0x7fa9b41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:12.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.388+0000 7fa9b9568700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9b409e410 0x7fa9b4133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:12.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.388+0000 7fa9b9568700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9b406b460 con 0x7fa9b409d6f0 2023-10-03T10:18:12.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.388+0000 7fa9b9568700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b409b120 con 0x7fa9b409e000 2023-10-03T10:18:12.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.388+0000 7fa9b9568700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b409a7e0 con 0x7fa9b409e410 2023-10-03T10:18:12.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.388+0000 7fa9b27fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9b409e000 0x7fa9b41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:12.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.388+0000 7fa9b27fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9b409e000 0x7fa9b41339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42192/0 (socket says 172.21.15.3:42192) 2023-10-03T10:18:12.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.388+0000 7fa9b27fc700 1 -- 172.21.15.3:0/1748746419 learned_addr learned my addr 172.21.15.3:0/1748746419 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:12.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.388+0000 7fa9b2ffd700 1 --2- 172.21.15.3:0/1748746419 >> v2:172.21.15.3:3300/0 conn(0x7fa9b409d6f0 0x7fa9b409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:12.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.388+0000 7fa9b37fe700 1 --2- 172.21.15.3:0/1748746419 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9b409e410 0x7fa9b4133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:12.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.388+0000 7fa9b2ffd700 1 -- 172.21.15.3:0/1748746419 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9b409e410 msgr2=0x7fa9b4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:12.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.388+0000 7fa9b2ffd700 1 --2- 172.21.15.3:0/1748746419 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9b409e410 0x7fa9b4133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.388+0000 7fa9b2ffd700 1 -- 172.21.15.3:0/1748746419 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9b409e000 msgr2=0x7fa9b41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:12.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.388+0000 7fa9b2ffd700 1 --2- 172.21.15.3:0/1748746419 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9b409e000 0x7fa9b41339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.388+0000 7fa9b2ffd700 1 -- 172.21.15.3:0/1748746419 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b409a500 con 0x7fa9b409d6f0 2023-10-03T10:18:12.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.388+0000 7fa9b27fc700 1 --2- 172.21.15.3:0/1748746419 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9b409e000 0x7fa9b41339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:12.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.388+0000 7fa9b2ffd700 1 --2- 172.21.15.3:0/1748746419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b409d6f0 0x7fa9b409dac0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4009840 tx=0x7fa9a4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4ea963b96b829f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:12.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.389+0000 7fa9b1ffb700 1 -- 172.21.15.3:0/1748746419 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4011830 con 0x7fa9b409d6f0 2023-10-03T10:18:12.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.389+0000 7fa9b1ffb700 1 -- 172.21.15.3:0/1748746419 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a4011990 con 0x7fa9b409d6f0 2023-10-03T10:18:12.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.389+0000 7fa9b1ffb700 1 -- 172.21.15.3:0/1748746419 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4011d20 con 0x7fa9b409d6f0 2023-10-03T10:18:12.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.389+0000 7fa9b9568700 1 -- 172.21.15.3:0/1748746419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b409d6f0 msgr2=0x7fa9b409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:12.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.389+0000 7fa9b9568700 1 --2- 172.21.15.3:0/1748746419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b409d6f0 0x7fa9b409dac0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4009840 tx=0x7fa9a4009b50 comp rx=0 tx=0).stop 2023-10-03T10:18:12.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.389+0000 7fa9b9568700 1 -- 172.21.15.3:0/1748746419 shutdown_connections 2023-10-03T10:18:12.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.390+0000 7fa9b9568700 1 --2- 172.21.15.3:0/1748746419 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9b409e410 0x7fa9b4133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.390+0000 7fa9b9568700 1 --2- 172.21.15.3:0/1748746419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b409d6f0 0x7fa9b409dac0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.390+0000 7fa9b9568700 1 --2- 172.21.15.3:0/1748746419 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9b409e000 0x7fa9b41339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.390+0000 7fa9b9568700 1 -- 172.21.15.3:0/1748746419 >> 172.21.15.3:0/1748746419 conn(0x7fa9b4066f50 msgr2=0x7fa9b409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:12.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.390+0000 7fa9b9568700 1 -- 172.21.15.3:0/1748746419 shutdown_connections 2023-10-03T10:18:12.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.390+0000 7fa9b9568700 1 -- 172.21.15.3:0/1748746419 wait complete. 2023-10-03T10:18:12.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.392+0000 7fa9b9568700 1 Processor -- start 2023-10-03T10:18:12.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.392+0000 7fa9b9568700 1 -- start start 2023-10-03T10:18:12.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.392+0000 7fa9b9568700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b409d6f0 0x7fa9b41a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:12.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.392+0000 7fa9b9568700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9b409e000 0x7fa9b41a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:12.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.392+0000 7fa9b9568700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9b409e410 0x7fa9b41c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:12.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.392+0000 7fa9b9568700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b40a1f20 con 0x7fa9b409d6f0 2023-10-03T10:18:12.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.392+0000 7fa9b9568700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b41a80a0 con 0x7fa9b409e000 2023-10-03T10:18:12.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.392+0000 7fa9b9568700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b41b91f0 con 0x7fa9b409e410 2023-10-03T10:18:12.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.392+0000 7fa9b2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b409d6f0 0x7fa9b41a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:12.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.392+0000 7fa9b27fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9b409e000 0x7fa9b41a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:12.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.392+0000 7fa9b2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b409d6f0 0x7fa9b41a5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46936/0 (socket says 172.21.15.3:46936) 2023-10-03T10:18:12.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.392+0000 7fa9b2ffd700 1 -- 172.21.15.3:0/68345275 learned_addr learned my addr 172.21.15.3:0/68345275 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:12.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.393+0000 7fa9b37fe700 1 --2- 172.21.15.3:0/68345275 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9b409e410 0x7fa9b41c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:12.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.393+0000 7fa9b27fc700 1 -- 172.21.15.3:0/68345275 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9b409e410 msgr2=0x7fa9b41c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:12.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.393+0000 7fa9b27fc700 1 --2- 172.21.15.3:0/68345275 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9b409e410 0x7fa9b41c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.393+0000 7fa9b27fc700 1 -- 172.21.15.3:0/68345275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b409d6f0 msgr2=0x7fa9b41a5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:12.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.393+0000 7fa9b27fc700 1 --2- 172.21.15.3:0/68345275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b409d6f0 0x7fa9b41a5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.393+0000 7fa9b27fc700 1 -- 172.21.15.3:0/68345275 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a8009550 con 0x7fa9b409e000 2023-10-03T10:18:12.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.393+0000 7fa9b27fc700 1 --2- 172.21.15.3:0/68345275 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9b409e000 0x7fa9b41a5670 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800e9d0 tx=0x7fa9a800ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:12.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.393+0000 7fa99b7fe700 1 -- 172.21.15.3:0/68345275 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a800ee70 con 0x7fa9b409e000 2023-10-03T10:18:12.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.393+0000 7fa99b7fe700 1 -- 172.21.15.3:0/68345275 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a8004030 con 0x7fa9b409e000 2023-10-03T10:18:12.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.393+0000 7fa99b7fe700 1 -- 172.21.15.3:0/68345275 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a80043c0 con 0x7fa9b409e000 2023-10-03T10:18:12.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.394+0000 7fa9b9568700 1 -- 172.21.15.3:0/68345275 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a40094a0 con 0x7fa9b409e000 2023-10-03T10:18:12.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.394+0000 7fa9b9568700 1 -- 172.21.15.3:0/68345275 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b41a6c60 con 0x7fa9b409e000 2023-10-03T10:18:12.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.395+0000 7fa99b7fe700 1 -- 172.21.15.3:0/68345275 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa9a800c850 con 0x7fa9b409e000 2023-10-03T10:18:12.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.395+0000 7fa99b7fe700 1 -- 172.21.15.3:0/68345275 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa9a803f9f0 con 0x7fa9b409e000 2023-10-03T10:18:12.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.395+0000 7fa9b9568700 1 -- 172.21.15.3:0/68345275 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9b41a81e0 con 0x7fa9b409e000 2023-10-03T10:18:12.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.401+0000 7fa99b7fe700 1 -- 172.21.15.3:0/68345275 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa9b41a81e0 con 0x7fa9b409e000 2023-10-03T10:18:12.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.495+0000 7fdeb4549700 1 Processor -- start 2023-10-03T10:18:12.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.495+0000 7fdeb4549700 1 -- start start 2023-10-03T10:18:12.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.495+0000 7fdeb4549700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdeac09d6f0 0x7fdeac09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:12.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.495+0000 7fdeb4549700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdeac09e000 0x7fdeac1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:12.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.495+0000 7fdeb4549700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdeac09e410 0x7fdeac133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:12.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.495+0000 7fdeb4549700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeac06b460 con 0x7fdeac09d6f0 2023-10-03T10:18:12.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.495+0000 7fdeb4549700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac09b120 con 0x7fdeac09e000 2023-10-03T10:18:12.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.495+0000 7fdeb4549700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac09a7e0 con 0x7fdeac09e410 2023-10-03T10:18:12.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.496+0000 7fdeb1ae4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdeac09e000 0x7fdeac1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:12.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.496+0000 7fdeb2ae6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdeac09e410 0x7fdeac133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:12.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.496+0000 7fdeb2ae6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdeac09e410 0x7fdeac133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47978/0 (socket says 172.21.15.3:47978) 2023-10-03T10:18:12.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.496+0000 7fdeb1ae4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdeac09e000 0x7fdeac1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42216/0 (socket says 172.21.15.3:42216) 2023-10-03T10:18:12.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.496+0000 7fdeb2ae6700 1 -- 172.21.15.3:0/608678879 learned_addr learned my addr 172.21.15.3:0/608678879 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:12.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.496+0000 7fdeb22e5700 1 --2- 172.21.15.3:0/608678879 >> v2:172.21.15.3:3300/0 conn(0x7fdeac09d6f0 0x7fdeac09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:12.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.496+0000 7fdeb2ae6700 1 -- 172.21.15.3:0/608678879 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdeac09e000 msgr2=0x7fdeac1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:12.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.496+0000 7fdeb2ae6700 1 --2- 172.21.15.3:0/608678879 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdeac09e000 0x7fdeac1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.496+0000 7fdeb2ae6700 1 -- 172.21.15.3:0/608678879 >> v2:172.21.15.3:3300/0 conn(0x7fdeac09d6f0 msgr2=0x7fdeac09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:12.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.496+0000 7fdeb2ae6700 1 --2- 172.21.15.3:0/608678879 >> v2:172.21.15.3:3300/0 conn(0x7fdeac09d6f0 0x7fdeac09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.496+0000 7fdeb2ae6700 1 -- 172.21.15.3:0/608678879 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeac09a500 con 0x7fdeac09e410 2023-10-03T10:18:12.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.496+0000 7fdeb1ae4700 1 --2- 172.21.15.3:0/608678879 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdeac09e000 0x7fdeac1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:12.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.496+0000 7fdeb22e5700 1 --2- 172.21.15.3:0/608678879 >> v2:172.21.15.3:3300/0 conn(0x7fdeac09d6f0 0x7fdeac09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:18:12.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.496+0000 7fdeb2ae6700 1 --2- 172.21.15.3:0/608678879 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdeac09e410 0x7fdeac133f30 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009e80 tx=0x7fdea400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f5ac712c41d08c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:12.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.497+0000 7fdeb12e3700 1 -- 172.21.15.3:0/608678879 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea400e070 con 0x7fdeac09e410 2023-10-03T10:18:12.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.497+0000 7fdeb12e3700 1 -- 172.21.15.3:0/608678879 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea400b8d0 con 0x7fdeac09e410 2023-10-03T10:18:12.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.497+0000 7fdeb12e3700 1 -- 172.21.15.3:0/608678879 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea400bc80 con 0x7fdeac09e410 2023-10-03T10:18:12.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.497+0000 7fdeb4549700 1 -- 172.21.15.3:0/608678879 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdeac09e410 msgr2=0x7fdeac133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:12.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.497+0000 7fdeb4549700 1 --2- 172.21.15.3:0/608678879 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdeac09e410 0x7fdeac133f30 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009e80 tx=0x7fdea400b330 comp rx=0 tx=0).stop 2023-10-03T10:18:12.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.497+0000 7fdeb4549700 1 -- 172.21.15.3:0/608678879 shutdown_connections 2023-10-03T10:18:12.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.497+0000 7fdeb4549700 1 --2- 172.21.15.3:0/608678879 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdeac09e410 0x7fdeac133f30 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.497+0000 7fdeb4549700 1 --2- 172.21.15.3:0/608678879 >> v2:172.21.15.3:3300/0 conn(0x7fdeac09d6f0 0x7fdeac09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.497+0000 7fdeb4549700 1 --2- 172.21.15.3:0/608678879 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdeac09e000 0x7fdeac1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.497+0000 7fdeb4549700 1 -- 172.21.15.3:0/608678879 >> 172.21.15.3:0/608678879 conn(0x7fdeac066f50 msgr2=0x7fdeac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:12.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.498+0000 7fdeb4549700 1 -- 172.21.15.3:0/608678879 shutdown_connections 2023-10-03T10:18:12.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.498+0000 7fdeb4549700 1 -- 172.21.15.3:0/608678879 wait complete. 2023-10-03T10:18:12.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.499+0000 7fdeb4549700 1 Processor -- start 2023-10-03T10:18:12.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.499+0000 7fdeb4549700 1 -- start start 2023-10-03T10:18:12.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.499+0000 7fdeb4549700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdeac09d6f0 0x7fdeac1a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:12.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.499+0000 7fdeb4549700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdeac09e000 0x7fdeac1a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:12.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.499+0000 7fdeb4549700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdeac09e410 0x7fdeac1c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:12.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.499+0000 7fdeb4549700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac1b9110 con 0x7fdeac09e000 2023-10-03T10:18:12.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.499+0000 7fdeb4549700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac1a4cf0 con 0x7fdeac09d6f0 2023-10-03T10:18:12.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.499+0000 7fdeb4549700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac1a6390 con 0x7fdeac09e410 2023-10-03T10:18:12.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.499+0000 7fdeb22e5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdeac09d6f0 0x7fdeac1a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:12.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.499+0000 7fdeb22e5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdeac09d6f0 0x7fdeac1a50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42230/0 (socket says 172.21.15.3:42230) 2023-10-03T10:18:12.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.499+0000 7fdeb22e5700 1 -- 172.21.15.3:0/2201590705 learned_addr learned my addr 172.21.15.3:0/2201590705 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:12.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.499+0000 7fdeb2ae6700 1 --2- 172.21.15.3:0/2201590705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdeac09e410 0x7fdeac1c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:12.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.500+0000 7fdeb1ae4700 1 --2- 172.21.15.3:0/2201590705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdeac09e000 0x7fdeac1a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:12.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.500+0000 7fdeb2ae6700 1 -- 172.21.15.3:0/2201590705 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdeac09d6f0 msgr2=0x7fdeac1a50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:12.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.500+0000 7fdeb2ae6700 1 --2- 172.21.15.3:0/2201590705 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdeac09d6f0 0x7fdeac1a50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.500+0000 7fdeb2ae6700 1 -- 172.21.15.3:0/2201590705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdeac09e000 msgr2=0x7fdeac1a5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:12.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.500+0000 7fdeb2ae6700 1 --2- 172.21.15.3:0/2201590705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdeac09e000 0x7fdeac1a5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.500+0000 7fdeb2ae6700 1 -- 172.21.15.3:0/2201590705 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9c0094a0 con 0x7fdeac09e410 2023-10-03T10:18:12.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.500+0000 7fdeb1ae4700 1 --2- 172.21.15.3:0/2201590705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdeac09e000 0x7fdeac1a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:18:12.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.500+0000 7fdeb2ae6700 1 --2- 172.21.15.3:0/2201590705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdeac09e410 0x7fdeac1c7f40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fdea4015000 tx=0x7fdea4004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:12.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.500+0000 7fdea2ffd700 1 -- 172.21.15.3:0/2201590705 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4004450 con 0x7fdeac09e410 2023-10-03T10:18:12.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.500+0000 7fdea2ffd700 1 -- 172.21.15.3:0/2201590705 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea40045f0 con 0x7fdeac09e410 2023-10-03T10:18:12.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.500+0000 7fdea2ffd700 1 -- 172.21.15.3:0/2201590705 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea40092e0 con 0x7fdeac09e410 2023-10-03T10:18:12.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.500+0000 7fdeb4549700 1 -- 172.21.15.3:0/2201590705 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea4009b30 con 0x7fdeac09e410 2023-10-03T10:18:12.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.500+0000 7fdeb4549700 1 -- 172.21.15.3:0/2201590705 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeac09c710 con 0x7fdeac09e410 2023-10-03T10:18:12.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.502+0000 7fdeb4549700 1 -- 172.21.15.3:0/2201590705 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeac074ea0 con 0x7fdeac09e410 2023-10-03T10:18:12.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.502+0000 7fdea2ffd700 1 -- 172.21.15.3:0/2201590705 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdea401c410 con 0x7fdeac09e410 2023-10-03T10:18:12.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.502+0000 7fdea2ffd700 1 -- 172.21.15.3:0/2201590705 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fdea4041320 con 0x7fdeac09e410 2023-10-03T10:18:12.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.512+0000 7fdea2ffd700 1 -- 172.21.15.3:0/2201590705 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdea401a7d0 con 0x7fdeac09e410 2023-10-03T10:18:12.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.516+0000 7fbd9a821700 1 -- 172.21.15.3:0/459568597 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbd74005590 con 0x7fbd9409d630 2023-10-03T10:18:12.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.517+0000 7fbd90ff9700 1 -- 172.21.15.3:0/459568597 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbd84017070 con 0x7fbd9409d630 2023-10-03T10:18:12.517 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:12.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.520+0000 7fbd9a821700 1 -- 172.21.15.3:0/459568597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd9409d630 msgr2=0x7fbd94121100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:12.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.520+0000 7fbd9a821700 1 --2- 172.21.15.3:0/459568597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd9409d630 0x7fbd94121100 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400ebe0 tx=0x7fbd8400eef0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.520+0000 7fbd9a821700 1 -- 172.21.15.3:0/459568597 shutdown_connections 2023-10-03T10:18:12.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.520+0000 7fbd9a821700 1 --2- 172.21.15.3:0/459568597 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd94133930 0x7fbd94121b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.520+0000 7fbd9a821700 1 --2- 172.21.15.3:0/459568597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd9409d630 0x7fbd94121100 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.520+0000 7fbd9a821700 1 --2- 172.21.15.3:0/459568597 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd940a0a30 0x7fbd94121640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.520+0000 7fbd9a821700 1 -- 172.21.15.3:0/459568597 >> 172.21.15.3:0/459568597 conn(0x7fbd940a3050 msgr2=0x7fbd94133210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:12.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.520+0000 7fbd9a821700 1 -- 172.21.15.3:0/459568597 shutdown_connections 2023-10-03T10:18:12.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.520+0000 7fbd9a821700 1 -- 172.21.15.3:0/459568597 wait complete. 2023-10-03T10:18:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.535+0000 7f7d802e5700 1 Processor -- start 2023-10-03T10:18:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.535+0000 7f7d802e5700 1 -- start start 2023-10-03T10:18:12.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.535+0000 7f7d802e5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d780a0a30 0x7f7d780a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:12.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.535+0000 7f7d802e5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7d780a1340 0x7f7d78133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:12.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.535+0000 7f7d802e5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d7809edd0 con 0x7f7d78133fa0 2023-10-03T10:18:12.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.535+0000 7f7d802e5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d7809e1f0 con 0x7f7d780a1340 2023-10-03T10:18:12.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.535+0000 7f7d802e5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7809dc50 con 0x7f7d780a0a30 2023-10-03T10:18:12.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.536+0000 7f7d7e081700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d780a0a30 0x7f7d780a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:12.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.536+0000 7f7d7e081700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d780a0a30 0x7f7d780a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42242/0 (socket says 172.21.15.3:42242) 2023-10-03T10:18:12.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.536+0000 7f7d7e081700 1 -- 172.21.15.3:0/1965109957 learned_addr learned my addr 172.21.15.3:0/1965109957 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:12.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.536+0000 7f7d7e081700 1 -- 172.21.15.3:0/1965109957 >> v2:172.21.15.3:3300/0 conn(0x7f7d780a1340 msgr2=0x7f7d78133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:12.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.536+0000 7f7d7e081700 1 --2- 172.21.15.3:0/1965109957 >> v2:172.21.15.3:3300/0 conn(0x7f7d780a1340 0x7f7d78133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.536+0000 7f7d7e081700 1 -- 172.21.15.3:0/1965109957 >> v1:172.21.15.3:6789/0 conn(0x7f7d78133fa0 legacy=0x7f7d7813ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:12.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.536+0000 7f7d7e081700 1 -- 172.21.15.3:0/1965109957 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d780a1fe0 con 0x7f7d780a0a30 2023-10-03T10:18:12.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.536+0000 7f7d7e081700 1 --2- 172.21.15.3:0/1965109957 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d780a0a30 0x7f7d780a0e00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f7d74009e50 tx=0x7f7d7400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b013cdd988128f2c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:12.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.536+0000 7f7d7d07f700 1 -- 172.21.15.3:0/1965109957 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7400e070 con 0x7f7d780a0a30 2023-10-03T10:18:12.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.536+0000 7f7d7d07f700 1 -- 172.21.15.3:0/1965109957 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d74003a20 con 0x7f7d780a0a30 2023-10-03T10:18:12.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.536+0000 7f7d7d07f700 1 -- 172.21.15.3:0/1965109957 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d74003dd0 con 0x7f7d780a0a30 2023-10-03T10:18:12.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.537+0000 7f7d802e5700 1 -- 172.21.15.3:0/1965109957 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d780a0a30 msgr2=0x7f7d780a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:12.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.537+0000 7f7d802e5700 1 --2- 172.21.15.3:0/1965109957 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d780a0a30 0x7f7d780a0e00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f7d74009e50 tx=0x7f7d7400b330 comp rx=0 tx=0).stop 2023-10-03T10:18:12.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.537+0000 7f7d802e5700 1 -- 172.21.15.3:0/1965109957 shutdown_connections 2023-10-03T10:18:12.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.537+0000 7f7d802e5700 1 --2- 172.21.15.3:0/1965109957 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d780a0a30 0x7f7d780a0e00 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.537+0000 7f7d802e5700 1 --2- 172.21.15.3:0/1965109957 >> v2:172.21.15.3:3300/0 conn(0x7f7d780a1340 0x7f7d78133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.537+0000 7f7d802e5700 1 -- 172.21.15.3:0/1965109957 >> 172.21.15.3:0/1965109957 conn(0x7f7d780a6550 msgr2=0x7f7d780a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:12.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.537+0000 7f7d802e5700 1 -- 172.21.15.3:0/1965109957 shutdown_connections 2023-10-03T10:18:12.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.537+0000 7f7d802e5700 1 -- 172.21.15.3:0/1965109957 wait complete. 2023-10-03T10:18:12.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.538+0000 7f7d802e5700 1 Processor -- start 2023-10-03T10:18:12.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.538+0000 7f7d802e5700 1 -- start start 2023-10-03T10:18:12.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.539+0000 7f7d802e5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d780a0a30 0x7f7d781c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:12.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.539+0000 7f7d802e5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d780a1340 0x7f7d781c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:12.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.539+0000 7f7d802e5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d78133fa0 0x7f7d781c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:12.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.539+0000 7f7d802e5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d78121f80 con 0x7f7d780a1340 2023-10-03T10:18:12.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.539+0000 7f7d802e5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d78120800 con 0x7f7d78133fa0 2023-10-03T10:18:12.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.539+0000 7f7d802e5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d780a0220 con 0x7f7d780a0a30 2023-10-03T10:18:12.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.539+0000 7f7d7e081700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d780a0a30 0x7f7d781c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:12.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.539+0000 7f7d7e081700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d780a0a30 0x7f7d781c8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47992/0 (socket says 172.21.15.3:47992) 2023-10-03T10:18:12.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.539+0000 7f7d7e081700 1 -- 172.21.15.3:0/3594932637 learned_addr learned my addr 172.21.15.3:0/3594932637 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:12.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.539+0000 7f7d7e882700 1 --2- 172.21.15.3:0/3594932637 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d78133fa0 0x7f7d781c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:12.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.540+0000 7f7d7e081700 1 -- 172.21.15.3:0/3594932637 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d78133fa0 msgr2=0x7f7d781c92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:12.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.540+0000 7f7d7e081700 1 --2- 172.21.15.3:0/3594932637 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d78133fa0 0x7f7d781c92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.540+0000 7f7d7e081700 1 -- 172.21.15.3:0/3594932637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d780a1340 msgr2=0x7f7d781c8d60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:18:12.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.540+0000 7f7d7e081700 1 --2- 172.21.15.3:0/3594932637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d780a1340 0x7f7d781c8d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.540+0000 7f7d7e081700 1 -- 172.21.15.3:0/3594932637 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d74009b30 con 0x7f7d780a0a30 2023-10-03T10:18:12.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.540+0000 7f7d7e882700 1 --2- 172.21.15.3:0/3594932637 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d78133fa0 0x7f7d781c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:12.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.540+0000 7f7d7e081700 1 --2- 172.21.15.3:0/3594932637 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d780a0a30 0x7f7d781c8820 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7d7809dd90 tx=0x7f7d74004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:12.542 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:12.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.540+0000 7f7d6effd700 1 -- 172.21.15.3:0/3594932637 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d74004440 con 0x7f7d780a0a30 2023-10-03T10:18:12.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.540+0000 7f7d802e5700 1 -- 172.21.15.3:0/3594932637 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d78121810 con 0x7f7d780a0a30 2023-10-03T10:18:12.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.540+0000 7f7d802e5700 1 -- 172.21.15.3:0/3594932637 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d781336a0 con 0x7f7d780a0a30 2023-10-03T10:18:12.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.541+0000 7f7d6effd700 1 -- 172.21.15.3:0/3594932637 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d740092e0 con 0x7f7d780a0a30 2023-10-03T10:18:12.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.541+0000 7f7d6effd700 1 -- 172.21.15.3:0/3594932637 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d740047a0 con 0x7f7d780a0a30 2023-10-03T10:18:12.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.541+0000 7f7d6effd700 1 -- 172.21.15.3:0/3594932637 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7d74019030 con 0x7f7d780a0a30 2023-10-03T10:18:12.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.541+0000 7f7d802e5700 1 -- 172.21.15.3:0/3594932637 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d600052e0 con 0x7f7d780a0a30 2023-10-03T10:18:12.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.542+0000 7f7d6effd700 1 -- 172.21.15.3:0/3594932637 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7d740406b0 con 0x7f7d780a0a30 2023-10-03T10:18:12.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.547+0000 7f7d6effd700 1 -- 172.21.15.3:0/3594932637 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7d74017020 con 0x7f7d780a0a30 2023-10-03T10:18:12.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.623+0000 7fa9b9568700 1 -- 172.21.15.3:0/68345275 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa9b41a81e0 con 0x7fa9b409e000 2023-10-03T10:18:12.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.623+0000 7fa99b7fe700 1 -- 172.21.15.3:0/68345275 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa9b41a81e0 con 0x7fa9b409e000 2023-10-03T10:18:12.623 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:12.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.625+0000 7fa9b9568700 1 -- 172.21.15.3:0/68345275 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9b409e000 msgr2=0x7fa9b41a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:12.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.625+0000 7fa9b9568700 1 --2- 172.21.15.3:0/68345275 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9b409e000 0x7fa9b41a5670 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800e9d0 tx=0x7fa9a800ece0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.625+0000 7fa9b9568700 1 -- 172.21.15.3:0/68345275 shutdown_connections 2023-10-03T10:18:12.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.625+0000 7fa9b9568700 1 --2- 172.21.15.3:0/68345275 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa9b409e410 0x7fa9b41c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.625+0000 7fa9b9568700 1 --2- 172.21.15.3:0/68345275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b409d6f0 0x7fa9b41a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.625+0000 7fa9b9568700 1 --2- 172.21.15.3:0/68345275 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa9b409e000 0x7fa9b41a5670 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.625+0000 7fa9b9568700 1 -- 172.21.15.3:0/68345275 >> 172.21.15.3:0/68345275 conn(0x7fa9b4066f50 msgr2=0x7fa9b41bbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:12.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.625+0000 7fa9b9568700 1 -- 172.21.15.3:0/68345275 shutdown_connections 2023-10-03T10:18:12.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.625+0000 7fa9b9568700 1 -- 172.21.15.3:0/68345275 wait complete. 2023-10-03T10:18:12.644 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:12.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.740+0000 7fdeb4549700 1 -- 172.21.15.3:0/2201590705 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fdeac1a7000 con 0x7fdeac09e410 2023-10-03T10:18:12.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.740+0000 7fdea2ffd700 1 -- 172.21.15.3:0/2201590705 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdea402a030 con 0x7fdeac09e410 2023-10-03T10:18:12.741 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:12.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.742+0000 7fdeb4549700 1 -- 172.21.15.3:0/2201590705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdeac09e410 msgr2=0x7fdeac1c7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:12.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.742+0000 7fdeb4549700 1 --2- 172.21.15.3:0/2201590705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdeac09e410 0x7fdeac1c7f40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fdea4015000 tx=0x7fdea4004ad0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.743+0000 7fdeb4549700 1 -- 172.21.15.3:0/2201590705 shutdown_connections 2023-10-03T10:18:12.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.743+0000 7fdeb4549700 1 --2- 172.21.15.3:0/2201590705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdeac09e410 0x7fdeac1c7f40 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.743+0000 7fdeb4549700 1 --2- 172.21.15.3:0/2201590705 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdeac09d6f0 0x7fdeac1a50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.743+0000 7fdeb4549700 1 --2- 172.21.15.3:0/2201590705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdeac09e000 0x7fdeac1a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.743+0000 7fdeb4549700 1 -- 172.21.15.3:0/2201590705 >> 172.21.15.3:0/2201590705 conn(0x7fdeac066f50 msgr2=0x7fdeac06f270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:12.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.743+0000 7fdeb4549700 1 -- 172.21.15.3:0/2201590705 shutdown_connections 2023-10-03T10:18:12.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.743+0000 7fdeb4549700 1 -- 172.21.15.3:0/2201590705 wait complete. 2023-10-03T10:18:12.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.753+0000 7f7d802e5700 1 -- 172.21.15.3:0/3594932637 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7d60005590 con 0x7f7d780a0a30 2023-10-03T10:18:12.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.753+0000 7f7d6effd700 1 -- 172.21.15.3:0/3594932637 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7d74004900 con 0x7f7d780a0a30 2023-10-03T10:18:12.753 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:12.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.756+0000 7f7d802e5700 1 -- 172.21.15.3:0/3594932637 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d780a0a30 msgr2=0x7f7d781c8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:12.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.756+0000 7f7d802e5700 1 --2- 172.21.15.3:0/3594932637 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d780a0a30 0x7f7d781c8820 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7d7809dd90 tx=0x7f7d74004ad0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.756+0000 7f7d802e5700 1 -- 172.21.15.3:0/3594932637 shutdown_connections 2023-10-03T10:18:12.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.756+0000 7f7d802e5700 1 --2- 172.21.15.3:0/3594932637 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d78133fa0 0x7f7d781c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.756+0000 7f7d802e5700 1 --2- 172.21.15.3:0/3594932637 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d780a0a30 0x7f7d781c8820 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.756+0000 7f7d802e5700 1 --2- 172.21.15.3:0/3594932637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d780a1340 0x7f7d781c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:12.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.756+0000 7f7d802e5700 1 -- 172.21.15.3:0/3594932637 >> 172.21.15.3:0/3594932637 conn(0x7f7d780a6550 msgr2=0x7f7d781b9130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:12.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.756+0000 7f7d802e5700 1 -- 172.21.15.3:0/3594932637 shutdown_connections 2023-10-03T10:18:12.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:12.756+0000 7f7d802e5700 1 -- 172.21.15.3:0/3594932637 wait complete. 2023-10-03T10:18:12.757 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:12.769 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:13.543 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:13.645 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:13.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.690+0000 7f0607876700 1 Processor -- start 2023-10-03T10:18:13.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.690+0000 7f0607876700 1 -- start start 2023-10-03T10:18:13.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.690+0000 7f0607876700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06000a0a30 0x7f06000a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:13.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.690+0000 7f0607876700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0600134170 0x7f060013bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:13.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.691+0000 7f0607876700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f060009edd0 con 0x7f06000a1340 2023-10-03T10:18:13.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.691+0000 7f0607876700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f060009e1f0 con 0x7f0600134170 2023-10-03T10:18:13.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.691+0000 7f0607876700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060009dc50 con 0x7f06000a0a30 2023-10-03T10:18:13.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.691+0000 7f0605612700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06000a0a30 0x7f06000a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:13.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.691+0000 7f0605612700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06000a0a30 0x7f06000a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59626/0 (socket says 172.21.15.3:59626) 2023-10-03T10:18:13.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.691+0000 7f0605612700 1 -- 172.21.15.3:0/3710610610 learned_addr learned my addr 172.21.15.3:0/3710610610 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:13.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.691+0000 7f0605e13700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0600134170 0x7f060013bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:13.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.691+0000 7f0605612700 1 -- 172.21.15.3:0/3710610610 >> v2:172.21.15.3:3300/0 conn(0x7f0600134170 msgr2=0x7f060013bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:13.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.691+0000 7f0605612700 1 --2- 172.21.15.3:0/3710610610 >> v2:172.21.15.3:3300/0 conn(0x7f0600134170 0x7f060013bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:13.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.691+0000 7f0605612700 1 -- 172.21.15.3:0/3710610610 >> v1:172.21.15.3:6789/0 conn(0x7f06000a1340 legacy=0x7f0600133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:13.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.691+0000 7f0605612700 1 -- 172.21.15.3:0/3710610610 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06000a1fe0 con 0x7f06000a0a30 2023-10-03T10:18:13.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.691+0000 7f0605e13700 1 --2- 172.21.15.3:0/3710610610 >> v2:172.21.15.3:3300/0 conn(0x7f0600134170 0x7f060013bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:13.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.691+0000 7f0605612700 1 --2- 172.21.15.3:0/3710610610 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06000a0a30 0x7f06000a0e00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0098f0 tx=0x7f05fc009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de611babf93df51 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:13.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.691+0000 7f05f7fff700 1 -- 172.21.15.3:0/3710610610 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc011830 con 0x7f06000a0a30 2023-10-03T10:18:13.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.691+0000 7f05f7fff700 1 -- 172.21.15.3:0/3710610610 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05fc011990 con 0x7f06000a0a30 2023-10-03T10:18:13.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.691+0000 7f05f7fff700 1 -- 172.21.15.3:0/3710610610 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc011d20 con 0x7f06000a0a30 2023-10-03T10:18:13.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.692+0000 7f0607876700 1 -- 172.21.15.3:0/3710610610 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06000a0a30 msgr2=0x7f06000a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:13.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.692+0000 7f0607876700 1 --2- 172.21.15.3:0/3710610610 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06000a0a30 0x7f06000a0e00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0098f0 tx=0x7f05fc009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:13.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.692+0000 7f0607876700 1 -- 172.21.15.3:0/3710610610 shutdown_connections 2023-10-03T10:18:13.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.692+0000 7f0607876700 1 --2- 172.21.15.3:0/3710610610 >> v2:172.21.15.3:3300/0 conn(0x7f0600134170 0x7f060013bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:13.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.692+0000 7f0607876700 1 --2- 172.21.15.3:0/3710610610 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f06000a0a30 0x7f06000a0e00 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:13.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.692+0000 7f0607876700 1 -- 172.21.15.3:0/3710610610 >> 172.21.15.3:0/3710610610 conn(0x7f06000a6550 msgr2=0x7f06000a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:13.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.692+0000 7f0607876700 1 -- 172.21.15.3:0/3710610610 shutdown_connections 2023-10-03T10:18:13.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.692+0000 7f0607876700 1 -- 172.21.15.3:0/3710610610 wait complete. 2023-10-03T10:18:13.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.693+0000 7f0607876700 1 Processor -- start 2023-10-03T10:18:13.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.693+0000 7f0607876700 1 -- start start 2023-10-03T10:18:13.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.694+0000 7f0607876700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f06000a0a30 0x7f0600122900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:13.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.694+0000 7f0607876700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06000a1340 0x7f0600122e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:13.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.694+0000 7f0607876700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0600134170 0x7f0600123380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:13.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.694+0000 7f0607876700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0600124690 con 0x7f06000a1340 2023-10-03T10:18:13.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.694+0000 7f0607876700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06000a4f40 con 0x7f0600134170 2023-10-03T10:18:13.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.694+0000 7f0607876700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0600126420 con 0x7f06000a0a30 2023-10-03T10:18:13.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.694+0000 7f0605612700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f06000a0a30 0x7f0600122900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:13.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.694+0000 7f0604e11700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06000a1340 0x7f0600122e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:13.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.694+0000 7f0605612700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f06000a0a30 0x7f0600122900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43342/0 (socket says 172.21.15.3:43342) 2023-10-03T10:18:13.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.694+0000 7f0605612700 1 -- 172.21.15.3:0/901242260 learned_addr learned my addr 172.21.15.3:0/901242260 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:13.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.694+0000 7f0605e13700 1 --2- 172.21.15.3:0/901242260 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0600134170 0x7f0600123380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:13.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.694+0000 7f0604e11700 1 -- 172.21.15.3:0/901242260 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f06000a0a30 msgr2=0x7f0600122900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:13.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.694+0000 7f0604e11700 1 --2- 172.21.15.3:0/901242260 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f06000a0a30 0x7f0600122900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:13.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.694+0000 7f0604e11700 1 -- 172.21.15.3:0/901242260 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0600134170 msgr2=0x7f0600123380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:13.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.694+0000 7f0604e11700 1 --2- 172.21.15.3:0/901242260 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0600134170 0x7f0600123380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:13.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.694+0000 7f0604e11700 1 -- 172.21.15.3:0/901242260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f8009a70 con 0x7f06000a1340 2023-10-03T10:18:13.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.694+0000 7f0604e11700 1 --2- 172.21.15.3:0/901242260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06000a1340 0x7f0600122e40 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f05f0000f60 tx=0x7f05f000c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:13.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.695+0000 7f05f5ffb700 1 -- 172.21.15.3:0/901242260 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f00052d0 con 0x7f06000a1340 2023-10-03T10:18:13.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.695+0000 7f05f5ffb700 1 -- 172.21.15.3:0/901242260 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f0005430 con 0x7f06000a1340 2023-10-03T10:18:13.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.695+0000 7f0607876700 1 -- 172.21.15.3:0/901242260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05fc009550 con 0x7f06000a1340 2023-10-03T10:18:13.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.695+0000 7f05f5ffb700 1 -- 172.21.15.3:0/901242260 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f00038c0 con 0x7f06000a1340 2023-10-03T10:18:13.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.695+0000 7f0607876700 1 -- 172.21.15.3:0/901242260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0600133420 con 0x7f06000a1340 2023-10-03T10:18:13.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.696+0000 7f05f5ffb700 1 -- 172.21.15.3:0/901242260 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f05f0003a20 con 0x7f06000a1340 2023-10-03T10:18:13.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.696+0000 7f05f5ffb700 1 -- 172.21.15.3:0/901242260 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f05f003e990 con 0x7f06000a1340 2023-10-03T10:18:13.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.696+0000 7f0607876700 1 -- 172.21.15.3:0/901242260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06000a1d40 con 0x7f06000a1340 2023-10-03T10:18:13.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.702+0000 7f05f5ffb700 1 -- 172.21.15.3:0/901242260 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f05f0017080 con 0x7f06000a1340 2023-10-03T10:18:13.758 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:13.770 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:13.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.786+0000 7f7ff8bfe700 1 Processor -- start 2023-10-03T10:18:13.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.786+0000 7f7ff8bfe700 1 -- start start 2023-10-03T10:18:13.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.787+0000 7f7ff8bfe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ff41338c0 0x7f7ff4133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:13.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.787+0000 7f7ff8bfe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7ff41341d0 0x7f7ff409de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:13.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.787+0000 7f7ff8bfe700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ff406f720 con 0x7f7ff409e390 2023-10-03T10:18:13.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.787+0000 7f7ff8bfe700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff4070c80 con 0x7f7ff41341d0 2023-10-03T10:18:13.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.787+0000 7f7ff8bfe700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff406fce0 con 0x7f7ff41338c0 2023-10-03T10:18:13.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.787+0000 7f7ff2d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7ff409e390 0x7f7ff413a850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53998/0 (socket says 172.21.15.3:53998) 2023-10-03T10:18:13.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.787+0000 7f7ff2d9d700 1 -- 172.21.15.3:0/1269318587 learned_addr learned my addr 172.21.15.3:0/1269318587 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:13.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.788+0000 7f7ff259c700 1 --2- 172.21.15.3:0/1269318587 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ff41338c0 0x7f7ff4133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:13.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.788+0000 7f7ff1d9b700 1 --2- 172.21.15.3:0/1269318587 >> v2:172.21.15.3:3300/0 conn(0x7f7ff41341d0 0x7f7ff409de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:13.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.788+0000 7f7ff1d9b700 1 -- 172.21.15.3:0/1269318587 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ff41338c0 msgr2=0x7f7ff4133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:13.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.788+0000 7f7ff1d9b700 1 --2- 172.21.15.3:0/1269318587 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ff41338c0 0x7f7ff4133c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:13.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.788+0000 7f7ff1d9b700 1 -- 172.21.15.3:0/1269318587 >> v1:172.21.15.3:6789/0 conn(0x7f7ff409e390 legacy=0x7f7ff413a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:13.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.788+0000 7f7ff1d9b700 1 -- 172.21.15.3:0/1269318587 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff41336a0 con 0x7f7ff41341d0 2023-10-03T10:18:13.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.788+0000 7f7ff1d9b700 1 --2- 172.21.15.3:0/1269318587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ff41341d0 0x7f7ff409de50 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8009840 tx=0x7f7fe8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2d625ec8757c74 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:13.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.788+0000 7f7ff159a700 1 -- 172.21.15.3:0/1269318587 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8011830 con 0x7f7ff41341d0 2023-10-03T10:18:13.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.788+0000 7f7ff159a700 1 -- 172.21.15.3:0/1269318587 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe8011990 con 0x7f7ff41341d0 2023-10-03T10:18:13.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.788+0000 7f7ff159a700 1 -- 172.21.15.3:0/1269318587 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8011d20 con 0x7f7ff41341d0 2023-10-03T10:18:13.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.789+0000 7f7ff8bfe700 1 -- 172.21.15.3:0/1269318587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ff41341d0 msgr2=0x7f7ff409de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:13.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.789+0000 7f7ff8bfe700 1 --2- 172.21.15.3:0/1269318587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ff41341d0 0x7f7ff409de50 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8009840 tx=0x7f7fe8009b50 comp rx=0 tx=0).stop 2023-10-03T10:18:13.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.789+0000 7f7ff8bfe700 1 -- 172.21.15.3:0/1269318587 shutdown_connections 2023-10-03T10:18:13.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.789+0000 7f7ff8bfe700 1 --2- 172.21.15.3:0/1269318587 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ff41338c0 0x7f7ff4133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:13.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.789+0000 7f7ff8bfe700 1 --2- 172.21.15.3:0/1269318587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ff41341d0 0x7f7ff409de50 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:13.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.789+0000 7f7ff8bfe700 1 -- 172.21.15.3:0/1269318587 >> 172.21.15.3:0/1269318587 conn(0x7f7ff40a2aa0 msgr2=0x7f7ff40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:13.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.790+0000 7f7ff8bfe700 1 -- 172.21.15.3:0/1269318587 shutdown_connections 2023-10-03T10:18:13.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.791+0000 7f7ff8bfe700 1 -- 172.21.15.3:0/1269318587 wait complete. 2023-10-03T10:18:13.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.792+0000 7f7ff8bfe700 1 Processor -- start 2023-10-03T10:18:13.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.792+0000 7f7ff8bfe700 1 -- start start 2023-10-03T10:18:13.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.792+0000 7f7ff8bfe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ff409e390 0x7f7ff411e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:13.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.792+0000 7f7ff8bfe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ff41338c0 0x7f7ff411edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:13.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.792+0000 7f7ff8bfe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ff41341d0 0x7f7ff411f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:13.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.792+0000 7f7ff8bfe700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff41223a0 con 0x7f7ff409e390 2023-10-03T10:18:13.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.792+0000 7f7ff8bfe700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff4121e00 con 0x7f7ff41338c0 2023-10-03T10:18:13.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.793+0000 7f7ff8bfe700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff412fd10 con 0x7f7ff41341d0 2023-10-03T10:18:13.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.793+0000 7f7ff2d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ff41341d0 0x7f7ff411f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:13.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.793+0000 7f7ff2d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ff41341d0 0x7f7ff411f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43354/0 (socket says 172.21.15.3:43354) 2023-10-03T10:18:13.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.793+0000 7f7ff2d9d700 1 -- 172.21.15.3:0/858162 learned_addr learned my addr 172.21.15.3:0/858162 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:13.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.793+0000 7f7ff259c700 1 --2- 172.21.15.3:0/858162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ff409e390 0x7f7ff411e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:13.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.793+0000 7f7ff259c700 1 -- 172.21.15.3:0/858162 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ff41341d0 msgr2=0x7f7ff411f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:13.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.793+0000 7f7ff259c700 1 --2- 172.21.15.3:0/858162 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ff41341d0 0x7f7ff411f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:13.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.793+0000 7f7ff259c700 1 -- 172.21.15.3:0/858162 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ff41338c0 msgr2=0x7f7ff411edc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:18:13.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.793+0000 7f7ff259c700 1 --2- 172.21.15.3:0/858162 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ff41338c0 0x7f7ff411edc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:13.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.793+0000 7f7ff259c700 1 -- 172.21.15.3:0/858162 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe80094a0 con 0x7f7ff409e390 2023-10-03T10:18:13.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.793+0000 7f7ff2d9d700 1 --2- 172.21.15.3:0/858162 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ff41341d0 0x7f7ff411f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:13.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.793+0000 7f7ff259c700 1 --2- 172.21.15.3:0/858162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ff409e390 0x7f7ff411e880 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400e900 tx=0x7f7fe400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:13.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.794+0000 7f7fe2ffd700 1 -- 172.21.15.3:0/858162 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe400ee50 con 0x7f7ff409e390 2023-10-03T10:18:13.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.794+0000 7f7fe2ffd700 1 -- 172.21.15.3:0/858162 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe400c850 con 0x7f7ff409e390 2023-10-03T10:18:13.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.794+0000 7f7fe2ffd700 1 -- 172.21.15.3:0/858162 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe400cbc0 con 0x7f7ff409e390 2023-10-03T10:18:13.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.794+0000 7f7ff8bfe700 1 -- 172.21.15.3:0/858162 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff40a1360 con 0x7f7ff409e390 2023-10-03T10:18:13.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.794+0000 7f7ff8bfe700 1 -- 172.21.15.3:0/858162 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff4122b30 con 0x7f7ff409e390 2023-10-03T10:18:13.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.795+0000 7f7fe2ffd700 1 -- 172.21.15.3:0/858162 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7fe400cd20 con 0x7f7ff409e390 2023-10-03T10:18:13.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.795+0000 7f7fe2ffd700 1 -- 172.21.15.3:0/858162 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7fe403fab0 con 0x7f7ff409e390 2023-10-03T10:18:13.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.796+0000 7f7ff8bfe700 1 -- 172.21.15.3:0/858162 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff4074ea0 con 0x7f7ff409e390 2023-10-03T10:18:13.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.801+0000 7f7fe2ffd700 1 -- 172.21.15.3:0/858162 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7fe401adc0 con 0x7f7ff409e390 2023-10-03T10:18:13.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.926+0000 7fab2fa34700 1 Processor -- start 2023-10-03T10:18:13.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.926+0000 7fab2fa34700 1 -- start start 2023-10-03T10:18:13.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.927+0000 7fab2fa34700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab28134630 0x7fab28136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:13.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.927+0000 7fab2fa34700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fab28136f50 0x7fab28139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:13.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.927+0000 7fab2fa34700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab2809b4f0 con 0x7fab28131810 2023-10-03T10:18:13.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.927+0000 7fab2fa34700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab281313c0 con 0x7fab28136f50 2023-10-03T10:18:13.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.927+0000 7fab2fa34700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fab2806e8c0 con 0x7fab28134630 2023-10-03T10:18:13.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.927+0000 7fab2d7d0700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fab28131810 0x7fab28131be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54008/0 (socket says 172.21.15.3:54008) 2023-10-03T10:18:13.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.927+0000 7fab2d7d0700 1 -- 172.21.15.3:0/3547151461 learned_addr learned my addr 172.21.15.3:0/3547151461 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:13.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.927+0000 7fab2cfcf700 1 --2- 172.21.15.3:0/3547151461 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab28134630 0x7fab28136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:13.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.927+0000 7fab2dfd1700 1 --2- 172.21.15.3:0/3547151461 >> v2:172.21.15.3:3300/0 conn(0x7fab28136f50 0x7fab28139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:13.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.928+0000 7fab2cfcf700 1 -- 172.21.15.3:0/3547151461 >> v2:172.21.15.3:3300/0 conn(0x7fab28136f50 msgr2=0x7fab28139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:13.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.928+0000 7fab2cfcf700 1 --2- 172.21.15.3:0/3547151461 >> v2:172.21.15.3:3300/0 conn(0x7fab28136f50 0x7fab28139330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:13.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.928+0000 7fab2cfcf700 1 -- 172.21.15.3:0/3547151461 >> v1:172.21.15.3:6789/0 conn(0x7fab28131810 legacy=0x7fab28131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:13.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.928+0000 7fab2cfcf700 1 -- 172.21.15.3:0/3547151461 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab28071d20 con 0x7fab28134630 2023-10-03T10:18:13.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.928+0000 7fab1ffff700 1 -- 172.21.15.3:0/3547151461 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048578307 0 0) 0x7fab2809b4f0 con 0x7fab28131810 2023-10-03T10:18:13.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.928+0000 7fab2cfcf700 1 --2- 172.21.15.3:0/3547151461 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab28134630 0x7fab28136a10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fab18009840 tx=0x7fab18009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a360a8a830416d56 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:13.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.928+0000 7fab1ffff700 1 -- 172.21.15.3:0/3547151461 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab18011830 con 0x7fab28134630 2023-10-03T10:18:13.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.928+0000 7fab1ffff700 1 -- 172.21.15.3:0/3547151461 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab18011990 con 0x7fab28134630 2023-10-03T10:18:13.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.928+0000 7fab1ffff700 1 -- 172.21.15.3:0/3547151461 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab18011d20 con 0x7fab28134630 2023-10-03T10:18:13.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.929+0000 7fab2fa34700 1 -- 172.21.15.3:0/3547151461 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab28134630 msgr2=0x7fab28136a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:13.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.929+0000 7fab2fa34700 1 --2- 172.21.15.3:0/3547151461 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab28134630 0x7fab28136a10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fab18009840 tx=0x7fab18009b50 comp rx=0 tx=0).stop 2023-10-03T10:18:13.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.929+0000 7fab2fa34700 1 -- 172.21.15.3:0/3547151461 shutdown_connections 2023-10-03T10:18:13.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.929+0000 7fab2fa34700 1 --2- 172.21.15.3:0/3547151461 >> v2:172.21.15.3:3300/0 conn(0x7fab28136f50 0x7fab28139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:13.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.929+0000 7fab2fa34700 1 --2- 172.21.15.3:0/3547151461 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab28134630 0x7fab28136a10 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:13.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.929+0000 7fab2fa34700 1 -- 172.21.15.3:0/3547151461 >> 172.21.15.3:0/3547151461 conn(0x7fab280a2aa0 msgr2=0x7fab280a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:13.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.929+0000 7fab2fa34700 1 -- 172.21.15.3:0/3547151461 shutdown_connections 2023-10-03T10:18:13.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.929+0000 7fab2fa34700 1 -- 172.21.15.3:0/3547151461 wait complete. 2023-10-03T10:18:13.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.930+0000 7fab2fa34700 1 Processor -- start 2023-10-03T10:18:13.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.931+0000 7fab2fa34700 1 -- start start 2023-10-03T10:18:13.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.931+0000 7fab2fa34700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab28131810 0x7fab2809c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:13.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.931+0000 7fab2fa34700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab28134630 0x7fab2809ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:13.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.931+0000 7fab2fa34700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab28136f50 0x7fab2809cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:13.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.931+0000 7fab2fa34700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2809e0d0 con 0x7fab28131810 2023-10-03T10:18:13.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.931+0000 7fab2fa34700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab28119110 con 0x7fab28136f50 2023-10-03T10:18:13.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.931+0000 7fab2fa34700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fab281198c0 con 0x7fab28134630 2023-10-03T10:18:13.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.931+0000 7fab2d7d0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab28131810 0x7fab2809c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:13.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.931+0000 7fab2dfd1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab28136f50 0x7fab2809cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:13.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.932+0000 7fab2d7d0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab28131810 0x7fab2809c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59246/0 (socket says 172.21.15.3:59246) 2023-10-03T10:18:13.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.932+0000 7fab2dfd1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab28136f50 0x7fab2809cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59642/0 (socket says 172.21.15.3:59642) 2023-10-03T10:18:13.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.932+0000 7fab2d7d0700 1 -- 172.21.15.3:0/2640133604 learned_addr learned my addr 172.21.15.3:0/2640133604 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:13.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.932+0000 7fab2cfcf700 1 --2- 172.21.15.3:0/2640133604 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab28134630 0x7fab2809ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:13.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.932+0000 7fab2d7d0700 1 -- 172.21.15.3:0/2640133604 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab28134630 msgr2=0x7fab2809ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:13.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.932+0000 7fab2d7d0700 1 --2- 172.21.15.3:0/2640133604 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab28134630 0x7fab2809ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:13.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.932+0000 7fab2d7d0700 1 -- 172.21.15.3:0/2640133604 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab28136f50 msgr2=0x7fab2809cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:13.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.932+0000 7fab2d7d0700 1 --2- 172.21.15.3:0/2640133604 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab28136f50 0x7fab2809cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:13.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.932+0000 7fab2d7d0700 1 -- 172.21.15.3:0/2640133604 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab180094a0 con 0x7fab28131810 2023-10-03T10:18:13.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.932+0000 7fab2d7d0700 1 --2- 172.21.15.3:0/2640133604 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab28131810 0x7fab2809c4c0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fab24001b90 tx=0x7fab24014050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:13.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.932+0000 7fab1dffb700 1 -- 172.21.15.3:0/2640133604 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab24017070 con 0x7fab28131810 2023-10-03T10:18:13.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.932+0000 7fab1dffb700 1 -- 172.21.15.3:0/2640133604 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab24012320 con 0x7fab28131810 2023-10-03T10:18:13.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.932+0000 7fab2fa34700 1 -- 172.21.15.3:0/2640133604 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab2811a2a0 con 0x7fab28131810 2023-10-03T10:18:13.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.933+0000 7fab1dffb700 1 -- 172.21.15.3:0/2640133604 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab240126d0 con 0x7fab28131810 2023-10-03T10:18:13.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.933+0000 7fab2fa34700 1 -- 172.21.15.3:0/2640133604 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2806eb00 con 0x7fab28131810 2023-10-03T10:18:13.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.934+0000 7fab1dffb700 1 -- 172.21.15.3:0/2640133604 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fab24004870 con 0x7fab28131810 2023-10-03T10:18:13.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.934+0000 7fab2fa34700 1 -- 172.21.15.3:0/2640133604 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab2811add0 con 0x7fab28131810 2023-10-03T10:18:13.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.934+0000 7fab1dffb700 1 -- 172.21.15.3:0/2640133604 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fab24047170 con 0x7fab28131810 2023-10-03T10:18:13.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.939+0000 7fab1dffb700 1 -- 172.21.15.3:0/2640133604 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fab24004d00 con 0x7fab28131810 2023-10-03T10:18:13.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.953+0000 7ff64db5d700 1 Processor -- start 2023-10-03T10:18:13.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.954+0000 7ff64db5d700 1 -- start start 2023-10-03T10:18:13.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.954+0000 7ff64db5d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff64809e930 0x7ff64809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:13.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.954+0000 7ff64db5d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff64809f650 0x7ff648131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:13.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.954+0000 7ff64db5d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff648070940 con 0x7ff64809f240 2023-10-03T10:18:13.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.954+0000 7ff64db5d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff64806fd60 con 0x7ff64809e930 2023-10-03T10:18:13.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.954+0000 7ff64db5d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff64806fa80 con 0x7ff64809f650 2023-10-03T10:18:13.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.954+0000 7ff6477fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff64809e930 0x7ff64809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:13.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.954+0000 7ff6477fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff64809e930 0x7ff64809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59252/0 (socket says 172.21.15.3:59252) 2023-10-03T10:18:13.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.954+0000 7ff6477fe700 1 -- 172.21.15.3:0/1682322574 learned_addr learned my addr 172.21.15.3:0/1682322574 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:13.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.954+0000 7ff646ffd700 1 --1- 172.21.15.3:0/1682322574 >> v1:172.21.15.3:6789/0 conn(0x7ff64809f240 0x7ff6481317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54018/0 (socket says 172.21.15.3:54018) 2023-10-03T10:18:13.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.955+0000 7ff647fff700 1 --2- 172.21.15.3:0/1682322574 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff64809f650 0x7ff648131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:13.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.955+0000 7ff6477fe700 1 -- 172.21.15.3:0/1682322574 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff64809f650 msgr2=0x7ff648131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:13.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.955+0000 7ff6477fe700 1 --2- 172.21.15.3:0/1682322574 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff64809f650 0x7ff648131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:13.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.955+0000 7ff6477fe700 1 -- 172.21.15.3:0/1682322574 >> v1:172.21.15.3:6789/0 conn(0x7ff64809f240 legacy=0x7ff6481317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:13.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.955+0000 7ff6477fe700 1 -- 172.21.15.3:0/1682322574 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64809fb60 con 0x7ff64809e930 2023-10-03T10:18:13.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.955+0000 7ff6477fe700 1 --2- 172.21.15.3:0/1682322574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff64809e930 0x7ff64809ed00 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7ff6380098f0 tx=0x7ff638009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414b5e1c70f3d85 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:13.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.955+0000 7ff6467fc700 1 -- 172.21.15.3:0/1682322574 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63800b8a0 con 0x7ff64809e930 2023-10-03T10:18:13.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.955+0000 7ff6467fc700 1 -- 172.21.15.3:0/1682322574 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63800ba00 con 0x7ff64809e930 2023-10-03T10:18:13.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.955+0000 7ff6467fc700 1 -- 172.21.15.3:0/1682322574 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63800bd90 con 0x7ff64809e930 2023-10-03T10:18:13.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.956+0000 7ff64db5d700 1 -- 172.21.15.3:0/1682322574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff64809e930 msgr2=0x7ff64809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:13.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.956+0000 7ff64db5d700 1 --2- 172.21.15.3:0/1682322574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff64809e930 0x7ff64809ed00 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7ff6380098f0 tx=0x7ff638009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:13.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.956+0000 7ff64db5d700 1 -- 172.21.15.3:0/1682322574 shutdown_connections 2023-10-03T10:18:13.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.956+0000 7ff64db5d700 1 --2- 172.21.15.3:0/1682322574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff64809e930 0x7ff64809ed00 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:13.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.956+0000 7ff64db5d700 1 --2- 172.21.15.3:0/1682322574 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff64809f650 0x7ff648131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:13.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.956+0000 7ff64db5d700 1 -- 172.21.15.3:0/1682322574 >> 172.21.15.3:0/1682322574 conn(0x7ff6480a4210 msgr2=0x7ff6480a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:13.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.956+0000 7ff64db5d700 1 -- 172.21.15.3:0/1682322574 shutdown_connections 2023-10-03T10:18:13.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.956+0000 7ff64db5d700 1 -- 172.21.15.3:0/1682322574 wait complete. 2023-10-03T10:18:13.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.957+0000 7ff64db5d700 1 Processor -- start 2023-10-03T10:18:13.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.957+0000 7ff64db5d700 1 -- start start 2023-10-03T10:18:13.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.958+0000 7ff64db5d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff64809e930 0x7ff6481a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:13.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.958+0000 7ff64db5d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff64809f240 0x7ff6481a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:13.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.958+0000 7ff64db5d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff64809f650 0x7ff6481a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:13.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.958+0000 7ff64db5d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6481a5220 con 0x7ff64809f650 2023-10-03T10:18:13.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.958+0000 7ff64db5d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6481a5980 con 0x7ff64809f240 2023-10-03T10:18:13.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.958+0000 7ff64db5d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6481a6bd0 con 0x7ff64809e930 2023-10-03T10:18:13.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.958+0000 7ff6477fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff64809e930 0x7ff6481a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:13.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.958+0000 7ff6477fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff64809e930 0x7ff6481a3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43378/0 (socket says 172.21.15.3:43378) 2023-10-03T10:18:13.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.958+0000 7ff6477fe700 1 -- 172.21.15.3:0/4262267822 learned_addr learned my addr 172.21.15.3:0/4262267822 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:13.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.958+0000 7ff646ffd700 1 --2- 172.21.15.3:0/4262267822 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff64809f240 0x7ff6481a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:13.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.958+0000 7ff647fff700 1 --2- 172.21.15.3:0/4262267822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff64809f650 0x7ff6481a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:13.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.959+0000 7ff646ffd700 1 -- 172.21.15.3:0/4262267822 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff64809e930 msgr2=0x7ff6481a3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:13.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.959+0000 7ff646ffd700 1 --2- 172.21.15.3:0/4262267822 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff64809e930 0x7ff6481a3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:13.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.959+0000 7ff646ffd700 1 -- 172.21.15.3:0/4262267822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff64809f650 msgr2=0x7ff6481a47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:13.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.959+0000 7ff646ffd700 1 --2- 172.21.15.3:0/4262267822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff64809f650 0x7ff6481a47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:13.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.959+0000 7ff646ffd700 1 -- 172.21.15.3:0/4262267822 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff638009550 con 0x7ff64809f240 2023-10-03T10:18:13.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.959+0000 7ff6477fe700 1 --2- 172.21.15.3:0/4262267822 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff64809e930 0x7ff6481a3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:13.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.959+0000 7ff646ffd700 1 --2- 172.21.15.3:0/4262267822 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff64809f240 0x7ff6481a42b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff63c00c450 tx=0x7ff63c00c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:13.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.959+0000 7ff62ffff700 1 -- 172.21.15.3:0/4262267822 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c005cb0 con 0x7ff64809f240 2023-10-03T10:18:13.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.959+0000 7ff62ffff700 1 -- 172.21.15.3:0/4262267822 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63c005e10 con 0x7ff64809f240 2023-10-03T10:18:13.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.959+0000 7ff64db5d700 1 -- 172.21.15.3:0/4262267822 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6480a0190 con 0x7ff64809f240 2023-10-03T10:18:13.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.959+0000 7ff62ffff700 1 -- 172.21.15.3:0/4262267822 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c0072d0 con 0x7ff64809f240 2023-10-03T10:18:13.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.959+0000 7ff64db5d700 1 -- 172.21.15.3:0/4262267822 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6480a3600 con 0x7ff64809f240 2023-10-03T10:18:13.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.960+0000 7ff62ffff700 1 -- 172.21.15.3:0/4262267822 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff63c007430 con 0x7ff64809f240 2023-10-03T10:18:13.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.961+0000 7ff62ffff700 1 -- 172.21.15.3:0/4262267822 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff63c03eae0 con 0x7ff64809f240 2023-10-03T10:18:13.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.961+0000 7ff64db5d700 1 -- 172.21.15.3:0/4262267822 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff648071d20 con 0x7ff64809f240 2023-10-03T10:18:13.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.970+0000 7ff62ffff700 1 -- 172.21.15.3:0/4262267822 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff63c017080 con 0x7ff64809f240 2023-10-03T10:18:13.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.986+0000 7f0607876700 1 -- 172.21.15.3:0/901242260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0600074ea0 con 0x7f06000a1340 2023-10-03T10:18:13.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.986+0000 7f05f5ffb700 1 -- 172.21.15.3:0/901242260 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f05f0011590 con 0x7f06000a1340 2023-10-03T10:18:13.986 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:13.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.988+0000 7f0607876700 1 -- 172.21.15.3:0/901242260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06000a1340 msgr2=0x7f0600122e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:13.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.988+0000 7f0607876700 1 --2- 172.21.15.3:0/901242260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06000a1340 0x7f0600122e40 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f05f0000f60 tx=0x7f05f000c730 comp rx=0 tx=0).stop 2023-10-03T10:18:13.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.988+0000 7f0607876700 1 -- 172.21.15.3:0/901242260 shutdown_connections 2023-10-03T10:18:13.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.988+0000 7f0607876700 1 --2- 172.21.15.3:0/901242260 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0600134170 0x7f0600123380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:13.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.988+0000 7f0607876700 1 --2- 172.21.15.3:0/901242260 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f06000a0a30 0x7f0600122900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:13.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.988+0000 7f0607876700 1 --2- 172.21.15.3:0/901242260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06000a1340 0x7f0600122e40 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:13.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.989+0000 7f0607876700 1 -- 172.21.15.3:0/901242260 >> 172.21.15.3:0/901242260 conn(0x7f06000a6550 msgr2=0x7f060009ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:13.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.989+0000 7f0607876700 1 -- 172.21.15.3:0/901242260 shutdown_connections 2023-10-03T10:18:13.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:13.989+0000 7f0607876700 1 -- 172.21.15.3:0/901242260 wait complete. 2023-10-03T10:18:14.003 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:14.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.031+0000 7f7ff8bfe700 1 -- 172.21.15.3:0/858162 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7ff406b9e0 con 0x7f7ff409e390 2023-10-03T10:18:14.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.032+0000 7f7fe2ffd700 1 -- 172.21.15.3:0/858162 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7fe4021030 con 0x7f7ff409e390 2023-10-03T10:18:14.077 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:14.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.034+0000 7f7ff8bfe700 1 -- 172.21.15.3:0/858162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ff409e390 msgr2=0x7f7ff411e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:14.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.034+0000 7f7ff8bfe700 1 --2- 172.21.15.3:0/858162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ff409e390 0x7f7ff411e880 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400e900 tx=0x7f7fe400ec10 comp rx=0 tx=0).stop 2023-10-03T10:18:14.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.034+0000 7f7ff8bfe700 1 -- 172.21.15.3:0/858162 shutdown_connections 2023-10-03T10:18:14.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.034+0000 7f7ff8bfe700 1 --2- 172.21.15.3:0/858162 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7ff41341d0 0x7f7ff411f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:14.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.034+0000 7f7ff8bfe700 1 --2- 172.21.15.3:0/858162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ff409e390 0x7f7ff411e880 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:14.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.034+0000 7f7ff8bfe700 1 --2- 172.21.15.3:0/858162 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7ff41338c0 0x7f7ff411edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:14.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.034+0000 7f7ff8bfe700 1 -- 172.21.15.3:0/858162 >> 172.21.15.3:0/858162 conn(0x7f7ff40a2aa0 msgr2=0x7f7ff41306a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:14.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.034+0000 7f7ff8bfe700 1 -- 172.21.15.3:0/858162 shutdown_connections 2023-10-03T10:18:14.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.034+0000 7f7ff8bfe700 1 -- 172.21.15.3:0/858162 wait complete. 2023-10-03T10:18:14.079 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:14.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.158+0000 7fab2fa34700 1 -- 172.21.15.3:0/2640133604 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fab28074ea0 con 0x7fab28131810 2023-10-03T10:18:14.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.158+0000 7fab1dffb700 1 -- 172.21.15.3:0/2640133604 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fab2401cb30 con 0x7fab28131810 2023-10-03T10:18:14.158 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:14.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.160+0000 7fab2fa34700 1 -- 172.21.15.3:0/2640133604 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab28131810 msgr2=0x7fab2809c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:14.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.160+0000 7fab2fa34700 1 --2- 172.21.15.3:0/2640133604 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab28131810 0x7fab2809c4c0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fab24001b90 tx=0x7fab24014050 comp rx=0 tx=0).stop 2023-10-03T10:18:14.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.161+0000 7fab2fa34700 1 -- 172.21.15.3:0/2640133604 shutdown_connections 2023-10-03T10:18:14.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.161+0000 7fab2fa34700 1 --2- 172.21.15.3:0/2640133604 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fab28136f50 0x7fab2809cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:14.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.161+0000 7fab2fa34700 1 --2- 172.21.15.3:0/2640133604 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab28131810 0x7fab2809c4c0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:14.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.161+0000 7fab2fa34700 1 --2- 172.21.15.3:0/2640133604 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fab28134630 0x7fab2809ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:14.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.161+0000 7fab2fa34700 1 -- 172.21.15.3:0/2640133604 >> 172.21.15.3:0/2640133604 conn(0x7fab280a2aa0 msgr2=0x7fab280a0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:14.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.161+0000 7fab2fa34700 1 -- 172.21.15.3:0/2640133604 shutdown_connections 2023-10-03T10:18:14.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.161+0000 7fab2fa34700 1 -- 172.21.15.3:0/2640133604 wait complete. 2023-10-03T10:18:14.174 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:14.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.192+0000 7ff64db5d700 1 -- 172.21.15.3:0/4262267822 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff648074ea0 con 0x7ff64809f240 2023-10-03T10:18:14.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.192+0000 7ff62ffff700 1 -- 172.21.15.3:0/4262267822 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff63c0076e0 con 0x7ff64809f240 2023-10-03T10:18:14.193 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:14.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.194+0000 7ff64db5d700 1 -- 172.21.15.3:0/4262267822 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff64809f240 msgr2=0x7ff6481a42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:14.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.194+0000 7ff64db5d700 1 --2- 172.21.15.3:0/4262267822 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff64809f240 0x7ff6481a42b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff63c00c450 tx=0x7ff63c00c760 comp rx=0 tx=0).stop 2023-10-03T10:18:14.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.195+0000 7ff64db5d700 1 -- 172.21.15.3:0/4262267822 shutdown_connections 2023-10-03T10:18:14.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.195+0000 7ff64db5d700 1 --2- 172.21.15.3:0/4262267822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff64809f650 0x7ff6481a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:14.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.195+0000 7ff64db5d700 1 --2- 172.21.15.3:0/4262267822 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff64809e930 0x7ff6481a3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:14.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.195+0000 7ff64db5d700 1 --2- 172.21.15.3:0/4262267822 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff64809f240 0x7ff6481a42b0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:14.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.195+0000 7ff64db5d700 1 -- 172.21.15.3:0/4262267822 >> 172.21.15.3:0/4262267822 conn(0x7ff6480a4210 msgr2=0x7ff6481b8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:14.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.195+0000 7ff64db5d700 1 -- 172.21.15.3:0/4262267822 shutdown_connections 2023-10-03T10:18:14.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:14.195+0000 7ff64db5d700 1 -- 172.21.15.3:0/4262267822 wait complete. 2023-10-03T10:18:14.208 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:15.003 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:15.080 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:15.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.151+0000 7f8ec010e700 1 Processor -- start 2023-10-03T10:18:15.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.151+0000 7f8ec010e700 1 -- start start 2023-10-03T10:18:15.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.151+0000 7f8ec010e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8eb809d630 0x7f8eb80a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:15.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.152+0000 7f8ec010e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8eb8138d80 0x7f8eb813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:15.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.152+0000 7f8ec010e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb80a2a70 con 0x7f8eb80a0a30 2023-10-03T10:18:15.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.152+0000 7f8ec010e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb80a01d0 con 0x7f8eb809d630 2023-10-03T10:18:15.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.152+0000 7f8ec010e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb809e2b0 con 0x7f8eb8138d80 2023-10-03T10:18:15.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.152+0000 7f8ebd6a9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8eb809d630 0x7f8eb80a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:15.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.152+0000 7f8ebd6a9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8eb809d630 0x7f8eb80a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59262/0 (socket says 172.21.15.3:59262) 2023-10-03T10:18:15.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.152+0000 7f8ebd6a9700 1 -- 172.21.15.3:0/2708997070 learned_addr learned my addr 172.21.15.3:0/2708997070 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:15.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.152+0000 7f8ebe6ab700 1 --2- 172.21.15.3:0/2708997070 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8eb8138d80 0x7f8eb813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:15.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.152+0000 7f8ebe6ab700 1 -- 172.21.15.3:0/2708997070 >> v2:172.21.15.3:3300/0 conn(0x7f8eb809d630 msgr2=0x7f8eb80a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:15.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.152+0000 7f8ebe6ab700 1 --2- 172.21.15.3:0/2708997070 >> v2:172.21.15.3:3300/0 conn(0x7f8eb809d630 0x7f8eb80a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.152+0000 7f8ebe6ab700 1 -- 172.21.15.3:0/2708997070 >> v1:172.21.15.3:6789/0 conn(0x7f8eb80a0a30 legacy=0x7f8eb80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:15.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.152+0000 7f8ebe6ab700 1 -- 172.21.15.3:0/2708997070 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb809dc50 con 0x7f8eb8138d80 2023-10-03T10:18:15.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.152+0000 7f8ebd6a9700 1 --2- 172.21.15.3:0/2708997070 >> v2:172.21.15.3:3300/0 conn(0x7f8eb809d630 0x7f8eb80a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:15.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.152+0000 7f8ebe6ab700 1 --2- 172.21.15.3:0/2708997070 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8eb8138d80 0x7f8eb813b160 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0009840 tx=0x7f8eb0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=215bfc8f36d4d3da server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:15.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.152+0000 7f8ebcea8700 1 -- 172.21.15.3:0/2708997070 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0011830 con 0x7f8eb8138d80 2023-10-03T10:18:15.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.152+0000 7f8ebcea8700 1 -- 172.21.15.3:0/2708997070 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb0011990 con 0x7f8eb8138d80 2023-10-03T10:18:15.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.153+0000 7f8ebcea8700 1 -- 172.21.15.3:0/2708997070 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0011d20 con 0x7f8eb8138d80 2023-10-03T10:18:15.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.153+0000 7f8ec010e700 1 -- 172.21.15.3:0/2708997070 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8eb8138d80 msgr2=0x7f8eb813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:15.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.153+0000 7f8ec010e700 1 --2- 172.21.15.3:0/2708997070 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8eb8138d80 0x7f8eb813b160 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0009840 tx=0x7f8eb0009b50 comp rx=0 tx=0).stop 2023-10-03T10:18:15.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.153+0000 7f8ec010e700 1 -- 172.21.15.3:0/2708997070 shutdown_connections 2023-10-03T10:18:15.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.153+0000 7f8ec010e700 1 --2- 172.21.15.3:0/2708997070 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8eb8138d80 0x7f8eb813b160 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.153+0000 7f8ec010e700 1 --2- 172.21.15.3:0/2708997070 >> v2:172.21.15.3:3300/0 conn(0x7f8eb809d630 0x7f8eb80a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.153+0000 7f8ec010e700 1 -- 172.21.15.3:0/2708997070 >> 172.21.15.3:0/2708997070 conn(0x7f8eb80a3050 msgr2=0x7f8eb80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:15.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.153+0000 7f8ec010e700 1 -- 172.21.15.3:0/2708997070 shutdown_connections 2023-10-03T10:18:15.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.153+0000 7f8ec010e700 1 -- 172.21.15.3:0/2708997070 wait complete. 2023-10-03T10:18:15.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.154+0000 7f8ec010e700 1 Processor -- start 2023-10-03T10:18:15.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.154+0000 7f8ec010e700 1 -- start start 2023-10-03T10:18:15.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.155+0000 7f8ec010e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8eb809d630 0x7f8eb8120620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:15.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.155+0000 7f8ec010e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8eb80a0a30 0x7f8eb8120b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:15.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.155+0000 7f8ec010e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8eb8138d80 0x7f8eb81210a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:15.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.155+0000 7f8ec010e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb8124140 con 0x7f8eb80a0a30 2023-10-03T10:18:15.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.155+0000 7f8ec010e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb81234f0 con 0x7f8eb809d630 2023-10-03T10:18:15.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.155+0000 7f8ec010e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb8138b20 con 0x7f8eb8138d80 2023-10-03T10:18:15.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.155+0000 7f8ebd6a9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8eb80a0a30 0x7f8eb8120b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:15.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.155+0000 7f8ebe6ab700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8eb8138d80 0x7f8eb81210a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:15.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.155+0000 7f8ebdeaa700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8eb809d630 0x7f8eb8120620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:15.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.155+0000 7f8ebd6a9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8eb80a0a30 0x7f8eb8120b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59276/0 (socket says 172.21.15.3:59276) 2023-10-03T10:18:15.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.155+0000 7f8ebd6a9700 1 -- 172.21.15.3:0/2988310125 learned_addr learned my addr 172.21.15.3:0/2988310125 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:15.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.155+0000 7f8ebe6ab700 1 -- 172.21.15.3:0/2988310125 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8eb809d630 msgr2=0x7f8eb8120620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:15.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.155+0000 7f8ebe6ab700 1 --2- 172.21.15.3:0/2988310125 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8eb809d630 0x7f8eb8120620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.155+0000 7f8ebe6ab700 1 -- 172.21.15.3:0/2988310125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8eb80a0a30 msgr2=0x7f8eb8120b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:15.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.155+0000 7f8ebe6ab700 1 --2- 172.21.15.3:0/2988310125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8eb80a0a30 0x7f8eb8120b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.155+0000 7f8ebe6ab700 1 -- 172.21.15.3:0/2988310125 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea8009550 con 0x7f8eb8138d80 2023-10-03T10:18:15.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.155+0000 7f8ebd6a9700 1 --2- 172.21.15.3:0/2988310125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8eb80a0a30 0x7f8eb8120b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:15.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.155+0000 7f8ebe6ab700 1 --2- 172.21.15.3:0/2988310125 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8eb8138d80 0x7f8eb81210a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0016000 tx=0x7f8eb0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:15.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.156+0000 7f8eae7fc700 1 -- 172.21.15.3:0/2988310125 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0004510 con 0x7f8eb8138d80 2023-10-03T10:18:15.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.156+0000 7f8eae7fc700 1 -- 172.21.15.3:0/2988310125 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb00036a0 con 0x7f8eb8138d80 2023-10-03T10:18:15.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.156+0000 7f8ec010e700 1 -- 172.21.15.3:0/2988310125 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb00094a0 con 0x7f8eb8138d80 2023-10-03T10:18:15.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.156+0000 7f8eae7fc700 1 -- 172.21.15.3:0/2988310125 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0003a30 con 0x7f8eb8138d80 2023-10-03T10:18:15.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.156+0000 7f8ec010e700 1 -- 172.21.15.3:0/2988310125 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb80a0460 con 0x7f8eb8138d80 2023-10-03T10:18:15.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.157+0000 7f8eae7fc700 1 -- 172.21.15.3:0/2988310125 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8eb0003b90 con 0x7f8eb8138d80 2023-10-03T10:18:15.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.157+0000 7f8ec010e700 1 -- 172.21.15.3:0/2988310125 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb8124990 con 0x7f8eb8138d80 2023-10-03T10:18:15.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.157+0000 7f8eae7fc700 1 -- 172.21.15.3:0/2988310125 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8eb00404e0 con 0x7f8eb8138d80 2023-10-03T10:18:15.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.162+0000 7f8eae7fc700 1 -- 172.21.15.3:0/2988310125 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8eb001a420 con 0x7f8eb8138d80 2023-10-03T10:18:15.175 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:15.209 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:15.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.222+0000 7fe473b7f700 1 Processor -- start 2023-10-03T10:18:15.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.223+0000 7fe473b7f700 1 -- start start 2023-10-03T10:18:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.223+0000 7fe473b7f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe46c134630 0x7fe46c136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.223+0000 7fe473b7f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe46c136f50 0x7fe46c139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.223+0000 7fe473b7f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe46c09b4f0 con 0x7fe46c131810 2023-10-03T10:18:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.223+0000 7fe473b7f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe46c1313c0 con 0x7fe46c134630 2023-10-03T10:18:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.223+0000 7fe473b7f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c06e8c0 con 0x7fe46c136f50 2023-10-03T10:18:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.223+0000 7fe47111a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe46c134630 0x7fe46c136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.223+0000 7fe47111a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe46c134630 0x7fe46c136a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59280/0 (socket says 172.21.15.3:59280) 2023-10-03T10:18:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.223+0000 7fe47111a700 1 -- 172.21.15.3:0/3089104308 learned_addr learned my addr 172.21.15.3:0/3089104308 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:15.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.224+0000 7fe47211c700 1 --2- 172.21.15.3:0/3089104308 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe46c136f50 0x7fe46c139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:15.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.224+0000 7fe47111a700 1 -- 172.21.15.3:0/3089104308 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe46c136f50 msgr2=0x7fe46c139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:15.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.224+0000 7fe47111a700 1 --2- 172.21.15.3:0/3089104308 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe46c136f50 0x7fe46c139330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.224+0000 7fe47111a700 1 -- 172.21.15.3:0/3089104308 >> v1:172.21.15.3:6789/0 conn(0x7fe46c131810 legacy=0x7fe46c131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:15.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.224+0000 7fe47111a700 1 -- 172.21.15.3:0/3089104308 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46c071d20 con 0x7fe46c134630 2023-10-03T10:18:15.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.224+0000 7fe47111a700 1 --2- 172.21.15.3:0/3089104308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe46c134630 0x7fe46c136a10 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fe45c0098f0 tx=0x7fe45c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b44911d5851e3e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:15.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.224+0000 7fe470919700 1 -- 172.21.15.3:0/3089104308 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45c011830 con 0x7fe46c134630 2023-10-03T10:18:15.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.224+0000 7fe470919700 1 -- 172.21.15.3:0/3089104308 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe45c011990 con 0x7fe46c134630 2023-10-03T10:18:15.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.224+0000 7fe470919700 1 -- 172.21.15.3:0/3089104308 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45c011d20 con 0x7fe46c134630 2023-10-03T10:18:15.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.225+0000 7fe473b7f700 1 -- 172.21.15.3:0/3089104308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe46c134630 msgr2=0x7fe46c136a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:15.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.225+0000 7fe473b7f700 1 --2- 172.21.15.3:0/3089104308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe46c134630 0x7fe46c136a10 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fe45c0098f0 tx=0x7fe45c009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:15.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.225+0000 7fe473b7f700 1 -- 172.21.15.3:0/3089104308 shutdown_connections 2023-10-03T10:18:15.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.225+0000 7fe473b7f700 1 --2- 172.21.15.3:0/3089104308 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe46c136f50 0x7fe46c139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.225+0000 7fe473b7f700 1 --2- 172.21.15.3:0/3089104308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe46c134630 0x7fe46c136a10 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.225+0000 7fe473b7f700 1 -- 172.21.15.3:0/3089104308 >> 172.21.15.3:0/3089104308 conn(0x7fe46c0a2aa0 msgr2=0x7fe46c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:15.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.225+0000 7fe473b7f700 1 -- 172.21.15.3:0/3089104308 shutdown_connections 2023-10-03T10:18:15.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.225+0000 7fe473b7f700 1 -- 172.21.15.3:0/3089104308 wait complete. 2023-10-03T10:18:15.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.226+0000 7fe473b7f700 1 Processor -- start 2023-10-03T10:18:15.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.226+0000 7fe473b7f700 1 -- start start 2023-10-03T10:18:15.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.227+0000 7fe473b7f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe46c131810 0x7fe46c118f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:15.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.227+0000 7fe473b7f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe46c134630 0x7fe46c119470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:15.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.227+0000 7fe473b7f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe46c136f50 0x7fe46c1199b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:15.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.227+0000 7fe473b7f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c11aa20 con 0x7fe46c131810 2023-10-03T10:18:15.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.227+0000 7fe473b7f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c11cb70 con 0x7fe46c134630 2023-10-03T10:18:15.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.227+0000 7fe473b7f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c11c6e0 con 0x7fe46c136f50 2023-10-03T10:18:15.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.227+0000 7fe47111a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe46c134630 0x7fe46c119470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:15.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.227+0000 7fe47191b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe46c131810 0x7fe46c118f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:15.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.227+0000 7fe47111a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe46c134630 0x7fe46c119470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59678/0 (socket says 172.21.15.3:59678) 2023-10-03T10:18:15.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.227+0000 7fe47191b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe46c131810 0x7fe46c118f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59296/0 (socket says 172.21.15.3:59296) 2023-10-03T10:18:15.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.227+0000 7fe47111a700 1 -- 172.21.15.3:0/1398372425 learned_addr learned my addr 172.21.15.3:0/1398372425 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:15.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.227+0000 7fe47211c700 1 --2- 172.21.15.3:0/1398372425 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe46c136f50 0x7fe46c1199b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:15.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.228+0000 7fe47111a700 1 -- 172.21.15.3:0/1398372425 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe46c136f50 msgr2=0x7fe46c1199b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:15.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.228+0000 7fe47111a700 1 --2- 172.21.15.3:0/1398372425 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe46c136f50 0x7fe46c1199b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.228+0000 7fe47111a700 1 -- 172.21.15.3:0/1398372425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe46c131810 msgr2=0x7fe46c118f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:15.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.228+0000 7fe47111a700 1 --2- 172.21.15.3:0/1398372425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe46c131810 0x7fe46c118f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.228+0000 7fe47111a700 1 -- 172.21.15.3:0/1398372425 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45c009550 con 0x7fe46c134630 2023-10-03T10:18:15.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.228+0000 7fe47111a700 1 --2- 172.21.15.3:0/1398372425 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe46c134630 0x7fe46c119470 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe45c009c40 tx=0x7fe45c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:15.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.228+0000 7fe4627fc700 1 -- 172.21.15.3:0/1398372425 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45c004480 con 0x7fe46c134630 2023-10-03T10:18:15.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.228+0000 7fe4627fc700 1 -- 172.21.15.3:0/1398372425 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe45c01f070 con 0x7fe46c134630 2023-10-03T10:18:15.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.228+0000 7fe4627fc700 1 -- 172.21.15.3:0/1398372425 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45c0038e0 con 0x7fe46c134630 2023-10-03T10:18:15.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.228+0000 7fe473b7f700 1 -- 172.21.15.3:0/1398372425 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe46c09a7a0 con 0x7fe46c134630 2023-10-03T10:18:15.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.228+0000 7fe473b7f700 1 -- 172.21.15.3:0/1398372425 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe46c06eb00 con 0x7fe46c134630 2023-10-03T10:18:15.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.229+0000 7fe4627fc700 1 -- 172.21.15.3:0/1398372425 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe45c021030 con 0x7fe46c134630 2023-10-03T10:18:15.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.229+0000 7fe473b7f700 1 -- 172.21.15.3:0/1398372425 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe46c11c540 con 0x7fe46c134630 2023-10-03T10:18:15.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.229+0000 7fe4627fc700 1 -- 172.21.15.3:0/1398372425 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe45c040bb0 con 0x7fe46c134630 2023-10-03T10:18:15.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.234+0000 7fe4627fc700 1 -- 172.21.15.3:0/1398372425 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe46c11c540 con 0x7fe46c134630 2023-10-03T10:18:15.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.339+0000 7f529e27e700 1 Processor -- start 2023-10-03T10:18:15.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.339+0000 7f529e27e700 1 -- start start 2023-10-03T10:18:15.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.339+0000 7f529e27e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f529809d630 0x7f52980a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:15.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.340+0000 7f529e27e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5298138d80 0x7f529813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:15.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.340+0000 7f529e27e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52980a2a70 con 0x7f52980a0a30 2023-10-03T10:18:15.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.340+0000 7f529e27e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52980a01d0 con 0x7f529809d630 2023-10-03T10:18:15.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.340+0000 7f529e27e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f529809e2b0 con 0x7f5298138d80 2023-10-03T10:18:15.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.340+0000 7f529c81b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5298138d80 0x7f529813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:15.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.340+0000 7f529c81b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5298138d80 0x7f529813b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43402/0 (socket says 172.21.15.3:43402) 2023-10-03T10:18:15.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.340+0000 7f529c81b700 1 -- 172.21.15.3:0/3971775280 learned_addr learned my addr 172.21.15.3:0/3971775280 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:15.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.340+0000 7f52977fe700 1 --2- 172.21.15.3:0/3971775280 >> v2:172.21.15.3:3300/0 conn(0x7f529809d630 0x7f52980a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:15.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.340+0000 7f529c81b700 1 -- 172.21.15.3:0/3971775280 >> v2:172.21.15.3:3300/0 conn(0x7f529809d630 msgr2=0x7f52980a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:15.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.340+0000 7f529c81b700 1 --2- 172.21.15.3:0/3971775280 >> v2:172.21.15.3:3300/0 conn(0x7f529809d630 0x7f52980a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.340+0000 7f529c81b700 1 -- 172.21.15.3:0/3971775280 >> v1:172.21.15.3:6789/0 conn(0x7f52980a0a30 legacy=0x7f52980a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:15.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.340+0000 7f529c81b700 1 -- 172.21.15.3:0/3971775280 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529809dc50 con 0x7f5298138d80 2023-10-03T10:18:15.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.340+0000 7f529c81b700 1 --2- 172.21.15.3:0/3971775280 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5298138d80 0x7f529813b160 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5290009e50 tx=0x7f529000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ac035c25713d7e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:15.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.341+0000 7f5296ffd700 1 -- 172.21.15.3:0/3971775280 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529000e070 con 0x7f5298138d80 2023-10-03T10:18:15.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.341+0000 7f5296ffd700 1 -- 172.21.15.3:0/3971775280 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529000b9c0 con 0x7f5298138d80 2023-10-03T10:18:15.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.341+0000 7f5296ffd700 1 -- 172.21.15.3:0/3971775280 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529000bd70 con 0x7f5298138d80 2023-10-03T10:18:15.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.342+0000 7f529e27e700 1 -- 172.21.15.3:0/3971775280 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5298138d80 msgr2=0x7f529813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:15.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.342+0000 7f529e27e700 1 --2- 172.21.15.3:0/3971775280 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5298138d80 0x7f529813b160 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5290009e50 tx=0x7f529000b330 comp rx=0 tx=0).stop 2023-10-03T10:18:15.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.342+0000 7f529e27e700 1 -- 172.21.15.3:0/3971775280 shutdown_connections 2023-10-03T10:18:15.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.342+0000 7f529e27e700 1 --2- 172.21.15.3:0/3971775280 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5298138d80 0x7f529813b160 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.342+0000 7f529e27e700 1 --2- 172.21.15.3:0/3971775280 >> v2:172.21.15.3:3300/0 conn(0x7f529809d630 0x7f52980a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.342+0000 7f529e27e700 1 -- 172.21.15.3:0/3971775280 >> 172.21.15.3:0/3971775280 conn(0x7f52980a3050 msgr2=0x7f52980a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:15.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.342+0000 7f529e27e700 1 -- 172.21.15.3:0/3971775280 shutdown_connections 2023-10-03T10:18:15.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.342+0000 7f529e27e700 1 -- 172.21.15.3:0/3971775280 wait complete. 2023-10-03T10:18:15.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.343+0000 7f529e27e700 1 Processor -- start 2023-10-03T10:18:15.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.344+0000 7f529e27e700 1 -- start start 2023-10-03T10:18:15.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.344+0000 7f529e27e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f529809d630 0x7f5298120620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:15.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.344+0000 7f529e27e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f52980a0a30 0x7f5298120b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:15.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.344+0000 7f529e27e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5298138d80 0x7f52981210a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:15.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.344+0000 7f529e27e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5298124140 con 0x7f5298138d80 2023-10-03T10:18:15.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.344+0000 7f529e27e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52981234f0 con 0x7f52980a0a30 2023-10-03T10:18:15.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.344+0000 7f529e27e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5298138b20 con 0x7f529809d630 2023-10-03T10:18:15.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.345+0000 7f5297fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f529809d630 0x7f5298120620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:15.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.345+0000 7f52977fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f52980a0a30 0x7f5298120b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:15.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.345+0000 7f52977fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f52980a0a30 0x7f5298120b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59688/0 (socket says 172.21.15.3:59688) 2023-10-03T10:18:15.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.345+0000 7f5297fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f529809d630 0x7f5298120620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43408/0 (socket says 172.21.15.3:43408) 2023-10-03T10:18:15.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.345+0000 7f52977fe700 1 -- 172.21.15.3:0/1806959437 learned_addr learned my addr 172.21.15.3:0/1806959437 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:15.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.345+0000 7f529c81b700 1 --2- 172.21.15.3:0/1806959437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5298138d80 0x7f52981210a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:15.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.345+0000 7f5297fff700 1 -- 172.21.15.3:0/1806959437 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f52980a0a30 msgr2=0x7f5298120b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:15.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.345+0000 7f5297fff700 1 --2- 172.21.15.3:0/1806959437 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f52980a0a30 0x7f5298120b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.345+0000 7f5297fff700 1 -- 172.21.15.3:0/1806959437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5298138d80 msgr2=0x7f52981210a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:15.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.345+0000 7f5297fff700 1 --2- 172.21.15.3:0/1806959437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5298138d80 0x7f52981210a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.345+0000 7f5297fff700 1 -- 172.21.15.3:0/1806959437 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5290009b30 con 0x7f529809d630 2023-10-03T10:18:15.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.346+0000 7f5297fff700 1 --2- 172.21.15.3:0/1806959437 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f529809d630 0x7f5298120620 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5288000f60 tx=0x7f528800c730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:15.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.346+0000 7f5294ff9700 1 -- 172.21.15.3:0/1806959437 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52880052d0 con 0x7f529809d630 2023-10-03T10:18:15.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.346+0000 7f5294ff9700 1 -- 172.21.15.3:0/1806959437 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5288005430 con 0x7f529809d630 2023-10-03T10:18:15.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.346+0000 7f529e27e700 1 -- 172.21.15.3:0/1806959437 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f529813afc0 con 0x7f529809d630 2023-10-03T10:18:15.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.346+0000 7f5294ff9700 1 -- 172.21.15.3:0/1806959437 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52880038c0 con 0x7f529809d630 2023-10-03T10:18:15.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.346+0000 7f529e27e700 1 -- 172.21.15.3:0/1806959437 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52981248d0 con 0x7f529809d630 2023-10-03T10:18:15.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.347+0000 7f529e27e700 1 -- 172.21.15.3:0/1806959437 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5298123dc0 con 0x7f529809d630 2023-10-03T10:18:15.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.348+0000 7f5294ff9700 1 -- 172.21.15.3:0/1806959437 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5288003a20 con 0x7f529809d630 2023-10-03T10:18:15.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.348+0000 7f5294ff9700 1 -- 172.21.15.3:0/1806959437 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f528803eab0 con 0x7f529809d630 2023-10-03T10:18:15.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.353+0000 7f5294ff9700 1 -- 172.21.15.3:0/1806959437 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5298123dc0 con 0x7f529809d630 2023-10-03T10:18:15.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.378+0000 7f8ec010e700 1 -- 172.21.15.3:0/2988310125 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8eb80747f0 con 0x7f8eb8138d80 2023-10-03T10:18:15.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.379+0000 7f8eae7fc700 1 -- 172.21.15.3:0/2988310125 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8eb0028090 con 0x7f8eb8138d80 2023-10-03T10:18:15.379 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:15.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.381+0000 7f8ec010e700 1 -- 172.21.15.3:0/2988310125 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8eb8138d80 msgr2=0x7f8eb81210a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:15.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.381+0000 7f8ec010e700 1 --2- 172.21.15.3:0/2988310125 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8eb8138d80 0x7f8eb81210a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0016000 tx=0x7f8eb0004750 comp rx=0 tx=0).stop 2023-10-03T10:18:15.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.381+0000 7f8ec010e700 1 -- 172.21.15.3:0/2988310125 shutdown_connections 2023-10-03T10:18:15.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.381+0000 7f8ec010e700 1 --2- 172.21.15.3:0/2988310125 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8eb8138d80 0x7f8eb81210a0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.381+0000 7f8ec010e700 1 --2- 172.21.15.3:0/2988310125 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8eb809d630 0x7f8eb8120620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.381+0000 7f8ec010e700 1 --2- 172.21.15.3:0/2988310125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8eb80a0a30 0x7f8eb8120b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.381+0000 7f8ec010e700 1 -- 172.21.15.3:0/2988310125 >> 172.21.15.3:0/2988310125 conn(0x7f8eb80a3050 msgr2=0x7f8eb8139150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:15.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.381+0000 7f8ec010e700 1 -- 172.21.15.3:0/2988310125 shutdown_connections 2023-10-03T10:18:15.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.381+0000 7f8ec010e700 1 -- 172.21.15.3:0/2988310125 wait complete. 2023-10-03T10:18:15.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.384+0000 7f5796e4e700 1 Processor -- start 2023-10-03T10:18:15.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.384+0000 7f5796e4e700 1 -- start start 2023-10-03T10:18:15.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.385+0000 7f5796e4e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f579009d6f0 0x7f579009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:15.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.385+0000 7f5796e4e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f579009e000 0x7f5790130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:15.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.385+0000 7f5796e4e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f579009e410 0x7f5790130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:15.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.385+0000 7f5796e4e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f579009f520 con 0x7f579009e000 2023-10-03T10:18:15.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.385+0000 7f5796e4e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579009f130 con 0x7f579009e410 2023-10-03T10:18:15.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.385+0000 7f5796e4e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f579009e930 con 0x7f579009d6f0 2023-10-03T10:18:15.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.385+0000 7f5794bea700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f579009d6f0 0x7f579009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:15.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.385+0000 7f5794bea700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f579009d6f0 0x7f579009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43418/0 (socket says 172.21.15.3:43418) 2023-10-03T10:18:15.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.385+0000 7f5794bea700 1 -- 172.21.15.3:0/4122989065 learned_addr learned my addr 172.21.15.3:0/4122989065 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:15.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.385+0000 7f57953eb700 1 --2- 172.21.15.3:0/4122989065 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f579009e410 0x7f5790130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:15.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.386+0000 7f5794bea700 1 -- 172.21.15.3:0/4122989065 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f579009e410 msgr2=0x7f5790130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:15.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.386+0000 7f5794bea700 1 --2- 172.21.15.3:0/4122989065 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f579009e410 0x7f5790130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.386+0000 7f5794bea700 1 -- 172.21.15.3:0/4122989065 >> v2:172.21.15.3:3300/0 conn(0x7f579009e000 msgr2=0x7f5790130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:15.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.386+0000 7f578ffff700 1 --2- 172.21.15.3:0/4122989065 >> v2:172.21.15.3:3300/0 conn(0x7f579009e000 0x7f5790130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:15.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.386+0000 7f5794bea700 1 --2- 172.21.15.3:0/4122989065 >> v2:172.21.15.3:3300/0 conn(0x7f579009e000 0x7f5790130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.386+0000 7f5794bea700 1 -- 172.21.15.3:0/4122989065 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579009b2d0 con 0x7f579009d6f0 2023-10-03T10:18:15.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.386+0000 7f578ffff700 1 --2- 172.21.15.3:0/4122989065 >> v2:172.21.15.3:3300/0 conn(0x7f579009e000 0x7f5790130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:18:15.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.386+0000 7f5794bea700 1 --2- 172.21.15.3:0/4122989065 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f579009d6f0 0x7f579009dac0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5784009920 tx=0x7f5784009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bab2b43969e451d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:15.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.386+0000 7f578f7fe700 1 -- 172.21.15.3:0/4122989065 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578400ba20 con 0x7f579009d6f0 2023-10-03T10:18:15.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.386+0000 7f578f7fe700 1 -- 172.21.15.3:0/4122989065 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578400bb80 con 0x7f579009d6f0 2023-10-03T10:18:15.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.386+0000 7f578f7fe700 1 -- 172.21.15.3:0/4122989065 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5784009e70 con 0x7f579009d6f0 2023-10-03T10:18:15.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.387+0000 7f5796e4e700 1 -- 172.21.15.3:0/4122989065 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f579009d6f0 msgr2=0x7f579009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:15.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.387+0000 7f5796e4e700 1 --2- 172.21.15.3:0/4122989065 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f579009d6f0 0x7f579009dac0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5784009920 tx=0x7f5784009c30 comp rx=0 tx=0).stop 2023-10-03T10:18:15.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.387+0000 7f5796e4e700 1 -- 172.21.15.3:0/4122989065 shutdown_connections 2023-10-03T10:18:15.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.387+0000 7f5796e4e700 1 --2- 172.21.15.3:0/4122989065 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f579009e410 0x7f5790130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.387+0000 7f5796e4e700 1 --2- 172.21.15.3:0/4122989065 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f579009d6f0 0x7f579009dac0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.387+0000 7f5796e4e700 1 --2- 172.21.15.3:0/4122989065 >> v2:172.21.15.3:3300/0 conn(0x7f579009e000 0x7f5790130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.387+0000 7f5796e4e700 1 -- 172.21.15.3:0/4122989065 >> 172.21.15.3:0/4122989065 conn(0x7f5790066f50 msgr2=0x7f579009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:15.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.387+0000 7f5796e4e700 1 -- 172.21.15.3:0/4122989065 shutdown_connections 2023-10-03T10:18:15.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.387+0000 7f5796e4e700 1 -- 172.21.15.3:0/4122989065 wait complete. 2023-10-03T10:18:15.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.389+0000 7f5796e4e700 1 Processor -- start 2023-10-03T10:18:15.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.389+0000 7f5796e4e700 1 -- start start 2023-10-03T10:18:15.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.390+0000 7f5796e4e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f579009d6f0 0x7f5790137160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:15.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.390+0000 7f5796e4e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f579009e000 0x7f57901376a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:15.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.390+0000 7f5796e4e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f579009e410 0x7f57901c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:15.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.390+0000 7f5796e4e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5790119d50 con 0x7f579009e000 2023-10-03T10:18:15.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.390+0000 7f5796e4e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5790119f30 con 0x7f579009e410 2023-10-03T10:18:15.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.390+0000 7f5796e4e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5790130e00 con 0x7f579009d6f0 2023-10-03T10:18:15.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.390+0000 7f578ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f579009e000 0x7f57901376a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:15.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.390+0000 7f578ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f579009e000 0x7f57901376a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59328/0 (socket says 172.21.15.3:59328) 2023-10-03T10:18:15.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.390+0000 7f578ffff700 1 -- 172.21.15.3:0/3958891495 learned_addr learned my addr 172.21.15.3:0/3958891495 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:15.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.390+0000 7f57953eb700 1 --2- 172.21.15.3:0/3958891495 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f579009e410 0x7f57901c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:15.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.390+0000 7f5794bea700 1 --2- 172.21.15.3:0/3958891495 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f579009d6f0 0x7f5790137160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:15.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.391+0000 7f578ffff700 1 -- 172.21.15.3:0/3958891495 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f579009d6f0 msgr2=0x7f5790137160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:15.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.391+0000 7f578ffff700 1 --2- 172.21.15.3:0/3958891495 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f579009d6f0 0x7f5790137160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.391+0000 7f578ffff700 1 -- 172.21.15.3:0/3958891495 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f579009e410 msgr2=0x7f57901c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:15.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.391+0000 7f578ffff700 1 --2- 172.21.15.3:0/3958891495 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f579009e410 0x7f57901c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.391+0000 7f578ffff700 1 -- 172.21.15.3:0/3958891495 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5784009550 con 0x7f579009e000 2023-10-03T10:18:15.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.391+0000 7f578ffff700 1 --2- 172.21.15.3:0/3958891495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f579009e000 0x7f57901376a0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f577c00c7f0 tx=0x7f577c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:15.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.391+0000 7f578d7fa700 1 -- 172.21.15.3:0/3958891495 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c004220 con 0x7f579009e000 2023-10-03T10:18:15.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.391+0000 7f578d7fa700 1 -- 172.21.15.3:0/3958891495 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f577c004380 con 0x7f579009e000 2023-10-03T10:18:15.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.391+0000 7f578d7fa700 1 -- 172.21.15.3:0/3958891495 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c0036a0 con 0x7f579009e000 2023-10-03T10:18:15.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.391+0000 7f5796e4e700 1 -- 172.21.15.3:0/3958891495 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5790136b10 con 0x7f579009e000 2023-10-03T10:18:15.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.391+0000 7f5796e4e700 1 -- 172.21.15.3:0/3958891495 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5790138a60 con 0x7f579009e000 2023-10-03T10:18:15.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.392+0000 7f578d7fa700 1 -- 172.21.15.3:0/3958891495 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f577c003800 con 0x7f579009e000 2023-10-03T10:18:15.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.393+0000 7f578d7fa700 1 -- 172.21.15.3:0/3958891495 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f577c03f930 con 0x7f579009e000 2023-10-03T10:18:15.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.393+0000 7f5796e4e700 1 -- 172.21.15.3:0/3958891495 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5790119bb0 con 0x7f579009e000 2023-10-03T10:18:15.398 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:15.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.399+0000 7f578d7fa700 1 -- 172.21.15.3:0/3958891495 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5790119bb0 con 0x7f579009e000 2023-10-03T10:18:15.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.464+0000 7fe473b7f700 1 -- 172.21.15.3:0/1398372425 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe46c11c540 con 0x7fe46c134630 2023-10-03T10:18:15.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.465+0000 7fe4627fc700 1 -- 172.21.15.3:0/1398372425 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe46c11c540 con 0x7fe46c134630 2023-10-03T10:18:15.465 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:15.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.467+0000 7fe473b7f700 1 -- 172.21.15.3:0/1398372425 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe46c134630 msgr2=0x7fe46c119470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:15.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.467+0000 7fe473b7f700 1 --2- 172.21.15.3:0/1398372425 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe46c134630 0x7fe46c119470 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe45c009c40 tx=0x7fe45c004750 comp rx=0 tx=0).stop 2023-10-03T10:18:15.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.467+0000 7fe473b7f700 1 -- 172.21.15.3:0/1398372425 shutdown_connections 2023-10-03T10:18:15.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.467+0000 7fe473b7f700 1 --2- 172.21.15.3:0/1398372425 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe46c136f50 0x7fe46c1199b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.467+0000 7fe473b7f700 1 --2- 172.21.15.3:0/1398372425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe46c131810 0x7fe46c118f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.467+0000 7fe473b7f700 1 --2- 172.21.15.3:0/1398372425 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe46c134630 0x7fe46c119470 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.467+0000 7fe473b7f700 1 -- 172.21.15.3:0/1398372425 >> 172.21.15.3:0/1398372425 conn(0x7fe46c0a2aa0 msgr2=0x7fe46c099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:15.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.467+0000 7fe473b7f700 1 -- 172.21.15.3:0/1398372425 shutdown_connections 2023-10-03T10:18:15.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.467+0000 7fe473b7f700 1 -- 172.21.15.3:0/1398372425 wait complete. 2023-10-03T10:18:15.481 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:15.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.608+0000 7f529e27e700 1 -- 172.21.15.3:0/1806959437 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f52980747f0 con 0x7f529809d630 2023-10-03T10:18:15.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.608+0000 7f5294ff9700 1 -- 172.21.15.3:0/1806959437 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5288017020 con 0x7f529809d630 2023-10-03T10:18:15.608 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:15.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.611+0000 7f529e27e700 1 -- 172.21.15.3:0/1806959437 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f529809d630 msgr2=0x7f5298120620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:15.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.611+0000 7f529e27e700 1 --2- 172.21.15.3:0/1806959437 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f529809d630 0x7f5298120620 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5288000f60 tx=0x7f528800c730 comp rx=0 tx=0).stop 2023-10-03T10:18:15.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.611+0000 7f529e27e700 1 -- 172.21.15.3:0/1806959437 shutdown_connections 2023-10-03T10:18:15.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.611+0000 7f529e27e700 1 --2- 172.21.15.3:0/1806959437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5298138d80 0x7f52981210a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.611+0000 7f529e27e700 1 --2- 172.21.15.3:0/1806959437 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f529809d630 0x7f5298120620 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.611+0000 7f529e27e700 1 --2- 172.21.15.3:0/1806959437 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f52980a0a30 0x7f5298120b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.611+0000 7f529e27e700 1 -- 172.21.15.3:0/1806959437 >> 172.21.15.3:0/1806959437 conn(0x7f52980a3050 msgr2=0x7f529809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:15.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.611+0000 7f529e27e700 1 -- 172.21.15.3:0/1806959437 shutdown_connections 2023-10-03T10:18:15.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.611+0000 7f529e27e700 1 -- 172.21.15.3:0/1806959437 wait complete. 2023-10-03T10:18:15.624 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:15.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.639+0000 7f5796e4e700 1 -- 172.21.15.3:0/3958891495 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5790119bb0 con 0x7f579009e000 2023-10-03T10:18:15.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.640+0000 7f578d7fa700 1 -- 172.21.15.3:0/3958891495 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5790119bb0 con 0x7f579009e000 2023-10-03T10:18:15.640 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:15.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.642+0000 7f5796e4e700 1 -- 172.21.15.3:0/3958891495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f579009e000 msgr2=0x7f57901376a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:15.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.642+0000 7f5796e4e700 1 --2- 172.21.15.3:0/3958891495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f579009e000 0x7f57901376a0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f577c00c7f0 tx=0x7f577c00cb00 comp rx=0 tx=0).stop 2023-10-03T10:18:15.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.642+0000 7f5796e4e700 1 -- 172.21.15.3:0/3958891495 shutdown_connections 2023-10-03T10:18:15.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.642+0000 7f5796e4e700 1 --2- 172.21.15.3:0/3958891495 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f579009e410 0x7f57901c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.642+0000 7f5796e4e700 1 --2- 172.21.15.3:0/3958891495 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f579009d6f0 0x7f5790137160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.642+0000 7f5796e4e700 1 --2- 172.21.15.3:0/3958891495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f579009e000 0x7f57901376a0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:15.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.642+0000 7f5796e4e700 1 -- 172.21.15.3:0/3958891495 >> 172.21.15.3:0/3958891495 conn(0x7f5790066f50 msgr2=0x7f5790070d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:15.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.642+0000 7f5796e4e700 1 -- 172.21.15.3:0/3958891495 shutdown_connections 2023-10-03T10:18:15.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:15.642+0000 7f5796e4e700 1 -- 172.21.15.3:0/3958891495 wait complete. 2023-10-03T10:18:15.656 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:16.399 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:16.482 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:16.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.548+0000 7fd86812f700 1 Processor -- start 2023-10-03T10:18:16.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.549+0000 7fd86812f700 1 -- start start 2023-10-03T10:18:16.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.549+0000 7fd86812f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd8600a0a30 0x7fd8600a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:16.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.549+0000 7fd86812f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd86009d630 0x7fd8600a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:16.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.549+0000 7fd86812f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8600a2a70 con 0x7fd860133930 2023-10-03T10:18:16.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.549+0000 7fd86812f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8600a01d0 con 0x7fd8600a0a30 2023-10-03T10:18:16.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.549+0000 7fd86812f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd86009e2b0 con 0x7fd86009d630 2023-10-03T10:18:16.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.549+0000 7fd865ecb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd8600a0a30 0x7fd8600a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:16.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.549+0000 7fd8656ca700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd86009d630 0x7fd8600a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:16.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.550+0000 7fd865ecb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd8600a0a30 0x7fd8600a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59342/0 (socket says 172.21.15.3:59342) 2023-10-03T10:18:16.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.550+0000 7fd8656ca700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd86009d630 0x7fd8600a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43446/0 (socket says 172.21.15.3:43446) 2023-10-03T10:18:16.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.550+0000 7fd865ecb700 1 -- 172.21.15.3:0/3411600277 learned_addr learned my addr 172.21.15.3:0/3411600277 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:16.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.550+0000 7fd865ecb700 1 -- 172.21.15.3:0/3411600277 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd86009d630 msgr2=0x7fd8600a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:16.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.550+0000 7fd865ecb700 1 --2- 172.21.15.3:0/3411600277 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd86009d630 0x7fd8600a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.550+0000 7fd865ecb700 1 -- 172.21.15.3:0/3411600277 >> v1:172.21.15.3:6789/0 conn(0x7fd860133930 legacy=0x7fd860133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:16.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.550+0000 7fd865ecb700 1 -- 172.21.15.3:0/3411600277 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd86009dc50 con 0x7fd8600a0a30 2023-10-03T10:18:16.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.550+0000 7fd8656ca700 1 --2- 172.21.15.3:0/3411600277 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd86009d630 0x7fd8600a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:16.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.550+0000 7fd865ecb700 1 --2- 172.21.15.3:0/3411600277 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8600a0a30 0x7fd8600a0e00 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fd850009e50 tx=0x7fd85000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36dd33f60d88a2af server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:16.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.550+0000 7fd864ec9700 1 -- 172.21.15.3:0/3411600277 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85000e070 con 0x7fd8600a0a30 2023-10-03T10:18:16.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.550+0000 7fd864ec9700 1 -- 172.21.15.3:0/3411600277 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85000b910 con 0x7fd8600a0a30 2023-10-03T10:18:16.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.550+0000 7fd864ec9700 1 -- 172.21.15.3:0/3411600277 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85000bcc0 con 0x7fd8600a0a30 2023-10-03T10:18:16.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.551+0000 7fd86812f700 1 -- 172.21.15.3:0/3411600277 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8600a0a30 msgr2=0x7fd8600a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:16.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.551+0000 7fd86812f700 1 --2- 172.21.15.3:0/3411600277 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8600a0a30 0x7fd8600a0e00 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fd850009e50 tx=0x7fd85000b330 comp rx=0 tx=0).stop 2023-10-03T10:18:16.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.551+0000 7fd86812f700 1 -- 172.21.15.3:0/3411600277 shutdown_connections 2023-10-03T10:18:16.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.551+0000 7fd86812f700 1 --2- 172.21.15.3:0/3411600277 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8600a0a30 0x7fd8600a0e00 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.551+0000 7fd86812f700 1 --2- 172.21.15.3:0/3411600277 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd86009d630 0x7fd8600a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.551+0000 7fd86812f700 1 -- 172.21.15.3:0/3411600277 >> 172.21.15.3:0/3411600277 conn(0x7fd8600a3050 msgr2=0x7fd8600a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:16.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.551+0000 7fd86812f700 1 -- 172.21.15.3:0/3411600277 shutdown_connections 2023-10-03T10:18:16.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.551+0000 7fd86812f700 1 -- 172.21.15.3:0/3411600277 wait complete. 2023-10-03T10:18:16.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.552+0000 7fd86812f700 1 Processor -- start 2023-10-03T10:18:16.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.552+0000 7fd86812f700 1 -- start start 2023-10-03T10:18:16.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.553+0000 7fd86812f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd86009d630 0x7fd8601c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:16.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.553+0000 7fd86812f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8600a0a30 0x7fd8601c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:16.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.553+0000 7fd86812f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd860133930 0x7fd8601c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:16.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.553+0000 7fd86812f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86011eea0 con 0x7fd86009d630 2023-10-03T10:18:16.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.553+0000 7fd86812f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8601207f0 con 0x7fd8600a0a30 2023-10-03T10:18:16.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.553+0000 7fd86812f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd860120600 con 0x7fd860133930 2023-10-03T10:18:16.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.553+0000 7fd865ecb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd86009d630 0x7fd8601c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:16.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.553+0000 7fd8666cc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd860133930 0x7fd8601c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:16.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.553+0000 7fd865ecb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd86009d630 0x7fd8601c8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59350/0 (socket says 172.21.15.3:59350) 2023-10-03T10:18:16.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.553+0000 7fd865ecb700 1 -- 172.21.15.3:0/936272259 learned_addr learned my addr 172.21.15.3:0/936272259 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:16.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.553+0000 7fd8656ca700 1 --2- 172.21.15.3:0/936272259 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8600a0a30 0x7fd8601c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:16.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.553+0000 7fd8666cc700 1 -- 172.21.15.3:0/936272259 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8600a0a30 msgr2=0x7fd8601c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:16.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.553+0000 7fd8666cc700 1 --2- 172.21.15.3:0/936272259 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8600a0a30 0x7fd8601c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.553+0000 7fd8666cc700 1 -- 172.21.15.3:0/936272259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd86009d630 msgr2=0x7fd8601c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:16.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.553+0000 7fd8666cc700 1 --2- 172.21.15.3:0/936272259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd86009d630 0x7fd8601c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.553+0000 7fd8666cc700 1 -- 172.21.15.3:0/936272259 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8540094a0 con 0x7fd860133930 2023-10-03T10:18:16.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.553+0000 7fd865ecb700 1 --2- 172.21.15.3:0/936272259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd86009d630 0x7fd8601c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:16.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.553+0000 7fd8666cc700 1 --2- 172.21.15.3:0/936272259 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd860133930 0x7fd8601c8ba0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fd858000f60 tx=0x7fd85800c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:16.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.554+0000 7fd85e7fc700 1 -- 172.21.15.3:0/936272259 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8580052d0 con 0x7fd860133930 2023-10-03T10:18:16.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.554+0000 7fd85e7fc700 1 -- 172.21.15.3:0/936272259 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd858005430 con 0x7fd860133930 2023-10-03T10:18:16.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.554+0000 7fd86812f700 1 -- 172.21.15.3:0/936272259 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd850009b30 con 0x7fd860133930 2023-10-03T10:18:16.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.554+0000 7fd85e7fc700 1 -- 172.21.15.3:0/936272259 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8580038c0 con 0x7fd860133930 2023-10-03T10:18:16.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.554+0000 7fd86812f700 1 -- 172.21.15.3:0/936272259 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd860122230 con 0x7fd860133930 2023-10-03T10:18:16.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.555+0000 7fd86812f700 1 -- 172.21.15.3:0/936272259 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd860122420 con 0x7fd860133930 2023-10-03T10:18:16.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.555+0000 7fd85e7fc700 1 -- 172.21.15.3:0/936272259 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd858003a20 con 0x7fd860133930 2023-10-03T10:18:16.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.555+0000 7fd85e7fc700 1 -- 172.21.15.3:0/936272259 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd85803edd0 con 0x7fd860133930 2023-10-03T10:18:16.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.564+0000 7fd85e7fc700 1 -- 172.21.15.3:0/936272259 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd858019b40 con 0x7fd860133930 2023-10-03T10:18:16.625 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:16.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.627+0000 7f25b2fa7700 1 Processor -- start 2023-10-03T10:18:16.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.627+0000 7f25b2fa7700 1 -- start start 2023-10-03T10:18:16.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.628+0000 7f25b2fa7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f25ac0a0a30 0x7f25ac0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:16.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.628+0000 7f25b2fa7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25ac09d630 0x7f25ac0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:16.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.628+0000 7f25b2fa7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25ac133930 0x7f25ac133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:16.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.628+0000 7f25b2fa7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25ac0a2a70 con 0x7f25ac0a0a30 2023-10-03T10:18:16.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.628+0000 7f25b2fa7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac0a01d0 con 0x7f25ac133930 2023-10-03T10:18:16.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.628+0000 7f25b2fa7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac09e2b0 con 0x7f25ac09d630 2023-10-03T10:18:16.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.628+0000 7f25abfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25ac09d630 0x7f25ac0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:16.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.628+0000 7f25abfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25ac09d630 0x7f25ac0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43460/0 (socket says 172.21.15.3:43460) 2023-10-03T10:18:16.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.628+0000 7f25abfff700 1 -- 172.21.15.3:0/3262490604 learned_addr learned my addr 172.21.15.3:0/3262490604 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:16.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.628+0000 7f25b1544700 1 --2- 172.21.15.3:0/3262490604 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25ac133930 0x7f25ac133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:16.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.628+0000 7f25b0d43700 1 --2- 172.21.15.3:0/3262490604 >> v2:172.21.15.3:3300/0 conn(0x7f25ac0a0a30 0x7f25ac0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:16.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.629+0000 7f25b1544700 1 -- 172.21.15.3:0/3262490604 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25ac09d630 msgr2=0x7f25ac0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:16.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.629+0000 7f25b1544700 1 --2- 172.21.15.3:0/3262490604 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25ac09d630 0x7f25ac0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.629+0000 7f25b1544700 1 -- 172.21.15.3:0/3262490604 >> v2:172.21.15.3:3300/0 conn(0x7f25ac0a0a30 msgr2=0x7f25ac0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:16.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.629+0000 7f25b1544700 1 --2- 172.21.15.3:0/3262490604 >> v2:172.21.15.3:3300/0 conn(0x7f25ac0a0a30 0x7f25ac0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.629+0000 7f25b1544700 1 -- 172.21.15.3:0/3262490604 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ac09dc50 con 0x7f25ac133930 2023-10-03T10:18:16.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.629+0000 7f25abfff700 1 --2- 172.21.15.3:0/3262490604 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25ac09d630 0x7f25ac0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:16.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.629+0000 7f25b1544700 1 --2- 172.21.15.3:0/3262490604 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25ac133930 0x7f25ac133db0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f25a4009d90 tx=0x7f25a400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbb4b54d7b0682c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.629+0000 7f25ab7fe700 1 -- 172.21.15.3:0/3262490604 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a400e070 con 0x7f25ac133930 2023-10-03T10:18:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.629+0000 7f25ab7fe700 1 -- 172.21.15.3:0/3262490604 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a400bb20 con 0x7f25ac133930 2023-10-03T10:18:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.629+0000 7f25ab7fe700 1 -- 172.21.15.3:0/3262490604 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a4012430 con 0x7f25ac133930 2023-10-03T10:18:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.630+0000 7f25b2fa7700 1 -- 172.21.15.3:0/3262490604 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25ac133930 msgr2=0x7f25ac133db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.630+0000 7f25b2fa7700 1 --2- 172.21.15.3:0/3262490604 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25ac133930 0x7f25ac133db0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f25a4009d90 tx=0x7f25a400b330 comp rx=0 tx=0).stop 2023-10-03T10:18:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.630+0000 7f25b2fa7700 1 -- 172.21.15.3:0/3262490604 shutdown_connections 2023-10-03T10:18:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.630+0000 7f25b2fa7700 1 --2- 172.21.15.3:0/3262490604 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25ac133930 0x7f25ac133db0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.630+0000 7f25b2fa7700 1 --2- 172.21.15.3:0/3262490604 >> v2:172.21.15.3:3300/0 conn(0x7f25ac0a0a30 0x7f25ac0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.630+0000 7f25b2fa7700 1 --2- 172.21.15.3:0/3262490604 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25ac09d630 0x7f25ac0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.630+0000 7f25b2fa7700 1 -- 172.21.15.3:0/3262490604 >> 172.21.15.3:0/3262490604 conn(0x7f25ac0a3050 msgr2=0x7f25ac0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.630+0000 7f25b2fa7700 1 -- 172.21.15.3:0/3262490604 shutdown_connections 2023-10-03T10:18:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.630+0000 7f25b2fa7700 1 -- 172.21.15.3:0/3262490604 wait complete. 2023-10-03T10:18:16.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.631+0000 7f25b2fa7700 1 Processor -- start 2023-10-03T10:18:16.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.631+0000 7f25b2fa7700 1 -- start start 2023-10-03T10:18:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.632+0000 7f25b2fa7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25ac09d630 0x7f25ac1c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.632+0000 7f25b2fa7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25ac0a0a30 0x7f25ac1c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.632+0000 7f25b2fa7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25ac133930 0x7f25ac1c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.632+0000 7f25b2fa7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac0a0410 con 0x7f25ac0a0a30 2023-10-03T10:18:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.632+0000 7f25b2fa7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac120630 con 0x7f25ac09d630 2023-10-03T10:18:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.632+0000 7f25b2fa7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac0a4030 con 0x7f25ac133930 2023-10-03T10:18:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.632+0000 7f25abfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25ac0a0a30 0x7f25ac1c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:16.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.632+0000 7f25abfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25ac0a0a30 0x7f25ac1c83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59366/0 (socket says 172.21.15.3:59366) 2023-10-03T10:18:16.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.632+0000 7f25abfff700 1 -- 172.21.15.3:0/959643227 learned_addr learned my addr 172.21.15.3:0/959643227 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:16.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.632+0000 7f25b0d43700 1 --2- 172.21.15.3:0/959643227 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25ac09d630 0x7f25ac1c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:16.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.632+0000 7f25b1544700 1 --2- 172.21.15.3:0/959643227 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25ac133930 0x7f25ac1c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:16.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.633+0000 7f25b0d43700 1 -- 172.21.15.3:0/959643227 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25ac133930 msgr2=0x7f25ac1c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:16.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.633+0000 7f25b0d43700 1 --2- 172.21.15.3:0/959643227 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25ac133930 0x7f25ac1c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.633+0000 7f25b0d43700 1 -- 172.21.15.3:0/959643227 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25ac0a0a30 msgr2=0x7f25ac1c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:16.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.633+0000 7f25b0d43700 1 --2- 172.21.15.3:0/959643227 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25ac0a0a30 0x7f25ac1c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.633+0000 7f25b0d43700 1 -- 172.21.15.3:0/959643227 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2594009550 con 0x7f25ac09d630 2023-10-03T10:18:16.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.633+0000 7f25b0d43700 1 --2- 172.21.15.3:0/959643227 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25ac09d630 0x7f25ac1c7e80 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f259c00e900 tx=0x7f259c00ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:16.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.633+0000 7f25a97fa700 1 -- 172.21.15.3:0/959643227 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c013680 con 0x7f25ac09d630 2023-10-03T10:18:16.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.633+0000 7f25a97fa700 1 -- 172.21.15.3:0/959643227 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259c0137e0 con 0x7f25ac09d630 2023-10-03T10:18:16.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.633+0000 7f25a97fa700 1 -- 172.21.15.3:0/959643227 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c013b50 con 0x7f25ac09d630 2023-10-03T10:18:16.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.633+0000 7f25b2fa7700 1 -- 172.21.15.3:0/959643227 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a4009a70 con 0x7f25ac09d630 2023-10-03T10:18:16.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.633+0000 7f25b2fa7700 1 -- 172.21.15.3:0/959643227 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25ac120ca0 con 0x7f25ac09d630 2023-10-03T10:18:16.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.634+0000 7f25a97fa700 1 -- 172.21.15.3:0/959643227 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f259c013cb0 con 0x7f25ac09d630 2023-10-03T10:18:16.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.634+0000 7f25a97fa700 1 -- 172.21.15.3:0/959643227 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f259c03f640 con 0x7f25ac09d630 2023-10-03T10:18:16.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.635+0000 7f25b2fa7700 1 -- 172.21.15.3:0/959643227 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25ac0747f0 con 0x7f25ac09d630 2023-10-03T10:18:16.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.640+0000 7f25a97fa700 1 -- 172.21.15.3:0/959643227 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f259c01bbe0 con 0x7f25ac09d630 2023-10-03T10:18:16.656 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:16.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.768+0000 7f8d9cd50700 1 Processor -- start 2023-10-03T10:18:16.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.768+0000 7f8d9cd50700 1 -- start start 2023-10-03T10:18:16.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.769+0000 7f8d9cd50700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d98130540 0x7f8d98130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:16.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.769+0000 7f8d9cd50700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d98130e50 0x7f8d9806f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:16.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.769+0000 7f8d9cd50700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8d9806fb20 0x7f8d9806ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:16.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.769+0000 7f8d9cd50700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d9809a390 con 0x7f8d9806fb20 2023-10-03T10:18:16.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.769+0000 7f8d9cd50700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d980716a0 con 0x7f8d98130e50 2023-10-03T10:18:16.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.769+0000 7f8d9cd50700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8d98071290 con 0x7f8d98130540 2023-10-03T10:18:16.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.769+0000 7f8d96d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8d9806fb20 0x7f8d9806ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:16.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.769+0000 7f8d96d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8d9806fb20 0x7f8d9806ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59378/0 (socket says 172.21.15.3:59378) 2023-10-03T10:18:16.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.769+0000 7f8d96d9d700 1 -- 172.21.15.3:0/3300418632 learned_addr learned my addr 172.21.15.3:0/3300418632 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:16.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.769+0000 7f8d95d9b700 1 --2- 172.21.15.3:0/3300418632 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d98130e50 0x7f8d9806f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:16.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.769+0000 7f8d9659c700 1 --2- 172.21.15.3:0/3300418632 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d98130540 0x7f8d98130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:16.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.770+0000 7f8d96d9d700 1 -- 172.21.15.3:0/3300418632 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d98130540 msgr2=0x7f8d98130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:16.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.770+0000 7f8d96d9d700 1 --2- 172.21.15.3:0/3300418632 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d98130540 0x7f8d98130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.770+0000 7f8d96d9d700 1 -- 172.21.15.3:0/3300418632 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d98130e50 msgr2=0x7f8d9806f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:16.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.770+0000 7f8d96d9d700 1 --2- 172.21.15.3:0/3300418632 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d98130e50 0x7f8d9806f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.770+0000 7f8d96d9d700 1 -- 172.21.15.3:0/3300418632 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d98130340 con 0x7f8d9806fb20 2023-10-03T10:18:16.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.770+0000 7f8d95d9b700 1 --2- 172.21.15.3:0/3300418632 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d98130e50 0x7f8d9806f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:16.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.770+0000 7f8d96d9d700 1 --2- 172.21.15.3:0/3300418632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d9806fb20 0x7f8d9806ffc0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f8d90009e80 tx=0x7f8d9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4956a1d28815066 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:16.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.770+0000 7f8d9559a700 1 -- 172.21.15.3:0/3300418632 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9000bad0 con 0x7f8d9806fb20 2023-10-03T10:18:16.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.770+0000 7f8d9559a700 1 -- 172.21.15.3:0/3300418632 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d9000bc30 con 0x7f8d9806fb20 2023-10-03T10:18:16.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.770+0000 7f8d9559a700 1 -- 172.21.15.3:0/3300418632 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90011460 con 0x7f8d9806fb20 2023-10-03T10:18:16.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.771+0000 7f8d9cd50700 1 -- 172.21.15.3:0/3300418632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d9806fb20 msgr2=0x7f8d9806ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:16.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.771+0000 7f8d9cd50700 1 --2- 172.21.15.3:0/3300418632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d9806fb20 0x7f8d9806ffc0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f8d90009e80 tx=0x7f8d9000b330 comp rx=0 tx=0).stop 2023-10-03T10:18:16.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.771+0000 7f8d9cd50700 1 -- 172.21.15.3:0/3300418632 shutdown_connections 2023-10-03T10:18:16.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.771+0000 7f8d9cd50700 1 --2- 172.21.15.3:0/3300418632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d9806fb20 0x7f8d9806ffc0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.771+0000 7f8d9cd50700 1 --2- 172.21.15.3:0/3300418632 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d98130540 0x7f8d98130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.771+0000 7f8d9cd50700 1 --2- 172.21.15.3:0/3300418632 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d98130e50 0x7f8d9806f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.771+0000 7f8d9cd50700 1 -- 172.21.15.3:0/3300418632 >> 172.21.15.3:0/3300418632 conn(0x7f8d980a2aa0 msgr2=0x7f8d980a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:16.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.771+0000 7f8d9cd50700 1 -- 172.21.15.3:0/3300418632 shutdown_connections 2023-10-03T10:18:16.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.771+0000 7f8d9cd50700 1 -- 172.21.15.3:0/3300418632 wait complete. 2023-10-03T10:18:16.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.772+0000 7f8d9cd50700 1 Processor -- start 2023-10-03T10:18:16.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.772+0000 7f8d9cd50700 1 -- start start 2023-10-03T10:18:16.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.773+0000 7f8d9cd50700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d9806fb20 0x7f8d9811a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:16.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.773+0000 7f8d9cd50700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d98130540 0x7f8d9811abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:16.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.773+0000 7f8d9cd50700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d98130e50 0x7f8d9811b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:16.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.773+0000 7f8d9cd50700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9811bb20 con 0x7f8d98130540 2023-10-03T10:18:16.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.773+0000 7f8d9cd50700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9811c280 con 0x7f8d98130e50 2023-10-03T10:18:16.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.773+0000 7f8d9cd50700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9811d4d0 con 0x7f8d9806fb20 2023-10-03T10:18:16.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.773+0000 7f8d95d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d98130540 0x7f8d9811abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:16.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.773+0000 7f8d9659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d9806fb20 0x7f8d9811a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:16.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.773+0000 7f8d95d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d98130540 0x7f8d9811abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59382/0 (socket says 172.21.15.3:59382) 2023-10-03T10:18:16.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.773+0000 7f8d95d9b700 1 -- 172.21.15.3:0/799054058 learned_addr learned my addr 172.21.15.3:0/799054058 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:16.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.773+0000 7f8d96d9d700 1 --2- 172.21.15.3:0/799054058 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d98130e50 0x7f8d9811b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:16.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.774+0000 7f8d95d9b700 1 -- 172.21.15.3:0/799054058 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d9806fb20 msgr2=0x7f8d9811a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:16.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.774+0000 7f8d95d9b700 1 --2- 172.21.15.3:0/799054058 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d9806fb20 0x7f8d9811a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.774+0000 7f8d95d9b700 1 -- 172.21.15.3:0/799054058 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d98130e50 msgr2=0x7f8d9811b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:16.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.774+0000 7f8d95d9b700 1 --2- 172.21.15.3:0/799054058 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d98130e50 0x7f8d9811b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.774+0000 7f8d95d9b700 1 -- 172.21.15.3:0/799054058 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8c0094a0 con 0x7f8d98130540 2023-10-03T10:18:16.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.774+0000 7f8d96d9d700 1 --2- 172.21.15.3:0/799054058 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d98130e50 0x7f8d9811b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:18:16.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.774+0000 7f8d95d9b700 1 --2- 172.21.15.3:0/799054058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d98130540 0x7f8d9811abb0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00ebe0 tx=0x7f8d8c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:16.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.774+0000 7f8d86ffd700 1 -- 172.21.15.3:0/799054058 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c00c910 con 0x7f8d98130540 2023-10-03T10:18:16.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.774+0000 7f8d86ffd700 1 -- 172.21.15.3:0/799054058 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d8c00ca70 con 0x7f8d98130540 2023-10-03T10:18:16.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.774+0000 7f8d86ffd700 1 -- 172.21.15.3:0/799054058 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c00ce00 con 0x7f8d98130540 2023-10-03T10:18:16.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.774+0000 7f8d9cd50700 1 -- 172.21.15.3:0/799054058 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d90009b30 con 0x7f8d98130540 2023-10-03T10:18:16.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.774+0000 7f8d9cd50700 1 -- 172.21.15.3:0/799054058 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d980a08a0 con 0x7f8d98130540 2023-10-03T10:18:16.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.775+0000 7f8d86ffd700 1 -- 172.21.15.3:0/799054058 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8d8c013470 con 0x7f8d98130540 2023-10-03T10:18:16.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.775+0000 7f8d9cd50700 1 -- 172.21.15.3:0/799054058 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d9809cd40 con 0x7f8d98130540 2023-10-03T10:18:16.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.776+0000 7f8d86ffd700 1 -- 172.21.15.3:0/799054058 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8d8c011350 con 0x7f8d98130540 2023-10-03T10:18:16.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.781+0000 7f8d86ffd700 1 -- 172.21.15.3:0/799054058 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8d8c044060 con 0x7f8d98130540 2023-10-03T10:18:16.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.798+0000 7f161375e700 1 Processor -- start 2023-10-03T10:18:16.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.798+0000 7f161375e700 1 -- start start 2023-10-03T10:18:16.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.798+0000 7f161375e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f160c09d6f0 0x7f160c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:16.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.798+0000 7f161375e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f160c09e410 0x7f160c130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:16.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.798+0000 7f161375e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f160c09f130 con 0x7f160c09e000 2023-10-03T10:18:16.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.798+0000 7f161375e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f160c09f520 con 0x7f160c09e410 2023-10-03T10:18:16.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.798+0000 7f161375e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f160c09e930 con 0x7f160c09d6f0 2023-10-03T10:18:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.799+0000 7f16114fa700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f160c09d6f0 0x7f160c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.799+0000 7f1610cf9700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f160c09e000 0x7f160c130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54064/0 (socket says 172.21.15.3:54064) 2023-10-03T10:18:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.799+0000 7f1610cf9700 1 -- 172.21.15.3:0/699281698 learned_addr learned my addr 172.21.15.3:0/699281698 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.799+0000 7f16114fa700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f160c09d6f0 0x7f160c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43494/0 (socket says 172.21.15.3:43494) 2023-10-03T10:18:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.799+0000 7f1611cfb700 1 --2- 172.21.15.3:0/699281698 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f160c09e410 0x7f160c130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.799+0000 7f16114fa700 1 -- 172.21.15.3:0/699281698 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f160c09e410 msgr2=0x7f160c130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.799+0000 7f16114fa700 1 --2- 172.21.15.3:0/699281698 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f160c09e410 0x7f160c130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.799+0000 7f16114fa700 1 -- 172.21.15.3:0/699281698 >> v1:172.21.15.3:6789/0 conn(0x7f160c09e000 legacy=0x7f160c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.799+0000 7f16114fa700 1 -- 172.21.15.3:0/699281698 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f160c09b2d0 con 0x7f160c09d6f0 2023-10-03T10:18:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.799+0000 7f1611cfb700 1 --2- 172.21.15.3:0/699281698 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f160c09e410 0x7f160c130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.799+0000 7f16114fa700 1 --2- 172.21.15.3:0/699281698 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f160c09d6f0 0x7f160c09dac0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1608009870 tx=0x7f1608009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44c90f4fc4f568a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.800+0000 7f1603fff700 1 -- 172.21.15.3:0/699281698 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1608011830 con 0x7f160c09d6f0 2023-10-03T10:18:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.800+0000 7f1603fff700 1 -- 172.21.15.3:0/699281698 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1608011990 con 0x7f160c09d6f0 2023-10-03T10:18:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.800+0000 7f1603fff700 1 -- 172.21.15.3:0/699281698 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1608011d20 con 0x7f160c09d6f0 2023-10-03T10:18:16.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.801+0000 7f161375e700 1 -- 172.21.15.3:0/699281698 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f160c09d6f0 msgr2=0x7f160c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:16.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.801+0000 7f161375e700 1 --2- 172.21.15.3:0/699281698 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f160c09d6f0 0x7f160c09dac0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1608009870 tx=0x7f1608009b80 comp rx=0 tx=0).stop 2023-10-03T10:18:16.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.801+0000 7f161375e700 1 -- 172.21.15.3:0/699281698 shutdown_connections 2023-10-03T10:18:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.801+0000 7f161375e700 1 --2- 172.21.15.3:0/699281698 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f160c09e410 0x7f160c130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.801+0000 7f161375e700 1 --2- 172.21.15.3:0/699281698 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f160c09d6f0 0x7f160c09dac0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.801+0000 7f161375e700 1 -- 172.21.15.3:0/699281698 >> 172.21.15.3:0/699281698 conn(0x7f160c066f50 msgr2=0x7f160c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.801+0000 7f161375e700 1 -- 172.21.15.3:0/699281698 shutdown_connections 2023-10-03T10:18:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.801+0000 7f161375e700 1 -- 172.21.15.3:0/699281698 wait complete. 2023-10-03T10:18:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.802+0000 7f161375e700 1 Processor -- start 2023-10-03T10:18:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.802+0000 7f161375e700 1 -- start start 2023-10-03T10:18:16.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.803+0000 7f161375e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f160c09d6f0 0x7f160c1301d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:16.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.803+0000 7f161375e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f160c09e000 0x7f160c130710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:16.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.803+0000 7f161375e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f160c09e410 0x7f160c130c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:16.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.803+0000 7f161375e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160c13ad10 con 0x7f160c09e410 2023-10-03T10:18:16.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.803+0000 7f161375e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f160c12f770 con 0x7f160c09d6f0 2023-10-03T10:18:16.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.803+0000 7f161375e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f160c13b380 con 0x7f160c09e000 2023-10-03T10:18:16.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.803+0000 7f16114fa700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f160c09d6f0 0x7f160c1301d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:16.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.803+0000 7f1611cfb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f160c09e410 0x7f160c130c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:16.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.803+0000 7f16114fa700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f160c09d6f0 0x7f160c1301d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59756/0 (socket says 172.21.15.3:59756) 2023-10-03T10:18:16.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.803+0000 7f16114fa700 1 -- 172.21.15.3:0/3125799796 learned_addr learned my addr 172.21.15.3:0/3125799796 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:16.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.803+0000 7f1610cf9700 1 --2- 172.21.15.3:0/3125799796 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f160c09e000 0x7f160c130710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:16.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.804+0000 7f16114fa700 1 -- 172.21.15.3:0/3125799796 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f160c09e000 msgr2=0x7f160c130710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:16.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.804+0000 7f16114fa700 1 --2- 172.21.15.3:0/3125799796 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f160c09e000 0x7f160c130710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.804+0000 7f16114fa700 1 -- 172.21.15.3:0/3125799796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f160c09e410 msgr2=0x7f160c130c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.804+0000 7f16114fa700 1 --2- 172.21.15.3:0/3125799796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f160c09e410 0x7f160c130c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.804+0000 7f16114fa700 1 -- 172.21.15.3:0/3125799796 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1604009a70 con 0x7f160c09d6f0 2023-10-03T10:18:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.804+0000 7f1611cfb700 1 --2- 172.21.15.3:0/3125799796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f160c09e410 0x7f160c130c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.804+0000 7f16114fa700 1 --2- 172.21.15.3:0/3125799796 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f160c09d6f0 0x7f160c1301d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f16080090b0 tx=0x7f1608004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.804+0000 7f1601ffb700 1 -- 172.21.15.3:0/3125799796 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1608004450 con 0x7f160c09d6f0 2023-10-03T10:18:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.804+0000 7f1601ffb700 1 -- 172.21.15.3:0/3125799796 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160801f070 con 0x7f160c09d6f0 2023-10-03T10:18:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.804+0000 7f1601ffb700 1 -- 172.21.15.3:0/3125799796 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16080038e0 con 0x7f160c09d6f0 2023-10-03T10:18:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.804+0000 7f161375e700 1 -- 172.21.15.3:0/3125799796 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16080094a0 con 0x7f160c09d6f0 2023-10-03T10:18:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.805+0000 7f161375e700 1 -- 172.21.15.3:0/3125799796 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f160c09c700 con 0x7f160c09d6f0 2023-10-03T10:18:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.806+0000 7f15ef7fe700 1 -- 172.21.15.3:0/3125799796 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15e80052e0 con 0x7f160c09d6f0 2023-10-03T10:18:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.806+0000 7f1601ffb700 1 -- 172.21.15.3:0/3125799796 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1608003a40 con 0x7f160c09d6f0 2023-10-03T10:18:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.807+0000 7f1601ffb700 1 -- 172.21.15.3:0/3125799796 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1608021070 con 0x7f160c09d6f0 2023-10-03T10:18:16.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.815+0000 7f1601ffb700 1 -- 172.21.15.3:0/3125799796 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f16080253e0 con 0x7f160c09d6f0 2023-10-03T10:18:16.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.826+0000 7fd86812f700 1 -- 172.21.15.3:0/936272259 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd8600747f0 con 0x7fd860133930 2023-10-03T10:18:16.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.826+0000 7fd85e7fc700 1 -- 172.21.15.3:0/936272259 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd858017030 con 0x7fd860133930 2023-10-03T10:18:16.827 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:16.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.829+0000 7fd86812f700 1 -- 172.21.15.3:0/936272259 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd860133930 msgr2=0x7fd8601c8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:16.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.829+0000 7fd86812f700 1 --2- 172.21.15.3:0/936272259 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd860133930 0x7fd8601c8ba0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fd858000f60 tx=0x7fd85800c6e0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.829+0000 7fd86812f700 1 -- 172.21.15.3:0/936272259 shutdown_connections 2023-10-03T10:18:16.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.829+0000 7fd86812f700 1 --2- 172.21.15.3:0/936272259 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd860133930 0x7fd8601c8ba0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.829+0000 7fd86812f700 1 --2- 172.21.15.3:0/936272259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd86009d630 0x7fd8601c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.829+0000 7fd86812f700 1 --2- 172.21.15.3:0/936272259 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8600a0a30 0x7fd8601c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.829+0000 7fd86812f700 1 -- 172.21.15.3:0/936272259 >> 172.21.15.3:0/936272259 conn(0x7fd8600a3050 msgr2=0x7fd86009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:16.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.830+0000 7fd86812f700 1 -- 172.21.15.3:0/936272259 shutdown_connections 2023-10-03T10:18:16.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.832+0000 7fd86812f700 1 -- 172.21.15.3:0/936272259 wait complete. 2023-10-03T10:18:16.846 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:16.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.903+0000 7f25b2fa7700 1 -- 172.21.15.3:0/959643227 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f25ac121eb0 con 0x7f25ac09d630 2023-10-03T10:18:16.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.903+0000 7f25a97fa700 1 -- 172.21.15.3:0/959643227 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f259c021020 con 0x7f25ac09d630 2023-10-03T10:18:16.904 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:16.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.906+0000 7f25b2fa7700 1 -- 172.21.15.3:0/959643227 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25ac09d630 msgr2=0x7f25ac1c7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:16.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.906+0000 7f25b2fa7700 1 --2- 172.21.15.3:0/959643227 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25ac09d630 0x7f25ac1c7e80 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f259c00e900 tx=0x7f259c00ec10 comp rx=0 tx=0).stop 2023-10-03T10:18:16.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.906+0000 7f25b2fa7700 1 -- 172.21.15.3:0/959643227 shutdown_connections 2023-10-03T10:18:16.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.906+0000 7f25b2fa7700 1 --2- 172.21.15.3:0/959643227 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f25ac133930 0x7f25ac1c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.906+0000 7f25b2fa7700 1 --2- 172.21.15.3:0/959643227 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f25ac09d630 0x7f25ac1c7e80 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.906+0000 7f25b2fa7700 1 --2- 172.21.15.3:0/959643227 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25ac0a0a30 0x7f25ac1c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.906+0000 7f25b2fa7700 1 -- 172.21.15.3:0/959643227 >> 172.21.15.3:0/959643227 conn(0x7f25ac0a3050 msgr2=0x7f25ac09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:16.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.906+0000 7f25b2fa7700 1 -- 172.21.15.3:0/959643227 shutdown_connections 2023-10-03T10:18:16.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.906+0000 7f25b2fa7700 1 -- 172.21.15.3:0/959643227 wait complete. 2023-10-03T10:18:16.920 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:16.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.995+0000 7f8d9cd50700 1 -- 172.21.15.3:0/799054058 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8d980718e0 con 0x7f8d98130540 2023-10-03T10:18:16.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.995+0000 7f8d86ffd700 1 -- 172.21.15.3:0/799054058 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8d8c017020 con 0x7f8d98130540 2023-10-03T10:18:16.996 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:16.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.997+0000 7f8d9cd50700 1 -- 172.21.15.3:0/799054058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d98130540 msgr2=0x7f8d9811abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:16.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.997+0000 7f8d9cd50700 1 --2- 172.21.15.3:0/799054058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d98130540 0x7f8d9811abb0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00ebe0 tx=0x7f8d8c00eef0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.997+0000 7f8d9cd50700 1 -- 172.21.15.3:0/799054058 shutdown_connections 2023-10-03T10:18:16.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.997+0000 7f8d9cd50700 1 --2- 172.21.15.3:0/799054058 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8d98130e50 0x7f8d9811b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.997+0000 7f8d9cd50700 1 --2- 172.21.15.3:0/799054058 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8d9806fb20 0x7f8d9811a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.997+0000 7f8d9cd50700 1 --2- 172.21.15.3:0/799054058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d98130540 0x7f8d9811abb0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:16.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.997+0000 7f8d9cd50700 1 -- 172.21.15.3:0/799054058 >> 172.21.15.3:0/799054058 conn(0x7f8d980a2aa0 msgr2=0x7f8d9812d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:16.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.997+0000 7f8d9cd50700 1 -- 172.21.15.3:0/799054058 shutdown_connections 2023-10-03T10:18:16.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:16.997+0000 7f8d9cd50700 1 -- 172.21.15.3:0/799054058 wait complete. 2023-10-03T10:18:17.011 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:17.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.024+0000 7f15ef7fe700 1 -- 172.21.15.3:0/3125799796 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f15e8005860 con 0x7f160c09d6f0 2023-10-03T10:18:17.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.024+0000 7f1601ffb700 1 -- 172.21.15.3:0/3125799796 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1608034090 con 0x7f160c09d6f0 2023-10-03T10:18:17.025 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:17.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.027+0000 7f15ef7fe700 1 -- 172.21.15.3:0/3125799796 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f160c09d6f0 msgr2=0x7f160c1301d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:17.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.027+0000 7f15ef7fe700 1 --2- 172.21.15.3:0/3125799796 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f160c09d6f0 0x7f160c1301d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f16080090b0 tx=0x7f1608004750 comp rx=0 tx=0).stop 2023-10-03T10:18:17.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.027+0000 7f15ef7fe700 1 -- 172.21.15.3:0/3125799796 shutdown_connections 2023-10-03T10:18:17.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.027+0000 7f15ef7fe700 1 --2- 172.21.15.3:0/3125799796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f160c09e410 0x7f160c130c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:17.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.027+0000 7f15ef7fe700 1 --2- 172.21.15.3:0/3125799796 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f160c09d6f0 0x7f160c1301d0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:17.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.027+0000 7f15ef7fe700 1 --2- 172.21.15.3:0/3125799796 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f160c09e000 0x7f160c130710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:17.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.027+0000 7f15ef7fe700 1 -- 172.21.15.3:0/3125799796 >> 172.21.15.3:0/3125799796 conn(0x7f160c066f50 msgr2=0x7f160c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:17.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.027+0000 7f15ef7fe700 1 -- 172.21.15.3:0/3125799796 shutdown_connections 2023-10-03T10:18:17.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.027+0000 7f15ef7fe700 1 -- 172.21.15.3:0/3125799796 wait complete. 2023-10-03T10:18:17.040 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:17.847 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:17.921 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:17.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.996+0000 7fb568ae7700 1 Processor -- start 2023-10-03T10:18:17.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.996+0000 7fb568ae7700 1 -- start start 2023-10-03T10:18:17.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.997+0000 7fb568ae7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5640a0a30 0x7fb5640a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:17.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.997+0000 7fb568ae7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5640a1340 0x7fb564133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:17.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.997+0000 7fb568ae7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb564133fa0 0x7fb564134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:17.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.997+0000 7fb568ae7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb56409edd0 con 0x7fb564133fa0 2023-10-03T10:18:17.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.997+0000 7fb568ae7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb56409e1f0 con 0x7fb5640a0a30 2023-10-03T10:18:17.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.997+0000 7fb568ae7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb56409dc50 con 0x7fb5640a1340 2023-10-03T10:18:17.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.997+0000 7fb56259c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5640a0a30 0x7fb5640a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:17.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.997+0000 7fb561d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5640a1340 0x7fb564133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:17.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.997+0000 7fb56259c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5640a0a30 0x7fb5640a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59760/0 (socket says 172.21.15.3:59760) 2023-10-03T10:18:17.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.997+0000 7fb56259c700 1 -- 172.21.15.3:0/972205823 learned_addr learned my addr 172.21.15.3:0/972205823 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:17.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.997+0000 7fb562d9d700 1 --2- 172.21.15.3:0/972205823 >> v2:172.21.15.3:3300/0 conn(0x7fb564133fa0 0x7fb564134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:17.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.997+0000 7fb56259c700 1 -- 172.21.15.3:0/972205823 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5640a1340 msgr2=0x7fb564133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:17.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.997+0000 7fb56259c700 1 --2- 172.21.15.3:0/972205823 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5640a1340 0x7fb564133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:17.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.997+0000 7fb56259c700 1 -- 172.21.15.3:0/972205823 >> v2:172.21.15.3:3300/0 conn(0x7fb564133fa0 msgr2=0x7fb564134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:17.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.997+0000 7fb56259c700 1 --2- 172.21.15.3:0/972205823 >> v2:172.21.15.3:3300/0 conn(0x7fb564133fa0 0x7fb564134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:17.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.997+0000 7fb56259c700 1 -- 172.21.15.3:0/972205823 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5640a1fe0 con 0x7fb5640a0a30 2023-10-03T10:18:17.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.997+0000 7fb562d9d700 1 --2- 172.21.15.3:0/972205823 >> v2:172.21.15.3:3300/0 conn(0x7fb564133fa0 0x7fb564134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:17.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.997+0000 7fb561d9b700 1 --2- 172.21.15.3:0/972205823 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5640a1340 0x7fb564133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:17.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.997+0000 7fb56259c700 1 --2- 172.21.15.3:0/972205823 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5640a0a30 0x7fb5640a0e00 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb55c0098f0 tx=0x7fb55c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ca6171d05d746e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:17.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.998+0000 7fb56159a700 1 -- 172.21.15.3:0/972205823 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c011830 con 0x7fb5640a0a30 2023-10-03T10:18:18.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.998+0000 7fb56159a700 1 -- 172.21.15.3:0/972205823 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55c011990 con 0x7fb5640a0a30 2023-10-03T10:18:18.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.998+0000 7fb56159a700 1 -- 172.21.15.3:0/972205823 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c011d20 con 0x7fb5640a0a30 2023-10-03T10:18:18.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.998+0000 7fb568ae7700 1 -- 172.21.15.3:0/972205823 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5640a0a30 msgr2=0x7fb5640a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:18.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.998+0000 7fb568ae7700 1 --2- 172.21.15.3:0/972205823 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5640a0a30 0x7fb5640a0e00 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb55c0098f0 tx=0x7fb55c009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:18.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.998+0000 7fb568ae7700 1 -- 172.21.15.3:0/972205823 shutdown_connections 2023-10-03T10:18:18.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.998+0000 7fb568ae7700 1 --2- 172.21.15.3:0/972205823 >> v2:172.21.15.3:3300/0 conn(0x7fb564133fa0 0x7fb564134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.998+0000 7fb568ae7700 1 --2- 172.21.15.3:0/972205823 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5640a0a30 0x7fb5640a0e00 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.998+0000 7fb568ae7700 1 --2- 172.21.15.3:0/972205823 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb5640a1340 0x7fb564133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.998+0000 7fb568ae7700 1 -- 172.21.15.3:0/972205823 >> 172.21.15.3:0/972205823 conn(0x7fb5640a6550 msgr2=0x7fb5640a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:18.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.998+0000 7fb568ae7700 1 -- 172.21.15.3:0/972205823 shutdown_connections 2023-10-03T10:18:18.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:17.998+0000 7fb568ae7700 1 -- 172.21.15.3:0/972205823 wait complete. 2023-10-03T10:18:18.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.000+0000 7fb568ae7700 1 Processor -- start 2023-10-03T10:18:18.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.000+0000 7fb568ae7700 1 -- start start 2023-10-03T10:18:18.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.000+0000 7fb568ae7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5640a0a30 0x7fb5641200a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:18.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.000+0000 7fb568ae7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5640a1340 0x7fb5641205e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:18.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.000+0000 7fb568ae7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb564133fa0 0x7fb564120b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:18.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.000+0000 7fb568ae7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb564123bc0 con 0x7fb5640a1340 2023-10-03T10:18:18.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.000+0000 7fb568ae7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb564122f90 con 0x7fb5640a0a30 2023-10-03T10:18:18.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.000+0000 7fb568ae7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb5640a34f0 con 0x7fb564133fa0 2023-10-03T10:18:18.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.000+0000 7fb562d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb564133fa0 0x7fb564120b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:18.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.000+0000 7fb56259c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5640a0a30 0x7fb5641200a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:18.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.000+0000 7fb561d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5640a1340 0x7fb5641205e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:18.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.000+0000 7fb562d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb564133fa0 0x7fb564120b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43506/0 (socket says 172.21.15.3:43506) 2023-10-03T10:18:18.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.000+0000 7fb562d9d700 1 -- 172.21.15.3:0/2269875287 learned_addr learned my addr 172.21.15.3:0/2269875287 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:18.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.000+0000 7fb56259c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5640a0a30 0x7fb5641200a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59770/0 (socket says 172.21.15.3:59770) 2023-10-03T10:18:18.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.001+0000 7fb561d9b700 1 -- 172.21.15.3:0/2269875287 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb564133fa0 msgr2=0x7fb564120b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:18.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.001+0000 7fb561d9b700 1 --2- 172.21.15.3:0/2269875287 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb564133fa0 0x7fb564120b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.001+0000 7fb561d9b700 1 -- 172.21.15.3:0/2269875287 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5640a0a30 msgr2=0x7fb5641200a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:18.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.001+0000 7fb561d9b700 1 --2- 172.21.15.3:0/2269875287 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5640a0a30 0x7fb5641200a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.001+0000 7fb561d9b700 1 -- 172.21.15.3:0/2269875287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54c0094a0 con 0x7fb5640a1340 2023-10-03T10:18:18.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.001+0000 7fb562d9d700 1 --2- 172.21.15.3:0/2269875287 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb564133fa0 0x7fb564120b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:18.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.001+0000 7fb56259c700 1 --2- 172.21.15.3:0/2269875287 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5640a0a30 0x7fb5641200a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:18.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.001+0000 7fb561d9b700 1 --2- 172.21.15.3:0/2269875287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5640a1340 0x7fb5641205e0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00e9b0 tx=0x7fb54c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:18.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.001+0000 7fb55affd700 1 -- 172.21.15.3:0/2269875287 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c00ee50 con 0x7fb5640a1340 2023-10-03T10:18:18.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.001+0000 7fb55affd700 1 -- 172.21.15.3:0/2269875287 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54c00c850 con 0x7fb5640a1340 2023-10-03T10:18:18.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.001+0000 7fb568ae7700 1 -- 172.21.15.3:0/2269875287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb554009a70 con 0x7fb5640a1340 2023-10-03T10:18:18.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.001+0000 7fb55affd700 1 -- 172.21.15.3:0/2269875287 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c00cc00 con 0x7fb5640a1340 2023-10-03T10:18:18.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.001+0000 7fb568ae7700 1 -- 172.21.15.3:0/2269875287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb55c009550 con 0x7fb5640a1340 2023-10-03T10:18:18.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.002+0000 7fb55affd700 1 -- 172.21.15.3:0/2269875287 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb54c013470 con 0x7fb5640a1340 2023-10-03T10:18:18.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.002+0000 7fb55affd700 1 -- 172.21.15.3:0/2269875287 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb54c03fad0 con 0x7fb5640a1340 2023-10-03T10:18:18.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.002+0000 7fb568ae7700 1 -- 172.21.15.3:0/2269875287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb564074ea0 con 0x7fb5640a1340 2023-10-03T10:18:18.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.007+0000 7fb55affd700 1 -- 172.21.15.3:0/2269875287 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb54c013720 con 0x7fb5640a1340 2023-10-03T10:18:18.012 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:18.042 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:18.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.065+0000 7fb97a2a6700 1 Processor -- start 2023-10-03T10:18:18.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.065+0000 7fb97a2a6700 1 -- start start 2023-10-03T10:18:18.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.066+0000 7fb97a2a6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9741338c0 0x7fb974133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:18.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.066+0000 7fb97a2a6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb9741341d0 0x7fb97409de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:18.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.066+0000 7fb97a2a6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb97409e390 0x7fb97409e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:18.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.066+0000 7fb97a2a6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb974070c80 con 0x7fb9741341d0 2023-10-03T10:18:18.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.066+0000 7fb97a2a6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb97409ed20 con 0x7fb9741338c0 2023-10-03T10:18:18.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.066+0000 7fb97a2a6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb97406f720 con 0x7fb97409e390 2023-10-03T10:18:18.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.066+0000 7fb9737fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb9741341d0 0x7fb97409de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:18.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.066+0000 7fb9737fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb9741341d0 0x7fb97409de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59414/0 (socket says 172.21.15.3:59414) 2023-10-03T10:18:18.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.066+0000 7fb9737fe700 1 -- 172.21.15.3:0/3640390167 learned_addr learned my addr 172.21.15.3:0/3640390167 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:18.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.066+0000 7fb973fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9741338c0 0x7fb974133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:18.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.066+0000 7fb978843700 1 --2- 172.21.15.3:0/3640390167 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb97409e390 0x7fb97409e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:18.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.067+0000 7fb973fff700 1 -- 172.21.15.3:0/3640390167 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb97409e390 msgr2=0x7fb97409e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:18.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.067+0000 7fb973fff700 1 --2- 172.21.15.3:0/3640390167 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb97409e390 0x7fb97409e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.067+0000 7fb973fff700 1 -- 172.21.15.3:0/3640390167 >> v2:172.21.15.3:3300/0 conn(0x7fb9741341d0 msgr2=0x7fb97409de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:18.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.067+0000 7fb973fff700 1 --2- 172.21.15.3:0/3640390167 >> v2:172.21.15.3:3300/0 conn(0x7fb9741341d0 0x7fb97409de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.067+0000 7fb973fff700 1 -- 172.21.15.3:0/3640390167 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97406fce0 con 0x7fb9741338c0 2023-10-03T10:18:18.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.067+0000 7fb973fff700 1 --2- 172.21.15.3:0/3640390167 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9741338c0 0x7fb974133c90 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb964009870 tx=0x7fb964009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38a2a9ee9bfae7d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:18.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.067+0000 7fb972ffd700 1 -- 172.21.15.3:0/3640390167 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb964011830 con 0x7fb9741338c0 2023-10-03T10:18:18.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.067+0000 7fb972ffd700 1 -- 172.21.15.3:0/3640390167 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb964011990 con 0x7fb9741338c0 2023-10-03T10:18:18.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.067+0000 7fb972ffd700 1 -- 172.21.15.3:0/3640390167 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb964011d20 con 0x7fb9741338c0 2023-10-03T10:18:18.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.068+0000 7fb97a2a6700 1 -- 172.21.15.3:0/3640390167 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9741338c0 msgr2=0x7fb974133c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:18.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.068+0000 7fb97a2a6700 1 --2- 172.21.15.3:0/3640390167 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9741338c0 0x7fb974133c90 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb964009870 tx=0x7fb964009b80 comp rx=0 tx=0).stop 2023-10-03T10:18:18.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.068+0000 7fb97a2a6700 1 -- 172.21.15.3:0/3640390167 shutdown_connections 2023-10-03T10:18:18.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.068+0000 7fb97a2a6700 1 --2- 172.21.15.3:0/3640390167 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb97409e390 0x7fb97409e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.068+0000 7fb97a2a6700 1 --2- 172.21.15.3:0/3640390167 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9741338c0 0x7fb974133c90 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.068+0000 7fb97a2a6700 1 --2- 172.21.15.3:0/3640390167 >> v2:172.21.15.3:3300/0 conn(0x7fb9741341d0 0x7fb97409de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.068+0000 7fb97a2a6700 1 -- 172.21.15.3:0/3640390167 >> 172.21.15.3:0/3640390167 conn(0x7fb9740a2aa0 msgr2=0x7fb9740a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:18.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.068+0000 7fb97a2a6700 1 -- 172.21.15.3:0/3640390167 shutdown_connections 2023-10-03T10:18:18.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.068+0000 7fb97a2a6700 1 -- 172.21.15.3:0/3640390167 wait complete. 2023-10-03T10:18:18.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.070+0000 7fb97a2a6700 1 Processor -- start 2023-10-03T10:18:18.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.070+0000 7fb97a2a6700 1 -- start start 2023-10-03T10:18:18.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.071+0000 7fb97a2a6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb97409e390 0x7fb97411b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:18.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.071+0000 7fb97a2a6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9741338c0 0x7fb97411bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:18.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.071+0000 7fb97a2a6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9741341d0 0x7fb97411c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:18.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.071+0000 7fb97a2a6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97411d250 con 0x7fb9741341d0 2023-10-03T10:18:18.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.071+0000 7fb97a2a6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb97411e4a0 con 0x7fb9741338c0 2023-10-03T10:18:18.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.071+0000 7fb97a2a6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb97411d830 con 0x7fb97409e390 2023-10-03T10:18:18.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.071+0000 7fb973fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb97409e390 0x7fb97411b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:18.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.071+0000 7fb973fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb97409e390 0x7fb97411b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43518/0 (socket says 172.21.15.3:43518) 2023-10-03T10:18:18.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.071+0000 7fb973fff700 1 -- 172.21.15.3:0/2036226268 learned_addr learned my addr 172.21.15.3:0/2036226268 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:18.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.071+0000 7fb978843700 1 --2- 172.21.15.3:0/2036226268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9741341d0 0x7fb97411c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:18.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.072+0000 7fb973fff700 1 -- 172.21.15.3:0/2036226268 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9741338c0 msgr2=0x7fb97411bbd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:18:18.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.072+0000 7fb973fff700 1 --2- 172.21.15.3:0/2036226268 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9741338c0 0x7fb97411bbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.072+0000 7fb973fff700 1 -- 172.21.15.3:0/2036226268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9741341d0 msgr2=0x7fb97411c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:18.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.072+0000 7fb973fff700 1 --2- 172.21.15.3:0/2036226268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9741341d0 0x7fb97411c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.072+0000 7fb973fff700 1 -- 172.21.15.3:0/2036226268 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9640094a0 con 0x7fb97409e390 2023-10-03T10:18:18.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.072+0000 7fb973fff700 1 --2- 172.21.15.3:0/2036226268 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb97409e390 0x7fb97411b690 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb964000c00 tx=0x7fb964004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:18.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.072+0000 7fb970ff9700 1 -- 172.21.15.3:0/2036226268 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9640036a0 con 0x7fb97409e390 2023-10-03T10:18:18.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.072+0000 7fb970ff9700 1 -- 172.21.15.3:0/2036226268 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb96401f070 con 0x7fb97409e390 2023-10-03T10:18:18.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.072+0000 7fb970ff9700 1 -- 172.21.15.3:0/2036226268 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb964003a20 con 0x7fb97409e390 2023-10-03T10:18:18.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.072+0000 7fb97a2a6700 1 -- 172.21.15.3:0/2036226268 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb97411ee80 con 0x7fb97409e390 2023-10-03T10:18:18.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.072+0000 7fb97a2a6700 1 -- 172.21.15.3:0/2036226268 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb97411e950 con 0x7fb97409e390 2023-10-03T10:18:18.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.073+0000 7fb97a2a6700 1 -- 172.21.15.3:0/2036226268 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9741bacd0 con 0x7fb97409e390 2023-10-03T10:18:18.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.074+0000 7fb970ff9700 1 -- 172.21.15.3:0/2036226268 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb964003b80 con 0x7fb97409e390 2023-10-03T10:18:18.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.074+0000 7fb970ff9700 1 -- 172.21.15.3:0/2036226268 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb964021070 con 0x7fb97409e390 2023-10-03T10:18:18.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.079+0000 7fb970ff9700 1 -- 172.21.15.3:0/2036226268 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb964003e30 con 0x7fb97409e390 2023-10-03T10:18:18.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.158+0000 7f1dd351e700 1 Processor -- start 2023-10-03T10:18:18.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.158+0000 7f1dd351e700 1 -- start start 2023-10-03T10:18:18.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.158+0000 7f1dd351e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1dcc06e570 0x7f1dcc06e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:18.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.158+0000 7f1dd351e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1dcc06ee80 0x7f1dcc134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:18.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.158+0000 7f1dd351e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dcc070ee0 con 0x7f1dcc09eca0 2023-10-03T10:18:18.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.158+0000 7f1dd351e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dcc071480 con 0x7f1dcc06ee80 2023-10-03T10:18:18.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.158+0000 7f1dd351e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc071b90 con 0x7f1dcc06e570 2023-10-03T10:18:18.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.159+0000 7f1dd12ba700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1dcc06e570 0x7f1dcc06e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:18.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.159+0000 7f1dd0ab9700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1dcc09eca0 0x7f1dcc134610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54070/0 (socket says 172.21.15.3:54070) 2023-10-03T10:18:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.159+0000 7f1dd12ba700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1dcc06e570 0x7f1dcc06e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59784/0 (socket says 172.21.15.3:59784) 2023-10-03T10:18:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.159+0000 7f1dd0ab9700 1 -- 172.21.15.3:0/3916491340 learned_addr learned my addr 172.21.15.3:0/3916491340 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.159+0000 7f1dd1abb700 1 --2- 172.21.15.3:0/3916491340 >> v2:172.21.15.3:3300/0 conn(0x7f1dcc06ee80 0x7f1dcc134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.159+0000 7f1dd12ba700 1 -- 172.21.15.3:0/3916491340 >> v2:172.21.15.3:3300/0 conn(0x7f1dcc06ee80 msgr2=0x7f1dcc134d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.159+0000 7f1dd12ba700 1 --2- 172.21.15.3:0/3916491340 >> v2:172.21.15.3:3300/0 conn(0x7f1dcc06ee80 0x7f1dcc134d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.159+0000 7f1dd12ba700 1 -- 172.21.15.3:0/3916491340 >> v1:172.21.15.3:6789/0 conn(0x7f1dcc09eca0 legacy=0x7f1dcc134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.159+0000 7f1dc3fff700 1 -- 172.21.15.3:0/3916491340 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4228744128 0 0) 0x7f1dcc070ee0 con 0x7f1dcc09eca0 2023-10-03T10:18:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.159+0000 7f1dd12ba700 1 -- 172.21.15.3:0/3916491340 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dcc0711a0 con 0x7f1dcc06e570 2023-10-03T10:18:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.159+0000 7f1dd1abb700 1 --2- 172.21.15.3:0/3916491340 >> v2:172.21.15.3:3300/0 conn(0x7f1dcc06ee80 0x7f1dcc134d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.159+0000 7f1dd12ba700 1 --2- 172.21.15.3:0/3916491340 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1dcc06e570 0x7f1dcc06e940 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009840 tx=0x7f1dc8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7b2ad15683ac1ec server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.160+0000 7f1dc3fff700 1 -- 172.21.15.3:0/3916491340 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8011830 con 0x7f1dcc06e570 2023-10-03T10:18:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.160+0000 7f1dc3fff700 1 -- 172.21.15.3:0/3916491340 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc8011990 con 0x7f1dcc06e570 2023-10-03T10:18:18.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.160+0000 7f1dc3fff700 1 -- 172.21.15.3:0/3916491340 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8011d20 con 0x7f1dcc06e570 2023-10-03T10:18:18.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.160+0000 7f1dd351e700 1 -- 172.21.15.3:0/3916491340 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1dcc06e570 msgr2=0x7f1dcc06e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:18.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.160+0000 7f1dd351e700 1 --2- 172.21.15.3:0/3916491340 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1dcc06e570 0x7f1dcc06e940 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009840 tx=0x7f1dc8009b50 comp rx=0 tx=0).stop 2023-10-03T10:18:18.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.160+0000 7f1dd351e700 1 -- 172.21.15.3:0/3916491340 shutdown_connections 2023-10-03T10:18:18.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.160+0000 7f1dd351e700 1 --2- 172.21.15.3:0/3916491340 >> v2:172.21.15.3:3300/0 conn(0x7f1dcc06ee80 0x7f1dcc134d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.160+0000 7f1dd351e700 1 --2- 172.21.15.3:0/3916491340 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1dcc06e570 0x7f1dcc06e940 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.160+0000 7f1dd351e700 1 -- 172.21.15.3:0/3916491340 >> 172.21.15.3:0/3916491340 conn(0x7f1dcc0a2aa0 msgr2=0x7f1dcc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:18.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.161+0000 7f1dd351e700 1 -- 172.21.15.3:0/3916491340 shutdown_connections 2023-10-03T10:18:18.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.161+0000 7f1dd351e700 1 -- 172.21.15.3:0/3916491340 wait complete. 2023-10-03T10:18:18.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.161+0000 7f1dd351e700 1 Processor -- start 2023-10-03T10:18:18.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.162+0000 7f1dd351e700 1 -- start start 2023-10-03T10:18:18.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.162+0000 7f1dd351e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1dcc06e570 0x7f1dcc134860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:18.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.162+0000 7f1dd351e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1dcc06ee80 0x7f1dcc139c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:18.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.162+0000 7f1dd351e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dcc134da0 0x7f1dcc13a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:18.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.162+0000 7f1dd351e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc13b0e0 con 0x7f1dcc134da0 2023-10-03T10:18:18.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.162+0000 7f1dd351e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc1356e0 con 0x7f1dcc06ee80 2023-10-03T10:18:18.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.162+0000 7f1dd351e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc134630 con 0x7f1dcc06e570 2023-10-03T10:18:18.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.162+0000 7f1dd12ba700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1dcc06e570 0x7f1dcc134860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:18.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.162+0000 7f1dd12ba700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1dcc06e570 0x7f1dcc134860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43534/0 (socket says 172.21.15.3:43534) 2023-10-03T10:18:18.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.162+0000 7f1dd12ba700 1 -- 172.21.15.3:0/646662084 learned_addr learned my addr 172.21.15.3:0/646662084 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:18.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.162+0000 7f1dd1abb700 1 --2- 172.21.15.3:0/646662084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dcc134da0 0x7f1dcc13a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:18.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.163+0000 7f1dd1abb700 1 -- 172.21.15.3:0/646662084 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1dcc06e570 msgr2=0x7f1dcc134860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:18.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.163+0000 7f1dd1abb700 1 --2- 172.21.15.3:0/646662084 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1dcc06e570 0x7f1dcc134860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.163+0000 7f1dd1abb700 1 -- 172.21.15.3:0/646662084 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1dcc06ee80 msgr2=0x7f1dcc139c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:18.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.163+0000 7f1dd1abb700 1 --2- 172.21.15.3:0/646662084 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1dcc06ee80 0x7f1dcc139c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.163+0000 7f1dd1abb700 1 -- 172.21.15.3:0/646662084 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc4009a70 con 0x7f1dcc134da0 2023-10-03T10:18:18.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.163+0000 7f1dd1abb700 1 --2- 172.21.15.3:0/646662084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dcc134da0 0x7f1dcc13a1c0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009e50 tx=0x7f1dc400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:18.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.163+0000 7f1dc1ffb700 1 -- 172.21.15.3:0/646662084 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc400cb40 con 0x7f1dcc134da0 2023-10-03T10:18:18.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.163+0000 7f1dc1ffb700 1 -- 172.21.15.3:0/646662084 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc400f040 con 0x7f1dcc134da0 2023-10-03T10:18:18.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.163+0000 7f1dc1ffb700 1 -- 172.21.15.3:0/646662084 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4003a20 con 0x7f1dcc134da0 2023-10-03T10:18:18.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.163+0000 7f1dd351e700 1 -- 172.21.15.3:0/646662084 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc80094a0 con 0x7f1dcc134da0 2023-10-03T10:18:18.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.163+0000 7f1dd351e700 1 -- 172.21.15.3:0/646662084 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dcc13d380 con 0x7f1dcc134da0 2023-10-03T10:18:18.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.164+0000 7f1dc1ffb700 1 -- 172.21.15.3:0/646662084 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1dc4014480 con 0x7f1dcc134da0 2023-10-03T10:18:18.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.165+0000 7f1dc1ffb700 1 -- 172.21.15.3:0/646662084 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1dc4041530 con 0x7f1dcc134da0 2023-10-03T10:18:18.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.165+0000 7f1dd351e700 1 -- 172.21.15.3:0/646662084 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dcc135240 con 0x7f1dcc134da0 2023-10-03T10:18:18.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.174+0000 7f1dc1ffb700 1 -- 172.21.15.3:0/646662084 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1dcc135240 con 0x7f1dcc134da0 2023-10-03T10:18:18.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.216+0000 7f86c7a42700 1 Processor -- start 2023-10-03T10:18:18.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.217+0000 7f86c7a42700 1 -- start start 2023-10-03T10:18:18.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.217+0000 7f86c7a42700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86c00a0a30 0x7f86c00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:18.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.217+0000 7f86c7a42700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f86c0138d80 0x7f86c013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:18.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.217+0000 7f86c7a42700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86c00a2a70 con 0x7f86c009d630 2023-10-03T10:18:18.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.217+0000 7f86c7a42700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c00a01d0 con 0x7f86c0138d80 2023-10-03T10:18:18.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.217+0000 7f86c7a42700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f86c009e2b0 con 0x7f86c00a0a30 2023-10-03T10:18:18.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.218+0000 7f86c57de700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86c00a0a30 0x7f86c00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:18.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.218+0000 7f86c57de700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86c00a0a30 0x7f86c00a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43544/0 (socket says 172.21.15.3:43544) 2023-10-03T10:18:18.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.218+0000 7f86c57de700 1 -- 172.21.15.3:0/1041615977 learned_addr learned my addr 172.21.15.3:0/1041615977 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:18.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.218+0000 7f86c5fdf700 1 --2- 172.21.15.3:0/1041615977 >> v2:172.21.15.3:3300/0 conn(0x7f86c0138d80 0x7f86c013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:18.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.218+0000 7f86c57de700 1 -- 172.21.15.3:0/1041615977 >> v2:172.21.15.3:3300/0 conn(0x7f86c0138d80 msgr2=0x7f86c013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:18.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.218+0000 7f86c57de700 1 --2- 172.21.15.3:0/1041615977 >> v2:172.21.15.3:3300/0 conn(0x7f86c0138d80 0x7f86c013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.218+0000 7f86c57de700 1 -- 172.21.15.3:0/1041615977 >> v1:172.21.15.3:6789/0 conn(0x7f86c009d630 legacy=0x7f86c00a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:18.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.218+0000 7f86c57de700 1 -- 172.21.15.3:0/1041615977 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c009dc50 con 0x7f86c00a0a30 2023-10-03T10:18:18.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.218+0000 7f86c57de700 1 --2- 172.21.15.3:0/1041615977 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86c00a0a30 0x7f86c00a0e00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0098f0 tx=0x7f86bc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba45ebbf1fdbf01e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:18.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.218+0000 7f86b7fff700 1 -- 172.21.15.3:0/1041615977 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc011830 con 0x7f86c00a0a30 2023-10-03T10:18:18.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.218+0000 7f86b7fff700 1 -- 172.21.15.3:0/1041615977 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86bc011990 con 0x7f86c00a0a30 2023-10-03T10:18:18.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.219+0000 7f86b7fff700 1 -- 172.21.15.3:0/1041615977 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc011d20 con 0x7f86c00a0a30 2023-10-03T10:18:18.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.219+0000 7f86c7a42700 1 -- 172.21.15.3:0/1041615977 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86c00a0a30 msgr2=0x7f86c00a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:18.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.219+0000 7f86c7a42700 1 --2- 172.21.15.3:0/1041615977 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86c00a0a30 0x7f86c00a0e00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0098f0 tx=0x7f86bc009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:18.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.220+0000 7f86c7a42700 1 -- 172.21.15.3:0/1041615977 shutdown_connections 2023-10-03T10:18:18.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.220+0000 7f86c7a42700 1 --2- 172.21.15.3:0/1041615977 >> v2:172.21.15.3:3300/0 conn(0x7f86c0138d80 0x7f86c013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.220+0000 7f86c7a42700 1 --2- 172.21.15.3:0/1041615977 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86c00a0a30 0x7f86c00a0e00 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.220+0000 7f86c7a42700 1 -- 172.21.15.3:0/1041615977 >> 172.21.15.3:0/1041615977 conn(0x7f86c00a3050 msgr2=0x7f86c00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:18.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.220+0000 7f86c7a42700 1 -- 172.21.15.3:0/1041615977 shutdown_connections 2023-10-03T10:18:18.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.220+0000 7f86c7a42700 1 -- 172.21.15.3:0/1041615977 wait complete. 2023-10-03T10:18:18.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.221+0000 7f86c7a42700 1 Processor -- start 2023-10-03T10:18:18.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.221+0000 7f86c7a42700 1 -- start start 2023-10-03T10:18:18.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.222+0000 7f86c7a42700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86c009d630 0x7f86c0120870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:18.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.222+0000 7f86c7a42700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86c00a0a30 0x7f86c0120db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:18.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.222+0000 7f86c7a42700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86c0138d80 0x7f86c01212f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:18.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.222+0000 7f86c7a42700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c0122c80 con 0x7f86c0138d80 2023-10-03T10:18:18.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.222+0000 7f86c7a42700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c01223b0 con 0x7f86c00a0a30 2023-10-03T10:18:18.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.222+0000 7f86c7a42700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f86c013b070 con 0x7f86c009d630 2023-10-03T10:18:18.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.222+0000 7f86c4fdd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86c00a0a30 0x7f86c0120db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:18.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.222+0000 7f86c4fdd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86c00a0a30 0x7f86c0120db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59814/0 (socket says 172.21.15.3:59814) 2023-10-03T10:18:18.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.222+0000 7f86c4fdd700 1 -- 172.21.15.3:0/844752802 learned_addr learned my addr 172.21.15.3:0/844752802 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:18.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.222+0000 7f86c5fdf700 1 --2- 172.21.15.3:0/844752802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86c0138d80 0x7f86c01212f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:18.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.223+0000 7f86c57de700 1 --2- 172.21.15.3:0/844752802 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86c009d630 0x7f86c0120870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:18.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.223+0000 7f86c4fdd700 1 -- 172.21.15.3:0/844752802 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86c009d630 msgr2=0x7f86c0120870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:18.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.223+0000 7f86c4fdd700 1 --2- 172.21.15.3:0/844752802 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86c009d630 0x7f86c0120870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.223+0000 7f86c4fdd700 1 -- 172.21.15.3:0/844752802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86c0138d80 msgr2=0x7f86c01212f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:18.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.223+0000 7f86c4fdd700 1 --2- 172.21.15.3:0/844752802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86c0138d80 0x7f86c01212f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.223+0000 7f86c4fdd700 1 -- 172.21.15.3:0/844752802 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc009550 con 0x7f86c00a0a30 2023-10-03T10:18:18.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.223+0000 7f86c57de700 1 --2- 172.21.15.3:0/844752802 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86c009d630 0x7f86c0120870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:18:18.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.223+0000 7f86c5fdf700 1 --2- 172.21.15.3:0/844752802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86c0138d80 0x7f86c01212f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:18:18.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.223+0000 7f86c4fdd700 1 --2- 172.21.15.3:0/844752802 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86c00a0a30 0x7f86c0120db0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f86b000c4b0 tx=0x7f86b000c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:18.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.223+0000 7f86b5ffb700 1 -- 172.21.15.3:0/844752802 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b00052d0 con 0x7f86c00a0a30 2023-10-03T10:18:18.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.223+0000 7f86b5ffb700 1 -- 172.21.15.3:0/844752802 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b0005430 con 0x7f86c00a0a30 2023-10-03T10:18:18.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.223+0000 7f86b5ffb700 1 -- 172.21.15.3:0/844752802 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b00038c0 con 0x7f86c00a0a30 2023-10-03T10:18:18.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.223+0000 7f86c7a42700 1 -- 172.21.15.3:0/844752802 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c0124320 con 0x7f86c00a0a30 2023-10-03T10:18:18.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.223+0000 7f86c7a42700 1 -- 172.21.15.3:0/844752802 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c0123fe0 con 0x7f86c00a0a30 2023-10-03T10:18:18.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.224+0000 7f86b5ffb700 1 -- 172.21.15.3:0/844752802 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f86b0003a20 con 0x7f86c00a0a30 2023-10-03T10:18:18.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.225+0000 7f86b5ffb700 1 -- 172.21.15.3:0/844752802 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f86b003eb90 con 0x7f86c00a0a30 2023-10-03T10:18:18.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.225+0000 7f86c7a42700 1 -- 172.21.15.3:0/844752802 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c00a0470 con 0x7f86c00a0a30 2023-10-03T10:18:18.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.230+0000 7f86b5ffb700 1 -- 172.21.15.3:0/844752802 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f86b0017070 con 0x7f86c00a0a30 2023-10-03T10:18:18.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.245+0000 7fb568ae7700 1 -- 172.21.15.3:0/2269875287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb5640a28e0 con 0x7fb5640a1340 2023-10-03T10:18:18.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.245+0000 7fb55affd700 1 -- 172.21.15.3:0/2269875287 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb54c011b40 con 0x7fb5640a1340 2023-10-03T10:18:18.245 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:18.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.248+0000 7fb568ae7700 1 -- 172.21.15.3:0/2269875287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5640a1340 msgr2=0x7fb5641205e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:18.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.248+0000 7fb568ae7700 1 --2- 172.21.15.3:0/2269875287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5640a1340 0x7fb5641205e0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00e9b0 tx=0x7fb54c00ecc0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.248+0000 7fb568ae7700 1 -- 172.21.15.3:0/2269875287 shutdown_connections 2023-10-03T10:18:18.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.248+0000 7fb568ae7700 1 --2- 172.21.15.3:0/2269875287 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb564133fa0 0x7fb564120b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.248+0000 7fb568ae7700 1 --2- 172.21.15.3:0/2269875287 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb5640a0a30 0x7fb5641200a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.248+0000 7fb568ae7700 1 --2- 172.21.15.3:0/2269875287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5640a1340 0x7fb5641205e0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.248+0000 7fb568ae7700 1 -- 172.21.15.3:0/2269875287 >> 172.21.15.3:0/2269875287 conn(0x7fb5640a6550 msgr2=0x7fb5641bae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:18.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.248+0000 7fb568ae7700 1 -- 172.21.15.3:0/2269875287 shutdown_connections 2023-10-03T10:18:18.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.249+0000 7fb568ae7700 1 -- 172.21.15.3:0/2269875287 wait complete. 2023-10-03T10:18:18.268 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:18.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.297+0000 7fb97a2a6700 1 -- 172.21.15.3:0/2036226268 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb97411e8f0 con 0x7fb97409e390 2023-10-03T10:18:18.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.297+0000 7fb970ff9700 1 -- 172.21.15.3:0/2036226268 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb964015070 con 0x7fb97409e390 2023-10-03T10:18:18.297 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:18.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.299+0000 7fb97a2a6700 1 -- 172.21.15.3:0/2036226268 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb97409e390 msgr2=0x7fb97411b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:18.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.299+0000 7fb97a2a6700 1 --2- 172.21.15.3:0/2036226268 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb97409e390 0x7fb97411b690 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb964000c00 tx=0x7fb964004750 comp rx=0 tx=0).stop 2023-10-03T10:18:18.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.299+0000 7fb97a2a6700 1 -- 172.21.15.3:0/2036226268 shutdown_connections 2023-10-03T10:18:18.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.299+0000 7fb97a2a6700 1 --2- 172.21.15.3:0/2036226268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9741341d0 0x7fb97411c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.299+0000 7fb97a2a6700 1 --2- 172.21.15.3:0/2036226268 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb97409e390 0x7fb97411b690 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.299+0000 7fb97a2a6700 1 --2- 172.21.15.3:0/2036226268 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb9741338c0 0x7fb97411bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.299+0000 7fb97a2a6700 1 -- 172.21.15.3:0/2036226268 >> 172.21.15.3:0/2036226268 conn(0x7fb9740a2aa0 msgr2=0x7fb974132dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:18.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.299+0000 7fb97a2a6700 1 -- 172.21.15.3:0/2036226268 shutdown_connections 2023-10-03T10:18:18.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.299+0000 7fb97a2a6700 1 -- 172.21.15.3:0/2036226268 wait complete. 2023-10-03T10:18:18.314 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:18.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.388+0000 7f1dd351e700 1 -- 172.21.15.3:0/646662084 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1dcc135240 con 0x7f1dcc134da0 2023-10-03T10:18:18.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.389+0000 7f1dc1ffb700 1 -- 172.21.15.3:0/646662084 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1dcc135240 con 0x7f1dcc134da0 2023-10-03T10:18:18.389 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:18.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.391+0000 7f1dd351e700 1 -- 172.21.15.3:0/646662084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dcc134da0 msgr2=0x7f1dcc13a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:18.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.391+0000 7f1dd351e700 1 --2- 172.21.15.3:0/646662084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dcc134da0 0x7f1dcc13a1c0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009e50 tx=0x7f1dc400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.391+0000 7f1dd351e700 1 -- 172.21.15.3:0/646662084 shutdown_connections 2023-10-03T10:18:18.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.391+0000 7f1dd351e700 1 --2- 172.21.15.3:0/646662084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dcc134da0 0x7f1dcc13a1c0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.391+0000 7f1dd351e700 1 --2- 172.21.15.3:0/646662084 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1dcc06e570 0x7f1dcc134860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.391+0000 7f1dd351e700 1 --2- 172.21.15.3:0/646662084 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1dcc06ee80 0x7f1dcc139c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.391+0000 7f1dd351e700 1 -- 172.21.15.3:0/646662084 >> 172.21.15.3:0/646662084 conn(0x7f1dcc0a2aa0 msgr2=0x7f1dcc0a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:18.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.391+0000 7f1dd351e700 1 -- 172.21.15.3:0/646662084 shutdown_connections 2023-10-03T10:18:18.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.391+0000 7f1dd351e700 1 -- 172.21.15.3:0/646662084 wait complete. 2023-10-03T10:18:18.404 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:18.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.457+0000 7f86c7a42700 1 -- 172.21.15.3:0/844752802 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f86c00747f0 con 0x7f86c00a0a30 2023-10-03T10:18:18.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.457+0000 7f86b5ffb700 1 -- 172.21.15.3:0/844752802 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f86b0017070 con 0x7f86c00a0a30 2023-10-03T10:18:18.458 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:18.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.459+0000 7f86c7a42700 1 -- 172.21.15.3:0/844752802 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86c00a0a30 msgr2=0x7f86c0120db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:18.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.459+0000 7f86c7a42700 1 --2- 172.21.15.3:0/844752802 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86c00a0a30 0x7f86c0120db0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f86b000c4b0 tx=0x7f86b000c870 comp rx=0 tx=0).stop 2023-10-03T10:18:18.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.459+0000 7f86c7a42700 1 -- 172.21.15.3:0/844752802 shutdown_connections 2023-10-03T10:18:18.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.459+0000 7f86c7a42700 1 --2- 172.21.15.3:0/844752802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86c0138d80 0x7f86c01212f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.459+0000 7f86c7a42700 1 --2- 172.21.15.3:0/844752802 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86c009d630 0x7f86c0120870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.459+0000 7f86c7a42700 1 --2- 172.21.15.3:0/844752802 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86c00a0a30 0x7f86c0120db0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:18.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.459+0000 7f86c7a42700 1 -- 172.21.15.3:0/844752802 >> 172.21.15.3:0/844752802 conn(0x7f86c00a3050 msgr2=0x7f86c009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:18.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.460+0000 7f86c7a42700 1 -- 172.21.15.3:0/844752802 shutdown_connections 2023-10-03T10:18:18.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:18.460+0000 7f86c7a42700 1 -- 172.21.15.3:0/844752802 wait complete. 2023-10-03T10:18:18.473 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:19.269 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:19.314 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:19.405 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:19.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.418+0000 7f84954ca700 1 Processor -- start 2023-10-03T10:18:19.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.418+0000 7f84954ca700 1 -- start start 2023-10-03T10:18:19.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.418+0000 7f84954ca700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f849009d6f0 0x7f849009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:19.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.418+0000 7f84954ca700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f849009e410 0x7f8490134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:19.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.418+0000 7f84954ca700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f849009b120 con 0x7f849009e000 2023-10-03T10:18:19.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.418+0000 7f84954ca700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f849006b460 con 0x7f849009e410 2023-10-03T10:18:19.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.418+0000 7f84954ca700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f849009a7e0 con 0x7f849009d6f0 2023-10-03T10:18:19.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.418+0000 7f848effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f849009d6f0 0x7f849009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:19.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.418+0000 7f848f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f849009e410 0x7f8490134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:19.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.418+0000 7f848effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f849009d6f0 0x7f849009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43552/0 (socket says 172.21.15.3:43552) 2023-10-03T10:18:19.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.418+0000 7f848effd700 1 -- 172.21.15.3:0/647135846 learned_addr learned my addr 172.21.15.3:0/647135846 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:19.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.418+0000 7f848f7fe700 1 --2- 172.21.15.3:0/647135846 >> v2:172.21.15.3:3300/0 conn(0x7f849009e410 0x7f8490134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59452/0 (socket says 172.21.15.3:59452) 2023-10-03T10:18:19.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.419+0000 7f848effd700 1 -- 172.21.15.3:0/647135846 >> v2:172.21.15.3:3300/0 conn(0x7f849009e410 msgr2=0x7f8490134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:19.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.419+0000 7f848effd700 1 --2- 172.21.15.3:0/647135846 >> v2:172.21.15.3:3300/0 conn(0x7f849009e410 0x7f8490134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.419+0000 7f848effd700 1 -- 172.21.15.3:0/647135846 >> v1:172.21.15.3:6789/0 conn(0x7f849009e000 legacy=0x7f84901339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:19.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.419+0000 7f848effd700 1 -- 172.21.15.3:0/647135846 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f849009a500 con 0x7f849009d6f0 2023-10-03T10:18:19.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.419+0000 7f848f7fe700 1 --2- 172.21.15.3:0/647135846 >> v2:172.21.15.3:3300/0 conn(0x7f849009e410 0x7f8490134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.419+0000 7f848effd700 1 --2- 172.21.15.3:0/647135846 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f849009d6f0 0x7f849009dac0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8480009920 tx=0x7f8480009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84a8cd74b9f054cb server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.419+0000 7f848dffb700 1 -- 172.21.15.3:0/647135846 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8480011830 con 0x7f849009d6f0 2023-10-03T10:18:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.419+0000 7f848dffb700 1 -- 172.21.15.3:0/647135846 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8480011990 con 0x7f849009d6f0 2023-10-03T10:18:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.419+0000 7f848dffb700 1 -- 172.21.15.3:0/647135846 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8480011d20 con 0x7f849009d6f0 2023-10-03T10:18:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.420+0000 7f84954ca700 1 -- 172.21.15.3:0/647135846 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f849009d6f0 msgr2=0x7f849009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.420+0000 7f84954ca700 1 --2- 172.21.15.3:0/647135846 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f849009d6f0 0x7f849009dac0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8480009920 tx=0x7f8480009c30 comp rx=0 tx=0).stop 2023-10-03T10:18:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.420+0000 7f84954ca700 1 -- 172.21.15.3:0/647135846 shutdown_connections 2023-10-03T10:18:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.420+0000 7f84954ca700 1 --2- 172.21.15.3:0/647135846 >> v2:172.21.15.3:3300/0 conn(0x7f849009e410 0x7f8490134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.420+0000 7f84954ca700 1 --2- 172.21.15.3:0/647135846 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f849009d6f0 0x7f849009dac0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.420+0000 7f84954ca700 1 -- 172.21.15.3:0/647135846 >> 172.21.15.3:0/647135846 conn(0x7f8490066f50 msgr2=0x7f849009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.420+0000 7f84954ca700 1 -- 172.21.15.3:0/647135846 shutdown_connections 2023-10-03T10:18:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.420+0000 7f84954ca700 1 -- 172.21.15.3:0/647135846 wait complete. 2023-10-03T10:18:19.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.422+0000 7f84954ca700 1 Processor -- start 2023-10-03T10:18:19.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.422+0000 7f84954ca700 1 -- start start 2023-10-03T10:18:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.423+0000 7f84954ca700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f849009d6f0 0x7f84901a6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.423+0000 7f84954ca700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f849009e000 0x7f84901a6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.423+0000 7f84954ca700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f849009e410 0x7f84901a6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.423+0000 7f84954ca700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84901a8eb0 con 0x7f849009e410 2023-10-03T10:18:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.423+0000 7f84954ca700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84901a9900 con 0x7f849009e000 2023-10-03T10:18:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.423+0000 7f84954ca700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f84901a89f0 con 0x7f849009d6f0 2023-10-03T10:18:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.423+0000 7f848effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f849009d6f0 0x7f84901a6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.423+0000 7f848f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f849009e410 0x7f84901a6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.423+0000 7f848effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f849009d6f0 0x7f84901a6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43554/0 (socket says 172.21.15.3:43554) 2023-10-03T10:18:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.423+0000 7f848effd700 1 -- 172.21.15.3:0/4100780586 learned_addr learned my addr 172.21.15.3:0/4100780586 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.423+0000 7f848e7fc700 1 --2- 172.21.15.3:0/4100780586 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f849009e000 0x7f84901a6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.424+0000 7f848f7fe700 1 -- 172.21.15.3:0/4100780586 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f849009d6f0 msgr2=0x7f84901a6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:19.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.424+0000 7f848f7fe700 1 --2- 172.21.15.3:0/4100780586 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f849009d6f0 0x7f84901a6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.424+0000 7f848f7fe700 1 -- 172.21.15.3:0/4100780586 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f849009e000 msgr2=0x7f84901a6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:19.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.424+0000 7f848f7fe700 1 --2- 172.21.15.3:0/4100780586 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f849009e000 0x7f84901a6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.424+0000 7f848f7fe700 1 -- 172.21.15.3:0/4100780586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8488009a70 con 0x7f849009e410 2023-10-03T10:18:19.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.424+0000 7f848effd700 1 --2- 172.21.15.3:0/4100780586 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f849009d6f0 0x7f84901a6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:19.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.424+0000 7f848e7fc700 1 --2- 172.21.15.3:0/4100780586 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f849009e000 0x7f84901a6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:18:19.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.424+0000 7f848f7fe700 1 --2- 172.21.15.3:0/4100780586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f849009e410 0x7f84901a6ab0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f8488009e50 tx=0x7f848800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:19.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.424+0000 7f84777fe700 1 -- 172.21.15.3:0/4100780586 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848800cb40 con 0x7f849009e410 2023-10-03T10:18:19.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.424+0000 7f84777fe700 1 -- 172.21.15.3:0/4100780586 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848800f040 con 0x7f849009e410 2023-10-03T10:18:19.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.424+0000 7f84954ca700 1 -- 172.21.15.3:0/4100780586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8480009550 con 0x7f849009e410 2023-10-03T10:18:19.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.424+0000 7f84777fe700 1 -- 172.21.15.3:0/4100780586 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8488003a20 con 0x7f849009e410 2023-10-03T10:18:19.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.424+0000 7f84954ca700 1 -- 172.21.15.3:0/4100780586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84901b94e0 con 0x7f849009e410 2023-10-03T10:18:19.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.426+0000 7f84954ca700 1 -- 172.21.15.3:0/4100780586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f849006e8f0 con 0x7f849009e410 2023-10-03T10:18:19.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.426+0000 7f84777fe700 1 -- 172.21.15.3:0/4100780586 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8488014480 con 0x7f849009e410 2023-10-03T10:18:19.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.426+0000 7f84777fe700 1 -- 172.21.15.3:0/4100780586 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8488040110 con 0x7f849009e410 2023-10-03T10:18:19.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.436+0000 7f84777fe700 1 -- 172.21.15.3:0/4100780586 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8488012930 con 0x7f849009e410 2023-10-03T10:18:19.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.464+0000 7f3381ac9700 1 Processor -- start 2023-10-03T10:18:19.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.464+0000 7f3381ac9700 1 -- start start 2023-10-03T10:18:19.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.465+0000 7f3381ac9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f337c09e930 0x7f337c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:19.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.465+0000 7f3381ac9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f337c09f240 0x7f337c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:19.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.465+0000 7f3381ac9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f337c09f650 0x7f337c131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:19.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.465+0000 7f3381ac9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f337c070940 con 0x7f337c09e930 2023-10-03T10:18:19.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.465+0000 7f3381ac9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f337c06fd60 con 0x7f337c09f240 2023-10-03T10:18:19.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.465+0000 7f3381ac9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f337c06fa80 con 0x7f337c09f650 2023-10-03T10:18:19.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.465+0000 7f337bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f337c09f650 0x7f337c131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:19.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.465+0000 7f337affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f337c09f240 0x7f337c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:19.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.465+0000 7f337affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f337c09f240 0x7f337c1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59846/0 (socket says 172.21.15.3:59846) 2023-10-03T10:18:19.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.465+0000 7f337b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f337c09e930 0x7f337c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:19.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.465+0000 7f337bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f337c09f650 0x7f337c131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43560/0 (socket says 172.21.15.3:43560) 2023-10-03T10:18:19.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.465+0000 7f337affd700 1 -- 172.21.15.3:0/3270505186 learned_addr learned my addr 172.21.15.3:0/3270505186 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:19.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.465+0000 7f337affd700 1 -- 172.21.15.3:0/3270505186 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f337c09f650 msgr2=0x7f337c131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:19.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.465+0000 7f337affd700 1 --2- 172.21.15.3:0/3270505186 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f337c09f650 0x7f337c131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.465+0000 7f337affd700 1 -- 172.21.15.3:0/3270505186 >> v2:172.21.15.3:3300/0 conn(0x7f337c09e930 msgr2=0x7f337c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:19.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.465+0000 7f337affd700 1 --2- 172.21.15.3:0/3270505186 >> v2:172.21.15.3:3300/0 conn(0x7f337c09e930 0x7f337c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.465+0000 7f337affd700 1 -- 172.21.15.3:0/3270505186 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f337c09fb60 con 0x7f337c09f240 2023-10-03T10:18:19.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.465+0000 7f337bfff700 1 --2- 172.21.15.3:0/3270505186 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f337c09f650 0x7f337c131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:19.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.465+0000 7f337b7fe700 1 --2- 172.21.15.3:0/3270505186 >> v2:172.21.15.3:3300/0 conn(0x7f337c09e930 0x7f337c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:19.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.466+0000 7f337affd700 1 --2- 172.21.15.3:0/3270505186 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f337c09f240 0x7f337c1317f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f337000dd10 tx=0x7f337000f330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc72bcd3efce83e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.466+0000 7f337a7fc700 1 -- 172.21.15.3:0/3270505186 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3370012070 con 0x7f337c09f240 2023-10-03T10:18:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.466+0000 7f337a7fc700 1 -- 172.21.15.3:0/3270505186 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337000d2e0 con 0x7f337c09f240 2023-10-03T10:18:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.466+0000 7f337a7fc700 1 -- 172.21.15.3:0/3270505186 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337000fb10 con 0x7f337c09f240 2023-10-03T10:18:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.466+0000 7f3381ac9700 1 -- 172.21.15.3:0/3270505186 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f337c09f240 msgr2=0x7f337c1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.466+0000 7f3381ac9700 1 --2- 172.21.15.3:0/3270505186 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f337c09f240 0x7f337c1317f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f337000dd10 tx=0x7f337000f330 comp rx=0 tx=0).stop 2023-10-03T10:18:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.466+0000 7f3381ac9700 1 -- 172.21.15.3:0/3270505186 shutdown_connections 2023-10-03T10:18:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.466+0000 7f3381ac9700 1 --2- 172.21.15.3:0/3270505186 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f337c09f650 0x7f337c131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.466+0000 7f3381ac9700 1 --2- 172.21.15.3:0/3270505186 >> v2:172.21.15.3:3300/0 conn(0x7f337c09e930 0x7f337c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.466+0000 7f3381ac9700 1 --2- 172.21.15.3:0/3270505186 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f337c09f240 0x7f337c1317f0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.466+0000 7f3381ac9700 1 -- 172.21.15.3:0/3270505186 >> 172.21.15.3:0/3270505186 conn(0x7f337c0a4210 msgr2=0x7f337c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:19.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.467+0000 7f3381ac9700 1 -- 172.21.15.3:0/3270505186 shutdown_connections 2023-10-03T10:18:19.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.467+0000 7f3381ac9700 1 -- 172.21.15.3:0/3270505186 wait complete. 2023-10-03T10:18:19.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.467+0000 7f3381ac9700 1 Processor -- start 2023-10-03T10:18:19.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.468+0000 7f3381ac9700 1 -- start start 2023-10-03T10:18:19.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.468+0000 7f3381ac9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f337c09e930 0x7f337c1a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:19.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.468+0000 7f3381ac9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f337c09f240 0x7f337c1a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:19.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.468+0000 7f3381ac9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f337c09f650 0x7f337c1c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:19.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.468+0000 7f3381ac9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337c1a3ee0 con 0x7f337c09f240 2023-10-03T10:18:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.468+0000 7f3381ac9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f337c0a33f0 con 0x7f337c09e930 2023-10-03T10:18:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.468+0000 7f3381ac9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f337c1a0bf0 con 0x7f337c09f650 2023-10-03T10:18:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.468+0000 7f337affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f337c09f240 0x7f337c1a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.468+0000 7f337b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f337c09e930 0x7f337c1a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.468+0000 7f337affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f337c09f240 0x7f337c1a14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59470/0 (socket says 172.21.15.3:59470) 2023-10-03T10:18:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.468+0000 7f337affd700 1 -- 172.21.15.3:0/3261640732 learned_addr learned my addr 172.21.15.3:0/3261640732 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.468+0000 7f337b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f337c09e930 0x7f337c1a0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59852/0 (socket says 172.21.15.3:59852) 2023-10-03T10:18:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.469+0000 7f337bfff700 1 --2- 172.21.15.3:0/3261640732 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f337c09f650 0x7f337c1c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.469+0000 7f337affd700 1 -- 172.21.15.3:0/3261640732 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f337c09f650 msgr2=0x7f337c1c3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.469+0000 7f337affd700 1 --2- 172.21.15.3:0/3261640732 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f337c09f650 0x7f337c1c3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.469+0000 7f337affd700 1 -- 172.21.15.3:0/3261640732 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f337c09e930 msgr2=0x7f337c1a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.469+0000 7f337affd700 1 --2- 172.21.15.3:0/3261640732 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f337c09e930 0x7f337c1a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.469+0000 7f337affd700 1 -- 172.21.15.3:0/3261640732 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f336c0094a0 con 0x7f337c09f240 2023-10-03T10:18:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.469+0000 7f337b7fe700 1 --2- 172.21.15.3:0/3261640732 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f337c09e930 0x7f337c1a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.469+0000 7f337affd700 1 --2- 172.21.15.3:0/3261640732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f337c09f240 0x7f337c1a14f0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f337000f370 tx=0x7f3370004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.469+0000 7f3363fff700 1 -- 172.21.15.3:0/3261640732 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337000de60 con 0x7f337c09f240 2023-10-03T10:18:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.469+0000 7f3363fff700 1 -- 172.21.15.3:0/3261640732 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33700036a0 con 0x7f337c09f240 2023-10-03T10:18:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.469+0000 7f3363fff700 1 -- 172.21.15.3:0/3261640732 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3370003a30 con 0x7f337c09f240 2023-10-03T10:18:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.469+0000 7f3381ac9700 1 -- 172.21.15.3:0/3261640732 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3374009a70 con 0x7f337c09f240 2023-10-03T10:18:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.469+0000 7f3381ac9700 1 -- 172.21.15.3:0/3261640732 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f337000d9f0 con 0x7f337c09f240 2023-10-03T10:18:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.470+0000 7f3381ac9700 1 -- 172.21.15.3:0/3261640732 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f337c074ea0 con 0x7f337c09f240 2023-10-03T10:18:19.474 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:19.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.470+0000 7f3363fff700 1 -- 172.21.15.3:0/3261640732 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3370003b90 con 0x7f337c09f240 2023-10-03T10:18:19.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.471+0000 7f3363fff700 1 -- 172.21.15.3:0/3261640732 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3370044940 con 0x7f337c09f240 2023-10-03T10:18:19.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.476+0000 7f3363fff700 1 -- 172.21.15.3:0/3261640732 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f337000b450 con 0x7f337c09f240 2023-10-03T10:18:19.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.550+0000 7f6495895700 1 Processor -- start 2023-10-03T10:18:19.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.550+0000 7f6495895700 1 -- start start 2023-10-03T10:18:19.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.551+0000 7f6495895700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6490133b00 0x7f6490133ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:19.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.551+0000 7f6495895700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f649009d940 0x7f649013bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:19.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.551+0000 7f6495895700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6490133900 con 0x7f6490134410 2023-10-03T10:18:19.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.551+0000 7f6495895700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6490099ba0 con 0x7f649009d940 2023-10-03T10:18:19.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.551+0000 7f6495895700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6490099dc0 con 0x7f6490133b00 2023-10-03T10:18:19.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.551+0000 7f648e7fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6490134410 0x7f649009d230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54092/0 (socket says 172.21.15.3:54092) 2023-10-03T10:18:19.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.551+0000 7f648f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f649009d940 0x7f649013bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:19.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.551+0000 7f648e7fc700 1 -- 172.21.15.3:0/493485420 learned_addr learned my addr 172.21.15.3:0/493485420 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:19.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.551+0000 7f648effd700 1 --2- 172.21.15.3:0/493485420 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6490133b00 0x7f6490133ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:19.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.551+0000 7f648f7fe700 1 -- 172.21.15.3:0/493485420 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6490133b00 msgr2=0x7f6490133ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:19.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.551+0000 7f648f7fe700 1 --2- 172.21.15.3:0/493485420 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6490133b00 0x7f6490133ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.551+0000 7f648f7fe700 1 -- 172.21.15.3:0/493485420 >> v1:172.21.15.3:6789/0 conn(0x7f6490134410 legacy=0x7f649009d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:19.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.552+0000 7f648f7fe700 1 -- 172.21.15.3:0/493485420 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6490099680 con 0x7f649009d940 2023-10-03T10:18:19.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.552+0000 7f648dffb700 1 -- 172.21.15.3:0/493485420 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3808456422 0 0) 0x7f6490133900 con 0x7f6490134410 2023-10-03T10:18:19.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.552+0000 7f648f7fe700 1 --2- 172.21.15.3:0/493485420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649009d940 0x7f649013bac0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f6488009840 tx=0x7f6488009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5913e396ed6c444 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:19.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.552+0000 7f648dffb700 1 -- 172.21.15.3:0/493485420 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6488011830 con 0x7f649009d940 2023-10-03T10:18:19.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.552+0000 7f648dffb700 1 -- 172.21.15.3:0/493485420 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6488011990 con 0x7f649009d940 2023-10-03T10:18:19.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.552+0000 7f648dffb700 1 -- 172.21.15.3:0/493485420 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6488011d20 con 0x7f649009d940 2023-10-03T10:18:19.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.553+0000 7f6495895700 1 -- 172.21.15.3:0/493485420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649009d940 msgr2=0x7f649013bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:19.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.553+0000 7f6495895700 1 --2- 172.21.15.3:0/493485420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649009d940 0x7f649013bac0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f6488009840 tx=0x7f6488009b50 comp rx=0 tx=0).stop 2023-10-03T10:18:19.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.553+0000 7f6495895700 1 -- 172.21.15.3:0/493485420 shutdown_connections 2023-10-03T10:18:19.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.553+0000 7f6495895700 1 --2- 172.21.15.3:0/493485420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649009d940 0x7f649013bac0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.553+0000 7f6495895700 1 --2- 172.21.15.3:0/493485420 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6490133b00 0x7f6490133ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.553+0000 7f6495895700 1 -- 172.21.15.3:0/493485420 >> 172.21.15.3:0/493485420 conn(0x7f64900a25a0 msgr2=0x7f64900a29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:19.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.553+0000 7f6495895700 1 -- 172.21.15.3:0/493485420 shutdown_connections 2023-10-03T10:18:19.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.553+0000 7f6495895700 1 -- 172.21.15.3:0/493485420 wait complete. 2023-10-03T10:18:19.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.554+0000 7f6495895700 1 Processor -- start 2023-10-03T10:18:19.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.554+0000 7f6495895700 1 -- start start 2023-10-03T10:18:19.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.555+0000 7f6495895700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649009d940 0x7f649012ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:19.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.555+0000 7f6495895700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6490133b00 0x7f649012e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:19.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.555+0000 7f6495895700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6490134410 0x7f649012e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:19.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.555+0000 7f6495895700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6490130790 con 0x7f649009d940 2023-10-03T10:18:19.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.555+0000 7f6495895700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f649008b1e0 con 0x7f6490134410 2023-10-03T10:18:19.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.555+0000 7f6495895700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f649008b450 con 0x7f6490133b00 2023-10-03T10:18:19.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.555+0000 7f648f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6490134410 0x7f649012e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:19.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.555+0000 7f648effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649009d940 0x7f649012ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:19.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.555+0000 7f648f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6490134410 0x7f649012e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59868/0 (socket says 172.21.15.3:59868) 2023-10-03T10:18:19.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.555+0000 7f648f7fe700 1 -- 172.21.15.3:0/2575212543 learned_addr learned my addr 172.21.15.3:0/2575212543 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:19.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.555+0000 7f648e7fc700 1 --2- 172.21.15.3:0/2575212543 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6490133b00 0x7f649012e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:19.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.556+0000 7f648f7fe700 1 -- 172.21.15.3:0/2575212543 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6490133b00 msgr2=0x7f649012e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:19.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.556+0000 7f648f7fe700 1 --2- 172.21.15.3:0/2575212543 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6490133b00 0x7f649012e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.556+0000 7f648f7fe700 1 -- 172.21.15.3:0/2575212543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649009d940 msgr2=0x7f649012ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:19.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.556+0000 7f648f7fe700 1 --2- 172.21.15.3:0/2575212543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649009d940 0x7f649012ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.556+0000 7f648f7fe700 1 -- 172.21.15.3:0/2575212543 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64880094a0 con 0x7f6490134410 2023-10-03T10:18:19.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.556+0000 7f648f7fe700 1 --2- 172.21.15.3:0/2575212543 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6490134410 0x7f649012e950 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f64880090b0 tx=0x7f6488004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:19.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.556+0000 7f6477fff700 1 -- 172.21.15.3:0/2575212543 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648801aa70 con 0x7f6490134410 2023-10-03T10:18:19.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.556+0000 7f6477fff700 1 -- 172.21.15.3:0/2575212543 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f648801abd0 con 0x7f6490134410 2023-10-03T10:18:19.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.556+0000 7f6495895700 1 -- 172.21.15.3:0/2575212543 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6490099140 con 0x7f6490134410 2023-10-03T10:18:19.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.556+0000 7f6477fff700 1 -- 172.21.15.3:0/2575212543 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64880243f0 con 0x7f6490134410 2023-10-03T10:18:19.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.556+0000 7f6495895700 1 -- 172.21.15.3:0/2575212543 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f649012f990 con 0x7f6490134410 2023-10-03T10:18:19.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.557+0000 7f6477fff700 1 -- 172.21.15.3:0/2575212543 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6488024550 con 0x7f6490134410 2023-10-03T10:18:19.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.557+0000 7f6477fff700 1 -- 172.21.15.3:0/2575212543 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6488040cb0 con 0x7f6490134410 2023-10-03T10:18:19.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.557+0000 7f6495895700 1 -- 172.21.15.3:0/2575212543 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f649012aa10 con 0x7f6490134410 2023-10-03T10:18:19.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.567+0000 7f6477fff700 1 -- 172.21.15.3:0/2575212543 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6488015020 con 0x7f6490134410 2023-10-03T10:18:19.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.646+0000 7fc2991f0700 1 Processor -- start 2023-10-03T10:18:19.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.646+0000 7fc2991f0700 1 -- start start 2023-10-03T10:18:19.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.647+0000 7fc2991f0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc29409e1d0 0x7fc294135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:19.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.647+0000 7fc2991f0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc294135e50 0x7fc294138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:19.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.647+0000 7fc2991f0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc29409f130 con 0x7fc29409d6f0 2023-10-03T10:18:19.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.647+0000 7fc2991f0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc29409f520 con 0x7fc29409e1d0 2023-10-03T10:18:19.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.647+0000 7fc2991f0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc29409e930 con 0x7fc294135e50 2023-10-03T10:18:19.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.647+0000 7fc292d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc29409d6f0 0x7fc29409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54108/0 (socket says 172.21.15.3:54108) 2023-10-03T10:18:19.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.647+0000 7fc292d9d700 1 -- 172.21.15.3:0/464778066 learned_addr learned my addr 172.21.15.3:0/464778066 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:19.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.647+0000 7fc29259c700 1 --2- 172.21.15.3:0/464778066 >> v2:172.21.15.3:3300/0 conn(0x7fc29409e1d0 0x7fc294135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:19.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.647+0000 7fc29359e700 1 --2- 172.21.15.3:0/464778066 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc294135e50 0x7fc294138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:19.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.648+0000 7fc291d9b700 1 -- 172.21.15.3:0/464778066 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3745538211 0 0) 0x7fc29409f130 con 0x7fc29409d6f0 2023-10-03T10:18:19.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.648+0000 7fc291d9b700 1 -- 172.21.15.3:0/464778066 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc27c0079c0 con 0x7fc29409d6f0 2023-10-03T10:18:19.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.648+0000 7fc29359e700 1 -- 172.21.15.3:0/464778066 >> v2:172.21.15.3:3300/0 conn(0x7fc29409e1d0 msgr2=0x7fc294135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:19.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.648+0000 7fc29359e700 1 --2- 172.21.15.3:0/464778066 >> v2:172.21.15.3:3300/0 conn(0x7fc29409e1d0 0x7fc294135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.648+0000 7fc29359e700 1 -- 172.21.15.3:0/464778066 >> v1:172.21.15.3:6789/0 conn(0x7fc29409d6f0 legacy=0x7fc29409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:19.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.648+0000 7fc29359e700 1 -- 172.21.15.3:0/464778066 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc29409b2d0 con 0x7fc294135e50 2023-10-03T10:18:19.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.648+0000 7fc29259c700 1 --2- 172.21.15.3:0/464778066 >> v2:172.21.15.3:3300/0 conn(0x7fc29409e1d0 0x7fc294135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:19.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.648+0000 7fc29359e700 1 --2- 172.21.15.3:0/464778066 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc294135e50 0x7fc294138230 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009d90 tx=0x7fc28c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32c0d0df569f7677 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:19.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.648+0000 7fc291d9b700 1 -- 172.21.15.3:0/464778066 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c00e070 con 0x7fc294135e50 2023-10-03T10:18:19.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.648+0000 7fc291d9b700 1 -- 172.21.15.3:0/464778066 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28c00bb20 con 0x7fc294135e50 2023-10-03T10:18:19.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.648+0000 7fc291d9b700 1 -- 172.21.15.3:0/464778066 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c012430 con 0x7fc294135e50 2023-10-03T10:18:19.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.649+0000 7fc2991f0700 1 -- 172.21.15.3:0/464778066 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc294135e50 msgr2=0x7fc294138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:19.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.649+0000 7fc2991f0700 1 --2- 172.21.15.3:0/464778066 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc294135e50 0x7fc294138230 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009d90 tx=0x7fc28c00b330 comp rx=0 tx=0).stop 2023-10-03T10:18:19.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.649+0000 7fc2991f0700 1 -- 172.21.15.3:0/464778066 shutdown_connections 2023-10-03T10:18:19.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.649+0000 7fc2991f0700 1 --2- 172.21.15.3:0/464778066 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc294135e50 0x7fc294138230 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.649+0000 7fc2991f0700 1 --2- 172.21.15.3:0/464778066 >> v2:172.21.15.3:3300/0 conn(0x7fc29409e1d0 0x7fc294135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.649+0000 7fc2991f0700 1 -- 172.21.15.3:0/464778066 >> 172.21.15.3:0/464778066 conn(0x7fc294066f50 msgr2=0x7fc29409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:19.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.649+0000 7fc2991f0700 1 -- 172.21.15.3:0/464778066 shutdown_connections 2023-10-03T10:18:19.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.649+0000 7fc2991f0700 1 -- 172.21.15.3:0/464778066 wait complete. 2023-10-03T10:18:19.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.651+0000 7fc2991f0700 1 Processor -- start 2023-10-03T10:18:19.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.651+0000 7fc2991f0700 1 -- start start 2023-10-03T10:18:19.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.652+0000 7fc2991f0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc29409d6f0 0x7fc294119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:19.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.652+0000 7fc2991f0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc29409e1d0 0x7fc2941199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:19.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.652+0000 7fc2991f0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc294135e50 0x7fc294119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:19.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.652+0000 7fc2991f0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29409c700 con 0x7fc29409d6f0 2023-10-03T10:18:19.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.652+0000 7fc2991f0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2940a3bd0 con 0x7fc29409e1d0 2023-10-03T10:18:19.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.652+0000 7fc2991f0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc29411c360 con 0x7fc294135e50 2023-10-03T10:18:19.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.652+0000 7fc29259c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc29409e1d0 0x7fc2941199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:19.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.652+0000 7fc29259c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc29409e1d0 0x7fc2941199c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59886/0 (socket says 172.21.15.3:59886) 2023-10-03T10:18:19.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.652+0000 7fc29259c700 1 -- 172.21.15.3:0/3191687341 learned_addr learned my addr 172.21.15.3:0/3191687341 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:19.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.652+0000 7fc29359e700 1 --2- 172.21.15.3:0/3191687341 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc294135e50 0x7fc294119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:19.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.652+0000 7fc292d9d700 1 --2- 172.21.15.3:0/3191687341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc29409d6f0 0x7fc294119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:19.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.653+0000 7fc29359e700 1 -- 172.21.15.3:0/3191687341 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc29409e1d0 msgr2=0x7fc2941199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:19.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.653+0000 7fc29359e700 1 --2- 172.21.15.3:0/3191687341 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc29409e1d0 0x7fc2941199c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.653+0000 7fc29359e700 1 -- 172.21.15.3:0/3191687341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc29409d6f0 msgr2=0x7fc294119480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:19.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.653+0000 7fc29359e700 1 --2- 172.21.15.3:0/3191687341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc29409d6f0 0x7fc294119480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.653+0000 7fc29359e700 1 -- 172.21.15.3:0/3191687341 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2880094a0 con 0x7fc294135e50 2023-10-03T10:18:19.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.653+0000 7fc29359e700 1 --2- 172.21.15.3:0/3191687341 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc294135e50 0x7fc294119f00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fc28c016000 tx=0x7fc28c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:19.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.653+0000 7fc27b7fe700 1 -- 172.21.15.3:0/3191687341 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c012430 con 0x7fc294135e50 2023-10-03T10:18:19.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.653+0000 7fc27b7fe700 1 -- 172.21.15.3:0/3191687341 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28c020070 con 0x7fc294135e50 2023-10-03T10:18:19.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.653+0000 7fc2991f0700 1 -- 172.21.15.3:0/3191687341 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc28c009a70 con 0x7fc294135e50 2023-10-03T10:18:19.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.653+0000 7fc27b7fe700 1 -- 172.21.15.3:0/3191687341 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c003e50 con 0x7fc294135e50 2023-10-03T10:18:19.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.653+0000 7fc2991f0700 1 -- 172.21.15.3:0/3191687341 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc29411b4b0 con 0x7fc294135e50 2023-10-03T10:18:19.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.654+0000 7fc27b7fe700 1 -- 172.21.15.3:0/3191687341 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc28c004040 con 0x7fc294135e50 2023-10-03T10:18:19.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.655+0000 7fc2991f0700 1 -- 172.21.15.3:0/3191687341 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc29411bf10 con 0x7fc294135e50 2023-10-03T10:18:19.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.655+0000 7fc27b7fe700 1 -- 172.21.15.3:0/3191687341 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc28c022070 con 0x7fc294135e50 2023-10-03T10:18:19.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.662+0000 7f84954ca700 1 -- 172.21.15.3:0/4100780586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8490074ea0 con 0x7f849009e410 2023-10-03T10:18:19.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.662+0000 7f84777fe700 1 -- 172.21.15.3:0/4100780586 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8488012340 con 0x7f849009e410 2023-10-03T10:18:19.663 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:19.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.664+0000 7fc27b7fe700 1 -- 172.21.15.3:0/3191687341 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc28c011750 con 0x7fc294135e50 2023-10-03T10:18:19.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.666+0000 7f84954ca700 1 -- 172.21.15.3:0/4100780586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f849009e410 msgr2=0x7f84901a6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:19.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.666+0000 7f84954ca700 1 --2- 172.21.15.3:0/4100780586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f849009e410 0x7f84901a6ab0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f8488009e50 tx=0x7f848800c3a0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.666+0000 7f84954ca700 1 -- 172.21.15.3:0/4100780586 shutdown_connections 2023-10-03T10:18:19.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.666+0000 7f84954ca700 1 --2- 172.21.15.3:0/4100780586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f849009e410 0x7f84901a6ab0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.666+0000 7f84954ca700 1 --2- 172.21.15.3:0/4100780586 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f849009d6f0 0x7f84901a6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.666+0000 7f84954ca700 1 --2- 172.21.15.3:0/4100780586 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f849009e000 0x7f84901a6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.666+0000 7f84954ca700 1 -- 172.21.15.3:0/4100780586 >> 172.21.15.3:0/4100780586 conn(0x7f8490066f50 msgr2=0x7f84901badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:19.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.666+0000 7f84954ca700 1 -- 172.21.15.3:0/4100780586 shutdown_connections 2023-10-03T10:18:19.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.666+0000 7f84954ca700 1 -- 172.21.15.3:0/4100780586 wait complete. 2023-10-03T10:18:19.685 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:19.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.741+0000 7f3381ac9700 1 -- 172.21.15.3:0/3261640732 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f337c1a4020 con 0x7f337c09f240 2023-10-03T10:18:19.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.741+0000 7f3363fff700 1 -- 172.21.15.3:0/3261640732 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f337001fb90 con 0x7f337c09f240 2023-10-03T10:18:19.741 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:19.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.743+0000 7f3381ac9700 1 -- 172.21.15.3:0/3261640732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f337c09f240 msgr2=0x7f337c1a14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:19.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.743+0000 7f3381ac9700 1 --2- 172.21.15.3:0/3261640732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f337c09f240 0x7f337c1a14f0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f337000f370 tx=0x7f3370004750 comp rx=0 tx=0).stop 2023-10-03T10:18:19.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.743+0000 7f3381ac9700 1 -- 172.21.15.3:0/3261640732 shutdown_connections 2023-10-03T10:18:19.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.743+0000 7f3381ac9700 1 --2- 172.21.15.3:0/3261640732 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f337c09f650 0x7f337c1c3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.743+0000 7f3381ac9700 1 --2- 172.21.15.3:0/3261640732 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f337c09e930 0x7f337c1a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.743+0000 7f3381ac9700 1 --2- 172.21.15.3:0/3261640732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f337c09f240 0x7f337c1a14f0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.743+0000 7f3381ac9700 1 -- 172.21.15.3:0/3261640732 >> 172.21.15.3:0/3261640732 conn(0x7f337c0a4210 msgr2=0x7f337c06c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:19.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.744+0000 7f3381ac9700 1 -- 172.21.15.3:0/3261640732 shutdown_connections 2023-10-03T10:18:19.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.744+0000 7f3381ac9700 1 -- 172.21.15.3:0/3261640732 wait complete. 2023-10-03T10:18:19.758 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:19.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.782+0000 7f6495895700 1 -- 172.21.15.3:0/2575212543 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f649006ca70 con 0x7f6490134410 2023-10-03T10:18:19.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.782+0000 7f6477fff700 1 -- 172.21.15.3:0/2575212543 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6488015020 con 0x7f6490134410 2023-10-03T10:18:19.782 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:19.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.784+0000 7f6495895700 1 -- 172.21.15.3:0/2575212543 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6490134410 msgr2=0x7f649012e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:19.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.784+0000 7f6495895700 1 --2- 172.21.15.3:0/2575212543 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6490134410 0x7f649012e950 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f64880090b0 tx=0x7f6488004750 comp rx=0 tx=0).stop 2023-10-03T10:18:19.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.784+0000 7f6495895700 1 -- 172.21.15.3:0/2575212543 shutdown_connections 2023-10-03T10:18:19.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.784+0000 7f6495895700 1 --2- 172.21.15.3:0/2575212543 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6490134410 0x7f649012e950 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.784+0000 7f6495895700 1 --2- 172.21.15.3:0/2575212543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649009d940 0x7f649012ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.784+0000 7f6495895700 1 --2- 172.21.15.3:0/2575212543 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6490133b00 0x7f649012e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.784+0000 7f6495895700 1 -- 172.21.15.3:0/2575212543 >> 172.21.15.3:0/2575212543 conn(0x7f64900a25a0 msgr2=0x7f64900a0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:19.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.784+0000 7f6495895700 1 -- 172.21.15.3:0/2575212543 shutdown_connections 2023-10-03T10:18:19.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.784+0000 7f6495895700 1 -- 172.21.15.3:0/2575212543 wait complete. 2023-10-03T10:18:19.803 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:19.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.880+0000 7fc2991f0700 1 -- 172.21.15.3:0/3191687341 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc294074ea0 con 0x7fc294135e50 2023-10-03T10:18:19.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.880+0000 7fc27b7fe700 1 -- 172.21.15.3:0/3191687341 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc28c0208e0 con 0x7fc294135e50 2023-10-03T10:18:19.881 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:19.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.883+0000 7fc2991f0700 1 -- 172.21.15.3:0/3191687341 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc294135e50 msgr2=0x7fc294119f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:19.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.883+0000 7fc2991f0700 1 --2- 172.21.15.3:0/3191687341 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc294135e50 0x7fc294119f00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fc28c016000 tx=0x7fc28c004ad0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.883+0000 7fc2991f0700 1 -- 172.21.15.3:0/3191687341 shutdown_connections 2023-10-03T10:18:19.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.883+0000 7fc2991f0700 1 --2- 172.21.15.3:0/3191687341 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc294135e50 0x7fc294119f00 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.883+0000 7fc2991f0700 1 --2- 172.21.15.3:0/3191687341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc29409d6f0 0x7fc294119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.883+0000 7fc2991f0700 1 --2- 172.21.15.3:0/3191687341 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc29409e1d0 0x7fc2941199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:19.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.883+0000 7fc2991f0700 1 -- 172.21.15.3:0/3191687341 >> 172.21.15.3:0/3191687341 conn(0x7fc294066f50 msgr2=0x7fc294132780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:19.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.883+0000 7fc2991f0700 1 -- 172.21.15.3:0/3191687341 shutdown_connections 2023-10-03T10:18:19.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:19.883+0000 7fc2991f0700 1 -- 172.21.15.3:0/3191687341 wait complete. 2023-10-03T10:18:19.896 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:20.686 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:20.758 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:20.804 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:20.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.836+0000 7f0bac30a700 1 Processor -- start 2023-10-03T10:18:20.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.836+0000 7f0bac30a700 1 -- start start 2023-10-03T10:18:20.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.837+0000 7f0bac30a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0ba4130540 0x7f0ba4130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:20.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.837+0000 7f0bac30a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ba406fcf0 0x7f0ba4138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:20.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.837+0000 7f0bac30a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ba40716a0 con 0x7f0ba4130e50 2023-10-03T10:18:20.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.837+0000 7f0bac30a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba409a390 con 0x7f0ba4130540 2023-10-03T10:18:20.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.837+0000 7f0bac30a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba4071290 con 0x7f0ba406fcf0 2023-10-03T10:18:20.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.837+0000 7f0baa0a6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0ba4130540 0x7f0ba4130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:20.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.837+0000 7f0ba98a5700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0ba4130e50 0x7f0ba406f5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54118/0 (socket says 172.21.15.3:54118) 2023-10-03T10:18:20.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.837+0000 7f0ba98a5700 1 -- 172.21.15.3:0/714455639 learned_addr learned my addr 172.21.15.3:0/714455639 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:20.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.837+0000 7f0baa8a7700 1 --2- 172.21.15.3:0/714455639 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ba406fcf0 0x7f0ba4138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:20.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.837+0000 7f0ba90a4700 1 -- 172.21.15.3:0/714455639 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2888101025 0 0) 0x7f0ba40716a0 con 0x7f0ba4130e50 2023-10-03T10:18:20.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.837+0000 7f0baa0a6700 1 -- 172.21.15.3:0/714455639 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ba406fcf0 msgr2=0x7f0ba4138570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:20.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.837+0000 7f0baa0a6700 1 --2- 172.21.15.3:0/714455639 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ba406fcf0 0x7f0ba4138570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:20.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.837+0000 7f0baa0a6700 1 -- 172.21.15.3:0/714455639 >> v1:172.21.15.3:6789/0 conn(0x7f0ba4130e50 legacy=0x7f0ba406f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:20.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.838+0000 7f0baa0a6700 1 -- 172.21.15.3:0/714455639 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba4130340 con 0x7f0ba4130540 2023-10-03T10:18:20.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.838+0000 7f0baa8a7700 1 --2- 172.21.15.3:0/714455639 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ba406fcf0 0x7f0ba4138570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:20.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.838+0000 7f0baa0a6700 1 --2- 172.21.15.3:0/714455639 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0ba4130540 0x7f0ba4130910 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0ba00131b0 tx=0x7f0ba00134c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb9a430ccaea2872 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:20.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.838+0000 7f0ba90a4700 1 -- 172.21.15.3:0/714455639 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0016070 con 0x7f0ba4130540 2023-10-03T10:18:20.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.838+0000 7f0ba90a4700 1 -- 172.21.15.3:0/714455639 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba0013d30 con 0x7f0ba4130540 2023-10-03T10:18:20.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.838+0000 7f0ba90a4700 1 -- 172.21.15.3:0/714455639 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0013ea0 con 0x7f0ba4130540 2023-10-03T10:18:20.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.839+0000 7f0bac30a700 1 -- 172.21.15.3:0/714455639 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0ba4130540 msgr2=0x7f0ba4130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:20.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.839+0000 7f0bac30a700 1 --2- 172.21.15.3:0/714455639 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0ba4130540 0x7f0ba4130910 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0ba00131b0 tx=0x7f0ba00134c0 comp rx=0 tx=0).stop 2023-10-03T10:18:20.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.839+0000 7f0bac30a700 1 -- 172.21.15.3:0/714455639 shutdown_connections 2023-10-03T10:18:20.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.839+0000 7f0bac30a700 1 --2- 172.21.15.3:0/714455639 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0ba4130540 0x7f0ba4130910 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:20.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.839+0000 7f0bac30a700 1 --2- 172.21.15.3:0/714455639 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ba406fcf0 0x7f0ba4138570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:20.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.839+0000 7f0bac30a700 1 -- 172.21.15.3:0/714455639 >> 172.21.15.3:0/714455639 conn(0x7f0ba40a2aa0 msgr2=0x7f0ba40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:20.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.839+0000 7f0bac30a700 1 -- 172.21.15.3:0/714455639 shutdown_connections 2023-10-03T10:18:20.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.839+0000 7f0bac30a700 1 -- 172.21.15.3:0/714455639 wait complete. 2023-10-03T10:18:20.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.840+0000 7f0bac30a700 1 Processor -- start 2023-10-03T10:18:20.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.840+0000 7f0bac30a700 1 -- start start 2023-10-03T10:18:20.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.840+0000 7f0bac30a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0ba406fcf0 0x7f0ba411a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:20.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.840+0000 7f0bac30a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ba4130540 0x7f0ba411aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:20.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.840+0000 7f0bac30a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ba4130e50 0x7f0ba411b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:20.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.840+0000 7f0bac30a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba411d410 con 0x7f0ba4130540 2023-10-03T10:18:20.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.840+0000 7f0bac30a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba411cf50 con 0x7f0ba406fcf0 2023-10-03T10:18:20.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.840+0000 7f0bac30a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba411c1f0 con 0x7f0ba4130e50 2023-10-03T10:18:20.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.841+0000 7f0baa0a6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0ba406fcf0 0x7f0ba411a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:20.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.841+0000 7f0baa0a6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0ba406fcf0 0x7f0ba411a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59914/0 (socket says 172.21.15.3:59914) 2023-10-03T10:18:20.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.841+0000 7f0baa0a6700 1 -- 172.21.15.3:0/4113781740 learned_addr learned my addr 172.21.15.3:0/4113781740 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:20.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.841+0000 7f0baa8a7700 1 --2- 172.21.15.3:0/4113781740 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ba4130e50 0x7f0ba411b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:20.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.841+0000 7f0ba98a5700 1 --2- 172.21.15.3:0/4113781740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ba4130540 0x7f0ba411aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:20.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.841+0000 7f0baa0a6700 1 -- 172.21.15.3:0/4113781740 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ba4130e50 msgr2=0x7f0ba411b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:20.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.841+0000 7f0baa0a6700 1 --2- 172.21.15.3:0/4113781740 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ba4130e50 0x7f0ba411b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:20.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.841+0000 7f0baa0a6700 1 -- 172.21.15.3:0/4113781740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ba4130540 msgr2=0x7f0ba411aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:20.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.841+0000 7f0baa0a6700 1 --2- 172.21.15.3:0/4113781740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ba4130540 0x7f0ba411aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:20.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.841+0000 7f0baa0a6700 1 -- 172.21.15.3:0/4113781740 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9c009a70 con 0x7f0ba406fcf0 2023-10-03T10:18:20.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.841+0000 7f0ba98a5700 1 --2- 172.21.15.3:0/4113781740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ba4130540 0x7f0ba411aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:18:20.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.841+0000 7f0baa0a6700 1 --2- 172.21.15.3:0/4113781740 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0ba406fcf0 0x7f0ba411a590 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0000c00 tx=0x7f0ba0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:20.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.841+0000 7f0b9affd700 1 -- 172.21.15.3:0/4113781740 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0013ea0 con 0x7f0ba406fcf0 2023-10-03T10:18:20.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.841+0000 7f0b9affd700 1 -- 172.21.15.3:0/4113781740 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba0024640 con 0x7f0ba406fcf0 2023-10-03T10:18:20.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.842+0000 7f0bac30a700 1 -- 172.21.15.3:0/4113781740 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba0011d40 con 0x7f0ba406fcf0 2023-10-03T10:18:20.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.842+0000 7f0b9affd700 1 -- 172.21.15.3:0/4113781740 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba00249d0 con 0x7f0ba406fcf0 2023-10-03T10:18:20.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.842+0000 7f0bac30a700 1 -- 172.21.15.3:0/4113781740 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba40718e0 con 0x7f0ba406fcf0 2023-10-03T10:18:20.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.843+0000 7f0bac30a700 1 -- 172.21.15.3:0/4113781740 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ba409cd40 con 0x7f0ba406fcf0 2023-10-03T10:18:20.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.843+0000 7f0b9affd700 1 -- 172.21.15.3:0/4113781740 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0ba0024b30 con 0x7f0ba406fcf0 2023-10-03T10:18:20.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.843+0000 7f0b9affd700 1 -- 172.21.15.3:0/4113781740 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0ba00494e0 con 0x7f0ba406fcf0 2023-10-03T10:18:20.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.848+0000 7f0b9affd700 1 -- 172.21.15.3:0/4113781740 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0ba002db60 con 0x7f0ba406fcf0 2023-10-03T10:18:20.897 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:20.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.914+0000 7ff99b070700 1 Processor -- start 2023-10-03T10:18:20.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.915+0000 7ff99b070700 1 -- start start 2023-10-03T10:18:20.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.915+0000 7ff99b070700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff994136710 0x7ff994138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:20.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.915+0000 7ff99b070700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff994139030 0x7ff99413b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:20.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.915+0000 7ff99b070700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff99406ef30 con 0x7ff9941338a0 2023-10-03T10:18:20.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.915+0000 7ff99b070700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff99406fef0 con 0x7ff994136710 2023-10-03T10:18:20.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.915+0000 7ff99b070700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff9941334b0 con 0x7ff994139030 2023-10-03T10:18:20.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.915+0000 7ff99960d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff994139030 0x7ff99413b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:20.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.915+0000 7ff993fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff994136710 0x7ff994138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:20.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.915+0000 7ff99960d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff994139030 0x7ff99413b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43618/0 (socket says 172.21.15.3:43618) 2023-10-03T10:18:20.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.915+0000 7ff99960d700 1 -- 172.21.15.3:0/2011302058 learned_addr learned my addr 172.21.15.3:0/2011302058 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:20.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.916+0000 7ff99960d700 1 -- 172.21.15.3:0/2011302058 >> v2:172.21.15.3:3300/0 conn(0x7ff994136710 msgr2=0x7ff994138af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:20.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.916+0000 7ff99960d700 1 --2- 172.21.15.3:0/2011302058 >> v2:172.21.15.3:3300/0 conn(0x7ff994136710 0x7ff994138af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:20.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.916+0000 7ff99960d700 1 -- 172.21.15.3:0/2011302058 >> v1:172.21.15.3:6789/0 conn(0x7ff9941338a0 legacy=0x7ff994133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:20.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.916+0000 7ff99960d700 1 -- 172.21.15.3:0/2011302058 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99406fc30 con 0x7ff994139030 2023-10-03T10:18:20.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.916+0000 7ff993fff700 1 --2- 172.21.15.3:0/2011302058 >> v2:172.21.15.3:3300/0 conn(0x7ff994136710 0x7ff994138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:20.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.916+0000 7ff99960d700 1 --2- 172.21.15.3:0/2011302058 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff994139030 0x7ff99413b410 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009e50 tx=0x7ff98c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1090aee55352517c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:20.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.916+0000 7ff9937fe700 1 -- 172.21.15.3:0/2011302058 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c00e070 con 0x7ff994139030 2023-10-03T10:18:20.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.916+0000 7ff9937fe700 1 -- 172.21.15.3:0/2011302058 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98c00b910 con 0x7ff994139030 2023-10-03T10:18:20.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.916+0000 7ff9937fe700 1 -- 172.21.15.3:0/2011302058 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c00bcc0 con 0x7ff994139030 2023-10-03T10:18:20.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.917+0000 7ff99b070700 1 -- 172.21.15.3:0/2011302058 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff994139030 msgr2=0x7ff99413b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:20.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.917+0000 7ff99b070700 1 --2- 172.21.15.3:0/2011302058 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff994139030 0x7ff99413b410 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009e50 tx=0x7ff98c00b330 comp rx=0 tx=0).stop 2023-10-03T10:18:20.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.917+0000 7ff99b070700 1 -- 172.21.15.3:0/2011302058 shutdown_connections 2023-10-03T10:18:20.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.917+0000 7ff99b070700 1 --2- 172.21.15.3:0/2011302058 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff994139030 0x7ff99413b410 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:20.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.917+0000 7ff99b070700 1 --2- 172.21.15.3:0/2011302058 >> v2:172.21.15.3:3300/0 conn(0x7ff994136710 0x7ff994138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:20.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.917+0000 7ff99b070700 1 -- 172.21.15.3:0/2011302058 >> 172.21.15.3:0/2011302058 conn(0x7ff9940a2aa0 msgr2=0x7ff9940a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:20.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.917+0000 7ff99b070700 1 -- 172.21.15.3:0/2011302058 shutdown_connections 2023-10-03T10:18:20.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.917+0000 7ff99b070700 1 -- 172.21.15.3:0/2011302058 wait complete. 2023-10-03T10:18:20.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.919+0000 7ff99b070700 1 Processor -- start 2023-10-03T10:18:20.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.919+0000 7ff99b070700 1 -- start start 2023-10-03T10:18:20.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.919+0000 7ff99b070700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9941338a0 0x7ff9941c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:20.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.919+0000 7ff99b070700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff994136710 0x7ff9941c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:20.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.919+0000 7ff99b070700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff994139030 0x7ff994119d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:20.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.920+0000 7ff99b070700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99411bab0 con 0x7ff994136710 2023-10-03T10:18:20.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.920+0000 7ff99b070700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff99411b8c0 con 0x7ff994139030 2023-10-03T10:18:20.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.920+0000 7ff99b070700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff99411d8f0 con 0x7ff9941338a0 2023-10-03T10:18:20.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.920+0000 7ff998e0c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9941338a0 0x7ff9941c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:20.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.920+0000 7ff998e0c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9941338a0 0x7ff9941c85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43626/0 (socket says 172.21.15.3:43626) 2023-10-03T10:18:20.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.920+0000 7ff998e0c700 1 -- 172.21.15.3:0/2791315234 learned_addr learned my addr 172.21.15.3:0/2791315234 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:20.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.920+0000 7ff993fff700 1 --2- 172.21.15.3:0/2791315234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff994136710 0x7ff9941c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:20.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.920+0000 7ff998e0c700 1 -- 172.21.15.3:0/2791315234 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff994139030 msgr2=0x7ff994119d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:20.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.920+0000 7ff998e0c700 1 --2- 172.21.15.3:0/2791315234 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff994139030 0x7ff994119d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:20.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.920+0000 7ff998e0c700 1 -- 172.21.15.3:0/2791315234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff994136710 msgr2=0x7ff9941c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:20.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.920+0000 7ff998e0c700 1 --2- 172.21.15.3:0/2791315234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff994136710 0x7ff9941c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:20.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.920+0000 7ff998e0c700 1 -- 172.21.15.3:0/2791315234 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9800094a0 con 0x7ff9941338a0 2023-10-03T10:18:20.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.920+0000 7ff993fff700 1 --2- 172.21.15.3:0/2791315234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff994136710 0x7ff9941c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:18:20.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.920+0000 7ff998e0c700 1 --2- 172.21.15.3:0/2791315234 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9941338a0 0x7ff9941c85c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff98800c4e0 tx=0x7ff98800c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:20.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.921+0000 7ff9917fa700 1 -- 172.21.15.3:0/2791315234 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9880052d0 con 0x7ff9941338a0 2023-10-03T10:18:20.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.921+0000 7ff9917fa700 1 -- 172.21.15.3:0/2791315234 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff988005430 con 0x7ff9941338a0 2023-10-03T10:18:20.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.921+0000 7ff99b070700 1 -- 172.21.15.3:0/2791315234 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff98c009b30 con 0x7ff9941338a0 2023-10-03T10:18:20.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.921+0000 7ff9917fa700 1 -- 172.21.15.3:0/2791315234 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9880038c0 con 0x7ff9941338a0 2023-10-03T10:18:20.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.921+0000 7ff99b070700 1 -- 172.21.15.3:0/2791315234 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff994138b00 con 0x7ff9941338a0 2023-10-03T10:18:20.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.922+0000 7ff9917fa700 1 -- 172.21.15.3:0/2791315234 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff988003a20 con 0x7ff9941338a0 2023-10-03T10:18:20.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.922+0000 7ff99b070700 1 -- 172.21.15.3:0/2791315234 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff99406f640 con 0x7ff9941338a0 2023-10-03T10:18:20.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.922+0000 7ff9917fa700 1 -- 172.21.15.3:0/2791315234 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff98803ed00 con 0x7ff9941338a0 2023-10-03T10:18:20.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.931+0000 7ff9917fa700 1 -- 172.21.15.3:0/2791315234 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff988018260 con 0x7ff9941338a0 2023-10-03T10:18:20.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.954+0000 7f1120955700 1 Processor -- start 2023-10-03T10:18:20.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.954+0000 7f1120955700 1 -- start start 2023-10-03T10:18:20.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.954+0000 7f1120955700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f111c09d6f0 0x7f111c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:20.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.954+0000 7f1120955700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f111c09e000 0x7f111c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:20.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.955+0000 7f1120955700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f111c09e410 0x7f111c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:20.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.955+0000 7f1120955700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f111c09f520 con 0x7f111c09e410 2023-10-03T10:18:20.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.955+0000 7f1120955700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f111c09f130 con 0x7f111c09d6f0 2023-10-03T10:18:20.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.955+0000 7f1120955700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f111c09e930 con 0x7f111c09e000 2023-10-03T10:18:20.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.955+0000 7f1119d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f111c09e000 0x7f111c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:20.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.955+0000 7f111a59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f111c09d6f0 0x7f111c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:20.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.955+0000 7f1119d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f111c09e000 0x7f111c130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43640/0 (socket says 172.21.15.3:43640) 2023-10-03T10:18:20.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.955+0000 7f1119d9b700 1 -- 172.21.15.3:0/19883561 learned_addr learned my addr 172.21.15.3:0/19883561 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:20.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.955+0000 7f111ad9d700 1 --2- 172.21.15.3:0/19883561 >> v2:172.21.15.3:3300/0 conn(0x7f111c09e410 0x7f111c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:20.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.955+0000 7f1119d9b700 1 -- 172.21.15.3:0/19883561 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f111c09d6f0 msgr2=0x7f111c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:20.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.955+0000 7f1119d9b700 1 --2- 172.21.15.3:0/19883561 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f111c09d6f0 0x7f111c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:20.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.955+0000 7f1119d9b700 1 -- 172.21.15.3:0/19883561 >> v2:172.21.15.3:3300/0 conn(0x7f111c09e410 msgr2=0x7f111c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:20.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.955+0000 7f1119d9b700 1 --2- 172.21.15.3:0/19883561 >> v2:172.21.15.3:3300/0 conn(0x7f111c09e410 0x7f111c130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:20.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.955+0000 7f1119d9b700 1 -- 172.21.15.3:0/19883561 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111c09b2d0 con 0x7f111c09e000 2023-10-03T10:18:20.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.955+0000 7f1119d9b700 1 --2- 172.21.15.3:0/19883561 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f111c09e000 0x7f111c130660 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f111000b9d0 tx=0x7f111000bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73289f18932944f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:20.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.956+0000 7f111959a700 1 -- 172.21.15.3:0/19883561 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111000cc20 con 0x7f111c09e000 2023-10-03T10:18:20.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.956+0000 7f111959a700 1 -- 172.21.15.3:0/19883561 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111000cd80 con 0x7f111c09e000 2023-10-03T10:18:20.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.956+0000 7f111959a700 1 -- 172.21.15.3:0/19883561 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111000f680 con 0x7f111c09e000 2023-10-03T10:18:20.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.956+0000 7f1120955700 1 -- 172.21.15.3:0/19883561 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f111c09e000 msgr2=0x7f111c130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:20.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.956+0000 7f1120955700 1 --2- 172.21.15.3:0/19883561 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f111c09e000 0x7f111c130660 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f111000b9d0 tx=0x7f111000bce0 comp rx=0 tx=0).stop 2023-10-03T10:18:20.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.956+0000 7f1120955700 1 -- 172.21.15.3:0/19883561 shutdown_connections 2023-10-03T10:18:20.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.957+0000 7f1120955700 1 --2- 172.21.15.3:0/19883561 >> v2:172.21.15.3:3300/0 conn(0x7f111c09e410 0x7f111c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:20.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.957+0000 7f1120955700 1 --2- 172.21.15.3:0/19883561 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f111c09d6f0 0x7f111c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:20.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.957+0000 7f1120955700 1 --2- 172.21.15.3:0/19883561 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f111c09e000 0x7f111c130660 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:20.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.957+0000 7f1120955700 1 -- 172.21.15.3:0/19883561 >> 172.21.15.3:0/19883561 conn(0x7f111c066f50 msgr2=0x7f111c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:20.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.957+0000 7f1120955700 1 -- 172.21.15.3:0/19883561 shutdown_connections 2023-10-03T10:18:20.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.957+0000 7f1120955700 1 -- 172.21.15.3:0/19883561 wait complete. 2023-10-03T10:18:20.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.958+0000 7f1120955700 1 Processor -- start 2023-10-03T10:18:20.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.958+0000 7f1120955700 1 -- start start 2023-10-03T10:18:20.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.959+0000 7f1120955700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f111c09d6f0 0x7f111c11e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:20.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.959+0000 7f1120955700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f111c09e000 0x7f111c11e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:20.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.959+0000 7f1120955700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f111c09e410 0x7f111c11eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:20.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.959+0000 7f1120955700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c121bd0 con 0x7f111c09d6f0 2023-10-03T10:18:20.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.959+0000 7f1120955700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f111c0a1d00 con 0x7f111c09e000 2023-10-03T10:18:20.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.959+0000 7f1120955700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f111c09c700 con 0x7f111c09e410 2023-10-03T10:18:20.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.959+0000 7f111ad9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f111c09e410 0x7f111c11eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:20.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.959+0000 7f111a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f111c09d6f0 0x7f111c11e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:20.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.959+0000 7f111ad9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f111c09e410 0x7f111c11eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43644/0 (socket says 172.21.15.3:43644) 2023-10-03T10:18:20.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.959+0000 7f111ad9d700 1 -- 172.21.15.3:0/711072887 learned_addr learned my addr 172.21.15.3:0/711072887 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:20.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.959+0000 7f1119d9b700 1 --2- 172.21.15.3:0/711072887 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f111c09e000 0x7f111c11e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:20.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.960+0000 7f111ad9d700 1 -- 172.21.15.3:0/711072887 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f111c09e000 msgr2=0x7f111c11e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:20.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.960+0000 7f111ad9d700 1 --2- 172.21.15.3:0/711072887 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f111c09e000 0x7f111c11e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:20.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.960+0000 7f111ad9d700 1 -- 172.21.15.3:0/711072887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f111c09d6f0 msgr2=0x7f111c11e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:20.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.960+0000 7f111ad9d700 1 --2- 172.21.15.3:0/711072887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f111c09d6f0 0x7f111c11e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:20.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.960+0000 7f111ad9d700 1 -- 172.21.15.3:0/711072887 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111000b600 con 0x7f111c09e410 2023-10-03T10:18:20.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.960+0000 7f111a59c700 1 --2- 172.21.15.3:0/711072887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f111c09d6f0 0x7f111c11e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:20.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.960+0000 7f111ad9d700 1 --2- 172.21.15.3:0/711072887 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f111c09e410 0x7f111c11eb30 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f11140117b0 tx=0x7f1114011ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:20.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.960+0000 7f110affd700 1 -- 172.21.15.3:0/711072887 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1114013020 con 0x7f111c09e410 2023-10-03T10:18:20.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.960+0000 7f110affd700 1 -- 172.21.15.3:0/711072887 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1114013180 con 0x7f111c09e410 2023-10-03T10:18:20.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.960+0000 7f110affd700 1 -- 172.21.15.3:0/711072887 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11140108a0 con 0x7f111c09e410 2023-10-03T10:18:20.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.960+0000 7f1120955700 1 -- 172.21.15.3:0/711072887 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f111c121630 con 0x7f111c09e410 2023-10-03T10:18:20.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.960+0000 7f1120955700 1 -- 172.21.15.3:0/711072887 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f111c120210 con 0x7f111c09e410 2023-10-03T10:18:20.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.962+0000 7f1120955700 1 -- 172.21.15.3:0/711072887 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f111c074ea0 con 0x7f111c09e410 2023-10-03T10:18:20.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.962+0000 7f110affd700 1 -- 172.21.15.3:0/711072887 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1114010a00 con 0x7f111c09e410 2023-10-03T10:18:20.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.962+0000 7f110affd700 1 -- 172.21.15.3:0/711072887 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1114044af0 con 0x7f111c09e410 2023-10-03T10:18:20.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:20.967+0000 7f110affd700 1 -- 172.21.15.3:0/711072887 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1114046e60 con 0x7f111c09e410 2023-10-03T10:18:21.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.046+0000 7fc7b66b4700 1 Processor -- start 2023-10-03T10:18:21.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.046+0000 7fc7b66b4700 1 -- start start 2023-10-03T10:18:21.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.047+0000 7fc7b66b4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc7b009e850 0x7fc7b009ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:21.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.047+0000 7fc7b66b4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc7b009f160 0x7fc7b0133bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:21.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.047+0000 7fc7b66b4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7b006bf30 con 0x7fc7b009f5b0 2023-10-03T10:18:21.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.047+0000 7fc7b66b4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7b009ffb0 con 0x7fc7b009e850 2023-10-03T10:18:21.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.047+0000 7fc7b66b4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b006cb10 con 0x7fc7b009f160 2023-10-03T10:18:21.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.047+0000 7fc7affff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc7b009e850 0x7fc7b009ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:21.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.047+0000 7fc7affff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc7b009e850 0x7fc7b009ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59578/0 (socket says 172.21.15.3:59578) 2023-10-03T10:18:21.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.047+0000 7fc7affff700 1 -- 172.21.15.3:0/3726291823 learned_addr learned my addr 172.21.15.3:0/3726291823 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:21.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.047+0000 7fc7af7fe700 1 --2- 172.21.15.3:0/3726291823 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc7b009f160 0x7fc7b0133bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:21.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.047+0000 7fc7affff700 1 -- 172.21.15.3:0/3726291823 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc7b009f160 msgr2=0x7fc7b0133bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:21.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.047+0000 7fc7affff700 1 --2- 172.21.15.3:0/3726291823 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc7b009f160 0x7fc7b0133bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:21.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.047+0000 7fc7affff700 1 -- 172.21.15.3:0/3726291823 >> v1:172.21.15.3:6789/0 conn(0x7fc7b009f5b0 legacy=0x7fc7b0134130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:21.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.047+0000 7fc7affff700 1 -- 172.21.15.3:0/3726291823 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b00a0270 con 0x7fc7b009e850 2023-10-03T10:18:21.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.048+0000 7fc7affff700 1 --2- 172.21.15.3:0/3726291823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7b009e850 0x7fc7b009ec20 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fc7a00098f0 tx=0x7fc7a0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c66f8ace72d0aa server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:21.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.048+0000 7fc7aeffd700 1 -- 172.21.15.3:0/3726291823 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0011830 con 0x7fc7b009e850 2023-10-03T10:18:21.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.048+0000 7fc7aeffd700 1 -- 172.21.15.3:0/3726291823 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a0011990 con 0x7fc7b009e850 2023-10-03T10:18:21.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.048+0000 7fc7aeffd700 1 -- 172.21.15.3:0/3726291823 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0011d20 con 0x7fc7b009e850 2023-10-03T10:18:21.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.048+0000 7fc7b66b4700 1 -- 172.21.15.3:0/3726291823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7b009e850 msgr2=0x7fc7b009ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:21.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.048+0000 7fc7b66b4700 1 --2- 172.21.15.3:0/3726291823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7b009e850 0x7fc7b009ec20 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fc7a00098f0 tx=0x7fc7a0009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:21.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.049+0000 7fc7b66b4700 1 -- 172.21.15.3:0/3726291823 shutdown_connections 2023-10-03T10:18:21.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.049+0000 7fc7b66b4700 1 --2- 172.21.15.3:0/3726291823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7b009e850 0x7fc7b009ec20 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:21.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.049+0000 7fc7b66b4700 1 --2- 172.21.15.3:0/3726291823 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc7b009f160 0x7fc7b0133bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:21.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.049+0000 7fc7b66b4700 1 -- 172.21.15.3:0/3726291823 >> 172.21.15.3:0/3726291823 conn(0x7fc7b00a3f70 msgr2=0x7fc7b00a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:21.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.049+0000 7fc7b66b4700 1 -- 172.21.15.3:0/3726291823 shutdown_connections 2023-10-03T10:18:21.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.049+0000 7fc7b66b4700 1 -- 172.21.15.3:0/3726291823 wait complete. 2023-10-03T10:18:21.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.050+0000 7fc7b66b4700 1 Processor -- start 2023-10-03T10:18:21.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.050+0000 7fc7b66b4700 1 -- start start 2023-10-03T10:18:21.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.051+0000 7fc7b66b4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc7b009e850 0x7fc7b01228e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:21.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.051+0000 7fc7b66b4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc7b009f160 0x7fc7b0122e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:21.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.051+0000 7fc7b66b4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7b009f5b0 0x7fc7b0123360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:21.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.051+0000 7fc7b66b4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b0123db0 con 0x7fc7b009f5b0 2023-10-03T10:18:21.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.051+0000 7fc7b66b4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b0126320 con 0x7fc7b009e850 2023-10-03T10:18:21.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.051+0000 7fc7b66b4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b01244b0 con 0x7fc7b009f160 2023-10-03T10:18:21.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.051+0000 7fc7affff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc7b009e850 0x7fc7b01228e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:21.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.051+0000 7fc7affff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc7b009e850 0x7fc7b01228e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59948/0 (socket says 172.21.15.3:59948) 2023-10-03T10:18:21.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.051+0000 7fc7affff700 1 -- 172.21.15.3:0/379144452 learned_addr learned my addr 172.21.15.3:0/379144452 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:21.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.051+0000 7fc7af7fe700 1 --2- 172.21.15.3:0/379144452 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc7b009f160 0x7fc7b0122e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:21.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.051+0000 7fc7b4c51700 1 --2- 172.21.15.3:0/379144452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7b009f5b0 0x7fc7b0123360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:21.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.052+0000 7fc7affff700 1 -- 172.21.15.3:0/379144452 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc7b009f160 msgr2=0x7fc7b0122e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:21.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.052+0000 7fc7affff700 1 --2- 172.21.15.3:0/379144452 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc7b009f160 0x7fc7b0122e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:21.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.052+0000 7fc7affff700 1 -- 172.21.15.3:0/379144452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7b009f5b0 msgr2=0x7fc7b0123360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:21.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.052+0000 7fc7affff700 1 --2- 172.21.15.3:0/379144452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7b009f5b0 0x7fc7b0123360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:21.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.052+0000 7fc7affff700 1 -- 172.21.15.3:0/379144452 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a0009550 con 0x7fc7b009e850 2023-10-03T10:18:21.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.052+0000 7fc7affff700 1 --2- 172.21.15.3:0/379144452 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc7b009e850 0x7fc7b01228e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0009c40 tx=0x7fc7a0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:21.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.052+0000 7fc7acff9700 1 -- 172.21.15.3:0/379144452 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0004590 con 0x7fc7b009e850 2023-10-03T10:18:21.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.052+0000 7fc7acff9700 1 -- 172.21.15.3:0/379144452 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a001e070 con 0x7fc7b009e850 2023-10-03T10:18:21.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.052+0000 7fc7b66b4700 1 -- 172.21.15.3:0/379144452 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b006f930 con 0x7fc7b009e850 2023-10-03T10:18:21.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.052+0000 7fc7acff9700 1 -- 172.21.15.3:0/379144452 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a00038c0 con 0x7fc7b009e850 2023-10-03T10:18:21.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.052+0000 7fc7b66b4700 1 -- 172.21.15.3:0/379144452 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b0125d80 con 0x7fc7b009e850 2023-10-03T10:18:21.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.053+0000 7fc7acff9700 1 -- 172.21.15.3:0/379144452 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc7a0003a20 con 0x7fc7b009e850 2023-10-03T10:18:21.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.053+0000 7fc7b66b4700 1 -- 172.21.15.3:0/379144452 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b0133360 con 0x7fc7b009e850 2023-10-03T10:18:21.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.053+0000 7fc7acff9700 1 -- 172.21.15.3:0/379144452 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc7a000bce0 con 0x7fc7b009e850 2023-10-03T10:18:21.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.058+0000 7fc7acff9700 1 -- 172.21.15.3:0/379144452 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc7a002d7d0 con 0x7fc7b009e850 2023-10-03T10:18:21.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.136+0000 7f0bac30a700 1 -- 172.21.15.3:0/4113781740 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0ba4074ea0 con 0x7f0ba406fcf0 2023-10-03T10:18:21.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.136+0000 7f0b9affd700 1 -- 172.21.15.3:0/4113781740 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0ba0023e40 con 0x7f0ba406fcf0 2023-10-03T10:18:21.136 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:21.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.138+0000 7f0bac30a700 1 -- 172.21.15.3:0/4113781740 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0ba406fcf0 msgr2=0x7f0ba411a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:21.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.138+0000 7f0bac30a700 1 --2- 172.21.15.3:0/4113781740 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0ba406fcf0 0x7f0ba411a590 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0000c00 tx=0x7f0ba0004750 comp rx=0 tx=0).stop 2023-10-03T10:18:21.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.139+0000 7f0bac30a700 1 -- 172.21.15.3:0/4113781740 shutdown_connections 2023-10-03T10:18:21.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.139+0000 7f0bac30a700 1 --2- 172.21.15.3:0/4113781740 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0ba4130e50 0x7f0ba411b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:21.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.139+0000 7f0bac30a700 1 --2- 172.21.15.3:0/4113781740 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0ba406fcf0 0x7f0ba411a590 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:21.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.139+0000 7f0bac30a700 1 --2- 172.21.15.3:0/4113781740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ba4130540 0x7f0ba411aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:21.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.139+0000 7f0bac30a700 1 -- 172.21.15.3:0/4113781740 >> 172.21.15.3:0/4113781740 conn(0x7f0ba40a2aa0 msgr2=0x7f0ba412d570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:21.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.140+0000 7f0bac30a700 1 -- 172.21.15.3:0/4113781740 shutdown_connections 2023-10-03T10:18:21.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.140+0000 7f0bac30a700 1 -- 172.21.15.3:0/4113781740 wait complete. 2023-10-03T10:18:21.153 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:21.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.175+0000 7f1120955700 1 -- 172.21.15.3:0/711072887 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f111c06c360 con 0x7f111c09e410 2023-10-03T10:18:21.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.175+0000 7f110affd700 1 -- 172.21.15.3:0/711072887 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1114010cb0 con 0x7f111c09e410 2023-10-03T10:18:21.176 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:21.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.178+0000 7f1120955700 1 -- 172.21.15.3:0/711072887 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f111c09e410 msgr2=0x7f111c11eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:21.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.178+0000 7f1120955700 1 --2- 172.21.15.3:0/711072887 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f111c09e410 0x7f111c11eb30 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f11140117b0 tx=0x7f1114011ac0 comp rx=0 tx=0).stop 2023-10-03T10:18:21.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.178+0000 7f1120955700 1 -- 172.21.15.3:0/711072887 shutdown_connections 2023-10-03T10:18:21.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.178+0000 7f1120955700 1 --2- 172.21.15.3:0/711072887 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f111c09e410 0x7f111c11eb30 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:21.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.178+0000 7f1120955700 1 --2- 172.21.15.3:0/711072887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f111c09d6f0 0x7f111c11e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:21.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.178+0000 7f1120955700 1 --2- 172.21.15.3:0/711072887 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f111c09e000 0x7f111c11e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:21.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.178+0000 7f1120955700 1 -- 172.21.15.3:0/711072887 >> 172.21.15.3:0/711072887 conn(0x7f111c066f50 msgr2=0x7f111c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:21.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.178+0000 7f1120955700 1 -- 172.21.15.3:0/711072887 shutdown_connections 2023-10-03T10:18:21.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.178+0000 7f1120955700 1 -- 172.21.15.3:0/711072887 wait complete. 2023-10-03T10:18:21.192 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:21.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.206+0000 7ff99b070700 1 -- 172.21.15.3:0/2791315234 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff994074ea0 con 0x7ff9941338a0 2023-10-03T10:18:21.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.207+0000 7ff9917fa700 1 -- 172.21.15.3:0/2791315234 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff988017070 con 0x7ff9941338a0 2023-10-03T10:18:21.207 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:21.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.209+0000 7ff99b070700 1 -- 172.21.15.3:0/2791315234 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9941338a0 msgr2=0x7ff9941c85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:21.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.209+0000 7ff99b070700 1 --2- 172.21.15.3:0/2791315234 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9941338a0 0x7ff9941c85c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff98800c4e0 tx=0x7ff98800c8a0 comp rx=0 tx=0).stop 2023-10-03T10:18:21.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.209+0000 7ff99b070700 1 -- 172.21.15.3:0/2791315234 shutdown_connections 2023-10-03T10:18:21.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.209+0000 7ff99b070700 1 --2- 172.21.15.3:0/2791315234 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff994139030 0x7ff994119d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:21.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.209+0000 7ff99b070700 1 --2- 172.21.15.3:0/2791315234 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9941338a0 0x7ff9941c85c0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:21.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.209+0000 7ff99b070700 1 --2- 172.21.15.3:0/2791315234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff994136710 0x7ff9941c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:21.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.209+0000 7ff99b070700 1 -- 172.21.15.3:0/2791315234 >> 172.21.15.3:0/2791315234 conn(0x7ff9940a2aa0 msgr2=0x7ff994130050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:21.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.209+0000 7ff99b070700 1 -- 172.21.15.3:0/2791315234 shutdown_connections 2023-10-03T10:18:21.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.209+0000 7ff99b070700 1 -- 172.21.15.3:0/2791315234 wait complete. 2023-10-03T10:18:21.223 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:21.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.266+0000 7fc7b66b4700 1 -- 172.21.15.3:0/379144452 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc7b00747f0 con 0x7fc7b009e850 2023-10-03T10:18:21.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.267+0000 7fc7acff9700 1 -- 172.21.15.3:0/379144452 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc7a0032070 con 0x7fc7b009e850 2023-10-03T10:18:21.267 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:21.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.269+0000 7fc7b66b4700 1 -- 172.21.15.3:0/379144452 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc7b009e850 msgr2=0x7fc7b01228e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:21.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.269+0000 7fc7b66b4700 1 --2- 172.21.15.3:0/379144452 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc7b009e850 0x7fc7b01228e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0009c40 tx=0x7fc7a0004750 comp rx=0 tx=0).stop 2023-10-03T10:18:21.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.269+0000 7fc7b66b4700 1 -- 172.21.15.3:0/379144452 shutdown_connections 2023-10-03T10:18:21.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.269+0000 7fc7b66b4700 1 --2- 172.21.15.3:0/379144452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7b009f5b0 0x7fc7b0123360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:21.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.269+0000 7fc7b66b4700 1 --2- 172.21.15.3:0/379144452 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc7b009e850 0x7fc7b01228e0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:21.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.269+0000 7fc7b66b4700 1 --2- 172.21.15.3:0/379144452 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc7b009f160 0x7fc7b0122e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:21.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.269+0000 7fc7b66b4700 1 -- 172.21.15.3:0/379144452 >> 172.21.15.3:0/379144452 conn(0x7fc7b00a3f70 msgr2=0x7fc7b0071190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:21.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.269+0000 7fc7b66b4700 1 -- 172.21.15.3:0/379144452 shutdown_connections 2023-10-03T10:18:21.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:21.269+0000 7fc7b66b4700 1 -- 172.21.15.3:0/379144452 wait complete. 2023-10-03T10:18:21.283 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:22.154 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:22.193 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:22.224 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:22.283 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:22.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.314+0000 7fd53296a700 1 Processor -- start 2023-10-03T10:18:22.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.314+0000 7fd53296a700 1 -- start start 2023-10-03T10:18:22.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.315+0000 7fd53296a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd52c09d6f0 0x7fd52c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:22.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.315+0000 7fd53296a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd52c09e000 0x7fd52c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:22.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.315+0000 7fd53296a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd52c09e410 0x7fd52c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:22.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.315+0000 7fd53296a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd52c09f520 con 0x7fd52c09d6f0 2023-10-03T10:18:22.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.315+0000 7fd53296a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c09f130 con 0x7fd52c09e410 2023-10-03T10:18:22.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.315+0000 7fd53296a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c09e930 con 0x7fd52c09e000 2023-10-03T10:18:22.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.315+0000 7fd52b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd52c09e000 0x7fd52c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:22.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.315+0000 7fd52b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd52c09e000 0x7fd52c130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43660/0 (socket says 172.21.15.3:43660) 2023-10-03T10:18:22.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.315+0000 7fd52b7fe700 1 -- 172.21.15.3:0/3168575764 learned_addr learned my addr 172.21.15.3:0/3168575764 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:22.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.315+0000 7fd530f07700 1 --2- 172.21.15.3:0/3168575764 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd52c09e410 0x7fd52c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:22.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.316+0000 7fd52bfff700 1 --2- 172.21.15.3:0/3168575764 >> v2:172.21.15.3:3300/0 conn(0x7fd52c09d6f0 0x7fd52c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:22.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.316+0000 7fd530f07700 1 -- 172.21.15.3:0/3168575764 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd52c09e000 msgr2=0x7fd52c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:22.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.316+0000 7fd530f07700 1 --2- 172.21.15.3:0/3168575764 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd52c09e000 0x7fd52c130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.316+0000 7fd530f07700 1 -- 172.21.15.3:0/3168575764 >> v2:172.21.15.3:3300/0 conn(0x7fd52c09d6f0 msgr2=0x7fd52c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:22.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.316+0000 7fd530f07700 1 --2- 172.21.15.3:0/3168575764 >> v2:172.21.15.3:3300/0 conn(0x7fd52c09d6f0 0x7fd52c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.316+0000 7fd530f07700 1 -- 172.21.15.3:0/3168575764 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52c09b2d0 con 0x7fd52c09e410 2023-10-03T10:18:22.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.316+0000 7fd52b7fe700 1 --2- 172.21.15.3:0/3168575764 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd52c09e000 0x7fd52c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:22.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.316+0000 7fd530f07700 1 --2- 172.21.15.3:0/3168575764 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd52c09e410 0x7fd52c130ba0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fd524009d90 tx=0x7fd52400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4abea15f27e56ff server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:22.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.316+0000 7fd52affd700 1 -- 172.21.15.3:0/3168575764 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52400e070 con 0x7fd52c09e410 2023-10-03T10:18:22.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.316+0000 7fd52affd700 1 -- 172.21.15.3:0/3168575764 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52400bb20 con 0x7fd52c09e410 2023-10-03T10:18:22.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.316+0000 7fd52affd700 1 -- 172.21.15.3:0/3168575764 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd524012430 con 0x7fd52c09e410 2023-10-03T10:18:22.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.317+0000 7fd53296a700 1 -- 172.21.15.3:0/3168575764 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd52c09e410 msgr2=0x7fd52c130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:22.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.317+0000 7fd53296a700 1 --2- 172.21.15.3:0/3168575764 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd52c09e410 0x7fd52c130ba0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fd524009d90 tx=0x7fd52400b330 comp rx=0 tx=0).stop 2023-10-03T10:18:22.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.317+0000 7fd53296a700 1 -- 172.21.15.3:0/3168575764 shutdown_connections 2023-10-03T10:18:22.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.317+0000 7fd53296a700 1 --2- 172.21.15.3:0/3168575764 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd52c09e410 0x7fd52c130ba0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.317+0000 7fd53296a700 1 --2- 172.21.15.3:0/3168575764 >> v2:172.21.15.3:3300/0 conn(0x7fd52c09d6f0 0x7fd52c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.317+0000 7fd53296a700 1 --2- 172.21.15.3:0/3168575764 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd52c09e000 0x7fd52c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.317+0000 7fd53296a700 1 -- 172.21.15.3:0/3168575764 >> 172.21.15.3:0/3168575764 conn(0x7fd52c066f50 msgr2=0x7fd52c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:22.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.317+0000 7fd53296a700 1 -- 172.21.15.3:0/3168575764 shutdown_connections 2023-10-03T10:18:22.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.317+0000 7fd53296a700 1 -- 172.21.15.3:0/3168575764 wait complete. 2023-10-03T10:18:22.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.318+0000 7fd53296a700 1 Processor -- start 2023-10-03T10:18:22.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.319+0000 7fd53296a700 1 -- start start 2023-10-03T10:18:22.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.319+0000 7fd53296a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd52c09d6f0 0x7fd52c11e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:22.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.319+0000 7fd53296a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd52c09e000 0x7fd52c11e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:22.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.319+0000 7fd53296a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd52c09e410 0x7fd52c11ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:22.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.319+0000 7fd53296a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c11fdb0 con 0x7fd52c09e000 2023-10-03T10:18:22.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.319+0000 7fd53296a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c1214f0 con 0x7fd52c09d6f0 2023-10-03T10:18:22.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.319+0000 7fd53296a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c09c770 con 0x7fd52c09e410 2023-10-03T10:18:22.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.319+0000 7fd52b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd52c09e000 0x7fd52c11e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:22.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.319+0000 7fd52b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd52c09e000 0x7fd52c11e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59596/0 (socket says 172.21.15.3:59596) 2023-10-03T10:18:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.319+0000 7fd52b7fe700 1 -- 172.21.15.3:0/400214771 learned_addr learned my addr 172.21.15.3:0/400214771 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.320+0000 7fd530f07700 1 --2- 172.21.15.3:0/400214771 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd52c09e410 0x7fd52c11ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.320+0000 7fd52b7fe700 1 -- 172.21.15.3:0/400214771 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd52c09e410 msgr2=0x7fd52c11ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.320+0000 7fd52b7fe700 1 --2- 172.21.15.3:0/400214771 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd52c09e410 0x7fd52c11ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.320+0000 7fd52b7fe700 1 -- 172.21.15.3:0/400214771 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd52c09d6f0 msgr2=0x7fd52c11e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:22.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.320+0000 7fd52b7fe700 1 --2- 172.21.15.3:0/400214771 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd52c09d6f0 0x7fd52c11e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.320+0000 7fd52b7fe700 1 -- 172.21.15.3:0/400214771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd520009550 con 0x7fd52c09e000 2023-10-03T10:18:22.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.320+0000 7fd52b7fe700 1 --2- 172.21.15.3:0/400214771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd52c09e000 0x7fd52c11e6e0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fd52000fdb0 tx=0x7fd52000d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:22.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.320+0000 7fd528ff9700 1 -- 172.21.15.3:0/400214771 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52000c070 con 0x7fd52c09e000 2023-10-03T10:18:22.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.320+0000 7fd528ff9700 1 -- 172.21.15.3:0/400214771 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52000dc00 con 0x7fd52c09e000 2023-10-03T10:18:22.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.320+0000 7fd528ff9700 1 -- 172.21.15.3:0/400214771 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52001a480 con 0x7fd52c09e000 2023-10-03T10:18:22.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.320+0000 7fd53296a700 1 -- 172.21.15.3:0/400214771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd524009a70 con 0x7fd52c09e000 2023-10-03T10:18:22.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.320+0000 7fd53296a700 1 -- 172.21.15.3:0/400214771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd52c0a3ad0 con 0x7fd52c09e000 2023-10-03T10:18:22.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.322+0000 7fd53296a700 1 -- 172.21.15.3:0/400214771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd52c11ac20 con 0x7fd52c09e000 2023-10-03T10:18:22.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.322+0000 7fd528ff9700 1 -- 172.21.15.3:0/400214771 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd520011450 con 0x7fd52c09e000 2023-10-03T10:18:22.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.322+0000 7fd528ff9700 1 -- 172.21.15.3:0/400214771 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd520015070 con 0x7fd52c09e000 2023-10-03T10:18:22.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.331+0000 7fd528ff9700 1 -- 172.21.15.3:0/400214771 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd520017920 con 0x7fd52c09e000 2023-10-03T10:18:22.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.362+0000 7feee1fea700 1 Processor -- start 2023-10-03T10:18:22.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.362+0000 7feee1fea700 1 -- start start 2023-10-03T10:18:22.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.363+0000 7feee1fea700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feedc09e930 0x7feedc09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:22.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.363+0000 7feee1fea700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feedc09f240 0x7feedc1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:22.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.363+0000 7feee1fea700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feedc070940 con 0x7feedc09f650 2023-10-03T10:18:22.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.363+0000 7feee1fea700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feedc06fd60 con 0x7feedc09f240 2023-10-03T10:18:22.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.363+0000 7feee1fea700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feedc06fa80 con 0x7feedc09e930 2023-10-03T10:18:22.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.363+0000 7feedb7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feedc09e930 0x7feedc09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:22.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.363+0000 7feedb7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feedc09e930 0x7feedc09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43678/0 (socket says 172.21.15.3:43678) 2023-10-03T10:18:22.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.363+0000 7feedb7fe700 1 -- 172.21.15.3:0/1293405851 learned_addr learned my addr 172.21.15.3:0/1293405851 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:22.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.363+0000 7feed2ffd700 1 --2- 172.21.15.3:0/1293405851 >> v2:172.21.15.3:3300/0 conn(0x7feedc09f240 0x7feedc1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:22.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.363+0000 7feed2ffd700 1 -- 172.21.15.3:0/1293405851 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feedc09e930 msgr2=0x7feedc09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:22.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.363+0000 7feed2ffd700 1 --2- 172.21.15.3:0/1293405851 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feedc09e930 0x7feedc09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.364+0000 7feed2ffd700 1 -- 172.21.15.3:0/1293405851 >> v1:172.21.15.3:6789/0 conn(0x7feedc09f650 legacy=0x7feedc131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:22.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.364+0000 7feedaffd700 1 -- 172.21.15.3:0/1293405851 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390411763 0 0) 0x7feedc070940 con 0x7feedc09f650 2023-10-03T10:18:22.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.364+0000 7feed2ffd700 1 -- 172.21.15.3:0/1293405851 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feedc09fb60 con 0x7feedc09f240 2023-10-03T10:18:22.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.364+0000 7feedb7fe700 1 --2- 172.21.15.3:0/1293405851 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feedc09e930 0x7feedc09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:22.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.364+0000 7feed2ffd700 1 --2- 172.21.15.3:0/1293405851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feedc09f240 0x7feedc1317f0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7feec4009840 tx=0x7feec4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611117554de2e961 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:22.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.364+0000 7feedaffd700 1 -- 172.21.15.3:0/1293405851 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec4011830 con 0x7feedc09f240 2023-10-03T10:18:22.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.364+0000 7feedaffd700 1 -- 172.21.15.3:0/1293405851 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec4011990 con 0x7feedc09f240 2023-10-03T10:18:22.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.364+0000 7feedaffd700 1 -- 172.21.15.3:0/1293405851 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec4011d20 con 0x7feedc09f240 2023-10-03T10:18:22.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.365+0000 7feee1fea700 1 -- 172.21.15.3:0/1293405851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feedc09f240 msgr2=0x7feedc1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:22.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.365+0000 7feee1fea700 1 --2- 172.21.15.3:0/1293405851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feedc09f240 0x7feedc1317f0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7feec4009840 tx=0x7feec4009b50 comp rx=0 tx=0).stop 2023-10-03T10:18:22.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.365+0000 7feee1fea700 1 -- 172.21.15.3:0/1293405851 shutdown_connections 2023-10-03T10:18:22.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.365+0000 7feee1fea700 1 --2- 172.21.15.3:0/1293405851 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feedc09e930 0x7feedc09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.365+0000 7feee1fea700 1 --2- 172.21.15.3:0/1293405851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feedc09f240 0x7feedc1317f0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.365+0000 7feee1fea700 1 -- 172.21.15.3:0/1293405851 >> 172.21.15.3:0/1293405851 conn(0x7feedc0a4210 msgr2=0x7feedc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:22.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.365+0000 7feee1fea700 1 -- 172.21.15.3:0/1293405851 shutdown_connections 2023-10-03T10:18:22.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.365+0000 7feee1fea700 1 -- 172.21.15.3:0/1293405851 wait complete. 2023-10-03T10:18:22.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.367+0000 7feee1fea700 1 Processor -- start 2023-10-03T10:18:22.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.367+0000 7feee1fea700 1 -- start start 2023-10-03T10:18:22.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.368+0000 7feee1fea700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feedc09e930 0x7feedc1a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:22.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.368+0000 7feee1fea700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feedc09f240 0x7feedc1a4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:22.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.368+0000 7feee1fea700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feedc09f650 0x7feedc1a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:22.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.368+0000 7feee1fea700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc1a5f20 con 0x7feedc09f650 2023-10-03T10:18:22.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.368+0000 7feee1fea700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feedc1a6be0 con 0x7feedc09e930 2023-10-03T10:18:22.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.368+0000 7feee1fea700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0a32f0 con 0x7feedc09f240 2023-10-03T10:18:22.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.368+0000 7feedb7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feedc09e930 0x7feedc1a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:22.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.368+0000 7feedb7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feedc09e930 0x7feedc1a3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59990/0 (socket says 172.21.15.3:59990) 2023-10-03T10:18:22.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.368+0000 7feedb7fe700 1 -- 172.21.15.3:0/2566933662 learned_addr learned my addr 172.21.15.3:0/2566933662 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:22.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.368+0000 7feed2ffd700 1 --2- 172.21.15.3:0/2566933662 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feedc09f240 0x7feedc1a4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:22.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.368+0000 7feedbfff700 1 --2- 172.21.15.3:0/2566933662 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feedc09f650 0x7feedc1a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:22.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.369+0000 7feedb7fe700 1 -- 172.21.15.3:0/2566933662 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feedc09f240 msgr2=0x7feedc1a4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:22.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.369+0000 7feedb7fe700 1 --2- 172.21.15.3:0/2566933662 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feedc09f240 0x7feedc1a4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.369+0000 7feedb7fe700 1 -- 172.21.15.3:0/2566933662 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feedc09f650 msgr2=0x7feedc1a4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:22.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.369+0000 7feedb7fe700 1 --2- 172.21.15.3:0/2566933662 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feedc09f650 0x7feedc1a4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.369+0000 7feedb7fe700 1 -- 172.21.15.3:0/2566933662 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feecc009550 con 0x7feedc09e930 2023-10-03T10:18:22.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.369+0000 7feedb7fe700 1 --2- 172.21.15.3:0/2566933662 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feedc09e930 0x7feedc1a3cf0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7feecc00ebe0 tx=0x7feecc00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:22.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.369+0000 7feed8ff9700 1 -- 172.21.15.3:0/2566933662 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc00c850 con 0x7feedc09e930 2023-10-03T10:18:22.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.369+0000 7feed8ff9700 1 -- 172.21.15.3:0/2566933662 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feecc00c9b0 con 0x7feedc09e930 2023-10-03T10:18:22.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.369+0000 7feed8ff9700 1 -- 172.21.15.3:0/2566933662 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc00cd20 con 0x7feedc09e930 2023-10-03T10:18:22.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.369+0000 7feee1fea700 1 -- 172.21.15.3:0/2566933662 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec40094a0 con 0x7feedc09e930 2023-10-03T10:18:22.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.369+0000 7feee1fea700 1 -- 172.21.15.3:0/2566933662 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feedc0a0190 con 0x7feedc09e930 2023-10-03T10:18:22.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.370+0000 7feed8ff9700 1 -- 172.21.15.3:0/2566933662 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feecc013470 con 0x7feedc09e930 2023-10-03T10:18:22.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.370+0000 7feee1fea700 1 -- 172.21.15.3:0/2566933662 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feedc1a0b30 con 0x7feedc09e930 2023-10-03T10:18:22.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.371+0000 7feed8ff9700 1 -- 172.21.15.3:0/2566933662 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7feecc040c20 con 0x7feedc09e930 2023-10-03T10:18:22.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.379+0000 7feed8ff9700 1 -- 172.21.15.3:0/2566933662 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feecc01b3b0 con 0x7feedc09e930 2023-10-03T10:18:22.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.391+0000 7fc7c7979700 1 Processor -- start 2023-10-03T10:18:22.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.392+0000 7fc7c7979700 1 -- start start 2023-10-03T10:18:22.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.393+0000 7fc7c7979700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc7c009d6f0 0x7fc7c009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:22.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.393+0000 7fc7c7979700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc7c009e000 0x7fc7c01339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:22.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.393+0000 7fc7c7979700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc7c009e410 0x7fc7c0133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:22.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.393+0000 7fc7c7979700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c006b460 con 0x7fc7c009e410 2023-10-03T10:18:22.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.393+0000 7fc7c7979700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c009b120 con 0x7fc7c009e000 2023-10-03T10:18:22.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.393+0000 7fc7c7979700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c009a7e0 con 0x7fc7c009d6f0 2023-10-03T10:18:22.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.393+0000 7fc7c4f14700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc7c009e000 0x7fc7c01339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:22.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.393+0000 7fc7c4f14700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc7c009e000 0x7fc7c01339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59998/0 (socket says 172.21.15.3:59998) 2023-10-03T10:18:22.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.393+0000 7fc7c4f14700 1 -- 172.21.15.3:0/1649846147 learned_addr learned my addr 172.21.15.3:0/1649846147 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:22.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.393+0000 7fc7c5f16700 1 --2- 172.21.15.3:0/1649846147 >> v2:172.21.15.3:3300/0 conn(0x7fc7c009e410 0x7fc7c0133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:22.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.393+0000 7fc7c5715700 1 --2- 172.21.15.3:0/1649846147 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc7c009d6f0 0x7fc7c009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:22.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.393+0000 7fc7c4f14700 1 -- 172.21.15.3:0/1649846147 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc7c009d6f0 msgr2=0x7fc7c009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:22.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.393+0000 7fc7c4f14700 1 --2- 172.21.15.3:0/1649846147 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc7c009d6f0 0x7fc7c009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.393+0000 7fc7c4f14700 1 -- 172.21.15.3:0/1649846147 >> v2:172.21.15.3:3300/0 conn(0x7fc7c009e410 msgr2=0x7fc7c0133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:22.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.393+0000 7fc7c4f14700 1 --2- 172.21.15.3:0/1649846147 >> v2:172.21.15.3:3300/0 conn(0x7fc7c009e410 0x7fc7c0133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.393+0000 7fc7c4f14700 1 -- 172.21.15.3:0/1649846147 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c009a500 con 0x7fc7c009e000 2023-10-03T10:18:22.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.393+0000 7fc7c5f16700 1 --2- 172.21.15.3:0/1649846147 >> v2:172.21.15.3:3300/0 conn(0x7fc7c009e410 0x7fc7c0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:22.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.394+0000 7fc7c4f14700 1 --2- 172.21.15.3:0/1649846147 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc7c009e000 0x7fc7c01339f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0009920 tx=0x7fc7b0009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91f1dc3af39e3b4d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:22.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.394+0000 7fc7b7fff700 1 -- 172.21.15.3:0/1649846147 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0014e90 con 0x7fc7c009e000 2023-10-03T10:18:22.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.394+0000 7fc7b7fff700 1 -- 172.21.15.3:0/1649846147 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b001c440 con 0x7fc7c009e000 2023-10-03T10:18:22.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.394+0000 7fc7b7fff700 1 -- 172.21.15.3:0/1649846147 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b001c7d0 con 0x7fc7c009e000 2023-10-03T10:18:22.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.395+0000 7fc7c7979700 1 -- 172.21.15.3:0/1649846147 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc7c009e000 msgr2=0x7fc7c01339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:22.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.395+0000 7fc7c7979700 1 --2- 172.21.15.3:0/1649846147 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc7c009e000 0x7fc7c01339f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0009920 tx=0x7fc7b0009c30 comp rx=0 tx=0).stop 2023-10-03T10:18:22.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.395+0000 7fc7c7979700 1 -- 172.21.15.3:0/1649846147 shutdown_connections 2023-10-03T10:18:22.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.395+0000 7fc7c7979700 1 --2- 172.21.15.3:0/1649846147 >> v2:172.21.15.3:3300/0 conn(0x7fc7c009e410 0x7fc7c0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.395+0000 7fc7c7979700 1 --2- 172.21.15.3:0/1649846147 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc7c009d6f0 0x7fc7c009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.395+0000 7fc7c7979700 1 --2- 172.21.15.3:0/1649846147 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc7c009e000 0x7fc7c01339f0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.395+0000 7fc7c7979700 1 -- 172.21.15.3:0/1649846147 >> 172.21.15.3:0/1649846147 conn(0x7fc7c0066f50 msgr2=0x7fc7c009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:22.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.395+0000 7fc7c7979700 1 -- 172.21.15.3:0/1649846147 shutdown_connections 2023-10-03T10:18:22.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.395+0000 7fc7c7979700 1 -- 172.21.15.3:0/1649846147 wait complete. 2023-10-03T10:18:22.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.396+0000 7fc7c7979700 1 Processor -- start 2023-10-03T10:18:22.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.396+0000 7fc7c7979700 1 -- start start 2023-10-03T10:18:22.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.397+0000 7fc7c7979700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc7c009d6f0 0x7fc7c0119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:22.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.397+0000 7fc7c7979700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7c009e000 0x7fc7c011a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:22.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.397+0000 7fc7c7979700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc7c009e410 0x7fc7c011a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:22.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.397+0000 7fc7c7979700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c011d7d0 con 0x7fc7c009e000 2023-10-03T10:18:22.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.397+0000 7fc7c7979700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c011cd80 con 0x7fc7c009e410 2023-10-03T10:18:22.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.397+0000 7fc7c7979700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c011b3f0 con 0x7fc7c009d6f0 2023-10-03T10:18:22.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.397+0000 7fc7c5715700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc7c009d6f0 0x7fc7c0119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:22.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.397+0000 7fc7c5715700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc7c009d6f0 0x7fc7c0119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43704/0 (socket says 172.21.15.3:43704) 2023-10-03T10:18:22.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.397+0000 7fc7c5715700 1 -- 172.21.15.3:0/2452618144 learned_addr learned my addr 172.21.15.3:0/2452618144 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:22.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.398+0000 7fc7c4f14700 1 --2- 172.21.15.3:0/2452618144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7c009e000 0x7fc7c011a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:22.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.398+0000 7fc7c5f16700 1 --2- 172.21.15.3:0/2452618144 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc7c009e410 0x7fc7c011a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:22.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.398+0000 7fc7c5715700 1 -- 172.21.15.3:0/2452618144 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc7c009e410 msgr2=0x7fc7c011a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:22.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.398+0000 7fc7c5715700 1 --2- 172.21.15.3:0/2452618144 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc7c009e410 0x7fc7c011a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.398+0000 7fc7c5715700 1 -- 172.21.15.3:0/2452618144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7c009e000 msgr2=0x7fc7c011a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:22.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.398+0000 7fc7c5715700 1 --2- 172.21.15.3:0/2452618144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7c009e000 0x7fc7c011a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.398+0000 7fc7c5715700 1 -- 172.21.15.3:0/2452618144 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b8009a70 con 0x7fc7c009d6f0 2023-10-03T10:18:22.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.398+0000 7fc7c5715700 1 --2- 172.21.15.3:0/2452618144 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc7c009d6f0 0x7fc7c0119f40 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00e9b0 tx=0x7fc7bc00ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:22.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.398+0000 7fc7b5ffb700 1 -- 172.21.15.3:0/2452618144 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc009710 con 0x7fc7c009d6f0 2023-10-03T10:18:22.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.399+0000 7fc7b5ffb700 1 -- 172.21.15.3:0/2452618144 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7bc00ee50 con 0x7fc7c009d6f0 2023-10-03T10:18:22.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.399+0000 7fc7b5ffb700 1 -- 172.21.15.3:0/2452618144 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc00ca50 con 0x7fc7c009d6f0 2023-10-03T10:18:22.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.399+0000 7fc7c7979700 1 -- 172.21.15.3:0/2452618144 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b0009550 con 0x7fc7c009d6f0 2023-10-03T10:18:22.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.400+0000 7fc7c7979700 1 -- 172.21.15.3:0/2452618144 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c00a2c80 con 0x7fc7c009d6f0 2023-10-03T10:18:22.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.400+0000 7fc7b5ffb700 1 -- 172.21.15.3:0/2452618144 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc7bc00cbb0 con 0x7fc7c009d6f0 2023-10-03T10:18:22.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.401+0000 7fc7b5ffb700 1 -- 172.21.15.3:0/2452618144 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc7bc03f790 con 0x7fc7c009d6f0 2023-10-03T10:18:22.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.401+0000 7fc7c7979700 1 -- 172.21.15.3:0/2452618144 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c0074ea0 con 0x7fc7c009d6f0 2023-10-03T10:18:22.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.406+0000 7fc7b5ffb700 1 -- 172.21.15.3:0/2452618144 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc7bc021020 con 0x7fc7c009d6f0 2023-10-03T10:18:22.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.446+0000 7fd8bb8c6700 1 Processor -- start 2023-10-03T10:18:22.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.447+0000 7fd8bb8c6700 1 -- start start 2023-10-03T10:18:22.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.447+0000 7fd8bb8c6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8b409d6f0 0x7fd8b409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:22.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.448+0000 7fd8bb8c6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd8b409e000 0x7fd8b4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:22.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.448+0000 7fd8bb8c6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b409f130 con 0x7fd8b409e410 2023-10-03T10:18:22.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.448+0000 7fd8bb8c6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b409f520 con 0x7fd8b409e000 2023-10-03T10:18:22.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.448+0000 7fd8bb8c6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b409e930 con 0x7fd8b409d6f0 2023-10-03T10:18:22.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.448+0000 7fd8b9662700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8b409d6f0 0x7fd8b409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:22.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.448+0000 7fd8b9662700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8b409d6f0 0x7fd8b409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60024/0 (socket says 172.21.15.3:60024) 2023-10-03T10:18:22.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.448+0000 7fd8b9662700 1 -- 172.21.15.3:0/1152974607 learned_addr learned my addr 172.21.15.3:0/1152974607 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:22.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.448+0000 7fd8b8e61700 1 --2- 172.21.15.3:0/1152974607 >> v2:172.21.15.3:3300/0 conn(0x7fd8b409e000 0x7fd8b4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:22.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.448+0000 7fd8b9662700 1 -- 172.21.15.3:0/1152974607 >> v2:172.21.15.3:3300/0 conn(0x7fd8b409e000 msgr2=0x7fd8b4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:22.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.448+0000 7fd8b9662700 1 --2- 172.21.15.3:0/1152974607 >> v2:172.21.15.3:3300/0 conn(0x7fd8b409e000 0x7fd8b4130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.448+0000 7fd8b9662700 1 -- 172.21.15.3:0/1152974607 >> v1:172.21.15.3:6789/0 conn(0x7fd8b409e410 legacy=0x7fd8b4130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:22.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.448+0000 7fd8abfff700 1 -- 172.21.15.3:0/1152974607 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2204988206 0 0) 0x7fd8b409f130 con 0x7fd8b409e410 2023-10-03T10:18:22.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.448+0000 7fd8b9662700 1 -- 172.21.15.3:0/1152974607 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b409b2d0 con 0x7fd8b409d6f0 2023-10-03T10:18:22.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.448+0000 7fd8b9662700 1 --2- 172.21.15.3:0/1152974607 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8b409d6f0 0x7fd8b409dac0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000bb90 tx=0x7fd8b000bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b027e54a55e028eb server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:22.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.449+0000 7fd8abfff700 1 -- 172.21.15.3:0/1152974607 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b000dd50 con 0x7fd8b409d6f0 2023-10-03T10:18:22.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.449+0000 7fd8abfff700 1 -- 172.21.15.3:0/1152974607 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b0010470 con 0x7fd8b409d6f0 2023-10-03T10:18:22.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.449+0000 7fd8abfff700 1 -- 172.21.15.3:0/1152974607 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0010800 con 0x7fd8b409d6f0 2023-10-03T10:18:22.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.450+0000 7fd8bb8c6700 1 -- 172.21.15.3:0/1152974607 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8b409d6f0 msgr2=0x7fd8b409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:22.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.450+0000 7fd8bb8c6700 1 --2- 172.21.15.3:0/1152974607 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8b409d6f0 0x7fd8b409dac0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000bb90 tx=0x7fd8b000bea0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.450+0000 7fd8bb8c6700 1 -- 172.21.15.3:0/1152974607 shutdown_connections 2023-10-03T10:18:22.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.450+0000 7fd8bb8c6700 1 --2- 172.21.15.3:0/1152974607 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8b409d6f0 0x7fd8b409dac0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.450+0000 7fd8bb8c6700 1 --2- 172.21.15.3:0/1152974607 >> v2:172.21.15.3:3300/0 conn(0x7fd8b409e000 0x7fd8b4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.450+0000 7fd8bb8c6700 1 -- 172.21.15.3:0/1152974607 >> 172.21.15.3:0/1152974607 conn(0x7fd8b4066f50 msgr2=0x7fd8b409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:22.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.450+0000 7fd8bb8c6700 1 -- 172.21.15.3:0/1152974607 shutdown_connections 2023-10-03T10:18:22.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.450+0000 7fd8bb8c6700 1 -- 172.21.15.3:0/1152974607 wait complete. 2023-10-03T10:18:22.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.451+0000 7fd8bb8c6700 1 Processor -- start 2023-10-03T10:18:22.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.451+0000 7fd8bb8c6700 1 -- start start 2023-10-03T10:18:22.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.452+0000 7fd8bb8c6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8b409d6f0 0x7fd8b41223f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:22.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.452+0000 7fd8bb8c6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8b409e000 0x7fd8b4122930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:22.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.452+0000 7fd8bb8c6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd8b409e410 0x7fd8b4122e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:22.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.452+0000 7fd8bb8c6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b4125f10 con 0x7fd8b409e000 2023-10-03T10:18:22.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.452+0000 7fd8bb8c6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b4124000 con 0x7fd8b409d6f0 2023-10-03T10:18:22.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.452+0000 7fd8bb8c6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b409a0c0 con 0x7fd8b409e410 2023-10-03T10:18:22.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.452+0000 7fd8b9662700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8b409d6f0 0x7fd8b41223f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:22.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.452+0000 7fd8b8e61700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8b409e000 0x7fd8b4122930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:22.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.452+0000 7fd8b9662700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8b409d6f0 0x7fd8b41223f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:60026/0 (socket says 172.21.15.3:60026) 2023-10-03T10:18:22.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.452+0000 7fd8b9662700 1 -- 172.21.15.3:0/1298108595 learned_addr learned my addr 172.21.15.3:0/1298108595 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:22.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.453+0000 7fd8b9e63700 1 --2- 172.21.15.3:0/1298108595 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd8b409e410 0x7fd8b4122e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:22.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.453+0000 7fd8b9662700 1 -- 172.21.15.3:0/1298108595 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd8b409e410 msgr2=0x7fd8b4122e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:22.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.453+0000 7fd8b9662700 1 --2- 172.21.15.3:0/1298108595 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd8b409e410 0x7fd8b4122e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.453+0000 7fd8b9662700 1 -- 172.21.15.3:0/1298108595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8b409e000 msgr2=0x7fd8b4122930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:22.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.453+0000 7fd8b9662700 1 --2- 172.21.15.3:0/1298108595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8b409e000 0x7fd8b4122930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.453+0000 7fd8b9662700 1 -- 172.21.15.3:0/1298108595 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b000b7f0 con 0x7fd8b409d6f0 2023-10-03T10:18:22.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.453+0000 7fd8b9662700 1 --2- 172.21.15.3:0/1298108595 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8b409d6f0 0x7fd8b41223f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000bee0 tx=0x7fd8b0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:22.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.453+0000 7fd8b8e61700 1 --2- 172.21.15.3:0/1298108595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8b409e000 0x7fd8b4122930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:22.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.453+0000 7fd8a9ffb700 1 -- 172.21.15.3:0/1298108595 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0003ab0 con 0x7fd8b409d6f0 2023-10-03T10:18:22.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.453+0000 7fd8a9ffb700 1 -- 172.21.15.3:0/1298108595 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b0003c10 con 0x7fd8b409d6f0 2023-10-03T10:18:22.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.453+0000 7fd8bb8c6700 1 -- 172.21.15.3:0/1298108595 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b409c700 con 0x7fd8b409d6f0 2023-10-03T10:18:22.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.454+0000 7fd8a9ffb700 1 -- 172.21.15.3:0/1298108595 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b00105e0 con 0x7fd8b409d6f0 2023-10-03T10:18:22.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.454+0000 7fd8bb8c6700 1 -- 172.21.15.3:0/1298108595 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b4126410 con 0x7fd8b409d6f0 2023-10-03T10:18:22.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.455+0000 7fd8a9ffb700 1 -- 172.21.15.3:0/1298108595 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd8b0023440 con 0x7fd8b409d6f0 2023-10-03T10:18:22.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.456+0000 7fd8a9ffb700 1 -- 172.21.15.3:0/1298108595 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd8b0040e80 con 0x7fd8b409d6f0 2023-10-03T10:18:22.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.456+0000 7fd8bb8c6700 1 -- 172.21.15.3:0/1298108595 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b4125a90 con 0x7fd8b409d6f0 2023-10-03T10:18:22.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.461+0000 7fd8a9ffb700 1 -- 172.21.15.3:0/1298108595 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd8b4125a90 con 0x7fd8b409d6f0 2023-10-03T10:18:22.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.554+0000 7fd53296a700 1 -- 172.21.15.3:0/400214771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd52c074ea0 con 0x7fd52c09e000 2023-10-03T10:18:22.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.554+0000 7fd528ff9700 1 -- 172.21.15.3:0/400214771 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd52001f030 con 0x7fd52c09e000 2023-10-03T10:18:22.554 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:22.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.556+0000 7fd53296a700 1 -- 172.21.15.3:0/400214771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd52c09e000 msgr2=0x7fd52c11e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:22.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.556+0000 7fd53296a700 1 --2- 172.21.15.3:0/400214771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd52c09e000 0x7fd52c11e6e0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fd52000fdb0 tx=0x7fd52000d440 comp rx=0 tx=0).stop 2023-10-03T10:18:22.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.556+0000 7fd53296a700 1 -- 172.21.15.3:0/400214771 shutdown_connections 2023-10-03T10:18:22.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.556+0000 7fd53296a700 1 --2- 172.21.15.3:0/400214771 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd52c09e410 0x7fd52c11ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.556+0000 7fd53296a700 1 --2- 172.21.15.3:0/400214771 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd52c09d6f0 0x7fd52c11e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.556+0000 7fd53296a700 1 --2- 172.21.15.3:0/400214771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd52c09e000 0x7fd52c11e6e0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.556+0000 7fd53296a700 1 -- 172.21.15.3:0/400214771 >> 172.21.15.3:0/400214771 conn(0x7fd52c066f50 msgr2=0x7fd52c130890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:22.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.556+0000 7fd53296a700 1 -- 172.21.15.3:0/400214771 shutdown_connections 2023-10-03T10:18:22.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.557+0000 7fd53296a700 1 -- 172.21.15.3:0/400214771 wait complete. 2023-10-03T10:18:22.571 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:22.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.592+0000 7feee1fea700 1 -- 172.21.15.3:0/2566933662 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7feedc074ea0 con 0x7feedc09e930 2023-10-03T10:18:22.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.592+0000 7feed8ff9700 1 -- 172.21.15.3:0/2566933662 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feecc011350 con 0x7feedc09e930 2023-10-03T10:18:22.592 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:22.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.594+0000 7feee1fea700 1 -- 172.21.15.3:0/2566933662 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feedc09e930 msgr2=0x7feedc1a3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:22.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.594+0000 7feee1fea700 1 --2- 172.21.15.3:0/2566933662 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feedc09e930 0x7feedc1a3cf0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7feecc00ebe0 tx=0x7feecc00eef0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.594+0000 7feee1fea700 1 -- 172.21.15.3:0/2566933662 shutdown_connections 2023-10-03T10:18:22.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.594+0000 7feee1fea700 1 --2- 172.21.15.3:0/2566933662 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feedc09f650 0x7feedc1a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.594+0000 7feee1fea700 1 --2- 172.21.15.3:0/2566933662 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feedc09e930 0x7feedc1a3cf0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.594+0000 7feee1fea700 1 --2- 172.21.15.3:0/2566933662 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feedc09f240 0x7feedc1a4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.594+0000 7feee1fea700 1 -- 172.21.15.3:0/2566933662 >> 172.21.15.3:0/2566933662 conn(0x7feedc0a4210 msgr2=0x7feedc1b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:22.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.594+0000 7feee1fea700 1 -- 172.21.15.3:0/2566933662 shutdown_connections 2023-10-03T10:18:22.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.594+0000 7feee1fea700 1 -- 172.21.15.3:0/2566933662 wait complete. 2023-10-03T10:18:22.609 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:22.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.663+0000 7fc7c7979700 1 -- 172.21.15.3:0/2452618144 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc7c011d630 con 0x7fc7c009d6f0 2023-10-03T10:18:22.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.664+0000 7fc7b5ffb700 1 -- 172.21.15.3:0/2452618144 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc7c011d630 con 0x7fc7c009d6f0 2023-10-03T10:18:22.664 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:22.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.666+0000 7fc7c7979700 1 -- 172.21.15.3:0/2452618144 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc7c009d6f0 msgr2=0x7fc7c0119f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:22.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.666+0000 7fc7c7979700 1 --2- 172.21.15.3:0/2452618144 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc7c009d6f0 0x7fc7c0119f40 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00e9b0 tx=0x7fc7bc00ecc0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.666+0000 7fc7c7979700 1 -- 172.21.15.3:0/2452618144 shutdown_connections 2023-10-03T10:18:22.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.666+0000 7fc7c7979700 1 --2- 172.21.15.3:0/2452618144 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc7c009e410 0x7fc7c011a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.666+0000 7fc7c7979700 1 --2- 172.21.15.3:0/2452618144 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc7c009d6f0 0x7fc7c0119f40 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.666+0000 7fc7c7979700 1 --2- 172.21.15.3:0/2452618144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7c009e000 0x7fc7c011a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.666+0000 7fc7c7979700 1 -- 172.21.15.3:0/2452618144 >> 172.21.15.3:0/2452618144 conn(0x7fc7c0066f50 msgr2=0x7fc7c00a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:22.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.666+0000 7fc7c7979700 1 -- 172.21.15.3:0/2452618144 shutdown_connections 2023-10-03T10:18:22.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.666+0000 7fc7c7979700 1 -- 172.21.15.3:0/2452618144 wait complete. 2023-10-03T10:18:22.680 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:22.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.697+0000 7fd8bb8c6700 1 -- 172.21.15.3:0/1298108595 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd8b4125a90 con 0x7fd8b409d6f0 2023-10-03T10:18:22.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.697+0000 7fd8a9ffb700 1 -- 172.21.15.3:0/1298108595 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd8b0019400 con 0x7fd8b409d6f0 2023-10-03T10:18:22.697 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:22.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.699+0000 7fd8bb8c6700 1 -- 172.21.15.3:0/1298108595 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8b409d6f0 msgr2=0x7fd8b41223f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:22.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.699+0000 7fd8bb8c6700 1 --2- 172.21.15.3:0/1298108595 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8b409d6f0 0x7fd8b41223f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000bee0 tx=0x7fd8b0004750 comp rx=0 tx=0).stop 2023-10-03T10:18:22.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.700+0000 7fd8bb8c6700 1 -- 172.21.15.3:0/1298108595 shutdown_connections 2023-10-03T10:18:22.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.700+0000 7fd8bb8c6700 1 --2- 172.21.15.3:0/1298108595 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd8b409e410 0x7fd8b4122e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.700+0000 7fd8bb8c6700 1 --2- 172.21.15.3:0/1298108595 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8b409d6f0 0x7fd8b41223f0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.700+0000 7fd8bb8c6700 1 --2- 172.21.15.3:0/1298108595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8b409e000 0x7fd8b4122930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:22.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.700+0000 7fd8bb8c6700 1 -- 172.21.15.3:0/1298108595 >> 172.21.15.3:0/1298108595 conn(0x7fd8b4066f50 msgr2=0x7fd8b4070ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:22.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.700+0000 7fd8bb8c6700 1 -- 172.21.15.3:0/1298108595 shutdown_connections 2023-10-03T10:18:22.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:22.700+0000 7fd8bb8c6700 1 -- 172.21.15.3:0/1298108595 wait complete. 2023-10-03T10:18:22.713 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:23.572 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:23.609 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:23.681 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:23.714 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:23.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.722+0000 7f5fecae5700 1 Processor -- start 2023-10-03T10:18:23.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.722+0000 7f5fecae5700 1 -- start start 2023-10-03T10:18:23.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.723+0000 7f5fecae5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fe81338a0 0x7f5fe8133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:23.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.723+0000 7f5fecae5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5fe809de50 0x7f5fe809e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:23.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.723+0000 7f5fecae5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fe806ef30 con 0x7f5fe809e760 2023-10-03T10:18:23.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.723+0000 7f5fecae5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe806fef0 con 0x7f5fe809de50 2023-10-03T10:18:23.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.723+0000 7f5fecae5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe81334b0 con 0x7f5fe81338a0 2023-10-03T10:18:23.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.723+0000 7f5fe659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fe81338a0 0x7f5fe8133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:23.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.723+0000 7f5fe5d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5fe809de50 0x7f5fe809e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:23.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.723+0000 7f5fe659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fe81338a0 0x7f5fe8133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51908/0 (socket says 172.21.15.3:51908) 2023-10-03T10:18:23.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.723+0000 7f5fe659c700 1 -- 172.21.15.3:0/1169393529 learned_addr learned my addr 172.21.15.3:0/1169393529 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:23.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.723+0000 7f5fe659c700 1 -- 172.21.15.3:0/1169393529 >> v2:172.21.15.3:3300/0 conn(0x7f5fe809de50 msgr2=0x7f5fe809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:23.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.723+0000 7f5fe659c700 1 --2- 172.21.15.3:0/1169393529 >> v2:172.21.15.3:3300/0 conn(0x7f5fe809de50 0x7f5fe809e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:23.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.723+0000 7f5fe659c700 1 -- 172.21.15.3:0/1169393529 >> v1:172.21.15.3:6789/0 conn(0x7f5fe809e760 legacy=0x7f5fe813a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:23.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.724+0000 7f5fe659c700 1 -- 172.21.15.3:0/1169393529 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe806fc30 con 0x7f5fe81338a0 2023-10-03T10:18:23.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.723+0000 7f5fe559a700 1 -- 172.21.15.3:0/1169393529 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3550837569 0 0) 0x7f5fe806ef30 con 0x7f5fe809e760 2023-10-03T10:18:23.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.724+0000 7f5fe5d9b700 1 --2- 172.21.15.3:0/1169393529 >> v2:172.21.15.3:3300/0 conn(0x7f5fe809de50 0x7f5fe809e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:18:23.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.724+0000 7f5fe659c700 1 --2- 172.21.15.3:0/1169393529 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fe81338a0 0x7f5fe8133c70 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5fd80098f0 tx=0x7f5fd8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8c953fb502f7878 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:23.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.724+0000 7f5fe559a700 1 -- 172.21.15.3:0/1169393529 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd800b8a0 con 0x7f5fe81338a0 2023-10-03T10:18:23.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.724+0000 7f5fe559a700 1 -- 172.21.15.3:0/1169393529 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd800ba00 con 0x7f5fe81338a0 2023-10-03T10:18:23.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.724+0000 7f5fe559a700 1 -- 172.21.15.3:0/1169393529 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd800bd90 con 0x7f5fe81338a0 2023-10-03T10:18:23.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.725+0000 7f5fecae5700 1 -- 172.21.15.3:0/1169393529 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fe81338a0 msgr2=0x7f5fe8133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:23.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.725+0000 7f5fecae5700 1 --2- 172.21.15.3:0/1169393529 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fe81338a0 0x7f5fe8133c70 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5fd80098f0 tx=0x7f5fd8009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:23.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.725+0000 7f5fecae5700 1 -- 172.21.15.3:0/1169393529 shutdown_connections 2023-10-03T10:18:23.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.725+0000 7f5fecae5700 1 --2- 172.21.15.3:0/1169393529 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fe81338a0 0x7f5fe8133c70 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:23.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.725+0000 7f5fecae5700 1 --2- 172.21.15.3:0/1169393529 >> v2:172.21.15.3:3300/0 conn(0x7f5fe809de50 0x7f5fe809e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:23.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.725+0000 7f5fecae5700 1 -- 172.21.15.3:0/1169393529 >> 172.21.15.3:0/1169393529 conn(0x7f5fe80a2aa0 msgr2=0x7f5fe80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:23.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.725+0000 7f5fecae5700 1 -- 172.21.15.3:0/1169393529 shutdown_connections 2023-10-03T10:18:23.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.725+0000 7f5fecae5700 1 -- 172.21.15.3:0/1169393529 wait complete. 2023-10-03T10:18:23.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.727+0000 7f5fecae5700 1 Processor -- start 2023-10-03T10:18:23.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.727+0000 7f5fecae5700 1 -- start start 2023-10-03T10:18:23.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.727+0000 7f5fecae5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fe809de50 0x7f5fe811a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:23.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.727+0000 7f5fecae5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fe809e760 0x7f5fe811aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:23.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.727+0000 7f5fecae5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fe81338a0 0x7f5fe811b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:23.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.727+0000 7f5fecae5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe811e0c0 con 0x7f5fe809de50 2023-10-03T10:18:23.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.727+0000 7f5fecae5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe80a0b50 con 0x7f5fe81338a0 2023-10-03T10:18:23.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.727+0000 7f5fecae5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe811d850 con 0x7f5fe809e760 2023-10-03T10:18:23.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.727+0000 7f5fe6d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fe81338a0 0x7f5fe811b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:23.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.727+0000 7f5fe659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fe809de50 0x7f5fe811a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:23.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.727+0000 7f5fe5d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fe809e760 0x7f5fe811aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:23.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.727+0000 7f5fe6d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fe81338a0 0x7f5fe811b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51918/0 (socket says 172.21.15.3:51918) 2023-10-03T10:18:23.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.727+0000 7f5fe6d9d700 1 -- 172.21.15.3:0/4120488762 learned_addr learned my addr 172.21.15.3:0/4120488762 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:23.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.727+0000 7f5fe659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fe809de50 0x7f5fe811a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58448/0 (socket says 172.21.15.3:58448) 2023-10-03T10:18:23.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.728+0000 7f5fe6d9d700 1 -- 172.21.15.3:0/4120488762 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fe809e760 msgr2=0x7f5fe811aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:23.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.728+0000 7f5fe6d9d700 1 --2- 172.21.15.3:0/4120488762 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fe809e760 0x7f5fe811aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:23.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.728+0000 7f5fe6d9d700 1 -- 172.21.15.3:0/4120488762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fe809de50 msgr2=0x7f5fe811a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:23.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.728+0000 7f5fe6d9d700 1 --2- 172.21.15.3:0/4120488762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fe809de50 0x7f5fe811a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:23.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.728+0000 7f5fe6d9d700 1 -- 172.21.15.3:0/4120488762 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd8009550 con 0x7f5fe81338a0 2023-10-03T10:18:23.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.728+0000 7f5fe5d9b700 1 --2- 172.21.15.3:0/4120488762 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fe809e760 0x7f5fe811aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:18:23.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.728+0000 7f5fe6d9d700 1 --2- 172.21.15.3:0/4120488762 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fe81338a0 0x7f5fe811b020 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0001560 tx=0x7f5fe000cb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:23.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.728+0000 7f5fd6ffd700 1 -- 172.21.15.3:0/4120488762 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0003e30 con 0x7f5fe81338a0 2023-10-03T10:18:23.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.728+0000 7f5fd6ffd700 1 -- 172.21.15.3:0/4120488762 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe0005070 con 0x7f5fe81338a0 2023-10-03T10:18:23.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.728+0000 7f5fecae5700 1 -- 172.21.15.3:0/4120488762 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe806f640 con 0x7f5fe81338a0 2023-10-03T10:18:23.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.728+0000 7f5fd6ffd700 1 -- 172.21.15.3:0/4120488762 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0005420 con 0x7f5fe81338a0 2023-10-03T10:18:23.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.728+0000 7f5fecae5700 1 -- 172.21.15.3:0/4120488762 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe811d400 con 0x7f5fe81338a0 2023-10-03T10:18:23.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.729+0000 7f5fd6ffd700 1 -- 172.21.15.3:0/4120488762 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5fe0005580 con 0x7f5fe81338a0 2023-10-03T10:18:23.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.730+0000 7f5fecae5700 1 -- 172.21.15.3:0/4120488762 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe811da80 con 0x7f5fe81338a0 2023-10-03T10:18:23.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.730+0000 7f5fd6ffd700 1 -- 172.21.15.3:0/4120488762 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5fe003ef70 con 0x7f5fe81338a0 2023-10-03T10:18:23.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.735+0000 7f5fd6ffd700 1 -- 172.21.15.3:0/4120488762 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5fe811da80 con 0x7f5fe81338a0 2023-10-03T10:18:23.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.766+0000 7f18dffee700 1 Processor -- start 2023-10-03T10:18:23.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.766+0000 7f18dffee700 1 -- start start 2023-10-03T10:18:23.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.767+0000 7f18dffee700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18d809e930 0x7f18d809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:23.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.767+0000 7f18dffee700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f18d809f240 0x7f18d81317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:23.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.767+0000 7f18dffee700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f18d809f650 0x7f18d8131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:23.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.767+0000 7f18dffee700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d8070940 con 0x7f18d809f240 2023-10-03T10:18:23.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.767+0000 7f18dffee700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d806fd60 con 0x7f18d809f650 2023-10-03T10:18:23.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.767+0000 7f18dffee700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f18d806fa80 con 0x7f18d809e930 2023-10-03T10:18:23.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.767+0000 7f18ddd8a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18d809e930 0x7f18d809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:23.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.767+0000 7f18ddd8a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18d809e930 0x7f18d809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52070/0 (socket says 172.21.15.3:52070) 2023-10-03T10:18:23.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.767+0000 7f18ddd8a700 1 -- 172.21.15.3:0/3629933297 learned_addr learned my addr 172.21.15.3:0/3629933297 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:23.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.767+0000 7f18de58b700 1 --2- 172.21.15.3:0/3629933297 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f18d809f650 0x7f18d8131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:23.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.767+0000 7f18dd589700 1 --2- 172.21.15.3:0/3629933297 >> v2:172.21.15.3:3300/0 conn(0x7f18d809f240 0x7f18d81317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:23.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.768+0000 7f18ddd8a700 1 -- 172.21.15.3:0/3629933297 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f18d809f650 msgr2=0x7f18d8131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:23.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.768+0000 7f18ddd8a700 1 --2- 172.21.15.3:0/3629933297 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f18d809f650 0x7f18d8131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:23.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.768+0000 7f18ddd8a700 1 -- 172.21.15.3:0/3629933297 >> v2:172.21.15.3:3300/0 conn(0x7f18d809f240 msgr2=0x7f18d81317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:23.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.768+0000 7f18ddd8a700 1 --2- 172.21.15.3:0/3629933297 >> v2:172.21.15.3:3300/0 conn(0x7f18d809f240 0x7f18d81317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:23.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.768+0000 7f18ddd8a700 1 -- 172.21.15.3:0/3629933297 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d809fb60 con 0x7f18d809e930 2023-10-03T10:18:23.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.768+0000 7f18ddd8a700 1 --2- 172.21.15.3:0/3629933297 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18d809e930 0x7f18d809ed00 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f18d40098f0 tx=0x7f18d4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9467a48e6f2ce076 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:23.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.768+0000 7f18dcd88700 1 -- 172.21.15.3:0/3629933297 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4011830 con 0x7f18d809e930 2023-10-03T10:18:23.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.768+0000 7f18dcd88700 1 -- 172.21.15.3:0/3629933297 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d4011990 con 0x7f18d809e930 2023-10-03T10:18:23.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.768+0000 7f18dcd88700 1 -- 172.21.15.3:0/3629933297 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4011d20 con 0x7f18d809e930 2023-10-03T10:18:23.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.769+0000 7f18dffee700 1 -- 172.21.15.3:0/3629933297 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18d809e930 msgr2=0x7f18d809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:23.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.769+0000 7f18dffee700 1 --2- 172.21.15.3:0/3629933297 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18d809e930 0x7f18d809ed00 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f18d40098f0 tx=0x7f18d4009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:23.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.769+0000 7f18dffee700 1 -- 172.21.15.3:0/3629933297 shutdown_connections 2023-10-03T10:18:23.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.769+0000 7f18dffee700 1 --2- 172.21.15.3:0/3629933297 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f18d809f650 0x7f18d8131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:23.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.769+0000 7f18dffee700 1 --2- 172.21.15.3:0/3629933297 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18d809e930 0x7f18d809ed00 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.769+0000 7f18dffee700 1 --2- 172.21.15.3:0/3629933297 >> v2:172.21.15.3:3300/0 conn(0x7f18d809f240 0x7f18d81317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.769+0000 7f18dffee700 1 -- 172.21.15.3:0/3629933297 >> 172.21.15.3:0/3629933297 conn(0x7f18d80a4210 msgr2=0x7f18d80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.769+0000 7f18dffee700 1 -- 172.21.15.3:0/3629933297 shutdown_connections 2023-10-03T10:18:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.769+0000 7f18dffee700 1 -- 172.21.15.3:0/3629933297 wait complete. 2023-10-03T10:18:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.771+0000 7f18dffee700 1 Processor -- start 2023-10-03T10:18:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.771+0000 7f18dffee700 1 -- start start 2023-10-03T10:18:23.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.772+0000 7f18dffee700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18d809e930 0x7f18d806b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:23.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.772+0000 7f18dffee700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18d809f240 0x7f18d806bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:23.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.772+0000 7f18dffee700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f18d809f650 0x7f18d806c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:23.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.772+0000 7f18dffee700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d806cd80 con 0x7f18d809e930 2023-10-03T10:18:23.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.772+0000 7f18dffee700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d81a8970 con 0x7f18d809f650 2023-10-03T10:18:23.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.772+0000 7f18dffee700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f18d81b7260 con 0x7f18d809f240 2023-10-03T10:18:23.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.772+0000 7f18de58b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f18d809f650 0x7f18d806c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:23.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.772+0000 7f18de58b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f18d809f650 0x7f18d806c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51936/0 (socket says 172.21.15.3:51936) 2023-10-03T10:18:23.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.772+0000 7f18de58b700 1 -- 172.21.15.3:0/245899122 learned_addr learned my addr 172.21.15.3:0/245899122 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:23.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.772+0000 7f18ddd8a700 1 --2- 172.21.15.3:0/245899122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18d809e930 0x7f18d806b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.772+0000 7f18dd589700 1 --2- 172.21.15.3:0/245899122 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18d809f240 0x7f18d806bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.772+0000 7f18de58b700 1 -- 172.21.15.3:0/245899122 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18d809f240 msgr2=0x7f18d806bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.772+0000 7f18de58b700 1 --2- 172.21.15.3:0/245899122 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18d809f240 0x7f18d806bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.772+0000 7f18de58b700 1 -- 172.21.15.3:0/245899122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18d809e930 msgr2=0x7f18d806b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.772+0000 7f18de58b700 1 --2- 172.21.15.3:0/245899122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18d809e930 0x7f18d806b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.772+0000 7f18de58b700 1 -- 172.21.15.3:0/245899122 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d4009550 con 0x7f18d809f650 2023-10-03T10:18:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.772+0000 7f18ddd8a700 1 --2- 172.21.15.3:0/245899122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18d809e930 0x7f18d806b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.773+0000 7f18de58b700 1 --2- 172.21.15.3:0/245899122 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f18d809f650 0x7f18d806c330 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f18d0009e30 tx=0x7f18d000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.773+0000 7f18ce7fc700 1 -- 172.21.15.3:0/245899122 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d0009ca0 con 0x7f18d809f650 2023-10-03T10:18:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.773+0000 7f18ce7fc700 1 -- 172.21.15.3:0/245899122 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d000f040 con 0x7f18d809f650 2023-10-03T10:18:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.773+0000 7f18dffee700 1 -- 172.21.15.3:0/245899122 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d80a0190 con 0x7f18d809f650 2023-10-03T10:18:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.773+0000 7f18ce7fc700 1 -- 172.21.15.3:0/245899122 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d0014480 con 0x7f18d809f650 2023-10-03T10:18:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.774+0000 7f18ce7fc700 1 -- 172.21.15.3:0/245899122 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f18d00145e0 con 0x7f18d809f650 2023-10-03T10:18:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.774+0000 7f18dffee700 1 -- 172.21.15.3:0/245899122 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d81a9190 con 0x7f18d809f650 2023-10-03T10:18:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.774+0000 7f18ce7fc700 1 -- 172.21.15.3:0/245899122 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f18d001c7b0 con 0x7f18d809f650 2023-10-03T10:18:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.775+0000 7f18dffee700 1 -- 172.21.15.3:0/245899122 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d81a8cf0 con 0x7f18d809f650 2023-10-03T10:18:23.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.784+0000 7f18ce7fc700 1 -- 172.21.15.3:0/245899122 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f18d0018020 con 0x7f18d809f650 2023-10-03T10:18:23.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.835+0000 7f8412237700 1 Processor -- start 2023-10-03T10:18:23.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.835+0000 7f8412237700 1 -- start start 2023-10-03T10:18:23.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.835+0000 7f8412237700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f840c12cf60 0x7f840c06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:23.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.835+0000 7f8412237700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f840c12f8b0 0x7f840c12fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:23.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.835+0000 7f8412237700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f840c130260 0x7f840c136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:23.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.835+0000 7f8412237700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c12d330 con 0x7f840c12cf60 2023-10-03T10:18:23.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.835+0000 7f8412237700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c12cd90 con 0x7f840c12f8b0 2023-10-03T10:18:23.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.835+0000 7f8412237700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f840c1306d0 con 0x7f840c130260 2023-10-03T10:18:23.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.836+0000 7f840b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f840c12cf60 0x7f840c06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:23.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.836+0000 7f840affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f840c12f8b0 0x7f840c12fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:23.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.836+0000 7f840b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f840c12cf60 0x7f840c06f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58490/0 (socket says 172.21.15.3:58490) 2023-10-03T10:18:23.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.836+0000 7f840b7fe700 1 -- 172.21.15.3:0/1471564264 learned_addr learned my addr 172.21.15.3:0/1471564264 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:23.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.836+0000 7f840affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f840c12f8b0 0x7f840c12fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51950/0 (socket says 172.21.15.3:51950) 2023-10-03T10:18:23.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.836+0000 7f840bfff700 1 --2- 172.21.15.3:0/1471564264 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f840c130260 0x7f840c136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:23.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.836+0000 7f840bfff700 1 -- 172.21.15.3:0/1471564264 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f840c12f8b0 msgr2=0x7f840c12fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:23.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.836+0000 7f840bfff700 1 --2- 172.21.15.3:0/1471564264 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f840c12f8b0 0x7f840c12fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:23.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.836+0000 7f840bfff700 1 -- 172.21.15.3:0/1471564264 >> v2:172.21.15.3:3300/0 conn(0x7f840c12cf60 msgr2=0x7f840c06f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:23.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.836+0000 7f840bfff700 1 --2- 172.21.15.3:0/1471564264 >> v2:172.21.15.3:3300/0 conn(0x7f840c12cf60 0x7f840c06f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:23.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.836+0000 7f840bfff700 1 -- 172.21.15.3:0/1471564264 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840c071780 con 0x7f840c130260 2023-10-03T10:18:23.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.836+0000 7f840bfff700 1 --2- 172.21.15.3:0/1471564264 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f840c130260 0x7f840c136c20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f8404009dc0 tx=0x7f840400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9997c79591039ace server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:23.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.837+0000 7f840a7fc700 1 -- 172.21.15.3:0/1471564264 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840400e070 con 0x7f840c130260 2023-10-03T10:18:23.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.837+0000 7f840a7fc700 1 -- 172.21.15.3:0/1471564264 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840400bae0 con 0x7f840c130260 2023-10-03T10:18:23.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.837+0000 7f840a7fc700 1 -- 172.21.15.3:0/1471564264 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840400be90 con 0x7f840c130260 2023-10-03T10:18:23.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.838+0000 7f8412237700 1 -- 172.21.15.3:0/1471564264 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f840c130260 msgr2=0x7f840c136c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:23.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.838+0000 7f8412237700 1 --2- 172.21.15.3:0/1471564264 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f840c130260 0x7f840c136c20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f8404009dc0 tx=0x7f840400b330 comp rx=0 tx=0).stop 2023-10-03T10:18:23.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.838+0000 7f8412237700 1 -- 172.21.15.3:0/1471564264 shutdown_connections 2023-10-03T10:18:23.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.838+0000 7f8412237700 1 --2- 172.21.15.3:0/1471564264 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f840c130260 0x7f840c136c20 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:23.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.838+0000 7f8412237700 1 --2- 172.21.15.3:0/1471564264 >> v2:172.21.15.3:3300/0 conn(0x7f840c12cf60 0x7f840c06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:23.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.838+0000 7f8412237700 1 --2- 172.21.15.3:0/1471564264 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f840c12f8b0 0x7f840c12fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:23.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.838+0000 7f8412237700 1 -- 172.21.15.3:0/1471564264 >> 172.21.15.3:0/1471564264 conn(0x7f840c0a2aa0 msgr2=0x7f840c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:23.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.838+0000 7f8412237700 1 -- 172.21.15.3:0/1471564264 shutdown_connections 2023-10-03T10:18:23.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.838+0000 7f8412237700 1 -- 172.21.15.3:0/1471564264 wait complete. 2023-10-03T10:18:23.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.839+0000 7f8412237700 1 Processor -- start 2023-10-03T10:18:23.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.840+0000 7f8412237700 1 -- start start 2023-10-03T10:18:23.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.840+0000 7f8412237700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f840c12cf60 0x7f840c1c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:23.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.840+0000 7f8412237700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f840c12f8b0 0x7f840c1c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:23.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.840+0000 7f8412237700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f840c130260 0x7f840c1c4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:23.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.840+0000 7f8412237700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c09d090 con 0x7f840c130260 2023-10-03T10:18:23.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.840+0000 7f8412237700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c129fa0 con 0x7f840c12cf60 2023-10-03T10:18:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.840+0000 7f8412237700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f840c09df20 con 0x7f840c12f8b0 2023-10-03T10:18:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.841+0000 7f840affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f840c12f8b0 0x7f840c1c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.841+0000 7f840bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f840c130260 0x7f840c1c4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.841+0000 7f840affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f840c12f8b0 0x7f840c1c46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52108/0 (socket says 172.21.15.3:52108) 2023-10-03T10:18:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.841+0000 7f840affd700 1 -- 172.21.15.3:0/589191589 learned_addr learned my addr 172.21.15.3:0/589191589 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.841+0000 7f840b7fe700 1 --2- 172.21.15.3:0/589191589 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f840c12cf60 0x7f840c1c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.841+0000 7f840affd700 1 -- 172.21.15.3:0/589191589 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f840c12cf60 msgr2=0x7f840c1c41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.841+0000 7f840affd700 1 --2- 172.21.15.3:0/589191589 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f840c12cf60 0x7f840c1c41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.841+0000 7f840affd700 1 -- 172.21.15.3:0/589191589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f840c130260 msgr2=0x7f840c1c4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.841+0000 7f840affd700 1 --2- 172.21.15.3:0/589191589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f840c130260 0x7f840c1c4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.841+0000 7f840affd700 1 -- 172.21.15.3:0/589191589 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8404009a70 con 0x7f840c12f8b0 2023-10-03T10:18:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.841+0000 7f840bfff700 1 --2- 172.21.15.3:0/589191589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f840c130260 0x7f840c1c4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.841+0000 7f840affd700 1 --2- 172.21.15.3:0/589191589 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f840c12f8b0 0x7f840c1c46e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f840000e9b0 tx=0x7f840000ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.842+0000 7f83f3fff700 1 -- 172.21.15.3:0/589191589 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8400009710 con 0x7f840c12f8b0 2023-10-03T10:18:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.842+0000 7f83f3fff700 1 -- 172.21.15.3:0/589191589 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840000ee50 con 0x7f840c12f8b0 2023-10-03T10:18:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.842+0000 7f83f3fff700 1 -- 172.21.15.3:0/589191589 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840000ca50 con 0x7f840c12f8b0 2023-10-03T10:18:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.843+0000 7f8412237700 1 -- 172.21.15.3:0/589191589 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f840c09e880 con 0x7f840c12f8b0 2023-10-03T10:18:23.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.843+0000 7f8412237700 1 -- 172.21.15.3:0/589191589 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f840c1193d0 con 0x7f840c12f8b0 2023-10-03T10:18:23.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.844+0000 7f83f3fff700 1 -- 172.21.15.3:0/589191589 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f840000cbb0 con 0x7f840c12f8b0 2023-10-03T10:18:23.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.844+0000 7f8412237700 1 -- 172.21.15.3:0/589191589 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f840c074ea0 con 0x7f840c12f8b0 2023-10-03T10:18:23.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.844+0000 7f83f3fff700 1 -- 172.21.15.3:0/589191589 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f840003f6e0 con 0x7f840c12f8b0 2023-10-03T10:18:23.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.849+0000 7f83f3fff700 1 -- 172.21.15.3:0/589191589 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f840001be10 con 0x7f840c12f8b0 2023-10-03T10:18:23.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.877+0000 7f203180b700 1 Processor -- start 2023-10-03T10:18:23.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.877+0000 7f203180b700 1 -- start start 2023-10-03T10:18:23.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.878+0000 7f203180b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f202c1316c0 0x7f202c131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:23.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.878+0000 7f203180b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f202c06ae80 0x7f202c139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:23.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.878+0000 7f203180b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f202c09e130 con 0x7f202c09d9f0 2023-10-03T10:18:23.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.878+0000 7f203180b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f202c1312b0 con 0x7f202c06ae80 2023-10-03T10:18:23.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.878+0000 7f203180b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202c09f6e0 con 0x7f202c1316c0 2023-10-03T10:18:23.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.878+0000 7f202a7fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f202c09d9f0 0x7f202c06a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47074/0 (socket says 172.21.15.3:47074) 2023-10-03T10:18:23.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.878+0000 7f202a7fc700 1 -- 172.21.15.3:0/2466729175 learned_addr learned my addr 172.21.15.3:0/2466729175 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:23.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.878+0000 7f202b7fe700 1 --2- 172.21.15.3:0/2466729175 >> v2:172.21.15.3:3300/0 conn(0x7f202c06ae80 0x7f202c139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:23.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.878+0000 7f202affd700 1 --2- 172.21.15.3:0/2466729175 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f202c1316c0 0x7f202c131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:23.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.878+0000 7f2029ffb700 1 -- 172.21.15.3:0/2466729175 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3144425467 0 0) 0x7f2020001110 con 0x7f202c09d9f0 2023-10-03T10:18:23.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.878+0000 7f2029ffb700 1 -- 172.21.15.3:0/2466729175 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20140035b0 con 0x7f202c09d9f0 2023-10-03T10:18:23.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.879+0000 7f202b7fe700 1 -- 172.21.15.3:0/2466729175 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f202c1316c0 msgr2=0x7f202c131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:23.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.879+0000 7f202b7fe700 1 --2- 172.21.15.3:0/2466729175 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f202c1316c0 0x7f202c131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:23.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.879+0000 7f202b7fe700 1 -- 172.21.15.3:0/2466729175 >> v1:172.21.15.3:6789/0 conn(0x7f202c09d9f0 legacy=0x7f202c06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:23.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.879+0000 7f202b7fe700 1 -- 172.21.15.3:0/2466729175 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202c09e760 con 0x7f202c06ae80 2023-10-03T10:18:23.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.879+0000 7f202affd700 1 --2- 172.21.15.3:0/2466729175 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f202c1316c0 0x7f202c131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:23.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.879+0000 7f202b7fe700 1 --2- 172.21.15.3:0/2466729175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c06ae80 0x7f202c139550 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f2024009d90 tx=0x7f202400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17ea4c6fcf12ff6 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:23.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.879+0000 7f2029ffb700 1 -- 172.21.15.3:0/2466729175 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202400e070 con 0x7f202c06ae80 2023-10-03T10:18:23.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.879+0000 7f2029ffb700 1 -- 172.21.15.3:0/2466729175 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202400bb20 con 0x7f202c06ae80 2023-10-03T10:18:23.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.879+0000 7f2029ffb700 1 -- 172.21.15.3:0/2466729175 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2024012430 con 0x7f202c06ae80 2023-10-03T10:18:23.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.880+0000 7f203180b700 1 -- 172.21.15.3:0/2466729175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c06ae80 msgr2=0x7f202c139550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:23.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.880+0000 7f203180b700 1 --2- 172.21.15.3:0/2466729175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c06ae80 0x7f202c139550 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f2024009d90 tx=0x7f202400b330 comp rx=0 tx=0).stop 2023-10-03T10:18:23.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.880+0000 7f203180b700 1 -- 172.21.15.3:0/2466729175 shutdown_connections 2023-10-03T10:18:23.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.880+0000 7f203180b700 1 --2- 172.21.15.3:0/2466729175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c06ae80 0x7f202c139550 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:23.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.880+0000 7f203180b700 1 --2- 172.21.15.3:0/2466729175 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f202c1316c0 0x7f202c131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:23.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.880+0000 7f203180b700 1 -- 172.21.15.3:0/2466729175 >> 172.21.15.3:0/2466729175 conn(0x7f202c066f50 msgr2=0x7f202c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:23.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.880+0000 7f203180b700 1 -- 172.21.15.3:0/2466729175 shutdown_connections 2023-10-03T10:18:23.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.880+0000 7f203180b700 1 -- 172.21.15.3:0/2466729175 wait complete. 2023-10-03T10:18:23.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.881+0000 7f203180b700 1 Processor -- start 2023-10-03T10:18:23.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.882+0000 7f203180b700 1 -- start start 2023-10-03T10:18:23.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.882+0000 7f203180b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f202c06ae80 0x7f202c139e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:23.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.882+0000 7f203180b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c09d9f0 0x7f202c0713e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:23.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.882+0000 7f203180b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f202c1316c0 0x7f202c071920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:23.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.882+0000 7f203180b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c134500 con 0x7f202c09d9f0 2023-10-03T10:18:23.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.882+0000 7f203180b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202c134700 con 0x7f202c06ae80 2023-10-03T10:18:23.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.882+0000 7f203180b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f202c1335f0 con 0x7f202c1316c0 2023-10-03T10:18:23.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.883+0000 7f202b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f202c1316c0 0x7f202c071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:23.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.883+0000 7f202a7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c09d9f0 0x7f202c0713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:23.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.883+0000 7f202b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f202c1316c0 0x7f202c071920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52118/0 (socket says 172.21.15.3:52118) 2023-10-03T10:18:23.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.883+0000 7f202b7fe700 1 -- 172.21.15.3:0/3146684153 learned_addr learned my addr 172.21.15.3:0/3146684153 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:23.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.883+0000 7f202affd700 1 --2- 172.21.15.3:0/3146684153 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f202c06ae80 0x7f202c139e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:23.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.883+0000 7f202a7fc700 1 -- 172.21.15.3:0/3146684153 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f202c1316c0 msgr2=0x7f202c071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:23.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.883+0000 7f202a7fc700 1 --2- 172.21.15.3:0/3146684153 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f202c1316c0 0x7f202c071920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:23.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.883+0000 7f202a7fc700 1 -- 172.21.15.3:0/3146684153 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f202c06ae80 msgr2=0x7f202c139e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:23.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.883+0000 7f202a7fc700 1 --2- 172.21.15.3:0/3146684153 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f202c06ae80 0x7f202c139e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:23.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.883+0000 7f202a7fc700 1 -- 172.21.15.3:0/3146684153 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f201c0094a0 con 0x7f202c09d9f0 2023-10-03T10:18:23.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.883+0000 7f202a7fc700 1 --2- 172.21.15.3:0/3146684153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c09d9f0 0x7f202c0713e0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f202c06deb0 tx=0x7f202000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:23.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.883+0000 7f2013fff700 1 -- 172.21.15.3:0/3146684153 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2020005d70 con 0x7f202c09d9f0 2023-10-03T10:18:23.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.884+0000 7f2013fff700 1 -- 172.21.15.3:0/3146684153 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2020001d30 con 0x7f202c09d9f0 2023-10-03T10:18:23.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.884+0000 7f2013fff700 1 -- 172.21.15.3:0/3146684153 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20200072f0 con 0x7f202c09d9f0 2023-10-03T10:18:23.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.884+0000 7f203180b700 1 -- 172.21.15.3:0/3146684153 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2024009a70 con 0x7f202c09d9f0 2023-10-03T10:18:23.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.884+0000 7f203180b700 1 -- 172.21.15.3:0/3146684153 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f202c13a5f0 con 0x7f202c09d9f0 2023-10-03T10:18:23.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.885+0000 7f2013fff700 1 -- 172.21.15.3:0/3146684153 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2020007450 con 0x7f202c09d9f0 2023-10-03T10:18:23.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.885+0000 7f203180b700 1 -- 172.21.15.3:0/3146684153 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f202c074ea0 con 0x7f202c09d9f0 2023-10-03T10:18:23.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.885+0000 7f2013fff700 1 -- 172.21.15.3:0/3146684153 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f202003e960 con 0x7f202c09d9f0 2023-10-03T10:18:23.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.891+0000 7f2013fff700 1 -- 172.21.15.3:0/3146684153 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2020017080 con 0x7f202c09d9f0 2023-10-03T10:18:23.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.956+0000 7f5fecae5700 1 -- 172.21.15.3:0/4120488762 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5fe811da80 con 0x7f5fe81338a0 2023-10-03T10:18:23.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.957+0000 7f5fd6ffd700 1 -- 172.21.15.3:0/4120488762 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5fe811da80 con 0x7f5fe81338a0 2023-10-03T10:18:23.957 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:23.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.959+0000 7f5fecae5700 1 -- 172.21.15.3:0/4120488762 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fe81338a0 msgr2=0x7f5fe811b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:23.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.959+0000 7f5fecae5700 1 --2- 172.21.15.3:0/4120488762 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fe81338a0 0x7f5fe811b020 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0001560 tx=0x7f5fe000cb50 comp rx=0 tx=0).stop 2023-10-03T10:18:23.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.959+0000 7f5fecae5700 1 -- 172.21.15.3:0/4120488762 shutdown_connections 2023-10-03T10:18:23.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.959+0000 7f5fecae5700 1 --2- 172.21.15.3:0/4120488762 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5fe81338a0 0x7f5fe811b020 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:23.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.959+0000 7f5fecae5700 1 --2- 172.21.15.3:0/4120488762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fe809de50 0x7f5fe811a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:23.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.959+0000 7f5fecae5700 1 --2- 172.21.15.3:0/4120488762 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5fe809e760 0x7f5fe811aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:23.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.959+0000 7f5fecae5700 1 -- 172.21.15.3:0/4120488762 >> 172.21.15.3:0/4120488762 conn(0x7f5fe80a2aa0 msgr2=0x7f5fe8130730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:23.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.959+0000 7f5fecae5700 1 -- 172.21.15.3:0/4120488762 shutdown_connections 2023-10-03T10:18:23.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.959+0000 7f5fecae5700 1 -- 172.21.15.3:0/4120488762 wait complete. 2023-10-03T10:18:23.973 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:23.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.998+0000 7f18dffee700 1 -- 172.21.15.3:0/245899122 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f18d81a0b30 con 0x7f18d809f650 2023-10-03T10:18:23.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:23.998+0000 7f18ce7fc700 1 -- 172.21.15.3:0/245899122 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f18d0045060 con 0x7f18d809f650 2023-10-03T10:18:23.999 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:24.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.001+0000 7f18dffee700 1 -- 172.21.15.3:0/245899122 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f18d809f650 msgr2=0x7f18d806c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:24.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.001+0000 7f18dffee700 1 --2- 172.21.15.3:0/245899122 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f18d809f650 0x7f18d806c330 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f18d0009e30 tx=0x7f18d000c3a0 comp rx=0 tx=0).stop 2023-10-03T10:18:24.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.001+0000 7f18dffee700 1 -- 172.21.15.3:0/245899122 shutdown_connections 2023-10-03T10:18:24.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.001+0000 7f18dffee700 1 --2- 172.21.15.3:0/245899122 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f18d809f650 0x7f18d806c330 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:24.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.001+0000 7f18dffee700 1 --2- 172.21.15.3:0/245899122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18d809e930 0x7f18d806b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:24.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.001+0000 7f18dffee700 1 --2- 172.21.15.3:0/245899122 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f18d809f240 0x7f18d806bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:24.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.001+0000 7f18dffee700 1 -- 172.21.15.3:0/245899122 >> 172.21.15.3:0/245899122 conn(0x7f18d80a4210 msgr2=0x7f18d81b8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:24.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.001+0000 7f18dffee700 1 -- 172.21.15.3:0/245899122 shutdown_connections 2023-10-03T10:18:24.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.001+0000 7f18dffee700 1 -- 172.21.15.3:0/245899122 wait complete. 2023-10-03T10:18:24.015 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:24.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.061+0000 7f8412237700 1 -- 172.21.15.3:0/589191589 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f840c09ea70 con 0x7f840c12f8b0 2023-10-03T10:18:24.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.061+0000 7f83f3fff700 1 -- 172.21.15.3:0/589191589 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f840001b3b0 con 0x7f840c12f8b0 2023-10-03T10:18:24.061 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:24.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.063+0000 7f8412237700 1 -- 172.21.15.3:0/589191589 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f840c12f8b0 msgr2=0x7f840c1c46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:24.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.063+0000 7f8412237700 1 --2- 172.21.15.3:0/589191589 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f840c12f8b0 0x7f840c1c46e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f840000e9b0 tx=0x7f840000ecc0 comp rx=0 tx=0).stop 2023-10-03T10:18:24.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.063+0000 7f8412237700 1 -- 172.21.15.3:0/589191589 shutdown_connections 2023-10-03T10:18:24.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.063+0000 7f8412237700 1 --2- 172.21.15.3:0/589191589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f840c130260 0x7f840c1c4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:24.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.063+0000 7f8412237700 1 --2- 172.21.15.3:0/589191589 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f840c12cf60 0x7f840c1c41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:24.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.063+0000 7f8412237700 1 --2- 172.21.15.3:0/589191589 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f840c12f8b0 0x7f840c1c46e0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:24.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.063+0000 7f8412237700 1 -- 172.21.15.3:0/589191589 >> 172.21.15.3:0/589191589 conn(0x7f840c0a2aa0 msgr2=0x7f840c1b7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:24.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.064+0000 7f8412237700 1 -- 172.21.15.3:0/589191589 shutdown_connections 2023-10-03T10:18:24.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.064+0000 7f8412237700 1 -- 172.21.15.3:0/589191589 wait complete. 2023-10-03T10:18:24.077 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:24.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.100+0000 7f203180b700 1 -- 172.21.15.3:0/3146684153 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f202c09e130 con 0x7f202c09d9f0 2023-10-03T10:18:24.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.100+0000 7f2013fff700 1 -- 172.21.15.3:0/3146684153 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f202c09e130 con 0x7f202c09d9f0 2023-10-03T10:18:24.100 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:24.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.102+0000 7f203180b700 1 -- 172.21.15.3:0/3146684153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c09d9f0 msgr2=0x7f202c0713e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:24.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.102+0000 7f203180b700 1 --2- 172.21.15.3:0/3146684153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c09d9f0 0x7f202c0713e0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f202c06deb0 tx=0x7f202000c9e0 comp rx=0 tx=0).stop 2023-10-03T10:18:24.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.102+0000 7f203180b700 1 -- 172.21.15.3:0/3146684153 shutdown_connections 2023-10-03T10:18:24.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.102+0000 7f203180b700 1 --2- 172.21.15.3:0/3146684153 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f202c1316c0 0x7f202c071920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:24.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.102+0000 7f203180b700 1 --2- 172.21.15.3:0/3146684153 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f202c06ae80 0x7f202c139e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:24.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.102+0000 7f203180b700 1 --2- 172.21.15.3:0/3146684153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f202c09d9f0 0x7f202c0713e0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:24.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.102+0000 7f203180b700 1 -- 172.21.15.3:0/3146684153 >> 172.21.15.3:0/3146684153 conn(0x7f202c066f50 msgr2=0x7f202c09b930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:24.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.103+0000 7f203180b700 1 -- 172.21.15.3:0/3146684153 shutdown_connections 2023-10-03T10:18:24.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:24.103+0000 7f203180b700 1 -- 172.21.15.3:0/3146684153 wait complete. 2023-10-03T10:18:24.116 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:24.974 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:25.016 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:25.078 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:25.116 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:25.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.124+0000 7fc17e42f700 1 Processor -- start 2023-10-03T10:18:25.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.124+0000 7fc17e42f700 1 -- start start 2023-10-03T10:18:25.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.125+0000 7fc17e42f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc1781305f0 0x7fc1781309c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:25.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.125+0000 7fc17e42f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc17806f660 0x7fc17806fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:25.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.125+0000 7fc17e42f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1781301c0 con 0x7fc17806ff90 2023-10-03T10:18:25.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.125+0000 7fc17e42f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc178071ff0 con 0x7fc17806f660 2023-10-03T10:18:25.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.125+0000 7fc17e42f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc178070e70 con 0x7fc1781305f0 2023-10-03T10:18:25.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.125+0000 7fc1777fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc17806f660 0x7fc17806fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:25.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.126+0000 7fc1777fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc17806f660 0x7fc17806fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58524/0 (socket says 172.21.15.3:58524) 2023-10-03T10:18:25.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.126+0000 7fc1777fe700 1 -- 172.21.15.3:0/2553325591 learned_addr learned my addr 172.21.15.3:0/2553325591 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:25.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.126+0000 7fc177fff700 1 --2- 172.21.15.3:0/2553325591 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc1781305f0 0x7fc1781309c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:25.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.126+0000 7fc1777fe700 1 -- 172.21.15.3:0/2553325591 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc1781305f0 msgr2=0x7fc1781309c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:25.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.126+0000 7fc1777fe700 1 --2- 172.21.15.3:0/2553325591 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc1781305f0 0x7fc1781309c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.126+0000 7fc1777fe700 1 -- 172.21.15.3:0/2553325591 >> v1:172.21.15.3:6789/0 conn(0x7fc17806ff90 legacy=0x7fc178137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:25.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.126+0000 7fc1777fe700 1 -- 172.21.15.3:0/2553325591 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1780722b0 con 0x7fc17806f660 2023-10-03T10:18:25.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.126+0000 7fc1777fe700 1 --2- 172.21.15.3:0/2553325591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc17806f660 0x7fc17806fa50 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0098f0 tx=0x7fc16c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8679c946313555b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:25.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.126+0000 7fc176ffd700 1 -- 172.21.15.3:0/2553325591 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c011830 con 0x7fc17806f660 2023-10-03T10:18:25.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.127+0000 7fc176ffd700 1 -- 172.21.15.3:0/2553325591 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16c011990 con 0x7fc17806f660 2023-10-03T10:18:25.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.127+0000 7fc176ffd700 1 -- 172.21.15.3:0/2553325591 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c011d20 con 0x7fc17806f660 2023-10-03T10:18:25.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.127+0000 7fc17e42f700 1 -- 172.21.15.3:0/2553325591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc17806f660 msgr2=0x7fc17806fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:25.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.127+0000 7fc17e42f700 1 --2- 172.21.15.3:0/2553325591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc17806f660 0x7fc17806fa50 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0098f0 tx=0x7fc16c009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:25.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.127+0000 7fc17e42f700 1 -- 172.21.15.3:0/2553325591 shutdown_connections 2023-10-03T10:18:25.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.127+0000 7fc17e42f700 1 --2- 172.21.15.3:0/2553325591 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc1781305f0 0x7fc1781309c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.127+0000 7fc17e42f700 1 --2- 172.21.15.3:0/2553325591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc17806f660 0x7fc17806fa50 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.127+0000 7fc17e42f700 1 -- 172.21.15.3:0/2553325591 >> 172.21.15.3:0/2553325591 conn(0x7fc1780a2aa0 msgr2=0x7fc1780a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:25.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.127+0000 7fc17e42f700 1 -- 172.21.15.3:0/2553325591 shutdown_connections 2023-10-03T10:18:25.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.128+0000 7fc17e42f700 1 -- 172.21.15.3:0/2553325591 wait complete. 2023-10-03T10:18:25.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.129+0000 7fc17e42f700 1 Processor -- start 2023-10-03T10:18:25.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.129+0000 7fc17e42f700 1 -- start start 2023-10-03T10:18:25.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.130+0000 7fc17e42f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc17806f660 0x7fc1781c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:25.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.130+0000 7fc17e42f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc17806ff90 0x7fc1781c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:25.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.130+0000 7fc17e42f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc1781305f0 0x7fc1781c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:25.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.130+0000 7fc17e42f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17813b9e0 con 0x7fc17806f660 2023-10-03T10:18:25.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.130+0000 7fc17e42f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17813afb0 con 0x7fc1781305f0 2023-10-03T10:18:25.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.130+0000 7fc17e42f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc178071820 con 0x7fc17806ff90 2023-10-03T10:18:25.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.130+0000 7fc177fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc17806f660 0x7fc1781c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:25.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.130+0000 7fc177fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc17806f660 0x7fc1781c8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58536/0 (socket says 172.21.15.3:58536) 2023-10-03T10:18:25.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.130+0000 7fc177fff700 1 -- 172.21.15.3:0/1217926835 learned_addr learned my addr 172.21.15.3:0/1217926835 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:25.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.130+0000 7fc17c9cc700 1 --2- 172.21.15.3:0/1217926835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc1781305f0 0x7fc1781c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:25.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.130+0000 7fc1777fe700 1 --2- 172.21.15.3:0/1217926835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc17806ff90 0x7fc1781c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:25.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.131+0000 7fc177fff700 1 -- 172.21.15.3:0/1217926835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc17806ff90 msgr2=0x7fc1781c8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:25.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.131+0000 7fc177fff700 1 --2- 172.21.15.3:0/1217926835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc17806ff90 0x7fc1781c8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.131+0000 7fc177fff700 1 -- 172.21.15.3:0/1217926835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc1781305f0 msgr2=0x7fc1781c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:25.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.131+0000 7fc177fff700 1 --2- 172.21.15.3:0/1217926835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc1781305f0 0x7fc1781c8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.131+0000 7fc177fff700 1 -- 172.21.15.3:0/1217926835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16c009550 con 0x7fc17806f660 2023-10-03T10:18:25.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.131+0000 7fc177fff700 1 --2- 172.21.15.3:0/1217926835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc17806f660 0x7fc1781c8230 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fc16800e9e0 tx=0x7fc16800ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:25.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.131+0000 7fc174ff9700 1 -- 172.21.15.3:0/1217926835 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc168009710 con 0x7fc17806f660 2023-10-03T10:18:25.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.131+0000 7fc174ff9700 1 -- 172.21.15.3:0/1217926835 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16800ee80 con 0x7fc17806f660 2023-10-03T10:18:25.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.131+0000 7fc174ff9700 1 -- 172.21.15.3:0/1217926835 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16800ca50 con 0x7fc17806f660 2023-10-03T10:18:25.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.131+0000 7fc17e42f700 1 -- 172.21.15.3:0/1217926835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc178118d70 con 0x7fc17806f660 2023-10-03T10:18:25.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.132+0000 7fc17e42f700 1 -- 172.21.15.3:0/1217926835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc178138a40 con 0x7fc17806f660 2023-10-03T10:18:25.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.133+0000 7fc17e42f700 1 -- 172.21.15.3:0/1217926835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc178074ea0 con 0x7fc17806f660 2023-10-03T10:18:25.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.133+0000 7fc174ff9700 1 -- 172.21.15.3:0/1217926835 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc16800cbb0 con 0x7fc17806f660 2023-10-03T10:18:25.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.134+0000 7fc174ff9700 1 -- 172.21.15.3:0/1217926835 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc16803f800 con 0x7fc17806f660 2023-10-03T10:18:25.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.143+0000 7fc174ff9700 1 -- 172.21.15.3:0/1217926835 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc1680232f0 con 0x7fc17806f660 2023-10-03T10:18:25.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.183+0000 7fe1f457c700 1 Processor -- start 2023-10-03T10:18:25.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.183+0000 7fe1f457c700 1 -- start start 2023-10-03T10:18:25.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.184+0000 7fe1f457c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1ec09e930 0x7fe1ec09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:25.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.184+0000 7fe1f457c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1ec1342f0 0x7fe1ec13be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:25.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.184+0000 7fe1f457c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1ec06c9f0 con 0x7fe1ec09f240 2023-10-03T10:18:25.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.184+0000 7fe1f457c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec06c670 con 0x7fe1ec1342f0 2023-10-03T10:18:25.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.184+0000 7fe1f457c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec06bc90 con 0x7fe1ec09e930 2023-10-03T10:18:25.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.185+0000 7fe1f2318700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1ec09e930 0x7fe1ec09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:25.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.185+0000 7fe1f2b19700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1ec1342f0 0x7fe1ec13be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:25.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.185+0000 7fe1f2318700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1ec09e930 0x7fe1ec09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52140/0 (socket says 172.21.15.3:52140) 2023-10-03T10:18:25.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.185+0000 7fe1f2318700 1 -- 172.21.15.3:0/2402723185 learned_addr learned my addr 172.21.15.3:0/2402723185 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:25.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.185+0000 7fe1f2318700 1 -- 172.21.15.3:0/2402723185 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1ec1342f0 msgr2=0x7fe1ec13be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:25.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.185+0000 7fe1f2318700 1 --2- 172.21.15.3:0/2402723185 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1ec1342f0 0x7fe1ec13be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.185+0000 7fe1f2318700 1 -- 172.21.15.3:0/2402723185 >> v1:172.21.15.3:6789/0 conn(0x7fe1ec09f240 legacy=0x7fe1ec133be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:25.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.185+0000 7fe1f2318700 1 -- 172.21.15.3:0/2402723185 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1ec06d860 con 0x7fe1ec09e930 2023-10-03T10:18:25.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.185+0000 7fe1f2b19700 1 --2- 172.21.15.3:0/2402723185 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1ec1342f0 0x7fe1ec13be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:18:25.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.185+0000 7fe1f2318700 1 --2- 172.21.15.3:0/2402723185 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1ec09e930 0x7fe1ec09ed00 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8009950 tx=0x7fe1e8009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b569aaa982e4211 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:25.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.186+0000 7fe1f1316700 1 -- 172.21.15.3:0/2402723185 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8009ea0 con 0x7fe1ec09e930 2023-10-03T10:18:25.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.186+0000 7fe1f1316700 1 -- 172.21.15.3:0/2402723185 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e800ba20 con 0x7fe1ec09e930 2023-10-03T10:18:25.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.186+0000 7fe1f1316700 1 -- 172.21.15.3:0/2402723185 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e800bdb0 con 0x7fe1ec09e930 2023-10-03T10:18:25.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.188+0000 7fe1f457c700 1 -- 172.21.15.3:0/2402723185 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1ec09e930 msgr2=0x7fe1ec09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:25.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.188+0000 7fe1f457c700 1 --2- 172.21.15.3:0/2402723185 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1ec09e930 0x7fe1ec09ed00 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8009950 tx=0x7fe1e8009c60 comp rx=0 tx=0).stop 2023-10-03T10:18:25.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.188+0000 7fe1f457c700 1 -- 172.21.15.3:0/2402723185 shutdown_connections 2023-10-03T10:18:25.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.188+0000 7fe1f457c700 1 --2- 172.21.15.3:0/2402723185 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1ec1342f0 0x7fe1ec13be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.188+0000 7fe1f457c700 1 --2- 172.21.15.3:0/2402723185 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1ec09e930 0x7fe1ec09ed00 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.188+0000 7fe1f457c700 1 -- 172.21.15.3:0/2402723185 >> 172.21.15.3:0/2402723185 conn(0x7fe1ec0a4210 msgr2=0x7fe1ec0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:25.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.189+0000 7fe1f457c700 1 -- 172.21.15.3:0/2402723185 shutdown_connections 2023-10-03T10:18:25.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.189+0000 7fe1f457c700 1 -- 172.21.15.3:0/2402723185 wait complete. 2023-10-03T10:18:25.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.190+0000 7fe1f457c700 1 Processor -- start 2023-10-03T10:18:25.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.190+0000 7fe1f457c700 1 -- start start 2023-10-03T10:18:25.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.191+0000 7fe1f457c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1ec09e930 0x7fe1ec122a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:25.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.191+0000 7fe1f457c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1ec09f240 0x7fe1ec122f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:25.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.191+0000 7fe1f457c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1ec1342f0 0x7fe1ec1234d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:25.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.191+0000 7fe1f457c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec126290 con 0x7fe1ec09f240 2023-10-03T10:18:25.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.191+0000 7fe1f457c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec124770 con 0x7fe1ec1342f0 2023-10-03T10:18:25.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.191+0000 7fe1f457c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec06fbe0 con 0x7fe1ec09e930 2023-10-03T10:18:25.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.191+0000 7fe1f1b17700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1ec09f240 0x7fe1ec122f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:25.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.191+0000 7fe1f2318700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1ec09e930 0x7fe1ec122a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:25.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.191+0000 7fe1f1b17700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1ec09f240 0x7fe1ec122f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58544/0 (socket says 172.21.15.3:58544) 2023-10-03T10:18:25.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.191+0000 7fe1f1b17700 1 -- 172.21.15.3:0/76757269 learned_addr learned my addr 172.21.15.3:0/76757269 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:25.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.191+0000 7fe1f2318700 1 -- 172.21.15.3:0/76757269 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1ec1342f0 msgr2=0x7fe1ec1234d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:25.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.191+0000 7fe1f2318700 1 --2- 172.21.15.3:0/76757269 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1ec1342f0 0x7fe1ec1234d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.191+0000 7fe1f2318700 1 -- 172.21.15.3:0/76757269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1ec09f240 msgr2=0x7fe1ec122f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:25.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.191+0000 7fe1f2318700 1 --2- 172.21.15.3:0/76757269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1ec09f240 0x7fe1ec122f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.191+0000 7fe1f2318700 1 -- 172.21.15.3:0/76757269 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e8009550 con 0x7fe1ec09e930 2023-10-03T10:18:25.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.191+0000 7fe1f1b17700 1 --2- 172.21.15.3:0/76757269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1ec09f240 0x7fe1ec122f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:25.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.192+0000 7fe1f2318700 1 --2- 172.21.15.3:0/76757269 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1ec09e930 0x7fe1ec122a50 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8016000 tx=0x7fe1e8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:25.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.192+0000 7fe1e2ffd700 1 -- 172.21.15.3:0/76757269 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e800be90 con 0x7fe1ec09e930 2023-10-03T10:18:25.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.192+0000 7fe1e2ffd700 1 -- 172.21.15.3:0/76757269 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e801f070 con 0x7fe1ec09e930 2023-10-03T10:18:25.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.192+0000 7fe1e2ffd700 1 -- 172.21.15.3:0/76757269 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e80036a0 con 0x7fe1ec09e930 2023-10-03T10:18:25.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.192+0000 7fe1f457c700 1 -- 172.21.15.3:0/76757269 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1ec06cbb0 con 0x7fe1ec09e930 2023-10-03T10:18:25.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.192+0000 7fe1f457c700 1 -- 172.21.15.3:0/76757269 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1ec126d00 con 0x7fe1ec09e930 2023-10-03T10:18:25.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.194+0000 7fe1f457c700 1 -- 172.21.15.3:0/76757269 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1ec06cee0 con 0x7fe1ec09e930 2023-10-03T10:18:25.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.194+0000 7fe1e2ffd700 1 -- 172.21.15.3:0/76757269 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe1e8021030 con 0x7fe1ec09e930 2023-10-03T10:18:25.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.194+0000 7fe1e2ffd700 1 -- 172.21.15.3:0/76757269 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe1e8040a10 con 0x7fe1ec09e930 2023-10-03T10:18:25.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.199+0000 7fe1e2ffd700 1 -- 172.21.15.3:0/76757269 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe1e8019970 con 0x7fe1ec09e930 2023-10-03T10:18:25.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.247+0000 7fdfbf21f700 1 Processor -- start 2023-10-03T10:18:25.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.247+0000 7fdfbf21f700 1 -- start start 2023-10-03T10:18:25.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.247+0000 7fdfbf21f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfb8134630 0x7fdfb8136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:25.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.247+0000 7fdfbf21f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdfb8136f50 0x7fdfb8139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:25.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.247+0000 7fdfbf21f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfb809b4f0 con 0x7fdfb8131810 2023-10-03T10:18:25.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.248+0000 7fdfbf21f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb81313c0 con 0x7fdfb8134630 2023-10-03T10:18:25.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.248+0000 7fdfbf21f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb806e8c0 con 0x7fdfb8136f50 2023-10-03T10:18:25.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.248+0000 7fdfb7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfb8134630 0x7fdfb8136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:25.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.248+0000 7fdfbd7bc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdfb8136f50 0x7fdfb8139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:25.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.248+0000 7fdfb7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfb8134630 0x7fdfb8136a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52008/0 (socket says 172.21.15.3:52008) 2023-10-03T10:18:25.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.248+0000 7fdfb7fff700 1 -- 172.21.15.3:0/32705207 learned_addr learned my addr 172.21.15.3:0/32705207 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:25.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.248+0000 7fdfbd7bc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdfb8136f50 0x7fdfb8139330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52166/0 (socket says 172.21.15.3:52166) 2023-10-03T10:18:25.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.248+0000 7fdfb7fff700 1 -- 172.21.15.3:0/32705207 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdfb8136f50 msgr2=0x7fdfb8139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:25.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.248+0000 7fdfb7fff700 1 --2- 172.21.15.3:0/32705207 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdfb8136f50 0x7fdfb8139330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.248+0000 7fdfb7fff700 1 -- 172.21.15.3:0/32705207 >> v1:172.21.15.3:6789/0 conn(0x7fdfb8131810 legacy=0x7fdfb8131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:25.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.248+0000 7fdfb7fff700 1 -- 172.21.15.3:0/32705207 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb8071d20 con 0x7fdfb8134630 2023-10-03T10:18:25.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.248+0000 7fdfb7fff700 1 --2- 172.21.15.3:0/32705207 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfb8134630 0x7fdfb8136a10 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fdfa400dd40 tx=0x7fdfa400f330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4917f48963105dd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:25.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.249+0000 7fdfb77fe700 1 -- 172.21.15.3:0/32705207 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4012070 con 0x7fdfb8134630 2023-10-03T10:18:25.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.249+0000 7fdfb77fe700 1 -- 172.21.15.3:0/32705207 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa400de90 con 0x7fdfb8134630 2023-10-03T10:18:25.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.249+0000 7fdfb77fe700 1 -- 172.21.15.3:0/32705207 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa400fb30 con 0x7fdfb8134630 2023-10-03T10:18:25.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.249+0000 7fdfbf21f700 1 -- 172.21.15.3:0/32705207 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfb8134630 msgr2=0x7fdfb8136a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:25.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.249+0000 7fdfbf21f700 1 --2- 172.21.15.3:0/32705207 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfb8134630 0x7fdfb8136a10 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fdfa400dd40 tx=0x7fdfa400f330 comp rx=0 tx=0).stop 2023-10-03T10:18:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.249+0000 7fdfbf21f700 1 -- 172.21.15.3:0/32705207 shutdown_connections 2023-10-03T10:18:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.249+0000 7fdfbf21f700 1 --2- 172.21.15.3:0/32705207 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdfb8136f50 0x7fdfb8139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.249+0000 7fdfbf21f700 1 --2- 172.21.15.3:0/32705207 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfb8134630 0x7fdfb8136a10 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.249+0000 7fdfbf21f700 1 -- 172.21.15.3:0/32705207 >> 172.21.15.3:0/32705207 conn(0x7fdfb80a2aa0 msgr2=0x7fdfb80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.249+0000 7fdfbf21f700 1 -- 172.21.15.3:0/32705207 shutdown_connections 2023-10-03T10:18:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.249+0000 7fdfbf21f700 1 -- 172.21.15.3:0/32705207 wait complete. 2023-10-03T10:18:25.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.251+0000 7fdfbf21f700 1 Processor -- start 2023-10-03T10:18:25.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.251+0000 7fdfbf21f700 1 -- start start 2023-10-03T10:18:25.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.252+0000 7fdfbf21f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfb8131810 0x7fdfb809c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:25.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.252+0000 7fdfbf21f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdfb8134630 0x7fdfb809cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:25.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.252+0000 7fdfbf21f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfb8136f50 0x7fdfb809d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:25.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.252+0000 7fdfbf21f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb8118d70 con 0x7fdfb8136f50 2023-10-03T10:18:25.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.252+0000 7fdfbf21f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb8136ab0 con 0x7fdfb8131810 2023-10-03T10:18:25.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.252+0000 7fdfbf21f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb806eb00 con 0x7fdfb8134630 2023-10-03T10:18:25.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.252+0000 7fdfbcfbb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfb8131810 0x7fdfb809c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.252+0000 7fdfbcfbb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfb8131810 0x7fdfb809c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52010/0 (socket says 172.21.15.3:52010) 2023-10-03T10:18:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.252+0000 7fdfbcfbb700 1 -- 172.21.15.3:0/2064761750 learned_addr learned my addr 172.21.15.3:0/2064761750 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.252+0000 7fdfbd7bc700 1 --2- 172.21.15.3:0/2064761750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfb8136f50 0x7fdfb809d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.252+0000 7fdfb7fff700 1 --2- 172.21.15.3:0/2064761750 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdfb8134630 0x7fdfb809cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.252+0000 7fdfbcfbb700 1 -- 172.21.15.3:0/2064761750 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdfb8134630 msgr2=0x7fdfb809cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.252+0000 7fdfbcfbb700 1 --2- 172.21.15.3:0/2064761750 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdfb8134630 0x7fdfb809cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.252+0000 7fdfbcfbb700 1 -- 172.21.15.3:0/2064761750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfb8136f50 msgr2=0x7fdfb809d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.252+0000 7fdfbcfbb700 1 --2- 172.21.15.3:0/2064761750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfb8136f50 0x7fdfb809d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.252+0000 7fdfbcfbb700 1 -- 172.21.15.3:0/2064761750 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa400d9f0 con 0x7fdfb8131810 2023-10-03T10:18:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.252+0000 7fdfb7fff700 1 --2- 172.21.15.3:0/2064761750 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdfb8134630 0x7fdfb809cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:18:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.252+0000 7fdfbd7bc700 1 --2- 172.21.15.3:0/2064761750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfb8136f50 0x7fdfb809d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.252+0000 7fdfbcfbb700 1 --2- 172.21.15.3:0/2064761750 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfb8131810 0x7fdfb809c700 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00c4e0 tx=0x7fdfac00c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:25.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.253+0000 7fdfb57fa700 1 -- 172.21.15.3:0/2064761750 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac0052d0 con 0x7fdfb8131810 2023-10-03T10:18:25.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.253+0000 7fdfb57fa700 1 -- 172.21.15.3:0/2064761750 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfac005430 con 0x7fdfb8131810 2023-10-03T10:18:25.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.253+0000 7fdfb57fa700 1 -- 172.21.15.3:0/2064761750 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac0038c0 con 0x7fdfb8131810 2023-10-03T10:18:25.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.253+0000 7fdfbf21f700 1 -- 172.21.15.3:0/2064761750 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb81199e0 con 0x7fdfb8131810 2023-10-03T10:18:25.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.253+0000 7fdfbf21f700 1 -- 172.21.15.3:0/2064761750 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb811ae30 con 0x7fdfb8131810 2023-10-03T10:18:25.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.254+0000 7fdfb57fa700 1 -- 172.21.15.3:0/2064761750 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdfac003a20 con 0x7fdfb8131810 2023-10-03T10:18:25.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.254+0000 7fdfbf21f700 1 -- 172.21.15.3:0/2064761750 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfb8074ea0 con 0x7fdfb8131810 2023-10-03T10:18:25.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.254+0000 7fdfb57fa700 1 -- 172.21.15.3:0/2064761750 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fdfac03eb90 con 0x7fdfb8131810 2023-10-03T10:18:25.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.263+0000 7fdfb57fa700 1 -- 172.21.15.3:0/2064761750 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdfac018260 con 0x7fdfb8131810 2023-10-03T10:18:25.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.275+0000 7f8e5050e700 1 Processor -- start 2023-10-03T10:18:25.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.275+0000 7f8e5050e700 1 -- start start 2023-10-03T10:18:25.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.275+0000 7f8e5050e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e4809de00 0x7f8e4809e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:25.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.275+0000 7f8e5050e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8e4813d1f0 0x7f8e4813f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:25.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.275+0000 7f8e5050e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e4809a810 con 0x7f8e4806a770 2023-10-03T10:18:25.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.275+0000 7f8e5050e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e4806ab40 con 0x7f8e4813d1f0 2023-10-03T10:18:25.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.275+0000 7f8e5050e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4809ebc0 con 0x7f8e4809de00 2023-10-03T10:18:25.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.276+0000 7f8e4daa9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e4809de00 0x7f8e4809e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:25.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.276+0000 7f8e4daa9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e4809de00 0x7f8e4809e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52016/0 (socket says 172.21.15.3:52016) 2023-10-03T10:18:25.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.276+0000 7f8e4daa9700 1 -- 172.21.15.3:0/3255095952 learned_addr learned my addr 172.21.15.3:0/3255095952 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:25.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.276+0000 7f8e4eaab700 1 --2- 172.21.15.3:0/3255095952 >> v2:172.21.15.3:3300/0 conn(0x7f8e4813d1f0 0x7f8e4813f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:25.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.276+0000 7f8e4daa9700 1 -- 172.21.15.3:0/3255095952 >> v2:172.21.15.3:3300/0 conn(0x7f8e4813d1f0 msgr2=0x7f8e4813f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:25.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.276+0000 7f8e4daa9700 1 --2- 172.21.15.3:0/3255095952 >> v2:172.21.15.3:3300/0 conn(0x7f8e4813d1f0 0x7f8e4813f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.276+0000 7f8e4daa9700 1 -- 172.21.15.3:0/3255095952 >> v1:172.21.15.3:6789/0 conn(0x7f8e4806a770 legacy=0x7f8e4809d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:25.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.276+0000 7f8e4daa9700 1 -- 172.21.15.3:0/3255095952 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4809e920 con 0x7f8e4809de00 2023-10-03T10:18:25.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.276+0000 7f8e4daa9700 1 --2- 172.21.15.3:0/3255095952 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e4809de00 0x7f8e4809e250 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8e380098f0 tx=0x7f8e38009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7aed86ad14cbda8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:25.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.276+0000 7f8e4d2a8700 1 -- 172.21.15.3:0/3255095952 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38011830 con 0x7f8e4809de00 2023-10-03T10:18:25.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.277+0000 7f8e4d2a8700 1 -- 172.21.15.3:0/3255095952 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e38011990 con 0x7f8e4809de00 2023-10-03T10:18:25.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.277+0000 7f8e4d2a8700 1 -- 172.21.15.3:0/3255095952 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38011d00 con 0x7f8e4809de00 2023-10-03T10:18:25.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.277+0000 7f8e5050e700 1 -- 172.21.15.3:0/3255095952 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e4809de00 msgr2=0x7f8e4809e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:25.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.277+0000 7f8e5050e700 1 --2- 172.21.15.3:0/3255095952 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e4809de00 0x7f8e4809e250 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8e380098f0 tx=0x7f8e38009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:25.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.278+0000 7f8e5050e700 1 -- 172.21.15.3:0/3255095952 shutdown_connections 2023-10-03T10:18:25.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.278+0000 7f8e5050e700 1 --2- 172.21.15.3:0/3255095952 >> v2:172.21.15.3:3300/0 conn(0x7f8e4813d1f0 0x7f8e4813f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.278+0000 7f8e5050e700 1 --2- 172.21.15.3:0/3255095952 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e4809de00 0x7f8e4809e250 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.278+0000 7f8e5050e700 1 -- 172.21.15.3:0/3255095952 >> 172.21.15.3:0/3255095952 conn(0x7f8e48066f50 msgr2=0x7f8e4809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:25.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.278+0000 7f8e5050e700 1 -- 172.21.15.3:0/3255095952 shutdown_connections 2023-10-03T10:18:25.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.278+0000 7f8e5050e700 1 -- 172.21.15.3:0/3255095952 wait complete. 2023-10-03T10:18:25.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.279+0000 7f8e5050e700 1 Processor -- start 2023-10-03T10:18:25.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.279+0000 7f8e5050e700 1 -- start start 2023-10-03T10:18:25.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.280+0000 7f8e5050e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8e4806a770 0x7f8e481260a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:25.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.280+0000 7f8e5050e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e4809de00 0x7f8e481265e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:25.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.280+0000 7f8e5050e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e4813d1f0 0x7f8e48126b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:25.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.280+0000 7f8e5050e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e481290c0 con 0x7f8e4813d1f0 2023-10-03T10:18:25.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.280+0000 7f8e5050e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e480a3840 con 0x7f8e4809de00 2023-10-03T10:18:25.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.280+0000 7f8e5050e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4809f1a0 con 0x7f8e4806a770 2023-10-03T10:18:25.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.280+0000 7f8e4daa9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e4809de00 0x7f8e481265e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:25.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.280+0000 7f8e4daa9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e4809de00 0x7f8e481265e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52024/0 (socket says 172.21.15.3:52024) 2023-10-03T10:18:25.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.280+0000 7f8e4daa9700 1 -- 172.21.15.3:0/3214832980 learned_addr learned my addr 172.21.15.3:0/3214832980 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:25.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.280+0000 7f8e4eaab700 1 --2- 172.21.15.3:0/3214832980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e4813d1f0 0x7f8e48126b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:25.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.280+0000 7f8e4daa9700 1 -- 172.21.15.3:0/3214832980 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8e4806a770 msgr2=0x7f8e481260a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:25.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.280+0000 7f8e4daa9700 1 --2- 172.21.15.3:0/3214832980 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8e4806a770 0x7f8e481260a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.280+0000 7f8e4daa9700 1 -- 172.21.15.3:0/3214832980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e4813d1f0 msgr2=0x7f8e48126b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:25.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.280+0000 7f8e4daa9700 1 --2- 172.21.15.3:0/3214832980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e4813d1f0 0x7f8e48126b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.280+0000 7f8e4daa9700 1 -- 172.21.15.3:0/3214832980 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e38009550 con 0x7f8e4809de00 2023-10-03T10:18:25.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.281+0000 7f8e4daa9700 1 --2- 172.21.15.3:0/3214832980 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e4809de00 0x7f8e481265e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8e4809a9a0 tx=0x7f8e38004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:25.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.281+0000 7f8e3effd700 1 -- 172.21.15.3:0/3214832980 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e380036a0 con 0x7f8e4809de00 2023-10-03T10:18:25.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.281+0000 7f8e3effd700 1 -- 172.21.15.3:0/3214832980 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3801f070 con 0x7f8e4809de00 2023-10-03T10:18:25.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.281+0000 7f8e3effd700 1 -- 172.21.15.3:0/3214832980 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38003a60 con 0x7f8e4809de00 2023-10-03T10:18:25.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.281+0000 7f8e5050e700 1 -- 172.21.15.3:0/3214832980 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e48127a90 con 0x7f8e4809de00 2023-10-03T10:18:25.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.281+0000 7f8e5050e700 1 -- 172.21.15.3:0/3214832980 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e48071840 con 0x7f8e4809de00 2023-10-03T10:18:25.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.282+0000 7f8e3effd700 1 -- 172.21.15.3:0/3214832980 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8e38003bc0 con 0x7f8e4809de00 2023-10-03T10:18:25.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.282+0000 7f8e3effd700 1 -- 172.21.15.3:0/3214832980 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8e38021070 con 0x7f8e4809de00 2023-10-03T10:18:25.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.283+0000 7f8e5050e700 1 -- 172.21.15.3:0/3214832980 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e48074ea0 con 0x7f8e4809de00 2023-10-03T10:18:25.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.288+0000 7f8e3effd700 1 -- 172.21.15.3:0/3214832980 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8e38015080 con 0x7f8e4809de00 2023-10-03T10:18:25.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.384+0000 7fc17e42f700 1 -- 172.21.15.3:0/1217926835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc17812e6f0 con 0x7fc17806f660 2023-10-03T10:18:25.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.384+0000 7fc174ff9700 1 -- 172.21.15.3:0/1217926835 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc168021030 con 0x7fc17806f660 2023-10-03T10:18:25.385 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:25.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.387+0000 7fc17e42f700 1 -- 172.21.15.3:0/1217926835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc17806f660 msgr2=0x7fc1781c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:25.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.387+0000 7fc17e42f700 1 --2- 172.21.15.3:0/1217926835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc17806f660 0x7fc1781c8230 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fc16800e9e0 tx=0x7fc16800ecf0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.387+0000 7fc17e42f700 1 -- 172.21.15.3:0/1217926835 shutdown_connections 2023-10-03T10:18:25.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.387+0000 7fc17e42f700 1 --2- 172.21.15.3:0/1217926835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc1781305f0 0x7fc1781c8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.387+0000 7fc17e42f700 1 --2- 172.21.15.3:0/1217926835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc17806f660 0x7fc1781c8230 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.387+0000 7fc17e42f700 1 --2- 172.21.15.3:0/1217926835 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc17806ff90 0x7fc1781c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.387+0000 7fc17e42f700 1 -- 172.21.15.3:0/1217926835 >> 172.21.15.3:0/1217926835 conn(0x7fc1780a2aa0 msgr2=0x7fc1780a1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:25.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.387+0000 7fc17e42f700 1 -- 172.21.15.3:0/1217926835 shutdown_connections 2023-10-03T10:18:25.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.387+0000 7fc17e42f700 1 -- 172.21.15.3:0/1217926835 wait complete. 2023-10-03T10:18:25.401 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:25.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.462+0000 7fe1f457c700 1 -- 172.21.15.3:0/76757269 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe1ec074ea0 con 0x7fe1ec09e930 2023-10-03T10:18:25.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.462+0000 7fe1e2ffd700 1 -- 172.21.15.3:0/76757269 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe1e8015070 con 0x7fe1ec09e930 2023-10-03T10:18:25.462 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:25.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.464+0000 7fe1f457c700 1 -- 172.21.15.3:0/76757269 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1ec09e930 msgr2=0x7fe1ec122a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:25.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.464+0000 7fe1f457c700 1 --2- 172.21.15.3:0/76757269 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1ec09e930 0x7fe1ec122a50 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8016000 tx=0x7fe1e8004750 comp rx=0 tx=0).stop 2023-10-03T10:18:25.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.465+0000 7fe1f457c700 1 -- 172.21.15.3:0/76757269 shutdown_connections 2023-10-03T10:18:25.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.465+0000 7fe1f457c700 1 --2- 172.21.15.3:0/76757269 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe1ec1342f0 0x7fe1ec1234d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.465+0000 7fe1f457c700 1 --2- 172.21.15.3:0/76757269 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe1ec09e930 0x7fe1ec122a50 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.465+0000 7fe1f457c700 1 --2- 172.21.15.3:0/76757269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1ec09f240 0x7fe1ec122f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.465+0000 7fe1f457c700 1 -- 172.21.15.3:0/76757269 >> 172.21.15.3:0/76757269 conn(0x7fe1ec0a4210 msgr2=0x7fe1ec0a2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:25.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.465+0000 7fe1f457c700 1 -- 172.21.15.3:0/76757269 shutdown_connections 2023-10-03T10:18:25.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.465+0000 7fe1f457c700 1 -- 172.21.15.3:0/76757269 wait complete. 2023-10-03T10:18:25.479 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:25.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.482+0000 7fdfbf21f700 1 -- 172.21.15.3:0/2064761750 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdfb806b9e0 con 0x7fdfb8131810 2023-10-03T10:18:25.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.482+0000 7fdfb57fa700 1 -- 172.21.15.3:0/2064761750 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdfac017030 con 0x7fdfb8131810 2023-10-03T10:18:25.482 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:25.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.484+0000 7fdfbf21f700 1 -- 172.21.15.3:0/2064761750 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfb8131810 msgr2=0x7fdfb809c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:25.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.484+0000 7fdfbf21f700 1 --2- 172.21.15.3:0/2064761750 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfb8131810 0x7fdfb809c700 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00c4e0 tx=0x7fdfac00c7f0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.485+0000 7fdfbf21f700 1 -- 172.21.15.3:0/2064761750 shutdown_connections 2023-10-03T10:18:25.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.485+0000 7fdfbf21f700 1 --2- 172.21.15.3:0/2064761750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfb8136f50 0x7fdfb809d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.485+0000 7fdfbf21f700 1 --2- 172.21.15.3:0/2064761750 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfb8131810 0x7fdfb809c700 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.485+0000 7fdfbf21f700 1 --2- 172.21.15.3:0/2064761750 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdfb8134630 0x7fdfb809cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.485+0000 7fdfbf21f700 1 -- 172.21.15.3:0/2064761750 >> 172.21.15.3:0/2064761750 conn(0x7fdfb80a2aa0 msgr2=0x7fdfb806f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:25.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.485+0000 7fdfbf21f700 1 -- 172.21.15.3:0/2064761750 shutdown_connections 2023-10-03T10:18:25.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.485+0000 7fdfbf21f700 1 -- 172.21.15.3:0/2064761750 wait complete. 2023-10-03T10:18:25.498 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:25.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.523+0000 7f8e5050e700 1 -- 172.21.15.3:0/3214832980 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8e4809a3d0 con 0x7f8e4809de00 2023-10-03T10:18:25.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.523+0000 7f8e3effd700 1 -- 172.21.15.3:0/3214832980 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8e3802b030 con 0x7f8e4809de00 2023-10-03T10:18:25.524 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:25.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.526+0000 7f8e5050e700 1 -- 172.21.15.3:0/3214832980 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e4809de00 msgr2=0x7f8e481265e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:25.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.526+0000 7f8e5050e700 1 --2- 172.21.15.3:0/3214832980 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e4809de00 0x7f8e481265e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8e4809a9a0 tx=0x7f8e38004750 comp rx=0 tx=0).stop 2023-10-03T10:18:25.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.526+0000 7f8e5050e700 1 -- 172.21.15.3:0/3214832980 shutdown_connections 2023-10-03T10:18:25.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.526+0000 7f8e5050e700 1 --2- 172.21.15.3:0/3214832980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e4813d1f0 0x7f8e48126b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.526+0000 7f8e5050e700 1 --2- 172.21.15.3:0/3214832980 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8e4806a770 0x7f8e481260a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.526+0000 7f8e5050e700 1 --2- 172.21.15.3:0/3214832980 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e4809de00 0x7f8e481265e0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:25.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.526+0000 7f8e5050e700 1 -- 172.21.15.3:0/3214832980 >> 172.21.15.3:0/3214832980 conn(0x7f8e48066f50 msgr2=0x7f8e480a1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:25.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.526+0000 7f8e5050e700 1 -- 172.21.15.3:0/3214832980 shutdown_connections 2023-10-03T10:18:25.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:25.526+0000 7f8e5050e700 1 -- 172.21.15.3:0/3214832980 wait complete. 2023-10-03T10:18:25.539 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:26.402 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:26.480 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:26.499 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:26.539 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:26.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.551+0000 7fb24c4c4700 1 Processor -- start 2023-10-03T10:18:26.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.551+0000 7fb24c4c4700 1 -- start start 2023-10-03T10:18:26.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.552+0000 7fb24c4c4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb24409d6f0 0x7fb24409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:26.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.552+0000 7fb24c4c4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb24409e440 0x7fb244134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:26.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.552+0000 7fb24c4c4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb24409b120 con 0x7fb24409e000 2023-10-03T10:18:26.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.552+0000 7fb24c4c4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb24406b460 con 0x7fb24409d6f0 2023-10-03T10:18:26.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.552+0000 7fb24c4c4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb24409a7e0 con 0x7fb24409e440 2023-10-03T10:18:26.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.553+0000 7fb24aa61700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb24409e440 0x7fb244134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:26.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.553+0000 7fb24a260700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb24409d6f0 0x7fb24409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:26.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.553+0000 7fb24aa61700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb24409e440 0x7fb244134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52030/0 (socket says 172.21.15.3:52030) 2023-10-03T10:18:26.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.553+0000 7fb24aa61700 1 -- 172.21.15.3:0/901352099 learned_addr learned my addr 172.21.15.3:0/901352099 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:26.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.553+0000 7fb24a260700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb24409d6f0 0x7fb24409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58578/0 (socket says 172.21.15.3:58578) 2023-10-03T10:18:26.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.553+0000 7fb24aa61700 1 -- 172.21.15.3:0/901352099 >> v2:172.21.15.3:3300/0 conn(0x7fb24409d6f0 msgr2=0x7fb24409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:26.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.553+0000 7fb24aa61700 1 --2- 172.21.15.3:0/901352099 >> v2:172.21.15.3:3300/0 conn(0x7fb24409d6f0 0x7fb24409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.553+0000 7fb24aa61700 1 -- 172.21.15.3:0/901352099 >> v1:172.21.15.3:6789/0 conn(0x7fb24409e000 legacy=0x7fb2441339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:26.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.553+0000 7fb24aa61700 1 -- 172.21.15.3:0/901352099 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24409a500 con 0x7fb24409e440 2023-10-03T10:18:26.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.553+0000 7fb24aa61700 1 --2- 172.21.15.3:0/901352099 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb24409e440 0x7fb244134100 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb23c009e80 tx=0x7fb23c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24cde0dcfd94bfef server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:26.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.553+0000 7fb24925e700 1 -- 172.21.15.3:0/901352099 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c00e070 con 0x7fb24409e440 2023-10-03T10:18:26.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.554+0000 7fb24925e700 1 -- 172.21.15.3:0/901352099 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb23c00b8d0 con 0x7fb24409e440 2023-10-03T10:18:26.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.554+0000 7fb24925e700 1 -- 172.21.15.3:0/901352099 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c00bc80 con 0x7fb24409e440 2023-10-03T10:18:26.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.554+0000 7fb24c4c4700 1 -- 172.21.15.3:0/901352099 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb24409e440 msgr2=0x7fb244134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:26.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.554+0000 7fb24c4c4700 1 --2- 172.21.15.3:0/901352099 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb24409e440 0x7fb244134100 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb23c009e80 tx=0x7fb23c00b330 comp rx=0 tx=0).stop 2023-10-03T10:18:26.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.554+0000 7fb24c4c4700 1 -- 172.21.15.3:0/901352099 shutdown_connections 2023-10-03T10:18:26.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.554+0000 7fb24c4c4700 1 --2- 172.21.15.3:0/901352099 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb24409e440 0x7fb244134100 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.554+0000 7fb24c4c4700 1 --2- 172.21.15.3:0/901352099 >> v2:172.21.15.3:3300/0 conn(0x7fb24409d6f0 0x7fb24409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.554+0000 7fb24c4c4700 1 -- 172.21.15.3:0/901352099 >> 172.21.15.3:0/901352099 conn(0x7fb244066f50 msgr2=0x7fb24409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:26.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.555+0000 7fb24c4c4700 1 -- 172.21.15.3:0/901352099 shutdown_connections 2023-10-03T10:18:26.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.555+0000 7fb24c4c4700 1 -- 172.21.15.3:0/901352099 wait complete. 2023-10-03T10:18:26.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.556+0000 7fb24c4c4700 1 Processor -- start 2023-10-03T10:18:26.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.556+0000 7fb24c4c4700 1 -- start start 2023-10-03T10:18:26.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.556+0000 7fb24c4c4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb24409d6f0 0x7fb2441a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:26.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.556+0000 7fb24c4c4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb24409e000 0x7fb2441a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:26.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.556+0000 7fb24c4c4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb24409e440 0x7fb2441a2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:26.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.556+0000 7fb24c4c4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2441a5f50 con 0x7fb24409e440 2023-10-03T10:18:26.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.556+0000 7fb24c4c4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2441a5590 con 0x7fb24409d6f0 2023-10-03T10:18:26.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.556+0000 7fb24c4c4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2441a48d0 con 0x7fb24409e000 2023-10-03T10:18:26.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.556+0000 7fb24aa61700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb24409e440 0x7fb2441a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:26.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.556+0000 7fb24aa61700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb24409e440 0x7fb2441a2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58594/0 (socket says 172.21.15.3:58594) 2023-10-03T10:18:26.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.556+0000 7fb24aa61700 1 -- 172.21.15.3:0/3552701563 learned_addr learned my addr 172.21.15.3:0/3552701563 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:26.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.556+0000 7fb249a5f700 1 --2- 172.21.15.3:0/3552701563 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb24409e000 0x7fb2441a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:26.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.557+0000 7fb24a260700 1 --2- 172.21.15.3:0/3552701563 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb24409d6f0 0x7fb2441a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:26.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.557+0000 7fb24aa61700 1 -- 172.21.15.3:0/3552701563 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb24409e000 msgr2=0x7fb2441a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:26.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.557+0000 7fb24aa61700 1 --2- 172.21.15.3:0/3552701563 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb24409e000 0x7fb2441a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.557+0000 7fb24aa61700 1 -- 172.21.15.3:0/3552701563 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb24409d6f0 msgr2=0x7fb2441a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:26.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.557+0000 7fb24aa61700 1 --2- 172.21.15.3:0/3552701563 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb24409d6f0 0x7fb2441a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.557+0000 7fb24aa61700 1 -- 172.21.15.3:0/3552701563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23c009b30 con 0x7fb24409e440 2023-10-03T10:18:26.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.557+0000 7fb24aa61700 1 --2- 172.21.15.3:0/3552701563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb24409e440 0x7fb2441a2540 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fb24409a920 tx=0x7fb23c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:26.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.557+0000 7fb23affd700 1 -- 172.21.15.3:0/3552701563 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c004800 con 0x7fb24409e440 2023-10-03T10:18:26.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.558+0000 7fb23affd700 1 -- 172.21.15.3:0/3552701563 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb23c004960 con 0x7fb24409e440 2023-10-03T10:18:26.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.558+0000 7fb24c4c4700 1 -- 172.21.15.3:0/3552701563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2440a2000 con 0x7fb24409e440 2023-10-03T10:18:26.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.558+0000 7fb23affd700 1 -- 172.21.15.3:0/3552701563 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c01c680 con 0x7fb24409e440 2023-10-03T10:18:26.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.558+0000 7fb24c4c4700 1 -- 172.21.15.3:0/3552701563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2441a4d20 con 0x7fb24409e440 2023-10-03T10:18:26.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.559+0000 7fb23affd700 1 -- 172.21.15.3:0/3552701563 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb23c01c7e0 con 0x7fb24409e440 2023-10-03T10:18:26.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.559+0000 7fb23affd700 1 -- 172.21.15.3:0/3552701563 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb23c041510 con 0x7fb24409e440 2023-10-03T10:18:26.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.560+0000 7fb238ff9700 1 -- 172.21.15.3:0/3552701563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2441a4f50 con 0x7fb24409e440 2023-10-03T10:18:26.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.569+0000 7fb23affd700 1 -- 172.21.15.3:0/3552701563 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb2441a4f50 con 0x7fb24409e440 2023-10-03T10:18:26.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.633+0000 7f86c0ed4700 1 Processor -- start 2023-10-03T10:18:26.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.634+0000 7f86c0ed4700 1 -- start start 2023-10-03T10:18:26.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.635+0000 7f86c0ed4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f86bc09e850 0x7f86bc09ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:26.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.635+0000 7f86c0ed4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86bc09f160 0x7f86bc133bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:26.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.635+0000 7f86c0ed4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86bc134130 0x7f86bc1345b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:26.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.635+0000 7f86c0ed4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86bc09ae60 con 0x7f86bc09e850 2023-10-03T10:18:26.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.635+0000 7f86c0ed4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc09f960 con 0x7f86bc09f160 2023-10-03T10:18:26.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.635+0000 7f86c0ed4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc06cb10 con 0x7f86bc134130 2023-10-03T10:18:26.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.635+0000 7f86b9d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86bc09f160 0x7f86bc133bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:26.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.635+0000 7f86b9d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86bc09f160 0x7f86bc133bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52042/0 (socket says 172.21.15.3:52042) 2023-10-03T10:18:26.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.635+0000 7f86b9d9b700 1 -- 172.21.15.3:0/1060537053 learned_addr learned my addr 172.21.15.3:0/1060537053 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:26.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.635+0000 7f86bad9d700 1 --2- 172.21.15.3:0/1060537053 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86bc134130 0x7f86bc1345b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:26.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.635+0000 7f86ba59c700 1 --2- 172.21.15.3:0/1060537053 >> v2:172.21.15.3:3300/0 conn(0x7f86bc09e850 0x7f86bc09ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:26.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.635+0000 7f86b9d9b700 1 -- 172.21.15.3:0/1060537053 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86bc134130 msgr2=0x7f86bc1345b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:26.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.635+0000 7f86b9d9b700 1 --2- 172.21.15.3:0/1060537053 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86bc134130 0x7f86bc1345b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.635+0000 7f86b9d9b700 1 -- 172.21.15.3:0/1060537053 >> v2:172.21.15.3:3300/0 conn(0x7f86bc09e850 msgr2=0x7f86bc09ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:26.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.635+0000 7f86b9d9b700 1 --2- 172.21.15.3:0/1060537053 >> v2:172.21.15.3:3300/0 conn(0x7f86bc09e850 0x7f86bc09ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.635+0000 7f86b9d9b700 1 -- 172.21.15.3:0/1060537053 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc06b990 con 0x7f86bc09f160 2023-10-03T10:18:26.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.635+0000 7f86bad9d700 1 --2- 172.21.15.3:0/1060537053 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86bc134130 0x7f86bc1345b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:26.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.635+0000 7f86b9d9b700 1 --2- 172.21.15.3:0/1060537053 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86bc09f160 0x7f86bc133bf0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f86b00098f0 tx=0x7f86b0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b02669b89349f66 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:26.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.636+0000 7f86b959a700 1 -- 172.21.15.3:0/1060537053 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b0011830 con 0x7f86bc09f160 2023-10-03T10:18:26.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.636+0000 7f86b959a700 1 -- 172.21.15.3:0/1060537053 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b0011990 con 0x7f86bc09f160 2023-10-03T10:18:26.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.636+0000 7f86b959a700 1 -- 172.21.15.3:0/1060537053 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b0011d20 con 0x7f86bc09f160 2023-10-03T10:18:26.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.636+0000 7f86c0ed4700 1 -- 172.21.15.3:0/1060537053 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86bc09f160 msgr2=0x7f86bc133bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:26.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.636+0000 7f86c0ed4700 1 --2- 172.21.15.3:0/1060537053 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86bc09f160 0x7f86bc133bf0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f86b00098f0 tx=0x7f86b0009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:26.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.637+0000 7f86c0ed4700 1 -- 172.21.15.3:0/1060537053 shutdown_connections 2023-10-03T10:18:26.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.637+0000 7f86c0ed4700 1 --2- 172.21.15.3:0/1060537053 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86bc134130 0x7f86bc1345b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.637+0000 7f86c0ed4700 1 --2- 172.21.15.3:0/1060537053 >> v2:172.21.15.3:3300/0 conn(0x7f86bc09e850 0x7f86bc09ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.637+0000 7f86c0ed4700 1 --2- 172.21.15.3:0/1060537053 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86bc09f160 0x7f86bc133bf0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.637+0000 7f86c0ed4700 1 -- 172.21.15.3:0/1060537053 >> 172.21.15.3:0/1060537053 conn(0x7f86bc0a3f70 msgr2=0x7f86bc0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:26.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.637+0000 7f86c0ed4700 1 -- 172.21.15.3:0/1060537053 shutdown_connections 2023-10-03T10:18:26.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.637+0000 7f86c0ed4700 1 -- 172.21.15.3:0/1060537053 wait complete. 2023-10-03T10:18:26.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.638+0000 7f86c0ed4700 1 Processor -- start 2023-10-03T10:18:26.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.638+0000 7f86c0ed4700 1 -- start start 2023-10-03T10:18:26.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.638+0000 7f86c0ed4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86bc09e850 0x7f86bc1225a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:26.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.638+0000 7f86c0ed4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86bc09f160 0x7f86bc122ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:26.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.638+0000 7f86c0ed4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86bc134130 0x7f86bc123020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:26.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.638+0000 7f86c0ed4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc125de0 con 0x7f86bc134130 2023-10-03T10:18:26.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.638+0000 7f86c0ed4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc125040 con 0x7f86bc09e850 2023-10-03T10:18:26.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.638+0000 7f86c0ed4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc06cd30 con 0x7f86bc09f160 2023-10-03T10:18:26.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.639+0000 7f86ba59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86bc09e850 0x7f86bc1225a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:26.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.639+0000 7f86ba59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86bc09e850 0x7f86bc1225a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52046/0 (socket says 172.21.15.3:52046) 2023-10-03T10:18:26.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.639+0000 7f86ba59c700 1 -- 172.21.15.3:0/2262076644 learned_addr learned my addr 172.21.15.3:0/2262076644 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:26.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.639+0000 7f86bad9d700 1 --2- 172.21.15.3:0/2262076644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86bc134130 0x7f86bc123020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:26.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.639+0000 7f86b9d9b700 1 --2- 172.21.15.3:0/2262076644 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86bc09f160 0x7f86bc122ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:26.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.639+0000 7f86bad9d700 1 -- 172.21.15.3:0/2262076644 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86bc09f160 msgr2=0x7f86bc122ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:26.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.639+0000 7f86bad9d700 1 --2- 172.21.15.3:0/2262076644 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86bc09f160 0x7f86bc122ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.639+0000 7f86bad9d700 1 -- 172.21.15.3:0/2262076644 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86bc09e850 msgr2=0x7f86bc1225a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:26.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.639+0000 7f86bad9d700 1 --2- 172.21.15.3:0/2262076644 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86bc09e850 0x7f86bc1225a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.639+0000 7f86bad9d700 1 -- 172.21.15.3:0/2262076644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b4009a70 con 0x7f86bc134130 2023-10-03T10:18:26.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.639+0000 7f86bad9d700 1 --2- 172.21.15.3:0/2262076644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86bc134130 0x7f86bc123020 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009e50 tx=0x7f86b400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:26.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.640+0000 7f86aaffd700 1 -- 172.21.15.3:0/2262076644 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b400ca90 con 0x7f86bc134130 2023-10-03T10:18:26.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.640+0000 7f86c0ed4700 1 -- 172.21.15.3:0/2262076644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b0009550 con 0x7f86bc134130 2023-10-03T10:18:26.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.640+0000 7f86c0ed4700 1 -- 172.21.15.3:0/2262076644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86bc126110 con 0x7f86bc134130 2023-10-03T10:18:26.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.640+0000 7f86aaffd700 1 -- 172.21.15.3:0/2262076644 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b400f040 con 0x7f86bc134130 2023-10-03T10:18:26.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.640+0000 7f86aaffd700 1 -- 172.21.15.3:0/2262076644 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b400cdf0 con 0x7f86bc134130 2023-10-03T10:18:26.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.641+0000 7f86aaffd700 1 -- 172.21.15.3:0/2262076644 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f86b4003a20 con 0x7f86bc134130 2023-10-03T10:18:26.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.641+0000 7f86aaffd700 1 -- 172.21.15.3:0/2262076644 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f86b403fbd0 con 0x7f86bc134130 2023-10-03T10:18:26.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.642+0000 7f86c0ed4700 1 -- 172.21.15.3:0/2262076644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86a00052e0 con 0x7f86bc134130 2023-10-03T10:18:26.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.647+0000 7f86aaffd700 1 -- 172.21.15.3:0/2262076644 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f86b4012750 con 0x7f86bc134130 2023-10-03T10:18:26.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.716+0000 7f5f17e6d700 1 Processor -- start 2023-10-03T10:18:26.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.717+0000 7f5f17e6d700 1 -- start start 2023-10-03T10:18:26.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.717+0000 7f5f17e6d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f100a0a30 0x7f5f100a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:26.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.717+0000 7f5f17e6d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f10138d80 0x7f5f1013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:26.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.717+0000 7f5f17e6d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f100a2a70 con 0x7f5f1009d630 2023-10-03T10:18:26.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.717+0000 7f5f17e6d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f100a01d0 con 0x7f5f100a0a30 2023-10-03T10:18:26.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.717+0000 7f5f17e6d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1009e2b0 con 0x7f5f10138d80 2023-10-03T10:18:26.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.717+0000 7f5f15c09700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f100a0a30 0x7f5f100a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:26.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.718+0000 7f5f15c09700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f100a0a30 0x7f5f100a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52062/0 (socket says 172.21.15.3:52062) 2023-10-03T10:18:26.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.718+0000 7f5f15c09700 1 -- 172.21.15.3:0/3743786282 learned_addr learned my addr 172.21.15.3:0/3743786282 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:26.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.718+0000 7f5f1640a700 1 --2- 172.21.15.3:0/3743786282 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f10138d80 0x7f5f1013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:26.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.718+0000 7f5f14c07700 1 -- 172.21.15.3:0/3743786282 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1320663043 0 0) 0x7f5f100a2a70 con 0x7f5f1009d630 2023-10-03T10:18:26.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.718+0000 7f5f14c07700 1 -- 172.21.15.3:0/3743786282 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5efc0035b0 con 0x7f5f1009d630 2023-10-03T10:18:26.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.718+0000 7f5f15c09700 1 -- 172.21.15.3:0/3743786282 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f10138d80 msgr2=0x7f5f1013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:26.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.718+0000 7f5f15c09700 1 --2- 172.21.15.3:0/3743786282 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f10138d80 0x7f5f1013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.718+0000 7f5f15c09700 1 -- 172.21.15.3:0/3743786282 >> v1:172.21.15.3:6789/0 conn(0x7f5f1009d630 legacy=0x7f5f100a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:26.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.718+0000 7f5f15c09700 1 -- 172.21.15.3:0/3743786282 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1009dc50 con 0x7f5f100a0a30 2023-10-03T10:18:26.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.718+0000 7f5f15c09700 1 --2- 172.21.15.3:0/3743786282 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f100a0a30 0x7f5f100a0e00 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c009920 tx=0x7f5f0c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbf24934ef82aa02 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:26.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.719+0000 7f5f14c07700 1 -- 172.21.15.3:0/3743786282 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c011830 con 0x7f5f100a0a30 2023-10-03T10:18:26.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.719+0000 7f5f14c07700 1 -- 172.21.15.3:0/3743786282 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0c011990 con 0x7f5f100a0a30 2023-10-03T10:18:26.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.719+0000 7f5f14c07700 1 -- 172.21.15.3:0/3743786282 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c011d20 con 0x7f5f100a0a30 2023-10-03T10:18:26.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.719+0000 7f5f17e6d700 1 -- 172.21.15.3:0/3743786282 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f100a0a30 msgr2=0x7f5f100a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:26.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.719+0000 7f5f17e6d700 1 --2- 172.21.15.3:0/3743786282 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f100a0a30 0x7f5f100a0e00 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c009920 tx=0x7f5f0c009c30 comp rx=0 tx=0).stop 2023-10-03T10:18:26.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.719+0000 7f5f17e6d700 1 -- 172.21.15.3:0/3743786282 shutdown_connections 2023-10-03T10:18:26.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.719+0000 7f5f17e6d700 1 --2- 172.21.15.3:0/3743786282 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f100a0a30 0x7f5f100a0e00 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.719+0000 7f5f17e6d700 1 --2- 172.21.15.3:0/3743786282 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f10138d80 0x7f5f1013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.719+0000 7f5f17e6d700 1 -- 172.21.15.3:0/3743786282 >> 172.21.15.3:0/3743786282 conn(0x7f5f100a3050 msgr2=0x7f5f100a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:26.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.719+0000 7f5f17e6d700 1 -- 172.21.15.3:0/3743786282 shutdown_connections 2023-10-03T10:18:26.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.719+0000 7f5f17e6d700 1 -- 172.21.15.3:0/3743786282 wait complete. 2023-10-03T10:18:26.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.720+0000 7f5f17e6d700 1 Processor -- start 2023-10-03T10:18:26.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.721+0000 7f5f17e6d700 1 -- start start 2023-10-03T10:18:26.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.721+0000 7f5f17e6d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f1009d630 0x7f5f10133730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:26.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.721+0000 7f5f17e6d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f100a0a30 0x7f5f10133c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:26.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.721+0000 7f5f17e6d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f10138d80 0x7f5f101341b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:26.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.721+0000 7f5f17e6d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f101361c0 con 0x7f5f10138d80 2023-10-03T10:18:26.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.721+0000 7f5f17e6d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f10136ee0 con 0x7f5f100a0a30 2023-10-03T10:18:26.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.721+0000 7f5f17e6d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f100a4bb0 con 0x7f5f1009d630 2023-10-03T10:18:26.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.721+0000 7f5f15408700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f100a0a30 0x7f5f10133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:26.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.721+0000 7f5f15408700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f100a0a30 0x7f5f10133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52074/0 (socket says 172.21.15.3:52074) 2023-10-03T10:18:26.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.721+0000 7f5f15408700 1 -- 172.21.15.3:0/2805308552 learned_addr learned my addr 172.21.15.3:0/2805308552 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:26.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.721+0000 7f5f1640a700 1 --2- 172.21.15.3:0/2805308552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f10138d80 0x7f5f101341b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:26.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.722+0000 7f5f15408700 1 -- 172.21.15.3:0/2805308552 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f1009d630 msgr2=0x7f5f10133730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:26.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.722+0000 7f5f15c09700 1 --2- 172.21.15.3:0/2805308552 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f1009d630 0x7f5f10133730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:26.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.722+0000 7f5f15408700 1 --2- 172.21.15.3:0/2805308552 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f1009d630 0x7f5f10133730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.722+0000 7f5f15408700 1 -- 172.21.15.3:0/2805308552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f10138d80 msgr2=0x7f5f101341b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:26.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.722+0000 7f5f15408700 1 --2- 172.21.15.3:0/2805308552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f10138d80 0x7f5f101341b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.722+0000 7f5f15408700 1 -- 172.21.15.3:0/2805308552 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0c009550 con 0x7f5f100a0a30 2023-10-03T10:18:26.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.722+0000 7f5f1640a700 1 --2- 172.21.15.3:0/2805308552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f10138d80 0x7f5f101341b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:18:26.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.722+0000 7f5f15c09700 1 --2- 172.21.15.3:0/2805308552 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f1009d630 0x7f5f10133730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:18:26.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.722+0000 7f5f15408700 1 --2- 172.21.15.3:0/2805308552 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f100a0a30 0x7f5f10133c70 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000c4f0 tx=0x7f5f0000c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:26.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.722+0000 7f5f067fc700 1 -- 172.21.15.3:0/2805308552 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f000052d0 con 0x7f5f100a0a30 2023-10-03T10:18:26.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.722+0000 7f5f067fc700 1 -- 172.21.15.3:0/2805308552 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f00005430 con 0x7f5f100a0a30 2023-10-03T10:18:26.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.722+0000 7f5f17e6d700 1 -- 172.21.15.3:0/2805308552 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f100a0410 con 0x7f5f100a0a30 2023-10-03T10:18:26.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.722+0000 7f5f067fc700 1 -- 172.21.15.3:0/2805308552 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f00003880 con 0x7f5f100a0a30 2023-10-03T10:18:26.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.722+0000 7f5f17e6d700 1 -- 172.21.15.3:0/2805308552 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f10137950 con 0x7f5f100a0a30 2023-10-03T10:18:26.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.723+0000 7f5f067fc700 1 -- 172.21.15.3:0/2805308552 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5f000039e0 con 0x7f5f100a0a30 2023-10-03T10:18:26.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.723+0000 7f5f067fc700 1 -- 172.21.15.3:0/2805308552 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5f0003ec00 con 0x7f5f100a0a30 2023-10-03T10:18:26.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.723+0000 7f5f17e6d700 1 -- 172.21.15.3:0/2805308552 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f100747f0 con 0x7f5f100a0a30 2023-10-03T10:18:26.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.727+0000 7f46ecbc0700 1 Processor -- start 2023-10-03T10:18:26.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.727+0000 7f46ecbc0700 1 -- start start 2023-10-03T10:18:26.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.728+0000 7f46ecbc0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46e809d6f0 0x7f46e809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:26.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.728+0000 7f46ecbc0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f46e809e000 0x7f46e8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:26.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.728+0000 7f46ecbc0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46e809e410 0x7f46e8130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:26.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.728+0000 7f46ecbc0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e809f520 con 0x7f46e809e000 2023-10-03T10:18:26.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.728+0000 7f46ecbc0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e809f130 con 0x7f46e809e410 2023-10-03T10:18:26.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.728+0000 7f46ecbc0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f46e809e930 con 0x7f46e809d6f0 2023-10-03T10:18:26.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.728+0000 7f46e659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46e809d6f0 0x7f46e809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:26.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.728+0000 7f46e659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46e809d6f0 0x7f46e809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52242/0 (socket says 172.21.15.3:52242) 2023-10-03T10:18:26.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.728+0000 7f46e659c700 1 -- 172.21.15.3:0/2257642979 learned_addr learned my addr 172.21.15.3:0/2257642979 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:26.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.728+0000 7f46e6d9d700 1 --2- 172.21.15.3:0/2257642979 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46e809e410 0x7f46e8130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:26.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.729+0000 7f46e5d9b700 1 --2- 172.21.15.3:0/2257642979 >> v2:172.21.15.3:3300/0 conn(0x7f46e809e000 0x7f46e8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:26.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.729+0000 7f46e6d9d700 1 -- 172.21.15.3:0/2257642979 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46e809d6f0 msgr2=0x7f46e809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:26.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.729+0000 7f46e6d9d700 1 --2- 172.21.15.3:0/2257642979 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46e809d6f0 0x7f46e809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.729+0000 7f46e6d9d700 1 -- 172.21.15.3:0/2257642979 >> v2:172.21.15.3:3300/0 conn(0x7f46e809e000 msgr2=0x7f46e8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:26.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.729+0000 7f46e6d9d700 1 --2- 172.21.15.3:0/2257642979 >> v2:172.21.15.3:3300/0 conn(0x7f46e809e000 0x7f46e8130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.729+0000 7f46e6d9d700 1 -- 172.21.15.3:0/2257642979 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e809b2d0 con 0x7f46e809e410 2023-10-03T10:18:26.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.729+0000 7f46e6d9d700 1 --2- 172.21.15.3:0/2257642979 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46e809e410 0x7f46e8130ba0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f46e0009d90 tx=0x7f46e000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f94f38db9b22e39 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:26.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.729+0000 7f5f067fc700 1 -- 172.21.15.3:0/2805308552 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5f00017080 con 0x7f5f100a0a30 2023-10-03T10:18:26.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.730+0000 7f46e559a700 1 -- 172.21.15.3:0/2257642979 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e000e070 con 0x7f46e809e410 2023-10-03T10:18:26.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.730+0000 7f46e559a700 1 -- 172.21.15.3:0/2257642979 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e000bb20 con 0x7f46e809e410 2023-10-03T10:18:26.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.730+0000 7f46e559a700 1 -- 172.21.15.3:0/2257642979 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e0012430 con 0x7f46e809e410 2023-10-03T10:18:26.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.731+0000 7f46ecbc0700 1 -- 172.21.15.3:0/2257642979 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46e809e410 msgr2=0x7f46e8130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:26.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.731+0000 7f46ecbc0700 1 --2- 172.21.15.3:0/2257642979 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46e809e410 0x7f46e8130ba0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f46e0009d90 tx=0x7f46e000b330 comp rx=0 tx=0).stop 2023-10-03T10:18:26.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.731+0000 7f46ecbc0700 1 -- 172.21.15.3:0/2257642979 shutdown_connections 2023-10-03T10:18:26.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.731+0000 7f46ecbc0700 1 --2- 172.21.15.3:0/2257642979 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46e809e410 0x7f46e8130ba0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.731+0000 7f46ecbc0700 1 --2- 172.21.15.3:0/2257642979 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46e809d6f0 0x7f46e809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.731+0000 7f46ecbc0700 1 --2- 172.21.15.3:0/2257642979 >> v2:172.21.15.3:3300/0 conn(0x7f46e809e000 0x7f46e8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.731+0000 7f46ecbc0700 1 -- 172.21.15.3:0/2257642979 >> 172.21.15.3:0/2257642979 conn(0x7f46e8066f50 msgr2=0x7f46e809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:26.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.732+0000 7f46e5d9b700 1 --2- 172.21.15.3:0/2257642979 >> v2:172.21.15.3:3300/0 conn(0x7f46e809e000 0x7f46e8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:26.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.732+0000 7f46ecbc0700 1 -- 172.21.15.3:0/2257642979 shutdown_connections 2023-10-03T10:18:26.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.732+0000 7f46ecbc0700 1 -- 172.21.15.3:0/2257642979 wait complete. 2023-10-03T10:18:26.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.733+0000 7f46ecbc0700 1 Processor -- start 2023-10-03T10:18:26.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.734+0000 7f46ecbc0700 1 -- start start 2023-10-03T10:18:26.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.734+0000 7f46ecbc0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46e809d6f0 0x7f46e811df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:26.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.734+0000 7f46ecbc0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46e809e410 0x7f46e811e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:26.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.734+0000 7f46ecbc0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46e811ea00 0x7f46e81ca470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:26.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.734+0000 7f46ecbc0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e8120140 con 0x7f46e809e410 2023-10-03T10:18:26.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.734+0000 7f46ecbc0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e8121a00 con 0x7f46e811ea00 2023-10-03T10:18:26.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.734+0000 7f46ecbc0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f46e8120dd0 con 0x7f46e809d6f0 2023-10-03T10:18:26.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.735+0000 7f46e659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46e809d6f0 0x7f46e811df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:26.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.735+0000 7f46e5d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46e809e410 0x7f46e811e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:26.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.735+0000 7f46e659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46e809d6f0 0x7f46e811df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52256/0 (socket says 172.21.15.3:52256) 2023-10-03T10:18:26.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.735+0000 7f46e659c700 1 -- 172.21.15.3:0/80399623 learned_addr learned my addr 172.21.15.3:0/80399623 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:26.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.735+0000 7f46e6d9d700 1 --2- 172.21.15.3:0/80399623 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46e811ea00 0x7f46e81ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:26.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.735+0000 7f46e659c700 1 -- 172.21.15.3:0/80399623 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46e811ea00 msgr2=0x7f46e81ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:26.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.735+0000 7f46e659c700 1 --2- 172.21.15.3:0/80399623 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46e811ea00 0x7f46e81ca470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.735+0000 7f46e659c700 1 -- 172.21.15.3:0/80399623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46e809e410 msgr2=0x7f46e811e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:26.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.735+0000 7f46e659c700 1 --2- 172.21.15.3:0/80399623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46e809e410 0x7f46e811e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.735+0000 7f46e659c700 1 -- 172.21.15.3:0/80399623 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e0009a70 con 0x7f46e809d6f0 2023-10-03T10:18:26.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.735+0000 7f46e5d9b700 1 --2- 172.21.15.3:0/80399623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46e809e410 0x7f46e811e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:26.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.735+0000 7f46e659c700 1 --2- 172.21.15.3:0/80399623 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46e809d6f0 0x7f46e811df80 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f46d8014140 tx=0x7f46d8014450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:26.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.735+0000 7f46d6ffd700 1 -- 172.21.15.3:0/80399623 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d8016050 con 0x7f46e809d6f0 2023-10-03T10:18:26.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.735+0000 7f46ecbc0700 1 -- 172.21.15.3:0/80399623 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e80a1890 con 0x7f46e809d6f0 2023-10-03T10:18:26.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.736+0000 7f46ecbc0700 1 -- 172.21.15.3:0/80399623 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e811faf0 con 0x7f46e809d6f0 2023-10-03T10:18:26.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.736+0000 7f46d6ffd700 1 -- 172.21.15.3:0/80399623 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d8017040 con 0x7f46e809d6f0 2023-10-03T10:18:26.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.736+0000 7f46d6ffd700 1 -- 172.21.15.3:0/80399623 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d8014ee0 con 0x7f46e809d6f0 2023-10-03T10:18:26.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.737+0000 7f46d6ffd700 1 -- 172.21.15.3:0/80399623 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f46d8005c20 con 0x7f46e809d6f0 2023-10-03T10:18:26.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.737+0000 7f46ecbc0700 1 -- 172.21.15.3:0/80399623 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46cc0052e0 con 0x7f46e809d6f0 2023-10-03T10:18:26.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.737+0000 7f46d6ffd700 1 -- 172.21.15.3:0/80399623 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f46d80477f0 con 0x7f46e809d6f0 2023-10-03T10:18:26.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.742+0000 7f46d6ffd700 1 -- 172.21.15.3:0/80399623 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f46d8023370 con 0x7f46e809d6f0 2023-10-03T10:18:26.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.879+0000 7f86c0ed4700 1 -- 172.21.15.3:0/2262076644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f86a0005590 con 0x7f86bc134130 2023-10-03T10:18:26.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.879+0000 7f86aaffd700 1 -- 172.21.15.3:0/2262076644 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f86b401c3b0 con 0x7f86bc134130 2023-10-03T10:18:26.880 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:26.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.881+0000 7f86c0ed4700 1 -- 172.21.15.3:0/2262076644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86bc134130 msgr2=0x7f86bc123020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:26.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.881+0000 7f86c0ed4700 1 --2- 172.21.15.3:0/2262076644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86bc134130 0x7f86bc123020 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009e50 tx=0x7f86b400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.881+0000 7f86c0ed4700 1 -- 172.21.15.3:0/2262076644 shutdown_connections 2023-10-03T10:18:26.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.881+0000 7f86c0ed4700 1 --2- 172.21.15.3:0/2262076644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86bc134130 0x7f86bc123020 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.881+0000 7f86c0ed4700 1 --2- 172.21.15.3:0/2262076644 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f86bc09e850 0x7f86bc1225a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.881+0000 7f86c0ed4700 1 --2- 172.21.15.3:0/2262076644 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f86bc09f160 0x7f86bc122ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.881+0000 7f86c0ed4700 1 -- 172.21.15.3:0/2262076644 >> 172.21.15.3:0/2262076644 conn(0x7f86bc0a3f70 msgr2=0x7f86bc070a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:26.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.882+0000 7f86c0ed4700 1 -- 172.21.15.3:0/2262076644 shutdown_connections 2023-10-03T10:18:26.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.882+0000 7f86c0ed4700 1 -- 172.21.15.3:0/2262076644 wait complete. 2023-10-03T10:18:26.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.890+0000 7fb238ff9700 1 -- 172.21.15.3:0/3552701563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb2441a4f50 con 0x7fb24409e440 2023-10-03T10:18:26.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.891+0000 7fb23affd700 1 -- 172.21.15.3:0/3552701563 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb2441a4f50 con 0x7fb24409e440 2023-10-03T10:18:26.891 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:26.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.893+0000 7fb238ff9700 1 -- 172.21.15.3:0/3552701563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb24409e440 msgr2=0x7fb2441a2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:26.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.893+0000 7fb238ff9700 1 --2- 172.21.15.3:0/3552701563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb24409e440 0x7fb2441a2540 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fb24409a920 tx=0x7fb23c004ad0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.893+0000 7fb238ff9700 1 -- 172.21.15.3:0/3552701563 shutdown_connections 2023-10-03T10:18:26.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.893+0000 7fb238ff9700 1 --2- 172.21.15.3:0/3552701563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb24409e440 0x7fb2441a2540 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.893+0000 7fb238ff9700 1 --2- 172.21.15.3:0/3552701563 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb24409d6f0 0x7fb2441a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.893+0000 7fb238ff9700 1 --2- 172.21.15.3:0/3552701563 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb24409e000 0x7fb2441a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.893+0000 7fb238ff9700 1 -- 172.21.15.3:0/3552701563 >> 172.21.15.3:0/3552701563 conn(0x7fb244066f50 msgr2=0x7fb2441b6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:26.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.893+0000 7fb238ff9700 1 -- 172.21.15.3:0/3552701563 shutdown_connections 2023-10-03T10:18:26.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.893+0000 7fb238ff9700 1 -- 172.21.15.3:0/3552701563 wait complete. 2023-10-03T10:18:26.896 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:26.908 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:26.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.938+0000 7f5f17e6d700 1 -- 172.21.15.3:0/2805308552 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5f10072980 con 0x7f5f100a0a30 2023-10-03T10:18:26.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.939+0000 7f5f067fc700 1 -- 172.21.15.3:0/2805308552 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5f00019b40 con 0x7f5f100a0a30 2023-10-03T10:18:26.939 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:26.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.941+0000 7f5f17e6d700 1 -- 172.21.15.3:0/2805308552 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f100a0a30 msgr2=0x7f5f10133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:26.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.941+0000 7f5f17e6d700 1 --2- 172.21.15.3:0/2805308552 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f100a0a30 0x7f5f10133c70 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000c4f0 tx=0x7f5f0000c8b0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.941+0000 7f5f17e6d700 1 -- 172.21.15.3:0/2805308552 shutdown_connections 2023-10-03T10:18:26.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.941+0000 7f5f17e6d700 1 --2- 172.21.15.3:0/2805308552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f10138d80 0x7f5f101341b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.941+0000 7f5f17e6d700 1 --2- 172.21.15.3:0/2805308552 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5f1009d630 0x7f5f10133730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.941+0000 7f5f17e6d700 1 --2- 172.21.15.3:0/2805308552 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5f100a0a30 0x7f5f10133c70 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.941+0000 7f5f17e6d700 1 -- 172.21.15.3:0/2805308552 >> 172.21.15.3:0/2805308552 conn(0x7f5f100a3050 msgr2=0x7f5f1013ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:26.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.941+0000 7f5f17e6d700 1 -- 172.21.15.3:0/2805308552 shutdown_connections 2023-10-03T10:18:26.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.941+0000 7f5f17e6d700 1 -- 172.21.15.3:0/2805308552 wait complete. 2023-10-03T10:18:26.955 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:26.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.981+0000 7f46ecbc0700 1 -- 172.21.15.3:0/80399623 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f46cc005590 con 0x7f46e809d6f0 2023-10-03T10:18:26.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.981+0000 7f46d6ffd700 1 -- 172.21.15.3:0/80399623 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f46d8033090 con 0x7f46e809d6f0 2023-10-03T10:18:26.982 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:26.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.983+0000 7f46ecbc0700 1 -- 172.21.15.3:0/80399623 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46e809d6f0 msgr2=0x7f46e811df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:26.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.983+0000 7f46ecbc0700 1 --2- 172.21.15.3:0/80399623 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46e809d6f0 0x7f46e811df80 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f46d8014140 tx=0x7f46d8014450 comp rx=0 tx=0).stop 2023-10-03T10:18:26.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.983+0000 7f46ecbc0700 1 -- 172.21.15.3:0/80399623 shutdown_connections 2023-10-03T10:18:26.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.983+0000 7f46ecbc0700 1 --2- 172.21.15.3:0/80399623 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f46e811ea00 0x7f46e81ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.983+0000 7f46ecbc0700 1 --2- 172.21.15.3:0/80399623 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f46e809d6f0 0x7f46e811df80 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.983+0000 7f46ecbc0700 1 --2- 172.21.15.3:0/80399623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46e809e410 0x7f46e811e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:26.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.983+0000 7f46ecbc0700 1 -- 172.21.15.3:0/80399623 >> 172.21.15.3:0/80399623 conn(0x7f46e8066f50 msgr2=0x7f46e81309b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:26.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.983+0000 7f46ecbc0700 1 -- 172.21.15.3:0/80399623 shutdown_connections 2023-10-03T10:18:26.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:26.983+0000 7f46ecbc0700 1 -- 172.21.15.3:0/80399623 wait complete. 2023-10-03T10:18:26.997 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:27.897 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:27.909 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:27.956 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:27.998 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:28.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.049+0000 7f6aa5998700 1 Processor -- start 2023-10-03T10:18:28.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.049+0000 7f6aa5998700 1 -- start start 2023-10-03T10:18:28.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.049+0000 7f6aa5998700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6aa012cf60 0x7f6aa006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:28.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.049+0000 7f6aa5998700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6aa012f8b0 0x7f6aa012fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:28.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.049+0000 7f6aa5998700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa012cd90 con 0x7f6aa0130260 2023-10-03T10:18:28.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.049+0000 7f6aa5998700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aa012d330 con 0x7f6aa012cf60 2023-10-03T10:18:28.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.049+0000 7f6aa5998700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa01306d0 con 0x7f6aa012f8b0 2023-10-03T10:18:28.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.050+0000 7f6a9e7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6aa012f8b0 0x7f6aa012fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:28.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.050+0000 7f6a9effd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6aa012cf60 0x7f6aa006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:28.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.050+0000 7f6a9e7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6aa012f8b0 0x7f6aa012fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52264/0 (socket says 172.21.15.3:52264) 2023-10-03T10:18:28.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.050+0000 7f6a9e7fc700 1 -- 172.21.15.3:0/28021546 learned_addr learned my addr 172.21.15.3:0/28021546 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:28.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.050+0000 7f6a9e7fc700 1 -- 172.21.15.3:0/28021546 >> v2:172.21.15.3:3300/0 conn(0x7f6aa012cf60 msgr2=0x7f6aa006f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:28.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.050+0000 7f6a9e7fc700 1 --2- 172.21.15.3:0/28021546 >> v2:172.21.15.3:3300/0 conn(0x7f6aa012cf60 0x7f6aa006f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.050+0000 7f6a9e7fc700 1 -- 172.21.15.3:0/28021546 >> v1:172.21.15.3:6789/0 conn(0x7f6aa0130260 legacy=0x7f6aa0136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:28.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.050+0000 7f6a9e7fc700 1 -- 172.21.15.3:0/28021546 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa0071780 con 0x7f6aa012f8b0 2023-10-03T10:18:28.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.050+0000 7f6a9e7fc700 1 --2- 172.21.15.3:0/28021546 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6aa012f8b0 0x7f6aa012fd20 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f6a940098f0 tx=0x7f6a94009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=542ce8e6d02ae4b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:28.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.050+0000 7f6a9dffb700 1 -- 172.21.15.3:0/28021546 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a94011830 con 0x7f6aa012f8b0 2023-10-03T10:18:28.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.051+0000 7f6a9dffb700 1 -- 172.21.15.3:0/28021546 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a94011990 con 0x7f6aa012f8b0 2023-10-03T10:18:28.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.051+0000 7f6a9dffb700 1 -- 172.21.15.3:0/28021546 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a94011d20 con 0x7f6aa012f8b0 2023-10-03T10:18:28.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.051+0000 7f6aa5998700 1 -- 172.21.15.3:0/28021546 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6aa012f8b0 msgr2=0x7f6aa012fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:28.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.051+0000 7f6aa5998700 1 --2- 172.21.15.3:0/28021546 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6aa012f8b0 0x7f6aa012fd20 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f6a940098f0 tx=0x7f6a94009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:28.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.051+0000 7f6aa5998700 1 -- 172.21.15.3:0/28021546 shutdown_connections 2023-10-03T10:18:28.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.051+0000 7f6aa5998700 1 --2- 172.21.15.3:0/28021546 >> v2:172.21.15.3:3300/0 conn(0x7f6aa012cf60 0x7f6aa006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.051+0000 7f6aa5998700 1 --2- 172.21.15.3:0/28021546 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6aa012f8b0 0x7f6aa012fd20 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.051+0000 7f6aa5998700 1 -- 172.21.15.3:0/28021546 >> 172.21.15.3:0/28021546 conn(0x7f6aa00a2aa0 msgr2=0x7f6aa00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:28.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.051+0000 7f6aa5998700 1 -- 172.21.15.3:0/28021546 shutdown_connections 2023-10-03T10:18:28.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.051+0000 7f6aa5998700 1 -- 172.21.15.3:0/28021546 wait complete. 2023-10-03T10:18:28.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.052+0000 7f6aa5998700 1 Processor -- start 2023-10-03T10:18:28.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.053+0000 7f6aa5998700 1 -- start start 2023-10-03T10:18:28.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.053+0000 7f6aa5998700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6aa012cf60 0x7f6aa0099dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:28.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.053+0000 7f6aa5998700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6aa012f8b0 0x7f6aa009a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:28.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.053+0000 7f6aa5998700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6aa0130260 0x7f6aa009a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:28.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.053+0000 7f6aa5998700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa0120110 con 0x7f6aa012cf60 2023-10-03T10:18:28.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.053+0000 7f6aa5998700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa012f560 con 0x7f6aa0130260 2023-10-03T10:18:28.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.053+0000 7f6aa5998700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa0129fa0 con 0x7f6aa012f8b0 2023-10-03T10:18:28.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.053+0000 7f6a9e7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6aa012f8b0 0x7f6aa009a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:28.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.053+0000 7f6a9e7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6aa012f8b0 0x7f6aa009a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52274/0 (socket says 172.21.15.3:52274) 2023-10-03T10:18:28.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.054+0000 7f6a9e7fc700 1 -- 172.21.15.3:0/3021209731 learned_addr learned my addr 172.21.15.3:0/3021209731 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:28.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.053+0000 7f6a9effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6aa012cf60 0x7f6aa0099dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:28.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.054+0000 7f6a9f7fe700 1 --2- 172.21.15.3:0/3021209731 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6aa0130260 0x7f6aa009a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:28.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.054+0000 7f6a9effd700 1 -- 172.21.15.3:0/3021209731 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6aa012f8b0 msgr2=0x7f6aa009a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:28.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.054+0000 7f6a9effd700 1 --2- 172.21.15.3:0/3021209731 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6aa012f8b0 0x7f6aa009a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.054+0000 7f6a9effd700 1 -- 172.21.15.3:0/3021209731 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6aa0130260 msgr2=0x7f6aa009a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:28.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.054+0000 7f6a9effd700 1 --2- 172.21.15.3:0/3021209731 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6aa0130260 0x7f6aa009a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.054+0000 7f6a9effd700 1 -- 172.21.15.3:0/3021209731 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a94009550 con 0x7f6aa012cf60 2023-10-03T10:18:28.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.054+0000 7f6a9e7fc700 1 --2- 172.21.15.3:0/3021209731 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6aa012f8b0 0x7f6aa009a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:28.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.054+0000 7f6a9f7fe700 1 --2- 172.21.15.3:0/3021209731 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6aa0130260 0x7f6aa009a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:28.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.054+0000 7f6a9effd700 1 --2- 172.21.15.3:0/3021209731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6aa012cf60 0x7f6aa0099dd0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000e910 tx=0x7f6a9000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.054+0000 7f6a87fff700 1 -- 172.21.15.3:0/3021209731 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a90010820 con 0x7f6aa012cf60 2023-10-03T10:18:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.054+0000 7f6a87fff700 1 -- 172.21.15.3:0/3021209731 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a90010980 con 0x7f6aa012cf60 2023-10-03T10:18:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.055+0000 7f6aa5998700 1 -- 172.21.15.3:0/3021209731 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa011f7c0 con 0x7f6aa012cf60 2023-10-03T10:18:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.055+0000 7f6a87fff700 1 -- 172.21.15.3:0/3021209731 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a90010d10 con 0x7f6aa012cf60 2023-10-03T10:18:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.055+0000 7f6aa5998700 1 -- 172.21.15.3:0/3021209731 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa0120b80 con 0x7f6aa012cf60 2023-10-03T10:18:28.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.056+0000 7f6aa5998700 1 -- 172.21.15.3:0/3021209731 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa011fd30 con 0x7f6aa012cf60 2023-10-03T10:18:28.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.056+0000 7f6a87fff700 1 -- 172.21.15.3:0/3021209731 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6a900113f0 con 0x7f6aa012cf60 2023-10-03T10:18:28.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.056+0000 7f6a87fff700 1 -- 172.21.15.3:0/3021209731 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6a9001ae60 con 0x7f6aa012cf60 2023-10-03T10:18:28.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.062+0000 7f6a87fff700 1 -- 172.21.15.3:0/3021209731 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6a90044060 con 0x7f6aa012cf60 2023-10-03T10:18:28.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.089+0000 7f78ebae7700 1 Processor -- start 2023-10-03T10:18:28.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.089+0000 7f78ebae7700 1 -- start start 2023-10-03T10:18:28.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.090+0000 7f78ebae7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78e406e570 0x7f78e406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:28.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.090+0000 7f78ebae7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f78e409eca0 0x7f78e406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:28.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.090+0000 7f78ebae7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78e4070ee0 con 0x7f78e4134610 2023-10-03T10:18:28.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.090+0000 7f78ebae7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78e4071480 con 0x7f78e409eca0 2023-10-03T10:18:28.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.090+0000 7f78ebae7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e406f3c0 con 0x7f78e406e570 2023-10-03T10:18:28.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.090+0000 7f78e9883700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78e406e570 0x7f78e406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:28.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.090+0000 7f78ea084700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f78e4134610 0x7f78e4134a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47136/0 (socket says 172.21.15.3:47136) 2023-10-03T10:18:28.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.090+0000 7f78ea084700 1 -- 172.21.15.3:0/449858588 learned_addr learned my addr 172.21.15.3:0/449858588 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:28.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.090+0000 7f78e9082700 1 --2- 172.21.15.3:0/449858588 >> v2:172.21.15.3:3300/0 conn(0x7f78e409eca0 0x7f78e406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:28.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.090+0000 7f78e8881700 1 -- 172.21.15.3:0/449858588 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1509756846 0 0) 0x7f78e4070ee0 con 0x7f78e4134610 2023-10-03T10:18:28.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.090+0000 7f78e9883700 1 -- 172.21.15.3:0/449858588 >> v2:172.21.15.3:3300/0 conn(0x7f78e409eca0 msgr2=0x7f78e406ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:28.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.090+0000 7f78e9883700 1 --2- 172.21.15.3:0/449858588 >> v2:172.21.15.3:3300/0 conn(0x7f78e409eca0 0x7f78e406ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.090+0000 7f78e9883700 1 -- 172.21.15.3:0/449858588 >> v1:172.21.15.3:6789/0 conn(0x7f78e4134610 legacy=0x7f78e4134a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:28.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.090+0000 7f78e9883700 1 -- 172.21.15.3:0/449858588 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e4071b90 con 0x7f78e406e570 2023-10-03T10:18:28.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.090+0000 7f78e9082700 1 --2- 172.21.15.3:0/449858588 >> v2:172.21.15.3:3300/0 conn(0x7f78e409eca0 0x7f78e406ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:28.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.091+0000 7f78e9883700 1 --2- 172.21.15.3:0/449858588 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78e406e570 0x7f78e406e940 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f78e0009840 tx=0x7f78e0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7d75ec739c3f3e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:28.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.091+0000 7f78e8881700 1 -- 172.21.15.3:0/449858588 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e0011830 con 0x7f78e406e570 2023-10-03T10:18:28.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.091+0000 7f78e8881700 1 -- 172.21.15.3:0/449858588 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e0011990 con 0x7f78e406e570 2023-10-03T10:18:28.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.091+0000 7f78e8881700 1 -- 172.21.15.3:0/449858588 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e0011d20 con 0x7f78e406e570 2023-10-03T10:18:28.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.092+0000 7f78ebae7700 1 -- 172.21.15.3:0/449858588 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78e406e570 msgr2=0x7f78e406e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:28.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.092+0000 7f78ebae7700 1 --2- 172.21.15.3:0/449858588 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78e406e570 0x7f78e406e940 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f78e0009840 tx=0x7f78e0009b50 comp rx=0 tx=0).stop 2023-10-03T10:18:28.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.092+0000 7f78ebae7700 1 -- 172.21.15.3:0/449858588 shutdown_connections 2023-10-03T10:18:28.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.092+0000 7f78ebae7700 1 --2- 172.21.15.3:0/449858588 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78e406e570 0x7f78e406e940 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.092+0000 7f78ebae7700 1 --2- 172.21.15.3:0/449858588 >> v2:172.21.15.3:3300/0 conn(0x7f78e409eca0 0x7f78e406ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.092+0000 7f78ebae7700 1 -- 172.21.15.3:0/449858588 >> 172.21.15.3:0/449858588 conn(0x7f78e40a2aa0 msgr2=0x7f78e40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:28.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.092+0000 7f78ebae7700 1 -- 172.21.15.3:0/449858588 shutdown_connections 2023-10-03T10:18:28.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.092+0000 7f78ebae7700 1 -- 172.21.15.3:0/449858588 wait complete. 2023-10-03T10:18:28.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.093+0000 7f78ebae7700 1 Processor -- start 2023-10-03T10:18:28.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.093+0000 7f78ebae7700 1 -- start start 2023-10-03T10:18:28.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.093+0000 7f78ebae7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78e406e570 0x7f78e41214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:28.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.093+0000 7f78ebae7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f78e4134610 0x7f78e4121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:28.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.093+0000 7f78ebae7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78e4121f60 0x7f78e41d2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:28.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.093+0000 7f78ebae7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e4134380 con 0x7f78e406e570 2023-10-03T10:18:28.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.093+0000 7f78ebae7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e4123df0 con 0x7f78e4121f60 2023-10-03T10:18:28.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.094+0000 7f78ebae7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f78e4123190 con 0x7f78e4134610 2023-10-03T10:18:28.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.094+0000 7f78e9883700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78e406e570 0x7f78e41214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:28.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.094+0000 7f78e9883700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78e406e570 0x7f78e41214e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58664/0 (socket says 172.21.15.3:58664) 2023-10-03T10:18:28.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.094+0000 7f78e9883700 1 -- 172.21.15.3:0/3805103707 learned_addr learned my addr 172.21.15.3:0/3805103707 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:28.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.094+0000 7f78e9082700 1 --2- 172.21.15.3:0/3805103707 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f78e4134610 0x7f78e4121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:28.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.094+0000 7f78ea084700 1 --2- 172.21.15.3:0/3805103707 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78e4121f60 0x7f78e41d2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:28.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.094+0000 7f78e9883700 1 -- 172.21.15.3:0/3805103707 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f78e4134610 msgr2=0x7f78e4121a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:28.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.094+0000 7f78e9883700 1 --2- 172.21.15.3:0/3805103707 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f78e4134610 0x7f78e4121a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.094+0000 7f78e9883700 1 -- 172.21.15.3:0/3805103707 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78e4121f60 msgr2=0x7f78e41d2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:28.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.094+0000 7f78e9883700 1 --2- 172.21.15.3:0/3805103707 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78e4121f60 0x7f78e41d2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.094+0000 7f78e9883700 1 -- 172.21.15.3:0/3805103707 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d40094a0 con 0x7f78e406e570 2023-10-03T10:18:28.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.094+0000 7f78e9082700 1 --2- 172.21.15.3:0/3805103707 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f78e4134610 0x7f78e4121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:28.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.095+0000 7f78e9883700 1 --2- 172.21.15.3:0/3805103707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78e406e570 0x7f78e41214e0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f78e00090b0 tx=0x7f78e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:28.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.095+0000 7f78da7fc700 1 -- 172.21.15.3:0/3805103707 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e00036a0 con 0x7f78e406e570 2023-10-03T10:18:28.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.095+0000 7f78da7fc700 1 -- 172.21.15.3:0/3805103707 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e001f070 con 0x7f78e406e570 2023-10-03T10:18:28.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.095+0000 7f78ebae7700 1 -- 172.21.15.3:0/3805103707 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e00094a0 con 0x7f78e406e570 2023-10-03T10:18:28.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.095+0000 7f78da7fc700 1 -- 172.21.15.3:0/3805103707 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e0003a60 con 0x7f78e406e570 2023-10-03T10:18:28.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.095+0000 7f78ebae7700 1 -- 172.21.15.3:0/3805103707 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e4124240 con 0x7f78e406e570 2023-10-03T10:18:28.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.096+0000 7f78da7fc700 1 -- 172.21.15.3:0/3805103707 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f78e0003bc0 con 0x7f78e406e570 2023-10-03T10:18:28.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.096+0000 7f78da7fc700 1 -- 172.21.15.3:0/3805103707 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f78e0021070 con 0x7f78e406e570 2023-10-03T10:18:28.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.096+0000 7f78ebae7700 1 -- 172.21.15.3:0/3805103707 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e409b1e0 con 0x7f78e406e570 2023-10-03T10:18:28.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.101+0000 7f78da7fc700 1 -- 172.21.15.3:0/3805103707 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f78e002b020 con 0x7f78e406e570 2023-10-03T10:18:28.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.150+0000 7fa596c70700 1 Processor -- start 2023-10-03T10:18:28.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.150+0000 7fa596c70700 1 -- start start 2023-10-03T10:18:28.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.150+0000 7fa596c70700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5900a0a30 0x7fa5900a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:28.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.150+0000 7fa596c70700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa59009d630 0x7fa5900a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:28.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.150+0000 7fa596c70700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa590133930 0x7fa590133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:28.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.150+0000 7fa596c70700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5900a2a70 con 0x7fa590133930 2023-10-03T10:18:28.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.150+0000 7fa596c70700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5900a01d0 con 0x7fa5900a0a30 2023-10-03T10:18:28.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.150+0000 7fa596c70700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa59009e2b0 con 0x7fa59009d630 2023-10-03T10:18:28.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.151+0000 7fa594a0c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5900a0a30 0x7fa5900a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:28.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.151+0000 7fa594a0c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5900a0a30 0x7fa5900a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52154/0 (socket says 172.21.15.3:52154) 2023-10-03T10:18:28.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.151+0000 7fa594a0c700 1 -- 172.21.15.3:0/4213720614 learned_addr learned my addr 172.21.15.3:0/4213720614 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:28.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.151+0000 7fa58ffff700 1 --2- 172.21.15.3:0/4213720614 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa59009d630 0x7fa5900a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:28.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.151+0000 7fa59520d700 1 --2- 172.21.15.3:0/4213720614 >> v2:172.21.15.3:3300/0 conn(0x7fa590133930 0x7fa590133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:28.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.151+0000 7fa58ffff700 1 -- 172.21.15.3:0/4213720614 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5900a0a30 msgr2=0x7fa5900a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:28.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.151+0000 7fa58ffff700 1 --2- 172.21.15.3:0/4213720614 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5900a0a30 0x7fa5900a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.151+0000 7fa58ffff700 1 -- 172.21.15.3:0/4213720614 >> v2:172.21.15.3:3300/0 conn(0x7fa590133930 msgr2=0x7fa590133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:28.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.151+0000 7fa58ffff700 1 --2- 172.21.15.3:0/4213720614 >> v2:172.21.15.3:3300/0 conn(0x7fa590133930 0x7fa590133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.151+0000 7fa58ffff700 1 -- 172.21.15.3:0/4213720614 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59009dc50 con 0x7fa59009d630 2023-10-03T10:18:28.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.151+0000 7fa59520d700 1 --2- 172.21.15.3:0/4213720614 >> v2:172.21.15.3:3300/0 conn(0x7fa590133930 0x7fa590133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:18:28.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.151+0000 7fa58ffff700 1 --2- 172.21.15.3:0/4213720614 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa59009d630 0x7fa5900a1340 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa584009840 tx=0x7fa584009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51bc7a719324d3bf server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:28.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.151+0000 7fa58f7fe700 1 -- 172.21.15.3:0/4213720614 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa584011830 con 0x7fa59009d630 2023-10-03T10:18:28.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.152+0000 7fa58f7fe700 1 -- 172.21.15.3:0/4213720614 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa584011990 con 0x7fa59009d630 2023-10-03T10:18:28.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.152+0000 7fa58f7fe700 1 -- 172.21.15.3:0/4213720614 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa584011d20 con 0x7fa59009d630 2023-10-03T10:18:28.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.152+0000 7fa596c70700 1 -- 172.21.15.3:0/4213720614 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa59009d630 msgr2=0x7fa5900a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:28.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.152+0000 7fa596c70700 1 --2- 172.21.15.3:0/4213720614 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa59009d630 0x7fa5900a1340 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa584009840 tx=0x7fa584009b50 comp rx=0 tx=0).stop 2023-10-03T10:18:28.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.152+0000 7fa596c70700 1 -- 172.21.15.3:0/4213720614 shutdown_connections 2023-10-03T10:18:28.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.152+0000 7fa596c70700 1 --2- 172.21.15.3:0/4213720614 >> v2:172.21.15.3:3300/0 conn(0x7fa590133930 0x7fa590133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.152+0000 7fa596c70700 1 --2- 172.21.15.3:0/4213720614 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa5900a0a30 0x7fa5900a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.152+0000 7fa596c70700 1 --2- 172.21.15.3:0/4213720614 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa59009d630 0x7fa5900a1340 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.152+0000 7fa596c70700 1 -- 172.21.15.3:0/4213720614 >> 172.21.15.3:0/4213720614 conn(0x7fa5900a3050 msgr2=0x7fa5900a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:28.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.152+0000 7fa596c70700 1 -- 172.21.15.3:0/4213720614 shutdown_connections 2023-10-03T10:18:28.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.153+0000 7fa596c70700 1 -- 172.21.15.3:0/4213720614 wait complete. 2023-10-03T10:18:28.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.154+0000 7fa596c70700 1 Processor -- start 2023-10-03T10:18:28.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.154+0000 7fa596c70700 1 -- start start 2023-10-03T10:18:28.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.155+0000 7fa596c70700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa59009d630 0x7fa5901c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:28.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.155+0000 7fa596c70700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5900a0a30 0x7fa5901c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:28.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.155+0000 7fa596c70700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa590133930 0x7fa5901c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:28.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.155+0000 7fa596c70700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5901207a0 con 0x7fa5900a0a30 2023-10-03T10:18:28.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.155+0000 7fa596c70700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa590122110 con 0x7fa59009d630 2023-10-03T10:18:28.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.155+0000 7fa596c70700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5900a0410 con 0x7fa590133930 2023-10-03T10:18:28.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.155+0000 7fa594a0c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa59009d630 0x7fa5901c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:28.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.155+0000 7fa594a0c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa59009d630 0x7fa5901c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52162/0 (socket says 172.21.15.3:52162) 2023-10-03T10:18:28.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.155+0000 7fa594a0c700 1 -- 172.21.15.3:0/2392424284 learned_addr learned my addr 172.21.15.3:0/2392424284 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:28.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.155+0000 7fa58ffff700 1 --2- 172.21.15.3:0/2392424284 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5900a0a30 0x7fa5901c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:28.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.156+0000 7fa594a0c700 1 -- 172.21.15.3:0/2392424284 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa590133930 msgr2=0x7fa5901c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:28.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.156+0000 7fa594a0c700 1 --2- 172.21.15.3:0/2392424284 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa590133930 0x7fa5901c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.156+0000 7fa594a0c700 1 -- 172.21.15.3:0/2392424284 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5900a0a30 msgr2=0x7fa5901c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:28.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.156+0000 7fa594a0c700 1 --2- 172.21.15.3:0/2392424284 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5900a0a30 0x7fa5901c86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.156+0000 7fa594a0c700 1 -- 172.21.15.3:0/2392424284 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5840094a0 con 0x7fa59009d630 2023-10-03T10:18:28.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.156+0000 7fa594a0c700 1 --2- 172.21.15.3:0/2392424284 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa59009d630 0x7fa5901c8190 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa58000e940 tx=0x7fa58000ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:28.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.156+0000 7fa58d7fa700 1 -- 172.21.15.3:0/2392424284 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5800096a0 con 0x7fa59009d630 2023-10-03T10:18:28.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.156+0000 7fa58d7fa700 1 -- 172.21.15.3:0/2392424284 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58000ee90 con 0x7fa59009d630 2023-10-03T10:18:28.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.156+0000 7fa58d7fa700 1 -- 172.21.15.3:0/2392424284 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58000ca90 con 0x7fa59009d630 2023-10-03T10:18:28.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.156+0000 7fa596c70700 1 -- 172.21.15.3:0/2392424284 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa590121720 con 0x7fa59009d630 2023-10-03T10:18:28.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.156+0000 7fa596c70700 1 -- 172.21.15.3:0/2392424284 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa590133640 con 0x7fa59009d630 2023-10-03T10:18:28.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.157+0000 7fa58d7fa700 1 -- 172.21.15.3:0/2392424284 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa58000cbf0 con 0x7fa59009d630 2023-10-03T10:18:28.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.158+0000 7fa596c70700 1 -- 172.21.15.3:0/2392424284 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5900747f0 con 0x7fa59009d630 2023-10-03T10:18:28.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.158+0000 7fa58d7fa700 1 -- 172.21.15.3:0/2392424284 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa58003f790 con 0x7fa59009d630 2023-10-03T10:18:28.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.163+0000 7fa58d7fa700 1 -- 172.21.15.3:0/2392424284 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa580017070 con 0x7fa59009d630 2023-10-03T10:18:28.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.206+0000 7f949697d700 1 Processor -- start 2023-10-03T10:18:28.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.206+0000 7f949697d700 1 -- start start 2023-10-03T10:18:28.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.207+0000 7f949697d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f949009d6f0 0x7f949009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:28.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.207+0000 7f949697d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f949009e410 0x7f9490130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:28.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.207+0000 7f949697d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f949009f130 con 0x7f949009e000 2023-10-03T10:18:28.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.207+0000 7f949697d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949009f520 con 0x7f949009e410 2023-10-03T10:18:28.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.207+0000 7f949697d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f949009e930 con 0x7f949009d6f0 2023-10-03T10:18:28.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.207+0000 7f9494f1a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f949009e410 0x7f9490130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:28.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.207+0000 7f948ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f949009d6f0 0x7f949009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:28.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.207+0000 7f948f7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f949009e000 0x7f9490130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47138/0 (socket says 172.21.15.3:47138) 2023-10-03T10:18:28.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.207+0000 7f948f7fe700 1 -- 172.21.15.3:0/3226858577 learned_addr learned my addr 172.21.15.3:0/3226858577 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:28.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.208+0000 7f948ffff700 1 -- 172.21.15.3:0/3226858577 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f949009e410 msgr2=0x7f9490130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:28.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.208+0000 7f948ffff700 1 --2- 172.21.15.3:0/3226858577 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f949009e410 0x7f9490130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.208+0000 7f948ffff700 1 -- 172.21.15.3:0/3226858577 >> v1:172.21.15.3:6789/0 conn(0x7f949009e000 legacy=0x7f9490130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:28.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.208+0000 7f948ffff700 1 -- 172.21.15.3:0/3226858577 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949009b2d0 con 0x7f949009d6f0 2023-10-03T10:18:28.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.208+0000 7f9494f1a700 1 --2- 172.21.15.3:0/3226858577 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f949009e410 0x7f9490130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:28.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.208+0000 7f948ffff700 1 --2- 172.21.15.3:0/3226858577 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f949009d6f0 0x7f949009dac0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9480009870 tx=0x7f9480009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cf66756161c98df server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:28.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.208+0000 7f948effd700 1 -- 172.21.15.3:0/3226858577 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9480004230 con 0x7f949009d6f0 2023-10-03T10:18:28.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.208+0000 7f948effd700 1 -- 172.21.15.3:0/3226858577 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9480004390 con 0x7f949009d6f0 2023-10-03T10:18:28.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.208+0000 7f948effd700 1 -- 172.21.15.3:0/3226858577 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94800036a0 con 0x7f949009d6f0 2023-10-03T10:18:28.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.209+0000 7f949697d700 1 -- 172.21.15.3:0/3226858577 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f949009d6f0 msgr2=0x7f949009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:28.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.209+0000 7f949697d700 1 --2- 172.21.15.3:0/3226858577 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f949009d6f0 0x7f949009dac0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9480009870 tx=0x7f9480009b80 comp rx=0 tx=0).stop 2023-10-03T10:18:28.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.209+0000 7f949697d700 1 -- 172.21.15.3:0/3226858577 shutdown_connections 2023-10-03T10:18:28.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.209+0000 7f949697d700 1 --2- 172.21.15.3:0/3226858577 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f949009e410 0x7f9490130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.209+0000 7f949697d700 1 --2- 172.21.15.3:0/3226858577 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f949009d6f0 0x7f949009dac0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.209+0000 7f949697d700 1 -- 172.21.15.3:0/3226858577 >> 172.21.15.3:0/3226858577 conn(0x7f9490066f50 msgr2=0x7f949009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:28.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.209+0000 7f949697d700 1 -- 172.21.15.3:0/3226858577 shutdown_connections 2023-10-03T10:18:28.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.209+0000 7f949697d700 1 -- 172.21.15.3:0/3226858577 wait complete. 2023-10-03T10:18:28.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.210+0000 7f949697d700 1 Processor -- start 2023-10-03T10:18:28.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.211+0000 7f949697d700 1 -- start start 2023-10-03T10:18:28.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.211+0000 7f949697d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f949009d6f0 0x7f949012dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:28.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.211+0000 7f949697d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f949009e000 0x7f949012e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:28.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.211+0000 7f949697d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f949009e410 0x7f949012e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:28.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.211+0000 7f949697d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9490131040 con 0x7f949009e000 2023-10-03T10:18:28.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.211+0000 7f949697d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949012ef70 con 0x7f949009e410 2023-10-03T10:18:28.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.211+0000 7f949697d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f949012f6d0 con 0x7f949009d6f0 2023-10-03T10:18:28.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.211+0000 7f9494f1a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f949009e410 0x7f949012e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:28.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.211+0000 7f9494f1a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f949009e410 0x7f949012e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52182/0 (socket says 172.21.15.3:52182) 2023-10-03T10:18:28.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.211+0000 7f9494f1a700 1 -- 172.21.15.3:0/3189175301 learned_addr learned my addr 172.21.15.3:0/3189175301 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:28.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.212+0000 7f948ffff700 1 --2- 172.21.15.3:0/3189175301 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f949009d6f0 0x7f949012dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:28.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.212+0000 7f948f7fe700 1 --2- 172.21.15.3:0/3189175301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f949009e000 0x7f949012e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:28.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.212+0000 7f9494f1a700 1 -- 172.21.15.3:0/3189175301 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f949009d6f0 msgr2=0x7f949012dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:28.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.212+0000 7f9494f1a700 1 --2- 172.21.15.3:0/3189175301 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f949009d6f0 0x7f949012dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.212+0000 7f9494f1a700 1 -- 172.21.15.3:0/3189175301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f949009e000 msgr2=0x7f949012e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:28.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.212+0000 7f9494f1a700 1 --2- 172.21.15.3:0/3189175301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f949009e000 0x7f949012e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.212+0000 7f9494f1a700 1 -- 172.21.15.3:0/3189175301 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9488009a70 con 0x7f949009e410 2023-10-03T10:18:28.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.212+0000 7f948ffff700 1 --2- 172.21.15.3:0/3189175301 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f949009d6f0 0x7f949012dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:18:28.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.212+0000 7f948f7fe700 1 --2- 172.21.15.3:0/3189175301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f949009e000 0x7f949012e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:18:28.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.212+0000 7f9494f1a700 1 --2- 172.21.15.3:0/3189175301 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f949009e410 0x7f949012e540 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f948800efc0 tx=0x7f948800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:28.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.212+0000 7f948cff9700 1 -- 172.21.15.3:0/3189175301 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9488009d70 con 0x7f949009e410 2023-10-03T10:18:28.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.212+0000 7f948cff9700 1 -- 172.21.15.3:0/3189175301 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948800f040 con 0x7f949009e410 2023-10-03T10:18:28.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.212+0000 7f949697d700 1 -- 172.21.15.3:0/3189175301 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94800094a0 con 0x7f949009e410 2023-10-03T10:18:28.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.212+0000 7f949697d700 1 -- 172.21.15.3:0/3189175301 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949009c700 con 0x7f949009e410 2023-10-03T10:18:28.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.212+0000 7f948cff9700 1 -- 172.21.15.3:0/3189175301 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948800ce00 con 0x7f949009e410 2023-10-03T10:18:28.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.213+0000 7f948cff9700 1 -- 172.21.15.3:0/3189175301 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f94880144a0 con 0x7f949009e410 2023-10-03T10:18:28.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.214+0000 7f949697d700 1 -- 172.21.15.3:0/3189175301 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94700052e0 con 0x7f949009e410 2023-10-03T10:18:28.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.214+0000 7f948cff9700 1 -- 172.21.15.3:0/3189175301 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f94880400d0 con 0x7f949009e410 2023-10-03T10:18:28.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.219+0000 7f948cff9700 1 -- 172.21.15.3:0/3189175301 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9488012700 con 0x7f949009e410 2023-10-03T10:18:28.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.314+0000 7f6aa5998700 1 -- 172.21.15.3:0/3021209731 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6aa0074ea0 con 0x7f6aa012cf60 2023-10-03T10:18:28.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.314+0000 7f6a87fff700 1 -- 172.21.15.3:0/3021209731 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6a90044090 con 0x7f6aa012cf60 2023-10-03T10:18:28.315 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.317+0000 7f6aa5998700 1 -- 172.21.15.3:0/3021209731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6aa012cf60 msgr2=0x7f6aa0099dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.317+0000 7f6aa5998700 1 --2- 172.21.15.3:0/3021209731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6aa012cf60 0x7f6aa0099dd0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000e910 tx=0x7f6a9000ec20 comp rx=0 tx=0).stop 2023-10-03T10:18:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.317+0000 7f6aa5998700 1 -- 172.21.15.3:0/3021209731 shutdown_connections 2023-10-03T10:18:28.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.317+0000 7f6aa5998700 1 --2- 172.21.15.3:0/3021209731 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6aa0130260 0x7f6aa009a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.317+0000 7f6aa5998700 1 --2- 172.21.15.3:0/3021209731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6aa012cf60 0x7f6aa0099dd0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.317+0000 7f6aa5998700 1 --2- 172.21.15.3:0/3021209731 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6aa012f8b0 0x7f6aa009a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.317+0000 7f6aa5998700 1 -- 172.21.15.3:0/3021209731 >> 172.21.15.3:0/3021209731 conn(0x7f6aa00a2aa0 msgr2=0x7f6aa009e320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:28.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.317+0000 7f6aa5998700 1 -- 172.21.15.3:0/3021209731 shutdown_connections 2023-10-03T10:18:28.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.317+0000 7f6aa5998700 1 -- 172.21.15.3:0/3021209731 wait complete. 2023-10-03T10:18:28.332 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:28.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.357+0000 7f78ebae7700 1 -- 172.21.15.3:0/3805103707 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f78e4122330 con 0x7f78e406e570 2023-10-03T10:18:28.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.357+0000 7f78da7fc700 1 -- 172.21.15.3:0/3805103707 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f78e001a950 con 0x7f78e406e570 2023-10-03T10:18:28.358 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:28.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.361+0000 7f78ebae7700 1 -- 172.21.15.3:0/3805103707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78e406e570 msgr2=0x7f78e41214e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:28.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.361+0000 7f78ebae7700 1 --2- 172.21.15.3:0/3805103707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78e406e570 0x7f78e41214e0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f78e00090b0 tx=0x7f78e0004750 comp rx=0 tx=0).stop 2023-10-03T10:18:28.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.361+0000 7f78ebae7700 1 -- 172.21.15.3:0/3805103707 shutdown_connections 2023-10-03T10:18:28.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.361+0000 7f78ebae7700 1 --2- 172.21.15.3:0/3805103707 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78e4121f60 0x7f78e41d2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.361+0000 7f78ebae7700 1 --2- 172.21.15.3:0/3805103707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78e406e570 0x7f78e41214e0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.361+0000 7f78ebae7700 1 --2- 172.21.15.3:0/3805103707 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f78e4134610 0x7f78e4121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.361+0000 7f78ebae7700 1 -- 172.21.15.3:0/3805103707 >> 172.21.15.3:0/3805103707 conn(0x7f78e40a2aa0 msgr2=0x7f78e41bf140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:28.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.361+0000 7f78ebae7700 1 -- 172.21.15.3:0/3805103707 shutdown_connections 2023-10-03T10:18:28.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.361+0000 7f78ebae7700 1 -- 172.21.15.3:0/3805103707 wait complete. 2023-10-03T10:18:28.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.373+0000 7fa596c70700 1 -- 172.21.15.3:0/2392424284 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa590074390 con 0x7fa59009d630 2023-10-03T10:18:28.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.373+0000 7fa58d7fa700 1 -- 172.21.15.3:0/2392424284 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa5800232f0 con 0x7fa59009d630 2023-10-03T10:18:28.373 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.375+0000 7fa596c70700 1 -- 172.21.15.3:0/2392424284 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa59009d630 msgr2=0x7fa5901c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.375+0000 7fa596c70700 1 --2- 172.21.15.3:0/2392424284 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa59009d630 0x7fa5901c8190 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa58000e940 tx=0x7fa58000ec50 comp rx=0 tx=0).stop 2023-10-03T10:18:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.375+0000 7fa596c70700 1 -- 172.21.15.3:0/2392424284 shutdown_connections 2023-10-03T10:18:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.375+0000 7fa596c70700 1 --2- 172.21.15.3:0/2392424284 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa590133930 0x7fa5901c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.375+0000 7fa596c70700 1 --2- 172.21.15.3:0/2392424284 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa59009d630 0x7fa5901c8190 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.375+0000 7fa596c70700 1 --2- 172.21.15.3:0/2392424284 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5900a0a30 0x7fa5901c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.375+0000 7fa596c70700 1 -- 172.21.15.3:0/2392424284 >> 172.21.15.3:0/2392424284 conn(0x7fa5900a3050 msgr2=0x7fa59009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.375+0000 7fa596c70700 1 -- 172.21.15.3:0/2392424284 shutdown_connections 2023-10-03T10:18:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.375+0000 7fa596c70700 1 -- 172.21.15.3:0/2392424284 wait complete. 2023-10-03T10:18:28.379 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:28.389 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:28.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.445+0000 7f949697d700 1 -- 172.21.15.3:0/3189175301 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9470005590 con 0x7f949009e410 2023-10-03T10:18:28.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.445+0000 7f948cff9700 1 -- 172.21.15.3:0/3189175301 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9488012340 con 0x7f949009e410 2023-10-03T10:18:28.446 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:28.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.447+0000 7f949697d700 1 -- 172.21.15.3:0/3189175301 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f949009e410 msgr2=0x7f949012e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:28.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.447+0000 7f949697d700 1 --2- 172.21.15.3:0/3189175301 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f949009e410 0x7f949012e540 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f948800efc0 tx=0x7f948800c3a0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.447+0000 7f949697d700 1 -- 172.21.15.3:0/3189175301 shutdown_connections 2023-10-03T10:18:28.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.447+0000 7f949697d700 1 --2- 172.21.15.3:0/3189175301 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f949009e410 0x7f949012e540 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.447+0000 7f949697d700 1 --2- 172.21.15.3:0/3189175301 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f949009d6f0 0x7f949012dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.447+0000 7f949697d700 1 --2- 172.21.15.3:0/3189175301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f949009e000 0x7f949012e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:28.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.447+0000 7f949697d700 1 -- 172.21.15.3:0/3189175301 >> 172.21.15.3:0/3189175301 conn(0x7f9490066f50 msgr2=0x7f94900a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:28.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.447+0000 7f949697d700 1 -- 172.21.15.3:0/3189175301 shutdown_connections 2023-10-03T10:18:28.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:28.447+0000 7f949697d700 1 -- 172.21.15.3:0/3189175301 wait complete. 2023-10-03T10:18:28.461 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:29.333 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:29.380 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:29.390 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:29.462 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:29.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.488+0000 7f7b17fe3700 1 Processor -- start 2023-10-03T10:18:29.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.488+0000 7f7b17fe3700 1 -- start start 2023-10-03T10:18:29.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.489+0000 7f7b17fe3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b1009d6f0 0x7f7b1009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:29.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.489+0000 7f7b17fe3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b1009e410 0x7f7b10130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:29.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.489+0000 7f7b17fe3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b1009f130 con 0x7f7b1009e000 2023-10-03T10:18:29.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.489+0000 7f7b17fe3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1009f520 con 0x7f7b1009e410 2023-10-03T10:18:29.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.489+0000 7f7b17fe3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1009e930 con 0x7f7b1009d6f0 2023-10-03T10:18:29.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.489+0000 7f7b1557e700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7b1009e000 0x7f7b10130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47140/0 (socket says 172.21.15.3:47140) 2023-10-03T10:18:29.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.489+0000 7f7b1557e700 1 -- 172.21.15.3:0/876119357 learned_addr learned my addr 172.21.15.3:0/876119357 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:29.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.489+0000 7f7b16580700 1 --2- 172.21.15.3:0/876119357 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b1009e410 0x7f7b10130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:29.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.489+0000 7f7b14d7d700 1 -- 172.21.15.3:0/876119357 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 177625969 0 0) 0x7f7b1009f130 con 0x7f7b1009e000 2023-10-03T10:18:29.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.489+0000 7f7b16580700 1 -- 172.21.15.3:0/876119357 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b1009d6f0 msgr2=0x7f7b1009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:29.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.489+0000 7f7b15d7f700 1 --2- 172.21.15.3:0/876119357 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b1009d6f0 0x7f7b1009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:29.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.490+0000 7f7b16580700 1 --2- 172.21.15.3:0/876119357 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b1009d6f0 0x7f7b1009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.490+0000 7f7b16580700 1 -- 172.21.15.3:0/876119357 >> v1:172.21.15.3:6789/0 conn(0x7f7b1009e000 legacy=0x7f7b10130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:29.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.490+0000 7f7b16580700 1 -- 172.21.15.3:0/876119357 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1009b2d0 con 0x7f7b1009e410 2023-10-03T10:18:29.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.490+0000 7f7b15d7f700 1 --2- 172.21.15.3:0/876119357 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b1009d6f0 0x7f7b1009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:18:29.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.490+0000 7f7b16580700 1 --2- 172.21.15.3:0/876119357 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b1009e410 0x7f7b10130d70 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009dc0 tx=0x7f7b0800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b38a366f9fea700 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:29.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.490+0000 7f7b14d7d700 1 -- 172.21.15.3:0/876119357 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0800bdd0 con 0x7f7b1009e410 2023-10-03T10:18:29.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.490+0000 7f7b14d7d700 1 -- 172.21.15.3:0/876119357 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b08011460 con 0x7f7b1009e410 2023-10-03T10:18:29.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.490+0000 7f7b14d7d700 1 -- 172.21.15.3:0/876119357 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b08011810 con 0x7f7b1009e410 2023-10-03T10:18:29.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.491+0000 7f7b17fe3700 1 -- 172.21.15.3:0/876119357 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b1009e410 msgr2=0x7f7b10130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:29.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.491+0000 7f7b17fe3700 1 --2- 172.21.15.3:0/876119357 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b1009e410 0x7f7b10130d70 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009dc0 tx=0x7f7b0800b330 comp rx=0 tx=0).stop 2023-10-03T10:18:29.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.491+0000 7f7b17fe3700 1 -- 172.21.15.3:0/876119357 shutdown_connections 2023-10-03T10:18:29.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.491+0000 7f7b17fe3700 1 --2- 172.21.15.3:0/876119357 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b1009e410 0x7f7b10130d70 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.491+0000 7f7b17fe3700 1 --2- 172.21.15.3:0/876119357 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b1009d6f0 0x7f7b1009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.491+0000 7f7b17fe3700 1 -- 172.21.15.3:0/876119357 >> 172.21.15.3:0/876119357 conn(0x7f7b10066f50 msgr2=0x7f7b1009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:29.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.491+0000 7f7b17fe3700 1 -- 172.21.15.3:0/876119357 shutdown_connections 2023-10-03T10:18:29.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.491+0000 7f7b17fe3700 1 -- 172.21.15.3:0/876119357 wait complete. 2023-10-03T10:18:29.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.493+0000 7f7b17fe3700 1 Processor -- start 2023-10-03T10:18:29.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.493+0000 7f7b17fe3700 1 -- start start 2023-10-03T10:18:29.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.493+0000 7f7b17fe3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b1009d6f0 0x7f7b1011def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:29.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.493+0000 7f7b17fe3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b1009e000 0x7f7b1011e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:29.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.493+0000 7f7b17fe3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b1009e410 0x7f7b1011e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:29.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.493+0000 7f7b17fe3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b101217d0 con 0x7f7b1009e410 2023-10-03T10:18:29.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.493+0000 7f7b17fe3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b101302d0 con 0x7f7b1009d6f0 2023-10-03T10:18:29.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.493+0000 7f7b17fe3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7b10120d50 con 0x7f7b1009e000 2023-10-03T10:18:29.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.494+0000 7f7b15d7f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b1009d6f0 0x7f7b1011def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:29.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.494+0000 7f7b16580700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b1009e410 0x7f7b1011e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:29.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.494+0000 7f7b16580700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b1009e410 0x7f7b1011e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58690/0 (socket says 172.21.15.3:58690) 2023-10-03T10:18:29.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.494+0000 7f7b15d7f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b1009d6f0 0x7f7b1011def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52208/0 (socket says 172.21.15.3:52208) 2023-10-03T10:18:29.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.494+0000 7f7b16580700 1 -- 172.21.15.3:0/591873594 learned_addr learned my addr 172.21.15.3:0/591873594 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:29.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.494+0000 7f7b1557e700 1 --2- 172.21.15.3:0/591873594 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b1009e000 0x7f7b1011e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:29.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.494+0000 7f7b16580700 1 -- 172.21.15.3:0/591873594 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b1009e000 msgr2=0x7f7b1011e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:29.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.494+0000 7f7b16580700 1 --2- 172.21.15.3:0/591873594 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b1009e000 0x7f7b1011e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.494+0000 7f7b16580700 1 -- 172.21.15.3:0/591873594 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b1009d6f0 msgr2=0x7f7b1011def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:29.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.494+0000 7f7b16580700 1 --2- 172.21.15.3:0/591873594 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b1009d6f0 0x7f7b1011def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.494+0000 7f7b16580700 1 -- 172.21.15.3:0/591873594 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b08009a70 con 0x7f7b1009e410 2023-10-03T10:18:29.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.494+0000 7f7b16580700 1 --2- 172.21.15.3:0/591873594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b1009e410 0x7f7b1011e970 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f7b08003f50 tx=0x7f7b08004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:29.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.495+0000 7f7b067fc700 1 -- 172.21.15.3:0/591873594 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b08011c00 con 0x7f7b1009e410 2023-10-03T10:18:29.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.495+0000 7f7b067fc700 1 -- 172.21.15.3:0/591873594 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b08011d60 con 0x7f7b1009e410 2023-10-03T10:18:29.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.495+0000 7f7b17fe3700 1 -- 172.21.15.3:0/591873594 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b101200e0 con 0x7f7b1009e410 2023-10-03T10:18:29.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.495+0000 7f7b067fc700 1 -- 172.21.15.3:0/591873594 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b08004280 con 0x7f7b1009e410 2023-10-03T10:18:29.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.495+0000 7f7b17fe3700 1 -- 172.21.15.3:0/591873594 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b1009c700 con 0x7f7b1009e410 2023-10-03T10:18:29.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.497+0000 7f7b067fc700 1 -- 172.21.15.3:0/591873594 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7b080043e0 con 0x7f7b1009e410 2023-10-03T10:18:29.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.497+0000 7f7b067fc700 1 -- 172.21.15.3:0/591873594 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7b080412b0 con 0x7f7b1009e410 2023-10-03T10:18:29.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.497+0000 7f7b17fe3700 1 -- 172.21.15.3:0/591873594 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b1011ac20 con 0x7f7b1009e410 2023-10-03T10:18:29.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.502+0000 7f7b067fc700 1 -- 172.21.15.3:0/591873594 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7b08019400 con 0x7f7b1009e410 2023-10-03T10:18:29.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.534+0000 7ff1eb925700 1 Processor -- start 2023-10-03T10:18:29.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.535+0000 7ff1eb925700 1 -- start start 2023-10-03T10:18:29.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.535+0000 7ff1eb925700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1e409d6f0 0x7ff1e409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:29.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.535+0000 7ff1eb925700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1e409e000 0x7ff1e41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:29.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.535+0000 7ff1eb925700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff1e409e410 0x7ff1e4133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:29.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.535+0000 7ff1eb925700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1e406b460 con 0x7ff1e409e410 2023-10-03T10:18:29.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.535+0000 7ff1eb925700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e409b120 con 0x7ff1e409e000 2023-10-03T10:18:29.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.535+0000 7ff1eb925700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e409a7e0 con 0x7ff1e409d6f0 2023-10-03T10:18:29.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.535+0000 7ff1e9ec2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff1e409e410 0x7ff1e4133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:29.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.535+0000 7ff1e9ec2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff1e409e410 0x7ff1e4133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58698/0 (socket says 172.21.15.3:58698) 2023-10-03T10:18:29.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.535+0000 7ff1e9ec2700 1 -- 172.21.15.3:0/2041212713 learned_addr learned my addr 172.21.15.3:0/2041212713 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:29.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.535+0000 7ff1e8ec0700 1 --2- 172.21.15.3:0/2041212713 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1e409e000 0x7ff1e41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:29.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.536+0000 7ff1e96c1700 1 --2- 172.21.15.3:0/2041212713 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1e409d6f0 0x7ff1e409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:29.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.536+0000 7ff1e9ec2700 1 -- 172.21.15.3:0/2041212713 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1e409d6f0 msgr2=0x7ff1e409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:29.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.536+0000 7ff1e9ec2700 1 --2- 172.21.15.3:0/2041212713 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1e409d6f0 0x7ff1e409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.536+0000 7ff1e9ec2700 1 -- 172.21.15.3:0/2041212713 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1e409e000 msgr2=0x7ff1e41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:29.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.536+0000 7ff1e9ec2700 1 --2- 172.21.15.3:0/2041212713 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1e409e000 0x7ff1e41339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.536+0000 7ff1e9ec2700 1 -- 172.21.15.3:0/2041212713 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e409a500 con 0x7ff1e409e410 2023-10-03T10:18:29.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.536+0000 7ff1e9ec2700 1 --2- 172.21.15.3:0/2041212713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1e409e410 0x7ff1e4133f30 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc009e50 tx=0x7ff1dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc150a434258a231 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:29.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.536+0000 7ff1dbfff700 1 -- 172.21.15.3:0/2041212713 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc00e070 con 0x7ff1e409e410 2023-10-03T10:18:29.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.536+0000 7ff1dbfff700 1 -- 172.21.15.3:0/2041212713 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1dc00b910 con 0x7ff1e409e410 2023-10-03T10:18:29.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.536+0000 7ff1dbfff700 1 -- 172.21.15.3:0/2041212713 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc00bcc0 con 0x7ff1e409e410 2023-10-03T10:18:29.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.537+0000 7ff1eb925700 1 -- 172.21.15.3:0/2041212713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1e409e410 msgr2=0x7ff1e4133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:29.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.537+0000 7ff1eb925700 1 --2- 172.21.15.3:0/2041212713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1e409e410 0x7ff1e4133f30 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc009e50 tx=0x7ff1dc00b330 comp rx=0 tx=0).stop 2023-10-03T10:18:29.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.537+0000 7ff1eb925700 1 -- 172.21.15.3:0/2041212713 shutdown_connections 2023-10-03T10:18:29.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.537+0000 7ff1eb925700 1 --2- 172.21.15.3:0/2041212713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1e409e410 0x7ff1e4133f30 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.537+0000 7ff1eb925700 1 --2- 172.21.15.3:0/2041212713 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1e409d6f0 0x7ff1e409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.537+0000 7ff1eb925700 1 --2- 172.21.15.3:0/2041212713 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1e409e000 0x7ff1e41339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.537+0000 7ff1eb925700 1 -- 172.21.15.3:0/2041212713 >> 172.21.15.3:0/2041212713 conn(0x7ff1e4066f50 msgr2=0x7ff1e409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:29.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.537+0000 7ff1eb925700 1 -- 172.21.15.3:0/2041212713 shutdown_connections 2023-10-03T10:18:29.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.537+0000 7ff1eb925700 1 -- 172.21.15.3:0/2041212713 wait complete. 2023-10-03T10:18:29.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.538+0000 7ff1eb925700 1 Processor -- start 2023-10-03T10:18:29.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.538+0000 7ff1eb925700 1 -- start start 2023-10-03T10:18:29.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.539+0000 7ff1eb925700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1e409d6f0 0x7ff1e41a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:29.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.539+0000 7ff1eb925700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1e409e000 0x7ff1e41a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:29.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.539+0000 7ff1eb925700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1e409e410 0x7ff1e41c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:29.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.539+0000 7ff1eb925700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e41a5c90 con 0x7ff1e409d6f0 2023-10-03T10:18:29.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.539+0000 7ff1eb925700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e41b9140 con 0x7ff1e409e000 2023-10-03T10:18:29.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.539+0000 7ff1eb925700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e41a4d20 con 0x7ff1e409e410 2023-10-03T10:18:29.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.539+0000 7ff1e8ec0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1e409e000 0x7ff1e41a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:29.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.539+0000 7ff1e8ec0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1e409e000 0x7ff1e41a5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52236/0 (socket says 172.21.15.3:52236) 2023-10-03T10:18:29.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.539+0000 7ff1e8ec0700 1 -- 172.21.15.3:0/2402840219 learned_addr learned my addr 172.21.15.3:0/2402840219 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:29.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.539+0000 7ff1e9ec2700 1 --2- 172.21.15.3:0/2402840219 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1e409e410 0x7ff1e41c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:29.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.539+0000 7ff1e96c1700 1 --2- 172.21.15.3:0/2402840219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1e409d6f0 0x7ff1e41a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:29.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.540+0000 7ff1e8ec0700 1 -- 172.21.15.3:0/2402840219 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1e409e410 msgr2=0x7ff1e41c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:29.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.540+0000 7ff1e8ec0700 1 --2- 172.21.15.3:0/2402840219 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1e409e410 0x7ff1e41c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.540+0000 7ff1e8ec0700 1 -- 172.21.15.3:0/2402840219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1e409d6f0 msgr2=0x7ff1e41a5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:29.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.540+0000 7ff1e8ec0700 1 --2- 172.21.15.3:0/2402840219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1e409d6f0 0x7ff1e41a5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.540+0000 7ff1e8ec0700 1 -- 172.21.15.3:0/2402840219 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1dc009b30 con 0x7ff1e409e000 2023-10-03T10:18:29.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.540+0000 7ff1e8ec0700 1 --2- 172.21.15.3:0/2402840219 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1e409e000 0x7ff1e41a5750 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff1d400e9b0 tx=0x7ff1d400ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:29.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.540+0000 7ff1d9ffb700 1 -- 172.21.15.3:0/2402840219 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4009710 con 0x7ff1e409e000 2023-10-03T10:18:29.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.540+0000 7ff1d9ffb700 1 -- 172.21.15.3:0/2402840219 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d400c850 con 0x7ff1e409e000 2023-10-03T10:18:29.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.540+0000 7ff1eb925700 1 -- 172.21.15.3:0/2402840219 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e41a83d0 con 0x7ff1e409e000 2023-10-03T10:18:29.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.540+0000 7ff1eb925700 1 -- 172.21.15.3:0/2402840219 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e41a7d30 con 0x7ff1e409e000 2023-10-03T10:18:29.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.540+0000 7ff1d9ffb700 1 -- 172.21.15.3:0/2402840219 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d400cbe0 con 0x7ff1e409e000 2023-10-03T10:18:29.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.541+0000 7ff1d9ffb700 1 -- 172.21.15.3:0/2402840219 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff1d400cd40 con 0x7ff1e409e000 2023-10-03T10:18:29.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.542+0000 7ff1d9ffb700 1 -- 172.21.15.3:0/2402840219 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff1d403fac0 con 0x7ff1e409e000 2023-10-03T10:18:29.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.542+0000 7ff1eb925700 1 -- 172.21.15.3:0/2402840219 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c40052e0 con 0x7ff1e409e000 2023-10-03T10:18:29.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.551+0000 7ff1d9ffb700 1 -- 172.21.15.3:0/2402840219 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff1d4021080 con 0x7ff1e409e000 2023-10-03T10:18:29.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.580+0000 7f6177fab700 1 Processor -- start 2023-10-03T10:18:29.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.580+0000 7f6177fab700 1 -- start start 2023-10-03T10:18:29.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.580+0000 7f6177fab700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6170131810 0x7f6170131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:29.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.580+0000 7f6177fab700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f617009a580 0x7f6170139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:29.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.581+0000 7f6177fab700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61701313c0 con 0x7f6170099a40 2023-10-03T10:18:29.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.581+0000 7f6177fab700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f617006e8c0 con 0x7f6170131810 2023-10-03T10:18:29.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.581+0000 7f6177fab700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6170071d20 con 0x7f617009a580 2023-10-03T10:18:29.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.581+0000 7f6175d47700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6170131810 0x7f6170131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:29.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.581+0000 7f6176548700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f617009a580 0x7f6170139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:29.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.581+0000 7f6175d47700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6170131810 0x7f6170131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52242/0 (socket says 172.21.15.3:52242) 2023-10-03T10:18:29.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.581+0000 7f6176548700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f617009a580 0x7f6170139660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52360/0 (socket says 172.21.15.3:52360) 2023-10-03T10:18:29.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.581+0000 7f6175d47700 1 -- 172.21.15.3:0/151812662 learned_addr learned my addr 172.21.15.3:0/151812662 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.581+0000 7f6176548700 1 -- 172.21.15.3:0/151812662 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6170131810 msgr2=0x7f6170131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.581+0000 7f6176548700 1 --2- 172.21.15.3:0/151812662 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6170131810 0x7f6170131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.581+0000 7f6176548700 1 -- 172.21.15.3:0/151812662 >> v1:172.21.15.3:6789/0 conn(0x7f6170099a40 legacy=0x7f6170099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.581+0000 7f6176548700 1 -- 172.21.15.3:0/151812662 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61700721a0 con 0x7f617009a580 2023-10-03T10:18:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.581+0000 7f6176548700 1 --2- 172.21.15.3:0/151812662 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f617009a580 0x7f6170139660 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6168009dc0 tx=0x7f616800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d624dfd7453766f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.582+0000 7f6174d45700 1 -- 172.21.15.3:0/151812662 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616800e070 con 0x7f617009a580 2023-10-03T10:18:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.582+0000 7f6174d45700 1 -- 172.21.15.3:0/151812662 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616800bb20 con 0x7f617009a580 2023-10-03T10:18:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.582+0000 7f6174d45700 1 -- 172.21.15.3:0/151812662 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6168012430 con 0x7f617009a580 2023-10-03T10:18:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.582+0000 7f6177fab700 1 -- 172.21.15.3:0/151812662 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f617009a580 msgr2=0x7f6170139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.582+0000 7f6177fab700 1 --2- 172.21.15.3:0/151812662 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f617009a580 0x7f6170139660 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6168009dc0 tx=0x7f616800b330 comp rx=0 tx=0).stop 2023-10-03T10:18:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.582+0000 7f6177fab700 1 -- 172.21.15.3:0/151812662 shutdown_connections 2023-10-03T10:18:29.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.582+0000 7f6177fab700 1 --2- 172.21.15.3:0/151812662 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6170131810 0x7f6170131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.582+0000 7f6177fab700 1 --2- 172.21.15.3:0/151812662 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f617009a580 0x7f6170139660 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.582+0000 7f6177fab700 1 -- 172.21.15.3:0/151812662 >> 172.21.15.3:0/151812662 conn(0x7f61700a2aa0 msgr2=0x7f61700a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:29.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.583+0000 7f6177fab700 1 -- 172.21.15.3:0/151812662 shutdown_connections 2023-10-03T10:18:29.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.583+0000 7f6177fab700 1 -- 172.21.15.3:0/151812662 wait complete. 2023-10-03T10:18:29.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.584+0000 7f6177fab700 1 Processor -- start 2023-10-03T10:18:29.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.584+0000 7f6177fab700 1 -- start start 2023-10-03T10:18:29.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.584+0000 7f6177fab700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6170099a40 0x7f617011d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:29.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.584+0000 7f6177fab700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f617009a580 0x7f617011d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:29.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.584+0000 7f6177fab700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6170131810 0x7f617011db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:29.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.584+0000 7f6177fab700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61701207b0 con 0x7f6170131810 2023-10-03T10:18:29.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.584+0000 7f6177fab700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6170120990 con 0x7f617009a580 2023-10-03T10:18:29.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.584+0000 7f6177fab700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f617011f300 con 0x7f6170099a40 2023-10-03T10:18:29.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.584+0000 7f6175546700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f617009a580 0x7f617011d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:29.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.584+0000 7f6175546700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f617009a580 0x7f617011d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52248/0 (socket says 172.21.15.3:52248) 2023-10-03T10:18:29.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.584+0000 7f6175546700 1 -- 172.21.15.3:0/4021345748 learned_addr learned my addr 172.21.15.3:0/4021345748 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:29.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.585+0000 7f6175d47700 1 --2- 172.21.15.3:0/4021345748 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6170099a40 0x7f617011d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:29.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.585+0000 7f6176548700 1 --2- 172.21.15.3:0/4021345748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6170131810 0x7f617011db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:29.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.585+0000 7f6175546700 1 -- 172.21.15.3:0/4021345748 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6170099a40 msgr2=0x7f617011d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:29.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.585+0000 7f6175546700 1 --2- 172.21.15.3:0/4021345748 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6170099a40 0x7f617011d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.585+0000 7f6175546700 1 -- 172.21.15.3:0/4021345748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6170131810 msgr2=0x7f617011db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:29.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.585+0000 7f6175546700 1 --2- 172.21.15.3:0/4021345748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6170131810 0x7f617011db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.585+0000 7f6175546700 1 -- 172.21.15.3:0/4021345748 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6168009a70 con 0x7f617009a580 2023-10-03T10:18:29.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.585+0000 7f6176548700 1 --2- 172.21.15.3:0/4021345748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6170131810 0x7f617011db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:18:29.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.585+0000 7f6175d47700 1 --2- 172.21.15.3:0/4021345748 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6170099a40 0x7f617011d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:18:29.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.585+0000 7f6175546700 1 --2- 172.21.15.3:0/4021345748 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f617009a580 0x7f617011d620 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f616000c4b0 tx=0x7f616000c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:29.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.585+0000 7f61667fc700 1 -- 172.21.15.3:0/4021345748 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61600052d0 con 0x7f617009a580 2023-10-03T10:18:29.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.585+0000 7f61667fc700 1 -- 172.21.15.3:0/4021345748 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6160005430 con 0x7f617009a580 2023-10-03T10:18:29.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.585+0000 7f6177fab700 1 -- 172.21.15.3:0/4021345748 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f617011eb20 con 0x7f617009a580 2023-10-03T10:18:29.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.585+0000 7f61667fc700 1 -- 172.21.15.3:0/4021345748 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61600038c0 con 0x7f617009a580 2023-10-03T10:18:29.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.585+0000 7f6177fab700 1 -- 172.21.15.3:0/4021345748 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61701203f0 con 0x7f617009a580 2023-10-03T10:18:29.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.586+0000 7f61667fc700 1 -- 172.21.15.3:0/4021345748 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6160003a20 con 0x7f617009a580 2023-10-03T10:18:29.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.587+0000 7f6177fab700 1 -- 172.21.15.3:0/4021345748 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61701205e0 con 0x7f617009a580 2023-10-03T10:18:29.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.587+0000 7f61667fc700 1 -- 172.21.15.3:0/4021345748 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f616003ebe0 con 0x7f617009a580 2023-10-03T10:18:29.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.596+0000 7f61667fc700 1 -- 172.21.15.3:0/4021345748 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f616000e280 con 0x7f617009a580 2023-10-03T10:18:29.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.626+0000 7f4cec2f0700 1 Processor -- start 2023-10-03T10:18:29.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.626+0000 7f4cec2f0700 1 -- start start 2023-10-03T10:18:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.627+0000 7f4cec2f0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4ce4131810 0x7f4ce4131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.627+0000 7f4cec2f0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4ce4099a40 0x7f4ce4099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.627+0000 7f4cec2f0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4ce409a3b0 0x7f4ce4138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.627+0000 7f4cec2f0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ce409b580 con 0x7f4ce409a3b0 2023-10-03T10:18:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.627+0000 7f4cec2f0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce41313c0 con 0x7f4ce4131810 2023-10-03T10:18:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.627+0000 7f4cec2f0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce406e8c0 con 0x7f4ce4099a40 2023-10-03T10:18:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.627+0000 7f4cea08c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4ce4131810 0x7f4ce4131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.628+0000 7f4cea08c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4ce4131810 0x7f4ce4131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52254/0 (socket says 172.21.15.3:52254) 2023-10-03T10:18:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.628+0000 7f4cea08c700 1 -- 172.21.15.3:0/69652492 learned_addr learned my addr 172.21.15.3:0/69652492 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.628+0000 7f4ce988b700 1 --2- 172.21.15.3:0/69652492 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4ce4099a40 0x7f4ce4099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.628+0000 7f4cea88d700 1 --2- 172.21.15.3:0/69652492 >> v2:172.21.15.3:3300/0 conn(0x7f4ce409a3b0 0x7f4ce4138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.628+0000 7f4cea08c700 1 -- 172.21.15.3:0/69652492 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4ce4099a40 msgr2=0x7f4ce4099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:29.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.628+0000 7f4cea08c700 1 --2- 172.21.15.3:0/69652492 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4ce4099a40 0x7f4ce4099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.628+0000 7f4cea08c700 1 -- 172.21.15.3:0/69652492 >> v2:172.21.15.3:3300/0 conn(0x7f4ce409a3b0 msgr2=0x7f4ce4138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:29.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.628+0000 7f4cea08c700 1 --2- 172.21.15.3:0/69652492 >> v2:172.21.15.3:3300/0 conn(0x7f4ce409a3b0 0x7f4ce4138650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.628+0000 7f4cea08c700 1 -- 172.21.15.3:0/69652492 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce409a820 con 0x7f4ce4131810 2023-10-03T10:18:29.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.628+0000 7f4cea08c700 1 --2- 172.21.15.3:0/69652492 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4ce4131810 0x7f4ce4131be0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00098f0 tx=0x7f4ce0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62bc96c18baa45f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:29.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.628+0000 7f4ce908a700 1 -- 172.21.15.3:0/69652492 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0011830 con 0x7f4ce4131810 2023-10-03T10:18:29.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.629+0000 7f4ce908a700 1 -- 172.21.15.3:0/69652492 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce0011990 con 0x7f4ce4131810 2023-10-03T10:18:29.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.629+0000 7f4ce908a700 1 -- 172.21.15.3:0/69652492 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0011d20 con 0x7f4ce4131810 2023-10-03T10:18:29.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.629+0000 7f4cec2f0700 1 -- 172.21.15.3:0/69652492 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4ce4131810 msgr2=0x7f4ce4131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:29.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.629+0000 7f4cec2f0700 1 --2- 172.21.15.3:0/69652492 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4ce4131810 0x7f4ce4131be0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00098f0 tx=0x7f4ce0009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:29.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.630+0000 7f4cec2f0700 1 -- 172.21.15.3:0/69652492 shutdown_connections 2023-10-03T10:18:29.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.630+0000 7f4cec2f0700 1 --2- 172.21.15.3:0/69652492 >> v2:172.21.15.3:3300/0 conn(0x7f4ce409a3b0 0x7f4ce4138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.630+0000 7f4cec2f0700 1 --2- 172.21.15.3:0/69652492 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4ce4131810 0x7f4ce4131be0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.630+0000 7f4cec2f0700 1 --2- 172.21.15.3:0/69652492 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4ce4099a40 0x7f4ce4099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.630+0000 7f4cec2f0700 1 -- 172.21.15.3:0/69652492 >> 172.21.15.3:0/69652492 conn(0x7f4ce40a2aa0 msgr2=0x7f4ce40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:29.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.630+0000 7f4cec2f0700 1 -- 172.21.15.3:0/69652492 shutdown_connections 2023-10-03T10:18:29.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.630+0000 7f4cec2f0700 1 -- 172.21.15.3:0/69652492 wait complete. 2023-10-03T10:18:29.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.631+0000 7f4cec2f0700 1 Processor -- start 2023-10-03T10:18:29.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.631+0000 7f4cec2f0700 1 -- start start 2023-10-03T10:18:29.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.632+0000 7f4cec2f0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4ce4099a40 0x7f4ce406f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:29.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.632+0000 7f4cec2f0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ce409a3b0 0x7f4ce406fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:29.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.632+0000 7f4cec2f0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4ce4131810 0x7f4ce4070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:29.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.632+0000 7f4cec2f0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce41309d0 con 0x7f4ce409a3b0 2023-10-03T10:18:29.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.632+0000 7f4cec2f0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce40711c0 con 0x7f4ce4099a40 2023-10-03T10:18:29.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.632+0000 7f4cec2f0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce406eb00 con 0x7f4ce4131810 2023-10-03T10:18:29.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.632+0000 7f4ce988b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ce409a3b0 0x7f4ce406fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:29.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.632+0000 7f4ce988b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ce409a3b0 0x7f4ce406fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58732/0 (socket says 172.21.15.3:58732) 2023-10-03T10:18:29.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.632+0000 7f4ce988b700 1 -- 172.21.15.3:0/3991955378 learned_addr learned my addr 172.21.15.3:0/3991955378 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:29.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.632+0000 7f4cea08c700 1 --2- 172.21.15.3:0/3991955378 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4ce4099a40 0x7f4ce406f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:29.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.632+0000 7f4cea88d700 1 --2- 172.21.15.3:0/3991955378 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4ce4131810 0x7f4ce4070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:29.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.632+0000 7f4ce988b700 1 -- 172.21.15.3:0/3991955378 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4ce4131810 msgr2=0x7f4ce4070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:29.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.632+0000 7f4ce988b700 1 --2- 172.21.15.3:0/3991955378 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4ce4131810 0x7f4ce4070090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.632+0000 7f4ce988b700 1 -- 172.21.15.3:0/3991955378 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4ce4099a40 msgr2=0x7f4ce406f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:29.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.632+0000 7f4ce988b700 1 --2- 172.21.15.3:0/3991955378 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4ce4099a40 0x7f4ce406f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.632+0000 7f4ce988b700 1 -- 172.21.15.3:0/3991955378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce0009550 con 0x7f4ce409a3b0 2023-10-03T10:18:29.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.632+0000 7f4ce988b700 1 --2- 172.21.15.3:0/3991955378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ce409a3b0 0x7f4ce406fb50 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400e980 tx=0x7f4cd400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:29.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.633+0000 7f4cdaffd700 1 -- 172.21.15.3:0/3991955378 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4009710 con 0x7f4ce409a3b0 2023-10-03T10:18:29.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.633+0000 7f4cdaffd700 1 -- 172.21.15.3:0/3991955378 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd400c690 con 0x7f4ce409a3b0 2023-10-03T10:18:29.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.633+0000 7f4cdaffd700 1 -- 172.21.15.3:0/3991955378 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd400ca20 con 0x7f4ce409a3b0 2023-10-03T10:18:29.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.633+0000 7f4cec2f0700 1 -- 172.21.15.3:0/3991955378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce4120d80 con 0x7f4ce409a3b0 2023-10-03T10:18:29.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.633+0000 7f4cec2f0700 1 -- 172.21.15.3:0/3991955378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce4121820 con 0x7f4ce409a3b0 2023-10-03T10:18:29.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.634+0000 7f4cec2f0700 1 -- 172.21.15.3:0/3991955378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce4120f10 con 0x7f4ce409a3b0 2023-10-03T10:18:29.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.634+0000 7f4cdaffd700 1 -- 172.21.15.3:0/3991955378 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4cd400cb80 con 0x7f4ce409a3b0 2023-10-03T10:18:29.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.634+0000 7f4cdaffd700 1 -- 172.21.15.3:0/3991955378 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4cd403f710 con 0x7f4ce409a3b0 2023-10-03T10:18:29.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.643+0000 7f4cdaffd700 1 -- 172.21.15.3:0/3991955378 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4ce4120f10 con 0x7f4ce409a3b0 2023-10-03T10:18:29.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.725+0000 7f7b17fe3700 1 -- 172.21.15.3:0/591873594 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7b10074ea0 con 0x7f7b1009e410 2023-10-03T10:18:29.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.725+0000 7f7b067fc700 1 -- 172.21.15.3:0/591873594 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7b0802bb50 con 0x7f7b1009e410 2023-10-03T10:18:29.725 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:29.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.727+0000 7f7b17fe3700 1 -- 172.21.15.3:0/591873594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b1009e410 msgr2=0x7f7b1011e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:29.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.727+0000 7f7b17fe3700 1 --2- 172.21.15.3:0/591873594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b1009e410 0x7f7b1011e970 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f7b08003f50 tx=0x7f7b08004ad0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.727+0000 7f7b17fe3700 1 -- 172.21.15.3:0/591873594 shutdown_connections 2023-10-03T10:18:29.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.727+0000 7f7b17fe3700 1 --2- 172.21.15.3:0/591873594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b1009e410 0x7f7b1011e970 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.727+0000 7f7b17fe3700 1 --2- 172.21.15.3:0/591873594 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b1009d6f0 0x7f7b1011def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.728+0000 7f7b17fe3700 1 --2- 172.21.15.3:0/591873594 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b1009e000 0x7f7b1011e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.728+0000 7f7b17fe3700 1 -- 172.21.15.3:0/591873594 >> 172.21.15.3:0/591873594 conn(0x7f7b10066f50 msgr2=0x7f7b10130a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:29.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.728+0000 7f7b17fe3700 1 -- 172.21.15.3:0/591873594 shutdown_connections 2023-10-03T10:18:29.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.728+0000 7f7b17fe3700 1 -- 172.21.15.3:0/591873594 wait complete. 2023-10-03T10:18:29.741 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:29.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.768+0000 7ff1eb925700 1 -- 172.21.15.3:0/2402840219 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff1c4005590 con 0x7ff1e409e000 2023-10-03T10:18:29.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.768+0000 7ff1d9ffb700 1 -- 172.21.15.3:0/2402840219 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff1d4017020 con 0x7ff1e409e000 2023-10-03T10:18:29.768 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:29.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.771+0000 7ff1eb925700 1 -- 172.21.15.3:0/2402840219 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1e409e000 msgr2=0x7ff1e41a5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:29.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.771+0000 7ff1eb925700 1 --2- 172.21.15.3:0/2402840219 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1e409e000 0x7ff1e41a5750 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff1d400e9b0 tx=0x7ff1d400ecc0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.771+0000 7ff1eb925700 1 -- 172.21.15.3:0/2402840219 shutdown_connections 2023-10-03T10:18:29.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.771+0000 7ff1eb925700 1 --2- 172.21.15.3:0/2402840219 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1e409e410 0x7ff1e41c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.771+0000 7ff1eb925700 1 --2- 172.21.15.3:0/2402840219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1e409d6f0 0x7ff1e41a5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.771+0000 7ff1eb925700 1 --2- 172.21.15.3:0/2402840219 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1e409e000 0x7ff1e41a5750 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.771+0000 7ff1eb925700 1 -- 172.21.15.3:0/2402840219 >> 172.21.15.3:0/2402840219 conn(0x7ff1e4066f50 msgr2=0x7ff1e41bbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:29.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.771+0000 7ff1eb925700 1 -- 172.21.15.3:0/2402840219 shutdown_connections 2023-10-03T10:18:29.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.771+0000 7ff1eb925700 1 -- 172.21.15.3:0/2402840219 wait complete. 2023-10-03T10:18:29.784 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:29.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.810+0000 7f6177fab700 1 -- 172.21.15.3:0/4021345748 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6170074ea0 con 0x7f617009a580 2023-10-03T10:18:29.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.810+0000 7f61667fc700 1 -- 172.21.15.3:0/4021345748 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6160017030 con 0x7f617009a580 2023-10-03T10:18:29.810 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:29.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.813+0000 7f6177fab700 1 -- 172.21.15.3:0/4021345748 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f617009a580 msgr2=0x7f617011d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:29.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.813+0000 7f6177fab700 1 --2- 172.21.15.3:0/4021345748 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f617009a580 0x7f617011d620 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f616000c4b0 tx=0x7f616000c870 comp rx=0 tx=0).stop 2023-10-03T10:18:29.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.813+0000 7f6177fab700 1 -- 172.21.15.3:0/4021345748 shutdown_connections 2023-10-03T10:18:29.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.813+0000 7f6177fab700 1 --2- 172.21.15.3:0/4021345748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6170131810 0x7f617011db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.813+0000 7f6177fab700 1 --2- 172.21.15.3:0/4021345748 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6170099a40 0x7f617011d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.813+0000 7f6177fab700 1 --2- 172.21.15.3:0/4021345748 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f617009a580 0x7f617011d620 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.813+0000 7f6177fab700 1 -- 172.21.15.3:0/4021345748 >> 172.21.15.3:0/4021345748 conn(0x7f61700a2aa0 msgr2=0x7f617012ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:29.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.813+0000 7f6177fab700 1 -- 172.21.15.3:0/4021345748 shutdown_connections 2023-10-03T10:18:29.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.813+0000 7f6177fab700 1 -- 172.21.15.3:0/4021345748 wait complete. 2023-10-03T10:18:29.830 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:29.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.862+0000 7f4cec2f0700 1 -- 172.21.15.3:0/3991955378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4ce4120f10 con 0x7f4ce409a3b0 2023-10-03T10:18:29.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.863+0000 7f4cdaffd700 1 -- 172.21.15.3:0/3991955378 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4ce4120f10 con 0x7f4ce409a3b0 2023-10-03T10:18:29.862 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:29.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.865+0000 7f4cec2f0700 1 -- 172.21.15.3:0/3991955378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ce409a3b0 msgr2=0x7f4ce406fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:29.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.865+0000 7f4cec2f0700 1 --2- 172.21.15.3:0/3991955378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ce409a3b0 0x7f4ce406fb50 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400e980 tx=0x7f4cd400ec90 comp rx=0 tx=0).stop 2023-10-03T10:18:29.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.865+0000 7f4cec2f0700 1 -- 172.21.15.3:0/3991955378 shutdown_connections 2023-10-03T10:18:29.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.865+0000 7f4cec2f0700 1 --2- 172.21.15.3:0/3991955378 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4ce4131810 0x7f4ce4070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.865+0000 7f4cec2f0700 1 --2- 172.21.15.3:0/3991955378 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4ce4099a40 0x7f4ce406f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.865+0000 7f4cec2f0700 1 --2- 172.21.15.3:0/3991955378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ce409a3b0 0x7f4ce406fb50 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:29.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.865+0000 7f4cec2f0700 1 -- 172.21.15.3:0/3991955378 >> 172.21.15.3:0/3991955378 conn(0x7f4ce40a2aa0 msgr2=0x7f4ce41b8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:29.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.865+0000 7f4cec2f0700 1 -- 172.21.15.3:0/3991955378 shutdown_connections 2023-10-03T10:18:29.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:29.865+0000 7f4cec2f0700 1 -- 172.21.15.3:0/3991955378 wait complete. 2023-10-03T10:18:29.877 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:30.741 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:30.784 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:30.831 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:30.878 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:30.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.892+0000 7f1c326e5700 1 Processor -- start 2023-10-03T10:18:30.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.892+0000 7f1c326e5700 1 -- start start 2023-10-03T10:18:30.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.892+0000 7f1c326e5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c2c06a770 0x7f1c2c09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:30.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.892+0000 7f1c326e5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c2c09dc30 0x7f1c2c09e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:30.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.893+0000 7f1c326e5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c2c09a810 con 0x7f1c2c13c1e0 2023-10-03T10:18:30.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.893+0000 7f1c326e5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c06ab40 con 0x7f1c2c06a770 2023-10-03T10:18:30.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.893+0000 7f1c326e5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c09ebc0 con 0x7f1c2c09dc30 2023-10-03T10:18:30.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.893+0000 7f1c2b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c2c09dc30 0x7f1c2c09e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:30.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.893+0000 7f1c2b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c2c09dc30 0x7f1c2c09e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52398/0 (socket says 172.21.15.3:52398) 2023-10-03T10:18:30.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.893+0000 7f1c2b7fe700 1 -- 172.21.15.3:0/1652518586 learned_addr learned my addr 172.21.15.3:0/1652518586 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:30.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.893+0000 7f1c2bfff700 1 --2- 172.21.15.3:0/1652518586 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c2c06a770 0x7f1c2c09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:30.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.893+0000 7f1c2affd700 1 -- 172.21.15.3:0/1652518586 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1094518355 0 0) 0x7f1c2c09a810 con 0x7f1c2c13c1e0 2023-10-03T10:18:30.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.893+0000 7f1c2affd700 1 -- 172.21.15.3:0/1652518586 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c140035b0 con 0x7f1c2c13c1e0 2023-10-03T10:18:30.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.893+0000 7f1c2b7fe700 1 -- 172.21.15.3:0/1652518586 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c2c06a770 msgr2=0x7f1c2c09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:30.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.893+0000 7f1c2b7fe700 1 --2- 172.21.15.3:0/1652518586 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c2c06a770 0x7f1c2c09d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:30.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.893+0000 7f1c2b7fe700 1 -- 172.21.15.3:0/1652518586 >> v1:172.21.15.3:6789/0 conn(0x7f1c2c13c1e0 legacy=0x7f1c2c13e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:30.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.893+0000 7f1c2b7fe700 1 -- 172.21.15.3:0/1652518586 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2c09e920 con 0x7f1c2c09dc30 2023-10-03T10:18:30.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.894+0000 7f1c2bfff700 1 --2- 172.21.15.3:0/1652518586 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c2c06a770 0x7f1c2c09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:30.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.894+0000 7f1c2b7fe700 1 --2- 172.21.15.3:0/1652518586 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c2c09dc30 0x7f1c2c09e080 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009950 tx=0x7f1c20009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48921bd33e7a5507 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:30.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.894+0000 7f1c2affd700 1 -- 172.21.15.3:0/1652518586 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c20011830 con 0x7f1c2c09dc30 2023-10-03T10:18:30.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.894+0000 7f1c2affd700 1 -- 172.21.15.3:0/1652518586 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c20011990 con 0x7f1c2c09dc30 2023-10-03T10:18:30.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.894+0000 7f1c2affd700 1 -- 172.21.15.3:0/1652518586 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c20011d40 con 0x7f1c2c09dc30 2023-10-03T10:18:30.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.895+0000 7f1c326e5700 1 -- 172.21.15.3:0/1652518586 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c2c09dc30 msgr2=0x7f1c2c09e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:30.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.895+0000 7f1c326e5700 1 --2- 172.21.15.3:0/1652518586 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c2c09dc30 0x7f1c2c09e080 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009950 tx=0x7f1c20009c60 comp rx=0 tx=0).stop 2023-10-03T10:18:30.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.895+0000 7f1c326e5700 1 -- 172.21.15.3:0/1652518586 shutdown_connections 2023-10-03T10:18:30.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.895+0000 7f1c326e5700 1 --2- 172.21.15.3:0/1652518586 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c2c06a770 0x7f1c2c09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:30.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.895+0000 7f1c326e5700 1 --2- 172.21.15.3:0/1652518586 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c2c09dc30 0x7f1c2c09e080 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:30.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.895+0000 7f1c326e5700 1 -- 172.21.15.3:0/1652518586 >> 172.21.15.3:0/1652518586 conn(0x7f1c2c066f50 msgr2=0x7f1c2c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:30.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.895+0000 7f1c326e5700 1 -- 172.21.15.3:0/1652518586 shutdown_connections 2023-10-03T10:18:30.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.895+0000 7f1c326e5700 1 -- 172.21.15.3:0/1652518586 wait complete. 2023-10-03T10:18:30.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.896+0000 7f1c326e5700 1 Processor -- start 2023-10-03T10:18:30.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.897+0000 7f1c326e5700 1 -- start start 2023-10-03T10:18:30.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.897+0000 7f1c326e5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c2c06a770 0x7f1c2c1cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:30.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.897+0000 7f1c326e5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c2c09dc30 0x7f1c2c1ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:30.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.897+0000 7f1c326e5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c2c13c1e0 0x7f1c2c1270b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:30.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.897+0000 7f1c326e5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c1c12c0 con 0x7f1c2c13c1e0 2023-10-03T10:18:30.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.897+0000 7f1c326e5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c0a1b50 con 0x7f1c2c09dc30 2023-10-03T10:18:30.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.897+0000 7f1c326e5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c09f1a0 con 0x7f1c2c06a770 2023-10-03T10:18:30.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.897+0000 7f1c2bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c2c06a770 0x7f1c2c1cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:30.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.897+0000 7f1c2bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c2c06a770 0x7f1c2c1cc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52412/0 (socket says 172.21.15.3:52412) 2023-10-03T10:18:30.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.897+0000 7f1c2bfff700 1 -- 172.21.15.3:0/62999053 learned_addr learned my addr 172.21.15.3:0/62999053 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:30.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.897+0000 7f1c2b7fe700 1 --2- 172.21.15.3:0/62999053 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c2c09dc30 0x7f1c2c1ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:30.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.898+0000 7f1c30c82700 1 --2- 172.21.15.3:0/62999053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c2c13c1e0 0x7f1c2c1270b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:30.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.898+0000 7f1c2bfff700 1 -- 172.21.15.3:0/62999053 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c2c09dc30 msgr2=0x7f1c2c1ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:30.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.898+0000 7f1c2bfff700 1 --2- 172.21.15.3:0/62999053 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c2c09dc30 0x7f1c2c1ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:30.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.898+0000 7f1c2bfff700 1 -- 172.21.15.3:0/62999053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c2c13c1e0 msgr2=0x7f1c2c1270b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:30.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.898+0000 7f1c2bfff700 1 --2- 172.21.15.3:0/62999053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c2c13c1e0 0x7f1c2c1270b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:30.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.898+0000 7f1c2bfff700 1 -- 172.21.15.3:0/62999053 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1c0094a0 con 0x7f1c2c06a770 2023-10-03T10:18:30.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.898+0000 7f1c2bfff700 1 --2- 172.21.15.3:0/62999053 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c2c06a770 0x7f1c2c1cc960 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c00ebe0 tx=0x7f1c1c00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:30.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.898+0000 7f1c28ff9700 1 -- 172.21.15.3:0/62999053 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c00c850 con 0x7f1c2c06a770 2023-10-03T10:18:30.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.898+0000 7f1c28ff9700 1 -- 172.21.15.3:0/62999053 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1c00c9b0 con 0x7f1c2c06a770 2023-10-03T10:18:30.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.898+0000 7f1c326e5700 1 -- 172.21.15.3:0/62999053 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c20009550 con 0x7f1c2c06a770 2023-10-03T10:18:30.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.899+0000 7f1c28ff9700 1 -- 172.21.15.3:0/62999053 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c00cd20 con 0x7f1c2c06a770 2023-10-03T10:18:30.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.899+0000 7f1c326e5700 1 -- 172.21.15.3:0/62999053 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c2c12a630 con 0x7f1c2c06a770 2023-10-03T10:18:30.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.900+0000 7f1c326e5700 1 -- 172.21.15.3:0/62999053 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c2c12ab10 con 0x7f1c2c06a770 2023-10-03T10:18:30.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.901+0000 7f1c28ff9700 1 -- 172.21.15.3:0/62999053 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1c1c013470 con 0x7f1c2c06a770 2023-10-03T10:18:30.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.901+0000 7f1c28ff9700 1 -- 172.21.15.3:0/62999053 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1c1c03fda0 con 0x7f1c2c06a770 2023-10-03T10:18:30.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.909+0000 7f1c28ff9700 1 -- 172.21.15.3:0/62999053 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1c1c00ce80 con 0x7f1c2c06a770 2023-10-03T10:18:30.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.939+0000 7f1eb9178700 1 Processor -- start 2023-10-03T10:18:30.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.939+0000 7f1eb9178700 1 -- start start 2023-10-03T10:18:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.939+0000 7f1eb9178700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1eb41316c0 0x7f1eb4131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.939+0000 7f1eb9178700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1eb409d9f0 0x7f1eb406a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.939+0000 7f1eb9178700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1eb406acb0 0x7f1eb406b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.939+0000 7f1eb9178700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb406b630 con 0x7f1eb409d9f0 2023-10-03T10:18:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.939+0000 7f1eb9178700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb409e130 con 0x7f1eb406acb0 2023-10-03T10:18:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.939+0000 7f1eb9178700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb41312b0 con 0x7f1eb41316c0 2023-10-03T10:18:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.940+0000 7f1eb259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1eb409d9f0 0x7f1eb406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.940+0000 7f1eb259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1eb409d9f0 0x7f1eb406a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58748/0 (socket says 172.21.15.3:58748) 2023-10-03T10:18:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.940+0000 7f1eb259c700 1 -- 172.21.15.3:0/138132705 learned_addr learned my addr 172.21.15.3:0/138132705 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.940+0000 7f1eb2d9d700 1 --2- 172.21.15.3:0/138132705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1eb41316c0 0x7f1eb4131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:30.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.940+0000 7f1eb359e700 1 --2- 172.21.15.3:0/138132705 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1eb406acb0 0x7f1eb406b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:30.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.940+0000 7f1eb259c700 1 -- 172.21.15.3:0/138132705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1eb41316c0 msgr2=0x7f1eb4131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:30.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.940+0000 7f1eb259c700 1 --2- 172.21.15.3:0/138132705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1eb41316c0 0x7f1eb4131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:30.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.940+0000 7f1eb259c700 1 -- 172.21.15.3:0/138132705 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1eb406acb0 msgr2=0x7f1eb406b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:30.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.940+0000 7f1eb259c700 1 --2- 172.21.15.3:0/138132705 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1eb406acb0 0x7f1eb406b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:30.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.940+0000 7f1eb259c700 1 -- 172.21.15.3:0/138132705 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb409f6e0 con 0x7f1eb409d9f0 2023-10-03T10:18:30.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.940+0000 7f1eb359e700 1 --2- 172.21.15.3:0/138132705 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1eb406acb0 0x7f1eb406b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:30.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.940+0000 7f1eb259c700 1 --2- 172.21.15.3:0/138132705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1eb409d9f0 0x7f1eb406a770 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80098f0 tx=0x7f1ea8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c126db7eb8ef96 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:30.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.941+0000 7f1eb1d9b700 1 -- 172.21.15.3:0/138132705 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8011830 con 0x7f1eb409d9f0 2023-10-03T10:18:30.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.941+0000 7f1eb1d9b700 1 -- 172.21.15.3:0/138132705 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea8011990 con 0x7f1eb409d9f0 2023-10-03T10:18:30.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.941+0000 7f1eb1d9b700 1 -- 172.21.15.3:0/138132705 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8011d20 con 0x7f1eb409d9f0 2023-10-03T10:18:30.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.941+0000 7f1eb9178700 1 -- 172.21.15.3:0/138132705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1eb409d9f0 msgr2=0x7f1eb406a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:30.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.941+0000 7f1eb9178700 1 --2- 172.21.15.3:0/138132705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1eb409d9f0 0x7f1eb406a770 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80098f0 tx=0x7f1ea8009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:30.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.941+0000 7f1eb9178700 1 -- 172.21.15.3:0/138132705 shutdown_connections 2023-10-03T10:18:30.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.941+0000 7f1eb9178700 1 --2- 172.21.15.3:0/138132705 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1eb406acb0 0x7f1eb406b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:30.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.941+0000 7f1eb9178700 1 --2- 172.21.15.3:0/138132705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1eb41316c0 0x7f1eb4131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:30.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.941+0000 7f1eb9178700 1 --2- 172.21.15.3:0/138132705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1eb409d9f0 0x7f1eb406a770 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:30.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.941+0000 7f1eb9178700 1 -- 172.21.15.3:0/138132705 >> 172.21.15.3:0/138132705 conn(0x7f1eb4066f50 msgr2=0x7f1eb409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:30.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.942+0000 7f1eb9178700 1 -- 172.21.15.3:0/138132705 shutdown_connections 2023-10-03T10:18:30.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.942+0000 7f1eb9178700 1 -- 172.21.15.3:0/138132705 wait complete. 2023-10-03T10:18:30.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.943+0000 7f1eb9178700 1 Processor -- start 2023-10-03T10:18:30.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.943+0000 7f1eb9178700 1 -- start start 2023-10-03T10:18:30.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.943+0000 7f1eb9178700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1eb406acb0 0x7f1eb40709c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:30.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.944+0000 7f1eb9178700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1eb409d9f0 0x7f1eb4070f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:30.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.944+0000 7f1eb9178700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1eb41316c0 0x7f1eb4071440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:30.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.944+0000 7f1eb9178700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb412b900 con 0x7f1eb409d9f0 2023-10-03T10:18:30.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.944+0000 7f1eb9178700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb411a9d0 con 0x7f1eb406acb0 2023-10-03T10:18:30.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.944+0000 7f1eb9178700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb41198c0 con 0x7f1eb41316c0 2023-10-03T10:18:30.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.944+0000 7f1eb2d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1eb406acb0 0x7f1eb40709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:30.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.944+0000 7f1eb2d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1eb406acb0 0x7f1eb40709c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52318/0 (socket says 172.21.15.3:52318) 2023-10-03T10:18:30.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.944+0000 7f1eb2d9d700 1 -- 172.21.15.3:0/3245155431 learned_addr learned my addr 172.21.15.3:0/3245155431 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:30.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.944+0000 7f1eb359e700 1 --2- 172.21.15.3:0/3245155431 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1eb41316c0 0x7f1eb4071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:30.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.944+0000 7f1eb259c700 1 --2- 172.21.15.3:0/3245155431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1eb409d9f0 0x7f1eb4070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:30.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.944+0000 7f1eb2d9d700 1 -- 172.21.15.3:0/3245155431 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1eb41316c0 msgr2=0x7f1eb4071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:30.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.944+0000 7f1eb2d9d700 1 --2- 172.21.15.3:0/3245155431 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1eb41316c0 0x7f1eb4071440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:30.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.944+0000 7f1eb2d9d700 1 -- 172.21.15.3:0/3245155431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1eb409d9f0 msgr2=0x7f1eb4070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:30.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.944+0000 7f1eb2d9d700 1 --2- 172.21.15.3:0/3245155431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1eb409d9f0 0x7f1eb4070f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:30.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.944+0000 7f1eb2d9d700 1 -- 172.21.15.3:0/3245155431 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eac009a70 con 0x7f1eb406acb0 2023-10-03T10:18:30.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.944+0000 7f1eb359e700 1 --2- 172.21.15.3:0/3245155431 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1eb41316c0 0x7f1eb4071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:18:30.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.944+0000 7f1eb2d9d700 1 --2- 172.21.15.3:0/3245155431 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1eb406acb0 0x7f1eb40709c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400e9e0 tx=0x7f1ea400ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:30.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.945+0000 7f1e9b7fe700 1 -- 172.21.15.3:0/3245155431 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4009710 con 0x7f1eb406acb0 2023-10-03T10:18:30.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.945+0000 7f1e9b7fe700 1 -- 172.21.15.3:0/3245155431 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea400c850 con 0x7f1eb406acb0 2023-10-03T10:18:30.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.945+0000 7f1e9b7fe700 1 -- 172.21.15.3:0/3245155431 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea400cbc0 con 0x7f1eb406acb0 2023-10-03T10:18:30.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.945+0000 7f1eb9178700 1 -- 172.21.15.3:0/3245155431 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea8009550 con 0x7f1eb406acb0 2023-10-03T10:18:30.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.945+0000 7f1eb9178700 1 -- 172.21.15.3:0/3245155431 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb411ac90 con 0x7f1eb406acb0 2023-10-03T10:18:30.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.946+0000 7f1e9b7fe700 1 -- 172.21.15.3:0/3245155431 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1ea400cd20 con 0x7f1eb406acb0 2023-10-03T10:18:30.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.946+0000 7f1eb9178700 1 -- 172.21.15.3:0/3245155431 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eb411ae80 con 0x7f1eb406acb0 2023-10-03T10:18:30.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.946+0000 7f1e9b7fe700 1 -- 172.21.15.3:0/3245155431 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1ea403f790 con 0x7f1eb406acb0 2023-10-03T10:18:30.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.956+0000 7f1e9b7fe700 1 -- 172.21.15.3:0/3245155431 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1ea401be10 con 0x7f1eb406acb0 2023-10-03T10:18:30.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.980+0000 7fdea19b8700 1 Processor -- start 2023-10-03T10:18:30.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.980+0000 7fdea19b8700 1 -- start start 2023-10-03T10:18:30.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.981+0000 7fdea19b8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde9c131630 0x7fde9c131a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:30.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.981+0000 7fdea19b8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fde9c06eba0 0x7fde9c139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:30.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.981+0000 7fdea19b8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde9c071a00 con 0x7fde9c131f40 2023-10-03T10:18:30.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.981+0000 7fdea19b8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c131430 con 0x7fde9c06eba0 2023-10-03T10:18:30.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.981+0000 7fdea19b8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c09a510 con 0x7fde9c131630 2023-10-03T10:18:30.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.981+0000 7fde9b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fde9c06eba0 0x7fde9c139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:30.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.981+0000 7fde9b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fde9c06eba0 0x7fde9c139660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52324/0 (socket says 172.21.15.3:52324) 2023-10-03T10:18:30.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.981+0000 7fde9affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde9c131630 0x7fde9c131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:30.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.981+0000 7fde9b7fe700 1 -- 172.21.15.3:0/3199303559 learned_addr learned my addr 172.21.15.3:0/3199303559 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:30.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.981+0000 7fde9a7fc700 1 --1- 172.21.15.3:0/3199303559 >> v1:172.21.15.3:6789/0 conn(0x7fde9c131f40 0x7fde9c06e490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47168/0 (socket says 172.21.15.3:47168) 2023-10-03T10:18:30.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.981+0000 7fde9affd700 1 -- 172.21.15.3:0/3199303559 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fde9c06eba0 msgr2=0x7fde9c139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:30.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.981+0000 7fde9affd700 1 --2- 172.21.15.3:0/3199303559 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fde9c06eba0 0x7fde9c139660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:30.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.981+0000 7fde9affd700 1 -- 172.21.15.3:0/3199303559 >> v1:172.21.15.3:6789/0 conn(0x7fde9c131f40 legacy=0x7fde9c06e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:30.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.981+0000 7fde9affd700 1 -- 172.21.15.3:0/3199303559 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9c0721c0 con 0x7fde9c131630 2023-10-03T10:18:30.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.982+0000 7fde9affd700 1 --2- 172.21.15.3:0/3199303559 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde9c131630 0x7fde9c131a00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009870 tx=0x7fde8c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93433fe8fb2716f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:30.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.982+0000 7fde99ffb700 1 -- 172.21.15.3:0/3199303559 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c011830 con 0x7fde9c131630 2023-10-03T10:18:30.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.982+0000 7fde99ffb700 1 -- 172.21.15.3:0/3199303559 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8c011990 con 0x7fde9c131630 2023-10-03T10:18:30.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.982+0000 7fde99ffb700 1 -- 172.21.15.3:0/3199303559 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c011d20 con 0x7fde9c131630 2023-10-03T10:18:30.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.983+0000 7fdea19b8700 1 -- 172.21.15.3:0/3199303559 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde9c131630 msgr2=0x7fde9c131a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:30.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.983+0000 7fdea19b8700 1 --2- 172.21.15.3:0/3199303559 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde9c131630 0x7fde9c131a00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009870 tx=0x7fde8c009b80 comp rx=0 tx=0).stop 2023-10-03T10:18:30.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.983+0000 7fdea19b8700 1 -- 172.21.15.3:0/3199303559 shutdown_connections 2023-10-03T10:18:30.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.983+0000 7fdea19b8700 1 --2- 172.21.15.3:0/3199303559 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fde9c06eba0 0x7fde9c139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:30.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.983+0000 7fdea19b8700 1 --2- 172.21.15.3:0/3199303559 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde9c131630 0x7fde9c131a00 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:30.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.983+0000 7fdea19b8700 1 -- 172.21.15.3:0/3199303559 >> 172.21.15.3:0/3199303559 conn(0x7fde9c0a2aa0 msgr2=0x7fde9c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:30.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.983+0000 7fdea19b8700 1 -- 172.21.15.3:0/3199303559 shutdown_connections 2023-10-03T10:18:30.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.983+0000 7fdea19b8700 1 -- 172.21.15.3:0/3199303559 wait complete. 2023-10-03T10:18:30.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.985+0000 7fdea19b8700 1 Processor -- start 2023-10-03T10:18:30.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.985+0000 7fdea19b8700 1 -- start start 2023-10-03T10:18:30.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.986+0000 7fdea19b8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde9c06eba0 0x7fde9c11b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:30.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.986+0000 7fdea19b8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde9c131630 0x7fde9c11bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:30.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.986+0000 7fdea19b8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fde9c131f40 0x7fde9c11c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:30.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.986+0000 7fdea19b8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c11cd80 con 0x7fde9c06eba0 2023-10-03T10:18:30.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.986+0000 7fdea19b8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c12ec10 con 0x7fde9c131f40 2023-10-03T10:18:30.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.986+0000 7fdea19b8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c11e680 con 0x7fde9c131630 2023-10-03T10:18:30.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.986+0000 7fde9b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fde9c131f40 0x7fde9c11c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:30.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.986+0000 7fde9b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fde9c131f40 0x7fde9c11c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52332/0 (socket says 172.21.15.3:52332) 2023-10-03T10:18:30.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.986+0000 7fde9b7fe700 1 -- 172.21.15.3:0/2547458830 learned_addr learned my addr 172.21.15.3:0/2547458830 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:30.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.986+0000 7fde9a7fc700 1 --2- 172.21.15.3:0/2547458830 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde9c131630 0x7fde9c11bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:30.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.986+0000 7fde9affd700 1 --2- 172.21.15.3:0/2547458830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde9c06eba0 0x7fde9c11b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:30.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.986+0000 7fde9a7fc700 1 -- 172.21.15.3:0/2547458830 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fde9c131f40 msgr2=0x7fde9c11c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:30.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.986+0000 7fde9a7fc700 1 --2- 172.21.15.3:0/2547458830 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fde9c131f40 0x7fde9c11c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:30.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.986+0000 7fde9a7fc700 1 -- 172.21.15.3:0/2547458830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde9c06eba0 msgr2=0x7fde9c11b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:30.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.986+0000 7fde9a7fc700 1 --2- 172.21.15.3:0/2547458830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde9c06eba0 0x7fde9c11b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:30.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.986+0000 7fde9a7fc700 1 -- 172.21.15.3:0/2547458830 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8c0094a0 con 0x7fde9c131630 2023-10-03T10:18:30.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.987+0000 7fde9a7fc700 1 --2- 172.21.15.3:0/2547458830 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde9c131630 0x7fde9c11bdf0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fde9000c450 tx=0x7fde9000c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:30.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.987+0000 7fde83fff700 1 -- 172.21.15.3:0/2547458830 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde90005cb0 con 0x7fde9c131630 2023-10-03T10:18:30.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.987+0000 7fde83fff700 1 -- 172.21.15.3:0/2547458830 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde90005e10 con 0x7fde9c131630 2023-10-03T10:18:30.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.987+0000 7fdea19b8700 1 -- 172.21.15.3:0/2547458830 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde9c11ead0 con 0x7fde9c131630 2023-10-03T10:18:30.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.987+0000 7fde83fff700 1 -- 172.21.15.3:0/2547458830 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde900072d0 con 0x7fde9c131630 2023-10-03T10:18:30.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.988+0000 7fde83fff700 1 -- 172.21.15.3:0/2547458830 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fde90007430 con 0x7fde9c131630 2023-10-03T10:18:30.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.988+0000 7fdea19b8700 1 -- 172.21.15.3:0/2547458830 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde9c09f940 con 0x7fde9c131630 2023-10-03T10:18:30.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.990+0000 7fde83fff700 1 -- 172.21.15.3:0/2547458830 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fde9000ce20 con 0x7fde9c131630 2023-10-03T10:18:30.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.990+0000 7fdea19b8700 1 -- 172.21.15.3:0/2547458830 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde9c11f180 con 0x7fde9c131630 2023-10-03T10:18:30.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:30.999+0000 7fde83fff700 1 -- 172.21.15.3:0/2547458830 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fde90017030 con 0x7fde9c131630 2023-10-03T10:18:31.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.037+0000 7f7b21e28700 1 Processor -- start 2023-10-03T10:18:31.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.037+0000 7f7b21e28700 1 -- start start 2023-10-03T10:18:31.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.037+0000 7f7b21e28700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b1c09d6f0 0x7f7b1c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:31.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.038+0000 7f7b21e28700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7b1c09e000 0x7f7b1c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:31.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.038+0000 7f7b21e28700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b1c09e410 0x7f7b1c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:31.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.038+0000 7f7b21e28700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b1c06b460 con 0x7f7b1c09e000 2023-10-03T10:18:31.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.038+0000 7f7b21e28700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c09b120 con 0x7f7b1c09e410 2023-10-03T10:18:31.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.038+0000 7f7b21e28700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c09a7e0 con 0x7f7b1c09d6f0 2023-10-03T10:18:31.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.038+0000 7f7b1b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b1c09d6f0 0x7f7b1c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:31.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.038+0000 7f7b1affd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7b1c09e000 0x7f7b1c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:31.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.038+0000 7f7b1b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b1c09d6f0 0x7f7b1c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52430/0 (socket says 172.21.15.3:52430) 2023-10-03T10:18:31.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.038+0000 7f7b1b7fe700 1 -- 172.21.15.3:0/2932158419 learned_addr learned my addr 172.21.15.3:0/2932158419 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:31.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.038+0000 7f7b1bfff700 1 --2- 172.21.15.3:0/2932158419 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b1c09e410 0x7f7b1c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:31.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.038+0000 7f7b1b7fe700 1 -- 172.21.15.3:0/2932158419 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b1c09e410 msgr2=0x7f7b1c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:31.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.038+0000 7f7b1b7fe700 1 --2- 172.21.15.3:0/2932158419 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b1c09e410 0x7f7b1c133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:31.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.038+0000 7f7b1b7fe700 1 -- 172.21.15.3:0/2932158419 >> v2:172.21.15.3:3300/0 conn(0x7f7b1c09e000 msgr2=0x7f7b1c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:31.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.038+0000 7f7b1b7fe700 1 --2- 172.21.15.3:0/2932158419 >> v2:172.21.15.3:3300/0 conn(0x7f7b1c09e000 0x7f7b1c1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:31.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.038+0000 7f7b1b7fe700 1 -- 172.21.15.3:0/2932158419 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1c09a500 con 0x7f7b1c09d6f0 2023-10-03T10:18:31.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.038+0000 7f7b1affd700 1 --2- 172.21.15.3:0/2932158419 >> v2:172.21.15.3:3300/0 conn(0x7f7b1c09e000 0x7f7b1c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:31.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.038+0000 7f7b1b7fe700 1 --2- 172.21.15.3:0/2932158419 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b1c09d6f0 0x7f7b1c09dac0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009920 tx=0x7f7b0c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d37655222dc86b39 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:31.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.039+0000 7f7b1a7fc700 1 -- 172.21.15.3:0/2932158419 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c011830 con 0x7f7b1c09d6f0 2023-10-03T10:18:31.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.039+0000 7f7b1a7fc700 1 -- 172.21.15.3:0/2932158419 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0c011990 con 0x7f7b1c09d6f0 2023-10-03T10:18:31.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.039+0000 7f7b1a7fc700 1 -- 172.21.15.3:0/2932158419 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c011d20 con 0x7f7b1c09d6f0 2023-10-03T10:18:31.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.040+0000 7f7b21e28700 1 -- 172.21.15.3:0/2932158419 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b1c09d6f0 msgr2=0x7f7b1c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:31.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.040+0000 7f7b21e28700 1 --2- 172.21.15.3:0/2932158419 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b1c09d6f0 0x7f7b1c09dac0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009920 tx=0x7f7b0c009c30 comp rx=0 tx=0).stop 2023-10-03T10:18:31.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.040+0000 7f7b21e28700 1 -- 172.21.15.3:0/2932158419 shutdown_connections 2023-10-03T10:18:31.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.040+0000 7f7b21e28700 1 --2- 172.21.15.3:0/2932158419 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b1c09e410 0x7f7b1c133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:31.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.040+0000 7f7b21e28700 1 --2- 172.21.15.3:0/2932158419 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b1c09d6f0 0x7f7b1c09dac0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:31.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.040+0000 7f7b21e28700 1 --2- 172.21.15.3:0/2932158419 >> v2:172.21.15.3:3300/0 conn(0x7f7b1c09e000 0x7f7b1c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:31.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.040+0000 7f7b21e28700 1 -- 172.21.15.3:0/2932158419 >> 172.21.15.3:0/2932158419 conn(0x7f7b1c066f50 msgr2=0x7f7b1c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:31.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.040+0000 7f7b21e28700 1 -- 172.21.15.3:0/2932158419 shutdown_connections 2023-10-03T10:18:31.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.040+0000 7f7b21e28700 1 -- 172.21.15.3:0/2932158419 wait complete. 2023-10-03T10:18:31.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.041+0000 7f7b21e28700 1 Processor -- start 2023-10-03T10:18:31.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.042+0000 7f7b21e28700 1 -- start start 2023-10-03T10:18:31.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.042+0000 7f7b21e28700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b1c09d6f0 0x7f7b1c1a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:31.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.042+0000 7f7b21e28700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b1c09e000 0x7f7b1c1a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:31.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.043+0000 7f7b21e28700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b1c09e410 0x7f7b1c1a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:31.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.043+0000 7f7b21e28700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c1a8150 con 0x7f7b1c09e410 2023-10-03T10:18:31.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.043+0000 7f7b21e28700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c1a92a0 con 0x7f7b1c09e000 2023-10-03T10:18:31.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.043+0000 7f7b21e28700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c1a7c20 con 0x7f7b1c09d6f0 2023-10-03T10:18:31.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.043+0000 7f7b1b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b1c09d6f0 0x7f7b1c1a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:31.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.043+0000 7f7b1affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b1c09e000 0x7f7b1c1a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:31.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.043+0000 7f7b1b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b1c09d6f0 0x7f7b1c1a4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52440/0 (socket says 172.21.15.3:52440) 2023-10-03T10:18:31.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.043+0000 7f7b1b7fe700 1 -- 172.21.15.3:0/2333718653 learned_addr learned my addr 172.21.15.3:0/2333718653 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:31.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.043+0000 7f7b1bfff700 1 --2- 172.21.15.3:0/2333718653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b1c09e410 0x7f7b1c1a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:31.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.043+0000 7f7b1affd700 1 -- 172.21.15.3:0/2333718653 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b1c09d6f0 msgr2=0x7f7b1c1a4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:31.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.043+0000 7f7b1affd700 1 --2- 172.21.15.3:0/2333718653 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b1c09d6f0 0x7f7b1c1a4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:31.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.043+0000 7f7b1affd700 1 -- 172.21.15.3:0/2333718653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b1c09e410 msgr2=0x7f7b1c1a58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:31.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.043+0000 7f7b1affd700 1 --2- 172.21.15.3:0/2333718653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b1c09e410 0x7f7b1c1a58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:31.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.043+0000 7f7b1affd700 1 -- 172.21.15.3:0/2333718653 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b100094a0 con 0x7f7b1c09e000 2023-10-03T10:18:31.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.043+0000 7f7b1affd700 1 --2- 172.21.15.3:0/2333718653 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b1c09e000 0x7f7b1c1a5390 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f7b1000e9b0 tx=0x7f7b1000ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:31.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.044+0000 7f7b03fff700 1 -- 172.21.15.3:0/2333718653 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1000ee50 con 0x7f7b1c09e000 2023-10-03T10:18:31.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.044+0000 7f7b03fff700 1 -- 172.21.15.3:0/2333718653 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1000c850 con 0x7f7b1c09e000 2023-10-03T10:18:31.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.044+0000 7f7b21e28700 1 -- 172.21.15.3:0/2333718653 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b0c009550 con 0x7f7b1c09e000 2023-10-03T10:18:31.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.044+0000 7f7b03fff700 1 -- 172.21.15.3:0/2333718653 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1000cbc0 con 0x7f7b1c09e000 2023-10-03T10:18:31.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.044+0000 7f7b21e28700 1 -- 172.21.15.3:0/2333718653 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b1c09c710 con 0x7f7b1c09e000 2023-10-03T10:18:31.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.045+0000 7f7b03fff700 1 -- 172.21.15.3:0/2333718653 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7b1000cd20 con 0x7f7b1c09e000 2023-10-03T10:18:31.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.045+0000 7f7b03fff700 1 -- 172.21.15.3:0/2333718653 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7b1003f7d0 con 0x7f7b1c09e000 2023-10-03T10:18:31.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.045+0000 7f7b21e28700 1 -- 172.21.15.3:0/2333718653 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b1c074ea0 con 0x7f7b1c09e000 2023-10-03T10:18:31.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.055+0000 7f7b03fff700 1 -- 172.21.15.3:0/2333718653 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7b10017080 con 0x7f7b1c09e000 2023-10-03T10:18:31.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.169+0000 7f1c326e5700 1 -- 172.21.15.3:0/62999053 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1c2c12ab10 con 0x7f1c2c06a770 2023-10-03T10:18:31.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.169+0000 7f1c28ff9700 1 -- 172.21.15.3:0/62999053 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1c2c12ab10 con 0x7f1c2c06a770 2023-10-03T10:18:31.169 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:31.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.172+0000 7f1c326e5700 1 -- 172.21.15.3:0/62999053 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c2c06a770 msgr2=0x7f1c2c1cc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:31.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.172+0000 7f1c326e5700 1 --2- 172.21.15.3:0/62999053 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c2c06a770 0x7f1c2c1cc960 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c00ebe0 tx=0x7f1c1c00eef0 comp rx=0 tx=0).stop 2023-10-03T10:18:31.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.173+0000 7f1c326e5700 1 -- 172.21.15.3:0/62999053 shutdown_connections 2023-10-03T10:18:31.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.173+0000 7f1c326e5700 1 --2- 172.21.15.3:0/62999053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c2c13c1e0 0x7f1c2c1270b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:31.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.173+0000 7f1c326e5700 1 --2- 172.21.15.3:0/62999053 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c2c06a770 0x7f1c2c1cc960 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:31.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.173+0000 7f1c326e5700 1 --2- 172.21.15.3:0/62999053 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c2c09dc30 0x7f1c2c1ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:31.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.173+0000 7f1c326e5700 1 -- 172.21.15.3:0/62999053 >> 172.21.15.3:0/62999053 conn(0x7f1c2c066f50 msgr2=0x7f1c2c09b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:31.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.173+0000 7f1c326e5700 1 -- 172.21.15.3:0/62999053 shutdown_connections 2023-10-03T10:18:31.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.173+0000 7f1c326e5700 1 -- 172.21.15.3:0/62999053 wait complete. 2023-10-03T10:18:31.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.176+0000 7f1eb9178700 1 -- 172.21.15.3:0/3245155431 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1eb4074ea0 con 0x7f1eb406acb0 2023-10-03T10:18:31.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.176+0000 7f1e9b7fe700 1 -- 172.21.15.3:0/3245155431 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1ea4017090 con 0x7f1eb406acb0 2023-10-03T10:18:31.176 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:31.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.178+0000 7f1eb9178700 1 -- 172.21.15.3:0/3245155431 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1eb406acb0 msgr2=0x7f1eb40709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:31.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.178+0000 7f1eb9178700 1 --2- 172.21.15.3:0/3245155431 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1eb406acb0 0x7f1eb40709c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400e9e0 tx=0x7f1ea400ecf0 comp rx=0 tx=0).stop 2023-10-03T10:18:31.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.178+0000 7f1eb9178700 1 -- 172.21.15.3:0/3245155431 shutdown_connections 2023-10-03T10:18:31.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.178+0000 7f1eb9178700 1 --2- 172.21.15.3:0/3245155431 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1eb41316c0 0x7f1eb4071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:31.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.178+0000 7f1eb9178700 1 --2- 172.21.15.3:0/3245155431 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1eb406acb0 0x7f1eb40709c0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:31.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.178+0000 7f1eb9178700 1 --2- 172.21.15.3:0/3245155431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1eb409d9f0 0x7f1eb4070f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:31.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.178+0000 7f1eb9178700 1 -- 172.21.15.3:0/3245155431 >> 172.21.15.3:0/3245155431 conn(0x7f1eb4066f50 msgr2=0x7f1eb40a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:31.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.178+0000 7f1eb9178700 1 -- 172.21.15.3:0/3245155431 shutdown_connections 2023-10-03T10:18:31.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.178+0000 7f1eb9178700 1 -- 172.21.15.3:0/3245155431 wait complete. 2023-10-03T10:18:31.193 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:31.194 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:31.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.223+0000 7fdea19b8700 1 -- 172.21.15.3:0/2547458830 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fde9c074ea0 con 0x7fde9c131630 2023-10-03T10:18:31.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.224+0000 7fde83fff700 1 -- 172.21.15.3:0/2547458830 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fde90044060 con 0x7fde9c131630 2023-10-03T10:18:31.226 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:31.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.226+0000 7fdea19b8700 1 -- 172.21.15.3:0/2547458830 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde9c131630 msgr2=0x7fde9c11bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:31.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.226+0000 7fdea19b8700 1 --2- 172.21.15.3:0/2547458830 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde9c131630 0x7fde9c11bdf0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fde9000c450 tx=0x7fde9000c760 comp rx=0 tx=0).stop 2023-10-03T10:18:31.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.226+0000 7fdea19b8700 1 -- 172.21.15.3:0/2547458830 shutdown_connections 2023-10-03T10:18:31.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.226+0000 7fdea19b8700 1 --2- 172.21.15.3:0/2547458830 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fde9c131f40 0x7fde9c11c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:31.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.226+0000 7fdea19b8700 1 --2- 172.21.15.3:0/2547458830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde9c06eba0 0x7fde9c11b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:31.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.226+0000 7fdea19b8700 1 --2- 172.21.15.3:0/2547458830 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde9c131630 0x7fde9c11bdf0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:31.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.226+0000 7fdea19b8700 1 -- 172.21.15.3:0/2547458830 >> 172.21.15.3:0/2547458830 conn(0x7fde9c0a2aa0 msgr2=0x7fde9c09d680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:31.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.226+0000 7fdea19b8700 1 -- 172.21.15.3:0/2547458830 shutdown_connections 2023-10-03T10:18:31.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.226+0000 7fdea19b8700 1 -- 172.21.15.3:0/2547458830 wait complete. 2023-10-03T10:18:31.240 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:31.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.287+0000 7f7b21e28700 1 -- 172.21.15.3:0/2333718653 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7b1c06c360 con 0x7f7b1c09e000 2023-10-03T10:18:31.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.288+0000 7f7b03fff700 1 -- 172.21.15.3:0/2333718653 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7b10021020 con 0x7f7b1c09e000 2023-10-03T10:18:31.288 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:31.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.290+0000 7f7b21e28700 1 -- 172.21.15.3:0/2333718653 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b1c09e000 msgr2=0x7f7b1c1a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:31.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.290+0000 7f7b21e28700 1 --2- 172.21.15.3:0/2333718653 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b1c09e000 0x7f7b1c1a5390 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f7b1000e9b0 tx=0x7f7b1000ecc0 comp rx=0 tx=0).stop 2023-10-03T10:18:31.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.290+0000 7f7b21e28700 1 -- 172.21.15.3:0/2333718653 shutdown_connections 2023-10-03T10:18:31.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.290+0000 7f7b21e28700 1 --2- 172.21.15.3:0/2333718653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b1c09e410 0x7f7b1c1a58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:31.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.290+0000 7f7b21e28700 1 --2- 172.21.15.3:0/2333718653 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7b1c09d6f0 0x7f7b1c1a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:31.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.290+0000 7f7b21e28700 1 --2- 172.21.15.3:0/2333718653 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7b1c09e000 0x7f7b1c1a5390 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:31.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.290+0000 7f7b21e28700 1 -- 172.21.15.3:0/2333718653 >> 172.21.15.3:0/2333718653 conn(0x7f7b1c066f50 msgr2=0x7f7b1c1bb010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:31.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.290+0000 7f7b21e28700 1 -- 172.21.15.3:0/2333718653 shutdown_connections 2023-10-03T10:18:31.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:31.290+0000 7f7b21e28700 1 -- 172.21.15.3:0/2333718653 wait complete. 2023-10-03T10:18:31.304 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:32.194 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:32.195 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:32.241 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:32.304 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:32.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.358+0000 7f05bde50700 1 Processor -- start 2023-10-03T10:18:32.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.358+0000 7f05bde50700 1 -- start start 2023-10-03T10:18:32.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.359+0000 7f05bde50700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f05b80a0a30 0x7f05b80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:32.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.359+0000 7f05bde50700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f05b809d630 0x7f05b80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:32.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.359+0000 7f05bde50700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05b80a2a70 con 0x7f05b8133930 2023-10-03T10:18:32.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.359+0000 7f05bde50700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b80a01d0 con 0x7f05b80a0a30 2023-10-03T10:18:32.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.359+0000 7f05bde50700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f05b809e2b0 con 0x7f05b809d630 2023-10-03T10:18:32.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.359+0000 7f05b77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f05b80a0a30 0x7f05b80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:32.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.359+0000 7f05b77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f05b80a0a30 0x7f05b80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52360/0 (socket says 172.21.15.3:52360) 2023-10-03T10:18:32.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.359+0000 7f05b6ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f05b809d630 0x7f05b80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:32.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.359+0000 7f05b77fe700 1 -- 172.21.15.3:0/612518278 learned_addr learned my addr 172.21.15.3:0/612518278 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:32.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.359+0000 7f05b6ffd700 1 -- 172.21.15.3:0/612518278 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f05b80a0a30 msgr2=0x7f05b80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:32.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.359+0000 7f05b6ffd700 1 --2- 172.21.15.3:0/612518278 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f05b80a0a30 0x7f05b80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.359+0000 7f05b6ffd700 1 -- 172.21.15.3:0/612518278 >> v1:172.21.15.3:6789/0 conn(0x7f05b8133930 legacy=0x7f05b8133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:32.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.359+0000 7f05b6ffd700 1 -- 172.21.15.3:0/612518278 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b809dc50 con 0x7f05b809d630 2023-10-03T10:18:32.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.360+0000 7f05b77fe700 1 --2- 172.21.15.3:0/612518278 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f05b80a0a30 0x7f05b80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:32.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.360+0000 7f05b6ffd700 1 --2- 172.21.15.3:0/612518278 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f05b809d630 0x7f05b80a1340 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f05ac009870 tx=0x7f05ac009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70eb072e689e9b03 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:32.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.360+0000 7f2e2a534700 1 Processor -- start 2023-10-03T10:18:32.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.360+0000 7f05b67fc700 1 -- 172.21.15.3:0/612518278 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac011830 con 0x7f05b809d630 2023-10-03T10:18:32.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.360+0000 7f05b67fc700 1 -- 172.21.15.3:0/612518278 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ac011990 con 0x7f05b809d630 2023-10-03T10:18:32.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.360+0000 7f05b67fc700 1 -- 172.21.15.3:0/612518278 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac011d20 con 0x7f05b809d630 2023-10-03T10:18:32.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.360+0000 7f2e2a534700 1 -- start start 2023-10-03T10:18:32.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.361+0000 7f2e2a534700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e240a0a30 0x7f2e240a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:32.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.361+0000 7f2e2a534700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2e240a1340 0x7f2e24133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:32.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.361+0000 7f2e2a534700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e24133fa0 0x7f2e24134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:32.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.361+0000 7f2e2a534700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e2409edd0 con 0x7f2e240a1340 2023-10-03T10:18:32.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.361+0000 7f2e2a534700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2409e1f0 con 0x7f2e240a0a30 2023-10-03T10:18:32.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.361+0000 7f2e2a534700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2409dc50 con 0x7f2e24133fa0 2023-10-03T10:18:32.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.361+0000 7f05bde50700 1 -- 172.21.15.3:0/612518278 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f05b809d630 msgr2=0x7f05b80a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:32.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.361+0000 7f05bde50700 1 --2- 172.21.15.3:0/612518278 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f05b809d630 0x7f05b80a1340 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f05ac009870 tx=0x7f05ac009b80 comp rx=0 tx=0).stop 2023-10-03T10:18:32.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.361+0000 7f2e23fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e240a0a30 0x7f2e240a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:32.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.361+0000 7f2e23fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e240a0a30 0x7f2e240a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52374/0 (socket says 172.21.15.3:52374) 2023-10-03T10:18:32.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.361+0000 7f2e23fff700 1 -- 172.21.15.3:0/685834558 learned_addr learned my addr 172.21.15.3:0/685834558 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:32.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.361+0000 7f2e237fe700 1 --2- 172.21.15.3:0/685834558 >> v2:172.21.15.3:3300/0 conn(0x7f2e240a1340 0x7f2e24133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:32.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.361+0000 7f2e28ad1700 1 --2- 172.21.15.3:0/685834558 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e24133fa0 0x7f2e24134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:32.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.361+0000 7f05bde50700 1 -- 172.21.15.3:0/612518278 shutdown_connections 2023-10-03T10:18:32.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.361+0000 7f05bde50700 1 --2- 172.21.15.3:0/612518278 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f05b80a0a30 0x7f05b80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.361+0000 7f05bde50700 1 --2- 172.21.15.3:0/612518278 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f05b809d630 0x7f05b80a1340 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.361+0000 7f05bde50700 1 -- 172.21.15.3:0/612518278 >> 172.21.15.3:0/612518278 conn(0x7f05b80a3050 msgr2=0x7f05b80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:32.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.362+0000 7f2e23fff700 1 -- 172.21.15.3:0/685834558 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e24133fa0 msgr2=0x7f2e24134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:32.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.362+0000 7f2e23fff700 1 --2- 172.21.15.3:0/685834558 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e24133fa0 0x7f2e24134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.362+0000 7f2e23fff700 1 -- 172.21.15.3:0/685834558 >> v2:172.21.15.3:3300/0 conn(0x7f2e240a1340 msgr2=0x7f2e24133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:32.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.362+0000 7f2e23fff700 1 --2- 172.21.15.3:0/685834558 >> v2:172.21.15.3:3300/0 conn(0x7f2e240a1340 0x7f2e24133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.362+0000 7f2e23fff700 1 -- 172.21.15.3:0/685834558 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e240a1fe0 con 0x7f2e240a0a30 2023-10-03T10:18:32.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.362+0000 7f2e237fe700 1 --2- 172.21.15.3:0/685834558 >> v2:172.21.15.3:3300/0 conn(0x7f2e240a1340 0x7f2e24133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:32.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.362+0000 7f05bde50700 1 -- 172.21.15.3:0/612518278 shutdown_connections 2023-10-03T10:18:32.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.362+0000 7f05bde50700 1 -- 172.21.15.3:0/612518278 wait complete. 2023-10-03T10:18:32.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.362+0000 7f2e23fff700 1 --2- 172.21.15.3:0/685834558 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e240a0a30 0x7f2e240a0e00 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009920 tx=0x7f2e14009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=317b46ca62dccc17 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:32.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.362+0000 7f2e22ffd700 1 -- 172.21.15.3:0/685834558 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e14011830 con 0x7f2e240a0a30 2023-10-03T10:18:32.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.362+0000 7f2e22ffd700 1 -- 172.21.15.3:0/685834558 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e14011990 con 0x7f2e240a0a30 2023-10-03T10:18:32.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.362+0000 7f2e22ffd700 1 -- 172.21.15.3:0/685834558 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e14011d20 con 0x7f2e240a0a30 2023-10-03T10:18:32.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.363+0000 7f05bde50700 1 Processor -- start 2023-10-03T10:18:32.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.363+0000 7f2e2a534700 1 -- 172.21.15.3:0/685834558 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e240a0a30 msgr2=0x7f2e240a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:32.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.363+0000 7f2e2a534700 1 --2- 172.21.15.3:0/685834558 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e240a0a30 0x7f2e240a0e00 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009920 tx=0x7f2e14009c30 comp rx=0 tx=0).stop 2023-10-03T10:18:32.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.363+0000 7f05bde50700 1 -- start start 2023-10-03T10:18:32.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.364+0000 7f05bde50700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05b809d630 0x7f05b811f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:32.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.364+0000 7f05bde50700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f05b80a0a30 0x7f05b811fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:32.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.364+0000 7f05bde50700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f05b8133930 0x7f05b81c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:32.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.364+0000 7f05bde50700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b8122340 con 0x7f05b809d630 2023-10-03T10:18:32.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.364+0000 7f05bde50700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b81333b0 con 0x7f05b8133930 2023-10-03T10:18:32.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.364+0000 7f05bde50700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f05b8122540 con 0x7f05b80a0a30 2023-10-03T10:18:32.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.364+0000 7f05b7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f05b8133930 0x7f05b81c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:32.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.364+0000 7f05b6ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f05b80a0a30 0x7f05b811fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:32.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.364+0000 7f05b7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f05b8133930 0x7f05b81c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52378/0 (socket says 172.21.15.3:52378) 2023-10-03T10:18:32.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.364+0000 7f05b7fff700 1 -- 172.21.15.3:0/2095230536 learned_addr learned my addr 172.21.15.3:0/2095230536 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:32.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.364+0000 7f05b77fe700 1 --2- 172.21.15.3:0/2095230536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05b809d630 0x7f05b811f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:32.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.364+0000 7f2e2a534700 1 -- 172.21.15.3:0/685834558 shutdown_connections 2023-10-03T10:18:32.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.364+0000 7f2e2a534700 1 --2- 172.21.15.3:0/685834558 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e24133fa0 0x7f2e24134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.364+0000 7f2e2a534700 1 --2- 172.21.15.3:0/685834558 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e240a0a30 0x7f2e240a0e00 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.364+0000 7f2e2a534700 1 --2- 172.21.15.3:0/685834558 >> v2:172.21.15.3:3300/0 conn(0x7f2e240a1340 0x7f2e24133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.364+0000 7f2e2a534700 1 -- 172.21.15.3:0/685834558 >> 172.21.15.3:0/685834558 conn(0x7f2e240a6550 msgr2=0x7f2e240a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:32.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.364+0000 7f05b6ffd700 1 -- 172.21.15.3:0/2095230536 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f05b8133930 msgr2=0x7f05b81c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:32.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.364+0000 7f05b6ffd700 1 --2- 172.21.15.3:0/2095230536 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f05b8133930 0x7f05b81c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.364+0000 7f05b6ffd700 1 -- 172.21.15.3:0/2095230536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05b809d630 msgr2=0x7f05b811f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:32.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.364+0000 7f05b6ffd700 1 --2- 172.21.15.3:0/2095230536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05b809d630 0x7f05b811f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.364+0000 7f05b6ffd700 1 -- 172.21.15.3:0/2095230536 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a8009550 con 0x7f05b80a0a30 2023-10-03T10:18:32.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.364+0000 7f2e2a534700 1 -- 172.21.15.3:0/685834558 shutdown_connections 2023-10-03T10:18:32.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.364+0000 7f2e2a534700 1 -- 172.21.15.3:0/685834558 wait complete. 2023-10-03T10:18:32.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.365+0000 7f05b6ffd700 1 --2- 172.21.15.3:0/2095230536 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f05b80a0a30 0x7f05b811fa60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f05b80a1f80 tx=0x7f05ac004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:32.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.365+0000 7f059ffff700 1 -- 172.21.15.3:0/2095230536 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac004240 con 0x7f05b80a0a30 2023-10-03T10:18:32.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.365+0000 7f059ffff700 1 -- 172.21.15.3:0/2095230536 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ac01f070 con 0x7f05b80a0a30 2023-10-03T10:18:32.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.365+0000 7f05bde50700 1 -- 172.21.15.3:0/2095230536 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ac0094a0 con 0x7f05b80a0a30 2023-10-03T10:18:32.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.365+0000 7f05bde50700 1 -- 172.21.15.3:0/2095230536 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b811ef30 con 0x7f05b80a0a30 2023-10-03T10:18:32.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.366+0000 7f059ffff700 1 -- 172.21.15.3:0/2095230536 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac0045e0 con 0x7f05b80a0a30 2023-10-03T10:18:32.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.366+0000 7f059ffff700 1 -- 172.21.15.3:0/2095230536 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f05ac021030 con 0x7f05b80a0a30 2023-10-03T10:18:32.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.366+0000 7f059ffff700 1 -- 172.21.15.3:0/2095230536 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f05ac0407d0 con 0x7f05b80a0a30 2023-10-03T10:18:32.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.366+0000 7f2e2a534700 1 Processor -- start 2023-10-03T10:18:32.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.366+0000 7f2e2a534700 1 -- start start 2023-10-03T10:18:32.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.367+0000 7f2e2a534700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e240a0a30 0x7f2e2412f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:32.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.367+0000 7f2e2a534700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e240a1340 0x7f2e2412fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:32.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.367+0000 7f2e2a534700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e24133fa0 0x7f2e241302b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:32.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.367+0000 7f2e2a534700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2407d890 con 0x7f2e240a1340 2023-10-03T10:18:32.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.367+0000 7f2e2a534700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2407da70 con 0x7f2e24133fa0 2023-10-03T10:18:32.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.367+0000 7f2e2a534700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2408aad0 con 0x7f2e240a0a30 2023-10-03T10:18:32.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.367+0000 7f2e23fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e240a0a30 0x7f2e2412f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:32.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.367+0000 7f2e23fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e240a0a30 0x7f2e2412f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52468/0 (socket says 172.21.15.3:52468) 2023-10-03T10:18:32.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.367+0000 7f2e23fff700 1 -- 172.21.15.3:0/2941258693 learned_addr learned my addr 172.21.15.3:0/2941258693 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:32.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.367+0000 7f2e28ad1700 1 --2- 172.21.15.3:0/2941258693 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e24133fa0 0x7f2e241302b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:32.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.367+0000 7f2e237fe700 1 --2- 172.21.15.3:0/2941258693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e240a1340 0x7f2e2412fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:32.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.368+0000 7f2e23fff700 1 -- 172.21.15.3:0/2941258693 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e24133fa0 msgr2=0x7f2e241302b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:32.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.368+0000 7f2e23fff700 1 --2- 172.21.15.3:0/2941258693 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e24133fa0 0x7f2e241302b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.368+0000 7f2e23fff700 1 -- 172.21.15.3:0/2941258693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e240a1340 msgr2=0x7f2e2412fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:32.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.368+0000 7f2e23fff700 1 --2- 172.21.15.3:0/2941258693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e240a1340 0x7f2e2412fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.368+0000 7f05bde50700 1 -- 172.21.15.3:0/2095230536 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b80a0410 con 0x7f05b80a0a30 2023-10-03T10:18:32.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.368+0000 7f2e23fff700 1 -- 172.21.15.3:0/2941258693 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e180094a0 con 0x7f2e240a0a30 2023-10-03T10:18:32.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.368+0000 7f2e23fff700 1 --2- 172.21.15.3:0/2941258693 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e240a0a30 0x7f2e2412f830 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2e140098f0 tx=0x7f2e14004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:32.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.373+0000 7f2e20ff9700 1 -- 172.21.15.3:0/2941258693 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e14004300 con 0x7f2e240a0a30 2023-10-03T10:18:32.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.373+0000 7f2e20ff9700 1 -- 172.21.15.3:0/2941258693 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1401f070 con 0x7f2e240a0a30 2023-10-03T10:18:32.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.373+0000 7f2e20ff9700 1 -- 172.21.15.3:0/2941258693 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e140036a0 con 0x7f2e240a0a30 2023-10-03T10:18:32.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.373+0000 7f2e2a534700 1 -- 172.21.15.3:0/2941258693 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e14009550 con 0x7f2e240a0a30 2023-10-03T10:18:32.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.373+0000 7f059ffff700 1 -- 172.21.15.3:0/2095230536 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f05ac0199a0 con 0x7f05b80a0a30 2023-10-03T10:18:32.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.373+0000 7f2e2a534700 1 -- 172.21.15.3:0/2941258693 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e240a0220 con 0x7f2e240a0a30 2023-10-03T10:18:32.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.375+0000 7f2e2a534700 1 -- 172.21.15.3:0/2941258693 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e24074ea0 con 0x7f2e240a0a30 2023-10-03T10:18:32.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.375+0000 7f2e20ff9700 1 -- 172.21.15.3:0/2941258693 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2e14021030 con 0x7f2e240a0a30 2023-10-03T10:18:32.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.376+0000 7f2e20ff9700 1 -- 172.21.15.3:0/2941258693 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2e14040f30 con 0x7f2e240a0a30 2023-10-03T10:18:32.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.380+0000 7f2e20ff9700 1 -- 172.21.15.3:0/2941258693 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2e140239c0 con 0x7f2e240a0a30 2023-10-03T10:18:32.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.400+0000 7f76aa097700 1 Processor -- start 2023-10-03T10:18:32.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.400+0000 7f76aa097700 1 -- start start 2023-10-03T10:18:32.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.401+0000 7f76aa097700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76a40a0a30 0x7f76a40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:32.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.401+0000 7f76aa097700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76a4138d80 0x7f76a413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:32.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.401+0000 7f76aa097700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a40a2a70 con 0x7f76a409d630 2023-10-03T10:18:32.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.401+0000 7f76aa097700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a40a01d0 con 0x7f76a4138d80 2023-10-03T10:18:32.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.401+0000 7f76aa097700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76a409e2b0 con 0x7f76a40a0a30 2023-10-03T10:18:32.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.401+0000 7f76a2ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f76a409d630 0x7f76a40a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47198/0 (socket says 172.21.15.3:47198) 2023-10-03T10:18:32.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.401+0000 7f76a2ffd700 1 -- 172.21.15.3:0/954822643 learned_addr learned my addr 172.21.15.3:0/954822643 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:32.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.401+0000 7f76a37fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76a40a0a30 0x7f76a40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:32.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.401+0000 7f76a3fff700 1 --2- 172.21.15.3:0/954822643 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76a4138d80 0x7f76a413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:32.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.402+0000 7f76a27fc700 1 -- 172.21.15.3:0/954822643 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1798002115 0 0) 0x7f76a40a2a70 con 0x7f76a409d630 2023-10-03T10:18:32.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.402+0000 7f76a27fc700 1 -- 172.21.15.3:0/954822643 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f768c0035b0 con 0x7f76a409d630 2023-10-03T10:18:32.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.402+0000 7f76a3fff700 1 -- 172.21.15.3:0/954822643 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76a40a0a30 msgr2=0x7f76a40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:32.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.402+0000 7f76a3fff700 1 --2- 172.21.15.3:0/954822643 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76a40a0a30 0x7f76a40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.402+0000 7f76a3fff700 1 -- 172.21.15.3:0/954822643 >> v1:172.21.15.3:6789/0 conn(0x7f76a409d630 legacy=0x7f76a40a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:32.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.402+0000 7f76a3fff700 1 -- 172.21.15.3:0/954822643 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a409dc50 con 0x7f76a4138d80 2023-10-03T10:18:32.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.402+0000 7f76a3fff700 1 --2- 172.21.15.3:0/954822643 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76a4138d80 0x7f76a413b160 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f769c009dc0 tx=0x7f769c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aac0c2a9ff79d92 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:32.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.402+0000 7f76a27fc700 1 -- 172.21.15.3:0/954822643 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c00e070 con 0x7f76a4138d80 2023-10-03T10:18:32.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.402+0000 7f76a27fc700 1 -- 172.21.15.3:0/954822643 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769c00bb20 con 0x7f76a4138d80 2023-10-03T10:18:32.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.402+0000 7f76a27fc700 1 -- 172.21.15.3:0/954822643 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c00be90 con 0x7f76a4138d80 2023-10-03T10:18:32.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.403+0000 7f76aa097700 1 -- 172.21.15.3:0/954822643 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76a4138d80 msgr2=0x7f76a413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:32.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.403+0000 7f76aa097700 1 --2- 172.21.15.3:0/954822643 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76a4138d80 0x7f76a413b160 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f769c009dc0 tx=0x7f769c00b330 comp rx=0 tx=0).stop 2023-10-03T10:18:32.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.403+0000 7f76aa097700 1 -- 172.21.15.3:0/954822643 shutdown_connections 2023-10-03T10:18:32.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.403+0000 7f76aa097700 1 --2- 172.21.15.3:0/954822643 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76a4138d80 0x7f76a413b160 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.403+0000 7f76aa097700 1 --2- 172.21.15.3:0/954822643 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76a40a0a30 0x7f76a40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.403+0000 7f76aa097700 1 -- 172.21.15.3:0/954822643 >> 172.21.15.3:0/954822643 conn(0x7f76a40a3050 msgr2=0x7f76a40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:32.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.403+0000 7f76aa097700 1 -- 172.21.15.3:0/954822643 shutdown_connections 2023-10-03T10:18:32.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.403+0000 7f76aa097700 1 -- 172.21.15.3:0/954822643 wait complete. 2023-10-03T10:18:32.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.404+0000 7f76aa097700 1 Processor -- start 2023-10-03T10:18:32.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.404+0000 7f76aa097700 1 -- start start 2023-10-03T10:18:32.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.405+0000 7f76aa097700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76a409d630 0x7f76a411c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:32.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.405+0000 7f76aa097700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76a40a0a30 0x7f76a411cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:32.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.405+0000 7f76aa097700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76a4138d80 0x7f76a411d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:32.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.405+0000 7f76aa097700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a411feb0 con 0x7f76a409d630 2023-10-03T10:18:32.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.405+0000 7f76aa097700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a409d320 con 0x7f76a40a0a30 2023-10-03T10:18:32.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.405+0000 7f76aa097700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76a40a0410 con 0x7f76a4138d80 2023-10-03T10:18:32.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.405+0000 7f76a37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76a409d630 0x7f76a411c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:32.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.405+0000 7f76a3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76a4138d80 0x7f76a411d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:32.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.405+0000 7f76a37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76a409d630 0x7f76a411c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58810/0 (socket says 172.21.15.3:58810) 2023-10-03T10:18:32.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.405+0000 7f76a2ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76a40a0a30 0x7f76a411cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:32.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.405+0000 7f76a37fe700 1 -- 172.21.15.3:0/1866713661 learned_addr learned my addr 172.21.15.3:0/1866713661 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:32.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.406+0000 7f76a2ffd700 1 -- 172.21.15.3:0/1866713661 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76a4138d80 msgr2=0x7f76a411d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:32.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.406+0000 7f76a2ffd700 1 --2- 172.21.15.3:0/1866713661 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76a4138d80 0x7f76a411d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.406+0000 7f76a2ffd700 1 -- 172.21.15.3:0/1866713661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76a409d630 msgr2=0x7f76a411c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.406+0000 7f76a2ffd700 1 --2- 172.21.15.3:0/1866713661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76a409d630 0x7f76a411c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.406+0000 7f76a2ffd700 1 -- 172.21.15.3:0/1866713661 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f769c009a70 con 0x7f76a40a0a30 2023-10-03T10:18:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.406+0000 7f76a3fff700 1 --2- 172.21.15.3:0/1866713661 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76a4138d80 0x7f76a411d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.406+0000 7f76a2ffd700 1 --2- 172.21.15.3:0/1866713661 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76a40a0a30 0x7f76a411cd00 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f769800c460 tx=0x7f769800c770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.406+0000 7f768bfff700 1 -- 172.21.15.3:0/1866713661 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7698005d70 con 0x7f76a40a0a30 2023-10-03T10:18:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.406+0000 7f768bfff700 1 -- 172.21.15.3:0/1866713661 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76980070d0 con 0x7f76a40a0a30 2023-10-03T10:18:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.406+0000 7f76aa097700 1 -- 172.21.15.3:0/1866713661 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a411f910 con 0x7f76a40a0a30 2023-10-03T10:18:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.406+0000 7f768bfff700 1 -- 172.21.15.3:0/1866713661 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7698007440 con 0x7f76a40a0a30 2023-10-03T10:18:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.406+0000 7f76aa097700 1 -- 172.21.15.3:0/1866713661 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a411e500 con 0x7f76a40a0a30 2023-10-03T10:18:32.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.408+0000 7f76aa097700 1 -- 172.21.15.3:0/1866713661 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a413b620 con 0x7f76a40a0a30 2023-10-03T10:18:32.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.408+0000 7f768bfff700 1 -- 172.21.15.3:0/1866713661 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f76980075a0 con 0x7f76a40a0a30 2023-10-03T10:18:32.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.408+0000 7f768bfff700 1 -- 172.21.15.3:0/1866713661 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f769803ebf0 con 0x7f76a40a0a30 2023-10-03T10:18:32.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.416+0000 7f768bfff700 1 -- 172.21.15.3:0/1866713661 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f769801bb10 con 0x7f76a40a0a30 2023-10-03T10:18:32.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.472+0000 7fa6a2506700 1 Processor -- start 2023-10-03T10:18:32.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.472+0000 7fa6a2506700 1 -- start start 2023-10-03T10:18:32.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.473+0000 7fa6a2506700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa69c09d6f0 0x7fa69c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:32.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.473+0000 7fa6a2506700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa69c09e000 0x7fa69c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:32.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.473+0000 7fa6a2506700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa69c09e410 0x7fa69c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:32.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.473+0000 7fa6a2506700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa69c09f520 con 0x7fa69c09e000 2023-10-03T10:18:32.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.473+0000 7fa6a2506700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa69c09f130 con 0x7fa69c09d6f0 2023-10-03T10:18:32.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.473+0000 7fa6a2506700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa69c09e930 con 0x7fa69c09e410 2023-10-03T10:18:32.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.473+0000 7fa69bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa69c09d6f0 0x7fa69c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:32.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.474+0000 7fa69bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa69c09d6f0 0x7fa69c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52434/0 (socket says 172.21.15.3:52434) 2023-10-03T10:18:32.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.474+0000 7fa69bfff700 1 -- 172.21.15.3:0/2149925493 learned_addr learned my addr 172.21.15.3:0/2149925493 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:32.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.474+0000 7fa69b7fe700 1 --2- 172.21.15.3:0/2149925493 >> v2:172.21.15.3:3300/0 conn(0x7fa69c09e000 0x7fa69c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:32.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.474+0000 7fa6a0aa3700 1 --2- 172.21.15.3:0/2149925493 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa69c09e410 0x7fa69c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:32.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.474+0000 7fa69b7fe700 1 -- 172.21.15.3:0/2149925493 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa69c09e410 msgr2=0x7fa69c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:32.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.474+0000 7fa69b7fe700 1 --2- 172.21.15.3:0/2149925493 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa69c09e410 0x7fa69c130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.474+0000 7fa69b7fe700 1 -- 172.21.15.3:0/2149925493 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa69c09d6f0 msgr2=0x7fa69c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:32.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.474+0000 7fa69b7fe700 1 --2- 172.21.15.3:0/2149925493 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa69c09d6f0 0x7fa69c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.474+0000 7fa69b7fe700 1 -- 172.21.15.3:0/2149925493 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69c09b2d0 con 0x7fa69c09e000 2023-10-03T10:18:32.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.474+0000 7fa69b7fe700 1 --2- 172.21.15.3:0/2149925493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa69c09e000 0x7fa69c130660 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fa69000bb10 tx=0x7fa69000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd6698863ff1f01 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:32.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.474+0000 7fa69affd700 1 -- 172.21.15.3:0/2149925493 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69000dc30 con 0x7fa69c09e000 2023-10-03T10:18:32.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.475+0000 7fa69affd700 1 -- 172.21.15.3:0/2149925493 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69000dd90 con 0x7fa69c09e000 2023-10-03T10:18:32.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.475+0000 7fa69affd700 1 -- 172.21.15.3:0/2149925493 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa690012990 con 0x7fa69c09e000 2023-10-03T10:18:32.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.475+0000 7fa6a2506700 1 -- 172.21.15.3:0/2149925493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa69c09e000 msgr2=0x7fa69c130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:32.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.475+0000 7fa6a2506700 1 --2- 172.21.15.3:0/2149925493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa69c09e000 0x7fa69c130660 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fa69000bb10 tx=0x7fa69000be20 comp rx=0 tx=0).stop 2023-10-03T10:18:32.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.475+0000 7fa6a2506700 1 -- 172.21.15.3:0/2149925493 shutdown_connections 2023-10-03T10:18:32.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.475+0000 7fa6a2506700 1 --2- 172.21.15.3:0/2149925493 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa69c09e410 0x7fa69c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.475+0000 7fa6a2506700 1 --2- 172.21.15.3:0/2149925493 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa69c09d6f0 0x7fa69c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.475+0000 7fa6a2506700 1 --2- 172.21.15.3:0/2149925493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa69c09e000 0x7fa69c130660 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.475+0000 7fa6a2506700 1 -- 172.21.15.3:0/2149925493 >> 172.21.15.3:0/2149925493 conn(0x7fa69c066f50 msgr2=0x7fa69c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:32.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.475+0000 7fa6a2506700 1 -- 172.21.15.3:0/2149925493 shutdown_connections 2023-10-03T10:18:32.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.475+0000 7fa6a2506700 1 -- 172.21.15.3:0/2149925493 wait complete. 2023-10-03T10:18:32.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.477+0000 7fa6a2506700 1 Processor -- start 2023-10-03T10:18:32.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.477+0000 7fa6a2506700 1 -- start start 2023-10-03T10:18:32.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.478+0000 7fa6a2506700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa69c09d6f0 0x7fa69c11e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:32.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.478+0000 7fa6a2506700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa69c09e000 0x7fa69c11e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:32.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.478+0000 7fa6a2506700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa69c09e410 0x7fa69c11eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:32.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.478+0000 7fa6a2506700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69c121bb0 con 0x7fa69c09d6f0 2023-10-03T10:18:32.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.478+0000 7fa6a2506700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa69c0a1af0 con 0x7fa69c09e410 2023-10-03T10:18:32.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.478+0000 7fa6a2506700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa69c09c700 con 0x7fa69c09e000 2023-10-03T10:18:32.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.478+0000 7fa69b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa69c09e000 0x7fa69c11e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:32.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.478+0000 7fa69b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa69c09e000 0x7fa69c11e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:52490/0 (socket says 172.21.15.3:52490) 2023-10-03T10:18:32.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.478+0000 7fa69b7fe700 1 -- 172.21.15.3:0/3316452195 learned_addr learned my addr 172.21.15.3:0/3316452195 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:32.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.478+0000 7fa6a0aa3700 1 --2- 172.21.15.3:0/3316452195 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa69c09e410 0x7fa69c11eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:32.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.478+0000 7fa69bfff700 1 --2- 172.21.15.3:0/3316452195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa69c09d6f0 0x7fa69c11e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:32.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.478+0000 7fa69b7fe700 1 -- 172.21.15.3:0/3316452195 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa69c09e410 msgr2=0x7fa69c11eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:32.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.478+0000 7fa69b7fe700 1 --2- 172.21.15.3:0/3316452195 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa69c09e410 0x7fa69c11eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.478+0000 7fa69b7fe700 1 -- 172.21.15.3:0/3316452195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa69c09d6f0 msgr2=0x7fa69c11e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:32.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.479+0000 7fa69b7fe700 1 --2- 172.21.15.3:0/3316452195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa69c09d6f0 0x7fa69c11e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.479+0000 7fa69b7fe700 1 -- 172.21.15.3:0/3316452195 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69000b740 con 0x7fa69c09e000 2023-10-03T10:18:32.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.479+0000 7fa69b7fe700 1 --2- 172.21.15.3:0/3316452195 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa69c09e000 0x7fa69c11e5d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fa690008010 tx=0x7fa690004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:32.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.479+0000 7fa698ff9700 1 -- 172.21.15.3:0/3316452195 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa690020070 con 0x7fa69c09e000 2023-10-03T10:18:32.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.479+0000 7fa698ff9700 1 -- 172.21.15.3:0/3316452195 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa690012e80 con 0x7fa69c09e000 2023-10-03T10:18:32.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.479+0000 7fa6a2506700 1 -- 172.21.15.3:0/3316452195 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa69c121610 con 0x7fa69c09e000 2023-10-03T10:18:32.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.479+0000 7fa698ff9700 1 -- 172.21.15.3:0/3316452195 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa690004440 con 0x7fa69c09e000 2023-10-03T10:18:32.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.479+0000 7fa6a2506700 1 -- 172.21.15.3:0/3316452195 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa69c1201f0 con 0x7fa69c09e000 2023-10-03T10:18:32.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.480+0000 7fa698ff9700 1 -- 172.21.15.3:0/3316452195 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa69000b040 con 0x7fa69c09e000 2023-10-03T10:18:32.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.481+0000 7fa6a2506700 1 -- 172.21.15.3:0/3316452195 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa69c074ea0 con 0x7fa69c09e000 2023-10-03T10:18:32.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.481+0000 7fa698ff9700 1 -- 172.21.15.3:0/3316452195 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa6900445c0 con 0x7fa69c09e000 2023-10-03T10:18:32.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.490+0000 7fa698ff9700 1 -- 172.21.15.3:0/3316452195 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa690022720 con 0x7fa69c09e000 2023-10-03T10:18:32.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.608+0000 7f2e2a534700 1 -- 172.21.15.3:0/2941258693 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2e240a28e0 con 0x7f2e240a0a30 2023-10-03T10:18:32.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.608+0000 7f2e20ff9700 1 -- 172.21.15.3:0/2941258693 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2e14015070 con 0x7f2e240a0a30 2023-10-03T10:18:32.609 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:32.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.612+0000 7f2e2a534700 1 -- 172.21.15.3:0/2941258693 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e240a0a30 msgr2=0x7f2e2412f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:32.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.612+0000 7f2e2a534700 1 --2- 172.21.15.3:0/2941258693 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e240a0a30 0x7f2e2412f830 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2e140098f0 tx=0x7f2e14004750 comp rx=0 tx=0).stop 2023-10-03T10:18:32.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.612+0000 7f2e2a534700 1 -- 172.21.15.3:0/2941258693 shutdown_connections 2023-10-03T10:18:32.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.612+0000 7f2e2a534700 1 --2- 172.21.15.3:0/2941258693 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e24133fa0 0x7f2e241302b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.612+0000 7f2e2a534700 1 --2- 172.21.15.3:0/2941258693 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2e240a0a30 0x7f2e2412f830 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.612+0000 7f2e2a534700 1 --2- 172.21.15.3:0/2941258693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e240a1340 0x7f2e2412fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.612+0000 7f2e2a534700 1 -- 172.21.15.3:0/2941258693 >> 172.21.15.3:0/2941258693 conn(0x7f2e240a6550 msgr2=0x7f2e2409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:32.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.612+0000 7f2e2a534700 1 -- 172.21.15.3:0/2941258693 shutdown_connections 2023-10-03T10:18:32.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.612+0000 7f2e2a534700 1 -- 172.21.15.3:0/2941258693 wait complete. 2023-10-03T10:18:32.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.614+0000 7f05bde50700 1 -- 172.21.15.3:0/2095230536 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f05b80747f0 con 0x7f05b80a0a30 2023-10-03T10:18:32.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.615+0000 7f059ffff700 1 -- 172.21.15.3:0/2095230536 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f05ac015020 con 0x7f05b80a0a30 2023-10-03T10:18:32.615 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:32.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.617+0000 7f05bde50700 1 -- 172.21.15.3:0/2095230536 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f05b80a0a30 msgr2=0x7f05b811fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:32.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.617+0000 7f05bde50700 1 --2- 172.21.15.3:0/2095230536 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f05b80a0a30 0x7f05b811fa60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f05b80a1f80 tx=0x7f05ac004750 comp rx=0 tx=0).stop 2023-10-03T10:18:32.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.619+0000 7f05bde50700 1 -- 172.21.15.3:0/2095230536 shutdown_connections 2023-10-03T10:18:32.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.619+0000 7f05bde50700 1 --2- 172.21.15.3:0/2095230536 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f05b8133930 0x7f05b81c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.619+0000 7f05bde50700 1 --2- 172.21.15.3:0/2095230536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05b809d630 0x7f05b811f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.619+0000 7f05bde50700 1 --2- 172.21.15.3:0/2095230536 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f05b80a0a30 0x7f05b811fa60 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.619+0000 7f05bde50700 1 -- 172.21.15.3:0/2095230536 >> 172.21.15.3:0/2095230536 conn(0x7f05b80a3050 msgr2=0x7f05b81bbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:32.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.619+0000 7f05bde50700 1 -- 172.21.15.3:0/2095230536 shutdown_connections 2023-10-03T10:18:32.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.619+0000 7f05bde50700 1 -- 172.21.15.3:0/2095230536 wait complete. 2023-10-03T10:18:32.634 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:32.634 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:32.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.682+0000 7f76aa097700 1 -- 172.21.15.3:0/1866713661 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f76a40747f0 con 0x7f76a40a0a30 2023-10-03T10:18:32.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.682+0000 7f768bfff700 1 -- 172.21.15.3:0/1866713661 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f76980192c0 con 0x7f76a40a0a30 2023-10-03T10:18:32.683 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:32.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.684+0000 7f76aa097700 1 -- 172.21.15.3:0/1866713661 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76a40a0a30 msgr2=0x7f76a411cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:32.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.684+0000 7f76aa097700 1 --2- 172.21.15.3:0/1866713661 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76a40a0a30 0x7f76a411cd00 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f769800c460 tx=0x7f769800c770 comp rx=0 tx=0).stop 2023-10-03T10:18:32.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.684+0000 7f76aa097700 1 -- 172.21.15.3:0/1866713661 shutdown_connections 2023-10-03T10:18:32.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.684+0000 7f76aa097700 1 --2- 172.21.15.3:0/1866713661 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f76a4138d80 0x7f76a411d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.684+0000 7f76aa097700 1 --2- 172.21.15.3:0/1866713661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76a409d630 0x7f76a411c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.684+0000 7f76aa097700 1 --2- 172.21.15.3:0/1866713661 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f76a40a0a30 0x7f76a411cd00 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.684+0000 7f76aa097700 1 -- 172.21.15.3:0/1866713661 >> 172.21.15.3:0/1866713661 conn(0x7f76a40a3050 msgr2=0x7f76a409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:32.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.684+0000 7f76aa097700 1 -- 172.21.15.3:0/1866713661 shutdown_connections 2023-10-03T10:18:32.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.684+0000 7f76aa097700 1 -- 172.21.15.3:0/1866713661 wait complete. 2023-10-03T10:18:32.698 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:32.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.746+0000 7fa6a2506700 1 -- 172.21.15.3:0/3316452195 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa69c120ef0 con 0x7fa69c09e000 2023-10-03T10:18:32.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.747+0000 7fa698ff9700 1 -- 172.21.15.3:0/3316452195 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa690029020 con 0x7fa69c09e000 2023-10-03T10:18:32.747 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:32.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.749+0000 7fa6a2506700 1 -- 172.21.15.3:0/3316452195 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa69c09e000 msgr2=0x7fa69c11e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:32.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.749+0000 7fa6a2506700 1 --2- 172.21.15.3:0/3316452195 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa69c09e000 0x7fa69c11e5d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fa690008010 tx=0x7fa690004750 comp rx=0 tx=0).stop 2023-10-03T10:18:32.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.749+0000 7fa6a2506700 1 -- 172.21.15.3:0/3316452195 shutdown_connections 2023-10-03T10:18:32.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.749+0000 7fa6a2506700 1 --2- 172.21.15.3:0/3316452195 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa69c09e410 0x7fa69c11eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.749+0000 7fa6a2506700 1 --2- 172.21.15.3:0/3316452195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa69c09d6f0 0x7fa69c11e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.749+0000 7fa6a2506700 1 --2- 172.21.15.3:0/3316452195 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa69c09e000 0x7fa69c11e5d0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:32.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.749+0000 7fa6a2506700 1 -- 172.21.15.3:0/3316452195 >> 172.21.15.3:0/3316452195 conn(0x7fa69c066f50 msgr2=0x7fa69c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:32.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.749+0000 7fa6a2506700 1 -- 172.21.15.3:0/3316452195 shutdown_connections 2023-10-03T10:18:32.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:32.749+0000 7fa6a2506700 1 -- 172.21.15.3:0/3316452195 wait complete. 2023-10-03T10:18:32.762 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:33.635 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:33.635 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:33.699 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:33.764 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:33.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.785+0000 7f70a0c49700 1 Processor -- start 2023-10-03T10:18:33.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.785+0000 7f70a0c49700 1 -- start start 2023-10-03T10:18:33.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.786+0000 7f70a0c49700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f709c131810 0x7f709c131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:33.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.786+0000 7f70a0c49700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f709c09a580 0x7f709c139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:33.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.786+0000 7f70a0c49700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f709c1313c0 con 0x7f709c099a40 2023-10-03T10:18:33.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.786+0000 7f70a0c49700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709c06e8c0 con 0x7f709c09a580 2023-10-03T10:18:33.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.786+0000 7f70a0c49700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f709c071d20 con 0x7f709c131810 2023-10-03T10:18:33.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.786+0000 7f7099d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f709c099a40 0x7f709c099e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43598/0 (socket says 172.21.15.3:43598) 2023-10-03T10:18:33.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.786+0000 7f7099d9b700 1 -- 172.21.15.3:0/1497586754 learned_addr learned my addr 172.21.15.3:0/1497586754 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:33.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.786+0000 7f709ad9d700 1 --2- 172.21.15.3:0/1497586754 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f709c09a580 0x7f709c139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:33.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.786+0000 7f709959a700 1 -- 172.21.15.3:0/1497586754 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2086642460 0 0) 0x7f709c1313c0 con 0x7f709c099a40 2023-10-03T10:18:33.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.786+0000 7f709a59c700 1 --2- 172.21.15.3:0/1497586754 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f709c131810 0x7f709c131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:33.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.786+0000 7f709959a700 1 -- 172.21.15.3:0/1497586754 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70840035b0 con 0x7f709c099a40 2023-10-03T10:18:33.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.786+0000 7f709ad9d700 1 -- 172.21.15.3:0/1497586754 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f709c131810 msgr2=0x7f709c131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:33.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.786+0000 7f709ad9d700 1 --2- 172.21.15.3:0/1497586754 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f709c131810 0x7f709c131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:33.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.786+0000 7f709ad9d700 1 -- 172.21.15.3:0/1497586754 >> v1:172.21.15.3:6789/0 conn(0x7f709c099a40 legacy=0x7f709c099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:33.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.786+0000 7f709959a700 1 -- 172.21.15.3:0/1497586754 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2511984448 0 0) 0x7f70840035b0 con 0x7f709c099a40 2023-10-03T10:18:33.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.786+0000 7f709ad9d700 1 -- 172.21.15.3:0/1497586754 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709c0721a0 con 0x7f709c09a580 2023-10-03T10:18:33.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.787+0000 7f709ad9d700 1 --2- 172.21.15.3:0/1497586754 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f709c09a580 0x7f709c139660 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f7094009dc0 tx=0x7f709400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec6c7756586f8727 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.787+0000 7f709959a700 1 -- 172.21.15.3:0/1497586754 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709400e070 con 0x7f709c09a580 2023-10-03T10:18:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.787+0000 7f709959a700 1 -- 172.21.15.3:0/1497586754 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7094003a20 con 0x7f709c09a580 2023-10-03T10:18:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.787+0000 7f709959a700 1 -- 172.21.15.3:0/1497586754 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090001670 con 0x7f709c09a580 2023-10-03T10:18:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.788+0000 7fcfde12a700 1 Processor -- start 2023-10-03T10:18:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.788+0000 7fcfde12a700 1 -- start start 2023-10-03T10:18:33.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.788+0000 7f70a0c49700 1 -- 172.21.15.3:0/1497586754 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f709c09a580 msgr2=0x7f709c139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:33.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.788+0000 7f70a0c49700 1 --2- 172.21.15.3:0/1497586754 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f709c09a580 0x7f709c139660 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f7094009dc0 tx=0x7f709400b330 comp rx=0 tx=0).stop 2023-10-03T10:18:33.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.788+0000 7f70a0c49700 1 -- 172.21.15.3:0/1497586754 shutdown_connections 2023-10-03T10:18:33.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.788+0000 7f70a0c49700 1 --2- 172.21.15.3:0/1497586754 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f709c09a580 0x7f709c139660 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:33.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.788+0000 7f70a0c49700 1 --2- 172.21.15.3:0/1497586754 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f709c131810 0x7f709c131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:33.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.788+0000 7f70a0c49700 1 -- 172.21.15.3:0/1497586754 >> 172.21.15.3:0/1497586754 conn(0x7f709c0a2aa0 msgr2=0x7f709c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:33.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.789+0000 7fcfde12a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcfd8130b30 0x7fcfd8138890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:33.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.789+0000 7fcfde12a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd8138dd0 0x7fcfd813b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:33.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.789+0000 7fcfde12a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfd8099ef0 con 0x7fcfd8071b40 2023-10-03T10:18:33.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.789+0000 7fcfde12a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd8099500 con 0x7fcfd8130b30 2023-10-03T10:18:33.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.789+0000 7fcfde12a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd8130f50 con 0x7fcfd8138dd0 2023-10-03T10:18:33.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.789+0000 7fcfd7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd8138dd0 0x7fcfd813b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:33.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.789+0000 7f70a0c49700 1 -- 172.21.15.3:0/1497586754 shutdown_connections 2023-10-03T10:18:33.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.789+0000 7fcfd7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd8138dd0 0x7fcfd813b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50132/0 (socket says 172.21.15.3:50132) 2023-10-03T10:18:33.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.789+0000 7fcfd7fff700 1 -- 172.21.15.3:0/1941632151 learned_addr learned my addr 172.21.15.3:0/1941632151 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:33.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.789+0000 7fcfd6ffd700 1 --2- 172.21.15.3:0/1941632151 >> v2:172.21.15.3:3300/0 conn(0x7fcfd8130b30 0x7fcfd8138890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:33.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.789+0000 7fcfd7fff700 1 -- 172.21.15.3:0/1941632151 >> v2:172.21.15.3:3300/0 conn(0x7fcfd8130b30 msgr2=0x7fcfd8138890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:33.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.789+0000 7fcfd7fff700 1 --2- 172.21.15.3:0/1941632151 >> v2:172.21.15.3:3300/0 conn(0x7fcfd8130b30 0x7fcfd8138890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:33.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.789+0000 7fcfd7fff700 1 -- 172.21.15.3:0/1941632151 >> v1:172.21.15.3:6789/0 conn(0x7fcfd8071b40 legacy=0x7fcfd8130420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:33.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.789+0000 7fcfd7fff700 1 -- 172.21.15.3:0/1941632151 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd8099c10 con 0x7fcfd8138dd0 2023-10-03T10:18:33.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.790+0000 7fcfd7fff700 1 --2- 172.21.15.3:0/1941632151 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd8138dd0 0x7fcfd813b1b0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009e50 tx=0x7fcfd000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc53b02bc35b45ae server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:33.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.790+0000 7fcfd67fc700 1 -- 172.21.15.3:0/1941632151 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd000e070 con 0x7fcfd8138dd0 2023-10-03T10:18:33.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.790+0000 7fcfd67fc700 1 -- 172.21.15.3:0/1941632151 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd0003a20 con 0x7fcfd8138dd0 2023-10-03T10:18:33.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.790+0000 7fcfd67fc700 1 -- 172.21.15.3:0/1941632151 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0003dd0 con 0x7fcfd8138dd0 2023-10-03T10:18:33.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.789+0000 7f70a0c49700 1 -- 172.21.15.3:0/1497586754 wait complete. 2023-10-03T10:18:33.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.790+0000 7f70a0c49700 1 Processor -- start 2023-10-03T10:18:33.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.790+0000 7f70a0c49700 1 -- start start 2023-10-03T10:18:33.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.791+0000 7fcfde12a700 1 -- 172.21.15.3:0/1941632151 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd8138dd0 msgr2=0x7fcfd813b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:33.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.791+0000 7fcfde12a700 1 --2- 172.21.15.3:0/1941632151 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd8138dd0 0x7fcfd813b1b0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009e50 tx=0x7fcfd000b330 comp rx=0 tx=0).stop 2023-10-03T10:18:33.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.791+0000 7f70a0c49700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f709c099a40 0x7f709c1c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:33.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.791+0000 7f70a0c49700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f709c09a580 0x7f709c1c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:33.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.791+0000 7f70a0c49700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f709c131810 0x7f709c1c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:33.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.791+0000 7f70a0c49700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c12de30 con 0x7f709c131810 2023-10-03T10:18:33.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.791+0000 7f70a0c49700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709c06eb00 con 0x7f709c099a40 2023-10-03T10:18:33.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.791+0000 7f70a0c49700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f709c12bfa0 con 0x7f709c09a580 2023-10-03T10:18:33.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.791+0000 7f7099d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f709c09a580 0x7f709c1c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:33.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.791+0000 7f7099d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f709c09a580 0x7f709c1c84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42250/0 (socket says 172.21.15.3:42250) 2023-10-03T10:18:33.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.791+0000 7f7099d9b700 1 -- 172.21.15.3:0/3477596115 learned_addr learned my addr 172.21.15.3:0/3477596115 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:33.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.791+0000 7f709a59c700 1 --2- 172.21.15.3:0/3477596115 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f709c099a40 0x7f709c1c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:33.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.791+0000 7fcfde12a700 1 -- 172.21.15.3:0/1941632151 shutdown_connections 2023-10-03T10:18:33.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.791+0000 7fcfde12a700 1 --2- 172.21.15.3:0/1941632151 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd8138dd0 0x7fcfd813b1b0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:33.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.791+0000 7fcfde12a700 1 --2- 172.21.15.3:0/1941632151 >> v2:172.21.15.3:3300/0 conn(0x7fcfd8130b30 0x7fcfd8138890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:33.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.791+0000 7fcfde12a700 1 -- 172.21.15.3:0/1941632151 >> 172.21.15.3:0/1941632151 conn(0x7fcfd809e7e0 msgr2=0x7fcfd809ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:33.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.792+0000 7f7099d9b700 1 -- 172.21.15.3:0/3477596115 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f709c099a40 msgr2=0x7f709c1c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:33.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.792+0000 7f7099d9b700 1 --2- 172.21.15.3:0/3477596115 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f709c099a40 0x7f709c1c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:33.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.792+0000 7f7099d9b700 1 -- 172.21.15.3:0/3477596115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f709c131810 msgr2=0x7f709c1c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:33.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.792+0000 7f7099d9b700 1 --2- 172.21.15.3:0/3477596115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f709c131810 0x7f709c1c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:33.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.792+0000 7f7099d9b700 1 -- 172.21.15.3:0/3477596115 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7094009a70 con 0x7f709c09a580 2023-10-03T10:18:33.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.792+0000 7f7099d9b700 1 --2- 172.21.15.3:0/3477596115 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f709c09a580 0x7f709c1c84d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f709000c780 tx=0x7f709000ca90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:33.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.792+0000 7fcfde12a700 1 -- 172.21.15.3:0/1941632151 shutdown_connections 2023-10-03T10:18:33.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.792+0000 7fcfde12a700 1 -- 172.21.15.3:0/1941632151 wait complete. 2023-10-03T10:18:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.793+0000 7f708affd700 1 -- 172.21.15.3:0/3477596115 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090004c80 con 0x7f709c09a580 2023-10-03T10:18:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.793+0000 7f708affd700 1 -- 172.21.15.3:0/3477596115 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7090004de0 con 0x7f709c09a580 2023-10-03T10:18:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.793+0000 7f708affd700 1 -- 172.21.15.3:0/3477596115 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090007310 con 0x7f709c09a580 2023-10-03T10:18:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.793+0000 7f70a0c49700 1 -- 172.21.15.3:0/3477596115 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f709c12d800 con 0x7f709c09a580 2023-10-03T10:18:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.793+0000 7f70a0c49700 1 -- 172.21.15.3:0/3477596115 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f709c12e980 con 0x7f709c09a580 2023-10-03T10:18:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.793+0000 7fcfde12a700 1 Processor -- start 2023-10-03T10:18:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.794+0000 7f70a0c49700 1 -- 172.21.15.3:0/3477596115 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f709c074ea0 con 0x7f709c09a580 2023-10-03T10:18:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.794+0000 7fcfde12a700 1 -- start start 2023-10-03T10:18:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.794+0000 7fcfde12a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd8071b40 0x7fcfd81cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.794+0000 7fcfde12a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd8130b30 0x7fcfd81ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.794+0000 7fcfde12a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfd8138dd0 0x7fcfd81373c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:33.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.794+0000 7fcfde12a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd806f460 con 0x7fcfd8138dd0 2023-10-03T10:18:33.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.795+0000 7fcfde12a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd806dce0 con 0x7fcfd8071b40 2023-10-03T10:18:33.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.795+0000 7fcfde12a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd813b230 con 0x7fcfd8130b30 2023-10-03T10:18:33.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.795+0000 7fcfd7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfd8138dd0 0x7fcfd81373c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:33.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.795+0000 7fcfd7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfd8138dd0 0x7fcfd81373c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60172/0 (socket says 172.21.15.3:60172) 2023-10-03T10:18:33.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.795+0000 7fcfd7fff700 1 -- 172.21.15.3:0/3761002268 learned_addr learned my addr 172.21.15.3:0/3761002268 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:33.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.795+0000 7fcfd6ffd700 1 --2- 172.21.15.3:0/3761002268 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd8130b30 0x7fcfd81ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:33.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.795+0000 7f708affd700 1 -- 172.21.15.3:0/3477596115 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7090007470 con 0x7f709c09a580 2023-10-03T10:18:33.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.795+0000 7f708affd700 1 -- 172.21.15.3:0/3477596115 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f709003fa50 con 0x7f709c09a580 2023-10-03T10:18:33.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.796+0000 7fcfd6ffd700 1 -- 172.21.15.3:0/3761002268 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd8071b40 msgr2=0x7fcfd81cc870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:18:33.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.796+0000 7fcfd6ffd700 1 --2- 172.21.15.3:0/3761002268 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd8071b40 0x7fcfd81cc870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:33.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.796+0000 7fcfd6ffd700 1 -- 172.21.15.3:0/3761002268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfd8138dd0 msgr2=0x7fcfd81373c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:33.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.796+0000 7fcfd6ffd700 1 --2- 172.21.15.3:0/3761002268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfd8138dd0 0x7fcfd81373c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:33.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.796+0000 7fcfd6ffd700 1 -- 172.21.15.3:0/3761002268 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd0009b30 con 0x7fcfd8130b30 2023-10-03T10:18:33.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.796+0000 7fcfd6ffd700 1 --2- 172.21.15.3:0/3761002268 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd8130b30 0x7fcfd81ccdb0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00e7e0 tx=0x7fcfcc00eaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:33.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.799+0000 7fcfbffff700 1 -- 172.21.15.3:0/3761002268 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc00c070 con 0x7fcfd8130b30 2023-10-03T10:18:33.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.800+0000 7fcfbffff700 1 -- 172.21.15.3:0/3761002268 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfcc013610 con 0x7fcfd8130b30 2023-10-03T10:18:33.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.800+0000 7fcfbffff700 1 -- 172.21.15.3:0/3761002268 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc013980 con 0x7fcfd8130b30 2023-10-03T10:18:33.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.800+0000 7fcfde12a700 1 -- 172.21.15.3:0/3761002268 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd809c150 con 0x7fcfd8130b30 2023-10-03T10:18:33.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.800+0000 7fcfde12a700 1 -- 172.21.15.3:0/3761002268 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd806d730 con 0x7fcfd8130b30 2023-10-03T10:18:33.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.800+0000 7f708affd700 1 -- 172.21.15.3:0/3477596115 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f70900133c0 con 0x7f709c09a580 2023-10-03T10:18:33.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.801+0000 7fcfbffff700 1 -- 172.21.15.3:0/3761002268 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcfcc013ae0 con 0x7fcfd8130b30 2023-10-03T10:18:33.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.801+0000 7fcfde12a700 1 -- 172.21.15.3:0/3761002268 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd80747f0 con 0x7fcfd8130b30 2023-10-03T10:18:33.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.801+0000 7fcfbffff700 1 -- 172.21.15.3:0/3761002268 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fcfcc03f2f0 con 0x7fcfd8130b30 2023-10-03T10:18:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.811+0000 7fcfbffff700 1 -- 172.21.15.3:0/3761002268 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcfcc017070 con 0x7fcfd8130b30 2023-10-03T10:18:33.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.853+0000 7fa30b44d700 1 Processor -- start 2023-10-03T10:18:33.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.853+0000 7fa30b44d700 1 -- start start 2023-10-03T10:18:33.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.853+0000 7fa30b44d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa30406c0e0 0x7fa30409ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:33.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.853+0000 7fa30b44d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa304134ec0 0x7fa3041379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:33.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.853+0000 7fa30b44d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa304137f20 0x7fa304138390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:33.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.853+0000 7fa30b44d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa30406c4b0 con 0x7fa304137f20 2023-10-03T10:18:33.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.853+0000 7fa30b44d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3040a09c0 con 0x7fa30406c0e0 2023-10-03T10:18:33.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.853+0000 7fa30b44d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa304134980 con 0x7fa304134ec0 2023-10-03T10:18:33.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.853+0000 7fa3091e9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa30406c0e0 0x7fa30409ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:33.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.854+0000 7fa3091e9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa30406c0e0 0x7fa30409ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50146/0 (socket says 172.21.15.3:50146) 2023-10-03T10:18:33.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.854+0000 7fa3091e9700 1 -- 172.21.15.3:0/147323928 learned_addr learned my addr 172.21.15.3:0/147323928 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:33.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.854+0000 7fa3089e8700 1 --2- 172.21.15.3:0/147323928 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa304134ec0 0x7fa3041379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:33.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.854+0000 7fa3099ea700 1 --2- 172.21.15.3:0/147323928 >> v2:172.21.15.3:3300/0 conn(0x7fa304137f20 0x7fa304138390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:33.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.854+0000 7fa3091e9700 1 -- 172.21.15.3:0/147323928 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa304134ec0 msgr2=0x7fa3041379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:33.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.854+0000 7fa3091e9700 1 --2- 172.21.15.3:0/147323928 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa304134ec0 0x7fa3041379e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:33.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.854+0000 7fa3091e9700 1 -- 172.21.15.3:0/147323928 >> v2:172.21.15.3:3300/0 conn(0x7fa304137f20 msgr2=0x7fa304138390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:33.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.854+0000 7fa3091e9700 1 --2- 172.21.15.3:0/147323928 >> v2:172.21.15.3:3300/0 conn(0x7fa304137f20 0x7fa304138390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:33.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.854+0000 7fa3091e9700 1 -- 172.21.15.3:0/147323928 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3040a0600 con 0x7fa30406c0e0 2023-10-03T10:18:33.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.854+0000 7fa3091e9700 1 --2- 172.21.15.3:0/147323928 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa30406c0e0 0x7fa30409ac90 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fa300009920 tx=0x7fa300009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adb946a506b79a15 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:33.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.854+0000 7fa2fbfff700 1 -- 172.21.15.3:0/147323928 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa300011830 con 0x7fa30406c0e0 2023-10-03T10:18:33.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.855+0000 7fa2fbfff700 1 -- 172.21.15.3:0/147323928 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa300011990 con 0x7fa30406c0e0 2023-10-03T10:18:33.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.855+0000 7fa2fbfff700 1 -- 172.21.15.3:0/147323928 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa300011d20 con 0x7fa30406c0e0 2023-10-03T10:18:33.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.855+0000 7fa30b44d700 1 -- 172.21.15.3:0/147323928 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa30406c0e0 msgr2=0x7fa30409ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:33.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.855+0000 7fa30b44d700 1 --2- 172.21.15.3:0/147323928 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa30406c0e0 0x7fa30409ac90 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fa300009920 tx=0x7fa300009c30 comp rx=0 tx=0).stop 2023-10-03T10:18:33.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.855+0000 7fa30b44d700 1 -- 172.21.15.3:0/147323928 shutdown_connections 2023-10-03T10:18:33.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.855+0000 7fa30b44d700 1 --2- 172.21.15.3:0/147323928 >> v2:172.21.15.3:3300/0 conn(0x7fa304137f20 0x7fa304138390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:33.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.855+0000 7fa30b44d700 1 --2- 172.21.15.3:0/147323928 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa30406c0e0 0x7fa30409ac90 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:33.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.855+0000 7fa30b44d700 1 --2- 172.21.15.3:0/147323928 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa304134ec0 0x7fa3041379e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:33.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.855+0000 7fa30b44d700 1 -- 172.21.15.3:0/147323928 >> 172.21.15.3:0/147323928 conn(0x7fa3040a4210 msgr2=0x7fa3040a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:33.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.856+0000 7fa30b44d700 1 -- 172.21.15.3:0/147323928 shutdown_connections 2023-10-03T10:18:33.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.856+0000 7fa30b44d700 1 -- 172.21.15.3:0/147323928 wait complete. 2023-10-03T10:18:33.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.857+0000 7fa30b44d700 1 Processor -- start 2023-10-03T10:18:33.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.857+0000 7fa30b44d700 1 -- start start 2023-10-03T10:18:33.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.857+0000 7fa30b44d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa30406c0e0 0x7fa304134140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:33.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.857+0000 7fa30b44d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa304134ec0 0x7fa30411a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:33.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.857+0000 7fa30b44d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa304137f20 0x7fa30411aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:33.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.857+0000 7fa30b44d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30411e110 con 0x7fa304134ec0 2023-10-03T10:18:33.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.857+0000 7fa30b44d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa30411c5f0 con 0x7fa30406c0e0 2023-10-03T10:18:33.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.857+0000 7fa30b44d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa30411ca90 con 0x7fa304137f20 2023-10-03T10:18:33.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.857+0000 7fa3091e9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa30406c0e0 0x7fa304134140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:33.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.857+0000 7fa3091e9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa30406c0e0 0x7fa304134140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50158/0 (socket says 172.21.15.3:50158) 2023-10-03T10:18:33.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.857+0000 7fa3091e9700 1 -- 172.21.15.3:0/3376418187 learned_addr learned my addr 172.21.15.3:0/3376418187 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:33.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.857+0000 7fa3099ea700 1 --2- 172.21.15.3:0/3376418187 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa304137f20 0x7fa30411aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:33.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.858+0000 7fa3091e9700 1 -- 172.21.15.3:0/3376418187 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa304137f20 msgr2=0x7fa30411aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:33.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.858+0000 7fa3091e9700 1 --2- 172.21.15.3:0/3376418187 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa304137f20 0x7fa30411aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:33.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.858+0000 7fa3091e9700 1 -- 172.21.15.3:0/3376418187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa304134ec0 msgr2=0x7fa30411a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:33.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.858+0000 7fa3091e9700 1 --2- 172.21.15.3:0/3376418187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa304134ec0 0x7fa30411a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:33.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.858+0000 7fa3091e9700 1 -- 172.21.15.3:0/3376418187 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa300009550 con 0x7fa30406c0e0 2023-10-03T10:18:33.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.858+0000 7fa3091e9700 1 --2- 172.21.15.3:0/3376418187 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa30406c0e0 0x7fa304134140 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa300014040 tx=0x7fa300004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:33.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.858+0000 7fa2f9ffb700 1 -- 172.21.15.3:0/3376418187 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa300004510 con 0x7fa30406c0e0 2023-10-03T10:18:33.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.858+0000 7fa2f9ffb700 1 -- 172.21.15.3:0/3376418187 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30001e070 con 0x7fa30406c0e0 2023-10-03T10:18:33.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.858+0000 7fa2f9ffb700 1 -- 172.21.15.3:0/3376418187 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3000038e0 con 0x7fa30406c0e0 2023-10-03T10:18:33.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.858+0000 7fa30b44d700 1 -- 172.21.15.3:0/3376418187 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3040a14c0 con 0x7fa30406c0e0 2023-10-03T10:18:33.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.859+0000 7fa30b44d700 1 -- 172.21.15.3:0/3376418187 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa304138e40 con 0x7fa30406c0e0 2023-10-03T10:18:33.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.860+0000 7fa2f9ffb700 1 -- 172.21.15.3:0/3376418187 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa300003a40 con 0x7fa30406c0e0 2023-10-03T10:18:33.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.860+0000 7fa2e77fe700 1 -- 172.21.15.3:0/3376418187 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e00052e0 con 0x7fa30406c0e0 2023-10-03T10:18:33.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.860+0000 7fa2f9ffb700 1 -- 172.21.15.3:0/3376418187 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa300020070 con 0x7fa30406c0e0 2023-10-03T10:18:33.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.865+0000 7fa2f9ffb700 1 -- 172.21.15.3:0/3376418187 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa300010380 con 0x7fa30406c0e0 2023-10-03T10:18:33.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.929+0000 7f9e2f2bf700 1 Processor -- start 2023-10-03T10:18:33.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.929+0000 7f9e2f2bf700 1 -- start start 2023-10-03T10:18:33.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.930+0000 7f9e2f2bf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e2809d6f0 0x7f9e2809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:33.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.930+0000 7f9e2f2bf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e2809e000 0x7f9e28130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:33.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.930+0000 7f9e2f2bf700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e2809f130 con 0x7f9e2809e410 2023-10-03T10:18:33.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.930+0000 7f9e2f2bf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2809f520 con 0x7f9e2809e000 2023-10-03T10:18:33.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.930+0000 7f9e2f2bf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2809e930 con 0x7f9e2809d6f0 2023-10-03T10:18:33.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.930+0000 7f9e2d05b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e2809d6f0 0x7f9e2809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:33.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.930+0000 7f9e2d05b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e2809d6f0 0x7f9e2809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42270/0 (socket says 172.21.15.3:42270) 2023-10-03T10:18:33.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.930+0000 7f9e2d05b700 1 -- 172.21.15.3:0/2994266033 learned_addr learned my addr 172.21.15.3:0/2994266033 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:33.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.930+0000 7f9e2c85a700 1 --2- 172.21.15.3:0/2994266033 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e2809e000 0x7f9e28130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:33.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.931+0000 7f9e2d05b700 1 -- 172.21.15.3:0/2994266033 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e2809e000 msgr2=0x7f9e28130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:33.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.931+0000 7f9e2d05b700 1 --2- 172.21.15.3:0/2994266033 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e2809e000 0x7f9e28130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:33.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.931+0000 7f9e2d05b700 1 -- 172.21.15.3:0/2994266033 >> v1:172.21.15.3:6789/0 conn(0x7f9e2809e410 legacy=0x7f9e28130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:33.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.931+0000 7f9e2d05b700 1 -- 172.21.15.3:0/2994266033 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2809b2d0 con 0x7f9e2809d6f0 2023-10-03T10:18:33.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.931+0000 7f9e2d05b700 1 --2- 172.21.15.3:0/2994266033 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e2809d6f0 0x7f9e2809dac0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9e24009950 tx=0x7f9e24009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4ca15030b2e58af server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:33.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.931+0000 7f9e1ffff700 1 -- 172.21.15.3:0/2994266033 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e24011830 con 0x7f9e2809d6f0 2023-10-03T10:18:33.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.931+0000 7f9e1ffff700 1 -- 172.21.15.3:0/2994266033 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e24011990 con 0x7f9e2809d6f0 2023-10-03T10:18:33.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.931+0000 7f9e1ffff700 1 -- 172.21.15.3:0/2994266033 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e24011d40 con 0x7f9e2809d6f0 2023-10-03T10:18:33.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.932+0000 7f9e2f2bf700 1 -- 172.21.15.3:0/2994266033 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e2809d6f0 msgr2=0x7f9e2809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:33.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.932+0000 7f9e2f2bf700 1 --2- 172.21.15.3:0/2994266033 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e2809d6f0 0x7f9e2809dac0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9e24009950 tx=0x7f9e24009c60 comp rx=0 tx=0).stop 2023-10-03T10:18:33.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.932+0000 7f9e2f2bf700 1 -- 172.21.15.3:0/2994266033 shutdown_connections 2023-10-03T10:18:33.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.932+0000 7f9e2f2bf700 1 --2- 172.21.15.3:0/2994266033 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e2809d6f0 0x7f9e2809dac0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:33.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.932+0000 7f9e2f2bf700 1 --2- 172.21.15.3:0/2994266033 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e2809e000 0x7f9e28130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:33.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.932+0000 7f9e2f2bf700 1 -- 172.21.15.3:0/2994266033 >> 172.21.15.3:0/2994266033 conn(0x7f9e28066f50 msgr2=0x7f9e2809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:33.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.932+0000 7f9e2f2bf700 1 -- 172.21.15.3:0/2994266033 shutdown_connections 2023-10-03T10:18:33.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.932+0000 7f9e2f2bf700 1 -- 172.21.15.3:0/2994266033 wait complete. 2023-10-03T10:18:33.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.933+0000 7f9e2f2bf700 1 Processor -- start 2023-10-03T10:18:33.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.933+0000 7f9e2f2bf700 1 -- start start 2023-10-03T10:18:33.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.934+0000 7f9e2f2bf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e2809d6f0 0x7f9e28122550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:33.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.934+0000 7f9e2f2bf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e2809e000 0x7f9e28122a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:33.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.934+0000 7f9e2f2bf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e2809e410 0x7f9e28122fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:33.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.934+0000 7f9e2f2bf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e28125c40 con 0x7f9e2809e000 2023-10-03T10:18:33.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.934+0000 7f9e2f2bf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e280a1d00 con 0x7f9e2809d6f0 2023-10-03T10:18:33.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.934+0000 7f9e2f2bf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2809c700 con 0x7f9e2809e410 2023-10-03T10:18:33.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.934+0000 7f9e2d85c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e2809e410 0x7f9e28122fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:33.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.934+0000 7f9e2d85c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e2809e410 0x7f9e28122fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42278/0 (socket says 172.21.15.3:42278) 2023-10-03T10:18:33.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.934+0000 7f9e2d85c700 1 -- 172.21.15.3:0/2868593695 learned_addr learned my addr 172.21.15.3:0/2868593695 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:33.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.934+0000 7f9e2c85a700 1 --2- 172.21.15.3:0/2868593695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e2809e000 0x7f9e28122a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:33.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.934+0000 7f9e2d05b700 1 --2- 172.21.15.3:0/2868593695 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e2809d6f0 0x7f9e28122550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:33.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.934+0000 7f9e2d85c700 1 -- 172.21.15.3:0/2868593695 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e2809d6f0 msgr2=0x7f9e28122550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:33.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.934+0000 7f9e2d85c700 1 --2- 172.21.15.3:0/2868593695 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e2809d6f0 0x7f9e28122550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:33.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.934+0000 7f9e2d85c700 1 -- 172.21.15.3:0/2868593695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e2809e000 msgr2=0x7f9e28122a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:33.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.934+0000 7f9e2d85c700 1 --2- 172.21.15.3:0/2868593695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e2809e000 0x7f9e28122a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:33.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.934+0000 7f9e2d85c700 1 -- 172.21.15.3:0/2868593695 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e24009550 con 0x7f9e2809e410 2023-10-03T10:18:33.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.935+0000 7f9e2d85c700 1 --2- 172.21.15.3:0/2868593695 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e2809e410 0x7f9e28122fd0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9e2000cc50 tx=0x7f9e2000a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:33.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.935+0000 7f9e1dffb700 1 -- 172.21.15.3:0/2868593695 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2000ad80 con 0x7f9e2809e410 2023-10-03T10:18:33.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.935+0000 7f9e1dffb700 1 -- 172.21.15.3:0/2868593695 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2000ce50 con 0x7f9e2809e410 2023-10-03T10:18:33.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.935+0000 7f9e2f2bf700 1 -- 172.21.15.3:0/2868593695 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e28124800 con 0x7f9e2809e410 2023-10-03T10:18:33.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.935+0000 7f9e1dffb700 1 -- 172.21.15.3:0/2868593695 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e20003c40 con 0x7f9e2809e410 2023-10-03T10:18:33.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.935+0000 7f9e2f2bf700 1 -- 172.21.15.3:0/2868593695 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e28125f20 con 0x7f9e2809e410 2023-10-03T10:18:33.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.936+0000 7f9e2f2bf700 1 -- 172.21.15.3:0/2868593695 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e281258e0 con 0x7f9e2809e410 2023-10-03T10:18:33.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.936+0000 7f9e1dffb700 1 -- 172.21.15.3:0/2868593695 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9e200050d0 con 0x7f9e2809e410 2023-10-03T10:18:33.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.937+0000 7f9e1dffb700 1 -- 172.21.15.3:0/2868593695 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9e20047350 con 0x7f9e2809e410 2023-10-03T10:18:33.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:33.946+0000 7f9e1dffb700 1 -- 172.21.15.3:0/2868593695 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9e20044060 con 0x7f9e2809e410 2023-10-03T10:18:34.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.019+0000 7f70a0c49700 1 -- 172.21.15.3:0/3477596115 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f709c12cda0 con 0x7f709c09a580 2023-10-03T10:18:34.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.020+0000 7f708affd700 1 -- 172.21.15.3:0/3477596115 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7090011360 con 0x7f709c09a580 2023-10-03T10:18:34.020 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:34.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.022+0000 7f70a0c49700 1 -- 172.21.15.3:0/3477596115 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f709c09a580 msgr2=0x7f709c1c84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:34.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.022+0000 7f70a0c49700 1 --2- 172.21.15.3:0/3477596115 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f709c09a580 0x7f709c1c84d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f709000c780 tx=0x7f709000ca90 comp rx=0 tx=0).stop 2023-10-03T10:18:34.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.022+0000 7f70a0c49700 1 -- 172.21.15.3:0/3477596115 shutdown_connections 2023-10-03T10:18:34.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.022+0000 7f70a0c49700 1 --2- 172.21.15.3:0/3477596115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f709c131810 0x7f709c1c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:34.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.022+0000 7f70a0c49700 1 --2- 172.21.15.3:0/3477596115 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f709c099a40 0x7f709c1c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:34.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.022+0000 7f70a0c49700 1 --2- 172.21.15.3:0/3477596115 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f709c09a580 0x7f709c1c84d0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:34.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.022+0000 7f70a0c49700 1 -- 172.21.15.3:0/3477596115 >> 172.21.15.3:0/3477596115 conn(0x7f709c0a2aa0 msgr2=0x7f709c0a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:34.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.022+0000 7f70a0c49700 1 -- 172.21.15.3:0/3477596115 shutdown_connections 2023-10-03T10:18:34.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.022+0000 7f70a0c49700 1 -- 172.21.15.3:0/3477596115 wait complete. 2023-10-03T10:18:34.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.035+0000 7fcfde12a700 1 -- 172.21.15.3:0/3761002268 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcfd806b250 con 0x7fcfd8130b30 2023-10-03T10:18:34.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.035+0000 7fcfbffff700 1 -- 172.21.15.3:0/3761002268 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcfcc0106d0 con 0x7fcfd8130b30 2023-10-03T10:18:34.036 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:34.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.038+0000 7fcfde12a700 1 -- 172.21.15.3:0/3761002268 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd8130b30 msgr2=0x7fcfd81ccdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:34.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.038+0000 7fcfde12a700 1 --2- 172.21.15.3:0/3761002268 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd8130b30 0x7fcfd81ccdb0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00e7e0 tx=0x7fcfcc00eaf0 comp rx=0 tx=0).stop 2023-10-03T10:18:34.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.038+0000 7fcfde12a700 1 -- 172.21.15.3:0/3761002268 shutdown_connections 2023-10-03T10:18:34.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.038+0000 7fcfde12a700 1 --2- 172.21.15.3:0/3761002268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfd8138dd0 0x7fcfd81373c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:34.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.038+0000 7fcfde12a700 1 --2- 172.21.15.3:0/3761002268 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd8071b40 0x7fcfd81cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:34.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.038+0000 7fcfde12a700 1 --2- 172.21.15.3:0/3761002268 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd8130b30 0x7fcfd81ccdb0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:34.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.038+0000 7fcfde12a700 1 -- 172.21.15.3:0/3761002268 >> 172.21.15.3:0/3761002268 conn(0x7fcfd809e7e0 msgr2=0x7fcfd809fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:34.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.038+0000 7fcfde12a700 1 -- 172.21.15.3:0/3761002268 shutdown_connections 2023-10-03T10:18:34.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.038+0000 7fcfde12a700 1 -- 172.21.15.3:0/3761002268 wait complete. 2023-10-03T10:18:34.041 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:34.053 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:34.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.106+0000 7fa2e77fe700 1 -- 172.21.15.3:0/3376418187 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa2e0005590 con 0x7fa30406c0e0 2023-10-03T10:18:34.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.107+0000 7fa2f9ffb700 1 -- 172.21.15.3:0/3376418187 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa30002a070 con 0x7fa30406c0e0 2023-10-03T10:18:34.107 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:34.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.109+0000 7fa2e77fe700 1 -- 172.21.15.3:0/3376418187 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa30406c0e0 msgr2=0x7fa304134140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:34.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.109+0000 7fa2e77fe700 1 --2- 172.21.15.3:0/3376418187 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa30406c0e0 0x7fa304134140 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa300014040 tx=0x7fa300004750 comp rx=0 tx=0).stop 2023-10-03T10:18:34.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.109+0000 7fa2e77fe700 1 -- 172.21.15.3:0/3376418187 shutdown_connections 2023-10-03T10:18:34.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.109+0000 7fa2e77fe700 1 --2- 172.21.15.3:0/3376418187 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa304137f20 0x7fa30411aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:34.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.109+0000 7fa2e77fe700 1 --2- 172.21.15.3:0/3376418187 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa30406c0e0 0x7fa304134140 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:34.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.109+0000 7fa2e77fe700 1 --2- 172.21.15.3:0/3376418187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa304134ec0 0x7fa30411a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:34.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.109+0000 7fa2e77fe700 1 -- 172.21.15.3:0/3376418187 >> 172.21.15.3:0/3376418187 conn(0x7fa3040a4210 msgr2=0x7fa304099c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:34.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.109+0000 7fa2e77fe700 1 -- 172.21.15.3:0/3376418187 shutdown_connections 2023-10-03T10:18:34.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.109+0000 7fa2e77fe700 1 -- 172.21.15.3:0/3376418187 wait complete. 2023-10-03T10:18:34.123 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:34.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.163+0000 7f9e2f2bf700 1 -- 172.21.15.3:0/2868593695 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9e28074ea0 con 0x7f9e2809e410 2023-10-03T10:18:34.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.164+0000 7f9e1dffb700 1 -- 172.21.15.3:0/2868593695 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9e20044030 con 0x7f9e2809e410 2023-10-03T10:18:34.165 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:34.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.166+0000 7f9e2f2bf700 1 -- 172.21.15.3:0/2868593695 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e2809e410 msgr2=0x7f9e28122fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:34.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.166+0000 7f9e2f2bf700 1 --2- 172.21.15.3:0/2868593695 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e2809e410 0x7f9e28122fd0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9e2000cc50 tx=0x7f9e2000a340 comp rx=0 tx=0).stop 2023-10-03T10:18:34.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.166+0000 7f9e2f2bf700 1 -- 172.21.15.3:0/2868593695 shutdown_connections 2023-10-03T10:18:34.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.166+0000 7f9e2f2bf700 1 --2- 172.21.15.3:0/2868593695 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9e2809e410 0x7f9e28122fd0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:34.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.166+0000 7f9e2f2bf700 1 --2- 172.21.15.3:0/2868593695 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9e2809d6f0 0x7f9e28122550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:34.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.166+0000 7f9e2f2bf700 1 --2- 172.21.15.3:0/2868593695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e2809e000 0x7f9e28122a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:34.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.166+0000 7f9e2f2bf700 1 -- 172.21.15.3:0/2868593695 >> 172.21.15.3:0/2868593695 conn(0x7f9e28066f50 msgr2=0x7f9e280a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:34.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.166+0000 7f9e2f2bf700 1 -- 172.21.15.3:0/2868593695 shutdown_connections 2023-10-03T10:18:34.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:34.166+0000 7f9e2f2bf700 1 -- 172.21.15.3:0/2868593695 wait complete. 2023-10-03T10:18:34.179 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:35.042 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:35.053 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:35.124 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:35.181 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:35.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.199+0000 7f4bdc9ee700 1 Processor -- start 2023-10-03T10:18:35.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.199+0000 7f4bdc9ee700 1 -- start start 2023-10-03T10:18:35.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.200+0000 7f4bdc9ee700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bd8133540 0x7f4bd8135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:35.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.200+0000 7f4bdc9ee700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4bd8135e60 0x7f4bd8138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:35.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.200+0000 7f4bdc9ee700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd80716a0 con 0x7f4bd8130540 2023-10-03T10:18:35.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.200+0000 7f4bdc9ee700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bd8071290 con 0x7f4bd8135e60 2023-10-03T10:18:35.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.200+0000 7f4bdc9ee700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd8130340 con 0x7f4bd8133540 2023-10-03T10:18:35.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.200+0000 7f4bd659c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4bd8130540 0x7f4bd8130910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43628/0 (socket says 172.21.15.3:43628) 2023-10-03T10:18:35.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.200+0000 7f4bd5d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bd8133540 0x7f4bd8135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:35.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.200+0000 7f4bd659c700 1 -- 172.21.15.3:0/486500511 learned_addr learned my addr 172.21.15.3:0/486500511 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:35.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.200+0000 7f4bd6d9d700 1 --2- 172.21.15.3:0/486500511 >> v2:172.21.15.3:3300/0 conn(0x7f4bd8135e60 0x7f4bd8138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:35.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.200+0000 7f4bd6d9d700 1 -- 172.21.15.3:0/486500511 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bd8133540 msgr2=0x7f4bd8135920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:35.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.200+0000 7f4bd6d9d700 1 --2- 172.21.15.3:0/486500511 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bd8133540 0x7f4bd8135920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.200+0000 7f4bd6d9d700 1 -- 172.21.15.3:0/486500511 >> v1:172.21.15.3:6789/0 conn(0x7f4bd8130540 legacy=0x7f4bd8130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:35.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.200+0000 7f4bd6d9d700 1 -- 172.21.15.3:0/486500511 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd8070b80 con 0x7f4bd8135e60 2023-10-03T10:18:35.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.200+0000 7f4bd559a700 1 -- 172.21.15.3:0/486500511 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 361867646 0 0) 0x7f4bd80716a0 con 0x7f4bd8130540 2023-10-03T10:18:35.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.201+0000 7f4bd5d9b700 1 --2- 172.21.15.3:0/486500511 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bd8133540 0x7f4bd8135920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:35.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.201+0000 7f4bd6d9d700 1 --2- 172.21.15.3:0/486500511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bd8135e60 0x7f4bd8138240 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0009d90 tx=0x7f4bd000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efba88343fa1c311 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:35.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.201+0000 7f4bd559a700 1 -- 172.21.15.3:0/486500511 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd000e070 con 0x7f4bd8135e60 2023-10-03T10:18:35.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.201+0000 7f4bd559a700 1 -- 172.21.15.3:0/486500511 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd000bb20 con 0x7f4bd8135e60 2023-10-03T10:18:35.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.201+0000 7f4bd559a700 1 -- 172.21.15.3:0/486500511 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0012430 con 0x7f4bd8135e60 2023-10-03T10:18:35.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.202+0000 7f4bdc9ee700 1 -- 172.21.15.3:0/486500511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bd8135e60 msgr2=0x7f4bd8138240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:35.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.202+0000 7f4bdc9ee700 1 --2- 172.21.15.3:0/486500511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bd8135e60 0x7f4bd8138240 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0009d90 tx=0x7f4bd000b330 comp rx=0 tx=0).stop 2023-10-03T10:18:35.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.202+0000 7f4bdc9ee700 1 -- 172.21.15.3:0/486500511 shutdown_connections 2023-10-03T10:18:35.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.202+0000 7f4bdc9ee700 1 --2- 172.21.15.3:0/486500511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bd8135e60 0x7f4bd8138240 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.202+0000 7f4bdc9ee700 1 --2- 172.21.15.3:0/486500511 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bd8133540 0x7f4bd8135920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.202+0000 7f4bdc9ee700 1 -- 172.21.15.3:0/486500511 >> 172.21.15.3:0/486500511 conn(0x7f4bd80a2aa0 msgr2=0x7f4bd80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:35.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.202+0000 7f4bdc9ee700 1 -- 172.21.15.3:0/486500511 shutdown_connections 2023-10-03T10:18:35.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.202+0000 7f4bdc9ee700 1 -- 172.21.15.3:0/486500511 wait complete. 2023-10-03T10:18:35.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.204+0000 7f4bdc9ee700 1 Processor -- start 2023-10-03T10:18:35.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.204+0000 7f4bdc9ee700 1 -- start start 2023-10-03T10:18:35.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.205+0000 7f4bdc9ee700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bd8130540 0x7f4bd8118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:35.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.205+0000 7f4bdc9ee700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4bd8133540 0x7f4bd8119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:35.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.205+0000 7f4bdc9ee700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bd8135e60 0x7f4bd8119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:35.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.205+0000 7f4bdc9ee700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd812da10 con 0x7f4bd8130540 2023-10-03T10:18:35.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.205+0000 7f4bdc9ee700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd811c6c0 con 0x7f4bd8135e60 2023-10-03T10:18:35.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.205+0000 7f4bdc9ee700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd80718e0 con 0x7f4bd8133540 2023-10-03T10:18:35.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.205+0000 7f4bd5d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4bd8133540 0x7f4bd8119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:35.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.205+0000 7f4bd5d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4bd8133540 0x7f4bd8119210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42282/0 (socket says 172.21.15.3:42282) 2023-10-03T10:18:35.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.205+0000 7f4bd5d9b700 1 -- 172.21.15.3:0/3353844199 learned_addr learned my addr 172.21.15.3:0/3353844199 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:35.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.205+0000 7f4bd6d9d700 1 --2- 172.21.15.3:0/3353844199 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bd8135e60 0x7f4bd8119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:35.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.206+0000 7f4bd5d9b700 1 -- 172.21.15.3:0/3353844199 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bd8135e60 msgr2=0x7f4bd8119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:35.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.206+0000 7f4bd5d9b700 1 --2- 172.21.15.3:0/3353844199 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bd8135e60 0x7f4bd8119750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.206+0000 7f4bd5d9b700 1 -- 172.21.15.3:0/3353844199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bd8130540 msgr2=0x7f4bd8118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:35.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.206+0000 7f4bd5d9b700 1 --2- 172.21.15.3:0/3353844199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bd8130540 0x7f4bd8118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.206+0000 7f4bd5d9b700 1 -- 172.21.15.3:0/3353844199 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bcc0094a0 con 0x7f4bd8133540 2023-10-03T10:18:35.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.206+0000 7f4bd5d9b700 1 --2- 172.21.15.3:0/3353844199 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4bd8133540 0x7f4bd8119210 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc00ebe0 tx=0x7f4bcc00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:35.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.207+0000 7f4bc6ffd700 1 -- 172.21.15.3:0/3353844199 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc009800 con 0x7f4bd8133540 2023-10-03T10:18:35.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.207+0000 7f4bc6ffd700 1 -- 172.21.15.3:0/3353844199 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bcc00c910 con 0x7f4bd8133540 2023-10-03T10:18:35.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.207+0000 7f4bdc9ee700 1 -- 172.21.15.3:0/3353844199 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd0009a70 con 0x7f4bd8133540 2023-10-03T10:18:35.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.207+0000 7f4bc6ffd700 1 -- 172.21.15.3:0/3353844199 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc00cc80 con 0x7f4bd8133540 2023-10-03T10:18:35.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.207+0000 7f4bdc9ee700 1 -- 172.21.15.3:0/3353844199 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd811bc90 con 0x7f4bd8133540 2023-10-03T10:18:35.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.208+0000 7f4bdc9ee700 1 -- 172.21.15.3:0/3353844199 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd811c500 con 0x7f4bd8133540 2023-10-03T10:18:35.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.209+0000 7f4bc6ffd700 1 -- 172.21.15.3:0/3353844199 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4bcc013470 con 0x7f4bd8133540 2023-10-03T10:18:35.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.209+0000 7f4bc6ffd700 1 -- 172.21.15.3:0/3353844199 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4bcc03fbf0 con 0x7f4bd8133540 2023-10-03T10:18:35.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.215+0000 7f4bc6ffd700 1 -- 172.21.15.3:0/3353844199 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4bd811c500 con 0x7f4bd8133540 2023-10-03T10:18:35.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.237+0000 7f4bee02b700 1 Processor -- start 2023-10-03T10:18:35.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.237+0000 7f4bee02b700 1 -- start start 2023-10-03T10:18:35.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.237+0000 7f4bee02b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4be806f5d0 0x7f4be812ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:35.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.237+0000 7f4bee02b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4be8135c20 0x7f4be8138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:35.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.238+0000 7f4bee02b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be806fa40 con 0x7f4be812cf60 2023-10-03T10:18:35.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.238+0000 7f4bee02b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be812cd90 con 0x7f4be806f5d0 2023-10-03T10:18:35.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.238+0000 7f4bee02b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be8071780 con 0x7f4be8135c20 2023-10-03T10:18:35.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.238+0000 7f4be6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4be806f5d0 0x7f4be812ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:35.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.238+0000 7f4be6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4be806f5d0 0x7f4be812ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60236/0 (socket says 172.21.15.3:60236) 2023-10-03T10:18:35.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.238+0000 7f4be6ffd700 1 -- 172.21.15.3:0/1956450970 learned_addr learned my addr 172.21.15.3:0/1956450970 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:35.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.238+0000 7f4be77fe700 1 --1- 172.21.15.3:0/1956450970 >> v1:172.21.15.3:6789/0 conn(0x7f4be812cf60 0x7f4be812f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43644/0 (socket says 172.21.15.3:43644) 2023-10-03T10:18:35.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.238+0000 7f4be7fff700 1 --2- 172.21.15.3:0/1956450970 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4be8135c20 0x7f4be8138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:35.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.238+0000 7f4be7fff700 1 -- 172.21.15.3:0/1956450970 >> v2:172.21.15.3:3300/0 conn(0x7f4be806f5d0 msgr2=0x7f4be812ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:35.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.238+0000 7f4be7fff700 1 --2- 172.21.15.3:0/1956450970 >> v2:172.21.15.3:3300/0 conn(0x7f4be806f5d0 0x7f4be812ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.238+0000 7f4be7fff700 1 -- 172.21.15.3:0/1956450970 >> v1:172.21.15.3:6789/0 conn(0x7f4be812cf60 legacy=0x7f4be812f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:35.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.238+0000 7f4be7fff700 1 -- 172.21.15.3:0/1956450970 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be8071a40 con 0x7f4be8135c20 2023-10-03T10:18:35.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.238+0000 7f4be6ffd700 1 --2- 172.21.15.3:0/1956450970 >> v2:172.21.15.3:3300/0 conn(0x7f4be806f5d0 0x7f4be812ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:35.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.239+0000 7f4be7fff700 1 --2- 172.21.15.3:0/1956450970 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4be8135c20 0x7f4be8138000 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009d90 tx=0x7f4be000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2cb97ac9d7a8613 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:35.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.239+0000 7f4be67fc700 1 -- 172.21.15.3:0/1956450970 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be000e070 con 0x7f4be8135c20 2023-10-03T10:18:35.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.239+0000 7f4be67fc700 1 -- 172.21.15.3:0/1956450970 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be000bb20 con 0x7f4be8135c20 2023-10-03T10:18:35.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.239+0000 7f4be67fc700 1 -- 172.21.15.3:0/1956450970 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be0012430 con 0x7f4be8135c20 2023-10-03T10:18:35.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.240+0000 7f4bee02b700 1 -- 172.21.15.3:0/1956450970 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4be8135c20 msgr2=0x7f4be8138000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:35.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.240+0000 7f4bee02b700 1 --2- 172.21.15.3:0/1956450970 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4be8135c20 0x7f4be8138000 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009d90 tx=0x7f4be000b330 comp rx=0 tx=0).stop 2023-10-03T10:18:35.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.240+0000 7f4bee02b700 1 -- 172.21.15.3:0/1956450970 shutdown_connections 2023-10-03T10:18:35.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.240+0000 7f4bee02b700 1 --2- 172.21.15.3:0/1956450970 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4be8135c20 0x7f4be8138000 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.240+0000 7f4bee02b700 1 --2- 172.21.15.3:0/1956450970 >> v2:172.21.15.3:3300/0 conn(0x7f4be806f5d0 0x7f4be812ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.240+0000 7f4bee02b700 1 -- 172.21.15.3:0/1956450970 >> 172.21.15.3:0/1956450970 conn(0x7f4be80a2aa0 msgr2=0x7f4be80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:35.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.240+0000 7f4bee02b700 1 -- 172.21.15.3:0/1956450970 shutdown_connections 2023-10-03T10:18:35.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.240+0000 7f4bee02b700 1 -- 172.21.15.3:0/1956450970 wait complete. 2023-10-03T10:18:35.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.241+0000 7f4bee02b700 1 Processor -- start 2023-10-03T10:18:35.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.241+0000 7f4bee02b700 1 -- start start 2023-10-03T10:18:35.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.242+0000 7f4bee02b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4be806f5d0 0x7f4be80a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:35.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.242+0000 7f4bee02b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4be812cf60 0x7f4be80a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:35.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.242+0000 7f4bee02b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4be8135c20 0x7f4be80a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:35.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.242+0000 7f4bee02b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be8127ce0 con 0x7f4be806f5d0 2023-10-03T10:18:35.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.242+0000 7f4bee02b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be8129fa0 con 0x7f4be812cf60 2023-10-03T10:18:35.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.242+0000 7f4bee02b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4be81284e0 con 0x7f4be8135c20 2023-10-03T10:18:35.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.242+0000 7f4be77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4be806f5d0 0x7f4be80a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:35.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.242+0000 7f4be77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4be806f5d0 0x7f4be80a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60240/0 (socket says 172.21.15.3:60240) 2023-10-03T10:18:35.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.242+0000 7f4be77fe700 1 -- 172.21.15.3:0/2698094236 learned_addr learned my addr 172.21.15.3:0/2698094236 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:35.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.242+0000 7f4be6ffd700 1 --2- 172.21.15.3:0/2698094236 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4be812cf60 0x7f4be80a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:35.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.242+0000 7f4be7fff700 1 --2- 172.21.15.3:0/2698094236 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4be8135c20 0x7f4be80a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:35.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.242+0000 7f4be77fe700 1 -- 172.21.15.3:0/2698094236 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4be8135c20 msgr2=0x7f4be80a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:35.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.242+0000 7f4be77fe700 1 --2- 172.21.15.3:0/2698094236 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4be8135c20 0x7f4be80a1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.242+0000 7f4be77fe700 1 -- 172.21.15.3:0/2698094236 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4be812cf60 msgr2=0x7f4be80a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:35.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.242+0000 7f4be77fe700 1 --2- 172.21.15.3:0/2698094236 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4be812cf60 0x7f4be80a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.242+0000 7f4be77fe700 1 -- 172.21.15.3:0/2698094236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bdc009550 con 0x7f4be806f5d0 2023-10-03T10:18:35.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.242+0000 7f4be6ffd700 1 --2- 172.21.15.3:0/2698094236 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4be812cf60 0x7f4be80a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:35.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.243+0000 7f4be77fe700 1 --2- 172.21.15.3:0/2698094236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4be806f5d0 0x7f4be80a0580 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f4bd800c520 tx=0x7f4bd800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:35.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.243+0000 7f4bcffff700 1 -- 172.21.15.3:0/2698094236 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8005cb0 con 0x7f4be806f5d0 2023-10-03T10:18:35.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.243+0000 7f4bcffff700 1 -- 172.21.15.3:0/2698094236 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd8005e10 con 0x7f4be806f5d0 2023-10-03T10:18:35.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.243+0000 7f4bcffff700 1 -- 172.21.15.3:0/2698094236 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd80072d0 con 0x7f4be806f5d0 2023-10-03T10:18:35.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.244+0000 7f4bee02b700 1 -- 172.21.15.3:0/2698094236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be0009a70 con 0x7f4be806f5d0 2023-10-03T10:18:35.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.244+0000 7f4bee02b700 1 -- 172.21.15.3:0/2698094236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be81275d0 con 0x7f4be806f5d0 2023-10-03T10:18:35.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.245+0000 7f4bee02b700 1 -- 172.21.15.3:0/2698094236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be80a1b00 con 0x7f4be806f5d0 2023-10-03T10:18:35.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.245+0000 7f4bcffff700 1 -- 172.21.15.3:0/2698094236 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4bd8007430 con 0x7f4be806f5d0 2023-10-03T10:18:35.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.245+0000 7f4bcffff700 1 -- 172.21.15.3:0/2698094236 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4bd803ea20 con 0x7f4be806f5d0 2023-10-03T10:18:35.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.254+0000 7f4bcffff700 1 -- 172.21.15.3:0/2698094236 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4bd801b3b0 con 0x7f4be806f5d0 2023-10-03T10:18:35.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.284+0000 7fc338f3f700 1 Processor -- start 2023-10-03T10:18:35.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.285+0000 7fc338f3f700 1 -- start start 2023-10-03T10:18:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.286+0000 7fc338f3f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc334130670 0x7fc334130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.286+0000 7fc338f3f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc33406f680 0x7fc33406fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.286+0000 7fc338f3f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc33406ffd0 0x7fc334137530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.286+0000 7fc338f3f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc33409a350 con 0x7fc33406f680 2023-10-03T10:18:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.286+0000 7fc338f3f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc334130260 con 0x7fc33406ffd0 2023-10-03T10:18:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.286+0000 7fc338f3f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc334071ff0 con 0x7fc334130670 2023-10-03T10:18:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.286+0000 7fc33259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc334130670 0x7fc334130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.286+0000 7fc33259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc334130670 0x7fc334130a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42300/0 (socket says 172.21.15.3:42300) 2023-10-03T10:18:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.286+0000 7fc33259c700 1 -- 172.21.15.3:0/284855236 learned_addr learned my addr 172.21.15.3:0/284855236 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.286+0000 7fc332d9d700 1 --2- 172.21.15.3:0/284855236 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc33406ffd0 0x7fc334137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.286+0000 7fc331d9b700 1 --2- 172.21.15.3:0/284855236 >> v2:172.21.15.3:3300/0 conn(0x7fc33406f680 0x7fc33406fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:35.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.286+0000 7fc33259c700 1 -- 172.21.15.3:0/284855236 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc33406ffd0 msgr2=0x7fc334137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:35.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.286+0000 7fc33259c700 1 --2- 172.21.15.3:0/284855236 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc33406ffd0 0x7fc334137530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.286+0000 7fc33259c700 1 -- 172.21.15.3:0/284855236 >> v2:172.21.15.3:3300/0 conn(0x7fc33406f680 msgr2=0x7fc33406fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:35.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.286+0000 7fc33259c700 1 --2- 172.21.15.3:0/284855236 >> v2:172.21.15.3:3300/0 conn(0x7fc33406f680 0x7fc33406fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.286+0000 7fc33259c700 1 -- 172.21.15.3:0/284855236 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc334070470 con 0x7fc334130670 2023-10-03T10:18:35.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.287+0000 7fc331d9b700 1 --2- 172.21.15.3:0/284855236 >> v2:172.21.15.3:3300/0 conn(0x7fc33406f680 0x7fc33406fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:35.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.287+0000 7fc33259c700 1 --2- 172.21.15.3:0/284855236 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc334130670 0x7fc334130a40 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc32400bbc0 tx=0x7fc32400bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6d7b7e53d786195 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:35.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.287+0000 7fc33159a700 1 -- 172.21.15.3:0/284855236 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32400dcf0 con 0x7fc334130670 2023-10-03T10:18:35.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.287+0000 7fc33159a700 1 -- 172.21.15.3:0/284855236 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32400de50 con 0x7fc334130670 2023-10-03T10:18:35.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.287+0000 7fc33159a700 1 -- 172.21.15.3:0/284855236 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc324010690 con 0x7fc334130670 2023-10-03T10:18:35.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.288+0000 7fc338f3f700 1 -- 172.21.15.3:0/284855236 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc334130670 msgr2=0x7fc334130a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:35.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.288+0000 7fc338f3f700 1 --2- 172.21.15.3:0/284855236 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc334130670 0x7fc334130a40 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc32400bbc0 tx=0x7fc32400bed0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.288+0000 7fc338f3f700 1 -- 172.21.15.3:0/284855236 shutdown_connections 2023-10-03T10:18:35.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.288+0000 7fc338f3f700 1 --2- 172.21.15.3:0/284855236 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc33406ffd0 0x7fc334137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.288+0000 7fc338f3f700 1 --2- 172.21.15.3:0/284855236 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc334130670 0x7fc334130a40 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.288+0000 7fc338f3f700 1 --2- 172.21.15.3:0/284855236 >> v2:172.21.15.3:3300/0 conn(0x7fc33406f680 0x7fc33406fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.288+0000 7fc338f3f700 1 -- 172.21.15.3:0/284855236 >> 172.21.15.3:0/284855236 conn(0x7fc3340a2aa0 msgr2=0x7fc3340a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.288+0000 7fc338f3f700 1 -- 172.21.15.3:0/284855236 shutdown_connections 2023-10-03T10:18:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.289+0000 7fc338f3f700 1 -- 172.21.15.3:0/284855236 wait complete. 2023-10-03T10:18:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.290+0000 7fc338f3f700 1 Processor -- start 2023-10-03T10:18:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.290+0000 7fc338f3f700 1 -- start start 2023-10-03T10:18:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.290+0000 7fc338f3f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc33406f680 0x7fc33411b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.290+0000 7fc338f3f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc33406ffd0 0x7fc33411beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.290+0000 7fc338f3f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc334130670 0x7fc33411c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.290+0000 7fc338f3f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33411f4a0 con 0x7fc334130670 2023-10-03T10:18:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.290+0000 7fc338f3f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc33411ddc0 con 0x7fc33406ffd0 2023-10-03T10:18:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.290+0000 7fc338f3f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc33411daa0 con 0x7fc33406f680 2023-10-03T10:18:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.290+0000 7fc332d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc334130670 0x7fc33411c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.291+0000 7fc332d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc334130670 0x7fc33411c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60258/0 (socket says 172.21.15.3:60258) 2023-10-03T10:18:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.291+0000 7fc332d9d700 1 -- 172.21.15.3:0/1237539706 learned_addr learned my addr 172.21.15.3:0/1237539706 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.291+0000 7fc33259c700 1 --2- 172.21.15.3:0/1237539706 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc33406f680 0x7fc33411b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.291+0000 7fc331d9b700 1 --2- 172.21.15.3:0/1237539706 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc33406ffd0 0x7fc33411beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.291+0000 7fc331d9b700 1 -- 172.21.15.3:0/1237539706 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc33406f680 msgr2=0x7fc33411b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.291+0000 7fc331d9b700 1 --2- 172.21.15.3:0/1237539706 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc33406f680 0x7fc33411b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.291+0000 7fc331d9b700 1 -- 172.21.15.3:0/1237539706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc334130670 msgr2=0x7fc33411c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.291+0000 7fc331d9b700 1 --2- 172.21.15.3:0/1237539706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc334130670 0x7fc33411c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.291+0000 7fc331d9b700 1 -- 172.21.15.3:0/1237539706 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3280094a0 con 0x7fc33406ffd0 2023-10-03T10:18:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.292+0000 7fc331d9b700 1 --2- 172.21.15.3:0/1237539706 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc33406ffd0 0x7fc33411beb0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc32800e980 tx=0x7fc32800ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:35.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.292+0000 7fc322ffd700 1 -- 172.21.15.3:0/1237539706 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc328013980 con 0x7fc33406ffd0 2023-10-03T10:18:35.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.292+0000 7fc322ffd700 1 -- 172.21.15.3:0/1237539706 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc328013ae0 con 0x7fc33406ffd0 2023-10-03T10:18:35.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.292+0000 7fc322ffd700 1 -- 172.21.15.3:0/1237539706 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc328013e70 con 0x7fc33406ffd0 2023-10-03T10:18:35.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.292+0000 7fc338f3f700 1 -- 172.21.15.3:0/1237539706 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc32400b7f0 con 0x7fc33406ffd0 2023-10-03T10:18:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.292+0000 7fc338f3f700 1 -- 172.21.15.3:0/1237539706 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc33411eb90 con 0x7fc33406ffd0 2023-10-03T10:18:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.293+0000 7fc322ffd700 1 -- 172.21.15.3:0/1237539706 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc32801b440 con 0x7fc33406ffd0 2023-10-03T10:18:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.293+0000 7fc338f3f700 1 -- 172.21.15.3:0/1237539706 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc33411ed80 con 0x7fc33406ffd0 2023-10-03T10:18:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.293+0000 7fc322ffd700 1 -- 172.21.15.3:0/1237539706 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc3280437c0 con 0x7fc33406ffd0 2023-10-03T10:18:35.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.303+0000 7fc322ffd700 1 -- 172.21.15.3:0/1237539706 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc3280243b0 con 0x7fc33406ffd0 2023-10-03T10:18:35.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.340+0000 7f91bcb98700 1 Processor -- start 2023-10-03T10:18:35.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.340+0000 7f91bcb98700 1 -- start start 2023-10-03T10:18:35.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.341+0000 7f91bcb98700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f91b809e930 0x7f91b809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:35.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.341+0000 7f91bcb98700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f91b809f240 0x7f91b81317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:35.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.341+0000 7f91bcb98700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b8070940 con 0x7f91b809f650 2023-10-03T10:18:35.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.341+0000 7f91bcb98700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91b806fd60 con 0x7f91b809f240 2023-10-03T10:18:35.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.341+0000 7f91bcb98700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91b806fa80 con 0x7f91b809e930 2023-10-03T10:18:35.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.341+0000 7f91b659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f91b809e930 0x7f91b809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:35.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.341+0000 7f91b5d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f91b809f240 0x7f91b81317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:35.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.341+0000 7f91b659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f91b809e930 0x7f91b809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42314/0 (socket says 172.21.15.3:42314) 2023-10-03T10:18:35.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.341+0000 7f91b659c700 1 -- 172.21.15.3:0/90460985 learned_addr learned my addr 172.21.15.3:0/90460985 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:35.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.341+0000 7f91b659c700 1 -- 172.21.15.3:0/90460985 >> v2:172.21.15.3:3300/0 conn(0x7f91b809f240 msgr2=0x7f91b81317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:35.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.341+0000 7f91b659c700 1 --2- 172.21.15.3:0/90460985 >> v2:172.21.15.3:3300/0 conn(0x7f91b809f240 0x7f91b81317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.341+0000 7f91b659c700 1 -- 172.21.15.3:0/90460985 >> v1:172.21.15.3:6789/0 conn(0x7f91b809f650 legacy=0x7f91b8131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:35.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.341+0000 7f91b659c700 1 -- 172.21.15.3:0/90460985 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b809fb60 con 0x7f91b809e930 2023-10-03T10:18:35.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.342+0000 7f91b659c700 1 --2- 172.21.15.3:0/90460985 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f91b809e930 0x7f91b809ed00 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f91a80098f0 tx=0x7f91a8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=608badcb6250c8f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:35.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.342+0000 7f91b559a700 1 -- 172.21.15.3:0/90460985 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a8011830 con 0x7f91b809e930 2023-10-03T10:18:35.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.342+0000 7f91b559a700 1 -- 172.21.15.3:0/90460985 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a8011990 con 0x7f91b809e930 2023-10-03T10:18:35.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.342+0000 7f91b559a700 1 -- 172.21.15.3:0/90460985 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a8011d20 con 0x7f91b809e930 2023-10-03T10:18:35.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.342+0000 7f91bcb98700 1 -- 172.21.15.3:0/90460985 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f91b809e930 msgr2=0x7f91b809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:35.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.342+0000 7f91bcb98700 1 --2- 172.21.15.3:0/90460985 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f91b809e930 0x7f91b809ed00 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f91a80098f0 tx=0x7f91a8009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:35.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.343+0000 7f91bcb98700 1 -- 172.21.15.3:0/90460985 shutdown_connections 2023-10-03T10:18:35.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.343+0000 7f91bcb98700 1 --2- 172.21.15.3:0/90460985 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f91b809e930 0x7f91b809ed00 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.343+0000 7f91bcb98700 1 --2- 172.21.15.3:0/90460985 >> v2:172.21.15.3:3300/0 conn(0x7f91b809f240 0x7f91b81317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.343+0000 7f91bcb98700 1 -- 172.21.15.3:0/90460985 >> 172.21.15.3:0/90460985 conn(0x7f91b80a4210 msgr2=0x7f91b80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:35.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.343+0000 7f91bcb98700 1 -- 172.21.15.3:0/90460985 shutdown_connections 2023-10-03T10:18:35.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.343+0000 7f91bcb98700 1 -- 172.21.15.3:0/90460985 wait complete. 2023-10-03T10:18:35.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.344+0000 7f91bcb98700 1 Processor -- start 2023-10-03T10:18:35.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.344+0000 7f91bcb98700 1 -- start start 2023-10-03T10:18:35.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.345+0000 7f91bcb98700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f91b809e930 0x7f91b81c44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:35.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.345+0000 7f91bcb98700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f91b809f240 0x7f91b81c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:35.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.345+0000 7f91bcb98700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91b809f650 0x7f91b81c0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:35.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.345+0000 7f91bcb98700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b81a52c0 con 0x7f91b809f650 2023-10-03T10:18:35.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.345+0000 7f91bcb98700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b81a67b0 con 0x7f91b809e930 2023-10-03T10:18:35.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.345+0000 7f91bcb98700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91b80a32d0 con 0x7f91b809f240 2023-10-03T10:18:35.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.345+0000 7f91b659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f91b809e930 0x7f91b81c44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:35.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.345+0000 7f91b659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f91b809e930 0x7f91b81c44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50234/0 (socket says 172.21.15.3:50234) 2023-10-03T10:18:35.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.345+0000 7f91b659c700 1 -- 172.21.15.3:0/2016440888 learned_addr learned my addr 172.21.15.3:0/2016440888 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:35.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.345+0000 7f91b6d9d700 1 --2- 172.21.15.3:0/2016440888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91b809f650 0x7f91b81c0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:35.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.345+0000 7f91b5d9b700 1 --2- 172.21.15.3:0/2016440888 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f91b809f240 0x7f91b81c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:35.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.346+0000 7f91b659c700 1 -- 172.21.15.3:0/2016440888 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f91b809f240 msgr2=0x7f91b81c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:35.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.346+0000 7f91b659c700 1 --2- 172.21.15.3:0/2016440888 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f91b809f240 0x7f91b81c49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.346+0000 7f91b659c700 1 -- 172.21.15.3:0/2016440888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91b809f650 msgr2=0x7f91b81c0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:35.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.346+0000 7f91b659c700 1 --2- 172.21.15.3:0/2016440888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91b809f650 0x7f91b81c0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.346+0000 7f91b659c700 1 -- 172.21.15.3:0/2016440888 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a8009550 con 0x7f91b809e930 2023-10-03T10:18:35.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.346+0000 7f91b659c700 1 --2- 172.21.15.3:0/2016440888 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f91b809e930 0x7f91b81c44b0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f91a8000c00 tx=0x7f91a8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:35.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.346+0000 7f91a6ffd700 1 -- 172.21.15.3:0/2016440888 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a8004590 con 0x7f91b809e930 2023-10-03T10:18:35.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.346+0000 7f91a6ffd700 1 -- 172.21.15.3:0/2016440888 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a80036a0 con 0x7f91b809e930 2023-10-03T10:18:35.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.346+0000 7f91bcb98700 1 -- 172.21.15.3:0/2016440888 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b80a0190 con 0x7f91b809e930 2023-10-03T10:18:35.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.346+0000 7f91bcb98700 1 -- 172.21.15.3:0/2016440888 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b81a5b70 con 0x7f91b809e930 2023-10-03T10:18:35.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.347+0000 7f91a6ffd700 1 -- 172.21.15.3:0/2016440888 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a8003a50 con 0x7f91b809e930 2023-10-03T10:18:35.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.348+0000 7f91bcb98700 1 -- 172.21.15.3:0/2016440888 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b81a7600 con 0x7f91b809e930 2023-10-03T10:18:35.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.348+0000 7f91a6ffd700 1 -- 172.21.15.3:0/2016440888 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f91a8003bb0 con 0x7f91b809e930 2023-10-03T10:18:35.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.349+0000 7f91a6ffd700 1 -- 172.21.15.3:0/2016440888 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f91a8021030 con 0x7f91b809e930 2023-10-03T10:18:35.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.357+0000 7f91a6ffd700 1 -- 172.21.15.3:0/2016440888 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f91a8030480 con 0x7f91b809e930 2023-10-03T10:18:35.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.452+0000 7f4bdc9ee700 1 -- 172.21.15.3:0/3353844199 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4bd811c500 con 0x7f4bd8133540 2023-10-03T10:18:35.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.453+0000 7f4bc6ffd700 1 -- 172.21.15.3:0/3353844199 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4bcc021020 con 0x7f4bd8133540 2023-10-03T10:18:35.453 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:35.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.455+0000 7f4bdc9ee700 1 -- 172.21.15.3:0/3353844199 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4bd8133540 msgr2=0x7f4bd8119210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:35.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.455+0000 7f4bdc9ee700 1 --2- 172.21.15.3:0/3353844199 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4bd8133540 0x7f4bd8119210 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc00ebe0 tx=0x7f4bcc00eef0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.455+0000 7f4bdc9ee700 1 -- 172.21.15.3:0/3353844199 shutdown_connections 2023-10-03T10:18:35.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.455+0000 7f4bdc9ee700 1 --2- 172.21.15.3:0/3353844199 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4bd8135e60 0x7f4bd8119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.455+0000 7f4bdc9ee700 1 --2- 172.21.15.3:0/3353844199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bd8130540 0x7f4bd8118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.455+0000 7f4bdc9ee700 1 --2- 172.21.15.3:0/3353844199 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4bd8133540 0x7f4bd8119210 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.455+0000 7f4bdc9ee700 1 -- 172.21.15.3:0/3353844199 >> 172.21.15.3:0/3353844199 conn(0x7f4bd80a2aa0 msgr2=0x7f4bd806ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:35.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.455+0000 7f4bdc9ee700 1 -- 172.21.15.3:0/3353844199 shutdown_connections 2023-10-03T10:18:35.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.455+0000 7f4bdc9ee700 1 -- 172.21.15.3:0/3353844199 wait complete. 2023-10-03T10:18:35.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.469+0000 7f4bee02b700 1 -- 172.21.15.3:0/2698094236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4be8127760 con 0x7f4be806f5d0 2023-10-03T10:18:35.470 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:35.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.469+0000 7f4bcffff700 1 -- 172.21.15.3:0/2698094236 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4be8127760 con 0x7f4be806f5d0 2023-10-03T10:18:35.470 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:35.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.472+0000 7f4bee02b700 1 -- 172.21.15.3:0/2698094236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4be806f5d0 msgr2=0x7f4be80a0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:35.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.472+0000 7f4bee02b700 1 --2- 172.21.15.3:0/2698094236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4be806f5d0 0x7f4be80a0580 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f4bd800c520 tx=0x7f4bd800c830 comp rx=0 tx=0).stop 2023-10-03T10:18:35.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.472+0000 7f4bee02b700 1 -- 172.21.15.3:0/2698094236 shutdown_connections 2023-10-03T10:18:35.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.472+0000 7f4bee02b700 1 --2- 172.21.15.3:0/2698094236 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4be8135c20 0x7f4be80a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.472+0000 7f4bee02b700 1 --2- 172.21.15.3:0/2698094236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4be806f5d0 0x7f4be80a0580 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.472+0000 7f4bee02b700 1 --2- 172.21.15.3:0/2698094236 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4be812cf60 0x7f4be80a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.472+0000 7f4bee02b700 1 -- 172.21.15.3:0/2698094236 >> 172.21.15.3:0/2698094236 conn(0x7f4be80a2aa0 msgr2=0x7f4be8129100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:35.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.472+0000 7f4bee02b700 1 -- 172.21.15.3:0/2698094236 shutdown_connections 2023-10-03T10:18:35.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.472+0000 7f4bee02b700 1 -- 172.21.15.3:0/2698094236 wait complete. 2023-10-03T10:18:35.486 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:35.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.514+0000 7fc338f3f700 1 -- 172.21.15.3:0/1237539706 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc33409ecb0 con 0x7fc33406ffd0 2023-10-03T10:18:35.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.514+0000 7fc322ffd700 1 -- 172.21.15.3:0/1237539706 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc328020070 con 0x7fc33406ffd0 2023-10-03T10:18:35.514 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:35.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.516+0000 7fc338f3f700 1 -- 172.21.15.3:0/1237539706 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc33406ffd0 msgr2=0x7fc33411beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:35.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.516+0000 7fc338f3f700 1 --2- 172.21.15.3:0/1237539706 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc33406ffd0 0x7fc33411beb0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc32800e980 tx=0x7fc32800ec90 comp rx=0 tx=0).stop 2023-10-03T10:18:35.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.516+0000 7fc338f3f700 1 -- 172.21.15.3:0/1237539706 shutdown_connections 2023-10-03T10:18:35.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.516+0000 7fc338f3f700 1 --2- 172.21.15.3:0/1237539706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc334130670 0x7fc33411c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.516+0000 7fc338f3f700 1 --2- 172.21.15.3:0/1237539706 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc33406f680 0x7fc33411b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.516+0000 7fc338f3f700 1 --2- 172.21.15.3:0/1237539706 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc33406ffd0 0x7fc33411beb0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.516+0000 7fc338f3f700 1 -- 172.21.15.3:0/1237539706 >> 172.21.15.3:0/1237539706 conn(0x7fc3340a2aa0 msgr2=0x7fc33406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:35.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.516+0000 7fc338f3f700 1 -- 172.21.15.3:0/1237539706 shutdown_connections 2023-10-03T10:18:35.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.516+0000 7fc338f3f700 1 -- 172.21.15.3:0/1237539706 wait complete. 2023-10-03T10:18:35.530 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:35.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.569+0000 7f91bcb98700 1 -- 172.21.15.3:0/2016440888 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f91b81a5b70 con 0x7f91b809e930 2023-10-03T10:18:35.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.569+0000 7f91a6ffd700 1 -- 172.21.15.3:0/2016440888 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f91a8015030 con 0x7f91b809e930 2023-10-03T10:18:35.570 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:35.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.572+0000 7f91bcb98700 1 -- 172.21.15.3:0/2016440888 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f91b809e930 msgr2=0x7f91b81c44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:35.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.572+0000 7f91bcb98700 1 --2- 172.21.15.3:0/2016440888 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f91b809e930 0x7f91b81c44b0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f91a8000c00 tx=0x7f91a8004750 comp rx=0 tx=0).stop 2023-10-03T10:18:35.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.572+0000 7f91bcb98700 1 -- 172.21.15.3:0/2016440888 shutdown_connections 2023-10-03T10:18:35.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.572+0000 7f91bcb98700 1 --2- 172.21.15.3:0/2016440888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91b809f650 0x7f91b81c0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.572+0000 7f91bcb98700 1 --2- 172.21.15.3:0/2016440888 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f91b809e930 0x7f91b81c44b0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.572+0000 7f91bcb98700 1 --2- 172.21.15.3:0/2016440888 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f91b809f240 0x7f91b81c49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:35.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.572+0000 7f91bcb98700 1 -- 172.21.15.3:0/2016440888 >> 172.21.15.3:0/2016440888 conn(0x7f91b80a4210 msgr2=0x7f91b80a12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:35.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.572+0000 7f91bcb98700 1 -- 172.21.15.3:0/2016440888 shutdown_connections 2023-10-03T10:18:35.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:35.572+0000 7f91bcb98700 1 -- 172.21.15.3:0/2016440888 wait complete. 2023-10-03T10:18:35.585 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:36.471 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:36.487 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:36.532 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:36.586 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:36.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.625+0000 7f85e6b6c700 1 Processor -- start 2023-10-03T10:18:36.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.625+0000 7f85e6b6c700 1 -- start start 2023-10-03T10:18:36.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.626+0000 7f85e6b6c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85e009d6f0 0x7f85e009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:36.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.626+0000 7f85e6b6c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f85e009e000 0x7f85e01339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:36.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.626+0000 7f85e6b6c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85e009e410 0x7f85e0133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:36.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.626+0000 7f85e6b6c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e006b460 con 0x7f85e009e000 2023-10-03T10:18:36.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.626+0000 7f85e6b6c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e009b120 con 0x7f85e009d6f0 2023-10-03T10:18:36.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.626+0000 7f85e6b6c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85e009a7e0 con 0x7f85e009e410 2023-10-03T10:18:36.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.626+0000 7f85e4908700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85e009d6f0 0x7f85e009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:36.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.626+0000 7f85e4908700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85e009d6f0 0x7f85e009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50240/0 (socket says 172.21.15.3:50240) 2023-10-03T10:18:36.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.626+0000 7f85e4908700 1 -- 172.21.15.3:0/2760234006 learned_addr learned my addr 172.21.15.3:0/2760234006 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:36.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.626+0000 7f85dffff700 1 --2- 172.21.15.3:0/2760234006 >> v2:172.21.15.3:3300/0 conn(0x7f85e009e000 0x7f85e01339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:36.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.626+0000 7f85e5109700 1 --2- 172.21.15.3:0/2760234006 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85e009e410 0x7f85e0133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:36.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.627+0000 7f85dffff700 1 -- 172.21.15.3:0/2760234006 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85e009e410 msgr2=0x7f85e0133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:36.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.627+0000 7f85dffff700 1 --2- 172.21.15.3:0/2760234006 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85e009e410 0x7f85e0133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.627+0000 7f85dffff700 1 -- 172.21.15.3:0/2760234006 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85e009d6f0 msgr2=0x7f85e009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:36.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.627+0000 7f85dffff700 1 --2- 172.21.15.3:0/2760234006 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85e009d6f0 0x7f85e009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.627+0000 7f85dffff700 1 -- 172.21.15.3:0/2760234006 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e009a500 con 0x7f85e009e000 2023-10-03T10:18:36.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.627+0000 7f85e4908700 1 --2- 172.21.15.3:0/2760234006 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85e009d6f0 0x7f85e009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:36.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.627+0000 7f85dffff700 1 --2- 172.21.15.3:0/2760234006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85e009e000 0x7f85e01339f0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f85cc009840 tx=0x7f85cc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f7d6a5147d46a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:36.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.627+0000 7f85df7fe700 1 -- 172.21.15.3:0/2760234006 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85cc011830 con 0x7f85e009e000 2023-10-03T10:18:36.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.627+0000 7f85df7fe700 1 -- 172.21.15.3:0/2760234006 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85cc011990 con 0x7f85e009e000 2023-10-03T10:18:36.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.627+0000 7f85df7fe700 1 -- 172.21.15.3:0/2760234006 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85cc011d20 con 0x7f85e009e000 2023-10-03T10:18:36.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.628+0000 7f85e6b6c700 1 -- 172.21.15.3:0/2760234006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85e009e000 msgr2=0x7f85e01339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:36.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.628+0000 7f85e6b6c700 1 --2- 172.21.15.3:0/2760234006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85e009e000 0x7f85e01339f0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f85cc009840 tx=0x7f85cc009b50 comp rx=0 tx=0).stop 2023-10-03T10:18:36.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.629+0000 7f85e6b6c700 1 -- 172.21.15.3:0/2760234006 shutdown_connections 2023-10-03T10:18:36.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.629+0000 7f85e6b6c700 1 --2- 172.21.15.3:0/2760234006 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85e009e410 0x7f85e0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.629+0000 7f85e6b6c700 1 --2- 172.21.15.3:0/2760234006 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85e009d6f0 0x7f85e009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.629+0000 7f85e6b6c700 1 --2- 172.21.15.3:0/2760234006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85e009e000 0x7f85e01339f0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.629+0000 7f85e6b6c700 1 -- 172.21.15.3:0/2760234006 >> 172.21.15.3:0/2760234006 conn(0x7f85e0066f50 msgr2=0x7f85e009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:36.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.629+0000 7f85e6b6c700 1 -- 172.21.15.3:0/2760234006 shutdown_connections 2023-10-03T10:18:36.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.629+0000 7f85e6b6c700 1 -- 172.21.15.3:0/2760234006 wait complete. 2023-10-03T10:18:36.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.630+0000 7f85e6b6c700 1 Processor -- start 2023-10-03T10:18:36.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.630+0000 7f85e6b6c700 1 -- start start 2023-10-03T10:18:36.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.631+0000 7f85e6b6c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85e009d6f0 0x7f85e01c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:36.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.631+0000 7f85e6b6c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85e009e000 0x7f85e01c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:36.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.631+0000 7f85e6b6c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85e009e410 0x7f85e01c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:36.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.631+0000 7f85e6b6c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e01a9620 con 0x7f85e009e410 2023-10-03T10:18:36.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.631+0000 7f85e6b6c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e01c5aa0 con 0x7f85e009e000 2023-10-03T10:18:36.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.631+0000 7f85e6b6c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85e01a7910 con 0x7f85e009d6f0 2023-10-03T10:18:36.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.631+0000 7f85e4908700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85e009d6f0 0x7f85e01c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:36.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.631+0000 7f85e4908700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85e009d6f0 0x7f85e01c5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42332/0 (socket says 172.21.15.3:42332) 2023-10-03T10:18:36.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.631+0000 7f85e4908700 1 -- 172.21.15.3:0/1940649507 learned_addr learned my addr 172.21.15.3:0/1940649507 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:36.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.632+0000 7f85e4908700 1 -- 172.21.15.3:0/1940649507 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85e009e000 msgr2=0x7f85e01c5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:36.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.632+0000 7f85dffff700 1 --2- 172.21.15.3:0/1940649507 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85e009e000 0x7f85e01c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:36.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.632+0000 7f85e4908700 1 --2- 172.21.15.3:0/1940649507 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85e009e000 0x7f85e01c5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.632+0000 7f85e4908700 1 -- 172.21.15.3:0/1940649507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85e009e410 msgr2=0x7f85e01c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:36.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.632+0000 7f85e4908700 1 --2- 172.21.15.3:0/1940649507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85e009e410 0x7f85e01c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.632+0000 7f85e4908700 1 -- 172.21.15.3:0/1940649507 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d4009550 con 0x7f85e009d6f0 2023-10-03T10:18:36.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.632+0000 7f85dffff700 1 --2- 172.21.15.3:0/1940649507 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85e009e000 0x7f85e01c5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:18:36.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.632+0000 7f85e4908700 1 --2- 172.21.15.3:0/1940649507 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85e009d6f0 0x7f85e01c5020 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009ea0 tx=0x7f85d400eec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:36.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.632+0000 7f85dd7fa700 1 -- 172.21.15.3:0/1940649507 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d4017d80 con 0x7f85e009d6f0 2023-10-03T10:18:36.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.633+0000 7f85e6b6c700 1 -- 172.21.15.3:0/1940649507 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85cc0094a0 con 0x7f85e009d6f0 2023-10-03T10:18:36.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.633+0000 7f85dd7fa700 1 -- 172.21.15.3:0/1940649507 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d400c540 con 0x7f85e009d6f0 2023-10-03T10:18:36.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.633+0000 7f85dd7fa700 1 -- 172.21.15.3:0/1940649507 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d400c8b0 con 0x7f85e009d6f0 2023-10-03T10:18:36.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.633+0000 7f85e6b6c700 1 -- 172.21.15.3:0/1940649507 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e01a8b60 con 0x7f85e009d6f0 2023-10-03T10:18:36.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.634+0000 7f85dd7fa700 1 -- 172.21.15.3:0/1940649507 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f85d400ca10 con 0x7f85e009d6f0 2023-10-03T10:18:36.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.634+0000 7f85dd7fa700 1 -- 172.21.15.3:0/1940649507 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f85d4049ba0 con 0x7f85e009d6f0 2023-10-03T10:18:36.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.634+0000 7f85e6b6c700 1 -- 172.21.15.3:0/1940649507 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e0074ea0 con 0x7f85e009d6f0 2023-10-03T10:18:36.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.643+0000 7f85dd7fa700 1 -- 172.21.15.3:0/1940649507 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f85d4024070 con 0x7f85e009d6f0 2023-10-03T10:18:36.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.668+0000 7f9ed33b3700 1 Processor -- start 2023-10-03T10:18:36.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.668+0000 7f9ed33b3700 1 -- start start 2023-10-03T10:18:36.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.669+0000 7f9ed33b3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9ecc0a0a30 0x7f9ecc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:36.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.669+0000 7f9ed33b3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ecc09d630 0x7f9ecc0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:36.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.669+0000 7f9ed33b3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ecc0a2a70 con 0x7f9ecc133930 2023-10-03T10:18:36.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.669+0000 7f9ed33b3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ecc0a01d0 con 0x7f9ecc0a0a30 2023-10-03T10:18:36.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.669+0000 7f9ed33b3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc09e2b0 con 0x7f9ecc09d630 2023-10-03T10:18:36.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.669+0000 7f9ed094e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ecc09d630 0x7f9ecc0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:36.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.669+0000 7f9ed094e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ecc09d630 0x7f9ecc0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42336/0 (socket says 172.21.15.3:42336) 2023-10-03T10:18:36.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.669+0000 7f9ed094e700 1 -- 172.21.15.3:0/3072460649 learned_addr learned my addr 172.21.15.3:0/3072460649 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:36.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.669+0000 7f9ed114f700 1 --2- 172.21.15.3:0/3072460649 >> v2:172.21.15.3:3300/0 conn(0x7f9ecc0a0a30 0x7f9ecc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:36.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.669+0000 7f9ed094e700 1 -- 172.21.15.3:0/3072460649 >> v2:172.21.15.3:3300/0 conn(0x7f9ecc0a0a30 msgr2=0x7f9ecc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:36.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.669+0000 7f9ed094e700 1 --2- 172.21.15.3:0/3072460649 >> v2:172.21.15.3:3300/0 conn(0x7f9ecc0a0a30 0x7f9ecc0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.669+0000 7f9ed094e700 1 -- 172.21.15.3:0/3072460649 >> v1:172.21.15.3:6789/0 conn(0x7f9ecc133930 legacy=0x7f9ecc133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:36.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.669+0000 7f9ed094e700 1 -- 172.21.15.3:0/3072460649 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ecc09dc50 con 0x7f9ecc09d630 2023-10-03T10:18:36.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.670+0000 7f9ed094e700 1 --2- 172.21.15.3:0/3072460649 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ecc09d630 0x7f9ecc0a1340 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0098f0 tx=0x7f9ebc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af52bcd3848d90aa server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:36.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.670+0000 7f9ec3fff700 1 -- 172.21.15.3:0/3072460649 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc004230 con 0x7f9ecc09d630 2023-10-03T10:18:36.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.670+0000 7f9ec3fff700 1 -- 172.21.15.3:0/3072460649 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ebc004390 con 0x7f9ecc09d630 2023-10-03T10:18:36.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.670+0000 7f9ec3fff700 1 -- 172.21.15.3:0/3072460649 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc0036a0 con 0x7f9ecc09d630 2023-10-03T10:18:36.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.671+0000 7f9ed33b3700 1 -- 172.21.15.3:0/3072460649 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ecc09d630 msgr2=0x7f9ecc0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:36.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.671+0000 7f9ed33b3700 1 --2- 172.21.15.3:0/3072460649 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ecc09d630 0x7f9ecc0a1340 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0098f0 tx=0x7f9ebc009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:36.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.671+0000 7f9ed33b3700 1 -- 172.21.15.3:0/3072460649 shutdown_connections 2023-10-03T10:18:36.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.671+0000 7f9ed33b3700 1 --2- 172.21.15.3:0/3072460649 >> v2:172.21.15.3:3300/0 conn(0x7f9ecc0a0a30 0x7f9ecc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.671+0000 7f9ed33b3700 1 --2- 172.21.15.3:0/3072460649 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ecc09d630 0x7f9ecc0a1340 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.671+0000 7f9ed33b3700 1 -- 172.21.15.3:0/3072460649 >> 172.21.15.3:0/3072460649 conn(0x7f9ecc0a3050 msgr2=0x7f9ecc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:36.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.671+0000 7f9ed33b3700 1 -- 172.21.15.3:0/3072460649 shutdown_connections 2023-10-03T10:18:36.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.671+0000 7f9ed33b3700 1 -- 172.21.15.3:0/3072460649 wait complete. 2023-10-03T10:18:36.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.672+0000 7f9ed33b3700 1 Processor -- start 2023-10-03T10:18:36.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.672+0000 7f9ed33b3700 1 -- start start 2023-10-03T10:18:36.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.673+0000 7f9ed33b3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ecc09d630 0x7f9ecc1c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:36.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.673+0000 7f9ed33b3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ecc0a0a30 0x7f9ecc1c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:36.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.673+0000 7f9ed33b3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ecc133930 0x7f9ecc1c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:36.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.673+0000 7f9ed33b3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc120710 con 0x7f9ecc0a0a30 2023-10-03T10:18:36.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.673+0000 7f9ed33b3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc122110 con 0x7f9ecc133930 2023-10-03T10:18:36.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.673+0000 7f9ed33b3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc121230 con 0x7f9ecc09d630 2023-10-03T10:18:36.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.673+0000 7f9ed094e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ecc0a0a30 0x7f9ecc1c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:36.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.673+0000 7f9ed094e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ecc0a0a30 0x7f9ecc1c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60326/0 (socket says 172.21.15.3:60326) 2023-10-03T10:18:36.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.673+0000 7f9ed094e700 1 -- 172.21.15.3:0/2283102980 learned_addr learned my addr 172.21.15.3:0/2283102980 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:36.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.673+0000 7f9ed114f700 1 --2- 172.21.15.3:0/2283102980 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ecc09d630 0x7f9ecc1c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:36.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.673+0000 7f9ed094e700 1 -- 172.21.15.3:0/2283102980 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ecc09d630 msgr2=0x7f9ecc1c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:36.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.673+0000 7f9ed094e700 1 --2- 172.21.15.3:0/2283102980 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ecc09d630 0x7f9ecc1c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.673+0000 7f9ed094e700 1 -- 172.21.15.3:0/2283102980 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ecc133930 msgr2=0x7f9ecc1c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:36.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.673+0000 7f9ed094e700 1 --2- 172.21.15.3:0/2283102980 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ecc133930 0x7f9ecc1c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.673+0000 7f9ed094e700 1 -- 172.21.15.3:0/2283102980 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ebc009550 con 0x7f9ecc0a0a30 2023-10-03T10:18:36.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.673+0000 7f9ed094e700 1 --2- 172.21.15.3:0/2283102980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ecc0a0a30 0x7f9ecc1c86b0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc09e3f0 tx=0x7f9ebc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:36.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.674+0000 7f9ec1ffb700 1 -- 172.21.15.3:0/2283102980 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc003d60 con 0x7f9ecc0a0a30 2023-10-03T10:18:36.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.674+0000 7f9ec1ffb700 1 -- 172.21.15.3:0/2283102980 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ebc00b660 con 0x7f9ecc0a0a30 2023-10-03T10:18:36.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.674+0000 7f9ed33b3700 1 -- 172.21.15.3:0/2283102980 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ecc1204e0 con 0x7f9ecc0a0a30 2023-10-03T10:18:36.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.674+0000 7f9ec1ffb700 1 -- 172.21.15.3:0/2283102980 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc00ba10 con 0x7f9ecc0a0a30 2023-10-03T10:18:36.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.674+0000 7f9ed33b3700 1 -- 172.21.15.3:0/2283102980 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ecc133380 con 0x7f9ecc0a0a30 2023-10-03T10:18:36.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.675+0000 7f9ec1ffb700 1 -- 172.21.15.3:0/2283102980 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9ebc021070 con 0x7f9ecc0a0a30 2023-10-03T10:18:36.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.675+0000 7f9ec1ffb700 1 -- 172.21.15.3:0/2283102980 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9ebc040f10 con 0x7f9ecc0a0a30 2023-10-03T10:18:36.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.675+0000 7f9ed33b3700 1 -- 172.21.15.3:0/2283102980 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ecc0747f0 con 0x7f9ecc0a0a30 2023-10-03T10:18:36.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.680+0000 7f9ec1ffb700 1 -- 172.21.15.3:0/2283102980 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9ebc0193c0 con 0x7f9ecc0a0a30 2023-10-03T10:18:36.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.730+0000 7fb58b169700 1 Processor -- start 2023-10-03T10:18:36.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.730+0000 7fb58b169700 1 -- start start 2023-10-03T10:18:36.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.731+0000 7fb58b169700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb58409d6f0 0x7fb58409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:36.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.731+0000 7fb58b169700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb58409e000 0x7fb5841339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:36.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.731+0000 7fb58b169700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb58409e410 0x7fb584133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:36.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.731+0000 7fb58b169700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb58406b460 con 0x7fb58409e410 2023-10-03T10:18:36.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.731+0000 7fb58b169700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb58409b120 con 0x7fb58409d6f0 2023-10-03T10:18:36.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.731+0000 7fb58b169700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb58409a7e0 con 0x7fb58409e000 2023-10-03T10:18:36.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.731+0000 7fb588f05700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb58409d6f0 0x7fb58409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:36.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.731+0000 7fb588f05700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb58409d6f0 0x7fb58409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50284/0 (socket says 172.21.15.3:50284) 2023-10-03T10:18:36.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.731+0000 7fb588f05700 1 -- 172.21.15.3:0/1167756721 learned_addr learned my addr 172.21.15.3:0/1167756721 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:36.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.731+0000 7fb583fff700 1 --2- 172.21.15.3:0/1167756721 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb58409e000 0x7fb5841339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:36.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.731+0000 7fb589706700 1 --2- 172.21.15.3:0/1167756721 >> v2:172.21.15.3:3300/0 conn(0x7fb58409e410 0x7fb584133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:36.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.732+0000 7fb588f05700 1 -- 172.21.15.3:0/1167756721 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb58409e000 msgr2=0x7fb5841339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:36.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.732+0000 7fb588f05700 1 --2- 172.21.15.3:0/1167756721 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb58409e000 0x7fb5841339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.732+0000 7fb588f05700 1 -- 172.21.15.3:0/1167756721 >> v2:172.21.15.3:3300/0 conn(0x7fb58409e410 msgr2=0x7fb584133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.732+0000 7fb588f05700 1 --2- 172.21.15.3:0/1167756721 >> v2:172.21.15.3:3300/0 conn(0x7fb58409e410 0x7fb584133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.732+0000 7fb588f05700 1 -- 172.21.15.3:0/1167756721 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58409a500 con 0x7fb58409d6f0 2023-10-03T10:18:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.732+0000 7fb589706700 1 --2- 172.21.15.3:0/1167756721 >> v2:172.21.15.3:3300/0 conn(0x7fb58409e410 0x7fb584133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.732+0000 7fb588f05700 1 --2- 172.21.15.3:0/1167756721 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb58409d6f0 0x7fb58409dac0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fb578009920 tx=0x7fb578009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19b3f98406b5082c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.732+0000 7fb5837fe700 1 -- 172.21.15.3:0/1167756721 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb578011830 con 0x7fb58409d6f0 2023-10-03T10:18:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.732+0000 7fb5837fe700 1 -- 172.21.15.3:0/1167756721 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb578011990 con 0x7fb58409d6f0 2023-10-03T10:18:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.732+0000 7fb5837fe700 1 -- 172.21.15.3:0/1167756721 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb578011d20 con 0x7fb58409d6f0 2023-10-03T10:18:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.733+0000 7fb58b169700 1 -- 172.21.15.3:0/1167756721 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb58409d6f0 msgr2=0x7fb58409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.733+0000 7fb58b169700 1 --2- 172.21.15.3:0/1167756721 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb58409d6f0 0x7fb58409dac0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fb578009920 tx=0x7fb578009c30 comp rx=0 tx=0).stop 2023-10-03T10:18:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.733+0000 7fb58b169700 1 -- 172.21.15.3:0/1167756721 shutdown_connections 2023-10-03T10:18:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.733+0000 7fb58b169700 1 --2- 172.21.15.3:0/1167756721 >> v2:172.21.15.3:3300/0 conn(0x7fb58409e410 0x7fb584133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.733+0000 7fb58b169700 1 --2- 172.21.15.3:0/1167756721 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb58409d6f0 0x7fb58409dac0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.733+0000 7fb58b169700 1 --2- 172.21.15.3:0/1167756721 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb58409e000 0x7fb5841339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.733+0000 7fb58b169700 1 -- 172.21.15.3:0/1167756721 >> 172.21.15.3:0/1167756721 conn(0x7fb584066f50 msgr2=0x7fb58409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:36.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.733+0000 7fb58b169700 1 -- 172.21.15.3:0/1167756721 shutdown_connections 2023-10-03T10:18:36.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.733+0000 7fb58b169700 1 -- 172.21.15.3:0/1167756721 wait complete. 2023-10-03T10:18:36.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.734+0000 7fb58b169700 1 Processor -- start 2023-10-03T10:18:36.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.734+0000 7fb58b169700 1 -- start start 2023-10-03T10:18:36.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.735+0000 7fb58b169700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb58409d6f0 0x7fb5841a6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:36.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.735+0000 7fb58b169700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb58409e000 0x7fb5841a6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:36.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.735+0000 7fb58b169700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb58409e410 0x7fb5841a6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:36.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.735+0000 7fb58b169700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5841a9a60 con 0x7fb58409d6f0 2023-10-03T10:18:36.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.735+0000 7fb58b169700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5841a8130 con 0x7fb58409e410 2023-10-03T10:18:36.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.735+0000 7fb58b169700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb5841a91f0 con 0x7fb58409e000 2023-10-03T10:18:36.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.735+0000 7fb588f05700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb58409d6f0 0x7fb5841a6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:36.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.735+0000 7fb588f05700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb58409d6f0 0x7fb5841a6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60348/0 (socket says 172.21.15.3:60348) 2023-10-03T10:18:36.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.735+0000 7fb588f05700 1 -- 172.21.15.3:0/4091836564 learned_addr learned my addr 172.21.15.3:0/4091836564 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:36.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.735+0000 7fb589706700 1 --2- 172.21.15.3:0/4091836564 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb58409e410 0x7fb5841a6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:36.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.735+0000 7fb583fff700 1 --2- 172.21.15.3:0/4091836564 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb58409e000 0x7fb5841a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:36.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.735+0000 7fb588f05700 1 -- 172.21.15.3:0/4091836564 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb58409e000 msgr2=0x7fb5841a6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:36.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.735+0000 7fb588f05700 1 --2- 172.21.15.3:0/4091836564 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb58409e000 0x7fb5841a6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.735+0000 7fb588f05700 1 -- 172.21.15.3:0/4091836564 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb58409e410 msgr2=0x7fb5841a6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:36.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.735+0000 7fb588f05700 1 --2- 172.21.15.3:0/4091836564 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb58409e410 0x7fb5841a6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.735+0000 7fb588f05700 1 -- 172.21.15.3:0/4091836564 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb57c009a70 con 0x7fb58409d6f0 2023-10-03T10:18:36.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.736+0000 7fb588f05700 1 --2- 172.21.15.3:0/4091836564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb58409d6f0 0x7fb5841a6020 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fb578000c00 tx=0x7fb578004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:36.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.736+0000 7fb5817fa700 1 -- 172.21.15.3:0/4091836564 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb578004240 con 0x7fb58409d6f0 2023-10-03T10:18:36.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.736+0000 7fb5817fa700 1 -- 172.21.15.3:0/4091836564 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57801f070 con 0x7fb58409d6f0 2023-10-03T10:18:36.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.736+0000 7fb58b169700 1 -- 172.21.15.3:0/4091836564 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb578009550 con 0x7fb58409d6f0 2023-10-03T10:18:36.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.736+0000 7fb5817fa700 1 -- 172.21.15.3:0/4091836564 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5780036a0 con 0x7fb58409d6f0 2023-10-03T10:18:36.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.736+0000 7fb58b169700 1 -- 172.21.15.3:0/4091836564 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5841a7b50 con 0x7fb58409d6f0 2023-10-03T10:18:36.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.737+0000 7fb5817fa700 1 -- 172.21.15.3:0/4091836564 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb578021030 con 0x7fb58409d6f0 2023-10-03T10:18:36.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.737+0000 7fb5817fa700 1 -- 172.21.15.3:0/4091836564 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb578040700 con 0x7fb58409d6f0 2023-10-03T10:18:36.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.737+0000 7fb58b169700 1 -- 172.21.15.3:0/4091836564 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb584074ea0 con 0x7fb58409d6f0 2023-10-03T10:18:36.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.742+0000 7fb5817fa700 1 -- 172.21.15.3:0/4091836564 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb578015080 con 0x7fb58409d6f0 2023-10-03T10:18:36.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.756+0000 7f31a9135700 1 Processor -- start 2023-10-03T10:18:36.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.756+0000 7f31a9135700 1 -- start start 2023-10-03T10:18:36.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.756+0000 7f31a9135700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f31a409e1d0 0x7f31a4135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:36.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.756+0000 7f31a9135700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31a4135e50 0x7f31a4138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:36.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.756+0000 7f31a9135700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a409f130 con 0x7f31a409d6f0 2023-10-03T10:18:36.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.756+0000 7f31a9135700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a409f520 con 0x7f31a409e1d0 2023-10-03T10:18:36.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.756+0000 7f31a9135700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a409e930 con 0x7f31a4135e50 2023-10-03T10:18:36.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.756+0000 7f31a259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f31a409e1d0 0x7f31a4135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:36.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.757+0000 7f31a2d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f31a409d6f0 0x7f31a409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43672/0 (socket says 172.21.15.3:43672) 2023-10-03T10:18:36.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.757+0000 7f31a259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f31a409e1d0 0x7f31a4135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60350/0 (socket says 172.21.15.3:60350) 2023-10-03T10:18:36.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.757+0000 7f31a2d9d700 1 -- 172.21.15.3:0/2875111632 learned_addr learned my addr 172.21.15.3:0/2875111632 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:36.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.757+0000 7f31a359e700 1 --2- 172.21.15.3:0/2875111632 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31a4135e50 0x7f31a4138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:36.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.757+0000 7f31a259c700 1 -- 172.21.15.3:0/2875111632 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31a4135e50 msgr2=0x7f31a4138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:36.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.757+0000 7f31a259c700 1 --2- 172.21.15.3:0/2875111632 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31a4135e50 0x7f31a4138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.757+0000 7f31a259c700 1 -- 172.21.15.3:0/2875111632 >> v1:172.21.15.3:6789/0 conn(0x7f31a409d6f0 legacy=0x7f31a409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:36.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.757+0000 7f31a259c700 1 -- 172.21.15.3:0/2875111632 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a409b2d0 con 0x7f31a409e1d0 2023-10-03T10:18:36.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.757+0000 7f31a259c700 1 --2- 172.21.15.3:0/2875111632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31a409e1d0 0x7f31a4135910 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f3198009840 tx=0x7f3198009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3d870979e5b593 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:36.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.757+0000 7f31a1d9b700 1 -- 172.21.15.3:0/2875111632 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3198011830 con 0x7f31a409e1d0 2023-10-03T10:18:36.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.757+0000 7f31a1d9b700 1 -- 172.21.15.3:0/2875111632 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3198011990 con 0x7f31a409e1d0 2023-10-03T10:18:36.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.757+0000 7f31a1d9b700 1 -- 172.21.15.3:0/2875111632 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3198011d20 con 0x7f31a409e1d0 2023-10-03T10:18:36.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.758+0000 7f31a9135700 1 -- 172.21.15.3:0/2875111632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31a409e1d0 msgr2=0x7f31a4135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:36.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.758+0000 7f31a9135700 1 --2- 172.21.15.3:0/2875111632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31a409e1d0 0x7f31a4135910 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f3198009840 tx=0x7f3198009b50 comp rx=0 tx=0).stop 2023-10-03T10:18:36.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.758+0000 7f31a9135700 1 -- 172.21.15.3:0/2875111632 shutdown_connections 2023-10-03T10:18:36.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.758+0000 7f31a9135700 1 --2- 172.21.15.3:0/2875111632 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31a4135e50 0x7f31a4138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.758+0000 7f31a9135700 1 --2- 172.21.15.3:0/2875111632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31a409e1d0 0x7f31a4135910 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.758+0000 7f31a9135700 1 -- 172.21.15.3:0/2875111632 >> 172.21.15.3:0/2875111632 conn(0x7f31a4066f50 msgr2=0x7f31a409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:36.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.758+0000 7f31a9135700 1 -- 172.21.15.3:0/2875111632 shutdown_connections 2023-10-03T10:18:36.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.759+0000 7f31a9135700 1 -- 172.21.15.3:0/2875111632 wait complete. 2023-10-03T10:18:36.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.760+0000 7f31a9135700 1 Processor -- start 2023-10-03T10:18:36.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.760+0000 7f31a9135700 1 -- start start 2023-10-03T10:18:36.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.761+0000 7f31a9135700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31a409d6f0 0x7f31a4121c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:36.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.761+0000 7f31a9135700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31a409e1d0 0x7f31a41221d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:36.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.761+0000 7f31a9135700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31a4135e50 0x7f31a4122710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:36.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.761+0000 7f31a9135700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a4125150 con 0x7f31a4135e50 2023-10-03T10:18:36.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.761+0000 7f31a9135700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a4072180 con 0x7f31a409d6f0 2023-10-03T10:18:36.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.761+0000 7f31a9135700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31a41244b0 con 0x7f31a409e1d0 2023-10-03T10:18:36.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.761+0000 7f31a259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31a409e1d0 0x7f31a41221d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:36.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.761+0000 7f31a2d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31a409d6f0 0x7f31a4121c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:36.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.761+0000 7f31a259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31a409e1d0 0x7f31a41221d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42392/0 (socket says 172.21.15.3:42392) 2023-10-03T10:18:36.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.761+0000 7f31a259c700 1 -- 172.21.15.3:0/3672128257 learned_addr learned my addr 172.21.15.3:0/3672128257 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:36.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.761+0000 7f31a359e700 1 --2- 172.21.15.3:0/3672128257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31a4135e50 0x7f31a4122710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:36.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.762+0000 7f31a2d9d700 1 -- 172.21.15.3:0/3672128257 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31a409e1d0 msgr2=0x7f31a41221d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:36.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.762+0000 7f31a2d9d700 1 --2- 172.21.15.3:0/3672128257 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31a409e1d0 0x7f31a41221d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.762+0000 7f31a2d9d700 1 -- 172.21.15.3:0/3672128257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31a4135e50 msgr2=0x7f31a4122710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:36.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.762+0000 7f31a2d9d700 1 --2- 172.21.15.3:0/3672128257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31a4135e50 0x7f31a4122710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.762+0000 7f31a2d9d700 1 -- 172.21.15.3:0/3672128257 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31980094a0 con 0x7f31a409d6f0 2023-10-03T10:18:36.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.762+0000 7f31a359e700 1 --2- 172.21.15.3:0/3672128257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31a4135e50 0x7f31a4122710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:36.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.762+0000 7f31a2d9d700 1 --2- 172.21.15.3:0/3672128257 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31a409d6f0 0x7f31a4121c90 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f319400c3f0 tx=0x7f319400c700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:36.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.762+0000 7f318b7fe700 1 -- 172.21.15.3:0/3672128257 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3194005b60 con 0x7f31a409d6f0 2023-10-03T10:18:36.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.762+0000 7f318b7fe700 1 -- 172.21.15.3:0/3672128257 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3194005cc0 con 0x7f31a409d6f0 2023-10-03T10:18:36.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.762+0000 7f31a9135700 1 -- 172.21.15.3:0/3672128257 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a40a1d00 con 0x7f31a409d6f0 2023-10-03T10:18:36.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.762+0000 7f318b7fe700 1 -- 172.21.15.3:0/3672128257 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31940072d0 con 0x7f31a409d6f0 2023-10-03T10:18:36.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.762+0000 7f31a9135700 1 -- 172.21.15.3:0/3672128257 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a4124790 con 0x7f31a409d6f0 2023-10-03T10:18:36.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.764+0000 7f31a9135700 1 -- 172.21.15.3:0/3672128257 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a4074ea0 con 0x7f31a409d6f0 2023-10-03T10:18:36.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.765+0000 7f318b7fe700 1 -- 172.21.15.3:0/3672128257 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3194007430 con 0x7f31a409d6f0 2023-10-03T10:18:36.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.765+0000 7f318b7fe700 1 -- 172.21.15.3:0/3672128257 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f319403e970 con 0x7f31a409d6f0 2023-10-03T10:18:36.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.771+0000 7f318b7fe700 1 -- 172.21.15.3:0/3672128257 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f31940076e0 con 0x7f31a409d6f0 2023-10-03T10:18:36.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.894+0000 7f85e6b6c700 1 -- 172.21.15.3:0/1940649507 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f85e01a98e0 con 0x7f85e009d6f0 2023-10-03T10:18:36.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.894+0000 7f85dd7fa700 1 -- 172.21.15.3:0/1940649507 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f85d4024030 con 0x7f85e009d6f0 2023-10-03T10:18:36.895 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:36.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.897+0000 7f85e6b6c700 1 -- 172.21.15.3:0/1940649507 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85e009d6f0 msgr2=0x7f85e01c5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:36.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.897+0000 7f85e6b6c700 1 --2- 172.21.15.3:0/1940649507 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85e009d6f0 0x7f85e01c5020 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009ea0 tx=0x7f85d400eec0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.897+0000 7f85e6b6c700 1 -- 172.21.15.3:0/1940649507 shutdown_connections 2023-10-03T10:18:36.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.897+0000 7f85e6b6c700 1 --2- 172.21.15.3:0/1940649507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85e009e410 0x7f85e01c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.897+0000 7f85e6b6c700 1 --2- 172.21.15.3:0/1940649507 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85e009d6f0 0x7f85e01c5020 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.897+0000 7f85e6b6c700 1 --2- 172.21.15.3:0/1940649507 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85e009e000 0x7f85e01c5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.897+0000 7f85e6b6c700 1 -- 172.21.15.3:0/1940649507 >> 172.21.15.3:0/1940649507 conn(0x7f85e0066f50 msgr2=0x7f85e01badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:36.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.897+0000 7f85e6b6c700 1 -- 172.21.15.3:0/1940649507 shutdown_connections 2023-10-03T10:18:36.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.897+0000 7f85e6b6c700 1 -- 172.21.15.3:0/1940649507 wait complete. 2023-10-03T10:18:36.912 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:36.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.933+0000 7f9ed33b3700 1 -- 172.21.15.3:0/2283102980 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9ecc072980 con 0x7f9ecc0a0a30 2023-10-03T10:18:36.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.933+0000 7f9ec1ffb700 1 -- 172.21.15.3:0/2283102980 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9ebc01abc0 con 0x7f9ecc0a0a30 2023-10-03T10:18:36.933 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:36.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.937+0000 7f9ed33b3700 1 -- 172.21.15.3:0/2283102980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ecc0a0a30 msgr2=0x7f9ecc1c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:36.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.937+0000 7f9ed33b3700 1 --2- 172.21.15.3:0/2283102980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ecc0a0a30 0x7f9ecc1c86b0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc09e3f0 tx=0x7f9ebc004750 comp rx=0 tx=0).stop 2023-10-03T10:18:36.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.937+0000 7f9ed33b3700 1 -- 172.21.15.3:0/2283102980 shutdown_connections 2023-10-03T10:18:36.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.937+0000 7f9ed33b3700 1 --2- 172.21.15.3:0/2283102980 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ecc133930 0x7f9ecc1c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.937+0000 7f9ed33b3700 1 --2- 172.21.15.3:0/2283102980 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ecc09d630 0x7f9ecc1c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.937+0000 7f9ed33b3700 1 --2- 172.21.15.3:0/2283102980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ecc0a0a30 0x7f9ecc1c86b0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.937+0000 7f9ed33b3700 1 -- 172.21.15.3:0/2283102980 >> 172.21.15.3:0/2283102980 conn(0x7f9ecc0a3050 msgr2=0x7f9ecc09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:36.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.937+0000 7f9ed33b3700 1 -- 172.21.15.3:0/2283102980 shutdown_connections 2023-10-03T10:18:36.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.937+0000 7f9ed33b3700 1 -- 172.21.15.3:0/2283102980 wait complete. 2023-10-03T10:18:36.955 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:36.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.958+0000 7fb58b169700 1 -- 172.21.15.3:0/4091836564 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb5841a8da0 con 0x7fb58409d6f0 2023-10-03T10:18:36.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.958+0000 7fb5817fa700 1 -- 172.21.15.3:0/4091836564 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb5780239c0 con 0x7fb58409d6f0 2023-10-03T10:18:36.959 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:36.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.961+0000 7fb58b169700 1 -- 172.21.15.3:0/4091836564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb58409d6f0 msgr2=0x7fb5841a6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:36.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.961+0000 7fb58b169700 1 --2- 172.21.15.3:0/4091836564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb58409d6f0 0x7fb5841a6020 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fb578000c00 tx=0x7fb578004750 comp rx=0 tx=0).stop 2023-10-03T10:18:36.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.961+0000 7fb58b169700 1 -- 172.21.15.3:0/4091836564 shutdown_connections 2023-10-03T10:18:36.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.961+0000 7fb58b169700 1 --2- 172.21.15.3:0/4091836564 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb58409e410 0x7fb5841a6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.961+0000 7fb58b169700 1 --2- 172.21.15.3:0/4091836564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb58409d6f0 0x7fb5841a6020 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.961+0000 7fb58b169700 1 --2- 172.21.15.3:0/4091836564 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb58409e000 0x7fb5841a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:36.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.961+0000 7fb58b169700 1 -- 172.21.15.3:0/4091836564 >> 172.21.15.3:0/4091836564 conn(0x7fb584066f50 msgr2=0x7fb5841bafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:36.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.961+0000 7fb58b169700 1 -- 172.21.15.3:0/4091836564 shutdown_connections 2023-10-03T10:18:36.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:36.961+0000 7fb58b169700 1 -- 172.21.15.3:0/4091836564 wait complete. 2023-10-03T10:18:36.974 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:37.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:37.020+0000 7f31a9135700 1 -- 172.21.15.3:0/3672128257 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f31a4123580 con 0x7f31a409d6f0 2023-10-03T10:18:37.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:37.021+0000 7f318b7fe700 1 -- 172.21.15.3:0/3672128257 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3194017020 con 0x7f31a409d6f0 2023-10-03T10:18:37.021 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:37.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:37.023+0000 7f31a9135700 1 -- 172.21.15.3:0/3672128257 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31a409d6f0 msgr2=0x7f31a4121c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:37.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:37.023+0000 7f31a9135700 1 --2- 172.21.15.3:0/3672128257 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31a409d6f0 0x7f31a4121c90 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f319400c3f0 tx=0x7f319400c700 comp rx=0 tx=0).stop 2023-10-03T10:18:37.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:37.023+0000 7f31a9135700 1 -- 172.21.15.3:0/3672128257 shutdown_connections 2023-10-03T10:18:37.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:37.023+0000 7f31a9135700 1 --2- 172.21.15.3:0/3672128257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31a4135e50 0x7f31a4122710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:37.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:37.023+0000 7f31a9135700 1 --2- 172.21.15.3:0/3672128257 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31a409d6f0 0x7f31a4121c90 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:37.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:37.023+0000 7f31a9135700 1 --2- 172.21.15.3:0/3672128257 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f31a409e1d0 0x7f31a41221d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:37.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:37.023+0000 7f31a9135700 1 -- 172.21.15.3:0/3672128257 >> 172.21.15.3:0/3672128257 conn(0x7f31a4066f50 msgr2=0x7f31a406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:37.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:37.023+0000 7f31a9135700 1 -- 172.21.15.3:0/3672128257 shutdown_connections 2023-10-03T10:18:37.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:37.023+0000 7f31a9135700 1 -- 172.21.15.3:0/3672128257 wait complete. 2023-10-03T10:18:37.037 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:37.914 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:37.956 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:37.976 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:38.038 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:38.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.069+0000 7fc0d66d6700 1 Processor -- start 2023-10-03T10:18:38.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.070+0000 7fc0d66d6700 1 -- start start 2023-10-03T10:18:38.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.070+0000 7fc0d66d6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc0d0133510 0x7fc0d01358f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:38.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.070+0000 7fc0d66d6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0d0135e30 0x7fc0d0138210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:38.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.070+0000 7fc0d66d6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0d0130260 con 0x7fc0d0130670 2023-10-03T10:18:38.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.070+0000 7fc0d66d6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d009a350 con 0x7fc0d0133510 2023-10-03T10:18:38.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.070+0000 7fc0d66d6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d0071ff0 con 0x7fc0d0135e30 2023-10-03T10:18:38.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.070+0000 7fc0cf7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc0d0133510 0x7fc0d01358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:38.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.070+0000 7fc0cf7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc0d0133510 0x7fc0d01358f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50330/0 (socket says 172.21.15.3:50330) 2023-10-03T10:18:38.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.070+0000 7fc0cf7fe700 1 -- 172.21.15.3:0/1509738057 learned_addr learned my addr 172.21.15.3:0/1509738057 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:38.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.071+0000 7fc0cf7fe700 1 -- 172.21.15.3:0/1509738057 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0d0135e30 msgr2=0x7fc0d0138210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:38.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.071+0000 7fc0cf7fe700 1 --2- 172.21.15.3:0/1509738057 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0d0135e30 0x7fc0d0138210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.071+0000 7fc0cf7fe700 1 -- 172.21.15.3:0/1509738057 >> v1:172.21.15.3:6789/0 conn(0x7fc0d0130670 legacy=0x7fc0d0130a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:38.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.071+0000 7fc0cf7fe700 1 -- 172.21.15.3:0/1509738057 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d009ecf0 con 0x7fc0d0133510 2023-10-03T10:18:38.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.071+0000 7fc0cf7fe700 1 --2- 172.21.15.3:0/1509738057 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc0d0133510 0x7fc0d01358f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4009920 tx=0x7fc0c4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e804e55312c7831f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:38.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.071+0000 7fc0ceffd700 1 -- 172.21.15.3:0/1509738057 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c400ba60 con 0x7fc0d0133510 2023-10-03T10:18:38.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.071+0000 7fc0ceffd700 1 -- 172.21.15.3:0/1509738057 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c400bbc0 con 0x7fc0d0133510 2023-10-03T10:18:38.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.071+0000 7fc0ceffd700 1 -- 172.21.15.3:0/1509738057 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4009e70 con 0x7fc0d0133510 2023-10-03T10:18:38.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.072+0000 7fc0d66d6700 1 -- 172.21.15.3:0/1509738057 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc0d0133510 msgr2=0x7fc0d01358f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:38.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.072+0000 7fc0d66d6700 1 --2- 172.21.15.3:0/1509738057 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc0d0133510 0x7fc0d01358f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4009920 tx=0x7fc0c4009c30 comp rx=0 tx=0).stop 2023-10-03T10:18:38.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.073+0000 7fc0d66d6700 1 -- 172.21.15.3:0/1509738057 shutdown_connections 2023-10-03T10:18:38.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.073+0000 7fc0d66d6700 1 --2- 172.21.15.3:0/1509738057 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0d0135e30 0x7fc0d0138210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.073+0000 7fc0d66d6700 1 --2- 172.21.15.3:0/1509738057 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc0d0133510 0x7fc0d01358f0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.073+0000 7fc0d66d6700 1 -- 172.21.15.3:0/1509738057 >> 172.21.15.3:0/1509738057 conn(0x7fc0d00a2aa0 msgr2=0x7fc0d00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:38.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.073+0000 7fc0d66d6700 1 -- 172.21.15.3:0/1509738057 shutdown_connections 2023-10-03T10:18:38.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.073+0000 7fc0d66d6700 1 -- 172.21.15.3:0/1509738057 wait complete. 2023-10-03T10:18:38.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.074+0000 7fc0d66d6700 1 Processor -- start 2023-10-03T10:18:38.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.074+0000 7fc0d66d6700 1 -- start start 2023-10-03T10:18:38.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.074+0000 7fc0d66d6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc0d0130670 0x7fc0d009a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:38.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.074+0000 7fc0d66d6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0d0133510 0x7fc0d009aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:38.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.074+0000 7fc0d66d6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0d0135e30 0x7fc0d009afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:38.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.074+0000 7fc0d66d6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d009cf00 con 0x7fc0d0135e30 2023-10-03T10:18:38.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.074+0000 7fc0d66d6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d01392c0 con 0x7fc0d0130670 2023-10-03T10:18:38.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.074+0000 7fc0d66d6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d0071820 con 0x7fc0d0133510 2023-10-03T10:18:38.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.075+0000 7fc0cffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc0d0130670 0x7fc0d009a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:38.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.075+0000 7fc0cffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc0d0130670 0x7fc0d009a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50338/0 (socket says 172.21.15.3:50338) 2023-10-03T10:18:38.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.075+0000 7fc0cffff700 1 -- 172.21.15.3:0/979159142 learned_addr learned my addr 172.21.15.3:0/979159142 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.075+0000 7fc0d4c73700 1 --2- 172.21.15.3:0/979159142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0d0135e30 0x7fc0d009afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.075+0000 7fc0cf7fe700 1 --2- 172.21.15.3:0/979159142 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0d0133510 0x7fc0d009aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.075+0000 7fc0cffff700 1 -- 172.21.15.3:0/979159142 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0d0133510 msgr2=0x7fc0d009aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.075+0000 7fc0cffff700 1 --2- 172.21.15.3:0/979159142 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0d0133510 0x7fc0d009aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.075+0000 7fc0cffff700 1 -- 172.21.15.3:0/979159142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0d0135e30 msgr2=0x7fc0d009afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.075+0000 7fc0cffff700 1 --2- 172.21.15.3:0/979159142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0d0135e30 0x7fc0d009afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.075+0000 7fc0cffff700 1 -- 172.21.15.3:0/979159142 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c4009550 con 0x7fc0d0130670 2023-10-03T10:18:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.075+0000 7fc0cf7fe700 1 --2- 172.21.15.3:0/979159142 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0d0133510 0x7fc0d009aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.075+0000 7fc0d4c73700 1 --2- 172.21.15.3:0/979159142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0d0135e30 0x7fc0d009afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.075+0000 7fc0cffff700 1 --2- 172.21.15.3:0/979159142 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc0d0130670 0x7fc0d009a550 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000c490 tx=0x7fc0c000c7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:38.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.076+0000 7fc0ccff9700 1 -- 172.21.15.3:0/979159142 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c000cea0 con 0x7fc0d0130670 2023-10-03T10:18:38.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.076+0000 7fc0ccff9700 1 -- 172.21.15.3:0/979159142 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c00052f0 con 0x7fc0d0130670 2023-10-03T10:18:38.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.076+0000 7fc0d66d6700 1 -- 172.21.15.3:0/979159142 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d009c550 con 0x7fc0d0130670 2023-10-03T10:18:38.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.076+0000 7fc0ccff9700 1 -- 172.21.15.3:0/979159142 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0006860 con 0x7fc0d0130670 2023-10-03T10:18:38.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.076+0000 7fc0d66d6700 1 -- 172.21.15.3:0/979159142 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d009dae0 con 0x7fc0d0130670 2023-10-03T10:18:38.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.077+0000 7fc0d66d6700 1 -- 172.21.15.3:0/979159142 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0d009c8c0 con 0x7fc0d0130670 2023-10-03T10:18:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.077+0000 7fc0ccff9700 1 -- 172.21.15.3:0/979159142 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc0c0003680 con 0x7fc0d0130670 2023-10-03T10:18:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.077+0000 7fc0ccff9700 1 -- 172.21.15.3:0/979159142 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc0c003e9f0 con 0x7fc0d0130670 2023-10-03T10:18:38.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.083+0000 7fc0ccff9700 1 -- 172.21.15.3:0/979159142 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc0c0010b10 con 0x7fc0d0130670 2023-10-03T10:18:38.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.109+0000 7f3ad584d700 1 Processor -- start 2023-10-03T10:18:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.109+0000 7f3ad584d700 1 -- start start 2023-10-03T10:18:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.109+0000 7f3ad584d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ad012cf60 0x7f3ad006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.109+0000 7f3ad584d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3ad012f8b0 0x7f3ad012fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.109+0000 7f3ad584d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ad0130260 0x7f3ad0136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.109+0000 7f3ad584d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ad012d330 con 0x7f3ad012f8b0 2023-10-03T10:18:38.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.110+0000 7f3ad584d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad012cd90 con 0x7f3ad0130260 2023-10-03T10:18:38.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.110+0000 7f3ad584d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad01306d0 con 0x7f3ad012cf60 2023-10-03T10:18:38.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.110+0000 7f3ace7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3ad012f8b0 0x7f3ad012fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:38.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.110+0000 7f3ace7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3ad012f8b0 0x7f3ad012fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60374/0 (socket says 172.21.15.3:60374) 2023-10-03T10:18:38.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.110+0000 7f3ace7fc700 1 -- 172.21.15.3:0/1691291178 learned_addr learned my addr 172.21.15.3:0/1691291178 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:38.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.110+0000 7f3aceffd700 1 --2- 172.21.15.3:0/1691291178 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ad012cf60 0x7f3ad006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.110+0000 7f3aceffd700 1 -- 172.21.15.3:0/1691291178 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ad0130260 msgr2=0x7f3ad0136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.110+0000 7f3aceffd700 1 --2- 172.21.15.3:0/1691291178 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ad0130260 0x7f3ad0136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.110+0000 7f3aceffd700 1 -- 172.21.15.3:0/1691291178 >> v2:172.21.15.3:3300/0 conn(0x7f3ad012f8b0 msgr2=0x7f3ad012fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.110+0000 7f3aceffd700 1 --2- 172.21.15.3:0/1691291178 >> v2:172.21.15.3:3300/0 conn(0x7f3ad012f8b0 0x7f3ad012fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.110+0000 7f3aceffd700 1 -- 172.21.15.3:0/1691291178 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad0071780 con 0x7f3ad012cf60 2023-10-03T10:18:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.110+0000 7f3ace7fc700 1 --2- 172.21.15.3:0/1691291178 >> v2:172.21.15.3:3300/0 conn(0x7f3ad012f8b0 0x7f3ad012fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.111+0000 7f3aceffd700 1 --2- 172.21.15.3:0/1691291178 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ad012cf60 0x7f3ad006f5d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0009870 tx=0x7f3ac0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faeacd04aecca5f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.111+0000 7f3acdffb700 1 -- 172.21.15.3:0/1691291178 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0012b80 con 0x7f3ad012cf60 2023-10-03T10:18:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.111+0000 7f3acdffb700 1 -- 172.21.15.3:0/1691291178 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac0012ce0 con 0x7f3ad012cf60 2023-10-03T10:18:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.111+0000 7f3acdffb700 1 -- 172.21.15.3:0/1691291178 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac001b620 con 0x7f3ad012cf60 2023-10-03T10:18:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.112+0000 7f3ad584d700 1 -- 172.21.15.3:0/1691291178 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ad012cf60 msgr2=0x7f3ad006f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.112+0000 7f3ad584d700 1 --2- 172.21.15.3:0/1691291178 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ad012cf60 0x7f3ad006f5d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0009870 tx=0x7f3ac0009b80 comp rx=0 tx=0).stop 2023-10-03T10:18:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.112+0000 7f3ad584d700 1 -- 172.21.15.3:0/1691291178 shutdown_connections 2023-10-03T10:18:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.112+0000 7f3ad584d700 1 --2- 172.21.15.3:0/1691291178 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ad0130260 0x7f3ad0136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.112+0000 7f3ad584d700 1 --2- 172.21.15.3:0/1691291178 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ad012cf60 0x7f3ad006f5d0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.112+0000 7f3ad584d700 1 --2- 172.21.15.3:0/1691291178 >> v2:172.21.15.3:3300/0 conn(0x7f3ad012f8b0 0x7f3ad012fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.112+0000 7f3ad584d700 1 -- 172.21.15.3:0/1691291178 >> 172.21.15.3:0/1691291178 conn(0x7f3ad00a2aa0 msgr2=0x7f3ad00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.112+0000 7f3ad584d700 1 -- 172.21.15.3:0/1691291178 shutdown_connections 2023-10-03T10:18:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.112+0000 7f3ad584d700 1 -- 172.21.15.3:0/1691291178 wait complete. 2023-10-03T10:18:38.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.113+0000 7f3ad584d700 1 Processor -- start 2023-10-03T10:18:38.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.113+0000 7f3ad584d700 1 -- start start 2023-10-03T10:18:38.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.114+0000 7f3ad584d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ad012cf60 0x7f3ad01c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:38.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.114+0000 7f3ad584d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ad012f8b0 0x7f3ad01c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:38.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.114+0000 7f3ad584d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ad0130260 0x7f3ad01c4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:38.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.114+0000 7f3ad584d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad009ec50 con 0x7f3ad012cf60 2023-10-03T10:18:38.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.114+0000 7f3ad584d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad0129b90 con 0x7f3ad0130260 2023-10-03T10:18:38.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.114+0000 7f3ad584d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad009cec0 con 0x7f3ad012f8b0 2023-10-03T10:18:38.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.114+0000 7f3ace7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ad012f8b0 0x7f3ad01c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:38.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.114+0000 7f3aceffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ad012cf60 0x7f3ad01c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:38.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.114+0000 7f3ace7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ad012f8b0 0x7f3ad01c46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42432/0 (socket says 172.21.15.3:42432) 2023-10-03T10:18:38.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.114+0000 7f3ace7fc700 1 -- 172.21.15.3:0/2457209761 learned_addr learned my addr 172.21.15.3:0/2457209761 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:38.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.114+0000 7f3acf7fe700 1 --2- 172.21.15.3:0/2457209761 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ad0130260 0x7f3ad01c4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:38.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.115+0000 7f3aceffd700 1 -- 172.21.15.3:0/2457209761 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ad012f8b0 msgr2=0x7f3ad01c46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:38.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.115+0000 7f3aceffd700 1 --2- 172.21.15.3:0/2457209761 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ad012f8b0 0x7f3ad01c46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.115+0000 7f3aceffd700 1 -- 172.21.15.3:0/2457209761 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ad0130260 msgr2=0x7f3ad01c4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.115+0000 7f3aceffd700 1 --2- 172.21.15.3:0/2457209761 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ad0130260 0x7f3ad01c4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.115+0000 7f3aceffd700 1 -- 172.21.15.3:0/2457209761 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac400a6f0 con 0x7f3ad012cf60 2023-10-03T10:18:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.115+0000 7f3aceffd700 1 --2- 172.21.15.3:0/2457209761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ad012cf60 0x7f3ad01c41a0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f3ac001a000 tx=0x7f3ac0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.115+0000 7f3ab7fff700 1 -- 172.21.15.3:0/2457209761 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0020070 con 0x7f3ad012cf60 2023-10-03T10:18:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.115+0000 7f3ab7fff700 1 -- 172.21.15.3:0/2457209761 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac00036a0 con 0x7f3ad012cf60 2023-10-03T10:18:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.115+0000 7f3ad584d700 1 -- 172.21.15.3:0/2457209761 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac00094a0 con 0x7f3ad012cf60 2023-10-03T10:18:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.115+0000 7f3ad584d700 1 -- 172.21.15.3:0/2457209761 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad009e6b0 con 0x7f3ad012cf60 2023-10-03T10:18:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.115+0000 7f3ab7fff700 1 -- 172.21.15.3:0/2457209761 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0003a30 con 0x7f3ad012cf60 2023-10-03T10:18:38.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.116+0000 7f3ad584d700 1 -- 172.21.15.3:0/2457209761 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab00052e0 con 0x7f3ad012cf60 2023-10-03T10:18:38.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.117+0000 7f3ab7fff700 1 -- 172.21.15.3:0/2457209761 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3ac0024070 con 0x7f3ad012cf60 2023-10-03T10:18:38.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.117+0000 7f3ab7fff700 1 -- 172.21.15.3:0/2457209761 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3ac0043ae0 con 0x7f3ad012cf60 2023-10-03T10:18:38.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.122+0000 7f3ab7fff700 1 -- 172.21.15.3:0/2457209761 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3ac001b400 con 0x7f3ad012cf60 2023-10-03T10:18:38.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.160+0000 7f22b2b58700 1 Processor -- start 2023-10-03T10:18:38.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.160+0000 7f22b2b58700 1 -- start start 2023-10-03T10:18:38.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.161+0000 7f22b2b58700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f22ac09d6f0 0x7f22ac09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:38.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.161+0000 7f22b2b58700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f22ac09e000 0x7f22ac1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:38.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.161+0000 7f22b2b58700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22ac09b120 con 0x7f22ac09e410 2023-10-03T10:18:38.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.161+0000 7f22b2b58700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac06b460 con 0x7f22ac09d6f0 2023-10-03T10:18:38.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.161+0000 7f22b2b58700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac09a7e0 con 0x7f22ac09e000 2023-10-03T10:18:38.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.161+0000 7f22b08f4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f22ac09d6f0 0x7f22ac09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:38.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.161+0000 7f22b08f4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f22ac09d6f0 0x7f22ac09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50370/0 (socket says 172.21.15.3:50370) 2023-10-03T10:18:38.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.161+0000 7f22abfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f22ac09e000 0x7f22ac1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:38.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.161+0000 7f22b08f4700 1 -- 172.21.15.3:0/3751040989 learned_addr learned my addr 172.21.15.3:0/3751040989 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:38.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.161+0000 7f22b08f4700 1 -- 172.21.15.3:0/3751040989 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f22ac09e000 msgr2=0x7f22ac1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:38.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.161+0000 7f22ab7fe700 1 -- 172.21.15.3:0/3751040989 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 296645850 0 0) 0x7f22ac09b120 con 0x7f22ac09e410 2023-10-03T10:18:38.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.161+0000 7f22b08f4700 1 --2- 172.21.15.3:0/3751040989 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f22ac09e000 0x7f22ac1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.161+0000 7f22b08f4700 1 -- 172.21.15.3:0/3751040989 >> v1:172.21.15.3:6789/0 conn(0x7f22ac09e410 legacy=0x7f22ac133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:38.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.161+0000 7f22b08f4700 1 -- 172.21.15.3:0/3751040989 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ac09a500 con 0x7f22ac09d6f0 2023-10-03T10:18:38.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.162+0000 7f22b08f4700 1 --2- 172.21.15.3:0/3751040989 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f22ac09d6f0 0x7f22ac09dac0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f22a0009920 tx=0x7f22a0009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79e045cafe8cd2f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:38.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.162+0000 7f22ab7fe700 1 -- 172.21.15.3:0/3751040989 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a0011830 con 0x7f22ac09d6f0 2023-10-03T10:18:38.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.162+0000 7f22ab7fe700 1 -- 172.21.15.3:0/3751040989 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a0011990 con 0x7f22ac09d6f0 2023-10-03T10:18:38.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.162+0000 7f22ab7fe700 1 -- 172.21.15.3:0/3751040989 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a0011d20 con 0x7f22ac09d6f0 2023-10-03T10:18:38.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.163+0000 7f22b2b58700 1 -- 172.21.15.3:0/3751040989 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f22ac09d6f0 msgr2=0x7f22ac09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:38.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.163+0000 7f22b2b58700 1 --2- 172.21.15.3:0/3751040989 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f22ac09d6f0 0x7f22ac09dac0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f22a0009920 tx=0x7f22a0009c30 comp rx=0 tx=0).stop 2023-10-03T10:18:38.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.163+0000 7f22b2b58700 1 -- 172.21.15.3:0/3751040989 shutdown_connections 2023-10-03T10:18:38.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.163+0000 7f22b2b58700 1 --2- 172.21.15.3:0/3751040989 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f22ac09d6f0 0x7f22ac09dac0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.163+0000 7f22b2b58700 1 --2- 172.21.15.3:0/3751040989 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f22ac09e000 0x7f22ac1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.163+0000 7f22b2b58700 1 -- 172.21.15.3:0/3751040989 >> 172.21.15.3:0/3751040989 conn(0x7f22ac066f50 msgr2=0x7f22ac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:38.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.163+0000 7f22b2b58700 1 -- 172.21.15.3:0/3751040989 shutdown_connections 2023-10-03T10:18:38.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.163+0000 7f22b2b58700 1 -- 172.21.15.3:0/3751040989 wait complete. 2023-10-03T10:18:38.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.165+0000 7f22b2b58700 1 Processor -- start 2023-10-03T10:18:38.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.165+0000 7f22b2b58700 1 -- start start 2023-10-03T10:18:38.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.166+0000 7f22b2b58700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22ac09d6f0 0x7f22ac1222a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:38.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.166+0000 7f22b2b58700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f22ac09e000 0x7f22ac1227e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:38.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.166+0000 7f22b2b58700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f22ac09e410 0x7f22ac122d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:38.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.166+0000 7f22b2b58700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac1258b0 con 0x7f22ac09d6f0 2023-10-03T10:18:38.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.166+0000 7f22b2b58700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac124d40 con 0x7f22ac09e000 2023-10-03T10:18:38.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.166+0000 7f22b2b58700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac09c710 con 0x7f22ac09e410 2023-10-03T10:18:38.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.166+0000 7f22abfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f22ac09e000 0x7f22ac1227e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:38.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.166+0000 7f22b08f4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22ac09d6f0 0x7f22ac1222a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:38.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.166+0000 7f22abfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f22ac09e000 0x7f22ac1227e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50372/0 (socket says 172.21.15.3:50372) 2023-10-03T10:18:38.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.166+0000 7f22b08f4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22ac09d6f0 0x7f22ac1222a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60392/0 (socket says 172.21.15.3:60392) 2023-10-03T10:18:38.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.166+0000 7f22abfff700 1 -- 172.21.15.3:0/3851093428 learned_addr learned my addr 172.21.15.3:0/3851093428 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:38.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.166+0000 7f22b10f5700 1 --2- 172.21.15.3:0/3851093428 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f22ac09e410 0x7f22ac122d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:38.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.166+0000 7f22b10f5700 1 -- 172.21.15.3:0/3851093428 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f22ac09e000 msgr2=0x7f22ac1227e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:38.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.166+0000 7f22b10f5700 1 --2- 172.21.15.3:0/3851093428 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f22ac09e000 0x7f22ac1227e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.166+0000 7f22b10f5700 1 -- 172.21.15.3:0/3851093428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22ac09d6f0 msgr2=0x7f22ac1222a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:38.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.166+0000 7f22b10f5700 1 --2- 172.21.15.3:0/3851093428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22ac09d6f0 0x7f22ac1222a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.166+0000 7f22b10f5700 1 -- 172.21.15.3:0/3851093428 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a0009550 con 0x7f22ac09e410 2023-10-03T10:18:38.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.166+0000 7f22b08f4700 1 --2- 172.21.15.3:0/3851093428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22ac09d6f0 0x7f22ac1222a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:38.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.167+0000 7f22b10f5700 1 --2- 172.21.15.3:0/3851093428 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f22ac09e410 0x7f22ac122d20 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f22ac09e8b0 tx=0x7f22a400caa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:38.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.167+0000 7f22a97fa700 1 -- 172.21.15.3:0/3851093428 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a4003e30 con 0x7f22ac09e410 2023-10-03T10:18:38.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.167+0000 7f22a97fa700 1 -- 172.21.15.3:0/3851093428 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a4005070 con 0x7f22ac09e410 2023-10-03T10:18:38.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.167+0000 7f22a97fa700 1 -- 172.21.15.3:0/3851093428 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a4005420 con 0x7f22ac09e410 2023-10-03T10:18:38.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.167+0000 7f22b2b58700 1 -- 172.21.15.3:0/3851093428 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ac125550 con 0x7f22ac09e410 2023-10-03T10:18:38.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.167+0000 7f22b2b58700 1 -- 172.21.15.3:0/3851093428 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ac133670 con 0x7f22ac09e410 2023-10-03T10:18:38.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.168+0000 7f22a97fa700 1 -- 172.21.15.3:0/3851093428 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f22a4005580 con 0x7f22ac09e410 2023-10-03T10:18:38.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.168+0000 7f22a97fa700 1 -- 172.21.15.3:0/3851093428 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f22a403ec40 con 0x7f22ac09e410 2023-10-03T10:18:38.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.168+0000 7f22b2b58700 1 -- 172.21.15.3:0/3851093428 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22ac074ea0 con 0x7f22ac09e410 2023-10-03T10:18:38.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.174+0000 7f22a97fa700 1 -- 172.21.15.3:0/3851093428 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f22a4017080 con 0x7f22ac09e410 2023-10-03T10:18:38.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.196+0000 7f94c2007700 1 Processor -- start 2023-10-03T10:18:38.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.196+0000 7f94c2007700 1 -- start start 2023-10-03T10:18:38.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.197+0000 7f94c2007700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f94bc09de00 0x7f94bc09e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:38.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.197+0000 7f94c2007700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94bc13d1f0 0x7f94bc13f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:38.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.197+0000 7f94c2007700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94bc09a810 con 0x7f94bc06a770 2023-10-03T10:18:38.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.197+0000 7f94c2007700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94bc06ab40 con 0x7f94bc09de00 2023-10-03T10:18:38.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.197+0000 7f94c2007700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc09ebc0 con 0x7f94bc13d1f0 2023-10-03T10:18:38.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.197+0000 7f94baffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f94bc09de00 0x7f94bc09e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:38.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.197+0000 7f94baffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f94bc09de00 0x7f94bc09e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60408/0 (socket says 172.21.15.3:60408) 2023-10-03T10:18:38.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.197+0000 7f94baffd700 1 -- 172.21.15.3:0/1084638978 learned_addr learned my addr 172.21.15.3:0/1084638978 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:38.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.197+0000 7f94bbfff700 1 --2- 172.21.15.3:0/1084638978 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94bc13d1f0 0x7f94bc13f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:38.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.197+0000 7f94baffd700 1 -- 172.21.15.3:0/1084638978 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94bc13d1f0 msgr2=0x7f94bc13f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:38.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.197+0000 7f94baffd700 1 --2- 172.21.15.3:0/1084638978 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94bc13d1f0 0x7f94bc13f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.197+0000 7f94baffd700 1 -- 172.21.15.3:0/1084638978 >> v1:172.21.15.3:6789/0 conn(0x7f94bc06a770 legacy=0x7f94bc09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:38.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.197+0000 7f94baffd700 1 -- 172.21.15.3:0/1084638978 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94bc09e920 con 0x7f94bc09de00 2023-10-03T10:18:38.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.198+0000 7f94baffd700 1 --2- 172.21.15.3:0/1084638978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94bc09de00 0x7f94bc09e250 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f94b00098f0 tx=0x7f94b0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d54e7618cf37f13 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:38.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.198+0000 7f94ba7fc700 1 -- 172.21.15.3:0/1084638978 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b0011830 con 0x7f94bc09de00 2023-10-03T10:18:38.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.198+0000 7f94ba7fc700 1 -- 172.21.15.3:0/1084638978 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b0011990 con 0x7f94bc09de00 2023-10-03T10:18:38.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.198+0000 7f94ba7fc700 1 -- 172.21.15.3:0/1084638978 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b0011d00 con 0x7f94bc09de00 2023-10-03T10:18:38.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.199+0000 7f94c2007700 1 -- 172.21.15.3:0/1084638978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94bc09de00 msgr2=0x7f94bc09e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:38.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.199+0000 7f94c2007700 1 --2- 172.21.15.3:0/1084638978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94bc09de00 0x7f94bc09e250 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f94b00098f0 tx=0x7f94b0009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:38.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.199+0000 7f94c2007700 1 -- 172.21.15.3:0/1084638978 shutdown_connections 2023-10-03T10:18:38.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.199+0000 7f94c2007700 1 --2- 172.21.15.3:0/1084638978 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94bc13d1f0 0x7f94bc13f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.199+0000 7f94c2007700 1 --2- 172.21.15.3:0/1084638978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94bc09de00 0x7f94bc09e250 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.199+0000 7f94c2007700 1 -- 172.21.15.3:0/1084638978 >> 172.21.15.3:0/1084638978 conn(0x7f94bc066f50 msgr2=0x7f94bc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:38.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.199+0000 7f94c2007700 1 -- 172.21.15.3:0/1084638978 shutdown_connections 2023-10-03T10:18:38.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.199+0000 7f94c2007700 1 -- 172.21.15.3:0/1084638978 wait complete. 2023-10-03T10:18:38.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.201+0000 7f94c2007700 1 Processor -- start 2023-10-03T10:18:38.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.201+0000 7f94c2007700 1 -- start start 2023-10-03T10:18:38.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.202+0000 7f94c2007700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94bc06a770 0x7f94bc1260a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:38.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.202+0000 7f94c2007700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94bc09de00 0x7f94bc1265e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:38.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.202+0000 7f94c2007700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94bc13d1f0 0x7f94bc126b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:38.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.202+0000 7f94c2007700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc128680 con 0x7f94bc13d1f0 2023-10-03T10:18:38.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.202+0000 7f94c2007700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc072260 con 0x7f94bc09de00 2023-10-03T10:18:38.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.202+0000 7f94c2007700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc128880 con 0x7f94bc06a770 2023-10-03T10:18:38.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.202+0000 7f94bbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94bc13d1f0 0x7f94bc126b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:38.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.202+0000 7f94bb7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94bc06a770 0x7f94bc1260a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:38.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.202+0000 7f94bbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94bc13d1f0 0x7f94bc126b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60410/0 (socket says 172.21.15.3:60410) 2023-10-03T10:18:38.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.202+0000 7f94bbfff700 1 -- 172.21.15.3:0/3092112894 learned_addr learned my addr 172.21.15.3:0/3092112894 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:38.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.202+0000 7f94bbfff700 1 -- 172.21.15.3:0/3092112894 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94bc06a770 msgr2=0x7f94bc1260a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:38.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.202+0000 7f94bbfff700 1 --2- 172.21.15.3:0/3092112894 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94bc06a770 0x7f94bc1260a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.202+0000 7f94bbfff700 1 -- 172.21.15.3:0/3092112894 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94bc09de00 msgr2=0x7f94bc1265e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:38.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.202+0000 7f94bbfff700 1 --2- 172.21.15.3:0/3092112894 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94bc09de00 0x7f94bc1265e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.202+0000 7f94bbfff700 1 -- 172.21.15.3:0/3092112894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b0009550 con 0x7f94bc13d1f0 2023-10-03T10:18:38.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.203+0000 7f94bbfff700 1 --2- 172.21.15.3:0/3092112894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94bc13d1f0 0x7f94bc126b20 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f94b400efc0 tx=0x7f94b400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:38.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.203+0000 7f94a3fff700 1 -- 172.21.15.3:0/3092112894 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b400cd40 con 0x7f94bc13d1f0 2023-10-03T10:18:38.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.203+0000 7f94c2007700 1 -- 172.21.15.3:0/3092112894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94bc1275e0 con 0x7f94bc13d1f0 2023-10-03T10:18:38.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.203+0000 7f94c2007700 1 -- 172.21.15.3:0/3092112894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94bc127b40 con 0x7f94bc13d1f0 2023-10-03T10:18:38.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.203+0000 7f94a3fff700 1 -- 172.21.15.3:0/3092112894 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b400f040 con 0x7f94bc13d1f0 2023-10-03T10:18:38.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.203+0000 7f94a3fff700 1 -- 172.21.15.3:0/3092112894 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b40146c0 con 0x7f94bc13d1f0 2023-10-03T10:18:38.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.204+0000 7f94c2007700 1 -- 172.21.15.3:0/3092112894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f949c0052e0 con 0x7f94bc13d1f0 2023-10-03T10:18:38.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.204+0000 7f94a3fff700 1 -- 172.21.15.3:0/3092112894 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f94b4014820 con 0x7f94bc13d1f0 2023-10-03T10:18:38.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.204+0000 7f94a3fff700 1 -- 172.21.15.3:0/3092112894 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f94b4040380 con 0x7f94bc13d1f0 2023-10-03T10:18:38.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.214+0000 7f94a3fff700 1 -- 172.21.15.3:0/3092112894 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f94b4007920 con 0x7f94bc13d1f0 2023-10-03T10:18:38.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.303+0000 7fc0d66d6700 1 -- 172.21.15.3:0/979159142 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc0d0074ea0 con 0x7fc0d0130670 2023-10-03T10:18:38.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.303+0000 7fc0ccff9700 1 -- 172.21.15.3:0/979159142 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc0c0010700 con 0x7fc0d0130670 2023-10-03T10:18:38.304 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.305+0000 7fc0d66d6700 1 -- 172.21.15.3:0/979159142 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc0d0130670 msgr2=0x7fc0d009a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.305+0000 7fc0d66d6700 1 --2- 172.21.15.3:0/979159142 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc0d0130670 0x7fc0d009a550 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000c490 tx=0x7fc0c000c7a0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.305+0000 7fc0d66d6700 1 -- 172.21.15.3:0/979159142 shutdown_connections 2023-10-03T10:18:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.305+0000 7fc0d66d6700 1 --2- 172.21.15.3:0/979159142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0d0135e30 0x7fc0d009afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.305+0000 7fc0d66d6700 1 --2- 172.21.15.3:0/979159142 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc0d0130670 0x7fc0d009a550 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.305+0000 7fc0d66d6700 1 --2- 172.21.15.3:0/979159142 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc0d0133510 0x7fc0d009aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.305+0000 7fc0d66d6700 1 -- 172.21.15.3:0/979159142 >> 172.21.15.3:0/979159142 conn(0x7fc0d00a2aa0 msgr2=0x7fc0d012e130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.306+0000 7fc0d66d6700 1 -- 172.21.15.3:0/979159142 shutdown_connections 2023-10-03T10:18:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.306+0000 7fc0d66d6700 1 -- 172.21.15.3:0/979159142 wait complete. 2023-10-03T10:18:38.320 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:38.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.342+0000 7f3ad584d700 1 -- 172.21.15.3:0/2457209761 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3ab0005150 con 0x7f3ad012cf60 2023-10-03T10:18:38.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.343+0000 7f3ab7fff700 1 -- 172.21.15.3:0/2457209761 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3ac002b030 con 0x7f3ad012cf60 2023-10-03T10:18:38.343 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:38.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.345+0000 7f3ad584d700 1 -- 172.21.15.3:0/2457209761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ad012cf60 msgr2=0x7f3ad01c41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:38.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.345+0000 7f3ad584d700 1 --2- 172.21.15.3:0/2457209761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ad012cf60 0x7f3ad01c41a0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f3ac001a000 tx=0x7f3ac0004750 comp rx=0 tx=0).stop 2023-10-03T10:18:38.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.345+0000 7f3ad584d700 1 -- 172.21.15.3:0/2457209761 shutdown_connections 2023-10-03T10:18:38.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.345+0000 7f3ad584d700 1 --2- 172.21.15.3:0/2457209761 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ad0130260 0x7f3ad01c4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.345+0000 7f3ad584d700 1 --2- 172.21.15.3:0/2457209761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ad012cf60 0x7f3ad01c41a0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.345+0000 7f3ad584d700 1 --2- 172.21.15.3:0/2457209761 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ad012f8b0 0x7f3ad01c46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.345+0000 7f3ad584d700 1 -- 172.21.15.3:0/2457209761 >> 172.21.15.3:0/2457209761 conn(0x7f3ad00a2aa0 msgr2=0x7f3ad006e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:38.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.345+0000 7f3ad584d700 1 -- 172.21.15.3:0/2457209761 shutdown_connections 2023-10-03T10:18:38.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.345+0000 7f3ad584d700 1 -- 172.21.15.3:0/2457209761 wait complete. 2023-10-03T10:18:38.359 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:38.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.389+0000 7f22b2b58700 1 -- 172.21.15.3:0/3851093428 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f22ac06c360 con 0x7f22ac09e410 2023-10-03T10:18:38.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.389+0000 7f22a97fa700 1 -- 172.21.15.3:0/3851093428 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f22a4005830 con 0x7f22ac09e410 2023-10-03T10:18:38.390 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:38.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.392+0000 7f22b2b58700 1 -- 172.21.15.3:0/3851093428 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f22ac09e410 msgr2=0x7f22ac122d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:38.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.392+0000 7f22b2b58700 1 --2- 172.21.15.3:0/3851093428 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f22ac09e410 0x7f22ac122d20 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f22ac09e8b0 tx=0x7f22a400caa0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.392+0000 7f22b2b58700 1 -- 172.21.15.3:0/3851093428 shutdown_connections 2023-10-03T10:18:38.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.392+0000 7f22b2b58700 1 --2- 172.21.15.3:0/3851093428 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f22ac09e410 0x7f22ac122d20 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.392+0000 7f22b2b58700 1 --2- 172.21.15.3:0/3851093428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22ac09d6f0 0x7f22ac1222a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.392+0000 7f22b2b58700 1 --2- 172.21.15.3:0/3851093428 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f22ac09e000 0x7f22ac1227e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.392+0000 7f22b2b58700 1 -- 172.21.15.3:0/3851093428 >> 172.21.15.3:0/3851093428 conn(0x7f22ac066f50 msgr2=0x7f22ac09b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:38.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.392+0000 7f22b2b58700 1 -- 172.21.15.3:0/3851093428 shutdown_connections 2023-10-03T10:18:38.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.392+0000 7f22b2b58700 1 -- 172.21.15.3:0/3851093428 wait complete. 2023-10-03T10:18:38.406 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:38.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.424+0000 7f94c2007700 1 -- 172.21.15.3:0/3092112894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f949c005590 con 0x7f94bc13d1f0 2023-10-03T10:18:38.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.424+0000 7f94a3fff700 1 -- 172.21.15.3:0/3092112894 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f94b4007330 con 0x7f94bc13d1f0 2023-10-03T10:18:38.424 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:38.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.426+0000 7f94c2007700 1 -- 172.21.15.3:0/3092112894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94bc13d1f0 msgr2=0x7f94bc126b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:38.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.426+0000 7f94c2007700 1 --2- 172.21.15.3:0/3092112894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94bc13d1f0 0x7f94bc126b20 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f94b400efc0 tx=0x7f94b400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.426+0000 7f94c2007700 1 -- 172.21.15.3:0/3092112894 shutdown_connections 2023-10-03T10:18:38.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.426+0000 7f94c2007700 1 --2- 172.21.15.3:0/3092112894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94bc13d1f0 0x7f94bc126b20 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.426+0000 7f94c2007700 1 --2- 172.21.15.3:0/3092112894 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94bc06a770 0x7f94bc1260a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.426+0000 7f94c2007700 1 --2- 172.21.15.3:0/3092112894 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94bc09de00 0x7f94bc1265e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:38.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.426+0000 7f94c2007700 1 -- 172.21.15.3:0/3092112894 >> 172.21.15.3:0/3092112894 conn(0x7f94bc066f50 msgr2=0x7f94bc0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:38.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.427+0000 7f94c2007700 1 -- 172.21.15.3:0/3092112894 shutdown_connections 2023-10-03T10:18:38.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:38.427+0000 7f94c2007700 1 -- 172.21.15.3:0/3092112894 wait complete. 2023-10-03T10:18:38.440 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:39.321 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:39.360 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:39.406 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:39.441 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:39.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.479+0000 7f849e3fa700 1 Processor -- start 2023-10-03T10:18:39.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.479+0000 7f849e3fa700 1 -- start start 2023-10-03T10:18:39.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.479+0000 7f849e3fa700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f849809e1d0 0x7f8498138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:39.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.479+0000 7f849e3fa700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8498139170 0x7f849813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:39.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.479+0000 7f849e3fa700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f849809b120 con 0x7f849809d6f0 2023-10-03T10:18:39.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.479+0000 7f849e3fa700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f849806b460 con 0x7f8498139170 2023-10-03T10:18:39.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.479+0000 7f849e3fa700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f849809a7e0 con 0x7f849809e1d0 2023-10-03T10:18:39.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.480+0000 7f849c997700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8498139170 0x7f849813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:39.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.480+0000 7f849c997700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8498139170 0x7f849813b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60426/0 (socket says 172.21.15.3:60426) 2023-10-03T10:18:39.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.480+0000 7f849c997700 1 -- 172.21.15.3:0/2926887870 learned_addr learned my addr 172.21.15.3:0/2926887870 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:39.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.480+0000 7f84977fe700 1 --2- 172.21.15.3:0/2926887870 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f849809e1d0 0x7f8498138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:39.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.480+0000 7f849c997700 1 -- 172.21.15.3:0/2926887870 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f849809e1d0 msgr2=0x7f8498138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:39.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.480+0000 7f849c997700 1 --2- 172.21.15.3:0/2926887870 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f849809e1d0 0x7f8498138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.480+0000 7f849c997700 1 -- 172.21.15.3:0/2926887870 >> v1:172.21.15.3:6789/0 conn(0x7f849809d6f0 legacy=0x7f849809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:39.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.480+0000 7f849c997700 1 -- 172.21.15.3:0/2926887870 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f849809a500 con 0x7f8498139170 2023-10-03T10:18:39.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.480+0000 7f849c997700 1 --2- 172.21.15.3:0/2926887870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8498139170 0x7f849813b550 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f8490009e50 tx=0x7f849000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49fa8e63194b2419 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:39.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.481+0000 7f8496ffd700 1 -- 172.21.15.3:0/2926887870 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849000e070 con 0x7f8498139170 2023-10-03T10:18:39.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.481+0000 7f8496ffd700 1 -- 172.21.15.3:0/2926887870 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f849000b9c0 con 0x7f8498139170 2023-10-03T10:18:39.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.481+0000 7f8496ffd700 1 -- 172.21.15.3:0/2926887870 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849000bd50 con 0x7f8498139170 2023-10-03T10:18:39.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.481+0000 7f849e3fa700 1 -- 172.21.15.3:0/2926887870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8498139170 msgr2=0x7f849813b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:39.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.481+0000 7f849e3fa700 1 --2- 172.21.15.3:0/2926887870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8498139170 0x7f849813b550 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f8490009e50 tx=0x7f849000b330 comp rx=0 tx=0).stop 2023-10-03T10:18:39.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.482+0000 7f849e3fa700 1 -- 172.21.15.3:0/2926887870 shutdown_connections 2023-10-03T10:18:39.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.482+0000 7f849e3fa700 1 --2- 172.21.15.3:0/2926887870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8498139170 0x7f849813b550 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.482+0000 7f849e3fa700 1 --2- 172.21.15.3:0/2926887870 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f849809e1d0 0x7f8498138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.482+0000 7f849e3fa700 1 -- 172.21.15.3:0/2926887870 >> 172.21.15.3:0/2926887870 conn(0x7f8498066f50 msgr2=0x7f849809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:39.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.482+0000 7f849e3fa700 1 -- 172.21.15.3:0/2926887870 shutdown_connections 2023-10-03T10:18:39.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.483+0000 7f849e3fa700 1 -- 172.21.15.3:0/2926887870 wait complete. 2023-10-03T10:18:39.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.484+0000 7f849e3fa700 1 Processor -- start 2023-10-03T10:18:39.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.484+0000 7f849e3fa700 1 -- start start 2023-10-03T10:18:39.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.485+0000 7f849e3fa700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f849809d6f0 0x7f849811e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:39.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.485+0000 7f849e3fa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f849809e1d0 0x7f849811eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:39.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.485+0000 7f849e3fa700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8498139170 0x7f849811f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:39.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.485+0000 7f849e3fa700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8498134090 con 0x7f849809e1d0 2023-10-03T10:18:39.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.485+0000 7f849e3fa700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f849809c7a0 con 0x7f849809d6f0 2023-10-03T10:18:39.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.485+0000 7f849e3fa700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8498072040 con 0x7f8498139170 2023-10-03T10:18:39.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.485+0000 7f849c997700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8498139170 0x7f849811f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:39.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.485+0000 7f8497fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f849809d6f0 0x7f849811e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:39.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.485+0000 7f849c997700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8498139170 0x7f849811f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42472/0 (socket says 172.21.15.3:42472) 2023-10-03T10:18:39.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.485+0000 7f849c997700 1 -- 172.21.15.3:0/1039449326 learned_addr learned my addr 172.21.15.3:0/1039449326 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:39.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.485+0000 7f84977fe700 1 --2- 172.21.15.3:0/1039449326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f849809e1d0 0x7f849811eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:39.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.485+0000 7f849c997700 1 -- 172.21.15.3:0/1039449326 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f849809d6f0 msgr2=0x7f849811e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:39.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.485+0000 7f849c997700 1 --2- 172.21.15.3:0/1039449326 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f849809d6f0 0x7f849811e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.485+0000 7f849c997700 1 -- 172.21.15.3:0/1039449326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f849809e1d0 msgr2=0x7f849811eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:39.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.485+0000 7f849c997700 1 --2- 172.21.15.3:0/1039449326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f849809e1d0 0x7f849811eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.485+0000 7f849c997700 1 -- 172.21.15.3:0/1039449326 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8490009b30 con 0x7f8498139170 2023-10-03T10:18:39.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.485+0000 7f8497fff700 1 --2- 172.21.15.3:0/1039449326 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f849809d6f0 0x7f849811e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:39.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.486+0000 7f849c997700 1 --2- 172.21.15.3:0/1039449326 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8498139170 0x7f849811f1e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f849000be60 tx=0x7f8490004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:39.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.486+0000 7f8494ff9700 1 -- 172.21.15.3:0/1039449326 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8490022d40 con 0x7f8498139170 2023-10-03T10:18:39.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.486+0000 7f849e3fa700 1 -- 172.21.15.3:0/1039449326 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8498121a30 con 0x7f8498139170 2023-10-03T10:18:39.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.486+0000 7f8494ff9700 1 -- 172.21.15.3:0/1039449326 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8490026070 con 0x7f8498139170 2023-10-03T10:18:39.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.486+0000 7f8494ff9700 1 -- 172.21.15.3:0/1039449326 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8490022ea0 con 0x7f8498139170 2023-10-03T10:18:39.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.486+0000 7f849e3fa700 1 -- 172.21.15.3:0/1039449326 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8498121fa0 con 0x7f8498139170 2023-10-03T10:18:39.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.487+0000 7f847e7fc700 1 -- 172.21.15.3:0/1039449326 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84700052e0 con 0x7f8498139170 2023-10-03T10:18:39.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.492+0000 7f8494ff9700 1 -- 172.21.15.3:0/1039449326 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8490028030 con 0x7f8498139170 2023-10-03T10:18:39.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.492+0000 7f8494ff9700 1 -- 172.21.15.3:0/1039449326 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f84900492c0 con 0x7f8498139170 2023-10-03T10:18:39.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.497+0000 7f8494ff9700 1 -- 172.21.15.3:0/1039449326 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8490011980 con 0x7f8498139170 2023-10-03T10:18:39.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.518+0000 7f1ade079700 1 Processor -- start 2023-10-03T10:18:39.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.518+0000 7f1ade079700 1 -- start start 2023-10-03T10:18:39.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.519+0000 7f1ade079700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ad806e570 0x7f1ad806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:39.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.519+0000 7f1ade079700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1ad809eca0 0x7f1ad806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:39.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.519+0000 7f1ade079700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ad8134610 0x7f1ad8134a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:39.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.519+0000 7f1ade079700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad8070ee0 con 0x7f1ad809eca0 2023-10-03T10:18:39.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.519+0000 7f1ade079700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad8071480 con 0x7f1ad8134610 2023-10-03T10:18:39.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.519+0000 7f1ade079700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad806f3c0 con 0x7f1ad806e570 2023-10-03T10:18:39.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.519+0000 7f1ad6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1ad809eca0 0x7f1ad806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:39.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.519+0000 7f1ad6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1ad809eca0 0x7f1ad806ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60450/0 (socket says 172.21.15.3:60450) 2023-10-03T10:18:39.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.519+0000 7f1ad6ffd700 1 -- 172.21.15.3:0/4044839527 learned_addr learned my addr 172.21.15.3:0/4044839527 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:39.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.519+0000 7f1ad7fff700 1 --2- 172.21.15.3:0/4044839527 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ad8134610 0x7f1ad8134a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:39.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.519+0000 7f1ad77fe700 1 --2- 172.21.15.3:0/4044839527 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ad806e570 0x7f1ad806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:39.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.519+0000 7f1ad6ffd700 1 -- 172.21.15.3:0/4044839527 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ad806e570 msgr2=0x7f1ad806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:39.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.519+0000 7f1ad6ffd700 1 --2- 172.21.15.3:0/4044839527 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ad806e570 0x7f1ad806e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.519+0000 7f1ad6ffd700 1 -- 172.21.15.3:0/4044839527 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ad8134610 msgr2=0x7f1ad8134a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:39.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.519+0000 7f1ad6ffd700 1 --2- 172.21.15.3:0/4044839527 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ad8134610 0x7f1ad8134a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.519+0000 7f1ad6ffd700 1 -- 172.21.15.3:0/4044839527 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad8071b90 con 0x7f1ad809eca0 2023-10-03T10:18:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.520+0000 7f1ad6ffd700 1 --2- 172.21.15.3:0/4044839527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ad809eca0 0x7f1ad806ee80 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f1acc0098f0 tx=0x7f1acc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7fbff43427a6ffd server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.520+0000 7f1ad67fc700 1 -- 172.21.15.3:0/4044839527 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc011830 con 0x7f1ad809eca0 2023-10-03T10:18:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.520+0000 7f1ad67fc700 1 -- 172.21.15.3:0/4044839527 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1acc011990 con 0x7f1ad809eca0 2023-10-03T10:18:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.520+0000 7f1ad67fc700 1 -- 172.21.15.3:0/4044839527 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc011d20 con 0x7f1ad809eca0 2023-10-03T10:18:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.521+0000 7f1ade079700 1 -- 172.21.15.3:0/4044839527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ad809eca0 msgr2=0x7f1ad806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.521+0000 7f1ade079700 1 --2- 172.21.15.3:0/4044839527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ad809eca0 0x7f1ad806ee80 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f1acc0098f0 tx=0x7f1acc009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.521+0000 7f1ade079700 1 -- 172.21.15.3:0/4044839527 shutdown_connections 2023-10-03T10:18:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.521+0000 7f1ade079700 1 --2- 172.21.15.3:0/4044839527 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ad8134610 0x7f1ad8134a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.521+0000 7f1ade079700 1 --2- 172.21.15.3:0/4044839527 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ad806e570 0x7f1ad806e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.521+0000 7f1ade079700 1 --2- 172.21.15.3:0/4044839527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ad809eca0 0x7f1ad806ee80 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.521+0000 7f1ade079700 1 -- 172.21.15.3:0/4044839527 >> 172.21.15.3:0/4044839527 conn(0x7f1ad80a2aa0 msgr2=0x7f1ad80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:39.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.521+0000 7f1ade079700 1 -- 172.21.15.3:0/4044839527 shutdown_connections 2023-10-03T10:18:39.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.521+0000 7f1ade079700 1 -- 172.21.15.3:0/4044839527 wait complete. 2023-10-03T10:18:39.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.523+0000 7f1ade079700 1 Processor -- start 2023-10-03T10:18:39.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.523+0000 7f1ade079700 1 -- start start 2023-10-03T10:18:39.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.524+0000 7f1ade079700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ad806e570 0x7f1ad81214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:39.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.524+0000 7f1ade079700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ad8134610 0x7f1ad8121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:39.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.524+0000 7f1ade079700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ad809e500 0x7f1ad8121f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:39.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.524+0000 7f1ade079700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad81248f0 con 0x7f1ad8134610 2023-10-03T10:18:39.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.524+0000 7f1ade079700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad80a2080 con 0x7f1ad809e500 2023-10-03T10:18:39.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.524+0000 7f1ade079700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad8124080 con 0x7f1ad806e570 2023-10-03T10:18:39.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.524+0000 7f1ad7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ad809e500 0x7f1ad8121f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:39.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.524+0000 7f1ad7fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ad809e500 0x7f1ad8121f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50414/0 (socket says 172.21.15.3:50414) 2023-10-03T10:18:39.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.524+0000 7f1ad7fff700 1 -- 172.21.15.3:0/1535226705 learned_addr learned my addr 172.21.15.3:0/1535226705 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:39.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.524+0000 7f1ad6ffd700 1 --2- 172.21.15.3:0/1535226705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ad8134610 0x7f1ad8121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:39.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.524+0000 7f1ad77fe700 1 --2- 172.21.15.3:0/1535226705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ad806e570 0x7f1ad81214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:39.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.524+0000 7f1ad6ffd700 1 -- 172.21.15.3:0/1535226705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ad806e570 msgr2=0x7f1ad81214e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:39.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.524+0000 7f1ad6ffd700 1 --2- 172.21.15.3:0/1535226705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ad806e570 0x7f1ad81214e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.524+0000 7f1ad6ffd700 1 -- 172.21.15.3:0/1535226705 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ad809e500 msgr2=0x7f1ad8121f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:39.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.524+0000 7f1ad6ffd700 1 --2- 172.21.15.3:0/1535226705 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ad809e500 0x7f1ad8121f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.524+0000 7f1ad6ffd700 1 -- 172.21.15.3:0/1535226705 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1acc009550 con 0x7f1ad8134610 2023-10-03T10:18:39.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.525+0000 7f1ad6ffd700 1 --2- 172.21.15.3:0/1535226705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ad8134610 0x7f1ad8121a20 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f1acc016000 tx=0x7f1acc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:39.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.525+0000 7f1abffff700 1 -- 172.21.15.3:0/1535226705 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc004450 con 0x7f1ad8134610 2023-10-03T10:18:39.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.525+0000 7f1abffff700 1 -- 172.21.15.3:0/1535226705 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1acc01f070 con 0x7f1ad8134610 2023-10-03T10:18:39.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.525+0000 7f1abffff700 1 -- 172.21.15.3:0/1535226705 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc0038e0 con 0x7f1ad8134610 2023-10-03T10:18:39.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.525+0000 7f1ade079700 1 -- 172.21.15.3:0/1535226705 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad8123c30 con 0x7f1ad8134610 2023-10-03T10:18:39.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.525+0000 7f1ade079700 1 -- 172.21.15.3:0/1535226705 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad81229e0 con 0x7f1ad8134610 2023-10-03T10:18:39.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.526+0000 7f1ade079700 1 -- 172.21.15.3:0/1535226705 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad809b1e0 con 0x7f1ad8134610 2023-10-03T10:18:39.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.526+0000 7f1abffff700 1 -- 172.21.15.3:0/1535226705 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1acc003a40 con 0x7f1ad8134610 2023-10-03T10:18:39.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.527+0000 7f1abffff700 1 -- 172.21.15.3:0/1535226705 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1acc021070 con 0x7f1ad8134610 2023-10-03T10:18:39.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.532+0000 7f1abffff700 1 -- 172.21.15.3:0/1535226705 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1acc02f3a0 con 0x7f1ad8134610 2023-10-03T10:18:39.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.566+0000 7f8cc1110700 1 Processor -- start 2023-10-03T10:18:39.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.566+0000 7f8cc1110700 1 -- start start 2023-10-03T10:18:39.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.567+0000 7f8cc1110700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cbc0a0a30 0x7f8cbc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:39.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.567+0000 7f8cc1110700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8cbc0a1340 0x7f8cbc133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:39.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.567+0000 7f8cc1110700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cbc09edd0 con 0x7f8cbc133fa0 2023-10-03T10:18:39.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.567+0000 7f8cc1110700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc09e1f0 con 0x7f8cbc0a1340 2023-10-03T10:18:39.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.567+0000 7f8cc1110700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc09dc50 con 0x7f8cbc0a0a30 2023-10-03T10:18:39.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.567+0000 7f8cbb59e700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8cbc133fa0 0x7f8cbc13ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43710/0 (socket says 172.21.15.3:43710) 2023-10-03T10:18:39.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.567+0000 7f8cbb59e700 1 -- 172.21.15.3:0/3347124160 learned_addr learned my addr 172.21.15.3:0/3347124160 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:39.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.567+0000 7f8cb9d9b700 1 -- 172.21.15.3:0/3347124160 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2177694431 0 0) 0x7f8cbc09edd0 con 0x7f8cbc133fa0 2023-10-03T10:18:39.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.567+0000 7f8cb9d9b700 1 -- 172.21.15.3:0/3347124160 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ca40035b0 con 0x7f8cbc133fa0 2023-10-03T10:18:39.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.567+0000 7f8cbad9d700 1 --2- 172.21.15.3:0/3347124160 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cbc0a0a30 0x7f8cbc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:39.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.568+0000 7f8cba59c700 1 --2- 172.21.15.3:0/3347124160 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8cbc0a1340 0x7f8cbc133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:39.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.568+0000 7f8cb9d9b700 1 -- 172.21.15.3:0/3347124160 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3627556007 0 0) 0x7f8ca40035b0 con 0x7f8cbc133fa0 2023-10-03T10:18:39.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.568+0000 7f8cb9d9b700 1 -- 172.21.15.3:0/3347124160 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cbc0a0a30 msgr2=0x7f8cbc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:39.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.568+0000 7f8cb9d9b700 1 --2- 172.21.15.3:0/3347124160 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cbc0a0a30 0x7f8cbc0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.568+0000 7f8cb9d9b700 1 -- 172.21.15.3:0/3347124160 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8cbc0a1340 msgr2=0x7f8cbc133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:39.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.568+0000 7f8cb9d9b700 1 --2- 172.21.15.3:0/3347124160 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8cbc0a1340 0x7f8cbc133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.568+0000 7f8cb9d9b700 1 -- 172.21.15.3:0/3347124160 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cbc0a1fe0 con 0x7f8cbc133fa0 2023-10-03T10:18:39.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.568+0000 7f8cb9d9b700 1 -- 172.21.15.3:0/3347124160 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1156569835 0 0) 0x7f8cb40019f0 con 0x7f8cbc133fa0 2023-10-03T10:18:39.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.568+0000 7f8cb9d9b700 1 -- 172.21.15.3:0/3347124160 >> v1:172.21.15.3:6789/0 conn(0x7f8cbc133fa0 legacy=0x7f8cbc13ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:39.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.568+0000 7f8cb9d9b700 1 --2- 172.21.15.3:0/3347124160 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ca4003a00 0x7f8ca4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.568+0000 7f8cb9d9b700 1 --2- 172.21.15.3:0/3347124160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ca4004390 0x7f8ca4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.568+0000 7f8cb9d9b700 1 --2- 172.21.15.3:0/3347124160 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cbc0a0a30 0x7f8ca40050a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.568+0000 7f8cb9d9b700 1 -- 172.21.15.3:0/3347124160 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc09e1f0 con 0x7f8ca4004390 2023-10-03T10:18:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.568+0000 7f8cb9d9b700 1 -- 172.21.15.3:0/3347124160 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc09dc50 con 0x7f8ca4003a00 2023-10-03T10:18:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.568+0000 7f8cb9d9b700 1 -- 172.21.15.3:0/3347124160 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca400ba10 con 0x7f8cbc0a0a30 2023-10-03T10:18:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.568+0000 7f8cba59c700 1 --2- 172.21.15.3:0/3347124160 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8cbc0a1340 0x7f8cbc133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:18:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.568+0000 7f8cbad9d700 1 --2- 172.21.15.3:0/3347124160 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ca4003a00 0x7f8ca4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.568+0000 7f8cbb59e700 1 --2- 172.21.15.3:0/3347124160 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cbc0a0a30 0x7f8ca40050a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.568+0000 7f8cba59c700 1 --2- 172.21.15.3:0/3347124160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ca4004390 0x7f8ca4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.568+0000 7f8cbad9d700 1 -- 172.21.15.3:0/3347124160 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cbc0a0a30 msgr2=0x7f8ca40050a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.568+0000 7f8cbad9d700 1 --2- 172.21.15.3:0/3347124160 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cbc0a0a30 0x7f8ca40050a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.568+0000 7f8cbad9d700 1 -- 172.21.15.3:0/3347124160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ca4004390 msgr2=0x7f8ca4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.568+0000 7f8cbad9d700 1 --2- 172.21.15.3:0/3347124160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ca4004390 0x7f8ca4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.568+0000 7f8cbad9d700 1 -- 172.21.15.3:0/3347124160 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca40035b0 con 0x7f8ca4003a00 2023-10-03T10:18:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.569+0000 7f8cbad9d700 1 --2- 172.21.15.3:0/3347124160 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ca4003a00 0x7f8ca4003e10 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8cac009b60 tx=0x7f8ca4005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4abc20e35f373f85 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.569+0000 7f8cb9d9b700 1 -- 172.21.15.3:0/3347124160 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac00e6c0 con 0x7f8ca4003a00 2023-10-03T10:18:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.569+0000 7f8cb9d9b700 1 -- 172.21.15.3:0/3347124160 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cac00e820 con 0x7f8ca4003a00 2023-10-03T10:18:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.569+0000 7f8cb9d9b700 1 -- 172.21.15.3:0/3347124160 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac00eb90 con 0x7f8ca4003a00 2023-10-03T10:18:39.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.570+0000 7f8cc1110700 1 -- 172.21.15.3:0/3347124160 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ca4003a00 msgr2=0x7f8ca4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:39.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.570+0000 7f8cc1110700 1 --2- 172.21.15.3:0/3347124160 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ca4003a00 0x7f8ca4003e10 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8cac009b60 tx=0x7f8ca4005880 comp rx=0 tx=0).stop 2023-10-03T10:18:39.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.570+0000 7f8cc1110700 1 -- 172.21.15.3:0/3347124160 shutdown_connections 2023-10-03T10:18:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.570+0000 7f8cc1110700 1 --2- 172.21.15.3:0/3347124160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ca4004390 0x7f8ca4004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.570+0000 7f8cc1110700 1 --2- 172.21.15.3:0/3347124160 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ca4003a00 0x7f8ca4003e10 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.570+0000 7f8cc1110700 1 --2- 172.21.15.3:0/3347124160 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8cbc0a0a30 0x7f8ca40050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.570+0000 7f8cc1110700 1 -- 172.21.15.3:0/3347124160 >> 172.21.15.3:0/3347124160 conn(0x7f8cbc0a6550 msgr2=0x7f8cbc0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.570+0000 7f8cc1110700 1 -- 172.21.15.3:0/3347124160 shutdown_connections 2023-10-03T10:18:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.570+0000 7f8cc1110700 1 -- 172.21.15.3:0/3347124160 wait complete. 2023-10-03T10:18:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.571+0000 7f8cc1110700 1 Processor -- start 2023-10-03T10:18:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.571+0000 7f8cc1110700 1 -- start start 2023-10-03T10:18:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.572+0000 7f8cc1110700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8ca4003a00 0x7f8cbc1c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.572+0000 7f8cc1110700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ca4004390 0x7f8cbc1c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.572+0000 7f8cc1110700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cbc0a0a30 0x7f8cbc1c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.572+0000 7f8cc1110700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc0a0220 con 0x7f8cbc0a0a30 2023-10-03T10:18:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.572+0000 7f8cc1110700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc125270 con 0x7f8ca4004390 2023-10-03T10:18:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.572+0000 7f8cc1110700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc124710 con 0x7f8ca4003a00 2023-10-03T10:18:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.572+0000 7f8cba59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ca4004390 0x7f8cbc1c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.572+0000 7f8cba59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ca4004390 0x7f8cbc1c8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50440/0 (socket says 172.21.15.3:50440) 2023-10-03T10:18:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.572+0000 7f8cba59c700 1 -- 172.21.15.3:0/1787499480 learned_addr learned my addr 172.21.15.3:0/1787499480 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.572+0000 7f8cbb59e700 1 --2- 172.21.15.3:0/1787499480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cbc0a0a30 0x7f8cbc1c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.572+0000 7f8cbad9d700 1 --2- 172.21.15.3:0/1787499480 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8ca4003a00 0x7f8cbc1c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.572+0000 7f8cba59c700 1 -- 172.21.15.3:0/1787499480 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8ca4003a00 msgr2=0x7f8cbc1c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.572+0000 7f8cba59c700 1 --2- 172.21.15.3:0/1787499480 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8ca4003a00 0x7f8cbc1c8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.572+0000 7f8cba59c700 1 -- 172.21.15.3:0/1787499480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cbc0a0a30 msgr2=0x7f8cbc1c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.572+0000 7f8cba59c700 1 --2- 172.21.15.3:0/1787499480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cbc0a0a30 0x7f8cbc1c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.572+0000 7f8cba59c700 1 -- 172.21.15.3:0/1787499480 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cbc0a25e0 con 0x7f8ca4004390 2023-10-03T10:18:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.573+0000 7f8cba59c700 1 --2- 172.21.15.3:0/1787499480 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ca4004390 0x7f8cbc1c8780 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0010870 tx=0x7f8cb0010b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.573+0000 7f8ca37fe700 1 -- 172.21.15.3:0/1787499480 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0004230 con 0x7f8ca4004390 2023-10-03T10:18:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.573+0000 7f8ca37fe700 1 -- 172.21.15.3:0/1787499480 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb000c050 con 0x7f8ca4004390 2023-10-03T10:18:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.573+0000 7f8cc1110700 1 -- 172.21.15.3:0/1787499480 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cbc0a3610 con 0x7f8ca4004390 2023-10-03T10:18:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.573+0000 7f8ca37fe700 1 -- 172.21.15.3:0/1787499480 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb00036a0 con 0x7f8ca4004390 2023-10-03T10:18:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.573+0000 7f8cc1110700 1 -- 172.21.15.3:0/1787499480 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cbc126c30 con 0x7f8ca4004390 2023-10-03T10:18:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.574+0000 7f8ca37fe700 1 -- 172.21.15.3:0/1787499480 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8cb0003800 con 0x7f8ca4004390 2023-10-03T10:18:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.574+0000 7f8ca17fa700 1 -- 172.21.15.3:0/1787499480 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cbc126100 con 0x7f8ca4004390 2023-10-03T10:18:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.574+0000 7f8ca37fe700 1 -- 172.21.15.3:0/1787499480 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8cb003fa80 con 0x7f8ca4004390 2023-10-03T10:18:39.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.579+0000 7f8ca37fe700 1 -- 172.21.15.3:0/1787499480 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8cb0015680 con 0x7f8ca4004390 2023-10-03T10:18:39.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.630+0000 7fbef969c700 1 Processor -- start 2023-10-03T10:18:39.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.630+0000 7fbef969c700 1 -- start start 2023-10-03T10:18:39.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.630+0000 7fbef969c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbef41337f0 0x7fbef4138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:39.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.630+0000 7fbef969c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbef4138dc0 0x7fbef413b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:39.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.630+0000 7fbef969c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef406b990 con 0x7fbef4133420 2023-10-03T10:18:39.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.630+0000 7fbef969c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef409b0e0 con 0x7fbef41337f0 2023-10-03T10:18:39.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.630+0000 7fbef969c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbef4133c20 con 0x7fbef4138dc0 2023-10-03T10:18:39.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.631+0000 7fbef27fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbef41337f0 0x7fbef4138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:39.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.631+0000 7fbef37fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbef4138dc0 0x7fbef413b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:39.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.631+0000 7fbef27fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbef41337f0 0x7fbef4138880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50450/0 (socket says 172.21.15.3:50450) 2023-10-03T10:18:39.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.631+0000 7fbef27fc700 1 -- 172.21.15.3:0/1941025677 learned_addr learned my addr 172.21.15.3:0/1941025677 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:39.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.631+0000 7fbef27fc700 1 -- 172.21.15.3:0/1941025677 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbef4138dc0 msgr2=0x7fbef413b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:39.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.631+0000 7fbef27fc700 1 --2- 172.21.15.3:0/1941025677 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbef4138dc0 0x7fbef413b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.631+0000 7fbef27fc700 1 -- 172.21.15.3:0/1941025677 >> v1:172.21.15.3:6789/0 conn(0x7fbef4133420 legacy=0x7fbef4133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:39.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.631+0000 7fbef27fc700 1 -- 172.21.15.3:0/1941025677 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef40a0590 con 0x7fbef41337f0 2023-10-03T10:18:39.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.631+0000 7fbef37fe700 1 --2- 172.21.15.3:0/1941025677 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbef4138dc0 0x7fbef413b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:39.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.631+0000 7fbef27fc700 1 --2- 172.21.15.3:0/1941025677 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbef41337f0 0x7fbef4138880 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009920 tx=0x7fbee8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=295dfa000c18f1bb server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:39.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.632+0000 7fbef1ffb700 1 -- 172.21.15.3:0/1941025677 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee8011830 con 0x7fbef41337f0 2023-10-03T10:18:39.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.632+0000 7fbef1ffb700 1 -- 172.21.15.3:0/1941025677 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee8011990 con 0x7fbef41337f0 2023-10-03T10:18:39.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.632+0000 7fbef1ffb700 1 -- 172.21.15.3:0/1941025677 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee8011d20 con 0x7fbef41337f0 2023-10-03T10:18:39.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.632+0000 7fbef969c700 1 -- 172.21.15.3:0/1941025677 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbef41337f0 msgr2=0x7fbef4138880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:39.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.632+0000 7fbef969c700 1 --2- 172.21.15.3:0/1941025677 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbef41337f0 0x7fbef4138880 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009920 tx=0x7fbee8009c30 comp rx=0 tx=0).stop 2023-10-03T10:18:39.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.632+0000 7fbef969c700 1 -- 172.21.15.3:0/1941025677 shutdown_connections 2023-10-03T10:18:39.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.632+0000 7fbef969c700 1 --2- 172.21.15.3:0/1941025677 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbef4138dc0 0x7fbef413b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.632+0000 7fbef969c700 1 --2- 172.21.15.3:0/1941025677 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbef41337f0 0x7fbef4138880 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.632+0000 7fbef969c700 1 -- 172.21.15.3:0/1941025677 >> 172.21.15.3:0/1941025677 conn(0x7fbef40a4210 msgr2=0x7fbef40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:39.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.633+0000 7fbef969c700 1 -- 172.21.15.3:0/1941025677 shutdown_connections 2023-10-03T10:18:39.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.633+0000 7fbef969c700 1 -- 172.21.15.3:0/1941025677 wait complete. 2023-10-03T10:18:39.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.634+0000 7fbef969c700 1 Processor -- start 2023-10-03T10:18:39.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.634+0000 7fbef969c700 1 -- start start 2023-10-03T10:18:39.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.635+0000 7fbef969c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbef4133420 0x7fbef412d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:39.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.635+0000 7fbef969c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbef41337f0 0x7fbef412d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:39.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.635+0000 7fbef969c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbef4138dc0 0x7fbef41c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:39.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.635+0000 7fbef969c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef409d0f0 con 0x7fbef4133420 2023-10-03T10:18:39.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.635+0000 7fbef969c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef409deb0 con 0x7fbef4138dc0 2023-10-03T10:18:39.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.635+0000 7fbef969c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbef409b970 con 0x7fbef41337f0 2023-10-03T10:18:39.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.635+0000 7fbef37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbef4138dc0 0x7fbef41c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:39.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.635+0000 7fbef37fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbef4138dc0 0x7fbef41c8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50464/0 (socket says 172.21.15.3:50464) 2023-10-03T10:18:39.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.635+0000 7fbef27fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbef41337f0 0x7fbef412d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:39.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.635+0000 7fbef37fe700 1 -- 172.21.15.3:0/3274452075 learned_addr learned my addr 172.21.15.3:0/3274452075 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:39.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.635+0000 7fbef2ffd700 1 --2- 172.21.15.3:0/3274452075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbef4133420 0x7fbef412d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:39.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.636+0000 7fbef37fe700 1 -- 172.21.15.3:0/3274452075 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbef41337f0 msgr2=0x7fbef412d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:39.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.636+0000 7fbef37fe700 1 --2- 172.21.15.3:0/3274452075 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbef41337f0 0x7fbef412d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.636+0000 7fbef37fe700 1 -- 172.21.15.3:0/3274452075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbef4133420 msgr2=0x7fbef412d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:39.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.636+0000 7fbef37fe700 1 --2- 172.21.15.3:0/3274452075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbef4133420 0x7fbef412d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.636+0000 7fbef37fe700 1 -- 172.21.15.3:0/3274452075 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeec009a70 con 0x7fbef4138dc0 2023-10-03T10:18:39.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.636+0000 7fbef2ffd700 1 --2- 172.21.15.3:0/3274452075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbef4133420 0x7fbef412d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:39.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.636+0000 7fbef27fc700 1 --2- 172.21.15.3:0/3274452075 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbef41337f0 0x7fbef412d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:39.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.636+0000 7fbef37fe700 1 --2- 172.21.15.3:0/3274452075 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbef4138dc0 0x7fbef41c8660 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00efb0 tx=0x7fbeec00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:39.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.636+0000 7fbedb7fe700 1 -- 172.21.15.3:0/3274452075 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec00cc90 con 0x7fbef4138dc0 2023-10-03T10:18:39.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.636+0000 7fbedb7fe700 1 -- 172.21.15.3:0/3274452075 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeec00f040 con 0x7fbef4138dc0 2023-10-03T10:18:39.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.636+0000 7fbef969c700 1 -- 172.21.15.3:0/3274452075 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee8009550 con 0x7fbef4138dc0 2023-10-03T10:18:39.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.636+0000 7fbedb7fe700 1 -- 172.21.15.3:0/3274452075 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec014480 con 0x7fbef4138dc0 2023-10-03T10:18:39.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.636+0000 7fbef969c700 1 -- 172.21.15.3:0/3274452075 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef409bbb0 con 0x7fbef4138dc0 2023-10-03T10:18:39.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.637+0000 7fbef969c700 1 -- 172.21.15.3:0/3274452075 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef4074ea0 con 0x7fbef4138dc0 2023-10-03T10:18:39.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.638+0000 7fbedb7fe700 1 -- 172.21.15.3:0/3274452075 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbeec0145e0 con 0x7fbef4138dc0 2023-10-03T10:18:39.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.638+0000 7fbedb7fe700 1 -- 172.21.15.3:0/3274452075 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fbeec0408a0 con 0x7fbef4138dc0 2023-10-03T10:18:39.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.647+0000 7fbedb7fe700 1 -- 172.21.15.3:0/3274452075 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbeec012930 con 0x7fbef4138dc0 2023-10-03T10:18:39.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.717+0000 7f847e7fc700 1 -- 172.21.15.3:0/1039449326 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8470005590 con 0x7f8498139170 2023-10-03T10:18:39.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.718+0000 7f8494ff9700 1 -- 172.21.15.3:0/1039449326 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8490011390 con 0x7f8498139170 2023-10-03T10:18:39.718 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:39.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.720+0000 7f847e7fc700 1 -- 172.21.15.3:0/1039449326 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8498139170 msgr2=0x7f849811f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:39.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.720+0000 7f847e7fc700 1 --2- 172.21.15.3:0/1039449326 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8498139170 0x7f849811f1e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f849000be60 tx=0x7f8490004ad0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.721+0000 7f847e7fc700 1 -- 172.21.15.3:0/1039449326 shutdown_connections 2023-10-03T10:18:39.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.721+0000 7f847e7fc700 1 --2- 172.21.15.3:0/1039449326 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8498139170 0x7f849811f1e0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.721+0000 7f847e7fc700 1 --2- 172.21.15.3:0/1039449326 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f849809d6f0 0x7f849811e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.721+0000 7f847e7fc700 1 --2- 172.21.15.3:0/1039449326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f849809e1d0 0x7f849811eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.721+0000 7f847e7fc700 1 -- 172.21.15.3:0/1039449326 >> 172.21.15.3:0/1039449326 conn(0x7f8498066f50 msgr2=0x7f8498138090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:39.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.722+0000 7f847e7fc700 1 -- 172.21.15.3:0/1039449326 shutdown_connections 2023-10-03T10:18:39.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.722+0000 7f847e7fc700 1 -- 172.21.15.3:0/1039449326 wait complete. 2023-10-03T10:18:39.740 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:39.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.778+0000 7f1ade079700 1 -- 172.21.15.3:0/1535226705 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1ad8074ea0 con 0x7f1ad8134610 2023-10-03T10:18:39.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.779+0000 7f1abffff700 1 -- 172.21.15.3:0/1535226705 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1acc02b070 con 0x7f1ad8134610 2023-10-03T10:18:39.779 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:39.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.781+0000 7f1ade079700 1 -- 172.21.15.3:0/1535226705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ad8134610 msgr2=0x7f1ad8121a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:39.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.781+0000 7f1ade079700 1 --2- 172.21.15.3:0/1535226705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ad8134610 0x7f1ad8121a20 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f1acc016000 tx=0x7f1acc004750 comp rx=0 tx=0).stop 2023-10-03T10:18:39.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.781+0000 7f1ade079700 1 -- 172.21.15.3:0/1535226705 shutdown_connections 2023-10-03T10:18:39.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.781+0000 7f1ade079700 1 --2- 172.21.15.3:0/1535226705 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1ad809e500 0x7f1ad8121f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.781+0000 7f1ade079700 1 --2- 172.21.15.3:0/1535226705 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1ad806e570 0x7f1ad81214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.781+0000 7f1ade079700 1 --2- 172.21.15.3:0/1535226705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ad8134610 0x7f1ad8121a20 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.781+0000 7f1ade079700 1 -- 172.21.15.3:0/1535226705 >> 172.21.15.3:0/1535226705 conn(0x7f1ad80a2aa0 msgr2=0x7f1ad81bf290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:39.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.781+0000 7f1ade079700 1 -- 172.21.15.3:0/1535226705 shutdown_connections 2023-10-03T10:18:39.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.781+0000 7f1ade079700 1 -- 172.21.15.3:0/1535226705 wait complete. 2023-10-03T10:18:39.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.792+0000 7f8ca17fa700 1 -- 172.21.15.3:0/1787499480 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8cbc074ea0 con 0x7f8ca4004390 2023-10-03T10:18:39.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.792+0000 7f8ca37fe700 1 -- 172.21.15.3:0/1787499480 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8cb002c090 con 0x7f8ca4004390 2023-10-03T10:18:39.793 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:39.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.795+0000 7f8ca17fa700 1 -- 172.21.15.3:0/1787499480 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ca4004390 msgr2=0x7f8cbc1c8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:39.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.795+0000 7f8ca17fa700 1 --2- 172.21.15.3:0/1787499480 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ca4004390 0x7f8cbc1c8780 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0010870 tx=0x7f8cb0010b80 comp rx=0 tx=0).stop 2023-10-03T10:18:39.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.795+0000 7f8ca17fa700 1 -- 172.21.15.3:0/1787499480 shutdown_connections 2023-10-03T10:18:39.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.795+0000 7f8ca17fa700 1 --2- 172.21.15.3:0/1787499480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cbc0a0a30 0x7f8cbc1c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.795+0000 7f8ca17fa700 1 --2- 172.21.15.3:0/1787499480 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8ca4003a00 0x7f8cbc1c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.795+0000 7f8ca17fa700 1 --2- 172.21.15.3:0/1787499480 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ca4004390 0x7f8cbc1c8780 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.795+0000 7f8ca17fa700 1 -- 172.21.15.3:0/1787499480 >> 172.21.15.3:0/1787499480 conn(0x7f8cbc0a6550 msgr2=0x7f8cbc11c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:39.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.795+0000 7f8ca17fa700 1 -- 172.21.15.3:0/1787499480 shutdown_connections 2023-10-03T10:18:39.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.795+0000 7f8ca17fa700 1 -- 172.21.15.3:0/1787499480 wait complete. 2023-10-03T10:18:39.800 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:39.809 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:39.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.875+0000 7fbef969c700 1 -- 172.21.15.3:0/3274452075 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbef409cf50 con 0x7fbef4138dc0 2023-10-03T10:18:39.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.875+0000 7fbedb7fe700 1 -- 172.21.15.3:0/3274452075 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbef409cf50 con 0x7fbef4138dc0 2023-10-03T10:18:39.877 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:39.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.878+0000 7fbef969c700 1 -- 172.21.15.3:0/3274452075 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbef4138dc0 msgr2=0x7fbef41c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:39.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.878+0000 7fbef969c700 1 --2- 172.21.15.3:0/3274452075 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbef4138dc0 0x7fbef41c8660 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00efb0 tx=0x7fbeec00c3a0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.878+0000 7fbef969c700 1 -- 172.21.15.3:0/3274452075 shutdown_connections 2023-10-03T10:18:39.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.878+0000 7fbef969c700 1 --2- 172.21.15.3:0/3274452075 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbef4138dc0 0x7fbef41c8660 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.878+0000 7fbef969c700 1 --2- 172.21.15.3:0/3274452075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbef4133420 0x7fbef412d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.878+0000 7fbef969c700 1 --2- 172.21.15.3:0/3274452075 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbef41337f0 0x7fbef412d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:39.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.878+0000 7fbef969c700 1 -- 172.21.15.3:0/3274452075 >> 172.21.15.3:0/3274452075 conn(0x7fbef40a4210 msgr2=0x7fbef40a29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:39.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.878+0000 7fbef969c700 1 -- 172.21.15.3:0/3274452075 shutdown_connections 2023-10-03T10:18:39.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:39.878+0000 7fbef969c700 1 -- 172.21.15.3:0/3274452075 wait complete. 2023-10-03T10:18:39.891 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:40.741 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:40.800 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:40.810 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:40.892 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:40.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.894+0000 7f4828e52700 1 Processor -- start 2023-10-03T10:18:40.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.894+0000 7f4828e52700 1 -- start start 2023-10-03T10:18:40.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.894+0000 7f4828e52700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f482409e930 0x7f482409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:40.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.894+0000 7f4828e52700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f482409f240 0x7f48241317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:40.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.895+0000 7f4828e52700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4824070940 con 0x7f482409f650 2023-10-03T10:18:40.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.895+0000 7f4828e52700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f482406fd60 con 0x7f482409f240 2023-10-03T10:18:40.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.895+0000 7f4828e52700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f482406fa80 con 0x7f482409e930 2023-10-03T10:18:40.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.895+0000 7f4821d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f482409f240 0x7f48241317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:40.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.895+0000 7f4821d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f482409f240 0x7f48241317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60498/0 (socket says 172.21.15.3:60498) 2023-10-03T10:18:40.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.895+0000 7f4821d9b700 1 -- 172.21.15.3:0/1258484911 learned_addr learned my addr 172.21.15.3:0/1258484911 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:40.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.895+0000 7f4821d9b700 1 -- 172.21.15.3:0/1258484911 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f482409e930 msgr2=0x7f482409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:40.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.895+0000 7f4821d9b700 1 --2- 172.21.15.3:0/1258484911 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f482409e930 0x7f482409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:40.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.895+0000 7f4821d9b700 1 -- 172.21.15.3:0/1258484911 >> v1:172.21.15.3:6789/0 conn(0x7f482409f650 legacy=0x7f4824131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:40.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.895+0000 7f4821d9b700 1 -- 172.21.15.3:0/1258484911 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482409fb60 con 0x7f482409f240 2023-10-03T10:18:40.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.896+0000 7f4821d9b700 1 --2- 172.21.15.3:0/1258484911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f482409f240 0x7f48241317f0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f480c0098f0 tx=0x7f480c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89a6e9be34097a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:40.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.896+0000 7f482159a700 1 -- 172.21.15.3:0/1258484911 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c00b8a0 con 0x7f482409f240 2023-10-03T10:18:40.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.896+0000 7f482159a700 1 -- 172.21.15.3:0/1258484911 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480c00ba00 con 0x7f482409f240 2023-10-03T10:18:40.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.896+0000 7f482159a700 1 -- 172.21.15.3:0/1258484911 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c00bd90 con 0x7f482409f240 2023-10-03T10:18:40.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.897+0000 7f4828e52700 1 -- 172.21.15.3:0/1258484911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f482409f240 msgr2=0x7f48241317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:40.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.897+0000 7f4828e52700 1 --2- 172.21.15.3:0/1258484911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f482409f240 0x7f48241317f0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f480c0098f0 tx=0x7f480c009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:40.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.897+0000 7f4828e52700 1 -- 172.21.15.3:0/1258484911 shutdown_connections 2023-10-03T10:18:40.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.897+0000 7f4828e52700 1 --2- 172.21.15.3:0/1258484911 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f482409e930 0x7f482409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:40.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.897+0000 7f4828e52700 1 --2- 172.21.15.3:0/1258484911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f482409f240 0x7f48241317f0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:40.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.897+0000 7f4828e52700 1 -- 172.21.15.3:0/1258484911 >> 172.21.15.3:0/1258484911 conn(0x7f48240a4210 msgr2=0x7f48240a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:40.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.897+0000 7f4828e52700 1 -- 172.21.15.3:0/1258484911 shutdown_connections 2023-10-03T10:18:40.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.897+0000 7f4828e52700 1 -- 172.21.15.3:0/1258484911 wait complete. 2023-10-03T10:18:40.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.898+0000 7f4828e52700 1 Processor -- start 2023-10-03T10:18:40.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.898+0000 7f4828e52700 1 -- start start 2023-10-03T10:18:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.899+0000 7f4828e52700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f482409e930 0x7f48241a3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.899+0000 7f4828e52700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f482409f240 0x7f48241a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.899+0000 7f4828e52700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f482409f650 0x7f48241a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.899+0000 7f4828e52700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48241a7870 con 0x7f482409e930 2023-10-03T10:18:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.899+0000 7f4828e52700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48241a5a00 con 0x7f482409f240 2023-10-03T10:18:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.899+0000 7f4828e52700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f48240a3640 con 0x7f482409f650 2023-10-03T10:18:40.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.899+0000 7f4821d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f482409f240 0x7f48241a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:40.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.899+0000 7f482259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f482409e930 0x7f48241a3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:40.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.899+0000 7f4821d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f482409f240 0x7f48241a4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50474/0 (socket says 172.21.15.3:50474) 2023-10-03T10:18:40.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.899+0000 7f4821d9b700 1 -- 172.21.15.3:0/734620170 learned_addr learned my addr 172.21.15.3:0/734620170 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:40.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.899+0000 7f4822d9d700 1 --2- 172.21.15.3:0/734620170 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f482409f650 0x7f48241a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:40.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.899+0000 7f482259c700 1 -- 172.21.15.3:0/734620170 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f482409f650 msgr2=0x7f48241a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:40.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.899+0000 7f482259c700 1 --2- 172.21.15.3:0/734620170 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f482409f650 0x7f48241a47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:40.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.899+0000 7f482259c700 1 -- 172.21.15.3:0/734620170 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f482409f240 msgr2=0x7f48241a4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:40.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.899+0000 7f482259c700 1 --2- 172.21.15.3:0/734620170 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f482409f240 0x7f48241a4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:40.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.899+0000 7f482259c700 1 -- 172.21.15.3:0/734620170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f480c009550 con 0x7f482409e930 2023-10-03T10:18:40.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.900+0000 7f482259c700 1 --2- 172.21.15.3:0/734620170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f482409e930 0x7f48241a3d50 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f481400b450 tx=0x7f481400b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:40.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.900+0000 7f481affd700 1 -- 172.21.15.3:0/734620170 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4814011830 con 0x7f482409e930 2023-10-03T10:18:40.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.900+0000 7f481affd700 1 -- 172.21.15.3:0/734620170 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4814011990 con 0x7f482409e930 2023-10-03T10:18:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.900+0000 7f4828e52700 1 -- 172.21.15.3:0/734620170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48240a0190 con 0x7f482409e930 2023-10-03T10:18:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.900+0000 7f481affd700 1 -- 172.21.15.3:0/734620170 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4814011d40 con 0x7f482409e930 2023-10-03T10:18:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.900+0000 7f4828e52700 1 -- 172.21.15.3:0/734620170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48241a6760 con 0x7f482409e930 2023-10-03T10:18:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.901+0000 7f481affd700 1 -- 172.21.15.3:0/734620170 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f481401a400 con 0x7f482409e930 2023-10-03T10:18:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.901+0000 7f481affd700 1 -- 172.21.15.3:0/734620170 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f481403f570 con 0x7f482409e930 2023-10-03T10:18:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.901+0000 7f4828e52700 1 -- 172.21.15.3:0/734620170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4824074ea0 con 0x7f482409e930 2023-10-03T10:18:40.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.907+0000 7f481affd700 1 -- 172.21.15.3:0/734620170 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4814015020 con 0x7f482409e930 2023-10-03T10:18:40.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.947+0000 7fc08144e700 1 Processor -- start 2023-10-03T10:18:40.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.948+0000 7fc08144e700 1 -- start start 2023-10-03T10:18:40.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.948+0000 7fc08144e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc07c09d6f0 0x7fc07c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:40.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.948+0000 7fc08144e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc07c09e000 0x7fc07c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:40.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.948+0000 7fc08144e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc07c09e410 0x7fc07c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:40.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.948+0000 7fc08144e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc07c06b460 con 0x7fc07c09e410 2023-10-03T10:18:40.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.948+0000 7fc08144e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c09b120 con 0x7fc07c09d6f0 2023-10-03T10:18:40.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.948+0000 7fc08144e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c09a7e0 con 0x7fc07c09e000 2023-10-03T10:18:40.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.948+0000 7fc07a7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc07c09e000 0x7fc07c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:40.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.949+0000 7fc07a7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc07c09e000 0x7fc07c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42584/0 (socket says 172.21.15.3:42584) 2023-10-03T10:18:40.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.949+0000 7fc07affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc07c09d6f0 0x7fc07c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:40.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.949+0000 7fc07a7fc700 1 -- 172.21.15.3:0/1196968621 learned_addr learned my addr 172.21.15.3:0/1196968621 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:40.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.949+0000 7fc07b7fe700 1 --2- 172.21.15.3:0/1196968621 >> v2:172.21.15.3:3300/0 conn(0x7fc07c09e410 0x7fc07c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:40.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.949+0000 7fc07a7fc700 1 -- 172.21.15.3:0/1196968621 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc07c09d6f0 msgr2=0x7fc07c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:40.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.949+0000 7fc07a7fc700 1 --2- 172.21.15.3:0/1196968621 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc07c09d6f0 0x7fc07c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:40.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.949+0000 7fc07a7fc700 1 -- 172.21.15.3:0/1196968621 >> v2:172.21.15.3:3300/0 conn(0x7fc07c09e410 msgr2=0x7fc07c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:40.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.949+0000 7fc07a7fc700 1 --2- 172.21.15.3:0/1196968621 >> v2:172.21.15.3:3300/0 conn(0x7fc07c09e410 0x7fc07c133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:40.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.949+0000 7fc07a7fc700 1 -- 172.21.15.3:0/1196968621 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07c09a500 con 0x7fc07c09e000 2023-10-03T10:18:40.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.949+0000 7fc07b7fe700 1 --2- 172.21.15.3:0/1196968621 >> v2:172.21.15.3:3300/0 conn(0x7fc07c09e410 0x7fc07c133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:40.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.949+0000 7fc07a7fc700 1 --2- 172.21.15.3:0/1196968621 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc07c09e000 0x7fc07c1339f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc070009920 tx=0x7fc070009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f06cf0a7bea75151 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:40.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.949+0000 7fc079ffb700 1 -- 172.21.15.3:0/1196968621 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc070011830 con 0x7fc07c09e000 2023-10-03T10:18:40.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.949+0000 7fc079ffb700 1 -- 172.21.15.3:0/1196968621 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc070011990 con 0x7fc07c09e000 2023-10-03T10:18:40.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.949+0000 7fc079ffb700 1 -- 172.21.15.3:0/1196968621 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc070011d20 con 0x7fc07c09e000 2023-10-03T10:18:40.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.950+0000 7fc08144e700 1 -- 172.21.15.3:0/1196968621 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc07c09e000 msgr2=0x7fc07c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:40.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.950+0000 7fc08144e700 1 --2- 172.21.15.3:0/1196968621 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc07c09e000 0x7fc07c1339f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc070009920 tx=0x7fc070009c30 comp rx=0 tx=0).stop 2023-10-03T10:18:40.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.950+0000 7fc08144e700 1 -- 172.21.15.3:0/1196968621 shutdown_connections 2023-10-03T10:18:40.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.950+0000 7fc08144e700 1 --2- 172.21.15.3:0/1196968621 >> v2:172.21.15.3:3300/0 conn(0x7fc07c09e410 0x7fc07c133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:40.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.950+0000 7fc08144e700 1 --2- 172.21.15.3:0/1196968621 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc07c09d6f0 0x7fc07c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:40.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.950+0000 7fc08144e700 1 --2- 172.21.15.3:0/1196968621 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc07c09e000 0x7fc07c1339f0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:40.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.950+0000 7fc08144e700 1 -- 172.21.15.3:0/1196968621 >> 172.21.15.3:0/1196968621 conn(0x7fc07c066f50 msgr2=0x7fc07c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:40.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.950+0000 7fc08144e700 1 -- 172.21.15.3:0/1196968621 shutdown_connections 2023-10-03T10:18:40.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.950+0000 7fc08144e700 1 -- 172.21.15.3:0/1196968621 wait complete. 2023-10-03T10:18:40.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.952+0000 7fc08144e700 1 Processor -- start 2023-10-03T10:18:40.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.952+0000 7fc08144e700 1 -- start start 2023-10-03T10:18:40.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.953+0000 7fc08144e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc07c09d6f0 0x7fc07c1b91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:40.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.953+0000 7fc08144e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc07c09e000 0x7fc07c1a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:40.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.953+0000 7fc08144e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc07c09e410 0x7fc07c1a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:40.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.953+0000 7fc08144e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c1aa2c0 con 0x7fc07c09d6f0 2023-10-03T10:18:40.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.953+0000 7fc08144e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c1a9900 con 0x7fc07c09e000 2023-10-03T10:18:40.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.953+0000 7fc08144e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c1a9090 con 0x7fc07c09e410 2023-10-03T10:18:40.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.953+0000 7fc07affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc07c09d6f0 0x7fc07c1b91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:40.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.953+0000 7fc07affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc07c09d6f0 0x7fc07c1b91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60524/0 (socket says 172.21.15.3:60524) 2023-10-03T10:18:40.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.953+0000 7fc07affd700 1 -- 172.21.15.3:0/3709996434 learned_addr learned my addr 172.21.15.3:0/3709996434 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:40.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.953+0000 7fc07a7fc700 1 --2- 172.21.15.3:0/3709996434 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc07c09e000 0x7fc07c1a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:40.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.953+0000 7fc07b7fe700 1 --2- 172.21.15.3:0/3709996434 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc07c09e410 0x7fc07c1a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:40.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.954+0000 7fc07a7fc700 1 -- 172.21.15.3:0/3709996434 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc07c09e410 msgr2=0x7fc07c1a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:40.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.954+0000 7fc07a7fc700 1 --2- 172.21.15.3:0/3709996434 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc07c09e410 0x7fc07c1a6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:40.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.954+0000 7fc07a7fc700 1 -- 172.21.15.3:0/3709996434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc07c09d6f0 msgr2=0x7fc07c1b91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:40.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.954+0000 7fc07a7fc700 1 --2- 172.21.15.3:0/3709996434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc07c09d6f0 0x7fc07c1b91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:40.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.954+0000 7fc07a7fc700 1 -- 172.21.15.3:0/3709996434 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc074009a70 con 0x7fc07c09e000 2023-10-03T10:18:40.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.954+0000 7fc07b7fe700 1 --2- 172.21.15.3:0/3709996434 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc07c09e410 0x7fc07c1a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:40.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.954+0000 7fc07a7fc700 1 --2- 172.21.15.3:0/3709996434 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc07c09e000 0x7fc07c1a5de0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fc07c09e9b0 tx=0x7fc070004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:40.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.954+0000 7fc0637fe700 1 -- 172.21.15.3:0/3709996434 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc070004240 con 0x7fc07c09e000 2023-10-03T10:18:40.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.954+0000 7fc0637fe700 1 -- 172.21.15.3:0/3709996434 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07001f070 con 0x7fc07c09e000 2023-10-03T10:18:40.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.954+0000 7fc08144e700 1 -- 172.21.15.3:0/3709996434 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc070009550 con 0x7fc07c09e000 2023-10-03T10:18:40.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.954+0000 7fc0637fe700 1 -- 172.21.15.3:0/3709996434 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0700036a0 con 0x7fc07c09e000 2023-10-03T10:18:40.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.954+0000 7fc08144e700 1 -- 172.21.15.3:0/3709996434 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc07c09c710 con 0x7fc07c09e000 2023-10-03T10:18:40.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.955+0000 7fc0617fa700 1 -- 172.21.15.3:0/3709996434 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0540052e0 con 0x7fc07c09e000 2023-10-03T10:18:40.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.956+0000 7fc0637fe700 1 -- 172.21.15.3:0/3709996434 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc070021030 con 0x7fc07c09e000 2023-10-03T10:18:40.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.956+0000 7fc0637fe700 1 -- 172.21.15.3:0/3709996434 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc070040cf0 con 0x7fc07c09e000 2023-10-03T10:18:40.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:40.961+0000 7fc0637fe700 1 -- 172.21.15.3:0/3709996434 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc0700199a0 con 0x7fc07c09e000 2023-10-03T10:18:41.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.034+0000 7f9877f9c700 1 Processor -- start 2023-10-03T10:18:41.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.034+0000 7f9877f9c700 1 -- start start 2023-10-03T10:18:41.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.035+0000 7f9877f9c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f98701337f0 0x7f9870138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:41.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.035+0000 7f9877f9c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9870138dc0 0x7f987013b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:41.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.035+0000 7f9877f9c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f987006b990 con 0x7f9870133420 2023-10-03T10:18:41.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.035+0000 7f9877f9c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f987009b0e0 con 0x7f98701337f0 2023-10-03T10:18:41.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.035+0000 7f9877f9c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9870133c20 con 0x7f9870138dc0 2023-10-03T10:18:41.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.035+0000 7f9876539700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9870138dc0 0x7f987013b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:41.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.036+0000 7f9876539700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9870138dc0 0x7f987013b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50490/0 (socket says 172.21.15.3:50490) 2023-10-03T10:18:41.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.036+0000 7f9876539700 1 -- 172.21.15.3:0/2016080629 learned_addr learned my addr 172.21.15.3:0/2016080629 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:41.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.036+0000 7f9875537700 1 --2- 172.21.15.3:0/2016080629 >> v2:172.21.15.3:3300/0 conn(0x7f98701337f0 0x7f9870138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:41.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.036+0000 7f9876539700 1 -- 172.21.15.3:0/2016080629 >> v2:172.21.15.3:3300/0 conn(0x7f98701337f0 msgr2=0x7f9870138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:41.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.036+0000 7f9876539700 1 --2- 172.21.15.3:0/2016080629 >> v2:172.21.15.3:3300/0 conn(0x7f98701337f0 0x7f9870138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:41.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.036+0000 7f9876539700 1 -- 172.21.15.3:0/2016080629 >> v1:172.21.15.3:6789/0 conn(0x7f9870133420 legacy=0x7f9870133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:41.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.036+0000 7f9874d36700 1 -- 172.21.15.3:0/2016080629 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3553532071 0 0) 0x7f987006b990 con 0x7f9870133420 2023-10-03T10:18:41.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.036+0000 7f9876539700 1 -- 172.21.15.3:0/2016080629 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98700a0590 con 0x7f9870138dc0 2023-10-03T10:18:41.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.036+0000 7f9875537700 1 --2- 172.21.15.3:0/2016080629 >> v2:172.21.15.3:3300/0 conn(0x7f98701337f0 0x7f9870138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:41.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.036+0000 7f9876539700 1 --2- 172.21.15.3:0/2016080629 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9870138dc0 0x7f987013b1a0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f9868009e50 tx=0x7f986800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f5bf78be042f6c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:41.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.036+0000 7f9874d36700 1 -- 172.21.15.3:0/2016080629 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986800e070 con 0x7f9870138dc0 2023-10-03T10:18:41.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.036+0000 7f9874d36700 1 -- 172.21.15.3:0/2016080629 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986800b910 con 0x7f9870138dc0 2023-10-03T10:18:41.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.036+0000 7f9874d36700 1 -- 172.21.15.3:0/2016080629 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986800bcc0 con 0x7f9870138dc0 2023-10-03T10:18:41.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.037+0000 7f9877f9c700 1 -- 172.21.15.3:0/2016080629 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9870138dc0 msgr2=0x7f987013b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:41.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.037+0000 7f9877f9c700 1 --2- 172.21.15.3:0/2016080629 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9870138dc0 0x7f987013b1a0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f9868009e50 tx=0x7f986800b330 comp rx=0 tx=0).stop 2023-10-03T10:18:41.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.038+0000 7f9877f9c700 1 -- 172.21.15.3:0/2016080629 shutdown_connections 2023-10-03T10:18:41.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.038+0000 7f9877f9c700 1 --2- 172.21.15.3:0/2016080629 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9870138dc0 0x7f987013b1a0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:41.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.038+0000 7f9877f9c700 1 --2- 172.21.15.3:0/2016080629 >> v2:172.21.15.3:3300/0 conn(0x7f98701337f0 0x7f9870138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:41.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.038+0000 7f9877f9c700 1 -- 172.21.15.3:0/2016080629 >> 172.21.15.3:0/2016080629 conn(0x7f98700a4210 msgr2=0x7f98700a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:41.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.038+0000 7f9877f9c700 1 -- 172.21.15.3:0/2016080629 shutdown_connections 2023-10-03T10:18:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.038+0000 7f8c7a100700 1 Processor -- start 2023-10-03T10:18:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.038+0000 7f8c7a100700 1 -- start start 2023-10-03T10:18:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.038+0000 7f8c7a100700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c7409d6f0 0x7f8c7409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.038+0000 7f8c7a100700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8c7409e000 0x7f8c74130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.039+0000 7f9877f9c700 1 -- 172.21.15.3:0/2016080629 wait complete. 2023-10-03T10:18:41.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.038+0000 7f8c7a100700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c7409f130 con 0x7f8c7409e410 2023-10-03T10:18:41.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.038+0000 7f8c7a100700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c7409f520 con 0x7f8c7409e000 2023-10-03T10:18:41.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.038+0000 7f8c7a100700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7409e930 con 0x7f8c7409d6f0 2023-10-03T10:18:41.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.039+0000 7f8c72ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8c7409e000 0x7f8c74130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:41.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.039+0000 7f8c72ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8c7409e000 0x7f8c74130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60542/0 (socket says 172.21.15.3:60542) 2023-10-03T10:18:41.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.039+0000 7f8c73fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8c7409e410 0x7f8c74130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43736/0 (socket says 172.21.15.3:43736) 2023-10-03T10:18:41.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.039+0000 7f8c72ffd700 1 -- 172.21.15.3:0/3758054669 learned_addr learned my addr 172.21.15.3:0/3758054669 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:41.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.039+0000 7f8c737fe700 1 --2- 172.21.15.3:0/3758054669 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c7409d6f0 0x7f8c7409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:41.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.039+0000 7f8c737fe700 1 -- 172.21.15.3:0/3758054669 >> v2:172.21.15.3:3300/0 conn(0x7f8c7409e000 msgr2=0x7f8c74130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:41.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.039+0000 7f8c737fe700 1 --2- 172.21.15.3:0/3758054669 >> v2:172.21.15.3:3300/0 conn(0x7f8c7409e000 0x7f8c74130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:41.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.039+0000 7f8c737fe700 1 -- 172.21.15.3:0/3758054669 >> v1:172.21.15.3:6789/0 conn(0x7f8c7409e410 legacy=0x7f8c74130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:41.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.039+0000 7f8c737fe700 1 -- 172.21.15.3:0/3758054669 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7409b2d0 con 0x7f8c7409d6f0 2023-10-03T10:18:41.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.039+0000 7f8c727fc700 1 -- 172.21.15.3:0/3758054669 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1705274688 0 0) 0x7f8c7409f130 con 0x7f8c7409e410 2023-10-03T10:18:41.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.039+0000 7f8c737fe700 1 --2- 172.21.15.3:0/3758054669 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c7409d6f0 0x7f8c7409dac0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f8c64009840 tx=0x7f8c64009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21aeb6f624216af5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:41.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.040+0000 7f8c727fc700 1 -- 172.21.15.3:0/3758054669 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c64011830 con 0x7f8c7409d6f0 2023-10-03T10:18:41.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.040+0000 7f8c727fc700 1 -- 172.21.15.3:0/3758054669 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c64011990 con 0x7f8c7409d6f0 2023-10-03T10:18:41.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.040+0000 7f8c727fc700 1 -- 172.21.15.3:0/3758054669 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c64011d20 con 0x7f8c7409d6f0 2023-10-03T10:18:41.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.040+0000 7f9877f9c700 1 Processor -- start 2023-10-03T10:18:41.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.040+0000 7f9877f9c700 1 -- start start 2023-10-03T10:18:41.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.040+0000 7f8c7a100700 1 -- 172.21.15.3:0/3758054669 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c7409d6f0 msgr2=0x7f8c7409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:41.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.040+0000 7f8c7a100700 1 --2- 172.21.15.3:0/3758054669 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c7409d6f0 0x7f8c7409dac0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f8c64009840 tx=0x7f8c64009b50 comp rx=0 tx=0).stop 2023-10-03T10:18:41.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.040+0000 7f8c7a100700 1 -- 172.21.15.3:0/3758054669 shutdown_connections 2023-10-03T10:18:41.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.040+0000 7f8c7a100700 1 --2- 172.21.15.3:0/3758054669 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c7409d6f0 0x7f8c7409dac0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:41.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.040+0000 7f8c7a100700 1 --2- 172.21.15.3:0/3758054669 >> v2:172.21.15.3:3300/0 conn(0x7f8c7409e000 0x7f8c74130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:41.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.041+0000 7f8c7a100700 1 -- 172.21.15.3:0/3758054669 >> 172.21.15.3:0/3758054669 conn(0x7f8c74066f50 msgr2=0x7f8c7409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:41.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.041+0000 7f8c7a100700 1 -- 172.21.15.3:0/3758054669 shutdown_connections 2023-10-03T10:18:41.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.041+0000 7f8c7a100700 1 -- 172.21.15.3:0/3758054669 wait complete. 2023-10-03T10:18:41.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.041+0000 7f9877f9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9870133420 0x7f987009be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:41.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.041+0000 7f9877f9c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f98701337f0 0x7f987012ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:41.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.041+0000 7f9877f9c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9870138dc0 0x7f987012f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:41.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.041+0000 7f9877f9c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987009b5e0 con 0x7f9870133420 2023-10-03T10:18:41.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.041+0000 7f9877f9c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9870138930 con 0x7f9870138dc0 2023-10-03T10:18:41.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.041+0000 7f9877f9c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f987012d510 con 0x7f98701337f0 2023-10-03T10:18:41.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.041+0000 7f9875537700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f98701337f0 0x7f987012ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:41.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.041+0000 7f9875537700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f98701337f0 0x7f987012ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42626/0 (socket says 172.21.15.3:42626) 2023-10-03T10:18:41.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.041+0000 7f9875537700 1 -- 172.21.15.3:0/2104306950 learned_addr learned my addr 172.21.15.3:0/2104306950 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:41.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.041+0000 7f9875d38700 1 --2- 172.21.15.3:0/2104306950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9870133420 0x7f987009be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:41.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.042+0000 7f9875d38700 1 -- 172.21.15.3:0/2104306950 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f98701337f0 msgr2=0x7f987012ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:41.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.042+0000 7f9875d38700 1 --2- 172.21.15.3:0/2104306950 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f98701337f0 0x7f987012ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:41.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.042+0000 7f9875d38700 1 -- 172.21.15.3:0/2104306950 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9870138dc0 msgr2=0x7f987012f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:41.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.042+0000 7f9875d38700 1 --2- 172.21.15.3:0/2104306950 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9870138dc0 0x7f987012f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:41.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.042+0000 7f9875d38700 1 -- 172.21.15.3:0/2104306950 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98600094a0 con 0x7f9870133420 2023-10-03T10:18:41.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.042+0000 7f9875537700 1 --2- 172.21.15.3:0/2104306950 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f98701337f0 0x7f987012ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:41.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.042+0000 7f9875d38700 1 --2- 172.21.15.3:0/2104306950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9870133420 0x7f987009be40 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f986c00c430 tx=0x7f986c00c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:41.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.042+0000 7f8c7a100700 1 Processor -- start 2023-10-03T10:18:41.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.042+0000 7f98667fc700 1 -- 172.21.15.3:0/2104306950 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c0054e0 con 0x7f9870133420 2023-10-03T10:18:41.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.042+0000 7f98667fc700 1 -- 172.21.15.3:0/2104306950 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986c003680 con 0x7f9870133420 2023-10-03T10:18:41.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.042+0000 7f9877f9c700 1 -- 172.21.15.3:0/2104306950 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9868009b30 con 0x7f9870133420 2023-10-03T10:18:41.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.042+0000 7f8c7a100700 1 -- start start 2023-10-03T10:18:41.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.043+0000 7f8c7a100700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c7409d6f0 0x7f8c7411de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:41.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.043+0000 7f8c7a100700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c7409e000 0x7f8c7411e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:41.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.043+0000 7f8c7a100700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c7409e410 0x7f8c7411e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:41.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.043+0000 7f8c7a100700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7411f270 con 0x7f8c7409e410 2023-10-03T10:18:41.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.043+0000 7f8c7a100700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c74120000 con 0x7f8c7409d6f0 2023-10-03T10:18:41.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.043+0000 7f8c7a100700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c741210c0 con 0x7f8c7409e000 2023-10-03T10:18:41.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.043+0000 7f8c72ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c7409e000 0x7f8c7411e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:41.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.043+0000 7f8c72ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c7409e000 0x7f8c7411e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:42634/0 (socket says 172.21.15.3:42634) 2023-10-03T10:18:41.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.043+0000 7f8c72ffd700 1 -- 172.21.15.3:0/3714025687 learned_addr learned my addr 172.21.15.3:0/3714025687 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:41.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.043+0000 7f8c72ffd700 1 -- 172.21.15.3:0/3714025687 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c7409d6f0 msgr2=0x7f8c7411de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:41.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.043+0000 7f8c72ffd700 1 --2- 172.21.15.3:0/3714025687 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c7409d6f0 0x7f8c7411de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:41.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.043+0000 7f8c72ffd700 1 -- 172.21.15.3:0/3714025687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c7409e410 msgr2=0x7f8c7411e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:41.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.043+0000 7f8c72ffd700 1 --2- 172.21.15.3:0/3714025687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c7409e410 0x7f8c7411e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:41.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.043+0000 7f8c72ffd700 1 -- 172.21.15.3:0/3714025687 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c640094a0 con 0x7f8c7409e000 2023-10-03T10:18:41.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.043+0000 7f8c72ffd700 1 --2- 172.21.15.3:0/3714025687 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c7409e000 0x7f8c7411e350 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800e910 tx=0x7f8c6800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:41.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.044+0000 7f8c5bfff700 1 -- 172.21.15.3:0/3714025687 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6800c850 con 0x7f8c7409e000 2023-10-03T10:18:41.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.044+0000 7f8c5bfff700 1 -- 172.21.15.3:0/3714025687 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6800c9b0 con 0x7f8c7409e000 2023-10-03T10:18:41.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.044+0000 7f8c5bfff700 1 -- 172.21.15.3:0/3714025687 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6800cd20 con 0x7f8c7409e000 2023-10-03T10:18:41.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.044+0000 7f8c7a100700 1 -- 172.21.15.3:0/3714025687 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c7409c700 con 0x7f8c7409e000 2023-10-03T10:18:41.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.044+0000 7f8c7a100700 1 -- 172.21.15.3:0/3714025687 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c7411fa20 con 0x7f8c7409e000 2023-10-03T10:18:41.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.042+0000 7f98667fc700 1 -- 172.21.15.3:0/2104306950 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c003a30 con 0x7f9870133420 2023-10-03T10:18:41.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.042+0000 7f9877f9c700 1 -- 172.21.15.3:0/2104306950 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f987009c960 con 0x7f9870133420 2023-10-03T10:18:41.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.045+0000 7f9853fff700 1 -- 172.21.15.3:0/2104306950 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9870131fb0 con 0x7f9870133420 2023-10-03T10:18:41.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.045+0000 7f8c5bfff700 1 -- 172.21.15.3:0/3714025687 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8c68013470 con 0x7f8c7409e000 2023-10-03T10:18:41.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.045+0000 7f8c5bfff700 1 -- 172.21.15.3:0/3714025687 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8c6803fbe0 con 0x7f8c7409e000 2023-10-03T10:18:41.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.046+0000 7f98667fc700 1 -- 172.21.15.3:0/2104306950 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f986c003b90 con 0x7f9870133420 2023-10-03T10:18:41.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.046+0000 7f98667fc700 1 -- 172.21.15.3:0/2104306950 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f986c03eb60 con 0x7f9870133420 2023-10-03T10:18:41.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.046+0000 7f8c7a100700 1 -- 172.21.15.3:0/3714025687 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c7411ac20 con 0x7f8c7409e000 2023-10-03T10:18:41.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.055+0000 7f8c5bfff700 1 -- 172.21.15.3:0/3714025687 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8c68017020 con 0x7f8c7409e000 2023-10-03T10:18:41.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.055+0000 7f98667fc700 1 -- 172.21.15.3:0/2104306950 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f986c003e40 con 0x7f9870133420 2023-10-03T10:18:41.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.182+0000 7f4828e52700 1 -- 172.21.15.3:0/734620170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f482406b250 con 0x7f482409e930 2023-10-03T10:18:41.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.182+0000 7f481affd700 1 -- 172.21.15.3:0/734620170 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4814029090 con 0x7f482409e930 2023-10-03T10:18:41.183 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:41.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.184+0000 7f4828e52700 1 -- 172.21.15.3:0/734620170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f482409e930 msgr2=0x7f48241a3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:41.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.184+0000 7f4828e52700 1 --2- 172.21.15.3:0/734620170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f482409e930 0x7f48241a3d50 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f481400b450 tx=0x7f481400b760 comp rx=0 tx=0).stop 2023-10-03T10:18:41.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.184+0000 7f4828e52700 1 -- 172.21.15.3:0/734620170 shutdown_connections 2023-10-03T10:18:41.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.184+0000 7f4828e52700 1 --2- 172.21.15.3:0/734620170 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f482409f650 0x7f48241a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:41.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.184+0000 7f4828e52700 1 --2- 172.21.15.3:0/734620170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f482409e930 0x7f48241a3d50 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:41.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.184+0000 7f4828e52700 1 --2- 172.21.15.3:0/734620170 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f482409f240 0x7f48241a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:41.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.184+0000 7f4828e52700 1 -- 172.21.15.3:0/734620170 >> 172.21.15.3:0/734620170 conn(0x7f48240a4210 msgr2=0x7f48241b8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:41.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.184+0000 7f4828e52700 1 -- 172.21.15.3:0/734620170 shutdown_connections 2023-10-03T10:18:41.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.184+0000 7f4828e52700 1 -- 172.21.15.3:0/734620170 wait complete. 2023-10-03T10:18:41.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.188+0000 7fc0617fa700 1 -- 172.21.15.3:0/3709996434 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc054005860 con 0x7fc07c09e000 2023-10-03T10:18:41.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.188+0000 7fc0637fe700 1 -- 172.21.15.3:0/3709996434 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc070015070 con 0x7fc07c09e000 2023-10-03T10:18:41.189 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:41.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.191+0000 7fc0617fa700 1 -- 172.21.15.3:0/3709996434 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc07c09e000 msgr2=0x7fc07c1a5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:41.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.191+0000 7fc0617fa700 1 --2- 172.21.15.3:0/3709996434 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc07c09e000 0x7fc07c1a5de0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fc07c09e9b0 tx=0x7fc070004750 comp rx=0 tx=0).stop 2023-10-03T10:18:41.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.191+0000 7fc0617fa700 1 -- 172.21.15.3:0/3709996434 shutdown_connections 2023-10-03T10:18:41.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.191+0000 7fc0617fa700 1 --2- 172.21.15.3:0/3709996434 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc07c09e410 0x7fc07c1a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:41.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.191+0000 7fc0617fa700 1 --2- 172.21.15.3:0/3709996434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc07c09d6f0 0x7fc07c1b91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:41.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.191+0000 7fc0617fa700 1 --2- 172.21.15.3:0/3709996434 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc07c09e000 0x7fc07c1a5de0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:41.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.191+0000 7fc0617fa700 1 -- 172.21.15.3:0/3709996434 >> 172.21.15.3:0/3709996434 conn(0x7fc07c066f50 msgr2=0x7fc07c1badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:41.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.191+0000 7fc0617fa700 1 -- 172.21.15.3:0/3709996434 shutdown_connections 2023-10-03T10:18:41.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.191+0000 7fc0617fa700 1 -- 172.21.15.3:0/3709996434 wait complete. 2023-10-03T10:18:41.201 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:41.206 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:41.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.263+0000 7f8c7a100700 1 -- 172.21.15.3:0/3714025687 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8c74074ea0 con 0x7f8c7409e000 2023-10-03T10:18:41.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.264+0000 7f8c5bfff700 1 -- 172.21.15.3:0/3714025687 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8c6802b090 con 0x7f8c7409e000 2023-10-03T10:18:41.264 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:41.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.266+0000 7f8c7a100700 1 -- 172.21.15.3:0/3714025687 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c7409e000 msgr2=0x7f8c7411e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:41.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.266+0000 7f8c7a100700 1 --2- 172.21.15.3:0/3714025687 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c7409e000 0x7f8c7411e350 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800e910 tx=0x7f8c6800ecd0 comp rx=0 tx=0).stop 2023-10-03T10:18:41.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.266+0000 7f8c7a100700 1 -- 172.21.15.3:0/3714025687 shutdown_connections 2023-10-03T10:18:41.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.266+0000 7f8c7a100700 1 --2- 172.21.15.3:0/3714025687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c7409e410 0x7f8c7411e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:41.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.266+0000 7f8c7a100700 1 --2- 172.21.15.3:0/3714025687 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8c7409d6f0 0x7f8c7411de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:41.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.266+0000 7f8c7a100700 1 --2- 172.21.15.3:0/3714025687 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8c7409e000 0x7f8c7411e350 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:41.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.266+0000 7f8c7a100700 1 -- 172.21.15.3:0/3714025687 >> 172.21.15.3:0/3714025687 conn(0x7f8c74066f50 msgr2=0x7f8c74130a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:41.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.266+0000 7f8c7a100700 1 -- 172.21.15.3:0/3714025687 shutdown_connections 2023-10-03T10:18:41.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.266+0000 7f8c7a100700 1 -- 172.21.15.3:0/3714025687 wait complete. 2023-10-03T10:18:41.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.279+0000 7f9853fff700 1 -- 172.21.15.3:0/2104306950 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9870131d90 con 0x7f9870133420 2023-10-03T10:18:41.280 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:41.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.279+0000 7f98667fc700 1 -- 172.21.15.3:0/2104306950 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f986c0192c0 con 0x7f9870133420 2023-10-03T10:18:41.280 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:41.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.282+0000 7f9853fff700 1 -- 172.21.15.3:0/2104306950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9870133420 msgr2=0x7f987009be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:41.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.282+0000 7f9853fff700 1 --2- 172.21.15.3:0/2104306950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9870133420 0x7f987009be40 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f986c00c430 tx=0x7f986c00c740 comp rx=0 tx=0).stop 2023-10-03T10:18:41.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.282+0000 7f9853fff700 1 -- 172.21.15.3:0/2104306950 shutdown_connections 2023-10-03T10:18:41.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.282+0000 7f9853fff700 1 --2- 172.21.15.3:0/2104306950 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9870138dc0 0x7f987012f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:41.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.282+0000 7f9853fff700 1 --2- 172.21.15.3:0/2104306950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9870133420 0x7f987009be40 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:41.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.282+0000 7f9853fff700 1 --2- 172.21.15.3:0/2104306950 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f98701337f0 0x7f987012ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:41.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.282+0000 7f9853fff700 1 -- 172.21.15.3:0/2104306950 >> 172.21.15.3:0/2104306950 conn(0x7f98700a4210 msgr2=0x7f987013a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:41.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.282+0000 7f9853fff700 1 -- 172.21.15.3:0/2104306950 shutdown_connections 2023-10-03T10:18:41.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:41.282+0000 7f9853fff700 1 -- 172.21.15.3:0/2104306950 wait complete. 2023-10-03T10:18:41.295 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:42.202 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:42.207 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:42.280 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:42.295 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:42.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.354+0000 7f4c3bac2700 1 Processor -- start 2023-10-03T10:18:42.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.355+0000 7f4c3bac2700 1 -- start start 2023-10-03T10:18:42.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.355+0000 7f4c3bac2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4c3409e1d0 0x7f4c34138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:42.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.355+0000 7f4c3bac2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c34139170 0x7f4c3413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:42.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.355+0000 7f4c3bac2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c3409b120 con 0x7f4c3409d6f0 2023-10-03T10:18:42.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.355+0000 7f4c3bac2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3406b460 con 0x7f4c34139170 2023-10-03T10:18:42.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.355+0000 7f4c3bac2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3409a7e0 con 0x7f4c3409e1d0 2023-10-03T10:18:42.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.355+0000 7f4c3985e700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4c3409d6f0 0x7f4c3409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43746/0 (socket says 172.21.15.3:43746) 2023-10-03T10:18:42.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.355+0000 7f4c3985e700 1 -- 172.21.15.3:0/1123709314 learned_addr learned my addr 172.21.15.3:0/1123709314 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:42.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.356+0000 7f4c3a05f700 1 --2- 172.21.15.3:0/1123709314 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c34139170 0x7f4c3413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:42.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.356+0000 7f4c3905d700 1 --2- 172.21.15.3:0/1123709314 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4c3409e1d0 0x7f4c34138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:42.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.356+0000 7f4c3885c700 1 -- 172.21.15.3:0/1123709314 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3536024537 0 0) 0x7f4c3409b120 con 0x7f4c3409d6f0 2023-10-03T10:18:42.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.356+0000 7f4c3885c700 1 -- 172.21.15.3:0/1123709314 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c200035b0 con 0x7f4c3409d6f0 2023-10-03T10:18:42.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.356+0000 7f4c3a05f700 1 -- 172.21.15.3:0/1123709314 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4c3409e1d0 msgr2=0x7f4c34138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:42.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.356+0000 7f4c3a05f700 1 --2- 172.21.15.3:0/1123709314 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4c3409e1d0 0x7f4c34138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.356+0000 7f4c3a05f700 1 -- 172.21.15.3:0/1123709314 >> v1:172.21.15.3:6789/0 conn(0x7f4c3409d6f0 legacy=0x7f4c3409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:42.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.356+0000 7f4c3a05f700 1 -- 172.21.15.3:0/1123709314 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3409a500 con 0x7f4c34139170 2023-10-03T10:18:42.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.356+0000 7f4c3a05f700 1 --2- 172.21.15.3:0/1123709314 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c34139170 0x7f4c3413b550 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009dc0 tx=0x7f4c2c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f344edbe97a1350 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:42.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.357+0000 7f4c3885c700 1 -- 172.21.15.3:0/1123709314 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c00e070 con 0x7f4c34139170 2023-10-03T10:18:42.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.357+0000 7f4c3885c700 1 -- 172.21.15.3:0/1123709314 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2c00bb20 con 0x7f4c34139170 2023-10-03T10:18:42.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.357+0000 7f4c3885c700 1 -- 172.21.15.3:0/1123709314 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c012430 con 0x7f4c34139170 2023-10-03T10:18:42.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.357+0000 7f4c3bac2700 1 -- 172.21.15.3:0/1123709314 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c34139170 msgr2=0x7f4c3413b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:42.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.357+0000 7f4c3bac2700 1 --2- 172.21.15.3:0/1123709314 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c34139170 0x7f4c3413b550 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009dc0 tx=0x7f4c2c00b330 comp rx=0 tx=0).stop 2023-10-03T10:18:42.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.357+0000 7f4c3bac2700 1 -- 172.21.15.3:0/1123709314 shutdown_connections 2023-10-03T10:18:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.357+0000 7f4c3bac2700 1 --2- 172.21.15.3:0/1123709314 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c34139170 0x7f4c3413b550 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.357+0000 7f4c3bac2700 1 --2- 172.21.15.3:0/1123709314 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4c3409e1d0 0x7f4c34138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.357+0000 7f4c3bac2700 1 -- 172.21.15.3:0/1123709314 >> 172.21.15.3:0/1123709314 conn(0x7f4c34066f50 msgr2=0x7f4c3409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.358+0000 7f4c3bac2700 1 -- 172.21.15.3:0/1123709314 shutdown_connections 2023-10-03T10:18:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.358+0000 7f4c3bac2700 1 -- 172.21.15.3:0/1123709314 wait complete. 2023-10-03T10:18:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.359+0000 7f4c3bac2700 1 Processor -- start 2023-10-03T10:18:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.359+0000 7f4c3bac2700 1 -- start start 2023-10-03T10:18:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.360+0000 7f4c3bac2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3409d6f0 0x7f4c3411e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.360+0000 7f4c3bac2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c3409e1d0 0x7f4c3411ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.360+0000 7f4c3bac2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4c34139170 0x7f4c3411f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.360+0000 7f4c3bac2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3409c710 con 0x7f4c3409d6f0 2023-10-03T10:18:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.360+0000 7f4c3bac2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c34122050 con 0x7f4c3409e1d0 2023-10-03T10:18:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.360+0000 7f4c3bac2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4c34121aa0 con 0x7f4c34139170 2023-10-03T10:18:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.360+0000 7f4c3905d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c3409e1d0 0x7f4c3411ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.360+0000 7f4c3985e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3409d6f0 0x7f4c3411e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.360+0000 7f4c3905d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c3409e1d0 0x7f4c3411ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50548/0 (socket says 172.21.15.3:50548) 2023-10-03T10:18:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.360+0000 7f4c3905d700 1 -- 172.21.15.3:0/3656040537 learned_addr learned my addr 172.21.15.3:0/3656040537 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.360+0000 7f4c3a05f700 1 --2- 172.21.15.3:0/3656040537 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4c34139170 0x7f4c3411f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.361+0000 7f4c3905d700 1 -- 172.21.15.3:0/3656040537 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4c34139170 msgr2=0x7f4c3411f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.361+0000 7f4c3905d700 1 --2- 172.21.15.3:0/3656040537 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4c34139170 0x7f4c3411f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.361+0000 7f4c3905d700 1 -- 172.21.15.3:0/3656040537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3409d6f0 msgr2=0x7f4c3411e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.361+0000 7f4c3905d700 1 --2- 172.21.15.3:0/3656040537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3409d6f0 0x7f4c3411e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.361+0000 7f4c3905d700 1 -- 172.21.15.3:0/3656040537 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c009a70 con 0x7f4c3409e1d0 2023-10-03T10:18:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.361+0000 7f4c3985e700 1 --2- 172.21.15.3:0/3656040537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3409d6f0 0x7f4c3411e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.361+0000 7f4c3905d700 1 --2- 172.21.15.3:0/3656040537 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c3409e1d0 0x7f4c3411ee20 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400e9b0 tx=0x7f4c2400ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.361+0000 7f4c2a7fc700 1 -- 172.21.15.3:0/3656040537 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c24009710 con 0x7f4c3409e1d0 2023-10-03T10:18:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.361+0000 7f4c2a7fc700 1 -- 172.21.15.3:0/3656040537 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2400ee50 con 0x7f4c3409e1d0 2023-10-03T10:18:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.361+0000 7f4c3bac2700 1 -- 172.21.15.3:0/3656040537 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c34071eb0 con 0x7f4c3409e1d0 2023-10-03T10:18:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.361+0000 7f4c2a7fc700 1 -- 172.21.15.3:0/3656040537 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2400ca50 con 0x7f4c3409e1d0 2023-10-03T10:18:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.361+0000 7f4c3bac2700 1 -- 172.21.15.3:0/3656040537 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c34120c10 con 0x7f4c3409e1d0 2023-10-03T10:18:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.363+0000 7f4c3bac2700 1 -- 172.21.15.3:0/3656040537 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c34122190 con 0x7f4c3409e1d0 2023-10-03T10:18:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.363+0000 7f4c2a7fc700 1 -- 172.21.15.3:0/3656040537 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4c2400cbb0 con 0x7f4c3409e1d0 2023-10-03T10:18:42.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.363+0000 7f4c2a7fc700 1 -- 172.21.15.3:0/3656040537 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4c2401a360 con 0x7f4c3409e1d0 2023-10-03T10:18:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.373+0000 7f4c2a7fc700 1 -- 172.21.15.3:0/3656040537 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4c34122190 con 0x7f4c3409e1d0 2023-10-03T10:18:42.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.398+0000 7fccb729c700 1 Processor -- start 2023-10-03T10:18:42.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.398+0000 7fccb729c700 1 -- start start 2023-10-03T10:18:42.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.399+0000 7fccb729c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fccb009d6f0 0x7fccb009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:42.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.399+0000 7fccb729c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fccb009e000 0x7fccb0130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:42.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.399+0000 7fccb729c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fccb009e410 0x7fccb0130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:42.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.399+0000 7fccb729c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccb009f520 con 0x7fccb009e410 2023-10-03T10:18:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.399+0000 7fccb729c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb009f130 con 0x7fccb009d6f0 2023-10-03T10:18:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.399+0000 7fccb729c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fccb009e930 con 0x7fccb009e000 2023-10-03T10:18:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.399+0000 7fccb5038700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fccb009d6f0 0x7fccb009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.399+0000 7fccb5839700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fccb009e410 0x7fccb0130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.399+0000 7fccb5038700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fccb009d6f0 0x7fccb009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50554/0 (socket says 172.21.15.3:50554) 2023-10-03T10:18:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.399+0000 7fccb5038700 1 -- 172.21.15.3:0/3942951990 learned_addr learned my addr 172.21.15.3:0/3942951990 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.399+0000 7fccb4837700 1 --2- 172.21.15.3:0/3942951990 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fccb009e000 0x7fccb0130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.399+0000 7fccb5839700 1 -- 172.21.15.3:0/3942951990 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fccb009e000 msgr2=0x7fccb0130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.399+0000 7fccb5839700 1 --2- 172.21.15.3:0/3942951990 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fccb009e000 0x7fccb0130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.399+0000 7fccb5839700 1 -- 172.21.15.3:0/3942951990 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fccb009d6f0 msgr2=0x7fccb009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.399+0000 7fccb5839700 1 --2- 172.21.15.3:0/3942951990 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fccb009d6f0 0x7fccb009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.399+0000 7fccb5839700 1 -- 172.21.15.3:0/3942951990 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb009b2d0 con 0x7fccb009e410 2023-10-03T10:18:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.400+0000 7fccb5839700 1 --2- 172.21.15.3:0/3942951990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccb009e410 0x7fccb0130ba0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009d90 tx=0x7fcca800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363a4476fde16a52 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.400+0000 7fcca7fff700 1 -- 172.21.15.3:0/3942951990 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca800e070 con 0x7fccb009e410 2023-10-03T10:18:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.400+0000 7fcca7fff700 1 -- 172.21.15.3:0/3942951990 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca800bb20 con 0x7fccb009e410 2023-10-03T10:18:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.400+0000 7fcca7fff700 1 -- 172.21.15.3:0/3942951990 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca8012430 con 0x7fccb009e410 2023-10-03T10:18:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.401+0000 7fccb729c700 1 -- 172.21.15.3:0/3942951990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccb009e410 msgr2=0x7fccb0130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.401+0000 7fccb729c700 1 --2- 172.21.15.3:0/3942951990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccb009e410 0x7fccb0130ba0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009d90 tx=0x7fcca800b330 comp rx=0 tx=0).stop 2023-10-03T10:18:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.401+0000 7fccb729c700 1 -- 172.21.15.3:0/3942951990 shutdown_connections 2023-10-03T10:18:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.401+0000 7fccb729c700 1 --2- 172.21.15.3:0/3942951990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccb009e410 0x7fccb0130ba0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.401+0000 7fccb729c700 1 --2- 172.21.15.3:0/3942951990 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fccb009d6f0 0x7fccb009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.401+0000 7fccb729c700 1 --2- 172.21.15.3:0/3942951990 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fccb009e000 0x7fccb0130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.401+0000 7fccb729c700 1 -- 172.21.15.3:0/3942951990 >> 172.21.15.3:0/3942951990 conn(0x7fccb0066f50 msgr2=0x7fccb009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.401+0000 7fccb729c700 1 -- 172.21.15.3:0/3942951990 shutdown_connections 2023-10-03T10:18:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.401+0000 7fccb729c700 1 -- 172.21.15.3:0/3942951990 wait complete. 2023-10-03T10:18:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.402+0000 7fccb729c700 1 Processor -- start 2023-10-03T10:18:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.402+0000 7fccb729c700 1 -- start start 2023-10-03T10:18:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.402+0000 7fccb729c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fccb009d6f0 0x7fccb0122290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.402+0000 7fccb729c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fccb009e000 0x7fccb01227d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.402+0000 7fccb729c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccb009e410 0x7fccb0122d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.402+0000 7fccb729c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb009c700 con 0x7fccb009e410 2023-10-03T10:18:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.402+0000 7fccb729c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb0123f00 con 0x7fccb009e000 2023-10-03T10:18:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.402+0000 7fccb729c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fccb0125a20 con 0x7fccb009d6f0 2023-10-03T10:18:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.403+0000 7fccb4837700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fccb009e000 0x7fccb01227d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.403+0000 7fccb4837700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fccb009e000 0x7fccb01227d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50562/0 (socket says 172.21.15.3:50562) 2023-10-03T10:18:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.403+0000 7fccb4837700 1 -- 172.21.15.3:0/1056215981 learned_addr learned my addr 172.21.15.3:0/1056215981 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.403+0000 7fccb5038700 1 --2- 172.21.15.3:0/1056215981 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fccb009d6f0 0x7fccb0122290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.403+0000 7fccb5839700 1 --2- 172.21.15.3:0/1056215981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccb009e410 0x7fccb0122d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.403+0000 7fccb4837700 1 -- 172.21.15.3:0/1056215981 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fccb009d6f0 msgr2=0x7fccb0122290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.403+0000 7fccb4837700 1 --2- 172.21.15.3:0/1056215981 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fccb009d6f0 0x7fccb0122290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.403+0000 7fccb4837700 1 -- 172.21.15.3:0/1056215981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccb009e410 msgr2=0x7fccb0122d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.403+0000 7fccb4837700 1 --2- 172.21.15.3:0/1056215981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccb009e410 0x7fccb0122d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.403+0000 7fccb4837700 1 -- 172.21.15.3:0/1056215981 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca8009a70 con 0x7fccb009e000 2023-10-03T10:18:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.403+0000 7fccb4837700 1 --2- 172.21.15.3:0/1056215981 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fccb009e000 0x7fccb01227d0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fcca000e980 tx=0x7fcca000ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.403+0000 7fcca5ffb700 1 -- 172.21.15.3:0/1056215981 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca0009710 con 0x7fccb009e000 2023-10-03T10:18:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.404+0000 7fcca5ffb700 1 -- 172.21.15.3:0/1056215981 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca000c690 con 0x7fccb009e000 2023-10-03T10:18:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.404+0000 7fccb729c700 1 -- 172.21.15.3:0/1056215981 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb00a3980 con 0x7fccb009e000 2023-10-03T10:18:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.404+0000 7fcca5ffb700 1 -- 172.21.15.3:0/1056215981 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca000ca20 con 0x7fccb009e000 2023-10-03T10:18:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.404+0000 7fccb729c700 1 -- 172.21.15.3:0/1056215981 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb0126490 con 0x7fccb009e000 2023-10-03T10:18:42.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.405+0000 7fcca5ffb700 1 -- 172.21.15.3:0/1056215981 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcca000cb80 con 0x7fccb009e000 2023-10-03T10:18:42.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.405+0000 7fcca5ffb700 1 -- 172.21.15.3:0/1056215981 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fcca003faa0 con 0x7fccb009e000 2023-10-03T10:18:42.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.405+0000 7fccb729c700 1 -- 172.21.15.3:0/1056215981 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccb0074ea0 con 0x7fccb009e000 2023-10-03T10:18:42.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.410+0000 7fcca5ffb700 1 -- 172.21.15.3:0/1056215981 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcca001b3b0 con 0x7fccb009e000 2023-10-03T10:18:42.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.444+0000 7f6c185d7700 1 Processor -- start 2023-10-03T10:18:42.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.444+0000 7f6c185d7700 1 -- start start 2023-10-03T10:18:42.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.445+0000 7f6c185d7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6c1006e570 0x7f6c1006e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:42.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.445+0000 7f6c185d7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6c1006ee80 0x7f6c10134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:42.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.445+0000 7f6c185d7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c10070ee0 con 0x7f6c1009eca0 2023-10-03T10:18:42.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.445+0000 7f6c185d7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c10071480 con 0x7f6c1006ee80 2023-10-03T10:18:42.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.445+0000 7f6c185d7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c10071b90 con 0x7f6c1006e570 2023-10-03T10:18:42.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.445+0000 7f6c16b74700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6c1006ee80 0x7f6c10134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:42.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.445+0000 7f6c16b74700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6c1006ee80 0x7f6c10134d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50576/0 (socket says 172.21.15.3:50576) 2023-10-03T10:18:42.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.445+0000 7f6c16b74700 1 -- 172.21.15.3:0/2428490261 learned_addr learned my addr 172.21.15.3:0/2428490261 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:42.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.445+0000 7f6c16373700 1 --2- 172.21.15.3:0/2428490261 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6c1006e570 0x7f6c1006e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:42.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.445+0000 7f6c15371700 1 -- 172.21.15.3:0/2428490261 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1831283013 0 0) 0x7f6c10070ee0 con 0x7f6c1009eca0 2023-10-03T10:18:42.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.445+0000 7f6c15371700 1 -- 172.21.15.3:0/2428490261 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bfc0035b0 con 0x7f6c1009eca0 2023-10-03T10:18:42.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.445+0000 7f6c16373700 1 -- 172.21.15.3:0/2428490261 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6c1006ee80 msgr2=0x7f6c10134d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:42.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.445+0000 7f6c16373700 1 --2- 172.21.15.3:0/2428490261 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6c1006ee80 0x7f6c10134d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.445+0000 7f6c16373700 1 -- 172.21.15.3:0/2428490261 >> v1:172.21.15.3:6789/0 conn(0x7f6c1009eca0 legacy=0x7f6c10134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:42.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.445+0000 7f6c16373700 1 -- 172.21.15.3:0/2428490261 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c100711a0 con 0x7f6c1006e570 2023-10-03T10:18:42.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.445+0000 7f6c16b74700 1 --2- 172.21.15.3:0/2428490261 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6c1006ee80 0x7f6c10134d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:42.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.446+0000 7f6c16373700 1 --2- 172.21.15.3:0/2428490261 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6c1006e570 0x7f6c1006e940 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009870 tx=0x7f6c0c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b39fb16ba3f61103 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:42.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.446+0000 7f6c15371700 1 -- 172.21.15.3:0/2428490261 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c011830 con 0x7f6c1006e570 2023-10-03T10:18:42.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.446+0000 7f6c15371700 1 -- 172.21.15.3:0/2428490261 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0c011990 con 0x7f6c1006e570 2023-10-03T10:18:42.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.446+0000 7f6c15371700 1 -- 172.21.15.3:0/2428490261 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c011d00 con 0x7f6c1006e570 2023-10-03T10:18:42.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.447+0000 7f6c185d7700 1 -- 172.21.15.3:0/2428490261 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6c1006e570 msgr2=0x7f6c1006e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:42.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.447+0000 7f6c185d7700 1 --2- 172.21.15.3:0/2428490261 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6c1006e570 0x7f6c1006e940 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009870 tx=0x7f6c0c009b80 comp rx=0 tx=0).stop 2023-10-03T10:18:42.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.447+0000 7f6c185d7700 1 -- 172.21.15.3:0/2428490261 shutdown_connections 2023-10-03T10:18:42.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.447+0000 7f6c185d7700 1 --2- 172.21.15.3:0/2428490261 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6c1006ee80 0x7f6c10134d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.447+0000 7f6c185d7700 1 --2- 172.21.15.3:0/2428490261 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6c1006e570 0x7f6c1006e940 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.447+0000 7f6c185d7700 1 -- 172.21.15.3:0/2428490261 >> 172.21.15.3:0/2428490261 conn(0x7f6c100a2aa0 msgr2=0x7f6c100a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:42.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.447+0000 7f6c185d7700 1 -- 172.21.15.3:0/2428490261 shutdown_connections 2023-10-03T10:18:42.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.447+0000 7f6c185d7700 1 -- 172.21.15.3:0/2428490261 wait complete. 2023-10-03T10:18:42.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.448+0000 7f6c185d7700 1 Processor -- start 2023-10-03T10:18:42.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.448+0000 7f6c185d7700 1 -- start start 2023-10-03T10:18:42.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.448+0000 7f6c185d7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6c1006e570 0x7f6c1013bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:42.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.448+0000 7f6c185d7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c1006ee80 0x7f6c10125ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:42.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.448+0000 7f6c185d7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6c1013ec90 0x7f6c101263e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:42.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.449+0000 7f6c185d7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c101275a0 con 0x7f6c1006ee80 2023-10-03T10:18:42.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.449+0000 7f6c185d7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1013d130 con 0x7f6c1013ec90 2023-10-03T10:18:42.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.449+0000 7f6c185d7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c10127790 con 0x7f6c1006e570 2023-10-03T10:18:42.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.449+0000 7f6c15b72700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c1006ee80 0x7f6c10125ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:42.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.449+0000 7f6c15b72700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c1006ee80 0x7f6c10125ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60594/0 (socket says 172.21.15.3:60594) 2023-10-03T10:18:42.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.449+0000 7f6c15b72700 1 -- 172.21.15.3:0/1667146326 learned_addr learned my addr 172.21.15.3:0/1667146326 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:42.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.449+0000 7f6c16373700 1 --2- 172.21.15.3:0/1667146326 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6c1006e570 0x7f6c1013bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:42.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.449+0000 7f6c16b74700 1 --2- 172.21.15.3:0/1667146326 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6c1013ec90 0x7f6c101263e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:42.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.449+0000 7f6c15b72700 1 -- 172.21.15.3:0/1667146326 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6c1006e570 msgr2=0x7f6c1013bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:42.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.449+0000 7f6c15b72700 1 --2- 172.21.15.3:0/1667146326 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6c1006e570 0x7f6c1013bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.449+0000 7f6c15b72700 1 -- 172.21.15.3:0/1667146326 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6c1013ec90 msgr2=0x7f6c101263e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:42.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.449+0000 7f6c15b72700 1 --2- 172.21.15.3:0/1667146326 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6c1013ec90 0x7f6c101263e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.449+0000 7f6c15b72700 1 -- 172.21.15.3:0/1667146326 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c08009b30 con 0x7f6c1006ee80 2023-10-03T10:18:42.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.449+0000 7f6c15b72700 1 --2- 172.21.15.3:0/1667146326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c1006ee80 0x7f6c10125ea0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f6c0000c530 tx=0x7f6c0000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:42.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.450+0000 7f6c06ffd700 1 -- 172.21.15.3:0/1667146326 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c000052d0 con 0x7f6c1006ee80 2023-10-03T10:18:42.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.450+0000 7f6c06ffd700 1 -- 172.21.15.3:0/1667146326 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c00005430 con 0x7f6c1006ee80 2023-10-03T10:18:42.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.450+0000 7f6c185d7700 1 -- 172.21.15.3:0/1667146326 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c0c0094a0 con 0x7f6c1006ee80 2023-10-03T10:18:42.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.450+0000 7f6c06ffd700 1 -- 172.21.15.3:0/1667146326 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c000038c0 con 0x7f6c1006ee80 2023-10-03T10:18:42.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.450+0000 7f6c185d7700 1 -- 172.21.15.3:0/1667146326 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c1013e560 con 0x7f6c1006ee80 2023-10-03T10:18:42.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.451+0000 7f6c185d7700 1 -- 172.21.15.3:0/1667146326 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c1013e370 con 0x7f6c1006ee80 2023-10-03T10:18:42.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.452+0000 7f6c06ffd700 1 -- 172.21.15.3:0/1667146326 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6c00003a20 con 0x7f6c1006ee80 2023-10-03T10:18:42.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.452+0000 7f6c06ffd700 1 -- 172.21.15.3:0/1667146326 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6c0003ef30 con 0x7f6c1006ee80 2023-10-03T10:18:42.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.461+0000 7f6c06ffd700 1 -- 172.21.15.3:0/1667146326 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6c00011590 con 0x7f6c1006ee80 2023-10-03T10:18:42.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.510+0000 7fcdef6fb700 1 Processor -- start 2023-10-03T10:18:42.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.510+0000 7fcdef6fb700 1 -- start start 2023-10-03T10:18:42.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.511+0000 7fcdef6fb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcde80a0a30 0x7fcde80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:42.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.511+0000 7fcdef6fb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcde809d630 0x7fcde80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:42.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.511+0000 7fcdef6fb700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcde80a2a70 con 0x7fcde8133930 2023-10-03T10:18:42.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.511+0000 7fcdef6fb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde80a01d0 con 0x7fcde809d630 2023-10-03T10:18:42.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.511+0000 7fcdef6fb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde809e2b0 con 0x7fcde80a0a30 2023-10-03T10:18:42.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.511+0000 7fcded497700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcde80a0a30 0x7fcde80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:42.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.511+0000 7fcdecc96700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcde809d630 0x7fcde80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:42.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.511+0000 7fcded497700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcde80a0a30 0x7fcde80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:50580/0 (socket says 172.21.15.3:50580) 2023-10-03T10:18:42.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.511+0000 7fcdecc96700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcde809d630 0x7fcde80a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60596/0 (socket says 172.21.15.3:60596) 2023-10-03T10:18:42.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.511+0000 7fcded497700 1 -- 172.21.15.3:0/3903013210 learned_addr learned my addr 172.21.15.3:0/3903013210 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:42.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.512+0000 7fcddffff700 1 -- 172.21.15.3:0/3903013210 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 628773428 0 0) 0x7fcde80a2a70 con 0x7fcde8133930 2023-10-03T10:18:42.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.512+0000 7fcded497700 1 -- 172.21.15.3:0/3903013210 >> v2:172.21.15.3:3300/0 conn(0x7fcde809d630 msgr2=0x7fcde80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:42.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.512+0000 7fcded497700 1 --2- 172.21.15.3:0/3903013210 >> v2:172.21.15.3:3300/0 conn(0x7fcde809d630 0x7fcde80a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.512+0000 7fcded497700 1 -- 172.21.15.3:0/3903013210 >> v1:172.21.15.3:6789/0 conn(0x7fcde8133930 legacy=0x7fcde8133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:42.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.512+0000 7fcded497700 1 -- 172.21.15.3:0/3903013210 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde809dc50 con 0x7fcde80a0a30 2023-10-03T10:18:42.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.512+0000 7fcded497700 1 --2- 172.21.15.3:0/3903013210 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcde80a0a30 0x7fcde80a0e00 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fcde40098f0 tx=0x7fcde4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=436a04f73d312025 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:42.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.512+0000 7fcddffff700 1 -- 172.21.15.3:0/3903013210 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde4011830 con 0x7fcde80a0a30 2023-10-03T10:18:42.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.512+0000 7fcddffff700 1 -- 172.21.15.3:0/3903013210 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde4011990 con 0x7fcde80a0a30 2023-10-03T10:18:42.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.512+0000 7fcddffff700 1 -- 172.21.15.3:0/3903013210 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde4011d20 con 0x7fcde80a0a30 2023-10-03T10:18:42.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.513+0000 7fcdef6fb700 1 -- 172.21.15.3:0/3903013210 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcde80a0a30 msgr2=0x7fcde80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:42.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.513+0000 7fcdef6fb700 1 --2- 172.21.15.3:0/3903013210 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcde80a0a30 0x7fcde80a0e00 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fcde40098f0 tx=0x7fcde4009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:42.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.513+0000 7fcdef6fb700 1 -- 172.21.15.3:0/3903013210 shutdown_connections 2023-10-03T10:18:42.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.513+0000 7fcdef6fb700 1 --2- 172.21.15.3:0/3903013210 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcde80a0a30 0x7fcde80a0e00 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.513+0000 7fcdef6fb700 1 --2- 172.21.15.3:0/3903013210 >> v2:172.21.15.3:3300/0 conn(0x7fcde809d630 0x7fcde80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.513+0000 7fcdef6fb700 1 -- 172.21.15.3:0/3903013210 >> 172.21.15.3:0/3903013210 conn(0x7fcde80a3050 msgr2=0x7fcde80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:42.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.513+0000 7fcdef6fb700 1 -- 172.21.15.3:0/3903013210 shutdown_connections 2023-10-03T10:18:42.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.513+0000 7fcdef6fb700 1 -- 172.21.15.3:0/3903013210 wait complete. 2023-10-03T10:18:42.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.515+0000 7fcdef6fb700 1 Processor -- start 2023-10-03T10:18:42.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.515+0000 7fcdef6fb700 1 -- start start 2023-10-03T10:18:42.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.516+0000 7fcdef6fb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcde809d630 0x7fcde806dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:42.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.516+0000 7fcdef6fb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcde80a0a30 0x7fcde806e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:42.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.516+0000 7fcdef6fb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcde8133930 0x7fcde806e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:42.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.516+0000 7fcdef6fb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde80715e0 con 0x7fcde809d630 2023-10-03T10:18:42.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.516+0000 7fcdef6fb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde806f9f0 con 0x7fcde80a0a30 2023-10-03T10:18:42.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.516+0000 7fcdef6fb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcde80a5410 con 0x7fcde8133930 2023-10-03T10:18:42.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.516+0000 7fcded497700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcde809d630 0x7fcde806dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:42.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.516+0000 7fcded497700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcde809d630 0x7fcde806dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60604/0 (socket says 172.21.15.3:60604) 2023-10-03T10:18:42.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.516+0000 7fcded497700 1 -- 172.21.15.3:0/1700389764 learned_addr learned my addr 172.21.15.3:0/1700389764 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:42.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.516+0000 7fcdedc98700 1 --2- 172.21.15.3:0/1700389764 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcde8133930 0x7fcde806e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:42.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.516+0000 7fcdecc96700 1 --2- 172.21.15.3:0/1700389764 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcde80a0a30 0x7fcde806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:42.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.516+0000 7fcded497700 1 -- 172.21.15.3:0/1700389764 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcde8133930 msgr2=0x7fcde806e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:42.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.516+0000 7fcded497700 1 --2- 172.21.15.3:0/1700389764 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcde8133930 0x7fcde806e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.516+0000 7fcded497700 1 -- 172.21.15.3:0/1700389764 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcde80a0a30 msgr2=0x7fcde806e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:42.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.516+0000 7fcded497700 1 --2- 172.21.15.3:0/1700389764 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcde80a0a30 0x7fcde806e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.516+0000 7fcded497700 1 -- 172.21.15.3:0/1700389764 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde4009550 con 0x7fcde809d630 2023-10-03T10:18:42.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.517+0000 7fcded497700 1 --2- 172.21.15.3:0/1700389764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcde809d630 0x7fcde806dac0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fcde4009c40 tx=0x7fcde4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:42.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.517+0000 7fcdddffb700 1 -- 172.21.15.3:0/1700389764 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde4004550 con 0x7fcde809d630 2023-10-03T10:18:42.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.517+0000 7fcdddffb700 1 -- 172.21.15.3:0/1700389764 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde40036a0 con 0x7fcde809d630 2023-10-03T10:18:42.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.517+0000 7fcdddffb700 1 -- 172.21.15.3:0/1700389764 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde4003a30 con 0x7fcde809d630 2023-10-03T10:18:42.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.517+0000 7fcdef6fb700 1 -- 172.21.15.3:0/1700389764 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde80a0410 con 0x7fcde809d630 2023-10-03T10:18:42.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.517+0000 7fcdef6fb700 1 -- 172.21.15.3:0/1700389764 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde8070080 con 0x7fcde809d630 2023-10-03T10:18:42.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.519+0000 7fcdef6fb700 1 -- 172.21.15.3:0/1700389764 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde8071180 con 0x7fcde809d630 2023-10-03T10:18:42.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.524+0000 7fcdddffb700 1 -- 172.21.15.3:0/1700389764 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcde4021070 con 0x7fcde809d630 2023-10-03T10:18:42.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.525+0000 7fcdddffb700 1 -- 172.21.15.3:0/1700389764 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fcde4040f90 con 0x7fcde809d630 2023-10-03T10:18:42.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.529+0000 7fcdddffb700 1 -- 172.21.15.3:0/1700389764 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcde4019730 con 0x7fcde809d630 2023-10-03T10:18:42.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.616+0000 7f4c3bac2700 1 -- 172.21.15.3:0/3656040537 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4c34122190 con 0x7f4c3409e1d0 2023-10-03T10:18:42.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.617+0000 7f4c2a7fc700 1 -- 172.21.15.3:0/3656040537 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4c34122190 con 0x7f4c3409e1d0 2023-10-03T10:18:42.617 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:42.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.620+0000 7f4c3bac2700 1 -- 172.21.15.3:0/3656040537 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c3409e1d0 msgr2=0x7f4c3411ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:42.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.620+0000 7f4c3bac2700 1 --2- 172.21.15.3:0/3656040537 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c3409e1d0 0x7f4c3411ee20 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400e9b0 tx=0x7f4c2400ecc0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.620+0000 7f4c3bac2700 1 -- 172.21.15.3:0/3656040537 shutdown_connections 2023-10-03T10:18:42.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.620+0000 7f4c3bac2700 1 --2- 172.21.15.3:0/3656040537 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4c34139170 0x7f4c3411f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.620+0000 7f4c3bac2700 1 --2- 172.21.15.3:0/3656040537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3409d6f0 0x7f4c3411e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.620+0000 7f4c3bac2700 1 --2- 172.21.15.3:0/3656040537 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c3409e1d0 0x7f4c3411ee20 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.620+0000 7f4c3bac2700 1 -- 172.21.15.3:0/3656040537 >> 172.21.15.3:0/3656040537 conn(0x7f4c34066f50 msgr2=0x7f4c34136b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:42.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.620+0000 7f4c3bac2700 1 -- 172.21.15.3:0/3656040537 shutdown_connections 2023-10-03T10:18:42.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.620+0000 7f4c3bac2700 1 -- 172.21.15.3:0/3656040537 wait complete. 2023-10-03T10:18:42.635 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:42.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.671+0000 7fccb729c700 1 -- 172.21.15.3:0/1056215981 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fccb006c360 con 0x7fccb009e000 2023-10-03T10:18:42.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.672+0000 7fcca5ffb700 1 -- 172.21.15.3:0/1056215981 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcca0017020 con 0x7fccb009e000 2023-10-03T10:18:42.672 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:42.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.675+0000 7fccb729c700 1 -- 172.21.15.3:0/1056215981 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fccb009e000 msgr2=0x7fccb01227d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:42.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.675+0000 7fccb729c700 1 --2- 172.21.15.3:0/1056215981 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fccb009e000 0x7fccb01227d0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fcca000e980 tx=0x7fcca000ec90 comp rx=0 tx=0).stop 2023-10-03T10:18:42.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.675+0000 7fccb729c700 1 -- 172.21.15.3:0/1056215981 shutdown_connections 2023-10-03T10:18:42.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.675+0000 7fccb729c700 1 --2- 172.21.15.3:0/1056215981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccb009e410 0x7fccb0122d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.675+0000 7fccb729c700 1 --2- 172.21.15.3:0/1056215981 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fccb009d6f0 0x7fccb0122290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.675+0000 7fccb729c700 1 --2- 172.21.15.3:0/1056215981 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fccb009e000 0x7fccb01227d0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.675+0000 7fccb729c700 1 -- 172.21.15.3:0/1056215981 >> 172.21.15.3:0/1056215981 conn(0x7fccb0066f50 msgr2=0x7fccb0070d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:42.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.675+0000 7fccb729c700 1 -- 172.21.15.3:0/1056215981 shutdown_connections 2023-10-03T10:18:42.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.675+0000 7fccb729c700 1 -- 172.21.15.3:0/1056215981 wait complete. 2023-10-03T10:18:42.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.681+0000 7f6c185d7700 1 -- 172.21.15.3:0/1667146326 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6c10074ea0 con 0x7f6c1006ee80 2023-10-03T10:18:42.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.681+0000 7f6c06ffd700 1 -- 172.21.15.3:0/1667146326 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6c00017030 con 0x7f6c1006ee80 2023-10-03T10:18:42.681 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:42.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.683+0000 7f6c185d7700 1 -- 172.21.15.3:0/1667146326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c1006ee80 msgr2=0x7f6c10125ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:42.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.683+0000 7f6c185d7700 1 --2- 172.21.15.3:0/1667146326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c1006ee80 0x7f6c10125ea0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f6c0000c530 tx=0x7f6c0000c840 comp rx=0 tx=0).stop 2023-10-03T10:18:42.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.683+0000 7f6c185d7700 1 -- 172.21.15.3:0/1667146326 shutdown_connections 2023-10-03T10:18:42.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.683+0000 7f6c185d7700 1 --2- 172.21.15.3:0/1667146326 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6c1013ec90 0x7f6c101263e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.683+0000 7f6c185d7700 1 --2- 172.21.15.3:0/1667146326 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6c1006e570 0x7f6c1013bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.683+0000 7f6c185d7700 1 --2- 172.21.15.3:0/1667146326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c1006ee80 0x7f6c10125ea0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.683+0000 7f6c185d7700 1 -- 172.21.15.3:0/1667146326 >> 172.21.15.3:0/1667146326 conn(0x7f6c100a2aa0 msgr2=0x7f6c100a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:42.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.683+0000 7f6c185d7700 1 -- 172.21.15.3:0/1667146326 shutdown_connections 2023-10-03T10:18:42.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.683+0000 7f6c185d7700 1 -- 172.21.15.3:0/1667146326 wait complete. 2023-10-03T10:18:42.691 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:42.698 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:42.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.754+0000 7fcdef6fb700 1 -- 172.21.15.3:0/1700389764 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcde80747f0 con 0x7fcde809d630 2023-10-03T10:18:42.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.754+0000 7fcdddffb700 1 -- 172.21.15.3:0/1700389764 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcde4029030 con 0x7fcde809d630 2023-10-03T10:18:42.754 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:42.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.756+0000 7fcdef6fb700 1 -- 172.21.15.3:0/1700389764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcde809d630 msgr2=0x7fcde806dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:42.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.756+0000 7fcdef6fb700 1 --2- 172.21.15.3:0/1700389764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcde809d630 0x7fcde806dac0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fcde4009c40 tx=0x7fcde4004750 comp rx=0 tx=0).stop 2023-10-03T10:18:42.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.756+0000 7fcdef6fb700 1 -- 172.21.15.3:0/1700389764 shutdown_connections 2023-10-03T10:18:42.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.756+0000 7fcdef6fb700 1 --2- 172.21.15.3:0/1700389764 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcde8133930 0x7fcde806e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.756+0000 7fcdef6fb700 1 --2- 172.21.15.3:0/1700389764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcde809d630 0x7fcde806dac0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.756+0000 7fcdef6fb700 1 --2- 172.21.15.3:0/1700389764 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcde80a0a30 0x7fcde806e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:42.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.756+0000 7fcdef6fb700 1 -- 172.21.15.3:0/1700389764 >> 172.21.15.3:0/1700389764 conn(0x7fcde80a3050 msgr2=0x7fcde81bb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:42.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.756+0000 7fcdef6fb700 1 -- 172.21.15.3:0/1700389764 shutdown_connections 2023-10-03T10:18:42.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:42.756+0000 7fcdef6fb700 1 -- 172.21.15.3:0/1700389764 wait complete. 2023-10-03T10:18:42.770 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:43.636 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:43.692 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:43.699 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:43.771 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:43.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.788+0000 7eff7567e700 1 Processor -- start 2023-10-03T10:18:43.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.789+0000 7eff7567e700 1 -- start start 2023-10-03T10:18:43.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.789+0000 7eff7567e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7eff7009d6f0 0x7eff7009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:43.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.789+0000 7eff7567e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff7009e000 0x7eff70130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:43.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.789+0000 7eff7567e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff7009f130 con 0x7eff7009e410 2023-10-03T10:18:43.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.789+0000 7eff7567e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff7009f520 con 0x7eff7009d6f0 2023-10-03T10:18:43.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.789+0000 7eff7567e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7009e930 con 0x7eff7009e000 2023-10-03T10:18:43.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.789+0000 7eff6e7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff7009e000 0x7eff70130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:43.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.789+0000 7eff6effd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7eff7009d6f0 0x7eff7009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:43.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.789+0000 7eff6e7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff7009e000 0x7eff70130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44460/0 (socket says 172.21.15.3:44460) 2023-10-03T10:18:43.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.789+0000 7eff6e7fc700 1 -- 172.21.15.3:0/2468784822 learned_addr learned my addr 172.21.15.3:0/2468784822 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:43.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.790+0000 7eff6effd700 1 -- 172.21.15.3:0/2468784822 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff7009e000 msgr2=0x7eff70130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:43.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.790+0000 7eff6effd700 1 --2- 172.21.15.3:0/2468784822 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff7009e000 0x7eff70130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:43.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.790+0000 7eff6effd700 1 -- 172.21.15.3:0/2468784822 >> v1:172.21.15.3:6789/0 conn(0x7eff7009e410 legacy=0x7eff70130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:43.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.790+0000 7eff6effd700 1 -- 172.21.15.3:0/2468784822 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff7009b2d0 con 0x7eff7009d6f0 2023-10-03T10:18:43.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.790+0000 7eff6effd700 1 --2- 172.21.15.3:0/2468784822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff7009d6f0 0x7eff7009dac0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7eff6000b8f0 tx=0x7eff6000bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54f058ad8b188f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:43.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.790+0000 7eff6dffb700 1 -- 172.21.15.3:0/2468784822 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6000c9f0 con 0x7eff7009d6f0 2023-10-03T10:18:43.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.790+0000 7eff6dffb700 1 -- 172.21.15.3:0/2468784822 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff6000cb50 con 0x7eff7009d6f0 2023-10-03T10:18:43.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.790+0000 7eff6dffb700 1 -- 172.21.15.3:0/2468784822 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6000f460 con 0x7eff7009d6f0 2023-10-03T10:18:43.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.791+0000 7eff7567e700 1 -- 172.21.15.3:0/2468784822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff7009d6f0 msgr2=0x7eff7009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:43.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.791+0000 7eff7567e700 1 --2- 172.21.15.3:0/2468784822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff7009d6f0 0x7eff7009dac0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7eff6000b8f0 tx=0x7eff6000bc00 comp rx=0 tx=0).stop 2023-10-03T10:18:43.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.792+0000 7eff7567e700 1 -- 172.21.15.3:0/2468784822 shutdown_connections 2023-10-03T10:18:43.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.792+0000 7eff7567e700 1 --2- 172.21.15.3:0/2468784822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff7009d6f0 0x7eff7009dac0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:43.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.792+0000 7eff7567e700 1 --2- 172.21.15.3:0/2468784822 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff7009e000 0x7eff70130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:43.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.792+0000 7eff7567e700 1 -- 172.21.15.3:0/2468784822 >> 172.21.15.3:0/2468784822 conn(0x7eff70066f50 msgr2=0x7eff7009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:43.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.792+0000 7eff7567e700 1 -- 172.21.15.3:0/2468784822 shutdown_connections 2023-10-03T10:18:43.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.792+0000 7eff7567e700 1 -- 172.21.15.3:0/2468784822 wait complete. 2023-10-03T10:18:43.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.793+0000 7eff7567e700 1 Processor -- start 2023-10-03T10:18:43.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.793+0000 7eff7567e700 1 -- start start 2023-10-03T10:18:43.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.794+0000 7eff7567e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff7009d6f0 0x7eff7011dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:43.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.794+0000 7eff7567e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff7009e000 0x7eff7011e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:43.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.794+0000 7eff7567e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff7009e410 0x7eff7011ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:43.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.794+0000 7eff7567e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7011f450 con 0x7eff7009e410 2023-10-03T10:18:43.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.794+0000 7eff7567e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff70130390 con 0x7eff7009e000 2023-10-03T10:18:43.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.794+0000 7eff7567e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff7009c700 con 0x7eff7009d6f0 2023-10-03T10:18:43.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.794+0000 7eff6effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff7009d6f0 0x7eff7011dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:43.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.794+0000 7eff6f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff7009e410 0x7eff7011ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:43.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.794+0000 7eff6effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff7009d6f0 0x7eff7011dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:37576/0 (socket says 172.21.15.3:37576) 2023-10-03T10:18:43.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.794+0000 7eff6effd700 1 -- 172.21.15.3:0/2746724458 learned_addr learned my addr 172.21.15.3:0/2746724458 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:43.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.795+0000 7eff6e7fc700 1 --2- 172.21.15.3:0/2746724458 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff7009e000 0x7eff7011e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:43.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.795+0000 7eff6f7fe700 1 -- 172.21.15.3:0/2746724458 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff7009d6f0 msgr2=0x7eff7011dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:43.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.795+0000 7eff6f7fe700 1 --2- 172.21.15.3:0/2746724458 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff7009d6f0 0x7eff7011dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:43.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.795+0000 7eff6f7fe700 1 -- 172.21.15.3:0/2746724458 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff7009e000 msgr2=0x7eff7011e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:43.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.795+0000 7eff6f7fe700 1 --2- 172.21.15.3:0/2746724458 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff7009e000 0x7eff7011e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:43.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.795+0000 7eff6f7fe700 1 -- 172.21.15.3:0/2746724458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff6000b550 con 0x7eff7009e410 2023-10-03T10:18:43.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.795+0000 7eff6f7fe700 1 --2- 172.21.15.3:0/2746724458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff7009e410 0x7eff7011ea70 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7eff6800cb90 tx=0x7eff6800cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:43.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.795+0000 7eff577fe700 1 -- 172.21.15.3:0/2746724458 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff68003c00 con 0x7eff7009e410 2023-10-03T10:18:43.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.795+0000 7eff577fe700 1 -- 172.21.15.3:0/2746724458 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff68003d60 con 0x7eff7009e410 2023-10-03T10:18:43.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.795+0000 7eff577fe700 1 -- 172.21.15.3:0/2746724458 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6800aaa0 con 0x7eff7009e410 2023-10-03T10:18:43.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.795+0000 7eff7567e700 1 -- 172.21.15.3:0/2746724458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff701217f0 con 0x7eff7009e410 2023-10-03T10:18:43.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.796+0000 7eff7567e700 1 -- 172.21.15.3:0/2746724458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff70120e10 con 0x7eff7009e410 2023-10-03T10:18:43.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.797+0000 7eff7567e700 1 -- 172.21.15.3:0/2746724458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff70074ea0 con 0x7eff7009e410 2023-10-03T10:18:43.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.797+0000 7eff577fe700 1 -- 172.21.15.3:0/2746724458 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7eff6800ac00 con 0x7eff7009e410 2023-10-03T10:18:43.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.797+0000 7eff577fe700 1 -- 172.21.15.3:0/2746724458 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7eff6803f770 con 0x7eff7009e410 2023-10-03T10:18:43.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.806+0000 7eff577fe700 1 -- 172.21.15.3:0/2746724458 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7eff680196c0 con 0x7eff7009e410 2023-10-03T10:18:43.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.858+0000 7fca13c8d700 1 Processor -- start 2023-10-03T10:18:43.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.859+0000 7fca13c8d700 1 -- start start 2023-10-03T10:18:43.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.859+0000 7fca13c8d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fca0c09d6f0 0x7fca0c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.859+0000 7fca13c8d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca0c09e440 0x7fca0c134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.859+0000 7fca13c8d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca0c09b120 con 0x7fca0c09e000 2023-10-03T10:18:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.859+0000 7fca13c8d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca0c06b460 con 0x7fca0c09d6f0 2023-10-03T10:18:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.859+0000 7fca13c8d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c09a7e0 con 0x7fca0c09e440 2023-10-03T10:18:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.860+0000 7fca11a29700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fca0c09d6f0 0x7fca0c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.860+0000 7fca1222a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca0c09e440 0x7fca0c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.860+0000 7fca11a29700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fca0c09d6f0 0x7fca0c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52372/0 (socket says 172.21.15.3:52372) 2023-10-03T10:18:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.860+0000 7fca11228700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fca0c09e000 0x7fca0c1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47692/0 (socket says 172.21.15.3:47692) 2023-10-03T10:18:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.860+0000 7fca11a29700 1 -- 172.21.15.3:0/2730213787 learned_addr learned my addr 172.21.15.3:0/2730213787 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:43.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.860+0000 7fca1222a700 1 -- 172.21.15.3:0/2730213787 >> v2:172.21.15.3:3300/0 conn(0x7fca0c09d6f0 msgr2=0x7fca0c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:43.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.860+0000 7fca1222a700 1 --2- 172.21.15.3:0/2730213787 >> v2:172.21.15.3:3300/0 conn(0x7fca0c09d6f0 0x7fca0c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:43.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.860+0000 7fca1222a700 1 -- 172.21.15.3:0/2730213787 >> v1:172.21.15.3:6789/0 conn(0x7fca0c09e000 legacy=0x7fca0c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:43.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.860+0000 7fca1222a700 1 -- 172.21.15.3:0/2730213787 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0c09a500 con 0x7fca0c09e440 2023-10-03T10:18:43.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.860+0000 7fca1222a700 1 --2- 172.21.15.3:0/2730213787 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca0c09e440 0x7fca0c134100 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fca04009dc0 tx=0x7fca0400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf163c8d9d6cfc3f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:43.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.861+0000 7fca10a27700 1 -- 172.21.15.3:0/2730213787 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0400e070 con 0x7fca0c09e440 2023-10-03T10:18:43.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.861+0000 7fca10a27700 1 -- 172.21.15.3:0/2730213787 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0400bae0 con 0x7fca0c09e440 2023-10-03T10:18:43.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.861+0000 7fca10a27700 1 -- 172.21.15.3:0/2730213787 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0400be90 con 0x7fca0c09e440 2023-10-03T10:18:43.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.861+0000 7fca13c8d700 1 -- 172.21.15.3:0/2730213787 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca0c09e440 msgr2=0x7fca0c134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:43.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.861+0000 7fca13c8d700 1 --2- 172.21.15.3:0/2730213787 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca0c09e440 0x7fca0c134100 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fca04009dc0 tx=0x7fca0400b330 comp rx=0 tx=0).stop 2023-10-03T10:18:43.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.861+0000 7fca13c8d700 1 -- 172.21.15.3:0/2730213787 shutdown_connections 2023-10-03T10:18:43.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.861+0000 7fca13c8d700 1 --2- 172.21.15.3:0/2730213787 >> v2:172.21.15.3:3300/0 conn(0x7fca0c09d6f0 0x7fca0c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:43.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.861+0000 7fca13c8d700 1 --2- 172.21.15.3:0/2730213787 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca0c09e440 0x7fca0c134100 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:43.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.861+0000 7fca13c8d700 1 -- 172.21.15.3:0/2730213787 >> 172.21.15.3:0/2730213787 conn(0x7fca0c066f50 msgr2=0x7fca0c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:43.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.862+0000 7fca13c8d700 1 -- 172.21.15.3:0/2730213787 shutdown_connections 2023-10-03T10:18:43.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.862+0000 7fca13c8d700 1 -- 172.21.15.3:0/2730213787 wait complete. 2023-10-03T10:18:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.863+0000 7fca13c8d700 1 Processor -- start 2023-10-03T10:18:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.863+0000 7fca13c8d700 1 -- start start 2023-10-03T10:18:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.863+0000 7fca13c8d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca0c09d6f0 0x7fca0c1c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.863+0000 7fca13c8d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca0c09e000 0x7fca0c1c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.863+0000 7fca13c8d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca0c09e440 0x7fca0c1c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.863+0000 7fca13c8d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c1a80b0 con 0x7fca0c09d6f0 2023-10-03T10:18:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.863+0000 7fca13c8d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c0a1f90 con 0x7fca0c09e000 2023-10-03T10:18:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.863+0000 7fca13c8d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c1b9550 con 0x7fca0c09e440 2023-10-03T10:18:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.863+0000 7fca11a29700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca0c09d6f0 0x7fca0c1c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.863+0000 7fca11a29700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca0c09d6f0 0x7fca0c1c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52388/0 (socket says 172.21.15.3:52388) 2023-10-03T10:18:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.863+0000 7fca11a29700 1 -- 172.21.15.3:0/1391576250 learned_addr learned my addr 172.21.15.3:0/1391576250 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.863+0000 7fca1222a700 1 --2- 172.21.15.3:0/1391576250 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca0c09e440 0x7fca0c1c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.864+0000 7fca11228700 1 --2- 172.21.15.3:0/1391576250 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca0c09e000 0x7fca0c1c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:43.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.864+0000 7fca1222a700 1 -- 172.21.15.3:0/1391576250 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca0c09e000 msgr2=0x7fca0c1c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:43.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.864+0000 7fca1222a700 1 --2- 172.21.15.3:0/1391576250 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca0c09e000 0x7fca0c1c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:43.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.864+0000 7fca1222a700 1 -- 172.21.15.3:0/1391576250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca0c09d6f0 msgr2=0x7fca0c1c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:43.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.864+0000 7fca1222a700 1 --2- 172.21.15.3:0/1391576250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca0c09d6f0 0x7fca0c1c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:43.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.864+0000 7fca1222a700 1 -- 172.21.15.3:0/1391576250 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca04009a70 con 0x7fca0c09e440 2023-10-03T10:18:43.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.864+0000 7fca11228700 1 --2- 172.21.15.3:0/1391576250 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca0c09e000 0x7fca0c1c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:18:43.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.864+0000 7fca11a29700 1 --2- 172.21.15.3:0/1391576250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca0c09d6f0 0x7fca0c1c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:43.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.864+0000 7fca1222a700 1 --2- 172.21.15.3:0/1391576250 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca0c09e440 0x7fca0c1c49d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fca04016000 tx=0x7fca04004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:43.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.864+0000 7fca027fc700 1 -- 172.21.15.3:0/1391576250 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca04004680 con 0x7fca0c09e440 2023-10-03T10:18:43.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.864+0000 7fca027fc700 1 -- 172.21.15.3:0/1391576250 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca04020070 con 0x7fca0c09e440 2023-10-03T10:18:43.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.864+0000 7fca13c8d700 1 -- 172.21.15.3:0/1391576250 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca0c1a8980 con 0x7fca0c09e440 2023-10-03T10:18:43.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.864+0000 7fca027fc700 1 -- 172.21.15.3:0/1391576250 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca04027480 con 0x7fca0c09e440 2023-10-03T10:18:43.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.864+0000 7fca13c8d700 1 -- 172.21.15.3:0/1391576250 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca0c1a79c0 con 0x7fca0c09e440 2023-10-03T10:18:43.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.865+0000 7fca027fc700 1 -- 172.21.15.3:0/1391576250 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fca04012430 con 0x7fca0c09e440 2023-10-03T10:18:43.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.866+0000 7fca027fc700 1 -- 172.21.15.3:0/1391576250 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fca04022070 con 0x7fca0c09e440 2023-10-03T10:18:43.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.866+0000 7fca13c8d700 1 -- 172.21.15.3:0/1391576250 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca0c1a79c0 con 0x7fca0c09e440 2023-10-03T10:18:43.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.876+0000 7fca027fc700 1 -- 172.21.15.3:0/1391576250 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fca0c1a79c0 con 0x7fca0c09e440 2023-10-03T10:18:43.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.882+0000 7f62c3061700 1 Processor -- start 2023-10-03T10:18:43.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.882+0000 7f62c3061700 1 -- start start 2023-10-03T10:18:43.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.882+0000 7f62c3061700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f62bc06a770 0x7f62bc06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:43.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.882+0000 7f62c3061700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f62bc131d10 0x7f62bc139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:43.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.882+0000 7f62c3061700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62bc09ece0 con 0x7f62bc06b080 2023-10-03T10:18:43.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.882+0000 7f62c3061700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62bc09e760 con 0x7f62bc131d10 2023-10-03T10:18:43.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.882+0000 7f62c3061700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc09e040 con 0x7f62bc06a770 2023-10-03T10:18:43.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.882+0000 7f62bbfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f62bc06b080 0x7f62bc131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47702/0 (socket says 172.21.15.3:47702) 2023-10-03T10:18:43.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.882+0000 7f62bbfff700 1 -- 172.21.15.3:0/2930079479 learned_addr learned my addr 172.21.15.3:0/2930079479 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:43.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.882+0000 7f62c0dfd700 1 --2- 172.21.15.3:0/2930079479 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f62bc06a770 0x7f62bc06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:43.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.883+0000 7f62c15fe700 1 --2- 172.21.15.3:0/2930079479 >> v2:172.21.15.3:3300/0 conn(0x7f62bc131d10 0x7f62bc139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:43.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.883+0000 7f62bb7fe700 1 -- 172.21.15.3:0/2930079479 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 471569309 0 0) 0x7f62bc09ece0 con 0x7f62bc06b080 2023-10-03T10:18:43.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.883+0000 7f62bb7fe700 1 -- 172.21.15.3:0/2930079479 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62ac0035b0 con 0x7f62bc06b080 2023-10-03T10:18:43.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.883+0000 7f62c15fe700 1 -- 172.21.15.3:0/2930079479 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f62bc06a770 msgr2=0x7f62bc06ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:43.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.883+0000 7f62c15fe700 1 --2- 172.21.15.3:0/2930079479 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f62bc06a770 0x7f62bc06ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:43.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.883+0000 7f62c15fe700 1 -- 172.21.15.3:0/2930079479 >> v1:172.21.15.3:6789/0 conn(0x7f62bc06b080 legacy=0x7f62bc131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:43.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.883+0000 7f62c15fe700 1 -- 172.21.15.3:0/2930079479 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62bc09f630 con 0x7f62bc131d10 2023-10-03T10:18:43.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.883+0000 7f62c15fe700 1 --2- 172.21.15.3:0/2930079479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62bc131d10 0x7f62bc139860 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f62b400f370 tx=0x7f62b400f680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b46ca237be6e5fb server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:43.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.883+0000 7f62bb7fe700 1 -- 172.21.15.3:0/2930079479 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b4012070 con 0x7f62bc131d10 2023-10-03T10:18:43.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.883+0000 7f62bb7fe700 1 -- 172.21.15.3:0/2930079479 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62b400fe70 con 0x7f62bc131d10 2023-10-03T10:18:43.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.883+0000 7f62bb7fe700 1 -- 172.21.15.3:0/2930079479 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b400d2f0 con 0x7f62bc131d10 2023-10-03T10:18:43.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.884+0000 7f62c3061700 1 -- 172.21.15.3:0/2930079479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62bc131d10 msgr2=0x7f62bc139860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:43.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.884+0000 7f62c3061700 1 --2- 172.21.15.3:0/2930079479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62bc131d10 0x7f62bc139860 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f62b400f370 tx=0x7f62b400f680 comp rx=0 tx=0).stop 2023-10-03T10:18:43.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.884+0000 7f62c3061700 1 -- 172.21.15.3:0/2930079479 shutdown_connections 2023-10-03T10:18:43.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.884+0000 7f62c3061700 1 --2- 172.21.15.3:0/2930079479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62bc131d10 0x7f62bc139860 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:43.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.884+0000 7f62c3061700 1 --2- 172.21.15.3:0/2930079479 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f62bc06a770 0x7f62bc06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:43.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.884+0000 7f62c3061700 1 -- 172.21.15.3:0/2930079479 >> 172.21.15.3:0/2930079479 conn(0x7f62bc066f50 msgr2=0x7f62bc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:43.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.884+0000 7f62c3061700 1 -- 172.21.15.3:0/2930079479 shutdown_connections 2023-10-03T10:18:43.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.884+0000 7f62c3061700 1 -- 172.21.15.3:0/2930079479 wait complete. 2023-10-03T10:18:43.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.886+0000 7f62c3061700 1 Processor -- start 2023-10-03T10:18:43.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.886+0000 7f62c3061700 1 -- start start 2023-10-03T10:18:43.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.887+0000 7f62c3061700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f62bc06a770 0x7f62bc11e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:43.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.887+0000 7f62c3061700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f62bc06b080 0x7f62bc11e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:43.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.887+0000 7f62c3061700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62bc131d10 0x7f62bc11ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.887+0000 7f62c3061700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc130cf0 con 0x7f62bc131d10 2023-10-03T10:18:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.887+0000 7f62c3061700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc130e80 con 0x7f62bc06b080 2023-10-03T10:18:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.887+0000 7f62c3061700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc09a6f0 con 0x7f62bc06a770 2023-10-03T10:18:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.887+0000 7f62bbfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f62bc06b080 0x7f62bc11e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.887+0000 7f62bbfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f62bc06b080 0x7f62bc11e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44502/0 (socket says 172.21.15.3:44502) 2023-10-03T10:18:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.887+0000 7f62bbfff700 1 -- 172.21.15.3:0/3264248439 learned_addr learned my addr 172.21.15.3:0/3264248439 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.887+0000 7f62c15fe700 1 --2- 172.21.15.3:0/3264248439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62bc131d10 0x7f62bc11ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:43.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.888+0000 7f62bbfff700 1 -- 172.21.15.3:0/3264248439 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f62bc06a770 msgr2=0x7f62bc11e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:43.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.888+0000 7f62bbfff700 1 --2- 172.21.15.3:0/3264248439 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f62bc06a770 0x7f62bc11e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:43.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.888+0000 7f62bbfff700 1 -- 172.21.15.3:0/3264248439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62bc131d10 msgr2=0x7f62bc11ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:43.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.888+0000 7f62bbfff700 1 --2- 172.21.15.3:0/3264248439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62bc131d10 0x7f62bc11ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:43.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.888+0000 7f62bbfff700 1 -- 172.21.15.3:0/3264248439 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b400f050 con 0x7f62bc06b080 2023-10-03T10:18:43.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.888+0000 7f62bbfff700 1 --2- 172.21.15.3:0/3264248439 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f62bc06b080 0x7f62bc11e8a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f62a8005910 tx=0x7f62a800ca90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:43.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.888+0000 7f62b97fa700 1 -- 172.21.15.3:0/3264248439 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a8005d70 con 0x7f62bc06b080 2023-10-03T10:18:43.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.888+0000 7f62b97fa700 1 -- 172.21.15.3:0/3264248439 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a80070d0 con 0x7f62bc06b080 2023-10-03T10:18:43.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.888+0000 7f62c3061700 1 -- 172.21.15.3:0/3264248439 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62bc071e60 con 0x7f62bc06b080 2023-10-03T10:18:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.888+0000 7f62c3061700 1 -- 172.21.15.3:0/3264248439 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62bc122610 con 0x7f62bc06b080 2023-10-03T10:18:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.888+0000 7f62b97fa700 1 -- 172.21.15.3:0/3264248439 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a8007440 con 0x7f62bc06b080 2023-10-03T10:18:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.889+0000 7f62b97fa700 1 -- 172.21.15.3:0/3264248439 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f62a80075a0 con 0x7f62bc06b080 2023-10-03T10:18:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.889+0000 7f62b97fa700 1 -- 172.21.15.3:0/3264248439 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f62a803ec50 con 0x7f62bc06b080 2023-10-03T10:18:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.890+0000 7f62c3061700 1 -- 172.21.15.3:0/3264248439 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62bc0a3ab0 con 0x7f62bc06b080 2023-10-03T10:18:43.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.895+0000 7f62b97fa700 1 -- 172.21.15.3:0/3264248439 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f62a8019d20 con 0x7f62bc06b080 2023-10-03T10:18:43.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.929+0000 7f904b6c1700 1 Processor -- start 2023-10-03T10:18:43.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.930+0000 7f904b6c1700 1 -- start start 2023-10-03T10:18:43.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.930+0000 7f904b6c1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f904409d630 0x7f90440a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:43.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.930+0000 7f904b6c1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9044138d80 0x7f904413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:43.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.930+0000 7f904b6c1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90440a2a70 con 0x7f90440a0a30 2023-10-03T10:18:43.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.930+0000 7f904b6c1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90440a01d0 con 0x7f904409d630 2023-10-03T10:18:43.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.930+0000 7f904b6c1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f904409e2b0 con 0x7f9044138d80 2023-10-03T10:18:43.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.930+0000 7f9048c5c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f904409d630 0x7f90440a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:43.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.930+0000 7f9049c5e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9044138d80 0x7f904413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:43.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.930+0000 7f9048c5c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f904409d630 0x7f90440a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44508/0 (socket says 172.21.15.3:44508) 2023-10-03T10:18:43.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.930+0000 7f9048c5c700 1 -- 172.21.15.3:0/2197832632 learned_addr learned my addr 172.21.15.3:0/2197832632 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:43.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.931+0000 7f9049c5e700 1 -- 172.21.15.3:0/2197832632 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f904409d630 msgr2=0x7f90440a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:43.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.931+0000 7f9049c5e700 1 --2- 172.21.15.3:0/2197832632 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f904409d630 0x7f90440a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:43.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.931+0000 7f9049c5e700 1 -- 172.21.15.3:0/2197832632 >> v1:172.21.15.3:6789/0 conn(0x7f90440a0a30 legacy=0x7f90440a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:43.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.931+0000 7f9049c5e700 1 -- 172.21.15.3:0/2197832632 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f904409dc50 con 0x7f9044138d80 2023-10-03T10:18:43.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.931+0000 7f9049c5e700 1 --2- 172.21.15.3:0/2197832632 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9044138d80 0x7f904413b160 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f903c009dc0 tx=0x7f903c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d617b675853c4a8f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:43.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.931+0000 7f903bfff700 1 -- 172.21.15.3:0/2197832632 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c00e070 con 0x7f9044138d80 2023-10-03T10:18:43.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.931+0000 7f903bfff700 1 -- 172.21.15.3:0/2197832632 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903c00bb20 con 0x7f9044138d80 2023-10-03T10:18:43.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.931+0000 7f903bfff700 1 -- 172.21.15.3:0/2197832632 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c012430 con 0x7f9044138d80 2023-10-03T10:18:43.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.932+0000 7f904b6c1700 1 -- 172.21.15.3:0/2197832632 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9044138d80 msgr2=0x7f904413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:43.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.932+0000 7f904b6c1700 1 --2- 172.21.15.3:0/2197832632 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9044138d80 0x7f904413b160 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f903c009dc0 tx=0x7f903c00b330 comp rx=0 tx=0).stop 2023-10-03T10:18:43.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.932+0000 7f904b6c1700 1 -- 172.21.15.3:0/2197832632 shutdown_connections 2023-10-03T10:18:43.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.932+0000 7f904b6c1700 1 --2- 172.21.15.3:0/2197832632 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9044138d80 0x7f904413b160 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:43.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.932+0000 7f904b6c1700 1 --2- 172.21.15.3:0/2197832632 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f904409d630 0x7f90440a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:43.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.932+0000 7f904b6c1700 1 -- 172.21.15.3:0/2197832632 >> 172.21.15.3:0/2197832632 conn(0x7f90440a3050 msgr2=0x7f90440a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:43.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.932+0000 7f904b6c1700 1 -- 172.21.15.3:0/2197832632 shutdown_connections 2023-10-03T10:18:43.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.932+0000 7f904b6c1700 1 -- 172.21.15.3:0/2197832632 wait complete. 2023-10-03T10:18:43.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.933+0000 7f904b6c1700 1 Processor -- start 2023-10-03T10:18:43.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.933+0000 7f904b6c1700 1 -- start start 2023-10-03T10:18:43.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.934+0000 7f904b6c1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f904409d630 0x7f9044133ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:43.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.934+0000 7f904b6c1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f90440a0a30 0x7f9044133ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:43.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.934+0000 7f904b6c1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9044138d80 0x7f9044134530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:43.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.934+0000 7f904b6c1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90440a4e10 con 0x7f904409d630 2023-10-03T10:18:43.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.934+0000 7f904b6c1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9044138890 con 0x7f9044138d80 2023-10-03T10:18:43.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.934+0000 7f904b6c1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f90441382a0 con 0x7f90440a0a30 2023-10-03T10:18:43.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.934+0000 7f904945d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f904409d630 0x7f9044133ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:43.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.934+0000 7f9048c5c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f90440a0a30 0x7f9044133ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:43.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.934+0000 7f9049c5e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9044138d80 0x7f9044134530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:43.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.934+0000 7f904945d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f904409d630 0x7f9044133ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52410/0 (socket says 172.21.15.3:52410) 2023-10-03T10:18:43.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.934+0000 7f9048c5c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f90440a0a30 0x7f9044133ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:37620/0 (socket says 172.21.15.3:37620) 2023-10-03T10:18:43.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.934+0000 7f9049c5e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9044138d80 0x7f9044134530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44514/0 (socket says 172.21.15.3:44514) 2023-10-03T10:18:43.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.934+0000 7f904945d700 1 -- 172.21.15.3:0/3648656462 learned_addr learned my addr 172.21.15.3:0/3648656462 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:43.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.934+0000 7f904945d700 1 -- 172.21.15.3:0/3648656462 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f90440a0a30 msgr2=0x7f9044133ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:43.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.934+0000 7f904945d700 1 --2- 172.21.15.3:0/3648656462 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f90440a0a30 0x7f9044133ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:43.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.934+0000 7f904945d700 1 -- 172.21.15.3:0/3648656462 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9044138d80 msgr2=0x7f9044134530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:43.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.934+0000 7f904945d700 1 --2- 172.21.15.3:0/3648656462 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9044138d80 0x7f9044134530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:43.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.934+0000 7f904945d700 1 -- 172.21.15.3:0/3648656462 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f903c009a70 con 0x7f904409d630 2023-10-03T10:18:43.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.935+0000 7f904945d700 1 --2- 172.21.15.3:0/3648656462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f904409d630 0x7f9044133ab0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f904000c510 tx=0x7f904000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:43.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.935+0000 7f9039ffb700 1 -- 172.21.15.3:0/3648656462 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90400052d0 con 0x7f904409d630 2023-10-03T10:18:43.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.935+0000 7f9039ffb700 1 -- 172.21.15.3:0/3648656462 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9040005430 con 0x7f904409d630 2023-10-03T10:18:43.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.935+0000 7f9039ffb700 1 -- 172.21.15.3:0/3648656462 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90400038c0 con 0x7f904409d630 2023-10-03T10:18:43.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.935+0000 7f904b6c1700 1 -- 172.21.15.3:0/3648656462 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9044137540 con 0x7f904409d630 2023-10-03T10:18:43.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.935+0000 7f904b6c1700 1 -- 172.21.15.3:0/3648656462 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9044138460 con 0x7f904409d630 2023-10-03T10:18:43.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.936+0000 7f904b6c1700 1 -- 172.21.15.3:0/3648656462 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90440747f0 con 0x7f904409d630 2023-10-03T10:18:43.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.937+0000 7f9039ffb700 1 -- 172.21.15.3:0/3648656462 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9040003a20 con 0x7f904409d630 2023-10-03T10:18:43.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.937+0000 7f9039ffb700 1 -- 172.21.15.3:0/3648656462 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f904003eca0 con 0x7f904409d630 2023-10-03T10:18:43.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:43.942+0000 7f9039ffb700 1 -- 172.21.15.3:0/3648656462 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f90400192c0 con 0x7f904409d630 2023-10-03T10:18:44.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.089+0000 7eff7567e700 1 -- 172.21.15.3:0/2746724458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7eff7006c360 con 0x7eff7009e410 2023-10-03T10:18:44.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.089+0000 7eff577fe700 1 -- 172.21.15.3:0/2746724458 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7eff68011380 con 0x7eff7009e410 2023-10-03T10:18:44.090 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:44.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.093+0000 7eff7567e700 1 -- 172.21.15.3:0/2746724458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff7009e410 msgr2=0x7eff7011ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:44.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.093+0000 7eff7567e700 1 --2- 172.21.15.3:0/2746724458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff7009e410 0x7eff7011ea70 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7eff6800cb90 tx=0x7eff6800cea0 comp rx=0 tx=0).stop 2023-10-03T10:18:44.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.093+0000 7eff7567e700 1 -- 172.21.15.3:0/2746724458 shutdown_connections 2023-10-03T10:18:44.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.093+0000 7eff7567e700 1 --2- 172.21.15.3:0/2746724458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff7009e410 0x7eff7011ea70 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:44.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.093+0000 7eff7567e700 1 --2- 172.21.15.3:0/2746724458 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff7009d6f0 0x7eff7011dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:44.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.093+0000 7eff7567e700 1 --2- 172.21.15.3:0/2746724458 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff7009e000 0x7eff7011e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:44.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.093+0000 7eff7567e700 1 -- 172.21.15.3:0/2746724458 >> 172.21.15.3:0/2746724458 conn(0x7eff70066f50 msgr2=0x7eff7006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:44.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.093+0000 7eff7567e700 1 -- 172.21.15.3:0/2746724458 shutdown_connections 2023-10-03T10:18:44.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.093+0000 7eff7567e700 1 -- 172.21.15.3:0/2746724458 wait complete. 2023-10-03T10:18:44.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.102+0000 7f62c3061700 1 -- 172.21.15.3:0/3264248439 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f62bc074ea0 con 0x7f62bc06b080 2023-10-03T10:18:44.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.102+0000 7f62b97fa700 1 -- 172.21.15.3:0/3264248439 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f62a8017030 con 0x7f62bc06b080 2023-10-03T10:18:44.102 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:44.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.104+0000 7f62c3061700 1 -- 172.21.15.3:0/3264248439 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f62bc06b080 msgr2=0x7f62bc11e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:44.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.104+0000 7f62c3061700 1 --2- 172.21.15.3:0/3264248439 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f62bc06b080 0x7f62bc11e8a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f62a8005910 tx=0x7f62a800ca90 comp rx=0 tx=0).stop 2023-10-03T10:18:44.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.105+0000 7f62c3061700 1 -- 172.21.15.3:0/3264248439 shutdown_connections 2023-10-03T10:18:44.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.105+0000 7f62c3061700 1 --2- 172.21.15.3:0/3264248439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62bc131d10 0x7f62bc11ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:44.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.105+0000 7f62c3061700 1 --2- 172.21.15.3:0/3264248439 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f62bc06a770 0x7f62bc11e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:44.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.105+0000 7f62c3061700 1 --2- 172.21.15.3:0/3264248439 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f62bc06b080 0x7f62bc11e8a0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:44.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.105+0000 7f62c3061700 1 -- 172.21.15.3:0/3264248439 >> 172.21.15.3:0/3264248439 conn(0x7f62bc066f50 msgr2=0x7f62bc0a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:44.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.105+0000 7f62c3061700 1 -- 172.21.15.3:0/3264248439 shutdown_connections 2023-10-03T10:18:44.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.105+0000 7f62c3061700 1 -- 172.21.15.3:0/3264248439 wait complete. 2023-10-03T10:18:44.108 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:44.119 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:44.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.136+0000 7fca13c8d700 1 -- 172.21.15.3:0/1391576250 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fca0c1a79c0 con 0x7fca0c09e440 2023-10-03T10:18:44.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.136+0000 7fca027fc700 1 -- 172.21.15.3:0/1391576250 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fca0c1a79c0 con 0x7fca0c09e440 2023-10-03T10:18:44.136 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:44.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.138+0000 7fca13c8d700 1 -- 172.21.15.3:0/1391576250 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca0c09e440 msgr2=0x7fca0c1c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:44.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.138+0000 7fca13c8d700 1 --2- 172.21.15.3:0/1391576250 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca0c09e440 0x7fca0c1c49d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fca04016000 tx=0x7fca04004ad0 comp rx=0 tx=0).stop 2023-10-03T10:18:44.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.138+0000 7fca13c8d700 1 -- 172.21.15.3:0/1391576250 shutdown_connections 2023-10-03T10:18:44.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.138+0000 7fca13c8d700 1 --2- 172.21.15.3:0/1391576250 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fca0c09e440 0x7fca0c1c49d0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:44.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.138+0000 7fca13c8d700 1 --2- 172.21.15.3:0/1391576250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca0c09d6f0 0x7fca0c1c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:44.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.138+0000 7fca13c8d700 1 --2- 172.21.15.3:0/1391576250 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fca0c09e000 0x7fca0c1c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:44.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.138+0000 7fca13c8d700 1 -- 172.21.15.3:0/1391576250 >> 172.21.15.3:0/1391576250 conn(0x7fca0c066f50 msgr2=0x7fca0c1bad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:44.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.138+0000 7fca13c8d700 1 -- 172.21.15.3:0/1391576250 shutdown_connections 2023-10-03T10:18:44.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.138+0000 7fca13c8d700 1 -- 172.21.15.3:0/1391576250 wait complete. 2023-10-03T10:18:44.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.150+0000 7f904b6c1700 1 -- 172.21.15.3:0/3648656462 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9044137320 con 0x7f904409d630 2023-10-03T10:18:44.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.151+0000 7f9039ffb700 1 -- 172.21.15.3:0/3648656462 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9040017020 con 0x7f904409d630 2023-10-03T10:18:44.151 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:44.152 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:44.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.153+0000 7f904b6c1700 1 -- 172.21.15.3:0/3648656462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f904409d630 msgr2=0x7f9044133ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:44.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.153+0000 7f904b6c1700 1 --2- 172.21.15.3:0/3648656462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f904409d630 0x7f9044133ab0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f904000c510 tx=0x7f904000c820 comp rx=0 tx=0).stop 2023-10-03T10:18:44.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.153+0000 7f904b6c1700 1 -- 172.21.15.3:0/3648656462 shutdown_connections 2023-10-03T10:18:44.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.153+0000 7f904b6c1700 1 --2- 172.21.15.3:0/3648656462 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9044138d80 0x7f9044134530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:44.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.153+0000 7f904b6c1700 1 --2- 172.21.15.3:0/3648656462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f904409d630 0x7f9044133ab0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:44.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.153+0000 7f904b6c1700 1 --2- 172.21.15.3:0/3648656462 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f90440a0a30 0x7f9044133ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:44.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.153+0000 7f904b6c1700 1 -- 172.21.15.3:0/3648656462 >> 172.21.15.3:0/3648656462 conn(0x7f90440a3050 msgr2=0x7f904409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:44.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.153+0000 7f904b6c1700 1 -- 172.21.15.3:0/3648656462 shutdown_connections 2023-10-03T10:18:44.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:44.153+0000 7f904b6c1700 1 -- 172.21.15.3:0/3648656462 wait complete. 2023-10-03T10:18:44.167 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:45.109 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:45.120 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:45.152 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:45.168 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:45.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.265+0000 7f761b740700 1 Processor -- start 2023-10-03T10:18:45.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.266+0000 7f761b740700 1 -- start start 2023-10-03T10:18:45.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.266+0000 7f761b740700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f761409f410 0x7f7614136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:45.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.266+0000 7f761b740700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7614136fe0 0x7f76141393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:45.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.266+0000 7f761b740700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7614070940 con 0x7f761409e930 2023-10-03T10:18:45.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.266+0000 7f761b740700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f761406fd60 con 0x7f761409f410 2023-10-03T10:18:45.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.266+0000 7f761b740700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f761406fa80 con 0x7f7614136fe0 2023-10-03T10:18:45.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.266+0000 7f76194dc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f761409e930 0x7f761409ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47722/0 (socket says 172.21.15.3:47722) 2023-10-03T10:18:45.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.266+0000 7f76194dc700 1 -- 172.21.15.3:0/3643247028 learned_addr learned my addr 172.21.15.3:0/3643247028 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:45.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.266+0000 7f7619cdd700 1 --2- 172.21.15.3:0/3643247028 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7614136fe0 0x7f76141393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:45.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.267+0000 7f7618cdb700 1 --2- 172.21.15.3:0/3643247028 >> v2:172.21.15.3:3300/0 conn(0x7f761409f410 0x7f7614136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:45.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.267+0000 7f760bfff700 1 -- 172.21.15.3:0/3643247028 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871295720 0 0) 0x7f7614070940 con 0x7f761409e930 2023-10-03T10:18:45.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.267+0000 7f760bfff700 1 -- 172.21.15.3:0/3643247028 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76000035b0 con 0x7f761409e930 2023-10-03T10:18:45.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.267+0000 7f7619cdd700 1 -- 172.21.15.3:0/3643247028 >> v2:172.21.15.3:3300/0 conn(0x7f761409f410 msgr2=0x7f7614136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:45.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.267+0000 7f7619cdd700 1 --2- 172.21.15.3:0/3643247028 >> v2:172.21.15.3:3300/0 conn(0x7f761409f410 0x7f7614136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.267+0000 7f7619cdd700 1 -- 172.21.15.3:0/3643247028 >> v1:172.21.15.3:6789/0 conn(0x7f761409e930 legacy=0x7f761409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:45.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.267+0000 7f7619cdd700 1 -- 172.21.15.3:0/3643247028 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f761409fb60 con 0x7f7614136fe0 2023-10-03T10:18:45.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.267+0000 7f7618cdb700 1 --2- 172.21.15.3:0/3643247028 >> v2:172.21.15.3:3300/0 conn(0x7f761409f410 0x7f7614136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:45.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.267+0000 7f7619cdd700 1 --2- 172.21.15.3:0/3643247028 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7614136fe0 0x7f76141393c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f760c009d90 tx=0x7f760c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49e6dd98802269fc server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:45.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.267+0000 7f760bfff700 1 -- 172.21.15.3:0/3643247028 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760c00e070 con 0x7f7614136fe0 2023-10-03T10:18:45.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.267+0000 7f760bfff700 1 -- 172.21.15.3:0/3643247028 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f760c00bae0 con 0x7f7614136fe0 2023-10-03T10:18:45.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.267+0000 7f760bfff700 1 -- 172.21.15.3:0/3643247028 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760c00be90 con 0x7f7614136fe0 2023-10-03T10:18:45.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.268+0000 7f761b740700 1 -- 172.21.15.3:0/3643247028 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7614136fe0 msgr2=0x7f76141393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:45.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.268+0000 7f761b740700 1 --2- 172.21.15.3:0/3643247028 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7614136fe0 0x7f76141393c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f760c009d90 tx=0x7f760c00b330 comp rx=0 tx=0).stop 2023-10-03T10:18:45.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.269+0000 7f761b740700 1 -- 172.21.15.3:0/3643247028 shutdown_connections 2023-10-03T10:18:45.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.269+0000 7f761b740700 1 --2- 172.21.15.3:0/3643247028 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7614136fe0 0x7f76141393c0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.269+0000 7f761b740700 1 --2- 172.21.15.3:0/3643247028 >> v2:172.21.15.3:3300/0 conn(0x7f761409f410 0x7f7614136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.269+0000 7f761b740700 1 -- 172.21.15.3:0/3643247028 >> 172.21.15.3:0/3643247028 conn(0x7f76140a4210 msgr2=0x7f76140a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:45.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.269+0000 7f761b740700 1 -- 172.21.15.3:0/3643247028 shutdown_connections 2023-10-03T10:18:45.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.269+0000 7f761b740700 1 -- 172.21.15.3:0/3643247028 wait complete. 2023-10-03T10:18:45.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.270+0000 7f761b740700 1 Processor -- start 2023-10-03T10:18:45.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.270+0000 7f761b740700 1 -- start start 2023-10-03T10:18:45.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.271+0000 7f761b740700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f761409e930 0x7f761411e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:45.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.271+0000 7f761b740700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f761409f410 0x7f761411eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:45.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.271+0000 7f761b740700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7614136fe0 0x7f761411f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:45.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.271+0000 7f761b740700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7614133f80 con 0x7f7614136fe0 2023-10-03T10:18:45.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.271+0000 7f761b740700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7614120890 con 0x7f761409e930 2023-10-03T10:18:45.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.271+0000 7f761b740700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7614120370 con 0x7f761409f410 2023-10-03T10:18:45.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.271+0000 7f7619cdd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7614136fe0 0x7f761411f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:45.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.271+0000 7f7619cdd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7614136fe0 0x7f761411f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52432/0 (socket says 172.21.15.3:52432) 2023-10-03T10:18:45.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.271+0000 7f7619cdd700 1 -- 172.21.15.3:0/2922238585 learned_addr learned my addr 172.21.15.3:0/2922238585 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:45.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.271+0000 7f76194dc700 1 --2- 172.21.15.3:0/2922238585 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f761409e930 0x7f761411e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:45.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.271+0000 7f7618cdb700 1 --2- 172.21.15.3:0/2922238585 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f761409f410 0x7f761411eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:45.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.272+0000 7f7619cdd700 1 -- 172.21.15.3:0/2922238585 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f761409f410 msgr2=0x7f761411eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:45.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.272+0000 7f7619cdd700 1 --2- 172.21.15.3:0/2922238585 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f761409f410 0x7f761411eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.272+0000 7f7619cdd700 1 -- 172.21.15.3:0/2922238585 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f761409e930 msgr2=0x7f761411e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:45.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.272+0000 7f7619cdd700 1 --2- 172.21.15.3:0/2922238585 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f761409e930 0x7f761411e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.272+0000 7f7619cdd700 1 -- 172.21.15.3:0/2922238585 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76040094a0 con 0x7f7614136fe0 2023-10-03T10:18:45.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.272+0000 7f76194dc700 1 --2- 172.21.15.3:0/2922238585 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f761409e930 0x7f761411e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:45.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.272+0000 7f7619cdd700 1 --2- 172.21.15.3:0/2922238585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7614136fe0 0x7f761411f0d0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f760c00b680 tx=0x7f760c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:45.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.272+0000 7f7609ffb700 1 -- 172.21.15.3:0/2922238585 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760c0044c0 con 0x7f7614136fe0 2023-10-03T10:18:45.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.272+0000 7f7609ffb700 1 -- 172.21.15.3:0/2922238585 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f760c004620 con 0x7f7614136fe0 2023-10-03T10:18:45.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.272+0000 7f761b740700 1 -- 172.21.15.3:0/2922238585 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f760c009a70 con 0x7f7614136fe0 2023-10-03T10:18:45.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.272+0000 7f7609ffb700 1 -- 172.21.15.3:0/2922238585 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760c012430 con 0x7f7614136fe0 2023-10-03T10:18:45.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.273+0000 7f761b740700 1 -- 172.21.15.3:0/2922238585 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7614120a50 con 0x7f7614136fe0 2023-10-03T10:18:45.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.274+0000 7f7609ffb700 1 -- 172.21.15.3:0/2922238585 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f760c012590 con 0x7f7614136fe0 2023-10-03T10:18:45.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.274+0000 7f7609ffb700 1 -- 172.21.15.3:0/2922238585 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f760c0415b0 con 0x7f7614136fe0 2023-10-03T10:18:45.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.274+0000 7f761b740700 1 -- 172.21.15.3:0/2922238585 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7614074ea0 con 0x7f7614136fe0 2023-10-03T10:18:45.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.283+0000 7f7609ffb700 1 -- 172.21.15.3:0/2922238585 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f760c011750 con 0x7f7614136fe0 2023-10-03T10:18:45.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.306+0000 7fd06815c700 1 Processor -- start 2023-10-03T10:18:45.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.306+0000 7fd06815c700 1 -- start start 2023-10-03T10:18:45.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.307+0000 7fd06815c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd06009e930 0x7fd06009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:45.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.307+0000 7fd06815c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd06009f240 0x7fd0601317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:45.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.307+0000 7fd06815c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd06009f650 0x7fd060131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:45.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.307+0000 7fd06815c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd060070940 con 0x7fd06009e930 2023-10-03T10:18:45.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.307+0000 7fd06815c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06006fd60 con 0x7fd06009f240 2023-10-03T10:18:45.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.307+0000 7fd06815c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd06006fa80 con 0x7fd06009f650 2023-10-03T10:18:45.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.307+0000 7fd0656f7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd06009f240 0x7fd0601317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:45.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.307+0000 7fd0656f7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd06009f240 0x7fd0601317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44538/0 (socket says 172.21.15.3:44538) 2023-10-03T10:18:45.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.307+0000 7fd0656f7700 1 -- 172.21.15.3:0/364055562 learned_addr learned my addr 172.21.15.3:0/364055562 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:45.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.307+0000 7fd065ef8700 1 --2- 172.21.15.3:0/364055562 >> v2:172.21.15.3:3300/0 conn(0x7fd06009e930 0x7fd06009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:45.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.307+0000 7fd0666f9700 1 --2- 172.21.15.3:0/364055562 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd06009f650 0x7fd060131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:45.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.307+0000 7fd065ef8700 1 -- 172.21.15.3:0/364055562 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd06009f650 msgr2=0x7fd060131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:45.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.307+0000 7fd065ef8700 1 --2- 172.21.15.3:0/364055562 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd06009f650 0x7fd060131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.307+0000 7fd065ef8700 1 -- 172.21.15.3:0/364055562 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd06009f240 msgr2=0x7fd0601317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:45.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.307+0000 7fd065ef8700 1 --2- 172.21.15.3:0/364055562 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd06009f240 0x7fd0601317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.307+0000 7fd065ef8700 1 -- 172.21.15.3:0/364055562 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06009fb60 con 0x7fd06009e930 2023-10-03T10:18:45.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.308+0000 7fd065ef8700 1 --2- 172.21.15.3:0/364055562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd06009e930 0x7fd06009ed00 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009840 tx=0x7fd05c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f7a09c08c4703eb server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:45.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.308+0000 7fd064ef6700 1 -- 172.21.15.3:0/364055562 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c010a90 con 0x7fd06009e930 2023-10-03T10:18:45.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.308+0000 7fd064ef6700 1 -- 172.21.15.3:0/364055562 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05c010bf0 con 0x7fd06009e930 2023-10-03T10:18:45.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.308+0000 7fd064ef6700 1 -- 172.21.15.3:0/364055562 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c018440 con 0x7fd06009e930 2023-10-03T10:18:45.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.309+0000 7fd06815c700 1 -- 172.21.15.3:0/364055562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd06009e930 msgr2=0x7fd06009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:45.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.309+0000 7fd06815c700 1 --2- 172.21.15.3:0/364055562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd06009e930 0x7fd06009ed00 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009840 tx=0x7fd05c009b50 comp rx=0 tx=0).stop 2023-10-03T10:18:45.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.309+0000 7fd06815c700 1 -- 172.21.15.3:0/364055562 shutdown_connections 2023-10-03T10:18:45.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.309+0000 7fd06815c700 1 --2- 172.21.15.3:0/364055562 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd06009f650 0x7fd060131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.309+0000 7fd06815c700 1 --2- 172.21.15.3:0/364055562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd06009e930 0x7fd06009ed00 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.309+0000 7fd06815c700 1 --2- 172.21.15.3:0/364055562 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd06009f240 0x7fd0601317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.309+0000 7fd06815c700 1 -- 172.21.15.3:0/364055562 >> 172.21.15.3:0/364055562 conn(0x7fd0600a4210 msgr2=0x7fd0600a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:45.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.309+0000 7fd06815c700 1 -- 172.21.15.3:0/364055562 shutdown_connections 2023-10-03T10:18:45.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.309+0000 7fd06815c700 1 -- 172.21.15.3:0/364055562 wait complete. 2023-10-03T10:18:45.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.310+0000 7fd06815c700 1 Processor -- start 2023-10-03T10:18:45.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.310+0000 7fd06815c700 1 -- start start 2023-10-03T10:18:45.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.311+0000 7fd06815c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd06009e930 0x7fd06011d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:45.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.311+0000 7fd06815c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd06009f240 0x7fd06011d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:45.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.311+0000 7fd06815c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd06009f650 0x7fd06011dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:45.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.311+0000 7fd06815c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06011f710 con 0x7fd06009e930 2023-10-03T10:18:45.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.311+0000 7fd06815c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06011f3f0 con 0x7fd06009f240 2023-10-03T10:18:45.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.311+0000 7fd06815c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd0600a2f40 con 0x7fd06009f650 2023-10-03T10:18:45.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.311+0000 7fd065ef8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd06009e930 0x7fd06011d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:45.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.311+0000 7fd0666f9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd06009f650 0x7fd06011dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:45.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.311+0000 7fd0656f7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd06009f240 0x7fd06011d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:45.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.311+0000 7fd065ef8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd06009e930 0x7fd06011d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52450/0 (socket says 172.21.15.3:52450) 2023-10-03T10:18:45.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.311+0000 7fd065ef8700 1 -- 172.21.15.3:0/2002886364 learned_addr learned my addr 172.21.15.3:0/2002886364 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:45.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.312+0000 7fd065ef8700 1 -- 172.21.15.3:0/2002886364 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd06009f650 msgr2=0x7fd06011dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:45.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.312+0000 7fd065ef8700 1 --2- 172.21.15.3:0/2002886364 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd06009f650 0x7fd06011dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.312+0000 7fd065ef8700 1 -- 172.21.15.3:0/2002886364 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd06009f240 msgr2=0x7fd06011d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:45.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.312+0000 7fd065ef8700 1 --2- 172.21.15.3:0/2002886364 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd06009f240 0x7fd06011d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.312+0000 7fd065ef8700 1 -- 172.21.15.3:0/2002886364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd05c0094a0 con 0x7fd06009e930 2023-10-03T10:18:45.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.312+0000 7fd0666f9700 1 --2- 172.21.15.3:0/2002886364 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd06009f650 0x7fd06011dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:45.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.312+0000 7fd0656f7700 1 --2- 172.21.15.3:0/2002886364 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd06009f240 0x7fd06011d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:18:45.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.312+0000 7fd065ef8700 1 --2- 172.21.15.3:0/2002886364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd06009e930 0x7fd06011d2c0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009fd0 tx=0x7fd05c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:45.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.312+0000 7fd0567fc700 1 -- 172.21.15.3:0/2002886364 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c003b70 con 0x7fd06009e930 2023-10-03T10:18:45.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.312+0000 7fd0567fc700 1 -- 172.21.15.3:0/2002886364 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05c003cd0 con 0x7fd06009e930 2023-10-03T10:18:45.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.312+0000 7fd06815c700 1 -- 172.21.15.3:0/2002886364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0600a0190 con 0x7fd06009e930 2023-10-03T10:18:45.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.312+0000 7fd0567fc700 1 -- 172.21.15.3:0/2002886364 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c0236b0 con 0x7fd06009e930 2023-10-03T10:18:45.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.312+0000 7fd06815c700 1 -- 172.21.15.3:0/2002886364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd060120df0 con 0x7fd06009e930 2023-10-03T10:18:45.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.313+0000 7fd0567fc700 1 -- 172.21.15.3:0/2002886364 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd05c01f070 con 0x7fd06009e930 2023-10-03T10:18:45.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.313+0000 7fd0567fc700 1 -- 172.21.15.3:0/2002886364 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd05c045020 con 0x7fd06009e930 2023-10-03T10:18:45.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.314+0000 7fd06815c700 1 -- 172.21.15.3:0/2002886364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0601206c0 con 0x7fd06009e930 2023-10-03T10:18:45.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.316+0000 7f742dfbd700 1 Processor -- start 2023-10-03T10:18:45.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.316+0000 7f742dfbd700 1 -- start start 2023-10-03T10:18:45.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.316+0000 7f742dfbd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f74281338a0 0x7f7428133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:45.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.316+0000 7f742dfbd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f742809de50 0x7f742809e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:45.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.316+0000 7f742dfbd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f742806ef30 con 0x7f742809e760 2023-10-03T10:18:45.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.316+0000 7f742dfbd700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f742806fef0 con 0x7f74281338a0 2023-10-03T10:18:45.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.316+0000 7f742dfbd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74281334b0 con 0x7f742809de50 2023-10-03T10:18:45.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.316+0000 7f7427fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f742809e760 0x7f742813a730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47730/0 (socket says 172.21.15.3:47730) 2023-10-03T10:18:45.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.316+0000 7f7427fff700 1 -- 172.21.15.3:0/3857352143 learned_addr learned my addr 172.21.15.3:0/3857352143 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:45.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.317+0000 7f7426ffd700 1 --2- 172.21.15.3:0/3857352143 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f742809de50 0x7f742809e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:45.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.317+0000 7f7426ffd700 1 -- 172.21.15.3:0/3857352143 >> v2:172.21.15.3:3300/0 conn(0x7f74281338a0 msgr2=0x7f7428133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:45.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.317+0000 7f7426ffd700 1 --2- 172.21.15.3:0/3857352143 >> v2:172.21.15.3:3300/0 conn(0x7f74281338a0 0x7f7428133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.317+0000 7f7426ffd700 1 -- 172.21.15.3:0/3857352143 >> v1:172.21.15.3:6789/0 conn(0x7f742809e760 legacy=0x7f742813a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:45.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.317+0000 7f7426ffd700 1 -- 172.21.15.3:0/3857352143 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f742806fc30 con 0x7f742809de50 2023-10-03T10:18:45.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.317+0000 7f7426ffd700 1 --2- 172.21.15.3:0/3857352143 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f742809de50 0x7f742809e220 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f741c009840 tx=0x7f741c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=495871dd75d6d61 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:45.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.317+0000 7f74267fc700 1 -- 172.21.15.3:0/3857352143 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c011830 con 0x7f742809de50 2023-10-03T10:18:45.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.317+0000 7f74267fc700 1 -- 172.21.15.3:0/3857352143 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741c011990 con 0x7f742809de50 2023-10-03T10:18:45.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.317+0000 7f74267fc700 1 -- 172.21.15.3:0/3857352143 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c011d20 con 0x7f742809de50 2023-10-03T10:18:45.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.318+0000 7f742dfbd700 1 -- 172.21.15.3:0/3857352143 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f742809de50 msgr2=0x7f742809e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:45.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.318+0000 7f742dfbd700 1 --2- 172.21.15.3:0/3857352143 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f742809de50 0x7f742809e220 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f741c009840 tx=0x7f741c009b50 comp rx=0 tx=0).stop 2023-10-03T10:18:45.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.318+0000 7f742dfbd700 1 -- 172.21.15.3:0/3857352143 shutdown_connections 2023-10-03T10:18:45.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.318+0000 7f742dfbd700 1 --2- 172.21.15.3:0/3857352143 >> v2:172.21.15.3:3300/0 conn(0x7f74281338a0 0x7f7428133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.318+0000 7f742dfbd700 1 --2- 172.21.15.3:0/3857352143 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f742809de50 0x7f742809e220 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.318+0000 7f742dfbd700 1 -- 172.21.15.3:0/3857352143 >> 172.21.15.3:0/3857352143 conn(0x7f74280a2aa0 msgr2=0x7f74280a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:45.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.318+0000 7f742dfbd700 1 -- 172.21.15.3:0/3857352143 shutdown_connections 2023-10-03T10:18:45.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.318+0000 7f742dfbd700 1 -- 172.21.15.3:0/3857352143 wait complete. 2023-10-03T10:18:45.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.319+0000 7f742dfbd700 1 Processor -- start 2023-10-03T10:18:45.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.319+0000 7fd0567fc700 1 -- 172.21.15.3:0/2002886364 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd05c01d020 con 0x7fd06009e930 2023-10-03T10:18:45.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.320+0000 7f742dfbd700 1 -- start start 2023-10-03T10:18:45.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.320+0000 7f742dfbd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f742809de50 0x7f742811ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:45.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.320+0000 7f742dfbd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f742809e760 0x7f742811f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:45.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.320+0000 7f742dfbd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74281338a0 0x7f742811f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:45.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.320+0000 7f742dfbd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742812f780 con 0x7f742809de50 2023-10-03T10:18:45.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.320+0000 7f742dfbd700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74281221c0 con 0x7f74281338a0 2023-10-03T10:18:45.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.320+0000 7f742dfbd700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74281226d0 con 0x7f742809e760 2023-10-03T10:18:45.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.320+0000 7f7427fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74281338a0 0x7f742811f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:45.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.320+0000 7f7427fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74281338a0 0x7f742811f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44560/0 (socket says 172.21.15.3:44560) 2023-10-03T10:18:45.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.320+0000 7f7427fff700 1 -- 172.21.15.3:0/1736875322 learned_addr learned my addr 172.21.15.3:0/1736875322 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:45.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.321+0000 7f74277fe700 1 --2- 172.21.15.3:0/1736875322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f742809de50 0x7f742811ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:45.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.321+0000 7f7426ffd700 1 --2- 172.21.15.3:0/1736875322 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f742809e760 0x7f742811f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:45.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.321+0000 7f7427fff700 1 -- 172.21.15.3:0/1736875322 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f742809e760 msgr2=0x7f742811f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:45.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.321+0000 7f7427fff700 1 --2- 172.21.15.3:0/1736875322 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f742809e760 0x7f742811f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.321+0000 7f7427fff700 1 -- 172.21.15.3:0/1736875322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f742809de50 msgr2=0x7f742811ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:45.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.321+0000 7f7427fff700 1 --2- 172.21.15.3:0/1736875322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f742809de50 0x7f742811ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.321+0000 7f7427fff700 1 -- 172.21.15.3:0/1736875322 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f741c0094a0 con 0x7f74281338a0 2023-10-03T10:18:45.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.321+0000 7f7426ffd700 1 --2- 172.21.15.3:0/1736875322 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f742809e760 0x7f742811f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:18:45.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.321+0000 7f7427fff700 1 --2- 172.21.15.3:0/1736875322 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74281338a0 0x7f742811f6c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f742000cbc0 tx=0x7f742000ced0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:45.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.321+0000 7f740ffff700 1 -- 172.21.15.3:0/1736875322 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742000aaa0 con 0x7f74281338a0 2023-10-03T10:18:45.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.321+0000 7f740ffff700 1 -- 172.21.15.3:0/1736875322 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742000ac00 con 0x7f74281338a0 2023-10-03T10:18:45.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.322+0000 7f740ffff700 1 -- 172.21.15.3:0/1736875322 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7420012400 con 0x7f74281338a0 2023-10-03T10:18:45.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.323+0000 7f742dfbd700 1 -- 172.21.15.3:0/1736875322 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74280a10c0 con 0x7f74281338a0 2023-10-03T10:18:45.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.323+0000 7f742dfbd700 1 -- 172.21.15.3:0/1736875322 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74281223f0 con 0x7f74281338a0 2023-10-03T10:18:45.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.324+0000 7f742dfbd700 1 -- 172.21.15.3:0/1736875322 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7428074ea0 con 0x7f74281338a0 2023-10-03T10:18:45.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.325+0000 7f740ffff700 1 -- 172.21.15.3:0/1736875322 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f742001b3b0 con 0x7f74281338a0 2023-10-03T10:18:45.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.325+0000 7f740ffff700 1 -- 172.21.15.3:0/1736875322 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f742003f450 con 0x7f74281338a0 2023-10-03T10:18:45.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.330+0000 7f740ffff700 1 -- 172.21.15.3:0/1736875322 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f74200192c0 con 0x7f74281338a0 2023-10-03T10:18:45.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.352+0000 7fafacbc3700 1 Processor -- start 2023-10-03T10:18:45.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.352+0000 7fafacbc3700 1 -- start start 2023-10-03T10:18:45.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.353+0000 7fafacbc3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fafa809c650 0x7fafa809ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:45.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.353+0000 7fafacbc3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fafa809cf60 0x7fafa8132080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:45.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.353+0000 7fafacbc3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafa809ba70 con 0x7fafa80a0090 2023-10-03T10:18:45.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.353+0000 7fafacbc3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa809e8f0 con 0x7fafa809cf60 2023-10-03T10:18:45.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.353+0000 7fafacbc3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fafa809d400 con 0x7fafa809c650 2023-10-03T10:18:45.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.353+0000 7fafa5d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fafa80a0090 0x7fafa8131970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47734/0 (socket says 172.21.15.3:47734) 2023-10-03T10:18:45.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.353+0000 7fafa5d9b700 1 -- 172.21.15.3:0/1287059254 learned_addr learned my addr 172.21.15.3:0/1287059254 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:45.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.353+0000 7fafa659c700 1 --2- 172.21.15.3:0/1287059254 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fafa809c650 0x7fafa809ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.353+0000 7fafa6d9d700 1 --2- 172.21.15.3:0/1287059254 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fafa809cf60 0x7fafa8132080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.353+0000 7fafa559a700 1 -- 172.21.15.3:0/1287059254 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2533674945 0 0) 0x7fafa809ba70 con 0x7fafa80a0090 2023-10-03T10:18:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.354+0000 7fafa659c700 1 -- 172.21.15.3:0/1287059254 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fafa809cf60 msgr2=0x7fafa8132080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.354+0000 7fafa659c700 1 --2- 172.21.15.3:0/1287059254 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fafa809cf60 0x7fafa8132080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.354+0000 7fafa659c700 1 -- 172.21.15.3:0/1287059254 >> v1:172.21.15.3:6789/0 conn(0x7fafa80a0090 legacy=0x7fafa8131970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:45.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.354+0000 7fafa659c700 1 -- 172.21.15.3:0/1287059254 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa80a0760 con 0x7fafa809c650 2023-10-03T10:18:45.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.354+0000 7fafa659c700 1 --2- 172.21.15.3:0/1287059254 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fafa809c650 0x7fafa809ca20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7faf98009870 tx=0x7faf98009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63642d937e495555 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:45.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.354+0000 7fafa559a700 1 -- 172.21.15.3:0/1287059254 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf98011830 con 0x7fafa809c650 2023-10-03T10:18:45.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.354+0000 7fafa559a700 1 -- 172.21.15.3:0/1287059254 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf98011990 con 0x7fafa809c650 2023-10-03T10:18:45.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.354+0000 7fafa559a700 1 -- 172.21.15.3:0/1287059254 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf98011d20 con 0x7fafa809c650 2023-10-03T10:18:45.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.355+0000 7fafacbc3700 1 -- 172.21.15.3:0/1287059254 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fafa809c650 msgr2=0x7fafa809ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:45.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.355+0000 7fafacbc3700 1 --2- 172.21.15.3:0/1287059254 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fafa809c650 0x7fafa809ca20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7faf98009870 tx=0x7faf98009b80 comp rx=0 tx=0).stop 2023-10-03T10:18:45.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.355+0000 7fafacbc3700 1 -- 172.21.15.3:0/1287059254 shutdown_connections 2023-10-03T10:18:45.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.355+0000 7fafacbc3700 1 --2- 172.21.15.3:0/1287059254 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fafa809cf60 0x7fafa8132080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.355+0000 7fafacbc3700 1 --2- 172.21.15.3:0/1287059254 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fafa809c650 0x7fafa809ca20 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.355+0000 7fafacbc3700 1 -- 172.21.15.3:0/1287059254 >> 172.21.15.3:0/1287059254 conn(0x7fafa80a3f70 msgr2=0x7fafa80a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:45.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.355+0000 7fafacbc3700 1 -- 172.21.15.3:0/1287059254 shutdown_connections 2023-10-03T10:18:45.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.355+0000 7fafacbc3700 1 -- 172.21.15.3:0/1287059254 wait complete. 2023-10-03T10:18:45.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.356+0000 7fafacbc3700 1 Processor -- start 2023-10-03T10:18:45.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.356+0000 7fafacbc3700 1 -- start start 2023-10-03T10:18:45.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.357+0000 7fafacbc3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fafa809c650 0x7fafa8070ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:45.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.357+0000 7fafacbc3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafa809cf60 0x7fafa8071000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:45.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.357+0000 7fafacbc3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fafa80a0090 0x7fafa8071540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:45.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.357+0000 7fafacbc3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa812a4d0 con 0x7fafa809cf60 2023-10-03T10:18:45.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.357+0000 7fafacbc3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa812a700 con 0x7fafa80a0090 2023-10-03T10:18:45.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.357+0000 7fafacbc3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fafa812abe0 con 0x7fafa809c650 2023-10-03T10:18:45.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.357+0000 7fafa659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fafa809c650 0x7fafa8070ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.357+0000 7fafa5d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafa809cf60 0x7fafa8071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.357+0000 7fafa6d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fafa80a0090 0x7fafa8071540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.357+0000 7fafa659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fafa809c650 0x7fafa8070ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:37662/0 (socket says 172.21.15.3:37662) 2023-10-03T10:18:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.357+0000 7fafa659c700 1 -- 172.21.15.3:0/3799193156 learned_addr learned my addr 172.21.15.3:0/3799193156 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.357+0000 7fafa659c700 1 -- 172.21.15.3:0/3799193156 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fafa80a0090 msgr2=0x7fafa8071540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.357+0000 7fafa659c700 1 --2- 172.21.15.3:0/3799193156 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fafa80a0090 0x7fafa8071540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.357+0000 7fafa659c700 1 -- 172.21.15.3:0/3799193156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafa809cf60 msgr2=0x7fafa8071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.357+0000 7fafa659c700 1 --2- 172.21.15.3:0/3799193156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafa809cf60 0x7fafa8071000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.357+0000 7fafa659c700 1 -- 172.21.15.3:0/3799193156 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf980094a0 con 0x7fafa809c650 2023-10-03T10:18:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.357+0000 7fafa5d9b700 1 --2- 172.21.15.3:0/3799193156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafa809cf60 0x7fafa8071000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.358+0000 7fafa659c700 1 --2- 172.21.15.3:0/3799193156 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fafa809c650 0x7fafa8070ac0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7faf98000c00 tx=0x7faf98004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.358+0000 7faf96ffd700 1 -- 172.21.15.3:0/3799193156 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf98004590 con 0x7fafa809c650 2023-10-03T10:18:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.358+0000 7faf96ffd700 1 -- 172.21.15.3:0/3799193156 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9801f070 con 0x7fafa809c650 2023-10-03T10:18:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.358+0000 7fafacbc3700 1 -- 172.21.15.3:0/3799193156 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafa809d980 con 0x7fafa809c650 2023-10-03T10:18:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.358+0000 7faf96ffd700 1 -- 172.21.15.3:0/3799193156 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf980038e0 con 0x7fafa809c650 2023-10-03T10:18:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.358+0000 7fafacbc3700 1 -- 172.21.15.3:0/3799193156 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafa8072530 con 0x7fafa809c650 2023-10-03T10:18:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.359+0000 7faf96ffd700 1 -- 172.21.15.3:0/3799193156 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faf98003a40 con 0x7fafa809c650 2023-10-03T10:18:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.359+0000 7faf96ffd700 1 -- 172.21.15.3:0/3799193156 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7faf98021070 con 0x7fafa809c650 2023-10-03T10:18:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.359+0000 7fafacbc3700 1 -- 172.21.15.3:0/3799193156 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafa80747f0 con 0x7fafa809c650 2023-10-03T10:18:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.364+0000 7faf96ffd700 1 -- 172.21.15.3:0/3799193156 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faf98015020 con 0x7fafa809c650 2023-10-03T10:18:45.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.531+0000 7fd06815c700 1 -- 172.21.15.3:0/2002886364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd060074ea0 con 0x7fd06009e930 2023-10-03T10:18:45.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.531+0000 7fd0567fc700 1 -- 172.21.15.3:0/2002886364 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd05c003e40 con 0x7fd06009e930 2023-10-03T10:18:45.532 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:45.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.533+0000 7fd06815c700 1 -- 172.21.15.3:0/2002886364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd06009e930 msgr2=0x7fd06011d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:45.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.533+0000 7fd06815c700 1 --2- 172.21.15.3:0/2002886364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd06009e930 0x7fd06011d2c0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009fd0 tx=0x7fd05c004750 comp rx=0 tx=0).stop 2023-10-03T10:18:45.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.533+0000 7fd06815c700 1 -- 172.21.15.3:0/2002886364 shutdown_connections 2023-10-03T10:18:45.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.533+0000 7fd06815c700 1 --2- 172.21.15.3:0/2002886364 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd06009f650 0x7fd06011dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.533+0000 7fd06815c700 1 --2- 172.21.15.3:0/2002886364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd06009e930 0x7fd06011d2c0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.533+0000 7fd06815c700 1 --2- 172.21.15.3:0/2002886364 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd06009f240 0x7fd06011d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.533+0000 7fd06815c700 1 -- 172.21.15.3:0/2002886364 >> 172.21.15.3:0/2002886364 conn(0x7fd0600a4210 msgr2=0x7fd0600a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:45.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.533+0000 7fd06815c700 1 -- 172.21.15.3:0/2002886364 shutdown_connections 2023-10-03T10:18:45.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.533+0000 7fd06815c700 1 -- 172.21.15.3:0/2002886364 wait complete. 2023-10-03T10:18:45.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.544+0000 7f761b740700 1 -- 172.21.15.3:0/2922238585 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7614121b30 con 0x7f7614136fe0 2023-10-03T10:18:45.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.545+0000 7f7609ffb700 1 -- 172.21.15.3:0/2922238585 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7614121b30 con 0x7f7614136fe0 2023-10-03T10:18:45.545 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:45.548 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:45.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.547+0000 7f761b740700 1 -- 172.21.15.3:0/2922238585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7614136fe0 msgr2=0x7f761411f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:45.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.547+0000 7f761b740700 1 --2- 172.21.15.3:0/2922238585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7614136fe0 0x7f761411f0d0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f760c00b680 tx=0x7f760c004ad0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.547+0000 7f761b740700 1 -- 172.21.15.3:0/2922238585 shutdown_connections 2023-10-03T10:18:45.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.547+0000 7f761b740700 1 --2- 172.21.15.3:0/2922238585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7614136fe0 0x7f761411f0d0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.547+0000 7f761b740700 1 --2- 172.21.15.3:0/2922238585 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f761409e930 0x7f761411e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.547+0000 7f761b740700 1 --2- 172.21.15.3:0/2922238585 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f761409f410 0x7f761411eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.547+0000 7f761b740700 1 -- 172.21.15.3:0/2922238585 >> 172.21.15.3:0/2922238585 conn(0x7f76140a4210 msgr2=0x7f761409b430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:45.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.547+0000 7f761b740700 1 -- 172.21.15.3:0/2922238585 shutdown_connections 2023-10-03T10:18:45.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.547+0000 7f761b740700 1 -- 172.21.15.3:0/2922238585 wait complete. 2023-10-03T10:18:45.561 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:45.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.575+0000 7fafacbc3700 1 -- 172.21.15.3:0/3799193156 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fafa8072840 con 0x7fafa809c650 2023-10-03T10:18:45.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.575+0000 7faf96ffd700 1 -- 172.21.15.3:0/3799193156 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faf98034070 con 0x7fafa809c650 2023-10-03T10:18:45.575 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:45.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.575+0000 7f742dfbd700 1 -- 172.21.15.3:0/1736875322 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f742806b9e0 con 0x7f74281338a0 2023-10-03T10:18:45.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.575+0000 7f740ffff700 1 -- 172.21.15.3:0/1736875322 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7420017030 con 0x7f74281338a0 2023-10-03T10:18:45.576 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:45.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.577+0000 7fafacbc3700 1 -- 172.21.15.3:0/3799193156 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fafa809c650 msgr2=0x7fafa8070ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:45.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.577+0000 7fafacbc3700 1 --2- 172.21.15.3:0/3799193156 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fafa809c650 0x7fafa8070ac0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7faf98000c00 tx=0x7faf98004750 comp rx=0 tx=0).stop 2023-10-03T10:18:45.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.577+0000 7fafacbc3700 1 -- 172.21.15.3:0/3799193156 shutdown_connections 2023-10-03T10:18:45.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.577+0000 7fafacbc3700 1 --2- 172.21.15.3:0/3799193156 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fafa80a0090 0x7fafa8071540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.577+0000 7fafacbc3700 1 --2- 172.21.15.3:0/3799193156 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fafa809c650 0x7fafa8070ac0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.577+0000 7fafacbc3700 1 --2- 172.21.15.3:0/3799193156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafa809cf60 0x7fafa8071000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.577+0000 7fafacbc3700 1 -- 172.21.15.3:0/3799193156 >> 172.21.15.3:0/3799193156 conn(0x7fafa80a3f70 msgr2=0x7fafa809eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:45.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.578+0000 7fafacbc3700 1 -- 172.21.15.3:0/3799193156 shutdown_connections 2023-10-03T10:18:45.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.578+0000 7fafacbc3700 1 -- 172.21.15.3:0/3799193156 wait complete. 2023-10-03T10:18:45.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.578+0000 7f742dfbd700 1 -- 172.21.15.3:0/1736875322 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74281338a0 msgr2=0x7f742811f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:45.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.578+0000 7f742dfbd700 1 --2- 172.21.15.3:0/1736875322 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74281338a0 0x7f742811f6c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f742000cbc0 tx=0x7f742000ced0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.578+0000 7f742dfbd700 1 -- 172.21.15.3:0/1736875322 shutdown_connections 2023-10-03T10:18:45.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.578+0000 7f742dfbd700 1 --2- 172.21.15.3:0/1736875322 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74281338a0 0x7f742811f6c0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.578+0000 7f742dfbd700 1 --2- 172.21.15.3:0/1736875322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f742809de50 0x7f742811ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.578+0000 7f742dfbd700 1 --2- 172.21.15.3:0/1736875322 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f742809e760 0x7f742811f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:45.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.578+0000 7f742dfbd700 1 -- 172.21.15.3:0/1736875322 >> 172.21.15.3:0/1736875322 conn(0x7f74280a2aa0 msgr2=0x7f742809ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:45.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.578+0000 7f742dfbd700 1 -- 172.21.15.3:0/1736875322 shutdown_connections 2023-10-03T10:18:45.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:45.578+0000 7f742dfbd700 1 -- 172.21.15.3:0/1736875322 wait complete. 2023-10-03T10:18:45.593 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:45.595 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:46.549 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:46.562 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:46.594 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:46.595 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:46.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.704+0000 7fb99dc77700 1 Processor -- start 2023-10-03T10:18:46.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.705+0000 7fb99dc77700 1 -- start start 2023-10-03T10:18:46.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.705+0000 7fb99dc77700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb99809d6f0 0x7fb99809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:46.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.705+0000 7fb99dc77700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb99809e000 0x7fb998130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.705+0000 7fb99dc77700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb99809e410 0x7fb998130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.705+0000 7fb99dc77700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb99809f520 con 0x7fb99809e410 2023-10-03T10:18:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.705+0000 7fb99dc77700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb99809f130 con 0x7fb99809e000 2023-10-03T10:18:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.705+0000 7fb99dc77700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb99809e930 con 0x7fb99809d6f0 2023-10-03T10:18:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.705+0000 7fb996ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb99809e000 0x7fb998130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.705+0000 7fb996ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb99809e000 0x7fb998130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44576/0 (socket says 172.21.15.3:44576) 2023-10-03T10:18:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.705+0000 7fb996ffd700 1 -- 172.21.15.3:0/3328268962 learned_addr learned my addr 172.21.15.3:0/3328268962 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.705+0000 7fb997fff700 1 --2- 172.21.15.3:0/3328268962 >> v2:172.21.15.3:3300/0 conn(0x7fb99809e410 0x7fb998130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.706+0000 7fb9977fe700 1 --2- 172.21.15.3:0/3328268962 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb99809d6f0 0x7fb99809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.706+0000 7fb996ffd700 1 -- 172.21.15.3:0/3328268962 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb99809d6f0 msgr2=0x7fb99809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.706+0000 7fb996ffd700 1 --2- 172.21.15.3:0/3328268962 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb99809d6f0 0x7fb99809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.706+0000 7fb996ffd700 1 -- 172.21.15.3:0/3328268962 >> v2:172.21.15.3:3300/0 conn(0x7fb99809e410 msgr2=0x7fb998130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.706+0000 7fb996ffd700 1 --2- 172.21.15.3:0/3328268962 >> v2:172.21.15.3:3300/0 conn(0x7fb99809e410 0x7fb998130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.706+0000 7fb996ffd700 1 -- 172.21.15.3:0/3328268962 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99809b2d0 con 0x7fb99809e000 2023-10-03T10:18:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.706+0000 7fb997fff700 1 --2- 172.21.15.3:0/3328268962 >> v2:172.21.15.3:3300/0 conn(0x7fb99809e410 0x7fb998130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.706+0000 7fb996ffd700 1 --2- 172.21.15.3:0/3328268962 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb99809e000 0x7fb998130660 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb98c00dd50 tx=0x7fb98c00f330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c1efee2fd66424d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.706+0000 7fb9967fc700 1 -- 172.21.15.3:0/3328268962 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98c012070 con 0x7fb99809e000 2023-10-03T10:18:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.706+0000 7fb9967fc700 1 -- 172.21.15.3:0/3328268962 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98c00fae0 con 0x7fb99809e000 2023-10-03T10:18:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.706+0000 7fb9967fc700 1 -- 172.21.15.3:0/3328268962 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98c00dea0 con 0x7fb99809e000 2023-10-03T10:18:46.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.707+0000 7fb99dc77700 1 -- 172.21.15.3:0/3328268962 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb99809e000 msgr2=0x7fb998130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:46.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.707+0000 7fb99dc77700 1 --2- 172.21.15.3:0/3328268962 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb99809e000 0x7fb998130660 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb98c00dd50 tx=0x7fb98c00f330 comp rx=0 tx=0).stop 2023-10-03T10:18:46.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.707+0000 7fb99dc77700 1 -- 172.21.15.3:0/3328268962 shutdown_connections 2023-10-03T10:18:46.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.707+0000 7fb99dc77700 1 --2- 172.21.15.3:0/3328268962 >> v2:172.21.15.3:3300/0 conn(0x7fb99809e410 0x7fb998130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.707+0000 7fb99dc77700 1 --2- 172.21.15.3:0/3328268962 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb99809d6f0 0x7fb99809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.707+0000 7fb99dc77700 1 --2- 172.21.15.3:0/3328268962 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb99809e000 0x7fb998130660 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.707+0000 7fb99dc77700 1 -- 172.21.15.3:0/3328268962 >> 172.21.15.3:0/3328268962 conn(0x7fb998066f50 msgr2=0x7fb99809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:46.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.707+0000 7fb99dc77700 1 -- 172.21.15.3:0/3328268962 shutdown_connections 2023-10-03T10:18:46.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.707+0000 7fb99dc77700 1 -- 172.21.15.3:0/3328268962 wait complete. 2023-10-03T10:18:46.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.709+0000 7fb99dc77700 1 Processor -- start 2023-10-03T10:18:46.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.709+0000 7fb99dc77700 1 -- start start 2023-10-03T10:18:46.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.710+0000 7fb99dc77700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb99809d6f0 0x7fb99811df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.710+0000 7fb99dc77700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb99809e000 0x7fb99811e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.710+0000 7fb99dc77700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb99809e410 0x7fb99811ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.710+0000 7fb99dc77700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb998120190 con 0x7fb99809e000 2023-10-03T10:18:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.710+0000 7fb99dc77700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb998121a50 con 0x7fb99809e410 2023-10-03T10:18:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.710+0000 7fb99dc77700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb998120e20 con 0x7fb99809d6f0 2023-10-03T10:18:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.710+0000 7fb997fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb99809e410 0x7fb99811ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.710+0000 7fb9977fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb99809d6f0 0x7fb99811df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.710+0000 7fb9977fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb99809d6f0 0x7fb99811df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:37680/0 (socket says 172.21.15.3:37680) 2023-10-03T10:18:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.710+0000 7fb997fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb99809e410 0x7fb99811ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44588/0 (socket says 172.21.15.3:44588) 2023-10-03T10:18:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.710+0000 7fb9977fe700 1 -- 172.21.15.3:0/553834879 learned_addr learned my addr 172.21.15.3:0/553834879 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.710+0000 7fb996ffd700 1 --2- 172.21.15.3:0/553834879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb99809e000 0x7fb99811e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.710+0000 7fb997fff700 1 -- 172.21.15.3:0/553834879 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb99809d6f0 msgr2=0x7fb99811df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:46.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.710+0000 7fb997fff700 1 --2- 172.21.15.3:0/553834879 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb99809d6f0 0x7fb99811df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.710+0000 7fb997fff700 1 -- 172.21.15.3:0/553834879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb99809e000 msgr2=0x7fb99811e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:46.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.710+0000 7fb997fff700 1 --2- 172.21.15.3:0/553834879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb99809e000 0x7fb99811e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.710+0000 7fb997fff700 1 -- 172.21.15.3:0/553834879 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb990009a70 con 0x7fb99809e410 2023-10-03T10:18:46.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.711+0000 7fb9977fe700 1 --2- 172.21.15.3:0/553834879 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb99809d6f0 0x7fb99811df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:46.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.711+0000 7fb997fff700 1 --2- 172.21.15.3:0/553834879 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb99809e410 0x7fb99811ea00 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fb99000efc0 tx=0x7fb99000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:46.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.711+0000 7fb97ffff700 1 -- 172.21.15.3:0/553834879 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99000ca90 con 0x7fb99809e410 2023-10-03T10:18:46.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.711+0000 7fb97ffff700 1 -- 172.21.15.3:0/553834879 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99000f040 con 0x7fb99809e410 2023-10-03T10:18:46.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.711+0000 7fb99dc77700 1 -- 172.21.15.3:0/553834879 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb98c00da00 con 0x7fb99809e410 2023-10-03T10:18:46.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.711+0000 7fb97ffff700 1 -- 172.21.15.3:0/553834879 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99000ce30 con 0x7fb99809e410 2023-10-03T10:18:46.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.711+0000 7fb99dc77700 1 -- 172.21.15.3:0/553834879 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9980a1720 con 0x7fb99809e410 2023-10-03T10:18:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.712+0000 7fb97ffff700 1 -- 172.21.15.3:0/553834879 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb990014480 con 0x7fb99809e410 2023-10-03T10:18:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.712+0000 7fb97ffff700 1 -- 172.21.15.3:0/553834879 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb99003fd20 con 0x7fb99809e410 2023-10-03T10:18:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.712+0000 7fb99dc77700 1 -- 172.21.15.3:0/553834879 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb99809c760 con 0x7fb99809e410 2023-10-03T10:18:46.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.717+0000 7fb97ffff700 1 -- 172.21.15.3:0/553834879 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb990012930 con 0x7fb99809e410 2023-10-03T10:18:46.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.749+0000 7f79c107d700 1 Processor -- start 2023-10-03T10:18:46.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.749+0000 7f79c107d700 1 -- start start 2023-10-03T10:18:46.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.749+0000 7f441e75e700 1 Processor -- start 2023-10-03T10:18:46.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.749+0000 7f441e75e700 1 -- start start 2023-10-03T10:18:46.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.750+0000 7f79c107d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f79bc09e1d0 0x7f79bc138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:46.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.750+0000 7f79c107d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79bc139170 0x7f79bc13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:46.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.750+0000 7f79c107d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79bc09b120 con 0x7f79bc09d6f0 2023-10-03T10:18:46.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.750+0000 7f79c107d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79bc06b460 con 0x7f79bc09e1d0 2023-10-03T10:18:46.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.750+0000 7f79c107d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc09a7e0 con 0x7f79bc139170 2023-10-03T10:18:46.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.750+0000 7f79ba59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f79bc09e1d0 0x7f79bc138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:46.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.750+0000 7f79ba59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f79bc09e1d0 0x7f79bc138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52500/0 (socket says 172.21.15.3:52500) 2023-10-03T10:18:46.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.750+0000 7f79ba59c700 1 -- 172.21.15.3:0/2718833211 learned_addr learned my addr 172.21.15.3:0/2718833211 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:46.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.750+0000 7f441e75e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4418130540 0x7f4418130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:46.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.750+0000 7f441e75e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4418130e50 0x7f441806f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:46.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.750+0000 7f441e75e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44180716a0 con 0x7f441806fb20 2023-10-03T10:18:46.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.750+0000 7f441e75e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f441809a390 con 0x7f4418130e50 2023-10-03T10:18:46.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.750+0000 7f441e75e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4418071290 con 0x7f4418130540 2023-10-03T10:18:46.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.750+0000 7f441ccfb700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f441806fb20 0x7f4418137560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47744/0 (socket says 172.21.15.3:47744) 2023-10-03T10:18:46.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.750+0000 7f441ccfb700 1 -- 172.21.15.3:0/974836294 learned_addr learned my addr 172.21.15.3:0/974836294 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:46.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.750+0000 7f44177fe700 1 --2- 172.21.15.3:0/974836294 >> v2:172.21.15.3:3300/0 conn(0x7f4418130e50 0x7f441806f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:46.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.751+0000 7f44177fe700 1 -- 172.21.15.3:0/974836294 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4418130540 msgr2=0x7f4418130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:46.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.751+0000 7f44177fe700 1 --2- 172.21.15.3:0/974836294 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4418130540 0x7f4418130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.751+0000 7f44177fe700 1 -- 172.21.15.3:0/974836294 >> v1:172.21.15.3:6789/0 conn(0x7f441806fb20 legacy=0x7f4418137560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:46.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.751+0000 7f44177fe700 1 -- 172.21.15.3:0/974836294 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4418130340 con 0x7f4418130e50 2023-10-03T10:18:46.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.751+0000 7f79b9d9b700 1 -- 172.21.15.3:0/2718833211 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2462988373 0 0) 0x7f79bc09b120 con 0x7f79bc09d6f0 2023-10-03T10:18:46.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.751+0000 7f79b9d9b700 1 -- 172.21.15.3:0/2718833211 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79a40035b0 con 0x7f79bc09d6f0 2023-10-03T10:18:46.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.751+0000 7f79ba59c700 1 -- 172.21.15.3:0/2718833211 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79bc139170 msgr2=0x7f79bc13b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:46.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.751+0000 7f79ba59c700 1 --2- 172.21.15.3:0/2718833211 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79bc139170 0x7f79bc13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.751+0000 7f79ba59c700 1 -- 172.21.15.3:0/2718833211 >> v1:172.21.15.3:6789/0 conn(0x7f79bc09d6f0 legacy=0x7f79bc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:46.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.751+0000 7f79ba59c700 1 -- 172.21.15.3:0/2718833211 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79bc09a500 con 0x7f79bc09e1d0 2023-10-03T10:18:46.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.751+0000 7f44177fe700 1 --2- 172.21.15.3:0/974836294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4418130e50 0x7f441806f5e0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f440c009840 tx=0x7f440c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01607c58c5b00fd server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:46.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.751+0000 7f4416ffd700 1 -- 172.21.15.3:0/974836294 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c011830 con 0x7f4418130e50 2023-10-03T10:18:46.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.751+0000 7f79ba59c700 1 --2- 172.21.15.3:0/2718833211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79bc09e1d0 0x7f79bc138c30 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f79b0009920 tx=0x7f79b0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=183debf8b1a6fb7f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:46.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.751+0000 7f4416ffd700 1 -- 172.21.15.3:0/974836294 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440c011990 con 0x7f4418130e50 2023-10-03T10:18:46.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.751+0000 7f4416ffd700 1 -- 172.21.15.3:0/974836294 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c011d20 con 0x7f4418130e50 2023-10-03T10:18:46.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.751+0000 7f79b9d9b700 1 -- 172.21.15.3:0/2718833211 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b0004230 con 0x7f79bc09e1d0 2023-10-03T10:18:46.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.751+0000 7f79b9d9b700 1 -- 172.21.15.3:0/2718833211 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b0004390 con 0x7f79bc09e1d0 2023-10-03T10:18:46.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.751+0000 7f79b9d9b700 1 -- 172.21.15.3:0/2718833211 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b00036a0 con 0x7f79bc09e1d0 2023-10-03T10:18:46.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.752+0000 7f441e75e700 1 -- 172.21.15.3:0/974836294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4418130e50 msgr2=0x7f441806f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:46.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.752+0000 7f441e75e700 1 --2- 172.21.15.3:0/974836294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4418130e50 0x7f441806f5e0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f440c009840 tx=0x7f440c009b50 comp rx=0 tx=0).stop 2023-10-03T10:18:46.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.752+0000 7f79c107d700 1 -- 172.21.15.3:0/2718833211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79bc09e1d0 msgr2=0x7f79bc138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:46.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.752+0000 7f79c107d700 1 --2- 172.21.15.3:0/2718833211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79bc09e1d0 0x7f79bc138c30 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f79b0009920 tx=0x7f79b0009c30 comp rx=0 tx=0).stop 2023-10-03T10:18:46.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.752+0000 7f441e75e700 1 -- 172.21.15.3:0/974836294 shutdown_connections 2023-10-03T10:18:46.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.752+0000 7f441e75e700 1 --2- 172.21.15.3:0/974836294 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4418130540 0x7f4418130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.752+0000 7f79c107d700 1 -- 172.21.15.3:0/2718833211 shutdown_connections 2023-10-03T10:18:46.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.752+0000 7f79c107d700 1 --2- 172.21.15.3:0/2718833211 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79bc139170 0x7f79bc13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.752+0000 7f79c107d700 1 --2- 172.21.15.3:0/2718833211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79bc09e1d0 0x7f79bc138c30 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.752+0000 7f441e75e700 1 --2- 172.21.15.3:0/974836294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4418130e50 0x7f441806f5e0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.752+0000 7f441e75e700 1 -- 172.21.15.3:0/974836294 >> 172.21.15.3:0/974836294 conn(0x7f44180a2aa0 msgr2=0x7f44180a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:46.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.752+0000 7f79c107d700 1 -- 172.21.15.3:0/2718833211 >> 172.21.15.3:0/2718833211 conn(0x7f79bc066f50 msgr2=0x7f79bc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:46.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.752+0000 7f441e75e700 1 -- 172.21.15.3:0/974836294 shutdown_connections 2023-10-03T10:18:46.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.752+0000 7f441e75e700 1 -- 172.21.15.3:0/974836294 wait complete. 2023-10-03T10:18:46.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.752+0000 7f79c107d700 1 -- 172.21.15.3:0/2718833211 shutdown_connections 2023-10-03T10:18:46.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.753+0000 7f79c107d700 1 -- 172.21.15.3:0/2718833211 wait complete. 2023-10-03T10:18:46.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.753+0000 7f441e75e700 1 Processor -- start 2023-10-03T10:18:46.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.754+0000 7f441e75e700 1 -- start start 2023-10-03T10:18:46.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.754+0000 7f441e75e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f441806fb20 0x7f4418118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:46.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.754+0000 7f441e75e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4418130540 0x7f4418119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:46.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.754+0000 7f441e75e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4418130e50 0x7f4418119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:46.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.754+0000 7f441e75e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441811be90 con 0x7f441806fb20 2023-10-03T10:18:46.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.754+0000 7f441e75e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f441811af80 con 0x7f4418130e50 2023-10-03T10:18:46.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.754+0000 7f441e75e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f441811a220 con 0x7f4418130540 2023-10-03T10:18:46.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.754+0000 7f79c107d700 1 Processor -- start 2023-10-03T10:18:46.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.754+0000 7f4417fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f441806fb20 0x7f4418118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:46.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.755+0000 7f4417fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f441806fb20 0x7f4418118cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52520/0 (socket says 172.21.15.3:52520) 2023-10-03T10:18:46.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.754+0000 7f79c107d700 1 -- start start 2023-10-03T10:18:46.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.755+0000 7f4417fff700 1 -- 172.21.15.3:0/2705135373 learned_addr learned my addr 172.21.15.3:0/2705135373 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:46.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.755+0000 7f4417fff700 1 -- 172.21.15.3:0/2705135373 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4418130540 msgr2=0x7f4418119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:46.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.755+0000 7f4417fff700 1 --2- 172.21.15.3:0/2705135373 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4418130540 0x7f4418119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.755+0000 7f4417fff700 1 -- 172.21.15.3:0/2705135373 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4418130e50 msgr2=0x7f4418119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:46.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.755+0000 7f4417fff700 1 --2- 172.21.15.3:0/2705135373 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4418130e50 0x7f4418119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.755+0000 7f4417fff700 1 -- 172.21.15.3:0/2705135373 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440c0094a0 con 0x7f441806fb20 2023-10-03T10:18:46.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.755+0000 7f79c107d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79bc09d6f0 0x7f79bc1c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:46.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.755+0000 7f4417fff700 1 --2- 172.21.15.3:0/2705135373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f441806fb20 0x7f4418118cd0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f440800d9e0 tx=0x7f440800dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:46.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.755+0000 7f4414ff9700 1 -- 172.21.15.3:0/2705135373 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440800ecd0 con 0x7f441806fb20 2023-10-03T10:18:46.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.756+0000 7f441e75e700 1 -- 172.21.15.3:0/2705135373 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44180a1f30 con 0x7f441806fb20 2023-10-03T10:18:46.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.756+0000 7f4414ff9700 1 -- 172.21.15.3:0/2705135373 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440800ee30 con 0x7f441806fb20 2023-10-03T10:18:46.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.756+0000 7f4414ff9700 1 -- 172.21.15.3:0/2705135373 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408016680 con 0x7f441806fb20 2023-10-03T10:18:46.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.756+0000 7f441e75e700 1 -- 172.21.15.3:0/2705135373 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f441811b440 con 0x7f441806fb20 2023-10-03T10:18:46.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.755+0000 7f79c107d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79bc09e1d0 0x7f79bc1c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:46.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.755+0000 7f79c107d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79bc139170 0x7f79bc120800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:46.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.755+0000 7f79c107d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc1240d0 con 0x7f79bc09e1d0 2023-10-03T10:18:46.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.755+0000 7f79c107d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc122410 con 0x7f79bc139170 2023-10-03T10:18:46.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.755+0000 7f79c107d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc071f60 con 0x7f79bc09d6f0 2023-10-03T10:18:46.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.756+0000 7f79bb59e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79bc139170 0x7f79bc120800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:46.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.756+0000 7f79bb59e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79bc139170 0x7f79bc120800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44612/0 (socket says 172.21.15.3:44612) 2023-10-03T10:18:46.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.756+0000 7f79bb59e700 1 -- 172.21.15.3:0/2272321008 learned_addr learned my addr 172.21.15.3:0/2272321008 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:46.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.756+0000 7f79bb59e700 1 -- 172.21.15.3:0/2272321008 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79bc09d6f0 msgr2=0x7f79bc1c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:46.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.756+0000 7f79bb59e700 1 --2- 172.21.15.3:0/2272321008 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79bc09d6f0 0x7f79bc1c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.757+0000 7f79bb59e700 1 -- 172.21.15.3:0/2272321008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79bc09e1d0 msgr2=0x7f79bc1c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:46.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.757+0000 7f79bb59e700 1 --2- 172.21.15.3:0/2272321008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79bc09e1d0 0x7f79bc1c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.757+0000 7f79bb59e700 1 -- 172.21.15.3:0/2272321008 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b0009550 con 0x7f79bc139170 2023-10-03T10:18:46.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.757+0000 7f79bb59e700 1 --2- 172.21.15.3:0/2272321008 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79bc139170 0x7f79bc120800 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f79b400ba10 tx=0x7f79b400bdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:46.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.757+0000 7f79a37fe700 1 -- 172.21.15.3:0/2272321008 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b400c9b0 con 0x7f79bc139170 2023-10-03T10:18:46.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.757+0000 7f441e75e700 1 -- 172.21.15.3:0/2705135373 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4418074ea0 con 0x7f441806fb20 2023-10-03T10:18:46.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.757+0000 7f79c107d700 1 -- 172.21.15.3:0/2272321008 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79bc09c710 con 0x7f79bc139170 2023-10-03T10:18:46.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.757+0000 7f79c107d700 1 -- 172.21.15.3:0/2272321008 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79bc1c9080 con 0x7f79bc139170 2023-10-03T10:18:46.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.757+0000 7f79a37fe700 1 -- 172.21.15.3:0/2272321008 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b400cb30 con 0x7f79bc139170 2023-10-03T10:18:46.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.757+0000 7f79a37fe700 1 -- 172.21.15.3:0/2272321008 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b4014440 con 0x7f79bc139170 2023-10-03T10:18:46.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.758+0000 7f79a37fe700 1 -- 172.21.15.3:0/2272321008 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f79b40145a0 con 0x7f79bc139170 2023-10-03T10:18:46.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.758+0000 7f4414ff9700 1 -- 172.21.15.3:0/2705135373 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f440801f3b0 con 0x7f441806fb20 2023-10-03T10:18:46.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.758+0000 7f79a37fe700 1 -- 172.21.15.3:0/2272321008 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f79b4006e40 con 0x7f79bc139170 2023-10-03T10:18:46.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.759+0000 7f79c107d700 1 -- 172.21.15.3:0/2272321008 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f799c0052e0 con 0x7f79bc139170 2023-10-03T10:18:46.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.759+0000 7f4414ff9700 1 -- 172.21.15.3:0/2705135373 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f440801f660 con 0x7f441806fb20 2023-10-03T10:18:46.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.767+0000 7f4414ff9700 1 -- 172.21.15.3:0/2705135373 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f440801e360 con 0x7f441806fb20 2023-10-03T10:18:46.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.768+0000 7f79a37fe700 1 -- 172.21.15.3:0/2272321008 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f79b401d3b0 con 0x7f79bc139170 2023-10-03T10:18:46.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.824+0000 7f7bd2b07700 1 Processor -- start 2023-10-03T10:18:46.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.824+0000 7f7bd2b07700 1 -- start start 2023-10-03T10:18:46.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.825+0000 7f7bd2b07700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7bcc09d6f0 0x7f7bcc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:46.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.825+0000 7f7bd2b07700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7bcc09e000 0x7f7bcc1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:46.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.825+0000 7f7bd2b07700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7bcc09e410 0x7f7bcc133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:46.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.825+0000 7f7bd2b07700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bcc06b460 con 0x7f7bcc09e000 2023-10-03T10:18:46.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.825+0000 7f7bd2b07700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc09b120 con 0x7f7bcc09e410 2023-10-03T10:18:46.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.825+0000 7f7bd2b07700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc09a7e0 con 0x7f7bcc09d6f0 2023-10-03T10:18:46.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.825+0000 7f7bd10a4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7bcc09e410 0x7f7bcc133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:46.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.825+0000 7f7bd08a3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7bcc09d6f0 0x7f7bcc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:46.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.825+0000 7f7bd10a4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7bcc09e410 0x7f7bcc133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44624/0 (socket says 172.21.15.3:44624) 2023-10-03T10:18:46.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.825+0000 7f7bd10a4700 1 -- 172.21.15.3:0/756903860 learned_addr learned my addr 172.21.15.3:0/756903860 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:46.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.825+0000 7f7bcbfff700 1 --2- 172.21.15.3:0/756903860 >> v2:172.21.15.3:3300/0 conn(0x7f7bcc09e000 0x7f7bcc1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.826+0000 7f7bd08a3700 1 -- 172.21.15.3:0/756903860 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7bcc09e410 msgr2=0x7f7bcc133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.826+0000 7f7bd08a3700 1 --2- 172.21.15.3:0/756903860 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7bcc09e410 0x7f7bcc133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.826+0000 7f7bd08a3700 1 -- 172.21.15.3:0/756903860 >> v2:172.21.15.3:3300/0 conn(0x7f7bcc09e000 msgr2=0x7f7bcc1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.826+0000 7f7bd08a3700 1 --2- 172.21.15.3:0/756903860 >> v2:172.21.15.3:3300/0 conn(0x7f7bcc09e000 0x7f7bcc1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.826+0000 7f7bd08a3700 1 -- 172.21.15.3:0/756903860 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc09a500 con 0x7f7bcc09d6f0 2023-10-03T10:18:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.826+0000 7f7bd08a3700 1 --2- 172.21.15.3:0/756903860 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7bcc09d6f0 0x7f7bcc09dac0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0009870 tx=0x7f7bc0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=616ca18ec017e20 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.826+0000 7f7bcb7fe700 1 -- 172.21.15.3:0/756903860 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0011830 con 0x7f7bcc09d6f0 2023-10-03T10:18:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.826+0000 7f7bcb7fe700 1 -- 172.21.15.3:0/756903860 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc0011990 con 0x7f7bcc09d6f0 2023-10-03T10:18:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.826+0000 7f7bcb7fe700 1 -- 172.21.15.3:0/756903860 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0011d20 con 0x7f7bcc09d6f0 2023-10-03T10:18:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.827+0000 7f7bd2b07700 1 -- 172.21.15.3:0/756903860 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7bcc09d6f0 msgr2=0x7f7bcc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.827+0000 7f7bd2b07700 1 --2- 172.21.15.3:0/756903860 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7bcc09d6f0 0x7f7bcc09dac0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0009870 tx=0x7f7bc0009b80 comp rx=0 tx=0).stop 2023-10-03T10:18:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.827+0000 7f7bd2b07700 1 -- 172.21.15.3:0/756903860 shutdown_connections 2023-10-03T10:18:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.827+0000 7f7bd2b07700 1 --2- 172.21.15.3:0/756903860 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7bcc09e410 0x7f7bcc133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.827+0000 7f7bd2b07700 1 --2- 172.21.15.3:0/756903860 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7bcc09d6f0 0x7f7bcc09dac0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.827+0000 7f7bd2b07700 1 --2- 172.21.15.3:0/756903860 >> v2:172.21.15.3:3300/0 conn(0x7f7bcc09e000 0x7f7bcc1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.827+0000 7f7bd2b07700 1 -- 172.21.15.3:0/756903860 >> 172.21.15.3:0/756903860 conn(0x7f7bcc066f50 msgr2=0x7f7bcc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.827+0000 7f7bd2b07700 1 -- 172.21.15.3:0/756903860 shutdown_connections 2023-10-03T10:18:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.827+0000 7f7bd2b07700 1 -- 172.21.15.3:0/756903860 wait complete. 2023-10-03T10:18:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.829+0000 7f7bd2b07700 1 Processor -- start 2023-10-03T10:18:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.829+0000 7f7bd2b07700 1 -- start start 2023-10-03T10:18:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.829+0000 7f7bd2b07700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7bcc09d6f0 0x7f7bcc1a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.829+0000 7f7bd2b07700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7bcc09e000 0x7f7bcc1a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.829+0000 7f7bd2b07700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bcc09e410 0x7f7bcc1a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.829+0000 7f7bd2b07700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc09c710 con 0x7f7bcc09e410 2023-10-03T10:18:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.829+0000 7f7bd2b07700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc1aa270 con 0x7f7bcc09d6f0 2023-10-03T10:18:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.829+0000 7f7bd2b07700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc1a79a0 con 0x7f7bcc09e000 2023-10-03T10:18:46.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.830+0000 7f7bcbfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7bcc09e000 0x7f7bcc1a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:46.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.830+0000 7f7bd08a3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7bcc09d6f0 0x7f7bcc1a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:46.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.830+0000 7f7bcbfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7bcc09e000 0x7f7bcc1a6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:37730/0 (socket says 172.21.15.3:37730) 2023-10-03T10:18:46.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.830+0000 7f7bcbfff700 1 -- 172.21.15.3:0/3475418148 learned_addr learned my addr 172.21.15.3:0/3475418148 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:46.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.830+0000 7f7bd10a4700 1 --2- 172.21.15.3:0/3475418148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bcc09e410 0x7f7bcc1a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.830+0000 7f7bd08a3700 1 -- 172.21.15.3:0/3475418148 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7bcc09e000 msgr2=0x7f7bcc1a6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.830+0000 7f7bd08a3700 1 --2- 172.21.15.3:0/3475418148 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7bcc09e000 0x7f7bcc1a6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.830+0000 7f7bd08a3700 1 -- 172.21.15.3:0/3475418148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bcc09e410 msgr2=0x7f7bcc1a68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.830+0000 7f7bd08a3700 1 --2- 172.21.15.3:0/3475418148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bcc09e410 0x7f7bcc1a68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.830+0000 7f7bd08a3700 1 -- 172.21.15.3:0/3475418148 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc00094a0 con 0x7f7bcc09d6f0 2023-10-03T10:18:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.830+0000 7f7bd08a3700 1 --2- 172.21.15.3:0/3475418148 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7bcc09d6f0 0x7f7bcc1a5e20 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc06e470 tx=0x7f7bc0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:46.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.831+0000 7f7bc97fa700 1 -- 172.21.15.3:0/3475418148 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0004510 con 0x7f7bcc09d6f0 2023-10-03T10:18:46.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.831+0000 7f7bc97fa700 1 -- 172.21.15.3:0/3475418148 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc001f070 con 0x7f7bcc09d6f0 2023-10-03T10:18:46.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.831+0000 7f7bd2b07700 1 -- 172.21.15.3:0/3475418148 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bcc1a98b0 con 0x7f7bcc09d6f0 2023-10-03T10:18:46.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.831+0000 7f7bc97fa700 1 -- 172.21.15.3:0/3475418148 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc00038c0 con 0x7f7bcc09d6f0 2023-10-03T10:18:46.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.831+0000 7f7bd2b07700 1 -- 172.21.15.3:0/3475418148 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bcc1a8bf0 con 0x7f7bcc09d6f0 2023-10-03T10:18:46.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.832+0000 7f7bc97fa700 1 -- 172.21.15.3:0/3475418148 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7bc0003a20 con 0x7f7bcc09d6f0 2023-10-03T10:18:46.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.832+0000 7f7bd2b07700 1 -- 172.21.15.3:0/3475418148 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bcc074ea0 con 0x7f7bcc09d6f0 2023-10-03T10:18:46.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.832+0000 7f7bc97fa700 1 -- 172.21.15.3:0/3475418148 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7bc0021070 con 0x7f7bcc09d6f0 2023-10-03T10:18:46.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.842+0000 7f7bc97fa700 1 -- 172.21.15.3:0/3475418148 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7bc0024380 con 0x7f7bcc09d6f0 2023-10-03T10:18:46.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.973+0000 7fb99dc77700 1 -- 172.21.15.3:0/553834879 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb998074ea0 con 0x7fb99809e410 2023-10-03T10:18:46.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.974+0000 7fb97ffff700 1 -- 172.21.15.3:0/553834879 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb99002a090 con 0x7fb99809e410 2023-10-03T10:18:46.974 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:46.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.976+0000 7fb99dc77700 1 -- 172.21.15.3:0/553834879 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb99809e410 msgr2=0x7fb99811ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:46.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.976+0000 7fb99dc77700 1 --2- 172.21.15.3:0/553834879 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb99809e410 0x7fb99811ea00 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fb99000efc0 tx=0x7fb99000c3a0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.976+0000 7fb99dc77700 1 -- 172.21.15.3:0/553834879 shutdown_connections 2023-10-03T10:18:46.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.976+0000 7fb99dc77700 1 --2- 172.21.15.3:0/553834879 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb99809e410 0x7fb99811ea00 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.976+0000 7fb99dc77700 1 --2- 172.21.15.3:0/553834879 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb99809d6f0 0x7fb99811df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.976+0000 7fb99dc77700 1 --2- 172.21.15.3:0/553834879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb99809e000 0x7fb99811e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.976+0000 7fb99dc77700 1 -- 172.21.15.3:0/553834879 >> 172.21.15.3:0/553834879 conn(0x7fb998066f50 msgr2=0x7fb99806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:46.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.977+0000 7fb99dc77700 1 -- 172.21.15.3:0/553834879 shutdown_connections 2023-10-03T10:18:46.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.977+0000 7fb99dc77700 1 -- 172.21.15.3:0/553834879 wait complete. 2023-10-03T10:18:46.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.983+0000 7f441e75e700 1 -- 172.21.15.3:0/2705135373 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f441806b9e0 con 0x7f441806fb20 2023-10-03T10:18:46.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.983+0000 7f4414ff9700 1 -- 172.21.15.3:0/2705135373 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f440801b070 con 0x7f441806fb20 2023-10-03T10:18:46.984 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:46.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.983+0000 7f79c107d700 1 -- 172.21.15.3:0/2272321008 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f799c005590 con 0x7f79bc139170 2023-10-03T10:18:46.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.984+0000 7f79a37fe700 1 -- 172.21.15.3:0/2272321008 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f79b4019020 con 0x7f79bc139170 2023-10-03T10:18:46.985 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:46.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.986+0000 7f441e75e700 1 -- 172.21.15.3:0/2705135373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f441806fb20 msgr2=0x7f4418118cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:46.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.986+0000 7f441e75e700 1 --2- 172.21.15.3:0/2705135373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f441806fb20 0x7f4418118cd0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f440800d9e0 tx=0x7f440800dda0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.986+0000 7f441e75e700 1 -- 172.21.15.3:0/2705135373 shutdown_connections 2023-10-03T10:18:46.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.986+0000 7f441e75e700 1 --2- 172.21.15.3:0/2705135373 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4418130e50 0x7f4418119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.986+0000 7f441e75e700 1 --2- 172.21.15.3:0/2705135373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f441806fb20 0x7f4418118cd0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.986+0000 7f441e75e700 1 --2- 172.21.15.3:0/2705135373 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4418130540 0x7f4418119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.986+0000 7f441e75e700 1 -- 172.21.15.3:0/2705135373 >> 172.21.15.3:0/2705135373 conn(0x7f44180a2aa0 msgr2=0x7f441812d570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:46.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.986+0000 7f441e75e700 1 -- 172.21.15.3:0/2705135373 shutdown_connections 2023-10-03T10:18:46.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.986+0000 7f441e75e700 1 -- 172.21.15.3:0/2705135373 wait complete. 2023-10-03T10:18:46.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.987+0000 7f79c107d700 1 -- 172.21.15.3:0/2272321008 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79bc139170 msgr2=0x7f79bc120800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:46.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.987+0000 7f79c107d700 1 --2- 172.21.15.3:0/2272321008 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79bc139170 0x7f79bc120800 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f79b400ba10 tx=0x7f79b400bdd0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.987+0000 7f79c107d700 1 -- 172.21.15.3:0/2272321008 shutdown_connections 2023-10-03T10:18:46.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.987+0000 7f79c107d700 1 --2- 172.21.15.3:0/2272321008 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79bc139170 0x7f79bc120800 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.987+0000 7f79c107d700 1 --2- 172.21.15.3:0/2272321008 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79bc09d6f0 0x7f79bc1c8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.987+0000 7f79c107d700 1 --2- 172.21.15.3:0/2272321008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79bc09e1d0 0x7f79bc1c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:46.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.987+0000 7f79c107d700 1 -- 172.21.15.3:0/2272321008 >> 172.21.15.3:0/2272321008 conn(0x7f79bc066f50 msgr2=0x7f79bc136b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:46.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.988+0000 7f79c107d700 1 -- 172.21.15.3:0/2272321008 shutdown_connections 2023-10-03T10:18:46.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:46.988+0000 7f79c107d700 1 -- 172.21.15.3:0/2272321008 wait complete. 2023-10-03T10:18:46.993 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:47.003 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:47.004 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:47.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:47.050+0000 7f7bd2b07700 1 -- 172.21.15.3:0/3475418148 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7bcc06c360 con 0x7f7bcc09d6f0 2023-10-03T10:18:47.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:47.050+0000 7f7bc97fa700 1 -- 172.21.15.3:0/3475418148 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7bc002b070 con 0x7f7bcc09d6f0 2023-10-03T10:18:47.051 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:47.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:47.052+0000 7f7bd2b07700 1 -- 172.21.15.3:0/3475418148 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7bcc09d6f0 msgr2=0x7f7bcc1a5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:47.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:47.052+0000 7f7bd2b07700 1 --2- 172.21.15.3:0/3475418148 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7bcc09d6f0 0x7f7bcc1a5e20 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc06e470 tx=0x7f7bc0004750 comp rx=0 tx=0).stop 2023-10-03T10:18:47.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:47.052+0000 7f7bd2b07700 1 -- 172.21.15.3:0/3475418148 shutdown_connections 2023-10-03T10:18:47.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:47.052+0000 7f7bd2b07700 1 --2- 172.21.15.3:0/3475418148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bcc09e410 0x7f7bcc1a68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:47.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:47.052+0000 7f7bd2b07700 1 --2- 172.21.15.3:0/3475418148 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7bcc09d6f0 0x7f7bcc1a5e20 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:47.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:47.052+0000 7f7bd2b07700 1 --2- 172.21.15.3:0/3475418148 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7bcc09e000 0x7f7bcc1a6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:47.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:47.052+0000 7f7bd2b07700 1 -- 172.21.15.3:0/3475418148 >> 172.21.15.3:0/3475418148 conn(0x7f7bcc066f50 msgr2=0x7f7bcc1bae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:47.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:47.053+0000 7f7bd2b07700 1 -- 172.21.15.3:0/3475418148 shutdown_connections 2023-10-03T10:18:47.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:47.053+0000 7f7bd2b07700 1 -- 172.21.15.3:0/3475418148 wait complete. 2023-10-03T10:18:47.066 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:47.994 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:48.004 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:48.005 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:48.067 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:48.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.157+0000 7f9eefa6d700 1 Processor -- start 2023-10-03T10:18:48.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.157+0000 7f9eefa6d700 1 -- start start 2023-10-03T10:18:48.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.158+0000 7f9eefa6d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ee809e930 0x7f9ee809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:48.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.158+0000 7f9eefa6d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ee809f240 0x7f9ee81317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:48.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.158+0000 7f9eefa6d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee8070940 con 0x7f9ee809f650 2023-10-03T10:18:48.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.158+0000 7f9eefa6d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee806fd60 con 0x7f9ee809f240 2023-10-03T10:18:48.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.158+0000 7f9eefa6d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee806fa80 con 0x7f9ee809e930 2023-10-03T10:18:48.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.158+0000 7f9eed809700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ee809e930 0x7f9ee809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:48.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.158+0000 7f9eed008700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ee809f240 0x7f9ee81317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:48.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.158+0000 7f9eed809700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ee809e930 0x7f9ee809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:37734/0 (socket says 172.21.15.3:37734) 2023-10-03T10:18:48.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.158+0000 7f9eed008700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ee809f240 0x7f9ee81317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44640/0 (socket says 172.21.15.3:44640) 2023-10-03T10:18:48.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.158+0000 7f9eed809700 1 -- 172.21.15.3:0/1228587352 learned_addr learned my addr 172.21.15.3:0/1228587352 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:48.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.159+0000 7f9eed008700 1 -- 172.21.15.3:0/1228587352 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ee809e930 msgr2=0x7f9ee809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:48.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.159+0000 7f9eed008700 1 --2- 172.21.15.3:0/1228587352 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ee809e930 0x7f9ee809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.159+0000 7f9eed008700 1 -- 172.21.15.3:0/1228587352 >> v1:172.21.15.3:6789/0 conn(0x7f9ee809f650 legacy=0x7f9ee8131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:48.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.159+0000 7f9eed008700 1 -- 172.21.15.3:0/1228587352 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee809fb60 con 0x7f9ee809f240 2023-10-03T10:18:48.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.159+0000 7f9eed008700 1 --2- 172.21.15.3:0/1228587352 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ee809f240 0x7f9ee81317f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009870 tx=0x7f9ed8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a37f844f83ad88e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:48.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.159+0000 7f9eec807700 1 -- 172.21.15.3:0/1228587352 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8011830 con 0x7f9ee809f240 2023-10-03T10:18:48.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.159+0000 7f9eec807700 1 -- 172.21.15.3:0/1228587352 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed8011990 con 0x7f9ee809f240 2023-10-03T10:18:48.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.159+0000 7f9eec807700 1 -- 172.21.15.3:0/1228587352 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8011d20 con 0x7f9ee809f240 2023-10-03T10:18:48.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.160+0000 7f9eefa6d700 1 -- 172.21.15.3:0/1228587352 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ee809f240 msgr2=0x7f9ee81317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:48.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.160+0000 7f9eefa6d700 1 --2- 172.21.15.3:0/1228587352 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ee809f240 0x7f9ee81317f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009870 tx=0x7f9ed8009b80 comp rx=0 tx=0).stop 2023-10-03T10:18:48.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.160+0000 7f9eefa6d700 1 -- 172.21.15.3:0/1228587352 shutdown_connections 2023-10-03T10:18:48.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.160+0000 7f9eefa6d700 1 --2- 172.21.15.3:0/1228587352 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ee809e930 0x7f9ee809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.160+0000 7f9eefa6d700 1 --2- 172.21.15.3:0/1228587352 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ee809f240 0x7f9ee81317f0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.160+0000 7f9eefa6d700 1 -- 172.21.15.3:0/1228587352 >> 172.21.15.3:0/1228587352 conn(0x7f9ee80a4210 msgr2=0x7f9ee80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:48.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.160+0000 7f9eefa6d700 1 -- 172.21.15.3:0/1228587352 shutdown_connections 2023-10-03T10:18:48.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.160+0000 7f9eefa6d700 1 -- 172.21.15.3:0/1228587352 wait complete. 2023-10-03T10:18:48.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.162+0000 7f9eefa6d700 1 Processor -- start 2023-10-03T10:18:48.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.162+0000 7f9eefa6d700 1 -- start start 2023-10-03T10:18:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.162+0000 7f9eefa6d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ee809e930 0x7f9ee81a2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.162+0000 7f9eefa6d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee809f240 0x7f9ee81a3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.162+0000 7f9eefa6d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ee809f650 0x7f9ee81a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.162+0000 7f9eefa6d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee81a6730 con 0x7f9ee809f240 2023-10-03T10:18:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.162+0000 7f9eefa6d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee81b6240 con 0x7f9ee809f650 2023-10-03T10:18:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.162+0000 7f9eefa6d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee81a5cb0 con 0x7f9ee809e930 2023-10-03T10:18:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.162+0000 7f9eed809700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ee809e930 0x7f9ee81a2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.162+0000 7f9eed809700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ee809e930 0x7f9ee81a2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:37738/0 (socket says 172.21.15.3:37738) 2023-10-03T10:18:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.162+0000 7f9eed809700 1 -- 172.21.15.3:0/3572264847 learned_addr learned my addr 172.21.15.3:0/3572264847 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.162+0000 7f9eee00a700 1 --2- 172.21.15.3:0/3572264847 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ee809f650 0x7f9ee81a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.162+0000 7f9eed008700 1 --2- 172.21.15.3:0/3572264847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee809f240 0x7f9ee81a3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:48.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.163+0000 7f9eee00a700 1 -- 172.21.15.3:0/3572264847 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ee809e930 msgr2=0x7f9ee81a2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:48.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.163+0000 7f9eee00a700 1 --2- 172.21.15.3:0/3572264847 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ee809e930 0x7f9ee81a2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.163+0000 7f9eee00a700 1 -- 172.21.15.3:0/3572264847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee809f240 msgr2=0x7f9ee81a3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:48.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.163+0000 7f9eee00a700 1 --2- 172.21.15.3:0/3572264847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee809f240 0x7f9ee81a3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.163+0000 7f9eee00a700 1 -- 172.21.15.3:0/3572264847 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed80094a0 con 0x7f9ee809f650 2023-10-03T10:18:48.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.163+0000 7f9eed809700 1 --2- 172.21.15.3:0/3572264847 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ee809e930 0x7f9ee81a2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:48.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.163+0000 7f9eee00a700 1 --2- 172.21.15.3:0/3572264847 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ee809f650 0x7f9ee81a38d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f9ee809faf0 tx=0x7f9ee000ce70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:48.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.163+0000 7f9ede7fc700 1 -- 172.21.15.3:0/3572264847 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0003c00 con 0x7f9ee809f650 2023-10-03T10:18:48.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.163+0000 7f9ede7fc700 1 -- 172.21.15.3:0/3572264847 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee0003d60 con 0x7f9ee809f650 2023-10-03T10:18:48.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.163+0000 7f9eefa6d700 1 -- 172.21.15.3:0/3572264847 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee80a0190 con 0x7f9ee809f650 2023-10-03T10:18:48.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.163+0000 7f9ede7fc700 1 -- 172.21.15.3:0/3572264847 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee000a9c0 con 0x7f9ee809f650 2023-10-03T10:18:48.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.163+0000 7f9eefa6d700 1 -- 172.21.15.3:0/3572264847 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee80a38e0 con 0x7f9ee809f650 2023-10-03T10:18:48.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.165+0000 7f9ede7fc700 1 -- 172.21.15.3:0/3572264847 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9ee000ab20 con 0x7f9ee809f650 2023-10-03T10:18:48.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.165+0000 7f9ede7fc700 1 -- 172.21.15.3:0/3572264847 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9ee0005880 con 0x7f9ee809f650 2023-10-03T10:18:48.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.165+0000 7f9eefa6d700 1 -- 172.21.15.3:0/3572264847 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee8074ea0 con 0x7f9ee809f650 2023-10-03T10:18:48.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.174+0000 7f9ede7fc700 1 -- 172.21.15.3:0/3572264847 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9ee0044060 con 0x7f9ee809f650 2023-10-03T10:18:48.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.187+0000 7fe9ea275700 1 Processor -- start 2023-10-03T10:18:48.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.187+0000 7fe9ea275700 1 -- start start 2023-10-03T10:18:48.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.188+0000 7fe9ea275700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe9e409d6f0 0x7fe9e409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:48.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.188+0000 7fe9ea275700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9e409e000 0x7fe9e4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:48.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.188+0000 7fe9ea275700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9e409f130 con 0x7fe9e409e410 2023-10-03T10:18:48.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.188+0000 7fe9ea275700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9e409f520 con 0x7fe9e409d6f0 2023-10-03T10:18:48.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.188+0000 7fe9ea275700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e409e930 con 0x7fe9e409e000 2023-10-03T10:18:48.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.188+0000 7fe9e37fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9e409e000 0x7fe9e4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:48.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.188+0000 7fe9e37fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9e409e000 0x7fe9e4130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:37754/0 (socket says 172.21.15.3:37754) 2023-10-03T10:18:48.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.188+0000 7fe9e37fe700 1 -- 172.21.15.3:0/1596905192 learned_addr learned my addr 172.21.15.3:0/1596905192 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:48.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.188+0000 7fe9e37fe700 1 -- 172.21.15.3:0/1596905192 >> v2:172.21.15.3:3300/0 conn(0x7fe9e409d6f0 msgr2=0x7fe9e409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:48.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.188+0000 7fe9e37fe700 1 --2- 172.21.15.3:0/1596905192 >> v2:172.21.15.3:3300/0 conn(0x7fe9e409d6f0 0x7fe9e409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.188+0000 7fe9e37fe700 1 -- 172.21.15.3:0/1596905192 >> v1:172.21.15.3:6789/0 conn(0x7fe9e409e410 legacy=0x7fe9e4130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:48.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.188+0000 7fe9e37fe700 1 -- 172.21.15.3:0/1596905192 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e409b2d0 con 0x7fe9e409e000 2023-10-03T10:18:48.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.189+0000 7fe9e37fe700 1 --2- 172.21.15.3:0/1596905192 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9e409e000 0x7fe9e4130660 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe9d800bb90 tx=0x7fe9d800bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b684c6c5341b4e00 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:48.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.189+0000 7fe9e2ffd700 1 -- 172.21.15.3:0/1596905192 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d800dd50 con 0x7fe9e409e000 2023-10-03T10:18:48.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.189+0000 7fe9e2ffd700 1 -- 172.21.15.3:0/1596905192 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d8010470 con 0x7fe9e409e000 2023-10-03T10:18:48.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.189+0000 7fe9e2ffd700 1 -- 172.21.15.3:0/1596905192 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8010800 con 0x7fe9e409e000 2023-10-03T10:18:48.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.189+0000 7fe9ea275700 1 -- 172.21.15.3:0/1596905192 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9e409e000 msgr2=0x7fe9e4130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:48.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.189+0000 7fe9ea275700 1 --2- 172.21.15.3:0/1596905192 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9e409e000 0x7fe9e4130660 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe9d800bb90 tx=0x7fe9d800bea0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.190+0000 7fe9ea275700 1 -- 172.21.15.3:0/1596905192 shutdown_connections 2023-10-03T10:18:48.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.190+0000 7fe9ea275700 1 --2- 172.21.15.3:0/1596905192 >> v2:172.21.15.3:3300/0 conn(0x7fe9e409d6f0 0x7fe9e409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.190+0000 7fe9ea275700 1 --2- 172.21.15.3:0/1596905192 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9e409e000 0x7fe9e4130660 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.190+0000 7fe9ea275700 1 -- 172.21.15.3:0/1596905192 >> 172.21.15.3:0/1596905192 conn(0x7fe9e4066f50 msgr2=0x7fe9e409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:48.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.190+0000 7fe9ea275700 1 -- 172.21.15.3:0/1596905192 shutdown_connections 2023-10-03T10:18:48.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.190+0000 7fe9ea275700 1 -- 172.21.15.3:0/1596905192 wait complete. 2023-10-03T10:18:48.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.190+0000 7f3274771700 1 Processor -- start 2023-10-03T10:18:48.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.191+0000 7fe9ea275700 1 Processor -- start 2023-10-03T10:18:48.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.191+0000 7f3274771700 1 -- start start 2023-10-03T10:18:48.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.191+0000 7f3274771700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f326c0a0a30 0x7f326c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:48.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.191+0000 7f3274771700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f326c09d630 0x7f326c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:48.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.191+0000 7f3274771700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f326c133930 0x7f326c133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:48.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.191+0000 7f3274771700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f326c0a2a70 con 0x7f326c0a0a30 2023-10-03T10:18:48.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.191+0000 7f3274771700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326c0a01d0 con 0x7f326c09d630 2023-10-03T10:18:48.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.191+0000 7f3274771700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f326c09e2b0 con 0x7f326c133930 2023-10-03T10:18:48.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.191+0000 7f3271d0c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f326c09d630 0x7f326c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:48.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.191+0000 7f327250d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f326c0a0a30 0x7f326c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:48.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.191+0000 7f3271d0c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f326c09d630 0x7f326c0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44666/0 (socket says 172.21.15.3:44666) 2023-10-03T10:18:48.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.192+0000 7f3271d0c700 1 -- 172.21.15.3:0/3797362565 learned_addr learned my addr 172.21.15.3:0/3797362565 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:48.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.192+0000 7f3271d0c700 1 -- 172.21.15.3:0/3797362565 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f326c133930 msgr2=0x7f326c133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:48.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.192+0000 7f3271d0c700 1 --2- 172.21.15.3:0/3797362565 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f326c133930 0x7f326c133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.192+0000 7f3271d0c700 1 -- 172.21.15.3:0/3797362565 >> v2:172.21.15.3:3300/0 conn(0x7f326c0a0a30 msgr2=0x7f326c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:48.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.192+0000 7f3271d0c700 1 --2- 172.21.15.3:0/3797362565 >> v2:172.21.15.3:3300/0 conn(0x7f326c0a0a30 0x7f326c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.192+0000 7f3271d0c700 1 -- 172.21.15.3:0/3797362565 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326c09dc50 con 0x7f326c09d630 2023-10-03T10:18:48.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.192+0000 7f3271d0c700 1 --2- 172.21.15.3:0/3797362565 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f326c09d630 0x7f326c0a1340 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f325c0098f0 tx=0x7f325c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26fbf617b3c3d494 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:48.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.191+0000 7fe9ea275700 1 -- start start 2023-10-03T10:18:48.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.192+0000 7fe9ea275700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9e409d6f0 0x7fe9e411d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:48.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.192+0000 7fe9ea275700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9e409e000 0x7fe9e411d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:48.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.192+0000 7fe9ea275700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9e409e410 0x7fe9e411db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:48.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.192+0000 7fe9ea275700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e411e5a0 con 0x7fe9e409e000 2023-10-03T10:18:48.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.192+0000 7fe9ea275700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e4120bf0 con 0x7fe9e409e410 2023-10-03T10:18:48.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.192+0000 7fe9ea275700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e411f300 con 0x7fe9e409d6f0 2023-10-03T10:18:48.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.192+0000 7fe9e8812700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9e409e410 0x7fe9e411db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:48.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.192+0000 7fe9e8812700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9e409e410 0x7fe9e411db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44674/0 (socket says 172.21.15.3:44674) 2023-10-03T10:18:48.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.192+0000 7fe9e8812700 1 -- 172.21.15.3:0/198101825 learned_addr learned my addr 172.21.15.3:0/198101825 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:48.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.192+0000 7fe9e3fff700 1 --2- 172.21.15.3:0/198101825 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9e409d6f0 0x7fe9e411d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:48.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.193+0000 7fe9e37fe700 1 --2- 172.21.15.3:0/198101825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9e409e000 0x7fe9e411d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:48.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.193+0000 7fe9e8812700 1 -- 172.21.15.3:0/198101825 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9e409d6f0 msgr2=0x7fe9e411d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:48.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.193+0000 7fe9e8812700 1 --2- 172.21.15.3:0/198101825 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9e409d6f0 0x7fe9e411d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.193+0000 7fe9e8812700 1 -- 172.21.15.3:0/198101825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9e409e000 msgr2=0x7fe9e411d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:48.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.193+0000 7fe9e8812700 1 --2- 172.21.15.3:0/198101825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9e409e000 0x7fe9e411d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.193+0000 7fe9e8812700 1 -- 172.21.15.3:0/198101825 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d800b7f0 con 0x7fe9e409e410 2023-10-03T10:18:48.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.193+0000 7f327150b700 1 -- 172.21.15.3:0/3797362565 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c011830 con 0x7f326c09d630 2023-10-03T10:18:48.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.193+0000 7f327150b700 1 -- 172.21.15.3:0/3797362565 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325c011990 con 0x7f326c09d630 2023-10-03T10:18:48.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.193+0000 7f327150b700 1 -- 172.21.15.3:0/3797362565 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c011d20 con 0x7f326c09d630 2023-10-03T10:18:48.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.193+0000 7fe9e37fe700 1 --2- 172.21.15.3:0/198101825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9e409e000 0x7fe9e411d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:18:48.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.193+0000 7fe9e8812700 1 --2- 172.21.15.3:0/198101825 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9e409e410 0x7fe9e411db50 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00cc50 tx=0x7fe9dc00cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:48.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.194+0000 7f3274771700 1 -- 172.21.15.3:0/3797362565 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f326c09d630 msgr2=0x7f326c0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:48.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.194+0000 7f3274771700 1 --2- 172.21.15.3:0/3797362565 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f326c09d630 0x7f326c0a1340 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f325c0098f0 tx=0x7f325c009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:48.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.194+0000 7fe9e0ff9700 1 -- 172.21.15.3:0/198101825 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc003c00 con 0x7fe9e409e410 2023-10-03T10:18:48.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.194+0000 7fe9e0ff9700 1 -- 172.21.15.3:0/198101825 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9dc003d60 con 0x7fe9e409e410 2023-10-03T10:18:48.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.194+0000 7f3274771700 1 -- 172.21.15.3:0/3797362565 shutdown_connections 2023-10-03T10:18:48.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.194+0000 7f3274771700 1 --2- 172.21.15.3:0/3797362565 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f326c133930 0x7f326c133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.194+0000 7f3274771700 1 --2- 172.21.15.3:0/3797362565 >> v2:172.21.15.3:3300/0 conn(0x7f326c0a0a30 0x7f326c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.194+0000 7f3274771700 1 --2- 172.21.15.3:0/3797362565 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f326c09d630 0x7f326c0a1340 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.194+0000 7f3274771700 1 -- 172.21.15.3:0/3797362565 >> 172.21.15.3:0/3797362565 conn(0x7f326c0a3050 msgr2=0x7f326c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:48.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.194+0000 7fe9ea275700 1 -- 172.21.15.3:0/198101825 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e412f4b0 con 0x7fe9e409e410 2023-10-03T10:18:48.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.194+0000 7fe9ea275700 1 -- 172.21.15.3:0/198101825 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e40a1e70 con 0x7fe9e409e410 2023-10-03T10:18:48.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.194+0000 7fe9e0ff9700 1 -- 172.21.15.3:0/198101825 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc00ab60 con 0x7fe9e409e410 2023-10-03T10:18:48.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.194+0000 7f3274771700 1 -- 172.21.15.3:0/3797362565 shutdown_connections 2023-10-03T10:18:48.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.195+0000 7fe9e0ff9700 1 -- 172.21.15.3:0/198101825 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe9dc00acc0 con 0x7fe9e409e410 2023-10-03T10:18:48.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.195+0000 7f3274771700 1 -- 172.21.15.3:0/3797362565 wait complete. 2023-10-03T10:18:48.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.195+0000 7fe9ea275700 1 -- 172.21.15.3:0/198101825 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c40052e0 con 0x7fe9e409e410 2023-10-03T10:18:48.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.195+0000 7fe9e0ff9700 1 -- 172.21.15.3:0/198101825 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe9dc03f730 con 0x7fe9e409e410 2023-10-03T10:18:48.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.196+0000 7f3274771700 1 Processor -- start 2023-10-03T10:18:48.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.197+0000 7f3274771700 1 -- start start 2023-10-03T10:18:48.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.197+0000 7f3274771700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f326c09d630 0x7f326c1c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:48.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.197+0000 7f3274771700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f326c0a0a30 0x7f326c1c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:48.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.197+0000 7f3274771700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f326c133930 0x7f326c1c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:48.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.197+0000 7f3274771700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c11eea0 con 0x7f326c09d630 2023-10-03T10:18:48.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.197+0000 7f3274771700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326c1207f0 con 0x7f326c0a0a30 2023-10-03T10:18:48.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.197+0000 7f3274771700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f326c120600 con 0x7f326c133930 2023-10-03T10:18:48.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.198+0000 7f327250d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f326c09d630 0x7f326c1c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:48.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.198+0000 7f327250d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f326c09d630 0x7f326c1c8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52576/0 (socket says 172.21.15.3:52576) 2023-10-03T10:18:48.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.198+0000 7f327250d700 1 -- 172.21.15.3:0/2901076346 learned_addr learned my addr 172.21.15.3:0/2901076346 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:48.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.198+0000 7f327250d700 1 -- 172.21.15.3:0/2901076346 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f326c133930 msgr2=0x7f326c1c8ba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:18:48.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.198+0000 7f327250d700 1 --2- 172.21.15.3:0/2901076346 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f326c133930 0x7f326c1c8ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.198+0000 7f327250d700 1 -- 172.21.15.3:0/2901076346 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f326c0a0a30 msgr2=0x7f326c1c8660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:18:48.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.198+0000 7f327250d700 1 --2- 172.21.15.3:0/2901076346 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f326c0a0a30 0x7f326c1c8660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.198+0000 7f327250d700 1 -- 172.21.15.3:0/2901076346 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f325c009550 con 0x7f326c09d630 2023-10-03T10:18:48.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.198+0000 7f327250d700 1 --2- 172.21.15.3:0/2901076346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f326c09d630 0x7f326c1c8120 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f326800e9b0 tx=0x7f326800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:48.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.198+0000 7f3262ffd700 1 -- 172.21.15.3:0/2901076346 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326800c850 con 0x7f326c09d630 2023-10-03T10:18:48.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.198+0000 7f3262ffd700 1 -- 172.21.15.3:0/2901076346 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326800c9b0 con 0x7f326c09d630 2023-10-03T10:18:48.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.198+0000 7f3274771700 1 -- 172.21.15.3:0/2901076346 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f326c122510 con 0x7f326c09d630 2023-10-03T10:18:48.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.198+0000 7f3274771700 1 -- 172.21.15.3:0/2901076346 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326c120df0 con 0x7f326c09d630 2023-10-03T10:18:48.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.199+0000 7f3262ffd700 1 -- 172.21.15.3:0/2901076346 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326800cd20 con 0x7f326c09d630 2023-10-03T10:18:48.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.200+0000 7f3274771700 1 -- 172.21.15.3:0/2901076346 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32540052e0 con 0x7f326c09d630 2023-10-03T10:18:48.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.200+0000 7f3262ffd700 1 -- 172.21.15.3:0/2901076346 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3268013470 con 0x7f326c09d630 2023-10-03T10:18:48.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.200+0000 7f3262ffd700 1 -- 172.21.15.3:0/2901076346 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f326803f910 con 0x7f326c09d630 2023-10-03T10:18:48.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.204+0000 7fe9e0ff9700 1 -- 172.21.15.3:0/198101825 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe9dc0198f0 con 0x7fe9e409e410 2023-10-03T10:18:48.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.206+0000 7f3262ffd700 1 -- 172.21.15.3:0/2901076346 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f32680232f0 con 0x7f326c09d630 2023-10-03T10:18:48.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.263+0000 7f4d25a90700 1 Processor -- start 2023-10-03T10:18:48.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.263+0000 7f4d25a90700 1 -- start start 2023-10-03T10:18:48.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.264+0000 7f4d25a90700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d20133540 0x7f4d20135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:48.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.264+0000 7f4d25a90700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d20135e60 0x7f4d20138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:48.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.264+0000 7f4d25a90700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d200716a0 con 0x7f4d20130540 2023-10-03T10:18:48.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.264+0000 7f4d25a90700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d20071290 con 0x7f4d20133540 2023-10-03T10:18:48.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.264+0000 7f4d25a90700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d20130340 con 0x7f4d20135e60 2023-10-03T10:18:48.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.264+0000 7f4d1f7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4d20130540 0x7f4d20130910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47776/0 (socket says 172.21.15.3:47776) 2023-10-03T10:18:48.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.264+0000 7f4d1f7fe700 1 -- 172.21.15.3:0/2149974216 learned_addr learned my addr 172.21.15.3:0/2149974216 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:48.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.264+0000 7f4d1effd700 1 --2- 172.21.15.3:0/2149974216 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d20133540 0x7f4d20135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:48.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.264+0000 7f4d1ffff700 1 --2- 172.21.15.3:0/2149974216 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d20135e60 0x7f4d20138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:48.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.264+0000 7f4d1effd700 1 -- 172.21.15.3:0/2149974216 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d20135e60 msgr2=0x7f4d20138240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:48.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.264+0000 7f4d1effd700 1 --2- 172.21.15.3:0/2149974216 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d20135e60 0x7f4d20138240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.264+0000 7f4d1e7fc700 1 -- 172.21.15.3:0/2149974216 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1689145165 0 0) 0x7f4d200716a0 con 0x7f4d20130540 2023-10-03T10:18:48.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.264+0000 7f4d1effd700 1 -- 172.21.15.3:0/2149974216 >> v1:172.21.15.3:6789/0 conn(0x7f4d20130540 legacy=0x7f4d20130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:48.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.264+0000 7f4d1effd700 1 -- 172.21.15.3:0/2149974216 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d20070b80 con 0x7f4d20133540 2023-10-03T10:18:48.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.265+0000 7f4d1effd700 1 --2- 172.21.15.3:0/2149974216 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d20133540 0x7f4d20135920 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009870 tx=0x7f4d14009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec6a06e798323a0e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:48.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.265+0000 7f4d1e7fc700 1 -- 172.21.15.3:0/2149974216 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14011830 con 0x7f4d20133540 2023-10-03T10:18:48.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.265+0000 7f4d1e7fc700 1 -- 172.21.15.3:0/2149974216 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d14011990 con 0x7f4d20133540 2023-10-03T10:18:48.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.265+0000 7f4d1e7fc700 1 -- 172.21.15.3:0/2149974216 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14011d20 con 0x7f4d20133540 2023-10-03T10:18:48.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.266+0000 7f4d25a90700 1 -- 172.21.15.3:0/2149974216 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d20133540 msgr2=0x7f4d20135920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:48.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.266+0000 7f4d25a90700 1 --2- 172.21.15.3:0/2149974216 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d20133540 0x7f4d20135920 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009870 tx=0x7f4d14009b80 comp rx=0 tx=0).stop 2023-10-03T10:18:48.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.266+0000 7f4d25a90700 1 -- 172.21.15.3:0/2149974216 shutdown_connections 2023-10-03T10:18:48.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.266+0000 7f4d25a90700 1 --2- 172.21.15.3:0/2149974216 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d20135e60 0x7f4d20138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.266+0000 7f4d25a90700 1 --2- 172.21.15.3:0/2149974216 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d20133540 0x7f4d20135920 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.266+0000 7f4d25a90700 1 -- 172.21.15.3:0/2149974216 >> 172.21.15.3:0/2149974216 conn(0x7f4d200a2aa0 msgr2=0x7f4d200a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:48.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.266+0000 7f4d25a90700 1 -- 172.21.15.3:0/2149974216 shutdown_connections 2023-10-03T10:18:48.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.266+0000 7f4d25a90700 1 -- 172.21.15.3:0/2149974216 wait complete. 2023-10-03T10:18:48.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.267+0000 7f4d25a90700 1 Processor -- start 2023-10-03T10:18:48.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.267+0000 7f4d25a90700 1 -- start start 2023-10-03T10:18:48.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.268+0000 7f4d25a90700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d20130540 0x7f4d20118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:48.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.268+0000 7f4d25a90700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d20133540 0x7f4d20119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:48.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.268+0000 7f4d25a90700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d20135e60 0x7f4d20119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:48.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.268+0000 7f4d25a90700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2009e220 con 0x7f4d20133540 2023-10-03T10:18:48.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.268+0000 7f4d25a90700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2011b480 con 0x7f4d20130540 2023-10-03T10:18:48.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.268+0000 7f4d25a90700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d200a2060 con 0x7f4d20135e60 2023-10-03T10:18:48.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.268+0000 7f4d1f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d20130540 0x7f4d20118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:48.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.268+0000 7f4d1f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d20130540 0x7f4d20118cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44690/0 (socket says 172.21.15.3:44690) 2023-10-03T10:18:48.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.268+0000 7f4d1f7fe700 1 -- 172.21.15.3:0/1145571686 learned_addr learned my addr 172.21.15.3:0/1145571686 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:48.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.268+0000 7f4d1ffff700 1 --2- 172.21.15.3:0/1145571686 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d20135e60 0x7f4d20119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:48.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.268+0000 7f4d1effd700 1 --2- 172.21.15.3:0/1145571686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d20133540 0x7f4d20119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:48.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.268+0000 7f4d1f7fe700 1 -- 172.21.15.3:0/1145571686 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d20135e60 msgr2=0x7f4d20119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:48.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.268+0000 7f4d1f7fe700 1 --2- 172.21.15.3:0/1145571686 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d20135e60 0x7f4d20119750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.268+0000 7f4d1f7fe700 1 -- 172.21.15.3:0/1145571686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d20133540 msgr2=0x7f4d20119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:48.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.268+0000 7f4d1f7fe700 1 --2- 172.21.15.3:0/1145571686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d20133540 0x7f4d20119210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.269+0000 7f4d1f7fe700 1 -- 172.21.15.3:0/1145571686 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d140094a0 con 0x7f4d20130540 2023-10-03T10:18:48.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.269+0000 7f4d1effd700 1 --2- 172.21.15.3:0/1145571686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d20133540 0x7f4d20119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:18:48.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.269+0000 7f4d1f7fe700 1 --2- 172.21.15.3:0/1145571686 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d20130540 0x7f4d20118cd0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000c520 tx=0x7f4d1000c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:48.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.269+0000 7f4d07fff700 1 -- 172.21.15.3:0/1145571686 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d10004c80 con 0x7f4d20130540 2023-10-03T10:18:48.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.269+0000 7f4d07fff700 1 -- 172.21.15.3:0/1145571686 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d10004e20 con 0x7f4d20130540 2023-10-03T10:18:48.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.269+0000 7f4d07fff700 1 -- 172.21.15.3:0/1145571686 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d10007310 con 0x7f4d20130540 2023-10-03T10:18:48.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.269+0000 7f4d25a90700 1 -- 172.21.15.3:0/1145571686 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d2009d780 con 0x7f4d20130540 2023-10-03T10:18:48.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.269+0000 7f4d25a90700 1 -- 172.21.15.3:0/1145571686 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d2011a0b0 con 0x7f4d20130540 2023-10-03T10:18:48.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.270+0000 7f4d07fff700 1 -- 172.21.15.3:0/1145571686 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4d10007470 con 0x7f4d20130540 2023-10-03T10:18:48.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.271+0000 7f4d25a90700 1 -- 172.21.15.3:0/1145571686 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d20074ea0 con 0x7f4d20130540 2023-10-03T10:18:48.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.271+0000 7f4d07fff700 1 -- 172.21.15.3:0/1145571686 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4d1003f500 con 0x7f4d20130540 2023-10-03T10:18:48.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.280+0000 7f4d07fff700 1 -- 172.21.15.3:0/1145571686 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4d100133c0 con 0x7f4d20130540 2023-10-03T10:18:48.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.395+0000 7f9eefa6d700 1 -- 172.21.15.3:0/3572264847 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9ee80a3360 con 0x7f9ee809f650 2023-10-03T10:18:48.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.396+0000 7f9ede7fc700 1 -- 172.21.15.3:0/3572264847 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9ee0012420 con 0x7f9ee809f650 2023-10-03T10:18:48.396 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.398+0000 7f9eefa6d700 1 -- 172.21.15.3:0/3572264847 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ee809f650 msgr2=0x7f9ee81a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.398+0000 7f9eefa6d700 1 --2- 172.21.15.3:0/3572264847 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ee809f650 0x7f9ee81a38d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f9ee809faf0 tx=0x7f9ee000ce70 comp rx=0 tx=0).stop 2023-10-03T10:18:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.398+0000 7f9eefa6d700 1 -- 172.21.15.3:0/3572264847 shutdown_connections 2023-10-03T10:18:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.398+0000 7f9eefa6d700 1 --2- 172.21.15.3:0/3572264847 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9ee809f650 0x7f9ee81a38d0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.398+0000 7f9eefa6d700 1 --2- 172.21.15.3:0/3572264847 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9ee809e930 0x7f9ee81a2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.398+0000 7f9eefa6d700 1 --2- 172.21.15.3:0/3572264847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee809f240 0x7f9ee81a3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.398+0000 7f9eefa6d700 1 -- 172.21.15.3:0/3572264847 >> 172.21.15.3:0/3572264847 conn(0x7f9ee80a4210 msgr2=0x7f9ee81b7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.399+0000 7f9eefa6d700 1 -- 172.21.15.3:0/3572264847 shutdown_connections 2023-10-03T10:18:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.399+0000 7f9eefa6d700 1 -- 172.21.15.3:0/3572264847 wait complete. 2023-10-03T10:18:48.413 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:48.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.425+0000 7fe9ea275700 1 -- 172.21.15.3:0/198101825 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe9c4005590 con 0x7fe9e409e410 2023-10-03T10:18:48.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.426+0000 7fe9e0ff9700 1 -- 172.21.15.3:0/198101825 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe9dc011380 con 0x7fe9e409e410 2023-10-03T10:18:48.426 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:48.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.428+0000 7fe9ea275700 1 -- 172.21.15.3:0/198101825 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9e409e410 msgr2=0x7fe9e411db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:48.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.428+0000 7fe9ea275700 1 --2- 172.21.15.3:0/198101825 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9e409e410 0x7fe9e411db50 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00cc50 tx=0x7fe9dc00cf60 comp rx=0 tx=0).stop 2023-10-03T10:18:48.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.428+0000 7fe9ea275700 1 -- 172.21.15.3:0/198101825 shutdown_connections 2023-10-03T10:18:48.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.428+0000 7fe9ea275700 1 --2- 172.21.15.3:0/198101825 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9e409e410 0x7fe9e411db50 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.428+0000 7fe9ea275700 1 --2- 172.21.15.3:0/198101825 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9e409d6f0 0x7fe9e411d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.428+0000 7fe9ea275700 1 --2- 172.21.15.3:0/198101825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9e409e000 0x7fe9e411d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.428+0000 7fe9ea275700 1 -- 172.21.15.3:0/198101825 >> 172.21.15.3:0/198101825 conn(0x7fe9e4066f50 msgr2=0x7fe9e412f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:48.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.429+0000 7fe9ea275700 1 -- 172.21.15.3:0/198101825 shutdown_connections 2023-10-03T10:18:48.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.429+0000 7fe9ea275700 1 -- 172.21.15.3:0/198101825 wait complete. 2023-10-03T10:18:48.442 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:48.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.446+0000 7f3274771700 1 -- 172.21.15.3:0/2901076346 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3254005590 con 0x7f326c09d630 2023-10-03T10:18:48.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.447+0000 7f3262ffd700 1 -- 172.21.15.3:0/2901076346 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3268021070 con 0x7f326c09d630 2023-10-03T10:18:48.447 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:48.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.449+0000 7f3274771700 1 -- 172.21.15.3:0/2901076346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f326c09d630 msgr2=0x7f326c1c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:48.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.449+0000 7f3274771700 1 --2- 172.21.15.3:0/2901076346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f326c09d630 0x7f326c1c8120 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f326800e9b0 tx=0x7f326800ed70 comp rx=0 tx=0).stop 2023-10-03T10:18:48.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.449+0000 7f3274771700 1 -- 172.21.15.3:0/2901076346 shutdown_connections 2023-10-03T10:18:48.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.449+0000 7f3274771700 1 --2- 172.21.15.3:0/2901076346 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f326c133930 0x7f326c1c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.449+0000 7f3274771700 1 --2- 172.21.15.3:0/2901076346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f326c09d630 0x7f326c1c8120 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.449+0000 7f3274771700 1 --2- 172.21.15.3:0/2901076346 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f326c0a0a30 0x7f326c1c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.449+0000 7f3274771700 1 -- 172.21.15.3:0/2901076346 >> 172.21.15.3:0/2901076346 conn(0x7f326c0a3050 msgr2=0x7f326c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:48.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.449+0000 7f3274771700 1 -- 172.21.15.3:0/2901076346 shutdown_connections 2023-10-03T10:18:48.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.449+0000 7f3274771700 1 -- 172.21.15.3:0/2901076346 wait complete. 2023-10-03T10:18:48.463 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:48.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.491+0000 7f4d25a90700 1 -- 172.21.15.3:0/1145571686 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4d2006b9e0 con 0x7f4d20130540 2023-10-03T10:18:48.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.491+0000 7f4d07fff700 1 -- 172.21.15.3:0/1145571686 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4d10011360 con 0x7f4d20130540 2023-10-03T10:18:48.492 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:48.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.494+0000 7f4d25a90700 1 -- 172.21.15.3:0/1145571686 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d20130540 msgr2=0x7f4d20118cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:48.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.494+0000 7f4d25a90700 1 --2- 172.21.15.3:0/1145571686 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d20130540 0x7f4d20118cd0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000c520 tx=0x7f4d1000c8e0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.494+0000 7f4d25a90700 1 -- 172.21.15.3:0/1145571686 shutdown_connections 2023-10-03T10:18:48.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.494+0000 7f4d25a90700 1 --2- 172.21.15.3:0/1145571686 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4d20135e60 0x7f4d20119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.494+0000 7f4d25a90700 1 --2- 172.21.15.3:0/1145571686 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4d20130540 0x7f4d20118cd0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.494+0000 7f4d25a90700 1 --2- 172.21.15.3:0/1145571686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d20133540 0x7f4d20119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:48.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.494+0000 7f4d25a90700 1 -- 172.21.15.3:0/1145571686 >> 172.21.15.3:0/1145571686 conn(0x7f4d200a2aa0 msgr2=0x7f4d20132040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:48.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.494+0000 7f4d25a90700 1 -- 172.21.15.3:0/1145571686 shutdown_connections 2023-10-03T10:18:48.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:48.494+0000 7f4d25a90700 1 -- 172.21.15.3:0/1145571686 wait complete. 2023-10-03T10:18:48.507 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:49.413 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:49.443 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:49.464 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:49.508 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:49.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.566+0000 7fc6ff2ab700 1 Processor -- start 2023-10-03T10:18:49.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.567+0000 7fc6ff2ab700 1 -- start start 2023-10-03T10:18:49.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.567+0000 7fc6ff2ab700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6f812cc20 0x7fc6f812f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:49.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.567+0000 7fc6ff2ab700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc6f812c2f0 0x7fc6f812fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:49.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.567+0000 7fc6ff2ab700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f809a510 con 0x7fc6f812cff0 2023-10-03T10:18:49.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.567+0000 7fc6ff2ab700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6f812c7d0 con 0x7fc6f812c2f0 2023-10-03T10:18:49.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.567+0000 7fc6ff2ab700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f809c190 con 0x7fc6f812cc20 2023-10-03T10:18:49.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.567+0000 7fc6fc846700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc6f812c2f0 0x7fc6f812fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:49.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.567+0000 7fc6fc846700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc6f812c2f0 0x7fc6f812fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52594/0 (socket says 172.21.15.3:52594) 2023-10-03T10:18:49.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.567+0000 7fc6fc846700 1 -- 172.21.15.3:0/857287556 learned_addr learned my addr 172.21.15.3:0/857287556 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:49.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.568+0000 7fc6fd047700 1 --2- 172.21.15.3:0/857287556 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6f812cc20 0x7fc6f812f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:49.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.568+0000 7fc6fc846700 1 -- 172.21.15.3:0/857287556 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6f812cc20 msgr2=0x7fc6f812f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:49.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.568+0000 7fc6fc846700 1 --2- 172.21.15.3:0/857287556 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6f812cc20 0x7fc6f812f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.568+0000 7fc6fc846700 1 -- 172.21.15.3:0/857287556 >> v1:172.21.15.3:6789/0 conn(0x7fc6f812cff0 legacy=0x7fc6f8136cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:49.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.568+0000 7fc6fc846700 1 -- 172.21.15.3:0/857287556 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f809b540 con 0x7fc6f812c2f0 2023-10-03T10:18:49.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.568+0000 7fc6fc846700 1 --2- 172.21.15.3:0/857287556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6f812c2f0 0x7fc6f812fe70 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80098f0 tx=0x7fc6e8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35470dd842a08408 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:49.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.568+0000 7fc6effff700 1 -- 172.21.15.3:0/857287556 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e800c900 con 0x7fc6f812c2f0 2023-10-03T10:18:49.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.568+0000 7fc6effff700 1 -- 172.21.15.3:0/857287556 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e800ca60 con 0x7fc6f812c2f0 2023-10-03T10:18:49.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.568+0000 7fc6effff700 1 -- 172.21.15.3:0/857287556 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e800cdf0 con 0x7fc6f812c2f0 2023-10-03T10:18:49.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.569+0000 7fc6ff2ab700 1 -- 172.21.15.3:0/857287556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6f812c2f0 msgr2=0x7fc6f812fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:49.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.569+0000 7fc6ff2ab700 1 --2- 172.21.15.3:0/857287556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6f812c2f0 0x7fc6f812fe70 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80098f0 tx=0x7fc6e8009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:49.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.569+0000 7fc6ff2ab700 1 -- 172.21.15.3:0/857287556 shutdown_connections 2023-10-03T10:18:49.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.569+0000 7fc6ff2ab700 1 --2- 172.21.15.3:0/857287556 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6f812cc20 0x7fc6f812f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.569+0000 7fc6ff2ab700 1 --2- 172.21.15.3:0/857287556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6f812c2f0 0x7fc6f812fe70 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.569+0000 7fc6ff2ab700 1 -- 172.21.15.3:0/857287556 >> 172.21.15.3:0/857287556 conn(0x7fc6f809f470 msgr2=0x7fc6f80a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:49.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.569+0000 7fc6ff2ab700 1 -- 172.21.15.3:0/857287556 shutdown_connections 2023-10-03T10:18:49.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.569+0000 7fc6ff2ab700 1 -- 172.21.15.3:0/857287556 wait complete. 2023-10-03T10:18:49.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.571+0000 7fc6ff2ab700 1 Processor -- start 2023-10-03T10:18:49.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.571+0000 7fc6ff2ab700 1 -- start start 2023-10-03T10:18:49.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.572+0000 7fc6ff2ab700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6f812c2f0 0x7fc6f8070fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:49.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.572+0000 7fc6ff2ab700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6f812cc20 0x7fc6f80714e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:49.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.572+0000 7fc6ff2ab700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6f812cff0 0x7fc6f8071a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:49.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.572+0000 7fc6ff2ab700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f8127280 con 0x7fc6f812cc20 2023-10-03T10:18:49.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.572+0000 7fc6ff2ab700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f808f580 con 0x7fc6f812c2f0 2023-10-03T10:18:49.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.572+0000 7fc6ff2ab700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f812b670 con 0x7fc6f812cff0 2023-10-03T10:18:49.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.572+0000 7fc6fd047700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6f812c2f0 0x7fc6f8070fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:49.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.572+0000 7fc6fd047700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6f812c2f0 0x7fc6f8070fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44692/0 (socket says 172.21.15.3:44692) 2023-10-03T10:18:49.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.572+0000 7fc6fd047700 1 -- 172.21.15.3:0/3892133392 learned_addr learned my addr 172.21.15.3:0/3892133392 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:49.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.572+0000 7fc6fc846700 1 --2- 172.21.15.3:0/3892133392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6f812cc20 0x7fc6f80714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:49.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.572+0000 7fc6fd848700 1 --2- 172.21.15.3:0/3892133392 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6f812cff0 0x7fc6f8071a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:49.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.572+0000 7fc6fd047700 1 -- 172.21.15.3:0/3892133392 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6f812cff0 msgr2=0x7fc6f8071a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:49.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.572+0000 7fc6fd047700 1 --2- 172.21.15.3:0/3892133392 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6f812cff0 0x7fc6f8071a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.572+0000 7fc6fd047700 1 -- 172.21.15.3:0/3892133392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6f812cc20 msgr2=0x7fc6f80714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:49.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.572+0000 7fc6fd047700 1 --2- 172.21.15.3:0/3892133392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6f812cc20 0x7fc6f80714e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.572+0000 7fc6fd047700 1 -- 172.21.15.3:0/3892133392 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e8009550 con 0x7fc6f812c2f0 2023-10-03T10:18:49.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.572+0000 7fc6fc846700 1 --2- 172.21.15.3:0/3892133392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6f812cc20 0x7fc6f80714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:18:49.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.572+0000 7fc6fd047700 1 --2- 172.21.15.3:0/3892133392 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6f812c2f0 0x7fc6f8070fa0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fc6f400e9b0 tx=0x7fc6f400ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.573+0000 7fc6edffb700 1 -- 172.21.15.3:0/3892133392 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4009710 con 0x7fc6f812c2f0 2023-10-03T10:18:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.573+0000 7fc6edffb700 1 -- 172.21.15.3:0/3892133392 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f400c850 con 0x7fc6f812c2f0 2023-10-03T10:18:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.573+0000 7fc6ff2ab700 1 -- 172.21.15.3:0/3892133392 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f8127760 con 0x7fc6f812c2f0 2023-10-03T10:18:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.573+0000 7fc6edffb700 1 -- 172.21.15.3:0/3892133392 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f400cbc0 con 0x7fc6f812c2f0 2023-10-03T10:18:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.573+0000 7fc6ff2ab700 1 -- 172.21.15.3:0/3892133392 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f81279d0 con 0x7fc6f812c2f0 2023-10-03T10:18:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.574+0000 7fc6edffb700 1 -- 172.21.15.3:0/3892133392 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc6f400cd20 con 0x7fc6f812c2f0 2023-10-03T10:18:49.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.575+0000 7fc6edffb700 1 -- 172.21.15.3:0/3892133392 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc6f403fb30 con 0x7fc6f812c2f0 2023-10-03T10:18:49.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.575+0000 7fc6ff2ab700 1 -- 172.21.15.3:0/3892133392 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f80747f0 con 0x7fc6f812c2f0 2023-10-03T10:18:49.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.582+0000 7fc6edffb700 1 -- 172.21.15.3:0/3892133392 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc6f4017080 con 0x7fc6f812c2f0 2023-10-03T10:18:49.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.609+0000 7f9527415700 1 Processor -- start 2023-10-03T10:18:49.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.609+0000 7f9527415700 1 -- start start 2023-10-03T10:18:49.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.610+0000 7f9527415700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f952012cf60 0x7f952006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:49.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.610+0000 7f9527415700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f952012f8b0 0x7f952012fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:49.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.610+0000 7f9527415700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f952012cd90 con 0x7f9520130260 2023-10-03T10:18:49.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.610+0000 7f9527415700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952012d330 con 0x7f952012cf60 2023-10-03T10:18:49.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.610+0000 7f9527415700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95201306d0 con 0x7f952012f8b0 2023-10-03T10:18:49.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.610+0000 7f95249b0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f952012f8b0 0x7f952012fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:49.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.610+0000 7f95251b1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f952012cf60 0x7f952006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:49.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.610+0000 7f95249b0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f952012f8b0 0x7f952012fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:37816/0 (socket says 172.21.15.3:37816) 2023-10-03T10:18:49.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.610+0000 7f95249b0700 1 -- 172.21.15.3:0/3269883991 learned_addr learned my addr 172.21.15.3:0/3269883991 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:49.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.610+0000 7f95249b0700 1 -- 172.21.15.3:0/3269883991 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f952012cf60 msgr2=0x7f952006f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:49.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.610+0000 7f95249b0700 1 --2- 172.21.15.3:0/3269883991 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f952012cf60 0x7f952006f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.610+0000 7f95249b0700 1 -- 172.21.15.3:0/3269883991 >> v1:172.21.15.3:6789/0 conn(0x7f9520130260 legacy=0x7f9520136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:49.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.611+0000 7f95249b0700 1 -- 172.21.15.3:0/3269883991 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9520071780 con 0x7f952012f8b0 2023-10-03T10:18:49.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.611+0000 7f95251b1700 1 --2- 172.21.15.3:0/3269883991 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f952012cf60 0x7f952006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:49.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.611+0000 7f95249b0700 1 --2- 172.21.15.3:0/3269883991 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f952012f8b0 0x7f952012fd20 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f951000b9d0 tx=0x7f951000bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b80b5351feebd5ad server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:49.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.611+0000 7f9517fff700 1 -- 172.21.15.3:0/3269883991 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951000cc20 con 0x7f952012f8b0 2023-10-03T10:18:49.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.611+0000 7f9517fff700 1 -- 172.21.15.3:0/3269883991 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951000cd80 con 0x7f952012f8b0 2023-10-03T10:18:49.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.611+0000 7f9517fff700 1 -- 172.21.15.3:0/3269883991 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9510004230 con 0x7f952012f8b0 2023-10-03T10:18:49.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.612+0000 7f9527415700 1 -- 172.21.15.3:0/3269883991 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f952012f8b0 msgr2=0x7f952012fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:49.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.612+0000 7f9527415700 1 --2- 172.21.15.3:0/3269883991 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f952012f8b0 0x7f952012fd20 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f951000b9d0 tx=0x7f951000bce0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.612+0000 7f9527415700 1 -- 172.21.15.3:0/3269883991 shutdown_connections 2023-10-03T10:18:49.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.612+0000 7f9527415700 1 --2- 172.21.15.3:0/3269883991 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f952012cf60 0x7f952006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.612+0000 7f9527415700 1 --2- 172.21.15.3:0/3269883991 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f952012f8b0 0x7f952012fd20 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.612+0000 7f9527415700 1 -- 172.21.15.3:0/3269883991 >> 172.21.15.3:0/3269883991 conn(0x7f95200a2aa0 msgr2=0x7f95200a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:49.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.612+0000 7f9527415700 1 -- 172.21.15.3:0/3269883991 shutdown_connections 2023-10-03T10:18:49.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.612+0000 7f9527415700 1 -- 172.21.15.3:0/3269883991 wait complete. 2023-10-03T10:18:49.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.613+0000 7f9527415700 1 Processor -- start 2023-10-03T10:18:49.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.613+0000 7f9527415700 1 -- start start 2023-10-03T10:18:49.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.614+0000 7f9527415700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f952012cf60 0x7f952009bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:49.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.614+0000 7f9527415700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f952012f8b0 0x7f952009c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:49.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.614+0000 7f9527415700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9520130260 0x7f95201c4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:49.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.614+0000 7f9527415700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95201297d0 con 0x7f952012cf60 2023-10-03T10:18:49.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.614+0000 7f9527415700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9520129fa0 con 0x7f952012f8b0 2023-10-03T10:18:49.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.614+0000 7f9527415700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f952009ec40 con 0x7f9520130260 2023-10-03T10:18:49.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.614+0000 7f95251b1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f952012cf60 0x7f952009bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:49.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.614+0000 7f95251b1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f952012cf60 0x7f952009bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52608/0 (socket says 172.21.15.3:52608) 2023-10-03T10:18:49.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.614+0000 7f95251b1700 1 -- 172.21.15.3:0/581321420 learned_addr learned my addr 172.21.15.3:0/581321420 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:49.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.614+0000 7f95259b2700 1 --2- 172.21.15.3:0/581321420 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9520130260 0x7f95201c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:49.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.614+0000 7f95249b0700 1 --2- 172.21.15.3:0/581321420 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f952012f8b0 0x7f952009c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:49.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.614+0000 7f95259b2700 1 -- 172.21.15.3:0/581321420 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f952012f8b0 msgr2=0x7f952009c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:49.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.614+0000 7f95259b2700 1 --2- 172.21.15.3:0/581321420 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f952012f8b0 0x7f952009c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.614+0000 7f95259b2700 1 -- 172.21.15.3:0/581321420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f952012cf60 msgr2=0x7f952009bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:49.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.614+0000 7f95259b2700 1 --2- 172.21.15.3:0/581321420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f952012cf60 0x7f952009bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.614+0000 7f95259b2700 1 -- 172.21.15.3:0/581321420 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c00d940 con 0x7f9520130260 2023-10-03T10:18:49.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.614+0000 7f95251b1700 1 --2- 172.21.15.3:0/581321420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f952012cf60 0x7f952009bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:49.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.614+0000 7f95259b2700 1 --2- 172.21.15.3:0/581321420 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9520130260 0x7f95201c4030 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f951800cae0 tx=0x7f951800cdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:49.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.615+0000 7f9515ffb700 1 -- 172.21.15.3:0/581321420 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951800a9e0 con 0x7f9520130260 2023-10-03T10:18:49.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.615+0000 7f9515ffb700 1 -- 172.21.15.3:0/581321420 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951800ab40 con 0x7f9520130260 2023-10-03T10:18:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.615+0000 7f9527415700 1 -- 172.21.15.3:0/581321420 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951000b600 con 0x7f9520130260 2023-10-03T10:18:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.615+0000 7f9515ffb700 1 -- 172.21.15.3:0/581321420 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9518012420 con 0x7f9520130260 2023-10-03T10:18:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.615+0000 7f9527415700 1 -- 172.21.15.3:0/581321420 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95201299a0 con 0x7f9520130260 2023-10-03T10:18:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.616+0000 7f9527415700 1 -- 172.21.15.3:0/581321420 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f952009ea80 con 0x7f9520130260 2023-10-03T10:18:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.617+0000 7f9515ffb700 1 -- 172.21.15.3:0/581321420 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9518012580 con 0x7f9520130260 2023-10-03T10:18:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.617+0000 7f9515ffb700 1 -- 172.21.15.3:0/581321420 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f95180401a0 con 0x7f9520130260 2023-10-03T10:18:49.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.622+0000 7f9515ffb700 1 -- 172.21.15.3:0/581321420 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9518023740 con 0x7f9520130260 2023-10-03T10:18:49.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.648+0000 7f8aea8f7700 1 Processor -- start 2023-10-03T10:18:49.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.648+0000 7f8aea8f7700 1 -- start start 2023-10-03T10:18:49.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.648+0000 7f8aea8f7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8ae409d6f0 0x7f8ae409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:49.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.648+0000 7f8aea8f7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8ae409e000 0x7f8ae4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:49.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.648+0000 7f8aea8f7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae409f130 con 0x7f8ae409e410 2023-10-03T10:18:49.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.648+0000 7f8aea8f7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae409f520 con 0x7f8ae409e000 2023-10-03T10:18:49.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.648+0000 7f8aea8f7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae409e930 con 0x7f8ae409d6f0 2023-10-03T10:18:49.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.649+0000 7f8ae37fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8ae409e000 0x7f8ae4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:49.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.649+0000 7f8ae37fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8ae409e000 0x7f8ae4130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52616/0 (socket says 172.21.15.3:52616) 2023-10-03T10:18:49.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.649+0000 7f8ae3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8ae409d6f0 0x7f8ae409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:49.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.649+0000 7f8ae37fe700 1 -- 172.21.15.3:0/1371174981 learned_addr learned my addr 172.21.15.3:0/1371174981 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:49.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.649+0000 7f8ae3fff700 1 -- 172.21.15.3:0/1371174981 >> v2:172.21.15.3:3300/0 conn(0x7f8ae409e000 msgr2=0x7f8ae4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:49.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.649+0000 7f8ae3fff700 1 --2- 172.21.15.3:0/1371174981 >> v2:172.21.15.3:3300/0 conn(0x7f8ae409e000 0x7f8ae4130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.649+0000 7f8ae3fff700 1 -- 172.21.15.3:0/1371174981 >> v1:172.21.15.3:6789/0 conn(0x7f8ae409e410 legacy=0x7f8ae4130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:49.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.649+0000 7f8ae3fff700 1 -- 172.21.15.3:0/1371174981 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae409b2d0 con 0x7f8ae409d6f0 2023-10-03T10:18:49.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.649+0000 7f8ae37fe700 1 --2- 172.21.15.3:0/1371174981 >> v2:172.21.15.3:3300/0 conn(0x7f8ae409e000 0x7f8ae4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:49.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.649+0000 7f8ae3fff700 1 --2- 172.21.15.3:0/1371174981 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8ae409d6f0 0x7f8ae409dac0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009840 tx=0x7f8ad4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d40e8e02bf09f3e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:49.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.649+0000 7f8ae2ffd700 1 -- 172.21.15.3:0/1371174981 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4011830 con 0x7f8ae409d6f0 2023-10-03T10:18:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.650+0000 7f8ae2ffd700 1 -- 172.21.15.3:0/1371174981 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad4011990 con 0x7f8ae409d6f0 2023-10-03T10:18:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.650+0000 7f8ae2ffd700 1 -- 172.21.15.3:0/1371174981 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4011d20 con 0x7f8ae409d6f0 2023-10-03T10:18:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.650+0000 7f8aea8f7700 1 -- 172.21.15.3:0/1371174981 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8ae409d6f0 msgr2=0x7f8ae409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.650+0000 7f8aea8f7700 1 --2- 172.21.15.3:0/1371174981 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8ae409d6f0 0x7f8ae409dac0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009840 tx=0x7f8ad4009b50 comp rx=0 tx=0).stop 2023-10-03T10:18:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.650+0000 7f8aea8f7700 1 -- 172.21.15.3:0/1371174981 shutdown_connections 2023-10-03T10:18:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.650+0000 7f8aea8f7700 1 --2- 172.21.15.3:0/1371174981 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8ae409d6f0 0x7f8ae409dac0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.650+0000 7f8aea8f7700 1 --2- 172.21.15.3:0/1371174981 >> v2:172.21.15.3:3300/0 conn(0x7f8ae409e000 0x7f8ae4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.650+0000 7f8aea8f7700 1 -- 172.21.15.3:0/1371174981 >> 172.21.15.3:0/1371174981 conn(0x7f8ae4066f50 msgr2=0x7f8ae409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:49.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.650+0000 7f8aea8f7700 1 -- 172.21.15.3:0/1371174981 shutdown_connections 2023-10-03T10:18:49.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.650+0000 7f8aea8f7700 1 -- 172.21.15.3:0/1371174981 wait complete. 2023-10-03T10:18:49.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.651+0000 7f8aea8f7700 1 Processor -- start 2023-10-03T10:18:49.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.652+0000 7f8aea8f7700 1 -- start start 2023-10-03T10:18:49.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.652+0000 7f8aea8f7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8ae409d6f0 0x7f8ae412eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:49.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.652+0000 7f8aea8f7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae409e000 0x7f8ae412f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.652+0000 7f8aea8f7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ae409e410 0x7f8ae412f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.652+0000 7f8aea8f7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae412ffc0 con 0x7f8ae409e000 2023-10-03T10:18:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.652+0000 7f8aea8f7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae40724c0 con 0x7f8ae409e410 2023-10-03T10:18:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.652+0000 7f8aea8f7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae4132980 con 0x7f8ae409d6f0 2023-10-03T10:18:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.652+0000 7f8ae37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae409e000 0x7f8ae412f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.652+0000 7f8ae3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8ae409d6f0 0x7f8ae412eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.652+0000 7f8ae37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae409e000 0x7f8ae412f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52628/0 (socket says 172.21.15.3:52628) 2023-10-03T10:18:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.652+0000 7f8ae37fe700 1 -- 172.21.15.3:0/1033808025 learned_addr learned my addr 172.21.15.3:0/1033808025 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.652+0000 7f8ae3fff700 1 --2- 172.21.15.3:0/1033808025 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8ae409d6f0 0x7f8ae412eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:37840/0 (socket says 172.21.15.3:37840) 2023-10-03T10:18:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.652+0000 7f8ae8e94700 1 --2- 172.21.15.3:0/1033808025 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ae409e410 0x7f8ae412f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.653+0000 7f8ae8e94700 1 -- 172.21.15.3:0/1033808025 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8ae409d6f0 msgr2=0x7f8ae412eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.653+0000 7f8ae8e94700 1 --2- 172.21.15.3:0/1033808025 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8ae409d6f0 0x7f8ae412eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.653+0000 7f8ae8e94700 1 -- 172.21.15.3:0/1033808025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae409e000 msgr2=0x7f8ae412f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.653+0000 7f8ae8e94700 1 --2- 172.21.15.3:0/1033808025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae409e000 0x7f8ae412f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.653+0000 7f8ae8e94700 1 -- 172.21.15.3:0/1033808025 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad8009550 con 0x7f8ae409e410 2023-10-03T10:18:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.653+0000 7f8ae8e94700 1 --2- 172.21.15.3:0/1033808025 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ae409e410 0x7f8ae412f5e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8ae406dff0 tx=0x7f8adc00ce70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.653+0000 7f8ae37fe700 1 --2- 172.21.15.3:0/1033808025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae409e000 0x7f8ae412f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.653+0000 7f8ae0ff9700 1 -- 172.21.15.3:0/1033808025 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc003c00 con 0x7f8ae409e410 2023-10-03T10:18:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.653+0000 7f8ae0ff9700 1 -- 172.21.15.3:0/1033808025 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8adc003d60 con 0x7f8ae409e410 2023-10-03T10:18:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.653+0000 7f8aea8f7700 1 -- 172.21.15.3:0/1033808025 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad40094a0 con 0x7f8ae409e410 2023-10-03T10:18:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.653+0000 7f8ae0ff9700 1 -- 172.21.15.3:0/1033808025 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc00a9a0 con 0x7f8ae409e410 2023-10-03T10:18:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.653+0000 7f8aea8f7700 1 -- 172.21.15.3:0/1033808025 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae41305f0 con 0x7f8ae409e410 2023-10-03T10:18:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.654+0000 7f8ae0ff9700 1 -- 172.21.15.3:0/1033808025 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8adc00ab00 con 0x7f8ae409e410 2023-10-03T10:18:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.654+0000 7f8aea8f7700 1 -- 172.21.15.3:0/1033808025 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae4074ea0 con 0x7f8ae409e410 2023-10-03T10:18:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.655+0000 7f8ae0ff9700 1 -- 172.21.15.3:0/1033808025 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8adc0473b0 con 0x7f8ae409e410 2023-10-03T10:18:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.660+0000 7f8ae0ff9700 1 -- 172.21.15.3:0/1033808025 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8adc00adb0 con 0x7f8ae409e410 2023-10-03T10:18:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.703+0000 7fbffec47700 1 Processor -- start 2023-10-03T10:18:49.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.703+0000 7fbffec47700 1 -- start start 2023-10-03T10:18:49.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.704+0000 7fbffec47700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbff809d6f0 0x7fbff809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:49.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.704+0000 7fbffec47700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbff809e000 0x7fbff8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:49.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.704+0000 7fbffec47700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbff809e410 0x7fbff8130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:49.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.704+0000 7fbffec47700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff809f520 con 0x7fbff809d6f0 2023-10-03T10:18:49.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.704+0000 7fbffec47700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff809f130 con 0x7fbff809e000 2023-10-03T10:18:49.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.704+0000 7fbffec47700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbff809e930 con 0x7fbff809e410 2023-10-03T10:18:49.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.704+0000 7fbffd1e4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbff809e410 0x7fbff8130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:49.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.704+0000 7fbffc9e3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbff809d6f0 0x7fbff809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:49.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.704+0000 7fbffd1e4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbff809e410 0x7fbff8130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:37850/0 (socket says 172.21.15.3:37850) 2023-10-03T10:18:49.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.704+0000 7fbffd1e4700 1 -- 172.21.15.3:0/231941787 learned_addr learned my addr 172.21.15.3:0/231941787 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:49.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.704+0000 7fbff7fff700 1 --2- 172.21.15.3:0/231941787 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbff809e000 0x7fbff8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:49.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.705+0000 7fbffd1e4700 1 -- 172.21.15.3:0/231941787 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbff809e000 msgr2=0x7fbff8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:49.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.705+0000 7fbffd1e4700 1 --2- 172.21.15.3:0/231941787 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbff809e000 0x7fbff8130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.705+0000 7fbffd1e4700 1 -- 172.21.15.3:0/231941787 >> v2:172.21.15.3:3300/0 conn(0x7fbff809d6f0 msgr2=0x7fbff809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:49.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.705+0000 7fbffd1e4700 1 --2- 172.21.15.3:0/231941787 >> v2:172.21.15.3:3300/0 conn(0x7fbff809d6f0 0x7fbff809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.705+0000 7fbffd1e4700 1 -- 172.21.15.3:0/231941787 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff809b2d0 con 0x7fbff809e410 2023-10-03T10:18:49.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.705+0000 7fbffd1e4700 1 --2- 172.21.15.3:0/231941787 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbff809e410 0x7fbff8130ba0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fbff0009e80 tx=0x7fbff000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ee9cd72b3b7ae04 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:49.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.705+0000 7fbff77fe700 1 -- 172.21.15.3:0/231941787 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff000e070 con 0x7fbff809e410 2023-10-03T10:18:49.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.705+0000 7fbff77fe700 1 -- 172.21.15.3:0/231941787 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff000b8d0 con 0x7fbff809e410 2023-10-03T10:18:49.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.705+0000 7fbff77fe700 1 -- 172.21.15.3:0/231941787 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff000bc80 con 0x7fbff809e410 2023-10-03T10:18:49.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.706+0000 7fbffec47700 1 -- 172.21.15.3:0/231941787 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbff809e410 msgr2=0x7fbff8130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:49.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.706+0000 7fbffec47700 1 --2- 172.21.15.3:0/231941787 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbff809e410 0x7fbff8130ba0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fbff0009e80 tx=0x7fbff000b330 comp rx=0 tx=0).stop 2023-10-03T10:18:49.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.706+0000 7fbffec47700 1 -- 172.21.15.3:0/231941787 shutdown_connections 2023-10-03T10:18:49.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.706+0000 7fbffec47700 1 --2- 172.21.15.3:0/231941787 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbff809e410 0x7fbff8130ba0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.706+0000 7fbffec47700 1 --2- 172.21.15.3:0/231941787 >> v2:172.21.15.3:3300/0 conn(0x7fbff809d6f0 0x7fbff809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.706+0000 7fbffec47700 1 --2- 172.21.15.3:0/231941787 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbff809e000 0x7fbff8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.706+0000 7fbffec47700 1 -- 172.21.15.3:0/231941787 >> 172.21.15.3:0/231941787 conn(0x7fbff8066f50 msgr2=0x7fbff809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:49.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.706+0000 7fbffec47700 1 -- 172.21.15.3:0/231941787 shutdown_connections 2023-10-03T10:18:49.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.706+0000 7fbffec47700 1 -- 172.21.15.3:0/231941787 wait complete. 2023-10-03T10:18:49.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.707+0000 7fbffec47700 1 Processor -- start 2023-10-03T10:18:49.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.707+0000 7fbffec47700 1 -- start start 2023-10-03T10:18:49.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.708+0000 7fbffec47700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbff809d6f0 0x7fbff8120100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:49.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.708+0000 7fbffec47700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbff809e000 0x7fbff8120640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:49.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.708+0000 7fbffec47700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbff809e410 0x7fbff8120b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:49.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.708+0000 7fbffec47700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff81238d0 con 0x7fbff809e000 2023-10-03T10:18:49.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.708+0000 7fbffec47700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff80a1e80 con 0x7fbff809e410 2023-10-03T10:18:49.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.708+0000 7fbffec47700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbff8130360 con 0x7fbff809d6f0 2023-10-03T10:18:49.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.708+0000 7fbffc9e3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbff809d6f0 0x7fbff8120100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:49.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.708+0000 7fbffc9e3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbff809d6f0 0x7fbff8120100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:37858/0 (socket says 172.21.15.3:37858) 2023-10-03T10:18:49.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.708+0000 7fbffc9e3700 1 -- 172.21.15.3:0/1924529478 learned_addr learned my addr 172.21.15.3:0/1924529478 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:49.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.708+0000 7fbff7fff700 1 --2- 172.21.15.3:0/1924529478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbff809e000 0x7fbff8120640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:49.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.708+0000 7fbffd1e4700 1 --2- 172.21.15.3:0/1924529478 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbff809e410 0x7fbff8120b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:49.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.708+0000 7fbff7fff700 1 -- 172.21.15.3:0/1924529478 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbff809d6f0 msgr2=0x7fbff8120100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:49.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.708+0000 7fbff7fff700 1 --2- 172.21.15.3:0/1924529478 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbff809d6f0 0x7fbff8120100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.708+0000 7fbff7fff700 1 -- 172.21.15.3:0/1924529478 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbff809e410 msgr2=0x7fbff8120b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:49.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.708+0000 7fbff7fff700 1 --2- 172.21.15.3:0/1924529478 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbff809e410 0x7fbff8120b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.708+0000 7fbff7fff700 1 -- 172.21.15.3:0/1924529478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff0009b30 con 0x7fbff809e000 2023-10-03T10:18:49.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.708+0000 7fbffc9e3700 1 --2- 172.21.15.3:0/1924529478 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbff809d6f0 0x7fbff8120100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:49.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.709+0000 7fbff7fff700 1 --2- 172.21.15.3:0/1924529478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbff809e000 0x7fbff8120640 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400e900 tx=0x7fbfe400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:49.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.709+0000 7fbff57fa700 1 -- 172.21.15.3:0/1924529478 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4013680 con 0x7fbff809e000 2023-10-03T10:18:49.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.709+0000 7fbff57fa700 1 -- 172.21.15.3:0/1924529478 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe40137e0 con 0x7fbff809e000 2023-10-03T10:18:49.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.709+0000 7fbff57fa700 1 -- 172.21.15.3:0/1924529478 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4013b50 con 0x7fbff809e000 2023-10-03T10:18:49.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.710+0000 7fbffec47700 1 -- 172.21.15.3:0/1924529478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff8123b20 con 0x7fbff809e000 2023-10-03T10:18:49.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.710+0000 7fbffec47700 1 -- 172.21.15.3:0/1924529478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff81229a0 con 0x7fbff809e000 2023-10-03T10:18:49.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.711+0000 7fbffec47700 1 -- 172.21.15.3:0/1924529478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff8074ea0 con 0x7fbff809e000 2023-10-03T10:18:49.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.711+0000 7fbff57fa700 1 -- 172.21.15.3:0/1924529478 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbfe4013cb0 con 0x7fbff809e000 2023-10-03T10:18:49.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.711+0000 7fbff57fa700 1 -- 172.21.15.3:0/1924529478 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fbfe403f690 con 0x7fbff809e000 2023-10-03T10:18:49.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.721+0000 7fbff57fa700 1 -- 172.21.15.3:0/1924529478 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbfe401bc10 con 0x7fbff809e000 2023-10-03T10:18:49.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.843+0000 7fc6ff2ab700 1 -- 172.21.15.3:0/3892133392 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc6f806b250 con 0x7fc6f812c2f0 2023-10-03T10:18:49.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.843+0000 7fc6edffb700 1 -- 172.21.15.3:0/3892133392 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc6f4021020 con 0x7fc6f812c2f0 2023-10-03T10:18:49.843 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:49.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.846+0000 7fc6ff2ab700 1 -- 172.21.15.3:0/3892133392 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6f812c2f0 msgr2=0x7fc6f8070fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:49.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.846+0000 7fc6ff2ab700 1 --2- 172.21.15.3:0/3892133392 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6f812c2f0 0x7fc6f8070fa0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fc6f400e9b0 tx=0x7fc6f400ecc0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.847+0000 7fc6ff2ab700 1 -- 172.21.15.3:0/3892133392 shutdown_connections 2023-10-03T10:18:49.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.847+0000 7fc6ff2ab700 1 --2- 172.21.15.3:0/3892133392 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc6f812cff0 0x7fc6f8071a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.847+0000 7fc6ff2ab700 1 --2- 172.21.15.3:0/3892133392 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc6f812c2f0 0x7fc6f8070fa0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.847+0000 7fc6ff2ab700 1 --2- 172.21.15.3:0/3892133392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6f812cc20 0x7fc6f80714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.847+0000 7fc6ff2ab700 1 -- 172.21.15.3:0/3892133392 >> 172.21.15.3:0/3892133392 conn(0x7fc6f809f470 msgr2=0x7fc6f80a1890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:49.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.847+0000 7fc6ff2ab700 1 -- 172.21.15.3:0/3892133392 shutdown_connections 2023-10-03T10:18:49.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.847+0000 7fc6ff2ab700 1 -- 172.21.15.3:0/3892133392 wait complete. 2023-10-03T10:18:49.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.864+0000 7f9527415700 1 -- 172.21.15.3:0/581321420 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f952009ea80 con 0x7f9520130260 2023-10-03T10:18:49.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.865+0000 7f9515ffb700 1 -- 172.21.15.3:0/581321420 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9518029020 con 0x7f9520130260 2023-10-03T10:18:49.865 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:49.866 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:49.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.867+0000 7f9527415700 1 -- 172.21.15.3:0/581321420 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9520130260 msgr2=0x7f95201c4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:49.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.867+0000 7f9527415700 1 --2- 172.21.15.3:0/581321420 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9520130260 0x7f95201c4030 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f951800cae0 tx=0x7f951800cdf0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.867+0000 7f9527415700 1 -- 172.21.15.3:0/581321420 shutdown_connections 2023-10-03T10:18:49.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.867+0000 7f9527415700 1 --2- 172.21.15.3:0/581321420 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9520130260 0x7f95201c4030 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.867+0000 7f9527415700 1 --2- 172.21.15.3:0/581321420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f952012cf60 0x7f952009bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.867+0000 7f9527415700 1 --2- 172.21.15.3:0/581321420 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f952012f8b0 0x7f952009c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.867+0000 7f9527415700 1 -- 172.21.15.3:0/581321420 >> 172.21.15.3:0/581321420 conn(0x7f95200a2aa0 msgr2=0x7f952006e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:49.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.867+0000 7f9527415700 1 -- 172.21.15.3:0/581321420 shutdown_connections 2023-10-03T10:18:49.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.867+0000 7f9527415700 1 -- 172.21.15.3:0/581321420 wait complete. 2023-10-03T10:18:49.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.869+0000 7f8aea8f7700 1 -- 172.21.15.3:0/1033808025 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8ae4130de0 con 0x7f8ae409e410 2023-10-03T10:18:49.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.869+0000 7f8ae0ff9700 1 -- 172.21.15.3:0/1033808025 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8adc011380 con 0x7f8ae409e410 2023-10-03T10:18:49.869 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:49.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.872+0000 7f8aea8f7700 1 -- 172.21.15.3:0/1033808025 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ae409e410 msgr2=0x7f8ae412f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:49.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.872+0000 7f8aea8f7700 1 --2- 172.21.15.3:0/1033808025 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ae409e410 0x7f8ae412f5e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8ae406dff0 tx=0x7f8adc00ce70 comp rx=0 tx=0).stop 2023-10-03T10:18:49.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.872+0000 7f8aea8f7700 1 -- 172.21.15.3:0/1033808025 shutdown_connections 2023-10-03T10:18:49.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.872+0000 7f8aea8f7700 1 --2- 172.21.15.3:0/1033808025 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8ae409e410 0x7f8ae412f5e0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.872+0000 7f8aea8f7700 1 --2- 172.21.15.3:0/1033808025 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8ae409d6f0 0x7f8ae412eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.872+0000 7f8aea8f7700 1 --2- 172.21.15.3:0/1033808025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae409e000 0x7f8ae412f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.872+0000 7f8aea8f7700 1 -- 172.21.15.3:0/1033808025 >> 172.21.15.3:0/1033808025 conn(0x7f8ae4066f50 msgr2=0x7f8ae40a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:49.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.872+0000 7f8aea8f7700 1 -- 172.21.15.3:0/1033808025 shutdown_connections 2023-10-03T10:18:49.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.872+0000 7f8aea8f7700 1 -- 172.21.15.3:0/1033808025 wait complete. 2023-10-03T10:18:49.883 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:49.887 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:49.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.941+0000 7fbffec47700 1 -- 172.21.15.3:0/1924529478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbff81234e0 con 0x7fbff809e000 2023-10-03T10:18:49.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.942+0000 7fbff57fa700 1 -- 172.21.15.3:0/1924529478 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbfe4021020 con 0x7fbff809e000 2023-10-03T10:18:49.942 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:49.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.944+0000 7fbffec47700 1 -- 172.21.15.3:0/1924529478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbff809e000 msgr2=0x7fbff8120640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:49.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.944+0000 7fbffec47700 1 --2- 172.21.15.3:0/1924529478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbff809e000 0x7fbff8120640 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400e900 tx=0x7fbfe400ec10 comp rx=0 tx=0).stop 2023-10-03T10:18:49.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.944+0000 7fbffec47700 1 -- 172.21.15.3:0/1924529478 shutdown_connections 2023-10-03T10:18:49.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.944+0000 7fbffec47700 1 --2- 172.21.15.3:0/1924529478 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbff809e410 0x7fbff8120b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.944+0000 7fbffec47700 1 --2- 172.21.15.3:0/1924529478 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbff809d6f0 0x7fbff8120100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.944+0000 7fbffec47700 1 --2- 172.21.15.3:0/1924529478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbff809e000 0x7fbff8120640 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:49.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.944+0000 7fbffec47700 1 -- 172.21.15.3:0/1924529478 >> 172.21.15.3:0/1924529478 conn(0x7fbff8066f50 msgr2=0x7fbff8070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:49.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.944+0000 7fbffec47700 1 -- 172.21.15.3:0/1924529478 shutdown_connections 2023-10-03T10:18:49.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:49.944+0000 7fbffec47700 1 -- 172.21.15.3:0/1924529478 wait complete. 2023-10-03T10:18:49.957 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:50.867 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:50.883 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:50.887 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:50.959 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:51.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.021+0000 7fc88d25c700 1 Processor -- start 2023-10-03T10:18:51.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.021+0000 7fc88d25c700 1 -- start start 2023-10-03T10:18:51.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.021+0000 7fc88d25c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc88812cf60 0x7fc88806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:51.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.021+0000 7fc88d25c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc888130430 0x7fc888137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:51.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.021+0000 7fc88d25c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc88812cd90 con 0x7fc88812f8b0 2023-10-03T10:18:51.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.021+0000 7fc88d25c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc88812d330 con 0x7fc888130430 2023-10-03T10:18:51.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.021+0000 7fc88d25c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc888071780 con 0x7fc88812cf60 2023-10-03T10:18:51.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.021+0000 7fc886d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc88812cf60 0x7fc88806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:51.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.022+0000 7fc886d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc88812cf60 0x7fc88806f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44738/0 (socket says 172.21.15.3:44738) 2023-10-03T10:18:51.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.022+0000 7fc88759e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc888130430 0x7fc888137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:51.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.022+0000 7fc886d9d700 1 -- 172.21.15.3:0/4243091771 learned_addr learned my addr 172.21.15.3:0/4243091771 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:51.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.022+0000 7fc886d9d700 1 -- 172.21.15.3:0/4243091771 >> v2:172.21.15.3:3300/0 conn(0x7fc888130430 msgr2=0x7fc888137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:51.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.022+0000 7fc886d9d700 1 --2- 172.21.15.3:0/4243091771 >> v2:172.21.15.3:3300/0 conn(0x7fc888130430 0x7fc888137c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.022+0000 7fc886d9d700 1 -- 172.21.15.3:0/4243091771 >> v1:172.21.15.3:6789/0 conn(0x7fc88812f8b0 legacy=0x7fc88812fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:51.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.022+0000 7fc886d9d700 1 -- 172.21.15.3:0/4243091771 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc888071a40 con 0x7fc88812cf60 2023-10-03T10:18:51.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.022+0000 7fc886d9d700 1 --2- 172.21.15.3:0/4243091771 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc88812cf60 0x7fc88806f5d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc87800b9a0 tx=0x7fc87800bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f82f9dbb1b7e19c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:51.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.022+0000 7fc885d9b700 1 -- 172.21.15.3:0/4243091771 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87800caa0 con 0x7fc88812cf60 2023-10-03T10:18:51.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.023+0000 7fc885d9b700 1 -- 172.21.15.3:0/4243091771 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87800cc00 con 0x7fc88812cf60 2023-10-03T10:18:51.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.023+0000 7fc885d9b700 1 -- 172.21.15.3:0/4243091771 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87800f460 con 0x7fc88812cf60 2023-10-03T10:18:51.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.023+0000 7fc88d25c700 1 -- 172.21.15.3:0/4243091771 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc88812cf60 msgr2=0x7fc88806f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:51.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.023+0000 7fc88d25c700 1 --2- 172.21.15.3:0/4243091771 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc88812cf60 0x7fc88806f5d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc87800b9a0 tx=0x7fc87800bcb0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.023+0000 7fc88d25c700 1 -- 172.21.15.3:0/4243091771 shutdown_connections 2023-10-03T10:18:51.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.023+0000 7fc88d25c700 1 --2- 172.21.15.3:0/4243091771 >> v2:172.21.15.3:3300/0 conn(0x7fc888130430 0x7fc888137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.023+0000 7fc88d25c700 1 --2- 172.21.15.3:0/4243091771 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc88812cf60 0x7fc88806f5d0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.023+0000 7fc88d25c700 1 -- 172.21.15.3:0/4243091771 >> 172.21.15.3:0/4243091771 conn(0x7fc8880a2aa0 msgr2=0x7fc8880a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:51.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.023+0000 7fc88d25c700 1 -- 172.21.15.3:0/4243091771 shutdown_connections 2023-10-03T10:18:51.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.024+0000 7fc88d25c700 1 -- 172.21.15.3:0/4243091771 wait complete. 2023-10-03T10:18:51.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.025+0000 7fc88d25c700 1 Processor -- start 2023-10-03T10:18:51.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.025+0000 7fc88d25c700 1 -- start start 2023-10-03T10:18:51.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.025+0000 7fc88d25c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc88812cf60 0x7fc8881c4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:51.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.025+0000 7fc88d25c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc88812f8b0 0x7fc8881c4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:51.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.025+0000 7fc88d25c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc888130430 0x7fc88809af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:51.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.025+0000 7fc88d25c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88809e8d0 con 0x7fc888130430 2023-10-03T10:18:51.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.025+0000 7fc88d25c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc88809eab0 con 0x7fc88812f8b0 2023-10-03T10:18:51.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.025+0000 7fc88d25c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8881b9010 con 0x7fc88812cf60 2023-10-03T10:18:51.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.026+0000 7fc88659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc88812f8b0 0x7fc8881c4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:51.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.026+0000 7fc886d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc88812cf60 0x7fc8881c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:51.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.026+0000 7fc886d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc88812cf60 0x7fc8881c4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:37874/0 (socket says 172.21.15.3:37874) 2023-10-03T10:18:51.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.026+0000 7fc886d9d700 1 -- 172.21.15.3:0/1470396168 learned_addr learned my addr 172.21.15.3:0/1470396168 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:51.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.026+0000 7fc88759e700 1 --2- 172.21.15.3:0/1470396168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc888130430 0x7fc88809af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:51.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.026+0000 7fc886d9d700 1 -- 172.21.15.3:0/1470396168 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc88812f8b0 msgr2=0x7fc8881c4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:51.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.026+0000 7fc886d9d700 1 --2- 172.21.15.3:0/1470396168 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc88812f8b0 0x7fc8881c4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.026+0000 7fc886d9d700 1 -- 172.21.15.3:0/1470396168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc888130430 msgr2=0x7fc88809af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:51.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.026+0000 7fc886d9d700 1 --2- 172.21.15.3:0/1470396168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc888130430 0x7fc88809af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.026+0000 7fc886d9d700 1 -- 172.21.15.3:0/1470396168 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87800b600 con 0x7fc88812cf60 2023-10-03T10:18:51.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.026+0000 7fc886d9d700 1 --2- 172.21.15.3:0/1470396168 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc88812cf60 0x7fc8881c4610 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc8780090b0 tx=0x7fc878004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:51.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.027+0000 7fc86f7fe700 1 -- 172.21.15.3:0/1470396168 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87801c070 con 0x7fc88812cf60 2023-10-03T10:18:51.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.027+0000 7fc86f7fe700 1 -- 172.21.15.3:0/1470396168 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87800c8a0 con 0x7fc88812cf60 2023-10-03T10:18:51.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.027+0000 7fc88d25c700 1 -- 172.21.15.3:0/1470396168 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8881296e0 con 0x7fc88812cf60 2023-10-03T10:18:51.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.027+0000 7fc86f7fe700 1 -- 172.21.15.3:0/1470396168 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc878004250 con 0x7fc88812cf60 2023-10-03T10:18:51.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.027+0000 7fc88d25c700 1 -- 172.21.15.3:0/1470396168 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc88812d7b0 con 0x7fc88812cf60 2023-10-03T10:18:51.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.028+0000 7fc86f7fe700 1 -- 172.21.15.3:0/1470396168 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc8780043b0 con 0x7fc88812cf60 2023-10-03T10:18:51.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.028+0000 7fc88d25c700 1 -- 172.21.15.3:0/1470396168 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc88809e730 con 0x7fc88812cf60 2023-10-03T10:18:51.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.028+0000 7fc86f7fe700 1 -- 172.21.15.3:0/1470396168 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc878040ee0 con 0x7fc88812cf60 2023-10-03T10:18:51.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.033+0000 7fc86f7fe700 1 -- 172.21.15.3:0/1470396168 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc88809e730 con 0x7fc88812cf60 2023-10-03T10:18:51.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.061+0000 7fa84782e700 1 Processor -- start 2023-10-03T10:18:51.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.062+0000 7f77fabc2700 1 Processor -- start 2023-10-03T10:18:51.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.062+0000 7f77fabc2700 1 -- start start 2023-10-03T10:18:51.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.062+0000 7fa84782e700 1 -- start start 2023-10-03T10:18:51.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.063+0000 7f77fabc2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f77f409d6f0 0x7f77f409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:51.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.063+0000 7fa84782e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa84009e850 0x7fa84009ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:51.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.063+0000 7f77fabc2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77f409e000 0x7f77f41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:51.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.063+0000 7fa84782e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa84009f160 0x7fa840133c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:51.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.063+0000 7fa84782e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa84006bec0 con 0x7fa84009f5b0 2023-10-03T10:18:51.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.063+0000 7fa84782e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8400a0030 con 0x7fa84009f160 2023-10-03T10:18:51.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.063+0000 7fa84782e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa8400a0830 con 0x7fa84009e850 2023-10-03T10:18:51.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.063+0000 7f77fabc2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f409b120 con 0x7f77f409e410 2023-10-03T10:18:51.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.063+0000 7f77fabc2700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f406b460 con 0x7f77f409d6f0 2023-10-03T10:18:51.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.063+0000 7f77fabc2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77f409a7e0 con 0x7f77f409e000 2023-10-03T10:18:51.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.063+0000 7fa8455ca700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa84009e850 0x7fa84009ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:51.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.063+0000 7fa8455ca700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa84009e850 0x7fa84009ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:37878/0 (socket says 172.21.15.3:37878) 2023-10-03T10:18:51.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.063+0000 7fa8455ca700 1 -- 172.21.15.3:0/1718204326 learned_addr learned my addr 172.21.15.3:0/1718204326 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:51.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.063+0000 7fa844dc9700 1 --2- 172.21.15.3:0/1718204326 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa84009f160 0x7fa840133c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:51.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.063+0000 7f77f3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77f409e000 0x7f77f41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:51.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.063+0000 7f77f3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77f409e000 0x7f77f41339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:37876/0 (socket says 172.21.15.3:37876) 2023-10-03T10:18:51.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.063+0000 7f77f3fff700 1 -- 172.21.15.3:0/4166445590 learned_addr learned my addr 172.21.15.3:0/4166445590 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:51.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.063+0000 7fa8455ca700 1 -- 172.21.15.3:0/1718204326 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa84009f160 msgr2=0x7fa840133c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:51.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.063+0000 7fa8455ca700 1 --2- 172.21.15.3:0/1718204326 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa84009f160 0x7fa840133c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.063+0000 7fa8455ca700 1 -- 172.21.15.3:0/1718204326 >> v1:172.21.15.3:6789/0 conn(0x7fa84009f5b0 legacy=0x7fa8401341a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:51.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.063+0000 7fa8455ca700 1 -- 172.21.15.3:0/1718204326 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84006caa0 con 0x7fa84009e850 2023-10-03T10:18:51.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.064+0000 7f77f3fff700 1 -- 172.21.15.3:0/4166445590 >> v2:172.21.15.3:3300/0 conn(0x7f77f409d6f0 msgr2=0x7f77f409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:51.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.064+0000 7f77f3fff700 1 --2- 172.21.15.3:0/4166445590 >> v2:172.21.15.3:3300/0 conn(0x7f77f409d6f0 0x7f77f409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.064+0000 7f77f3fff700 1 -- 172.21.15.3:0/4166445590 >> v1:172.21.15.3:6789/0 conn(0x7f77f409e410 legacy=0x7f77f4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:51.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.064+0000 7f77f3fff700 1 -- 172.21.15.3:0/4166445590 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f409a500 con 0x7f77f409e000 2023-10-03T10:18:51.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.064+0000 7fa8455ca700 1 --2- 172.21.15.3:0/1718204326 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa84009e850 0x7fa84009ec20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009920 tx=0x7fa83c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fc6af88fb439380 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.064+0000 7fa837fff700 1 -- 172.21.15.3:0/1718204326 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c004230 con 0x7fa84009e850 2023-10-03T10:18:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.064+0000 7f77f3fff700 1 --2- 172.21.15.3:0/4166445590 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77f409e000 0x7f77f41339f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009920 tx=0x7f77e0009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ed1313874a88518 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.064+0000 7fa837fff700 1 -- 172.21.15.3:0/1718204326 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83c004390 con 0x7fa84009e850 2023-10-03T10:18:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.064+0000 7fa837fff700 1 -- 172.21.15.3:0/1718204326 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c0036a0 con 0x7fa84009e850 2023-10-03T10:18:51.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.064+0000 7f77f37fe700 1 -- 172.21.15.3:0/4166445590 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e0004230 con 0x7f77f409e000 2023-10-03T10:18:51.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.064+0000 7f77f37fe700 1 -- 172.21.15.3:0/4166445590 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e0004390 con 0x7f77f409e000 2023-10-03T10:18:51.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.064+0000 7f77f37fe700 1 -- 172.21.15.3:0/4166445590 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e0009e70 con 0x7f77f409e000 2023-10-03T10:18:51.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.065+0000 7f77fabc2700 1 -- 172.21.15.3:0/4166445590 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77f409e000 msgr2=0x7f77f41339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:51.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.065+0000 7f77fabc2700 1 --2- 172.21.15.3:0/4166445590 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77f409e000 0x7f77f41339f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009920 tx=0x7f77e0009c30 comp rx=0 tx=0).stop 2023-10-03T10:18:51.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.065+0000 7fa84782e700 1 -- 172.21.15.3:0/1718204326 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa84009e850 msgr2=0x7fa84009ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:51.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.065+0000 7fa84782e700 1 --2- 172.21.15.3:0/1718204326 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa84009e850 0x7fa84009ec20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009920 tx=0x7fa83c009c30 comp rx=0 tx=0).stop 2023-10-03T10:18:51.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.065+0000 7f77fabc2700 1 -- 172.21.15.3:0/4166445590 shutdown_connections 2023-10-03T10:18:51.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.065+0000 7f77fabc2700 1 --2- 172.21.15.3:0/4166445590 >> v2:172.21.15.3:3300/0 conn(0x7f77f409d6f0 0x7f77f409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.065+0000 7f77fabc2700 1 --2- 172.21.15.3:0/4166445590 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77f409e000 0x7f77f41339f0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.065+0000 7f77fabc2700 1 -- 172.21.15.3:0/4166445590 >> 172.21.15.3:0/4166445590 conn(0x7f77f4066f50 msgr2=0x7f77f409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:51.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.065+0000 7fa84782e700 1 -- 172.21.15.3:0/1718204326 shutdown_connections 2023-10-03T10:18:51.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.065+0000 7fa84782e700 1 --2- 172.21.15.3:0/1718204326 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa84009e850 0x7fa84009ec20 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.065+0000 7fa84782e700 1 --2- 172.21.15.3:0/1718204326 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa84009f160 0x7fa840133c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.065+0000 7fa84782e700 1 -- 172.21.15.3:0/1718204326 >> 172.21.15.3:0/1718204326 conn(0x7fa8400a3f70 msgr2=0x7fa8400a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:51.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.065+0000 7f77fabc2700 1 -- 172.21.15.3:0/4166445590 shutdown_connections 2023-10-03T10:18:51.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.065+0000 7fa84782e700 1 -- 172.21.15.3:0/1718204326 shutdown_connections 2023-10-03T10:18:51.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.065+0000 7f77fabc2700 1 -- 172.21.15.3:0/4166445590 wait complete. 2023-10-03T10:18:51.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.065+0000 7fa84782e700 1 -- 172.21.15.3:0/1718204326 wait complete. 2023-10-03T10:18:51.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.067+0000 7f77fabc2700 1 Processor -- start 2023-10-03T10:18:51.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.067+0000 7fa84782e700 1 Processor -- start 2023-10-03T10:18:51.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.067+0000 7f77fabc2700 1 -- start start 2023-10-03T10:18:51.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.067+0000 7fa84782e700 1 -- start start 2023-10-03T10:18:51.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.068+0000 7fa84782e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa84009e850 0x7fa840120600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:51.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.068+0000 7fa84782e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa84009f160 0x7fa840120b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:51.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.068+0000 7fa84782e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa84009f5b0 0x7fa840121080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:51.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.068+0000 7fa84782e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8400a35d0 con 0x7fa84009f5b0 2023-10-03T10:18:51.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.068+0000 7fa84782e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa840122a70 con 0x7fa84009f160 2023-10-03T10:18:51.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.068+0000 7fa84782e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa840124120 con 0x7fa84009e850 2023-10-03T10:18:51.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.068+0000 7fa8455ca700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa84009e850 0x7fa840120600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:51.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.068+0000 7fa8455ca700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa84009e850 0x7fa840120600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:37890/0 (socket says 172.21.15.3:37890) 2023-10-03T10:18:51.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.068+0000 7fa8455ca700 1 -- 172.21.15.3:0/1788680500 learned_addr learned my addr 172.21.15.3:0/1788680500 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:51.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.068+0000 7f77fabc2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77f409d6f0 0x7f77f41a5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:51.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.068+0000 7fa844dc9700 1 --2- 172.21.15.3:0/1788680500 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa84009f160 0x7fa840120b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:51.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.068+0000 7f77fabc2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77f409e000 0x7f77f41a6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:51.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.068+0000 7f77fabc2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77f409e410 0x7f77f41a6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:51.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.068+0000 7f77fabc2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f41a8ec0 con 0x7f77f409e410 2023-10-03T10:18:51.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.068+0000 7f77fabc2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f41b93e0 con 0x7f77f409e000 2023-10-03T10:18:51.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.068+0000 7f77fabc2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77f41a7450 con 0x7f77f409d6f0 2023-10-03T10:18:51.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.068+0000 7f77f895e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77f409d6f0 0x7f77f41a5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:51.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.068+0000 7f77f895e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77f409d6f0 0x7f77f41a5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:37902/0 (socket says 172.21.15.3:37902) 2023-10-03T10:18:51.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.068+0000 7f77f895e700 1 -- 172.21.15.3:0/323814887 learned_addr learned my addr 172.21.15.3:0/323814887 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:51.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.068+0000 7f77f3fff700 1 --2- 172.21.15.3:0/323814887 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77f409e000 0x7f77f41a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:51.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.068+0000 7fa8455ca700 1 -- 172.21.15.3:0/1788680500 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa84009f160 msgr2=0x7fa840120b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:51.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.068+0000 7fa8455ca700 1 --2- 172.21.15.3:0/1788680500 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa84009f160 0x7fa840120b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.068+0000 7fa8455ca700 1 -- 172.21.15.3:0/1788680500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa84009f5b0 msgr2=0x7fa840121080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:51.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.068+0000 7fa8455ca700 1 --2- 172.21.15.3:0/1788680500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa84009f5b0 0x7fa840121080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.068+0000 7fa8455ca700 1 -- 172.21.15.3:0/1788680500 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83c009550 con 0x7fa84009e850 2023-10-03T10:18:51.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.069+0000 7fa8455ca700 1 --2- 172.21.15.3:0/1788680500 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa84009e850 0x7fa840120600 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa83c000c00 tx=0x7fa83c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:51.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.069+0000 7f77f3fff700 1 -- 172.21.15.3:0/323814887 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77f409d6f0 msgr2=0x7f77f41a5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:51.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.069+0000 7f77f3fff700 1 --2- 172.21.15.3:0/323814887 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77f409d6f0 0x7f77f41a5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.069+0000 7f77f3fff700 1 -- 172.21.15.3:0/323814887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77f409e410 msgr2=0x7f77f41a6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:51.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.069+0000 7f77f3fff700 1 --2- 172.21.15.3:0/323814887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77f409e410 0x7f77f41a6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.069+0000 7f77f3fff700 1 -- 172.21.15.3:0/323814887 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e0009550 con 0x7f77f409e000 2023-10-03T10:18:51.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.069+0000 7fa835ffb700 1 -- 172.21.15.3:0/1788680500 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c003e10 con 0x7fa84009e850 2023-10-03T10:18:51.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.069+0000 7fa84782e700 1 -- 172.21.15.3:0/1788680500 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa84006ccc0 con 0x7fa84009e850 2023-10-03T10:18:51.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.069+0000 7fa835ffb700 1 -- 172.21.15.3:0/1788680500 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83c011630 con 0x7fa84009e850 2023-10-03T10:18:51.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.069+0000 7fa835ffb700 1 -- 172.21.15.3:0/1788680500 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c0119a0 con 0x7fa84009e850 2023-10-03T10:18:51.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.069+0000 7fa84782e700 1 -- 172.21.15.3:0/1788680500 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa840123bd0 con 0x7fa84009e850 2023-10-03T10:18:51.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.070+0000 7f77f3fff700 1 --2- 172.21.15.3:0/323814887 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77f409e000 0x7f77f41a6530 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f77f409ac00 tx=0x7f77e0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:51.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.070+0000 7f77f17fa700 1 -- 172.21.15.3:0/323814887 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e000bc20 con 0x7f77f409e000 2023-10-03T10:18:51.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.070+0000 7f77f17fa700 1 -- 172.21.15.3:0/323814887 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e001f070 con 0x7f77f409e000 2023-10-03T10:18:51.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.070+0000 7f77f17fa700 1 -- 172.21.15.3:0/323814887 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e00036a0 con 0x7f77f409e000 2023-10-03T10:18:51.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.070+0000 7f77fabc2700 1 -- 172.21.15.3:0/323814887 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f41a81e0 con 0x7f77f409e000 2023-10-03T10:18:51.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.070+0000 7f77fabc2700 1 -- 172.21.15.3:0/323814887 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f40a21b0 con 0x7f77f409e000 2023-10-03T10:18:51.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.070+0000 7fa84782e700 1 -- 172.21.15.3:0/1788680500 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa84009caf0 con 0x7fa84009e850 2023-10-03T10:18:51.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.071+0000 7fa835ffb700 1 -- 172.21.15.3:0/1788680500 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa83c021070 con 0x7fa84009e850 2023-10-03T10:18:51.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.071+0000 7f77fabc2700 1 -- 172.21.15.3:0/323814887 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f409c710 con 0x7f77f409e000 2023-10-03T10:18:51.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.072+0000 7fa835ffb700 1 -- 172.21.15.3:0/1788680500 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa83c040f30 con 0x7fa84009e850 2023-10-03T10:18:51.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.073+0000 7f77f17fa700 1 -- 172.21.15.3:0/323814887 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f77e0021030 con 0x7f77f409e000 2023-10-03T10:18:51.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.073+0000 7f77f17fa700 1 -- 172.21.15.3:0/323814887 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f77e00411e0 con 0x7f77f409e000 2023-10-03T10:18:51.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.079+0000 7fa835ffb700 1 -- 172.21.15.3:0/1788680500 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa83c011b00 con 0x7fa84009e850 2023-10-03T10:18:51.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.080+0000 7f77f17fa700 1 -- 172.21.15.3:0/323814887 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f77e0019780 con 0x7f77f409e000 2023-10-03T10:18:51.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.122+0000 7f74b8b63700 1 Processor -- start 2023-10-03T10:18:51.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.122+0000 7f74b8b63700 1 -- start start 2023-10-03T10:18:51.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.122+0000 7f74b8b63700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74b40a0a30 0x7f74b40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:51.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.122+0000 7f74b8b63700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74b4138d80 0x7f74b413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:51.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.122+0000 7f74b8b63700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74b40a2a70 con 0x7f74b409d630 2023-10-03T10:18:51.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.122+0000 7f74b8b63700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b40a01d0 con 0x7f74b40a0a30 2023-10-03T10:18:51.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.122+0000 7f74b8b63700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74b409e2b0 con 0x7f74b4138d80 2023-10-03T10:18:51.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.123+0000 7f74b1d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f74b409d630 0x7f74b40a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47828/0 (socket says 172.21.15.3:47828) 2023-10-03T10:18:51.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.123+0000 7f74b1d9b700 1 -- 172.21.15.3:0/3150575932 learned_addr learned my addr 172.21.15.3:0/3150575932 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:51.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.123+0000 7f74b2d9d700 1 --2- 172.21.15.3:0/3150575932 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74b4138d80 0x7f74b413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:51.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.123+0000 7f74b259c700 1 --2- 172.21.15.3:0/3150575932 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74b40a0a30 0x7f74b40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:51.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.123+0000 7f74b159a700 1 -- 172.21.15.3:0/3150575932 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3106606306 0 0) 0x7f74b40a2a70 con 0x7f74b409d630 2023-10-03T10:18:51.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.123+0000 7f74b159a700 1 -- 172.21.15.3:0/3150575932 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f749c0035b0 con 0x7f74b409d630 2023-10-03T10:18:51.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.123+0000 7f74b2d9d700 1 -- 172.21.15.3:0/3150575932 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74b40a0a30 msgr2=0x7f74b40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:51.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.123+0000 7f74b2d9d700 1 --2- 172.21.15.3:0/3150575932 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74b40a0a30 0x7f74b40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.123+0000 7f74b2d9d700 1 -- 172.21.15.3:0/3150575932 >> v1:172.21.15.3:6789/0 conn(0x7f74b409d630 legacy=0x7f74b40a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:51.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.123+0000 7f74b2d9d700 1 -- 172.21.15.3:0/3150575932 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b409dc50 con 0x7f74b4138d80 2023-10-03T10:18:51.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.124+0000 7f74b2d9d700 1 --2- 172.21.15.3:0/3150575932 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74b4138d80 0x7f74b413b160 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f74ac009dc0 tx=0x7f74ac00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1287fa98708ace3e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:51.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.124+0000 7f74b159a700 1 -- 172.21.15.3:0/3150575932 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac00e070 con 0x7f74b4138d80 2023-10-03T10:18:51.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.124+0000 7f74b159a700 1 -- 172.21.15.3:0/3150575932 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74ac00bb20 con 0x7f74b4138d80 2023-10-03T10:18:51.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.124+0000 7f74b159a700 1 -- 172.21.15.3:0/3150575932 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac012430 con 0x7f74b4138d80 2023-10-03T10:18:51.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.124+0000 7f74b8b63700 1 -- 172.21.15.3:0/3150575932 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74b4138d80 msgr2=0x7f74b413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:51.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.124+0000 7f74b8b63700 1 --2- 172.21.15.3:0/3150575932 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74b4138d80 0x7f74b413b160 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f74ac009dc0 tx=0x7f74ac00b330 comp rx=0 tx=0).stop 2023-10-03T10:18:51.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.125+0000 7f74b8b63700 1 -- 172.21.15.3:0/3150575932 shutdown_connections 2023-10-03T10:18:51.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.125+0000 7f74b8b63700 1 --2- 172.21.15.3:0/3150575932 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74b40a0a30 0x7f74b40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.125+0000 7f74b8b63700 1 --2- 172.21.15.3:0/3150575932 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74b4138d80 0x7f74b413b160 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.125+0000 7f74b8b63700 1 -- 172.21.15.3:0/3150575932 >> 172.21.15.3:0/3150575932 conn(0x7f74b40a3050 msgr2=0x7f74b40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:51.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.125+0000 7f74b8b63700 1 -- 172.21.15.3:0/3150575932 shutdown_connections 2023-10-03T10:18:51.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.125+0000 7f74b8b63700 1 -- 172.21.15.3:0/3150575932 wait complete. 2023-10-03T10:18:51.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.126+0000 7f74b8b63700 1 Processor -- start 2023-10-03T10:18:51.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.126+0000 7f74b8b63700 1 -- start start 2023-10-03T10:18:51.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.126+0000 7f74b8b63700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74b409d630 0x7f74b412c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:51.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.126+0000 7f74b8b63700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74b40a0a30 0x7f74b412c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:51.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.126+0000 7f74b8b63700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74b4138d80 0x7f74b412ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:51.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.126+0000 7f74b8b63700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b412f4a0 con 0x7f74b409d630 2023-10-03T10:18:51.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.126+0000 7f74b8b63700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b40a4a90 con 0x7f74b40a0a30 2023-10-03T10:18:51.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.126+0000 7f74b8b63700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74b412eb50 con 0x7f74b4138d80 2023-10-03T10:18:51.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.127+0000 7f74b1d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74b40a0a30 0x7f74b412c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:51.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.127+0000 7f74b259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74b409d630 0x7f74b412c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:51.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.127+0000 7f74b259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74b409d630 0x7f74b412c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52738/0 (socket says 172.21.15.3:52738) 2023-10-03T10:18:51.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.127+0000 7f74b259c700 1 -- 172.21.15.3:0/1428388370 learned_addr learned my addr 172.21.15.3:0/1428388370 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:51.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.127+0000 7f74b2d9d700 1 --2- 172.21.15.3:0/1428388370 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74b4138d80 0x7f74b412ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:51.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.127+0000 7f74b259c700 1 -- 172.21.15.3:0/1428388370 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74b4138d80 msgr2=0x7f74b412ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:51.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.127+0000 7f74b259c700 1 --2- 172.21.15.3:0/1428388370 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74b4138d80 0x7f74b412ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.127+0000 7f74b259c700 1 -- 172.21.15.3:0/1428388370 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74b40a0a30 msgr2=0x7f74b412c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:51.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.127+0000 7f74b259c700 1 --2- 172.21.15.3:0/1428388370 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74b40a0a30 0x7f74b412c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.127+0000 7f74b259c700 1 -- 172.21.15.3:0/1428388370 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ac009a70 con 0x7f74b409d630 2023-10-03T10:18:51.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.127+0000 7f74b1d9b700 1 --2- 172.21.15.3:0/1428388370 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74b40a0a30 0x7f74b412c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:51.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.127+0000 7f74b259c700 1 --2- 172.21.15.3:0/1428388370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74b409d630 0x7f74b412c450 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f74a400e9b0 tx=0x7f74a400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:51.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.127+0000 7f74a2ffd700 1 -- 172.21.15.3:0/1428388370 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a4009710 con 0x7f74b409d630 2023-10-03T10:18:51.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.128+0000 7f74a2ffd700 1 -- 172.21.15.3:0/1428388370 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a400c850 con 0x7f74b409d630 2023-10-03T10:18:51.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.128+0000 7f74a2ffd700 1 -- 172.21.15.3:0/1428388370 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a400cbc0 con 0x7f74b409d630 2023-10-03T10:18:51.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.128+0000 7f74b8b63700 1 -- 172.21.15.3:0/1428388370 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b412ff80 con 0x7f74b409d630 2023-10-03T10:18:51.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.128+0000 7f74b8b63700 1 -- 172.21.15.3:0/1428388370 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b412dfb0 con 0x7f74b409d630 2023-10-03T10:18:51.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.129+0000 7f74b8b63700 1 -- 172.21.15.3:0/1428388370 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74b40747f0 con 0x7f74b409d630 2023-10-03T10:18:51.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.129+0000 7f74a2ffd700 1 -- 172.21.15.3:0/1428388370 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f74a400cd20 con 0x7f74b409d630 2023-10-03T10:18:51.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.129+0000 7f74a2ffd700 1 -- 172.21.15.3:0/1428388370 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f74a403f7a0 con 0x7f74b409d630 2023-10-03T10:18:51.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.139+0000 7f74a2ffd700 1 -- 172.21.15.3:0/1428388370 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f74a4013470 con 0x7f74b409d630 2023-10-03T10:18:51.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.247+0000 7fc88d25c700 1 -- 172.21.15.3:0/1470396168 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc88809e730 con 0x7fc88812cf60 2023-10-03T10:18:51.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.247+0000 7fc86f7fe700 1 -- 172.21.15.3:0/1470396168 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc88809e730 con 0x7fc88812cf60 2023-10-03T10:18:51.247 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.250+0000 7fc88d25c700 1 -- 172.21.15.3:0/1470396168 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc88812cf60 msgr2=0x7fc8881c4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.250+0000 7fc88d25c700 1 --2- 172.21.15.3:0/1470396168 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc88812cf60 0x7fc8881c4610 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc8780090b0 tx=0x7fc878004750 comp rx=0 tx=0).stop 2023-10-03T10:18:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.250+0000 7fc88d25c700 1 -- 172.21.15.3:0/1470396168 shutdown_connections 2023-10-03T10:18:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.250+0000 7fc88d25c700 1 --2- 172.21.15.3:0/1470396168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc888130430 0x7fc88809af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.250+0000 7fc88d25c700 1 --2- 172.21.15.3:0/1470396168 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc88812cf60 0x7fc8881c4610 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.250+0000 7fc88d25c700 1 --2- 172.21.15.3:0/1470396168 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc88812f8b0 0x7fc8881c4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.250+0000 7fc88d25c700 1 -- 172.21.15.3:0/1470396168 >> 172.21.15.3:0/1470396168 conn(0x7fc8880a2aa0 msgr2=0x7fc88806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.250+0000 7fc88d25c700 1 -- 172.21.15.3:0/1470396168 shutdown_connections 2023-10-03T10:18:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.250+0000 7fc88d25c700 1 -- 172.21.15.3:0/1470396168 wait complete. 2023-10-03T10:18:51.264 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:51.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.294+0000 7f77fabc2700 1 -- 172.21.15.3:0/323814887 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f77f409c710 con 0x7f77f409e000 2023-10-03T10:18:51.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.294+0000 7f77f17fa700 1 -- 172.21.15.3:0/323814887 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f77e001b440 con 0x7f77f409e000 2023-10-03T10:18:51.295 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:51.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.296+0000 7f77fabc2700 1 -- 172.21.15.3:0/323814887 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77f409e000 msgr2=0x7f77f41a6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:51.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.296+0000 7f77fabc2700 1 --2- 172.21.15.3:0/323814887 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77f409e000 0x7f77f41a6530 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f77f409ac00 tx=0x7f77e0004750 comp rx=0 tx=0).stop 2023-10-03T10:18:51.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.297+0000 7f77fabc2700 1 -- 172.21.15.3:0/323814887 shutdown_connections 2023-10-03T10:18:51.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.297+0000 7f77fabc2700 1 --2- 172.21.15.3:0/323814887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77f409e410 0x7f77f41a6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.297+0000 7f77fabc2700 1 --2- 172.21.15.3:0/323814887 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f77f409d6f0 0x7f77f41a5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.297+0000 7f77fabc2700 1 --2- 172.21.15.3:0/323814887 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f77f409e000 0x7f77f41a6530 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.297+0000 7f77fabc2700 1 -- 172.21.15.3:0/323814887 >> 172.21.15.3:0/323814887 conn(0x7f77f4066f50 msgr2=0x7f77f41bae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:51.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.297+0000 7f77fabc2700 1 -- 172.21.15.3:0/323814887 shutdown_connections 2023-10-03T10:18:51.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.297+0000 7f77fabc2700 1 -- 172.21.15.3:0/323814887 wait complete. 2023-10-03T10:18:51.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.304+0000 7fa84782e700 1 -- 172.21.15.3:0/1788680500 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa8400747f0 con 0x7fa84009e850 2023-10-03T10:18:51.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.304+0000 7fa835ffb700 1 -- 172.21.15.3:0/1788680500 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa83c01be00 con 0x7fa84009e850 2023-10-03T10:18:51.305 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:51.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.307+0000 7fa84782e700 1 -- 172.21.15.3:0/1788680500 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa84009e850 msgr2=0x7fa840120600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:51.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.307+0000 7fa84782e700 1 --2- 172.21.15.3:0/1788680500 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa84009e850 0x7fa840120600 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa83c000c00 tx=0x7fa83c004750 comp rx=0 tx=0).stop 2023-10-03T10:18:51.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.307+0000 7fa84782e700 1 -- 172.21.15.3:0/1788680500 shutdown_connections 2023-10-03T10:18:51.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.307+0000 7fa84782e700 1 --2- 172.21.15.3:0/1788680500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa84009f5b0 0x7fa840121080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.307+0000 7fa84782e700 1 --2- 172.21.15.3:0/1788680500 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa84009e850 0x7fa840120600 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.307+0000 7fa84782e700 1 --2- 172.21.15.3:0/1788680500 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa84009f160 0x7fa840120b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.307+0000 7fa84782e700 1 -- 172.21.15.3:0/1788680500 >> 172.21.15.3:0/1788680500 conn(0x7fa8400a3f70 msgr2=0x7fa8401bb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:51.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.307+0000 7fa84782e700 1 -- 172.21.15.3:0/1788680500 shutdown_connections 2023-10-03T10:18:51.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.307+0000 7fa84782e700 1 -- 172.21.15.3:0/1788680500 wait complete. 2023-10-03T10:18:51.314 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:51.322 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:51.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.350+0000 7f74b8b63700 1 -- 172.21.15.3:0/1428388370 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f74b412dfb0 con 0x7f74b409d630 2023-10-03T10:18:51.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.350+0000 7f74a2ffd700 1 -- 172.21.15.3:0/1428388370 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f74b412dfb0 con 0x7f74b409d630 2023-10-03T10:18:51.350 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:51.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.352+0000 7f74b8b63700 1 -- 172.21.15.3:0/1428388370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74b409d630 msgr2=0x7f74b412c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:51.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.352+0000 7f74b8b63700 1 --2- 172.21.15.3:0/1428388370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74b409d630 0x7f74b412c450 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f74a400e9b0 tx=0x7f74a400ecc0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.352+0000 7f74b8b63700 1 -- 172.21.15.3:0/1428388370 shutdown_connections 2023-10-03T10:18:51.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.352+0000 7f74b8b63700 1 --2- 172.21.15.3:0/1428388370 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f74b4138d80 0x7f74b412ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.352+0000 7f74b8b63700 1 --2- 172.21.15.3:0/1428388370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74b409d630 0x7f74b412c450 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.352+0000 7f74b8b63700 1 --2- 172.21.15.3:0/1428388370 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f74b40a0a30 0x7f74b412c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:51.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.352+0000 7f74b8b63700 1 -- 172.21.15.3:0/1428388370 >> 172.21.15.3:0/1428388370 conn(0x7f74b40a3050 msgr2=0x7f74b412acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:51.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.352+0000 7f74b8b63700 1 -- 172.21.15.3:0/1428388370 shutdown_connections 2023-10-03T10:18:51.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:51.352+0000 7f74b8b63700 1 -- 172.21.15.3:0/1428388370 wait complete. 2023-10-03T10:18:51.366 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:52.265 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:52.315 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:52.323 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:52.367 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:52.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.419+0000 7f189333c700 1 Processor -- start 2023-10-03T10:18:52.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.420+0000 7f189333c700 1 -- start start 2023-10-03T10:18:52.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.420+0000 7f189333c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f188c0a0a30 0x7f188c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:52.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.420+0000 7f189333c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f188c09d630 0x7f188c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:52.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.420+0000 7f189333c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f188c0a2a70 con 0x7f188c133930 2023-10-03T10:18:52.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.420+0000 7f189333c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f188c0a01d0 con 0x7f188c0a0a30 2023-10-03T10:18:52.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.420+0000 7f189333c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f188c09e2b0 con 0x7f188c09d630 2023-10-03T10:18:52.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.421+0000 7f18910d8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f188c0a0a30 0x7f188c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:52.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.421+0000 7f18910d8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f188c0a0a30 0x7f188c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44782/0 (socket says 172.21.15.3:44782) 2023-10-03T10:18:52.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.421+0000 7f18910d8700 1 -- 172.21.15.3:0/2741532042 learned_addr learned my addr 172.21.15.3:0/2741532042 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:52.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.421+0000 7f18908d7700 1 --2- 172.21.15.3:0/2741532042 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f188c09d630 0x7f188c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:52.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.421+0000 7f18908d7700 1 -- 172.21.15.3:0/2741532042 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f188c0a0a30 msgr2=0x7f188c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:52.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.421+0000 7f18908d7700 1 --2- 172.21.15.3:0/2741532042 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f188c0a0a30 0x7f188c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.421+0000 7f18908d7700 1 -- 172.21.15.3:0/2741532042 >> v1:172.21.15.3:6789/0 conn(0x7f188c133930 legacy=0x7f188c133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:52.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.421+0000 7f18908d7700 1 -- 172.21.15.3:0/2741532042 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188c09dc50 con 0x7f188c09d630 2023-10-03T10:18:52.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.421+0000 7f18908d7700 1 --2- 172.21.15.3:0/2741532042 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f188c09d630 0x7f188c0a1340 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f187c009870 tx=0x7f187c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa750934a98d42b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:52.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.422+0000 7f1883fff700 1 -- 172.21.15.3:0/2741532042 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c011830 con 0x7f188c09d630 2023-10-03T10:18:52.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.422+0000 7f1883fff700 1 -- 172.21.15.3:0/2741532042 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187c011990 con 0x7f188c09d630 2023-10-03T10:18:52.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.422+0000 7f1883fff700 1 -- 172.21.15.3:0/2741532042 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c011d20 con 0x7f188c09d630 2023-10-03T10:18:52.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.422+0000 7f189333c700 1 -- 172.21.15.3:0/2741532042 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f188c09d630 msgr2=0x7f188c0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:52.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.423+0000 7f189333c700 1 --2- 172.21.15.3:0/2741532042 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f188c09d630 0x7f188c0a1340 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f187c009870 tx=0x7f187c009b80 comp rx=0 tx=0).stop 2023-10-03T10:18:52.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.423+0000 7f189333c700 1 -- 172.21.15.3:0/2741532042 shutdown_connections 2023-10-03T10:18:52.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.423+0000 7f189333c700 1 --2- 172.21.15.3:0/2741532042 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f188c0a0a30 0x7f188c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.423+0000 7f189333c700 1 --2- 172.21.15.3:0/2741532042 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f188c09d630 0x7f188c0a1340 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.423+0000 7f189333c700 1 -- 172.21.15.3:0/2741532042 >> 172.21.15.3:0/2741532042 conn(0x7f188c0a3050 msgr2=0x7f188c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:52.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.423+0000 7f189333c700 1 -- 172.21.15.3:0/2741532042 shutdown_connections 2023-10-03T10:18:52.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.423+0000 7f189333c700 1 -- 172.21.15.3:0/2741532042 wait complete. 2023-10-03T10:18:52.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.424+0000 7f189333c700 1 Processor -- start 2023-10-03T10:18:52.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.424+0000 7f189333c700 1 -- start start 2023-10-03T10:18:52.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.424+0000 7f189333c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f188c09d630 0x7f188c11f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:52.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.424+0000 7f189333c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f188c0a0a30 0x7f188c11f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:52.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.424+0000 7f189333c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f188c133930 0x7f188c1c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:52.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.424+0000 7f189333c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c11eee0 con 0x7f188c09d630 2023-10-03T10:18:52.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.424+0000 7f189333c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f188c121f80 con 0x7f188c133930 2023-10-03T10:18:52.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.424+0000 7f189333c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f188c11fe50 con 0x7f188c0a0a30 2023-10-03T10:18:52.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.425+0000 7f18908d7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f188c0a0a30 0x7f188c11f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:52.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.425+0000 7f18908d7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f188c0a0a30 0x7f188c11f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:37940/0 (socket says 172.21.15.3:37940) 2023-10-03T10:18:52.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.425+0000 7f18908d7700 1 -- 172.21.15.3:0/4039408326 learned_addr learned my addr 172.21.15.3:0/4039408326 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:52.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.425+0000 7f18918d9700 1 --2- 172.21.15.3:0/4039408326 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f188c133930 0x7f188c1c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:52.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.425+0000 7f18910d8700 1 --2- 172.21.15.3:0/4039408326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f188c09d630 0x7f188c11f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:52.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.425+0000 7f18908d7700 1 -- 172.21.15.3:0/4039408326 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f188c133930 msgr2=0x7f188c1c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:52.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.425+0000 7f18908d7700 1 --2- 172.21.15.3:0/4039408326 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f188c133930 0x7f188c1c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.425+0000 7f18908d7700 1 -- 172.21.15.3:0/4039408326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f188c09d630 msgr2=0x7f188c11f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:52.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.425+0000 7f18908d7700 1 --2- 172.21.15.3:0/4039408326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f188c09d630 0x7f188c11f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.425+0000 7f18908d7700 1 -- 172.21.15.3:0/4039408326 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f187c0094a0 con 0x7f188c0a0a30 2023-10-03T10:18:52.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.425+0000 7f18908d7700 1 --2- 172.21.15.3:0/4039408326 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f188c0a0a30 0x7f188c11f910 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f187c009bc0 tx=0x7f187c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:52.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.425+0000 7f1881ffb700 1 -- 172.21.15.3:0/4039408326 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c004240 con 0x7f188c0a0a30 2023-10-03T10:18:52.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.425+0000 7f1881ffb700 1 -- 172.21.15.3:0/4039408326 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187c01f070 con 0x7f188c0a0a30 2023-10-03T10:18:52.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.426+0000 7f1881ffb700 1 -- 172.21.15.3:0/4039408326 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c0045e0 con 0x7f188c0a0a30 2023-10-03T10:18:52.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.426+0000 7f189333c700 1 -- 172.21.15.3:0/4039408326 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f188c122620 con 0x7f188c0a0a30 2023-10-03T10:18:52.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.426+0000 7f189333c700 1 -- 172.21.15.3:0/4039408326 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f188c0a4030 con 0x7f188c0a0a30 2023-10-03T10:18:52.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.427+0000 7f1881ffb700 1 -- 172.21.15.3:0/4039408326 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f187c021030 con 0x7f188c0a0a30 2023-10-03T10:18:52.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.427+0000 7f1881ffb700 1 -- 172.21.15.3:0/4039408326 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f187c0408d0 con 0x7f188c0a0a30 2023-10-03T10:18:52.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.427+0000 7f189333c700 1 -- 172.21.15.3:0/4039408326 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f188c0747f0 con 0x7f188c0a0a30 2023-10-03T10:18:52.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.433+0000 7f1881ffb700 1 -- 172.21.15.3:0/4039408326 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f187c0237e0 con 0x7f188c0a0a30 2023-10-03T10:18:52.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.462+0000 7ff9b71d8700 1 Processor -- start 2023-10-03T10:18:52.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.462+0000 7ff9b71d8700 1 -- start start 2023-10-03T10:18:52.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.462+0000 7ff9b71d8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff9b009ac90 0x7ff9b013cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:52.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.462+0000 7ff9b71d8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9b013d240 0x7ff9b013f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:52.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.462+0000 7ff9b71d8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b0134980 con 0x7ff9b006c0e0 2023-10-03T10:18:52.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.462+0000 7ff9b71d8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b006c4b0 con 0x7ff9b009ac90 2023-10-03T10:18:52.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.462+0000 7ff9b71d8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b009b0e0 con 0x7ff9b013d240 2023-10-03T10:18:52.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.463+0000 7ff9affff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff9b009ac90 0x7ff9b013cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:52.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.463+0000 7ff9affff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff9b009ac90 0x7ff9b013cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44794/0 (socket says 172.21.15.3:44794) 2023-10-03T10:18:52.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.463+0000 7ff9affff700 1 -- 172.21.15.3:0/3683332692 learned_addr learned my addr 172.21.15.3:0/3683332692 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:52.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.463+0000 7ff9b4f74700 1 --1- 172.21.15.3:0/3683332692 >> v1:172.21.15.3:6789/0 conn(0x7ff9b006c0e0 0x7ff9b0134ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47854/0 (socket says 172.21.15.3:47854) 2023-10-03T10:18:52.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.463+0000 7ff9b5775700 1 --2- 172.21.15.3:0/3683332692 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9b013d240 0x7ff9b013f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:52.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.463+0000 7ff9af7fe700 1 -- 172.21.15.3:0/3683332692 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3213331848 0 0) 0x7ff9b0134980 con 0x7ff9b006c0e0 2023-10-03T10:18:52.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.463+0000 7ff9affff700 1 -- 172.21.15.3:0/3683332692 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9b013d240 msgr2=0x7ff9b013f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:52.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.463+0000 7ff9affff700 1 --2- 172.21.15.3:0/3683332692 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9b013d240 0x7ff9b013f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.463+0000 7ff9affff700 1 -- 172.21.15.3:0/3683332692 >> v1:172.21.15.3:6789/0 conn(0x7ff9b006c0e0 legacy=0x7ff9b0134ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:52.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.463+0000 7ff9affff700 1 -- 172.21.15.3:0/3683332692 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b00a0600 con 0x7ff9b009ac90 2023-10-03T10:18:52.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.463+0000 7ff9affff700 1 --2- 172.21.15.3:0/3683332692 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff9b009ac90 0x7ff9b013cd00 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ff99c009920 tx=0x7ff99c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ddcccc67e9dd9c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:52.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.464+0000 7ff9af7fe700 1 -- 172.21.15.3:0/3683332692 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c011830 con 0x7ff9b009ac90 2023-10-03T10:18:52.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.464+0000 7ff9af7fe700 1 -- 172.21.15.3:0/3683332692 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99c011990 con 0x7ff9b009ac90 2023-10-03T10:18:52.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.464+0000 7ff9af7fe700 1 -- 172.21.15.3:0/3683332692 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c011d20 con 0x7ff9b009ac90 2023-10-03T10:18:52.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.464+0000 7ff9b71d8700 1 -- 172.21.15.3:0/3683332692 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff9b009ac90 msgr2=0x7ff9b013cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:52.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.464+0000 7ff9b71d8700 1 --2- 172.21.15.3:0/3683332692 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff9b009ac90 0x7ff9b013cd00 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ff99c009920 tx=0x7ff99c009c30 comp rx=0 tx=0).stop 2023-10-03T10:18:52.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.464+0000 7ff9b71d8700 1 -- 172.21.15.3:0/3683332692 shutdown_connections 2023-10-03T10:18:52.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.464+0000 7ff9b71d8700 1 --2- 172.21.15.3:0/3683332692 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9b013d240 0x7ff9b013f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.464+0000 7ff9b71d8700 1 --2- 172.21.15.3:0/3683332692 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff9b009ac90 0x7ff9b013cd00 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.464+0000 7ff9b71d8700 1 -- 172.21.15.3:0/3683332692 >> 172.21.15.3:0/3683332692 conn(0x7ff9b00a4210 msgr2=0x7ff9b00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:52.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.465+0000 7ff9b71d8700 1 -- 172.21.15.3:0/3683332692 shutdown_connections 2023-10-03T10:18:52.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.465+0000 7ff9b71d8700 1 -- 172.21.15.3:0/3683332692 wait complete. 2023-10-03T10:18:52.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.466+0000 7ff9b71d8700 1 Processor -- start 2023-10-03T10:18:52.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.466+0000 7ff9b71d8700 1 -- start start 2023-10-03T10:18:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.467+0000 7ff9b71d8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9b006c0e0 0x7ff9b0132bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.467+0000 7ff9b71d8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff9b009ac90 0x7ff9b0133110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.467+0000 7ff9b71d8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9b013d240 0x7ff9b0133650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.467+0000 7ff9b71d8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b009dac0 con 0x7ff9b013d240 2023-10-03T10:18:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.467+0000 7ff9b71d8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b0139120 con 0x7ff9b009ac90 2023-10-03T10:18:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.467+0000 7ff9b71d8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b00a03f0 con 0x7ff9b006c0e0 2023-10-03T10:18:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.467+0000 7ff9b4f74700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9b006c0e0 0x7ff9b0132bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.467+0000 7ff9affff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff9b009ac90 0x7ff9b0133110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.467+0000 7ff9b4f74700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9b006c0e0 0x7ff9b0132bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:37948/0 (socket says 172.21.15.3:37948) 2023-10-03T10:18:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.467+0000 7ff9affff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff9b009ac90 0x7ff9b0133110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44808/0 (socket says 172.21.15.3:44808) 2023-10-03T10:18:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.467+0000 7ff9b4f74700 1 -- 172.21.15.3:0/2482661298 learned_addr learned my addr 172.21.15.3:0/2482661298 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:52.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.467+0000 7ff9b5775700 1 --2- 172.21.15.3:0/2482661298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9b013d240 0x7ff9b0133650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:52.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.468+0000 7ff9b5775700 1 -- 172.21.15.3:0/2482661298 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9b006c0e0 msgr2=0x7ff9b0132bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:52.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.468+0000 7ff9b5775700 1 --2- 172.21.15.3:0/2482661298 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9b006c0e0 0x7ff9b0132bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.468+0000 7ff9b5775700 1 -- 172.21.15.3:0/2482661298 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff9b009ac90 msgr2=0x7ff9b0133110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:52.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.468+0000 7ff9b5775700 1 --2- 172.21.15.3:0/2482661298 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff9b009ac90 0x7ff9b0133110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.468+0000 7ff9b5775700 1 -- 172.21.15.3:0/2482661298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99c009550 con 0x7ff9b013d240 2023-10-03T10:18:52.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.468+0000 7ff9b4f74700 1 --2- 172.21.15.3:0/2482661298 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9b006c0e0 0x7ff9b0132bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:52.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.468+0000 7ff9b5775700 1 --2- 172.21.15.3:0/2482661298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9b013d240 0x7ff9b0133650 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800ef90 tx=0x7ff9a800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:52.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.468+0000 7ff9ad7fa700 1 -- 172.21.15.3:0/2482661298 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a800cc90 con 0x7ff9b013d240 2023-10-03T10:18:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.468+0000 7ff9ad7fa700 1 -- 172.21.15.3:0/2482661298 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a800f040 con 0x7ff9b013d240 2023-10-03T10:18:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.468+0000 7ff9b71d8700 1 -- 172.21.15.3:0/2482661298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b0134000 con 0x7ff9b013d240 2023-10-03T10:18:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.468+0000 7ff9ad7fa700 1 -- 172.21.15.3:0/2482661298 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8009e60 con 0x7ff9b013d240 2023-10-03T10:18:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.468+0000 7ff9b71d8700 1 -- 172.21.15.3:0/2482661298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b009ccf0 con 0x7ff9b013d240 2023-10-03T10:18:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.469+0000 7ff9b71d8700 1 -- 172.21.15.3:0/2482661298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b009e3c0 con 0x7ff9b013d240 2023-10-03T10:18:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.469+0000 7ff9ad7fa700 1 -- 172.21.15.3:0/2482661298 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff9a8014480 con 0x7ff9b013d240 2023-10-03T10:18:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.470+0000 7ff9ad7fa700 1 -- 172.21.15.3:0/2482661298 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff9a803fdc0 con 0x7ff9b013d240 2023-10-03T10:18:52.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.479+0000 7ff9ad7fa700 1 -- 172.21.15.3:0/2482661298 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff9a801a8b0 con 0x7ff9b013d240 2023-10-03T10:18:52.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.549+0000 7faf088bc700 1 Processor -- start 2023-10-03T10:18:52.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.549+0000 7faf088bc700 1 -- start start 2023-10-03T10:18:52.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.550+0000 7faf088bc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7faf041338c0 0x7faf04133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:52.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.550+0000 7faf088bc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faf0409e560 0x7faf0413b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:52.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.550+0000 7faf088bc700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf0406f720 con 0x7faf041341d0 2023-10-03T10:18:52.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.550+0000 7faf088bc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf04070c80 con 0x7faf041338c0 2023-10-03T10:18:52.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.550+0000 7faf088bc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf0406fce0 con 0x7faf0409e560 2023-10-03T10:18:52.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.550+0000 7faf01d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7faf041341d0 0x7faf0409de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47866/0 (socket says 172.21.15.3:47866) 2023-10-03T10:18:52.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.550+0000 7faf01d9b700 1 -- 172.21.15.3:0/4262230817 learned_addr learned my addr 172.21.15.3:0/4262230817 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:52.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.550+0000 7faf02d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faf0409e560 0x7faf0413b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:52.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.550+0000 7faf0259c700 1 --2- 172.21.15.3:0/4262230817 >> v2:172.21.15.3:3300/0 conn(0x7faf041338c0 0x7faf04133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:52.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.551+0000 7faf02d9d700 1 -- 172.21.15.3:0/4262230817 >> v2:172.21.15.3:3300/0 conn(0x7faf041338c0 msgr2=0x7faf04133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:52.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.551+0000 7faf02d9d700 1 --2- 172.21.15.3:0/4262230817 >> v2:172.21.15.3:3300/0 conn(0x7faf041338c0 0x7faf04133c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.551+0000 7faf02d9d700 1 -- 172.21.15.3:0/4262230817 >> v1:172.21.15.3:6789/0 conn(0x7faf041341d0 legacy=0x7faf0409de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:52.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.551+0000 7faf02d9d700 1 -- 172.21.15.3:0/4262230817 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf041336a0 con 0x7faf0409e560 2023-10-03T10:18:52.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.551+0000 7faf0259c700 1 --2- 172.21.15.3:0/4262230817 >> v2:172.21.15.3:3300/0 conn(0x7faf041338c0 0x7faf04133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:52.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.551+0000 7faf02d9d700 1 --2- 172.21.15.3:0/4262230817 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faf0409e560 0x7faf0413b860 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7faefc009d90 tx=0x7faefc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e88bbe1cbab564d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:52.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.551+0000 7faf0159a700 1 -- 172.21.15.3:0/4262230817 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc00e070 con 0x7faf0409e560 2023-10-03T10:18:52.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.551+0000 7faf0159a700 1 -- 172.21.15.3:0/4262230817 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faefc00bb20 con 0x7faf0409e560 2023-10-03T10:18:52.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.551+0000 7faf0159a700 1 -- 172.21.15.3:0/4262230817 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc012430 con 0x7faf0409e560 2023-10-03T10:18:52.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.552+0000 7faf088bc700 1 -- 172.21.15.3:0/4262230817 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faf0409e560 msgr2=0x7faf0413b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:52.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.552+0000 7faf088bc700 1 --2- 172.21.15.3:0/4262230817 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faf0409e560 0x7faf0413b860 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7faefc009d90 tx=0x7faefc00b330 comp rx=0 tx=0).stop 2023-10-03T10:18:52.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.552+0000 7faf088bc700 1 -- 172.21.15.3:0/4262230817 shutdown_connections 2023-10-03T10:18:52.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.552+0000 7faf088bc700 1 --2- 172.21.15.3:0/4262230817 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faf0409e560 0x7faf0413b860 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.552+0000 7faf088bc700 1 --2- 172.21.15.3:0/4262230817 >> v2:172.21.15.3:3300/0 conn(0x7faf041338c0 0x7faf04133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.552+0000 7faf088bc700 1 -- 172.21.15.3:0/4262230817 >> 172.21.15.3:0/4262230817 conn(0x7faf040a2aa0 msgr2=0x7faf040a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:52.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.552+0000 7faf088bc700 1 -- 172.21.15.3:0/4262230817 shutdown_connections 2023-10-03T10:18:52.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.552+0000 7faf088bc700 1 -- 172.21.15.3:0/4262230817 wait complete. 2023-10-03T10:18:52.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.554+0000 7faf088bc700 1 Processor -- start 2023-10-03T10:18:52.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.554+0000 7faf088bc700 1 -- start start 2023-10-03T10:18:52.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.554+0000 7faf088bc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7faf0409e560 0x7faf0411c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:52.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.554+0000 7faf088bc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf041338c0 0x7faf0411ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:52.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.554+0000 7faf088bc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faf041341d0 0x7faf0411d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:52.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.555+0000 7faf088bc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0412ddb0 con 0x7faf041338c0 2023-10-03T10:18:52.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.555+0000 7faf088bc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf04120110 con 0x7faf041341d0 2023-10-03T10:18:52.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.555+0000 7faf088bc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf0406f960 con 0x7faf0409e560 2023-10-03T10:18:52.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.555+0000 7faf0259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7faf0409e560 0x7faf0411c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:52.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.555+0000 7faf02d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faf041341d0 0x7faf0411d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:52.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.555+0000 7faf0259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7faf0409e560 0x7faf0411c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:37960/0 (socket says 172.21.15.3:37960) 2023-10-03T10:18:52.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.555+0000 7faf02d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faf041341d0 0x7faf0411d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44826/0 (socket says 172.21.15.3:44826) 2023-10-03T10:18:52.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.555+0000 7faf0259c700 1 -- 172.21.15.3:0/1839818659 learned_addr learned my addr 172.21.15.3:0/1839818659 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:52.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.555+0000 7faf01d9b700 1 --2- 172.21.15.3:0/1839818659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf041338c0 0x7faf0411ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.555+0000 7faf0259c700 1 -- 172.21.15.3:0/1839818659 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faf041341d0 msgr2=0x7faf0411d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.555+0000 7faf0259c700 1 --2- 172.21.15.3:0/1839818659 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faf041341d0 0x7faf0411d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.555+0000 7faf0259c700 1 -- 172.21.15.3:0/1839818659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf041338c0 msgr2=0x7faf0411ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.555+0000 7faf0259c700 1 --2- 172.21.15.3:0/1839818659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf041338c0 0x7faf0411ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.555+0000 7faf0259c700 1 -- 172.21.15.3:0/1839818659 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef40094a0 con 0x7faf0409e560 2023-10-03T10:18:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.555+0000 7faf01d9b700 1 --2- 172.21.15.3:0/1839818659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf041338c0 0x7faf0411ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.555+0000 7faf0259c700 1 --2- 172.21.15.3:0/1839818659 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7faf0409e560 0x7faf0411c8d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7faef400ebe0 tx=0x7faef400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.556+0000 7faef2ffd700 1 -- 172.21.15.3:0/1839818659 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef400c850 con 0x7faf0409e560 2023-10-03T10:18:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.556+0000 7faef2ffd700 1 -- 172.21.15.3:0/1839818659 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef400c9b0 con 0x7faf0409e560 2023-10-03T10:18:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.556+0000 7faf088bc700 1 -- 172.21.15.3:0/1839818659 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faefc009a70 con 0x7faf0409e560 2023-10-03T10:18:52.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.556+0000 7faef2ffd700 1 -- 172.21.15.3:0/1839818659 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef400cd40 con 0x7faf0409e560 2023-10-03T10:18:52.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.556+0000 7faf088bc700 1 -- 172.21.15.3:0/1839818659 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf04120b80 con 0x7faf0409e560 2023-10-03T10:18:52.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.557+0000 7faef2ffd700 1 -- 172.21.15.3:0/1839818659 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faef4013470 con 0x7faf0409e560 2023-10-03T10:18:52.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.557+0000 7faf088bc700 1 -- 172.21.15.3:0/1839818659 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf04074ea0 con 0x7faf0409e560 2023-10-03T10:18:52.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.557+0000 7faef2ffd700 1 -- 172.21.15.3:0/1839818659 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7faef403fb20 con 0x7faf0409e560 2023-10-03T10:18:52.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.560+0000 7f09a7105700 1 Processor -- start 2023-10-03T10:18:52.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.560+0000 7f09a7105700 1 -- start start 2023-10-03T10:18:52.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.561+0000 7f09a7105700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09a009e1d0 0x7f09a0138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:52.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.561+0000 7f09a7105700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f09a0139170 0x7f09a013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:52.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.561+0000 7f09a7105700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a009b120 con 0x7f09a009d6f0 2023-10-03T10:18:52.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.561+0000 7f09a7105700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09a006b460 con 0x7f09a0139170 2023-10-03T10:18:52.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.561+0000 7f09a7105700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a009a7e0 con 0x7f09a009e1d0 2023-10-03T10:18:52.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.561+0000 7f099ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09a009e1d0 0x7f09a0138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:52.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.561+0000 7f09a4ea1700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f09a009d6f0 0x7f09a009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47882/0 (socket says 172.21.15.3:47882) 2023-10-03T10:18:52.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.561+0000 7f09a4ea1700 1 -- 172.21.15.3:0/1582898757 learned_addr learned my addr 172.21.15.3:0/1582898757 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:52.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.561+0000 7f09a56a2700 1 --2- 172.21.15.3:0/1582898757 >> v2:172.21.15.3:3300/0 conn(0x7f09a0139170 0x7f09a013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:52.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.562+0000 7f09a56a2700 1 -- 172.21.15.3:0/1582898757 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09a009e1d0 msgr2=0x7f09a0138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:52.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.562+0000 7f099f7fe700 1 -- 172.21.15.3:0/1582898757 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1393967096 0 0) 0x7f09a009b120 con 0x7f09a009d6f0 2023-10-03T10:18:52.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.562+0000 7f09a56a2700 1 --2- 172.21.15.3:0/1582898757 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09a009e1d0 0x7f09a0138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.562+0000 7f09a56a2700 1 -- 172.21.15.3:0/1582898757 >> v1:172.21.15.3:6789/0 conn(0x7f09a009d6f0 legacy=0x7f09a009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:52.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.562+0000 7f09a56a2700 1 -- 172.21.15.3:0/1582898757 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a009a500 con 0x7f09a0139170 2023-10-03T10:18:52.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.562+0000 7f099ffff700 1 --2- 172.21.15.3:0/1582898757 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09a009e1d0 0x7f09a0138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:52.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.562+0000 7f09a56a2700 1 --2- 172.21.15.3:0/1582898757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a0139170 0x7f09a013b550 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f0998009d90 tx=0x7f099800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e9c47d551f0ad9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:52.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.562+0000 7f099f7fe700 1 -- 172.21.15.3:0/1582898757 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099800e070 con 0x7f09a0139170 2023-10-03T10:18:52.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.562+0000 7f099f7fe700 1 -- 172.21.15.3:0/1582898757 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099800bb20 con 0x7f09a0139170 2023-10-03T10:18:52.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.562+0000 7f099f7fe700 1 -- 172.21.15.3:0/1582898757 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0998012430 con 0x7f09a0139170 2023-10-03T10:18:52.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.563+0000 7f09a7105700 1 -- 172.21.15.3:0/1582898757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a0139170 msgr2=0x7f09a013b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:52.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.563+0000 7f09a7105700 1 --2- 172.21.15.3:0/1582898757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a0139170 0x7f09a013b550 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f0998009d90 tx=0x7f099800b330 comp rx=0 tx=0).stop 2023-10-03T10:18:52.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.563+0000 7f09a7105700 1 -- 172.21.15.3:0/1582898757 shutdown_connections 2023-10-03T10:18:52.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.563+0000 7f09a7105700 1 --2- 172.21.15.3:0/1582898757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a0139170 0x7f09a013b550 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.563+0000 7f09a7105700 1 --2- 172.21.15.3:0/1582898757 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09a009e1d0 0x7f09a0138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.563+0000 7f09a7105700 1 -- 172.21.15.3:0/1582898757 >> 172.21.15.3:0/1582898757 conn(0x7f09a0066f50 msgr2=0x7f09a009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:52.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.563+0000 7f09a7105700 1 -- 172.21.15.3:0/1582898757 shutdown_connections 2023-10-03T10:18:52.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.563+0000 7f09a7105700 1 -- 172.21.15.3:0/1582898757 wait complete. 2023-10-03T10:18:52.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.564+0000 7f09a7105700 1 Processor -- start 2023-10-03T10:18:52.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.564+0000 7f09a7105700 1 -- start start 2023-10-03T10:18:52.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.565+0000 7f09a7105700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09a009d6f0 0x7f09a01218a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:52.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.565+0000 7f09a7105700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a009e1d0 0x7f09a0121de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:52.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.565+0000 7f09a7105700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f09a0139170 0x7f09a0122320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:52.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.565+0000 7f09a7105700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a0124f00 con 0x7f09a009e1d0 2023-10-03T10:18:52.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.565+0000 7f09a7105700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a01250e0 con 0x7f09a009d6f0 2023-10-03T10:18:52.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.565+0000 7f09a7105700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f09a009c710 con 0x7f09a0139170 2023-10-03T10:18:52.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.565+0000 7f09a4ea1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09a009d6f0 0x7f09a01218a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:52.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.565+0000 7f099ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a009e1d0 0x7f09a0121de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:52.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.565+0000 7f09a56a2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f09a0139170 0x7f09a0122320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:52.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.565+0000 7f099ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a009e1d0 0x7f09a0121de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52788/0 (socket says 172.21.15.3:52788) 2023-10-03T10:18:52.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.565+0000 7f09a4ea1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09a009d6f0 0x7f09a01218a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:44850/0 (socket says 172.21.15.3:44850) 2023-10-03T10:18:52.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.565+0000 7f099ffff700 1 -- 172.21.15.3:0/3664955932 learned_addr learned my addr 172.21.15.3:0/3664955932 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:52.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.565+0000 7f09a56a2700 1 -- 172.21.15.3:0/3664955932 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09a009d6f0 msgr2=0x7f09a01218a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:52.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.565+0000 7f09a56a2700 1 --2- 172.21.15.3:0/3664955932 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09a009d6f0 0x7f09a01218a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.565+0000 7f09a56a2700 1 -- 172.21.15.3:0/3664955932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a009e1d0 msgr2=0x7f09a0121de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:52.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.565+0000 7f09a56a2700 1 --2- 172.21.15.3:0/3664955932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a009e1d0 0x7f09a0121de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.565+0000 7f09a56a2700 1 -- 172.21.15.3:0/3664955932 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098c0094a0 con 0x7f09a0139170 2023-10-03T10:18:52.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.565+0000 7f099ffff700 1 --2- 172.21.15.3:0/3664955932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a009e1d0 0x7f09a0121de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:52.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.566+0000 7f09a56a2700 1 --2- 172.21.15.3:0/3664955932 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f09a0139170 0x7f09a0122320 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f0998016000 tx=0x7f0998004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:52.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.566+0000 7f099d7fa700 1 -- 172.21.15.3:0/3664955932 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0998012430 con 0x7f09a0139170 2023-10-03T10:18:52.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.566+0000 7f099d7fa700 1 -- 172.21.15.3:0/3664955932 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0998020070 con 0x7f09a0139170 2023-10-03T10:18:52.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.566+0000 7f09a7105700 1 -- 172.21.15.3:0/3664955932 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0998009a70 con 0x7f09a0139170 2023-10-03T10:18:52.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.566+0000 7f099d7fa700 1 -- 172.21.15.3:0/3664955932 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099801ba30 con 0x7f09a0139170 2023-10-03T10:18:52.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.567+0000 7f09a7105700 1 -- 172.21.15.3:0/3664955932 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a0072150 con 0x7f09a0139170 2023-10-03T10:18:52.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.567+0000 7faef2ffd700 1 -- 172.21.15.3:0/1839818659 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faef4013720 con 0x7faf0409e560 2023-10-03T10:18:52.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.568+0000 7f09a7105700 1 -- 172.21.15.3:0/3664955932 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a009c920 con 0x7f09a0139170 2023-10-03T10:18:52.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.568+0000 7f099d7fa700 1 -- 172.21.15.3:0/3664955932 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f099801bb90 con 0x7f09a0139170 2023-10-03T10:18:52.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.569+0000 7f099d7fa700 1 -- 172.21.15.3:0/3664955932 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0998022070 con 0x7f09a0139170 2023-10-03T10:18:52.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.578+0000 7f099d7fa700 1 -- 172.21.15.3:0/3664955932 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f099801be40 con 0x7f09a0139170 2023-10-03T10:18:52.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.689+0000 7ff9b71d8700 1 -- 172.21.15.3:0/2482661298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff9b009dc30 con 0x7ff9b013d240 2023-10-03T10:18:52.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.690+0000 7ff9ad7fa700 1 -- 172.21.15.3:0/2482661298 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff9a801a2c0 con 0x7ff9b013d240 2023-10-03T10:18:52.690 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:52.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.692+0000 7ff9b71d8700 1 -- 172.21.15.3:0/2482661298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9b013d240 msgr2=0x7ff9b0133650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:52.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.692+0000 7ff9b71d8700 1 --2- 172.21.15.3:0/2482661298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9b013d240 0x7ff9b0133650 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800ef90 tx=0x7ff9a800c3a0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.692+0000 7ff9b71d8700 1 -- 172.21.15.3:0/2482661298 shutdown_connections 2023-10-03T10:18:52.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.692+0000 7ff9b71d8700 1 --2- 172.21.15.3:0/2482661298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9b013d240 0x7ff9b0133650 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.692+0000 7ff9b71d8700 1 --2- 172.21.15.3:0/2482661298 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff9b006c0e0 0x7ff9b0132bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.692+0000 7ff9b71d8700 1 --2- 172.21.15.3:0/2482661298 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff9b009ac90 0x7ff9b0133110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.692+0000 7ff9b71d8700 1 -- 172.21.15.3:0/2482661298 >> 172.21.15.3:0/2482661298 conn(0x7ff9b00a4210 msgr2=0x7ff9b00a0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:52.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.692+0000 7ff9b71d8700 1 -- 172.21.15.3:0/2482661298 shutdown_connections 2023-10-03T10:18:52.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.692+0000 7ff9b71d8700 1 -- 172.21.15.3:0/2482661298 wait complete. 2023-10-03T10:18:52.707 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:52.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.711+0000 7f189333c700 1 -- 172.21.15.3:0/4039408326 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f188c1334b0 con 0x7f188c0a0a30 2023-10-03T10:18:52.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.711+0000 7f1881ffb700 1 -- 172.21.15.3:0/4039408326 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f187c0281c0 con 0x7f188c0a0a30 2023-10-03T10:18:52.712 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:52.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.714+0000 7f189333c700 1 -- 172.21.15.3:0/4039408326 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f188c0a0a30 msgr2=0x7f188c11f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:52.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.714+0000 7f189333c700 1 --2- 172.21.15.3:0/4039408326 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f188c0a0a30 0x7f188c11f910 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f187c009bc0 tx=0x7f187c004750 comp rx=0 tx=0).stop 2023-10-03T10:18:52.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.714+0000 7f189333c700 1 -- 172.21.15.3:0/4039408326 shutdown_connections 2023-10-03T10:18:52.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.714+0000 7f189333c700 1 --2- 172.21.15.3:0/4039408326 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f188c133930 0x7f188c1c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.714+0000 7f189333c700 1 --2- 172.21.15.3:0/4039408326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f188c09d630 0x7f188c11f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.714+0000 7f189333c700 1 --2- 172.21.15.3:0/4039408326 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f188c0a0a30 0x7f188c11f910 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.714+0000 7f189333c700 1 -- 172.21.15.3:0/4039408326 >> 172.21.15.3:0/4039408326 conn(0x7f188c0a3050 msgr2=0x7f188c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:52.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.714+0000 7f189333c700 1 -- 172.21.15.3:0/4039408326 shutdown_connections 2023-10-03T10:18:52.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.714+0000 7f189333c700 1 -- 172.21.15.3:0/4039408326 wait complete. 2023-10-03T10:18:52.728 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:52.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.778+0000 7faf088bc700 1 -- 172.21.15.3:0/1839818659 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7faf0406b9e0 con 0x7faf0409e560 2023-10-03T10:18:52.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.778+0000 7faef2ffd700 1 -- 172.21.15.3:0/1839818659 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faef4017030 con 0x7faf0409e560 2023-10-03T10:18:52.779 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:52.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.781+0000 7faf088bc700 1 -- 172.21.15.3:0/1839818659 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7faf0409e560 msgr2=0x7faf0411c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:52.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.781+0000 7faf088bc700 1 --2- 172.21.15.3:0/1839818659 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7faf0409e560 0x7faf0411c8d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7faef400ebe0 tx=0x7faef400eef0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.781+0000 7faf088bc700 1 -- 172.21.15.3:0/1839818659 shutdown_connections 2023-10-03T10:18:52.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.781+0000 7faf088bc700 1 --2- 172.21.15.3:0/1839818659 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7faf041341d0 0x7faf0411d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.781+0000 7faf088bc700 1 --2- 172.21.15.3:0/1839818659 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7faf0409e560 0x7faf0411c8d0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.781+0000 7faf088bc700 1 --2- 172.21.15.3:0/1839818659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf041338c0 0x7faf0411ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.781+0000 7faf088bc700 1 -- 172.21.15.3:0/1839818659 >> 172.21.15.3:0/1839818659 conn(0x7faf040a2aa0 msgr2=0x7faf0409f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:52.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.781+0000 7faf088bc700 1 -- 172.21.15.3:0/1839818659 shutdown_connections 2023-10-03T10:18:52.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.781+0000 7faf088bc700 1 -- 172.21.15.3:0/1839818659 wait complete. 2023-10-03T10:18:52.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.784+0000 7f09a7105700 1 -- 172.21.15.3:0/3664955932 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f09a0125220 con 0x7f09a0139170 2023-10-03T10:18:52.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.784+0000 7f099d7fa700 1 -- 172.21.15.3:0/3664955932 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f09a0125220 con 0x7f09a0139170 2023-10-03T10:18:52.784 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:52.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.786+0000 7f09a7105700 1 -- 172.21.15.3:0/3664955932 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f09a0139170 msgr2=0x7f09a0122320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:52.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.786+0000 7f09a7105700 1 --2- 172.21.15.3:0/3664955932 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f09a0139170 0x7f09a0122320 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f0998016000 tx=0x7f0998004ad0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.786+0000 7f09a7105700 1 -- 172.21.15.3:0/3664955932 shutdown_connections 2023-10-03T10:18:52.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.786+0000 7f09a7105700 1 --2- 172.21.15.3:0/3664955932 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f09a0139170 0x7f09a0122320 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.786+0000 7f09a7105700 1 --2- 172.21.15.3:0/3664955932 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f09a009d6f0 0x7f09a01218a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.786+0000 7f09a7105700 1 --2- 172.21.15.3:0/3664955932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a009e1d0 0x7f09a0121de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:52.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.786+0000 7f09a7105700 1 -- 172.21.15.3:0/3664955932 >> 172.21.15.3:0/3664955932 conn(0x7f09a0066f50 msgr2=0x7f09a0070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:52.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.786+0000 7f09a7105700 1 -- 172.21.15.3:0/3664955932 shutdown_connections 2023-10-03T10:18:52.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:52.786+0000 7f09a7105700 1 -- 172.21.15.3:0/3664955932 wait complete. 2023-10-03T10:18:52.796 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:52.800 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:53.708 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:53.729 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:53.797 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:53.801 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:53.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.866+0000 7fc5ac7fc700 1 Processor -- start 2023-10-03T10:18:53.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.866+0000 7fc5ac7fc700 1 -- start start 2023-10-03T10:18:53.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.867+0000 7fc5ac7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5a41338a0 0x7fc5a4133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:53.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.867+0000 7fc5ac7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5a409de50 0x7fc5a409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:53.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.867+0000 7fc5ac7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc5a409e760 0x7fc5a413a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:53.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.867+0000 7fc5ac7fc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a406fef0 con 0x7fc5a409e760 2023-10-03T10:18:53.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.867+0000 7fc5ac7fc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a406ef30 con 0x7fc5a409de50 2023-10-03T10:18:53.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.867+0000 7fc5ac7fc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a41334b0 con 0x7fc5a41338a0 2023-10-03T10:18:53.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.867+0000 7fc5aa598700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5a41338a0 0x7fc5a4133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:53.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.867+0000 7fc5a9d97700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5a409de50 0x7fc5a409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:53.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.867+0000 7fc5aa598700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5a41338a0 0x7fc5a4133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47722/0 (socket says 172.21.15.3:47722) 2023-10-03T10:18:53.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.867+0000 7fc5aa598700 1 -- 172.21.15.3:0/1183521461 learned_addr learned my addr 172.21.15.3:0/1183521461 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:53.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.868+0000 7fc5aa598700 1 -- 172.21.15.3:0/1183521461 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5a409de50 msgr2=0x7fc5a409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:53.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.868+0000 7fc5aa598700 1 --2- 172.21.15.3:0/1183521461 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5a409de50 0x7fc5a409e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:53.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.868+0000 7fc5aa598700 1 -- 172.21.15.3:0/1183521461 >> v2:172.21.15.3:3300/0 conn(0x7fc5a409e760 msgr2=0x7fc5a413a730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-03T10:18:53.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.868+0000 7fc5aa598700 1 --2- 172.21.15.3:0/1183521461 >> v2:172.21.15.3:3300/0 conn(0x7fc5a409e760 0x7fc5a413a730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:53.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.868+0000 7fc5aa598700 1 -- 172.21.15.3:0/1183521461 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a406fc30 con 0x7fc5a41338a0 2023-10-03T10:18:53.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.868+0000 7fc5a9d97700 1 --2- 172.21.15.3:0/1183521461 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5a409de50 0x7fc5a409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:53.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.868+0000 7fc5aa598700 1 --2- 172.21.15.3:0/1183521461 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5a41338a0 0x7fc5a4133c70 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009920 tx=0x7fc5a0009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f054e60eea50efb server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:53.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.868+0000 7fc5a9596700 1 -- 172.21.15.3:0/1183521461 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0011830 con 0x7fc5a41338a0 2023-10-03T10:18:53.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.868+0000 7fc5a9596700 1 -- 172.21.15.3:0/1183521461 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a0011990 con 0x7fc5a41338a0 2023-10-03T10:18:53.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.868+0000 7fc5a9596700 1 -- 172.21.15.3:0/1183521461 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0011d20 con 0x7fc5a41338a0 2023-10-03T10:18:53.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.869+0000 7fc5ac7fc700 1 -- 172.21.15.3:0/1183521461 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5a41338a0 msgr2=0x7fc5a4133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:53.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.869+0000 7fc5ac7fc700 1 --2- 172.21.15.3:0/1183521461 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5a41338a0 0x7fc5a4133c70 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009920 tx=0x7fc5a0009c30 comp rx=0 tx=0).stop 2023-10-03T10:18:53.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.870+0000 7fc5ac7fc700 1 -- 172.21.15.3:0/1183521461 shutdown_connections 2023-10-03T10:18:53.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.870+0000 7fc5ac7fc700 1 --2- 172.21.15.3:0/1183521461 >> v2:172.21.15.3:3300/0 conn(0x7fc5a409e760 0x7fc5a413a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:53.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.870+0000 7fc5ac7fc700 1 --2- 172.21.15.3:0/1183521461 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5a41338a0 0x7fc5a4133c70 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:53.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.870+0000 7fc5ac7fc700 1 --2- 172.21.15.3:0/1183521461 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5a409de50 0x7fc5a409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:53.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.870+0000 7fc5ac7fc700 1 -- 172.21.15.3:0/1183521461 >> 172.21.15.3:0/1183521461 conn(0x7fc5a40a2aa0 msgr2=0x7fc5a40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:53.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.870+0000 7fc5ac7fc700 1 -- 172.21.15.3:0/1183521461 shutdown_connections 2023-10-03T10:18:53.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.870+0000 7fc5ac7fc700 1 -- 172.21.15.3:0/1183521461 wait complete. 2023-10-03T10:18:53.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.871+0000 7fc5ac7fc700 1 Processor -- start 2023-10-03T10:18:53.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.871+0000 7fc5ac7fc700 1 -- start start 2023-10-03T10:18:53.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.872+0000 7fc5ac7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5a409de50 0x7fc5a41cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:53.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.872+0000 7fc5ac7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5a409e760 0x7fc5a41ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:53.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.872+0000 7fc5ac7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5a41338a0 0x7fc5a41cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:53.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.872+0000 7fc5ac7fc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a4124f10 con 0x7fc5a41338a0 2023-10-03T10:18:53.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.872+0000 7fc5ac7fc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a40a1280 con 0x7fc5a409de50 2023-10-03T10:18:53.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.872+0000 7fc5ac7fc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a4125c10 con 0x7fc5a409e760 2023-10-03T10:18:53.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.872+0000 7fc5a9d97700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5a409e760 0x7fc5a41ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:53.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.872+0000 7fc5aa598700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5a409de50 0x7fc5a41cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:53.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.872+0000 7fc5a9d97700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5a409e760 0x7fc5a41ccb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47734/0 (socket says 172.21.15.3:47734) 2023-10-03T10:18:53.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.872+0000 7fc5a9d97700 1 -- 172.21.15.3:0/2904323045 learned_addr learned my addr 172.21.15.3:0/2904323045 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:53.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.872+0000 7fc5aa598700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5a409de50 0x7fc5a41cc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42022/0 (socket says 172.21.15.3:42022) 2023-10-03T10:18:53.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.872+0000 7fc5aad99700 1 --2- 172.21.15.3:0/2904323045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5a41338a0 0x7fc5a41cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:53.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.872+0000 7fc5a9d97700 1 -- 172.21.15.3:0/2904323045 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5a409de50 msgr2=0x7fc5a41cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:53.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.872+0000 7fc5a9d97700 1 --2- 172.21.15.3:0/2904323045 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5a409de50 0x7fc5a41cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:53.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.872+0000 7fc5a9d97700 1 -- 172.21.15.3:0/2904323045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5a41338a0 msgr2=0x7fc5a41cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:53.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.872+0000 7fc5a9d97700 1 --2- 172.21.15.3:0/2904323045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5a41338a0 0x7fc5a41cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:53.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.872+0000 7fc5a9d97700 1 -- 172.21.15.3:0/2904323045 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5940094a0 con 0x7fc5a409e760 2023-10-03T10:18:53.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.872+0000 7fc5aa598700 1 --2- 172.21.15.3:0/2904323045 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5a409de50 0x7fc5a41cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:53.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.873+0000 7fc5a9d97700 1 --2- 172.21.15.3:0/2904323045 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5a409e760 0x7fc5a41ccb60 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fc59400ebb0 tx=0x7fc59400eec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:53.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.873+0000 7fc59affd700 1 -- 172.21.15.3:0/2904323045 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59400c850 con 0x7fc5a409e760 2023-10-03T10:18:53.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.873+0000 7fc59affd700 1 -- 172.21.15.3:0/2904323045 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59400c9b0 con 0x7fc5a409e760 2023-10-03T10:18:53.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.873+0000 7fc5ac7fc700 1 -- 172.21.15.3:0/2904323045 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a0009550 con 0x7fc5a409e760 2023-10-03T10:18:53.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.873+0000 7fc59affd700 1 -- 172.21.15.3:0/2904323045 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59400cd40 con 0x7fc5a409e760 2023-10-03T10:18:53.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.873+0000 7fc5ac7fc700 1 -- 172.21.15.3:0/2904323045 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a4122970 con 0x7fc5a409e760 2023-10-03T10:18:53.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.874+0000 7fc59affd700 1 -- 172.21.15.3:0/2904323045 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc594013470 con 0x7fc5a409e760 2023-10-03T10:18:53.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.874+0000 7fc59affd700 1 -- 172.21.15.3:0/2904323045 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc59403f770 con 0x7fc5a409e760 2023-10-03T10:18:53.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.875+0000 7fc5ac7fc700 1 -- 172.21.15.3:0/2904323045 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a4074ea0 con 0x7fc5a409e760 2023-10-03T10:18:53.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.884+0000 7fc59affd700 1 -- 172.21.15.3:0/2904323045 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc594017080 con 0x7fc5a409e760 2023-10-03T10:18:53.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.909+0000 7f6c4b666700 1 Processor -- start 2023-10-03T10:18:53.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.910+0000 7f6c4b666700 1 -- start start 2023-10-03T10:18:53.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.910+0000 7f6c4b666700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6c4409e930 0x7f6c4409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:53.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.910+0000 7f6c4b666700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6c4409f650 0x7f6c44131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:53.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.910+0000 7f6c4b666700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c44070940 con 0x7f6c4409f240 2023-10-03T10:18:53.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.910+0000 7f6c4b666700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c4406fd60 con 0x7f6c4409f650 2023-10-03T10:18:53.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.910+0000 7f6c4b666700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4406fa80 con 0x7f6c4409e930 2023-10-03T10:18:53.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.911+0000 7f6c49402700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6c4409e930 0x7f6c4409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:53.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.911+0000 7f6c48c01700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6c4409f240 0x7f6c441317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33368/0 (socket says 172.21.15.3:33368) 2023-10-03T10:18:53.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.911+0000 7f6c48c01700 1 -- 172.21.15.3:0/2584900231 learned_addr learned my addr 172.21.15.3:0/2584900231 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:53.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.911+0000 7f6c49c03700 1 --2- 172.21.15.3:0/2584900231 >> v2:172.21.15.3:3300/0 conn(0x7f6c4409f650 0x7f6c44131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:53.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.911+0000 7f6c3bfff700 1 -- 172.21.15.3:0/2584900231 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 75342188 0 0) 0x7f6c44070940 con 0x7f6c4409f240 2023-10-03T10:18:53.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.911+0000 7f6c3bfff700 1 -- 172.21.15.3:0/2584900231 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c300035b0 con 0x7f6c4409f240 2023-10-03T10:18:53.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.911+0000 7f6c49402700 1 -- 172.21.15.3:0/2584900231 >> v2:172.21.15.3:3300/0 conn(0x7f6c4409f650 msgr2=0x7f6c44131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:53.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.911+0000 7f6c49402700 1 --2- 172.21.15.3:0/2584900231 >> v2:172.21.15.3:3300/0 conn(0x7f6c4409f650 0x7f6c44131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:53.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.911+0000 7f6c49402700 1 -- 172.21.15.3:0/2584900231 >> v1:172.21.15.3:6789/0 conn(0x7f6c4409f240 legacy=0x7f6c441317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:53.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.911+0000 7f6c49402700 1 -- 172.21.15.3:0/2584900231 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4409fb60 con 0x7f6c4409e930 2023-10-03T10:18:53.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.911+0000 7f6c49c03700 1 --2- 172.21.15.3:0/2584900231 >> v2:172.21.15.3:3300/0 conn(0x7f6c4409f650 0x7f6c44131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:18:53.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.912+0000 7f6c49402700 1 --2- 172.21.15.3:0/2584900231 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6c4409e930 0x7f6c4409ed00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6c40009840 tx=0x7f6c40009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a19c8bf7d2ec4c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:53.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.912+0000 7f6c3bfff700 1 -- 172.21.15.3:0/2584900231 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c40011830 con 0x7f6c4409e930 2023-10-03T10:18:53.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.912+0000 7f6c3bfff700 1 -- 172.21.15.3:0/2584900231 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c40011990 con 0x7f6c4409e930 2023-10-03T10:18:53.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.912+0000 7f6c3bfff700 1 -- 172.21.15.3:0/2584900231 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c40011d20 con 0x7f6c4409e930 2023-10-03T10:18:53.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.913+0000 7f6c4b666700 1 -- 172.21.15.3:0/2584900231 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6c4409e930 msgr2=0x7f6c4409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:53.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.913+0000 7f6c4b666700 1 --2- 172.21.15.3:0/2584900231 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6c4409e930 0x7f6c4409ed00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6c40009840 tx=0x7f6c40009b50 comp rx=0 tx=0).stop 2023-10-03T10:18:53.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.913+0000 7f6c4b666700 1 -- 172.21.15.3:0/2584900231 shutdown_connections 2023-10-03T10:18:53.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.913+0000 7f6c4b666700 1 --2- 172.21.15.3:0/2584900231 >> v2:172.21.15.3:3300/0 conn(0x7f6c4409f650 0x7f6c44131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:53.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.913+0000 7f6c4b666700 1 --2- 172.21.15.3:0/2584900231 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6c4409e930 0x7f6c4409ed00 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:53.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.913+0000 7f6c4b666700 1 -- 172.21.15.3:0/2584900231 >> 172.21.15.3:0/2584900231 conn(0x7f6c440a4210 msgr2=0x7f6c440a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:53.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.913+0000 7f6c4b666700 1 -- 172.21.15.3:0/2584900231 shutdown_connections 2023-10-03T10:18:53.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.913+0000 7f6c4b666700 1 -- 172.21.15.3:0/2584900231 wait complete. 2023-10-03T10:18:53.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.914+0000 7f6c4b666700 1 Processor -- start 2023-10-03T10:18:53.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.914+0000 7f6c4b666700 1 -- start start 2023-10-03T10:18:53.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.915+0000 7f6c4b666700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c4409e930 0x7f6c441a3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:53.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.915+0000 7f6c4b666700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6c4409f240 0x7f6c441a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:53.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.915+0000 7f6c4b666700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6c4409f650 0x7f6c441a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:53.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.915+0000 7f6c4b666700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c441a79d0 con 0x7f6c4409e930 2023-10-03T10:18:53.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.915+0000 7f6c4b666700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c441a60a0 con 0x7f6c4409f240 2023-10-03T10:18:53.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.915+0000 7f6c4b666700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c441a7160 con 0x7f6c4409f650 2023-10-03T10:18:53.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.915+0000 7f6c49402700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c4409e930 0x7f6c441a3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:53.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.915+0000 7f6c49402700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c4409e930 0x7f6c441a3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49540/0 (socket says 172.21.15.3:49540) 2023-10-03T10:18:53.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.915+0000 7f6c49402700 1 -- 172.21.15.3:0/1282491714 learned_addr learned my addr 172.21.15.3:0/1282491714 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:53.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.915+0000 7f6c48c01700 1 --2- 172.21.15.3:0/1282491714 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6c4409f240 0x7f6c441a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:53.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.915+0000 7f6c49c03700 1 --2- 172.21.15.3:0/1282491714 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6c4409f650 0x7f6c441a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:53.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.915+0000 7f6c48c01700 1 -- 172.21.15.3:0/1282491714 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6c4409f650 msgr2=0x7f6c441a4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:53.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.916+0000 7f6c48c01700 1 --2- 172.21.15.3:0/1282491714 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6c4409f650 0x7f6c441a4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:53.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.916+0000 7f6c48c01700 1 -- 172.21.15.3:0/1282491714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c4409e930 msgr2=0x7f6c441a3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:53.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.916+0000 7f6c48c01700 1 --2- 172.21.15.3:0/1282491714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c4409e930 0x7f6c441a3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:53.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.916+0000 7f6c48c01700 1 -- 172.21.15.3:0/1282491714 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c400094a0 con 0x7f6c4409f240 2023-10-03T10:18:53.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.916+0000 7f6c48c01700 1 --2- 172.21.15.3:0/1282491714 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6c4409f240 0x7f6c441a4440 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400c460 tx=0x7f6c3400c770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:53.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.916+0000 7f6c39ffb700 1 -- 172.21.15.3:0/1282491714 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c34005d70 con 0x7f6c4409f240 2023-10-03T10:18:53.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.916+0000 7f6c39ffb700 1 -- 172.21.15.3:0/1282491714 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c340070d0 con 0x7f6c4409f240 2023-10-03T10:18:53.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.916+0000 7f6c4b666700 1 -- 172.21.15.3:0/1282491714 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c440a0190 con 0x7f6c4409f240 2023-10-03T10:18:53.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.916+0000 7f6c39ffb700 1 -- 172.21.15.3:0/1282491714 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c34007480 con 0x7f6c4409f240 2023-10-03T10:18:53.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.916+0000 7f6c4b666700 1 -- 172.21.15.3:0/1282491714 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c441b72a0 con 0x7f6c4409f240 2023-10-03T10:18:53.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.917+0000 7f6c39ffb700 1 -- 172.21.15.3:0/1282491714 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6c340075e0 con 0x7f6c4409f240 2023-10-03T10:18:53.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.917+0000 7f6c39ffb700 1 -- 172.21.15.3:0/1282491714 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6c3403ea60 con 0x7f6c4409f240 2023-10-03T10:18:53.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.919+0000 7f6c4b666700 1 -- 172.21.15.3:0/1282491714 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c441a7390 con 0x7f6c4409f240 2023-10-03T10:18:53.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.924+0000 7f6c39ffb700 1 -- 172.21.15.3:0/1282491714 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6c441a7390 con 0x7f6c4409f240 2023-10-03T10:18:53.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.948+0000 7fbc812e1700 1 Processor -- start 2023-10-03T10:18:53.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.948+0000 7fbc812e1700 1 -- start start 2023-10-03T10:18:53.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.949+0000 7fbc812e1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc7c09e1d0 0x7fbc7c138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:53.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.949+0000 7fbc812e1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbc7c139170 0x7fbc7c13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:53.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.949+0000 7fbc812e1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc7c09b120 con 0x7fbc7c09d6f0 2023-10-03T10:18:53.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.949+0000 7fbc812e1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc7c06b460 con 0x7fbc7c139170 2023-10-03T10:18:53.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.949+0000 7fbc812e1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7c09a7e0 con 0x7fbc7c09e1d0 2023-10-03T10:18:53.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.949+0000 7fbc727fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc7c09e1d0 0x7fbc7c138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:53.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.949+0000 7fbc727fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc7c09e1d0 0x7fbc7c138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42032/0 (socket says 172.21.15.3:42032) 2023-10-03T10:18:53.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.949+0000 7fbc727fc700 1 -- 172.21.15.3:0/3936913574 learned_addr learned my addr 172.21.15.3:0/3936913574 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:53.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.949+0000 7fbc7b7fe700 1 --2- 172.21.15.3:0/3936913574 >> v2:172.21.15.3:3300/0 conn(0x7fbc7c139170 0x7fbc7c13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:53.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.949+0000 7fbc727fc700 1 -- 172.21.15.3:0/3936913574 >> v2:172.21.15.3:3300/0 conn(0x7fbc7c139170 msgr2=0x7fbc7c13b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:53.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.949+0000 7fbc727fc700 1 --2- 172.21.15.3:0/3936913574 >> v2:172.21.15.3:3300/0 conn(0x7fbc7c139170 0x7fbc7c13b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:53.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.949+0000 7fbc727fc700 1 -- 172.21.15.3:0/3936913574 >> v1:172.21.15.3:6789/0 conn(0x7fbc7c09d6f0 legacy=0x7fbc7c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:53.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.949+0000 7fbc727fc700 1 -- 172.21.15.3:0/3936913574 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc7c09a500 con 0x7fbc7c09e1d0 2023-10-03T10:18:53.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.949+0000 7fbc7b7fe700 1 --2- 172.21.15.3:0/3936913574 >> v2:172.21.15.3:3300/0 conn(0x7fbc7c139170 0x7fbc7c13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:53.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.950+0000 7fbc727fc700 1 --2- 172.21.15.3:0/3936913574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc7c09e1d0 0x7fbc7c138c30 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fbc64009920 tx=0x7fbc64009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4df96f2d2e31407 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:53.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.950+0000 7fbc7a7fc700 1 -- 172.21.15.3:0/3936913574 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64011830 con 0x7fbc7c09e1d0 2023-10-03T10:18:53.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.950+0000 7fbc7a7fc700 1 -- 172.21.15.3:0/3936913574 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc64011990 con 0x7fbc7c09e1d0 2023-10-03T10:18:53.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.950+0000 7fbc7a7fc700 1 -- 172.21.15.3:0/3936913574 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64011d20 con 0x7fbc7c09e1d0 2023-10-03T10:18:53.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.951+0000 7fbc812e1700 1 -- 172.21.15.3:0/3936913574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc7c09e1d0 msgr2=0x7fbc7c138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:53.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.951+0000 7fbc812e1700 1 --2- 172.21.15.3:0/3936913574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc7c09e1d0 0x7fbc7c138c30 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fbc64009920 tx=0x7fbc64009c30 comp rx=0 tx=0).stop 2023-10-03T10:18:53.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.951+0000 7fbc812e1700 1 -- 172.21.15.3:0/3936913574 shutdown_connections 2023-10-03T10:18:53.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.951+0000 7fbc812e1700 1 --2- 172.21.15.3:0/3936913574 >> v2:172.21.15.3:3300/0 conn(0x7fbc7c139170 0x7fbc7c13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:53.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.951+0000 7fbc812e1700 1 --2- 172.21.15.3:0/3936913574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc7c09e1d0 0x7fbc7c138c30 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:53.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.951+0000 7fbc812e1700 1 -- 172.21.15.3:0/3936913574 >> 172.21.15.3:0/3936913574 conn(0x7fbc7c066f50 msgr2=0x7fbc7c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:53.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.951+0000 7fbc812e1700 1 -- 172.21.15.3:0/3936913574 shutdown_connections 2023-10-03T10:18:53.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.951+0000 7fbc812e1700 1 -- 172.21.15.3:0/3936913574 wait complete. 2023-10-03T10:18:53.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.952+0000 7fbc812e1700 1 Processor -- start 2023-10-03T10:18:53.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.952+0000 7fbc812e1700 1 -- start start 2023-10-03T10:18:53.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.953+0000 7fbc812e1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc7c09d6f0 0x7fbc7c122470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:53.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.953+0000 7fbc812e1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc7c09e1d0 0x7fbc7c1229b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:53.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.953+0000 7fbc812e1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc7c139170 0x7fbc7c1c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:53.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.953+0000 7fbc812e1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7c122000 con 0x7fbc7c139170 2023-10-03T10:18:53.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.953+0000 7fbc812e1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7c124fd0 con 0x7fbc7c09e1d0 2023-10-03T10:18:53.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.953+0000 7fbc812e1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7c0a2a90 con 0x7fbc7c09d6f0 2023-10-03T10:18:53.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.953+0000 7fbc7affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc7c09d6f0 0x7fbc7c122470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:53.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.953+0000 7fbc7affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc7c09d6f0 0x7fbc7c122470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47770/0 (socket says 172.21.15.3:47770) 2023-10-03T10:18:53.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.953+0000 7fbc7affd700 1 -- 172.21.15.3:0/1055331902 learned_addr learned my addr 172.21.15.3:0/1055331902 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:53.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.953+0000 7fbc727fc700 1 --2- 172.21.15.3:0/1055331902 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc7c09e1d0 0x7fbc7c1229b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:53.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.954+0000 7fbc7b7fe700 1 --2- 172.21.15.3:0/1055331902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc7c139170 0x7fbc7c1c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:53.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.954+0000 7fbc727fc700 1 -- 172.21.15.3:0/1055331902 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc7c09d6f0 msgr2=0x7fbc7c122470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:53.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.954+0000 7fbc727fc700 1 --2- 172.21.15.3:0/1055331902 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc7c09d6f0 0x7fbc7c122470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:53.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.954+0000 7fbc727fc700 1 -- 172.21.15.3:0/1055331902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc7c139170 msgr2=0x7fbc7c1c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:53.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.954+0000 7fbc727fc700 1 --2- 172.21.15.3:0/1055331902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc7c139170 0x7fbc7c1c8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:53.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.954+0000 7fbc727fc700 1 -- 172.21.15.3:0/1055331902 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc74009a70 con 0x7fbc7c09e1d0 2023-10-03T10:18:53.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.954+0000 7fbc7affd700 1 --2- 172.21.15.3:0/1055331902 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc7c09d6f0 0x7fbc7c122470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:53.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.954+0000 7fbc727fc700 1 --2- 172.21.15.3:0/1055331902 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc7c09e1d0 0x7fbc7c1229b0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fbc64016000 tx=0x7fbc64004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:53.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.954+0000 7fbc73fff700 1 -- 172.21.15.3:0/1055331902 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc640044d0 con 0x7fbc7c09e1d0 2023-10-03T10:18:53.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.954+0000 7fbc73fff700 1 -- 172.21.15.3:0/1055331902 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc640036a0 con 0x7fbc7c09e1d0 2023-10-03T10:18:53.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.954+0000 7fbc73fff700 1 -- 172.21.15.3:0/1055331902 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64003a50 con 0x7fbc7c09e1d0 2023-10-03T10:18:53.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.954+0000 7fbc812e1700 1 -- 172.21.15.3:0/1055331902 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc64009550 con 0x7fbc7c09e1d0 2023-10-03T10:18:53.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.954+0000 7fbc812e1700 1 -- 172.21.15.3:0/1055331902 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc7c125d00 con 0x7fbc7c09e1d0 2023-10-03T10:18:53.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.956+0000 7fbc73fff700 1 -- 172.21.15.3:0/1055331902 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbc64003bb0 con 0x7fbc7c09e1d0 2023-10-03T10:18:53.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.956+0000 7fbc717fa700 1 -- 172.21.15.3:0/1055331902 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc540052e0 con 0x7fbc7c09e1d0 2023-10-03T10:18:53.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.956+0000 7fbc73fff700 1 -- 172.21.15.3:0/1055331902 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fbc64021030 con 0x7fbc7c09e1d0 2023-10-03T10:18:53.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:53.961+0000 7fbc73fff700 1 -- 172.21.15.3:0/1055331902 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbc6401a7b0 con 0x7fbc7c09e1d0 2023-10-03T10:18:54.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.020+0000 7f51aec56700 1 Processor -- start 2023-10-03T10:18:54.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.020+0000 7f51aec56700 1 -- start start 2023-10-03T10:18:54.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.021+0000 7f51aec56700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f51a80a0a30 0x7f51a80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:54.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.021+0000 7f51aec56700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f51a80a1340 0x7f51a8133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:54.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.021+0000 7f51aec56700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a809edd0 con 0x7f51a8133fa0 2023-10-03T10:18:54.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.021+0000 7f51aec56700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a809e1f0 con 0x7f51a80a0a30 2023-10-03T10:18:54.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.021+0000 7f51aec56700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f51a809dc50 con 0x7f51a80a1340 2023-10-03T10:18:54.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.021+0000 7f51a7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f51a80a1340 0x7f51a8133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:54.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.021+0000 7f51a7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f51a80a1340 0x7f51a8133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47784/0 (socket says 172.21.15.3:47784) 2023-10-03T10:18:54.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.021+0000 7f51a7fff700 1 -- 172.21.15.3:0/157254506 learned_addr learned my addr 172.21.15.3:0/157254506 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:54.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.021+0000 7f51ac9f2700 1 --2- 172.21.15.3:0/157254506 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f51a80a0a30 0x7f51a80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:54.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.021+0000 7f51a7fff700 1 -- 172.21.15.3:0/157254506 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f51a80a0a30 msgr2=0x7f51a80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:54.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.021+0000 7f51a7fff700 1 --2- 172.21.15.3:0/157254506 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f51a80a0a30 0x7f51a80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:54.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.021+0000 7f51a7fff700 1 -- 172.21.15.3:0/157254506 >> v1:172.21.15.3:6789/0 conn(0x7f51a8133fa0 legacy=0x7f51a813ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:54.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.021+0000 7f51a7fff700 1 -- 172.21.15.3:0/157254506 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a80a1fe0 con 0x7f51a80a1340 2023-10-03T10:18:54.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.021+0000 7f51ac9f2700 1 --2- 172.21.15.3:0/157254506 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f51a80a0a30 0x7f51a80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:18:54.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.021+0000 7f51a7fff700 1 --2- 172.21.15.3:0/157254506 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f51a80a1340 0x7f51a8133a60 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5194009920 tx=0x7f5194009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68e3fa21fae767aa server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:54.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.022+0000 7f51a77fe700 1 -- 172.21.15.3:0/157254506 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519400ba60 con 0x7f51a80a1340 2023-10-03T10:18:54.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.022+0000 7f51a77fe700 1 -- 172.21.15.3:0/157254506 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519400bbc0 con 0x7f51a80a1340 2023-10-03T10:18:54.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.022+0000 7f51a77fe700 1 -- 172.21.15.3:0/157254506 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194009e70 con 0x7f51a80a1340 2023-10-03T10:18:54.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.022+0000 7f51aec56700 1 -- 172.21.15.3:0/157254506 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f51a80a1340 msgr2=0x7f51a8133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:54.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.022+0000 7f51aec56700 1 --2- 172.21.15.3:0/157254506 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f51a80a1340 0x7f51a8133a60 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5194009920 tx=0x7f5194009c30 comp rx=0 tx=0).stop 2023-10-03T10:18:54.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.023+0000 7f51aec56700 1 -- 172.21.15.3:0/157254506 shutdown_connections 2023-10-03T10:18:54.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.023+0000 7f51aec56700 1 --2- 172.21.15.3:0/157254506 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f51a80a0a30 0x7f51a80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:54.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.023+0000 7f51aec56700 1 --2- 172.21.15.3:0/157254506 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f51a80a1340 0x7f51a8133a60 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:54.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.023+0000 7f51aec56700 1 -- 172.21.15.3:0/157254506 >> 172.21.15.3:0/157254506 conn(0x7f51a80a6550 msgr2=0x7f51a80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:54.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.023+0000 7f51aec56700 1 -- 172.21.15.3:0/157254506 shutdown_connections 2023-10-03T10:18:54.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.023+0000 7f51aec56700 1 -- 172.21.15.3:0/157254506 wait complete. 2023-10-03T10:18:54.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.024+0000 7f51aec56700 1 Processor -- start 2023-10-03T10:18:54.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.024+0000 7f51aec56700 1 -- start start 2023-10-03T10:18:54.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.025+0000 7f51aec56700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f51a80a0a30 0x7f51a811e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:54.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.025+0000 7f51aec56700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f51a80a1340 0x7f51a811eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:54.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.025+0000 7f51aec56700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51a8133fa0 0x7f51a811f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:54.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.025+0000 7f51aec56700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a8122110 con 0x7f51a8133fa0 2023-10-03T10:18:54.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.025+0000 7f51aec56700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a81205f0 con 0x7f51a80a0a30 2023-10-03T10:18:54.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.025+0000 7f51aec56700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f51a8133430 con 0x7f51a80a1340 2023-10-03T10:18:54.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.025+0000 7f51ac9f2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f51a80a0a30 0x7f51a811e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:54.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.025+0000 7f51ac9f2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f51a80a0a30 0x7f51a811e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42058/0 (socket says 172.21.15.3:42058) 2023-10-03T10:18:54.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.025+0000 7f51ad1f3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51a8133fa0 0x7f51a811f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:54.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.025+0000 7f51ac9f2700 1 -- 172.21.15.3:0/202456768 learned_addr learned my addr 172.21.15.3:0/202456768 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:54.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.026+0000 7f51ac9f2700 1 -- 172.21.15.3:0/202456768 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f51a80a1340 msgr2=0x7f51a811eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:54.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.026+0000 7f51ac9f2700 1 --2- 172.21.15.3:0/202456768 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f51a80a1340 0x7f51a811eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:54.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.026+0000 7f51ac9f2700 1 -- 172.21.15.3:0/202456768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51a8133fa0 msgr2=0x7f51a811f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:54.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.026+0000 7f51ac9f2700 1 --2- 172.21.15.3:0/202456768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51a8133fa0 0x7f51a811f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:54.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.026+0000 7f51ac9f2700 1 -- 172.21.15.3:0/202456768 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5194009550 con 0x7f51a80a0a30 2023-10-03T10:18:54.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.026+0000 7f51ad1f3700 1 --2- 172.21.15.3:0/202456768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51a8133fa0 0x7f51a811f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:54.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.026+0000 7f51ac9f2700 1 --2- 172.21.15.3:0/202456768 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f51a80a0a30 0x7f51a811e960 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f519c00e810 tx=0x7f519c00eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:54.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.026+0000 7f51a57fa700 1 -- 172.21.15.3:0/202456768 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c013610 con 0x7f51a80a0a30 2023-10-03T10:18:54.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.026+0000 7f51a57fa700 1 -- 172.21.15.3:0/202456768 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519c013770 con 0x7f51a80a0a30 2023-10-03T10:18:54.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.026+0000 7f51a57fa700 1 -- 172.21.15.3:0/202456768 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c013b20 con 0x7f51a80a0a30 2023-10-03T10:18:54.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.027+0000 7f51aec56700 1 -- 172.21.15.3:0/202456768 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a80a25e0 con 0x7f51a80a0a30 2023-10-03T10:18:54.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.027+0000 7f51aec56700 1 -- 172.21.15.3:0/202456768 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a8121e50 con 0x7f51a80a0a30 2023-10-03T10:18:54.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.029+0000 7f51aec56700 1 -- 172.21.15.3:0/202456768 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a8074ea0 con 0x7f51a80a0a30 2023-10-03T10:18:54.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.029+0000 7f51a57fa700 1 -- 172.21.15.3:0/202456768 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f519c013c80 con 0x7f51a80a0a30 2023-10-03T10:18:54.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.029+0000 7f51a57fa700 1 -- 172.21.15.3:0/202456768 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f519c0126a0 con 0x7f51a80a0a30 2023-10-03T10:18:54.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.039+0000 7f51a57fa700 1 -- 172.21.15.3:0/202456768 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f519c044060 con 0x7f51a80a0a30 2023-10-03T10:18:54.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.099+0000 7fc5ac7fc700 1 -- 172.21.15.3:0/2904323045 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc5a406b9e0 con 0x7fc5a409e760 2023-10-03T10:18:54.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.099+0000 7fc59affd700 1 -- 172.21.15.3:0/2904323045 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc594021020 con 0x7fc5a409e760 2023-10-03T10:18:54.099 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:54.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.103+0000 7fc5ac7fc700 1 -- 172.21.15.3:0/2904323045 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5a409e760 msgr2=0x7fc5a41ccb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:54.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.103+0000 7fc5ac7fc700 1 --2- 172.21.15.3:0/2904323045 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5a409e760 0x7fc5a41ccb60 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fc59400ebb0 tx=0x7fc59400eec0 comp rx=0 tx=0).stop 2023-10-03T10:18:54.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.103+0000 7fc5ac7fc700 1 -- 172.21.15.3:0/2904323045 shutdown_connections 2023-10-03T10:18:54.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.103+0000 7fc5ac7fc700 1 --2- 172.21.15.3:0/2904323045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5a41338a0 0x7fc5a41cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:54.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.103+0000 7fc5ac7fc700 1 --2- 172.21.15.3:0/2904323045 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc5a409de50 0x7fc5a41cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:54.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.103+0000 7fc5ac7fc700 1 --2- 172.21.15.3:0/2904323045 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc5a409e760 0x7fc5a41ccb60 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:54.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.103+0000 7fc5ac7fc700 1 -- 172.21.15.3:0/2904323045 >> 172.21.15.3:0/2904323045 conn(0x7fc5a40a2aa0 msgr2=0x7fc5a409ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:54.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.103+0000 7fc5ac7fc700 1 -- 172.21.15.3:0/2904323045 shutdown_connections 2023-10-03T10:18:54.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.103+0000 7fc5ac7fc700 1 -- 172.21.15.3:0/2904323045 wait complete. 2023-10-03T10:18:54.122 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:54.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.167+0000 7f6c4b666700 1 -- 172.21.15.3:0/1282491714 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6c441a7390 con 0x7f6c4409f240 2023-10-03T10:18:54.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.167+0000 7f6c39ffb700 1 -- 172.21.15.3:0/1282491714 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6c441a7390 con 0x7f6c4409f240 2023-10-03T10:18:54.168 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:54.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.170+0000 7f6c4b666700 1 -- 172.21.15.3:0/1282491714 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6c4409f240 msgr2=0x7f6c441a4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:54.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.170+0000 7f6c4b666700 1 --2- 172.21.15.3:0/1282491714 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6c4409f240 0x7f6c441a4440 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400c460 tx=0x7f6c3400c770 comp rx=0 tx=0).stop 2023-10-03T10:18:54.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.170+0000 7f6c4b666700 1 -- 172.21.15.3:0/1282491714 shutdown_connections 2023-10-03T10:18:54.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.170+0000 7f6c4b666700 1 --2- 172.21.15.3:0/1282491714 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6c4409f650 0x7f6c441a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:54.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.170+0000 7f6c4b666700 1 --2- 172.21.15.3:0/1282491714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c4409e930 0x7f6c441a3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:54.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.170+0000 7f6c4b666700 1 --2- 172.21.15.3:0/1282491714 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6c4409f240 0x7f6c441a4440 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:54.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.170+0000 7f6c4b666700 1 -- 172.21.15.3:0/1282491714 >> 172.21.15.3:0/1282491714 conn(0x7f6c440a4210 msgr2=0x7f6c441b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:54.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.170+0000 7f6c4b666700 1 -- 172.21.15.3:0/1282491714 shutdown_connections 2023-10-03T10:18:54.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.170+0000 7f6c4b666700 1 -- 172.21.15.3:0/1282491714 wait complete. 2023-10-03T10:18:54.184 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:54.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.192+0000 7fbc717fa700 1 -- 172.21.15.3:0/1055331902 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbc54005590 con 0x7fbc7c09e1d0 2023-10-03T10:18:54.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.192+0000 7fbc73fff700 1 -- 172.21.15.3:0/1055331902 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbc64015070 con 0x7fbc7c09e1d0 2023-10-03T10:18:54.192 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:54.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.194+0000 7fbc717fa700 1 -- 172.21.15.3:0/1055331902 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc7c09e1d0 msgr2=0x7fbc7c1229b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:54.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.194+0000 7fbc717fa700 1 --2- 172.21.15.3:0/1055331902 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc7c09e1d0 0x7fbc7c1229b0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fbc64016000 tx=0x7fbc64004750 comp rx=0 tx=0).stop 2023-10-03T10:18:54.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.194+0000 7fbc717fa700 1 -- 172.21.15.3:0/1055331902 shutdown_connections 2023-10-03T10:18:54.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.194+0000 7fbc717fa700 1 --2- 172.21.15.3:0/1055331902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc7c139170 0x7fbc7c1c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:54.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.194+0000 7fbc717fa700 1 --2- 172.21.15.3:0/1055331902 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc7c09d6f0 0x7fbc7c122470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:54.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.194+0000 7fbc717fa700 1 --2- 172.21.15.3:0/1055331902 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc7c09e1d0 0x7fbc7c1229b0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:54.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.194+0000 7fbc717fa700 1 -- 172.21.15.3:0/1055331902 >> 172.21.15.3:0/1055331902 conn(0x7fbc7c066f50 msgr2=0x7fbc7c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:54.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.194+0000 7fbc717fa700 1 -- 172.21.15.3:0/1055331902 shutdown_connections 2023-10-03T10:18:54.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.194+0000 7fbc717fa700 1 -- 172.21.15.3:0/1055331902 wait complete. 2023-10-03T10:18:54.208 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:54.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.245+0000 7f51aec56700 1 -- 172.21.15.3:0/202456768 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f51a80a28e0 con 0x7f51a80a0a30 2023-10-03T10:18:54.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.245+0000 7f51a57fa700 1 -- 172.21.15.3:0/202456768 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f519c044070 con 0x7f51a80a0a30 2023-10-03T10:18:54.246 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:54.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.248+0000 7f51aec56700 1 -- 172.21.15.3:0/202456768 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f51a80a0a30 msgr2=0x7f51a811e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:54.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.248+0000 7f51aec56700 1 --2- 172.21.15.3:0/202456768 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f51a80a0a30 0x7f51a811e960 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f519c00e810 tx=0x7f519c00eb20 comp rx=0 tx=0).stop 2023-10-03T10:18:54.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.248+0000 7f51aec56700 1 -- 172.21.15.3:0/202456768 shutdown_connections 2023-10-03T10:18:54.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.248+0000 7f51aec56700 1 --2- 172.21.15.3:0/202456768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51a8133fa0 0x7f51a811f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:54.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.248+0000 7f51aec56700 1 --2- 172.21.15.3:0/202456768 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f51a80a0a30 0x7f51a811e960 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:54.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.248+0000 7f51aec56700 1 --2- 172.21.15.3:0/202456768 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f51a80a1340 0x7f51a811eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:54.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.248+0000 7f51aec56700 1 -- 172.21.15.3:0/202456768 >> 172.21.15.3:0/202456768 conn(0x7f51a80a6550 msgr2=0x7f51a81b9120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:54.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.248+0000 7f51aec56700 1 -- 172.21.15.3:0/202456768 shutdown_connections 2023-10-03T10:18:54.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:54.248+0000 7f51aec56700 1 -- 172.21.15.3:0/202456768 wait complete. 2023-10-03T10:18:54.261 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:55.123 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:55.185 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:55.209 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:55.262 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:55.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.272+0000 7f791144d700 1 Processor -- start 2023-10-03T10:18:55.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.272+0000 7f791144d700 1 -- start start 2023-10-03T10:18:55.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.273+0000 7f791144d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f790c09e1d0 0x7f790c135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:55.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.273+0000 7f791144d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f790c135e50 0x7f790c138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:55.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.273+0000 7f791144d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f790c09f130 con 0x7f790c09d6f0 2023-10-03T10:18:55.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.273+0000 7f791144d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f790c09f520 con 0x7f790c09e1d0 2023-10-03T10:18:55.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.273+0000 7f791144d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f790c09e930 con 0x7f790c135e50 2023-10-03T10:18:55.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.273+0000 7f790b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f790c135e50 0x7f790c138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:55.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.273+0000 7f790a7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f790c09e1d0 0x7f790c135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:55.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.273+0000 7f790a7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f790c09e1d0 0x7f790c135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49570/0 (socket says 172.21.15.3:49570) 2023-10-03T10:18:55.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.273+0000 7f790b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f790c135e50 0x7f790c138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47806/0 (socket says 172.21.15.3:47806) 2023-10-03T10:18:55.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.273+0000 7f790a7fc700 1 -- 172.21.15.3:0/3556870443 learned_addr learned my addr 172.21.15.3:0/3556870443 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:55.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.274+0000 7f790b7fe700 1 -- 172.21.15.3:0/3556870443 >> v2:172.21.15.3:3300/0 conn(0x7f790c09e1d0 msgr2=0x7f790c135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:55.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.274+0000 7f790b7fe700 1 --2- 172.21.15.3:0/3556870443 >> v2:172.21.15.3:3300/0 conn(0x7f790c09e1d0 0x7f790c135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.274+0000 7f790b7fe700 1 -- 172.21.15.3:0/3556870443 >> v1:172.21.15.3:6789/0 conn(0x7f790c09d6f0 legacy=0x7f790c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:55.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.274+0000 7f790b7fe700 1 -- 172.21.15.3:0/3556870443 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790c09b2d0 con 0x7f790c135e50 2023-10-03T10:18:55.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.274+0000 7f790a7fc700 1 --2- 172.21.15.3:0/3556870443 >> v2:172.21.15.3:3300/0 conn(0x7f790c09e1d0 0x7f790c135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:55.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.274+0000 7f790b7fe700 1 --2- 172.21.15.3:0/3556870443 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f790c135e50 0x7f790c138230 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f7904009d90 tx=0x7f790400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c9dd7d1cb12a0c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:55.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.274+0000 7f7909ffb700 1 -- 172.21.15.3:0/3556870443 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790400e070 con 0x7f790c135e50 2023-10-03T10:18:55.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.274+0000 7f7909ffb700 1 -- 172.21.15.3:0/3556870443 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790400bb20 con 0x7f790c135e50 2023-10-03T10:18:55.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.274+0000 7f7909ffb700 1 -- 172.21.15.3:0/3556870443 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7904012430 con 0x7f790c135e50 2023-10-03T10:18:55.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.275+0000 7f791144d700 1 -- 172.21.15.3:0/3556870443 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f790c135e50 msgr2=0x7f790c138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:55.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.275+0000 7f791144d700 1 --2- 172.21.15.3:0/3556870443 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f790c135e50 0x7f790c138230 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f7904009d90 tx=0x7f790400b330 comp rx=0 tx=0).stop 2023-10-03T10:18:55.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.275+0000 7f791144d700 1 -- 172.21.15.3:0/3556870443 shutdown_connections 2023-10-03T10:18:55.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.275+0000 7f791144d700 1 --2- 172.21.15.3:0/3556870443 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f790c135e50 0x7f790c138230 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.275+0000 7f791144d700 1 --2- 172.21.15.3:0/3556870443 >> v2:172.21.15.3:3300/0 conn(0x7f790c09e1d0 0x7f790c135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.275+0000 7f791144d700 1 -- 172.21.15.3:0/3556870443 >> 172.21.15.3:0/3556870443 conn(0x7f790c066f50 msgr2=0x7f790c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:55.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.275+0000 7f791144d700 1 -- 172.21.15.3:0/3556870443 shutdown_connections 2023-10-03T10:18:55.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.276+0000 7f791144d700 1 -- 172.21.15.3:0/3556870443 wait complete. 2023-10-03T10:18:55.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.277+0000 7f791144d700 1 Processor -- start 2023-10-03T10:18:55.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.277+0000 7f791144d700 1 -- start start 2023-10-03T10:18:55.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.278+0000 7f791144d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f790c09d6f0 0x7f790c134c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:55.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.278+0000 7f791144d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f790c09e1d0 0x7f790c135190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:55.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.278+0000 7f791144d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f790c135e50 0x7f790c1356d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:55.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.278+0000 7f791144d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790c13b4f0 con 0x7f790c135e50 2023-10-03T10:18:55.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.278+0000 7f791144d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f790c133440 con 0x7f790c09d6f0 2023-10-03T10:18:55.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.278+0000 7f791144d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f790c13ac10 con 0x7f790c09e1d0 2023-10-03T10:18:55.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.278+0000 7f790a7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f790c09e1d0 0x7f790c135190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:55.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.278+0000 7f790a7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f790c09e1d0 0x7f790c135190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47812/0 (socket says 172.21.15.3:47812) 2023-10-03T10:18:55.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.278+0000 7f790a7fc700 1 -- 172.21.15.3:0/1765638737 learned_addr learned my addr 172.21.15.3:0/1765638737 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:55.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.278+0000 7f790b7fe700 1 --2- 172.21.15.3:0/1765638737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f790c135e50 0x7f790c1356d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:55.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.279+0000 7f790affd700 1 --2- 172.21.15.3:0/1765638737 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f790c09d6f0 0x7f790c134c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:55.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.279+0000 7f790a7fc700 1 -- 172.21.15.3:0/1765638737 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f790c09d6f0 msgr2=0x7f790c134c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:55.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.279+0000 7f790a7fc700 1 --2- 172.21.15.3:0/1765638737 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f790c09d6f0 0x7f790c134c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.279+0000 7f790a7fc700 1 -- 172.21.15.3:0/1765638737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f790c135e50 msgr2=0x7f790c1356d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:55.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.279+0000 7f790a7fc700 1 --2- 172.21.15.3:0/1765638737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f790c135e50 0x7f790c1356d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.279+0000 7f790a7fc700 1 -- 172.21.15.3:0/1765638737 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7900009550 con 0x7f790c09e1d0 2023-10-03T10:18:55.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.279+0000 7f790a7fc700 1 --2- 172.21.15.3:0/1765638737 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f790c09e1d0 0x7f790c135190 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f790000ebe0 tx=0x7f790000eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:55.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.279+0000 7f78f37fe700 1 -- 172.21.15.3:0/1765638737 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790000c850 con 0x7f790c09e1d0 2023-10-03T10:18:55.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.279+0000 7f78f37fe700 1 -- 172.21.15.3:0/1765638737 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790000c9b0 con 0x7f790c09e1d0 2023-10-03T10:18:55.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.279+0000 7f78f37fe700 1 -- 172.21.15.3:0/1765638737 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790000cd40 con 0x7f790c09e1d0 2023-10-03T10:18:55.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.279+0000 7f791144d700 1 -- 172.21.15.3:0/1765638737 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7904009a70 con 0x7f790c09e1d0 2023-10-03T10:18:55.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.280+0000 7f791144d700 1 -- 172.21.15.3:0/1765638737 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f790c09c700 con 0x7f790c09e1d0 2023-10-03T10:18:55.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.282+0000 7f791144d700 1 -- 172.21.15.3:0/1765638737 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f790c074ea0 con 0x7f790c09e1d0 2023-10-03T10:18:55.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.282+0000 7f78f37fe700 1 -- 172.21.15.3:0/1765638737 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7900013470 con 0x7f790c09e1d0 2023-10-03T10:18:55.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.282+0000 7f78f37fe700 1 -- 172.21.15.3:0/1765638737 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f790003fc40 con 0x7f790c09e1d0 2023-10-03T10:18:55.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.291+0000 7f78f37fe700 1 -- 172.21.15.3:0/1765638737 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f790001b3b0 con 0x7f790c09e1d0 2023-10-03T10:18:55.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.335+0000 7f4f9c3f9700 1 Processor -- start 2023-10-03T10:18:55.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.335+0000 7f4f9c3f9700 1 -- start start 2023-10-03T10:18:55.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.336+0000 7f4f9c3f9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4f94130670 0x7f4f94130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:55.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.336+0000 7f4f9c3f9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f9406f680 0x7f4f9406fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:55.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.336+0000 7f4f9c3f9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4f9406ffd0 0x7f4f94137530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:55.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.336+0000 7f4f9c3f9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f9409a350 con 0x7f4f9406ffd0 2023-10-03T10:18:55.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.336+0000 7f4f9c3f9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f94130260 con 0x7f4f9406f680 2023-10-03T10:18:55.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.336+0000 7f4f9c3f9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f94071ff0 con 0x7f4f94130670 2023-10-03T10:18:55.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.337+0000 7f4f9a996700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4f9406ffd0 0x7f4f94137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:55.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.337+0000 7f4f9a195700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4f94130670 0x7f4f94130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:55.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.337+0000 7f4f9a996700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4f9406ffd0 0x7f4f94137530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49588/0 (socket says 172.21.15.3:49588) 2023-10-03T10:18:55.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.337+0000 7f4f9a996700 1 -- 172.21.15.3:0/2559315962 learned_addr learned my addr 172.21.15.3:0/2559315962 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:55.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.337+0000 7f4f99994700 1 --2- 172.21.15.3:0/2559315962 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f9406f680 0x7f4f9406fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:55.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.337+0000 7f4f9a195700 1 -- 172.21.15.3:0/2559315962 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f9406f680 msgr2=0x7f4f9406fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:55.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.337+0000 7f4f9a195700 1 --2- 172.21.15.3:0/2559315962 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f9406f680 0x7f4f9406fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.337+0000 7f4f9a195700 1 -- 172.21.15.3:0/2559315962 >> v2:172.21.15.3:3300/0 conn(0x7f4f9406ffd0 msgr2=0x7f4f94137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:55.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.337+0000 7f4f9a195700 1 --2- 172.21.15.3:0/2559315962 >> v2:172.21.15.3:3300/0 conn(0x7f4f9406ffd0 0x7f4f94137530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.337+0000 7f4f9a195700 1 -- 172.21.15.3:0/2559315962 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f94070470 con 0x7f4f94130670 2023-10-03T10:18:55.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.337+0000 7f4f99994700 1 --2- 172.21.15.3:0/2559315962 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f9406f680 0x7f4f9406fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:18:55.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.337+0000 7f4f9a195700 1 --2- 172.21.15.3:0/2559315962 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4f94130670 0x7f4f94130a40 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009870 tx=0x7f4f90009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aec0b3139f597e94 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:55.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.337+0000 7f4f99193700 1 -- 172.21.15.3:0/2559315962 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f90011830 con 0x7f4f94130670 2023-10-03T10:18:55.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.338+0000 7f4f99193700 1 -- 172.21.15.3:0/2559315962 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f90011990 con 0x7f4f94130670 2023-10-03T10:18:55.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.338+0000 7f4f99193700 1 -- 172.21.15.3:0/2559315962 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f90011d20 con 0x7f4f94130670 2023-10-03T10:18:55.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.339+0000 7f4f9c3f9700 1 -- 172.21.15.3:0/2559315962 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4f94130670 msgr2=0x7f4f94130a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:55.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.339+0000 7f4f9c3f9700 1 --2- 172.21.15.3:0/2559315962 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4f94130670 0x7f4f94130a40 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009870 tx=0x7f4f90009b80 comp rx=0 tx=0).stop 2023-10-03T10:18:55.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.339+0000 7f4f9c3f9700 1 -- 172.21.15.3:0/2559315962 shutdown_connections 2023-10-03T10:18:55.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.339+0000 7f4f9c3f9700 1 --2- 172.21.15.3:0/2559315962 >> v2:172.21.15.3:3300/0 conn(0x7f4f9406ffd0 0x7f4f94137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.339+0000 7f4f9c3f9700 1 --2- 172.21.15.3:0/2559315962 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4f94130670 0x7f4f94130a40 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.339+0000 7f4f9c3f9700 1 --2- 172.21.15.3:0/2559315962 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f9406f680 0x7f4f9406fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.339+0000 7f4f9c3f9700 1 -- 172.21.15.3:0/2559315962 >> 172.21.15.3:0/2559315962 conn(0x7f4f940a2aa0 msgr2=0x7f4f940a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:55.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.339+0000 7f4f9c3f9700 1 -- 172.21.15.3:0/2559315962 shutdown_connections 2023-10-03T10:18:55.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.339+0000 7f4f9c3f9700 1 -- 172.21.15.3:0/2559315962 wait complete. 2023-10-03T10:18:55.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.340+0000 7f4f9c3f9700 1 Processor -- start 2023-10-03T10:18:55.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.340+0000 7f4f9c3f9700 1 -- start start 2023-10-03T10:18:55.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.340+0000 7f4f9c3f9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f9406f680 0x7f4f941c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:55.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.340+0000 7f4f9c3f9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f9406ffd0 0x7f4f941c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:55.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.340+0000 7f4f9c3f9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4f94130670 0x7f4f941c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:55.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.340+0000 7f4f9c3f9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f94071870 con 0x7f4f9406f680 2023-10-03T10:18:55.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.340+0000 7f4f9c3f9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9412e4e0 con 0x7f4f9406ffd0 2023-10-03T10:18:55.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.341+0000 7f4f9c3f9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9412c970 con 0x7f4f94130670 2023-10-03T10:18:55.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.341+0000 7f4f9a195700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f9406f680 0x7f4f941c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:55.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.341+0000 7f4f9a195700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f9406f680 0x7f4f941c7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49604/0 (socket says 172.21.15.3:49604) 2023-10-03T10:18:55.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.341+0000 7f4f9a195700 1 -- 172.21.15.3:0/3754015519 learned_addr learned my addr 172.21.15.3:0/3754015519 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:55.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.341+0000 7f4f9a996700 1 --2- 172.21.15.3:0/3754015519 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4f94130670 0x7f4f941c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:55.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.341+0000 7f4f9a195700 1 -- 172.21.15.3:0/3754015519 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4f94130670 msgr2=0x7f4f941c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:55.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.341+0000 7f4f9a195700 1 --2- 172.21.15.3:0/3754015519 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4f94130670 0x7f4f941c8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.341+0000 7f4f9a195700 1 -- 172.21.15.3:0/3754015519 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f9406ffd0 msgr2=0x7f4f941c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:55.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.341+0000 7f4f9a195700 1 --2- 172.21.15.3:0/3754015519 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f9406ffd0 0x7f4f941c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.341+0000 7f4f9a195700 1 -- 172.21.15.3:0/3754015519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f900094a0 con 0x7f4f9406f680 2023-10-03T10:18:55.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.341+0000 7f4f9a996700 1 --2- 172.21.15.3:0/3754015519 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4f94130670 0x7f4f941c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:18:55.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.341+0000 7f4f9a195700 1 --2- 172.21.15.3:0/3754015519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f9406f680 0x7f4f941c7fd0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f4f90000c00 tx=0x7f4f90004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:55.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.342+0000 7f4f8affd700 1 -- 172.21.15.3:0/3754015519 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f900036a0 con 0x7f4f9406f680 2023-10-03T10:18:55.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.342+0000 7f4f8affd700 1 -- 172.21.15.3:0/3754015519 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f90003800 con 0x7f4f9406f680 2023-10-03T10:18:55.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.342+0000 7f4f9c3f9700 1 -- 172.21.15.3:0/3754015519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f9412dc00 con 0x7f4f9406f680 2023-10-03T10:18:55.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.342+0000 7f4f8affd700 1 -- 172.21.15.3:0/3754015519 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f90003b70 con 0x7f4f9406f680 2023-10-03T10:18:55.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.342+0000 7f4f9c3f9700 1 -- 172.21.15.3:0/3754015519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f9412d6a0 con 0x7f4f9406f680 2023-10-03T10:18:55.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.343+0000 7f4f8affd700 1 -- 172.21.15.3:0/3754015519 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4f90003cd0 con 0x7f4f9406f680 2023-10-03T10:18:55.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.343+0000 7f4f9c3f9700 1 -- 172.21.15.3:0/3754015519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f94074ea0 con 0x7f4f9406f680 2023-10-03T10:18:55.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.343+0000 7f4f8affd700 1 -- 172.21.15.3:0/3754015519 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4f90040a90 con 0x7f4f9406f680 2023-10-03T10:18:55.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.352+0000 7f4f8affd700 1 -- 172.21.15.3:0/3754015519 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4f9002d3b0 con 0x7f4f9406f680 2023-10-03T10:18:55.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.386+0000 7fdd5b78f700 1 Processor -- start 2023-10-03T10:18:55.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.386+0000 7fdd5b78f700 1 -- start start 2023-10-03T10:18:55.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.387+0000 7fdd5b78f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdd540a0a30 0x7fdd540a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:55.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.387+0000 7fdd5b78f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdd5409d630 0x7fdd540a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:55.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.387+0000 7fdd5b78f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdd54133930 0x7fdd54133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:55.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.387+0000 7fdd5b78f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd540a2a70 con 0x7fdd5409d630 2023-10-03T10:18:55.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.387+0000 7fdd5b78f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd540a01d0 con 0x7fdd54133930 2023-10-03T10:18:55.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.387+0000 7fdd5b78f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5409e2b0 con 0x7fdd540a0a30 2023-10-03T10:18:55.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.387+0000 7fdd5952b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdd540a0a30 0x7fdd540a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:55.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.387+0000 7fdd59d2c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdd54133930 0x7fdd54133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:55.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.387+0000 7fdd59d2c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdd54133930 0x7fdd54133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42110/0 (socket says 172.21.15.3:42110) 2023-10-03T10:18:55.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.387+0000 7fdd59d2c700 1 -- 172.21.15.3:0/1923555358 learned_addr learned my addr 172.21.15.3:0/1923555358 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:55.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.387+0000 7fdd5952b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdd540a0a30 0x7fdd540a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47834/0 (socket says 172.21.15.3:47834) 2023-10-03T10:18:55.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.388+0000 7fdd58d2a700 1 --2- 172.21.15.3:0/1923555358 >> v2:172.21.15.3:3300/0 conn(0x7fdd5409d630 0x7fdd540a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:55.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.388+0000 7fdd59d2c700 1 -- 172.21.15.3:0/1923555358 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdd540a0a30 msgr2=0x7fdd540a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:55.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.388+0000 7fdd59d2c700 1 --2- 172.21.15.3:0/1923555358 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdd540a0a30 0x7fdd540a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.388+0000 7fdd59d2c700 1 -- 172.21.15.3:0/1923555358 >> v2:172.21.15.3:3300/0 conn(0x7fdd5409d630 msgr2=0x7fdd540a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:55.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.388+0000 7fdd59d2c700 1 --2- 172.21.15.3:0/1923555358 >> v2:172.21.15.3:3300/0 conn(0x7fdd5409d630 0x7fdd540a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.388+0000 7fdd59d2c700 1 -- 172.21.15.3:0/1923555358 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5409dc50 con 0x7fdd54133930 2023-10-03T10:18:55.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.388+0000 7fdd59d2c700 1 --2- 172.21.15.3:0/1923555358 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdd54133930 0x7fdd54133dd0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009e50 tx=0x7fdd4c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31e012389418e44a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:55.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.388+0000 7fdd4bfff700 1 -- 172.21.15.3:0/1923555358 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c00e070 con 0x7fdd54133930 2023-10-03T10:18:55.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.388+0000 7fdd4bfff700 1 -- 172.21.15.3:0/1923555358 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c00b910 con 0x7fdd54133930 2023-10-03T10:18:55.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.388+0000 7fdd4bfff700 1 -- 172.21.15.3:0/1923555358 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c00bcc0 con 0x7fdd54133930 2023-10-03T10:18:55.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.389+0000 7fdd5b78f700 1 -- 172.21.15.3:0/1923555358 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdd54133930 msgr2=0x7fdd54133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:55.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.389+0000 7fdd5b78f700 1 --2- 172.21.15.3:0/1923555358 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdd54133930 0x7fdd54133dd0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009e50 tx=0x7fdd4c00b330 comp rx=0 tx=0).stop 2023-10-03T10:18:55.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.389+0000 7fdd5b78f700 1 -- 172.21.15.3:0/1923555358 shutdown_connections 2023-10-03T10:18:55.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.389+0000 7fdd5b78f700 1 --2- 172.21.15.3:0/1923555358 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdd54133930 0x7fdd54133dd0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.389+0000 7fdd5b78f700 1 --2- 172.21.15.3:0/1923555358 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdd540a0a30 0x7fdd540a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.389+0000 7fdd5b78f700 1 --2- 172.21.15.3:0/1923555358 >> v2:172.21.15.3:3300/0 conn(0x7fdd5409d630 0x7fdd540a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.389+0000 7fdd5b78f700 1 -- 172.21.15.3:0/1923555358 >> 172.21.15.3:0/1923555358 conn(0x7fdd540a3050 msgr2=0x7fdd540a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:55.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.389+0000 7fdd5b78f700 1 -- 172.21.15.3:0/1923555358 shutdown_connections 2023-10-03T10:18:55.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.390+0000 7fdd5b78f700 1 -- 172.21.15.3:0/1923555358 wait complete. 2023-10-03T10:18:55.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.390+0000 7fdd5b78f700 1 Processor -- start 2023-10-03T10:18:55.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.391+0000 7fdd5b78f700 1 -- start start 2023-10-03T10:18:55.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.391+0000 7fdd5b78f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdd5409d630 0x7fdd541c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:55.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.391+0000 7fdd5b78f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdd540a0a30 0x7fdd541c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:55.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.391+0000 7fdd5b78f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd54133930 0x7fdd541c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:55.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.391+0000 7fdd5b78f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd54133530 con 0x7fdd54133930 2023-10-03T10:18:55.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.391+0000 7fdd5b78f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd54121f20 con 0x7fdd5409d630 2023-10-03T10:18:55.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.391+0000 7fdd5b78f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdd54120730 con 0x7fdd540a0a30 2023-10-03T10:18:55.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.391+0000 7fdd59d2c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd54133930 0x7fdd541c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:55.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.391+0000 7fdd5952b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdd5409d630 0x7fdd541c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:55.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.391+0000 7fdd59d2c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd54133930 0x7fdd541c8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49632/0 (socket says 172.21.15.3:49632) 2023-10-03T10:18:55.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.391+0000 7fdd5952b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdd5409d630 0x7fdd541c7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42112/0 (socket says 172.21.15.3:42112) 2023-10-03T10:18:55.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.391+0000 7fdd59d2c700 1 -- 172.21.15.3:0/1090794031 learned_addr learned my addr 172.21.15.3:0/1090794031 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:55.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.392+0000 7fdd58d2a700 1 --2- 172.21.15.3:0/1090794031 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdd540a0a30 0x7fdd541c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:55.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.392+0000 7fdd59d2c700 1 -- 172.21.15.3:0/1090794031 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdd540a0a30 msgr2=0x7fdd541c84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:55.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.392+0000 7fdd59d2c700 1 --2- 172.21.15.3:0/1090794031 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdd540a0a30 0x7fdd541c84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.392+0000 7fdd59d2c700 1 -- 172.21.15.3:0/1090794031 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdd5409d630 msgr2=0x7fdd541c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:55.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.392+0000 7fdd59d2c700 1 --2- 172.21.15.3:0/1090794031 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdd5409d630 0x7fdd541c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.392+0000 7fdd59d2c700 1 -- 172.21.15.3:0/1090794031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4c009b30 con 0x7fdd54133930 2023-10-03T10:18:55.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.392+0000 7fdd59d2c700 1 --2- 172.21.15.3:0/1090794031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd54133930 0x7fdd541c8a00 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c00b680 tx=0x7fdd4c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:55.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.392+0000 7fdd49ffb700 1 -- 172.21.15.3:0/1090794031 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c004450 con 0x7fdd54133930 2023-10-03T10:18:55.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.392+0000 7fdd49ffb700 1 -- 172.21.15.3:0/1090794031 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c0045d0 con 0x7fdd54133930 2023-10-03T10:18:55.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.392+0000 7fdd49ffb700 1 -- 172.21.15.3:0/1090794031 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c00ce80 con 0x7fdd54133930 2023-10-03T10:18:55.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.393+0000 7fdd5b78f700 1 -- 172.21.15.3:0/1090794031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd540a5d70 con 0x7fdd54133930 2023-10-03T10:18:55.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.393+0000 7fdd5b78f700 1 -- 172.21.15.3:0/1090794031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd54122c50 con 0x7fdd54133930 2023-10-03T10:18:55.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.394+0000 7fdd49ffb700 1 -- 172.21.15.3:0/1090794031 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdd4c025460 con 0x7fdd54133930 2023-10-03T10:18:55.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.394+0000 7fdd49ffb700 1 -- 172.21.15.3:0/1090794031 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fdd4c041250 con 0x7fdd54133930 2023-10-03T10:18:55.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.394+0000 7fdd5b78f700 1 -- 172.21.15.3:0/1090794031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd540747f0 con 0x7fdd54133930 2023-10-03T10:18:55.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.399+0000 7fdd49ffb700 1 -- 172.21.15.3:0/1090794031 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdd4c01a7d0 con 0x7fdd54133930 2023-10-03T10:18:55.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.423+0000 7fe9eba36700 1 Processor -- start 2023-10-03T10:18:55.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.423+0000 7fe9eba36700 1 -- start start 2023-10-03T10:18:55.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.424+0000 7fe9eba36700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9e41338a0 0x7fe9e4133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:55.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.424+0000 7fe9eba36700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9e409de50 0x7fe9e409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:55.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.424+0000 7fe9eba36700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe9e409e760 0x7fe9e413a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:55.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.424+0000 7fe9eba36700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9e406fef0 con 0x7fe9e409e760 2023-10-03T10:18:55.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.424+0000 7fe9eba36700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e406ef30 con 0x7fe9e409de50 2023-10-03T10:18:55.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.424+0000 7fe9eba36700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e41334b0 con 0x7fe9e41338a0 2023-10-03T10:18:55.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.424+0000 7fe9e97d2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9e41338a0 0x7fe9e4133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:55.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.424+0000 7fe9e8fd1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9e409de50 0x7fe9e409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:55.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.424+0000 7fe9e97d2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9e41338a0 0x7fe9e4133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47862/0 (socket says 172.21.15.3:47862) 2023-10-03T10:18:55.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.424+0000 7fe9e97d2700 1 -- 172.21.15.3:0/2095426120 learned_addr learned my addr 172.21.15.3:0/2095426120 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:55.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.424+0000 7fe9e8fd1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9e409de50 0x7fe9e409e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42128/0 (socket says 172.21.15.3:42128) 2023-10-03T10:18:55.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.424+0000 7fe9e9fd3700 1 --2- 172.21.15.3:0/2095426120 >> v2:172.21.15.3:3300/0 conn(0x7fe9e409e760 0x7fe9e413a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:55.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.424+0000 7fe9e8fd1700 1 -- 172.21.15.3:0/2095426120 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9e41338a0 msgr2=0x7fe9e4133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:55.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.424+0000 7fe9e8fd1700 1 --2- 172.21.15.3:0/2095426120 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9e41338a0 0x7fe9e4133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.424+0000 7fe9e8fd1700 1 -- 172.21.15.3:0/2095426120 >> v2:172.21.15.3:3300/0 conn(0x7fe9e409e760 msgr2=0x7fe9e413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:55.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.424+0000 7fe9e8fd1700 1 --2- 172.21.15.3:0/2095426120 >> v2:172.21.15.3:3300/0 conn(0x7fe9e409e760 0x7fe9e413a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.424+0000 7fe9e8fd1700 1 -- 172.21.15.3:0/2095426120 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e406fc30 con 0x7fe9e409de50 2023-10-03T10:18:55.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.424+0000 7fe9e9fd3700 1 --2- 172.21.15.3:0/2095426120 >> v2:172.21.15.3:3300/0 conn(0x7fe9e409e760 0x7fe9e413a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:18:55.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.425+0000 7fe9e8fd1700 1 --2- 172.21.15.3:0/2095426120 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9e409de50 0x7fe9e409e220 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4009870 tx=0x7fe9d4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea598ddc917e99c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:55.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.425+0000 7fe9dbfff700 1 -- 172.21.15.3:0/2095426120 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4011830 con 0x7fe9e409de50 2023-10-03T10:18:55.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.425+0000 7fe9dbfff700 1 -- 172.21.15.3:0/2095426120 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d4011990 con 0x7fe9e409de50 2023-10-03T10:18:55.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.425+0000 7fe9dbfff700 1 -- 172.21.15.3:0/2095426120 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4011d20 con 0x7fe9e409de50 2023-10-03T10:18:55.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.426+0000 7fe9eba36700 1 -- 172.21.15.3:0/2095426120 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9e409de50 msgr2=0x7fe9e409e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:55.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.426+0000 7fe9eba36700 1 --2- 172.21.15.3:0/2095426120 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9e409de50 0x7fe9e409e220 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4009870 tx=0x7fe9d4009b80 comp rx=0 tx=0).stop 2023-10-03T10:18:55.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.426+0000 7fe9eba36700 1 -- 172.21.15.3:0/2095426120 shutdown_connections 2023-10-03T10:18:55.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.426+0000 7fe9eba36700 1 --2- 172.21.15.3:0/2095426120 >> v2:172.21.15.3:3300/0 conn(0x7fe9e409e760 0x7fe9e413a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.426+0000 7fe9eba36700 1 --2- 172.21.15.3:0/2095426120 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9e41338a0 0x7fe9e4133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.426+0000 7fe9eba36700 1 --2- 172.21.15.3:0/2095426120 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9e409de50 0x7fe9e409e220 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.426+0000 7fe9eba36700 1 -- 172.21.15.3:0/2095426120 >> 172.21.15.3:0/2095426120 conn(0x7fe9e40a2aa0 msgr2=0x7fe9e40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:55.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.426+0000 7fe9eba36700 1 -- 172.21.15.3:0/2095426120 shutdown_connections 2023-10-03T10:18:55.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.426+0000 7fe9eba36700 1 -- 172.21.15.3:0/2095426120 wait complete. 2023-10-03T10:18:55.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.427+0000 7fe9eba36700 1 Processor -- start 2023-10-03T10:18:55.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.427+0000 7fe9eba36700 1 -- start start 2023-10-03T10:18:55.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.428+0000 7fe9eba36700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9e409de50 0x7fe9e411ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:55.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.428+0000 7fe9eba36700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9e409e760 0x7fe9e411efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:55.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.428+0000 7fe9eba36700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9e41338a0 0x7fe9e411f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:55.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.428+0000 7fe9eba36700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e412fa60 con 0x7fe9e409e760 2023-10-03T10:18:55.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.428+0000 7fe9eba36700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e4122200 con 0x7fe9e409de50 2023-10-03T10:18:55.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.428+0000 7fe9eba36700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e40a0fe0 con 0x7fe9e41338a0 2023-10-03T10:18:55.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.428+0000 7fe9e97d2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9e409de50 0x7fe9e411ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:55.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.428+0000 7fe9e97d2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9e409de50 0x7fe9e411ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42130/0 (socket says 172.21.15.3:42130) 2023-10-03T10:18:55.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.428+0000 7fe9e97d2700 1 -- 172.21.15.3:0/535262682 learned_addr learned my addr 172.21.15.3:0/535262682 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:55.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.428+0000 7fe9e8fd1700 1 --2- 172.21.15.3:0/535262682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9e409e760 0x7fe9e411efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:55.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.428+0000 7fe9e9fd3700 1 --2- 172.21.15.3:0/535262682 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9e41338a0 0x7fe9e411f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:55.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.428+0000 7fe9e8fd1700 1 -- 172.21.15.3:0/535262682 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9e41338a0 msgr2=0x7fe9e411f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:55.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.428+0000 7fe9e8fd1700 1 --2- 172.21.15.3:0/535262682 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9e41338a0 0x7fe9e411f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.428+0000 7fe9e8fd1700 1 -- 172.21.15.3:0/535262682 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9e409de50 msgr2=0x7fe9e411ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:55.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.428+0000 7fe9e8fd1700 1 --2- 172.21.15.3:0/535262682 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9e409de50 0x7fe9e411ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.428+0000 7fe9e8fd1700 1 -- 172.21.15.3:0/535262682 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d40094a0 con 0x7fe9e409e760 2023-10-03T10:18:55.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.428+0000 7fe9e97d2700 1 --2- 172.21.15.3:0/535262682 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9e409de50 0x7fe9e411ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:55.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.429+0000 7fe9e8fd1700 1 --2- 172.21.15.3:0/535262682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9e409e760 0x7fe9e411efd0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4016000 tx=0x7fe9d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:55.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.429+0000 7fe9d9ffb700 1 -- 172.21.15.3:0/535262682 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4004450 con 0x7fe9e409e760 2023-10-03T10:18:55.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.429+0000 7fe9d9ffb700 1 -- 172.21.15.3:0/535262682 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d401e070 con 0x7fe9e409e760 2023-10-03T10:18:55.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.429+0000 7fe9d9ffb700 1 -- 172.21.15.3:0/535262682 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d40038e0 con 0x7fe9e409e760 2023-10-03T10:18:55.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.429+0000 7fe9eba36700 1 -- 172.21.15.3:0/535262682 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e411fed0 con 0x7fe9e409e760 2023-10-03T10:18:55.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.429+0000 7fe9eba36700 1 -- 172.21.15.3:0/535262682 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e41206c0 con 0x7fe9e409e760 2023-10-03T10:18:55.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.430+0000 7fe9d9ffb700 1 -- 172.21.15.3:0/535262682 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe9d4003a40 con 0x7fe9e409e760 2023-10-03T10:18:55.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.430+0000 7fe9d9ffb700 1 -- 172.21.15.3:0/535262682 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe9d4040340 con 0x7fe9e409e760 2023-10-03T10:18:55.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.430+0000 7fe9eba36700 1 -- 172.21.15.3:0/535262682 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9e4121f70 con 0x7fe9e409e760 2023-10-03T10:18:55.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.440+0000 7fe9d9ffb700 1 -- 172.21.15.3:0/535262682 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe9d4015080 con 0x7fe9e409e760 2023-10-03T10:18:55.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.519+0000 7f791144d700 1 -- 172.21.15.3:0/1765638737 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f790c132960 con 0x7f790c09e1d0 2023-10-03T10:18:55.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.520+0000 7f78f37fe700 1 -- 172.21.15.3:0/1765638737 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7900024350 con 0x7f790c09e1d0 2023-10-03T10:18:55.520 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.522+0000 7f791144d700 1 -- 172.21.15.3:0/1765638737 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f790c09e1d0 msgr2=0x7f790c135190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:55.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.522+0000 7f791144d700 1 --2- 172.21.15.3:0/1765638737 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f790c09e1d0 0x7f790c135190 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f790000ebe0 tx=0x7f790000eef0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.522+0000 7f791144d700 1 -- 172.21.15.3:0/1765638737 shutdown_connections 2023-10-03T10:18:55.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.522+0000 7f791144d700 1 --2- 172.21.15.3:0/1765638737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f790c135e50 0x7f790c1356d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.522+0000 7f791144d700 1 --2- 172.21.15.3:0/1765638737 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f790c09d6f0 0x7f790c134c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.522+0000 7f791144d700 1 --2- 172.21.15.3:0/1765638737 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f790c09e1d0 0x7f790c135190 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.522+0000 7f791144d700 1 -- 172.21.15.3:0/1765638737 >> 172.21.15.3:0/1765638737 conn(0x7f790c066f50 msgr2=0x7f790c06e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:55.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.522+0000 7f791144d700 1 -- 172.21.15.3:0/1765638737 shutdown_connections 2023-10-03T10:18:55.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.522+0000 7f791144d700 1 -- 172.21.15.3:0/1765638737 wait complete. 2023-10-03T10:18:55.537 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:55.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.581+0000 7f4f9c3f9700 1 -- 172.21.15.3:0/3754015519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4f94072210 con 0x7f4f9406f680 2023-10-03T10:18:55.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.581+0000 7f4f8affd700 1 -- 172.21.15.3:0/3754015519 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4f90015030 con 0x7f4f9406f680 2023-10-03T10:18:55.582 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:55.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.583+0000 7f4f9c3f9700 1 -- 172.21.15.3:0/3754015519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f9406f680 msgr2=0x7f4f941c7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:55.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.583+0000 7f4f9c3f9700 1 --2- 172.21.15.3:0/3754015519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f9406f680 0x7f4f941c7fd0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f4f90000c00 tx=0x7f4f90004750 comp rx=0 tx=0).stop 2023-10-03T10:18:55.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.584+0000 7f4f9c3f9700 1 -- 172.21.15.3:0/3754015519 shutdown_connections 2023-10-03T10:18:55.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.584+0000 7f4f9c3f9700 1 --2- 172.21.15.3:0/3754015519 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4f94130670 0x7f4f941c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.584+0000 7f4f9c3f9700 1 --2- 172.21.15.3:0/3754015519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f9406f680 0x7f4f941c7fd0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.584+0000 7f4f9c3f9700 1 --2- 172.21.15.3:0/3754015519 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4f9406ffd0 0x7f4f941c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.584+0000 7f4f9c3f9700 1 -- 172.21.15.3:0/3754015519 >> 172.21.15.3:0/3754015519 conn(0x7f4f940a2aa0 msgr2=0x7f4f9406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:55.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.584+0000 7f4f9c3f9700 1 -- 172.21.15.3:0/3754015519 shutdown_connections 2023-10-03T10:18:55.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.584+0000 7f4f9c3f9700 1 -- 172.21.15.3:0/3754015519 wait complete. 2023-10-03T10:18:55.598 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:55.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.621+0000 7fdd5b78f700 1 -- 172.21.15.3:0/1090794031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fdd54072980 con 0x7fdd54133930 2023-10-03T10:18:55.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.621+0000 7fdd49ffb700 1 -- 172.21.15.3:0/1090794031 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdd4c02a030 con 0x7fdd54133930 2023-10-03T10:18:55.622 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:55.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.624+0000 7fdd5b78f700 1 -- 172.21.15.3:0/1090794031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd54133930 msgr2=0x7fdd541c8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:55.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.624+0000 7fdd5b78f700 1 --2- 172.21.15.3:0/1090794031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd54133930 0x7fdd541c8a00 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c00b680 tx=0x7fdd4c004ad0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.624+0000 7fdd5b78f700 1 -- 172.21.15.3:0/1090794031 shutdown_connections 2023-10-03T10:18:55.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.624+0000 7fdd5b78f700 1 --2- 172.21.15.3:0/1090794031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd54133930 0x7fdd541c8a00 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.624+0000 7fdd5b78f700 1 --2- 172.21.15.3:0/1090794031 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdd5409d630 0x7fdd541c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.624+0000 7fdd5b78f700 1 --2- 172.21.15.3:0/1090794031 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdd540a0a30 0x7fdd541c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.624+0000 7fdd5b78f700 1 -- 172.21.15.3:0/1090794031 >> 172.21.15.3:0/1090794031 conn(0x7fdd540a3050 msgr2=0x7fdd5409ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:55.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.624+0000 7fdd5b78f700 1 -- 172.21.15.3:0/1090794031 shutdown_connections 2023-10-03T10:18:55.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.624+0000 7fdd5b78f700 1 -- 172.21.15.3:0/1090794031 wait complete. 2023-10-03T10:18:55.638 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:55.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.661+0000 7fe9eba36700 1 -- 172.21.15.3:0/535262682 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe9e4122d30 con 0x7fe9e409e760 2023-10-03T10:18:55.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.661+0000 7fe9d9ffb700 1 -- 172.21.15.3:0/535262682 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe9d402d450 con 0x7fe9e409e760 2023-10-03T10:18:55.661 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:55.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.663+0000 7fe9eba36700 1 -- 172.21.15.3:0/535262682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9e409e760 msgr2=0x7fe9e411efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:55.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.663+0000 7fe9eba36700 1 --2- 172.21.15.3:0/535262682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9e409e760 0x7fe9e411efd0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4016000 tx=0x7fe9d4004750 comp rx=0 tx=0).stop 2023-10-03T10:18:55.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.663+0000 7fe9eba36700 1 -- 172.21.15.3:0/535262682 shutdown_connections 2023-10-03T10:18:55.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.663+0000 7fe9eba36700 1 --2- 172.21.15.3:0/535262682 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe9e41338a0 0x7fe9e411f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.663+0000 7fe9eba36700 1 --2- 172.21.15.3:0/535262682 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe9e409de50 0x7fe9e411ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.663+0000 7fe9eba36700 1 --2- 172.21.15.3:0/535262682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9e409e760 0x7fe9e411efd0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:55.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.663+0000 7fe9eba36700 1 -- 172.21.15.3:0/535262682 >> 172.21.15.3:0/535262682 conn(0x7fe9e40a2aa0 msgr2=0x7fe9e409ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:55.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.663+0000 7fe9eba36700 1 -- 172.21.15.3:0/535262682 shutdown_connections 2023-10-03T10:18:55.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:55.663+0000 7fe9eba36700 1 -- 172.21.15.3:0/535262682 wait complete. 2023-10-03T10:18:55.677 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:56.538 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:56.599 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:56.638 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:56.678 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:56.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.688+0000 7feb8cb42700 1 Processor -- start 2023-10-03T10:18:56.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.688+0000 7feb8cb42700 1 -- start start 2023-10-03T10:18:56.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.689+0000 7feb8cb42700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feb88136710 0x7feb88138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:56.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.689+0000 7feb8cb42700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feb88139030 0x7feb8813b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:56.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.689+0000 7feb8cb42700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb8806ef30 con 0x7feb881338a0 2023-10-03T10:18:56.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.689+0000 7feb8cb42700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb8806fef0 con 0x7feb88136710 2023-10-03T10:18:56.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.689+0000 7feb8cb42700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb881334b0 con 0x7feb88139030 2023-10-03T10:18:56.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.689+0000 7feb85d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feb88136710 0x7feb88138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:56.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.689+0000 7feb85d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feb88136710 0x7feb88138af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49650/0 (socket says 172.21.15.3:49650) 2023-10-03T10:18:56.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.689+0000 7feb86d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feb88139030 0x7feb8813b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:56.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.689+0000 7feb85d9b700 1 -- 172.21.15.3:0/683458826 learned_addr learned my addr 172.21.15.3:0/683458826 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:56.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.689+0000 7feb85d9b700 1 -- 172.21.15.3:0/683458826 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feb88139030 msgr2=0x7feb8813b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:56.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.689+0000 7feb85d9b700 1 --2- 172.21.15.3:0/683458826 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feb88139030 0x7feb8813b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:56.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.689+0000 7feb85d9b700 1 -- 172.21.15.3:0/683458826 >> v1:172.21.15.3:6789/0 conn(0x7feb881338a0 legacy=0x7feb88133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:56.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.689+0000 7feb85d9b700 1 -- 172.21.15.3:0/683458826 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb8806fc30 con 0x7feb88136710 2023-10-03T10:18:56.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.690+0000 7feb85d9b700 1 --2- 172.21.15.3:0/683458826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb88136710 0x7feb88138af0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7feb7c0098f0 tx=0x7feb7c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6238872dd18508a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:56.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.690+0000 7feb8559a700 1 -- 172.21.15.3:0/683458826 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7c004230 con 0x7feb88136710 2023-10-03T10:18:56.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.690+0000 7feb8559a700 1 -- 172.21.15.3:0/683458826 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb7c004390 con 0x7feb88136710 2023-10-03T10:18:56.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.690+0000 7feb8559a700 1 -- 172.21.15.3:0/683458826 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7c0036a0 con 0x7feb88136710 2023-10-03T10:18:56.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.691+0000 7feb8cb42700 1 -- 172.21.15.3:0/683458826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb88136710 msgr2=0x7feb88138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:56.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.691+0000 7feb8cb42700 1 --2- 172.21.15.3:0/683458826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb88136710 0x7feb88138af0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7feb7c0098f0 tx=0x7feb7c009c00 comp rx=0 tx=0).stop 2023-10-03T10:18:56.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.691+0000 7feb8cb42700 1 -- 172.21.15.3:0/683458826 shutdown_connections 2023-10-03T10:18:56.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.691+0000 7feb8cb42700 1 --2- 172.21.15.3:0/683458826 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feb88139030 0x7feb8813b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:56.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.691+0000 7feb8cb42700 1 --2- 172.21.15.3:0/683458826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb88136710 0x7feb88138af0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:56.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.691+0000 7feb8cb42700 1 -- 172.21.15.3:0/683458826 >> 172.21.15.3:0/683458826 conn(0x7feb880a2aa0 msgr2=0x7feb880a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:56.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.691+0000 7feb8cb42700 1 -- 172.21.15.3:0/683458826 shutdown_connections 2023-10-03T10:18:56.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.691+0000 7feb8cb42700 1 -- 172.21.15.3:0/683458826 wait complete. 2023-10-03T10:18:56.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.692+0000 7feb8cb42700 1 Processor -- start 2023-10-03T10:18:56.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.692+0000 7feb8cb42700 1 -- start start 2023-10-03T10:18:56.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.693+0000 7feb8cb42700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feb881338a0 0x7feb8809e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:56.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.693+0000 7feb8cb42700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb88136710 0x7feb8809e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:56.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.693+0000 7feb8cb42700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feb88139030 0x7feb881cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:56.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.693+0000 7feb8cb42700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb88122a60 con 0x7feb88136710 2023-10-03T10:18:56.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.693+0000 7feb8cb42700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb88130a80 con 0x7feb88139030 2023-10-03T10:18:56.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.693+0000 7feb8cb42700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feb8809deb0 con 0x7feb881338a0 2023-10-03T10:18:56.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.693+0000 7feb85d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb88136710 0x7feb8809e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:56.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.693+0000 7feb85d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb88136710 0x7feb8809e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49664/0 (socket says 172.21.15.3:49664) 2023-10-03T10:18:56.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.693+0000 7feb85d9b700 1 -- 172.21.15.3:0/1917997579 learned_addr learned my addr 172.21.15.3:0/1917997579 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:56.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.693+0000 7feb8659c700 1 --2- 172.21.15.3:0/1917997579 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feb881338a0 0x7feb8809e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:56.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.693+0000 7feb86d9d700 1 --2- 172.21.15.3:0/1917997579 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feb88139030 0x7feb881cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:56.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.693+0000 7feb85d9b700 1 -- 172.21.15.3:0/1917997579 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feb881338a0 msgr2=0x7feb8809e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:56.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.694+0000 7feb85d9b700 1 --2- 172.21.15.3:0/1917997579 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feb881338a0 0x7feb8809e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:56.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.694+0000 7feb85d9b700 1 -- 172.21.15.3:0/1917997579 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feb88139030 msgr2=0x7feb881cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:56.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.694+0000 7feb85d9b700 1 --2- 172.21.15.3:0/1917997579 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feb88139030 0x7feb881cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:56.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.694+0000 7feb85d9b700 1 -- 172.21.15.3:0/1917997579 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7c009550 con 0x7feb88136710 2023-10-03T10:18:56.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.694+0000 7feb8659c700 1 --2- 172.21.15.3:0/1917997579 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feb881338a0 0x7feb8809e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:18:56.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.694+0000 7feb85d9b700 1 --2- 172.21.15.3:0/1917997579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb88136710 0x7feb8809e990 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009130 tx=0x7feb7c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:56.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.694+0000 7feb76ffd700 1 -- 172.21.15.3:0/1917997579 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7c003c80 con 0x7feb88136710 2023-10-03T10:18:56.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.694+0000 7feb76ffd700 1 -- 172.21.15.3:0/1917997579 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb7c01f070 con 0x7feb88136710 2023-10-03T10:18:56.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.694+0000 7feb8cb42700 1 -- 172.21.15.3:0/1917997579 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb88138b00 con 0x7feb88136710 2023-10-03T10:18:56.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.694+0000 7feb76ffd700 1 -- 172.21.15.3:0/1917997579 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7c011630 con 0x7feb88136710 2023-10-03T10:18:56.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.694+0000 7feb8cb42700 1 -- 172.21.15.3:0/1917997579 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb8809f4f0 con 0x7feb88136710 2023-10-03T10:18:56.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.695+0000 7feb76ffd700 1 -- 172.21.15.3:0/1917997579 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feb7c021030 con 0x7feb88136710 2023-10-03T10:18:56.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.695+0000 7feb76ffd700 1 -- 172.21.15.3:0/1917997579 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7feb7c040c50 con 0x7feb88136710 2023-10-03T10:18:56.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.695+0000 7feb8cb42700 1 -- 172.21.15.3:0/1917997579 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb88074ea0 con 0x7feb88136710 2023-10-03T10:18:56.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.701+0000 7feb76ffd700 1 -- 172.21.15.3:0/1917997579 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feb7c019990 con 0x7feb88136710 2023-10-03T10:18:56.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.748+0000 7f4ed90c1700 1 Processor -- start 2023-10-03T10:18:56.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.748+0000 7f4ed90c1700 1 -- start start 2023-10-03T10:18:56.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.749+0000 7f4ed90c1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4ed40a0a30 0x7f4ed40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:56.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.749+0000 7f4ed90c1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4ed40a1340 0x7f4ed4133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:56.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.749+0000 7f4ed90c1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4ed4133fa0 0x7f4ed4134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:56.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.749+0000 7f4ed90c1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ed409edd0 con 0x7f4ed40a0a30 2023-10-03T10:18:56.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.749+0000 7f4ed90c1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed409e1f0 con 0x7f4ed4133fa0 2023-10-03T10:18:56.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.749+0000 7f4ed90c1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed409dc50 con 0x7f4ed40a1340 2023-10-03T10:18:56.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.749+0000 7f4ed2d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4ed40a0a30 0x7f4ed40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:56.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.749+0000 7f4ed2d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4ed40a0a30 0x7f4ed40a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49676/0 (socket says 172.21.15.3:49676) 2023-10-03T10:18:56.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.749+0000 7f4ed2d9d700 1 -- 172.21.15.3:0/1639788545 learned_addr learned my addr 172.21.15.3:0/1639788545 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:56.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.749+0000 7f4ed359e700 1 --2- 172.21.15.3:0/1639788545 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4ed4133fa0 0x7f4ed4134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:56.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.749+0000 7f4ecbfff700 1 --2- 172.21.15.3:0/1639788545 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4ed40a1340 0x7f4ed4133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:56.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.749+0000 7f4ed359e700 1 -- 172.21.15.3:0/1639788545 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4ed40a1340 msgr2=0x7f4ed4133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:56.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.749+0000 7f4ed359e700 1 --2- 172.21.15.3:0/1639788545 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4ed40a1340 0x7f4ed4133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:56.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.749+0000 7f4ed359e700 1 -- 172.21.15.3:0/1639788545 >> v2:172.21.15.3:3300/0 conn(0x7f4ed40a0a30 msgr2=0x7f4ed40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:56.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.749+0000 7f4ed359e700 1 --2- 172.21.15.3:0/1639788545 >> v2:172.21.15.3:3300/0 conn(0x7f4ed40a0a30 0x7f4ed40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:56.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.750+0000 7f4ed359e700 1 -- 172.21.15.3:0/1639788545 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed40a1fe0 con 0x7f4ed4133fa0 2023-10-03T10:18:56.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.750+0000 7f4ed359e700 1 --2- 172.21.15.3:0/1639788545 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4ed4133fa0 0x7f4ed4134410 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc009dc0 tx=0x7f4ecc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1338e7c92d0f3078 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:56.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.750+0000 7f4ed259c700 1 -- 172.21.15.3:0/1639788545 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc00e070 con 0x7f4ed4133fa0 2023-10-03T10:18:56.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.750+0000 7f4ed259c700 1 -- 172.21.15.3:0/1639788545 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ecc00bae0 con 0x7f4ed4133fa0 2023-10-03T10:18:56.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.750+0000 7f4ed259c700 1 -- 172.21.15.3:0/1639788545 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc00be90 con 0x7f4ed4133fa0 2023-10-03T10:18:56.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.751+0000 7f4ed90c1700 1 -- 172.21.15.3:0/1639788545 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4ed4133fa0 msgr2=0x7f4ed4134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:56.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.751+0000 7f4ed90c1700 1 --2- 172.21.15.3:0/1639788545 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4ed4133fa0 0x7f4ed4134410 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc009dc0 tx=0x7f4ecc00b330 comp rx=0 tx=0).stop 2023-10-03T10:18:56.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.751+0000 7f4ed90c1700 1 -- 172.21.15.3:0/1639788545 shutdown_connections 2023-10-03T10:18:56.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.751+0000 7f4ed90c1700 1 --2- 172.21.15.3:0/1639788545 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4ed4133fa0 0x7f4ed4134410 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:56.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.751+0000 7f4ed90c1700 1 --2- 172.21.15.3:0/1639788545 >> v2:172.21.15.3:3300/0 conn(0x7f4ed40a0a30 0x7f4ed40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:56.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.751+0000 7f4ed90c1700 1 --2- 172.21.15.3:0/1639788545 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4ed40a1340 0x7f4ed4133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:56.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.751+0000 7f4ed90c1700 1 -- 172.21.15.3:0/1639788545 >> 172.21.15.3:0/1639788545 conn(0x7f4ed40a6550 msgr2=0x7f4ed40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:56.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.751+0000 7f4ed90c1700 1 -- 172.21.15.3:0/1639788545 shutdown_connections 2023-10-03T10:18:56.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.751+0000 7f4ed90c1700 1 -- 172.21.15.3:0/1639788545 wait complete. 2023-10-03T10:18:56.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.752+0000 7f4ed90c1700 1 Processor -- start 2023-10-03T10:18:56.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.752+0000 7f4ed90c1700 1 -- start start 2023-10-03T10:18:56.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.753+0000 7f4ed90c1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ed40a0a30 0x7f4ed4120180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:56.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.753+0000 7f4ed90c1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4ed40a1340 0x7f4ed41206c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:56.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.753+0000 7f4ed90c1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4ed4133fa0 0x7f4ed4120c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:56.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.753+0000 7f4ed90c1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed4122f40 con 0x7f4ed40a0a30 2023-10-03T10:18:56.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.753+0000 7f4ed90c1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed4123920 con 0x7f4ed40a1340 2023-10-03T10:18:56.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.753+0000 7f4ed90c1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed4121e00 con 0x7f4ed4133fa0 2023-10-03T10:18:56.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.753+0000 7f4ecbfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4ed40a1340 0x7f4ed41206c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:56.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.753+0000 7f4ecbfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4ed40a1340 0x7f4ed41206c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42174/0 (socket says 172.21.15.3:42174) 2023-10-03T10:18:56.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.753+0000 7f4ecbfff700 1 -- 172.21.15.3:0/2931828643 learned_addr learned my addr 172.21.15.3:0/2931828643 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:56.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.753+0000 7f4ed359e700 1 --2- 172.21.15.3:0/2931828643 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4ed4133fa0 0x7f4ed4120c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:56.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.753+0000 7f4ed2d9d700 1 --2- 172.21.15.3:0/2931828643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ed40a0a30 0x7f4ed4120180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:56.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.753+0000 7f4ecbfff700 1 -- 172.21.15.3:0/2931828643 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4ed4133fa0 msgr2=0x7f4ed4120c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:56.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.753+0000 7f4ecbfff700 1 --2- 172.21.15.3:0/2931828643 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4ed4133fa0 0x7f4ed4120c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:56.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.753+0000 7f4ecbfff700 1 -- 172.21.15.3:0/2931828643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ed40a0a30 msgr2=0x7f4ed4120180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:56.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.753+0000 7f4ecbfff700 1 --2- 172.21.15.3:0/2931828643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ed40a0a30 0x7f4ed4120180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:56.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.753+0000 7f4ecbfff700 1 -- 172.21.15.3:0/2931828643 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ecc009a70 con 0x7f4ed40a1340 2023-10-03T10:18:56.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.754+0000 7f4ecbfff700 1 --2- 172.21.15.3:0/2931828643 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4ed40a1340 0x7f4ed41206c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00e810 tx=0x7f4ebc00eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:56.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.754+0000 7f4ecb7fe700 1 -- 172.21.15.3:0/2931828643 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc01f440 con 0x7f4ed40a1340 2023-10-03T10:18:56.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.754+0000 7f4ecb7fe700 1 -- 172.21.15.3:0/2931828643 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ebc01f5a0 con 0x7f4ed40a1340 2023-10-03T10:18:56.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.754+0000 7f4ecb7fe700 1 -- 172.21.15.3:0/2931828643 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc01f950 con 0x7f4ed40a1340 2023-10-03T10:18:56.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.754+0000 7f4ed90c1700 1 -- 172.21.15.3:0/2931828643 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed40a3430 con 0x7f4ed40a1340 2023-10-03T10:18:56.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.754+0000 7f4ed90c1700 1 -- 172.21.15.3:0/2931828643 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed4123c00 con 0x7f4ed40a1340 2023-10-03T10:18:56.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.755+0000 7f4ed90c1700 1 -- 172.21.15.3:0/2931828643 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ed4074ea0 con 0x7f4ed40a1340 2023-10-03T10:18:56.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.755+0000 7f4ecb7fe700 1 -- 172.21.15.3:0/2931828643 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4ebc01fab0 con 0x7f4ed40a1340 2023-10-03T10:18:56.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.755+0000 7f4ecb7fe700 1 -- 172.21.15.3:0/2931828643 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4ebc0474c0 con 0x7f4ed40a1340 2023-10-03T10:18:56.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.760+0000 7f4ecb7fe700 1 -- 172.21.15.3:0/2931828643 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4ebc01fd60 con 0x7f4ed40a1340 2023-10-03T10:18:56.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.824+0000 7f0f49c72700 1 Processor -- start 2023-10-03T10:18:56.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.824+0000 7f0f49c72700 1 -- start start 2023-10-03T10:18:56.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.825+0000 7f0f49c72700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f4409de00 0x7f0f4409e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:56.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.825+0000 7f0f49c72700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f4413d1f0 0x7f0f4413f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:56.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.825+0000 7f0f49c72700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f4409a810 con 0x7f0f4406a770 2023-10-03T10:18:56.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.825+0000 7f0f49c72700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4406ab40 con 0x7f0f4409de00 2023-10-03T10:18:56.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.825+0000 7f0f49c72700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4409ebc0 con 0x7f0f4413d1f0 2023-10-03T10:18:56.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.825+0000 7f0f42ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f4409de00 0x7f0f4409e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:56.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.825+0000 7f0f42ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f4409de00 0x7f0f4409e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42178/0 (socket says 172.21.15.3:42178) 2023-10-03T10:18:56.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.825+0000 7f0f42ffd700 1 -- 172.21.15.3:0/601651348 learned_addr learned my addr 172.21.15.3:0/601651348 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:56.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.825+0000 7f0f42ffd700 1 -- 172.21.15.3:0/601651348 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f4413d1f0 msgr2=0x7f0f4413f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:56.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.825+0000 7f0f42ffd700 1 --2- 172.21.15.3:0/601651348 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f4413d1f0 0x7f0f4413f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:56.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.826+0000 7f0f42ffd700 1 -- 172.21.15.3:0/601651348 >> v1:172.21.15.3:6789/0 conn(0x7f0f4406a770 legacy=0x7f0f4409d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:56.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.826+0000 7f0f42ffd700 1 -- 172.21.15.3:0/601651348 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4409e920 con 0x7f0f4409de00 2023-10-03T10:18:56.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.826+0000 7f0f42ffd700 1 --2- 172.21.15.3:0/601651348 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f4409de00 0x7f0f4409e250 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0f38009920 tx=0x7f0f38009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28eb55d240f1f1f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:56.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.826+0000 7f0f427fc700 1 -- 172.21.15.3:0/601651348 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3800ba60 con 0x7f0f4409de00 2023-10-03T10:18:56.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.826+0000 7f0f427fc700 1 -- 172.21.15.3:0/601651348 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f3800bbc0 con 0x7f0f4409de00 2023-10-03T10:18:56.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.826+0000 7f0f427fc700 1 -- 172.21.15.3:0/601651348 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f38009e70 con 0x7f0f4409de00 2023-10-03T10:18:56.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.827+0000 7f0f49c72700 1 -- 172.21.15.3:0/601651348 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f4409de00 msgr2=0x7f0f4409e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:56.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.827+0000 7f0f49c72700 1 --2- 172.21.15.3:0/601651348 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f4409de00 0x7f0f4409e250 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0f38009920 tx=0x7f0f38009c30 comp rx=0 tx=0).stop 2023-10-03T10:18:56.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.827+0000 7f0f49c72700 1 -- 172.21.15.3:0/601651348 shutdown_connections 2023-10-03T10:18:56.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.827+0000 7f0f49c72700 1 --2- 172.21.15.3:0/601651348 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f4413d1f0 0x7f0f4413f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:56.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.827+0000 7f0f49c72700 1 --2- 172.21.15.3:0/601651348 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f4409de00 0x7f0f4409e250 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:56.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.827+0000 7f0f49c72700 1 -- 172.21.15.3:0/601651348 >> 172.21.15.3:0/601651348 conn(0x7f0f44066f50 msgr2=0x7f0f4409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:56.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.827+0000 7f0f49c72700 1 -- 172.21.15.3:0/601651348 shutdown_connections 2023-10-03T10:18:56.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.827+0000 7f0f49c72700 1 -- 172.21.15.3:0/601651348 wait complete. 2023-10-03T10:18:56.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.828+0000 7f0f49c72700 1 Processor -- start 2023-10-03T10:18:56.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.829+0000 7f0f49c72700 1 -- start start 2023-10-03T10:18:56.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.829+0000 7f0f49c72700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f4406a770 0x7f0f4413b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:56.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.829+0000 7f0f49c72700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f4409de00 0x7f0f4413bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:56.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.829+0000 7f0f49c72700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f4413d1f0 0x7f0f4413c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:56.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.829+0000 7f0f49c72700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4409bff0 con 0x7f0f4409de00 2023-10-03T10:18:56.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.829+0000 7f0f49c72700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f44071bf0 con 0x7f0f4406a770 2023-10-03T10:18:56.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.829+0000 7f0f49c72700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f44135f20 con 0x7f0f4413d1f0 2023-10-03T10:18:56.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.830+0000 7f0f437fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f4406a770 0x7f0f4413b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:56.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.830+0000 7f0f437fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f4406a770 0x7f0f4413b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42190/0 (socket says 172.21.15.3:42190) 2023-10-03T10:18:56.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.830+0000 7f0f437fe700 1 -- 172.21.15.3:0/3648205420 learned_addr learned my addr 172.21.15.3:0/3648205420 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:56.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.830+0000 7f0f42ffd700 1 --2- 172.21.15.3:0/3648205420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f4409de00 0x7f0f4413bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:56.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.830+0000 7f0f43fff700 1 --2- 172.21.15.3:0/3648205420 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f4413d1f0 0x7f0f4413c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:56.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.830+0000 7f0f437fe700 1 -- 172.21.15.3:0/3648205420 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f4413d1f0 msgr2=0x7f0f4413c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:56.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.830+0000 7f0f437fe700 1 --2- 172.21.15.3:0/3648205420 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f4413d1f0 0x7f0f4413c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:56.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.830+0000 7f0f437fe700 1 -- 172.21.15.3:0/3648205420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f4409de00 msgr2=0x7f0f4413bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:56.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.830+0000 7f0f437fe700 1 --2- 172.21.15.3:0/3648205420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f4409de00 0x7f0f4413bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:56.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.830+0000 7f0f437fe700 1 -- 172.21.15.3:0/3648205420 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f38009550 con 0x7f0f4406a770 2023-10-03T10:18:56.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.830+0000 7f0f437fe700 1 --2- 172.21.15.3:0/3648205420 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f4406a770 0x7f0f4413b850 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f0f3400c4e0 tx=0x7f0f3400c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:56.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.830+0000 7f0f2bfff700 1 -- 172.21.15.3:0/3648205420 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f340052d0 con 0x7f0f4406a770 2023-10-03T10:18:56.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.830+0000 7f0f49c72700 1 -- 172.21.15.3:0/3648205420 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f4411deb0 con 0x7f0f4406a770 2023-10-03T10:18:56.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.831+0000 7f0f2bfff700 1 -- 172.21.15.3:0/3648205420 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f34005430 con 0x7f0f4406a770 2023-10-03T10:18:56.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.831+0000 7f0f49c72700 1 -- 172.21.15.3:0/3648205420 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f4409f1a0 con 0x7f0f4406a770 2023-10-03T10:18:56.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.831+0000 7f0f2bfff700 1 -- 172.21.15.3:0/3648205420 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f340038c0 con 0x7f0f4406a770 2023-10-03T10:18:56.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.832+0000 7f0f2bfff700 1 -- 172.21.15.3:0/3648205420 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0f34003a20 con 0x7f0f4406a770 2023-10-03T10:18:56.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.832+0000 7f0f49c72700 1 -- 172.21.15.3:0/3648205420 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f44074ea0 con 0x7f0f4406a770 2023-10-03T10:18:56.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.832+0000 7f0f2bfff700 1 -- 172.21.15.3:0/3648205420 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0f3403ec90 con 0x7f0f4406a770 2023-10-03T10:18:56.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.835+0000 7f0195c1e700 1 Processor -- start 2023-10-03T10:18:56.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.835+0000 7f0195c1e700 1 -- start start 2023-10-03T10:18:56.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.836+0000 7f0195c1e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0190136710 0x7f0190138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:56.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.836+0000 7f0195c1e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0190139030 0x7f019013b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:56.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.836+0000 7f0195c1e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f019006ef30 con 0x7f01901338a0 2023-10-03T10:18:56.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.836+0000 7f0195c1e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f019006fef0 con 0x7f0190136710 2023-10-03T10:18:56.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.836+0000 7f0195c1e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01901334b0 con 0x7f0190139030 2023-10-03T10:18:56.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.836+0000 7f018ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0190139030 0x7f019013b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:56.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.836+0000 7f018ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0190139030 0x7f019013b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42194/0 (socket says 172.21.15.3:42194) 2023-10-03T10:18:56.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.836+0000 7f018ffff700 1 -- 172.21.15.3:0/757767733 learned_addr learned my addr 172.21.15.3:0/757767733 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:56.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.836+0000 7f018effd700 1 --2- 172.21.15.3:0/757767733 >> v2:172.21.15.3:3300/0 conn(0x7f0190136710 0x7f0190138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:56.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.837+0000 7f018ffff700 1 -- 172.21.15.3:0/757767733 >> v2:172.21.15.3:3300/0 conn(0x7f0190136710 msgr2=0x7f0190138af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:56.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.837+0000 7f018ffff700 1 --2- 172.21.15.3:0/757767733 >> v2:172.21.15.3:3300/0 conn(0x7f0190136710 0x7f0190138af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:56.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.837+0000 7f018ffff700 1 -- 172.21.15.3:0/757767733 >> v1:172.21.15.3:6789/0 conn(0x7f01901338a0 legacy=0x7f0190133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:56.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.837+0000 7f018ffff700 1 -- 172.21.15.3:0/757767733 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019006fc30 con 0x7f0190139030 2023-10-03T10:18:56.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.837+0000 7f018ffff700 1 --2- 172.21.15.3:0/757767733 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0190139030 0x7f019013b410 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f0188009e50 tx=0x7f018800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2934f24e014ca21d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:56.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.841+0000 7f018e7fc700 1 -- 172.21.15.3:0/757767733 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018800e070 con 0x7f0190139030 2023-10-03T10:18:56.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.841+0000 7f018e7fc700 1 -- 172.21.15.3:0/757767733 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f018800b9c0 con 0x7f0190139030 2023-10-03T10:18:56.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.841+0000 7f018e7fc700 1 -- 172.21.15.3:0/757767733 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018800bd70 con 0x7f0190139030 2023-10-03T10:18:56.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.841+0000 7f0f2bfff700 1 -- 172.21.15.3:0/3648205420 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0f34017070 con 0x7f0f4406a770 2023-10-03T10:18:56.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.842+0000 7f0195c1e700 1 -- 172.21.15.3:0/757767733 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0190139030 msgr2=0x7f019013b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:56.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.842+0000 7f0195c1e700 1 --2- 172.21.15.3:0/757767733 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0190139030 0x7f019013b410 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f0188009e50 tx=0x7f018800b330 comp rx=0 tx=0).stop 2023-10-03T10:18:56.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.842+0000 7f0195c1e700 1 -- 172.21.15.3:0/757767733 shutdown_connections 2023-10-03T10:18:56.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.842+0000 7f0195c1e700 1 --2- 172.21.15.3:0/757767733 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0190139030 0x7f019013b410 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:56.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.842+0000 7f0195c1e700 1 --2- 172.21.15.3:0/757767733 >> v2:172.21.15.3:3300/0 conn(0x7f0190136710 0x7f0190138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:56.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.842+0000 7f0195c1e700 1 -- 172.21.15.3:0/757767733 >> 172.21.15.3:0/757767733 conn(0x7f01900a2aa0 msgr2=0x7f01900a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:56.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.842+0000 7f0195c1e700 1 -- 172.21.15.3:0/757767733 shutdown_connections 2023-10-03T10:18:56.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.842+0000 7f0195c1e700 1 -- 172.21.15.3:0/757767733 wait complete. 2023-10-03T10:18:56.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.843+0000 7f0195c1e700 1 Processor -- start 2023-10-03T10:18:56.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.844+0000 7f0195c1e700 1 -- start start 2023-10-03T10:18:56.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.844+0000 7f0195c1e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01901338a0 0x7f01901c8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:56.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.844+0000 7f0195c1e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0190136710 0x7f01901c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:56.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.844+0000 7f0195c1e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0190139030 0x7f01901c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:56.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.844+0000 7f0195c1e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01900a2440 con 0x7f01901338a0 2023-10-03T10:18:56.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.844+0000 7f0195c1e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f019006f640 con 0x7f0190139030 2023-10-03T10:18:56.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.844+0000 7f0195c1e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f019011cc80 con 0x7f0190136710 2023-10-03T10:18:56.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.845+0000 7f018effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0190136710 0x7f01901c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:56.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.845+0000 7f018ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0190139030 0x7f01901c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:56.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.845+0000 7f018effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0190136710 0x7f01901c8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47926/0 (socket says 172.21.15.3:47926) 2023-10-03T10:18:56.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.845+0000 7f018effd700 1 -- 172.21.15.3:0/616026293 learned_addr learned my addr 172.21.15.3:0/616026293 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:56.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.845+0000 7f018f7fe700 1 --2- 172.21.15.3:0/616026293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01901338a0 0x7f01901c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:56.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.845+0000 7f018effd700 1 -- 172.21.15.3:0/616026293 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0190139030 msgr2=0x7f01901c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:56.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.845+0000 7f018effd700 1 --2- 172.21.15.3:0/616026293 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0190139030 0x7f01901c8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:56.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.845+0000 7f018effd700 1 -- 172.21.15.3:0/616026293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01901338a0 msgr2=0x7f01901c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:56.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.845+0000 7f018effd700 1 --2- 172.21.15.3:0/616026293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01901338a0 0x7f01901c8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:56.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.845+0000 7f018effd700 1 -- 172.21.15.3:0/616026293 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0188009b30 con 0x7f0190136710 2023-10-03T10:18:56.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.846+0000 7f018effd700 1 --2- 172.21.15.3:0/616026293 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0190136710 0x7f01901c8860 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f018400e840 tx=0x7f018400eb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:56.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.846+0000 7f0177fff700 1 -- 172.21.15.3:0/616026293 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0184013610 con 0x7f0190136710 2023-10-03T10:18:56.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.846+0000 7f0177fff700 1 -- 172.21.15.3:0/616026293 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0184013770 con 0x7f0190136710 2023-10-03T10:18:56.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.846+0000 7f0195c1e700 1 -- 172.21.15.3:0/616026293 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f019011e4e0 con 0x7f0190136710 2023-10-03T10:18:56.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.846+0000 7f0177fff700 1 -- 172.21.15.3:0/616026293 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0184013b20 con 0x7f0190136710 2023-10-03T10:18:56.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.846+0000 7f0195c1e700 1 -- 172.21.15.3:0/616026293 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f019011d7e0 con 0x7f0190136710 2023-10-03T10:18:56.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.847+0000 7f0177fff700 1 -- 172.21.15.3:0/616026293 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0184013c80 con 0x7f0190136710 2023-10-03T10:18:56.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.847+0000 7f0177fff700 1 -- 172.21.15.3:0/616026293 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f018403f400 con 0x7f0190136710 2023-10-03T10:18:56.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.853+0000 7f0195c1e700 1 -- 172.21.15.3:0/616026293 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f019011e830 con 0x7f0190136710 2023-10-03T10:18:56.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.858+0000 7f0177fff700 1 -- 172.21.15.3:0/616026293 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0184012370 con 0x7f0190136710 2023-10-03T10:18:56.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.963+0000 7feb8cb42700 1 -- 172.21.15.3:0/1917997579 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7feb88074ea0 con 0x7feb88136710 2023-10-03T10:18:56.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.964+0000 7feb76ffd700 1 -- 172.21.15.3:0/1917997579 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feb7c01b440 con 0x7feb88136710 2023-10-03T10:18:56.964 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:56.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.966+0000 7feb8cb42700 1 -- 172.21.15.3:0/1917997579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb88136710 msgr2=0x7feb8809e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:56.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.966+0000 7feb8cb42700 1 --2- 172.21.15.3:0/1917997579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb88136710 0x7feb8809e990 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009130 tx=0x7feb7c004750 comp rx=0 tx=0).stop 2023-10-03T10:18:56.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.966+0000 7feb8cb42700 1 -- 172.21.15.3:0/1917997579 shutdown_connections 2023-10-03T10:18:56.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.966+0000 7feb8cb42700 1 --2- 172.21.15.3:0/1917997579 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feb88139030 0x7feb881cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:56.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.966+0000 7feb8cb42700 1 --2- 172.21.15.3:0/1917997579 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7feb881338a0 0x7feb8809e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:56.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.966+0000 7feb8cb42700 1 --2- 172.21.15.3:0/1917997579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb88136710 0x7feb8809e990 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:56.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.966+0000 7feb8cb42700 1 -- 172.21.15.3:0/1917997579 >> 172.21.15.3:0/1917997579 conn(0x7feb880a2aa0 msgr2=0x7feb881372b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:56.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.967+0000 7feb8cb42700 1 -- 172.21.15.3:0/1917997579 shutdown_connections 2023-10-03T10:18:56.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.967+0000 7feb8cb42700 1 -- 172.21.15.3:0/1917997579 wait complete. 2023-10-03T10:18:56.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.980+0000 7f4ed90c1700 1 -- 172.21.15.3:0/2931828643 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4ed4123a60 con 0x7f4ed40a1340 2023-10-03T10:18:56.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.980+0000 7f4ecb7fe700 1 -- 172.21.15.3:0/2931828643 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4ed4123a60 con 0x7f4ed40a1340 2023-10-03T10:18:56.980 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:56.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.983+0000 7f4ed90c1700 1 -- 172.21.15.3:0/2931828643 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4ed40a1340 msgr2=0x7f4ed41206c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:56.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.983+0000 7f4ed90c1700 1 --2- 172.21.15.3:0/2931828643 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4ed40a1340 0x7f4ed41206c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00e810 tx=0x7f4ebc00eb20 comp rx=0 tx=0).stop 2023-10-03T10:18:56.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.983+0000 7f4ed90c1700 1 -- 172.21.15.3:0/2931828643 shutdown_connections 2023-10-03T10:18:56.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.983+0000 7f4ed90c1700 1 --2- 172.21.15.3:0/2931828643 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4ed4133fa0 0x7f4ed4120c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:56.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.983+0000 7f4ed90c1700 1 --2- 172.21.15.3:0/2931828643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ed40a0a30 0x7f4ed4120180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:56.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.983+0000 7f4ed90c1700 1 --2- 172.21.15.3:0/2931828643 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4ed40a1340 0x7f4ed41206c0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:56.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.983+0000 7f4ed90c1700 1 -- 172.21.15.3:0/2931828643 >> 172.21.15.3:0/2931828643 conn(0x7f4ed40a6550 msgr2=0x7f4ed409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:56.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.984+0000 7f4ed90c1700 1 -- 172.21.15.3:0/2931828643 shutdown_connections 2023-10-03T10:18:56.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:56.984+0000 7f4ed90c1700 1 -- 172.21.15.3:0/2931828643 wait complete. 2023-10-03T10:18:56.986 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:56.998 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:57.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:57.077+0000 7f0f49c72700 1 -- 172.21.15.3:0/3648205420 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0f4411e520 con 0x7f0f4406a770 2023-10-03T10:18:57.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:57.077+0000 7f0f2bfff700 1 -- 172.21.15.3:0/3648205420 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0f34019d20 con 0x7f0f4406a770 2023-10-03T10:18:57.078 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:57.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:57.081+0000 7f0f49c72700 1 -- 172.21.15.3:0/3648205420 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f4406a770 msgr2=0x7f0f4413b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:57.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:57.081+0000 7f0f49c72700 1 --2- 172.21.15.3:0/3648205420 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f4406a770 0x7f0f4413b850 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f0f3400c4e0 tx=0x7f0f3400c7f0 comp rx=0 tx=0).stop 2023-10-03T10:18:57.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:57.081+0000 7f0f49c72700 1 -- 172.21.15.3:0/3648205420 shutdown_connections 2023-10-03T10:18:57.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:57.081+0000 7f0f49c72700 1 --2- 172.21.15.3:0/3648205420 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f4413d1f0 0x7f0f4413c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:57.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:57.081+0000 7f0f49c72700 1 --2- 172.21.15.3:0/3648205420 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f4406a770 0x7f0f4413b850 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:57.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:57.081+0000 7f0f49c72700 1 --2- 172.21.15.3:0/3648205420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f4409de00 0x7f0f4413bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:57.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:57.081+0000 7f0f49c72700 1 -- 172.21.15.3:0/3648205420 >> 172.21.15.3:0/3648205420 conn(0x7f0f44066f50 msgr2=0x7f0f440a2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:57.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:57.081+0000 7f0f49c72700 1 -- 172.21.15.3:0/3648205420 shutdown_connections 2023-10-03T10:18:57.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:57.081+0000 7f0f49c72700 1 -- 172.21.15.3:0/3648205420 wait complete. 2023-10-03T10:18:57.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:57.093+0000 7f0195c1e700 1 -- 172.21.15.3:0/616026293 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f019006ea80 con 0x7f0190136710 2023-10-03T10:18:57.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:57.094+0000 7f0177fff700 1 -- 172.21.15.3:0/616026293 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0184010310 con 0x7f0190136710 2023-10-03T10:18:57.094 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:57.095 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:57.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:57.096+0000 7f0195c1e700 1 -- 172.21.15.3:0/616026293 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0190136710 msgr2=0x7f01901c8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:57.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:57.096+0000 7f0195c1e700 1 --2- 172.21.15.3:0/616026293 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0190136710 0x7f01901c8860 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f018400e840 tx=0x7f018400eb50 comp rx=0 tx=0).stop 2023-10-03T10:18:57.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:57.096+0000 7f0195c1e700 1 -- 172.21.15.3:0/616026293 shutdown_connections 2023-10-03T10:18:57.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:57.096+0000 7f0195c1e700 1 --2- 172.21.15.3:0/616026293 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0190139030 0x7f01901c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:57.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:57.097+0000 7f0195c1e700 1 --2- 172.21.15.3:0/616026293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01901338a0 0x7f01901c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:57.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:57.097+0000 7f0195c1e700 1 --2- 172.21.15.3:0/616026293 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0190136710 0x7f01901c8860 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:57.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:57.097+0000 7f0195c1e700 1 -- 172.21.15.3:0/616026293 >> 172.21.15.3:0/616026293 conn(0x7f01900a2aa0 msgr2=0x7f01901344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:57.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:57.097+0000 7f0195c1e700 1 -- 172.21.15.3:0/616026293 shutdown_connections 2023-10-03T10:18:57.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:57.097+0000 7f0195c1e700 1 -- 172.21.15.3:0/616026293 wait complete. 2023-10-03T10:18:57.110 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:57.986 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:57.999 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:58.095 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:58.111 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:58.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.138+0000 7ff1f7767700 1 Processor -- start 2023-10-03T10:18:58.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.138+0000 7ff1f7767700 1 -- start start 2023-10-03T10:18:58.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.139+0000 7ff1f7767700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff1f009e1d0 0x7ff1f0138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:58.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.139+0000 7ff1f7767700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1f0139170 0x7ff1f013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:58.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.139+0000 7ff1f7767700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1f009b120 con 0x7ff1f009d6f0 2023-10-03T10:18:58.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.139+0000 7ff1f7767700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1f006b460 con 0x7ff1f009e1d0 2023-10-03T10:18:58.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.139+0000 7ff1f7767700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f009a7e0 con 0x7ff1f0139170 2023-10-03T10:18:58.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.139+0000 7ff1f5d04700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1f0139170 0x7ff1f013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:58.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.139+0000 7ff1f4d02700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff1f009e1d0 0x7ff1f0138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:58.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.139+0000 7ff1f5d04700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1f0139170 0x7ff1f013b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42208/0 (socket says 172.21.15.3:42208) 2023-10-03T10:18:58.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.139+0000 7ff1f5d04700 1 -- 172.21.15.3:0/305427867 learned_addr learned my addr 172.21.15.3:0/305427867 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:58.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.139+0000 7ff1f4d02700 1 --2- 172.21.15.3:0/305427867 >> v2:172.21.15.3:3300/0 conn(0x7ff1f009e1d0 0x7ff1f0138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49718/0 (socket says 172.21.15.3:49718) 2023-10-03T10:18:58.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.139+0000 7ff1f5d04700 1 -- 172.21.15.3:0/305427867 >> v2:172.21.15.3:3300/0 conn(0x7ff1f009e1d0 msgr2=0x7ff1f0138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:58.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.139+0000 7ff1f5d04700 1 --2- 172.21.15.3:0/305427867 >> v2:172.21.15.3:3300/0 conn(0x7ff1f009e1d0 0x7ff1f0138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.140+0000 7ff1f5d04700 1 -- 172.21.15.3:0/305427867 >> v1:172.21.15.3:6789/0 conn(0x7ff1f009d6f0 legacy=0x7ff1f009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:58.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.140+0000 7ff1f5d04700 1 -- 172.21.15.3:0/305427867 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f009a500 con 0x7ff1f0139170 2023-10-03T10:18:58.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.140+0000 7ff1f4d02700 1 --2- 172.21.15.3:0/305427867 >> v2:172.21.15.3:3300/0 conn(0x7ff1f009e1d0 0x7ff1f0138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:58.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.140+0000 7ff1f5d04700 1 --2- 172.21.15.3:0/305427867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1f0139170 0x7ff1f013b550 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009e80 tx=0x7ff1e800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94ce22757f3a7aa5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:58.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.140+0000 7ff1e7fff700 1 -- 172.21.15.3:0/305427867 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e800e070 con 0x7ff1f0139170 2023-10-03T10:18:58.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.140+0000 7ff1e7fff700 1 -- 172.21.15.3:0/305427867 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e800b8d0 con 0x7ff1f0139170 2023-10-03T10:18:58.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.140+0000 7ff1e7fff700 1 -- 172.21.15.3:0/305427867 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e800bc80 con 0x7ff1f0139170 2023-10-03T10:18:58.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.141+0000 7ff1f7767700 1 -- 172.21.15.3:0/305427867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1f0139170 msgr2=0x7ff1f013b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:58.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.141+0000 7ff1f7767700 1 --2- 172.21.15.3:0/305427867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1f0139170 0x7ff1f013b550 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009e80 tx=0x7ff1e800b330 comp rx=0 tx=0).stop 2023-10-03T10:18:58.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.141+0000 7ff1f7767700 1 -- 172.21.15.3:0/305427867 shutdown_connections 2023-10-03T10:18:58.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.141+0000 7ff1f7767700 1 --2- 172.21.15.3:0/305427867 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1f0139170 0x7ff1f013b550 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.141+0000 7ff1f7767700 1 --2- 172.21.15.3:0/305427867 >> v2:172.21.15.3:3300/0 conn(0x7ff1f009e1d0 0x7ff1f0138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.141+0000 7ff1f7767700 1 -- 172.21.15.3:0/305427867 >> 172.21.15.3:0/305427867 conn(0x7ff1f0066f50 msgr2=0x7ff1f009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:58.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.141+0000 7ff1f7767700 1 -- 172.21.15.3:0/305427867 shutdown_connections 2023-10-03T10:18:58.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.141+0000 7ff1f7767700 1 -- 172.21.15.3:0/305427867 wait complete. 2023-10-03T10:18:58.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.142+0000 7ff1f7767700 1 Processor -- start 2023-10-03T10:18:58.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.142+0000 7ff1f7767700 1 -- start start 2023-10-03T10:18:58.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.143+0000 7ff1f7767700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1f009d6f0 0x7ff1f01c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:58.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.143+0000 7ff1f7767700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1f009e1d0 0x7ff1f01c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:58.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.143+0000 7ff1f7767700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1f0139170 0x7ff1f011e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:58.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.143+0000 7ff1f7767700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f0122110 con 0x7ff1f009d6f0 2023-10-03T10:18:58.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.143+0000 7ff1f7767700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f0120400 con 0x7ff1f0139170 2023-10-03T10:18:58.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.143+0000 7ff1f7767700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f0121d70 con 0x7ff1f009e1d0 2023-10-03T10:18:58.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.143+0000 7ff1f5d04700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1f0139170 0x7ff1f011e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:58.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.143+0000 7ff1f5503700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1f009d6f0 0x7ff1f01c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:58.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.143+0000 7ff1f5d04700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1f0139170 0x7ff1f011e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42212/0 (socket says 172.21.15.3:42212) 2023-10-03T10:18:58.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.143+0000 7ff1f5d04700 1 -- 172.21.15.3:0/2879058566 learned_addr learned my addr 172.21.15.3:0/2879058566 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:58.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.143+0000 7ff1f4d02700 1 --2- 172.21.15.3:0/2879058566 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1f009e1d0 0x7ff1f01c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:58.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.143+0000 7ff1f5d04700 1 -- 172.21.15.3:0/2879058566 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1f009e1d0 msgr2=0x7ff1f01c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:58.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.143+0000 7ff1f5d04700 1 --2- 172.21.15.3:0/2879058566 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1f009e1d0 0x7ff1f01c8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.143+0000 7ff1f5d04700 1 -- 172.21.15.3:0/2879058566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1f009d6f0 msgr2=0x7ff1f01c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:58.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.143+0000 7ff1f5d04700 1 --2- 172.21.15.3:0/2879058566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1f009d6f0 0x7ff1f01c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.143+0000 7ff1f5d04700 1 -- 172.21.15.3:0/2879058566 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e00094a0 con 0x7ff1f0139170 2023-10-03T10:18:58.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.143+0000 7ff1f4d02700 1 --2- 172.21.15.3:0/2879058566 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1f009e1d0 0x7ff1f01c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:58.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.143+0000 7ff1f5503700 1 --2- 172.21.15.3:0/2879058566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1f009d6f0 0x7ff1f01c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:58.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.144+0000 7ff1f5d04700 1 --2- 172.21.15.3:0/2879058566 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1f0139170 0x7ff1f011e7f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8012000 tx=0x7ff1e8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:58.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.144+0000 7ff1e5ffb700 1 -- 172.21.15.3:0/2879058566 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8004800 con 0x7ff1f0139170 2023-10-03T10:18:58.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.144+0000 7ff1e5ffb700 1 -- 172.21.15.3:0/2879058566 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e8004960 con 0x7ff1f0139170 2023-10-03T10:18:58.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.144+0000 7ff1f7767700 1 -- 172.21.15.3:0/2879058566 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e8009b30 con 0x7ff1f0139170 2023-10-03T10:18:58.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.144+0000 7ff1e5ffb700 1 -- 172.21.15.3:0/2879058566 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e801c680 con 0x7ff1f0139170 2023-10-03T10:18:58.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.144+0000 7ff1f7767700 1 -- 172.21.15.3:0/2879058566 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f0122aa0 con 0x7ff1f0139170 2023-10-03T10:18:58.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.145+0000 7ff1f7767700 1 -- 172.21.15.3:0/2879058566 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1f0074ea0 con 0x7ff1f0139170 2023-10-03T10:18:58.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.146+0000 7ff1e5ffb700 1 -- 172.21.15.3:0/2879058566 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff1e801c7e0 con 0x7ff1f0139170 2023-10-03T10:18:58.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.146+0000 7ff1e5ffb700 1 -- 172.21.15.3:0/2879058566 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff1e80414d0 con 0x7ff1f0139170 2023-10-03T10:18:58.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.152+0000 7ff1e5ffb700 1 -- 172.21.15.3:0/2879058566 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff1e80243b0 con 0x7ff1f0139170 2023-10-03T10:18:58.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.177+0000 7fd3c71c0700 1 Processor -- start 2023-10-03T10:18:58.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.177+0000 7fd3c71c0700 1 -- start start 2023-10-03T10:18:58.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.178+0000 7fd3c71c0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3c009e1d0 0x7fd3c0138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:58.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.178+0000 7fd3c71c0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3c0139170 0x7fd3c013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:58.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.178+0000 7fd3c71c0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3c009b120 con 0x7fd3c009d6f0 2023-10-03T10:18:58.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.178+0000 7fd3c71c0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c006b460 con 0x7fd3c009e1d0 2023-10-03T10:18:58.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.178+0000 7fd3c71c0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c009a7e0 con 0x7fd3c0139170 2023-10-03T10:18:58.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.178+0000 7fd3c4f5c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd3c009d6f0 0x7fd3c009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33448/0 (socket says 172.21.15.3:33448) 2023-10-03T10:18:58.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.178+0000 7fd3c4f5c700 1 -- 172.21.15.3:0/3732148937 learned_addr learned my addr 172.21.15.3:0/3732148937 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:18:58.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.179+0000 7fd3bffff700 1 --2- 172.21.15.3:0/3732148937 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3c009e1d0 0x7fd3c0138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:58.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.179+0000 7fd3bf7fe700 1 -- 172.21.15.3:0/3732148937 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3044798128 0 0) 0x7fd3c009b120 con 0x7fd3c009d6f0 2023-10-03T10:18:58.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.179+0000 7fd3c575d700 1 --2- 172.21.15.3:0/3732148937 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3c0139170 0x7fd3c013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:58.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.179+0000 7fd3bf7fe700 1 -- 172.21.15.3:0/3732148937 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3b00035b0 con 0x7fd3c009d6f0 2023-10-03T10:18:58.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.179+0000 7fd3bffff700 1 -- 172.21.15.3:0/3732148937 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3c0139170 msgr2=0x7fd3c013b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:58.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.179+0000 7fd3bffff700 1 --2- 172.21.15.3:0/3732148937 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3c0139170 0x7fd3c013b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.179+0000 7fd3bffff700 1 -- 172.21.15.3:0/3732148937 >> v1:172.21.15.3:6789/0 conn(0x7fd3c009d6f0 legacy=0x7fd3c009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:58.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.179+0000 7fd3bf7fe700 1 -- 172.21.15.3:0/3732148937 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3538528567 0 0) 0x7fd3b00035b0 con 0x7fd3c009d6f0 2023-10-03T10:18:58.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.179+0000 7fd3bffff700 1 -- 172.21.15.3:0/3732148937 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c009a500 con 0x7fd3c009e1d0 2023-10-03T10:18:58.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.179+0000 7fd3bffff700 1 --2- 172.21.15.3:0/3732148937 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3c009e1d0 0x7fd3c0138c30 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009870 tx=0x7fd3ac009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6b520d6402cae3b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:58.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.180+0000 7fd3bf7fe700 1 -- 172.21.15.3:0/3732148937 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4001670 con 0x7fd3c009e1d0 2023-10-03T10:18:58.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.180+0000 7fd3bf7fe700 1 -- 172.21.15.3:0/3732148937 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ac004030 con 0x7fd3c009e1d0 2023-10-03T10:18:58.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.180+0000 7fd3bf7fe700 1 -- 172.21.15.3:0/3732148937 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac0043c0 con 0x7fd3c009e1d0 2023-10-03T10:18:58.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.181+0000 7fd3c71c0700 1 -- 172.21.15.3:0/3732148937 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3c009e1d0 msgr2=0x7fd3c0138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:58.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.181+0000 7fd3c71c0700 1 --2- 172.21.15.3:0/3732148937 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3c009e1d0 0x7fd3c0138c30 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009870 tx=0x7fd3ac009b80 comp rx=0 tx=0).stop 2023-10-03T10:18:58.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.181+0000 7fd3c71c0700 1 -- 172.21.15.3:0/3732148937 shutdown_connections 2023-10-03T10:18:58.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.181+0000 7fd3c71c0700 1 --2- 172.21.15.3:0/3732148937 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3c0139170 0x7fd3c013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.181+0000 7fd3c71c0700 1 --2- 172.21.15.3:0/3732148937 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3c009e1d0 0x7fd3c0138c30 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.181+0000 7fd3c71c0700 1 -- 172.21.15.3:0/3732148937 >> 172.21.15.3:0/3732148937 conn(0x7fd3c0066f50 msgr2=0x7fd3c009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:58.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.181+0000 7fd3c71c0700 1 -- 172.21.15.3:0/3732148937 shutdown_connections 2023-10-03T10:18:58.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.181+0000 7fd3c71c0700 1 -- 172.21.15.3:0/3732148937 wait complete. 2023-10-03T10:18:58.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.182+0000 7fd3c71c0700 1 Processor -- start 2023-10-03T10:18:58.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.182+0000 7fd3c71c0700 1 -- start start 2023-10-03T10:18:58.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.183+0000 7fd3c71c0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3c009d6f0 0x7fd3c01207f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:58.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.183+0000 7fd3c71c0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3c009e1d0 0x7fd3c0120d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:58.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.183+0000 7fd3c71c0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3c0139170 0x7fd3c0121270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:58.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.183+0000 7fd3c71c0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c0136100 con 0x7fd3c009e1d0 2023-10-03T10:18:58.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.183+0000 7fd3c71c0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c0122400 con 0x7fd3c009d6f0 2023-10-03T10:18:58.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.183+0000 7fd3c71c0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c01366e0 con 0x7fd3c0139170 2023-10-03T10:18:58.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.183+0000 7fd3bffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3c009e1d0 0x7fd3c0120d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:58.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.183+0000 7fd3bffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3c009e1d0 0x7fd3c0120d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49730/0 (socket says 172.21.15.3:49730) 2023-10-03T10:18:58.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.183+0000 7fd3bffff700 1 -- 172.21.15.3:0/951896310 learned_addr learned my addr 172.21.15.3:0/951896310 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:58.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.183+0000 7fd3c4f5c700 1 --2- 172.21.15.3:0/951896310 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3c009d6f0 0x7fd3c01207f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:58.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.183+0000 7fd3c575d700 1 --2- 172.21.15.3:0/951896310 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3c0139170 0x7fd3c0121270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:58.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.184+0000 7fd3bffff700 1 -- 172.21.15.3:0/951896310 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3c0139170 msgr2=0x7fd3c0121270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:58.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.184+0000 7fd3bffff700 1 --2- 172.21.15.3:0/951896310 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3c0139170 0x7fd3c0121270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.184+0000 7fd3bffff700 1 -- 172.21.15.3:0/951896310 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3c009d6f0 msgr2=0x7fd3c01207f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:58.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.184+0000 7fd3bffff700 1 --2- 172.21.15.3:0/951896310 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3c009d6f0 0x7fd3c01207f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.184+0000 7fd3bffff700 1 -- 172.21.15.3:0/951896310 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ac0094a0 con 0x7fd3c009e1d0 2023-10-03T10:18:58.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.184+0000 7fd3bffff700 1 --2- 172.21.15.3:0/951896310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3c009e1d0 0x7fd3c0120d30 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009ed0 tx=0x7fd3ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:58.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.184+0000 7fd3bd7fa700 1 -- 172.21.15.3:0/951896310 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac011630 con 0x7fd3c009e1d0 2023-10-03T10:18:58.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.184+0000 7fd3c71c0700 1 -- 172.21.15.3:0/951896310 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3c0122c60 con 0x7fd3c009e1d0 2023-10-03T10:18:58.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.184+0000 7fd3c71c0700 1 -- 172.21.15.3:0/951896310 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c00a3ad0 con 0x7fd3c009e1d0 2023-10-03T10:18:58.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.184+0000 7fd3bd7fa700 1 -- 172.21.15.3:0/951896310 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ac01f070 con 0x7fd3c009e1d0 2023-10-03T10:18:58.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.184+0000 7fd3bd7fa700 1 -- 172.21.15.3:0/951896310 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac011990 con 0x7fd3c009e1d0 2023-10-03T10:18:58.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.185+0000 7fd3c71c0700 1 -- 172.21.15.3:0/951896310 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3c0074ea0 con 0x7fd3c009e1d0 2023-10-03T10:18:58.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.186+0000 7fd3bd7fa700 1 -- 172.21.15.3:0/951896310 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd3ac011af0 con 0x7fd3c009e1d0 2023-10-03T10:18:58.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.186+0000 7fd3bd7fa700 1 -- 172.21.15.3:0/951896310 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd3ac021070 con 0x7fd3c009e1d0 2023-10-03T10:18:58.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.195+0000 7fd3bd7fa700 1 -- 172.21.15.3:0/951896310 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd3ac011da0 con 0x7fd3c009e1d0 2023-10-03T10:18:58.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.258+0000 7f21bb6da700 1 Processor -- start 2023-10-03T10:18:58.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.258+0000 7f21bb6da700 1 -- start start 2023-10-03T10:18:58.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.259+0000 7f21bb6da700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21b409d6f0 0x7f21b409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:58.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.259+0000 7f21bb6da700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21b409e000 0x7f21b41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:58.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.259+0000 7f21bb6da700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f21b409e410 0x7f21b4133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:58.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.259+0000 7f21bb6da700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21b406b460 con 0x7f21b409e410 2023-10-03T10:18:58.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.259+0000 7f21bb6da700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21b409b120 con 0x7f21b409d6f0 2023-10-03T10:18:58.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.259+0000 7f21bb6da700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f21b409a7e0 con 0x7f21b409e000 2023-10-03T10:18:58.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.259+0000 7f21b8c75700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21b409e000 0x7f21b41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:58.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.259+0000 7f21b8c75700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21b409e000 0x7f21b41339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47944/0 (socket says 172.21.15.3:47944) 2023-10-03T10:18:58.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.259+0000 7f21b8c75700 1 -- 172.21.15.3:0/3191596204 learned_addr learned my addr 172.21.15.3:0/3191596204 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:58.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.259+0000 7f21b9476700 1 --2- 172.21.15.3:0/3191596204 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21b409d6f0 0x7f21b409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:58.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.259+0000 7f21b9c77700 1 --2- 172.21.15.3:0/3191596204 >> v2:172.21.15.3:3300/0 conn(0x7f21b409e410 0x7f21b4133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:58.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.259+0000 7f21b8c75700 1 -- 172.21.15.3:0/3191596204 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21b409d6f0 msgr2=0x7f21b409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:58.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.259+0000 7f21b8c75700 1 --2- 172.21.15.3:0/3191596204 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21b409d6f0 0x7f21b409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.259+0000 7f21b8c75700 1 -- 172.21.15.3:0/3191596204 >> v2:172.21.15.3:3300/0 conn(0x7f21b409e410 msgr2=0x7f21b4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:58.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.259+0000 7f21b8c75700 1 --2- 172.21.15.3:0/3191596204 >> v2:172.21.15.3:3300/0 conn(0x7f21b409e410 0x7f21b4133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.259+0000 7f21b8c75700 1 -- 172.21.15.3:0/3191596204 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b409a500 con 0x7f21b409e000 2023-10-03T10:18:58.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.260+0000 7f21b8c75700 1 --2- 172.21.15.3:0/3191596204 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21b409e000 0x7f21b41339f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f21a4009920 tx=0x7f21a4009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d11c209bd6efd6d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:58.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.260+0000 7f21abfff700 1 -- 172.21.15.3:0/3191596204 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a4011830 con 0x7f21b409e000 2023-10-03T10:18:58.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.260+0000 7f21abfff700 1 -- 172.21.15.3:0/3191596204 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a4011990 con 0x7f21b409e000 2023-10-03T10:18:58.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.260+0000 7f21abfff700 1 -- 172.21.15.3:0/3191596204 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a4011d20 con 0x7f21b409e000 2023-10-03T10:18:58.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.261+0000 7f21bb6da700 1 -- 172.21.15.3:0/3191596204 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21b409e000 msgr2=0x7f21b41339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:58.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.261+0000 7f21bb6da700 1 --2- 172.21.15.3:0/3191596204 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21b409e000 0x7f21b41339f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f21a4009920 tx=0x7f21a4009c30 comp rx=0 tx=0).stop 2023-10-03T10:18:58.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.261+0000 7f21bb6da700 1 -- 172.21.15.3:0/3191596204 shutdown_connections 2023-10-03T10:18:58.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.261+0000 7f21bb6da700 1 --2- 172.21.15.3:0/3191596204 >> v2:172.21.15.3:3300/0 conn(0x7f21b409e410 0x7f21b4133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.261+0000 7f21bb6da700 1 --2- 172.21.15.3:0/3191596204 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21b409d6f0 0x7f21b409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.261+0000 7f21bb6da700 1 --2- 172.21.15.3:0/3191596204 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21b409e000 0x7f21b41339f0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.261+0000 7f21bb6da700 1 -- 172.21.15.3:0/3191596204 >> 172.21.15.3:0/3191596204 conn(0x7f21b4066f50 msgr2=0x7f21b409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:58.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.261+0000 7f21bb6da700 1 -- 172.21.15.3:0/3191596204 shutdown_connections 2023-10-03T10:18:58.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.261+0000 7f21bb6da700 1 -- 172.21.15.3:0/3191596204 wait complete. 2023-10-03T10:18:58.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.262+0000 7f21bb6da700 1 Processor -- start 2023-10-03T10:18:58.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.263+0000 7f21bb6da700 1 -- start start 2023-10-03T10:18:58.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.263+0000 7f21bb6da700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21b409d6f0 0x7f21b41a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:58.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.263+0000 7f21bb6da700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21b409e000 0x7f21b41a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:58.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.263+0000 7f21bb6da700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21b409e410 0x7f21b41a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:58.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.263+0000 7f21bb6da700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b41a9650 con 0x7f21b409e000 2023-10-03T10:18:58.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.263+0000 7f21bb6da700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21b41aa210 con 0x7f21b409d6f0 2023-10-03T10:18:58.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.263+0000 7f21bb6da700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f21b41a8c00 con 0x7f21b409e410 2023-10-03T10:18:58.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.263+0000 7f21b9c77700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21b409e410 0x7f21b41a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:58.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.263+0000 7f21b9c77700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21b409e410 0x7f21b41a67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47960/0 (socket says 172.21.15.3:47960) 2023-10-03T10:18:58.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.263+0000 7f21b9c77700 1 -- 172.21.15.3:0/37460080 learned_addr learned my addr 172.21.15.3:0/37460080 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:58.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.263+0000 7f21b9476700 1 --2- 172.21.15.3:0/37460080 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21b409d6f0 0x7f21b41a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:58.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.263+0000 7f21b8c75700 1 --2- 172.21.15.3:0/37460080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21b409e000 0x7f21b41a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:58.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.264+0000 7f21b9c77700 1 -- 172.21.15.3:0/37460080 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21b409d6f0 msgr2=0x7f21b41a5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:58.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.264+0000 7f21b9c77700 1 --2- 172.21.15.3:0/37460080 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21b409d6f0 0x7f21b41a5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.264+0000 7f21b9c77700 1 -- 172.21.15.3:0/37460080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21b409e000 msgr2=0x7f21b41a6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:58.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.264+0000 7f21b9c77700 1 --2- 172.21.15.3:0/37460080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21b409e000 0x7f21b41a6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.264+0000 7f21b9c77700 1 -- 172.21.15.3:0/37460080 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a4009550 con 0x7f21b409e410 2023-10-03T10:18:58.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.264+0000 7f21b9c77700 1 --2- 172.21.15.3:0/37460080 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21b409e410 0x7f21b41a67b0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f21ac007e90 tx=0x7f21ac00e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:58.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.264+0000 7f21a9ffb700 1 -- 172.21.15.3:0/37460080 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac00ed30 con 0x7f21b409e410 2023-10-03T10:18:58.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.264+0000 7f21a9ffb700 1 -- 172.21.15.3:0/37460080 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21ac00ee90 con 0x7f21b409e410 2023-10-03T10:18:58.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.264+0000 7f21a9ffb700 1 -- 172.21.15.3:0/37460080 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac014640 con 0x7f21b409e410 2023-10-03T10:18:58.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.265+0000 7f21bb6da700 1 -- 172.21.15.3:0/37460080 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21b41a9850 con 0x7f21b409e410 2023-10-03T10:18:58.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.265+0000 7f21bb6da700 1 -- 172.21.15.3:0/37460080 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b41a87a0 con 0x7f21b409e410 2023-10-03T10:18:58.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.265+0000 7f21a9ffb700 1 -- 172.21.15.3:0/37460080 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f21ac0147a0 con 0x7f21b409e410 2023-10-03T10:18:58.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.266+0000 7f21a9ffb700 1 -- 172.21.15.3:0/37460080 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f21ac03fba0 con 0x7f21b409e410 2023-10-03T10:18:58.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.266+0000 7f21bb6da700 1 -- 172.21.15.3:0/37460080 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21b41a8740 con 0x7f21b409e410 2023-10-03T10:18:58.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.271+0000 7f21a9ffb700 1 -- 172.21.15.3:0/37460080 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f21ac01b900 con 0x7f21b409e410 2023-10-03T10:18:58.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.287+0000 7f25048bb700 1 Processor -- start 2023-10-03T10:18:58.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.287+0000 7f25048bb700 1 -- start start 2023-10-03T10:18:58.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.288+0000 7f25048bb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f250009d6f0 0x7f250009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:58.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.288+0000 7f25048bb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f250009e000 0x7f2500130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:58.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.288+0000 7f25048bb700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f250009f130 con 0x7f250009e410 2023-10-03T10:18:58.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.288+0000 7f25048bb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f250009f520 con 0x7f250009e000 2023-10-03T10:18:58.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.288+0000 7f25048bb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f250009e930 con 0x7f250009d6f0 2023-10-03T10:18:58.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.288+0000 7f24fe59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f250009d6f0 0x7f250009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:58.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.288+0000 7f24fe59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f250009d6f0 0x7f250009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:47968/0 (socket says 172.21.15.3:47968) 2023-10-03T10:18:58.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.288+0000 7f24fe59c700 1 -- 172.21.15.3:0/2936203410 learned_addr learned my addr 172.21.15.3:0/2936203410 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:58.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.288+0000 7f24fdd9b700 1 --2- 172.21.15.3:0/2936203410 >> v2:172.21.15.3:3300/0 conn(0x7f250009e000 0x7f2500130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:58.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.288+0000 7f24fe59c700 1 -- 172.21.15.3:0/2936203410 >> v2:172.21.15.3:3300/0 conn(0x7f250009e000 msgr2=0x7f2500130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:58.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.288+0000 7f24fe59c700 1 --2- 172.21.15.3:0/2936203410 >> v2:172.21.15.3:3300/0 conn(0x7f250009e000 0x7f2500130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.288+0000 7f24fe59c700 1 -- 172.21.15.3:0/2936203410 >> v1:172.21.15.3:6789/0 conn(0x7f250009e410 legacy=0x7f2500130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:58.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.288+0000 7f24fe59c700 1 -- 172.21.15.3:0/2936203410 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f250009b2d0 con 0x7f250009d6f0 2023-10-03T10:18:58.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.289+0000 7f24fe59c700 1 --2- 172.21.15.3:0/2936203410 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f250009d6f0 0x7f250009dac0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f24f000bb90 tx=0x7f24f000bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12811ac5ea0563de server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:58.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.289+0000 7f24fd59a700 1 -- 172.21.15.3:0/2936203410 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f000dc30 con 0x7f250009d6f0 2023-10-03T10:18:58.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.289+0000 7f24fd59a700 1 -- 172.21.15.3:0/2936203410 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f000dd90 con 0x7f250009d6f0 2023-10-03T10:18:58.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.289+0000 7f24fd59a700 1 -- 172.21.15.3:0/2936203410 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f0004250 con 0x7f250009d6f0 2023-10-03T10:18:58.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.289+0000 7f25048bb700 1 -- 172.21.15.3:0/2936203410 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f250009d6f0 msgr2=0x7f250009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:58.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.289+0000 7f25048bb700 1 --2- 172.21.15.3:0/2936203410 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f250009d6f0 0x7f250009dac0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f24f000bb90 tx=0x7f24f000bea0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.290+0000 7f25048bb700 1 -- 172.21.15.3:0/2936203410 shutdown_connections 2023-10-03T10:18:58.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.290+0000 7f25048bb700 1 --2- 172.21.15.3:0/2936203410 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f250009d6f0 0x7f250009dac0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.290+0000 7f25048bb700 1 --2- 172.21.15.3:0/2936203410 >> v2:172.21.15.3:3300/0 conn(0x7f250009e000 0x7f2500130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.290+0000 7f25048bb700 1 -- 172.21.15.3:0/2936203410 >> 172.21.15.3:0/2936203410 conn(0x7f2500066f50 msgr2=0x7f250009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:58.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.290+0000 7f25048bb700 1 -- 172.21.15.3:0/2936203410 shutdown_connections 2023-10-03T10:18:58.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.290+0000 7f25048bb700 1 -- 172.21.15.3:0/2936203410 wait complete. 2023-10-03T10:18:58.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.291+0000 7f25048bb700 1 Processor -- start 2023-10-03T10:18:58.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.291+0000 7f25048bb700 1 -- start start 2023-10-03T10:18:58.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.291+0000 7f25048bb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f250009d6f0 0x7f250011e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:58.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.291+0000 7f25048bb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f250009e000 0x7f250011e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:58.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.291+0000 7f25048bb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f250009e410 0x7f250011edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:58.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.291+0000 7f25048bb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25001217f0 con 0x7f250009d6f0 2023-10-03T10:18:58.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.291+0000 7f25048bb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25000a1d90 con 0x7f250009e000 2023-10-03T10:18:58.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.291+0000 7f25048bb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f250009c700 con 0x7f250009e410 2023-10-03T10:18:58.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.292+0000 7f24fe59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f250009d6f0 0x7f250011e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:58.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.292+0000 7f24fdd9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f250009e000 0x7f250011e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:58.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.292+0000 7f24fed9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f250009e410 0x7f250011edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:58.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.292+0000 7f24fe59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f250009d6f0 0x7f250011e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49758/0 (socket says 172.21.15.3:49758) 2023-10-03T10:18:58.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.292+0000 7f24fdd9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f250009e000 0x7f250011e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42270/0 (socket says 172.21.15.3:42270) 2023-10-03T10:18:58.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.292+0000 7f24fe59c700 1 -- 172.21.15.3:0/663841785 learned_addr learned my addr 172.21.15.3:0/663841785 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:58.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.292+0000 7f24fdd9b700 1 -- 172.21.15.3:0/663841785 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f250009e410 msgr2=0x7f250011edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:58.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.292+0000 7f24fdd9b700 1 --2- 172.21.15.3:0/663841785 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f250009e410 0x7f250011edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.292+0000 7f24fdd9b700 1 -- 172.21.15.3:0/663841785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f250009d6f0 msgr2=0x7f250011e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:58.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.292+0000 7f24fdd9b700 1 --2- 172.21.15.3:0/663841785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f250009d6f0 0x7f250011e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.292+0000 7f24fdd9b700 1 -- 172.21.15.3:0/663841785 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f000b7f0 con 0x7f250009e000 2023-10-03T10:18:58.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.292+0000 7f24fe59c700 1 --2- 172.21.15.3:0/663841785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f250009d6f0 0x7f250011e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:58.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.292+0000 7f24fdd9b700 1 --2- 172.21.15.3:0/663841785 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f250009e000 0x7f250011e880 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f24f400e8d0 tx=0x7f24f400ebe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:58.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.292+0000 7f24eeffd700 1 -- 172.21.15.3:0/663841785 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f4013680 con 0x7f250009e000 2023-10-03T10:18:58.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.293+0000 7f24eeffd700 1 -- 172.21.15.3:0/663841785 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f40137e0 con 0x7f250009e000 2023-10-03T10:18:58.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.293+0000 7f25048bb700 1 -- 172.21.15.3:0/663841785 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2500121440 con 0x7f250009e000 2023-10-03T10:18:58.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.293+0000 7f24eeffd700 1 -- 172.21.15.3:0/663841785 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f4013b50 con 0x7f250009e000 2023-10-03T10:18:58.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.293+0000 7f25048bb700 1 -- 172.21.15.3:0/663841785 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2500120750 con 0x7f250009e000 2023-10-03T10:18:58.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.294+0000 7f25048bb700 1 -- 172.21.15.3:0/663841785 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2500074ea0 con 0x7f250009e000 2023-10-03T10:18:58.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.294+0000 7f24eeffd700 1 -- 172.21.15.3:0/663841785 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f24f4013cb0 con 0x7f250009e000 2023-10-03T10:18:58.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.295+0000 7f24eeffd700 1 -- 172.21.15.3:0/663841785 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f24f403ff70 con 0x7f250009e000 2023-10-03T10:18:58.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.299+0000 7f24eeffd700 1 -- 172.21.15.3:0/663841785 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f24f401a360 con 0x7f250009e000 2023-10-03T10:18:58.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.408+0000 7fd3c71c0700 1 -- 172.21.15.3:0/951896310 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd3c006c360 con 0x7fd3c009e1d0 2023-10-03T10:18:58.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.409+0000 7fd3bd7fa700 1 -- 172.21.15.3:0/951896310 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd3ac033020 con 0x7fd3c009e1d0 2023-10-03T10:18:58.409 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:58.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.412+0000 7fd3c71c0700 1 -- 172.21.15.3:0/951896310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3c009e1d0 msgr2=0x7fd3c0120d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:58.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.412+0000 7fd3c71c0700 1 --2- 172.21.15.3:0/951896310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3c009e1d0 0x7fd3c0120d30 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009ed0 tx=0x7fd3ac004750 comp rx=0 tx=0).stop 2023-10-03T10:18:58.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.412+0000 7fd3c71c0700 1 -- 172.21.15.3:0/951896310 shutdown_connections 2023-10-03T10:18:58.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.412+0000 7fd3c71c0700 1 --2- 172.21.15.3:0/951896310 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd3c0139170 0x7fd3c0121270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.412+0000 7fd3c71c0700 1 --2- 172.21.15.3:0/951896310 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3c009d6f0 0x7fd3c01207f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.412+0000 7fd3c71c0700 1 --2- 172.21.15.3:0/951896310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3c009e1d0 0x7fd3c0120d30 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.412+0000 7fd3c71c0700 1 -- 172.21.15.3:0/951896310 >> 172.21.15.3:0/951896310 conn(0x7fd3c0066f50 msgr2=0x7fd3c0070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:58.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.412+0000 7fd3c71c0700 1 -- 172.21.15.3:0/951896310 shutdown_connections 2023-10-03T10:18:58.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.412+0000 7fd3c71c0700 1 -- 172.21.15.3:0/951896310 wait complete. 2023-10-03T10:18:58.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.419+0000 7ff1f7767700 1 -- 172.21.15.3:0/2879058566 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff1f009f420 con 0x7ff1f0139170 2023-10-03T10:18:58.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.420+0000 7ff1e5ffb700 1 -- 172.21.15.3:0/2879058566 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff1e8023b50 con 0x7ff1f0139170 2023-10-03T10:18:58.420 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:58.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.422+0000 7ff1f7767700 1 -- 172.21.15.3:0/2879058566 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1f0139170 msgr2=0x7ff1f011e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:58.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.422+0000 7ff1f7767700 1 --2- 172.21.15.3:0/2879058566 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1f0139170 0x7ff1f011e7f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8012000 tx=0x7ff1e8004ad0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.422+0000 7ff1f7767700 1 -- 172.21.15.3:0/2879058566 shutdown_connections 2023-10-03T10:18:58.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.422+0000 7ff1f7767700 1 --2- 172.21.15.3:0/2879058566 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1f0139170 0x7ff1f011e7f0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.422+0000 7ff1f7767700 1 --2- 172.21.15.3:0/2879058566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1f009d6f0 0x7ff1f01c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.422+0000 7ff1f7767700 1 --2- 172.21.15.3:0/2879058566 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1f009e1d0 0x7ff1f01c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.422+0000 7ff1f7767700 1 -- 172.21.15.3:0/2879058566 >> 172.21.15.3:0/2879058566 conn(0x7ff1f0066f50 msgr2=0x7ff1f00a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:58.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.422+0000 7ff1f7767700 1 -- 172.21.15.3:0/2879058566 shutdown_connections 2023-10-03T10:18:58.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.422+0000 7ff1f7767700 1 -- 172.21.15.3:0/2879058566 wait complete. 2023-10-03T10:18:58.427 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:58.442 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:58.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.507+0000 7f25048bb700 1 -- 172.21.15.3:0/663841785 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f250006c360 con 0x7f250009e000 2023-10-03T10:18:58.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.507+0000 7f24eeffd700 1 -- 172.21.15.3:0/663841785 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f24f4017030 con 0x7f250009e000 2023-10-03T10:18:58.508 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:58.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.509+0000 7f25048bb700 1 -- 172.21.15.3:0/663841785 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f250009e000 msgr2=0x7f250011e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:58.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.510+0000 7f25048bb700 1 --2- 172.21.15.3:0/663841785 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f250009e000 0x7f250011e880 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f24f400e8d0 tx=0x7f24f400ebe0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.510+0000 7f25048bb700 1 -- 172.21.15.3:0/663841785 shutdown_connections 2023-10-03T10:18:58.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.510+0000 7f25048bb700 1 --2- 172.21.15.3:0/663841785 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f250009e410 0x7f250011edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.510+0000 7f25048bb700 1 --2- 172.21.15.3:0/663841785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f250009d6f0 0x7f250011e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.510+0000 7f25048bb700 1 --2- 172.21.15.3:0/663841785 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f250009e000 0x7f250011e880 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.510+0000 7f25048bb700 1 -- 172.21.15.3:0/663841785 >> 172.21.15.3:0/663841785 conn(0x7f2500066f50 msgr2=0x7f250006f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:58.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.510+0000 7f25048bb700 1 -- 172.21.15.3:0/663841785 shutdown_connections 2023-10-03T10:18:58.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.510+0000 7f25048bb700 1 -- 172.21.15.3:0/663841785 wait complete. 2023-10-03T10:18:58.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.515+0000 7f21bb6da700 1 -- 172.21.15.3:0/37460080 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f21b409c710 con 0x7f21b409e410 2023-10-03T10:18:58.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.516+0000 7f21a9ffb700 1 -- 172.21.15.3:0/37460080 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f21ac01b310 con 0x7f21b409e410 2023-10-03T10:18:58.516 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.518+0000 7f21bb6da700 1 -- 172.21.15.3:0/37460080 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21b409e410 msgr2=0x7f21b41a67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.518+0000 7f21bb6da700 1 --2- 172.21.15.3:0/37460080 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21b409e410 0x7f21b41a67b0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f21ac007e90 tx=0x7f21ac00e3c0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.518+0000 7f21bb6da700 1 -- 172.21.15.3:0/37460080 shutdown_connections 2023-10-03T10:18:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.518+0000 7f21bb6da700 1 --2- 172.21.15.3:0/37460080 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21b409e410 0x7f21b41a67b0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.518+0000 7f21bb6da700 1 --2- 172.21.15.3:0/37460080 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21b409d6f0 0x7f21b41a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.518+0000 7f21bb6da700 1 --2- 172.21.15.3:0/37460080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21b409e000 0x7f21b41a6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.518+0000 7f21bb6da700 1 -- 172.21.15.3:0/37460080 >> 172.21.15.3:0/37460080 conn(0x7f21b4066f50 msgr2=0x7f21b41bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:58.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.518+0000 7f21bb6da700 1 -- 172.21.15.3:0/37460080 shutdown_connections 2023-10-03T10:18:58.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:58.518+0000 7f21bb6da700 1 -- 172.21.15.3:0/37460080 wait complete. 2023-10-03T10:18:58.524 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:58.532 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:18:59.428 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:18:59.443 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:18:59.526 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:18:59.533 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:18:59.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.580+0000 7f6599535700 1 Processor -- start 2023-10-03T10:18:59.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.580+0000 7f6599535700 1 -- start start 2023-10-03T10:18:59.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.581+0000 7f6599535700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65941338c0 0x7f6594133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:59.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.581+0000 7f6599535700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f659409e560 0x7f659413b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:59.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.581+0000 7f6599535700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f659406f720 con 0x7f65941341d0 2023-10-03T10:18:59.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.581+0000 7f6599535700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6594070c80 con 0x7f659409e560 2023-10-03T10:18:59.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.581+0000 7f6599535700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f659406fce0 con 0x7f65941338c0 2023-10-03T10:18:59.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.581+0000 7f65937fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f659409e560 0x7f659413b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:59.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.581+0000 7f6592ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65941338c0 0x7f6594133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:59.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.581+0000 7f65937fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f659409e560 0x7f659413b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49760/0 (socket says 172.21.15.3:49760) 2023-10-03T10:18:59.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.581+0000 7f65937fe700 1 -- 172.21.15.3:0/167377201 learned_addr learned my addr 172.21.15.3:0/167377201 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.581+0000 7f6592ffd700 1 -- 172.21.15.3:0/167377201 >> v2:172.21.15.3:3300/0 conn(0x7f659409e560 msgr2=0x7f659413b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.581+0000 7f6592ffd700 1 --2- 172.21.15.3:0/167377201 >> v2:172.21.15.3:3300/0 conn(0x7f659409e560 0x7f659413b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.581+0000 7f6592ffd700 1 -- 172.21.15.3:0/167377201 >> v1:172.21.15.3:6789/0 conn(0x7f65941341d0 legacy=0x7f659409de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:59.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.581+0000 7f6592ffd700 1 -- 172.21.15.3:0/167377201 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65941336a0 con 0x7f65941338c0 2023-10-03T10:18:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.582+0000 7f6592ffd700 1 --2- 172.21.15.3:0/167377201 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65941338c0 0x7f6594133c90 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6584009840 tx=0x7f6584009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=821fc3d75e67f22a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.582+0000 7f6591ffb700 1 -- 172.21.15.3:0/167377201 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6584011830 con 0x7f65941338c0 2023-10-03T10:18:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.582+0000 7f6591ffb700 1 -- 172.21.15.3:0/167377201 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6584011990 con 0x7f65941338c0 2023-10-03T10:18:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.582+0000 7f6591ffb700 1 -- 172.21.15.3:0/167377201 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6584011d20 con 0x7f65941338c0 2023-10-03T10:18:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.583+0000 7f6599535700 1 -- 172.21.15.3:0/167377201 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65941338c0 msgr2=0x7f6594133c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.583+0000 7f6599535700 1 --2- 172.21.15.3:0/167377201 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65941338c0 0x7f6594133c90 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6584009840 tx=0x7f6584009b50 comp rx=0 tx=0).stop 2023-10-03T10:18:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.583+0000 7f6599535700 1 -- 172.21.15.3:0/167377201 shutdown_connections 2023-10-03T10:18:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.583+0000 7f6599535700 1 --2- 172.21.15.3:0/167377201 >> v2:172.21.15.3:3300/0 conn(0x7f659409e560 0x7f659413b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.583+0000 7f6599535700 1 --2- 172.21.15.3:0/167377201 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65941338c0 0x7f6594133c90 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.583+0000 7f6599535700 1 -- 172.21.15.3:0/167377201 >> 172.21.15.3:0/167377201 conn(0x7f65940a2aa0 msgr2=0x7f65940a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.583+0000 7f6599535700 1 -- 172.21.15.3:0/167377201 shutdown_connections 2023-10-03T10:18:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.583+0000 7f6599535700 1 -- 172.21.15.3:0/167377201 wait complete. 2023-10-03T10:18:59.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.584+0000 7f6599535700 1 Processor -- start 2023-10-03T10:18:59.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.584+0000 7f6599535700 1 -- start start 2023-10-03T10:18:59.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.585+0000 7f6599535700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f659409e560 0x7f659411fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:59.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.585+0000 7f6599535700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65941338c0 0x7f6594120090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:59.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.585+0000 7f6599535700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65941341d0 0x7f65941205d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:59.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.585+0000 7f6599535700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659406f960 con 0x7f65941338c0 2023-10-03T10:18:59.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.585+0000 7f6599535700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6594123730 con 0x7f659409e560 2023-10-03T10:18:59.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.585+0000 7f6599535700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f65941217c0 con 0x7f65941341d0 2023-10-03T10:18:59.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.585+0000 7f6592ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f659409e560 0x7f659411fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:59.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.585+0000 7f6592ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f659409e560 0x7f659411fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42276/0 (socket says 172.21.15.3:42276) 2023-10-03T10:18:59.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.585+0000 7f6592ffd700 1 -- 172.21.15.3:0/2435466620 learned_addr learned my addr 172.21.15.3:0/2435466620 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:59.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.585+0000 7f65927fc700 1 --2- 172.21.15.3:0/2435466620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65941338c0 0x7f6594120090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:59.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.585+0000 7f65937fe700 1 --2- 172.21.15.3:0/2435466620 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65941341d0 0x7f65941205d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:59.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.585+0000 7f65927fc700 1 -- 172.21.15.3:0/2435466620 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65941341d0 msgr2=0x7f65941205d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:59.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.586+0000 7f65927fc700 1 --2- 172.21.15.3:0/2435466620 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65941341d0 0x7f65941205d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.586+0000 7f65927fc700 1 -- 172.21.15.3:0/2435466620 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f659409e560 msgr2=0x7f659411fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:59.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.586+0000 7f65927fc700 1 --2- 172.21.15.3:0/2435466620 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f659409e560 0x7f659411fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.586+0000 7f65927fc700 1 -- 172.21.15.3:0/2435466620 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65840094a0 con 0x7f65941338c0 2023-10-03T10:18:59.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.586+0000 7f65927fc700 1 --2- 172.21.15.3:0/2435466620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65941338c0 0x7f6594120090 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f6588000f60 tx=0x7f658800c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:59.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.586+0000 7f657b7fe700 1 -- 172.21.15.3:0/2435466620 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65880050d0 con 0x7f65941338c0 2023-10-03T10:18:59.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.586+0000 7f6599535700 1 -- 172.21.15.3:0/2435466620 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6594123090 con 0x7f65941338c0 2023-10-03T10:18:59.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.586+0000 7f657b7fe700 1 -- 172.21.15.3:0/2435466620 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6588005230 con 0x7f65941338c0 2023-10-03T10:18:59.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.586+0000 7f657b7fe700 1 -- 172.21.15.3:0/2435466620 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6588003680 con 0x7f65941338c0 2023-10-03T10:18:59.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.586+0000 7f6599535700 1 -- 172.21.15.3:0/2435466620 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6594130b20 con 0x7f65941338c0 2023-10-03T10:18:59.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.587+0000 7f657b7fe700 1 -- 172.21.15.3:0/2435466620 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f65880037e0 con 0x7f65941338c0 2023-10-03T10:18:59.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.587+0000 7f657b7fe700 1 -- 172.21.15.3:0/2435466620 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f658803e9c0 con 0x7f65941338c0 2023-10-03T10:18:59.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.588+0000 7f6599535700 1 -- 172.21.15.3:0/2435466620 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6594130670 con 0x7f65941338c0 2023-10-03T10:18:59.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.594+0000 7f657b7fe700 1 -- 172.21.15.3:0/2435466620 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6588017080 con 0x7f65941338c0 2023-10-03T10:18:59.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.622+0000 7eff933e1700 1 Processor -- start 2023-10-03T10:18:59.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.622+0000 7eff933e1700 1 -- start start 2023-10-03T10:18:59.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.622+0000 7eff933e1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff8c133420 0x7eff8c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:59.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.622+0000 7eff933e1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff8c133e30 0x7eff8c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:59.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.622+0000 7eff933e1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7eff8c134260 0x7eff8c1304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:59.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.622+0000 7eff933e1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff8c06b990 con 0x7eff8c134260 2023-10-03T10:18:59.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.622+0000 7eff933e1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c0a0590 con 0x7eff8c133420 2023-10-03T10:18:59.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.622+0000 7eff933e1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c09fb80 con 0x7eff8c133e30 2023-10-03T10:18:59.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.623+0000 7eff9117d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff8c133420 0x7eff8c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:59.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.623+0000 7eff9117d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff8c133420 0x7eff8c1337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42288/0 (socket says 172.21.15.3:42288) 2023-10-03T10:18:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.623+0000 7eff9117d700 1 -- 172.21.15.3:0/1324162141 learned_addr learned my addr 172.21.15.3:0/1324162141 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.623+0000 7eff9097c700 1 --2- 172.21.15.3:0/1324162141 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff8c133e30 0x7eff8c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.623+0000 7eff9197e700 1 --2- 172.21.15.3:0/1324162141 >> v2:172.21.15.3:3300/0 conn(0x7eff8c134260 0x7eff8c1304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.623+0000 7eff9097c700 1 -- 172.21.15.3:0/1324162141 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff8c133420 msgr2=0x7eff8c1337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.623+0000 7eff9097c700 1 --2- 172.21.15.3:0/1324162141 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff8c133420 0x7eff8c1337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.623+0000 7eff9097c700 1 -- 172.21.15.3:0/1324162141 >> v2:172.21.15.3:3300/0 conn(0x7eff8c134260 msgr2=0x7eff8c1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.623+0000 7eff9097c700 1 --2- 172.21.15.3:0/1324162141 >> v2:172.21.15.3:3300/0 conn(0x7eff8c134260 0x7eff8c1304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.623+0000 7eff9097c700 1 -- 172.21.15.3:0/1324162141 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff8c0a0030 con 0x7eff8c133e30 2023-10-03T10:18:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.623+0000 7eff9197e700 1 --2- 172.21.15.3:0/1324162141 >> v2:172.21.15.3:3300/0 conn(0x7eff8c134260 0x7eff8c1304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.623+0000 7eff9097c700 1 --2- 172.21.15.3:0/1324162141 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff8c133e30 0x7eff8c12ff90 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7eff7c009840 tx=0x7eff7c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1b2dbb8442664d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:59.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.624+0000 7eff83fff700 1 -- 172.21.15.3:0/1324162141 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c011830 con 0x7eff8c133e30 2023-10-03T10:18:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.624+0000 7eff83fff700 1 -- 172.21.15.3:0/1324162141 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7c011990 con 0x7eff8c133e30 2023-10-03T10:18:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.624+0000 7eff83fff700 1 -- 172.21.15.3:0/1324162141 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c011d20 con 0x7eff8c133e30 2023-10-03T10:18:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.624+0000 7eff933e1700 1 -- 172.21.15.3:0/1324162141 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff8c133e30 msgr2=0x7eff8c12ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.624+0000 7eff933e1700 1 --2- 172.21.15.3:0/1324162141 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff8c133e30 0x7eff8c12ff90 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7eff7c009840 tx=0x7eff7c009b50 comp rx=0 tx=0).stop 2023-10-03T10:18:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.624+0000 7eff933e1700 1 -- 172.21.15.3:0/1324162141 shutdown_connections 2023-10-03T10:18:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.624+0000 7eff933e1700 1 --2- 172.21.15.3:0/1324162141 >> v2:172.21.15.3:3300/0 conn(0x7eff8c134260 0x7eff8c1304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.625+0000 7eff933e1700 1 --2- 172.21.15.3:0/1324162141 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff8c133420 0x7eff8c1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.625+0000 7eff933e1700 1 --2- 172.21.15.3:0/1324162141 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff8c133e30 0x7eff8c12ff90 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.625+0000 7eff933e1700 1 -- 172.21.15.3:0/1324162141 >> 172.21.15.3:0/1324162141 conn(0x7eff8c0a4210 msgr2=0x7eff8c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.625+0000 7eff933e1700 1 -- 172.21.15.3:0/1324162141 shutdown_connections 2023-10-03T10:18:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.625+0000 7eff933e1700 1 -- 172.21.15.3:0/1324162141 wait complete. 2023-10-03T10:18:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.626+0000 7eff933e1700 1 Processor -- start 2023-10-03T10:18:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.626+0000 7eff933e1700 1 -- start start 2023-10-03T10:18:59.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.627+0000 7eff933e1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff8c133420 0x7eff8c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:59.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.627+0000 7eff933e1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff8c133e30 0x7eff8c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:59.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.627+0000 7eff933e1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff8c134260 0x7eff8c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:59.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.627+0000 7eff933e1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c12d5e0 con 0x7eff8c134260 2023-10-03T10:18:59.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.627+0000 7eff933e1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c131490 con 0x7eff8c133e30 2023-10-03T10:18:59.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.627+0000 7eff933e1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c11fcc0 con 0x7eff8c133420 2023-10-03T10:18:59.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.627+0000 7eff9117d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff8c133420 0x7eff8c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:59.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.627+0000 7eff9117d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff8c133420 0x7eff8c09c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48030/0 (socket says 172.21.15.3:48030) 2023-10-03T10:18:59.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.627+0000 7eff9117d700 1 -- 172.21.15.3:0/1180578859 learned_addr learned my addr 172.21.15.3:0/1180578859 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:59.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.627+0000 7eff9197e700 1 --2- 172.21.15.3:0/1180578859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff8c134260 0x7eff8c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:59.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.627+0000 7eff9097c700 1 --2- 172.21.15.3:0/1180578859 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff8c133e30 0x7eff8c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:59.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.627+0000 7eff9117d700 1 -- 172.21.15.3:0/1180578859 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff8c133e30 msgr2=0x7eff8c09c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:59.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.627+0000 7eff9117d700 1 --2- 172.21.15.3:0/1180578859 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff8c133e30 0x7eff8c09c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.627+0000 7eff9117d700 1 -- 172.21.15.3:0/1180578859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff8c134260 msgr2=0x7eff8c09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:59.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.627+0000 7eff9117d700 1 --2- 172.21.15.3:0/1180578859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff8c134260 0x7eff8c09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.627+0000 7eff9117d700 1 -- 172.21.15.3:0/1180578859 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff84009a70 con 0x7eff8c133420 2023-10-03T10:18:59.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.628+0000 7eff9197e700 1 --2- 172.21.15.3:0/1180578859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff8c134260 0x7eff8c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:18:59.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.628+0000 7eff9117d700 1 --2- 172.21.15.3:0/1180578859 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff8c133420 0x7eff8c09c480 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7eff8800e910 tx=0x7eff8800ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:59.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.628+0000 7eff81ffb700 1 -- 172.21.15.3:0/1180578859 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff880096a0 con 0x7eff8c133420 2023-10-03T10:18:59.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.628+0000 7eff81ffb700 1 -- 172.21.15.3:0/1180578859 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8800ee60 con 0x7eff8c133420 2023-10-03T10:18:59.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.628+0000 7eff933e1700 1 -- 172.21.15.3:0/1180578859 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff7c0094a0 con 0x7eff8c133420 2023-10-03T10:18:59.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.628+0000 7eff81ffb700 1 -- 172.21.15.3:0/1180578859 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8800ca90 con 0x7eff8c133420 2023-10-03T10:18:59.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.628+0000 7eff933e1700 1 -- 172.21.15.3:0/1180578859 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff8c11f9b0 con 0x7eff8c133420 2023-10-03T10:18:59.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.629+0000 7eff933e1700 1 -- 172.21.15.3:0/1180578859 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff8c11fee0 con 0x7eff8c133420 2023-10-03T10:18:59.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.630+0000 7eff81ffb700 1 -- 172.21.15.3:0/1180578859 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7eff8800cbf0 con 0x7eff8c133420 2023-10-03T10:18:59.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.630+0000 7eff81ffb700 1 -- 172.21.15.3:0/1180578859 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7eff8803ff00 con 0x7eff8c133420 2023-10-03T10:18:59.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.635+0000 7eff81ffb700 1 -- 172.21.15.3:0/1180578859 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7eff88024350 con 0x7eff8c133420 2023-10-03T10:18:59.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.703+0000 7fae9dd60700 1 Processor -- start 2023-10-03T10:18:59.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.703+0000 7fae9dd60700 1 -- start start 2023-10-03T10:18:59.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.703+0000 7fae9dd60700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fae9809d6f0 0x7fae9809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:59.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.703+0000 7fae9dd60700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fae9809e000 0x7fae981339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:59.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.703+0000 7fae9dd60700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae9809b120 con 0x7fae9809e410 2023-10-03T10:18:59.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.703+0000 7fae9dd60700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae9806b460 con 0x7fae9809d6f0 2023-10-03T10:18:59.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.703+0000 7fae9dd60700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9809a7e0 con 0x7fae9809e000 2023-10-03T10:18:59.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.704+0000 7fae96ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fae9809e000 0x7fae981339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:59.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.704+0000 7fae96ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fae9809e000 0x7fae981339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42300/0 (socket says 172.21.15.3:42300) 2023-10-03T10:18:59.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.704+0000 7fae96ffd700 1 -- 172.21.15.3:0/298215131 learned_addr learned my addr 172.21.15.3:0/298215131 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:59.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.704+0000 7fae977fe700 1 --2- 172.21.15.3:0/298215131 >> v2:172.21.15.3:3300/0 conn(0x7fae9809d6f0 0x7fae9809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:59.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.704+0000 7fae977fe700 1 -- 172.21.15.3:0/298215131 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fae9809e000 msgr2=0x7fae981339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:59.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.704+0000 7fae977fe700 1 --2- 172.21.15.3:0/298215131 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fae9809e000 0x7fae981339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.704+0000 7fae977fe700 1 -- 172.21.15.3:0/298215131 >> v1:172.21.15.3:6789/0 conn(0x7fae9809e410 legacy=0x7fae98133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:59.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.704+0000 7fae977fe700 1 -- 172.21.15.3:0/298215131 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9809a500 con 0x7fae9809d6f0 2023-10-03T10:18:59.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.704+0000 7fae96ffd700 1 --2- 172.21.15.3:0/298215131 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fae9809e000 0x7fae981339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:18:59.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.704+0000 7fae977fe700 1 --2- 172.21.15.3:0/298215131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae9809d6f0 0x7fae9809dac0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fae88009840 tx=0x7fae88009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339b2f06d3ece542 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:59.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.705+0000 7fae967fc700 1 -- 172.21.15.3:0/298215131 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae88011830 con 0x7fae9809d6f0 2023-10-03T10:18:59.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.705+0000 7fae967fc700 1 -- 172.21.15.3:0/298215131 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae88011990 con 0x7fae9809d6f0 2023-10-03T10:18:59.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.705+0000 7fae967fc700 1 -- 172.21.15.3:0/298215131 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae88011d20 con 0x7fae9809d6f0 2023-10-03T10:18:59.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.706+0000 7fae9dd60700 1 -- 172.21.15.3:0/298215131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae9809d6f0 msgr2=0x7fae9809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:59.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.706+0000 7fae9dd60700 1 --2- 172.21.15.3:0/298215131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae9809d6f0 0x7fae9809dac0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fae88009840 tx=0x7fae88009b50 comp rx=0 tx=0).stop 2023-10-03T10:18:59.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.706+0000 7fae9dd60700 1 -- 172.21.15.3:0/298215131 shutdown_connections 2023-10-03T10:18:59.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.706+0000 7fae9dd60700 1 --2- 172.21.15.3:0/298215131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae9809d6f0 0x7fae9809dac0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.706+0000 7fae9dd60700 1 --2- 172.21.15.3:0/298215131 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fae9809e000 0x7fae981339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.706+0000 7fae9dd60700 1 -- 172.21.15.3:0/298215131 >> 172.21.15.3:0/298215131 conn(0x7fae98066f50 msgr2=0x7fae9809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:59.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.706+0000 7fae9dd60700 1 -- 172.21.15.3:0/298215131 shutdown_connections 2023-10-03T10:18:59.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.706+0000 7fae9dd60700 1 -- 172.21.15.3:0/298215131 wait complete. 2023-10-03T10:18:59.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.707+0000 7fae9dd60700 1 Processor -- start 2023-10-03T10:18:59.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.708+0000 7fae9dd60700 1 -- start start 2023-10-03T10:18:59.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.708+0000 7fae9dd60700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae9809d6f0 0x7fae981a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:59.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.708+0000 7fae9dd60700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae9809e000 0x7fae981a63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:59.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.708+0000 7fae9dd60700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fae9809e410 0x7fae981a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:59.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.708+0000 7fae9dd60700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae981a9130 con 0x7fae9809d6f0 2023-10-03T10:18:59.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.708+0000 7fae9dd60700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9809c710 con 0x7fae9809e410 2023-10-03T10:18:59.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.708+0000 7fae9dd60700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae980a3250 con 0x7fae9809e000 2023-10-03T10:18:59.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.709+0000 7fae977fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae9809d6f0 0x7fae981a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:59.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.709+0000 7fae977fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae9809d6f0 0x7fae981a5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49804/0 (socket says 172.21.15.3:49804) 2023-10-03T10:18:59.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.709+0000 7fae977fe700 1 -- 172.21.15.3:0/734957265 learned_addr learned my addr 172.21.15.3:0/734957265 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:59.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.709+0000 7fae96ffd700 1 --2- 172.21.15.3:0/734957265 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae9809e000 0x7fae981a63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:59.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.709+0000 7fae97fff700 1 --2- 172.21.15.3:0/734957265 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fae9809e410 0x7fae981a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:59.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.709+0000 7fae977fe700 1 -- 172.21.15.3:0/734957265 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae9809e000 msgr2=0x7fae981a63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:59.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.709+0000 7fae977fe700 1 --2- 172.21.15.3:0/734957265 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae9809e000 0x7fae981a63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.709+0000 7fae977fe700 1 -- 172.21.15.3:0/734957265 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fae9809e410 msgr2=0x7fae981a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:59.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.709+0000 7fae977fe700 1 --2- 172.21.15.3:0/734957265 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fae9809e410 0x7fae981a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.709+0000 7fae977fe700 1 -- 172.21.15.3:0/734957265 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae8c009550 con 0x7fae9809d6f0 2023-10-03T10:18:59.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.710+0000 7fae977fe700 1 --2- 172.21.15.3:0/734957265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae9809d6f0 0x7fae981a5e80 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fae88009b90 tx=0x7fae88004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:59.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.710+0000 7fae7ffff700 1 -- 172.21.15.3:0/734957265 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae880036a0 con 0x7fae9809d6f0 2023-10-03T10:18:59.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.710+0000 7fae7ffff700 1 -- 172.21.15.3:0/734957265 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8801e070 con 0x7fae9809d6f0 2023-10-03T10:18:59.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.710+0000 7fae9dd60700 1 -- 172.21.15.3:0/734957265 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae880094a0 con 0x7fae9809d6f0 2023-10-03T10:18:59.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.710+0000 7fae7ffff700 1 -- 172.21.15.3:0/734957265 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae88003a60 con 0x7fae9809d6f0 2023-10-03T10:18:59.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.710+0000 7fae9dd60700 1 -- 172.21.15.3:0/734957265 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae981a8ce0 con 0x7fae9809d6f0 2023-10-03T10:18:59.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.711+0000 7fae7ffff700 1 -- 172.21.15.3:0/734957265 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fae88003bc0 con 0x7fae9809d6f0 2023-10-03T10:18:59.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.711+0000 7fae9dd60700 1 -- 172.21.15.3:0/734957265 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae981a9360 con 0x7fae9809d6f0 2023-10-03T10:18:59.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.712+0000 7fae7ffff700 1 -- 172.21.15.3:0/734957265 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fae88040600 con 0x7fae9809d6f0 2023-10-03T10:18:59.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.717+0000 7fae7ffff700 1 -- 172.21.15.3:0/734957265 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fae981a9360 con 0x7fae9809d6f0 2023-10-03T10:18:59.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.720+0000 7effb518a700 1 Processor -- start 2023-10-03T10:18:59.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.720+0000 7effb518a700 1 -- start start 2023-10-03T10:18:59.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.721+0000 7effb518a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7effb0133420 0x7effb01337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:59.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.721+0000 7effb518a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effb0133e30 0x7effb012ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:59.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.721+0000 7effb518a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb006b990 con 0x7effb0134260 2023-10-03T10:18:59.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.721+0000 7effb518a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb00a0590 con 0x7effb0133420 2023-10-03T10:18:59.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.721+0000 7effb518a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7effb009fb80 con 0x7effb0133e30 2023-10-03T10:18:59.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.721+0000 7effa7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effb0133e30 0x7effb012ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:59.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.721+0000 7effa7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effb0133e30 0x7effb012ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48046/0 (socket says 172.21.15.3:48046) 2023-10-03T10:18:59.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.721+0000 7effa7fff700 1 -- 172.21.15.3:0/4638217 learned_addr learned my addr 172.21.15.3:0/4638217 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:59.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.721+0000 7effa7fff700 1 -- 172.21.15.3:0/4638217 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7effb0133420 msgr2=0x7effb01337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:18:59.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.721+0000 7effa7fff700 1 --2- 172.21.15.3:0/4638217 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7effb0133420 0x7effb01337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.721+0000 7effa7fff700 1 -- 172.21.15.3:0/4638217 >> v1:172.21.15.3:6789/0 conn(0x7effb0134260 legacy=0x7effb01304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:59.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.721+0000 7effa7fff700 1 -- 172.21.15.3:0/4638217 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb00a0030 con 0x7effb0133e30 2023-10-03T10:18:59.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.722+0000 7effa7fff700 1 --2- 172.21.15.3:0/4638217 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effb0133e30 0x7effb012ff90 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7eff98009920 tx=0x7eff98009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79fc38a0d3e486e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:59.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.722+0000 7effae59c700 1 -- 172.21.15.3:0/4638217 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9800ba60 con 0x7effb0133e30 2023-10-03T10:18:59.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.722+0000 7effae59c700 1 -- 172.21.15.3:0/4638217 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9800bbc0 con 0x7effb0133e30 2023-10-03T10:18:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.722+0000 7effae59c700 1 -- 172.21.15.3:0/4638217 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff98009e70 con 0x7effb0133e30 2023-10-03T10:18:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.723+0000 7effb518a700 1 -- 172.21.15.3:0/4638217 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effb0133e30 msgr2=0x7effb012ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.723+0000 7effb518a700 1 --2- 172.21.15.3:0/4638217 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effb0133e30 0x7effb012ff90 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7eff98009920 tx=0x7eff98009c30 comp rx=0 tx=0).stop 2023-10-03T10:18:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.723+0000 7effb518a700 1 -- 172.21.15.3:0/4638217 shutdown_connections 2023-10-03T10:18:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.723+0000 7effb518a700 1 --2- 172.21.15.3:0/4638217 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7effb0133420 0x7effb01337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.723+0000 7effb518a700 1 --2- 172.21.15.3:0/4638217 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effb0133e30 0x7effb012ff90 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.723+0000 7effb518a700 1 -- 172.21.15.3:0/4638217 >> 172.21.15.3:0/4638217 conn(0x7effb00a4210 msgr2=0x7effb00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.723+0000 7effb518a700 1 -- 172.21.15.3:0/4638217 shutdown_connections 2023-10-03T10:18:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.723+0000 7effb518a700 1 -- 172.21.15.3:0/4638217 wait complete. 2023-10-03T10:18:59.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.724+0000 7effb518a700 1 Processor -- start 2023-10-03T10:18:59.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.724+0000 7effb518a700 1 -- start start 2023-10-03T10:18:59.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.725+0000 7effb518a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effb0133420 0x7effb009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:59.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.725+0000 7effb518a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effb0133e30 0x7effb009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.725+0000 7effb518a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7effb0134260 0x7effb009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:18:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.725+0000 7effb518a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb0131920 con 0x7effb0133e30 2023-10-03T10:18:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.725+0000 7effb518a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb01204b0 con 0x7effb0134260 2023-10-03T10:18:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.725+0000 7effb518a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7effb0120060 con 0x7effb0133420 2023-10-03T10:18:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.725+0000 7effaed9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effb0133420 0x7effb009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.725+0000 7effaed9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effb0133420 0x7effb009c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48060/0 (socket says 172.21.15.3:48060) 2023-10-03T10:18:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.725+0000 7effaed9d700 1 -- 172.21.15.3:0/3495730345 learned_addr learned my addr 172.21.15.3:0/3495730345 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:18:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.725+0000 7effa7fff700 1 --2- 172.21.15.3:0/3495730345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effb0133e30 0x7effb009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.725+0000 7effaf59e700 1 --2- 172.21.15.3:0/3495730345 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7effb0134260 0x7effb009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:18:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.725+0000 7effa7fff700 1 -- 172.21.15.3:0/3495730345 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effb0133420 msgr2=0x7effb009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.725+0000 7effa7fff700 1 --2- 172.21.15.3:0/3495730345 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effb0133420 0x7effb009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.725+0000 7effa7fff700 1 -- 172.21.15.3:0/3495730345 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7effb0134260 msgr2=0x7effb009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.725+0000 7effa7fff700 1 --2- 172.21.15.3:0/3495730345 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7effb0134260 0x7effb009cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.725+0000 7effa7fff700 1 -- 172.21.15.3:0/3495730345 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff98009550 con 0x7effb0133e30 2023-10-03T10:18:59.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.726+0000 7effa7fff700 1 --2- 172.21.15.3:0/3495730345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effb0133e30 0x7effb009c9c0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7eff98016000 tx=0x7eff98004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:18:59.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.726+0000 7effa77fe700 1 -- 172.21.15.3:0/3495730345 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff98004550 con 0x7effb0133e30 2023-10-03T10:18:59.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.726+0000 7effa77fe700 1 -- 172.21.15.3:0/3495730345 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff980036a0 con 0x7effb0133e30 2023-10-03T10:18:59.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.726+0000 7effa77fe700 1 -- 172.21.15.3:0/3495730345 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff98003a10 con 0x7effb0133e30 2023-10-03T10:18:59.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.726+0000 7effb518a700 1 -- 172.21.15.3:0/3495730345 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb011f9e0 con 0x7effb0133e30 2023-10-03T10:18:59.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.726+0000 7effb518a700 1 -- 172.21.15.3:0/3495730345 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb012d770 con 0x7effb0133e30 2023-10-03T10:18:59.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.727+0000 7effa77fe700 1 -- 172.21.15.3:0/3495730345 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7eff98021070 con 0x7effb0133e30 2023-10-03T10:18:59.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.727+0000 7effa57fa700 1 -- 172.21.15.3:0/3495730345 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb01202a0 con 0x7effb0133e30 2023-10-03T10:18:59.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.728+0000 7effa77fe700 1 -- 172.21.15.3:0/3495730345 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7eff98040b60 con 0x7effb0133e30 2023-10-03T10:18:59.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.733+0000 7effa77fe700 1 -- 172.21.15.3:0/3495730345 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7eff9802e3f0 con 0x7effb0133e30 2023-10-03T10:18:59.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.838+0000 7f6599535700 1 -- 172.21.15.3:0/2435466620 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6594074ea0 con 0x7f65941338c0 2023-10-03T10:18:59.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.838+0000 7f657b7fe700 1 -- 172.21.15.3:0/2435466620 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6588011590 con 0x7f65941338c0 2023-10-03T10:18:59.839 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:18:59.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.841+0000 7f6599535700 1 -- 172.21.15.3:0/2435466620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65941338c0 msgr2=0x7f6594120090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:59.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.841+0000 7f6599535700 1 --2- 172.21.15.3:0/2435466620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65941338c0 0x7f6594120090 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f6588000f60 tx=0x7f658800c730 comp rx=0 tx=0).stop 2023-10-03T10:18:59.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.841+0000 7f6599535700 1 -- 172.21.15.3:0/2435466620 shutdown_connections 2023-10-03T10:18:59.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.841+0000 7f6599535700 1 --2- 172.21.15.3:0/2435466620 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65941341d0 0x7f65941205d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.841+0000 7f6599535700 1 --2- 172.21.15.3:0/2435466620 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f659409e560 0x7f659411fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.841+0000 7f6599535700 1 --2- 172.21.15.3:0/2435466620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65941338c0 0x7f6594120090 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.841+0000 7f6599535700 1 -- 172.21.15.3:0/2435466620 >> 172.21.15.3:0/2435466620 conn(0x7f65940a2aa0 msgr2=0x7f659409fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:59.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.841+0000 7f6599535700 1 -- 172.21.15.3:0/2435466620 shutdown_connections 2023-10-03T10:18:59.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.841+0000 7f6599535700 1 -- 172.21.15.3:0/2435466620 wait complete. 2023-10-03T10:18:59.860 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:18:59.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.873+0000 7eff933e1700 1 -- 172.21.15.3:0/1180578859 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7eff8c074ea0 con 0x7eff8c133420 2023-10-03T10:18:59.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.873+0000 7eff81ffb700 1 -- 172.21.15.3:0/1180578859 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7eff880123d0 con 0x7eff8c133420 2023-10-03T10:18:59.873 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:18:59.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.875+0000 7eff933e1700 1 -- 172.21.15.3:0/1180578859 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff8c133420 msgr2=0x7eff8c09c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:59.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.875+0000 7eff933e1700 1 --2- 172.21.15.3:0/1180578859 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff8c133420 0x7eff8c09c480 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7eff8800e910 tx=0x7eff8800ec20 comp rx=0 tx=0).stop 2023-10-03T10:18:59.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.875+0000 7eff933e1700 1 -- 172.21.15.3:0/1180578859 shutdown_connections 2023-10-03T10:18:59.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.875+0000 7eff933e1700 1 --2- 172.21.15.3:0/1180578859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff8c134260 0x7eff8c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.875+0000 7eff933e1700 1 --2- 172.21.15.3:0/1180578859 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7eff8c133420 0x7eff8c09c480 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.875+0000 7eff933e1700 1 --2- 172.21.15.3:0/1180578859 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7eff8c133e30 0x7eff8c09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.875+0000 7eff933e1700 1 -- 172.21.15.3:0/1180578859 >> 172.21.15.3:0/1180578859 conn(0x7eff8c0a4210 msgr2=0x7eff8c0a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:59.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.876+0000 7eff933e1700 1 -- 172.21.15.3:0/1180578859 shutdown_connections 2023-10-03T10:18:59.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.876+0000 7eff933e1700 1 -- 172.21.15.3:0/1180578859 wait complete. 2023-10-03T10:18:59.889 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:18:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.942+0000 7fae9dd60700 1 -- 172.21.15.3:0/734957265 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fae981a9360 con 0x7fae9809d6f0 2023-10-03T10:18:59.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.943+0000 7fae7ffff700 1 -- 172.21.15.3:0/734957265 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fae981a9360 con 0x7fae9809d6f0 2023-10-03T10:18:59.943 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:18:59.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.945+0000 7fae9dd60700 1 -- 172.21.15.3:0/734957265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae9809d6f0 msgr2=0x7fae981a5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:59.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.945+0000 7fae9dd60700 1 --2- 172.21.15.3:0/734957265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae9809d6f0 0x7fae981a5e80 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fae88009b90 tx=0x7fae88004750 comp rx=0 tx=0).stop 2023-10-03T10:18:59.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.945+0000 7fae9dd60700 1 -- 172.21.15.3:0/734957265 shutdown_connections 2023-10-03T10:18:59.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.945+0000 7fae9dd60700 1 --2- 172.21.15.3:0/734957265 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fae9809e410 0x7fae981a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.945+0000 7fae9dd60700 1 --2- 172.21.15.3:0/734957265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae9809d6f0 0x7fae981a5e80 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.945+0000 7fae9dd60700 1 --2- 172.21.15.3:0/734957265 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fae9809e000 0x7fae981a63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.945+0000 7fae9dd60700 1 -- 172.21.15.3:0/734957265 >> 172.21.15.3:0/734957265 conn(0x7fae98066f50 msgr2=0x7fae981bad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:59.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.945+0000 7fae9dd60700 1 -- 172.21.15.3:0/734957265 shutdown_connections 2023-10-03T10:18:59.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.945+0000 7fae9dd60700 1 -- 172.21.15.3:0/734957265 wait complete. 2023-10-03T10:18:59.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.949+0000 7effa57fa700 1 -- 172.21.15.3:0/3495730345 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7effb011fe60 con 0x7effb0133e30 2023-10-03T10:18:59.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.949+0000 7effa77fe700 1 -- 172.21.15.3:0/3495730345 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7eff98015020 con 0x7effb0133e30 2023-10-03T10:18:59.949 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:18:59.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.951+0000 7effa57fa700 1 -- 172.21.15.3:0/3495730345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effb0133e30 msgr2=0x7effb009c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:18:59.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.951+0000 7effa57fa700 1 --2- 172.21.15.3:0/3495730345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effb0133e30 0x7effb009c9c0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7eff98016000 tx=0x7eff98004750 comp rx=0 tx=0).stop 2023-10-03T10:18:59.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.951+0000 7effa57fa700 1 -- 172.21.15.3:0/3495730345 shutdown_connections 2023-10-03T10:18:59.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.951+0000 7effa57fa700 1 --2- 172.21.15.3:0/3495730345 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7effb0134260 0x7effb009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.951+0000 7effa57fa700 1 --2- 172.21.15.3:0/3495730345 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7effb0133420 0x7effb009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.951+0000 7effa57fa700 1 --2- 172.21.15.3:0/3495730345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effb0133e30 0x7effb009c9c0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:18:59.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.951+0000 7effa57fa700 1 -- 172.21.15.3:0/3495730345 >> 172.21.15.3:0/3495730345 conn(0x7effb00a4210 msgr2=0x7effb00a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:18:59.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.952+0000 7effa57fa700 1 -- 172.21.15.3:0/3495730345 shutdown_connections 2023-10-03T10:18:59.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:18:59.952+0000 7effa57fa700 1 -- 172.21.15.3:0/3495730345 wait complete. 2023-10-03T10:18:59.960 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:18:59.966 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:00.861 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:00.890 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:00.960 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:00.967 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:01.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.010+0000 7ffb90c46700 1 Processor -- start 2023-10-03T10:19:01.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.010+0000 7ffb90c46700 1 -- start start 2023-10-03T10:19:01.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.011+0000 7ffb90c46700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffb8c09d6f0 0x7ffb8c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:01.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.011+0000 7ffb90c46700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffb8c09e410 0x7ffb8c134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:01.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.011+0000 7ffb90c46700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb8c09b120 con 0x7ffb8c09e000 2023-10-03T10:19:01.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.011+0000 7ffb90c46700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c06b460 con 0x7ffb8c09e410 2023-10-03T10:19:01.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.011+0000 7ffb90c46700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c09a7e0 con 0x7ffb8c09d6f0 2023-10-03T10:19:01.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.011+0000 7ffb8a59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffb8c09d6f0 0x7ffb8c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:01.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.011+0000 7ffb89d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ffb8c09e000 0x7ffb8c1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33520/0 (socket says 172.21.15.3:33520) 2023-10-03T10:19:01.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.011+0000 7ffb89d9b700 1 -- 172.21.15.3:0/2373740808 learned_addr learned my addr 172.21.15.3:0/2373740808 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:01.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.011+0000 7ffb8ad9d700 1 --2- 172.21.15.3:0/2373740808 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffb8c09e410 0x7ffb8c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:01.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.012+0000 7ffb8a59c700 1 -- 172.21.15.3:0/2373740808 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffb8c09e410 msgr2=0x7ffb8c134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:01.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.012+0000 7ffb8a59c700 1 --2- 172.21.15.3:0/2373740808 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffb8c09e410 0x7ffb8c134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.012+0000 7ffb8a59c700 1 -- 172.21.15.3:0/2373740808 >> v1:172.21.15.3:6789/0 conn(0x7ffb8c09e000 legacy=0x7ffb8c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:01.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.012+0000 7ffb8a59c700 1 -- 172.21.15.3:0/2373740808 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c09a500 con 0x7ffb8c09d6f0 2023-10-03T10:19:01.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.012+0000 7ffb8a59c700 1 --2- 172.21.15.3:0/2373740808 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffb8c09d6f0 0x7ffb8c09dac0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0098a0 tx=0x7ffb7c009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c38a66dad8a6e3b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:01.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.012+0000 7ffb8959a700 1 -- 172.21.15.3:0/2373740808 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c011830 con 0x7ffb8c09d6f0 2023-10-03T10:19:01.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.012+0000 7ffb8959a700 1 -- 172.21.15.3:0/2373740808 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7c011990 con 0x7ffb8c09d6f0 2023-10-03T10:19:01.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.012+0000 7ffb8959a700 1 -- 172.21.15.3:0/2373740808 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c011d20 con 0x7ffb8c09d6f0 2023-10-03T10:19:01.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.013+0000 7ffb90c46700 1 -- 172.21.15.3:0/2373740808 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffb8c09d6f0 msgr2=0x7ffb8c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:01.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.013+0000 7ffb90c46700 1 --2- 172.21.15.3:0/2373740808 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffb8c09d6f0 0x7ffb8c09dac0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0098a0 tx=0x7ffb7c009bb0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.013+0000 7ffb90c46700 1 -- 172.21.15.3:0/2373740808 shutdown_connections 2023-10-03T10:19:01.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.013+0000 7ffb90c46700 1 --2- 172.21.15.3:0/2373740808 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffb8c09e410 0x7ffb8c134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.013+0000 7ffb90c46700 1 --2- 172.21.15.3:0/2373740808 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffb8c09d6f0 0x7ffb8c09dac0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.013+0000 7ffb90c46700 1 -- 172.21.15.3:0/2373740808 >> 172.21.15.3:0/2373740808 conn(0x7ffb8c066f50 msgr2=0x7ffb8c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:01.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.013+0000 7ffb90c46700 1 -- 172.21.15.3:0/2373740808 shutdown_connections 2023-10-03T10:19:01.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.013+0000 7ffb90c46700 1 -- 172.21.15.3:0/2373740808 wait complete. 2023-10-03T10:19:01.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.015+0000 7ffb90c46700 1 Processor -- start 2023-10-03T10:19:01.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.015+0000 7ffb90c46700 1 -- start start 2023-10-03T10:19:01.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.015+0000 7ffb90c46700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffb8c09d6f0 0x7ffb8c122580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:01.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.015+0000 7ffb90c46700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffb8c09e000 0x7ffb8c122ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:01.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.015+0000 7ffb90c46700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb8c09e410 0x7ffb8c123000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:01.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.015+0000 7ffb90c46700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c125b00 con 0x7ffb8c09e410 2023-10-03T10:19:01.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.015+0000 7ffb90c46700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c1260a0 con 0x7ffb8c09e000 2023-10-03T10:19:01.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.015+0000 7ffb90c46700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c1249c0 con 0x7ffb8c09d6f0 2023-10-03T10:19:01.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.015+0000 7ffb8a59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffb8c09d6f0 0x7ffb8c122580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:01.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.015+0000 7ffb8a59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffb8c09d6f0 0x7ffb8c122580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48080/0 (socket says 172.21.15.3:48080) 2023-10-03T10:19:01.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.015+0000 7ffb8a59c700 1 -- 172.21.15.3:0/3178888456 learned_addr learned my addr 172.21.15.3:0/3178888456 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:01.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.015+0000 7ffb8ad9d700 1 --2- 172.21.15.3:0/3178888456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb8c09e410 0x7ffb8c123000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:01.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.016+0000 7ffb89d9b700 1 --2- 172.21.15.3:0/3178888456 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffb8c09e000 0x7ffb8c122ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:01.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.016+0000 7ffb8a59c700 1 -- 172.21.15.3:0/3178888456 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffb8c09e000 msgr2=0x7ffb8c122ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:01.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.016+0000 7ffb8a59c700 1 --2- 172.21.15.3:0/3178888456 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffb8c09e000 0x7ffb8c122ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.016+0000 7ffb8a59c700 1 -- 172.21.15.3:0/3178888456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb8c09e410 msgr2=0x7ffb8c123000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:01.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.016+0000 7ffb8a59c700 1 --2- 172.21.15.3:0/3178888456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb8c09e410 0x7ffb8c123000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.016+0000 7ffb8a59c700 1 -- 172.21.15.3:0/3178888456 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb7c0094a0 con 0x7ffb8c09d6f0 2023-10-03T10:19:01.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.016+0000 7ffb89d9b700 1 --2- 172.21.15.3:0/3178888456 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffb8c09e000 0x7ffb8c122ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:01.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.016+0000 7ffb8ad9d700 1 --2- 172.21.15.3:0/3178888456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb8c09e410 0x7ffb8c123000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:01.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.016+0000 7ffb8a59c700 1 --2- 172.21.15.3:0/3178888456 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffb8c09d6f0 0x7ffb8c122580 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0090b0 tx=0x7ffb7c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:01.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.016+0000 7ffb7affd700 1 -- 172.21.15.3:0/3178888456 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c004550 con 0x7ffb8c09d6f0 2023-10-03T10:19:01.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.016+0000 7ffb7affd700 1 -- 172.21.15.3:0/3178888456 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7c01f070 con 0x7ffb8c09d6f0 2023-10-03T10:19:01.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.016+0000 7ffb7affd700 1 -- 172.21.15.3:0/3178888456 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c0038e0 con 0x7ffb8c09d6f0 2023-10-03T10:19:01.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.017+0000 7ffb90c46700 1 -- 172.21.15.3:0/3178888456 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb8c0a1d00 con 0x7ffb8c09d6f0 2023-10-03T10:19:01.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.017+0000 7ffb90c46700 1 -- 172.21.15.3:0/3178888456 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8c125dc0 con 0x7ffb8c09d6f0 2023-10-03T10:19:01.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.017+0000 7ffb7affd700 1 -- 172.21.15.3:0/3178888456 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffb7c003a40 con 0x7ffb8c09d6f0 2023-10-03T10:19:01.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.018+0000 7ffb7affd700 1 -- 172.21.15.3:0/3178888456 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ffb7c021070 con 0x7ffb8c09d6f0 2023-10-03T10:19:01.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.018+0000 7ffb90c46700 1 -- 172.21.15.3:0/3178888456 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb8c074ea0 con 0x7ffb8c09d6f0 2023-10-03T10:19:01.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.024+0000 7ffb7affd700 1 -- 172.21.15.3:0/3178888456 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffb7c0253e0 con 0x7ffb8c09d6f0 2023-10-03T10:19:01.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.053+0000 7fbf7176c700 1 Processor -- start 2023-10-03T10:19:01.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.053+0000 7fbf7176c700 1 -- start start 2023-10-03T10:19:01.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.053+0000 7fbf7176c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf6c130540 0x7fbf6c130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:01.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.053+0000 7fbf7176c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbf6c130e50 0x7fbf6c06f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:01.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.053+0000 7fbf7176c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf6c06fb20 0x7fbf6c06ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:01.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.053+0000 7fbf7176c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf6c0704b0 con 0x7fbf6c130e50 2023-10-03T10:19:01.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.053+0000 7fbf7176c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c09a390 con 0x7fbf6c130540 2023-10-03T10:19:01.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.053+0000 7fbf7176c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c0716a0 con 0x7fbf6c06fb20 2023-10-03T10:19:01.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.054+0000 7fbf6affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf6c130540 0x7fbf6c130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:01.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.054+0000 7fbf6affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf6c130540 0x7fbf6c130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42354/0 (socket says 172.21.15.3:42354) 2023-10-03T10:19:01.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.054+0000 7fbf6affd700 1 -- 172.21.15.3:0/792923148 learned_addr learned my addr 172.21.15.3:0/792923148 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:01.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.054+0000 7fbf6b7fe700 1 --2- 172.21.15.3:0/792923148 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf6c06fb20 0x7fbf6c06ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:01.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.054+0000 7fbf6a7fc700 1 --2- 172.21.15.3:0/792923148 >> v2:172.21.15.3:3300/0 conn(0x7fbf6c130e50 0x7fbf6c06f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:01.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.054+0000 7fbf6affd700 1 -- 172.21.15.3:0/792923148 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf6c06fb20 msgr2=0x7fbf6c06ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:01.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.054+0000 7fbf6affd700 1 --2- 172.21.15.3:0/792923148 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf6c06fb20 0x7fbf6c06ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.054+0000 7fbf6affd700 1 -- 172.21.15.3:0/792923148 >> v2:172.21.15.3:3300/0 conn(0x7fbf6c130e50 msgr2=0x7fbf6c06f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:01.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.054+0000 7fbf6affd700 1 --2- 172.21.15.3:0/792923148 >> v2:172.21.15.3:3300/0 conn(0x7fbf6c130e50 0x7fbf6c06f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.054+0000 7fbf6affd700 1 -- 172.21.15.3:0/792923148 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf6c071290 con 0x7fbf6c130540 2023-10-03T10:19:01.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.054+0000 7fbf6b7fe700 1 --2- 172.21.15.3:0/792923148 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf6c06fb20 0x7fbf6c06ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:01.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.054+0000 7fbf6a7fc700 1 --2- 172.21.15.3:0/792923148 >> v2:172.21.15.3:3300/0 conn(0x7fbf6c130e50 0x7fbf6c06f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:19:01.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.054+0000 7fbf6affd700 1 --2- 172.21.15.3:0/792923148 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf6c130540 0x7fbf6c130910 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00bb90 tx=0x7fbf5c00bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c71ce09e554d2466 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:01.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.055+0000 7fbf69ffb700 1 -- 172.21.15.3:0/792923148 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c00dd30 con 0x7fbf6c130540 2023-10-03T10:19:01.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.055+0000 7fbf69ffb700 1 -- 172.21.15.3:0/792923148 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5c00de90 con 0x7fbf6c130540 2023-10-03T10:19:01.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.055+0000 7fbf69ffb700 1 -- 172.21.15.3:0/792923148 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c010690 con 0x7fbf6c130540 2023-10-03T10:19:01.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.055+0000 7fbf7176c700 1 -- 172.21.15.3:0/792923148 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf6c130540 msgr2=0x7fbf6c130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:01.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.055+0000 7fbf7176c700 1 --2- 172.21.15.3:0/792923148 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf6c130540 0x7fbf6c130910 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00bb90 tx=0x7fbf5c00bea0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.056+0000 7fbf7176c700 1 -- 172.21.15.3:0/792923148 shutdown_connections 2023-10-03T10:19:01.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.056+0000 7fbf7176c700 1 --2- 172.21.15.3:0/792923148 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf6c06fb20 0x7fbf6c06ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.056+0000 7fbf7176c700 1 --2- 172.21.15.3:0/792923148 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf6c130540 0x7fbf6c130910 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.056+0000 7fbf7176c700 1 --2- 172.21.15.3:0/792923148 >> v2:172.21.15.3:3300/0 conn(0x7fbf6c130e50 0x7fbf6c06f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.056+0000 7fbf7176c700 1 -- 172.21.15.3:0/792923148 >> 172.21.15.3:0/792923148 conn(0x7fbf6c0a2aa0 msgr2=0x7fbf6c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:01.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.056+0000 7fbf7176c700 1 -- 172.21.15.3:0/792923148 shutdown_connections 2023-10-03T10:19:01.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.056+0000 7fbf7176c700 1 -- 172.21.15.3:0/792923148 wait complete. 2023-10-03T10:19:01.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.057+0000 7fbf7176c700 1 Processor -- start 2023-10-03T10:19:01.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.057+0000 7fbf7176c700 1 -- start start 2023-10-03T10:19:01.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.057+0000 7fbf7176c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf6c06fb20 0x7fbf6c11a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:01.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.057+0000 7fbf7176c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf6c130540 0x7fbf6c11aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:01.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.057+0000 7fbf7176c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf6c130e50 0x7fbf6c11af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:01.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.057+0000 7fbf7176c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c11c250 con 0x7fbf6c130e50 2023-10-03T10:19:01.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.057+0000 7fbf7176c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c11d390 con 0x7fbf6c06fb20 2023-10-03T10:19:01.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.057+0000 7fbf7176c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c0718e0 con 0x7fbf6c130540 2023-10-03T10:19:01.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.058+0000 7fbf6affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf6c06fb20 0x7fbf6c11a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:01.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.058+0000 7fbf6affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf6c06fb20 0x7fbf6c11a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42356/0 (socket says 172.21.15.3:42356) 2023-10-03T10:19:01.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.058+0000 7fbf6affd700 1 -- 172.21.15.3:0/3073484528 learned_addr learned my addr 172.21.15.3:0/3073484528 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:01.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.058+0000 7fbf6a7fc700 1 --2- 172.21.15.3:0/3073484528 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf6c130540 0x7fbf6c11aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:01.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.058+0000 7fbf6b7fe700 1 --2- 172.21.15.3:0/3073484528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf6c130e50 0x7fbf6c11af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:01.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.058+0000 7fbf6affd700 1 -- 172.21.15.3:0/3073484528 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf6c130540 msgr2=0x7fbf6c11aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:01.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.058+0000 7fbf6affd700 1 --2- 172.21.15.3:0/3073484528 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf6c130540 0x7fbf6c11aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.058+0000 7fbf6affd700 1 -- 172.21.15.3:0/3073484528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf6c130e50 msgr2=0x7fbf6c11af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:01.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.058+0000 7fbf6affd700 1 --2- 172.21.15.3:0/3073484528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf6c130e50 0x7fbf6c11af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.058+0000 7fbf6affd700 1 -- 172.21.15.3:0/3073484528 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf64009a70 con 0x7fbf6c06fb20 2023-10-03T10:19:01.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.058+0000 7fbf6affd700 1 --2- 172.21.15.3:0/3073484528 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf6c06fb20 0x7fbf6c11a510 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c000c00 tx=0x7fbf5c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:01.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.059+0000 7fbf537fe700 1 -- 172.21.15.3:0/3073484528 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c003ab0 con 0x7fbf6c06fb20 2023-10-03T10:19:01.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.059+0000 7fbf537fe700 1 -- 172.21.15.3:0/3073484528 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5c01e070 con 0x7fbf6c06fb20 2023-10-03T10:19:01.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.059+0000 7fbf537fe700 1 -- 172.21.15.3:0/3073484528 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c003e10 con 0x7fbf6c06fb20 2023-10-03T10:19:01.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.059+0000 7fbf7176c700 1 -- 172.21.15.3:0/3073484528 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf5c00b7f0 con 0x7fbf6c06fb20 2023-10-03T10:19:01.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.059+0000 7fbf7176c700 1 -- 172.21.15.3:0/3073484528 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf6c11ced0 con 0x7fbf6c06fb20 2023-10-03T10:19:01.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.060+0000 7fbf7176c700 1 -- 172.21.15.3:0/3073484528 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf6c09cd40 con 0x7fbf6c06fb20 2023-10-03T10:19:01.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.061+0000 7fbf537fe700 1 -- 172.21.15.3:0/3073484528 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbf5c019400 con 0x7fbf6c06fb20 2023-10-03T10:19:01.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.061+0000 7fbf537fe700 1 -- 172.21.15.3:0/3073484528 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fbf5c042ec0 con 0x7fbf6c06fb20 2023-10-03T10:19:01.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.066+0000 7fbf537fe700 1 -- 172.21.15.3:0/3073484528 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbf5c0233d0 con 0x7fbf6c06fb20 2023-10-03T10:19:01.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.108+0000 7f03a2cf1700 1 Processor -- start 2023-10-03T10:19:01.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.108+0000 7f03a2cf1700 1 -- start start 2023-10-03T10:19:01.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.109+0000 7f03a2cf1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f039c09d6f0 0x7f039c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:01.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.109+0000 7f03a2cf1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f039c09e000 0x7f039c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:01.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.109+0000 7f03a2cf1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f039c09e410 0x7f039c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:01.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.109+0000 7f03a2cf1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f039c06b460 con 0x7f039c09d6f0 2023-10-03T10:19:01.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.109+0000 7f03a2cf1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039c09b120 con 0x7f039c09e410 2023-10-03T10:19:01.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.109+0000 7f03a2cf1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f039c09a7e0 con 0x7f039c09e000 2023-10-03T10:19:01.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.109+0000 7f039bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f039c09e000 0x7f039c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:01.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.109+0000 7f039bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f039c09e000 0x7f039c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48116/0 (socket says 172.21.15.3:48116) 2023-10-03T10:19:01.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.109+0000 7f039bfff700 1 -- 172.21.15.3:0/3206379646 learned_addr learned my addr 172.21.15.3:0/3206379646 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:01.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.109+0000 7f03a128e700 1 --2- 172.21.15.3:0/3206379646 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f039c09e410 0x7f039c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:01.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.109+0000 7f03a0a8d700 1 --2- 172.21.15.3:0/3206379646 >> v2:172.21.15.3:3300/0 conn(0x7f039c09d6f0 0x7f039c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:01.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.110+0000 7f039bfff700 1 -- 172.21.15.3:0/3206379646 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f039c09e410 msgr2=0x7f039c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:01.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.110+0000 7f039bfff700 1 --2- 172.21.15.3:0/3206379646 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f039c09e410 0x7f039c133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.110+0000 7f039bfff700 1 -- 172.21.15.3:0/3206379646 >> v2:172.21.15.3:3300/0 conn(0x7f039c09d6f0 msgr2=0x7f039c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:01.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.110+0000 7f039bfff700 1 --2- 172.21.15.3:0/3206379646 >> v2:172.21.15.3:3300/0 conn(0x7f039c09d6f0 0x7f039c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.110+0000 7f039bfff700 1 -- 172.21.15.3:0/3206379646 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c09a500 con 0x7f039c09e000 2023-10-03T10:19:01.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.110+0000 7f039bfff700 1 --2- 172.21.15.3:0/3206379646 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f039c09e000 0x7f039c1339f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f0388009920 tx=0x7f0388009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c82f9ebebc02bc65 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:01.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.110+0000 7f039b7fe700 1 -- 172.21.15.3:0/3206379646 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0388011830 con 0x7f039c09e000 2023-10-03T10:19:01.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.110+0000 7f039b7fe700 1 -- 172.21.15.3:0/3206379646 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0388011990 con 0x7f039c09e000 2023-10-03T10:19:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.110+0000 7f039b7fe700 1 -- 172.21.15.3:0/3206379646 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0388011d20 con 0x7f039c09e000 2023-10-03T10:19:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.111+0000 7f03a2cf1700 1 -- 172.21.15.3:0/3206379646 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f039c09e000 msgr2=0x7f039c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.111+0000 7f03a2cf1700 1 --2- 172.21.15.3:0/3206379646 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f039c09e000 0x7f039c1339f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f0388009920 tx=0x7f0388009c30 comp rx=0 tx=0).stop 2023-10-03T10:19:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.111+0000 7f03a2cf1700 1 -- 172.21.15.3:0/3206379646 shutdown_connections 2023-10-03T10:19:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.111+0000 7f03a2cf1700 1 --2- 172.21.15.3:0/3206379646 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f039c09e410 0x7f039c133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.111+0000 7f03a2cf1700 1 --2- 172.21.15.3:0/3206379646 >> v2:172.21.15.3:3300/0 conn(0x7f039c09d6f0 0x7f039c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.111+0000 7f03a2cf1700 1 --2- 172.21.15.3:0/3206379646 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f039c09e000 0x7f039c1339f0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.111+0000 7f03a2cf1700 1 -- 172.21.15.3:0/3206379646 >> 172.21.15.3:0/3206379646 conn(0x7f039c066f50 msgr2=0x7f039c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.111+0000 7f03a2cf1700 1 -- 172.21.15.3:0/3206379646 shutdown_connections 2023-10-03T10:19:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.111+0000 7f03a2cf1700 1 -- 172.21.15.3:0/3206379646 wait complete. 2023-10-03T10:19:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.112+0000 7f03a2cf1700 1 Processor -- start 2023-10-03T10:19:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.112+0000 7f03a2cf1700 1 -- start start 2023-10-03T10:19:01.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.113+0000 7f03a2cf1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f039c09d6f0 0x7f039c1ab9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:01.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.113+0000 7f03a2cf1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f039c09e000 0x7f039c1abee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:01.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.113+0000 7f03a2cf1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f039c09e410 0x7f039c1ac420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:01.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.113+0000 7f03a2cf1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c1ade30 con 0x7f039c09e410 2023-10-03T10:19:01.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.113+0000 7f03a2cf1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039c1bad90 con 0x7f039c09d6f0 2023-10-03T10:19:01.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.113+0000 7f03a2cf1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f039c1ace00 con 0x7f039c09e000 2023-10-03T10:19:01.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.113+0000 7f03a128e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f039c09e410 0x7f039c1ac420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:01.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.113+0000 7f03a128e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f039c09e410 0x7f039c1ac420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49866/0 (socket says 172.21.15.3:49866) 2023-10-03T10:19:01.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.113+0000 7f03a128e700 1 -- 172.21.15.3:0/871323081 learned_addr learned my addr 172.21.15.3:0/871323081 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:01.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.113+0000 7f039bfff700 1 --2- 172.21.15.3:0/871323081 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f039c09e000 0x7f039c1abee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:01.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.113+0000 7f03a0a8d700 1 --2- 172.21.15.3:0/871323081 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f039c09d6f0 0x7f039c1ab9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:01.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.114+0000 7f03a128e700 1 -- 172.21.15.3:0/871323081 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f039c09e000 msgr2=0x7f039c1abee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:01.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.114+0000 7f03a128e700 1 --2- 172.21.15.3:0/871323081 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f039c09e000 0x7f039c1abee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.114+0000 7f03a128e700 1 -- 172.21.15.3:0/871323081 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f039c09d6f0 msgr2=0x7f039c1ab9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:01.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.114+0000 7f03a128e700 1 --2- 172.21.15.3:0/871323081 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f039c09d6f0 0x7f039c1ab9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.114+0000 7f03a128e700 1 -- 172.21.15.3:0/871323081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0388009550 con 0x7f039c09e410 2023-10-03T10:19:01.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.114+0000 7f03a128e700 1 --2- 172.21.15.3:0/871323081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f039c09e410 0x7f039c1ac420 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f0394009dd0 tx=0x7f039400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:01.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.114+0000 7f03997fa700 1 -- 172.21.15.3:0/871323081 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0394009c40 con 0x7f039c09e410 2023-10-03T10:19:01.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.114+0000 7f03997fa700 1 -- 172.21.15.3:0/871323081 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039400f040 con 0x7f039c09e410 2023-10-03T10:19:01.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.114+0000 7f03a2cf1700 1 -- 172.21.15.3:0/871323081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f039c1af470 con 0x7f039c09e410 2023-10-03T10:19:01.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.114+0000 7f03997fa700 1 -- 172.21.15.3:0/871323081 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039400ceb0 con 0x7f039c09e410 2023-10-03T10:19:01.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.114+0000 7f03a2cf1700 1 -- 172.21.15.3:0/871323081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f039c1ae7b0 con 0x7f039c09e410 2023-10-03T10:19:01.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.116+0000 7f03a2cf1700 1 -- 172.21.15.3:0/871323081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f039c074ea0 con 0x7f039c09e410 2023-10-03T10:19:01.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.116+0000 7f03997fa700 1 -- 172.21.15.3:0/871323081 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0394014480 con 0x7f039c09e410 2023-10-03T10:19:01.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.116+0000 7f03997fa700 1 -- 172.21.15.3:0/871323081 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f039403fdf0 con 0x7f039c09e410 2023-10-03T10:19:01.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.125+0000 7f03997fa700 1 -- 172.21.15.3:0/871323081 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f039401a8b0 con 0x7f039c09e410 2023-10-03T10:19:01.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.160+0000 7f96a60ba700 1 Processor -- start 2023-10-03T10:19:01.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.160+0000 7f96a60ba700 1 -- start start 2023-10-03T10:19:01.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.161+0000 7f96a60ba700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f96a00a0a30 0x7f96a00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:01.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.161+0000 7f96a60ba700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f96a009d630 0x7f96a00a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:01.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.161+0000 7f96a60ba700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96a00a2a70 con 0x7f96a0133930 2023-10-03T10:19:01.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.161+0000 7f96a60ba700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a00a01d0 con 0x7f96a00a0a30 2023-10-03T10:19:01.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.161+0000 7f96a60ba700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f96a009e2b0 con 0x7f96a009d630 2023-10-03T10:19:01.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.161+0000 7f969ffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f96a0133930 0x7f96a0133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33534/0 (socket says 172.21.15.3:33534) 2023-10-03T10:19:01.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.162+0000 7f969ffff700 1 -- 172.21.15.3:0/3030045646 learned_addr learned my addr 172.21.15.3:0/3030045646 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:01.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.162+0000 7f969effd700 1 --2- 172.21.15.3:0/3030045646 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f96a009d630 0x7f96a00a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:01.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.162+0000 7f969f7fe700 1 --2- 172.21.15.3:0/3030045646 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f96a00a0a30 0x7f96a00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:01.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.162+0000 7f969effd700 1 -- 172.21.15.3:0/3030045646 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f96a00a0a30 msgr2=0x7f96a00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:01.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.162+0000 7f969effd700 1 --2- 172.21.15.3:0/3030045646 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f96a00a0a30 0x7f96a00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.162+0000 7f969effd700 1 -- 172.21.15.3:0/3030045646 >> v1:172.21.15.3:6789/0 conn(0x7f96a0133930 legacy=0x7f96a0133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:01.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.162+0000 7f969effd700 1 -- 172.21.15.3:0/3030045646 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a009dc50 con 0x7f96a009d630 2023-10-03T10:19:01.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.162+0000 7f969f7fe700 1 --2- 172.21.15.3:0/3030045646 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f96a00a0a30 0x7f96a00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:01.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.162+0000 7f969effd700 1 --2- 172.21.15.3:0/3030045646 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f96a009d630 0x7f96a00a1340 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9694009870 tx=0x7f9694009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4256f275ab66ff5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:01.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.162+0000 7f969e7fc700 1 -- 172.21.15.3:0/3030045646 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9694011830 con 0x7f96a009d630 2023-10-03T10:19:01.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.163+0000 7f969e7fc700 1 -- 172.21.15.3:0/3030045646 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9694011990 con 0x7f96a009d630 2023-10-03T10:19:01.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.163+0000 7f969e7fc700 1 -- 172.21.15.3:0/3030045646 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9694011d00 con 0x7f96a009d630 2023-10-03T10:19:01.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.163+0000 7f96a60ba700 1 -- 172.21.15.3:0/3030045646 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f96a009d630 msgr2=0x7f96a00a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:01.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.163+0000 7f96a60ba700 1 --2- 172.21.15.3:0/3030045646 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f96a009d630 0x7f96a00a1340 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9694009870 tx=0x7f9694009b80 comp rx=0 tx=0).stop 2023-10-03T10:19:01.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.163+0000 7f96a60ba700 1 -- 172.21.15.3:0/3030045646 shutdown_connections 2023-10-03T10:19:01.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.163+0000 7f96a60ba700 1 --2- 172.21.15.3:0/3030045646 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f96a00a0a30 0x7f96a00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.163+0000 7f96a60ba700 1 --2- 172.21.15.3:0/3030045646 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f96a009d630 0x7f96a00a1340 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.164+0000 7f96a60ba700 1 -- 172.21.15.3:0/3030045646 >> 172.21.15.3:0/3030045646 conn(0x7f96a00a3050 msgr2=0x7f96a00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:01.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.164+0000 7f96a60ba700 1 -- 172.21.15.3:0/3030045646 shutdown_connections 2023-10-03T10:19:01.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.164+0000 7f96a60ba700 1 -- 172.21.15.3:0/3030045646 wait complete. 2023-10-03T10:19:01.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.165+0000 7f96a60ba700 1 Processor -- start 2023-10-03T10:19:01.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.165+0000 7f96a60ba700 1 -- start start 2023-10-03T10:19:01.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.166+0000 7f96a60ba700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f96a009d630 0x7f96a011f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:01.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.166+0000 7f96a60ba700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96a00a0a30 0x7f96a011fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:01.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.166+0000 7f96a60ba700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f96a0133930 0x7f96a01c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:01.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.166+0000 7f96a60ba700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a0122360 con 0x7f96a00a0a30 2023-10-03T10:19:01.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.166+0000 7f96a60ba700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a00a40a0 con 0x7f96a009d630 2023-10-03T10:19:01.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.166+0000 7f96a60ba700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f96a011f0d0 con 0x7f96a0133930 2023-10-03T10:19:01.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.166+0000 7f969f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f96a009d630 0x7f96a011f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:01.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.166+0000 7f969f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f96a009d630 0x7f96a011f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42390/0 (socket says 172.21.15.3:42390) 2023-10-03T10:19:01.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.166+0000 7f969f7fe700 1 -- 172.21.15.3:0/2743212278 learned_addr learned my addr 172.21.15.3:0/2743212278 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:01.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.166+0000 7f969effd700 1 --2- 172.21.15.3:0/2743212278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96a00a0a30 0x7f96a011fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:01.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.166+0000 7f969ffff700 1 --2- 172.21.15.3:0/2743212278 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f96a0133930 0x7f96a01c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:01.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.167+0000 7f969f7fe700 1 -- 172.21.15.3:0/2743212278 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f96a0133930 msgr2=0x7f96a01c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:01.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.167+0000 7f969f7fe700 1 --2- 172.21.15.3:0/2743212278 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f96a0133930 0x7f96a01c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.167+0000 7f969f7fe700 1 -- 172.21.15.3:0/2743212278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96a00a0a30 msgr2=0x7f96a011fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:01.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.167+0000 7f969f7fe700 1 --2- 172.21.15.3:0/2743212278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96a00a0a30 0x7f96a011fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.167+0000 7f969f7fe700 1 -- 172.21.15.3:0/2743212278 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96900094a0 con 0x7f96a009d630 2023-10-03T10:19:01.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.167+0000 7f969f7fe700 1 --2- 172.21.15.3:0/2743212278 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f96a009d630 0x7f96a011f540 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f969000ec10 tx=0x7f969000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:01.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.167+0000 7f9687fff700 1 -- 172.21.15.3:0/2743212278 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969000c850 con 0x7f96a009d630 2023-10-03T10:19:01.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.167+0000 7f9687fff700 1 -- 172.21.15.3:0/2743212278 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969000c9b0 con 0x7f96a009d630 2023-10-03T10:19:01.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.167+0000 7f9687fff700 1 -- 172.21.15.3:0/2743212278 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969000cd40 con 0x7f96a009d630 2023-10-03T10:19:01.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.167+0000 7f96a60ba700 1 -- 172.21.15.3:0/2743212278 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96940094a0 con 0x7f96a009d630 2023-10-03T10:19:01.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.167+0000 7f96a60ba700 1 -- 172.21.15.3:0/2743212278 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a0133620 con 0x7f96a009d630 2023-10-03T10:19:01.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.169+0000 7f9687fff700 1 -- 172.21.15.3:0/2743212278 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9690013470 con 0x7f96a009d630 2023-10-03T10:19:01.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.169+0000 7f96a60ba700 1 -- 172.21.15.3:0/2743212278 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96a01221c0 con 0x7f96a009d630 2023-10-03T10:19:01.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.169+0000 7f9687fff700 1 -- 172.21.15.3:0/2743212278 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f969003fc20 con 0x7f96a009d630 2023-10-03T10:19:01.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.178+0000 7f9687fff700 1 -- 172.21.15.3:0/2743212278 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f96a01221c0 con 0x7f96a009d630 2023-10-03T10:19:01.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.263+0000 7ffb90c46700 1 -- 172.21.15.3:0/3178888456 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ffb8c06c360 con 0x7ffb8c09d6f0 2023-10-03T10:19:01.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.264+0000 7ffb7affd700 1 -- 172.21.15.3:0/3178888456 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffb7c015020 con 0x7ffb8c09d6f0 2023-10-03T10:19:01.264 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:01.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.266+0000 7ffb90c46700 1 -- 172.21.15.3:0/3178888456 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffb8c09d6f0 msgr2=0x7ffb8c122580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:01.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.266+0000 7ffb90c46700 1 --2- 172.21.15.3:0/3178888456 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffb8c09d6f0 0x7ffb8c122580 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0090b0 tx=0x7ffb7c004750 comp rx=0 tx=0).stop 2023-10-03T10:19:01.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.266+0000 7ffb90c46700 1 -- 172.21.15.3:0/3178888456 shutdown_connections 2023-10-03T10:19:01.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.266+0000 7ffb90c46700 1 --2- 172.21.15.3:0/3178888456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb8c09e410 0x7ffb8c123000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.266+0000 7ffb90c46700 1 --2- 172.21.15.3:0/3178888456 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffb8c09d6f0 0x7ffb8c122580 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.266+0000 7ffb90c46700 1 --2- 172.21.15.3:0/3178888456 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffb8c09e000 0x7ffb8c122ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.266+0000 7ffb90c46700 1 -- 172.21.15.3:0/3178888456 >> 172.21.15.3:0/3178888456 conn(0x7ffb8c066f50 msgr2=0x7ffb8c06f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:01.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.267+0000 7ffb90c46700 1 -- 172.21.15.3:0/3178888456 shutdown_connections 2023-10-03T10:19:01.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.267+0000 7ffb90c46700 1 -- 172.21.15.3:0/3178888456 wait complete. 2023-10-03T10:19:01.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.279+0000 7fbf7176c700 1 -- 172.21.15.3:0/3073484528 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbf6c074ea0 con 0x7fbf6c06fb20 2023-10-03T10:19:01.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.280+0000 7fbf537fe700 1 -- 172.21.15.3:0/3073484528 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbf5c029020 con 0x7fbf6c06fb20 2023-10-03T10:19:01.280 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:01.281 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:01.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.282+0000 7fbf7176c700 1 -- 172.21.15.3:0/3073484528 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf6c06fb20 msgr2=0x7fbf6c11a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:01.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.282+0000 7fbf7176c700 1 --2- 172.21.15.3:0/3073484528 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf6c06fb20 0x7fbf6c11a510 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c000c00 tx=0x7fbf5c004750 comp rx=0 tx=0).stop 2023-10-03T10:19:01.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.282+0000 7fbf7176c700 1 -- 172.21.15.3:0/3073484528 shutdown_connections 2023-10-03T10:19:01.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.282+0000 7fbf7176c700 1 --2- 172.21.15.3:0/3073484528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf6c130e50 0x7fbf6c11af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.282+0000 7fbf7176c700 1 --2- 172.21.15.3:0/3073484528 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbf6c06fb20 0x7fbf6c11a510 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.282+0000 7fbf7176c700 1 --2- 172.21.15.3:0/3073484528 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbf6c130540 0x7fbf6c11aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.282+0000 7fbf7176c700 1 -- 172.21.15.3:0/3073484528 >> 172.21.15.3:0/3073484528 conn(0x7fbf6c0a2aa0 msgr2=0x7fbf6c12d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:01.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.282+0000 7fbf7176c700 1 -- 172.21.15.3:0/3073484528 shutdown_connections 2023-10-03T10:19:01.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.282+0000 7fbf7176c700 1 -- 172.21.15.3:0/3073484528 wait complete. 2023-10-03T10:19:01.301 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.345+0000 7f03a2cf1700 1 -- 172.21.15.3:0/871323081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f039c06c360 con 0x7f039c09e410 2023-10-03T10:19:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.345+0000 7f03997fa700 1 -- 172.21.15.3:0/871323081 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0394018020 con 0x7f039c09e410 2023-10-03T10:19:01.346 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:01.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.348+0000 7f03a2cf1700 1 -- 172.21.15.3:0/871323081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f039c09e410 msgr2=0x7f039c1ac420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:01.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.348+0000 7f03a2cf1700 1 --2- 172.21.15.3:0/871323081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f039c09e410 0x7f039c1ac420 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f0394009dd0 tx=0x7f039400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.348+0000 7f03a2cf1700 1 -- 172.21.15.3:0/871323081 shutdown_connections 2023-10-03T10:19:01.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.348+0000 7f03a2cf1700 1 --2- 172.21.15.3:0/871323081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f039c09e410 0x7f039c1ac420 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.348+0000 7f03a2cf1700 1 --2- 172.21.15.3:0/871323081 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f039c09d6f0 0x7f039c1ab9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.348+0000 7f03a2cf1700 1 --2- 172.21.15.3:0/871323081 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f039c09e000 0x7f039c1abee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.348+0000 7f03a2cf1700 1 -- 172.21.15.3:0/871323081 >> 172.21.15.3:0/871323081 conn(0x7f039c066f50 msgr2=0x7f039c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:01.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.348+0000 7f03a2cf1700 1 -- 172.21.15.3:0/871323081 shutdown_connections 2023-10-03T10:19:01.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.348+0000 7f03a2cf1700 1 -- 172.21.15.3:0/871323081 wait complete. 2023-10-03T10:19:01.361 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:01.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.386+0000 7f96a60ba700 1 -- 172.21.15.3:0/2743212278 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f96a01221c0 con 0x7f96a009d630 2023-10-03T10:19:01.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.386+0000 7f9687fff700 1 -- 172.21.15.3:0/2743212278 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f96a01221c0 con 0x7f96a009d630 2023-10-03T10:19:01.387 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:01.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.389+0000 7f96a60ba700 1 -- 172.21.15.3:0/2743212278 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f96a009d630 msgr2=0x7f96a011f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:01.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.389+0000 7f96a60ba700 1 --2- 172.21.15.3:0/2743212278 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f96a009d630 0x7f96a011f540 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f969000ec10 tx=0x7f969000ef20 comp rx=0 tx=0).stop 2023-10-03T10:19:01.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.389+0000 7f96a60ba700 1 -- 172.21.15.3:0/2743212278 shutdown_connections 2023-10-03T10:19:01.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.389+0000 7f96a60ba700 1 --2- 172.21.15.3:0/2743212278 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f96a0133930 0x7f96a01c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.389+0000 7f96a60ba700 1 --2- 172.21.15.3:0/2743212278 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f96a009d630 0x7f96a011f540 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.389+0000 7f96a60ba700 1 --2- 172.21.15.3:0/2743212278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96a00a0a30 0x7f96a011fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:01.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.389+0000 7f96a60ba700 1 -- 172.21.15.3:0/2743212278 >> 172.21.15.3:0/2743212278 conn(0x7f96a00a3050 msgr2=0x7f96a009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:01.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.389+0000 7f96a60ba700 1 -- 172.21.15.3:0/2743212278 shutdown_connections 2023-10-03T10:19:01.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:01.389+0000 7f96a60ba700 1 -- 172.21.15.3:0/2743212278 wait complete. 2023-10-03T10:19:01.402 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:02.282 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:02.302 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:02.362 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:02.403 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:02.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.433+0000 7f91c1390700 1 Processor -- start 2023-10-03T10:19:02.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.433+0000 7f91c1390700 1 -- start start 2023-10-03T10:19:02.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.433+0000 7f91c1390700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f91bc09d6f0 0x7f91bc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:02.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.433+0000 7f91c1390700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f91bc09e000 0x7f91bc130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:02.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.433+0000 7f91c1390700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91bc09f130 con 0x7f91bc09e410 2023-10-03T10:19:02.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.433+0000 7f91c1390700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91bc09f520 con 0x7f91bc09e000 2023-10-03T10:19:02.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.433+0000 7f91c1390700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc09e930 con 0x7f91bc09d6f0 2023-10-03T10:19:02.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.434+0000 7f91ba7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f91bc09e000 0x7f91bc130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:02.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.434+0000 7f91baffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f91bc09d6f0 0x7f91bc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:02.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.434+0000 7f91baffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f91bc09d6f0 0x7f91bc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48162/0 (socket says 172.21.15.3:48162) 2023-10-03T10:19:02.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.434+0000 7f91ba7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f91bc09e000 0x7f91bc130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49882/0 (socket says 172.21.15.3:49882) 2023-10-03T10:19:02.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.434+0000 7f91baffd700 1 -- 172.21.15.3:0/2854690820 learned_addr learned my addr 172.21.15.3:0/2854690820 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:02.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.434+0000 7f91baffd700 1 -- 172.21.15.3:0/2854690820 >> v2:172.21.15.3:3300/0 conn(0x7f91bc09e000 msgr2=0x7f91bc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:02.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.434+0000 7f91baffd700 1 --2- 172.21.15.3:0/2854690820 >> v2:172.21.15.3:3300/0 conn(0x7f91bc09e000 0x7f91bc130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.434+0000 7f91baffd700 1 -- 172.21.15.3:0/2854690820 >> v1:172.21.15.3:6789/0 conn(0x7f91bc09e410 legacy=0x7f91bc130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:02.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.434+0000 7f91baffd700 1 -- 172.21.15.3:0/2854690820 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91bc09b2d0 con 0x7f91bc09d6f0 2023-10-03T10:19:02.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.434+0000 7f91baffd700 1 --2- 172.21.15.3:0/2854690820 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f91bc09d6f0 0x7f91bc09dac0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00dd20 tx=0x7f91ac00f330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e23da976f390673 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:02.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.434+0000 7f91b9ffb700 1 -- 172.21.15.3:0/2854690820 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac012070 con 0x7f91bc09d6f0 2023-10-03T10:19:02.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.435+0000 7f91b9ffb700 1 -- 172.21.15.3:0/2854690820 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ac00de70 con 0x7f91bc09d6f0 2023-10-03T10:19:02.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.435+0000 7f91b9ffb700 1 -- 172.21.15.3:0/2854690820 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac00fd20 con 0x7f91bc09d6f0 2023-10-03T10:19:02.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.435+0000 7f91c1390700 1 -- 172.21.15.3:0/2854690820 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f91bc09d6f0 msgr2=0x7f91bc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:02.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.435+0000 7f91c1390700 1 --2- 172.21.15.3:0/2854690820 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f91bc09d6f0 0x7f91bc09dac0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00dd20 tx=0x7f91ac00f330 comp rx=0 tx=0).stop 2023-10-03T10:19:02.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.435+0000 7f91c1390700 1 -- 172.21.15.3:0/2854690820 shutdown_connections 2023-10-03T10:19:02.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.435+0000 7f91c1390700 1 --2- 172.21.15.3:0/2854690820 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f91bc09d6f0 0x7f91bc09dac0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.435+0000 7f91c1390700 1 --2- 172.21.15.3:0/2854690820 >> v2:172.21.15.3:3300/0 conn(0x7f91bc09e000 0x7f91bc130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.435+0000 7f91c1390700 1 -- 172.21.15.3:0/2854690820 >> 172.21.15.3:0/2854690820 conn(0x7f91bc066f50 msgr2=0x7f91bc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:02.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.436+0000 7f91c1390700 1 -- 172.21.15.3:0/2854690820 shutdown_connections 2023-10-03T10:19:02.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.436+0000 7f91c1390700 1 -- 172.21.15.3:0/2854690820 wait complete. 2023-10-03T10:19:02.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.437+0000 7f91c1390700 1 Processor -- start 2023-10-03T10:19:02.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.437+0000 7f91c1390700 1 -- start start 2023-10-03T10:19:02.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.438+0000 7f91c1390700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f91bc09d6f0 0x7f91bc11dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:02.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.438+0000 7f91c1390700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91bc09e000 0x7f91bc11e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:02.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.438+0000 7f91c1390700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f91bc09e410 0x7f91bc11ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:02.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.438+0000 7f91c1390700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc11f4a0 con 0x7f91bc09e000 2023-10-03T10:19:02.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.438+0000 7f91c1390700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc121af0 con 0x7f91bc09d6f0 2023-10-03T10:19:02.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.438+0000 7f91c1390700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc120e30 con 0x7f91bc09e410 2023-10-03T10:19:02.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.438+0000 7f91baffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f91bc09d6f0 0x7f91bc11dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:02.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.438+0000 7f91baffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f91bc09d6f0 0x7f91bc11dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42404/0 (socket says 172.21.15.3:42404) 2023-10-03T10:19:02.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.438+0000 7f91baffd700 1 -- 172.21.15.3:0/1816280414 learned_addr learned my addr 172.21.15.3:0/1816280414 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:02.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.438+0000 7f91ba7fc700 1 --2- 172.21.15.3:0/1816280414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91bc09e000 0x7f91bc11e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:02.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.438+0000 7f91bb7fe700 1 --2- 172.21.15.3:0/1816280414 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f91bc09e410 0x7f91bc11ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:02.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.439+0000 7f91baffd700 1 -- 172.21.15.3:0/1816280414 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f91bc09e410 msgr2=0x7f91bc11ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:02.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.439+0000 7f91baffd700 1 --2- 172.21.15.3:0/1816280414 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f91bc09e410 0x7f91bc11ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.439+0000 7f91baffd700 1 -- 172.21.15.3:0/1816280414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91bc09e000 msgr2=0x7f91bc11e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:02.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.439+0000 7f91baffd700 1 --2- 172.21.15.3:0/1816280414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91bc09e000 0x7f91bc11e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.439+0000 7f91baffd700 1 -- 172.21.15.3:0/1816280414 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ac00da00 con 0x7f91bc09d6f0 2023-10-03T10:19:02.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.439+0000 7f91bb7fe700 1 --2- 172.21.15.3:0/1816280414 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f91bc09e410 0x7f91bc11ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:02.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.439+0000 7f91baffd700 1 --2- 172.21.15.3:0/1816280414 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f91bc09d6f0 0x7f91bc11dfd0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f91ac018040 tx=0x7f91ac004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:02.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.439+0000 7f91a37fe700 1 -- 172.21.15.3:0/1816280414 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac003b30 con 0x7f91bc09d6f0 2023-10-03T10:19:02.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.439+0000 7f91a37fe700 1 -- 172.21.15.3:0/1816280414 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ac003cb0 con 0x7f91bc09d6f0 2023-10-03T10:19:02.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.439+0000 7f91a37fe700 1 -- 172.21.15.3:0/1816280414 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac016670 con 0x7f91bc09d6f0 2023-10-03T10:19:02.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.439+0000 7f91c1390700 1 -- 172.21.15.3:0/1816280414 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91bc1201c0 con 0x7f91bc09d6f0 2023-10-03T10:19:02.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.439+0000 7f91c1390700 1 -- 172.21.15.3:0/1816280414 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91bc0a1d00 con 0x7f91bc09d6f0 2023-10-03T10:19:02.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.440+0000 7f91a37fe700 1 -- 172.21.15.3:0/1816280414 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f91ac0167d0 con 0x7f91bc09d6f0 2023-10-03T10:19:02.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.441+0000 7f91a37fe700 1 -- 172.21.15.3:0/1816280414 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f91ac045180 con 0x7f91bc09d6f0 2023-10-03T10:19:02.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.441+0000 7f91c1390700 1 -- 172.21.15.3:0/1816280414 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91bc09c700 con 0x7f91bc09d6f0 2023-10-03T10:19:02.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.451+0000 7f91a37fe700 1 -- 172.21.15.3:0/1816280414 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f91ac032430 con 0x7f91bc09d6f0 2023-10-03T10:19:02.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.488+0000 7f79bd1a6700 1 Processor -- start 2023-10-03T10:19:02.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.488+0000 7f79bd1a6700 1 -- start start 2023-10-03T10:19:02.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.489+0000 7f79bd1a6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f79b809d6f0 0x7f79b809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:02.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.489+0000 7f79bd1a6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79b809e000 0x7f79b8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:02.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.489+0000 7f79bd1a6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b809f130 con 0x7f79b809e410 2023-10-03T10:19:02.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.489+0000 7f79bd1a6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79b809f520 con 0x7f79b809d6f0 2023-10-03T10:19:02.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.489+0000 7f79bd1a6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b809e930 con 0x7f79b809e000 2023-10-03T10:19:02.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.489+0000 7f79b659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79b809e000 0x7f79b8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:02.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.489+0000 7f79b6d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f79b809d6f0 0x7f79b809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:02.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.489+0000 7f79b659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79b809e000 0x7f79b8130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42406/0 (socket says 172.21.15.3:42406) 2023-10-03T10:19:02.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.489+0000 7f79b659c700 1 -- 172.21.15.3:0/1048700739 learned_addr learned my addr 172.21.15.3:0/1048700739 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:02.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.490+0000 7f79b6d9d700 1 -- 172.21.15.3:0/1048700739 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79b809e000 msgr2=0x7f79b8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:02.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.490+0000 7f79b6d9d700 1 --2- 172.21.15.3:0/1048700739 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79b809e000 0x7f79b8130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.490+0000 7f79b6d9d700 1 -- 172.21.15.3:0/1048700739 >> v1:172.21.15.3:6789/0 conn(0x7f79b809e410 legacy=0x7f79b8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:02.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.490+0000 7f79b6d9d700 1 -- 172.21.15.3:0/1048700739 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b809b2d0 con 0x7f79b809d6f0 2023-10-03T10:19:02.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.490+0000 7f79b5d9b700 1 -- 172.21.15.3:0/1048700739 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3362667298 0 0) 0x7f79b809f130 con 0x7f79b809e410 2023-10-03T10:19:02.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.490+0000 7f79b6d9d700 1 --2- 172.21.15.3:0/1048700739 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79b809d6f0 0x7f79b809dac0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009d90 tx=0x7f79a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5141eadc383dd18a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:02.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.490+0000 7f79b5d9b700 1 -- 172.21.15.3:0/1048700739 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a800e070 con 0x7f79b809d6f0 2023-10-03T10:19:02.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.490+0000 7f79b5d9b700 1 -- 172.21.15.3:0/1048700739 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a800bb20 con 0x7f79b809d6f0 2023-10-03T10:19:02.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.490+0000 7f79b5d9b700 1 -- 172.21.15.3:0/1048700739 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a8012430 con 0x7f79b809d6f0 2023-10-03T10:19:02.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.491+0000 7f79bd1a6700 1 -- 172.21.15.3:0/1048700739 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79b809d6f0 msgr2=0x7f79b809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:02.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.491+0000 7f79bd1a6700 1 --2- 172.21.15.3:0/1048700739 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79b809d6f0 0x7f79b809dac0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009d90 tx=0x7f79a800b330 comp rx=0 tx=0).stop 2023-10-03T10:19:02.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.491+0000 7f79bd1a6700 1 -- 172.21.15.3:0/1048700739 shutdown_connections 2023-10-03T10:19:02.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.491+0000 7f79bd1a6700 1 --2- 172.21.15.3:0/1048700739 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79b809d6f0 0x7f79b809dac0 secure :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009d90 tx=0x7f79a800b330 comp rx=0 tx=0).stop 2023-10-03T10:19:02.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.491+0000 7f79bd1a6700 1 --2- 172.21.15.3:0/1048700739 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79b809e000 0x7f79b8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.491+0000 7f79bd1a6700 1 -- 172.21.15.3:0/1048700739 >> 172.21.15.3:0/1048700739 conn(0x7f79b8066f50 msgr2=0x7f79b809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:02.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.491+0000 7f79bd1a6700 1 -- 172.21.15.3:0/1048700739 shutdown_connections 2023-10-03T10:19:02.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.491+0000 7f79bd1a6700 1 -- 172.21.15.3:0/1048700739 wait complete. 2023-10-03T10:19:02.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.493+0000 7f79bd1a6700 1 Processor -- start 2023-10-03T10:19:02.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.495+0000 7f79bd1a6700 1 -- start start 2023-10-03T10:19:02.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.495+0000 7f79bd1a6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79b809e000 0x7f79b8120340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:02.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.495+0000 7f79bd1a6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79b809e410 0x7f79b8120880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:02.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.495+0000 7f79bd1a6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79b8120dc0 0x7f79b81cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:02.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.495+0000 7f79bd1a6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b8123dd0 con 0x7f79b809e410 2023-10-03T10:19:02.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.496+0000 7f79bd1a6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b81224e0 con 0x7f79b8120dc0 2023-10-03T10:19:02.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.496+0000 7f79bd1a6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79b809c700 con 0x7f79b809e000 2023-10-03T10:19:02.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.496+0000 7f79b659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79b809e410 0x7f79b8120880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:02.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.496+0000 7f79b659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79b809e410 0x7f79b8120880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49914/0 (socket says 172.21.15.3:49914) 2023-10-03T10:19:02.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.496+0000 7f79b659c700 1 -- 172.21.15.3:0/1002028553 learned_addr learned my addr 172.21.15.3:0/1002028553 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:02.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.496+0000 7f79b6d9d700 1 --2- 172.21.15.3:0/1002028553 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79b809e000 0x7f79b8120340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:02.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.496+0000 7f79b759e700 1 --2- 172.21.15.3:0/1002028553 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79b8120dc0 0x7f79b81cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:02.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.496+0000 7f79b659c700 1 -- 172.21.15.3:0/1002028553 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79b809e000 msgr2=0x7f79b8120340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:02.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.496+0000 7f79b659c700 1 --2- 172.21.15.3:0/1002028553 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79b809e000 0x7f79b8120340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.496+0000 7f79b659c700 1 -- 172.21.15.3:0/1002028553 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79b8120dc0 msgr2=0x7f79b81cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:02.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.496+0000 7f79b659c700 1 --2- 172.21.15.3:0/1002028553 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79b8120dc0 0x7f79b81cc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.496+0000 7f79b659c700 1 -- 172.21.15.3:0/1002028553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a8009a70 con 0x7f79b809e410 2023-10-03T10:19:02.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.496+0000 7f79b6d9d700 1 --2- 172.21.15.3:0/1002028553 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79b809e000 0x7f79b8120340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:02.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.496+0000 7f79b659c700 1 --2- 172.21.15.3:0/1002028553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79b809e410 0x7f79b8120880 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00e910 tx=0x7f79ac00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:02.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.497+0000 7f799f7fe700 1 -- 172.21.15.3:0/1002028553 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ac0096a0 con 0x7f79b809e410 2023-10-03T10:19:02.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.497+0000 7f799f7fe700 1 -- 172.21.15.3:0/1002028553 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ac00ee60 con 0x7f79b809e410 2023-10-03T10:19:02.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.497+0000 7f79bd1a6700 1 -- 172.21.15.3:0/1002028553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b8121ec0 con 0x7f79b809e410 2023-10-03T10:19:02.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.497+0000 7f79bd1a6700 1 -- 172.21.15.3:0/1002028553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b81226b0 con 0x7f79b809e410 2023-10-03T10:19:02.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.497+0000 7f799f7fe700 1 -- 172.21.15.3:0/1002028553 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ac00ca50 con 0x7f79b809e410 2023-10-03T10:19:02.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.498+0000 7f799f7fe700 1 -- 172.21.15.3:0/1002028553 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f79ac00cbb0 con 0x7f79b809e410 2023-10-03T10:19:02.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.498+0000 7f79bd1a6700 1 -- 172.21.15.3:0/1002028553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79980052e0 con 0x7f79b809e410 2023-10-03T10:19:02.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.499+0000 7f799f7fe700 1 -- 172.21.15.3:0/1002028553 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f79ac03f7d0 con 0x7f79b809e410 2023-10-03T10:19:02.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.504+0000 7f799f7fe700 1 -- 172.21.15.3:0/1002028553 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f79ac0232f0 con 0x7f79b809e410 2023-10-03T10:19:02.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.524+0000 7f85da319700 1 Processor -- start 2023-10-03T10:19:02.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.524+0000 7f85da319700 1 -- start start 2023-10-03T10:19:02.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.525+0000 7f85da319700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85d406e570 0x7f85d406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:02.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.525+0000 7f85da319700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85d406ee80 0x7f85d4134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:02.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.525+0000 7f85da319700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85d4070ee0 con 0x7f85d409eca0 2023-10-03T10:19:02.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.525+0000 7f85da319700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d4071480 con 0x7f85d406ee80 2023-10-03T10:19:02.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.525+0000 7f85da319700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85d4071b90 con 0x7f85d406e570 2023-10-03T10:19:02.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.525+0000 7f85d37fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f85d409eca0 0x7f85d4134610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33562/0 (socket says 172.21.15.3:33562) 2023-10-03T10:19:02.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.525+0000 7f85d37fe700 1 -- 172.21.15.3:0/2530677218 learned_addr learned my addr 172.21.15.3:0/2530677218 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:02.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.526+0000 7f85d88b6700 1 --2- 172.21.15.3:0/2530677218 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85d406ee80 0x7f85d4134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:02.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.526+0000 7f85d3fff700 1 --2- 172.21.15.3:0/2530677218 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85d406e570 0x7f85d406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:02.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.526+0000 7f85d3fff700 1 -- 172.21.15.3:0/2530677218 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85d406ee80 msgr2=0x7f85d4134d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:02.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.526+0000 7f85d3fff700 1 --2- 172.21.15.3:0/2530677218 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85d406ee80 0x7f85d4134d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.526+0000 7f85d3fff700 1 -- 172.21.15.3:0/2530677218 >> v1:172.21.15.3:6789/0 conn(0x7f85d409eca0 legacy=0x7f85d4134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:02.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.526+0000 7f85d3fff700 1 -- 172.21.15.3:0/2530677218 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d40711a0 con 0x7f85d406e570 2023-10-03T10:19:02.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.526+0000 7f85d2ffd700 1 -- 172.21.15.3:0/2530677218 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 596070546 0 0) 0x7f85d4070ee0 con 0x7f85d409eca0 2023-10-03T10:19:02.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.526+0000 7f85d3fff700 1 --2- 172.21.15.3:0/2530677218 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85d406e570 0x7f85d406e940 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f85c4009870 tx=0x7f85c4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=909640f82eb00f8a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:02.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.527+0000 7f85d2ffd700 1 -- 172.21.15.3:0/2530677218 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c4011830 con 0x7f85d406e570 2023-10-03T10:19:02.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.527+0000 7f85d2ffd700 1 -- 172.21.15.3:0/2530677218 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c4011990 con 0x7f85d406e570 2023-10-03T10:19:02.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.527+0000 7f85d2ffd700 1 -- 172.21.15.3:0/2530677218 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c4011d20 con 0x7f85d406e570 2023-10-03T10:19:02.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.527+0000 7f85da319700 1 -- 172.21.15.3:0/2530677218 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85d406e570 msgr2=0x7f85d406e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:02.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.527+0000 7f85da319700 1 --2- 172.21.15.3:0/2530677218 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85d406e570 0x7f85d406e940 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f85c4009870 tx=0x7f85c4009b80 comp rx=0 tx=0).stop 2023-10-03T10:19:02.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.528+0000 7f85da319700 1 -- 172.21.15.3:0/2530677218 shutdown_connections 2023-10-03T10:19:02.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.528+0000 7f85da319700 1 --2- 172.21.15.3:0/2530677218 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85d406ee80 0x7f85d4134d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.528+0000 7f85da319700 1 --2- 172.21.15.3:0/2530677218 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85d406e570 0x7f85d406e940 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.528+0000 7f85da319700 1 -- 172.21.15.3:0/2530677218 >> 172.21.15.3:0/2530677218 conn(0x7f85d40a2aa0 msgr2=0x7f85d40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:02.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.528+0000 7f85da319700 1 -- 172.21.15.3:0/2530677218 shutdown_connections 2023-10-03T10:19:02.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.528+0000 7f85da319700 1 -- 172.21.15.3:0/2530677218 wait complete. 2023-10-03T10:19:02.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.529+0000 7f85da319700 1 Processor -- start 2023-10-03T10:19:02.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.529+0000 7f85da319700 1 -- start start 2023-10-03T10:19:02.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.529+0000 7f85da319700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85d406e570 0x7f85d413ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:02.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.529+0000 7f85da319700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85d406ee80 0x7f85d4124e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:02.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.529+0000 7f85da319700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85d413d9b0 0x7f85d4125340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:02.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.529+0000 7f85da319700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d4140550 con 0x7f85d406e570 2023-10-03T10:19:02.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.529+0000 7f85da319700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d41265b0 con 0x7f85d413d9b0 2023-10-03T10:19:02.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.529+0000 7f85da319700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85d413d5b0 con 0x7f85d406ee80 2023-10-03T10:19:02.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.530+0000 7f85d3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85d406e570 0x7f85d413ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:02.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.530+0000 7f85d37fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85d406ee80 0x7f85d4124e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:02.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.530+0000 7f85d3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85d406e570 0x7f85d413ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49924/0 (socket says 172.21.15.3:49924) 2023-10-03T10:19:02.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.530+0000 7f85d3fff700 1 -- 172.21.15.3:0/2410479793 learned_addr learned my addr 172.21.15.3:0/2410479793 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:02.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.530+0000 7f85d88b6700 1 --2- 172.21.15.3:0/2410479793 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85d413d9b0 0x7f85d4125340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:02.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.530+0000 7f85d3fff700 1 -- 172.21.15.3:0/2410479793 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85d406ee80 msgr2=0x7f85d4124e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:02.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.530+0000 7f85d3fff700 1 --2- 172.21.15.3:0/2410479793 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85d406ee80 0x7f85d4124e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.530+0000 7f85d3fff700 1 -- 172.21.15.3:0/2410479793 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85d413d9b0 msgr2=0x7f85d4125340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:02.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.530+0000 7f85d3fff700 1 --2- 172.21.15.3:0/2410479793 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85d413d9b0 0x7f85d4125340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.530+0000 7f85d3fff700 1 -- 172.21.15.3:0/2410479793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c40094a0 con 0x7f85d406e570 2023-10-03T10:19:02.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.530+0000 7f85d3fff700 1 --2- 172.21.15.3:0/2410479793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85d406e570 0x7f85d413ab00 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f85c4009840 tx=0x7f85c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:02.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.531+0000 7f85d0ff9700 1 -- 172.21.15.3:0/2410479793 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c4004510 con 0x7f85d406e570 2023-10-03T10:19:02.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.531+0000 7f85d0ff9700 1 -- 172.21.15.3:0/2410479793 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c401f070 con 0x7f85d406e570 2023-10-03T10:19:02.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.531+0000 7f85d0ff9700 1 -- 172.21.15.3:0/2410479793 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c40038c0 con 0x7f85d406e570 2023-10-03T10:19:02.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.531+0000 7f85da319700 1 -- 172.21.15.3:0/2410479793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d4135690 con 0x7f85d406e570 2023-10-03T10:19:02.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.531+0000 7f85da319700 1 -- 172.21.15.3:0/2410479793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85d41267f0 con 0x7f85d406e570 2023-10-03T10:19:02.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.532+0000 7f85d0ff9700 1 -- 172.21.15.3:0/2410479793 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f85c4003a20 con 0x7f85d406e570 2023-10-03T10:19:02.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.532+0000 7f85da319700 1 -- 172.21.15.3:0/2410479793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85d413d3f0 con 0x7f85d406e570 2023-10-03T10:19:02.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.533+0000 7f85d0ff9700 1 -- 172.21.15.3:0/2410479793 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f85c4021070 con 0x7f85d406e570 2023-10-03T10:19:02.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.542+0000 7f85d0ff9700 1 -- 172.21.15.3:0/2410479793 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f85d413d3f0 con 0x7f85d406e570 2023-10-03T10:19:02.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.572+0000 7fb813ebb700 1 Processor -- start 2023-10-03T10:19:02.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.572+0000 7fb813ebb700 1 -- start start 2023-10-03T10:19:02.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.573+0000 7fb813ebb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb80c1338a0 0x7fb80c133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:02.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.573+0000 7fb813ebb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb80c09de50 0x7fb80c09e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:02.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.573+0000 7fb813ebb700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb80c06ef30 con 0x7fb80c09e760 2023-10-03T10:19:02.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.573+0000 7fb813ebb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c06fef0 con 0x7fb80c09de50 2023-10-03T10:19:02.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.573+0000 7fb813ebb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c1334b0 con 0x7fb80c1338a0 2023-10-03T10:19:02.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.573+0000 7fb811456700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb80c09de50 0x7fb80c09e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:02.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.573+0000 7fb811456700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb80c09de50 0x7fb80c09e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42442/0 (socket says 172.21.15.3:42442) 2023-10-03T10:19:02.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.573+0000 7fb811456700 1 -- 172.21.15.3:0/3223237590 learned_addr learned my addr 172.21.15.3:0/3223237590 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:02.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.573+0000 7fb811c57700 1 --2- 172.21.15.3:0/3223237590 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb80c1338a0 0x7fb80c133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:02.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.573+0000 7fb811456700 1 -- 172.21.15.3:0/3223237590 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb80c1338a0 msgr2=0x7fb80c133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:02.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.573+0000 7fb811456700 1 --2- 172.21.15.3:0/3223237590 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb80c1338a0 0x7fb80c133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.573+0000 7fb811456700 1 -- 172.21.15.3:0/3223237590 >> v1:172.21.15.3:6789/0 conn(0x7fb80c09e760 legacy=0x7fb80c13a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:02.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.573+0000 7fb811456700 1 -- 172.21.15.3:0/3223237590 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80c06fc30 con 0x7fb80c09de50 2023-10-03T10:19:02.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.574+0000 7fb811456700 1 --2- 172.21.15.3:0/3223237590 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb80c09de50 0x7fb80c09e220 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009920 tx=0x7fb7fc009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4057266831d99aa8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:02.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.574+0000 7fb810c55700 1 -- 172.21.15.3:0/3223237590 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc00ba60 con 0x7fb80c09de50 2023-10-03T10:19:02.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.574+0000 7fb810c55700 1 -- 172.21.15.3:0/3223237590 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7fc00bbc0 con 0x7fb80c09de50 2023-10-03T10:19:02.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.574+0000 7fb810c55700 1 -- 172.21.15.3:0/3223237590 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc009e70 con 0x7fb80c09de50 2023-10-03T10:19:02.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.574+0000 7fb813ebb700 1 -- 172.21.15.3:0/3223237590 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb80c09de50 msgr2=0x7fb80c09e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:02.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.574+0000 7fb813ebb700 1 --2- 172.21.15.3:0/3223237590 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb80c09de50 0x7fb80c09e220 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009920 tx=0x7fb7fc009c30 comp rx=0 tx=0).stop 2023-10-03T10:19:02.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.575+0000 7fb813ebb700 1 -- 172.21.15.3:0/3223237590 shutdown_connections 2023-10-03T10:19:02.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.575+0000 7fb813ebb700 1 --2- 172.21.15.3:0/3223237590 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb80c1338a0 0x7fb80c133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.575+0000 7fb813ebb700 1 --2- 172.21.15.3:0/3223237590 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb80c09de50 0x7fb80c09e220 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.575+0000 7fb813ebb700 1 -- 172.21.15.3:0/3223237590 >> 172.21.15.3:0/3223237590 conn(0x7fb80c0a2aa0 msgr2=0x7fb80c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:02.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.575+0000 7fb813ebb700 1 -- 172.21.15.3:0/3223237590 shutdown_connections 2023-10-03T10:19:02.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.575+0000 7fb813ebb700 1 -- 172.21.15.3:0/3223237590 wait complete. 2023-10-03T10:19:02.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.576+0000 7fb813ebb700 1 Processor -- start 2023-10-03T10:19:02.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.576+0000 7fb813ebb700 1 -- start start 2023-10-03T10:19:02.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.577+0000 7fb813ebb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb80c09de50 0x7fb80c11eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:02.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.577+0000 7fb813ebb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb80c09e760 0x7fb80c11f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:02.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.577+0000 7fb813ebb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80c1338a0 0x7fb80c11f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:02.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.577+0000 7fb813ebb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c121cd0 con 0x7fb80c1338a0 2023-10-03T10:19:02.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.577+0000 7fb813ebb700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c12f680 con 0x7fb80c09e760 2023-10-03T10:19:02.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.577+0000 7fb813ebb700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c1206e0 con 0x7fb80c09de50 2023-10-03T10:19:02.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.577+0000 7fb811c57700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb80c09de50 0x7fb80c11eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:02.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.577+0000 7fb811c57700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb80c09de50 0x7fb80c11eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48224/0 (socket says 172.21.15.3:48224) 2023-10-03T10:19:02.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.577+0000 7fb811c57700 1 -- 172.21.15.3:0/129585332 learned_addr learned my addr 172.21.15.3:0/129585332 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:02.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.577+0000 7fb811456700 1 --2- 172.21.15.3:0/129585332 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb80c09e760 0x7fb80c11f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:02.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.577+0000 7fb812458700 1 --2- 172.21.15.3:0/129585332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80c1338a0 0x7fb80c11f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:02.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.578+0000 7fb811c57700 1 -- 172.21.15.3:0/129585332 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb80c09e760 msgr2=0x7fb80c11f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:02.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.578+0000 7fb811c57700 1 --2- 172.21.15.3:0/129585332 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb80c09e760 0x7fb80c11f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.578+0000 7fb811c57700 1 -- 172.21.15.3:0/129585332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80c1338a0 msgr2=0x7fb80c11f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:02.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.578+0000 7fb811c57700 1 --2- 172.21.15.3:0/129585332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80c1338a0 0x7fb80c11f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.578+0000 7fb811c57700 1 -- 172.21.15.3:0/129585332 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7fc009550 con 0x7fb80c09de50 2023-10-03T10:19:02.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.578+0000 7fb811c57700 1 --2- 172.21.15.3:0/129585332 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb80c09de50 0x7fb80c11eb50 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb80800e810 tx=0x7fb80800eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:02.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.578+0000 7fb8027fc700 1 -- 172.21.15.3:0/129585332 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80800c070 con 0x7fb80c09de50 2023-10-03T10:19:02.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.578+0000 7fb8027fc700 1 -- 172.21.15.3:0/129585332 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb808013610 con 0x7fb80c09de50 2023-10-03T10:19:02.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.578+0000 7fb8027fc700 1 -- 172.21.15.3:0/129585332 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8080139c0 con 0x7fb80c09de50 2023-10-03T10:19:02.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.578+0000 7fb813ebb700 1 -- 172.21.15.3:0/129585332 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb80c0a1150 con 0x7fb80c09de50 2023-10-03T10:19:02.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.578+0000 7fb813ebb700 1 -- 172.21.15.3:0/129585332 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb80c122310 con 0x7fb80c09de50 2023-10-03T10:19:02.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.579+0000 7fb8027fc700 1 -- 172.21.15.3:0/129585332 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb808013b20 con 0x7fb80c09de50 2023-10-03T10:19:02.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.579+0000 7fb8027fc700 1 -- 172.21.15.3:0/129585332 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb80803f480 con 0x7fb80c09de50 2023-10-03T10:19:02.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.579+0000 7fb813ebb700 1 -- 172.21.15.3:0/129585332 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb80c11ffb0 con 0x7fb80c09de50 2023-10-03T10:19:02.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.589+0000 7fb8027fc700 1 -- 172.21.15.3:0/129585332 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb8080108e0 con 0x7fb80c09de50 2023-10-03T10:19:02.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.672+0000 7f91c1390700 1 -- 172.21.15.3:0/1816280414 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f91bc074ea0 con 0x7f91bc09d6f0 2023-10-03T10:19:02.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.672+0000 7f91a37fe700 1 -- 172.21.15.3:0/1816280414 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f91ac01eb70 con 0x7f91bc09d6f0 2023-10-03T10:19:02.673 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:02.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.675+0000 7f91c1390700 1 -- 172.21.15.3:0/1816280414 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f91bc09d6f0 msgr2=0x7f91bc11dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:02.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.675+0000 7f91c1390700 1 --2- 172.21.15.3:0/1816280414 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f91bc09d6f0 0x7f91bc11dfd0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f91ac018040 tx=0x7f91ac004750 comp rx=0 tx=0).stop 2023-10-03T10:19:02.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.675+0000 7f91c1390700 1 -- 172.21.15.3:0/1816280414 shutdown_connections 2023-10-03T10:19:02.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.675+0000 7f91c1390700 1 --2- 172.21.15.3:0/1816280414 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f91bc09e410 0x7f91bc11ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.675+0000 7f91c1390700 1 --2- 172.21.15.3:0/1816280414 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f91bc09d6f0 0x7f91bc11dfd0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.675+0000 7f91c1390700 1 --2- 172.21.15.3:0/1816280414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91bc09e000 0x7f91bc11e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.675+0000 7f91c1390700 1 -- 172.21.15.3:0/1816280414 >> 172.21.15.3:0/1816280414 conn(0x7f91bc066f50 msgr2=0x7f91bc070ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:02.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.675+0000 7f91c1390700 1 -- 172.21.15.3:0/1816280414 shutdown_connections 2023-10-03T10:19:02.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.675+0000 7f91c1390700 1 -- 172.21.15.3:0/1816280414 wait complete. 2023-10-03T10:19:02.689 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:02.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.721+0000 7f79bd1a6700 1 -- 172.21.15.3:0/1002028553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7998005590 con 0x7f79b809e410 2023-10-03T10:19:02.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.721+0000 7f799f7fe700 1 -- 172.21.15.3:0/1002028553 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f79ac021070 con 0x7f79b809e410 2023-10-03T10:19:02.722 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:02.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.724+0000 7f79bd1a6700 1 -- 172.21.15.3:0/1002028553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79b809e410 msgr2=0x7f79b8120880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:02.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.724+0000 7f79bd1a6700 1 --2- 172.21.15.3:0/1002028553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79b809e410 0x7f79b8120880 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00e910 tx=0x7f79ac00ec20 comp rx=0 tx=0).stop 2023-10-03T10:19:02.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.724+0000 7f79bd1a6700 1 -- 172.21.15.3:0/1002028553 shutdown_connections 2023-10-03T10:19:02.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.724+0000 7f79bd1a6700 1 --2- 172.21.15.3:0/1002028553 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79b8120dc0 0x7f79b81cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.724+0000 7f79bd1a6700 1 --2- 172.21.15.3:0/1002028553 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79b809e000 0x7f79b8120340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.724+0000 7f79bd1a6700 1 --2- 172.21.15.3:0/1002028553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79b809e410 0x7f79b8120880 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.724+0000 7f79bd1a6700 1 -- 172.21.15.3:0/1002028553 >> 172.21.15.3:0/1002028553 conn(0x7f79b8066f50 msgr2=0x7f79b806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:02.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.724+0000 7f79bd1a6700 1 -- 172.21.15.3:0/1002028553 shutdown_connections 2023-10-03T10:19:02.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.724+0000 7f79bd1a6700 1 -- 172.21.15.3:0/1002028553 wait complete. 2023-10-03T10:19:02.738 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:02.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.754+0000 7f85da319700 1 -- 172.21.15.3:0/2410479793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f85d413d3f0 con 0x7f85d406e570 2023-10-03T10:19:02.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.754+0000 7f85d0ff9700 1 -- 172.21.15.3:0/2410479793 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f85d413d3f0 con 0x7f85d406e570 2023-10-03T10:19:02.754 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:02.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.757+0000 7f85da319700 1 -- 172.21.15.3:0/2410479793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85d406e570 msgr2=0x7f85d413ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:02.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.757+0000 7f85da319700 1 --2- 172.21.15.3:0/2410479793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85d406e570 0x7f85d413ab00 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f85c4009840 tx=0x7f85c4004750 comp rx=0 tx=0).stop 2023-10-03T10:19:02.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.757+0000 7f85da319700 1 -- 172.21.15.3:0/2410479793 shutdown_connections 2023-10-03T10:19:02.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.757+0000 7f85da319700 1 --2- 172.21.15.3:0/2410479793 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f85d413d9b0 0x7f85d4125340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.757+0000 7f85da319700 1 --2- 172.21.15.3:0/2410479793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85d406e570 0x7f85d413ab00 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.757+0000 7f85da319700 1 --2- 172.21.15.3:0/2410479793 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f85d406ee80 0x7f85d4124e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.757+0000 7f85da319700 1 -- 172.21.15.3:0/2410479793 >> 172.21.15.3:0/2410479793 conn(0x7f85d40a2aa0 msgr2=0x7f85d40a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:02.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.757+0000 7f85da319700 1 -- 172.21.15.3:0/2410479793 shutdown_connections 2023-10-03T10:19:02.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.757+0000 7f85da319700 1 -- 172.21.15.3:0/2410479793 wait complete. 2023-10-03T10:19:02.771 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:02.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.798+0000 7fb813ebb700 1 -- 172.21.15.3:0/129585332 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb80c074ea0 con 0x7fb80c09de50 2023-10-03T10:19:02.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.798+0000 7fb8027fc700 1 -- 172.21.15.3:0/129585332 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb80802b090 con 0x7fb80c09de50 2023-10-03T10:19:02.798 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:02.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.800+0000 7fb813ebb700 1 -- 172.21.15.3:0/129585332 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb80c09de50 msgr2=0x7fb80c11eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:02.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.800+0000 7fb813ebb700 1 --2- 172.21.15.3:0/129585332 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb80c09de50 0x7fb80c11eb50 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb80800e810 tx=0x7fb80800eb20 comp rx=0 tx=0).stop 2023-10-03T10:19:02.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.800+0000 7fb813ebb700 1 -- 172.21.15.3:0/129585332 shutdown_connections 2023-10-03T10:19:02.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.800+0000 7fb813ebb700 1 --2- 172.21.15.3:0/129585332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80c1338a0 0x7fb80c11f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.800+0000 7fb813ebb700 1 --2- 172.21.15.3:0/129585332 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb80c09de50 0x7fb80c11eb50 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.800+0000 7fb813ebb700 1 --2- 172.21.15.3:0/129585332 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb80c09e760 0x7fb80c11f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:02.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.800+0000 7fb813ebb700 1 -- 172.21.15.3:0/129585332 >> 172.21.15.3:0/129585332 conn(0x7fb80c0a2aa0 msgr2=0x7fb80c09ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:02.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.800+0000 7fb813ebb700 1 -- 172.21.15.3:0/129585332 shutdown_connections 2023-10-03T10:19:02.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:02.800+0000 7fb813ebb700 1 -- 172.21.15.3:0/129585332 wait complete. 2023-10-03T10:19:02.814 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:03.690 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:03.739 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:03.771 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:03.815 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:03.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.847+0000 7fd8c7bf5700 1 Processor -- start 2023-10-03T10:19:03.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.847+0000 7fd8c7bf5700 1 -- start start 2023-10-03T10:19:03.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.848+0000 7fd8c7bf5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd8c01338a0 0x7fd8c0133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:03.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.848+0000 7fd8c7bf5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd8c009e930 0x7fd8c013b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:03.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.848+0000 7fd8c7bf5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c006ef30 con 0x7fd8c009de50 2023-10-03T10:19:03.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.848+0000 7fd8c7bf5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c006fef0 con 0x7fd8c009e930 2023-10-03T10:19:03.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.848+0000 7fd8c7bf5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c01334b0 con 0x7fd8c01338a0 2023-10-03T10:19:03.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.849+0000 7fd8c6192700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd8c009e930 0x7fd8c013b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:03.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.849+0000 7fd8c5991700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd8c01338a0 0x7fd8c0133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:03.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.849+0000 7fd8c6192700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd8c009e930 0x7fd8c013b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46540/0 (socket says 172.21.15.3:46540) 2023-10-03T10:19:03.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.849+0000 7fd8c6192700 1 -- 172.21.15.3:0/2000960465 learned_addr learned my addr 172.21.15.3:0/2000960465 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:03.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.849+0000 7fd8c5991700 1 -- 172.21.15.3:0/2000960465 >> v2:172.21.15.3:3300/0 conn(0x7fd8c009e930 msgr2=0x7fd8c013b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:03.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.849+0000 7fd8c5991700 1 --2- 172.21.15.3:0/2000960465 >> v2:172.21.15.3:3300/0 conn(0x7fd8c009e930 0x7fd8c013b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:03.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.849+0000 7fd8c5991700 1 -- 172.21.15.3:0/2000960465 >> v1:172.21.15.3:6789/0 conn(0x7fd8c009de50 legacy=0x7fd8c009e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:03.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.849+0000 7fd8c5991700 1 -- 172.21.15.3:0/2000960465 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c006fc30 con 0x7fd8c01338a0 2023-10-03T10:19:03.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.849+0000 7fd8c6192700 1 --2- 172.21.15.3:0/2000960465 >> v2:172.21.15.3:3300/0 conn(0x7fd8c009e930 0x7fd8c013b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:03.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.849+0000 7fd8c5991700 1 --2- 172.21.15.3:0/2000960465 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd8c01338a0 0x7fd8c0133c70 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc009840 tx=0x7fd8bc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae61b1d8e64233ca server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:03.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.849+0000 7fd8c498f700 1 -- 172.21.15.3:0/2000960465 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc011830 con 0x7fd8c01338a0 2023-10-03T10:19:03.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.850+0000 7fd8c498f700 1 -- 172.21.15.3:0/2000960465 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8bc011990 con 0x7fd8c01338a0 2023-10-03T10:19:03.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.850+0000 7fd8c498f700 1 -- 172.21.15.3:0/2000960465 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc011d20 con 0x7fd8c01338a0 2023-10-03T10:19:03.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.851+0000 7fd8c7bf5700 1 -- 172.21.15.3:0/2000960465 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd8c01338a0 msgr2=0x7fd8c0133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:03.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.851+0000 7fd8c7bf5700 1 --2- 172.21.15.3:0/2000960465 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd8c01338a0 0x7fd8c0133c70 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc009840 tx=0x7fd8bc009b50 comp rx=0 tx=0).stop 2023-10-03T10:19:03.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.851+0000 7fd8c7bf5700 1 -- 172.21.15.3:0/2000960465 shutdown_connections 2023-10-03T10:19:03.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.851+0000 7fd8c7bf5700 1 --2- 172.21.15.3:0/2000960465 >> v2:172.21.15.3:3300/0 conn(0x7fd8c009e930 0x7fd8c013b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:03.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.851+0000 7fd8c7bf5700 1 --2- 172.21.15.3:0/2000960465 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd8c01338a0 0x7fd8c0133c70 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:03.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.851+0000 7fd8c7bf5700 1 -- 172.21.15.3:0/2000960465 >> 172.21.15.3:0/2000960465 conn(0x7fd8c00a2aa0 msgr2=0x7fd8c00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:03.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.851+0000 7fd8c7bf5700 1 -- 172.21.15.3:0/2000960465 shutdown_connections 2023-10-03T10:19:03.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.851+0000 7fd8c7bf5700 1 -- 172.21.15.3:0/2000960465 wait complete. 2023-10-03T10:19:03.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.852+0000 7fd8c7bf5700 1 Processor -- start 2023-10-03T10:19:03.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.852+0000 7fd8c7bf5700 1 -- start start 2023-10-03T10:19:03.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.853+0000 7fd8c7bf5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8c009de50 0x7fd8c011d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:03.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.853+0000 7fd8c7bf5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd8c009e930 0x7fd8c011dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:03.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.853+0000 7fd8c7bf5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8c01338a0 0x7fd8c011e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:03.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.853+0000 7fd8c7bf5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c011f350 con 0x7fd8c01338a0 2023-10-03T10:19:03.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.853+0000 7fd8c7bf5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c006f640 con 0x7fd8c009de50 2023-10-03T10:19:03.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.853+0000 7fd8c7bf5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c0120f80 con 0x7fd8c009e930 2023-10-03T10:19:03.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.853+0000 7fd8c5991700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8c009de50 0x7fd8c011d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:03.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.853+0000 7fd8c5991700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8c009de50 0x7fd8c011d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41368/0 (socket says 172.21.15.3:41368) 2023-10-03T10:19:03.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.853+0000 7fd8c5991700 1 -- 172.21.15.3:0/1168639048 learned_addr learned my addr 172.21.15.3:0/1168639048 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:03.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.853+0000 7fd8c5190700 1 --2- 172.21.15.3:0/1168639048 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd8c009e930 0x7fd8c011dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:03.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.854+0000 7fd8c5991700 1 -- 172.21.15.3:0/1168639048 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd8c009e930 msgr2=0x7fd8c011dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:03.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.854+0000 7fd8c5991700 1 --2- 172.21.15.3:0/1168639048 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd8c009e930 0x7fd8c011dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:03.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.854+0000 7fd8c5991700 1 -- 172.21.15.3:0/1168639048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8c01338a0 msgr2=0x7fd8c011e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:03.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.854+0000 7fd8c5991700 1 --2- 172.21.15.3:0/1168639048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8c01338a0 0x7fd8c011e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:03.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.854+0000 7fd8c5991700 1 -- 172.21.15.3:0/1168639048 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b8009b30 con 0x7fd8c009de50 2023-10-03T10:19:03.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.854+0000 7fd8c5991700 1 --2- 172.21.15.3:0/1168639048 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8c009de50 0x7fd8c011d820 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc009fd0 tx=0x7fd8bc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:03.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.854+0000 7fd8b67fc700 1 -- 172.21.15.3:0/1168639048 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc0036a0 con 0x7fd8c009de50 2023-10-03T10:19:03.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.854+0000 7fd8b67fc700 1 -- 172.21.15.3:0/1168639048 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8bc01f070 con 0x7fd8c009de50 2023-10-03T10:19:03.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.854+0000 7fd8c7bf5700 1 -- 172.21.15.3:0/1168639048 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8bc0094a0 con 0x7fd8c009de50 2023-10-03T10:19:03.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.854+0000 7fd8b67fc700 1 -- 172.21.15.3:0/1168639048 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc003a60 con 0x7fd8c009de50 2023-10-03T10:19:03.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.854+0000 7fd8c7bf5700 1 -- 172.21.15.3:0/1168639048 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c00a1050 con 0x7fd8c009de50 2023-10-03T10:19:03.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.855+0000 7fd8b67fc700 1 -- 172.21.15.3:0/1168639048 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd8bc003bc0 con 0x7fd8c009de50 2023-10-03T10:19:03.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.856+0000 7fd8b67fc700 1 -- 172.21.15.3:0/1168639048 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd8bc021070 con 0x7fd8c009de50 2023-10-03T10:19:03.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.856+0000 7fd8c7bf5700 1 -- 172.21.15.3:0/1168639048 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c011a470 con 0x7fd8c009de50 2023-10-03T10:19:03.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.866+0000 7fd8b67fc700 1 -- 172.21.15.3:0/1168639048 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd8bc02b080 con 0x7fd8c009de50 2023-10-03T10:19:03.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.890+0000 7f65e4607700 1 Processor -- start 2023-10-03T10:19:03.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.890+0000 7f65e4607700 1 -- start start 2023-10-03T10:19:03.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.891+0000 7f65e4607700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65dc09d6f0 0x7f65dc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:03.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.891+0000 7f65e4607700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65dc09e000 0x7f65dc1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.891+0000 7f65e4607700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f65dc09e410 0x7f65dc133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.891+0000 7f65e4607700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65dc06b460 con 0x7f65dc09e410 2023-10-03T10:19:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.891+0000 7f65e4607700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc09b120 con 0x7f65dc09e000 2023-10-03T10:19:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.891+0000 7f65e4607700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc09a7e0 con 0x7f65dc09d6f0 2023-10-03T10:19:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.891+0000 7f65e1ba2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65dc09e000 0x7f65dc1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.891+0000 7f65e2ba4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f65dc09e410 0x7f65dc133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.891+0000 7f65e1ba2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65dc09e000 0x7f65dc1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41376/0 (socket says 172.21.15.3:41376) 2023-10-03T10:19:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.891+0000 7f65e2ba4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f65dc09e410 0x7f65dc133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46554/0 (socket says 172.21.15.3:46554) 2023-10-03T10:19:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.891+0000 7f65e1ba2700 1 -- 172.21.15.3:0/1633942837 learned_addr learned my addr 172.21.15.3:0/1633942837 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:03.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.891+0000 7f65e23a3700 1 --2- 172.21.15.3:0/1633942837 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65dc09d6f0 0x7f65dc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:03.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.892+0000 7f65e2ba4700 1 -- 172.21.15.3:0/1633942837 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65dc09d6f0 msgr2=0x7f65dc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:03.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.892+0000 7f65e2ba4700 1 --2- 172.21.15.3:0/1633942837 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65dc09d6f0 0x7f65dc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:03.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.892+0000 7f65e2ba4700 1 -- 172.21.15.3:0/1633942837 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65dc09e000 msgr2=0x7f65dc1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:03.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.892+0000 7f65e2ba4700 1 --2- 172.21.15.3:0/1633942837 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65dc09e000 0x7f65dc1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.892+0000 7f65e2ba4700 1 -- 172.21.15.3:0/1633942837 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65dc09a500 con 0x7f65dc09e410 2023-10-03T10:19:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.892+0000 7f65e23a3700 1 --2- 172.21.15.3:0/1633942837 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65dc09d6f0 0x7f65dc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.892+0000 7f65e1ba2700 1 --2- 172.21.15.3:0/1633942837 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65dc09e000 0x7f65dc1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.892+0000 7f65e2ba4700 1 --2- 172.21.15.3:0/1633942837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65dc09e410 0x7f65dc133f30 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f65d4009d90 tx=0x7f65d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5966172e15609f98 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.892+0000 7f65e13a1700 1 -- 172.21.15.3:0/1633942837 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d400e070 con 0x7f65dc09e410 2023-10-03T10:19:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.892+0000 7f65e13a1700 1 -- 172.21.15.3:0/1633942837 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d400bb20 con 0x7f65dc09e410 2023-10-03T10:19:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.892+0000 7f65e13a1700 1 -- 172.21.15.3:0/1633942837 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d4012430 con 0x7f65dc09e410 2023-10-03T10:19:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.893+0000 7f65e4607700 1 -- 172.21.15.3:0/1633942837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65dc09e410 msgr2=0x7f65dc133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.893+0000 7f65e4607700 1 --2- 172.21.15.3:0/1633942837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65dc09e410 0x7f65dc133f30 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f65d4009d90 tx=0x7f65d400b330 comp rx=0 tx=0).stop 2023-10-03T10:19:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.893+0000 7f65e4607700 1 -- 172.21.15.3:0/1633942837 shutdown_connections 2023-10-03T10:19:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.893+0000 7f65e4607700 1 --2- 172.21.15.3:0/1633942837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65dc09e410 0x7f65dc133f30 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:03.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.893+0000 7f65e4607700 1 --2- 172.21.15.3:0/1633942837 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65dc09d6f0 0x7f65dc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:03.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.893+0000 7f65e4607700 1 --2- 172.21.15.3:0/1633942837 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65dc09e000 0x7f65dc1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:03.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.893+0000 7f65e4607700 1 -- 172.21.15.3:0/1633942837 >> 172.21.15.3:0/1633942837 conn(0x7f65dc066f50 msgr2=0x7f65dc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:03.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.893+0000 7f65e4607700 1 -- 172.21.15.3:0/1633942837 shutdown_connections 2023-10-03T10:19:03.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.893+0000 7f65e4607700 1 -- 172.21.15.3:0/1633942837 wait complete. 2023-10-03T10:19:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.894+0000 7f65e4607700 1 Processor -- start 2023-10-03T10:19:03.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.894+0000 7f65e4607700 1 -- start start 2023-10-03T10:19:03.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.895+0000 7f65e4607700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65dc09d6f0 0x7f65dc1a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:03.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.895+0000 7f65e4607700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65dc09e000 0x7f65dc1a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:03.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.895+0000 7f65e4607700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65dc09e410 0x7f65dc1c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:03.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.895+0000 7f65e4607700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc1a6510 con 0x7f65dc09e410 2023-10-03T10:19:03.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.895+0000 7f65e4607700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc1b94d0 con 0x7f65dc09d6f0 2023-10-03T10:19:03.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.895+0000 7f65e4607700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc1b9290 con 0x7f65dc09e000 2023-10-03T10:19:03.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.895+0000 7f65e23a3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65dc09d6f0 0x7f65dc1a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:03.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.895+0000 7f65e23a3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65dc09d6f0 0x7f65dc1a52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41382/0 (socket says 172.21.15.3:41382) 2023-10-03T10:19:03.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.895+0000 7f65e23a3700 1 -- 172.21.15.3:0/2718756667 learned_addr learned my addr 172.21.15.3:0/2718756667 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:03.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.895+0000 7f65e2ba4700 1 --2- 172.21.15.3:0/2718756667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65dc09e410 0x7f65dc1c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:03.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.896+0000 7f65e2ba4700 1 -- 172.21.15.3:0/2718756667 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65dc09e000 msgr2=0x7f65dc1a5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:03.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.896+0000 7f65e2ba4700 1 --2- 172.21.15.3:0/2718756667 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65dc09e000 0x7f65dc1a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:03.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.896+0000 7f65e2ba4700 1 -- 172.21.15.3:0/2718756667 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65dc09d6f0 msgr2=0x7f65dc1a52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:03.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.896+0000 7f65e2ba4700 1 --2- 172.21.15.3:0/2718756667 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65dc09d6f0 0x7f65dc1a52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:03.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.896+0000 7f65e2ba4700 1 -- 172.21.15.3:0/2718756667 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65cc009550 con 0x7f65dc09e410 2023-10-03T10:19:03.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.896+0000 7f65e23a3700 1 --2- 172.21.15.3:0/2718756667 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65dc09d6f0 0x7f65dc1a52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:03.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.896+0000 7f65e2ba4700 1 --2- 172.21.15.3:0/2718756667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65dc09e410 0x7f65dc1c8150 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f65dc09e8b0 tx=0x7f65d4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:03.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.896+0000 7f65d2ffd700 1 -- 172.21.15.3:0/2718756667 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d4004930 con 0x7f65dc09e410 2023-10-03T10:19:03.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.896+0000 7f65d2ffd700 1 -- 172.21.15.3:0/2718756667 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d4020070 con 0x7f65dc09e410 2023-10-03T10:19:03.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.896+0000 7f65e4607700 1 -- 172.21.15.3:0/2718756667 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d4009a70 con 0x7f65dc09e410 2023-10-03T10:19:03.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.896+0000 7f65d2ffd700 1 -- 172.21.15.3:0/2718756667 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d40276c0 con 0x7f65dc09e410 2023-10-03T10:19:03.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.896+0000 7f65e4607700 1 -- 172.21.15.3:0/2718756667 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65dc1a5d70 con 0x7f65dc09e410 2023-10-03T10:19:03.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.897+0000 7f65d2ffd700 1 -- 172.21.15.3:0/2718756667 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f65d4012430 con 0x7f65dc09e410 2023-10-03T10:19:03.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.897+0000 7f65d2ffd700 1 -- 172.21.15.3:0/2718756667 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f65d4022070 con 0x7f65dc09e410 2023-10-03T10:19:03.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.897+0000 7f65d0ff9700 1 -- 172.21.15.3:0/2718756667 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65b80052e0 con 0x7f65dc09e410 2023-10-03T10:19:03.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.903+0000 7f65d2ffd700 1 -- 172.21.15.3:0/2718756667 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f65d4011980 con 0x7f65dc09e410 2023-10-03T10:19:03.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.941+0000 7fd336239700 1 Processor -- start 2023-10-03T10:19:03.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.941+0000 7fd336239700 1 -- start start 2023-10-03T10:19:03.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.941+0000 7fd336239700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd33009d630 0x7fd3300a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:03.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.941+0000 7fd336239700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd330138d80 0x7fd33013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:03.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.941+0000 7fd336239700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3300a2a70 con 0x7fd3300a0a30 2023-10-03T10:19:03.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.941+0000 7fd336239700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3300a01d0 con 0x7fd330138d80 2023-10-03T10:19:03.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.941+0000 7fd336239700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd33009e2b0 con 0x7fd33009d630 2023-10-03T10:19:03.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.941+0000 7fd32ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd330138d80 0x7fd33013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:03.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.941+0000 7fd32f7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd3300a0a30 0x7fd3300a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53594/0 (socket says 172.21.15.3:53594) 2023-10-03T10:19:03.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.941+0000 7fd32f7fe700 1 -- 172.21.15.3:0/3374082525 learned_addr learned my addr 172.21.15.3:0/3374082525 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:03.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.942+0000 7fd32ffff700 1 -- 172.21.15.3:0/3374082525 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd33009d630 msgr2=0x7fd3300a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:03.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.942+0000 7fd32ffff700 1 --2- 172.21.15.3:0/3374082525 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd33009d630 0x7fd3300a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:03.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.942+0000 7fd32ffff700 1 -- 172.21.15.3:0/3374082525 >> v1:172.21.15.3:6789/0 conn(0x7fd3300a0a30 legacy=0x7fd3300a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:03.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.942+0000 7fd32effd700 1 -- 172.21.15.3:0/3374082525 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3825757912 0 0) 0x7fd3300a2a70 con 0x7fd3300a0a30 2023-10-03T10:19:03.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.942+0000 7fd32ffff700 1 -- 172.21.15.3:0/3374082525 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33009dc50 con 0x7fd330138d80 2023-10-03T10:19:03.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.942+0000 7fd32ffff700 1 --2- 172.21.15.3:0/3374082525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd330138d80 0x7fd33013b160 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fd328009d90 tx=0x7fd32800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff6a6843482b4b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:03.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.942+0000 7fd32effd700 1 -- 172.21.15.3:0/3374082525 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32800e070 con 0x7fd330138d80 2023-10-03T10:19:03.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.942+0000 7fd32effd700 1 -- 172.21.15.3:0/3374082525 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32800bbd0 con 0x7fd330138d80 2023-10-03T10:19:03.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.942+0000 7fd32effd700 1 -- 172.21.15.3:0/3374082525 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd328012430 con 0x7fd330138d80 2023-10-03T10:19:03.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.943+0000 7fd336239700 1 -- 172.21.15.3:0/3374082525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd330138d80 msgr2=0x7fd33013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:03.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.943+0000 7fd336239700 1 --2- 172.21.15.3:0/3374082525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd330138d80 0x7fd33013b160 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fd328009d90 tx=0x7fd32800b330 comp rx=0 tx=0).stop 2023-10-03T10:19:03.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.943+0000 7fd336239700 1 -- 172.21.15.3:0/3374082525 shutdown_connections 2023-10-03T10:19:03.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.943+0000 7fd336239700 1 --2- 172.21.15.3:0/3374082525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd330138d80 0x7fd33013b160 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:03.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.943+0000 7fd336239700 1 --2- 172.21.15.3:0/3374082525 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd33009d630 0x7fd3300a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:03.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.943+0000 7fd336239700 1 -- 172.21.15.3:0/3374082525 >> 172.21.15.3:0/3374082525 conn(0x7fd3300a3050 msgr2=0x7fd3300a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:03.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.943+0000 7fd336239700 1 -- 172.21.15.3:0/3374082525 shutdown_connections 2023-10-03T10:19:03.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.943+0000 7fd336239700 1 -- 172.21.15.3:0/3374082525 wait complete. 2023-10-03T10:19:03.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.944+0000 7fd336239700 1 Processor -- start 2023-10-03T10:19:03.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.944+0000 7fd336239700 1 -- start start 2023-10-03T10:19:03.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.945+0000 7fd336239700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd33009d630 0x7fd330133760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:03.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.945+0000 7fd336239700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3300a0a30 0x7fd330133ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:03.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.945+0000 7fd336239700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd330138d80 0x7fd3301341e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:03.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.945+0000 7fd336239700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd330136900 con 0x7fd33009d630 2023-10-03T10:19:03.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.945+0000 7fd336239700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd330135b30 con 0x7fd3300a0a30 2023-10-03T10:19:03.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.945+0000 7fd336239700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd3300a4980 con 0x7fd330138d80 2023-10-03T10:19:03.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.945+0000 7fd32f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd33009d630 0x7fd330133760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:03.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.945+0000 7fd32f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd33009d630 0x7fd330133760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46582/0 (socket says 172.21.15.3:46582) 2023-10-03T10:19:03.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.945+0000 7fd32f7fe700 1 -- 172.21.15.3:0/2894791610 learned_addr learned my addr 172.21.15.3:0/2894791610 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:03.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.945+0000 7fd32ffff700 1 --2- 172.21.15.3:0/2894791610 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd330138d80 0x7fd3301341e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:03.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.946+0000 7fd327fff700 1 --2- 172.21.15.3:0/2894791610 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3300a0a30 0x7fd330133ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:03.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.946+0000 7fd32f7fe700 1 -- 172.21.15.3:0/2894791610 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd330138d80 msgr2=0x7fd3301341e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:03.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.946+0000 7fd32f7fe700 1 --2- 172.21.15.3:0/2894791610 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd330138d80 0x7fd3301341e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:03.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.946+0000 7fd32f7fe700 1 -- 172.21.15.3:0/2894791610 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3300a0a30 msgr2=0x7fd330133ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:03.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.946+0000 7fd32f7fe700 1 --2- 172.21.15.3:0/2894791610 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3300a0a30 0x7fd330133ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:03.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.946+0000 7fd32f7fe700 1 -- 172.21.15.3:0/2894791610 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd328009a70 con 0x7fd33009d630 2023-10-03T10:19:03.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.946+0000 7fd327fff700 1 --2- 172.21.15.3:0/2894791610 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3300a0a30 0x7fd330133ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:03.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.946+0000 7fd32ffff700 1 --2- 172.21.15.3:0/2894791610 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd330138d80 0x7fd3301341e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:03.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.946+0000 7fd32f7fe700 1 --2- 172.21.15.3:0/2894791610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd33009d630 0x7fd330133760 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fd32000c520 tx=0x7fd32000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:03.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.946+0000 7fd32cff9700 1 -- 172.21.15.3:0/2894791610 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd320004c80 con 0x7fd33009d630 2023-10-03T10:19:03.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.946+0000 7fd32cff9700 1 -- 172.21.15.3:0/2894791610 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd320004de0 con 0x7fd33009d630 2023-10-03T10:19:03.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.946+0000 7fd32cff9700 1 -- 172.21.15.3:0/2894791610 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd320007310 con 0x7fd33009d630 2023-10-03T10:19:03.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.946+0000 7fd336239700 1 -- 172.21.15.3:0/2894791610 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd330136bd0 con 0x7fd33009d630 2023-10-03T10:19:03.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.946+0000 7fd336239700 1 -- 172.21.15.3:0/2894791610 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd330136500 con 0x7fd33009d630 2023-10-03T10:19:03.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.947+0000 7fd32cff9700 1 -- 172.21.15.3:0/2894791610 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd320007470 con 0x7fd33009d630 2023-10-03T10:19:03.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.948+0000 7fd32cff9700 1 -- 172.21.15.3:0/2894791610 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd320040d60 con 0x7fd33009d630 2023-10-03T10:19:03.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.948+0000 7fd336239700 1 -- 172.21.15.3:0/2894791610 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd330137de0 con 0x7fd33009d630 2023-10-03T10:19:03.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.953+0000 7fd32cff9700 1 -- 172.21.15.3:0/2894791610 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd320044060 con 0x7fd33009d630 2023-10-03T10:19:03.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.981+0000 7f0d8edd4700 1 Processor -- start 2023-10-03T10:19:03.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.981+0000 7f0d8edd4700 1 -- start start 2023-10-03T10:19:03.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.982+0000 7f0d8edd4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0d8809c650 0x7f0d8809ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:03.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.982+0000 7f0d8edd4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0d8809cf60 0x7f0d88132080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:03.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.982+0000 7f0d8edd4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d8809ba70 con 0x7f0d880a0090 2023-10-03T10:19:03.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.982+0000 7f0d8edd4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d8809e8f0 con 0x7f0d8809c650 2023-10-03T10:19:03.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.982+0000 7f0d8edd4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d880a0760 con 0x7f0d8809cf60 2023-10-03T10:19:03.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.982+0000 7f0d8d371700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0d8809cf60 0x7f0d88132080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:03.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.982+0000 7f0d8d371700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0d8809cf60 0x7f0d88132080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41386/0 (socket says 172.21.15.3:41386) 2023-10-03T10:19:03.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.982+0000 7f0d8d371700 1 -- 172.21.15.3:0/1182882091 learned_addr learned my addr 172.21.15.3:0/1182882091 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:03.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.982+0000 7f0d8cb70700 1 --2- 172.21.15.3:0/1182882091 >> v2:172.21.15.3:3300/0 conn(0x7f0d8809c650 0x7f0d8809ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:03.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.983+0000 7f0d8d371700 1 -- 172.21.15.3:0/1182882091 >> v2:172.21.15.3:3300/0 conn(0x7f0d8809c650 msgr2=0x7f0d8809ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:03.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.983+0000 7f0d8d371700 1 --2- 172.21.15.3:0/1182882091 >> v2:172.21.15.3:3300/0 conn(0x7f0d8809c650 0x7f0d8809ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:03.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.983+0000 7f0d8d371700 1 -- 172.21.15.3:0/1182882091 >> v1:172.21.15.3:6789/0 conn(0x7f0d880a0090 legacy=0x7f0d88131970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:03.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.983+0000 7f0d8d371700 1 -- 172.21.15.3:0/1182882091 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8809a080 con 0x7f0d8809cf60 2023-10-03T10:19:03.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.983+0000 7f0d8cb70700 1 --2- 172.21.15.3:0/1182882091 >> v2:172.21.15.3:3300/0 conn(0x7f0d8809c650 0x7f0d8809ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:03.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.983+0000 7f0d8d371700 1 --2- 172.21.15.3:0/1182882091 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0d8809cf60 0x7f0d88132080 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009e50 tx=0x7f0d8000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82ca557e583e821f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:03.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.983+0000 7f0d877fe700 1 -- 172.21.15.3:0/1182882091 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8000e070 con 0x7f0d8809cf60 2023-10-03T10:19:03.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.983+0000 7f0d877fe700 1 -- 172.21.15.3:0/1182882091 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8000b910 con 0x7f0d8809cf60 2023-10-03T10:19:03.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.983+0000 7f0d877fe700 1 -- 172.21.15.3:0/1182882091 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8000bcc0 con 0x7f0d8809cf60 2023-10-03T10:19:03.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.984+0000 7f0d8edd4700 1 -- 172.21.15.3:0/1182882091 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0d8809cf60 msgr2=0x7f0d88132080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:03.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.984+0000 7f0d8edd4700 1 --2- 172.21.15.3:0/1182882091 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0d8809cf60 0x7f0d88132080 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009e50 tx=0x7f0d8000b330 comp rx=0 tx=0).stop 2023-10-03T10:19:03.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.984+0000 7f0d8edd4700 1 -- 172.21.15.3:0/1182882091 shutdown_connections 2023-10-03T10:19:03.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.984+0000 7f0d8edd4700 1 --2- 172.21.15.3:0/1182882091 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0d8809cf60 0x7f0d88132080 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:03.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.984+0000 7f0d8edd4700 1 --2- 172.21.15.3:0/1182882091 >> v2:172.21.15.3:3300/0 conn(0x7f0d8809c650 0x7f0d8809ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:03.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.984+0000 7f0d8edd4700 1 -- 172.21.15.3:0/1182882091 >> 172.21.15.3:0/1182882091 conn(0x7f0d880a3f70 msgr2=0x7f0d880a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:03.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.984+0000 7f0d8edd4700 1 -- 172.21.15.3:0/1182882091 shutdown_connections 2023-10-03T10:19:03.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.984+0000 7f0d8edd4700 1 -- 172.21.15.3:0/1182882091 wait complete. 2023-10-03T10:19:03.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.985+0000 7f0d8edd4700 1 Processor -- start 2023-10-03T10:19:03.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.985+0000 7f0d8edd4700 1 -- start start 2023-10-03T10:19:03.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.986+0000 7f0d8edd4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0d8809c650 0x7f0d8811a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:03.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.986+0000 7f0d8edd4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d8809cf60 0x7f0d8811ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:03.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.986+0000 7f0d8edd4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0d880a0090 0x7f0d8811b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:03.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.986+0000 7f0d8edd4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8811e350 con 0x7f0d8809cf60 2023-10-03T10:19:03.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.986+0000 7f0d8edd4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8811d990 con 0x7f0d880a0090 2023-10-03T10:19:03.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.986+0000 7f0d8edd4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8809d980 con 0x7f0d8809c650 2023-10-03T10:19:03.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.986+0000 7f0d87fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d8809cf60 0x7f0d8811ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:03.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.986+0000 7f0d8cb70700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0d8809c650 0x7f0d8811a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:03.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.986+0000 7f0d87fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d8809cf60 0x7f0d8811ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46598/0 (socket says 172.21.15.3:46598) 2023-10-03T10:19:03.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.986+0000 7f0d87fff700 1 -- 172.21.15.3:0/3175543746 learned_addr learned my addr 172.21.15.3:0/3175543746 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:03.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.986+0000 7f0d8d371700 1 --2- 172.21.15.3:0/3175543746 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0d880a0090 0x7f0d8811b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:03.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.986+0000 7f0d87fff700 1 -- 172.21.15.3:0/3175543746 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0d8809c650 msgr2=0x7f0d8811a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:03.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.986+0000 7f0d87fff700 1 --2- 172.21.15.3:0/3175543746 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0d8809c650 0x7f0d8811a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:03.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.986+0000 7f0d87fff700 1 -- 172.21.15.3:0/3175543746 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0d880a0090 msgr2=0x7f0d8811b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:03.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.986+0000 7f0d87fff700 1 --2- 172.21.15.3:0/3175543746 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0d880a0090 0x7f0d8811b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:03.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.986+0000 7f0d87fff700 1 -- 172.21.15.3:0/3175543746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c00f610 con 0x7f0d8809cf60 2023-10-03T10:19:03.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.986+0000 7f0d8d371700 1 --2- 172.21.15.3:0/3175543746 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0d880a0090 0x7f0d8811b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:03.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.987+0000 7f0d87fff700 1 --2- 172.21.15.3:0/3175543746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d8809cf60 0x7f0d8811ab60 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400c510 tx=0x7f0d7400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:03.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.987+0000 7f0d857fa700 1 -- 172.21.15.3:0/3175543746 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d740052d0 con 0x7f0d8809cf60 2023-10-03T10:19:03.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.987+0000 7f0d857fa700 1 -- 172.21.15.3:0/3175543746 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d74005430 con 0x7f0d8809cf60 2023-10-03T10:19:03.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.987+0000 7f0d8edd4700 1 -- 172.21.15.3:0/3175543746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d80009b30 con 0x7f0d8809cf60 2023-10-03T10:19:03.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.987+0000 7f0d857fa700 1 -- 172.21.15.3:0/3175543746 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d740038c0 con 0x7f0d8809cf60 2023-10-03T10:19:03.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.987+0000 7f0d8edd4700 1 -- 172.21.15.3:0/3175543746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d8811c1d0 con 0x7f0d8809cf60 2023-10-03T10:19:03.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.988+0000 7f0d857fa700 1 -- 172.21.15.3:0/3175543746 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0d74003a20 con 0x7f0d8809cf60 2023-10-03T10:19:03.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.988+0000 7f0d857fa700 1 -- 172.21.15.3:0/3175543746 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0d74011a60 con 0x7f0d8809cf60 2023-10-03T10:19:03.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.989+0000 7f0d8edd4700 1 -- 172.21.15.3:0/3175543746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d880747f0 con 0x7f0d8809cf60 2023-10-03T10:19:03.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:03.994+0000 7f0d857fa700 1 -- 172.21.15.3:0/3175543746 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0d74044060 con 0x7f0d8809cf60 2023-10-03T10:19:04.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.120+0000 7fd8c7bf5700 1 -- 172.21.15.3:0/1168639048 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd8c0074ea0 con 0x7fd8c009de50 2023-10-03T10:19:04.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.120+0000 7fd8b67fc700 1 -- 172.21.15.3:0/1168639048 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd8bc015020 con 0x7fd8c009de50 2023-10-03T10:19:04.121 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:04.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.122+0000 7fd8c7bf5700 1 -- 172.21.15.3:0/1168639048 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8c009de50 msgr2=0x7fd8c011d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:04.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.122+0000 7fd8c7bf5700 1 --2- 172.21.15.3:0/1168639048 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8c009de50 0x7fd8c011d820 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc009fd0 tx=0x7fd8bc004750 comp rx=0 tx=0).stop 2023-10-03T10:19:04.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.123+0000 7fd8c7bf5700 1 -- 172.21.15.3:0/1168639048 shutdown_connections 2023-10-03T10:19:04.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.123+0000 7fd8c7bf5700 1 --2- 172.21.15.3:0/1168639048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8c01338a0 0x7fd8c011e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:04.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.123+0000 7fd8c7bf5700 1 --2- 172.21.15.3:0/1168639048 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8c009de50 0x7fd8c011d820 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:04.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.123+0000 7fd8c7bf5700 1 --2- 172.21.15.3:0/1168639048 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd8c009e930 0x7fd8c011dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:04.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.123+0000 7fd8c7bf5700 1 -- 172.21.15.3:0/1168639048 >> 172.21.15.3:0/1168639048 conn(0x7fd8c00a2aa0 msgr2=0x7fd8c01badd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:04.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.123+0000 7fd8c7bf5700 1 -- 172.21.15.3:0/1168639048 shutdown_connections 2023-10-03T10:19:04.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.123+0000 7fd8c7bf5700 1 -- 172.21.15.3:0/1168639048 wait complete. 2023-10-03T10:19:04.137 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:04.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.156+0000 7f65d0ff9700 1 -- 172.21.15.3:0/2718756667 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f65b8005590 con 0x7f65dc09e410 2023-10-03T10:19:04.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.156+0000 7f65d2ffd700 1 -- 172.21.15.3:0/2718756667 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f65d4036090 con 0x7f65dc09e410 2023-10-03T10:19:04.156 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:04.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.158+0000 7f65d0ff9700 1 -- 172.21.15.3:0/2718756667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65dc09e410 msgr2=0x7f65dc1c8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:04.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.158+0000 7f65d0ff9700 1 --2- 172.21.15.3:0/2718756667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65dc09e410 0x7f65dc1c8150 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f65dc09e8b0 tx=0x7f65d4004ad0 comp rx=0 tx=0).stop 2023-10-03T10:19:04.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.158+0000 7f65d0ff9700 1 -- 172.21.15.3:0/2718756667 shutdown_connections 2023-10-03T10:19:04.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.158+0000 7f65d0ff9700 1 --2- 172.21.15.3:0/2718756667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65dc09e410 0x7f65dc1c8150 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:04.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.158+0000 7f65d0ff9700 1 --2- 172.21.15.3:0/2718756667 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f65dc09d6f0 0x7f65dc1a52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:04.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.158+0000 7f65d0ff9700 1 --2- 172.21.15.3:0/2718756667 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f65dc09e000 0x7f65dc1a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:04.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.158+0000 7f65d0ff9700 1 -- 172.21.15.3:0/2718756667 >> 172.21.15.3:0/2718756667 conn(0x7f65dc066f50 msgr2=0x7f65dc0a1580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:04.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.158+0000 7f65d0ff9700 1 -- 172.21.15.3:0/2718756667 shutdown_connections 2023-10-03T10:19:04.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.158+0000 7f65d0ff9700 1 -- 172.21.15.3:0/2718756667 wait complete. 2023-10-03T10:19:04.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.166+0000 7fd336239700 1 -- 172.21.15.3:0/2894791610 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd330136fb0 con 0x7fd33009d630 2023-10-03T10:19:04.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.166+0000 7fd32cff9700 1 -- 172.21.15.3:0/2894791610 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd3200123b0 con 0x7fd33009d630 2023-10-03T10:19:04.167 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:04.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.169+0000 7fd336239700 1 -- 172.21.15.3:0/2894791610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd33009d630 msgr2=0x7fd330133760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:04.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.169+0000 7fd336239700 1 --2- 172.21.15.3:0/2894791610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd33009d630 0x7fd330133760 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fd32000c520 tx=0x7fd32000c830 comp rx=0 tx=0).stop 2023-10-03T10:19:04.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.169+0000 7fd336239700 1 -- 172.21.15.3:0/2894791610 shutdown_connections 2023-10-03T10:19:04.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.169+0000 7fd336239700 1 --2- 172.21.15.3:0/2894791610 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd330138d80 0x7fd3301341e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:04.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.169+0000 7fd336239700 1 --2- 172.21.15.3:0/2894791610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd33009d630 0x7fd330133760 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:04.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.169+0000 7fd336239700 1 --2- 172.21.15.3:0/2894791610 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd3300a0a30 0x7fd330133ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:04.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.169+0000 7fd336239700 1 -- 172.21.15.3:0/2894791610 >> 172.21.15.3:0/2894791610 conn(0x7fd3300a3050 msgr2=0x7fd33009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:04.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.169+0000 7fd336239700 1 -- 172.21.15.3:0/2894791610 shutdown_connections 2023-10-03T10:19:04.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.169+0000 7fd336239700 1 -- 172.21.15.3:0/2894791610 wait complete. 2023-10-03T10:19:04.172 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:04.183 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:04.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.201+0000 7f0d8edd4700 1 -- 172.21.15.3:0/3175543746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0d88072840 con 0x7f0d8809cf60 2023-10-03T10:19:04.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.201+0000 7f0d857fa700 1 -- 172.21.15.3:0/3175543746 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0d7402f090 con 0x7f0d8809cf60 2023-10-03T10:19:04.201 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:04.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.203+0000 7f0d8edd4700 1 -- 172.21.15.3:0/3175543746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d8809cf60 msgr2=0x7f0d8811ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:04.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.203+0000 7f0d8edd4700 1 --2- 172.21.15.3:0/3175543746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d8809cf60 0x7f0d8811ab60 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400c510 tx=0x7f0d7400c820 comp rx=0 tx=0).stop 2023-10-03T10:19:04.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.203+0000 7f0d8edd4700 1 -- 172.21.15.3:0/3175543746 shutdown_connections 2023-10-03T10:19:04.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.203+0000 7f0d8edd4700 1 --2- 172.21.15.3:0/3175543746 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0d880a0090 0x7f0d8811b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:04.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.203+0000 7f0d8edd4700 1 --2- 172.21.15.3:0/3175543746 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0d8809c650 0x7f0d8811a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:04.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.203+0000 7f0d8edd4700 1 --2- 172.21.15.3:0/3175543746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d8809cf60 0x7f0d8811ab60 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:04.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.203+0000 7f0d8edd4700 1 -- 172.21.15.3:0/3175543746 >> 172.21.15.3:0/3175543746 conn(0x7f0d880a3f70 msgr2=0x7f0d8812ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:04.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.203+0000 7f0d8edd4700 1 -- 172.21.15.3:0/3175543746 shutdown_connections 2023-10-03T10:19:04.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:04.203+0000 7f0d8edd4700 1 -- 172.21.15.3:0/3175543746 wait complete. 2023-10-03T10:19:04.217 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:05.138 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:05.173 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:05.184 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:05.218 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:05.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.301+0000 7fbd78834700 1 Processor -- start 2023-10-03T10:19:05.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.301+0000 7fbd78834700 1 -- start start 2023-10-03T10:19:05.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.302+0000 7fbd78834700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409a710 0x7fbd7412fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:05.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.302+0000 7fbd78834700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd741303e0 0x7fbd7412c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:05.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.302+0000 7fbd78834700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd741307f0 con 0x7fbd7412c860 2023-10-03T10:19:05.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.302+0000 7fbd78834700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7409aae0 con 0x7fbd741303e0 2023-10-03T10:19:05.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.302+0000 7fbd78834700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd74099c70 con 0x7fbd7409a710 2023-10-03T10:19:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.303+0000 7fbd7259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409a710 0x7fbd7412fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.303+0000 7fbd7259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409a710 0x7fbd7412fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43762/0 (socket says 172.21.15.3:43762) 2023-10-03T10:19:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.303+0000 7fbd7259c700 1 -- 172.21.15.3:0/1503217943 learned_addr learned my addr 172.21.15.3:0/1503217943 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.303+0000 7fbd71d9b700 1 --2- 172.21.15.3:0/1503217943 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd741303e0 0x7fbd7412c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.303+0000 7fbd7259c700 1 -- 172.21.15.3:0/1503217943 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd741303e0 msgr2=0x7fbd7412c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.303+0000 7fbd7259c700 1 --2- 172.21.15.3:0/1503217943 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd741303e0 0x7fbd7412c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.303+0000 7fbd7259c700 1 -- 172.21.15.3:0/1503217943 >> v1:172.21.15.3:6789/0 conn(0x7fbd7412c860 legacy=0x7fbd74136c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.303+0000 7fbd7259c700 1 -- 172.21.15.3:0/1503217943 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd74071450 con 0x7fbd7409a710 2023-10-03T10:19:05.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.303+0000 7fbd7259c700 1 --2- 172.21.15.3:0/1503217943 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409a710 0x7fbd7412fea0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fbd64009950 tx=0x7fbd64009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90f3da7caf4d3db7 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:05.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.304+0000 7fbd7159a700 1 -- 172.21.15.3:0/1503217943 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd64011830 con 0x7fbd7409a710 2023-10-03T10:19:05.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.304+0000 7fbd7159a700 1 -- 172.21.15.3:0/1503217943 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd64011990 con 0x7fbd7409a710 2023-10-03T10:19:05.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.304+0000 7fbd7159a700 1 -- 172.21.15.3:0/1503217943 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd64011d40 con 0x7fbd7409a710 2023-10-03T10:19:05.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.305+0000 7fbd78834700 1 -- 172.21.15.3:0/1503217943 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409a710 msgr2=0x7fbd7412fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:05.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.305+0000 7fbd78834700 1 --2- 172.21.15.3:0/1503217943 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409a710 0x7fbd7412fea0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fbd64009950 tx=0x7fbd64009c60 comp rx=0 tx=0).stop 2023-10-03T10:19:05.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.305+0000 7fbd78834700 1 -- 172.21.15.3:0/1503217943 shutdown_connections 2023-10-03T10:19:05.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.305+0000 7fbd78834700 1 --2- 172.21.15.3:0/1503217943 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409a710 0x7fbd7412fea0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.305+0000 7fbd78834700 1 --2- 172.21.15.3:0/1503217943 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd741303e0 0x7fbd7412c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.305+0000 7fbd78834700 1 -- 172.21.15.3:0/1503217943 >> 172.21.15.3:0/1503217943 conn(0x7fbd7406a770 msgr2=0x7fbd7406ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:05.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.305+0000 7fbd78834700 1 -- 172.21.15.3:0/1503217943 shutdown_connections 2023-10-03T10:19:05.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.306+0000 7fbd78834700 1 -- 172.21.15.3:0/1503217943 wait complete. 2023-10-03T10:19:05.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.307+0000 7fbd78834700 1 Processor -- start 2023-10-03T10:19:05.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.308+0000 7fbd78834700 1 -- start start 2023-10-03T10:19:05.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.308+0000 7fbd78834700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409a710 0x7fbd741b5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:05.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.308+0000 7fbd78834700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd7412c860 0x7fbd741b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:05.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.308+0000 7fbd78834700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd741303e0 0x7fbd741b63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:05.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.308+0000 7fbd78834700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd74129f30 con 0x7fbd7412c860 2023-10-03T10:19:05.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.308+0000 7fbd78834700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd741b83e0 con 0x7fbd741303e0 2023-10-03T10:19:05.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.308+0000 7fbd78834700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd741b8730 con 0x7fbd7409a710 2023-10-03T10:19:05.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.309+0000 7fbd7259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409a710 0x7fbd741b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:05.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.309+0000 7fbd7259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409a710 0x7fbd741b5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43766/0 (socket says 172.21.15.3:43766) 2023-10-03T10:19:05.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.309+0000 7fbd7259c700 1 -- 172.21.15.3:0/1227062673 learned_addr learned my addr 172.21.15.3:0/1227062673 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:05.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.309+0000 7fbd72d9d700 1 --2- 172.21.15.3:0/1227062673 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd741303e0 0x7fbd741b63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:05.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.309+0000 7fbd71d9b700 1 --2- 172.21.15.3:0/1227062673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd7412c860 0x7fbd741b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:05.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.309+0000 7fbd7259c700 1 -- 172.21.15.3:0/1227062673 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd741303e0 msgr2=0x7fbd741b63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:05.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.309+0000 7fbd7259c700 1 --2- 172.21.15.3:0/1227062673 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd741303e0 0x7fbd741b63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.309+0000 7fbd7259c700 1 -- 172.21.15.3:0/1227062673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd7412c860 msgr2=0x7fbd741b5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:05.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.309+0000 7fbd7259c700 1 --2- 172.21.15.3:0/1227062673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd7412c860 0x7fbd741b5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.309+0000 7fbd7259c700 1 -- 172.21.15.3:0/1227062673 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd64009550 con 0x7fbd7409a710 2023-10-03T10:19:05.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.309+0000 7fbd71d9b700 1 --2- 172.21.15.3:0/1227062673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd7412c860 0x7fbd741b5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:19:05.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.309+0000 7fbd72d9d700 1 --2- 172.21.15.3:0/1227062673 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd741303e0 0x7fbd741b63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:05.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.309+0000 7fbd7259c700 1 --2- 172.21.15.3:0/1227062673 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409a710 0x7fbd741b5960 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fbd64000c00 tx=0x7fbd64004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:05.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.310+0000 7fbd62ffd700 1 -- 172.21.15.3:0/1227062673 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd640036a0 con 0x7fbd7409a710 2023-10-03T10:19:05.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.310+0000 7fbd62ffd700 1 -- 172.21.15.3:0/1227062673 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6401e070 con 0x7fbd7409a710 2023-10-03T10:19:05.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.310+0000 7fbd62ffd700 1 -- 172.21.15.3:0/1227062673 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd64003a40 con 0x7fbd7409a710 2023-10-03T10:19:05.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.310+0000 7fbd78834700 1 -- 172.21.15.3:0/1227062673 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd7406f630 con 0x7fbd7409a710 2023-10-03T10:19:05.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.310+0000 7fbd78834700 1 -- 172.21.15.3:0/1227062673 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd741b7180 con 0x7fbd7409a710 2023-10-03T10:19:05.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.311+0000 7fbd62ffd700 1 -- 172.21.15.3:0/1227062673 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbd64003ba0 con 0x7fbd7409a710 2023-10-03T10:19:05.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.311+0000 7fbd78834700 1 -- 172.21.15.3:0/1227062673 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd7404f2a0 con 0x7fbd7409a710 2023-10-03T10:19:05.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.311+0000 7fbd62ffd700 1 -- 172.21.15.3:0/1227062673 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fbd64040690 con 0x7fbd7409a710 2023-10-03T10:19:05.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.321+0000 7fbd62ffd700 1 -- 172.21.15.3:0/1227062673 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbd64023770 con 0x7fbd7409a710 2023-10-03T10:19:05.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.336+0000 7ff26ca2a700 1 Processor -- start 2023-10-03T10:19:05.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.337+0000 7ff26ca2a700 1 -- start start 2023-10-03T10:19:05.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.338+0000 7ff26ca2a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2681316c0 0x7ff268131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:05.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.338+0000 7ff26ca2a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff26809d9f0 0x7ff26806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:05.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.338+0000 7ff26ca2a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff26806acb0 0x7ff26806b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:05.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.338+0000 7ff26ca2a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff26806b630 con 0x7ff26809d9f0 2023-10-03T10:19:05.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.338+0000 7ff26ca2a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26809e130 con 0x7ff26806acb0 2023-10-03T10:19:05.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.338+0000 7ff26ca2a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff2681312b0 con 0x7ff2681316c0 2023-10-03T10:19:05.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.338+0000 7ff266d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff26806acb0 0x7ff26806b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:05.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.338+0000 7ff266d9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff26806acb0 0x7ff26806b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41430/0 (socket says 172.21.15.3:41430) 2023-10-03T10:19:05.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.338+0000 7ff266d9d700 1 -- 172.21.15.3:0/738540893 learned_addr learned my addr 172.21.15.3:0/738540893 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:05.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.338+0000 7ff26659c700 1 --2- 172.21.15.3:0/738540893 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2681316c0 0x7ff268131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:05.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.338+0000 7ff265d9b700 1 --2- 172.21.15.3:0/738540893 >> v2:172.21.15.3:3300/0 conn(0x7ff26809d9f0 0x7ff26806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:05.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.338+0000 7ff266d9d700 1 -- 172.21.15.3:0/738540893 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2681316c0 msgr2=0x7ff268131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:05.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.338+0000 7ff266d9d700 1 --2- 172.21.15.3:0/738540893 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2681316c0 0x7ff268131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.338+0000 7ff266d9d700 1 -- 172.21.15.3:0/738540893 >> v2:172.21.15.3:3300/0 conn(0x7ff26809d9f0 msgr2=0x7ff26806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:05.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.338+0000 7ff266d9d700 1 --2- 172.21.15.3:0/738540893 >> v2:172.21.15.3:3300/0 conn(0x7ff26809d9f0 0x7ff26806a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.338+0000 7ff266d9d700 1 -- 172.21.15.3:0/738540893 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26809f6e0 con 0x7ff26806acb0 2023-10-03T10:19:05.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.339+0000 7ff266d9d700 1 --2- 172.21.15.3:0/738540893 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff26806acb0 0x7ff26806b0f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ff260009e80 tx=0x7ff26000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=779cd7d9d5ccb5f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:05.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.339+0000 7ff26559a700 1 -- 172.21.15.3:0/738540893 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26000e070 con 0x7ff26806acb0 2023-10-03T10:19:05.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.339+0000 7ff26559a700 1 -- 172.21.15.3:0/738540893 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26000b8d0 con 0x7ff26806acb0 2023-10-03T10:19:05.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.339+0000 7ff26559a700 1 -- 172.21.15.3:0/738540893 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26000bc80 con 0x7ff26806acb0 2023-10-03T10:19:05.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.340+0000 7ff26ca2a700 1 -- 172.21.15.3:0/738540893 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff26806acb0 msgr2=0x7ff26806b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:05.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.340+0000 7ff26ca2a700 1 --2- 172.21.15.3:0/738540893 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff26806acb0 0x7ff26806b0f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ff260009e80 tx=0x7ff26000b330 comp rx=0 tx=0).stop 2023-10-03T10:19:05.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.340+0000 7ff26ca2a700 1 -- 172.21.15.3:0/738540893 shutdown_connections 2023-10-03T10:19:05.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.340+0000 7ff26ca2a700 1 --2- 172.21.15.3:0/738540893 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff26806acb0 0x7ff26806b0f0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.340+0000 7ff26ca2a700 1 --2- 172.21.15.3:0/738540893 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff2681316c0 0x7ff268131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.340+0000 7ff26ca2a700 1 --2- 172.21.15.3:0/738540893 >> v2:172.21.15.3:3300/0 conn(0x7ff26809d9f0 0x7ff26806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.340+0000 7ff26ca2a700 1 -- 172.21.15.3:0/738540893 >> 172.21.15.3:0/738540893 conn(0x7ff268066f50 msgr2=0x7ff26809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:05.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.340+0000 7ff26ca2a700 1 -- 172.21.15.3:0/738540893 shutdown_connections 2023-10-03T10:19:05.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.340+0000 7ff26ca2a700 1 -- 172.21.15.3:0/738540893 wait complete. 2023-10-03T10:19:05.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.342+0000 7ff26ca2a700 1 Processor -- start 2023-10-03T10:19:05.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.342+0000 7ff26ca2a700 1 -- start start 2023-10-03T10:19:05.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.343+0000 7ff26ca2a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff26806acb0 0x7ff268119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:05.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.343+0000 7ff26ca2a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff26809d9f0 0x7ff2681199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:05.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.343+0000 7ff26ca2a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2681316c0 0x7ff268119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:05.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.343+0000 7ff26ca2a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26811af20 con 0x7ff2681316c0 2023-10-03T10:19:05.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.343+0000 7ff26ca2a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff268072210 con 0x7ff26809d9f0 2023-10-03T10:19:05.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.343+0000 7ff26ca2a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff26811b7b0 con 0x7ff26806acb0 2023-10-03T10:19:05.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.343+0000 7ff266d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2681316c0 0x7ff268119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:05.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.343+0000 7ff266d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2681316c0 0x7ff268119f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46630/0 (socket says 172.21.15.3:46630) 2023-10-03T10:19:05.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.343+0000 7ff26659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff26806acb0 0x7ff268119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:05.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.343+0000 7ff266d9d700 1 -- 172.21.15.3:0/3910106332 learned_addr learned my addr 172.21.15.3:0/3910106332 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:05.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.343+0000 7ff265d9b700 1 --2- 172.21.15.3:0/3910106332 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff26809d9f0 0x7ff2681199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:05.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.343+0000 7ff265d9b700 1 -- 172.21.15.3:0/3910106332 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff26806acb0 msgr2=0x7ff268119480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:05.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.343+0000 7ff265d9b700 1 --2- 172.21.15.3:0/3910106332 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff26806acb0 0x7ff268119480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.343+0000 7ff265d9b700 1 -- 172.21.15.3:0/3910106332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2681316c0 msgr2=0x7ff268119f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:05.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.343+0000 7ff265d9b700 1 --2- 172.21.15.3:0/3910106332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2681316c0 0x7ff268119f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.343+0000 7ff265d9b700 1 -- 172.21.15.3:0/3910106332 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff260009b30 con 0x7ff26809d9f0 2023-10-03T10:19:05.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.343+0000 7ff26659c700 1 --2- 172.21.15.3:0/3910106332 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff26806acb0 0x7ff268119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:05.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.344+0000 7ff265d9b700 1 --2- 172.21.15.3:0/3910106332 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff26809d9f0 0x7ff2681199c0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff26806b7d0 tx=0x7ff25c00ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:05.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.344+0000 7ff256ffd700 1 -- 172.21.15.3:0/3910106332 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c013680 con 0x7ff26809d9f0 2023-10-03T10:19:05.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.344+0000 7ff256ffd700 1 -- 172.21.15.3:0/3910106332 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25c0137e0 con 0x7ff26809d9f0 2023-10-03T10:19:05.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.344+0000 7ff26ca2a700 1 -- 172.21.15.3:0/3910106332 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff26811a7f0 con 0x7ff26809d9f0 2023-10-03T10:19:05.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.344+0000 7ff256ffd700 1 -- 172.21.15.3:0/3910106332 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c013b50 con 0x7ff26809d9f0 2023-10-03T10:19:05.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.344+0000 7ff26ca2a700 1 -- 172.21.15.3:0/3910106332 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2680a35a0 con 0x7ff26809d9f0 2023-10-03T10:19:05.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.345+0000 7ff256ffd700 1 -- 172.21.15.3:0/3910106332 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff25c013cb0 con 0x7ff26809d9f0 2023-10-03T10:19:05.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.345+0000 7ff256ffd700 1 -- 172.21.15.3:0/3910106332 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff25c03fac0 con 0x7ff26809d9f0 2023-10-03T10:19:05.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.345+0000 7ff26ca2a700 1 -- 172.21.15.3:0/3910106332 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff268074ea0 con 0x7ff26809d9f0 2023-10-03T10:19:05.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.355+0000 7ff256ffd700 1 -- 172.21.15.3:0/3910106332 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff25c021080 con 0x7ff26809d9f0 2023-10-03T10:19:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.379+0000 7f5739c8e700 1 Processor -- start 2023-10-03T10:19:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.380+0000 7f5739c8e700 1 -- start start 2023-10-03T10:19:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.380+0000 7f5739c8e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57341338a0 0x7f5734133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:05.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.380+0000 7f5739c8e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f573409e930 0x7f573413b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:05.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.380+0000 7f5739c8e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f573406ef30 con 0x7f573409de50 2023-10-03T10:19:05.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.380+0000 7f5739c8e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f573406fef0 con 0x7f573409e930 2023-10-03T10:19:05.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.380+0000 7f5739c8e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f57341334b0 con 0x7f57341338a0 2023-10-03T10:19:05.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.381+0000 7f57337fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57341338a0 0x7f5734133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.381+0000 7f5732ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f573409de50 0x7f573409e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53614/0 (socket says 172.21.15.3:53614) 2023-10-03T10:19:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.381+0000 7f5732ffd700 1 -- 172.21.15.3:0/1469101157 learned_addr learned my addr 172.21.15.3:0/1469101157 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.381+0000 7f5733fff700 1 --2- 172.21.15.3:0/1469101157 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f573409e930 0x7f573413b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.381+0000 7f57327fc700 1 -- 172.21.15.3:0/1469101157 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2706843483 0 0) 0x7f573406ef30 con 0x7f573409de50 2023-10-03T10:19:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.381+0000 7f57327fc700 1 -- 172.21.15.3:0/1469101157 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f571c0035b0 con 0x7f573409de50 2023-10-03T10:19:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.381+0000 7f57337fe700 1 -- 172.21.15.3:0/1469101157 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f573409e930 msgr2=0x7f573413b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.381+0000 7f57337fe700 1 --2- 172.21.15.3:0/1469101157 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f573409e930 0x7f573413b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.381+0000 7f57337fe700 1 -- 172.21.15.3:0/1469101157 >> v1:172.21.15.3:6789/0 conn(0x7f573409de50 legacy=0x7f573409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.381+0000 7f57337fe700 1 -- 172.21.15.3:0/1469101157 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573406fc30 con 0x7f57341338a0 2023-10-03T10:19:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.381+0000 7f5733fff700 1 --2- 172.21.15.3:0/1469101157 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f573409e930 0x7f573413b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.382+0000 7f57337fe700 1 --2- 172.21.15.3:0/1469101157 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57341338a0 0x7f5734133c70 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5724009870 tx=0x7f5724009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=651b31b8a44f78c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.382+0000 7f57327fc700 1 -- 172.21.15.3:0/1469101157 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5724011830 con 0x7f57341338a0 2023-10-03T10:19:05.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.382+0000 7f57327fc700 1 -- 172.21.15.3:0/1469101157 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5724011990 con 0x7f57341338a0 2023-10-03T10:19:05.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.382+0000 7f57327fc700 1 -- 172.21.15.3:0/1469101157 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5724011d00 con 0x7f57341338a0 2023-10-03T10:19:05.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.382+0000 7f5739c8e700 1 -- 172.21.15.3:0/1469101157 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57341338a0 msgr2=0x7f5734133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:05.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.382+0000 7f5739c8e700 1 --2- 172.21.15.3:0/1469101157 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57341338a0 0x7f5734133c70 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5724009870 tx=0x7f5724009b80 comp rx=0 tx=0).stop 2023-10-03T10:19:05.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.383+0000 7f5739c8e700 1 -- 172.21.15.3:0/1469101157 shutdown_connections 2023-10-03T10:19:05.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.383+0000 7f5739c8e700 1 --2- 172.21.15.3:0/1469101157 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f573409e930 0x7f573413b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.383+0000 7f5739c8e700 1 --2- 172.21.15.3:0/1469101157 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57341338a0 0x7f5734133c70 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.383+0000 7f5739c8e700 1 -- 172.21.15.3:0/1469101157 >> 172.21.15.3:0/1469101157 conn(0x7f57340a2aa0 msgr2=0x7f57340a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:05.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.383+0000 7f5739c8e700 1 -- 172.21.15.3:0/1469101157 shutdown_connections 2023-10-03T10:19:05.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.383+0000 7f5739c8e700 1 -- 172.21.15.3:0/1469101157 wait complete. 2023-10-03T10:19:05.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.384+0000 7f5739c8e700 1 Processor -- start 2023-10-03T10:19:05.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.384+0000 7f5739c8e700 1 -- start start 2023-10-03T10:19:05.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.385+0000 7f5739c8e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f573409de50 0x7f573411d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:05.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.385+0000 7f5739c8e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f573409e930 0x7f573411db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:05.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.385+0000 7f5739c8e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57341338a0 0x7f573411e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:05.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.385+0000 7f5739c8e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57341208e0 con 0x7f573409e930 2023-10-03T10:19:05.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.385+0000 7f5739c8e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f573406f640 con 0x7f573409de50 2023-10-03T10:19:05.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.385+0000 7f5739c8e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f57340a0d00 con 0x7f57341338a0 2023-10-03T10:19:05.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.385+0000 7f5733fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57341338a0 0x7f573411e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:05.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.385+0000 7f5733fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57341338a0 0x7f573411e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43802/0 (socket says 172.21.15.3:43802) 2023-10-03T10:19:05.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.385+0000 7f5733fff700 1 -- 172.21.15.3:0/2827110206 learned_addr learned my addr 172.21.15.3:0/2827110206 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:05.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.385+0000 7f57337fe700 1 --2- 172.21.15.3:0/2827110206 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f573409de50 0x7f573411d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:05.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.386+0000 7f5732ffd700 1 --2- 172.21.15.3:0/2827110206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f573409e930 0x7f573411db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:05.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.386+0000 7f5733fff700 1 -- 172.21.15.3:0/2827110206 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f573409de50 msgr2=0x7f573411d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:05.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.386+0000 7f5733fff700 1 --2- 172.21.15.3:0/2827110206 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f573409de50 0x7f573411d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.386+0000 7f5733fff700 1 -- 172.21.15.3:0/2827110206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f573409e930 msgr2=0x7f573411db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:05.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.386+0000 7f5733fff700 1 --2- 172.21.15.3:0/2827110206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f573409e930 0x7f573411db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.386+0000 7f5733fff700 1 -- 172.21.15.3:0/2827110206 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57240094a0 con 0x7f57341338a0 2023-10-03T10:19:05.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.386+0000 7f5733fff700 1 --2- 172.21.15.3:0/2827110206 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57341338a0 0x7f573411e0b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f572c009fa0 tx=0x7f572c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:05.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.386+0000 7f571bfff700 1 -- 172.21.15.3:0/2827110206 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c00de60 con 0x7f57341338a0 2023-10-03T10:19:05.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.386+0000 7f571bfff700 1 -- 172.21.15.3:0/2827110206 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572c00f040 con 0x7f57341338a0 2023-10-03T10:19:05.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.386+0000 7f5739c8e700 1 -- 172.21.15.3:0/2827110206 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f573411f240 con 0x7f57341338a0 2023-10-03T10:19:05.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.386+0000 7f571bfff700 1 -- 172.21.15.3:0/2827110206 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c003c80 con 0x7f57341338a0 2023-10-03T10:19:05.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.386+0000 7f5739c8e700 1 -- 172.21.15.3:0/2827110206 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5734121650 con 0x7f57341338a0 2023-10-03T10:19:05.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.387+0000 7f5739c8e700 1 -- 172.21.15.3:0/2827110206 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5734120b10 con 0x7f57341338a0 2023-10-03T10:19:05.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.388+0000 7f571bfff700 1 -- 172.21.15.3:0/2827110206 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f572c004030 con 0x7f57341338a0 2023-10-03T10:19:05.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.388+0000 7f571bfff700 1 -- 172.21.15.3:0/2827110206 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f572c03f860 con 0x7f57341338a0 2023-10-03T10:19:05.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.389+0000 7fc03a5ac700 1 Processor -- start 2023-10-03T10:19:05.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.389+0000 7fc03a5ac700 1 -- start start 2023-10-03T10:19:05.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.390+0000 7fc03a5ac700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc03409d6f0 0x7fc03409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:05.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.390+0000 7fc03a5ac700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc03409e000 0x7fc0341339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:05.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.390+0000 7fc03a5ac700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc03409e410 0x7fc034133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:05.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.390+0000 7fc03a5ac700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc03406b460 con 0x7fc03409e410 2023-10-03T10:19:05.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.390+0000 7fc03a5ac700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03409b120 con 0x7fc03409e000 2023-10-03T10:19:05.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.390+0000 7fc03a5ac700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc03409a7e0 con 0x7fc03409d6f0 2023-10-03T10:19:05.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.390+0000 7fc033fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc03409d6f0 0x7fc03409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:05.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.390+0000 7fc033fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc03409d6f0 0x7fc03409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43812/0 (socket says 172.21.15.3:43812) 2023-10-03T10:19:05.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.390+0000 7fc033fff700 1 -- 172.21.15.3:0/3588584437 learned_addr learned my addr 172.21.15.3:0/3588584437 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:05.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.390+0000 7fc038b49700 1 --2- 172.21.15.3:0/3588584437 >> v2:172.21.15.3:3300/0 conn(0x7fc03409e410 0x7fc034133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:05.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.390+0000 7fc033fff700 1 -- 172.21.15.3:0/3588584437 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc03409e000 msgr2=0x7fc0341339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:05.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.390+0000 7fc033fff700 1 --2- 172.21.15.3:0/3588584437 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc03409e000 0x7fc0341339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.390+0000 7fc033fff700 1 -- 172.21.15.3:0/3588584437 >> v2:172.21.15.3:3300/0 conn(0x7fc03409e410 msgr2=0x7fc034133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:05.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.390+0000 7fc033fff700 1 --2- 172.21.15.3:0/3588584437 >> v2:172.21.15.3:3300/0 conn(0x7fc03409e410 0x7fc034133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.390+0000 7fc033fff700 1 -- 172.21.15.3:0/3588584437 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03409a500 con 0x7fc03409d6f0 2023-10-03T10:19:05.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.390+0000 7fc038b49700 1 --2- 172.21.15.3:0/3588584437 >> v2:172.21.15.3:3300/0 conn(0x7fc03409e410 0x7fc034133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:19:05.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.391+0000 7fc033fff700 1 --2- 172.21.15.3:0/3588584437 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc03409d6f0 0x7fc03409dac0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc024009920 tx=0x7fc024009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daace260add483d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:05.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.397+0000 7fc032ffd700 1 -- 172.21.15.3:0/3588584437 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02400ba20 con 0x7fc03409d6f0 2023-10-03T10:19:05.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.397+0000 7fc032ffd700 1 -- 172.21.15.3:0/3588584437 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02400bb80 con 0x7fc03409d6f0 2023-10-03T10:19:05.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.397+0000 7fc032ffd700 1 -- 172.21.15.3:0/3588584437 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc024009e70 con 0x7fc03409d6f0 2023-10-03T10:19:05.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.397+0000 7f571bfff700 1 -- 172.21.15.3:0/2827110206 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f572c01e3b0 con 0x7f57341338a0 2023-10-03T10:19:05.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.398+0000 7fc03a5ac700 1 -- 172.21.15.3:0/3588584437 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc03409d6f0 msgr2=0x7fc03409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:05.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.398+0000 7fc03a5ac700 1 --2- 172.21.15.3:0/3588584437 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc03409d6f0 0x7fc03409dac0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc024009920 tx=0x7fc024009c30 comp rx=0 tx=0).stop 2023-10-03T10:19:05.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.398+0000 7fc03a5ac700 1 -- 172.21.15.3:0/3588584437 shutdown_connections 2023-10-03T10:19:05.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.398+0000 7fc03a5ac700 1 --2- 172.21.15.3:0/3588584437 >> v2:172.21.15.3:3300/0 conn(0x7fc03409e410 0x7fc034133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.398+0000 7fc03a5ac700 1 --2- 172.21.15.3:0/3588584437 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc03409d6f0 0x7fc03409dac0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.398+0000 7fc03a5ac700 1 --2- 172.21.15.3:0/3588584437 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc03409e000 0x7fc0341339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.398+0000 7fc03a5ac700 1 -- 172.21.15.3:0/3588584437 >> 172.21.15.3:0/3588584437 conn(0x7fc034066f50 msgr2=0x7fc03409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:05.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.398+0000 7fc03a5ac700 1 -- 172.21.15.3:0/3588584437 shutdown_connections 2023-10-03T10:19:05.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.398+0000 7fc03a5ac700 1 -- 172.21.15.3:0/3588584437 wait complete. 2023-10-03T10:19:05.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.400+0000 7fc03a5ac700 1 Processor -- start 2023-10-03T10:19:05.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.400+0000 7fc03a5ac700 1 -- start start 2023-10-03T10:19:05.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.401+0000 7fc03a5ac700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc03409d6f0 0x7fc0341a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:05.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.401+0000 7fc03a5ac700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc03409e000 0x7fc0341a6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:05.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.401+0000 7fc03a5ac700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc03409e410 0x7fc0341a6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:05.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.401+0000 7fc03a5ac700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0341a98e0 con 0x7fc03409e410 2023-10-03T10:19:05.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.401+0000 7fc03a5ac700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03409c710 con 0x7fc03409d6f0 2023-10-03T10:19:05.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.401+0000 7fc03a5ac700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc0340a31a0 con 0x7fc03409e000 2023-10-03T10:19:05.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.401+0000 7fc033fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc03409d6f0 0x7fc0341a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:05.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.401+0000 7fc0337fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc03409e000 0x7fc0341a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:05.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.401+0000 7fc033fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc03409d6f0 0x7fc0341a5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41462/0 (socket says 172.21.15.3:41462) 2023-10-03T10:19:05.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.401+0000 7fc033fff700 1 -- 172.21.15.3:0/1403985653 learned_addr learned my addr 172.21.15.3:0/1403985653 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:05.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.401+0000 7fc038b49700 1 --2- 172.21.15.3:0/1403985653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc03409e410 0x7fc0341a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:05.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.402+0000 7fc0337fe700 1 -- 172.21.15.3:0/1403985653 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc03409d6f0 msgr2=0x7fc0341a5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:05.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.402+0000 7fc0337fe700 1 --2- 172.21.15.3:0/1403985653 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc03409d6f0 0x7fc0341a5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.402+0000 7fc0337fe700 1 -- 172.21.15.3:0/1403985653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc03409e410 msgr2=0x7fc0341a6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:05.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.402+0000 7fc0337fe700 1 --2- 172.21.15.3:0/1403985653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc03409e410 0x7fc0341a6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.402+0000 7fc0337fe700 1 -- 172.21.15.3:0/1403985653 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc024009550 con 0x7fc03409e000 2023-10-03T10:19:05.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.402+0000 7fc033fff700 1 --2- 172.21.15.3:0/1403985653 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc03409d6f0 0x7fc0341a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:05.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.402+0000 7fc038b49700 1 --2- 172.21.15.3:0/1403985653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc03409e410 0x7fc0341a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:05.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.402+0000 7fc0337fe700 1 --2- 172.21.15.3:0/1403985653 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc03409e000 0x7fc0341a6350 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc03406e510 tx=0x7fc02800b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:05.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.402+0000 7fc030ff9700 1 -- 172.21.15.3:0/1403985653 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc028011830 con 0x7fc03409e000 2023-10-03T10:19:05.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.402+0000 7fc030ff9700 1 -- 172.21.15.3:0/1403985653 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc028011990 con 0x7fc03409e000 2023-10-03T10:19:05.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.402+0000 7fc030ff9700 1 -- 172.21.15.3:0/1403985653 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc028011d40 con 0x7fc03409e000 2023-10-03T10:19:05.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.402+0000 7fc03a5ac700 1 -- 172.21.15.3:0/1403985653 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0341a7fb0 con 0x7fc03409e000 2023-10-03T10:19:05.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.403+0000 7fc03a5ac700 1 -- 172.21.15.3:0/1403985653 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0341a79d0 con 0x7fc03409e000 2023-10-03T10:19:05.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.404+0000 7fc030ff9700 1 -- 172.21.15.3:0/1403985653 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc02801a400 con 0x7fc03409e000 2023-10-03T10:19:05.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.404+0000 7fc030ff9700 1 -- 172.21.15.3:0/1403985653 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc02803f6a0 con 0x7fc03409e000 2023-10-03T10:19:05.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.404+0000 7fc03a5ac700 1 -- 172.21.15.3:0/1403985653 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0341a9460 con 0x7fc03409e000 2023-10-03T10:19:05.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.414+0000 7fc030ff9700 1 -- 172.21.15.3:0/1403985653 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc0341a9460 con 0x7fc03409e000 2023-10-03T10:19:05.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.542+0000 7fbd78834700 1 -- 172.21.15.3:0/1227062673 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbd7406ba40 con 0x7fbd7409a710 2023-10-03T10:19:05.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.542+0000 7fbd62ffd700 1 -- 172.21.15.3:0/1227062673 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbd64015030 con 0x7fbd7409a710 2023-10-03T10:19:05.542 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:05.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.544+0000 7fbd78834700 1 -- 172.21.15.3:0/1227062673 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409a710 msgr2=0x7fbd741b5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:05.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.544+0000 7fbd78834700 1 --2- 172.21.15.3:0/1227062673 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409a710 0x7fbd741b5960 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fbd64000c00 tx=0x7fbd64004750 comp rx=0 tx=0).stop 2023-10-03T10:19:05.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.544+0000 7fbd78834700 1 -- 172.21.15.3:0/1227062673 shutdown_connections 2023-10-03T10:19:05.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.544+0000 7fbd78834700 1 --2- 172.21.15.3:0/1227062673 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd741303e0 0x7fbd741b63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.544+0000 7fbd78834700 1 --2- 172.21.15.3:0/1227062673 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbd7409a710 0x7fbd741b5960 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.544+0000 7fbd78834700 1 --2- 172.21.15.3:0/1227062673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd7412c860 0x7fbd741b5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.544+0000 7fbd78834700 1 -- 172.21.15.3:0/1227062673 >> 172.21.15.3:0/1227062673 conn(0x7fbd7406a770 msgr2=0x7fbd7412e680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:05.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.544+0000 7fbd78834700 1 -- 172.21.15.3:0/1227062673 shutdown_connections 2023-10-03T10:19:05.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.545+0000 7fbd78834700 1 -- 172.21.15.3:0/1227062673 wait complete. 2023-10-03T10:19:05.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.570+0000 7ff26ca2a700 1 -- 172.21.15.3:0/3910106332 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff26809e980 con 0x7ff26809d9f0 2023-10-03T10:19:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.570+0000 7ff256ffd700 1 -- 172.21.15.3:0/3910106332 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff25c017020 con 0x7ff26809d9f0 2023-10-03T10:19:05.571 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:05.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.573+0000 7ff26ca2a700 1 -- 172.21.15.3:0/3910106332 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff26809d9f0 msgr2=0x7ff2681199c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:05.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.573+0000 7ff26ca2a700 1 --2- 172.21.15.3:0/3910106332 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff26809d9f0 0x7ff2681199c0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff26806b7d0 tx=0x7ff25c00ebb0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.573+0000 7ff26ca2a700 1 -- 172.21.15.3:0/3910106332 shutdown_connections 2023-10-03T10:19:05.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.573+0000 7ff26ca2a700 1 --2- 172.21.15.3:0/3910106332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2681316c0 0x7ff268119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.573+0000 7ff26ca2a700 1 --2- 172.21.15.3:0/3910106332 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff26806acb0 0x7ff268119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.573+0000 7ff26ca2a700 1 --2- 172.21.15.3:0/3910106332 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff26809d9f0 0x7ff2681199c0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.573+0000 7ff26ca2a700 1 -- 172.21.15.3:0/3910106332 >> 172.21.15.3:0/3910106332 conn(0x7ff268066f50 msgr2=0x7ff26809bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:05.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.573+0000 7ff26ca2a700 1 -- 172.21.15.3:0/3910106332 shutdown_connections 2023-10-03T10:19:05.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.573+0000 7ff26ca2a700 1 -- 172.21.15.3:0/3910106332 wait complete. 2023-10-03T10:19:05.594 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:05.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.623+0000 7f5739c8e700 1 -- 172.21.15.3:0/2827110206 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5734120cf0 con 0x7f57341338a0 2023-10-03T10:19:05.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.623+0000 7f571bfff700 1 -- 172.21.15.3:0/2827110206 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f572c01a070 con 0x7f57341338a0 2023-10-03T10:19:05.623 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:05.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.625+0000 7f5739c8e700 1 -- 172.21.15.3:0/2827110206 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57341338a0 msgr2=0x7f573411e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:05.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.625+0000 7f5739c8e700 1 --2- 172.21.15.3:0/2827110206 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57341338a0 0x7f573411e0b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f572c009fa0 tx=0x7f572c00d3b0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.625+0000 7f5739c8e700 1 -- 172.21.15.3:0/2827110206 shutdown_connections 2023-10-03T10:19:05.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.625+0000 7f5739c8e700 1 --2- 172.21.15.3:0/2827110206 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f57341338a0 0x7f573411e0b0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.625+0000 7f5739c8e700 1 --2- 172.21.15.3:0/2827110206 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f573409de50 0x7f573411d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.625+0000 7f5739c8e700 1 --2- 172.21.15.3:0/2827110206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f573409e930 0x7f573411db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.625+0000 7f5739c8e700 1 -- 172.21.15.3:0/2827110206 >> 172.21.15.3:0/2827110206 conn(0x7f57340a2aa0 msgr2=0x7f57341b9410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:05.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.626+0000 7f5739c8e700 1 -- 172.21.15.3:0/2827110206 shutdown_connections 2023-10-03T10:19:05.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.626+0000 7f5739c8e700 1 -- 172.21.15.3:0/2827110206 wait complete. 2023-10-03T10:19:05.644 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:05.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.652+0000 7fc03a5ac700 1 -- 172.21.15.3:0/1403985653 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc0341a9460 con 0x7fc03409e000 2023-10-03T10:19:05.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.653+0000 7fc030ff9700 1 -- 172.21.15.3:0/1403985653 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc0341a9460 con 0x7fc03409e000 2023-10-03T10:19:05.653 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:05.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.655+0000 7fc03a5ac700 1 -- 172.21.15.3:0/1403985653 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc03409e000 msgr2=0x7fc0341a6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:05.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.655+0000 7fc03a5ac700 1 --2- 172.21.15.3:0/1403985653 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc03409e000 0x7fc0341a6350 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc03406e510 tx=0x7fc02800b730 comp rx=0 tx=0).stop 2023-10-03T10:19:05.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.655+0000 7fc03a5ac700 1 -- 172.21.15.3:0/1403985653 shutdown_connections 2023-10-03T10:19:05.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.655+0000 7fc03a5ac700 1 --2- 172.21.15.3:0/1403985653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc03409e410 0x7fc0341a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.655+0000 7fc03a5ac700 1 --2- 172.21.15.3:0/1403985653 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc03409d6f0 0x7fc0341a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.655+0000 7fc03a5ac700 1 --2- 172.21.15.3:0/1403985653 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc03409e000 0x7fc0341a6350 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:05.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.655+0000 7fc03a5ac700 1 -- 172.21.15.3:0/1403985653 >> 172.21.15.3:0/1403985653 conn(0x7fc034066f50 msgr2=0x7fc0341bae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:05.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.655+0000 7fc03a5ac700 1 -- 172.21.15.3:0/1403985653 shutdown_connections 2023-10-03T10:19:05.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:05.655+0000 7fc03a5ac700 1 -- 172.21.15.3:0/1403985653 wait complete. 2023-10-03T10:19:05.669 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:06.560 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:06.595 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:06.645 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:06.670 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:06.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.712+0000 7fc176cfc700 1 Processor -- start 2023-10-03T10:19:06.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.712+0000 7fc176cfc700 1 -- start start 2023-10-03T10:19:06.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.713+0000 7fc176cfc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc17009d6f0 0x7fc17009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:06.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.713+0000 7fc176cfc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc17009e000 0x7fc1701339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:06.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.713+0000 7fc176cfc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc17009e410 0x7fc170133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:06.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.713+0000 7fc176cfc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc17006b460 con 0x7fc17009d6f0 2023-10-03T10:19:06.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.713+0000 7fc176cfc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17009b120 con 0x7fc17009e000 2023-10-03T10:19:06.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.713+0000 7fc176cfc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc17009a7e0 con 0x7fc17009e410 2023-10-03T10:19:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.713+0000 7fc174a98700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc17009d6f0 0x7fc17009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.713+0000 7fc174a98700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc17009d6f0 0x7fc17009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46680/0 (socket says 172.21.15.3:46680) 2023-10-03T10:19:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.713+0000 7fc174a98700 1 -- 172.21.15.3:0/1639966679 learned_addr learned my addr 172.21.15.3:0/1639966679 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.713+0000 7fc175299700 1 --2- 172.21.15.3:0/1639966679 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc17009e410 0x7fc170133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.714+0000 7fc16ffff700 1 --2- 172.21.15.3:0/1639966679 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc17009e000 0x7fc1701339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.714+0000 7fc174a98700 1 -- 172.21.15.3:0/1639966679 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc17009e410 msgr2=0x7fc170133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.714+0000 7fc174a98700 1 --2- 172.21.15.3:0/1639966679 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc17009e410 0x7fc170133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.714+0000 7fc174a98700 1 -- 172.21.15.3:0/1639966679 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc17009e000 msgr2=0x7fc1701339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.714+0000 7fc174a98700 1 --2- 172.21.15.3:0/1639966679 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc17009e000 0x7fc1701339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.714+0000 7fc174a98700 1 -- 172.21.15.3:0/1639966679 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17009a500 con 0x7fc17009d6f0 2023-10-03T10:19:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.714+0000 7fc174a98700 1 --2- 172.21.15.3:0/1639966679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc17009d6f0 0x7fc17009dac0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fc1640098f0 tx=0x7fc164009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb324f195b45c60 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.714+0000 7fc16f7fe700 1 -- 172.21.15.3:0/1639966679 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164011830 con 0x7fc17009d6f0 2023-10-03T10:19:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.714+0000 7fc16f7fe700 1 -- 172.21.15.3:0/1639966679 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc164011990 con 0x7fc17009d6f0 2023-10-03T10:19:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.714+0000 7fc16f7fe700 1 -- 172.21.15.3:0/1639966679 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164011d20 con 0x7fc17009d6f0 2023-10-03T10:19:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.715+0000 7fc176cfc700 1 -- 172.21.15.3:0/1639966679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc17009d6f0 msgr2=0x7fc17009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.715+0000 7fc176cfc700 1 --2- 172.21.15.3:0/1639966679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc17009d6f0 0x7fc17009dac0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fc1640098f0 tx=0x7fc164009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.715+0000 7fc176cfc700 1 -- 172.21.15.3:0/1639966679 shutdown_connections 2023-10-03T10:19:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.715+0000 7fc176cfc700 1 --2- 172.21.15.3:0/1639966679 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc17009e410 0x7fc170133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.715+0000 7fc176cfc700 1 --2- 172.21.15.3:0/1639966679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc17009d6f0 0x7fc17009dac0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.715+0000 7fc176cfc700 1 --2- 172.21.15.3:0/1639966679 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc17009e000 0x7fc1701339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.715+0000 7fc176cfc700 1 -- 172.21.15.3:0/1639966679 >> 172.21.15.3:0/1639966679 conn(0x7fc170066f50 msgr2=0x7fc17009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.715+0000 7fc176cfc700 1 -- 172.21.15.3:0/1639966679 shutdown_connections 2023-10-03T10:19:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.715+0000 7fc176cfc700 1 -- 172.21.15.3:0/1639966679 wait complete. 2023-10-03T10:19:06.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.716+0000 7fc176cfc700 1 Processor -- start 2023-10-03T10:19:06.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.717+0000 7fc176cfc700 1 -- start start 2023-10-03T10:19:06.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.717+0000 7fc176cfc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc17009d6f0 0x7fc1701a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.717+0000 7fc176cfc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc17009e000 0x7fc1701a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.717+0000 7fc176cfc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc17009e410 0x7fc1701c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.717+0000 7fc176cfc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1700a1f20 con 0x7fc17009e410 2023-10-03T10:19:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.717+0000 7fc176cfc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1701a80a0 con 0x7fc17009d6f0 2023-10-03T10:19:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.717+0000 7fc176cfc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc1701b91f0 con 0x7fc17009e000 2023-10-03T10:19:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.717+0000 7fc174a98700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc17009d6f0 0x7fc1701a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.717+0000 7fc174a98700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc17009d6f0 0x7fc1701a5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41486/0 (socket says 172.21.15.3:41486) 2023-10-03T10:19:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.717+0000 7fc174a98700 1 -- 172.21.15.3:0/1161431749 learned_addr learned my addr 172.21.15.3:0/1161431749 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.718+0000 7fc175299700 1 --2- 172.21.15.3:0/1161431749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc17009e410 0x7fc1701c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.718+0000 7fc16ffff700 1 --2- 172.21.15.3:0/1161431749 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc17009e000 0x7fc1701a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.718+0000 7fc174a98700 1 -- 172.21.15.3:0/1161431749 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc17009e000 msgr2=0x7fc1701a5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.718+0000 7fc174a98700 1 --2- 172.21.15.3:0/1161431749 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc17009e000 0x7fc1701a5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.718+0000 7fc174a98700 1 -- 172.21.15.3:0/1161431749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc17009e410 msgr2=0x7fc1701c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:06.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.718+0000 7fc174a98700 1 --2- 172.21.15.3:0/1161431749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc17009e410 0x7fc1701c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.718+0000 7fc174a98700 1 -- 172.21.15.3:0/1161431749 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc164009550 con 0x7fc17009d6f0 2023-10-03T10:19:06.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.718+0000 7fc16ffff700 1 --2- 172.21.15.3:0/1161431749 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc17009e000 0x7fc1701a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:06.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.718+0000 7fc174a98700 1 --2- 172.21.15.3:0/1161431749 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc17009d6f0 0x7fc1701a5130 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fc164011db0 tx=0x7fc164004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:06.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.718+0000 7fc16d7fa700 1 -- 172.21.15.3:0/1161431749 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164004240 con 0x7fc17009d6f0 2023-10-03T10:19:06.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.719+0000 7fc16d7fa700 1 -- 172.21.15.3:0/1161431749 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16401f070 con 0x7fc17009d6f0 2023-10-03T10:19:06.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.719+0000 7fc16d7fa700 1 -- 172.21.15.3:0/1161431749 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1640045a0 con 0x7fc17009d6f0 2023-10-03T10:19:06.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.719+0000 7fc176cfc700 1 -- 172.21.15.3:0/1161431749 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1701a4d40 con 0x7fc17009d6f0 2023-10-03T10:19:06.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.719+0000 7fc176cfc700 1 -- 172.21.15.3:0/1161431749 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1701a6470 con 0x7fc17009d6f0 2023-10-03T10:19:06.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.720+0000 7fc176cfc700 1 -- 172.21.15.3:0/1161431749 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1701a6980 con 0x7fc17009d6f0 2023-10-03T10:19:06.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.721+0000 7fc16d7fa700 1 -- 172.21.15.3:0/1161431749 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc164021030 con 0x7fc17009d6f0 2023-10-03T10:19:06.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.721+0000 7fc16d7fa700 1 -- 172.21.15.3:0/1161431749 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc164040d40 con 0x7fc17009d6f0 2023-10-03T10:19:06.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.726+0000 7fc16d7fa700 1 -- 172.21.15.3:0/1161431749 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc1640036a0 con 0x7fc17009d6f0 2023-10-03T10:19:06.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.758+0000 7f8f77b6b700 1 Processor -- start 2023-10-03T10:19:06.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.758+0000 7f8f77b6b700 1 -- start start 2023-10-03T10:19:06.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.758+0000 7f8f77b6b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f70133540 0x7f8f70135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:06.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.758+0000 7f8f77b6b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8f70135e60 0x7f8f70138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:06.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.758+0000 7f8f77b6b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f700716a0 con 0x7f8f70130540 2023-10-03T10:19:06.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.758+0000 7f8f77b6b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f70071290 con 0x7f8f70135e60 2023-10-03T10:19:06.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.758+0000 7f8f77b6b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8f70130340 con 0x7f8f70133540 2023-10-03T10:19:06.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.759+0000 7f8f75106700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f70133540 0x7f8f70135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:06.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.759+0000 7f8f75106700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f70133540 0x7f8f70135920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43850/0 (socket says 172.21.15.3:43850) 2023-10-03T10:19:06.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.759+0000 7f8f75106700 1 -- 172.21.15.3:0/1650679281 learned_addr learned my addr 172.21.15.3:0/1650679281 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:06.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.759+0000 7f8f76108700 1 --2- 172.21.15.3:0/1650679281 >> v2:172.21.15.3:3300/0 conn(0x7f8f70135e60 0x7f8f70138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:06.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.759+0000 7f8f75106700 1 -- 172.21.15.3:0/1650679281 >> v2:172.21.15.3:3300/0 conn(0x7f8f70135e60 msgr2=0x7f8f70138240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:06.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.759+0000 7f8f75106700 1 --2- 172.21.15.3:0/1650679281 >> v2:172.21.15.3:3300/0 conn(0x7f8f70135e60 0x7f8f70138240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.759+0000 7f8f75106700 1 -- 172.21.15.3:0/1650679281 >> v1:172.21.15.3:6789/0 conn(0x7f8f70130540 legacy=0x7f8f70130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:06.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.759+0000 7f8f75106700 1 -- 172.21.15.3:0/1650679281 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f70070b80 con 0x7f8f70133540 2023-10-03T10:19:06.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.759+0000 7f8f75106700 1 --2- 172.21.15.3:0/1650679281 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f70133540 0x7f8f70135920 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000bb90 tx=0x7f8f6000bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94d2191b07967700 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:06.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.760+0000 7f8f74905700 1 -- 172.21.15.3:0/1650679281 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6000dd30 con 0x7f8f70133540 2023-10-03T10:19:06.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.760+0000 7f8f74905700 1 -- 172.21.15.3:0/1650679281 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6000de90 con 0x7f8f70133540 2023-10-03T10:19:06.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.760+0000 7f8f74905700 1 -- 172.21.15.3:0/1650679281 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f60010690 con 0x7f8f70133540 2023-10-03T10:19:06.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.760+0000 7f8f77b6b700 1 -- 172.21.15.3:0/1650679281 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f70133540 msgr2=0x7f8f70135920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:06.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.760+0000 7f8f77b6b700 1 --2- 172.21.15.3:0/1650679281 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f70133540 0x7f8f70135920 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000bb90 tx=0x7f8f6000bea0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.760+0000 7f8f77b6b700 1 -- 172.21.15.3:0/1650679281 shutdown_connections 2023-10-03T10:19:06.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.760+0000 7f8f77b6b700 1 --2- 172.21.15.3:0/1650679281 >> v2:172.21.15.3:3300/0 conn(0x7f8f70135e60 0x7f8f70138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.760+0000 7f8f77b6b700 1 --2- 172.21.15.3:0/1650679281 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f70133540 0x7f8f70135920 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.761+0000 7f8f77b6b700 1 -- 172.21.15.3:0/1650679281 >> 172.21.15.3:0/1650679281 conn(0x7f8f700a2aa0 msgr2=0x7f8f700a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:06.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.761+0000 7f8f77b6b700 1 -- 172.21.15.3:0/1650679281 shutdown_connections 2023-10-03T10:19:06.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.761+0000 7f8f77b6b700 1 -- 172.21.15.3:0/1650679281 wait complete. 2023-10-03T10:19:06.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.762+0000 7f8f77b6b700 1 Processor -- start 2023-10-03T10:19:06.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.762+0000 7f8f77b6b700 1 -- start start 2023-10-03T10:19:06.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.763+0000 7f8f77b6b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f70130540 0x7f8f701c4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:06.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.763+0000 7f8f77b6b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f70133540 0x7f8f701c4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:06.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.763+0000 7f8f77b6b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8f70135e60 0x7f8f7009ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:06.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.763+0000 7f8f77b6b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f700a22f0 con 0x7f8f70133540 2023-10-03T10:19:06.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.763+0000 7f8f77b6b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7011a740 con 0x7f8f70135e60 2023-10-03T10:19:06.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.763+0000 7f8f77b6b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7011a1d0 con 0x7f8f70130540 2023-10-03T10:19:06.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.763+0000 7f8f75106700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f70133540 0x7f8f701c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:06.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.763+0000 7f8f75907700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f70130540 0x7f8f701c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:06.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.763+0000 7f8f75907700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f70130540 0x7f8f701c4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43856/0 (socket says 172.21.15.3:43856) 2023-10-03T10:19:06.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.763+0000 7f8f75106700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f70133540 0x7f8f701c4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46702/0 (socket says 172.21.15.3:46702) 2023-10-03T10:19:06.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.763+0000 7f8f75907700 1 -- 172.21.15.3:0/2003891009 learned_addr learned my addr 172.21.15.3:0/2003891009 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:06.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.764+0000 7f8f75106700 1 -- 172.21.15.3:0/2003891009 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f70130540 msgr2=0x7f8f701c4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:06.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.764+0000 7f8f76108700 1 --2- 172.21.15.3:0/2003891009 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8f70135e60 0x7f8f7009ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:06.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.764+0000 7f8f75106700 1 --2- 172.21.15.3:0/2003891009 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f70130540 0x7f8f701c4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.764+0000 7f8f75106700 1 -- 172.21.15.3:0/2003891009 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8f70135e60 msgr2=0x7f8f7009ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:06.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.764+0000 7f8f75106700 1 --2- 172.21.15.3:0/2003891009 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8f70135e60 0x7f8f7009ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.764+0000 7f8f75106700 1 -- 172.21.15.3:0/2003891009 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6000b7f0 con 0x7f8f70133540 2023-10-03T10:19:06.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.764+0000 7f8f76108700 1 --2- 172.21.15.3:0/2003891009 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8f70135e60 0x7f8f7009ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:19:06.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.764+0000 7f8f75907700 1 --2- 172.21.15.3:0/2003891009 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f70130540 0x7f8f701c4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:06.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.764+0000 7f8f75106700 1 --2- 172.21.15.3:0/2003891009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f70133540 0x7f8f701c4780 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f8f60018000 tx=0x7f8f60004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:06.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.764+0000 7f8f667fc700 1 -- 172.21.15.3:0/2003891009 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f60003bf0 con 0x7f8f70133540 2023-10-03T10:19:06.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.764+0000 7f8f667fc700 1 -- 172.21.15.3:0/2003891009 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f60003d70 con 0x7f8f70133540 2023-10-03T10:19:06.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.764+0000 7f8f667fc700 1 -- 172.21.15.3:0/2003891009 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f600106b0 con 0x7f8f70133540 2023-10-03T10:19:06.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.764+0000 7f8f77b6b700 1 -- 172.21.15.3:0/2003891009 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f7011b1b0 con 0x7f8f70133540 2023-10-03T10:19:06.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.764+0000 7f8f77b6b700 1 -- 172.21.15.3:0/2003891009 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f70119270 con 0x7f8f70133540 2023-10-03T10:19:06.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.765+0000 7f8f667fc700 1 -- 172.21.15.3:0/2003891009 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8f60019400 con 0x7f8f70133540 2023-10-03T10:19:06.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.765+0000 7f8f77b6b700 1 -- 172.21.15.3:0/2003891009 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f7011a580 con 0x7f8f70133540 2023-10-03T10:19:06.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.766+0000 7f8f667fc700 1 -- 172.21.15.3:0/2003891009 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8f60040ee0 con 0x7f8f70133540 2023-10-03T10:19:06.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.775+0000 7f8f667fc700 1 -- 172.21.15.3:0/2003891009 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8f6000e950 con 0x7f8f70133540 2023-10-03T10:19:06.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.804+0000 7f98b9036700 1 Processor -- start 2023-10-03T10:19:06.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.805+0000 7f98b9036700 1 -- start start 2023-10-03T10:19:06.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.805+0000 7f98b9036700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f98b406a770 0x7f98b409d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:06.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.805+0000 7f98b9036700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f98b413d1f0 0x7f98b413f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:06.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.805+0000 7f98b9036700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b409a810 con 0x7f98b409dc30 2023-10-03T10:19:06.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.805+0000 7f98b9036700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b406ab40 con 0x7f98b413d1f0 2023-10-03T10:19:06.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.805+0000 7f98b9036700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98b409ebc0 con 0x7f98b406a770 2023-10-03T10:19:06.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.806+0000 7f98abfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f98b409dc30 0x7f98b409e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53632/0 (socket says 172.21.15.3:53632) 2023-10-03T10:19:06.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.806+0000 7f98abfff700 1 -- 172.21.15.3:0/2108985447 learned_addr learned my addr 172.21.15.3:0/2108985447 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:06.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.806+0000 7f98b2d9d700 1 --2- 172.21.15.3:0/2108985447 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f98b406a770 0x7f98b409d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:06.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.806+0000 7f98b359e700 1 --2- 172.21.15.3:0/2108985447 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f98b413d1f0 0x7f98b413f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:06.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.806+0000 7f98b359e700 1 -- 172.21.15.3:0/2108985447 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f98b406a770 msgr2=0x7f98b409d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:06.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.806+0000 7f98b359e700 1 --2- 172.21.15.3:0/2108985447 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f98b406a770 0x7f98b409d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.806+0000 7f98b359e700 1 -- 172.21.15.3:0/2108985447 >> v1:172.21.15.3:6789/0 conn(0x7f98b409dc30 legacy=0x7f98b409e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:06.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.806+0000 7f98b359e700 1 -- 172.21.15.3:0/2108985447 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b409e920 con 0x7f98b413d1f0 2023-10-03T10:19:06.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.806+0000 7f98b2d9d700 1 --2- 172.21.15.3:0/2108985447 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f98b406a770 0x7f98b409d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:06.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.806+0000 7f98b359e700 1 --2- 172.21.15.3:0/2108985447 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f98b413d1f0 0x7f98b413f5d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009dc0 tx=0x7f98ac00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebb2890dfdc18bf3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:06.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.807+0000 7f98b259c700 1 -- 172.21.15.3:0/2108985447 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac00e070 con 0x7f98b413d1f0 2023-10-03T10:19:06.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.807+0000 7f98b259c700 1 -- 172.21.15.3:0/2108985447 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ac00bb20 con 0x7f98b413d1f0 2023-10-03T10:19:06.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.807+0000 7f98b259c700 1 -- 172.21.15.3:0/2108985447 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac012430 con 0x7f98b413d1f0 2023-10-03T10:19:06.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.808+0000 7f98b9036700 1 -- 172.21.15.3:0/2108985447 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f98b413d1f0 msgr2=0x7f98b413f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:06.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.808+0000 7f98b9036700 1 --2- 172.21.15.3:0/2108985447 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f98b413d1f0 0x7f98b413f5d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009dc0 tx=0x7f98ac00b330 comp rx=0 tx=0).stop 2023-10-03T10:19:06.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.808+0000 7f98b9036700 1 -- 172.21.15.3:0/2108985447 shutdown_connections 2023-10-03T10:19:06.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.808+0000 7f98b9036700 1 --2- 172.21.15.3:0/2108985447 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f98b413d1f0 0x7f98b413f5d0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.808+0000 7f98b9036700 1 --2- 172.21.15.3:0/2108985447 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f98b406a770 0x7f98b409d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.808+0000 7f98b9036700 1 -- 172.21.15.3:0/2108985447 >> 172.21.15.3:0/2108985447 conn(0x7f98b4066f50 msgr2=0x7f98b409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:06.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.808+0000 7f98b9036700 1 -- 172.21.15.3:0/2108985447 shutdown_connections 2023-10-03T10:19:06.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.808+0000 7f98b9036700 1 -- 172.21.15.3:0/2108985447 wait complete. 2023-10-03T10:19:06.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.809+0000 7f98b9036700 1 Processor -- start 2023-10-03T10:19:06.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.809+0000 7f98b9036700 1 -- start start 2023-10-03T10:19:06.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.810+0000 7f98b9036700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98b406a770 0x7f98b41cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:06.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.810+0000 7f98b9036700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f98b409dc30 0x7f98b41ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:06.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.810+0000 7f98b9036700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f98b413d1f0 0x7f98b41cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:06.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.810+0000 7f98b9036700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b4071ed0 con 0x7f98b406a770 2023-10-03T10:19:06.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.810+0000 7f98b9036700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b4072450 con 0x7f98b409dc30 2023-10-03T10:19:06.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.810+0000 7f98b9036700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98b406fbc0 con 0x7f98b413d1f0 2023-10-03T10:19:06.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.810+0000 7f98abfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f98b409dc30 0x7f98b41ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:06.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.810+0000 7f98abfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f98b409dc30 0x7f98b41ccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41510/0 (socket says 172.21.15.3:41510) 2023-10-03T10:19:06.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.810+0000 7f98abfff700 1 -- 172.21.15.3:0/1398955119 learned_addr learned my addr 172.21.15.3:0/1398955119 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:06.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.810+0000 7f98b2d9d700 1 --2- 172.21.15.3:0/1398955119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98b406a770 0x7f98b41cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:06.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.810+0000 7f98b359e700 1 --2- 172.21.15.3:0/1398955119 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f98b413d1f0 0x7f98b41cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:06.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.810+0000 7f98abfff700 1 -- 172.21.15.3:0/1398955119 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f98b413d1f0 msgr2=0x7f98b41cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:06.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.810+0000 7f98abfff700 1 --2- 172.21.15.3:0/1398955119 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f98b413d1f0 0x7f98b41cd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.810+0000 7f98abfff700 1 -- 172.21.15.3:0/1398955119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98b406a770 msgr2=0x7f98b41cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:06.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.810+0000 7f98abfff700 1 --2- 172.21.15.3:0/1398955119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98b406a770 0x7f98b41cc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.810+0000 7f98abfff700 1 -- 172.21.15.3:0/1398955119 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a40094a0 con 0x7f98b409dc30 2023-10-03T10:19:06.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.810+0000 7f98b359e700 1 --2- 172.21.15.3:0/1398955119 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f98b413d1f0 0x7f98b41cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:06.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.810+0000 7f98abfff700 1 --2- 172.21.15.3:0/1398955119 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f98b409dc30 0x7f98b41ccf00 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f989c00c500 tx=0x7f989c00c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:06.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.811+0000 7f98ab7fe700 1 -- 172.21.15.3:0/1398955119 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c005d70 con 0x7f98b409dc30 2023-10-03T10:19:06.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.811+0000 7f98ab7fe700 1 -- 172.21.15.3:0/1398955119 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c0070d0 con 0x7f98b409dc30 2023-10-03T10:19:06.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.811+0000 7f98ab7fe700 1 -- 172.21.15.3:0/1398955119 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c007440 con 0x7f98b409dc30 2023-10-03T10:19:06.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.811+0000 7f98b9036700 1 -- 172.21.15.3:0/1398955119 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ac009a70 con 0x7f98b409dc30 2023-10-03T10:19:06.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.811+0000 7f98b9036700 1 -- 172.21.15.3:0/1398955119 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b406f8b0 con 0x7f98b409dc30 2023-10-03T10:19:06.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.812+0000 7f98ab7fe700 1 -- 172.21.15.3:0/1398955119 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f989c0075a0 con 0x7f98b409dc30 2023-10-03T10:19:06.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.812+0000 7f98ab7fe700 1 -- 172.21.15.3:0/1398955119 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f989c03eac0 con 0x7f98b409dc30 2023-10-03T10:19:06.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.812+0000 7f98b9036700 1 -- 172.21.15.3:0/1398955119 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b4074ea0 con 0x7f98b409dc30 2023-10-03T10:19:06.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.822+0000 7f98ab7fe700 1 -- 172.21.15.3:0/1398955119 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f989c017020 con 0x7f98b409dc30 2023-10-03T10:19:06.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.850+0000 7fa3dd52b700 1 Processor -- start 2023-10-03T10:19:06.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.850+0000 7fa3dd52b700 1 -- start start 2023-10-03T10:19:06.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.851+0000 7fa3dd52b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3d8133420 0x7fa3d81337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:06.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.851+0000 7fa3dd52b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3d8134260 0x7fa3d81306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:06.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.851+0000 7fa3dd52b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3d806b990 con 0x7fa3d8133e30 2023-10-03T10:19:06.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.851+0000 7fa3dd52b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d80a0590 con 0x7fa3d8134260 2023-10-03T10:19:06.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.851+0000 7fa3dd52b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d809fb80 con 0x7fa3d8133420 2023-10-03T10:19:06.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.852+0000 7fa3d67fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa3d8133e30 0x7fa3d812ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53636/0 (socket says 172.21.15.3:53636) 2023-10-03T10:19:06.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.852+0000 7fa3d6ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3d8133420 0x7fa3d81337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:06.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.852+0000 7fa3d67fc700 1 -- 172.21.15.3:0/24396051 learned_addr learned my addr 172.21.15.3:0/24396051 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:06.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.852+0000 7fa3d77fe700 1 --2- 172.21.15.3:0/24396051 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3d8134260 0x7fa3d81306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:06.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.852+0000 7fa3d77fe700 1 -- 172.21.15.3:0/24396051 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3d8133420 msgr2=0x7fa3d81337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:06.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.852+0000 7fa3d77fe700 1 --2- 172.21.15.3:0/24396051 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3d8133420 0x7fa3d81337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.852+0000 7fa3d77fe700 1 -- 172.21.15.3:0/24396051 >> v1:172.21.15.3:6789/0 conn(0x7fa3d8133e30 legacy=0x7fa3d812ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:06.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.852+0000 7fa3d77fe700 1 -- 172.21.15.3:0/24396051 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d80a0030 con 0x7fa3d8134260 2023-10-03T10:19:06.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.852+0000 7fa3d77fe700 1 --2- 172.21.15.3:0/24396051 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3d8134260 0x7fa3d81306a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009dc0 tx=0x7fa3d000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f45bfb66343febeb server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:06.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.852+0000 7fa3d5ffb700 1 -- 172.21.15.3:0/24396051 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d000e070 con 0x7fa3d8134260 2023-10-03T10:19:06.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.852+0000 7fa3d5ffb700 1 -- 172.21.15.3:0/24396051 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d000bb20 con 0x7fa3d8134260 2023-10-03T10:19:06.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.853+0000 7fa3d5ffb700 1 -- 172.21.15.3:0/24396051 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0012430 con 0x7fa3d8134260 2023-10-03T10:19:06.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.853+0000 7fa3dd52b700 1 -- 172.21.15.3:0/24396051 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3d8134260 msgr2=0x7fa3d81306a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:06.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.853+0000 7fa3dd52b700 1 --2- 172.21.15.3:0/24396051 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3d8134260 0x7fa3d81306a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009dc0 tx=0x7fa3d000b330 comp rx=0 tx=0).stop 2023-10-03T10:19:06.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.854+0000 7fa3dd52b700 1 -- 172.21.15.3:0/24396051 shutdown_connections 2023-10-03T10:19:06.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.854+0000 7fa3dd52b700 1 --2- 172.21.15.3:0/24396051 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3d8134260 0x7fa3d81306a0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.854+0000 7fa3dd52b700 1 --2- 172.21.15.3:0/24396051 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3d8133420 0x7fa3d81337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.854+0000 7fa3dd52b700 1 -- 172.21.15.3:0/24396051 >> 172.21.15.3:0/24396051 conn(0x7fa3d80a4210 msgr2=0x7fa3d80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:06.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.854+0000 7fa3dd52b700 1 -- 172.21.15.3:0/24396051 shutdown_connections 2023-10-03T10:19:06.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.854+0000 7fa3dd52b700 1 -- 172.21.15.3:0/24396051 wait complete. 2023-10-03T10:19:06.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.855+0000 7fa3dd52b700 1 Processor -- start 2023-10-03T10:19:06.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.856+0000 7fa3dd52b700 1 -- start start 2023-10-03T10:19:06.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.856+0000 7fa3dd52b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3d8133420 0x7fa3d809c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:06.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.856+0000 7fa3dd52b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3d8133e30 0x7fa3d809cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:06.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.856+0000 7fa3dd52b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3d8134260 0x7fa3d809d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:06.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.856+0000 7fa3dd52b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d81203f0 con 0x7fa3d8133e30 2023-10-03T10:19:06.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.856+0000 7fa3dd52b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d812d820 con 0x7fa3d8133420 2023-10-03T10:19:06.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.856+0000 7fa3dd52b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d81208f0 con 0x7fa3d8134260 2023-10-03T10:19:06.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.856+0000 7fa3d6ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3d8133420 0x7fa3d809c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:06.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.856+0000 7fa3d6ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3d8133420 0x7fa3d809c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41538/0 (socket says 172.21.15.3:41538) 2023-10-03T10:19:06.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.856+0000 7fa3d6ffd700 1 -- 172.21.15.3:0/788908220 learned_addr learned my addr 172.21.15.3:0/788908220 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:06.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.857+0000 7fa3d67fc700 1 --2- 172.21.15.3:0/788908220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3d8133e30 0x7fa3d809cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:06.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.857+0000 7fa3d77fe700 1 --2- 172.21.15.3:0/788908220 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3d8134260 0x7fa3d809d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:06.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.857+0000 7fa3d6ffd700 1 -- 172.21.15.3:0/788908220 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3d8134260 msgr2=0x7fa3d809d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:06.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.857+0000 7fa3d6ffd700 1 --2- 172.21.15.3:0/788908220 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3d8134260 0x7fa3d809d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.857+0000 7fa3d6ffd700 1 -- 172.21.15.3:0/788908220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3d8133e30 msgr2=0x7fa3d809cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:06.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.857+0000 7fa3d6ffd700 1 --2- 172.21.15.3:0/788908220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3d8133e30 0x7fa3d809cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.857+0000 7fa3d6ffd700 1 -- 172.21.15.3:0/788908220 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d0009a70 con 0x7fa3d8133420 2023-10-03T10:19:06.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.857+0000 7fa3d6ffd700 1 --2- 172.21.15.3:0/788908220 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3d8133420 0x7fa3d809c6e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa3c800e9b0 tx=0x7fa3c800ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:06.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.857+0000 7fa3bf7fe700 1 -- 172.21.15.3:0/788908220 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c800c850 con 0x7fa3d8133420 2023-10-03T10:19:06.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.857+0000 7fa3bf7fe700 1 -- 172.21.15.3:0/788908220 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c800c9b0 con 0x7fa3d8133420 2023-10-03T10:19:06.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.857+0000 7fa3dd52b700 1 -- 172.21.15.3:0/788908220 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d8120bb0 con 0x7fa3d8133420 2023-10-03T10:19:06.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.857+0000 7fa3bf7fe700 1 -- 172.21.15.3:0/788908220 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c800cd20 con 0x7fa3d8133420 2023-10-03T10:19:06.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.858+0000 7fa3dd52b700 1 -- 172.21.15.3:0/788908220 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d81201c0 con 0x7fa3d8133420 2023-10-03T10:19:06.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.859+0000 7fa3bf7fe700 1 -- 172.21.15.3:0/788908220 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa3c8013470 con 0x7fa3d8133420 2023-10-03T10:19:06.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.859+0000 7fa3dd52b700 1 -- 172.21.15.3:0/788908220 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b80052e0 con 0x7fa3d8133420 2023-10-03T10:19:06.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.859+0000 7fa3bf7fe700 1 -- 172.21.15.3:0/788908220 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa3c803f7d0 con 0x7fa3d8133420 2023-10-03T10:19:06.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.868+0000 7fa3bf7fe700 1 -- 172.21.15.3:0/788908220 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa3c80232f0 con 0x7fa3d8133420 2023-10-03T10:19:06.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.969+0000 7fc176cfc700 1 -- 172.21.15.3:0/1161431749 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc170074ea0 con 0x7fc17009d6f0 2023-10-03T10:19:06.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.970+0000 7fc16d7fa700 1 -- 172.21.15.3:0/1161431749 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc164015020 con 0x7fc17009d6f0 2023-10-03T10:19:06.970 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:06.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.972+0000 7fc176cfc700 1 -- 172.21.15.3:0/1161431749 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc17009d6f0 msgr2=0x7fc1701a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:06.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.972+0000 7fc176cfc700 1 --2- 172.21.15.3:0/1161431749 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc17009d6f0 0x7fc1701a5130 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fc164011db0 tx=0x7fc164004750 comp rx=0 tx=0).stop 2023-10-03T10:19:06.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.972+0000 7fc176cfc700 1 -- 172.21.15.3:0/1161431749 shutdown_connections 2023-10-03T10:19:06.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.972+0000 7fc176cfc700 1 --2- 172.21.15.3:0/1161431749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc17009e410 0x7fc1701c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.972+0000 7fc176cfc700 1 --2- 172.21.15.3:0/1161431749 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc17009d6f0 0x7fc1701a5130 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.972+0000 7fc176cfc700 1 --2- 172.21.15.3:0/1161431749 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc17009e000 0x7fc1701a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.972+0000 7fc176cfc700 1 -- 172.21.15.3:0/1161431749 >> 172.21.15.3:0/1161431749 conn(0x7fc170066f50 msgr2=0x7fc1701bbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:06.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.972+0000 7fc176cfc700 1 -- 172.21.15.3:0/1161431749 shutdown_connections 2023-10-03T10:19:06.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.972+0000 7fc176cfc700 1 -- 172.21.15.3:0/1161431749 wait complete. 2023-10-03T10:19:06.987 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:06.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.994+0000 7f8f77b6b700 1 -- 172.21.15.3:0/2003891009 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8f7009ed40 con 0x7f8f70133540 2023-10-03T10:19:06.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.995+0000 7f8f667fc700 1 -- 172.21.15.3:0/2003891009 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8f60031090 con 0x7f8f70133540 2023-10-03T10:19:06.995 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:06.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.997+0000 7f8f77b6b700 1 -- 172.21.15.3:0/2003891009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f70133540 msgr2=0x7f8f701c4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:06.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.997+0000 7f8f77b6b700 1 --2- 172.21.15.3:0/2003891009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f70133540 0x7f8f701c4780 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f8f60018000 tx=0x7f8f60004750 comp rx=0 tx=0).stop 2023-10-03T10:19:06.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.997+0000 7f8f77b6b700 1 -- 172.21.15.3:0/2003891009 shutdown_connections 2023-10-03T10:19:06.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.997+0000 7f8f77b6b700 1 --2- 172.21.15.3:0/2003891009 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8f70135e60 0x7f8f7009ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.997+0000 7f8f77b6b700 1 --2- 172.21.15.3:0/2003891009 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f70130540 0x7f8f701c4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.997+0000 7f8f77b6b700 1 --2- 172.21.15.3:0/2003891009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f70133540 0x7f8f701c4780 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:06.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.997+0000 7f8f77b6b700 1 -- 172.21.15.3:0/2003891009 >> 172.21.15.3:0/2003891009 conn(0x7f8f700a2aa0 msgr2=0x7f8f70131300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:06.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.997+0000 7f8f77b6b700 1 -- 172.21.15.3:0/2003891009 shutdown_connections 2023-10-03T10:19:06.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:06.997+0000 7f8f77b6b700 1 -- 172.21.15.3:0/2003891009 wait complete. 2023-10-03T10:19:07.012 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:07.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:07.042+0000 7f98b9036700 1 -- 172.21.15.3:0/1398955119 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f98b406fdc0 con 0x7f98b409dc30 2023-10-03T10:19:07.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:07.042+0000 7f98ab7fe700 1 -- 172.21.15.3:0/1398955119 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f98b406fdc0 con 0x7f98b409dc30 2023-10-03T10:19:07.043 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:07.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:07.045+0000 7f98b9036700 1 -- 172.21.15.3:0/1398955119 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f98b409dc30 msgr2=0x7f98b41ccf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:07.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:07.045+0000 7f98b9036700 1 --2- 172.21.15.3:0/1398955119 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f98b409dc30 0x7f98b41ccf00 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f989c00c500 tx=0x7f989c00c810 comp rx=0 tx=0).stop 2023-10-03T10:19:07.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:07.045+0000 7f98b9036700 1 -- 172.21.15.3:0/1398955119 shutdown_connections 2023-10-03T10:19:07.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:07.045+0000 7f98b9036700 1 --2- 172.21.15.3:0/1398955119 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f98b413d1f0 0x7f98b41cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:07.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:07.045+0000 7f98b9036700 1 --2- 172.21.15.3:0/1398955119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98b406a770 0x7f98b41cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:07.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:07.045+0000 7f98b9036700 1 --2- 172.21.15.3:0/1398955119 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f98b409dc30 0x7f98b41ccf00 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:07.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:07.045+0000 7f98b9036700 1 -- 172.21.15.3:0/1398955119 >> 172.21.15.3:0/1398955119 conn(0x7f98b4066f50 msgr2=0x7f98b413d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:07.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:07.045+0000 7f98b9036700 1 -- 172.21.15.3:0/1398955119 shutdown_connections 2023-10-03T10:19:07.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:07.045+0000 7f98b9036700 1 -- 172.21.15.3:0/1398955119 wait complete. 2023-10-03T10:19:07.059 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:07.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:07.090+0000 7fa3dd52b700 1 -- 172.21.15.3:0/788908220 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa3b8005590 con 0x7fa3d8133420 2023-10-03T10:19:07.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:07.091+0000 7fa3bf7fe700 1 -- 172.21.15.3:0/788908220 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa3c8017030 con 0x7fa3d8133420 2023-10-03T10:19:07.091 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:07.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:07.093+0000 7fa3dd52b700 1 -- 172.21.15.3:0/788908220 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3d8133420 msgr2=0x7fa3d809c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:07.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:07.093+0000 7fa3dd52b700 1 --2- 172.21.15.3:0/788908220 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3d8133420 0x7fa3d809c6e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa3c800e9b0 tx=0x7fa3c800ecc0 comp rx=0 tx=0).stop 2023-10-03T10:19:07.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:07.093+0000 7fa3dd52b700 1 -- 172.21.15.3:0/788908220 shutdown_connections 2023-10-03T10:19:07.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:07.093+0000 7fa3dd52b700 1 --2- 172.21.15.3:0/788908220 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3d8134260 0x7fa3d809d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:07.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:07.093+0000 7fa3dd52b700 1 --2- 172.21.15.3:0/788908220 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3d8133420 0x7fa3d809c6e0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:07.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:07.093+0000 7fa3dd52b700 1 --2- 172.21.15.3:0/788908220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3d8133e30 0x7fa3d809cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:07.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:07.093+0000 7fa3dd52b700 1 -- 172.21.15.3:0/788908220 >> 172.21.15.3:0/788908220 conn(0x7fa3d80a4210 msgr2=0x7fa3d80a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:07.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:07.093+0000 7fa3dd52b700 1 -- 172.21.15.3:0/788908220 shutdown_connections 2023-10-03T10:19:07.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:07.093+0000 7fa3dd52b700 1 -- 172.21.15.3:0/788908220 wait complete. 2023-10-03T10:19:07.106 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:07.987 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:08.013 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:08.059 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:08.107 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:08.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.148+0000 7f4d03c78700 1 Processor -- start 2023-10-03T10:19:08.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.148+0000 7f4d03c78700 1 -- start start 2023-10-03T10:19:08.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.149+0000 7f4d03c78700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cfc130540 0x7f4cfc130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:08.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.149+0000 7f4d03c78700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4cfc130e50 0x7f4cfc06f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:08.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.149+0000 7f4d03c78700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cfc0716a0 con 0x7f4cfc06fb20 2023-10-03T10:19:08.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.149+0000 7f4d03c78700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cfc09a390 con 0x7f4cfc130e50 2023-10-03T10:19:08.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.149+0000 7f4d03c78700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc071290 con 0x7f4cfc130540 2023-10-03T10:19:08.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.149+0000 7f4d02215700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4cfc06fb20 0x7f4cfc137560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53644/0 (socket says 172.21.15.3:53644) 2023-10-03T10:19:08.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.149+0000 7f4d02215700 1 -- 172.21.15.3:0/2863084959 learned_addr learned my addr 172.21.15.3:0/2863084959 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:08.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.149+0000 7f4d01a14700 1 --2- 172.21.15.3:0/2863084959 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cfc130540 0x7f4cfc130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:08.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.150+0000 7f4d01213700 1 --2- 172.21.15.3:0/2863084959 >> v2:172.21.15.3:3300/0 conn(0x7f4cfc130e50 0x7f4cfc06f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:08.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.150+0000 7f4d01a14700 1 -- 172.21.15.3:0/2863084959 >> v2:172.21.15.3:3300/0 conn(0x7f4cfc130e50 msgr2=0x7f4cfc06f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:08.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.150+0000 7f4d01a14700 1 --2- 172.21.15.3:0/2863084959 >> v2:172.21.15.3:3300/0 conn(0x7f4cfc130e50 0x7f4cfc06f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.150+0000 7f4d01a14700 1 -- 172.21.15.3:0/2863084959 >> v1:172.21.15.3:6789/0 conn(0x7f4cfc06fb20 legacy=0x7f4cfc137560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:08.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.150+0000 7f4d00a12700 1 -- 172.21.15.3:0/2863084959 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2330738827 0 0) 0x7f4cfc0716a0 con 0x7f4cfc06fb20 2023-10-03T10:19:08.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.150+0000 7f4d01a14700 1 -- 172.21.15.3:0/2863084959 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cfc130340 con 0x7f4cfc130540 2023-10-03T10:19:08.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.150+0000 7f4d01a14700 1 --2- 172.21.15.3:0/2863084959 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cfc130540 0x7f4cfc130910 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009870 tx=0x7f4cf0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e2e6c4f7ad964af server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:08.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.150+0000 7f4d00a12700 1 -- 172.21.15.3:0/2863084959 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0011830 con 0x7f4cfc130540 2023-10-03T10:19:08.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.150+0000 7f4d00a12700 1 -- 172.21.15.3:0/2863084959 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf0011990 con 0x7f4cfc130540 2023-10-03T10:19:08.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.150+0000 7f4d00a12700 1 -- 172.21.15.3:0/2863084959 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0011d20 con 0x7f4cfc130540 2023-10-03T10:19:08.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.151+0000 7f4d03c78700 1 -- 172.21.15.3:0/2863084959 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cfc130540 msgr2=0x7f4cfc130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:08.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.151+0000 7f4d03c78700 1 --2- 172.21.15.3:0/2863084959 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cfc130540 0x7f4cfc130910 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009870 tx=0x7f4cf0009b80 comp rx=0 tx=0).stop 2023-10-03T10:19:08.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.152+0000 7f4d03c78700 1 -- 172.21.15.3:0/2863084959 shutdown_connections 2023-10-03T10:19:08.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.152+0000 7f4d03c78700 1 --2- 172.21.15.3:0/2863084959 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cfc130540 0x7f4cfc130910 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.152+0000 7f4d03c78700 1 --2- 172.21.15.3:0/2863084959 >> v2:172.21.15.3:3300/0 conn(0x7f4cfc130e50 0x7f4cfc06f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.152+0000 7f4d03c78700 1 -- 172.21.15.3:0/2863084959 >> 172.21.15.3:0/2863084959 conn(0x7f4cfc0a2aa0 msgr2=0x7f4cfc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:08.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.152+0000 7f4d03c78700 1 -- 172.21.15.3:0/2863084959 shutdown_connections 2023-10-03T10:19:08.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.152+0000 7f4d03c78700 1 -- 172.21.15.3:0/2863084959 wait complete. 2023-10-03T10:19:08.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.153+0000 7f4d03c78700 1 Processor -- start 2023-10-03T10:19:08.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.154+0000 7f4d03c78700 1 -- start start 2023-10-03T10:19:08.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.154+0000 7f4d03c78700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cfc06fb20 0x7f4cfc1cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:08.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.154+0000 7f4d03c78700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cfc130540 0x7f4cfc1cff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:08.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.154+0000 7f4d03c78700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cfc130e50 0x7f4cfc1d24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:08.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.154+0000 7f4d03c78700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc0718e0 con 0x7f4cfc130540 2023-10-03T10:19:08.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.154+0000 7f4d03c78700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc12bef0 con 0x7f4cfc06fb20 2023-10-03T10:19:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.154+0000 7f4d03c78700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc06f160 con 0x7f4cfc130e50 2023-10-03T10:19:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.154+0000 7f4d01a14700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cfc06fb20 0x7f4cfc1cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.154+0000 7f4d01a14700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cfc06fb20 0x7f4cfc1cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41556/0 (socket says 172.21.15.3:41556) 2023-10-03T10:19:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.154+0000 7f4d01a14700 1 -- 172.21.15.3:0/1347999640 learned_addr learned my addr 172.21.15.3:0/1347999640 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.154+0000 7f4d02215700 1 --2- 172.21.15.3:0/1347999640 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cfc130e50 0x7f4cfc1d24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.155+0000 7f4d01213700 1 --2- 172.21.15.3:0/1347999640 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cfc130540 0x7f4cfc1cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.155+0000 7f4d02215700 1 -- 172.21.15.3:0/1347999640 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cfc06fb20 msgr2=0x7f4cfc1cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.155+0000 7f4d02215700 1 --2- 172.21.15.3:0/1347999640 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cfc06fb20 0x7f4cfc1cda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.155+0000 7f4d02215700 1 -- 172.21.15.3:0/1347999640 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cfc130540 msgr2=0x7f4cfc1cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.155+0000 7f4d02215700 1 --2- 172.21.15.3:0/1347999640 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cfc130540 0x7f4cfc1cff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.155+0000 7f4d02215700 1 -- 172.21.15.3:0/1347999640 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf00094a0 con 0x7f4cfc130e50 2023-10-03T10:19:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.155+0000 7f4d01213700 1 --2- 172.21.15.3:0/1347999640 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cfc130540 0x7f4cfc1cff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.155+0000 7f4d02215700 1 --2- 172.21.15.3:0/1347999640 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cfc130e50 0x7f4cfc1d24a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f4cf400cbe0 tx=0x7f4cf400cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:08.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.155+0000 7f4cee7fc700 1 -- 172.21.15.3:0/1347999640 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf400aaf0 con 0x7f4cfc130e50 2023-10-03T10:19:08.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.155+0000 7f4cee7fc700 1 -- 172.21.15.3:0/1347999640 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf400ac70 con 0x7f4cfc130e50 2023-10-03T10:19:08.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.155+0000 7f4d03c78700 1 -- 172.21.15.3:0/1347999640 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cfc0a1960 con 0x7f4cfc130e50 2023-10-03T10:19:08.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.156+0000 7f4cee7fc700 1 -- 172.21.15.3:0/1347999640 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf40070d0 con 0x7f4cfc130e50 2023-10-03T10:19:08.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.156+0000 7f4d03c78700 1 -- 172.21.15.3:0/1347999640 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cfc121870 con 0x7f4cfc130e50 2023-10-03T10:19:08.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.157+0000 7f4cee7fc700 1 -- 172.21.15.3:0/1347999640 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4cf4007230 con 0x7f4cfc130e50 2023-10-03T10:19:08.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.157+0000 7f4cee7fc700 1 -- 172.21.15.3:0/1347999640 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4cf40434d0 con 0x7f4cfc130e50 2023-10-03T10:19:08.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.158+0000 7f4d03c78700 1 -- 172.21.15.3:0/1347999640 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cfc09dd40 con 0x7f4cfc130e50 2023-10-03T10:19:08.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.164+0000 7f4cee7fc700 1 -- 172.21.15.3:0/1347999640 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4cf401a8b0 con 0x7f4cfc130e50 2023-10-03T10:19:08.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.181+0000 7f741836b700 1 Processor -- start 2023-10-03T10:19:08.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.181+0000 7f741836b700 1 -- start start 2023-10-03T10:19:08.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.181+0000 7f741836b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f741009d6f0 0x7f741009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:08.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.181+0000 7f741836b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f741009e000 0x7f7410130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:08.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.181+0000 7f741836b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f741009f130 con 0x7f741009e410 2023-10-03T10:19:08.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.181+0000 7f741836b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f741009f520 con 0x7f741009d6f0 2023-10-03T10:19:08.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.181+0000 7f741836b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f741009e930 con 0x7f741009e000 2023-10-03T10:19:08.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.182+0000 7f7415906700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f741009e000 0x7f7410130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:08.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.182+0000 7f7415906700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f741009e000 0x7f7410130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41560/0 (socket says 172.21.15.3:41560) 2023-10-03T10:19:08.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.182+0000 7f7415906700 1 -- 172.21.15.3:0/1923903883 learned_addr learned my addr 172.21.15.3:0/1923903883 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:08.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.182+0000 7f7416107700 1 --2- 172.21.15.3:0/1923903883 >> v2:172.21.15.3:3300/0 conn(0x7f741009d6f0 0x7f741009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:08.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.182+0000 7f7415906700 1 -- 172.21.15.3:0/1923903883 >> v2:172.21.15.3:3300/0 conn(0x7f741009d6f0 msgr2=0x7f741009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:08.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.182+0000 7f7415906700 1 --2- 172.21.15.3:0/1923903883 >> v2:172.21.15.3:3300/0 conn(0x7f741009d6f0 0x7f741009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.182+0000 7f7415906700 1 -- 172.21.15.3:0/1923903883 >> v1:172.21.15.3:6789/0 conn(0x7f741009e410 legacy=0x7f7410130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:08.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.182+0000 7f7415906700 1 -- 172.21.15.3:0/1923903883 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f741009b2d0 con 0x7f741009e000 2023-10-03T10:19:08.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.182+0000 7f7415105700 1 -- 172.21.15.3:0/1923903883 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1063820083 0 0) 0x7f741009f130 con 0x7f741009e410 2023-10-03T10:19:08.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.182+0000 7f7415906700 1 --2- 172.21.15.3:0/1923903883 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f741009e000 0x7f7410130660 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f74000098f0 tx=0x7f7400009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c807cbd9cfabed1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:08.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.183+0000 7f7415105700 1 -- 172.21.15.3:0/1923903883 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740000b8a0 con 0x7f741009e000 2023-10-03T10:19:08.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.183+0000 7f7415105700 1 -- 172.21.15.3:0/1923903883 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740000ba00 con 0x7f741009e000 2023-10-03T10:19:08.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.183+0000 7f7415105700 1 -- 172.21.15.3:0/1923903883 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740000bd90 con 0x7f741009e000 2023-10-03T10:19:08.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.183+0000 7f741836b700 1 -- 172.21.15.3:0/1923903883 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f741009e000 msgr2=0x7f7410130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:08.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.183+0000 7f741836b700 1 --2- 172.21.15.3:0/1923903883 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f741009e000 0x7f7410130660 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f74000098f0 tx=0x7f7400009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:08.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.184+0000 7f741836b700 1 -- 172.21.15.3:0/1923903883 shutdown_connections 2023-10-03T10:19:08.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.184+0000 7f741836b700 1 --2- 172.21.15.3:0/1923903883 >> v2:172.21.15.3:3300/0 conn(0x7f741009d6f0 0x7f741009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.184+0000 7f741836b700 1 --2- 172.21.15.3:0/1923903883 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f741009e000 0x7f7410130660 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.184+0000 7f741836b700 1 -- 172.21.15.3:0/1923903883 >> 172.21.15.3:0/1923903883 conn(0x7f7410066f50 msgr2=0x7f741009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:08.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.184+0000 7f741836b700 1 -- 172.21.15.3:0/1923903883 shutdown_connections 2023-10-03T10:19:08.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.184+0000 7f741836b700 1 -- 172.21.15.3:0/1923903883 wait complete. 2023-10-03T10:19:08.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.185+0000 7f741836b700 1 Processor -- start 2023-10-03T10:19:08.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.185+0000 7f741836b700 1 -- start start 2023-10-03T10:19:08.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.186+0000 7f741836b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f741009d6f0 0x7f741011ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:08.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.186+0000 7f741836b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f741009e000 0x7f741011e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:08.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.186+0000 7f741836b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f741009e410 0x7f741011e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:08.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.186+0000 7f741836b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741011fa00 con 0x7f741009e000 2023-10-03T10:19:08.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.186+0000 7f741836b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7410120c50 con 0x7f741009e410 2023-10-03T10:19:08.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.186+0000 7f741836b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f741011ffe0 con 0x7f741009d6f0 2023-10-03T10:19:08.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.186+0000 7f7416107700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f741009d6f0 0x7f741011ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:08.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.186+0000 7f7416107700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f741009d6f0 0x7f741011ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43924/0 (socket says 172.21.15.3:43924) 2023-10-03T10:19:08.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.186+0000 7f7416107700 1 -- 172.21.15.3:0/4027132858 learned_addr learned my addr 172.21.15.3:0/4027132858 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:08.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.186+0000 7f7416908700 1 --2- 172.21.15.3:0/4027132858 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f741009e410 0x7f741011e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:08.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.186+0000 7f7415906700 1 --2- 172.21.15.3:0/4027132858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f741009e000 0x7f741011e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:08.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.187+0000 7f7416107700 1 -- 172.21.15.3:0/4027132858 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f741009e410 msgr2=0x7f741011e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:08.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.187+0000 7f7416107700 1 --2- 172.21.15.3:0/4027132858 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f741009e410 0x7f741011e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.187+0000 7f7416107700 1 -- 172.21.15.3:0/4027132858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f741009e000 msgr2=0x7f741011e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:08.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.187+0000 7f7416107700 1 --2- 172.21.15.3:0/4027132858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f741009e000 0x7f741011e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.187+0000 7f7416107700 1 -- 172.21.15.3:0/4027132858 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7400009550 con 0x7f741009d6f0 2023-10-03T10:19:08.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.187+0000 7f7416107700 1 --2- 172.21.15.3:0/4027132858 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f741009d6f0 0x7f741011ddf0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f740c009f70 tx=0x7f740c00eaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:08.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.187+0000 7f7406ffd700 1 -- 172.21.15.3:0/4027132858 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c00c070 con 0x7f741009d6f0 2023-10-03T10:19:08.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.187+0000 7f7406ffd700 1 -- 172.21.15.3:0/4027132858 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740c013b20 con 0x7f741009d6f0 2023-10-03T10:19:08.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.187+0000 7f741836b700 1 -- 172.21.15.3:0/4027132858 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7410121630 con 0x7f741009d6f0 2023-10-03T10:19:08.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.187+0000 7f7406ffd700 1 -- 172.21.15.3:0/4027132858 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c01b3f0 con 0x7f741009d6f0 2023-10-03T10:19:08.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.187+0000 7f741836b700 1 -- 172.21.15.3:0/4027132858 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f741009c700 con 0x7f741009d6f0 2023-10-03T10:19:08.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.188+0000 7f7406ffd700 1 -- 172.21.15.3:0/4027132858 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f740c01b550 con 0x7f741009d6f0 2023-10-03T10:19:08.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.188+0000 7f741836b700 1 -- 172.21.15.3:0/4027132858 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f741011ac20 con 0x7f741009d6f0 2023-10-03T10:19:08.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.189+0000 7f7406ffd700 1 -- 172.21.15.3:0/4027132858 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f740c0461c0 con 0x7f741009d6f0 2023-10-03T10:19:08.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.198+0000 7f7406ffd700 1 -- 172.21.15.3:0/4027132858 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f740c048060 con 0x7f741009d6f0 2023-10-03T10:19:08.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.223+0000 7f646642c700 1 Processor -- start 2023-10-03T10:19:08.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.223+0000 7f646642c700 1 -- start start 2023-10-03T10:19:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.223+0000 7f646642c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f646006a770 0x7f646006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.223+0000 7f646642c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f646006b080 0x7f6460131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.224+0000 7f646642c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f646009ece0 con 0x7f6460131b40 2023-10-03T10:19:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.224+0000 7f646642c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f646009e760 con 0x7f646006a770 2023-10-03T10:19:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.224+0000 7f646642c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646009e040 con 0x7f646006b080 2023-10-03T10:19:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.224+0000 7f645f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f646006b080 0x7f6460131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.224+0000 7f645f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f646006b080 0x7f6460131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41580/0 (socket says 172.21.15.3:41580) 2023-10-03T10:19:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.224+0000 7f645f7fe700 1 -- 172.21.15.3:0/2721858406 learned_addr learned my addr 172.21.15.3:0/2721858406 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:08.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.224+0000 7f645ffff700 1 --2- 172.21.15.3:0/2721858406 >> v2:172.21.15.3:3300/0 conn(0x7f646006a770 0x7f646006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.224+0000 7f645f7fe700 1 -- 172.21.15.3:0/2721858406 >> v2:172.21.15.3:3300/0 conn(0x7f646006a770 msgr2=0x7f646006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.224+0000 7f645f7fe700 1 --2- 172.21.15.3:0/2721858406 >> v2:172.21.15.3:3300/0 conn(0x7f646006a770 0x7f646006ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.224+0000 7f645f7fe700 1 -- 172.21.15.3:0/2721858406 >> v1:172.21.15.3:6789/0 conn(0x7f6460131b40 legacy=0x7f6460138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.224+0000 7f645f7fe700 1 -- 172.21.15.3:0/2721858406 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646009f630 con 0x7f646006b080 2023-10-03T10:19:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.224+0000 7f645f7fe700 1 --2- 172.21.15.3:0/2721858406 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f646006b080 0x7f6460131600 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f64540098f0 tx=0x7f6454009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1d9d1cfff576d75 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.225+0000 7f645effd700 1 -- 172.21.15.3:0/2721858406 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6454004230 con 0x7f646006b080 2023-10-03T10:19:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.225+0000 7f645effd700 1 -- 172.21.15.3:0/2721858406 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6454004390 con 0x7f646006b080 2023-10-03T10:19:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.225+0000 7f645effd700 1 -- 172.21.15.3:0/2721858406 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64540036a0 con 0x7f646006b080 2023-10-03T10:19:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.226+0000 7f646642c700 1 -- 172.21.15.3:0/2721858406 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f646006b080 msgr2=0x7f6460131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.226+0000 7f646642c700 1 --2- 172.21.15.3:0/2721858406 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f646006b080 0x7f6460131600 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f64540098f0 tx=0x7f6454009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:08.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.226+0000 7f646642c700 1 -- 172.21.15.3:0/2721858406 shutdown_connections 2023-10-03T10:19:08.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.226+0000 7f646642c700 1 --2- 172.21.15.3:0/2721858406 >> v2:172.21.15.3:3300/0 conn(0x7f646006a770 0x7f646006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.226+0000 7f646642c700 1 --2- 172.21.15.3:0/2721858406 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f646006b080 0x7f6460131600 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.226+0000 7f646642c700 1 -- 172.21.15.3:0/2721858406 >> 172.21.15.3:0/2721858406 conn(0x7f6460066f50 msgr2=0x7f646009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.226+0000 7f646642c700 1 -- 172.21.15.3:0/2721858406 shutdown_connections 2023-10-03T10:19:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.226+0000 7f646642c700 1 -- 172.21.15.3:0/2721858406 wait complete. 2023-10-03T10:19:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.227+0000 7f646642c700 1 Processor -- start 2023-10-03T10:19:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.227+0000 7f646642c700 1 -- start start 2023-10-03T10:19:08.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.228+0000 7f646642c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f646006a770 0x7f646009add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:08.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.228+0000 7f646642c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f646006b080 0x7f646009b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:08.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.228+0000 7f646642c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6460131b40 0x7f64601c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:08.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.228+0000 7f646642c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646009b850 con 0x7f646006a770 2023-10-03T10:19:08.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.228+0000 7f646642c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6460121c50 con 0x7f646006b080 2023-10-03T10:19:08.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.228+0000 7f646642c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6460121f10 con 0x7f6460131b40 2023-10-03T10:19:08.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.228+0000 7f645ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f646006a770 0x7f646009add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:08.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.228+0000 7f645ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f646006a770 0x7f646009add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46760/0 (socket says 172.21.15.3:46760) 2023-10-03T10:19:08.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.228+0000 7f645ffff700 1 -- 172.21.15.3:0/1123630065 learned_addr learned my addr 172.21.15.3:0/1123630065 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:08.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.228+0000 7f645f7fe700 1 --2- 172.21.15.3:0/1123630065 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f646006b080 0x7f646009b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:08.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.228+0000 7f64649c9700 1 --2- 172.21.15.3:0/1123630065 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6460131b40 0x7f64601c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:08.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.229+0000 7f645f7fe700 1 -- 172.21.15.3:0/1123630065 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6460131b40 msgr2=0x7f64601c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:08.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.229+0000 7f645f7fe700 1 --2- 172.21.15.3:0/1123630065 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6460131b40 0x7f64601c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.229+0000 7f645f7fe700 1 -- 172.21.15.3:0/1123630065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f646006a770 msgr2=0x7f646009add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:08.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.229+0000 7f645f7fe700 1 --2- 172.21.15.3:0/1123630065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f646006a770 0x7f646009add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.229+0000 7f645f7fe700 1 -- 172.21.15.3:0/1123630065 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6454009550 con 0x7f646006b080 2023-10-03T10:19:08.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.229+0000 7f645ffff700 1 --2- 172.21.15.3:0/1123630065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f646006a770 0x7f646009add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:08.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.229+0000 7f645f7fe700 1 --2- 172.21.15.3:0/1123630065 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f646006b080 0x7f646009b310 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f6454015000 tx=0x7f6454004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:08.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.229+0000 7f645cff9700 1 -- 172.21.15.3:0/1123630065 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6454003ba0 con 0x7f646006b080 2023-10-03T10:19:08.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.229+0000 7f645cff9700 1 -- 172.21.15.3:0/1123630065 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645401f070 con 0x7f646006b080 2023-10-03T10:19:08.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.229+0000 7f645cff9700 1 -- 172.21.15.3:0/1123630065 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6454011630 con 0x7f646006b080 2023-10-03T10:19:08.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.230+0000 7f646642c700 1 -- 172.21.15.3:0/1123630065 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f646009c010 con 0x7f646006b080 2023-10-03T10:19:08.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.230+0000 7f646642c700 1 -- 172.21.15.3:0/1123630065 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64601221f0 con 0x7f646006b080 2023-10-03T10:19:08.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.231+0000 7f645cff9700 1 -- 172.21.15.3:0/1123630065 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6454021030 con 0x7f646006b080 2023-10-03T10:19:08.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.231+0000 7f646642c700 1 -- 172.21.15.3:0/1123630065 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6460131280 con 0x7f646006b080 2023-10-03T10:19:08.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.231+0000 7f645cff9700 1 -- 172.21.15.3:0/1123630065 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6454040ca0 con 0x7f646006b080 2023-10-03T10:19:08.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.241+0000 7f645cff9700 1 -- 172.21.15.3:0/1123630065 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f64540239c0 con 0x7f646006b080 2023-10-03T10:19:08.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.273+0000 7f5e20ed2700 1 Processor -- start 2023-10-03T10:19:08.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.274+0000 7f5e20ed2700 1 -- start start 2023-10-03T10:19:08.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.275+0000 7f5e20ed2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e1c09d6f0 0x7f5e1c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:08.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.275+0000 7f5e20ed2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5e1c09e000 0x7f5e1c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:08.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.275+0000 7f5e20ed2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e1c09b120 con 0x7f5e1c09e410 2023-10-03T10:19:08.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.275+0000 7f5e20ed2700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e1c06b460 con 0x7f5e1c09e000 2023-10-03T10:19:08.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.275+0000 7f5e20ed2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1c09a7e0 con 0x7f5e1c09d6f0 2023-10-03T10:19:08.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.275+0000 7f5e19d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5e1c09e000 0x7f5e1c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:08.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.275+0000 7f5e1a59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e1c09d6f0 0x7f5e1c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:08.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.275+0000 7f5e19d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5e1c09e000 0x7f5e1c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46766/0 (socket says 172.21.15.3:46766) 2023-10-03T10:19:08.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.275+0000 7f5e1a59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e1c09d6f0 0x7f5e1c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41602/0 (socket says 172.21.15.3:41602) 2023-10-03T10:19:08.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.275+0000 7f5e19d9b700 1 -- 172.21.15.3:0/644221296 learned_addr learned my addr 172.21.15.3:0/644221296 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:08.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.275+0000 7f5e19d9b700 1 -- 172.21.15.3:0/644221296 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e1c09d6f0 msgr2=0x7f5e1c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:08.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.275+0000 7f5e19d9b700 1 --2- 172.21.15.3:0/644221296 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e1c09d6f0 0x7f5e1c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.275+0000 7f5e19d9b700 1 -- 172.21.15.3:0/644221296 >> v1:172.21.15.3:6789/0 conn(0x7f5e1c09e410 legacy=0x7f5e1c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:08.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.275+0000 7f5e19d9b700 1 -- 172.21.15.3:0/644221296 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1c09a500 con 0x7f5e1c09e000 2023-10-03T10:19:08.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.276+0000 7f5e19d9b700 1 --2- 172.21.15.3:0/644221296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e1c09e000 0x7f5e1c1339f0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f5e100098f0 tx=0x7f5e10009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a681497549f8d2cd server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:08.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.276+0000 7f5e1959a700 1 -- 172.21.15.3:0/644221296 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10011830 con 0x7f5e1c09e000 2023-10-03T10:19:08.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.276+0000 7f5e1959a700 1 -- 172.21.15.3:0/644221296 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e10011990 con 0x7f5e1c09e000 2023-10-03T10:19:08.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.276+0000 7f5e1959a700 1 -- 172.21.15.3:0/644221296 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10011d20 con 0x7f5e1c09e000 2023-10-03T10:19:08.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.276+0000 7f5e20ed2700 1 -- 172.21.15.3:0/644221296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e1c09e000 msgr2=0x7f5e1c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:08.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.276+0000 7f5e20ed2700 1 --2- 172.21.15.3:0/644221296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e1c09e000 0x7f5e1c1339f0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f5e100098f0 tx=0x7f5e10009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:08.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.277+0000 7f5e20ed2700 1 -- 172.21.15.3:0/644221296 shutdown_connections 2023-10-03T10:19:08.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.277+0000 7f5e20ed2700 1 --2- 172.21.15.3:0/644221296 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e1c09d6f0 0x7f5e1c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.277+0000 7f5e20ed2700 1 --2- 172.21.15.3:0/644221296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e1c09e000 0x7f5e1c1339f0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.277+0000 7f5e20ed2700 1 -- 172.21.15.3:0/644221296 >> 172.21.15.3:0/644221296 conn(0x7f5e1c066f50 msgr2=0x7f5e1c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:08.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.277+0000 7f5e20ed2700 1 -- 172.21.15.3:0/644221296 shutdown_connections 2023-10-03T10:19:08.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.277+0000 7f5e20ed2700 1 -- 172.21.15.3:0/644221296 wait complete. 2023-10-03T10:19:08.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.278+0000 7f5e20ed2700 1 Processor -- start 2023-10-03T10:19:08.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.279+0000 7f5e20ed2700 1 -- start start 2023-10-03T10:19:08.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.279+0000 7f5e20ed2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e1c09d6f0 0x7f5e1c1a5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:08.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.279+0000 7f5e20ed2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e1c09e000 0x7f5e1c1a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:08.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.279+0000 7f5e20ed2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e1c09e410 0x7f5e1c1a6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:08.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.279+0000 7f5e20ed2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1c1a7b30 con 0x7f5e1c09e000 2023-10-03T10:19:08.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.279+0000 7f5e20ed2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1c1a8150 con 0x7f5e1c09e410 2023-10-03T10:19:08.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.279+0000 7f5e20ed2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1c1a8d80 con 0x7f5e1c09d6f0 2023-10-03T10:19:08.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.279+0000 7f5e1a59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e1c09d6f0 0x7f5e1c1a5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:08.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.280+0000 7f5e1a59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e1c09d6f0 0x7f5e1c1a5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43938/0 (socket says 172.21.15.3:43938) 2023-10-03T10:19:08.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.280+0000 7f5e1a59c700 1 -- 172.21.15.3:0/276024613 learned_addr learned my addr 172.21.15.3:0/276024613 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:08.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.280+0000 7f5e1ad9d700 1 --2- 172.21.15.3:0/276024613 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e1c09e410 0x7f5e1c1a6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:08.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.280+0000 7f5e19d9b700 1 --2- 172.21.15.3:0/276024613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e1c09e000 0x7f5e1c1a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:08.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.280+0000 7f5e1a59c700 1 -- 172.21.15.3:0/276024613 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e1c09e410 msgr2=0x7f5e1c1a6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:08.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.280+0000 7f5e1a59c700 1 --2- 172.21.15.3:0/276024613 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e1c09e410 0x7f5e1c1a6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.280+0000 7f5e1a59c700 1 -- 172.21.15.3:0/276024613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e1c09e000 msgr2=0x7f5e1c1a64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:08.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.280+0000 7f5e1a59c700 1 --2- 172.21.15.3:0/276024613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e1c09e000 0x7f5e1c1a64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.280+0000 7f5e1a59c700 1 -- 172.21.15.3:0/276024613 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e10009550 con 0x7f5e1c09d6f0 2023-10-03T10:19:08.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.280+0000 7f5e19d9b700 1 --2- 172.21.15.3:0/276024613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e1c09e000 0x7f5e1c1a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:08.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.280+0000 7f5e1a59c700 1 --2- 172.21.15.3:0/276024613 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e1c09d6f0 0x7f5e1c1a5fb0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c00e9b0 tx=0x7f5e0c00ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:08.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.281+0000 7f5e0affd700 1 -- 172.21.15.3:0/276024613 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c009710 con 0x7f5e1c09d6f0 2023-10-03T10:19:08.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.281+0000 7f5e0affd700 1 -- 172.21.15.3:0/276024613 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e0c00ee50 con 0x7f5e1c09d6f0 2023-10-03T10:19:08.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.281+0000 7f5e20ed2700 1 -- 172.21.15.3:0/276024613 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e1c1a91d0 con 0x7f5e1c09d6f0 2023-10-03T10:19:08.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.281+0000 7f5e0affd700 1 -- 172.21.15.3:0/276024613 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c00ca50 con 0x7f5e1c09d6f0 2023-10-03T10:19:08.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.281+0000 7f5e20ed2700 1 -- 172.21.15.3:0/276024613 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e1c1a7400 con 0x7f5e1c09d6f0 2023-10-03T10:19:08.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.282+0000 7f5e20ed2700 1 -- 172.21.15.3:0/276024613 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e1c074ea0 con 0x7f5e1c09d6f0 2023-10-03T10:19:08.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.283+0000 7f5e0affd700 1 -- 172.21.15.3:0/276024613 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5e0c00cbb0 con 0x7f5e1c09d6f0 2023-10-03T10:19:08.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.283+0000 7f5e0affd700 1 -- 172.21.15.3:0/276024613 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5e0c03ff00 con 0x7f5e1c09d6f0 2023-10-03T10:19:08.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.288+0000 7f5e0affd700 1 -- 172.21.15.3:0/276024613 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5e0c011350 con 0x7f5e1c09d6f0 2023-10-03T10:19:08.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.388+0000 7f4d03c78700 1 -- 172.21.15.3:0/1347999640 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4cfc12cc10 con 0x7f4cfc130e50 2023-10-03T10:19:08.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.388+0000 7f4cee7fc700 1 -- 172.21.15.3:0/1347999640 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4cf4018020 con 0x7f4cfc130e50 2023-10-03T10:19:08.389 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:08.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.390+0000 7f4d03c78700 1 -- 172.21.15.3:0/1347999640 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cfc130e50 msgr2=0x7f4cfc1d24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:08.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.390+0000 7f4d03c78700 1 --2- 172.21.15.3:0/1347999640 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cfc130e50 0x7f4cfc1d24a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f4cf400cbe0 tx=0x7f4cf400cef0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.391+0000 7f4d03c78700 1 -- 172.21.15.3:0/1347999640 shutdown_connections 2023-10-03T10:19:08.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.391+0000 7f4d03c78700 1 --2- 172.21.15.3:0/1347999640 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4cfc130e50 0x7f4cfc1d24a0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.391+0000 7f4d03c78700 1 --2- 172.21.15.3:0/1347999640 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4cfc06fb20 0x7f4cfc1cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.391+0000 7f4d03c78700 1 --2- 172.21.15.3:0/1347999640 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cfc130540 0x7f4cfc1cff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.391+0000 7f4d03c78700 1 -- 172.21.15.3:0/1347999640 >> 172.21.15.3:0/1347999640 conn(0x7f4cfc0a2aa0 msgr2=0x7f4cfc12d670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:08.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.391+0000 7f4d03c78700 1 -- 172.21.15.3:0/1347999640 shutdown_connections 2023-10-03T10:19:08.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.391+0000 7f4d03c78700 1 -- 172.21.15.3:0/1347999640 wait complete. 2023-10-03T10:19:08.405 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:08.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.411+0000 7f741836b700 1 -- 172.21.15.3:0/4027132858 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7410074ea0 con 0x7f741009d6f0 2023-10-03T10:19:08.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.411+0000 7f7406ffd700 1 -- 172.21.15.3:0/4027132858 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f740c048090 con 0x7f741009d6f0 2023-10-03T10:19:08.411 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:08.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.414+0000 7f741836b700 1 -- 172.21.15.3:0/4027132858 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f741009d6f0 msgr2=0x7f741011ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:08.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.414+0000 7f741836b700 1 --2- 172.21.15.3:0/4027132858 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f741009d6f0 0x7f741011ddf0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f740c009f70 tx=0x7f740c00eaf0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.414+0000 7f741836b700 1 -- 172.21.15.3:0/4027132858 shutdown_connections 2023-10-03T10:19:08.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.414+0000 7f741836b700 1 --2- 172.21.15.3:0/4027132858 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f741009e410 0x7f741011e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.414+0000 7f741836b700 1 --2- 172.21.15.3:0/4027132858 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f741009d6f0 0x7f741011ddf0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.414+0000 7f741836b700 1 --2- 172.21.15.3:0/4027132858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f741009e000 0x7f741011e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.414+0000 7f741836b700 1 -- 172.21.15.3:0/4027132858 >> 172.21.15.3:0/4027132858 conn(0x7f7410066f50 msgr2=0x7f7410130a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:08.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.414+0000 7f741836b700 1 -- 172.21.15.3:0/4027132858 shutdown_connections 2023-10-03T10:19:08.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.414+0000 7f741836b700 1 -- 172.21.15.3:0/4027132858 wait complete. 2023-10-03T10:19:08.428 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:08.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.453+0000 7f646642c700 1 -- 172.21.15.3:0/1123630065 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6460131280 con 0x7f646006b080 2023-10-03T10:19:08.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.453+0000 7f645cff9700 1 -- 172.21.15.3:0/1123630065 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6454024430 con 0x7f646006b080 2023-10-03T10:19:08.454 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:08.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.456+0000 7f646642c700 1 -- 172.21.15.3:0/1123630065 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f646006b080 msgr2=0x7f646009b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:08.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.456+0000 7f646642c700 1 --2- 172.21.15.3:0/1123630065 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f646006b080 0x7f646009b310 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f6454015000 tx=0x7f6454004750 comp rx=0 tx=0).stop 2023-10-03T10:19:08.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.456+0000 7f646642c700 1 -- 172.21.15.3:0/1123630065 shutdown_connections 2023-10-03T10:19:08.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.456+0000 7f646642c700 1 --2- 172.21.15.3:0/1123630065 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6460131b40 0x7f64601c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.456+0000 7f646642c700 1 --2- 172.21.15.3:0/1123630065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f646006a770 0x7f646009add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.456+0000 7f646642c700 1 --2- 172.21.15.3:0/1123630065 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f646006b080 0x7f646009b310 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.456+0000 7f646642c700 1 -- 172.21.15.3:0/1123630065 >> 172.21.15.3:0/1123630065 conn(0x7f6460066f50 msgr2=0x7f64600a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:08.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.456+0000 7f646642c700 1 -- 172.21.15.3:0/1123630065 shutdown_connections 2023-10-03T10:19:08.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.456+0000 7f646642c700 1 -- 172.21.15.3:0/1123630065 wait complete. 2023-10-03T10:19:08.470 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:08.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.494+0000 7f5e20ed2700 1 -- 172.21.15.3:0/276024613 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5e1c1a9530 con 0x7f5e1c09d6f0 2023-10-03T10:19:08.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.495+0000 7f5e0affd700 1 -- 172.21.15.3:0/276024613 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5e0c024b00 con 0x7f5e1c09d6f0 2023-10-03T10:19:08.495 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:08.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.497+0000 7f5e20ed2700 1 -- 172.21.15.3:0/276024613 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e1c09d6f0 msgr2=0x7f5e1c1a5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:08.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.497+0000 7f5e20ed2700 1 --2- 172.21.15.3:0/276024613 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e1c09d6f0 0x7f5e1c1a5fb0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c00e9b0 tx=0x7f5e0c00ecc0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.497+0000 7f5e20ed2700 1 -- 172.21.15.3:0/276024613 shutdown_connections 2023-10-03T10:19:08.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.497+0000 7f5e20ed2700 1 --2- 172.21.15.3:0/276024613 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5e1c09e410 0x7f5e1c1a6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.497+0000 7f5e20ed2700 1 --2- 172.21.15.3:0/276024613 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5e1c09d6f0 0x7f5e1c1a5fb0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.497+0000 7f5e20ed2700 1 --2- 172.21.15.3:0/276024613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e1c09e000 0x7f5e1c1a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:08.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.497+0000 7f5e20ed2700 1 -- 172.21.15.3:0/276024613 >> 172.21.15.3:0/276024613 conn(0x7f5e1c066f50 msgr2=0x7f5e1c1baed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:08.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.497+0000 7f5e20ed2700 1 -- 172.21.15.3:0/276024613 shutdown_connections 2023-10-03T10:19:08.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:08.497+0000 7f5e20ed2700 1 -- 172.21.15.3:0/276024613 wait complete. 2023-10-03T10:19:08.511 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:09.406 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:09.428 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:09.471 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:09.512 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:09.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.557+0000 7f814e181700 1 Processor -- start 2023-10-03T10:19:09.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.557+0000 7f814e181700 1 -- start start 2023-10-03T10:19:09.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.558+0000 7f814e181700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f81481316d0 0x7f8148131aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:09.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.558+0000 7f814e181700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8148131fe0 0x7f814806e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:09.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.558+0000 7f814e181700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f814806ea90 0x7f814806eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:09.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.558+0000 7f814e181700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f814809a5f0 con 0x7f81481316d0 2023-10-03T10:19:09.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.558+0000 7f814e181700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8148131510 con 0x7f8148131fe0 2023-10-03T10:19:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.558+0000 7f814e181700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8148099e70 con 0x7f814806ea90 2023-10-03T10:19:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.558+0000 7f8146ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8148131fe0 0x7f814806e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.558+0000 7f8147fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f814806ea90 0x7f814806eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.558+0000 7f8146ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8148131fe0 0x7f814806e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41626/0 (socket says 172.21.15.3:41626) 2023-10-03T10:19:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.558+0000 7f8146ffd700 1 -- 172.21.15.3:0/2013937903 learned_addr learned my addr 172.21.15.3:0/2013937903 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.559+0000 7f81477fe700 1 --2- 172.21.15.3:0/2013937903 >> v2:172.21.15.3:3300/0 conn(0x7f81481316d0 0x7f8148131aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.559+0000 7f8147fff700 1 -- 172.21.15.3:0/2013937903 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8148131fe0 msgr2=0x7f814806e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.559+0000 7f8147fff700 1 --2- 172.21.15.3:0/2013937903 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8148131fe0 0x7f814806e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.559+0000 7f8147fff700 1 -- 172.21.15.3:0/2013937903 >> v2:172.21.15.3:3300/0 conn(0x7f81481316d0 msgr2=0x7f8148131aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.559+0000 7f8147fff700 1 --2- 172.21.15.3:0/2013937903 >> v2:172.21.15.3:3300/0 conn(0x7f81481316d0 0x7f8148131aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.559+0000 7f8147fff700 1 -- 172.21.15.3:0/2013937903 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8148066680 con 0x7f814806ea90 2023-10-03T10:19:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.559+0000 7f81477fe700 1 --2- 172.21.15.3:0/2013937903 >> v2:172.21.15.3:3300/0 conn(0x7f81481316d0 0x7f8148131aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:19:09.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.559+0000 7f8147fff700 1 --2- 172.21.15.3:0/2013937903 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f814806ea90 0x7f814806eee0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8140009d90 tx=0x7f814000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47c11a668e556554 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:09.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.559+0000 7f81467fc700 1 -- 172.21.15.3:0/2013937903 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814000e070 con 0x7f814806ea90 2023-10-03T10:19:09.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.559+0000 7f81467fc700 1 -- 172.21.15.3:0/2013937903 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814000bb20 con 0x7f814806ea90 2023-10-03T10:19:09.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.559+0000 7f81467fc700 1 -- 172.21.15.3:0/2013937903 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8140012430 con 0x7f814806ea90 2023-10-03T10:19:09.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.560+0000 7f814e181700 1 -- 172.21.15.3:0/2013937903 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f814806ea90 msgr2=0x7f814806eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:09.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.560+0000 7f814e181700 1 --2- 172.21.15.3:0/2013937903 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f814806ea90 0x7f814806eee0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8140009d90 tx=0x7f814000b330 comp rx=0 tx=0).stop 2023-10-03T10:19:09.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.560+0000 7f814e181700 1 -- 172.21.15.3:0/2013937903 shutdown_connections 2023-10-03T10:19:09.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.560+0000 7f814e181700 1 --2- 172.21.15.3:0/2013937903 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f814806ea90 0x7f814806eee0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.560+0000 7f814e181700 1 --2- 172.21.15.3:0/2013937903 >> v2:172.21.15.3:3300/0 conn(0x7f81481316d0 0x7f8148131aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.560+0000 7f814e181700 1 --2- 172.21.15.3:0/2013937903 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8148131fe0 0x7f814806e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.560+0000 7f814e181700 1 -- 172.21.15.3:0/2013937903 >> 172.21.15.3:0/2013937903 conn(0x7f81480a2aa0 msgr2=0x7f81480a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:09.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.560+0000 7f814e181700 1 -- 172.21.15.3:0/2013937903 shutdown_connections 2023-10-03T10:19:09.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.560+0000 7f814e181700 1 -- 172.21.15.3:0/2013937903 wait complete. 2023-10-03T10:19:09.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.562+0000 7f814e181700 1 Processor -- start 2023-10-03T10:19:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.562+0000 7f814e181700 1 -- start start 2023-10-03T10:19:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.563+0000 7f814e181700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f814806ea90 0x7f8148132da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.563+0000 7f814e181700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81481316d0 0x7f81481332e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.563+0000 7f814e181700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8148131fe0 0x7f8148133820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.563+0000 7f814e181700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8148136370 con 0x7f81481316d0 2023-10-03T10:19:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.563+0000 7f814e181700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814812e650 con 0x7f814806ea90 2023-10-03T10:19:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.563+0000 7f814e181700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f814809a0b0 con 0x7f8148131fe0 2023-10-03T10:19:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.563+0000 7f8147fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8148131fe0 0x7f8148133820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.563+0000 7f81477fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f814806ea90 0x7f8148132da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.563+0000 7f8147fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8148131fe0 0x7f8148133820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43950/0 (socket says 172.21.15.3:43950) 2023-10-03T10:19:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.563+0000 7f8147fff700 1 -- 172.21.15.3:0/1512609749 learned_addr learned my addr 172.21.15.3:0/1512609749 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.563+0000 7f8146ffd700 1 --2- 172.21.15.3:0/1512609749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81481316d0 0x7f81481332e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:09.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.563+0000 7f8147fff700 1 -- 172.21.15.3:0/1512609749 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f814806ea90 msgr2=0x7f8148132da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:09.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.563+0000 7f8147fff700 1 --2- 172.21.15.3:0/1512609749 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f814806ea90 0x7f8148132da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.563+0000 7f8147fff700 1 -- 172.21.15.3:0/1512609749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81481316d0 msgr2=0x7f81481332e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:09.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.563+0000 7f8147fff700 1 --2- 172.21.15.3:0/1512609749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81481316d0 0x7f81481332e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.563+0000 7f8147fff700 1 -- 172.21.15.3:0/1512609749 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8140009a70 con 0x7f8148131fe0 2023-10-03T10:19:09.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.564+0000 7f8147fff700 1 --2- 172.21.15.3:0/1512609749 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8148131fe0 0x7f8148133820 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f8140014040 tx=0x7f8140004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:09.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.564+0000 7f812ffff700 1 -- 172.21.15.3:0/1512609749 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8140012430 con 0x7f8148131fe0 2023-10-03T10:19:09.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.564+0000 7f812ffff700 1 -- 172.21.15.3:0/1512609749 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8140020070 con 0x7f8148131fe0 2023-10-03T10:19:09.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.564+0000 7f814e181700 1 -- 172.21.15.3:0/1512609749 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8148134ea0 con 0x7f8148131fe0 2023-10-03T10:19:09.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.564+0000 7f812ffff700 1 -- 172.21.15.3:0/1512609749 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8140003ea0 con 0x7f8148131fe0 2023-10-03T10:19:09.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.564+0000 7f814e181700 1 -- 172.21.15.3:0/1512609749 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8148136550 con 0x7f8148131fe0 2023-10-03T10:19:09.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.565+0000 7f812ffff700 1 -- 172.21.15.3:0/1512609749 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f814002ac80 con 0x7f8148131fe0 2023-10-03T10:19:09.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.565+0000 7f812ffff700 1 -- 172.21.15.3:0/1512609749 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8140022070 con 0x7f8148131fe0 2023-10-03T10:19:09.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.565+0000 7f814e181700 1 -- 172.21.15.3:0/1512609749 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81481361a0 con 0x7f8148131fe0 2023-10-03T10:19:09.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.571+0000 7f812ffff700 1 -- 172.21.15.3:0/1512609749 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8140004040 con 0x7f8148131fe0 2023-10-03T10:19:09.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.602+0000 7fb59e529700 1 Processor -- start 2023-10-03T10:19:09.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.603+0000 7fb59e529700 1 -- start start 2023-10-03T10:19:09.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.603+0000 7fb59e529700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb59809d6f0 0x7fb59809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:09.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.603+0000 7fb59e529700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb59809e000 0x7fb5981339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:09.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.603+0000 7fb59e529700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb59809b120 con 0x7fb59809e410 2023-10-03T10:19:09.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.603+0000 7fb59e529700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb59806b460 con 0x7fb59809d6f0 2023-10-03T10:19:09.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.603+0000 7fb59e529700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb59809a7e0 con 0x7fb59809e000 2023-10-03T10:19:09.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.604+0000 7fb5977fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb59809e000 0x7fb5981339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:09.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.604+0000 7fb5977fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb59809e000 0x7fb5981339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43964/0 (socket says 172.21.15.3:43964) 2023-10-03T10:19:09.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.604+0000 7fb5977fe700 1 -- 172.21.15.3:0/2713442710 learned_addr learned my addr 172.21.15.3:0/2713442710 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:09.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.604+0000 7fb597fff700 1 --2- 172.21.15.3:0/2713442710 >> v2:172.21.15.3:3300/0 conn(0x7fb59809d6f0 0x7fb59809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:09.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.604+0000 7fb597fff700 1 -- 172.21.15.3:0/2713442710 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb59809e000 msgr2=0x7fb5981339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:09.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.604+0000 7fb597fff700 1 --2- 172.21.15.3:0/2713442710 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb59809e000 0x7fb5981339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.604+0000 7fb597fff700 1 -- 172.21.15.3:0/2713442710 >> v1:172.21.15.3:6789/0 conn(0x7fb59809e410 legacy=0x7fb598133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:09.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.604+0000 7fb597fff700 1 -- 172.21.15.3:0/2713442710 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59809a500 con 0x7fb59809d6f0 2023-10-03T10:19:09.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.604+0000 7fb596ffd700 1 -- 172.21.15.3:0/2713442710 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3282354073 0 0) 0x7fb59809b120 con 0x7fb59809e410 2023-10-03T10:19:09.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.604+0000 7fb5977fe700 1 --2- 172.21.15.3:0/2713442710 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb59809e000 0x7fb5981339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:09.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.604+0000 7fb597fff700 1 --2- 172.21.15.3:0/2713442710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb59809d6f0 0x7fb59809dac0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fb588009840 tx=0x7fb588009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f282fecca2441781 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:09.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.605+0000 7fb596ffd700 1 -- 172.21.15.3:0/2713442710 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb588011830 con 0x7fb59809d6f0 2023-10-03T10:19:09.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.605+0000 7fb596ffd700 1 -- 172.21.15.3:0/2713442710 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb588011990 con 0x7fb59809d6f0 2023-10-03T10:19:09.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.605+0000 7fb596ffd700 1 -- 172.21.15.3:0/2713442710 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb588011d20 con 0x7fb59809d6f0 2023-10-03T10:19:09.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.605+0000 7fb59e529700 1 -- 172.21.15.3:0/2713442710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb59809d6f0 msgr2=0x7fb59809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:09.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.605+0000 7fb59e529700 1 --2- 172.21.15.3:0/2713442710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb59809d6f0 0x7fb59809dac0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fb588009840 tx=0x7fb588009b50 comp rx=0 tx=0).stop 2023-10-03T10:19:09.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.606+0000 7fb59e529700 1 -- 172.21.15.3:0/2713442710 shutdown_connections 2023-10-03T10:19:09.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.606+0000 7fb59e529700 1 --2- 172.21.15.3:0/2713442710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb59809d6f0 0x7fb59809dac0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.606+0000 7fb59e529700 1 --2- 172.21.15.3:0/2713442710 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb59809e000 0x7fb5981339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.606+0000 7fb59e529700 1 -- 172.21.15.3:0/2713442710 >> 172.21.15.3:0/2713442710 conn(0x7fb598066f50 msgr2=0x7fb59809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:09.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.606+0000 7fb59e529700 1 -- 172.21.15.3:0/2713442710 shutdown_connections 2023-10-03T10:19:09.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.606+0000 7fb59e529700 1 -- 172.21.15.3:0/2713442710 wait complete. 2023-10-03T10:19:09.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.607+0000 7fb59e529700 1 Processor -- start 2023-10-03T10:19:09.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.607+0000 7fb59e529700 1 -- start start 2023-10-03T10:19:09.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.607+0000 7fb59e529700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb59809d6f0 0x7fb5981c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:09.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.607+0000 7fb59e529700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb59809e000 0x7fb5981c5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:09.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.607+0000 7fb59e529700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb59809e410 0x7fb5981c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:09.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.607+0000 7fb59e529700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5981c48e0 con 0x7fb59809e000 2023-10-03T10:19:09.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.607+0000 7fb59e529700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5980a1f00 con 0x7fb59809d6f0 2023-10-03T10:19:09.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.607+0000 7fb59e529700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb59809c710 con 0x7fb59809e410 2023-10-03T10:19:09.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.608+0000 7fb597fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb59809d6f0 0x7fb5981c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:09.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.608+0000 7fb59cac6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb59809e410 0x7fb5981c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:09.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.608+0000 7fb597fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb59809d6f0 0x7fb5981c50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41650/0 (socket says 172.21.15.3:41650) 2023-10-03T10:19:09.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.608+0000 7fb597fff700 1 -- 172.21.15.3:0/3513505436 learned_addr learned my addr 172.21.15.3:0/3513505436 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:09.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.608+0000 7fb5977fe700 1 --2- 172.21.15.3:0/3513505436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb59809e000 0x7fb5981c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:09.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.608+0000 7fb59cac6700 1 -- 172.21.15.3:0/3513505436 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb59809d6f0 msgr2=0x7fb5981c50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:09.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.608+0000 7fb59cac6700 1 --2- 172.21.15.3:0/3513505436 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb59809d6f0 0x7fb5981c50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.608+0000 7fb59cac6700 1 -- 172.21.15.3:0/3513505436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb59809e000 msgr2=0x7fb5981c5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:09.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.608+0000 7fb59cac6700 1 --2- 172.21.15.3:0/3513505436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb59809e000 0x7fb5981c5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.608+0000 7fb59cac6700 1 -- 172.21.15.3:0/3513505436 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58c009550 con 0x7fb59809e410 2023-10-03T10:19:09.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.608+0000 7fb59cac6700 1 --2- 172.21.15.3:0/3513505436 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb59809e410 0x7fb5981c7f90 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fb59000c730 tx=0x7fb59000ca40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:09.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.609+0000 7fb594ff9700 1 -- 172.21.15.3:0/3513505436 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb590003e30 con 0x7fb59809e410 2023-10-03T10:19:09.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.609+0000 7fb594ff9700 1 -- 172.21.15.3:0/3513505436 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb590005070 con 0x7fb59809e410 2023-10-03T10:19:09.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.609+0000 7fb594ff9700 1 -- 172.21.15.3:0/3513505436 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb590005420 con 0x7fb59809e410 2023-10-03T10:19:09.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.609+0000 7fb59e529700 1 -- 172.21.15.3:0/3513505436 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5880094a0 con 0x7fb59809e410 2023-10-03T10:19:09.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.609+0000 7fb59e529700 1 -- 172.21.15.3:0/3513505436 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5981a9b00 con 0x7fb59809e410 2023-10-03T10:19:09.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.610+0000 7fb594ff9700 1 -- 172.21.15.3:0/3513505436 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb590005580 con 0x7fb59809e410 2023-10-03T10:19:09.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.610+0000 7fb594ff9700 1 -- 172.21.15.3:0/3513505436 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb59003f1e0 con 0x7fb59809e410 2023-10-03T10:19:09.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.610+0000 7fb59e529700 1 -- 172.21.15.3:0/3513505436 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5981abaf0 con 0x7fb59809e410 2023-10-03T10:19:09.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.615+0000 7fb594ff9700 1 -- 172.21.15.3:0/3513505436 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb590010300 con 0x7fb59809e410 2023-10-03T10:19:09.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.645+0000 7f2cf28c4700 1 Processor -- start 2023-10-03T10:19:09.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.645+0000 7f2cf28c4700 1 -- start start 2023-10-03T10:19:09.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.646+0000 7f2cf28c4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2cec09f330 0x7f2cec138ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:09.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.646+0000 7f2cf28c4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2cec1393e0 0x7f2cec13b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:09.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.646+0000 7f2cf28c4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cec09f960 con 0x7f2cec09e850 2023-10-03T10:19:09.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.646+0000 7f2cf28c4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cec09f720 con 0x7f2cec09f330 2023-10-03T10:19:09.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.646+0000 7f2cf28c4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec09ae60 con 0x7f2cec1393e0 2023-10-03T10:19:09.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.646+0000 7f2cf0e61700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2cec1393e0 0x7f2cec13b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:09.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.646+0000 7f2cf0e61700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2cec1393e0 0x7f2cec13b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41654/0 (socket says 172.21.15.3:41654) 2023-10-03T10:19:09.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.646+0000 7f2cf0e61700 1 -- 172.21.15.3:0/521959489 learned_addr learned my addr 172.21.15.3:0/521959489 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:09.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.646+0000 7f2ceb7fe700 1 --2- 172.21.15.3:0/521959489 >> v2:172.21.15.3:3300/0 conn(0x7f2cec09f330 0x7f2cec138ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:09.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.647+0000 7f2cf0e61700 1 -- 172.21.15.3:0/521959489 >> v2:172.21.15.3:3300/0 conn(0x7f2cec09f330 msgr2=0x7f2cec138ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:09.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.647+0000 7f2cf0e61700 1 --2- 172.21.15.3:0/521959489 >> v2:172.21.15.3:3300/0 conn(0x7f2cec09f330 0x7f2cec138ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.647+0000 7f2cf0e61700 1 -- 172.21.15.3:0/521959489 >> v1:172.21.15.3:6789/0 conn(0x7f2cec09e850 legacy=0x7f2cec09ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:09.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.647+0000 7f2cf0e61700 1 -- 172.21.15.3:0/521959489 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cec06cb10 con 0x7f2cec1393e0 2023-10-03T10:19:09.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.647+0000 7f2cf0e61700 1 --2- 172.21.15.3:0/521959489 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2cec1393e0 0x7f2cec13b7c0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009e50 tx=0x7f2ce400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a37f2228513d0c75 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:09.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.647+0000 7f2ceaffd700 1 -- 172.21.15.3:0/521959489 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce400e070 con 0x7f2cec1393e0 2023-10-03T10:19:09.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.647+0000 7f2ceaffd700 1 -- 172.21.15.3:0/521959489 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce400b910 con 0x7f2cec1393e0 2023-10-03T10:19:09.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.647+0000 7f2ceaffd700 1 -- 172.21.15.3:0/521959489 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce400bcc0 con 0x7f2cec1393e0 2023-10-03T10:19:09.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.648+0000 7f2cf28c4700 1 -- 172.21.15.3:0/521959489 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2cec1393e0 msgr2=0x7f2cec13b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:09.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.648+0000 7f2cf28c4700 1 --2- 172.21.15.3:0/521959489 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2cec1393e0 0x7f2cec13b7c0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009e50 tx=0x7f2ce400b330 comp rx=0 tx=0).stop 2023-10-03T10:19:09.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.648+0000 7f2cf28c4700 1 -- 172.21.15.3:0/521959489 shutdown_connections 2023-10-03T10:19:09.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.648+0000 7f2cf28c4700 1 --2- 172.21.15.3:0/521959489 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2cec1393e0 0x7f2cec13b7c0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.648+0000 7f2cf28c4700 1 --2- 172.21.15.3:0/521959489 >> v2:172.21.15.3:3300/0 conn(0x7f2cec09f330 0x7f2cec138ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.648+0000 7f2cf28c4700 1 -- 172.21.15.3:0/521959489 >> 172.21.15.3:0/521959489 conn(0x7f2cec0a3f70 msgr2=0x7f2cec0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:09.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.648+0000 7f2cf28c4700 1 -- 172.21.15.3:0/521959489 shutdown_connections 2023-10-03T10:19:09.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.648+0000 7f2cf28c4700 1 -- 172.21.15.3:0/521959489 wait complete. 2023-10-03T10:19:09.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.649+0000 7f2cf28c4700 1 Processor -- start 2023-10-03T10:19:09.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.650+0000 7f2cf28c4700 1 -- start start 2023-10-03T10:19:09.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.650+0000 7f2cf28c4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2cec09e850 0x7f2cec11ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:09.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.650+0000 7f2cf28c4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2cec09f330 0x7f2cec11ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:09.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.650+0000 7f2cf28c4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cec1393e0 0x7f2cec11f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:09.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.650+0000 7f2cf28c4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec122210 con 0x7f2cec1393e0 2023-10-03T10:19:09.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.650+0000 7f2cf28c4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec121c60 con 0x7f2cec09e850 2023-10-03T10:19:09.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.650+0000 7f2cf28c4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec06f810 con 0x7f2cec09f330 2023-10-03T10:19:09.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.650+0000 7f2cebfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2cec09e850 0x7f2cec11ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:09.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.650+0000 7f2cebfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2cec09e850 0x7f2cec11ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41668/0 (socket says 172.21.15.3:41668) 2023-10-03T10:19:09.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.650+0000 7f2cebfff700 1 -- 172.21.15.3:0/649714062 learned_addr learned my addr 172.21.15.3:0/649714062 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:09.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.650+0000 7f2cf0e61700 1 --2- 172.21.15.3:0/649714062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cec1393e0 0x7f2cec11f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:09.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.650+0000 7f2ceb7fe700 1 --2- 172.21.15.3:0/649714062 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2cec09f330 0x7f2cec11ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:09.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.651+0000 7f2ceb7fe700 1 -- 172.21.15.3:0/649714062 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2cec09e850 msgr2=0x7f2cec11ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:09.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.651+0000 7f2ceb7fe700 1 --2- 172.21.15.3:0/649714062 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2cec09e850 0x7f2cec11ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.651+0000 7f2ceb7fe700 1 -- 172.21.15.3:0/649714062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cec1393e0 msgr2=0x7f2cec11f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:09.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.651+0000 7f2ceb7fe700 1 --2- 172.21.15.3:0/649714062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cec1393e0 0x7f2cec11f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.651+0000 7f2ceb7fe700 1 -- 172.21.15.3:0/649714062 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce4009b30 con 0x7f2cec09f330 2023-10-03T10:19:09.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.651+0000 7f2cebfff700 1 --2- 172.21.15.3:0/649714062 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2cec09e850 0x7f2cec11ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:09.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.651+0000 7f2cf0e61700 1 --2- 172.21.15.3:0/649714062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cec1393e0 0x7f2cec11f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:09.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.651+0000 7f2ceb7fe700 1 --2- 172.21.15.3:0/649714062 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2cec09f330 0x7f2cec11ef70 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2ce000e900 tx=0x7f2ce000ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:09.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.651+0000 7f2ce8ff9700 1 -- 172.21.15.3:0/649714062 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0013680 con 0x7f2cec09f330 2023-10-03T10:19:09.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.651+0000 7f2ce8ff9700 1 -- 172.21.15.3:0/649714062 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce00137e0 con 0x7f2cec09f330 2023-10-03T10:19:09.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.651+0000 7f2cf28c4700 1 -- 172.21.15.3:0/649714062 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cec06cd30 con 0x7f2cec09f330 2023-10-03T10:19:09.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.651+0000 7f2cf28c4700 1 -- 172.21.15.3:0/649714062 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cec120dd0 con 0x7f2cec09f330 2023-10-03T10:19:09.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.652+0000 7f2ce8ff9700 1 -- 172.21.15.3:0/649714062 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0013b50 con 0x7f2cec09f330 2023-10-03T10:19:09.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.653+0000 7f2ce8ff9700 1 -- 172.21.15.3:0/649714062 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2ce0013cb0 con 0x7f2cec09f330 2023-10-03T10:19:09.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.653+0000 7f2ce8ff9700 1 -- 172.21.15.3:0/649714062 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2ce00421c0 con 0x7f2cec09f330 2023-10-03T10:19:09.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.654+0000 7f2cf28c4700 1 -- 172.21.15.3:0/649714062 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ccc0052e0 con 0x7f2cec09f330 2023-10-03T10:19:09.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.659+0000 7f2ce8ff9700 1 -- 172.21.15.3:0/649714062 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2ce0017070 con 0x7f2cec09f330 2023-10-03T10:19:09.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.733+0000 7f360a3a2700 1 Processor -- start 2023-10-03T10:19:09.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.733+0000 7f360a3a2700 1 -- start start 2023-10-03T10:19:09.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.734+0000 7f360a3a2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f36040a0a30 0x7f36040a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:09.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.734+0000 7f360a3a2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3604134170 0x7f360413bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:09.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.734+0000 7f360a3a2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f360409edd0 con 0x7f36040a1340 2023-10-03T10:19:09.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.734+0000 7f360a3a2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f360409e1f0 con 0x7f36040a0a30 2023-10-03T10:19:09.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.734+0000 7f360a3a2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f360409dc50 con 0x7f3604134170 2023-10-03T10:19:09.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.734+0000 7f3603fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f36040a0a30 0x7f36040a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:09.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.734+0000 7f3603fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f36040a0a30 0x7f36040a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41680/0 (socket says 172.21.15.3:41680) 2023-10-03T10:19:09.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.734+0000 7f3603fff700 1 -- 172.21.15.3:0/4079979029 learned_addr learned my addr 172.21.15.3:0/4079979029 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:09.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.734+0000 7f360893f700 1 --2- 172.21.15.3:0/4079979029 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3604134170 0x7f360413bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:09.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.734+0000 7f3603fff700 1 -- 172.21.15.3:0/4079979029 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3604134170 msgr2=0x7f360413bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:09.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.734+0000 7f3603fff700 1 --2- 172.21.15.3:0/4079979029 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3604134170 0x7f360413bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.734+0000 7f3603fff700 1 -- 172.21.15.3:0/4079979029 >> v1:172.21.15.3:6789/0 conn(0x7f36040a1340 legacy=0x7f3604133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:09.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.734+0000 7f3603fff700 1 -- 172.21.15.3:0/4079979029 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36040a1fe0 con 0x7f36040a0a30 2023-10-03T10:19:09.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.734+0000 7f360893f700 1 --2- 172.21.15.3:0/4079979029 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3604134170 0x7f360413bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:09.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.735+0000 7f3603fff700 1 --2- 172.21.15.3:0/4079979029 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f36040a0a30 0x7f36040a0e00 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f35f4009920 tx=0x7f35f4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8306fcf8737a59a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:09.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.735+0000 7f3602ffd700 1 -- 172.21.15.3:0/4079979029 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f4011830 con 0x7f36040a0a30 2023-10-03T10:19:09.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.735+0000 7f3602ffd700 1 -- 172.21.15.3:0/4079979029 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f4011990 con 0x7f36040a0a30 2023-10-03T10:19:09.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.735+0000 7f3602ffd700 1 -- 172.21.15.3:0/4079979029 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f4011d20 con 0x7f36040a0a30 2023-10-03T10:19:09.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.735+0000 7f360a3a2700 1 -- 172.21.15.3:0/4079979029 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f36040a0a30 msgr2=0x7f36040a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:09.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.735+0000 7f360a3a2700 1 --2- 172.21.15.3:0/4079979029 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f36040a0a30 0x7f36040a0e00 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f35f4009920 tx=0x7f35f4009c30 comp rx=0 tx=0).stop 2023-10-03T10:19:09.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.735+0000 7f360a3a2700 1 -- 172.21.15.3:0/4079979029 shutdown_connections 2023-10-03T10:19:09.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.735+0000 7f360a3a2700 1 --2- 172.21.15.3:0/4079979029 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f36040a0a30 0x7f36040a0e00 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.736+0000 7f360a3a2700 1 --2- 172.21.15.3:0/4079979029 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3604134170 0x7f360413bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.736+0000 7f360a3a2700 1 -- 172.21.15.3:0/4079979029 >> 172.21.15.3:0/4079979029 conn(0x7f36040a6550 msgr2=0x7f36040a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:09.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.736+0000 7f360a3a2700 1 -- 172.21.15.3:0/4079979029 shutdown_connections 2023-10-03T10:19:09.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.736+0000 7f360a3a2700 1 -- 172.21.15.3:0/4079979029 wait complete. 2023-10-03T10:19:09.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.737+0000 7f360a3a2700 1 Processor -- start 2023-10-03T10:19:09.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.737+0000 7f360a3a2700 1 -- start start 2023-10-03T10:19:09.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.738+0000 7f360a3a2700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f36040a0a30 0x7f360411e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:09.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.738+0000 7f360a3a2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f36040a1340 0x7f360411ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:09.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.738+0000 7f360a3a2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3604134170 0x7f360411f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:09.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.738+0000 7f360a3a2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3604120360 con 0x7f3604134170 2023-10-03T10:19:09.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.738+0000 7f360a3a2700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3604133620 con 0x7f36040a1340 2023-10-03T10:19:09.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.738+0000 7f360a3a2700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3604121c30 con 0x7f36040a0a30 2023-10-03T10:19:09.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.738+0000 7f3603fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f36040a0a30 0x7f360411e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:09.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.738+0000 7f3603fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f36040a0a30 0x7f360411e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:43998/0 (socket says 172.21.15.3:43998) 2023-10-03T10:19:09.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.738+0000 7f3603fff700 1 -- 172.21.15.3:0/1051639597 learned_addr learned my addr 172.21.15.3:0/1051639597 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:09.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.739+0000 7f360893f700 1 --2- 172.21.15.3:0/1051639597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3604134170 0x7f360411f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:09.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.739+0000 7f36037fe700 1 --2- 172.21.15.3:0/1051639597 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f36040a1340 0x7f360411ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:09.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.739+0000 7f3603fff700 1 -- 172.21.15.3:0/1051639597 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f36040a1340 msgr2=0x7f360411ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:09.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.739+0000 7f3603fff700 1 --2- 172.21.15.3:0/1051639597 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f36040a1340 0x7f360411ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.739+0000 7f3603fff700 1 -- 172.21.15.3:0/1051639597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3604134170 msgr2=0x7f360411f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:09.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.739+0000 7f3603fff700 1 --2- 172.21.15.3:0/1051639597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3604134170 0x7f360411f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.739+0000 7f3603fff700 1 -- 172.21.15.3:0/1051639597 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35fc009a70 con 0x7f36040a0a30 2023-10-03T10:19:09.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.739+0000 7f360893f700 1 --2- 172.21.15.3:0/1051639597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3604134170 0x7f360411f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:09.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.739+0000 7f36037fe700 1 --2- 172.21.15.3:0/1051639597 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f36040a1340 0x7f360411ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:09.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.739+0000 7f3603fff700 1 --2- 172.21.15.3:0/1051639597 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f36040a0a30 0x7f360411e740 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f35f4000c00 tx=0x7f35f4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:09.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.739+0000 7f3600ff9700 1 -- 172.21.15.3:0/1051639597 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f4004590 con 0x7f36040a0a30 2023-10-03T10:19:09.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.739+0000 7f360a3a2700 1 -- 172.21.15.3:0/1051639597 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f4009550 con 0x7f36040a0a30 2023-10-03T10:19:09.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.739+0000 7f3600ff9700 1 -- 172.21.15.3:0/1051639597 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f40036a0 con 0x7f36040a0a30 2023-10-03T10:19:09.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.739+0000 7f3600ff9700 1 -- 172.21.15.3:0/1051639597 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f4003a30 con 0x7f36040a0a30 2023-10-03T10:19:09.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.739+0000 7f360a3a2700 1 -- 172.21.15.3:0/1051639597 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36040a25e0 con 0x7f36040a0a30 2023-10-03T10:19:09.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.740+0000 7f3600ff9700 1 -- 172.21.15.3:0/1051639597 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f35f4003b90 con 0x7f36040a0a30 2023-10-03T10:19:09.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.740+0000 7f3600ff9700 1 -- 172.21.15.3:0/1051639597 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f35f4021030 con 0x7f36040a0a30 2023-10-03T10:19:09.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.741+0000 7f35ea7fc700 1 -- 172.21.15.3:0/1051639597 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35dc0052e0 con 0x7f36040a0a30 2023-10-03T10:19:09.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.746+0000 7f3600ff9700 1 -- 172.21.15.3:0/1051639597 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f35f402b080 con 0x7f36040a0a30 2023-10-03T10:19:09.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.834+0000 7fb59e529700 1 -- 172.21.15.3:0/3513505436 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb598074ea0 con 0x7fb59809e410 2023-10-03T10:19:09.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.834+0000 7fb594ff9700 1 -- 172.21.15.3:0/3513505436 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb590019ab0 con 0x7fb59809e410 2023-10-03T10:19:09.835 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:09.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.837+0000 7fb59e529700 1 -- 172.21.15.3:0/3513505436 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb59809e410 msgr2=0x7fb5981c7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:09.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.837+0000 7fb59e529700 1 --2- 172.21.15.3:0/3513505436 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb59809e410 0x7fb5981c7f90 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fb59000c730 tx=0x7fb59000ca40 comp rx=0 tx=0).stop 2023-10-03T10:19:09.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.837+0000 7fb59e529700 1 -- 172.21.15.3:0/3513505436 shutdown_connections 2023-10-03T10:19:09.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.837+0000 7fb59e529700 1 --2- 172.21.15.3:0/3513505436 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb59809e410 0x7fb5981c7f90 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.837+0000 7fb59e529700 1 --2- 172.21.15.3:0/3513505436 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb59809d6f0 0x7fb5981c50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.837+0000 7fb59e529700 1 --2- 172.21.15.3:0/3513505436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb59809e000 0x7fb5981c5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.837+0000 7fb59e529700 1 -- 172.21.15.3:0/3513505436 >> 172.21.15.3:0/3513505436 conn(0x7fb598066f50 msgr2=0x7fb59806f270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:09.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.837+0000 7fb59e529700 1 -- 172.21.15.3:0/3513505436 shutdown_connections 2023-10-03T10:19:09.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.837+0000 7fb59e529700 1 -- 172.21.15.3:0/3513505436 wait complete. 2023-10-03T10:19:09.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.850+0000 7f814e181700 1 -- 172.21.15.3:0/1512609749 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8148074ea0 con 0x7f8148131fe0 2023-10-03T10:19:09.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.851+0000 7f812ffff700 1 -- 172.21.15.3:0/1512609749 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8140011b40 con 0x7f8148131fe0 2023-10-03T10:19:09.851 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:09.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.853+0000 7f814e181700 1 -- 172.21.15.3:0/1512609749 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8148131fe0 msgr2=0x7f8148133820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:09.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.853+0000 7f814e181700 1 --2- 172.21.15.3:0/1512609749 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8148131fe0 0x7f8148133820 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f8140014040 tx=0x7f8140004ad0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.853+0000 7f814e181700 1 -- 172.21.15.3:0/1512609749 shutdown_connections 2023-10-03T10:19:09.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.853+0000 7f814e181700 1 --2- 172.21.15.3:0/1512609749 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8148131fe0 0x7f8148133820 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.853+0000 7f814e181700 1 --2- 172.21.15.3:0/1512609749 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f814806ea90 0x7f8148132da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.853+0000 7f814e181700 1 --2- 172.21.15.3:0/1512609749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81481316d0 0x7f81481332e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.853+0000 7f814e181700 1 -- 172.21.15.3:0/1512609749 >> 172.21.15.3:0/1512609749 conn(0x7f81480a2aa0 msgr2=0x7f8148070e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:09.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.853+0000 7f814e181700 1 -- 172.21.15.3:0/1512609749 shutdown_connections 2023-10-03T10:19:09.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.854+0000 7f814e181700 1 -- 172.21.15.3:0/1512609749 wait complete. 2023-10-03T10:19:09.856 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:09.868 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:09.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.879+0000 7f2cf28c4700 1 -- 172.21.15.3:0/649714062 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2ccc005590 con 0x7f2cec09f330 2023-10-03T10:19:09.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.879+0000 7f2ce8ff9700 1 -- 172.21.15.3:0/649714062 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2ce0017020 con 0x7f2cec09f330 2023-10-03T10:19:09.879 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:09.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.881+0000 7f2cf28c4700 1 -- 172.21.15.3:0/649714062 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2cec09f330 msgr2=0x7f2cec11ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:09.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.881+0000 7f2cf28c4700 1 --2- 172.21.15.3:0/649714062 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2cec09f330 0x7f2cec11ef70 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2ce000e900 tx=0x7f2ce000ec10 comp rx=0 tx=0).stop 2023-10-03T10:19:09.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.881+0000 7f2cf28c4700 1 -- 172.21.15.3:0/649714062 shutdown_connections 2023-10-03T10:19:09.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.881+0000 7f2cf28c4700 1 --2- 172.21.15.3:0/649714062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cec1393e0 0x7f2cec11f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.881+0000 7f2cf28c4700 1 --2- 172.21.15.3:0/649714062 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2cec09e850 0x7f2cec11ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.881+0000 7f2cf28c4700 1 --2- 172.21.15.3:0/649714062 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2cec09f330 0x7f2cec11ef70 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.881+0000 7f2cf28c4700 1 -- 172.21.15.3:0/649714062 >> 172.21.15.3:0/649714062 conn(0x7f2cec0a3f70 msgr2=0x7f2cec0710f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:09.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.881+0000 7f2cf28c4700 1 -- 172.21.15.3:0/649714062 shutdown_connections 2023-10-03T10:19:09.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.881+0000 7f2cf28c4700 1 -- 172.21.15.3:0/649714062 wait complete. 2023-10-03T10:19:09.896 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:09.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.956+0000 7f35ea7fc700 1 -- 172.21.15.3:0/1051639597 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f35dc005590 con 0x7f36040a0a30 2023-10-03T10:19:09.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.956+0000 7f3600ff9700 1 -- 172.21.15.3:0/1051639597 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f35f4015020 con 0x7f36040a0a30 2023-10-03T10:19:09.957 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:09.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.958+0000 7f35ea7fc700 1 -- 172.21.15.3:0/1051639597 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f36040a0a30 msgr2=0x7f360411e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:09.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.958+0000 7f35ea7fc700 1 --2- 172.21.15.3:0/1051639597 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f36040a0a30 0x7f360411e740 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f35f4000c00 tx=0x7f35f4004750 comp rx=0 tx=0).stop 2023-10-03T10:19:09.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.958+0000 7f35ea7fc700 1 -- 172.21.15.3:0/1051639597 shutdown_connections 2023-10-03T10:19:09.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.958+0000 7f35ea7fc700 1 --2- 172.21.15.3:0/1051639597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3604134170 0x7f360411f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.958+0000 7f35ea7fc700 1 --2- 172.21.15.3:0/1051639597 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f36040a0a30 0x7f360411e740 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.958+0000 7f35ea7fc700 1 --2- 172.21.15.3:0/1051639597 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f36040a1340 0x7f360411ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:09.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.958+0000 7f35ea7fc700 1 -- 172.21.15.3:0/1051639597 >> 172.21.15.3:0/1051639597 conn(0x7f36040a6550 msgr2=0x7f36041b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:09.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.959+0000 7f35ea7fc700 1 -- 172.21.15.3:0/1051639597 shutdown_connections 2023-10-03T10:19:09.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:09.959+0000 7f35ea7fc700 1 -- 172.21.15.3:0/1051639597 wait complete. 2023-10-03T10:19:09.972 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:10.857 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:10.868 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:10.897 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:10.973 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:11.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.027+0000 7f6b57a4c700 1 Processor -- start 2023-10-03T10:19:11.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.027+0000 7f6b57a4c700 1 -- start start 2023-10-03T10:19:11.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.028+0000 7f6b57a4c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6b5009d6f0 0x7f6b5009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:11.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.028+0000 7f6b57a4c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6b5009e000 0x7f6b50130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:11.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.028+0000 7f6b57a4c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5009f130 con 0x7f6b5009e410 2023-10-03T10:19:11.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.028+0000 7f6b57a4c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b5009f520 con 0x7f6b5009e000 2023-10-03T10:19:11.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.028+0000 7f6b57a4c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5009e930 con 0x7f6b5009d6f0 2023-10-03T10:19:11.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.028+0000 7f6b557e8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6b5009d6f0 0x7f6b5009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:11.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.028+0000 7f6b54fe7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6b5009e000 0x7f6b50130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:11.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.028+0000 7f6b557e8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6b5009d6f0 0x7f6b5009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41694/0 (socket says 172.21.15.3:41694) 2023-10-03T10:19:11.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.028+0000 7f6b54fe7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6b5009e000 0x7f6b50130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46848/0 (socket says 172.21.15.3:46848) 2023-10-03T10:19:11.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.028+0000 7f6b557e8700 1 -- 172.21.15.3:0/3801473773 learned_addr learned my addr 172.21.15.3:0/3801473773 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:11.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.029+0000 7f6b557e8700 1 -- 172.21.15.3:0/3801473773 >> v2:172.21.15.3:3300/0 conn(0x7f6b5009e000 msgr2=0x7f6b50130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:11.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.029+0000 7f6b557e8700 1 --2- 172.21.15.3:0/3801473773 >> v2:172.21.15.3:3300/0 conn(0x7f6b5009e000 0x7f6b50130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.029+0000 7f6b557e8700 1 -- 172.21.15.3:0/3801473773 >> v1:172.21.15.3:6789/0 conn(0x7f6b5009e410 legacy=0x7f6b50130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:11.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.029+0000 7f6b557e8700 1 -- 172.21.15.3:0/3801473773 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5009b2d0 con 0x7f6b5009d6f0 2023-10-03T10:19:11.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.029+0000 7f6b557e8700 1 --2- 172.21.15.3:0/3801473773 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6b5009d6f0 0x7f6b5009dac0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c00bb90 tx=0x7f6b4c00bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d6c3383ffa528a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:11.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.029+0000 7f6b47fff700 1 -- 172.21.15.3:0/3801473773 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c00dd30 con 0x7f6b5009d6f0 2023-10-03T10:19:11.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.029+0000 7f6b47fff700 1 -- 172.21.15.3:0/3801473773 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4c00de90 con 0x7f6b5009d6f0 2023-10-03T10:19:11.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.029+0000 7f6b47fff700 1 -- 172.21.15.3:0/3801473773 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c010690 con 0x7f6b5009d6f0 2023-10-03T10:19:11.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.030+0000 7f6b57a4c700 1 -- 172.21.15.3:0/3801473773 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6b5009d6f0 msgr2=0x7f6b5009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:11.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.030+0000 7f6b57a4c700 1 --2- 172.21.15.3:0/3801473773 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6b5009d6f0 0x7f6b5009dac0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c00bb90 tx=0x7f6b4c00bea0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.030+0000 7f6b57a4c700 1 -- 172.21.15.3:0/3801473773 shutdown_connections 2023-10-03T10:19:11.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.030+0000 7f6b57a4c700 1 --2- 172.21.15.3:0/3801473773 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6b5009d6f0 0x7f6b5009dac0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.030+0000 7f6b57a4c700 1 --2- 172.21.15.3:0/3801473773 >> v2:172.21.15.3:3300/0 conn(0x7f6b5009e000 0x7f6b50130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.030+0000 7f6b57a4c700 1 -- 172.21.15.3:0/3801473773 >> 172.21.15.3:0/3801473773 conn(0x7f6b50066f50 msgr2=0x7f6b5009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:11.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.031+0000 7f6b57a4c700 1 -- 172.21.15.3:0/3801473773 shutdown_connections 2023-10-03T10:19:11.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.031+0000 7f6b57a4c700 1 -- 172.21.15.3:0/3801473773 wait complete. 2023-10-03T10:19:11.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.032+0000 7f6b57a4c700 1 Processor -- start 2023-10-03T10:19:11.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.032+0000 7f6b57a4c700 1 -- start start 2023-10-03T10:19:11.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.033+0000 7f6b57a4c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b5009d6f0 0x7f6b5011e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:11.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.033+0000 7f6b57a4c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6b5009e000 0x7f6b5011e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:11.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.033+0000 7f6b57a4c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6b5009e410 0x7f6b5011eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:11.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.033+0000 7f6b57a4c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b50121bb0 con 0x7f6b5009d6f0 2023-10-03T10:19:11.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.033+0000 7f6b57a4c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b500a1d00 con 0x7f6b5009e000 2023-10-03T10:19:11.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.033+0000 7f6b57a4c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5009c700 con 0x7f6b5009e410 2023-10-03T10:19:11.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.033+0000 7f6b557e8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b5009d6f0 0x7f6b5011e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:11.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.033+0000 7f6b557e8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b5009d6f0 0x7f6b5011e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46866/0 (socket says 172.21.15.3:46866) 2023-10-03T10:19:11.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.033+0000 7f6b557e8700 1 -- 172.21.15.3:0/1200627412 learned_addr learned my addr 172.21.15.3:0/1200627412 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:11.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.033+0000 7f6b54fe7700 1 --2- 172.21.15.3:0/1200627412 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6b5009e000 0x7f6b5011e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:11.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.033+0000 7f6b55fe9700 1 --2- 172.21.15.3:0/1200627412 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6b5009e410 0x7f6b5011eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:11.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.034+0000 7f6b557e8700 1 -- 172.21.15.3:0/1200627412 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6b5009e410 msgr2=0x7f6b5011eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:11.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.034+0000 7f6b557e8700 1 --2- 172.21.15.3:0/1200627412 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6b5009e410 0x7f6b5011eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.034+0000 7f6b557e8700 1 -- 172.21.15.3:0/1200627412 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6b5009e000 msgr2=0x7f6b5011e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:11.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.034+0000 7f6b557e8700 1 --2- 172.21.15.3:0/1200627412 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6b5009e000 0x7f6b5011e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.034+0000 7f6b557e8700 1 -- 172.21.15.3:0/1200627412 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b4c00b7f0 con 0x7f6b5009d6f0 2023-10-03T10:19:11.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.034+0000 7f6b557e8700 1 --2- 172.21.15.3:0/1200627412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b5009d6f0 0x7f6b5011e0e0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c000c00 tx=0x7f6b4c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:11.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.034+0000 7f6b45ffb700 1 -- 172.21.15.3:0/1200627412 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c003b70 con 0x7f6b5009d6f0 2023-10-03T10:19:11.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.034+0000 7f6b45ffb700 1 -- 172.21.15.3:0/1200627412 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4c003cd0 con 0x7f6b5009d6f0 2023-10-03T10:19:11.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.034+0000 7f6b57a4c700 1 -- 172.21.15.3:0/1200627412 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b50121610 con 0x7f6b5009d6f0 2023-10-03T10:19:11.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.034+0000 7f6b45ffb700 1 -- 172.21.15.3:0/1200627412 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c010670 con 0x7f6b5009d6f0 2023-10-03T10:19:11.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.034+0000 7f6b57a4c700 1 -- 172.21.15.3:0/1200627412 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b501201f0 con 0x7f6b5009d6f0 2023-10-03T10:19:11.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.035+0000 7f6b45ffb700 1 -- 172.21.15.3:0/1200627412 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6b4c019400 con 0x7f6b5009d6f0 2023-10-03T10:19:11.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.035+0000 7f6b45ffb700 1 -- 172.21.15.3:0/1200627412 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6b4c040e40 con 0x7f6b5009d6f0 2023-10-03T10:19:11.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.037+0000 7f6b57a4c700 1 -- 172.21.15.3:0/1200627412 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b300052e0 con 0x7f6b5009d6f0 2023-10-03T10:19:11.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.042+0000 7f6b45ffb700 1 -- 172.21.15.3:0/1200627412 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6b4c00f3d0 con 0x7f6b5009d6f0 2023-10-03T10:19:11.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.055+0000 7f1cb5e84700 1 Processor -- start 2023-10-03T10:19:11.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.055+0000 7f1cb5e84700 1 -- start start 2023-10-03T10:19:11.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.056+0000 7f1cb5e84700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1cb0133420 0x7f1cb01337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:11.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.056+0000 7f1cb5e84700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cb0133e30 0x7f1cb012ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:11.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.056+0000 7f1cb5e84700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1cb0134260 0x7f1cb01304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:11.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.056+0000 7f1cb5e84700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cb006b990 con 0x7f1cb0134260 2023-10-03T10:19:11.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.056+0000 7f1cb5e84700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb00a0590 con 0x7f1cb0133e30 2023-10-03T10:19:11.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.056+0000 7f1cb5e84700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb009fb80 con 0x7f1cb0133420 2023-10-03T10:19:11.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.056+0000 7f1caeffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cb0133e30 0x7f1cb012ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:11.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.056+0000 7f1caeffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cb0133e30 0x7f1cb012ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41712/0 (socket says 172.21.15.3:41712) 2023-10-03T10:19:11.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.056+0000 7f1caeffd700 1 -- 172.21.15.3:0/1489137188 learned_addr learned my addr 172.21.15.3:0/1489137188 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:11.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.056+0000 7f1caeffd700 1 -- 172.21.15.3:0/1489137188 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1cb0133420 msgr2=0x7f1cb01337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:11.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.056+0000 7f1caeffd700 1 --2- 172.21.15.3:0/1489137188 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1cb0133420 0x7f1cb01337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.056+0000 7f1caeffd700 1 -- 172.21.15.3:0/1489137188 >> v2:172.21.15.3:3300/0 conn(0x7f1cb0134260 msgr2=0x7f1cb01304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:11.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.056+0000 7f1caffff700 1 --2- 172.21.15.3:0/1489137188 >> v2:172.21.15.3:3300/0 conn(0x7f1cb0134260 0x7f1cb01304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:11.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.056+0000 7f1caeffd700 1 --2- 172.21.15.3:0/1489137188 >> v2:172.21.15.3:3300/0 conn(0x7f1cb0134260 0x7f1cb01304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.056+0000 7f1caeffd700 1 -- 172.21.15.3:0/1489137188 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb00a0030 con 0x7f1cb0133e30 2023-10-03T10:19:11.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.056+0000 7f1caffff700 1 --2- 172.21.15.3:0/1489137188 >> v2:172.21.15.3:3300/0 conn(0x7f1cb0134260 0x7f1cb01304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:19:11.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.057+0000 7f1caeffd700 1 --2- 172.21.15.3:0/1489137188 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cb0133e30 0x7f1cb012ff90 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40098f0 tx=0x7f1ca4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9de868a792bea615 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:11.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.057+0000 7f1cae7fc700 1 -- 172.21.15.3:0/1489137188 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca400b8a0 con 0x7f1cb0133e30 2023-10-03T10:19:11.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.057+0000 7f1cae7fc700 1 -- 172.21.15.3:0/1489137188 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca400ba00 con 0x7f1cb0133e30 2023-10-03T10:19:11.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.057+0000 7f1cae7fc700 1 -- 172.21.15.3:0/1489137188 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca400bd90 con 0x7f1cb0133e30 2023-10-03T10:19:11.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.057+0000 7f1c45bd3700 1 Processor -- start 2023-10-03T10:19:11.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.057+0000 7f1c45bd3700 1 -- start start 2023-10-03T10:19:11.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.058+0000 7f1cb5e84700 1 -- 172.21.15.3:0/1489137188 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cb0133e30 msgr2=0x7f1cb012ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:11.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.058+0000 7f1cb5e84700 1 --2- 172.21.15.3:0/1489137188 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cb0133e30 0x7f1cb012ff90 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40098f0 tx=0x7f1ca4009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:11.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.058+0000 7f1cb5e84700 1 -- 172.21.15.3:0/1489137188 shutdown_connections 2023-10-03T10:19:11.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.058+0000 7f1c45bd3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c4009e930 0x7f1c4009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:11.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.058+0000 7f1c45bd3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1c4009f240 0x7f1c401317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:11.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.058+0000 7f1c45bd3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c40070940 con 0x7f1c4009f650 2023-10-03T10:19:11.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.058+0000 7f1c45bd3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c4006fd60 con 0x7f1c4009f240 2023-10-03T10:19:11.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.058+0000 7f1c45bd3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4006fa80 con 0x7f1c4009e930 2023-10-03T10:19:11.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.058+0000 7f1cb5e84700 1 --2- 172.21.15.3:0/1489137188 >> v2:172.21.15.3:3300/0 conn(0x7f1cb0134260 0x7f1cb01304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.058+0000 7f1cb5e84700 1 --2- 172.21.15.3:0/1489137188 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1cb0133420 0x7f1cb01337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.058+0000 7f1cb5e84700 1 --2- 172.21.15.3:0/1489137188 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cb0133e30 0x7f1cb012ff90 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.058+0000 7f1cb5e84700 1 -- 172.21.15.3:0/1489137188 >> 172.21.15.3:0/1489137188 conn(0x7f1cb00a4210 msgr2=0x7f1cb00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:11.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.058+0000 7f1c3effd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1c4009f240 0x7f1c401317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:11.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.058+0000 7f1c3effd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1c4009f240 0x7f1c401317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46882/0 (socket says 172.21.15.3:46882) 2023-10-03T10:19:11.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.058+0000 7f1c3effd700 1 -- 172.21.15.3:0/4122979586 learned_addr learned my addr 172.21.15.3:0/4122979586 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:11.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.058+0000 7f1c3f7fe700 1 --2- 172.21.15.3:0/4122979586 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c4009e930 0x7f1c4009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:11.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.058+0000 7f1cb5e84700 1 -- 172.21.15.3:0/1489137188 shutdown_connections 2023-10-03T10:19:11.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.059+0000 7f1cb5e84700 1 -- 172.21.15.3:0/1489137188 wait complete. 2023-10-03T10:19:11.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.058+0000 7f1c3f7fe700 1 -- 172.21.15.3:0/4122979586 >> v2:172.21.15.3:3300/0 conn(0x7f1c4009f240 msgr2=0x7f1c401317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:11.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.059+0000 7f1c3f7fe700 1 --2- 172.21.15.3:0/4122979586 >> v2:172.21.15.3:3300/0 conn(0x7f1c4009f240 0x7f1c401317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.059+0000 7f1c3f7fe700 1 -- 172.21.15.3:0/4122979586 >> v1:172.21.15.3:6789/0 conn(0x7f1c4009f650 legacy=0x7f1c40131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:11.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.059+0000 7f1c3f7fe700 1 -- 172.21.15.3:0/4122979586 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4009fb60 con 0x7f1c4009e930 2023-10-03T10:19:11.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.059+0000 7f1c3effd700 1 --2- 172.21.15.3:0/4122979586 >> v2:172.21.15.3:3300/0 conn(0x7f1c4009f240 0x7f1c401317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:11.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.059+0000 7f1c3f7fe700 1 --2- 172.21.15.3:0/4122979586 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c4009e930 0x7f1c4009ed00 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000dc60 tx=0x7f1c3000df70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b75e798c5989b0c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:11.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.059+0000 7f1c3e7fc700 1 -- 172.21.15.3:0/4122979586 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c30012070 con 0x7f1c4009e930 2023-10-03T10:19:11.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.059+0000 7f1c3e7fc700 1 -- 172.21.15.3:0/4122979586 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3000d2e0 con 0x7f1c4009e930 2023-10-03T10:19:11.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.059+0000 7f1c3e7fc700 1 -- 172.21.15.3:0/4122979586 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3000f900 con 0x7f1c4009e930 2023-10-03T10:19:11.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.060+0000 7f1cb5e84700 1 Processor -- start 2023-10-03T10:19:11.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.060+0000 7f1c45bd3700 1 -- 172.21.15.3:0/4122979586 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c4009e930 msgr2=0x7f1c4009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:11.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.060+0000 7f1c45bd3700 1 --2- 172.21.15.3:0/4122979586 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c4009e930 0x7f1c4009ed00 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000dc60 tx=0x7f1c3000df70 comp rx=0 tx=0).stop 2023-10-03T10:19:11.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.060+0000 7f1c45bd3700 1 -- 172.21.15.3:0/4122979586 shutdown_connections 2023-10-03T10:19:11.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.060+0000 7f1c45bd3700 1 --2- 172.21.15.3:0/4122979586 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c4009e930 0x7f1c4009ed00 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.060+0000 7f1c45bd3700 1 --2- 172.21.15.3:0/4122979586 >> v2:172.21.15.3:3300/0 conn(0x7f1c4009f240 0x7f1c401317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.060+0000 7f1c45bd3700 1 -- 172.21.15.3:0/4122979586 >> 172.21.15.3:0/4122979586 conn(0x7f1c400a4210 msgr2=0x7f1c400a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:11.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.060+0000 7f1c45bd3700 1 -- 172.21.15.3:0/4122979586 shutdown_connections 2023-10-03T10:19:11.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.060+0000 7f1c45bd3700 1 -- 172.21.15.3:0/4122979586 wait complete. 2023-10-03T10:19:11.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.060+0000 7f1cb5e84700 1 -- start start 2023-10-03T10:19:11.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.061+0000 7f1cb5e84700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cb0133420 0x7f1cb009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:11.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.061+0000 7f1cb5e84700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1cb0133e30 0x7f1cb009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:11.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.061+0000 7f1cb5e84700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cb0134260 0x7f1cb009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:11.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.061+0000 7f1cb5e84700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb009e390 con 0x7f1cb0134260 2023-10-03T10:19:11.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.061+0000 7f1cb5e84700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb011fcc0 con 0x7f1cb0133420 2023-10-03T10:19:11.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.061+0000 7f1cb5e84700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb011ff10 con 0x7f1cb0133e30 2023-10-03T10:19:11.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.061+0000 7f1caeffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1cb0133e30 0x7f1cb009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:11.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.061+0000 7f1caeffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1cb0133e30 0x7f1cb009c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:44036/0 (socket says 172.21.15.3:44036) 2023-10-03T10:19:11.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.061+0000 7f1caeffd700 1 -- 172.21.15.3:0/949420806 learned_addr learned my addr 172.21.15.3:0/949420806 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:11.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.061+0000 7f1caffff700 1 --2- 172.21.15.3:0/949420806 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cb0134260 0x7f1cb009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:11.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.062+0000 7f1caf7fe700 1 --2- 172.21.15.3:0/949420806 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cb0133420 0x7f1cb009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:11.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.062+0000 7f1c45bd3700 1 Processor -- start 2023-10-03T10:19:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.062+0000 7f1caeffd700 1 -- 172.21.15.3:0/949420806 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cb0133420 msgr2=0x7f1cb009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.062+0000 7f1caeffd700 1 --2- 172.21.15.3:0/949420806 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cb0133420 0x7f1cb009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.062+0000 7f1caeffd700 1 -- 172.21.15.3:0/949420806 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cb0134260 msgr2=0x7f1cb009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.062+0000 7f1caeffd700 1 --2- 172.21.15.3:0/949420806 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cb0134260 0x7f1cb009cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.062+0000 7f1caeffd700 1 -- 172.21.15.3:0/949420806 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca4009550 con 0x7f1cb0133e30 2023-10-03T10:19:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.062+0000 7f1caeffd700 1 --2- 172.21.15.3:0/949420806 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1cb0133e30 0x7f1cb009c9c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4013000 tx=0x7f1ca4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.062+0000 7f1c97fff700 1 -- 172.21.15.3:0/949420806 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4004590 con 0x7f1cb0133e30 2023-10-03T10:19:11.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.062+0000 7f1c97fff700 1 -- 172.21.15.3:0/949420806 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca40036a0 con 0x7f1cb0133e30 2023-10-03T10:19:11.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.062+0000 7f1c97fff700 1 -- 172.21.15.3:0/949420806 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4003a10 con 0x7f1cb0133e30 2023-10-03T10:19:11.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.062+0000 7f1cb5e84700 1 -- 172.21.15.3:0/949420806 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb0120980 con 0x7f1cb0133e30 2023-10-03T10:19:11.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.062+0000 7f1cb5e84700 1 -- 172.21.15.3:0/949420806 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb009dde0 con 0x7f1cb0133e30 2023-10-03T10:19:11.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.063+0000 7f1c97fff700 1 -- 172.21.15.3:0/949420806 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1ca4021070 con 0x7f1cb0133e30 2023-10-03T10:19:11.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.064+0000 7f1c97fff700 1 -- 172.21.15.3:0/949420806 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1ca4040d70 con 0x7f1cb0133e30 2023-10-03T10:19:11.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.062+0000 7f1c45bd3700 1 -- start start 2023-10-03T10:19:11.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.063+0000 7f1c45bd3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c4009e930 0x7f1c401c0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:11.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.063+0000 7f1c45bd3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c4009f240 0x7f1c401c1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:11.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.063+0000 7f1c45bd3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c4009f650 0x7f1c401c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:11.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.063+0000 7f1c45bd3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c401c18b0 con 0x7f1c4009f240 2023-10-03T10:19:11.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.063+0000 7f1c45bd3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c401a52d0 con 0x7f1c4009f650 2023-10-03T10:19:11.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.063+0000 7f1c45bd3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c40071d20 con 0x7f1c4009e930 2023-10-03T10:19:11.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.063+0000 7f1c3ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c4009f650 0x7f1c401c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:11.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.063+0000 7f1c3ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c4009f650 0x7f1c401c3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41742/0 (socket says 172.21.15.3:41742) 2023-10-03T10:19:11.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.063+0000 7f1c3ffff700 1 -- 172.21.15.3:0/319964063 learned_addr learned my addr 172.21.15.3:0/319964063 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:11.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.063+0000 7f1c3effd700 1 --2- 172.21.15.3:0/319964063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c4009f240 0x7f1c401c1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:11.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.064+0000 7f1cb5e84700 1 -- 172.21.15.3:0/949420806 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb012dba0 con 0x7f1cb0133e30 2023-10-03T10:19:11.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.065+0000 7f1c3f7fe700 1 --2- 172.21.15.3:0/319964063 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c4009e930 0x7f1c401c0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:11.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.065+0000 7f1c3ffff700 1 -- 172.21.15.3:0/319964063 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c4009e930 msgr2=0x7f1c401c0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:11.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.065+0000 7f1c3ffff700 1 --2- 172.21.15.3:0/319964063 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c4009e930 0x7f1c401c0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.065+0000 7f1c3ffff700 1 -- 172.21.15.3:0/319964063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c4009f240 msgr2=0x7f1c401c1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:11.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.065+0000 7f1c3ffff700 1 --2- 172.21.15.3:0/319964063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c4009f240 0x7f1c401c1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.065+0000 7f1c3ffff700 1 -- 172.21.15.3:0/319964063 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c34009550 con 0x7f1c4009f650 2023-10-03T10:19:11.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.065+0000 7f1c3f7fe700 1 --2- 172.21.15.3:0/319964063 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c4009e930 0x7f1c401c0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:19:11.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.065+0000 7f1c3ffff700 1 --2- 172.21.15.3:0/319964063 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c4009f650 0x7f1c401c3e20 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800ccb0 tx=0x7f1c3800a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:11.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.066+0000 7f1c27fff700 1 -- 172.21.15.3:0/319964063 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c38003c00 con 0x7f1c4009f650 2023-10-03T10:19:11.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.066+0000 7f1c27fff700 1 -- 172.21.15.3:0/319964063 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c38003d60 con 0x7f1c4009f650 2023-10-03T10:19:11.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.066+0000 7f1c27fff700 1 -- 172.21.15.3:0/319964063 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3800ab80 con 0x7f1c4009f650 2023-10-03T10:19:11.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.066+0000 7f1c45bd3700 1 -- 172.21.15.3:0/319964063 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c3000d940 con 0x7f1c4009f650 2023-10-03T10:19:11.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.066+0000 7f1c45bd3700 1 -- 172.21.15.3:0/319964063 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c400a0190 con 0x7f1c4009f650 2023-10-03T10:19:11.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.067+0000 7f1c27fff700 1 -- 172.21.15.3:0/319964063 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1c3800ace0 con 0x7f1c4009f650 2023-10-03T10:19:11.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.067+0000 7f1c27fff700 1 -- 172.21.15.3:0/319964063 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1c3803f790 con 0x7f1c4009f650 2023-10-03T10:19:11.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.070+0000 7f1c97fff700 1 -- 172.21.15.3:0/949420806 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1ca4029020 con 0x7f1cb0133e30 2023-10-03T10:19:11.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.070+0000 7f1c45bd3700 1 -- 172.21.15.3:0/319964063 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c200052e0 con 0x7f1c4009f650 2023-10-03T10:19:11.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.075+0000 7f1c27fff700 1 -- 172.21.15.3:0/319964063 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1c380198f0 con 0x7f1c4009f650 2023-10-03T10:19:11.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.122+0000 7fc4c66ea700 1 Processor -- start 2023-10-03T10:19:11.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.122+0000 7fc4c66ea700 1 -- start start 2023-10-03T10:19:11.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.123+0000 7fc4c66ea700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc4c00a0a30 0x7fc4c00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:11.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.123+0000 7fc4c66ea700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc4c00a1340 0x7fc4c0133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:11.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.123+0000 7fc4c66ea700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc4c0133fa0 0x7fc4c0134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:11.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.123+0000 7fc4c66ea700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4c009edd0 con 0x7fc4c00a0a30 2023-10-03T10:19:11.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.123+0000 7fc4c66ea700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c009e1f0 con 0x7fc4c0133fa0 2023-10-03T10:19:11.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.123+0000 7fc4c66ea700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c009dc50 con 0x7fc4c00a1340 2023-10-03T10:19:11.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.123+0000 7fc4bffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc4c00a0a30 0x7fc4c00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:11.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.123+0000 7fc4c4c87700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc4c0133fa0 0x7fc4c0134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:11.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.123+0000 7fc4bffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc4c00a0a30 0x7fc4c00a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46908/0 (socket says 172.21.15.3:46908) 2023-10-03T10:19:11.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.123+0000 7fc4bffff700 1 -- 172.21.15.3:0/964133337 learned_addr learned my addr 172.21.15.3:0/964133337 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:11.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.123+0000 7fc4c4c87700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc4c0133fa0 0x7fc4c0134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41754/0 (socket says 172.21.15.3:41754) 2023-10-03T10:19:11.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.123+0000 7fc4bf7fe700 1 --2- 172.21.15.3:0/964133337 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc4c00a1340 0x7fc4c0133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:11.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.123+0000 7fc4c4c87700 1 -- 172.21.15.3:0/964133337 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc4c00a1340 msgr2=0x7fc4c0133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:11.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.123+0000 7fc4c4c87700 1 --2- 172.21.15.3:0/964133337 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc4c00a1340 0x7fc4c0133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.123+0000 7fc4c4c87700 1 -- 172.21.15.3:0/964133337 >> v2:172.21.15.3:3300/0 conn(0x7fc4c00a0a30 msgr2=0x7fc4c00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:11.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.123+0000 7fc4c4c87700 1 --2- 172.21.15.3:0/964133337 >> v2:172.21.15.3:3300/0 conn(0x7fc4c00a0a30 0x7fc4c00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.123+0000 7fc4c4c87700 1 -- 172.21.15.3:0/964133337 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c00a1fe0 con 0x7fc4c0133fa0 2023-10-03T10:19:11.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.124+0000 7fc4bffff700 1 --2- 172.21.15.3:0/964133337 >> v2:172.21.15.3:3300/0 conn(0x7fc4c00a0a30 0x7fc4c00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:11.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.124+0000 7fc4c4c87700 1 --2- 172.21.15.3:0/964133337 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc4c0133fa0 0x7fc4c0134410 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009dc0 tx=0x7fc4b800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f24a6d201f7795d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:11.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.124+0000 7fc4beffd700 1 -- 172.21.15.3:0/964133337 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b800e070 con 0x7fc4c0133fa0 2023-10-03T10:19:11.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.124+0000 7fc4beffd700 1 -- 172.21.15.3:0/964133337 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b800bae0 con 0x7fc4c0133fa0 2023-10-03T10:19:11.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.124+0000 7fc4beffd700 1 -- 172.21.15.3:0/964133337 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b800be90 con 0x7fc4c0133fa0 2023-10-03T10:19:11.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.125+0000 7fc4c66ea700 1 -- 172.21.15.3:0/964133337 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc4c0133fa0 msgr2=0x7fc4c0134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:11.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.125+0000 7fc4c66ea700 1 --2- 172.21.15.3:0/964133337 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc4c0133fa0 0x7fc4c0134410 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009dc0 tx=0x7fc4b800b330 comp rx=0 tx=0).stop 2023-10-03T10:19:11.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.125+0000 7fc4c66ea700 1 -- 172.21.15.3:0/964133337 shutdown_connections 2023-10-03T10:19:11.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.125+0000 7fc4c66ea700 1 --2- 172.21.15.3:0/964133337 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc4c0133fa0 0x7fc4c0134410 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.125+0000 7fc4c66ea700 1 --2- 172.21.15.3:0/964133337 >> v2:172.21.15.3:3300/0 conn(0x7fc4c00a0a30 0x7fc4c00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.125+0000 7fc4c66ea700 1 --2- 172.21.15.3:0/964133337 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc4c00a1340 0x7fc4c0133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.125+0000 7fc4c66ea700 1 -- 172.21.15.3:0/964133337 >> 172.21.15.3:0/964133337 conn(0x7fc4c00a6550 msgr2=0x7fc4c00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:11.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.125+0000 7fc4c66ea700 1 -- 172.21.15.3:0/964133337 shutdown_connections 2023-10-03T10:19:11.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.125+0000 7fc4c66ea700 1 -- 172.21.15.3:0/964133337 wait complete. 2023-10-03T10:19:11.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.126+0000 7fc4c66ea700 1 Processor -- start 2023-10-03T10:19:11.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.127+0000 7fc4c66ea700 1 -- start start 2023-10-03T10:19:11.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.128+0000 7fc4c66ea700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc4c00a0a30 0x7fc4c0120230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:11.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.128+0000 7fc4c66ea700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc4c00a1340 0x7fc4c0120770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:11.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.128+0000 7fc4c66ea700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4c0133fa0 0x7fc4c0120cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:11.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.128+0000 7fc4c66ea700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c0121e40 con 0x7fc4c0133fa0 2023-10-03T10:19:11.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.128+0000 7fc4c66ea700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c0123090 con 0x7fc4c00a1340 2023-10-03T10:19:11.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.128+0000 7fc4c66ea700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c0123b10 con 0x7fc4c00a0a30 2023-10-03T10:19:11.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.128+0000 7fc4bffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc4c00a0a30 0x7fc4c0120230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:11.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.128+0000 7fc4bffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc4c00a0a30 0x7fc4c0120230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:44058/0 (socket says 172.21.15.3:44058) 2023-10-03T10:19:11.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.128+0000 7fc4bffff700 1 -- 172.21.15.3:0/925992821 learned_addr learned my addr 172.21.15.3:0/925992821 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:11.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.128+0000 7fc4bf7fe700 1 --2- 172.21.15.3:0/925992821 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc4c00a1340 0x7fc4c0120770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:11.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.128+0000 7fc4c4c87700 1 --2- 172.21.15.3:0/925992821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4c0133fa0 0x7fc4c0120cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:11.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.128+0000 7fc4bffff700 1 -- 172.21.15.3:0/925992821 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc4c00a1340 msgr2=0x7fc4c0120770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:11.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.128+0000 7fc4bffff700 1 --2- 172.21.15.3:0/925992821 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc4c00a1340 0x7fc4c0120770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.128+0000 7fc4bffff700 1 -- 172.21.15.3:0/925992821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4c0133fa0 msgr2=0x7fc4c0120cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:11.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.128+0000 7fc4bffff700 1 --2- 172.21.15.3:0/925992821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4c0133fa0 0x7fc4c0120cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.128+0000 7fc4bffff700 1 -- 172.21.15.3:0/925992821 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b0009550 con 0x7fc4c00a0a30 2023-10-03T10:19:11.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.128+0000 7fc4c4c87700 1 --2- 172.21.15.3:0/925992821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4c0133fa0 0x7fc4c0120cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:11.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.128+0000 7fc4bffff700 1 --2- 172.21.15.3:0/925992821 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc4c00a0a30 0x7fc4c0120230 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000ebe0 tx=0x7fc4b0009e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:11.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.129+0000 7fc4bcff9700 1 -- 172.21.15.3:0/925992821 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b000c8b0 con 0x7fc4c00a0a30 2023-10-03T10:19:11.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.129+0000 7fc4bcff9700 1 -- 172.21.15.3:0/925992821 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b000ca10 con 0x7fc4c00a0a30 2023-10-03T10:19:11.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.129+0000 7fc4bcff9700 1 -- 172.21.15.3:0/925992821 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b000cd80 con 0x7fc4c00a0a30 2023-10-03T10:19:11.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.129+0000 7fc4c66ea700 1 -- 172.21.15.3:0/925992821 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b8009a70 con 0x7fc4c00a0a30 2023-10-03T10:19:11.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.129+0000 7fc4c66ea700 1 -- 172.21.15.3:0/925992821 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c00a3480 con 0x7fc4c00a0a30 2023-10-03T10:19:11.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.130+0000 7fc4bcff9700 1 -- 172.21.15.3:0/925992821 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc4b0013470 con 0x7fc4c00a0a30 2023-10-03T10:19:11.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.130+0000 7fc4c66ea700 1 -- 172.21.15.3:0/925992821 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c00a0220 con 0x7fc4c00a0a30 2023-10-03T10:19:11.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.130+0000 7fc4bcff9700 1 -- 172.21.15.3:0/925992821 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc4b003fd60 con 0x7fc4c00a0a30 2023-10-03T10:19:11.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.139+0000 7fc4bcff9700 1 -- 172.21.15.3:0/925992821 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc4b001b3b0 con 0x7fc4c00a0a30 2023-10-03T10:19:11.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.264+0000 7f6b57a4c700 1 -- 172.21.15.3:0/1200627412 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6b30005590 con 0x7f6b5009d6f0 2023-10-03T10:19:11.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.265+0000 7f6b45ffb700 1 -- 172.21.15.3:0/1200627412 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6b4c00e360 con 0x7f6b5009d6f0 2023-10-03T10:19:11.265 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:11.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.267+0000 7f6b57a4c700 1 -- 172.21.15.3:0/1200627412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b5009d6f0 msgr2=0x7f6b5011e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:11.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.267+0000 7f6b57a4c700 1 --2- 172.21.15.3:0/1200627412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b5009d6f0 0x7f6b5011e0e0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c000c00 tx=0x7f6b4c004750 comp rx=0 tx=0).stop 2023-10-03T10:19:11.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.267+0000 7f6b57a4c700 1 -- 172.21.15.3:0/1200627412 shutdown_connections 2023-10-03T10:19:11.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.267+0000 7f6b57a4c700 1 --2- 172.21.15.3:0/1200627412 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6b5009e410 0x7f6b5011eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.267+0000 7f6b57a4c700 1 --2- 172.21.15.3:0/1200627412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b5009d6f0 0x7f6b5011e0e0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.267+0000 7f6b57a4c700 1 --2- 172.21.15.3:0/1200627412 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6b5009e000 0x7f6b5011e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.267+0000 7f6b57a4c700 1 -- 172.21.15.3:0/1200627412 >> 172.21.15.3:0/1200627412 conn(0x7f6b50066f50 msgr2=0x7f6b5006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:11.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.267+0000 7f6b57a4c700 1 -- 172.21.15.3:0/1200627412 shutdown_connections 2023-10-03T10:19:11.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.267+0000 7f6b57a4c700 1 -- 172.21.15.3:0/1200627412 wait complete. 2023-10-03T10:19:11.286 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:11.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.338+0000 7f1cb5e84700 1 -- 172.21.15.3:0/949420806 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1cb0074ea0 con 0x7f1cb0133e30 2023-10-03T10:19:11.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.338+0000 7f1c97fff700 1 -- 172.21.15.3:0/949420806 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1ca4015070 con 0x7f1cb0133e30 2023-10-03T10:19:11.339 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:11.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.341+0000 7f1cb5e84700 1 -- 172.21.15.3:0/949420806 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1cb0133e30 msgr2=0x7f1cb009c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:11.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.341+0000 7f1cb5e84700 1 --2- 172.21.15.3:0/949420806 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1cb0133e30 0x7f1cb009c9c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4013000 tx=0x7f1ca4004750 comp rx=0 tx=0).stop 2023-10-03T10:19:11.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.341+0000 7f1cb5e84700 1 -- 172.21.15.3:0/949420806 shutdown_connections 2023-10-03T10:19:11.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.341+0000 7f1cb5e84700 1 --2- 172.21.15.3:0/949420806 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cb0134260 0x7f1cb009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.341+0000 7f1cb5e84700 1 --2- 172.21.15.3:0/949420806 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1cb0133420 0x7f1cb009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.341+0000 7f1cb5e84700 1 --2- 172.21.15.3:0/949420806 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1cb0133e30 0x7f1cb009c9c0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.341+0000 7f1cb5e84700 1 -- 172.21.15.3:0/949420806 >> 172.21.15.3:0/949420806 conn(0x7f1cb00a4210 msgr2=0x7f1cb00a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:11.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.341+0000 7f1cb5e84700 1 -- 172.21.15.3:0/949420806 shutdown_connections 2023-10-03T10:19:11.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.341+0000 7f1cb5e84700 1 -- 172.21.15.3:0/949420806 wait complete. 2023-10-03T10:19:11.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.344+0000 7f1c45bd3700 1 -- 172.21.15.3:0/319964063 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1c20005590 con 0x7f1c4009f650 2023-10-03T10:19:11.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.344+0000 7f1c27fff700 1 -- 172.21.15.3:0/319964063 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1c38001cb0 con 0x7f1c4009f650 2023-10-03T10:19:11.344 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:11.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.346+0000 7f1c45bd3700 1 -- 172.21.15.3:0/319964063 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c4009f650 msgr2=0x7f1c401c3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:11.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.346+0000 7f1c45bd3700 1 --2- 172.21.15.3:0/319964063 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c4009f650 0x7f1c401c3e20 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800ccb0 tx=0x7f1c3800a060 comp rx=0 tx=0).stop 2023-10-03T10:19:11.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.347+0000 7f1c45bd3700 1 -- 172.21.15.3:0/319964063 shutdown_connections 2023-10-03T10:19:11.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.347+0000 7f1c45bd3700 1 --2- 172.21.15.3:0/319964063 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1c4009f650 0x7f1c401c3e20 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.347+0000 7f1c45bd3700 1 --2- 172.21.15.3:0/319964063 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1c4009e930 0x7f1c401c0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.347+0000 7f1c45bd3700 1 --2- 172.21.15.3:0/319964063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c4009f240 0x7f1c401c1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.347+0000 7f1c45bd3700 1 -- 172.21.15.3:0/319964063 >> 172.21.15.3:0/319964063 conn(0x7f1c400a4210 msgr2=0x7f1c401b6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:11.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.347+0000 7f1c45bd3700 1 -- 172.21.15.3:0/319964063 shutdown_connections 2023-10-03T10:19:11.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.347+0000 7f1c45bd3700 1 -- 172.21.15.3:0/319964063 wait complete. 2023-10-03T10:19:11.356 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:11.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.356+0000 7fc4c66ea700 1 -- 172.21.15.3:0/925992821 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc4c0074ea0 con 0x7fc4c00a0a30 2023-10-03T10:19:11.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.357+0000 7fc4bcff9700 1 -- 172.21.15.3:0/925992821 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc4b0017030 con 0x7fc4c00a0a30 2023-10-03T10:19:11.357 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:11.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.359+0000 7fc4c66ea700 1 -- 172.21.15.3:0/925992821 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc4c00a0a30 msgr2=0x7fc4c0120230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:11.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.359+0000 7fc4c66ea700 1 --2- 172.21.15.3:0/925992821 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc4c00a0a30 0x7fc4c0120230 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000ebe0 tx=0x7fc4b0009e70 comp rx=0 tx=0).stop 2023-10-03T10:19:11.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.359+0000 7fc4c66ea700 1 -- 172.21.15.3:0/925992821 shutdown_connections 2023-10-03T10:19:11.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.359+0000 7fc4c66ea700 1 --2- 172.21.15.3:0/925992821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4c0133fa0 0x7fc4c0120cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.359+0000 7fc4c66ea700 1 --2- 172.21.15.3:0/925992821 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc4c00a0a30 0x7fc4c0120230 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.359+0000 7fc4c66ea700 1 --2- 172.21.15.3:0/925992821 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc4c00a1340 0x7fc4c0120770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:11.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.359+0000 7fc4c66ea700 1 -- 172.21.15.3:0/925992821 >> 172.21.15.3:0/925992821 conn(0x7fc4c00a6550 msgr2=0x7fc4c009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:11.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.360+0000 7fc4c66ea700 1 -- 172.21.15.3:0/925992821 shutdown_connections 2023-10-03T10:19:11.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:11.360+0000 7fc4c66ea700 1 -- 172.21.15.3:0/925992821 wait complete. 2023-10-03T10:19:11.362 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:11.373 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:12.287 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:12.357 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:12.363 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:12.374 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:12.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.440+0000 7f29db18b700 1 Processor -- start 2023-10-03T10:19:12.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.440+0000 7f29db18b700 1 -- start start 2023-10-03T10:19:12.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.441+0000 7f29db18b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29d4131810 0x7f29d4131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:12.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.441+0000 7f29db18b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f29d4099a40 0x7f29d4099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:12.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.441+0000 7f29db18b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d409a850 con 0x7f29d409a3b0 2023-10-03T10:19:12.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.441+0000 7f29db18b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29d409b580 con 0x7f29d4099a40 2023-10-03T10:19:12.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.441+0000 7f29db18b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29d41313c0 con 0x7f29d4131810 2023-10-03T10:19:12.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.441+0000 7f29d8f27700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29d4131810 0x7f29d4131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:12.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.441+0000 7f29d8f27700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29d4131810 0x7f29d4131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:44070/0 (socket says 172.21.15.3:44070) 2023-10-03T10:19:12.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.441+0000 7f29d8f27700 1 -- 172.21.15.3:0/3266659782 learned_addr learned my addr 172.21.15.3:0/3266659782 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:12.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.441+0000 7f29d3fff700 1 --2- 172.21.15.3:0/3266659782 >> v2:172.21.15.3:3300/0 conn(0x7f29d4099a40 0x7f29d4099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:12.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.442+0000 7f29d8f27700 1 -- 172.21.15.3:0/3266659782 >> v2:172.21.15.3:3300/0 conn(0x7f29d4099a40 msgr2=0x7f29d4099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:12.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.442+0000 7f29d8f27700 1 --2- 172.21.15.3:0/3266659782 >> v2:172.21.15.3:3300/0 conn(0x7f29d4099a40 0x7f29d4099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.442+0000 7f29d8f27700 1 -- 172.21.15.3:0/3266659782 >> v1:172.21.15.3:6789/0 conn(0x7f29d409a3b0 legacy=0x7f29d4138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:12.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.442+0000 7f29d8f27700 1 -- 172.21.15.3:0/3266659782 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d406e8c0 con 0x7f29d4131810 2023-10-03T10:19:12.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.442+0000 7f29d8f27700 1 --2- 172.21.15.3:0/3266659782 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29d4131810 0x7f29d4131be0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f29c8009920 tx=0x7f29c8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d436576526715c5c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:12.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.442+0000 7f29d37fe700 1 -- 172.21.15.3:0/3266659782 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c8011830 con 0x7f29d4131810 2023-10-03T10:19:12.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.442+0000 7f29d37fe700 1 -- 172.21.15.3:0/3266659782 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c8011990 con 0x7f29d4131810 2023-10-03T10:19:12.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.442+0000 7f29d37fe700 1 -- 172.21.15.3:0/3266659782 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c8011d20 con 0x7f29d4131810 2023-10-03T10:19:12.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.443+0000 7f29db18b700 1 -- 172.21.15.3:0/3266659782 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29d4131810 msgr2=0x7f29d4131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:12.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.443+0000 7f29db18b700 1 --2- 172.21.15.3:0/3266659782 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29d4131810 0x7f29d4131be0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f29c8009920 tx=0x7f29c8009c30 comp rx=0 tx=0).stop 2023-10-03T10:19:12.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.443+0000 7f29db18b700 1 -- 172.21.15.3:0/3266659782 shutdown_connections 2023-10-03T10:19:12.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.443+0000 7f29db18b700 1 --2- 172.21.15.3:0/3266659782 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29d4131810 0x7f29d4131be0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.443+0000 7f29db18b700 1 --2- 172.21.15.3:0/3266659782 >> v2:172.21.15.3:3300/0 conn(0x7f29d4099a40 0x7f29d4099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.443+0000 7f29db18b700 1 -- 172.21.15.3:0/3266659782 >> 172.21.15.3:0/3266659782 conn(0x7f29d40a2aa0 msgr2=0x7f29d40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:12.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.443+0000 7f29db18b700 1 -- 172.21.15.3:0/3266659782 shutdown_connections 2023-10-03T10:19:12.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.443+0000 7f29db18b700 1 -- 172.21.15.3:0/3266659782 wait complete. 2023-10-03T10:19:12.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.444+0000 7f29db18b700 1 Processor -- start 2023-10-03T10:19:12.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.445+0000 7f29db18b700 1 -- start start 2023-10-03T10:19:12.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.445+0000 7f29db18b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29d4099a40 0x7f29d406f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:12.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.445+0000 7f29db18b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d409a3b0 0x7f29d406fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:12.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.445+0000 7f29db18b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29d4131810 0x7f29d4070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:12.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.446+0000 7f29db18b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d411f040 con 0x7f29d409a3b0 2023-10-03T10:19:12.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.446+0000 7f29db18b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d411f240 con 0x7f29d4099a40 2023-10-03T10:19:12.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.446+0000 7f29db18b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29d411e130 con 0x7f29d4131810 2023-10-03T10:19:12.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.446+0000 7f29d8f27700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29d4099a40 0x7f29d406f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:12.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.446+0000 7f29d8f27700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29d4099a40 0x7f29d406f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41780/0 (socket says 172.21.15.3:41780) 2023-10-03T10:19:12.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.446+0000 7f29d8f27700 1 -- 172.21.15.3:0/3748619919 learned_addr learned my addr 172.21.15.3:0/3748619919 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:12.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.446+0000 7f29d3fff700 1 --2- 172.21.15.3:0/3748619919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d409a3b0 0x7f29d406fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:12.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.446+0000 7f29d9728700 1 --2- 172.21.15.3:0/3748619919 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29d4131810 0x7f29d4070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:12.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.446+0000 7f29d3fff700 1 -- 172.21.15.3:0/3748619919 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29d4131810 msgr2=0x7f29d4070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:12.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.446+0000 7f29d3fff700 1 --2- 172.21.15.3:0/3748619919 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29d4131810 0x7f29d4070090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.446+0000 7f29d3fff700 1 -- 172.21.15.3:0/3748619919 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29d4099a40 msgr2=0x7f29d406f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:12.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.446+0000 7f29d3fff700 1 --2- 172.21.15.3:0/3748619919 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29d4099a40 0x7f29d406f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.446+0000 7f29d3fff700 1 -- 172.21.15.3:0/3748619919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c8009550 con 0x7f29d409a3b0 2023-10-03T10:19:12.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.446+0000 7f29d8f27700 1 --2- 172.21.15.3:0/3748619919 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29d4099a40 0x7f29d406f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:12.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.446+0000 7f29d9728700 1 --2- 172.21.15.3:0/3748619919 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29d4131810 0x7f29d4070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:12.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.447+0000 7f29d3fff700 1 --2- 172.21.15.3:0/3748619919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d409a3b0 0x7f29d406fb50 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f29c000e8d0 tx=0x7f29c000ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:12.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.447+0000 7f29d17fa700 1 -- 172.21.15.3:0/3748619919 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c0013680 con 0x7f29d409a3b0 2023-10-03T10:19:12.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.447+0000 7f29d17fa700 1 -- 172.21.15.3:0/3748619919 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c00137e0 con 0x7f29d409a3b0 2023-10-03T10:19:12.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.447+0000 7f29db18b700 1 -- 172.21.15.3:0/3748619919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d412cbb0 con 0x7f29d409a3b0 2023-10-03T10:19:12.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.447+0000 7f29d17fa700 1 -- 172.21.15.3:0/3748619919 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c0013b50 con 0x7f29d409a3b0 2023-10-03T10:19:12.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.447+0000 7f29db18b700 1 -- 172.21.15.3:0/3748619919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d406eb00 con 0x7f29d409a3b0 2023-10-03T10:19:12.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.448+0000 7f29d17fa700 1 -- 172.21.15.3:0/3748619919 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f29c0013cb0 con 0x7f29d409a3b0 2023-10-03T10:19:12.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.448+0000 7f29d17fa700 1 -- 172.21.15.3:0/3748619919 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f29c003f9c0 con 0x7f29d409a3b0 2023-10-03T10:19:12.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.448+0000 7f29db18b700 1 -- 172.21.15.3:0/3748619919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29d4074ea0 con 0x7f29d409a3b0 2023-10-03T10:19:12.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.458+0000 7f29d17fa700 1 -- 172.21.15.3:0/3748619919 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f29c0017080 con 0x7f29d409a3b0 2023-10-03T10:19:12.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.505+0000 7f111b965700 1 Processor -- start 2023-10-03T10:19:12.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.505+0000 7f111b965700 1 -- start start 2023-10-03T10:19:12.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.506+0000 7f111b965700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f111406b250 0x7f1114136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:12.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.506+0000 7f111b965700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1114136d80 0x7f1114139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:12.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.506+0000 7f111b965700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f111409ece0 con 0x7f111406a770 2023-10-03T10:19:12.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.506+0000 7f111b965700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f111406b640 con 0x7f111406b250 2023-10-03T10:19:12.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.506+0000 7f111b965700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f111409e760 con 0x7f1114136d80 2023-10-03T10:19:12.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.506+0000 7f1118f00700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f111406b250 0x7f1114136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:12.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.506+0000 7f1118f00700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f111406b250 0x7f1114136840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41782/0 (socket says 172.21.15.3:41782) 2023-10-03T10:19:12.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.506+0000 7f1118f00700 1 -- 172.21.15.3:0/1203505639 learned_addr learned my addr 172.21.15.3:0/1203505639 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:12.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.506+0000 7f1119701700 1 --1- 172.21.15.3:0/1203505639 >> v1:172.21.15.3:6789/0 conn(0x7f111406a770 0x7f111406ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53734/0 (socket says 172.21.15.3:53734) 2023-10-03T10:19:12.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.506+0000 7f1119f02700 1 --2- 172.21.15.3:0/1203505639 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1114136d80 0x7f1114139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:12.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.506+0000 7f1118f00700 1 -- 172.21.15.3:0/1203505639 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1114136d80 msgr2=0x7f1114139160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:12.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.506+0000 7f110bfff700 1 -- 172.21.15.3:0/1203505639 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3500779611 0 0) 0x7f111409ece0 con 0x7f111406a770 2023-10-03T10:19:12.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.506+0000 7f1118f00700 1 --2- 172.21.15.3:0/1203505639 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1114136d80 0x7f1114139160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.506+0000 7f1118f00700 1 -- 172.21.15.3:0/1203505639 >> v1:172.21.15.3:6789/0 conn(0x7f111406a770 legacy=0x7f111406ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:12.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.506+0000 7f1118f00700 1 -- 172.21.15.3:0/1203505639 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111409e040 con 0x7f111406b250 2023-10-03T10:19:12.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.506+0000 7f1119f02700 1 --2- 172.21.15.3:0/1203505639 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1114136d80 0x7f1114139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:12.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.507+0000 7f1118f00700 1 --2- 172.21.15.3:0/1203505639 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f111406b250 0x7f1114136840 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f1104009920 tx=0x7f1104009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c902882d666e0454 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:12.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.507+0000 7f110bfff700 1 -- 172.21.15.3:0/1203505639 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1104011830 con 0x7f111406b250 2023-10-03T10:19:12.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.507+0000 7f110bfff700 1 -- 172.21.15.3:0/1203505639 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1104011990 con 0x7f111406b250 2023-10-03T10:19:12.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.507+0000 7f110bfff700 1 -- 172.21.15.3:0/1203505639 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1104011d20 con 0x7f111406b250 2023-10-03T10:19:12.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.507+0000 7f111b965700 1 -- 172.21.15.3:0/1203505639 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f111406b250 msgr2=0x7f1114136840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:12.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.507+0000 7f111b965700 1 --2- 172.21.15.3:0/1203505639 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f111406b250 0x7f1114136840 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f1104009920 tx=0x7f1104009c30 comp rx=0 tx=0).stop 2023-10-03T10:19:12.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.508+0000 7f111b965700 1 -- 172.21.15.3:0/1203505639 shutdown_connections 2023-10-03T10:19:12.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.508+0000 7f111b965700 1 --2- 172.21.15.3:0/1203505639 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1114136d80 0x7f1114139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.508+0000 7f111b965700 1 --2- 172.21.15.3:0/1203505639 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f111406b250 0x7f1114136840 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.508+0000 7f111b965700 1 -- 172.21.15.3:0/1203505639 >> 172.21.15.3:0/1203505639 conn(0x7f1114066f50 msgr2=0x7f111409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:12.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.508+0000 7f111b965700 1 -- 172.21.15.3:0/1203505639 shutdown_connections 2023-10-03T10:19:12.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.508+0000 7f111b965700 1 -- 172.21.15.3:0/1203505639 wait complete. 2023-10-03T10:19:12.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.509+0000 7f111b965700 1 Processor -- start 2023-10-03T10:19:12.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.509+0000 7f111b965700 1 -- start start 2023-10-03T10:19:12.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.509+0000 7f111b965700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f111406a770 0x7f111409ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:12.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.509+0000 7f111b965700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1114136d80 0x7f111409b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:12.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.509+0000 7f111b965700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11141c8310 0x7f11141c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:12.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.509+0000 7f111b965700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1114131490 con 0x7f11141c8310 2023-10-03T10:19:12.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.509+0000 7f111b965700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1114121200 con 0x7f1114136d80 2023-10-03T10:19:12.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.509+0000 7f111b965700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f111409b800 con 0x7f111406a770 2023-10-03T10:19:12.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.509+0000 7f1119701700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f111406a770 0x7f111409ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:12.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.509+0000 7f1119701700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f111406a770 0x7f111409ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:44102/0 (socket says 172.21.15.3:44102) 2023-10-03T10:19:12.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.509+0000 7f1119701700 1 -- 172.21.15.3:0/2882154422 learned_addr learned my addr 172.21.15.3:0/2882154422 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:12.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.510+0000 7f1118f00700 1 --2- 172.21.15.3:0/2882154422 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1114136d80 0x7f111409b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:12.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.510+0000 7f1119f02700 1 --2- 172.21.15.3:0/2882154422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11141c8310 0x7f11141c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:12.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.510+0000 7f1119701700 1 -- 172.21.15.3:0/2882154422 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1114136d80 msgr2=0x7f111409b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:12.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.510+0000 7f1119701700 1 --2- 172.21.15.3:0/2882154422 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1114136d80 0x7f111409b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.510+0000 7f1119701700 1 -- 172.21.15.3:0/2882154422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11141c8310 msgr2=0x7f11141c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:12.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.510+0000 7f1119701700 1 --2- 172.21.15.3:0/2882154422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11141c8310 0x7f11141c8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.510+0000 7f1119701700 1 -- 172.21.15.3:0/2882154422 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f110c009a70 con 0x7f111406a770 2023-10-03T10:19:12.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.510+0000 7f1119f02700 1 --2- 172.21.15.3:0/2882154422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11141c8310 0x7f11141c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:12.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.510+0000 7f1119701700 1 --2- 172.21.15.3:0/2882154422 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f111406a770 0x7f111409ad80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f111000c520 tx=0x7f111000c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:12.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.510+0000 7f1109ffb700 1 -- 172.21.15.3:0/2882154422 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1110004c80 con 0x7f111406a770 2023-10-03T10:19:12.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.510+0000 7f1109ffb700 1 -- 172.21.15.3:0/2882154422 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1110004de0 con 0x7f111406a770 2023-10-03T10:19:12.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.510+0000 7f111b965700 1 -- 172.21.15.3:0/2882154422 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1104009550 con 0x7f111406a770 2023-10-03T10:19:12.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.510+0000 7f1109ffb700 1 -- 172.21.15.3:0/2882154422 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1110007310 con 0x7f111406a770 2023-10-03T10:19:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.511+0000 7f111b965700 1 -- 172.21.15.3:0/2882154422 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1114120500 con 0x7f111406a770 2023-10-03T10:19:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.512+0000 7f1109ffb700 1 -- 172.21.15.3:0/2882154422 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1110007470 con 0x7f111406a770 2023-10-03T10:19:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.512+0000 7f1109ffb700 1 -- 172.21.15.3:0/2882154422 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1110040d20 con 0x7f111406a770 2023-10-03T10:19:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.512+0000 7f111b965700 1 -- 172.21.15.3:0/2882154422 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f111406e940 con 0x7f111406a770 2023-10-03T10:19:12.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.518+0000 7f1109ffb700 1 -- 172.21.15.3:0/2882154422 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1110044060 con 0x7f111406a770 2023-10-03T10:19:12.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.552+0000 7fde2d64d700 1 Processor -- start 2023-10-03T10:19:12.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.552+0000 7fde2d64d700 1 -- start start 2023-10-03T10:19:12.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.553+0000 7fde2d64d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fde2809e930 0x7fde2809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:12.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.553+0000 7fde2d64d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde2809f650 0x7fde28131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:12.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.553+0000 7fde2d64d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde28070940 con 0x7fde2809f240 2023-10-03T10:19:12.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.553+0000 7fde2d64d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde2806fd60 con 0x7fde2809e930 2023-10-03T10:19:12.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.553+0000 7fde2d64d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde2806fa80 con 0x7fde2809f650 2023-10-03T10:19:12.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.553+0000 7fde277fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde2809f650 0x7fde28131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:12.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.553+0000 7fde26ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fde2809e930 0x7fde2809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:12.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.553+0000 7fde277fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde2809f650 0x7fde28131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:44114/0 (socket says 172.21.15.3:44114) 2023-10-03T10:19:12.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.553+0000 7fde277fe700 1 -- 172.21.15.3:0/1145043021 learned_addr learned my addr 172.21.15.3:0/1145043021 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:12.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.553+0000 7fde277fe700 1 -- 172.21.15.3:0/1145043021 >> v2:172.21.15.3:3300/0 conn(0x7fde2809e930 msgr2=0x7fde2809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:12.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.553+0000 7fde277fe700 1 --2- 172.21.15.3:0/1145043021 >> v2:172.21.15.3:3300/0 conn(0x7fde2809e930 0x7fde2809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.554+0000 7fde277fe700 1 -- 172.21.15.3:0/1145043021 >> v1:172.21.15.3:6789/0 conn(0x7fde2809f240 legacy=0x7fde281317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:12.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.554+0000 7fde277fe700 1 -- 172.21.15.3:0/1145043021 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde2809fb60 con 0x7fde2809f650 2023-10-03T10:19:12.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.554+0000 7fde26ffd700 1 --2- 172.21.15.3:0/1145043021 >> v2:172.21.15.3:3300/0 conn(0x7fde2809e930 0x7fde2809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:12.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.554+0000 7fde277fe700 1 --2- 172.21.15.3:0/1145043021 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde2809f650 0x7fde28131f00 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fde20009e50 tx=0x7fde2000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90d66204922302e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:12.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.554+0000 7fde25ffb700 1 -- 172.21.15.3:0/1145043021 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2000e070 con 0x7fde2809f650 2023-10-03T10:19:12.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.554+0000 7fde25ffb700 1 -- 172.21.15.3:0/1145043021 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2000b910 con 0x7fde2809f650 2023-10-03T10:19:12.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.554+0000 7fde25ffb700 1 -- 172.21.15.3:0/1145043021 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2000bcc0 con 0x7fde2809f650 2023-10-03T10:19:12.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.555+0000 7fde2d64d700 1 -- 172.21.15.3:0/1145043021 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde2809f650 msgr2=0x7fde28131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:12.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.555+0000 7fde2d64d700 1 --2- 172.21.15.3:0/1145043021 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde2809f650 0x7fde28131f00 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fde20009e50 tx=0x7fde2000b330 comp rx=0 tx=0).stop 2023-10-03T10:19:12.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.555+0000 7fde2d64d700 1 -- 172.21.15.3:0/1145043021 shutdown_connections 2023-10-03T10:19:12.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.555+0000 7fde2d64d700 1 --2- 172.21.15.3:0/1145043021 >> v2:172.21.15.3:3300/0 conn(0x7fde2809e930 0x7fde2809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.555+0000 7fde2d64d700 1 --2- 172.21.15.3:0/1145043021 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde2809f650 0x7fde28131f00 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.555+0000 7fde2d64d700 1 -- 172.21.15.3:0/1145043021 >> 172.21.15.3:0/1145043021 conn(0x7fde280a4210 msgr2=0x7fde280a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:12.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.555+0000 7fde2d64d700 1 -- 172.21.15.3:0/1145043021 shutdown_connections 2023-10-03T10:19:12.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.555+0000 7fde2d64d700 1 -- 172.21.15.3:0/1145043021 wait complete. 2023-10-03T10:19:12.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.556+0000 7fde2d64d700 1 Processor -- start 2023-10-03T10:19:12.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.556+0000 7fde2d64d700 1 -- start start 2023-10-03T10:19:12.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.557+0000 7fde2d64d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fde2809e930 0x7fde2806b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:12.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.557+0000 7fde2d64d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde2809f240 0x7fde2806bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:12.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.557+0000 7fde2d64d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde2809f650 0x7fde2806c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:12.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.557+0000 7fde2d64d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2806cd80 con 0x7fde2809f650 2023-10-03T10:19:12.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.557+0000 7fde2d64d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde281b72c0 con 0x7fde2809e930 2023-10-03T10:19:12.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.557+0000 7fde2d64d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde281a9470 con 0x7fde2809f240 2023-10-03T10:19:12.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.557+0000 7fde277fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde2809f650 0x7fde2806c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:12.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.557+0000 7fde267fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde2809f240 0x7fde2806bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:12.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.557+0000 7fde277fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde2809f650 0x7fde2806c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46952/0 (socket says 172.21.15.3:46952) 2023-10-03T10:19:12.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.557+0000 7fde267fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde2809f240 0x7fde2806bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:44128/0 (socket says 172.21.15.3:44128) 2023-10-03T10:19:12.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.557+0000 7fde277fe700 1 -- 172.21.15.3:0/3265553710 learned_addr learned my addr 172.21.15.3:0/3265553710 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:12.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.557+0000 7fde277fe700 1 -- 172.21.15.3:0/3265553710 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde2809f240 msgr2=0x7fde2806bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:12.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.557+0000 7fde277fe700 1 --2- 172.21.15.3:0/3265553710 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde2809f240 0x7fde2806bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.558+0000 7fde277fe700 1 -- 172.21.15.3:0/3265553710 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fde2809e930 msgr2=0x7fde2806b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:12.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.558+0000 7fde277fe700 1 --2- 172.21.15.3:0/3265553710 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fde2809e930 0x7fde2806b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.558+0000 7fde277fe700 1 -- 172.21.15.3:0/3265553710 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde180094a0 con 0x7fde2809f650 2023-10-03T10:19:12.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.558+0000 7fde277fe700 1 --2- 172.21.15.3:0/3265553710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde2809f650 0x7fde2806c330 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fde2000b680 tx=0x7fde20004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:12.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.558+0000 7fde0f7fe700 1 -- 172.21.15.3:0/3265553710 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde200048e0 con 0x7fde2809f650 2023-10-03T10:19:12.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.558+0000 7fde2d64d700 1 -- 172.21.15.3:0/3265553710 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde20009b30 con 0x7fde2809f650 2023-10-03T10:19:12.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.558+0000 7fde0f7fe700 1 -- 172.21.15.3:0/3265553710 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde200092e0 con 0x7fde2809f650 2023-10-03T10:19:12.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.558+0000 7fde0f7fe700 1 -- 172.21.15.3:0/3265553710 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2001c660 con 0x7fde2809f650 2023-10-03T10:19:12.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.558+0000 7fde2d64d700 1 -- 172.21.15.3:0/3265553710 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde280a0190 con 0x7fde2809f650 2023-10-03T10:19:12.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.559+0000 7fde0f7fe700 1 -- 172.21.15.3:0/3265553710 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fde2001c7c0 con 0x7fde2809f650 2023-10-03T10:19:12.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.559+0000 7fde0f7fe700 1 -- 172.21.15.3:0/3265553710 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fde20041350 con 0x7fde2809f650 2023-10-03T10:19:12.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.559+0000 7fde2d64d700 1 -- 172.21.15.3:0/3265553710 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde281a91a0 con 0x7fde2809f650 2023-10-03T10:19:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.565+0000 7fde0f7fe700 1 -- 172.21.15.3:0/3265553710 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fde200257d0 con 0x7fde2809f650 2023-10-03T10:19:12.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.599+0000 7fc9c5879700 1 Processor -- start 2023-10-03T10:19:12.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.599+0000 7fc9c5879700 1 -- start start 2023-10-03T10:19:12.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.600+0000 7fc9c5879700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc9c00a0a30 0x7fc9c00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:12.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.600+0000 7fc9c5879700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9c0138d80 0x7fc9c013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:12.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.600+0000 7fc9c5879700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9c00a2a70 con 0x7fc9c009d630 2023-10-03T10:19:12.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.600+0000 7fc9c5879700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9c00a01d0 con 0x7fc9c00a0a30 2023-10-03T10:19:12.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.600+0000 7fc9c5879700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c009e2b0 con 0x7fc9c0138d80 2023-10-03T10:19:12.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.600+0000 7fc9bf7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9c0138d80 0x7fc9c013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:12.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.600+0000 7fc9bf7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9c0138d80 0x7fc9c013b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41816/0 (socket says 172.21.15.3:41816) 2023-10-03T10:19:12.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.600+0000 7fc9bf7fe700 1 -- 172.21.15.3:0/2862570851 learned_addr learned my addr 172.21.15.3:0/2862570851 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:12.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.600+0000 7fc9bf7fe700 1 -- 172.21.15.3:0/2862570851 >> v2:172.21.15.3:3300/0 conn(0x7fc9c00a0a30 msgr2=0x7fc9c00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:12.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.600+0000 7fc9bf7fe700 1 --2- 172.21.15.3:0/2862570851 >> v2:172.21.15.3:3300/0 conn(0x7fc9c00a0a30 0x7fc9c00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.600+0000 7fc9bf7fe700 1 -- 172.21.15.3:0/2862570851 >> v1:172.21.15.3:6789/0 conn(0x7fc9c009d630 legacy=0x7fc9c00a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:12.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.600+0000 7fc9bf7fe700 1 -- 172.21.15.3:0/2862570851 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c009dc50 con 0x7fc9c0138d80 2023-10-03T10:19:12.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.601+0000 7fc9bf7fe700 1 --2- 172.21.15.3:0/2862570851 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9c0138d80 0x7fc9c013b160 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8009e50 tx=0x7fc9b800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a955811733c8d088 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:12.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.601+0000 7fc9bdffb700 1 -- 172.21.15.3:0/2862570851 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b800e070 con 0x7fc9c0138d80 2023-10-03T10:19:12.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.601+0000 7fc9bdffb700 1 -- 172.21.15.3:0/2862570851 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b8003a20 con 0x7fc9c0138d80 2023-10-03T10:19:12.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.601+0000 7fc9bdffb700 1 -- 172.21.15.3:0/2862570851 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8003dd0 con 0x7fc9c0138d80 2023-10-03T10:19:12.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.602+0000 7fc9c5879700 1 -- 172.21.15.3:0/2862570851 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9c0138d80 msgr2=0x7fc9c013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:12.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.602+0000 7fc9c5879700 1 --2- 172.21.15.3:0/2862570851 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9c0138d80 0x7fc9c013b160 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8009e50 tx=0x7fc9b800b330 comp rx=0 tx=0).stop 2023-10-03T10:19:12.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.602+0000 7fc9c5879700 1 -- 172.21.15.3:0/2862570851 shutdown_connections 2023-10-03T10:19:12.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.602+0000 7fc9c5879700 1 --2- 172.21.15.3:0/2862570851 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9c0138d80 0x7fc9c013b160 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.602+0000 7fc9c5879700 1 --2- 172.21.15.3:0/2862570851 >> v2:172.21.15.3:3300/0 conn(0x7fc9c00a0a30 0x7fc9c00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.602+0000 7fc9c5879700 1 -- 172.21.15.3:0/2862570851 >> 172.21.15.3:0/2862570851 conn(0x7fc9c00a3050 msgr2=0x7fc9c00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:12.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.602+0000 7fc9c5879700 1 -- 172.21.15.3:0/2862570851 shutdown_connections 2023-10-03T10:19:12.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.602+0000 7fc9c5879700 1 -- 172.21.15.3:0/2862570851 wait complete. 2023-10-03T10:19:12.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.603+0000 7fc9c5879700 1 Processor -- start 2023-10-03T10:19:12.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.603+0000 7fc9c5879700 1 -- start start 2023-10-03T10:19:12.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.603+0000 7fc9c5879700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9c009d630 0x7fc9c01336e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:12.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.603+0000 7fc9c5879700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9c00a0a30 0x7fc9c0133c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:12.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.603+0000 7fc9c5879700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9c0138d80 0x7fc9c0134160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:12.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.603+0000 7fc9c5879700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c0134bb0 con 0x7fc9c009d630 2023-10-03T10:19:12.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.604+0000 7fc9c5879700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c01365b0 con 0x7fc9c00a0a30 2023-10-03T10:19:12.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.604+0000 7fc9c5879700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c0136f20 con 0x7fc9c0138d80 2023-10-03T10:19:12.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.604+0000 7fc9bf7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9c0138d80 0x7fc9c0134160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:12.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.604+0000 7fc9bf7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9c0138d80 0x7fc9c0134160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:44138/0 (socket says 172.21.15.3:44138) 2023-10-03T10:19:12.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.604+0000 7fc9bf7fe700 1 -- 172.21.15.3:0/1904234311 learned_addr learned my addr 172.21.15.3:0/1904234311 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:12.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.604+0000 7fc9be7fc700 1 --2- 172.21.15.3:0/1904234311 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9c00a0a30 0x7fc9c0133c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:12.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.604+0000 7fc9beffd700 1 --2- 172.21.15.3:0/1904234311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9c009d630 0x7fc9c01336e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:12.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.604+0000 7fc9be7fc700 1 -- 172.21.15.3:0/1904234311 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9c0138d80 msgr2=0x7fc9c0134160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:12.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.604+0000 7fc9be7fc700 1 --2- 172.21.15.3:0/1904234311 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9c0138d80 0x7fc9c0134160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.604+0000 7fc9be7fc700 1 -- 172.21.15.3:0/1904234311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9c009d630 msgr2=0x7fc9c01336e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:12.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.604+0000 7fc9be7fc700 1 --2- 172.21.15.3:0/1904234311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9c009d630 0x7fc9c01336e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.604+0000 7fc9be7fc700 1 -- 172.21.15.3:0/1904234311 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b8009b30 con 0x7fc9c00a0a30 2023-10-03T10:19:12.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.604+0000 7fc9be7fc700 1 --2- 172.21.15.3:0/1904234311 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9c00a0a30 0x7fc9c0133c20 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400c3e0 tx=0x7fc9b400c6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:12.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.605+0000 7fc9a7fff700 1 -- 172.21.15.3:0/1904234311 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b40052d0 con 0x7fc9c00a0a30 2023-10-03T10:19:12.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.605+0000 7fc9a7fff700 1 -- 172.21.15.3:0/1904234311 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b4005450 con 0x7fc9c00a0a30 2023-10-03T10:19:12.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.605+0000 7fc9c5879700 1 -- 172.21.15.3:0/1904234311 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9c00a4c70 con 0x7fc9c00a0a30 2023-10-03T10:19:12.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.605+0000 7fc9a7fff700 1 -- 172.21.15.3:0/1904234311 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b40038c0 con 0x7fc9c00a0a30 2023-10-03T10:19:12.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.605+0000 7fc9c5879700 1 -- 172.21.15.3:0/1904234311 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9c0137990 con 0x7fc9c00a0a30 2023-10-03T10:19:12.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.606+0000 7fc9a7fff700 1 -- 172.21.15.3:0/1904234311 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc9b4003a20 con 0x7fc9c00a0a30 2023-10-03T10:19:12.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.606+0000 7fc9a7fff700 1 -- 172.21.15.3:0/1904234311 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc9b403ea40 con 0x7fc9c00a0a30 2023-10-03T10:19:12.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.607+0000 7fc9c5879700 1 -- 172.21.15.3:0/1904234311 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9c00747f0 con 0x7fc9c00a0a30 2023-10-03T10:19:12.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.612+0000 7fc9a7fff700 1 -- 172.21.15.3:0/1904234311 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc9b4017020 con 0x7fc9c00a0a30 2023-10-03T10:19:12.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.727+0000 7f111b965700 1 -- 172.21.15.3:0/2882154422 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1114071e60 con 0x7f111406a770 2023-10-03T10:19:12.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.728+0000 7f1109ffb700 1 -- 172.21.15.3:0/2882154422 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f11100123b0 con 0x7f111406a770 2023-10-03T10:19:12.728 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:12.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.730+0000 7f111b965700 1 -- 172.21.15.3:0/2882154422 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f111406a770 msgr2=0x7f111409ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:12.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.730+0000 7f111b965700 1 --2- 172.21.15.3:0/2882154422 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f111406a770 0x7f111409ad80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f111000c520 tx=0x7f111000c830 comp rx=0 tx=0).stop 2023-10-03T10:19:12.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.730+0000 7f111b965700 1 -- 172.21.15.3:0/2882154422 shutdown_connections 2023-10-03T10:19:12.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.730+0000 7f111b965700 1 --2- 172.21.15.3:0/2882154422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11141c8310 0x7f11141c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.730+0000 7f111b965700 1 --2- 172.21.15.3:0/2882154422 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f111406a770 0x7f111409ad80 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.730+0000 7f111b965700 1 --2- 172.21.15.3:0/2882154422 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1114136d80 0x7f111409b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.730+0000 7f111b965700 1 -- 172.21.15.3:0/2882154422 >> 172.21.15.3:0/2882154422 conn(0x7f1114066f50 msgr2=0x7f1114135f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:12.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.730+0000 7f111b965700 1 -- 172.21.15.3:0/2882154422 shutdown_connections 2023-10-03T10:19:12.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.730+0000 7f111b965700 1 -- 172.21.15.3:0/2882154422 wait complete. 2023-10-03T10:19:12.745 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:12.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.775+0000 7fde2d64d700 1 -- 172.21.15.3:0/3265553710 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fde280a34f0 con 0x7fde2809f650 2023-10-03T10:19:12.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.775+0000 7fde0f7fe700 1 -- 172.21.15.3:0/3265553710 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fde20034090 con 0x7fde2809f650 2023-10-03T10:19:12.776 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:12.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.778+0000 7fde2d64d700 1 -- 172.21.15.3:0/3265553710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde2809f650 msgr2=0x7fde2806c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:12.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.778+0000 7fde2d64d700 1 --2- 172.21.15.3:0/3265553710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde2809f650 0x7fde2806c330 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fde2000b680 tx=0x7fde20004ad0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.778+0000 7fde2d64d700 1 -- 172.21.15.3:0/3265553710 shutdown_connections 2023-10-03T10:19:12.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.778+0000 7fde2d64d700 1 --2- 172.21.15.3:0/3265553710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde2809f650 0x7fde2806c330 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.778+0000 7fde2d64d700 1 --2- 172.21.15.3:0/3265553710 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fde2809e930 0x7fde2806b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.778+0000 7fde2d64d700 1 --2- 172.21.15.3:0/3265553710 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde2809f240 0x7fde2806bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.778+0000 7fde2d64d700 1 -- 172.21.15.3:0/3265553710 >> 172.21.15.3:0/3265553710 conn(0x7fde280a4210 msgr2=0x7fde281b8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:12.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.778+0000 7fde2d64d700 1 -- 172.21.15.3:0/3265553710 shutdown_connections 2023-10-03T10:19:12.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.778+0000 7fde2d64d700 1 -- 172.21.15.3:0/3265553710 wait complete. 2023-10-03T10:19:12.792 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:12.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.795+0000 7f29db18b700 1 -- 172.21.15.3:0/3748619919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f29d411ee80 con 0x7f29d409a3b0 2023-10-03T10:19:12.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.795+0000 7f29d17fa700 1 -- 172.21.15.3:0/3748619919 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f29c0021020 con 0x7f29d409a3b0 2023-10-03T10:19:12.796 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:12.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.798+0000 7f29db18b700 1 -- 172.21.15.3:0/3748619919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d409a3b0 msgr2=0x7f29d406fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:12.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.798+0000 7f29db18b700 1 --2- 172.21.15.3:0/3748619919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d409a3b0 0x7f29d406fb50 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f29c000e8d0 tx=0x7f29c000ebe0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.798+0000 7f29db18b700 1 -- 172.21.15.3:0/3748619919 shutdown_connections 2023-10-03T10:19:12.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.798+0000 7f29db18b700 1 --2- 172.21.15.3:0/3748619919 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f29d4131810 0x7f29d4070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.798+0000 7f29db18b700 1 --2- 172.21.15.3:0/3748619919 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f29d4099a40 0x7f29d406f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.798+0000 7f29db18b700 1 --2- 172.21.15.3:0/3748619919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d409a3b0 0x7f29d406fb50 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.798+0000 7f29db18b700 1 -- 172.21.15.3:0/3748619919 >> 172.21.15.3:0/3748619919 conn(0x7f29d40a2aa0 msgr2=0x7f29d412e580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:12.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.798+0000 7f29db18b700 1 -- 172.21.15.3:0/3748619919 shutdown_connections 2023-10-03T10:19:12.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.798+0000 7f29db18b700 1 -- 172.21.15.3:0/3748619919 wait complete. 2023-10-03T10:19:12.812 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:12.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.875+0000 7fc9c5879700 1 -- 172.21.15.3:0/1904234311 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc9c0072980 con 0x7fc9c00a0a30 2023-10-03T10:19:12.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.876+0000 7fc9a7fff700 1 -- 172.21.15.3:0/1904234311 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc9b4019b40 con 0x7fc9c00a0a30 2023-10-03T10:19:12.876 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:12.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.878+0000 7fc9c5879700 1 -- 172.21.15.3:0/1904234311 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9c00a0a30 msgr2=0x7fc9c0133c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:12.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.878+0000 7fc9c5879700 1 --2- 172.21.15.3:0/1904234311 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9c00a0a30 0x7fc9c0133c20 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400c3e0 tx=0x7fc9b400c6f0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.878+0000 7fc9c5879700 1 -- 172.21.15.3:0/1904234311 shutdown_connections 2023-10-03T10:19:12.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.878+0000 7fc9c5879700 1 --2- 172.21.15.3:0/1904234311 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc9c0138d80 0x7fc9c0134160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.878+0000 7fc9c5879700 1 --2- 172.21.15.3:0/1904234311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9c009d630 0x7fc9c01336e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.878+0000 7fc9c5879700 1 --2- 172.21.15.3:0/1904234311 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc9c00a0a30 0x7fc9c0133c20 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:12.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.878+0000 7fc9c5879700 1 -- 172.21.15.3:0/1904234311 >> 172.21.15.3:0/1904234311 conn(0x7fc9c00a3050 msgr2=0x7fc9c009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:12.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.878+0000 7fc9c5879700 1 -- 172.21.15.3:0/1904234311 shutdown_connections 2023-10-03T10:19:12.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:12.878+0000 7fc9c5879700 1 -- 172.21.15.3:0/1904234311 wait complete. 2023-10-03T10:19:12.892 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:13.747 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:13.792 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:13.812 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:13.892 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:13.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.896+0000 7f68b448d700 1 Processor -- start 2023-10-03T10:19:13.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.896+0000 7f68b448d700 1 -- start start 2023-10-03T10:19:13.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.897+0000 7f68b448d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f68ac130670 0x7f68ac130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:13.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.897+0000 7f68b448d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f68ac06f680 0x7f68ac06fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:13.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.897+0000 7f68b448d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f68ac06ffd0 0x7f68ac137530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:13.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.897+0000 7f68b448d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68ac09a350 con 0x7f68ac06f680 2023-10-03T10:19:13.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.897+0000 7f68b448d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac130260 con 0x7f68ac130670 2023-10-03T10:19:13.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.897+0000 7f68b448d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac071ff0 con 0x7f68ac06ffd0 2023-10-03T10:19:13.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.897+0000 7f68b2229700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f68ac130670 0x7f68ac130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:13.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.897+0000 7f68b2229700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f68ac130670 0x7f68ac130a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:54112/0 (socket says 172.21.15.3:54112) 2023-10-03T10:19:13.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.897+0000 7f68b2229700 1 -- 172.21.15.3:0/3949251956 learned_addr learned my addr 172.21.15.3:0/3949251956 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:13.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.897+0000 7f68b1a28700 1 --2- 172.21.15.3:0/3949251956 >> v2:172.21.15.3:3300/0 conn(0x7f68ac06f680 0x7f68ac06fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:13.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.897+0000 7f68b2a2a700 1 --2- 172.21.15.3:0/3949251956 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f68ac06ffd0 0x7f68ac137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:13.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.897+0000 7f68b1a28700 1 -- 172.21.15.3:0/3949251956 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f68ac06ffd0 msgr2=0x7f68ac137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:13.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.897+0000 7f68b1a28700 1 --2- 172.21.15.3:0/3949251956 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f68ac06ffd0 0x7f68ac137530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:13.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.897+0000 7f68b1a28700 1 -- 172.21.15.3:0/3949251956 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f68ac130670 msgr2=0x7f68ac130a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:13.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.897+0000 7f68b1a28700 1 --2- 172.21.15.3:0/3949251956 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f68ac130670 0x7f68ac130a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:13.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.897+0000 7f68b1a28700 1 -- 172.21.15.3:0/3949251956 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ac070470 con 0x7f68ac06f680 2023-10-03T10:19:13.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.897+0000 7f68b2a2a700 1 --2- 172.21.15.3:0/3949251956 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f68ac06ffd0 0x7f68ac137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:19:13.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.897+0000 7f68b2229700 1 --2- 172.21.15.3:0/3949251956 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f68ac130670 0x7f68ac130a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:13.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.898+0000 7f68b1a28700 1 --2- 172.21.15.3:0/3949251956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68ac06f680 0x7f68ac06fa90 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f689c009840 tx=0x7f689c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f28ec85d2df4f20 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:13.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.898+0000 7f68b1227700 1 -- 172.21.15.3:0/3949251956 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c011830 con 0x7f68ac06f680 2023-10-03T10:19:13.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.898+0000 7f68b1227700 1 -- 172.21.15.3:0/3949251956 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689c011990 con 0x7f68ac06f680 2023-10-03T10:19:13.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.898+0000 7f68b1227700 1 -- 172.21.15.3:0/3949251956 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c011d20 con 0x7f68ac06f680 2023-10-03T10:19:13.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.899+0000 7f68b448d700 1 -- 172.21.15.3:0/3949251956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68ac06f680 msgr2=0x7f68ac06fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:13.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.899+0000 7f68b448d700 1 --2- 172.21.15.3:0/3949251956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68ac06f680 0x7f68ac06fa90 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f689c009840 tx=0x7f689c009b50 comp rx=0 tx=0).stop 2023-10-03T10:19:13.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.899+0000 7f68b448d700 1 -- 172.21.15.3:0/3949251956 shutdown_connections 2023-10-03T10:19:13.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.899+0000 7f68b448d700 1 --2- 172.21.15.3:0/3949251956 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f68ac06ffd0 0x7f68ac137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:13.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.899+0000 7f68b448d700 1 --2- 172.21.15.3:0/3949251956 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f68ac130670 0x7f68ac130a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:13.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.899+0000 7f68b448d700 1 --2- 172.21.15.3:0/3949251956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68ac06f680 0x7f68ac06fa90 secure :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f689c009840 tx=0x7f689c009b50 comp rx=0 tx=0).stop 2023-10-03T10:19:13.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.899+0000 7f68b448d700 1 -- 172.21.15.3:0/3949251956 >> 172.21.15.3:0/3949251956 conn(0x7f68ac0a2aa0 msgr2=0x7f68ac0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:13.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.899+0000 7f68b448d700 1 -- 172.21.15.3:0/3949251956 shutdown_connections 2023-10-03T10:19:13.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.899+0000 7f68b448d700 1 -- 172.21.15.3:0/3949251956 wait complete. 2023-10-03T10:19:13.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.900+0000 7f68b448d700 1 Processor -- start 2023-10-03T10:19:13.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.900+0000 7f68b448d700 1 -- start start 2023-10-03T10:19:13.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.901+0000 7f68b448d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f68ac06ffd0 0x7f68ac133cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:13.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.901+0000 7f68b448d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f68ac130670 0x7f68ac134230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:13.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.901+0000 7f68b448d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68ac12dab0 0x7f68ac134770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:13.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.901+0000 7f68b448d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac119b50 con 0x7f68ac12dab0 2023-10-03T10:19:13.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.901+0000 7f68b448d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac135f30 con 0x7f68ac06ffd0 2023-10-03T10:19:13.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.901+0000 7f68b448d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac118eb0 con 0x7f68ac130670 2023-10-03T10:19:13.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.901+0000 7f68b2a2a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68ac12dab0 0x7f68ac134770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:13.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.901+0000 7f68b2a2a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68ac12dab0 0x7f68ac134770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54410/0 (socket says 172.21.15.3:54410) 2023-10-03T10:19:13.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.901+0000 7f68b2a2a700 1 -- 172.21.15.3:0/2858486807 learned_addr learned my addr 172.21.15.3:0/2858486807 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:13.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.901+0000 7f68b1a28700 1 --2- 172.21.15.3:0/2858486807 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f68ac130670 0x7f68ac134230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:13.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.901+0000 7f68b2229700 1 --2- 172.21.15.3:0/2858486807 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f68ac06ffd0 0x7f68ac133cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:13.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.901+0000 7f68b1a28700 1 -- 172.21.15.3:0/2858486807 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f68ac06ffd0 msgr2=0x7f68ac133cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:13.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.901+0000 7f68b1a28700 1 --2- 172.21.15.3:0/2858486807 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f68ac06ffd0 0x7f68ac133cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:13.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.902+0000 7f68b1a28700 1 -- 172.21.15.3:0/2858486807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68ac12dab0 msgr2=0x7f68ac134770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:13.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.902+0000 7f68b1a28700 1 --2- 172.21.15.3:0/2858486807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68ac12dab0 0x7f68ac134770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:13.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.902+0000 7f68b1a28700 1 -- 172.21.15.3:0/2858486807 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a8009550 con 0x7f68ac130670 2023-10-03T10:19:13.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.902+0000 7f68b2229700 1 --2- 172.21.15.3:0/2858486807 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f68ac06ffd0 0x7f68ac133cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:19:13.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.902+0000 7f68b1a28700 1 --2- 172.21.15.3:0/2858486807 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f68ac130670 0x7f68ac134230 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f689c016000 tx=0x7f689c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:13.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.902+0000 7f68a2ffd700 1 -- 172.21.15.3:0/2858486807 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c004450 con 0x7f68ac130670 2023-10-03T10:19:13.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.902+0000 7f68a2ffd700 1 -- 172.21.15.3:0/2858486807 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689c01f070 con 0x7f68ac130670 2023-10-03T10:19:13.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.902+0000 7f68a2ffd700 1 -- 172.21.15.3:0/2858486807 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c0038c0 con 0x7f68ac130670 2023-10-03T10:19:13.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.902+0000 7f68b448d700 1 -- 172.21.15.3:0/2858486807 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f689c0094a0 con 0x7f68ac130670 2023-10-03T10:19:13.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.902+0000 7f68b448d700 1 -- 172.21.15.3:0/2858486807 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68ac12e700 con 0x7f68ac130670 2023-10-03T10:19:13.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.903+0000 7f68a2ffd700 1 -- 172.21.15.3:0/2858486807 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f689c003a20 con 0x7f68ac130670 2023-10-03T10:19:13.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.903+0000 7f68b448d700 1 -- 172.21.15.3:0/2858486807 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68ac09dd00 con 0x7f68ac130670 2023-10-03T10:19:13.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.904+0000 7f68a2ffd700 1 -- 172.21.15.3:0/2858486807 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f689c021070 con 0x7f68ac130670 2023-10-03T10:19:13.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.910+0000 7f68a2ffd700 1 -- 172.21.15.3:0/2858486807 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f689c003cd0 con 0x7f68ac130670 2023-10-03T10:19:13.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.950+0000 7f0491bd4700 1 Processor -- start 2023-10-03T10:19:13.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.951+0000 7f0491bd4700 1 -- start start 2023-10-03T10:19:13.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.952+0000 7f0491bd4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f048c09eca0 0x7f048c13cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:13.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.952+0000 7f0491bd4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f048c06f050 0x7f048c13f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:13.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.952+0000 7f0491bd4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f048c070ee0 con 0x7f048c06e570 2023-10-03T10:19:13.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.952+0000 7f0491bd4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048c071980 con 0x7f048c09eca0 2023-10-03T10:19:13.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.952+0000 7f0491bd4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f048c071480 con 0x7f048c06f050 2023-10-03T10:19:13.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.952+0000 7f048affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f048c09eca0 0x7f048c13cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:13.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.952+0000 7f048affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f048c09eca0 0x7f048c13cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:54134/0 (socket says 172.21.15.3:54134) 2023-10-03T10:19:13.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.952+0000 7f048affd700 1 -- 172.21.15.3:0/3673861049 learned_addr learned my addr 172.21.15.3:0/3673861049 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:13.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.952+0000 7f048bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f048c06f050 0x7f048c13f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:13.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.952+0000 7f048bfff700 1 -- 172.21.15.3:0/3673861049 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f048c09eca0 msgr2=0x7f048c13cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:13.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.952+0000 7f048bfff700 1 --2- 172.21.15.3:0/3673861049 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f048c09eca0 0x7f048c13cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:13.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.952+0000 7f048bfff700 1 -- 172.21.15.3:0/3673861049 >> v1:172.21.15.3:6789/0 conn(0x7f048c06e570 legacy=0x7f048c06e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:13.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.952+0000 7f048bfff700 1 -- 172.21.15.3:0/3673861049 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048c071b90 con 0x7f048c06f050 2023-10-03T10:19:13.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.952+0000 7f048a7fc700 1 -- 172.21.15.3:0/3673861049 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2265318820 0 0) 0x7f048c070ee0 con 0x7f048c06e570 2023-10-03T10:19:13.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.952+0000 7f048affd700 1 --2- 172.21.15.3:0/3673861049 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f048c09eca0 0x7f048c13cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:13.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.953+0000 7f048bfff700 1 --2- 172.21.15.3:0/3673861049 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f048c06f050 0x7f048c13f3f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0484009dc0 tx=0x7f048400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf4b308e4597c61a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:13.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.953+0000 7f048a7fc700 1 -- 172.21.15.3:0/3673861049 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048400e070 con 0x7f048c06f050 2023-10-03T10:19:13.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.953+0000 7f048a7fc700 1 -- 172.21.15.3:0/3673861049 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048400bb20 con 0x7f048c06f050 2023-10-03T10:19:13.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.953+0000 7f048a7fc700 1 -- 172.21.15.3:0/3673861049 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0484012430 con 0x7f048c06f050 2023-10-03T10:19:13.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.954+0000 7f0491bd4700 1 -- 172.21.15.3:0/3673861049 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f048c06f050 msgr2=0x7f048c13f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:13.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.954+0000 7f0491bd4700 1 --2- 172.21.15.3:0/3673861049 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f048c06f050 0x7f048c13f3f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0484009dc0 tx=0x7f048400b330 comp rx=0 tx=0).stop 2023-10-03T10:19:13.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.954+0000 7f0491bd4700 1 -- 172.21.15.3:0/3673861049 shutdown_connections 2023-10-03T10:19:13.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.954+0000 7f0491bd4700 1 --2- 172.21.15.3:0/3673861049 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f048c06f050 0x7f048c13f3f0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:13.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.954+0000 7f0491bd4700 1 --2- 172.21.15.3:0/3673861049 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f048c09eca0 0x7f048c13cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:13.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.954+0000 7f0491bd4700 1 -- 172.21.15.3:0/3673861049 >> 172.21.15.3:0/3673861049 conn(0x7f048c0a2aa0 msgr2=0x7f048c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:13.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.954+0000 7f0491bd4700 1 -- 172.21.15.3:0/3673861049 shutdown_connections 2023-10-03T10:19:13.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.954+0000 7f0491bd4700 1 -- 172.21.15.3:0/3673861049 wait complete. 2023-10-03T10:19:13.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.956+0000 7f0491bd4700 1 Processor -- start 2023-10-03T10:19:13.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.956+0000 7f0491bd4700 1 -- start start 2023-10-03T10:19:13.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.957+0000 7f0491bd4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f048c06e570 0x7f048c1265d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:13.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.957+0000 7f0491bd4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f048c06f050 0x7f048c126b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:13.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.957+0000 7f0491bd4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f048c127050 0x7f048c1d2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:13.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.957+0000 7f0491bd4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c127aa0 con 0x7f048c127050 2023-10-03T10:19:13.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.957+0000 7f0491bd4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048c1294a0 con 0x7f048c06e570 2023-10-03T10:19:13.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.957+0000 7f0491bd4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f048c13ea10 con 0x7f048c06f050 2023-10-03T10:19:13.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.957+0000 7f048affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f048c06f050 0x7f048c126b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:13.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.957+0000 7f048affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f048c06f050 0x7f048c126b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41504/0 (socket says 172.21.15.3:41504) 2023-10-03T10:19:13.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.957+0000 7f048affd700 1 -- 172.21.15.3:0/4053108976 learned_addr learned my addr 172.21.15.3:0/4053108976 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:13.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.957+0000 7f048b7fe700 1 --2- 172.21.15.3:0/4053108976 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f048c06e570 0x7f048c1265d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:13.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.957+0000 7f048bfff700 1 --2- 172.21.15.3:0/4053108976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f048c127050 0x7f048c1d2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:13.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.957+0000 7f048affd700 1 -- 172.21.15.3:0/4053108976 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f048c06e570 msgr2=0x7f048c1265d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:13.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.957+0000 7f048affd700 1 --2- 172.21.15.3:0/4053108976 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f048c06e570 0x7f048c1265d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:13.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.957+0000 7f048affd700 1 -- 172.21.15.3:0/4053108976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f048c127050 msgr2=0x7f048c1d2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:13.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.957+0000 7f048affd700 1 --2- 172.21.15.3:0/4053108976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f048c127050 0x7f048c1d2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:13.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.957+0000 7f048affd700 1 -- 172.21.15.3:0/4053108976 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0480009550 con 0x7f048c06f050 2023-10-03T10:19:13.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.958+0000 7f048affd700 1 --2- 172.21.15.3:0/4053108976 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f048c06f050 0x7f048c126b10 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f048000ec10 tx=0x7f048000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:13.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.958+0000 7f0473fff700 1 -- 172.21.15.3:0/4053108976 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048000c850 con 0x7f048c06f050 2023-10-03T10:19:13.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.958+0000 7f0473fff700 1 -- 172.21.15.3:0/4053108976 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048000c9b0 con 0x7f048c06f050 2023-10-03T10:19:13.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.958+0000 7f0473fff700 1 -- 172.21.15.3:0/4053108976 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048000cd20 con 0x7f048c06f050 2023-10-03T10:19:13.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.958+0000 7f0491bd4700 1 -- 172.21.15.3:0/4053108976 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0484009a70 con 0x7f048c06f050 2023-10-03T10:19:13.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.958+0000 7f0491bd4700 1 -- 172.21.15.3:0/4053108976 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f048c070610 con 0x7f048c06f050 2023-10-03T10:19:13.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.959+0000 7f0473fff700 1 -- 172.21.15.3:0/4053108976 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0480013470 con 0x7f048c06f050 2023-10-03T10:19:13.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.959+0000 7f0473fff700 1 -- 172.21.15.3:0/4053108976 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f048003fa10 con 0x7f048c06f050 2023-10-03T10:19:13.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.959+0000 7f0491bd4700 1 -- 172.21.15.3:0/4053108976 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f048c074ea0 con 0x7f048c06f050 2023-10-03T10:19:13.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.965+0000 7f0473fff700 1 -- 172.21.15.3:0/4053108976 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0480021080 con 0x7f048c06f050 2023-10-03T10:19:13.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.987+0000 7f0bc9151700 1 Processor -- start 2023-10-03T10:19:13.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.987+0000 7f0bc9151700 1 -- start start 2023-10-03T10:19:13.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.987+0000 7f0bc9151700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0bc406a770 0x7f0bc409d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:13.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.987+0000 7f0bc9151700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0bc409dc30 0x7f0bc409e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:13.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.987+0000 7f0bc9151700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0bc413c1e0 0x7f0bc413e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:13.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.987+0000 7f0bc9151700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bc406ab40 con 0x7f0bc413c1e0 2023-10-03T10:19:13.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.987+0000 7f0bc9151700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc409ebc0 con 0x7f0bc409dc30 2023-10-03T10:19:13.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.987+0000 7f0bc9151700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc409a810 con 0x7f0bc406a770 2023-10-03T10:19:13.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.988+0000 7f0bc2d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0bc406a770 0x7f0bc409d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:13.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.988+0000 7f0bc2d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0bc406a770 0x7f0bc409d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41508/0 (socket says 172.21.15.3:41508) 2023-10-03T10:19:13.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.988+0000 7f0bc2d9d700 1 -- 172.21.15.3:0/1457244044 learned_addr learned my addr 172.21.15.3:0/1457244044 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:13.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.988+0000 7f0bc259c700 1 --2- 172.21.15.3:0/1457244044 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0bc409dc30 0x7f0bc409e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:13.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.988+0000 7f0bc359e700 1 --2- 172.21.15.3:0/1457244044 >> v2:172.21.15.3:3300/0 conn(0x7f0bc413c1e0 0x7f0bc413e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:13.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.988+0000 7f0bc259c700 1 -- 172.21.15.3:0/1457244044 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0bc406a770 msgr2=0x7f0bc409d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:13.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.988+0000 7f0bc259c700 1 --2- 172.21.15.3:0/1457244044 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0bc406a770 0x7f0bc409d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:13.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.988+0000 7f0bc259c700 1 -- 172.21.15.3:0/1457244044 >> v2:172.21.15.3:3300/0 conn(0x7f0bc413c1e0 msgr2=0x7f0bc413e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:13.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.988+0000 7f0bc259c700 1 --2- 172.21.15.3:0/1457244044 >> v2:172.21.15.3:3300/0 conn(0x7f0bc413c1e0 0x7f0bc413e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:13.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.988+0000 7f0bc259c700 1 -- 172.21.15.3:0/1457244044 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc409e920 con 0x7f0bc409dc30 2023-10-03T10:19:13.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.988+0000 7f0bc2d9d700 1 --2- 172.21.15.3:0/1457244044 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0bc406a770 0x7f0bc409d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:13.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.988+0000 7f0bc359e700 1 --2- 172.21.15.3:0/1457244044 >> v2:172.21.15.3:3300/0 conn(0x7f0bc413c1e0 0x7f0bc413e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:13.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.988+0000 7f0bc259c700 1 --2- 172.21.15.3:0/1457244044 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0bc409dc30 0x7f0bc409e080 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8009870 tx=0x7f0bb8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2337577affc3e1fa server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:13.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.989+0000 7f0bc1d9b700 1 -- 172.21.15.3:0/1457244044 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8011830 con 0x7f0bc409dc30 2023-10-03T10:19:13.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.989+0000 7f0bc1d9b700 1 -- 172.21.15.3:0/1457244044 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb8011990 con 0x7f0bc409dc30 2023-10-03T10:19:13.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.989+0000 7f0bc1d9b700 1 -- 172.21.15.3:0/1457244044 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8011d20 con 0x7f0bc409dc30 2023-10-03T10:19:13.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.990+0000 7f0bc9151700 1 -- 172.21.15.3:0/1457244044 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0bc409dc30 msgr2=0x7f0bc409e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:13.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.990+0000 7f0bc9151700 1 --2- 172.21.15.3:0/1457244044 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0bc409dc30 0x7f0bc409e080 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8009870 tx=0x7f0bb8009b80 comp rx=0 tx=0).stop 2023-10-03T10:19:13.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.990+0000 7f0bc9151700 1 -- 172.21.15.3:0/1457244044 shutdown_connections 2023-10-03T10:19:13.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.990+0000 7f0bc9151700 1 --2- 172.21.15.3:0/1457244044 >> v2:172.21.15.3:3300/0 conn(0x7f0bc413c1e0 0x7f0bc413e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:13.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.990+0000 7f0bc9151700 1 --2- 172.21.15.3:0/1457244044 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0bc406a770 0x7f0bc409d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:13.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.990+0000 7f0bc9151700 1 --2- 172.21.15.3:0/1457244044 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0bc409dc30 0x7f0bc409e080 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:13.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.990+0000 7f0bc9151700 1 -- 172.21.15.3:0/1457244044 >> 172.21.15.3:0/1457244044 conn(0x7f0bc4066f50 msgr2=0x7f0bc409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:13.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.990+0000 7f0bc9151700 1 -- 172.21.15.3:0/1457244044 shutdown_connections 2023-10-03T10:19:13.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.990+0000 7f0bc9151700 1 -- 172.21.15.3:0/1457244044 wait complete. 2023-10-03T10:19:13.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.992+0000 7f0bc9151700 1 Processor -- start 2023-10-03T10:19:13.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.992+0000 7f0bc9151700 1 -- start start 2023-10-03T10:19:13.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.992+0000 7f0bc9151700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0bc406a770 0x7f0bc413b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:13.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.992+0000 7f0bc9151700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0bc409dc30 0x7f0bc411d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:13.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.992+0000 7f0bc9151700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0bc413c1e0 0x7f0bc411ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:13.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.992+0000 7f0bc9151700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc411ef80 con 0x7f0bc406a770 2023-10-03T10:19:13.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.992+0000 7f0bc9151700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc41200e0 con 0x7f0bc409dc30 2023-10-03T10:19:13.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.992+0000 7f0bc9151700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc40a3880 con 0x7f0bc413c1e0 2023-10-03T10:19:13.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.993+0000 7f0bc359e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0bc413c1e0 0x7f0bc411ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:13.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.993+0000 7f0bc359e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0bc413c1e0 0x7f0bc411ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41512/0 (socket says 172.21.15.3:41512) 2023-10-03T10:19:13.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.993+0000 7f0bc359e700 1 -- 172.21.15.3:0/3876076280 learned_addr learned my addr 172.21.15.3:0/3876076280 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:13.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.993+0000 7f0bc259c700 1 --2- 172.21.15.3:0/3876076280 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0bc409dc30 0x7f0bc411d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:13.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.993+0000 7f0bc2d9d700 1 --2- 172.21.15.3:0/3876076280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0bc406a770 0x7f0bc413b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:13.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.993+0000 7f0bc359e700 1 -- 172.21.15.3:0/3876076280 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0bc409dc30 msgr2=0x7f0bc411d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:13.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.993+0000 7f0bc359e700 1 --2- 172.21.15.3:0/3876076280 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0bc409dc30 0x7f0bc411d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:13.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.993+0000 7f0bc359e700 1 -- 172.21.15.3:0/3876076280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0bc406a770 msgr2=0x7f0bc413b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:13.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.993+0000 7f0bc359e700 1 --2- 172.21.15.3:0/3876076280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0bc406a770 0x7f0bc413b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:13.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.993+0000 7f0bc359e700 1 -- 172.21.15.3:0/3876076280 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb4009550 con 0x7f0bc413c1e0 2023-10-03T10:19:13.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.993+0000 7f0bc2d9d700 1 --2- 172.21.15.3:0/3876076280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0bc406a770 0x7f0bc413b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:13.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.993+0000 7f0bc259c700 1 --2- 172.21.15.3:0/3876076280 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0bc409dc30 0x7f0bc411d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:13.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.993+0000 7f0bc359e700 1 --2- 172.21.15.3:0/3876076280 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0bc413c1e0 0x7f0bc411ddd0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00efc0 tx=0x7f0bbc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:13.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.994+0000 7f0bab7fe700 1 -- 172.21.15.3:0/3876076280 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc009d70 con 0x7f0bc413c1e0 2023-10-03T10:19:13.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.994+0000 7f0bab7fe700 1 -- 172.21.15.3:0/3876076280 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bbc00f040 con 0x7f0bc413c1e0 2023-10-03T10:19:13.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.994+0000 7f0bab7fe700 1 -- 172.21.15.3:0/3876076280 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc00ce00 con 0x7f0bc413c1e0 2023-10-03T10:19:13.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.994+0000 7f0bc9151700 1 -- 172.21.15.3:0/3876076280 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bbc009a70 con 0x7f0bc413c1e0 2023-10-03T10:19:13.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.994+0000 7f0bc9151700 1 -- 172.21.15.3:0/3876076280 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb80094a0 con 0x7f0bc413c1e0 2023-10-03T10:19:13.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.995+0000 7f0bc9151700 1 -- 172.21.15.3:0/3876076280 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bc411faa0 con 0x7f0bc413c1e0 2023-10-03T10:19:13.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.997+0000 7f0bab7fe700 1 -- 172.21.15.3:0/3876076280 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0bbc003a20 con 0x7f0bc413c1e0 2023-10-03T10:19:13.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:13.997+0000 7f0bab7fe700 1 -- 172.21.15.3:0/3876076280 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0bbc0409f0 con 0x7f0bc413c1e0 2023-10-03T10:19:14.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.004+0000 7f0bab7fe700 1 -- 172.21.15.3:0/3876076280 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0bbc01c7c0 con 0x7f0bc413c1e0 2023-10-03T10:19:14.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.044+0000 7f3eac8d9700 1 Processor -- start 2023-10-03T10:19:14.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.044+0000 7f3eac8d9700 1 -- start start 2023-10-03T10:19:14.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.045+0000 7f3eac8d9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3ea809d6f0 0x7f3ea809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:14.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.045+0000 7f3eac8d9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ea809e440 0x7f3ea8134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:14.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.045+0000 7f3eac8d9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea809b120 con 0x7f3ea809e000 2023-10-03T10:19:14.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.045+0000 7f3eac8d9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ea806b460 con 0x7f3ea809d6f0 2023-10-03T10:19:14.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.045+0000 7f3eac8d9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea809a7e0 con 0x7f3ea809e440 2023-10-03T10:19:14.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.045+0000 7f3ea6d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ea809e440 0x7f3ea8134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:14.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.045+0000 7f3ea6d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ea809e440 0x7f3ea8134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41516/0 (socket says 172.21.15.3:41516) 2023-10-03T10:19:14.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.045+0000 7f3ea6d9d700 1 -- 172.21.15.3:0/620461162 learned_addr learned my addr 172.21.15.3:0/620461162 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:14.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.045+0000 7f3ea659c700 1 --2- 172.21.15.3:0/620461162 >> v2:172.21.15.3:3300/0 conn(0x7f3ea809d6f0 0x7f3ea809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:14.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.045+0000 7f3ea6d9d700 1 -- 172.21.15.3:0/620461162 >> v2:172.21.15.3:3300/0 conn(0x7f3ea809d6f0 msgr2=0x7f3ea809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:14.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.045+0000 7f3ea6d9d700 1 --2- 172.21.15.3:0/620461162 >> v2:172.21.15.3:3300/0 conn(0x7f3ea809d6f0 0x7f3ea809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:14.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.045+0000 7f3ea6d9d700 1 -- 172.21.15.3:0/620461162 >> v1:172.21.15.3:6789/0 conn(0x7f3ea809e000 legacy=0x7f3ea81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:14.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.045+0000 7f3ea6d9d700 1 -- 172.21.15.3:0/620461162 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea809a500 con 0x7f3ea809e440 2023-10-03T10:19:14.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.045+0000 7f3ea659c700 1 --2- 172.21.15.3:0/620461162 >> v2:172.21.15.3:3300/0 conn(0x7f3ea809d6f0 0x7f3ea809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:14.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.046+0000 7f3ea6d9d700 1 --2- 172.21.15.3:0/620461162 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ea809e440 0x7f3ea8134100 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0009e80 tx=0x7f3ea000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=563dd70512e91581 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:14.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.046+0000 7f3ea559a700 1 -- 172.21.15.3:0/620461162 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea000e070 con 0x7f3ea809e440 2023-10-03T10:19:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.046+0000 7f3ea559a700 1 -- 172.21.15.3:0/620461162 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea0003a20 con 0x7f3ea809e440 2023-10-03T10:19:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.046+0000 7f3ea559a700 1 -- 172.21.15.3:0/620461162 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0003dd0 con 0x7f3ea809e440 2023-10-03T10:19:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.046+0000 7f3eac8d9700 1 -- 172.21.15.3:0/620461162 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ea809e440 msgr2=0x7f3ea8134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.046+0000 7f3eac8d9700 1 --2- 172.21.15.3:0/620461162 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ea809e440 0x7f3ea8134100 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0009e80 tx=0x7f3ea000b330 comp rx=0 tx=0).stop 2023-10-03T10:19:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.047+0000 7f3eac8d9700 1 -- 172.21.15.3:0/620461162 shutdown_connections 2023-10-03T10:19:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.047+0000 7f3eac8d9700 1 --2- 172.21.15.3:0/620461162 >> v2:172.21.15.3:3300/0 conn(0x7f3ea809d6f0 0x7f3ea809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.047+0000 7f3eac8d9700 1 --2- 172.21.15.3:0/620461162 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ea809e440 0x7f3ea8134100 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.047+0000 7f3eac8d9700 1 -- 172.21.15.3:0/620461162 >> 172.21.15.3:0/620461162 conn(0x7f3ea8066f50 msgr2=0x7f3ea809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.047+0000 7f3eac8d9700 1 -- 172.21.15.3:0/620461162 shutdown_connections 2023-10-03T10:19:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.047+0000 7f3eac8d9700 1 -- 172.21.15.3:0/620461162 wait complete. 2023-10-03T10:19:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.048+0000 7f3eac8d9700 1 Processor -- start 2023-10-03T10:19:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.048+0000 7f3eac8d9700 1 -- start start 2023-10-03T10:19:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.048+0000 7f3eac8d9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ea809d6f0 0x7f3ea81a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.048+0000 7f3eac8d9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ea809e000 0x7f3ea81a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.048+0000 7f3eac8d9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ea809e440 0x7f3ea81a6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.048+0000 7f3eac8d9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea81a98d0 con 0x7f3ea809e000 2023-10-03T10:19:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.048+0000 7f3eac8d9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea81aa270 con 0x7f3ea809e440 2023-10-03T10:19:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.048+0000 7f3eac8d9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea81a9060 con 0x7f3ea809d6f0 2023-10-03T10:19:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.049+0000 7f3ea659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ea809d6f0 0x7f3ea81a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.049+0000 7f3ea659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ea809d6f0 0x7f3ea81a5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41526/0 (socket says 172.21.15.3:41526) 2023-10-03T10:19:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.049+0000 7f3ea659c700 1 -- 172.21.15.3:0/858232224 learned_addr learned my addr 172.21.15.3:0/858232224 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.049+0000 7f3ea6d9d700 1 --2- 172.21.15.3:0/858232224 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ea809e440 0x7f3ea81a6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.049+0000 7f3ea659c700 1 -- 172.21.15.3:0/858232224 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ea809e440 msgr2=0x7f3ea81a6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.049+0000 7f3ea659c700 1 --2- 172.21.15.3:0/858232224 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ea809e440 0x7f3ea81a6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.049+0000 7f3ea659c700 1 -- 172.21.15.3:0/858232224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ea809e000 msgr2=0x7f3ea81a62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:14.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.049+0000 7f3ea659c700 1 --2- 172.21.15.3:0/858232224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ea809e000 0x7f3ea81a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:14.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.049+0000 7f3ea659c700 1 -- 172.21.15.3:0/858232224 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea0009b30 con 0x7f3ea809d6f0 2023-10-03T10:19:14.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.049+0000 7f3ea659c700 1 --2- 172.21.15.3:0/858232224 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ea809d6f0 0x7f3ea81a5db0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800e810 tx=0x7f3e9800ebd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:14.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.049+0000 7f3e96ffd700 1 -- 172.21.15.3:0/858232224 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e98013610 con 0x7f3ea809d6f0 2023-10-03T10:19:14.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.049+0000 7f3e96ffd700 1 -- 172.21.15.3:0/858232224 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e98013770 con 0x7f3ea809d6f0 2023-10-03T10:19:14.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.049+0000 7f3e96ffd700 1 -- 172.21.15.3:0/858232224 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e98013b20 con 0x7f3ea809d6f0 2023-10-03T10:19:14.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.049+0000 7f3eac8d9700 1 -- 172.21.15.3:0/858232224 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea80a2020 con 0x7f3ea809d6f0 2023-10-03T10:19:14.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.050+0000 7f3eac8d9700 1 -- 172.21.15.3:0/858232224 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea81a79c0 con 0x7f3ea809d6f0 2023-10-03T10:19:14.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.051+0000 7f3e96ffd700 1 -- 172.21.15.3:0/858232224 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3e98013c80 con 0x7f3ea809d6f0 2023-10-03T10:19:14.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.051+0000 7f3e96ffd700 1 -- 172.21.15.3:0/858232224 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3e9803f4a0 con 0x7f3ea809d6f0 2023-10-03T10:19:14.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.051+0000 7f3eac8d9700 1 -- 172.21.15.3:0/858232224 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea8074ea0 con 0x7f3ea809d6f0 2023-10-03T10:19:14.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.061+0000 7f3e96ffd700 1 -- 172.21.15.3:0/858232224 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3e9800c390 con 0x7f3ea809d6f0 2023-10-03T10:19:14.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.130+0000 7f68b448d700 1 -- 172.21.15.3:0/2858486807 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f68ac074ea0 con 0x7f68ac130670 2023-10-03T10:19:14.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.130+0000 7f68a2ffd700 1 -- 172.21.15.3:0/2858486807 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f689c015020 con 0x7f68ac130670 2023-10-03T10:19:14.131 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:14.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.133+0000 7f68b448d700 1 -- 172.21.15.3:0/2858486807 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f68ac130670 msgr2=0x7f68ac134230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:14.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.133+0000 7f68b448d700 1 --2- 172.21.15.3:0/2858486807 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f68ac130670 0x7f68ac134230 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f689c016000 tx=0x7f689c004750 comp rx=0 tx=0).stop 2023-10-03T10:19:14.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.133+0000 7f68b448d700 1 -- 172.21.15.3:0/2858486807 shutdown_connections 2023-10-03T10:19:14.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.133+0000 7f68b448d700 1 --2- 172.21.15.3:0/2858486807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68ac12dab0 0x7f68ac134770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:14.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.133+0000 7f68b448d700 1 --2- 172.21.15.3:0/2858486807 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f68ac06ffd0 0x7f68ac133cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:14.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.133+0000 7f68b448d700 1 --2- 172.21.15.3:0/2858486807 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f68ac130670 0x7f68ac134230 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:14.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.133+0000 7f68b448d700 1 -- 172.21.15.3:0/2858486807 >> 172.21.15.3:0/2858486807 conn(0x7f68ac0a2aa0 msgr2=0x7f68ac0a1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:14.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.133+0000 7f68b448d700 1 -- 172.21.15.3:0/2858486807 shutdown_connections 2023-10-03T10:19:14.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.133+0000 7f68b448d700 1 -- 172.21.15.3:0/2858486807 wait complete. 2023-10-03T10:19:14.148 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:14.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.183+0000 7f0491bd4700 1 -- 172.21.15.3:0/4053108976 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f048c1289c0 con 0x7f048c06f050 2023-10-03T10:19:14.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.183+0000 7f0473fff700 1 -- 172.21.15.3:0/4053108976 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f04800232f0 con 0x7f048c06f050 2023-10-03T10:19:14.184 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:14.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.186+0000 7f0491bd4700 1 -- 172.21.15.3:0/4053108976 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f048c06f050 msgr2=0x7f048c126b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:14.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.186+0000 7f0491bd4700 1 --2- 172.21.15.3:0/4053108976 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f048c06f050 0x7f048c126b10 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f048000ec10 tx=0x7f048000ef20 comp rx=0 tx=0).stop 2023-10-03T10:19:14.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.186+0000 7f0491bd4700 1 -- 172.21.15.3:0/4053108976 shutdown_connections 2023-10-03T10:19:14.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.186+0000 7f0491bd4700 1 --2- 172.21.15.3:0/4053108976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f048c127050 0x7f048c1d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:14.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.186+0000 7f0491bd4700 1 --2- 172.21.15.3:0/4053108976 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f048c06e570 0x7f048c1265d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:14.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.186+0000 7f0491bd4700 1 --2- 172.21.15.3:0/4053108976 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f048c06f050 0x7f048c126b10 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:14.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.186+0000 7f0491bd4700 1 -- 172.21.15.3:0/4053108976 >> 172.21.15.3:0/4053108976 conn(0x7f048c0a2aa0 msgr2=0x7f048c0a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:14.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.186+0000 7f0491bd4700 1 -- 172.21.15.3:0/4053108976 shutdown_connections 2023-10-03T10:19:14.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.186+0000 7f0491bd4700 1 -- 172.21.15.3:0/4053108976 wait complete. 2023-10-03T10:19:14.200 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:14.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.217+0000 7f0bc9151700 1 -- 172.21.15.3:0/3876076280 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0bc411fd60 con 0x7f0bc413c1e0 2023-10-03T10:19:14.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.217+0000 7f0bab7fe700 1 -- 172.21.15.3:0/3876076280 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0bc411fd60 con 0x7f0bc413c1e0 2023-10-03T10:19:14.218 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:14.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.220+0000 7f0bc9151700 1 -- 172.21.15.3:0/3876076280 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0bc413c1e0 msgr2=0x7f0bc411ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:14.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.220+0000 7f0bc9151700 1 --2- 172.21.15.3:0/3876076280 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0bc413c1e0 0x7f0bc411ddd0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00efc0 tx=0x7f0bbc00c3a0 comp rx=0 tx=0).stop 2023-10-03T10:19:14.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.220+0000 7f0bc9151700 1 -- 172.21.15.3:0/3876076280 shutdown_connections 2023-10-03T10:19:14.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.220+0000 7f0bc9151700 1 --2- 172.21.15.3:0/3876076280 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0bc413c1e0 0x7f0bc411ddd0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:14.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.220+0000 7f0bc9151700 1 --2- 172.21.15.3:0/3876076280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0bc406a770 0x7f0bc413b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:14.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.220+0000 7f0bc9151700 1 --2- 172.21.15.3:0/3876076280 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0bc409dc30 0x7f0bc411d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:14.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.220+0000 7f0bc9151700 1 -- 172.21.15.3:0/3876076280 >> 172.21.15.3:0/3876076280 conn(0x7f0bc4066f50 msgr2=0x7f0bc40a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:14.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.220+0000 7f0bc9151700 1 -- 172.21.15.3:0/3876076280 shutdown_connections 2023-10-03T10:19:14.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.220+0000 7f0bc9151700 1 -- 172.21.15.3:0/3876076280 wait complete. 2023-10-03T10:19:14.234 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:14.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.269+0000 7f3eac8d9700 1 -- 172.21.15.3:0/858232224 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3ea809f420 con 0x7f3ea809d6f0 2023-10-03T10:19:14.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.270+0000 7f3e96ffd700 1 -- 172.21.15.3:0/858232224 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3e9801b3b0 con 0x7f3ea809d6f0 2023-10-03T10:19:14.270 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:14.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.272+0000 7f3eac8d9700 1 -- 172.21.15.3:0/858232224 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ea809d6f0 msgr2=0x7f3ea81a5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:14.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.272+0000 7f3eac8d9700 1 --2- 172.21.15.3:0/858232224 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ea809d6f0 0x7f3ea81a5db0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800e810 tx=0x7f3e9800ebd0 comp rx=0 tx=0).stop 2023-10-03T10:19:14.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.272+0000 7f3eac8d9700 1 -- 172.21.15.3:0/858232224 shutdown_connections 2023-10-03T10:19:14.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.272+0000 7f3eac8d9700 1 --2- 172.21.15.3:0/858232224 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ea809e440 0x7f3ea81a6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:14.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.272+0000 7f3eac8d9700 1 --2- 172.21.15.3:0/858232224 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ea809d6f0 0x7f3ea81a5db0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:14.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.272+0000 7f3eac8d9700 1 --2- 172.21.15.3:0/858232224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ea809e000 0x7f3ea81a62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:14.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.272+0000 7f3eac8d9700 1 -- 172.21.15.3:0/858232224 >> 172.21.15.3:0/858232224 conn(0x7f3ea8066f50 msgr2=0x7f3ea81bad10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:14.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.272+0000 7f3eac8d9700 1 -- 172.21.15.3:0/858232224 shutdown_connections 2023-10-03T10:19:14.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:14.272+0000 7f3eac8d9700 1 -- 172.21.15.3:0/858232224 wait complete. 2023-10-03T10:19:14.285 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:15.149 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:15.201 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:15.234 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:15.286 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:15.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.301+0000 7f6671b28700 1 Processor -- start 2023-10-03T10:19:15.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.301+0000 7f6671b28700 1 -- start start 2023-10-03T10:19:15.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.302+0000 7f6671b28700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f666c136710 0x7f666c138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:15.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.302+0000 7f6671b28700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f666c139030 0x7f666c13b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:15.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.302+0000 7f6671b28700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f666c06ef30 con 0x7f666c1338a0 2023-10-03T10:19:15.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.302+0000 7f6671b28700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f666c06fef0 con 0x7f666c139030 2023-10-03T10:19:15.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.302+0000 7f6671b28700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f666c1334b0 con 0x7f666c136710 2023-10-03T10:19:15.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.302+0000 7f666affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f666c136710 0x7f666c138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:15.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.302+0000 7f666affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f666c136710 0x7f666c138af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41532/0 (socket says 172.21.15.3:41532) 2023-10-03T10:19:15.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.302+0000 7f666affd700 1 -- 172.21.15.3:0/539651626 learned_addr learned my addr 172.21.15.3:0/539651626 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:15.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.302+0000 7f666bfff700 1 --2- 172.21.15.3:0/539651626 >> v2:172.21.15.3:3300/0 conn(0x7f666c139030 0x7f666c13b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:15.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.302+0000 7f666a7fc700 1 -- 172.21.15.3:0/539651626 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 343588164 0 0) 0x7f666c06ef30 con 0x7f666c1338a0 2023-10-03T10:19:15.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.302+0000 7f666a7fc700 1 -- 172.21.15.3:0/539651626 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66540035b0 con 0x7f666c1338a0 2023-10-03T10:19:15.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.303+0000 7f666bfff700 1 -- 172.21.15.3:0/539651626 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f666c136710 msgr2=0x7f666c138af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:15.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.303+0000 7f666bfff700 1 --2- 172.21.15.3:0/539651626 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f666c136710 0x7f666c138af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.303+0000 7f666bfff700 1 -- 172.21.15.3:0/539651626 >> v1:172.21.15.3:6789/0 conn(0x7f666c1338a0 legacy=0x7f666c133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:15.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.303+0000 7f666bfff700 1 -- 172.21.15.3:0/539651626 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f666c06fc30 con 0x7f666c139030 2023-10-03T10:19:15.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.303+0000 7f666bfff700 1 --2- 172.21.15.3:0/539651626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f666c139030 0x7f666c13b410 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f6664009d90 tx=0x7f666400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a4ccd02a734645 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:15.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.303+0000 7f666a7fc700 1 -- 172.21.15.3:0/539651626 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666400e070 con 0x7f666c139030 2023-10-03T10:19:15.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.303+0000 7f666a7fc700 1 -- 172.21.15.3:0/539651626 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666400bb20 con 0x7f666c139030 2023-10-03T10:19:15.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.303+0000 7f666a7fc700 1 -- 172.21.15.3:0/539651626 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6664012430 con 0x7f666c139030 2023-10-03T10:19:15.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.304+0000 7f6671b28700 1 -- 172.21.15.3:0/539651626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f666c139030 msgr2=0x7f666c13b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:15.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.304+0000 7f6671b28700 1 --2- 172.21.15.3:0/539651626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f666c139030 0x7f666c13b410 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f6664009d90 tx=0x7f666400b330 comp rx=0 tx=0).stop 2023-10-03T10:19:15.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.304+0000 7f6671b28700 1 -- 172.21.15.3:0/539651626 shutdown_connections 2023-10-03T10:19:15.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.304+0000 7f6671b28700 1 --2- 172.21.15.3:0/539651626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f666c139030 0x7f666c13b410 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.304+0000 7f6671b28700 1 --2- 172.21.15.3:0/539651626 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f666c136710 0x7f666c138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.304+0000 7f6671b28700 1 -- 172.21.15.3:0/539651626 >> 172.21.15.3:0/539651626 conn(0x7f666c0a2aa0 msgr2=0x7f666c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:15.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.304+0000 7f6671b28700 1 -- 172.21.15.3:0/539651626 shutdown_connections 2023-10-03T10:19:15.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.305+0000 7f6671b28700 1 -- 172.21.15.3:0/539651626 wait complete. 2023-10-03T10:19:15.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.306+0000 7f6671b28700 1 Processor -- start 2023-10-03T10:19:15.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.306+0000 7f6671b28700 1 -- start start 2023-10-03T10:19:15.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.306+0000 7f6671b28700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f666c1338a0 0x7f666c11abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:15.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.306+0000 7f6671b28700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f666c136710 0x7f666c11b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:15.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.306+0000 7f6671b28700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f666c139030 0x7f666c11b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:15.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.306+0000 7f6671b28700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666c11e770 con 0x7f666c136710 2023-10-03T10:19:15.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.306+0000 7f6671b28700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f666c11de60 con 0x7f666c139030 2023-10-03T10:19:15.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.307+0000 7f6671b28700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f666c06f640 con 0x7f666c1338a0 2023-10-03T10:19:15.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.307+0000 7f666b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f666c1338a0 0x7f666c11abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:15.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.307+0000 7f666b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f666c1338a0 0x7f666c11abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41538/0 (socket says 172.21.15.3:41538) 2023-10-03T10:19:15.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.307+0000 7f666b7fe700 1 -- 172.21.15.3:0/3763254067 learned_addr learned my addr 172.21.15.3:0/3763254067 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:15.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.307+0000 7f666bfff700 1 --2- 172.21.15.3:0/3763254067 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f666c139030 0x7f666c11b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:15.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.307+0000 7f666affd700 1 --2- 172.21.15.3:0/3763254067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f666c136710 0x7f666c11b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:15.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.307+0000 7f666bfff700 1 -- 172.21.15.3:0/3763254067 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f666c1338a0 msgr2=0x7f666c11abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:15.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.307+0000 7f666bfff700 1 --2- 172.21.15.3:0/3763254067 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f666c1338a0 0x7f666c11abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.307+0000 7f666bfff700 1 -- 172.21.15.3:0/3763254067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f666c136710 msgr2=0x7f666c11b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:15.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.307+0000 7f666bfff700 1 --2- 172.21.15.3:0/3763254067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f666c136710 0x7f666c11b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.307+0000 7f666bfff700 1 -- 172.21.15.3:0/3763254067 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6664009a70 con 0x7f666c139030 2023-10-03T10:19:15.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.307+0000 7f666bfff700 1 --2- 172.21.15.3:0/3763254067 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f666c139030 0x7f666c11b630 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6664016000 tx=0x7f6664004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:15.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.308+0000 7f6653fff700 1 -- 172.21.15.3:0/3763254067 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6664012430 con 0x7f666c139030 2023-10-03T10:19:15.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.308+0000 7f6653fff700 1 -- 172.21.15.3:0/3763254067 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6664020070 con 0x7f666c139030 2023-10-03T10:19:15.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.308+0000 7f6671b28700 1 -- 172.21.15.3:0/3763254067 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f666c138c90 con 0x7f666c139030 2023-10-03T10:19:15.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.308+0000 7f6653fff700 1 -- 172.21.15.3:0/3763254067 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6664004280 con 0x7f666c139030 2023-10-03T10:19:15.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.308+0000 7f6671b28700 1 -- 172.21.15.3:0/3763254067 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f666c11e4a0 con 0x7f666c139030 2023-10-03T10:19:15.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.309+0000 7f6653fff700 1 -- 172.21.15.3:0/3763254067 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f66640043e0 con 0x7f666c139030 2023-10-03T10:19:15.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.309+0000 7f6653fff700 1 -- 172.21.15.3:0/3763254067 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6664022070 con 0x7f666c139030 2023-10-03T10:19:15.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.309+0000 7f6671b28700 1 -- 172.21.15.3:0/3763254067 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f666c11e2c0 con 0x7f666c139030 2023-10-03T10:19:15.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.314+0000 7f6653fff700 1 -- 172.21.15.3:0/3763254067 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6664012e10 con 0x7f666c139030 2023-10-03T10:19:15.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.352+0000 7fadec773700 1 Processor -- start 2023-10-03T10:19:15.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.352+0000 7fadec773700 1 -- start start 2023-10-03T10:19:15.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.353+0000 7fadec773700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fade406a770 0x7fade406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:15.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.353+0000 7fadec773700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fade406b080 0x7fade4131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:15.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.353+0000 7fadec773700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade409ece0 con 0x7fade4131b40 2023-10-03T10:19:15.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.353+0000 7fadec773700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade409e760 con 0x7fade406a770 2023-10-03T10:19:15.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.353+0000 7fadec773700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fade409e040 con 0x7fade406b080 2023-10-03T10:19:15.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.353+0000 7fadea50f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fade406a770 0x7fade406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:15.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.353+0000 7fade9d0e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fade406b080 0x7fade4131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:15.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.353+0000 7fadea50f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fade406a770 0x7fade406ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54488/0 (socket says 172.21.15.3:54488) 2023-10-03T10:19:15.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.353+0000 7fade9d0e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fade406b080 0x7fade4131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41550/0 (socket says 172.21.15.3:41550) 2023-10-03T10:19:15.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.353+0000 7fadea50f700 1 -- 172.21.15.3:0/3372219436 learned_addr learned my addr 172.21.15.3:0/3372219436 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:15.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.353+0000 7fadead10700 1 --1- 172.21.15.3:0/3372219436 >> v1:172.21.15.3:6789/0 conn(0x7fade4131b40 0x7fade4138850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51670/0 (socket says 172.21.15.3:51670) 2023-10-03T10:19:15.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.354+0000 7fade950d700 1 -- 172.21.15.3:0/3372219436 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3137125405 0 0) 0x7fade409ece0 con 0x7fade4131b40 2023-10-03T10:19:15.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.354+0000 7fade950d700 1 -- 172.21.15.3:0/3372219436 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadd00035b0 con 0x7fade4131b40 2023-10-03T10:19:15.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.354+0000 7fadea50f700 1 -- 172.21.15.3:0/3372219436 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fade406b080 msgr2=0x7fade4131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:15.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.354+0000 7fadea50f700 1 --2- 172.21.15.3:0/3372219436 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fade406b080 0x7fade4131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.354+0000 7fadea50f700 1 -- 172.21.15.3:0/3372219436 >> v1:172.21.15.3:6789/0 conn(0x7fade4131b40 legacy=0x7fade4138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:15.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.354+0000 7fadea50f700 1 -- 172.21.15.3:0/3372219436 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade409f630 con 0x7fade406a770 2023-10-03T10:19:15.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.354+0000 7fade9d0e700 1 --2- 172.21.15.3:0/3372219436 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fade406b080 0x7fade4131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:15.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.354+0000 7fadea50f700 1 --2- 172.21.15.3:0/3372219436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fade406a770 0x7fade406ab40 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fade00098f0 tx=0x7fade0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed6ced775f0c423 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:15.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.354+0000 7fade950d700 1 -- 172.21.15.3:0/3372219436 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade0011830 con 0x7fade406a770 2023-10-03T10:19:15.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.354+0000 7fade950d700 1 -- 172.21.15.3:0/3372219436 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade0011990 con 0x7fade406a770 2023-10-03T10:19:15.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.354+0000 7fade950d700 1 -- 172.21.15.3:0/3372219436 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade0011d20 con 0x7fade406a770 2023-10-03T10:19:15.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.355+0000 7fadec773700 1 -- 172.21.15.3:0/3372219436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fade406a770 msgr2=0x7fade406ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:15.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.355+0000 7fadec773700 1 --2- 172.21.15.3:0/3372219436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fade406a770 0x7fade406ab40 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fade00098f0 tx=0x7fade0009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:15.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.355+0000 7fadec773700 1 -- 172.21.15.3:0/3372219436 shutdown_connections 2023-10-03T10:19:15.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.355+0000 7fadec773700 1 --2- 172.21.15.3:0/3372219436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fade406a770 0x7fade406ab40 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.355+0000 7fadec773700 1 --2- 172.21.15.3:0/3372219436 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fade406b080 0x7fade4131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.355+0000 7fadec773700 1 -- 172.21.15.3:0/3372219436 >> 172.21.15.3:0/3372219436 conn(0x7fade4066f50 msgr2=0x7fade409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:15.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.355+0000 7fadec773700 1 -- 172.21.15.3:0/3372219436 shutdown_connections 2023-10-03T10:19:15.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.355+0000 7fadec773700 1 -- 172.21.15.3:0/3372219436 wait complete. 2023-10-03T10:19:15.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.356+0000 7fadec773700 1 Processor -- start 2023-10-03T10:19:15.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.357+0000 7fadec773700 1 -- start start 2023-10-03T10:19:15.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.357+0000 7fadec773700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fade406a770 0x7fade41c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:15.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.357+0000 7fadec773700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fade406b080 0x7fade41c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:15.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.357+0000 7fadec773700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fade4131b40 0x7fade41c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:15.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.357+0000 7fadec773700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade4122ac0 con 0x7fade406a770 2023-10-03T10:19:15.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.357+0000 7fadec773700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade4122eb0 con 0x7fade406b080 2023-10-03T10:19:15.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.357+0000 7fadec773700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fade4122350 con 0x7fade4131b40 2023-10-03T10:19:15.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.357+0000 7fadea50f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fade406a770 0x7fade41c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:15.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.357+0000 7fadea50f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fade406a770 0x7fade41c5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54500/0 (socket says 172.21.15.3:54500) 2023-10-03T10:19:15.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.357+0000 7fadea50f700 1 -- 172.21.15.3:0/824353186 learned_addr learned my addr 172.21.15.3:0/824353186 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:15.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.357+0000 7fadead10700 1 --2- 172.21.15.3:0/824353186 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fade4131b40 0x7fade41c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:15.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.357+0000 7fade9d0e700 1 --2- 172.21.15.3:0/824353186 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fade406b080 0x7fade41c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:15.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.358+0000 7fadea50f700 1 -- 172.21.15.3:0/824353186 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fade4131b40 msgr2=0x7fade41c68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:15.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.358+0000 7fadea50f700 1 --2- 172.21.15.3:0/824353186 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fade4131b40 0x7fade41c68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.358+0000 7fadea50f700 1 -- 172.21.15.3:0/824353186 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fade406b080 msgr2=0x7fade41c6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:15.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.358+0000 7fadea50f700 1 --2- 172.21.15.3:0/824353186 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fade406b080 0x7fade41c6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.358+0000 7fadea50f700 1 -- 172.21.15.3:0/824353186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd40094a0 con 0x7fade406a770 2023-10-03T10:19:15.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.358+0000 7fade9d0e700 1 --2- 172.21.15.3:0/824353186 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fade406b080 0x7fade41c6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:15.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.358+0000 7fadea50f700 1 --2- 172.21.15.3:0/824353186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fade406a770 0x7fade41c5e30 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fade0000c00 tx=0x7fade0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:15.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.358+0000 7faddaffd700 1 -- 172.21.15.3:0/824353186 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade0004380 con 0x7fade406a770 2023-10-03T10:19:15.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.358+0000 7faddaffd700 1 -- 172.21.15.3:0/824353186 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade001f070 con 0x7fade406a770 2023-10-03T10:19:15.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.358+0000 7fadec773700 1 -- 172.21.15.3:0/824353186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade0009550 con 0x7fade406a770 2023-10-03T10:19:15.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.358+0000 7faddaffd700 1 -- 172.21.15.3:0/824353186 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade00036a0 con 0x7fade406a770 2023-10-03T10:19:15.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.358+0000 7fadec773700 1 -- 172.21.15.3:0/824353186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade4123b40 con 0x7fade406a770 2023-10-03T10:19:15.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.360+0000 7faddaffd700 1 -- 172.21.15.3:0/824353186 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fade0021030 con 0x7fade406a770 2023-10-03T10:19:15.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.360+0000 7fadd8ff9700 1 -- 172.21.15.3:0/824353186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc00052e0 con 0x7fade406a770 2023-10-03T10:19:15.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.360+0000 7faddaffd700 1 -- 172.21.15.3:0/824353186 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fade00409e0 con 0x7fade406a770 2023-10-03T10:19:15.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.365+0000 7faddaffd700 1 -- 172.21.15.3:0/824353186 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fade00199a0 con 0x7fade406a770 2023-10-03T10:19:15.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.398+0000 7f53cbc68700 1 Processor -- start 2023-10-03T10:19:15.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.398+0000 7f53cbc68700 1 -- start start 2023-10-03T10:19:15.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.399+0000 7f53cbc68700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f53c409d6f0 0x7f53c409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:15.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.399+0000 7f53cbc68700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f53c409e000 0x7f53c41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:15.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.399+0000 7f53cbc68700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c409b120 con 0x7f53c409e410 2023-10-03T10:19:15.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.399+0000 7f53cbc68700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c406b460 con 0x7f53c409d6f0 2023-10-03T10:19:15.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.399+0000 7f53cbc68700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c409a7e0 con 0x7f53c409e000 2023-10-03T10:19:15.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.399+0000 7f53c9203700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f53c409e000 0x7f53c41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:15.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.399+0000 7f53c9a04700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f53c409d6f0 0x7f53c409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:15.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.399+0000 7f53c9a04700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f53c409d6f0 0x7f53c409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54516/0 (socket says 172.21.15.3:54516) 2023-10-03T10:19:15.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.399+0000 7f53c9203700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f53c409e000 0x7f53c41339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:54186/0 (socket says 172.21.15.3:54186) 2023-10-03T10:19:15.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.399+0000 7f53c9a04700 1 -- 172.21.15.3:0/929741123 learned_addr learned my addr 172.21.15.3:0/929741123 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:15.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.400+0000 7f53c9203700 1 -- 172.21.15.3:0/929741123 >> v2:172.21.15.3:3300/0 conn(0x7f53c409d6f0 msgr2=0x7f53c409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:15.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.400+0000 7f53c9203700 1 --2- 172.21.15.3:0/929741123 >> v2:172.21.15.3:3300/0 conn(0x7f53c409d6f0 0x7f53c409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.400+0000 7f53c9203700 1 -- 172.21.15.3:0/929741123 >> v1:172.21.15.3:6789/0 conn(0x7f53c409e410 legacy=0x7f53c4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:15.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.400+0000 7f53c9203700 1 -- 172.21.15.3:0/929741123 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c409a500 con 0x7f53c409e000 2023-10-03T10:19:15.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.400+0000 7f53c9a04700 1 --2- 172.21.15.3:0/929741123 >> v2:172.21.15.3:3300/0 conn(0x7f53c409d6f0 0x7f53c409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:15.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.400+0000 7f53c9203700 1 --2- 172.21.15.3:0/929741123 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f53c409e000 0x7f53c41339f0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f53b4009870 tx=0x7f53b4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aecd7af4ad5368ab server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:15.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.400+0000 7f53c8a02700 1 -- 172.21.15.3:0/929741123 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b4011830 con 0x7f53c409e000 2023-10-03T10:19:15.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.400+0000 7f53c8a02700 1 -- 172.21.15.3:0/929741123 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b4011990 con 0x7f53c409e000 2023-10-03T10:19:15.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.400+0000 7f53c8a02700 1 -- 172.21.15.3:0/929741123 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b4011d20 con 0x7f53c409e000 2023-10-03T10:19:15.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.401+0000 7f53cbc68700 1 -- 172.21.15.3:0/929741123 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f53c409e000 msgr2=0x7f53c41339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:15.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.401+0000 7f53cbc68700 1 --2- 172.21.15.3:0/929741123 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f53c409e000 0x7f53c41339f0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f53b4009870 tx=0x7f53b4009b80 comp rx=0 tx=0).stop 2023-10-03T10:19:15.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.401+0000 7f53cbc68700 1 -- 172.21.15.3:0/929741123 shutdown_connections 2023-10-03T10:19:15.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.401+0000 7f53cbc68700 1 --2- 172.21.15.3:0/929741123 >> v2:172.21.15.3:3300/0 conn(0x7f53c409d6f0 0x7f53c409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.401+0000 7f53cbc68700 1 --2- 172.21.15.3:0/929741123 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f53c409e000 0x7f53c41339f0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.401+0000 7f53cbc68700 1 -- 172.21.15.3:0/929741123 >> 172.21.15.3:0/929741123 conn(0x7f53c4066f50 msgr2=0x7f53c409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:15.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.401+0000 7f53cbc68700 1 -- 172.21.15.3:0/929741123 shutdown_connections 2023-10-03T10:19:15.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.401+0000 7f53cbc68700 1 -- 172.21.15.3:0/929741123 wait complete. 2023-10-03T10:19:15.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.402+0000 7f53cbc68700 1 Processor -- start 2023-10-03T10:19:15.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.403+0000 7f53cbc68700 1 -- start start 2023-10-03T10:19:15.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.403+0000 7f53cbc68700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53c409d6f0 0x7f53c41a5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:15.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.403+0000 7f53cbc68700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f53c409e000 0x7f53c41a6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:15.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.403+0000 7f53cbc68700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53c409e410 0x7f53c41a6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:15.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.403+0000 7f53cbc68700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c41a7490 con 0x7f53c409d6f0 2023-10-03T10:19:15.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.403+0000 7f53cbc68700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c41a7ba0 con 0x7f53c409e000 2023-10-03T10:19:15.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.403+0000 7f53cbc68700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53c41a8df0 con 0x7f53c409e410 2023-10-03T10:19:15.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.404+0000 7f53c9a04700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53c409d6f0 0x7f53c41a5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:15.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.404+0000 7f53c9a04700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53c409d6f0 0x7f53c41a5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54530/0 (socket says 172.21.15.3:54530) 2023-10-03T10:19:15.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.404+0000 7f53c9a04700 1 -- 172.21.15.3:0/2676481459 learned_addr learned my addr 172.21.15.3:0/2676481459 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:15.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.404+0000 7f53c9203700 1 --2- 172.21.15.3:0/2676481459 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f53c409e000 0x7f53c41a6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:15.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.404+0000 7f53c9203700 1 -- 172.21.15.3:0/2676481459 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53c409e410 msgr2=0x7f53c41a6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:15.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.404+0000 7f53c9203700 1 --2- 172.21.15.3:0/2676481459 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53c409e410 0x7f53c41a6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.404+0000 7f53c9203700 1 -- 172.21.15.3:0/2676481459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53c409d6f0 msgr2=0x7f53c41a5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:15.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.404+0000 7f53c9203700 1 --2- 172.21.15.3:0/2676481459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53c409d6f0 0x7f53c41a5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.404+0000 7f53c9203700 1 -- 172.21.15.3:0/2676481459 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c0009550 con 0x7f53c409e000 2023-10-03T10:19:15.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.404+0000 7f53c9a04700 1 --2- 172.21.15.3:0/2676481459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53c409d6f0 0x7f53c41a5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:15.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.404+0000 7f53c9203700 1 --2- 172.21.15.3:0/2676481459 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f53c409e000 0x7f53c41a6520 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f53b4016000 tx=0x7f53b4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:15.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.404+0000 7f53ba7fc700 1 -- 172.21.15.3:0/2676481459 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b40036a0 con 0x7f53c409e000 2023-10-03T10:19:15.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.404+0000 7f53ba7fc700 1 -- 172.21.15.3:0/2676481459 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b401f070 con 0x7f53c409e000 2023-10-03T10:19:15.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.404+0000 7f53ba7fc700 1 -- 172.21.15.3:0/2676481459 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b4003a00 con 0x7f53c409e000 2023-10-03T10:19:15.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.405+0000 7f53cbc68700 1 -- 172.21.15.3:0/2676481459 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b40094a0 con 0x7f53c409e000 2023-10-03T10:19:15.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.405+0000 7f53cbc68700 1 -- 172.21.15.3:0/2676481459 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c40a31b0 con 0x7f53c409e000 2023-10-03T10:19:15.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.406+0000 7f53ba7fc700 1 -- 172.21.15.3:0/2676481459 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f53b4003b60 con 0x7f53c409e000 2023-10-03T10:19:15.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.406+0000 7f53ba7fc700 1 -- 172.21.15.3:0/2676481459 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f53b4021070 con 0x7f53c409e000 2023-10-03T10:19:15.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.406+0000 7f53cbc68700 1 -- 172.21.15.3:0/2676481459 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c41a8180 con 0x7f53c409e000 2023-10-03T10:19:15.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.411+0000 7f53ba7fc700 1 -- 172.21.15.3:0/2676481459 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f53b4015070 con 0x7f53c409e000 2023-10-03T10:19:15.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.446+0000 7ff78cd6f700 1 Processor -- start 2023-10-03T10:19:15.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.446+0000 7ff78cd6f700 1 -- start start 2023-10-03T10:19:15.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.447+0000 7ff78cd6f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78809d6f0 0x7ff78809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:15.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.447+0000 7ff78cd6f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff78809e000 0x7ff7881339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:15.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.447+0000 7ff78cd6f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78809e410 0x7ff788133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:15.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.447+0000 7ff78cd6f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff78806b460 con 0x7ff78809e000 2023-10-03T10:19:15.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.447+0000 7ff78cd6f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78809b120 con 0x7ff78809e410 2023-10-03T10:19:15.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.447+0000 7ff78cd6f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff78809a7e0 con 0x7ff78809d6f0 2023-10-03T10:19:15.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.447+0000 7ff785d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff78809e000 0x7ff7881339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:15.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.447+0000 7ff78659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78809d6f0 0x7ff78809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:15.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.447+0000 7ff78659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78809d6f0 0x7ff78809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41596/0 (socket says 172.21.15.3:41596) 2023-10-03T10:19:15.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.447+0000 7ff785d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff78809e000 0x7ff7881339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54534/0 (socket says 172.21.15.3:54534) 2023-10-03T10:19:15.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.447+0000 7ff785d9b700 1 -- 172.21.15.3:0/708806909 learned_addr learned my addr 172.21.15.3:0/708806909 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:15.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.448+0000 7ff786d9d700 1 --2- 172.21.15.3:0/708806909 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78809e410 0x7ff788133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:15.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.448+0000 7ff785d9b700 1 -- 172.21.15.3:0/708806909 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78809d6f0 msgr2=0x7ff78809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:15.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.448+0000 7ff785d9b700 1 --2- 172.21.15.3:0/708806909 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78809d6f0 0x7ff78809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.448+0000 7ff785d9b700 1 -- 172.21.15.3:0/708806909 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78809e410 msgr2=0x7ff788133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:15.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.448+0000 7ff785d9b700 1 --2- 172.21.15.3:0/708806909 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78809e410 0x7ff788133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.448+0000 7ff785d9b700 1 -- 172.21.15.3:0/708806909 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78809a500 con 0x7ff78809e000 2023-10-03T10:19:15.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.448+0000 7ff786d9d700 1 --2- 172.21.15.3:0/708806909 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78809e410 0x7ff788133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:19:15.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.448+0000 7ff785d9b700 1 --2- 172.21.15.3:0/708806909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78809e000 0x7ff7881339f0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0098f0 tx=0x7ff77c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791ea8c4333b1ac9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:15.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.448+0000 7ff78559a700 1 -- 172.21.15.3:0/708806909 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c011830 con 0x7ff78809e000 2023-10-03T10:19:15.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.448+0000 7ff78559a700 1 -- 172.21.15.3:0/708806909 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77c011990 con 0x7ff78809e000 2023-10-03T10:19:15.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.448+0000 7ff78559a700 1 -- 172.21.15.3:0/708806909 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c011d20 con 0x7ff78809e000 2023-10-03T10:19:15.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.449+0000 7ff78cd6f700 1 -- 172.21.15.3:0/708806909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78809e000 msgr2=0x7ff7881339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:15.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.449+0000 7ff78cd6f700 1 --2- 172.21.15.3:0/708806909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78809e000 0x7ff7881339f0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0098f0 tx=0x7ff77c009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:15.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.449+0000 7ff78cd6f700 1 -- 172.21.15.3:0/708806909 shutdown_connections 2023-10-03T10:19:15.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.449+0000 7ff78cd6f700 1 --2- 172.21.15.3:0/708806909 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78809e410 0x7ff788133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.449+0000 7ff78cd6f700 1 --2- 172.21.15.3:0/708806909 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78809d6f0 0x7ff78809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.449+0000 7ff78cd6f700 1 --2- 172.21.15.3:0/708806909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78809e000 0x7ff7881339f0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.449+0000 7ff78cd6f700 1 -- 172.21.15.3:0/708806909 >> 172.21.15.3:0/708806909 conn(0x7ff788066f50 msgr2=0x7ff78809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:15.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.449+0000 7ff78cd6f700 1 -- 172.21.15.3:0/708806909 shutdown_connections 2023-10-03T10:19:15.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.449+0000 7ff78cd6f700 1 -- 172.21.15.3:0/708806909 wait complete. 2023-10-03T10:19:15.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.451+0000 7ff78cd6f700 1 Processor -- start 2023-10-03T10:19:15.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.451+0000 7ff78cd6f700 1 -- start start 2023-10-03T10:19:15.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.452+0000 7ff78cd6f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78809d6f0 0x7ff788122650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:15.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.452+0000 7ff78cd6f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78809e000 0x7ff788122b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:15.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.452+0000 7ff78cd6f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78809e410 0x7ff7881230d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:15.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.452+0000 7ff78cd6f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff788125e90 con 0x7ff78809e000 2023-10-03T10:19:15.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.452+0000 7ff78cd6f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff788124370 con 0x7ff78809d6f0 2023-10-03T10:19:15.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.452+0000 7ff78cd6f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff78809c710 con 0x7ff78809e410 2023-10-03T10:19:15.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.452+0000 7ff78659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78809d6f0 0x7ff788122650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:15.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.452+0000 7ff785d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78809e000 0x7ff788122b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:15.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.452+0000 7ff78659c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78809d6f0 0x7ff788122650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:54220/0 (socket says 172.21.15.3:54220) 2023-10-03T10:19:15.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.452+0000 7ff78659c700 1 -- 172.21.15.3:0/3371517975 learned_addr learned my addr 172.21.15.3:0/3371517975 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:15.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.452+0000 7ff786d9d700 1 --2- 172.21.15.3:0/3371517975 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78809e410 0x7ff7881230d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:15.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.453+0000 7ff785d9b700 1 -- 172.21.15.3:0/3371517975 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78809e410 msgr2=0x7ff7881230d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:15.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.453+0000 7ff785d9b700 1 --2- 172.21.15.3:0/3371517975 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78809e410 0x7ff7881230d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.453+0000 7ff785d9b700 1 -- 172.21.15.3:0/3371517975 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78809d6f0 msgr2=0x7ff788122650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:15.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.453+0000 7ff785d9b700 1 --2- 172.21.15.3:0/3371517975 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78809d6f0 0x7ff788122650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.453+0000 7ff785d9b700 1 -- 172.21.15.3:0/3371517975 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77c009550 con 0x7ff78809e000 2023-10-03T10:19:15.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.453+0000 7ff78659c700 1 --2- 172.21.15.3:0/3371517975 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78809d6f0 0x7ff788122650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:15.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.453+0000 7ff785d9b700 1 --2- 172.21.15.3:0/3371517975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78809e000 0x7ff788122b90 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7ff78806e510 tx=0x7ff77c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:15.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.453+0000 7ff776ffd700 1 -- 172.21.15.3:0/3371517975 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c004300 con 0x7ff78809e000 2023-10-03T10:19:15.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.453+0000 7ff776ffd700 1 -- 172.21.15.3:0/3371517975 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77c01f070 con 0x7ff78809e000 2023-10-03T10:19:15.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.453+0000 7ff776ffd700 1 -- 172.21.15.3:0/3371517975 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c0036a0 con 0x7ff78809e000 2023-10-03T10:19:15.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.453+0000 7ff78cd6f700 1 -- 172.21.15.3:0/3371517975 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7880a1d00 con 0x7ff78809e000 2023-10-03T10:19:15.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.453+0000 7ff78cd6f700 1 -- 172.21.15.3:0/3371517975 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff788126900 con 0x7ff78809e000 2023-10-03T10:19:15.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.454+0000 7ff776ffd700 1 -- 172.21.15.3:0/3371517975 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff77c021030 con 0x7ff78809e000 2023-10-03T10:19:15.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.455+0000 7ff78cd6f700 1 -- 172.21.15.3:0/3371517975 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7880729a0 con 0x7ff78809e000 2023-10-03T10:19:15.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.455+0000 7ff776ffd700 1 -- 172.21.15.3:0/3371517975 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff77c045020 con 0x7ff78809e000 2023-10-03T10:19:15.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.460+0000 7ff776ffd700 1 -- 172.21.15.3:0/3371517975 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff77c015020 con 0x7ff78809e000 2023-10-03T10:19:15.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.536+0000 7f6671b28700 1 -- 172.21.15.3:0/3763254067 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f666c074ea0 con 0x7f666c139030 2023-10-03T10:19:15.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.536+0000 7f6653fff700 1 -- 172.21.15.3:0/3763254067 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6664011390 con 0x7f666c139030 2023-10-03T10:19:15.536 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:15.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.538+0000 7f6671b28700 1 -- 172.21.15.3:0/3763254067 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f666c139030 msgr2=0x7f666c11b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:15.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.538+0000 7f6671b28700 1 --2- 172.21.15.3:0/3763254067 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f666c139030 0x7f666c11b630 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6664016000 tx=0x7f6664004ad0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.539+0000 7f6671b28700 1 -- 172.21.15.3:0/3763254067 shutdown_connections 2023-10-03T10:19:15.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.539+0000 7f6671b28700 1 --2- 172.21.15.3:0/3763254067 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f666c139030 0x7f666c11b630 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.539+0000 7f6671b28700 1 --2- 172.21.15.3:0/3763254067 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f666c1338a0 0x7f666c11abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.539+0000 7f6671b28700 1 --2- 172.21.15.3:0/3763254067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f666c136710 0x7f666c11b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.539+0000 7f6671b28700 1 -- 172.21.15.3:0/3763254067 >> 172.21.15.3:0/3763254067 conn(0x7f666c0a2aa0 msgr2=0x7f666c1344d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:15.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.539+0000 7f6671b28700 1 -- 172.21.15.3:0/3763254067 shutdown_connections 2023-10-03T10:19:15.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.539+0000 7f6671b28700 1 -- 172.21.15.3:0/3763254067 wait complete. 2023-10-03T10:19:15.553 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:15.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.626+0000 7f53cbc68700 1 -- 172.21.15.3:0/2676481459 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f53c4074ea0 con 0x7f53c409e000 2023-10-03T10:19:15.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.626+0000 7f53ba7fc700 1 -- 172.21.15.3:0/2676481459 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f53b4003e10 con 0x7f53c409e000 2023-10-03T10:19:15.627 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:15.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.629+0000 7f53cbc68700 1 -- 172.21.15.3:0/2676481459 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f53c409e000 msgr2=0x7f53c41a6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:15.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.629+0000 7f53cbc68700 1 --2- 172.21.15.3:0/2676481459 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f53c409e000 0x7f53c41a6520 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f53b4016000 tx=0x7f53b4004750 comp rx=0 tx=0).stop 2023-10-03T10:19:15.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.629+0000 7f53cbc68700 1 -- 172.21.15.3:0/2676481459 shutdown_connections 2023-10-03T10:19:15.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.629+0000 7f53cbc68700 1 --2- 172.21.15.3:0/2676481459 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f53c409e410 0x7f53c41a6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.629+0000 7f53cbc68700 1 --2- 172.21.15.3:0/2676481459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53c409d6f0 0x7f53c41a5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.629+0000 7f53cbc68700 1 --2- 172.21.15.3:0/2676481459 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f53c409e000 0x7f53c41a6520 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.629+0000 7f53cbc68700 1 -- 172.21.15.3:0/2676481459 >> 172.21.15.3:0/2676481459 conn(0x7f53c4066f50 msgr2=0x7f53c41baf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:15.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.629+0000 7f53cbc68700 1 -- 172.21.15.3:0/2676481459 shutdown_connections 2023-10-03T10:19:15.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.629+0000 7f53cbc68700 1 -- 172.21.15.3:0/2676481459 wait complete. 2023-10-03T10:19:15.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.631+0000 7fadd8ff9700 1 -- 172.21.15.3:0/824353186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fadc0005590 con 0x7fade406a770 2023-10-03T10:19:15.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.631+0000 7faddaffd700 1 -- 172.21.15.3:0/824353186 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fade00193b0 con 0x7fade406a770 2023-10-03T10:19:15.632 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:15.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.634+0000 7fadd8ff9700 1 -- 172.21.15.3:0/824353186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fade406a770 msgr2=0x7fade41c5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:15.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.634+0000 7fadd8ff9700 1 --2- 172.21.15.3:0/824353186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fade406a770 0x7fade41c5e30 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fade0000c00 tx=0x7fade0004750 comp rx=0 tx=0).stop 2023-10-03T10:19:15.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.634+0000 7fadd8ff9700 1 -- 172.21.15.3:0/824353186 shutdown_connections 2023-10-03T10:19:15.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.634+0000 7fadd8ff9700 1 --2- 172.21.15.3:0/824353186 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fade4131b40 0x7fade41c68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.634+0000 7fadd8ff9700 1 --2- 172.21.15.3:0/824353186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fade406a770 0x7fade41c5e30 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.634+0000 7fadd8ff9700 1 --2- 172.21.15.3:0/824353186 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fade406b080 0x7fade41c6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.634+0000 7fadd8ff9700 1 -- 172.21.15.3:0/824353186 >> 172.21.15.3:0/824353186 conn(0x7fade4066f50 msgr2=0x7fade406ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:15.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.636+0000 7fadd8ff9700 1 -- 172.21.15.3:0/824353186 shutdown_connections 2023-10-03T10:19:15.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.636+0000 7fadd8ff9700 1 -- 172.21.15.3:0/824353186 wait complete. 2023-10-03T10:19:15.644 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:15.650 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:15.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.716+0000 7ff78cd6f700 1 -- 172.21.15.3:0/3371517975 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff788074ea0 con 0x7ff78809e000 2023-10-03T10:19:15.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.716+0000 7ff776ffd700 1 -- 172.21.15.3:0/3371517975 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff77c00b9d0 con 0x7ff78809e000 2023-10-03T10:19:15.717 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:15.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.718+0000 7ff78cd6f700 1 -- 172.21.15.3:0/3371517975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78809e000 msgr2=0x7ff788122b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:15.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.718+0000 7ff78cd6f700 1 --2- 172.21.15.3:0/3371517975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78809e000 0x7ff788122b90 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7ff78806e510 tx=0x7ff77c004750 comp rx=0 tx=0).stop 2023-10-03T10:19:15.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.718+0000 7ff78cd6f700 1 -- 172.21.15.3:0/3371517975 shutdown_connections 2023-10-03T10:19:15.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.718+0000 7ff78cd6f700 1 --2- 172.21.15.3:0/3371517975 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff78809e410 0x7ff7881230d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.718+0000 7ff78cd6f700 1 --2- 172.21.15.3:0/3371517975 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff78809d6f0 0x7ff788122650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.718+0000 7ff78cd6f700 1 --2- 172.21.15.3:0/3371517975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff78809e000 0x7ff788122b90 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:15.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.718+0000 7ff78cd6f700 1 -- 172.21.15.3:0/3371517975 >> 172.21.15.3:0/3371517975 conn(0x7ff788066f50 msgr2=0x7ff7880a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:15.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.718+0000 7ff78cd6f700 1 -- 172.21.15.3:0/3371517975 shutdown_connections 2023-10-03T10:19:15.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:15.718+0000 7ff78cd6f700 1 -- 172.21.15.3:0/3371517975 wait complete. 2023-10-03T10:19:15.732 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:16.554 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:16.644 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:16.651 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:16.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.701+0000 7ff1e086b700 1 Processor -- start 2023-10-03T10:19:16.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.701+0000 7ff1e086b700 1 -- start start 2023-10-03T10:19:16.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.702+0000 7ff1e086b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff1dc09d6f0 0x7ff1dc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:16.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.702+0000 7ff1e086b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1dc09e000 0x7ff1dc1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:16.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.702+0000 7ff1e086b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1dc09e410 0x7ff1dc133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:16.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.702+0000 7ff1e086b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1dc06b460 con 0x7ff1dc09d6f0 2023-10-03T10:19:16.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.702+0000 7ff1e086b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc09b120 con 0x7ff1dc09e000 2023-10-03T10:19:16.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.702+0000 7ff1e086b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc09a7e0 con 0x7ff1dc09e410 2023-10-03T10:19:16.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.702+0000 7ff1d9d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1dc09e000 0x7ff1dc1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:16.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.702+0000 7ff1d9d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1dc09e000 0x7ff1dc1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:54236/0 (socket says 172.21.15.3:54236) 2023-10-03T10:19:16.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.702+0000 7ff1d9d9b700 1 -- 172.21.15.3:0/3873943672 learned_addr learned my addr 172.21.15.3:0/3873943672 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:16.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.702+0000 7ff1dad9d700 1 --2- 172.21.15.3:0/3873943672 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1dc09e410 0x7ff1dc133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:16.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.702+0000 7ff1da59c700 1 --2- 172.21.15.3:0/3873943672 >> v2:172.21.15.3:3300/0 conn(0x7ff1dc09d6f0 0x7ff1dc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:16.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.702+0000 7ff1d9d9b700 1 -- 172.21.15.3:0/3873943672 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1dc09e410 msgr2=0x7ff1dc133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:16.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.702+0000 7ff1d9d9b700 1 --2- 172.21.15.3:0/3873943672 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1dc09e410 0x7ff1dc133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:16.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.702+0000 7ff1d9d9b700 1 -- 172.21.15.3:0/3873943672 >> v2:172.21.15.3:3300/0 conn(0x7ff1dc09d6f0 msgr2=0x7ff1dc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:16.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.702+0000 7ff1d9d9b700 1 --2- 172.21.15.3:0/3873943672 >> v2:172.21.15.3:3300/0 conn(0x7ff1dc09d6f0 0x7ff1dc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:16.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.702+0000 7ff1d9d9b700 1 -- 172.21.15.3:0/3873943672 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1dc09a500 con 0x7ff1dc09e000 2023-10-03T10:19:16.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.702+0000 7ff1dad9d700 1 --2- 172.21.15.3:0/3873943672 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1dc09e410 0x7ff1dc133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:16.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.702+0000 7ff1da59c700 1 --2- 172.21.15.3:0/3873943672 >> v2:172.21.15.3:3300/0 conn(0x7ff1dc09d6f0 0x7ff1dc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:16.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.703+0000 7ff1d9d9b700 1 --2- 172.21.15.3:0/3873943672 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1dc09e000 0x7ff1dc1339f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009920 tx=0x7ff1d0009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b53762f339a7fbf6 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:16.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.703+0000 7ff1d959a700 1 -- 172.21.15.3:0/3873943672 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0011830 con 0x7ff1dc09e000 2023-10-03T10:19:16.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.703+0000 7ff1d959a700 1 -- 172.21.15.3:0/3873943672 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d0011990 con 0x7ff1dc09e000 2023-10-03T10:19:16.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.703+0000 7ff1d959a700 1 -- 172.21.15.3:0/3873943672 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0011d20 con 0x7ff1dc09e000 2023-10-03T10:19:16.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.704+0000 7ff1e086b700 1 -- 172.21.15.3:0/3873943672 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1dc09e000 msgr2=0x7ff1dc1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:16.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.704+0000 7ff1e086b700 1 --2- 172.21.15.3:0/3873943672 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1dc09e000 0x7ff1dc1339f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009920 tx=0x7ff1d0009c30 comp rx=0 tx=0).stop 2023-10-03T10:19:16.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.704+0000 7ff1e086b700 1 -- 172.21.15.3:0/3873943672 shutdown_connections 2023-10-03T10:19:16.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.704+0000 7ff1e086b700 1 --2- 172.21.15.3:0/3873943672 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1dc09e410 0x7ff1dc133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:16.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.704+0000 7ff1e086b700 1 --2- 172.21.15.3:0/3873943672 >> v2:172.21.15.3:3300/0 conn(0x7ff1dc09d6f0 0x7ff1dc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:16.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.704+0000 7ff1e086b700 1 --2- 172.21.15.3:0/3873943672 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1dc09e000 0x7ff1dc1339f0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:16.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.704+0000 7ff1e086b700 1 -- 172.21.15.3:0/3873943672 >> 172.21.15.3:0/3873943672 conn(0x7ff1dc066f50 msgr2=0x7ff1dc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:16.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.704+0000 7ff1e086b700 1 -- 172.21.15.3:0/3873943672 shutdown_connections 2023-10-03T10:19:16.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.704+0000 7ff1e086b700 1 -- 172.21.15.3:0/3873943672 wait complete. 2023-10-03T10:19:16.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.706+0000 7ff1e086b700 1 Processor -- start 2023-10-03T10:19:16.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.706+0000 7ff1e086b700 1 -- start start 2023-10-03T10:19:16.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.706+0000 7ff1e086b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1dc09d6f0 0x7ff1dc1a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:16.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.706+0000 7ff1e086b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1dc09e000 0x7ff1dc1a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:16.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.706+0000 7ff1e086b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1dc09e410 0x7ff1dc1a6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:16.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.706+0000 7ff1e086b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc1a98e0 con 0x7ff1dc09e000 2023-10-03T10:19:16.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.706+0000 7ff1e086b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc1a7ff0 con 0x7ff1dc09e410 2023-10-03T10:19:16.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.706+0000 7ff1e086b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc1aa280 con 0x7ff1dc09d6f0 2023-10-03T10:19:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.707+0000 7ff1da59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1dc09d6f0 0x7ff1dc1a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.707+0000 7ff1da59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1dc09d6f0 0x7ff1dc1a5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41630/0 (socket says 172.21.15.3:41630) 2023-10-03T10:19:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.707+0000 7ff1da59c700 1 -- 172.21.15.3:0/3011045976 learned_addr learned my addr 172.21.15.3:0/3011045976 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.707+0000 7ff1da59c700 1 -- 172.21.15.3:0/3011045976 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1dc09e410 msgr2=0x7ff1dc1a6840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:19:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.707+0000 7ff1da59c700 1 --2- 172.21.15.3:0/3011045976 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1dc09e410 0x7ff1dc1a6840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.707+0000 7ff1da59c700 1 -- 172.21.15.3:0/3011045976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1dc09e000 msgr2=0x7ff1dc1a6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.707+0000 7ff1da59c700 1 --2- 172.21.15.3:0/3011045976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1dc09e000 0x7ff1dc1a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.707+0000 7ff1da59c700 1 -- 172.21.15.3:0/3011045976 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1cc0094a0 con 0x7ff1dc09d6f0 2023-10-03T10:19:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.707+0000 7ff1da59c700 1 --2- 172.21.15.3:0/3011045976 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1dc09d6f0 0x7ff1dc1a5dc0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00ebe0 tx=0x7ff1cc009ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.707+0000 7ff1caffd700 1 -- 172.21.15.3:0/3011045976 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc00c910 con 0x7ff1dc09d6f0 2023-10-03T10:19:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.708+0000 7ff1caffd700 1 -- 172.21.15.3:0/3011045976 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1cc00ca70 con 0x7ff1dc09d6f0 2023-10-03T10:19:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.708+0000 7ff1caffd700 1 -- 172.21.15.3:0/3011045976 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc00cde0 con 0x7ff1dc09d6f0 2023-10-03T10:19:16.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.708+0000 7ff1e086b700 1 -- 172.21.15.3:0/3011045976 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d4009a70 con 0x7ff1dc09d6f0 2023-10-03T10:19:16.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.708+0000 7ff1e086b700 1 -- 172.21.15.3:0/3011045976 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d0009550 con 0x7ff1dc09d6f0 2023-10-03T10:19:16.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.709+0000 7ff1caffd700 1 -- 172.21.15.3:0/3011045976 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff1cc013470 con 0x7ff1dc09d6f0 2023-10-03T10:19:16.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.709+0000 7ff1caffd700 1 -- 172.21.15.3:0/3011045976 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff1cc03fd30 con 0x7ff1dc09d6f0 2023-10-03T10:19:16.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.709+0000 7ff1e086b700 1 -- 172.21.15.3:0/3011045976 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1dc074ea0 con 0x7ff1dc09d6f0 2023-10-03T10:19:16.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.714+0000 7ff1caffd700 1 -- 172.21.15.3:0/3011045976 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff1cc021080 con 0x7ff1dc09d6f0 2023-10-03T10:19:16.733 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:16.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.802+0000 7fcfd4de8700 1 Processor -- start 2023-10-03T10:19:16.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.802+0000 7fcfd4de8700 1 -- start start 2023-10-03T10:19:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.803+0000 7fcfd4de8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd009d630 0x7fcfd00a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:16.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.803+0000 7fcfd4de8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcfd0138d80 0x7fcfd013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:16.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.803+0000 7fcfd4de8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfd00a2a70 con 0x7fcfd00a0a30 2023-10-03T10:19:16.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.803+0000 7fcfd4de8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd00a01d0 con 0x7fcfd0138d80 2023-10-03T10:19:16.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.803+0000 7fcfd4de8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd009e2b0 con 0x7fcfd009d630 2023-10-03T10:19:16.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.803+0000 7fcfcdd9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd009d630 0x7fcfd00a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:16.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.803+0000 7fcfcdd9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd009d630 0x7fcfd00a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41634/0 (socket says 172.21.15.3:41634) 2023-10-03T10:19:16.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.803+0000 7fcfcdd9b700 1 -- 172.21.15.3:0/2689569610 learned_addr learned my addr 172.21.15.3:0/2689569610 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:16.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.803+0000 7fcfced9d700 1 --2- 172.21.15.3:0/2689569610 >> v2:172.21.15.3:3300/0 conn(0x7fcfd0138d80 0x7fcfd013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:16.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.804+0000 7fcfcdd9b700 1 -- 172.21.15.3:0/2689569610 >> v2:172.21.15.3:3300/0 conn(0x7fcfd0138d80 msgr2=0x7fcfd013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:16.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.804+0000 7fcfcdd9b700 1 --2- 172.21.15.3:0/2689569610 >> v2:172.21.15.3:3300/0 conn(0x7fcfd0138d80 0x7fcfd013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:16.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.804+0000 7fcfcdd9b700 1 -- 172.21.15.3:0/2689569610 >> v1:172.21.15.3:6789/0 conn(0x7fcfd00a0a30 legacy=0x7fcfd00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:16.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.804+0000 7fcfcdd9b700 1 -- 172.21.15.3:0/2689569610 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd009dc50 con 0x7fcfd009d630 2023-10-03T10:19:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.804+0000 7fcfcdd9b700 1 --2- 172.21.15.3:0/2689569610 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd009d630 0x7fcfd00a1510 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fcfc40098f0 tx=0x7fcfc4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=970994540fae42c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.804+0000 7fcfcd59a700 1 -- 172.21.15.3:0/2689569610 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4011830 con 0x7fcfd009d630 2023-10-03T10:19:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.804+0000 7fcfcd59a700 1 -- 172.21.15.3:0/2689569610 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc4011990 con 0x7fcfd009d630 2023-10-03T10:19:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.804+0000 7fcfcd59a700 1 -- 172.21.15.3:0/2689569610 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4011d20 con 0x7fcfd009d630 2023-10-03T10:19:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.805+0000 7fcfd4de8700 1 -- 172.21.15.3:0/2689569610 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd009d630 msgr2=0x7fcfd00a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.805+0000 7fcfd4de8700 1 --2- 172.21.15.3:0/2689569610 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd009d630 0x7fcfd00a1510 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fcfc40098f0 tx=0x7fcfc4009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.805+0000 7fcfd4de8700 1 -- 172.21.15.3:0/2689569610 shutdown_connections 2023-10-03T10:19:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.805+0000 7fcfd4de8700 1 --2- 172.21.15.3:0/2689569610 >> v2:172.21.15.3:3300/0 conn(0x7fcfd0138d80 0x7fcfd013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.805+0000 7fcfd4de8700 1 --2- 172.21.15.3:0/2689569610 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd009d630 0x7fcfd00a1510 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.805+0000 7fcfd4de8700 1 -- 172.21.15.3:0/2689569610 >> 172.21.15.3:0/2689569610 conn(0x7fcfd00a3050 msgr2=0x7fcfd00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.805+0000 7fcfd4de8700 1 -- 172.21.15.3:0/2689569610 shutdown_connections 2023-10-03T10:19:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.805+0000 7fcfd4de8700 1 -- 172.21.15.3:0/2689569610 wait complete. 2023-10-03T10:19:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.807+0000 7fcfd4de8700 1 Processor -- start 2023-10-03T10:19:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.807+0000 7fcfd4de8700 1 -- start start 2023-10-03T10:19:16.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.808+0000 7fcfd4de8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfd009d630 0x7fcfd0133b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:16.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.808+0000 7fcfd4de8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd00a0a30 0x7fcfd01340b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:16.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.808+0000 7fcfd4de8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd0138d80 0x7fcfd01345f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:16.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.808+0000 7fcfd4de8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd0137560 con 0x7fcfd009d630 2023-10-03T10:19:16.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.808+0000 7fcfd4de8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd013af80 con 0x7fcfd00a0a30 2023-10-03T10:19:16.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.808+0000 7fcfd4de8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd00a0410 con 0x7fcfd0138d80 2023-10-03T10:19:16.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.808+0000 7fcfcdd9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd00a0a30 0x7fcfd01340b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:16.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.808+0000 7fcfcdd9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd00a0a30 0x7fcfd01340b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:54244/0 (socket says 172.21.15.3:54244) 2023-10-03T10:19:16.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.808+0000 7fcfcdd9b700 1 -- 172.21.15.3:0/3142206443 learned_addr learned my addr 172.21.15.3:0/3142206443 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:16.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.808+0000 7fcfced9d700 1 --2- 172.21.15.3:0/3142206443 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd0138d80 0x7fcfd01345f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:16.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.809+0000 7fcfce59c700 1 --2- 172.21.15.3:0/3142206443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfd009d630 0x7fcfd0133b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:16.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.809+0000 7fcfcdd9b700 1 -- 172.21.15.3:0/3142206443 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd0138d80 msgr2=0x7fcfd01345f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:16.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.809+0000 7fcfcdd9b700 1 --2- 172.21.15.3:0/3142206443 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd0138d80 0x7fcfd01345f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:16.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.809+0000 7fcfcdd9b700 1 -- 172.21.15.3:0/3142206443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfd009d630 msgr2=0x7fcfd0133b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:16.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.809+0000 7fcfcdd9b700 1 --2- 172.21.15.3:0/3142206443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfd009d630 0x7fcfd0133b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:16.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.809+0000 7fcfcdd9b700 1 -- 172.21.15.3:0/3142206443 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc4009550 con 0x7fcfd00a0a30 2023-10-03T10:19:16.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.809+0000 7fcfcdd9b700 1 --2- 172.21.15.3:0/3142206443 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd00a0a30 0x7fcfd01340b0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4000c00 tx=0x7fcfc4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:16.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.809+0000 7fcfbeffd700 1 -- 172.21.15.3:0/3142206443 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4004570 con 0x7fcfd00a0a30 2023-10-03T10:19:16.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.809+0000 7fcfbeffd700 1 -- 172.21.15.3:0/3142206443 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc401f070 con 0x7fcfd00a0a30 2023-10-03T10:19:16.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.809+0000 7fcfbeffd700 1 -- 172.21.15.3:0/3142206443 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc40038e0 con 0x7fcfd00a0a30 2023-10-03T10:19:16.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.810+0000 7fcfd4de8700 1 -- 172.21.15.3:0/3142206443 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd01381e0 con 0x7fcfd00a0a30 2023-10-03T10:19:16.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.810+0000 7fcfd4de8700 1 -- 172.21.15.3:0/3142206443 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd0136cb0 con 0x7fcfd00a0a30 2023-10-03T10:19:16.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.811+0000 7fcfbeffd700 1 -- 172.21.15.3:0/3142206443 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcfc4021030 con 0x7fcfd00a0a30 2023-10-03T10:19:16.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.811+0000 7fcfbeffd700 1 -- 172.21.15.3:0/3142206443 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fcfc4040fb0 con 0x7fcfd00a0a30 2023-10-03T10:19:16.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.812+0000 7fcfbcff9700 1 -- 172.21.15.3:0/3142206443 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa80052e0 con 0x7fcfd00a0a30 2023-10-03T10:19:16.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.817+0000 7fcfbeffd700 1 -- 172.21.15.3:0/3142206443 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcfc4015020 con 0x7fcfd00a0a30 2023-10-03T10:19:16.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.864+0000 7f3884b22700 1 Processor -- start 2023-10-03T10:19:16.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.864+0000 7f3884b22700 1 -- start start 2023-10-03T10:19:16.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.864+0000 7f3884b22700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38801338a0 0x7f3880133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:16.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.864+0000 7f3884b22700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f388009de50 0x7f388009e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:16.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.864+0000 7f3884b22700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f388006ef30 con 0x7f388009e760 2023-10-03T10:19:16.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.865+0000 7f3884b22700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388006fef0 con 0x7f388009de50 2023-10-03T10:19:16.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.865+0000 7f3884b22700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38801334b0 con 0x7f38801338a0 2023-10-03T10:19:16.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.865+0000 7f387ed9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f388009e760 0x7f388013a730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51700/0 (socket says 172.21.15.3:51700) 2023-10-03T10:19:16.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.865+0000 7f387ed9d700 1 -- 172.21.15.3:0/690338800 learned_addr learned my addr 172.21.15.3:0/690338800 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:16.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.865+0000 7f387dd9b700 1 --2- 172.21.15.3:0/690338800 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f388009de50 0x7f388009e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:16.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.865+0000 7f387e59c700 1 --2- 172.21.15.3:0/690338800 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38801338a0 0x7f3880133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:16.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.865+0000 7f387d59a700 1 -- 172.21.15.3:0/690338800 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3080565849 0 0) 0x7f388006ef30 con 0x7f388009e760 2023-10-03T10:19:16.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.865+0000 7f387d59a700 1 -- 172.21.15.3:0/690338800 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38680035b0 con 0x7f388009e760 2023-10-03T10:19:16.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.865+0000 7f387dd9b700 1 -- 172.21.15.3:0/690338800 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38801338a0 msgr2=0x7f3880133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:16.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.865+0000 7f387dd9b700 1 --2- 172.21.15.3:0/690338800 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38801338a0 0x7f3880133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:16.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.865+0000 7f387dd9b700 1 -- 172.21.15.3:0/690338800 >> v1:172.21.15.3:6789/0 conn(0x7f388009e760 legacy=0x7f388013a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:16.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.865+0000 7f387dd9b700 1 -- 172.21.15.3:0/690338800 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388006fc30 con 0x7f388009de50 2023-10-03T10:19:16.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.865+0000 7f387e59c700 1 --2- 172.21.15.3:0/690338800 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38801338a0 0x7f3880133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:16.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.866+0000 7f387dd9b700 1 --2- 172.21.15.3:0/690338800 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f388009de50 0x7f388009e220 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f3874009870 tx=0x7f3874009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c078c1d76e1c6310 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:16.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.866+0000 7f387d59a700 1 -- 172.21.15.3:0/690338800 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3874011830 con 0x7f388009de50 2023-10-03T10:19:16.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.866+0000 7f387d59a700 1 -- 172.21.15.3:0/690338800 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3874011990 con 0x7f388009de50 2023-10-03T10:19:16.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.866+0000 7f387d59a700 1 -- 172.21.15.3:0/690338800 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3874011d20 con 0x7f388009de50 2023-10-03T10:19:16.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.866+0000 7f3884b22700 1 -- 172.21.15.3:0/690338800 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f388009de50 msgr2=0x7f388009e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:16.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.866+0000 7f3884b22700 1 --2- 172.21.15.3:0/690338800 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f388009de50 0x7f388009e220 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f3874009870 tx=0x7f3874009b80 comp rx=0 tx=0).stop 2023-10-03T10:19:16.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.867+0000 7f3884b22700 1 -- 172.21.15.3:0/690338800 shutdown_connections 2023-10-03T10:19:16.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.867+0000 7f3884b22700 1 --2- 172.21.15.3:0/690338800 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f38801338a0 0x7f3880133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:16.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.867+0000 7f3884b22700 1 --2- 172.21.15.3:0/690338800 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f388009de50 0x7f388009e220 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:16.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.867+0000 7f3884b22700 1 -- 172.21.15.3:0/690338800 >> 172.21.15.3:0/690338800 conn(0x7f38800a2aa0 msgr2=0x7f38800a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:16.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.867+0000 7f3884b22700 1 -- 172.21.15.3:0/690338800 shutdown_connections 2023-10-03T10:19:16.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.867+0000 7f3884b22700 1 -- 172.21.15.3:0/690338800 wait complete. 2023-10-03T10:19:16.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.868+0000 7f3884b22700 1 Processor -- start 2023-10-03T10:19:16.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.868+0000 7f3884b22700 1 -- start start 2023-10-03T10:19:16.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.869+0000 7f3884b22700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f388009de50 0x7f38801214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:16.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.869+0000 7f3884b22700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f388009e760 0x7f3880121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:16.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.869+0000 7f3884b22700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38801338a0 0x7f3880121f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:16.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.869+0000 7f3884b22700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3880124490 con 0x7f388009e760 2023-10-03T10:19:16.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.869+0000 7f3884b22700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388006f640 con 0x7f38801338a0 2023-10-03T10:19:16.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.869+0000 7f3884b22700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3880124b60 con 0x7f388009de50 2023-10-03T10:19:16.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.869+0000 7f387dd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f388009e760 0x7f3880121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:16.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.869+0000 7f387dd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f388009e760 0x7f3880121a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54580/0 (socket says 172.21.15.3:54580) 2023-10-03T10:19:16.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.869+0000 7f387dd9b700 1 -- 172.21.15.3:0/1855642062 learned_addr learned my addr 172.21.15.3:0/1855642062 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:16.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.869+0000 7f387ed9d700 1 --2- 172.21.15.3:0/1855642062 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38801338a0 0x7f3880121f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:16.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.869+0000 7f387e59c700 1 --2- 172.21.15.3:0/1855642062 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f388009de50 0x7f38801214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:16.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.870+0000 7f387dd9b700 1 -- 172.21.15.3:0/1855642062 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f388009de50 msgr2=0x7f38801214e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:16.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.870+0000 7f387dd9b700 1 --2- 172.21.15.3:0/1855642062 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f388009de50 0x7f38801214e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:16.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.870+0000 7f387dd9b700 1 -- 172.21.15.3:0/1855642062 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38801338a0 msgr2=0x7f3880121f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:16.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.870+0000 7f387dd9b700 1 --2- 172.21.15.3:0/1855642062 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38801338a0 0x7f3880121f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:16.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.870+0000 7f387dd9b700 1 -- 172.21.15.3:0/1855642062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38740094a0 con 0x7f388009e760 2023-10-03T10:19:16.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.870+0000 7f387ed9d700 1 --2- 172.21.15.3:0/1855642062 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38801338a0 0x7f3880121f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:16.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.870+0000 7f387dd9b700 1 --2- 172.21.15.3:0/1855642062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f388009e760 0x7f3880121a20 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f3874000c00 tx=0x7f3874004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:16.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.870+0000 7f386effd700 1 -- 172.21.15.3:0/1855642062 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3874004590 con 0x7f388009e760 2023-10-03T10:19:16.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.870+0000 7f386effd700 1 -- 172.21.15.3:0/1855642062 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387401e070 con 0x7f388009e760 2023-10-03T10:19:16.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.870+0000 7f386effd700 1 -- 172.21.15.3:0/1855642062 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38740038e0 con 0x7f388009e760 2023-10-03T10:19:16.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.870+0000 7f3884b22700 1 -- 172.21.15.3:0/1855642062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3880124dd0 con 0x7f388009e760 2023-10-03T10:19:16.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.870+0000 7f3884b22700 1 -- 172.21.15.3:0/1855642062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3880124830 con 0x7f388009e760 2023-10-03T10:19:16.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.872+0000 7f386effd700 1 -- 172.21.15.3:0/1855642062 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3874003a40 con 0x7f388009e760 2023-10-03T10:19:16.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.872+0000 7f3884b22700 1 -- 172.21.15.3:0/1855642062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3880074ea0 con 0x7f388009e760 2023-10-03T10:19:16.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.872+0000 7f386effd700 1 -- 172.21.15.3:0/1855642062 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f38740406f0 con 0x7f388009e760 2023-10-03T10:19:16.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.882+0000 7f386effd700 1 -- 172.21.15.3:0/1855642062 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3874015020 con 0x7f388009e760 2023-10-03T10:19:16.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.881+0000 7f59c59df700 1 Processor -- start 2023-10-03T10:19:16.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.882+0000 7f59c59df700 1 -- start start 2023-10-03T10:19:16.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.882+0000 7f59c59df700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59c006c0e0 0x7f59c009ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:16.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.882+0000 7f59c59df700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59c01380f0 0x7f59c013fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:16.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.882+0000 7f59c59df700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59c0134980 con 0x7f59c0134ec0 2023-10-03T10:19:16.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.882+0000 7f59c59df700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c006c4b0 con 0x7f59c006c0e0 2023-10-03T10:19:16.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.882+0000 7f59c59df700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f59c00a09c0 con 0x7f59c01380f0 2023-10-03T10:19:16.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.883+0000 7f59beffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59c006c0e0 0x7f59c009ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:16.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.883+0000 7f59beffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59c006c0e0 0x7f59c009ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:54264/0 (socket says 172.21.15.3:54264) 2023-10-03T10:19:16.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.883+0000 7f59beffd700 1 -- 172.21.15.3:0/887226071 learned_addr learned my addr 172.21.15.3:0/887226071 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:16.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.883+0000 7f59bf7fe700 1 --2- 172.21.15.3:0/887226071 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59c01380f0 0x7f59c013fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:16.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.883+0000 7f59beffd700 1 -- 172.21.15.3:0/887226071 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59c01380f0 msgr2=0x7f59c013fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:16.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.883+0000 7f59beffd700 1 --2- 172.21.15.3:0/887226071 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59c01380f0 0x7f59c013fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:16.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.883+0000 7f59beffd700 1 -- 172.21.15.3:0/887226071 >> v1:172.21.15.3:6789/0 conn(0x7f59c0134ec0 legacy=0x7f59c01379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:16.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.883+0000 7f59beffd700 1 -- 172.21.15.3:0/887226071 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c00a0600 con 0x7f59c006c0e0 2023-10-03T10:19:16.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.883+0000 7f59beffd700 1 --2- 172.21.15.3:0/887226071 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59c006c0e0 0x7f59c009ac90 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f59b0009920 tx=0x7f59b0009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30b800afbd8af826 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:16.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.884+0000 7f59bdffb700 1 -- 172.21.15.3:0/887226071 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b0011830 con 0x7f59c006c0e0 2023-10-03T10:19:16.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.884+0000 7f59bdffb700 1 -- 172.21.15.3:0/887226071 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b0011990 con 0x7f59c006c0e0 2023-10-03T10:19:16.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.884+0000 7f59bdffb700 1 -- 172.21.15.3:0/887226071 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b0011d20 con 0x7f59c006c0e0 2023-10-03T10:19:16.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.885+0000 7f59c59df700 1 -- 172.21.15.3:0/887226071 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59c006c0e0 msgr2=0x7f59c009ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:16.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.885+0000 7f59c59df700 1 --2- 172.21.15.3:0/887226071 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59c006c0e0 0x7f59c009ac90 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f59b0009920 tx=0x7f59b0009c30 comp rx=0 tx=0).stop 2023-10-03T10:19:16.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.885+0000 7f59c59df700 1 -- 172.21.15.3:0/887226071 shutdown_connections 2023-10-03T10:19:16.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.885+0000 7f59c59df700 1 --2- 172.21.15.3:0/887226071 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59c006c0e0 0x7f59c009ac90 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:16.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.885+0000 7f59c59df700 1 --2- 172.21.15.3:0/887226071 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59c01380f0 0x7f59c013fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:16.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.885+0000 7f59c59df700 1 -- 172.21.15.3:0/887226071 >> 172.21.15.3:0/887226071 conn(0x7f59c00a4210 msgr2=0x7f59c00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:16.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.885+0000 7f59c59df700 1 -- 172.21.15.3:0/887226071 shutdown_connections 2023-10-03T10:19:16.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.885+0000 7f59c59df700 1 -- 172.21.15.3:0/887226071 wait complete. 2023-10-03T10:19:16.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.886+0000 7f59c59df700 1 Processor -- start 2023-10-03T10:19:16.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.887+0000 7f59c59df700 1 -- start start 2023-10-03T10:19:16.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.887+0000 7f59c59df700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59c006c0e0 0x7f59c01ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:16.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.887+0000 7f59c59df700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59c0134ec0 0x7f59c01cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:16.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.887+0000 7f59c59df700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59c01380f0 0x7f59c011a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:16.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.887+0000 7f59c59df700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c011e470 con 0x7f59c0134ec0 2023-10-03T10:19:16.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.887+0000 7f59c59df700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c0134400 con 0x7f59c01380f0 2023-10-03T10:19:16.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.887+0000 7f59c59df700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f59c011dd00 con 0x7f59c006c0e0 2023-10-03T10:19:16.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.888+0000 7f59bf7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59c01380f0 0x7f59c011a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:16.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.888+0000 7f59be7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59c0134ec0 0x7f59c01cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:16.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.888+0000 7f59bf7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59c01380f0 0x7f59c011a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:54270/0 (socket says 172.21.15.3:54270) 2023-10-03T10:19:16.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.888+0000 7f59bf7fe700 1 -- 172.21.15.3:0/3563349513 learned_addr learned my addr 172.21.15.3:0/3563349513 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:16.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.888+0000 7f59beffd700 1 --2- 172.21.15.3:0/3563349513 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59c006c0e0 0x7f59c01ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:16.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.888+0000 7f59bf7fe700 1 -- 172.21.15.3:0/3563349513 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59c006c0e0 msgr2=0x7f59c01ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:16.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.888+0000 7f59bf7fe700 1 --2- 172.21.15.3:0/3563349513 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59c006c0e0 0x7f59c01ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:16.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.888+0000 7f59bf7fe700 1 -- 172.21.15.3:0/3563349513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59c0134ec0 msgr2=0x7f59c01cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:16.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.888+0000 7f59bf7fe700 1 --2- 172.21.15.3:0/3563349513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59c0134ec0 0x7f59c01cd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:16.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.888+0000 7f59bf7fe700 1 -- 172.21.15.3:0/3563349513 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b0009550 con 0x7f59c01380f0 2023-10-03T10:19:16.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.888+0000 7f59bf7fe700 1 --2- 172.21.15.3:0/3563349513 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59c01380f0 0x7f59c011a680 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f59b800efc0 tx=0x7f59b800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:16.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.888+0000 7f59a7fff700 1 -- 172.21.15.3:0/3563349513 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b8009ea0 con 0x7f59c01380f0 2023-10-03T10:19:16.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.889+0000 7f59a7fff700 1 -- 172.21.15.3:0/3563349513 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b800f040 con 0x7f59c01380f0 2023-10-03T10:19:16.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.889+0000 7f59c59df700 1 -- 172.21.15.3:0/3563349513 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59c011c090 con 0x7f59c01380f0 2023-10-03T10:19:16.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.889+0000 7f59a7fff700 1 -- 172.21.15.3:0/3563349513 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b800ced0 con 0x7f59c01380f0 2023-10-03T10:19:16.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.889+0000 7f59c59df700 1 -- 172.21.15.3:0/3563349513 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c011df70 con 0x7f59c01380f0 2023-10-03T10:19:16.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.890+0000 7f59c59df700 1 -- 172.21.15.3:0/3563349513 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59c0074ea0 con 0x7f59c01380f0 2023-10-03T10:19:16.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.890+0000 7f59a7fff700 1 -- 172.21.15.3:0/3563349513 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f59b8014480 con 0x7f59c01380f0 2023-10-03T10:19:16.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.890+0000 7f59a7fff700 1 -- 172.21.15.3:0/3563349513 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f59b803fee0 con 0x7f59c01380f0 2023-10-03T10:19:16.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.899+0000 7f59a7fff700 1 -- 172.21.15.3:0/3563349513 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f59b801a8b0 con 0x7f59c01380f0 2023-10-03T10:19:16.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.941+0000 7ff1e086b700 1 -- 172.21.15.3:0/3011045976 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff1dc1a79d0 con 0x7ff1dc09d6f0 2023-10-03T10:19:16.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.942+0000 7ff1caffd700 1 -- 172.21.15.3:0/3011045976 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff1cc01b3b0 con 0x7ff1dc09d6f0 2023-10-03T10:19:16.942 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:16.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.944+0000 7ff1e086b700 1 -- 172.21.15.3:0/3011045976 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1dc09d6f0 msgr2=0x7ff1dc1a5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:16.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.944+0000 7ff1e086b700 1 --2- 172.21.15.3:0/3011045976 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1dc09d6f0 0x7ff1dc1a5dc0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00ebe0 tx=0x7ff1cc009ec0 comp rx=0 tx=0).stop 2023-10-03T10:19:16.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.944+0000 7ff1e086b700 1 -- 172.21.15.3:0/3011045976 shutdown_connections 2023-10-03T10:19:16.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.944+0000 7ff1e086b700 1 --2- 172.21.15.3:0/3011045976 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff1dc09e410 0x7ff1dc1a6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:16.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.944+0000 7ff1e086b700 1 --2- 172.21.15.3:0/3011045976 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff1dc09d6f0 0x7ff1dc1a5dc0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:16.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.944+0000 7ff1e086b700 1 --2- 172.21.15.3:0/3011045976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1dc09e000 0x7ff1dc1a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:16.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.944+0000 7ff1e086b700 1 -- 172.21.15.3:0/3011045976 >> 172.21.15.3:0/3011045976 conn(0x7ff1dc066f50 msgr2=0x7ff1dc1bab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:16.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.944+0000 7ff1e086b700 1 -- 172.21.15.3:0/3011045976 shutdown_connections 2023-10-03T10:19:16.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:16.944+0000 7ff1e086b700 1 -- 172.21.15.3:0/3011045976 wait complete. 2023-10-03T10:19:16.959 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:17.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.043+0000 7fcfbcff9700 1 -- 172.21.15.3:0/3142206443 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcfa8005590 con 0x7fcfd00a0a30 2023-10-03T10:19:17.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.043+0000 7fcfbeffd700 1 -- 172.21.15.3:0/3142206443 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcfc401a7d0 con 0x7fcfd00a0a30 2023-10-03T10:19:17.043 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:17.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.045+0000 7fcfbcff9700 1 -- 172.21.15.3:0/3142206443 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd00a0a30 msgr2=0x7fcfd01340b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:17.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.045+0000 7fcfbcff9700 1 --2- 172.21.15.3:0/3142206443 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd00a0a30 0x7fcfd01340b0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4000c00 tx=0x7fcfc4004750 comp rx=0 tx=0).stop 2023-10-03T10:19:17.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.046+0000 7fcfbcff9700 1 -- 172.21.15.3:0/3142206443 shutdown_connections 2023-10-03T10:19:17.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.046+0000 7fcfbcff9700 1 --2- 172.21.15.3:0/3142206443 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcfd0138d80 0x7fcfd01345f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:17.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.046+0000 7fcfbcff9700 1 --2- 172.21.15.3:0/3142206443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfd009d630 0x7fcfd0133b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:17.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.046+0000 7fcfbcff9700 1 --2- 172.21.15.3:0/3142206443 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcfd00a0a30 0x7fcfd01340b0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:17.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.046+0000 7fcfbcff9700 1 -- 172.21.15.3:0/3142206443 >> 172.21.15.3:0/3142206443 conn(0x7fcfd00a3050 msgr2=0x7fcfd01399e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:17.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.046+0000 7fcfbcff9700 1 -- 172.21.15.3:0/3142206443 shutdown_connections 2023-10-03T10:19:17.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.046+0000 7fcfbcff9700 1 -- 172.21.15.3:0/3142206443 wait complete. 2023-10-03T10:19:17.060 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.097+0000 7f3884b22700 1 -- 172.21.15.3:0/1855642062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f388006b9e0 con 0x7f388009e760 2023-10-03T10:19:17.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.098+0000 7f386effd700 1 -- 172.21.15.3:0/1855642062 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3874003cf0 con 0x7f388009e760 2023-10-03T10:19:17.098 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:17.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.100+0000 7f3884b22700 1 -- 172.21.15.3:0/1855642062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f388009e760 msgr2=0x7f3880121a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:17.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.100+0000 7f3884b22700 1 --2- 172.21.15.3:0/1855642062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f388009e760 0x7f3880121a20 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f3874000c00 tx=0x7f3874004750 comp rx=0 tx=0).stop 2023-10-03T10:19:17.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.100+0000 7f3884b22700 1 -- 172.21.15.3:0/1855642062 shutdown_connections 2023-10-03T10:19:17.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.100+0000 7f3884b22700 1 --2- 172.21.15.3:0/1855642062 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f38801338a0 0x7f3880121f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:17.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.100+0000 7f3884b22700 1 --2- 172.21.15.3:0/1855642062 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f388009de50 0x7f38801214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:17.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.100+0000 7f3884b22700 1 --2- 172.21.15.3:0/1855642062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f388009e760 0x7f3880121a20 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:17.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.100+0000 7f3884b22700 1 -- 172.21.15.3:0/1855642062 >> 172.21.15.3:0/1855642062 conn(0x7f38800a2aa0 msgr2=0x7f388009ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:17.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.100+0000 7f3884b22700 1 -- 172.21.15.3:0/1855642062 shutdown_connections 2023-10-03T10:19:17.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.100+0000 7f3884b22700 1 -- 172.21.15.3:0/1855642062 wait complete. 2023-10-03T10:19:17.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.107+0000 7f59c59df700 1 -- 172.21.15.3:0/3563349513 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f59c006b250 con 0x7f59c01380f0 2023-10-03T10:19:17.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.108+0000 7f59a7fff700 1 -- 172.21.15.3:0/3563349513 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f59b8018030 con 0x7f59c01380f0 2023-10-03T10:19:17.108 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:17.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.110+0000 7f59c59df700 1 -- 172.21.15.3:0/3563349513 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59c01380f0 msgr2=0x7f59c011a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:17.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.110+0000 7f59c59df700 1 --2- 172.21.15.3:0/3563349513 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59c01380f0 0x7f59c011a680 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f59b800efc0 tx=0x7f59b800c3a0 comp rx=0 tx=0).stop 2023-10-03T10:19:17.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.110+0000 7f59c59df700 1 -- 172.21.15.3:0/3563349513 shutdown_connections 2023-10-03T10:19:17.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.110+0000 7f59c59df700 1 --2- 172.21.15.3:0/3563349513 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59c01380f0 0x7f59c011a680 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:17.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.110+0000 7f59c59df700 1 --2- 172.21.15.3:0/3563349513 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f59c006c0e0 0x7f59c01ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:17.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.110+0000 7f59c59df700 1 --2- 172.21.15.3:0/3563349513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59c0134ec0 0x7f59c01cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:17.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.110+0000 7f59c59df700 1 -- 172.21.15.3:0/3563349513 >> 172.21.15.3:0/3563349513 conn(0x7f59c00a4210 msgr2=0x7f59c00a4810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:17.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.110+0000 7f59c59df700 1 -- 172.21.15.3:0/3563349513 shutdown_connections 2023-10-03T10:19:17.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:17.110+0000 7f59c59df700 1 -- 172.21.15.3:0/3563349513 wait complete. 2023-10-03T10:19:17.114 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:17.124 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:17.960 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:18.061 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:18.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.108+0000 7febeec83700 1 Processor -- start 2023-10-03T10:19:18.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.108+0000 7febeec83700 1 -- start start 2023-10-03T10:19:18.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.108+0000 7febeec83700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7febe80a0a30 0x7febe80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:18.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.108+0000 7febeec83700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febe809d630 0x7febe80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:18.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.108+0000 7febeec83700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febe80a2a70 con 0x7febe8133930 2023-10-03T10:19:18.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.108+0000 7febeec83700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febe80a01d0 con 0x7febe80a0a30 2023-10-03T10:19:18.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.108+0000 7febeec83700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7febe809e2b0 con 0x7febe809d630 2023-10-03T10:19:18.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.108+0000 7febe7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febe809d630 0x7febe80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:18.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.108+0000 7febe7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febe809d630 0x7febe80a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41686/0 (socket says 172.21.15.3:41686) 2023-10-03T10:19:18.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.108+0000 7febeca1f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7febe80a0a30 0x7febe80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:18.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.108+0000 7febe7fff700 1 -- 172.21.15.3:0/2137288957 learned_addr learned my addr 172.21.15.3:0/2137288957 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:18.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.109+0000 7febe7fff700 1 -- 172.21.15.3:0/2137288957 >> v2:172.21.15.3:3300/0 conn(0x7febe80a0a30 msgr2=0x7febe80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:18.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.109+0000 7febe7fff700 1 --2- 172.21.15.3:0/2137288957 >> v2:172.21.15.3:3300/0 conn(0x7febe80a0a30 0x7febe80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.109+0000 7febe7fff700 1 -- 172.21.15.3:0/2137288957 >> v1:172.21.15.3:6789/0 conn(0x7febe8133930 legacy=0x7febe8133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:18.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.109+0000 7febe7fff700 1 -- 172.21.15.3:0/2137288957 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe809dc50 con 0x7febe809d630 2023-10-03T10:19:18.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.109+0000 7febeca1f700 1 --2- 172.21.15.3:0/2137288957 >> v2:172.21.15.3:3300/0 conn(0x7febe80a0a30 0x7febe80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:18.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.109+0000 7febe7fff700 1 --2- 172.21.15.3:0/2137288957 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febe809d630 0x7febe80a1340 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7febdc0098f0 tx=0x7febdc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=911723e1480f240e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:18.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.109+0000 7febe77fe700 1 -- 172.21.15.3:0/2137288957 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc011830 con 0x7febe809d630 2023-10-03T10:19:18.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.109+0000 7febe77fe700 1 -- 172.21.15.3:0/2137288957 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febdc011990 con 0x7febe809d630 2023-10-03T10:19:18.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.109+0000 7febe77fe700 1 -- 172.21.15.3:0/2137288957 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc011d20 con 0x7febe809d630 2023-10-03T10:19:18.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.110+0000 7febeec83700 1 -- 172.21.15.3:0/2137288957 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febe809d630 msgr2=0x7febe80a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:18.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.110+0000 7febeec83700 1 --2- 172.21.15.3:0/2137288957 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febe809d630 0x7febe80a1340 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7febdc0098f0 tx=0x7febdc009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:18.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.110+0000 7febeec83700 1 -- 172.21.15.3:0/2137288957 shutdown_connections 2023-10-03T10:19:18.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.110+0000 7febeec83700 1 --2- 172.21.15.3:0/2137288957 >> v2:172.21.15.3:3300/0 conn(0x7febe80a0a30 0x7febe80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.110+0000 7febeec83700 1 --2- 172.21.15.3:0/2137288957 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febe809d630 0x7febe80a1340 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.110+0000 7febeec83700 1 -- 172.21.15.3:0/2137288957 >> 172.21.15.3:0/2137288957 conn(0x7febe80a3050 msgr2=0x7febe80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:18.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.110+0000 7febeec83700 1 -- 172.21.15.3:0/2137288957 shutdown_connections 2023-10-03T10:19:18.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.110+0000 7febeec83700 1 -- 172.21.15.3:0/2137288957 wait complete. 2023-10-03T10:19:18.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.111+0000 7febeec83700 1 Processor -- start 2023-10-03T10:19:18.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.111+0000 7febeec83700 1 -- start start 2023-10-03T10:19:18.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.112+0000 7febeec83700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febe809d630 0x7febe81c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:18.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.112+0000 7febeec83700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febe80a0a30 0x7febe81c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:18.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.112+0000 7febeec83700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7febe8133930 0x7febe81c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:18.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.112+0000 7febeec83700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe8133530 con 0x7febe809d630 2023-10-03T10:19:18.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.112+0000 7febeec83700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe8120710 con 0x7febe8133930 2023-10-03T10:19:18.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.112+0000 7febeec83700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7febe8121f00 con 0x7febe80a0a30 2023-10-03T10:19:18.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.113+0000 7febe7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febe80a0a30 0x7febe81c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:18.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.113+0000 7febeca1f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febe809d630 0x7febe81c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:18.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.113+0000 7febe7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febe80a0a30 0x7febe81c86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41690/0 (socket says 172.21.15.3:41690) 2023-10-03T10:19:18.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.113+0000 7febe7fff700 1 -- 172.21.15.3:0/905847900 learned_addr learned my addr 172.21.15.3:0/905847900 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:18.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.113+0000 7febeca1f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febe809d630 0x7febe81c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54610/0 (socket says 172.21.15.3:54610) 2023-10-03T10:19:18.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.113+0000 7febed220700 1 --2- 172.21.15.3:0/905847900 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7febe8133930 0x7febe81c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:18.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.113+0000 7febe7fff700 1 -- 172.21.15.3:0/905847900 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7febe8133930 msgr2=0x7febe81c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:18.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.113+0000 7febe7fff700 1 --2- 172.21.15.3:0/905847900 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7febe8133930 0x7febe81c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.113+0000 7febe7fff700 1 -- 172.21.15.3:0/905847900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febe809d630 msgr2=0x7febe81c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:18.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.113+0000 7febe7fff700 1 --2- 172.21.15.3:0/905847900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febe809d630 0x7febe81c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.113+0000 7febe7fff700 1 -- 172.21.15.3:0/905847900 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd80094a0 con 0x7febe80a0a30 2023-10-03T10:19:18.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.113+0000 7febeca1f700 1 --2- 172.21.15.3:0/905847900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febe809d630 0x7febe81c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:18.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.113+0000 7febe7fff700 1 --2- 172.21.15.3:0/905847900 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febe80a0a30 0x7febe81c86d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7febdc009c40 tx=0x7febdc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:18.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.113+0000 7febe57fa700 1 -- 172.21.15.3:0/905847900 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc0036a0 con 0x7febe80a0a30 2023-10-03T10:19:18.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.113+0000 7febe57fa700 1 -- 172.21.15.3:0/905847900 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febdc003800 con 0x7febe80a0a30 2023-10-03T10:19:18.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.113+0000 7febeec83700 1 -- 172.21.15.3:0/905847900 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febdc009550 con 0x7febe80a0a30 2023-10-03T10:19:18.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.114+0000 7febe57fa700 1 -- 172.21.15.3:0/905847900 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc003bb0 con 0x7febe80a0a30 2023-10-03T10:19:18.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.114+0000 7febeec83700 1 -- 172.21.15.3:0/905847900 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe80a0460 con 0x7febe80a0a30 2023-10-03T10:19:18.116 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:18.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.115+0000 7febeec83700 1 -- 172.21.15.3:0/905847900 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe8122400 con 0x7febe80a0a30 2023-10-03T10:19:18.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.115+0000 7febe57fa700 1 -- 172.21.15.3:0/905847900 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7febdc021070 con 0x7febe80a0a30 2023-10-03T10:19:18.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.115+0000 7febe57fa700 1 -- 172.21.15.3:0/905847900 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7febdc040e70 con 0x7febe80a0a30 2023-10-03T10:19:18.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.120+0000 7febe57fa700 1 -- 172.21.15.3:0/905847900 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7febdc02d3a0 con 0x7febe80a0a30 2023-10-03T10:19:18.124 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:18.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.209+0000 7f6d9ea6a700 1 Processor -- start 2023-10-03T10:19:18.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.209+0000 7f6d9ea6a700 1 -- start start 2023-10-03T10:19:18.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.209+0000 7f6d9ea6a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d9809e1d0 0x7f6d98135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:18.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.209+0000 7f6d9ea6a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6d98135e50 0x7f6d98138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:18.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.209+0000 7f6d9ea6a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d9809f130 con 0x7f6d9809d6f0 2023-10-03T10:19:18.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.209+0000 7f6d9ea6a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d9809f520 con 0x7f6d98135e50 2023-10-03T10:19:18.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.209+0000 7f6d9ea6a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9809e930 con 0x7f6d9809e1d0 2023-10-03T10:19:18.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.209+0000 7f6d97fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d9809e1d0 0x7f6d98135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:18.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.209+0000 7f6d97fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d9809e1d0 0x7f6d98135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:54292/0 (socket says 172.21.15.3:54292) 2023-10-03T10:19:18.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.210+0000 7f6d97fff700 1 -- 172.21.15.3:0/3001086835 learned_addr learned my addr 172.21.15.3:0/3001086835 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:18.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.210+0000 7f6d9d007700 1 --2- 172.21.15.3:0/3001086835 >> v2:172.21.15.3:3300/0 conn(0x7f6d98135e50 0x7f6d98138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.210+0000 7f6d97fff700 1 -- 172.21.15.3:0/3001086835 >> v2:172.21.15.3:3300/0 conn(0x7f6d98135e50 msgr2=0x7f6d98138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.210+0000 7f6d97fff700 1 --2- 172.21.15.3:0/3001086835 >> v2:172.21.15.3:3300/0 conn(0x7f6d98135e50 0x7f6d98138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.210+0000 7f6d97fff700 1 -- 172.21.15.3:0/3001086835 >> v1:172.21.15.3:6789/0 conn(0x7f6d9809d6f0 legacy=0x7f6d9809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.210+0000 7f6d97fff700 1 -- 172.21.15.3:0/3001086835 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9809b2d0 con 0x7f6d9809e1d0 2023-10-03T10:19:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.210+0000 7f6d9d007700 1 --2- 172.21.15.3:0/3001086835 >> v2:172.21.15.3:3300/0 conn(0x7f6d98135e50 0x7f6d98138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.210+0000 7f6d97fff700 1 --2- 172.21.15.3:0/3001086835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d9809e1d0 0x7f6d98135910 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009920 tx=0x7f6d84009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aeb577325c760c7f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.210+0000 7f6d977fe700 1 -- 172.21.15.3:0/3001086835 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d84004230 con 0x7f6d9809e1d0 2023-10-03T10:19:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.210+0000 7f6d977fe700 1 -- 172.21.15.3:0/3001086835 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d84004390 con 0x7f6d9809e1d0 2023-10-03T10:19:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.210+0000 7f6d977fe700 1 -- 172.21.15.3:0/3001086835 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d84009e70 con 0x7f6d9809e1d0 2023-10-03T10:19:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.211+0000 7f6d9ea6a700 1 -- 172.21.15.3:0/3001086835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d9809e1d0 msgr2=0x7f6d98135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.211+0000 7f6d9ea6a700 1 --2- 172.21.15.3:0/3001086835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d9809e1d0 0x7f6d98135910 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009920 tx=0x7f6d84009c30 comp rx=0 tx=0).stop 2023-10-03T10:19:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.211+0000 7f6d9ea6a700 1 -- 172.21.15.3:0/3001086835 shutdown_connections 2023-10-03T10:19:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.211+0000 7f6d9ea6a700 1 --2- 172.21.15.3:0/3001086835 >> v2:172.21.15.3:3300/0 conn(0x7f6d98135e50 0x7f6d98138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.211+0000 7f6d9ea6a700 1 --2- 172.21.15.3:0/3001086835 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d9809e1d0 0x7f6d98135910 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.211+0000 7f6d9ea6a700 1 -- 172.21.15.3:0/3001086835 >> 172.21.15.3:0/3001086835 conn(0x7f6d98066f50 msgr2=0x7f6d9809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:18.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.211+0000 7f6d9ea6a700 1 -- 172.21.15.3:0/3001086835 shutdown_connections 2023-10-03T10:19:18.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.211+0000 7f6d9ea6a700 1 -- 172.21.15.3:0/3001086835 wait complete. 2023-10-03T10:19:18.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.212+0000 7f6d9ea6a700 1 Processor -- start 2023-10-03T10:19:18.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.213+0000 7f6d9ea6a700 1 -- start start 2023-10-03T10:19:18.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.213+0000 7f6d9ea6a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d9809d6f0 0x7f6d9811e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:18.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.213+0000 7f6d9ea6a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6d9809e1d0 0x7f6d9811e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:18.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.213+0000 7f6d9ea6a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d98135e50 0x7f6d981c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:18.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.213+0000 7f6d9ea6a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9809c770 con 0x7f6d9809d6f0 2023-10-03T10:19:18.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.213+0000 7f6d9ea6a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d981303e0 con 0x7f6d98135e50 2023-10-03T10:19:18.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.213+0000 7f6d9ea6a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9811eea0 con 0x7f6d9809e1d0 2023-10-03T10:19:18.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.213+0000 7f6d9c806700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d9809d6f0 0x7f6d9811e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:18.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.213+0000 7f6d97fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6d9809e1d0 0x7f6d9811e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:18.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.214+0000 7f6d9c806700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d9809d6f0 0x7f6d9811e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54618/0 (socket says 172.21.15.3:54618) 2023-10-03T10:19:18.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.214+0000 7f6d9c806700 1 -- 172.21.15.3:0/1205527944 learned_addr learned my addr 172.21.15.3:0/1205527944 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:18.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.214+0000 7f6d9d007700 1 --2- 172.21.15.3:0/1205527944 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d98135e50 0x7f6d981c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:18.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.214+0000 7f6d9c806700 1 -- 172.21.15.3:0/1205527944 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6d9809e1d0 msgr2=0x7f6d9811e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:18.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.214+0000 7f6d9c806700 1 --2- 172.21.15.3:0/1205527944 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6d9809e1d0 0x7f6d9811e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.214+0000 7f6d9c806700 1 -- 172.21.15.3:0/1205527944 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d98135e50 msgr2=0x7f6d981c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:18.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.214+0000 7f6d9c806700 1 --2- 172.21.15.3:0/1205527944 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d98135e50 0x7f6d981c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.214+0000 7f6d9c806700 1 -- 172.21.15.3:0/1205527944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d84009550 con 0x7f6d9809d6f0 2023-10-03T10:19:18.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.214+0000 7f6d97fff700 1 --2- 172.21.15.3:0/1205527944 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6d9809e1d0 0x7f6d9811e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:18.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.214+0000 7f6d9c806700 1 --2- 172.21.15.3:0/1205527944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d9809d6f0 0x7f6d9811e420 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00c4d0 tx=0x7f6d8c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:18.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.214+0000 7f6d957fa700 1 -- 172.21.15.3:0/1205527944 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c011790 con 0x7f6d9809d6f0 2023-10-03T10:19:18.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.215+0000 7f6d957fa700 1 -- 172.21.15.3:0/1205527944 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8c0118f0 con 0x7f6d9809d6f0 2023-10-03T10:19:18.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.215+0000 7f6d9ea6a700 1 -- 172.21.15.3:0/1205527944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d98120ff0 con 0x7f6d9809d6f0 2023-10-03T10:19:18.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.215+0000 7f6d957fa700 1 -- 172.21.15.3:0/1205527944 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c011ca0 con 0x7f6d9809d6f0 2023-10-03T10:19:18.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.215+0000 7f6d9ea6a700 1 -- 172.21.15.3:0/1205527944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d981202f0 con 0x7f6d9809d6f0 2023-10-03T10:19:18.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.216+0000 7f6d957fa700 1 -- 172.21.15.3:0/1205527944 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6d8c01c460 con 0x7f6d9809d6f0 2023-10-03T10:19:18.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.216+0000 7f6d9ea6a700 1 -- 172.21.15.3:0/1205527944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d9809ed10 con 0x7f6d9809d6f0 2023-10-03T10:19:18.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.216+0000 7f6d957fa700 1 -- 172.21.15.3:0/1205527944 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6d8c03f470 con 0x7f6d9809d6f0 2023-10-03T10:19:18.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.225+0000 7f6d957fa700 1 -- 172.21.15.3:0/1205527944 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6d8c0106a0 con 0x7f6d9809d6f0 2023-10-03T10:19:18.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.266+0000 7fd609bc7700 1 Processor -- start 2023-10-03T10:19:18.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.266+0000 7fd609bc7700 1 -- start start 2023-10-03T10:19:18.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.266+0000 7fd609bc7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd60409e1d0 0x7fd604138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:18.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.266+0000 7fd609bc7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd604139170 0x7fd60413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:18.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.266+0000 7fd609bc7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd60409b120 con 0x7fd60409d6f0 2023-10-03T10:19:18.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.266+0000 7fd609bc7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd60406b460 con 0x7fd604139170 2023-10-03T10:19:18.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.266+0000 7fd609bc7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd60409a7e0 con 0x7fd60409e1d0 2023-10-03T10:19:18.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.267+0000 7fd6037fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd60409d6f0 0x7fd60409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51744/0 (socket says 172.21.15.3:51744) 2023-10-03T10:19:18.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.267+0000 7fd6037fe700 1 -- 172.21.15.3:0/1567852450 learned_addr learned my addr 172.21.15.3:0/1567852450 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:18.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.267+0000 7fd603fff700 1 --2- 172.21.15.3:0/1567852450 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd604139170 0x7fd60413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:18.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.267+0000 7fd6027fc700 1 -- 172.21.15.3:0/1567852450 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3974276230 0 0) 0x7fd60409b120 con 0x7fd60409d6f0 2023-10-03T10:19:18.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.267+0000 7fd6027fc700 1 -- 172.21.15.3:0/1567852450 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5ec0035b0 con 0x7fd60409d6f0 2023-10-03T10:19:18.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.267+0000 7fd602ffd700 1 --2- 172.21.15.3:0/1567852450 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd60409e1d0 0x7fd604138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:18.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.267+0000 7fd602ffd700 1 -- 172.21.15.3:0/1567852450 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd604139170 msgr2=0x7fd60413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:18.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.267+0000 7fd602ffd700 1 --2- 172.21.15.3:0/1567852450 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd604139170 0x7fd60413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.267+0000 7fd602ffd700 1 -- 172.21.15.3:0/1567852450 >> v1:172.21.15.3:6789/0 conn(0x7fd60409d6f0 legacy=0x7fd60409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:18.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.267+0000 7fd602ffd700 1 -- 172.21.15.3:0/1567852450 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd60409a500 con 0x7fd60409e1d0 2023-10-03T10:19:18.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.268+0000 7fd602ffd700 1 --2- 172.21.15.3:0/1567852450 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd60409e1d0 0x7fd604138c30 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80098a0 tx=0x7fd5f8009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50ec8888ee3eec35 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:18.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.268+0000 7fd6027fc700 1 -- 172.21.15.3:0/1567852450 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8011830 con 0x7fd60409e1d0 2023-10-03T10:19:18.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.268+0000 7fd6027fc700 1 -- 172.21.15.3:0/1567852450 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f8011990 con 0x7fd60409e1d0 2023-10-03T10:19:18.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.268+0000 7fd6027fc700 1 -- 172.21.15.3:0/1567852450 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8011d20 con 0x7fd60409e1d0 2023-10-03T10:19:18.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.268+0000 7f06622e7700 1 Processor -- start 2023-10-03T10:19:18.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.268+0000 7f06622e7700 1 -- start start 2023-10-03T10:19:18.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.269+0000 7fd609bc7700 1 -- 172.21.15.3:0/1567852450 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd60409e1d0 msgr2=0x7fd604138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:18.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.269+0000 7fd609bc7700 1 --2- 172.21.15.3:0/1567852450 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd60409e1d0 0x7fd604138c30 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80098a0 tx=0x7fd5f8009bb0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.269+0000 7fd609bc7700 1 -- 172.21.15.3:0/1567852450 shutdown_connections 2023-10-03T10:19:18.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.269+0000 7fd609bc7700 1 --2- 172.21.15.3:0/1567852450 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd604139170 0x7fd60413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.269+0000 7fd609bc7700 1 --2- 172.21.15.3:0/1567852450 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd60409e1d0 0x7fd604138c30 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.269+0000 7fd609bc7700 1 -- 172.21.15.3:0/1567852450 >> 172.21.15.3:0/1567852450 conn(0x7fd604066f50 msgr2=0x7fd60409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:18.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.269+0000 7fd609bc7700 1 -- 172.21.15.3:0/1567852450 shutdown_connections 2023-10-03T10:19:18.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.269+0000 7fd609bc7700 1 -- 172.21.15.3:0/1567852450 wait complete. 2023-10-03T10:19:18.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.269+0000 7f06622e7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f065c130540 0x7f065c130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:18.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.269+0000 7f06622e7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f065c130e50 0x7f065c06f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:18.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.269+0000 7f06622e7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f065c0716a0 con 0x7f065c06fb20 2023-10-03T10:19:18.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.269+0000 7f06622e7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065c09a390 con 0x7f065c130540 2023-10-03T10:19:18.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.269+0000 7f06622e7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f065c071290 con 0x7f065c130e50 2023-10-03T10:19:18.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.269+0000 7f0660884700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f065c06fb20 0x7f065c137560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51760/0 (socket says 172.21.15.3:51760) 2023-10-03T10:19:18.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.269+0000 7f0660884700 1 -- 172.21.15.3:0/112759782 learned_addr learned my addr 172.21.15.3:0/112759782 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:18.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.270+0000 7f065affd700 1 -- 172.21.15.3:0/112759782 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 702736988 0 0) 0x7f065c0716a0 con 0x7f065c06fb20 2023-10-03T10:19:18.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.270+0000 7f065affd700 1 -- 172.21.15.3:0/112759782 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06440035b0 con 0x7f065c06fb20 2023-10-03T10:19:18.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.270+0000 7f065affd700 1 -- 172.21.15.3:0/112759782 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 752022981 0 0) 0x7f06440035b0 con 0x7f065c06fb20 2023-10-03T10:19:18.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.270+0000 7f065affd700 1 -- 172.21.15.3:0/112759782 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f065c130e50 msgr2=0x7f065c06f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:18.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.270+0000 7f065affd700 1 --2- 172.21.15.3:0/112759782 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f065c130e50 0x7f065c06f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.270+0000 7f065affd700 1 -- 172.21.15.3:0/112759782 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f065c130540 msgr2=0x7f065c130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:18.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.270+0000 7f065affd700 1 --2- 172.21.15.3:0/112759782 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f065c130540 0x7f065c130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.270+0000 7f065affd700 1 -- 172.21.15.3:0/112759782 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065c130340 con 0x7f065c06fb20 2023-10-03T10:19:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.270+0000 7f065affd700 1 -- 172.21.15.3:0/112759782 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1156569835 0 0) 0x7f06540019f0 con 0x7f065c06fb20 2023-10-03T10:19:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.270+0000 7f065affd700 1 -- 172.21.15.3:0/112759782 >> v1:172.21.15.3:6789/0 conn(0x7f065c06fb20 legacy=0x7f065c137560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.270+0000 7fd609bc7700 1 Processor -- start 2023-10-03T10:19:18.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.270+0000 7f065affd700 1 --2- 172.21.15.3:0/112759782 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f065c130540 0x7f0644003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:18.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.270+0000 7f065affd700 1 --2- 172.21.15.3:0/112759782 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f065c130e50 0x7f06440041a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:18.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.270+0000 7f065affd700 1 --2- 172.21.15.3:0/112759782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0644004900 0x7f0644004d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:18.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.270+0000 7f065affd700 1 -- 172.21.15.3:0/112759782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c09a390 con 0x7f0644004900 2023-10-03T10:19:18.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.270+0000 7f065affd700 1 -- 172.21.15.3:0/112759782 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065c071290 con 0x7f065c130540 2023-10-03T10:19:18.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.270+0000 7f065affd700 1 -- 172.21.15.3:0/112759782 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f06440055a0 con 0x7f065c130e50 2023-10-03T10:19:18.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.270+0000 7f0660884700 1 --2- 172.21.15.3:0/112759782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0644004900 0x7f0644004d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:18.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.271+0000 7f065b7fe700 1 --2- 172.21.15.3:0/112759782 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f065c130e50 0x7f06440041a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:18.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.271+0000 7f0660884700 1 -- 172.21.15.3:0/112759782 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f065c130e50 msgr2=0x7f06440041a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:18.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.271+0000 7f0660884700 1 --2- 172.21.15.3:0/112759782 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f065c130e50 0x7f06440041a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.271+0000 7f0660884700 1 -- 172.21.15.3:0/112759782 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f065c130540 msgr2=0x7f0644003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:18.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.271+0000 7f0660884700 1 --2- 172.21.15.3:0/112759782 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f065c130540 0x7f0644003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.271+0000 7f0660884700 1 -- 172.21.15.3:0/112759782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065c130340 con 0x7f0644004900 2023-10-03T10:19:18.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.271+0000 7f0660884700 1 --2- 172.21.15.3:0/112759782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0644004900 0x7f0644004d70 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f065400a940 tx=0x7f0644005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8855f76066723ac9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:18.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.271+0000 7f065affd700 1 -- 172.21.15.3:0/112759782 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065400a3e0 con 0x7f0644004900 2023-10-03T10:19:18.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.271+0000 7f065affd700 1 -- 172.21.15.3:0/112759782 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0654012600 con 0x7f0644004900 2023-10-03T10:19:18.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.271+0000 7f065affd700 1 -- 172.21.15.3:0/112759782 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06540129b0 con 0x7f0644004900 2023-10-03T10:19:18.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.271+0000 7fd609bc7700 1 -- start start 2023-10-03T10:19:18.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.271+0000 7fd609bc7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd60409d6f0 0x7fd6041cd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:18.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.271+0000 7fd609bc7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd60409e1d0 0x7fd6041cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:18.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.271+0000 7fd609bc7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd604139170 0x7fd6041d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:18.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.271+0000 7fd609bc7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd604128010 con 0x7fd604139170 2023-10-03T10:19:18.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.271+0000 7fd609bc7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6041293e0 con 0x7fd60409e1d0 2023-10-03T10:19:18.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.271+0000 7fd609bc7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd604127720 con 0x7fd60409d6f0 2023-10-03T10:19:18.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.272+0000 7fd6037fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd60409d6f0 0x7fd6041cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:18.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.272+0000 7fd602ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd60409e1d0 0x7fd6041cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:18.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.272+0000 7fd6037fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd60409d6f0 0x7fd6041cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41740/0 (socket says 172.21.15.3:41740) 2023-10-03T10:19:18.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.272+0000 7fd602ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd60409e1d0 0x7fd6041cfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:54334/0 (socket says 172.21.15.3:54334) 2023-10-03T10:19:18.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.272+0000 7fd6037fe700 1 -- 172.21.15.3:0/1561286135 learned_addr learned my addr 172.21.15.3:0/1561286135 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:18.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.272+0000 7fd602ffd700 1 -- 172.21.15.3:0/1561286135 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd60409d6f0 msgr2=0x7fd6041cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:18.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.272+0000 7fd602ffd700 1 --2- 172.21.15.3:0/1561286135 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd60409d6f0 0x7fd6041cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.272+0000 7fd602ffd700 1 -- 172.21.15.3:0/1561286135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd604139170 msgr2=0x7fd6041d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:18.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.272+0000 7fd602ffd700 1 --2- 172.21.15.3:0/1561286135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd604139170 0x7fd6041d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.272+0000 7fd602ffd700 1 -- 172.21.15.3:0/1561286135 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f80094a0 con 0x7fd60409e1d0 2023-10-03T10:19:18.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.272+0000 7fd6037fe700 1 --2- 172.21.15.3:0/1561286135 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd60409d6f0 0x7fd6041cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:18.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.272+0000 7f06622e7700 1 -- 172.21.15.3:0/112759782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0644004900 msgr2=0x7f0644004d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:18.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.272+0000 7f06622e7700 1 --2- 172.21.15.3:0/112759782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0644004900 0x7f0644004d70 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f065400a940 tx=0x7f0644005880 comp rx=0 tx=0).stop 2023-10-03T10:19:18.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.272+0000 7f06622e7700 1 -- 172.21.15.3:0/112759782 shutdown_connections 2023-10-03T10:19:18.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.272+0000 7f06622e7700 1 --2- 172.21.15.3:0/112759782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0644004900 0x7f0644004d70 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.272+0000 7f06622e7700 1 --2- 172.21.15.3:0/112759782 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f065c130540 0x7f0644003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.272+0000 7f06622e7700 1 --2- 172.21.15.3:0/112759782 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f065c130e50 0x7f06440041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.272+0000 7f06622e7700 1 -- 172.21.15.3:0/112759782 >> 172.21.15.3:0/112759782 conn(0x7f065c0a2aa0 msgr2=0x7f065c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:18.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.272+0000 7f06622e7700 1 -- 172.21.15.3:0/112759782 shutdown_connections 2023-10-03T10:19:18.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.272+0000 7f06622e7700 1 -- 172.21.15.3:0/112759782 wait complete. 2023-10-03T10:19:18.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.272+0000 7fd602ffd700 1 --2- 172.21.15.3:0/1561286135 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd60409e1d0 0x7fd6041cfe70 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8016000 tx=0x7fd5f8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:18.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.273+0000 7fd5ebfff700 1 -- 172.21.15.3:0/1561286135 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8004550 con 0x7fd60409e1d0 2023-10-03T10:19:18.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.273+0000 7fd5ebfff700 1 -- 172.21.15.3:0/1561286135 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f801f070 con 0x7fd60409e1d0 2023-10-03T10:19:18.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.273+0000 7fd5ebfff700 1 -- 172.21.15.3:0/1561286135 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f80038e0 con 0x7fd60409e1d0 2023-10-03T10:19:18.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.273+0000 7fd609bc7700 1 -- 172.21.15.3:0/1561286135 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd604127cd0 con 0x7fd60409e1d0 2023-10-03T10:19:18.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.274+0000 7fd609bc7700 1 -- 172.21.15.3:0/1561286135 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd604071f10 con 0x7fd60409e1d0 2023-10-03T10:19:18.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.273+0000 7f06622e7700 1 Processor -- start 2023-10-03T10:19:18.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.274+0000 7f06622e7700 1 -- start start 2023-10-03T10:19:18.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.275+0000 7f06622e7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0644004900 0x7f065c09c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:18.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.275+0000 7f06622e7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f065c06fb20 0x7f065c09cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:18.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.275+0000 7f06622e7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f065c130540 0x7f065c09d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:18.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.275+0000 7f06622e7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c126d60 con 0x7f065c06fb20 2023-10-03T10:19:18.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.275+0000 7f06622e7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065c09e320 con 0x7f065c130540 2023-10-03T10:19:18.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.275+0000 7f06622e7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f065c0a0580 con 0x7f0644004900 2023-10-03T10:19:18.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.275+0000 7f065bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0644004900 0x7f065c09c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:18.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.275+0000 7fd5e9ffb700 1 -- 172.21.15.3:0/1561286135 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5dc0052e0 con 0x7fd60409e1d0 2023-10-03T10:19:18.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.275+0000 7fd5ebfff700 1 -- 172.21.15.3:0/1561286135 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd5f8003a40 con 0x7fd60409e1d0 2023-10-03T10:19:18.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.275+0000 7fd5ebfff700 1 -- 172.21.15.3:0/1561286135 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd5f8021070 con 0x7fd60409e1d0 2023-10-03T10:19:18.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.275+0000 7f065bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0644004900 0x7f065c09c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41744/0 (socket says 172.21.15.3:41744) 2023-10-03T10:19:18.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.275+0000 7f065bfff700 1 -- 172.21.15.3:0/1354471837 learned_addr learned my addr 172.21.15.3:0/1354471837 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:18.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.275+0000 7f065b7fe700 1 --2- 172.21.15.3:0/1354471837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f065c06fb20 0x7f065c09cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:18.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.275+0000 7f0660884700 1 --2- 172.21.15.3:0/1354471837 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f065c130540 0x7f065c09d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:18.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.275+0000 7f065bfff700 1 -- 172.21.15.3:0/1354471837 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f065c130540 msgr2=0x7f065c09d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:18.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.275+0000 7f065bfff700 1 --2- 172.21.15.3:0/1354471837 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f065c130540 0x7f065c09d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.275+0000 7f065bfff700 1 -- 172.21.15.3:0/1354471837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f065c06fb20 msgr2=0x7f065c09cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:18.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.275+0000 7f065bfff700 1 --2- 172.21.15.3:0/1354471837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f065c06fb20 0x7f065c09cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.275+0000 7f065bfff700 1 -- 172.21.15.3:0/1354471837 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065c125c50 con 0x7f0644004900 2023-10-03T10:19:18.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.276+0000 7f065bfff700 1 --2- 172.21.15.3:0/1354471837 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0644004900 0x7f065c09c630 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f064c00b580 tx=0x7f064c00b890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:18.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.278+0000 7f0658ff9700 1 -- 172.21.15.3:0/1354471837 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c011830 con 0x7f0644004900 2023-10-03T10:19:18.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.278+0000 7f0658ff9700 1 -- 172.21.15.3:0/1354471837 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064c011990 con 0x7f0644004900 2023-10-03T10:19:18.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.278+0000 7f0658ff9700 1 -- 172.21.15.3:0/1354471837 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c011d20 con 0x7f0644004900 2023-10-03T10:19:18.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.278+0000 7f06622e7700 1 -- 172.21.15.3:0/1354471837 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f065c127300 con 0x7f0644004900 2023-10-03T10:19:18.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.278+0000 7f06622e7700 1 -- 172.21.15.3:0/1354471837 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f065c127020 con 0x7f0644004900 2023-10-03T10:19:18.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.279+0000 7f06622e7700 1 -- 172.21.15.3:0/1354471837 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f065c074ea0 con 0x7f0644004900 2023-10-03T10:19:18.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.283+0000 7f0658ff9700 1 -- 172.21.15.3:0/1354471837 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f064c01a400 con 0x7f0644004900 2023-10-03T10:19:18.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.283+0000 7f0658ff9700 1 -- 172.21.15.3:0/1354471837 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f064c03f6d0 con 0x7f0644004900 2023-10-03T10:19:18.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.285+0000 7fd5ebfff700 1 -- 172.21.15.3:0/1561286135 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd5f8026480 con 0x7fd60409e1d0 2023-10-03T10:19:18.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.285+0000 7f0658ff9700 1 -- 172.21.15.3:0/1354471837 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f064c00f8d0 con 0x7f0644004900 2023-10-03T10:19:18.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.348+0000 7febeec83700 1 -- 172.21.15.3:0/905847900 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7febe8074770 con 0x7febe80a0a30 2023-10-03T10:19:18.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.349+0000 7febe57fa700 1 -- 172.21.15.3:0/905847900 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7febdc015020 con 0x7febe80a0a30 2023-10-03T10:19:18.349 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:18.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.351+0000 7febeec83700 1 -- 172.21.15.3:0/905847900 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febe80a0a30 msgr2=0x7febe81c86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:18.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.351+0000 7febeec83700 1 --2- 172.21.15.3:0/905847900 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febe80a0a30 0x7febe81c86d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7febdc009c40 tx=0x7febdc004750 comp rx=0 tx=0).stop 2023-10-03T10:19:18.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.351+0000 7febeec83700 1 -- 172.21.15.3:0/905847900 shutdown_connections 2023-10-03T10:19:18.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.351+0000 7febeec83700 1 --2- 172.21.15.3:0/905847900 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7febe8133930 0x7febe81c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.351+0000 7febeec83700 1 --2- 172.21.15.3:0/905847900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febe809d630 0x7febe81c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.351+0000 7febeec83700 1 --2- 172.21.15.3:0/905847900 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febe80a0a30 0x7febe81c86d0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.351+0000 7febeec83700 1 -- 172.21.15.3:0/905847900 >> 172.21.15.3:0/905847900 conn(0x7febe80a3050 msgr2=0x7febe81bbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:18.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.351+0000 7febeec83700 1 -- 172.21.15.3:0/905847900 shutdown_connections 2023-10-03T10:19:18.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.351+0000 7febeec83700 1 -- 172.21.15.3:0/905847900 wait complete. 2023-10-03T10:19:18.371 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:18.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.439+0000 7f6d9ea6a700 1 -- 172.21.15.3:0/1205527944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6d98074ea0 con 0x7f6d9809d6f0 2023-10-03T10:19:18.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.439+0000 7f6d957fa700 1 -- 172.21.15.3:0/1205527944 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6d8c0102e0 con 0x7f6d9809d6f0 2023-10-03T10:19:18.440 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:18.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.442+0000 7f6d9ea6a700 1 -- 172.21.15.3:0/1205527944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d9809d6f0 msgr2=0x7f6d9811e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.442+0000 7f6d9ea6a700 1 --2- 172.21.15.3:0/1205527944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d9809d6f0 0x7f6d9811e420 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00c4d0 tx=0x7f6d8c00c7e0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.442+0000 7f6d9ea6a700 1 -- 172.21.15.3:0/1205527944 shutdown_connections 2023-10-03T10:19:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.442+0000 7f6d9ea6a700 1 --2- 172.21.15.3:0/1205527944 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6d98135e50 0x7f6d981c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.442+0000 7f6d9ea6a700 1 --2- 172.21.15.3:0/1205527944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d9809d6f0 0x7f6d9811e420 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.442+0000 7f6d9ea6a700 1 --2- 172.21.15.3:0/1205527944 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6d9809e1d0 0x7f6d9811e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.442+0000 7f6d9ea6a700 1 -- 172.21.15.3:0/1205527944 >> 172.21.15.3:0/1205527944 conn(0x7f6d98066f50 msgr2=0x7f6d9806efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.442+0000 7f6d9ea6a700 1 -- 172.21.15.3:0/1205527944 shutdown_connections 2023-10-03T10:19:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.442+0000 7f6d9ea6a700 1 -- 172.21.15.3:0/1205527944 wait complete. 2023-10-03T10:19:18.456 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:18.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.493+0000 7f06622e7700 1 -- 172.21.15.3:0/1354471837 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f065c127af0 con 0x7f0644004900 2023-10-03T10:19:18.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.493+0000 7f0658ff9700 1 -- 172.21.15.3:0/1354471837 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f064c022350 con 0x7f0644004900 2023-10-03T10:19:18.494 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:18.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.496+0000 7f06622e7700 1 -- 172.21.15.3:0/1354471837 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0644004900 msgr2=0x7f065c09c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:18.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.496+0000 7f06622e7700 1 --2- 172.21.15.3:0/1354471837 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0644004900 0x7f065c09c630 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f064c00b580 tx=0x7f064c00b890 comp rx=0 tx=0).stop 2023-10-03T10:19:18.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.496+0000 7f06622e7700 1 -- 172.21.15.3:0/1354471837 shutdown_connections 2023-10-03T10:19:18.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.496+0000 7f06622e7700 1 --2- 172.21.15.3:0/1354471837 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f065c130540 0x7f065c09d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.496+0000 7f06622e7700 1 --2- 172.21.15.3:0/1354471837 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0644004900 0x7f065c09c630 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.496+0000 7f06622e7700 1 --2- 172.21.15.3:0/1354471837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f065c06fb20 0x7f065c09cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.496+0000 7f06622e7700 1 -- 172.21.15.3:0/1354471837 >> 172.21.15.3:0/1354471837 conn(0x7f065c0a2aa0 msgr2=0x7f065c06e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:18.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.496+0000 7f06622e7700 1 -- 172.21.15.3:0/1354471837 shutdown_connections 2023-10-03T10:19:18.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.496+0000 7f06622e7700 1 -- 172.21.15.3:0/1354471837 wait complete. 2023-10-03T10:19:18.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.505+0000 7fd5e9ffb700 1 -- 172.21.15.3:0/1561286135 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd5dc005590 con 0x7fd60409e1d0 2023-10-03T10:19:18.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.505+0000 7fd5ebfff700 1 -- 172.21.15.3:0/1561286135 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd5f802b070 con 0x7fd60409e1d0 2023-10-03T10:19:18.505 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:18.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.507+0000 7fd5e9ffb700 1 -- 172.21.15.3:0/1561286135 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd60409e1d0 msgr2=0x7fd6041cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:18.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.507+0000 7fd5e9ffb700 1 --2- 172.21.15.3:0/1561286135 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd60409e1d0 0x7fd6041cfe70 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8016000 tx=0x7fd5f8004750 comp rx=0 tx=0).stop 2023-10-03T10:19:18.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.507+0000 7fd5e9ffb700 1 -- 172.21.15.3:0/1561286135 shutdown_connections 2023-10-03T10:19:18.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.507+0000 7fd5e9ffb700 1 --2- 172.21.15.3:0/1561286135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd604139170 0x7fd6041d23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.507+0000 7fd5e9ffb700 1 --2- 172.21.15.3:0/1561286135 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd60409d6f0 0x7fd6041cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.507+0000 7fd5e9ffb700 1 --2- 172.21.15.3:0/1561286135 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd60409e1d0 0x7fd6041cfe70 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:18.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.507+0000 7fd5e9ffb700 1 -- 172.21.15.3:0/1561286135 >> 172.21.15.3:0/1561286135 conn(0x7fd604066f50 msgr2=0x7fd604070c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:18.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.508+0000 7fd5e9ffb700 1 -- 172.21.15.3:0/1561286135 shutdown_connections 2023-10-03T10:19:18.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:18.508+0000 7fd5e9ffb700 1 -- 172.21.15.3:0/1561286135 wait complete. 2023-10-03T10:19:18.510 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:18.521 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:19.372 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:19.457 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:19.511 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:19.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.521+0000 7f904be53700 1 Processor -- start 2023-10-03T10:19:19.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.521+0000 7f904be53700 1 -- start start 2023-10-03T10:19:19.522 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:19.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.522+0000 7f904be53700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9044133420 0x7f90441337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:19.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.522+0000 7f904be53700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9044133e30 0x7f904412ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:19.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.522+0000 7f904be53700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f904406b990 con 0x7f9044134260 2023-10-03T10:19:19.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.522+0000 7f904be53700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90440a0590 con 0x7f9044133420 2023-10-03T10:19:19.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.522+0000 7f904be53700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f904409fb80 con 0x7f9044133e30 2023-10-03T10:19:19.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.522+0000 7f90493ee700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9044133e30 0x7f904412ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:19.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.522+0000 7f90493ee700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9044133e30 0x7f904412ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:54354/0 (socket says 172.21.15.3:54354) 2023-10-03T10:19:19.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.522+0000 7f90493ee700 1 -- 172.21.15.3:0/1599814202 learned_addr learned my addr 172.21.15.3:0/1599814202 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:19.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.522+0000 7f9049bef700 1 --2- 172.21.15.3:0/1599814202 >> v2:172.21.15.3:3300/0 conn(0x7f9044133420 0x7f90441337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:19.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.523+0000 7f90493ee700 1 -- 172.21.15.3:0/1599814202 >> v2:172.21.15.3:3300/0 conn(0x7f9044133420 msgr2=0x7f90441337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:19.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.523+0000 7f90493ee700 1 --2- 172.21.15.3:0/1599814202 >> v2:172.21.15.3:3300/0 conn(0x7f9044133420 0x7f90441337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.523+0000 7f90493ee700 1 -- 172.21.15.3:0/1599814202 >> v1:172.21.15.3:6789/0 conn(0x7f9044134260 legacy=0x7f90441304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:19.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.523+0000 7f90493ee700 1 -- 172.21.15.3:0/1599814202 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90440a0030 con 0x7f9044133e30 2023-10-03T10:19:19.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.523+0000 7f90493ee700 1 --2- 172.21.15.3:0/1599814202 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9044133e30 0x7f904412ff90 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f90340098f0 tx=0x7f9034009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15b6e49c5dda82aa server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:19.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.523+0000 7f9048bed700 1 -- 172.21.15.3:0/1599814202 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9034011830 con 0x7f9044133e30 2023-10-03T10:19:19.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.523+0000 7f9048bed700 1 -- 172.21.15.3:0/1599814202 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9034011990 con 0x7f9044133e30 2023-10-03T10:19:19.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.523+0000 7f9048bed700 1 -- 172.21.15.3:0/1599814202 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9034011d20 con 0x7f9044133e30 2023-10-03T10:19:19.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.524+0000 7f904be53700 1 -- 172.21.15.3:0/1599814202 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9044133e30 msgr2=0x7f904412ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:19.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.524+0000 7f904be53700 1 --2- 172.21.15.3:0/1599814202 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9044133e30 0x7f904412ff90 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f90340098f0 tx=0x7f9034009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:19.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.524+0000 7f904be53700 1 -- 172.21.15.3:0/1599814202 shutdown_connections 2023-10-03T10:19:19.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.524+0000 7f904be53700 1 --2- 172.21.15.3:0/1599814202 >> v2:172.21.15.3:3300/0 conn(0x7f9044133420 0x7f90441337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.524+0000 7f904be53700 1 --2- 172.21.15.3:0/1599814202 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9044133e30 0x7f904412ff90 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.524+0000 7f904be53700 1 -- 172.21.15.3:0/1599814202 >> 172.21.15.3:0/1599814202 conn(0x7f90440a4210 msgr2=0x7f90440a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:19.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.525+0000 7f904be53700 1 -- 172.21.15.3:0/1599814202 shutdown_connections 2023-10-03T10:19:19.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.525+0000 7f904be53700 1 -- 172.21.15.3:0/1599814202 wait complete. 2023-10-03T10:19:19.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.526+0000 7f904be53700 1 Processor -- start 2023-10-03T10:19:19.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.526+0000 7f904be53700 1 -- start start 2023-10-03T10:19:19.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.526+0000 7f904be53700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9044133420 0x7f904409c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:19.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.526+0000 7f904be53700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9044133e30 0x7f904409cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:19.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.526+0000 7f904be53700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9044134260 0x7f904409d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:19.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.527+0000 7f904be53700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90441201b0 con 0x7f9044134260 2023-10-03T10:19:19.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.527+0000 7f904be53700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9044131a70 con 0x7f9044133420 2023-10-03T10:19:19.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.527+0000 7f904be53700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f904409dfe0 con 0x7f9044133e30 2023-10-03T10:19:19.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.527+0000 7f9049bef700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9044133420 0x7f904409c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:19.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.527+0000 7f9049bef700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9044133420 0x7f904409c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:54364/0 (socket says 172.21.15.3:54364) 2023-10-03T10:19:19.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.527+0000 7f9049bef700 1 -- 172.21.15.3:0/3824207717 learned_addr learned my addr 172.21.15.3:0/3824207717 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:19.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.527+0000 7f904a3f0700 1 --2- 172.21.15.3:0/3824207717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9044134260 0x7f904409d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:19.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.527+0000 7f904a3f0700 1 -- 172.21.15.3:0/3824207717 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9044133e30 msgr2=0x7f904409cc20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:19:19.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.527+0000 7f904a3f0700 1 --2- 172.21.15.3:0/3824207717 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9044133e30 0x7f904409cc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.527+0000 7f904a3f0700 1 -- 172.21.15.3:0/3824207717 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9044133420 msgr2=0x7f904409c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:19.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.527+0000 7f904a3f0700 1 --2- 172.21.15.3:0/3824207717 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9044133420 0x7f904409c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.527+0000 7f904a3f0700 1 -- 172.21.15.3:0/3824207717 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9034009550 con 0x7f9044134260 2023-10-03T10:19:19.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.527+0000 7f904a3f0700 1 --2- 172.21.15.3:0/3824207717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9044134260 0x7f904409d160 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f904409f3e0 tx=0x7f903c00ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:19.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.528+0000 7f903a7fc700 1 -- 172.21.15.3:0/3824207717 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c003c00 con 0x7f9044134260 2023-10-03T10:19:19.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.528+0000 7f903a7fc700 1 -- 172.21.15.3:0/3824207717 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903c003d60 con 0x7f9044134260 2023-10-03T10:19:19.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.528+0000 7f904be53700 1 -- 172.21.15.3:0/3824207717 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f904411fe80 con 0x7f9044134260 2023-10-03T10:19:19.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.528+0000 7f903a7fc700 1 -- 172.21.15.3:0/3824207717 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c00aa60 con 0x7f9044134260 2023-10-03T10:19:19.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.528+0000 7f904be53700 1 -- 172.21.15.3:0/3824207717 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f904411fb00 con 0x7f9044134260 2023-10-03T10:19:19.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.529+0000 7f903a7fc700 1 -- 172.21.15.3:0/3824207717 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f903c00abc0 con 0x7f9044134260 2023-10-03T10:19:19.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.529+0000 7f903a7fc700 1 -- 172.21.15.3:0/3824207717 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f903c03efb0 con 0x7f9044134260 2023-10-03T10:19:19.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.529+0000 7f904be53700 1 -- 172.21.15.3:0/3824207717 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90441202f0 con 0x7f9044134260 2023-10-03T10:19:19.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.535+0000 7f903a7fc700 1 -- 172.21.15.3:0/3824207717 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f90441202f0 con 0x7f9044134260 2023-10-03T10:19:19.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.615+0000 7f0c887c5700 1 Processor -- start 2023-10-03T10:19:19.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.615+0000 7f0c887c5700 1 -- start start 2023-10-03T10:19:19.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.616+0000 7f0c887c5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c8009d9f0 0x7f0c80136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:19.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.616+0000 7f0c887c5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c80136a70 0x7f0c80138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:19.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.616+0000 7f0c887c5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c8009e130 con 0x7f0c801316c0 2023-10-03T10:19:19.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.616+0000 7f0c887c5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c801321a0 con 0x7f0c80136a70 2023-10-03T10:19:19.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.616+0000 7f0c887c5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c801312b0 con 0x7f0c8009d9f0 2023-10-03T10:19:19.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.616+0000 7f0c86561700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0c801316c0 0x7f0c80131a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51776/0 (socket says 172.21.15.3:51776) 2023-10-03T10:19:19.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.616+0000 7f0c85d60700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c8009d9f0 0x7f0c80136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:19.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.616+0000 7f0c86561700 1 -- 172.21.15.3:0/719960230 learned_addr learned my addr 172.21.15.3:0/719960230 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:19.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.616+0000 7f0c86d62700 1 --2- 172.21.15.3:0/719960230 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c80136a70 0x7f0c80138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:19.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.616+0000 7f0c85d60700 1 -- 172.21.15.3:0/719960230 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c80136a70 msgr2=0x7f0c80138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:19.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.616+0000 7f0c85d60700 1 --2- 172.21.15.3:0/719960230 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c80136a70 0x7f0c80138e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.616+0000 7f0c85d60700 1 -- 172.21.15.3:0/719960230 >> v1:172.21.15.3:6789/0 conn(0x7f0c801316c0 legacy=0x7f0c80131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:19.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.616+0000 7f0c85d60700 1 -- 172.21.15.3:0/719960230 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8009f6e0 con 0x7f0c8009d9f0 2023-10-03T10:19:19.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.616+0000 7f0c8555f700 1 -- 172.21.15.3:0/719960230 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3791107227 0 0) 0x7f0c7c001110 con 0x7f0c801316c0 2023-10-03T10:19:19.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.617+0000 7f0c85d60700 1 --2- 172.21.15.3:0/719960230 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c8009d9f0 0x7f0c80136530 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f0c700098a0 tx=0x7f0c70009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28da5ab48b8942e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:19.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.617+0000 7f0c8555f700 1 -- 172.21.15.3:0/719960230 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70011830 con 0x7f0c8009d9f0 2023-10-03T10:19:19.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.617+0000 7f0c8555f700 1 -- 172.21.15.3:0/719960230 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c70011990 con 0x7f0c8009d9f0 2023-10-03T10:19:19.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.617+0000 7f0c8555f700 1 -- 172.21.15.3:0/719960230 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70011d20 con 0x7f0c8009d9f0 2023-10-03T10:19:19.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.618+0000 7f0c887c5700 1 -- 172.21.15.3:0/719960230 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c8009d9f0 msgr2=0x7f0c80136530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:19.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.618+0000 7f0c887c5700 1 --2- 172.21.15.3:0/719960230 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c8009d9f0 0x7f0c80136530 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f0c700098a0 tx=0x7f0c70009bb0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.618+0000 7f0c887c5700 1 -- 172.21.15.3:0/719960230 shutdown_connections 2023-10-03T10:19:19.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.618+0000 7f0c887c5700 1 --2- 172.21.15.3:0/719960230 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c80136a70 0x7f0c80138e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.618+0000 7f0c887c5700 1 --2- 172.21.15.3:0/719960230 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c8009d9f0 0x7f0c80136530 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.618+0000 7f0c887c5700 1 -- 172.21.15.3:0/719960230 >> 172.21.15.3:0/719960230 conn(0x7f0c80066f50 msgr2=0x7f0c8009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:19.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.618+0000 7f0c887c5700 1 -- 172.21.15.3:0/719960230 shutdown_connections 2023-10-03T10:19:19.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.618+0000 7f0c887c5700 1 -- 172.21.15.3:0/719960230 wait complete. 2023-10-03T10:19:19.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.619+0000 7f0c887c5700 1 Processor -- start 2023-10-03T10:19:19.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.620+0000 7f0c887c5700 1 -- start start 2023-10-03T10:19:19.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.620+0000 7f0c887c5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c8009d9f0 0x7f0c8012ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:19.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.620+0000 7f0c887c5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c801316c0 0x7f0c80134680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:19.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.620+0000 7f0c887c5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c80136a70 0x7f0c80134bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:19.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.620+0000 7f0c887c5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8009bb10 con 0x7f0c8009d9f0 2023-10-03T10:19:19.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.620+0000 7f0c887c5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c80072760 con 0x7f0c80136a70 2023-10-03T10:19:19.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.620+0000 7f0c887c5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8009a750 con 0x7f0c801316c0 2023-10-03T10:19:19.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.620+0000 7f0c86561700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c8009d9f0 0x7f0c8012ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:19.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.621+0000 7f0c86561700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c8009d9f0 0x7f0c8012ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54662/0 (socket says 172.21.15.3:54662) 2023-10-03T10:19:19.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.621+0000 7f0c86561700 1 -- 172.21.15.3:0/1720467067 learned_addr learned my addr 172.21.15.3:0/1720467067 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:19.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.621+0000 7f0c86d62700 1 --2- 172.21.15.3:0/1720467067 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c80136a70 0x7f0c80134bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:19.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.621+0000 7f0c85d60700 1 --2- 172.21.15.3:0/1720467067 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c801316c0 0x7f0c80134680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:19.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.621+0000 7f0c86d62700 1 -- 172.21.15.3:0/1720467067 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c801316c0 msgr2=0x7f0c80134680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:19.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.621+0000 7f0c86d62700 1 --2- 172.21.15.3:0/1720467067 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c801316c0 0x7f0c80134680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.621+0000 7f0c86d62700 1 -- 172.21.15.3:0/1720467067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c8009d9f0 msgr2=0x7f0c8012ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:19.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.621+0000 7f0c86d62700 1 --2- 172.21.15.3:0/1720467067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c8009d9f0 0x7f0c8012ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.621+0000 7f0c86d62700 1 -- 172.21.15.3:0/1720467067 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c700094a0 con 0x7f0c80136a70 2023-10-03T10:19:19.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.621+0000 7f0c86561700 1 --2- 172.21.15.3:0/1720467067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c8009d9f0 0x7f0c8012ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:19.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.621+0000 7f0c86d62700 1 --2- 172.21.15.3:0/1720467067 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c80136a70 0x7f0c80134bc0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f0c8012edd0 tx=0x7f0c7800ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:19.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.621+0000 7f0c76ffd700 1 -- 172.21.15.3:0/1720467067 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7800cc20 con 0x7f0c80136a70 2023-10-03T10:19:19.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.621+0000 7f0c76ffd700 1 -- 172.21.15.3:0/1720467067 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7800f040 con 0x7f0c80136a70 2023-10-03T10:19:19.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.621+0000 7f0c887c5700 1 -- 172.21.15.3:0/1720467067 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c80136550 con 0x7f0c80136a70 2023-10-03T10:19:19.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.622+0000 7f0c76ffd700 1 -- 172.21.15.3:0/1720467067 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c78009e60 con 0x7f0c80136a70 2023-10-03T10:19:19.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.622+0000 7f0c887c5700 1 -- 172.21.15.3:0/1720467067 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c8006eb50 con 0x7f0c80136a70 2023-10-03T10:19:19.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.622+0000 7f0c76ffd700 1 -- 172.21.15.3:0/1720467067 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0c78014480 con 0x7f0c80136a70 2023-10-03T10:19:19.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.623+0000 7f0c76ffd700 1 -- 172.21.15.3:0/1720467067 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0c780402b0 con 0x7f0c80136a70 2023-10-03T10:19:19.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.623+0000 7f0c887c5700 1 -- 172.21.15.3:0/1720467067 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c8012b900 con 0x7f0c80136a70 2023-10-03T10:19:19.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.633+0000 7f0c76ffd700 1 -- 172.21.15.3:0/1720467067 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0c78018070 con 0x7f0c80136a70 2023-10-03T10:19:19.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.659+0000 7ff1147cf700 1 Processor -- start 2023-10-03T10:19:19.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.659+0000 7ff1147cf700 1 -- start start 2023-10-03T10:19:19.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.659+0000 7ff1147cf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff10c09d6f0 0x7ff10c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:19.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.659+0000 7ff1147cf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff10c09e410 0x7ff10c130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:19.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.659+0000 7ff1147cf700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10c09f130 con 0x7ff10c09e000 2023-10-03T10:19:19.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.659+0000 7ff1147cf700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff10c09f520 con 0x7ff10c09e410 2023-10-03T10:19:19.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.659+0000 7ff1147cf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff10c09e930 con 0x7ff10c09d6f0 2023-10-03T10:19:19.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.660+0000 7ff111d6a700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff10c09e000 0x7ff10c130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51778/0 (socket says 172.21.15.3:51778) 2023-10-03T10:19:19.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.660+0000 7ff111d6a700 1 -- 172.21.15.3:0/3572233958 learned_addr learned my addr 172.21.15.3:0/3572233958 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:19.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.660+0000 7ff11256b700 1 --2- 172.21.15.3:0/3572233958 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff10c09d6f0 0x7ff10c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:19.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.660+0000 7ff112d6c700 1 --2- 172.21.15.3:0/3572233958 >> v2:172.21.15.3:3300/0 conn(0x7ff10c09e410 0x7ff10c130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:19.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.660+0000 7ff11256b700 1 -- 172.21.15.3:0/3572233958 >> v2:172.21.15.3:3300/0 conn(0x7ff10c09e410 msgr2=0x7ff10c130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:19.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.660+0000 7ff11256b700 1 --2- 172.21.15.3:0/3572233958 >> v2:172.21.15.3:3300/0 conn(0x7ff10c09e410 0x7ff10c130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.660+0000 7ff11256b700 1 -- 172.21.15.3:0/3572233958 >> v1:172.21.15.3:6789/0 conn(0x7ff10c09e000 legacy=0x7ff10c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:19.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.660+0000 7ff111569700 1 -- 172.21.15.3:0/3572233958 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1228007613 0 0) 0x7ff10c09f130 con 0x7ff10c09e000 2023-10-03T10:19:19.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.660+0000 7ff11256b700 1 -- 172.21.15.3:0/3572233958 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10c09b2d0 con 0x7ff10c09d6f0 2023-10-03T10:19:19.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.660+0000 7ff11256b700 1 --2- 172.21.15.3:0/3572233958 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff10c09d6f0 0x7ff10c09dac0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff108009840 tx=0x7ff108009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=803d892e0efe8062 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:19.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.661+0000 7ff111569700 1 -- 172.21.15.3:0/3572233958 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff108011830 con 0x7ff10c09d6f0 2023-10-03T10:19:19.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.661+0000 7ff111569700 1 -- 172.21.15.3:0/3572233958 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff108011990 con 0x7ff10c09d6f0 2023-10-03T10:19:19.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.661+0000 7ff111569700 1 -- 172.21.15.3:0/3572233958 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff108011d20 con 0x7ff10c09d6f0 2023-10-03T10:19:19.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.661+0000 7ff1147cf700 1 -- 172.21.15.3:0/3572233958 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff10c09d6f0 msgr2=0x7ff10c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:19.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.661+0000 7ff1147cf700 1 --2- 172.21.15.3:0/3572233958 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff10c09d6f0 0x7ff10c09dac0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff108009840 tx=0x7ff108009b50 comp rx=0 tx=0).stop 2023-10-03T10:19:19.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.661+0000 7ff1147cf700 1 -- 172.21.15.3:0/3572233958 shutdown_connections 2023-10-03T10:19:19.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.661+0000 7ff1147cf700 1 --2- 172.21.15.3:0/3572233958 >> v2:172.21.15.3:3300/0 conn(0x7ff10c09e410 0x7ff10c130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.661+0000 7ff1147cf700 1 --2- 172.21.15.3:0/3572233958 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff10c09d6f0 0x7ff10c09dac0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.661+0000 7ff1147cf700 1 -- 172.21.15.3:0/3572233958 >> 172.21.15.3:0/3572233958 conn(0x7ff10c066f50 msgr2=0x7ff10c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:19.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.662+0000 7ff1147cf700 1 -- 172.21.15.3:0/3572233958 shutdown_connections 2023-10-03T10:19:19.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.662+0000 7ff1147cf700 1 -- 172.21.15.3:0/3572233958 wait complete. 2023-10-03T10:19:19.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.663+0000 7ff1147cf700 1 Processor -- start 2023-10-03T10:19:19.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.663+0000 7ff1147cf700 1 -- start start 2023-10-03T10:19:19.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.663+0000 7ff1147cf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff10c09d6f0 0x7ff10c1222b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:19.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.663+0000 7ff1147cf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff10c09e000 0x7ff10c1227f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:19.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.663+0000 7ff1147cf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff10c09e410 0x7ff10c122d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:19.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.663+0000 7ff1147cf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10c125830 con 0x7ff10c09e410 2023-10-03T10:19:19.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.663+0000 7ff1147cf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff10c125dd0 con 0x7ff10c09e000 2023-10-03T10:19:19.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.663+0000 7ff1147cf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff10c1246e0 con 0x7ff10c09d6f0 2023-10-03T10:19:19.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.663+0000 7ff111d6a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff10c09e000 0x7ff10c1227f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:19.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.663+0000 7ff111d6a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff10c09e000 0x7ff10c1227f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:54396/0 (socket says 172.21.15.3:54396) 2023-10-03T10:19:19.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.663+0000 7ff111d6a700 1 -- 172.21.15.3:0/2651230861 learned_addr learned my addr 172.21.15.3:0/2651230861 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:19.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.663+0000 7ff112d6c700 1 --2- 172.21.15.3:0/2651230861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff10c09e410 0x7ff10c122d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:19.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.664+0000 7ff11256b700 1 --2- 172.21.15.3:0/2651230861 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff10c09d6f0 0x7ff10c1222b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:19.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.664+0000 7ff111d6a700 1 -- 172.21.15.3:0/2651230861 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff10c09d6f0 msgr2=0x7ff10c1222b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:19.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.664+0000 7ff111d6a700 1 --2- 172.21.15.3:0/2651230861 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff10c09d6f0 0x7ff10c1222b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.664+0000 7ff111d6a700 1 -- 172.21.15.3:0/2651230861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff10c09e410 msgr2=0x7ff10c122d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:19.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.664+0000 7ff111d6a700 1 --2- 172.21.15.3:0/2651230861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff10c09e410 0x7ff10c122d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.664+0000 7ff111d6a700 1 -- 172.21.15.3:0/2651230861 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1080094a0 con 0x7ff10c09e000 2023-10-03T10:19:19.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.664+0000 7ff11256b700 1 --2- 172.21.15.3:0/2651230861 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff10c09d6f0 0x7ff10c1222b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:19.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.664+0000 7ff111d6a700 1 --2- 172.21.15.3:0/2651230861 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff10c09e000 0x7ff10c1227f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc001eb0 tx=0x7ff0fc00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:19.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.664+0000 7ff102ffd700 1 -- 172.21.15.3:0/2651230861 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc004c80 con 0x7ff10c09e000 2023-10-03T10:19:19.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.664+0000 7ff102ffd700 1 -- 172.21.15.3:0/2651230861 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0fc004de0 con 0x7ff10c09e000 2023-10-03T10:19:19.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.664+0000 7ff102ffd700 1 -- 172.21.15.3:0/2651230861 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc007310 con 0x7ff10c09e000 2023-10-03T10:19:19.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.664+0000 7ff1147cf700 1 -- 172.21.15.3:0/2651230861 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff10c072150 con 0x7ff10c09e000 2023-10-03T10:19:19.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.664+0000 7ff1147cf700 1 -- 172.21.15.3:0/2651230861 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff10c125af0 con 0x7ff10c09e000 2023-10-03T10:19:19.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.665+0000 7ff102ffd700 1 -- 172.21.15.3:0/2651230861 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff0fc007470 con 0x7ff10c09e000 2023-10-03T10:19:19.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.665+0000 7ff1147cf700 1 -- 172.21.15.3:0/2651230861 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff10c074ea0 con 0x7ff10c09e000 2023-10-03T10:19:19.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.666+0000 7ff102ffd700 1 -- 172.21.15.3:0/2651230861 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff0fc03f3a0 con 0x7ff10c09e000 2023-10-03T10:19:19.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.667+0000 7f7e0c30c700 1 Processor -- start 2023-10-03T10:19:19.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.667+0000 7f7e0c30c700 1 -- start start 2023-10-03T10:19:19.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.668+0000 7f7e0c30c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7e0406a770 0x7f7e0409d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:19.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.668+0000 7f7e0c30c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e0409dc30 0x7f7e0409e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:19.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.668+0000 7f7e0c30c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e0413c1e0 0x7f7e0413e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:19.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.668+0000 7f7e0c30c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e0406ab40 con 0x7f7e0406a770 2023-10-03T10:19:19.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.668+0000 7f7e0c30c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0409ebc0 con 0x7f7e0413c1e0 2023-10-03T10:19:19.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.668+0000 7f7e0c30c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0409a810 con 0x7f7e0409dc30 2023-10-03T10:19:19.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.668+0000 7f7e0a8a9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e0413c1e0 0x7f7e0413e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:19.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.668+0000 7f7e0a8a9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e0413c1e0 0x7f7e0413e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:54410/0 (socket says 172.21.15.3:54410) 2023-10-03T10:19:19.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.668+0000 7f7e0a8a9700 1 -- 172.21.15.3:0/2451398426 learned_addr learned my addr 172.21.15.3:0/2451398426 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:19.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.668+0000 7f7e0a0a8700 1 --2- 172.21.15.3:0/2451398426 >> v2:172.21.15.3:3300/0 conn(0x7f7e0406a770 0x7f7e0409d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:19.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.668+0000 7f7e098a7700 1 --2- 172.21.15.3:0/2451398426 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e0409dc30 0x7f7e0409e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:19.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.668+0000 7f7e0a8a9700 1 -- 172.21.15.3:0/2451398426 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e0409dc30 msgr2=0x7f7e0409e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:19.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.668+0000 7f7e0a8a9700 1 --2- 172.21.15.3:0/2451398426 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e0409dc30 0x7f7e0409e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.668+0000 7f7e0a8a9700 1 -- 172.21.15.3:0/2451398426 >> v2:172.21.15.3:3300/0 conn(0x7f7e0406a770 msgr2=0x7f7e0409d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:19.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.668+0000 7f7e0a8a9700 1 --2- 172.21.15.3:0/2451398426 >> v2:172.21.15.3:3300/0 conn(0x7f7e0406a770 0x7f7e0409d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.668+0000 7f7e0a8a9700 1 -- 172.21.15.3:0/2451398426 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e0409e920 con 0x7f7e0413c1e0 2023-10-03T10:19:19.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.669+0000 7f7e098a7700 1 --2- 172.21.15.3:0/2451398426 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e0409dc30 0x7f7e0409e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:19.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.669+0000 7f7e0a8a9700 1 --2- 172.21.15.3:0/2451398426 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e0413c1e0 0x7f7e0413e5c0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc009e50 tx=0x7f7dfc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a81ad792542c578 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:19.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.671+0000 7f7e090a6700 1 -- 172.21.15.3:0/2451398426 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc00e070 con 0x7f7e0413c1e0 2023-10-03T10:19:19.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.671+0000 7f7e090a6700 1 -- 172.21.15.3:0/2451398426 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dfc00b910 con 0x7f7e0413c1e0 2023-10-03T10:19:19.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.671+0000 7f7e090a6700 1 -- 172.21.15.3:0/2451398426 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc00bcc0 con 0x7f7e0413c1e0 2023-10-03T10:19:19.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.671+0000 7ff102ffd700 1 -- 172.21.15.3:0/2651230861 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff0fc0106d0 con 0x7ff10c09e000 2023-10-03T10:19:19.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.671+0000 7f7e0c30c700 1 -- 172.21.15.3:0/2451398426 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e0413c1e0 msgr2=0x7f7e0413e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:19.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.671+0000 7f7e0c30c700 1 --2- 172.21.15.3:0/2451398426 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e0413c1e0 0x7f7e0413e5c0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc009e50 tx=0x7f7dfc00b330 comp rx=0 tx=0).stop 2023-10-03T10:19:19.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.672+0000 7f7e0c30c700 1 -- 172.21.15.3:0/2451398426 shutdown_connections 2023-10-03T10:19:19.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.672+0000 7f7e0c30c700 1 --2- 172.21.15.3:0/2451398426 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e0413c1e0 0x7f7e0413e5c0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.672+0000 7f7e0c30c700 1 --2- 172.21.15.3:0/2451398426 >> v2:172.21.15.3:3300/0 conn(0x7f7e0406a770 0x7f7e0409d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.672+0000 7f7e0c30c700 1 --2- 172.21.15.3:0/2451398426 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e0409dc30 0x7f7e0409e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.672+0000 7f7e0c30c700 1 -- 172.21.15.3:0/2451398426 >> 172.21.15.3:0/2451398426 conn(0x7f7e04066f50 msgr2=0x7f7e0409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:19.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.672+0000 7f7e0c30c700 1 -- 172.21.15.3:0/2451398426 shutdown_connections 2023-10-03T10:19:19.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.672+0000 7f7e0c30c700 1 -- 172.21.15.3:0/2451398426 wait complete. 2023-10-03T10:19:19.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.673+0000 7f7e0c30c700 1 Processor -- start 2023-10-03T10:19:19.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.673+0000 7f7e0c30c700 1 -- start start 2023-10-03T10:19:19.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.673+0000 7f7e0c30c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e0406a770 0x7f7e0413ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:19.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.673+0000 7f7e0c30c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e0409dc30 0x7f7e0413a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:19.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.673+0000 7f7e0c30c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e0413c1e0 0x7f7e0413ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:19.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.673+0000 7f7e0c30c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e040a1a00 con 0x7f7e0413c1e0 2023-10-03T10:19:19.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.673+0000 7f7e0c30c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0411ed80 con 0x7f7e0406a770 2023-10-03T10:19:19.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.673+0000 7f7e0c30c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0411d0c0 con 0x7f7e0409dc30 2023-10-03T10:19:19.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.674+0000 7f7e0a0a8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e0406a770 0x7f7e0413ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:19.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.674+0000 7f7e098a7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e0409dc30 0x7f7e0413a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:19.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.674+0000 7f7e0a0a8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e0406a770 0x7f7e0413ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:54422/0 (socket says 172.21.15.3:54422) 2023-10-03T10:19:19.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.674+0000 7f7e0a0a8700 1 -- 172.21.15.3:0/417618514 learned_addr learned my addr 172.21.15.3:0/417618514 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:19.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.674+0000 7f7e0a8a9700 1 --2- 172.21.15.3:0/417618514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e0413c1e0 0x7f7e0413ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:19.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.674+0000 7f7e0a8a9700 1 -- 172.21.15.3:0/417618514 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e0409dc30 msgr2=0x7f7e0413a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:19.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.674+0000 7f7e0a8a9700 1 --2- 172.21.15.3:0/417618514 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e0409dc30 0x7f7e0413a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.674+0000 7f7e0a8a9700 1 -- 172.21.15.3:0/417618514 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e0406a770 msgr2=0x7f7e0413ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:19.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.674+0000 7f7e0a8a9700 1 --2- 172.21.15.3:0/417618514 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e0406a770 0x7f7e0413ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.674+0000 7f7e0a8a9700 1 -- 172.21.15.3:0/417618514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dfc009b30 con 0x7f7e0413c1e0 2023-10-03T10:19:19.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.674+0000 7f7e0a8a9700 1 --2- 172.21.15.3:0/417618514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e0413c1e0 0x7f7e0413ab40 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f7e0406b400 tx=0x7f7dfc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:19.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.675+0000 7f7dfaffd700 1 -- 172.21.15.3:0/417618514 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc004450 con 0x7f7e0413c1e0 2023-10-03T10:19:19.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.675+0000 7f7dfaffd700 1 -- 172.21.15.3:0/417618514 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dfc0045d0 con 0x7f7e0413c1e0 2023-10-03T10:19:19.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.675+0000 7f7e0c30c700 1 -- 172.21.15.3:0/417618514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e0409f1a0 con 0x7f7e0413c1e0 2023-10-03T10:19:19.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.675+0000 7f7dfaffd700 1 -- 172.21.15.3:0/417618514 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc0092e0 con 0x7f7e0413c1e0 2023-10-03T10:19:19.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.675+0000 7f7e0c30c700 1 -- 172.21.15.3:0/417618514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e0411f440 con 0x7f7e0413c1e0 2023-10-03T10:19:19.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.676+0000 7f7e0c30c700 1 -- 172.21.15.3:0/417618514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e0413c040 con 0x7f7e0413c1e0 2023-10-03T10:19:19.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.676+0000 7f7dfaffd700 1 -- 172.21.15.3:0/417618514 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7dfc025460 con 0x7f7e0413c1e0 2023-10-03T10:19:19.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.676+0000 7f7dfaffd700 1 -- 172.21.15.3:0/417618514 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7dfc041050 con 0x7f7e0413c1e0 2023-10-03T10:19:19.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.686+0000 7f7dfaffd700 1 -- 172.21.15.3:0/417618514 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7e0413c040 con 0x7f7e0413c1e0 2023-10-03T10:19:19.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.772+0000 7f904be53700 1 -- 172.21.15.3:0/3824207717 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f90441202f0 con 0x7f9044134260 2023-10-03T10:19:19.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.772+0000 7f903a7fc700 1 -- 172.21.15.3:0/3824207717 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f90441202f0 con 0x7f9044134260 2023-10-03T10:19:19.772 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:19.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.774+0000 7f904be53700 1 -- 172.21.15.3:0/3824207717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9044134260 msgr2=0x7f904409d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:19.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.774+0000 7f904be53700 1 --2- 172.21.15.3:0/3824207717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9044134260 0x7f904409d160 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f904409f3e0 tx=0x7f903c00ce70 comp rx=0 tx=0).stop 2023-10-03T10:19:19.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.775+0000 7f904be53700 1 -- 172.21.15.3:0/3824207717 shutdown_connections 2023-10-03T10:19:19.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.775+0000 7f904be53700 1 --2- 172.21.15.3:0/3824207717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9044134260 0x7f904409d160 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.775+0000 7f904be53700 1 --2- 172.21.15.3:0/3824207717 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9044133420 0x7f904409c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.775+0000 7f904be53700 1 --2- 172.21.15.3:0/3824207717 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9044133e30 0x7f904409cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.775+0000 7f904be53700 1 -- 172.21.15.3:0/3824207717 >> 172.21.15.3:0/3824207717 conn(0x7f90440a4210 msgr2=0x7f90440a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:19.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.775+0000 7f904be53700 1 -- 172.21.15.3:0/3824207717 shutdown_connections 2023-10-03T10:19:19.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.775+0000 7f904be53700 1 -- 172.21.15.3:0/3824207717 wait complete. 2023-10-03T10:19:19.789 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:19.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.845+0000 7f0c887c5700 1 -- 172.21.15.3:0/1720467067 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0c80131480 con 0x7f0c80136a70 2023-10-03T10:19:19.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.846+0000 7f0c76ffd700 1 -- 172.21.15.3:0/1720467067 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0c7802c090 con 0x7f0c80136a70 2023-10-03T10:19:19.846 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:19.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.848+0000 7f0c887c5700 1 -- 172.21.15.3:0/1720467067 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c80136a70 msgr2=0x7f0c80134bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:19.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.848+0000 7f0c887c5700 1 --2- 172.21.15.3:0/1720467067 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c80136a70 0x7f0c80134bc0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f0c8012edd0 tx=0x7f0c7800ef90 comp rx=0 tx=0).stop 2023-10-03T10:19:19.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.848+0000 7f0c887c5700 1 -- 172.21.15.3:0/1720467067 shutdown_connections 2023-10-03T10:19:19.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.848+0000 7f0c887c5700 1 --2- 172.21.15.3:0/1720467067 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0c80136a70 0x7f0c80134bc0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.848+0000 7f0c887c5700 1 --2- 172.21.15.3:0/1720467067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c8009d9f0 0x7f0c8012ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.848+0000 7f0c887c5700 1 --2- 172.21.15.3:0/1720467067 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0c801316c0 0x7f0c80134680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.848+0000 7f0c887c5700 1 -- 172.21.15.3:0/1720467067 >> 172.21.15.3:0/1720467067 conn(0x7f0c80066f50 msgr2=0x7f0c80136e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:19.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.848+0000 7f0c887c5700 1 -- 172.21.15.3:0/1720467067 shutdown_connections 2023-10-03T10:19:19.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.848+0000 7f0c887c5700 1 -- 172.21.15.3:0/1720467067 wait complete. 2023-10-03T10:19:19.862 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:19.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.885+0000 7ff1147cf700 1 -- 172.21.15.3:0/2651230861 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff10c06c360 con 0x7ff10c09e000 2023-10-03T10:19:19.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.885+0000 7ff102ffd700 1 -- 172.21.15.3:0/2651230861 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff0fc011360 con 0x7ff10c09e000 2023-10-03T10:19:19.886 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:19.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.889+0000 7ff1147cf700 1 -- 172.21.15.3:0/2651230861 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff10c09e000 msgr2=0x7ff10c1227f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:19.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.889+0000 7ff1147cf700 1 --2- 172.21.15.3:0/2651230861 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff10c09e000 0x7ff10c1227f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc001eb0 tx=0x7ff0fc00c800 comp rx=0 tx=0).stop 2023-10-03T10:19:19.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.889+0000 7ff1147cf700 1 -- 172.21.15.3:0/2651230861 shutdown_connections 2023-10-03T10:19:19.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.889+0000 7ff1147cf700 1 --2- 172.21.15.3:0/2651230861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff10c09e410 0x7ff10c122d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.889+0000 7ff1147cf700 1 --2- 172.21.15.3:0/2651230861 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff10c09d6f0 0x7ff10c1222b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.889+0000 7ff1147cf700 1 --2- 172.21.15.3:0/2651230861 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff10c09e000 0x7ff10c1227f0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.889+0000 7ff1147cf700 1 -- 172.21.15.3:0/2651230861 >> 172.21.15.3:0/2651230861 conn(0x7ff10c066f50 msgr2=0x7ff10c070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:19.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.889+0000 7ff1147cf700 1 -- 172.21.15.3:0/2651230861 shutdown_connections 2023-10-03T10:19:19.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.889+0000 7ff1147cf700 1 -- 172.21.15.3:0/2651230861 wait complete. 2023-10-03T10:19:19.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.898+0000 7f7e0c30c700 1 -- 172.21.15.3:0/417618514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7e0413c040 con 0x7f7e0413c1e0 2023-10-03T10:19:19.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.899+0000 7f7dfaffd700 1 -- 172.21.15.3:0/417618514 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7e0413c040 con 0x7f7e0413c1e0 2023-10-03T10:19:19.899 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:19.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.901+0000 7f7e0c30c700 1 -- 172.21.15.3:0/417618514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e0413c1e0 msgr2=0x7f7e0413ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:19.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.901+0000 7f7e0c30c700 1 --2- 172.21.15.3:0/417618514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e0413c1e0 0x7f7e0413ab40 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f7e0406b400 tx=0x7f7dfc004ad0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.901+0000 7f7e0c30c700 1 -- 172.21.15.3:0/417618514 shutdown_connections 2023-10-03T10:19:19.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.901+0000 7f7e0c30c700 1 --2- 172.21.15.3:0/417618514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e0413c1e0 0x7f7e0413ab40 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.901+0000 7f7e0c30c700 1 --2- 172.21.15.3:0/417618514 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7e0406a770 0x7f7e0413ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.901+0000 7f7e0c30c700 1 --2- 172.21.15.3:0/417618514 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7e0409dc30 0x7f7e0413a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:19.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.902+0000 7f7e0c30c700 1 -- 172.21.15.3:0/417618514 >> 172.21.15.3:0/417618514 conn(0x7f7e04066f50 msgr2=0x7f7e0409b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:19.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.902+0000 7f7e0c30c700 1 -- 172.21.15.3:0/417618514 shutdown_connections 2023-10-03T10:19:19.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:19.902+0000 7f7e0c30c700 1 -- 172.21.15.3:0/417618514 wait complete. 2023-10-03T10:19:19.903 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:19.915 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:20.790 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:20.863 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:20.904 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:20.916 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:20.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.948+0000 7fceac339700 1 Processor -- start 2023-10-03T10:19:20.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.948+0000 7fceac339700 1 -- start start 2023-10-03T10:19:20.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.948+0000 7fceac339700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcea409e1d0 0x7fcea4135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:20.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.948+0000 7fceac339700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcea4135e50 0x7fcea4138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:20.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.948+0000 7fceac339700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea409f130 con 0x7fcea409d6f0 2023-10-03T10:19:20.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.948+0000 7fceac339700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea409f520 con 0x7fcea409e1d0 2023-10-03T10:19:20.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.948+0000 7fceac339700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea409e930 con 0x7fcea4135e50 2023-10-03T10:19:20.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.949+0000 7fcea98d4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcea409e1d0 0x7fcea4135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:20.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.949+0000 7fceaa8d6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcea4135e50 0x7fcea4138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:20.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.949+0000 7fcea98d4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcea409e1d0 0x7fcea4135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54708/0 (socket says 172.21.15.3:54708) 2023-10-03T10:19:20.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.949+0000 7fcea98d4700 1 -- 172.21.15.3:0/2008242294 learned_addr learned my addr 172.21.15.3:0/2008242294 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:20.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.949+0000 7fceaa8d6700 1 -- 172.21.15.3:0/2008242294 >> v2:172.21.15.3:3300/0 conn(0x7fcea409e1d0 msgr2=0x7fcea4135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:20.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.949+0000 7fceaa8d6700 1 --2- 172.21.15.3:0/2008242294 >> v2:172.21.15.3:3300/0 conn(0x7fcea409e1d0 0x7fcea4135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:20.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.949+0000 7fceaa8d6700 1 -- 172.21.15.3:0/2008242294 >> v1:172.21.15.3:6789/0 conn(0x7fcea409d6f0 legacy=0x7fcea409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:20.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.949+0000 7fceaa8d6700 1 -- 172.21.15.3:0/2008242294 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea409b2d0 con 0x7fcea4135e50 2023-10-03T10:19:20.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.949+0000 7fcea98d4700 1 --2- 172.21.15.3:0/2008242294 >> v2:172.21.15.3:3300/0 conn(0x7fcea409e1d0 0x7fcea4135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:20.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.949+0000 7fceaa8d6700 1 --2- 172.21.15.3:0/2008242294 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcea4135e50 0x7fcea4138230 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009d90 tx=0x7fce9c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a803370836584d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:20.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.949+0000 7fcea90d3700 1 -- 172.21.15.3:0/2008242294 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c00e070 con 0x7fcea4135e50 2023-10-03T10:19:20.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.950+0000 7fcea90d3700 1 -- 172.21.15.3:0/2008242294 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9c00bb20 con 0x7fcea4135e50 2023-10-03T10:19:20.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.950+0000 7fcea90d3700 1 -- 172.21.15.3:0/2008242294 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c012430 con 0x7fcea4135e50 2023-10-03T10:19:20.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.951+0000 7fceac339700 1 -- 172.21.15.3:0/2008242294 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcea4135e50 msgr2=0x7fcea4138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:20.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.951+0000 7fceac339700 1 --2- 172.21.15.3:0/2008242294 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcea4135e50 0x7fcea4138230 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009d90 tx=0x7fce9c00b330 comp rx=0 tx=0).stop 2023-10-03T10:19:20.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.951+0000 7fceac339700 1 -- 172.21.15.3:0/2008242294 shutdown_connections 2023-10-03T10:19:20.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.951+0000 7fceac339700 1 --2- 172.21.15.3:0/2008242294 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcea4135e50 0x7fcea4138230 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:20.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.951+0000 7fceac339700 1 --2- 172.21.15.3:0/2008242294 >> v2:172.21.15.3:3300/0 conn(0x7fcea409e1d0 0x7fcea4135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:20.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.951+0000 7fceac339700 1 -- 172.21.15.3:0/2008242294 >> 172.21.15.3:0/2008242294 conn(0x7fcea4066f50 msgr2=0x7fcea409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:20.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.951+0000 7fceac339700 1 -- 172.21.15.3:0/2008242294 shutdown_connections 2023-10-03T10:19:20.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.951+0000 7fceac339700 1 -- 172.21.15.3:0/2008242294 wait complete. 2023-10-03T10:19:20.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.952+0000 7fceac339700 1 Processor -- start 2023-10-03T10:19:20.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.952+0000 7fceac339700 1 -- start start 2023-10-03T10:19:20.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.953+0000 7fceac339700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcea409d6f0 0x7fcea41347b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:20.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.953+0000 7fceac339700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcea409e1d0 0x7fcea4134cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:20.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.953+0000 7fceac339700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcea4135e50 0x7fcea4135230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:20.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.953+0000 7fceac339700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea41199f0 con 0x7fcea409e1d0 2023-10-03T10:19:20.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.953+0000 7fceac339700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea411ac40 con 0x7fcea4135e50 2023-10-03T10:19:20.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.953+0000 7fceac339700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcea4135c10 con 0x7fcea409d6f0 2023-10-03T10:19:20.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.953+0000 7fceaa0d5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcea409d6f0 0x7fcea41347b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:20.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.953+0000 7fceaa0d5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcea409d6f0 0x7fcea41347b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41794/0 (socket says 172.21.15.3:41794) 2023-10-03T10:19:20.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.953+0000 7fceaa0d5700 1 -- 172.21.15.3:0/626859294 learned_addr learned my addr 172.21.15.3:0/626859294 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:20.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.954+0000 7fcea98d4700 1 --2- 172.21.15.3:0/626859294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcea409e1d0 0x7fcea4134cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:20.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.954+0000 7fceaa8d6700 1 --2- 172.21.15.3:0/626859294 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcea4135e50 0x7fcea4135230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:20.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.954+0000 7fcea98d4700 1 -- 172.21.15.3:0/626859294 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcea409d6f0 msgr2=0x7fcea41347b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:20.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.954+0000 7fcea98d4700 1 --2- 172.21.15.3:0/626859294 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcea409d6f0 0x7fcea41347b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:20.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.954+0000 7fcea98d4700 1 -- 172.21.15.3:0/626859294 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcea4135e50 msgr2=0x7fcea4135230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:20.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.954+0000 7fcea98d4700 1 --2- 172.21.15.3:0/626859294 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcea4135e50 0x7fcea4135230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:20.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.954+0000 7fcea98d4700 1 -- 172.21.15.3:0/626859294 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce94009550 con 0x7fcea409e1d0 2023-10-03T10:19:20.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.954+0000 7fceaa0d5700 1 --2- 172.21.15.3:0/626859294 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcea409d6f0 0x7fcea41347b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:20.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.954+0000 7fcea98d4700 1 --2- 172.21.15.3:0/626859294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcea409e1d0 0x7fcea4134cf0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fce9400e9d0 tx=0x7fce9400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:20.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.954+0000 7fce9affd700 1 -- 172.21.15.3:0/626859294 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9400ee70 con 0x7fcea409e1d0 2023-10-03T10:19:20.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.955+0000 7fce9affd700 1 -- 172.21.15.3:0/626859294 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce94004030 con 0x7fcea409e1d0 2023-10-03T10:19:20.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.955+0000 7fceac339700 1 -- 172.21.15.3:0/626859294 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce9c009a70 con 0x7fcea409e1d0 2023-10-03T10:19:20.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.955+0000 7fce9affd700 1 -- 172.21.15.3:0/626859294 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce940043c0 con 0x7fcea409e1d0 2023-10-03T10:19:20.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.955+0000 7fceac339700 1 -- 172.21.15.3:0/626859294 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea40a2290 con 0x7fcea409e1d0 2023-10-03T10:19:20.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.956+0000 7fce9affd700 1 -- 172.21.15.3:0/626859294 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fce9400c850 con 0x7fcea409e1d0 2023-10-03T10:19:20.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.956+0000 7fce9affd700 1 -- 172.21.15.3:0/626859294 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fce9403f930 con 0x7fcea409e1d0 2023-10-03T10:19:20.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.956+0000 7fceac339700 1 -- 172.21.15.3:0/626859294 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce8c0052e0 con 0x7fcea409e1d0 2023-10-03T10:19:20.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:20.965+0000 7fce9affd700 1 -- 172.21.15.3:0/626859294 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fce94011db0 con 0x7fcea409e1d0 2023-10-03T10:19:21.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.020+0000 7f94c699d700 1 Processor -- start 2023-10-03T10:19:21.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.020+0000 7f94c699d700 1 -- start start 2023-10-03T10:19:21.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.021+0000 7f94c699d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94c009d6f0 0x7f94c009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:21.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.021+0000 7f94c699d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94c009e000 0x7f94c01339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:21.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.021+0000 7f94c699d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f94c009e410 0x7f94c0133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:21.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.021+0000 7f94c699d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94c006b460 con 0x7f94c009e410 2023-10-03T10:19:21.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.021+0000 7f94c699d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c009b120 con 0x7f94c009d6f0 2023-10-03T10:19:21.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.021+0000 7f94c699d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94c009a7e0 con 0x7f94c009e000 2023-10-03T10:19:21.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.021+0000 7f94bffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94c009d6f0 0x7f94c009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:21.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.021+0000 7f94bffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94c009d6f0 0x7f94c009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:54446/0 (socket says 172.21.15.3:54446) 2023-10-03T10:19:21.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.021+0000 7f94bffff700 1 -- 172.21.15.3:0/2205974253 learned_addr learned my addr 172.21.15.3:0/2205974253 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:21.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.021+0000 7f94c4f3a700 1 --2- 172.21.15.3:0/2205974253 >> v2:172.21.15.3:3300/0 conn(0x7f94c009e410 0x7f94c0133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:21.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.021+0000 7f94bffff700 1 -- 172.21.15.3:0/2205974253 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94c009e000 msgr2=0x7f94c01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:21.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.021+0000 7f94bffff700 1 --2- 172.21.15.3:0/2205974253 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94c009e000 0x7f94c01339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.021+0000 7f94bffff700 1 -- 172.21.15.3:0/2205974253 >> v2:172.21.15.3:3300/0 conn(0x7f94c009e410 msgr2=0x7f94c0133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:21.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.021+0000 7f94bffff700 1 --2- 172.21.15.3:0/2205974253 >> v2:172.21.15.3:3300/0 conn(0x7f94c009e410 0x7f94c0133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.021+0000 7f94bffff700 1 -- 172.21.15.3:0/2205974253 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c009a500 con 0x7f94c009d6f0 2023-10-03T10:19:21.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.021+0000 7f94c4f3a700 1 --2- 172.21.15.3:0/2205974253 >> v2:172.21.15.3:3300/0 conn(0x7f94c009e410 0x7f94c0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:21.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.022+0000 7f94bffff700 1 --2- 172.21.15.3:0/2205974253 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94c009d6f0 0x7f94c009dac0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f94b0009920 tx=0x7f94b0009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4e5bd6d5a5a90e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:21.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.022+0000 7f94beffd700 1 -- 172.21.15.3:0/2205974253 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b0011830 con 0x7f94c009d6f0 2023-10-03T10:19:21.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.022+0000 7f94beffd700 1 -- 172.21.15.3:0/2205974253 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b0011990 con 0x7f94c009d6f0 2023-10-03T10:19:21.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.022+0000 7f94beffd700 1 -- 172.21.15.3:0/2205974253 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b0011d20 con 0x7f94c009d6f0 2023-10-03T10:19:21.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.022+0000 7f94c699d700 1 -- 172.21.15.3:0/2205974253 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94c009d6f0 msgr2=0x7f94c009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:21.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.022+0000 7f94c699d700 1 --2- 172.21.15.3:0/2205974253 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94c009d6f0 0x7f94c009dac0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f94b0009920 tx=0x7f94b0009c30 comp rx=0 tx=0).stop 2023-10-03T10:19:21.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.023+0000 7f94c699d700 1 -- 172.21.15.3:0/2205974253 shutdown_connections 2023-10-03T10:19:21.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.023+0000 7f94c699d700 1 --2- 172.21.15.3:0/2205974253 >> v2:172.21.15.3:3300/0 conn(0x7f94c009e410 0x7f94c0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.023+0000 7f94c699d700 1 --2- 172.21.15.3:0/2205974253 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94c009d6f0 0x7f94c009dac0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.023+0000 7f94c699d700 1 --2- 172.21.15.3:0/2205974253 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94c009e000 0x7f94c01339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.023+0000 7f94c699d700 1 -- 172.21.15.3:0/2205974253 >> 172.21.15.3:0/2205974253 conn(0x7f94c0066f50 msgr2=0x7f94c009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:21.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.023+0000 7f94c699d700 1 -- 172.21.15.3:0/2205974253 shutdown_connections 2023-10-03T10:19:21.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.023+0000 7f94c699d700 1 -- 172.21.15.3:0/2205974253 wait complete. 2023-10-03T10:19:21.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.024+0000 7f94c699d700 1 Processor -- start 2023-10-03T10:19:21.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.024+0000 7f94c699d700 1 -- start start 2023-10-03T10:19:21.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.025+0000 7f94c699d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94c009d6f0 0x7f94c01a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:21.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.025+0000 7f94c699d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94c009e000 0x7f94c01a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:21.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.025+0000 7f94c699d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94c009e410 0x7f94c01a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:21.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.025+0000 7f94c699d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c01a8a40 con 0x7f94c009e000 2023-10-03T10:19:21.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.025+0000 7f94c699d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c00a3290 con 0x7f94c009d6f0 2023-10-03T10:19:21.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.025+0000 7f94c699d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94c01a81d0 con 0x7f94c009e410 2023-10-03T10:19:21.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.025+0000 7f94bffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94c009d6f0 0x7f94c01a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:21.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.025+0000 7f94bf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94c009e000 0x7f94c01a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:21.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.025+0000 7f94bffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94c009d6f0 0x7f94c01a4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:54462/0 (socket says 172.21.15.3:54462) 2023-10-03T10:19:21.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.025+0000 7f94bffff700 1 -- 172.21.15.3:0/1842396898 learned_addr learned my addr 172.21.15.3:0/1842396898 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:21.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.025+0000 7f94c4f3a700 1 --2- 172.21.15.3:0/1842396898 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94c009e410 0x7f94c01a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:21.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.026+0000 7f94bf7fe700 1 -- 172.21.15.3:0/1842396898 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94c009e410 msgr2=0x7f94c01a5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:21.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.026+0000 7f94bf7fe700 1 --2- 172.21.15.3:0/1842396898 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94c009e410 0x7f94c01a5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.026+0000 7f94bf7fe700 1 -- 172.21.15.3:0/1842396898 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94c009d6f0 msgr2=0x7f94c01a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:21.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.026+0000 7f94bf7fe700 1 --2- 172.21.15.3:0/1842396898 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94c009d6f0 0x7f94c01a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.026+0000 7f94bf7fe700 1 -- 172.21.15.3:0/1842396898 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b8009a70 con 0x7f94c009e000 2023-10-03T10:19:21.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.026+0000 7f94bffff700 1 --2- 172.21.15.3:0/1842396898 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94c009d6f0 0x7f94c01a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:21.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.026+0000 7f94bf7fe700 1 --2- 172.21.15.3:0/1842396898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94c009e000 0x7f94c01a54f0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f94c006e510 tx=0x7f94b400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:21.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.026+0000 7f94bcff9700 1 -- 172.21.15.3:0/1842396898 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b4011830 con 0x7f94c009e000 2023-10-03T10:19:21.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.026+0000 7f94bcff9700 1 -- 172.21.15.3:0/1842396898 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b4011990 con 0x7f94c009e000 2023-10-03T10:19:21.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.026+0000 7f94c699d700 1 -- 172.21.15.3:0/1842396898 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b0009550 con 0x7f94c009e000 2023-10-03T10:19:21.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.026+0000 7f94bcff9700 1 -- 172.21.15.3:0/1842396898 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b4011d40 con 0x7f94c009e000 2023-10-03T10:19:21.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.026+0000 7f94c699d700 1 -- 172.21.15.3:0/1842396898 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c01a6b30 con 0x7f94c009e000 2023-10-03T10:19:21.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.028+0000 7f94c699d700 1 -- 172.21.15.3:0/1842396898 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c01a85c0 con 0x7f94c009e000 2023-10-03T10:19:21.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.029+0000 7f94bcff9700 1 -- 172.21.15.3:0/1842396898 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f94b401a400 con 0x7f94c009e000 2023-10-03T10:19:21.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.029+0000 7f94bcff9700 1 -- 172.21.15.3:0/1842396898 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f94b403f590 con 0x7f94c009e000 2023-10-03T10:19:21.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.034+0000 7f94bcff9700 1 -- 172.21.15.3:0/1842396898 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f94c01a85c0 con 0x7f94c009e000 2023-10-03T10:19:21.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.062+0000 7f9382ddf700 1 Processor -- start 2023-10-03T10:19:21.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.062+0000 7f9382ddf700 1 -- start start 2023-10-03T10:19:21.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.063+0000 7f9382ddf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f937c06f5d0 0x7f937c12ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:21.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.063+0000 7f9382ddf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f937c135c20 0x7f937c138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:21.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.063+0000 7f9382ddf700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f937c06fa40 con 0x7f937c12cf60 2023-10-03T10:19:21.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.063+0000 7f9382ddf700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f937c12cd90 con 0x7f937c06f5d0 2023-10-03T10:19:21.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.063+0000 7f9382ddf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f937c071780 con 0x7f937c135c20 2023-10-03T10:19:21.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.063+0000 7f937bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f937c06f5d0 0x7f937c12ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:21.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.063+0000 7f9380b7b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f937c12cf60 0x7f937c12f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51798/0 (socket says 172.21.15.3:51798) 2023-10-03T10:19:21.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.063+0000 7f9380b7b700 1 -- 172.21.15.3:0/773209341 learned_addr learned my addr 172.21.15.3:0/773209341 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:21.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.063+0000 7f938137c700 1 --2- 172.21.15.3:0/773209341 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f937c135c20 0x7f937c138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:21.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.063+0000 7f937bfff700 1 -- 172.21.15.3:0/773209341 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f937c135c20 msgr2=0x7f937c138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:21.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.063+0000 7f937bfff700 1 --2- 172.21.15.3:0/773209341 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f937c135c20 0x7f937c138000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.063+0000 7f937bfff700 1 -- 172.21.15.3:0/773209341 >> v1:172.21.15.3:6789/0 conn(0x7f937c12cf60 legacy=0x7f937c12f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:21.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.063+0000 7f937bfff700 1 -- 172.21.15.3:0/773209341 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f937c071a40 con 0x7f937c06f5d0 2023-10-03T10:19:21.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.064+0000 7f937bfff700 1 --2- 172.21.15.3:0/773209341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f937c06f5d0 0x7f937c12ffc0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f936800a9e0 tx=0x7f936800acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c558777b50d5878 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:21.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.064+0000 7f937b7fe700 1 -- 172.21.15.3:0/773209341 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936800cb70 con 0x7f937c06f5d0 2023-10-03T10:19:21.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.064+0000 7f937b7fe700 1 -- 172.21.15.3:0/773209341 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936800ccd0 con 0x7f937c06f5d0 2023-10-03T10:19:21.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.064+0000 7f937b7fe700 1 -- 172.21.15.3:0/773209341 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9368018620 con 0x7f937c06f5d0 2023-10-03T10:19:21.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.065+0000 7f9382ddf700 1 -- 172.21.15.3:0/773209341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f937c06f5d0 msgr2=0x7f937c12ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:21.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.065+0000 7f9382ddf700 1 --2- 172.21.15.3:0/773209341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f937c06f5d0 0x7f937c12ffc0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f936800a9e0 tx=0x7f936800acf0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.065+0000 7f9382ddf700 1 -- 172.21.15.3:0/773209341 shutdown_connections 2023-10-03T10:19:21.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.065+0000 7f9382ddf700 1 --2- 172.21.15.3:0/773209341 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f937c135c20 0x7f937c138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.065+0000 7f9382ddf700 1 --2- 172.21.15.3:0/773209341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f937c06f5d0 0x7f937c12ffc0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.065+0000 7f9382ddf700 1 -- 172.21.15.3:0/773209341 >> 172.21.15.3:0/773209341 conn(0x7f937c0a2aa0 msgr2=0x7f937c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:21.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.065+0000 7f9382ddf700 1 -- 172.21.15.3:0/773209341 shutdown_connections 2023-10-03T10:19:21.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.065+0000 7f9382ddf700 1 -- 172.21.15.3:0/773209341 wait complete. 2023-10-03T10:19:21.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.066+0000 7f9382ddf700 1 Processor -- start 2023-10-03T10:19:21.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.067+0000 7f9382ddf700 1 -- start start 2023-10-03T10:19:21.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.068+0000 7f9382ddf700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f937c06f5d0 0x7f937c126160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:21.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.068+0000 7f9382ddf700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f937c12cf60 0x7f937c1266a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:21.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.068+0000 7f9382ddf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f937c135c20 0x7f937c1c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:21.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.068+0000 7f9382ddf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937c1c4da0 con 0x7f937c135c20 2023-10-03T10:19:21.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.068+0000 7f9382ddf700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f937c09eac0 con 0x7f937c06f5d0 2023-10-03T10:19:21.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.068+0000 7f9382ddf700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f937c129fa0 con 0x7f937c12cf60 2023-10-03T10:19:21.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.068+0000 7f937bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f937c12cf60 0x7f937c1266a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:21.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.068+0000 7f938137c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f937c135c20 0x7f937c1c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:21.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.068+0000 7f937bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f937c12cf60 0x7f937c1266a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41822/0 (socket says 172.21.15.3:41822) 2023-10-03T10:19:21.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.068+0000 7f937bfff700 1 -- 172.21.15.3:0/1753384427 learned_addr learned my addr 172.21.15.3:0/1753384427 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:21.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.068+0000 7f937bfff700 1 -- 172.21.15.3:0/1753384427 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f937c06f5d0 msgr2=0x7f937c126160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:21.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.068+0000 7f937bfff700 1 --2- 172.21.15.3:0/1753384427 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f937c06f5d0 0x7f937c126160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.068+0000 7f937bfff700 1 -- 172.21.15.3:0/1753384427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f937c135c20 msgr2=0x7f937c1c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:21.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.068+0000 7f937bfff700 1 --2- 172.21.15.3:0/1753384427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f937c135c20 0x7f937c1c8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.068+0000 7f937bfff700 1 -- 172.21.15.3:0/1753384427 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936800a640 con 0x7f937c12cf60 2023-10-03T10:19:21.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.068+0000 7f938137c700 1 --2- 172.21.15.3:0/1753384427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f937c135c20 0x7f937c1c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:21.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.068+0000 7f937bfff700 1 --2- 172.21.15.3:0/1753384427 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f937c12cf60 0x7f937c1266a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f936800ad30 tx=0x7f93680047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:21.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.069+0000 7f93797fa700 1 -- 172.21.15.3:0/1753384427 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9368003ab0 con 0x7f937c12cf60 2023-10-03T10:19:21.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.069+0000 7f93797fa700 1 -- 172.21.15.3:0/1753384427 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9368003c30 con 0x7f937c12cf60 2023-10-03T10:19:21.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.069+0000 7f93797fa700 1 -- 172.21.15.3:0/1753384427 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9368021480 con 0x7f937c12cf60 2023-10-03T10:19:21.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.069+0000 7f9382ddf700 1 -- 172.21.15.3:0/1753384427 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f937c09dfe0 con 0x7f937c12cf60 2023-10-03T10:19:21.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.069+0000 7f9382ddf700 1 -- 172.21.15.3:0/1753384427 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f937c09cd90 con 0x7f937c12cf60 2023-10-03T10:19:21.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.070+0000 7f93797fa700 1 -- 172.21.15.3:0/1753384427 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f93680215e0 con 0x7f937c12cf60 2023-10-03T10:19:21.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.070+0000 7f93797fa700 1 -- 172.21.15.3:0/1753384427 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9368040d70 con 0x7f937c12cf60 2023-10-03T10:19:21.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.070+0000 7f9382ddf700 1 -- 172.21.15.3:0/1753384427 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f937c074ea0 con 0x7f937c12cf60 2023-10-03T10:19:21.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.080+0000 7f93797fa700 1 -- 172.21.15.3:0/1753384427 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9368021890 con 0x7f937c12cf60 2023-10-03T10:19:21.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.102+0000 7f7d8ab38700 1 Processor -- start 2023-10-03T10:19:21.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.103+0000 7f7d8ab38700 1 -- start start 2023-10-03T10:19:21.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.103+0000 7f7d8ab38700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d8409d6f0 0x7f7d8409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:21.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.103+0000 7f7d8ab38700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7d8409e410 0x7f7d84134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:21.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.103+0000 7f7d8ab38700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d8409b120 con 0x7f7d8409e000 2023-10-03T10:19:21.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.103+0000 7f7d8ab38700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d8406b460 con 0x7f7d8409e410 2023-10-03T10:19:21.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.103+0000 7f7d8ab38700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8409a7e0 con 0x7f7d8409d6f0 2023-10-03T10:19:21.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.104+0000 7f7d888d4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d8409d6f0 0x7f7d8409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:21.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.104+0000 7f7d888d4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d8409d6f0 0x7f7d8409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41836/0 (socket says 172.21.15.3:41836) 2023-10-03T10:19:21.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.104+0000 7f7d888d4700 1 -- 172.21.15.3:0/352995028 learned_addr learned my addr 172.21.15.3:0/352995028 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:21.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.104+0000 7f7d890d5700 1 --2- 172.21.15.3:0/352995028 >> v2:172.21.15.3:3300/0 conn(0x7f7d8409e410 0x7f7d84134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:21.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.104+0000 7f7d888d4700 1 -- 172.21.15.3:0/352995028 >> v2:172.21.15.3:3300/0 conn(0x7f7d8409e410 msgr2=0x7f7d84134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:21.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.104+0000 7f7d888d4700 1 --2- 172.21.15.3:0/352995028 >> v2:172.21.15.3:3300/0 conn(0x7f7d8409e410 0x7f7d84134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.104+0000 7f7d888d4700 1 -- 172.21.15.3:0/352995028 >> v1:172.21.15.3:6789/0 conn(0x7f7d8409e000 legacy=0x7f7d841339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:21.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.104+0000 7f7d888d4700 1 -- 172.21.15.3:0/352995028 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d8409a500 con 0x7f7d8409d6f0 2023-10-03T10:19:21.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.104+0000 7f7d888d4700 1 --2- 172.21.15.3:0/352995028 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d8409d6f0 0x7f7d8409dac0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009920 tx=0x7f7d78009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c8b17f6ad7a1a65 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:21.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.104+0000 7f7d837fe700 1 -- 172.21.15.3:0/352995028 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7800ba20 con 0x7f7d8409d6f0 2023-10-03T10:19:21.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.104+0000 7f7d837fe700 1 -- 172.21.15.3:0/352995028 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d7800bb80 con 0x7f7d8409d6f0 2023-10-03T10:19:21.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.104+0000 7f7d837fe700 1 -- 172.21.15.3:0/352995028 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d78009e70 con 0x7f7d8409d6f0 2023-10-03T10:19:21.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.105+0000 7f7d8ab38700 1 -- 172.21.15.3:0/352995028 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d8409d6f0 msgr2=0x7f7d8409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:21.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.105+0000 7f7d8ab38700 1 --2- 172.21.15.3:0/352995028 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d8409d6f0 0x7f7d8409dac0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009920 tx=0x7f7d78009c30 comp rx=0 tx=0).stop 2023-10-03T10:19:21.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.105+0000 7f7d8ab38700 1 -- 172.21.15.3:0/352995028 shutdown_connections 2023-10-03T10:19:21.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.105+0000 7f7d8ab38700 1 --2- 172.21.15.3:0/352995028 >> v2:172.21.15.3:3300/0 conn(0x7f7d8409e410 0x7f7d84134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.105+0000 7f7d8ab38700 1 --2- 172.21.15.3:0/352995028 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d8409d6f0 0x7f7d8409dac0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.105+0000 7f7d8ab38700 1 -- 172.21.15.3:0/352995028 >> 172.21.15.3:0/352995028 conn(0x7f7d84066f50 msgr2=0x7f7d8409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:21.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.105+0000 7f7d8ab38700 1 -- 172.21.15.3:0/352995028 shutdown_connections 2023-10-03T10:19:21.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.105+0000 7f7d8ab38700 1 -- 172.21.15.3:0/352995028 wait complete. 2023-10-03T10:19:21.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.107+0000 7f7d8ab38700 1 Processor -- start 2023-10-03T10:19:21.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.107+0000 7f7d8ab38700 1 -- start start 2023-10-03T10:19:21.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.108+0000 7f7d8ab38700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d8409d6f0 0x7f7d841a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:21.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.108+0000 7f7d8ab38700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d8409e000 0x7f7d841a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:21.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.108+0000 7f7d8ab38700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d8409e410 0x7f7d841a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:21.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.108+0000 7f7d8ab38700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d841a9720 con 0x7f7d8409d6f0 2023-10-03T10:19:21.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.108+0000 7f7d8ab38700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d841a9920 con 0x7f7d8409e000 2023-10-03T10:19:21.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.108+0000 7f7d8ab38700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d841a8810 con 0x7f7d8409e410 2023-10-03T10:19:21.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.108+0000 7f7d888d4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d8409d6f0 0x7f7d841a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:21.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.108+0000 7f7d888d4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d8409d6f0 0x7f7d841a5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54780/0 (socket says 172.21.15.3:54780) 2023-10-03T10:19:21.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.108+0000 7f7d888d4700 1 -- 172.21.15.3:0/2238355693 learned_addr learned my addr 172.21.15.3:0/2238355693 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:21.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.108+0000 7f7d890d5700 1 --2- 172.21.15.3:0/2238355693 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d8409e410 0x7f7d841a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:21.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.108+0000 7f7d888d4700 1 -- 172.21.15.3:0/2238355693 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d8409e410 msgr2=0x7f7d841a6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:21.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.108+0000 7f7d888d4700 1 --2- 172.21.15.3:0/2238355693 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d8409e410 0x7f7d841a6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.108+0000 7f7d888d4700 1 -- 172.21.15.3:0/2238355693 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d8409e000 msgr2=0x7f7d841a63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:21.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.108+0000 7f7d888d4700 1 --2- 172.21.15.3:0/2238355693 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d8409e000 0x7f7d841a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.109+0000 7f7d888d4700 1 -- 172.21.15.3:0/2238355693 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d78009550 con 0x7f7d8409d6f0 2023-10-03T10:19:21.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.109+0000 7f7d890d5700 1 --2- 172.21.15.3:0/2238355693 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d8409e410 0x7f7d841a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:21.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.109+0000 7f7d888d4700 1 --2- 172.21.15.3:0/2238355693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d8409d6f0 0x7f7d841a5e90 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009870 tx=0x7f7d78004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:21.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.109+0000 7f7d817fa700 1 -- 172.21.15.3:0/2238355693 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d78004300 con 0x7f7d8409d6f0 2023-10-03T10:19:21.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.109+0000 7f7d8ab38700 1 -- 172.21.15.3:0/2238355693 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d841a7ab0 con 0x7f7d8409d6f0 2023-10-03T10:19:21.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.109+0000 7f7d8ab38700 1 -- 172.21.15.3:0/2238355693 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d8409c710 con 0x7f7d8409d6f0 2023-10-03T10:19:21.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.109+0000 7f7d817fa700 1 -- 172.21.15.3:0/2238355693 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d7801f070 con 0x7f7d8409d6f0 2023-10-03T10:19:21.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.109+0000 7f7d817fa700 1 -- 172.21.15.3:0/2238355693 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d780036a0 con 0x7f7d8409d6f0 2023-10-03T10:19:21.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.110+0000 7f7d8ab38700 1 -- 172.21.15.3:0/2238355693 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d640052e0 con 0x7f7d8409d6f0 2023-10-03T10:19:21.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.111+0000 7f7d817fa700 1 -- 172.21.15.3:0/2238355693 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7d78021030 con 0x7f7d8409d6f0 2023-10-03T10:19:21.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.114+0000 7f7d817fa700 1 -- 172.21.15.3:0/2238355693 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7d7801b5f0 con 0x7f7d8409d6f0 2023-10-03T10:19:21.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.120+0000 7f7d817fa700 1 -- 172.21.15.3:0/2238355693 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7d780199a0 con 0x7f7d8409d6f0 2023-10-03T10:19:21.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.195+0000 7fceac339700 1 -- 172.21.15.3:0/626859294 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fce8c005590 con 0x7fcea409e1d0 2023-10-03T10:19:21.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.196+0000 7fce9affd700 1 -- 172.21.15.3:0/626859294 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fce94011350 con 0x7fcea409e1d0 2023-10-03T10:19:21.196 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:21.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.199+0000 7fceac339700 1 -- 172.21.15.3:0/626859294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcea409e1d0 msgr2=0x7fcea4134cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:21.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.199+0000 7fceac339700 1 --2- 172.21.15.3:0/626859294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcea409e1d0 0x7fcea4134cf0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fce9400e9d0 tx=0x7fce9400ece0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.199+0000 7fceac339700 1 -- 172.21.15.3:0/626859294 shutdown_connections 2023-10-03T10:19:21.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.199+0000 7fceac339700 1 --2- 172.21.15.3:0/626859294 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcea4135e50 0x7fcea4135230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.199+0000 7fceac339700 1 --2- 172.21.15.3:0/626859294 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcea409d6f0 0x7fcea41347b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.199+0000 7fceac339700 1 --2- 172.21.15.3:0/626859294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcea409e1d0 0x7fcea4134cf0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.199+0000 7fceac339700 1 -- 172.21.15.3:0/626859294 >> 172.21.15.3:0/626859294 conn(0x7fcea4066f50 msgr2=0x7fcea40a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:21.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.199+0000 7fceac339700 1 -- 172.21.15.3:0/626859294 shutdown_connections 2023-10-03T10:19:21.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.199+0000 7fceac339700 1 -- 172.21.15.3:0/626859294 wait complete. 2023-10-03T10:19:21.219 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:21.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.255+0000 7f94c699d700 1 -- 172.21.15.3:0/1842396898 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f94c01a85c0 con 0x7f94c009e000 2023-10-03T10:19:21.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.255+0000 7f94bcff9700 1 -- 172.21.15.3:0/1842396898 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f94c01a85c0 con 0x7f94c009e000 2023-10-03T10:19:21.256 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:21.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.259+0000 7f94c699d700 1 -- 172.21.15.3:0/1842396898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94c009e000 msgr2=0x7f94c01a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:21.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.259+0000 7f94c699d700 1 --2- 172.21.15.3:0/1842396898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94c009e000 0x7f94c01a54f0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f94c006e510 tx=0x7f94b400b730 comp rx=0 tx=0).stop 2023-10-03T10:19:21.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.259+0000 7f94c699d700 1 -- 172.21.15.3:0/1842396898 shutdown_connections 2023-10-03T10:19:21.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.259+0000 7f94c699d700 1 --2- 172.21.15.3:0/1842396898 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94c009e410 0x7f94c01a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.259+0000 7f94c699d700 1 --2- 172.21.15.3:0/1842396898 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94c009d6f0 0x7f94c01a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.259+0000 7f94c699d700 1 --2- 172.21.15.3:0/1842396898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94c009e000 0x7f94c01a54f0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.259+0000 7f94c699d700 1 -- 172.21.15.3:0/1842396898 >> 172.21.15.3:0/1842396898 conn(0x7f94c0066f50 msgr2=0x7f94c00a1580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:21.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.259+0000 7f94c699d700 1 -- 172.21.15.3:0/1842396898 shutdown_connections 2023-10-03T10:19:21.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.259+0000 7f94c699d700 1 -- 172.21.15.3:0/1842396898 wait complete. 2023-10-03T10:19:21.278 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:21.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.320+0000 7f9382ddf700 1 -- 172.21.15.3:0/1753384427 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f937c0713a0 con 0x7f937c12cf60 2023-10-03T10:19:21.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.321+0000 7f93797fa700 1 -- 172.21.15.3:0/1753384427 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9368026020 con 0x7f937c12cf60 2023-10-03T10:19:21.321 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:21.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.323+0000 7f9382ddf700 1 -- 172.21.15.3:0/1753384427 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f937c12cf60 msgr2=0x7f937c1266a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:21.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.323+0000 7f9382ddf700 1 --2- 172.21.15.3:0/1753384427 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f937c12cf60 0x7f937c1266a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f936800ad30 tx=0x7f93680047c0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.323+0000 7f9382ddf700 1 -- 172.21.15.3:0/1753384427 shutdown_connections 2023-10-03T10:19:21.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.323+0000 7f9382ddf700 1 --2- 172.21.15.3:0/1753384427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f937c135c20 0x7f937c1c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.323+0000 7f9382ddf700 1 --2- 172.21.15.3:0/1753384427 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f937c06f5d0 0x7f937c126160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.323+0000 7f9382ddf700 1 --2- 172.21.15.3:0/1753384427 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f937c12cf60 0x7f937c1266a0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.323+0000 7f9382ddf700 1 -- 172.21.15.3:0/1753384427 >> 172.21.15.3:0/1753384427 conn(0x7f937c0a2aa0 msgr2=0x7f937c06e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:21.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.323+0000 7f9382ddf700 1 -- 172.21.15.3:0/1753384427 shutdown_connections 2023-10-03T10:19:21.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.324+0000 7f9382ddf700 1 -- 172.21.15.3:0/1753384427 wait complete. 2023-10-03T10:19:21.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.332+0000 7f7d8ab38700 1 -- 172.21.15.3:0/2238355693 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7d64005590 con 0x7f7d8409d6f0 2023-10-03T10:19:21.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.332+0000 7f7d817fa700 1 -- 172.21.15.3:0/2238355693 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7d78015030 con 0x7f7d8409d6f0 2023-10-03T10:19:21.333 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:21.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.335+0000 7f7d8ab38700 1 -- 172.21.15.3:0/2238355693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d8409d6f0 msgr2=0x7f7d841a5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:21.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.335+0000 7f7d8ab38700 1 --2- 172.21.15.3:0/2238355693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d8409d6f0 0x7f7d841a5e90 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009870 tx=0x7f7d78004750 comp rx=0 tx=0).stop 2023-10-03T10:19:21.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.335+0000 7f7d8ab38700 1 -- 172.21.15.3:0/2238355693 shutdown_connections 2023-10-03T10:19:21.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.335+0000 7f7d8ab38700 1 --2- 172.21.15.3:0/2238355693 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7d8409e410 0x7f7d841a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.335+0000 7f7d8ab38700 1 --2- 172.21.15.3:0/2238355693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d8409d6f0 0x7f7d841a5e90 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.335+0000 7f7d8ab38700 1 --2- 172.21.15.3:0/2238355693 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d8409e000 0x7f7d841a63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:21.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.335+0000 7f7d8ab38700 1 -- 172.21.15.3:0/2238355693 >> 172.21.15.3:0/2238355693 conn(0x7f7d84066f50 msgr2=0x7f7d841bae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:21.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.335+0000 7f7d8ab38700 1 -- 172.21.15.3:0/2238355693 shutdown_connections 2023-10-03T10:19:21.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:21.335+0000 7f7d8ab38700 1 -- 172.21.15.3:0/2238355693 wait complete. 2023-10-03T10:19:21.338 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:21.349 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:22.220 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:22.278 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:22.338 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:22.349 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:22.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.372+0000 7fa3e2d63700 1 Processor -- start 2023-10-03T10:19:22.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.372+0000 7fa3e2d63700 1 -- start start 2023-10-03T10:19:22.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.372+0000 7fa3e2d63700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3dc09f410 0x7fa3dc136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:22.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.372+0000 7fa3e2d63700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3dc136fe0 0x7fa3dc1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:22.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.372+0000 7fa3e2d63700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3dc070940 con 0x7fa3dc09e930 2023-10-03T10:19:22.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.372+0000 7fa3e2d63700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3dc06fd60 con 0x7fa3dc09f410 2023-10-03T10:19:22.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.372+0000 7fa3e2d63700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc06fa80 con 0x7fa3dc136fe0 2023-10-03T10:19:22.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.372+0000 7fa3dbfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3dc09f410 0x7fa3dc136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:22.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.373+0000 7fa3dbfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3dc09f410 0x7fa3dc136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54794/0 (socket says 172.21.15.3:54794) 2023-10-03T10:19:22.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.373+0000 7fa3dbfff700 1 -- 172.21.15.3:0/2508918299 learned_addr learned my addr 172.21.15.3:0/2508918299 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:22.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.373+0000 7fa3e1300700 1 --2- 172.21.15.3:0/2508918299 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3dc136fe0 0x7fa3dc1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:22.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.373+0000 7fa3dbfff700 1 -- 172.21.15.3:0/2508918299 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3dc136fe0 msgr2=0x7fa3dc1393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:22.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.373+0000 7fa3dbfff700 1 --2- 172.21.15.3:0/2508918299 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3dc136fe0 0x7fa3dc1393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.373+0000 7fa3dbfff700 1 -- 172.21.15.3:0/2508918299 >> v1:172.21.15.3:6789/0 conn(0x7fa3dc09e930 legacy=0x7fa3dc09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:22.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.373+0000 7fa3dbfff700 1 -- 172.21.15.3:0/2508918299 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc09fb60 con 0x7fa3dc09f410 2023-10-03T10:19:22.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.373+0000 7fa3e1300700 1 --2- 172.21.15.3:0/2508918299 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3dc136fe0 0x7fa3dc1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:22.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.373+0000 7fa3dbfff700 1 --2- 172.21.15.3:0/2508918299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3dc09f410 0x7fa3dc136aa0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fa3c80098f0 tx=0x7fa3c8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b788fa0a27e747 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:22.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.373+0000 7fa3db7fe700 1 -- 172.21.15.3:0/2508918299 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8011830 con 0x7fa3dc09f410 2023-10-03T10:19:22.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.373+0000 7fa3db7fe700 1 -- 172.21.15.3:0/2508918299 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c8011990 con 0x7fa3dc09f410 2023-10-03T10:19:22.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.373+0000 7fa3db7fe700 1 -- 172.21.15.3:0/2508918299 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8011d20 con 0x7fa3dc09f410 2023-10-03T10:19:22.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.374+0000 7fa3e2d63700 1 -- 172.21.15.3:0/2508918299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3dc09f410 msgr2=0x7fa3dc136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:22.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.374+0000 7fa3e2d63700 1 --2- 172.21.15.3:0/2508918299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3dc09f410 0x7fa3dc136aa0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fa3c80098f0 tx=0x7fa3c8009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:22.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.374+0000 7fa3e2d63700 1 -- 172.21.15.3:0/2508918299 shutdown_connections 2023-10-03T10:19:22.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.374+0000 7fa3e2d63700 1 --2- 172.21.15.3:0/2508918299 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3dc136fe0 0x7fa3dc1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.374+0000 7fa3e2d63700 1 --2- 172.21.15.3:0/2508918299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3dc09f410 0x7fa3dc136aa0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.374+0000 7fa3e2d63700 1 -- 172.21.15.3:0/2508918299 >> 172.21.15.3:0/2508918299 conn(0x7fa3dc0a4210 msgr2=0x7fa3dc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:22.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.375+0000 7fa3e2d63700 1 -- 172.21.15.3:0/2508918299 shutdown_connections 2023-10-03T10:19:22.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.375+0000 7fa3e2d63700 1 -- 172.21.15.3:0/2508918299 wait complete. 2023-10-03T10:19:22.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.376+0000 7fa3e2d63700 1 Processor -- start 2023-10-03T10:19:22.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.376+0000 7fa3e2d63700 1 -- start start 2023-10-03T10:19:22.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.376+0000 7fa3e2d63700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3dc09e930 0x7fa3dc06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:22.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.376+0000 7fa3e2d63700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3dc09f410 0x7fa3dc06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:22.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.376+0000 7fa3e2d63700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3dc136fe0 0x7fa3dc06c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:22.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.376+0000 7fa3e2d63700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc1226a0 con 0x7fa3dc136fe0 2023-10-03T10:19:22.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.376+0000 7fa3e2d63700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc1325f0 con 0x7fa3dc09e930 2023-10-03T10:19:22.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.376+0000 7fa3e2d63700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc0a2f40 con 0x7fa3dc09f410 2023-10-03T10:19:22.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.376+0000 7fa3e0aff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3dc09e930 0x7fa3dc06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:22.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.377+0000 7fa3e0aff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3dc09e930 0x7fa3dc06b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:54480/0 (socket says 172.21.15.3:54480) 2023-10-03T10:19:22.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.377+0000 7fa3e0aff700 1 -- 172.21.15.3:0/307686690 learned_addr learned my addr 172.21.15.3:0/307686690 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:22.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.377+0000 7fa3e1300700 1 --2- 172.21.15.3:0/307686690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3dc136fe0 0x7fa3dc06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:22.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.377+0000 7fa3dbfff700 1 --2- 172.21.15.3:0/307686690 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3dc09f410 0x7fa3dc06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:22.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.377+0000 7fa3e1300700 1 -- 172.21.15.3:0/307686690 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3dc09f410 msgr2=0x7fa3dc06bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:22.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.377+0000 7fa3e1300700 1 --2- 172.21.15.3:0/307686690 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3dc09f410 0x7fa3dc06bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.377+0000 7fa3e1300700 1 -- 172.21.15.3:0/307686690 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3dc09e930 msgr2=0x7fa3dc06b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:22.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.377+0000 7fa3e1300700 1 --2- 172.21.15.3:0/307686690 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3dc09e930 0x7fa3dc06b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.377+0000 7fa3e1300700 1 -- 172.21.15.3:0/307686690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d4009a70 con 0x7fa3dc136fe0 2023-10-03T10:19:22.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.377+0000 7fa3e1300700 1 --2- 172.21.15.3:0/307686690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3dc136fe0 0x7fa3dc06c330 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400efc0 tx=0x7fa3d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:22.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.377+0000 7fa3d97fa700 1 -- 172.21.15.3:0/307686690 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d400ca90 con 0x7fa3dc136fe0 2023-10-03T10:19:22.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.378+0000 7fa3d97fa700 1 -- 172.21.15.3:0/307686690 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d400f040 con 0x7fa3dc136fe0 2023-10-03T10:19:22.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.378+0000 7fa3d97fa700 1 -- 172.21.15.3:0/307686690 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d400ce10 con 0x7fa3dc136fe0 2023-10-03T10:19:22.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.378+0000 7fa3e2d63700 1 -- 172.21.15.3:0/307686690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c8009550 con 0x7fa3dc136fe0 2023-10-03T10:19:22.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.378+0000 7fa3e2d63700 1 -- 172.21.15.3:0/307686690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3dc06cca0 con 0x7fa3dc136fe0 2023-10-03T10:19:22.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.379+0000 7fa3d97fa700 1 -- 172.21.15.3:0/307686690 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa3d4003a20 con 0x7fa3dc136fe0 2023-10-03T10:19:22.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.379+0000 7fa3d97fa700 1 -- 172.21.15.3:0/307686690 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa3d403ff20 con 0x7fa3dc136fe0 2023-10-03T10:19:22.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.380+0000 7fa3e2d63700 1 -- 172.21.15.3:0/307686690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3dc09e190 con 0x7fa3dc136fe0 2023-10-03T10:19:22.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.389+0000 7fa3d97fa700 1 -- 172.21.15.3:0/307686690 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa3d401b770 con 0x7fa3dc136fe0 2023-10-03T10:19:22.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.429+0000 7fa6872f5700 1 Processor -- start 2023-10-03T10:19:22.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.429+0000 7fa6872f5700 1 -- start start 2023-10-03T10:19:22.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.430+0000 7fa6872f5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa68009e1d0 0x7fa680138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:22.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.430+0000 7fa6872f5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa680139170 0x7fa68013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:22.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.430+0000 7fa6872f5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa68009b120 con 0x7fa68009d6f0 2023-10-03T10:19:22.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.430+0000 7fa6872f5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa68006b460 con 0x7fa68009e1d0 2023-10-03T10:19:22.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.430+0000 7fa6872f5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa68009a7e0 con 0x7fa680139170 2023-10-03T10:19:22.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.430+0000 7fa684890700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa68009e1d0 0x7fa680138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:22.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.430+0000 7fa685091700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa68009d6f0 0x7fa68009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51820/0 (socket says 172.21.15.3:51820) 2023-10-03T10:19:22.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.430+0000 7fa685091700 1 -- 172.21.15.3:0/771021382 learned_addr learned my addr 172.21.15.3:0/771021382 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:22.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.430+0000 7fa685892700 1 --2- 172.21.15.3:0/771021382 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa680139170 0x7fa68013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:22.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.430+0000 7fa684890700 1 -- 172.21.15.3:0/771021382 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa680139170 msgr2=0x7fa68013b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:22.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.430+0000 7fa684890700 1 --2- 172.21.15.3:0/771021382 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa680139170 0x7fa68013b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.430+0000 7fa684890700 1 -- 172.21.15.3:0/771021382 >> v1:172.21.15.3:6789/0 conn(0x7fa68009d6f0 legacy=0x7fa68009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:22.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.430+0000 7fa677fff700 1 -- 172.21.15.3:0/771021382 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 739203921 0 0) 0x7fa68009b120 con 0x7fa68009d6f0 2023-10-03T10:19:22.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.430+0000 7fa684890700 1 -- 172.21.15.3:0/771021382 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa68009a500 con 0x7fa68009e1d0 2023-10-03T10:19:22.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.431+0000 7fa684890700 1 --2- 172.21.15.3:0/771021382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa68009e1d0 0x7fa680138c30 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fa670009870 tx=0x7fa670009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f06266f7daf842 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:22.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.431+0000 7fa677fff700 1 -- 172.21.15.3:0/771021382 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670011830 con 0x7fa68009e1d0 2023-10-03T10:19:22.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.431+0000 7fa677fff700 1 -- 172.21.15.3:0/771021382 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa670011990 con 0x7fa68009e1d0 2023-10-03T10:19:22.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.431+0000 7fa677fff700 1 -- 172.21.15.3:0/771021382 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670011d20 con 0x7fa68009e1d0 2023-10-03T10:19:22.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.431+0000 7fa6872f5700 1 -- 172.21.15.3:0/771021382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa68009e1d0 msgr2=0x7fa680138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:22.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.431+0000 7fa6872f5700 1 --2- 172.21.15.3:0/771021382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa68009e1d0 0x7fa680138c30 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fa670009870 tx=0x7fa670009b80 comp rx=0 tx=0).stop 2023-10-03T10:19:22.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.432+0000 7fa6872f5700 1 -- 172.21.15.3:0/771021382 shutdown_connections 2023-10-03T10:19:22.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.432+0000 7fa6872f5700 1 --2- 172.21.15.3:0/771021382 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa680139170 0x7fa68013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.432+0000 7fa6872f5700 1 --2- 172.21.15.3:0/771021382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa68009e1d0 0x7fa680138c30 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.432+0000 7fa6872f5700 1 -- 172.21.15.3:0/771021382 >> 172.21.15.3:0/771021382 conn(0x7fa680066f50 msgr2=0x7fa68009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:22.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.432+0000 7fa6872f5700 1 -- 172.21.15.3:0/771021382 shutdown_connections 2023-10-03T10:19:22.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.432+0000 7fa6872f5700 1 -- 172.21.15.3:0/771021382 wait complete. 2023-10-03T10:19:22.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.433+0000 7fa6872f5700 1 Processor -- start 2023-10-03T10:19:22.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.433+0000 7fa6872f5700 1 -- start start 2023-10-03T10:19:22.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.433+0000 7fa6872f5700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa68009d6f0 0x7fa680121960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:22.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.433+0000 7fa6872f5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa68009e1d0 0x7fa680121ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:22.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.433+0000 7fa6872f5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa680139170 0x7fa6801223e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:22.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.433+0000 7fa6872f5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6800a3a20 con 0x7fa680139170 2023-10-03T10:19:22.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.433+0000 7fa6872f5700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6801251a0 con 0x7fa68009d6f0 2023-10-03T10:19:22.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.433+0000 7fa6872f5700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa680137850 con 0x7fa68009e1d0 2023-10-03T10:19:22.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.434+0000 7fa684890700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa68009e1d0 0x7fa680121ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:22.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.434+0000 7fa684890700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa68009e1d0 0x7fa680121ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41884/0 (socket says 172.21.15.3:41884) 2023-10-03T10:19:22.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.434+0000 7fa684890700 1 -- 172.21.15.3:0/1306353693 learned_addr learned my addr 172.21.15.3:0/1306353693 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:22.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.434+0000 7fa685892700 1 --2- 172.21.15.3:0/1306353693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa680139170 0x7fa6801223e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:22.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.434+0000 7fa685091700 1 --2- 172.21.15.3:0/1306353693 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa68009d6f0 0x7fa680121960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:22.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.434+0000 7fa684890700 1 -- 172.21.15.3:0/1306353693 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa68009d6f0 msgr2=0x7fa680121960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:22.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.434+0000 7fa684890700 1 --2- 172.21.15.3:0/1306353693 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa68009d6f0 0x7fa680121960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.434+0000 7fa684890700 1 -- 172.21.15.3:0/1306353693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa680139170 msgr2=0x7fa6801223e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:22.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.434+0000 7fa684890700 1 --2- 172.21.15.3:0/1306353693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa680139170 0x7fa6801223e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.434+0000 7fa684890700 1 -- 172.21.15.3:0/1306353693 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6700094a0 con 0x7fa68009e1d0 2023-10-03T10:19:22.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.434+0000 7fa685091700 1 --2- 172.21.15.3:0/1306353693 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa68009d6f0 0x7fa680121960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:19:22.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.434+0000 7fa685892700 1 --2- 172.21.15.3:0/1306353693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa680139170 0x7fa6801223e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:22.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.434+0000 7fa684890700 1 --2- 172.21.15.3:0/1306353693 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa68009e1d0 0x7fa680121ea0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fa670000c00 tx=0x7fa670004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:22.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.434+0000 7fa675ffb700 1 -- 172.21.15.3:0/1306353693 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670004240 con 0x7fa68009e1d0 2023-10-03T10:19:22.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.435+0000 7fa675ffb700 1 -- 172.21.15.3:0/1306353693 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6700043a0 con 0x7fa68009e1d0 2023-10-03T10:19:22.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.435+0000 7fa675ffb700 1 -- 172.21.15.3:0/1306353693 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670022e10 con 0x7fa68009e1d0 2023-10-03T10:19:22.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.435+0000 7fa6872f5700 1 -- 172.21.15.3:0/1306353693 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa680123680 con 0x7fa68009e1d0 2023-10-03T10:19:22.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.435+0000 7fa6872f5700 1 -- 172.21.15.3:0/1306353693 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa680123ad0 con 0x7fa68009e1d0 2023-10-03T10:19:22.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.436+0000 7fa6872f5700 1 -- 172.21.15.3:0/1306353693 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa680074ea0 con 0x7fa68009e1d0 2023-10-03T10:19:22.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.436+0000 7fa675ffb700 1 -- 172.21.15.3:0/1306353693 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa67002a070 con 0x7fa68009e1d0 2023-10-03T10:19:22.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.436+0000 7fa675ffb700 1 -- 172.21.15.3:0/1306353693 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa670048ab0 con 0x7fa68009e1d0 2023-10-03T10:19:22.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.445+0000 7fa675ffb700 1 -- 172.21.15.3:0/1306353693 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa67004ae60 con 0x7fa68009e1d0 2023-10-03T10:19:22.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.516+0000 7f9ae4130700 1 Processor -- start 2023-10-03T10:19:22.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.516+0000 7f9ae4130700 1 -- start start 2023-10-03T10:19:22.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.517+0000 7f9ae4130700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9adc09e1d0 0x7f9adc135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:22.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.517+0000 7f9ae4130700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9adc135e50 0x7f9adc138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:22.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.517+0000 7f9ae4130700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9adc09f130 con 0x7f9adc09d6f0 2023-10-03T10:19:22.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.517+0000 7f9ae4130700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc09f520 con 0x7f9adc135e50 2023-10-03T10:19:22.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.517+0000 7f9ae4130700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc09e930 con 0x7f9adc09e1d0 2023-10-03T10:19:22.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.517+0000 7f9ae26cd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9adc135e50 0x7f9adc138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:22.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.517+0000 7f9ae26cd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9adc135e50 0x7f9adc138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:54504/0 (socket says 172.21.15.3:54504) 2023-10-03T10:19:22.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.517+0000 7f9ae26cd700 1 -- 172.21.15.3:0/276620486 learned_addr learned my addr 172.21.15.3:0/276620486 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:22.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.517+0000 7f9ae16cb700 1 --2- 172.21.15.3:0/276620486 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9adc09e1d0 0x7f9adc135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:22.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.517+0000 7f9ae26cd700 1 -- 172.21.15.3:0/276620486 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9adc09e1d0 msgr2=0x7f9adc135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:22.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.517+0000 7f9ae26cd700 1 --2- 172.21.15.3:0/276620486 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9adc09e1d0 0x7f9adc135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.517+0000 7f9ae26cd700 1 -- 172.21.15.3:0/276620486 >> v1:172.21.15.3:6789/0 conn(0x7f9adc09d6f0 legacy=0x7f9adc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:22.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.517+0000 7f9ae26cd700 1 -- 172.21.15.3:0/276620486 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9adc09b2d0 con 0x7f9adc135e50 2023-10-03T10:19:22.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.517+0000 7f9ae16cb700 1 --2- 172.21.15.3:0/276620486 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9adc09e1d0 0x7f9adc135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:22.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.517+0000 7f9ae26cd700 1 --2- 172.21.15.3:0/276620486 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9adc135e50 0x7f9adc138230 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009eb0 tx=0x7f9ad400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e85609afafd75baf server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:22.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.518+0000 7f9ae0eca700 1 -- 172.21.15.3:0/276620486 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad400e070 con 0x7f9adc135e50 2023-10-03T10:19:22.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.518+0000 7f9ae0eca700 1 -- 172.21.15.3:0/276620486 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad4003a20 con 0x7f9adc135e50 2023-10-03T10:19:22.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.518+0000 7f9ae0eca700 1 -- 172.21.15.3:0/276620486 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4003dd0 con 0x7f9adc135e50 2023-10-03T10:19:22.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.519+0000 7f9ae4130700 1 -- 172.21.15.3:0/276620486 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9adc135e50 msgr2=0x7f9adc138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:22.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.519+0000 7f9ae4130700 1 --2- 172.21.15.3:0/276620486 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9adc135e50 0x7f9adc138230 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009eb0 tx=0x7f9ad400b330 comp rx=0 tx=0).stop 2023-10-03T10:19:22.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.519+0000 7f9ae4130700 1 -- 172.21.15.3:0/276620486 shutdown_connections 2023-10-03T10:19:22.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.519+0000 7f9ae4130700 1 --2- 172.21.15.3:0/276620486 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9adc135e50 0x7f9adc138230 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.519+0000 7f9ae4130700 1 --2- 172.21.15.3:0/276620486 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9adc09e1d0 0x7f9adc135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.519+0000 7f9ae4130700 1 -- 172.21.15.3:0/276620486 >> 172.21.15.3:0/276620486 conn(0x7f9adc066f50 msgr2=0x7f9adc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:22.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.519+0000 7f9ae4130700 1 -- 172.21.15.3:0/276620486 shutdown_connections 2023-10-03T10:19:22.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.519+0000 7f9ae4130700 1 -- 172.21.15.3:0/276620486 wait complete. 2023-10-03T10:19:22.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.520+0000 7f9ae4130700 1 Processor -- start 2023-10-03T10:19:22.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.520+0000 7f9ae4130700 1 -- start start 2023-10-03T10:19:22.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.520+0000 7f9ae4130700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9adc09d6f0 0x7f9adc134c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:22.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.520+0000 7f9ae4130700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9adc09e1d0 0x7f9adc1351d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:22.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.520+0000 7f9ae4130700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9adc135e50 0x7f9adc135710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:22.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.520+0000 7f9ae4130700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc133920 con 0x7f9adc09d6f0 2023-10-03T10:19:22.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.520+0000 7f9ae4130700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc1327b0 con 0x7f9adc09e1d0 2023-10-03T10:19:22.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.520+0000 7f9ae4130700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc133dd0 con 0x7f9adc135e50 2023-10-03T10:19:22.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.521+0000 7f9ae1ecc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9adc09d6f0 0x7f9adc134c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:22.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.521+0000 7f9ae16cb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9adc09e1d0 0x7f9adc1351d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:22.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.521+0000 7f9ae1ecc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9adc09d6f0 0x7f9adc134c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54850/0 (socket says 172.21.15.3:54850) 2023-10-03T10:19:22.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.521+0000 7f9ae1ecc700 1 -- 172.21.15.3:0/2015555224 learned_addr learned my addr 172.21.15.3:0/2015555224 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:22.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.521+0000 7f9ae26cd700 1 --2- 172.21.15.3:0/2015555224 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9adc135e50 0x7f9adc135710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:22.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.521+0000 7f9ae1ecc700 1 -- 172.21.15.3:0/2015555224 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9adc135e50 msgr2=0x7f9adc135710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:22.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.521+0000 7f9ae1ecc700 1 --2- 172.21.15.3:0/2015555224 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9adc135e50 0x7f9adc135710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.521+0000 7f9ae1ecc700 1 -- 172.21.15.3:0/2015555224 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9adc09e1d0 msgr2=0x7f9adc1351d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:22.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.521+0000 7f9ae1ecc700 1 --2- 172.21.15.3:0/2015555224 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9adc09e1d0 0x7f9adc1351d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.521+0000 7f9ae1ecc700 1 -- 172.21.15.3:0/2015555224 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad4009b30 con 0x7f9adc09d6f0 2023-10-03T10:19:22.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.521+0000 7f9ae1ecc700 1 --2- 172.21.15.3:0/2015555224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9adc09d6f0 0x7f9adc134c90 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f9ad800c4d0 tx=0x7f9ad800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:22.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.521+0000 7f9ad27fc700 1 -- 172.21.15.3:0/2015555224 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad800cea0 con 0x7f9adc09d6f0 2023-10-03T10:19:22.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.522+0000 7f9ad27fc700 1 -- 172.21.15.3:0/2015555224 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad8005500 con 0x7f9adc09d6f0 2023-10-03T10:19:22.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.522+0000 7f9ad27fc700 1 -- 172.21.15.3:0/2015555224 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad80038c0 con 0x7f9adc09d6f0 2023-10-03T10:19:22.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.522+0000 7f9ae4130700 1 -- 172.21.15.3:0/2015555224 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9adc1333f0 con 0x7f9adc09d6f0 2023-10-03T10:19:22.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.522+0000 7f9ae4130700 1 -- 172.21.15.3:0/2015555224 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9adc134840 con 0x7f9adc09d6f0 2023-10-03T10:19:22.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.523+0000 7f9ae4130700 1 -- 172.21.15.3:0/2015555224 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9adc071700 con 0x7f9adc09d6f0 2023-10-03T10:19:22.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.523+0000 7f9ad27fc700 1 -- 172.21.15.3:0/2015555224 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9ad8003a20 con 0x7f9adc09d6f0 2023-10-03T10:19:22.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.523+0000 7f9ad27fc700 1 -- 172.21.15.3:0/2015555224 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9ad803e8d0 con 0x7f9adc09d6f0 2023-10-03T10:19:22.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.524+0000 7f1e7cc11700 1 Processor -- start 2023-10-03T10:19:22.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.524+0000 7f1e7cc11700 1 -- start start 2023-10-03T10:19:22.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.525+0000 7f1e7cc11700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e78131810 0x7f1e78131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:22.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.525+0000 7f1e7cc11700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e78099a40 0x7f1e78099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:22.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.525+0000 7f1e7cc11700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e781313c0 con 0x7f1e7809a3b0 2023-10-03T10:19:22.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.525+0000 7f1e7cc11700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7809b580 con 0x7f1e78099a40 2023-10-03T10:19:22.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.525+0000 7f1e7cc11700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7806e8c0 con 0x7f1e78131810 2023-10-03T10:19:22.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.525+0000 7f1e75d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e78099a40 0x7f1e78099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:22.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.525+0000 7f1e76d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1e7809a3b0 0x7f1e78138650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51846/0 (socket says 172.21.15.3:51846) 2023-10-03T10:19:22.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.525+0000 7f1e76d9d700 1 -- 172.21.15.3:0/2137626076 learned_addr learned my addr 172.21.15.3:0/2137626076 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:22.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.525+0000 7f1e75d9b700 1 -- 172.21.15.3:0/2137626076 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e78131810 msgr2=0x7f1e78131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:22.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.525+0000 7f1e75d9b700 1 --2- 172.21.15.3:0/2137626076 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e78131810 0x7f1e78131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.525+0000 7f1e75d9b700 1 -- 172.21.15.3:0/2137626076 >> v1:172.21.15.3:6789/0 conn(0x7f1e7809a3b0 legacy=0x7f1e78138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:22.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.525+0000 7f1e75d9b700 1 -- 172.21.15.3:0/2137626076 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e7809a820 con 0x7f1e78099a40 2023-10-03T10:19:22.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.525+0000 7f1e75d9b700 1 --2- 172.21.15.3:0/2137626076 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e78099a40 0x7f1e78099e70 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c009870 tx=0x7f1e6c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12771a5d48ebcc1f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:22.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.526+0000 7f1e7559a700 1 -- 172.21.15.3:0/2137626076 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c011830 con 0x7f1e78099a40 2023-10-03T10:19:22.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.526+0000 7f1e7559a700 1 -- 172.21.15.3:0/2137626076 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6c011990 con 0x7f1e78099a40 2023-10-03T10:19:22.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.526+0000 7f1e7559a700 1 -- 172.21.15.3:0/2137626076 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c011d20 con 0x7f1e78099a40 2023-10-03T10:19:22.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.526+0000 7f1e7cc11700 1 -- 172.21.15.3:0/2137626076 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e78099a40 msgr2=0x7f1e78099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:22.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.526+0000 7f1e7cc11700 1 --2- 172.21.15.3:0/2137626076 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e78099a40 0x7f1e78099e70 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c009870 tx=0x7f1e6c009b80 comp rx=0 tx=0).stop 2023-10-03T10:19:22.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.527+0000 7f1e7cc11700 1 -- 172.21.15.3:0/2137626076 shutdown_connections 2023-10-03T10:19:22.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.527+0000 7f1e7cc11700 1 --2- 172.21.15.3:0/2137626076 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e78131810 0x7f1e78131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.527+0000 7f1e7cc11700 1 --2- 172.21.15.3:0/2137626076 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e78099a40 0x7f1e78099e70 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.527+0000 7f1e7cc11700 1 -- 172.21.15.3:0/2137626076 >> 172.21.15.3:0/2137626076 conn(0x7f1e780a2aa0 msgr2=0x7f1e780a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:22.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.527+0000 7f1e7cc11700 1 -- 172.21.15.3:0/2137626076 shutdown_connections 2023-10-03T10:19:22.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.527+0000 7f1e7cc11700 1 -- 172.21.15.3:0/2137626076 wait complete. 2023-10-03T10:19:22.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.528+0000 7f1e7cc11700 1 Processor -- start 2023-10-03T10:19:22.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.528+0000 7f1e7cc11700 1 -- start start 2023-10-03T10:19:22.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.529+0000 7f1e7cc11700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e78099a40 0x7f1e78119710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:22.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.529+0000 7f1e7cc11700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e7809a3b0 0x7f1e78119c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:22.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.529+0000 7f1e7cc11700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e78131810 0x7f1e7811a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:22.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.529+0000 7f1e7cc11700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7811ab70 con 0x7f1e7809a3b0 2023-10-03T10:19:22.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.529+0000 7f1e7cc11700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7811c120 con 0x7f1e78131810 2023-10-03T10:19:22.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.529+0000 7f1e7cc11700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7812cbc0 con 0x7f1e78099a40 2023-10-03T10:19:22.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.529+0000 7f1e7659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e78099a40 0x7f1e78119710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:22.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.529+0000 7f1e7659c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e78099a40 0x7f1e78119710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:41914/0 (socket says 172.21.15.3:41914) 2023-10-03T10:19:22.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.529+0000 7f1e7659c700 1 -- 172.21.15.3:0/1286873845 learned_addr learned my addr 172.21.15.3:0/1286873845 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:22.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.529+0000 7f1e75d9b700 1 --2- 172.21.15.3:0/1286873845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e7809a3b0 0x7f1e78119c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:22.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.529+0000 7f1e76d9d700 1 --2- 172.21.15.3:0/1286873845 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e78131810 0x7f1e7811a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:22.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.529+0000 7f1e7659c700 1 -- 172.21.15.3:0/1286873845 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e78131810 msgr2=0x7f1e7811a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:22.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.529+0000 7f1e7659c700 1 --2- 172.21.15.3:0/1286873845 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e78131810 0x7f1e7811a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.529+0000 7f1e7659c700 1 -- 172.21.15.3:0/1286873845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e7809a3b0 msgr2=0x7f1e78119c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:22.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.529+0000 7f1e7659c700 1 --2- 172.21.15.3:0/1286873845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e7809a3b0 0x7f1e78119c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.529+0000 7f1e7659c700 1 -- 172.21.15.3:0/1286873845 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e6c0094a0 con 0x7f1e78099a40 2023-10-03T10:19:22.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.529+0000 7f1e75d9b700 1 --2- 172.21.15.3:0/1286873845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e7809a3b0 0x7f1e78119c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:22.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.530+0000 7f1e7659c700 1 --2- 172.21.15.3:0/1286873845 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e78099a40 0x7f1e78119710 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800b530 tx=0x7f1e6800b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:22.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.530+0000 7f1e66ffd700 1 -- 172.21.15.3:0/1286873845 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68011830 con 0x7f1e78099a40 2023-10-03T10:19:22.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.530+0000 7f1e66ffd700 1 -- 172.21.15.3:0/1286873845 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e68011990 con 0x7f1e78099a40 2023-10-03T10:19:22.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.530+0000 7f1e66ffd700 1 -- 172.21.15.3:0/1286873845 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68011d40 con 0x7f1e78099a40 2023-10-03T10:19:22.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.530+0000 7f1e7cc11700 1 -- 172.21.15.3:0/1286873845 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e78130ae0 con 0x7f1e78099a40 2023-10-03T10:19:22.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.530+0000 7f1e7cc11700 1 -- 172.21.15.3:0/1286873845 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e7806eb60 con 0x7f1e78099a40 2023-10-03T10:19:22.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.531+0000 7f1e66ffd700 1 -- 172.21.15.3:0/1286873845 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1e6801a400 con 0x7f1e78099a40 2023-10-03T10:19:22.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.532+0000 7f1e7cc11700 1 -- 172.21.15.3:0/1286873845 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e7811da80 con 0x7f1e78099a40 2023-10-03T10:19:22.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.532+0000 7f1e66ffd700 1 -- 172.21.15.3:0/1286873845 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1e6803f710 con 0x7f1e78099a40 2023-10-03T10:19:22.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.532+0000 7f9ad27fc700 1 -- 172.21.15.3:0/2015555224 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9ad8018af0 con 0x7f9adc09d6f0 2023-10-03T10:19:22.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.537+0000 7f1e66ffd700 1 -- 172.21.15.3:0/1286873845 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1e6800f8d0 con 0x7f1e78099a40 2023-10-03T10:19:22.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.644+0000 7fa3e2d63700 1 -- 172.21.15.3:0/307686690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa3dc0a3de0 con 0x7fa3dc136fe0 2023-10-03T10:19:22.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.645+0000 7fa3d97fa700 1 -- 172.21.15.3:0/307686690 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa3d40133e0 con 0x7fa3dc136fe0 2023-10-03T10:19:22.645 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:22.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.647+0000 7fa3e2d63700 1 -- 172.21.15.3:0/307686690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3dc136fe0 msgr2=0x7fa3dc06c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:22.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.647+0000 7fa3e2d63700 1 --2- 172.21.15.3:0/307686690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3dc136fe0 0x7fa3dc06c330 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400efc0 tx=0x7fa3d400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.647+0000 7fa3e2d63700 1 -- 172.21.15.3:0/307686690 shutdown_connections 2023-10-03T10:19:22.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.647+0000 7fa3e2d63700 1 --2- 172.21.15.3:0/307686690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3dc136fe0 0x7fa3dc06c330 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.647+0000 7fa3e2d63700 1 --2- 172.21.15.3:0/307686690 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa3dc09e930 0x7fa3dc06b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.647+0000 7fa3e2d63700 1 --2- 172.21.15.3:0/307686690 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa3dc09f410 0x7fa3dc06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.647+0000 7fa3e2d63700 1 -- 172.21.15.3:0/307686690 >> 172.21.15.3:0/307686690 conn(0x7fa3dc0a4210 msgr2=0x7fa3dc134b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:22.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.647+0000 7fa3e2d63700 1 -- 172.21.15.3:0/307686690 shutdown_connections 2023-10-03T10:19:22.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.647+0000 7fa3e2d63700 1 -- 172.21.15.3:0/307686690 wait complete. 2023-10-03T10:19:22.662 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:22.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.663+0000 7fa6872f5700 1 -- 172.21.15.3:0/1306353693 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa68009f420 con 0x7fa68009e1d0 2023-10-03T10:19:22.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.663+0000 7fa675ffb700 1 -- 172.21.15.3:0/1306353693 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa670015070 con 0x7fa68009e1d0 2023-10-03T10:19:22.663 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:22.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.665+0000 7fa6872f5700 1 -- 172.21.15.3:0/1306353693 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa68009e1d0 msgr2=0x7fa680121ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:22.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.665+0000 7fa6872f5700 1 --2- 172.21.15.3:0/1306353693 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa68009e1d0 0x7fa680121ea0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fa670000c00 tx=0x7fa670004750 comp rx=0 tx=0).stop 2023-10-03T10:19:22.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.665+0000 7fa6872f5700 1 -- 172.21.15.3:0/1306353693 shutdown_connections 2023-10-03T10:19:22.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.665+0000 7fa6872f5700 1 --2- 172.21.15.3:0/1306353693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa680139170 0x7fa6801223e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.666+0000 7fa6872f5700 1 --2- 172.21.15.3:0/1306353693 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa68009d6f0 0x7fa680121960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.666+0000 7fa6872f5700 1 --2- 172.21.15.3:0/1306353693 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa68009e1d0 0x7fa680121ea0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.666+0000 7fa6872f5700 1 -- 172.21.15.3:0/1306353693 >> 172.21.15.3:0/1306353693 conn(0x7fa680066f50 msgr2=0x7fa680070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:22.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.666+0000 7fa6872f5700 1 -- 172.21.15.3:0/1306353693 shutdown_connections 2023-10-03T10:19:22.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.666+0000 7fa6872f5700 1 -- 172.21.15.3:0/1306353693 wait complete. 2023-10-03T10:19:22.680 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:22.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.740+0000 7f9ae4130700 1 -- 172.21.15.3:0/2015555224 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9adc074ea0 con 0x7f9adc09d6f0 2023-10-03T10:19:22.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.741+0000 7f9ad27fc700 1 -- 172.21.15.3:0/2015555224 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9ad8017030 con 0x7f9adc09d6f0 2023-10-03T10:19:22.741 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:22.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.743+0000 7f9ae4130700 1 -- 172.21.15.3:0/2015555224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9adc09d6f0 msgr2=0x7f9adc134c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:22.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.743+0000 7f9ae4130700 1 --2- 172.21.15.3:0/2015555224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9adc09d6f0 0x7f9adc134c90 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f9ad800c4d0 tx=0x7f9ad800c7e0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.743+0000 7f9ae4130700 1 -- 172.21.15.3:0/2015555224 shutdown_connections 2023-10-03T10:19:22.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.743+0000 7f9ae4130700 1 --2- 172.21.15.3:0/2015555224 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9adc135e50 0x7f9adc135710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.743+0000 7f9ae4130700 1 --2- 172.21.15.3:0/2015555224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9adc09d6f0 0x7f9adc134c90 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.743+0000 7f9ae4130700 1 --2- 172.21.15.3:0/2015555224 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9adc09e1d0 0x7f9adc1351d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.743+0000 7f9ae4130700 1 -- 172.21.15.3:0/2015555224 >> 172.21.15.3:0/2015555224 conn(0x7f9adc066f50 msgr2=0x7f9adc06e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:22.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.743+0000 7f9ae4130700 1 -- 172.21.15.3:0/2015555224 shutdown_connections 2023-10-03T10:19:22.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.743+0000 7f9ae4130700 1 -- 172.21.15.3:0/2015555224 wait complete. 2023-10-03T10:19:22.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.748+0000 7f1e7cc11700 1 -- 172.21.15.3:0/1286873845 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1e78074ea0 con 0x7f1e78099a40 2023-10-03T10:19:22.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.748+0000 7f1e66ffd700 1 -- 172.21.15.3:0/1286873845 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1e68015030 con 0x7f1e78099a40 2023-10-03T10:19:22.748 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:22.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.750+0000 7f1e7cc11700 1 -- 172.21.15.3:0/1286873845 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e78099a40 msgr2=0x7f1e78119710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:22.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.750+0000 7f1e7cc11700 1 --2- 172.21.15.3:0/1286873845 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e78099a40 0x7f1e78119710 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800b530 tx=0x7f1e6800b840 comp rx=0 tx=0).stop 2023-10-03T10:19:22.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.750+0000 7f1e7cc11700 1 -- 172.21.15.3:0/1286873845 shutdown_connections 2023-10-03T10:19:22.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.750+0000 7f1e7cc11700 1 --2- 172.21.15.3:0/1286873845 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1e78131810 0x7f1e7811a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.750+0000 7f1e7cc11700 1 --2- 172.21.15.3:0/1286873845 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1e78099a40 0x7f1e78119710 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.750+0000 7f1e7cc11700 1 --2- 172.21.15.3:0/1286873845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e7809a3b0 0x7f1e78119c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:22.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.750+0000 7f1e7cc11700 1 -- 172.21.15.3:0/1286873845 >> 172.21.15.3:0/1286873845 conn(0x7f1e780a2aa0 msgr2=0x7f1e7812e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:22.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.751+0000 7f1e7cc11700 1 -- 172.21.15.3:0/1286873845 shutdown_connections 2023-10-03T10:19:22.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:22.751+0000 7f1e7cc11700 1 -- 172.21.15.3:0/1286873845 wait complete. 2023-10-03T10:19:22.758 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:22.765 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:23.663 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:23.681 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:23.759 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:23.765 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:23.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.814+0000 7f597af02700 1 Processor -- start 2023-10-03T10:19:23.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.814+0000 7f597af02700 1 -- start start 2023-10-03T10:19:23.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.814+0000 7f597af02700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f597406e570 0x7f597406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:23.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.814+0000 7f597af02700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f597409eca0 0x7f597406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:23.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.814+0000 7f597af02700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5974134610 0x7f5974134a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:23.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.814+0000 7f597af02700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5974070ee0 con 0x7f597406e570 2023-10-03T10:19:23.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.814+0000 7f597af02700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5974071480 con 0x7f5974134610 2023-10-03T10:19:23.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.814+0000 7f597af02700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f597406f3c0 con 0x7f597409eca0 2023-10-03T10:19:23.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.815+0000 7f5973fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f597409eca0 0x7f597406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:23.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.815+0000 7f5973fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f597409eca0 0x7f597406ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36496/0 (socket says 172.21.15.3:36496) 2023-10-03T10:19:23.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.815+0000 7f5973fff700 1 -- 172.21.15.3:0/3879453067 learned_addr learned my addr 172.21.15.3:0/3879453067 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:23.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.815+0000 7f597949f700 1 --2- 172.21.15.3:0/3879453067 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5974134610 0x7f5974134a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:23.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.815+0000 7f5978c9e700 1 --2- 172.21.15.3:0/3879453067 >> v2:172.21.15.3:3300/0 conn(0x7f597406e570 0x7f597406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:23.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.815+0000 7f5973fff700 1 -- 172.21.15.3:0/3879453067 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5974134610 msgr2=0x7f5974134a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:23.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.815+0000 7f5973fff700 1 --2- 172.21.15.3:0/3879453067 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5974134610 0x7f5974134a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:23.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.815+0000 7f5973fff700 1 -- 172.21.15.3:0/3879453067 >> v2:172.21.15.3:3300/0 conn(0x7f597406e570 msgr2=0x7f597406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:23.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.815+0000 7f5973fff700 1 --2- 172.21.15.3:0/3879453067 >> v2:172.21.15.3:3300/0 conn(0x7f597406e570 0x7f597406e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:23.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.815+0000 7f5973fff700 1 -- 172.21.15.3:0/3879453067 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5974071b90 con 0x7f597409eca0 2023-10-03T10:19:23.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.815+0000 7f5978c9e700 1 --2- 172.21.15.3:0/3879453067 >> v2:172.21.15.3:3300/0 conn(0x7f597406e570 0x7f597406e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:23.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.815+0000 7f5973fff700 1 --2- 172.21.15.3:0/3879453067 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f597409eca0 0x7f597406ee80 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f59600098f0 tx=0x7f5960009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17c444ea4e004551 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:23.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.816+0000 7f59737fe700 1 -- 172.21.15.3:0/3879453067 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5960011830 con 0x7f597409eca0 2023-10-03T10:19:23.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.816+0000 7f59737fe700 1 -- 172.21.15.3:0/3879453067 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5960011990 con 0x7f597409eca0 2023-10-03T10:19:23.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.816+0000 7f59737fe700 1 -- 172.21.15.3:0/3879453067 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5960011d20 con 0x7f597409eca0 2023-10-03T10:19:23.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.816+0000 7f597af02700 1 -- 172.21.15.3:0/3879453067 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f597409eca0 msgr2=0x7f597406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:23.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.816+0000 7f597af02700 1 --2- 172.21.15.3:0/3879453067 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f597409eca0 0x7f597406ee80 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f59600098f0 tx=0x7f5960009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:23.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.817+0000 7f597af02700 1 -- 172.21.15.3:0/3879453067 shutdown_connections 2023-10-03T10:19:23.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.817+0000 7f597af02700 1 --2- 172.21.15.3:0/3879453067 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5974134610 0x7f5974134a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:23.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.817+0000 7f597af02700 1 --2- 172.21.15.3:0/3879453067 >> v2:172.21.15.3:3300/0 conn(0x7f597406e570 0x7f597406e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:23.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.817+0000 7f597af02700 1 --2- 172.21.15.3:0/3879453067 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f597409eca0 0x7f597406ee80 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:23.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.817+0000 7f597af02700 1 -- 172.21.15.3:0/3879453067 >> 172.21.15.3:0/3879453067 conn(0x7f59740a2aa0 msgr2=0x7f59740a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:23.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.817+0000 7f597af02700 1 -- 172.21.15.3:0/3879453067 shutdown_connections 2023-10-03T10:19:23.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.817+0000 7f597af02700 1 -- 172.21.15.3:0/3879453067 wait complete. 2023-10-03T10:19:23.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.818+0000 7f597af02700 1 Processor -- start 2023-10-03T10:19:23.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.818+0000 7f597af02700 1 -- start start 2023-10-03T10:19:23.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.819+0000 7f597af02700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f597406e570 0x7f5974127480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:23.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.819+0000 7f597af02700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5974134610 0x7f5974125ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:23.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.819+0000 7f597af02700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59741279c0 0x7f59741263e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:23.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.819+0000 7f597af02700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597413e330 con 0x7f597406e570 2023-10-03T10:19:23.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.819+0000 7f597af02700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597413b350 con 0x7f59741279c0 2023-10-03T10:19:23.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.819+0000 7f597af02700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f597413c110 con 0x7f5974134610 2023-10-03T10:19:23.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.819+0000 7f5978c9e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f597406e570 0x7f5974127480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:23.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.819+0000 7f5978c9e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f597406e570 0x7f5974127480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50790/0 (socket says 172.21.15.3:50790) 2023-10-03T10:19:23.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.819+0000 7f5978c9e700 1 -- 172.21.15.3:0/3310044693 learned_addr learned my addr 172.21.15.3:0/3310044693 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:23.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.819+0000 7f597949f700 1 --2- 172.21.15.3:0/3310044693 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59741279c0 0x7f59741263e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:23.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.819+0000 7f5973fff700 1 --2- 172.21.15.3:0/3310044693 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5974134610 0x7f5974125ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:23.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.819+0000 7f5978c9e700 1 -- 172.21.15.3:0/3310044693 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5974134610 msgr2=0x7f5974125ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:23.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.819+0000 7f5978c9e700 1 --2- 172.21.15.3:0/3310044693 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5974134610 0x7f5974125ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:23.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.819+0000 7f5978c9e700 1 -- 172.21.15.3:0/3310044693 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59741279c0 msgr2=0x7f59741263e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:23.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.819+0000 7f5978c9e700 1 --2- 172.21.15.3:0/3310044693 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59741279c0 0x7f59741263e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:23.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.820+0000 7f5978c9e700 1 -- 172.21.15.3:0/3310044693 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59680094a0 con 0x7f597406e570 2023-10-03T10:19:23.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.820+0000 7f597949f700 1 --2- 172.21.15.3:0/3310044693 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59741279c0 0x7f59741263e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:23.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.820+0000 7f5978c9e700 1 --2- 172.21.15.3:0/3310044693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f597406e570 0x7f5974127480 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f596800ebe0 tx=0x7f596800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:23.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.820+0000 7f59717fa700 1 -- 172.21.15.3:0/3310044693 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596800c850 con 0x7f597406e570 2023-10-03T10:19:23.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.820+0000 7f59717fa700 1 -- 172.21.15.3:0/3310044693 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596800c9b0 con 0x7f597406e570 2023-10-03T10:19:23.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.820+0000 7f597af02700 1 -- 172.21.15.3:0/3310044693 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5960009550 con 0x7f597406e570 2023-10-03T10:19:23.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.820+0000 7f59717fa700 1 -- 172.21.15.3:0/3310044693 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596800cd40 con 0x7f597406e570 2023-10-03T10:19:23.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.820+0000 7f597af02700 1 -- 172.21.15.3:0/3310044693 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5974070610 con 0x7f597406e570 2023-10-03T10:19:23.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.821+0000 7f59717fa700 1 -- 172.21.15.3:0/3310044693 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5968013470 con 0x7f597406e570 2023-10-03T10:19:23.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.821+0000 7f597af02700 1 -- 172.21.15.3:0/3310044693 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f597409d1f0 con 0x7f597406e570 2023-10-03T10:19:23.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.822+0000 7f59717fa700 1 -- 172.21.15.3:0/3310044693 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f596803f7f0 con 0x7f597406e570 2023-10-03T10:19:23.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.828+0000 7f59717fa700 1 -- 172.21.15.3:0/3310044693 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5968013720 con 0x7f597406e570 2023-10-03T10:19:23.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.863+0000 7f29560c0700 1 Processor -- start 2023-10-03T10:19:23.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.864+0000 7f29560c0700 1 -- start start 2023-10-03T10:19:23.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.864+0000 7f29560c0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f295009d6f0 0x7f295009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:23.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.864+0000 7f29560c0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f295009e000 0x7f29501339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:23.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.864+0000 7f29560c0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f295009b120 con 0x7f295009e410 2023-10-03T10:19:23.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.864+0000 7f29560c0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295006b460 con 0x7f295009d6f0 2023-10-03T10:19:23.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.864+0000 7f29560c0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f295009a7e0 con 0x7f295009e000 2023-10-03T10:19:23.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.865+0000 7f294ffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f295009e410 0x7f2950133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33102/0 (socket says 172.21.15.3:33102) 2023-10-03T10:19:23.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.865+0000 7f294ffff700 1 -- 172.21.15.3:0/2870299725 learned_addr learned my addr 172.21.15.3:0/2870299725 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:23.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.865+0000 7f294effd700 1 --2- 172.21.15.3:0/2870299725 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f295009e000 0x7f29501339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:23.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.865+0000 7f294f7fe700 1 --2- 172.21.15.3:0/2870299725 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f295009d6f0 0x7f295009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:23.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.865+0000 7f294e7fc700 1 -- 172.21.15.3:0/2870299725 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3369374057 0 0) 0x7f295009b120 con 0x7f295009e410 2023-10-03T10:19:23.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.865+0000 7f294e7fc700 1 -- 172.21.15.3:0/2870299725 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29380035b0 con 0x7f295009e410 2023-10-03T10:19:23.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.865+0000 7f294effd700 1 -- 172.21.15.3:0/2870299725 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f295009d6f0 msgr2=0x7f295009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:23.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.865+0000 7f294effd700 1 --2- 172.21.15.3:0/2870299725 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f295009d6f0 0x7f295009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:23.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.865+0000 7f294effd700 1 -- 172.21.15.3:0/2870299725 >> v1:172.21.15.3:6789/0 conn(0x7f295009e410 legacy=0x7f2950133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:23.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.865+0000 7f294effd700 1 -- 172.21.15.3:0/2870299725 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295009a500 con 0x7f295009e000 2023-10-03T10:19:23.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.865+0000 7f294effd700 1 --2- 172.21.15.3:0/2870299725 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f295009e000 0x7f29501339f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f29440098a0 tx=0x7f2944009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a86d40b297d97e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:23.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.866+0000 7f294e7fc700 1 -- 172.21.15.3:0/2870299725 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2944011830 con 0x7f295009e000 2023-10-03T10:19:23.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.866+0000 7f294e7fc700 1 -- 172.21.15.3:0/2870299725 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2944011990 con 0x7f295009e000 2023-10-03T10:19:23.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.866+0000 7f294e7fc700 1 -- 172.21.15.3:0/2870299725 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2944011d20 con 0x7f295009e000 2023-10-03T10:19:23.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.867+0000 7f29560c0700 1 -- 172.21.15.3:0/2870299725 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f295009e000 msgr2=0x7f29501339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:23.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.867+0000 7f29560c0700 1 --2- 172.21.15.3:0/2870299725 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f295009e000 0x7f29501339f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f29440098a0 tx=0x7f2944009bb0 comp rx=0 tx=0).stop 2023-10-03T10:19:23.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.867+0000 7f29560c0700 1 -- 172.21.15.3:0/2870299725 shutdown_connections 2023-10-03T10:19:23.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.867+0000 7f29560c0700 1 --2- 172.21.15.3:0/2870299725 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f295009d6f0 0x7f295009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:23.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.867+0000 7f29560c0700 1 --2- 172.21.15.3:0/2870299725 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f295009e000 0x7f29501339f0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:23.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.867+0000 7f29560c0700 1 -- 172.21.15.3:0/2870299725 >> 172.21.15.3:0/2870299725 conn(0x7f2950066f50 msgr2=0x7f295009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:23.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.867+0000 7f29560c0700 1 -- 172.21.15.3:0/2870299725 shutdown_connections 2023-10-03T10:19:23.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.867+0000 7f29560c0700 1 -- 172.21.15.3:0/2870299725 wait complete. 2023-10-03T10:19:23.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.868+0000 7f29560c0700 1 Processor -- start 2023-10-03T10:19:23.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.869+0000 7f29560c0700 1 -- start start 2023-10-03T10:19:23.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.869+0000 7f29560c0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f295009d6f0 0x7f29501194b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:23.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.869+0000 7f29560c0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f295009e000 0x7f29501199f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:23.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.870+0000 7f29560c0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f295009e410 0x7f2950119f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:23.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.870+0000 7f29560c0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295011ccf0 con 0x7f295009d6f0 2023-10-03T10:19:23.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.870+0000 7f29560c0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295009c710 con 0x7f295009e410 2023-10-03T10:19:23.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.870+0000 7f29560c0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29501ae4a0 con 0x7f295009e000 2023-10-03T10:19:23.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.870+0000 7f294f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f295009d6f0 0x7f29501194b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:23.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.870+0000 7f294f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f295009d6f0 0x7f29501194b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50798/0 (socket says 172.21.15.3:50798) 2023-10-03T10:19:23.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.870+0000 7f294f7fe700 1 -- 172.21.15.3:0/2610702342 learned_addr learned my addr 172.21.15.3:0/2610702342 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:23.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.870+0000 7f294ffff700 1 --2- 172.21.15.3:0/2610702342 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f295009e410 0x7f2950119f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:23.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.870+0000 7f294effd700 1 --2- 172.21.15.3:0/2610702342 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f295009e000 0x7f29501199f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:23.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.870+0000 7f294f7fe700 1 -- 172.21.15.3:0/2610702342 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f295009e000 msgr2=0x7f29501199f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:23.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.870+0000 7f294f7fe700 1 --2- 172.21.15.3:0/2610702342 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f295009e000 0x7f29501199f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:23.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.870+0000 7f294f7fe700 1 -- 172.21.15.3:0/2610702342 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f295009e410 msgr2=0x7f2950119f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:23.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.870+0000 7f294f7fe700 1 --2- 172.21.15.3:0/2610702342 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f295009e410 0x7f2950119f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:23.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.870+0000 7f294f7fe700 1 -- 172.21.15.3:0/2610702342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29440094a0 con 0x7f295009d6f0 2023-10-03T10:19:23.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.870+0000 7f294f7fe700 1 --2- 172.21.15.3:0/2610702342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f295009d6f0 0x7f29501194b0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f294000e9b0 tx=0x7f294000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:23.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.871+0000 7f2937fff700 1 -- 172.21.15.3:0/2610702342 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2940009710 con 0x7f295009d6f0 2023-10-03T10:19:23.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.871+0000 7f2937fff700 1 -- 172.21.15.3:0/2610702342 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294000c850 con 0x7f295009d6f0 2023-10-03T10:19:23.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.871+0000 7f29560c0700 1 -- 172.21.15.3:0/2610702342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29500a1e90 con 0x7f295009d6f0 2023-10-03T10:19:23.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.871+0000 7f2937fff700 1 -- 172.21.15.3:0/2610702342 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294000cbc0 con 0x7f295009d6f0 2023-10-03T10:19:23.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.871+0000 7f29560c0700 1 -- 172.21.15.3:0/2610702342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f295011a470 con 0x7f295009d6f0 2023-10-03T10:19:23.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.872+0000 7f2937fff700 1 -- 172.21.15.3:0/2610702342 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f294000cd20 con 0x7f295009d6f0 2023-10-03T10:19:23.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.872+0000 7f29560c0700 1 -- 172.21.15.3:0/2610702342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2950074ea0 con 0x7f295009d6f0 2023-10-03T10:19:23.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.872+0000 7f2937fff700 1 -- 172.21.15.3:0/2610702342 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f294003fb80 con 0x7f295009d6f0 2023-10-03T10:19:23.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.877+0000 7f2937fff700 1 -- 172.21.15.3:0/2610702342 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f294001b3b0 con 0x7f295009d6f0 2023-10-03T10:19:23.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.912+0000 7f5166115700 1 Processor -- start 2023-10-03T10:19:23.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.913+0000 7f5166115700 1 -- start start 2023-10-03T10:19:23.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.913+0000 7f5166115700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5160133a80 0x7f5160133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:23.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.913+0000 7f5166115700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5160130a70 0x7f516013b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:23.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.913+0000 7f5166115700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f516006bc00 con 0x7f516012ff20 2023-10-03T10:19:23.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.913+0000 7f5166115700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f516006c570 con 0x7f5160133a80 2023-10-03T10:19:23.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.913+0000 7f5166115700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f51600a0040 con 0x7f5160130a70 2023-10-03T10:19:23.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.913+0000 7f515ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5160130a70 0x7f516013b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:23.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.913+0000 7f515ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5160130a70 0x7f516013b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36532/0 (socket says 172.21.15.3:36532) 2023-10-03T10:19:23.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.913+0000 7f515ffff700 1 -- 172.21.15.3:0/3808580158 learned_addr learned my addr 172.21.15.3:0/3808580158 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:23.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.914+0000 7f515f7fe700 1 --2- 172.21.15.3:0/3808580158 >> v2:172.21.15.3:3300/0 conn(0x7f5160133a80 0x7f5160133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:23.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.914+0000 7f515ffff700 1 -- 172.21.15.3:0/3808580158 >> v2:172.21.15.3:3300/0 conn(0x7f5160133a80 msgr2=0x7f5160133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:23.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.914+0000 7f515ffff700 1 --2- 172.21.15.3:0/3808580158 >> v2:172.21.15.3:3300/0 conn(0x7f5160133a80 0x7f5160133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:23.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.914+0000 7f515ffff700 1 -- 172.21.15.3:0/3808580158 >> v1:172.21.15.3:6789/0 conn(0x7f516012ff20 legacy=0x7f5160130360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:23.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.914+0000 7f515ffff700 1 -- 172.21.15.3:0/3808580158 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5160133860 con 0x7f5160130a70 2023-10-03T10:19:23.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.914+0000 7f515ffff700 1 --2- 172.21.15.3:0/3808580158 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5160130a70 0x7f516013b850 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f5158009e50 tx=0x7f515800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=452be89a76b999b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:23.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.914+0000 7f515e7fc700 1 -- 172.21.15.3:0/3808580158 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515800e070 con 0x7f5160130a70 2023-10-03T10:19:23.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.914+0000 7f515e7fc700 1 -- 172.21.15.3:0/3808580158 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515800b9c0 con 0x7f5160130a70 2023-10-03T10:19:23.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.914+0000 7f515e7fc700 1 -- 172.21.15.3:0/3808580158 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515800bd70 con 0x7f5160130a70 2023-10-03T10:19:23.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.915+0000 7f5166115700 1 -- 172.21.15.3:0/3808580158 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5160130a70 msgr2=0x7f516013b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:23.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.915+0000 7f5166115700 1 --2- 172.21.15.3:0/3808580158 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5160130a70 0x7f516013b850 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f5158009e50 tx=0x7f515800b330 comp rx=0 tx=0).stop 2023-10-03T10:19:23.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.915+0000 7f5166115700 1 -- 172.21.15.3:0/3808580158 shutdown_connections 2023-10-03T10:19:23.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.915+0000 7f5166115700 1 --2- 172.21.15.3:0/3808580158 >> v2:172.21.15.3:3300/0 conn(0x7f5160133a80 0x7f5160133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:23.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.915+0000 7f5166115700 1 --2- 172.21.15.3:0/3808580158 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5160130a70 0x7f516013b850 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:23.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.915+0000 7f5166115700 1 -- 172.21.15.3:0/3808580158 >> 172.21.15.3:0/3808580158 conn(0x7f51600a4210 msgr2=0x7f51600a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:23.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.915+0000 7f5166115700 1 -- 172.21.15.3:0/3808580158 shutdown_connections 2023-10-03T10:19:23.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.915+0000 7f5166115700 1 -- 172.21.15.3:0/3808580158 wait complete. 2023-10-03T10:19:23.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.916+0000 7f5166115700 1 Processor -- start 2023-10-03T10:19:23.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.917+0000 7f5166115700 1 -- start start 2023-10-03T10:19:23.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.917+0000 7f5166115700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f516012ff20 0x7f516011de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:23.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.917+0000 7f5166115700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5160130a70 0x7f516011e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:23.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.917+0000 7f5166115700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5160133a80 0x7f516011c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:23.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.917+0000 7f5166115700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516011b840 con 0x7f5160133a80 2023-10-03T10:19:23.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.917+0000 7f5166115700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516012dbc0 con 0x7f516012ff20 2023-10-03T10:19:23.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.917+0000 7f5166115700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f516011bb00 con 0x7f5160130a70 2023-10-03T10:19:23.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.917+0000 7f515f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f516012ff20 0x7f516011de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:23.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.917+0000 7f515f7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f516012ff20 0x7f516011de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51652/0 (socket says 172.21.15.3:51652) 2023-10-03T10:19:23.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.917+0000 7f515f7fe700 1 -- 172.21.15.3:0/234921861 learned_addr learned my addr 172.21.15.3:0/234921861 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:23.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.917+0000 7f515ffff700 1 --2- 172.21.15.3:0/234921861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5160133a80 0x7f516011c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:23.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.918+0000 7f515effd700 1 --2- 172.21.15.3:0/234921861 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5160130a70 0x7f516011e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:23.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.918+0000 7f515f7fe700 1 -- 172.21.15.3:0/234921861 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5160130a70 msgr2=0x7f516011e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:23.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.918+0000 7f515f7fe700 1 --2- 172.21.15.3:0/234921861 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5160130a70 0x7f516011e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:23.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.918+0000 7f515f7fe700 1 -- 172.21.15.3:0/234921861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5160133a80 msgr2=0x7f516011c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:23.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.918+0000 7f515f7fe700 1 --2- 172.21.15.3:0/234921861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5160133a80 0x7f516011c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:23.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.918+0000 7f515f7fe700 1 -- 172.21.15.3:0/234921861 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5158009b30 con 0x7f516012ff20 2023-10-03T10:19:23.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.918+0000 7f515effd700 1 --2- 172.21.15.3:0/234921861 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5160130a70 0x7f516011e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:19:23.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.918+0000 7f515f7fe700 1 --2- 172.21.15.3:0/234921861 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f516012ff20 0x7f516011de00 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f515000e810 tx=0x7f515000eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:23.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.918+0000 7f5147fff700 1 -- 172.21.15.3:0/234921861 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5150013610 con 0x7f516012ff20 2023-10-03T10:19:23.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.918+0000 7f5147fff700 1 -- 172.21.15.3:0/234921861 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5150013770 con 0x7f516012ff20 2023-10-03T10:19:23.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.918+0000 7f5166115700 1 -- 172.21.15.3:0/234921861 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51600a20e0 con 0x7f516012ff20 2023-10-03T10:19:23.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.918+0000 7f5147fff700 1 -- 172.21.15.3:0/234921861 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5150013b00 con 0x7f516012ff20 2023-10-03T10:19:23.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.918+0000 7f5166115700 1 -- 172.21.15.3:0/234921861 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f516011c570 con 0x7f516012ff20 2023-10-03T10:19:23.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.920+0000 7f5147fff700 1 -- 172.21.15.3:0/234921861 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5150013c60 con 0x7f516012ff20 2023-10-03T10:19:23.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.920+0000 7f5166115700 1 -- 172.21.15.3:0/234921861 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5160074ea0 con 0x7f516012ff20 2023-10-03T10:19:23.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.920+0000 7f5147fff700 1 -- 172.21.15.3:0/234921861 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f515003f3d0 con 0x7f516012ff20 2023-10-03T10:19:23.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.929+0000 7f5147fff700 1 -- 172.21.15.3:0/234921861 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5150012370 con 0x7f516012ff20 2023-10-03T10:19:23.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.958+0000 7f920f034700 1 Processor -- start 2023-10-03T10:19:23.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.958+0000 7f920f034700 1 -- start start 2023-10-03T10:19:23.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.958+0000 7f920f034700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f920809c650 0x7f920809ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:23.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.958+0000 7f920f034700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f920809cf60 0x7f9208132080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:23.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.958+0000 7f920f034700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f920809ba70 con 0x7f92080a0090 2023-10-03T10:19:23.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.958+0000 7f920f034700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920809e8f0 con 0x7f920809c650 2023-10-03T10:19:23.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.958+0000 7f920f034700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f920809d400 con 0x7f920809cf60 2023-10-03T10:19:23.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.958+0000 7f920d5d1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f920809cf60 0x7f9208132080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:23.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.959+0000 7f920d5d1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f920809cf60 0x7f9208132080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36546/0 (socket says 172.21.15.3:36546) 2023-10-03T10:19:23.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.959+0000 7f920d5d1700 1 -- 172.21.15.3:0/3272878573 learned_addr learned my addr 172.21.15.3:0/3272878573 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:23.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.959+0000 7f920cdd0700 1 --2- 172.21.15.3:0/3272878573 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f920809c650 0x7f920809ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:23.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.959+0000 7f920d5d1700 1 -- 172.21.15.3:0/3272878573 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f920809c650 msgr2=0x7f920809ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:23.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.959+0000 7f920d5d1700 1 --2- 172.21.15.3:0/3272878573 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f920809c650 0x7f920809ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:23.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.959+0000 7f92077fe700 1 -- 172.21.15.3:0/3272878573 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1781603756 0 0) 0x7f920809ba70 con 0x7f92080a0090 2023-10-03T10:19:23.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.959+0000 7f920d5d1700 1 -- 172.21.15.3:0/3272878573 >> v1:172.21.15.3:6789/0 conn(0x7f92080a0090 legacy=0x7f9208131970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:23.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.959+0000 7f920d5d1700 1 -- 172.21.15.3:0/3272878573 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92080a0760 con 0x7f920809cf60 2023-10-03T10:19:23.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.959+0000 7f920d5d1700 1 --2- 172.21.15.3:0/3272878573 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f920809cf60 0x7f9208132080 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9200009e80 tx=0x7f920000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6452239207f3ad5a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:23.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.959+0000 7f92077fe700 1 -- 172.21.15.3:0/3272878573 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920000e070 con 0x7f920809cf60 2023-10-03T10:19:23.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.959+0000 7f92077fe700 1 -- 172.21.15.3:0/3272878573 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f920000b910 con 0x7f920809cf60 2023-10-03T10:19:23.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.959+0000 7f92077fe700 1 -- 172.21.15.3:0/3272878573 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920000bcc0 con 0x7f920809cf60 2023-10-03T10:19:23.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.960+0000 7f920f034700 1 -- 172.21.15.3:0/3272878573 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f920809cf60 msgr2=0x7f9208132080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:23.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.960+0000 7f920f034700 1 --2- 172.21.15.3:0/3272878573 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f920809cf60 0x7f9208132080 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9200009e80 tx=0x7f920000b330 comp rx=0 tx=0).stop 2023-10-03T10:19:23.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.960+0000 7f920f034700 1 -- 172.21.15.3:0/3272878573 shutdown_connections 2023-10-03T10:19:23.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.960+0000 7f920f034700 1 --2- 172.21.15.3:0/3272878573 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f920809c650 0x7f920809ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:23.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.960+0000 7f920f034700 1 --2- 172.21.15.3:0/3272878573 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f920809cf60 0x7f9208132080 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:23.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.960+0000 7f920f034700 1 -- 172.21.15.3:0/3272878573 >> 172.21.15.3:0/3272878573 conn(0x7f92080a3f70 msgr2=0x7f92080a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:23.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.960+0000 7f920f034700 1 -- 172.21.15.3:0/3272878573 shutdown_connections 2023-10-03T10:19:23.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.960+0000 7f920f034700 1 -- 172.21.15.3:0/3272878573 wait complete. 2023-10-03T10:19:23.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.962+0000 7f920f034700 1 Processor -- start 2023-10-03T10:19:23.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.962+0000 7f920f034700 1 -- start start 2023-10-03T10:19:23.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.963+0000 7f920f034700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f920809c650 0x7f920811bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:23.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.963+0000 7f920f034700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f920809cf60 0x7f920811c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:23.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.963+0000 7f920f034700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f92080a0090 0x7f920811ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:23.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.963+0000 7f920f034700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920811dad0 con 0x7f920809c650 2023-10-03T10:19:23.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.963+0000 7f920f034700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920811d3b0 con 0x7f920809cf60 2023-10-03T10:19:23.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.963+0000 7f920f034700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f920809d980 con 0x7f92080a0090 2023-10-03T10:19:23.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.963+0000 7f920d5d1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f92080a0090 0x7f920811ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:23.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.963+0000 7f920d5d1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f92080a0090 0x7f920811ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36552/0 (socket says 172.21.15.3:36552) 2023-10-03T10:19:23.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.963+0000 7f920d5d1700 1 -- 172.21.15.3:0/3717689449 learned_addr learned my addr 172.21.15.3:0/3717689449 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:23.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.963+0000 7f920cdd0700 1 --2- 172.21.15.3:0/3717689449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f920809c650 0x7f920811bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:23.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.963+0000 7f9207fff700 1 --2- 172.21.15.3:0/3717689449 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f920809cf60 0x7f920811c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:23.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.963+0000 7f920d5d1700 1 -- 172.21.15.3:0/3717689449 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f920809cf60 msgr2=0x7f920811c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:23.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.963+0000 7f920d5d1700 1 --2- 172.21.15.3:0/3717689449 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f920809cf60 0x7f920811c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:23.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.963+0000 7f920d5d1700 1 -- 172.21.15.3:0/3717689449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f920809c650 msgr2=0x7f920811bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:23.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.963+0000 7f920d5d1700 1 --2- 172.21.15.3:0/3717689449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f920809c650 0x7f920811bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:23.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.963+0000 7f920d5d1700 1 -- 172.21.15.3:0/3717689449 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9200009b30 con 0x7f92080a0090 2023-10-03T10:19:23.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.964+0000 7f920d5d1700 1 --2- 172.21.15.3:0/3717689449 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f92080a0090 0x7f920811ca10 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f920809d590 tx=0x7f9200004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:23.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.964+0000 7f92057fa700 1 -- 172.21.15.3:0/3717689449 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92000048c0 con 0x7f92080a0090 2023-10-03T10:19:23.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.964+0000 7f92057fa700 1 -- 172.21.15.3:0/3717689449 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92000092e0 con 0x7f92080a0090 2023-10-03T10:19:23.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.964+0000 7f920f034700 1 -- 172.21.15.3:0/3717689449 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f920811f700 con 0x7f92080a0090 2023-10-03T10:19:23.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.964+0000 7f92057fa700 1 -- 172.21.15.3:0/3717689449 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920001c660 con 0x7f92080a0090 2023-10-03T10:19:23.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.964+0000 7f920f034700 1 -- 172.21.15.3:0/3717689449 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f920811ed20 con 0x7f92080a0090 2023-10-03T10:19:23.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.965+0000 7f92057fa700 1 -- 172.21.15.3:0/3717689449 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f920001c7c0 con 0x7f92080a0090 2023-10-03T10:19:23.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.965+0000 7f920f034700 1 -- 172.21.15.3:0/3717689449 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9208070fb0 con 0x7f92080a0090 2023-10-03T10:19:23.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.966+0000 7f92057fa700 1 -- 172.21.15.3:0/3717689449 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9200024a00 con 0x7f92080a0090 2023-10-03T10:19:23.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:23.975+0000 7f92057fa700 1 -- 172.21.15.3:0/3717689449 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9200046060 con 0x7f92080a0090 2023-10-03T10:19:24.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.053+0000 7f597af02700 1 -- 172.21.15.3:0/3310044693 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5974074ea0 con 0x7f597406e570 2023-10-03T10:19:24.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.053+0000 7f59717fa700 1 -- 172.21.15.3:0/3310044693 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5968021020 con 0x7f597406e570 2023-10-03T10:19:24.053 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:24.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.055+0000 7f597af02700 1 -- 172.21.15.3:0/3310044693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f597406e570 msgr2=0x7f5974127480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:24.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.055+0000 7f597af02700 1 --2- 172.21.15.3:0/3310044693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f597406e570 0x7f5974127480 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f596800ebe0 tx=0x7f596800eef0 comp rx=0 tx=0).stop 2023-10-03T10:19:24.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.056+0000 7f597af02700 1 -- 172.21.15.3:0/3310044693 shutdown_connections 2023-10-03T10:19:24.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.056+0000 7f597af02700 1 --2- 172.21.15.3:0/3310044693 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59741279c0 0x7f59741263e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:24.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.056+0000 7f597af02700 1 --2- 172.21.15.3:0/3310044693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f597406e570 0x7f5974127480 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:24.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.056+0000 7f597af02700 1 --2- 172.21.15.3:0/3310044693 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5974134610 0x7f5974125ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:24.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.056+0000 7f597af02700 1 -- 172.21.15.3:0/3310044693 >> 172.21.15.3:0/3310044693 conn(0x7f59740a2aa0 msgr2=0x7f5974137c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:24.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.056+0000 7f597af02700 1 -- 172.21.15.3:0/3310044693 shutdown_connections 2023-10-03T10:19:24.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.056+0000 7f597af02700 1 -- 172.21.15.3:0/3310044693 wait complete. 2023-10-03T10:19:24.070 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:24.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.091+0000 7f29560c0700 1 -- 172.21.15.3:0/2610702342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f295006c360 con 0x7f295009d6f0 2023-10-03T10:19:24.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.092+0000 7f2937fff700 1 -- 172.21.15.3:0/2610702342 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2940021070 con 0x7f295009d6f0 2023-10-03T10:19:24.092 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:24.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.094+0000 7f29560c0700 1 -- 172.21.15.3:0/2610702342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f295009d6f0 msgr2=0x7f29501194b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:24.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.094+0000 7f29560c0700 1 --2- 172.21.15.3:0/2610702342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f295009d6f0 0x7f29501194b0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f294000e9b0 tx=0x7f294000ecc0 comp rx=0 tx=0).stop 2023-10-03T10:19:24.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.094+0000 7f29560c0700 1 -- 172.21.15.3:0/2610702342 shutdown_connections 2023-10-03T10:19:24.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.094+0000 7f29560c0700 1 --2- 172.21.15.3:0/2610702342 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f295009e410 0x7f2950119f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:24.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.094+0000 7f29560c0700 1 --2- 172.21.15.3:0/2610702342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f295009d6f0 0x7f29501194b0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:24.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.094+0000 7f29560c0700 1 --2- 172.21.15.3:0/2610702342 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f295009e000 0x7f29501199f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:24.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.094+0000 7f29560c0700 1 -- 172.21.15.3:0/2610702342 >> 172.21.15.3:0/2610702342 conn(0x7f2950066f50 msgr2=0x7f29501bac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:24.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.094+0000 7f29560c0700 1 -- 172.21.15.3:0/2610702342 shutdown_connections 2023-10-03T10:19:24.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.094+0000 7f29560c0700 1 -- 172.21.15.3:0/2610702342 wait complete. 2023-10-03T10:19:24.108 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:24.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.145+0000 7f5166115700 1 -- 172.21.15.3:0/234921861 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f516006b250 con 0x7f516012ff20 2023-10-03T10:19:24.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.146+0000 7f5147fff700 1 -- 172.21.15.3:0/234921861 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5150021070 con 0x7f516012ff20 2023-10-03T10:19:24.146 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:24.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.148+0000 7f5166115700 1 -- 172.21.15.3:0/234921861 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f516012ff20 msgr2=0x7f516011de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:24.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.148+0000 7f5166115700 1 --2- 172.21.15.3:0/234921861 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f516012ff20 0x7f516011de00 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f515000e810 tx=0x7f515000eb20 comp rx=0 tx=0).stop 2023-10-03T10:19:24.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.148+0000 7f5166115700 1 -- 172.21.15.3:0/234921861 shutdown_connections 2023-10-03T10:19:24.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.148+0000 7f5166115700 1 --2- 172.21.15.3:0/234921861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5160133a80 0x7f516011c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:24.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.148+0000 7f5166115700 1 --2- 172.21.15.3:0/234921861 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f516012ff20 0x7f516011de00 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:24.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.148+0000 7f5166115700 1 --2- 172.21.15.3:0/234921861 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5160130a70 0x7f516011e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:24.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.148+0000 7f5166115700 1 -- 172.21.15.3:0/234921861 >> 172.21.15.3:0/234921861 conn(0x7f51600a4210 msgr2=0x7f516009e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:24.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.148+0000 7f5166115700 1 -- 172.21.15.3:0/234921861 shutdown_connections 2023-10-03T10:19:24.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.148+0000 7f5166115700 1 -- 172.21.15.3:0/234921861 wait complete. 2023-10-03T10:19:24.162 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:24.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.187+0000 7f920f034700 1 -- 172.21.15.3:0/3717689449 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f92080747f0 con 0x7f92080a0090 2023-10-03T10:19:24.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.188+0000 7f92057fa700 1 -- 172.21.15.3:0/3717689449 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9200046030 con 0x7f92080a0090 2023-10-03T10:19:24.188 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:24.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.190+0000 7f920f034700 1 -- 172.21.15.3:0/3717689449 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f92080a0090 msgr2=0x7f920811ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.190+0000 7f920f034700 1 --2- 172.21.15.3:0/3717689449 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f92080a0090 0x7f920811ca10 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f920809d590 tx=0x7f9200004ad0 comp rx=0 tx=0).stop 2023-10-03T10:19:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.190+0000 7f920f034700 1 -- 172.21.15.3:0/3717689449 shutdown_connections 2023-10-03T10:19:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.190+0000 7f920f034700 1 --2- 172.21.15.3:0/3717689449 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f92080a0090 0x7f920811ca10 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.190+0000 7f920f034700 1 --2- 172.21.15.3:0/3717689449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f920809c650 0x7f920811bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.190+0000 7f920f034700 1 --2- 172.21.15.3:0/3717689449 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f920809cf60 0x7f920811c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.190+0000 7f920f034700 1 -- 172.21.15.3:0/3717689449 >> 172.21.15.3:0/3717689449 conn(0x7f92080a3f70 msgr2=0x7f920806d030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.190+0000 7f920f034700 1 -- 172.21.15.3:0/3717689449 shutdown_connections 2023-10-03T10:19:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:24.190+0000 7f920f034700 1 -- 172.21.15.3:0/3717689449 wait complete. 2023-10-03T10:19:24.204 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:25.071 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:25.109 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:25.162 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:25.204 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:25.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.222+0000 7f27a1e3b700 1 Processor -- start 2023-10-03T10:19:25.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.222+0000 7f27a1e3b700 1 -- start start 2023-10-03T10:19:25.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.223+0000 7f27a1e3b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f279c06e570 0x7f279c06e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:25.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.223+0000 7f27a1e3b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f279c06ee80 0x7f279c134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:25.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.223+0000 7f27a1e3b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f279c070ee0 con 0x7f279c09eca0 2023-10-03T10:19:25.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.223+0000 7f27a1e3b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f279c071480 con 0x7f279c06e570 2023-10-03T10:19:25.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.223+0000 7f27a1e3b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f279c071b90 con 0x7f279c06ee80 2023-10-03T10:19:25.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.223+0000 7f279bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f279c06ee80 0x7f279c134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:25.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.223+0000 7f279bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f279c06ee80 0x7f279c134d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36560/0 (socket says 172.21.15.3:36560) 2023-10-03T10:19:25.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.223+0000 7f279bfff700 1 -- 172.21.15.3:0/2979433017 learned_addr learned my addr 172.21.15.3:0/2979433017 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:25.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.223+0000 7f279b7fe700 1 --2- 172.21.15.3:0/2979433017 >> v2:172.21.15.3:3300/0 conn(0x7f279c06e570 0x7f279c06e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:25.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.224+0000 7f279bfff700 1 -- 172.21.15.3:0/2979433017 >> v2:172.21.15.3:3300/0 conn(0x7f279c06e570 msgr2=0x7f279c06e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:25.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.224+0000 7f279bfff700 1 --2- 172.21.15.3:0/2979433017 >> v2:172.21.15.3:3300/0 conn(0x7f279c06e570 0x7f279c06e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.224+0000 7f279bfff700 1 -- 172.21.15.3:0/2979433017 >> v1:172.21.15.3:6789/0 conn(0x7f279c09eca0 legacy=0x7f279c134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:25.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.224+0000 7f279bfff700 1 -- 172.21.15.3:0/2979433017 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f279c0711a0 con 0x7f279c06ee80 2023-10-03T10:19:25.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.224+0000 7f279bfff700 1 --2- 172.21.15.3:0/2979433017 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f279c06ee80 0x7f279c134d20 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f2794009e50 tx=0x7f279400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23db14d478fd6b01 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:25.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.224+0000 7f279a7fc700 1 -- 172.21.15.3:0/2979433017 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279400e070 con 0x7f279c06ee80 2023-10-03T10:19:25.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.224+0000 7f279a7fc700 1 -- 172.21.15.3:0/2979433017 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279400b910 con 0x7f279c06ee80 2023-10-03T10:19:25.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.224+0000 7f279a7fc700 1 -- 172.21.15.3:0/2979433017 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279400bcc0 con 0x7f279c06ee80 2023-10-03T10:19:25.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.225+0000 7f27a1e3b700 1 -- 172.21.15.3:0/2979433017 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f279c06ee80 msgr2=0x7f279c134d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:25.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.225+0000 7f27a1e3b700 1 --2- 172.21.15.3:0/2979433017 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f279c06ee80 0x7f279c134d20 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f2794009e50 tx=0x7f279400b330 comp rx=0 tx=0).stop 2023-10-03T10:19:25.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.225+0000 7f27a1e3b700 1 -- 172.21.15.3:0/2979433017 shutdown_connections 2023-10-03T10:19:25.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.225+0000 7f27a1e3b700 1 --2- 172.21.15.3:0/2979433017 >> v2:172.21.15.3:3300/0 conn(0x7f279c06e570 0x7f279c06e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.225+0000 7f27a1e3b700 1 --2- 172.21.15.3:0/2979433017 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f279c06ee80 0x7f279c134d20 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.225+0000 7f27a1e3b700 1 -- 172.21.15.3:0/2979433017 >> 172.21.15.3:0/2979433017 conn(0x7f279c0a2aa0 msgr2=0x7f279c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:25.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.225+0000 7f27a1e3b700 1 -- 172.21.15.3:0/2979433017 shutdown_connections 2023-10-03T10:19:25.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.225+0000 7f27a1e3b700 1 -- 172.21.15.3:0/2979433017 wait complete. 2023-10-03T10:19:25.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.226+0000 7f27a1e3b700 1 Processor -- start 2023-10-03T10:19:25.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.227+0000 7f27a1e3b700 1 -- start start 2023-10-03T10:19:25.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.227+0000 7f27a1e3b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f279c06e570 0x7f279c134a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:25.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.227+0000 7f27a1e3b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f279c06ee80 0x7f279c139e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:25.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.227+0000 7f27a1e3b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f279c13a3c0 0x7f279c13a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:25.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.227+0000 7f27a1e3b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279c13bb30 con 0x7f279c06ee80 2023-10-03T10:19:25.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.227+0000 7f27a1e3b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279c13b940 con 0x7f279c13a3c0 2023-10-03T10:19:25.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.227+0000 7f27a1e3b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f279c13b2c0 con 0x7f279c06e570 2023-10-03T10:19:25.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.227+0000 7f279bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f279c13a3c0 0x7f279c13a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:25.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.227+0000 7f279bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f279c13a3c0 0x7f279c13a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51672/0 (socket says 172.21.15.3:51672) 2023-10-03T10:19:25.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.227+0000 7f279bfff700 1 -- 172.21.15.3:0/1057036996 learned_addr learned my addr 172.21.15.3:0/1057036996 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:25.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.227+0000 7f279affd700 1 --2- 172.21.15.3:0/1057036996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f279c06ee80 0x7f279c139e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:25.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.228+0000 7f279b7fe700 1 --2- 172.21.15.3:0/1057036996 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f279c06e570 0x7f279c134a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:25.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.228+0000 7f279bfff700 1 -- 172.21.15.3:0/1057036996 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f279c06e570 msgr2=0x7f279c134a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:25.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.228+0000 7f279bfff700 1 --2- 172.21.15.3:0/1057036996 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f279c06e570 0x7f279c134a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.228+0000 7f279bfff700 1 -- 172.21.15.3:0/1057036996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f279c06ee80 msgr2=0x7f279c139e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:25.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.228+0000 7f279bfff700 1 --2- 172.21.15.3:0/1057036996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f279c06ee80 0x7f279c139e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.228+0000 7f279bfff700 1 -- 172.21.15.3:0/1057036996 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2794009b30 con 0x7f279c13a3c0 2023-10-03T10:19:25.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.228+0000 7f279bfff700 1 --2- 172.21.15.3:0/1057036996 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f279c13a3c0 0x7f279c13a830 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f279400b680 tx=0x7f2794004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:25.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.228+0000 7f2783fff700 1 -- 172.21.15.3:0/1057036996 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2794004830 con 0x7f279c13a3c0 2023-10-03T10:19:25.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.228+0000 7f2783fff700 1 -- 172.21.15.3:0/1057036996 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27940092e0 con 0x7f279c13a3c0 2023-10-03T10:19:25.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.228+0000 7f27a1e3b700 1 -- 172.21.15.3:0/1057036996 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f279c135920 con 0x7f279c13a3c0 2023-10-03T10:19:25.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.229+0000 7f27a1e3b700 1 -- 172.21.15.3:0/1057036996 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f279c070660 con 0x7f279c13a3c0 2023-10-03T10:19:25.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.229+0000 7f2783fff700 1 -- 172.21.15.3:0/1057036996 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279401c660 con 0x7f279c13a3c0 2023-10-03T10:19:25.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.229+0000 7f2783fff700 1 -- 172.21.15.3:0/1057036996 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f279401c7c0 con 0x7f279c13a3c0 2023-10-03T10:19:25.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.230+0000 7f2783fff700 1 -- 172.21.15.3:0/1057036996 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2794041530 con 0x7f279c13a3c0 2023-10-03T10:19:25.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.232+0000 7f27a1e3b700 1 -- 172.21.15.3:0/1057036996 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f277c0052e0 con 0x7f279c13a3c0 2023-10-03T10:19:25.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.241+0000 7f2783fff700 1 -- 172.21.15.3:0/1057036996 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f279402d940 con 0x7f279c13a3c0 2023-10-03T10:19:25.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.262+0000 7f2a5949a700 1 Processor -- start 2023-10-03T10:19:25.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.262+0000 7f2a5949a700 1 -- start start 2023-10-03T10:19:25.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.263+0000 7f2a5949a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2a5406e570 0x7f2a5406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:25.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.263+0000 7f2a5949a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2a5406ee80 0x7f2a54134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:25.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.263+0000 7f2a5949a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a54070ee0 con 0x7f2a5409eca0 2023-10-03T10:19:25.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.263+0000 7f2a5949a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a54071480 con 0x7f2a5406e570 2023-10-03T10:19:25.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.263+0000 7f2a5949a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a54071b90 con 0x7f2a5406ee80 2023-10-03T10:19:25.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.263+0000 7f2a52ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2a5406e570 0x7f2a5406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:25.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.263+0000 7f2a52ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2a5406e570 0x7f2a5406e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50858/0 (socket says 172.21.15.3:50858) 2023-10-03T10:19:25.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.263+0000 7f2a52ffd700 1 -- 172.21.15.3:0/3555138668 learned_addr learned my addr 172.21.15.3:0/3555138668 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:25.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.263+0000 7f2a537fe700 1 --2- 172.21.15.3:0/3555138668 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2a5406ee80 0x7f2a54134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:25.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.264+0000 7f2a537fe700 1 -- 172.21.15.3:0/3555138668 >> v2:172.21.15.3:3300/0 conn(0x7f2a5406e570 msgr2=0x7f2a5406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:25.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.264+0000 7f2a537fe700 1 --2- 172.21.15.3:0/3555138668 >> v2:172.21.15.3:3300/0 conn(0x7f2a5406e570 0x7f2a5406e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.264+0000 7f2a537fe700 1 -- 172.21.15.3:0/3555138668 >> v1:172.21.15.3:6789/0 conn(0x7f2a5409eca0 legacy=0x7f2a54134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:25.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.264+0000 7f2a537fe700 1 -- 172.21.15.3:0/3555138668 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a540711a0 con 0x7f2a5406ee80 2023-10-03T10:19:25.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.264+0000 7f2a537fe700 1 --2- 172.21.15.3:0/3555138668 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2a5406ee80 0x7f2a54134d20 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009d90 tx=0x7f2a4c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aea0f00b555e0624 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:25.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.264+0000 7f2a51ffb700 1 -- 172.21.15.3:0/3555138668 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c00e070 con 0x7f2a5406ee80 2023-10-03T10:19:25.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.264+0000 7f2a51ffb700 1 -- 172.21.15.3:0/3555138668 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4c00bb20 con 0x7f2a5406ee80 2023-10-03T10:19:25.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.264+0000 7f2a51ffb700 1 -- 172.21.15.3:0/3555138668 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c012430 con 0x7f2a5406ee80 2023-10-03T10:19:25.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.265+0000 7f2a5949a700 1 -- 172.21.15.3:0/3555138668 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2a5406ee80 msgr2=0x7f2a54134d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:25.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.265+0000 7f2a5949a700 1 --2- 172.21.15.3:0/3555138668 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2a5406ee80 0x7f2a54134d20 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009d90 tx=0x7f2a4c00b330 comp rx=0 tx=0).stop 2023-10-03T10:19:25.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.265+0000 7f2a5949a700 1 -- 172.21.15.3:0/3555138668 shutdown_connections 2023-10-03T10:19:25.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.265+0000 7f2a5949a700 1 --2- 172.21.15.3:0/3555138668 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2a5406ee80 0x7f2a54134d20 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.265+0000 7f2a5949a700 1 --2- 172.21.15.3:0/3555138668 >> v2:172.21.15.3:3300/0 conn(0x7f2a5406e570 0x7f2a5406e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.265+0000 7f2a5949a700 1 -- 172.21.15.3:0/3555138668 >> 172.21.15.3:0/3555138668 conn(0x7f2a540a2aa0 msgr2=0x7f2a540a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:25.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.265+0000 7f2a5949a700 1 -- 172.21.15.3:0/3555138668 shutdown_connections 2023-10-03T10:19:25.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.265+0000 7f2a5949a700 1 -- 172.21.15.3:0/3555138668 wait complete. 2023-10-03T10:19:25.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.267+0000 7f2a5949a700 1 Processor -- start 2023-10-03T10:19:25.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.267+0000 7f2a5949a700 1 -- start start 2023-10-03T10:19:25.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.267+0000 7f2a5949a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a5406e570 0x7f2a5413ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:25.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.267+0000 7f2a5949a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2a5406ee80 0x7f2a54124f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:25.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.267+0000 7f2a5949a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2a5413dce0 0x7f2a541254b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:25.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.267+0000 7f2a5949a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a541357b0 con 0x7f2a5406e570 2023-10-03T10:19:25.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.267+0000 7f2a5949a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5413d9e0 con 0x7f2a5406ee80 2023-10-03T10:19:25.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.267+0000 7f2a5949a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2a54126750 con 0x7f2a5413dce0 2023-10-03T10:19:25.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.267+0000 7f2a52ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a5406e570 0x7f2a5413ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:25.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.267+0000 7f2a527fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2a5406ee80 0x7f2a54124f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:25.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.267+0000 7f2a52ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a5406e570 0x7f2a5413ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50862/0 (socket says 172.21.15.3:50862) 2023-10-03T10:19:25.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.267+0000 7f2a52ffd700 1 -- 172.21.15.3:0/2466447970 learned_addr learned my addr 172.21.15.3:0/2466447970 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:25.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.268+0000 7f2a537fe700 1 --2- 172.21.15.3:0/2466447970 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2a5413dce0 0x7f2a541254b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:25.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.268+0000 7f2a527fc700 1 -- 172.21.15.3:0/2466447970 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2a5413dce0 msgr2=0x7f2a541254b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:25.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.268+0000 7f2a527fc700 1 --2- 172.21.15.3:0/2466447970 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2a5413dce0 0x7f2a541254b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.268+0000 7f2a527fc700 1 -- 172.21.15.3:0/2466447970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a5406e570 msgr2=0x7f2a5413ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:25.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.268+0000 7f2a527fc700 1 --2- 172.21.15.3:0/2466447970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a5406e570 0x7f2a5413ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.268+0000 7f2a527fc700 1 -- 172.21.15.3:0/2466447970 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4c009a70 con 0x7f2a5406ee80 2023-10-03T10:19:25.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.268+0000 7f2a52ffd700 1 --2- 172.21.15.3:0/2466447970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a5406e570 0x7f2a5413ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:25.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.268+0000 7f2a527fc700 1 --2- 172.21.15.3:0/2466447970 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2a5406ee80 0x7f2a54124f70 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2a48000f60 tx=0x7f2a4800c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:25.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.268+0000 7f2a3b7fe700 1 -- 172.21.15.3:0/2466447970 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a480050d0 con 0x7f2a5406ee80 2023-10-03T10:19:25.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.268+0000 7f2a3b7fe700 1 -- 172.21.15.3:0/2466447970 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a48005230 con 0x7f2a5406ee80 2023-10-03T10:19:25.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.268+0000 7f2a5949a700 1 -- 172.21.15.3:0/2466447970 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a5413be80 con 0x7f2a5406ee80 2023-10-03T10:19:25.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.268+0000 7f2a3b7fe700 1 -- 172.21.15.3:0/2466447970 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a48003680 con 0x7f2a5406ee80 2023-10-03T10:19:25.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.269+0000 7f2a5949a700 1 -- 172.21.15.3:0/2466447970 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5413d500 con 0x7f2a5406ee80 2023-10-03T10:19:25.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.269+0000 7f2a3b7fe700 1 -- 172.21.15.3:0/2466447970 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2a480037e0 con 0x7f2a5406ee80 2023-10-03T10:19:25.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.270+0000 7f2a3b7fe700 1 -- 172.21.15.3:0/2466447970 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2a4803e9b0 con 0x7f2a5406ee80 2023-10-03T10:19:25.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.270+0000 7f2a5949a700 1 -- 172.21.15.3:0/2466447970 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a5413d1a0 con 0x7f2a5406ee80 2023-10-03T10:19:25.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.279+0000 7f2a3b7fe700 1 -- 172.21.15.3:0/2466447970 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2a5413d1a0 con 0x7f2a5406ee80 2023-10-03T10:19:25.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.317+0000 7f8f74f48700 1 Processor -- start 2023-10-03T10:19:25.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.317+0000 7f8f74f48700 1 -- start start 2023-10-03T10:19:25.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.317+0000 7f8f74f48700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8f7009e930 0x7f8f7009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:25.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.317+0000 7f8f74f48700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f7009f240 0x7f8f701317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:25.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.317+0000 7f8f74f48700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f70070940 con 0x7f8f7009f650 2023-10-03T10:19:25.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.317+0000 7f8f74f48700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f7006fd60 con 0x7f8f7009e930 2023-10-03T10:19:25.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.317+0000 7f8f74f48700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7006fa80 con 0x7f8f7009f240 2023-10-03T10:19:25.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.318+0000 7f8f6e59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8f7009e930 0x7f8f7009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:25.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.318+0000 7f8f6e59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8f7009e930 0x7f8f7009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50870/0 (socket says 172.21.15.3:50870) 2023-10-03T10:19:25.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.318+0000 7f8f6e59c700 1 -- 172.21.15.3:0/2139656275 learned_addr learned my addr 172.21.15.3:0/2139656275 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:25.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.318+0000 7f8f6ed9d700 1 --1- 172.21.15.3:0/2139656275 >> v1:172.21.15.3:6789/0 conn(0x7f8f7009f650 0x7f8f70131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33132/0 (socket says 172.21.15.3:33132) 2023-10-03T10:19:25.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.318+0000 7f8f6dd9b700 1 --2- 172.21.15.3:0/2139656275 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f7009f240 0x7f8f701317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:25.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.318+0000 7f8f6dd9b700 1 -- 172.21.15.3:0/2139656275 >> v2:172.21.15.3:3300/0 conn(0x7f8f7009e930 msgr2=0x7f8f7009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:25.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.318+0000 7f8f6dd9b700 1 --2- 172.21.15.3:0/2139656275 >> v2:172.21.15.3:3300/0 conn(0x7f8f7009e930 0x7f8f7009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.318+0000 7f8f6dd9b700 1 -- 172.21.15.3:0/2139656275 >> v1:172.21.15.3:6789/0 conn(0x7f8f7009f650 legacy=0x7f8f70131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:25.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.318+0000 7f8f6dd9b700 1 -- 172.21.15.3:0/2139656275 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7009fb60 con 0x7f8f7009f240 2023-10-03T10:19:25.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.318+0000 7f8f6e59c700 1 --2- 172.21.15.3:0/2139656275 >> v2:172.21.15.3:3300/0 conn(0x7f8f7009e930 0x7f8f7009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:25.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.318+0000 7f8f6dd9b700 1 --2- 172.21.15.3:0/2139656275 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f7009f240 0x7f8f701317f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009840 tx=0x7f8f64009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2198a318519512b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:25.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.319+0000 7f8f6d59a700 1 -- 172.21.15.3:0/2139656275 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f64011830 con 0x7f8f7009f240 2023-10-03T10:19:25.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.319+0000 7f8f6d59a700 1 -- 172.21.15.3:0/2139656275 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f64011990 con 0x7f8f7009f240 2023-10-03T10:19:25.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.319+0000 7f8f6d59a700 1 -- 172.21.15.3:0/2139656275 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f64011d20 con 0x7f8f7009f240 2023-10-03T10:19:25.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.319+0000 7f8f74f48700 1 -- 172.21.15.3:0/2139656275 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f7009f240 msgr2=0x7f8f701317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:25.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.319+0000 7f8f74f48700 1 --2- 172.21.15.3:0/2139656275 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f7009f240 0x7f8f701317f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009840 tx=0x7f8f64009b50 comp rx=0 tx=0).stop 2023-10-03T10:19:25.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.319+0000 7f8f74f48700 1 -- 172.21.15.3:0/2139656275 shutdown_connections 2023-10-03T10:19:25.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.319+0000 7f8f74f48700 1 --2- 172.21.15.3:0/2139656275 >> v2:172.21.15.3:3300/0 conn(0x7f8f7009e930 0x7f8f7009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.319+0000 7f8f74f48700 1 --2- 172.21.15.3:0/2139656275 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f7009f240 0x7f8f701317f0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.319+0000 7f8f74f48700 1 -- 172.21.15.3:0/2139656275 >> 172.21.15.3:0/2139656275 conn(0x7f8f700a4210 msgr2=0x7f8f700a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:25.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.320+0000 7f8f74f48700 1 -- 172.21.15.3:0/2139656275 shutdown_connections 2023-10-03T10:19:25.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.320+0000 7f8f74f48700 1 -- 172.21.15.3:0/2139656275 wait complete. 2023-10-03T10:19:25.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.321+0000 7f8f74f48700 1 Processor -- start 2023-10-03T10:19:25.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.321+0000 7f8f74f48700 1 -- start start 2023-10-03T10:19:25.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.321+0000 7f8f74f48700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f7009e930 0x7f8f701a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:25.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.321+0000 7f8f74f48700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8f7009f240 0x7f8f701a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:25.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.321+0000 7f8f74f48700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f7009f650 0x7f8f701a6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:25.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.321+0000 7f8f74f48700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f701a71a0 con 0x7f8f7009f650 2023-10-03T10:19:25.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.321+0000 7f8f74f48700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f701a78b0 con 0x7f8f7009f240 2023-10-03T10:19:25.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.321+0000 7f8f74f48700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8f701a7ed0 con 0x7f8f7009e930 2023-10-03T10:19:25.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.322+0000 7f8f6dd9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8f7009f240 0x7f8f701a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:25.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.322+0000 7f8f6dd9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8f7009f240 0x7f8f701a6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51708/0 (socket says 172.21.15.3:51708) 2023-10-03T10:19:25.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.322+0000 7f8f6dd9b700 1 -- 172.21.15.3:0/3702089560 learned_addr learned my addr 172.21.15.3:0/3702089560 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:25.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.322+0000 7f8f6ed9d700 1 --2- 172.21.15.3:0/3702089560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f7009f650 0x7f8f701a6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:25.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.322+0000 7f8f6e59c700 1 --2- 172.21.15.3:0/3702089560 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f7009e930 0x7f8f701a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:25.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.322+0000 7f8f6dd9b700 1 -- 172.21.15.3:0/3702089560 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f7009e930 msgr2=0x7f8f701a5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:25.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.322+0000 7f8f6dd9b700 1 --2- 172.21.15.3:0/3702089560 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f7009e930 0x7f8f701a5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.322+0000 7f8f6dd9b700 1 -- 172.21.15.3:0/3702089560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f7009f650 msgr2=0x7f8f701a6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:25.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.322+0000 7f8f6dd9b700 1 --2- 172.21.15.3:0/3702089560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f7009f650 0x7f8f701a6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.322+0000 7f8f6dd9b700 1 -- 172.21.15.3:0/3702089560 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f60009550 con 0x7f8f7009f240 2023-10-03T10:19:25.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.322+0000 7f8f6ed9d700 1 --2- 172.21.15.3:0/3702089560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f7009f650 0x7f8f701a6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:25.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.322+0000 7f8f6dd9b700 1 --2- 172.21.15.3:0/3702089560 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8f7009f240 0x7f8f701a6230 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009fd0 tx=0x7f8f64004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:25.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.323+0000 7f8f5effd700 1 -- 172.21.15.3:0/3702089560 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f640036a0 con 0x7f8f7009f240 2023-10-03T10:19:25.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.323+0000 7f8f5effd700 1 -- 172.21.15.3:0/3702089560 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6401f070 con 0x7f8f7009f240 2023-10-03T10:19:25.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.323+0000 7f8f74f48700 1 -- 172.21.15.3:0/3702089560 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f640094a0 con 0x7f8f7009f240 2023-10-03T10:19:25.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.323+0000 7f8f5effd700 1 -- 172.21.15.3:0/3702089560 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f64003a60 con 0x7f8f7009f240 2023-10-03T10:19:25.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.323+0000 7f8f74f48700 1 -- 172.21.15.3:0/3702089560 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f700a0190 con 0x7f8f7009f240 2023-10-03T10:19:25.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.324+0000 7f8f5effd700 1 -- 172.21.15.3:0/3702089560 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8f64003bc0 con 0x7f8f7009f240 2023-10-03T10:19:25.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.324+0000 7f8f5effd700 1 -- 172.21.15.3:0/3702089560 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8f64021070 con 0x7f8f7009f240 2023-10-03T10:19:25.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.325+0000 7f8f74f48700 1 -- 172.21.15.3:0/3702089560 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f701a2b40 con 0x7f8f7009f240 2023-10-03T10:19:25.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.330+0000 7f8f5effd700 1 -- 172.21.15.3:0/3702089560 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8f64015080 con 0x7f8f7009f240 2023-10-03T10:19:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.365+0000 7fbc2949b700 1 Processor -- start 2023-10-03T10:19:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.365+0000 7fbc2949b700 1 -- start start 2023-10-03T10:19:25.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.366+0000 7fbc2949b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbc2406a770 0x7fbc2406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:25.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.366+0000 7fbc2949b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc2406b080 0x7fbc24131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:25.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.366+0000 7fbc2949b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc2409ece0 con 0x7fbc24131b40 2023-10-03T10:19:25.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.366+0000 7fbc2949b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc2409e760 con 0x7fbc2406a770 2023-10-03T10:19:25.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.366+0000 7fbc2949b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2409e040 con 0x7fbc2406b080 2023-10-03T10:19:25.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.366+0000 7fbc22ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbc2406a770 0x7fbc2406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:25.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.366+0000 7fbc22ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbc2406a770 0x7fbc2406ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50886/0 (socket says 172.21.15.3:50886) 2023-10-03T10:19:25.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.366+0000 7fbc22ffd700 1 -- 172.21.15.3:0/3078396378 learned_addr learned my addr 172.21.15.3:0/3078396378 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:25.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.366+0000 7fbc227fc700 1 --2- 172.21.15.3:0/3078396378 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc2406b080 0x7fbc24131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:25.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.366+0000 7fbc227fc700 1 -- 172.21.15.3:0/3078396378 >> v2:172.21.15.3:3300/0 conn(0x7fbc2406a770 msgr2=0x7fbc2406ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:25.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.366+0000 7fbc227fc700 1 --2- 172.21.15.3:0/3078396378 >> v2:172.21.15.3:3300/0 conn(0x7fbc2406a770 0x7fbc2406ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.366+0000 7fbc227fc700 1 -- 172.21.15.3:0/3078396378 >> v1:172.21.15.3:6789/0 conn(0x7fbc24131b40 legacy=0x7fbc24138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:25.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.366+0000 7fbc21ffb700 1 -- 172.21.15.3:0/3078396378 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1996966669 0 0) 0x7fbc2409ece0 con 0x7fbc24131b40 2023-10-03T10:19:25.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.366+0000 7fbc227fc700 1 -- 172.21.15.3:0/3078396378 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2409f630 con 0x7fbc2406b080 2023-10-03T10:19:25.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.366+0000 7fbc22ffd700 1 --2- 172.21.15.3:0/3078396378 >> v2:172.21.15.3:3300/0 conn(0x7fbc2406a770 0x7fbc2406ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:25.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.367+0000 7fbc227fc700 1 --2- 172.21.15.3:0/3078396378 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc2406b080 0x7fbc24131600 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fbc18009840 tx=0x7fbc18009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9541c1c14b2a6b14 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:25.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.367+0000 7fbc21ffb700 1 -- 172.21.15.3:0/3078396378 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc18011830 con 0x7fbc2406b080 2023-10-03T10:19:25.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.367+0000 7fbc21ffb700 1 -- 172.21.15.3:0/3078396378 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc18011990 con 0x7fbc2406b080 2023-10-03T10:19:25.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.367+0000 7fbc21ffb700 1 -- 172.21.15.3:0/3078396378 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc18011d20 con 0x7fbc2406b080 2023-10-03T10:19:25.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.368+0000 7fbc2949b700 1 -- 172.21.15.3:0/3078396378 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc2406b080 msgr2=0x7fbc24131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:25.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.368+0000 7fbc2949b700 1 --2- 172.21.15.3:0/3078396378 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc2406b080 0x7fbc24131600 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fbc18009840 tx=0x7fbc18009b50 comp rx=0 tx=0).stop 2023-10-03T10:19:25.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.368+0000 7fbc2949b700 1 -- 172.21.15.3:0/3078396378 shutdown_connections 2023-10-03T10:19:25.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.368+0000 7fbc2949b700 1 --2- 172.21.15.3:0/3078396378 >> v2:172.21.15.3:3300/0 conn(0x7fbc2406a770 0x7fbc2406ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.368+0000 7fbc2949b700 1 --2- 172.21.15.3:0/3078396378 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc2406b080 0x7fbc24131600 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.368+0000 7fbc2949b700 1 -- 172.21.15.3:0/3078396378 >> 172.21.15.3:0/3078396378 conn(0x7fbc24066f50 msgr2=0x7fbc2409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:25.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.368+0000 7fbc2949b700 1 -- 172.21.15.3:0/3078396378 shutdown_connections 2023-10-03T10:19:25.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.368+0000 7fbc2949b700 1 -- 172.21.15.3:0/3078396378 wait complete. 2023-10-03T10:19:25.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.370+0000 7fbc2949b700 1 Processor -- start 2023-10-03T10:19:25.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.370+0000 7fbc2949b700 1 -- start start 2023-10-03T10:19:25.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.370+0000 7fbc2949b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc2406a770 0x7fbc2411c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:25.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.371+0000 7fbc2949b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc2406b080 0x7fbc2411c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:25.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.371+0000 7fbc2949b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc24131b40 0x7fbc2411cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:25.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.371+0000 7fbc2949b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2411f970 con 0x7fbc2406b080 2023-10-03T10:19:25.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.371+0000 7fbc2949b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc24071e60 con 0x7fbc24131b40 2023-10-03T10:19:25.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.371+0000 7fbc2949b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2411e110 con 0x7fbc2406a770 2023-10-03T10:19:25.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.371+0000 7fbc22ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc2406a770 0x7fbc2411c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:25.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.371+0000 7fbc22ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc2406a770 0x7fbc2411c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36614/0 (socket says 172.21.15.3:36614) 2023-10-03T10:19:25.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.371+0000 7fbc22ffd700 1 -- 172.21.15.3:0/1521778780 learned_addr learned my addr 172.21.15.3:0/1521778780 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:25.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.371+0000 7fbc237fe700 1 --2- 172.21.15.3:0/1521778780 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc24131b40 0x7fbc2411cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:25.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.371+0000 7fbc227fc700 1 --2- 172.21.15.3:0/1521778780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc2406b080 0x7fbc2411c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:25.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.371+0000 7fbc227fc700 1 -- 172.21.15.3:0/1521778780 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc2406a770 msgr2=0x7fbc2411c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:25.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.371+0000 7fbc227fc700 1 --2- 172.21.15.3:0/1521778780 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc2406a770 0x7fbc2411c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.371+0000 7fbc227fc700 1 -- 172.21.15.3:0/1521778780 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc24131b40 msgr2=0x7fbc2411cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:25.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.371+0000 7fbc227fc700 1 --2- 172.21.15.3:0/1521778780 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc24131b40 0x7fbc2411cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.371+0000 7fbc227fc700 1 -- 172.21.15.3:0/1521778780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc14009550 con 0x7fbc2406b080 2023-10-03T10:19:25.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.371+0000 7fbc22ffd700 1 --2- 172.21.15.3:0/1521778780 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc2406a770 0x7fbc2411c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:25.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.371+0000 7fbc227fc700 1 --2- 172.21.15.3:0/1521778780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc2406b080 0x7fbc2411c9d0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fbc18016000 tx=0x7fbc18004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:25.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.372+0000 7fbc0b7fe700 1 -- 172.21.15.3:0/1521778780 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc180036a0 con 0x7fbc2406b080 2023-10-03T10:19:25.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.372+0000 7fbc0b7fe700 1 -- 172.21.15.3:0/1521778780 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1801f070 con 0x7fbc2406b080 2023-10-03T10:19:25.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.372+0000 7fbc2949b700 1 -- 172.21.15.3:0/1521778780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc180094a0 con 0x7fbc2406b080 2023-10-03T10:19:25.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.372+0000 7fbc0b7fe700 1 -- 172.21.15.3:0/1521778780 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc18003a20 con 0x7fbc2406b080 2023-10-03T10:19:25.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.372+0000 7fbc2949b700 1 -- 172.21.15.3:0/1521778780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc241315a0 con 0x7fbc2406b080 2023-10-03T10:19:25.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.373+0000 7fbc0b7fe700 1 -- 172.21.15.3:0/1521778780 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbc18003b80 con 0x7fbc2406b080 2023-10-03T10:19:25.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.373+0000 7fbc0b7fe700 1 -- 172.21.15.3:0/1521778780 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fbc18021070 con 0x7fbc2406b080 2023-10-03T10:19:25.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.373+0000 7fbc2949b700 1 -- 172.21.15.3:0/1521778780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc24074ea0 con 0x7fbc2406b080 2023-10-03T10:19:25.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.383+0000 7fbc0b7fe700 1 -- 172.21.15.3:0/1521778780 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbc18015070 con 0x7fbc2406b080 2023-10-03T10:19:25.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.459+0000 7f27a1e3b700 1 -- 172.21.15.3:0/1057036996 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f277c005590 con 0x7f279c13a3c0 2023-10-03T10:19:25.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.460+0000 7f2783fff700 1 -- 172.21.15.3:0/1057036996 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2794034090 con 0x7f279c13a3c0 2023-10-03T10:19:25.460 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:25.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.463+0000 7f27a1e3b700 1 -- 172.21.15.3:0/1057036996 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f279c13a3c0 msgr2=0x7f279c13a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:25.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.463+0000 7f27a1e3b700 1 --2- 172.21.15.3:0/1057036996 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f279c13a3c0 0x7f279c13a830 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f279400b680 tx=0x7f2794004ad0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.463+0000 7f27a1e3b700 1 -- 172.21.15.3:0/1057036996 shutdown_connections 2023-10-03T10:19:25.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.463+0000 7f27a1e3b700 1 --2- 172.21.15.3:0/1057036996 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f279c13a3c0 0x7f279c13a830 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.463+0000 7f27a1e3b700 1 --2- 172.21.15.3:0/1057036996 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f279c06e570 0x7f279c134a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.463+0000 7f27a1e3b700 1 --2- 172.21.15.3:0/1057036996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f279c06ee80 0x7f279c139e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.463+0000 7f27a1e3b700 1 -- 172.21.15.3:0/1057036996 >> 172.21.15.3:0/1057036996 conn(0x7f279c0a2aa0 msgr2=0x7f279c1373a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:25.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.464+0000 7f27a1e3b700 1 -- 172.21.15.3:0/1057036996 shutdown_connections 2023-10-03T10:19:25.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.464+0000 7f27a1e3b700 1 -- 172.21.15.3:0/1057036996 wait complete. 2023-10-03T10:19:25.478 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:25.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.501+0000 7f2a5949a700 1 -- 172.21.15.3:0/2466447970 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2a5413d1a0 con 0x7f2a5406ee80 2023-10-03T10:19:25.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.502+0000 7f2a3b7fe700 1 -- 172.21.15.3:0/2466447970 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2a5413d1a0 con 0x7f2a5406ee80 2023-10-03T10:19:25.502 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:25.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.504+0000 7f2a5949a700 1 -- 172.21.15.3:0/2466447970 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2a5406ee80 msgr2=0x7f2a54124f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:25.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.504+0000 7f2a5949a700 1 --2- 172.21.15.3:0/2466447970 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2a5406ee80 0x7f2a54124f70 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2a48000f60 tx=0x7f2a4800c730 comp rx=0 tx=0).stop 2023-10-03T10:19:25.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.504+0000 7f2a5949a700 1 -- 172.21.15.3:0/2466447970 shutdown_connections 2023-10-03T10:19:25.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.504+0000 7f2a5949a700 1 --2- 172.21.15.3:0/2466447970 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2a5413dce0 0x7f2a541254b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.504+0000 7f2a5949a700 1 --2- 172.21.15.3:0/2466447970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a5406e570 0x7f2a5413ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.504+0000 7f2a5949a700 1 --2- 172.21.15.3:0/2466447970 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2a5406ee80 0x7f2a54124f70 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.504+0000 7f2a5949a700 1 -- 172.21.15.3:0/2466447970 >> 172.21.15.3:0/2466447970 conn(0x7f2a540a2aa0 msgr2=0x7f2a540a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:25.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.504+0000 7f2a5949a700 1 -- 172.21.15.3:0/2466447970 shutdown_connections 2023-10-03T10:19:25.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.504+0000 7f2a5949a700 1 -- 172.21.15.3:0/2466447970 wait complete. 2023-10-03T10:19:25.520 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:25.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.552+0000 7f8f74f48700 1 -- 172.21.15.3:0/3702089560 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8f700a34f0 con 0x7f8f7009f240 2023-10-03T10:19:25.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.553+0000 7f8f5effd700 1 -- 172.21.15.3:0/3702089560 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8f64035090 con 0x7f8f7009f240 2023-10-03T10:19:25.553 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:25.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.555+0000 7f8f74f48700 1 -- 172.21.15.3:0/3702089560 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8f7009f240 msgr2=0x7f8f701a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:25.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.555+0000 7f8f74f48700 1 --2- 172.21.15.3:0/3702089560 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8f7009f240 0x7f8f701a6230 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009fd0 tx=0x7f8f64004750 comp rx=0 tx=0).stop 2023-10-03T10:19:25.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.555+0000 7f8f74f48700 1 -- 172.21.15.3:0/3702089560 shutdown_connections 2023-10-03T10:19:25.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.555+0000 7f8f74f48700 1 --2- 172.21.15.3:0/3702089560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f7009f650 0x7f8f701a6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.555+0000 7f8f74f48700 1 --2- 172.21.15.3:0/3702089560 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f8f7009e930 0x7f8f701a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.555+0000 7f8f74f48700 1 --2- 172.21.15.3:0/3702089560 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8f7009f240 0x7f8f701a6230 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.555+0000 7f8f74f48700 1 -- 172.21.15.3:0/3702089560 >> 172.21.15.3:0/3702089560 conn(0x7f8f700a4210 msgr2=0x7f8f7009dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:25.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.555+0000 7f8f74f48700 1 -- 172.21.15.3:0/3702089560 shutdown_connections 2023-10-03T10:19:25.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.555+0000 7f8f74f48700 1 -- 172.21.15.3:0/3702089560 wait complete. 2023-10-03T10:19:25.569 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:25.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.601+0000 7fbc2949b700 1 -- 172.21.15.3:0/1521778780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbc2406c360 con 0x7fbc2406b080 2023-10-03T10:19:25.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.601+0000 7fbc0b7fe700 1 -- 172.21.15.3:0/1521778780 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbc18019ba0 con 0x7fbc2406b080 2023-10-03T10:19:25.601 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:25.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.603+0000 7fbc2949b700 1 -- 172.21.15.3:0/1521778780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc2406b080 msgr2=0x7fbc2411c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:25.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.603+0000 7fbc2949b700 1 --2- 172.21.15.3:0/1521778780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc2406b080 0x7fbc2411c9d0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fbc18016000 tx=0x7fbc18004750 comp rx=0 tx=0).stop 2023-10-03T10:19:25.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.603+0000 7fbc2949b700 1 -- 172.21.15.3:0/1521778780 shutdown_connections 2023-10-03T10:19:25.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.603+0000 7fbc2949b700 1 --2- 172.21.15.3:0/1521778780 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbc24131b40 0x7fbc2411cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.603+0000 7fbc2949b700 1 --2- 172.21.15.3:0/1521778780 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbc2406a770 0x7fbc2411c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.603+0000 7fbc2949b700 1 --2- 172.21.15.3:0/1521778780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc2406b080 0x7fbc2411c9d0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:25.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.603+0000 7fbc2949b700 1 -- 172.21.15.3:0/1521778780 >> 172.21.15.3:0/1521778780 conn(0x7fbc24066f50 msgr2=0x7fbc24070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:25.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.603+0000 7fbc2949b700 1 -- 172.21.15.3:0/1521778780 shutdown_connections 2023-10-03T10:19:25.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:25.603+0000 7fbc2949b700 1 -- 172.21.15.3:0/1521778780 wait complete. 2023-10-03T10:19:25.617 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:26.479 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:26.520 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:26.569 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:26.618 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:26.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.630+0000 7fe7ac685700 1 Processor -- start 2023-10-03T10:19:26.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.630+0000 7fe7ac685700 1 -- start start 2023-10-03T10:19:26.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.630+0000 7fe7ac685700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe7a406a770 0x7fe7a409d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:26.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.631+0000 7fe7ac685700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7a409dc30 0x7fe7a409e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:26.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.631+0000 7fe7ac685700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a409a810 con 0x7fe7a413c1e0 2023-10-03T10:19:26.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.631+0000 7fe7ac685700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a406ab40 con 0x7fe7a406a770 2023-10-03T10:19:26.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.631+0000 7fe7ac685700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a409ebc0 con 0x7fe7a409dc30 2023-10-03T10:19:26.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.631+0000 7fe7a9c20700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7a409dc30 0x7fe7a409e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:26.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.631+0000 7fe7a9c20700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7a409dc30 0x7fe7a409e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36622/0 (socket says 172.21.15.3:36622) 2023-10-03T10:19:26.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.631+0000 7fe7a9c20700 1 -- 172.21.15.3:0/2079997252 learned_addr learned my addr 172.21.15.3:0/2079997252 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:26.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.631+0000 7fe7aa421700 1 --2- 172.21.15.3:0/2079997252 >> v2:172.21.15.3:3300/0 conn(0x7fe7a406a770 0x7fe7a409d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:26.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.631+0000 7fe7a9c20700 1 -- 172.21.15.3:0/2079997252 >> v2:172.21.15.3:3300/0 conn(0x7fe7a406a770 msgr2=0x7fe7a409d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:26.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.631+0000 7fe7a9c20700 1 --2- 172.21.15.3:0/2079997252 >> v2:172.21.15.3:3300/0 conn(0x7fe7a406a770 0x7fe7a409d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.631+0000 7fe7a9c20700 1 -- 172.21.15.3:0/2079997252 >> v1:172.21.15.3:6789/0 conn(0x7fe7a413c1e0 legacy=0x7fe7a413e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:26.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.631+0000 7fe7a9c20700 1 -- 172.21.15.3:0/2079997252 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a409e920 con 0x7fe7a409dc30 2023-10-03T10:19:26.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.632+0000 7fe7a9c20700 1 --2- 172.21.15.3:0/2079997252 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7a409dc30 0x7fe7a409e080 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe7940098f0 tx=0x7fe794009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d8d849775c44772 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:26.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.632+0000 7fe7a941f700 1 -- 172.21.15.3:0/2079997252 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79400b8a0 con 0x7fe7a409dc30 2023-10-03T10:19:26.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.632+0000 7fe7a941f700 1 -- 172.21.15.3:0/2079997252 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79400ba00 con 0x7fe7a409dc30 2023-10-03T10:19:26.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.632+0000 7fe7a941f700 1 -- 172.21.15.3:0/2079997252 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79400bd90 con 0x7fe7a409dc30 2023-10-03T10:19:26.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.633+0000 7fe7ac685700 1 -- 172.21.15.3:0/2079997252 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7a409dc30 msgr2=0x7fe7a409e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:26.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.633+0000 7fe7ac685700 1 --2- 172.21.15.3:0/2079997252 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7a409dc30 0x7fe7a409e080 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe7940098f0 tx=0x7fe794009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:26.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.633+0000 7fe7ac685700 1 -- 172.21.15.3:0/2079997252 shutdown_connections 2023-10-03T10:19:26.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.633+0000 7fe7ac685700 1 --2- 172.21.15.3:0/2079997252 >> v2:172.21.15.3:3300/0 conn(0x7fe7a406a770 0x7fe7a409d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.633+0000 7fe7ac685700 1 --2- 172.21.15.3:0/2079997252 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7a409dc30 0x7fe7a409e080 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.633+0000 7fe7ac685700 1 -- 172.21.15.3:0/2079997252 >> 172.21.15.3:0/2079997252 conn(0x7fe7a4066f50 msgr2=0x7fe7a409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:26.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.633+0000 7fe7ac685700 1 -- 172.21.15.3:0/2079997252 shutdown_connections 2023-10-03T10:19:26.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.633+0000 7fe7ac685700 1 -- 172.21.15.3:0/2079997252 wait complete. 2023-10-03T10:19:26.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.634+0000 7fe7ac685700 1 Processor -- start 2023-10-03T10:19:26.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.635+0000 7fe7ac685700 1 -- start start 2023-10-03T10:19:26.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.635+0000 7fe7ac685700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7a406a770 0x7fe7a411b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:26.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.635+0000 7fe7ac685700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7a409dc30 0x7fe7a413a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:26.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.635+0000 7fe7ac685700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7a413c1e0 0x7fe7a413acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:26.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.635+0000 7fe7ac685700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a411d500 con 0x7fe7a406a770 2023-10-03T10:19:26.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.635+0000 7fe7ac685700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a411c0b0 con 0x7fe7a409dc30 2023-10-03T10:19:26.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.635+0000 7fe7ac685700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a411dda0 con 0x7fe7a413c1e0 2023-10-03T10:19:26.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.635+0000 7fe7a9c20700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7a409dc30 0x7fe7a413a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:26.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.635+0000 7fe7a9c20700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7a409dc30 0x7fe7a413a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51728/0 (socket says 172.21.15.3:51728) 2023-10-03T10:19:26.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.635+0000 7fe7a9c20700 1 -- 172.21.15.3:0/3115107948 learned_addr learned my addr 172.21.15.3:0/3115107948 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:26.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.635+0000 7fe7aac22700 1 --2- 172.21.15.3:0/3115107948 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7a413c1e0 0x7fe7a413acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:26.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.636+0000 7fe7aac22700 1 -- 172.21.15.3:0/3115107948 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7a409dc30 msgr2=0x7fe7a413a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:26.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.636+0000 7fe7aac22700 1 --2- 172.21.15.3:0/3115107948 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7a409dc30 0x7fe7a413a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.636+0000 7fe7aac22700 1 -- 172.21.15.3:0/3115107948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7a406a770 msgr2=0x7fe7a411b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:26.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.636+0000 7fe7aac22700 1 --2- 172.21.15.3:0/3115107948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7a406a770 0x7fe7a411b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.636+0000 7fe7aac22700 1 -- 172.21.15.3:0/3115107948 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe794009550 con 0x7fe7a413c1e0 2023-10-03T10:19:26.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.636+0000 7fe7a9c20700 1 --2- 172.21.15.3:0/3115107948 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7a409dc30 0x7fe7a413a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:26.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.637+0000 7fe7aac22700 1 --2- 172.21.15.3:0/3115107948 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7a413c1e0 0x7fe7a413acc0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00cbc0 tx=0x7fe79c00ced0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:26.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.637+0000 7fe79affd700 1 -- 172.21.15.3:0/3115107948 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c003c00 con 0x7fe7a413c1e0 2023-10-03T10:19:26.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.637+0000 7fe79affd700 1 -- 172.21.15.3:0/3115107948 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79c003d60 con 0x7fe7a413c1e0 2023-10-03T10:19:26.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.637+0000 7fe7ac685700 1 -- 172.21.15.3:0/3115107948 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a4071ee0 con 0x7fe7a413c1e0 2023-10-03T10:19:26.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.637+0000 7fe79affd700 1 -- 172.21.15.3:0/3115107948 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c00aaa0 con 0x7fe7a413c1e0 2023-10-03T10:19:26.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.637+0000 7fe7ac685700 1 -- 172.21.15.3:0/3115107948 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a411e810 con 0x7fe7a413c1e0 2023-10-03T10:19:26.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.639+0000 7fe7ac685700 1 -- 172.21.15.3:0/3115107948 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a4074ea0 con 0x7fe7a413c1e0 2023-10-03T10:19:26.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.639+0000 7fe79affd700 1 -- 172.21.15.3:0/3115107948 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe79c00ac00 con 0x7fe7a413c1e0 2023-10-03T10:19:26.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.639+0000 7fe79affd700 1 -- 172.21.15.3:0/3115107948 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe79c040130 con 0x7fe7a413c1e0 2023-10-03T10:19:26.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.644+0000 7fe79affd700 1 -- 172.21.15.3:0/3115107948 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe79c0050d0 con 0x7fe7a413c1e0 2023-10-03T10:19:26.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.672+0000 7fdf957ba700 1 Processor -- start 2023-10-03T10:19:26.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.672+0000 7fdf957ba700 1 -- start start 2023-10-03T10:19:26.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.672+0000 7fdf957ba700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf9009d6d0 0x7fdf9009daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:26.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.673+0000 7fdf957ba700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdf9009dfe0 0x7fdf901335c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:26.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.673+0000 7fdf957ba700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf9009a570 con 0x7fdf90133b00 2023-10-03T10:19:26.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.673+0000 7fdf957ba700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf9009b1c0 con 0x7fdf9009dfe0 2023-10-03T10:19:26.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.673+0000 7fdf957ba700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9009f240 con 0x7fdf9009d6d0 2023-10-03T10:19:26.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.673+0000 7fdf8f7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdf90133b00 0x7fdf9013aa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33156/0 (socket says 172.21.15.3:33156) 2023-10-03T10:19:26.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.673+0000 7fdf8f7fe700 1 -- 172.21.15.3:0/756482963 learned_addr learned my addr 172.21.15.3:0/756482963 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:26.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.673+0000 7fdf8effd700 1 --2- 172.21.15.3:0/756482963 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf9009d6d0 0x7fdf9009daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:26.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.673+0000 7fdf8e7fc700 1 --2- 172.21.15.3:0/756482963 >> v2:172.21.15.3:3300/0 conn(0x7fdf9009dfe0 0x7fdf901335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:26.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.673+0000 7fdf8effd700 1 -- 172.21.15.3:0/756482963 >> v2:172.21.15.3:3300/0 conn(0x7fdf9009dfe0 msgr2=0x7fdf901335c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:26.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.673+0000 7fdf8effd700 1 --2- 172.21.15.3:0/756482963 >> v2:172.21.15.3:3300/0 conn(0x7fdf9009dfe0 0x7fdf901335c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.673+0000 7fdf8effd700 1 -- 172.21.15.3:0/756482963 >> v1:172.21.15.3:6789/0 conn(0x7fdf90133b00 legacy=0x7fdf9013aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:26.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.673+0000 7fdf8effd700 1 -- 172.21.15.3:0/756482963 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9009a2d0 con 0x7fdf9009d6d0 2023-10-03T10:19:26.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.673+0000 7fdf8effd700 1 --2- 172.21.15.3:0/756482963 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf9009d6d0 0x7fdf9009daa0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009840 tx=0x7fdf80009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96dae6879f405ff3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:26.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.674+0000 7fdf8dffb700 1 -- 172.21.15.3:0/756482963 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf80004230 con 0x7fdf9009d6d0 2023-10-03T10:19:26.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.674+0000 7fdf8dffb700 1 -- 172.21.15.3:0/756482963 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf80004390 con 0x7fdf9009d6d0 2023-10-03T10:19:26.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.674+0000 7fdf8dffb700 1 -- 172.21.15.3:0/756482963 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf800036a0 con 0x7fdf9009d6d0 2023-10-03T10:19:26.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.675+0000 7fdf957ba700 1 -- 172.21.15.3:0/756482963 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf9009d6d0 msgr2=0x7fdf9009daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:26.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.675+0000 7fdf957ba700 1 --2- 172.21.15.3:0/756482963 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf9009d6d0 0x7fdf9009daa0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009840 tx=0x7fdf80009b50 comp rx=0 tx=0).stop 2023-10-03T10:19:26.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.675+0000 7fdf957ba700 1 -- 172.21.15.3:0/756482963 shutdown_connections 2023-10-03T10:19:26.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.675+0000 7fdf957ba700 1 --2- 172.21.15.3:0/756482963 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf9009d6d0 0x7fdf9009daa0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.675+0000 7fdf957ba700 1 --2- 172.21.15.3:0/756482963 >> v2:172.21.15.3:3300/0 conn(0x7fdf9009dfe0 0x7fdf901335c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.675+0000 7fdf957ba700 1 -- 172.21.15.3:0/756482963 >> 172.21.15.3:0/756482963 conn(0x7fdf90099c20 msgr2=0x7fdf900a2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:26.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.675+0000 7fdf957ba700 1 -- 172.21.15.3:0/756482963 shutdown_connections 2023-10-03T10:19:26.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.675+0000 7fdf957ba700 1 -- 172.21.15.3:0/756482963 wait complete. 2023-10-03T10:19:26.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.677+0000 7fdf957ba700 1 Processor -- start 2023-10-03T10:19:26.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.677+0000 7fdf957ba700 1 -- start start 2023-10-03T10:19:26.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.678+0000 7fdf957ba700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf9009d6d0 0x7fdf9011e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:26.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.678+0000 7fdf957ba700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf9009dfe0 0x7fdf9011ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:26.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.678+0000 7fdf957ba700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf90133b00 0x7fdf9011f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:26.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.678+0000 7fdf957ba700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf90121d90 con 0x7fdf9009d6d0 2023-10-03T10:19:26.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.678+0000 7fdf957ba700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9012f3a0 con 0x7fdf9009dfe0 2023-10-03T10:19:26.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.678+0000 7fdf957ba700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf90122330 con 0x7fdf90133b00 2023-10-03T10:19:26.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.678+0000 7fdf8effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf9009d6d0 0x7fdf9011e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:26.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.678+0000 7fdf8effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf9009d6d0 0x7fdf9011e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50944/0 (socket says 172.21.15.3:50944) 2023-10-03T10:19:26.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.678+0000 7fdf8effd700 1 -- 172.21.15.3:0/1466830210 learned_addr learned my addr 172.21.15.3:0/1466830210 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:26.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.678+0000 7fdf8e7fc700 1 --2- 172.21.15.3:0/1466830210 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf9009dfe0 0x7fdf9011ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:26.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.678+0000 7fdf8effd700 1 -- 172.21.15.3:0/1466830210 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf90133b00 msgr2=0x7fdf9011f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:26.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.678+0000 7fdf8effd700 1 --2- 172.21.15.3:0/1466830210 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf90133b00 0x7fdf9011f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.678+0000 7fdf8effd700 1 -- 172.21.15.3:0/1466830210 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf9009dfe0 msgr2=0x7fdf9011ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:26.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.678+0000 7fdf8effd700 1 --2- 172.21.15.3:0/1466830210 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf9009dfe0 0x7fdf9011ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.678+0000 7fdf8effd700 1 -- 172.21.15.3:0/1466830210 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf800094a0 con 0x7fdf9009d6d0 2023-10-03T10:19:26.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.678+0000 7fdf8e7fc700 1 --2- 172.21.15.3:0/1466830210 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf9009dfe0 0x7fdf9011ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:26.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.679+0000 7fdf8effd700 1 --2- 172.21.15.3:0/1466830210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf9009d6d0 0x7fdf9011e810 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fdf80000c00 tx=0x7fdf80004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:26.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.679+0000 7fdf777fe700 1 -- 172.21.15.3:0/1466830210 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf80003d80 con 0x7fdf9009d6d0 2023-10-03T10:19:26.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.679+0000 7fdf777fe700 1 -- 172.21.15.3:0/1466830210 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8001f070 con 0x7fdf9009d6d0 2023-10-03T10:19:26.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.679+0000 7fdf957ba700 1 -- 172.21.15.3:0/1466830210 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf901328e0 con 0x7fdf9009d6d0 2023-10-03T10:19:26.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.679+0000 7fdf777fe700 1 -- 172.21.15.3:0/1466830210 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf80011870 con 0x7fdf9009d6d0 2023-10-03T10:19:26.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.679+0000 7fdf957ba700 1 -- 172.21.15.3:0/1466830210 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf90122050 con 0x7fdf9009d6d0 2023-10-03T10:19:26.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.680+0000 7fdf777fe700 1 -- 172.21.15.3:0/1466830210 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdf80021030 con 0x7fdf9009d6d0 2023-10-03T10:19:26.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.680+0000 7fdf957ba700 1 -- 172.21.15.3:0/1466830210 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf9006a9e0 con 0x7fdf9009d6d0 2023-10-03T10:19:26.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.680+0000 7fdf777fe700 1 -- 172.21.15.3:0/1466830210 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fdf80040940 con 0x7fdf9009d6d0 2023-10-03T10:19:26.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.685+0000 7fdf777fe700 1 -- 172.21.15.3:0/1466830210 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdf8001a440 con 0x7fdf9009d6d0 2023-10-03T10:19:26.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.717+0000 7f2108663700 1 Processor -- start 2023-10-03T10:19:26.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.717+0000 7f2108663700 1 -- start start 2023-10-03T10:19:26.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.718+0000 7f2108663700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210009d6f0 0x7f210009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:26.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.718+0000 7f2108663700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210009e000 0x7f21001339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:26.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.718+0000 7f2108663700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f210009b120 con 0x7f210009e410 2023-10-03T10:19:26.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.718+0000 7f2108663700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210006b460 con 0x7f210009e000 2023-10-03T10:19:26.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.718+0000 7f2108663700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f210009a7e0 con 0x7f210009d6f0 2023-10-03T10:19:26.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.718+0000 7f2105bfe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210009e000 0x7f21001339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:26.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.718+0000 7f2105bfe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210009e000 0x7f21001339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51764/0 (socket says 172.21.15.3:51764) 2023-10-03T10:19:26.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.718+0000 7f2106c00700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f210009e410 0x7f2100133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33160/0 (socket says 172.21.15.3:33160) 2023-10-03T10:19:26.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.718+0000 7f2105bfe700 1 -- 172.21.15.3:0/3004625821 learned_addr learned my addr 172.21.15.3:0/3004625821 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:26.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.718+0000 7f21063ff700 1 --2- 172.21.15.3:0/3004625821 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210009d6f0 0x7f210009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:26.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.719+0000 7f2105bfe700 1 -- 172.21.15.3:0/3004625821 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210009d6f0 msgr2=0x7f210009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:26.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.719+0000 7f2105bfe700 1 --2- 172.21.15.3:0/3004625821 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210009d6f0 0x7f210009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.719+0000 7f2105bfe700 1 -- 172.21.15.3:0/3004625821 >> v1:172.21.15.3:6789/0 conn(0x7f210009e410 legacy=0x7f2100133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:26.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.719+0000 7f2105bfe700 1 -- 172.21.15.3:0/3004625821 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210009a500 con 0x7f210009e000 2023-10-03T10:19:26.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.719+0000 7f2105bfe700 1 --2- 172.21.15.3:0/3004625821 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210009e000 0x7f21001339f0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f20f0009920 tx=0x7f20f0009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed627b7783a99cad server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:26.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.719+0000 7f21053fd700 1 -- 172.21.15.3:0/3004625821 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f0011830 con 0x7f210009e000 2023-10-03T10:19:26.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.719+0000 7f21053fd700 1 -- 172.21.15.3:0/3004625821 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f0011990 con 0x7f210009e000 2023-10-03T10:19:26.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.719+0000 7f21053fd700 1 -- 172.21.15.3:0/3004625821 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f0011d20 con 0x7f210009e000 2023-10-03T10:19:26.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.720+0000 7f2108663700 1 -- 172.21.15.3:0/3004625821 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210009e000 msgr2=0x7f21001339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:26.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.720+0000 7f2108663700 1 --2- 172.21.15.3:0/3004625821 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210009e000 0x7f21001339f0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f20f0009920 tx=0x7f20f0009c30 comp rx=0 tx=0).stop 2023-10-03T10:19:26.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.720+0000 7f2108663700 1 -- 172.21.15.3:0/3004625821 shutdown_connections 2023-10-03T10:19:26.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.720+0000 7f2108663700 1 --2- 172.21.15.3:0/3004625821 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210009d6f0 0x7f210009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.720+0000 7f2108663700 1 --2- 172.21.15.3:0/3004625821 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210009e000 0x7f21001339f0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.720+0000 7f2108663700 1 -- 172.21.15.3:0/3004625821 >> 172.21.15.3:0/3004625821 conn(0x7f2100066f50 msgr2=0x7f210009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:26.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.720+0000 7f2108663700 1 -- 172.21.15.3:0/3004625821 shutdown_connections 2023-10-03T10:19:26.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.720+0000 7f2108663700 1 -- 172.21.15.3:0/3004625821 wait complete. 2023-10-03T10:19:26.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.721+0000 7f2108663700 1 Processor -- start 2023-10-03T10:19:26.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.721+0000 7f2108663700 1 -- start start 2023-10-03T10:19:26.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.722+0000 7f2108663700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f210009d6f0 0x7f21001223c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:26.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.722+0000 7f2108663700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210009e000 0x7f2100122900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:26.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.722+0000 7f2108663700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210009e410 0x7f2100122e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:26.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.722+0000 7f2108663700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21000a2cb0 con 0x7f210009d6f0 2023-10-03T10:19:26.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.722+0000 7f2108663700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2100125c00 con 0x7f210009e410 2023-10-03T10:19:26.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.722+0000 7f2108663700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2100125e70 con 0x7f210009e000 2023-10-03T10:19:26.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.722+0000 7f2105bfe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210009e000 0x7f2100122900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:26.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.722+0000 7f2105bfe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210009e000 0x7f2100122900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36652/0 (socket says 172.21.15.3:36652) 2023-10-03T10:19:26.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.722+0000 7f2105bfe700 1 -- 172.21.15.3:0/516604446 learned_addr learned my addr 172.21.15.3:0/516604446 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:26.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.722+0000 7f21063ff700 1 --2- 172.21.15.3:0/516604446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f210009d6f0 0x7f21001223c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:26.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.722+0000 7f2106c00700 1 --2- 172.21.15.3:0/516604446 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210009e410 0x7f2100122e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:26.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.722+0000 7f2105bfe700 1 -- 172.21.15.3:0/516604446 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210009e410 msgr2=0x7f2100122e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:26.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.722+0000 7f2105bfe700 1 --2- 172.21.15.3:0/516604446 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210009e410 0x7f2100122e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.722+0000 7f2105bfe700 1 -- 172.21.15.3:0/516604446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f210009d6f0 msgr2=0x7f21001223c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:26.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.722+0000 7f2105bfe700 1 --2- 172.21.15.3:0/516604446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f210009d6f0 0x7f21001223c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.722+0000 7f2105bfe700 1 -- 172.21.15.3:0/516604446 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f0009550 con 0x7f210009e000 2023-10-03T10:19:26.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.723+0000 7f2105bfe700 1 --2- 172.21.15.3:0/516604446 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210009e000 0x7f2100122900 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f20f0000c00 tx=0x7f20f0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:26.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.723+0000 7f20f6ffd700 1 -- 172.21.15.3:0/516604446 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f0004450 con 0x7f210009e000 2023-10-03T10:19:26.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.723+0000 7f20f6ffd700 1 -- 172.21.15.3:0/516604446 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f00045b0 con 0x7f210009e000 2023-10-03T10:19:26.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.723+0000 7f2108663700 1 -- 172.21.15.3:0/516604446 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21001258d0 con 0x7f210009e000 2023-10-03T10:19:26.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.723+0000 7f20f6ffd700 1 -- 172.21.15.3:0/516604446 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f00038a0 con 0x7f210009e000 2023-10-03T10:19:26.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.723+0000 7f2108663700 1 -- 172.21.15.3:0/516604446 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2100123790 con 0x7f210009e000 2023-10-03T10:19:26.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.724+0000 7f20f6ffd700 1 -- 172.21.15.3:0/516604446 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f20f0003a00 con 0x7f210009e000 2023-10-03T10:19:26.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.724+0000 7f2108663700 1 -- 172.21.15.3:0/516604446 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2100125a60 con 0x7f210009e000 2023-10-03T10:19:26.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.724+0000 7f20f6ffd700 1 -- 172.21.15.3:0/516604446 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f20f0021030 con 0x7f210009e000 2023-10-03T10:19:26.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.734+0000 7f20f6ffd700 1 -- 172.21.15.3:0/516604446 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2100125a60 con 0x7f210009e000 2023-10-03T10:19:26.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.773+0000 7fa6982a0700 1 Processor -- start 2023-10-03T10:19:26.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.773+0000 7fa6982a0700 1 -- start start 2023-10-03T10:19:26.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.774+0000 7fa6982a0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa690131630 0x7fa690131a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:26.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.774+0000 7fa6982a0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa69006eba0 0x7fa690139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:26.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.774+0000 7fa6982a0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa690071a00 con 0x7fa690131f40 2023-10-03T10:19:26.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.774+0000 7fa6982a0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa690131430 con 0x7fa69006eba0 2023-10-03T10:19:26.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.774+0000 7fa6982a0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa69009a510 con 0x7fa690131630 2023-10-03T10:19:26.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.774+0000 7fa69603c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa690131630 0x7fa690131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:26.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.774+0000 7fa69603c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa690131630 0x7fa690131a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36654/0 (socket says 172.21.15.3:36654) 2023-10-03T10:19:26.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.774+0000 7fa69603c700 1 -- 172.21.15.3:0/2265379063 learned_addr learned my addr 172.21.15.3:0/2265379063 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:26.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.774+0000 7fa69683d700 1 --2- 172.21.15.3:0/2265379063 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa69006eba0 0x7fa690139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:26.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.775+0000 7fa69603c700 1 -- 172.21.15.3:0/2265379063 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa69006eba0 msgr2=0x7fa690139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:26.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.775+0000 7fa69603c700 1 --2- 172.21.15.3:0/2265379063 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa69006eba0 0x7fa690139660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.775+0000 7fa69603c700 1 -- 172.21.15.3:0/2265379063 >> v1:172.21.15.3:6789/0 conn(0x7fa690131f40 legacy=0x7fa69006e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:26.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.775+0000 7fa69603c700 1 -- 172.21.15.3:0/2265379063 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6900721c0 con 0x7fa690131630 2023-10-03T10:19:26.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.775+0000 7fa69603c700 1 --2- 172.21.15.3:0/2265379063 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa690131630 0x7fa690131a00 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa68c009920 tx=0x7fa68c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f89e84c6fbe16ae server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:26.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.775+0000 7fa69503a700 1 -- 172.21.15.3:0/2265379063 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c011830 con 0x7fa690131630 2023-10-03T10:19:26.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.775+0000 7fa69503a700 1 -- 172.21.15.3:0/2265379063 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68c011990 con 0x7fa690131630 2023-10-03T10:19:26.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.775+0000 7fa69503a700 1 -- 172.21.15.3:0/2265379063 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c011d20 con 0x7fa690131630 2023-10-03T10:19:26.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.776+0000 7fa6982a0700 1 -- 172.21.15.3:0/2265379063 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa690131630 msgr2=0x7fa690131a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:26.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.776+0000 7fa6982a0700 1 --2- 172.21.15.3:0/2265379063 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa690131630 0x7fa690131a00 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa68c009920 tx=0x7fa68c009c30 comp rx=0 tx=0).stop 2023-10-03T10:19:26.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.776+0000 7fa6982a0700 1 -- 172.21.15.3:0/2265379063 shutdown_connections 2023-10-03T10:19:26.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.776+0000 7fa6982a0700 1 --2- 172.21.15.3:0/2265379063 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa69006eba0 0x7fa690139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.776+0000 7fa6982a0700 1 --2- 172.21.15.3:0/2265379063 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa690131630 0x7fa690131a00 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.776+0000 7fa6982a0700 1 -- 172.21.15.3:0/2265379063 >> 172.21.15.3:0/2265379063 conn(0x7fa6900a2aa0 msgr2=0x7fa6900a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:26.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.776+0000 7fa6982a0700 1 -- 172.21.15.3:0/2265379063 shutdown_connections 2023-10-03T10:19:26.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.777+0000 7fa6982a0700 1 -- 172.21.15.3:0/2265379063 wait complete. 2023-10-03T10:19:26.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.778+0000 7fa6982a0700 1 Processor -- start 2023-10-03T10:19:26.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.778+0000 7fa6982a0700 1 -- start start 2023-10-03T10:19:26.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.779+0000 7fa6982a0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa69006eba0 0x7fa69012e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:26.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.779+0000 7fa6982a0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa690131630 0x7fa6901285b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:26.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.779+0000 7fa6982a0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa690131f40 0x7fa690128af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:26.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.779+0000 7fa6982a0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69009e580 con 0x7fa690131630 2023-10-03T10:19:26.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.779+0000 7fa6982a0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa690129790 con 0x7fa690131f40 2023-10-03T10:19:26.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.779+0000 7fa6982a0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa69012ab50 con 0x7fa69006eba0 2023-10-03T10:19:26.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.779+0000 7fa69603c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa69006eba0 0x7fa69012e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:26.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.779+0000 7fa69603c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa69006eba0 0x7fa69012e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36658/0 (socket says 172.21.15.3:36658) 2023-10-03T10:19:26.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.779+0000 7fa69603c700 1 -- 172.21.15.3:0/458537080 learned_addr learned my addr 172.21.15.3:0/458537080 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:26.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.779+0000 7fa69583b700 1 --2- 172.21.15.3:0/458537080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa690131630 0x7fa6901285b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:26.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.779+0000 7fa69683d700 1 --2- 172.21.15.3:0/458537080 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa690131f40 0x7fa690128af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:26.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.779+0000 7fa69603c700 1 -- 172.21.15.3:0/458537080 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa690131f40 msgr2=0x7fa690128af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:26.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.779+0000 7fa69603c700 1 --2- 172.21.15.3:0/458537080 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa690131f40 0x7fa690128af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.779+0000 7fa69603c700 1 -- 172.21.15.3:0/458537080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa690131630 msgr2=0x7fa6901285b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:26.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.779+0000 7fa69603c700 1 --2- 172.21.15.3:0/458537080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa690131630 0x7fa6901285b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.779+0000 7fa69603c700 1 -- 172.21.15.3:0/458537080 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa68c009550 con 0x7fa69006eba0 2023-10-03T10:19:26.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.779+0000 7fa69683d700 1 --2- 172.21.15.3:0/458537080 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa690131f40 0x7fa690128af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:26.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.780+0000 7fa69603c700 1 --2- 172.21.15.3:0/458537080 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa69006eba0 0x7fa69012e930 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa68c000c00 tx=0x7fa68c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:26.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.780+0000 7fa686ffd700 1 -- 172.21.15.3:0/458537080 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c004510 con 0x7fa69006eba0 2023-10-03T10:19:26.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.780+0000 7fa686ffd700 1 -- 172.21.15.3:0/458537080 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68c0036a0 con 0x7fa69006eba0 2023-10-03T10:19:26.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.780+0000 7fa686ffd700 1 -- 172.21.15.3:0/458537080 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c003a10 con 0x7fa69006eba0 2023-10-03T10:19:26.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.780+0000 7fa6982a0700 1 -- 172.21.15.3:0/458537080 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa690099f20 con 0x7fa69006eba0 2023-10-03T10:19:26.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.780+0000 7fa6982a0700 1 -- 172.21.15.3:0/458537080 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa690129e50 con 0x7fa69006eba0 2023-10-03T10:19:26.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.781+0000 7fa686ffd700 1 -- 172.21.15.3:0/458537080 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa68c003b70 con 0x7fa69006eba0 2023-10-03T10:19:26.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.781+0000 7fa686ffd700 1 -- 172.21.15.3:0/458537080 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa68c021030 con 0x7fa69006eba0 2023-10-03T10:19:26.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.781+0000 7fa6982a0700 1 -- 172.21.15.3:0/458537080 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa69012af30 con 0x7fa69006eba0 2023-10-03T10:19:26.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.786+0000 7fa686ffd700 1 -- 172.21.15.3:0/458537080 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa68c015080 con 0x7fa69006eba0 2023-10-03T10:19:26.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.863+0000 7fe7ac685700 1 -- 172.21.15.3:0/3115107948 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe7a409f1a0 con 0x7fe7a413c1e0 2023-10-03T10:19:26.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.864+0000 7fe79affd700 1 -- 172.21.15.3:0/3115107948 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe79c011380 con 0x7fe7a413c1e0 2023-10-03T10:19:26.864 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:26.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.867+0000 7fe7ac685700 1 -- 172.21.15.3:0/3115107948 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7a413c1e0 msgr2=0x7fe7a413acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:26.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.867+0000 7fe7ac685700 1 --2- 172.21.15.3:0/3115107948 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7a413c1e0 0x7fe7a413acc0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00cbc0 tx=0x7fe79c00ced0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.867+0000 7fe7ac685700 1 -- 172.21.15.3:0/3115107948 shutdown_connections 2023-10-03T10:19:26.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.867+0000 7fe7ac685700 1 --2- 172.21.15.3:0/3115107948 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe7a413c1e0 0x7fe7a413acc0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.867+0000 7fe7ac685700 1 --2- 172.21.15.3:0/3115107948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7a406a770 0x7fe7a411b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.867+0000 7fe7ac685700 1 --2- 172.21.15.3:0/3115107948 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7a409dc30 0x7fe7a413a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.867+0000 7fe7ac685700 1 -- 172.21.15.3:0/3115107948 >> 172.21.15.3:0/3115107948 conn(0x7fe7a4066f50 msgr2=0x7fe7a40a1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:26.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.868+0000 7fe7ac685700 1 -- 172.21.15.3:0/3115107948 shutdown_connections 2023-10-03T10:19:26.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.868+0000 7fe7ac685700 1 -- 172.21.15.3:0/3115107948 wait complete. 2023-10-03T10:19:26.887 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:26.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.901+0000 7fdf957ba700 1 -- 172.21.15.3:0/1466830210 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdf9006a9e0 con 0x7fdf9009d6d0 2023-10-03T10:19:26.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.902+0000 7fdf777fe700 1 -- 172.21.15.3:0/1466830210 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdf80023bc0 con 0x7fdf9009d6d0 2023-10-03T10:19:26.902 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:26.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.904+0000 7fdf957ba700 1 -- 172.21.15.3:0/1466830210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf9009d6d0 msgr2=0x7fdf9011e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:26.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.904+0000 7fdf957ba700 1 --2- 172.21.15.3:0/1466830210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf9009d6d0 0x7fdf9011e810 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fdf80000c00 tx=0x7fdf80004750 comp rx=0 tx=0).stop 2023-10-03T10:19:26.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.904+0000 7fdf957ba700 1 -- 172.21.15.3:0/1466830210 shutdown_connections 2023-10-03T10:19:26.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.904+0000 7fdf957ba700 1 --2- 172.21.15.3:0/1466830210 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf90133b00 0x7fdf9011f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.904+0000 7fdf957ba700 1 --2- 172.21.15.3:0/1466830210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf9009d6d0 0x7fdf9011e810 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.904+0000 7fdf957ba700 1 --2- 172.21.15.3:0/1466830210 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf9009dfe0 0x7fdf9011ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.904+0000 7fdf957ba700 1 -- 172.21.15.3:0/1466830210 >> 172.21.15.3:0/1466830210 conn(0x7fdf90099c20 msgr2=0x7fdf900a0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:26.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.904+0000 7fdf957ba700 1 -- 172.21.15.3:0/1466830210 shutdown_connections 2023-10-03T10:19:26.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.904+0000 7fdf957ba700 1 -- 172.21.15.3:0/1466830210 wait complete. 2023-10-03T10:19:26.918 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:26.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.956+0000 7f2108663700 1 -- 172.21.15.3:0/516604446 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2100125a60 con 0x7f210009e000 2023-10-03T10:19:26.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.956+0000 7f20f6ffd700 1 -- 172.21.15.3:0/516604446 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2100125a60 con 0x7f210009e000 2023-10-03T10:19:26.956 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:26.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.958+0000 7f2108663700 1 -- 172.21.15.3:0/516604446 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210009e000 msgr2=0x7f2100122900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:26.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.958+0000 7f2108663700 1 --2- 172.21.15.3:0/516604446 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210009e000 0x7f2100122900 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f20f0000c00 tx=0x7f20f0004750 comp rx=0 tx=0).stop 2023-10-03T10:19:26.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.958+0000 7f2108663700 1 -- 172.21.15.3:0/516604446 shutdown_connections 2023-10-03T10:19:26.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.958+0000 7f2108663700 1 --2- 172.21.15.3:0/516604446 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f210009e410 0x7f2100122e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.958+0000 7f2108663700 1 --2- 172.21.15.3:0/516604446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f210009d6f0 0x7f21001223c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.958+0000 7f2108663700 1 --2- 172.21.15.3:0/516604446 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f210009e000 0x7f2100122900 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.959+0000 7f2108663700 1 -- 172.21.15.3:0/516604446 >> 172.21.15.3:0/516604446 conn(0x7f2100066f50 msgr2=0x7f21000a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:26.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.959+0000 7f2108663700 1 -- 172.21.15.3:0/516604446 shutdown_connections 2023-10-03T10:19:26.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.959+0000 7f2108663700 1 -- 172.21.15.3:0/516604446 wait complete. 2023-10-03T10:19:26.973 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:26.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.995+0000 7fa6982a0700 1 -- 172.21.15.3:0/458537080 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa690074ea0 con 0x7fa69006eba0 2023-10-03T10:19:26.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.995+0000 7fa686ffd700 1 -- 172.21.15.3:0/458537080 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa68c02b020 con 0x7fa69006eba0 2023-10-03T10:19:26.998 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:26.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.997+0000 7fa6982a0700 1 -- 172.21.15.3:0/458537080 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa69006eba0 msgr2=0x7fa69012e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:26.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.997+0000 7fa6982a0700 1 --2- 172.21.15.3:0/458537080 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa69006eba0 0x7fa69012e930 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa68c000c00 tx=0x7fa68c004750 comp rx=0 tx=0).stop 2023-10-03T10:19:26.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.997+0000 7fa6982a0700 1 -- 172.21.15.3:0/458537080 shutdown_connections 2023-10-03T10:19:26.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.997+0000 7fa6982a0700 1 --2- 172.21.15.3:0/458537080 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa690131f40 0x7fa690128af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.997+0000 7fa6982a0700 1 --2- 172.21.15.3:0/458537080 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa69006eba0 0x7fa69012e930 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.997+0000 7fa6982a0700 1 --2- 172.21.15.3:0/458537080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa690131630 0x7fa6901285b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:26.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.997+0000 7fa6982a0700 1 -- 172.21.15.3:0/458537080 >> 172.21.15.3:0/458537080 conn(0x7fa6900a2aa0 msgr2=0x7fa690127270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:26.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.997+0000 7fa6982a0700 1 -- 172.21.15.3:0/458537080 shutdown_connections 2023-10-03T10:19:26.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:26.997+0000 7fa6982a0700 1 -- 172.21.15.3:0/458537080 wait complete. 2023-10-03T10:19:27.011 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:27.888 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:27.919 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:27.974 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:28.012 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:28.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.042+0000 7fe5d86a6700 1 Processor -- start 2023-10-03T10:19:28.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.042+0000 7fe5d86a6700 1 -- start start 2023-10-03T10:19:28.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.043+0000 7fe5d86a6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5d0130670 0x7fe5d0130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:28.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.043+0000 7fe5d86a6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5d006f680 0x7fe5d006fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:28.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.043+0000 7fe5d86a6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe5d006ffd0 0x7fe5d0137530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:28.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.043+0000 7fe5d86a6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d009a350 con 0x7fe5d006ffd0 2023-10-03T10:19:28.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.043+0000 7fe5d86a6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d0130260 con 0x7fe5d006f680 2023-10-03T10:19:28.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.043+0000 7fe5d86a6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d0071ff0 con 0x7fe5d0130670 2023-10-03T10:19:28.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.044+0000 7fe5d6442700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5d0130670 0x7fe5d0130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:28.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.044+0000 7fe5d6442700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5d0130670 0x7fe5d0130a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36670/0 (socket says 172.21.15.3:36670) 2023-10-03T10:19:28.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.044+0000 7fe5d6442700 1 -- 172.21.15.3:0/2302613840 learned_addr learned my addr 172.21.15.3:0/2302613840 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:28.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.044+0000 7fe5d6c43700 1 --2- 172.21.15.3:0/2302613840 >> v2:172.21.15.3:3300/0 conn(0x7fe5d006ffd0 0x7fe5d0137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:28.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.044+0000 7fe5d5c41700 1 --2- 172.21.15.3:0/2302613840 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5d006f680 0x7fe5d006fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:28.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.044+0000 7fe5d5c41700 1 -- 172.21.15.3:0/2302613840 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5d0130670 msgr2=0x7fe5d0130a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:28.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.044+0000 7fe5d5c41700 1 --2- 172.21.15.3:0/2302613840 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5d0130670 0x7fe5d0130a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.044+0000 7fe5d5c41700 1 -- 172.21.15.3:0/2302613840 >> v2:172.21.15.3:3300/0 conn(0x7fe5d006ffd0 msgr2=0x7fe5d0137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:28.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.044+0000 7fe5d5c41700 1 --2- 172.21.15.3:0/2302613840 >> v2:172.21.15.3:3300/0 conn(0x7fe5d006ffd0 0x7fe5d0137530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.044+0000 7fe5d5c41700 1 -- 172.21.15.3:0/2302613840 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d0070470 con 0x7fe5d006f680 2023-10-03T10:19:28.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.044+0000 7fe5d6c43700 1 --2- 172.21.15.3:0/2302613840 >> v2:172.21.15.3:3300/0 conn(0x7fe5d006ffd0 0x7fe5d0137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:28.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.044+0000 7fe5d5c41700 1 --2- 172.21.15.3:0/2302613840 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5d006f680 0x7fe5d006fa90 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009870 tx=0x7fe5c0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c20a3b97d37717d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:28.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.045+0000 7fe5d5440700 1 -- 172.21.15.3:0/2302613840 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0011830 con 0x7fe5d006f680 2023-10-03T10:19:28.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.045+0000 7fe5d5440700 1 -- 172.21.15.3:0/2302613840 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c0011990 con 0x7fe5d006f680 2023-10-03T10:19:28.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.045+0000 7fe5d5440700 1 -- 172.21.15.3:0/2302613840 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0011d20 con 0x7fe5d006f680 2023-10-03T10:19:28.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.046+0000 7fe5d86a6700 1 -- 172.21.15.3:0/2302613840 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5d006f680 msgr2=0x7fe5d006fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:28.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.046+0000 7fe5d86a6700 1 --2- 172.21.15.3:0/2302613840 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5d006f680 0x7fe5d006fa90 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009870 tx=0x7fe5c0009b80 comp rx=0 tx=0).stop 2023-10-03T10:19:28.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.046+0000 7fe5d86a6700 1 -- 172.21.15.3:0/2302613840 shutdown_connections 2023-10-03T10:19:28.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.046+0000 7fe5d86a6700 1 --2- 172.21.15.3:0/2302613840 >> v2:172.21.15.3:3300/0 conn(0x7fe5d006ffd0 0x7fe5d0137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.046+0000 7fe5d86a6700 1 --2- 172.21.15.3:0/2302613840 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5d0130670 0x7fe5d0130a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.046+0000 7fe5d86a6700 1 --2- 172.21.15.3:0/2302613840 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5d006f680 0x7fe5d006fa90 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.046+0000 7fe5d86a6700 1 -- 172.21.15.3:0/2302613840 >> 172.21.15.3:0/2302613840 conn(0x7fe5d00a2aa0 msgr2=0x7fe5d00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:28.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.046+0000 7fe5d86a6700 1 -- 172.21.15.3:0/2302613840 shutdown_connections 2023-10-03T10:19:28.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.046+0000 7fe5d86a6700 1 -- 172.21.15.3:0/2302613840 wait complete. 2023-10-03T10:19:28.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.048+0000 7fe5d86a6700 1 Processor -- start 2023-10-03T10:19:28.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.048+0000 7fe5d86a6700 1 -- start start 2023-10-03T10:19:28.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.048+0000 7fe5d86a6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5d006f680 0x7fe5d01214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:28.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.048+0000 7fe5d86a6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5d006ffd0 0x7fe5d0121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:28.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.048+0000 7fe5d86a6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5d0130670 0x7fe5d0121f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:28.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.048+0000 7fe5d86a6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d0123d10 con 0x7fe5d006ffd0 2023-10-03T10:19:28.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.048+0000 7fe5d86a6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d0123a50 con 0x7fe5d0130670 2023-10-03T10:19:28.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.048+0000 7fe5d86a6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d009dd90 con 0x7fe5d006f680 2023-10-03T10:19:28.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.048+0000 7fe5d6442700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5d006f680 0x7fe5d01214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:28.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.048+0000 7fe5d6c43700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5d0130670 0x7fe5d0121f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:28.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.048+0000 7fe5d6442700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5d006f680 0x7fe5d01214e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36680/0 (socket says 172.21.15.3:36680) 2023-10-03T10:19:28.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.048+0000 7fe5d6442700 1 -- 172.21.15.3:0/2522660710 learned_addr learned my addr 172.21.15.3:0/2522660710 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:28.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.048+0000 7fe5d6c43700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5d0130670 0x7fe5d0121f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51810/0 (socket says 172.21.15.3:51810) 2023-10-03T10:19:28.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.049+0000 7fe5d6442700 1 -- 172.21.15.3:0/2522660710 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5d0130670 msgr2=0x7fe5d0121f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:28.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.049+0000 7fe5d6442700 1 --2- 172.21.15.3:0/2522660710 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5d0130670 0x7fe5d0121f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.049+0000 7fe5d5c41700 1 --2- 172.21.15.3:0/2522660710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5d006ffd0 0x7fe5d0121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:28.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.049+0000 7fe5d6442700 1 -- 172.21.15.3:0/2522660710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5d006ffd0 msgr2=0x7fe5d0121a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:28.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.049+0000 7fe5d6442700 1 --2- 172.21.15.3:0/2522660710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5d006ffd0 0x7fe5d0121a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.049+0000 7fe5d6442700 1 -- 172.21.15.3:0/2522660710 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c8009a70 con 0x7fe5d006f680 2023-10-03T10:19:28.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.049+0000 7fe5d6c43700 1 --2- 172.21.15.3:0/2522660710 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5d0130670 0x7fe5d0121f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:28.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.049+0000 7fe5d5c41700 1 --2- 172.21.15.3:0/2522660710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5d006ffd0 0x7fe5d0121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:19:28.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.049+0000 7fe5d6442700 1 --2- 172.21.15.3:0/2522660710 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5d006f680 0x7fe5d01214e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00e910 tx=0x7fe5cc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:28.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.049+0000 7fe5c6ffd700 1 -- 172.21.15.3:0/2522660710 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc0096a0 con 0x7fe5d006f680 2023-10-03T10:19:28.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.050+0000 7fe5c6ffd700 1 -- 172.21.15.3:0/2522660710 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5cc00ee50 con 0x7fe5d006f680 2023-10-03T10:19:28.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.050+0000 7fe5c6ffd700 1 -- 172.21.15.3:0/2522660710 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc00ca90 con 0x7fe5d006f680 2023-10-03T10:19:28.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.050+0000 7fe5d86a6700 1 -- 172.21.15.3:0/2522660710 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c00094a0 con 0x7fe5d006f680 2023-10-03T10:19:28.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.050+0000 7fe5d86a6700 1 -- 172.21.15.3:0/2522660710 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d009e4f0 con 0x7fe5d006f680 2023-10-03T10:19:28.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.051+0000 7fe5d86a6700 1 -- 172.21.15.3:0/2522660710 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d0074ea0 con 0x7fe5d006f680 2023-10-03T10:19:28.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.051+0000 7fe5c6ffd700 1 -- 172.21.15.3:0/2522660710 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe5cc00cbf0 con 0x7fe5d006f680 2023-10-03T10:19:28.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.051+0000 7fe5c6ffd700 1 -- 172.21.15.3:0/2522660710 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe5cc03f650 con 0x7fe5d006f680 2023-10-03T10:19:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.057+0000 7fe5c6ffd700 1 -- 172.21.15.3:0/2522660710 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe5cc013470 con 0x7fe5d006f680 2023-10-03T10:19:28.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.084+0000 7f4257718700 1 Processor -- start 2023-10-03T10:19:28.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.084+0000 7f4257718700 1 -- start start 2023-10-03T10:19:28.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.085+0000 7f4257718700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f42501338a0 0x7f4250133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:28.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.085+0000 7f4257718700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f425009de50 0x7f425009e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:28.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.085+0000 7f4257718700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f425006ef30 con 0x7f425009e760 2023-10-03T10:19:28.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.085+0000 7f4257718700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425006fef0 con 0x7f425009de50 2023-10-03T10:19:28.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.085+0000 7f4257718700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f42501334b0 con 0x7f42501338a0 2023-10-03T10:19:28.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.085+0000 7f4254cb3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f425009de50 0x7f425009e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:28.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.085+0000 7f42554b4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f42501338a0 0x7f4250133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:28.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.085+0000 7f4254cb3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f425009de50 0x7f425009e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51818/0 (socket says 172.21.15.3:51818) 2023-10-03T10:19:28.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.085+0000 7f4254cb3700 1 -- 172.21.15.3:0/4200173320 learned_addr learned my addr 172.21.15.3:0/4200173320 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:28.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.085+0000 7f4254cb3700 1 -- 172.21.15.3:0/4200173320 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f42501338a0 msgr2=0x7f4250133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:28.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.085+0000 7f4254cb3700 1 --2- 172.21.15.3:0/4200173320 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f42501338a0 0x7f4250133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.085+0000 7f4254cb3700 1 -- 172.21.15.3:0/4200173320 >> v1:172.21.15.3:6789/0 conn(0x7f425009e760 legacy=0x7f425013a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:28.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.085+0000 7f4254cb3700 1 -- 172.21.15.3:0/4200173320 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425006fc30 con 0x7f425009de50 2023-10-03T10:19:28.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.086+0000 7f4254cb3700 1 --2- 172.21.15.3:0/4200173320 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f425009de50 0x7f425009e220 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f4240009920 tx=0x7f4240009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d6e1cbaba89136e server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:28.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.086+0000 7f4247fff700 1 -- 172.21.15.3:0/4200173320 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4240011830 con 0x7f425009de50 2023-10-03T10:19:28.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.086+0000 7f4247fff700 1 -- 172.21.15.3:0/4200173320 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4240011990 con 0x7f425009de50 2023-10-03T10:19:28.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.086+0000 7f4247fff700 1 -- 172.21.15.3:0/4200173320 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4240011d20 con 0x7f425009de50 2023-10-03T10:19:28.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.087+0000 7f4257718700 1 -- 172.21.15.3:0/4200173320 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f425009de50 msgr2=0x7f425009e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:28.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.087+0000 7f4257718700 1 --2- 172.21.15.3:0/4200173320 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f425009de50 0x7f425009e220 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f4240009920 tx=0x7f4240009c30 comp rx=0 tx=0).stop 2023-10-03T10:19:28.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.087+0000 7f4257718700 1 -- 172.21.15.3:0/4200173320 shutdown_connections 2023-10-03T10:19:28.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.087+0000 7f4257718700 1 --2- 172.21.15.3:0/4200173320 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f42501338a0 0x7f4250133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.087+0000 7f4257718700 1 --2- 172.21.15.3:0/4200173320 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f425009de50 0x7f425009e220 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.087+0000 7f4257718700 1 -- 172.21.15.3:0/4200173320 >> 172.21.15.3:0/4200173320 conn(0x7f42500a2aa0 msgr2=0x7f42500a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:28.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.087+0000 7f4257718700 1 -- 172.21.15.3:0/4200173320 shutdown_connections 2023-10-03T10:19:28.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.087+0000 7f4257718700 1 -- 172.21.15.3:0/4200173320 wait complete. 2023-10-03T10:19:28.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.088+0000 7f4257718700 1 Processor -- start 2023-10-03T10:19:28.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.088+0000 7f4257718700 1 -- start start 2023-10-03T10:19:28.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.089+0000 7f4257718700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f425009de50 0x7f425011fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:28.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.089+0000 7f4257718700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f425009e760 0x7f425011ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:28.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.089+0000 7f4257718700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f42501338a0 0x7f4250120490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:28.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.089+0000 7f4257718700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42500a1270 con 0x7f425009de50 2023-10-03T10:19:28.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.089+0000 7f4257718700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4250121730 con 0x7f425009e760 2023-10-03T10:19:28.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.089+0000 7f4257718700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f425006f640 con 0x7f42501338a0 2023-10-03T10:19:28.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.089+0000 7f42554b4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f425009de50 0x7f425011fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:28.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.089+0000 7f42554b4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f425009de50 0x7f425011fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50978/0 (socket says 172.21.15.3:50978) 2023-10-03T10:19:28.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.089+0000 7f42554b4700 1 -- 172.21.15.3:0/3101707584 learned_addr learned my addr 172.21.15.3:0/3101707584 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:28.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.089+0000 7f4255cb5700 1 --2- 172.21.15.3:0/3101707584 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f42501338a0 0x7f4250120490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:28.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.089+0000 7f4254cb3700 1 --2- 172.21.15.3:0/3101707584 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f425009e760 0x7f425011ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:28.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.089+0000 7f42554b4700 1 -- 172.21.15.3:0/3101707584 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f42501338a0 msgr2=0x7f4250120490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:28.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.089+0000 7f42554b4700 1 --2- 172.21.15.3:0/3101707584 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f42501338a0 0x7f4250120490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.089+0000 7f42554b4700 1 -- 172.21.15.3:0/3101707584 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f425009e760 msgr2=0x7f425011ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:28.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.089+0000 7f42554b4700 1 --2- 172.21.15.3:0/3101707584 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f425009e760 0x7f425011ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.089+0000 7f42554b4700 1 -- 172.21.15.3:0/3101707584 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4240009550 con 0x7f425009de50 2023-10-03T10:19:28.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.089+0000 7f4254cb3700 1 --2- 172.21.15.3:0/3101707584 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f425009e760 0x7f425011ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:19:28.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.090+0000 7f42554b4700 1 --2- 172.21.15.3:0/3101707584 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f425009de50 0x7f425011fa10 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f424c00e9b0 tx=0x7f424c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:28.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.090+0000 7f4245ffb700 1 -- 172.21.15.3:0/3101707584 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c009710 con 0x7f425009de50 2023-10-03T10:19:28.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.090+0000 7f4245ffb700 1 -- 172.21.15.3:0/3101707584 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424c00c850 con 0x7f425009de50 2023-10-03T10:19:28.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.090+0000 7f4257718700 1 -- 172.21.15.3:0/3101707584 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4250122f90 con 0x7f425009de50 2023-10-03T10:19:28.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.090+0000 7f4245ffb700 1 -- 172.21.15.3:0/3101707584 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c00cbc0 con 0x7f425009de50 2023-10-03T10:19:28.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.090+0000 7f4257718700 1 -- 172.21.15.3:0/3101707584 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4250123cc0 con 0x7f425009de50 2023-10-03T10:19:28.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.091+0000 7f4245ffb700 1 -- 172.21.15.3:0/3101707584 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f424c00cd20 con 0x7f425009de50 2023-10-03T10:19:28.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.092+0000 7f4245ffb700 1 -- 172.21.15.3:0/3101707584 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f424c03f810 con 0x7f425009de50 2023-10-03T10:19:28.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.092+0000 7f4257718700 1 -- 172.21.15.3:0/3101707584 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4250123120 con 0x7f425009de50 2023-10-03T10:19:28.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.097+0000 7f4245ffb700 1 -- 172.21.15.3:0/3101707584 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4250123120 con 0x7f425009de50 2023-10-03T10:19:28.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.132+0000 7fcb8f60d700 1 Processor -- start 2023-10-03T10:19:28.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.132+0000 7fcb8f60d700 1 -- start start 2023-10-03T10:19:28.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.133+0000 7fcb8f60d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcb8809d6f0 0x7fcb8809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:28.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.133+0000 7fcb8f60d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb8809e440 0x7fcb88134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:28.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.133+0000 7fcb8f60d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb8809b120 con 0x7fcb8809e000 2023-10-03T10:19:28.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.133+0000 7fcb8f60d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb8806b460 con 0x7fcb8809d6f0 2023-10-03T10:19:28.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.133+0000 7fcb8f60d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8809a7e0 con 0x7fcb8809e440 2023-10-03T10:19:28.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.133+0000 7fcb8d3a9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcb8809d6f0 0x7fcb8809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:28.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.133+0000 7fcb8cba8700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcb8809e000 0x7fcb881339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33186/0 (socket says 172.21.15.3:33186) 2023-10-03T10:19:28.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.133+0000 7fcb8cba8700 1 -- 172.21.15.3:0/4232318092 learned_addr learned my addr 172.21.15.3:0/4232318092 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:28.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.133+0000 7fcb8dbaa700 1 --2- 172.21.15.3:0/4232318092 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb8809e440 0x7fcb88134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:28.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.134+0000 7fcb8dbaa700 1 -- 172.21.15.3:0/4232318092 >> v2:172.21.15.3:3300/0 conn(0x7fcb8809d6f0 msgr2=0x7fcb8809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:28.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.134+0000 7fcb8dbaa700 1 --2- 172.21.15.3:0/4232318092 >> v2:172.21.15.3:3300/0 conn(0x7fcb8809d6f0 0x7fcb8809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.134+0000 7fcb8dbaa700 1 -- 172.21.15.3:0/4232318092 >> v1:172.21.15.3:6789/0 conn(0x7fcb8809e000 legacy=0x7fcb881339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:28.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.134+0000 7fcb8dbaa700 1 -- 172.21.15.3:0/4232318092 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8809a500 con 0x7fcb8809e440 2023-10-03T10:19:28.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.134+0000 7fcb8dbaa700 1 --2- 172.21.15.3:0/4232318092 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb8809e440 0x7fcb88134100 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009dc0 tx=0x7fcb8000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69932a46d32e006a server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:28.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.134+0000 7fcb7ffff700 1 -- 172.21.15.3:0/4232318092 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8000e070 con 0x7fcb8809e440 2023-10-03T10:19:28.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.134+0000 7fcb7ffff700 1 -- 172.21.15.3:0/4232318092 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8000bae0 con 0x7fcb8809e440 2023-10-03T10:19:28.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.134+0000 7fcb7ffff700 1 -- 172.21.15.3:0/4232318092 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8000be90 con 0x7fcb8809e440 2023-10-03T10:19:28.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.135+0000 7fcb8f60d700 1 -- 172.21.15.3:0/4232318092 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb8809e440 msgr2=0x7fcb88134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:28.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.135+0000 7fcb8f60d700 1 --2- 172.21.15.3:0/4232318092 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb8809e440 0x7fcb88134100 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009dc0 tx=0x7fcb8000b330 comp rx=0 tx=0).stop 2023-10-03T10:19:28.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.135+0000 7fcb8f60d700 1 -- 172.21.15.3:0/4232318092 shutdown_connections 2023-10-03T10:19:28.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.135+0000 7fcb8f60d700 1 --2- 172.21.15.3:0/4232318092 >> v2:172.21.15.3:3300/0 conn(0x7fcb8809d6f0 0x7fcb8809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.135+0000 7fcb8f60d700 1 --2- 172.21.15.3:0/4232318092 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb8809e440 0x7fcb88134100 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.135+0000 7fcb8f60d700 1 -- 172.21.15.3:0/4232318092 >> 172.21.15.3:0/4232318092 conn(0x7fcb88066f50 msgr2=0x7fcb8809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:28.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.136+0000 7fcb8f60d700 1 -- 172.21.15.3:0/4232318092 shutdown_connections 2023-10-03T10:19:28.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.136+0000 7fcb8f60d700 1 -- 172.21.15.3:0/4232318092 wait complete. 2023-10-03T10:19:28.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.137+0000 7fcb8f60d700 1 Processor -- start 2023-10-03T10:19:28.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.137+0000 7fcb8f60d700 1 -- start start 2023-10-03T10:19:28.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.138+0000 7fcb8f60d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb8809d6f0 0x7fcb881a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:28.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.138+0000 7fcb8f60d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb8809e000 0x7fcb881a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:28.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.138+0000 7fcb8f60d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb8809e440 0x7fcb881a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.138+0000 7fcb8f60d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb881a99d0 con 0x7fcb8809e000 2023-10-03T10:19:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.138+0000 7fcb8f60d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb881a7b60 con 0x7fcb8809e440 2023-10-03T10:19:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.138+0000 7fcb8f60d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb880a1f90 con 0x7fcb8809d6f0 2023-10-03T10:19:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.138+0000 7fcb8cba8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb8809e000 0x7fcb881a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.138+0000 7fcb8cba8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb8809e000 0x7fcb881a63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51002/0 (socket says 172.21.15.3:51002) 2023-10-03T10:19:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.138+0000 7fcb8cba8700 1 -- 172.21.15.3:0/2680684573 learned_addr learned my addr 172.21.15.3:0/2680684573 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.138+0000 7fcb8dbaa700 1 --2- 172.21.15.3:0/2680684573 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb8809e440 0x7fcb881a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.138+0000 7fcb8d3a9700 1 --2- 172.21.15.3:0/2680684573 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb8809d6f0 0x7fcb881a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.138+0000 7fcb8cba8700 1 -- 172.21.15.3:0/2680684573 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb8809d6f0 msgr2=0x7fcb881a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.138+0000 7fcb8cba8700 1 --2- 172.21.15.3:0/2680684573 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb8809d6f0 0x7fcb881a5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.139+0000 7fcb8cba8700 1 -- 172.21.15.3:0/2680684573 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb8809e440 msgr2=0x7fcb881a6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.139+0000 7fcb8cba8700 1 --2- 172.21.15.3:0/2680684573 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb8809e440 0x7fcb881a6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.139+0000 7fcb8cba8700 1 -- 172.21.15.3:0/2680684573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb80009a70 con 0x7fcb8809e000 2023-10-03T10:19:28.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.139+0000 7fcb8cba8700 1 --2- 172.21.15.3:0/2680684573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb8809e000 0x7fcb881a63f0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800c520 tx=0x7fcb7800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:28.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.139+0000 7fcb7dffb700 1 -- 172.21.15.3:0/2680684573 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78005d70 con 0x7fcb8809e000 2023-10-03T10:19:28.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.139+0000 7fcb7dffb700 1 -- 172.21.15.3:0/2680684573 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb780070d0 con 0x7fcb8809e000 2023-10-03T10:19:28.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.139+0000 7fcb7dffb700 1 -- 172.21.15.3:0/2680684573 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78007440 con 0x7fcb8809e000 2023-10-03T10:19:28.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.139+0000 7fcb8f60d700 1 -- 172.21.15.3:0/2680684573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8809c710 con 0x7fcb8809e000 2023-10-03T10:19:28.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.139+0000 7fcb8f60d700 1 -- 172.21.15.3:0/2680684573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb881a8dd0 con 0x7fcb8809e000 2023-10-03T10:19:28.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.140+0000 7fcb7dffb700 1 -- 172.21.15.3:0/2680684573 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcb780075a0 con 0x7fcb8809e000 2023-10-03T10:19:28.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.141+0000 7fcb7dffb700 1 -- 172.21.15.3:0/2680684573 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fcb7803ecb0 con 0x7fcb8809e000 2023-10-03T10:19:28.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.141+0000 7fcb8f60d700 1 -- 172.21.15.3:0/2680684573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb88074ea0 con 0x7fcb8809e000 2023-10-03T10:19:28.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.146+0000 7fcb7dffb700 1 -- 172.21.15.3:0/2680684573 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcb78017080 con 0x7fcb8809e000 2023-10-03T10:19:28.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.169+0000 7f814315b700 1 Processor -- start 2023-10-03T10:19:28.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.169+0000 7f814315b700 1 -- start start 2023-10-03T10:19:28.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.169+0000 7f814315b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f813c06f5d0 0x7f813c12ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:28.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.169+0000 7f814315b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f813c135c20 0x7f813c138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:28.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.169+0000 7f814315b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f813c06fa40 con 0x7f813c12cf60 2023-10-03T10:19:28.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.169+0000 7f814315b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f813c12cd90 con 0x7f813c135c20 2023-10-03T10:19:28.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.169+0000 7f814315b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f813c071780 con 0x7f813c06f5d0 2023-10-03T10:19:28.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.170+0000 7f8140ef7700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f813c12cf60 0x7f813c12f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33194/0 (socket says 172.21.15.3:33194) 2023-10-03T10:19:28.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.170+0000 7f8140ef7700 1 -- 172.21.15.3:0/2395043651 learned_addr learned my addr 172.21.15.3:0/2395043651 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:28.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.170+0000 7f81416f8700 1 --2- 172.21.15.3:0/2395043651 >> v2:172.21.15.3:3300/0 conn(0x7f813c135c20 0x7f813c138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:28.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.170+0000 7f813bfff700 1 --2- 172.21.15.3:0/2395043651 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f813c06f5d0 0x7f813c12ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:28.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.170+0000 7f813bfff700 1 -- 172.21.15.3:0/2395043651 >> v2:172.21.15.3:3300/0 conn(0x7f813c135c20 msgr2=0x7f813c138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:28.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.170+0000 7f813bfff700 1 --2- 172.21.15.3:0/2395043651 >> v2:172.21.15.3:3300/0 conn(0x7f813c135c20 0x7f813c138000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.170+0000 7f813bfff700 1 -- 172.21.15.3:0/2395043651 >> v1:172.21.15.3:6789/0 conn(0x7f813c12cf60 legacy=0x7f813c12f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:28.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.170+0000 7f813bfff700 1 -- 172.21.15.3:0/2395043651 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f813c071a40 con 0x7f813c06f5d0 2023-10-03T10:19:28.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.170+0000 7f813bfff700 1 --2- 172.21.15.3:0/2395043651 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f813c06f5d0 0x7f813c12ffc0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f8128009840 tx=0x7f8128009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4f508887235bbe3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:28.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.171+0000 7f813b7fe700 1 -- 172.21.15.3:0/2395043651 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8128011830 con 0x7f813c06f5d0 2023-10-03T10:19:28.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.171+0000 7f813b7fe700 1 -- 172.21.15.3:0/2395043651 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8128011990 con 0x7f813c06f5d0 2023-10-03T10:19:28.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.171+0000 7f813b7fe700 1 -- 172.21.15.3:0/2395043651 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8128011d20 con 0x7f813c06f5d0 2023-10-03T10:19:28.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.171+0000 7f814315b700 1 -- 172.21.15.3:0/2395043651 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f813c06f5d0 msgr2=0x7f813c12ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:28.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.171+0000 7f814315b700 1 --2- 172.21.15.3:0/2395043651 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f813c06f5d0 0x7f813c12ffc0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f8128009840 tx=0x7f8128009b50 comp rx=0 tx=0).stop 2023-10-03T10:19:28.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.172+0000 7f814315b700 1 -- 172.21.15.3:0/2395043651 shutdown_connections 2023-10-03T10:19:28.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.172+0000 7f814315b700 1 --2- 172.21.15.3:0/2395043651 >> v2:172.21.15.3:3300/0 conn(0x7f813c135c20 0x7f813c138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.172+0000 7f814315b700 1 --2- 172.21.15.3:0/2395043651 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f813c06f5d0 0x7f813c12ffc0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.172+0000 7f814315b700 1 -- 172.21.15.3:0/2395043651 >> 172.21.15.3:0/2395043651 conn(0x7f813c0a2aa0 msgr2=0x7f813c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:28.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.172+0000 7f814315b700 1 -- 172.21.15.3:0/2395043651 shutdown_connections 2023-10-03T10:19:28.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.172+0000 7f814315b700 1 -- 172.21.15.3:0/2395043651 wait complete. 2023-10-03T10:19:28.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.173+0000 7f814315b700 1 Processor -- start 2023-10-03T10:19:28.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.173+0000 7f814315b700 1 -- start start 2023-10-03T10:19:28.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.173+0000 7f814315b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f813c06f5d0 0x7f813c12e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:28.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.173+0000 7f814315b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f813c12cf60 0x7f813c12ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:28.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.173+0000 7f814315b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f813c135c20 0x7f813c1c4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:28.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.173+0000 7f814315b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813c129fa0 con 0x7f813c12cf60 2023-10-03T10:19:28.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.173+0000 7f814315b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f813c12c0c0 con 0x7f813c06f5d0 2023-10-03T10:19:28.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.173+0000 7f814315b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f813c1305e0 con 0x7f813c135c20 2023-10-03T10:19:28.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.174+0000 7f8140ef7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f813c06f5d0 0x7f813c12e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:28.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.174+0000 7f81416f8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f813c135c20 0x7f813c1c4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:28.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.174+0000 7f8140ef7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f813c06f5d0 0x7f813c12e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51844/0 (socket says 172.21.15.3:51844) 2023-10-03T10:19:28.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.174+0000 7f81416f8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f813c135c20 0x7f813c1c4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36722/0 (socket says 172.21.15.3:36722) 2023-10-03T10:19:28.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.174+0000 7f8140ef7700 1 -- 172.21.15.3:0/587047238 learned_addr learned my addr 172.21.15.3:0/587047238 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:28.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.174+0000 7f813bfff700 1 --2- 172.21.15.3:0/587047238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f813c12cf60 0x7f813c12ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:28.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.174+0000 7f813bfff700 1 -- 172.21.15.3:0/587047238 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f813c135c20 msgr2=0x7f813c1c4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:28.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.174+0000 7f813bfff700 1 --2- 172.21.15.3:0/587047238 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f813c135c20 0x7f813c1c4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.174+0000 7f813bfff700 1 -- 172.21.15.3:0/587047238 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f813c06f5d0 msgr2=0x7f813c12e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:28.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.174+0000 7f813bfff700 1 --2- 172.21.15.3:0/587047238 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f813c06f5d0 0x7f813c12e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.174+0000 7f813bfff700 1 -- 172.21.15.3:0/587047238 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81280094a0 con 0x7f813c12cf60 2023-10-03T10:19:28.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.174+0000 7f8140ef7700 1 --2- 172.21.15.3:0/587047238 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f813c06f5d0 0x7f813c12e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:28.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.174+0000 7f813bfff700 1 --2- 172.21.15.3:0/587047238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f813c12cf60 0x7f813c12ea00 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f8128016000 tx=0x7f8128004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:28.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.174+0000 7f81397fa700 1 -- 172.21.15.3:0/587047238 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81280036a0 con 0x7f813c12cf60 2023-10-03T10:19:28.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.175+0000 7f81397fa700 1 -- 172.21.15.3:0/587047238 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f812801f070 con 0x7f813c12cf60 2023-10-03T10:19:28.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.175+0000 7f81397fa700 1 -- 172.21.15.3:0/587047238 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8128003a60 con 0x7f813c12cf60 2023-10-03T10:19:28.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.175+0000 7f814315b700 1 -- 172.21.15.3:0/587047238 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f813c12ef40 con 0x7f813c12cf60 2023-10-03T10:19:28.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.175+0000 7f814315b700 1 -- 172.21.15.3:0/587047238 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f813c135550 con 0x7f813c12cf60 2023-10-03T10:19:28.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.176+0000 7f81397fa700 1 -- 172.21.15.3:0/587047238 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8128003bc0 con 0x7f813c12cf60 2023-10-03T10:19:28.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.176+0000 7f814315b700 1 -- 172.21.15.3:0/587047238 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f813c074ea0 con 0x7f813c12cf60 2023-10-03T10:19:28.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.176+0000 7f81397fa700 1 -- 172.21.15.3:0/587047238 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f8128021070 con 0x7f813c12cf60 2023-10-03T10:19:28.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.181+0000 7f81397fa700 1 -- 172.21.15.3:0/587047238 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f812802b070 con 0x7f813c12cf60 2023-10-03T10:19:28.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.277+0000 7fe5d86a6700 1 -- 172.21.15.3:0/2522660710 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe5d0072210 con 0x7fe5d006f680 2023-10-03T10:19:28.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.277+0000 7fe5c6ffd700 1 -- 172.21.15.3:0/2522660710 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe5cc021020 con 0x7fe5d006f680 2023-10-03T10:19:28.277 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:28.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.279+0000 7fe5d86a6700 1 -- 172.21.15.3:0/2522660710 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5d006f680 msgr2=0x7fe5d01214e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:28.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.279+0000 7fe5d86a6700 1 --2- 172.21.15.3:0/2522660710 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5d006f680 0x7fe5d01214e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00e910 tx=0x7fe5cc00ecd0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.279+0000 7fe5d86a6700 1 -- 172.21.15.3:0/2522660710 shutdown_connections 2023-10-03T10:19:28.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.279+0000 7fe5d86a6700 1 --2- 172.21.15.3:0/2522660710 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe5d0130670 0x7fe5d0121f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.279+0000 7fe5d86a6700 1 --2- 172.21.15.3:0/2522660710 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe5d006f680 0x7fe5d01214e0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.279+0000 7fe5d86a6700 1 --2- 172.21.15.3:0/2522660710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5d006ffd0 0x7fe5d0121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.279+0000 7fe5d86a6700 1 -- 172.21.15.3:0/2522660710 >> 172.21.15.3:0/2522660710 conn(0x7fe5d00a2aa0 msgr2=0x7fe5d006e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:28.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.280+0000 7fe5d86a6700 1 -- 172.21.15.3:0/2522660710 shutdown_connections 2023-10-03T10:19:28.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.280+0000 7fe5d86a6700 1 -- 172.21.15.3:0/2522660710 wait complete. 2023-10-03T10:19:28.299 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:28.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.325+0000 7f4257718700 1 -- 172.21.15.3:0/3101707584 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4250123120 con 0x7f425009de50 2023-10-03T10:19:28.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.325+0000 7f4245ffb700 1 -- 172.21.15.3:0/3101707584 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4250123120 con 0x7f425009de50 2023-10-03T10:19:28.326 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:28.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.327+0000 7f4257718700 1 -- 172.21.15.3:0/3101707584 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f425009de50 msgr2=0x7f425011fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:28.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.327+0000 7f4257718700 1 --2- 172.21.15.3:0/3101707584 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f425009de50 0x7f425011fa10 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f424c00e9b0 tx=0x7f424c00ecc0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.327+0000 7f4257718700 1 -- 172.21.15.3:0/3101707584 shutdown_connections 2023-10-03T10:19:28.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.327+0000 7f4257718700 1 --2- 172.21.15.3:0/3101707584 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f42501338a0 0x7f4250120490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.327+0000 7f4257718700 1 --2- 172.21.15.3:0/3101707584 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f425009de50 0x7f425011fa10 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.327+0000 7f4257718700 1 --2- 172.21.15.3:0/3101707584 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f425009e760 0x7f425011ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.328+0000 7f4257718700 1 -- 172.21.15.3:0/3101707584 >> 172.21.15.3:0/3101707584 conn(0x7f42500a2aa0 msgr2=0x7f425009ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:28.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.328+0000 7f4257718700 1 -- 172.21.15.3:0/3101707584 shutdown_connections 2023-10-03T10:19:28.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.328+0000 7f4257718700 1 -- 172.21.15.3:0/3101707584 wait complete. 2023-10-03T10:19:28.341 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:28.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.360+0000 7fcb8f60d700 1 -- 172.21.15.3:0/2680684573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fcb8809f420 con 0x7fcb8809e000 2023-10-03T10:19:28.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.360+0000 7fcb7dffb700 1 -- 172.21.15.3:0/2680684573 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcb78019b40 con 0x7fcb8809e000 2023-10-03T10:19:28.361 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:28.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.363+0000 7fcb8f60d700 1 -- 172.21.15.3:0/2680684573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb8809e000 msgr2=0x7fcb881a63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:28.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.363+0000 7fcb8f60d700 1 --2- 172.21.15.3:0/2680684573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb8809e000 0x7fcb881a63f0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800c520 tx=0x7fcb7800c8e0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.363+0000 7fcb8f60d700 1 -- 172.21.15.3:0/2680684573 shutdown_connections 2023-10-03T10:19:28.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.363+0000 7fcb8f60d700 1 --2- 172.21.15.3:0/2680684573 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcb8809e440 0x7fcb881a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.363+0000 7fcb8f60d700 1 --2- 172.21.15.3:0/2680684573 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcb8809d6f0 0x7fcb881a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.363+0000 7fcb8f60d700 1 --2- 172.21.15.3:0/2680684573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb8809e000 0x7fcb881a63f0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.363+0000 7fcb8f60d700 1 -- 172.21.15.3:0/2680684573 >> 172.21.15.3:0/2680684573 conn(0x7fcb88066f50 msgr2=0x7fcb881badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:28.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.363+0000 7fcb8f60d700 1 -- 172.21.15.3:0/2680684573 shutdown_connections 2023-10-03T10:19:28.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.363+0000 7fcb8f60d700 1 -- 172.21.15.3:0/2680684573 wait complete. 2023-10-03T10:19:28.377 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:28.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.397+0000 7f814315b700 1 -- 172.21.15.3:0/587047238 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f813c0713a0 con 0x7f813c12cf60 2023-10-03T10:19:28.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.397+0000 7f81397fa700 1 -- 172.21.15.3:0/587047238 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8128015030 con 0x7f813c12cf60 2023-10-03T10:19:28.398 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:28.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.400+0000 7f814315b700 1 -- 172.21.15.3:0/587047238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f813c12cf60 msgr2=0x7f813c12ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:28.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.400+0000 7f814315b700 1 --2- 172.21.15.3:0/587047238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f813c12cf60 0x7f813c12ea00 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f8128016000 tx=0x7f8128004750 comp rx=0 tx=0).stop 2023-10-03T10:19:28.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.400+0000 7f814315b700 1 -- 172.21.15.3:0/587047238 shutdown_connections 2023-10-03T10:19:28.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.400+0000 7f814315b700 1 --2- 172.21.15.3:0/587047238 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f813c135c20 0x7f813c1c4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.400+0000 7f814315b700 1 --2- 172.21.15.3:0/587047238 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f813c06f5d0 0x7f813c12e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.400+0000 7f814315b700 1 --2- 172.21.15.3:0/587047238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f813c12cf60 0x7f813c12ea00 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:28.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.400+0000 7f814315b700 1 -- 172.21.15.3:0/587047238 >> 172.21.15.3:0/587047238 conn(0x7f813c0a2aa0 msgr2=0x7f813c06e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:28.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.400+0000 7f814315b700 1 -- 172.21.15.3:0/587047238 shutdown_connections 2023-10-03T10:19:28.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:28.400+0000 7f814315b700 1 -- 172.21.15.3:0/587047238 wait complete. 2023-10-03T10:19:28.413 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:29.300 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:29.342 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:29.377 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:29.414 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:29.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.451+0000 7f028b2fc700 1 Processor -- start 2023-10-03T10:19:29.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.452+0000 7f028b2fc700 1 -- start start 2023-10-03T10:19:29.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.452+0000 7f028b2fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0284131810 0x7f0284131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:29.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.452+0000 7f028b2fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0284099a40 0x7f0284099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:29.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.452+0000 7f028b2fc700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02841313c0 con 0x7f028409a3b0 2023-10-03T10:19:29.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.452+0000 7f028b2fc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f028409b580 con 0x7f0284131810 2023-10-03T10:19:29.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.452+0000 7f028b2fc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f028406e8c0 con 0x7f0284099a40 2023-10-03T10:19:29.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.452+0000 7f0289098700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0284131810 0x7f0284131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:29.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.452+0000 7f0289098700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0284131810 0x7f0284131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51016/0 (socket says 172.21.15.3:51016) 2023-10-03T10:19:29.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.452+0000 7f0289098700 1 -- 172.21.15.3:0/817333457 learned_addr learned my addr 172.21.15.3:0/817333457 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:29.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.452+0000 7f0288897700 1 --2- 172.21.15.3:0/817333457 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0284099a40 0x7f0284099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:29.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.453+0000 7f0289098700 1 -- 172.21.15.3:0/817333457 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0284099a40 msgr2=0x7f0284099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:29.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.453+0000 7f0289098700 1 --2- 172.21.15.3:0/817333457 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0284099a40 0x7f0284099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.453+0000 7f0289098700 1 -- 172.21.15.3:0/817333457 >> v1:172.21.15.3:6789/0 conn(0x7f028409a3b0 legacy=0x7f0284138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:29.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.453+0000 7f0289098700 1 -- 172.21.15.3:0/817333457 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f028409a830 con 0x7f0284131810 2023-10-03T10:19:29.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.453+0000 7f0289098700 1 --2- 172.21.15.3:0/817333457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0284131810 0x7f0284131be0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f02800098f0 tx=0x7f0280009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3150ca2798cd5bcd server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:29.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.453+0000 7f027bfff700 1 -- 172.21.15.3:0/817333457 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0280011830 con 0x7f0284131810 2023-10-03T10:19:29.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.453+0000 7f027bfff700 1 -- 172.21.15.3:0/817333457 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0280011990 con 0x7f0284131810 2023-10-03T10:19:29.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.453+0000 7f027bfff700 1 -- 172.21.15.3:0/817333457 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0280011d20 con 0x7f0284131810 2023-10-03T10:19:29.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.454+0000 7f028b2fc700 1 -- 172.21.15.3:0/817333457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0284131810 msgr2=0x7f0284131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:29.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.454+0000 7f028b2fc700 1 --2- 172.21.15.3:0/817333457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0284131810 0x7f0284131be0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f02800098f0 tx=0x7f0280009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:29.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.454+0000 7f028b2fc700 1 -- 172.21.15.3:0/817333457 shutdown_connections 2023-10-03T10:19:29.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.454+0000 7f028b2fc700 1 --2- 172.21.15.3:0/817333457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0284131810 0x7f0284131be0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.454+0000 7f028b2fc700 1 --2- 172.21.15.3:0/817333457 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0284099a40 0x7f0284099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.454+0000 7f028b2fc700 1 -- 172.21.15.3:0/817333457 >> 172.21.15.3:0/817333457 conn(0x7f02840a2aa0 msgr2=0x7f02840a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:29.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.454+0000 7f028b2fc700 1 -- 172.21.15.3:0/817333457 shutdown_connections 2023-10-03T10:19:29.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.454+0000 7f028b2fc700 1 -- 172.21.15.3:0/817333457 wait complete. 2023-10-03T10:19:29.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.455+0000 7f028b2fc700 1 Processor -- start 2023-10-03T10:19:29.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.456+0000 7f028b2fc700 1 -- start start 2023-10-03T10:19:29.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.456+0000 7f028b2fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0284099a40 0x7f028411b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:29.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.456+0000 7f028b2fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f028409a3b0 0x7f028411bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:29.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.456+0000 7f028b2fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0284131810 0x7f028411c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:29.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.456+0000 7f028b2fc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028411f1a0 con 0x7f0284099a40 2023-10-03T10:19:29.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.456+0000 7f028b2fc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028411e550 con 0x7f0284131810 2023-10-03T10:19:29.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.456+0000 7f028b2fc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f028411d330 con 0x7f028409a3b0 2023-10-03T10:19:29.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.456+0000 7f0289899700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0284131810 0x7f028411c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:29.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.456+0000 7f0289899700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0284131810 0x7f028411c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51848/0 (socket says 172.21.15.3:51848) 2023-10-03T10:19:29.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.456+0000 7f0289899700 1 -- 172.21.15.3:0/3352677498 learned_addr learned my addr 172.21.15.3:0/3352677498 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:29.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.456+0000 7f0289098700 1 --2- 172.21.15.3:0/3352677498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0284099a40 0x7f028411b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:29.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.457+0000 7f0289899700 1 -- 172.21.15.3:0/3352677498 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f028409a3b0 msgr2=0x7f028411bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:29.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.457+0000 7f0288897700 1 --2- 172.21.15.3:0/3352677498 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f028409a3b0 0x7f028411bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:29.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.457+0000 7f0289899700 1 --2- 172.21.15.3:0/3352677498 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f028409a3b0 0x7f028411bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.457+0000 7f0289899700 1 -- 172.21.15.3:0/3352677498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0284099a40 msgr2=0x7f028411b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:29.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.457+0000 7f0289899700 1 --2- 172.21.15.3:0/3352677498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0284099a40 0x7f028411b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.457+0000 7f0289899700 1 -- 172.21.15.3:0/3352677498 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0280009550 con 0x7f0284131810 2023-10-03T10:19:29.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.457+0000 7f0289098700 1 --2- 172.21.15.3:0/3352677498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0284099a40 0x7f028411b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:29.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.457+0000 7f0288897700 1 --2- 172.21.15.3:0/3352677498 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f028409a3b0 0x7f028411bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:19:29.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.457+0000 7f0289899700 1 --2- 172.21.15.3:0/3352677498 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0284131810 0x7f028411c100 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f027c00cc50 tx=0x7f027c00cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:29.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.458+0000 7f0279ffb700 1 -- 172.21.15.3:0/3352677498 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c003c00 con 0x7f0284131810 2023-10-03T10:19:29.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.458+0000 7f0279ffb700 1 -- 172.21.15.3:0/3352677498 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f027c003d60 con 0x7f0284131810 2023-10-03T10:19:29.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.458+0000 7f028b2fc700 1 -- 172.21.15.3:0/3352677498 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02840a1d20 con 0x7f0284131810 2023-10-03T10:19:29.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.458+0000 7f0279ffb700 1 -- 172.21.15.3:0/3352677498 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c00ab60 con 0x7f0284131810 2023-10-03T10:19:29.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.458+0000 7f028b2fc700 1 -- 172.21.15.3:0/3352677498 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f028411f6a0 con 0x7f0284131810 2023-10-03T10:19:29.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.459+0000 7f028b2fc700 1 -- 172.21.15.3:0/3352677498 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0284074ea0 con 0x7f0284131810 2023-10-03T10:19:29.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.460+0000 7f0279ffb700 1 -- 172.21.15.3:0/3352677498 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f027c00acc0 con 0x7f0284131810 2023-10-03T10:19:29.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.461+0000 7f0279ffb700 1 -- 172.21.15.3:0/3352677498 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f027c03fe40 con 0x7f0284131810 2023-10-03T10:19:29.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.467+0000 7f0279ffb700 1 -- 172.21.15.3:0/3352677498 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f027c044060 con 0x7f0284131810 2023-10-03T10:19:29.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.500+0000 7fe5382a4700 1 Processor -- start 2023-10-03T10:19:29.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.500+0000 7fe5382a4700 1 -- start start 2023-10-03T10:19:29.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.501+0000 7fe5382a4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe53009d6f0 0x7fe53009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:29.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.501+0000 7fe5382a4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe53009e440 0x7fe530134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:29.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.501+0000 7fe5382a4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe53009b120 con 0x7fe53009e000 2023-10-03T10:19:29.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.501+0000 7fe5382a4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe53006b460 con 0x7fe53009d6f0 2023-10-03T10:19:29.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.501+0000 7fe5382a4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe53009a7e0 con 0x7fe53009e440 2023-10-03T10:19:29.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.502+0000 7fe536040700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe53009d6f0 0x7fe53009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:29.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.502+0000 7fe536040700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe53009d6f0 0x7fe53009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51032/0 (socket says 172.21.15.3:51032) 2023-10-03T10:19:29.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.502+0000 7fe536040700 1 -- 172.21.15.3:0/716294013 learned_addr learned my addr 172.21.15.3:0/716294013 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:29.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.502+0000 7fe536841700 1 --2- 172.21.15.3:0/716294013 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe53009e440 0x7fe530134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:29.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.502+0000 7fe536040700 1 -- 172.21.15.3:0/716294013 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe53009e440 msgr2=0x7fe530134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:29.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.502+0000 7fe536040700 1 --2- 172.21.15.3:0/716294013 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe53009e440 0x7fe530134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.502+0000 7fe536040700 1 -- 172.21.15.3:0/716294013 >> v1:172.21.15.3:6789/0 conn(0x7fe53009e000 legacy=0x7fe5301339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:29.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.502+0000 7fe536040700 1 -- 172.21.15.3:0/716294013 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe53009a500 con 0x7fe53009d6f0 2023-10-03T10:19:29.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.502+0000 7fe53503e700 1 -- 172.21.15.3:0/716294013 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1623748668 0 0) 0x7fe53009b120 con 0x7fe53009e000 2023-10-03T10:19:29.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.502+0000 7fe536040700 1 --2- 172.21.15.3:0/716294013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe53009d6f0 0x7fe53009dac0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fe52c009920 tx=0x7fe52c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc03f297de4ab5fc server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:29.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.503+0000 7fe53503e700 1 -- 172.21.15.3:0/716294013 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c011830 con 0x7fe53009d6f0 2023-10-03T10:19:29.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.503+0000 7fe53503e700 1 -- 172.21.15.3:0/716294013 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe52c011990 con 0x7fe53009d6f0 2023-10-03T10:19:29.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.503+0000 7fe53503e700 1 -- 172.21.15.3:0/716294013 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c011d20 con 0x7fe53009d6f0 2023-10-03T10:19:29.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.504+0000 7fe5382a4700 1 -- 172.21.15.3:0/716294013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe53009d6f0 msgr2=0x7fe53009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:29.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.504+0000 7fe5382a4700 1 --2- 172.21.15.3:0/716294013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe53009d6f0 0x7fe53009dac0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fe52c009920 tx=0x7fe52c009c30 comp rx=0 tx=0).stop 2023-10-03T10:19:29.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.504+0000 7fe5382a4700 1 -- 172.21.15.3:0/716294013 shutdown_connections 2023-10-03T10:19:29.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.504+0000 7fe5382a4700 1 --2- 172.21.15.3:0/716294013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe53009d6f0 0x7fe53009dac0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.504+0000 7fe5382a4700 1 --2- 172.21.15.3:0/716294013 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe53009e440 0x7fe530134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.504+0000 7fe5382a4700 1 -- 172.21.15.3:0/716294013 >> 172.21.15.3:0/716294013 conn(0x7fe530066f50 msgr2=0x7fe53009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:29.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.504+0000 7fe5382a4700 1 -- 172.21.15.3:0/716294013 shutdown_connections 2023-10-03T10:19:29.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.504+0000 7fe5382a4700 1 -- 172.21.15.3:0/716294013 wait complete. 2023-10-03T10:19:29.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.505+0000 7fe5382a4700 1 Processor -- start 2023-10-03T10:19:29.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.506+0000 7fe5382a4700 1 -- start start 2023-10-03T10:19:29.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.506+0000 7fe5382a4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe53009d6f0 0x7fe5301c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:29.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.506+0000 7fe5382a4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe53009e000 0x7fe5301c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:29.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.506+0000 7fe5382a4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe53009e440 0x7fe5301c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:29.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.506+0000 7fe5382a4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5301a4c90 con 0x7fe53009d6f0 2023-10-03T10:19:29.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.506+0000 7fe5382a4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5301a8100 con 0x7fe53009e440 2023-10-03T10:19:29.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.506+0000 7fe5382a4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe53009c710 con 0x7fe53009e000 2023-10-03T10:19:29.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.507+0000 7fe536040700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe53009d6f0 0x7fe5301c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:29.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.507+0000 7fe536040700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe53009d6f0 0x7fe5301c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51048/0 (socket says 172.21.15.3:51048) 2023-10-03T10:19:29.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.507+0000 7fe536040700 1 -- 172.21.15.3:0/3902085677 learned_addr learned my addr 172.21.15.3:0/3902085677 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:29.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.507+0000 7fe53583f700 1 --2- 172.21.15.3:0/3902085677 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe53009e000 0x7fe5301c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:29.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.507+0000 7fe536040700 1 -- 172.21.15.3:0/3902085677 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe53009e000 msgr2=0x7fe5301c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:29.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.507+0000 7fe536040700 1 --2- 172.21.15.3:0/3902085677 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe53009e000 0x7fe5301c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.507+0000 7fe536040700 1 -- 172.21.15.3:0/3902085677 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe53009e440 msgr2=0x7fe5301c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:29.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.507+0000 7fe536040700 1 --2- 172.21.15.3:0/3902085677 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe53009e440 0x7fe5301c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.507+0000 7fe536040700 1 -- 172.21.15.3:0/3902085677 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe52c009550 con 0x7fe53009d6f0 2023-10-03T10:19:29.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.507+0000 7fe536040700 1 --2- 172.21.15.3:0/3902085677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe53009d6f0 0x7fe5301c8090 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fe52c000c00 tx=0x7fe52c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:29.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.507+0000 7fe526ffd700 1 -- 172.21.15.3:0/3902085677 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c004510 con 0x7fe53009d6f0 2023-10-03T10:19:29.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.507+0000 7fe526ffd700 1 -- 172.21.15.3:0/3902085677 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe52c0036a0 con 0x7fe53009d6f0 2023-10-03T10:19:29.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.507+0000 7fe5382a4700 1 -- 172.21.15.3:0/3902085677 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5301a63f0 con 0x7fe53009d6f0 2023-10-03T10:19:29.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.508+0000 7fe526ffd700 1 -- 172.21.15.3:0/3902085677 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c003a50 con 0x7fe53009d6f0 2023-10-03T10:19:29.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.508+0000 7fe5382a4700 1 -- 172.21.15.3:0/3902085677 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5301a7060 con 0x7fe53009d6f0 2023-10-03T10:19:29.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.509+0000 7fe526ffd700 1 -- 172.21.15.3:0/3902085677 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe52c003bb0 con 0x7fe53009d6f0 2023-10-03T10:19:29.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.509+0000 7fe526ffd700 1 -- 172.21.15.3:0/3902085677 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe52c021030 con 0x7fe53009d6f0 2023-10-03T10:19:29.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.510+0000 7fe5382a4700 1 -- 172.21.15.3:0/3902085677 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe530074ea0 con 0x7fe53009d6f0 2023-10-03T10:19:29.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.515+0000 7fe526ffd700 1 -- 172.21.15.3:0/3902085677 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe52c02b080 con 0x7fe53009d6f0 2023-10-03T10:19:29.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.537+0000 7f94d9759700 1 Processor -- start 2023-10-03T10:19:29.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.537+0000 7f94d9759700 1 -- start start 2023-10-03T10:19:29.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.538+0000 7f94d9759700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f94d412cf60 0x7f94d406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:29.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.538+0000 7f94d9759700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94d4130430 0x7f94d4137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:29.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.538+0000 7f94d9759700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d412cd90 con 0x7f94d412f8b0 2023-10-03T10:19:29.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.538+0000 7f94d9759700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94d412d330 con 0x7f94d412cf60 2023-10-03T10:19:29.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.538+0000 7f94d9759700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d4071780 con 0x7f94d4130430 2023-10-03T10:19:29.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.538+0000 7f94d27fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f94d412f8b0 0x7f94d412fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33214/0 (socket says 172.21.15.3:33214) 2023-10-03T10:19:29.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.538+0000 7f94d27fc700 1 -- 172.21.15.3:0/1303024675 learned_addr learned my addr 172.21.15.3:0/1303024675 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:29.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.538+0000 7f94d2ffd700 1 --2- 172.21.15.3:0/1303024675 >> v2:172.21.15.3:3300/0 conn(0x7f94d412cf60 0x7f94d406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:29.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.538+0000 7f94d37fe700 1 --2- 172.21.15.3:0/1303024675 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94d4130430 0x7f94d4137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:29.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.538+0000 7f94d2ffd700 1 -- 172.21.15.3:0/1303024675 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94d4130430 msgr2=0x7f94d4137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:29.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.538+0000 7f94d2ffd700 1 --2- 172.21.15.3:0/1303024675 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94d4130430 0x7f94d4137c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.538+0000 7f94d2ffd700 1 -- 172.21.15.3:0/1303024675 >> v1:172.21.15.3:6789/0 conn(0x7f94d412f8b0 legacy=0x7f94d412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:29.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.538+0000 7f94d2ffd700 1 -- 172.21.15.3:0/1303024675 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d4071a40 con 0x7f94d412cf60 2023-10-03T10:19:29.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.538+0000 7f94d37fe700 1 --2- 172.21.15.3:0/1303024675 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94d4130430 0x7f94d4137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:29.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.538+0000 7f94d2ffd700 1 --2- 172.21.15.3:0/1303024675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94d412cf60 0x7f94d406f5d0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f94c40098a0 tx=0x7f94c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac3ce644f392ec9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:29.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.539+0000 7f94d1ffb700 1 -- 172.21.15.3:0/1303024675 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c4004220 con 0x7f94d412cf60 2023-10-03T10:19:29.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.539+0000 7f94d1ffb700 1 -- 172.21.15.3:0/1303024675 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c4004380 con 0x7f94d412cf60 2023-10-03T10:19:29.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.539+0000 7f94d1ffb700 1 -- 172.21.15.3:0/1303024675 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c40108c0 con 0x7f94d412cf60 2023-10-03T10:19:29.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.540+0000 7f94d9759700 1 -- 172.21.15.3:0/1303024675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94d412cf60 msgr2=0x7f94d406f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:29.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.540+0000 7f94d9759700 1 --2- 172.21.15.3:0/1303024675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94d412cf60 0x7f94d406f5d0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f94c40098a0 tx=0x7f94c4009bb0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.540+0000 7f94d9759700 1 -- 172.21.15.3:0/1303024675 shutdown_connections 2023-10-03T10:19:29.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.540+0000 7f94d9759700 1 --2- 172.21.15.3:0/1303024675 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94d4130430 0x7f94d4137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.540+0000 7f94d9759700 1 --2- 172.21.15.3:0/1303024675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94d412cf60 0x7f94d406f5d0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.540+0000 7f94d9759700 1 -- 172.21.15.3:0/1303024675 >> 172.21.15.3:0/1303024675 conn(0x7f94d40a2aa0 msgr2=0x7f94d40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:29.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.540+0000 7f94d9759700 1 -- 172.21.15.3:0/1303024675 shutdown_connections 2023-10-03T10:19:29.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.540+0000 7f94d9759700 1 -- 172.21.15.3:0/1303024675 wait complete. 2023-10-03T10:19:29.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.542+0000 7f94d9759700 1 Processor -- start 2023-10-03T10:19:29.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.542+0000 7f94d9759700 1 -- start start 2023-10-03T10:19:29.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.543+0000 7f94d9759700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94d412cf60 0x7f94d41c4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:29.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.543+0000 7f94d9759700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94d412f8b0 0x7f94d41c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:29.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.543+0000 7f94d9759700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94d4130430 0x7f94d41c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:29.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.543+0000 7f94d9759700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d409e7a0 con 0x7f94d412f8b0 2023-10-03T10:19:29.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.543+0000 7f94d9759700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d4129a40 con 0x7f94d4130430 2023-10-03T10:19:29.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.543+0000 7f94d9759700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94d409ccf0 con 0x7f94d412cf60 2023-10-03T10:19:29.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.543+0000 7f94d2ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94d412cf60 0x7f94d41c4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:29.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.543+0000 7f94d2ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94d412cf60 0x7f94d41c4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36772/0 (socket says 172.21.15.3:36772) 2023-10-03T10:19:29.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.543+0000 7f94d2ffd700 1 -- 172.21.15.3:0/3920017171 learned_addr learned my addr 172.21.15.3:0/3920017171 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:29.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.543+0000 7f94d27fc700 1 --2- 172.21.15.3:0/3920017171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94d412f8b0 0x7f94d41c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:29.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.543+0000 7f94d37fe700 1 --2- 172.21.15.3:0/3920017171 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94d4130430 0x7f94d41c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:29.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.543+0000 7f94d27fc700 1 -- 172.21.15.3:0/3920017171 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94d412cf60 msgr2=0x7f94d41c4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:29.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.543+0000 7f94d27fc700 1 --2- 172.21.15.3:0/3920017171 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94d412cf60 0x7f94d41c4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.543+0000 7f94d27fc700 1 -- 172.21.15.3:0/3920017171 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94d4130430 msgr2=0x7f94d41c4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:29.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.543+0000 7f94d27fc700 1 --2- 172.21.15.3:0/3920017171 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94d4130430 0x7f94d41c4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.543+0000 7f94d27fc700 1 -- 172.21.15.3:0/3920017171 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c40094a0 con 0x7f94d412f8b0 2023-10-03T10:19:29.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.544+0000 7f94d2ffd700 1 --2- 172.21.15.3:0/3920017171 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94d412cf60 0x7f94d41c4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:29.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.544+0000 7f94d27fc700 1 --2- 172.21.15.3:0/3920017171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94d412f8b0 0x7f94d41c4560 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f94c800c530 tx=0x7f94c800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:29.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.544+0000 7f94bb7fe700 1 -- 172.21.15.3:0/3920017171 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c800cea0 con 0x7f94d412f8b0 2023-10-03T10:19:29.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.544+0000 7f94bb7fe700 1 -- 172.21.15.3:0/3920017171 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c8005d70 con 0x7f94d412f8b0 2023-10-03T10:19:29.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.544+0000 7f94d9759700 1 -- 172.21.15.3:0/3920017171 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d409e4e0 con 0x7f94d412f8b0 2023-10-03T10:19:29.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.544+0000 7f94bb7fe700 1 -- 172.21.15.3:0/3920017171 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c80072d0 con 0x7f94d412f8b0 2023-10-03T10:19:29.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.545+0000 7f94d9759700 1 -- 172.21.15.3:0/3920017171 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d4119140 con 0x7f94d412f8b0 2023-10-03T10:19:29.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.545+0000 7f94bb7fe700 1 -- 172.21.15.3:0/3920017171 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f94c8007430 con 0x7f94d412f8b0 2023-10-03T10:19:29.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.546+0000 7f94bb7fe700 1 -- 172.21.15.3:0/3920017171 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f94c8040ba0 con 0x7f94d412f8b0 2023-10-03T10:19:29.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.546+0000 7f94d9759700 1 -- 172.21.15.3:0/3920017171 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d4074ea0 con 0x7f94d412f8b0 2023-10-03T10:19:29.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.556+0000 7f94bb7fe700 1 -- 172.21.15.3:0/3920017171 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f94c8044060 con 0x7f94d412f8b0 2023-10-03T10:19:29.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.585+0000 7f9c894e6700 1 Processor -- start 2023-10-03T10:19:29.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.585+0000 7f9c894e6700 1 -- start start 2023-10-03T10:19:29.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.585+0000 7f9c894e6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c8409e930 0x7f9c8409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:29.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.585+0000 7f9c894e6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9c8409f240 0x7f9c841317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:29.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.585+0000 7f9c894e6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c8409f650 0x7f9c84131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:29.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.585+0000 7f9c894e6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c84070940 con 0x7f9c8409f240 2023-10-03T10:19:29.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.585+0000 7f9c894e6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8406fd60 con 0x7f9c8409e930 2023-10-03T10:19:29.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.585+0000 7f9c894e6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8406fa80 con 0x7f9c8409f650 2023-10-03T10:19:29.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.585+0000 7f9c827fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9c8409f240 0x7f9c841317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:29.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.585+0000 7f9c827fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9c8409f240 0x7f9c841317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51072/0 (socket says 172.21.15.3:51072) 2023-10-03T10:19:29.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.585+0000 7f9c827fc700 1 -- 172.21.15.3:0/630856559 learned_addr learned my addr 172.21.15.3:0/630856559 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:29.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.586+0000 7f9c82ffd700 1 --2- 172.21.15.3:0/630856559 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c8409e930 0x7f9c8409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:29.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.586+0000 7f9c837fe700 1 --2- 172.21.15.3:0/630856559 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c8409f650 0x7f9c84131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:29.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.586+0000 7f9c827fc700 1 -- 172.21.15.3:0/630856559 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c8409f650 msgr2=0x7f9c84131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:29.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.586+0000 7f9c827fc700 1 --2- 172.21.15.3:0/630856559 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c8409f650 0x7f9c84131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.586+0000 7f9c827fc700 1 -- 172.21.15.3:0/630856559 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c8409e930 msgr2=0x7f9c8409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:29.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.586+0000 7f9c827fc700 1 --2- 172.21.15.3:0/630856559 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c8409e930 0x7f9c8409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.586+0000 7f9c827fc700 1 -- 172.21.15.3:0/630856559 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c8409fb60 con 0x7f9c8409f240 2023-10-03T10:19:29.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.586+0000 7f9c837fe700 1 --2- 172.21.15.3:0/630856559 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c8409f650 0x7f9c84131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:29.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.586+0000 7f9c827fc700 1 --2- 172.21.15.3:0/630856559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c8409f240 0x7f9c841317f0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009950 tx=0x7f9c78009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d783164be050c385 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:29.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.586+0000 7f9c81ffb700 1 -- 172.21.15.3:0/630856559 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c78004030 con 0x7f9c8409f240 2023-10-03T10:19:29.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.586+0000 7f9c81ffb700 1 -- 172.21.15.3:0/630856559 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c78004190 con 0x7f9c8409f240 2023-10-03T10:19:29.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.586+0000 7f9c81ffb700 1 -- 172.21.15.3:0/630856559 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c78004540 con 0x7f9c8409f240 2023-10-03T10:19:29.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.587+0000 7f9c894e6700 1 -- 172.21.15.3:0/630856559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c8409f240 msgr2=0x7f9c841317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:29.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.587+0000 7f9c894e6700 1 --2- 172.21.15.3:0/630856559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c8409f240 0x7f9c841317f0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009950 tx=0x7f9c78009c60 comp rx=0 tx=0).stop 2023-10-03T10:19:29.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.587+0000 7f9c894e6700 1 -- 172.21.15.3:0/630856559 shutdown_connections 2023-10-03T10:19:29.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.587+0000 7f9c894e6700 1 --2- 172.21.15.3:0/630856559 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c8409f650 0x7f9c84131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.587+0000 7f9c894e6700 1 --2- 172.21.15.3:0/630856559 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c8409e930 0x7f9c8409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.587+0000 7f9c894e6700 1 --2- 172.21.15.3:0/630856559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c8409f240 0x7f9c841317f0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.587+0000 7f9c894e6700 1 -- 172.21.15.3:0/630856559 >> 172.21.15.3:0/630856559 conn(0x7f9c840a4210 msgr2=0x7f9c840a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:29.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.587+0000 7f9c894e6700 1 -- 172.21.15.3:0/630856559 shutdown_connections 2023-10-03T10:19:29.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.587+0000 7f9c894e6700 1 -- 172.21.15.3:0/630856559 wait complete. 2023-10-03T10:19:29.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.589+0000 7f9c894e6700 1 Processor -- start 2023-10-03T10:19:29.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.589+0000 7f9c894e6700 1 -- start start 2023-10-03T10:19:29.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.589+0000 7f9c894e6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c8409e930 0x7f9c841c0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:29.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.589+0000 7f9c894e6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c8409f240 0x7f9c841c1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:29.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.589+0000 7f9c894e6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c8409f650 0x7f9c841c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:29.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.589+0000 7f9c894e6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c841a3880 con 0x7f9c8409f240 2023-10-03T10:19:29.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.589+0000 7f9c894e6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c841a3690 con 0x7f9c8409f650 2023-10-03T10:19:29.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.589+0000 7f9c894e6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c840a3630 con 0x7f9c8409e930 2023-10-03T10:19:29.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.590+0000 7f9c82ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c8409e930 0x7f9c841c0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:29.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.590+0000 7f9c82ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c8409e930 0x7f9c841c0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36788/0 (socket says 172.21.15.3:36788) 2023-10-03T10:19:29.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.590+0000 7f9c82ffd700 1 -- 172.21.15.3:0/3745250961 learned_addr learned my addr 172.21.15.3:0/3745250961 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:29.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.590+0000 7f9c827fc700 1 --2- 172.21.15.3:0/3745250961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c8409f240 0x7f9c841c1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:29.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.590+0000 7f9c837fe700 1 --2- 172.21.15.3:0/3745250961 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c8409f650 0x7f9c841c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:29.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.590+0000 7f9c82ffd700 1 -- 172.21.15.3:0/3745250961 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c8409f650 msgr2=0x7f9c841c3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:29.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.590+0000 7f9c82ffd700 1 --2- 172.21.15.3:0/3745250961 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c8409f650 0x7f9c841c3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.590+0000 7f9c82ffd700 1 -- 172.21.15.3:0/3745250961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c8409f240 msgr2=0x7f9c841c1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.590+0000 7f9c82ffd700 1 --2- 172.21.15.3:0/3745250961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c8409f240 0x7f9c841c1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.590+0000 7f9c82ffd700 1 -- 172.21.15.3:0/3745250961 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c78009550 con 0x7f9c8409e930 2023-10-03T10:19:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.590+0000 7f9c82ffd700 1 --2- 172.21.15.3:0/3745250961 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c8409e930 0x7f9c841c0e30 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400e930 tx=0x7f9c7400ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.590+0000 7f9c6b7fe700 1 -- 172.21.15.3:0/3745250961 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c74013680 con 0x7f9c8409e930 2023-10-03T10:19:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.590+0000 7f9c6b7fe700 1 -- 172.21.15.3:0/3745250961 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c740137e0 con 0x7f9c8409e930 2023-10-03T10:19:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.591+0000 7f9c894e6700 1 -- 172.21.15.3:0/3745250961 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c840a0190 con 0x7f9c8409e930 2023-10-03T10:19:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.591+0000 7f9c6b7fe700 1 -- 172.21.15.3:0/3745250961 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c74013b70 con 0x7f9c8409e930 2023-10-03T10:19:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.591+0000 7f9c894e6700 1 -- 172.21.15.3:0/3745250961 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c841a48e0 con 0x7f9c8409e930 2023-10-03T10:19:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.591+0000 7f9c6b7fe700 1 -- 172.21.15.3:0/3745250961 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9c74013cd0 con 0x7f9c8409e930 2023-10-03T10:19:29.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.592+0000 7f9c6b7fe700 1 -- 172.21.15.3:0/3745250961 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9c7403f6e0 con 0x7f9c8409e930 2023-10-03T10:19:29.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.592+0000 7f9c894e6700 1 -- 172.21.15.3:0/3745250961 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c84074ea0 con 0x7f9c8409e930 2023-10-03T10:19:29.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.597+0000 7f9c6b7fe700 1 -- 172.21.15.3:0/3745250961 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9c7400c970 con 0x7f9c8409e930 2023-10-03T10:19:29.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.692+0000 7f028b2fc700 1 -- 172.21.15.3:0/3352677498 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f028411ee20 con 0x7f0284131810 2023-10-03T10:19:29.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.692+0000 7f0279ffb700 1 -- 172.21.15.3:0/3352677498 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f028411ee20 con 0x7f0284131810 2023-10-03T10:19:29.693 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:29.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.694+0000 7f028b2fc700 1 -- 172.21.15.3:0/3352677498 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0284131810 msgr2=0x7f028411c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:29.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.694+0000 7f028b2fc700 1 --2- 172.21.15.3:0/3352677498 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0284131810 0x7f028411c100 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f027c00cc50 tx=0x7f027c00cf60 comp rx=0 tx=0).stop 2023-10-03T10:19:29.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.695+0000 7f028b2fc700 1 -- 172.21.15.3:0/3352677498 shutdown_connections 2023-10-03T10:19:29.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.695+0000 7f028b2fc700 1 --2- 172.21.15.3:0/3352677498 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0284131810 0x7f028411c100 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.695+0000 7f028b2fc700 1 --2- 172.21.15.3:0/3352677498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0284099a40 0x7f028411b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.695+0000 7f028b2fc700 1 --2- 172.21.15.3:0/3352677498 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f028409a3b0 0x7f028411bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.695+0000 7f028b2fc700 1 -- 172.21.15.3:0/3352677498 >> 172.21.15.3:0/3352677498 conn(0x7f02840a2aa0 msgr2=0x7f0284070f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:29.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.695+0000 7f028b2fc700 1 -- 172.21.15.3:0/3352677498 shutdown_connections 2023-10-03T10:19:29.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.695+0000 7f028b2fc700 1 -- 172.21.15.3:0/3352677498 wait complete. 2023-10-03T10:19:29.709 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:29.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.734+0000 7fe5382a4700 1 -- 172.21.15.3:0/3902085677 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe5300a1f70 con 0x7fe53009d6f0 2023-10-03T10:19:29.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.734+0000 7fe526ffd700 1 -- 172.21.15.3:0/3902085677 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe52c01a430 con 0x7fe53009d6f0 2023-10-03T10:19:29.734 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:29.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.736+0000 7fe5382a4700 1 -- 172.21.15.3:0/3902085677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe53009d6f0 msgr2=0x7fe5301c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:29.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.736+0000 7fe5382a4700 1 --2- 172.21.15.3:0/3902085677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe53009d6f0 0x7fe5301c8090 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fe52c000c00 tx=0x7fe52c004750 comp rx=0 tx=0).stop 2023-10-03T10:19:29.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.736+0000 7fe5382a4700 1 -- 172.21.15.3:0/3902085677 shutdown_connections 2023-10-03T10:19:29.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.736+0000 7fe5382a4700 1 --2- 172.21.15.3:0/3902085677 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe53009e440 0x7fe5301c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.736+0000 7fe5382a4700 1 --2- 172.21.15.3:0/3902085677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe53009d6f0 0x7fe5301c8090 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.736+0000 7fe5382a4700 1 --2- 172.21.15.3:0/3902085677 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe53009e000 0x7fe5301c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.736+0000 7fe5382a4700 1 -- 172.21.15.3:0/3902085677 >> 172.21.15.3:0/3902085677 conn(0x7fe530066f50 msgr2=0x7fe53006f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:29.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.736+0000 7fe5382a4700 1 -- 172.21.15.3:0/3902085677 shutdown_connections 2023-10-03T10:19:29.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.736+0000 7fe5382a4700 1 -- 172.21.15.3:0/3902085677 wait complete. 2023-10-03T10:19:29.751 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:29.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.766+0000 7f94d9759700 1 -- 172.21.15.3:0/3920017171 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f94d40713a0 con 0x7f94d412f8b0 2023-10-03T10:19:29.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.767+0000 7f94bb7fe700 1 -- 172.21.15.3:0/3920017171 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f94c8011d70 con 0x7f94d412f8b0 2023-10-03T10:19:29.767 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:29.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.769+0000 7f94d9759700 1 -- 172.21.15.3:0/3920017171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94d412f8b0 msgr2=0x7f94d41c4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:29.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.769+0000 7f94d9759700 1 --2- 172.21.15.3:0/3920017171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94d412f8b0 0x7f94d41c4560 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f94c800c530 tx=0x7f94c800c840 comp rx=0 tx=0).stop 2023-10-03T10:19:29.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.769+0000 7f94d9759700 1 -- 172.21.15.3:0/3920017171 shutdown_connections 2023-10-03T10:19:29.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.769+0000 7f94d9759700 1 --2- 172.21.15.3:0/3920017171 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f94d4130430 0x7f94d41c4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.769+0000 7f94d9759700 1 --2- 172.21.15.3:0/3920017171 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f94d412cf60 0x7f94d41c4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.769+0000 7f94d9759700 1 --2- 172.21.15.3:0/3920017171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94d412f8b0 0x7f94d41c4560 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.769+0000 7f94d9759700 1 -- 172.21.15.3:0/3920017171 >> 172.21.15.3:0/3920017171 conn(0x7f94d40a2aa0 msgr2=0x7f94d406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:29.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.769+0000 7f94d9759700 1 -- 172.21.15.3:0/3920017171 shutdown_connections 2023-10-03T10:19:29.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.769+0000 7f94d9759700 1 -- 172.21.15.3:0/3920017171 wait complete. 2023-10-03T10:19:29.783 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:29.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.806+0000 7f9c894e6700 1 -- 172.21.15.3:0/3745250961 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9c8406b250 con 0x7f9c8409e930 2023-10-03T10:19:29.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.807+0000 7f9c6b7fe700 1 -- 172.21.15.3:0/3745250961 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9c74021030 con 0x7f9c8409e930 2023-10-03T10:19:29.807 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:29.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.809+0000 7f9c894e6700 1 -- 172.21.15.3:0/3745250961 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c8409e930 msgr2=0x7f9c841c0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:29.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.809+0000 7f9c894e6700 1 --2- 172.21.15.3:0/3745250961 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c8409e930 0x7f9c841c0e30 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400e930 tx=0x7f9c7400ec40 comp rx=0 tx=0).stop 2023-10-03T10:19:29.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.809+0000 7f9c894e6700 1 -- 172.21.15.3:0/3745250961 shutdown_connections 2023-10-03T10:19:29.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.809+0000 7f9c894e6700 1 --2- 172.21.15.3:0/3745250961 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c8409f650 0x7f9c841c3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.809+0000 7f9c894e6700 1 --2- 172.21.15.3:0/3745250961 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c8409e930 0x7f9c841c0e30 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.809+0000 7f9c894e6700 1 --2- 172.21.15.3:0/3745250961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c8409f240 0x7f9c841c1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:29.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.809+0000 7f9c894e6700 1 -- 172.21.15.3:0/3745250961 >> 172.21.15.3:0/3745250961 conn(0x7f9c840a4210 msgr2=0x7f9c841b6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:29.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.809+0000 7f9c894e6700 1 -- 172.21.15.3:0/3745250961 shutdown_connections 2023-10-03T10:19:29.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:29.809+0000 7f9c894e6700 1 -- 172.21.15.3:0/3745250961 wait complete. 2023-10-03T10:19:29.823 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:30.710 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:30.752 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:30.784 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:30.824 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:30.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.864+0000 7f52c1f8c700 1 Processor -- start 2023-10-03T10:19:30.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.864+0000 7f52c1f8c700 1 -- start start 2023-10-03T10:19:30.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.865+0000 7f52c1f8c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f52bc136830 0x7f52bc138c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:30.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.865+0000 7f52c1f8c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f52bc139150 0x7f52bc13b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:30.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.865+0000 7f52c1f8c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52bc06f720 con 0x7f52bc1338c0 2023-10-03T10:19:30.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.865+0000 7f52c1f8c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52bc070c80 con 0x7f52bc139150 2023-10-03T10:19:30.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.865+0000 7f52c1f8c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc06fce0 con 0x7f52bc136830 2023-10-03T10:19:30.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.866+0000 7f52baffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f52bc136830 0x7f52bc138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:30.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.866+0000 7f52bbfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f52bc139150 0x7f52bc13b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:30.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.866+0000 7f52baffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f52bc136830 0x7f52bc138c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51884/0 (socket says 172.21.15.3:51884) 2023-10-03T10:19:30.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.866+0000 7f52bbfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f52bc139150 0x7f52bc13b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51098/0 (socket says 172.21.15.3:51098) 2023-10-03T10:19:30.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.866+0000 7f52baffd700 1 -- 172.21.15.3:0/1947119333 learned_addr learned my addr 172.21.15.3:0/1947119333 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:30.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.866+0000 7f52baffd700 1 -- 172.21.15.3:0/1947119333 >> v2:172.21.15.3:3300/0 conn(0x7f52bc139150 msgr2=0x7f52bc13b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:30.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.866+0000 7f52baffd700 1 --2- 172.21.15.3:0/1947119333 >> v2:172.21.15.3:3300/0 conn(0x7f52bc139150 0x7f52bc13b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:30.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.866+0000 7f52baffd700 1 -- 172.21.15.3:0/1947119333 >> v1:172.21.15.3:6789/0 conn(0x7f52bc1338c0 legacy=0x7f52bc133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:30.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.866+0000 7f52baffd700 1 -- 172.21.15.3:0/1947119333 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52bc1336a0 con 0x7f52bc136830 2023-10-03T10:19:30.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.866+0000 7f52bbfff700 1 --2- 172.21.15.3:0/1947119333 >> v2:172.21.15.3:3300/0 conn(0x7f52bc139150 0x7f52bc13b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:30.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.866+0000 7f52baffd700 1 --2- 172.21.15.3:0/1947119333 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f52bc136830 0x7f52bc138c10 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f52b00098f0 tx=0x7f52b0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80e8da1eeaeef816 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:30.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.867+0000 7f52ba7fc700 1 -- 172.21.15.3:0/1947119333 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b0011830 con 0x7f52bc136830 2023-10-03T10:19:30.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.867+0000 7f52ba7fc700 1 -- 172.21.15.3:0/1947119333 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b0011990 con 0x7f52bc136830 2023-10-03T10:19:30.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.867+0000 7f52ba7fc700 1 -- 172.21.15.3:0/1947119333 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b0011d20 con 0x7f52bc136830 2023-10-03T10:19:30.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.868+0000 7f52c1f8c700 1 -- 172.21.15.3:0/1947119333 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f52bc136830 msgr2=0x7f52bc138c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:30.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.868+0000 7f52c1f8c700 1 --2- 172.21.15.3:0/1947119333 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f52bc136830 0x7f52bc138c10 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f52b00098f0 tx=0x7f52b0009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:30.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.868+0000 7f52c1f8c700 1 -- 172.21.15.3:0/1947119333 shutdown_connections 2023-10-03T10:19:30.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.868+0000 7f52c1f8c700 1 --2- 172.21.15.3:0/1947119333 >> v2:172.21.15.3:3300/0 conn(0x7f52bc139150 0x7f52bc13b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:30.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.868+0000 7f52c1f8c700 1 --2- 172.21.15.3:0/1947119333 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f52bc136830 0x7f52bc138c10 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:30.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.868+0000 7f52c1f8c700 1 -- 172.21.15.3:0/1947119333 >> 172.21.15.3:0/1947119333 conn(0x7f52bc0a2aa0 msgr2=0x7f52bc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:30.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.868+0000 7f52c1f8c700 1 -- 172.21.15.3:0/1947119333 shutdown_connections 2023-10-03T10:19:30.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.868+0000 7f52c1f8c700 1 -- 172.21.15.3:0/1947119333 wait complete. 2023-10-03T10:19:30.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.869+0000 7f52c1f8c700 1 Processor -- start 2023-10-03T10:19:30.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.870+0000 7f52c1f8c700 1 -- start start 2023-10-03T10:19:30.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.870+0000 7f52c1f8c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52bc1338c0 0x7f52bc1c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:30.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.870+0000 7f52c1f8c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f52bc136830 0x7f52bc1c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:30.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.870+0000 7f52c1f8c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f52bc139150 0x7f52bc118f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:30.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.870+0000 7f52c1f8c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc1c91f0 con 0x7f52bc1338c0 2023-10-03T10:19:30.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.870+0000 7f52c1f8c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc11a930 con 0x7f52bc136830 2023-10-03T10:19:30.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.870+0000 7f52c1f8c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc06f9d0 con 0x7f52bc139150 2023-10-03T10:19:30.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.870+0000 7f52bb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52bc1338c0 0x7f52bc1c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:30.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.870+0000 7f52bb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52bc1338c0 0x7f52bc1c8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51106/0 (socket says 172.21.15.3:51106) 2023-10-03T10:19:30.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.870+0000 7f52bb7fe700 1 -- 172.21.15.3:0/4055954746 learned_addr learned my addr 172.21.15.3:0/4055954746 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:30.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.870+0000 7f52baffd700 1 --2- 172.21.15.3:0/4055954746 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f52bc136830 0x7f52bc1c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:30.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.871+0000 7f52bbfff700 1 --2- 172.21.15.3:0/4055954746 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f52bc139150 0x7f52bc118f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:30.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.871+0000 7f52baffd700 1 -- 172.21.15.3:0/4055954746 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f52bc139150 msgr2=0x7f52bc118f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:30.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.871+0000 7f52baffd700 1 --2- 172.21.15.3:0/4055954746 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f52bc139150 0x7f52bc118f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:30.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.871+0000 7f52baffd700 1 -- 172.21.15.3:0/4055954746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52bc1338c0 msgr2=0x7f52bc1c8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:30.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.871+0000 7f52baffd700 1 --2- 172.21.15.3:0/4055954746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52bc1338c0 0x7f52bc1c8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:30.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.871+0000 7f52baffd700 1 -- 172.21.15.3:0/4055954746 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b4009a70 con 0x7f52bc136830 2023-10-03T10:19:30.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.871+0000 7f52bb7fe700 1 --2- 172.21.15.3:0/4055954746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52bc1338c0 0x7f52bc1c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:30.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.871+0000 7f52baffd700 1 --2- 172.21.15.3:0/4055954746 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f52bc136830 0x7f52bc1c8cb0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f52b0016000 tx=0x7f52b0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:30.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.871+0000 7f52a3fff700 1 -- 172.21.15.3:0/4055954746 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b0004400 con 0x7f52bc136830 2023-10-03T10:19:30.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.871+0000 7f52a3fff700 1 -- 172.21.15.3:0/4055954746 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b001f070 con 0x7f52bc136830 2023-10-03T10:19:30.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.871+0000 7f52c1f8c700 1 -- 172.21.15.3:0/4055954746 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b0009550 con 0x7f52bc136830 2023-10-03T10:19:30.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.872+0000 7f52c1f8c700 1 -- 172.21.15.3:0/4055954746 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52bc1360d0 con 0x7f52bc136830 2023-10-03T10:19:30.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.872+0000 7f52a3fff700 1 -- 172.21.15.3:0/4055954746 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b00038e0 con 0x7f52bc136830 2023-10-03T10:19:30.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.873+0000 7f52a3fff700 1 -- 172.21.15.3:0/4055954746 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f52b0021030 con 0x7f52bc136830 2023-10-03T10:19:30.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.873+0000 7f52a3fff700 1 -- 172.21.15.3:0/4055954746 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f52b0040b60 con 0x7f52bc136830 2023-10-03T10:19:30.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.873+0000 7f52c1f8c700 1 -- 172.21.15.3:0/4055954746 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f529c0052e0 con 0x7f52bc136830 2023-10-03T10:19:30.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.878+0000 7f52a3fff700 1 -- 172.21.15.3:0/4055954746 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f52b00199a0 con 0x7f52bc136830 2023-10-03T10:19:30.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.911+0000 7f24b3172700 1 Processor -- start 2023-10-03T10:19:30.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.911+0000 7f24b3172700 1 -- start start 2023-10-03T10:19:30.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.911+0000 7f24b3172700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f24ac06f5d0 0x7f24ac12ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:30.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.911+0000 7f24b3172700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24ac135c20 0x7f24ac138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:30.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.912+0000 7f24b3172700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24ac06fa40 con 0x7f24ac12cf60 2023-10-03T10:19:30.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.912+0000 7f24b3172700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24ac12cd90 con 0x7f24ac06f5d0 2023-10-03T10:19:30.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.912+0000 7f24b3172700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac071780 con 0x7f24ac135c20 2023-10-03T10:19:30.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.912+0000 7f24b170f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24ac135c20 0x7f24ac138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:30.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.912+0000 7f24b170f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24ac135c20 0x7f24ac138000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36806/0 (socket says 172.21.15.3:36806) 2023-10-03T10:19:30.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.912+0000 7f24b170f700 1 -- 172.21.15.3:0/1438319096 learned_addr learned my addr 172.21.15.3:0/1438319096 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:30.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.912+0000 7f24b0f0e700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f24ac12cf60 0x7f24ac12f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33238/0 (socket says 172.21.15.3:33238) 2023-10-03T10:19:30.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.912+0000 7f24abfff700 1 --2- 172.21.15.3:0/1438319096 >> v2:172.21.15.3:3300/0 conn(0x7f24ac06f5d0 0x7f24ac12ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:30.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.912+0000 7f24b170f700 1 -- 172.21.15.3:0/1438319096 >> v2:172.21.15.3:3300/0 conn(0x7f24ac06f5d0 msgr2=0x7f24ac12ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:30.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.912+0000 7f24b170f700 1 --2- 172.21.15.3:0/1438319096 >> v2:172.21.15.3:3300/0 conn(0x7f24ac06f5d0 0x7f24ac12ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:30.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.912+0000 7f24b170f700 1 -- 172.21.15.3:0/1438319096 >> v1:172.21.15.3:6789/0 conn(0x7f24ac12cf60 legacy=0x7f24ac12f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:30.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.912+0000 7f24b170f700 1 -- 172.21.15.3:0/1438319096 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ac071a40 con 0x7f24ac135c20 2023-10-03T10:19:30.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.912+0000 7f24abfff700 1 --2- 172.21.15.3:0/1438319096 >> v2:172.21.15.3:3300/0 conn(0x7f24ac06f5d0 0x7f24ac12ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:30.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.913+0000 7f24b170f700 1 --2- 172.21.15.3:0/1438319096 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24ac135c20 0x7f24ac138000 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009e50 tx=0x7f24a400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba319df6b62fda94 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:30.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.913+0000 7f24ab7fe700 1 -- 172.21.15.3:0/1438319096 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a400e070 con 0x7f24ac135c20 2023-10-03T10:19:30.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.913+0000 7f24ab7fe700 1 -- 172.21.15.3:0/1438319096 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a400b910 con 0x7f24ac135c20 2023-10-03T10:19:30.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.913+0000 7f24ab7fe700 1 -- 172.21.15.3:0/1438319096 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a400bcc0 con 0x7f24ac135c20 2023-10-03T10:19:30.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.914+0000 7f24b3172700 1 -- 172.21.15.3:0/1438319096 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24ac135c20 msgr2=0x7f24ac138000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:30.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.914+0000 7f24b3172700 1 --2- 172.21.15.3:0/1438319096 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24ac135c20 0x7f24ac138000 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009e50 tx=0x7f24a400b330 comp rx=0 tx=0).stop 2023-10-03T10:19:30.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.914+0000 7f24b3172700 1 -- 172.21.15.3:0/1438319096 shutdown_connections 2023-10-03T10:19:30.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.914+0000 7f24b3172700 1 --2- 172.21.15.3:0/1438319096 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24ac135c20 0x7f24ac138000 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:30.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.914+0000 7f24b3172700 1 --2- 172.21.15.3:0/1438319096 >> v2:172.21.15.3:3300/0 conn(0x7f24ac06f5d0 0x7f24ac12ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:30.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.914+0000 7f24b3172700 1 -- 172.21.15.3:0/1438319096 >> 172.21.15.3:0/1438319096 conn(0x7f24ac0a2aa0 msgr2=0x7f24ac0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:30.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.914+0000 7f24b3172700 1 -- 172.21.15.3:0/1438319096 shutdown_connections 2023-10-03T10:19:30.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.914+0000 7f24b3172700 1 -- 172.21.15.3:0/1438319096 wait complete. 2023-10-03T10:19:30.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.915+0000 7f24b3172700 1 Processor -- start 2023-10-03T10:19:30.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.916+0000 7f24b3172700 1 -- start start 2023-10-03T10:19:30.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.916+0000 7f24b3172700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24ac06f5d0 0x7f24ac1c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:30.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.916+0000 7f24b3172700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24ac12cf60 0x7f24ac1c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:30.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.917+0000 7f24b3172700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24ac135c20 0x7f24ac1c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:30.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.917+0000 7f24b3172700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac131d40 con 0x7f24ac12cf60 2023-10-03T10:19:30.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.917+0000 7f24b3172700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac129fa0 con 0x7f24ac135c20 2023-10-03T10:19:30.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.917+0000 7f24b3172700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac135000 con 0x7f24ac06f5d0 2023-10-03T10:19:30.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.917+0000 7f24b0f0e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24ac06f5d0 0x7f24ac1c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.917+0000 7f24b0f0e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24ac06f5d0 0x7f24ac1c86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36822/0 (socket says 172.21.15.3:36822) 2023-10-03T10:19:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.917+0000 7f24b0f0e700 1 -- 172.21.15.3:0/3094820903 learned_addr learned my addr 172.21.15.3:0/3094820903 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.917+0000 7f24abfff700 1 --2- 172.21.15.3:0/3094820903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24ac12cf60 0x7f24ac1c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.917+0000 7f24b170f700 1 --2- 172.21.15.3:0/3094820903 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24ac135c20 0x7f24ac1c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.917+0000 7f24b0f0e700 1 -- 172.21.15.3:0/3094820903 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24ac135c20 msgr2=0x7f24ac1c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.917+0000 7f24b0f0e700 1 --2- 172.21.15.3:0/3094820903 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24ac135c20 0x7f24ac1c9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.917+0000 7f24b0f0e700 1 -- 172.21.15.3:0/3094820903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24ac12cf60 msgr2=0x7f24ac1c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.917+0000 7f24b0f0e700 1 --2- 172.21.15.3:0/3094820903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24ac12cf60 0x7f24ac1c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.917+0000 7f24b0f0e700 1 -- 172.21.15.3:0/3094820903 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24980094a0 con 0x7f24ac06f5d0 2023-10-03T10:19:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.917+0000 7f24b0f0e700 1 --2- 172.21.15.3:0/3094820903 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24ac06f5d0 0x7f24ac1c86d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f24a000c550 tx=0x7f24a000c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.917+0000 7f24b170f700 1 --2- 172.21.15.3:0/3094820903 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24ac135c20 0x7f24ac1c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.918+0000 7f24a97fa700 1 -- 172.21.15.3:0/3094820903 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0005cb0 con 0x7f24ac06f5d0 2023-10-03T10:19:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.918+0000 7f24a97fa700 1 -- 172.21.15.3:0/3094820903 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a0005e10 con 0x7f24ac06f5d0 2023-10-03T10:19:30.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.918+0000 7f24a97fa700 1 -- 172.21.15.3:0/3094820903 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a00072d0 con 0x7f24ac06f5d0 2023-10-03T10:19:30.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.918+0000 7f24b3172700 1 -- 172.21.15.3:0/3094820903 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a4009b30 con 0x7f24ac06f5d0 2023-10-03T10:19:30.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.918+0000 7f24b3172700 1 -- 172.21.15.3:0/3094820903 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24ac13bec0 con 0x7f24ac06f5d0 2023-10-03T10:19:30.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.919+0000 7f24a97fa700 1 -- 172.21.15.3:0/3094820903 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f24a0007430 con 0x7f24ac06f5d0 2023-10-03T10:19:30.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.919+0000 7f24a97fa700 1 -- 172.21.15.3:0/3094820903 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f24a003ebd0 con 0x7f24ac06f5d0 2023-10-03T10:19:30.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.920+0000 7f24b3172700 1 -- 172.21.15.3:0/3094820903 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24ac074ea0 con 0x7f24ac06f5d0 2023-10-03T10:19:30.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.929+0000 7f24a97fa700 1 -- 172.21.15.3:0/3094820903 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f24a0017080 con 0x7f24ac06f5d0 2023-10-03T10:19:30.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.948+0000 7f2783587700 1 Processor -- start 2023-10-03T10:19:30.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.948+0000 7f2783587700 1 -- start start 2023-10-03T10:19:30.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.949+0000 7f2783587700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f277c09d9f0 0x7f277c136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:30.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.949+0000 7f2783587700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f277c136a70 0x7f277c138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:30.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.949+0000 7f2783587700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f277c09e130 con 0x7f277c1316c0 2023-10-03T10:19:30.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.949+0000 7f2783587700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f277c1321a0 con 0x7f277c09d9f0 2023-10-03T10:19:30.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.949+0000 7f2783587700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277c1312b0 con 0x7f277c136a70 2023-10-03T10:19:30.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.949+0000 7f2780b22700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f277c09d9f0 0x7f277c136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:30.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.949+0000 7f2780b22700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f277c09d9f0 0x7f277c136530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51118/0 (socket says 172.21.15.3:51118) 2023-10-03T10:19:30.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.949+0000 7f2780b22700 1 -- 172.21.15.3:0/3907322245 learned_addr learned my addr 172.21.15.3:0/3907322245 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:30.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.949+0000 7f2781b24700 1 --2- 172.21.15.3:0/3907322245 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f277c136a70 0x7f277c138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:30.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.950+0000 7f2780b22700 1 -- 172.21.15.3:0/3907322245 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f277c136a70 msgr2=0x7f277c138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:30.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.950+0000 7f2780b22700 1 --2- 172.21.15.3:0/3907322245 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f277c136a70 0x7f277c138e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:30.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.950+0000 7f2780b22700 1 -- 172.21.15.3:0/3907322245 >> v1:172.21.15.3:6789/0 conn(0x7f277c1316c0 legacy=0x7f277c131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:30.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.950+0000 7f2780b22700 1 -- 172.21.15.3:0/3907322245 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f277c09f6e0 con 0x7f277c09d9f0 2023-10-03T10:19:30.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.950+0000 7f2780b22700 1 --2- 172.21.15.3:0/3907322245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f277c09d9f0 0x7f277c136530 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f276c0098f0 tx=0x7f276c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34abe20b2e8efb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:30.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.950+0000 7f2773fff700 1 -- 172.21.15.3:0/3907322245 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c011830 con 0x7f277c09d9f0 2023-10-03T10:19:30.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.950+0000 7f2773fff700 1 -- 172.21.15.3:0/3907322245 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276c011990 con 0x7f277c09d9f0 2023-10-03T10:19:30.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.950+0000 7f2773fff700 1 -- 172.21.15.3:0/3907322245 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c011d20 con 0x7f277c09d9f0 2023-10-03T10:19:30.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.951+0000 7f2783587700 1 -- 172.21.15.3:0/3907322245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f277c09d9f0 msgr2=0x7f277c136530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:30.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.951+0000 7f2783587700 1 --2- 172.21.15.3:0/3907322245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f277c09d9f0 0x7f277c136530 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f276c0098f0 tx=0x7f276c009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:30.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.951+0000 7f2783587700 1 -- 172.21.15.3:0/3907322245 shutdown_connections 2023-10-03T10:19:30.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.951+0000 7f2783587700 1 --2- 172.21.15.3:0/3907322245 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f277c136a70 0x7f277c138e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:30.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.951+0000 7f2783587700 1 --2- 172.21.15.3:0/3907322245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f277c09d9f0 0x7f277c136530 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:30.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.951+0000 7f2783587700 1 -- 172.21.15.3:0/3907322245 >> 172.21.15.3:0/3907322245 conn(0x7f277c066f50 msgr2=0x7f277c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:30.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.952+0000 7f2783587700 1 -- 172.21.15.3:0/3907322245 shutdown_connections 2023-10-03T10:19:30.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.952+0000 7f2783587700 1 -- 172.21.15.3:0/3907322245 wait complete. 2023-10-03T10:19:30.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.953+0000 7f2783587700 1 Processor -- start 2023-10-03T10:19:30.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.953+0000 7f2783587700 1 -- start start 2023-10-03T10:19:30.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.954+0000 7f2783587700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f277c09d9f0 0x7f277c070de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:30.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.954+0000 7f2783587700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f277c1316c0 0x7f277c071320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:30.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.954+0000 7f2783587700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f277c136a70 0x7f277c071860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:30.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.954+0000 7f2783587700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c06f730 con 0x7f277c09d9f0 2023-10-03T10:19:30.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.954+0000 7f2783587700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277c138e60 con 0x7f277c1316c0 2023-10-03T10:19:30.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.954+0000 7f2783587700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f277c12b900 con 0x7f277c136a70 2023-10-03T10:19:30.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.954+0000 7f2780b22700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f277c1316c0 0x7f277c071320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:30.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.954+0000 7f2781b24700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f277c136a70 0x7f277c071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:30.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.954+0000 7f2780b22700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f277c1316c0 0x7f277c071320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51916/0 (socket says 172.21.15.3:51916) 2023-10-03T10:19:30.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.954+0000 7f2780b22700 1 -- 172.21.15.3:0/734084887 learned_addr learned my addr 172.21.15.3:0/734084887 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:30.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.954+0000 7f2781323700 1 --2- 172.21.15.3:0/734084887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f277c09d9f0 0x7f277c070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:30.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.955+0000 7f2781b24700 1 -- 172.21.15.3:0/734084887 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f277c1316c0 msgr2=0x7f277c071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:30.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.955+0000 7f2781b24700 1 --2- 172.21.15.3:0/734084887 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f277c1316c0 0x7f277c071320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:30.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.955+0000 7f2781b24700 1 -- 172.21.15.3:0/734084887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f277c09d9f0 msgr2=0x7f277c070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:30.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.955+0000 7f2781b24700 1 --2- 172.21.15.3:0/734084887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f277c09d9f0 0x7f277c070de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:30.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.955+0000 7f2781b24700 1 -- 172.21.15.3:0/734084887 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f276c009550 con 0x7f277c136a70 2023-10-03T10:19:30.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.955+0000 7f2781b24700 1 --2- 172.21.15.3:0/734084887 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f277c136a70 0x7f277c071860 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f277400ef90 tx=0x7f277400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:30.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.955+0000 7f2771ffb700 1 -- 172.21.15.3:0/734084887 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277400cc90 con 0x7f277c136a70 2023-10-03T10:19:30.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.955+0000 7f2771ffb700 1 -- 172.21.15.3:0/734084887 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277400f040 con 0x7f277c136a70 2023-10-03T10:19:30.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.955+0000 7f2783587700 1 -- 172.21.15.3:0/734084887 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f277c06fc60 con 0x7f277c136a70 2023-10-03T10:19:30.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.955+0000 7f2771ffb700 1 -- 172.21.15.3:0/734084887 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27740146c0 con 0x7f277c136a70 2023-10-03T10:19:30.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.956+0000 7f2783587700 1 -- 172.21.15.3:0/734084887 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f277c070990 con 0x7f277c136a70 2023-10-03T10:19:30.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.956+0000 7f2771ffb700 1 -- 172.21.15.3:0/734084887 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2774014820 con 0x7f277c136a70 2023-10-03T10:19:30.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.957+0000 7f2771ffb700 1 -- 172.21.15.3:0/734084887 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f27740401a0 con 0x7f277c136a70 2023-10-03T10:19:30.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.957+0000 7f2783587700 1 -- 172.21.15.3:0/734084887 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f277c06fe80 con 0x7f277c136a70 2023-10-03T10:19:30.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:30.966+0000 7f2771ffb700 1 -- 172.21.15.3:0/734084887 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f277401d400 con 0x7f277c136a70 2023-10-03T10:19:31.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.003+0000 7f1b72ea4700 1 Processor -- start 2023-10-03T10:19:31.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.003+0000 7f1b72ea4700 1 -- start start 2023-10-03T10:19:31.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.003+0000 7f1b72ea4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1b6c09f330 0x7f1b6c138ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:31.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.003+0000 7f1b72ea4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b6c1393e0 0x7f1b6c13b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:31.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.003+0000 7f1b72ea4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b6c09f960 con 0x7f1b6c09e850 2023-10-03T10:19:31.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.003+0000 7f1b72ea4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b6c09f720 con 0x7f1b6c09f330 2023-10-03T10:19:31.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.003+0000 7f1b72ea4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c09ae60 con 0x7f1b6c1393e0 2023-10-03T10:19:31.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.004+0000 7f1b6bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1b6c09f330 0x7f1b6c138ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:31.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.004+0000 7f1b6bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1b6c09f330 0x7f1b6c138ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51142/0 (socket says 172.21.15.3:51142) 2023-10-03T10:19:31.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.004+0000 7f1b70c40700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1b6c09e850 0x7f1b6c09ec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33266/0 (socket says 172.21.15.3:33266) 2023-10-03T10:19:31.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.004+0000 7f1b6bfff700 1 -- 172.21.15.3:0/2471992211 learned_addr learned my addr 172.21.15.3:0/2471992211 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:31.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.004+0000 7f1b6bfff700 1 -- 172.21.15.3:0/2471992211 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b6c1393e0 msgr2=0x7f1b6c13b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:31.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.004+0000 7f1b6bfff700 1 --2- 172.21.15.3:0/2471992211 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b6c1393e0 0x7f1b6c13b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:31.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.004+0000 7f1b6bfff700 1 -- 172.21.15.3:0/2471992211 >> v1:172.21.15.3:6789/0 conn(0x7f1b6c09e850 legacy=0x7f1b6c09ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:31.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.004+0000 7f1b6bfff700 1 -- 172.21.15.3:0/2471992211 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6c06cb10 con 0x7f1b6c09f330 2023-10-03T10:19:31.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.004+0000 7f1b6bfff700 1 --2- 172.21.15.3:0/2471992211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b6c09f330 0x7f1b6c138ea0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f1b580098f0 tx=0x7f1b58009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a233e69f7206ba87 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:31.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.004+0000 7f1b6b7fe700 1 -- 172.21.15.3:0/2471992211 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5800b8a0 con 0x7f1b6c09f330 2023-10-03T10:19:31.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.005+0000 7f1b6b7fe700 1 -- 172.21.15.3:0/2471992211 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b5800ba00 con 0x7f1b6c09f330 2023-10-03T10:19:31.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.005+0000 7f1b6b7fe700 1 -- 172.21.15.3:0/2471992211 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5800bd90 con 0x7f1b6c09f330 2023-10-03T10:19:31.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.005+0000 7f1b72ea4700 1 -- 172.21.15.3:0/2471992211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b6c09f330 msgr2=0x7f1b6c138ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:31.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.005+0000 7f1b72ea4700 1 --2- 172.21.15.3:0/2471992211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b6c09f330 0x7f1b6c138ea0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f1b580098f0 tx=0x7f1b58009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:31.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.005+0000 7f1b72ea4700 1 -- 172.21.15.3:0/2471992211 shutdown_connections 2023-10-03T10:19:31.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.005+0000 7f1b72ea4700 1 --2- 172.21.15.3:0/2471992211 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b6c1393e0 0x7f1b6c13b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:31.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.005+0000 7f1b72ea4700 1 --2- 172.21.15.3:0/2471992211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b6c09f330 0x7f1b6c138ea0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:31.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.005+0000 7f1b72ea4700 1 -- 172.21.15.3:0/2471992211 >> 172.21.15.3:0/2471992211 conn(0x7f1b6c0a3f70 msgr2=0x7f1b6c0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:31.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.005+0000 7f1b72ea4700 1 -- 172.21.15.3:0/2471992211 shutdown_connections 2023-10-03T10:19:31.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.005+0000 7f1b72ea4700 1 -- 172.21.15.3:0/2471992211 wait complete. 2023-10-03T10:19:31.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.006+0000 7f1b72ea4700 1 Processor -- start 2023-10-03T10:19:31.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.006+0000 7f1b72ea4700 1 -- start start 2023-10-03T10:19:31.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.007+0000 7f1b72ea4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b6c09e850 0x7f1b6c1c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:31.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.007+0000 7f1b72ea4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b6c09f330 0x7f1b6c1c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:31.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.007+0000 7f1b72ea4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b6c1393e0 0x7f1b6c11fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:31.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.007+0000 7f1b72ea4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c1234d0 con 0x7f1b6c09f330 2023-10-03T10:19:31.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.007+0000 7f1b72ea4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c06f9e0 con 0x7f1b6c1393e0 2023-10-03T10:19:31.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.007+0000 7f1b72ea4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c123910 con 0x7f1b6c09e850 2023-10-03T10:19:31.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.007+0000 7f1b70c40700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b6c09e850 0x7f1b6c1c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:31.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.007+0000 7f1b70c40700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b6c09e850 0x7f1b6c1c8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36838/0 (socket says 172.21.15.3:36838) 2023-10-03T10:19:31.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.007+0000 7f1b71441700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b6c1393e0 0x7f1b6c11fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:31.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.007+0000 7f1b70c40700 1 -- 172.21.15.3:0/3794668687 learned_addr learned my addr 172.21.15.3:0/3794668687 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:31.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.007+0000 7f1b6bfff700 1 --2- 172.21.15.3:0/3794668687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b6c09f330 0x7f1b6c1c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:31.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.008+0000 7f1b70c40700 1 -- 172.21.15.3:0/3794668687 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b6c1393e0 msgr2=0x7f1b6c11fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:31.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.008+0000 7f1b70c40700 1 --2- 172.21.15.3:0/3794668687 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b6c1393e0 0x7f1b6c11fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:31.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.008+0000 7f1b70c40700 1 -- 172.21.15.3:0/3794668687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b6c09f330 msgr2=0x7f1b6c1c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:31.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.008+0000 7f1b70c40700 1 --2- 172.21.15.3:0/3794668687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b6c09f330 0x7f1b6c1c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:31.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.008+0000 7f1b70c40700 1 -- 172.21.15.3:0/3794668687 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b58009550 con 0x7f1b6c09e850 2023-10-03T10:19:31.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.008+0000 7f1b71441700 1 --2- 172.21.15.3:0/3794668687 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b6c1393e0 0x7f1b6c11fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:31.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.008+0000 7f1b70c40700 1 --2- 172.21.15.3:0/3794668687 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b6c09e850 0x7f1b6c1c8990 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000c550 tx=0x7f1b6000c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:31.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.008+0000 7f1b697fa700 1 -- 172.21.15.3:0/3794668687 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b60005cb0 con 0x7f1b6c09e850 2023-10-03T10:19:31.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.008+0000 7f1b697fa700 1 -- 172.21.15.3:0/3794668687 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b60005e10 con 0x7f1b6c09e850 2023-10-03T10:19:31.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.008+0000 7f1b697fa700 1 -- 172.21.15.3:0/3794668687 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b600072d0 con 0x7f1b6c09e850 2023-10-03T10:19:31.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.008+0000 7f1b72ea4700 1 -- 172.21.15.3:0/3794668687 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b6c06cd30 con 0x7f1b6c09e850 2023-10-03T10:19:31.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.008+0000 7f1b72ea4700 1 -- 172.21.15.3:0/3794668687 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6c1be630 con 0x7f1b6c09e850 2023-10-03T10:19:31.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.009+0000 7f1b697fa700 1 -- 172.21.15.3:0/3794668687 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1b60007430 con 0x7f1b6c09e850 2023-10-03T10:19:31.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.009+0000 7f1b697fa700 1 -- 172.21.15.3:0/3794668687 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1b6003eb40 con 0x7f1b6c09e850 2023-10-03T10:19:31.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.010+0000 7f1b72ea4700 1 -- 172.21.15.3:0/3794668687 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b6c0747f0 con 0x7f1b6c09e850 2023-10-03T10:19:31.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.015+0000 7f1b697fa700 1 -- 172.21.15.3:0/3794668687 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1b6001b3b0 con 0x7f1b6c09e850 2023-10-03T10:19:31.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.131+0000 7f52c1f8c700 1 -- 172.21.15.3:0/4055954746 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f529c005590 con 0x7f52bc136830 2023-10-03T10:19:31.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.131+0000 7f52a3fff700 1 -- 172.21.15.3:0/4055954746 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f52b0015030 con 0x7f52bc136830 2023-10-03T10:19:31.132 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:31.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.135+0000 7f52c1f8c700 1 -- 172.21.15.3:0/4055954746 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f52bc136830 msgr2=0x7f52bc1c8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:31.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.135+0000 7f52c1f8c700 1 --2- 172.21.15.3:0/4055954746 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f52bc136830 0x7f52bc1c8cb0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f52b0016000 tx=0x7f52b0004750 comp rx=0 tx=0).stop 2023-10-03T10:19:31.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.135+0000 7f52c1f8c700 1 -- 172.21.15.3:0/4055954746 shutdown_connections 2023-10-03T10:19:31.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.135+0000 7f52c1f8c700 1 --2- 172.21.15.3:0/4055954746 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f52bc139150 0x7f52bc118f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:31.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.135+0000 7f52c1f8c700 1 --2- 172.21.15.3:0/4055954746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52bc1338c0 0x7f52bc1c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:31.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.135+0000 7f52c1f8c700 1 --2- 172.21.15.3:0/4055954746 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f52bc136830 0x7f52bc1c8cb0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:31.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.135+0000 7f52c1f8c700 1 -- 172.21.15.3:0/4055954746 >> 172.21.15.3:0/4055954746 conn(0x7f52bc0a2aa0 msgr2=0x7f52bc12f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:31.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.135+0000 7f52c1f8c700 1 -- 172.21.15.3:0/4055954746 shutdown_connections 2023-10-03T10:19:31.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.135+0000 7f52c1f8c700 1 -- 172.21.15.3:0/4055954746 wait complete. 2023-10-03T10:19:31.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.141+0000 7f24b3172700 1 -- 172.21.15.3:0/3094820903 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f24ac0713a0 con 0x7f24ac06f5d0 2023-10-03T10:19:31.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.141+0000 7f24a97fa700 1 -- 172.21.15.3:0/3094820903 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f24a0012400 con 0x7f24ac06f5d0 2023-10-03T10:19:31.141 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:31.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.145+0000 7f24b3172700 1 -- 172.21.15.3:0/3094820903 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24ac06f5d0 msgr2=0x7f24ac1c86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:31.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.145+0000 7f24b3172700 1 --2- 172.21.15.3:0/3094820903 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24ac06f5d0 0x7f24ac1c86d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f24a000c550 tx=0x7f24a000c860 comp rx=0 tx=0).stop 2023-10-03T10:19:31.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.146+0000 7f24b3172700 1 -- 172.21.15.3:0/3094820903 shutdown_connections 2023-10-03T10:19:31.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.146+0000 7f24b3172700 1 --2- 172.21.15.3:0/3094820903 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f24ac135c20 0x7f24ac1c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:31.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.146+0000 7f24b3172700 1 --2- 172.21.15.3:0/3094820903 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f24ac06f5d0 0x7f24ac1c86d0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:31.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.146+0000 7f24b3172700 1 --2- 172.21.15.3:0/3094820903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24ac12cf60 0x7f24ac1c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:31.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.146+0000 7f24b3172700 1 -- 172.21.15.3:0/3094820903 >> 172.21.15.3:0/3094820903 conn(0x7f24ac0a2aa0 msgr2=0x7f24ac06e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:31.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.147+0000 7f24b3172700 1 -- 172.21.15.3:0/3094820903 shutdown_connections 2023-10-03T10:19:31.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.147+0000 7f24b3172700 1 -- 172.21.15.3:0/3094820903 wait complete. 2023-10-03T10:19:31.155 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:31.167 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:31.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.218+0000 7f2783587700 1 -- 172.21.15.3:0/734084887 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f277c074ea0 con 0x7f277c136a70 2023-10-03T10:19:31.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.219+0000 7f2771ffb700 1 -- 172.21.15.3:0/734084887 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2774012340 con 0x7f277c136a70 2023-10-03T10:19:31.219 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:31.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.221+0000 7f2783587700 1 -- 172.21.15.3:0/734084887 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f277c136a70 msgr2=0x7f277c071860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:31.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.221+0000 7f2783587700 1 --2- 172.21.15.3:0/734084887 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f277c136a70 0x7f277c071860 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f277400ef90 tx=0x7f277400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:19:31.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.221+0000 7f2783587700 1 -- 172.21.15.3:0/734084887 shutdown_connections 2023-10-03T10:19:31.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.221+0000 7f2783587700 1 --2- 172.21.15.3:0/734084887 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f277c136a70 0x7f277c071860 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:31.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.221+0000 7f2783587700 1 --2- 172.21.15.3:0/734084887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f277c09d9f0 0x7f277c070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:31.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.221+0000 7f2783587700 1 --2- 172.21.15.3:0/734084887 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f277c1316c0 0x7f277c071320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:31.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.221+0000 7f2783587700 1 -- 172.21.15.3:0/734084887 >> 172.21.15.3:0/734084887 conn(0x7f277c066f50 msgr2=0x7f277c136e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:31.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.222+0000 7f2783587700 1 -- 172.21.15.3:0/734084887 shutdown_connections 2023-10-03T10:19:31.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.222+0000 7f2783587700 1 -- 172.21.15.3:0/734084887 wait complete. 2023-10-03T10:19:31.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.234+0000 7f1b72ea4700 1 -- 172.21.15.3:0/3794668687 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1b6c06bbb0 con 0x7f1b6c09e850 2023-10-03T10:19:31.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.234+0000 7f1b697fa700 1 -- 172.21.15.3:0/3794668687 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1b60017020 con 0x7f1b6c09e850 2023-10-03T10:19:31.234 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:31.235 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:31.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.237+0000 7f1b72ea4700 1 -- 172.21.15.3:0/3794668687 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b6c09e850 msgr2=0x7f1b6c1c8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:31.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.237+0000 7f1b72ea4700 1 --2- 172.21.15.3:0/3794668687 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b6c09e850 0x7f1b6c1c8990 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000c550 tx=0x7f1b6000c860 comp rx=0 tx=0).stop 2023-10-03T10:19:31.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.237+0000 7f1b72ea4700 1 -- 172.21.15.3:0/3794668687 shutdown_connections 2023-10-03T10:19:31.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.237+0000 7f1b72ea4700 1 --2- 172.21.15.3:0/3794668687 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1b6c1393e0 0x7f1b6c11fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:31.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.237+0000 7f1b72ea4700 1 --2- 172.21.15.3:0/3794668687 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1b6c09e850 0x7f1b6c1c8990 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:31.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.237+0000 7f1b72ea4700 1 --2- 172.21.15.3:0/3794668687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b6c09f330 0x7f1b6c1c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:31.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.237+0000 7f1b72ea4700 1 -- 172.21.15.3:0/3794668687 >> 172.21.15.3:0/3794668687 conn(0x7f1b6c0a3f70 msgr2=0x7f1b6c135db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:31.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.237+0000 7f1b72ea4700 1 -- 172.21.15.3:0/3794668687 shutdown_connections 2023-10-03T10:19:31.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:31.237+0000 7f1b72ea4700 1 -- 172.21.15.3:0/3794668687 wait complete. 2023-10-03T10:19:31.250 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:32.157 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:32.167 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:32.237 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:32.251 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:32.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.310+0000 7f8964e06700 1 Processor -- start 2023-10-03T10:19:32.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.310+0000 7f8964e06700 1 -- start start 2023-10-03T10:19:32.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.311+0000 7f8964e06700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f896006a770 0x7f896009d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:32.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.311+0000 7f8964e06700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f896013d1f0 0x7f896013f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:32.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.311+0000 7f8964e06700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f896009a810 con 0x7f896009dc30 2023-10-03T10:19:32.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.311+0000 7f8964e06700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f896006ab40 con 0x7f896013d1f0 2023-10-03T10:19:32.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.311+0000 7f8964e06700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f896009ebc0 con 0x7f896006a770 2023-10-03T10:19:32.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.311+0000 7f895e59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f896006a770 0x7f896009d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:32.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.311+0000 7f895dd9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f896009dc30 0x7f896009e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33272/0 (socket says 172.21.15.3:33272) 2023-10-03T10:19:32.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.311+0000 7f895dd9b700 1 -- 172.21.15.3:0/4068034627 learned_addr learned my addr 172.21.15.3:0/4068034627 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.312+0000 7f895ed9d700 1 --2- 172.21.15.3:0/4068034627 >> v2:172.21.15.3:3300/0 conn(0x7f896013d1f0 0x7f896013f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.312+0000 7f895e59c700 1 -- 172.21.15.3:0/4068034627 >> v2:172.21.15.3:3300/0 conn(0x7f896013d1f0 msgr2=0x7f896013f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.312+0000 7f895e59c700 1 --2- 172.21.15.3:0/4068034627 >> v2:172.21.15.3:3300/0 conn(0x7f896013d1f0 0x7f896013f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.312+0000 7f895e59c700 1 -- 172.21.15.3:0/4068034627 >> v1:172.21.15.3:6789/0 conn(0x7f896009dc30 legacy=0x7f896009e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.312+0000 7f895e59c700 1 -- 172.21.15.3:0/4068034627 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f896009e920 con 0x7f896006a770 2023-10-03T10:19:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.312+0000 7f895e59c700 1 --2- 172.21.15.3:0/4068034627 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f896006a770 0x7f896009d6f0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f8950009870 tx=0x7f8950009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c33dab537e829965 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.312+0000 7f895d59a700 1 -- 172.21.15.3:0/4068034627 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950011830 con 0x7f896006a770 2023-10-03T10:19:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.312+0000 7f895d59a700 1 -- 172.21.15.3:0/4068034627 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8950011990 con 0x7f896006a770 2023-10-03T10:19:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.312+0000 7f895d59a700 1 -- 172.21.15.3:0/4068034627 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950011d20 con 0x7f896006a770 2023-10-03T10:19:32.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.313+0000 7f8964e06700 1 -- 172.21.15.3:0/4068034627 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f896006a770 msgr2=0x7f896009d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:32.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.313+0000 7f8964e06700 1 --2- 172.21.15.3:0/4068034627 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f896006a770 0x7f896009d6f0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f8950009870 tx=0x7f8950009b80 comp rx=0 tx=0).stop 2023-10-03T10:19:32.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.313+0000 7f8964e06700 1 -- 172.21.15.3:0/4068034627 shutdown_connections 2023-10-03T10:19:32.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.313+0000 7f8964e06700 1 --2- 172.21.15.3:0/4068034627 >> v2:172.21.15.3:3300/0 conn(0x7f896013d1f0 0x7f896013f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.313+0000 7f8964e06700 1 --2- 172.21.15.3:0/4068034627 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f896006a770 0x7f896009d6f0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.313+0000 7f8964e06700 1 -- 172.21.15.3:0/4068034627 >> 172.21.15.3:0/4068034627 conn(0x7f8960066f50 msgr2=0x7f896009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:32.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.313+0000 7f8964e06700 1 -- 172.21.15.3:0/4068034627 shutdown_connections 2023-10-03T10:19:32.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.313+0000 7f8964e06700 1 -- 172.21.15.3:0/4068034627 wait complete. 2023-10-03T10:19:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.315+0000 7f8964e06700 1 Processor -- start 2023-10-03T10:19:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.315+0000 7f8964e06700 1 -- start start 2023-10-03T10:19:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.315+0000 7f8964e06700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f896006a770 0x7f89601260a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.315+0000 7f8964e06700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f896009dc30 0x7f89601265e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:32.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.315+0000 7f8964e06700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f896013d1f0 0x7f8960126b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:32.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.315+0000 7f8964e06700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8960129150 con 0x7f896009dc30 2023-10-03T10:19:32.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.315+0000 7f8964e06700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8960128490 con 0x7f896006a770 2023-10-03T10:19:32.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.315+0000 7f8964e06700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f896009f1a0 con 0x7f896013d1f0 2023-10-03T10:19:32.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.315+0000 7f895dd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f896009dc30 0x7f89601265e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:32.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.315+0000 7f895ed9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f896013d1f0 0x7f8960126b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:32.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.315+0000 7f895dd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f896009dc30 0x7f89601265e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51178/0 (socket says 172.21.15.3:51178) 2023-10-03T10:19:32.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.315+0000 7f895ed9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f896013d1f0 0x7f8960126b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36852/0 (socket says 172.21.15.3:36852) 2023-10-03T10:19:32.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.315+0000 7f895dd9b700 1 -- 172.21.15.3:0/1504971549 learned_addr learned my addr 172.21.15.3:0/1504971549 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:32.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.316+0000 7f895e59c700 1 --2- 172.21.15.3:0/1504971549 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f896006a770 0x7f89601260a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:32.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.316+0000 7f895e59c700 1 -- 172.21.15.3:0/1504971549 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f896013d1f0 msgr2=0x7f8960126b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:32.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.316+0000 7f895e59c700 1 --2- 172.21.15.3:0/1504971549 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f896013d1f0 0x7f8960126b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.316+0000 7f895e59c700 1 -- 172.21.15.3:0/1504971549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f896009dc30 msgr2=0x7f89601265e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:32.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.316+0000 7f895e59c700 1 --2- 172.21.15.3:0/1504971549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f896009dc30 0x7f89601265e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.316+0000 7f895e59c700 1 -- 172.21.15.3:0/1504971549 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89500094a0 con 0x7f896006a770 2023-10-03T10:19:32.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.316+0000 7f895e59c700 1 --2- 172.21.15.3:0/1504971549 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f896006a770 0x7f89601260a0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f8950016000 tx=0x7f8950004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:32.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.316+0000 7f894effd700 1 -- 172.21.15.3:0/1504971549 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89500036a0 con 0x7f896006a770 2023-10-03T10:19:32.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.317+0000 7f894effd700 1 -- 172.21.15.3:0/1504971549 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f895001e070 con 0x7f896006a770 2023-10-03T10:19:32.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.317+0000 7f8964e06700 1 -- 172.21.15.3:0/1504971549 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8960128770 con 0x7f896006a770 2023-10-03T10:19:32.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.317+0000 7f894effd700 1 -- 172.21.15.3:0/1504971549 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950003a40 con 0x7f896006a770 2023-10-03T10:19:32.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.317+0000 7f8964e06700 1 -- 172.21.15.3:0/1504971549 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8960127460 con 0x7f896006a770 2023-10-03T10:19:32.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.318+0000 7f894effd700 1 -- 172.21.15.3:0/1504971549 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8950003ba0 con 0x7f896006a770 2023-10-03T10:19:32.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.318+0000 7f894effd700 1 -- 172.21.15.3:0/1504971549 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f89500405f0 con 0x7f896006a770 2023-10-03T10:19:32.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.318+0000 7f894cff9700 1 -- 172.21.15.3:0/1504971549 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89380052e0 con 0x7f896006a770 2023-10-03T10:19:32.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.323+0000 7f894effd700 1 -- 172.21.15.3:0/1504971549 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8950015020 con 0x7f896006a770 2023-10-03T10:19:32.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.349+0000 7f55725cc700 1 Processor -- start 2023-10-03T10:19:32.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.349+0000 7f55725cc700 1 -- start start 2023-10-03T10:19:32.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.350+0000 7f55725cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f556c0a0a30 0x7f556c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:32.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.350+0000 7f55725cc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f556c0a1340 0x7f556c133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:32.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.350+0000 7f55725cc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f556c133fa0 0x7f556c134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:32.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.350+0000 7f55725cc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f556c09edd0 con 0x7f556c0a1340 2023-10-03T10:19:32.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.350+0000 7f55725cc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f556c09e1f0 con 0x7f556c0a0a30 2023-10-03T10:19:32.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.350+0000 7f55725cc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f556c09dc50 con 0x7f556c133fa0 2023-10-03T10:19:32.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.350+0000 7f5570b69700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f556c133fa0 0x7f556c134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:32.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.350+0000 7f556bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f556c0a0a30 0x7f556c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:32.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.350+0000 7f5570b69700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f556c133fa0 0x7f556c134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36854/0 (socket says 172.21.15.3:36854) 2023-10-03T10:19:32.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.350+0000 7f5570b69700 1 -- 172.21.15.3:0/2394459514 learned_addr learned my addr 172.21.15.3:0/2394459514 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:32.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.350+0000 7f556b7fe700 1 --2- 172.21.15.3:0/2394459514 >> v2:172.21.15.3:3300/0 conn(0x7f556c0a1340 0x7f556c133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:32.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.350+0000 7f5570b69700 1 -- 172.21.15.3:0/2394459514 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f556c0a0a30 msgr2=0x7f556c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:32.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.350+0000 7f5570b69700 1 --2- 172.21.15.3:0/2394459514 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f556c0a0a30 0x7f556c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.350+0000 7f5570b69700 1 -- 172.21.15.3:0/2394459514 >> v2:172.21.15.3:3300/0 conn(0x7f556c0a1340 msgr2=0x7f556c133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:32.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.350+0000 7f5570b69700 1 --2- 172.21.15.3:0/2394459514 >> v2:172.21.15.3:3300/0 conn(0x7f556c0a1340 0x7f556c133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.350+0000 7f5570b69700 1 -- 172.21.15.3:0/2394459514 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f556c0a1fe0 con 0x7f556c133fa0 2023-10-03T10:19:32.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.351+0000 7f556bfff700 1 --2- 172.21.15.3:0/2394459514 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f556c0a0a30 0x7f556c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:32.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.351+0000 7f556b7fe700 1 --2- 172.21.15.3:0/2394459514 >> v2:172.21.15.3:3300/0 conn(0x7f556c0a1340 0x7f556c133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:19:32.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.351+0000 7f5570b69700 1 --2- 172.21.15.3:0/2394459514 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f556c133fa0 0x7f556c134410 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f5564009e80 tx=0x7f556400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5b8fb53f0180dff server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:32.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.351+0000 7f556affd700 1 -- 172.21.15.3:0/2394459514 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556400e070 con 0x7f556c133fa0 2023-10-03T10:19:32.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.351+0000 7f556affd700 1 -- 172.21.15.3:0/2394459514 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f556400b8d0 con 0x7f556c133fa0 2023-10-03T10:19:32.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.351+0000 7f556affd700 1 -- 172.21.15.3:0/2394459514 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556400bc80 con 0x7f556c133fa0 2023-10-03T10:19:32.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.352+0000 7f55725cc700 1 -- 172.21.15.3:0/2394459514 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f556c133fa0 msgr2=0x7f556c134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:32.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.352+0000 7f55725cc700 1 --2- 172.21.15.3:0/2394459514 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f556c133fa0 0x7f556c134410 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f5564009e80 tx=0x7f556400b330 comp rx=0 tx=0).stop 2023-10-03T10:19:32.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.352+0000 7f55725cc700 1 -- 172.21.15.3:0/2394459514 shutdown_connections 2023-10-03T10:19:32.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.352+0000 7f55725cc700 1 --2- 172.21.15.3:0/2394459514 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f556c133fa0 0x7f556c134410 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.352+0000 7f55725cc700 1 --2- 172.21.15.3:0/2394459514 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f556c0a0a30 0x7f556c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.352+0000 7f55725cc700 1 --2- 172.21.15.3:0/2394459514 >> v2:172.21.15.3:3300/0 conn(0x7f556c0a1340 0x7f556c133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.352+0000 7f55725cc700 1 -- 172.21.15.3:0/2394459514 >> 172.21.15.3:0/2394459514 conn(0x7f556c0a6550 msgr2=0x7f556c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:32.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.352+0000 7f55725cc700 1 -- 172.21.15.3:0/2394459514 shutdown_connections 2023-10-03T10:19:32.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.352+0000 7f55725cc700 1 -- 172.21.15.3:0/2394459514 wait complete. 2023-10-03T10:19:32.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.353+0000 7f55725cc700 1 Processor -- start 2023-10-03T10:19:32.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.353+0000 7f55725cc700 1 -- start start 2023-10-03T10:19:32.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.354+0000 7f55725cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f556c0a0a30 0x7f556c120060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:32.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.354+0000 7f55725cc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f556c0a1340 0x7f556c1205a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:32.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.354+0000 7f55725cc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f556c133fa0 0x7f556c120ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:32.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.354+0000 7f55725cc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c0a0220 con 0x7f556c0a1340 2023-10-03T10:19:32.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.354+0000 7f55725cc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f556c121d10 con 0x7f556c0a0a30 2023-10-03T10:19:32.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.354+0000 7f55725cc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f556c1238a0 con 0x7f556c133fa0 2023-10-03T10:19:32.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.354+0000 7f556bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f556c0a0a30 0x7f556c120060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:32.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.354+0000 7f556bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f556c0a0a30 0x7f556c120060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51968/0 (socket says 172.21.15.3:51968) 2023-10-03T10:19:32.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.354+0000 7f556bfff700 1 -- 172.21.15.3:0/482867807 learned_addr learned my addr 172.21.15.3:0/482867807 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:32.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.354+0000 7f5570b69700 1 --2- 172.21.15.3:0/482867807 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f556c133fa0 0x7f556c120ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:32.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.354+0000 7f556b7fe700 1 --2- 172.21.15.3:0/482867807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f556c0a1340 0x7f556c1205a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:32.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.354+0000 7f556bfff700 1 -- 172.21.15.3:0/482867807 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f556c133fa0 msgr2=0x7f556c120ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:32.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.354+0000 7f556bfff700 1 --2- 172.21.15.3:0/482867807 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f556c133fa0 0x7f556c120ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.354+0000 7f556bfff700 1 -- 172.21.15.3:0/482867807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f556c0a1340 msgr2=0x7f556c1205a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:32.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.354+0000 7f556bfff700 1 --2- 172.21.15.3:0/482867807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f556c0a1340 0x7f556c1205a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.354+0000 7f556bfff700 1 -- 172.21.15.3:0/482867807 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f555c0094a0 con 0x7f556c0a0a30 2023-10-03T10:19:32.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.355+0000 7f556bfff700 1 --2- 172.21.15.3:0/482867807 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f556c0a0a30 0x7f556c120060 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f555c00ebe0 tx=0x7f555c00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:32.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.355+0000 7f5568ff9700 1 -- 172.21.15.3:0/482867807 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c00c850 con 0x7f556c0a0a30 2023-10-03T10:19:32.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.355+0000 7f5568ff9700 1 -- 172.21.15.3:0/482867807 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f555c00c9b0 con 0x7f556c0a0a30 2023-10-03T10:19:32.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.355+0000 7f5568ff9700 1 -- 172.21.15.3:0/482867807 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c00cd40 con 0x7f556c0a0a30 2023-10-03T10:19:32.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.355+0000 7f55725cc700 1 -- 172.21.15.3:0/482867807 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5564009b30 con 0x7f556c0a0a30 2023-10-03T10:19:32.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.355+0000 7f55725cc700 1 -- 172.21.15.3:0/482867807 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f556c122ec0 con 0x7f556c0a0a30 2023-10-03T10:19:32.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.356+0000 7f5568ff9700 1 -- 172.21.15.3:0/482867807 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f555c013470 con 0x7f556c0a0a30 2023-10-03T10:19:32.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.356+0000 7f55725cc700 1 -- 172.21.15.3:0/482867807 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f556c074ea0 con 0x7f556c0a0a30 2023-10-03T10:19:32.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.356+0000 7f5568ff9700 1 -- 172.21.15.3:0/482867807 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f555c03fba0 con 0x7f556c0a0a30 2023-10-03T10:19:32.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.365+0000 7f5568ff9700 1 -- 172.21.15.3:0/482867807 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f555c00cea0 con 0x7f556c0a0a30 2023-10-03T10:19:32.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.399+0000 7f35de23f700 1 Processor -- start 2023-10-03T10:19:32.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.399+0000 7f35de23f700 1 -- start start 2023-10-03T10:19:32.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.399+0000 7f35de23f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35d809ac90 0x7f35d813cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:32.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.399+0000 7f35de23f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35d813d240 0x7f35d813f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:32.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.399+0000 7f35de23f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d8134980 con 0x7f35d806c0e0 2023-10-03T10:19:32.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.399+0000 7f35de23f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d806c4b0 con 0x7f35d809ac90 2023-10-03T10:19:32.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.399+0000 7f35de23f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f35d809b0e0 con 0x7f35d813d240 2023-10-03T10:19:32.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.399+0000 7f35d7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35d813d240 0x7f35d813f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:32.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.399+0000 7f35d77fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f35d806c0e0 0x7f35d8134ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33284/0 (socket says 172.21.15.3:33284) 2023-10-03T10:19:32.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.399+0000 7f35d77fe700 1 -- 172.21.15.3:0/1336619319 learned_addr learned my addr 172.21.15.3:0/1336619319 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:32.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.399+0000 7f35d7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35d813d240 0x7f35d813f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36880/0 (socket says 172.21.15.3:36880) 2023-10-03T10:19:32.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.400+0000 7f35d6ffd700 1 --2- 172.21.15.3:0/1336619319 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35d809ac90 0x7f35d813cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:32.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.400+0000 7f35d67fc700 1 -- 172.21.15.3:0/1336619319 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1463022723 0 0) 0x7f35d8134980 con 0x7f35d806c0e0 2023-10-03T10:19:32.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.400+0000 7f35d7fff700 1 -- 172.21.15.3:0/1336619319 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35d809ac90 msgr2=0x7f35d813cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:32.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.400+0000 7f35d7fff700 1 --2- 172.21.15.3:0/1336619319 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35d809ac90 0x7f35d813cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.400+0000 7f35d7fff700 1 -- 172.21.15.3:0/1336619319 >> v1:172.21.15.3:6789/0 conn(0x7f35d806c0e0 legacy=0x7f35d8134ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:32.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.400+0000 7f35d7fff700 1 -- 172.21.15.3:0/1336619319 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d80a0600 con 0x7f35d813d240 2023-10-03T10:19:32.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.400+0000 7f35d7fff700 1 --2- 172.21.15.3:0/1336619319 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35d813d240 0x7f35d813f620 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009dc0 tx=0x7f35d000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17caa4e492f4f19b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:32.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.400+0000 7f35d67fc700 1 -- 172.21.15.3:0/1336619319 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d000e070 con 0x7f35d813d240 2023-10-03T10:19:32.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.400+0000 7f35d67fc700 1 -- 172.21.15.3:0/1336619319 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d000bb20 con 0x7f35d813d240 2023-10-03T10:19:32.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.400+0000 7f35d67fc700 1 -- 172.21.15.3:0/1336619319 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d0012430 con 0x7f35d813d240 2023-10-03T10:19:32.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.401+0000 7f35de23f700 1 -- 172.21.15.3:0/1336619319 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35d813d240 msgr2=0x7f35d813f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:32.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.401+0000 7f35de23f700 1 --2- 172.21.15.3:0/1336619319 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35d813d240 0x7f35d813f620 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009dc0 tx=0x7f35d000b330 comp rx=0 tx=0).stop 2023-10-03T10:19:32.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.401+0000 7f35de23f700 1 -- 172.21.15.3:0/1336619319 shutdown_connections 2023-10-03T10:19:32.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.401+0000 7f35de23f700 1 --2- 172.21.15.3:0/1336619319 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35d813d240 0x7f35d813f620 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.401+0000 7f35de23f700 1 --2- 172.21.15.3:0/1336619319 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35d809ac90 0x7f35d813cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.401+0000 7f35de23f700 1 -- 172.21.15.3:0/1336619319 >> 172.21.15.3:0/1336619319 conn(0x7f35d80a4210 msgr2=0x7f35d80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:32.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.401+0000 7f35de23f700 1 -- 172.21.15.3:0/1336619319 shutdown_connections 2023-10-03T10:19:32.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.401+0000 7f35de23f700 1 -- 172.21.15.3:0/1336619319 wait complete. 2023-10-03T10:19:32.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.403+0000 7f35de23f700 1 Processor -- start 2023-10-03T10:19:32.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.403+0000 7f35de23f700 1 -- start start 2023-10-03T10:19:32.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.404+0000 7f35de23f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35d806c0e0 0x7f35d8138c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:32.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.404+0000 7f35de23f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35d809ac90 0x7f35d8132c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:32.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.404+0000 7f35de23f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35d813d240 0x7f35d8133180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:32.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.404+0000 7f35de23f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d809dcf0 con 0x7f35d809ac90 2023-10-03T10:19:32.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.404+0000 7f35de23f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d8134600 con 0x7f35d806c0e0 2023-10-03T10:19:32.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.404+0000 7f35de23f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f35d8134040 con 0x7f35d813d240 2023-10-03T10:19:32.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.404+0000 7f35d77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35d806c0e0 0x7f35d8138c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:32.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.404+0000 7f35d77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35d806c0e0 0x7f35d8138c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51998/0 (socket says 172.21.15.3:51998) 2023-10-03T10:19:32.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.404+0000 7f35d77fe700 1 -- 172.21.15.3:0/3710191556 learned_addr learned my addr 172.21.15.3:0/3710191556 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:32.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.404+0000 7f35d6ffd700 1 --2- 172.21.15.3:0/3710191556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35d809ac90 0x7f35d8132c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:32.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.404+0000 7f35d7fff700 1 --2- 172.21.15.3:0/3710191556 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35d813d240 0x7f35d8133180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:32.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.405+0000 7f35d77fe700 1 -- 172.21.15.3:0/3710191556 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35d813d240 msgr2=0x7f35d8133180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:32.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.405+0000 7f35d77fe700 1 --2- 172.21.15.3:0/3710191556 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35d813d240 0x7f35d8133180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.405+0000 7f35d77fe700 1 -- 172.21.15.3:0/3710191556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35d809ac90 msgr2=0x7f35d8132c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:32.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.405+0000 7f35d77fe700 1 --2- 172.21.15.3:0/3710191556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35d809ac90 0x7f35d8132c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.405+0000 7f35d77fe700 1 -- 172.21.15.3:0/3710191556 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d0009a70 con 0x7f35d806c0e0 2023-10-03T10:19:32.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.405+0000 7f35d77fe700 1 --2- 172.21.15.3:0/3710191556 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35d806c0e0 0x7f35d8138c60 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f35c800c520 tx=0x7f35c800c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:32.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.405+0000 7f35bffff700 1 -- 172.21.15.3:0/3710191556 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c8004c80 con 0x7f35d806c0e0 2023-10-03T10:19:32.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.405+0000 7f35bffff700 1 -- 172.21.15.3:0/3710191556 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c8004de0 con 0x7f35d806c0e0 2023-10-03T10:19:32.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.405+0000 7f35bffff700 1 -- 172.21.15.3:0/3710191556 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c8007310 con 0x7f35d806c0e0 2023-10-03T10:19:32.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.405+0000 7f35de23f700 1 -- 172.21.15.3:0/3710191556 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d809df80 con 0x7f35d806c0e0 2023-10-03T10:19:32.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.405+0000 7f35de23f700 1 -- 172.21.15.3:0/3710191556 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d809cae0 con 0x7f35d806c0e0 2023-10-03T10:19:32.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.406+0000 7f35bffff700 1 -- 172.21.15.3:0/3710191556 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f35c8007470 con 0x7f35d806c0e0 2023-10-03T10:19:32.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.407+0000 7f35bffff700 1 -- 172.21.15.3:0/3710191556 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f35c803f400 con 0x7f35d806c0e0 2023-10-03T10:19:32.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.407+0000 7f35de23f700 1 -- 172.21.15.3:0/3710191556 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d8074ea0 con 0x7f35d806c0e0 2023-10-03T10:19:32.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.412+0000 7f35bffff700 1 -- 172.21.15.3:0/3710191556 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f35c80123b0 con 0x7f35d806c0e0 2023-10-03T10:19:32.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.464+0000 7f12b4996700 1 Processor -- start 2023-10-03T10:19:32.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.464+0000 7f12b4996700 1 -- start start 2023-10-03T10:19:32.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.465+0000 7f12b4996700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12b01338a0 0x7f12b0133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:32.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.465+0000 7f12b4996700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12b009de50 0x7f12b009e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:32.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.465+0000 7f12b4996700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f12b009e760 0x7f12b013a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:32.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.465+0000 7f12b4996700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12b006fef0 con 0x7f12b009e760 2023-10-03T10:19:32.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.465+0000 7f12b4996700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b006ef30 con 0x7f12b01338a0 2023-10-03T10:19:32.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.465+0000 7f12b4996700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12b01334b0 con 0x7f12b009de50 2023-10-03T10:19:32.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.465+0000 7f12ae59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12b01338a0 0x7f12b0133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:32.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.466+0000 7f12ae59c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12b01338a0 0x7f12b0133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:52000/0 (socket says 172.21.15.3:52000) 2023-10-03T10:19:32.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.466+0000 7f12ae59c700 1 -- 172.21.15.3:0/2316793075 learned_addr learned my addr 172.21.15.3:0/2316793075 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:32.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.466+0000 7f12aed9d700 1 --2- 172.21.15.3:0/2316793075 >> v2:172.21.15.3:3300/0 conn(0x7f12b009e760 0x7f12b013a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:32.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.466+0000 7f12add9b700 1 --2- 172.21.15.3:0/2316793075 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12b009de50 0x7f12b009e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:32.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.466+0000 7f12add9b700 1 -- 172.21.15.3:0/2316793075 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12b01338a0 msgr2=0x7f12b0133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:32.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.466+0000 7f12add9b700 1 --2- 172.21.15.3:0/2316793075 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12b01338a0 0x7f12b0133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.466+0000 7f12add9b700 1 -- 172.21.15.3:0/2316793075 >> v2:172.21.15.3:3300/0 conn(0x7f12b009e760 msgr2=0x7f12b013a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:32.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.466+0000 7f12add9b700 1 --2- 172.21.15.3:0/2316793075 >> v2:172.21.15.3:3300/0 conn(0x7f12b009e760 0x7f12b013a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.466+0000 7f12add9b700 1 -- 172.21.15.3:0/2316793075 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b006fc30 con 0x7f12b009de50 2023-10-03T10:19:32.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.466+0000 7f12ae59c700 1 --2- 172.21.15.3:0/2316793075 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12b01338a0 0x7f12b0133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:32.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.466+0000 7f12aed9d700 1 --2- 172.21.15.3:0/2316793075 >> v2:172.21.15.3:3300/0 conn(0x7f12b009e760 0x7f12b013a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:32.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.466+0000 7f12add9b700 1 --2- 172.21.15.3:0/2316793075 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12b009de50 0x7f12b009e220 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009870 tx=0x7f12a4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ed0ae5c718d8690 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:32.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.467+0000 7f12ad59a700 1 -- 172.21.15.3:0/2316793075 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a4011830 con 0x7f12b009de50 2023-10-03T10:19:32.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.467+0000 7f12ad59a700 1 -- 172.21.15.3:0/2316793075 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a4011990 con 0x7f12b009de50 2023-10-03T10:19:32.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.467+0000 7f12ad59a700 1 -- 172.21.15.3:0/2316793075 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a4011d20 con 0x7f12b009de50 2023-10-03T10:19:32.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.467+0000 7f12b4996700 1 -- 172.21.15.3:0/2316793075 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12b009de50 msgr2=0x7f12b009e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:32.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.467+0000 7f12b4996700 1 --2- 172.21.15.3:0/2316793075 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12b009de50 0x7f12b009e220 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009870 tx=0x7f12a4009b80 comp rx=0 tx=0).stop 2023-10-03T10:19:32.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.467+0000 7f12b4996700 1 -- 172.21.15.3:0/2316793075 shutdown_connections 2023-10-03T10:19:32.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.467+0000 7f12b4996700 1 --2- 172.21.15.3:0/2316793075 >> v2:172.21.15.3:3300/0 conn(0x7f12b009e760 0x7f12b013a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.467+0000 7f12b4996700 1 --2- 172.21.15.3:0/2316793075 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12b01338a0 0x7f12b0133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.467+0000 7f12b4996700 1 --2- 172.21.15.3:0/2316793075 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12b009de50 0x7f12b009e220 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.467+0000 7f12b4996700 1 -- 172.21.15.3:0/2316793075 >> 172.21.15.3:0/2316793075 conn(0x7f12b00a2aa0 msgr2=0x7f12b00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:32.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.467+0000 7f12b4996700 1 -- 172.21.15.3:0/2316793075 shutdown_connections 2023-10-03T10:19:32.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.467+0000 7f12b4996700 1 -- 172.21.15.3:0/2316793075 wait complete. 2023-10-03T10:19:32.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.469+0000 7f12b4996700 1 Processor -- start 2023-10-03T10:19:32.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.469+0000 7f12b4996700 1 -- start start 2023-10-03T10:19:32.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.470+0000 7f12b4996700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12b009de50 0x7f12b011ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:32.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.470+0000 7f12b4996700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12b009e760 0x7f12b011f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:32.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.470+0000 7f12b4996700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12b01338a0 0x7f12b011f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:32.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.470+0000 7f12b4996700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b0121100 con 0x7f12b01338a0 2023-10-03T10:19:32.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.470+0000 7f12b4996700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b0122760 con 0x7f12b009e760 2023-10-03T10:19:32.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.470+0000 7f12b4996700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12b01200a0 con 0x7f12b009de50 2023-10-03T10:19:32.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.470+0000 7f12ae59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12b009de50 0x7f12b011ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:32.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.470+0000 7f12ae59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12b009de50 0x7f12b011ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:36908/0 (socket says 172.21.15.3:36908) 2023-10-03T10:19:32.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.470+0000 7f12ae59c700 1 -- 172.21.15.3:0/1923548781 learned_addr learned my addr 172.21.15.3:0/1923548781 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:32.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.470+0000 7f12aed9d700 1 --2- 172.21.15.3:0/1923548781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12b01338a0 0x7f12b011f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:32.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.470+0000 7f12add9b700 1 --2- 172.21.15.3:0/1923548781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12b009e760 0x7f12b011f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:32.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.471+0000 7f12ae59c700 1 -- 172.21.15.3:0/1923548781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12b009e760 msgr2=0x7f12b011f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:32.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.471+0000 7f12ae59c700 1 --2- 172.21.15.3:0/1923548781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12b009e760 0x7f12b011f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.471+0000 7f12ae59c700 1 -- 172.21.15.3:0/1923548781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12b01338a0 msgr2=0x7f12b011f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:32.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.471+0000 7f12ae59c700 1 --2- 172.21.15.3:0/1923548781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12b01338a0 0x7f12b011f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.471+0000 7f12ae59c700 1 -- 172.21.15.3:0/1923548781 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a8009a70 con 0x7f12b009de50 2023-10-03T10:19:32.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.471+0000 7f12aed9d700 1 --2- 172.21.15.3:0/1923548781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12b01338a0 0x7f12b011f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:32.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.471+0000 7f12ae59c700 1 --2- 172.21.15.3:0/1923548781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12b009de50 0x7f12b011ec40 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f12a000ec10 tx=0x7f12a000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:32.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.471+0000 7f129effd700 1 -- 172.21.15.3:0/1923548781 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a000c850 con 0x7f12b009de50 2023-10-03T10:19:32.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.471+0000 7f12b4996700 1 -- 172.21.15.3:0/1923548781 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a0009550 con 0x7f12b009de50 2023-10-03T10:19:32.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.471+0000 7f129effd700 1 -- 172.21.15.3:0/1923548781 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a000c9b0 con 0x7f12b009de50 2023-10-03T10:19:32.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.471+0000 7f129effd700 1 -- 172.21.15.3:0/1923548781 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a000cd20 con 0x7f12b009de50 2023-10-03T10:19:32.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.471+0000 7f12b4996700 1 -- 172.21.15.3:0/1923548781 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a40094a0 con 0x7f12b009de50 2023-10-03T10:19:32.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.472+0000 7f12b4996700 1 -- 172.21.15.3:0/1923548781 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b0122300 con 0x7f12b009de50 2023-10-03T10:19:32.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.473+0000 7f129effd700 1 -- 172.21.15.3:0/1923548781 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f12a0013470 con 0x7f12b009de50 2023-10-03T10:19:32.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.473+0000 7f129effd700 1 -- 172.21.15.3:0/1923548781 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f12a003fe10 con 0x7f12b009de50 2023-10-03T10:19:32.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.482+0000 7f129effd700 1 -- 172.21.15.3:0/1923548781 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f12a000ce80 con 0x7f12b009de50 2023-10-03T10:19:32.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.578+0000 7f894cff9700 1 -- 172.21.15.3:0/1504971549 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8938005590 con 0x7f896006a770 2023-10-03T10:19:32.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.578+0000 7f894effd700 1 -- 172.21.15.3:0/1504971549 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f895002d450 con 0x7f896006a770 2023-10-03T10:19:32.579 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:32.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.581+0000 7f894cff9700 1 -- 172.21.15.3:0/1504971549 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f896006a770 msgr2=0x7f89601260a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:32.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.581+0000 7f894cff9700 1 --2- 172.21.15.3:0/1504971549 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f896006a770 0x7f89601260a0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f8950016000 tx=0x7f8950004750 comp rx=0 tx=0).stop 2023-10-03T10:19:32.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.581+0000 7f894cff9700 1 -- 172.21.15.3:0/1504971549 shutdown_connections 2023-10-03T10:19:32.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.581+0000 7f894cff9700 1 --2- 172.21.15.3:0/1504971549 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f896013d1f0 0x7f8960126b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.581+0000 7f894cff9700 1 --2- 172.21.15.3:0/1504971549 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f896006a770 0x7f89601260a0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.581+0000 7f894cff9700 1 --2- 172.21.15.3:0/1504971549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f896009dc30 0x7f89601265e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.581+0000 7f894cff9700 1 -- 172.21.15.3:0/1504971549 >> 172.21.15.3:0/1504971549 conn(0x7f8960066f50 msgr2=0x7f89600a1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:32.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.582+0000 7f894cff9700 1 -- 172.21.15.3:0/1504971549 shutdown_connections 2023-10-03T10:19:32.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.582+0000 7f894cff9700 1 -- 172.21.15.3:0/1504971549 wait complete. 2023-10-03T10:19:32.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.583+0000 7f55725cc700 1 -- 172.21.15.3:0/482867807 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f556c123540 con 0x7f556c0a0a30 2023-10-03T10:19:32.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.584+0000 7f5568ff9700 1 -- 172.21.15.3:0/482867807 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f555c017070 con 0x7f556c0a0a30 2023-10-03T10:19:32.584 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:32.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.586+0000 7f55725cc700 1 -- 172.21.15.3:0/482867807 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f556c0a0a30 msgr2=0x7f556c120060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:32.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.586+0000 7f55725cc700 1 --2- 172.21.15.3:0/482867807 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f556c0a0a30 0x7f556c120060 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f555c00ebe0 tx=0x7f555c00eef0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.586+0000 7f55725cc700 1 -- 172.21.15.3:0/482867807 shutdown_connections 2023-10-03T10:19:32.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.586+0000 7f55725cc700 1 --2- 172.21.15.3:0/482867807 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f556c133fa0 0x7f556c120ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.586+0000 7f55725cc700 1 --2- 172.21.15.3:0/482867807 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f556c0a0a30 0x7f556c120060 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.586+0000 7f55725cc700 1 --2- 172.21.15.3:0/482867807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f556c0a1340 0x7f556c1205a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.586+0000 7f55725cc700 1 -- 172.21.15.3:0/482867807 >> 172.21.15.3:0/482867807 conn(0x7f556c0a6550 msgr2=0x7f556c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:32.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.587+0000 7f55725cc700 1 -- 172.21.15.3:0/482867807 shutdown_connections 2023-10-03T10:19:32.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.587+0000 7f55725cc700 1 -- 172.21.15.3:0/482867807 wait complete. 2023-10-03T10:19:32.598 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:32.603 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:32.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.624+0000 7f35de23f700 1 -- 172.21.15.3:0/3710191556 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f35d806b250 con 0x7f35d806c0e0 2023-10-03T10:19:32.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.625+0000 7f35bffff700 1 -- 172.21.15.3:0/3710191556 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f35c8011360 con 0x7f35d806c0e0 2023-10-03T10:19:32.625 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.627+0000 7f35de23f700 1 -- 172.21.15.3:0/3710191556 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35d806c0e0 msgr2=0x7f35d8138c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.627+0000 7f35de23f700 1 --2- 172.21.15.3:0/3710191556 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35d806c0e0 0x7f35d8138c60 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f35c800c520 tx=0x7f35c800c830 comp rx=0 tx=0).stop 2023-10-03T10:19:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.627+0000 7f35de23f700 1 -- 172.21.15.3:0/3710191556 shutdown_connections 2023-10-03T10:19:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.627+0000 7f35de23f700 1 --2- 172.21.15.3:0/3710191556 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f35d813d240 0x7f35d8133180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.627+0000 7f35de23f700 1 --2- 172.21.15.3:0/3710191556 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f35d806c0e0 0x7f35d8138c60 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.627+0000 7f35de23f700 1 --2- 172.21.15.3:0/3710191556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35d809ac90 0x7f35d8132c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.627+0000 7f35de23f700 1 -- 172.21.15.3:0/3710191556 >> 172.21.15.3:0/3710191556 conn(0x7f35d80a4210 msgr2=0x7f35d80a0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.627+0000 7f35de23f700 1 -- 172.21.15.3:0/3710191556 shutdown_connections 2023-10-03T10:19:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.627+0000 7f35de23f700 1 -- 172.21.15.3:0/3710191556 wait complete. 2023-10-03T10:19:32.641 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:32.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.699+0000 7f12b4996700 1 -- 172.21.15.3:0/1923548781 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f12b0122300 con 0x7f12b009de50 2023-10-03T10:19:32.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.700+0000 7f129effd700 1 -- 172.21.15.3:0/1923548781 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f12a0017020 con 0x7f12b009de50 2023-10-03T10:19:32.700 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:32.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.702+0000 7f12b4996700 1 -- 172.21.15.3:0/1923548781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12b009de50 msgr2=0x7f12b011ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:32.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.702+0000 7f12b4996700 1 --2- 172.21.15.3:0/1923548781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12b009de50 0x7f12b011ec40 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f12a000ec10 tx=0x7f12a000ef20 comp rx=0 tx=0).stop 2023-10-03T10:19:32.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.702+0000 7f12b4996700 1 -- 172.21.15.3:0/1923548781 shutdown_connections 2023-10-03T10:19:32.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.702+0000 7f12b4996700 1 --2- 172.21.15.3:0/1923548781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12b01338a0 0x7f12b011f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.702+0000 7f12b4996700 1 --2- 172.21.15.3:0/1923548781 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f12b009de50 0x7f12b011ec40 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.702+0000 7f12b4996700 1 --2- 172.21.15.3:0/1923548781 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f12b009e760 0x7f12b011f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:32.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.702+0000 7f12b4996700 1 -- 172.21.15.3:0/1923548781 >> 172.21.15.3:0/1923548781 conn(0x7f12b00a2aa0 msgr2=0x7f12b009ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:32.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.702+0000 7f12b4996700 1 -- 172.21.15.3:0/1923548781 shutdown_connections 2023-10-03T10:19:32.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:32.702+0000 7f12b4996700 1 -- 172.21.15.3:0/1923548781 wait complete. 2023-10-03T10:19:32.715 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:33.600 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:33.603 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:33.642 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:33.717 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:33.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.754+0000 7fc98bea8700 1 Processor -- start 2023-10-03T10:19:33.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.755+0000 7fc98bea8700 1 -- start start 2023-10-03T10:19:33.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.756+0000 7fc98bea8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc98409d6f0 0x7fc98409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:33.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.756+0000 7fc98bea8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc98409e000 0x7fc984130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:33.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.756+0000 7fc98bea8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc98409f130 con 0x7fc98409e410 2023-10-03T10:19:33.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.756+0000 7fc98bea8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc98409f520 con 0x7fc98409e000 2023-10-03T10:19:33.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.756+0000 7fc98bea8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc98409e930 con 0x7fc98409d6f0 2023-10-03T10:19:33.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.756+0000 7fc989443700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc98409e000 0x7fc984130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:33.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.756+0000 7fc989c44700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc98409d6f0 0x7fc98409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:33.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.756+0000 7fc989443700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc98409e000 0x7fc984130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40902/0 (socket says 172.21.15.3:40902) 2023-10-03T10:19:33.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.756+0000 7fc989443700 1 -- 172.21.15.3:0/3467700486 learned_addr learned my addr 172.21.15.3:0/3467700486 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:33.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.756+0000 7fc989c44700 1 -- 172.21.15.3:0/3467700486 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc98409e000 msgr2=0x7fc984130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:33.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.756+0000 7fc989c44700 1 --2- 172.21.15.3:0/3467700486 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc98409e000 0x7fc984130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:33.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.756+0000 7fc989c44700 1 -- 172.21.15.3:0/3467700486 >> v1:172.21.15.3:6789/0 conn(0x7fc98409e410 legacy=0x7fc984130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:33.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.756+0000 7fc989c44700 1 -- 172.21.15.3:0/3467700486 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc98409b2d0 con 0x7fc98409d6f0 2023-10-03T10:19:33.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.757+0000 7fc989c44700 1 --2- 172.21.15.3:0/3467700486 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc98409d6f0 0x7fc98409dac0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc9800098a0 tx=0x7fc980009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f95019180e09caa server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:33.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.757+0000 7fc988c42700 1 -- 172.21.15.3:0/3467700486 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc980011830 con 0x7fc98409d6f0 2023-10-03T10:19:33.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.757+0000 7fc988c42700 1 -- 172.21.15.3:0/3467700486 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc980011990 con 0x7fc98409d6f0 2023-10-03T10:19:33.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.757+0000 7fc988c42700 1 -- 172.21.15.3:0/3467700486 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc980011d20 con 0x7fc98409d6f0 2023-10-03T10:19:33.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.758+0000 7fc98bea8700 1 -- 172.21.15.3:0/3467700486 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc98409d6f0 msgr2=0x7fc98409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:33.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.758+0000 7fc98bea8700 1 --2- 172.21.15.3:0/3467700486 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc98409d6f0 0x7fc98409dac0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc9800098a0 tx=0x7fc980009bb0 comp rx=0 tx=0).stop 2023-10-03T10:19:33.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.758+0000 7fc98bea8700 1 -- 172.21.15.3:0/3467700486 shutdown_connections 2023-10-03T10:19:33.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.758+0000 7fc98bea8700 1 --2- 172.21.15.3:0/3467700486 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc98409d6f0 0x7fc98409dac0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:33.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.758+0000 7fc98bea8700 1 --2- 172.21.15.3:0/3467700486 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc98409e000 0x7fc984130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:33.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.758+0000 7fc98bea8700 1 -- 172.21.15.3:0/3467700486 >> 172.21.15.3:0/3467700486 conn(0x7fc984066f50 msgr2=0x7fc98409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:33.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.758+0000 7fc98bea8700 1 -- 172.21.15.3:0/3467700486 shutdown_connections 2023-10-03T10:19:33.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.758+0000 7fc98bea8700 1 -- 172.21.15.3:0/3467700486 wait complete. 2023-10-03T10:19:33.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.759+0000 7fc98bea8700 1 Processor -- start 2023-10-03T10:19:33.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.759+0000 7fc98bea8700 1 -- start start 2023-10-03T10:19:33.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.760+0000 7fc98bea8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc98409d6f0 0x7fc98411f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:33.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.760+0000 7fc98bea8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc98409e000 0x7fc98411f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:33.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.760+0000 7fc98bea8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc98409e410 0x7fc98411fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:33.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.760+0000 7fc98bea8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98409c700 con 0x7fc98409d6f0 2023-10-03T10:19:33.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.760+0000 7fc98bea8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9841214e0 con 0x7fc98409e410 2023-10-03T10:19:33.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.760+0000 7fc98bea8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc98406e840 con 0x7fc98409e000 2023-10-03T10:19:33.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.760+0000 7fc989443700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc98409e000 0x7fc98411f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:33.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.760+0000 7fc989443700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc98409e000 0x7fc98411f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48100/0 (socket says 172.21.15.3:48100) 2023-10-03T10:19:33.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.760+0000 7fc989443700 1 -- 172.21.15.3:0/3944752858 learned_addr learned my addr 172.21.15.3:0/3944752858 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:33.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.760+0000 7fc98a445700 1 --2- 172.21.15.3:0/3944752858 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc98409e410 0x7fc98411fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:33.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.760+0000 7fc989c44700 1 --2- 172.21.15.3:0/3944752858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc98409d6f0 0x7fc98411f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:33.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.761+0000 7fc989443700 1 -- 172.21.15.3:0/3944752858 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc98409e410 msgr2=0x7fc98411fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:33.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.761+0000 7fc989443700 1 --2- 172.21.15.3:0/3944752858 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc98409e410 0x7fc98411fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:33.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.761+0000 7fc989443700 1 -- 172.21.15.3:0/3944752858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc98409d6f0 msgr2=0x7fc98411f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:33.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.761+0000 7fc989443700 1 --2- 172.21.15.3:0/3944752858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc98409d6f0 0x7fc98411f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:33.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.761+0000 7fc989443700 1 -- 172.21.15.3:0/3944752858 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9800094a0 con 0x7fc98409e000 2023-10-03T10:19:33.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.761+0000 7fc989443700 1 --2- 172.21.15.3:0/3944752858 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc98409e000 0x7fc98411f5a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc974010af0 tx=0x7fc974010e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:33.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.761+0000 7fc97a7fc700 1 -- 172.21.15.3:0/3944752858 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97400ec60 con 0x7fc98409e000 2023-10-03T10:19:33.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.761+0000 7fc97a7fc700 1 -- 172.21.15.3:0/3944752858 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97400edc0 con 0x7fc98409e000 2023-10-03T10:19:33.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.761+0000 7fc97a7fc700 1 -- 172.21.15.3:0/3944752858 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc974011670 con 0x7fc98409e000 2023-10-03T10:19:33.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.761+0000 7fc98bea8700 1 -- 172.21.15.3:0/3944752858 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc984121a90 con 0x7fc98409e000 2023-10-03T10:19:33.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.761+0000 7fc98bea8700 1 -- 172.21.15.3:0/3944752858 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9841228a0 con 0x7fc98409e000 2023-10-03T10:19:33.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.762+0000 7fc97a7fc700 1 -- 172.21.15.3:0/3944752858 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc9740117d0 con 0x7fc98409e000 2023-10-03T10:19:33.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.763+0000 7fc97a7fc700 1 -- 172.21.15.3:0/3944752858 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc974041920 con 0x7fc98409e000 2023-10-03T10:19:33.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.763+0000 7fc98bea8700 1 -- 172.21.15.3:0/3944752858 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc984074ea0 con 0x7fc98409e000 2023-10-03T10:19:33.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.772+0000 7fc97a7fc700 1 -- 172.21.15.3:0/3944752858 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc97401e080 con 0x7fc98409e000 2023-10-03T10:19:33.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.794+0000 7f48a5445700 1 Processor -- start 2023-10-03T10:19:33.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.794+0000 7f48a5445700 1 -- start start 2023-10-03T10:19:33.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.795+0000 7f48a5445700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48a01338a0 0x7f48a0133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:33.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.795+0000 7f48a5445700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f48a009de50 0x7f48a009e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:33.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.795+0000 7f48a5445700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a006ef30 con 0x7f48a009e760 2023-10-03T10:19:33.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.795+0000 7f48a5445700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a006fef0 con 0x7f48a009de50 2023-10-03T10:19:33.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.795+0000 7f48a5445700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a01334b0 con 0x7f48a01338a0 2023-10-03T10:19:33.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.795+0000 7f489effd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48a01338a0 0x7f48a0133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:33.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.795+0000 7f489e7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f48a009de50 0x7f48a009e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:33.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.795+0000 7f489effd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48a01338a0 0x7f48a0133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40926/0 (socket says 172.21.15.3:40926) 2023-10-03T10:19:33.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.795+0000 7f489effd700 1 -- 172.21.15.3:0/2546363001 learned_addr learned my addr 172.21.15.3:0/2546363001 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:33.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.796+0000 7f489effd700 1 -- 172.21.15.3:0/2546363001 >> v2:172.21.15.3:3300/0 conn(0x7f48a009de50 msgr2=0x7f48a009e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:33.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.796+0000 7f489effd700 1 --2- 172.21.15.3:0/2546363001 >> v2:172.21.15.3:3300/0 conn(0x7f48a009de50 0x7f48a009e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.796+0000 7f489effd700 1 -- 172.21.15.3:0/2546363001 >> v1:172.21.15.3:6789/0 conn(0x7f48a009e760 legacy=0x7f48a013a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.796+0000 7f489effd700 1 -- 172.21.15.3:0/2546363001 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a006fc30 con 0x7f48a01338a0 2023-10-03T10:19:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.796+0000 7f489effd700 1 --2- 172.21.15.3:0/2546363001 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48a01338a0 0x7f48a0133c70 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f48900098f0 tx=0x7f4890009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acf9ca72d2d01cdd server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.796+0000 7f489e7fc700 1 --2- 172.21.15.3:0/2546363001 >> v2:172.21.15.3:3300/0 conn(0x7f48a009de50 0x7f48a009e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.796+0000 7f489dffb700 1 -- 172.21.15.3:0/2546363001 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489000b8a0 con 0x7f48a01338a0 2023-10-03T10:19:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.796+0000 7f489dffb700 1 -- 172.21.15.3:0/2546363001 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489000ba00 con 0x7f48a01338a0 2023-10-03T10:19:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.796+0000 7f489dffb700 1 -- 172.21.15.3:0/2546363001 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489000bd90 con 0x7f48a01338a0 2023-10-03T10:19:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.797+0000 7f48a5445700 1 -- 172.21.15.3:0/2546363001 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48a01338a0 msgr2=0x7f48a0133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.797+0000 7f48a5445700 1 --2- 172.21.15.3:0/2546363001 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48a01338a0 0x7f48a0133c70 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f48900098f0 tx=0x7f4890009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.797+0000 7f48a5445700 1 -- 172.21.15.3:0/2546363001 shutdown_connections 2023-10-03T10:19:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.797+0000 7f48a5445700 1 --2- 172.21.15.3:0/2546363001 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48a01338a0 0x7f48a0133c70 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.797+0000 7f48a5445700 1 --2- 172.21.15.3:0/2546363001 >> v2:172.21.15.3:3300/0 conn(0x7f48a009de50 0x7f48a009e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.797+0000 7f48a5445700 1 -- 172.21.15.3:0/2546363001 >> 172.21.15.3:0/2546363001 conn(0x7f48a00a2aa0 msgr2=0x7f48a00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.797+0000 7f48a5445700 1 -- 172.21.15.3:0/2546363001 shutdown_connections 2023-10-03T10:19:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.797+0000 7f48a5445700 1 -- 172.21.15.3:0/2546363001 wait complete. 2023-10-03T10:19:33.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.798+0000 7f48a5445700 1 Processor -- start 2023-10-03T10:19:33.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.799+0000 7f48a5445700 1 -- start start 2023-10-03T10:19:33.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.799+0000 7f48a5445700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48a009de50 0x7f48a01214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:33.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.799+0000 7f48a5445700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48a009e760 0x7f48a0121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:33.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.799+0000 7f48a5445700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a01338a0 0x7f48a0121f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:33.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.799+0000 7f48a5445700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a0124ae0 con 0x7f48a01338a0 2023-10-03T10:19:33.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.799+0000 7f48a5445700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a00a0fd0 con 0x7f48a009e760 2023-10-03T10:19:33.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.799+0000 7f48a5445700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f48a006f640 con 0x7f48a009de50 2023-10-03T10:19:33.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.799+0000 7f489effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48a009de50 0x7f48a01214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:33.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.799+0000 7f489effd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48a009de50 0x7f48a01214e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48102/0 (socket says 172.21.15.3:48102) 2023-10-03T10:19:33.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.799+0000 7f489effd700 1 -- 172.21.15.3:0/611831139 learned_addr learned my addr 172.21.15.3:0/611831139 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:33.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.799+0000 7f489e7fc700 1 --2- 172.21.15.3:0/611831139 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48a009e760 0x7f48a0121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:33.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.800+0000 7f489effd700 1 -- 172.21.15.3:0/611831139 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48a009e760 msgr2=0x7f48a0121a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:33.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.800+0000 7f489effd700 1 --2- 172.21.15.3:0/611831139 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48a009e760 0x7f48a0121a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:33.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.800+0000 7f489effd700 1 -- 172.21.15.3:0/611831139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a01338a0 msgr2=0x7f48a0121f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:33.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.800+0000 7f489effd700 1 --2- 172.21.15.3:0/611831139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a01338a0 0x7f48a0121f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:33.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.800+0000 7f489effd700 1 -- 172.21.15.3:0/611831139 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4890009550 con 0x7f48a009de50 2023-10-03T10:19:33.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.800+0000 7f489effd700 1 --2- 172.21.15.3:0/611831139 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48a009de50 0x7f48a01214e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f4890000c00 tx=0x7f4890004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:33.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.800+0000 7f48877fe700 1 -- 172.21.15.3:0/611831139 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4890004380 con 0x7f48a009de50 2023-10-03T10:19:33.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.801+0000 7f48877fe700 1 -- 172.21.15.3:0/611831139 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489001f070 con 0x7f48a009de50 2023-10-03T10:19:33.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.801+0000 7f48877fe700 1 -- 172.21.15.3:0/611831139 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48900036a0 con 0x7f48a009de50 2023-10-03T10:19:33.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.801+0000 7f48a5445700 1 -- 172.21.15.3:0/611831139 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a01254f0 con 0x7f48a009de50 2023-10-03T10:19:33.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.801+0000 7f48a5445700 1 -- 172.21.15.3:0/611831139 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a0124780 con 0x7f48a009de50 2023-10-03T10:19:33.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.802+0000 7f48a5445700 1 -- 172.21.15.3:0/611831139 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a0124910 con 0x7f48a009de50 2023-10-03T10:19:33.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.802+0000 7f48877fe700 1 -- 172.21.15.3:0/611831139 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4890021030 con 0x7f48a009de50 2023-10-03T10:19:33.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.803+0000 7f48877fe700 1 -- 172.21.15.3:0/611831139 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4890040f80 con 0x7f48a009de50 2023-10-03T10:19:33.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.807+0000 7f48877fe700 1 -- 172.21.15.3:0/611831139 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f48900239c0 con 0x7f48a009de50 2023-10-03T10:19:33.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.811+0000 7f5bbe16b700 1 Processor -- start 2023-10-03T10:19:33.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.812+0000 7f5bbe16b700 1 -- start start 2023-10-03T10:19:33.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.812+0000 7f5bbe16b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5bb809d6f0 0x7f5bb809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:33.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.812+0000 7f5bbe16b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5bb809e000 0x7f5bb8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:33.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.812+0000 7f5bbe16b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bb809f130 con 0x7f5bb809e410 2023-10-03T10:19:33.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.812+0000 7f5bbe16b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bb809f520 con 0x7f5bb809d6f0 2023-10-03T10:19:33.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.812+0000 7f5bbe16b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb809e930 con 0x7f5bb809e000 2023-10-03T10:19:33.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.812+0000 7f5bb77fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5bb809d6f0 0x7f5bb809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:33.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.812+0000 7f5bb6ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5bb809e000 0x7f5bb8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:33.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.812+0000 7f5bb77fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5bb809d6f0 0x7f5bb809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37602/0 (socket says 172.21.15.3:37602) 2023-10-03T10:19:33.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.812+0000 7f5bb77fe700 1 -- 172.21.15.3:0/1171903305 learned_addr learned my addr 172.21.15.3:0/1171903305 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:33.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.813+0000 7f5bb6ffd700 1 -- 172.21.15.3:0/1171903305 >> v2:172.21.15.3:3300/0 conn(0x7f5bb809d6f0 msgr2=0x7f5bb809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:33.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.813+0000 7f5bb6ffd700 1 --2- 172.21.15.3:0/1171903305 >> v2:172.21.15.3:3300/0 conn(0x7f5bb809d6f0 0x7f5bb809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:33.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.813+0000 7f5bb6ffd700 1 -- 172.21.15.3:0/1171903305 >> v1:172.21.15.3:6789/0 conn(0x7f5bb809e410 legacy=0x7f5bb8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:33.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.813+0000 7f5bb6ffd700 1 -- 172.21.15.3:0/1171903305 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb809b2d0 con 0x7f5bb809e000 2023-10-03T10:19:33.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.813+0000 7f5bb6ffd700 1 --2- 172.21.15.3:0/1171903305 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5bb809e000 0x7f5bb8130660 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009840 tx=0x7f5bac009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fad154b29edde828 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:33.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.813+0000 7f5bb67fc700 1 -- 172.21.15.3:0/1171903305 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac011830 con 0x7f5bb809e000 2023-10-03T10:19:33.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.813+0000 7f5bb67fc700 1 -- 172.21.15.3:0/1171903305 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bac011990 con 0x7f5bb809e000 2023-10-03T10:19:33.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.813+0000 7f5bb67fc700 1 -- 172.21.15.3:0/1171903305 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac011d20 con 0x7f5bb809e000 2023-10-03T10:19:33.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.814+0000 7f5bbe16b700 1 -- 172.21.15.3:0/1171903305 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5bb809e000 msgr2=0x7f5bb8130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:33.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.814+0000 7f5bbe16b700 1 --2- 172.21.15.3:0/1171903305 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5bb809e000 0x7f5bb8130660 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009840 tx=0x7f5bac009b50 comp rx=0 tx=0).stop 2023-10-03T10:19:33.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.814+0000 7f5bbe16b700 1 -- 172.21.15.3:0/1171903305 shutdown_connections 2023-10-03T10:19:33.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.814+0000 7f5bbe16b700 1 --2- 172.21.15.3:0/1171903305 >> v2:172.21.15.3:3300/0 conn(0x7f5bb809d6f0 0x7f5bb809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:33.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.814+0000 7f5bbe16b700 1 --2- 172.21.15.3:0/1171903305 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5bb809e000 0x7f5bb8130660 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:33.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.814+0000 7f5bbe16b700 1 -- 172.21.15.3:0/1171903305 >> 172.21.15.3:0/1171903305 conn(0x7f5bb8066f50 msgr2=0x7f5bb809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:33.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.814+0000 7f5bbe16b700 1 -- 172.21.15.3:0/1171903305 shutdown_connections 2023-10-03T10:19:33.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.814+0000 7f5bbe16b700 1 -- 172.21.15.3:0/1171903305 wait complete. 2023-10-03T10:19:33.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.815+0000 7f5bbe16b700 1 Processor -- start 2023-10-03T10:19:33.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.815+0000 7f5bbe16b700 1 -- start start 2023-10-03T10:19:33.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.816+0000 7f5bbe16b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5bb809d6f0 0x7f5bb81222f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:33.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.816+0000 7f5bbe16b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5bb809e000 0x7f5bb8122830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:33.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.816+0000 7f5bbe16b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bb809e410 0x7f5bb8122d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:33.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.816+0000 7f5bbe16b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb8125510 con 0x7f5bb809e410 2023-10-03T10:19:33.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.816+0000 7f5bbe16b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb8123f00 con 0x7f5bb809d6f0 2023-10-03T10:19:33.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.816+0000 7f5bbe16b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb81247d0 con 0x7f5bb809e000 2023-10-03T10:19:33.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.816+0000 7f5bb6ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5bb809e000 0x7f5bb8122830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:33.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.816+0000 7f5bb77fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5bb809d6f0 0x7f5bb81222f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:33.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.816+0000 7f5bb6ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5bb809e000 0x7f5bb8122830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48112/0 (socket says 172.21.15.3:48112) 2023-10-03T10:19:33.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.816+0000 7f5bb6ffd700 1 -- 172.21.15.3:0/1669614384 learned_addr learned my addr 172.21.15.3:0/1669614384 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:33.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.816+0000 7f5bb7fff700 1 --2- 172.21.15.3:0/1669614384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bb809e410 0x7f5bb8122d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:33.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.817+0000 7f5bb77fe700 1 -- 172.21.15.3:0/1669614384 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5bb809e000 msgr2=0x7f5bb8122830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:33.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.817+0000 7f5bb77fe700 1 --2- 172.21.15.3:0/1669614384 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5bb809e000 0x7f5bb8122830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:33.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.817+0000 7f5bb77fe700 1 -- 172.21.15.3:0/1669614384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bb809e410 msgr2=0x7f5bb8122d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:33.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.817+0000 7f5bb77fe700 1 --2- 172.21.15.3:0/1669614384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bb809e410 0x7f5bb8122d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:33.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.817+0000 7f5bb77fe700 1 -- 172.21.15.3:0/1669614384 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bac0094a0 con 0x7f5bb809d6f0 2023-10-03T10:19:33.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.817+0000 7f5bb77fe700 1 --2- 172.21.15.3:0/1669614384 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5bb809d6f0 0x7f5bb81222f0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f5ba800e8e0 tx=0x7f5ba800ebf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:33.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.817+0000 7f5b9ffff700 1 -- 172.21.15.3:0/1669614384 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8024690 con 0x7f5bb809d6f0 2023-10-03T10:19:33.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.817+0000 7f5b9ffff700 1 -- 172.21.15.3:0/1669614384 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba80247f0 con 0x7f5bb809d6f0 2023-10-03T10:19:33.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.817+0000 7f5b9ffff700 1 -- 172.21.15.3:0/1669614384 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8024b80 con 0x7f5bb809d6f0 2023-10-03T10:19:33.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.817+0000 7f5bbe16b700 1 -- 172.21.15.3:0/1669614384 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bb8125e10 con 0x7f5bb809d6f0 2023-10-03T10:19:33.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.817+0000 7f5bbe16b700 1 -- 172.21.15.3:0/1669614384 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb80a1d60 con 0x7f5bb809d6f0 2023-10-03T10:19:33.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.819+0000 7f5b9ffff700 1 -- 172.21.15.3:0/1669614384 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5ba8024ce0 con 0x7f5bb809d6f0 2023-10-03T10:19:33.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.819+0000 7f5bbe16b700 1 -- 172.21.15.3:0/1669614384 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bb81302a0 con 0x7f5bb809d6f0 2023-10-03T10:19:33.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.819+0000 7f5b9ffff700 1 -- 172.21.15.3:0/1669614384 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5ba802c9b0 con 0x7f5bb809d6f0 2023-10-03T10:19:33.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.824+0000 7f5b9ffff700 1 -- 172.21.15.3:0/1669614384 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5ba8028070 con 0x7f5bb809d6f0 2023-10-03T10:19:33.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.872+0000 7f21aed9a700 1 Processor -- start 2023-10-03T10:19:33.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.872+0000 7f21aed9a700 1 -- start start 2023-10-03T10:19:33.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.873+0000 7f21aed9a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21a806e570 0x7f21a806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:33.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.873+0000 7f21aed9a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21a806ee80 0x7f21a8134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:33.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.873+0000 7f21aed9a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21a8070ee0 con 0x7f21a809eca0 2023-10-03T10:19:33.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.873+0000 7f21aed9a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a8071480 con 0x7f21a806e570 2023-10-03T10:19:33.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.873+0000 7f21aed9a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f21a8071b90 con 0x7f21a806ee80 2023-10-03T10:19:33.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.873+0000 7f21acb36700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21a806e570 0x7f21a806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:33.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.873+0000 7f21acb36700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21a806e570 0x7f21a806e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40950/0 (socket says 172.21.15.3:40950) 2023-10-03T10:19:33.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.873+0000 7f21ad337700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21a806ee80 0x7f21a8134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:33.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.873+0000 7f21acb36700 1 -- 172.21.15.3:0/355217826 learned_addr learned my addr 172.21.15.3:0/355217826 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:33.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.874+0000 7f21a77fe700 1 -- 172.21.15.3:0/355217826 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3456471422 0 0) 0x7f21a8070ee0 con 0x7f21a809eca0 2023-10-03T10:19:33.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.874+0000 7f21ad337700 1 -- 172.21.15.3:0/355217826 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21a806e570 msgr2=0x7f21a806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:33.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.874+0000 7f21ad337700 1 --2- 172.21.15.3:0/355217826 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21a806e570 0x7f21a806e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:33.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.874+0000 7f21ad337700 1 -- 172.21.15.3:0/355217826 >> v1:172.21.15.3:6789/0 conn(0x7f21a809eca0 legacy=0x7f21a8134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:33.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.874+0000 7f21ad337700 1 -- 172.21.15.3:0/355217826 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a80711a0 con 0x7f21a806ee80 2023-10-03T10:19:33.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.874+0000 7f21acb36700 1 --2- 172.21.15.3:0/355217826 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21a806e570 0x7f21a806e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:33.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.874+0000 7f21ad337700 1 --2- 172.21.15.3:0/355217826 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21a806ee80 0x7f21a8134d20 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f21a0009dc0 tx=0x7f21a000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0e67b1ab3a17b84 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:33.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.874+0000 7f21a77fe700 1 -- 172.21.15.3:0/355217826 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a000e070 con 0x7f21a806ee80 2023-10-03T10:19:33.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.874+0000 7f21a77fe700 1 -- 172.21.15.3:0/355217826 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a000bb20 con 0x7f21a806ee80 2023-10-03T10:19:33.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.874+0000 7f21a77fe700 1 -- 172.21.15.3:0/355217826 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a0012430 con 0x7f21a806ee80 2023-10-03T10:19:33.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.875+0000 7f21aed9a700 1 -- 172.21.15.3:0/355217826 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21a806ee80 msgr2=0x7f21a8134d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:33.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.875+0000 7f21aed9a700 1 --2- 172.21.15.3:0/355217826 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21a806ee80 0x7f21a8134d20 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f21a0009dc0 tx=0x7f21a000b330 comp rx=0 tx=0).stop 2023-10-03T10:19:33.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.875+0000 7f21aed9a700 1 -- 172.21.15.3:0/355217826 shutdown_connections 2023-10-03T10:19:33.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.875+0000 7f21aed9a700 1 --2- 172.21.15.3:0/355217826 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21a806e570 0x7f21a806e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:33.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.875+0000 7f21aed9a700 1 --2- 172.21.15.3:0/355217826 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21a806ee80 0x7f21a8134d20 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:33.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.875+0000 7f21aed9a700 1 -- 172.21.15.3:0/355217826 >> 172.21.15.3:0/355217826 conn(0x7f21a80a2aa0 msgr2=0x7f21a80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:33.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.875+0000 7f21aed9a700 1 -- 172.21.15.3:0/355217826 shutdown_connections 2023-10-03T10:19:33.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.875+0000 7f21aed9a700 1 -- 172.21.15.3:0/355217826 wait complete. 2023-10-03T10:19:33.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.876+0000 7f21aed9a700 1 Processor -- start 2023-10-03T10:19:33.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.876+0000 7f21aed9a700 1 -- start start 2023-10-03T10:19:33.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.877+0000 7f21aed9a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21a806e570 0x7f21a813dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:33.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.877+0000 7f21aed9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21a806ee80 0x7f21a8124e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:33.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.877+0000 7f21aed9a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21a813b850 0x7f21a81253d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:33.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.877+0000 7f21aed9a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a80706a0 con 0x7f21a806ee80 2023-10-03T10:19:33.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.877+0000 7f21aed9a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a8126660 con 0x7f21a806e570 2023-10-03T10:19:33.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.877+0000 7f21aed9a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f21a813d180 con 0x7f21a813b850 2023-10-03T10:19:33.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.877+0000 7f21acb36700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21a806e570 0x7f21a813dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:33.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.877+0000 7f21acb36700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21a806e570 0x7f21a813dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40962/0 (socket says 172.21.15.3:40962) 2023-10-03T10:19:33.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.877+0000 7f21acb36700 1 -- 172.21.15.3:0/661631194 learned_addr learned my addr 172.21.15.3:0/661631194 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:33.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.877+0000 7f21a7fff700 1 --2- 172.21.15.3:0/661631194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21a806ee80 0x7f21a8124e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:33.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.877+0000 7f21ad337700 1 --2- 172.21.15.3:0/661631194 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21a813b850 0x7f21a81253d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:33.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.877+0000 7f21acb36700 1 -- 172.21.15.3:0/661631194 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21a813b850 msgr2=0x7f21a81253d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:33.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.877+0000 7f21acb36700 1 --2- 172.21.15.3:0/661631194 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21a813b850 0x7f21a81253d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:33.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.878+0000 7f21acb36700 1 -- 172.21.15.3:0/661631194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21a806ee80 msgr2=0x7f21a8124e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:33.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.878+0000 7f21acb36700 1 --2- 172.21.15.3:0/661631194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21a806ee80 0x7f21a8124e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:33.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.878+0000 7f21acb36700 1 -- 172.21.15.3:0/661631194 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219c009550 con 0x7f21a806e570 2023-10-03T10:19:33.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.878+0000 7f21acb36700 1 --2- 172.21.15.3:0/661631194 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21a806e570 0x7f21a813dc20 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f219c00ebe0 tx=0x7f219c00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:33.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.878+0000 7f21a57fa700 1 -- 172.21.15.3:0/661631194 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c00c850 con 0x7f21a806e570 2023-10-03T10:19:33.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.878+0000 7f21a57fa700 1 -- 172.21.15.3:0/661631194 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219c00c9b0 con 0x7f21a806e570 2023-10-03T10:19:33.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.878+0000 7f21aed9a700 1 -- 172.21.15.3:0/661631194 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a0009a70 con 0x7f21a806e570 2023-10-03T10:19:33.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.878+0000 7f21a57fa700 1 -- 172.21.15.3:0/661631194 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c00cd40 con 0x7f21a806e570 2023-10-03T10:19:33.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.878+0000 7f21aed9a700 1 -- 172.21.15.3:0/661631194 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a8126350 con 0x7f21a806e570 2023-10-03T10:19:33.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.879+0000 7f21aed9a700 1 -- 172.21.15.3:0/661631194 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21a8074ea0 con 0x7f21a806e570 2023-10-03T10:19:33.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.880+0000 7f21a57fa700 1 -- 172.21.15.3:0/661631194 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f219c013470 con 0x7f21a806e570 2023-10-03T10:19:33.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.880+0000 7f21a57fa700 1 -- 172.21.15.3:0/661631194 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f219c03f870 con 0x7f21a806e570 2023-10-03T10:19:33.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:33.889+0000 7f21a57fa700 1 -- 172.21.15.3:0/661631194 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f219c0232f0 con 0x7f21a806e570 2023-10-03T10:19:34.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.021+0000 7fc98bea8700 1 -- 172.21.15.3:0/3944752858 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc98406c360 con 0x7fc98409e000 2023-10-03T10:19:34.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.022+0000 7fc97a7fc700 1 -- 172.21.15.3:0/3944752858 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc974018990 con 0x7fc98409e000 2023-10-03T10:19:34.022 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:34.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.024+0000 7fc98bea8700 1 -- 172.21.15.3:0/3944752858 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc98409e000 msgr2=0x7fc98411f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:34.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.024+0000 7fc98bea8700 1 --2- 172.21.15.3:0/3944752858 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc98409e000 0x7fc98411f5a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc974010af0 tx=0x7fc974010e00 comp rx=0 tx=0).stop 2023-10-03T10:19:34.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.024+0000 7fc98bea8700 1 -- 172.21.15.3:0/3944752858 shutdown_connections 2023-10-03T10:19:34.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.024+0000 7fc98bea8700 1 --2- 172.21.15.3:0/3944752858 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc98409e410 0x7fc98411fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:34.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.024+0000 7fc98bea8700 1 --2- 172.21.15.3:0/3944752858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc98409d6f0 0x7fc98411f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:34.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.024+0000 7fc98bea8700 1 --2- 172.21.15.3:0/3944752858 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc98409e000 0x7fc98411f5a0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:34.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.024+0000 7fc98bea8700 1 -- 172.21.15.3:0/3944752858 >> 172.21.15.3:0/3944752858 conn(0x7fc984066f50 msgr2=0x7fc984119480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:34.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.024+0000 7fc98bea8700 1 -- 172.21.15.3:0/3944752858 shutdown_connections 2023-10-03T10:19:34.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.024+0000 7fc98bea8700 1 -- 172.21.15.3:0/3944752858 wait complete. 2023-10-03T10:19:34.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.035+0000 7f5bbe16b700 1 -- 172.21.15.3:0/1669614384 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5bb8074ea0 con 0x7f5bb809d6f0 2023-10-03T10:19:34.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.035+0000 7f5b9ffff700 1 -- 172.21.15.3:0/1669614384 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5ba8028030 con 0x7f5bb809d6f0 2023-10-03T10:19:34.035 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:34.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.038+0000 7f5bbe16b700 1 -- 172.21.15.3:0/1669614384 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5bb809d6f0 msgr2=0x7f5bb81222f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:34.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.038+0000 7f5bbe16b700 1 --2- 172.21.15.3:0/1669614384 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5bb809d6f0 0x7f5bb81222f0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f5ba800e8e0 tx=0x7f5ba800ebf0 comp rx=0 tx=0).stop 2023-10-03T10:19:34.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.038+0000 7f5bbe16b700 1 -- 172.21.15.3:0/1669614384 shutdown_connections 2023-10-03T10:19:34.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.038+0000 7f5bbe16b700 1 --2- 172.21.15.3:0/1669614384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bb809e410 0x7f5bb8122d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:34.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.038+0000 7f5bbe16b700 1 --2- 172.21.15.3:0/1669614384 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5bb809d6f0 0x7f5bb81222f0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:34.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.038+0000 7f5bbe16b700 1 --2- 172.21.15.3:0/1669614384 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5bb809e000 0x7f5bb8122830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:34.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.038+0000 7f5bbe16b700 1 -- 172.21.15.3:0/1669614384 >> 172.21.15.3:0/1669614384 conn(0x7f5bb8066f50 msgr2=0x7f5bb8130a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:34.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.038+0000 7f5bbe16b700 1 -- 172.21.15.3:0/1669614384 shutdown_connections 2023-10-03T10:19:34.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.038+0000 7f5bbe16b700 1 -- 172.21.15.3:0/1669614384 wait complete. 2023-10-03T10:19:34.040 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:34.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.050+0000 7f48a5445700 1 -- 172.21.15.3:0/611831139 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f48a0123120 con 0x7f48a009de50 2023-10-03T10:19:34.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.050+0000 7f48877fe700 1 -- 172.21.15.3:0/611831139 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4890015020 con 0x7f48a009de50 2023-10-03T10:19:34.051 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:34.052 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:34.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.053+0000 7f48a5445700 1 -- 172.21.15.3:0/611831139 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48a009de50 msgr2=0x7f48a01214e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:34.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.053+0000 7f48a5445700 1 --2- 172.21.15.3:0/611831139 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48a009de50 0x7f48a01214e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f4890000c00 tx=0x7f4890004750 comp rx=0 tx=0).stop 2023-10-03T10:19:34.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.053+0000 7f48a5445700 1 -- 172.21.15.3:0/611831139 shutdown_connections 2023-10-03T10:19:34.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.053+0000 7f48a5445700 1 --2- 172.21.15.3:0/611831139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a01338a0 0x7f48a0121f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:34.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.053+0000 7f48a5445700 1 --2- 172.21.15.3:0/611831139 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f48a009de50 0x7f48a01214e0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:34.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.053+0000 7f48a5445700 1 --2- 172.21.15.3:0/611831139 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f48a009e760 0x7f48a0121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:34.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.053+0000 7f48a5445700 1 -- 172.21.15.3:0/611831139 >> 172.21.15.3:0/611831139 conn(0x7f48a00a2aa0 msgr2=0x7f48a01ba640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:34.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.053+0000 7f48a5445700 1 -- 172.21.15.3:0/611831139 shutdown_connections 2023-10-03T10:19:34.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.053+0000 7f48a5445700 1 -- 172.21.15.3:0/611831139 wait complete. 2023-10-03T10:19:34.067 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:34.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.100+0000 7f21aed9a700 1 -- 172.21.15.3:0/661631194 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f21a806b9e0 con 0x7f21a806e570 2023-10-03T10:19:34.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.100+0000 7f21a57fa700 1 -- 172.21.15.3:0/661631194 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f219c021030 con 0x7f21a806e570 2023-10-03T10:19:34.101 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.102+0000 7f21aed9a700 1 -- 172.21.15.3:0/661631194 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21a806e570 msgr2=0x7f21a813dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.102+0000 7f21aed9a700 1 --2- 172.21.15.3:0/661631194 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21a806e570 0x7f21a813dc20 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f219c00ebe0 tx=0x7f219c00eef0 comp rx=0 tx=0).stop 2023-10-03T10:19:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.102+0000 7f21aed9a700 1 -- 172.21.15.3:0/661631194 shutdown_connections 2023-10-03T10:19:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.102+0000 7f21aed9a700 1 --2- 172.21.15.3:0/661631194 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f21a813b850 0x7f21a81253d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.102+0000 7f21aed9a700 1 --2- 172.21.15.3:0/661631194 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f21a806e570 0x7f21a813dc20 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.102+0000 7f21aed9a700 1 --2- 172.21.15.3:0/661631194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21a806ee80 0x7f21a8124e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:34.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.102+0000 7f21aed9a700 1 -- 172.21.15.3:0/661631194 >> 172.21.15.3:0/661631194 conn(0x7f21a80a2aa0 msgr2=0x7f21a80a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:34.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.103+0000 7f21aed9a700 1 -- 172.21.15.3:0/661631194 shutdown_connections 2023-10-03T10:19:34.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:34.103+0000 7f21aed9a700 1 -- 172.21.15.3:0/661631194 wait complete. 2023-10-03T10:19:34.116 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:35.041 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:35.053 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:35.068 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:35.117 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:35.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.202+0000 7fd471cce700 1 Processor -- start 2023-10-03T10:19:35.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.202+0000 7fd471cce700 1 -- start start 2023-10-03T10:19:35.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.203+0000 7fd471cce700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd46c09d6f0 0x7fd46c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:35.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.203+0000 7fd471cce700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd46c09e000 0x7fd46c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:35.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.203+0000 7fd471cce700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd46c09e410 0x7fd46c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:35.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.203+0000 7fd471cce700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd46c06b460 con 0x7fd46c09d6f0 2023-10-03T10:19:35.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.203+0000 7fd471cce700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c09b120 con 0x7fd46c09e000 2023-10-03T10:19:35.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.203+0000 7fd471cce700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c09a7e0 con 0x7fd46c09e410 2023-10-03T10:19:35.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.203+0000 7fd46affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd46c09e000 0x7fd46c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:35.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.203+0000 7fd46affd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd46c09e000 0x7fd46c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40964/0 (socket says 172.21.15.3:40964) 2023-10-03T10:19:35.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.203+0000 7fd46affd700 1 -- 172.21.15.3:0/1209923295 learned_addr learned my addr 172.21.15.3:0/1209923295 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:35.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.203+0000 7fd46b7fe700 1 --2- 172.21.15.3:0/1209923295 >> v2:172.21.15.3:3300/0 conn(0x7fd46c09d6f0 0x7fd46c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:35.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.204+0000 7fd46bfff700 1 --2- 172.21.15.3:0/1209923295 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd46c09e410 0x7fd46c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:35.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.204+0000 7fd46b7fe700 1 -- 172.21.15.3:0/1209923295 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd46c09e410 msgr2=0x7fd46c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:35.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.204+0000 7fd46b7fe700 1 --2- 172.21.15.3:0/1209923295 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd46c09e410 0x7fd46c133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.204+0000 7fd46b7fe700 1 -- 172.21.15.3:0/1209923295 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd46c09e000 msgr2=0x7fd46c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:35.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.204+0000 7fd46b7fe700 1 --2- 172.21.15.3:0/1209923295 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd46c09e000 0x7fd46c1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.204+0000 7fd46b7fe700 1 -- 172.21.15.3:0/1209923295 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46c09a500 con 0x7fd46c09d6f0 2023-10-03T10:19:35.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.204+0000 7fd46affd700 1 --2- 172.21.15.3:0/1209923295 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd46c09e000 0x7fd46c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:35.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.204+0000 7fd46b7fe700 1 --2- 172.21.15.3:0/1209923295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c09d6f0 0x7fd46c09dac0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009840 tx=0x7fd45c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f47fbd360ad6df8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:35.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.204+0000 7fd46a7fc700 1 -- 172.21.15.3:0/1209923295 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c011830 con 0x7fd46c09d6f0 2023-10-03T10:19:35.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.204+0000 7fd46a7fc700 1 -- 172.21.15.3:0/1209923295 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45c011990 con 0x7fd46c09d6f0 2023-10-03T10:19:35.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.204+0000 7fd46a7fc700 1 -- 172.21.15.3:0/1209923295 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c011d20 con 0x7fd46c09d6f0 2023-10-03T10:19:35.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.205+0000 7fd471cce700 1 -- 172.21.15.3:0/1209923295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c09d6f0 msgr2=0x7fd46c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:35.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.205+0000 7fd471cce700 1 --2- 172.21.15.3:0/1209923295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c09d6f0 0x7fd46c09dac0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009840 tx=0x7fd45c009b50 comp rx=0 tx=0).stop 2023-10-03T10:19:35.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.205+0000 7fd471cce700 1 -- 172.21.15.3:0/1209923295 shutdown_connections 2023-10-03T10:19:35.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.205+0000 7fd471cce700 1 --2- 172.21.15.3:0/1209923295 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd46c09e410 0x7fd46c133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.205+0000 7fd471cce700 1 --2- 172.21.15.3:0/1209923295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c09d6f0 0x7fd46c09dac0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.205+0000 7fd471cce700 1 --2- 172.21.15.3:0/1209923295 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd46c09e000 0x7fd46c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.205+0000 7fd471cce700 1 -- 172.21.15.3:0/1209923295 >> 172.21.15.3:0/1209923295 conn(0x7fd46c066f50 msgr2=0x7fd46c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:35.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.205+0000 7fd471cce700 1 -- 172.21.15.3:0/1209923295 shutdown_connections 2023-10-03T10:19:35.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.205+0000 7fd471cce700 1 -- 172.21.15.3:0/1209923295 wait complete. 2023-10-03T10:19:35.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.207+0000 7fd471cce700 1 Processor -- start 2023-10-03T10:19:35.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.207+0000 7fd471cce700 1 -- start start 2023-10-03T10:19:35.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.208+0000 7fd471cce700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd46c09d6f0 0x7fd46c1a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:35.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.208+0000 7fd471cce700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c09e000 0x7fd46c1a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:35.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.208+0000 7fd471cce700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd46c09e410 0x7fd46c1c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:35.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.208+0000 7fd471cce700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c0a1f20 con 0x7fd46c09e000 2023-10-03T10:19:35.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.208+0000 7fd471cce700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c1a80a0 con 0x7fd46c09e410 2023-10-03T10:19:35.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.208+0000 7fd471cce700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c1b91f0 con 0x7fd46c09d6f0 2023-10-03T10:19:35.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.208+0000 7fd46bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd46c09e410 0x7fd46c1c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:35.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.208+0000 7fd46b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd46c09d6f0 0x7fd46c1a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:35.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.208+0000 7fd46bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd46c09e410 0x7fd46c1c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40974/0 (socket says 172.21.15.3:40974) 2023-10-03T10:19:35.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.208+0000 7fd46bfff700 1 -- 172.21.15.3:0/798272561 learned_addr learned my addr 172.21.15.3:0/798272561 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:35.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.208+0000 7fd46affd700 1 --2- 172.21.15.3:0/798272561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c09e000 0x7fd46c1a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:35.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.208+0000 7fd46bfff700 1 -- 172.21.15.3:0/798272561 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd46c09d6f0 msgr2=0x7fd46c1a5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:35.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.208+0000 7fd46bfff700 1 --2- 172.21.15.3:0/798272561 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd46c09d6f0 0x7fd46c1a5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.208+0000 7fd46bfff700 1 -- 172.21.15.3:0/798272561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c09e000 msgr2=0x7fd46c1a5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:35.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.208+0000 7fd46bfff700 1 --2- 172.21.15.3:0/798272561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c09e000 0x7fd46c1a5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.208+0000 7fd46bfff700 1 -- 172.21.15.3:0/798272561 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd460009550 con 0x7fd46c09e410 2023-10-03T10:19:35.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.209+0000 7fd46bfff700 1 --2- 172.21.15.3:0/798272561 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd46c09e410 0x7fd46c1c8040 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fd464009dd0 tx=0x7fd46400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:35.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.209+0000 7fd453fff700 1 -- 172.21.15.3:0/798272561 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd464009c40 con 0x7fd46c09e410 2023-10-03T10:19:35.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.209+0000 7fd453fff700 1 -- 172.21.15.3:0/798272561 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46400f040 con 0x7fd46c09e410 2023-10-03T10:19:35.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.209+0000 7fd471cce700 1 -- 172.21.15.3:0/798272561 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd45c0094a0 con 0x7fd46c09e410 2023-10-03T10:19:35.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.209+0000 7fd453fff700 1 -- 172.21.15.3:0/798272561 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46400ceb0 con 0x7fd46c09e410 2023-10-03T10:19:35.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.209+0000 7fd471cce700 1 -- 172.21.15.3:0/798272561 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd46c1a6c60 con 0x7fd46c09e410 2023-10-03T10:19:35.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.210+0000 7fd453fff700 1 -- 172.21.15.3:0/798272561 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd464014480 con 0x7fd46c09e410 2023-10-03T10:19:35.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.210+0000 7fd471cce700 1 -- 172.21.15.3:0/798272561 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd46c1a64d0 con 0x7fd46c09e410 2023-10-03T10:19:35.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.210+0000 7fd453fff700 1 -- 172.21.15.3:0/798272561 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd4640401f0 con 0x7fd46c09e410 2023-10-03T10:19:35.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.219+0000 7fd453fff700 1 -- 172.21.15.3:0/798272561 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd464007920 con 0x7fd46c09e410 2023-10-03T10:19:35.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.239+0000 7f0fd2a6d700 1 Processor -- start 2023-10-03T10:19:35.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.239+0000 7f0fd2a6d700 1 -- start start 2023-10-03T10:19:35.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.239+0000 7f0fd2a6d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0fcc06a770 0x7f0fcc06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:35.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.239+0000 7f0fd2a6d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0fcc131d10 0x7f0fcc139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:35.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.239+0000 7f0fd2a6d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fcc09ece0 con 0x7f0fcc06b080 2023-10-03T10:19:35.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.239+0000 7f0fd2a6d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc09e760 con 0x7f0fcc131d10 2023-10-03T10:19:35.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.239+0000 7f0fd2a6d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc09e040 con 0x7f0fcc06a770 2023-10-03T10:19:35.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.240+0000 7f0fcbfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0fcc06b080 0x7f0fcc131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47378/0 (socket says 172.21.15.3:47378) 2023-10-03T10:19:35.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.240+0000 7f0fcbfff700 1 -- 172.21.15.3:0/2241106518 learned_addr learned my addr 172.21.15.3:0/2241106518 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:35.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.240+0000 7f0fd0809700 1 --2- 172.21.15.3:0/2241106518 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0fcc06a770 0x7f0fcc06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:35.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.240+0000 7f0fd0809700 1 -- 172.21.15.3:0/2241106518 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0fcc131d10 msgr2=0x7f0fcc139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:35.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.240+0000 7f0fd0809700 1 --2- 172.21.15.3:0/2241106518 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0fcc131d10 0x7f0fcc139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.240+0000 7f0fcb7fe700 1 -- 172.21.15.3:0/2241106518 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3284916873 0 0) 0x7f0fcc09ece0 con 0x7f0fcc06b080 2023-10-03T10:19:35.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.240+0000 7f0fd0809700 1 -- 172.21.15.3:0/2241106518 >> v1:172.21.15.3:6789/0 conn(0x7f0fcc06b080 legacy=0x7f0fcc131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:35.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.240+0000 7f0fd0809700 1 -- 172.21.15.3:0/2241106518 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fcc09f630 con 0x7f0fcc06a770 2023-10-03T10:19:35.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.240+0000 7f0fd0809700 1 --2- 172.21.15.3:0/2241106518 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0fcc06a770 0x7f0fcc06ab40 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0009870 tx=0x7f0fc0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48c9d51f5267c938 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:35.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.240+0000 7f0fcb7fe700 1 -- 172.21.15.3:0/2241106518 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0011830 con 0x7f0fcc06a770 2023-10-03T10:19:35.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.241+0000 7f0fcb7fe700 1 -- 172.21.15.3:0/2241106518 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc0011990 con 0x7f0fcc06a770 2023-10-03T10:19:35.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.241+0000 7f0fcb7fe700 1 -- 172.21.15.3:0/2241106518 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0011d20 con 0x7f0fcc06a770 2023-10-03T10:19:35.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.241+0000 7f0fd2a6d700 1 -- 172.21.15.3:0/2241106518 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0fcc06a770 msgr2=0x7f0fcc06ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:35.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.241+0000 7f0fd2a6d700 1 --2- 172.21.15.3:0/2241106518 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0fcc06a770 0x7f0fcc06ab40 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0009870 tx=0x7f0fc0009b80 comp rx=0 tx=0).stop 2023-10-03T10:19:35.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.241+0000 7f0fd2a6d700 1 -- 172.21.15.3:0/2241106518 shutdown_connections 2023-10-03T10:19:35.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.241+0000 7f0fd2a6d700 1 --2- 172.21.15.3:0/2241106518 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0fcc131d10 0x7f0fcc139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.241+0000 7f0fd2a6d700 1 --2- 172.21.15.3:0/2241106518 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0fcc06a770 0x7f0fcc06ab40 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.241+0000 7f0fd2a6d700 1 -- 172.21.15.3:0/2241106518 >> 172.21.15.3:0/2241106518 conn(0x7f0fcc066f50 msgr2=0x7f0fcc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:35.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.241+0000 7f87caf72700 1 Processor -- start 2023-10-03T10:19:35.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.242+0000 7f0fd2a6d700 1 -- 172.21.15.3:0/2241106518 shutdown_connections 2023-10-03T10:19:35.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.241+0000 7f87caf72700 1 -- start start 2023-10-03T10:19:35.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.242+0000 7f0fd2a6d700 1 -- 172.21.15.3:0/2241106518 wait complete. 2023-10-03T10:19:35.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.242+0000 7f87caf72700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f87c41338a0 0x7f87c4133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:35.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.242+0000 7f87caf72700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f87c409de50 0x7f87c409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:35.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.242+0000 7f87caf72700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87c406ef30 con 0x7f87c409e760 2023-10-03T10:19:35.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.242+0000 7f87caf72700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c406fef0 con 0x7f87c41338a0 2023-10-03T10:19:35.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.242+0000 7f87caf72700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f87c41334b0 con 0x7f87c409de50 2023-10-03T10:19:35.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.242+0000 7f87c8d0e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f87c41338a0 0x7f87c4133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:35.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.243+0000 7f87c8d0e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f87c41338a0 0x7f87c4133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40994/0 (socket says 172.21.15.3:40994) 2023-10-03T10:19:35.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.243+0000 7f87c8d0e700 1 -- 172.21.15.3:0/1630064535 learned_addr learned my addr 172.21.15.3:0/1630064535 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:35.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.243+0000 7f87c3fff700 1 --2- 172.21.15.3:0/1630064535 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f87c409de50 0x7f87c409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:35.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.243+0000 7f0fd2a6d700 1 Processor -- start 2023-10-03T10:19:35.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.243+0000 7f87c8d0e700 1 -- 172.21.15.3:0/1630064535 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f87c409de50 msgr2=0x7f87c409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:35.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.243+0000 7f87c8d0e700 1 --2- 172.21.15.3:0/1630064535 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f87c409de50 0x7f87c409e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.243+0000 7f87c8d0e700 1 -- 172.21.15.3:0/1630064535 >> v1:172.21.15.3:6789/0 conn(0x7f87c409e760 legacy=0x7f87c413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:35.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.243+0000 7f87c8d0e700 1 -- 172.21.15.3:0/1630064535 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c406fc30 con 0x7f87c41338a0 2023-10-03T10:19:35.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.243+0000 7f0fd2a6d700 1 -- start start 2023-10-03T10:19:35.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.243+0000 7f87c8d0e700 1 --2- 172.21.15.3:0/1630064535 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f87c41338a0 0x7f87c4133c70 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009920 tx=0x7f87b8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46a5ff09dd7ee44f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:35.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.243+0000 7f87c37fe700 1 -- 172.21.15.3:0/1630064535 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b8004230 con 0x7f87c41338a0 2023-10-03T10:19:35.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.244+0000 7f87c37fe700 1 -- 172.21.15.3:0/1630064535 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b8004390 con 0x7f87c41338a0 2023-10-03T10:19:35.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.244+0000 7f87c37fe700 1 -- 172.21.15.3:0/1630064535 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b80036a0 con 0x7f87c41338a0 2023-10-03T10:19:35.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.244+0000 7f0fd2a6d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0fcc06a770 0x7f0fcc126dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:35.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.244+0000 7f0fd2a6d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0fcc06b080 0x7f0fcc127300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:35.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.244+0000 7f0fd2a6d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fcc131d10 0x7f0fcc127840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:35.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.244+0000 7f0fd2a6d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc129000 con 0x7f0fcc131d10 2023-10-03T10:19:35.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.244+0000 7f0fd2a6d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc129c00 con 0x7f0fcc06a770 2023-10-03T10:19:35.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.244+0000 7f0fd2a6d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc071e60 con 0x7f0fcc06b080 2023-10-03T10:19:35.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.244+0000 7f0fd100a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fcc131d10 0x7f0fcc127840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:35.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.244+0000 7f0fd100a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fcc131d10 0x7f0fcc127840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37642/0 (socket says 172.21.15.3:37642) 2023-10-03T10:19:35.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.244+0000 7f0fd100a700 1 -- 172.21.15.3:0/2208267159 learned_addr learned my addr 172.21.15.3:0/2208267159 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:35.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.244+0000 7f0fd0809700 1 --2- 172.21.15.3:0/2208267159 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0fcc06a770 0x7f0fcc126dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:35.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.244+0000 7f0fcbfff700 1 --2- 172.21.15.3:0/2208267159 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0fcc06b080 0x7f0fcc127300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:35.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.245+0000 7f0fd0809700 1 -- 172.21.15.3:0/2208267159 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0fcc06b080 msgr2=0x7f0fcc127300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:35.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.245+0000 7f0fd0809700 1 --2- 172.21.15.3:0/2208267159 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0fcc06b080 0x7f0fcc127300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.245+0000 7f0fd0809700 1 -- 172.21.15.3:0/2208267159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fcc131d10 msgr2=0x7f0fcc127840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:35.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.245+0000 7f0fd0809700 1 --2- 172.21.15.3:0/2208267159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fcc131d10 0x7f0fcc127840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.245+0000 7f0fd0809700 1 -- 172.21.15.3:0/2208267159 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc00094a0 con 0x7f0fcc06a770 2023-10-03T10:19:35.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.245+0000 7f87caf72700 1 -- 172.21.15.3:0/1630064535 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f87c41338a0 msgr2=0x7f87c4133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:35.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.245+0000 7f87caf72700 1 --2- 172.21.15.3:0/1630064535 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f87c41338a0 0x7f87c4133c70 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009920 tx=0x7f87b8009c30 comp rx=0 tx=0).stop 2023-10-03T10:19:35.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.245+0000 7f87caf72700 1 -- 172.21.15.3:0/1630064535 shutdown_connections 2023-10-03T10:19:35.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.245+0000 7f87caf72700 1 --2- 172.21.15.3:0/1630064535 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f87c41338a0 0x7f87c4133c70 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.245+0000 7f87caf72700 1 --2- 172.21.15.3:0/1630064535 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f87c409de50 0x7f87c409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.245+0000 7f87caf72700 1 -- 172.21.15.3:0/1630064535 >> 172.21.15.3:0/1630064535 conn(0x7f87c40a2aa0 msgr2=0x7f87c40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:35.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.245+0000 7f87caf72700 1 -- 172.21.15.3:0/1630064535 shutdown_connections 2023-10-03T10:19:35.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.245+0000 7f0fd0809700 1 --2- 172.21.15.3:0/2208267159 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0fcc06a770 0x7f0fcc126dc0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0016000 tx=0x7f0fc0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:35.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.245+0000 7f87caf72700 1 -- 172.21.15.3:0/1630064535 wait complete. 2023-10-03T10:19:35.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.245+0000 7f0fc97fa700 1 -- 172.21.15.3:0/2208267159 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0004590 con 0x7f0fcc06a770 2023-10-03T10:19:35.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.245+0000 7f0fc97fa700 1 -- 172.21.15.3:0/2208267159 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc001f070 con 0x7f0fcc06a770 2023-10-03T10:19:35.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.245+0000 7f0fd2a6d700 1 -- 172.21.15.3:0/2208267159 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fcc1293a0 con 0x7f0fcc06a770 2023-10-03T10:19:35.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.245+0000 7f0fc97fa700 1 -- 172.21.15.3:0/2208267159 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc00038e0 con 0x7f0fcc06a770 2023-10-03T10:19:35.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.245+0000 7f0fd2a6d700 1 -- 172.21.15.3:0/2208267159 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fcc1282d0 con 0x7f0fcc06a770 2023-10-03T10:19:35.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.247+0000 7f87caf72700 1 Processor -- start 2023-10-03T10:19:35.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.247+0000 7f87caf72700 1 -- start start 2023-10-03T10:19:35.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.247+0000 7f0fc97fa700 1 -- 172.21.15.3:0/2208267159 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0fc0003a40 con 0x7f0fcc06a770 2023-10-03T10:19:35.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.247+0000 7f0fd2a6d700 1 -- 172.21.15.3:0/2208267159 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fcc074ea0 con 0x7f0fcc06a770 2023-10-03T10:19:35.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.247+0000 7f0fc97fa700 1 -- 172.21.15.3:0/2208267159 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0fc0021070 con 0x7f0fcc06a770 2023-10-03T10:19:35.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.248+0000 7f87caf72700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87c409de50 0x7f87c4119710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:35.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.248+0000 7f87caf72700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f87c409e760 0x7f87c4119c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:35.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.248+0000 7f87caf72700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f87c41338a0 0x7f87c411a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:35.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.248+0000 7f87caf72700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c411d230 con 0x7f87c409de50 2023-10-03T10:19:35.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.248+0000 7f87caf72700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c411c9c0 con 0x7f87c41338a0 2023-10-03T10:19:35.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.248+0000 7f87caf72700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f87c406f640 con 0x7f87c409e760 2023-10-03T10:19:35.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.248+0000 7f87c3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f87c409e760 0x7f87c4119c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:35.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.248+0000 7f87c3fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f87c409e760 0x7f87c4119c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48186/0 (socket says 172.21.15.3:48186) 2023-10-03T10:19:35.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.248+0000 7f87c3fff700 1 -- 172.21.15.3:0/1885780054 learned_addr learned my addr 172.21.15.3:0/1885780054 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:35.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.248+0000 7f87c8d0e700 1 --2- 172.21.15.3:0/1885780054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87c409de50 0x7f87c4119710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:35.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.248+0000 7f87c950f700 1 --2- 172.21.15.3:0/1885780054 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f87c41338a0 0x7f87c411a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:35.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.249+0000 7f87c950f700 1 -- 172.21.15.3:0/1885780054 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f87c409e760 msgr2=0x7f87c4119c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:35.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.249+0000 7f87c950f700 1 --2- 172.21.15.3:0/1885780054 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f87c409e760 0x7f87c4119c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.249+0000 7f87c950f700 1 -- 172.21.15.3:0/1885780054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87c409de50 msgr2=0x7f87c4119710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:35.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.249+0000 7f87c950f700 1 --2- 172.21.15.3:0/1885780054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87c409de50 0x7f87c4119710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.249+0000 7f87c950f700 1 -- 172.21.15.3:0/1885780054 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b8009550 con 0x7f87c41338a0 2023-10-03T10:19:35.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.249+0000 7f87c3fff700 1 --2- 172.21.15.3:0/1885780054 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f87c409e760 0x7f87c4119c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:35.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.249+0000 7f87c950f700 1 --2- 172.21.15.3:0/1885780054 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f87c41338a0 0x7f87c411a190 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f87bc00cb10 tx=0x7f87bc00ce20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:35.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.252+0000 7f0fc97fa700 1 -- 172.21.15.3:0/2208267159 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0fc000bdf0 con 0x7f0fcc06a770 2023-10-03T10:19:35.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.252+0000 7f87c17fa700 1 -- 172.21.15.3:0/1885780054 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc00a9e0 con 0x7f87c41338a0 2023-10-03T10:19:35.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.252+0000 7f87c17fa700 1 -- 172.21.15.3:0/1885780054 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87bc00ab40 con 0x7f87c41338a0 2023-10-03T10:19:35.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.252+0000 7f87c17fa700 1 -- 172.21.15.3:0/1885780054 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc012420 con 0x7f87c41338a0 2023-10-03T10:19:35.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.252+0000 7f87caf72700 1 -- 172.21.15.3:0/1885780054 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c40a0d40 con 0x7f87c41338a0 2023-10-03T10:19:35.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.252+0000 7f87caf72700 1 -- 172.21.15.3:0/1885780054 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c411bb30 con 0x7f87c41338a0 2023-10-03T10:19:35.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.254+0000 7f87caf72700 1 -- 172.21.15.3:0/1885780054 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87c411cbf0 con 0x7f87c41338a0 2023-10-03T10:19:35.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.254+0000 7f87c17fa700 1 -- 172.21.15.3:0/1885780054 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f87bc012580 con 0x7f87c41338a0 2023-10-03T10:19:35.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.255+0000 7f87c17fa700 1 -- 172.21.15.3:0/1885780054 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f87bc040240 con 0x7f87c41338a0 2023-10-03T10:19:35.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.259+0000 7f87c17fa700 1 -- 172.21.15.3:0/1885780054 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f87bc023940 con 0x7f87c41338a0 2023-10-03T10:19:35.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.280+0000 7fb00e210700 1 Processor -- start 2023-10-03T10:19:35.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.280+0000 7fb00e210700 1 -- start start 2023-10-03T10:19:35.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.280+0000 7fb00e210700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb00806c0e0 0x7fb00809ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:35.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.280+0000 7fb00e210700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb008134ec0 0x7fb0081379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:35.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.280+0000 7fb00e210700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb008134980 con 0x7fb008137f20 2023-10-03T10:19:35.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.280+0000 7fb00e210700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb00806c4b0 con 0x7fb00806c0e0 2023-10-03T10:19:35.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.280+0000 7fb00e210700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080a09c0 con 0x7fb008134ec0 2023-10-03T10:19:35.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.280+0000 7fb0077fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb00806c0e0 0x7fb00809ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:35.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.281+0000 7fb0077fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb00806c0e0 0x7fb00809ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37654/0 (socket says 172.21.15.3:37654) 2023-10-03T10:19:35.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.281+0000 7fb0077fe700 1 -- 172.21.15.3:0/2510595551 learned_addr learned my addr 172.21.15.3:0/2510595551 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:35.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.281+0000 7fb006ffd700 1 --2- 172.21.15.3:0/2510595551 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb008134ec0 0x7fb0081379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:35.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.281+0000 7fb0077fe700 1 -- 172.21.15.3:0/2510595551 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb008134ec0 msgr2=0x7fb0081379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:35.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.281+0000 7fb0077fe700 1 --2- 172.21.15.3:0/2510595551 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb008134ec0 0x7fb0081379e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.281+0000 7fb0077fe700 1 -- 172.21.15.3:0/2510595551 >> v1:172.21.15.3:6789/0 conn(0x7fb008137f20 legacy=0x7fb00813ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:35.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.281+0000 7fb0077fe700 1 -- 172.21.15.3:0/2510595551 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0080a0600 con 0x7fb00806c0e0 2023-10-03T10:19:35.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.281+0000 7fb0077fe700 1 --2- 172.21.15.3:0/2510595551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb00806c0e0 0x7fb00809ac90 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7faff80098f0 tx=0x7faff8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bfffe2ef712db24 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:35.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.281+0000 7fb0067fc700 1 -- 172.21.15.3:0/2510595551 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8011830 con 0x7fb00806c0e0 2023-10-03T10:19:35.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.281+0000 7fb0067fc700 1 -- 172.21.15.3:0/2510595551 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff8011990 con 0x7fb00806c0e0 2023-10-03T10:19:35.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.281+0000 7fb0067fc700 1 -- 172.21.15.3:0/2510595551 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8011d20 con 0x7fb00806c0e0 2023-10-03T10:19:35.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.282+0000 7fb00e210700 1 -- 172.21.15.3:0/2510595551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb00806c0e0 msgr2=0x7fb00809ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:35.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.282+0000 7fb00e210700 1 --2- 172.21.15.3:0/2510595551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb00806c0e0 0x7fb00809ac90 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7faff80098f0 tx=0x7faff8009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.282+0000 7fb00e210700 1 -- 172.21.15.3:0/2510595551 shutdown_connections 2023-10-03T10:19:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.282+0000 7fb00e210700 1 --2- 172.21.15.3:0/2510595551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb00806c0e0 0x7fb00809ac90 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.282+0000 7fb00e210700 1 --2- 172.21.15.3:0/2510595551 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb008134ec0 0x7fb0081379e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.282+0000 7fb00e210700 1 -- 172.21.15.3:0/2510595551 >> 172.21.15.3:0/2510595551 conn(0x7fb0080a4210 msgr2=0x7fb0080a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.282+0000 7fb00e210700 1 -- 172.21.15.3:0/2510595551 shutdown_connections 2023-10-03T10:19:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.282+0000 7fb00e210700 1 -- 172.21.15.3:0/2510595551 wait complete. 2023-10-03T10:19:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.283+0000 7fb00e210700 1 Processor -- start 2023-10-03T10:19:35.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.283+0000 7fb00e210700 1 -- start start 2023-10-03T10:19:35.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.284+0000 7fb00e210700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb00806c0e0 0x7fb0081ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:35.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.284+0000 7fb00e210700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb008134ec0 0x7fb0081cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:35.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.284+0000 7fb00e210700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb008137f20 0x7fb00811a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.284+0000 7fb00e210700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00811cf50 con 0x7fb008134ec0 2023-10-03T10:19:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.284+0000 7fb00e210700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0081272a0 con 0x7fb008137f20 2023-10-03T10:19:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.284+0000 7fb00e210700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080a03f0 con 0x7fb00806c0e0 2023-10-03T10:19:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.284+0000 7fb006ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb008134ec0 0x7fb0081cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.284+0000 7fb006ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb008134ec0 0x7fb0081cd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37656/0 (socket says 172.21.15.3:37656) 2023-10-03T10:19:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.284+0000 7fb006ffd700 1 -- 172.21.15.3:0/4086904978 learned_addr learned my addr 172.21.15.3:0/4086904978 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.284+0000 7fb0077fe700 1 --2- 172.21.15.3:0/4086904978 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb00806c0e0 0x7fb0081ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.284+0000 7fb007fff700 1 --2- 172.21.15.3:0/4086904978 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb008137f20 0x7fb00811a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.285+0000 7fb006ffd700 1 -- 172.21.15.3:0/4086904978 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb00806c0e0 msgr2=0x7fb0081ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.285+0000 7fb006ffd700 1 --2- 172.21.15.3:0/4086904978 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb00806c0e0 0x7fb0081ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.285+0000 7fb006ffd700 1 -- 172.21.15.3:0/4086904978 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb008137f20 msgr2=0x7fb00811a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.285+0000 7fb006ffd700 1 --2- 172.21.15.3:0/4086904978 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb008137f20 0x7fb00811a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.285+0000 7fb006ffd700 1 -- 172.21.15.3:0/4086904978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff8009550 con 0x7fb008134ec0 2023-10-03T10:19:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.285+0000 7fb006ffd700 1 --2- 172.21.15.3:0/4086904978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb008134ec0 0x7fb0081cd290 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7faffc00e9b0 tx=0x7faffc00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.285+0000 7fafeffff700 1 -- 172.21.15.3:0/4086904978 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc009710 con 0x7fb008134ec0 2023-10-03T10:19:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.285+0000 7fafeffff700 1 -- 172.21.15.3:0/4086904978 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faffc00ee50 con 0x7fb008134ec0 2023-10-03T10:19:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.285+0000 7fb00e210700 1 -- 172.21.15.3:0/4086904978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb00811dcb0 con 0x7fb008134ec0 2023-10-03T10:19:35.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.285+0000 7fafeffff700 1 -- 172.21.15.3:0/4086904978 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc00ca50 con 0x7fb008134ec0 2023-10-03T10:19:35.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.285+0000 7fb00e210700 1 -- 172.21.15.3:0/4086904978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0080a3d80 con 0x7fb008134ec0 2023-10-03T10:19:35.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.286+0000 7fafeffff700 1 -- 172.21.15.3:0/4086904978 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faffc00cbb0 con 0x7fb008134ec0 2023-10-03T10:19:35.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.286+0000 7fafeffff700 1 -- 172.21.15.3:0/4086904978 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7faffc03f790 con 0x7fb008134ec0 2023-10-03T10:19:35.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.287+0000 7fb00e210700 1 -- 172.21.15.3:0/4086904978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb00811d2d0 con 0x7fb008134ec0 2023-10-03T10:19:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.292+0000 7fafeffff700 1 -- 172.21.15.3:0/4086904978 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faffc021020 con 0x7fb008134ec0 2023-10-03T10:19:35.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.462+0000 7f0fd2a6d700 1 -- 172.21.15.3:0/2208267159 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0fcc06c360 con 0x7f0fcc06a770 2023-10-03T10:19:35.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.463+0000 7f0fc97fa700 1 -- 172.21.15.3:0/2208267159 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0fc002a070 con 0x7f0fcc06a770 2023-10-03T10:19:35.463 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:35.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.466+0000 7f0fd2a6d700 1 -- 172.21.15.3:0/2208267159 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0fcc06a770 msgr2=0x7f0fcc126dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:35.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.466+0000 7f0fd2a6d700 1 --2- 172.21.15.3:0/2208267159 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0fcc06a770 0x7f0fcc126dc0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0016000 tx=0x7f0fc0004750 comp rx=0 tx=0).stop 2023-10-03T10:19:35.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.466+0000 7f0fd2a6d700 1 -- 172.21.15.3:0/2208267159 shutdown_connections 2023-10-03T10:19:35.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.466+0000 7f0fd2a6d700 1 --2- 172.21.15.3:0/2208267159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fcc131d10 0x7f0fcc127840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.466+0000 7f0fd2a6d700 1 --2- 172.21.15.3:0/2208267159 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0fcc06a770 0x7f0fcc126dc0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.466+0000 7f0fd2a6d700 1 --2- 172.21.15.3:0/2208267159 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0fcc06b080 0x7f0fcc127300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.466+0000 7f0fd2a6d700 1 -- 172.21.15.3:0/2208267159 >> 172.21.15.3:0/2208267159 conn(0x7f0fcc066f50 msgr2=0x7f0fcc070d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:35.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.466+0000 7f0fd2a6d700 1 -- 172.21.15.3:0/2208267159 shutdown_connections 2023-10-03T10:19:35.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.466+0000 7f0fd2a6d700 1 -- 172.21.15.3:0/2208267159 wait complete. 2023-10-03T10:19:35.485 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:35.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.497+0000 7fd471cce700 1 -- 172.21.15.3:0/798272561 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd46c074ea0 con 0x7fd46c09e410 2023-10-03T10:19:35.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.497+0000 7fd453fff700 1 -- 172.21.15.3:0/798272561 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd464007330 con 0x7fd46c09e410 2023-10-03T10:19:35.497 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:35.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.500+0000 7fd471cce700 1 -- 172.21.15.3:0/798272561 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd46c09e410 msgr2=0x7fd46c1c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:35.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.500+0000 7fd471cce700 1 --2- 172.21.15.3:0/798272561 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd46c09e410 0x7fd46c1c8040 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fd464009dd0 tx=0x7fd46400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.500+0000 7fd471cce700 1 -- 172.21.15.3:0/798272561 shutdown_connections 2023-10-03T10:19:35.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.500+0000 7fd471cce700 1 --2- 172.21.15.3:0/798272561 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd46c09e410 0x7fd46c1c8040 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.500+0000 7fd471cce700 1 --2- 172.21.15.3:0/798272561 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd46c09d6f0 0x7fd46c1a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.500+0000 7fd471cce700 1 --2- 172.21.15.3:0/798272561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c09e000 0x7fd46c1a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.501+0000 7fd471cce700 1 -- 172.21.15.3:0/798272561 >> 172.21.15.3:0/798272561 conn(0x7fd46c066f50 msgr2=0x7fd46c1bbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:35.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.501+0000 7fd471cce700 1 -- 172.21.15.3:0/798272561 shutdown_connections 2023-10-03T10:19:35.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.501+0000 7fd471cce700 1 -- 172.21.15.3:0/798272561 wait complete. 2023-10-03T10:19:35.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.501+0000 7fb00e210700 1 -- 172.21.15.3:0/4086904978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb00811d7e0 con 0x7fb008134ec0 2023-10-03T10:19:35.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.501+0000 7fafeffff700 1 -- 172.21.15.3:0/4086904978 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faffc01be10 con 0x7fb008134ec0 2023-10-03T10:19:35.503 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:35.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.505+0000 7fb00e210700 1 -- 172.21.15.3:0/4086904978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb008134ec0 msgr2=0x7fb0081cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:35.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.505+0000 7fb00e210700 1 --2- 172.21.15.3:0/4086904978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb008134ec0 0x7fb0081cd290 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7faffc00e9b0 tx=0x7faffc00ecc0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.505+0000 7fb00e210700 1 -- 172.21.15.3:0/4086904978 shutdown_connections 2023-10-03T10:19:35.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.505+0000 7fb00e210700 1 --2- 172.21.15.3:0/4086904978 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb008137f20 0x7fb00811a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.505+0000 7fb00e210700 1 --2- 172.21.15.3:0/4086904978 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb00806c0e0 0x7fb0081ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.505+0000 7fb00e210700 1 --2- 172.21.15.3:0/4086904978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb008134ec0 0x7fb0081cd290 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.505+0000 7fb00e210700 1 -- 172.21.15.3:0/4086904978 >> 172.21.15.3:0/4086904978 conn(0x7fb0080a4210 msgr2=0x7fb0080a27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:35.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.505+0000 7fb00e210700 1 -- 172.21.15.3:0/4086904978 shutdown_connections 2023-10-03T10:19:35.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.505+0000 7fb00e210700 1 -- 172.21.15.3:0/4086904978 wait complete. 2023-10-03T10:19:35.517 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:35.521 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:35.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.544+0000 7f87caf72700 1 -- 172.21.15.3:0/1885780054 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f87c4074ea0 con 0x7f87c41338a0 2023-10-03T10:19:35.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.544+0000 7f87c17fa700 1 -- 172.21.15.3:0/1885780054 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f87bc029020 con 0x7f87c41338a0 2023-10-03T10:19:35.545 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:35.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.547+0000 7f87caf72700 1 -- 172.21.15.3:0/1885780054 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f87c41338a0 msgr2=0x7f87c411a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:35.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.547+0000 7f87caf72700 1 --2- 172.21.15.3:0/1885780054 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f87c41338a0 0x7f87c411a190 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f87bc00cb10 tx=0x7f87bc00ce20 comp rx=0 tx=0).stop 2023-10-03T10:19:35.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.547+0000 7f87caf72700 1 -- 172.21.15.3:0/1885780054 shutdown_connections 2023-10-03T10:19:35.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.547+0000 7f87caf72700 1 --2- 172.21.15.3:0/1885780054 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f87c41338a0 0x7f87c411a190 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.547+0000 7f87caf72700 1 --2- 172.21.15.3:0/1885780054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87c409de50 0x7f87c4119710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.547+0000 7f87caf72700 1 --2- 172.21.15.3:0/1885780054 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f87c409e760 0x7f87c4119c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:35.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.547+0000 7f87caf72700 1 -- 172.21.15.3:0/1885780054 >> 172.21.15.3:0/1885780054 conn(0x7f87c40a2aa0 msgr2=0x7f87c409ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:35.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.547+0000 7f87caf72700 1 -- 172.21.15.3:0/1885780054 shutdown_connections 2023-10-03T10:19:35.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:35.547+0000 7f87caf72700 1 -- 172.21.15.3:0/1885780054 wait complete. 2023-10-03T10:19:35.561 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:36.486 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:36.518 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:36.530 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:36.561 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:36.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.641+0000 7fddd4af9700 1 Processor -- start 2023-10-03T10:19:36.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.641+0000 7fddd4af9700 1 -- start start 2023-10-03T10:19:36.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.642+0000 7fddd4af9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fddd009ed20 0x7fddd0136640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:36.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.642+0000 7fddd4af9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fddd0136b80 0x7fddd0138f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:36.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.642+0000 7fddd4af9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddd009b420 con 0x7fddd01316c0 2023-10-03T10:19:36.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.643+0000 7fddd4af9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddd01314a0 con 0x7fddd009ed20 2023-10-03T10:19:36.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.643+0000 7fddd4af9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fddd0099ce0 con 0x7fddd0136b80 2023-10-03T10:19:36.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.643+0000 7fddcdd9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fddd009ed20 0x7fddd0136640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:36.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.643+0000 7fddcdd9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fddd009ed20 0x7fddd0136640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37664/0 (socket says 172.21.15.3:37664) 2023-10-03T10:19:36.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.643+0000 7fddcdd9b700 1 -- 172.21.15.3:0/1035273638 learned_addr learned my addr 172.21.15.3:0/1035273638 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:36.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.643+0000 7fddced9d700 1 --2- 172.21.15.3:0/1035273638 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fddd0136b80 0x7fddd0138f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:36.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.643+0000 7fddcdd9b700 1 -- 172.21.15.3:0/1035273638 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fddd0136b80 msgr2=0x7fddd0138f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:36.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.643+0000 7fddcdd9b700 1 --2- 172.21.15.3:0/1035273638 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fddd0136b80 0x7fddd0138f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.643+0000 7fddcdd9b700 1 -- 172.21.15.3:0/1035273638 >> v1:172.21.15.3:6789/0 conn(0x7fddd01316c0 legacy=0x7fddd0131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:36.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.643+0000 7fddcdd9b700 1 -- 172.21.15.3:0/1035273638 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd009a510 con 0x7fddd009ed20 2023-10-03T10:19:36.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.643+0000 7fddcdd9b700 1 --2- 172.21.15.3:0/1035273638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddd009ed20 0x7fddd0136640 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fddc40098f0 tx=0x7fddc4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1de083a8899573 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:36.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.644+0000 7fddcd59a700 1 -- 172.21.15.3:0/1035273638 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc4011830 con 0x7fddd009ed20 2023-10-03T10:19:36.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.644+0000 7fddcd59a700 1 -- 172.21.15.3:0/1035273638 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc4011990 con 0x7fddd009ed20 2023-10-03T10:19:36.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.644+0000 7fddcd59a700 1 -- 172.21.15.3:0/1035273638 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc4011d20 con 0x7fddd009ed20 2023-10-03T10:19:36.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.644+0000 7fddd4af9700 1 -- 172.21.15.3:0/1035273638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddd009ed20 msgr2=0x7fddd0136640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:36.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.644+0000 7fddd4af9700 1 --2- 172.21.15.3:0/1035273638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddd009ed20 0x7fddd0136640 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fddc40098f0 tx=0x7fddc4009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:36.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.645+0000 7fddd4af9700 1 -- 172.21.15.3:0/1035273638 shutdown_connections 2023-10-03T10:19:36.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.645+0000 7fddd4af9700 1 --2- 172.21.15.3:0/1035273638 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fddd0136b80 0x7fddd0138f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.645+0000 7fddd4af9700 1 --2- 172.21.15.3:0/1035273638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddd009ed20 0x7fddd0136640 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.645+0000 7fddd4af9700 1 -- 172.21.15.3:0/1035273638 >> 172.21.15.3:0/1035273638 conn(0x7fddd00a2aa0 msgr2=0x7fddd00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:36.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.645+0000 7fddd4af9700 1 -- 172.21.15.3:0/1035273638 shutdown_connections 2023-10-03T10:19:36.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.645+0000 7fddd4af9700 1 -- 172.21.15.3:0/1035273638 wait complete. 2023-10-03T10:19:36.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.646+0000 7fddd4af9700 1 Processor -- start 2023-10-03T10:19:36.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.646+0000 7fddd4af9700 1 -- start start 2023-10-03T10:19:36.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.647+0000 7fddd4af9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fddd009ed20 0x7fddd0134180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:36.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.647+0000 7fddd4af9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddd01316c0 0x7fddd01346c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:36.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.647+0000 7fddd4af9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fddd0136b80 0x7fddd0134c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:36.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.647+0000 7fddd4af9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd01355e0 con 0x7fddd01316c0 2023-10-03T10:19:36.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.647+0000 7fddd4af9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd0138f70 con 0x7fddd0136b80 2023-10-03T10:19:36.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.647+0000 7fddd4af9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fddd0099f20 con 0x7fddd009ed20 2023-10-03T10:19:36.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.647+0000 7fddced9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fddd0136b80 0x7fddd0134c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:36.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.647+0000 7fddced9d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fddd0136b80 0x7fddd0134c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41022/0 (socket says 172.21.15.3:41022) 2023-10-03T10:19:36.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.647+0000 7fddced9d700 1 -- 172.21.15.3:0/1099032610 learned_addr learned my addr 172.21.15.3:0/1099032610 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:36.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.647+0000 7fddce59c700 1 --2- 172.21.15.3:0/1099032610 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fddd009ed20 0x7fddd0134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:36.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.648+0000 7fddcdd9b700 1 --2- 172.21.15.3:0/1099032610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddd01316c0 0x7fddd01346c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:36.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.648+0000 7fddced9d700 1 -- 172.21.15.3:0/1099032610 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fddd009ed20 msgr2=0x7fddd0134180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:36.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.648+0000 7fddced9d700 1 --2- 172.21.15.3:0/1099032610 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fddd009ed20 0x7fddd0134180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.648+0000 7fddced9d700 1 -- 172.21.15.3:0/1099032610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddd01316c0 msgr2=0x7fddd01346c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:36.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.648+0000 7fddced9d700 1 --2- 172.21.15.3:0/1099032610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddd01316c0 0x7fddd01346c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.648+0000 7fddced9d700 1 -- 172.21.15.3:0/1099032610 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc4009550 con 0x7fddd0136b80 2023-10-03T10:19:36.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.648+0000 7fddced9d700 1 --2- 172.21.15.3:0/1099032610 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fddd0136b80 0x7fddd0134c00 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fddc800efc0 tx=0x7fddc800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:36.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.648+0000 7fddbeffd700 1 -- 172.21.15.3:0/1099032610 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc800cc90 con 0x7fddd0136b80 2023-10-03T10:19:36.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.648+0000 7fddbeffd700 1 -- 172.21.15.3:0/1099032610 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc800f040 con 0x7fddd0136b80 2023-10-03T10:19:36.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.648+0000 7fddd4af9700 1 -- 172.21.15.3:0/1099032610 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd0119770 con 0x7fddd0136b80 2023-10-03T10:19:36.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.648+0000 7fddd4af9700 1 -- 172.21.15.3:0/1099032610 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd011a150 con 0x7fddd0136b80 2023-10-03T10:19:36.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.648+0000 7fddbeffd700 1 -- 172.21.15.3:0/1099032610 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc8014480 con 0x7fddd0136b80 2023-10-03T10:19:36.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.650+0000 7fddd4af9700 1 -- 172.21.15.3:0/1099032610 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb40052e0 con 0x7fddd0136b80 2023-10-03T10:19:36.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.650+0000 7fddbeffd700 1 -- 172.21.15.3:0/1099032610 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fddc8014600 con 0x7fddd0136b80 2023-10-03T10:19:36.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.650+0000 7fddbeffd700 1 -- 172.21.15.3:0/1099032610 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fddc8040750 con 0x7fddd0136b80 2023-10-03T10:19:36.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.659+0000 7fddbeffd700 1 -- 172.21.15.3:0/1099032610 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fddc8012930 con 0x7fddd0136b80 2023-10-03T10:19:36.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.684+0000 7f572810e700 1 Processor -- start 2023-10-03T10:19:36.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.685+0000 7f572810e700 1 -- start start 2023-10-03T10:19:36.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.685+0000 7f572810e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5720136830 0x7f5720138c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:36.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.685+0000 7f572810e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5720139150 0x7f572013b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:36.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.685+0000 7f572810e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f572006f720 con 0x7f57201338c0 2023-10-03T10:19:36.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.685+0000 7f572810e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5720070c80 con 0x7f5720136830 2023-10-03T10:19:36.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.685+0000 7f572810e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f572006fce0 con 0x7f5720139150 2023-10-03T10:19:36.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.686+0000 7f57266ab700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5720139150 0x7f572013b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:36.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.686+0000 7f57256a9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5720136830 0x7f5720138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:36.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.686+0000 7f57266ab700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5720139150 0x7f572013b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48218/0 (socket says 172.21.15.3:48218) 2023-10-03T10:19:36.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.686+0000 7f57266ab700 1 -- 172.21.15.3:0/1885140073 learned_addr learned my addr 172.21.15.3:0/1885140073 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:36.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.686+0000 7f57256a9700 1 -- 172.21.15.3:0/1885140073 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5720139150 msgr2=0x7f572013b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:36.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.686+0000 7f57256a9700 1 --2- 172.21.15.3:0/1885140073 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5720139150 0x7f572013b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.686+0000 7f57256a9700 1 -- 172.21.15.3:0/1885140073 >> v1:172.21.15.3:6789/0 conn(0x7f57201338c0 legacy=0x7f5720133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:36.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.686+0000 7f57256a9700 1 -- 172.21.15.3:0/1885140073 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57201336a0 con 0x7f5720136830 2023-10-03T10:19:36.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.686+0000 7f57266ab700 1 --2- 172.21.15.3:0/1885140073 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5720139150 0x7f572013b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:36.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.686+0000 7f57256a9700 1 --2- 172.21.15.3:0/1885140073 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5720136830 0x7f5720138c10 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f5710009870 tx=0x7f5710009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b0a38dcec0fe10c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:36.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.686+0000 7f5724ea8700 1 -- 172.21.15.3:0/1885140073 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5710004230 con 0x7f5720136830 2023-10-03T10:19:36.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.686+0000 7f5724ea8700 1 -- 172.21.15.3:0/1885140073 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5710004390 con 0x7f5720136830 2023-10-03T10:19:36.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.686+0000 7f5724ea8700 1 -- 172.21.15.3:0/1885140073 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57100036a0 con 0x7f5720136830 2023-10-03T10:19:36.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.687+0000 7f572810e700 1 -- 172.21.15.3:0/1885140073 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5720136830 msgr2=0x7f5720138c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:36.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.687+0000 7f572810e700 1 --2- 172.21.15.3:0/1885140073 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5720136830 0x7f5720138c10 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f5710009870 tx=0x7f5710009b80 comp rx=0 tx=0).stop 2023-10-03T10:19:36.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.687+0000 7f572810e700 1 -- 172.21.15.3:0/1885140073 shutdown_connections 2023-10-03T10:19:36.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.687+0000 7f572810e700 1 --2- 172.21.15.3:0/1885140073 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5720139150 0x7f572013b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.687+0000 7f572810e700 1 --2- 172.21.15.3:0/1885140073 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5720136830 0x7f5720138c10 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.687+0000 7f572810e700 1 -- 172.21.15.3:0/1885140073 >> 172.21.15.3:0/1885140073 conn(0x7f57200a2aa0 msgr2=0x7f57200a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:36.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.687+0000 7f572810e700 1 -- 172.21.15.3:0/1885140073 shutdown_connections 2023-10-03T10:19:36.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.687+0000 7f572810e700 1 -- 172.21.15.3:0/1885140073 wait complete. 2023-10-03T10:19:36.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.689+0000 7f572810e700 1 Processor -- start 2023-10-03T10:19:36.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.689+0000 7f572810e700 1 -- start start 2023-10-03T10:19:36.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.689+0000 7f1af73c4700 1 Processor -- start 2023-10-03T10:19:36.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.690+0000 7f1af73c4700 1 -- start start 2023-10-03T10:19:36.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.690+0000 7f572810e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57201338c0 0x7f572009b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:36.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.690+0000 7f572810e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5720136830 0x7f572009bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:36.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.690+0000 7f572810e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5720139150 0x7f57201cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:36.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.690+0000 7f572810e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572009cec0 con 0x7f57201338c0 2023-10-03T10:19:36.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.690+0000 7f572810e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f572009e3e0 con 0x7f5720136830 2023-10-03T10:19:36.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.690+0000 7f572810e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f572012f3c0 con 0x7f5720139150 2023-10-03T10:19:36.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.690+0000 7f57266ab700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5720139150 0x7f57201cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:36.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.690+0000 7f57266ab700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5720139150 0x7f57201cc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48234/0 (socket says 172.21.15.3:48234) 2023-10-03T10:19:36.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.690+0000 7f57266ab700 1 -- 172.21.15.3:0/2045899527 learned_addr learned my addr 172.21.15.3:0/2045899527 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:36.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.690+0000 7f5725eaa700 1 --2- 172.21.15.3:0/2045899527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57201338c0 0x7f572009b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:36.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.690+0000 7f57266ab700 1 -- 172.21.15.3:0/2045899527 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5720136830 msgr2=0x7f572009bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:36.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.691+0000 7f57266ab700 1 --2- 172.21.15.3:0/2045899527 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5720136830 0x7f572009bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.691+0000 7f57266ab700 1 -- 172.21.15.3:0/2045899527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57201338c0 msgr2=0x7f572009b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:36.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.691+0000 7f57266ab700 1 --2- 172.21.15.3:0/2045899527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57201338c0 0x7f572009b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.690+0000 7f1af73c4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af0136710 0x7f1af0138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:36.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.690+0000 7f1af73c4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1af0139030 0x7f1af013b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:36.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.690+0000 7f1af73c4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af006ef30 con 0x7f1af01338a0 2023-10-03T10:19:36.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.690+0000 7f1af73c4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1af006fef0 con 0x7f1af0139030 2023-10-03T10:19:36.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.690+0000 7f1af73c4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af01334b0 con 0x7f1af0136710 2023-10-03T10:19:36.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.691+0000 7f57266ab700 1 -- 172.21.15.3:0/2045899527 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5718009b30 con 0x7f5720139150 2023-10-03T10:19:36.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.691+0000 7f5725eaa700 1 --2- 172.21.15.3:0/2045899527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57201338c0 0x7f572009b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:36.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.691+0000 7f1af495f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af0136710 0x7f1af0138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:36.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.691+0000 7f1af495f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af0136710 0x7f1af0138af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41058/0 (socket says 172.21.15.3:41058) 2023-10-03T10:19:36.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.691+0000 7f1af495f700 1 -- 172.21.15.3:0/1552123423 learned_addr learned my addr 172.21.15.3:0/1552123423 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:36.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.691+0000 7f1af5961700 1 --2- 172.21.15.3:0/1552123423 >> v2:172.21.15.3:3300/0 conn(0x7f1af0139030 0x7f1af013b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:36.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.691+0000 7f57266ab700 1 --2- 172.21.15.3:0/2045899527 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5720139150 0x7f57201cc770 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f571800ef90 tx=0x7f571800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:36.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.691+0000 7f1af495f700 1 -- 172.21.15.3:0/1552123423 >> v2:172.21.15.3:3300/0 conn(0x7f1af0139030 msgr2=0x7f1af013b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:36.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.691+0000 7f1af495f700 1 --2- 172.21.15.3:0/1552123423 >> v2:172.21.15.3:3300/0 conn(0x7f1af0139030 0x7f1af013b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.691+0000 7f1af495f700 1 -- 172.21.15.3:0/1552123423 >> v1:172.21.15.3:6789/0 conn(0x7f1af01338a0 legacy=0x7f1af0133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:36.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.691+0000 7f1af495f700 1 -- 172.21.15.3:0/1552123423 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af006fc30 con 0x7f1af0136710 2023-10-03T10:19:36.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.691+0000 7f1ae7fff700 1 -- 172.21.15.3:0/1552123423 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1041814261 0 0) 0x7f1af006ef30 con 0x7f1af01338a0 2023-10-03T10:19:36.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.691+0000 7f1af5961700 1 --2- 172.21.15.3:0/1552123423 >> v2:172.21.15.3:3300/0 conn(0x7f1af0139030 0x7f1af013b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:36.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.692+0000 7f1af495f700 1 --2- 172.21.15.3:0/1552123423 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af0136710 0x7f1af0138af0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f1ae00098f0 tx=0x7f1ae0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a0aea1f9cf876ce server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:36.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.692+0000 7f57167fc700 1 -- 172.21.15.3:0/2045899527 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5718003c20 con 0x7f5720139150 2023-10-03T10:19:36.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.692+0000 7f1ae7fff700 1 -- 172.21.15.3:0/1552123423 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0011830 con 0x7f1af0136710 2023-10-03T10:19:36.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.692+0000 7f57167fc700 1 -- 172.21.15.3:0/2045899527 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f571800f040 con 0x7f5720139150 2023-10-03T10:19:36.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.692+0000 7f57167fc700 1 -- 172.21.15.3:0/2045899527 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571800c940 con 0x7f5720139150 2023-10-03T10:19:36.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.692+0000 7f572810e700 1 -- 172.21.15.3:0/2045899527 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57100094a0 con 0x7f5720139150 2023-10-03T10:19:36.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.692+0000 7f572810e700 1 -- 172.21.15.3:0/2045899527 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f572009d750 con 0x7f5720139150 2023-10-03T10:19:36.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.692+0000 7f1ae7fff700 1 -- 172.21.15.3:0/1552123423 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae0011990 con 0x7f1af0136710 2023-10-03T10:19:36.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.692+0000 7f1ae7fff700 1 -- 172.21.15.3:0/1552123423 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0011d20 con 0x7f1af0136710 2023-10-03T10:19:36.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.693+0000 7f57167fc700 1 -- 172.21.15.3:0/2045899527 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f571800caa0 con 0x7f5720139150 2023-10-03T10:19:36.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.693+0000 7f57167fc700 1 -- 172.21.15.3:0/2045899527 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f571803fb50 con 0x7f5720139150 2023-10-03T10:19:36.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.693+0000 7f1af73c4700 1 -- 172.21.15.3:0/1552123423 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af0136710 msgr2=0x7f1af0138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:36.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.693+0000 7f1af73c4700 1 --2- 172.21.15.3:0/1552123423 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af0136710 0x7f1af0138af0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f1ae00098f0 tx=0x7f1ae0009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:36.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.693+0000 7f572810e700 1 -- 172.21.15.3:0/2045899527 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5720134320 con 0x7f5720139150 2023-10-03T10:19:36.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.693+0000 7f1af73c4700 1 -- 172.21.15.3:0/1552123423 shutdown_connections 2023-10-03T10:19:36.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.693+0000 7f1af73c4700 1 --2- 172.21.15.3:0/1552123423 >> v2:172.21.15.3:3300/0 conn(0x7f1af0139030 0x7f1af013b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.693+0000 7f1af73c4700 1 --2- 172.21.15.3:0/1552123423 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af0136710 0x7f1af0138af0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.693+0000 7f1af73c4700 1 -- 172.21.15.3:0/1552123423 >> 172.21.15.3:0/1552123423 conn(0x7f1af00a2aa0 msgr2=0x7f1af00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:36.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.694+0000 7f1af73c4700 1 -- 172.21.15.3:0/1552123423 shutdown_connections 2023-10-03T10:19:36.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.694+0000 7f1af73c4700 1 -- 172.21.15.3:0/1552123423 wait complete. 2023-10-03T10:19:36.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.695+0000 7f1af73c4700 1 Processor -- start 2023-10-03T10:19:36.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.695+0000 7f1af73c4700 1 -- start start 2023-10-03T10:19:36.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.696+0000 7f1af73c4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af01338a0 0x7f1af0118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:36.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.696+0000 7f1af73c4700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1af0136710 0x7f1af0119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:36.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.696+0000 7f1af73c4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af0139030 0x7f1af0119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:36.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.696+0000 7f1af73c4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af011a0a0 con 0x7f1af01338a0 2023-10-03T10:19:36.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.696+0000 7f1af73c4700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af011c080 con 0x7f1af0139030 2023-10-03T10:19:36.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.696+0000 7f1af73c4700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1af011a3a0 con 0x7f1af0136710 2023-10-03T10:19:36.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.696+0000 7f1af495f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1af0136710 0x7f1af0119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:36.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.696+0000 7f1af495f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1af0136710 0x7f1af0119210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48240/0 (socket says 172.21.15.3:48240) 2023-10-03T10:19:36.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.696+0000 7f1af495f700 1 -- 172.21.15.3:0/1168012791 learned_addr learned my addr 172.21.15.3:0/1168012791 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:36.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.697+0000 7f1af495f700 1 -- 172.21.15.3:0/1168012791 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af0139030 msgr2=0x7f1af0119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:36.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.697+0000 7f1af495f700 1 --2- 172.21.15.3:0/1168012791 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af0139030 0x7f1af0119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.697+0000 7f1af495f700 1 -- 172.21.15.3:0/1168012791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af01338a0 msgr2=0x7f1af0118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:36.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.697+0000 7f1af495f700 1 --2- 172.21.15.3:0/1168012791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af01338a0 0x7f1af0118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.697+0000 7f1af495f700 1 -- 172.21.15.3:0/1168012791 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae8009a70 con 0x7f1af0136710 2023-10-03T10:19:36.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.697+0000 7f1af495f700 1 --2- 172.21.15.3:0/1168012791 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1af0136710 0x7f1af0119210 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1af01335f0 tx=0x7f1ae0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:36.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.698+0000 7f1ae5ffb700 1 -- 172.21.15.3:0/1168012791 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0004510 con 0x7f1af0136710 2023-10-03T10:19:36.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.698+0000 7f1ae5ffb700 1 -- 172.21.15.3:0/1168012791 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae00036a0 con 0x7f1af0136710 2023-10-03T10:19:36.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.698+0000 7f1ae5ffb700 1 -- 172.21.15.3:0/1168012791 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0003a50 con 0x7f1af0136710 2023-10-03T10:19:36.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.699+0000 7f57167fc700 1 -- 172.21.15.3:0/2045899527 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5718014440 con 0x7f5720139150 2023-10-03T10:19:36.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.699+0000 7f1af73c4700 1 -- 172.21.15.3:0/1168012791 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae0009550 con 0x7f1af0136710 2023-10-03T10:19:36.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.700+0000 7f1af73c4700 1 -- 172.21.15.3:0/1168012791 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af006f640 con 0x7f1af0136710 2023-10-03T10:19:36.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.701+0000 7f1af73c4700 1 -- 172.21.15.3:0/1168012791 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af011b160 con 0x7f1af0136710 2023-10-03T10:19:36.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.701+0000 7f1ae5ffb700 1 -- 172.21.15.3:0/1168012791 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1ae0003bb0 con 0x7f1af0136710 2023-10-03T10:19:36.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.701+0000 7f1ae5ffb700 1 -- 172.21.15.3:0/1168012791 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1ae0040a10 con 0x7f1af0136710 2023-10-03T10:19:36.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.706+0000 7f1ae5ffb700 1 -- 172.21.15.3:0/1168012791 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1ae001a420 con 0x7f1af0136710 2023-10-03T10:19:36.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.726+0000 7f0e417a9700 1 Processor -- start 2023-10-03T10:19:36.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.726+0000 7f0e417a9700 1 -- start start 2023-10-03T10:19:36.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.726+0000 7f0e417a9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0e3c09e1d0 0x7f0e3c135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:36.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.726+0000 7f0e417a9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e3c135e50 0x7f0e3c138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:36.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.726+0000 7f0e417a9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e3c09f130 con 0x7f0e3c09d6f0 2023-10-03T10:19:36.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.726+0000 7f0e417a9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e3c09f520 con 0x7f0e3c09e1d0 2023-10-03T10:19:36.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.726+0000 7f0e417a9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c09e930 con 0x7f0e3c135e50 2023-10-03T10:19:36.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.727+0000 7f0e3a7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0e3c09e1d0 0x7f0e3c135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:36.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.727+0000 7f0e3b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e3c135e50 0x7f0e3c138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:36.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.727+0000 7f0e3a7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0e3c09e1d0 0x7f0e3c135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37706/0 (socket says 172.21.15.3:37706) 2023-10-03T10:19:36.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.727+0000 7f0e3b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e3c135e50 0x7f0e3c138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48244/0 (socket says 172.21.15.3:48244) 2023-10-03T10:19:36.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.727+0000 7f0e3a7fc700 1 -- 172.21.15.3:0/1897888878 learned_addr learned my addr 172.21.15.3:0/1897888878 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:36.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.727+0000 7f0e3b7fe700 1 -- 172.21.15.3:0/1897888878 >> v2:172.21.15.3:3300/0 conn(0x7f0e3c09e1d0 msgr2=0x7f0e3c135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:36.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.727+0000 7f0e3b7fe700 1 --2- 172.21.15.3:0/1897888878 >> v2:172.21.15.3:3300/0 conn(0x7f0e3c09e1d0 0x7f0e3c135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.727+0000 7f0e3b7fe700 1 -- 172.21.15.3:0/1897888878 >> v1:172.21.15.3:6789/0 conn(0x7f0e3c09d6f0 legacy=0x7f0e3c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:36.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.727+0000 7f0e3b7fe700 1 -- 172.21.15.3:0/1897888878 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3c09b2d0 con 0x7f0e3c135e50 2023-10-03T10:19:36.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.727+0000 7f0e3a7fc700 1 --2- 172.21.15.3:0/1897888878 >> v2:172.21.15.3:3300/0 conn(0x7f0e3c09e1d0 0x7f0e3c135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:36.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.727+0000 7f0e3b7fe700 1 --2- 172.21.15.3:0/1897888878 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e3c135e50 0x7f0e3c138230 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009d90 tx=0x7f0e3400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ee31e00c602e947 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:36.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.728+0000 7f0e39ffb700 1 -- 172.21.15.3:0/1897888878 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3400e070 con 0x7f0e3c135e50 2023-10-03T10:19:36.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.728+0000 7f0e39ffb700 1 -- 172.21.15.3:0/1897888878 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3400bae0 con 0x7f0e3c135e50 2023-10-03T10:19:36.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.728+0000 7f0e39ffb700 1 -- 172.21.15.3:0/1897888878 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3400be90 con 0x7f0e3c135e50 2023-10-03T10:19:36.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.728+0000 7f0e417a9700 1 -- 172.21.15.3:0/1897888878 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e3c135e50 msgr2=0x7f0e3c138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:36.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.728+0000 7f0e417a9700 1 --2- 172.21.15.3:0/1897888878 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e3c135e50 0x7f0e3c138230 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009d90 tx=0x7f0e3400b330 comp rx=0 tx=0).stop 2023-10-03T10:19:36.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.728+0000 7f0e417a9700 1 -- 172.21.15.3:0/1897888878 shutdown_connections 2023-10-03T10:19:36.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.728+0000 7f0e417a9700 1 --2- 172.21.15.3:0/1897888878 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e3c135e50 0x7f0e3c138230 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.728+0000 7f0e417a9700 1 --2- 172.21.15.3:0/1897888878 >> v2:172.21.15.3:3300/0 conn(0x7f0e3c09e1d0 0x7f0e3c135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.728+0000 7f0e417a9700 1 -- 172.21.15.3:0/1897888878 >> 172.21.15.3:0/1897888878 conn(0x7f0e3c066f50 msgr2=0x7f0e3c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:36.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.728+0000 7f0e417a9700 1 -- 172.21.15.3:0/1897888878 shutdown_connections 2023-10-03T10:19:36.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.728+0000 7f0e417a9700 1 -- 172.21.15.3:0/1897888878 wait complete. 2023-10-03T10:19:36.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.730+0000 7f0e417a9700 1 Processor -- start 2023-10-03T10:19:36.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.730+0000 7f0e417a9700 1 -- start start 2023-10-03T10:19:36.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.730+0000 7f0e417a9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e3c09d6f0 0x7f0e3c1397b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:36.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.730+0000 7f0e417a9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e3c09e1d0 0x7f0e3c139cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:36.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.730+0000 7f0e417a9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0e3c135e50 0x7f0e3c13a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:36.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.730+0000 7f0e417a9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c11fc70 con 0x7f0e3c09d6f0 2023-10-03T10:19:36.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.730+0000 7f0e417a9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c1212b0 con 0x7f0e3c135e50 2023-10-03T10:19:36.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.730+0000 7f0e417a9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c120680 con 0x7f0e3c09e1d0 2023-10-03T10:19:36.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.730+0000 7f0e3affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e3c09d6f0 0x7f0e3c1397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:36.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.730+0000 7f0e3b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0e3c135e50 0x7f0e3c13a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:36.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.731+0000 7f0e3affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e3c09d6f0 0x7f0e3c1397b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37720/0 (socket says 172.21.15.3:37720) 2023-10-03T10:19:36.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.731+0000 7f0e3b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0e3c135e50 0x7f0e3c13a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41088/0 (socket says 172.21.15.3:41088) 2023-10-03T10:19:36.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.731+0000 7f0e3affd700 1 -- 172.21.15.3:0/3378175600 learned_addr learned my addr 172.21.15.3:0/3378175600 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:36.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.731+0000 7f0e3a7fc700 1 --2- 172.21.15.3:0/3378175600 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e3c09e1d0 0x7f0e3c139cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:36.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.731+0000 7f0e3b7fe700 1 -- 172.21.15.3:0/3378175600 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e3c09e1d0 msgr2=0x7f0e3c139cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:36.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.731+0000 7f0e3b7fe700 1 --2- 172.21.15.3:0/3378175600 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e3c09e1d0 0x7f0e3c139cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.731+0000 7f0e3b7fe700 1 -- 172.21.15.3:0/3378175600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e3c09d6f0 msgr2=0x7f0e3c1397b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:36.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.731+0000 7f0e3b7fe700 1 --2- 172.21.15.3:0/3378175600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e3c09d6f0 0x7f0e3c1397b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.731+0000 7f0e3b7fe700 1 -- 172.21.15.3:0/3378175600 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e30009550 con 0x7f0e3c135e50 2023-10-03T10:19:36.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.731+0000 7f0e3b7fe700 1 --2- 172.21.15.3:0/3378175600 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0e3c135e50 0x7f0e3c13a230 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f0e34015000 tx=0x7f0e34004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.731+0000 7f0e237fe700 1 -- 172.21.15.3:0/3378175600 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e34004540 con 0x7f0e3c135e50 2023-10-03T10:19:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.732+0000 7f0e237fe700 1 -- 172.21.15.3:0/3378175600 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e340046c0 con 0x7f0e3c135e50 2023-10-03T10:19:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.732+0000 7f0e237fe700 1 -- 172.21.15.3:0/3378175600 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e34012430 con 0x7f0e3c135e50 2023-10-03T10:19:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.732+0000 7f0e417a9700 1 -- 172.21.15.3:0/3378175600 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e34009a70 con 0x7f0e3c135e50 2023-10-03T10:19:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.732+0000 7f0e417a9700 1 -- 172.21.15.3:0/3378175600 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3c135bd0 con 0x7f0e3c135e50 2023-10-03T10:19:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.733+0000 7f0e237fe700 1 -- 172.21.15.3:0/3378175600 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0e34012590 con 0x7f0e3c135e50 2023-10-03T10:19:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.733+0000 7f0e417a9700 1 -- 172.21.15.3:0/3378175600 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e3c071700 con 0x7f0e3c135e50 2023-10-03T10:19:36.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.733+0000 7f0e237fe700 1 -- 172.21.15.3:0/3378175600 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0e34041320 con 0x7f0e3c135e50 2023-10-03T10:19:36.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.742+0000 7f0e237fe700 1 -- 172.21.15.3:0/3378175600 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0e3402d940 con 0x7f0e3c135e50 2023-10-03T10:19:36.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.885+0000 7fddd4af9700 1 -- 172.21.15.3:0/1099032610 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fddb4005150 con 0x7fddd0136b80 2023-10-03T10:19:36.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.886+0000 7fddbeffd700 1 -- 172.21.15.3:0/1099032610 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fddc8012340 con 0x7fddd0136b80 2023-10-03T10:19:36.886 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:36.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.888+0000 7fddd4af9700 1 -- 172.21.15.3:0/1099032610 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fddd0136b80 msgr2=0x7fddd0134c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.888+0000 7fddd4af9700 1 --2- 172.21.15.3:0/1099032610 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fddd0136b80 0x7fddd0134c00 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fddc800efc0 tx=0x7fddc800c3a0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.888+0000 7fddd4af9700 1 -- 172.21.15.3:0/1099032610 shutdown_connections 2023-10-03T10:19:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.888+0000 7fddd4af9700 1 --2- 172.21.15.3:0/1099032610 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fddd0136b80 0x7fddd0134c00 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.888+0000 7fddd4af9700 1 --2- 172.21.15.3:0/1099032610 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fddd009ed20 0x7fddd0134180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.888+0000 7fddd4af9700 1 --2- 172.21.15.3:0/1099032610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddd01316c0 0x7fddd01346c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.888+0000 7fddd4af9700 1 -- 172.21.15.3:0/1099032610 >> 172.21.15.3:0/1099032610 conn(0x7fddd00a2aa0 msgr2=0x7fddd00a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.888+0000 7fddd4af9700 1 -- 172.21.15.3:0/1099032610 shutdown_connections 2023-10-03T10:19:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.888+0000 7fddd4af9700 1 -- 172.21.15.3:0/1099032610 wait complete. 2023-10-03T10:19:36.903 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:36.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.911+0000 7f572810e700 1 -- 172.21.15.3:0/2045899527 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5720074ea0 con 0x7f5720139150 2023-10-03T10:19:36.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.911+0000 7f57167fc700 1 -- 172.21.15.3:0/2045899527 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f571802c090 con 0x7f5720139150 2023-10-03T10:19:36.911 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:36.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.913+0000 7f572810e700 1 -- 172.21.15.3:0/2045899527 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5720139150 msgr2=0x7f57201cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:36.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.913+0000 7f572810e700 1 --2- 172.21.15.3:0/2045899527 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5720139150 0x7f57201cc770 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f571800ef90 tx=0x7f571800c3a0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.913+0000 7f572810e700 1 -- 172.21.15.3:0/2045899527 shutdown_connections 2023-10-03T10:19:36.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.913+0000 7f572810e700 1 --2- 172.21.15.3:0/2045899527 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5720139150 0x7f57201cc770 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.913+0000 7f572810e700 1 --2- 172.21.15.3:0/2045899527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57201338c0 0x7f572009b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.913+0000 7f572810e700 1 --2- 172.21.15.3:0/2045899527 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5720136830 0x7f572009bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.913+0000 7f572810e700 1 -- 172.21.15.3:0/2045899527 >> 172.21.15.3:0/2045899527 conn(0x7f57200a2aa0 msgr2=0x7f572009f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:36.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.914+0000 7f572810e700 1 -- 172.21.15.3:0/2045899527 shutdown_connections 2023-10-03T10:19:36.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.914+0000 7f572810e700 1 -- 172.21.15.3:0/2045899527 wait complete. 2023-10-03T10:19:36.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.917+0000 7f1af73c4700 1 -- 172.21.15.3:0/1168012791 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1af0074ea0 con 0x7f1af0136710 2023-10-03T10:19:36.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.917+0000 7f1ae5ffb700 1 -- 172.21.15.3:0/1168012791 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1ae0015020 con 0x7f1af0136710 2023-10-03T10:19:36.918 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:36.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.920+0000 7f1af73c4700 1 -- 172.21.15.3:0/1168012791 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1af0136710 msgr2=0x7f1af0119210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:36.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.920+0000 7f1af73c4700 1 --2- 172.21.15.3:0/1168012791 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1af0136710 0x7f1af0119210 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1af01335f0 tx=0x7f1ae0004750 comp rx=0 tx=0).stop 2023-10-03T10:19:36.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.920+0000 7f1af73c4700 1 -- 172.21.15.3:0/1168012791 shutdown_connections 2023-10-03T10:19:36.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.920+0000 7f1af73c4700 1 --2- 172.21.15.3:0/1168012791 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1af0139030 0x7f1af0119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.920+0000 7f1af73c4700 1 --2- 172.21.15.3:0/1168012791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af01338a0 0x7f1af0118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.920+0000 7f1af73c4700 1 --2- 172.21.15.3:0/1168012791 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1af0136710 0x7f1af0119210 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.920+0000 7f1af73c4700 1 -- 172.21.15.3:0/1168012791 >> 172.21.15.3:0/1168012791 conn(0x7f1af00a2aa0 msgr2=0x7f1af01344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:36.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.920+0000 7f1af73c4700 1 -- 172.21.15.3:0/1168012791 shutdown_connections 2023-10-03T10:19:36.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.920+0000 7f1af73c4700 1 -- 172.21.15.3:0/1168012791 wait complete. 2023-10-03T10:19:36.929 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:36.934 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:36.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.953+0000 7f0e417a9700 1 -- 172.21.15.3:0/3378175600 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0e3c074ea0 con 0x7f0e3c135e50 2023-10-03T10:19:36.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.953+0000 7f0e237fe700 1 -- 172.21.15.3:0/3378175600 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0e3402d350 con 0x7f0e3c135e50 2023-10-03T10:19:36.953 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:36.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.956+0000 7f0e417a9700 1 -- 172.21.15.3:0/3378175600 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0e3c135e50 msgr2=0x7f0e3c13a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:36.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.956+0000 7f0e417a9700 1 --2- 172.21.15.3:0/3378175600 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0e3c135e50 0x7f0e3c13a230 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f0e34015000 tx=0x7f0e34004ad0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.956+0000 7f0e417a9700 1 -- 172.21.15.3:0/3378175600 shutdown_connections 2023-10-03T10:19:36.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.956+0000 7f0e417a9700 1 --2- 172.21.15.3:0/3378175600 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0e3c135e50 0x7f0e3c13a230 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.956+0000 7f0e417a9700 1 --2- 172.21.15.3:0/3378175600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e3c09d6f0 0x7f0e3c1397b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.956+0000 7f0e417a9700 1 --2- 172.21.15.3:0/3378175600 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0e3c09e1d0 0x7f0e3c139cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:36.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.956+0000 7f0e417a9700 1 -- 172.21.15.3:0/3378175600 >> 172.21.15.3:0/3378175600 conn(0x7f0e3c066f50 msgr2=0x7f0e3c06e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:36.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.956+0000 7f0e417a9700 1 -- 172.21.15.3:0/3378175600 shutdown_connections 2023-10-03T10:19:36.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:36.956+0000 7f0e417a9700 1 -- 172.21.15.3:0/3378175600 wait complete. 2023-10-03T10:19:36.969 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:37.905 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:37.930 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:37.935 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:37.970 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:38.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.060+0000 7fa1b51e1700 1 Processor -- start 2023-10-03T10:19:38.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.060+0000 7fa1b51e1700 1 -- start start 2023-10-03T10:19:38.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.060+0000 7fa1b51e1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1b009f410 0x7fa1b0136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:38.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.060+0000 7fa1b51e1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa1b0136fe0 0x7fa1b01393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:38.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.060+0000 7fa1b51e1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1b0070940 con 0x7fa1b009e930 2023-10-03T10:19:38.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.060+0000 7fa1b51e1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b006fd60 con 0x7fa1b009f410 2023-10-03T10:19:38.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.060+0000 7fa1b51e1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b006fa80 con 0x7fa1b0136fe0 2023-10-03T10:19:38.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.060+0000 7fa1aed9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa1b009e930 0x7fa1b009ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47422/0 (socket says 172.21.15.3:47422) 2023-10-03T10:19:38.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.060+0000 7fa1aed9d700 1 -- 172.21.15.3:0/1504352678 learned_addr learned my addr 172.21.15.3:0/1504352678 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:38.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.060+0000 7fa1af59e700 1 --2- 172.21.15.3:0/1504352678 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa1b0136fe0 0x7fa1b01393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:38.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.060+0000 7fa1ae59c700 1 --2- 172.21.15.3:0/1504352678 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1b009f410 0x7fa1b0136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:38.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.061+0000 7fa1add9b700 1 -- 172.21.15.3:0/1504352678 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3541489877 0 0) 0x7fa1b0070940 con 0x7fa1b009e930 2023-10-03T10:19:38.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.061+0000 7fa1add9b700 1 -- 172.21.15.3:0/1504352678 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1980035b0 con 0x7fa1b009e930 2023-10-03T10:19:38.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.061+0000 7fa1af59e700 1 -- 172.21.15.3:0/1504352678 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1b009f410 msgr2=0x7fa1b0136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:38.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.061+0000 7fa1af59e700 1 --2- 172.21.15.3:0/1504352678 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1b009f410 0x7fa1b0136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.061+0000 7fa1af59e700 1 -- 172.21.15.3:0/1504352678 >> v1:172.21.15.3:6789/0 conn(0x7fa1b009e930 legacy=0x7fa1b009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:38.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.061+0000 7fa1af59e700 1 -- 172.21.15.3:0/1504352678 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b009fb60 con 0x7fa1b0136fe0 2023-10-03T10:19:38.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.061+0000 7fa1af59e700 1 --2- 172.21.15.3:0/1504352678 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa1b0136fe0 0x7fa1b01393c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009dc0 tx=0x7fa1a800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25ee6c3df84fae21 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:38.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.062+0000 7fa1add9b700 1 -- 172.21.15.3:0/1504352678 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a800e070 con 0x7fa1b0136fe0 2023-10-03T10:19:38.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.062+0000 7fa1add9b700 1 -- 172.21.15.3:0/1504352678 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a800bb20 con 0x7fa1b0136fe0 2023-10-03T10:19:38.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.062+0000 7fa1add9b700 1 -- 172.21.15.3:0/1504352678 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a800be90 con 0x7fa1b0136fe0 2023-10-03T10:19:38.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.062+0000 7fa1b51e1700 1 -- 172.21.15.3:0/1504352678 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa1b0136fe0 msgr2=0x7fa1b01393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:38.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.062+0000 7fa1b51e1700 1 --2- 172.21.15.3:0/1504352678 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa1b0136fe0 0x7fa1b01393c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009dc0 tx=0x7fa1a800b330 comp rx=0 tx=0).stop 2023-10-03T10:19:38.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.062+0000 7fa1b51e1700 1 -- 172.21.15.3:0/1504352678 shutdown_connections 2023-10-03T10:19:38.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.062+0000 7fa1b51e1700 1 --2- 172.21.15.3:0/1504352678 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa1b0136fe0 0x7fa1b01393c0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.062+0000 7fa1b51e1700 1 --2- 172.21.15.3:0/1504352678 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1b009f410 0x7fa1b0136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.063+0000 7fa1b51e1700 1 -- 172.21.15.3:0/1504352678 >> 172.21.15.3:0/1504352678 conn(0x7fa1b00a4210 msgr2=0x7fa1b00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:38.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.063+0000 7fa1b51e1700 1 -- 172.21.15.3:0/1504352678 shutdown_connections 2023-10-03T10:19:38.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.063+0000 7fa1b51e1700 1 -- 172.21.15.3:0/1504352678 wait complete. 2023-10-03T10:19:38.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.064+0000 7fa1b51e1700 1 Processor -- start 2023-10-03T10:19:38.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.064+0000 7fa1b51e1700 1 -- start start 2023-10-03T10:19:38.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.065+0000 7fa1b51e1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa1b009e930 0x7fa1b006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:38.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.065+0000 7fa1b51e1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1b009f410 0x7fa1b006bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:38.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.065+0000 7fa1b51e1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1b0136fe0 0x7fa1b006c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:38.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.065+0000 7fa1b51e1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b0071d20 con 0x7fa1b009f410 2023-10-03T10:19:38.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.065+0000 7fa1b51e1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b006d5d0 con 0x7fa1b0136fe0 2023-10-03T10:19:38.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.065+0000 7fa1b51e1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b0123e60 con 0x7fa1b009e930 2023-10-03T10:19:38.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.065+0000 7fa1af59e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1b0136fe0 0x7fa1b006c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:38.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.065+0000 7fa1af59e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1b0136fe0 0x7fa1b006c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41108/0 (socket says 172.21.15.3:41108) 2023-10-03T10:19:38.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.065+0000 7fa1af59e700 1 -- 172.21.15.3:0/1054884666 learned_addr learned my addr 172.21.15.3:0/1054884666 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:38.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.065+0000 7fa1aed9d700 1 --2- 172.21.15.3:0/1054884666 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa1b009e930 0x7fa1b006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:38.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.066+0000 7fa1ae59c700 1 --2- 172.21.15.3:0/1054884666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1b009f410 0x7fa1b006bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:38.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.066+0000 7fa1af59e700 1 -- 172.21.15.3:0/1054884666 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa1b009e930 msgr2=0x7fa1b006b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:38.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.066+0000 7fa1af59e700 1 --2- 172.21.15.3:0/1054884666 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa1b009e930 0x7fa1b006b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.066+0000 7fa1af59e700 1 -- 172.21.15.3:0/1054884666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1b009f410 msgr2=0x7fa1b006bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:38.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.066+0000 7fa1af59e700 1 --2- 172.21.15.3:0/1054884666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1b009f410 0x7fa1b006bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.066+0000 7fa1af59e700 1 -- 172.21.15.3:0/1054884666 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a8009a70 con 0x7fa1b0136fe0 2023-10-03T10:19:38.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.066+0000 7fa1af59e700 1 --2- 172.21.15.3:0/1054884666 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1b0136fe0 0x7fa1b006c330 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fa1a800b370 tx=0x7fa1a8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:38.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.066+0000 7fa1977fe700 1 -- 172.21.15.3:0/1054884666 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8007000 con 0x7fa1b0136fe0 2023-10-03T10:19:38.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.066+0000 7fa1977fe700 1 -- 172.21.15.3:0/1054884666 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a8012650 con 0x7fa1b0136fe0 2023-10-03T10:19:38.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.066+0000 7fa1b51e1700 1 -- 172.21.15.3:0/1054884666 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b009de80 con 0x7fa1b0136fe0 2023-10-03T10:19:38.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.066+0000 7fa1977fe700 1 -- 172.21.15.3:0/1054884666 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8012a00 con 0x7fa1b0136fe0 2023-10-03T10:19:38.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.067+0000 7fa1b51e1700 1 -- 172.21.15.3:0/1054884666 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b01248d0 con 0x7fa1b0136fe0 2023-10-03T10:19:38.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.068+0000 7fa1b51e1700 1 -- 172.21.15.3:0/1054884666 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1b0074ea0 con 0x7fa1b0136fe0 2023-10-03T10:19:38.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.068+0000 7fa1977fe700 1 -- 172.21.15.3:0/1054884666 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa1a8012b60 con 0x7fa1b0136fe0 2023-10-03T10:19:38.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.069+0000 7fa1977fe700 1 -- 172.21.15.3:0/1054884666 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa1a8041f60 con 0x7fa1b0136fe0 2023-10-03T10:19:38.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.077+0000 7fa1977fe700 1 -- 172.21.15.3:0/1054884666 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa1a8012e10 con 0x7fa1b0136fe0 2023-10-03T10:19:38.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.106+0000 7f307f59e700 1 Processor -- start 2023-10-03T10:19:38.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.106+0000 7f307f59e700 1 -- start start 2023-10-03T10:19:38.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.106+0000 7f307f59e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30801348a0 0x7f3080134c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:38.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.107+0000 7f307f59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f308009dc40 0x7f308013fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.107+0000 7f307f59e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f308009ebc0 con 0x7f308009d100 2023-10-03T10:19:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.107+0000 7f307f59e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3080095cb0 con 0x7f308009dc40 2023-10-03T10:19:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.107+0000 7f307f59e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f308009e980 con 0x7f30801348a0 2023-10-03T10:19:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.107+0000 7f307dd9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f308009d100 0x7f308009d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47434/0 (socket says 172.21.15.3:47434) 2023-10-03T10:19:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.107+0000 7f307dd9b700 1 -- 172.21.15.3:0/3190626018 learned_addr learned my addr 172.21.15.3:0/3190626018 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.107+0000 7f307ed9d700 1 --2- 172.21.15.3:0/3190626018 >> v2:172.21.15.3:3300/0 conn(0x7f308009dc40 0x7f308013fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.107+0000 7f307e59c700 1 --2- 172.21.15.3:0/3190626018 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30801348a0 0x7f3080134c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:38.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.107+0000 7f307ed9d700 1 -- 172.21.15.3:0/3190626018 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30801348a0 msgr2=0x7f3080134c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:38.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.107+0000 7f307ed9d700 1 --2- 172.21.15.3:0/3190626018 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30801348a0 0x7f3080134c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.107+0000 7f307ed9d700 1 -- 172.21.15.3:0/3190626018 >> v1:172.21.15.3:6789/0 conn(0x7f308009d100 legacy=0x7f308009d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:38.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.107+0000 7f307ed9d700 1 -- 172.21.15.3:0/3190626018 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3080094ec0 con 0x7f308009dc40 2023-10-03T10:19:38.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.108+0000 7fe2a8a98700 1 Processor -- start 2023-10-03T10:19:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.108+0000 7f307ed9d700 1 --2- 172.21.15.3:0/3190626018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308009dc40 0x7f308013fc40 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f3078009d90 tx=0x7f307800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a7ff67f245218c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.108+0000 7f307d59a700 1 -- 172.21.15.3:0/3190626018 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307800e070 con 0x7f308009dc40 2023-10-03T10:19:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.108+0000 7f307d59a700 1 -- 172.21.15.3:0/3190626018 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307800bb20 con 0x7f308009dc40 2023-10-03T10:19:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.108+0000 7f307d59a700 1 -- 172.21.15.3:0/3190626018 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3078012430 con 0x7f308009dc40 2023-10-03T10:19:38.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.108+0000 7fe2a8a98700 1 -- start start 2023-10-03T10:19:38.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.108+0000 7fe2a8a98700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe2a40a0a30 0x7fe2a40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:38.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.108+0000 7fe2a8a98700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe2a4138d80 0x7fe2a413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:38.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.108+0000 7fe2a8a98700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2a40a2a70 con 0x7fe2a409d630 2023-10-03T10:19:38.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.108+0000 7fe2a8a98700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a40a01d0 con 0x7fe2a40a0a30 2023-10-03T10:19:38.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.108+0000 7fe2a8a98700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a409e2b0 con 0x7fe2a4138d80 2023-10-03T10:19:38.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.109+0000 7fe2a2d9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe2a4138d80 0x7fe2a413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.109+0000 7fe2a1d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe2a409d630 0x7fe2a40a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47450/0 (socket says 172.21.15.3:47450) 2023-10-03T10:19:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.109+0000 7fe2a1d9b700 1 -- 172.21.15.3:0/276953731 learned_addr learned my addr 172.21.15.3:0/276953731 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.109+0000 7fe2a259c700 1 --2- 172.21.15.3:0/276953731 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe2a40a0a30 0x7fe2a40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.109+0000 7fe2a159a700 1 -- 172.21.15.3:0/276953731 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3110327600 0 0) 0x7fe2a40a2a70 con 0x7fe2a409d630 2023-10-03T10:19:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.109+0000 7f307f59e700 1 -- 172.21.15.3:0/3190626018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308009dc40 msgr2=0x7f308013fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.109+0000 7f307f59e700 1 --2- 172.21.15.3:0/3190626018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308009dc40 0x7f308013fc40 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f3078009d90 tx=0x7f307800b330 comp rx=0 tx=0).stop 2023-10-03T10:19:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.109+0000 7fe2a259c700 1 -- 172.21.15.3:0/276953731 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe2a4138d80 msgr2=0x7fe2a413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.109+0000 7fe2a259c700 1 --2- 172.21.15.3:0/276953731 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe2a4138d80 0x7fe2a413b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.109+0000 7fe2a259c700 1 -- 172.21.15.3:0/276953731 >> v1:172.21.15.3:6789/0 conn(0x7fe2a409d630 legacy=0x7fe2a40a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.109+0000 7fe2a259c700 1 -- 172.21.15.3:0/276953731 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a409dc50 con 0x7fe2a40a0a30 2023-10-03T10:19:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.109+0000 7f307f59e700 1 -- 172.21.15.3:0/3190626018 shutdown_connections 2023-10-03T10:19:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.109+0000 7f307f59e700 1 --2- 172.21.15.3:0/3190626018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308009dc40 0x7f308013fc40 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.109+0000 7f307f59e700 1 --2- 172.21.15.3:0/3190626018 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30801348a0 0x7f3080134c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.109+0000 7f307f59e700 1 -- 172.21.15.3:0/3190626018 >> 172.21.15.3:0/3190626018 conn(0x7f30800a2540 msgr2=0x7f30800a2940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.109+0000 7f307f59e700 1 -- 172.21.15.3:0/3190626018 shutdown_connections 2023-10-03T10:19:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.109+0000 7fe2a259c700 1 --2- 172.21.15.3:0/276953731 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe2a40a0a30 0x7fe2a40a0e00 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fe294009870 tx=0x7fe294009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1f0aaa036e489ac server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:38.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.109+0000 7f307f59e700 1 -- 172.21.15.3:0/3190626018 wait complete. 2023-10-03T10:19:38.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.111+0000 7fe2a159a700 1 -- 172.21.15.3:0/276953731 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe294011830 con 0x7fe2a40a0a30 2023-10-03T10:19:38.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.111+0000 7fe2a159a700 1 -- 172.21.15.3:0/276953731 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe294011990 con 0x7fe2a40a0a30 2023-10-03T10:19:38.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.111+0000 7fe2a159a700 1 -- 172.21.15.3:0/276953731 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe294011d20 con 0x7fe2a40a0a30 2023-10-03T10:19:38.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.111+0000 7f307f59e700 1 Processor -- start 2023-10-03T10:19:38.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.111+0000 7fe2a8a98700 1 -- 172.21.15.3:0/276953731 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe2a40a0a30 msgr2=0x7fe2a40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:38.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.111+0000 7fe2a8a98700 1 --2- 172.21.15.3:0/276953731 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe2a40a0a30 0x7fe2a40a0e00 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fe294009870 tx=0x7fe294009b80 comp rx=0 tx=0).stop 2023-10-03T10:19:38.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.112+0000 7fe2a8a98700 1 -- 172.21.15.3:0/276953731 shutdown_connections 2023-10-03T10:19:38.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.112+0000 7fe2a8a98700 1 --2- 172.21.15.3:0/276953731 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe2a40a0a30 0x7fe2a40a0e00 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.112+0000 7fe2a8a98700 1 --2- 172.21.15.3:0/276953731 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe2a4138d80 0x7fe2a413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.112+0000 7fe2a8a98700 1 -- 172.21.15.3:0/276953731 >> 172.21.15.3:0/276953731 conn(0x7fe2a40a3050 msgr2=0x7fe2a40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:38.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.111+0000 7f307f59e700 1 -- start start 2023-10-03T10:19:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.112+0000 7f307f59e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f308009d100 0x7f3080133b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.112+0000 7f307f59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308009dc40 0x7f3080134050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.112+0000 7f307f59e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30801348a0 0x7f308006e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.112+0000 7f307f59e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30800728e0 con 0x7f308009dc40 2023-10-03T10:19:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.112+0000 7f307f59e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3080072680 con 0x7f308009d100 2023-10-03T10:19:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.112+0000 7f307f59e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3080094080 con 0x7f30801348a0 2023-10-03T10:19:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.112+0000 7fe2a8a98700 1 -- 172.21.15.3:0/276953731 shutdown_connections 2023-10-03T10:19:38.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.112+0000 7f307dd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308009dc40 0x7f3080134050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:38.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.112+0000 7f307dd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308009dc40 0x7f3080134050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37746/0 (socket says 172.21.15.3:37746) 2023-10-03T10:19:38.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.112+0000 7f307dd9b700 1 -- 172.21.15.3:0/2673747560 learned_addr learned my addr 172.21.15.3:0/2673747560 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:38.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.112+0000 7f307e59c700 1 --2- 172.21.15.3:0/2673747560 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f308009d100 0x7f3080133b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:38.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.112+0000 7f307ed9d700 1 --2- 172.21.15.3:0/2673747560 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30801348a0 0x7f308006e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:38.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.113+0000 7f307dd9b700 1 -- 172.21.15.3:0/2673747560 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30801348a0 msgr2=0x7f308006e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:38.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.113+0000 7f307dd9b700 1 --2- 172.21.15.3:0/2673747560 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30801348a0 0x7f308006e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.113+0000 7f307dd9b700 1 -- 172.21.15.3:0/2673747560 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f308009d100 msgr2=0x7f3080133b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:38.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.113+0000 7f307dd9b700 1 --2- 172.21.15.3:0/2673747560 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f308009d100 0x7f3080133b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.113+0000 7f307dd9b700 1 -- 172.21.15.3:0/2673747560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3078009a70 con 0x7f308009dc40 2023-10-03T10:19:38.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.113+0000 7f307dd9b700 1 --2- 172.21.15.3:0/2673747560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308009dc40 0x7f3080134050 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f307400c450 tx=0x7f307400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:38.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.113+0000 7f306effd700 1 -- 172.21.15.3:0/2673747560 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074005d70 con 0x7f308009dc40 2023-10-03T10:19:38.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.113+0000 7f306effd700 1 -- 172.21.15.3:0/2673747560 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30740070d0 con 0x7f308009dc40 2023-10-03T10:19:38.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.113+0000 7f306effd700 1 -- 172.21.15.3:0/2673747560 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074007440 con 0x7f308009dc40 2023-10-03T10:19:38.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.112+0000 7fe2a8a98700 1 -- 172.21.15.3:0/276953731 wait complete. 2023-10-03T10:19:38.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.113+0000 7fe2a8a98700 1 Processor -- start 2023-10-03T10:19:38.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.113+0000 7fe2a8a98700 1 -- start start 2023-10-03T10:19:38.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.114+0000 7fe2a8a98700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2a409d630 0x7fe2a41cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:38.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.114+0000 7fe2a8a98700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe2a40a0a30 0x7fe2a41ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:38.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.114+0000 7fe2a8a98700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe2a4138d80 0x7fe2a406dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:38.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.114+0000 7fe2a8a98700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a40a0410 con 0x7fe2a409d630 2023-10-03T10:19:38.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.114+0000 7fe2a8a98700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a4071ba0 con 0x7fe2a40a0a30 2023-10-03T10:19:38.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.114+0000 7fe2a8a98700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a40715e0 con 0x7fe2a4138d80 2023-10-03T10:19:38.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.115+0000 7f307f59e700 1 -- 172.21.15.3:0/2673747560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3080071500 con 0x7f308009dc40 2023-10-03T10:19:38.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.115+0000 7f307f59e700 1 -- 172.21.15.3:0/2673747560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30800721a0 con 0x7f308009dc40 2023-10-03T10:19:38.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.114+0000 7fe2a1d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe2a40a0a30 0x7fe2a41ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:38.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.114+0000 7fe2a1d9b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe2a40a0a30 0x7fe2a41ccec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41126/0 (socket says 172.21.15.3:41126) 2023-10-03T10:19:38.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.114+0000 7fe2a1d9b700 1 -- 172.21.15.3:0/2840052226 learned_addr learned my addr 172.21.15.3:0/2840052226 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:38.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.114+0000 7fe2a259c700 1 --2- 172.21.15.3:0/2840052226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2a409d630 0x7fe2a41cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:38.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.114+0000 7fe2a2d9d700 1 --2- 172.21.15.3:0/2840052226 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe2a4138d80 0x7fe2a406dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:38.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.114+0000 7fe2a1d9b700 1 -- 172.21.15.3:0/2840052226 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe2a4138d80 msgr2=0x7fe2a406dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:38.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.115+0000 7fe2a1d9b700 1 --2- 172.21.15.3:0/2840052226 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe2a4138d80 0x7fe2a406dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.115+0000 7fe2a1d9b700 1 -- 172.21.15.3:0/2840052226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2a409d630 msgr2=0x7fe2a41cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:38.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.115+0000 7fe2a1d9b700 1 --2- 172.21.15.3:0/2840052226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2a409d630 0x7fe2a41cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.115+0000 7fe2a1d9b700 1 -- 172.21.15.3:0/2840052226 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2940094a0 con 0x7fe2a40a0a30 2023-10-03T10:19:38.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.115+0000 7fe2a2d9d700 1 --2- 172.21.15.3:0/2840052226 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe2a4138d80 0x7fe2a406dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:19:38.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.115+0000 7fe2a1d9b700 1 --2- 172.21.15.3:0/2840052226 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe2a40a0a30 0x7fe2a41ccec0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fe28c00c540 tx=0x7fe28c00c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:38.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.115+0000 7fe29affd700 1 -- 172.21.15.3:0/2840052226 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c004c80 con 0x7fe2a40a0a30 2023-10-03T10:19:38.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.115+0000 7fe29affd700 1 -- 172.21.15.3:0/2840052226 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28c004de0 con 0x7fe2a40a0a30 2023-10-03T10:19:38.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.115+0000 7fe29affd700 1 -- 172.21.15.3:0/2840052226 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c007310 con 0x7fe2a40a0a30 2023-10-03T10:19:38.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.116+0000 7fe2a8a98700 1 -- 172.21.15.3:0/2840052226 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2a4071040 con 0x7fe2a40a0a30 2023-10-03T10:19:38.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.116+0000 7fe2a8a98700 1 -- 172.21.15.3:0/2840052226 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2a41cd400 con 0x7fe2a40a0a30 2023-10-03T10:19:38.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.117+0000 7f307f59e700 1 -- 172.21.15.3:0/2673747560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f308004f2a0 con 0x7f308009dc40 2023-10-03T10:19:38.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.117+0000 7f306effd700 1 -- 172.21.15.3:0/2673747560 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f30740075a0 con 0x7f308009dc40 2023-10-03T10:19:38.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.117+0000 7f306effd700 1 -- 172.21.15.3:0/2673747560 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f307403ea90 con 0x7f308009dc40 2023-10-03T10:19:38.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.117+0000 7fe29affd700 1 -- 172.21.15.3:0/2840052226 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe28c007470 con 0x7fe2a40a0a30 2023-10-03T10:19:38.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.117+0000 7fe2a8a98700 1 -- 172.21.15.3:0/2840052226 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2a40747f0 con 0x7fe2a40a0a30 2023-10-03T10:19:38.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.117+0000 7fe29affd700 1 -- 172.21.15.3:0/2840052226 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fe28c03f530 con 0x7fe2a40a0a30 2023-10-03T10:19:38.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.127+0000 7fe29affd700 1 -- 172.21.15.3:0/2840052226 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe28c0123b0 con 0x7fe2a40a0a30 2023-10-03T10:19:38.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.127+0000 7f306effd700 1 -- 172.21.15.3:0/2673747560 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3074019b40 con 0x7f308009dc40 2023-10-03T10:19:38.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.153+0000 7ff33c32f700 1 Processor -- start 2023-10-03T10:19:38.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.153+0000 7ff33c32f700 1 -- start start 2023-10-03T10:19:38.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.154+0000 7ff33c32f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff33409d630 0x7ff3340a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:38.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.154+0000 7ff33c32f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff334138d80 0x7ff33413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:38.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.154+0000 7ff33c32f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3340a2a70 con 0x7ff3340a0a30 2023-10-03T10:19:38.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.154+0000 7ff33c32f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3340a01d0 con 0x7ff334138d80 2023-10-03T10:19:38.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.154+0000 7ff33c32f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff33409e2b0 con 0x7ff33409d630 2023-10-03T10:19:38.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.155+0000 7ff33a8cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff334138d80 0x7ff33413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:38.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.155+0000 7ff33a8cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff334138d80 0x7ff33413b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41128/0 (socket says 172.21.15.3:41128) 2023-10-03T10:19:38.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.155+0000 7ff33a8cc700 1 -- 172.21.15.3:0/2539539112 learned_addr learned my addr 172.21.15.3:0/2539539112 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:38.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.155+0000 7ff33a0cb700 1 --1- 172.21.15.3:0/2539539112 >> v1:172.21.15.3:6789/0 conn(0x7ff3340a0a30 0x7ff3340a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47464/0 (socket says 172.21.15.3:47464) 2023-10-03T10:19:38.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.155+0000 7ff3398ca700 1 --2- 172.21.15.3:0/2539539112 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff33409d630 0x7ff3340a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:38.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.155+0000 7ff33a8cc700 1 -- 172.21.15.3:0/2539539112 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff33409d630 msgr2=0x7ff3340a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:38.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.155+0000 7ff33a8cc700 1 --2- 172.21.15.3:0/2539539112 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff33409d630 0x7ff3340a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.155+0000 7ff33a8cc700 1 -- 172.21.15.3:0/2539539112 >> v1:172.21.15.3:6789/0 conn(0x7ff3340a0a30 legacy=0x7ff3340a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:38.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.155+0000 7ff33a8cc700 1 -- 172.21.15.3:0/2539539112 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33409dc50 con 0x7ff334138d80 2023-10-03T10:19:38.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.155+0000 7ff33a8cc700 1 --2- 172.21.15.3:0/2539539112 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff334138d80 0x7ff33413b160 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009e80 tx=0x7ff32c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dce0c3c65d2a9def server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:38.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.155+0000 7ff3390c9700 1 -- 172.21.15.3:0/2539539112 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c00e070 con 0x7ff334138d80 2023-10-03T10:19:38.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.156+0000 7ff3390c9700 1 -- 172.21.15.3:0/2539539112 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32c00b910 con 0x7ff334138d80 2023-10-03T10:19:38.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.156+0000 7ff3390c9700 1 -- 172.21.15.3:0/2539539112 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c00bcc0 con 0x7ff334138d80 2023-10-03T10:19:38.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.156+0000 7ff33c32f700 1 -- 172.21.15.3:0/2539539112 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff334138d80 msgr2=0x7ff33413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:38.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.156+0000 7ff33c32f700 1 --2- 172.21.15.3:0/2539539112 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff334138d80 0x7ff33413b160 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009e80 tx=0x7ff32c00b330 comp rx=0 tx=0).stop 2023-10-03T10:19:38.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.156+0000 7ff33c32f700 1 -- 172.21.15.3:0/2539539112 shutdown_connections 2023-10-03T10:19:38.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.156+0000 7ff33c32f700 1 --2- 172.21.15.3:0/2539539112 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff334138d80 0x7ff33413b160 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.156+0000 7ff33c32f700 1 --2- 172.21.15.3:0/2539539112 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff33409d630 0x7ff3340a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.156+0000 7ff33c32f700 1 -- 172.21.15.3:0/2539539112 >> 172.21.15.3:0/2539539112 conn(0x7ff3340a3050 msgr2=0x7ff3340a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:38.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.156+0000 7ff33c32f700 1 -- 172.21.15.3:0/2539539112 shutdown_connections 2023-10-03T10:19:38.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.157+0000 7ff33c32f700 1 -- 172.21.15.3:0/2539539112 wait complete. 2023-10-03T10:19:38.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.157+0000 7ff33c32f700 1 Processor -- start 2023-10-03T10:19:38.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.158+0000 7ff33c32f700 1 -- start start 2023-10-03T10:19:38.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.158+0000 7ff33c32f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff33409d630 0x7ff334133800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:38.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.158+0000 7ff33c32f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3340a0a30 0x7ff334133d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:38.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.158+0000 7ff33c32f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff334138d80 0x7ff334134280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:38.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.158+0000 7ff33c32f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff334137320 con 0x7ff3340a0a30 2023-10-03T10:19:38.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.158+0000 7ff33c32f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff33413ad30 con 0x7ff334138d80 2023-10-03T10:19:38.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.158+0000 7ff33c32f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff3340a0410 con 0x7ff33409d630 2023-10-03T10:19:38.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.158+0000 7ff33a8cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff334138d80 0x7ff334134280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:38.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.158+0000 7ff33a8cc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff334138d80 0x7ff334134280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41134/0 (socket says 172.21.15.3:41134) 2023-10-03T10:19:38.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.158+0000 7ff33a8cc700 1 -- 172.21.15.3:0/3873609491 learned_addr learned my addr 172.21.15.3:0/3873609491 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:38.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.158+0000 7ff33a0cb700 1 --2- 172.21.15.3:0/3873609491 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff33409d630 0x7ff334133800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:38.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.159+0000 7ff3398ca700 1 --2- 172.21.15.3:0/3873609491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3340a0a30 0x7ff334133d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:38.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.159+0000 7ff33a8cc700 1 -- 172.21.15.3:0/3873609491 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff33409d630 msgr2=0x7ff334133800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:38.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.159+0000 7ff33a8cc700 1 --2- 172.21.15.3:0/3873609491 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff33409d630 0x7ff334133800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.159+0000 7ff33a8cc700 1 -- 172.21.15.3:0/3873609491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3340a0a30 msgr2=0x7ff334133d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:38.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.159+0000 7ff33a8cc700 1 --2- 172.21.15.3:0/3873609491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3340a0a30 0x7ff334133d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.159+0000 7ff33a8cc700 1 -- 172.21.15.3:0/3873609491 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c009b30 con 0x7ff334138d80 2023-10-03T10:19:38.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.159+0000 7ff3398ca700 1 --2- 172.21.15.3:0/3873609491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3340a0a30 0x7ff334133d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:19:38.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.159+0000 7ff33a0cb700 1 --2- 172.21.15.3:0/3873609491 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff33409d630 0x7ff334133800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:38.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.159+0000 7ff33a8cc700 1 --2- 172.21.15.3:0/3873609491 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff334138d80 0x7ff334134280 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ff32c012000 tx=0x7ff32c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:38.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.159+0000 7ff32affd700 1 -- 172.21.15.3:0/3873609491 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c004240 con 0x7ff334138d80 2023-10-03T10:19:38.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.159+0000 7ff32affd700 1 -- 172.21.15.3:0/3873609491 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32c0043e0 con 0x7ff334138d80 2023-10-03T10:19:38.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.159+0000 7ff32affd700 1 -- 172.21.15.3:0/3873609491 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c004790 con 0x7ff334138d80 2023-10-03T10:19:38.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.159+0000 7ff33c32f700 1 -- 172.21.15.3:0/3873609491 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff334136c70 con 0x7ff334138d80 2023-10-03T10:19:38.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.159+0000 7ff33c32f700 1 -- 172.21.15.3:0/3873609491 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3341379a0 con 0x7ff334138d80 2023-10-03T10:19:38.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.161+0000 7ff33c32f700 1 -- 172.21.15.3:0/3873609491 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3340747f0 con 0x7ff334138d80 2023-10-03T10:19:38.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.161+0000 7ff32affd700 1 -- 172.21.15.3:0/3873609491 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff32c01c410 con 0x7ff334138d80 2023-10-03T10:19:38.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.161+0000 7ff32affd700 1 -- 172.21.15.3:0/3873609491 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff32c041120 con 0x7ff334138d80 2023-10-03T10:19:38.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.171+0000 7ff32affd700 1 -- 172.21.15.3:0/3873609491 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff32c01c6c0 con 0x7ff334138d80 2023-10-03T10:19:38.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.290+0000 7fa1b51e1700 1 -- 172.21.15.3:0/1054884666 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa1b006b250 con 0x7fa1b0136fe0 2023-10-03T10:19:38.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.290+0000 7fa1977fe700 1 -- 172.21.15.3:0/1054884666 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa1a801ab80 con 0x7fa1b0136fe0 2023-10-03T10:19:38.291 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:38.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.294+0000 7fa1b51e1700 1 -- 172.21.15.3:0/1054884666 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1b0136fe0 msgr2=0x7fa1b006c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:38.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.294+0000 7fa1b51e1700 1 --2- 172.21.15.3:0/1054884666 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1b0136fe0 0x7fa1b006c330 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fa1a800b370 tx=0x7fa1a8004ad0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.294+0000 7fa1b51e1700 1 -- 172.21.15.3:0/1054884666 shutdown_connections 2023-10-03T10:19:38.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.294+0000 7fa1b51e1700 1 --2- 172.21.15.3:0/1054884666 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1b0136fe0 0x7fa1b006c330 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.294+0000 7fa1b51e1700 1 --2- 172.21.15.3:0/1054884666 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa1b009e930 0x7fa1b006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.294+0000 7fa1b51e1700 1 --2- 172.21.15.3:0/1054884666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1b009f410 0x7fa1b006bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.294+0000 7fa1b51e1700 1 -- 172.21.15.3:0/1054884666 >> 172.21.15.3:0/1054884666 conn(0x7fa1b00a4210 msgr2=0x7fa1b0134890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:38.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.294+0000 7fa1b51e1700 1 -- 172.21.15.3:0/1054884666 shutdown_connections 2023-10-03T10:19:38.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.294+0000 7fa1b51e1700 1 -- 172.21.15.3:0/1054884666 wait complete. 2023-10-03T10:19:38.313 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:38.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.335+0000 7fe2a8a98700 1 -- 172.21.15.3:0/2840052226 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe2a40711d0 con 0x7fe2a40a0a30 2023-10-03T10:19:38.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.336+0000 7fe29affd700 1 -- 172.21.15.3:0/2840052226 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe28c01b340 con 0x7fe2a40a0a30 2023-10-03T10:19:38.336 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:38.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.339+0000 7fe2a8a98700 1 -- 172.21.15.3:0/2840052226 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe2a40a0a30 msgr2=0x7fe2a41ccec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:38.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.339+0000 7fe2a8a98700 1 --2- 172.21.15.3:0/2840052226 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe2a40a0a30 0x7fe2a41ccec0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fe28c00c540 tx=0x7fe28c00c850 comp rx=0 tx=0).stop 2023-10-03T10:19:38.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.339+0000 7fe2a8a98700 1 -- 172.21.15.3:0/2840052226 shutdown_connections 2023-10-03T10:19:38.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.339+0000 7fe2a8a98700 1 --2- 172.21.15.3:0/2840052226 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fe2a4138d80 0x7fe2a406dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.339+0000 7fe2a8a98700 1 --2- 172.21.15.3:0/2840052226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2a409d630 0x7fe2a41cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.339+0000 7fe2a8a98700 1 --2- 172.21.15.3:0/2840052226 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe2a40a0a30 0x7fe2a41ccec0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.339+0000 7fe2a8a98700 1 -- 172.21.15.3:0/2840052226 >> 172.21.15.3:0/2840052226 conn(0x7fe2a40a3050 msgr2=0x7fe2a40a6800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:38.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.339+0000 7fe2a8a98700 1 -- 172.21.15.3:0/2840052226 shutdown_connections 2023-10-03T10:19:38.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.339+0000 7fe2a8a98700 1 -- 172.21.15.3:0/2840052226 wait complete. 2023-10-03T10:19:38.358 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:38.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.371+0000 7f307f59e700 1 -- 172.21.15.3:0/2673747560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f308006b7c0 con 0x7f308009dc40 2023-10-03T10:19:38.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.371+0000 7f306effd700 1 -- 172.21.15.3:0/2673747560 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3074017070 con 0x7f308009dc40 2023-10-03T10:19:38.371 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:38.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.373+0000 7f307f59e700 1 -- 172.21.15.3:0/2673747560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308009dc40 msgr2=0x7f3080134050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:38.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.373+0000 7f307f59e700 1 --2- 172.21.15.3:0/2673747560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308009dc40 0x7f3080134050 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f307400c450 tx=0x7f307400c760 comp rx=0 tx=0).stop 2023-10-03T10:19:38.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.373+0000 7f307f59e700 1 -- 172.21.15.3:0/2673747560 shutdown_connections 2023-10-03T10:19:38.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.373+0000 7f307f59e700 1 --2- 172.21.15.3:0/2673747560 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f30801348a0 0x7f308006e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.373+0000 7f307f59e700 1 --2- 172.21.15.3:0/2673747560 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f308009d100 0x7f3080133b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.373+0000 7f307f59e700 1 --2- 172.21.15.3:0/2673747560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308009dc40 0x7f3080134050 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.373+0000 7f307f59e700 1 -- 172.21.15.3:0/2673747560 >> 172.21.15.3:0/2673747560 conn(0x7f30800a2540 msgr2=0x7f30800a16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:38.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.373+0000 7f307f59e700 1 -- 172.21.15.3:0/2673747560 shutdown_connections 2023-10-03T10:19:38.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.374+0000 7f307f59e700 1 -- 172.21.15.3:0/2673747560 wait complete. 2023-10-03T10:19:38.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.381+0000 7ff33c32f700 1 -- 172.21.15.3:0/3873609491 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff334072980 con 0x7ff334138d80 2023-10-03T10:19:38.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.381+0000 7ff32affd700 1 -- 172.21.15.3:0/3873609491 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff32c011390 con 0x7ff334138d80 2023-10-03T10:19:38.381 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:38.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.383+0000 7ff33c32f700 1 -- 172.21.15.3:0/3873609491 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff334138d80 msgr2=0x7ff334134280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:38.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.383+0000 7ff33c32f700 1 --2- 172.21.15.3:0/3873609491 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff334138d80 0x7ff334134280 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ff32c012000 tx=0x7ff32c004ad0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.383+0000 7ff33c32f700 1 -- 172.21.15.3:0/3873609491 shutdown_connections 2023-10-03T10:19:38.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.383+0000 7ff33c32f700 1 --2- 172.21.15.3:0/3873609491 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff334138d80 0x7ff334134280 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.383+0000 7ff33c32f700 1 --2- 172.21.15.3:0/3873609491 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff33409d630 0x7ff334133800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.383+0000 7ff33c32f700 1 --2- 172.21.15.3:0/3873609491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3340a0a30 0x7ff334133d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:38.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.383+0000 7ff33c32f700 1 -- 172.21.15.3:0/3873609491 >> 172.21.15.3:0/3873609491 conn(0x7ff3340a3050 msgr2=0x7ff33409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:38.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.384+0000 7ff33c32f700 1 -- 172.21.15.3:0/3873609491 shutdown_connections 2023-10-03T10:19:38.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:38.384+0000 7ff33c32f700 1 -- 172.21.15.3:0/3873609491 wait complete. 2023-10-03T10:19:38.389 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:38.398 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:39.314 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:39.359 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:39.390 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:39.399 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:39.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.478+0000 7fda66e2b700 1 Processor -- start 2023-10-03T10:19:39.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.478+0000 7fda66e2b700 1 -- start start 2023-10-03T10:19:39.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.479+0000 7fda66e2b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fda6009e1d0 0x7fda60138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:39.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.479+0000 7fda66e2b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda60139170 0x7fda6013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:39.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.479+0000 7fda66e2b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda6009b120 con 0x7fda6009d6f0 2023-10-03T10:19:39.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.479+0000 7fda66e2b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda6006b460 con 0x7fda6009e1d0 2023-10-03T10:19:39.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.479+0000 7fda66e2b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6009a7e0 con 0x7fda60139170 2023-10-03T10:19:39.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.479+0000 7fda653c8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda60139170 0x7fda6013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:39.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.479+0000 7fda653c8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda60139170 0x7fda6013b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41146/0 (socket says 172.21.15.3:41146) 2023-10-03T10:19:39.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.479+0000 7fda653c8700 1 -- 172.21.15.3:0/1387670692 learned_addr learned my addr 172.21.15.3:0/1387670692 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:39.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.479+0000 7fda5ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fda6009e1d0 0x7fda60138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:39.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.479+0000 7fda653c8700 1 -- 172.21.15.3:0/1387670692 >> v2:172.21.15.3:3300/0 conn(0x7fda6009e1d0 msgr2=0x7fda60138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:39.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.479+0000 7fda653c8700 1 --2- 172.21.15.3:0/1387670692 >> v2:172.21.15.3:3300/0 conn(0x7fda6009e1d0 0x7fda60138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.479+0000 7fda653c8700 1 -- 172.21.15.3:0/1387670692 >> v1:172.21.15.3:6789/0 conn(0x7fda6009d6f0 legacy=0x7fda6009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:39.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.480+0000 7fda653c8700 1 -- 172.21.15.3:0/1387670692 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6009a500 con 0x7fda60139170 2023-10-03T10:19:39.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.480+0000 7fda653c8700 1 --2- 172.21.15.3:0/1387670692 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda60139170 0x7fda6013b550 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fda58009e80 tx=0x7fda5800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=226f205b51ae91ec server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:39.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.480+0000 7fda5f7fe700 1 -- 172.21.15.3:0/1387670692 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5800e070 con 0x7fda60139170 2023-10-03T10:19:39.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.480+0000 7fda5f7fe700 1 -- 172.21.15.3:0/1387670692 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda5800b8d0 con 0x7fda60139170 2023-10-03T10:19:39.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.480+0000 7fda5f7fe700 1 -- 172.21.15.3:0/1387670692 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5800bc80 con 0x7fda60139170 2023-10-03T10:19:39.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.481+0000 7fda66e2b700 1 -- 172.21.15.3:0/1387670692 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda60139170 msgr2=0x7fda6013b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:39.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.481+0000 7fda66e2b700 1 --2- 172.21.15.3:0/1387670692 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda60139170 0x7fda6013b550 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fda58009e80 tx=0x7fda5800b330 comp rx=0 tx=0).stop 2023-10-03T10:19:39.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.481+0000 7fda66e2b700 1 -- 172.21.15.3:0/1387670692 shutdown_connections 2023-10-03T10:19:39.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.481+0000 7fda66e2b700 1 --2- 172.21.15.3:0/1387670692 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda60139170 0x7fda6013b550 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.481+0000 7fda66e2b700 1 --2- 172.21.15.3:0/1387670692 >> v2:172.21.15.3:3300/0 conn(0x7fda6009e1d0 0x7fda60138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.481+0000 7fda66e2b700 1 -- 172.21.15.3:0/1387670692 >> 172.21.15.3:0/1387670692 conn(0x7fda60066f50 msgr2=0x7fda6009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:39.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.481+0000 7fda66e2b700 1 -- 172.21.15.3:0/1387670692 shutdown_connections 2023-10-03T10:19:39.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.481+0000 7fda66e2b700 1 -- 172.21.15.3:0/1387670692 wait complete. 2023-10-03T10:19:39.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.483+0000 7fda66e2b700 1 Processor -- start 2023-10-03T10:19:39.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.483+0000 7fda66e2b700 1 -- start start 2023-10-03T10:19:39.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.484+0000 7fda66e2b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fda6009d6f0 0x7fda601c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:39.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.484+0000 7fda66e2b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda6009e1d0 0x7fda601c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:39.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.484+0000 7fda66e2b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda60139170 0x7fda6011e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:39.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.484+0000 7fda66e2b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda601220c0 con 0x7fda60139170 2023-10-03T10:19:39.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.484+0000 7fda66e2b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda601203b0 con 0x7fda6009e1d0 2023-10-03T10:19:39.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.484+0000 7fda66e2b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fda60134670 con 0x7fda6009d6f0 2023-10-03T10:19:39.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.484+0000 7fda653c8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda60139170 0x7fda6011e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:39.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.484+0000 7fda5ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda6009e1d0 0x7fda601c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:39.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.484+0000 7fda653c8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda60139170 0x7fda6011e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37772/0 (socket says 172.21.15.3:37772) 2023-10-03T10:19:39.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.484+0000 7fda653c8700 1 -- 172.21.15.3:0/1087476122 learned_addr learned my addr 172.21.15.3:0/1087476122 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:39.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.485+0000 7fda5ffff700 1 -- 172.21.15.3:0/1087476122 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fda6009d6f0 msgr2=0x7fda601c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:39.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.485+0000 7fda5ffff700 1 --2- 172.21.15.3:0/1087476122 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fda6009d6f0 0x7fda601c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.485+0000 7fda5ffff700 1 -- 172.21.15.3:0/1087476122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda60139170 msgr2=0x7fda6011e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:39.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.485+0000 7fda5ffff700 1 --2- 172.21.15.3:0/1087476122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda60139170 0x7fda6011e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.485+0000 7fda5ffff700 1 -- 172.21.15.3:0/1087476122 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda58009b30 con 0x7fda6009e1d0 2023-10-03T10:19:39.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.485+0000 7fda5ffff700 1 --2- 172.21.15.3:0/1087476122 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda6009e1d0 0x7fda601c8bb0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00e900 tx=0x7fda4c00ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:39.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.485+0000 7fda5d7fa700 1 -- 172.21.15.3:0/1087476122 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c00ee50 con 0x7fda6009e1d0 2023-10-03T10:19:39.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.485+0000 7fda5d7fa700 1 -- 172.21.15.3:0/1087476122 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4c00c850 con 0x7fda6009e1d0 2023-10-03T10:19:39.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.485+0000 7fda5d7fa700 1 -- 172.21.15.3:0/1087476122 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c00cbc0 con 0x7fda6009e1d0 2023-10-03T10:19:39.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.485+0000 7fda66e2b700 1 -- 172.21.15.3:0/1087476122 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda600726b0 con 0x7fda6009e1d0 2023-10-03T10:19:39.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.485+0000 7fda66e2b700 1 -- 172.21.15.3:0/1087476122 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda601c90f0 con 0x7fda6009e1d0 2023-10-03T10:19:39.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.486+0000 7fda5d7fa700 1 -- 172.21.15.3:0/1087476122 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fda4c00cd20 con 0x7fda6009e1d0 2023-10-03T10:19:39.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.486+0000 7fda5d7fa700 1 -- 172.21.15.3:0/1087476122 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fda4c03fbb0 con 0x7fda6009e1d0 2023-10-03T10:19:39.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.489+0000 7fda66e2b700 1 -- 172.21.15.3:0/1087476122 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda60074ea0 con 0x7fda6009e1d0 2023-10-03T10:19:39.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.494+0000 7fda5d7fa700 1 -- 172.21.15.3:0/1087476122 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fda4c021080 con 0x7fda6009e1d0 2023-10-03T10:19:39.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.514+0000 7fc84b9aa700 1 Processor -- start 2023-10-03T10:19:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.514+0000 7fc84b9aa700 1 -- start start 2023-10-03T10:19:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.514+0000 7fc84b9aa700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc84409e930 0x7fc84409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.515+0000 7fc84b9aa700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc84409f240 0x7fc8441317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.515+0000 7fc84b9aa700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc84409f650 0x7fc844131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.515+0000 7fc84b9aa700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc844070940 con 0x7fc84409f650 2023-10-03T10:19:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.515+0000 7fc84b9aa700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc84406fd60 con 0x7fc84409e930 2023-10-03T10:19:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.515+0000 7fc84b9aa700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc84406fa80 con 0x7fc84409f240 2023-10-03T10:19:39.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.515+0000 7fc849746700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc84409e930 0x7fc84409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:39.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.515+0000 7fc849746700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc84409e930 0x7fc84409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41168/0 (socket says 172.21.15.3:41168) 2023-10-03T10:19:39.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.515+0000 7fc849746700 1 -- 172.21.15.3:0/1527609301 learned_addr learned my addr 172.21.15.3:0/1527609301 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:39.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.515+0000 7fc849f47700 1 --2- 172.21.15.3:0/1527609301 >> v2:172.21.15.3:3300/0 conn(0x7fc84409f650 0x7fc844131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:39.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.515+0000 7fc849746700 1 -- 172.21.15.3:0/1527609301 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc84409f240 msgr2=0x7fc8441317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:39.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.515+0000 7fc849746700 1 --2- 172.21.15.3:0/1527609301 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc84409f240 0x7fc8441317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.515+0000 7fc849746700 1 -- 172.21.15.3:0/1527609301 >> v2:172.21.15.3:3300/0 conn(0x7fc84409f650 msgr2=0x7fc844131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:39.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.515+0000 7fc849746700 1 --2- 172.21.15.3:0/1527609301 >> v2:172.21.15.3:3300/0 conn(0x7fc84409f650 0x7fc844131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.516+0000 7fc849746700 1 -- 172.21.15.3:0/1527609301 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc84409fb60 con 0x7fc84409e930 2023-10-03T10:19:39.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.516+0000 7fc849f47700 1 --2- 172.21.15.3:0/1527609301 >> v2:172.21.15.3:3300/0 conn(0x7fc84409f650 0x7fc844131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:39.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.516+0000 7fc849746700 1 --2- 172.21.15.3:0/1527609301 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc84409e930 0x7fc84409ed00 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fc8400098f0 tx=0x7fc840009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8747ef236a2ca112 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:39.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.516+0000 7fc83bfff700 1 -- 172.21.15.3:0/1527609301 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84000b8a0 con 0x7fc84409e930 2023-10-03T10:19:39.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.516+0000 7fc83bfff700 1 -- 172.21.15.3:0/1527609301 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84000ba00 con 0x7fc84409e930 2023-10-03T10:19:39.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.516+0000 7fc83bfff700 1 -- 172.21.15.3:0/1527609301 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84000bd90 con 0x7fc84409e930 2023-10-03T10:19:39.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.517+0000 7fc84b9aa700 1 -- 172.21.15.3:0/1527609301 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc84409e930 msgr2=0x7fc84409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:39.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.517+0000 7fc84b9aa700 1 --2- 172.21.15.3:0/1527609301 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc84409e930 0x7fc84409ed00 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fc8400098f0 tx=0x7fc840009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:39.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.517+0000 7fc84b9aa700 1 -- 172.21.15.3:0/1527609301 shutdown_connections 2023-10-03T10:19:39.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.517+0000 7fc84b9aa700 1 --2- 172.21.15.3:0/1527609301 >> v2:172.21.15.3:3300/0 conn(0x7fc84409f650 0x7fc844131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.517+0000 7fc84b9aa700 1 --2- 172.21.15.3:0/1527609301 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc84409e930 0x7fc84409ed00 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.517+0000 7fc84b9aa700 1 --2- 172.21.15.3:0/1527609301 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc84409f240 0x7fc8441317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.517+0000 7fc84b9aa700 1 -- 172.21.15.3:0/1527609301 >> 172.21.15.3:0/1527609301 conn(0x7fc8440a4210 msgr2=0x7fc8440a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:39.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.517+0000 7fc84b9aa700 1 -- 172.21.15.3:0/1527609301 shutdown_connections 2023-10-03T10:19:39.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.517+0000 7fc84b9aa700 1 -- 172.21.15.3:0/1527609301 wait complete. 2023-10-03T10:19:39.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.518+0000 7fc84b9aa700 1 Processor -- start 2023-10-03T10:19:39.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.518+0000 7fc84b9aa700 1 -- start start 2023-10-03T10:19:39.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.519+0000 7fc84b9aa700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc84409e930 0x7fc84406b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:39.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.519+0000 7fc84b9aa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc84409f240 0x7fc84406bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:39.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.519+0000 7fc84b9aa700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc84409f650 0x7fc84406c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:39.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.519+0000 7fc84b9aa700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8441a7f40 con 0x7fc84409f240 2023-10-03T10:19:39.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.519+0000 7fc84b9aa700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc844071d20 con 0x7fc84409e930 2023-10-03T10:19:39.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.519+0000 7fc84b9aa700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8440a36e0 con 0x7fc84409f650 2023-10-03T10:19:39.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.519+0000 7fc849746700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc84409e930 0x7fc84406b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:39.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.519+0000 7fc849746700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc84409e930 0x7fc84406b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41174/0 (socket says 172.21.15.3:41174) 2023-10-03T10:19:39.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.519+0000 7fc848f45700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc84409f240 0x7fc84406bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.519+0000 7fc849746700 1 -- 172.21.15.3:0/2726845952 learned_addr learned my addr 172.21.15.3:0/2726845952 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.519+0000 7fc849f47700 1 --2- 172.21.15.3:0/2726845952 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc84409f650 0x7fc84406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.520+0000 7fc848f45700 1 -- 172.21.15.3:0/2726845952 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc84409f650 msgr2=0x7fc84406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.520+0000 7fc848f45700 1 --2- 172.21.15.3:0/2726845952 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc84409f650 0x7fc84406c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.520+0000 7fc848f45700 1 -- 172.21.15.3:0/2726845952 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc84409e930 msgr2=0x7fc84406b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.520+0000 7fc848f45700 1 --2- 172.21.15.3:0/2726845952 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc84409e930 0x7fc84406b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.520+0000 7fc848f45700 1 -- 172.21.15.3:0/2726845952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc840009550 con 0x7fc84409f240 2023-10-03T10:19:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.520+0000 7fc849746700 1 --2- 172.21.15.3:0/2726845952 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc84409e930 0x7fc84406b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.520+0000 7fc848f45700 1 --2- 172.21.15.3:0/2726845952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc84409f240 0x7fc84406bdf0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fc84406e6c0 tx=0x7fc83400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.520+0000 7fc839ffb700 1 -- 172.21.15.3:0/2726845952 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc834011830 con 0x7fc84409f240 2023-10-03T10:19:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.520+0000 7fc839ffb700 1 -- 172.21.15.3:0/2726845952 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc834011990 con 0x7fc84409f240 2023-10-03T10:19:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.520+0000 7fc84b9aa700 1 -- 172.21.15.3:0/2726845952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8440a0190 con 0x7fc84409f240 2023-10-03T10:19:39.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.520+0000 7fc839ffb700 1 -- 172.21.15.3:0/2726845952 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc834011d40 con 0x7fc84409f240 2023-10-03T10:19:39.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.520+0000 7fc84b9aa700 1 -- 172.21.15.3:0/2726845952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8441a82b0 con 0x7fc84409f240 2023-10-03T10:19:39.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.521+0000 7fc839ffb700 1 -- 172.21.15.3:0/2726845952 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc83401a400 con 0x7fc84409f240 2023-10-03T10:19:39.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.521+0000 7fc84b9aa700 1 -- 172.21.15.3:0/2726845952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc84409b430 con 0x7fc84409f240 2023-10-03T10:19:39.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.522+0000 7fc839ffb700 1 -- 172.21.15.3:0/2726845952 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc83403f600 con 0x7fc84409f240 2023-10-03T10:19:39.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.527+0000 7fc839ffb700 1 -- 172.21.15.3:0/2726845952 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc83400be30 con 0x7fc84409f240 2023-10-03T10:19:39.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.558+0000 7f79b050e700 1 Processor -- start 2023-10-03T10:19:39.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.558+0000 7f79b050e700 1 -- start start 2023-10-03T10:19:39.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.559+0000 7f79b050e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79a81337f0 0x7f79a8138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:39.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.559+0000 7f79b050e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f79a8138dc0 0x7f79a813b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:39.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.559+0000 7f79b050e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79a806b990 con 0x7f79a8133420 2023-10-03T10:19:39.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.559+0000 7f79b050e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79a809b0e0 con 0x7f79a8138dc0 2023-10-03T10:19:39.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.559+0000 7f79b050e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a8133c20 con 0x7f79a81337f0 2023-10-03T10:19:39.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.559+0000 7f79ae2aa700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f79a8133420 0x7f79a8133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47482/0 (socket says 172.21.15.3:47482) 2023-10-03T10:19:39.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.559+0000 7f79ae2aa700 1 -- 172.21.15.3:0/476105248 learned_addr learned my addr 172.21.15.3:0/476105248 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:39.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.559+0000 7f79aeaab700 1 --2- 172.21.15.3:0/476105248 >> v2:172.21.15.3:3300/0 conn(0x7f79a8138dc0 0x7f79a813b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:39.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.559+0000 7f79adaa9700 1 --2- 172.21.15.3:0/476105248 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79a81337f0 0x7f79a8138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:39.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.559+0000 7f79ad2a8700 1 -- 172.21.15.3:0/476105248 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 969085972 0 0) 0x7f79a806b990 con 0x7f79a8133420 2023-10-03T10:19:39.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.559+0000 7f79ad2a8700 1 -- 172.21.15.3:0/476105248 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79940035b0 con 0x7f79a8133420 2023-10-03T10:19:39.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.559+0000 7f79aeaab700 1 -- 172.21.15.3:0/476105248 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79a81337f0 msgr2=0x7f79a8138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:39.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.559+0000 7f79aeaab700 1 --2- 172.21.15.3:0/476105248 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79a81337f0 0x7f79a8138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.560+0000 7f79aeaab700 1 -- 172.21.15.3:0/476105248 >> v1:172.21.15.3:6789/0 conn(0x7f79a8133420 legacy=0x7f79a8133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:39.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.560+0000 7f79aeaab700 1 -- 172.21.15.3:0/476105248 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a80a0590 con 0x7f79a8138dc0 2023-10-03T10:19:39.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.560+0000 7f79aeaab700 1 --2- 172.21.15.3:0/476105248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79a8138dc0 0x7f79a813b1a0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f79a0009d90 tx=0x7f79a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecde6f594b4a7fd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:39.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.560+0000 7f79ad2a8700 1 -- 172.21.15.3:0/476105248 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a000e070 con 0x7f79a8138dc0 2023-10-03T10:19:39.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.560+0000 7f79ad2a8700 1 -- 172.21.15.3:0/476105248 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a000bb20 con 0x7f79a8138dc0 2023-10-03T10:19:39.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.560+0000 7f79ad2a8700 1 -- 172.21.15.3:0/476105248 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0012430 con 0x7f79a8138dc0 2023-10-03T10:19:39.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.561+0000 7f79b050e700 1 -- 172.21.15.3:0/476105248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79a8138dc0 msgr2=0x7f79a813b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:39.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.561+0000 7f79b050e700 1 --2- 172.21.15.3:0/476105248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79a8138dc0 0x7f79a813b1a0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f79a0009d90 tx=0x7f79a000b330 comp rx=0 tx=0).stop 2023-10-03T10:19:39.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.561+0000 7f79b050e700 1 -- 172.21.15.3:0/476105248 shutdown_connections 2023-10-03T10:19:39.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.561+0000 7f79b050e700 1 --2- 172.21.15.3:0/476105248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79a8138dc0 0x7f79a813b1a0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.561+0000 7f79b050e700 1 --2- 172.21.15.3:0/476105248 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79a81337f0 0x7f79a8138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.561+0000 7f79b050e700 1 -- 172.21.15.3:0/476105248 >> 172.21.15.3:0/476105248 conn(0x7f79a80a4210 msgr2=0x7f79a80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:39.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.562+0000 7f79b050e700 1 -- 172.21.15.3:0/476105248 shutdown_connections 2023-10-03T10:19:39.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.562+0000 7f79b050e700 1 -- 172.21.15.3:0/476105248 wait complete. 2023-10-03T10:19:39.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.563+0000 7f79b050e700 1 Processor -- start 2023-10-03T10:19:39.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.563+0000 7f79b050e700 1 -- start start 2023-10-03T10:19:39.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.563+0000 7f79b050e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79a8133420 0x7f79a80709a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:39.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.563+0000 7f79b050e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79a81337f0 0x7f79a8070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:39.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.563+0000 7f79b050e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79a8138dc0 0x7f79a8071420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:39.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.563+0000 7f79b050e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a809b890 con 0x7f79a8133420 2023-10-03T10:19:39.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.563+0000 7f79b050e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a81388a0 con 0x7f79a8138dc0 2023-10-03T10:19:39.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.563+0000 7f79b050e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79a809ba20 con 0x7f79a81337f0 2023-10-03T10:19:39.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.563+0000 7f79ae2aa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79a8133420 0x7f79a80709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:39.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.563+0000 7f79adaa9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79a81337f0 0x7f79a8070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:39.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.563+0000 7f79ae2aa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79a8133420 0x7f79a80709a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37800/0 (socket says 172.21.15.3:37800) 2023-10-03T10:19:39.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.563+0000 7f79aeaab700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79a8138dc0 0x7f79a8071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:39.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.563+0000 7f79ae2aa700 1 -- 172.21.15.3:0/1198684314 learned_addr learned my addr 172.21.15.3:0/1198684314 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:39.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.564+0000 7f79adaa9700 1 -- 172.21.15.3:0/1198684314 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79a8138dc0 msgr2=0x7f79a8071420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:39.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.564+0000 7f79adaa9700 1 --2- 172.21.15.3:0/1198684314 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79a8138dc0 0x7f79a8071420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.564+0000 7f79adaa9700 1 -- 172.21.15.3:0/1198684314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79a8133420 msgr2=0x7f79a80709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:39.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.564+0000 7f79adaa9700 1 --2- 172.21.15.3:0/1198684314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79a8133420 0x7f79a80709a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.564+0000 7f79adaa9700 1 -- 172.21.15.3:0/1198684314 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a0009a70 con 0x7f79a81337f0 2023-10-03T10:19:39.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.564+0000 7f79ae2aa700 1 --2- 172.21.15.3:0/1198684314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79a8133420 0x7f79a80709a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:39.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.564+0000 7f79adaa9700 1 --2- 172.21.15.3:0/1198684314 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79a81337f0 0x7f79a8070ee0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f799800e7e0 tx=0x7f799800eaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:39.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.564+0000 7f799effd700 1 -- 172.21.15.3:0/1198684314 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799800c070 con 0x7f79a81337f0 2023-10-03T10:19:39.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.564+0000 7f799effd700 1 -- 172.21.15.3:0/1198684314 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7998013610 con 0x7f79a81337f0 2023-10-03T10:19:39.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.564+0000 7f799effd700 1 -- 172.21.15.3:0/1198684314 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7998013980 con 0x7f79a81337f0 2023-10-03T10:19:39.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.564+0000 7f79b050e700 1 -- 172.21.15.3:0/1198684314 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a80724a0 con 0x7f79a81337f0 2023-10-03T10:19:39.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.564+0000 7f79b050e700 1 -- 172.21.15.3:0/1198684314 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a809c9e0 con 0x7f79a81337f0 2023-10-03T10:19:39.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.565+0000 7f799effd700 1 -- 172.21.15.3:0/1198684314 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7998013ae0 con 0x7f79a81337f0 2023-10-03T10:19:39.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.565+0000 7f79b050e700 1 -- 172.21.15.3:0/1198684314 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a8074ea0 con 0x7f79a81337f0 2023-10-03T10:19:39.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.566+0000 7f799effd700 1 -- 172.21.15.3:0/1198684314 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f799803f380 con 0x7f79a81337f0 2023-10-03T10:19:39.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.568+0000 7f509bc62700 1 Processor -- start 2023-10-03T10:19:39.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.569+0000 7f509bc62700 1 -- start start 2023-10-03T10:19:39.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.569+0000 7f509bc62700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f50941338a0 0x7f5094133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:39.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.569+0000 7f509bc62700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f509409de50 0x7f509409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:39.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.569+0000 7f509bc62700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f509406ef30 con 0x7f509409e760 2023-10-03T10:19:39.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.569+0000 7f509bc62700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f509406fef0 con 0x7f50941338a0 2023-10-03T10:19:39.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.569+0000 7f509bc62700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f50941334b0 con 0x7f509409de50 2023-10-03T10:19:39.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.569+0000 7f50991fd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f509409de50 0x7f509409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:39.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.569+0000 7f50991fd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f509409de50 0x7f509409e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48362/0 (socket says 172.21.15.3:48362) 2023-10-03T10:19:39.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.569+0000 7f50991fd700 1 -- 172.21.15.3:0/36537643 learned_addr learned my addr 172.21.15.3:0/36537643 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:39.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.569+0000 7f509a1ff700 1 --1- 172.21.15.3:0/36537643 >> v1:172.21.15.3:6789/0 conn(0x7f509409e760 0x7f509413a730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47492/0 (socket says 172.21.15.3:47492) 2023-10-03T10:19:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.569+0000 7f50999fe700 1 --2- 172.21.15.3:0/36537643 >> v2:172.21.15.3:3300/0 conn(0x7f50941338a0 0x7f5094133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.570+0000 7f50991fd700 1 -- 172.21.15.3:0/36537643 >> v2:172.21.15.3:3300/0 conn(0x7f50941338a0 msgr2=0x7f5094133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.570+0000 7f50991fd700 1 --2- 172.21.15.3:0/36537643 >> v2:172.21.15.3:3300/0 conn(0x7f50941338a0 0x7f5094133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.570+0000 7f50991fd700 1 -- 172.21.15.3:0/36537643 >> v1:172.21.15.3:6789/0 conn(0x7f509409e760 legacy=0x7f509413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.570+0000 7f50991fd700 1 -- 172.21.15.3:0/36537643 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509406fc30 con 0x7f509409de50 2023-10-03T10:19:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.570+0000 7f50991fd700 1 --2- 172.21.15.3:0/36537643 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f509409de50 0x7f509409e220 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f50840098f0 tx=0x7f5084009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b04e80292261ea7 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.575+0000 7f50989fc700 1 -- 172.21.15.3:0/36537643 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084011830 con 0x7f509409de50 2023-10-03T10:19:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.575+0000 7f50989fc700 1 -- 172.21.15.3:0/36537643 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5084011990 con 0x7f509409de50 2023-10-03T10:19:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.575+0000 7f50989fc700 1 -- 172.21.15.3:0/36537643 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084011d00 con 0x7f509409de50 2023-10-03T10:19:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.575+0000 7f799effd700 1 -- 172.21.15.3:0/1198684314 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7998010900 con 0x7f79a81337f0 2023-10-03T10:19:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.576+0000 7f509bc62700 1 -- 172.21.15.3:0/36537643 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f509409de50 msgr2=0x7f509409e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.576+0000 7f509bc62700 1 --2- 172.21.15.3:0/36537643 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f509409de50 0x7f509409e220 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f50840098f0 tx=0x7f5084009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.576+0000 7f509bc62700 1 -- 172.21.15.3:0/36537643 shutdown_connections 2023-10-03T10:19:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.576+0000 7f509bc62700 1 --2- 172.21.15.3:0/36537643 >> v2:172.21.15.3:3300/0 conn(0x7f50941338a0 0x7f5094133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.576+0000 7f509bc62700 1 --2- 172.21.15.3:0/36537643 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f509409de50 0x7f509409e220 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.576+0000 7f509bc62700 1 -- 172.21.15.3:0/36537643 >> 172.21.15.3:0/36537643 conn(0x7f50940a2aa0 msgr2=0x7f50940a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.576+0000 7f509bc62700 1 -- 172.21.15.3:0/36537643 shutdown_connections 2023-10-03T10:19:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.576+0000 7f509bc62700 1 -- 172.21.15.3:0/36537643 wait complete. 2023-10-03T10:19:39.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.578+0000 7f509bc62700 1 Processor -- start 2023-10-03T10:19:39.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.578+0000 7f509bc62700 1 -- start start 2023-10-03T10:19:39.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.579+0000 7f509bc62700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f509409de50 0x7f509411d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:39.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.579+0000 7f509bc62700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f509409e760 0x7f509411de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:39.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.579+0000 7f509bc62700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50941338a0 0x7f509411e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:39.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.579+0000 7f509bc62700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5094121400 con 0x7f50941338a0 2023-10-03T10:19:39.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.579+0000 7f509bc62700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509406f6b0 con 0x7f509409e760 2023-10-03T10:19:39.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.579+0000 7f509bc62700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f50940a0fd0 con 0x7f509409de50 2023-10-03T10:19:39.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.579+0000 7f509a1ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50941338a0 0x7f509411e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:39.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.579+0000 7f509a1ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50941338a0 0x7f509411e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37816/0 (socket says 172.21.15.3:37816) 2023-10-03T10:19:39.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.579+0000 7f509a1ff700 1 -- 172.21.15.3:0/1526461875 learned_addr learned my addr 172.21.15.3:0/1526461875 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:39.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.579+0000 7f50999fe700 1 --2- 172.21.15.3:0/1526461875 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f509409de50 0x7f509411d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:39.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.579+0000 7f50991fd700 1 --2- 172.21.15.3:0/1526461875 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f509409e760 0x7f509411de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:39.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.579+0000 7f509a1ff700 1 -- 172.21.15.3:0/1526461875 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f509409de50 msgr2=0x7f509411d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:39.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.579+0000 7f509a1ff700 1 --2- 172.21.15.3:0/1526461875 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f509409de50 0x7f509411d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.579+0000 7f509a1ff700 1 -- 172.21.15.3:0/1526461875 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f509409e760 msgr2=0x7f509411de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:39.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.579+0000 7f509a1ff700 1 --2- 172.21.15.3:0/1526461875 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f509409e760 0x7f509411de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.579+0000 7f509a1ff700 1 -- 172.21.15.3:0/1526461875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5084009550 con 0x7f50941338a0 2023-10-03T10:19:39.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.579+0000 7f509a1ff700 1 --2- 172.21.15.3:0/1526461875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50941338a0 0x7f509411e360 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f508c00cbd0 tx=0x7f508c00cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:39.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.580+0000 7f508a7fc700 1 -- 172.21.15.3:0/1526461875 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c00aa40 con 0x7f50941338a0 2023-10-03T10:19:39.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.580+0000 7f508a7fc700 1 -- 172.21.15.3:0/1526461875 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508c00aba0 con 0x7f50941338a0 2023-10-03T10:19:39.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.580+0000 7f509bc62700 1 -- 172.21.15.3:0/1526461875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f509411fd50 con 0x7f50941338a0 2023-10-03T10:19:39.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.580+0000 7f508a7fc700 1 -- 172.21.15.3:0/1526461875 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c007070 con 0x7f50941338a0 2023-10-03T10:19:39.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.580+0000 7f509bc62700 1 -- 172.21.15.3:0/1526461875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5094120b90 con 0x7f50941338a0 2023-10-03T10:19:39.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.581+0000 7f509bc62700 1 -- 172.21.15.3:0/1526461875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5094120740 con 0x7f50941338a0 2023-10-03T10:19:39.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.581+0000 7f508a7fc700 1 -- 172.21.15.3:0/1526461875 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f508c0071d0 con 0x7f50941338a0 2023-10-03T10:19:39.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.582+0000 7f508a7fc700 1 -- 172.21.15.3:0/1526461875 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f508c0117a0 con 0x7f50941338a0 2023-10-03T10:19:39.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.586+0000 7f508a7fc700 1 -- 172.21.15.3:0/1526461875 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f508c0108f0 con 0x7f50941338a0 2023-10-03T10:19:39.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.713+0000 7fda66e2b700 1 -- 172.21.15.3:0/1087476122 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fda6009f420 con 0x7fda6009e1d0 2023-10-03T10:19:39.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.714+0000 7fda5d7fa700 1 -- 172.21.15.3:0/1087476122 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fda4c017020 con 0x7fda6009e1d0 2023-10-03T10:19:39.714 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:39.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.716+0000 7fda66e2b700 1 -- 172.21.15.3:0/1087476122 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda6009e1d0 msgr2=0x7fda601c8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:39.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.716+0000 7fda66e2b700 1 --2- 172.21.15.3:0/1087476122 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda6009e1d0 0x7fda601c8bb0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00e900 tx=0x7fda4c00ec10 comp rx=0 tx=0).stop 2023-10-03T10:19:39.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.716+0000 7fda66e2b700 1 -- 172.21.15.3:0/1087476122 shutdown_connections 2023-10-03T10:19:39.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.716+0000 7fda66e2b700 1 --2- 172.21.15.3:0/1087476122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda60139170 0x7fda6011e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.716+0000 7fda66e2b700 1 --2- 172.21.15.3:0/1087476122 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fda6009d6f0 0x7fda601c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.716+0000 7fda66e2b700 1 --2- 172.21.15.3:0/1087476122 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fda6009e1d0 0x7fda601c8bb0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.716+0000 7fda66e2b700 1 -- 172.21.15.3:0/1087476122 >> 172.21.15.3:0/1087476122 conn(0x7fda60066f50 msgr2=0x7fda600a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:39.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.717+0000 7fda66e2b700 1 -- 172.21.15.3:0/1087476122 shutdown_connections 2023-10-03T10:19:39.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.717+0000 7fda66e2b700 1 -- 172.21.15.3:0/1087476122 wait complete. 2023-10-03T10:19:39.731 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:39.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.742+0000 7fc84b9aa700 1 -- 172.21.15.3:0/2726845952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc844074ea0 con 0x7fc84409f240 2023-10-03T10:19:39.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.742+0000 7fc839ffb700 1 -- 172.21.15.3:0/2726845952 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc834015020 con 0x7fc84409f240 2023-10-03T10:19:39.742 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:39.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.745+0000 7fc84b9aa700 1 -- 172.21.15.3:0/2726845952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc84409f240 msgr2=0x7fc84406bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:39.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.745+0000 7fc84b9aa700 1 --2- 172.21.15.3:0/2726845952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc84409f240 0x7fc84406bdf0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fc84406e6c0 tx=0x7fc83400b730 comp rx=0 tx=0).stop 2023-10-03T10:19:39.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.745+0000 7fc84b9aa700 1 -- 172.21.15.3:0/2726845952 shutdown_connections 2023-10-03T10:19:39.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.745+0000 7fc84b9aa700 1 --2- 172.21.15.3:0/2726845952 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc84409f650 0x7fc84406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.745+0000 7fc84b9aa700 1 --2- 172.21.15.3:0/2726845952 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc84409e930 0x7fc84406b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.745+0000 7fc84b9aa700 1 --2- 172.21.15.3:0/2726845952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc84409f240 0x7fc84406bdf0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.745+0000 7fc84b9aa700 1 -- 172.21.15.3:0/2726845952 >> 172.21.15.3:0/2726845952 conn(0x7fc8440a4210 msgr2=0x7fc8441b8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:39.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.745+0000 7fc84b9aa700 1 -- 172.21.15.3:0/2726845952 shutdown_connections 2023-10-03T10:19:39.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.745+0000 7fc84b9aa700 1 -- 172.21.15.3:0/2726845952 wait complete. 2023-10-03T10:19:39.759 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:39.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.783+0000 7f79b050e700 1 -- 172.21.15.3:0/1198684314 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f79a806b250 con 0x7f79a81337f0 2023-10-03T10:19:39.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.784+0000 7f799effd700 1 -- 172.21.15.3:0/1198684314 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7998010310 con 0x7f79a81337f0 2023-10-03T10:19:39.784 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:39.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.786+0000 7f79b050e700 1 -- 172.21.15.3:0/1198684314 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79a81337f0 msgr2=0x7f79a8070ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:39.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.786+0000 7f79b050e700 1 --2- 172.21.15.3:0/1198684314 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79a81337f0 0x7f79a8070ee0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f799800e7e0 tx=0x7f799800eaf0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.786+0000 7f79b050e700 1 -- 172.21.15.3:0/1198684314 shutdown_connections 2023-10-03T10:19:39.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.786+0000 7f79b050e700 1 --2- 172.21.15.3:0/1198684314 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f79a8138dc0 0x7f79a8071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.786+0000 7f79b050e700 1 --2- 172.21.15.3:0/1198684314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79a8133420 0x7f79a80709a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.786+0000 7f79b050e700 1 --2- 172.21.15.3:0/1198684314 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f79a81337f0 0x7f79a8070ee0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.786+0000 7f79b050e700 1 -- 172.21.15.3:0/1198684314 >> 172.21.15.3:0/1198684314 conn(0x7f79a80a4210 msgr2=0x7f79a80a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:39.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.786+0000 7f79b050e700 1 -- 172.21.15.3:0/1198684314 shutdown_connections 2023-10-03T10:19:39.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.786+0000 7f79b050e700 1 -- 172.21.15.3:0/1198684314 wait complete. 2023-10-03T10:19:39.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.797+0000 7f509bc62700 1 -- 172.21.15.3:0/1526461875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5094134180 con 0x7f50941338a0 2023-10-03T10:19:39.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.798+0000 7f508a7fc700 1 -- 172.21.15.3:0/1526461875 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f508c017030 con 0x7f50941338a0 2023-10-03T10:19:39.799 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:39.800 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:39.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.800+0000 7f509bc62700 1 -- 172.21.15.3:0/1526461875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50941338a0 msgr2=0x7f509411e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:39.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.800+0000 7f509bc62700 1 --2- 172.21.15.3:0/1526461875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50941338a0 0x7f509411e360 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f508c00cbd0 tx=0x7f508c00cee0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.800+0000 7f509bc62700 1 -- 172.21.15.3:0/1526461875 shutdown_connections 2023-10-03T10:19:39.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.800+0000 7f509bc62700 1 --2- 172.21.15.3:0/1526461875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50941338a0 0x7f509411e360 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.800+0000 7f509bc62700 1 --2- 172.21.15.3:0/1526461875 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f509409de50 0x7f509411d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.800+0000 7f509bc62700 1 --2- 172.21.15.3:0/1526461875 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f509409e760 0x7f509411de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:39.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.800+0000 7f509bc62700 1 -- 172.21.15.3:0/1526461875 >> 172.21.15.3:0/1526461875 conn(0x7f50940a2aa0 msgr2=0x7f50941bae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:39.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.800+0000 7f509bc62700 1 -- 172.21.15.3:0/1526461875 shutdown_connections 2023-10-03T10:19:39.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:39.800+0000 7f509bc62700 1 -- 172.21.15.3:0/1526461875 wait complete. 2023-10-03T10:19:39.814 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:40.732 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:40.760 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:40.801 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:40.815 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:40.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.897+0000 7fb939af9700 1 Processor -- start 2023-10-03T10:19:40.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.897+0000 7fb939af9700 1 -- start start 2023-10-03T10:19:40.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.898+0000 7fb939af9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb93409d6f0 0x7fb93409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:40.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.898+0000 7fb939af9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb93409e410 0x7fb934130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:40.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.898+0000 7fb939af9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb93409f130 con 0x7fb93409e000 2023-10-03T10:19:40.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.898+0000 7fb939af9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb93409f520 con 0x7fb93409e410 2023-10-03T10:19:40.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.898+0000 7fb939af9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb93409e930 con 0x7fb93409d6f0 2023-10-03T10:19:40.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.898+0000 7fb9337fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb93409d6f0 0x7fb93409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:40.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.899+0000 7fb9337fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb93409d6f0 0x7fb93409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41222/0 (socket says 172.21.15.3:41222) 2023-10-03T10:19:40.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.899+0000 7fb9337fe700 1 -- 172.21.15.3:0/2465040368 learned_addr learned my addr 172.21.15.3:0/2465040368 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:40.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.899+0000 7fb933fff700 1 --2- 172.21.15.3:0/2465040368 >> v2:172.21.15.3:3300/0 conn(0x7fb93409e410 0x7fb934130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:40.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.899+0000 7fb9337fe700 1 -- 172.21.15.3:0/2465040368 >> v2:172.21.15.3:3300/0 conn(0x7fb93409e410 msgr2=0x7fb934130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:40.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.899+0000 7fb9337fe700 1 --2- 172.21.15.3:0/2465040368 >> v2:172.21.15.3:3300/0 conn(0x7fb93409e410 0x7fb934130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:40.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.899+0000 7fb9337fe700 1 -- 172.21.15.3:0/2465040368 >> v1:172.21.15.3:6789/0 conn(0x7fb93409e000 legacy=0x7fb934130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:40.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.899+0000 7fb9337fe700 1 -- 172.21.15.3:0/2465040368 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb93409b2d0 con 0x7fb93409d6f0 2023-10-03T10:19:40.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.899+0000 7fb9337fe700 1 --2- 172.21.15.3:0/2465040368 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb93409d6f0 0x7fb93409dac0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fb9240098f0 tx=0x7fb924009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5b7a727a1a2e337 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:40.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.899+0000 7fb9327fc700 1 -- 172.21.15.3:0/2465040368 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb924011830 con 0x7fb93409d6f0 2023-10-03T10:19:40.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.899+0000 7fb9327fc700 1 -- 172.21.15.3:0/2465040368 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb924011990 con 0x7fb93409d6f0 2023-10-03T10:19:40.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.899+0000 7fb9327fc700 1 -- 172.21.15.3:0/2465040368 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb924011d20 con 0x7fb93409d6f0 2023-10-03T10:19:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.900+0000 7fb939af9700 1 -- 172.21.15.3:0/2465040368 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb93409d6f0 msgr2=0x7fb93409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.900+0000 7fb939af9700 1 --2- 172.21.15.3:0/2465040368 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb93409d6f0 0x7fb93409dac0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fb9240098f0 tx=0x7fb924009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.901+0000 7fb939af9700 1 -- 172.21.15.3:0/2465040368 shutdown_connections 2023-10-03T10:19:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.901+0000 7fb939af9700 1 --2- 172.21.15.3:0/2465040368 >> v2:172.21.15.3:3300/0 conn(0x7fb93409e410 0x7fb934130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.901+0000 7fb939af9700 1 --2- 172.21.15.3:0/2465040368 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb93409d6f0 0x7fb93409dac0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.901+0000 7fb939af9700 1 -- 172.21.15.3:0/2465040368 >> 172.21.15.3:0/2465040368 conn(0x7fb934066f50 msgr2=0x7fb93409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.901+0000 7fb939af9700 1 -- 172.21.15.3:0/2465040368 shutdown_connections 2023-10-03T10:19:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.901+0000 7fb939af9700 1 -- 172.21.15.3:0/2465040368 wait complete. 2023-10-03T10:19:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.902+0000 7fb939af9700 1 Processor -- start 2023-10-03T10:19:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.902+0000 7fb939af9700 1 -- start start 2023-10-03T10:19:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.902+0000 7fb939af9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb93409d6f0 0x7fb9341202f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.903+0000 7fb939af9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb93409e000 0x7fb934120830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.903+0000 7fb939af9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb93409e410 0x7fb934120d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.903+0000 7fb939af9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9341217c0 con 0x7fb93409d6f0 2023-10-03T10:19:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.903+0000 7fb939af9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb934123cd0 con 0x7fb93409e000 2023-10-03T10:19:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.903+0000 7fb939af9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb934121dc0 con 0x7fb93409e410 2023-10-03T10:19:40.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.903+0000 7fb9337fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb93409d6f0 0x7fb9341202f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:40.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.903+0000 7fb9337fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb93409d6f0 0x7fb9341202f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37834/0 (socket says 172.21.15.3:37834) 2023-10-03T10:19:40.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.903+0000 7fb9337fe700 1 -- 172.21.15.3:0/327112100 learned_addr learned my addr 172.21.15.3:0/327112100 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:40.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.903+0000 7fb932ffd700 1 --2- 172.21.15.3:0/327112100 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb93409e000 0x7fb934120830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:40.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.903+0000 7fb933fff700 1 --2- 172.21.15.3:0/327112100 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb93409e410 0x7fb934120d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:40.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.903+0000 7fb9337fe700 1 -- 172.21.15.3:0/327112100 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb93409e410 msgr2=0x7fb934120d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:40.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.903+0000 7fb9337fe700 1 --2- 172.21.15.3:0/327112100 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb93409e410 0x7fb934120d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:40.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.903+0000 7fb9337fe700 1 -- 172.21.15.3:0/327112100 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb93409e000 msgr2=0x7fb934120830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:40.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.903+0000 7fb9337fe700 1 --2- 172.21.15.3:0/327112100 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb93409e000 0x7fb934120830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:40.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.903+0000 7fb9337fe700 1 -- 172.21.15.3:0/327112100 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb924009550 con 0x7fb93409d6f0 2023-10-03T10:19:40.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.903+0000 7fb9337fe700 1 --2- 172.21.15.3:0/327112100 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb93409d6f0 0x7fb9341202f0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fb924009c40 tx=0x7fb924004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:40.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.904+0000 7fb91bfff700 1 -- 172.21.15.3:0/327112100 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb924004510 con 0x7fb93409d6f0 2023-10-03T10:19:40.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.904+0000 7fb91bfff700 1 -- 172.21.15.3:0/327112100 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb92401e070 con 0x7fb93409d6f0 2023-10-03T10:19:40.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.904+0000 7fb91bfff700 1 -- 172.21.15.3:0/327112100 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9240038c0 con 0x7fb93409d6f0 2023-10-03T10:19:40.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.904+0000 7fb939af9700 1 -- 172.21.15.3:0/327112100 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9340a2010 con 0x7fb93409d6f0 2023-10-03T10:19:40.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.904+0000 7fb939af9700 1 -- 172.21.15.3:0/327112100 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9341300d0 con 0x7fb93409d6f0 2023-10-03T10:19:40.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.905+0000 7fb91bfff700 1 -- 172.21.15.3:0/327112100 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb924003a20 con 0x7fb93409d6f0 2023-10-03T10:19:40.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.905+0000 7fb939af9700 1 -- 172.21.15.3:0/327112100 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9341223a0 con 0x7fb93409d6f0 2023-10-03T10:19:40.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.905+0000 7fb91bfff700 1 -- 172.21.15.3:0/327112100 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb92400ba20 con 0x7fb93409d6f0 2023-10-03T10:19:40.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.914+0000 7fb91bfff700 1 -- 172.21.15.3:0/327112100 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb924015030 con 0x7fb93409d6f0 2023-10-03T10:19:40.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.935+0000 7fbdcede3700 1 Processor -- start 2023-10-03T10:19:40.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.935+0000 7fbdcede3700 1 -- start start 2023-10-03T10:19:40.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.936+0000 7fbdcede3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbdc81338a0 0x7fbdc8133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:40.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.936+0000 7fbdcede3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbdc809de50 0x7fbdc809e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:40.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.936+0000 7fbdcede3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc806ef30 con 0x7fbdc809e760 2023-10-03T10:19:40.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.936+0000 7fbdcede3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc806fef0 con 0x7fbdc809de50 2023-10-03T10:19:40.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.936+0000 7fbdcede3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc81334b0 con 0x7fbdc81338a0 2023-10-03T10:19:40.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.936+0000 7fbdc7fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbdc809de50 0x7fbdc809e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:40.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.936+0000 7fbdcd380700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbdc809e760 0x7fbdc813a730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47502/0 (socket says 172.21.15.3:47502) 2023-10-03T10:19:40.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.936+0000 7fbdcd380700 1 -- 172.21.15.3:0/490112147 learned_addr learned my addr 172.21.15.3:0/490112147 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:40.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.936+0000 7fbdccb7f700 1 --2- 172.21.15.3:0/490112147 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbdc81338a0 0x7fbdc8133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:40.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.937+0000 7fbdc77fe700 1 -- 172.21.15.3:0/490112147 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 123208821 0 0) 0x7fbdc806ef30 con 0x7fbdc809e760 2023-10-03T10:19:40.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.937+0000 7fbdc77fe700 1 -- 172.21.15.3:0/490112147 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdb80035b0 con 0x7fbdc809e760 2023-10-03T10:19:40.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.937+0000 7fbdc7fff700 1 -- 172.21.15.3:0/490112147 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbdc81338a0 msgr2=0x7fbdc8133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:40.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.937+0000 7fbdc7fff700 1 --2- 172.21.15.3:0/490112147 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbdc81338a0 0x7fbdc8133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:40.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.937+0000 7fbdc7fff700 1 -- 172.21.15.3:0/490112147 >> v1:172.21.15.3:6789/0 conn(0x7fbdc809e760 legacy=0x7fbdc813a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:40.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.937+0000 7fbdc7fff700 1 -- 172.21.15.3:0/490112147 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc806fc30 con 0x7fbdc809de50 2023-10-03T10:19:40.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.937+0000 7fbdc7fff700 1 --2- 172.21.15.3:0/490112147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbdc809de50 0x7fbdc809e220 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4009840 tx=0x7fbdb4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795c2bb83f05d190 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:40.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.937+0000 7fbdc77fe700 1 -- 172.21.15.3:0/490112147 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4011830 con 0x7fbdc809de50 2023-10-03T10:19:40.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.937+0000 7fbdc77fe700 1 -- 172.21.15.3:0/490112147 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb4011990 con 0x7fbdc809de50 2023-10-03T10:19:40.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.937+0000 7fbdc77fe700 1 -- 172.21.15.3:0/490112147 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4011d00 con 0x7fbdc809de50 2023-10-03T10:19:40.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.938+0000 7fbdcede3700 1 -- 172.21.15.3:0/490112147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbdc809de50 msgr2=0x7fbdc809e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:40.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.938+0000 7fbdcede3700 1 --2- 172.21.15.3:0/490112147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbdc809de50 0x7fbdc809e220 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4009840 tx=0x7fbdb4009b50 comp rx=0 tx=0).stop 2023-10-03T10:19:40.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.938+0000 7fbdcede3700 1 -- 172.21.15.3:0/490112147 shutdown_connections 2023-10-03T10:19:40.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.938+0000 7fbdcede3700 1 --2- 172.21.15.3:0/490112147 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbdc81338a0 0x7fbdc8133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:40.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.938+0000 7fbdcede3700 1 --2- 172.21.15.3:0/490112147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbdc809de50 0x7fbdc809e220 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:40.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.938+0000 7fbdcede3700 1 -- 172.21.15.3:0/490112147 >> 172.21.15.3:0/490112147 conn(0x7fbdc80a2aa0 msgr2=0x7fbdc80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:40.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.938+0000 7fbdcede3700 1 -- 172.21.15.3:0/490112147 shutdown_connections 2023-10-03T10:19:40.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.938+0000 7fbdcede3700 1 -- 172.21.15.3:0/490112147 wait complete. 2023-10-03T10:19:40.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.940+0000 7fbdcede3700 1 Processor -- start 2023-10-03T10:19:40.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.940+0000 7fbdcede3700 1 -- start start 2023-10-03T10:19:40.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.941+0000 7fbdcede3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbdc809de50 0x7fbdc811a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:40.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.941+0000 7fbdcede3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbdc809e760 0x7fbdc811ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:40.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.941+0000 7fbdcede3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbdc81338a0 0x7fbdc811b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:40.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.941+0000 7fbdcede3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc811e0c0 con 0x7fbdc81338a0 2023-10-03T10:19:40.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.941+0000 7fbdcede3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc80a0d40 con 0x7fbdc809de50 2023-10-03T10:19:40.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.941+0000 7fbdcede3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc811d850 con 0x7fbdc809e760 2023-10-03T10:19:40.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.941+0000 7fbdccb7f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbdc809de50 0x7fbdc811a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:40.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.941+0000 7fbdcd380700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbdc81338a0 0x7fbdc811b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:40.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.941+0000 7fbdccb7f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbdc809de50 0x7fbdc811a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41228/0 (socket says 172.21.15.3:41228) 2023-10-03T10:19:40.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.941+0000 7fbdccb7f700 1 -- 172.21.15.3:0/2077922103 learned_addr learned my addr 172.21.15.3:0/2077922103 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:40.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.941+0000 7fbdc7fff700 1 --2- 172.21.15.3:0/2077922103 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbdc809e760 0x7fbdc811ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:40.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.941+0000 7fbdcd380700 1 -- 172.21.15.3:0/2077922103 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbdc809e760 msgr2=0x7fbdc811ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:40.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.941+0000 7fbdcd380700 1 --2- 172.21.15.3:0/2077922103 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbdc809e760 0x7fbdc811ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:40.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.941+0000 7fbdcd380700 1 -- 172.21.15.3:0/2077922103 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbdc809de50 msgr2=0x7fbdc811a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:40.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.941+0000 7fbdcd380700 1 --2- 172.21.15.3:0/2077922103 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbdc809de50 0x7fbdc811a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:40.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.941+0000 7fbdcd380700 1 -- 172.21.15.3:0/2077922103 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb40094a0 con 0x7fbdc81338a0 2023-10-03T10:19:40.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.942+0000 7fbdcd380700 1 --2- 172.21.15.3:0/2077922103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbdc81338a0 0x7fbdc811b150 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000ca80 tx=0x7fbdc000cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:40.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.942+0000 7fbdc57fa700 1 -- 172.21.15.3:0/2077922103 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc000a9e0 con 0x7fbdc81338a0 2023-10-03T10:19:40.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.942+0000 7fbdc57fa700 1 -- 172.21.15.3:0/2077922103 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc000ab40 con 0x7fbdc81338a0 2023-10-03T10:19:40.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.942+0000 7fbdcede3700 1 -- 172.21.15.3:0/2077922103 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc806f640 con 0x7fbdc81338a0 2023-10-03T10:19:40.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.942+0000 7fbdc57fa700 1 -- 172.21.15.3:0/2077922103 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0012400 con 0x7fbdc81338a0 2023-10-03T10:19:40.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.942+0000 7fbdcede3700 1 -- 172.21.15.3:0/2077922103 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc811d400 con 0x7fbdc81338a0 2023-10-03T10:19:40.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.943+0000 7fbdc57fa700 1 -- 172.21.15.3:0/2077922103 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbdc001b3b0 con 0x7fbdc81338a0 2023-10-03T10:19:40.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.943+0000 7fbdc57fa700 1 -- 172.21.15.3:0/2077922103 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fbdc003f2c0 con 0x7fbdc81338a0 2023-10-03T10:19:40.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.943+0000 7fbdcede3700 1 -- 172.21.15.3:0/2077922103 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc8074ea0 con 0x7fbdc81338a0 2023-10-03T10:19:40.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.952+0000 7fbdc57fa700 1 -- 172.21.15.3:0/2077922103 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbdc00192c0 con 0x7fbdc81338a0 2023-10-03T10:19:40.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.972+0000 7fb42778b700 1 Processor -- start 2023-10-03T10:19:40.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.972+0000 7fde04891700 1 Processor -- start 2023-10-03T10:19:40.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.972+0000 7fb42778b700 1 -- start start 2023-10-03T10:19:40.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.972+0000 7fde04891700 1 -- start start 2023-10-03T10:19:40.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.973+0000 7fb42778b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb42009d6f0 0x7fb42009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:40.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.973+0000 7fb42778b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb42009e000 0x7fb4201339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:40.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.973+0000 7fb42778b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb42009e410 0x7fb420133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:40.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.973+0000 7fb42778b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb42006b460 con 0x7fb42009d6f0 2023-10-03T10:19:40.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.973+0000 7fb42778b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb42009b120 con 0x7fb42009e410 2023-10-03T10:19:40.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.973+0000 7fb42778b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb42009a7e0 con 0x7fb42009e000 2023-10-03T10:19:40.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.973+0000 7fde04891700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde0009d6f0 0x7fde0009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:40.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.973+0000 7fde04891700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fde0009e000 0x7fde00130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:40.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.973+0000 7fde04891700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fde0009e410 0x7fde00130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:40.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.973+0000 7fde04891700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde0009f520 con 0x7fde0009e000 2023-10-03T10:19:40.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.973+0000 7fde04891700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0009f130 con 0x7fde0009e410 2023-10-03T10:19:40.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.973+0000 7fde04891700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde0009e930 con 0x7fde0009d6f0 2023-10-03T10:19:40.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.973+0000 7fb424d26700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb42009e000 0x7fb4201339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:40.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.973+0000 7fb424d26700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb42009e000 0x7fb4201339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48394/0 (socket says 172.21.15.3:48394) 2023-10-03T10:19:40.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.973+0000 7fb424d26700 1 -- 172.21.15.3:0/2256554468 learned_addr learned my addr 172.21.15.3:0/2256554468 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:40.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.973+0000 7fb425d28700 1 --2- 172.21.15.3:0/2256554468 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb42009e410 0x7fb420133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:40.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.973+0000 7fddfe59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde0009d6f0 0x7fde0009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:40.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.973+0000 7fddfe59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde0009d6f0 0x7fde0009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48396/0 (socket says 172.21.15.3:48396) 2023-10-03T10:19:40.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.973+0000 7fddfe59c700 1 -- 172.21.15.3:0/3029379894 learned_addr learned my addr 172.21.15.3:0/3029379894 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:40.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.974+0000 7fddfed9d700 1 --2- 172.21.15.3:0/3029379894 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fde0009e410 0x7fde00130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:40.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.974+0000 7fddfdd9b700 1 --2- 172.21.15.3:0/3029379894 >> v2:172.21.15.3:3300/0 conn(0x7fde0009e000 0x7fde00130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:40.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.974+0000 7fb424d26700 1 -- 172.21.15.3:0/2256554468 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb42009e410 msgr2=0x7fb420133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:40.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.974+0000 7fb424d26700 1 --2- 172.21.15.3:0/2256554468 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb42009e410 0x7fb420133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:40.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.974+0000 7fb424d26700 1 -- 172.21.15.3:0/2256554468 >> v2:172.21.15.3:3300/0 conn(0x7fb42009d6f0 msgr2=0x7fb42009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:40.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.974+0000 7fb424d26700 1 --2- 172.21.15.3:0/2256554468 >> v2:172.21.15.3:3300/0 conn(0x7fb42009d6f0 0x7fb42009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:40.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.974+0000 7fb424d26700 1 -- 172.21.15.3:0/2256554468 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42009a500 con 0x7fb42009e000 2023-10-03T10:19:40.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.974+0000 7fddfdd9b700 1 -- 172.21.15.3:0/3029379894 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde0009d6f0 msgr2=0x7fde0009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:40.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.974+0000 7fddfdd9b700 1 --2- 172.21.15.3:0/3029379894 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde0009d6f0 0x7fde0009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:40.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.974+0000 7fddfdd9b700 1 -- 172.21.15.3:0/3029379894 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fde0009e410 msgr2=0x7fde00130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:40.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.974+0000 7fddfdd9b700 1 --2- 172.21.15.3:0/3029379894 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fde0009e410 0x7fde00130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:40.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.974+0000 7fddfdd9b700 1 -- 172.21.15.3:0/3029379894 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0009b2d0 con 0x7fde0009e000 2023-10-03T10:19:40.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.974+0000 7fddfe59c700 1 --2- 172.21.15.3:0/3029379894 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde0009d6f0 0x7fde0009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:40.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.974+0000 7fb424d26700 1 --2- 172.21.15.3:0/2256554468 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb42009e000 0x7fb4201339f0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fb410009920 tx=0x7fb410009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac8a536828ad123c server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:40.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.974+0000 7fddfdd9b700 1 --2- 172.21.15.3:0/3029379894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde0009e000 0x7fde00130660 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fddf4009870 tx=0x7fddf4009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d109ace260633b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:40.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.974+0000 7fb417fff700 1 -- 172.21.15.3:0/2256554468 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb410011830 con 0x7fb42009e000 2023-10-03T10:19:40.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.975+0000 7fb417fff700 1 -- 172.21.15.3:0/2256554468 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb410011990 con 0x7fb42009e000 2023-10-03T10:19:40.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.975+0000 7fb417fff700 1 -- 172.21.15.3:0/2256554468 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb410011d20 con 0x7fb42009e000 2023-10-03T10:19:40.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.975+0000 7fddfd59a700 1 -- 172.21.15.3:0/3029379894 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf4011830 con 0x7fde0009e000 2023-10-03T10:19:40.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.975+0000 7fddfd59a700 1 -- 172.21.15.3:0/3029379894 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf4011990 con 0x7fde0009e000 2023-10-03T10:19:40.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.975+0000 7fddfd59a700 1 -- 172.21.15.3:0/3029379894 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf4011d20 con 0x7fde0009e000 2023-10-03T10:19:40.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.975+0000 7fb42778b700 1 -- 172.21.15.3:0/2256554468 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb42009e000 msgr2=0x7fb4201339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:40.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.975+0000 7fb42778b700 1 --2- 172.21.15.3:0/2256554468 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb42009e000 0x7fb4201339f0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fb410009920 tx=0x7fb410009c30 comp rx=0 tx=0).stop 2023-10-03T10:19:40.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.975+0000 7fde04891700 1 -- 172.21.15.3:0/3029379894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde0009e000 msgr2=0x7fde00130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:40.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.975+0000 7fde04891700 1 --2- 172.21.15.3:0/3029379894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde0009e000 0x7fde00130660 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fddf4009870 tx=0x7fddf4009b80 comp rx=0 tx=0).stop 2023-10-03T10:19:40.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.976+0000 7fb42778b700 1 -- 172.21.15.3:0/2256554468 shutdown_connections 2023-10-03T10:19:40.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.976+0000 7fb42778b700 1 --2- 172.21.15.3:0/2256554468 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb42009e410 0x7fb420133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:40.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.976+0000 7fb42778b700 1 --2- 172.21.15.3:0/2256554468 >> v2:172.21.15.3:3300/0 conn(0x7fb42009d6f0 0x7fb42009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:40.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.976+0000 7fb42778b700 1 --2- 172.21.15.3:0/2256554468 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb42009e000 0x7fb4201339f0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:40.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.976+0000 7fb42778b700 1 -- 172.21.15.3:0/2256554468 >> 172.21.15.3:0/2256554468 conn(0x7fb420066f50 msgr2=0x7fb42009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:40.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.976+0000 7fb42778b700 1 -- 172.21.15.3:0/2256554468 shutdown_connections 2023-10-03T10:19:40.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.976+0000 7fb42778b700 1 -- 172.21.15.3:0/2256554468 wait complete. 2023-10-03T10:19:40.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.976+0000 7fde04891700 1 -- 172.21.15.3:0/3029379894 shutdown_connections 2023-10-03T10:19:40.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.976+0000 7fde04891700 1 --2- 172.21.15.3:0/3029379894 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fde0009e410 0x7fde00130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:40.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.976+0000 7fde04891700 1 --2- 172.21.15.3:0/3029379894 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde0009d6f0 0x7fde0009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:40.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.976+0000 7fde04891700 1 --2- 172.21.15.3:0/3029379894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde0009e000 0x7fde00130660 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:40.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.976+0000 7fde04891700 1 -- 172.21.15.3:0/3029379894 >> 172.21.15.3:0/3029379894 conn(0x7fde00066f50 msgr2=0x7fde0009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:40.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.976+0000 7fde04891700 1 -- 172.21.15.3:0/3029379894 shutdown_connections 2023-10-03T10:19:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.977+0000 7fb42778b700 1 Processor -- start 2023-10-03T10:19:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.977+0000 7fb42778b700 1 -- start start 2023-10-03T10:19:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.976+0000 7fde04891700 1 -- 172.21.15.3:0/3029379894 wait complete. 2023-10-03T10:19:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.977+0000 7fde04891700 1 Processor -- start 2023-10-03T10:19:40.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.977+0000 7fb42778b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb42009d6f0 0x7fb4201a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:40.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.977+0000 7fb42778b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb42009e000 0x7fb4201a63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:40.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.977+0000 7fb42778b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb42009e410 0x7fb4201a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:40.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.977+0000 7fb42778b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4201a72e0 con 0x7fb42009d6f0 2023-10-03T10:19:40.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.977+0000 7fb42778b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4201a8070 con 0x7fb42009e000 2023-10-03T10:19:40.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.977+0000 7fb42778b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb4201a9130 con 0x7fb42009e410 2023-10-03T10:19:40.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.978+0000 7fb424d26700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb42009e000 0x7fb4201a63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:40.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.978+0000 7fb424d26700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb42009e000 0x7fb4201a63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41244/0 (socket says 172.21.15.3:41244) 2023-10-03T10:19:40.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.978+0000 7fb424d26700 1 -- 172.21.15.3:0/1142725996 learned_addr learned my addr 172.21.15.3:0/1142725996 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:40.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.978+0000 7fb425d28700 1 --2- 172.21.15.3:0/1142725996 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb42009e410 0x7fb4201a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:40.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.977+0000 7fde04891700 1 -- start start 2023-10-03T10:19:40.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.978+0000 7fde04891700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde0009d6f0 0x7fde0012f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:40.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.978+0000 7fde04891700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fde0009e000 0x7fde0012fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:40.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.978+0000 7fde04891700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde0009e410 0x7fde00130170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:40.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.978+0000 7fde04891700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0013e9a0 con 0x7fde0009e410 2023-10-03T10:19:40.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.978+0000 7fde04891700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde000a3b10 con 0x7fde0009e000 2023-10-03T10:19:40.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.978+0000 7fde04891700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde0013fb70 con 0x7fde0009d6f0 2023-10-03T10:19:40.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.978+0000 7fddfe59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde0009d6f0 0x7fde0012f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:40.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.978+0000 7fddfe59c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde0009d6f0 0x7fde0012f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48408/0 (socket says 172.21.15.3:48408) 2023-10-03T10:19:40.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.978+0000 7fddfe59c700 1 -- 172.21.15.3:0/3414949197 learned_addr learned my addr 172.21.15.3:0/3414949197 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:40.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.978+0000 7fb424d26700 1 -- 172.21.15.3:0/1142725996 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb42009e410 msgr2=0x7fb4201a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:40.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.978+0000 7fb424d26700 1 --2- 172.21.15.3:0/1142725996 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb42009e410 0x7fb4201a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:40.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.978+0000 7fb424d26700 1 -- 172.21.15.3:0/1142725996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb42009d6f0 msgr2=0x7fb4201a5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:40.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.978+0000 7fb424d26700 1 --2- 172.21.15.3:0/1142725996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb42009d6f0 0x7fb4201a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:40.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.978+0000 7fb424d26700 1 -- 172.21.15.3:0/1142725996 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb410009550 con 0x7fb42009e000 2023-10-03T10:19:40.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.978+0000 7fddfed9d700 1 --2- 172.21.15.3:0/3414949197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde0009e410 0x7fde00130170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:40.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.978+0000 7fddfdd9b700 1 --2- 172.21.15.3:0/3414949197 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fde0009e000 0x7fde0012fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:40.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.979+0000 7fb424d26700 1 --2- 172.21.15.3:0/1142725996 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb42009e000 0x7fb4201a63c0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fb4100098f0 tx=0x7fb410004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:40.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.979+0000 7fddfed9d700 1 -- 172.21.15.3:0/3414949197 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde0009d6f0 msgr2=0x7fde0012f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:40.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.979+0000 7fddfed9d700 1 --2- 172.21.15.3:0/3414949197 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde0009d6f0 0x7fde0012f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:40.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.979+0000 7fddfed9d700 1 -- 172.21.15.3:0/3414949197 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fde0009e000 msgr2=0x7fde0012fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:40.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.979+0000 7fddfed9d700 1 --2- 172.21.15.3:0/3414949197 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fde0009e000 0x7fde0012fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:40.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.979+0000 7fddfed9d700 1 -- 172.21.15.3:0/3414949197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf0009550 con 0x7fde0009e410 2023-10-03T10:19:40.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.979+0000 7fddfed9d700 1 --2- 172.21.15.3:0/3414949197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde0009e410 0x7fde00130170 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fddf800efc0 tx=0x7fddf800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:40.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.979+0000 7fddeeffd700 1 -- 172.21.15.3:0/3414949197 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf800cc90 con 0x7fde0009e410 2023-10-03T10:19:40.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.979+0000 7fb415ffb700 1 -- 172.21.15.3:0/1142725996 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb410004380 con 0x7fb42009e000 2023-10-03T10:19:40.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.979+0000 7fb415ffb700 1 -- 172.21.15.3:0/1142725996 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41001f070 con 0x7fb42009e000 2023-10-03T10:19:40.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.979+0000 7fb415ffb700 1 -- 172.21.15.3:0/1142725996 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4100036a0 con 0x7fb42009e000 2023-10-03T10:19:40.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.979+0000 7fddeeffd700 1 -- 172.21.15.3:0/3414949197 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf800f040 con 0x7fde0009e410 2023-10-03T10:19:40.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.979+0000 7fddeeffd700 1 -- 172.21.15.3:0/3414949197 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf80146c0 con 0x7fde0009e410 2023-10-03T10:19:40.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.979+0000 7fde04891700 1 -- 172.21.15.3:0/3414949197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf40094a0 con 0x7fde0009e410 2023-10-03T10:19:40.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.979+0000 7fde04891700 1 -- 172.21.15.3:0/3414949197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0013f890 con 0x7fde0009e410 2023-10-03T10:19:40.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.979+0000 7fb42778b700 1 -- 172.21.15.3:0/1142725996 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb42009c710 con 0x7fb42009e000 2023-10-03T10:19:40.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.980+0000 7fb42778b700 1 -- 172.21.15.3:0/1142725996 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4201a7a90 con 0x7fb42009e000 2023-10-03T10:19:40.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.980+0000 7fde04891700 1 -- 172.21.15.3:0/3414949197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde00074ea0 con 0x7fde0009e410 2023-10-03T10:19:40.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.981+0000 7fb42778b700 1 -- 172.21.15.3:0/1142725996 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4201a9360 con 0x7fb42009e000 2023-10-03T10:19:40.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.981+0000 7fb415ffb700 1 -- 172.21.15.3:0/1142725996 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb410021030 con 0x7fb42009e000 2023-10-03T10:19:40.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.981+0000 7fb415ffb700 1 -- 172.21.15.3:0/1142725996 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb410040960 con 0x7fb42009e000 2023-10-03T10:19:40.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.982+0000 7fddeeffd700 1 -- 172.21.15.3:0/3414949197 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fddf8014820 con 0x7fde0009e410 2023-10-03T10:19:40.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.982+0000 7fddeeffd700 1 -- 172.21.15.3:0/3414949197 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fddf8040950 con 0x7fde0009e410 2023-10-03T10:19:40.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.990+0000 7fddeeffd700 1 -- 172.21.15.3:0/3414949197 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fddf8012930 con 0x7fde0009e410 2023-10-03T10:19:40.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:40.991+0000 7fb415ffb700 1 -- 172.21.15.3:0/1142725996 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb4201a9360 con 0x7fb42009e000 2023-10-03T10:19:41.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.178+0000 7fb939af9700 1 -- 172.21.15.3:0/327112100 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb934074ea0 con 0x7fb93409d6f0 2023-10-03T10:19:41.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.178+0000 7fb91bfff700 1 -- 172.21.15.3:0/327112100 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb924045060 con 0x7fb93409d6f0 2023-10-03T10:19:41.179 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:41.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.181+0000 7fb939af9700 1 -- 172.21.15.3:0/327112100 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb93409d6f0 msgr2=0x7fb9341202f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:41.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.181+0000 7fb939af9700 1 --2- 172.21.15.3:0/327112100 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb93409d6f0 0x7fb9341202f0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fb924009c40 tx=0x7fb924004750 comp rx=0 tx=0).stop 2023-10-03T10:19:41.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.181+0000 7fb939af9700 1 -- 172.21.15.3:0/327112100 shutdown_connections 2023-10-03T10:19:41.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.181+0000 7fb939af9700 1 --2- 172.21.15.3:0/327112100 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb93409e410 0x7fb934120d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:41.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.181+0000 7fb939af9700 1 --2- 172.21.15.3:0/327112100 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb93409d6f0 0x7fb9341202f0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:41.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.181+0000 7fb939af9700 1 --2- 172.21.15.3:0/327112100 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb93409e000 0x7fb934120830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:41.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.181+0000 7fb939af9700 1 -- 172.21.15.3:0/327112100 >> 172.21.15.3:0/327112100 conn(0x7fb934066f50 msgr2=0x7fb93406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:41.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.181+0000 7fb939af9700 1 -- 172.21.15.3:0/327112100 shutdown_connections 2023-10-03T10:19:41.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.181+0000 7fb939af9700 1 -- 172.21.15.3:0/327112100 wait complete. 2023-10-03T10:19:41.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.196+0000 7fb42778b700 1 -- 172.21.15.3:0/1142725996 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb4201a9360 con 0x7fb42009e000 2023-10-03T10:19:41.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.196+0000 7fb415ffb700 1 -- 172.21.15.3:0/1142725996 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb4201a9360 con 0x7fb42009e000 2023-10-03T10:19:41.197 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:41.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.199+0000 7fb42778b700 1 -- 172.21.15.3:0/1142725996 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb42009e000 msgr2=0x7fb4201a63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:41.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.199+0000 7fb42778b700 1 --2- 172.21.15.3:0/1142725996 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb42009e000 0x7fb4201a63c0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fb4100098f0 tx=0x7fb410004750 comp rx=0 tx=0).stop 2023-10-03T10:19:41.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.199+0000 7fb42778b700 1 -- 172.21.15.3:0/1142725996 shutdown_connections 2023-10-03T10:19:41.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.199+0000 7fb42778b700 1 --2- 172.21.15.3:0/1142725996 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb42009e410 0x7fb4201a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:41.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.199+0000 7fb42778b700 1 --2- 172.21.15.3:0/1142725996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb42009d6f0 0x7fb4201a5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:41.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.199+0000 7fb42778b700 1 --2- 172.21.15.3:0/1142725996 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb42009e000 0x7fb4201a63c0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:41.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.199+0000 7fb42778b700 1 -- 172.21.15.3:0/1142725996 >> 172.21.15.3:0/1142725996 conn(0x7fb420066f50 msgr2=0x7fb4201bae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:41.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.200+0000 7fb42778b700 1 -- 172.21.15.3:0/1142725996 shutdown_connections 2023-10-03T10:19:41.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.200+0000 7fb42778b700 1 -- 172.21.15.3:0/1142725996 wait complete. 2023-10-03T10:19:41.201 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:41.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.200+0000 7fde04891700 1 -- 172.21.15.3:0/3414949197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fde0013e3e0 con 0x7fde0009e410 2023-10-03T10:19:41.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.200+0000 7fddeeffd700 1 -- 172.21.15.3:0/3414949197 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fddf802a030 con 0x7fde0009e410 2023-10-03T10:19:41.202 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:41.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.203+0000 7fde04891700 1 -- 172.21.15.3:0/3414949197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde0009e410 msgr2=0x7fde00130170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:41.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.203+0000 7fde04891700 1 --2- 172.21.15.3:0/3414949197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde0009e410 0x7fde00130170 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fddf800efc0 tx=0x7fddf800c3a0 comp rx=0 tx=0).stop 2023-10-03T10:19:41.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.203+0000 7fde04891700 1 -- 172.21.15.3:0/3414949197 shutdown_connections 2023-10-03T10:19:41.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.203+0000 7fde04891700 1 --2- 172.21.15.3:0/3414949197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde0009e410 0x7fde00130170 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:41.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.203+0000 7fde04891700 1 --2- 172.21.15.3:0/3414949197 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fde0009d6f0 0x7fde0012f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:41.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.203+0000 7fde04891700 1 --2- 172.21.15.3:0/3414949197 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fde0009e000 0x7fde0012fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:41.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.203+0000 7fde04891700 1 -- 172.21.15.3:0/3414949197 >> 172.21.15.3:0/3414949197 conn(0x7fde00066f50 msgr2=0x7fde0006ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:41.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.203+0000 7fde04891700 1 -- 172.21.15.3:0/3414949197 shutdown_connections 2023-10-03T10:19:41.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.203+0000 7fde04891700 1 -- 172.21.15.3:0/3414949197 wait complete. 2023-10-03T10:19:41.215 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:41.218 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:41.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.218+0000 7fbdcede3700 1 -- 172.21.15.3:0/2077922103 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbdc806b9e0 con 0x7fbdc81338a0 2023-10-03T10:19:41.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.219+0000 7fbdc57fa700 1 -- 172.21.15.3:0/2077922103 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbdc0017030 con 0x7fbdc81338a0 2023-10-03T10:19:41.219 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:41.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.221+0000 7fbdcede3700 1 -- 172.21.15.3:0/2077922103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbdc81338a0 msgr2=0x7fbdc811b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:41.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.221+0000 7fbdcede3700 1 --2- 172.21.15.3:0/2077922103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbdc81338a0 0x7fbdc811b150 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000ca80 tx=0x7fbdc000cd90 comp rx=0 tx=0).stop 2023-10-03T10:19:41.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.221+0000 7fbdcede3700 1 -- 172.21.15.3:0/2077922103 shutdown_connections 2023-10-03T10:19:41.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.221+0000 7fbdcede3700 1 --2- 172.21.15.3:0/2077922103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbdc81338a0 0x7fbdc811b150 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:41.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.221+0000 7fbdcede3700 1 --2- 172.21.15.3:0/2077922103 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbdc809de50 0x7fbdc811a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:41.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.221+0000 7fbdcede3700 1 --2- 172.21.15.3:0/2077922103 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fbdc809e760 0x7fbdc811ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:41.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.221+0000 7fbdcede3700 1 -- 172.21.15.3:0/2077922103 >> 172.21.15.3:0/2077922103 conn(0x7fbdc80a2aa0 msgr2=0x7fbdc8130730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:41.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.221+0000 7fbdcede3700 1 -- 172.21.15.3:0/2077922103 shutdown_connections 2023-10-03T10:19:41.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:41.221+0000 7fbdcede3700 1 -- 172.21.15.3:0/2077922103 wait complete. 2023-10-03T10:19:41.235 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:42.202 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:42.216 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:42.218 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:42.236 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:42.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.359+0000 7f2ccc7ae700 1 Processor -- start 2023-10-03T10:19:42.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.359+0000 7f2ccc7ae700 1 -- start start 2023-10-03T10:19:42.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.360+0000 7f2ccc7ae700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2cc409d9f0 0x7f2cc4136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:42.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.360+0000 7f2ccc7ae700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2cc4136a70 0x7f2cc4138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:42.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.360+0000 7f2ccc7ae700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cc409e130 con 0x7f2cc41316c0 2023-10-03T10:19:42.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.360+0000 7f2ccc7ae700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc41321a0 con 0x7f2cc409d9f0 2023-10-03T10:19:42.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.360+0000 7f2ccc7ae700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc41312b0 con 0x7f2cc4136a70 2023-10-03T10:19:42.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.360+0000 7f2cc9d49700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2cc409d9f0 0x7f2cc4136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:42.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.360+0000 7f2cc9d49700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2cc409d9f0 0x7f2cc4136530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41268/0 (socket says 172.21.15.3:41268) 2023-10-03T10:19:42.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.360+0000 7f2cc9d49700 1 -- 172.21.15.3:0/3652548537 learned_addr learned my addr 172.21.15.3:0/3652548537 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:42.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.360+0000 7f2ccad4b700 1 --2- 172.21.15.3:0/3652548537 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2cc4136a70 0x7f2cc4138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.360+0000 7f2ccad4b700 1 -- 172.21.15.3:0/3652548537 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2cc409d9f0 msgr2=0x7f2cc4136530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.360+0000 7f2ccad4b700 1 --2- 172.21.15.3:0/3652548537 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2cc409d9f0 0x7f2cc4136530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.360+0000 7f2ccad4b700 1 -- 172.21.15.3:0/3652548537 >> v1:172.21.15.3:6789/0 conn(0x7f2cc41316c0 legacy=0x7f2cc4131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.361+0000 7f2ccad4b700 1 -- 172.21.15.3:0/3652548537 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc409f6e0 con 0x7f2cc4136a70 2023-10-03T10:19:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.361+0000 7f2cc9548700 1 -- 172.21.15.3:0/3652548537 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2726027438 0 0) 0x7f2cc0001110 con 0x7f2cc41316c0 2023-10-03T10:19:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.361+0000 7f2cc9d49700 1 --2- 172.21.15.3:0/3652548537 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2cc409d9f0 0x7f2cc4136530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.361+0000 7f2ccad4b700 1 --2- 172.21.15.3:0/3652548537 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2cc4136a70 0x7f2cc4138e50 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc009df0 tx=0x7f2cbc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54a3b3ff3c6f23c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.361+0000 7f2cc9548700 1 -- 172.21.15.3:0/3652548537 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc00e070 con 0x7f2cc4136a70 2023-10-03T10:19:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.361+0000 7f2cc9548700 1 -- 172.21.15.3:0/3652548537 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cbc00bae0 con 0x7f2cc4136a70 2023-10-03T10:19:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.361+0000 7f2cc9548700 1 -- 172.21.15.3:0/3652548537 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc00be90 con 0x7f2cc4136a70 2023-10-03T10:19:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.362+0000 7f2ccc7ae700 1 -- 172.21.15.3:0/3652548537 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2cc4136a70 msgr2=0x7f2cc4138e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.362+0000 7f2ccc7ae700 1 --2- 172.21.15.3:0/3652548537 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2cc4136a70 0x7f2cc4138e50 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc009df0 tx=0x7f2cbc00b330 comp rx=0 tx=0).stop 2023-10-03T10:19:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.362+0000 7f2ccc7ae700 1 -- 172.21.15.3:0/3652548537 shutdown_connections 2023-10-03T10:19:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.362+0000 7f2ccc7ae700 1 --2- 172.21.15.3:0/3652548537 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2cc4136a70 0x7f2cc4138e50 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.362+0000 7f2ccc7ae700 1 --2- 172.21.15.3:0/3652548537 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2cc409d9f0 0x7f2cc4136530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.362+0000 7f2ccc7ae700 1 -- 172.21.15.3:0/3652548537 >> 172.21.15.3:0/3652548537 conn(0x7f2cc4066f50 msgr2=0x7f2cc409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:42.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.362+0000 7f2ccc7ae700 1 -- 172.21.15.3:0/3652548537 shutdown_connections 2023-10-03T10:19:42.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.362+0000 7f2ccc7ae700 1 -- 172.21.15.3:0/3652548537 wait complete. 2023-10-03T10:19:42.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.364+0000 7f2ccc7ae700 1 Processor -- start 2023-10-03T10:19:42.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.364+0000 7f2ccc7ae700 1 -- start start 2023-10-03T10:19:42.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.365+0000 7f2ccc7ae700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cc409d9f0 0x7f2cc4127560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:42.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.365+0000 7f2ccc7ae700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2cc41316c0 0x7f2cc4127aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:42.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.365+0000 7f2ccc7ae700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2cc4136a70 0x7f2cc4127fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:42.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.365+0000 7f2ccc7ae700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc412b690 con 0x7f2cc409d9f0 2023-10-03T10:19:42.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.365+0000 7f2ccc7ae700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc412b900 con 0x7f2cc4136a70 2023-10-03T10:19:42.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.365+0000 7f2ccc7ae700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc4134960 con 0x7f2cc41316c0 2023-10-03T10:19:42.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.365+0000 7f2ccad4b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2cc4136a70 0x7f2cc4127fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:42.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.365+0000 7f2cca54a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cc409d9f0 0x7f2cc4127560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:42.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.365+0000 7f2ccad4b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2cc4136a70 0x7f2cc4127fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41274/0 (socket says 172.21.15.3:41274) 2023-10-03T10:19:42.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.365+0000 7f2cca54a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cc409d9f0 0x7f2cc4127560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37894/0 (socket says 172.21.15.3:37894) 2023-10-03T10:19:42.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.365+0000 7f2ccad4b700 1 -- 172.21.15.3:0/2336547394 learned_addr learned my addr 172.21.15.3:0/2336547394 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:42.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.365+0000 7f2cc9d49700 1 --2- 172.21.15.3:0/2336547394 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2cc41316c0 0x7f2cc4127aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:42.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.365+0000 7f2cca54a700 1 -- 172.21.15.3:0/2336547394 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2cc41316c0 msgr2=0x7f2cc4127aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:42.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.365+0000 7f2cca54a700 1 --2- 172.21.15.3:0/2336547394 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2cc41316c0 0x7f2cc4127aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.365+0000 7f2cca54a700 1 -- 172.21.15.3:0/2336547394 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2cc4136a70 msgr2=0x7f2cc4127fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:42.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.365+0000 7f2cca54a700 1 --2- 172.21.15.3:0/2336547394 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2cc4136a70 0x7f2cc4127fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.365+0000 7f2cca54a700 1 -- 172.21.15.3:0/2336547394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb4009550 con 0x7f2cc409d9f0 2023-10-03T10:19:42.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.365+0000 7f2ccad4b700 1 --2- 172.21.15.3:0/2336547394 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2cc4136a70 0x7f2cc4127fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:42.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.366+0000 7f2cca54a700 1 --2- 172.21.15.3:0/2336547394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cc409d9f0 0x7f2cc4127560 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0000f60 tx=0x7f2cc000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:42.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.366+0000 7f2cbaffd700 1 -- 172.21.15.3:0/2336547394 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc00054e0 con 0x7f2cc409d9f0 2023-10-03T10:19:42.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.366+0000 7f2cbaffd700 1 -- 172.21.15.3:0/2336547394 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc0003680 con 0x7f2cc409d9f0 2023-10-03T10:19:42.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.366+0000 7f2cbaffd700 1 -- 172.21.15.3:0/2336547394 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0003a30 con 0x7f2cc409d9f0 2023-10-03T10:19:42.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.366+0000 7f2ccc7ae700 1 -- 172.21.15.3:0/2336547394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cbc009a70 con 0x7f2cc409d9f0 2023-10-03T10:19:42.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.366+0000 7f2ccc7ae700 1 -- 172.21.15.3:0/2336547394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc4136190 con 0x7f2cc409d9f0 2023-10-03T10:19:42.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.367+0000 7f2cbaffd700 1 -- 172.21.15.3:0/2336547394 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2cc0017020 con 0x7f2cc409d9f0 2023-10-03T10:19:42.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.367+0000 7f2cbaffd700 1 -- 172.21.15.3:0/2336547394 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2cc0042940 con 0x7f2cc409d9f0 2023-10-03T10:19:42.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.368+0000 7f2ccc7ae700 1 -- 172.21.15.3:0/2336547394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc409e130 con 0x7f2cc409d9f0 2023-10-03T10:19:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.373+0000 7f2cbaffd700 1 -- 172.21.15.3:0/2336547394 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2cc409e130 con 0x7f2cc409d9f0 2023-10-03T10:19:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.401+0000 7f5882396700 1 Processor -- start 2023-10-03T10:19:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.401+0000 7f5882396700 1 -- start start 2023-10-03T10:19:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.402+0000 7f5882396700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f587c1338c0 0x7f587c133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.402+0000 7f5882396700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f587c1341d0 0x7f587c09de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.402+0000 7f5882396700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f587c06f720 con 0x7f587c09e390 2023-10-03T10:19:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.402+0000 7f5882396700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f587c070c80 con 0x7f587c1341d0 2023-10-03T10:19:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.402+0000 7f5882396700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f587c06fce0 con 0x7f587c1338c0 2023-10-03T10:19:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.402+0000 7f587bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f587c1338c0 0x7f587c133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.402+0000 7f587bfff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f587c1338c0 0x7f587c133c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48440/0 (socket says 172.21.15.3:48440) 2023-10-03T10:19:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.402+0000 7f587bfff700 1 -- 172.21.15.3:0/425496724 learned_addr learned my addr 172.21.15.3:0/425496724 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.402+0000 7f587b7fe700 1 --2- 172.21.15.3:0/425496724 >> v2:172.21.15.3:3300/0 conn(0x7f587c1341d0 0x7f587c09de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.402+0000 7f587bfff700 1 -- 172.21.15.3:0/425496724 >> v2:172.21.15.3:3300/0 conn(0x7f587c1341d0 msgr2=0x7f587c09de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.402+0000 7f587bfff700 1 --2- 172.21.15.3:0/425496724 >> v2:172.21.15.3:3300/0 conn(0x7f587c1341d0 0x7f587c09de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.402+0000 7f587bfff700 1 -- 172.21.15.3:0/425496724 >> v1:172.21.15.3:6789/0 conn(0x7f587c09e390 legacy=0x7f587c13a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.402+0000 7f587bfff700 1 -- 172.21.15.3:0/425496724 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587c1336a0 con 0x7f587c1338c0 2023-10-03T10:19:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.402+0000 7f587b7fe700 1 --2- 172.21.15.3:0/425496724 >> v2:172.21.15.3:3300/0 conn(0x7f587c1341d0 0x7f587c09de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.403+0000 7f587bfff700 1 --2- 172.21.15.3:0/425496724 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f587c1338c0 0x7f587c133c90 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f586c0098f0 tx=0x7f586c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb0872eb43433c26 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.403+0000 7f587affd700 1 -- 172.21.15.3:0/425496724 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c011830 con 0x7f587c1338c0 2023-10-03T10:19:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.403+0000 7f587affd700 1 -- 172.21.15.3:0/425496724 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f586c011990 con 0x7f587c1338c0 2023-10-03T10:19:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.403+0000 7f587affd700 1 -- 172.21.15.3:0/425496724 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c011d20 con 0x7f587c1338c0 2023-10-03T10:19:42.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.404+0000 7f5882396700 1 -- 172.21.15.3:0/425496724 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f587c1338c0 msgr2=0x7f587c133c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:42.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.404+0000 7f5882396700 1 --2- 172.21.15.3:0/425496724 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f587c1338c0 0x7f587c133c90 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f586c0098f0 tx=0x7f586c009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:42.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.404+0000 7f5882396700 1 -- 172.21.15.3:0/425496724 shutdown_connections 2023-10-03T10:19:42.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.404+0000 7f5882396700 1 --2- 172.21.15.3:0/425496724 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f587c1338c0 0x7f587c133c90 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.404+0000 7f5882396700 1 --2- 172.21.15.3:0/425496724 >> v2:172.21.15.3:3300/0 conn(0x7f587c1341d0 0x7f587c09de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.404+0000 7f5882396700 1 -- 172.21.15.3:0/425496724 >> 172.21.15.3:0/425496724 conn(0x7f587c0a2aa0 msgr2=0x7f587c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:42.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.404+0000 7f5882396700 1 -- 172.21.15.3:0/425496724 shutdown_connections 2023-10-03T10:19:42.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.404+0000 7f5882396700 1 -- 172.21.15.3:0/425496724 wait complete. 2023-10-03T10:19:42.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.405+0000 7f5882396700 1 Processor -- start 2023-10-03T10:19:42.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.405+0000 7f5882396700 1 -- start start 2023-10-03T10:19:42.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.406+0000 7f5882396700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f587c09e390 0x7f587c11f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:42.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.406+0000 7f5882396700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587c1338c0 0x7f587c11fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:42.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.406+0000 7f5882396700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f587c1341d0 0x7f587c1202f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:42.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.406+0000 7f5882396700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587c1230b0 con 0x7f587c1338c0 2023-10-03T10:19:42.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.406+0000 7f5882396700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f587c122390 con 0x7f587c09e390 2023-10-03T10:19:42.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.406+0000 7f5882396700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f587c0a1260 con 0x7f587c1341d0 2023-10-03T10:19:42.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.406+0000 7f5880933700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f587c1341d0 0x7f587c1202f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:42.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.406+0000 7f587bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f587c09e390 0x7f587c11f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:42.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.406+0000 7f5880933700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f587c1341d0 0x7f587c1202f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48448/0 (socket says 172.21.15.3:48448) 2023-10-03T10:19:42.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.406+0000 7f5880933700 1 -- 172.21.15.3:0/3168280997 learned_addr learned my addr 172.21.15.3:0/3168280997 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:42.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.406+0000 7f587b7fe700 1 --2- 172.21.15.3:0/3168280997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587c1338c0 0x7f587c11fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:42.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.406+0000 7f5880933700 1 -- 172.21.15.3:0/3168280997 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f587c09e390 msgr2=0x7f587c11f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:42.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.406+0000 7f5880933700 1 --2- 172.21.15.3:0/3168280997 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f587c09e390 0x7f587c11f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.406+0000 7f5880933700 1 -- 172.21.15.3:0/3168280997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587c1338c0 msgr2=0x7f587c11fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:42.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.406+0000 7f5880933700 1 --2- 172.21.15.3:0/3168280997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587c1338c0 0x7f587c11fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.406+0000 7f5880933700 1 -- 172.21.15.3:0/3168280997 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58700094a0 con 0x7f587c1341d0 2023-10-03T10:19:42.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.406+0000 7f587b7fe700 1 --2- 172.21.15.3:0/3168280997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587c1338c0 0x7f587c11fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:42.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.407+0000 7f5880933700 1 --2- 172.21.15.3:0/3168280997 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f587c1341d0 0x7f587c1202f0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f587400cc50 tx=0x7f587400cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:42.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.407+0000 7f5878ff9700 1 -- 172.21.15.3:0/3168280997 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5874003c00 con 0x7f587c1341d0 2023-10-03T10:19:42.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.407+0000 7f5878ff9700 1 -- 172.21.15.3:0/3168280997 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5874003d60 con 0x7f587c1341d0 2023-10-03T10:19:42.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.407+0000 7f5882396700 1 -- 172.21.15.3:0/3168280997 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f586c009550 con 0x7f587c1341d0 2023-10-03T10:19:42.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.407+0000 7f5878ff9700 1 -- 172.21.15.3:0/3168280997 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587400aa50 con 0x7f587c1341d0 2023-10-03T10:19:42.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.407+0000 7f5882396700 1 -- 172.21.15.3:0/3168280997 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f587c123b20 con 0x7f587c1341d0 2023-10-03T10:19:42.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.408+0000 7f5878ff9700 1 -- 172.21.15.3:0/3168280997 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f587400abb0 con 0x7f587c1341d0 2023-10-03T10:19:42.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.408+0000 7f5878ff9700 1 -- 172.21.15.3:0/3168280997 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f587403f6e0 con 0x7f587c1341d0 2023-10-03T10:19:42.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.410+0000 7f5882396700 1 -- 172.21.15.3:0/3168280997 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f587c11c480 con 0x7f587c1341d0 2023-10-03T10:19:42.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.410+0000 7f4a179a8700 1 Processor -- start 2023-10-03T10:19:42.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.411+0000 7f4a179a8700 1 -- start start 2023-10-03T10:19:42.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.411+0000 7f4a179a8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a1009d6f0 0x7f4a1009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:42.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.411+0000 7f4a179a8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4a1009e000 0x7f4a10130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:42.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.411+0000 7f4a179a8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a1009e410 0x7f4a10130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:42.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.411+0000 7f4a179a8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a1009f520 con 0x7f4a1009e000 2023-10-03T10:19:42.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.411+0000 7f4a179a8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1009f130 con 0x7f4a1009e410 2023-10-03T10:19:42.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.411+0000 7f4a179a8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1009e930 con 0x7f4a1009d6f0 2023-10-03T10:19:42.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.412+0000 7f4a15744700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a1009d6f0 0x7f4a1009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:42.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.412+0000 7f4a15744700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a1009d6f0 0x7f4a1009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48454/0 (socket says 172.21.15.3:48454) 2023-10-03T10:19:42.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.412+0000 7f4a15744700 1 -- 172.21.15.3:0/1097809274 learned_addr learned my addr 172.21.15.3:0/1097809274 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:42.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.412+0000 7f4a15f45700 1 --2- 172.21.15.3:0/1097809274 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a1009e410 0x7f4a10130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:42.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.412+0000 7f4a14f43700 1 --2- 172.21.15.3:0/1097809274 >> v2:172.21.15.3:3300/0 conn(0x7f4a1009e000 0x7f4a10130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:42.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.412+0000 7f4a15744700 1 -- 172.21.15.3:0/1097809274 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a1009e410 msgr2=0x7f4a10130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:42.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.412+0000 7f4a15744700 1 --2- 172.21.15.3:0/1097809274 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a1009e410 0x7f4a10130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.412+0000 7f4a15744700 1 -- 172.21.15.3:0/1097809274 >> v2:172.21.15.3:3300/0 conn(0x7f4a1009e000 msgr2=0x7f4a10130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:42.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.412+0000 7f4a15744700 1 --2- 172.21.15.3:0/1097809274 >> v2:172.21.15.3:3300/0 conn(0x7f4a1009e000 0x7f4a10130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.412+0000 7f4a15744700 1 -- 172.21.15.3:0/1097809274 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1009b2d0 con 0x7f4a1009d6f0 2023-10-03T10:19:42.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.412+0000 7f4a15f45700 1 --2- 172.21.15.3:0/1097809274 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a1009e410 0x7f4a10130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:42.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.412+0000 7f4a15744700 1 --2- 172.21.15.3:0/1097809274 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a1009d6f0 0x7f4a1009dac0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00bbc0 tx=0x7f4a0c00bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fda615f54f42cfc8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:42.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.416+0000 7f4a07fff700 1 -- 172.21.15.3:0/1097809274 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c00dcf0 con 0x7f4a1009d6f0 2023-10-03T10:19:42.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.416+0000 7f4a07fff700 1 -- 172.21.15.3:0/1097809274 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0c00de50 con 0x7f4a1009d6f0 2023-10-03T10:19:42.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.416+0000 7f4a07fff700 1 -- 172.21.15.3:0/1097809274 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c010690 con 0x7f4a1009d6f0 2023-10-03T10:19:42.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.416+0000 7f5878ff9700 1 -- 172.21.15.3:0/3168280997 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5874012420 con 0x7f587c1341d0 2023-10-03T10:19:42.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.417+0000 7f4a179a8700 1 -- 172.21.15.3:0/1097809274 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a1009d6f0 msgr2=0x7f4a1009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:42.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.417+0000 7f4a179a8700 1 --2- 172.21.15.3:0/1097809274 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a1009d6f0 0x7f4a1009dac0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00bbc0 tx=0x7f4a0c00bed0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.417+0000 7f4a179a8700 1 -- 172.21.15.3:0/1097809274 shutdown_connections 2023-10-03T10:19:42.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.417+0000 7f4a179a8700 1 --2- 172.21.15.3:0/1097809274 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a1009e410 0x7f4a10130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.417+0000 7f4a179a8700 1 --2- 172.21.15.3:0/1097809274 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a1009d6f0 0x7f4a1009dac0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.417+0000 7f4a179a8700 1 --2- 172.21.15.3:0/1097809274 >> v2:172.21.15.3:3300/0 conn(0x7f4a1009e000 0x7f4a10130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.417+0000 7f4a179a8700 1 -- 172.21.15.3:0/1097809274 >> 172.21.15.3:0/1097809274 conn(0x7f4a10066f50 msgr2=0x7f4a1009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:42.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.418+0000 7f4a179a8700 1 -- 172.21.15.3:0/1097809274 shutdown_connections 2023-10-03T10:19:42.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.418+0000 7f4a179a8700 1 -- 172.21.15.3:0/1097809274 wait complete. 2023-10-03T10:19:42.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.419+0000 7f4a179a8700 1 Processor -- start 2023-10-03T10:19:42.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.419+0000 7f4a179a8700 1 -- start start 2023-10-03T10:19:42.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.420+0000 7f4a179a8700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a1009d6f0 0x7f4a1011e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:42.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.420+0000 7f4a179a8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a1009e000 0x7f4a1011e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:42.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.420+0000 7f4a179a8700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a1009e410 0x7f4a1011eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:42.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.420+0000 7f4a179a8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a10120230 con 0x7f4a1009e000 2023-10-03T10:19:42.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.420+0000 7f4a179a8700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a100a1a60 con 0x7f4a1009d6f0 2023-10-03T10:19:42.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.420+0000 7f4a179a8700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a10121320 con 0x7f4a1009e410 2023-10-03T10:19:42.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.420+0000 7f4a15744700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a1009d6f0 0x7f4a1011e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:42.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.420+0000 7f4a15744700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a1009d6f0 0x7f4a1011e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41298/0 (socket says 172.21.15.3:41298) 2023-10-03T10:19:42.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.420+0000 7f4a15744700 1 -- 172.21.15.3:0/3459576846 learned_addr learned my addr 172.21.15.3:0/3459576846 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:42.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.420+0000 7f4a15f45700 1 --2- 172.21.15.3:0/3459576846 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a1009e410 0x7f4a1011eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:42.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.420+0000 7f4a14f43700 1 --2- 172.21.15.3:0/3459576846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a1009e000 0x7f4a1011e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:42.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.421+0000 7f4a15744700 1 -- 172.21.15.3:0/3459576846 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a1009e410 msgr2=0x7f4a1011eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:42.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.421+0000 7f4a15744700 1 --2- 172.21.15.3:0/3459576846 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a1009e410 0x7f4a1011eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.421+0000 7f4a15744700 1 -- 172.21.15.3:0/3459576846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a1009e000 msgr2=0x7f4a1011e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:42.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.421+0000 7f4a15744700 1 --2- 172.21.15.3:0/3459576846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a1009e000 0x7f4a1011e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.421+0000 7f4a15744700 1 -- 172.21.15.3:0/3459576846 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a08009a70 con 0x7f4a1009d6f0 2023-10-03T10:19:42.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.422+0000 7f4a15744700 1 --2- 172.21.15.3:0/3459576846 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a1009d6f0 0x7f4a1011e070 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f4a1009ea70 tx=0x7f4a0c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:42.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.422+0000 7f4a05ffb700 1 -- 172.21.15.3:0/3459576846 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c01f070 con 0x7f4a1009d6f0 2023-10-03T10:19:42.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.422+0000 7f4a05ffb700 1 -- 172.21.15.3:0/3459576846 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0c0038b0 con 0x7f4a1009d6f0 2023-10-03T10:19:42.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.422+0000 7f4a05ffb700 1 -- 172.21.15.3:0/3459576846 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c003c60 con 0x7f4a1009d6f0 2023-10-03T10:19:42.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.422+0000 7f4a179a8700 1 -- 172.21.15.3:0/3459576846 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a0c00b7f0 con 0x7f4a1009d6f0 2023-10-03T10:19:42.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.422+0000 7f4a179a8700 1 -- 172.21.15.3:0/3459576846 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a10122320 con 0x7f4a1009d6f0 2023-10-03T10:19:42.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.423+0000 7f4a179a8700 1 -- 172.21.15.3:0/3459576846 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a10074ea0 con 0x7f4a1009d6f0 2023-10-03T10:19:42.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.424+0000 7f4a05ffb700 1 -- 172.21.15.3:0/3459576846 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4a0c019400 con 0x7f4a1009d6f0 2023-10-03T10:19:42.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.424+0000 7fdf36978700 1 Processor -- start 2023-10-03T10:19:42.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.425+0000 7fdf36978700 1 -- start start 2023-10-03T10:19:42.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.425+0000 7f4a05ffb700 1 -- 172.21.15.3:0/3459576846 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4a0c042d60 con 0x7f4a1009d6f0 2023-10-03T10:19:42.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.425+0000 7fdf36978700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf3009d6f0 0x7fdf3009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:42.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.425+0000 7fdf36978700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf3009e000 0x7fdf30130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:42.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.425+0000 7fdf36978700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdf3009e410 0x7fdf30130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:42.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.425+0000 7fdf36978700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf3009f520 con 0x7fdf3009e410 2023-10-03T10:19:42.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.425+0000 7fdf36978700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3009f130 con 0x7fdf3009d6f0 2023-10-03T10:19:42.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.425+0000 7fdf36978700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3009e930 con 0x7fdf3009e000 2023-10-03T10:19:42.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.425+0000 7fdf2f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf3009e000 0x7fdf30130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:42.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.425+0000 7fdf34f15700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdf3009e410 0x7fdf30130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:42.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.425+0000 7fdf2f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf3009e000 0x7fdf30130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48468/0 (socket says 172.21.15.3:48468) 2023-10-03T10:19:42.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.425+0000 7fdf2f7fe700 1 -- 172.21.15.3:0/986222202 learned_addr learned my addr 172.21.15.3:0/986222202 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:42.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.426+0000 7fdf2ffff700 1 --2- 172.21.15.3:0/986222202 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf3009d6f0 0x7fdf3009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:42.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.426+0000 7fdf34f15700 1 -- 172.21.15.3:0/986222202 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf3009e000 msgr2=0x7fdf30130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:42.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.426+0000 7fdf34f15700 1 --2- 172.21.15.3:0/986222202 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf3009e000 0x7fdf30130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.426+0000 7fdf34f15700 1 -- 172.21.15.3:0/986222202 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf3009d6f0 msgr2=0x7fdf3009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:42.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.426+0000 7fdf34f15700 1 --2- 172.21.15.3:0/986222202 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf3009d6f0 0x7fdf3009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.426+0000 7fdf34f15700 1 -- 172.21.15.3:0/986222202 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3009b2d0 con 0x7fdf3009e410 2023-10-03T10:19:42.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.426+0000 7fdf34f15700 1 --2- 172.21.15.3:0/986222202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf3009e410 0x7fdf30130ba0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fdf2000dc60 tx=0x7fdf2000df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e540385aa16a56f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:42.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.426+0000 7fdf2effd700 1 -- 172.21.15.3:0/986222202 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf20012070 con 0x7fdf3009e410 2023-10-03T10:19:42.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.426+0000 7fdf2effd700 1 -- 172.21.15.3:0/986222202 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2000d2e0 con 0x7fdf3009e410 2023-10-03T10:19:42.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.426+0000 7fdf2effd700 1 -- 172.21.15.3:0/986222202 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2000f900 con 0x7fdf3009e410 2023-10-03T10:19:42.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.427+0000 7fdf36978700 1 -- 172.21.15.3:0/986222202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf3009e410 msgr2=0x7fdf30130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:42.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.427+0000 7fdf36978700 1 --2- 172.21.15.3:0/986222202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf3009e410 0x7fdf30130ba0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fdf2000dc60 tx=0x7fdf2000df70 comp rx=0 tx=0).stop 2023-10-03T10:19:42.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.427+0000 7fdf36978700 1 -- 172.21.15.3:0/986222202 shutdown_connections 2023-10-03T10:19:42.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.427+0000 7fdf36978700 1 --2- 172.21.15.3:0/986222202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf3009e410 0x7fdf30130ba0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.427+0000 7fdf36978700 1 --2- 172.21.15.3:0/986222202 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf3009d6f0 0x7fdf3009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.427+0000 7fdf36978700 1 --2- 172.21.15.3:0/986222202 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf3009e000 0x7fdf30130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.427+0000 7fdf36978700 1 -- 172.21.15.3:0/986222202 >> 172.21.15.3:0/986222202 conn(0x7fdf30066f50 msgr2=0x7fdf3009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:42.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.429+0000 7f4a05ffb700 1 -- 172.21.15.3:0/3459576846 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4a0c02a720 con 0x7f4a1009d6f0 2023-10-03T10:19:42.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.433+0000 7fdf36978700 1 -- 172.21.15.3:0/986222202 shutdown_connections 2023-10-03T10:19:42.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.433+0000 7fdf36978700 1 -- 172.21.15.3:0/986222202 wait complete. 2023-10-03T10:19:42.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.434+0000 7fdf36978700 1 Processor -- start 2023-10-03T10:19:42.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.434+0000 7fdf36978700 1 -- start start 2023-10-03T10:19:42.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.435+0000 7fdf36978700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf3009d6f0 0x7fdf3011deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:42.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.435+0000 7fdf36978700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf3009e000 0x7fdf3011e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:42.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.435+0000 7fdf36978700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf3009e410 0x7fdf3011e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:42.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.435+0000 7fdf36978700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3011f340 con 0x7fdf3009e000 2023-10-03T10:19:42.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.435+0000 7fdf36978700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3011f9e0 con 0x7fdf3009e410 2023-10-03T10:19:42.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.435+0000 7fdf36978700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf30120000 con 0x7fdf3009d6f0 2023-10-03T10:19:42.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.436+0000 7fdf2ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf3009d6f0 0x7fdf3011deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:42.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.436+0000 7fdf2ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf3009d6f0 0x7fdf3011deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:48482/0 (socket says 172.21.15.3:48482) 2023-10-03T10:19:42.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.436+0000 7fdf2ffff700 1 -- 172.21.15.3:0/2977371405 learned_addr learned my addr 172.21.15.3:0/2977371405 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:42.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.436+0000 7fdf2f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf3009e000 0x7fdf3011e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:42.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.436+0000 7fdf2f7fe700 1 -- 172.21.15.3:0/2977371405 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf3009d6f0 msgr2=0x7fdf3011deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:42.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.436+0000 7fdf2f7fe700 1 --2- 172.21.15.3:0/2977371405 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf3009d6f0 0x7fdf3011deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.436+0000 7fdf2f7fe700 1 -- 172.21.15.3:0/2977371405 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf3009e410 msgr2=0x7fdf3011e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:42.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.436+0000 7fdf2f7fe700 1 --2- 172.21.15.3:0/2977371405 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf3009e410 0x7fdf3011e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.436+0000 7fdf2f7fe700 1 -- 172.21.15.3:0/2977371405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2000d940 con 0x7fdf3009e000 2023-10-03T10:19:42.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.436+0000 7fdf2ffff700 1 --2- 172.21.15.3:0/2977371405 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf3009d6f0 0x7fdf3011deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:42.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.436+0000 7fdf2f7fe700 1 --2- 172.21.15.3:0/2977371405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf3009e000 0x7fdf3011e3f0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400fbb0 tx=0x7fdf2400fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:42.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.436+0000 7fdf2cff9700 1 -- 172.21.15.3:0/2977371405 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2400dd10 con 0x7fdf3009e000 2023-10-03T10:19:42.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.436+0000 7fdf2cff9700 1 -- 172.21.15.3:0/2977371405 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2400de70 con 0x7fdf3009e000 2023-10-03T10:19:42.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.436+0000 7fdf36978700 1 -- 172.21.15.3:0/2977371405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf30121610 con 0x7fdf3009e000 2023-10-03T10:19:42.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.437+0000 7fdf2cff9700 1 -- 172.21.15.3:0/2977371405 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24018640 con 0x7fdf3009e000 2023-10-03T10:19:42.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.437+0000 7fdf36978700 1 -- 172.21.15.3:0/2977371405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3009c700 con 0x7fdf3009e000 2023-10-03T10:19:42.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.438+0000 7fdf2cff9700 1 -- 172.21.15.3:0/2977371405 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdf240187a0 con 0x7fdf3009e000 2023-10-03T10:19:42.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.438+0000 7fdf2cff9700 1 -- 172.21.15.3:0/2977371405 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fdf2403fe80 con 0x7fdf3009e000 2023-10-03T10:19:42.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.439+0000 7fdf36978700 1 -- 172.21.15.3:0/2977371405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf30074ea0 con 0x7fdf3009e000 2023-10-03T10:19:42.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.444+0000 7fdf2cff9700 1 -- 172.21.15.3:0/2977371405 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdf2401d020 con 0x7fdf3009e000 2023-10-03T10:19:42.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.626+0000 7f2ccc7ae700 1 -- 172.21.15.3:0/2336547394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2cc409e130 con 0x7f2cc409d9f0 2023-10-03T10:19:42.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.626+0000 7f2cbaffd700 1 -- 172.21.15.3:0/2336547394 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2cc409e130 con 0x7f2cc409d9f0 2023-10-03T10:19:42.626 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:42.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.629+0000 7f2ccc7ae700 1 -- 172.21.15.3:0/2336547394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cc409d9f0 msgr2=0x7f2cc4127560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:42.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.629+0000 7f2ccc7ae700 1 --2- 172.21.15.3:0/2336547394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cc409d9f0 0x7f2cc4127560 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0000f60 tx=0x7f2cc000ca30 comp rx=0 tx=0).stop 2023-10-03T10:19:42.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.630+0000 7f2ccc7ae700 1 -- 172.21.15.3:0/2336547394 shutdown_connections 2023-10-03T10:19:42.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.630+0000 7f2ccc7ae700 1 --2- 172.21.15.3:0/2336547394 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2cc4136a70 0x7f2cc4127fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.630+0000 7f2ccc7ae700 1 --2- 172.21.15.3:0/2336547394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cc409d9f0 0x7f2cc4127560 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.630+0000 7f2ccc7ae700 1 --2- 172.21.15.3:0/2336547394 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2cc41316c0 0x7f2cc4127aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.630+0000 7f2ccc7ae700 1 -- 172.21.15.3:0/2336547394 >> 172.21.15.3:0/2336547394 conn(0x7f2cc4066f50 msgr2=0x7f2cc4138730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:42.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.631+0000 7f2ccc7ae700 1 -- 172.21.15.3:0/2336547394 shutdown_connections 2023-10-03T10:19:42.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.631+0000 7f2ccc7ae700 1 -- 172.21.15.3:0/2336547394 wait complete. 2023-10-03T10:19:42.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.632+0000 7f5882396700 1 -- 172.21.15.3:0/3168280997 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f587c074ea0 con 0x7f587c1341d0 2023-10-03T10:19:42.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.632+0000 7f5878ff9700 1 -- 172.21.15.3:0/3168280997 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f587401a360 con 0x7f587c1341d0 2023-10-03T10:19:42.632 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:42.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.635+0000 7f5882396700 1 -- 172.21.15.3:0/3168280997 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f587c1341d0 msgr2=0x7f587c1202f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:42.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.635+0000 7f5882396700 1 --2- 172.21.15.3:0/3168280997 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f587c1341d0 0x7f587c1202f0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f587400cc50 tx=0x7f587400cf60 comp rx=0 tx=0).stop 2023-10-03T10:19:42.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.635+0000 7f5882396700 1 -- 172.21.15.3:0/3168280997 shutdown_connections 2023-10-03T10:19:42.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.635+0000 7f5882396700 1 --2- 172.21.15.3:0/3168280997 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f587c1341d0 0x7f587c1202f0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.635+0000 7f5882396700 1 --2- 172.21.15.3:0/3168280997 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f587c09e390 0x7f587c11f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.635+0000 7f5882396700 1 --2- 172.21.15.3:0/3168280997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587c1338c0 0x7f587c11fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.635+0000 7f5882396700 1 -- 172.21.15.3:0/3168280997 >> 172.21.15.3:0/3168280997 conn(0x7f587c0a2aa0 msgr2=0x7f587c131ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:42.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.642+0000 7f5882396700 1 -- 172.21.15.3:0/3168280997 shutdown_connections 2023-10-03T10:19:42.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.642+0000 7f5882396700 1 -- 172.21.15.3:0/3168280997 wait complete. 2023-10-03T10:19:42.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.646+0000 7f4a179a8700 1 -- 172.21.15.3:0/3459576846 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4a10121490 con 0x7f4a1009d6f0 2023-10-03T10:19:42.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.646+0000 7f4a05ffb700 1 -- 172.21.15.3:0/3459576846 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4a0c028020 con 0x7f4a1009d6f0 2023-10-03T10:19:42.647 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:42.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.650+0000 7f4a179a8700 1 -- 172.21.15.3:0/3459576846 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a1009d6f0 msgr2=0x7f4a1011e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:42.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.650+0000 7f4a179a8700 1 --2- 172.21.15.3:0/3459576846 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a1009d6f0 0x7f4a1011e070 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f4a1009ea70 tx=0x7f4a0c004750 comp rx=0 tx=0).stop 2023-10-03T10:19:42.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.650+0000 7f4a179a8700 1 -- 172.21.15.3:0/3459576846 shutdown_connections 2023-10-03T10:19:42.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.650+0000 7f4a179a8700 1 --2- 172.21.15.3:0/3459576846 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4a1009e410 0x7f4a1011eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.650+0000 7f4a179a8700 1 --2- 172.21.15.3:0/3459576846 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a1009d6f0 0x7f4a1011e070 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.650+0000 7f4a179a8700 1 --2- 172.21.15.3:0/3459576846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a1009e000 0x7f4a1011e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.650+0000 7f4a179a8700 1 -- 172.21.15.3:0/3459576846 >> 172.21.15.3:0/3459576846 conn(0x7f4a10066f50 msgr2=0x7f4a1006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:42.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.650+0000 7f4a179a8700 1 -- 172.21.15.3:0/3459576846 shutdown_connections 2023-10-03T10:19:42.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.650+0000 7f4a179a8700 1 -- 172.21.15.3:0/3459576846 wait complete. 2023-10-03T10:19:42.652 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:42.658 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:42.664 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:42.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.676+0000 7fdf36978700 1 -- 172.21.15.3:0/2977371405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdf3006c360 con 0x7fdf3009e000 2023-10-03T10:19:42.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.677+0000 7fdf2cff9700 1 -- 172.21.15.3:0/2977371405 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdf24017990 con 0x7fdf3009e000 2023-10-03T10:19:42.677 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:42.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.679+0000 7fdf36978700 1 -- 172.21.15.3:0/2977371405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf3009e000 msgr2=0x7fdf3011e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:42.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.679+0000 7fdf36978700 1 --2- 172.21.15.3:0/2977371405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf3009e000 0x7fdf3011e3f0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400fbb0 tx=0x7fdf2400fec0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.679+0000 7fdf36978700 1 -- 172.21.15.3:0/2977371405 shutdown_connections 2023-10-03T10:19:42.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.679+0000 7fdf36978700 1 --2- 172.21.15.3:0/2977371405 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdf3009e410 0x7fdf3011e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.679+0000 7fdf36978700 1 --2- 172.21.15.3:0/2977371405 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fdf3009d6f0 0x7fdf3011deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.679+0000 7fdf36978700 1 --2- 172.21.15.3:0/2977371405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf3009e000 0x7fdf3011e3f0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:42.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.679+0000 7fdf36978700 1 -- 172.21.15.3:0/2977371405 >> 172.21.15.3:0/2977371405 conn(0x7fdf30066f50 msgr2=0x7fdf30070d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:42.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.679+0000 7fdf36978700 1 -- 172.21.15.3:0/2977371405 shutdown_connections 2023-10-03T10:19:42.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:42.679+0000 7fdf36978700 1 -- 172.21.15.3:0/2977371405 wait complete. 2023-10-03T10:19:42.693 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:43.652 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:43.659 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:43.665 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:43.693 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:43.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.814+0000 7f93bbfdc700 1 Processor -- start 2023-10-03T10:19:43.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.815+0000 7f93bbfdc700 1 -- start start 2023-10-03T10:19:43.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.815+0000 7f93bbfdc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f93b4131810 0x7f93b4131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:43.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.815+0000 7f93bbfdc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f93b4099a40 0x7f93b4099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:43.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.815+0000 7f93bbfdc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f93b409a3b0 0x7f93b4138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:43.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.815+0000 7f93bbfdc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b409b580 con 0x7f93b409a3b0 2023-10-03T10:19:43.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.815+0000 7f93bbfdc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b41313c0 con 0x7f93b4099a40 2023-10-03T10:19:43.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.815+0000 7f93bbfdc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f93b406e8c0 con 0x7f93b4131810 2023-10-03T10:19:43.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.815+0000 7f93b9d78700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f93b4131810 0x7f93b4131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:43.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.816+0000 7f93b9d78700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f93b4131810 0x7f93b4131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:44496/0 (socket says 172.21.15.3:44496) 2023-10-03T10:19:43.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.816+0000 7f93b9d78700 1 -- 172.21.15.3:0/1697407195 learned_addr learned my addr 172.21.15.3:0/1697407195 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:43.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.816+0000 7f93b9577700 1 --2- 172.21.15.3:0/1697407195 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f93b4099a40 0x7f93b4099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:43.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.816+0000 7f93ba579700 1 --2- 172.21.15.3:0/1697407195 >> v2:172.21.15.3:3300/0 conn(0x7f93b409a3b0 0x7f93b4138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:43.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.816+0000 7f93b9d78700 1 -- 172.21.15.3:0/1697407195 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f93b4099a40 msgr2=0x7f93b4099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:43.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.816+0000 7f93b9d78700 1 --2- 172.21.15.3:0/1697407195 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f93b4099a40 0x7f93b4099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:43.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.816+0000 7f93b9d78700 1 -- 172.21.15.3:0/1697407195 >> v2:172.21.15.3:3300/0 conn(0x7f93b409a3b0 msgr2=0x7f93b4138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:43.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.816+0000 7f93b9d78700 1 --2- 172.21.15.3:0/1697407195 >> v2:172.21.15.3:3300/0 conn(0x7f93b409a3b0 0x7f93b4138650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:43.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.816+0000 7f93b9d78700 1 -- 172.21.15.3:0/1697407195 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b409a820 con 0x7f93b4131810 2023-10-03T10:19:43.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.816+0000 7f93b9d78700 1 --2- 172.21.15.3:0/1697407195 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f93b4131810 0x7f93b4131be0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f93b00098f0 tx=0x7f93b0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fae8dc1db009869 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:43.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.816+0000 7f93b8d76700 1 -- 172.21.15.3:0/1697407195 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0011830 con 0x7f93b4131810 2023-10-03T10:19:43.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.817+0000 7f93b8d76700 1 -- 172.21.15.3:0/1697407195 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b0011990 con 0x7f93b4131810 2023-10-03T10:19:43.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.817+0000 7f93b8d76700 1 -- 172.21.15.3:0/1697407195 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0011d20 con 0x7f93b4131810 2023-10-03T10:19:43.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.818+0000 7f93bbfdc700 1 -- 172.21.15.3:0/1697407195 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f93b4131810 msgr2=0x7f93b4131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:43.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.818+0000 7f93bbfdc700 1 --2- 172.21.15.3:0/1697407195 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f93b4131810 0x7f93b4131be0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f93b00098f0 tx=0x7f93b0009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:43.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.818+0000 7f93bbfdc700 1 -- 172.21.15.3:0/1697407195 shutdown_connections 2023-10-03T10:19:43.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.818+0000 7f93bbfdc700 1 --2- 172.21.15.3:0/1697407195 >> v2:172.21.15.3:3300/0 conn(0x7f93b409a3b0 0x7f93b4138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:43.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.818+0000 7f93bbfdc700 1 --2- 172.21.15.3:0/1697407195 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f93b4131810 0x7f93b4131be0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:43.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.818+0000 7f93bbfdc700 1 --2- 172.21.15.3:0/1697407195 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f93b4099a40 0x7f93b4099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:43.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.818+0000 7f93bbfdc700 1 -- 172.21.15.3:0/1697407195 >> 172.21.15.3:0/1697407195 conn(0x7f93b40a2aa0 msgr2=0x7f93b40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:43.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.818+0000 7f93bbfdc700 1 -- 172.21.15.3:0/1697407195 shutdown_connections 2023-10-03T10:19:43.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.818+0000 7f93bbfdc700 1 -- 172.21.15.3:0/1697407195 wait complete. 2023-10-03T10:19:43.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.819+0000 7f93bbfdc700 1 Processor -- start 2023-10-03T10:19:43.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.819+0000 7f93bbfdc700 1 -- start start 2023-10-03T10:19:43.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.819+0000 7f93bbfdc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93b4099a40 0x7f93b411b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:43.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.819+0000 7f93bbfdc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f93b409a3b0 0x7f93b411bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:43.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.819+0000 7f93bbfdc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f93b4131810 0x7f93b411c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:43.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.819+0000 7f93bbfdc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b411cb00 con 0x7f93b4099a40 2023-10-03T10:19:43.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.819+0000 7f93bbfdc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b411d890 con 0x7f93b409a3b0 2023-10-03T10:19:43.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.819+0000 7f93bbfdc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f93b411e950 con 0x7f93b4131810 2023-10-03T10:19:43.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.820+0000 7f93b9577700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f93b409a3b0 0x7f93b411bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:43.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.820+0000 7f93b9577700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f93b409a3b0 0x7f93b411bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:43104/0 (socket says 172.21.15.3:43104) 2023-10-03T10:19:43.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.820+0000 7f93b9577700 1 -- 172.21.15.3:0/2067880624 learned_addr learned my addr 172.21.15.3:0/2067880624 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:43.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.820+0000 7f93ba579700 1 --2- 172.21.15.3:0/2067880624 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f93b4131810 0x7f93b411c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:43.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.820+0000 7f93b9577700 1 -- 172.21.15.3:0/2067880624 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f93b4131810 msgr2=0x7f93b411c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:43.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.820+0000 7f93b9577700 1 --2- 172.21.15.3:0/2067880624 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f93b4131810 0x7f93b411c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:43.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.820+0000 7f93b9577700 1 -- 172.21.15.3:0/2067880624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93b4099a40 msgr2=0x7f93b411b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:43.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.820+0000 7f93b9d78700 1 --2- 172.21.15.3:0/2067880624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93b4099a40 0x7f93b411b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:43.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.820+0000 7f93b9577700 1 --2- 172.21.15.3:0/2067880624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93b4099a40 0x7f93b411b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:43.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.820+0000 7f93b9577700 1 -- 172.21.15.3:0/2067880624 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b0009550 con 0x7f93b409a3b0 2023-10-03T10:19:43.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.820+0000 7f93ba579700 1 --2- 172.21.15.3:0/2067880624 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f93b4131810 0x7f93b411c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:43.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.820+0000 7f93b9d78700 1 --2- 172.21.15.3:0/2067880624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93b4099a40 0x7f93b411b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:19:43.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.820+0000 7f93b9577700 1 --2- 172.21.15.3:0/2067880624 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f93b409a3b0 0x7f93b411bbe0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f93a4012f90 tx=0x7f93a40103a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:43.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.821+0000 7f93aa7fc700 1 -- 172.21.15.3:0/2067880624 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a400dbb0 con 0x7f93b409a3b0 2023-10-03T10:19:43.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.821+0000 7f93aa7fc700 1 -- 172.21.15.3:0/2067880624 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a4013040 con 0x7f93b409a3b0 2023-10-03T10:19:43.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.821+0000 7f93bbfdc700 1 -- 172.21.15.3:0/2067880624 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b406eb00 con 0x7f93b409a3b0 2023-10-03T10:19:43.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.821+0000 7f93aa7fc700 1 -- 172.21.15.3:0/2067880624 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a4010dd0 con 0x7f93b409a3b0 2023-10-03T10:19:43.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.821+0000 7f93bbfdc700 1 -- 172.21.15.3:0/2067880624 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b411d2b0 con 0x7f93b409a3b0 2023-10-03T10:19:43.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.822+0000 7f93aa7fc700 1 -- 172.21.15.3:0/2067880624 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f93a4018480 con 0x7f93b409a3b0 2023-10-03T10:19:43.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.822+0000 7f93aa7fc700 1 -- 172.21.15.3:0/2067880624 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f93a4043700 con 0x7f93b409a3b0 2023-10-03T10:19:43.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.822+0000 7f93bbfdc700 1 -- 172.21.15.3:0/2067880624 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93b4074ea0 con 0x7f93b409a3b0 2023-10-03T10:19:43.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.831+0000 7f93aa7fc700 1 -- 172.21.15.3:0/2067880624 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f93a4016930 con 0x7f93b409a3b0 2023-10-03T10:19:43.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.847+0000 7f2f3201e700 1 Processor -- start 2023-10-03T10:19:43.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.847+0000 7f2f3201e700 1 -- start start 2023-10-03T10:19:43.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.848+0000 7f2f3201e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2f2c09e1d0 0x7f2f2c135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:43.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.848+0000 7f2f3201e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2f2c135e50 0x7f2f2c138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:43.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.848+0000 7f2f3201e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2c09f130 con 0x7f2f2c09d6f0 2023-10-03T10:19:43.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.848+0000 7f2f3201e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c09f520 con 0x7f2f2c135e50 2023-10-03T10:19:43.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.848+0000 7f2f3201e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c09e930 con 0x7f2f2c09e1d0 2023-10-03T10:19:43.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.848+0000 7f2f2b7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2f2c09d6f0 0x7f2f2c09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60392/0 (socket says 172.21.15.3:60392) 2023-10-03T10:19:43.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.848+0000 7f2f2bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2f2c135e50 0x7f2f2c138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:43.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.848+0000 7f2f2b7fe700 1 -- 172.21.15.3:0/1348965319 learned_addr learned my addr 172.21.15.3:0/1348965319 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:43.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.848+0000 7f2f2affd700 1 --2- 172.21.15.3:0/1348965319 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2f2c09e1d0 0x7f2f2c135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:43.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.849+0000 7f2f2a7fc700 1 -- 172.21.15.3:0/1348965319 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3616007013 0 0) 0x7f2f2c09f130 con 0x7f2f2c09d6f0 2023-10-03T10:19:43.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.849+0000 7f2f2a7fc700 1 -- 172.21.15.3:0/1348965319 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f140035b0 con 0x7f2f2c09d6f0 2023-10-03T10:19:43.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.849+0000 7f2f2bfff700 1 -- 172.21.15.3:0/1348965319 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2f2c09e1d0 msgr2=0x7f2f2c135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:43.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.849+0000 7f2f2bfff700 1 --2- 172.21.15.3:0/1348965319 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2f2c09e1d0 0x7f2f2c135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:43.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.849+0000 7f2f2bfff700 1 -- 172.21.15.3:0/1348965319 >> v1:172.21.15.3:6789/0 conn(0x7f2f2c09d6f0 legacy=0x7f2f2c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:43.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.849+0000 7f2f2bfff700 1 -- 172.21.15.3:0/1348965319 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2c09b2d0 con 0x7f2f2c135e50 2023-10-03T10:19:43.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.849+0000 7f2f2bfff700 1 --2- 172.21.15.3:0/1348965319 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2f2c135e50 0x7f2f2c138230 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f2f2400bfd0 tx=0x7f2f2400d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ce3818dc514df0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:43.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.849+0000 7f2f2a7fc700 1 -- 172.21.15.3:0/1348965319 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2400d970 con 0x7f2f2c135e50 2023-10-03T10:19:43.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.849+0000 7f2f2a7fc700 1 -- 172.21.15.3:0/1348965319 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2400dad0 con 0x7f2f2c135e50 2023-10-03T10:19:43.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.849+0000 7f2f2a7fc700 1 -- 172.21.15.3:0/1348965319 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2400de80 con 0x7f2f2c135e50 2023-10-03T10:19:43.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.850+0000 7fcff5358700 1 Processor -- start 2023-10-03T10:19:43.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.850+0000 7fcff5358700 1 -- start start 2023-10-03T10:19:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.850+0000 7f2f3201e700 1 -- 172.21.15.3:0/1348965319 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2f2c135e50 msgr2=0x7f2f2c138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.850+0000 7f2f3201e700 1 --2- 172.21.15.3:0/1348965319 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2f2c135e50 0x7f2f2c138230 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f2f2400bfd0 tx=0x7f2f2400d330 comp rx=0 tx=0).stop 2023-10-03T10:19:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.851+0000 7f2f3201e700 1 -- 172.21.15.3:0/1348965319 shutdown_connections 2023-10-03T10:19:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.851+0000 7f2f3201e700 1 --2- 172.21.15.3:0/1348965319 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2f2c135e50 0x7f2f2c138230 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.851+0000 7f2f3201e700 1 --2- 172.21.15.3:0/1348965319 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2f2c09e1d0 0x7f2f2c135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.851+0000 7f2f3201e700 1 -- 172.21.15.3:0/1348965319 >> 172.21.15.3:0/1348965319 conn(0x7f2f2c066f50 msgr2=0x7f2f2c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:43.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.851+0000 7fcff5358700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcff009d6f0 0x7fcff009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:43.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.851+0000 7fcff5358700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcff009e440 0x7fcff0134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:43.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.851+0000 7fcff5358700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcff009b120 con 0x7fcff009e000 2023-10-03T10:19:43.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.851+0000 7fcff5358700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcff006b460 con 0x7fcff009d6f0 2023-10-03T10:19:43.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.851+0000 7fcff5358700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcff009a7e0 con 0x7fcff009e440 2023-10-03T10:19:43.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.851+0000 7f2f3201e700 1 -- 172.21.15.3:0/1348965319 shutdown_connections 2023-10-03T10:19:43.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.851+0000 7fcfeeffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcff009d6f0 0x7fcff009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:43.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.851+0000 7fcfeeffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcff009d6f0 0x7fcff009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38610/0 (socket says 172.21.15.3:38610) 2023-10-03T10:19:43.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.851+0000 7fcfeeffd700 1 -- 172.21.15.3:0/2617603241 learned_addr learned my addr 172.21.15.3:0/2617603241 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:43.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.851+0000 7f2f3201e700 1 -- 172.21.15.3:0/1348965319 wait complete. 2023-10-03T10:19:43.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.851+0000 7fcfef7fe700 1 --2- 172.21.15.3:0/2617603241 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcff009e440 0x7fcff0134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:43.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.851+0000 7fcfef7fe700 1 -- 172.21.15.3:0/2617603241 >> v2:172.21.15.3:3300/0 conn(0x7fcff009d6f0 msgr2=0x7fcff009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:43.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.851+0000 7fcfef7fe700 1 --2- 172.21.15.3:0/2617603241 >> v2:172.21.15.3:3300/0 conn(0x7fcff009d6f0 0x7fcff009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:43.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.851+0000 7fcfef7fe700 1 -- 172.21.15.3:0/2617603241 >> v1:172.21.15.3:6789/0 conn(0x7fcff009e000 legacy=0x7fcff01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:43.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.851+0000 7fcfef7fe700 1 -- 172.21.15.3:0/2617603241 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff009a500 con 0x7fcff009e440 2023-10-03T10:19:43.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.852+0000 7fcfef7fe700 1 --2- 172.21.15.3:0/2617603241 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcff009e440 0x7fcff0134100 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009dc0 tx=0x7fcfe800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef01be7afafdc7da server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:43.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.852+0000 7fcfedffb700 1 -- 172.21.15.3:0/2617603241 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe800e070 con 0x7fcff009e440 2023-10-03T10:19:43.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.852+0000 7fcfedffb700 1 -- 172.21.15.3:0/2617603241 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe800bae0 con 0x7fcff009e440 2023-10-03T10:19:43.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.852+0000 7fcfedffb700 1 -- 172.21.15.3:0/2617603241 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe800be90 con 0x7fcff009e440 2023-10-03T10:19:43.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.852+0000 7f2f3201e700 1 Processor -- start 2023-10-03T10:19:43.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.853+0000 7f2f3201e700 1 -- start start 2023-10-03T10:19:43.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.853+0000 7fcff5358700 1 -- 172.21.15.3:0/2617603241 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcff009e440 msgr2=0x7fcff0134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:43.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.853+0000 7fcff5358700 1 --2- 172.21.15.3:0/2617603241 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcff009e440 0x7fcff0134100 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009dc0 tx=0x7fcfe800b330 comp rx=0 tx=0).stop 2023-10-03T10:19:43.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.853+0000 7fcff5358700 1 -- 172.21.15.3:0/2617603241 shutdown_connections 2023-10-03T10:19:43.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.853+0000 7fcff5358700 1 --2- 172.21.15.3:0/2617603241 >> v2:172.21.15.3:3300/0 conn(0x7fcff009d6f0 0x7fcff009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:43.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.853+0000 7fcff5358700 1 --2- 172.21.15.3:0/2617603241 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcff009e440 0x7fcff0134100 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:43.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.853+0000 7fcff5358700 1 -- 172.21.15.3:0/2617603241 >> 172.21.15.3:0/2617603241 conn(0x7fcff0066f50 msgr2=0x7fcff009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:43.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.853+0000 7fcff5358700 1 -- 172.21.15.3:0/2617603241 shutdown_connections 2023-10-03T10:19:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.853+0000 7f2f3201e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2f2c09d6f0 0x7f2f2c11b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.853+0000 7f2f3201e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2f2c09e1d0 0x7f2f2c11b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.853+0000 7f2f3201e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f2c135e50 0x7f2f2c11bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.853+0000 7f2f3201e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c11ecb0 con 0x7f2f2c135e50 2023-10-03T10:19:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.853+0000 7f2f3201e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c09c700 con 0x7f2f2c09d6f0 2023-10-03T10:19:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.853+0000 7f2f3201e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c11e440 con 0x7f2f2c09e1d0 2023-10-03T10:19:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.854+0000 7f2f2b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2f2c09d6f0 0x7f2f2c11b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.854+0000 7f2f2b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2f2c09d6f0 0x7f2f2c11b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:43112/0 (socket says 172.21.15.3:43112) 2023-10-03T10:19:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.854+0000 7f2f2b7fe700 1 -- 172.21.15.3:0/2108206471 learned_addr learned my addr 172.21.15.3:0/2108206471 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.854+0000 7f2f2b7fe700 1 -- 172.21.15.3:0/2108206471 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2f2c09e1d0 msgr2=0x7f2f2c11b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.854+0000 7f2f2b7fe700 1 --2- 172.21.15.3:0/2108206471 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2f2c09e1d0 0x7f2f2c11b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.854+0000 7f2f2b7fe700 1 -- 172.21.15.3:0/2108206471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f2c135e50 msgr2=0x7f2f2c11bc10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:19:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.854+0000 7f2f2b7fe700 1 --2- 172.21.15.3:0/2108206471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f2c135e50 0x7f2f2c11bc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.854+0000 7f2f2b7fe700 1 -- 172.21.15.3:0/2108206471 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2400bc80 con 0x7f2f2c09d6f0 2023-10-03T10:19:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.854+0000 7f2f2b7fe700 1 --2- 172.21.15.3:0/2108206471 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2f2c09d6f0 0x7f2f2c11b190 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c001eb0 tx=0x7f2f1c00c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.854+0000 7f2f13fff700 1 -- 172.21.15.3:0/2108206471 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c005d70 con 0x7f2f2c09d6f0 2023-10-03T10:19:43.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.853+0000 7fcff5358700 1 -- 172.21.15.3:0/2617603241 wait complete. 2023-10-03T10:19:43.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.855+0000 7fcff5358700 1 Processor -- start 2023-10-03T10:19:43.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.855+0000 7f13a048a700 1 Processor -- start 2023-10-03T10:19:43.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.855+0000 7f13a048a700 1 -- start start 2023-10-03T10:19:43.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.854+0000 7f2f13fff700 1 -- 172.21.15.3:0/2108206471 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1c0070d0 con 0x7f2f2c09d6f0 2023-10-03T10:19:43.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.854+0000 7f2f13fff700 1 -- 172.21.15.3:0/2108206471 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c007460 con 0x7f2f2c09d6f0 2023-10-03T10:19:43.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.855+0000 7f2f3201e700 1 -- 172.21.15.3:0/2108206471 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f2c0a2140 con 0x7f2f2c09d6f0 2023-10-03T10:19:43.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.855+0000 7f2f3201e700 1 -- 172.21.15.3:0/2108206471 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2c11ce00 con 0x7f2f2c09d6f0 2023-10-03T10:19:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.855+0000 7fcff5358700 1 -- start start 2023-10-03T10:19:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.856+0000 7fcff5358700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcff009d6f0 0x7fcff0119770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.856+0000 7fcff5358700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcff009e000 0x7fcff0119cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.856+0000 7fcff5358700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcff009e440 0x7fcff011a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.856+0000 7fcff5358700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff011cd30 con 0x7fcff009d6f0 2023-10-03T10:19:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.856+0000 7fcff5358700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff01b94b0 con 0x7fcff009e000 2023-10-03T10:19:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.856+0000 7fcff5358700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcff011cf80 con 0x7fcff009e440 2023-10-03T10:19:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.856+0000 7fcfee7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcff009e000 0x7fcff0119cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.856+0000 7fcfee7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcff009e000 0x7fcff0119cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:43122/0 (socket says 172.21.15.3:43122) 2023-10-03T10:19:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.856+0000 7fcfee7fc700 1 -- 172.21.15.3:0/3829731224 learned_addr learned my addr 172.21.15.3:0/3829731224 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.856+0000 7f13a048a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f13980a0a30 0x7f13980a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.856+0000 7f13a048a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13980a1340 0x7f1398133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.856+0000 7f13a048a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1398133fa0 0x7f1398134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.856+0000 7f13a048a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f139809edd0 con 0x7f13980a0a30 2023-10-03T10:19:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.856+0000 7f13a048a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f139809e1f0 con 0x7f13980a1340 2023-10-03T10:19:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.856+0000 7f13a048a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f139809dc50 con 0x7f1398133fa0 2023-10-03T10:19:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.856+0000 7f139e226700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f13980a0a30 0x7f13980a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.856+0000 7f139e226700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f13980a0a30 0x7f13980a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38626/0 (socket says 172.21.15.3:38626) 2023-10-03T10:19:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.856+0000 7f139e226700 1 -- 172.21.15.3:0/521846961 learned_addr learned my addr 172.21.15.3:0/521846961 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:43.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.856+0000 7f2f13fff700 1 -- 172.21.15.3:0/2108206471 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2f1c0075c0 con 0x7f2f2c09d6f0 2023-10-03T10:19:43.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.856+0000 7fcfeeffd700 1 --2- 172.21.15.3:0/3829731224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcff009d6f0 0x7fcff0119770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:43.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.857+0000 7fcfee7fc700 1 -- 172.21.15.3:0/3829731224 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcff009e440 msgr2=0x7fcff011a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:43.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.857+0000 7fcfee7fc700 1 --2- 172.21.15.3:0/3829731224 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcff009e440 0x7fcff011a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:43.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.857+0000 7fcfee7fc700 1 -- 172.21.15.3:0/3829731224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcff009d6f0 msgr2=0x7fcff0119770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:43.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.857+0000 7fcfee7fc700 1 --2- 172.21.15.3:0/3829731224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcff009d6f0 0x7fcff0119770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:43.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.857+0000 7fcfee7fc700 1 -- 172.21.15.3:0/3829731224 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe8009a70 con 0x7fcff009e000 2023-10-03T10:19:43.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.857+0000 7f2f3201e700 1 -- 172.21.15.3:0/2108206471 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f2c074ea0 con 0x7f2f2c09d6f0 2023-10-03T10:19:43.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.857+0000 7f2f13fff700 1 -- 172.21.15.3:0/2108206471 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f2f1c03ec10 con 0x7f2f2c09d6f0 2023-10-03T10:19:43.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.857+0000 7f139da25700 1 --2- 172.21.15.3:0/521846961 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13980a1340 0x7f1398133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:43.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.857+0000 7f139e226700 1 -- 172.21.15.3:0/521846961 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1398133fa0 msgr2=0x7f1398134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:43.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.857+0000 7f139e226700 1 --2- 172.21.15.3:0/521846961 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1398133fa0 0x7f1398134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:43.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.857+0000 7f139e226700 1 -- 172.21.15.3:0/521846961 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13980a1340 msgr2=0x7f1398133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:43.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.857+0000 7f139e226700 1 --2- 172.21.15.3:0/521846961 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13980a1340 0x7f1398133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:43.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.857+0000 7f139e226700 1 -- 172.21.15.3:0/521846961 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13980a1fe0 con 0x7f13980a0a30 2023-10-03T10:19:43.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.857+0000 7f139da25700 1 --2- 172.21.15.3:0/521846961 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13980a1340 0x7f1398133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:43.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.857+0000 7f139e226700 1 --2- 172.21.15.3:0/521846961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13980a0a30 0x7f13980a0e00 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f13940098f0 tx=0x7f1394009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19aee354eae29191 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:43.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.858+0000 7f139d224700 1 -- 172.21.15.3:0/521846961 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1394011830 con 0x7f13980a0a30 2023-10-03T10:19:43.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.858+0000 7f139d224700 1 -- 172.21.15.3:0/521846961 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1394011990 con 0x7f13980a0a30 2023-10-03T10:19:43.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.859+0000 7f139d224700 1 -- 172.21.15.3:0/521846961 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1394011d20 con 0x7f13980a0a30 2023-10-03T10:19:43.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.859+0000 7f13a048a700 1 -- 172.21.15.3:0/521846961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13980a0a30 msgr2=0x7f13980a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:43.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.859+0000 7f13a048a700 1 --2- 172.21.15.3:0/521846961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13980a0a30 0x7f13980a0e00 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f13940098f0 tx=0x7f1394009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:43.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.859+0000 7f13a048a700 1 -- 172.21.15.3:0/521846961 shutdown_connections 2023-10-03T10:19:43.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.859+0000 7f13a048a700 1 --2- 172.21.15.3:0/521846961 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f1398133fa0 0x7f1398134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:43.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.859+0000 7f13a048a700 1 --2- 172.21.15.3:0/521846961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13980a0a30 0x7f13980a0e00 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:43.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.859+0000 7f13a048a700 1 --2- 172.21.15.3:0/521846961 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13980a1340 0x7f1398133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:43.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.859+0000 7f13a048a700 1 -- 172.21.15.3:0/521846961 >> 172.21.15.3:0/521846961 conn(0x7f13980a6550 msgr2=0x7f13980a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:43.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.859+0000 7f13a048a700 1 -- 172.21.15.3:0/521846961 shutdown_connections 2023-10-03T10:19:43.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.859+0000 7f13a048a700 1 -- 172.21.15.3:0/521846961 wait complete. 2023-10-03T10:19:43.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.860+0000 7f13a048a700 1 Processor -- start 2023-10-03T10:19:43.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.861+0000 7f13a048a700 1 -- start start 2023-10-03T10:19:43.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.861+0000 7f13a048a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f13980a0a30 0x7f1398120220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:43.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.861+0000 7f13a048a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13980a1340 0x7f1398120760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:43.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.861+0000 7f13a048a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1398133fa0 0x7f1398120ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:43.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.861+0000 7f13a048a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1398121e30 con 0x7f13980a1340 2023-10-03T10:19:43.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.861+0000 7f13a048a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1398123080 con 0x7f1398133fa0 2023-10-03T10:19:43.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.861+0000 7f13a048a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1398122410 con 0x7f13980a0a30 2023-10-03T10:19:43.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.861+0000 7f139ea27700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1398133fa0 0x7f1398120ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:43.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.861+0000 7f139ea27700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1398133fa0 0x7f1398120ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:43132/0 (socket says 172.21.15.3:43132) 2023-10-03T10:19:43.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.861+0000 7f139ea27700 1 -- 172.21.15.3:0/1174352977 learned_addr learned my addr 172.21.15.3:0/1174352977 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:43.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.861+0000 7f139da25700 1 --2- 172.21.15.3:0/1174352977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13980a1340 0x7f1398120760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:43.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.861+0000 7f139e226700 1 --2- 172.21.15.3:0/1174352977 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f13980a0a30 0x7f1398120220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:43.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.862+0000 7f139ea27700 1 -- 172.21.15.3:0/1174352977 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f13980a0a30 msgr2=0x7f1398120220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:43.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.862+0000 7f139ea27700 1 --2- 172.21.15.3:0/1174352977 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f13980a0a30 0x7f1398120220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:43.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.862+0000 7f139ea27700 1 -- 172.21.15.3:0/1174352977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13980a1340 msgr2=0x7f1398120760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:43.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.862+0000 7f139ea27700 1 --2- 172.21.15.3:0/1174352977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13980a1340 0x7f1398120760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:43.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.862+0000 7f139ea27700 1 -- 172.21.15.3:0/1174352977 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13880094a0 con 0x7f1398133fa0 2023-10-03T10:19:43.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.862+0000 7f139ea27700 1 --2- 172.21.15.3:0/1174352977 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1398133fa0 0x7f1398120ca0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f139000b9a0 tx=0x7f139000bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:43.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.862+0000 7f138effd700 1 -- 172.21.15.3:0/1174352977 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139000d850 con 0x7f1398133fa0 2023-10-03T10:19:43.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.862+0000 7fcfee7fc700 1 --2- 172.21.15.3:0/3829731224 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcff009e000 0x7fcff0119cb0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fcfe400c510 tx=0x7fcfe400c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:43.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.862+0000 7f138effd700 1 -- 172.21.15.3:0/1174352977 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f139000d9d0 con 0x7f1398133fa0 2023-10-03T10:19:43.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.862+0000 7f13a048a700 1 -- 172.21.15.3:0/1174352977 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1394009550 con 0x7f1398133fa0 2023-10-03T10:19:43.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.862+0000 7f138effd700 1 -- 172.21.15.3:0/1174352977 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139000dd80 con 0x7f1398133fa0 2023-10-03T10:19:43.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.862+0000 7f13a048a700 1 -- 172.21.15.3:0/1174352977 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13980a35a0 con 0x7f1398133fa0 2023-10-03T10:19:43.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.863+0000 7f2f13fff700 1 -- 172.21.15.3:0/2108206471 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2f1c019c90 con 0x7f2f2c09d6f0 2023-10-03T10:19:43.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.863+0000 7fcfd77fe700 1 -- 172.21.15.3:0/3829731224 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe40052d0 con 0x7fcff009e000 2023-10-03T10:19:43.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.863+0000 7fcfd77fe700 1 -- 172.21.15.3:0/3829731224 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe4005430 con 0x7fcff009e000 2023-10-03T10:19:43.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.863+0000 7fcfd77fe700 1 -- 172.21.15.3:0/3829731224 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe40038c0 con 0x7fcff009e000 2023-10-03T10:19:43.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.863+0000 7fcff5358700 1 -- 172.21.15.3:0/3829731224 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff011c350 con 0x7fcff009e000 2023-10-03T10:19:43.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.863+0000 7fcff5358700 1 -- 172.21.15.3:0/3829731224 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff011bce0 con 0x7fcff009e000 2023-10-03T10:19:43.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.865+0000 7fcfd77fe700 1 -- 172.21.15.3:0/3829731224 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcfe4003a20 con 0x7fcff009e000 2023-10-03T10:19:43.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.865+0000 7fcfd77fe700 1 -- 172.21.15.3:0/3829731224 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fcfe403ec40 con 0x7fcff009e000 2023-10-03T10:19:43.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.866+0000 7fcff5358700 1 -- 172.21.15.3:0/3829731224 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff0074ea0 con 0x7fcff009e000 2023-10-03T10:19:43.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.864+0000 7f138effd700 1 -- 172.21.15.3:0/1174352977 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f139000e420 con 0x7f1398133fa0 2023-10-03T10:19:43.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.864+0000 7f138effd700 1 -- 172.21.15.3:0/1174352977 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f1390006e40 con 0x7f1398133fa0 2023-10-03T10:19:43.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.865+0000 7f13a048a700 1 -- 172.21.15.3:0/1174352977 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1398074ea0 con 0x7f1398133fa0 2023-10-03T10:19:43.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.874+0000 7f138effd700 1 -- 172.21.15.3:0/1174352977 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f139001c020 con 0x7f1398133fa0 2023-10-03T10:19:43.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:43.883+0000 7fcfd77fe700 1 -- 172.21.15.3:0/3829731224 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcfe400e280 con 0x7fcff009e000 2023-10-03T10:19:44.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.068+0000 7f93bbfdc700 1 -- 172.21.15.3:0/2067880624 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f93b411e500 con 0x7f93b409a3b0 2023-10-03T10:19:44.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.069+0000 7f93aa7fc700 1 -- 172.21.15.3:0/2067880624 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f93a4016340 con 0x7f93b409a3b0 2023-10-03T10:19:44.069 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:44.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.071+0000 7f93bbfdc700 1 -- 172.21.15.3:0/2067880624 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f93b409a3b0 msgr2=0x7f93b411bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:44.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.071+0000 7f93bbfdc700 1 --2- 172.21.15.3:0/2067880624 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f93b409a3b0 0x7f93b411bbe0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f93a4012f90 tx=0x7f93a40103a0 comp rx=0 tx=0).stop 2023-10-03T10:19:44.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.071+0000 7f93bbfdc700 1 -- 172.21.15.3:0/2067880624 shutdown_connections 2023-10-03T10:19:44.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.071+0000 7f93bbfdc700 1 --2- 172.21.15.3:0/2067880624 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f93b4131810 0x7f93b411c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:44.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.071+0000 7f93bbfdc700 1 --2- 172.21.15.3:0/2067880624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93b4099a40 0x7f93b411b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:44.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.071+0000 7f93bbfdc700 1 --2- 172.21.15.3:0/2067880624 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f93b409a3b0 0x7f93b411bbe0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:44.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.071+0000 7f93bbfdc700 1 -- 172.21.15.3:0/2067880624 >> 172.21.15.3:0/2067880624 conn(0x7f93b40a2aa0 msgr2=0x7f93b4070e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:44.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.072+0000 7f93bbfdc700 1 -- 172.21.15.3:0/2067880624 shutdown_connections 2023-10-03T10:19:44.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.074+0000 7f93bbfdc700 1 -- 172.21.15.3:0/2067880624 wait complete. 2023-10-03T10:19:44.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.084+0000 7f2f3201e700 1 -- 172.21.15.3:0/2108206471 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2f2c06c360 con 0x7f2f2c09d6f0 2023-10-03T10:19:44.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.085+0000 7f2f13fff700 1 -- 172.21.15.3:0/2108206471 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2f1c017030 con 0x7f2f2c09d6f0 2023-10-03T10:19:44.085 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:44.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.088+0000 7f2f3201e700 1 -- 172.21.15.3:0/2108206471 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2f2c09d6f0 msgr2=0x7f2f2c11b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:44.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.088+0000 7f2f3201e700 1 --2- 172.21.15.3:0/2108206471 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2f2c09d6f0 0x7f2f2c11b190 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c001eb0 tx=0x7f2f1c00c860 comp rx=0 tx=0).stop 2023-10-03T10:19:44.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.088+0000 7f2f3201e700 1 -- 172.21.15.3:0/2108206471 shutdown_connections 2023-10-03T10:19:44.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.088+0000 7f2f3201e700 1 --2- 172.21.15.3:0/2108206471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f2c135e50 0x7f2f2c11bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:44.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.088+0000 7f2f3201e700 1 --2- 172.21.15.3:0/2108206471 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2f2c09d6f0 0x7f2f2c11b190 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:44.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.088+0000 7f2f3201e700 1 --2- 172.21.15.3:0/2108206471 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f2f2c09e1d0 0x7f2f2c11b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:44.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.088+0000 7f2f3201e700 1 -- 172.21.15.3:0/2108206471 >> 172.21.15.3:0/2108206471 conn(0x7f2f2c066f50 msgr2=0x7f2f2c132680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:44.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.088+0000 7f2f3201e700 1 -- 172.21.15.3:0/2108206471 shutdown_connections 2023-10-03T10:19:44.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.088+0000 7f2f3201e700 1 -- 172.21.15.3:0/2108206471 wait complete. 2023-10-03T10:19:44.091 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:44.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.091+0000 7fcff5358700 1 -- 172.21.15.3:0/3829731224 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcff009f420 con 0x7fcff009e000 2023-10-03T10:19:44.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.092+0000 7fcfd77fe700 1 -- 172.21.15.3:0/3829731224 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcfe4017030 con 0x7fcff009e000 2023-10-03T10:19:44.092 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:44.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.094+0000 7fcff5358700 1 -- 172.21.15.3:0/3829731224 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcff009e000 msgr2=0x7fcff0119cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:44.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.094+0000 7fcff5358700 1 --2- 172.21.15.3:0/3829731224 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcff009e000 0x7fcff0119cb0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fcfe400c510 tx=0x7fcfe400c8d0 comp rx=0 tx=0).stop 2023-10-03T10:19:44.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.094+0000 7fcff5358700 1 -- 172.21.15.3:0/3829731224 shutdown_connections 2023-10-03T10:19:44.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.094+0000 7fcff5358700 1 --2- 172.21.15.3:0/3829731224 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcff009e440 0x7fcff011a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:44.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.094+0000 7fcff5358700 1 --2- 172.21.15.3:0/3829731224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcff009d6f0 0x7fcff0119770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:44.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.094+0000 7fcff5358700 1 --2- 172.21.15.3:0/3829731224 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcff009e000 0x7fcff0119cb0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:44.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.094+0000 7fcff5358700 1 -- 172.21.15.3:0/3829731224 >> 172.21.15.3:0/3829731224 conn(0x7fcff0066f50 msgr2=0x7fcff01bad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:44.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.095+0000 7fcff5358700 1 -- 172.21.15.3:0/3829731224 shutdown_connections 2023-10-03T10:19:44.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.095+0000 7fcff5358700 1 -- 172.21.15.3:0/3829731224 wait complete. 2023-10-03T10:19:44.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.101+0000 7f13a048a700 1 -- 172.21.15.3:0/1174352977 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f13980a28e0 con 0x7f1398133fa0 2023-10-03T10:19:44.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.101+0000 7f138effd700 1 -- 172.21.15.3:0/1174352977 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f139000c960 con 0x7f1398133fa0 2023-10-03T10:19:44.102 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:44.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.104+0000 7f13a048a700 1 -- 172.21.15.3:0/1174352977 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1398133fa0 msgr2=0x7f1398120ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:44.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.104+0000 7f13a048a700 1 --2- 172.21.15.3:0/1174352977 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1398133fa0 0x7f1398120ca0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f139000b9a0 tx=0x7f139000bcb0 comp rx=0 tx=0).stop 2023-10-03T10:19:44.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.104+0000 7f13a048a700 1 -- 172.21.15.3:0/1174352977 shutdown_connections 2023-10-03T10:19:44.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.104+0000 7f13a048a700 1 --2- 172.21.15.3:0/1174352977 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1398133fa0 0x7f1398120ca0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:44.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.104+0000 7f13a048a700 1 --2- 172.21.15.3:0/1174352977 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f13980a0a30 0x7f1398120220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:44.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.104+0000 7f13a048a700 1 --2- 172.21.15.3:0/1174352977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13980a1340 0x7f1398120760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:44.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.104+0000 7f13a048a700 1 -- 172.21.15.3:0/1174352977 >> 172.21.15.3:0/1174352977 conn(0x7f13980a6550 msgr2=0x7f13981bb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:44.106 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:44.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.104+0000 7f13a048a700 1 -- 172.21.15.3:0/1174352977 shutdown_connections 2023-10-03T10:19:44.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:44.104+0000 7f13a048a700 1 -- 172.21.15.3:0/1174352977 wait complete. 2023-10-03T10:19:44.111 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:44.119 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:45.092 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:45.106 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:45.111 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:45.120 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:45.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.257+0000 7f7958640700 1 Processor -- start 2023-10-03T10:19:45.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.257+0000 7f7958640700 1 -- start start 2023-10-03T10:19:45.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.258+0000 7f7958640700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f795009e930 0x7f795009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:45.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.258+0000 7f7958640700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f795009f240 0x7f79501317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:45.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.258+0000 7f7958640700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7950070940 con 0x7f795009f650 2023-10-03T10:19:45.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.258+0000 7f7958640700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f795006fd60 con 0x7f795009e930 2023-10-03T10:19:45.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.258+0000 7f7958640700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f795006fa80 con 0x7f795009f240 2023-10-03T10:19:45.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.258+0000 7f79563dc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f795009e930 0x7f795009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:45.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.258+0000 7f7955bdb700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f795009f240 0x7f79501317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:45.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.258+0000 7f79563dc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f795009e930 0x7f795009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:43148/0 (socket says 172.21.15.3:43148) 2023-10-03T10:19:45.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.258+0000 7f79563dc700 1 -- 172.21.15.3:0/2095821501 learned_addr learned my addr 172.21.15.3:0/2095821501 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:45.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.258+0000 7f7956bdd700 1 --1- 172.21.15.3:0/2095821501 >> v1:172.21.15.3:6789/0 conn(0x7f795009f650 0x7f7950131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60396/0 (socket says 172.21.15.3:60396) 2023-10-03T10:19:45.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.259+0000 7f79563dc700 1 -- 172.21.15.3:0/2095821501 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f795009f240 msgr2=0x7f79501317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:45.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.259+0000 7f79563dc700 1 --2- 172.21.15.3:0/2095821501 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f795009f240 0x7f79501317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.259+0000 7f79563dc700 1 -- 172.21.15.3:0/2095821501 >> v1:172.21.15.3:6789/0 conn(0x7f795009f650 legacy=0x7f7950131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:45.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.259+0000 7f79553da700 1 -- 172.21.15.3:0/2095821501 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3159715829 0 0) 0x7f7950070940 con 0x7f795009f650 2023-10-03T10:19:45.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.259+0000 7f79563dc700 1 -- 172.21.15.3:0/2095821501 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f795009fb60 con 0x7f795009e930 2023-10-03T10:19:45.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.259+0000 7f79563dc700 1 --2- 172.21.15.3:0/2095821501 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f795009e930 0x7f795009ed00 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f794c00dd40 tx=0x7f794c00f330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cf23bbad2965466 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:45.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.259+0000 7f79553da700 1 -- 172.21.15.3:0/2095821501 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c012070 con 0x7f795009e930 2023-10-03T10:19:45.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.259+0000 7f79553da700 1 -- 172.21.15.3:0/2095821501 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794c00de90 con 0x7f795009e930 2023-10-03T10:19:45.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.259+0000 7f79553da700 1 -- 172.21.15.3:0/2095821501 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c00fb30 con 0x7f795009e930 2023-10-03T10:19:45.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.260+0000 7f7958640700 1 -- 172.21.15.3:0/2095821501 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f795009e930 msgr2=0x7f795009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:45.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.260+0000 7f7958640700 1 --2- 172.21.15.3:0/2095821501 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f795009e930 0x7f795009ed00 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f794c00dd40 tx=0x7f794c00f330 comp rx=0 tx=0).stop 2023-10-03T10:19:45.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.260+0000 7f7958640700 1 -- 172.21.15.3:0/2095821501 shutdown_connections 2023-10-03T10:19:45.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.260+0000 7f7958640700 1 --2- 172.21.15.3:0/2095821501 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f795009e930 0x7f795009ed00 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.260+0000 7f7958640700 1 --2- 172.21.15.3:0/2095821501 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f795009f240 0x7f79501317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.260+0000 7f7958640700 1 -- 172.21.15.3:0/2095821501 >> 172.21.15.3:0/2095821501 conn(0x7f79500a4210 msgr2=0x7f79500a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:45.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.260+0000 7f7958640700 1 -- 172.21.15.3:0/2095821501 shutdown_connections 2023-10-03T10:19:45.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.260+0000 7f7958640700 1 -- 172.21.15.3:0/2095821501 wait complete. 2023-10-03T10:19:45.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.261+0000 7f7958640700 1 Processor -- start 2023-10-03T10:19:45.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.261+0000 7f7958640700 1 -- start start 2023-10-03T10:19:45.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.262+0000 7f7958640700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f795009e930 0x7f79501a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:45.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.262+0000 7f7958640700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f795009f240 0x7f79501a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:45.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.262+0000 7f7958640700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f795009f650 0x7f79501c3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:45.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.262+0000 7f7958640700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79501b50d0 con 0x7f795009f650 2023-10-03T10:19:45.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.262+0000 7f7958640700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79501a1a50 con 0x7f795009f240 2023-10-03T10:19:45.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.262+0000 7f7958640700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79501a2d20 con 0x7f795009e930 2023-10-03T10:19:45.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.262+0000 7f7955bdb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f795009f240 0x7f79501a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:45.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.262+0000 7f7955bdb700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f795009f240 0x7f79501a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:43158/0 (socket says 172.21.15.3:43158) 2023-10-03T10:19:45.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.262+0000 7f7955bdb700 1 -- 172.21.15.3:0/2035618310 learned_addr learned my addr 172.21.15.3:0/2035618310 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:45.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.262+0000 7f7956bdd700 1 --2- 172.21.15.3:0/2035618310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f795009f650 0x7f79501c3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:45.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.262+0000 7f79563dc700 1 --2- 172.21.15.3:0/2035618310 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f795009e930 0x7f79501a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:45.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.262+0000 7f7955bdb700 1 -- 172.21.15.3:0/2035618310 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f795009e930 msgr2=0x7f79501a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:45.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.262+0000 7f7955bdb700 1 --2- 172.21.15.3:0/2035618310 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f795009e930 0x7f79501a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.262+0000 7f7955bdb700 1 -- 172.21.15.3:0/2035618310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f795009f650 msgr2=0x7f79501c3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:45.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.262+0000 7f7955bdb700 1 --2- 172.21.15.3:0/2035618310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f795009f650 0x7f79501c3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.263+0000 7f7955bdb700 1 -- 172.21.15.3:0/2035618310 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f794c00d9f0 con 0x7f795009f240 2023-10-03T10:19:45.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.263+0000 7f7955bdb700 1 --2- 172.21.15.3:0/2035618310 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f795009f240 0x7f79501a1510 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f794000e980 tx=0x7f794000ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:45.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.263+0000 7f79563dc700 1 --2- 172.21.15.3:0/2035618310 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f795009e930 0x7f79501a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:19:45.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.263+0000 7f7946ffd700 1 -- 172.21.15.3:0/2035618310 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794000eed0 con 0x7f795009f240 2023-10-03T10:19:45.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.263+0000 7f7946ffd700 1 -- 172.21.15.3:0/2035618310 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794000c850 con 0x7f795009f240 2023-10-03T10:19:45.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.263+0000 7f7958640700 1 -- 172.21.15.3:0/2035618310 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79500a0190 con 0x7f795009f240 2023-10-03T10:19:45.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.263+0000 7f7946ffd700 1 -- 172.21.15.3:0/2035618310 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794000cbc0 con 0x7f795009f240 2023-10-03T10:19:45.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.263+0000 7f7958640700 1 -- 172.21.15.3:0/2035618310 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79501a0c20 con 0x7f795009f240 2023-10-03T10:19:45.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.264+0000 7f7946ffd700 1 -- 172.21.15.3:0/2035618310 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f794000cd20 con 0x7f795009f240 2023-10-03T10:19:45.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.264+0000 7f7958640700 1 -- 172.21.15.3:0/2035618310 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79501a4080 con 0x7f795009f240 2023-10-03T10:19:45.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.265+0000 7f7946ffd700 1 -- 172.21.15.3:0/2035618310 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f794003f920 con 0x7f795009f240 2023-10-03T10:19:45.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.270+0000 7f7946ffd700 1 -- 172.21.15.3:0/2035618310 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f79400232f0 con 0x7f795009f240 2023-10-03T10:19:45.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.285+0000 7f78dfe54700 1 Processor -- start 2023-10-03T10:19:45.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.286+0000 7f78dfe54700 1 -- start start 2023-10-03T10:19:45.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.286+0000 7f78dfe54700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78d80a0a30 0x7f78d80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:45.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.286+0000 7f78dfe54700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f78d80a1340 0x7f78d8133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:45.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.286+0000 7f78dfe54700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d809edd0 con 0x7f78d8133fa0 2023-10-03T10:19:45.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.286+0000 7f78dfe54700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78d809e1f0 con 0x7f78d80a1340 2023-10-03T10:19:45.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.286+0000 7f78dfe54700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d809dc50 con 0x7f78d80a0a30 2023-10-03T10:19:45.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.286+0000 7f78ddbf0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78d80a0a30 0x7f78d80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:45.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.286+0000 7f78dd3ef700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f78d80a1340 0x7f78d8133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:45.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.287+0000 7f78ddbf0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78d80a0a30 0x7f78d80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:43164/0 (socket says 172.21.15.3:43164) 2023-10-03T10:19:45.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.287+0000 7f78ddbf0700 1 -- 172.21.15.3:0/214751974 learned_addr learned my addr 172.21.15.3:0/214751974 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:45.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.287+0000 7f78dcbee700 1 -- 172.21.15.3:0/214751974 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2689558043 0 0) 0x7f78d809edd0 con 0x7f78d8133fa0 2023-10-03T10:19:45.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.287+0000 7f78dcbee700 1 -- 172.21.15.3:0/214751974 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78c40035b0 con 0x7f78d8133fa0 2023-10-03T10:19:45.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.287+0000 7f78ddbf0700 1 -- 172.21.15.3:0/214751974 >> v2:172.21.15.3:3300/0 conn(0x7f78d80a1340 msgr2=0x7f78d8133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:45.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.287+0000 7f78ddbf0700 1 --2- 172.21.15.3:0/214751974 >> v2:172.21.15.3:3300/0 conn(0x7f78d80a1340 0x7f78d8133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.287+0000 7f78ddbf0700 1 -- 172.21.15.3:0/214751974 >> v1:172.21.15.3:6789/0 conn(0x7f78d8133fa0 legacy=0x7f78d813ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:45.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.287+0000 7f78ddbf0700 1 -- 172.21.15.3:0/214751974 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d80a1fe0 con 0x7f78d80a0a30 2023-10-03T10:19:45.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.287+0000 7f78dd3ef700 1 --2- 172.21.15.3:0/214751974 >> v2:172.21.15.3:3300/0 conn(0x7f78d80a1340 0x7f78d8133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:45.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.287+0000 7f78ddbf0700 1 --2- 172.21.15.3:0/214751974 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78d80a0a30 0x7f78d80a0e00 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f78d40098f0 tx=0x7f78d4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e0352a22040d3fa server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:45.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.287+0000 7f78dcbee700 1 -- 172.21.15.3:0/214751974 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d4011830 con 0x7f78d80a0a30 2023-10-03T10:19:45.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.287+0000 7f78dcbee700 1 -- 172.21.15.3:0/214751974 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d4011990 con 0x7f78d80a0a30 2023-10-03T10:19:45.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.288+0000 7f78dcbee700 1 -- 172.21.15.3:0/214751974 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d4011d20 con 0x7f78d80a0a30 2023-10-03T10:19:45.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.288+0000 7f78dfe54700 1 -- 172.21.15.3:0/214751974 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78d80a0a30 msgr2=0x7f78d80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:45.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.288+0000 7f78dfe54700 1 --2- 172.21.15.3:0/214751974 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78d80a0a30 0x7f78d80a0e00 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f78d40098f0 tx=0x7f78d4009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:45.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.288+0000 7f78dfe54700 1 -- 172.21.15.3:0/214751974 shutdown_connections 2023-10-03T10:19:45.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.288+0000 7f78dfe54700 1 --2- 172.21.15.3:0/214751974 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78d80a0a30 0x7f78d80a0e00 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.288+0000 7f78dfe54700 1 --2- 172.21.15.3:0/214751974 >> v2:172.21.15.3:3300/0 conn(0x7f78d80a1340 0x7f78d8133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.288+0000 7f78dfe54700 1 -- 172.21.15.3:0/214751974 >> 172.21.15.3:0/214751974 conn(0x7f78d80a6550 msgr2=0x7f78d80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:45.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.289+0000 7f78dfe54700 1 -- 172.21.15.3:0/214751974 shutdown_connections 2023-10-03T10:19:45.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.289+0000 7f78dfe54700 1 -- 172.21.15.3:0/214751974 wait complete. 2023-10-03T10:19:45.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.289+0000 7f9c0f0f7700 1 Processor -- start 2023-10-03T10:19:45.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.289+0000 7f9c0f0f7700 1 -- start start 2023-10-03T10:19:45.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.290+0000 7f9c0f0f7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c081337f0 0x7f9c08138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:45.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.290+0000 7f9c0f0f7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c08138dc0 0x7f9c0813b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:45.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.290+0000 7f9c0f0f7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c0806b990 con 0x7f9c08133420 2023-10-03T10:19:45.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.290+0000 7f9c0f0f7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0809b0e0 con 0x7f9c08138dc0 2023-10-03T10:19:45.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.290+0000 7f9c0f0f7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c08133c20 con 0x7f9c081337f0 2023-10-03T10:19:45.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.290+0000 7f78dfe54700 1 Processor -- start 2023-10-03T10:19:45.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.290+0000 7f9c0ce93700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9c08133420 0x7f9c08133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60412/0 (socket says 172.21.15.3:60412) 2023-10-03T10:19:45.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.290+0000 7f9c0d694700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c08138dc0 0x7f9c0813b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:45.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.290+0000 7f9c0ce93700 1 -- 172.21.15.3:0/109949823 learned_addr learned my addr 172.21.15.3:0/109949823 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:45.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.290+0000 7f9c0d694700 1 -- 172.21.15.3:0/109949823 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c081337f0 msgr2=0x7f9c08138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:45.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.290+0000 7f9c0d694700 1 --2- 172.21.15.3:0/109949823 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c081337f0 0x7f9c08138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.290+0000 7f9c077fe700 1 -- 172.21.15.3:0/109949823 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2009142170 0 0) 0x7f9c0806b990 con 0x7f9c08133420 2023-10-03T10:19:45.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.290+0000 7f9c0d694700 1 -- 172.21.15.3:0/109949823 >> v1:172.21.15.3:6789/0 conn(0x7f9c08133420 legacy=0x7f9c08133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:45.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.291+0000 7f9c0d694700 1 -- 172.21.15.3:0/109949823 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c080a0590 con 0x7f9c08138dc0 2023-10-03T10:19:45.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.291+0000 7f9c0d694700 1 --2- 172.21.15.3:0/109949823 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c08138dc0 0x7f9c0813b1a0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009dc0 tx=0x7f9c0000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0e97e055efea4df server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:45.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.290+0000 7f78dfe54700 1 -- start start 2023-10-03T10:19:45.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.291+0000 7f78dfe54700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78d80a0a30 0x7f78d8126d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:45.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.291+0000 7f78dfe54700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78d80a1340 0x7f78d81272c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:45.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.291+0000 7f78dfe54700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f78d8133fa0 0x7f78d81cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:45.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.291+0000 7f78dfe54700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d81285c0 con 0x7f78d80a0a30 2023-10-03T10:19:45.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.291+0000 7f78dfe54700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d80a3580 con 0x7f78d80a1340 2023-10-03T10:19:45.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.291+0000 7f78dfe54700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f78d80a0220 con 0x7f78d8133fa0 2023-10-03T10:19:45.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.291+0000 7f78dd3ef700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78d80a1340 0x7f78d81272c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:45.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.291+0000 7f78dd3ef700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78d80a1340 0x7f78d81272c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:43190/0 (socket says 172.21.15.3:43190) 2023-10-03T10:19:45.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.291+0000 7f78dd3ef700 1 -- 172.21.15.3:0/3663266726 learned_addr learned my addr 172.21.15.3:0/3663266726 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:45.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.291+0000 7f9c077fe700 1 -- 172.21.15.3:0/109949823 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0000e070 con 0x7f9c08138dc0 2023-10-03T10:19:45.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.291+0000 7f9c077fe700 1 -- 172.21.15.3:0/109949823 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0000bbd0 con 0x7f9c08138dc0 2023-10-03T10:19:45.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.291+0000 7f9c077fe700 1 -- 172.21.15.3:0/109949823 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c00012430 con 0x7f9c08138dc0 2023-10-03T10:19:45.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.292+0000 7f78dd3ef700 1 -- 172.21.15.3:0/3663266726 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f78d8133fa0 msgr2=0x7f78d81cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:45.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.292+0000 7f78dd3ef700 1 --2- 172.21.15.3:0/3663266726 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f78d8133fa0 0x7f78d81cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.292+0000 7f78dd3ef700 1 -- 172.21.15.3:0/3663266726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78d80a0a30 msgr2=0x7f78d8126d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-03T10:19:45.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.292+0000 7f78dd3ef700 1 --2- 172.21.15.3:0/3663266726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78d80a0a30 0x7f78d8126d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.292+0000 7f78dd3ef700 1 -- 172.21.15.3:0/3663266726 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c80094a0 con 0x7f78d80a1340 2023-10-03T10:19:45.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.292+0000 7f78dd3ef700 1 --2- 172.21.15.3:0/3663266726 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78d80a1340 0x7f78d81272c0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f78c800ebe0 tx=0x7f78c8009ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:45.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.292+0000 7f9c0f0f7700 1 -- 172.21.15.3:0/109949823 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c08138dc0 msgr2=0x7f9c0813b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:45.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.292+0000 7f9c0f0f7700 1 --2- 172.21.15.3:0/109949823 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c08138dc0 0x7f9c0813b1a0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009dc0 tx=0x7f9c0000b330 comp rx=0 tx=0).stop 2023-10-03T10:19:45.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.292+0000 7f9c0f0f7700 1 -- 172.21.15.3:0/109949823 shutdown_connections 2023-10-03T10:19:45.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.292+0000 7f9c0f0f7700 1 --2- 172.21.15.3:0/109949823 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c08138dc0 0x7f9c0813b1a0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.292+0000 7f9c0f0f7700 1 --2- 172.21.15.3:0/109949823 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c081337f0 0x7f9c08138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.292+0000 7f9c0f0f7700 1 -- 172.21.15.3:0/109949823 >> 172.21.15.3:0/109949823 conn(0x7f9c080a4210 msgr2=0x7f9c080a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:45.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.292+0000 7f9c0f0f7700 1 -- 172.21.15.3:0/109949823 shutdown_connections 2023-10-03T10:19:45.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.292+0000 7f9c0f0f7700 1 -- 172.21.15.3:0/109949823 wait complete. 2023-10-03T10:19:45.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.292+0000 7f78ce7fc700 1 -- 172.21.15.3:0/3663266726 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c800c910 con 0x7f78d80a1340 2023-10-03T10:19:45.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.292+0000 7f78ce7fc700 1 -- 172.21.15.3:0/3663266726 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c800ca70 con 0x7f78d80a1340 2023-10-03T10:19:45.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.292+0000 7f78ce7fc700 1 -- 172.21.15.3:0/3663266726 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c800cde0 con 0x7f78d80a1340 2023-10-03T10:19:45.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.292+0000 7f78dfe54700 1 -- 172.21.15.3:0/3663266726 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d4009550 con 0x7f78d80a1340 2023-10-03T10:19:45.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.292+0000 7f78dfe54700 1 -- 172.21.15.3:0/3663266726 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d8127800 con 0x7f78d80a1340 2023-10-03T10:19:45.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.293+0000 7f78ce7fc700 1 -- 172.21.15.3:0/3663266726 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f78c8013470 con 0x7f78d80a1340 2023-10-03T10:19:45.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.293+0000 7f9c0f0f7700 1 Processor -- start 2023-10-03T10:19:45.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.293+0000 7f9c0f0f7700 1 -- start start 2023-10-03T10:19:45.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.294+0000 7f78ce7fc700 1 -- 172.21.15.3:0/3663266726 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f78c803fa80 con 0x7f78d80a1340 2023-10-03T10:19:45.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.294+0000 7f78dfe54700 1 -- 172.21.15.3:0/3663266726 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d8129cc0 con 0x7f78d80a1340 2023-10-03T10:19:45.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.294+0000 7f9c0f0f7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c08133420 0x7f9c080709a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:45.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.294+0000 7f9c0f0f7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c081337f0 0x7f9c08070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:45.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.294+0000 7f9c0f0f7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c08138dc0 0x7f9c08071420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:45.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.294+0000 7f9c0f0f7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0809e420 con 0x7f9c08133420 2023-10-03T10:19:45.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.294+0000 7f9c0f0f7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0809d7e0 con 0x7f9c081337f0 2023-10-03T10:19:45.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.294+0000 7f9c0f0f7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0809b790 con 0x7f9c08138dc0 2023-10-03T10:19:45.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.295+0000 7f9c0d694700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c08138dc0 0x7f9c08071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:45.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.295+0000 7f9c0d694700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c08138dc0 0x7f9c08071420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:44592/0 (socket says 172.21.15.3:44592) 2023-10-03T10:19:45.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.295+0000 7f9c0d694700 1 -- 172.21.15.3:0/3280695519 learned_addr learned my addr 172.21.15.3:0/3280695519 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:45.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.295+0000 7f9c07fff700 1 --2- 172.21.15.3:0/3280695519 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c081337f0 0x7f9c08070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:45.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.295+0000 7f9c0ce93700 1 --2- 172.21.15.3:0/3280695519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c08133420 0x7f9c080709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:45.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.295+0000 7f9c0d694700 1 -- 172.21.15.3:0/3280695519 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c081337f0 msgr2=0x7f9c08070ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:45.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.295+0000 7f9c0d694700 1 --2- 172.21.15.3:0/3280695519 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c081337f0 0x7f9c08070ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.295+0000 7f9c0d694700 1 -- 172.21.15.3:0/3280695519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c08133420 msgr2=0x7f9c080709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:45.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.295+0000 7f9c0d694700 1 --2- 172.21.15.3:0/3280695519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c08133420 0x7f9c080709a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.295+0000 7f9c0d694700 1 -- 172.21.15.3:0/3280695519 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c00009a70 con 0x7f9c08138dc0 2023-10-03T10:19:45.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.295+0000 7f9c0d694700 1 --2- 172.21.15.3:0/3280695519 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c08138dc0 0x7f9c08071420 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f9c08134600 tx=0x7f9c00004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:45.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.297+0000 7f9c057fa700 1 -- 172.21.15.3:0/3280695519 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c00012430 con 0x7f9c08138dc0 2023-10-03T10:19:45.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.297+0000 7f9c057fa700 1 -- 172.21.15.3:0/3280695519 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c00020070 con 0x7f9c08138dc0 2023-10-03T10:19:45.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.297+0000 7f9c057fa700 1 -- 172.21.15.3:0/3280695519 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c00004280 con 0x7f9c08138dc0 2023-10-03T10:19:45.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.297+0000 7f9c0f0f7700 1 -- 172.21.15.3:0/3280695519 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c08138930 con 0x7f9c08138dc0 2023-10-03T10:19:45.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.297+0000 7f9c0f0f7700 1 -- 172.21.15.3:0/3280695519 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c0809bf10 con 0x7f9c08138dc0 2023-10-03T10:19:45.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.298+0000 7f9c057fa700 1 -- 172.21.15.3:0/3280695519 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9c000043e0 con 0x7f9c08138dc0 2023-10-03T10:19:45.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.298+0000 7f9c0f0f7700 1 -- 172.21.15.3:0/3280695519 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c0809c5c0 con 0x7f9c08138dc0 2023-10-03T10:19:45.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.298+0000 7f9c057fa700 1 -- 172.21.15.3:0/3280695519 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9c00022070 con 0x7f9c08138dc0 2023-10-03T10:19:45.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.302+0000 7f78ce7fc700 1 -- 172.21.15.3:0/3663266726 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f78c8017080 con 0x7f78d80a1340 2023-10-03T10:19:45.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.305+0000 7f9c057fa700 1 -- 172.21.15.3:0/3280695519 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9c00012e30 con 0x7f9c08138dc0 2023-10-03T10:19:45.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.313+0000 7f4de038e700 1 Processor -- start 2023-10-03T10:19:45.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.313+0000 7f4de038e700 1 -- start start 2023-10-03T10:19:45.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.314+0000 7f4de038e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4dd80a0a30 0x7f4dd80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:45.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.314+0000 7f4de038e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4dd8138d80 0x7f4dd813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:45.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.314+0000 7f4de038e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd80a2a70 con 0x7f4dd809d630 2023-10-03T10:19:45.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.314+0000 7f4de038e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd80a01d0 con 0x7f4dd80a0a30 2023-10-03T10:19:45.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.314+0000 7f4de038e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd809e2b0 con 0x7f4dd8138d80 2023-10-03T10:19:45.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.314+0000 7f4ddd929700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4dd809d630 0x7f4dd80a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60414/0 (socket says 172.21.15.3:60414) 2023-10-03T10:19:45.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.314+0000 7f4ddd929700 1 -- 172.21.15.3:0/708781018 learned_addr learned my addr 172.21.15.3:0/708781018 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:45.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.315+0000 7f4ddd128700 1 -- 172.21.15.3:0/708781018 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2111568951 0 0) 0x7f4dd80a2a70 con 0x7f4dd809d630 2023-10-03T10:19:45.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.315+0000 7f4ddd128700 1 -- 172.21.15.3:0/708781018 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dc40035b0 con 0x7f4dd809d630 2023-10-03T10:19:45.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.315+0000 7f4dde12a700 1 --2- 172.21.15.3:0/708781018 >> v2:172.21.15.3:3300/0 conn(0x7f4dd80a0a30 0x7f4dd80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:45.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.315+0000 7f4ddd128700 1 -- 172.21.15.3:0/708781018 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 503010343 0 0) 0x7f4dc40035b0 con 0x7f4dd809d630 2023-10-03T10:19:45.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.315+0000 7f4ddd128700 1 -- 172.21.15.3:0/708781018 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4dd8138d80 msgr2=0x7f4dd813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:45.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.315+0000 7f4ddd128700 1 --2- 172.21.15.3:0/708781018 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4dd8138d80 0x7f4dd813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.315+0000 7f4ddd128700 1 -- 172.21.15.3:0/708781018 >> v2:172.21.15.3:3300/0 conn(0x7f4dd80a0a30 msgr2=0x7f4dd80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:45.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.315+0000 7f4ddd128700 1 --2- 172.21.15.3:0/708781018 >> v2:172.21.15.3:3300/0 conn(0x7f4dd80a0a30 0x7f4dd80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.315+0000 7f4ddd128700 1 -- 172.21.15.3:0/708781018 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd809dc50 con 0x7f4dd809d630 2023-10-03T10:19:45.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.315+0000 7f4ddd128700 1 -- 172.21.15.3:0/708781018 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1156569835 0 0) 0x7f4dd00019f0 con 0x7f4dd809d630 2023-10-03T10:19:45.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.315+0000 7f4ddd128700 1 -- 172.21.15.3:0/708781018 >> v1:172.21.15.3:6789/0 conn(0x7f4dd809d630 legacy=0x7f4dd80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:45.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.315+0000 7f4ddd128700 1 --2- 172.21.15.3:0/708781018 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4dc4003a00 0x7f4dc4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:45.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.315+0000 7f4ddd128700 1 --2- 172.21.15.3:0/708781018 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4dc4004430 0x7f4dc40048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:45.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.315+0000 7f4ddd128700 1 --2- 172.21.15.3:0/708781018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4dc4005190 0x7f4dc400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:45.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.315+0000 7f4ddd128700 1 -- 172.21.15.3:0/708781018 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd80a01d0 con 0x7f4dc4005190 2023-10-03T10:19:45.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.315+0000 7f4ddd128700 1 -- 172.21.15.3:0/708781018 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd809e2b0 con 0x7f4dc4004430 2023-10-03T10:19:45.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.315+0000 7f4ddd128700 1 -- 172.21.15.3:0/708781018 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc400c160 con 0x7f4dc4003a00 2023-10-03T10:19:45.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.315+0000 7f4ddd929700 1 --2- 172.21.15.3:0/708781018 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4dc4004430 0x7f4dc40048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:45.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.316+0000 7f4dde12a700 1 --2- 172.21.15.3:0/708781018 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4dc4003a00 0x7f4dc4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:45.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.316+0000 7f4ddd929700 1 -- 172.21.15.3:0/708781018 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4dc4003a00 msgr2=0x7f4dc4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:45.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.316+0000 7f4ddd929700 1 --2- 172.21.15.3:0/708781018 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4dc4003a00 0x7f4dc4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.316+0000 7f4ddd929700 1 -- 172.21.15.3:0/708781018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4dc4005190 msgr2=0x7f4dc400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:45.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.316+0000 7f4ddd929700 1 --2- 172.21.15.3:0/708781018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4dc4005190 0x7f4dc400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.316+0000 7f4ddd929700 1 -- 172.21.15.3:0/708781018 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd809dc50 con 0x7f4dc4004430 2023-10-03T10:19:45.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.316+0000 7f4ddd929700 1 --2- 172.21.15.3:0/708781018 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4dc4004430 0x7f4dc40048f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f4dd000a8d0 tx=0x7f4dc4005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=383030e266f2c5e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:45.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.316+0000 7f4ddd128700 1 -- 172.21.15.3:0/708781018 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd000a340 con 0x7f4dc4004430 2023-10-03T10:19:45.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.316+0000 7f4ddd128700 1 -- 172.21.15.3:0/708781018 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd0012600 con 0x7f4dc4004430 2023-10-03T10:19:45.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.316+0000 7f4ddd128700 1 -- 172.21.15.3:0/708781018 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0012990 con 0x7f4dc4004430 2023-10-03T10:19:45.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.317+0000 7f4de038e700 1 -- 172.21.15.3:0/708781018 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4dc4004430 msgr2=0x7f4dc40048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:45.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.317+0000 7f4de038e700 1 --2- 172.21.15.3:0/708781018 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4dc4004430 0x7f4dc40048f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f4dd000a8d0 tx=0x7f4dc4005880 comp rx=0 tx=0).stop 2023-10-03T10:19:45.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.317+0000 7f4dde92b700 1 -- 172.21.15.3:0/708781018 reap_dead start 2023-10-03T10:19:45.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.317+0000 7f4de038e700 1 -- 172.21.15.3:0/708781018 shutdown_connections 2023-10-03T10:19:45.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.317+0000 7f4de038e700 1 -- 172.21.15.3:0/708781018 >> 172.21.15.3:0/708781018 conn(0x7f4dd80a3050 msgr2=0x7f4dd80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:45.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.317+0000 7f4de038e700 1 -- 172.21.15.3:0/708781018 shutdown_connections 2023-10-03T10:19:45.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.317+0000 7f4de038e700 1 -- 172.21.15.3:0/708781018 wait complete. 2023-10-03T10:19:45.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.318+0000 7f4de038e700 1 Processor -- start 2023-10-03T10:19:45.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.319+0000 7f4de038e700 1 -- start start 2023-10-03T10:19:45.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.319+0000 7f4de038e700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4dd811f450 0x7f4dd811f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:45.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.319+0000 7f4de038e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4dd81c8300 0x7f4dd81c8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:45.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.319+0000 7f4de038e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4dd81c8c50 0x7f4dd81cf630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:45.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.319+0000 7f4de038e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd80a0410 con 0x7f4dd81c8300 2023-10-03T10:19:45.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.319+0000 7f4de038e700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd80a5000 con 0x7f4dd811f450 2023-10-03T10:19:45.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.319+0000 7f4de038e700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd811f0a0 con 0x7f4dd81c8c50 2023-10-03T10:19:45.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.319+0000 7f4dde12a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4dd811f450 0x7f4dd811f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:45.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.319+0000 7f4dde12a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4dd811f450 0x7f4dd811f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:43236/0 (socket says 172.21.15.3:43236) 2023-10-03T10:19:45.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.319+0000 7f4dde12a700 1 -- 172.21.15.3:0/2490984625 learned_addr learned my addr 172.21.15.3:0/2490984625 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:45.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.319+0000 7f4ddd929700 1 --2- 172.21.15.3:0/2490984625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4dd81c8300 0x7f4dd81c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:45.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.320+0000 7f4dde92b700 1 --2- 172.21.15.3:0/2490984625 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4dd81c8c50 0x7f4dd81cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:45.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.320+0000 7f4ddd929700 1 -- 172.21.15.3:0/2490984625 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4dd81c8c50 msgr2=0x7f4dd81cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:45.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.320+0000 7f4ddd929700 1 --2- 172.21.15.3:0/2490984625 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4dd81c8c50 0x7f4dd81cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.320+0000 7f4ddd929700 1 -- 172.21.15.3:0/2490984625 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4dd811f450 msgr2=0x7f4dd811f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:45.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.320+0000 7f4ddd929700 1 --2- 172.21.15.3:0/2490984625 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4dd811f450 0x7f4dd811f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.320+0000 7f4ddd929700 1 -- 172.21.15.3:0/2490984625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd8122070 con 0x7f4dd81c8300 2023-10-03T10:19:45.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.320+0000 7f4ddd929700 1 --2- 172.21.15.3:0/2490984625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4dd81c8300 0x7f4dd81c8710 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0015000 tx=0x7f4dd0001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:45.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.320+0000 7f4dceffd700 1 -- 172.21.15.3:0/2490984625 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd000a340 con 0x7f4dd81c8300 2023-10-03T10:19:45.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.320+0000 7f4dceffd700 1 -- 172.21.15.3:0/2490984625 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd001e640 con 0x7f4dd81c8300 2023-10-03T10:19:45.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.320+0000 7f4de038e700 1 -- 172.21.15.3:0/2490984625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd8120ef0 con 0x7f4dd81c8300 2023-10-03T10:19:45.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.320+0000 7f4dceffd700 1 -- 172.21.15.3:0/2490984625 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0012ea0 con 0x7f4dd81c8300 2023-10-03T10:19:45.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.321+0000 7f4de038e700 1 -- 172.21.15.3:0/2490984625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd8122da0 con 0x7f4dd81c8300 2023-10-03T10:19:45.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.321+0000 7f4dceffd700 1 -- 172.21.15.3:0/2490984625 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4dd001e7f0 con 0x7f4dd81c8300 2023-10-03T10:19:45.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.322+0000 7f4dceffd700 1 -- 172.21.15.3:0/2490984625 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f4dd0041c60 con 0x7f4dd81c8300 2023-10-03T10:19:45.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.322+0000 7f4de038e700 1 -- 172.21.15.3:0/2490984625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd80747f0 con 0x7f4dd81c8300 2023-10-03T10:19:45.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.327+0000 7f4dceffd700 1 -- 172.21.15.3:0/2490984625 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4dd002b020 con 0x7f4dd81c8300 2023-10-03T10:19:45.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.517+0000 7f7958640700 1 -- 172.21.15.3:0/2035618310 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7950071d50 con 0x7f795009f240 2023-10-03T10:19:45.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.517+0000 7f7946ffd700 1 -- 172.21.15.3:0/2035618310 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7940021090 con 0x7f795009f240 2023-10-03T10:19:45.518 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:45.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.520+0000 7f7958640700 1 -- 172.21.15.3:0/2035618310 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f795009f240 msgr2=0x7f79501a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:45.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.520+0000 7f7958640700 1 --2- 172.21.15.3:0/2035618310 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f795009f240 0x7f79501a1510 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f794000e980 tx=0x7f794000ec90 comp rx=0 tx=0).stop 2023-10-03T10:19:45.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.521+0000 7f7958640700 1 -- 172.21.15.3:0/2035618310 shutdown_connections 2023-10-03T10:19:45.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.521+0000 7f7958640700 1 --2- 172.21.15.3:0/2035618310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f795009f650 0x7f79501c3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.521+0000 7f7958640700 1 --2- 172.21.15.3:0/2035618310 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f795009e930 0x7f79501a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.521+0000 7f7958640700 1 --2- 172.21.15.3:0/2035618310 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f795009f240 0x7f79501a1510 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.521+0000 7f7958640700 1 -- 172.21.15.3:0/2035618310 >> 172.21.15.3:0/2035618310 conn(0x7f79500a4210 msgr2=0x7f79501b7890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:45.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.521+0000 7f7958640700 1 -- 172.21.15.3:0/2035618310 shutdown_connections 2023-10-03T10:19:45.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.521+0000 7f7958640700 1 -- 172.21.15.3:0/2035618310 wait complete. 2023-10-03T10:19:45.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.536+0000 7f4de038e700 1 -- 172.21.15.3:0/2490984625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4dd8072980 con 0x7f4dd81c8300 2023-10-03T10:19:45.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.536+0000 7f4dceffd700 1 -- 172.21.15.3:0/2490984625 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4dd0037090 con 0x7f4dd81c8300 2023-10-03T10:19:45.537 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:45.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.537+0000 7f9c0f0f7700 1 -- 172.21.15.3:0/3280695519 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9c08074ea0 con 0x7f9c08138dc0 2023-10-03T10:19:45.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.537+0000 7f9c057fa700 1 -- 172.21.15.3:0/3280695519 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9c0002c070 con 0x7f9c08138dc0 2023-10-03T10:19:45.538 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:45.540 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:45.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.540+0000 7f4de038e700 1 -- 172.21.15.3:0/2490984625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4dd81c8300 msgr2=0x7f4dd81c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:45.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.540+0000 7f4de038e700 1 --2- 172.21.15.3:0/2490984625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4dd81c8300 0x7f4dd81c8710 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0015000 tx=0x7f4dd0001cb0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.540+0000 7f4de038e700 1 -- 172.21.15.3:0/2490984625 shutdown_connections 2023-10-03T10:19:45.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.540+0000 7f4de038e700 1 --2- 172.21.15.3:0/2490984625 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f4dd81c8c50 0x7f4dd81cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.540+0000 7f4de038e700 1 --2- 172.21.15.3:0/2490984625 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4dd811f450 0x7f4dd811f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.540+0000 7f4de038e700 1 --2- 172.21.15.3:0/2490984625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4dd81c8300 0x7f4dd81c8710 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.540+0000 7f4de038e700 1 -- 172.21.15.3:0/2490984625 >> 172.21.15.3:0/2490984625 conn(0x7f4dd80a3050 msgr2=0x7f4dd8137da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:45.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.540+0000 7f4de038e700 1 -- 172.21.15.3:0/2490984625 shutdown_connections 2023-10-03T10:19:45.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.540+0000 7f4de038e700 1 -- 172.21.15.3:0/2490984625 wait complete. 2023-10-03T10:19:45.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.540+0000 7f9c0f0f7700 1 -- 172.21.15.3:0/3280695519 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c08138dc0 msgr2=0x7f9c08071420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:45.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.540+0000 7f9c0f0f7700 1 --2- 172.21.15.3:0/3280695519 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c08138dc0 0x7f9c08071420 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f9c08134600 tx=0x7f9c00004ad0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.540+0000 7f9c0f0f7700 1 -- 172.21.15.3:0/3280695519 shutdown_connections 2023-10-03T10:19:45.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.540+0000 7f9c0f0f7700 1 --2- 172.21.15.3:0/3280695519 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9c08138dc0 0x7f9c08071420 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.540+0000 7f9c0f0f7700 1 --2- 172.21.15.3:0/3280695519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c08133420 0x7f9c080709a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.540+0000 7f9c0f0f7700 1 --2- 172.21.15.3:0/3280695519 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9c081337f0 0x7f9c08070ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.540+0000 7f9c0f0f7700 1 -- 172.21.15.3:0/3280695519 >> 172.21.15.3:0/3280695519 conn(0x7f9c080a4210 msgr2=0x7f9c080a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:45.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.540+0000 7f9c0f0f7700 1 -- 172.21.15.3:0/3280695519 shutdown_connections 2023-10-03T10:19:45.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.540+0000 7f9c0f0f7700 1 -- 172.21.15.3:0/3280695519 wait complete. 2023-10-03T10:19:45.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.541+0000 7f78dfe54700 1 -- 172.21.15.3:0/3663266726 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f78d8074ea0 con 0x7f78d80a1340 2023-10-03T10:19:45.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.541+0000 7f78ce7fc700 1 -- 172.21.15.3:0/3663266726 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f78c80232f0 con 0x7f78d80a1340 2023-10-03T10:19:45.545 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:45.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.543+0000 7f78dfe54700 1 -- 172.21.15.3:0/3663266726 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78d80a1340 msgr2=0x7f78d81272c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:45.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.543+0000 7f78dfe54700 1 --2- 172.21.15.3:0/3663266726 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78d80a1340 0x7f78d81272c0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f78c800ebe0 tx=0x7f78c8009ec0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.544+0000 7f78dfe54700 1 -- 172.21.15.3:0/3663266726 shutdown_connections 2023-10-03T10:19:45.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.544+0000 7f78dfe54700 1 --2- 172.21.15.3:0/3663266726 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f78d8133fa0 0x7f78d81cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.544+0000 7f78dfe54700 1 --2- 172.21.15.3:0/3663266726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78d80a0a30 0x7f78d8126d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.544+0000 7f78dfe54700 1 --2- 172.21.15.3:0/3663266726 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f78d80a1340 0x7f78d81272c0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:45.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.544+0000 7f78dfe54700 1 -- 172.21.15.3:0/3663266726 >> 172.21.15.3:0/3663266726 conn(0x7f78d80a6550 msgr2=0x7f78d81332a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:45.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.544+0000 7f78dfe54700 1 -- 172.21.15.3:0/3663266726 shutdown_connections 2023-10-03T10:19:45.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:45.544+0000 7f78dfe54700 1 -- 172.21.15.3:0/3663266726 wait complete. 2023-10-03T10:19:45.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:45.564 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:45.565 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:46.541 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:46.560 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:46.565 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:46.565 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:46.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.698+0000 7fd6ffe18700 1 Processor -- start 2023-10-03T10:19:46.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.699+0000 7fd6ffe18700 1 -- start start 2023-10-03T10:19:46.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.699+0000 7fd6ffe18700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd6f8134630 0x7fd6f8136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:46.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.699+0000 7fd6ffe18700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6f8136f50 0x7fd6f8139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:46.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.699+0000 7fd6ffe18700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f809b4f0 con 0x7fd6f8131810 2023-10-03T10:19:46.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.699+0000 7fd6ffe18700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f81313c0 con 0x7fd6f8134630 2023-10-03T10:19:46.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.699+0000 7fd6ffe18700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f806e8c0 con 0x7fd6f8136f50 2023-10-03T10:19:46.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.699+0000 7fd6fd3b3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd6f8134630 0x7fd6f8136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:46.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.699+0000 7fd6fd3b3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd6f8134630 0x7fd6f8136a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38692/0 (socket says 172.21.15.3:38692) 2023-10-03T10:19:46.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.699+0000 7fd6fd3b3700 1 -- 172.21.15.3:0/3253912053 learned_addr learned my addr 172.21.15.3:0/3253912053 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:46.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.699+0000 7fd6fe3b5700 1 --2- 172.21.15.3:0/3253912053 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6f8136f50 0x7fd6f8139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:46.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.700+0000 7fd6fe3b5700 1 -- 172.21.15.3:0/3253912053 >> v2:172.21.15.3:3300/0 conn(0x7fd6f8134630 msgr2=0x7fd6f8136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:46.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.700+0000 7fd6fe3b5700 1 --2- 172.21.15.3:0/3253912053 >> v2:172.21.15.3:3300/0 conn(0x7fd6f8134630 0x7fd6f8136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:46.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.700+0000 7fd6fe3b5700 1 -- 172.21.15.3:0/3253912053 >> v1:172.21.15.3:6789/0 conn(0x7fd6f8131810 legacy=0x7fd6f8131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:46.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.700+0000 7fd6fe3b5700 1 -- 172.21.15.3:0/3253912053 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f8071d20 con 0x7fd6f8136f50 2023-10-03T10:19:46.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.700+0000 7fd6fd3b3700 1 --2- 172.21.15.3:0/3253912053 >> v2:172.21.15.3:3300/0 conn(0x7fd6f8134630 0x7fd6f8136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:46.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.700+0000 7fd6fe3b5700 1 --2- 172.21.15.3:0/3253912053 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6f8136f50 0x7fd6f8139330 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000f370 tx=0x7fd6f000f680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5add48337fea9d30 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:46.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.700+0000 7fd6fcbb2700 1 -- 172.21.15.3:0/3253912053 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0012070 con 0x7fd6f8136f50 2023-10-03T10:19:46.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.700+0000 7fd6fcbb2700 1 -- 172.21.15.3:0/3253912053 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f000fe70 con 0x7fd6f8136f50 2023-10-03T10:19:46.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.700+0000 7fd6fcbb2700 1 -- 172.21.15.3:0/3253912053 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f000d2f0 con 0x7fd6f8136f50 2023-10-03T10:19:46.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.701+0000 7fd6ffe18700 1 -- 172.21.15.3:0/3253912053 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6f8136f50 msgr2=0x7fd6f8139330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:46.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.701+0000 7fd6ffe18700 1 --2- 172.21.15.3:0/3253912053 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6f8136f50 0x7fd6f8139330 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000f370 tx=0x7fd6f000f680 comp rx=0 tx=0).stop 2023-10-03T10:19:46.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.702+0000 7fd6ffe18700 1 -- 172.21.15.3:0/3253912053 shutdown_connections 2023-10-03T10:19:46.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.702+0000 7fd6ffe18700 1 --2- 172.21.15.3:0/3253912053 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6f8136f50 0x7fd6f8139330 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:46.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.702+0000 7fd6ffe18700 1 --2- 172.21.15.3:0/3253912053 >> v2:172.21.15.3:3300/0 conn(0x7fd6f8134630 0x7fd6f8136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:46.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.702+0000 7fd6ffe18700 1 -- 172.21.15.3:0/3253912053 >> 172.21.15.3:0/3253912053 conn(0x7fd6f80a2aa0 msgr2=0x7fd6f80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:46.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.702+0000 7fd6ffe18700 1 -- 172.21.15.3:0/3253912053 shutdown_connections 2023-10-03T10:19:46.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.702+0000 7fd6ffe18700 1 -- 172.21.15.3:0/3253912053 wait complete. 2023-10-03T10:19:46.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.703+0000 7fd6ffe18700 1 Processor -- start 2023-10-03T10:19:46.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.703+0000 7fd6ffe18700 1 -- start start 2023-10-03T10:19:46.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.704+0000 7fd6ffe18700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6f8131810 0x7fd6f809c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:46.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.704+0000 7fd6ffe18700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6f8134630 0x7fd6f809cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:46.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.704+0000 7fd6ffe18700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6f8136f50 0x7fd6f809d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:46.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.704+0000 7fd6ffe18700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f8118d60 con 0x7fd6f8134630 2023-10-03T10:19:46.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.704+0000 7fd6ffe18700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f8136bb0 con 0x7fd6f8131810 2023-10-03T10:19:46.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.704+0000 7fd6ffe18700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f8119dc0 con 0x7fd6f8136f50 2023-10-03T10:19:46.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.704+0000 7fd6fdbb4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6f8131810 0x7fd6f809c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:46.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.704+0000 7fd6fdbb4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6f8131810 0x7fd6f809c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:43250/0 (socket says 172.21.15.3:43250) 2023-10-03T10:19:46.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.704+0000 7fd6fdbb4700 1 -- 172.21.15.3:0/4154416798 learned_addr learned my addr 172.21.15.3:0/4154416798 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:46.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.704+0000 7fd6fd3b3700 1 --2- 172.21.15.3:0/4154416798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6f8134630 0x7fd6f809cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:46.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.704+0000 7fd6fe3b5700 1 --2- 172.21.15.3:0/4154416798 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6f8136f50 0x7fd6f809d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:46.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.705+0000 7fd6fdbb4700 1 -- 172.21.15.3:0/4154416798 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6f8136f50 msgr2=0x7fd6f809d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:46.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.705+0000 7fd6fdbb4700 1 --2- 172.21.15.3:0/4154416798 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6f8136f50 0x7fd6f809d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.705+0000 7fd6fdbb4700 1 -- 172.21.15.3:0/4154416798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6f8134630 msgr2=0x7fd6f809cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.705+0000 7fd6fdbb4700 1 --2- 172.21.15.3:0/4154416798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6f8134630 0x7fd6f809cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.705+0000 7fd6fdbb4700 1 -- 172.21.15.3:0/4154416798 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e8009550 con 0x7fd6f8131810 2023-10-03T10:19:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.705+0000 7fd6fdbb4700 1 --2- 172.21.15.3:0/4154416798 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6f8131810 0x7fd6f809c630 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd6f400c450 tx=0x7fd6f400c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.705+0000 7fd6ee7fc700 1 -- 172.21.15.3:0/4154416798 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f40052d0 con 0x7fd6f8131810 2023-10-03T10:19:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.705+0000 7fd6ee7fc700 1 -- 172.21.15.3:0/4154416798 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f4005430 con 0x7fd6f8131810 2023-10-03T10:19:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.705+0000 7fd6ee7fc700 1 -- 172.21.15.3:0/4154416798 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f40038c0 con 0x7fd6f8131810 2023-10-03T10:19:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.705+0000 7fd6ffe18700 1 -- 172.21.15.3:0/4154416798 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f000f050 con 0x7fd6f8131810 2023-10-03T10:19:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.705+0000 7fd6ffe18700 1 -- 172.21.15.3:0/4154416798 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f8118f10 con 0x7fd6f8131810 2023-10-03T10:19:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.707+0000 7fd6ee7fc700 1 -- 172.21.15.3:0/4154416798 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd6f4003a20 con 0x7fd6f8131810 2023-10-03T10:19:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.707+0000 7fd6ffe18700 1 -- 172.21.15.3:0/4154416798 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f8074ea0 con 0x7fd6f8131810 2023-10-03T10:19:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.707+0000 7fd6ee7fc700 1 -- 172.21.15.3:0/4154416798 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fd6f403eb60 con 0x7fd6f8131810 2023-10-03T10:19:46.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.716+0000 7fd6ee7fc700 1 -- 172.21.15.3:0/4154416798 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd6f4011590 con 0x7fd6f8131810 2023-10-03T10:19:46.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.745+0000 7fc99c2c6700 1 Processor -- start 2023-10-03T10:19:46.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.745+0000 7fc99c2c6700 1 -- start start 2023-10-03T10:19:46.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.745+0000 7f0445384700 1 Processor -- start 2023-10-03T10:19:46.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.745+0000 7f0445384700 1 -- start start 2023-10-03T10:19:46.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.746+0000 7fc99c2c6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc99409e1d0 0x7fc994138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:46.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.746+0000 7fc99c2c6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc994139170 0x7fc99413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:46.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.746+0000 7fc99c2c6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99409b120 con 0x7fc99409d6f0 2023-10-03T10:19:46.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.746+0000 7fc99c2c6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99406b460 con 0x7fc99409e1d0 2023-10-03T10:19:46.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.746+0000 7fc99c2c6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc99409a7e0 con 0x7fc994139170 2023-10-03T10:19:46.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.746+0000 7fc99a062700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc99409d6f0 0x7fc99409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60432/0 (socket says 172.21.15.3:60432) 2023-10-03T10:19:46.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.746+0000 7fc99a062700 1 -- 172.21.15.3:0/1606771143 learned_addr learned my addr 172.21.15.3:0/1606771143 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:46.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.746+0000 7f0445384700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f04401305f0 0x7f04401309c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:46.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.746+0000 7f0445384700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0440070060 0x7f0440139740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:46.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.746+0000 7f0445384700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04400704e0 con 0x7f044006e550 2023-10-03T10:19:46.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.746+0000 7f0445384700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04401301c0 con 0x7f0440070060 2023-10-03T10:19:46.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.746+0000 7f0445384700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0440071ff0 con 0x7f04401305f0 2023-10-03T10:19:46.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.746+0000 7f043f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0440070060 0x7f0440139740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:46.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.746+0000 7f043effd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f04401305f0 0x7f04401309c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:46.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.746+0000 7f043effd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f04401305f0 0x7f04401309c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:43260/0 (socket says 172.21.15.3:43260) 2023-10-03T10:19:46.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.746+0000 7f043f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0440070060 0x7f0440139740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38722/0 (socket says 172.21.15.3:38722) 2023-10-03T10:19:46.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.746+0000 7f043effd700 1 -- 172.21.15.3:0/2869731628 learned_addr learned my addr 172.21.15.3:0/2869731628 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:46.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.746+0000 7fc999861700 1 --2- 172.21.15.3:0/1606771143 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc99409e1d0 0x7fc994138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:46.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.747+0000 7fc999060700 1 -- 172.21.15.3:0/1606771143 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1161926590 0 0) 0x7fc99409b120 con 0x7fc99409d6f0 2023-10-03T10:19:46.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.747+0000 7fc999060700 1 -- 172.21.15.3:0/1606771143 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9800035b0 con 0x7fc99409d6f0 2023-10-03T10:19:46.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.747+0000 7fc99a863700 1 --2- 172.21.15.3:0/1606771143 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc994139170 0x7fc99413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:46.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.747+0000 7f043f7fe700 1 -- 172.21.15.3:0/2869731628 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f04401305f0 msgr2=0x7f04401309c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:46.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.747+0000 7f043f7fe700 1 --2- 172.21.15.3:0/2869731628 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f04401305f0 0x7f04401309c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:46.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.747+0000 7f043f7fe700 1 -- 172.21.15.3:0/2869731628 >> v1:172.21.15.3:6789/0 conn(0x7f044006e550 legacy=0x7f044006e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:46.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.747+0000 7fc999861700 1 -- 172.21.15.3:0/1606771143 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc994139170 msgr2=0x7fc99413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:46.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.747+0000 7fc999861700 1 --2- 172.21.15.3:0/1606771143 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc994139170 0x7fc99413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:46.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.747+0000 7fc999861700 1 -- 172.21.15.3:0/1606771143 >> v1:172.21.15.3:6789/0 conn(0x7fc99409d6f0 legacy=0x7fc99409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:46.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.747+0000 7fc999861700 1 -- 172.21.15.3:0/1606771143 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99409a500 con 0x7fc99409e1d0 2023-10-03T10:19:46.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.747+0000 7f043f7fe700 1 -- 172.21.15.3:0/2869731628 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0440070e70 con 0x7f0440070060 2023-10-03T10:19:46.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.747+0000 7fc999861700 1 --2- 172.21.15.3:0/1606771143 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc99409e1d0 0x7fc994138c30 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fc984009870 tx=0x7fc984009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abf67b90d67f327 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:46.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.747+0000 7fc99a863700 1 --2- 172.21.15.3:0/1606771143 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc994139170 0x7fc99413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-03T10:19:46.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.747+0000 7f043f7fe700 1 --2- 172.21.15.3:0/2869731628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0440070060 0x7f0440139740 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f0438009d90 tx=0x7f043800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621296415c445ea1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:46.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.747+0000 7fc999060700 1 -- 172.21.15.3:0/1606771143 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984011830 con 0x7fc99409e1d0 2023-10-03T10:19:46.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.747+0000 7fc999060700 1 -- 172.21.15.3:0/1606771143 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc984011990 con 0x7fc99409e1d0 2023-10-03T10:19:46.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.747+0000 7fc999060700 1 -- 172.21.15.3:0/1606771143 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984011d00 con 0x7fc99409e1d0 2023-10-03T10:19:46.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.747+0000 7f043dffb700 1 -- 172.21.15.3:0/2869731628 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043800e070 con 0x7f0440070060 2023-10-03T10:19:46.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.748+0000 7f043dffb700 1 -- 172.21.15.3:0/2869731628 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f043800bbd0 con 0x7f0440070060 2023-10-03T10:19:46.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.748+0000 7f043dffb700 1 -- 172.21.15.3:0/2869731628 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0438012430 con 0x7f0440070060 2023-10-03T10:19:46.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.748+0000 7f0445384700 1 -- 172.21.15.3:0/2869731628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0440070060 msgr2=0x7f0440139740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:46.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.748+0000 7f0445384700 1 --2- 172.21.15.3:0/2869731628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0440070060 0x7f0440139740 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f0438009d90 tx=0x7f043800b330 comp rx=0 tx=0).stop 2023-10-03T10:19:46.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.748+0000 7fc99c2c6700 1 -- 172.21.15.3:0/1606771143 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc99409e1d0 msgr2=0x7fc994138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:46.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.748+0000 7fc99c2c6700 1 --2- 172.21.15.3:0/1606771143 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc99409e1d0 0x7fc994138c30 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fc984009870 tx=0x7fc984009b80 comp rx=0 tx=0).stop 2023-10-03T10:19:46.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.748+0000 7fc99c2c6700 1 -- 172.21.15.3:0/1606771143 shutdown_connections 2023-10-03T10:19:46.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.748+0000 7fc99c2c6700 1 --2- 172.21.15.3:0/1606771143 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc994139170 0x7fc99413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:46.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.748+0000 7fc99c2c6700 1 --2- 172.21.15.3:0/1606771143 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc99409e1d0 0x7fc994138c30 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:46.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.748+0000 7fc99c2c6700 1 -- 172.21.15.3:0/1606771143 >> 172.21.15.3:0/1606771143 conn(0x7fc994066f50 msgr2=0x7fc99409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:46.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.749+0000 7fc99c2c6700 1 -- 172.21.15.3:0/1606771143 shutdown_connections 2023-10-03T10:19:46.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.749+0000 7fc99c2c6700 1 -- 172.21.15.3:0/1606771143 wait complete. 2023-10-03T10:19:46.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.749+0000 7febcbce7700 1 Processor -- start 2023-10-03T10:19:46.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.749+0000 7f0445384700 1 -- 172.21.15.3:0/2869731628 shutdown_connections 2023-10-03T10:19:46.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.749+0000 7f0445384700 1 --2- 172.21.15.3:0/2869731628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0440070060 0x7f0440139740 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:46.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.749+0000 7f0445384700 1 --2- 172.21.15.3:0/2869731628 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f04401305f0 0x7f04401309c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:46.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.749+0000 7f0445384700 1 -- 172.21.15.3:0/2869731628 >> 172.21.15.3:0/2869731628 conn(0x7f04400a2aa0 msgr2=0x7f04400a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:46.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.749+0000 7f0445384700 1 -- 172.21.15.3:0/2869731628 shutdown_connections 2023-10-03T10:19:46.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.749+0000 7f0445384700 1 -- 172.21.15.3:0/2869731628 wait complete. 2023-10-03T10:19:46.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.749+0000 7febcbce7700 1 -- start start 2023-10-03T10:19:46.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.750+0000 7febcbce7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7febc4130540 0x7febc4130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:46.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.750+0000 7febcbce7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febc4130e50 0x7febc406f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:46.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.750+0000 7fc99c2c6700 1 Processor -- start 2023-10-03T10:19:46.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.750+0000 7febcbce7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febc40716a0 con 0x7febc406fb20 2023-10-03T10:19:46.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.750+0000 7febcbce7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc409a390 con 0x7febc4130540 2023-10-03T10:19:46.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.750+0000 7febcbce7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7febc4071290 con 0x7febc4130e50 2023-10-03T10:19:46.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.750+0000 7febca284700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7febc406fb20 0x7febc4137560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60436/0 (socket says 172.21.15.3:60436) 2023-10-03T10:19:46.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.750+0000 7febca284700 1 -- 172.21.15.3:0/469996382 learned_addr learned my addr 172.21.15.3:0/469996382 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:46.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.750+0000 7f0445384700 1 Processor -- start 2023-10-03T10:19:46.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.750+0000 7fc99c2c6700 1 -- start start 2023-10-03T10:19:46.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.750+0000 7febc9a83700 1 --2- 172.21.15.3:0/469996382 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7febc4130540 0x7febc4130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:46.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.750+0000 7febc9282700 1 --2- 172.21.15.3:0/469996382 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febc4130e50 0x7febc406f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:46.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.751+0000 7febc9282700 1 -- 172.21.15.3:0/469996382 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7febc4130540 msgr2=0x7febc4130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:46.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.751+0000 7febc9282700 1 --2- 172.21.15.3:0/469996382 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7febc4130540 0x7febc4130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:46.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.751+0000 7febc9282700 1 -- 172.21.15.3:0/469996382 >> v1:172.21.15.3:6789/0 conn(0x7febc406fb20 legacy=0x7febc4137560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:46.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.751+0000 7febc9282700 1 -- 172.21.15.3:0/469996382 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc4130340 con 0x7febc4130e50 2023-10-03T10:19:46.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.751+0000 7febc9a83700 1 --2- 172.21.15.3:0/469996382 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7febc4130540 0x7febc4130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:46.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.751+0000 7fc99c2c6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc99409d6f0 0x7fc9941c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:46.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.751+0000 7fc99c2c6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc99409e1d0 0x7fc9941c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:46.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.751+0000 7fc99c2c6700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc994139170 0x7fc99411e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:46.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.751+0000 7fc99c2c6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9941220f0 con 0x7fc99409d6f0 2023-10-03T10:19:46.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.751+0000 7fc99c2c6700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9941222d0 con 0x7fc99409e1d0 2023-10-03T10:19:46.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.751+0000 7fc99c2c6700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9941be250 con 0x7fc994139170 2023-10-03T10:19:46.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.751+0000 7fc99a062700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc99409d6f0 0x7fc9941c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:46.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.751+0000 7fc99a062700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc99409d6f0 0x7fc9941c8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38736/0 (socket says 172.21.15.3:38736) 2023-10-03T10:19:46.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.751+0000 7fc99a062700 1 -- 172.21.15.3:0/3578920825 learned_addr learned my addr 172.21.15.3:0/3578920825 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:46.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.751+0000 7fc999861700 1 --2- 172.21.15.3:0/3578920825 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc99409e1d0 0x7fc9941c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:46.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.751+0000 7fc99a863700 1 --2- 172.21.15.3:0/3578920825 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc994139170 0x7fc99411e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:46.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.752+0000 7fc99a863700 1 -- 172.21.15.3:0/3578920825 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc99409e1d0 msgr2=0x7fc9941c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:46.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.752+0000 7fc99a863700 1 --2- 172.21.15.3:0/3578920825 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc99409e1d0 0x7fc9941c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:46.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.752+0000 7fc99a863700 1 -- 172.21.15.3:0/3578920825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc99409d6f0 msgr2=0x7fc9941c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:46.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.752+0000 7fc99a863700 1 --2- 172.21.15.3:0/3578920825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc99409d6f0 0x7fc9941c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:46.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.752+0000 7fc99a863700 1 -- 172.21.15.3:0/3578920825 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9840094a0 con 0x7fc994139170 2023-10-03T10:19:46.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.752+0000 7fc99a062700 1 --2- 172.21.15.3:0/3578920825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc99409d6f0 0x7fc9941c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:46.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.752+0000 7fc999861700 1 --2- 172.21.15.3:0/3578920825 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc99409e1d0 0x7fc9941c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:46.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.752+0000 7fc99a863700 1 --2- 172.21.15.3:0/3578920825 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc994139170 0x7fc99411e7b0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00ce20 tx=0x7fc98c00ce50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:46.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.752+0000 7febc9282700 1 --2- 172.21.15.3:0/469996382 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febc4130e50 0x7febc406f5e0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7febb400dcc0 tx=0x7febb400f050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffe76319609c8a80 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:46.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.751+0000 7f0445384700 1 -- start start 2023-10-03T10:19:46.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.751+0000 7f0445384700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f044006e550 0x7f0440139cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:46.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.751+0000 7f0445384700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0440070060 0x7f044013a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:46.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.751+0000 7f0445384700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f04401305f0 0x7f044013a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:46.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.751+0000 7f0445384700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044013d540 con 0x7f044006e550 2023-10-03T10:19:46.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.751+0000 7f0445384700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f044013b0d0 con 0x7f04401305f0 2023-10-03T10:19:46.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.751+0000 7f0445384700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f044013d740 con 0x7f0440070060 2023-10-03T10:19:46.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.752+0000 7f043e7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0440070060 0x7f044013a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:46.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.752+0000 7f043e7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0440070060 0x7f044013a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:44680/0 (socket says 172.21.15.3:44680) 2023-10-03T10:19:46.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.752+0000 7f043e7fc700 1 -- 172.21.15.3:0/212557905 learned_addr learned my addr 172.21.15.3:0/212557905 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:46.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.752+0000 7f043effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f044006e550 0x7f0440139cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:46.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.752+0000 7fc98affd700 1 -- 172.21.15.3:0/3578920825 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c00ad00 con 0x7fc994139170 2023-10-03T10:19:46.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.752+0000 7fc98affd700 1 -- 172.21.15.3:0/3578920825 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98c015440 con 0x7fc994139170 2023-10-03T10:19:46.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.752+0000 7fc98affd700 1 -- 172.21.15.3:0/3578920825 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c00aea0 con 0x7fc994139170 2023-10-03T10:19:46.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.752+0000 7febc8a81700 1 -- 172.21.15.3:0/469996382 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb4012070 con 0x7febc4130e50 2023-10-03T10:19:46.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.752+0000 7febc8a81700 1 -- 172.21.15.3:0/469996382 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb400d2e0 con 0x7febc4130e50 2023-10-03T10:19:46.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.752+0000 7febc8a81700 1 -- 172.21.15.3:0/469996382 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb4004230 con 0x7febc4130e50 2023-10-03T10:19:46.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.752+0000 7f043f7fe700 1 --2- 172.21.15.3:0/212557905 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f04401305f0 0x7f044013a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:46.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.752+0000 7f043effd700 1 -- 172.21.15.3:0/212557905 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0440070060 msgr2=0x7f044013a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:46.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.752+0000 7f043effd700 1 --2- 172.21.15.3:0/212557905 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0440070060 0x7f044013a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:46.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.752+0000 7f043effd700 1 -- 172.21.15.3:0/212557905 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f04401305f0 msgr2=0x7f044013a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:46.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.752+0000 7f043effd700 1 --2- 172.21.15.3:0/212557905 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f04401305f0 0x7f044013a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:46.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.752+0000 7f043effd700 1 -- 172.21.15.3:0/212557905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0438009a70 con 0x7f044006e550 2023-10-03T10:19:46.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.752+0000 7f043e7fc700 1 --2- 172.21.15.3:0/212557905 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0440070060 0x7f044013a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:46.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.753+0000 7f043effd700 1 --2- 172.21.15.3:0/212557905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f044006e550 0x7f0440139cb0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f043000d800 tx=0x7f043000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:46.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.753+0000 7f04277fe700 1 -- 172.21.15.3:0/212557905 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04300125f0 con 0x7f044006e550 2023-10-03T10:19:46.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.753+0000 7f04277fe700 1 -- 172.21.15.3:0/212557905 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0430012750 con 0x7f044006e550 2023-10-03T10:19:46.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.753+0000 7f0445384700 1 -- 172.21.15.3:0/212557905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0440133ba0 con 0x7f044006e550 2023-10-03T10:19:46.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.753+0000 7f04277fe700 1 -- 172.21.15.3:0/212557905 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0430012b00 con 0x7f044006e550 2023-10-03T10:19:46.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.753+0000 7f0445384700 1 -- 172.21.15.3:0/212557905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0440071820 con 0x7f044006e550 2023-10-03T10:19:46.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.753+0000 7fc99c2c6700 1 -- 172.21.15.3:0/3578920825 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc994134680 con 0x7fc994139170 2023-10-03T10:19:46.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.753+0000 7fc99c2c6700 1 -- 172.21.15.3:0/3578920825 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9941203c0 con 0x7fc994139170 2023-10-03T10:19:46.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.753+0000 7febcbce7700 1 -- 172.21.15.3:0/469996382 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febc4130e50 msgr2=0x7febc406f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:46.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.753+0000 7febcbce7700 1 --2- 172.21.15.3:0/469996382 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febc4130e50 0x7febc406f5e0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7febb400dcc0 tx=0x7febb400f050 comp rx=0 tx=0).stop 2023-10-03T10:19:46.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.754+0000 7febcbce7700 1 -- 172.21.15.3:0/469996382 shutdown_connections 2023-10-03T10:19:46.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.754+0000 7febcbce7700 1 --2- 172.21.15.3:0/469996382 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7febc4130540 0x7febc4130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:46.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.754+0000 7febcbce7700 1 --2- 172.21.15.3:0/469996382 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febc4130e50 0x7febc406f5e0 secure :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7febb400dcc0 tx=0x7febb400f050 comp rx=0 tx=0).stop 2023-10-03T10:19:46.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.754+0000 7febcbce7700 1 -- 172.21.15.3:0/469996382 >> 172.21.15.3:0/469996382 conn(0x7febc40a2aa0 msgr2=0x7febc40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:46.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.754+0000 7fc99c2c6700 1 -- 172.21.15.3:0/3578920825 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc994074ea0 con 0x7fc994139170 2023-10-03T10:19:46.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.754+0000 7f0445384700 1 -- 172.21.15.3:0/212557905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f044009dd00 con 0x7f044006e550 2023-10-03T10:19:46.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.755+0000 7f04277fe700 1 -- 172.21.15.3:0/212557905 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0430012c60 con 0x7f044006e550 2023-10-03T10:19:46.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.755+0000 7fc98affd700 1 -- 172.21.15.3:0/3578920825 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc98c0157f0 con 0x7fc994139170 2023-10-03T10:19:46.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.755+0000 7febcbce7700 1 -- 172.21.15.3:0/469996382 shutdown_connections 2023-10-03T10:19:46.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.755+0000 7febcbce7700 1 -- 172.21.15.3:0/469996382 wait complete. 2023-10-03T10:19:46.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.755+0000 7fc98affd700 1 -- 172.21.15.3:0/3578920825 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc98c040340 con 0x7fc994139170 2023-10-03T10:19:46.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.756+0000 7febcbce7700 1 Processor -- start 2023-10-03T10:19:46.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.756+0000 7febcbce7700 1 -- start start 2023-10-03T10:19:46.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.757+0000 7febcbce7700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7febc406fb20 0x7febc409bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:46.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.757+0000 7febcbce7700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febc4130540 0x7febc409c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:46.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.757+0000 7febcbce7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febc409c9f0 0x7febc41ca640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:46.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.757+0000 7febcbce7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc409d3d0 con 0x7febc409c9f0 2023-10-03T10:19:46.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.757+0000 7febcbce7700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc411a100 con 0x7febc406fb20 2023-10-03T10:19:46.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.757+0000 7febcbce7700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7febc40718e0 con 0x7febc4130540 2023-10-03T10:19:46.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.757+0000 7febca284700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febc409c9f0 0x7febc41ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:46.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.757+0000 7febca284700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febc409c9f0 0x7febc41ca640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38758/0 (socket says 172.21.15.3:38758) 2023-10-03T10:19:46.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.757+0000 7febca284700 1 -- 172.21.15.3:0/3389285130 learned_addr learned my addr 172.21.15.3:0/3389285130 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:46.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.757+0000 7febca284700 1 -- 172.21.15.3:0/3389285130 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febc4130540 msgr2=0x7febc409c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:46.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.757+0000 7febca284700 1 --2- 172.21.15.3:0/3389285130 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febc4130540 0x7febc409c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:46.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.757+0000 7febca284700 1 -- 172.21.15.3:0/3389285130 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7febc406fb20 msgr2=0x7febc409bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:46.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.757+0000 7febca284700 1 --2- 172.21.15.3:0/3389285130 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7febc406fb20 0x7febc409bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:46.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.757+0000 7febca284700 1 -- 172.21.15.3:0/3389285130 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc00094a0 con 0x7febc409c9f0 2023-10-03T10:19:46.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.758+0000 7febca284700 1 --2- 172.21.15.3:0/3389285130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febc409c9f0 0x7febc41ca640 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7febbc002060 tx=0x7febbc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:46.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.755+0000 7f04277fe700 1 -- 172.21.15.3:0/212557905 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f043003f910 con 0x7f044006e550 2023-10-03T10:19:46.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.761+0000 7febba7fc700 1 -- 172.21.15.3:0/3389285130 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc00ad80 con 0x7febc409c9f0 2023-10-03T10:19:46.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.761+0000 7febba7fc700 1 -- 172.21.15.3:0/3389285130 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febbc0070d0 con 0x7febc409c9f0 2023-10-03T10:19:46.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.761+0000 7febba7fc700 1 -- 172.21.15.3:0/3389285130 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc007460 con 0x7febc409c9f0 2023-10-03T10:19:46.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.761+0000 7f04277fe700 1 -- 172.21.15.3:0/212557905 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0430011750 con 0x7f044006e550 2023-10-03T10:19:46.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.762+0000 7febcbce7700 1 -- 172.21.15.3:0/3389285130 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb400d940 con 0x7febc409c9f0 2023-10-03T10:19:46.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.762+0000 7febcbce7700 1 -- 172.21.15.3:0/3389285130 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc4119030 con 0x7febc409c9f0 2023-10-03T10:19:46.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.762+0000 7febba7fc700 1 -- 172.21.15.3:0/3389285130 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7febbc0075c0 con 0x7febc409c9f0 2023-10-03T10:19:46.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.763+0000 7febcbce7700 1 -- 172.21.15.3:0/3389285130 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febc4074ea0 con 0x7febc409c9f0 2023-10-03T10:19:46.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.763+0000 7febba7fc700 1 -- 172.21.15.3:0/3389285130 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7febbc03f9f0 con 0x7febc409c9f0 2023-10-03T10:19:46.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.764+0000 7fc98affd700 1 -- 172.21.15.3:0/3578920825 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc98c014990 con 0x7fc994139170 2023-10-03T10:19:46.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.771+0000 7febba7fc700 1 -- 172.21.15.3:0/3389285130 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7febbc01acd0 con 0x7febc409c9f0 2023-10-03T10:19:46.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.941+0000 7fd6ffe18700 1 -- 172.21.15.3:0/4154416798 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd6f8119ff0 con 0x7fd6f8131810 2023-10-03T10:19:46.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.942+0000 7fd6ee7fc700 1 -- 172.21.15.3:0/4154416798 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd6f8119ff0 con 0x7fd6f8131810 2023-10-03T10:19:46.942 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:46.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.944+0000 7fd6ffe18700 1 -- 172.21.15.3:0/4154416798 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6f8131810 msgr2=0x7fd6f809c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:46.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.944+0000 7fd6ffe18700 1 --2- 172.21.15.3:0/4154416798 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6f8131810 0x7fd6f809c630 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd6f400c450 tx=0x7fd6f400c760 comp rx=0 tx=0).stop 2023-10-03T10:19:46.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.944+0000 7fd6ffe18700 1 -- 172.21.15.3:0/4154416798 shutdown_connections 2023-10-03T10:19:46.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.944+0000 7fd6ffe18700 1 --2- 172.21.15.3:0/4154416798 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fd6f8136f50 0x7fd6f809d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:46.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.944+0000 7fd6ffe18700 1 --2- 172.21.15.3:0/4154416798 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd6f8131810 0x7fd6f809c630 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:46.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.944+0000 7fd6ffe18700 1 --2- 172.21.15.3:0/4154416798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6f8134630 0x7fd6f809cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:46.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.944+0000 7fd6ffe18700 1 -- 172.21.15.3:0/4154416798 >> 172.21.15.3:0/4154416798 conn(0x7fd6f80a2aa0 msgr2=0x7fd6f8134a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:46.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.944+0000 7fd6ffe18700 1 -- 172.21.15.3:0/4154416798 shutdown_connections 2023-10-03T10:19:46.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.944+0000 7fd6ffe18700 1 -- 172.21.15.3:0/4154416798 wait complete. 2023-10-03T10:19:46.958 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.971+0000 7f0445384700 1 -- 172.21.15.3:0/212557905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f044013bde0 con 0x7f044006e550 2023-10-03T10:19:46.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.972+0000 7f04277fe700 1 -- 172.21.15.3:0/212557905 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0430013390 con 0x7f044006e550 2023-10-03T10:19:46.972 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:46.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.976+0000 7f0445384700 1 -- 172.21.15.3:0/212557905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f044006e550 msgr2=0x7f0440139cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:46.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.976+0000 7f0445384700 1 --2- 172.21.15.3:0/212557905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f044006e550 0x7f0440139cb0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f043000d800 tx=0x7f043000db10 comp rx=0 tx=0).stop 2023-10-03T10:19:46.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.976+0000 7f0445384700 1 -- 172.21.15.3:0/212557905 shutdown_connections 2023-10-03T10:19:46.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.976+0000 7f0445384700 1 --2- 172.21.15.3:0/212557905 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f04401305f0 0x7f044013a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:46.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.976+0000 7f0445384700 1 --2- 172.21.15.3:0/212557905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f044006e550 0x7f0440139cb0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:46.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.976+0000 7f0445384700 1 --2- 172.21.15.3:0/212557905 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0440070060 0x7f044013a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:46.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.976+0000 7f0445384700 1 -- 172.21.15.3:0/212557905 >> 172.21.15.3:0/212557905 conn(0x7f04400a2aa0 msgr2=0x7f04400a1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:46.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.976+0000 7f0445384700 1 -- 172.21.15.3:0/212557905 shutdown_connections 2023-10-03T10:19:46.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.976+0000 7f0445384700 1 -- 172.21.15.3:0/212557905 wait complete. 2023-10-03T10:19:46.995 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:46.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.998+0000 7fc99c2c6700 1 -- 172.21.15.3:0/3578920825 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc994121a90 con 0x7fc994139170 2023-10-03T10:19:46.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:46.998+0000 7fc98affd700 1 -- 172.21.15.3:0/3578920825 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc98c01a030 con 0x7fc994139170 2023-10-03T10:19:46.999 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:47.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:47.000+0000 7fc99c2c6700 1 -- 172.21.15.3:0/3578920825 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc994139170 msgr2=0x7fc99411e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:47.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:47.000+0000 7fc99c2c6700 1 --2- 172.21.15.3:0/3578920825 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc994139170 0x7fc99411e7b0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00ce20 tx=0x7fc98c00ce50 comp rx=0 tx=0).stop 2023-10-03T10:19:47.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:47.000+0000 7fc99c2c6700 1 -- 172.21.15.3:0/3578920825 shutdown_connections 2023-10-03T10:19:47.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:47.000+0000 7fc99c2c6700 1 --2- 172.21.15.3:0/3578920825 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc994139170 0x7fc99411e7b0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:47.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:47.000+0000 7fc99c2c6700 1 --2- 172.21.15.3:0/3578920825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc99409d6f0 0x7fc9941c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:47.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:47.000+0000 7fc99c2c6700 1 --2- 172.21.15.3:0/3578920825 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc99409e1d0 0x7fc9941c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:47.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:47.000+0000 7fc99c2c6700 1 -- 172.21.15.3:0/3578920825 >> 172.21.15.3:0/3578920825 conn(0x7fc994066f50 msgr2=0x7fc99409b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:47.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:47.001+0000 7fc99c2c6700 1 -- 172.21.15.3:0/3578920825 shutdown_connections 2023-10-03T10:19:47.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:47.001+0000 7fc99c2c6700 1 -- 172.21.15.3:0/3578920825 wait complete. 2023-10-03T10:19:47.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:47.009+0000 7febcbce7700 1 -- 172.21.15.3:0/3389285130 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7febc409da50 con 0x7febc409c9f0 2023-10-03T10:19:47.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:47.010+0000 7febba7fc700 1 -- 172.21.15.3:0/3389285130 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7febbc01a2c0 con 0x7febc409c9f0 2023-10-03T10:19:47.010 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:47.012+0000 7febcbce7700 1 -- 172.21.15.3:0/3389285130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febc409c9f0 msgr2=0x7febc41ca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:47.012+0000 7febcbce7700 1 --2- 172.21.15.3:0/3389285130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febc409c9f0 0x7febc41ca640 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7febbc002060 tx=0x7febbc00a340 comp rx=0 tx=0).stop 2023-10-03T10:19:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:47.012+0000 7febcbce7700 1 -- 172.21.15.3:0/3389285130 shutdown_connections 2023-10-03T10:19:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:47.012+0000 7febcbce7700 1 --2- 172.21.15.3:0/3389285130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febc409c9f0 0x7febc41ca640 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:47.012+0000 7febcbce7700 1 --2- 172.21.15.3:0/3389285130 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7febc406fb20 0x7febc409bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:47.012+0000 7febcbce7700 1 --2- 172.21.15.3:0/3389285130 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7febc4130540 0x7febc409c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:47.012+0000 7febcbce7700 1 -- 172.21.15.3:0/3389285130 >> 172.21.15.3:0/3389285130 conn(0x7febc40a2aa0 msgr2=0x7febc412b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:47.012+0000 7febcbce7700 1 -- 172.21.15.3:0/3389285130 shutdown_connections 2023-10-03T10:19:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:47.012+0000 7febcbce7700 1 -- 172.21.15.3:0/3389285130 wait complete. 2023-10-03T10:19:47.014 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:47.026 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:47.960 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:47.996 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:48.016 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:48.027 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:48.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.114+0000 7f7dfdb2f700 1 Processor -- start 2023-10-03T10:19:48.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.114+0000 7f7dfdb2f700 1 -- start start 2023-10-03T10:19:48.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.115+0000 7f7dfdb2f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7df806f5d0 0x7f7df812ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:48.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.115+0000 7f7dfdb2f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7df8135c20 0x7f7df8138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:48.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.115+0000 7f7dfdb2f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7df806fa40 con 0x7f7df812cf60 2023-10-03T10:19:48.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.115+0000 7f7dfdb2f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7df812cd90 con 0x7f7df8135c20 2023-10-03T10:19:48.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.115+0000 7f7dfdb2f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df8071780 con 0x7f7df806f5d0 2023-10-03T10:19:48.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.115+0000 7f7df6ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7df806f5d0 0x7f7df812ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:48.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.115+0000 7f7df6ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7df806f5d0 0x7f7df812ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:43298/0 (socket says 172.21.15.3:43298) 2023-10-03T10:19:48.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.115+0000 7f7df6ffd700 1 -- 172.21.15.3:0/2240840889 learned_addr learned my addr 172.21.15.3:0/2240840889 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:48.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.115+0000 7f7df7fff700 1 --2- 172.21.15.3:0/2240840889 >> v2:172.21.15.3:3300/0 conn(0x7f7df8135c20 0x7f7df8138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:48.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.115+0000 7f7df6ffd700 1 -- 172.21.15.3:0/2240840889 >> v2:172.21.15.3:3300/0 conn(0x7f7df8135c20 msgr2=0x7f7df8138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:48.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.116+0000 7f7df6ffd700 1 --2- 172.21.15.3:0/2240840889 >> v2:172.21.15.3:3300/0 conn(0x7f7df8135c20 0x7f7df8138000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.116+0000 7f7df6ffd700 1 -- 172.21.15.3:0/2240840889 >> v1:172.21.15.3:6789/0 conn(0x7f7df812cf60 legacy=0x7f7df812f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:48.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.116+0000 7f7df6ffd700 1 -- 172.21.15.3:0/2240840889 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df8071a40 con 0x7f7df806f5d0 2023-10-03T10:19:48.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.116+0000 7f7df6ffd700 1 --2- 172.21.15.3:0/2240840889 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7df806f5d0 0x7f7df812ffc0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0098f0 tx=0x7f7dec009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5c64ead2598a05d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:48.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.116+0000 7f7df67fc700 1 -- 172.21.15.3:0/2240840889 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec010a90 con 0x7f7df806f5d0 2023-10-03T10:19:48.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.116+0000 7f7df67fc700 1 -- 172.21.15.3:0/2240840889 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dec010bf0 con 0x7f7df806f5d0 2023-10-03T10:19:48.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.116+0000 7f7df67fc700 1 -- 172.21.15.3:0/2240840889 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec018440 con 0x7f7df806f5d0 2023-10-03T10:19:48.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.117+0000 7f7dfdb2f700 1 -- 172.21.15.3:0/2240840889 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7df806f5d0 msgr2=0x7f7df812ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:48.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.117+0000 7f7dfdb2f700 1 --2- 172.21.15.3:0/2240840889 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7df806f5d0 0x7f7df812ffc0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0098f0 tx=0x7f7dec009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:48.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.117+0000 7f7dfdb2f700 1 -- 172.21.15.3:0/2240840889 shutdown_connections 2023-10-03T10:19:48.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.117+0000 7f7dfdb2f700 1 --2- 172.21.15.3:0/2240840889 >> v2:172.21.15.3:3300/0 conn(0x7f7df8135c20 0x7f7df8138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.117+0000 7f7dfdb2f700 1 --2- 172.21.15.3:0/2240840889 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7df806f5d0 0x7f7df812ffc0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.117+0000 7f7dfdb2f700 1 -- 172.21.15.3:0/2240840889 >> 172.21.15.3:0/2240840889 conn(0x7f7df80a2aa0 msgr2=0x7f7df80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:48.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.117+0000 7f7dfdb2f700 1 -- 172.21.15.3:0/2240840889 shutdown_connections 2023-10-03T10:19:48.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.117+0000 7f7dfdb2f700 1 -- 172.21.15.3:0/2240840889 wait complete. 2023-10-03T10:19:48.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.118+0000 7f7dfdb2f700 1 Processor -- start 2023-10-03T10:19:48.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.119+0000 7f7dfdb2f700 1 -- start start 2023-10-03T10:19:48.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.119+0000 7f7dfdb2f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df806f5d0 0x7f7df8099d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:48.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.119+0000 7f7dfdb2f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7df812cf60 0x7f7df809a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:48.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.119+0000 7f7dfdb2f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7df8135c20 0x7f7df809a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:48.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.119+0000 7f7dfdb2f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df809c280 con 0x7f7df806f5d0 2023-10-03T10:19:48.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.119+0000 7f7dfdb2f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df8129fa0 con 0x7f7df812cf60 2023-10-03T10:19:48.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.119+0000 7f7dfdb2f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7df809d650 con 0x7f7df8135c20 2023-10-03T10:19:48.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.120+0000 7f7df7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7df8135c20 0x7f7df809a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:48.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.120+0000 7f7df6ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7df812cf60 0x7f7df809a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:48.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.120+0000 7f7df7fff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7df8135c20 0x7f7df809a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:44696/0 (socket says 172.21.15.3:44696) 2023-10-03T10:19:48.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.120+0000 7f7df6ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7df812cf60 0x7f7df809a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:43310/0 (socket says 172.21.15.3:43310) 2023-10-03T10:19:48.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.120+0000 7f7df7fff700 1 -- 172.21.15.3:0/3503431524 learned_addr learned my addr 172.21.15.3:0/3503431524 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:48.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.120+0000 7f7df77fe700 1 --2- 172.21.15.3:0/3503431524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df806f5d0 0x7f7df8099d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:48.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.120+0000 7f7df7fff700 1 -- 172.21.15.3:0/3503431524 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7df812cf60 msgr2=0x7f7df809a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:48.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.120+0000 7f7df7fff700 1 --2- 172.21.15.3:0/3503431524 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7df812cf60 0x7f7df809a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.120+0000 7f7df7fff700 1 -- 172.21.15.3:0/3503431524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df806f5d0 msgr2=0x7f7df8099d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:48.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.120+0000 7f7df7fff700 1 --2- 172.21.15.3:0/3503431524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df806f5d0 0x7f7df8099d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.120+0000 7f7df7fff700 1 -- 172.21.15.3:0/3503431524 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dec009550 con 0x7f7df8135c20 2023-10-03T10:19:48.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.120+0000 7f7df77fe700 1 --2- 172.21.15.3:0/3503431524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df806f5d0 0x7f7df8099d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:48.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.120+0000 7f7df7fff700 1 --2- 172.21.15.3:0/3503431524 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7df8135c20 0x7f7df809a7d0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f7df000efc0 tx=0x7f7df000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:48.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.121+0000 7f7ddffff700 1 -- 172.21.15.3:0/3503431524 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df000cc90 con 0x7f7df8135c20 2023-10-03T10:19:48.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.121+0000 7f7ddffff700 1 -- 172.21.15.3:0/3503431524 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df000f040 con 0x7f7df8135c20 2023-10-03T10:19:48.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.121+0000 7f7ddffff700 1 -- 172.21.15.3:0/3503431524 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df0014480 con 0x7f7df8135c20 2023-10-03T10:19:48.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.121+0000 7f7dfdb2f700 1 -- 172.21.15.3:0/3503431524 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df809cde0 con 0x7f7df8135c20 2023-10-03T10:19:48.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.121+0000 7f7dfdb2f700 1 -- 172.21.15.3:0/3503431524 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df8129770 con 0x7f7df8135c20 2023-10-03T10:19:48.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.122+0000 7f7ddffff700 1 -- 172.21.15.3:0/3503431524 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7df0014600 con 0x7f7df8135c20 2023-10-03T10:19:48.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.122+0000 7f7dfdb2f700 1 -- 172.21.15.3:0/3503431524 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7df809aeb0 con 0x7f7df8135c20 2023-10-03T10:19:48.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.122+0000 7f7ddffff700 1 -- 172.21.15.3:0/3503431524 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7df003fed0 con 0x7f7df8135c20 2023-10-03T10:19:48.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.132+0000 7f7ddffff700 1 -- 172.21.15.3:0/3503431524 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7df00118b0 con 0x7f7df8135c20 2023-10-03T10:19:48.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.161+0000 7f5ee0982700 1 Processor -- start 2023-10-03T10:19:48.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.161+0000 7f5ee0982700 1 -- start start 2023-10-03T10:19:48.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.162+0000 7f5ee0982700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5edc06e570 0x7f5edc06e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:48.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.162+0000 7f5ee0982700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5edc06ee80 0x7f5edc134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:48.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.162+0000 7f5ee0982700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5edc070ee0 con 0x7f5edc09eca0 2023-10-03T10:19:48.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.162+0000 7f5ee0982700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5edc071480 con 0x7f5edc06e570 2023-10-03T10:19:48.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.162+0000 7f5ee0982700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5edc071b90 con 0x7f5edc06ee80 2023-10-03T10:19:48.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.162+0000 7f5edad9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5edc06ee80 0x7f5edc134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:48.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.162+0000 7f5eda59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5edc06e570 0x7f5edc06e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:48.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.162+0000 7f5edad9d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5edc06ee80 0x7f5edc134d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:44706/0 (socket says 172.21.15.3:44706) 2023-10-03T10:19:48.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.162+0000 7f5ed9d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5edc09eca0 0x7f5edc134610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60456/0 (socket says 172.21.15.3:60456) 2023-10-03T10:19:48.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.162+0000 7f5edad9d700 1 -- 172.21.15.3:0/3969358693 learned_addr learned my addr 172.21.15.3:0/3969358693 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:48.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.162+0000 7f5eda59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5edc06e570 0x7f5edc06e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38776/0 (socket says 172.21.15.3:38776) 2023-10-03T10:19:48.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.162+0000 7f5ed959a700 1 -- 172.21.15.3:0/3969358693 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1122141618 0 0) 0x7f5edc070ee0 con 0x7f5edc09eca0 2023-10-03T10:19:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.162+0000 7f5ed959a700 1 -- 172.21.15.3:0/3969358693 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ec40035b0 con 0x7f5edc09eca0 2023-10-03T10:19:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.162+0000 7f5edad9d700 1 -- 172.21.15.3:0/3969358693 >> v2:172.21.15.3:3300/0 conn(0x7f5edc06e570 msgr2=0x7f5edc06e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.163+0000 7f5edad9d700 1 --2- 172.21.15.3:0/3969358693 >> v2:172.21.15.3:3300/0 conn(0x7f5edc06e570 0x7f5edc06e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.163+0000 7f5edad9d700 1 -- 172.21.15.3:0/3969358693 >> v1:172.21.15.3:6789/0 conn(0x7f5edc09eca0 legacy=0x7f5edc134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.163+0000 7f5edad9d700 1 -- 172.21.15.3:0/3969358693 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5edc0711a0 con 0x7f5edc06ee80 2023-10-03T10:19:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.163+0000 7f5eda59c700 1 --2- 172.21.15.3:0/3969358693 >> v2:172.21.15.3:3300/0 conn(0x7f5edc06e570 0x7f5edc06e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.163+0000 7f5edad9d700 1 --2- 172.21.15.3:0/3969358693 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5edc06ee80 0x7f5edc134d20 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009e50 tx=0x7f5ed400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcff9020e8a46230 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.163+0000 7f5ed959a700 1 -- 172.21.15.3:0/3969358693 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed400e070 con 0x7f5edc06ee80 2023-10-03T10:19:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.163+0000 7f5ed959a700 1 -- 172.21.15.3:0/3969358693 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed400b910 con 0x7f5edc06ee80 2023-10-03T10:19:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.163+0000 7f5ed959a700 1 -- 172.21.15.3:0/3969358693 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed400bcc0 con 0x7f5edc06ee80 2023-10-03T10:19:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.164+0000 7f5ee0982700 1 -- 172.21.15.3:0/3969358693 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5edc06ee80 msgr2=0x7f5edc134d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:48.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.164+0000 7f5ee0982700 1 --2- 172.21.15.3:0/3969358693 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5edc06ee80 0x7f5edc134d20 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009e50 tx=0x7f5ed400b330 comp rx=0 tx=0).stop 2023-10-03T10:19:48.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.164+0000 7f5ee0982700 1 -- 172.21.15.3:0/3969358693 shutdown_connections 2023-10-03T10:19:48.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.164+0000 7f5ee0982700 1 --2- 172.21.15.3:0/3969358693 >> v2:172.21.15.3:3300/0 conn(0x7f5edc06e570 0x7f5edc06e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.164+0000 7f5ee0982700 1 --2- 172.21.15.3:0/3969358693 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5edc06ee80 0x7f5edc134d20 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.164+0000 7f5ee0982700 1 -- 172.21.15.3:0/3969358693 >> 172.21.15.3:0/3969358693 conn(0x7f5edc0a2aa0 msgr2=0x7f5edc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:48.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.164+0000 7f5ee0982700 1 -- 172.21.15.3:0/3969358693 shutdown_connections 2023-10-03T10:19:48.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.164+0000 7f5ee0982700 1 -- 172.21.15.3:0/3969358693 wait complete. 2023-10-03T10:19:48.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.165+0000 7f5ee0982700 1 Processor -- start 2023-10-03T10:19:48.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.165+0000 7f5ee0982700 1 -- start start 2023-10-03T10:19:48.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.166+0000 7f5ee0982700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5edc06e570 0x7f5edc13b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:48.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.166+0000 7f5ee0982700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5edc06ee80 0x7f5edc1348b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:48.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.166+0000 7f5ee0982700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5edc134df0 0x7f5edc139e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:48.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.166+0000 7f5ee0982700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5edc13d0c0 con 0x7f5edc134df0 2023-10-03T10:19:48.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.166+0000 7f5ee0982700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5edc13d380 con 0x7f5edc06e570 2023-10-03T10:19:48.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.166+0000 7f5ee0982700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5edc13ddf0 con 0x7f5edc06ee80 2023-10-03T10:19:48.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.166+0000 7f5edad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5edc134df0 0x7f5edc139e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:48.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.166+0000 7f5edad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5edc134df0 0x7f5edc139e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38786/0 (socket says 172.21.15.3:38786) 2023-10-03T10:19:48.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.166+0000 7f5edad9d700 1 -- 172.21.15.3:0/1332965309 learned_addr learned my addr 172.21.15.3:0/1332965309 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:48.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.166+0000 7f5ed9d9b700 1 --2- 172.21.15.3:0/1332965309 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5edc06ee80 0x7f5edc1348b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:48.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.166+0000 7f5eda59c700 1 --2- 172.21.15.3:0/1332965309 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5edc06e570 0x7f5edc13b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:48.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.166+0000 7f5edad9d700 1 -- 172.21.15.3:0/1332965309 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5edc06ee80 msgr2=0x7f5edc1348b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:48.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.166+0000 7f5edad9d700 1 --2- 172.21.15.3:0/1332965309 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5edc06ee80 0x7f5edc1348b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.166+0000 7f5edad9d700 1 -- 172.21.15.3:0/1332965309 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5edc06e570 msgr2=0x7f5edc13b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:48.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.166+0000 7f5edad9d700 1 --2- 172.21.15.3:0/1332965309 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5edc06e570 0x7f5edc13b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.166+0000 7f5edad9d700 1 -- 172.21.15.3:0/1332965309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ecc0094a0 con 0x7f5edc134df0 2023-10-03T10:19:48.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.167+0000 7f5eda59c700 1 --2- 172.21.15.3:0/1332965309 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5edc06e570 0x7f5edc13b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:48.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.167+0000 7f5edad9d700 1 --2- 172.21.15.3:0/1332965309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5edc134df0 0x7f5edc139e80 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f5edc071cd0 tx=0x7f5ed4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:48.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.167+0000 7f5ecaffd700 1 -- 172.21.15.3:0/1332965309 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed40048b0 con 0x7f5edc134df0 2023-10-03T10:19:48.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.167+0000 7f5ecaffd700 1 -- 172.21.15.3:0/1332965309 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed40099b0 con 0x7f5edc134df0 2023-10-03T10:19:48.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.167+0000 7f5ecaffd700 1 -- 172.21.15.3:0/1332965309 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed401c680 con 0x7f5edc134df0 2023-10-03T10:19:48.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.168+0000 7f5ee0982700 1 -- 172.21.15.3:0/1332965309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed4009b30 con 0x7f5edc134df0 2023-10-03T10:19:48.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.168+0000 7f5ee0982700 1 -- 172.21.15.3:0/1332965309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5edc13a3c0 con 0x7f5edc134df0 2023-10-03T10:19:48.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.169+0000 7f5ee0982700 1 -- 172.21.15.3:0/1332965309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5edc074ea0 con 0x7f5edc134df0 2023-10-03T10:19:48.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.169+0000 7f5ecaffd700 1 -- 172.21.15.3:0/1332965309 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5ed401c7e0 con 0x7f5edc134df0 2023-10-03T10:19:48.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.169+0000 7f5ecaffd700 1 -- 172.21.15.3:0/1332965309 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5ed4041230 con 0x7f5edc134df0 2023-10-03T10:19:48.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.178+0000 7f5ecaffd700 1 -- 172.21.15.3:0/1332965309 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5ed402d940 con 0x7f5edc134df0 2023-10-03T10:19:48.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.198+0000 7f3efa472700 1 Processor -- start 2023-10-03T10:19:48.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.198+0000 7f3efa472700 1 -- start start 2023-10-03T10:19:48.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.198+0000 7f3efa472700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ef406c0e0 0x7f3ef409ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:48.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.198+0000 7f3efa472700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3ef4134ec0 0x7f3ef41379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:48.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.198+0000 7f3efa472700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef4134980 con 0x7f3ef4137f20 2023-10-03T10:19:48.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.198+0000 7f3efa472700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ef406c4b0 con 0x7f3ef4134ec0 2023-10-03T10:19:48.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.198+0000 7f3efa472700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef40a09c0 con 0x7f3ef406c0e0 2023-10-03T10:19:48.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.199+0000 7f3ef3fff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ef406c0e0 0x7f3ef409ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:48.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.199+0000 7f3ef8a0f700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3ef4137f20 0x7f3ef413ed10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60470/0 (socket says 172.21.15.3:60470) 2023-10-03T10:19:48.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.199+0000 7f3ef8a0f700 1 -- 172.21.15.3:0/443911945 learned_addr learned my addr 172.21.15.3:0/443911945 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:48.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.199+0000 7f3ef37fe700 1 --2- 172.21.15.3:0/443911945 >> v2:172.21.15.3:3300/0 conn(0x7f3ef4134ec0 0x7f3ef41379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:48.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.199+0000 7f7ccbda3700 1 Processor -- start 2023-10-03T10:19:48.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.199+0000 7f3ef37fe700 1 -- 172.21.15.3:0/443911945 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ef406c0e0 msgr2=0x7f3ef409ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:48.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.199+0000 7f3ef37fe700 1 --2- 172.21.15.3:0/443911945 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ef406c0e0 0x7f3ef409ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.199+0000 7f3ef37fe700 1 -- 172.21.15.3:0/443911945 >> v1:172.21.15.3:6789/0 conn(0x7f3ef4137f20 legacy=0x7f3ef413ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:48.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.199+0000 7f3ef37fe700 1 -- 172.21.15.3:0/443911945 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef40a0600 con 0x7f3ef4134ec0 2023-10-03T10:19:48.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.199+0000 7f3ef2ffd700 1 -- 172.21.15.3:0/443911945 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2940870556 0 0) 0x7f3ef4134980 con 0x7f3ef4137f20 2023-10-03T10:19:48.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.199+0000 7f7ccbda3700 1 -- start start 2023-10-03T10:19:48.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.199+0000 7f3ef37fe700 1 --2- 172.21.15.3:0/443911945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ef4134ec0 0x7f3ef41379e0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8009840 tx=0x7f3ee8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdee039c52d9fd server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:48.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.200+0000 7f3ef2ffd700 1 -- 172.21.15.3:0/443911945 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8011830 con 0x7f3ef4134ec0 2023-10-03T10:19:48.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.200+0000 7f3ef2ffd700 1 -- 172.21.15.3:0/443911945 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee8011990 con 0x7f3ef4134ec0 2023-10-03T10:19:48.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.200+0000 7f3ef2ffd700 1 -- 172.21.15.3:0/443911945 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8011d20 con 0x7f3ef4134ec0 2023-10-03T10:19:48.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.200+0000 7f7ccbda3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cc4066410 0x7f7cc406e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:48.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.200+0000 7f7ccbda3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cc406ef70 0x7f7cc412fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:48.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.200+0000 7f7ccbda3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cc4099b50 con 0x7f7cc406eb80 2023-10-03T10:19:48.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.200+0000 7f7ccbda3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc40667e0 con 0x7f7cc406ef70 2023-10-03T10:19:48.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.200+0000 7f7ccbda3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc4099f60 con 0x7f7cc4066410 2023-10-03T10:19:48.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.200+0000 7f7cca340700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cc406ef70 0x7f7cc412fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:48.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.200+0000 7f7cca340700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cc406ef70 0x7f7cc412fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:43352/0 (socket says 172.21.15.3:43352) 2023-10-03T10:19:48.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.200+0000 7f7cca340700 1 -- 172.21.15.3:0/4191667648 learned_addr learned my addr 172.21.15.3:0/4191667648 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:48.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.201+0000 7f7cca340700 1 -- 172.21.15.3:0/4191667648 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cc4066410 msgr2=0x7f7cc406e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:48.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.201+0000 7f7cca340700 1 --2- 172.21.15.3:0/4191667648 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cc4066410 0x7f7cc406e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.201+0000 7f7cca340700 1 -- 172.21.15.3:0/4191667648 >> v1:172.21.15.3:6789/0 conn(0x7f7cc406eb80 legacy=0x7f7cc412f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:48.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.201+0000 7f7cca340700 1 -- 172.21.15.3:0/4191667648 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc409a920 con 0x7f7cc406ef70 2023-10-03T10:19:48.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.201+0000 7f3efa472700 1 -- 172.21.15.3:0/443911945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ef4134ec0 msgr2=0x7f3ef41379e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:48.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.201+0000 7f3efa472700 1 --2- 172.21.15.3:0/443911945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ef4134ec0 0x7f3ef41379e0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8009840 tx=0x7f3ee8009b50 comp rx=0 tx=0).stop 2023-10-03T10:19:48.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.201+0000 7f3efa472700 1 -- 172.21.15.3:0/443911945 shutdown_connections 2023-10-03T10:19:48.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.201+0000 7f3efa472700 1 --2- 172.21.15.3:0/443911945 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ef406c0e0 0x7f3ef409ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.201+0000 7f3efa472700 1 --2- 172.21.15.3:0/443911945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ef4134ec0 0x7f3ef41379e0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.201+0000 7f3efa472700 1 -- 172.21.15.3:0/443911945 >> 172.21.15.3:0/443911945 conn(0x7f3ef40a4210 msgr2=0x7f3ef40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:48.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.201+0000 7f3efa472700 1 -- 172.21.15.3:0/443911945 shutdown_connections 2023-10-03T10:19:48.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.201+0000 7f7cca340700 1 --2- 172.21.15.3:0/4191667648 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cc406ef70 0x7f7cc412fa30 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0009920 tx=0x7f7cc0009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a04a292748818be1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:48.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.201+0000 7f3efa472700 1 -- 172.21.15.3:0/443911945 wait complete. 2023-10-03T10:19:48.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.201+0000 7f7cc8b3d700 1 -- 172.21.15.3:0/4191667648 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc000ba60 con 0x7f7cc406ef70 2023-10-03T10:19:48.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.201+0000 7f7cc8b3d700 1 -- 172.21.15.3:0/4191667648 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cc000bbc0 con 0x7f7cc406ef70 2023-10-03T10:19:48.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.201+0000 7f7cc8b3d700 1 -- 172.21.15.3:0/4191667648 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0009e70 con 0x7f7cc406ef70 2023-10-03T10:19:48.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.202+0000 7f7ccbda3700 1 -- 172.21.15.3:0/4191667648 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cc406ef70 msgr2=0x7f7cc412fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:48.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.202+0000 7f7ccbda3700 1 --2- 172.21.15.3:0/4191667648 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cc406ef70 0x7f7cc412fa30 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0009920 tx=0x7f7cc0009c30 comp rx=0 tx=0).stop 2023-10-03T10:19:48.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.202+0000 7f3efa472700 1 Processor -- start 2023-10-03T10:19:48.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.202+0000 7f3efa472700 1 -- start start 2023-10-03T10:19:48.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.203+0000 7f7ccbda3700 1 -- 172.21.15.3:0/4191667648 shutdown_connections 2023-10-03T10:19:48.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.203+0000 7f7ccbda3700 1 --2- 172.21.15.3:0/4191667648 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cc406ef70 0x7f7cc412fa30 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.203+0000 7f7ccbda3700 1 --2- 172.21.15.3:0/4191667648 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cc4066410 0x7f7cc406e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.203+0000 7f7ccbda3700 1 -- 172.21.15.3:0/4191667648 >> 172.21.15.3:0/4191667648 conn(0x7f7cc40a2aa0 msgr2=0x7f7cc40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:48.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.203+0000 7f3efa472700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ef406c0e0 0x7f3ef41326b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:48.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.203+0000 7f3efa472700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ef4134ec0 0x7f3ef4132bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:48.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.203+0000 7f3efa472700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ef4137f20 0x7f3ef41334e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:48.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.203+0000 7f3efa472700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef4133130 con 0x7f3ef4134ec0 2023-10-03T10:19:48.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.203+0000 7f3efa472700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef4137a70 con 0x7f3ef4137f20 2023-10-03T10:19:48.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.203+0000 7f3efa472700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef40a03f0 con 0x7f3ef406c0e0 2023-10-03T10:19:48.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.203+0000 7f7ccbda3700 1 -- 172.21.15.3:0/4191667648 shutdown_connections 2023-10-03T10:19:48.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.203+0000 7f3ef8a0f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ef4137f20 0x7f3ef41334e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:48.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.203+0000 7f3ef8a0f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ef4137f20 0x7f3ef41334e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:43368/0 (socket says 172.21.15.3:43368) 2023-10-03T10:19:48.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.203+0000 7f3ef8a0f700 1 -- 172.21.15.3:0/3686667272 learned_addr learned my addr 172.21.15.3:0/3686667272 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:48.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.203+0000 7f3ef3fff700 1 --2- 172.21.15.3:0/3686667272 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ef406c0e0 0x7f3ef41326b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:48.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.203+0000 7f3ef37fe700 1 --2- 172.21.15.3:0/3686667272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ef4134ec0 0x7f3ef4132bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:48.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.203+0000 7f3ef8a0f700 1 -- 172.21.15.3:0/3686667272 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ef406c0e0 msgr2=0x7f3ef41326b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:48.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.203+0000 7f3ef8a0f700 1 --2- 172.21.15.3:0/3686667272 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ef406c0e0 0x7f3ef41326b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.203+0000 7f3ef8a0f700 1 -- 172.21.15.3:0/3686667272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ef4134ec0 msgr2=0x7f3ef4132bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:48.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.203+0000 7f3ef8a0f700 1 --2- 172.21.15.3:0/3686667272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ef4134ec0 0x7f3ef4132bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.203+0000 7f3ef8a0f700 1 -- 172.21.15.3:0/3686667272 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee80094a0 con 0x7f3ef4137f20 2023-10-03T10:19:48.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.204+0000 7f3ef8a0f700 1 --2- 172.21.15.3:0/3686667272 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ef4137f20 0x7f3ef41334e0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00cc70 tx=0x7f3eec00cf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:48.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.204+0000 7f3ef0ff9700 1 -- 172.21.15.3:0/3686667272 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec00aac0 con 0x7f3ef4137f20 2023-10-03T10:19:48.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.204+0000 7f3ef0ff9700 1 -- 172.21.15.3:0/3686667272 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eec00ac40 con 0x7f3ef4137f20 2023-10-03T10:19:48.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.204+0000 7f3efa472700 1 -- 172.21.15.3:0/3686667272 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef411d180 con 0x7f3ef4137f20 2023-10-03T10:19:48.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.204+0000 7f3efa472700 1 -- 172.21.15.3:0/3686667272 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef411bb20 con 0x7f3ef4137f20 2023-10-03T10:19:48.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.204+0000 7f3ef0ff9700 1 -- 172.21.15.3:0/3686667272 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec0070d0 con 0x7f3ef4137f20 2023-10-03T10:19:48.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.203+0000 7f7ccbda3700 1 -- 172.21.15.3:0/4191667648 wait complete. 2023-10-03T10:19:48.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.204+0000 7f7ccbda3700 1 Processor -- start 2023-10-03T10:19:48.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.204+0000 7f7ccbda3700 1 -- start start 2023-10-03T10:19:48.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.205+0000 7f7ccbda3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cc4066410 0x7f7cc41d1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:48.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.205+0000 7f7ccbda3700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cc406eb80 0x7f7cc41d23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:48.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.205+0000 7f7ccbda3700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cc406ef70 0x7f7cc41d4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:48.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.205+0000 7f7ccbda3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc41b86b0 con 0x7f7cc4066410 2023-10-03T10:19:48.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.205+0000 7f7ccbda3700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc41d28e0 con 0x7f7cc406ef70 2023-10-03T10:19:48.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.205+0000 7f7ccbda3700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc412d230 con 0x7f7cc406eb80 2023-10-03T10:19:48.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.205+0000 7f7cca340700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cc406ef70 0x7f7cc41d4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:48.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.205+0000 7f7cca340700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cc406ef70 0x7f7cc41d4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:43384/0 (socket says 172.21.15.3:43384) 2023-10-03T10:19:48.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.205+0000 7f7cca340700 1 -- 172.21.15.3:0/2111870656 learned_addr learned my addr 172.21.15.3:0/2111870656 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:48.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.205+0000 7f7cc933e700 1 --2- 172.21.15.3:0/2111870656 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cc406eb80 0x7f7cc41d23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:48.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.205+0000 7f3efa472700 1 -- 172.21.15.3:0/3686667272 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ed40052e0 con 0x7f3ef4137f20 2023-10-03T10:19:48.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.206+0000 7f3ef0ff9700 1 -- 172.21.15.3:0/3686667272 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3eec007250 con 0x7f3ef4137f20 2023-10-03T10:19:48.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.206+0000 7f7cc9b3f700 1 --2- 172.21.15.3:0/2111870656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cc4066410 0x7f7cc41d1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:48.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.206+0000 7f3ef0ff9700 1 -- 172.21.15.3:0/3686667272 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3eec03f630 con 0x7f3ef4137f20 2023-10-03T10:19:48.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.206+0000 7f7cca340700 1 -- 172.21.15.3:0/2111870656 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cc406eb80 msgr2=0x7f7cc41d23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:48.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.206+0000 7f7cca340700 1 --2- 172.21.15.3:0/2111870656 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cc406eb80 0x7f7cc41d23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.206+0000 7f7cca340700 1 -- 172.21.15.3:0/2111870656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cc4066410 msgr2=0x7f7cc41d1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:48.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.206+0000 7f7cca340700 1 --2- 172.21.15.3:0/2111870656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cc4066410 0x7f7cc41d1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.206+0000 7f7cca340700 1 -- 172.21.15.3:0/2111870656 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc0009550 con 0x7f7cc406ef70 2023-10-03T10:19:48.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.206+0000 7f7cca340700 1 --2- 172.21.15.3:0/2111870656 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cc406ef70 0x7f7cc41d4ea0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4099a40 tx=0x7f7cc0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:48.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.211+0000 7f7cba7fc700 1 -- 172.21.15.3:0/2111870656 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0011830 con 0x7f7cc406ef70 2023-10-03T10:19:48.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.211+0000 7f7cba7fc700 1 -- 172.21.15.3:0/2111870656 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cc0011990 con 0x7f7cc406ef70 2023-10-03T10:19:48.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.211+0000 7f7ccbda3700 1 -- 172.21.15.3:0/2111870656 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cc412d7e0 con 0x7f7cc406ef70 2023-10-03T10:19:48.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.211+0000 7f7cba7fc700 1 -- 172.21.15.3:0/2111870656 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0011d00 con 0x7f7cc406ef70 2023-10-03T10:19:48.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.211+0000 7f7ccbda3700 1 -- 172.21.15.3:0/2111870656 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cc41b88b0 con 0x7f7cc406ef70 2023-10-03T10:19:48.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.211+0000 7f3ef0ff9700 1 -- 172.21.15.3:0/3686667272 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3eec007500 con 0x7f3ef4137f20 2023-10-03T10:19:48.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.212+0000 7f7ccbda3700 1 -- 172.21.15.3:0/2111870656 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cc412fef0 con 0x7f7cc406ef70 2023-10-03T10:19:48.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.212+0000 7f7cba7fc700 1 -- 172.21.15.3:0/2111870656 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7cc001a420 con 0x7f7cc406ef70 2023-10-03T10:19:48.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.212+0000 7f7cba7fc700 1 -- 172.21.15.3:0/2111870656 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f7cc00408f0 con 0x7f7cc406ef70 2023-10-03T10:19:48.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.222+0000 7f7cba7fc700 1 -- 172.21.15.3:0/2111870656 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7cc001a6d0 con 0x7f7cc406ef70 2023-10-03T10:19:48.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.350+0000 7f7dfdb2f700 1 -- 172.21.15.3:0/3503431524 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7df8074ea0 con 0x7f7df8135c20 2023-10-03T10:19:48.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.351+0000 7f7ddffff700 1 -- 172.21.15.3:0/3503431524 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7df0018030 con 0x7f7df8135c20 2023-10-03T10:19:48.351 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:48.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.353+0000 7f7dfdb2f700 1 -- 172.21.15.3:0/3503431524 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7df8135c20 msgr2=0x7f7df809a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:48.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.353+0000 7f7dfdb2f700 1 --2- 172.21.15.3:0/3503431524 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7df8135c20 0x7f7df809a7d0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f7df000efc0 tx=0x7f7df000c3a0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.353+0000 7f7dfdb2f700 1 -- 172.21.15.3:0/3503431524 shutdown_connections 2023-10-03T10:19:48.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.353+0000 7f7dfdb2f700 1 --2- 172.21.15.3:0/3503431524 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7df8135c20 0x7f7df809a7d0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.353+0000 7f7dfdb2f700 1 --2- 172.21.15.3:0/3503431524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df806f5d0 0x7f7df8099d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.353+0000 7f7dfdb2f700 1 --2- 172.21.15.3:0/3503431524 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7df812cf60 0x7f7df809a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.353+0000 7f7dfdb2f700 1 -- 172.21.15.3:0/3503431524 >> 172.21.15.3:0/3503431524 conn(0x7f7df80a2aa0 msgr2=0x7f7df806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:48.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.353+0000 7f7dfdb2f700 1 -- 172.21.15.3:0/3503431524 shutdown_connections 2023-10-03T10:19:48.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.353+0000 7f7dfdb2f700 1 -- 172.21.15.3:0/3503431524 wait complete. 2023-10-03T10:19:48.368 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:48.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.392+0000 7f5ee0982700 1 -- 172.21.15.3:0/1332965309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5edc06b9e0 con 0x7f5edc134df0 2023-10-03T10:19:48.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.393+0000 7f5ecaffd700 1 -- 172.21.15.3:0/1332965309 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5ed402d350 con 0x7f5edc134df0 2023-10-03T10:19:48.393 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.395+0000 7f5ee0982700 1 -- 172.21.15.3:0/1332965309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5edc134df0 msgr2=0x7f5edc139e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.395+0000 7f5ee0982700 1 --2- 172.21.15.3:0/1332965309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5edc134df0 0x7f5edc139e80 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f5edc071cd0 tx=0x7f5ed4004ad0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.395+0000 7f5ee0982700 1 -- 172.21.15.3:0/1332965309 shutdown_connections 2023-10-03T10:19:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.395+0000 7f5ee0982700 1 --2- 172.21.15.3:0/1332965309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5edc134df0 0x7f5edc139e80 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.395+0000 7f5ee0982700 1 --2- 172.21.15.3:0/1332965309 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5edc06e570 0x7f5edc13b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.395+0000 7f5ee0982700 1 --2- 172.21.15.3:0/1332965309 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5edc06ee80 0x7f5edc1348b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.395+0000 7f5ee0982700 1 -- 172.21.15.3:0/1332965309 >> 172.21.15.3:0/1332965309 conn(0x7f5edc0a2aa0 msgr2=0x7f5edc0a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:48.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.396+0000 7f5ee0982700 1 -- 172.21.15.3:0/1332965309 shutdown_connections 2023-10-03T10:19:48.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.396+0000 7f5ee0982700 1 -- 172.21.15.3:0/1332965309 wait complete. 2023-10-03T10:19:48.409 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:48.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.424+0000 7f3efa472700 1 -- 172.21.15.3:0/3686667272 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3ed4005590 con 0x7f3ef4137f20 2023-10-03T10:19:48.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.424+0000 7f3ef0ff9700 1 -- 172.21.15.3:0/3686667272 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3eec01a2c0 con 0x7f3ef4137f20 2023-10-03T10:19:48.425 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.427+0000 7f3efa472700 1 -- 172.21.15.3:0/3686667272 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ef4137f20 msgr2=0x7f3ef41334e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.427+0000 7f3efa472700 1 --2- 172.21.15.3:0/3686667272 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ef4137f20 0x7f3ef41334e0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00cc70 tx=0x7f3eec00cf80 comp rx=0 tx=0).stop 2023-10-03T10:19:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.427+0000 7f3efa472700 1 -- 172.21.15.3:0/3686667272 shutdown_connections 2023-10-03T10:19:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.427+0000 7f3efa472700 1 --2- 172.21.15.3:0/3686667272 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3ef4137f20 0x7f3ef41334e0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.427+0000 7f3efa472700 1 --2- 172.21.15.3:0/3686667272 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3ef406c0e0 0x7f3ef41326b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.427+0000 7f3efa472700 1 --2- 172.21.15.3:0/3686667272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ef4134ec0 0x7f3ef4132bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.427+0000 7f3efa472700 1 -- 172.21.15.3:0/3686667272 >> 172.21.15.3:0/3686667272 conn(0x7f3ef40a4210 msgr2=0x7f3ef40a0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:48.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.427+0000 7f3efa472700 1 -- 172.21.15.3:0/3686667272 shutdown_connections 2023-10-03T10:19:48.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.427+0000 7f3efa472700 1 -- 172.21.15.3:0/3686667272 wait complete. 2023-10-03T10:19:48.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.430+0000 7f7ccbda3700 1 -- 172.21.15.3:0/2111870656 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7cc4074ea0 con 0x7f7cc406ef70 2023-10-03T10:19:48.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.431+0000 7f7cba7fc700 1 -- 172.21.15.3:0/2111870656 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7cc0028020 con 0x7f7cc406ef70 2023-10-03T10:19:48.431 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:48.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.433+0000 7f7ccbda3700 1 -- 172.21.15.3:0/2111870656 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cc406ef70 msgr2=0x7f7cc41d4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:48.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.433+0000 7f7ccbda3700 1 --2- 172.21.15.3:0/2111870656 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cc406ef70 0x7f7cc41d4ea0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4099a40 tx=0x7f7cc0004750 comp rx=0 tx=0).stop 2023-10-03T10:19:48.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.433+0000 7f7ccbda3700 1 -- 172.21.15.3:0/2111870656 shutdown_connections 2023-10-03T10:19:48.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.433+0000 7f7ccbda3700 1 --2- 172.21.15.3:0/2111870656 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cc406ef70 0x7f7cc41d4ea0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.433+0000 7f7ccbda3700 1 --2- 172.21.15.3:0/2111870656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cc4066410 0x7f7cc41d1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.433+0000 7f7ccbda3700 1 --2- 172.21.15.3:0/2111870656 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f7cc406eb80 0x7f7cc41d23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:48.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.433+0000 7f7ccbda3700 1 -- 172.21.15.3:0/2111870656 >> 172.21.15.3:0/2111870656 conn(0x7f7cc40a2aa0 msgr2=0x7f7cc4070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:48.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.433+0000 7f7ccbda3700 1 -- 172.21.15.3:0/2111870656 shutdown_connections 2023-10-03T10:19:48.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:48.433+0000 7f7ccbda3700 1 -- 172.21.15.3:0/2111870656 wait complete. 2023-10-03T10:19:48.442 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:48.447 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:49.369 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:49.410 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-03T10:19:49.442 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:49.448 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-03T10:19:49.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.534+0000 7fc71696b700 1 Processor -- start 2023-10-03T10:19:49.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.535+0000 7fc71696b700 1 -- start start 2023-10-03T10:19:49.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.535+0000 7fc71696b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc71009d6f0 0x7fc71009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:49.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.535+0000 7fc71696b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc71009e000 0x7fc7101339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:49.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.535+0000 7fc71696b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc71009e410 0x7fc710133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:49.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.535+0000 7fc71696b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc71006b460 con 0x7fc71009e000 2023-10-03T10:19:49.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.535+0000 7fc71696b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc71009b120 con 0x7fc71009d6f0 2023-10-03T10:19:49.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.535+0000 7fc71696b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc71009a7e0 con 0x7fc71009e410 2023-10-03T10:19:49.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.536+0000 7fc70f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc71009e000 0x7fc7101339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:49.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.536+0000 7fc70ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc71009d6f0 0x7fc71009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:49.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.536+0000 7fc70f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc71009e000 0x7fc7101339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38828/0 (socket says 172.21.15.3:38828) 2023-10-03T10:19:49.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.536+0000 7fc70f7fe700 1 -- 172.21.15.3:0/2112560656 learned_addr learned my addr 172.21.15.3:0/2112560656 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:49.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.536+0000 7fc714f08700 1 --2- 172.21.15.3:0/2112560656 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc71009e410 0x7fc710133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:49.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.536+0000 7fc70f7fe700 1 -- 172.21.15.3:0/2112560656 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc71009e410 msgr2=0x7fc710133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:49.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.536+0000 7fc70f7fe700 1 --2- 172.21.15.3:0/2112560656 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc71009e410 0x7fc710133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.536+0000 7fc70f7fe700 1 -- 172.21.15.3:0/2112560656 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc71009d6f0 msgr2=0x7fc71009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:49.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.536+0000 7fc70f7fe700 1 --2- 172.21.15.3:0/2112560656 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc71009d6f0 0x7fc71009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.536+0000 7fc70f7fe700 1 -- 172.21.15.3:0/2112560656 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71009a500 con 0x7fc71009e000 2023-10-03T10:19:49.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.536+0000 7fc70ffff700 1 --2- 172.21.15.3:0/2112560656 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc71009d6f0 0x7fc71009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:49.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.536+0000 7fc70f7fe700 1 --2- 172.21.15.3:0/2112560656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc71009e000 0x7fc7101339f0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fc7040098f0 tx=0x7fc704009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7109fd4cc9bb013 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:49.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.537+0000 7fc70effd700 1 -- 172.21.15.3:0/2112560656 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc704011830 con 0x7fc71009e000 2023-10-03T10:19:49.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.537+0000 7fc70effd700 1 -- 172.21.15.3:0/2112560656 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc704011990 con 0x7fc71009e000 2023-10-03T10:19:49.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.537+0000 7fc70effd700 1 -- 172.21.15.3:0/2112560656 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc704011d20 con 0x7fc71009e000 2023-10-03T10:19:49.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.538+0000 7fc71696b700 1 -- 172.21.15.3:0/2112560656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc71009e000 msgr2=0x7fc7101339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:49.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.538+0000 7fc71696b700 1 --2- 172.21.15.3:0/2112560656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc71009e000 0x7fc7101339f0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fc7040098f0 tx=0x7fc704009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:49.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.538+0000 7fc71696b700 1 -- 172.21.15.3:0/2112560656 shutdown_connections 2023-10-03T10:19:49.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.538+0000 7fc71696b700 1 --2- 172.21.15.3:0/2112560656 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc71009e410 0x7fc710133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.538+0000 7fc71696b700 1 --2- 172.21.15.3:0/2112560656 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc71009d6f0 0x7fc71009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.538+0000 7fc71696b700 1 --2- 172.21.15.3:0/2112560656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc71009e000 0x7fc7101339f0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.538+0000 7fc71696b700 1 -- 172.21.15.3:0/2112560656 >> 172.21.15.3:0/2112560656 conn(0x7fc710066f50 msgr2=0x7fc71009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:49.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.538+0000 7fc71696b700 1 -- 172.21.15.3:0/2112560656 shutdown_connections 2023-10-03T10:19:49.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.538+0000 7fc71696b700 1 -- 172.21.15.3:0/2112560656 wait complete. 2023-10-03T10:19:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.539+0000 7fc71696b700 1 Processor -- start 2023-10-03T10:19:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.540+0000 7fc71696b700 1 -- start start 2023-10-03T10:19:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.540+0000 7fc71696b700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc71009d6f0 0x7fc7101a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.540+0000 7fc71696b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc71009e000 0x7fc7101a6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.540+0000 7fc71696b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc71009e410 0x7fc7101a6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.540+0000 7fc71696b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7101a9cf0 con 0x7fc71009e410 2023-10-03T10:19:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.541+0000 7fc71696b700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7100a1fe0 con 0x7fc71009d6f0 2023-10-03T10:19:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.541+0000 7fc71696b700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc71009c710 con 0x7fc71009e000 2023-10-03T10:19:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.541+0000 7fc70ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc71009d6f0 0x7fc7101a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.541+0000 7fc70ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc71009d6f0 0x7fc7101a61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:43404/0 (socket says 172.21.15.3:43404) 2023-10-03T10:19:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.541+0000 7fc714f08700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc71009e410 0x7fc7101a6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.541+0000 7fc70ffff700 1 -- 172.21.15.3:0/1336207521 learned_addr learned my addr 172.21.15.3:0/1336207521 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:49.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.541+0000 7fc70f7fe700 1 --2- 172.21.15.3:0/1336207521 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc71009e000 0x7fc7101a6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:49.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.541+0000 7fc70ffff700 1 -- 172.21.15.3:0/1336207521 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc71009e000 msgr2=0x7fc7101a6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:49.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.541+0000 7fc70ffff700 1 --2- 172.21.15.3:0/1336207521 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc71009e000 0x7fc7101a6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.541+0000 7fc70ffff700 1 -- 172.21.15.3:0/1336207521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc71009e410 msgr2=0x7fc7101a6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:49.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.541+0000 7fc70ffff700 1 --2- 172.21.15.3:0/1336207521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc71009e410 0x7fc7101a6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.541+0000 7fc70ffff700 1 -- 172.21.15.3:0/1336207521 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7000094a0 con 0x7fc71009d6f0 2023-10-03T10:19:49.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.541+0000 7fc70f7fe700 1 --2- 172.21.15.3:0/1336207521 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc71009e000 0x7fc7101a6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-03T10:19:49.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.541+0000 7fc714f08700 1 --2- 172.21.15.3:0/1336207521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc71009e410 0x7fc7101a6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:49.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.541+0000 7fc70ffff700 1 --2- 172.21.15.3:0/1336207521 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc71009d6f0 0x7fc7101a61d0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fc70000ebe0 tx=0x7fc70000eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:49.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.542+0000 7fc70cff9700 1 -- 172.21.15.3:0/1336207521 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc700009800 con 0x7fc71009d6f0 2023-10-03T10:19:49.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.542+0000 7fc70cff9700 1 -- 172.21.15.3:0/1336207521 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70000c910 con 0x7fc71009d6f0 2023-10-03T10:19:49.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.542+0000 7fc70cff9700 1 -- 172.21.15.3:0/1336207521 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70000cc80 con 0x7fc71009d6f0 2023-10-03T10:19:49.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.542+0000 7fc71696b700 1 -- 172.21.15.3:0/1336207521 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc704009550 con 0x7fc71009d6f0 2023-10-03T10:19:49.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.542+0000 7fc71696b700 1 -- 172.21.15.3:0/1336207521 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7101a9480 con 0x7fc71009d6f0 2023-10-03T10:19:49.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.543+0000 7fc70cff9700 1 -- 172.21.15.3:0/1336207521 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc700013470 con 0x7fc71009d6f0 2023-10-03T10:19:49.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.543+0000 7fc71696b700 1 -- 172.21.15.3:0/1336207521 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7101a8390 con 0x7fc71009d6f0 2023-10-03T10:19:49.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.543+0000 7fc70cff9700 1 -- 172.21.15.3:0/1336207521 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fc70003fc90 con 0x7fc71009d6f0 2023-10-03T10:19:49.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.550+0000 7fc70cff9700 1 -- 172.21.15.3:0/1336207521 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc70001b3b0 con 0x7fc71009d6f0 2023-10-03T10:19:49.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.575+0000 7f58df428700 1 Processor -- start 2023-10-03T10:19:49.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.575+0000 7f58df428700 1 -- start start 2023-10-03T10:19:49.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.576+0000 7f58df428700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f58d81338a0 0x7f58d8133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:49.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.576+0000 7f58df428700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f58d809de50 0x7f58d809e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:49.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.576+0000 7f58df428700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58d806ef30 con 0x7f58d809e760 2023-10-03T10:19:49.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.576+0000 7f58df428700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58d806fef0 con 0x7f58d809de50 2023-10-03T10:19:49.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.576+0000 7f58df428700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d81334b0 con 0x7f58d81338a0 2023-10-03T10:19:49.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.576+0000 7f58dc9c3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f58d809de50 0x7f58d809e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:49.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.576+0000 7f58dd1c4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f58d81338a0 0x7f58d8133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:49.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.576+0000 7f58dc9c3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f58d809de50 0x7f58d809e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38840/0 (socket says 172.21.15.3:38840) 2023-10-03T10:19:49.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.576+0000 7f58dc9c3700 1 -- 172.21.15.3:0/3848339684 learned_addr learned my addr 172.21.15.3:0/3848339684 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:49.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.576+0000 7f58dd9c5700 1 --1- 172.21.15.3:0/3848339684 >> v1:172.21.15.3:6789/0 conn(0x7f58d809e760 0x7f58d813a730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60494/0 (socket says 172.21.15.3:60494) 2023-10-03T10:19:49.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.577+0000 7f58dc9c3700 1 -- 172.21.15.3:0/3848339684 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f58d81338a0 msgr2=0x7f58d8133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.577+0000 7f58dc9c3700 1 --2- 172.21.15.3:0/3848339684 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f58d81338a0 0x7f58d8133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.577+0000 7f58dc9c3700 1 -- 172.21.15.3:0/3848339684 >> v1:172.21.15.3:6789/0 conn(0x7f58d809e760 legacy=0x7f58d813a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.577+0000 7f58cffff700 1 -- 172.21.15.3:0/3848339684 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2229053218 0 0) 0x7f58d806ef30 con 0x7f58d809e760 2023-10-03T10:19:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.577+0000 7f58dc9c3700 1 -- 172.21.15.3:0/3848339684 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d806fc30 con 0x7f58d809de50 2023-10-03T10:19:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.577+0000 7f58dc9c3700 1 --2- 172.21.15.3:0/3848339684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58d809de50 0x7f58d809e220 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f58c80098f0 tx=0x7f58c8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6a4ecae70fd9b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.577+0000 7f58cffff700 1 -- 172.21.15.3:0/3848339684 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8011830 con 0x7f58d809de50 2023-10-03T10:19:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.577+0000 7f58cffff700 1 -- 172.21.15.3:0/3848339684 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c8011990 con 0x7f58d809de50 2023-10-03T10:19:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.577+0000 7f58cffff700 1 -- 172.21.15.3:0/3848339684 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8011d20 con 0x7f58d809de50 2023-10-03T10:19:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.578+0000 7f58df428700 1 -- 172.21.15.3:0/3848339684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58d809de50 msgr2=0x7f58d809e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.578+0000 7f58df428700 1 --2- 172.21.15.3:0/3848339684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58d809de50 0x7f58d809e220 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f58c80098f0 tx=0x7f58c8009c00 comp rx=0 tx=0).stop 2023-10-03T10:19:49.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.578+0000 7f58df428700 1 -- 172.21.15.3:0/3848339684 shutdown_connections 2023-10-03T10:19:49.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.578+0000 7f58df428700 1 --2- 172.21.15.3:0/3848339684 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f58d81338a0 0x7f58d8133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.578+0000 7f58df428700 1 --2- 172.21.15.3:0/3848339684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58d809de50 0x7f58d809e220 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.578+0000 7f58df428700 1 -- 172.21.15.3:0/3848339684 >> 172.21.15.3:0/3848339684 conn(0x7f58d80a2aa0 msgr2=0x7f58d80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:49.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.578+0000 7f58df428700 1 -- 172.21.15.3:0/3848339684 shutdown_connections 2023-10-03T10:19:49.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.579+0000 7f58df428700 1 -- 172.21.15.3:0/3848339684 wait complete. 2023-10-03T10:19:49.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.580+0000 7f58df428700 1 Processor -- start 2023-10-03T10:19:49.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.580+0000 7f58df428700 1 -- start start 2023-10-03T10:19:49.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.581+0000 7f58df428700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f58d809de50 0x7f58d811b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:49.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.581+0000 7f58df428700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58d809e760 0x7f58d811bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:49.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.581+0000 7f58df428700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f58d81338a0 0x7f58d811c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:49.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.581+0000 7f58df428700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d811e0a0 con 0x7f58d809e760 2023-10-03T10:19:49.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.581+0000 7f58df428700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d80a0d70 con 0x7f58d809de50 2023-10-03T10:19:49.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.581+0000 7f58df428700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f58d806f640 con 0x7f58d81338a0 2023-10-03T10:19:49.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.581+0000 7f58dd1c4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f58d809de50 0x7f58d811b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:49.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.581+0000 7f58dd1c4700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f58d809de50 0x7f58d811b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:43428/0 (socket says 172.21.15.3:43428) 2023-10-03T10:19:49.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.581+0000 7f58dd1c4700 1 -- 172.21.15.3:0/1168283671 learned_addr learned my addr 172.21.15.3:0/1168283671 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:49.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.581+0000 7f58dc9c3700 1 --2- 172.21.15.3:0/1168283671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58d809e760 0x7f58d811bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:49.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.581+0000 7f58dd9c5700 1 --2- 172.21.15.3:0/1168283671 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f58d81338a0 0x7f58d811c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:49.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.581+0000 7f58dc9c3700 1 -- 172.21.15.3:0/1168283671 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f58d81338a0 msgr2=0x7f58d811c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:49.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.581+0000 7f58dc9c3700 1 --2- 172.21.15.3:0/1168283671 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f58d81338a0 0x7f58d811c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.582+0000 7f58dc9c3700 1 -- 172.21.15.3:0/1168283671 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f58d809de50 msgr2=0x7f58d811b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:49.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.582+0000 7f58dc9c3700 1 --2- 172.21.15.3:0/1168283671 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f58d809de50 0x7f58d811b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.582+0000 7f58dc9c3700 1 -- 172.21.15.3:0/1168283671 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c8009550 con 0x7f58d809e760 2023-10-03T10:19:49.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.582+0000 7f58dd1c4700 1 --2- 172.21.15.3:0/1168283671 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f58d809de50 0x7f58d811b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:49.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.582+0000 7f58dc9c3700 1 --2- 172.21.15.3:0/1168283671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58d809e760 0x7f58d811bd40 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f58c8016000 tx=0x7f58c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:49.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.582+0000 7f58cdffb700 1 -- 172.21.15.3:0/1168283671 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8004450 con 0x7f58d809e760 2023-10-03T10:19:49.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.582+0000 7f58cdffb700 1 -- 172.21.15.3:0/1168283671 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c801e070 con 0x7f58d809e760 2023-10-03T10:19:49.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.582+0000 7f58df428700 1 -- 172.21.15.3:0/1168283671 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d811d2d0 con 0x7f58d809e760 2023-10-03T10:19:49.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.582+0000 7f58cdffb700 1 -- 172.21.15.3:0/1168283671 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c80038e0 con 0x7f58d809e760 2023-10-03T10:19:49.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.582+0000 7f58df428700 1 -- 172.21.15.3:0/1168283671 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d811da20 con 0x7f58d809e760 2023-10-03T10:19:49.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.583+0000 7f58cdffb700 1 -- 172.21.15.3:0/1168283671 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f58c8003a40 con 0x7f58d809e760 2023-10-03T10:19:49.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.584+0000 7f58cdffb700 1 -- 172.21.15.3:0/1168283671 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f58c8040410 con 0x7f58d809e760 2023-10-03T10:19:49.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.584+0000 7f58df428700 1 -- 172.21.15.3:0/1168283671 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d811edd0 con 0x7f58d809e760 2023-10-03T10:19:49.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.593+0000 7f58cdffb700 1 -- 172.21.15.3:0/1168283671 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f58d811edd0 con 0x7f58d809e760 2023-10-03T10:19:49.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.615+0000 7f3dd230f700 1 Processor -- start 2023-10-03T10:19:49.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.615+0000 7f3dd230f700 1 -- start start 2023-10-03T10:19:49.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.616+0000 7f3dd230f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3dcc09e1d0 0x7f3dcc138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:49.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.616+0000 7f3dd230f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3dcc139170 0x7f3dcc13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:49.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.616+0000 7f3dd230f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dcc09b120 con 0x7f3dcc09d6f0 2023-10-03T10:19:49.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.616+0000 7f3dd230f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dcc06b460 con 0x7f3dcc09e1d0 2023-10-03T10:19:49.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.616+0000 7f3dd230f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc09a7e0 con 0x7f3dcc139170 2023-10-03T10:19:49.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.616+0000 7f3dcb7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3dcc09e1d0 0x7f3dcc138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:49.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.616+0000 7f3dcb7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3dcc09e1d0 0x7f3dcc138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38854/0 (socket says 172.21.15.3:38854) 2023-10-03T10:19:49.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.616+0000 7f3dcb7fe700 1 -- 172.21.15.3:0/4289442774 learned_addr learned my addr 172.21.15.3:0/4289442774 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:49.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.616+0000 7f3dd08ac700 1 --2- 172.21.15.3:0/4289442774 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3dcc139170 0x7f3dcc13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:49.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.617+0000 7f3dcb7fe700 1 -- 172.21.15.3:0/4289442774 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3dcc139170 msgr2=0x7f3dcc13b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:49.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.617+0000 7f3dcb7fe700 1 --2- 172.21.15.3:0/4289442774 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3dcc139170 0x7f3dcc13b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.617+0000 7f3dcb7fe700 1 -- 172.21.15.3:0/4289442774 >> v1:172.21.15.3:6789/0 conn(0x7f3dcc09d6f0 legacy=0x7f3dcc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:49.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.617+0000 7f3dcaffd700 1 -- 172.21.15.3:0/4289442774 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 994856735 0 0) 0x7f3dcc09b120 con 0x7f3dcc09d6f0 2023-10-03T10:19:49.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.617+0000 7f3dcb7fe700 1 -- 172.21.15.3:0/4289442774 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dcc09a500 con 0x7f3dcc09e1d0 2023-10-03T10:19:49.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.617+0000 7f3dcb7fe700 1 --2- 172.21.15.3:0/4289442774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3dcc09e1d0 0x7f3dcc138c30 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0009920 tx=0x7f3dc0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36e9544745bf88b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:49.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.617+0000 7f3dcaffd700 1 -- 172.21.15.3:0/4289442774 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0011830 con 0x7f3dcc09e1d0 2023-10-03T10:19:49.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.617+0000 7f3dcaffd700 1 -- 172.21.15.3:0/4289442774 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc0011990 con 0x7f3dcc09e1d0 2023-10-03T10:19:49.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.617+0000 7f3dcaffd700 1 -- 172.21.15.3:0/4289442774 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0011d20 con 0x7f3dcc09e1d0 2023-10-03T10:19:49.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.618+0000 7f3dd230f700 1 -- 172.21.15.3:0/4289442774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3dcc09e1d0 msgr2=0x7f3dcc138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:49.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.618+0000 7f3dd230f700 1 --2- 172.21.15.3:0/4289442774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3dcc09e1d0 0x7f3dcc138c30 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0009920 tx=0x7f3dc0009c30 comp rx=0 tx=0).stop 2023-10-03T10:19:49.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.618+0000 7f3dd230f700 1 -- 172.21.15.3:0/4289442774 shutdown_connections 2023-10-03T10:19:49.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.618+0000 7f3dd230f700 1 --2- 172.21.15.3:0/4289442774 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3dcc139170 0x7f3dcc13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.618+0000 7f3dd230f700 1 --2- 172.21.15.3:0/4289442774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3dcc09e1d0 0x7f3dcc138c30 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.618+0000 7f3dd230f700 1 -- 172.21.15.3:0/4289442774 >> 172.21.15.3:0/4289442774 conn(0x7f3dcc066f50 msgr2=0x7f3dcc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:49.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.618+0000 7f3dd230f700 1 -- 172.21.15.3:0/4289442774 shutdown_connections 2023-10-03T10:19:49.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.618+0000 7f3dd230f700 1 -- 172.21.15.3:0/4289442774 wait complete. 2023-10-03T10:19:49.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.619+0000 7f7107c9f700 1 Processor -- start 2023-10-03T10:19:49.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.619+0000 7f3dd230f700 1 Processor -- start 2023-10-03T10:19:49.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.619+0000 7f7107c9f700 1 -- start start 2023-10-03T10:19:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.619+0000 7f3dd230f700 1 -- start start 2023-10-03T10:19:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.620+0000 7f3dd230f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3dcc09d6f0 0x7f3dcc11e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.620+0000 7f3dd230f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3dcc09e1d0 0x7f3dcc11ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.620+0000 7f3dd230f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3dcc139170 0x7f3dcc11f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.620+0000 7f3dd230f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc1206a0 con 0x7f3dcc139170 2023-10-03T10:19:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.620+0000 7f3dd230f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc09c710 con 0x7f3dcc09e1d0 2023-10-03T10:19:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.620+0000 7f3dd230f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc09a0c0 con 0x7f3dcc09d6f0 2023-10-03T10:19:49.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.620+0000 7f7107c9f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f710009d6f0 0x7f710009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:49.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.620+0000 7f7107c9f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f710009e410 0x7f7100130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:49.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.620+0000 7f7107c9f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f710009f130 con 0x7f710009e000 2023-10-03T10:19:49.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.620+0000 7f7107c9f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f710009f520 con 0x7f710009e410 2023-10-03T10:19:49.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.620+0000 7f7107c9f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f710009e930 con 0x7f710009d6f0 2023-10-03T10:19:49.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.620+0000 7f3dd08ac700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3dcc139170 0x7f3dcc11f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:49.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.620+0000 7f3dd08ac700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3dcc139170 0x7f3dcc11f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38870/0 (socket says 172.21.15.3:38870) 2023-10-03T10:19:49.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.620+0000 7f3dd08ac700 1 -- 172.21.15.3:0/564800677 learned_addr learned my addr 172.21.15.3:0/564800677 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:49.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.620+0000 7f7105a3b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f710009d6f0 0x7f710009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:49.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.620+0000 7f7105a3b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f710009d6f0 0x7f710009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:44814/0 (socket says 172.21.15.3:44814) 2023-10-03T10:19:49.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.620+0000 7f7105a3b700 1 -- 172.21.15.3:0/2962006955 learned_addr learned my addr 172.21.15.3:0/2962006955 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:49.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.620+0000 7f710623c700 1 --2- 172.21.15.3:0/2962006955 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f710009e410 0x7f7100130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:49.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.620+0000 7f3dcb7fe700 1 --2- 172.21.15.3:0/564800677 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3dcc09e1d0 0x7f3dcc11ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:49.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.621+0000 7f3dd08ac700 1 -- 172.21.15.3:0/564800677 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3dcc09d6f0 msgr2=0x7f3dcc11e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:49.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.621+0000 7f3dd08ac700 1 --2- 172.21.15.3:0/564800677 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3dcc09d6f0 0x7f3dcc11e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.621+0000 7f3dd08ac700 1 -- 172.21.15.3:0/564800677 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3dcc09e1d0 msgr2=0x7f3dcc11ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:49.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.621+0000 7f3dd08ac700 1 --2- 172.21.15.3:0/564800677 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3dcc09e1d0 0x7f3dcc11ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.621+0000 7f3dd08ac700 1 -- 172.21.15.3:0/564800677 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc0009550 con 0x7f3dcc139170 2023-10-03T10:19:49.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.621+0000 7f7105a3b700 1 -- 172.21.15.3:0/2962006955 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f710009e410 msgr2=0x7f7100130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:49.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.621+0000 7f7105a3b700 1 --2- 172.21.15.3:0/2962006955 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f710009e410 0x7f7100130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.621+0000 7f7105a3b700 1 -- 172.21.15.3:0/2962006955 >> v1:172.21.15.3:6789/0 conn(0x7f710009e000 legacy=0x7f7100130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:49.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.621+0000 7f7105a3b700 1 -- 172.21.15.3:0/2962006955 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f710009b2d0 con 0x7f710009d6f0 2023-10-03T10:19:49.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.621+0000 7f3dd08ac700 1 --2- 172.21.15.3:0/564800677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3dcc139170 0x7f3dcc11f460 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400efc0 tx=0x7f3dc400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:49.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.621+0000 7f7105a3b700 1 --2- 172.21.15.3:0/2962006955 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f710009d6f0 0x7f710009dac0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009920 tx=0x7f70fc009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b1d19475a5994b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:49.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.621+0000 7f3dc8ff9700 1 -- 172.21.15.3:0/564800677 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc400cd40 con 0x7f3dcc139170 2023-10-03T10:19:49.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.621+0000 7f3dc8ff9700 1 -- 172.21.15.3:0/564800677 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc400f040 con 0x7f3dcc139170 2023-10-03T10:19:49.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.621+0000 7f3dd230f700 1 -- 172.21.15.3:0/564800677 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dcc121bc0 con 0x7f3dcc139170 2023-10-03T10:19:49.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.621+0000 7f7104a39700 1 -- 172.21.15.3:0/2962006955 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc004230 con 0x7f710009d6f0 2023-10-03T10:19:49.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.621+0000 7f7104a39700 1 -- 172.21.15.3:0/2962006955 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70fc004390 con 0x7f710009d6f0 2023-10-03T10:19:49.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.621+0000 7f7104a39700 1 -- 172.21.15.3:0/2962006955 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc009e70 con 0x7f710009d6f0 2023-10-03T10:19:49.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.621+0000 7f3dc8ff9700 1 -- 172.21.15.3:0/564800677 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc40146c0 con 0x7f3dcc139170 2023-10-03T10:19:49.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.621+0000 7f3dd230f700 1 -- 172.21.15.3:0/564800677 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dcc120cf0 con 0x7f3dcc139170 2023-10-03T10:19:49.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.622+0000 7f3dc8ff9700 1 -- 172.21.15.3:0/564800677 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3dc4014820 con 0x7f3dcc139170 2023-10-03T10:19:49.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.622+0000 7f3dd230f700 1 -- 172.21.15.3:0/564800677 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dcc074ea0 con 0x7f3dcc139170 2023-10-03T10:19:49.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.623+0000 7f7107c9f700 1 -- 172.21.15.3:0/2962006955 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f710009d6f0 msgr2=0x7f710009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:49.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.623+0000 7f3dc8ff9700 1 -- 172.21.15.3:0/564800677 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3dc4040350 con 0x7f3dcc139170 2023-10-03T10:19:49.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.623+0000 7f7107c9f700 1 --2- 172.21.15.3:0/2962006955 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f710009d6f0 0x7f710009dac0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009920 tx=0x7f70fc009c30 comp rx=0 tx=0).stop 2023-10-03T10:19:49.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.623+0000 7f7107c9f700 1 -- 172.21.15.3:0/2962006955 shutdown_connections 2023-10-03T10:19:49.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.623+0000 7f7107c9f700 1 --2- 172.21.15.3:0/2962006955 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f710009e410 0x7f7100130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.623+0000 7f7107c9f700 1 --2- 172.21.15.3:0/2962006955 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f710009d6f0 0x7f710009dac0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.623+0000 7f7107c9f700 1 -- 172.21.15.3:0/2962006955 >> 172.21.15.3:0/2962006955 conn(0x7f7100066f50 msgr2=0x7f710009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:49.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.623+0000 7f7107c9f700 1 -- 172.21.15.3:0/2962006955 shutdown_connections 2023-10-03T10:19:49.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.623+0000 7f7107c9f700 1 -- 172.21.15.3:0/2962006955 wait complete. 2023-10-03T10:19:49.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.625+0000 7f7107c9f700 1 Processor -- start 2023-10-03T10:19:49.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.625+0000 7f7107c9f700 1 -- start start 2023-10-03T10:19:49.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.625+0000 7f7107c9f700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f710009d6f0 0x7f710011dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:49.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.625+0000 7f7107c9f700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f710009e000 0x7f710011e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:49.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.625+0000 7f7107c9f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f710009e410 0x7f710011ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:49.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.625+0000 7f7107c9f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7100121810 con 0x7f710009e410 2023-10-03T10:19:49.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.625+0000 7f7107c9f700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7100120200 con 0x7f710009e000 2023-10-03T10:19:49.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.625+0000 7f7107c9f700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7100121280 con 0x7f710009d6f0 2023-10-03T10:19:49.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.626+0000 7f7105a3b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f710009d6f0 0x7f710011dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:49.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.626+0000 7f7105a3b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f710009d6f0 0x7f710011dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:44816/0 (socket says 172.21.15.3:44816) 2023-10-03T10:19:49.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.626+0000 7f7105a3b700 1 -- 172.21.15.3:0/4291165768 learned_addr learned my addr 172.21.15.3:0/4291165768 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:49.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.626+0000 7f710623c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f710009e410 0x7f710011ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:49.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.626+0000 7f710523a700 1 --2- 172.21.15.3:0/4291165768 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f710009e000 0x7f710011e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:49.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.626+0000 7f710523a700 1 -- 172.21.15.3:0/4291165768 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f710009d6f0 msgr2=0x7f710011dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:49.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.626+0000 7f710523a700 1 --2- 172.21.15.3:0/4291165768 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f710009d6f0 0x7f710011dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.626+0000 7f710523a700 1 -- 172.21.15.3:0/4291165768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f710009e410 msgr2=0x7f710011ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:49.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.626+0000 7f710523a700 1 --2- 172.21.15.3:0/4291165768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f710009e410 0x7f710011ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.626+0000 7f710523a700 1 -- 172.21.15.3:0/4291165768 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70fc009550 con 0x7f710009e000 2023-10-03T10:19:49.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.626+0000 7f7105a3b700 1 --2- 172.21.15.3:0/4291165768 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f710009d6f0 0x7f710011dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:49.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.626+0000 7f710523a700 1 --2- 172.21.15.3:0/4291165768 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f710009e000 0x7f710011e510 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f71000a3e20 tx=0x7f70f000c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:49.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.627+0000 7f70f67fc700 1 -- 172.21.15.3:0/4291165768 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f0012620 con 0x7f710009e000 2023-10-03T10:19:49.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.627+0000 7f70f67fc700 1 -- 172.21.15.3:0/4291165768 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f0012780 con 0x7f710009e000 2023-10-03T10:19:49.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.627+0000 7f70f67fc700 1 -- 172.21.15.3:0/4291165768 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f0012b30 con 0x7f710009e000 2023-10-03T10:19:49.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.627+0000 7f7107c9f700 1 -- 172.21.15.3:0/4291165768 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f710009c700 con 0x7f710009e000 2023-10-03T10:19:49.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.627+0000 7f7107c9f700 1 -- 172.21.15.3:0/4291165768 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71000a1d60 con 0x7f710009e000 2023-10-03T10:19:49.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.628+0000 7f70f67fc700 1 -- 172.21.15.3:0/4291165768 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f70f001c460 con 0x7f710009e000 2023-10-03T10:19:49.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.628+0000 7f7107c9f700 1 -- 172.21.15.3:0/4291165768 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7100074ea0 con 0x7f710009e000 2023-10-03T10:19:49.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.628+0000 7f70f67fc700 1 -- 172.21.15.3:0/4291165768 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f70f003f480 con 0x7f710009e000 2023-10-03T10:19:49.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.629+0000 7f3dc8ff9700 1 -- 172.21.15.3:0/564800677 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3dc4014ad0 con 0x7f3dcc139170 2023-10-03T10:19:49.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.633+0000 7f70f67fc700 1 -- 172.21.15.3:0/4291165768 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f70f00106f0 con 0x7f710009e000 2023-10-03T10:19:49.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.773+0000 7fc71696b700 1 -- 172.21.15.3:0/1336207521 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc710074ea0 con 0x7fc71009d6f0 2023-10-03T10:19:49.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.774+0000 7fc70cff9700 1 -- 172.21.15.3:0/1336207521 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc700011350 con 0x7fc71009d6f0 2023-10-03T10:19:49.773 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:49.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.776+0000 7fc71696b700 1 -- 172.21.15.3:0/1336207521 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc71009d6f0 msgr2=0x7fc7101a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:49.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.776+0000 7fc71696b700 1 --2- 172.21.15.3:0/1336207521 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc71009d6f0 0x7fc7101a61d0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fc70000ebe0 tx=0x7fc70000eef0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.776+0000 7fc71696b700 1 -- 172.21.15.3:0/1336207521 shutdown_connections 2023-10-03T10:19:49.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.776+0000 7fc71696b700 1 --2- 172.21.15.3:0/1336207521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc71009e410 0x7fc7101a6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.776+0000 7fc71696b700 1 --2- 172.21.15.3:0/1336207521 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc71009d6f0 0x7fc7101a61d0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.776+0000 7fc71696b700 1 --2- 172.21.15.3:0/1336207521 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fc71009e000 0x7fc7101a6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.776+0000 7fc71696b700 1 -- 172.21.15.3:0/1336207521 >> 172.21.15.3:0/1336207521 conn(0x7fc710066f50 msgr2=0x7fc7101bac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:49.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.776+0000 7fc71696b700 1 -- 172.21.15.3:0/1336207521 shutdown_connections 2023-10-03T10:19:49.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.776+0000 7fc71696b700 1 -- 172.21.15.3:0/1336207521 wait complete. 2023-10-03T10:19:49.789 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:49.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.807+0000 7f58df428700 1 -- 172.21.15.3:0/1168283671 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f58d811edd0 con 0x7f58d809e760 2023-10-03T10:19:49.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.808+0000 7f58cdffb700 1 -- 172.21.15.3:0/1168283671 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f58d811edd0 con 0x7f58d809e760 2023-10-03T10:19:49.807 INFO:teuthology.orchestra.run.smithi003.stdout:124554051587 2023-10-03T10:19:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.810+0000 7f58df428700 1 -- 172.21.15.3:0/1168283671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58d809e760 msgr2=0x7f58d811bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.810+0000 7f58df428700 1 --2- 172.21.15.3:0/1168283671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58d809e760 0x7f58d811bd40 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f58c8016000 tx=0x7f58c8004750 comp rx=0 tx=0).stop 2023-10-03T10:19:49.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.810+0000 7f58df428700 1 -- 172.21.15.3:0/1168283671 shutdown_connections 2023-10-03T10:19:49.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.810+0000 7f58df428700 1 --2- 172.21.15.3:0/1168283671 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f58d81338a0 0x7f58d811c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.810+0000 7f58df428700 1 --2- 172.21.15.3:0/1168283671 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f58d809de50 0x7f58d811b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.810+0000 7f58df428700 1 --2- 172.21.15.3:0/1168283671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58d809e760 0x7f58d811bd40 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.810+0000 7f58df428700 1 -- 172.21.15.3:0/1168283671 >> 172.21.15.3:0/1168283671 conn(0x7f58d80a2aa0 msgr2=0x7f58d8130630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:49.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.810+0000 7f58df428700 1 -- 172.21.15.3:0/1168283671 shutdown_connections 2023-10-03T10:19:49.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.810+0000 7f58df428700 1 -- 172.21.15.3:0/1168283671 wait complete. 2023-10-03T10:19:49.823 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-10-03T10:19:49.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.837+0000 7f3dd230f700 1 -- 172.21.15.3:0/564800677 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3dcc09f420 con 0x7f3dcc139170 2023-10-03T10:19:49.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.838+0000 7f3dc8ff9700 1 -- 172.21.15.3:0/564800677 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3dc4018020 con 0x7f3dcc139170 2023-10-03T10:19:49.837 INFO:teuthology.orchestra.run.smithi003.stdout:137438953474 2023-10-03T10:19:49.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.840+0000 7f3dd230f700 1 -- 172.21.15.3:0/564800677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3dcc139170 msgr2=0x7f3dcc11f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:49.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.840+0000 7f3dd230f700 1 --2- 172.21.15.3:0/564800677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3dcc139170 0x7f3dcc11f460 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400efc0 tx=0x7f3dc400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.840+0000 7f3dd230f700 1 -- 172.21.15.3:0/564800677 shutdown_connections 2023-10-03T10:19:49.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.840+0000 7f3dd230f700 1 --2- 172.21.15.3:0/564800677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3dcc139170 0x7f3dcc11f460 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.840+0000 7f3dd230f700 1 --2- 172.21.15.3:0/564800677 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f3dcc09d6f0 0x7f3dcc11e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.840+0000 7f3dd230f700 1 --2- 172.21.15.3:0/564800677 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3dcc09e1d0 0x7f3dcc11ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.840+0000 7f3dd230f700 1 -- 172.21.15.3:0/564800677 >> 172.21.15.3:0/564800677 conn(0x7f3dcc066f50 msgr2=0x7f3dcc138070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:49.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.840+0000 7f3dd230f700 1 -- 172.21.15.3:0/564800677 shutdown_connections 2023-10-03T10:19:49.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.841+0000 7f3dd230f700 1 -- 172.21.15.3:0/564800677 wait complete. 2023-10-03T10:19:49.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.843+0000 7f7107c9f700 1 -- 172.21.15.3:0/4291165768 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f710006c360 con 0x7f710009e000 2023-10-03T10:19:49.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.844+0000 7f70f67fc700 1 -- 172.21.15.3:0/4291165768 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f70f001a360 con 0x7f710009e000 2023-10-03T10:19:49.843 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:49.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.846+0000 7f7107c9f700 1 -- 172.21.15.3:0/4291165768 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f710009e000 msgr2=0x7f710011e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:49.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.846+0000 7f7107c9f700 1 --2- 172.21.15.3:0/4291165768 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f710009e000 0x7f710011e510 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f71000a3e20 tx=0x7f70f000c6e0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.846+0000 7f7107c9f700 1 -- 172.21.15.3:0/4291165768 shutdown_connections 2023-10-03T10:19:49.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.846+0000 7f7107c9f700 1 --2- 172.21.15.3:0/4291165768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f710009e410 0x7f710011ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.846+0000 7f7107c9f700 1 --2- 172.21.15.3:0/4291165768 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f710009d6f0 0x7f710011dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.846+0000 7f7107c9f700 1 --2- 172.21.15.3:0/4291165768 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f710009e000 0x7f710011e510 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:49.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.846+0000 7f7107c9f700 1 -- 172.21.15.3:0/4291165768 >> 172.21.15.3:0/4291165768 conn(0x7f7100066f50 msgr2=0x7f710006f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:49.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.846+0000 7f7107c9f700 1 -- 172.21.15.3:0/4291165768 shutdown_connections 2023-10-03T10:19:49.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:49.846+0000 7f7107c9f700 1 -- 172.21.15.3:0/4291165768 wait complete. 2023-10-03T10:19:49.854 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-10-03T10:19:49.859 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:19:50.790 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-03T10:19:50.824 INFO:tasks.ceph:ignoring flush pg stats error, probably testing upgrade: timed out waiting for mon to be updated with osd.0: 124554051587 < 124554051588 2023-10-03T10:19:50.825 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-03T10:19:50.825 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-03T10:19:50.860 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-03T10:19:50.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.940+0000 7f5ceef0d700 1 Processor -- start 2023-10-03T10:19:50.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.940+0000 7f5ceef0d700 1 -- start start 2023-10-03T10:19:50.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.941+0000 7f5ceef0d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ce809d6f0 0x7f5ce809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:50.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.941+0000 7f5ceef0d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ce809e410 0x7f5ce8134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:50.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.941+0000 7f5ceef0d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce809b120 con 0x7f5ce809e000 2023-10-03T10:19:50.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.941+0000 7f5ceef0d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce806b460 con 0x7f5ce809e410 2023-10-03T10:19:50.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.941+0000 7f5ceef0d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce809a7e0 con 0x7f5ce809d6f0 2023-10-03T10:19:50.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.941+0000 7f5cecca9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ce809d6f0 0x7f5ce809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:50.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.941+0000 7f5ced4aa700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ce809e410 0x7f5ce8134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:50.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.941+0000 7f5ce7fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5ce809e000 0x7f5ce81339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60520/0 (socket says 172.21.15.3:60520) 2023-10-03T10:19:50.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.941+0000 7f5ce7fff700 1 -- 172.21.15.3:0/1713619633 learned_addr learned my addr 172.21.15.3:0/1713619633 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:50.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.941+0000 7f5ced4aa700 1 -- 172.21.15.3:0/1713619633 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ce809d6f0 msgr2=0x7f5ce809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:50.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.941+0000 7f5ce77fe700 1 -- 172.21.15.3:0/1713619633 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1502864234 0 0) 0x7f5ce809b120 con 0x7f5ce809e000 2023-10-03T10:19:50.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.941+0000 7f5ced4aa700 1 --2- 172.21.15.3:0/1713619633 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ce809d6f0 0x7f5ce809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:50.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.941+0000 7f5ced4aa700 1 -- 172.21.15.3:0/1713619633 >> v1:172.21.15.3:6789/0 conn(0x7f5ce809e000 legacy=0x7f5ce81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:50.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.941+0000 7f5ced4aa700 1 -- 172.21.15.3:0/1713619633 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce809a500 con 0x7f5ce809e410 2023-10-03T10:19:50.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.941+0000 7f5cecca9700 1 --2- 172.21.15.3:0/1713619633 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ce809d6f0 0x7f5ce809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:50.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.942+0000 7f5ced4aa700 1 --2- 172.21.15.3:0/1713619633 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ce809e410 0x7f5ce8134100 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009dc0 tx=0x7f5ce000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35e665b77aa70821 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:50.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.942+0000 7f5ce77fe700 1 -- 172.21.15.3:0/1713619633 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce000e070 con 0x7f5ce809e410 2023-10-03T10:19:50.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.942+0000 7f5ce77fe700 1 -- 172.21.15.3:0/1713619633 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce000bb20 con 0x7f5ce809e410 2023-10-03T10:19:50.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.942+0000 7f5ce77fe700 1 -- 172.21.15.3:0/1713619633 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0012430 con 0x7f5ce809e410 2023-10-03T10:19:50.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.943+0000 7f5ceef0d700 1 -- 172.21.15.3:0/1713619633 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ce809e410 msgr2=0x7f5ce8134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:50.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.943+0000 7f5ceef0d700 1 --2- 172.21.15.3:0/1713619633 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ce809e410 0x7f5ce8134100 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009dc0 tx=0x7f5ce000b330 comp rx=0 tx=0).stop 2023-10-03T10:19:50.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.943+0000 7f5ceef0d700 1 -- 172.21.15.3:0/1713619633 shutdown_connections 2023-10-03T10:19:50.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.943+0000 7f5ceef0d700 1 --2- 172.21.15.3:0/1713619633 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ce809e410 0x7f5ce8134100 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:50.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.943+0000 7f5ceef0d700 1 --2- 172.21.15.3:0/1713619633 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ce809d6f0 0x7f5ce809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:50.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.943+0000 7f5ceef0d700 1 -- 172.21.15.3:0/1713619633 >> 172.21.15.3:0/1713619633 conn(0x7f5ce8066f50 msgr2=0x7f5ce809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:50.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.943+0000 7f5ceef0d700 1 -- 172.21.15.3:0/1713619633 shutdown_connections 2023-10-03T10:19:50.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.943+0000 7f5ceef0d700 1 -- 172.21.15.3:0/1713619633 wait complete. 2023-10-03T10:19:50.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.944+0000 7f5ceef0d700 1 Processor -- start 2023-10-03T10:19:50.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.944+0000 7f5ceef0d700 1 -- start start 2023-10-03T10:19:50.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.945+0000 7f5ceef0d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ce809d6f0 0x7f5ce81c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:50.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.945+0000 7f5ceef0d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ce809e000 0x7f5ce81c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:50.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.945+0000 7f5ceef0d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ce809e410 0x7f5ce81c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:50.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.945+0000 7f5ceef0d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce81a6f20 con 0x7f5ce809e000 2023-10-03T10:19:50.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.945+0000 7f5ceef0d700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce80a1d00 con 0x7f5ce809d6f0 2023-10-03T10:19:50.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.945+0000 7f5ceef0d700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce81a6350 con 0x7f5ce809e410 2023-10-03T10:19:50.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.945+0000 7f5cecca9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ce809d6f0 0x7f5ce81c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:50.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.945+0000 7f5cecca9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ce809d6f0 0x7f5ce81c7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:43458/0 (socket says 172.21.15.3:43458) 2023-10-03T10:19:50.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.945+0000 7f5cecca9700 1 -- 172.21.15.3:0/3486358828 learned_addr learned my addr 172.21.15.3:0/3486358828 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:50.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.945+0000 7f5ce7fff700 1 --2- 172.21.15.3:0/3486358828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ce809e000 0x7f5ce81c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:50.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.945+0000 7f5ced4aa700 1 --2- 172.21.15.3:0/3486358828 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ce809e410 0x7f5ce81c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:50.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.946+0000 7f5cecca9700 1 -- 172.21.15.3:0/3486358828 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ce809e410 msgr2=0x7f5ce81c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:50.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.946+0000 7f5cecca9700 1 --2- 172.21.15.3:0/3486358828 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ce809e410 0x7f5ce81c89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:50.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.946+0000 7f5cecca9700 1 -- 172.21.15.3:0/3486358828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ce809e000 msgr2=0x7f5ce81c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:50.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.946+0000 7f5cecca9700 1 --2- 172.21.15.3:0/3486358828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ce809e000 0x7f5ce81c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:50.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.946+0000 7f5cecca9700 1 -- 172.21.15.3:0/3486358828 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cdc0094a0 con 0x7f5ce809d6f0 2023-10-03T10:19:50.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.946+0000 7f5cecca9700 1 --2- 172.21.15.3:0/3486358828 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ce809d6f0 0x7f5ce81c7f40 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00ebe0 tx=0x7f5cdc00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:50.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.946+0000 7f5ce57fa700 1 -- 172.21.15.3:0/3486358828 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc00c850 con 0x7f5ce809d6f0 2023-10-03T10:19:50.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.946+0000 7f5ce57fa700 1 -- 172.21.15.3:0/3486358828 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cdc00c9b0 con 0x7f5ce809d6f0 2023-10-03T10:19:50.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.946+0000 7f5ceef0d700 1 -- 172.21.15.3:0/3486358828 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ce0009a70 con 0x7f5ce809d6f0 2023-10-03T10:19:50.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.946+0000 7f5ce57fa700 1 -- 172.21.15.3:0/3486358828 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc00cd40 con 0x7f5ce809d6f0 2023-10-03T10:19:50.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.946+0000 7f5ceef0d700 1 -- 172.21.15.3:0/3486358828 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ce81a6a30 con 0x7f5ce809d6f0 2023-10-03T10:19:50.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.947+0000 7f5ce57fa700 1 -- 172.21.15.3:0/3486358828 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5cdc013470 con 0x7f5ce809d6f0 2023-10-03T10:19:50.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.947+0000 7f5ceef0d700 1 -- 172.21.15.3:0/3486358828 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ce8074ea0 con 0x7f5ce809d6f0 2023-10-03T10:19:50.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.948+0000 7f5ce57fa700 1 -- 172.21.15.3:0/3486358828 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5cdc03fb50 con 0x7f5ce809d6f0 2023-10-03T10:19:50.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.952+0000 7f5ce57fa700 1 -- 172.21.15.3:0/3486358828 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5cdc013720 con 0x7f5ce809d6f0 2023-10-03T10:19:50.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.978+0000 7f0b14206700 1 Processor -- start 2023-10-03T10:19:50.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.978+0000 7f0b14206700 1 -- start start 2023-10-03T10:19:50.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.979+0000 7f0b14206700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b0c0a0a30 0x7f0b0c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:50.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.979+0000 7f0b14206700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b0c138d80 0x7f0b0c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:50.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.979+0000 7f0b14206700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b0c0a2a70 con 0x7f0b0c09d630 2023-10-03T10:19:50.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.979+0000 7f0b14206700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c0a01d0 con 0x7f0b0c0a0a30 2023-10-03T10:19:50.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.979+0000 7f0b14206700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c09e2b0 con 0x7f0b0c138d80 2023-10-03T10:19:50.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.979+0000 7f0b117a1700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0b0c09d630 0x7f0b0c0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60530/0 (socket says 172.21.15.3:60530) 2023-10-03T10:19:50.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.979+0000 7f0b11fa2700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b0c0a0a30 0x7f0b0c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:50.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.979+0000 7f0b117a1700 1 -- 172.21.15.3:0/1863075453 learned_addr learned my addr 172.21.15.3:0/1863075453 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:19:50.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.979+0000 7f0b127a3700 1 --2- 172.21.15.3:0/1863075453 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b0c138d80 0x7f0b0c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:50.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.979+0000 7f0b10fa0700 1 -- 172.21.15.3:0/1863075453 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1652542688 0 0) 0x7f0b0c0a2a70 con 0x7f0b0c09d630 2023-10-03T10:19:50.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.979+0000 7f0b10fa0700 1 -- 172.21.15.3:0/1863075453 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0af80035b0 con 0x7f0b0c09d630 2023-10-03T10:19:50.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.979+0000 7f0b11fa2700 1 -- 172.21.15.3:0/1863075453 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b0c138d80 msgr2=0x7f0b0c13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:50.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.979+0000 7f0b11fa2700 1 --2- 172.21.15.3:0/1863075453 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b0c138d80 0x7f0b0c13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:50.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.979+0000 7f0b11fa2700 1 -- 172.21.15.3:0/1863075453 >> v1:172.21.15.3:6789/0 conn(0x7f0b0c09d630 legacy=0x7f0b0c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:50.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.979+0000 7f0b11fa2700 1 -- 172.21.15.3:0/1863075453 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0c09dc50 con 0x7f0b0c0a0a30 2023-10-03T10:19:50.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.979+0000 7f0b127a3700 1 --2- 172.21.15.3:0/1863075453 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b0c138d80 0x7f0b0c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:50.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.980+0000 7f0b11fa2700 1 --2- 172.21.15.3:0/1863075453 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b0c0a0a30 0x7f0b0c0a0e00 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009870 tx=0x7f0b08009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecaa98b23576fe59 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:50.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.980+0000 7f0b10fa0700 1 -- 172.21.15.3:0/1863075453 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b08011830 con 0x7f0b0c0a0a30 2023-10-03T10:19:50.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.980+0000 7f0b10fa0700 1 -- 172.21.15.3:0/1863075453 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b08011990 con 0x7f0b0c0a0a30 2023-10-03T10:19:50.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.980+0000 7f0b10fa0700 1 -- 172.21.15.3:0/1863075453 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b08011d20 con 0x7f0b0c0a0a30 2023-10-03T10:19:50.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.980+0000 7f0b14206700 1 -- 172.21.15.3:0/1863075453 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b0c0a0a30 msgr2=0x7f0b0c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:50.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.980+0000 7f0b14206700 1 --2- 172.21.15.3:0/1863075453 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b0c0a0a30 0x7f0b0c0a0e00 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009870 tx=0x7f0b08009b80 comp rx=0 tx=0).stop 2023-10-03T10:19:50.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.981+0000 7f0b14206700 1 -- 172.21.15.3:0/1863075453 shutdown_connections 2023-10-03T10:19:50.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.981+0000 7f0b14206700 1 --2- 172.21.15.3:0/1863075453 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b0c0a0a30 0x7f0b0c0a0e00 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:50.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.981+0000 7f0b14206700 1 --2- 172.21.15.3:0/1863075453 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b0c138d80 0x7f0b0c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:50.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.981+0000 7f0b14206700 1 -- 172.21.15.3:0/1863075453 >> 172.21.15.3:0/1863075453 conn(0x7f0b0c0a3050 msgr2=0x7f0b0c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:50.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.981+0000 7f0b14206700 1 -- 172.21.15.3:0/1863075453 shutdown_connections 2023-10-03T10:19:50.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.981+0000 7f0b14206700 1 -- 172.21.15.3:0/1863075453 wait complete. 2023-10-03T10:19:50.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.982+0000 7f0b14206700 1 Processor -- start 2023-10-03T10:19:50.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.982+0000 7f0b14206700 1 -- start start 2023-10-03T10:19:50.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.982+0000 7f0b14206700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b0c09d630 0x7f0b0c133760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:50.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.982+0000 7f0b14206700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b0c0a0a30 0x7f0b0c133ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:50.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.982+0000 7f0b14206700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b0c138d80 0x7f0b0c1341e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:50.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.982+0000 7f0b14206700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c1365a0 con 0x7f0b0c138d80 2023-10-03T10:19:50.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.982+0000 7f0b14206700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c138b10 con 0x7f0b0c09d630 2023-10-03T10:19:50.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.982+0000 7f0b14206700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c136f10 con 0x7f0b0c0a0a30 2023-10-03T10:19:50.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.982+0000 7f0b117a1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b0c0a0a30 0x7f0b0c133ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:50.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.982+0000 7f0b117a1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b0c0a0a30 0x7f0b0c133ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:44862/0 (socket says 172.21.15.3:44862) 2023-10-03T10:19:50.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.982+0000 7f0b117a1700 1 -- 172.21.15.3:0/771479155 learned_addr learned my addr 172.21.15.3:0/771479155 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:50.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.983+0000 7f0b127a3700 1 --2- 172.21.15.3:0/771479155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b0c138d80 0x7f0b0c1341e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:50.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.983+0000 7f0b11fa2700 1 --2- 172.21.15.3:0/771479155 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b0c09d630 0x7f0b0c133760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:50.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.983+0000 7f0b117a1700 1 -- 172.21.15.3:0/771479155 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b0c09d630 msgr2=0x7f0b0c133760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:50.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.983+0000 7f0b117a1700 1 --2- 172.21.15.3:0/771479155 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0b0c09d630 0x7f0b0c133760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:50.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.983+0000 7f0b117a1700 1 -- 172.21.15.3:0/771479155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b0c138d80 msgr2=0x7f0b0c1341e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:50.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.983+0000 7f0b117a1700 1 --2- 172.21.15.3:0/771479155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b0c138d80 0x7f0b0c1341e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:50.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.983+0000 7f0b117a1700 1 -- 172.21.15.3:0/771479155 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b04009a70 con 0x7f0b0c0a0a30 2023-10-03T10:19:50.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.983+0000 7f0b117a1700 1 --2- 172.21.15.3:0/771479155 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0b0c0a0a30 0x7f0b0c133ca0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f0afc00c700 tx=0x7f0afc00ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:50.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.983+0000 7f0b027fc700 1 -- 172.21.15.3:0/771479155 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc005d70 con 0x7f0b0c0a0a30 2023-10-03T10:19:50.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.983+0000 7f0b027fc700 1 -- 172.21.15.3:0/771479155 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0afc0070d0 con 0x7f0b0c0a0a30 2023-10-03T10:19:50.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.983+0000 7f0b14206700 1 -- 172.21.15.3:0/771479155 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b080094a0 con 0x7f0b0c0a0a30 2023-10-03T10:19:50.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.983+0000 7f0b027fc700 1 -- 172.21.15.3:0/771479155 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc007440 con 0x7f0b0c0a0a30 2023-10-03T10:19:50.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.983+0000 7f0b14206700 1 -- 172.21.15.3:0/771479155 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b0c1371f0 con 0x7f0b0c0a0a30 2023-10-03T10:19:50.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.984+0000 7f0b027fc700 1 -- 172.21.15.3:0/771479155 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0afc0075a0 con 0x7f0b0c0a0a30 2023-10-03T10:19:50.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.985+0000 7f0b027fc700 1 -- 172.21.15.3:0/771479155 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0afc03eaa0 con 0x7f0b0c0a0a30 2023-10-03T10:19:50.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.985+0000 7f0b14206700 1 -- 172.21.15.3:0/771479155 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b0c137050 con 0x7f0b0c0a0a30 2023-10-03T10:19:50.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:50.990+0000 7f0b027fc700 1 -- 172.21.15.3:0/771479155 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0b0c137050 con 0x7f0b0c0a0a30 2023-10-03T10:19:51.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.021+0000 7f38477e9700 1 Processor -- start 2023-10-03T10:19:51.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.021+0000 7f38477e9700 1 -- start start 2023-10-03T10:19:51.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.022+0000 7f38477e9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f384009d6f0 0x7f384009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:51.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.022+0000 7f38477e9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f384009e000 0x7f3840130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:51.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.022+0000 7f38477e9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f384009f130 con 0x7f384009e410 2023-10-03T10:19:51.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.022+0000 7f38477e9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f384009f520 con 0x7f384009e000 2023-10-03T10:19:51.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.022+0000 7f38477e9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384009e930 con 0x7f384009d6f0 2023-10-03T10:19:51.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.022+0000 7f3845585700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f384009d6f0 0x7f384009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:51.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.022+0000 7f3844d84700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f384009e000 0x7f3840130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:51.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.022+0000 7f3844d84700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f384009e000 0x7f3840130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38908/0 (socket says 172.21.15.3:38908) 2023-10-03T10:19:51.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.022+0000 7f3845585700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f384009d6f0 0x7f384009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:43496/0 (socket says 172.21.15.3:43496) 2023-10-03T10:19:51.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.022+0000 7f3844d84700 1 -- 172.21.15.3:0/148839491 learned_addr learned my addr 172.21.15.3:0/148839491 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:51.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.022+0000 7f3844d84700 1 -- 172.21.15.3:0/148839491 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f384009d6f0 msgr2=0x7f384009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:19:51.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.022+0000 7f3844d84700 1 --2- 172.21.15.3:0/148839491 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f384009d6f0 0x7f384009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:51.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.022+0000 7f3844d84700 1 -- 172.21.15.3:0/148839491 >> v1:172.21.15.3:6789/0 conn(0x7f384009e410 legacy=0x7f3840130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:51.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.022+0000 7f3837fff700 1 -- 172.21.15.3:0/148839491 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3615786750 0 0) 0x7f384009f130 con 0x7f384009e410 2023-10-03T10:19:51.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.022+0000 7f3844d84700 1 -- 172.21.15.3:0/148839491 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384009b2d0 con 0x7f384009e000 2023-10-03T10:19:51.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.022+0000 7f3845585700 1 --2- 172.21.15.3:0/148839491 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f384009d6f0 0x7f384009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:51.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.023+0000 7f3844d84700 1 --2- 172.21.15.3:0/148839491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f384009e000 0x7f3840130660 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f383000b9a0 tx=0x7f383000bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c920994f51e31ca1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:51.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.023+0000 7f3837fff700 1 -- 172.21.15.3:0/148839491 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383000caa0 con 0x7f384009e000 2023-10-03T10:19:51.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.023+0000 7f3837fff700 1 -- 172.21.15.3:0/148839491 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383000cc00 con 0x7f384009e000 2023-10-03T10:19:51.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.023+0000 7f3837fff700 1 -- 172.21.15.3:0/148839491 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383000f460 con 0x7f384009e000 2023-10-03T10:19:51.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.023+0000 7f38477e9700 1 -- 172.21.15.3:0/148839491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f384009e000 msgr2=0x7f3840130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:51.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.023+0000 7f38477e9700 1 --2- 172.21.15.3:0/148839491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f384009e000 0x7f3840130660 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f383000b9a0 tx=0x7f383000bcb0 comp rx=0 tx=0).stop 2023-10-03T10:19:51.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.023+0000 7f38477e9700 1 -- 172.21.15.3:0/148839491 shutdown_connections 2023-10-03T10:19:51.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.023+0000 7f38477e9700 1 --2- 172.21.15.3:0/148839491 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f384009d6f0 0x7f384009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:51.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.023+0000 7f38477e9700 1 --2- 172.21.15.3:0/148839491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f384009e000 0x7f3840130660 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:51.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.023+0000 7f38477e9700 1 -- 172.21.15.3:0/148839491 >> 172.21.15.3:0/148839491 conn(0x7f3840066f50 msgr2=0x7f384009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:51.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.024+0000 7f38477e9700 1 -- 172.21.15.3:0/148839491 shutdown_connections 2023-10-03T10:19:51.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.024+0000 7f38477e9700 1 -- 172.21.15.3:0/148839491 wait complete. 2023-10-03T10:19:51.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.025+0000 7f38477e9700 1 Processor -- start 2023-10-03T10:19:51.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.025+0000 7f38477e9700 1 -- start start 2023-10-03T10:19:51.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.025+0000 7f38477e9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f384009d6f0 0x7f384011de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:51.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.025+0000 7f38477e9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f384009e000 0x7f384011e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:51.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.025+0000 7f38477e9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f384009e410 0x7f384011e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:19:51.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.025+0000 7f38477e9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384011f280 con 0x7f384009d6f0 2023-10-03T10:19:51.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.025+0000 7f38477e9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3840120010 con 0x7f384009e000 2023-10-03T10:19:51.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.025+0000 7f38477e9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38401210d0 con 0x7f384009e410 2023-10-03T10:19:51.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.025+0000 7f3844d84700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f384009e000 0x7f384011e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:51.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.025+0000 7f3844d84700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f384009e000 0x7f384011e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:43504/0 (socket says 172.21.15.3:43504) 2023-10-03T10:19:51.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.025+0000 7f3844d84700 1 -- 172.21.15.3:0/2235936589 learned_addr learned my addr 172.21.15.3:0/2235936589 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:19:51.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.025+0000 7f3845585700 1 --2- 172.21.15.3:0/2235936589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f384009d6f0 0x7f384011de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:51.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.025+0000 7f3845d86700 1 --2- 172.21.15.3:0/2235936589 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f384009e410 0x7f384011e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:19:51.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.026+0000 7f3845585700 1 -- 172.21.15.3:0/2235936589 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f384009e410 msgr2=0x7f384011e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:51.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.026+0000 7f3845585700 1 --2- 172.21.15.3:0/2235936589 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f384009e410 0x7f384011e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:51.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.026+0000 7f3845585700 1 -- 172.21.15.3:0/2235936589 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f384009e000 msgr2=0x7f384011e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:51.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.026+0000 7f3845585700 1 --2- 172.21.15.3:0/2235936589 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f384009e000 0x7f384011e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:51.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.026+0000 7f3845585700 1 -- 172.21.15.3:0/2235936589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f383c00d940 con 0x7f384009d6f0 2023-10-03T10:19:51.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.026+0000 7f3844d84700 1 --2- 172.21.15.3:0/2235936589 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f384009e000 0x7f384011e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:19:51.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.026+0000 7f3845585700 1 --2- 172.21.15.3:0/2235936589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f384009d6f0 0x7f384011de20 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f383c00dc90 tx=0x7f383c0100c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:19:51.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.026+0000 7f3835ffb700 1 -- 172.21.15.3:0/2235936589 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c0108e0 con 0x7f384009d6f0 2023-10-03T10:19:51.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.026+0000 7f3835ffb700 1 -- 172.21.15.3:0/2235936589 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383c013040 con 0x7f384009d6f0 2023-10-03T10:19:51.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.026+0000 7f38477e9700 1 -- 172.21.15.3:0/2235936589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f383000b600 con 0x7f384009d6f0 2023-10-03T10:19:51.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.026+0000 7f3835ffb700 1 -- 172.21.15.3:0/2235936589 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c010c80 con 0x7f384009d6f0 2023-10-03T10:19:51.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.026+0000 7f38477e9700 1 -- 172.21.15.3:0/2235936589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38400720b0 con 0x7f384009d6f0 2023-10-03T10:19:51.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.027+0000 7f3835ffb700 1 -- 172.21.15.3:0/2235936589 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f383c018480 con 0x7f384009d6f0 2023-10-03T10:19:51.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.027+0000 7f38477e9700 1 -- 172.21.15.3:0/2235936589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f384011ac20 con 0x7f384009d6f0 2023-10-03T10:19:51.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.028+0000 7f3835ffb700 1 -- 172.21.15.3:0/2235936589 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f383c043520 con 0x7f384009d6f0 2023-10-03T10:19:51.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.032+0000 7f3835ffb700 1 -- 172.21.15.3:0/2235936589 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f383c016930 con 0x7f384009d6f0 2023-10-03T10:19:51.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.183+0000 7f5ceef0d700 1 -- 172.21.15.3:0/3486358828 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5ce81a7b10 con 0x7f5ce809d6f0 2023-10-03T10:19:51.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.184+0000 7f5ce57fa700 1 -- 172.21.15.3:0/3486358828 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5cdc011960 con 0x7f5ce809d6f0 2023-10-03T10:19:51.184 INFO:teuthology.orchestra.run.smithi003.stdout:133143986179 2023-10-03T10:19:51.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.186+0000 7f5ceef0d700 1 -- 172.21.15.3:0/3486358828 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ce809d6f0 msgr2=0x7f5ce81c7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:51.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.186+0000 7f5ceef0d700 1 --2- 172.21.15.3:0/3486358828 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ce809d6f0 0x7f5ce81c7f40 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00ebe0 tx=0x7f5cdc00eef0 comp rx=0 tx=0).stop 2023-10-03T10:19:51.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.186+0000 7f5ceef0d700 1 -- 172.21.15.3:0/3486358828 shutdown_connections 2023-10-03T10:19:51.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.186+0000 7f5ceef0d700 1 --2- 172.21.15.3:0/3486358828 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5ce809e410 0x7f5ce81c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:51.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.186+0000 7f5ceef0d700 1 --2- 172.21.15.3:0/3486358828 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5ce809d6f0 0x7f5ce81c7f40 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:51.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.186+0000 7f5ceef0d700 1 --2- 172.21.15.3:0/3486358828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ce809e000 0x7f5ce81c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:51.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.186+0000 7f5ceef0d700 1 -- 172.21.15.3:0/3486358828 >> 172.21.15.3:0/3486358828 conn(0x7f5ce8066f50 msgr2=0x7f5ce81bbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:51.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.186+0000 7f5ceef0d700 1 -- 172.21.15.3:0/3486358828 shutdown_connections 2023-10-03T10:19:51.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.186+0000 7f5ceef0d700 1 -- 172.21.15.3:0/3486358828 wait complete. 2023-10-03T10:19:51.200 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986179 for osd.1 2023-10-03T10:19:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.248+0000 7f38477e9700 1 -- 172.21.15.3:0/2235936589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3840074ea0 con 0x7f384009d6f0 2023-10-03T10:19:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.249+0000 7f3835ffb700 1 -- 172.21.15.3:0/2235936589 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f383c016340 con 0x7f384009d6f0 2023-10-03T10:19:51.249 INFO:teuthology.orchestra.run.smithi003.stdout:133143986178 2023-10-03T10:19:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.251+0000 7f38477e9700 1 -- 172.21.15.3:0/2235936589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f384009d6f0 msgr2=0x7f384011de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:19:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.251+0000 7f38477e9700 1 --2- 172.21.15.3:0/2235936589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f384009d6f0 0x7f384011de20 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f383c00dc90 tx=0x7f383c0100c0 comp rx=0 tx=0).stop 2023-10-03T10:19:51.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.251+0000 7f38477e9700 1 -- 172.21.15.3:0/2235936589 shutdown_connections 2023-10-03T10:19:51.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.251+0000 7f38477e9700 1 --2- 172.21.15.3:0/2235936589 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f384009e410 0x7f384011e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:51.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.251+0000 7f38477e9700 1 --2- 172.21.15.3:0/2235936589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f384009d6f0 0x7f384011de20 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:51.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.251+0000 7f38477e9700 1 --2- 172.21.15.3:0/2235936589 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f384009e000 0x7f384011e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:19:51.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.251+0000 7f38477e9700 1 -- 172.21.15.3:0/2235936589 >> 172.21.15.3:0/2235936589 conn(0x7f3840066f50 msgr2=0x7f3840130a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:19:51.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.251+0000 7f38477e9700 1 -- 172.21.15.3:0/2235936589 shutdown_connections 2023-10-03T10:19:51.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:19:51.251+0000 7f38477e9700 1 -- 172.21.15.3:0/2235936589 wait complete. 2023-10-03T10:19:51.265 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-10-03T10:21:50.872 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-10-03T10:21:50.873 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph.py", line 1496, in healthy manager.wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi003 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-10-03T10:21:50.940 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=da0dc0d18c3f4803878317e6040c6e0e Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph.py", line 1496, in healthy manager.wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi003 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-10-03T10:21:50.944 DEBUG:teuthology.run_tasks:Unwinding manager ceph.restart 2023-10-03T10:21:50.954 DEBUG:teuthology.run_tasks:Unwinding manager ceph.stop 2023-10-03T10:21:50.963 DEBUG:teuthology.run_tasks:Unwinding manager ceph.restart 2023-10-03T10:21:50.972 DEBUG:teuthology.run_tasks:Unwinding manager ceph.restart 2023-10-03T10:21:50.982 DEBUG:teuthology.run_tasks:Unwinding manager install.upgrade 2023-10-03T10:21:50.991 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-10-03T10:21:51.000 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-10-03T10:21:51.001 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-10-03T10:21:51.001 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-03T10:21:51.031 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-10-03T10:21:51.031 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-03T10:21:51.094 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-03T10:21:51.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.240+0000 7f9aa1337700 1 Processor -- start 2023-10-03T10:21:51.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.240+0000 7f9aa1337700 1 -- start start 2023-10-03T10:21:51.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.241+0000 7f9aa1337700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9a9c0a09c0 0x7f9a9c0a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:51.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.241+0000 7f9aa1337700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a9c09d5c0 0x7f9a9c0a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:51.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.241+0000 7f9aa1337700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a9c133920 0x7f9a9c133d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.241+0000 7f9aa1337700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a9c0a1810 con 0x7f9a9c0a09c0 2023-10-03T10:21:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.241+0000 7f9aa1337700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c09ea10 con 0x7f9a9c133920 2023-10-03T10:21:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.241+0000 7f9aa1337700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c09e210 con 0x7f9a9c09d5c0 2023-10-03T10:21:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.241+0000 7f9a9b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a9c133920 0x7f9a9c133d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.241+0000 7f9a927fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a9c09d5c0 0x7f9a9c0a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.241+0000 7f9a9b7fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a9c133920 0x7f9a9c133d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59790/0 (socket says 172.21.15.3:59790) 2023-10-03T10:21:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.241+0000 7f9a927fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a9c09d5c0 0x7f9a9c0a12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54970/0 (socket says 172.21.15.3:54970) 2023-10-03T10:21:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.241+0000 7f9a9b7fe700 1 -- 172.21.15.3:0/4065549119 learned_addr learned my addr 172.21.15.3:0/4065549119 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:21:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.241+0000 7f9a9affd700 1 --2- 172.21.15.3:0/4065549119 >> v2:172.21.15.3:3300/0 conn(0x7f9a9c0a09c0 0x7f9a9c0a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.242+0000 7f9a9b7fe700 1 -- 172.21.15.3:0/4065549119 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a9c09d5c0 msgr2=0x7f9a9c0a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:21:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.242+0000 7f9a9b7fe700 1 --2- 172.21.15.3:0/4065549119 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a9c09d5c0 0x7f9a9c0a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.242+0000 7f9a9b7fe700 1 -- 172.21.15.3:0/4065549119 >> v2:172.21.15.3:3300/0 conn(0x7f9a9c0a09c0 msgr2=0x7f9a9c0a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:21:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.242+0000 7f9a9b7fe700 1 --2- 172.21.15.3:0/4065549119 >> v2:172.21.15.3:3300/0 conn(0x7f9a9c0a09c0 0x7f9a9c0a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.242+0000 7f9a9b7fe700 1 -- 172.21.15.3:0/4065549119 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9c0a26f0 con 0x7f9a9c133920 2023-10-03T10:21:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.242+0000 7f9a927fc700 1 --2- 172.21.15.3:0/4065549119 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a9c09d5c0 0x7f9a9c0a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:21:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.242+0000 7f9a9affd700 1 --2- 172.21.15.3:0/4065549119 >> v2:172.21.15.3:3300/0 conn(0x7f9a9c0a09c0 0x7f9a9c0a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:21:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.242+0000 7f9a9b7fe700 1 --2- 172.21.15.3:0/4065549119 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a9c133920 0x7f9a9c133d70 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9a94009e50 tx=0x7f9a9400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54f54eb1962ef18d server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:21:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.242+0000 7f9a9a7fc700 1 -- 172.21.15.3:0/4065549119 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9400e070 con 0x7f9a9c133920 2023-10-03T10:21:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.242+0000 7f9a9a7fc700 1 -- 172.21.15.3:0/4065549119 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a9400b910 con 0x7f9a9c133920 2023-10-03T10:21:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.242+0000 7f9a9a7fc700 1 -- 172.21.15.3:0/4065549119 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9400bcc0 con 0x7f9a9c133920 2023-10-03T10:21:51.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.243+0000 7f9aa1337700 1 -- 172.21.15.3:0/4065549119 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a9c133920 msgr2=0x7f9a9c133d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:51.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.243+0000 7f9aa1337700 1 --2- 172.21.15.3:0/4065549119 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a9c133920 0x7f9a9c133d70 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9a94009e50 tx=0x7f9a9400b330 comp rx=0 tx=0).stop 2023-10-03T10:21:51.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.243+0000 7f9aa1337700 1 -- 172.21.15.3:0/4065549119 shutdown_connections 2023-10-03T10:21:51.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.243+0000 7f9aa1337700 1 --2- 172.21.15.3:0/4065549119 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a9c133920 0x7f9a9c133d70 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.243+0000 7f9aa1337700 1 --2- 172.21.15.3:0/4065549119 >> v2:172.21.15.3:3300/0 conn(0x7f9a9c0a09c0 0x7f9a9c0a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.243+0000 7f9aa1337700 1 --2- 172.21.15.3:0/4065549119 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a9c09d5c0 0x7f9a9c0a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.243+0000 7f9aa1337700 1 -- 172.21.15.3:0/4065549119 >> 172.21.15.3:0/4065549119 conn(0x7f9a9c0a2fe0 msgr2=0x7f9a9c0a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:21:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.243+0000 7f9aa1337700 1 -- 172.21.15.3:0/4065549119 shutdown_connections 2023-10-03T10:21:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.243+0000 7f9aa1337700 1 -- 172.21.15.3:0/4065549119 wait complete. 2023-10-03T10:21:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.244+0000 7f9aa1337700 1 Processor -- start 2023-10-03T10:21:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.244+0000 7f9aa1337700 1 -- start start 2023-10-03T10:21:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.244+0000 7f9aa1337700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a9c09d5c0 0x7f9a9c06dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.244+0000 7f9aa1337700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a9c0a09c0 0x7f9a9c06e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.244+0000 7f9aa1337700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a9c133920 0x7f9a9c1cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.244+0000 7f9aa1337700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c070fd0 con 0x7f9a9c09d5c0 2023-10-03T10:21:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.244+0000 7f9aa1337700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c0719e0 con 0x7f9a9c133920 2023-10-03T10:21:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.244+0000 7f9aa1337700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c06ee90 con 0x7f9a9c0a09c0 2023-10-03T10:21:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.244+0000 7f9a9affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a9c09d5c0 0x7f9a9c06dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.244+0000 7f9a9affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a9c09d5c0 0x7f9a9c06dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42548/0 (socket says 172.21.15.3:42548) 2023-10-03T10:21:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.244+0000 7f9a927fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a9c0a09c0 0x7f9a9c06e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.244+0000 7f9a9affd700 1 -- 172.21.15.3:0/1753031103 learned_addr learned my addr 172.21.15.3:0/1753031103 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:21:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.245+0000 7f9a9b7fe700 1 --2- 172.21.15.3:0/1753031103 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a9c133920 0x7f9a9c1cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.245+0000 7f9a9affd700 1 -- 172.21.15.3:0/1753031103 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a9c0a09c0 msgr2=0x7f9a9c06e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.245+0000 7f9a9affd700 1 --2- 172.21.15.3:0/1753031103 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a9c0a09c0 0x7f9a9c06e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.245+0000 7f9a9affd700 1 -- 172.21.15.3:0/1753031103 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a9c133920 msgr2=0x7f9a9c1cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.245+0000 7f9a9affd700 1 --2- 172.21.15.3:0/1753031103 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a9c133920 0x7f9a9c1cc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.245+0000 7f9a9affd700 1 -- 172.21.15.3:0/1753031103 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a840094a0 con 0x7f9a9c09d5c0 2023-10-03T10:21:51.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.245+0000 7f9a927fc700 1 --2- 172.21.15.3:0/1753031103 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a9c0a09c0 0x7f9a9c06e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:21:51.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.245+0000 7f9a9b7fe700 1 --2- 172.21.15.3:0/1753031103 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a9c133920 0x7f9a9c1cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:21:51.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.245+0000 7f9a9affd700 1 --2- 172.21.15.3:0/1753031103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a9c09d5c0 0x7f9a9c06dd30 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c00ebe0 tx=0x7f9a8c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:21:51.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.245+0000 7f9a93fff700 1 -- 172.21.15.3:0/1753031103 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c00c7f0 con 0x7f9a9c09d5c0 2023-10-03T10:21:51.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.245+0000 7f9a93fff700 1 -- 172.21.15.3:0/1753031103 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8c00c950 con 0x7f9a9c09d5c0 2023-10-03T10:21:51.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.245+0000 7f9a93fff700 1 -- 172.21.15.3:0/1753031103 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c00cce0 con 0x7f9a9c09d5c0 2023-10-03T10:21:51.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.245+0000 7f9aa1337700 1 -- 172.21.15.3:0/1753031103 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a8c0094a0 con 0x7f9a9c09d5c0 2023-10-03T10:21:51.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.245+0000 7f9aa1337700 1 -- 172.21.15.3:0/1753031103 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a94009b30 con 0x7f9a9c09d5c0 2023-10-03T10:21:51.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.246+0000 7f9a93fff700 1 -- 172.21.15.3:0/1753031103 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9a8c013470 con 0x7f9a9c09d5c0 2023-10-03T10:21:51.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.246+0000 7f9a93fff700 1 -- 172.21.15.3:0/1753031103 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f9a8c03f740 con 0x7f9a9c09d5c0 2023-10-03T10:21:51.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.246+0000 7f9aa1337700 1 -- 172.21.15.3:0/1753031103 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a9c071320 con 0x7f9a9c09d5c0 2023-10-03T10:21:51.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.251+0000 7f9a93fff700 1 -- 172.21.15.3:0/1753031103 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9a8c013720 con 0x7f9a9c09d5c0 2023-10-03T10:21:51.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.456+0000 7f9aa1337700 1 -- 172.21.15.3:0/1753031103 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a9c0747f0 con 0x7f9a9c09d5c0 2023-10-03T10:21:51.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.457+0000 7f9a93fff700 1 -- 172.21.15.3:0/1753031103 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+13448 (secure 0 0 0) 0x7f9a8c01b970 con 0x7f9a9c09d5c0 2023-10-03T10:21:51.458 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:21:51.458 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":40,"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","created":"2023-10-03T10:09:28.703441+0000","modified":"2023-10-03T10:13:12.182630+0000","last_up_change":"2023-10-03T10:12:32.260880+0000","last_in_change":"2023-10-03T10:09:30.156546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-03T10:09:36.623971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-03T10:09:38.210614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-03T10:09:42.695873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-03T10:09:43.115588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cdcdd930-3c5b-4486-a021-7132f90fb6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":29,"up_thru":32,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6804","nonce":129556}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6805","nonce":129556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6807","nonce":129556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6806","nonce":129556}]},"public_addr":"172.21.15.3:6804/129556","cluster_addr":"172.21.15.3:6805/129556","heartbeat_back_addr":"172.21.15.3:6807/129556","heartbeat_front_addr":"172.21.15.3:6806/129556","state":["exists","up"]},{"osd":1,"uuid":"c8e58877-6762-4741-a8c1-567ebc97b708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":31,"up_thru":32,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6808","nonce":129654}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6809","nonce":129654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6811","nonce":129654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6810","nonce":129654}]},"public_addr":"172.21.15.3:6808/129654","cluster_addr":"172.21.15.3:6809/129654","heartbeat_back_addr":"172.21.15.3:6811/129654","heartbeat_front_addr":"172.21.15.3:6810/129654","state":["exists","up"]},{"osd":2,"uuid":"68a2a21a-7a8e-4f82-8433-5201faa50777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":25,"up_from":31,"up_thru":32,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6800","nonce":129764}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6801","nonce":129764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6803","nonce":129764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6802","nonce":129764}]},"public_addr":"172.21.15.3:6800/129764","cluster_addr":"172.21.15.3:6801/129764","heartbeat_back_addr":"172.21.15.3:6803/129764","heartbeat_front_addr":"172.21.15.3:6802/129764","state":["exists","up"]},{"osd":3,"uuid":"c4df2903-4e1e-4208-bf77-95b9fa749241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":26,"up_from":32,"up_thru":32,"down_at":27,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6812","nonce":129912}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6813","nonce":129912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6815","nonce":129912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6814","nonce":129912}]},"public_addr":"172.21.15.3:6812/129912","cluster_addr":"172.21.15.3:6813/129912","heartbeat_back_addr":"172.21.15.3:6815/129912","heartbeat_front_addr":"172.21.15.3:6814/129912","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.369708+0000","dead_epoch":24},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.257825+0000","dead_epoch":25},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.337931+0000","dead_epoch":26},{"osd":3,"down_stamp":"2023-10-03T10:12:26.212430+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.085222+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1","mappings":[{"from":3,"to":0}]},{"pgid":"4.8","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.19","mappings":[{"from":3,"to":0}]},{"pgid":"4.2a","mappings":[{"from":1,"to":2}]},{"pgid":"4.2d","mappings":[{"from":3,"to":0}]},{"pgid":"4.33","mappings":[{"from":3,"to":0}]},{"pgid":"4.39","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/3264604155":"2023-10-04T10:13:12.182591+0000","172.21.15.3:0/1911521769":"2023-10-04T10:13:12.182591+0000","172.21.15.3:0/3289486158":"2023-10-04T10:13:12.182591+0000","172.21.15.3:6816/118296":"2023-10-04T10:12:13.305411+0000","172.21.15.3:0/3246155339":"2023-10-04T10:13:12.182591+0000","172.21.15.3:0/2091814084":"2023-10-04T10:12:13.305411+0000","172.21.15.3:6817/2582564779":"2023-10-04T10:12:38.734689+0000","172.21.15.3:0/3571619151":"2023-10-04T10:13:12.182591+0000","172.21.15.3:0/3919724564":"2023-10-04T10:12:13.305411+0000","172.21.15.3:6816/127025":"2023-10-04T10:13:12.182591+0000","172.21.15.3:0/2059330378":"2023-10-04T10:13:12.182591+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-03T10:21:51.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.460+0000 7f9aa1337700 1 -- 172.21.15.3:0/1753031103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a9c09d5c0 msgr2=0x7f9a9c06dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:51.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.460+0000 7f9aa1337700 1 --2- 172.21.15.3:0/1753031103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a9c09d5c0 0x7f9a9c06dd30 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c00ebe0 tx=0x7f9a8c00eef0 comp rx=0 tx=0).stop 2023-10-03T10:21:51.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.460+0000 7f9aa1337700 1 -- 172.21.15.3:0/1753031103 shutdown_connections 2023-10-03T10:21:51.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.460+0000 7f9aa1337700 1 --2- 172.21.15.3:0/1753031103 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9a9c133920 0x7f9a9c1cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:51.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.460+0000 7f9aa1337700 1 --2- 172.21.15.3:0/1753031103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a9c09d5c0 0x7f9a9c06dd30 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:51.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.460+0000 7f9aa1337700 1 --2- 172.21.15.3:0/1753031103 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f9a9c0a09c0 0x7f9a9c06e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:51.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.460+0000 7f9aa1337700 1 -- 172.21.15.3:0/1753031103 >> 172.21.15.3:0/1753031103 conn(0x7f9a9c0a2fe0 msgr2=0x7f9a9c1bbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:21:51.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.460+0000 7f9aa1337700 1 -- 172.21.15.3:0/1753031103 shutdown_connections 2023-10-03T10:21:51.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.460+0000 7f9aa1337700 1 -- 172.21.15.3:0/1753031103 wait complete. 2023-10-03T10:21:51.474 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-03T10:09:36.623971+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}, 'mgr_devicehealth': {}}, 'read_balance': {'score_acting': 4, 'score_stable': 4, 'optimal_score': 0.5, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-03T10:09:38.210614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-03T10:09:42.695873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-03T10:09:43.115588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.059999942779541, 'score_stable': 1.059999942779541, 'optimal_score': 1, 'raw_score_acting': 1.059999942779541, 'raw_score_stable': 1.059999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-03T10:21:51.475 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-03T10:21:51.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.615+0000 7f38645f1700 1 Processor -- start 2023-10-03T10:21:51.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.615+0000 7f38645f1700 1 -- start start 2023-10-03T10:21:51.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.615+0000 7f38645f1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f385c071cd0 0x7f385c0720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:51.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.615+0000 7f38645f1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f385c09f370 0x7f385c13bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:51.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.615+0000 7f38645f1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f385c06cb80 con 0x7f385c09e850 2023-10-03T10:21:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.615+0000 7f38645f1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385c0a0680 con 0x7f385c071cd0 2023-10-03T10:21:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.615+0000 7f38645f1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f385c06c0a0 con 0x7f385c09f370 2023-10-03T10:21:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.615+0000 7f386238d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f385c071cd0 0x7f385c0720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.615+0000 7f3862b8e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f385c09f370 0x7f385c13bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.616+0000 7f3862b8e700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f385c09f370 0x7f385c13bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:54986/0 (socket says 172.21.15.3:54986) 2023-10-03T10:21:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.616+0000 7f386238d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f385c071cd0 0x7f385c0720a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59808/0 (socket says 172.21.15.3:59808) 2023-10-03T10:21:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.616+0000 7f3862b8e700 1 -- 172.21.15.3:0/377788238 learned_addr learned my addr 172.21.15.3:0/377788238 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:21:51.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.616+0000 7f3862b8e700 1 -- 172.21.15.3:0/377788238 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f385c071cd0 msgr2=0x7f385c0720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:21:51.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.616+0000 7f3862b8e700 1 --2- 172.21.15.3:0/377788238 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f385c071cd0 0x7f385c0720a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:51.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.616+0000 7f3862b8e700 1 -- 172.21.15.3:0/377788238 >> v1:172.21.15.3:6789/0 conn(0x7f385c09e850 legacy=0x7f385c09ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:51.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.616+0000 7f3862b8e700 1 -- 172.21.15.3:0/377788238 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385c09f810 con 0x7f385c09f370 2023-10-03T10:21:51.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.616+0000 7f386138b700 1 -- 172.21.15.3:0/377788238 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3547278052 0 0) 0x7f385c06cb80 con 0x7f385c09e850 2023-10-03T10:21:51.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.616+0000 7f386238d700 1 --2- 172.21.15.3:0/377788238 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f385c071cd0 0x7f385c0720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:21:51.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.616+0000 7f3862b8e700 1 --2- 172.21.15.3:0/377788238 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f385c09f370 0x7f385c13bbb0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f3854009920 tx=0x7f3854009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d48e83625e8177b server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:21:51.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.616+0000 7f386138b700 1 -- 172.21.15.3:0/377788238 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854011830 con 0x7f385c09f370 2023-10-03T10:21:51.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.616+0000 7f386138b700 1 -- 172.21.15.3:0/377788238 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3854011990 con 0x7f385c09f370 2023-10-03T10:21:51.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.616+0000 7f386138b700 1 -- 172.21.15.3:0/377788238 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854011d20 con 0x7f385c09f370 2023-10-03T10:21:51.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.617+0000 7f38645f1700 1 -- 172.21.15.3:0/377788238 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f385c09f370 msgr2=0x7f385c13bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:51.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.617+0000 7f38645f1700 1 --2- 172.21.15.3:0/377788238 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f385c09f370 0x7f385c13bbb0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f3854009920 tx=0x7f3854009c30 comp rx=0 tx=0).stop 2023-10-03T10:21:51.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.617+0000 7f38645f1700 1 -- 172.21.15.3:0/377788238 shutdown_connections 2023-10-03T10:21:51.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.617+0000 7f38645f1700 1 --2- 172.21.15.3:0/377788238 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f385c071cd0 0x7f385c0720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:51.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.617+0000 7f38645f1700 1 --2- 172.21.15.3:0/377788238 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f385c09f370 0x7f385c13bbb0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:51.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.617+0000 7f38645f1700 1 -- 172.21.15.3:0/377788238 >> 172.21.15.3:0/377788238 conn(0x7f385c0a4130 msgr2=0x7f385c0a4530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:21:51.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.617+0000 7f38645f1700 1 -- 172.21.15.3:0/377788238 shutdown_connections 2023-10-03T10:21:51.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.617+0000 7f38645f1700 1 -- 172.21.15.3:0/377788238 wait complete. 2023-10-03T10:21:51.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.618+0000 7f38645f1700 1 Processor -- start 2023-10-03T10:21:51.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.618+0000 7f38645f1700 1 -- start start 2023-10-03T10:21:51.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.618+0000 7f38645f1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f385c071cd0 0x7f385c1a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:51.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.618+0000 7f38645f1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f385c09e850 0x7f385c1a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:51.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.618+0000 7f38645f1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f385c09f370 0x7f385c1a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:51.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.618+0000 7f38645f1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c1a6730 con 0x7f385c09f370 2023-10-03T10:21:51.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.618+0000 7f38645f1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385c1a8d80 con 0x7f385c09e850 2023-10-03T10:21:51.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.618+0000 7f38645f1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f385c1a8150 con 0x7f385c071cd0 2023-10-03T10:21:51.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.619+0000 7f386238d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f385c071cd0 0x7f385c1a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:51.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.619+0000 7f3862b8e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f385c09f370 0x7f385c1a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:51.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.619+0000 7f3861b8c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f385c09e850 0x7f385c1a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:51.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.619+0000 7f386238d700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f385c071cd0 0x7f385c1a5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55002/0 (socket says 172.21.15.3:55002) 2023-10-03T10:21:51.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.619+0000 7f386238d700 1 -- 172.21.15.3:0/1539765938 learned_addr learned my addr 172.21.15.3:0/1539765938 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:21:51.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.619+0000 7f3862b8e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f385c09f370 0x7f385c1a5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42560/0 (socket says 172.21.15.3:42560) 2023-10-03T10:21:51.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.619+0000 7f3861b8c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f385c09e850 0x7f385c1a57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59812/0 (socket says 172.21.15.3:59812) 2023-10-03T10:21:51.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.619+0000 7f386238d700 1 -- 172.21.15.3:0/1539765938 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f385c09e850 msgr2=0x7f385c1a57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:51.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.619+0000 7f386238d700 1 --2- 172.21.15.3:0/1539765938 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f385c09e850 0x7f385c1a57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:51.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.619+0000 7f386238d700 1 -- 172.21.15.3:0/1539765938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f385c09f370 msgr2=0x7f385c1a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:51.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.619+0000 7f386238d700 1 --2- 172.21.15.3:0/1539765938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f385c09f370 0x7f385c1a5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:51.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.619+0000 7f386238d700 1 -- 172.21.15.3:0/1539765938 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3858009a70 con 0x7f385c071cd0 2023-10-03T10:21:51.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.619+0000 7f386238d700 1 --2- 172.21.15.3:0/1539765938 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f385c071cd0 0x7f385c1a5260 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f3858009ec0 tx=0x7f385800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:21:51.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.619+0000 7f3852ffd700 1 -- 172.21.15.3:0/1539765938 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385800cdd0 con 0x7f385c071cd0 2023-10-03T10:21:51.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.619+0000 7f3852ffd700 1 -- 172.21.15.3:0/1539765938 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385800f040 con 0x7f385c071cd0 2023-10-03T10:21:51.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.619+0000 7f38645f1700 1 -- 172.21.15.3:0/1539765938 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3854009550 con 0x7f385c071cd0 2023-10-03T10:21:51.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.619+0000 7f3852ffd700 1 -- 172.21.15.3:0/1539765938 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38580146a0 con 0x7f385c071cd0 2023-10-03T10:21:51.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.619+0000 7f38645f1700 1 -- 172.21.15.3:0/1539765938 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f385c06cda0 con 0x7f385c071cd0 2023-10-03T10:21:51.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.620+0000 7f3852ffd700 1 -- 172.21.15.3:0/1539765938 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3858014800 con 0x7f385c071cd0 2023-10-03T10:21:51.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.620+0000 7f3852ffd700 1 -- 172.21.15.3:0/1539765938 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f3858040300 con 0x7f385c071cd0 2023-10-03T10:21:51.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.620+0000 7f38645f1700 1 -- 172.21.15.3:0/1539765938 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f385c09fd60 con 0x7f385c071cd0 2023-10-03T10:21:51.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.625+0000 7f3852ffd700 1 -- 172.21.15.3:0/1539765938 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f385801d400 con 0x7f385c071cd0 2023-10-03T10:21:51.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.829+0000 7f38645f1700 1 -- 172.21.15.3:0/1539765938 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f385c1a8900 con 0x7f385c071cd0 2023-10-03T10:21:51.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.829+0000 7f3852ffd700 1 -- 172.21.15.3:0/1539765938 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f385c1a8900 con 0x7f385c071cd0 2023-10-03T10:21:51.830 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-03T10:21:51.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.831+0000 7f38645f1700 1 -- 172.21.15.3:0/1539765938 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f385c071cd0 msgr2=0x7f385c1a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:51.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.831+0000 7f38645f1700 1 --2- 172.21.15.3:0/1539765938 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f385c071cd0 0x7f385c1a5260 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f3858009ec0 tx=0x7f385800c3a0 comp rx=0 tx=0).stop 2023-10-03T10:21:51.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.832+0000 7f38645f1700 1 -- 172.21.15.3:0/1539765938 shutdown_connections 2023-10-03T10:21:51.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.832+0000 7f38645f1700 1 --2- 172.21.15.3:0/1539765938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f385c09f370 0x7f385c1a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:51.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.832+0000 7f38645f1700 1 --2- 172.21.15.3:0/1539765938 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f385c071cd0 0x7f385c1a5260 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:51.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.832+0000 7f38645f1700 1 --2- 172.21.15.3:0/1539765938 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f385c09e850 0x7f385c1a57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:51.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.832+0000 7f38645f1700 1 -- 172.21.15.3:0/1539765938 >> 172.21.15.3:0/1539765938 conn(0x7f385c0a4130 msgr2=0x7f385c1bb210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:21:51.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.832+0000 7f38645f1700 1 -- 172.21.15.3:0/1539765938 shutdown_connections 2023-10-03T10:21:51.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.832+0000 7f38645f1700 1 -- 172.21.15.3:0/1539765938 wait complete. 2023-10-03T10:21:51.845 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-03T10:21:51.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.985+0000 7f2631926700 1 Processor -- start 2023-10-03T10:21:51.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.985+0000 7f2631926700 1 -- start start 2023-10-03T10:21:51.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.985+0000 7f2631926700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f262c0a09c0 0x7f262c0a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:51.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.985+0000 7f2631926700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f262c09d5c0 0x7f262c0a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:51.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.985+0000 7f2631926700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f262c0a1810 con 0x7f262c133920 2023-10-03T10:21:51.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.985+0000 7f2631926700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f262c09ea10 con 0x7f262c0a09c0 2023-10-03T10:21:51.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.985+0000 7f2631926700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f262c09e210 con 0x7f262c09d5c0 2023-10-03T10:21:51.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.985+0000 7f262a7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f262c09d5c0 0x7f262c0a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:51.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.985+0000 7f262affd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f262c0a09c0 0x7f262c0a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:51.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.985+0000 7f262affd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f262c0a09c0 0x7f262c0a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42566/0 (socket says 172.21.15.3:42566) 2023-10-03T10:21:51.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.985+0000 7f262a7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f262c09d5c0 0x7f262c0a12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55008/0 (socket says 172.21.15.3:55008) 2023-10-03T10:21:51.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.985+0000 7f262affd700 1 -- 172.21.15.3:0/850274744 learned_addr learned my addr 172.21.15.3:0/850274744 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:21:51.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.986+0000 7f262affd700 1 -- 172.21.15.3:0/850274744 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f262c09d5c0 msgr2=0x7f262c0a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:21:51.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.986+0000 7f262affd700 1 --2- 172.21.15.3:0/850274744 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f262c09d5c0 0x7f262c0a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:51.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.986+0000 7f262affd700 1 -- 172.21.15.3:0/850274744 >> v1:172.21.15.3:6789/0 conn(0x7f262c133920 legacy=0x7f262c133d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:51.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.986+0000 7f262affd700 1 -- 172.21.15.3:0/850274744 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262c0a26f0 con 0x7f262c0a09c0 2023-10-03T10:21:51.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.986+0000 7f262a7fc700 1 --2- 172.21.15.3:0/850274744 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f262c09d5c0 0x7f262c0a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:21:51.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.986+0000 7f262affd700 1 --2- 172.21.15.3:0/850274744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f262c0a09c0 0x7f262c0a0d90 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f261c0098f0 tx=0x7f261c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df140cc53e5836e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:21:51.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.986+0000 7f2629ffb700 1 -- 172.21.15.3:0/850274744 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c011830 con 0x7f262c0a09c0 2023-10-03T10:21:51.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.986+0000 7f2629ffb700 1 -- 172.21.15.3:0/850274744 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261c011990 con 0x7f262c0a09c0 2023-10-03T10:21:51.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.986+0000 7f2629ffb700 1 -- 172.21.15.3:0/850274744 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c011d20 con 0x7f262c0a09c0 2023-10-03T10:21:51.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.987+0000 7f2631926700 1 -- 172.21.15.3:0/850274744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f262c0a09c0 msgr2=0x7f262c0a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:51.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.987+0000 7f2631926700 1 --2- 172.21.15.3:0/850274744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f262c0a09c0 0x7f262c0a0d90 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f261c0098f0 tx=0x7f261c009c00 comp rx=0 tx=0).stop 2023-10-03T10:21:51.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.987+0000 7f2631926700 1 -- 172.21.15.3:0/850274744 shutdown_connections 2023-10-03T10:21:51.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.987+0000 7f2631926700 1 --2- 172.21.15.3:0/850274744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f262c0a09c0 0x7f262c0a0d90 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:51.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.987+0000 7f2631926700 1 --2- 172.21.15.3:0/850274744 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f262c09d5c0 0x7f262c0a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:51.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.987+0000 7f2631926700 1 -- 172.21.15.3:0/850274744 >> 172.21.15.3:0/850274744 conn(0x7f262c0a2fe0 msgr2=0x7f262c0a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:21:51.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.987+0000 7f2631926700 1 -- 172.21.15.3:0/850274744 shutdown_connections 2023-10-03T10:21:51.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.987+0000 7f2631926700 1 -- 172.21.15.3:0/850274744 wait complete. 2023-10-03T10:21:51.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.988+0000 7f2631926700 1 Processor -- start 2023-10-03T10:21:51.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.988+0000 7f2631926700 1 -- start start 2023-10-03T10:21:51.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.988+0000 7f2631926700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f262c09d5c0 0x7f262c11e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:51.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.988+0000 7f2631926700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f262c133920 0x7f262c11e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:51.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.988+0000 7f2631926700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f262c122070 0x7f262c11ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:51.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.988+0000 7f2631926700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262c120f20 con 0x7f262c122070 2023-10-03T10:21:51.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.988+0000 7f2631926700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f262c11f500 con 0x7f262c133920 2023-10-03T10:21:51.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.988+0000 7f2631926700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f262c121970 con 0x7f262c09d5c0 2023-10-03T10:21:51.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.989+0000 7f262affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f262c09d5c0 0x7f262c11e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:51.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.989+0000 7f262a7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f262c133920 0x7f262c11e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:51.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.989+0000 7f262affd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f262c09d5c0 0x7f262c11e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55022/0 (socket says 172.21.15.3:55022) 2023-10-03T10:21:51.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.989+0000 7f262affd700 1 -- 172.21.15.3:0/153652901 learned_addr learned my addr 172.21.15.3:0/153652901 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:21:51.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.989+0000 7f262a7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f262c133920 0x7f262c11e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59824/0 (socket says 172.21.15.3:59824) 2023-10-03T10:21:51.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.989+0000 7f262b7fe700 1 --2- 172.21.15.3:0/153652901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f262c122070 0x7f262c11ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:51.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.989+0000 7f262affd700 1 -- 172.21.15.3:0/153652901 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f262c133920 msgr2=0x7f262c11e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:51.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.989+0000 7f262affd700 1 --2- 172.21.15.3:0/153652901 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f262c133920 0x7f262c11e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:51.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.989+0000 7f262affd700 1 -- 172.21.15.3:0/153652901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f262c122070 msgr2=0x7f262c11ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:51.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.989+0000 7f262affd700 1 --2- 172.21.15.3:0/153652901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f262c122070 0x7f262c11ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:51.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.989+0000 7f262affd700 1 -- 172.21.15.3:0/153652901 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26200094a0 con 0x7f262c09d5c0 2023-10-03T10:21:51.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.989+0000 7f262b7fe700 1 --2- 172.21.15.3:0/153652901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f262c122070 0x7f262c11ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:21:51.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.989+0000 7f262affd700 1 --2- 172.21.15.3:0/153652901 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f262c09d5c0 0x7f262c11e050 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f261c009c40 tx=0x7f261c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:21:51.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.989+0000 7f2613fff700 1 -- 172.21.15.3:0/153652901 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c004340 con 0x7f262c09d5c0 2023-10-03T10:21:51.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.989+0000 7f2613fff700 1 -- 172.21.15.3:0/153652901 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261c01f070 con 0x7f262c09d5c0 2023-10-03T10:21:51.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.989+0000 7f2613fff700 1 -- 172.21.15.3:0/153652901 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c0036a0 con 0x7f262c09d5c0 2023-10-03T10:21:51.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.989+0000 7f2631926700 1 -- 172.21.15.3:0/153652901 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f261c009550 con 0x7f262c09d5c0 2023-10-03T10:21:51.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.989+0000 7f2631926700 1 -- 172.21.15.3:0/153652901 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f262c120ab0 con 0x7f262c09d5c0 2023-10-03T10:21:51.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.990+0000 7f2613fff700 1 -- 172.21.15.3:0/153652901 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f261c021030 con 0x7f262c09d5c0 2023-10-03T10:21:51.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.990+0000 7f2613fff700 1 -- 172.21.15.3:0/153652901 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f261c040ba0 con 0x7f262c09d5c0 2023-10-03T10:21:51.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.991+0000 7f2631926700 1 -- 172.21.15.3:0/153652901 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f262c09cec0 con 0x7f262c09d5c0 2023-10-03T10:21:51.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:51.996+0000 7f2613fff700 1 -- 172.21.15.3:0/153652901 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f261c0199a0 con 0x7f262c09d5c0 2023-10-03T10:21:52.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.199+0000 7f2631926700 1 -- 172.21.15.3:0/153652901 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f262c121ab0 con 0x7f262c09d5c0 2023-10-03T10:21:52.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.200+0000 7f2613fff700 1 -- 172.21.15.3:0/153652901 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7f261c01b430 con 0x7f262c09d5c0 2023-10-03T10:21:52.200 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-03T10:21:52.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.202+0000 7f2631926700 1 -- 172.21.15.3:0/153652901 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f262c09d5c0 msgr2=0x7f262c11e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:52.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.202+0000 7f2631926700 1 --2- 172.21.15.3:0/153652901 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f262c09d5c0 0x7f262c11e050 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f261c009c40 tx=0x7f261c004750 comp rx=0 tx=0).stop 2023-10-03T10:21:52.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.202+0000 7f2631926700 1 -- 172.21.15.3:0/153652901 shutdown_connections 2023-10-03T10:21:52.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.202+0000 7f2631926700 1 --2- 172.21.15.3:0/153652901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f262c122070 0x7f262c11ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:52.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.202+0000 7f2631926700 1 --2- 172.21.15.3:0/153652901 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f262c09d5c0 0x7f262c11e050 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:52.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.202+0000 7f2631926700 1 --2- 172.21.15.3:0/153652901 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f262c133920 0x7f262c11e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:52.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.202+0000 7f2631926700 1 -- 172.21.15.3:0/153652901 >> 172.21.15.3:0/153652901 conn(0x7f262c0a2fe0 msgr2=0x7f262c132e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:21:52.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.202+0000 7f2631926700 1 -- 172.21.15.3:0/153652901 shutdown_connections 2023-10-03T10:21:52.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.202+0000 7f2631926700 1 -- 172.21.15.3:0/153652901 wait complete. 2023-10-03T10:21:52.216 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-03T10:21:52.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.356+0000 7fcabf2d1700 1 Processor -- start 2023-10-03T10:21:52.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.357+0000 7fcabf2d1700 1 -- start start 2023-10-03T10:21:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.357+0000 7fcabf2d1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcab8131b30 0x7fcab8131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.357+0000 7fcabf2d1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcab809e850 0x7fcab809ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:52.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.357+0000 7fcabf2d1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab80a01c0 con 0x7fcab809f1c0 2023-10-03T10:21:52.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.357+0000 7fcabf2d1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab809b560 con 0x7fcab809e850 2023-10-03T10:21:52.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.357+0000 7fcabf2d1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab809a260 con 0x7fcab8131b30 2023-10-03T10:21:52.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.357+0000 7fcabc86c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcab809e850 0x7fcab809ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:52.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.357+0000 7fcabd06d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcab8131b30 0x7fcab8131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:52.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.357+0000 7fcabc86c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcab809e850 0x7fcab809ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42594/0 (socket says 172.21.15.3:42594) 2023-10-03T10:21:52.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.357+0000 7fcabd06d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcab8131b30 0x7fcab8131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59836/0 (socket says 172.21.15.3:59836) 2023-10-03T10:21:52.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.357+0000 7fcabc86c700 1 -- 172.21.15.3:0/3864859958 learned_addr learned my addr 172.21.15.3:0/3864859958 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:21:52.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.357+0000 7fcabd86e700 1 --1- 172.21.15.3:0/3864859958 >> v1:172.21.15.3:6789/0 conn(0x7fcab809f1c0 0x7fcab81389c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52900/0 (socket says 172.21.15.3:52900) 2023-10-03T10:21:52.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.358+0000 7fcabd06d700 1 -- 172.21.15.3:0/3864859958 >> v2:172.21.15.3:3300/0 conn(0x7fcab809e850 msgr2=0x7fcab809ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:21:52.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.358+0000 7fcabd06d700 1 --2- 172.21.15.3:0/3864859958 >> v2:172.21.15.3:3300/0 conn(0x7fcab809e850 0x7fcab809ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:52.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.358+0000 7fcabd06d700 1 -- 172.21.15.3:0/3864859958 >> v1:172.21.15.3:6789/0 conn(0x7fcab809f1c0 legacy=0x7fcab81389c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:52.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.358+0000 7fcabd06d700 1 -- 172.21.15.3:0/3864859958 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab80a07a0 con 0x7fcab8131b30 2023-10-03T10:21:52.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.358+0000 7fcabc86c700 1 --2- 172.21.15.3:0/3864859958 >> v2:172.21.15.3:3300/0 conn(0x7fcab809e850 0x7fcab809ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:21:52.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.358+0000 7fcabd06d700 1 --2- 172.21.15.3:0/3864859958 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcab8131b30 0x7fcab8131f00 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009840 tx=0x7fcab4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e53ddf426d254e01 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:21:52.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.358+0000 7fcaabfff700 1 -- 172.21.15.3:0/3864859958 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab4011830 con 0x7fcab8131b30 2023-10-03T10:21:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.358+0000 7fcaabfff700 1 -- 172.21.15.3:0/3864859958 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab4011990 con 0x7fcab8131b30 2023-10-03T10:21:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.358+0000 7fcaabfff700 1 -- 172.21.15.3:0/3864859958 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab4011d20 con 0x7fcab8131b30 2023-10-03T10:21:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.359+0000 7fcabf2d1700 1 -- 172.21.15.3:0/3864859958 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcab8131b30 msgr2=0x7fcab8131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.359+0000 7fcabf2d1700 1 --2- 172.21.15.3:0/3864859958 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcab8131b30 0x7fcab8131f00 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009840 tx=0x7fcab4009b50 comp rx=0 tx=0).stop 2023-10-03T10:21:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.359+0000 7fcabf2d1700 1 -- 172.21.15.3:0/3864859958 shutdown_connections 2023-10-03T10:21:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.359+0000 7fcabf2d1700 1 --2- 172.21.15.3:0/3864859958 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcab8131b30 0x7fcab8131f00 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.359+0000 7fcabf2d1700 1 --2- 172.21.15.3:0/3864859958 >> v2:172.21.15.3:3300/0 conn(0x7fcab809e850 0x7fcab809ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:52.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.359+0000 7fcabf2d1700 1 -- 172.21.15.3:0/3864859958 >> 172.21.15.3:0/3864859958 conn(0x7fcab80a4130 msgr2=0x7fcab80a4530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:21:52.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.359+0000 7fcabf2d1700 1 -- 172.21.15.3:0/3864859958 shutdown_connections 2023-10-03T10:21:52.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.359+0000 7fcabf2d1700 1 -- 172.21.15.3:0/3864859958 wait complete. 2023-10-03T10:21:52.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.360+0000 7fcabf2d1700 1 Processor -- start 2023-10-03T10:21:52.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.360+0000 7fcabf2d1700 1 -- start start 2023-10-03T10:21:52.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.360+0000 7fcabf2d1700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcab809e850 0x7fcab811e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:52.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.361+0000 7fcabf2d1700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcab809f1c0 0x7fcab811e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:52.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.361+0000 7fcabf2d1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab8131b30 0x7fcab811ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:52.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.361+0000 7fcabf2d1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab8121ca0 con 0x7fcab8131b30 2023-10-03T10:21:52.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.361+0000 7fcabf2d1700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab8121070 con 0x7fcab809e850 2023-10-03T10:21:52.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.361+0000 7fcabf2d1700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcab806d890 con 0x7fcab809f1c0 2023-10-03T10:21:52.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.361+0000 7fcabd06d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcab809e850 0x7fcab811e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:52.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.361+0000 7fcabc86c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcab809f1c0 0x7fcab811e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:52.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.361+0000 7fcabc86c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcab809f1c0 0x7fcab811e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55038/0 (socket says 172.21.15.3:55038) 2023-10-03T10:21:52.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.361+0000 7fcabc86c700 1 -- 172.21.15.3:0/1454927506 learned_addr learned my addr 172.21.15.3:0/1454927506 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:21:52.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.361+0000 7fcabd06d700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcab809e850 0x7fcab811e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:59846/0 (socket says 172.21.15.3:59846) 2023-10-03T10:21:52.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.361+0000 7fcabd86e700 1 --2- 172.21.15.3:0/1454927506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab8131b30 0x7fcab811ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:52.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.361+0000 7fcabc86c700 1 -- 172.21.15.3:0/1454927506 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcab809e850 msgr2=0x7fcab811e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:52.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.361+0000 7fcabc86c700 1 --2- 172.21.15.3:0/1454927506 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcab809e850 0x7fcab811e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:52.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.361+0000 7fcabc86c700 1 -- 172.21.15.3:0/1454927506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab8131b30 msgr2=0x7fcab811ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:52.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.361+0000 7fcabc86c700 1 --2- 172.21.15.3:0/1454927506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab8131b30 0x7fcab811ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:52.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.361+0000 7fcabc86c700 1 -- 172.21.15.3:0/1454927506 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaac009550 con 0x7fcab809f1c0 2023-10-03T10:21:52.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.361+0000 7fcabd86e700 1 --2- 172.21.15.3:0/1454927506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab8131b30 0x7fcab811ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:21:52.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.361+0000 7fcabd06d700 1 --2- 172.21.15.3:0/1454927506 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcab809e850 0x7fcab811e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:21:52.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.361+0000 7fcabc86c700 1 --2- 172.21.15.3:0/1454927506 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcab809f1c0 0x7fcab811e750 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00ebe0 tx=0x7fcaac00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:21:52.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.361+0000 7fcaa9ffb700 1 -- 172.21.15.3:0/1454927506 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac00c850 con 0x7fcab809f1c0 2023-10-03T10:21:52.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.361+0000 7fcaa9ffb700 1 -- 172.21.15.3:0/1454927506 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaac00c9b0 con 0x7fcab809f1c0 2023-10-03T10:21:52.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.361+0000 7fcaa9ffb700 1 -- 172.21.15.3:0/1454927506 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac00cd60 con 0x7fcab809f1c0 2023-10-03T10:21:52.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.361+0000 7fcabf2d1700 1 -- 172.21.15.3:0/1454927506 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab40094a0 con 0x7fcab809f1c0 2023-10-03T10:21:52.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.362+0000 7fcabf2d1700 1 -- 172.21.15.3:0/1454927506 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab809da10 con 0x7fcab809f1c0 2023-10-03T10:21:52.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.363+0000 7fcabf2d1700 1 -- 172.21.15.3:0/1454927506 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab8121820 con 0x7fcab809f1c0 2023-10-03T10:21:52.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.363+0000 7fcaa9ffb700 1 -- 172.21.15.3:0/1454927506 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcaac013470 con 0x7fcab809f1c0 2023-10-03T10:21:52.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.363+0000 7fcaa9ffb700 1 -- 172.21.15.3:0/1454927506 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fcaac03fef0 con 0x7fcab809f1c0 2023-10-03T10:21:52.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.368+0000 7fcaa9ffb700 1 -- 172.21.15.3:0/1454927506 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcaac013720 con 0x7fcab809f1c0 2023-10-03T10:21:52.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.570+0000 7fcabf2d1700 1 -- 172.21.15.3:0/1454927506 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcab8121820 con 0x7fcab809f1c0 2023-10-03T10:21:52.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.571+0000 7fcaa9ffb700 1 -- 172.21.15.3:0/1454927506 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcab8121820 con 0x7fcab809f1c0 2023-10-03T10:21:52.571 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-10-03T10:21:52.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.573+0000 7fcabf2d1700 1 -- 172.21.15.3:0/1454927506 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcab809f1c0 msgr2=0x7fcab811e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:52.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.573+0000 7fcabf2d1700 1 --2- 172.21.15.3:0/1454927506 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcab809f1c0 0x7fcab811e750 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00ebe0 tx=0x7fcaac00eef0 comp rx=0 tx=0).stop 2023-10-03T10:21:52.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.573+0000 7fcabf2d1700 1 -- 172.21.15.3:0/1454927506 shutdown_connections 2023-10-03T10:21:52.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.573+0000 7fcabf2d1700 1 --2- 172.21.15.3:0/1454927506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab8131b30 0x7fcab811ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:52.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.573+0000 7fcabf2d1700 1 --2- 172.21.15.3:0/1454927506 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcab809e850 0x7fcab811e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:52.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.573+0000 7fcabf2d1700 1 --2- 172.21.15.3:0/1454927506 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fcab809f1c0 0x7fcab811e750 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:52.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.573+0000 7fcabf2d1700 1 -- 172.21.15.3:0/1454927506 >> 172.21.15.3:0/1454927506 conn(0x7fcab80a4130 msgr2=0x7fcab8071890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:21:52.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.573+0000 7fcabf2d1700 1 -- 172.21.15.3:0/1454927506 shutdown_connections 2023-10-03T10:21:52.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.573+0000 7fcabf2d1700 1 -- 172.21.15.3:0/1454927506 wait complete. 2023-10-03T10:21:52.587 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-03T10:21:52.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.727+0000 7fb26174c700 1 Processor -- start 2023-10-03T10:21:52.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.727+0000 7fb26174c700 1 -- start start 2023-10-03T10:21:52.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.727+0000 7fb26174c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb25c09a520 0x7fb25c138d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:52.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.727+0000 7fb26174c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb25c1392a0 0x7fb25c13b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:52.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.727+0000 7fb26174c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb25c0a0030 con 0x7fb25c099a40 2023-10-03T10:21:52.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.727+0000 7fb26174c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb25c09a910 con 0x7fb25c1392a0 2023-10-03T10:21:52.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.727+0000 7fb26174c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c09f360 con 0x7fb25c09a520 2023-10-03T10:21:52.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.728+0000 7fb25affd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb25c099a40 0x7fb25c099e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52906/0 (socket says 172.21.15.3:52906) 2023-10-03T10:21:52.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.728+0000 7fb25a7fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb25c09a520 0x7fb25c138d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:52.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.728+0000 7fb25affd700 1 -- 172.21.15.3:0/2934714843 learned_addr learned my addr 172.21.15.3:0/2934714843 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:21:52.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.728+0000 7fb25b7fe700 1 --2- 172.21.15.3:0/2934714843 >> v2:172.21.15.3:3300/0 conn(0x7fb25c1392a0 0x7fb25c13b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:52.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.728+0000 7fb25a7fc700 1 -- 172.21.15.3:0/2934714843 >> v2:172.21.15.3:3300/0 conn(0x7fb25c1392a0 msgr2=0x7fb25c13b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:21:52.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.728+0000 7fb25a7fc700 1 --2- 172.21.15.3:0/2934714843 >> v2:172.21.15.3:3300/0 conn(0x7fb25c1392a0 0x7fb25c13b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:52.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.728+0000 7fb25a7fc700 1 -- 172.21.15.3:0/2934714843 >> v1:172.21.15.3:6789/0 conn(0x7fb25c099a40 legacy=0x7fb25c099e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:52.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.728+0000 7fb25a7fc700 1 -- 172.21.15.3:0/2934714843 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25c06c950 con 0x7fb25c09a520 2023-10-03T10:21:52.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.728+0000 7fb25b7fe700 1 --2- 172.21.15.3:0/2934714843 >> v2:172.21.15.3:3300/0 conn(0x7fb25c1392a0 0x7fb25c13b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:21:52.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.728+0000 7fb25a7fc700 1 --2- 172.21.15.3:0/2934714843 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb25c09a520 0x7fb25c138d60 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fb244009840 tx=0x7fb244009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2fc794bef0fb8c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:21:52.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.728+0000 7fb259ffb700 1 -- 172.21.15.3:0/2934714843 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb244011830 con 0x7fb25c09a520 2023-10-03T10:21:52.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.728+0000 7fb259ffb700 1 -- 172.21.15.3:0/2934714843 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb244011990 con 0x7fb25c09a520 2023-10-03T10:21:52.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.728+0000 7fb259ffb700 1 -- 172.21.15.3:0/2934714843 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb244011d20 con 0x7fb25c09a520 2023-10-03T10:21:52.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.729+0000 7fb26174c700 1 -- 172.21.15.3:0/2934714843 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb25c09a520 msgr2=0x7fb25c138d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:52.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.729+0000 7fb26174c700 1 --2- 172.21.15.3:0/2934714843 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb25c09a520 0x7fb25c138d60 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fb244009840 tx=0x7fb244009b50 comp rx=0 tx=0).stop 2023-10-03T10:21:52.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.729+0000 7fb26174c700 1 -- 172.21.15.3:0/2934714843 shutdown_connections 2023-10-03T10:21:52.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.729+0000 7fb26174c700 1 --2- 172.21.15.3:0/2934714843 >> v2:172.21.15.3:3300/0 conn(0x7fb25c1392a0 0x7fb25c13b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:52.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.729+0000 7fb26174c700 1 --2- 172.21.15.3:0/2934714843 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb25c09a520 0x7fb25c138d60 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:52.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.729+0000 7fb26174c700 1 -- 172.21.15.3:0/2934714843 >> 172.21.15.3:0/2934714843 conn(0x7fb25c0a3f50 msgr2=0x7fb25c0a4350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:21:52.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.729+0000 7fb26174c700 1 -- 172.21.15.3:0/2934714843 shutdown_connections 2023-10-03T10:21:52.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.729+0000 7fb26174c700 1 -- 172.21.15.3:0/2934714843 wait complete. 2023-10-03T10:21:52.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.730+0000 7fb26174c700 1 Processor -- start 2023-10-03T10:21:52.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.730+0000 7fb26174c700 1 -- start start 2023-10-03T10:21:52.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.731+0000 7fb26174c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb25c099a40 0x7fb25c11ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:52.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.731+0000 7fb26174c700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb25c09a520 0x7fb25c11ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:52.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.731+0000 7fb26174c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb25c1392a0 0x7fb25c11f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:52.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.731+0000 7fb26174c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c06cb70 con 0x7fb25c099a40 2023-10-03T10:21:52.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.731+0000 7fb26174c700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c120c10 con 0x7fb25c09a520 2023-10-03T10:21:52.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.731+0000 7fb26174c700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c134510 con 0x7fb25c1392a0 2023-10-03T10:21:52.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.731+0000 7fb25b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb25c1392a0 0x7fb25c11f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:52.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.731+0000 7fb25a7fc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb25c09a520 0x7fb25c11ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:52.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.731+0000 7fb25b7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb25c1392a0 0x7fb25c11f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55058/0 (socket says 172.21.15.3:55058) 2023-10-03T10:21:52.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.731+0000 7fb25b7fe700 1 -- 172.21.15.3:0/4231402066 learned_addr learned my addr 172.21.15.3:0/4231402066 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:21:52.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.731+0000 7fb25affd700 1 --2- 172.21.15.3:0/4231402066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb25c099a40 0x7fb25c11ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:52.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.731+0000 7fb25b7fe700 1 -- 172.21.15.3:0/4231402066 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb25c09a520 msgr2=0x7fb25c11ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:52.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.731+0000 7fb25b7fe700 1 --2- 172.21.15.3:0/4231402066 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb25c09a520 0x7fb25c11ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.731+0000 7fb25b7fe700 1 -- 172.21.15.3:0/4231402066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb25c099a40 msgr2=0x7fb25c11ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.731+0000 7fb25b7fe700 1 --2- 172.21.15.3:0/4231402066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb25c099a40 0x7fb25c11ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.731+0000 7fb25b7fe700 1 -- 172.21.15.3:0/4231402066 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb254009a70 con 0x7fb25c1392a0 2023-10-03T10:21:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.731+0000 7fb25a7fc700 1 --2- 172.21.15.3:0/4231402066 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb25c09a520 0x7fb25c11ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:21:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.731+0000 7fb25b7fe700 1 --2- 172.21.15.3:0/4231402066 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb25c1392a0 0x7fb25c11f480 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fb254009f00 tx=0x7fb25400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:21:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.731+0000 7fb2537fe700 1 -- 172.21.15.3:0/4231402066 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb254009d70 con 0x7fb25c1392a0 2023-10-03T10:21:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.732+0000 7fb2537fe700 1 -- 172.21.15.3:0/4231402066 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25400f040 con 0x7fb25c1392a0 2023-10-03T10:21:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.732+0000 7fb2537fe700 1 -- 172.21.15.3:0/4231402066 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25400cd70 con 0x7fb25c1392a0 2023-10-03T10:21:52.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.732+0000 7fb26174c700 1 -- 172.21.15.3:0/4231402066 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2440094a0 con 0x7fb25c1392a0 2023-10-03T10:21:52.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.732+0000 7fb26174c700 1 -- 172.21.15.3:0/4231402066 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb25c120de0 con 0x7fb25c1392a0 2023-10-03T10:21:52.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.732+0000 7fb2537fe700 1 -- 172.21.15.3:0/4231402066 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb254014480 con 0x7fb25c1392a0 2023-10-03T10:21:52.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.732+0000 7fb2537fe700 1 -- 172.21.15.3:0/4231402066 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb25403fd00 con 0x7fb25c1392a0 2023-10-03T10:21:52.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.733+0000 7fb26174c700 1 -- 172.21.15.3:0/4231402066 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb25c04f2a0 con 0x7fb25c1392a0 2023-10-03T10:21:52.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.738+0000 7fb2537fe700 1 -- 172.21.15.3:0/4231402066 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb254012930 con 0x7fb25c1392a0 2023-10-03T10:21:52.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.939+0000 7fb26174c700 1 -- 172.21.15.3:0/4231402066 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb25c0727f0 con 0x7fb25c1392a0 2023-10-03T10:21:52.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.939+0000 7fb2537fe700 1 -- 172.21.15.3:0/4231402066 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb25402a0a0 con 0x7fb25c1392a0 2023-10-03T10:21:52.939 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-10-03T10:21:52.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.941+0000 7fb26174c700 1 -- 172.21.15.3:0/4231402066 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb25c1392a0 msgr2=0x7fb25c11f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:52.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.941+0000 7fb26174c700 1 --2- 172.21.15.3:0/4231402066 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb25c1392a0 0x7fb25c11f480 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fb254009f00 tx=0x7fb25400c3a0 comp rx=0 tx=0).stop 2023-10-03T10:21:52.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.941+0000 7fb26174c700 1 -- 172.21.15.3:0/4231402066 shutdown_connections 2023-10-03T10:21:52.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.941+0000 7fb26174c700 1 --2- 172.21.15.3:0/4231402066 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb25c1392a0 0x7fb25c11f480 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:52.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.941+0000 7fb26174c700 1 --2- 172.21.15.3:0/4231402066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb25c099a40 0x7fb25c11ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:52.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.941+0000 7fb26174c700 1 --2- 172.21.15.3:0/4231402066 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb25c09a520 0x7fb25c11ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:52.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.941+0000 7fb26174c700 1 -- 172.21.15.3:0/4231402066 >> 172.21.15.3:0/4231402066 conn(0x7fb25c0a3f50 msgr2=0x7fb25c1382c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:21:52.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.941+0000 7fb26174c700 1 -- 172.21.15.3:0/4231402066 shutdown_connections 2023-10-03T10:21:52.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:52.941+0000 7fb26174c700 1 -- 172.21.15.3:0/4231402066 wait complete. 2023-10-03T10:21:52.955 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-03T10:21:53.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.092+0000 7ff663310700 1 Processor -- start 2023-10-03T10:21:53.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.093+0000 7ff663310700 1 -- start start 2023-10-03T10:21:53.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.093+0000 7ff663310700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff65c0a0a30 0x7ff65c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:53.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.093+0000 7ff663310700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff65c134170 0x7ff65c13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:53.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.093+0000 7ff663310700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff65c09edd0 con 0x7ff65c0a1340 2023-10-03T10:21:53.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.093+0000 7ff663310700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff65c09e1f0 con 0x7ff65c134170 2023-10-03T10:21:53.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.093+0000 7ff663310700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c09dc50 con 0x7ff65c0a0a30 2023-10-03T10:21:53.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.093+0000 7ff6610ac700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff65c0a0a30 0x7ff65c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:53.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.093+0000 7ff6618ad700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff65c134170 0x7ff65c13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:53.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.093+0000 7ff6610ac700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff65c0a0a30 0x7ff65c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55062/0 (socket says 172.21.15.3:55062) 2023-10-03T10:21:53.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.093+0000 7ff6610ac700 1 -- 172.21.15.3:0/1335610630 learned_addr learned my addr 172.21.15.3:0/1335610630 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:21:53.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.094+0000 7ff6610ac700 1 -- 172.21.15.3:0/1335610630 >> v2:172.21.15.3:3300/0 conn(0x7ff65c134170 msgr2=0x7ff65c13bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:21:53.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.094+0000 7ff6610ac700 1 --2- 172.21.15.3:0/1335610630 >> v2:172.21.15.3:3300/0 conn(0x7ff65c134170 0x7ff65c13bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:53.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.094+0000 7ff6610ac700 1 -- 172.21.15.3:0/1335610630 >> v1:172.21.15.3:6789/0 conn(0x7ff65c0a1340 legacy=0x7ff65c133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:53.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.094+0000 7ff6610ac700 1 -- 172.21.15.3:0/1335610630 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65c0a1fe0 con 0x7ff65c0a0a30 2023-10-03T10:21:53.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.094+0000 7ff6618ad700 1 --2- 172.21.15.3:0/1335610630 >> v2:172.21.15.3:3300/0 conn(0x7ff65c134170 0x7ff65c13bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:21:53.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.094+0000 7ff6610ac700 1 --2- 172.21.15.3:0/1335610630 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff65c0a0a30 0x7ff65c0a0e00 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7ff6580098f0 tx=0x7ff658009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e89f6446ce7a4df server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:21:53.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.094+0000 7ff653fff700 1 -- 172.21.15.3:0/1335610630 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff658011830 con 0x7ff65c0a0a30 2023-10-03T10:21:53.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.094+0000 7ff653fff700 1 -- 172.21.15.3:0/1335610630 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff658011990 con 0x7ff65c0a0a30 2023-10-03T10:21:53.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.094+0000 7ff653fff700 1 -- 172.21.15.3:0/1335610630 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff658011d20 con 0x7ff65c0a0a30 2023-10-03T10:21:53.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.095+0000 7ff663310700 1 -- 172.21.15.3:0/1335610630 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff65c0a0a30 msgr2=0x7ff65c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:53.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.095+0000 7ff663310700 1 --2- 172.21.15.3:0/1335610630 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff65c0a0a30 0x7ff65c0a0e00 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7ff6580098f0 tx=0x7ff658009c00 comp rx=0 tx=0).stop 2023-10-03T10:21:53.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.095+0000 7ff663310700 1 -- 172.21.15.3:0/1335610630 shutdown_connections 2023-10-03T10:21:53.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.095+0000 7ff663310700 1 --2- 172.21.15.3:0/1335610630 >> v2:172.21.15.3:3300/0 conn(0x7ff65c134170 0x7ff65c13bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:53.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.095+0000 7ff663310700 1 --2- 172.21.15.3:0/1335610630 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff65c0a0a30 0x7ff65c0a0e00 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:53.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.095+0000 7ff663310700 1 -- 172.21.15.3:0/1335610630 >> 172.21.15.3:0/1335610630 conn(0x7ff65c0a6550 msgr2=0x7ff65c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:21:53.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.095+0000 7ff663310700 1 -- 172.21.15.3:0/1335610630 shutdown_connections 2023-10-03T10:21:53.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.095+0000 7ff663310700 1 -- 172.21.15.3:0/1335610630 wait complete. 2023-10-03T10:21:53.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.096+0000 7ff663310700 1 Processor -- start 2023-10-03T10:21:53.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.096+0000 7ff663310700 1 -- start start 2023-10-03T10:21:53.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.096+0000 7ff663310700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff65c0a0a30 0x7ff65c122900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:53.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.096+0000 7ff663310700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff65c0a1340 0x7ff65c122e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:53.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.096+0000 7ff663310700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff65c134170 0x7ff65c123380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:53.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.096+0000 7ff663310700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c124690 con 0x7ff65c134170 2023-10-03T10:21:53.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.096+0000 7ff663310700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c0a4f40 con 0x7ff65c0a1340 2023-10-03T10:21:53.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.096+0000 7ff663310700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c126420 con 0x7ff65c0a0a30 2023-10-03T10:21:53.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.096+0000 7ff6610ac700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff65c0a0a30 0x7ff65c122900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:53.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.096+0000 7ff6610ac700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff65c0a0a30 0x7ff65c122900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:55078/0 (socket says 172.21.15.3:55078) 2023-10-03T10:21:53.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.096+0000 7ff6608ab700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff65c0a1340 0x7ff65c122e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:53.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.096+0000 7ff6610ac700 1 -- 172.21.15.3:0/4024505138 learned_addr learned my addr 172.21.15.3:0/4024505138 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:21:53.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.096+0000 7ff6618ad700 1 --2- 172.21.15.3:0/4024505138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff65c134170 0x7ff65c123380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:53.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.097+0000 7ff6610ac700 1 -- 172.21.15.3:0/4024505138 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff65c0a1340 msgr2=0x7ff65c122e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:53.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.097+0000 7ff6610ac700 1 --2- 172.21.15.3:0/4024505138 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff65c0a1340 0x7ff65c122e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:53.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.097+0000 7ff6610ac700 1 -- 172.21.15.3:0/4024505138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff65c134170 msgr2=0x7ff65c123380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:53.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.097+0000 7ff6610ac700 1 --2- 172.21.15.3:0/4024505138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff65c134170 0x7ff65c123380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:53.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.097+0000 7ff6610ac700 1 -- 172.21.15.3:0/4024505138 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff654009a70 con 0x7ff65c0a0a30 2023-10-03T10:21:53.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.097+0000 7ff6610ac700 1 --2- 172.21.15.3:0/4024505138 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff65c0a0a30 0x7ff65c122900 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7ff658000c00 tx=0x7ff658004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:21:53.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.097+0000 7ff651ffb700 1 -- 172.21.15.3:0/4024505138 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6580043c0 con 0x7ff65c0a0a30 2023-10-03T10:21:53.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.097+0000 7ff651ffb700 1 -- 172.21.15.3:0/4024505138 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65801f070 con 0x7ff65c0a0a30 2023-10-03T10:21:53.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.097+0000 7ff663310700 1 -- 172.21.15.3:0/4024505138 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff658009550 con 0x7ff65c0a0a30 2023-10-03T10:21:53.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.097+0000 7ff651ffb700 1 -- 172.21.15.3:0/4024505138 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6580038c0 con 0x7ff65c0a0a30 2023-10-03T10:21:53.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.097+0000 7ff663310700 1 -- 172.21.15.3:0/4024505138 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff65c133420 con 0x7ff65c0a0a30 2023-10-03T10:21:53.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.098+0000 7ff651ffb700 1 -- 172.21.15.3:0/4024505138 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff658021030 con 0x7ff65c0a0a30 2023-10-03T10:21:53.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.098+0000 7ff651ffb700 1 -- 172.21.15.3:0/4024505138 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ff658040900 con 0x7ff65c0a0a30 2023-10-03T10:21:53.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.098+0000 7ff663310700 1 -- 172.21.15.3:0/4024505138 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff65c0a1d40 con 0x7ff65c0a0a30 2023-10-03T10:21:53.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.104+0000 7ff651ffb700 1 -- 172.21.15.3:0/4024505138 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff658015080 con 0x7ff65c0a0a30 2023-10-03T10:21:53.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.326+0000 7ff663310700 1 -- 172.21.15.3:0/4024505138 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff65c074ea0 con 0x7ff65c0a0a30 2023-10-03T10:21:53.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.326+0000 7ff651ffb700 1 -- 172.21.15.3:0/4024505138 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 11 entries v40) v1 ==== 81+0+628 (secure 0 0 0) 0x7ff658004560 con 0x7ff65c0a0a30 2023-10-03T10:21:53.327 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/3264604155 2023-10-04T10:13:12.182591+0000 2023-10-03T10:21:53.327 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1911521769 2023-10-04T10:13:12.182591+0000 2023-10-03T10:21:53.327 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/3289486158 2023-10-04T10:13:12.182591+0000 2023-10-03T10:21:53.327 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6816/118296 2023-10-04T10:12:13.305411+0000 2023-10-03T10:21:53.328 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/3246155339 2023-10-04T10:13:12.182591+0000 2023-10-03T10:21:53.328 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2091814084 2023-10-04T10:12:13.305411+0000 2023-10-03T10:21:53.328 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6817/2582564779 2023-10-04T10:12:38.734689+0000 2023-10-03T10:21:53.328 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/3571619151 2023-10-04T10:13:12.182591+0000 2023-10-03T10:21:53.328 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/3919724564 2023-10-04T10:12:13.305411+0000 2023-10-03T10:21:53.328 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6816/127025 2023-10-04T10:13:12.182591+0000 2023-10-03T10:21:53.328 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2059330378 2023-10-04T10:13:12.182591+0000 2023-10-03T10:21:53.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.329+0000 7ff663310700 1 -- 172.21.15.3:0/4024505138 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff65c0a0a30 msgr2=0x7ff65c122900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:53.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.329+0000 7ff663310700 1 --2- 172.21.15.3:0/4024505138 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff65c0a0a30 0x7ff65c122900 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7ff658000c00 tx=0x7ff658004750 comp rx=0 tx=0).stop 2023-10-03T10:21:53.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.329+0000 7ff663310700 1 -- 172.21.15.3:0/4024505138 shutdown_connections 2023-10-03T10:21:53.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.329+0000 7ff663310700 1 --2- 172.21.15.3:0/4024505138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff65c134170 0x7ff65c123380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:53.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.329+0000 7ff663310700 1 --2- 172.21.15.3:0/4024505138 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ff65c0a0a30 0x7ff65c122900 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:53.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.329+0000 7ff663310700 1 --2- 172.21.15.3:0/4024505138 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff65c0a1340 0x7ff65c122e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:53.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.329+0000 7ff663310700 1 -- 172.21.15.3:0/4024505138 >> 172.21.15.3:0/4024505138 conn(0x7ff65c0a6550 msgr2=0x7ff65c09ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:21:53.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.329+0000 7ff663310700 1 -- 172.21.15.3:0/4024505138 shutdown_connections 2023-10-03T10:21:53.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.329+0000 7ff663310700 1 -- 172.21.15.3:0/4024505138 wait complete. 2023-10-03T10:21:53.331 INFO:teuthology.orchestra.run.smithi003.stderr:listed 11 entries 2023-10-03T10:21:53.343 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-10-03T10:21:53.343 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-03T10:21:53.367 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-03T10:21:53.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.504+0000 7ffbc6679700 1 Processor -- start 2023-10-03T10:21:53.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.504+0000 7ffbc6679700 1 -- start start 2023-10-03T10:21:53.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.504+0000 7ffbc6679700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffbc009d6f0 0x7ffbc009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:53.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.505+0000 7ffbc6679700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffbc009e000 0x7ffbc0130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:53.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.505+0000 7ffbc6679700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc009f130 con 0x7ffbc009e410 2023-10-03T10:21:53.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.505+0000 7ffbc6679700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc009f520 con 0x7ffbc009d6f0 2023-10-03T10:21:53.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.505+0000 7ffbc6679700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc009e930 con 0x7ffbc009e000 2023-10-03T10:21:53.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.505+0000 7ffbbf7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffbc009e000 0x7ffbc0130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:53.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.505+0000 7ffbbffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffbc009d6f0 0x7ffbc009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:53.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.505+0000 7ffbc4c16700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ffbc009e410 0x7ffbc0130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37842/0 (socket says 172.21.15.3:37842) 2023-10-03T10:21:53.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.505+0000 7ffbc4c16700 1 -- 172.21.15.3:0/2775139690 learned_addr learned my addr 172.21.15.3:0/2775139690 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:21:53.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.505+0000 7ffbbeffd700 1 -- 172.21.15.3:0/2775139690 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2690335695 0 0) 0x7ffbc009f130 con 0x7ffbc009e410 2023-10-03T10:21:53.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.505+0000 7ffbbf7fe700 1 -- 172.21.15.3:0/2775139690 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffbc009d6f0 msgr2=0x7ffbc009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:21:53.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.505+0000 7ffbbf7fe700 1 --2- 172.21.15.3:0/2775139690 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffbc009d6f0 0x7ffbc009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:53.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.505+0000 7ffbbf7fe700 1 -- 172.21.15.3:0/2775139690 >> v1:172.21.15.3:6789/0 conn(0x7ffbc009e410 legacy=0x7ffbc0130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:53.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.505+0000 7ffbbf7fe700 1 -- 172.21.15.3:0/2775139690 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc009b2d0 con 0x7ffbc009e000 2023-10-03T10:21:53.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.505+0000 7ffbbffff700 1 --2- 172.21.15.3:0/2775139690 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffbc009d6f0 0x7ffbc009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:21:53.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.505+0000 7ffbbf7fe700 1 --2- 172.21.15.3:0/2775139690 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffbc009e000 0x7ffbc0130660 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400bb40 tx=0x7ffbb400be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf61295e0a46f48 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:21:53.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.505+0000 7ffbbeffd700 1 -- 172.21.15.3:0/2775139690 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb400dc30 con 0x7ffbc009e000 2023-10-03T10:21:53.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.505+0000 7ffbbeffd700 1 -- 172.21.15.3:0/2775139690 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb400dd90 con 0x7ffbc009e000 2023-10-03T10:21:53.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.506+0000 7ffbbeffd700 1 -- 172.21.15.3:0/2775139690 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4010690 con 0x7ffbc009e000 2023-10-03T10:21:53.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.506+0000 7ffbc6679700 1 -- 172.21.15.3:0/2775139690 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffbc009e000 msgr2=0x7ffbc0130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:53.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.506+0000 7ffbc6679700 1 --2- 172.21.15.3:0/2775139690 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffbc009e000 0x7ffbc0130660 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400bb40 tx=0x7ffbb400be50 comp rx=0 tx=0).stop 2023-10-03T10:21:53.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.506+0000 7ffbc6679700 1 -- 172.21.15.3:0/2775139690 shutdown_connections 2023-10-03T10:21:53.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.506+0000 7ffbc6679700 1 --2- 172.21.15.3:0/2775139690 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffbc009d6f0 0x7ffbc009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:53.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.506+0000 7ffbc6679700 1 --2- 172.21.15.3:0/2775139690 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffbc009e000 0x7ffbc0130660 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:53.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.506+0000 7ffbc6679700 1 -- 172.21.15.3:0/2775139690 >> 172.21.15.3:0/2775139690 conn(0x7ffbc0066f50 msgr2=0x7ffbc009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:21:53.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.506+0000 7ffbc6679700 1 -- 172.21.15.3:0/2775139690 shutdown_connections 2023-10-03T10:21:53.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.506+0000 7ffbc6679700 1 -- 172.21.15.3:0/2775139690 wait complete. 2023-10-03T10:21:53.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.507+0000 7ffbc6679700 1 Processor -- start 2023-10-03T10:21:53.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.507+0000 7ffbc6679700 1 -- start start 2023-10-03T10:21:53.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.508+0000 7ffbc6679700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffbc009d6f0 0x7ffbc011dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:53.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.508+0000 7ffbc6679700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffbc009e000 0x7ffbc011e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:53.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.508+0000 7ffbc6679700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbc009e410 0x7ffbc011e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:53.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.508+0000 7ffbc6679700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc01218b0 con 0x7ffbc009e410 2023-10-03T10:21:53.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.508+0000 7ffbc6679700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc0120c80 con 0x7ffbc009d6f0 2023-10-03T10:21:53.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.508+0000 7ffbc6679700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc0122250 con 0x7ffbc009e000 2023-10-03T10:21:53.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.508+0000 7ffbbf7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffbc009e000 0x7ffbc011e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:53.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.508+0000 7ffbc4c16700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbc009e410 0x7ffbc011e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:53.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.508+0000 7ffbbf7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffbc009e000 0x7ffbc011e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:59716/0 (socket says 172.21.15.3:59716) 2023-10-03T10:21:53.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.508+0000 7ffbbf7fe700 1 -- 172.21.15.3:0/616362356 learned_addr learned my addr 172.21.15.3:0/616362356 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:21:53.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.508+0000 7ffbbffff700 1 --2- 172.21.15.3:0/616362356 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffbc009d6f0 0x7ffbc011dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:53.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.508+0000 7ffbbf7fe700 1 -- 172.21.15.3:0/616362356 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffbc009d6f0 msgr2=0x7ffbc011dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:53.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.508+0000 7ffbbf7fe700 1 --2- 172.21.15.3:0/616362356 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffbc009d6f0 0x7ffbc011dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:53.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.508+0000 7ffbbf7fe700 1 -- 172.21.15.3:0/616362356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbc009e410 msgr2=0x7ffbc011e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:53.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.508+0000 7ffbbf7fe700 1 --2- 172.21.15.3:0/616362356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbc009e410 0x7ffbc011e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:53.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.508+0000 7ffbbf7fe700 1 -- 172.21.15.3:0/616362356 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb00094a0 con 0x7ffbc009e000 2023-10-03T10:21:53.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.508+0000 7ffbc4c16700 1 --2- 172.21.15.3:0/616362356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbc009e410 0x7ffbc011e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:21:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.509+0000 7ffbbf7fe700 1 --2- 172.21.15.3:0/616362356 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffbc009e000 0x7ffbc011e2d0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4015000 tx=0x7ffbb4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:21:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.509+0000 7ffbbcff9700 1 -- 172.21.15.3:0/616362356 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb401d040 con 0x7ffbc009e000 2023-10-03T10:21:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.509+0000 7ffbbcff9700 1 -- 172.21.15.3:0/616362356 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb4024680 con 0x7ffbc009e000 2023-10-03T10:21:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.509+0000 7ffbc6679700 1 -- 172.21.15.3:0/616362356 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb400b740 con 0x7ffbc009e000 2023-10-03T10:21:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.509+0000 7ffbbcff9700 1 -- 172.21.15.3:0/616362356 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4024a30 con 0x7ffbc009e000 2023-10-03T10:21:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.509+0000 7ffbc6679700 1 -- 172.21.15.3:0/616362356 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc009c700 con 0x7ffbc009e000 2023-10-03T10:21:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.509+0000 7ffbbcff9700 1 -- 172.21.15.3:0/616362356 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffbb4024b90 con 0x7ffbc009e000 2023-10-03T10:21:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.510+0000 7ffbc6679700 1 -- 172.21.15.3:0/616362356 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc011ac20 con 0x7ffbc009e000 2023-10-03T10:21:53.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.510+0000 7ffbbcff9700 1 -- 172.21.15.3:0/616362356 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7ffbb4044140 con 0x7ffbc009e000 2023-10-03T10:21:53.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.515+0000 7ffbbcff9700 1 -- 172.21.15.3:0/616362356 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffbb4024e40 con 0x7ffbc009e000 2023-10-03T10:21:53.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.720+0000 7ffbc6679700 1 -- 172.21.15.3:0/616362356 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffbc0074ea0 con 0x7ffbc009e000 2023-10-03T10:21:53.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.721+0000 7ffbbcff9700 1 -- 172.21.15.3:0/616362356 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+13448 (secure 0 0 0) 0x7ffbb403c090 con 0x7ffbc009e000 2023-10-03T10:21:53.721 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-10-03T10:21:53.721 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":40,"fsid":"68e08e7d-a212-4fd3-bd42-88e2411aa299","created":"2023-10-03T10:09:28.703441+0000","modified":"2023-10-03T10:13:12.182630+0000","last_up_change":"2023-10-03T10:12:32.260880+0000","last_in_change":"2023-10-03T10:09:30.156546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-03T10:09:36.623971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-03T10:09:38.210614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-03T10:09:42.695873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-03T10:09:43.115588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cdcdd930-3c5b-4486-a021-7132f90fb6e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":29,"up_thru":32,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6804","nonce":129556}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6805","nonce":129556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6807","nonce":129556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6806","nonce":129556}]},"public_addr":"172.21.15.3:6804/129556","cluster_addr":"172.21.15.3:6805/129556","heartbeat_back_addr":"172.21.15.3:6807/129556","heartbeat_front_addr":"172.21.15.3:6806/129556","state":["exists","up"]},{"osd":1,"uuid":"c8e58877-6762-4741-a8c1-567ebc97b708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":31,"up_thru":32,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6808","nonce":129654}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6809","nonce":129654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6811","nonce":129654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6810","nonce":129654}]},"public_addr":"172.21.15.3:6808/129654","cluster_addr":"172.21.15.3:6809/129654","heartbeat_back_addr":"172.21.15.3:6811/129654","heartbeat_front_addr":"172.21.15.3:6810/129654","state":["exists","up"]},{"osd":2,"uuid":"68a2a21a-7a8e-4f82-8433-5201faa50777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":25,"up_from":31,"up_thru":32,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6800","nonce":129764}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6801","nonce":129764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6803","nonce":129764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6802","nonce":129764}]},"public_addr":"172.21.15.3:6800/129764","cluster_addr":"172.21.15.3:6801/129764","heartbeat_back_addr":"172.21.15.3:6803/129764","heartbeat_front_addr":"172.21.15.3:6802/129764","state":["exists","up"]},{"osd":3,"uuid":"c4df2903-4e1e-4208-bf77-95b9fa749241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":26,"up_from":32,"up_thru":32,"down_at":27,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6812","nonce":129912}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6813","nonce":129912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6815","nonce":129912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.3:6814","nonce":129912}]},"public_addr":"172.21.15.3:6812/129912","cluster_addr":"172.21.15.3:6813/129912","heartbeat_back_addr":"172.21.15.3:6815/129912","heartbeat_front_addr":"172.21.15.3:6814/129912","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.369708+0000","dead_epoch":24},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.257825+0000","dead_epoch":25},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.337931+0000","dead_epoch":26},{"osd":3,"down_stamp":"2023-10-03T10:12:26.212430+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-03T10:09:35.085222+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1","mappings":[{"from":3,"to":0}]},{"pgid":"4.8","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.19","mappings":[{"from":3,"to":0}]},{"pgid":"4.2a","mappings":[{"from":1,"to":2}]},{"pgid":"4.2d","mappings":[{"from":3,"to":0}]},{"pgid":"4.33","mappings":[{"from":3,"to":0}]},{"pgid":"4.39","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/3264604155":"2023-10-04T10:13:12.182591+0000","172.21.15.3:0/1911521769":"2023-10-04T10:13:12.182591+0000","172.21.15.3:0/3289486158":"2023-10-04T10:13:12.182591+0000","172.21.15.3:6816/118296":"2023-10-04T10:12:13.305411+0000","172.21.15.3:0/3246155339":"2023-10-04T10:13:12.182591+0000","172.21.15.3:0/2091814084":"2023-10-04T10:12:13.305411+0000","172.21.15.3:6817/2582564779":"2023-10-04T10:12:38.734689+0000","172.21.15.3:0/3571619151":"2023-10-04T10:13:12.182591+0000","172.21.15.3:0/3919724564":"2023-10-04T10:12:13.305411+0000","172.21.15.3:6816/127025":"2023-10-04T10:13:12.182591+0000","172.21.15.3:0/2059330378":"2023-10-04T10:13:12.182591+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-03T10:21:53.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.723+0000 7ffbc6679700 1 -- 172.21.15.3:0/616362356 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffbc009e000 msgr2=0x7ffbc011e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:53.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.723+0000 7ffbc6679700 1 --2- 172.21.15.3:0/616362356 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffbc009e000 0x7ffbc011e2d0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4015000 tx=0x7ffbb4004750 comp rx=0 tx=0).stop 2023-10-03T10:21:53.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.723+0000 7ffbc6679700 1 -- 172.21.15.3:0/616362356 shutdown_connections 2023-10-03T10:21:53.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.723+0000 7ffbc6679700 1 --2- 172.21.15.3:0/616362356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbc009e410 0x7ffbc011e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:53.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.723+0000 7ffbc6679700 1 --2- 172.21.15.3:0/616362356 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ffbc009d6f0 0x7ffbc011dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:53.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.723+0000 7ffbc6679700 1 --2- 172.21.15.3:0/616362356 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7ffbc009e000 0x7ffbc011e2d0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:53.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.723+0000 7ffbc6679700 1 -- 172.21.15.3:0/616362356 >> 172.21.15.3:0/616362356 conn(0x7ffbc0066f50 msgr2=0x7ffbc0130a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:21:53.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.723+0000 7ffbc6679700 1 -- 172.21.15.3:0/616362356 shutdown_connections 2023-10-03T10:21:53.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.723+0000 7ffbc6679700 1 -- 172.21.15.3:0/616362356 wait complete. 2023-10-03T10:21:53.737 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-03T10:09:36.623971+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}, 'mgr_devicehealth': {}}, 'read_balance': {'score_acting': 4, 'score_stable': 4, 'optimal_score': 0.5, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-03T10:09:38.210614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-03T10:09:42.695873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-03T10:09:43.115588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.059999942779541, 'score_stable': 1.059999942779541, 'optimal_score': 1, 'raw_score_acting': 1.059999942779541, 'raw_score_stable': 1.059999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-03T10:21:53.738 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-03T10:21:53.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.877+0000 7f499d4a9700 1 Processor -- start 2023-10-03T10:21:53.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.877+0000 7f499d4a9700 1 -- start start 2023-10-03T10:21:53.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.878+0000 7f499d4a9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f499809d7d0 0x7f499809dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:53.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.878+0000 7f499d4a9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f499809e0e0 0x7f4998130770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:53.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.878+0000 7f499d4a9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4998130cb0 0x7f4998131150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:53.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.878+0000 7f499d4a9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499809b620 con 0x7f499809e0e0 2023-10-03T10:21:53.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.878+0000 7f499d4a9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499809f370 con 0x7f4998130cb0 2023-10-03T10:21:53.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.878+0000 7f499d4a9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f499809b800 con 0x7f499809d7d0 2023-10-03T10:21:53.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.878+0000 7f4996ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f499809d7d0 0x7f499809dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:53.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.878+0000 7f49977fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4998130cb0 0x7f4998131150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:53.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.878+0000 7f49967fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f499809e0e0 0x7f4998130770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:53.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.878+0000 7f4996ffd700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f499809d7d0 0x7f499809dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:59724/0 (socket says 172.21.15.3:59724) 2023-10-03T10:21:53.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.878+0000 7f49967fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f499809e0e0 0x7f4998130770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38868/0 (socket says 172.21.15.3:38868) 2023-10-03T10:21:53.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.878+0000 7f49977fe700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4998130cb0 0x7f4998131150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46206/0 (socket says 172.21.15.3:46206) 2023-10-03T10:21:53.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.878+0000 7f4996ffd700 1 -- 172.21.15.3:0/2206792939 learned_addr learned my addr 172.21.15.3:0/2206792939 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:21:53.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.878+0000 7f49967fc700 1 -- 172.21.15.3:0/2206792939 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f499809d7d0 msgr2=0x7f499809dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:21:53.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.878+0000 7f49967fc700 1 --2- 172.21.15.3:0/2206792939 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f499809d7d0 0x7f499809dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:53.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.878+0000 7f49967fc700 1 -- 172.21.15.3:0/2206792939 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4998130cb0 msgr2=0x7f4998131150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:21:53.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.878+0000 7f49967fc700 1 --2- 172.21.15.3:0/2206792939 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4998130cb0 0x7f4998131150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:53.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.878+0000 7f49967fc700 1 -- 172.21.15.3:0/2206792939 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499809ef80 con 0x7f499809e0e0 2023-10-03T10:21:53.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.878+0000 7f4996ffd700 1 --2- 172.21.15.3:0/2206792939 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f499809d7d0 0x7f499809dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:21:53.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.878+0000 7f49977fe700 1 --2- 172.21.15.3:0/2206792939 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4998130cb0 0x7f4998131150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:21:53.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.878+0000 7f49967fc700 1 --2- 172.21.15.3:0/2206792939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f499809e0e0 0x7f4998130770 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f4980009840 tx=0x7f4980009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406f3583888a538 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:21:53.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.879+0000 7f4995ffb700 1 -- 172.21.15.3:0/2206792939 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4980011830 con 0x7f499809e0e0 2023-10-03T10:21:53.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.879+0000 7f4995ffb700 1 -- 172.21.15.3:0/2206792939 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4980011990 con 0x7f499809e0e0 2023-10-03T10:21:53.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.879+0000 7f4995ffb700 1 -- 172.21.15.3:0/2206792939 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4980011d20 con 0x7f499809e0e0 2023-10-03T10:21:53.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.879+0000 7f499d4a9700 1 -- 172.21.15.3:0/2206792939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f499809e0e0 msgr2=0x7f4998130770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:53.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.879+0000 7f499d4a9700 1 --2- 172.21.15.3:0/2206792939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f499809e0e0 0x7f4998130770 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f4980009840 tx=0x7f4980009b50 comp rx=0 tx=0).stop 2023-10-03T10:21:53.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.879+0000 7f499d4a9700 1 -- 172.21.15.3:0/2206792939 shutdown_connections 2023-10-03T10:21:53.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.879+0000 7f499d4a9700 1 --2- 172.21.15.3:0/2206792939 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4998130cb0 0x7f4998131150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:53.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.879+0000 7f499d4a9700 1 --2- 172.21.15.3:0/2206792939 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f499809d7d0 0x7f499809dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:53.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.879+0000 7f499d4a9700 1 --2- 172.21.15.3:0/2206792939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f499809e0e0 0x7f4998130770 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:53.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.879+0000 7f499d4a9700 1 -- 172.21.15.3:0/2206792939 >> 172.21.15.3:0/2206792939 conn(0x7f4998066d70 msgr2=0x7f49980a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:21:53.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.879+0000 7f499d4a9700 1 -- 172.21.15.3:0/2206792939 shutdown_connections 2023-10-03T10:21:53.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.879+0000 7f499d4a9700 1 -- 172.21.15.3:0/2206792939 wait complete. 2023-10-03T10:21:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.880+0000 7f499d4a9700 1 Processor -- start 2023-10-03T10:21:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.881+0000 7f499d4a9700 1 -- start start 2023-10-03T10:21:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.881+0000 7f499d4a9700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f499809d7d0 0x7f49981211d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.881+0000 7f499d4a9700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f499809e0e0 0x7f4998121710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:53.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.881+0000 7f499d4a9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4998130cb0 0x7f4998121c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:53.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.881+0000 7f499d4a9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4998123660 con 0x7f4998130cb0 2023-10-03T10:21:53.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.881+0000 7f499d4a9700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4998124ca0 con 0x7f499809d7d0 2023-10-03T10:21:53.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.881+0000 7f499d4a9700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f49980a0180 con 0x7f499809e0e0 2023-10-03T10:21:53.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.881+0000 7f4996ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f499809d7d0 0x7f49981211d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:53.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.881+0000 7f49967fc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f499809e0e0 0x7f4998121710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:53.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.881+0000 7f4996ffd700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f499809d7d0 0x7f49981211d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46220/0 (socket says 172.21.15.3:46220) 2023-10-03T10:21:53.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.881+0000 7f49977fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4998130cb0 0x7f4998121c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.881+0000 7f4996ffd700 1 -- 172.21.15.3:0/385706666 learned_addr learned my addr 172.21.15.3:0/385706666 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:21:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.882+0000 7f4996ffd700 1 -- 172.21.15.3:0/385706666 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f499809e0e0 msgr2=0x7f4998121710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.882+0000 7f4996ffd700 1 --2- 172.21.15.3:0/385706666 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f499809e0e0 0x7f4998121710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.882+0000 7f4996ffd700 1 -- 172.21.15.3:0/385706666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4998130cb0 msgr2=0x7f4998121c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.882+0000 7f4996ffd700 1 --2- 172.21.15.3:0/385706666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4998130cb0 0x7f4998121c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.882+0000 7f4996ffd700 1 -- 172.21.15.3:0/385706666 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4988009550 con 0x7f499809d7d0 2023-10-03T10:21:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.882+0000 7f49967fc700 1 --2- 172.21.15.3:0/385706666 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f499809e0e0 0x7f4998121710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:21:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.882+0000 7f49977fe700 1 --2- 172.21.15.3:0/385706666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4998130cb0 0x7f4998121c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:21:53.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.882+0000 7f4996ffd700 1 --2- 172.21.15.3:0/385706666 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f499809d7d0 0x7f49981211d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f498800e930 tx=0x7f498800ec40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:21:53.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.882+0000 7f498f7fe700 1 -- 172.21.15.3:0/385706666 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4988004030 con 0x7f499809d7d0 2023-10-03T10:21:53.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.882+0000 7f498f7fe700 1 -- 172.21.15.3:0/385706666 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4988004190 con 0x7f499809d7d0 2023-10-03T10:21:53.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.882+0000 7f499d4a9700 1 -- 172.21.15.3:0/385706666 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f499000bc80 con 0x7f499809d7d0 2023-10-03T10:21:53.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.882+0000 7f498f7fe700 1 -- 172.21.15.3:0/385706666 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4988004520 con 0x7f499809d7d0 2023-10-03T10:21:53.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.882+0000 7f499d4a9700 1 -- 172.21.15.3:0/385706666 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49800094a0 con 0x7f499809d7d0 2023-10-03T10:21:53.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.883+0000 7f498f7fe700 1 -- 172.21.15.3:0/385706666 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f49880036a0 con 0x7f499809d7d0 2023-10-03T10:21:53.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.883+0000 7f499d4a9700 1 -- 172.21.15.3:0/385706666 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4998130590 con 0x7f499809d7d0 2023-10-03T10:21:53.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.883+0000 7f498f7fe700 1 -- 172.21.15.3:0/385706666 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f498803f850 con 0x7f499809d7d0 2023-10-03T10:21:53.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:53.888+0000 7f498f7fe700 1 -- 172.21.15.3:0/385706666 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4988011bd0 con 0x7f499809d7d0 2023-10-03T10:21:54.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.089+0000 7f499d4a9700 1 -- 172.21.15.3:0/385706666 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f499804f2a0 con 0x7f499809d7d0 2023-10-03T10:21:54.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.089+0000 7f498f7fe700 1 -- 172.21.15.3:0/385706666 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4988011e00 con 0x7f499809d7d0 2023-10-03T10:21:54.090 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-10-03T10:21:54.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.092+0000 7f499d4a9700 1 -- 172.21.15.3:0/385706666 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f499809d7d0 msgr2=0x7f49981211d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:54.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.092+0000 7f499d4a9700 1 --2- 172.21.15.3:0/385706666 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f499809d7d0 0x7f49981211d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f498800e930 tx=0x7f498800ec40 comp rx=0 tx=0).stop 2023-10-03T10:21:54.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.092+0000 7f499d4a9700 1 -- 172.21.15.3:0/385706666 shutdown_connections 2023-10-03T10:21:54.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.092+0000 7f499d4a9700 1 --2- 172.21.15.3:0/385706666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4998130cb0 0x7f4998121c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:54.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.092+0000 7f499d4a9700 1 --2- 172.21.15.3:0/385706666 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f499809d7d0 0x7f49981211d0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:54.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.092+0000 7f499d4a9700 1 --2- 172.21.15.3:0/385706666 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f499809e0e0 0x7f4998121710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:54.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.092+0000 7f499d4a9700 1 -- 172.21.15.3:0/385706666 >> 172.21.15.3:0/385706666 conn(0x7f4998066d70 msgr2=0x7f4998071ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:21:54.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.092+0000 7f499d4a9700 1 -- 172.21.15.3:0/385706666 shutdown_connections 2023-10-03T10:21:54.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.092+0000 7f499d4a9700 1 -- 172.21.15.3:0/385706666 wait complete. 2023-10-03T10:21:54.106 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-03T10:21:54.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.245+0000 7f5d87d1a700 1 Processor -- start 2023-10-03T10:21:54.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.246+0000 7f5d87d1a700 1 -- start start 2023-10-03T10:21:54.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.246+0000 7f5d87d1a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d80136780 0x7f5d80138b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:54.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.246+0000 7f5d87d1a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5d801390a0 0x7f5d8013b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:54.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.246+0000 7f5d87d1a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d801301c0 con 0x7f5d801303c0 2023-10-03T10:21:54.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.246+0000 7f5d87d1a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d8009f620 con 0x7f5d801390a0 2023-10-03T10:21:54.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.246+0000 7f5d87d1a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8009e1b0 con 0x7f5d80136780 2023-10-03T10:21:54.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.246+0000 7f5d862b7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5d801390a0 0x7f5d8013b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:54.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.246+0000 7f5d852b5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d80136780 0x7f5d80138b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:54.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.246+0000 7f5d852b5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d80136780 0x7f5d80138b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:59740/0 (socket says 172.21.15.3:59740) 2023-10-03T10:21:54.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.246+0000 7f5d862b7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5d801390a0 0x7f5d8013b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38892/0 (socket says 172.21.15.3:38892) 2023-10-03T10:21:54.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.246+0000 7f5d852b5700 1 -- 172.21.15.3:0/388525956 learned_addr learned my addr 172.21.15.3:0/388525956 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:21:54.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.247+0000 7f5d852b5700 1 -- 172.21.15.3:0/388525956 >> v2:172.21.15.3:3300/0 conn(0x7f5d801390a0 msgr2=0x7f5d8013b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:21:54.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.247+0000 7f5d852b5700 1 --2- 172.21.15.3:0/388525956 >> v2:172.21.15.3:3300/0 conn(0x7f5d801390a0 0x7f5d8013b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:54.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.247+0000 7f5d852b5700 1 -- 172.21.15.3:0/388525956 >> v1:172.21.15.3:6789/0 conn(0x7f5d801303c0 legacy=0x7f5d80130790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:54.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.247+0000 7f5d852b5700 1 -- 172.21.15.3:0/388525956 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d800a3a60 con 0x7f5d80136780 2023-10-03T10:21:54.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.247+0000 7f5d862b7700 1 --2- 172.21.15.3:0/388525956 >> v2:172.21.15.3:3300/0 conn(0x7f5d801390a0 0x7f5d8013b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:21:54.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.247+0000 7f5d852b5700 1 --2- 172.21.15.3:0/388525956 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d80136780 0x7f5d80138b60 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f5d700098f0 tx=0x7f5d70009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ceee327db4b43586 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:21:54.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.247+0000 7f5d84ab4700 1 -- 172.21.15.3:0/388525956 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70011830 con 0x7f5d80136780 2023-10-03T10:21:54.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.247+0000 7f5d84ab4700 1 -- 172.21.15.3:0/388525956 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d70011990 con 0x7f5d80136780 2023-10-03T10:21:54.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.247+0000 7f5d84ab4700 1 -- 172.21.15.3:0/388525956 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70011d20 con 0x7f5d80136780 2023-10-03T10:21:54.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.248+0000 7f5d87d1a700 1 -- 172.21.15.3:0/388525956 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d80136780 msgr2=0x7f5d80138b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:54.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.248+0000 7f5d87d1a700 1 --2- 172.21.15.3:0/388525956 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d80136780 0x7f5d80138b60 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f5d700098f0 tx=0x7f5d70009c00 comp rx=0 tx=0).stop 2023-10-03T10:21:54.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.248+0000 7f5d87d1a700 1 -- 172.21.15.3:0/388525956 shutdown_connections 2023-10-03T10:21:54.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.248+0000 7f5d87d1a700 1 --2- 172.21.15.3:0/388525956 >> v2:172.21.15.3:3300/0 conn(0x7f5d801390a0 0x7f5d8013b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:54.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.248+0000 7f5d87d1a700 1 --2- 172.21.15.3:0/388525956 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d80136780 0x7f5d80138b60 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:54.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.248+0000 7f5d87d1a700 1 -- 172.21.15.3:0/388525956 >> 172.21.15.3:0/388525956 conn(0x7f5d80066d70 msgr2=0x7f5d800a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:21:54.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.248+0000 7f5d87d1a700 1 -- 172.21.15.3:0/388525956 shutdown_connections 2023-10-03T10:21:54.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.248+0000 7f5d87d1a700 1 -- 172.21.15.3:0/388525956 wait complete. 2023-10-03T10:21:54.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.249+0000 7f5d87d1a700 1 Processor -- start 2023-10-03T10:21:54.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.249+0000 7f5d87d1a700 1 -- start start 2023-10-03T10:21:54.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.249+0000 7f5d87d1a700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d801303c0 0x7f5d800713a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:54.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.249+0000 7f5d87d1a700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d80136780 0x7f5d800719c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:54.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.249+0000 7f5d87d1a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d801390a0 0x7f5d80071f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:54.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.249+0000 7f5d87d1a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d80070450 con 0x7f5d801390a0 2023-10-03T10:21:54.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.249+0000 7f5d87d1a700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d80072660 con 0x7f5d801303c0 2023-10-03T10:21:54.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.249+0000 7f5d87d1a700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d80070190 con 0x7f5d80136780 2023-10-03T10:21:54.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.249+0000 7f5d852b5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d80136780 0x7f5d800719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:54.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.249+0000 7f5d85ab6700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d801303c0 0x7f5d800713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:54.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.249+0000 7f5d852b5700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d80136780 0x7f5d800719c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:59748/0 (socket says 172.21.15.3:59748) 2023-10-03T10:21:54.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.249+0000 7f5d852b5700 1 -- 172.21.15.3:0/3025606861 learned_addr learned my addr 172.21.15.3:0/3025606861 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:21:54.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.249+0000 7f5d862b7700 1 --2- 172.21.15.3:0/3025606861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d801390a0 0x7f5d80071f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:54.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.250+0000 7f5d85ab6700 1 -- 172.21.15.3:0/3025606861 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d80136780 msgr2=0x7f5d800719c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:54.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.250+0000 7f5d85ab6700 1 --2- 172.21.15.3:0/3025606861 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d80136780 0x7f5d800719c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:54.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.250+0000 7f5d85ab6700 1 -- 172.21.15.3:0/3025606861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d801390a0 msgr2=0x7f5d80071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:54.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.250+0000 7f5d85ab6700 1 --2- 172.21.15.3:0/3025606861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d801390a0 0x7f5d80071f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:54.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.250+0000 7f5d85ab6700 1 -- 172.21.15.3:0/3025606861 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d78009a70 con 0x7f5d801303c0 2023-10-03T10:21:54.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.250+0000 7f5d852b5700 1 --2- 172.21.15.3:0/3025606861 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d80136780 0x7f5d800719c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:21:54.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.250+0000 7f5d862b7700 1 --2- 172.21.15.3:0/3025606861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d801390a0 0x7f5d80071f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:21:54.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.250+0000 7f5d85ab6700 1 --2- 172.21.15.3:0/3025606861 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d801303c0 0x7f5d800713a0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c000f60 tx=0x7f5d7c00c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:21:54.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.250+0000 7f5d767fc700 1 -- 172.21.15.3:0/3025606861 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0052d0 con 0x7f5d801303c0 2023-10-03T10:21:54.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.250+0000 7f5d767fc700 1 -- 172.21.15.3:0/3025606861 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7c005430 con 0x7f5d801303c0 2023-10-03T10:21:54.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.250+0000 7f5d87d1a700 1 -- 172.21.15.3:0/3025606861 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d70009550 con 0x7f5d801303c0 2023-10-03T10:21:54.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.250+0000 7f5d767fc700 1 -- 172.21.15.3:0/3025606861 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0038c0 con 0x7f5d801303c0 2023-10-03T10:21:54.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.250+0000 7f5d87d1a700 1 -- 172.21.15.3:0/3025606861 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8006e860 con 0x7f5d801303c0 2023-10-03T10:21:54.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.251+0000 7f5d767fc700 1 -- 172.21.15.3:0/3025606861 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5d7c003a20 con 0x7f5d801303c0 2023-10-03T10:21:54.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.251+0000 7f5d767fc700 1 -- 172.21.15.3:0/3025606861 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f5d7c03e9d0 con 0x7f5d801303c0 2023-10-03T10:21:54.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.251+0000 7f5d87d1a700 1 -- 172.21.15.3:0/3025606861 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d800705a0 con 0x7f5d801303c0 2023-10-03T10:21:54.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.256+0000 7f5d767fc700 1 -- 172.21.15.3:0/3025606861 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5d7c017070 con 0x7f5d801303c0 2023-10-03T10:21:54.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.456+0000 7f5d87d1a700 1 -- 172.21.15.3:0/3025606861 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5d8004f2a0 con 0x7f5d801303c0 2023-10-03T10:21:54.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.457+0000 7f5d767fc700 1 -- 172.21.15.3:0/3025606861 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5d7c011590 con 0x7f5d801303c0 2023-10-03T10:21:54.458 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-10-03T10:21:54.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.459+0000 7f5d87d1a700 1 -- 172.21.15.3:0/3025606861 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d801303c0 msgr2=0x7f5d800713a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:54.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.459+0000 7f5d87d1a700 1 --2- 172.21.15.3:0/3025606861 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d801303c0 0x7f5d800713a0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c000f60 tx=0x7f5d7c00c730 comp rx=0 tx=0).stop 2023-10-03T10:21:54.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.459+0000 7f5d87d1a700 1 -- 172.21.15.3:0/3025606861 shutdown_connections 2023-10-03T10:21:54.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.459+0000 7f5d87d1a700 1 --2- 172.21.15.3:0/3025606861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d801390a0 0x7f5d80071f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:54.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.459+0000 7f5d87d1a700 1 --2- 172.21.15.3:0/3025606861 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5d801303c0 0x7f5d800713a0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:54.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.459+0000 7f5d87d1a700 1 --2- 172.21.15.3:0/3025606861 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f5d80136780 0x7f5d800719c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:54.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.459+0000 7f5d87d1a700 1 -- 172.21.15.3:0/3025606861 >> 172.21.15.3:0/3025606861 conn(0x7f5d80066d70 msgr2=0x7f5d80134560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:21:54.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.459+0000 7f5d87d1a700 1 -- 172.21.15.3:0/3025606861 shutdown_connections 2023-10-03T10:21:54.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.460+0000 7f5d87d1a700 1 -- 172.21.15.3:0/3025606861 wait complete. 2023-10-03T10:21:54.473 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-03T10:21:54.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.613+0000 7fa212fdc700 1 Processor -- start 2023-10-03T10:21:54.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.613+0000 7fa212fdc700 1 -- start start 2023-10-03T10:21:54.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.614+0000 7fa212fdc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa20c131b30 0x7fa20c131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:54.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.614+0000 7fa212fdc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa20c09e850 0x7fa20c09ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:54.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.614+0000 7fa212fdc700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa20c0a01c0 con 0x7fa20c09f1c0 2023-10-03T10:21:54.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.614+0000 7fa212fdc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa20c09b560 con 0x7fa20c131b30 2023-10-03T10:21:54.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.614+0000 7fa212fdc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c09a260 con 0x7fa20c09e850 2023-10-03T10:21:54.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.614+0000 7fa210d78700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa20c131b30 0x7fa20c131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:54.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.614+0000 7fa20bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa20c09e850 0x7fa20c09ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:54.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.614+0000 7fa20bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa20c09e850 0x7fa20c09ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46242/0 (socket says 172.21.15.3:46242) 2023-10-03T10:21:54.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.614+0000 7fa210d78700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa20c131b30 0x7fa20c131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38900/0 (socket says 172.21.15.3:38900) 2023-10-03T10:21:54.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.614+0000 7fa20bfff700 1 -- 172.21.15.3:0/686269038 learned_addr learned my addr 172.21.15.3:0/686269038 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:21:54.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.614+0000 7fa20bfff700 1 -- 172.21.15.3:0/686269038 >> v2:172.21.15.3:3300/0 conn(0x7fa20c131b30 msgr2=0x7fa20c131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:21:54.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.614+0000 7fa20bfff700 1 --2- 172.21.15.3:0/686269038 >> v2:172.21.15.3:3300/0 conn(0x7fa20c131b30 0x7fa20c131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:54.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.614+0000 7fa20bfff700 1 -- 172.21.15.3:0/686269038 >> v1:172.21.15.3:6789/0 conn(0x7fa20c09f1c0 legacy=0x7fa20c1389c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:54.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.614+0000 7fa20bfff700 1 -- 172.21.15.3:0/686269038 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20c0a07a0 con 0x7fa20c09e850 2023-10-03T10:21:54.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.614+0000 7fa210d78700 1 --2- 172.21.15.3:0/686269038 >> v2:172.21.15.3:3300/0 conn(0x7fa20c131b30 0x7fa20c131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:21:54.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.614+0000 7fa20bfff700 1 --2- 172.21.15.3:0/686269038 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa20c09e850 0x7fa20c09ec80 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fa2000098f0 tx=0x7fa200009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce7bd41b8d58c6bd server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:21:54.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.615+0000 7fa20b7fe700 1 -- 172.21.15.3:0/686269038 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa200011830 con 0x7fa20c09e850 2023-10-03T10:21:54.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.615+0000 7fa20b7fe700 1 -- 172.21.15.3:0/686269038 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa200011990 con 0x7fa20c09e850 2023-10-03T10:21:54.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.615+0000 7fa20b7fe700 1 -- 172.21.15.3:0/686269038 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa200011d20 con 0x7fa20c09e850 2023-10-03T10:21:54.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.615+0000 7fa212fdc700 1 -- 172.21.15.3:0/686269038 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa20c09e850 msgr2=0x7fa20c09ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:54.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.615+0000 7fa212fdc700 1 --2- 172.21.15.3:0/686269038 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa20c09e850 0x7fa20c09ec80 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fa2000098f0 tx=0x7fa200009c00 comp rx=0 tx=0).stop 2023-10-03T10:21:54.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.615+0000 7fa212fdc700 1 -- 172.21.15.3:0/686269038 shutdown_connections 2023-10-03T10:21:54.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.615+0000 7fa212fdc700 1 --2- 172.21.15.3:0/686269038 >> v2:172.21.15.3:3300/0 conn(0x7fa20c131b30 0x7fa20c131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:54.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.615+0000 7fa212fdc700 1 --2- 172.21.15.3:0/686269038 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa20c09e850 0x7fa20c09ec80 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:54.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.615+0000 7fa212fdc700 1 -- 172.21.15.3:0/686269038 >> 172.21.15.3:0/686269038 conn(0x7fa20c0a4130 msgr2=0x7fa20c0a4530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:21:54.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.615+0000 7fa212fdc700 1 -- 172.21.15.3:0/686269038 shutdown_connections 2023-10-03T10:21:54.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.615+0000 7fa212fdc700 1 -- 172.21.15.3:0/686269038 wait complete. 2023-10-03T10:21:54.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.616+0000 7fa212fdc700 1 Processor -- start 2023-10-03T10:21:54.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.616+0000 7fa212fdc700 1 -- start start 2023-10-03T10:21:54.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.617+0000 7fa212fdc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa20c09e850 0x7fa20c11af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:54.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.617+0000 7fa212fdc700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa20c09f1c0 0x7fa20c11b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:54.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.617+0000 7fa212fdc700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa20c131b30 0x7fa20c11ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:54.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.617+0000 7fa212fdc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c11c460 con 0x7fa20c09e850 2023-10-03T10:21:54.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.617+0000 7fa212fdc700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c11d1f0 con 0x7fa20c09f1c0 2023-10-03T10:21:54.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.617+0000 7fa212fdc700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c11eab0 con 0x7fa20c131b30 2023-10-03T10:21:54.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.617+0000 7fa210d78700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa20c09e850 0x7fa20c11af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:54.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.617+0000 7fa20bfff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa20c09f1c0 0x7fa20c11b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:54.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.617+0000 7fa210d78700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa20c09e850 0x7fa20c11af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38916/0 (socket says 172.21.15.3:38916) 2023-10-03T10:21:54.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.617+0000 7fa210d78700 1 -- 172.21.15.3:0/2751416709 learned_addr learned my addr 172.21.15.3:0/2751416709 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:21:54.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.617+0000 7fa211579700 1 --2- 172.21.15.3:0/2751416709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa20c131b30 0x7fa20c11ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:54.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.618+0000 7fa20bfff700 1 -- 172.21.15.3:0/2751416709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa20c131b30 msgr2=0x7fa20c11ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:54.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.618+0000 7fa20bfff700 1 --2- 172.21.15.3:0/2751416709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa20c131b30 0x7fa20c11ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:54.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.618+0000 7fa20bfff700 1 -- 172.21.15.3:0/2751416709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa20c09e850 msgr2=0x7fa20c11af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:54.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.618+0000 7fa20bfff700 1 --2- 172.21.15.3:0/2751416709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa20c09e850 0x7fa20c11af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:54.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.618+0000 7fa20bfff700 1 -- 172.21.15.3:0/2751416709 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1fc0094a0 con 0x7fa20c09f1c0 2023-10-03T10:21:54.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.618+0000 7fa210d78700 1 --2- 172.21.15.3:0/2751416709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa20c09e850 0x7fa20c11af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:21:54.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.618+0000 7fa20bfff700 1 --2- 172.21.15.3:0/2751416709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa20c09f1c0 0x7fa20c11b4d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fa200016000 tx=0x7fa200004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:21:54.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.618+0000 7fa2097fa700 1 -- 172.21.15.3:0/2751416709 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa200004400 con 0x7fa20c09f1c0 2023-10-03T10:21:54.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.618+0000 7fa2097fa700 1 -- 172.21.15.3:0/2751416709 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa20001f070 con 0x7fa20c09f1c0 2023-10-03T10:21:54.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.618+0000 7fa212fdc700 1 -- 172.21.15.3:0/2751416709 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa200009550 con 0x7fa20c09f1c0 2023-10-03T10:21:54.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.618+0000 7fa2097fa700 1 -- 172.21.15.3:0/2751416709 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2000038e0 con 0x7fa20c09f1c0 2023-10-03T10:21:54.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.618+0000 7fa212fdc700 1 -- 172.21.15.3:0/2751416709 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa20c09da10 con 0x7fa20c09f1c0 2023-10-03T10:21:54.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.619+0000 7fa2097fa700 1 -- 172.21.15.3:0/2751416709 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa200021030 con 0x7fa20c09f1c0 2023-10-03T10:21:54.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.619+0000 7fa2097fa700 1 -- 172.21.15.3:0/2751416709 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fa200040940 con 0x7fa20c09f1c0 2023-10-03T10:21:54.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.619+0000 7fa212fdc700 1 -- 172.21.15.3:0/2751416709 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa20c04f2a0 con 0x7fa20c09f1c0 2023-10-03T10:21:54.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.624+0000 7fa2097fa700 1 -- 172.21.15.3:0/2751416709 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa200004560 con 0x7fa20c09f1c0 2023-10-03T10:21:54.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.827+0000 7fa212fdc700 1 -- 172.21.15.3:0/2751416709 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa20c0728b0 con 0x7fa20c09f1c0 2023-10-03T10:21:54.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.827+0000 7fa2097fa700 1 -- 172.21.15.3:0/2751416709 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa200015030 con 0x7fa20c09f1c0 2023-10-03T10:21:54.828 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-10-03T10:21:54.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.830+0000 7fa212fdc700 1 -- 172.21.15.3:0/2751416709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa20c09f1c0 msgr2=0x7fa20c11b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:54.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.830+0000 7fa212fdc700 1 --2- 172.21.15.3:0/2751416709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa20c09f1c0 0x7fa20c11b4d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fa200016000 tx=0x7fa200004750 comp rx=0 tx=0).stop 2023-10-03T10:21:54.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.830+0000 7fa212fdc700 1 -- 172.21.15.3:0/2751416709 shutdown_connections 2023-10-03T10:21:54.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.830+0000 7fa212fdc700 1 --2- 172.21.15.3:0/2751416709 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fa20c131b30 0x7fa20c11ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:54.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.830+0000 7fa212fdc700 1 --2- 172.21.15.3:0/2751416709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa20c09e850 0x7fa20c11af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:54.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.830+0000 7fa212fdc700 1 --2- 172.21.15.3:0/2751416709 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa20c09f1c0 0x7fa20c11b4d0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:54.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.830+0000 7fa212fdc700 1 -- 172.21.15.3:0/2751416709 >> 172.21.15.3:0/2751416709 conn(0x7fa20c0a4130 msgr2=0x7fa20c131050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:21:54.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.830+0000 7fa212fdc700 1 -- 172.21.15.3:0/2751416709 shutdown_connections 2023-10-03T10:21:54.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.830+0000 7fa212fdc700 1 -- 172.21.15.3:0/2751416709 wait complete. 2023-10-03T10:21:54.844 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-03T10:21:54.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.983+0000 7f0f97262700 1 Processor -- start 2023-10-03T10:21:54.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.984+0000 7f0f97262700 1 -- start start 2023-10-03T10:21:54.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.984+0000 7f0f97262700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f90071d60 0x7f0f90072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:54.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.984+0000 7f0f97262700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f9009f350 0x7f0f9013bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:54.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.984+0000 7f0f97262700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f900a0280 con 0x7f0f9009e850 2023-10-03T10:21:54.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.984+0000 7f0f97262700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f900a0020 con 0x7f0f9009f350 2023-10-03T10:21:54.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.984+0000 7f0f97262700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9006ba00 con 0x7f0f90071d60 2023-10-03T10:21:54.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.985+0000 7f0f8ffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0f9009e850 0x7f0f9009ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37872/0 (socket says 172.21.15.3:37872) 2023-10-03T10:21:54.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.985+0000 7f0f8ffff700 1 -- 172.21.15.3:0/1987744219 learned_addr learned my addr 172.21.15.3:0/1987744219 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-10-03T10:21:54.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.985+0000 7f0f957ff700 1 --2- 172.21.15.3:0/1987744219 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f9009f350 0x7f0f9013bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:54.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.985+0000 7f0f94ffe700 1 --2- 172.21.15.3:0/1987744219 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f90071d60 0x7f0f90072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:54.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.985+0000 7f0f94ffe700 1 -- 172.21.15.3:0/1987744219 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f9009f350 msgr2=0x7f0f9013bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:21:54.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.985+0000 7f0f94ffe700 1 --2- 172.21.15.3:0/1987744219 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f9009f350 0x7f0f9013bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:54.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.985+0000 7f0f94ffe700 1 -- 172.21.15.3:0/1987744219 >> v1:172.21.15.3:6789/0 conn(0x7f0f9009e850 legacy=0x7f0f9009ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:54.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.985+0000 7f0f94ffe700 1 -- 172.21.15.3:0/1987744219 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f9006cb80 con 0x7f0f90071d60 2023-10-03T10:21:54.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.985+0000 7f0f8f7fe700 1 -- 172.21.15.3:0/1987744219 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3725727396 0 0) 0x7f0f900a0280 con 0x7f0f9009e850 2023-10-03T10:21:54.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.985+0000 7f0f94ffe700 1 --2- 172.21.15.3:0/1987744219 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f90071d60 0x7f0f90072130 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f0f80009870 tx=0x7f0f80009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d75298ec8224a738 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:21:54.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.985+0000 7f0f8f7fe700 1 -- 172.21.15.3:0/1987744219 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f80011830 con 0x7f0f90071d60 2023-10-03T10:21:54.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.985+0000 7f0f8f7fe700 1 -- 172.21.15.3:0/1987744219 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f80011990 con 0x7f0f90071d60 2023-10-03T10:21:54.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.985+0000 7f0f8f7fe700 1 -- 172.21.15.3:0/1987744219 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f80011d20 con 0x7f0f90071d60 2023-10-03T10:21:54.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.986+0000 7f0f97262700 1 -- 172.21.15.3:0/1987744219 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f90071d60 msgr2=0x7f0f90072130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:54.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.986+0000 7f0f97262700 1 --2- 172.21.15.3:0/1987744219 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f90071d60 0x7f0f90072130 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f0f80009870 tx=0x7f0f80009b80 comp rx=0 tx=0).stop 2023-10-03T10:21:54.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.986+0000 7f0f97262700 1 -- 172.21.15.3:0/1987744219 shutdown_connections 2023-10-03T10:21:54.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.986+0000 7f0f97262700 1 --2- 172.21.15.3:0/1987744219 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f9009f350 0x7f0f9013bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:54.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.986+0000 7f0f97262700 1 --2- 172.21.15.3:0/1987744219 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f90071d60 0x7f0f90072130 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:54.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.986+0000 7f0f97262700 1 -- 172.21.15.3:0/1987744219 >> 172.21.15.3:0/1987744219 conn(0x7f0f900a4130 msgr2=0x7f0f900a4530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:21:54.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.986+0000 7f0f97262700 1 -- 172.21.15.3:0/1987744219 shutdown_connections 2023-10-03T10:21:54.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.986+0000 7f0f97262700 1 -- 172.21.15.3:0/1987744219 wait complete. 2023-10-03T10:21:54.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.987+0000 7f0f97262700 1 Processor -- start 2023-10-03T10:21:54.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.987+0000 7f0f97262700 1 -- start start 2023-10-03T10:21:54.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.988+0000 7f0f97262700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f90071d60 0x7f0f901a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:54.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.988+0000 7f0f97262700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f9009e850 0x7f0f901a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:54.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.988+0000 7f0f97262700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f9009f350 0x7f0f901a41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:54.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.988+0000 7f0f97262700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f901a6be0 con 0x7f0f9009f350 2023-10-03T10:21:54.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.988+0000 7f0f97262700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f901a6370 con 0x7f0f9009e850 2023-10-03T10:21:54.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.988+0000 7f0f97262700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f900a37e0 con 0x7f0f90071d60 2023-10-03T10:21:54.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.988+0000 7f0f8ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f9009e850 0x7f0f901a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:54.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.988+0000 7f0f94ffe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f90071d60 0x7f0f901a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:54.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.988+0000 7f0f957ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f9009f350 0x7f0f901a41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:54.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.988+0000 7f0f8ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f9009e850 0x7f0f901a3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46270/0 (socket says 172.21.15.3:46270) 2023-10-03T10:21:54.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.988+0000 7f0f8ffff700 1 -- 172.21.15.3:0/1609694555 learned_addr learned my addr 172.21.15.3:0/1609694555 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:21:54.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.988+0000 7f0f94ffe700 1 -- 172.21.15.3:0/1609694555 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f9009e850 msgr2=0x7f0f901a3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:54.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.988+0000 7f0f94ffe700 1 --2- 172.21.15.3:0/1609694555 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f9009e850 0x7f0f901a3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:54.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.988+0000 7f0f94ffe700 1 -- 172.21.15.3:0/1609694555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f9009f350 msgr2=0x7f0f901a41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:54.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.988+0000 7f0f94ffe700 1 --2- 172.21.15.3:0/1609694555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f9009f350 0x7f0f901a41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:54.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.988+0000 7f0f94ffe700 1 -- 172.21.15.3:0/1609694555 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f800094a0 con 0x7f0f90071d60 2023-10-03T10:21:54.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.988+0000 7f0f8ffff700 1 --2- 172.21.15.3:0/1609694555 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f9009e850 0x7f0f901a3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:21:54.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.989+0000 7f0f94ffe700 1 --2- 172.21.15.3:0/1609694555 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f90071d60 0x7f0f901a3720 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f0f80016000 tx=0x7f0f80004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:21:54.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.989+0000 7f0f8d7fa700 1 -- 172.21.15.3:0/1609694555 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f80004510 con 0x7f0f90071d60 2023-10-03T10:21:54.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.989+0000 7f0f8d7fa700 1 -- 172.21.15.3:0/1609694555 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8001f070 con 0x7f0f90071d60 2023-10-03T10:21:54.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.989+0000 7f0f97262700 1 -- 172.21.15.3:0/1609694555 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f9006cda0 con 0x7f0f90071d60 2023-10-03T10:21:54.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.989+0000 7f0f8d7fa700 1 -- 172.21.15.3:0/1609694555 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f800038c0 con 0x7f0f90071d60 2023-10-03T10:21:54.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.989+0000 7f0f97262700 1 -- 172.21.15.3:0/1609694555 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f901a5f20 con 0x7f0f90071d60 2023-10-03T10:21:54.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.989+0000 7f0f8d7fa700 1 -- 172.21.15.3:0/1609694555 <== mon.2 v2:172.21.15.3:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0f80003a20 con 0x7f0f90071d60 2023-10-03T10:21:54.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.990+0000 7f0f8d7fa700 1 -- 172.21.15.3:0/1609694555 <== mon.2 v2:172.21.15.3:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f0f80021070 con 0x7f0f90071d60 2023-10-03T10:21:54.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.990+0000 7f0f97262700 1 -- 172.21.15.3:0/1609694555 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f901a6780 con 0x7f0f90071d60 2023-10-03T10:21:54.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:54.995+0000 7f0f8d7fa700 1 -- 172.21.15.3:0/1609694555 <== mon.2 v2:172.21.15.3:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0f8001a430 con 0x7f0f90071d60 2023-10-03T10:21:55.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.199+0000 7f0f97262700 1 -- 172.21.15.3:0/1609694555 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0f9004f2a0 con 0x7f0f90071d60 2023-10-03T10:21:55.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.200+0000 7f0f8d7fa700 1 -- 172.21.15.3:0/1609694555 <== mon.2 v2:172.21.15.3:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0f80015020 con 0x7f0f90071d60 2023-10-03T10:21:55.200 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-10-03T10:21:55.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.202+0000 7f0f97262700 1 -- 172.21.15.3:0/1609694555 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f90071d60 msgr2=0x7f0f901a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:55.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.202+0000 7f0f97262700 1 --2- 172.21.15.3:0/1609694555 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f90071d60 0x7f0f901a3720 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f0f80016000 tx=0x7f0f80004750 comp rx=0 tx=0).stop 2023-10-03T10:21:55.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.202+0000 7f0f97262700 1 -- 172.21.15.3:0/1609694555 shutdown_connections 2023-10-03T10:21:55.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.202+0000 7f0f97262700 1 --2- 172.21.15.3:0/1609694555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f9009f350 0x7f0f901a41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:55.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.202+0000 7f0f97262700 1 --2- 172.21.15.3:0/1609694555 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f0f90071d60 0x7f0f901a3720 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:55.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.202+0000 7f0f97262700 1 --2- 172.21.15.3:0/1609694555 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0f9009e850 0x7f0f901a3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:55.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.202+0000 7f0f97262700 1 -- 172.21.15.3:0/1609694555 >> 172.21.15.3:0/1609694555 conn(0x7f0f900a4130 msgr2=0x7f0f900706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:21:55.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.202+0000 7f0f97262700 1 -- 172.21.15.3:0/1609694555 shutdown_connections 2023-10-03T10:21:55.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.202+0000 7f0f97262700 1 -- 172.21.15.3:0/1609694555 wait complete. 2023-10-03T10:21:55.216 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-03T10:21:55.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.353+0000 7f60766d0700 1 Processor -- start 2023-10-03T10:21:55.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.353+0000 7f60766d0700 1 -- start start 2023-10-03T10:21:55.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.353+0000 7f60766d0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6070131830 0x7f6070131c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:55.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.354+0000 7f60766d0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f607006ae80 0x7f6070139590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:55.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.354+0000 7f60766d0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607009dba0 con 0x7f607009e230 2023-10-03T10:21:55.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.354+0000 7f60766d0700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6070132140 con 0x7f607006ae80 2023-10-03T10:21:55.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.354+0000 7f60766d0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f60701313e0 con 0x7f6070131830 2023-10-03T10:21:55.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.354+0000 7f6074c6d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f607006ae80 0x7f6070139590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:55.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.354+0000 7f606ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6070131830 0x7f6070131c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:55.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.354+0000 7f606ffff700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6070131830 0x7f6070131c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:59798/0 (socket says 172.21.15.3:59798) 2023-10-03T10:21:55.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.354+0000 7f6074c6d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f607006ae80 0x7f6070139590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38938/0 (socket says 172.21.15.3:38938) 2023-10-03T10:21:55.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.354+0000 7f606ffff700 1 -- 172.21.15.3:0/2236323932 learned_addr learned my addr 172.21.15.3:0/2236323932 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:21:55.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.354+0000 7f606ffff700 1 -- 172.21.15.3:0/2236323932 >> v2:172.21.15.3:3300/0 conn(0x7f607006ae80 msgr2=0x7f6070139590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:21:55.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.354+0000 7f606ffff700 1 --2- 172.21.15.3:0/2236323932 >> v2:172.21.15.3:3300/0 conn(0x7f607006ae80 0x7f6070139590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:55.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.354+0000 7f606ffff700 1 -- 172.21.15.3:0/2236323932 >> v1:172.21.15.3:6789/0 conn(0x7f607009e230 legacy=0x7f607006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:55.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.354+0000 7f606ffff700 1 -- 172.21.15.3:0/2236323932 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607009d6d0 con 0x7f6070131830 2023-10-03T10:21:55.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.354+0000 7f6074c6d700 1 --2- 172.21.15.3:0/2236323932 >> v2:172.21.15.3:3300/0 conn(0x7f607006ae80 0x7f6070139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:21:55.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.354+0000 7f606ffff700 1 --2- 172.21.15.3:0/2236323932 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6070131830 0x7f6070131c00 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f60600098f0 tx=0x7f6060009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b21a5da1f98bbab5 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:21:55.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.354+0000 7f606effd700 1 -- 172.21.15.3:0/2236323932 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6060011830 con 0x7f6070131830 2023-10-03T10:21:55.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.354+0000 7f606effd700 1 -- 172.21.15.3:0/2236323932 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6060011990 con 0x7f6070131830 2023-10-03T10:21:55.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.354+0000 7f606effd700 1 -- 172.21.15.3:0/2236323932 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6060011d20 con 0x7f6070131830 2023-10-03T10:21:55.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.355+0000 7f60766d0700 1 -- 172.21.15.3:0/2236323932 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6070131830 msgr2=0x7f6070131c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:55.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.355+0000 7f60766d0700 1 --2- 172.21.15.3:0/2236323932 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6070131830 0x7f6070131c00 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f60600098f0 tx=0x7f6060009c00 comp rx=0 tx=0).stop 2023-10-03T10:21:55.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.355+0000 7f60766d0700 1 -- 172.21.15.3:0/2236323932 shutdown_connections 2023-10-03T10:21:55.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.355+0000 7f60766d0700 1 --2- 172.21.15.3:0/2236323932 >> v2:172.21.15.3:3300/0 conn(0x7f607006ae80 0x7f6070139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:55.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.355+0000 7f60766d0700 1 --2- 172.21.15.3:0/2236323932 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f6070131830 0x7f6070131c00 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:55.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.355+0000 7f60766d0700 1 -- 172.21.15.3:0/2236323932 >> 172.21.15.3:0/2236323932 conn(0x7f6070099c20 msgr2=0x7f60700a2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:21:55.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.355+0000 7f60766d0700 1 -- 172.21.15.3:0/2236323932 shutdown_connections 2023-10-03T10:21:55.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.355+0000 7f60766d0700 1 -- 172.21.15.3:0/2236323932 wait complete. 2023-10-03T10:21:55.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.356+0000 7f60766d0700 1 Processor -- start 2023-10-03T10:21:55.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.356+0000 7f60766d0700 1 -- start start 2023-10-03T10:21:55.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.357+0000 7f60766d0700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f607006ae80 0x7f6070070ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:55.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.357+0000 7f60766d0700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f607009e230 0x7f6070071000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:55.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.357+0000 7f60766d0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6070131830 0x7f6070071540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:21:55.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.357+0000 7f60766d0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607012b8a0 con 0x7f6070131830 2023-10-03T10:21:55.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.357+0000 7f60766d0700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f607009ab50 con 0x7f607006ae80 2023-10-03T10:21:55.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.357+0000 7f60766d0700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f60700727a0 con 0x7f607009e230 2023-10-03T10:21:55.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.357+0000 7f606f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f607009e230 0x7f6070071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:55.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.357+0000 7f6074c6d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6070131830 0x7f6070071540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:55.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.357+0000 7f606ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f607006ae80 0x7f6070070ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:21:55.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.357+0000 7f606f7fe700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f607009e230 0x7f6070071000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:59802/0 (socket says 172.21.15.3:59802) 2023-10-03T10:21:55.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.357+0000 7f606ffff700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f607006ae80 0x7f6070070ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:46280/0 (socket says 172.21.15.3:46280) 2023-10-03T10:21:55.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.357+0000 7f606f7fe700 1 -- 172.21.15.3:0/1942415318 learned_addr learned my addr 172.21.15.3:0/1942415318 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:21:55.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.357+0000 7f606ffff700 1 -- 172.21.15.3:0/1942415318 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f607009e230 msgr2=0x7f6070071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:55.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.357+0000 7f606ffff700 1 --2- 172.21.15.3:0/1942415318 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f607009e230 0x7f6070071000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:55.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.357+0000 7f606ffff700 1 -- 172.21.15.3:0/1942415318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6070131830 msgr2=0x7f6070071540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:55.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.357+0000 7f606ffff700 1 --2- 172.21.15.3:0/1942415318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6070131830 0x7f6070071540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:55.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.357+0000 7f606ffff700 1 -- 172.21.15.3:0/1942415318 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6068009a70 con 0x7f607006ae80 2023-10-03T10:21:55.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.357+0000 7f6074c6d700 1 --2- 172.21.15.3:0/1942415318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6070131830 0x7f6070071540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:21:55.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.357+0000 7f606f7fe700 1 --2- 172.21.15.3:0/1942415318 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f607009e230 0x7f6070071000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:21:55.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.358+0000 7f606ffff700 1 --2- 172.21.15.3:0/1942415318 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f607006ae80 0x7f6070070ac0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f6060016000 tx=0x7f6060004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:21:55.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.358+0000 7f606cff9700 1 -- 172.21.15.3:0/1942415318 <== mon.1 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60600043c0 con 0x7f607006ae80 2023-10-03T10:21:55.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.358+0000 7f606cff9700 1 -- 172.21.15.3:0/1942415318 <== mon.1 v2:172.21.15.3:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606001f070 con 0x7f607006ae80 2023-10-03T10:21:55.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.358+0000 7f60766d0700 1 -- 172.21.15.3:0/1942415318 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6060009550 con 0x7f607006ae80 2023-10-03T10:21:55.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.358+0000 7f606cff9700 1 -- 172.21.15.3:0/1942415318 <== mon.1 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60600036a0 con 0x7f607006ae80 2023-10-03T10:21:55.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.358+0000 7f60766d0700 1 -- 172.21.15.3:0/1942415318 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607009ba70 con 0x7f607006ae80 2023-10-03T10:21:55.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.359+0000 7f606cff9700 1 -- 172.21.15.3:0/1942415318 <== mon.1 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6060021030 con 0x7f607006ae80 2023-10-03T10:21:55.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.359+0000 7f60766d0700 1 -- 172.21.15.3:0/1942415318 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f607004f2a0 con 0x7f607006ae80 2023-10-03T10:21:55.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.359+0000 7f606cff9700 1 -- 172.21.15.3:0/1942415318 <== mon.1 v2:172.21.15.3:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7f6060040950 con 0x7f607006ae80 2023-10-03T10:21:55.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.364+0000 7f606cff9700 1 -- 172.21.15.3:0/1942415318 <== mon.1 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f60600239c0 con 0x7f607006ae80 2023-10-03T10:21:55.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.583+0000 7f60766d0700 1 -- 172.21.15.3:0/1942415318 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f60700a2ed0 con 0x7f607006ae80 2023-10-03T10:21:55.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.583+0000 7f606cff9700 1 -- 172.21.15.3:0/1942415318 <== mon.1 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 11 entries v40) v1 ==== 81+0+628 (secure 0 0 0) 0x7f6060015070 con 0x7f607006ae80 2023-10-03T10:21:55.584 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/3264604155 2023-10-04T10:13:12.182591+0000 2023-10-03T10:21:55.584 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1911521769 2023-10-04T10:13:12.182591+0000 2023-10-03T10:21:55.584 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/3289486158 2023-10-04T10:13:12.182591+0000 2023-10-03T10:21:55.584 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6816/118296 2023-10-04T10:12:13.305411+0000 2023-10-03T10:21:55.584 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/3246155339 2023-10-04T10:13:12.182591+0000 2023-10-03T10:21:55.584 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2091814084 2023-10-04T10:12:13.305411+0000 2023-10-03T10:21:55.584 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6817/2582564779 2023-10-04T10:12:38.734689+0000 2023-10-03T10:21:55.585 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/3571619151 2023-10-04T10:13:12.182591+0000 2023-10-03T10:21:55.585 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/3919724564 2023-10-04T10:12:13.305411+0000 2023-10-03T10:21:55.585 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6816/127025 2023-10-04T10:13:12.182591+0000 2023-10-03T10:21:55.585 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2059330378 2023-10-04T10:13:12.182591+0000 2023-10-03T10:21:55.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.585+0000 7f60766d0700 1 -- 172.21.15.3:0/1942415318 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f607006ae80 msgr2=0x7f6070070ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:21:55.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.585+0000 7f60766d0700 1 --2- 172.21.15.3:0/1942415318 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f607006ae80 0x7f6070070ac0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f6060016000 tx=0x7f6060004750 comp rx=0 tx=0).stop 2023-10-03T10:21:55.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.586+0000 7f60766d0700 1 -- 172.21.15.3:0/1942415318 shutdown_connections 2023-10-03T10:21:55.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.586+0000 7f60766d0700 1 --2- 172.21.15.3:0/1942415318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6070131830 0x7f6070071540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:55.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.586+0000 7f60766d0700 1 --2- 172.21.15.3:0/1942415318 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f607006ae80 0x7f6070070ac0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:55.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.586+0000 7f60766d0700 1 --2- 172.21.15.3:0/1942415318 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7f607009e230 0x7f6070071000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:21:55.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.586+0000 7f60766d0700 1 -- 172.21.15.3:0/1942415318 >> 172.21.15.3:0/1942415318 conn(0x7f6070099c20 msgr2=0x7f607012f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:21:55.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.586+0000 7f60766d0700 1 -- 172.21.15.3:0/1942415318 shutdown_connections 2023-10-03T10:21:55.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:21:55.586+0000 7f60766d0700 1 -- 172.21.15.3:0/1942415318 wait complete. 2023-10-03T10:21:55.587 INFO:teuthology.orchestra.run.smithi003.stderr:listed 11 entries 2023-10-03T10:21:55.600 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi022.front.sepia.ceph.com... 2023-10-03T10:21:55.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-03T10:21:55.600 DEBUG:teuthology.orchestra.run.smithi022:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-03T10:21:55.647 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:ceph-fuse[112317]: fuse finished with error 0 and tester_r 0 2023-10-03T10:21:55.648 INFO:teuthology.orchestra.run:waiting for 300 2023-10-03T10:22:01.749 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2023-10-03T10:22:01.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-03T10:22:01.750 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-03T10:22:01.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-03T10:22:01.785 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2023-10-03T10:22:01.849 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-03T10:22:01.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-03T10:22:01.850 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-03T10:22:01.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-03T10:22:01.938 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link delete ceph-brx 2023-10-03T10:22:02.025 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-03T10:22:02.036 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-03T10:22:02.037 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-03T10:22:02.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.182+0000 7fb498dde700 1 Processor -- start 2023-10-03T10:22:02.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.182+0000 7fb498dde700 1 -- start start 2023-10-03T10:22:02.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.182+0000 7fb498dde700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb49409f410 0x7fb494136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:22:02.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.182+0000 7fb498dde700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb494136fe0 0x7fb4941393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:22:02.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.182+0000 7fb498dde700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb494070940 con 0x7fb49409e930 2023-10-03T10:22:02.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.182+0000 7fb498dde700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb49406fd60 con 0x7fb494136fe0 2023-10-03T10:22:02.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.182+0000 7fb498dde700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb49406fa80 con 0x7fb49409f410 2023-10-03T10:22:02.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.182+0000 7fb492d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb494136fe0 0x7fb4941393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:22:02.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.182+0000 7fb491d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb49409f410 0x7fb494136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:22:02.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.182+0000 7fb492d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb494136fe0 0x7fb4941393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38958/0 (socket says 172.21.15.3:38958) 2023-10-03T10:22:02.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.182+0000 7fb491d9b700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb49409f410 0x7fb494136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3302/0 says I am v2:172.21.15.3:59812/0 (socket says 172.21.15.3:59812) 2023-10-03T10:22:02.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.183+0000 7fb492d9d700 1 -- 172.21.15.3:0/2374105088 learned_addr learned my addr 172.21.15.3:0/2374105088 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:22:02.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.183+0000 7fb491d9b700 1 -- 172.21.15.3:0/2374105088 >> v2:172.21.15.3:3300/0 conn(0x7fb494136fe0 msgr2=0x7fb4941393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-03T10:22:02.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.183+0000 7fb491d9b700 1 --2- 172.21.15.3:0/2374105088 >> v2:172.21.15.3:3300/0 conn(0x7fb494136fe0 0x7fb4941393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:22:02.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.183+0000 7fb491d9b700 1 -- 172.21.15.3:0/2374105088 >> v1:172.21.15.3:6789/0 conn(0x7fb49409e930 legacy=0x7fb49409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:22:02.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.183+0000 7fb491d9b700 1 -- 172.21.15.3:0/2374105088 --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb49409fb60 con 0x7fb49409f410 2023-10-03T10:22:02.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.183+0000 7fb492d9d700 1 --2- 172.21.15.3:0/2374105088 >> v2:172.21.15.3:3300/0 conn(0x7fb494136fe0 0x7fb4941393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:22:02.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.183+0000 7fb491d9b700 1 --2- 172.21.15.3:0/2374105088 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb49409f410 0x7fb494136aa0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fb488009840 tx=0x7fb488009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33578c32a40d160f server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:22:02.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.183+0000 7fb49159a700 1 -- 172.21.15.3:0/2374105088 <== mon.2 v2:172.21.15.3:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb488011830 con 0x7fb49409f410 2023-10-03T10:22:02.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.183+0000 7fb49159a700 1 -- 172.21.15.3:0/2374105088 <== mon.2 v2:172.21.15.3:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb488011990 con 0x7fb49409f410 2023-10-03T10:22:02.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.183+0000 7fb49159a700 1 -- 172.21.15.3:0/2374105088 <== mon.2 v2:172.21.15.3:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb488011d20 con 0x7fb49409f410 2023-10-03T10:22:02.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.184+0000 7fb498dde700 1 -- 172.21.15.3:0/2374105088 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb49409f410 msgr2=0x7fb494136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:22:02.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.184+0000 7fb498dde700 1 --2- 172.21.15.3:0/2374105088 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb49409f410 0x7fb494136aa0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fb488009840 tx=0x7fb488009b50 comp rx=0 tx=0).stop 2023-10-03T10:22:02.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.184+0000 7fb498dde700 1 -- 172.21.15.3:0/2374105088 shutdown_connections 2023-10-03T10:22:02.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.184+0000 7fb498dde700 1 --2- 172.21.15.3:0/2374105088 >> v2:172.21.15.3:3300/0 conn(0x7fb494136fe0 0x7fb4941393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:22:02.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.184+0000 7fb498dde700 1 --2- 172.21.15.3:0/2374105088 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb49409f410 0x7fb494136aa0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:22:02.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.184+0000 7fb498dde700 1 -- 172.21.15.3:0/2374105088 >> 172.21.15.3:0/2374105088 conn(0x7fb4940a4210 msgr2=0x7fb4940a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-03T10:22:02.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.184+0000 7fb498dde700 1 -- 172.21.15.3:0/2374105088 shutdown_connections 2023-10-03T10:22:02.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.184+0000 7fb498dde700 1 -- 172.21.15.3:0/2374105088 wait complete. 2023-10-03T10:22:02.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.185+0000 7fb498dde700 1 Processor -- start 2023-10-03T10:22:02.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.185+0000 7fb498dde700 1 -- start start 2023-10-03T10:22:02.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.186+0000 7fb498dde700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb49409e930 0x7fb49411c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:22:02.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.186+0000 7fb498dde700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb49409f410 0x7fb49411cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:22:02.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.186+0000 7fb498dde700 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb494136fe0 0x7fb49411d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-03T10:22:02.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.186+0000 7fb498dde700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49411fe50 con 0x7fb49409f410 2023-10-03T10:22:02.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.186+0000 7fb498dde700 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4940a2f40 con 0x7fb494136fe0 2023-10-03T10:22:02.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.186+0000 7fb498dde700 1 -- --> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb49411f8a0 con 0x7fb49409e930 2023-10-03T10:22:02.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.186+0000 7fb491d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb49409f410 0x7fb49411cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:22:02.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.186+0000 7fb49259c700 1 --2- >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb49409e930 0x7fb49411c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:22:02.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.186+0000 7fb491d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb49409f410 0x7fb49411cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38962/0 (socket says 172.21.15.3:38962) 2023-10-03T10:22:02.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.186+0000 7fb491d9b700 1 -- 172.21.15.3:0/1404176117 learned_addr learned my addr 172.21.15.3:0/1404176117 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-10-03T10:22:02.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.186+0000 7fb492d9d700 1 --2- 172.21.15.3:0/1404176117 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb494136fe0 0x7fb49411d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-03T10:22:02.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.186+0000 7fb491d9b700 1 -- 172.21.15.3:0/1404176117 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb49409e930 msgr2=0x7fb49411c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:22:02.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.186+0000 7fb491d9b700 1 --2- 172.21.15.3:0/1404176117 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb49409e930 0x7fb49411c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:22:02.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.186+0000 7fb491d9b700 1 -- 172.21.15.3:0/1404176117 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb494136fe0 msgr2=0x7fb49411d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-03T10:22:02.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.186+0000 7fb491d9b700 1 --2- 172.21.15.3:0/1404176117 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb494136fe0 0x7fb49411d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-03T10:22:02.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.186+0000 7fb491d9b700 1 -- 172.21.15.3:0/1404176117 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c009b30 con 0x7fb49409f410 2023-10-03T10:22:02.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.186+0000 7fb492d9d700 1 --2- 172.21.15.3:0/1404176117 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb494136fe0 0x7fb49411d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:22:02.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.186+0000 7fb49259c700 1 --2- 172.21.15.3:0/1404176117 >> [v2:172.21.15.3:3302/0,v1:172.21.15.3:6791/0] conn(0x7fb49409e930 0x7fb49411c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-03T10:22:02.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.186+0000 7fb491d9b700 1 --2- 172.21.15.3:0/1404176117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb49409f410 0x7fb49411cba0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fb488009fd0 tx=0x7fb488004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-03T10:22:02.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.186+0000 7fb482ffd700 1 -- 172.21.15.3:0/1404176117 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4880036a0 con 0x7fb49409f410 2023-10-03T10:22:02.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.186+0000 7fb482ffd700 1 -- 172.21.15.3:0/1404176117 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48801f070 con 0x7fb49409f410 2023-10-03T10:22:02.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.186+0000 7fb498dde700 1 -- 172.21.15.3:0/1404176117 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4880094a0 con 0x7fb49409f410 2023-10-03T10:22:02.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.186+0000 7fb482ffd700 1 -- 172.21.15.3:0/1404176117 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb488003a60 con 0x7fb49409f410 2023-10-03T10:22:02.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.187+0000 7fb498dde700 1 -- 172.21.15.3:0/1404176117 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb49411e330 con 0x7fb49409f410 2023-10-03T10:22:02.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.187+0000 7fb482ffd700 1 -- 172.21.15.3:0/1404176117 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb488003bc0 con 0x7fb49409f410 2023-10-03T10:22:02.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.187+0000 7fb482ffd700 1 -- 172.21.15.3:0/1404176117 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4422+0+0 (secure 0 0 0) 0x7fb488021070 con 0x7fb49409f410 2023-10-03T10:22:02.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.188+0000 7fb498dde700 1 -- 172.21.15.3:0/1404176117 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb494132500 con 0x7fb49409f410 2023-10-03T10:22:02.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-10-03T10:22:02.193+0000 7fb482ffd700 1 -- 172.21.15.3:0/1404176117 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb488015020 con 0x7fb49409f410 2023-10-03T10:24:02.081 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-10-03T10:24:02.081 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi003 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-10-03T10:24:02.091 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-03T10:24:02.092 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-03T10:24:02.092 INFO:teuthology.orchestra.run:waiting for 300 2023-10-03T10:24:02.105 INFO:tasks.ceph.mds.a:Stopped 2023-10-03T10:24:02.106 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-03T10:24:02.106 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-03T10:24:02.106 INFO:teuthology.orchestra.run:waiting for 300 2023-10-03T10:24:02.177 INFO:tasks.ceph.osd.0:Stopped 2023-10-03T10:24:02.177 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-03T10:24:02.178 INFO:teuthology.orchestra.run:waiting for 300 2023-10-03T10:24:02.244 INFO:tasks.ceph.osd.1:Stopped 2023-10-03T10:24:02.245 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-03T10:24:02.245 INFO:teuthology.orchestra.run:waiting for 300 2023-10-03T10:24:02.312 INFO:tasks.ceph.osd.2:Stopped 2023-10-03T10:24:02.313 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-03T10:24:02.313 INFO:teuthology.orchestra.run:waiting for 300 2023-10-03T10:24:02.383 INFO:tasks.ceph.osd.3:Stopped 2023-10-03T10:24:02.383 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-03T10:24:02.384 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-03T10:24:02.384 INFO:teuthology.orchestra.run:waiting for 300 2023-10-03T10:24:02.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-03T10:24:02.384 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi003 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi003 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x' 2023-10-03T10:24:02.395 INFO:tasks.ceph.mgr.x:Stopped 2023-10-03T10:24:02.396 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-03T10:24:02.396 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-03T10:24:02.396 INFO:teuthology.orchestra.run:waiting for 300 2023-10-03T10:24:02.437 INFO:tasks.ceph.mon.a:Stopped 2023-10-03T10:24:02.437 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-03T10:24:02.437 INFO:teuthology.orchestra.run:waiting for 300 2023-10-03T10:24:02.491 INFO:tasks.ceph.mon.b:Stopped 2023-10-03T10:24:02.491 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-03T10:24:02.491 INFO:teuthology.orchestra.run:waiting for 300 2023-10-03T10:24:02.544 INFO:tasks.ceph.mon.c:Stopped 2023-10-03T10:24:02.544 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-03T10:24:02.545 DEBUG:teuthology.orchestra.run.smithi003:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'overall HEALTH_' | egrep -v '\(FS_' | egrep -v '\(MDS_' | egrep -v '\(OSD_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(CACHE_POOL_' | egrep -v '\(POOL_' | egrep -v '\(MGR_DOWN\)' | egrep -v '\(PG_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v 'Monitor daemon marked osd' | egrep -v 'Behind on trimming' | egrep -v 'Manager daemon' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'scrub mismatch' | egrep -v ScrubResult | egrep -v 'wrongly marked' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'overall HEALTH_' | egrep -v '\(MON_MSGR2_NOT_ENABLED\)' | egrep -v 'slow request' | head -n 1 2023-10-03T10:24:02.635 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi003.front.sepia.ceph.com 2023-10-03T10:24:02.635 DEBUG:teuthology.orchestra.run.smithi003:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-03T10:24:02.846 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi003.front.sepia.ceph.com 2023-10-03T10:24:02.846 DEBUG:teuthology.orchestra.run.smithi003:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-03T10:24:02.994 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi003.front.sepia.ceph.com 2023-10-03T10:24:02.995 DEBUG:teuthology.orchestra.run.smithi003:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-03T10:24:03.078 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi003.front.sepia.ceph.com 2023-10-03T10:24:03.078 DEBUG:teuthology.orchestra.run.smithi003:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-03T10:24:03.215 INFO:tasks.ceph:Archiving mon data... 2023-10-03T10:24:03.216 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/dparmar-2023-10-03_09:48:43-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7409565/data/mon.a.tgz 2023-10-03T10:24:03.216 DEBUG:teuthology.orchestra.run.smithi003:> mktemp 2023-10-03T10:24:03.239 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.R1XuhchtFU 2023-10-03T10:24:03.240 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.R1XuhchtFU 2023-10-03T10:24:03.441 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0666 /tmp/tmp.R1XuhchtFU 2023-10-03T10:24:03.538 DEBUG:teuthology.orchestra.remote:smithi003:/tmp/tmp.R1XuhchtFU is 337KB 2023-10-03T10:24:03.635 DEBUG:teuthology.orchestra.run.smithi003:> rm -fr /tmp/tmp.R1XuhchtFU 2023-10-03T10:24:03.657 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/dparmar-2023-10-03_09:48:43-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7409565/data/mon.b.tgz 2023-10-03T10:24:03.658 DEBUG:teuthology.orchestra.run.smithi003:> mktemp 2023-10-03T10:24:03.720 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.70L2lLt9j8 2023-10-03T10:24:03.720 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.70L2lLt9j8 2023-10-03T10:24:03.864 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0666 /tmp/tmp.70L2lLt9j8 2023-10-03T10:24:03.959 DEBUG:teuthology.orchestra.remote:smithi003:/tmp/tmp.70L2lLt9j8 is 340KB 2023-10-03T10:24:04.054 DEBUG:teuthology.orchestra.run.smithi003:> rm -fr /tmp/tmp.70L2lLt9j8 2023-10-03T10:24:04.077 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/dparmar-2023-10-03_09:48:43-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7409565/data/mon.c.tgz 2023-10-03T10:24:04.077 DEBUG:teuthology.orchestra.run.smithi003:> mktemp 2023-10-03T10:24:04.140 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.MErkATt7Dq 2023-10-03T10:24:04.140 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.MErkATt7Dq 2023-10-03T10:24:04.283 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0666 /tmp/tmp.MErkATt7Dq 2023-10-03T10:24:04.377 DEBUG:teuthology.orchestra.remote:smithi003:/tmp/tmp.MErkATt7Dq is 340KB 2023-10-03T10:24:04.471 DEBUG:teuthology.orchestra.run.smithi003:> rm -fr /tmp/tmp.MErkATt7Dq 2023-10-03T10:24:04.494 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-03T10:24:04.495 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-03T10:24:04.537 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-03T10:24:04.575 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-03T10:24:04.575 DEBUG:teuthology.orchestra.run.smithi003:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-03T10:24:04.613 DEBUG:teuthology.orchestra.run.smithi022:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-03T10:24:04.646 INFO:teuthology.orchestra.run.smithi003.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-03T10:24:04.653 INFO:teuthology.orchestra.run.smithi022.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-03T10:24:04.660 INFO:tasks.ceph:Archiving crash dumps... 2023-10-03T10:24:04.662 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/lib/ceph/crash to /home/teuthworker/archive/dparmar-2023-10-03_09:48:43-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7409565/remote/smithi003/crash 2023-10-03T10:24:04.663 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-03T10:24:04.817 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/lib/ceph/crash to /home/teuthworker/archive/dparmar-2023-10-03_09:48:43-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7409565/remote/smithi022/crash 2023-10-03T10:24:04.818 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-03T10:24:04.865 INFO:tasks.ceph:Compressing logs... 2023-10-03T10:24:04.865 DEBUG:teuthology.orchestra.run.smithi003:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-03T10:24:04.868 DEBUG:teuthology.orchestra.run.smithi022:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-03T10:24:04.902 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163851.log 2023-10-03T10:24:04.902 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141238.log 2023-10-03T10:24:04.903 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.112317.log 2023-10-03T10:24:04.910 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.112317.log: 96.6% -- replaced with /var/log/ceph/ceph-client.0.112317.log.gz 2023-10-03T10:24:04.917 INFO:teuthology.orchestra.run.smithi022.stderr: 2023-10-03T10:24:04.918 INFO:teuthology.orchestra.run.smithi022.stderr:real 0m0.022s 2023-10-03T10:24:04.918 INFO:teuthology.orchestra.run.smithi022.stderr:user 0m0.019s 2023-10-03T10:24:04.918 INFO:teuthology.orchestra.run.smithi022.stderr:sys 0m0.013s 2023-10-03T10:24:04.976 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.163851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164145.log 2023-10-03T10:24:04.976 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.163851.log.gz 2023-10-03T10:24:04.976 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.141238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154525.log 2023-10-03T10:24:04.976 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.141238.log.gz 2023-10-03T10:24:04.976 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.164145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183568.log 2023-10-03T10:24:04.976 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.154525.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.164145.log.gz 2023-10-03T10:24:04.977 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171223.log 2023-10-03T10:24:04.977 INFO:teuthology.orchestra.run.smithi003.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.154525.log.gz 2023-10-03T10:24:04.977 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.183568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123123.log 2023-10-03T10:24:04.977 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.183568.log.gz 2023-10-03T10:24:04.977 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.171223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170811.log 2023-10-03T10:24:04.977 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.123123.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.171223.log.gz 2023-10-03T10:24:04.977 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140846.log 2023-10-03T10:24:04.977 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.123123.log.gz 2023-10-03T10:24:04.978 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.170811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125432.log 2023-10-03T10:24:04.978 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.170811.log.gz 2023-10-03T10:24:04.978 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.140846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193353.log 2023-10-03T10:24:04.978 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.125432.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.140846.log.gz 2023-10-03T10:24:04.978 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180406.log 2023-10-03T10:24:04.978 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.193353.log: gzip -5 --verbose 90.8%-- /var/log/ceph/ceph-client.admin.176452.log -- replaced with /var/log/ceph/ceph-client.admin.125432.log.gz 2023-10-03T10:24:04.978 INFO:teuthology.orchestra.run.smithi003.stderr: 2023-10-03T10:24:04.978 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.193353.log.gz 2023-10-03T10:24:04.979 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.180406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147444.log 2023-10-03T10:24:04.979 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.176452.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.180406.log.gz 2023-10-03T10:24:04.979 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175329.log 2023-10-03T10:24:04.979 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.176452.log.gz 2023-10-03T10:24:04.979 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.147444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179026.log 2023-10-03T10:24:04.979 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.147444.log.gz 2023-10-03T10:24:04.979 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.175329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180800.log 2023-10-03T10:24:04.980 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.175329.log.gz 2023-10-03T10:24:04.980 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.179026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142487.log 2023-10-03T10:24:04.980 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.179026.log.gz 2023-10-03T10:24:04.980 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.180800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127300.log 2023-10-03T10:24:04.980 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.180800.log.gz 2023-10-03T10:24:04.980 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.142487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179733.log 2023-10-03T10:24:04.980 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.142487.log.gz 2023-10-03T10:24:04.980 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.127300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140336.log 2023-10-03T10:24:04.980 INFO:teuthology.orchestra.run.smithi003.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.127300.log.gz 2023-10-03T10:24:04.981 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.179733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154722.log 2023-10-03T10:24:04.981 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.179733.log.gz 2023-10-03T10:24:04.981 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.140336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162744.log 2023-10-03T10:24:04.981 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.154722.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.140336.log.gz 2023-10-03T10:24:04.981 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144962.log 2023-10-03T10:24:04.981 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.154722.log.gz 2023-10-03T10:24:04.981 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.162744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145991.log 2023-10-03T10:24:04.981 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.162744.log.gz 2023-10-03T10:24:04.982 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.144962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195209.log 2023-10-03T10:24:04.982 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.145991.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.144962.log.gz 2023-10-03T10:24:04.982 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191662.log 2023-10-03T10:24:04.982 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.145991.log.gz 2023-10-03T10:24:04.982 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.195209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160903.log 2023-10-03T10:24:04.982 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.195209.log.gz 2023-10-03T10:24:04.982 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.191662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151095.log 2023-10-03T10:24:04.983 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.160903.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.191662.log.gz 2023-10-03T10:24:04.983 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171691.log 2023-10-03T10:24:04.983 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.160903.log.gz 2023-10-03T10:24:04.983 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.151095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161195.log 2023-10-03T10:24:04.983 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.151095.log.gz 2023-10-03T10:24:04.983 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.171691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171103.log 2023-10-03T10:24:04.983 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.171691.log.gz 2023-10-03T10:24:04.983 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.161195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136015.log 2023-10-03T10:24:04.984 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.161195.log.gz 2023-10-03T10:24:04.984 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.171103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187293.log 2023-10-03T10:24:04.984 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.171103.log.gz 2023-10-03T10:24:04.984 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.136015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147202.log 2023-10-03T10:24:04.984 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.187293.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.136015.log.gz 2023-10-03T10:24:04.984 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174051.log 2023-10-03T10:24:04.984 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.187293.log.gz 2023-10-03T10:24:04.984 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.147202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152906.log 2023-10-03T10:24:04.985 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.147202.log.gz 2023-10-03T10:24:04.985 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.174051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166007.log 2023-10-03T10:24:04.985 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.174051.log.gz 2023-10-03T10:24:04.985 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.152906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149161.log 2023-10-03T10:24:04.985 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.152906.log.gz 2023-10-03T10:24:04.985 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.166007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125952.log 2023-10-03T10:24:04.985 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.149161.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.166007.log.gz 2023-10-03T10:24:04.985 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189703.log 2023-10-03T10:24:04.986 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.149161.log.gz 2023-10-03T10:24:04.986 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167379.log 2023-10-03T10:24:04.986 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.125952.log: /var/log/ceph/ceph-client.admin.189703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153304.log 2023-10-03T10:24:04.986 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.125952.log.gz 2023-10-03T10:24:04.986 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.189703.log.gz 2023-10-03T10:24:04.986 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.167379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193796.log 2023-10-03T10:24:04.986 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.153304.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.167379.log.gz 2023-10-03T10:24:04.986 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151240.log 2023-10-03T10:24:04.986 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.153304.log.gz 2023-10-03T10:24:04.987 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.193796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164466.log 2023-10-03T10:24:04.987 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.193796.log.gz 2023-10-03T10:24:04.987 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.151240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184129.log 2023-10-03T10:24:04.987 INFO:teuthology.orchestra.run.smithi003.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.151240.log.gz 2023-10-03T10:24:04.987 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.164466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138316.log 2023-10-03T10:24:04.987 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.164466.log.gz 2023-10-03T10:24:04.987 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.184129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162667.log 2023-10-03T10:24:04.987 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.184129.log.gz 2023-10-03T10:24:04.988 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.138316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166791.log 2023-10-03T10:24:04.988 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.138316.log.gz 2023-10-03T10:24:04.988 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.162667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148351.log 2023-10-03T10:24:04.988 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.162667.log.gz 2023-10-03T10:24:04.988 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.166791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142879.log 2023-10-03T10:24:04.988 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.166791.log.gz 2023-10-03T10:24:04.988 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.148351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177262.log 2023-10-03T10:24:04.988 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.148351.log.gz 2023-10-03T10:24:04.989 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.142879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178438.log 2023-10-03T10:24:04.989 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.142879.log.gz 2023-10-03T10:24:04.989 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.177262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177680.log 2023-10-03T10:24:04.989 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.177262.log.gz 2023-10-03T10:24:04.989 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.178438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156983.log 2023-10-03T10:24:04.989 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.178438.log.gz 2023-10-03T10:24:04.989 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 /var/log/ceph/ceph-client.admin.177680.log: --verbose -- /var/log/ceph/ceph-client.admin.141068.log 2023-10-03T10:24:04.989 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.156983.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.177680.log.gz 2023-10-03T10:24:04.990 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189015.log 2023-10-03T10:24:04.990 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.156983.log.gz 2023-10-03T10:24:04.990 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.141068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124499.log 2023-10-03T10:24:04.990 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.141068.log.gz 2023-10-03T10:24:04.990 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138169.log 2023-10-03T10:24:04.990 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.189015.log: /var/log/ceph/ceph-client.admin.124499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124425.log 2023-10-03T10:24:04.990 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.189015.log.gz 2023-10-03T10:24:04.990 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.138169.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.124499.log.gz 2023-10-03T10:24:04.991 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185110.log 2023-10-03T10:24:04.991 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.138169.log.gz 2023-10-03T10:24:04.991 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.124425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175668.log 2023-10-03T10:24:04.991 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.124425.log.gz 2023-10-03T10:24:04.991 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.185110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183615.log 2023-10-03T10:24:04.991 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.185110.log.gz 2023-10-03T10:24:04.991 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.175668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139474.log 2023-10-03T10:24:04.991 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.175668.log.gz 2023-10-03T10:24:04.992 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.183615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144617.log 2023-10-03T10:24:04.992 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.183615.log.gz 2023-10-03T10:24:04.992 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.139474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154082.log 2023-10-03T10:24:04.992 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.139474.log.gz 2023-10-03T10:24:04.992 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.144617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184595.log 2023-10-03T10:24:04.992 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.144617.log.gz 2023-10-03T10:24:04.992 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.154082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189263.log 2023-10-03T10:24:04.992 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.154082.log.gz 2023-10-03T10:24:04.992 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.184595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191271.log 2023-10-03T10:24:04.993 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.184595.log.gz 2023-10-03T10:24:04.993 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.189263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129804.log 2023-10-03T10:24:04.993 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.189263.log.gz 2023-10-03T10:24:04.993 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.191271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173585.log 2023-10-03T10:24:04.993 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.191271.log.gz 2023-10-03T10:24:04.993 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.129804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146410.log 2023-10-03T10:24:04.993 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.129804.log.gz 2023-10-03T10:24:04.993 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.173585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125285.log 2023-10-03T10:24:04.994 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.173585.log.gz 2023-10-03T10:24:04.994 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.146410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187541.log 2023-10-03T10:24:04.994 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.146410.log.gz 2023-10-03T10:24:04.994 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.125285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191001.log 2023-10-03T10:24:04.994 INFO:teuthology.orchestra.run.smithi003.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.125285.log.gz 2023-10-03T10:24:04.994 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.187541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148994.log 2023-10-03T10:24:04.994 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.187541.log.gz 2023-10-03T10:24:04.994 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.191001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167648.log 2023-10-03T10:24:04.995 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.191001.log.gz 2023-10-03T10:24:04.995 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.148994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150458.log 2023-10-03T10:24:04.995 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.167648.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.148994.log.gz 2023-10-03T10:24:04.995 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159805.log 2023-10-03T10:24:04.995 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.167648.log.gz 2023-10-03T10:24:04.995 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.150458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142069.log 2023-10-03T10:24:04.995 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.150458.log.gz 2023-10-03T10:24:04.995 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.159805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160315.log 2023-10-03T10:24:04.996 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.142069.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.159805.log.gz 2023-10-03T10:24:04.996 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147371.log 2023-10-03T10:24:04.996 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.142069.log.gz 2023-10-03T10:24:04.996 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.160315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169927.log 2023-10-03T10:24:04.996 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.160315.log.gz 2023-10-03T10:24:04.996 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.147371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189407.log 2023-10-03T10:24:04.996 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.169927.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.147371.log.gz 2023-10-03T10:24:04.996 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186752.log 2023-10-03T10:24:04.996 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.169927.log.gz 2023-10-03T10:24:04.997 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.189407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169143.log 2023-10-03T10:24:04.997 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.189407.log.gz 2023-10-03T10:24:04.997 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.186752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161002.log 2023-10-03T10:24:04.997 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.169143.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.186752.log.gz 2023-10-03T10:24:04.997 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151628.log 2023-10-03T10:24:04.997 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.169143.log.gz 2023-10-03T10:24:04.997 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.161002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179248.log 2023-10-03T10:24:04.997 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.161002.log.gz 2023-10-03T10:24:04.998 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.151628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173191.log 2023-10-03T10:24:04.998 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.179248.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.151628.log.gz 2023-10-03T10:24:04.998 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188911.log 2023-10-03T10:24:04.998 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.179248.log.gz 2023-10-03T10:24:04.998 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.173191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125552.log 2023-10-03T10:24:04.998 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.173191.log.gz 2023-10-03T10:24:04.998 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.188911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172754.log 2023-10-03T10:24:04.998 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.125552.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.188911.log.gz 2023-10-03T10:24:04.999 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157571.log 2023-10-03T10:24:04.999 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.125552.log.gz 2023-10-03T10:24:04.999 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.172754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161348.log 2023-10-03T10:24:04.999 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.172754.log.gz 2023-10-03T10:24:04.999 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.157571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152020.log 2023-10-03T10:24:04.999 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.161348.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.157571.log.gz 2023-10-03T10:24:04.999 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190637.log 2023-10-03T10:24:04.999 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.161348.log.gz 2023-10-03T10:24:05.000 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.152020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124085.log 2023-10-03T10:24:05.000 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.152020.log.gz 2023-10-03T10:24:05.000 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.190637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158242.log 2023-10-03T10:24:05.000 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.124085.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.190637.log.gz 2023-10-03T10:24:05.000 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162863.log 2023-10-03T10:24:05.000 INFO:teuthology.orchestra.run.smithi003.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.124085.log.gz 2023-10-03T10:24:05.000 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.158242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145009.log 2023-10-03T10:24:05.000 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.158242.log.gz 2023-10-03T10:24:05.001 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.162863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187688.log 2023-10-03T10:24:05.001 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.145009.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.162863.log.gz 2023-10-03T10:24:05.001 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171177.log 2023-10-03T10:24:05.001 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.145009.log.gz 2023-10-03T10:24:05.001 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.187688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159975.log 2023-10-03T10:24:05.001 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.187688.log.gz 2023-10-03T10:24:05.001 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.171177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122915.log 2023-10-03T10:24:05.001 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.159975.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.171177.log.gz 2023-10-03T10:24:05.002 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183812.log 2023-10-03T10:24:05.002 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.159975.log.gz 2023-10-03T10:24:05.002 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.122915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181948.log 2023-10-03T10:24:05.002 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.122915.log.gz 2023-10-03T10:24:05.002 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.183812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194189.log 2023-10-03T10:24:05.002 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.181948.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.183812.log.gz 2023-10-03T10:24:05.002 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161125.log 2023-10-03T10:24:05.002 INFO:teuthology.orchestra.run.smithi003.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.181948.log.gz 2023-10-03T10:24:05.003 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.194189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173735.log 2023-10-03T10:24:05.003 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.194189.log.gz 2023-10-03T10:24:05.003 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.161125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165296.log 2023-10-03T10:24:05.003 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.173735.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.161125.log.gz 2023-10-03T10:24:05.003 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182954.log 2023-10-03T10:24:05.003 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.173735.log.gz 2023-10-03T10:24:05.003 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.165296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166695.log 2023-10-03T10:24:05.003 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.165296.log.gz 2023-10-03T10:24:05.004 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.182954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192986.log 2023-10-03T10:24:05.004 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.166695.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.182954.log.gz 2023-10-03T10:24:05.004 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195356.log 2023-10-03T10:24:05.004 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.166695.log.gz 2023-10-03T10:24:05.004 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.192986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183988.log 2023-10-03T10:24:05.004 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.192986.log.gz 2023-10-03T10:24:05.004 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.195356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127951.log 2023-10-03T10:24:05.004 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.183988.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.195356.log.gz 2023-10-03T10:24:05.005 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148379.log 2023-10-03T10:24:05.005 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.183988.log.gz 2023-10-03T10:24:05.005 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.127951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140062.log 2023-10-03T10:24:05.005 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.127951.log.gz 2023-10-03T10:24:05.005 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195454.log 2023-10-03T10:24:05.005 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.148379.log: /var/log/ceph/ceph-client.admin.140062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156860.log 2023-10-03T10:24:05.005 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.148379.log.gz 2023-10-03T10:24:05.005 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.140062.log.gz 2023-10-03T10:24:05.005 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.195454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137898.log 2023-10-03T10:24:05.006 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.195454.log.gz 2023-10-03T10:24:05.006 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.156860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140114.log 2023-10-03T10:24:05.006 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.137898.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.156860.log.gz 2023-10-03T10:24:05.006 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127121.log 2023-10-03T10:24:05.006 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.137898.log.gz 2023-10-03T10:24:05.006 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.140114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136330.log 2023-10-03T10:24:05.006 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.140114.log.gz 2023-10-03T10:24:05.006 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.127121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123172.log 2023-10-03T10:24:05.007 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.136330.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.127121.log.gz 2023-10-03T10:24:05.007 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193327.log 2023-10-03T10:24:05.007 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.136330.log.gz 2023-10-03T10:24:05.007 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.123172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127833.log 2023-10-03T10:24:05.007 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.123172.log.gz 2023-10-03T10:24:05.007 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.193327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189603.log 2023-10-03T10:24:05.007 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.193327.log.gz 2023-10-03T10:24:05.007 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.127833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183764.log 2023-10-03T10:24:05.008 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.189603.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.127833.log.gz 2023-10-03T10:24:05.008 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147640.log 2023-10-03T10:24:05.008 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.189603.log.gz 2023-10-03T10:24:05.008 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.183764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147518.log 2023-10-03T10:24:05.008 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.147640.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.183764.log.gz 2023-10-03T10:24:05.008 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149553.log 2023-10-03T10:24:05.008 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.147640.log.gz 2023-10-03T10:24:05.008 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.147518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142610.log 2023-10-03T10:24:05.009 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.147518.log.gz 2023-10-03T10:24:05.009 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.149553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154133.log 2023-10-03T10:24:05.009 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.142610.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.149553.log.gz 2023-10-03T10:24:05.009 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127502.log 2023-10-03T10:24:05.009 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.142610.log.gz 2023-10-03T10:24:05.009 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.154133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184940.log 2023-10-03T10:24:05.009 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.154133.log.gz 2023-10-03T10:24:05.009 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.127502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172783.log 2023-10-03T10:24:05.010 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.184940.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.127502.log.gz 2023-10-03T10:24:05.010 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191563.log 2023-10-03T10:24:05.010 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.184940.log.gz 2023-10-03T10:24:05.010 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.172783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183960.log 2023-10-03T10:24:05.010 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.172783.log.gz 2023-10-03T10:24:05.010 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.191563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135352.log 2023-10-03T10:24:05.010 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.183960.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.191563.log.gz 2023-10-03T10:24:05.010 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175815.log 2023-10-03T10:24:05.011 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.183960.log.gz 2023-10-03T10:24:05.011 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.135352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127172.log 2023-10-03T10:24:05.011 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.135352.log.gz 2023-10-03T10:24:05.011 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.175815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153546.log 2023-10-03T10:24:05.011 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.127172.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.175815.log.gz 2023-10-03T10:24:05.011 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165126.log 2023-10-03T10:24:05.011 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.127172.log.gz 2023-10-03T10:24:05.012 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.153546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152663.log 2023-10-03T10:24:05.012 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.153546.log.gz 2023-10-03T10:24:05.012 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.165126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149386.log 2023-10-03T10:24:05.012 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.152663.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.165126.log.gz 2023-10-03T10:24:05.012 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136969.log 2023-10-03T10:24:05.012 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.152663.log.gz 2023-10-03T10:24:05.012 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.149386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176897.log 2023-10-03T10:24:05.012 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.149386.log.gz 2023-10-03T10:24:05.013 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.136969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177288.log 2023-10-03T10:24:05.013 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.176897.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.136969.log.gz 2023-10-03T10:24:05.013 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173538.log 2023-10-03T10:24:05.013 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.176897.log.gz 2023-10-03T10:24:05.013 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.177288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146726.log 2023-10-03T10:24:05.013 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.177288.log.gz 2023-10-03T10:24:05.013 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.173538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153839.log 2023-10-03T10:24:05.013 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.173538.log.gz 2023-10-03T10:24:05.014 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.146726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160955.log 2023-10-03T10:24:05.014 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.146726.log.gz 2023-10-03T10:24:05.014 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.153839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127906.log 2023-10-03T10:24:05.014 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.153839.log.gz 2023-10-03T10:24:05.014 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.160955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173781.log 2023-10-03T10:24:05.014 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.127906.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.160955.log.gz 2023-10-03T10:24:05.014 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169190.log 2023-10-03T10:24:05.014 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.127906.log.gz 2023-10-03T10:24:05.015 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.173781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185379.log 2023-10-03T10:24:05.015 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.173781.log.gz 2023-10-03T10:24:05.015 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.169190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195160.log 2023-10-03T10:24:05.015 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.185379.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.169190.log.gz 2023-10-03T10:24:05.015 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168042.log 2023-10-03T10:24:05.015 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.185379.log.gz 2023-10-03T10:24:05.015 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.195160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122817.log 2023-10-03T10:24:05.015 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.195160.log.gz 2023-10-03T10:24:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.168042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188623.log 2023-10-03T10:24:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.122817.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.168042.log.gz 2023-10-03T10:24:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161543.log 2023-10-03T10:24:05.016 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.122817.log.gz 2023-10-03T10:24:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.188623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189995.log 2023-10-03T10:24:05.016 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.188623.log.gz 2023-10-03T10:24:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.161543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165323.log 2023-10-03T10:24:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.189995.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.161543.log.gz 2023-10-03T10:24:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141925.log 2023-10-03T10:24:05.017 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.189995.log.gz 2023-10-03T10:24:05.017 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.165323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185084.log 2023-10-03T10:24:05.017 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.165323.log.gz 2023-10-03T10:24:05.017 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.141925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174100.log 2023-10-03T10:24:05.017 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.185084.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.141925.log.gz 2023-10-03T10:24:05.017 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161321.log 2023-10-03T10:24:05.017 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.185084.log.gz 2023-10-03T10:24:05.017 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.174100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191785.log 2023-10-03T10:24:05.018 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.174100.log.gz 2023-10-03T10:24:05.018 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.161321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140898.log 2023-10-03T10:24:05.018 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.191785.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.161321.log.gz 2023-10-03T10:24:05.018 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144447.log 2023-10-03T10:24:05.018 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.191785.log.gz 2023-10-03T10:24:05.018 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.140898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188275.log 2023-10-03T10:24:05.018 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.140898.log.gz 2023-10-03T10:24:05.018 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.144447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128361.log 2023-10-03T10:24:05.019 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.188275.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.144447.log.gz 2023-10-03T10:24:05.019 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140113.log 2023-10-03T10:24:05.019 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.188275.log.gz 2023-10-03T10:24:05.019 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.128361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163655.log 2023-10-03T10:24:05.019 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.128361.log.gz 2023-10-03T10:24:05.019 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.140113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181872.log 2023-10-03T10:24:05.019 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.140113.log.gz 2023-10-03T10:24:05.019 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.163655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157401.log 2023-10-03T10:24:05.020 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.163655.log.gz 2023-10-03T10:24:05.020 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.181872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147910.log 2023-10-03T10:24:05.020 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.157401.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.181872.log.gz 2023-10-03T10:24:05.020 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173463.log 2023-10-03T10:24:05.020 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.147910.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.157401.log.gz 2023-10-03T10:24:05.020 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147052.log 2023-10-03T10:24:05.020 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.147910.log.gz 2023-10-03T10:24:05.020 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.173463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186874.log 2023-10-03T10:24:05.021 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.147052.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.173463.log.gz 2023-10-03T10:24:05.021 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129336.log 2023-10-03T10:24:05.021 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.147052.log.gz 2023-10-03T10:24:05.021 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.186874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145795.log 2023-10-03T10:24:05.021 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.186874.log.gz 2023-10-03T10:24:05.021 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.129336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191075.log 2023-10-03T10:24:05.021 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.145795.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.129336.log.gz 2023-10-03T10:24:05.021 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150533.log 2023-10-03T10:24:05.022 INFO:teuthology.orchestra.run.smithi003.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.145795.log.gz 2023-10-03T10:24:05.022 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.191075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129507.log 2023-10-03T10:24:05.022 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.191075.log.gz 2023-10-03T10:24:05.022 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.150533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143982.log 2023-10-03T10:24:05.022 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.150533.log.gz 2023-10-03T10:24:05.022 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.129507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184548.log 2023-10-03T10:24:05.022 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.129507.log.gz 2023-10-03T10:24:05.022 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.143982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180996.log 2023-10-03T10:24:05.023 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.143982.log.gz 2023-10-03T10:24:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.184548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163111.log 2023-10-03T10:24:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.180996.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.184548.log.gz 2023-10-03T10:24:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168359.log 2023-10-03T10:24:05.023 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.180996.log.gz 2023-10-03T10:24:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.163111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159413.log 2023-10-03T10:24:05.023 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.163111.log.gz 2023-10-03T10:24:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.168359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187097.log 2023-10-03T10:24:05.024 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.168359.log.gz 2023-10-03T10:24:05.024 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.159413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136722.log 2023-10-03T10:24:05.024 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.159413.log.gz 2023-10-03T10:24:05.024 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.187097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125040.log 2023-10-03T10:24:05.024 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.187097.log.gz 2023-10-03T10:24:05.024 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.136722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182928.log 2023-10-03T10:24:05.024 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.136722.log.gz 2023-10-03T10:24:05.024 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.125040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169339.log 2023-10-03T10:24:05.024 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.125040.log.gz 2023-10-03T10:24:05.025 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.182928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183542.log 2023-10-03T10:24:05.025 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.182928.log.gz 2023-10-03T10:24:05.025 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.169339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179147.log 2023-10-03T10:24:05.025 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.183542.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.169339.log.gz 2023-10-03T10:24:05.025 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146979.log 2023-10-03T10:24:05.025 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.183542.log.gz 2023-10-03T10:24:05.025 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.179147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124889.log 2023-10-03T10:24:05.025 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.179147.log.gz 2023-10-03T10:24:05.026 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.146979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148769.log 2023-10-03T10:24:05.026 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.124889.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.146979.log.gz 2023-10-03T10:24:05.026 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171895.log 2023-10-03T10:24:05.026 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.124889.log.gz 2023-10-03T10:24:05.026 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.148769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189433.log 2023-10-03T10:24:05.026 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.148769.log.gz 2023-10-03T10:24:05.026 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.171895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188497.log 2023-10-03T10:24:05.026 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.171895.log.gz 2023-10-03T10:24:05.027 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.189433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160341.log 2023-10-03T10:24:05.027 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.188497.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.189433.log.gz 2023-10-03T10:24:05.027 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161739.log 2023-10-03T10:24:05.027 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.188497.log.gz 2023-10-03T10:24:05.027 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.160341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166892.log 2023-10-03T10:24:05.027 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.160341.log.gz/var/log/ceph/ceph-client.admin.161739.log: 2023-10-03T10:24:05.027 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156495.log 2023-10-03T10:24:05.027 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.161739.log.gz 2023-10-03T10:24:05.028 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.166892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161394.log 2023-10-03T10:24:05.028 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.166892.log.gz 2023-10-03T10:24:05.028 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.156495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140702.log 2023-10-03T10:24:05.028 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.161394.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.156495.log.gz 2023-10-03T10:24:05.028 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176207.log 2023-10-03T10:24:05.028 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.140702.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.161394.log.gz 2023-10-03T10:24:05.028 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177579.log 2023-10-03T10:24:05.028 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.140702.log.gz 2023-10-03T10:24:05.028 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.176207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143886.log 2023-10-03T10:24:05.029 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.176207.log.gz 2023-10-03T10:24:05.029 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.177579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129238.log 2023-10-03T10:24:05.029 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.143886.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.177579.log.gz 2023-10-03T10:24:05.029 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181974.log 2023-10-03T10:24:05.029 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.129238.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.143886.log.gz 2023-10-03T10:24:05.029 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160929.log 2023-10-03T10:24:05.029 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.129238.log.gz 2023-10-03T10:24:05.029 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.181974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149206.log 2023-10-03T10:24:05.030 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.181974.log.gz 2023-10-03T10:24:05.030 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.160929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161099.log 2023-10-03T10:24:05.155 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.149206.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.160929.log.gz 2023-10-03T10:24:05.155 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118528.log 2023-10-03T10:24:05.155 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.161099.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.149206.log.gz 2023-10-03T10:24:05.156 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163085.log 2023-10-03T10:24:05.156 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.161099.log.gz 2023-10-03T10:24:05.156 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.118528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170515.log 2023-10-03T10:24:05.156 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.118528.log.gz 2023-10-03T10:24:05.156 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.163085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190191.log 2023-10-03T10:24:05.156 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.170515.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.163085.log.gz 2023-10-03T10:24:05.156 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124595.log 2023-10-03T10:24:05.157 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.170515.log.gz 2023-10-03T10:24:05.157 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.190191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187266.log 2023-10-03T10:24:05.157 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.190191.log.gz 2023-10-03T10:24:05.157 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.124595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147763.log 2023-10-03T10:24:05.157 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.187266.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.124595.log.gz 2023-10-03T10:24:05.157 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135401.log 2023-10-03T10:24:05.157 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.187266.log.gz 2023-10-03T10:24:05.157 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.147763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153520.log 2023-10-03T10:24:05.158 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.147763.log.gz 2023-10-03T10:24:05.158 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.135401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185723.log 2023-10-03T10:24:05.158 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.153520.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.135401.log.gz 2023-10-03T10:24:05.158 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192935.log 2023-10-03T10:24:05.158 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.153520.log.gz 2023-10-03T10:24:05.158 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.185723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127627.log 2023-10-03T10:24:05.158 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.185723.log.gz 2023-10-03T10:24:05.158 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.192935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192543.log 2023-10-03T10:24:05.159 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.127627.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.192935.log.gz 2023-10-03T10:24:05.159 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163158.log 2023-10-03T10:24:05.159 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.127627.log.gz 2023-10-03T10:24:05.159 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.192543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181453.log 2023-10-03T10:24:05.159 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.192543.log.gz 2023-10-03T10:24:05.159 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.163158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150703.log 2023-10-03T10:24:05.159 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.163158.log.gz 2023-10-03T10:24:05.159 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.181453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164047.log 2023-10-03T10:24:05.159 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.181453.log.gz 2023-10-03T10:24:05.160 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.150703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145672.log 2023-10-03T10:24:05.160 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.150703.log.gz 2023-10-03T10:24:05.160 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.164047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149598.log 2023-10-03T10:24:05.160 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.164047.log.gz 2023-10-03T10:24:05.160 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.145672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149919.log 2023-10-03T10:24:05.160 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.145672.log.gz 2023-10-03T10:24:05.160 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.149598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173977.log 2023-10-03T10:24:05.160 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.149598.log.gz 2023-10-03T10:24:05.161 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.149919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146187.log 2023-10-03T10:24:05.161 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.149919.log.gz 2023-10-03T10:24:05.161 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.173977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169386.log 2023-10-03T10:24:05.161 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.173977.log.gz 2023-10-03T10:24:05.161 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.146187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149992.log 2023-10-03T10:24:05.161 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.146187.log.gz 2023-10-03T10:24:05.161 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.169386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167846.log 2023-10-03T10:24:05.161 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.169386.log.gz 2023-10-03T10:24:05.162 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.149992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176701.log 2023-10-03T10:24:05.162 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.149992.log.gz 2023-10-03T10:24:05.162 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.167846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124032.log 2023-10-03T10:24:05.162 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.167846.log.gz 2023-10-03T10:24:05.162 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.176701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154919.log 2023-10-03T10:24:05.162 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.124032.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.176701.log.gz 2023-10-03T10:24:05.162 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177775.log 2023-10-03T10:24:05.162 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.124032.log.gz 2023-10-03T10:24:05.163 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.154919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127419.log 2023-10-03T10:24:05.163 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.154919.log.gz 2023-10-03T10:24:05.163 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.177775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136407.log 2023-10-03T10:24:05.163 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.127419.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.177775.log.gz 2023-10-03T10:24:05.163 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180847.log 2023-10-03T10:24:05.163 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.127419.log.gz 2023-10-03T10:24:05.163 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.136407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135546.log 2023-10-03T10:24:05.163 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.136407.log.gz 2023-10-03T10:24:05.164 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.180847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140676.log 2023-10-03T10:24:05.164 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.135546.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.180847.log.gz 2023-10-03T10:24:05.164 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192428.log 2023-10-03T10:24:05.164 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.135546.log.gz 2023-10-03T10:24:05.164 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.140676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136577.log 2023-10-03T10:24:05.164 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.140676.log.gz 2023-10-03T10:24:05.164 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.192428.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-03T10:24:05.164 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.136577.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.192428.log.gz 2023-10-03T10:24:05.165 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191223.log 2023-10-03T10:24:05.165 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.136577.log.gz 2023-10-03T10:24:05.165 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161197.log 2023-10-03T10:24:05.165 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.191223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188322.log 2023-10-03T10:24:05.165 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.191223.log.gz 2023-10-03T10:24:05.165 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.161197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150654.log 2023-10-03T10:24:05.165 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.188322.log: gzip 88.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.161197.log.gz--verbose 2023-10-03T10:24:05.165 INFO:teuthology.orchestra.run.smithi003.stderr:-- /var/log/ceph/ceph-client.admin.128793.log 2023-10-03T10:24:05.166 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.188322.log.gz 2023-10-03T10:24:05.166 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.150654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137605.log 2023-10-03T10:24:05.166 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.150654.log.gz 2023-10-03T10:24:05.166 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.128793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184791.log 2023-10-03T10:24:05.166 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.137605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141708.log 2023-10-03T10:24:05.166 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.128793.log.gz 2023-10-03T10:24:05.166 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.137605.log.gz 2023-10-03T10:24:05.166 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.184791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172091.log 2023-10-03T10:24:05.167 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.141708.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.184791.log.gz 2023-10-03T10:24:05.167 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137409.log 2023-10-03T10:24:05.167 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.141708.log.gz 2023-10-03T10:24:05.167 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.172091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180892.log 2023-10-03T10:24:05.167 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.172091.log.gz 2023-10-03T10:24:05.167 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.137409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173512.log 2023-10-03T10:24:05.167 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.180892.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.137409.log.gz 2023-10-03T10:24:05.168 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166054.log 2023-10-03T10:24:05.168 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.180892.log.gz 2023-10-03T10:24:05.168 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.173512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191393.log 2023-10-03T10:24:05.168 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.173512.log.gz 2023-10-03T10:24:05.168 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.166054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124693.log 2023-10-03T10:24:05.168 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.191393.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.166054.log.gz 2023-10-03T10:24:05.168 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135644.log 2023-10-03T10:24:05.168 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.191393.log.gz 2023-10-03T10:24:05.168 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194751.log 2023-10-03T10:24:05.169 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.124693.log: /var/log/ceph/ceph-client.admin.135644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147714.log 2023-10-03T10:24:05.169 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.124693.log.gz 2023-10-03T10:24:05.169 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.135644.log.gz 2023-10-03T10:24:05.169 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.194751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166303.log 2023-10-03T10:24:05.169 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.194751.log.gz 2023-10-03T10:24:05.169 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.147714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145158.log 2023-10-03T10:24:05.169 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6%/var/log/ceph/ceph-client.admin.166303.log: -- replaced with /var/log/ceph/ceph-client.admin.147714.log.gz 2023-10-03T10:24:05.169 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174758.log 2023-10-03T10:24:05.170 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.166303.log.gz 2023-10-03T10:24:05.170 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.145158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168237.log 2023-10-03T10:24:05.170 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.174758.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.145158.log.gz 2023-10-03T10:24:05.170 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125854.log 2023-10-03T10:24:05.170 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.174758.log.gz 2023-10-03T10:24:05.170 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.168237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127472.log 2023-10-03T10:24:05.170 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.125854.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.168237.log.gz 2023-10-03T10:24:05.170 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146214.log 2023-10-03T10:24:05.171 INFO:teuthology.orchestra.run.smithi003.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.125854.log.gz 2023-10-03T10:24:05.171 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146802.log 2023-10-03T10:24:05.171 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.127472.log: /var/log/ceph/ceph-client.admin.146214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122572.log 2023-10-03T10:24:05.171 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.127472.log.gz 2023-10-03T10:24:05.171 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.146214.log.gz 2023-10-03T10:24:05.171 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.146802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175106.log 2023-10-03T10:24:05.171 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.122572.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.146802.log.gz 2023-10-03T10:24:05.171 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125503.log 2023-10-03T10:24:05.172 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.122572.log.gz 2023-10-03T10:24:05.172 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.175106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159335.log 2023-10-03T10:24:05.172 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.175106.log.gz 2023-10-03T10:24:05.172 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.125503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143198.log 2023-10-03T10:24:05.172 INFO:teuthology.orchestra.run.smithi003.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.125503.log.gz 2023-10-03T10:24:05.172 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.159335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138170.log 2023-10-03T10:24:05.172 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.159335.log.gz 2023-10-03T10:24:05.172 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.143198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189290.log 2023-10-03T10:24:05.173 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2%/var/log/ceph/ceph-client.admin.138170.log: -- replaced with /var/log/ceph/ceph-client.admin.143198.log.gz 2023-10-03T10:24:05.173 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159974.log 2023-10-03T10:24:05.173 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.138170.log.gz 2023-10-03T10:24:05.173 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.189290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176599.log 2023-10-03T10:24:05.173 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.189290.log.gz 2023-10-03T10:24:05.173 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.159974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151317.log 2023-10-03T10:24:05.173 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.176599.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.159974.log.gz 2023-10-03T10:24:05.173 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138912.log 2023-10-03T10:24:05.174 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.176599.log.gz 2023-10-03T10:24:05.174 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.151317.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.111958.log 2023-10-03T10:24:05.174 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.151317.log.gz 2023-10-03T10:24:05.174 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.138912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141522.log 2023-10-03T10:24:05.174 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph.tmp-client.admin.111958.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.138912.log.gz 2023-10-03T10:24:05.174 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164734.log 2023-10-03T10:24:05.174 INFO:teuthology.orchestra.run.smithi003.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.111958.log.gz 2023-10-03T10:24:05.174 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.141522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143663.log 2023-10-03T10:24:05.175 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.164734.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.141522.log.gz 2023-10-03T10:24:05.175 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122866.log 2023-10-03T10:24:05.175 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.164734.log.gz 2023-10-03T10:24:05.175 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.143663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171495.log 2023-10-03T10:24:05.175 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.143663.log.gz 2023-10-03T10:24:05.175 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.122866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148498.log 2023-10-03T10:24:05.175 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.122866.log.gz 2023-10-03T10:24:05.175 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.171495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172679.log 2023-10-03T10:24:05.175 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.148498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188079.log 2023-10-03T10:24:05.176 INFO:teuthology.orchestra.run.smithi003.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.171495.log.gz 2023-10-03T10:24:05.176 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.148498.log.gz 2023-10-03T10:24:05.176 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.172679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159387.log 2023-10-03T10:24:05.176 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8%/var/log/ceph/ceph-client.admin.188079.log: -- replaced with /var/log/ceph/ceph-client.admin.172679.log.gz 2023-10-03T10:24:05.176 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156003.log 2023-10-03T10:24:05.176 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.188079.log.gz 2023-10-03T10:24:05.176 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.159387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142710.log 2023-10-03T10:24:05.176 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.156003.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.159387.log.gz 2023-10-03T10:24:05.177 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158551.log 2023-10-03T10:24:05.177 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.156003.log.gz 2023-10-03T10:24:05.177 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.142710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150115.log 2023-10-03T10:24:05.177 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.142710.log.gz 2023-10-03T10:24:05.177 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175031.log 2023-10-03T10:24:05.177 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.158551.log: /var/log/ceph/ceph-client.admin.150115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148155.log 2023-10-03T10:24:05.177 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.158551.log.gz 2023-10-03T10:24:05.178 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.150115.log.gz 2023-10-03T10:24:05.178 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.175031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173342.log 2023-10-03T10:24:05.178 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.175031.log.gz 2023-10-03T10:24:05.178 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.148155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150167.log 2023-10-03T10:24:05.178 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.148155.log.gz 2023-10-03T10:24:05.178 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.173342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186142.log 2023-10-03T10:24:05.178 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.173342.log.gz 2023-10-03T10:24:05.179 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.150167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179392.log 2023-10-03T10:24:05.179 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.186142.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.150167.log.gz 2023-10-03T10:24:05.179 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170907.log 2023-10-03T10:24:05.179 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.186142.log.gz 2023-10-03T10:24:05.179 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.179392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140454.log 2023-10-03T10:24:05.179 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.179392.log.gz 2023-10-03T10:24:05.179 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.170907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145403.log 2023-10-03T10:24:05.179 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.140454.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.170907.log.gz 2023-10-03T10:24:05.180 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127007.log 2023-10-03T10:24:05.180 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.145403.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.140454.log.gz 2023-10-03T10:24:05.180 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162163.log 2023-10-03T10:24:05.180 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.145403.log.gz 2023-10-03T10:24:05.180 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142265.log 2023-10-03T10:24:05.180 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.127007.log: /var/log/ceph/ceph-client.admin.162163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180968.log 2023-10-03T10:24:05.180 INFO:teuthology.orchestra.run.smithi003.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.127007.log.gz 2023-10-03T10:24:05.181 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.162163.log.gz 2023-10-03T10:24:05.181 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.142265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192007.log 2023-10-03T10:24:05.181 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.142265.log.gz 2023-10-03T10:24:05.181 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.180968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174369.log 2023-10-03T10:24:05.181 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.192007.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.180968.log.gz 2023-10-03T10:24:05.181 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168751.log 2023-10-03T10:24:05.181 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.192007.log.gz 2023-10-03T10:24:05.181 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.174369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152393.log 2023-10-03T10:24:05.182 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.174369.log.gz 2023-10-03T10:24:05.182 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.168751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124084.log 2023-10-03T10:24:05.182 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.168751.log.gz 2023-10-03T10:24:05.182 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.152393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176011.log 2023-10-03T10:24:05.182 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.152393.log.gz 2023-10-03T10:24:05.182 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.124084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129087.log 2023-10-03T10:24:05.182 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.124084.log.gz 2023-10-03T10:24:05.183 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.176011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178020.log 2023-10-03T10:24:05.183 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.176011.log.gz 2023-10-03T10:24:05.183 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.129087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151291.log 2023-10-03T10:24:05.183 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.129087.log.gz 2023-10-03T10:24:05.183 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.178020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159042.log 2023-10-03T10:24:05.183 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.178020.log.gz 2023-10-03T10:24:05.183 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.151291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165446.log 2023-10-03T10:24:05.183 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.151291.log.gz 2023-10-03T10:24:05.184 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.159042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166838.log 2023-10-03T10:24:05.184 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.159042.log.gz 2023-10-03T10:24:05.184 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.165446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172483.log 2023-10-03T10:24:05.184 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.165446.log.gz 2023-10-03T10:24:05.184 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.166838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153055.log 2023-10-03T10:24:05.184 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.166838.log.gz 2023-10-03T10:24:05.184 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.172483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180576.log 2023-10-03T10:24:05.185 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.172483.log.gz 2023-10-03T10:24:05.185 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.153055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181164.log 2023-10-03T10:24:05.185 INFO:teuthology.orchestra.run.smithi003.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.153055.log.gz 2023-10-03T10:24:05.185 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.180576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160759.log 2023-10-03T10:24:05.185 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.180576.log.gz 2023-10-03T10:24:05.185 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.181164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185529.log 2023-10-03T10:24:05.185 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.181164.log.gz 2023-10-03T10:24:05.185 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.160759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192203.log 2023-10-03T10:24:05.186 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.160759.log.gz 2023-10-03T10:24:05.186 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.185529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135693.log 2023-10-03T10:24:05.186 INFO:teuthology.orchestra.run.smithi003.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.185529.log.gz 2023-10-03T10:24:05.186 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.192203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118946.log 2023-10-03T10:24:05.186 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.192203.log.gz 2023-10-03T10:24:05.186 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.135693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186508.log 2023-10-03T10:24:05.186 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.118946.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.135693.log.gz 2023-10-03T10:24:05.186 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136408.log 2023-10-03T10:24:05.187 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.118946.log.gz 2023-10-03T10:24:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.186508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135228.log 2023-10-03T10:24:05.187 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.186508.log.gz 2023-10-03T10:24:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.136408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192739.log 2023-10-03T10:24:05.187 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.136408.log.gz 2023-10-03T10:24:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.135228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171615.log 2023-10-03T10:24:05.187 INFO:teuthology.orchestra.run.smithi003.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.135228.log.gz 2023-10-03T10:24:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.192739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135449.log 2023-10-03T10:24:05.188 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.192739.log.gz 2023-10-03T10:24:05.188 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.171615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163942.log 2023-10-03T10:24:05.188 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.171615.log.gz 2023-10-03T10:24:05.188 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.135449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123517.log 2023-10-03T10:24:05.188 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.135449.log.gz 2023-10-03T10:24:05.188 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.163942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163558.log 2023-10-03T10:24:05.188 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.163942.log.gz 2023-10-03T10:24:05.188 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.123517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168406.log 2023-10-03T10:24:05.188 INFO:teuthology.orchestra.run.smithi003.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.123517.log.gz 2023-10-03T10:24:05.189 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.163558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157887.log 2023-10-03T10:24:05.189 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.163558.log.gz 2023-10-03T10:24:05.189 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.168406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136211.log 2023-10-03T10:24:05.189 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.168406.log.gz 2023-10-03T10:24:05.189 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.157887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193991.log 2023-10-03T10:24:05.189 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.157887.log.gz 2023-10-03T10:24:05.189 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.136211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145942.log 2023-10-03T10:24:05.189 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.136211.log.gz 2023-10-03T10:24:05.190 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.193991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175525.log 2023-10-03T10:24:05.190 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.193991.log.gz 2023-10-03T10:24:05.190 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.145942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150262.log 2023-10-03T10:24:05.190 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.145942.log.gz 2023-10-03T10:24:05.190 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.175525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164269.log 2023-10-03T10:24:05.190 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.175525.log.gz 2023-10-03T10:24:05.190 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.150262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163354.log 2023-10-03T10:24:05.190 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.150262.log.gz 2023-10-03T10:24:05.191 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.164269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154819.log 2023-10-03T10:24:05.191 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.164269.log.gz 2023-10-03T10:24:05.191 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.163354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170223.log 2023-10-03T10:24:05.191 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.163354.log.gz 2023-10-03T10:24:05.191 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.154819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129694.log 2023-10-03T10:24:05.191 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.170223.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.154819.log.gz 2023-10-03T10:24:05.191 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170562.log 2023-10-03T10:24:05.191 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.170223.log.gz 2023-10-03T10:24:05.192 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.129694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137702.log 2023-10-03T10:24:05.192 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.129694.log.gz 2023-10-03T10:24:05.192 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.170562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193209.log 2023-10-03T10:24:05.192 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.170562.log.gz 2023-10-03T10:24:05.192 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.137702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143690.log 2023-10-03T10:24:05.192 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.137702.log.gz 2023-10-03T10:24:05.192 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.193209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186064.log 2023-10-03T10:24:05.192 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.193209.log.gz 2023-10-03T10:24:05.193 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.143690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141141.log 2023-10-03T10:24:05.193 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.143690.log.gz 2023-10-03T10:24:05.193 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.186064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157082.log 2023-10-03T10:24:05.193 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.186064.log.gz 2023-10-03T10:24:05.193 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.141141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169731.log 2023-10-03T10:24:05.193 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.141141.log.gz 2023-10-03T10:24:05.193 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.157082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173120.log 2023-10-03T10:24:05.193 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.169731.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.157082.log.gz 2023-10-03T10:24:05.194 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151824.log 2023-10-03T10:24:05.194 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.169731.log.gz 2023-10-03T10:24:05.194 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.173120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157989.log 2023-10-03T10:24:05.194 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.173120.log.gz 2023-10-03T10:24:05.194 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.151824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185280.log 2023-10-03T10:24:05.194 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.157989.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.151824.log.gz 2023-10-03T10:24:05.194 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162079.log 2023-10-03T10:24:05.194 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.157989.log.gz 2023-10-03T10:24:05.195 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.185280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192151.log 2023-10-03T10:24:05.195 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.185280.log.gz 2023-10-03T10:24:05.195 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.162079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180696.log 2023-10-03T10:24:05.195 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.192151.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.162079.log.gz 2023-10-03T10:24:05.195 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184326.log 2023-10-03T10:24:05.195 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.192151.log.gz 2023-10-03T10:24:05.195 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.180696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160732.log 2023-10-03T10:24:05.195 INFO:teuthology.orchestra.run.smithi003.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.180696.log.gz 2023-10-03T10:24:05.196 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.184326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192054.log 2023-10-03T10:24:05.196 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.160732.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.184326.log.gz 2023-10-03T10:24:05.196 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167018.log 2023-10-03T10:24:05.196 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.160732.log.gz 2023-10-03T10:24:05.196 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.192054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190068.log 2023-10-03T10:24:05.196 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.192054.log.gz 2023-10-03T10:24:05.196 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.167018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189041.log 2023-10-03T10:24:05.196 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.190068.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.167018.log.gz 2023-10-03T10:24:05.197 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123017.log 2023-10-03T10:24:05.197 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.190068.log.gz 2023-10-03T10:24:05.197 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.189041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163485.log 2023-10-03T10:24:05.197 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.189041.log.gz 2023-10-03T10:24:05.197 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192177.log 2023-10-03T10:24:05.197 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.123017.log: /var/log/ceph/ceph-client.admin.163485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194964.log 2023-10-03T10:24:05.197 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.123017.log.gz 2023-10-03T10:24:05.197 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.163485.log.gz 2023-10-03T10:24:05.198 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.192177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136356.log 2023-10-03T10:24:05.198 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.194964.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.192177.log.gz 2023-10-03T10:24:05.198 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190287.log 2023-10-03T10:24:05.198 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.194964.log.gz 2023-10-03T10:24:05.198 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.136356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178046.log 2023-10-03T10:24:05.198 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.136356.log.gz 2023-10-03T10:24:05.198 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.190287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187240.log 2023-10-03T10:24:05.198 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.190287.log.gz 2023-10-03T10:24:05.199 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.178046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141826.log 2023-10-03T10:24:05.199 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.178046.log.gz 2023-10-03T10:24:05.199 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.187240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142122.log 2023-10-03T10:24:05.199 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.141826.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.187240.log.gz 2023-10-03T10:24:05.199 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144280.log 2023-10-03T10:24:05.199 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.142122.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.141826.log.gz 2023-10-03T10:24:05.199 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182264.log 2023-10-03T10:24:05.199 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.142122.log.gz 2023-10-03T10:24:05.200 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.144280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144643.log 2023-10-03T10:24:05.200 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.144280.log.gz 2023-10-03T10:24:05.200 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.182264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127727.log 2023-10-03T10:24:05.200 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.182264.log.gz 2023-10-03T10:24:05.200 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.144643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147397.log 2023-10-03T10:24:05.200 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.144643.log.gz 2023-10-03T10:24:05.200 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.127727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191197.log 2023-10-03T10:24:05.200 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.127727.log.gz 2023-10-03T10:24:05.200 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.147397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157672.log 2023-10-03T10:24:05.201 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.191197.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.147397.log.gz 2023-10-03T10:24:05.201 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191759.log 2023-10-03T10:24:05.201 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.191197.log.gz 2023-10-03T10:24:05.201 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.157672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167284.log 2023-10-03T10:24:05.201 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.157672.log.gz 2023-10-03T10:24:05.201 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.191759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191419.log 2023-10-03T10:24:05.201 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.191759.log.gz 2023-10-03T10:24:05.201 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.167284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128578.log 2023-10-03T10:24:05.202 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.167284.log.gz 2023-10-03T10:24:05.202 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.191419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187735.log 2023-10-03T10:24:05.202 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.128578.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.191419.log.gz 2023-10-03T10:24:05.202 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140310.log 2023-10-03T10:24:05.202 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.128578.log.gz 2023-10-03T10:24:05.202 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.187735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167771.log 2023-10-03T10:24:05.202 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.187735.log.gz 2023-10-03T10:24:05.203 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.140310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184888.log 2023-10-03T10:24:05.203 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.140310.log.gz 2023-10-03T10:24:05.203 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.167771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178804.log 2023-10-03T10:24:05.203 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.184888.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.167771.log.gz 2023-10-03T10:24:05.203 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172334.log 2023-10-03T10:24:05.203 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.184888.log.gz 2023-10-03T10:24:05.203 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.178804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193157.log 2023-10-03T10:24:05.203 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.178804.log.gz 2023-10-03T10:24:05.203 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.172334.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-03T10:24:05.204 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.172334.log.gz 2023-10-03T10:24:05.204 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.193157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150756.log 2023-10-03T10:24:05.204 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.193157.log.gz 2023-10-03T10:24:05.204 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188819.log 2023-10-03T10:24:05.204 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.150756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179784.log 2023-10-03T10:24:05.204 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.150756.log.gz 2023-10-03T10:24:05.204 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.188819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157448.log 2023-10-03T10:24:05.204 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 87.3%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.188819.log.gz/var/log/ceph/ceph-client.admin.143075.log 2023-10-03T10:24:05.205 INFO:teuthology.orchestra.run.smithi003.stderr: 2023-10-03T10:24:05.205 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.179784.log: /var/log/ceph/ceph-client.admin.157448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143002.log 2023-10-03T10:24:05.205 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.179784.log.gz 2023-10-03T10:24:05.205 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.157448.log.gz 2023-10-03T10:24:05.205 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.143075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180176.log 2023-10-03T10:24:05.205 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose 87.7%-- /var/log/ceph/ceph-client.admin.135964.log -- replaced with /var/log/ceph/ceph-client.admin.143075.log.gz 2023-10-03T10:24:05.205 INFO:teuthology.orchestra.run.smithi003.stderr: 2023-10-03T10:24:05.205 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.143002.log: /var/log/ceph/ceph-client.admin.180176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172362.log 2023-10-03T10:24:05.206 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.143002.log.gz 2023-10-03T10:24:05.206 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.180176.log.gz 2023-10-03T10:24:05.206 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.135964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143467.log 2023-10-03T10:24:05.206 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190290.log 2023-10-03T10:24:05.206 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.135964.log.gz 2023-10-03T10:24:05.206 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.172362.log: /var/log/ceph/ceph-client.admin.143467.log: 88.0%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.172362.log.gz--verbose 2023-10-03T10:24:05.206 INFO:teuthology.orchestra.run.smithi003.stderr:-- /var/log/ceph/ceph-client.admin.193992.log 2023-10-03T10:24:05.206 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.143467.log.gz 2023-10-03T10:24:05.207 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.190290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157009.log 2023-10-03T10:24:05.207 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152530.log 2023-10-03T10:24:05.207 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.190290.log.gz 2023-10-03T10:24:05.207 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.193992.log: /var/log/ceph/ceph-client.admin.157009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150729.log 2023-10-03T10:24:05.207 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.193992.log.gz 2023-10-03T10:24:05.207 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.157009.log.gz 2023-10-03T10:24:05.207 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.152530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159165.log 2023-10-03T10:24:05.207 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.150729.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.152530.log.gz 2023-10-03T10:24:05.208 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128448.log 2023-10-03T10:24:05.208 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.150729.log.gz 2023-10-03T10:24:05.208 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.159165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138315.log 2023-10-03T10:24:05.208 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127299.log 2023-10-03T10:24:05.208 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.159165.log.gz 2023-10-03T10:24:05.208 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.128448.log: /var/log/ceph/ceph-client.admin.138315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194866.log 2023-10-03T10:24:05.208 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.128448.log.gz 2023-10-03T10:24:05.208 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.138315.log.gz 2023-10-03T10:24:05.209 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.127299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182562.log 2023-10-03T10:24:05.209 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168994.log 2023-10-03T10:24:05.209 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.194866.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.127299.log.gz 2023-10-03T10:24:05.209 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.182562.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.194866.log.gzgzip 2023-10-03T10:24:05.209 INFO:teuthology.orchestra.run.smithi003.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.154134.log 2023-10-03T10:24:05.209 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.182562.log.gz 2023-10-03T10:24:05.209 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.168994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151709.log 2023-10-03T10:24:05.209 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.168994.log.gz 2023-10-03T10:24:05.209 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.154134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193915.log 2023-10-03T10:24:05.210 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.151709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191171.log 2023-10-03T10:24:05.210 INFO:teuthology.orchestra.run.smithi003.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.154134.log.gz 2023-10-03T10:24:05.210 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.151709.log.gz 2023-10-03T10:24:05.210 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.193915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154670.log 2023-10-03T10:24:05.210 INFO:teuthology.orchestra.run.smithi003.stderr:gzip 87.7%-5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.193915.log.gz/var/log/ceph/ceph-client.admin.183738.log 2023-10-03T10:24:05.210 INFO:teuthology.orchestra.run.smithi003.stderr: 2023-10-03T10:24:05.210 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.191171.log: 95.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-03T10:24:05.210 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.191171.log.gz 2023-10-03T10:24:05.211 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.154670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156442.log 2023-10-03T10:24:05.211 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142095.log 2023-10-03T10:24:05.211 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.183738.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.154670.log.gz 2023-10-03T10:24:05.211 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.183738.log.gz 2023-10-03T10:24:05.211 INFO:teuthology.orchestra.run.smithi003.stderr:gzip /var/log/ceph/ceph-client.admin.156442.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.176991.log 2023-10-03T10:24:05.211 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.156442.log.gz 2023-10-03T10:24:05.211 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 /var/log/ceph/ceph-client.admin.142095.log: --verbose -- /var/log/ceph/ceph-client.admin.173708.log 2023-10-03T10:24:05.211 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.176991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153716.log 2023-10-03T10:24:05.212 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.142095.log.gz 2023-10-03T10:24:05.212 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.176991.log.gz 2023-10-03T10:24:05.212 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.173708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186947.log 2023-10-03T10:24:05.212 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.153716.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.173708.log.gz 2023-10-03T10:24:05.212 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172558.log 2023-10-03T10:24:05.212 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.153716.log.gz 2023-10-03T10:24:05.212 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.186947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123888.log 2023-10-03T10:24:05.212 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.172558.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.186947.log.gz 2023-10-03T10:24:05.213 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192250.log 2023-10-03T10:24:05.213 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.123888.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.172558.log.gz 2023-10-03T10:24:05.213 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191028.log 2023-10-03T10:24:05.213 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.192250.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.123888.log.gz 2023-10-03T10:24:05.213 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177824.log 2023-10-03T10:24:05.213 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.192250.log.gz 2023-10-03T10:24:05.213 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.191028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153686.log 2023-10-03T10:24:05.213 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.191028.log.gz 2023-10-03T10:24:05.214 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177824.log: /var/log/ceph/ceph-client.admin.124203.log 2023-10-03T10:24:05.214 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.153686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138986.log 2023-10-03T10:24:05.214 INFO:teuthology.orchestra.run.smithi003.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.177824.log.gz 2023-10-03T10:24:05.214 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.153686.log.gz 2023-10-03T10:24:05.214 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.124203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189487.log 2023-10-03T10:24:05.214 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.124203.log.gz 2023-10-03T10:24:05.214 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.138986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138939.log 2023-10-03T10:24:05.214 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.138986.log.gz 2023-10-03T10:24:05.215 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146579.log 2023-10-03T10:24:05.215 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.189487.log: /var/log/ceph/ceph-client.admin.138939.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.189487.log.gz 2023-10-03T10:24:05.215 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177092.log 2023-10-03T10:24:05.215 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.138939.log.gz 2023-10-03T10:24:05.215 INFO:teuthology.orchestra.run.smithi003.stderr:gzip /var/log/ceph/ceph-client.admin.146579.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.155219.log 2023-10-03T10:24:05.215 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.177092.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.146579.log.gz 2023-10-03T10:24:05.215 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137166.log 2023-10-03T10:24:05.215 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.177092.log.gz 2023-10-03T10:24:05.216 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.155219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152075.log 2023-10-03T10:24:05.216 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.155219.log.gz 2023-10-03T10:24:05.216 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.137166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136016.log 2023-10-03T10:24:05.216 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.137166.log.gz 2023-10-03T10:24:05.216 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177187.log 2023-10-03T10:24:05.216 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.152075.log: /var/log/ceph/ceph-client.admin.136016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176282.log 2023-10-03T10:24:05.216 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.152075.log.gz 2023-10-03T10:24:05.216 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.136016.log.gz 2023-10-03T10:24:05.217 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.177187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173071.log 2023-10-03T10:24:05.217 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.177187.log.gz 2023-10-03T10:24:05.217 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.176282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190413.log 2023-10-03T10:24:05.217 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.176282.log.gz 2023-10-03T10:24:05.217 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.173071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146652.log 2023-10-03T10:24:05.217 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.173071.log.gz 2023-10-03T10:24:05.217 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150850.log 2023-10-03T10:24:05.217 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.190413.log: /var/log/ceph/ceph-client.admin.146652.log: gzip 87.6%-5 -- replaced with /var/log/ceph/ceph-client.admin.190413.log.gz--verbose 2023-10-03T10:24:05.218 INFO:teuthology.orchestra.run.smithi003.stderr:-- /var/log/ceph/ceph-client.admin.181192.log 2023-10-03T10:24:05.218 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.146652.log.gz 2023-10-03T10:24:05.218 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.150850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157597.log 2023-10-03T10:24:05.218 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.181192.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.150850.log.gz 2023-10-03T10:24:05.218 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156690.log 2023-10-03T10:24:05.218 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.181192.log.gz 2023-10-03T10:24:05.218 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.157597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171568.log 2023-10-03T10:24:05.218 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.157597.log.gz 2023-10-03T10:24:05.219 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.156690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179196.log 2023-10-03T10:24:05.219 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.156690.log.gz 2023-10-03T10:24:05.219 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.171568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129434.log 2023-10-03T10:24:05.219 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.171568.log.gz 2023-10-03T10:24:05.219 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.179196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138368.log 2023-10-03T10:24:05.219 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.179196.log.gz 2023-10-03T10:24:05.219 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.129434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166446.log 2023-10-03T10:24:05.219 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.138368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189067.log 2023-10-03T10:24:05.220 INFO:teuthology.orchestra.run.smithi003.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.138368.log.gz 2023-10-03T10:24:05.220 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.166446.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.129434.log.gz 2023-10-03T10:24:05.220 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137986.log 2023-10-03T10:24:05.220 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.166446.log.gz 2023-10-03T10:24:05.220 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125707.log 2023-10-03T10:24:05.220 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.189067.log: /var/log/ceph/ceph-client.admin.137986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170392.log 2023-10-03T10:24:05.220 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.189067.log.gz 2023-10-03T10:24:05.220 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.125707.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.137986.log.gz 2023-10-03T10:24:05.221 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163683.log 2023-10-03T10:24:05.221 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.170392.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.125707.log.gz 2023-10-03T10:24:05.221 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162745.log 2023-10-03T10:24:05.221 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.163683.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.170392.log.gz 2023-10-03T10:24:05.221 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170758.log 2023-10-03T10:24:05.221 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.163683.log.gz 2023-10-03T10:24:05.221 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.162745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167891.log 2023-10-03T10:24:05.221 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.162745.log.gz 2023-10-03T10:24:05.221 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124939.log 2023-10-03T10:24:05.222 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.170758.log: /var/log/ceph/ceph-client.admin.167891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158185.log 2023-10-03T10:24:05.222 INFO:teuthology.orchestra.run.smithi003.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.170758.log.gz 2023-10-03T10:24:05.222 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.167891.log.gz 2023-10-03T10:24:05.222 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.124939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186260.log 2023-10-03T10:24:05.222 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.124939.log.gz 2023-10-03T10:24:05.222 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.158185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169067.log 2023-10-03T10:24:05.222 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.158185.log.gz 2023-10-03T10:24:05.222 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.186260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129038.log 2023-10-03T10:24:05.223 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.186260.log.gz 2023-10-03T10:24:05.223 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.169067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125383.log 2023-10-03T10:24:05.223 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.129038.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.169067.log.gz 2023-10-03T10:24:05.223 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156228.log 2023-10-03T10:24:05.223 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.125383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142461.log 2023-10-03T10:24:05.223 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.129038.log.gz 2023-10-03T10:24:05.223 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.125383.log.gz 2023-10-03T10:24:05.223 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.156228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155807.log 2023-10-03T10:24:05.224 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.156228.log.gz 2023-10-03T10:24:05.224 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.142461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179980.log 2023-10-03T10:24:05.224 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.142461.log.gz 2023-10-03T10:24:05.224 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182758.log 2023-10-03T10:24:05.224 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.155807.log: /var/log/ceph/ceph-client.admin.179980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147593.log 2023-10-03T10:24:05.224 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.155807.log.gz 2023-10-03T10:24:05.224 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.179980.log.gz 2023-10-03T10:24:05.224 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.182758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150507.log 2023-10-03T10:24:05.225 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.182758.log.gz 2023-10-03T10:24:05.225 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144813.log 2023-10-03T10:24:05.225 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.147593.log: /var/log/ceph/ceph-client.admin.150507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128695.log 2023-10-03T10:24:05.225 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.147593.log.gz 2023-10-03T10:24:05.225 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.150507.log.gz 2023-10-03T10:24:05.225 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157279.log 2023-10-03T10:24:05.225 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.144813.log: /var/log/ceph/ceph-client.admin.128695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165811.log 2023-10-03T10:24:05.225 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.144813.log.gz 2023-10-03T10:24:05.226 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.128695.log.gz 2023-10-03T10:24:05.226 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.157279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179810.log 2023-10-03T10:24:05.226 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.157279.log.gz 2023-10-03T10:24:05.226 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.165811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138886.log 2023-10-03T10:24:05.226 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.165811.log.gz 2023-10-03T10:24:05.226 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.179810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153643.log 2023-10-03T10:24:05.226 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.179810.log.gz 2023-10-03T10:24:05.226 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190094.log 2023-10-03T10:24:05.227 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.138886.log: /var/log/ceph/ceph-client.admin.153643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147322.log 2023-10-03T10:24:05.227 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.138886.log.gz 2023-10-03T10:24:05.227 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.153643.log.gz 2023-10-03T10:24:05.227 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.190094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171812.log 2023-10-03T10:24:05.227 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.190094.log.gz 2023-10-03T10:24:05.227 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174173.log 2023-10-03T10:24:05.227 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.147322.log: /var/log/ceph/ceph-client.admin.171812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125756.log 2023-10-03T10:24:05.227 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.147322.log.gz 2023-10-03T10:24:05.228 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.171812.log.gz 2023-10-03T10:24:05.228 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.174173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178634.log 2023-10-03T10:24:05.228 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.125756.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.174173.log.gz 2023-10-03T10:24:05.228 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181676.log 2023-10-03T10:24:05.228 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.178634.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.125756.log.gz 2023-10-03T10:24:05.228 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188845.log 2023-10-03T10:24:05.228 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.178634.log.gz 2023-10-03T10:24:05.228 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.181676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147175.log 2023-10-03T10:24:05.228 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.181676.log.gz 2023-10-03T10:24:05.229 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159727.log 2023-10-03T10:24:05.229 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.188845.log: /var/log/ceph/ceph-client.admin.147175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148181.log 2023-10-03T10:24:05.229 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.188845.log.gz 2023-10-03T10:24:05.229 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.147175.log.gz 2023-10-03T10:24:05.229 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.159727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180032.log 2023-10-03T10:24:05.229 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.159727.log.gz 2023-10-03T10:24:05.229 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155088.log 2023-10-03T10:24:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.148181.log: /var/log/ceph/ceph-client.admin.180032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174714.log 2023-10-03T10:24:05.230 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.148181.log.gz 2023-10-03T10:24:05.230 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.180032.log.gz 2023-10-03T10:24:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.155088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192399.log 2023-10-03T10:24:05.230 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.155088.log.gz 2023-10-03T10:24:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.174714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194385.log 2023-10-03T10:24:05.230 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.174714.log.gz 2023-10-03T10:24:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.192399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146930.log 2023-10-03T10:24:05.231 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.192399.log.gz 2023-10-03T10:24:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.194385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175423.log 2023-10-03T10:24:05.231 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.194385.log.gz 2023-10-03T10:24:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.146930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141434.log 2023-10-03T10:24:05.231 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.146930.log.gz 2023-10-03T10:24:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.175423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178830.log 2023-10-03T10:24:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.141434.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.175423.log.gz 2023-10-03T10:24:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153344.log 2023-10-03T10:24:05.231 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.141434.log.gz 2023-10-03T10:24:05.232 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.178830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171968.log 2023-10-03T10:24:05.232 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.178830.log.gz 2023-10-03T10:24:05.232 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.153344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194333.log 2023-10-03T10:24:05.232 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.153344.log.gz 2023-10-03T10:24:05.232 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.171968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187931.log 2023-10-03T10:24:05.232 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.171968.log.gz 2023-10-03T10:24:05.232 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.194333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162962.log 2023-10-03T10:24:05.232 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.194333.log.gz 2023-10-03T10:24:05.233 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.187931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188223.log 2023-10-03T10:24:05.233 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.187931.log.gz 2023-10-03T10:24:05.233 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.162962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152761.log 2023-10-03T10:24:05.233 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.162962.log.gz 2023-10-03T10:24:05.233 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.188223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154330.log 2023-10-03T10:24:05.233 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.188223.log.gz 2023-10-03T10:24:05.233 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.152761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179588.log 2023-10-03T10:24:05.233 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.152761.log.gz 2023-10-03T10:24:05.234 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.154330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135960.log 2023-10-03T10:24:05.234 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.154330.log.gz 2023-10-03T10:24:05.234 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.179588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192594.log 2023-10-03T10:24:05.234 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.179588.log.gz 2023-10-03T10:24:05.234 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.135960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164512.log 2023-10-03T10:24:05.234 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.135960.log.gz 2023-10-03T10:24:05.234 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.192594.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-03T10:24:05.234 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.192594.log.gz 2023-10-03T10:24:05.235 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.164512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125805.log 2023-10-03T10:24:05.235 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.164512.log.gz 2023-10-03T10:24:05.235 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164831.log 2023-10-03T10:24:05.235 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.125805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172985.log 2023-10-03T10:24:05.235 INFO:teuthology.orchestra.run.smithi003.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.125805.log.gz 2023-10-03T10:24:05.235 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.164831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177432.log 2023-10-03T10:24:05.235 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.164831.log.gz 2023-10-03T10:24:05.235 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.172985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139526.log 2023-10-03T10:24:05.236 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.172985.log.gz 2023-10-03T10:24:05.236 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.177432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178216.log 2023-10-03T10:24:05.236 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.177432.log.gz 2023-10-03T10:24:05.236 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172409.log 2023-10-03T10:24:05.236 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.139526.log: /var/log/ceph/ceph-client.admin.178216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178167.log 2023-10-03T10:24:05.236 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.139526.log.gz 2023-10-03T10:24:05.236 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.178216.log.gz 2023-10-03T10:24:05.236 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.172409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176648.log 2023-10-03T10:24:05.236 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.172409.log.gz 2023-10-03T10:24:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191615.log 2023-10-03T10:24:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.178167.log: /var/log/ceph/ceph-client.admin.176648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164857.log 2023-10-03T10:24:05.237 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.178167.log.gz 2023-10-03T10:24:05.237 INFO:teuthology.orchestra.run.smithi003.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.176648.log.gz 2023-10-03T10:24:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.191615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123762.log 2023-10-03T10:24:05.237 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.191615.log.gz-5 2023-10-03T10:24:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:--verbose -- /var/log/ceph/ceph-client.admin.164857.log: /var/log/ceph/ceph-client.admin.177236.log 2023-10-03T10:24:05.237 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.164857.log.gz 2023-10-03T10:24:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.123762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129385.log 2023-10-03T10:24:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153447.log 2023-10-03T10:24:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.177236.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.123762.log.gz 2023-10-03T10:24:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.129385.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.177236.log.gz 2023-10-03T10:24:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173932.log 2023-10-03T10:24:05.238 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.129385.log.gz 2023-10-03T10:24:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165873.log 2023-10-03T10:24:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.153447.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.153447.log.gz 2023-10-03T10:24:05.239 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.173932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138571.log 2023-10-03T10:24:05.239 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.173932.log.gz 2023-10-03T10:24:05.239 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.165873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160563.log 2023-10-03T10:24:05.239 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.165873.log.gz 2023-10-03T10:24:05.239 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.138571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150570.log 2023-10-03T10:24:05.239 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.138571.log.gz 2023-10-03T10:24:05.239 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.160563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128398.log 2023-10-03T10:24:05.239 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.160563.log.gz 2023-10-03T10:24:05.240 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178755.log 2023-10-03T10:24:05.240 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.150570.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.150570.log.gz 2023-10-03T10:24:05.240 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.128398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151487.log 2023-10-03T10:24:05.240 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137997.log 2023-10-03T10:24:05.240 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.128398.log.gz 2023-10-03T10:24:05.240 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.178755.log: /var/log/ceph/ceph-client.admin.151487.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.178755.log.gz 2023-10-03T10:24:05.240 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145260.log 2023-10-03T10:24:05.241 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.151487.log.gz 2023-10-03T10:24:05.241 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.137997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152493.log 2023-10-03T10:24:05.241 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.145260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158799.log 2023-10-03T10:24:05.241 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.137997.log.gz 2023-10-03T10:24:05.241 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.145260.log.gz 2023-10-03T10:24:05.241 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.152493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170954.log 2023-10-03T10:24:05.241 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- 88.8%/var/log/ceph/ceph-client.admin.194187.log 2023-10-03T10:24:05.241 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.152493.log.gz 2023-10-03T10:24:05.242 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.158799.log: /var/log/ceph/ceph-client.admin.170954.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.158799.log.gz 2023-10-03T10:24:05.242 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166080.log 2023-10-03T10:24:05.242 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.170954.log.gz 2023-10-03T10:24:05.242 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171542.log 2023-10-03T10:24:05.242 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.194187.log: /var/log/ceph/ceph-client.admin.166080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184986.log 2023-10-03T10:24:05.242 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.194187.log.gz 2023-10-03T10:24:05.243 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.166080.log.gz 2023-10-03T10:24:05.243 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.171542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183419.log 2023-10-03T10:24:05.243 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.171542.log.gz 2023-10-03T10:24:05.243 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148228.log 2023-10-03T10:24:05.243 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.184986.log: /var/log/ceph/ceph-client.admin.183419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138767.log 2023-10-03T10:24:05.243 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.184986.log.gz 2023-10-03T10:24:05.243 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.183419.log.gz 2023-10-03T10:24:05.243 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.148228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159583.log 2023-10-03T10:24:05.244 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.148228.log.gz 2023-10-03T10:24:05.244 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186312.log 2023-10-03T10:24:05.244 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.138767.log: /var/log/ceph/ceph-client.admin.159583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193795.log 2023-10-03T10:24:05.244 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.138767.log.gz 2023-10-03T10:24:05.244 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.186312.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.159583.log.gz 2023-10-03T10:24:05.244 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177876.log 2023-10-03T10:24:05.244 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.186312.log.gz 2023-10-03T10:24:05.244 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.193795.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-03T10:24:05.245 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.193795.log.gz 2023-10-03T10:24:05.245 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.177876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149945.log 2023-10-03T10:24:05.245 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.177876.log.gz 2023-10-03T10:24:05.245 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144421.log 2023-10-03T10:24:05.245 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.149945.log: /var/log/ceph/ceph-client.admin.144421.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.149945.log.gz 2023-10-03T10:24:05.245 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.144421.log.gz 2023-10-03T10:24:05.245 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151931.log 2023-10-03T10:24:05.245 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167622.log 2023-10-03T10:24:05.246 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.151931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139278.log 2023-10-03T10:24:05.246 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.151931.log.gz 2023-10-03T10:24:05.246 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.167622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149723.log 2023-10-03T10:24:05.246 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186115.log 2023-10-03T10:24:05.246 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.139278.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.167622.log.gz 2023-10-03T10:24:05.246 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.149723.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.139278.log.gz 2023-10-03T10:24:05.246 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.149723.log.gz 2023-10-03T10:24:05.246 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.186115.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.186115.log.gz 2023-10-03T10:24:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165053.log 2023-10-03T10:24:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123223.log 2023-10-03T10:24:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.165053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176309.log 2023-10-03T10:24:05.247 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.165053.log.gz 2023-10-03T10:24:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.123223.log: /var/log/ceph/ceph-client.admin.176309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151735.log 2023-10-03T10:24:05.247 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.123223.log.gz 2023-10-03T10:24:05.247 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.176309.log.gz 2023-10-03T10:24:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193599.log 2023-10-03T10:24:05.248 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.151735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193600.log 2023-10-03T10:24:05.248 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.151735.log.gz 2023-10-03T10:24:05.248 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.193599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137729.log 2023-10-03T10:24:05.248 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.193599.log.gz 2023-10-03T10:24:05.248 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.193600.log: /var/log/ceph/ceph-client.admin.137729.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.193600.log.gz 2023-10-03T10:24:05.248 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.137729.log.gz 2023-10-03T10:24:05.248 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190609.log 2023-10-03T10:24:05.248 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182068.log 2023-10-03T10:24:05.249 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.190609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189488.log 2023-10-03T10:24:05.249 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.190609.log.gz 2023-10-03T10:24:05.249 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.182068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180521.log 2023-10-03T10:24:05.249 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.182068.log.gz 2023-10-03T10:24:05.249 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129136.log 2023-10-03T10:24:05.249 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.189488.log: /var/log/ceph/ceph-client.admin.180521.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.189488.log.gz 2023-10-03T10:24:05.249 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.180521.log.gz 2023-10-03T10:24:05.249 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.129136.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.129136.log.gz 2023-10-03T10:24:05.250 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155836.log 2023-10-03T10:24:05.250 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154035.log 2023-10-03T10:24:05.250 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.155836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149282.log 2023-10-03T10:24:05.250 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.155836.log.gz 2023-10-03T10:24:05.250 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.154035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155925.log 2023-10-03T10:24:05.250 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.154035.log.gz-5 2023-10-03T10:24:05.250 INFO:teuthology.orchestra.run.smithi003.stderr:--verbose -- /var/log/ceph/ceph-client.admin.138367.log 2023-10-03T10:24:05.250 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.149282.log: /var/log/ceph/ceph-client.admin.155925.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.149282.log.gz 2023-10-03T10:24:05.251 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.138367.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.155925.log.gz 2023-10-03T10:24:05.251 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.138367.log.gz 2023-10-03T10:24:05.251 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180433.log 2023-10-03T10:24:05.251 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153350.log 2023-10-03T10:24:05.251 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.180433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154623.log 2023-10-03T10:24:05.251 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.180433.log.gz 2023-10-03T10:24:05.251 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.153350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123272.log 2023-10-03T10:24:05.251 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.153350.log.gz 2023-10-03T10:24:05.252 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141899.log 2023-10-03T10:24:05.252 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.154623.log: /var/log/ceph/ceph-client.admin.123272.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.154623.log.gz 2023-10-03T10:24:05.252 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.141899.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.123272.log.gz 2023-10-03T10:24:05.252 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138171.log 2023-10-03T10:24:05.252 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.141899.log.gz 2023-10-03T10:24:05.252 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161982.log 2023-10-03T10:24:05.252 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.138171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136799.log 2023-10-03T10:24:05.252 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.138171.log.gz 2023-10-03T10:24:05.252 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.161982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123419.log 2023-10-03T10:24:05.253 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195111.log 2023-10-03T10:24:05.253 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.161982.log.gz 2023-10-03T10:24:05.253 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.136799.log: /var/log/ceph/ceph-client.admin.123419.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.136799.log.gz 2023-10-03T10:24:05.253 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173146.log 2023-10-03T10:24:05.253 INFO:teuthology.orchestra.run.smithi003.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.123419.log.gz 2023-10-03T10:24:05.253 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.195111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149674.log 2023-10-03T10:24:05.253 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.195111.log.gz 2023-10-03T10:24:05.253 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.173146.log: /var/log/ceph/ceph-client.admin.149674.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.173146.log.gz 2023-10-03T10:24:05.254 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162301.log 2023-10-03T10:24:05.254 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.149674.log.gz 2023-10-03T10:24:05.254 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152958.log 2023-10-03T10:24:05.254 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.162301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139966.log 2023-10-03T10:24:05.254 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.162301.log.gz 2023-10-03T10:24:05.254 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.152958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167967.log 2023-10-03T10:24:05.254 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.152958.log.gz 2023-10-03T10:24:05.254 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.139966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183198.log 2023-10-03T10:24:05.255 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.139966.log.gz 2023-10-03T10:24:05.255 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.167967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140140.log 2023-10-03T10:24:05.255 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.167967.log.gz 2023-10-03T10:24:05.255 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.183198.log: /var/log/ceph/ceph-client.admin.140140.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.183198.log.gz 2023-10-03T10:24:05.255 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.140140.log.gz 2023-10-03T10:24:05.255 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150337.log 2023-10-03T10:24:05.255 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168675.log 2023-10-03T10:24:05.255 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.150337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154721.log 2023-10-03T10:24:05.256 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.150337.log.gz 2023-10-03T10:24:05.256 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.168675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141337.log 2023-10-03T10:24:05.256 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.168675.log.gz 2023-10-03T10:24:05.256 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.154721.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.154721.log.gz 2023-10-03T10:24:05.256 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.141337.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.141337.log.gz 2023-10-03T10:24:05.256 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165518.log 2023-10-03T10:24:05.256 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157963.log 2023-10-03T10:24:05.256 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.165518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135154.log 2023-10-03T10:24:05.257 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.165518.log.gz 2023-10-03T10:24:05.257 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.157963.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.157963.log.gz 2023-10-03T10:24:05.257 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.135154.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.135154.log.gz 2023-10-03T10:24:05.257 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169974.log 2023-10-03T10:24:05.257 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137801.log 2023-10-03T10:24:05.257 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.169974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157642.log 2023-10-03T10:24:05.257 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.169974.log.gz 2023-10-03T10:24:05.257 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.137801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136526.log 2023-10-03T10:24:05.258 INFO:teuthology.orchestra.run.smithi003.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.137801.log.gz 2023-10-03T10:24:05.258 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.157642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164298.log 2023-10-03T10:24:05.258 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.157642.log.gz 2023-10-03T10:24:05.258 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.136526.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.136526.log.gz 2023-10-03T10:24:05.258 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.164298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172728.log 2023-10-03T10:24:05.258 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.164298.log.gz 2023-10-03T10:24:05.258 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174492.log 2023-10-03T10:24:05.258 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.172728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175694.log 2023-10-03T10:24:05.259 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.172728.log.gz 2023-10-03T10:24:05.259 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.174492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136134.log 2023-10-03T10:24:05.259 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.175694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168602.log 2023-10-03T10:24:05.259 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.174492.log.gz 2023-10-03T10:24:05.259 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.136134.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.175694.log.gz 2023-10-03T10:24:05.259 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145354.log 2023-10-03T10:24:05.259 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2%/var/log/ceph/ceph-client.admin.168602.log: -- replaced with /var/log/ceph/ceph-client.admin.136134.log.gz 2023-10-03T10:24:05.259 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.168602.log.gz 2023-10-03T10:24:05.260 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.145354.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.145354.log.gz 2023-10-03T10:24:05.260 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168435.log 2023-10-03T10:24:05.260 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154328.log 2023-10-03T10:24:05.260 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.168435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161590.log 2023-10-03T10:24:05.260 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.168435.log.gz 2023-10-03T10:24:05.260 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.154328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166276.log 2023-10-03T10:24:05.260 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.154328.log.gz 2023-10-03T10:24:05.260 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.161590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184300.log 2023-10-03T10:24:05.261 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.161590.log.gz 2023-10-03T10:24:05.261 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.166276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180123.log 2023-10-03T10:24:05.261 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.166276.log.gz 2023-10-03T10:24:05.261 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.184300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152710.log 2023-10-03T10:24:05.261 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.180123.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.184300.log.gzgzip 2023-10-03T10:24:05.261 INFO:teuthology.orchestra.run.smithi003.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.139746.log 2023-10-03T10:24:05.261 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.152710.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.180123.log.gz 2023-10-03T10:24:05.261 INFO:teuthology.orchestra.run.smithi003.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.152710.log.gz 2023-10-03T10:24:05.262 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.139746.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.139746.log.gz 2023-10-03T10:24:05.262 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135820.log 2023-10-03T10:24:05.262 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181556.log 2023-10-03T10:24:05.262 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.135820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192621.log 2023-10-03T10:24:05.262 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.181556.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.135820.log.gz 2023-10-03T10:24:05.262 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.181556.log.gz 2023-10-03T10:24:05.262 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.192621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148574.log 2023-10-03T10:24:05.262 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.192621.log.gz 2023-10-03T10:24:05.263 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151046.log 2023-10-03T10:24:05.263 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.148574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167452.log 2023-10-03T10:24:05.263 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.148574.log.gz 2023-10-03T10:24:05.263 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.151046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122670.log 2023-10-03T10:24:05.263 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.151046.log.gz 2023-10-03T10:24:05.263 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.167452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183028.log 2023-10-03T10:24:05.263 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.122670.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.167452.log.gz 2023-10-03T10:24:05.263 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.183028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170588.log 2023-10-03T10:24:05.263 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.122670.log.gz 2023-10-03T10:24:05.264 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.183028.log.gz 2023-10-03T10:24:05.264 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136578.log 2023-10-03T10:24:05.264 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175302.log 2023-10-03T10:24:05.264 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.170588.log: /var/log/ceph/ceph-client.admin.136578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169635.log 2023-10-03T10:24:05.264 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.170588.log.gz 2023-10-03T10:24:05.264 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.136578.log.gz 2023-10-03T10:24:05.264 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.175302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151121.log 2023-10-03T10:24:05.264 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.175302.log.gz 2023-10-03T10:24:05.265 INFO:teuthology.orchestra.run.smithi003.stderr:gzip /var/log/ceph/ceph-client.admin.169635.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.181239.log 2023-10-03T10:24:05.265 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.151121.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.169635.log.gz 2023-10-03T10:24:05.265 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.151121.log.gz 2023-10-03T10:24:05.265 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.181239.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.181239.log.gz 2023-10-03T10:24:05.265 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169043.log 2023-10-03T10:24:05.265 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155114.log 2023-10-03T10:24:05.265 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.169043.log: /var/log/ceph/ceph-client.admin.155114.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.169043.log.gz 2023-10-03T10:24:05.265 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160393.log 2023-10-03T10:24:05.266 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.155114.log.gz 2023-10-03T10:24:05.266 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178608.log 2023-10-03T10:24:05.266 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.160393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171998.log 2023-10-03T10:24:05.266 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.160393.log.gz 2023-10-03T10:24:05.266 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.178608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181752.log 2023-10-03T10:24:05.266 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.178608.log.gz 2023-10-03T10:24:05.266 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.171998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153128.log 2023-10-03T10:24:05.266 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.171998.log.gz 2023-10-03T10:24:05.267 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.181752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169217.log 2023-10-03T10:24:05.267 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.181752.log.gz 2023-10-03T10:24:05.267 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.153128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139670.log 2023-10-03T10:24:05.267 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.169217.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.153128.log.gz-5 2023-10-03T10:24:05.267 INFO:teuthology.orchestra.run.smithi003.stderr:--verbose -- /var/log/ceph/ceph-client.admin.141729.log 2023-10-03T10:24:05.267 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.139670.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.169217.log.gz 2023-10-03T10:24:05.267 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.139670.log.gz 2023-10-03T10:24:05.267 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.141729.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.141729.log.gz 2023-10-03T10:24:05.268 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141533.log 2023-10-03T10:24:05.268 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168283.log 2023-10-03T10:24:05.268 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.141533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165249.log 2023-10-03T10:24:05.268 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.141533.log.gz 2023-10-03T10:24:05.268 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.168283.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.168283.log.gz 2023-10-03T10:24:05.268 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.165249.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.165249.log.gz 2023-10-03T10:24:05.268 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122311.log 2023-10-03T10:24:05.268 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126002.log 2023-10-03T10:24:05.269 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.122311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148743.log 2023-10-03T10:24:05.269 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.122311.log.gz 2023-10-03T10:24:05.269 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.126002.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.126002.log.gz 2023-10-03T10:24:05.269 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.148743.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.148743.log.gz 2023-10-03T10:24:05.269 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153742.log 2023-10-03T10:24:05.269 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192791.log 2023-10-03T10:24:05.269 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.153742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137797.log 2023-10-03T10:24:05.269 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.153742.log.gz/var/log/ceph/ceph-client.admin.192791.log: 2023-10-03T10:24:05.270 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168798.log 2023-10-03T10:24:05.270 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.192791.log.gz 2023-10-03T10:24:05.270 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.137797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156042.log 2023-10-03T10:24:05.270 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.168798.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.137797.log.gz 2023-10-03T10:24:05.270 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186534.log 2023-10-03T10:24:05.270 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.168798.log.gz 2023-10-03T10:24:05.270 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.156042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166642.log 2023-10-03T10:24:05.271 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.156042.log.gz--verbose 2023-10-03T10:24:05.271 INFO:teuthology.orchestra.run.smithi003.stderr:-- /var/log/ceph/ceph-client.admin.153516.log 2023-10-03T10:24:05.271 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.186534.log: /var/log/ceph/ceph-client.admin.166642.log: gzip -5 --verbose 87.7%-- /var/log/ceph/ceph-client.admin.183320.log -- replaced with /var/log/ceph/ceph-client.admin.186534.log.gz 2023-10-03T10:24:05.271 INFO:teuthology.orchestra.run.smithi003.stderr: 2023-10-03T10:24:05.271 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.153516.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.166642.log.gz 2023-10-03T10:24:05.271 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185946.log 2023-10-03T10:24:05.271 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.153516.log.gz 2023-10-03T10:24:05.271 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.183320.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.183320.log.gz 2023-10-03T10:24:05.271 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.185946.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.185946.log.gz 2023-10-03T10:24:05.272 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136996.log 2023-10-03T10:24:05.272 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127678.log 2023-10-03T10:24:05.272 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.136996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187831.log 2023-10-03T10:24:05.272 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.136996.log.gz 2023-10-03T10:24:05.272 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.127678.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.127678.log.gz 2023-10-03T10:24:05.272 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.187831.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.187831.log.gz 2023-10-03T10:24:05.272 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161936.log 2023-10-03T10:24:05.272 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160197.log 2023-10-03T10:24:05.273 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.161936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156638.log 2023-10-03T10:24:05.273 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.161936.log.gz 2023-10-03T10:24:05.273 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.160197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171007.log 2023-10-03T10:24:05.273 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.160197.log.gz 2023-10-03T10:24:05.273 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.156638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123370.log 2023-10-03T10:24:05.273 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.156638.log.gz 2023-10-03T10:24:05.273 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.171007.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.171007.log.gz 2023-10-03T10:24:05.273 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.123370.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.123370.log.gz 2023-10-03T10:24:05.274 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193404.log 2023-10-03T10:24:05.274 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153891.log 2023-10-03T10:24:05.274 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.193404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179539.log 2023-10-03T10:24:05.274 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.193404.log.gz 2023-10-03T10:24:05.274 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.153891.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.153891.log.gz 2023-10-03T10:24:05.274 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.179539.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.179539.log.gz 2023-10-03T10:24:05.274 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146775.log 2023-10-03T10:24:05.274 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141702.log 2023-10-03T10:24:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.146775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175498.log 2023-10-03T10:24:05.275 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.146775.log.gz 2023-10-03T10:24:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.141702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158407.log 2023-10-03T10:24:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.175498.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.141702.log.gz 2023-10-03T10:24:05.275 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.175498.log.gz 2023-10-03T10:24:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.158407.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.158407.log.gz 2023-10-03T10:24:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157767.log 2023-10-03T10:24:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180202.log 2023-10-03T10:24:05.276 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.157767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182588.log 2023-10-03T10:24:05.276 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.157767.log.gz 2023-10-03T10:24:05.276 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.180202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153155.log 2023-10-03T10:24:05.276 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.180202.log.gzgzip 2023-10-03T10:24:05.276 INFO:teuthology.orchestra.run.smithi003.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.137586.log 2023-10-03T10:24:05.276 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.182588.log: /var/log/ceph/ceph-client.admin.153155.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.182588.log.gz 2023-10-03T10:24:05.276 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.137586.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.153155.log.gz 2023-10-03T10:24:05.276 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.137586.log.gz 2023-10-03T10:24:05.277 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185868.log 2023-10-03T10:24:05.277 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126051.log 2023-10-03T10:24:05.277 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.185868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164662.log 2023-10-03T10:24:05.277 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.185868.log.gz 2023-10-03T10:24:05.277 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.126051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152127.log 2023-10-03T10:24:05.277 INFO:teuthology.orchestra.run.smithi003.stderr:gzip /var/log/ceph/ceph-client.admin.164662.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.155319.log 2023-10-03T10:24:05.277 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.126051.log.gz 2023-10-03T10:24:05.277 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.152127.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.164662.log.gz 2023-10-03T10:24:05.277 INFO:teuthology.orchestra.run.smithi003.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.152127.log.gz 2023-10-03T10:24:05.278 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.155319.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.155319.log.gz 2023-10-03T10:24:05.278 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175134.log 2023-10-03T10:24:05.278 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150166.log 2023-10-03T10:24:05.344 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.175134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182217.log 2023-10-03T10:24:05.344 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.175134.log.gz 2023-10-03T10:24:05.345 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.150166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170366.log 2023-10-03T10:24:05.345 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174565.log 2023-10-03T10:24:05.345 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.150166.log.gz 2023-10-03T10:24:05.345 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.182217.log: /var/log/ceph/ceph-client.admin.170366.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.182217.log.gz 2023-10-03T10:24:05.345 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.170366.log.gz 2023-10-03T10:24:05.345 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.174565.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.174565.log.gz 2023-10-03T10:24:05.345 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146456.log 2023-10-03T10:24:05.346 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161295.log 2023-10-03T10:24:05.346 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.146456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160119.log 2023-10-03T10:24:05.346 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.146456.log.gz 2023-10-03T10:24:05.346 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.161295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188722.log 2023-10-03T10:24:05.346 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177971.log 89.0% 2023-10-03T10:24:05.346 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.161295.log.gz 2023-10-03T10:24:05.346 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.160119.log: /var/log/ceph/ceph-client.admin.188722.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.160119.log.gz 2023-10-03T10:24:05.346 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179444.log 2023-10-03T10:24:05.347 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.188722.log.gz 2023-10-03T10:24:05.347 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.177971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166399.log 2023-10-03T10:24:05.347 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.177971.log.gz 2023-10-03T10:24:05.347 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.179444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176795.log 2023-10-03T10:24:05.347 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.179444.log.gz 2023-10-03T10:24:05.347 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.166399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138572.log 2023-10-03T10:24:05.347 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.176795.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.166399.log.gz 2023-10-03T10:24:05.347 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184718.log 2023-10-03T10:24:05.348 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.176795.log.gz 2023-10-03T10:24:05.348 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.138572.log: /var/log/ceph/ceph-client.admin.184718.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.138572.log.gz 2023-10-03T10:24:05.348 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.184718.log.gz 2023-10-03T10:24:05.348 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178363.log 2023-10-03T10:24:05.348 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194581.log 2023-10-03T10:24:05.348 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.178363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193719.log 2023-10-03T10:24:05.348 INFO:teuthology.orchestra.run.smithi003.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.178363.log.gz 2023-10-03T10:24:05.348 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.194581.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.194581.log.gz 2023-10-03T10:24:05.348 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.193719.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.193719.log.gz 2023-10-03T10:24:05.349 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137336.log 2023-10-03T10:24:05.349 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173659.log 2023-10-03T10:24:05.349 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.137336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148106.log 2023-10-03T10:24:05.349 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.137336.log.gz/var/log/ceph/ceph-client.admin.173659.log: 2023-10-03T10:24:05.349 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.173659.log.gz 2023-10-03T10:24:05.349 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.148106.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.148106.log.gz 2023-10-03T10:24:05.349 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139135.log 2023-10-03T10:24:05.349 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179836.log 2023-10-03T10:24:05.350 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.139135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143245.log 2023-10-03T10:24:05.350 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.139135.log.gz 2023-10-03T10:24:05.350 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.179836.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.179836.log.gz 2023-10-03T10:24:05.350 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.143245.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.143245.log.gz 2023-10-03T10:24:05.350 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179614.log 2023-10-03T10:24:05.350 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147006.log 2023-10-03T10:24:05.350 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.179614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144029.log 2023-10-03T10:24:05.350 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.179614.log.gz 2023-10-03T10:24:05.351 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.147006.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.147006.log.gz 2023-10-03T10:24:05.351 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.144029.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.144029.log.gz 2023-10-03T10:24:05.351 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168555.log 2023-10-03T10:24:05.351 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136160.log 2023-10-03T10:24:05.351 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.168555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194580.log 2023-10-03T10:24:05.351 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.168555.log.gz 2023-10-03T10:24:05.351 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.136160.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.136160.log.gz 2023-10-03T10:24:05.352 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.194580.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.194580.log.gz 2023-10-03T10:24:05.352 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160733.log 2023-10-03T10:24:05.352 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188126.log 2023-10-03T10:24:05.352 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.160733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180623.log 2023-10-03T10:24:05.352 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.160733.log.gz 2023-10-03T10:24:05.352 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.188126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155880.log 2023-10-03T10:24:05.352 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.188126.log.gz 2023-10-03T10:24:05.352 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.180623.log: /var/log/ceph/ceph-client.admin.155880.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.180623.log.gz 2023-10-03T10:24:05.353 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.155880.log.gz 2023-10-03T10:24:05.353 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176674.log 2023-10-03T10:24:05.353 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166987.log 2023-10-03T10:24:05.353 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.176674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189825.log 2023-10-03T10:24:05.353 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.176674.log.gz 2023-10-03T10:24:05.353 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.166987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188053.log 2023-10-03T10:24:05.353 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142291.log 2023-10-03T10:24:05.354 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.166987.log.gz 2023-10-03T10:24:05.354 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.189825.log: /var/log/ceph/ceph-client.admin.188053.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.189825.log.gz 2023-10-03T10:24:05.354 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.188053.log.gz 2023-10-03T10:24:05.354 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.142291.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.142291.log.gz 2023-10-03T10:24:05.354 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183516.log 2023-10-03T10:24:05.354 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159386.log 2023-10-03T10:24:05.354 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.183516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166595.log 2023-10-03T10:24:05.354 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.183516.log.gz 2023-10-03T10:24:05.355 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.159386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185184.log 2023-10-03T10:24:05.355 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.159386.log.gz 2023-10-03T10:24:05.355 INFO:teuthology.orchestra.run.smithi003.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.189114.log 2023-10-03T10:24:05.355 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.166595.log: /var/log/ceph/ceph-client.admin.185184.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.166595.log.gz 2023-10-03T10:24:05.355 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.189114.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.185184.log.gz 2023-10-03T10:24:05.355 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.189114.log.gz 2023-10-03T10:24:05.355 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143833.log 2023-10-03T10:24:05.355 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152957.log 2023-10-03T10:24:05.356 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.143833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146138.log 2023-10-03T10:24:05.356 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.143833.log.gz 2023-10-03T10:24:05.356 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.152957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141042.log 2023-10-03T10:24:05.356 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.152957.log.gz-5 2023-10-03T10:24:05.356 INFO:teuthology.orchestra.run.smithi003.stderr:--verbose -- /var/log/ceph/ceph-client.admin.140309.log 2023-10-03T10:24:05.356 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.146138.log: /var/log/ceph/ceph-client.admin.141042.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.146138.log.gz 2023-10-03T10:24:05.356 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2%/var/log/ceph/ceph-client.admin.140309.log: -- replaced with /var/log/ceph/ceph-client.admin.141042.log.gz 2023-10-03T10:24:05.356 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.140309.log.gz 2023-10-03T10:24:05.357 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138172.log 2023-10-03T10:24:05.357 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156591.log 2023-10-03T10:24:05.357 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.138172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189655.log 2023-10-03T10:24:05.357 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.138172.log.gz 2023-10-03T10:24:05.357 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.156591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145868.log 2023-10-03T10:24:05.357 INFO:teuthology.orchestra.run.smithi003.stderr:gzip 87.6%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.156591.log.gz-- 2023-10-03T10:24:05.357 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.139696.log 2023-10-03T10:24:05.357 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.189655.log: /var/log/ceph/ceph-client.admin.145868.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.189655.log.gz 2023-10-03T10:24:05.358 INFO:teuthology.orchestra.run.smithi003.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.145868.log.gz 2023-10-03T10:24:05.358 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.139696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176403.log 2023-10-03T10:24:05.358 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.139696.log.gz 2023-10-03T10:24:05.358 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163307.log 2023-10-03T10:24:05.358 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182981.log 2023-10-03T10:24:05.358 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.176403.log: /var/log/ceph/ceph-client.admin.163307.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.176403.log.gz 2023-10-03T10:24:05.358 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143102.log 2023-10-03T10:24:05.358 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.163307.log.gz 2023-10-03T10:24:05.359 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.182981.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.182981.log.gz 2023-10-03T10:24:05.359 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.143102.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.143102.log.gz 2023-10-03T10:24:05.359 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169830.log 2023-10-03T10:24:05.359 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161909.log 2023-10-03T10:24:05.359 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.169830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179000.log 2023-10-03T10:24:05.359 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.161909.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.169830.log.gzgzip 2023-10-03T10:24:05.359 INFO:teuthology.orchestra.run.smithi003.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.139939.log 2023-10-03T10:24:05.359 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.161909.log.gz 2023-10-03T10:24:05.360 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.179000.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.179000.log.gz 2023-10-03T10:24:05.360 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.139939.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.139939.log.gz 2023-10-03T10:24:05.360 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162471.log 2023-10-03T10:24:05.360 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148939.log 2023-10-03T10:24:05.360 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.162471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176112.log 2023-10-03T10:24:05.360 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.162471.log.gz 2023-10-03T10:24:05.360 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.148939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175890.log 2023-10-03T10:24:05.360 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.148939.log.gz 2023-10-03T10:24:05.360 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.176112.log: /var/log/ceph/ceph-client.admin.175890.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.176112.log.gz 2023-10-03T10:24:05.361 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.175890.log.gz 2023-10-03T10:24:05.361 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137310.log 2023-10-03T10:24:05.361 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156300.log 2023-10-03T10:24:05.361 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.137310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190486.log 2023-10-03T10:24:05.361 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.137310.log.gz 2023-10-03T10:24:05.361 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.156300.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.156300.log.gz 2023-10-03T10:24:05.361 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.190486.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.190486.log.gz 2023-10-03T10:24:05.361 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163511.log 2023-10-03T10:24:05.362 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169608.log 2023-10-03T10:24:05.362 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.163511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161687.log 2023-10-03T10:24:05.362 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.163511.log.gz 2023-10-03T10:24:05.362 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.169608.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.169608.log.gz 2023-10-03T10:24:05.362 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.161687.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.161687.log.gz 2023-10-03T10:24:05.362 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152859.log 2023-10-03T10:24:05.362 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158159.log 2023-10-03T10:24:05.362 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.152859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179343.log 2023-10-03T10:24:05.363 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.152859.log.gz 2023-10-03T10:24:05.363 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.158159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128646.log 2023-10-03T10:24:05.363 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.158159.log.gz 2023-10-03T10:24:05.363 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.179343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165615.log 2023-10-03T10:24:05.363 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.128646.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.179343.log.gz 2023-10-03T10:24:05.363 INFO:teuthology.orchestra.run.smithi003.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.128646.log.gz 2023-10-03T10:24:05.363 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.165615.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.165615.log.gz 2023-10-03T10:24:05.363 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188649.log 2023-10-03T10:24:05.364 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189898.log 2023-10-03T10:24:05.364 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.188649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146848.log 2023-10-03T10:24:05.364 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4%/var/log/ceph/ceph-client.admin.189898.log: -- replaced with /var/log/ceph/ceph-client.admin.188649.log.gz 2023-10-03T10:24:05.364 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176504.log 2023-10-03T10:24:05.364 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.189898.log.gz 2023-10-03T10:24:05.364 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.146848.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.146848.log.gz 2023-10-03T10:24:05.364 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.176504.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.176504.log.gz 2023-10-03T10:24:05.364 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179052.log 2023-10-03T10:24:05.365 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187635.log 2023-10-03T10:24:05.365 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.179052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179222.log 2023-10-03T10:24:05.365 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.179052.log.gz 2023-10-03T10:24:05.365 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.187635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148302.log 2023-10-03T10:24:05.365 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.187635.log.gz 2023-10-03T10:24:05.365 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.179222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187143.log 2023-10-03T10:24:05.365 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.179222.log.gz 2023-10-03T10:24:05.365 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.148302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136968.log 2023-10-03T10:24:05.365 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.148302.log.gz 2023-10-03T10:24:05.366 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.187143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134494.log 2023-10-03T10:24:05.366 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.187143.log.gz 2023-10-03T10:24:05.366 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.136968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193797.log 2023-10-03T10:24:05.366 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.136968.log.gz 2023-10-03T10:24:05.366 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.134494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174296.log 2023-10-03T10:24:05.366 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.134494.log.gz 2023-10-03T10:24:05.366 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.193797.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.193797.log.gz 2023-10-03T10:24:05.366 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.174296.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.174296.log.gz 2023-10-03T10:24:05.367 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165492.log 2023-10-03T10:24:05.367 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139304.log 2023-10-03T10:24:05.367 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.165492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127473.log 2023-10-03T10:24:05.367 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0%/var/log/ceph/ceph-client.admin.139304.log: -- replaced with /var/log/ceph/ceph-client.admin.165492.log.gz 2023-10-03T10:24:05.367 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.139304.log.gz 2023-10-03T10:24:05.367 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.127473.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.127473.log.gz 2023-10-03T10:24:05.367 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139748.log 2023-10-03T10:24:05.367 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156664.log 2023-10-03T10:24:05.368 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.139748.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-03T10:24:05.368 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.139748.log.gz 2023-10-03T10:24:05.368 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.156664.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.156664.log.gz 2023-10-03T10:24:05.368 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192790.log 2023-10-03T10:24:05.368 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158627.log 2023-10-03T10:24:05.368 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.192790.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.192790.log.gz 2023-10-03T10:24:05.368 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.158627.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.158627.log.gz 2023-10-03T10:24:05.368 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150378.log 2023-10-03T10:24:05.369 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195405.log 2023-10-03T10:24:05.369 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.150378.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.150378.log.gz 2023-10-03T10:24:05.369 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.195405.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.195405.log.gz 2023-10-03T10:24:05.369 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123321.log 2023-10-03T10:24:05.369 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144055.log 2023-10-03T10:24:05.369 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.123321.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.123321.log.gz 2023-10-03T10:24:05.369 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.144055.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.144055.log.gz 2023-10-03T10:24:05.369 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165910.log 2023-10-03T10:24:05.370 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142022.log 2023-10-03T10:24:05.370 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.165910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174518.log 2023-10-03T10:24:05.370 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.165910.log.gz 2023-10-03T10:24:05.370 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.142022.log: /var/log/ceph/ceph-client.admin.174518.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.142022.log.gz 2023-10-03T10:24:05.370 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.174518.log.gz 2023-10-03T10:24:05.370 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125138.log 2023-10-03T10:24:05.370 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157793.log 2023-10-03T10:24:05.370 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.125138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194111.log 2023-10-03T10:24:05.371 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.125138.log.gz 2023-10-03T10:24:05.371 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.157793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140507.log 2023-10-03T10:24:05.371 INFO:teuthology.orchestra.run.smithi003.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.157793.log.gz 2023-10-03T10:24:05.371 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.194111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166498.log 2023-10-03T10:24:05.371 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.194111.log.gz 2023-10-03T10:24:05.371 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.140507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142513.log 2023-10-03T10:24:05.371 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.140507.log.gz 2023-10-03T10:24:05.371 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.166498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128335.log 2023-10-03T10:24:05.372 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.166498.log.gz 2023-10-03T10:24:05.372 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.142513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136212.log 2023-10-03T10:24:05.372 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.142513.log.gz 2023-10-03T10:24:05.372 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.128335.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.128335.log.gz 2023-10-03T10:24:05.372 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.136212.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.136212.log.gz 2023-10-03T10:24:05.372 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164243.log 2023-10-03T10:24:05.372 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137140.log 2023-10-03T10:24:05.372 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.164243.log: /var/log/ceph/ceph-client.admin.137140.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.164243.log.gz 2023-10-03T10:24:05.373 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171150.log 2023-10-03T10:24:05.373 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.137140.log.gz 2023-10-03T10:24:05.373 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185724.log 2023-10-03T10:24:05.373 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.171150.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.171150.log.gz 2023-10-03T10:24:05.373 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.185724.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.185724.log.gz 2023-10-03T10:24:05.373 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153251.log 2023-10-03T10:24:05.373 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192398.log 2023-10-03T10:24:05.373 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.153251.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.153251.log.gz 2023-10-03T10:24:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.192398.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.192398.log.gz 2023-10-03T10:24:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147959.log 2023-10-03T10:24:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189799.log 2023-10-03T10:24:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.147959.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.147959.log.gz 2023-10-03T10:24:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.189799.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.189799.log.gz 2023-10-03T10:24:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160145.log 2023-10-03T10:24:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180380.log 2023-10-03T10:24:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.160145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152297.log 2023-10-03T10:24:05.375 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.160145.log.gz 2023-10-03T10:24:05.375 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.180380.log: /var/log/ceph/ceph-client.admin.152297.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.180380.log.gz 2023-10-03T10:24:05.375 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.152297.log.gz 2023-10-03T10:24:05.375 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181387.log 2023-10-03T10:24:05.375 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185575.log 2023-10-03T10:24:05.375 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.181387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165884.log 2023-10-03T10:24:05.375 INFO:teuthology.orchestra.run.smithi003.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.181387.log.gz 2023-10-03T10:24:05.375 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.185575.log: /var/log/ceph/ceph-client.admin.165884.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.185575.log.gz 2023-10-03T10:24:05.376 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.165884.log.gz 2023-10-03T10:24:05.376 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151434.log 2023-10-03T10:24:05.376 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174639.log 2023-10-03T10:24:05.376 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.151434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178268.log 2023-10-03T10:24:05.376 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.151434.log.gz 2023-10-03T10:24:05.376 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.174639.log: /var/log/ceph/ceph-client.admin.178268.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.174639.log.gz 2023-10-03T10:24:05.376 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.178268.log.gz 2023-10-03T10:24:05.376 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135499.log 2023-10-03T10:24:05.377 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177066.log 2023-10-03T10:24:05.377 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.135499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149086.log 2023-10-03T10:24:05.377 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.135499.log.gz 2023-10-03T10:24:05.377 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.177066.log: /var/log/ceph/ceph-client.admin.149086.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.177066.log.gz 2023-10-03T10:24:05.377 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.149086.log.gz 2023-10-03T10:24:05.377 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172214.log 2023-10-03T10:24:05.377 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137506.log 2023-10-03T10:24:05.377 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.172214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159778.log 2023-10-03T10:24:05.378 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.172214.log.gz 2023-10-03T10:24:05.378 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.137506.log: /var/log/ceph/ceph-client.admin.159778.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.137506.log.gz 2023-10-03T10:24:05.378 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.159778.log.gz 2023-10-03T10:24:05.378 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118578.log 2023-10-03T10:24:05.378 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172605.log 2023-10-03T10:24:05.378 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.118578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124302.log 2023-10-03T10:24:05.378 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.118578.log.gz 2023-10-03T10:24:05.378 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.172605.log: /var/log/ceph/ceph-client.admin.124302.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.172605.log.gz 2023-10-03T10:24:05.379 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.124302.log.gz 2023-10-03T10:24:05.379 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187070.log 2023-10-03T10:24:05.379 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186704.log 2023-10-03T10:24:05.379 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.187070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168871.log 2023-10-03T10:24:05.379 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.187070.log.gz 2023-10-03T10:24:05.379 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.186704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186848.log 2023-10-03T10:24:05.379 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.186704.log.gz 2023-10-03T10:24:05.379 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.168871.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.168871.log.gz 2023-10-03T10:24:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.186848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148547.log 2023-10-03T10:24:05.380 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.186848.log.gz 2023-10-03T10:24:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193601.log 2023-10-03T10:24:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.148547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184224.log 2023-10-03T10:24:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.193601.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.148547.log.gz 2023-10-03T10:24:05.380 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.193601.log.gz 2023-10-03T10:24:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.184224.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.184224.log.gz 2023-10-03T10:24:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158577.log 2023-10-03T10:24:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174688.log 2023-10-03T10:24:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.158577.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.158577.log.gz 2023-10-03T10:24:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.174688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188419.log 2023-10-03T10:24:05.381 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.174688.log.gz 2023-10-03T10:24:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169535.log 2023-10-03T10:24:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.188419.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.188419.log.gz 2023-10-03T10:24:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.169535.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.169535.log.gz 2023-10-03T10:24:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152271.log 2023-10-03T10:24:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187465.log 2023-10-03T10:24:05.382 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.152271.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.152271.log.gz 2023-10-03T10:24:05.382 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.187465.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.187465.log.gz 2023-10-03T10:24:05.382 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155684.log 2023-10-03T10:24:05.382 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124840.log 2023-10-03T10:24:05.382 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.155684.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.155684.log.gz 2023-10-03T10:24:05.382 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.124840.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.124840.log.gz 2023-10-03T10:24:05.382 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182340.log 2023-10-03T10:24:05.382 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141630.log 2023-10-03T10:24:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.182340.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.182340.log.gz 2023-10-03T10:24:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.141630.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.141630.log.gz 2023-10-03T10:24:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165697.log 2023-10-03T10:24:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151147.log 2023-10-03T10:24:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.165697.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.165697.log.gz 2023-10-03T10:24:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.151147.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.151147.log.gz 2023-10-03T10:24:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194699.log 2023-10-03T10:24:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150899.log 2023-10-03T10:24:05.384 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.194699.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.194699.log.gz 2023-10-03T10:24:05.384 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.150899.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.150899.log.gz 2023-10-03T10:24:05.384 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128190.log 2023-10-03T10:24:05.384 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146383.log 2023-10-03T10:24:05.384 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.128190.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.128190.log.gz 2023-10-03T10:24:05.384 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.146383.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.146383.log.gz 2023-10-03T10:24:05.384 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163059.log 2023-10-03T10:24:05.384 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122768.log 2023-10-03T10:24:05.385 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.163059.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.163059.log.gz 2023-10-03T10:24:05.385 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.122768.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.122768.log.gz 2023-10-03T10:24:05.385 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168014.log 2023-10-03T10:24:05.385 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156468.log 2023-10-03T10:24:05.385 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.168014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151683.log 2023-10-03T10:24:05.385 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.168014.log.gz 2023-10-03T10:24:05.385 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.156468.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.156468.log.gz 2023-10-03T10:24:05.385 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.151683.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.151683.log.gz 2023-10-03T10:24:05.386 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179640.log 2023-10-03T10:24:05.386 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125089.log 2023-10-03T10:24:05.386 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.179640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172875.log 2023-10-03T10:24:05.386 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.179640.log.gz 2023-10-03T10:24:05.386 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.125089.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.125089.log.gz 2023-10-03T10:24:05.386 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.172875.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.172875.log.gz 2023-10-03T10:24:05.386 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163950.log 2023-10-03T10:24:05.386 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123615.log 2023-10-03T10:24:05.387 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.163950.log: /var/log/ceph/ceph-client.admin.123615.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.163950.log.gz 2023-10-03T10:24:05.387 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.123615.log.gz 2023-10-03T10:24:05.387 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142905.log 2023-10-03T10:24:05.387 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149527.log 2023-10-03T10:24:05.387 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.142905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148620.log 2023-10-03T10:24:05.387 INFO:teuthology.orchestra.run.smithi003.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-03T10:24:05.387 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.142905.log.gz 2023-10-03T10:24:05.387 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.149527.log: /var/log/ceph/ceph-client.admin.148620.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.149527.log.gz 2023-10-03T10:24:05.388 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.148620.log.gz 2023-10-03T10:24:05.388 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184692.log 2023-10-03T10:24:05.388 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161491.log 2023-10-03T10:24:05.388 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.184692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143786.log 2023-10-03T10:24:05.388 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.184692.log.gz 2023-10-03T10:24:05.388 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.161491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156113.log 2023-10-03T10:24:05.388 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose 89.2%-- /var/log/ceph/ceph-client.admin.169263.log -- replaced with /var/log/ceph/ceph-client.admin.161491.log.gz 2023-10-03T10:24:05.388 INFO:teuthology.orchestra.run.smithi003.stderr: 2023-10-03T10:24:05.389 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.143786.log: /var/log/ceph/ceph-client.admin.156113.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.143786.log.gz 2023-10-03T10:24:05.389 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183223.log 2023-10-03T10:24:05.389 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.156113.log.gz 2023-10-03T10:24:05.389 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.169263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166250.log 2023-10-03T10:24:05.389 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.183223.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.169263.log.gz 2023-10-03T10:24:05.389 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174939.log 2023-10-03T10:24:05.389 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.183223.log.gz 2023-10-03T10:24:05.389 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.166250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143637.log 2023-10-03T10:24:05.390 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.166250.log.gz 2023-10-03T10:24:05.390 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.174939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123664.log 2023-10-03T10:24:05.390 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.174939.log.gz 2023-10-03T10:24:05.390 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.143637.log: /var/log/ceph/ceph-client.admin.123664.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.143637.log.gz 2023-10-03T10:24:05.390 INFO:teuthology.orchestra.run.smithi003.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.123664.log.gz 2023-10-03T10:24:05.390 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192347.log 2023-10-03T10:24:05.390 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159217.log 2023-10-03T10:24:05.390 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.192347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189852.log 2023-10-03T10:24:05.391 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.192347.log.gz 2023-10-03T10:24:05.391 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.159217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127301.log 2023-10-03T10:24:05.391 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185672.log 2023-10-03T10:24:05.391 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.159217.log.gz 2023-10-03T10:24:05.391 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.189852.log: /var/log/ceph/ceph-client.admin.127301.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.189852.log.gz 2023-10-03T10:24:05.391 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123889.log 2023-10-03T10:24:05.391 INFO:teuthology.orchestra.run.smithi003.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.127301.log.gz 2023-10-03T10:24:05.391 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.185672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126100.log 2023-10-03T10:24:05.391 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.185672.log.gz 2023-10-03T10:24:05.392 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.123889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158773.log 2023-10-03T10:24:05.392 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.126100.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.123889.log.gz 2023-10-03T10:24:05.392 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184522.log 2023-10-03T10:24:05.392 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.158773.log: 88.6% 90.7% -- replaced with /var/log/ceph/ceph-client.admin.158773.log.gz -- replaced with /var/log/ceph/ceph-client.admin.126100.log.gz 2023-10-03T10:24:05.392 INFO:teuthology.orchestra.run.smithi003.stderr: 2023-10-03T10:24:05.392 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.184522.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.184522.log.gz 2023-10-03T10:24:05.392 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139331.log 2023-10-03T10:24:05.392 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150952.log 2023-10-03T10:24:05.393 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.139331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143298.log 2023-10-03T10:24:05.393 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.139331.log.gz 2023-10-03T10:24:05.393 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.150952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175227.log 2023-10-03T10:24:05.393 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.150952.log.gz 2023-10-03T10:24:05.393 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.143298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123468.log 2023-10-03T10:24:05.393 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123887.log 2023-10-03T10:24:05.393 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.175227.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.143298.log.gz 2023-10-03T10:24:05.393 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.123468.log: gzip 89.4%-5 -- replaced with /var/log/ceph/ceph-client.admin.175227.log.gz--verbose 2023-10-03T10:24:05.394 INFO:teuthology.orchestra.run.smithi003.stderr:-- /var/log/ceph/ceph-client.admin.142683.log 2023-10-03T10:24:05.394 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.123468.log.gz 2023-10-03T10:24:05.394 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.123887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163281.log 2023-10-03T10:24:05.394 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.123887.log.gz 2023-10-03T10:24:05.394 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.142683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160537.log 2023-10-03T10:24:05.394 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.142683.log.gz 2023-10-03T10:24:05.394 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.163281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194503.log 2023-10-03T10:24:05.394 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.163281.log.gz 2023-10-03T10:24:05.395 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.160537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148890.log 2023-10-03T10:24:05.395 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.160537.log.gz 2023-10-03T10:24:05.395 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.194503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169582.log 2023-10-03T10:24:05.395 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.194503.log.gz 2023-10-03T10:24:05.395 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.148890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163753.log 2023-10-03T10:24:05.395 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.148890.log.gz 2023-10-03T10:24:05.395 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.169582.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.169582.log.gz 2023-10-03T10:24:05.395 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.163753.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.163753.log.gz 2023-10-03T10:24:05.395 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149331.log 2023-10-03T10:24:05.396 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157858.log 2023-10-03T10:24:05.396 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128237.log 2023-10-03T10:24:05.396 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.149331.log: /var/log/ceph/ceph-client.admin.157858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185137.log 2023-10-03T10:24:05.396 INFO:teuthology.orchestra.run.smithi003.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.149331.log.gz 2023-10-03T10:24:05.396 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.157858.log.gz/var/log/ceph/ceph-client.admin.128237.log: 2023-10-03T10:24:05.396 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128940.log 2023-10-03T10:24:05.396 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.128237.log.gz 2023-10-03T10:24:05.396 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.185137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188027.log 2023-10-03T10:24:05.397 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.128940.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.185137.log.gz 2023-10-03T10:24:05.397 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.128940.log.gz 2023-10-03T10:24:05.397 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.188027.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.188027.log.gz 2023-10-03T10:24:05.397 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124033.log 2023-10-03T10:24:05.397 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174443.log 2023-10-03T10:24:05.397 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.124033.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.124033.log.gz 2023-10-03T10:24:05.397 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.174443.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.174443.log.gz 2023-10-03T10:24:05.397 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155658.log 2023-10-03T10:24:05.398 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155062.log 2023-10-03T10:24:05.398 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.155658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144477.log 2023-10-03T10:24:05.398 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.155658.log.gz 2023-10-03T10:24:05.398 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.155062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129596.log 2023-10-03T10:24:05.398 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141292.log 2023-10-03T10:24:05.398 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.155062.log.gz 2023-10-03T10:24:05.398 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.144477.log: /var/log/ceph/ceph-client.admin.129596.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.144477.log.gz 2023-10-03T10:24:05.398 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181360.log 2023-10-03T10:24:05.399 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.129596.log.gz 2023-10-03T10:24:05.399 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.141292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147567.log 2023-10-03T10:24:05.399 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.141292.log.gz 2023-10-03T10:24:05.399 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.181360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135475.log 2023-10-03T10:24:05.399 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.147567.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.181360.log.gz 2023-10-03T10:24:05.399 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167695.log 2023-10-03T10:24:05.399 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.147567.log.gz 2023-10-03T10:24:05.399 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.135475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138768.log 2023-10-03T10:24:05.400 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158747.log 2023-10-03T10:24:05.400 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.167695.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.135475.log.gz 2023-10-03T10:24:05.400 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.138768.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.167695.log.gz 2023-10-03T10:24:05.400 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175864.log 2023-10-03T10:24:05.400 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.138768.log.gz 2023-10-03T10:24:05.400 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.158747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127302.log 2023-10-03T10:24:05.400 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.158747.log.gz 2023-10-03T10:24:05.400 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.175864.log: /var/log/ceph/ceph-client.admin.127302.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.175864.log.gz 2023-10-03T10:24:05.401 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170170.log 2023-10-03T10:24:05.401 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.127302.log.gz 2023-10-03T10:24:05.401 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155515.log 2023-10-03T10:24:05.401 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184352.log 2023-10-03T10:24:05.401 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.170170.log: /var/log/ceph/ceph-client.admin.155515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128842.log 2023-10-03T10:24:05.401 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.170170.log.gz 2023-10-03T10:24:05.401 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.155515.log.gz 2023-10-03T10:24:05.401 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.184352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174247.log 2023-10-03T10:24:05.401 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.184352.log.gz 2023-10-03T10:24:05.402 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.128842.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.128842.log.gz 2023-10-03T10:24:05.402 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.174247.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.174247.log.gz 2023-10-03T10:24:05.402 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138716.log 2023-10-03T10:24:05.402 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-10-03T10:24:05.402 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.138716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191981.log 2023-10-03T10:24:05.402 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.138716.log.gz 2023-10-03T10:24:05.402 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167219.log 2023-10-03T10:24:05.402 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176086.log 2023-10-03T10:24:05.403 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.191981.log: /var/log/ceph/ceph-client.admin.167219.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.191981.log.gz 2023-10-03T10:24:05.403 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.167219.log.gz 2023-10-03T10:24:05.403 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.176086.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.176086.log.gz 2023-10-03T10:24:05.403 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166668.log 2023-10-03T10:24:05.403 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183933.log 2023-10-03T10:24:05.403 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.166668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193131.log 2023-10-03T10:24:05.403 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.166668.log.gz 2023-10-03T10:24:05.403 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.183933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186678.log 2023-10-03T10:24:05.404 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.183933.log.gz 2023-10-03T10:24:05.404 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.193131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151879.log 2023-10-03T10:24:05.404 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.193131.log.gz 2023-10-03T10:24:05.404 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.186678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191367.log 2023-10-03T10:24:05.404 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.186678.log.gz 2023-10-03T10:24:05.404 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.151879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162547.log 2023-10-03T10:24:05.404 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.151879.log.gz 2023-10-03T10:24:05.404 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.191367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186456.log 2023-10-03T10:24:05.405 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.191367.log.gz 2023-10-03T10:24:05.405 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.162547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184400.log 2023-10-03T10:24:05.405 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.162547.log.gz 2023-10-03T10:24:05.405 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.186456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123713.log 2023-10-03T10:24:05.405 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.186456.log.gz 2023-10-03T10:24:05.405 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.184400.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-10-03T10:24:05.405 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.184400.log.gz 2023-10-03T10:24:05.405 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.123713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190021.log 2023-10-03T10:24:05.405 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.123713.log.gz 2023-10-03T10:24:05.406 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-03T10:24:05.406 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.190021.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.190021.log.gz 2023-10-03T10:24:05.406 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118625.log 2023-10-03T10:24:05.406 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.118625.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.118625.log.gz 2023-10-03T10:24:05.406 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172138.log 2023-10-03T10:24:05.406 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.172138.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.172138.log.gz 2023-10-03T10:24:05.406 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142853.log 2023-10-03T10:24:05.407 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.142853.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.142853.log.gz 2023-10-03T10:24:05.407 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158943.log 2023-10-03T10:24:05.407 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.158943.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.158943.log.gz 2023-10-03T10:24:05.407 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158995.log 2023-10-03T10:24:05.407 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.158995.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.158995.log.gz 2023-10-03T10:24:05.407 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190218.log 2023-10-03T10:24:05.407 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.190218.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.190218.log.gz 2023-10-03T10:24:05.407 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183124.log 2023-10-03T10:24:05.407 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.183124.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.183124.log.gz 2023-10-03T10:24:05.408 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167060.log 2023-10-03T10:24:05.408 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.167060.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.167060.log.gz 2023-10-03T10:24:05.408 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144839.log 2023-10-03T10:24:05.408 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.144839.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.144839.log.gz 2023-10-03T10:24:05.408 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151539.log 2023-10-03T10:24:05.408 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.151539.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.151539.log.gz 2023-10-03T10:24:05.408 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167575.log 2023-10-03T10:24:05.408 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.167575.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.167575.log.gz 2023-10-03T10:24:05.409 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152126.log 2023-10-03T10:24:05.409 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.152126.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.152126.log.gz 2023-10-03T10:24:05.409 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135305.log 2023-10-03T10:24:05.409 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.135305.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.135305.log.gz 2023-10-03T10:24:05.409 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182411.log 2023-10-03T10:24:05.409 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.182411.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.182411.log.gz 2023-10-03T10:24:05.409 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151538.log 2023-10-03T10:24:05.409 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.151538.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.151538.log.gz 2023-10-03T10:24:05.410 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159779.log 2023-10-03T10:24:05.410 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.159779.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.159779.log.gz 2023-10-03T10:24:05.410 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186359.log 2023-10-03T10:24:05.410 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.186359.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.186359.log.gz 2023-10-03T10:24:05.410 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171764.log 2023-10-03T10:24:05.410 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.171764.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.171764.log.gz 2023-10-03T10:24:05.410 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154278.log 2023-10-03T10:24:05.410 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.154278.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.154278.log.gz 2023-10-03T10:24:05.411 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159609.log 2023-10-03T10:24:05.411 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.159609.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.159609.log.gz 2023-10-03T10:24:05.411 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193013.log 2023-10-03T10:24:05.411 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.193013.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.193013.log.gz 2023-10-03T10:24:05.411 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145822.log 2023-10-03T10:24:05.411 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139108.log 2023-10-03T10:24:05.411 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.145822.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.145822.log.gz 2023-10-03T10:24:05.411 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.139108.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.139108.log.gz 2023-10-03T10:24:05.411 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181480.log 2023-10-03T10:24:05.412 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.181480.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.181480.log.gz 2023-10-03T10:24:05.412 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171299.log 2023-10-03T10:24:05.412 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.171299.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.171299.log.gz 2023-10-03T10:24:05.412 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186286.log 2023-10-03T10:24:05.412 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.186286.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.186286.log.gz 2023-10-03T10:24:05.412 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160196.log 2023-10-03T10:24:05.412 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.160196.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.160196.log.gz 2023-10-03T10:24:05.412 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162889.log 2023-10-03T10:24:05.413 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.162889.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.162889.log.gz 2023-10-03T10:24:05.413 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157375.log 2023-10-03T10:24:05.413 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.157375.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.157375.log.gz 2023-10-03T10:24:05.413 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167408.log 2023-10-03T10:24:05.413 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.167408.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.167408.log.gz 2023-10-03T10:24:05.413 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156395.log 2023-10-03T10:24:05.413 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.156395.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.156395.log.gz 2023-10-03T10:24:05.413 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122966.log 2023-10-03T10:24:05.414 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.122966.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.122966.log.gz 2023-10-03T10:24:05.414 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180649.log 2023-10-03T10:24:05.414 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.180649.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.180649.log.gz 2023-10-03T10:24:05.414 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170027.log 2023-10-03T10:24:05.414 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.170027.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.170027.log.gz 2023-10-03T10:24:05.414 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154231.log 2023-10-03T10:24:05.414 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.154231.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.154231.log.gz 2023-10-03T10:24:05.414 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190482.log 2023-10-03T10:24:05.415 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.190482.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.190482.log.gz 2023-10-03T10:24:05.415 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170000.log 2023-10-03T10:24:05.415 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.170000.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.170000.log.gz 2023-10-03T10:24:05.415 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137925.log 2023-10-03T10:24:05.415 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.137925.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.137925.log.gz 2023-10-03T10:24:05.415 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191467.log 2023-10-03T10:24:05.415 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.191467.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.191467.log.gz 2023-10-03T10:24:05.415 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194188.log 2023-10-03T10:24:05.415 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.194188.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.194188.log.gz 2023-10-03T10:24:05.416 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170784.log 2023-10-03T10:24:05.416 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.170784.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.170784.log.gz 2023-10-03T10:24:05.416 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184744.log 2023-10-03T10:24:05.416 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.184744.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.184744.log.gz 2023-10-03T10:24:05.416 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144082.log 2023-10-03T10:24:05.416 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.144082.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.144082.log.gz 2023-10-03T10:24:05.416 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162178.log 2023-10-03T10:24:05.416 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.162178.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.162178.log.gz 2023-10-03T10:24:05.417 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146064.log 2023-10-03T10:24:05.417 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.146064.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.146064.log.gz 2023-10-03T10:24:05.417 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149778.log 2023-10-03T10:24:05.417 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.149778.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.149778.log.gz 2023-10-03T10:24:05.417 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176060.log 2023-10-03T10:24:05.417 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.176060.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.176060.log.gz 2023-10-03T10:24:05.417 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187339.log 2023-10-03T10:24:05.442 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.187339.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.187339.log.gz 2023-10-03T10:24:05.443 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145205.log 2023-10-03T10:24:05.443 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.145205.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.145205.log.gz 2023-10-03T10:24:05.443 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135817.log 2023-10-03T10:24:05.443 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.135817.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.135817.log.gz 2023-10-03T10:24:05.443 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159216.log 2023-10-03T10:24:05.443 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.159216.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.159216.log.gz 2023-10-03T10:24:05.443 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145599.log 2023-10-03T10:24:05.444 INFO:teuthology.orchestra.run.smithi003.stderr: 94.7% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-10-03T10:24:05.444 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.145599.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.145599.log.gz 2023-10-03T10:24:05.444 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170319.log 2023-10-03T10:24:05.444 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-10-03T10:24:05.444 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.170319.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.170319.log.gz 2023-10-03T10:24:05.444 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164539.log 2023-10-03T10:24:05.444 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.164539.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.164539.log.gz 2023-10-03T10:24:05.444 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149357.log 2023-10-03T10:24:05.445 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.149357.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.149357.log.gz 2023-10-03T10:24:05.445 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144766.log 2023-10-03T10:24:05.445 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.144766.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.144766.log.gz 2023-10-03T10:24:05.445 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136798.log 2023-10-03T10:24:05.445 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.136798.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.136798.log.gz 2023-10-03T10:24:05.445 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129287.log 2023-10-03T10:24:05.445 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.129287.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.129287.log.gz 2023-10-03T10:24:05.445 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128891.log 2023-10-03T10:24:05.446 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.128891.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.128891.log.gz 2023-10-03T10:24:05.446 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193405.log 2023-10-03T10:24:05.446 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.193405.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.193405.log.gz 2023-10-03T10:24:05.446 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185750.log 2023-10-03T10:24:05.446 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.185750.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.185750.log.gz 2023-10-03T10:24:05.446 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148816.log 2023-10-03T10:24:05.446 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.148816.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.148816.log.gz 2023-10-03T10:24:05.446 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137214.log 2023-10-03T10:24:05.447 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.137214.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.137214.log.gz 2023-10-03T10:24:05.447 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142414.log 2023-10-03T10:24:05.447 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.142414.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.142414.log.gz 2023-10-03T10:24:05.447 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173316.log 2023-10-03T10:24:05.447 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.173316.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.173316.log.gz 2023-10-03T10:24:05.447 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187884.log 2023-10-03T10:24:05.447 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.187884.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.187884.log.gz 2023-10-03T10:24:05.447 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162915.log 2023-10-03T10:24:05.447 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.162915.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.162915.log.gz 2023-10-03T10:24:05.448 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194384.log 2023-10-03T10:24:05.448 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.194384.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.194384.log.gz 2023-10-03T10:24:05.448 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191589.log 2023-10-03T10:24:05.448 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.191589.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.191589.log.gz 2023-10-03T10:24:05.448 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176870.log 2023-10-03T10:24:05.448 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.176870.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.176870.log.gz 2023-10-03T10:24:05.448 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140480.log 2023-10-03T10:24:05.448 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.140480.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.140480.log.gz 2023-10-03T10:24:05.449 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124255.log 2023-10-03T10:24:05.449 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.124255.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.124255.log.gz 2023-10-03T10:24:05.449 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142806.log 2023-10-03T10:24:05.449 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.142806.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.142806.log.gz 2023-10-03T10:24:05.449 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188445.log 2023-10-03T10:24:05.449 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.188445.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.188445.log.gz 2023-10-03T10:24:05.449 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124991.log 2023-10-03T10:24:05.449 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.124991.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.124991.log.gz 2023-10-03T10:24:05.450 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156272.log 2023-10-03T10:24:05.450 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.156272.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.156272.log.gz 2023-10-03T10:24:05.450 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148032.log 2023-10-03T10:24:05.450 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.148032.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.148032.log.gz 2023-10-03T10:24:05.450 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167480.log 2023-10-03T10:24:05.450 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.167480.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.167480.log.gz 2023-10-03T10:24:05.450 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155488.log 2023-10-03T10:24:05.450 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.155488.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.155488.log.gz 2023-10-03T10:24:05.451 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179418.log 2023-10-03T10:24:05.451 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.179418.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.179418.log.gz 2023-10-03T10:24:05.451 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178661.log 2023-10-03T10:24:05.451 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.178661.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.178661.log.gz 2023-10-03T10:24:05.451 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164635.log 2023-10-03T10:24:05.451 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.164635.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.164635.log.gz 2023-10-03T10:24:05.451 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175916.log 2023-10-03T10:24:05.451 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.175916.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.175916.log.gz 2023-10-03T10:24:05.452 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163754.log 2023-10-03T10:24:05.452 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.163754.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.163754.log.gz 2023-10-03T10:24:05.452 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143394.log 2023-10-03T10:24:05.452 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.143394.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.143394.log.gz 2023-10-03T10:24:05.452 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168087.log 2023-10-03T10:24:05.452 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.168087.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.168087.log.gz 2023-10-03T10:24:05.452 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175472.log 2023-10-03T10:24:05.453 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.175472.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.175472.log.gz 2023-10-03T10:24:05.453 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190387.log 2023-10-03T10:24:05.453 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.190387.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.190387.log.gz 2023-10-03T10:24:05.453 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165027.log 2023-10-03T10:24:05.453 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.165027.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.165027.log.gz 2023-10-03T10:24:05.453 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158650.log 2023-10-03T10:24:05.453 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.158650.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.158650.log.gz 2023-10-03T10:24:05.453 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151905.log 2023-10-03T10:24:05.453 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.151905.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.151905.log.gz 2023-10-03T10:24:05.454 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147248.log 2023-10-03T10:24:05.454 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.147248.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.147248.log.gz 2023-10-03T10:24:05.454 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187857.log 2023-10-03T10:24:05.454 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.187857.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.187857.log.gz 2023-10-03T10:24:05.454 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125236.log 2023-10-03T10:24:05.454 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141115.log 2023-10-03T10:24:05.454 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.125236.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.125236.log.gz 2023-10-03T10:24:05.455 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.141115.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.141115.log.gz 2023-10-03T10:24:05.455 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139570.log 2023-10-03T10:24:05.455 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.139570.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.139570.log.gz 2023-10-03T10:24:05.455 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183346.log 2023-10-03T10:24:05.455 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.183346.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.183346.log.gz 2023-10-03T10:24:05.455 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147836.log 2023-10-03T10:24:05.455 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.147836.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.147836.log.gz 2023-10-03T10:24:05.455 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181847.log 2023-10-03T10:24:05.455 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.181847.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.181847.log.gz 2023-10-03T10:24:05.456 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165080.log 2023-10-03T10:24:05.456 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.165080.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.165080.log.gz 2023-10-03T10:24:05.456 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182786.log 2023-10-03T10:24:05.456 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139866.log 2023-10-03T10:24:05.456 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.182786.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.182786.log.gz 2023-10-03T10:24:05.456 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.139866.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.139866.log.gz 2023-10-03T10:24:05.456 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174835.log 2023-10-03T10:24:05.456 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.174835.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.174835.log.gz 2023-10-03T10:24:05.457 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184914.log 2023-10-03T10:24:05.457 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.184914.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.184914.log.gz 2023-10-03T10:24:05.457 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143441.log 2023-10-03T10:24:05.457 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.143441.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.143441.log.gz 2023-10-03T10:24:05.457 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148424.log 2023-10-03T10:24:05.457 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.148424.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.148424.log.gz 2023-10-03T10:24:05.457 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191858.log 2023-10-03T10:24:05.457 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.191858.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.191858.log.gz 2023-10-03T10:24:05.458 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144225.log 2023-10-03T10:24:05.458 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.144225.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.144225.log.gz 2023-10-03T10:24:05.458 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168479.log 2023-10-03T10:24:05.458 INFO:teuthology.orchestra.run.smithi003.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-03T10:24:05.458 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.168479.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.168479.log.gz 2023-10-03T10:24:05.458 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135595.log 2023-10-03T10:24:05.458 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171419.log 2023-10-03T10:24:05.458 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.135595.log: /var/log/ceph/ceph-client.admin.171419.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.135595.log.gz 2023-10-03T10:24:05.459 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.171419.log.gz 2023-10-03T10:24:05.459 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184156.log 2023-10-03T10:24:05.459 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138494.log 2023-10-03T10:24:05.459 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.184156.log: /var/log/ceph/ceph-client.admin.138494.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.184156.log.gz 2023-10-03T10:24:05.459 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.138494.log.gz 2023-10-03T10:24:05.459 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162354.log 2023-10-03T10:24:05.459 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193208.log 2023-10-03T10:24:05.459 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.162354.log: /var/log/ceph/ceph-client.admin.193208.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.162354.log.gz 2023-10-03T10:24:05.459 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.193208.log.gz 2023-10-03T10:24:05.460 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174322.log 2023-10-03T10:24:05.460 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-03T10:24:05.460 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.174322.log: /var/log/ceph/ceph.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.174322.log.gz 2023-10-03T10:24:05.460 INFO:teuthology.orchestra.run.smithi003.stderr: 94.2% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-03T10:24:05.460 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152762.log 2023-10-03T10:24:05.460 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122503.log 2023-10-03T10:24:05.460 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.152762.log: /var/log/ceph/ceph-client.admin.122503.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.152762.log.gz 2023-10-03T10:24:05.461 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.122503.log.gz 2023-10-03T10:24:05.461 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175619.log 2023-10-03T10:24:05.461 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.175619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178242.log 2023-10-03T10:24:05.461 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.175619.log.gz 2023-10-03T10:24:05.461 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139892.log 2023-10-03T10:24:05.461 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.178242.log: /var/log/ceph/ceph-client.admin.139892.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.178242.log.gz 2023-10-03T10:24:05.461 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.139892.log.gz 2023-10-03T10:24:05.461 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176478.log 2023-10-03T10:24:05.462 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178412.log 2023-10-03T10:24:05.462 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.176478.log: /var/log/ceph/ceph-client.admin.178412.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.176478.log.gz 2023-10-03T10:24:05.462 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.178412.log.gz 2023-10-03T10:24:05.462 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190836.log 2023-10-03T10:24:05.462 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178856.log 2023-10-03T10:24:05.462 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.190836.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.190836.log.gz 2023-10-03T10:24:05.462 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.178856.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.178856.log.gz 2023-10-03T10:24:05.463 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146018.log 2023-10-03T10:24:05.463 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190878.log 2023-10-03T10:24:05.463 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.146018.log: /var/log/ceph/ceph-client.admin.190878.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.146018.log.gz 2023-10-03T10:24:05.463 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.190878.log.gz 2023-10-03T10:24:05.463 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160392.log 2023-10-03T10:24:05.463 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187044.log 2023-10-03T10:24:05.463 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.160392.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.160392.log.gz 2023-10-03T10:24:05.463 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.187044.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.187044.log.gz 2023-10-03T10:24:05.463 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194529.log 2023-10-03T10:24:05.464 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146530.log 2023-10-03T10:24:05.464 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.194529.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.194529.log.gz 2023-10-03T10:24:05.464 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.146530.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.146530.log.gz 2023-10-03T10:24:05.464 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135819.log 2023-10-03T10:24:05.464 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189629.log 2023-10-03T10:24:05.464 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.135819.log: /var/log/ceph/ceph-client.admin.189629.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.135819.log.gz 2023-10-03T10:24:05.464 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.189629.log.gz 2023-10-03T10:24:05.464 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144673.log 2023-10-03T10:24:05.465 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135425.log 2023-10-03T10:24:05.465 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.144673.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.144673.log.gz 2023-10-03T10:24:05.465 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.135425.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.135425.log.gz 2023-10-03T10:24:05.465 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143271.log 2023-10-03T10:24:05.465 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161517.log 2023-10-03T10:24:05.465 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.143271.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.143271.log.gz/var/log/ceph/ceph-client.admin.161517.log: 2023-10-03T10:24:05.465 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.161517.log.gz 2023-10-03T10:24:05.465 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191955.log 2023-10-03T10:24:05.466 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155266.log 2023-10-03T10:24:05.466 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.191955.log: /var/log/ceph/ceph-client.admin.155266.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.191955.log.gz 2023-10-03T10:24:05.466 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140896.log 2023-10-03T10:24:05.466 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.155266.log.gz 2023-10-03T10:24:05.466 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184130.log 2023-10-03T10:24:05.466 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.140896.log: /var/log/ceph/ceph-client.admin.184130.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.140896.log.gz 2023-10-03T10:24:05.466 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.184130.log.gz 2023-10-03T10:24:05.466 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145550.log 2023-10-03T10:24:05.467 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129911.log 2023-10-03T10:24:05.467 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.145550.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.145550.log.gz 2023-10-03T10:24:05.467 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.129911.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.129911.log.gz 2023-10-03T10:24:05.467 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177458.log 2023-10-03T10:24:05.467 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145746.log 2023-10-03T10:24:05.467 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.177458.log: /var/log/ceph/ceph-client.admin.145746.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.177458.log.gz 2023-10-03T10:24:05.467 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.145746.log.gz 2023-10-03T10:24:05.467 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186116.log 2023-10-03T10:24:05.478 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181582.log 2023-10-03T10:24:05.478 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.186116.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.186116.log.gz 2023-10-03T10:24:05.479 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.181582.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.181582.log.gz 2023-10-03T10:24:05.479 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141895.log 2023-10-03T10:24:05.479 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149749.log 2023-10-03T10:24:05.479 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.141895.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.141895.log.gz 2023-10-03T10:24:05.479 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.149749.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.149749.log.gz 2023-10-03T10:24:05.479 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124742.log 2023-10-03T10:24:05.479 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140749.log 2023-10-03T10:24:05.480 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.124742.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.124742.log.gz 2023-10-03T10:24:05.480 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.140749.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.140749.log.gz 2023-10-03T10:24:05.480 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122719.log 2023-10-03T10:24:05.480 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143494.log 2023-10-03T10:24:05.480 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.122719.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.122719.log.gz 2023-10-03T10:24:05.480 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.143494.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.143494.log.gz 2023-10-03T10:24:05.480 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190779.log 2023-10-03T10:24:05.480 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140554.log 2023-10-03T10:24:05.481 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.190779.log: /var/log/ceph/ceph-client.admin.140554.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.190779.log.gz 2023-10-03T10:24:05.481 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.140554.log.gz 2023-10-03T10:24:05.481 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164146.log 2023-10-03T10:24:05.481 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142657.log 2023-10-03T10:24:05.481 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.164146.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.164146.log.gz 2023-10-03T10:24:05.481 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.142657.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.142657.log.gz 2023-10-03T10:24:05.481 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173267.log 2023-10-03T10:24:05.481 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138690.log 2023-10-03T10:24:05.482 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.173267.log: /var/log/ceph/ceph-client.admin.138690.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.173267.log.gz 2023-10-03T10:24:05.482 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.138690.log.gz 2023-10-03T10:24:05.482 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183934.log 2023-10-03T10:24:05.482 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155611.log 2023-10-03T10:24:05.482 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.183934.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.183934.log.gz 2023-10-03T10:24:05.482 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.155611.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.155611.log.gz 2023-10-03T10:24:05.482 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150311.log 2023-10-03T10:24:05.483 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152567.log 2023-10-03T10:24:05.483 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.150311.log: /var/log/ceph/ceph-client.admin.152567.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.150311.log.gz 2023-10-03T10:24:05.483 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.152567.log.gz 2023-10-03T10:24:05.483 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182170.log 2023-10-03T10:24:05.483 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179929.log 2023-10-03T10:24:05.483 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.182170.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.182170.log.gz/var/log/ceph/ceph-client.admin.179929.log: 2023-10-03T10:24:05.483 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.179929.log.gz 2023-10-03T10:24:05.483 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194725.log 2023-10-03T10:24:05.484 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170980.log 2023-10-03T10:24:05.484 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.194725.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.194725.log.gz 2023-10-03T10:24:05.484 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.170980.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.170980.log.gz 2023-10-03T10:24:05.484 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176844.log 2023-10-03T10:24:05.484 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125658.log 2023-10-03T10:24:05.484 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.176844.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.176844.log.gz 2023-10-03T10:24:05.485 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.125658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143590.log 2023-10-03T10:24:05.485 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.125658.log.gz 2023-10-03T10:24:05.485 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181778.log 2023-10-03T10:24:05.485 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.143590.log: /var/log/ceph/ceph-client.admin.181778.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.143590.log.gz 2023-10-03T10:24:05.485 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.181778.log.gz 2023-10-03T10:24:05.485 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164708.log 2023-10-03T10:24:05.485 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177654.log 2023-10-03T10:24:05.485 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.164708.log: 88.4%/var/log/ceph/ceph-client.admin.177654.log: -- replaced with /var/log/ceph/ceph-client.admin.164708.log.gz 2023-10-03T10:24:05.486 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.177654.log.gz 2023-10-03T10:24:05.486 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163877.log 2023-10-03T10:24:05.486 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186900.log 2023-10-03T10:24:05.486 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.163877.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.163877.log.gz 2023-10-03T10:24:05.486 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.186900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143859.log 2023-10-03T10:24:05.486 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.186900.log.gz 2023-10-03T10:24:05.486 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122380.log 2023-10-03T10:24:05.487 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.143859.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.143859.log.gz 2023-10-03T10:24:05.487 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167096.log 2023-10-03T10:24:05.487 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.122380.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.122380.log.gz 2023-10-03T10:24:05.487 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128164.log 2023-10-03T10:24:05.487 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.167096.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.167096.log.gz 2023-10-03T10:24:05.487 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156787.log 2023-10-03T10:24:05.487 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.128164.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.128164.log.gz 2023-10-03T10:24:05.487 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.156787.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.156787.log.gz 2023-10-03T10:24:05.488 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136970.log 2023-10-03T10:24:05.488 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160001.log 2023-10-03T10:24:05.488 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.136970.log: /var/log/ceph/ceph-client.admin.160001.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.136970.log.gz 2023-10-03T10:24:05.488 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.160001.log.gz 2023-10-03T10:24:05.488 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159923.log 2023-10-03T10:24:05.488 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128138.log 2023-10-03T10:24:05.488 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.159923.log: /var/log/ceph/ceph-client.admin.128138.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.159923.log.gz 2023-10-03T10:24:05.488 INFO:teuthology.orchestra.run.smithi003.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.128138.log.gz 2023-10-03T10:24:05.489 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139378.log 2023-10-03T10:24:05.489 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154892.log 2023-10-03T10:24:05.489 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.139378.log: /var/log/ceph/ceph-client.admin.154892.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.139378.log.gz 2023-10-03T10:24:05.489 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.154892.log.gz 2023-10-03T10:24:05.489 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170123.log 2023-10-03T10:24:05.489 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157233.log 2023-10-03T10:24:05.489 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.170123.log: /var/log/ceph/ceph-client.admin.157233.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.170123.log.gz 2023-10-03T10:24:05.489 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.157233.log.gz 2023-10-03T10:24:05.490 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192987.log 2023-10-03T10:24:05.490 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144868.log 2023-10-03T10:24:05.490 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.192987.log: /var/log/ceph/ceph-client.admin.144868.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.192987.log.gz 2023-10-03T10:24:05.490 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.144868.log.gz 2023-10-03T10:24:05.490 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155415.log 2023-10-03T10:24:05.490 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152323.log 2023-10-03T10:24:05.490 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.155415.log: /var/log/ceph/ceph-client.admin.152323.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.155415.log.gz 2023-10-03T10:24:05.490 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.152323.log.gz 2023-10-03T10:24:05.491 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160511.log 2023-10-03T10:24:05.491 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162549.log 2023-10-03T10:24:05.491 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.160511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137365.log 2023-10-03T10:24:05.491 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.160511.log.gz 2023-10-03T10:24:05.491 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.162549.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.162549.log.gz 2023-10-03T10:24:05.491 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.137365.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.137365.log.gz 2023-10-03T10:24:05.491 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187439.log 2023-10-03T10:24:05.495 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185476.log 2023-10-03T10:24:05.495 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.187439.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.187439.log.gz 2023-10-03T10:24:05.495 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.185476.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.185476.log.gz 2023-10-03T10:24:05.496 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158455.log 2023-10-03T10:24:05.496 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178072.log 2023-10-03T10:24:05.496 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.158455.log: /var/log/ceph/ceph-client.admin.178072.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.158455.log.gz 2023-10-03T10:24:05.496 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.178072.log.gz 2023-10-03T10:24:05.496 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181088.log 2023-10-03T10:24:05.496 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159531.log 2023-10-03T10:24:05.496 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.181088.log: 87.8%/var/log/ceph/ceph-client.admin.159531.log: -- replaced with /var/log/ceph/ceph-client.admin.181088.log.gz 2023-10-03T10:24:05.496 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.159531.log.gz 2023-10-03T10:24:05.497 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157495.log 2023-10-03T10:24:05.497 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195013.log 2023-10-03T10:24:05.497 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.157495.log: 88.1%/var/log/ceph/ceph-client.admin.195013.log: -- replaced with /var/log/ceph/ceph-client.admin.157495.log.gz 2023-10-03T10:24:05.497 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.195013.log.gz 2023-10-03T10:24:05.497 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161883.log 2023-10-03T10:24:05.497 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170615.log 2023-10-03T10:24:05.497 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.161883.log: /var/log/ceph/ceph-client.admin.170615.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.161883.log.gz 2023-10-03T10:24:05.498 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.170615.log.gz 2023-10-03T10:24:05.498 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125187.log 2023-10-03T10:24:05.498 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149870.log 2023-10-03T10:24:05.498 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.125187.log: /var/log/ceph/ceph-client.admin.149870.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.125187.log.gz 2023-10-03T10:24:05.498 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.149870.log.gz 2023-10-03T10:24:05.498 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156199.log 2023-10-03T10:24:05.498 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139500.log 2023-10-03T10:24:05.498 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.156199.log: /var/log/ceph/ceph-client.admin.139500.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.156199.log.gz 2023-10-03T10:24:05.499 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.139500.log.gz 2023-10-03T10:24:05.499 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174126.log 2023-10-03T10:24:05.499 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167183.log 2023-10-03T10:24:05.499 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.174126.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.174126.log.gz 2023-10-03T10:24:05.499 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.167183.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.167183.log.gz 2023-10-03T10:24:05.499 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129186.log 2023-10-03T10:24:05.499 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193523.log 2023-10-03T10:24:05.499 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.129186.log: /var/log/ceph/ceph-client.admin.193523.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.129186.log.gz 2023-10-03T10:24:05.500 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.193523.log.gz 2023-10-03T10:24:05.500 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173855.log 2023-10-03T10:24:05.500 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137114.log 2023-10-03T10:24:05.500 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.173855.log: /var/log/ceph/ceph-client.admin.137114.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.173855.log.gz 2023-10-03T10:24:05.500 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.137114.log.gz 2023-10-03T10:24:05.500 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136800.log 2023-10-03T10:24:05.500 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145626.log 2023-10-03T10:24:05.501 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.136800.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.136800.log.gz 2023-10-03T10:24:05.501 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.145626.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.145626.log.gz 2023-10-03T10:24:05.501 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180772.log 2023-10-03T10:24:05.501 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154866.log 2023-10-03T10:24:05.501 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.180772.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.180772.log.gz 2023-10-03T10:24:05.501 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.154866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182366.log 2023-10-03T10:24:05.501 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.154866.log.gz 2023-10-03T10:24:05.501 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169778.log 2023-10-03T10:24:05.504 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.182366.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.182366.log.gz 2023-10-03T10:24:05.504 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177383.log 2023-10-03T10:24:05.505 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.169778.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.169778.log.gz 2023-10-03T10:24:05.505 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.177383.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.177383.log.gz 2023-10-03T10:24:05.505 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182732.log 2023-10-03T10:24:05.505 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173389.log 2023-10-03T10:24:05.505 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.182732.log: /var/log/ceph/ceph-client.admin.173389.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.182732.log.gz 2023-10-03T10:24:05.505 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.173389.log.gz 2023-10-03T10:24:05.505 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194915.log 2023-10-03T10:24:05.505 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172287.log 2023-10-03T10:24:05.506 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.194915.log: /var/log/ceph/ceph-client.admin.172287.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.194915.log.gz 2023-10-03T10:24:05.506 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.172287.log.gz 2023-10-03T10:24:05.506 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178559.log 2023-10-03T10:24:05.506 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158239.log 2023-10-03T10:24:05.506 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.178559.log: /var/log/ceph/ceph-client.admin.158239.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.178559.log.gz 2023-10-03T10:24:05.506 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.158239.log.gz 2023-10-03T10:24:05.506 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172166.log 2023-10-03T10:24:05.507 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166472.log 2023-10-03T10:24:05.507 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.172166.log: 88.1%/var/log/ceph/ceph-client.admin.166472.log: -- replaced with /var/log/ceph/ceph-client.admin.172166.log.gz 2023-10-03T10:24:05.507 INFO:teuthology.orchestra.run.smithi003.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.166472.log.gz 2023-10-03T10:24:05.507 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177484.log 2023-10-03T10:24:05.507 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168826.log 2023-10-03T10:24:05.507 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.177484.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.177484.log.gz 2023-10-03T10:24:05.507 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.168826.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.168826.log.gz 2023-10-03T10:24:05.508 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190975.log 2023-10-03T10:24:05.509 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124452.log 2023-10-03T10:24:05.510 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.190975.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.190975.log.gz/var/log/ceph/ceph-client.admin.124452.log: 2023-10-03T10:24:05.510 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.124452.log.gz 2023-10-03T10:24:05.510 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185306.log 2023-10-03T10:24:05.510 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160785.log 2023-10-03T10:24:05.510 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.185306.log: /var/log/ceph/ceph-client.admin.160785.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.185306.log.gz 2023-10-03T10:24:05.510 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148965.log 2023-10-03T10:24:05.510 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.160785.log.gz 2023-10-03T10:24:05.511 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185919.log 2023-10-03T10:24:05.511 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.148965.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.148965.log.gz 2023-10-03T10:24:05.511 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158969.log 2023-10-03T10:24:05.511 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.185919.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.185919.log.gz 2023-10-03T10:24:05.511 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182461.log 2023-10-03T10:24:05.513 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.158969.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.158969.log.gz 2023-10-03T10:24:05.513 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169804.log 2023-10-03T10:24:05.515 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.182461.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.182461.log.gz 2023-10-03T10:24:05.515 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123074.log 2023-10-03T10:24:05.515 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.169804.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.169804.log.gz 2023-10-03T10:24:05.515 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190682.log 2023-10-03T10:24:05.516 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.123074.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.123074.log.gz 2023-10-03T10:24:05.516 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168638.log 2023-10-03T10:24:05.516 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.190682.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.190682.log.gz 2023-10-03T10:24:05.516 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.168638.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.168638.log.gz 2023-10-03T10:24:05.516 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169412.log 2023-10-03T10:24:05.516 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147985.log 2023-10-03T10:24:05.516 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.169412.log: /var/log/ceph/ceph-client.admin.147985.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.169412.log.gz 2023-10-03T10:24:05.517 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171738.log 2023-10-03T10:24:05.517 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.147985.log.gz 2023-10-03T10:24:05.517 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140650.log 2023-10-03T10:24:05.519 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.171738.log: /var/log/ceph/ceph-client.admin.140650.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.171738.log.gz 2023-10-03T10:24:05.519 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.140650.log.gz 2023-10-03T10:24:05.519 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153102.log 2023-10-03T10:24:05.520 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175720.log 2023-10-03T10:24:05.520 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.153102.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.153102.log.gz 2023-10-03T10:24:05.520 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.175720.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.175720.log.gz 2023-10-03T10:24:05.520 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137533.log 2023-10-03T10:24:05.520 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151343.log 2023-10-03T10:24:05.520 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.137533.log: /var/log/ceph/ceph-client.admin.151343.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.137533.log.gz 2023-10-03T10:24:05.520 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.151343.log.gz 2023-10-03T10:24:05.520 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165223.log 2023-10-03T10:24:05.521 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168210.log 2023-10-03T10:24:05.522 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.165223.log: /var/log/ceph/ceph-client.admin.168210.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.165223.log.gz 2023-10-03T10:24:05.522 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.168210.log.gz 2023-10-03T10:24:05.523 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164074.log 2023-10-03T10:24:05.523 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184496.log 2023-10-03T10:24:05.523 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.164074.log: 89.1%/var/log/ceph/ceph-client.admin.184496.log: -- replaced with /var/log/ceph/ceph-client.admin.164074.log.gz 2023-10-03T10:24:05.523 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.184496.log.gz 2023-10-03T10:24:05.523 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189262.log 2023-10-03T10:24:05.523 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181042.log 2023-10-03T10:24:05.523 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.189262.log: /var/log/ceph/ceph-client.admin.181042.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.189262.log.gz 2023-10-03T10:24:05.523 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.181042.log.gz 2023-10-03T10:24:05.524 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156076.log 2023-10-03T10:24:05.525 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175276.log 2023-10-03T10:24:05.525 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.156076.log: 87.9%/var/log/ceph/ceph-client.admin.175276.log: -- replaced with /var/log/ceph/ceph-client.admin.156076.log.gz 2023-10-03T10:24:05.525 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.175276.log.gz 2023-10-03T10:24:05.525 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125609.log 2023-10-03T10:24:05.525 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163459.log 2023-10-03T10:24:05.526 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.125609.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.125609.log.gz 2023-10-03T10:24:05.526 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.163459.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.163459.log.gz 2023-10-03T10:24:05.526 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164342.log 2023-10-03T10:24:05.526 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148694.log 2023-10-03T10:24:05.527 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.164342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175080.log 2023-10-03T10:24:05.527 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.164342.log.gz 2023-10-03T10:24:05.527 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.148694.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.148694.log.gz 2023-10-03T10:24:05.527 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.175080.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.175080.log.gz 2023-10-03T10:24:05.528 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146334.log 2023-10-03T10:24:05.528 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156840.log 2023-10-03T10:24:05.529 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.146334.log: /var/log/ceph/ceph-client.admin.156840.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.146334.log.gz 2023-10-03T10:24:05.529 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.156840.log.gz 2023-10-03T10:24:05.529 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163255.log 2023-10-03T10:24:05.529 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147789.log 2023-10-03T10:24:05.530 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.163255.log: /var/log/ceph/ceph-client.admin.147789.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.163255.log.gz 2023-10-03T10:24:05.530 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192817.log 2023-10-03T10:24:05.530 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.147789.log.gz 2023-10-03T10:24:05.530 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187492.log 2023-10-03T10:24:05.531 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.192817.log: /var/log/ceph/ceph-client.admin.187492.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.192817.log.gz 2023-10-03T10:24:05.531 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.187492.log.gz 2023-10-03T10:24:05.532 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146260.log 2023-10-03T10:24:05.532 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154427.log 2023-10-03T10:24:05.533 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.146260.log: /var/log/ceph/ceph-client.admin.154427.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.146260.log.gz 2023-10-03T10:24:05.533 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.154427.log.gz 2023-10-03T10:24:05.533 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170711.log 2023-10-03T10:24:05.533 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140945.log 2023-10-03T10:24:05.533 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.170711.log: /var/log/ceph/ceph-client.admin.140945.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.170711.log.gz 2023-10-03T10:24:05.534 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.140945.log.gz 2023-10-03T10:24:05.534 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147126.log 2023-10-03T10:24:05.535 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180317.log 2023-10-03T10:24:05.535 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.147126.log: /var/log/ceph/ceph-client.admin.180317.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.147126.log.gz 2023-10-03T10:24:05.536 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.180317.log.gz 2023-10-03T10:24:05.536 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171372.log 2023-10-03T10:24:05.536 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171346.log 2023-10-03T10:24:05.536 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.171372.log: /var/log/ceph/ceph-client.admin.171346.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.171372.log.gz 2023-10-03T10:24:05.536 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.171346.log.gz 2023-10-03T10:24:05.537 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118233.log 2023-10-03T10:24:05.537 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162105.log 2023-10-03T10:24:05.538 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.118233.log: /var/log/ceph/ceph-client.admin.162105.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.118233.log.gz 2023-10-03T10:24:05.538 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.162105.log.gz 2023-10-03T10:24:05.538 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159164.log 2023-10-03T10:24:05.539 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185502.log 2023-10-03T10:24:05.539 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.159164.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.159164.log.gz 2023-10-03T10:24:05.539 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.185502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157078.log 2023-10-03T10:24:05.539 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.185502.log.gz 2023-10-03T10:24:05.540 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181284.log 2023-10-03T10:24:05.540 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.157078.log: 87.8%/var/log/ceph/ceph-client.admin.181284.log: -- replaced with /var/log/ceph/ceph-client.admin.157078.log.gz 2023-10-03T10:24:05.540 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.181284.log.gz 2023-10-03T10:24:05.540 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166107.log 2023-10-03T10:24:05.541 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144374.log 2023-10-03T10:24:05.541 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.166107.log: /var/log/ceph/ceph-client.admin.144374.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.166107.log.gz 2023-10-03T10:24:05.541 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.144374.log.gz 2023-10-03T10:24:05.541 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158381.log 2023-10-03T10:24:05.542 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162275.log 2023-10-03T10:24:05.543 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.158381.log: /var/log/ceph/ceph-client.admin.162275.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.158381.log.gz 2023-10-03T10:24:05.543 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.162275.log.gz 2023-10-03T10:24:05.543 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177628.log 2023-10-03T10:24:05.543 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167256.log 2023-10-03T10:24:05.544 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.177628.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.177628.log.gz 2023-10-03T10:24:05.544 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.167256.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.167256.log.gz 2023-10-03T10:24:05.544 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177040.log 2023-10-03T10:24:05.545 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118834.log 2023-10-03T10:24:05.545 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.177040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150066.log 2023-10-03T10:24:05.545 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.177040.log.gz 2023-10-03T10:24:05.546 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.118834.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.118834.log.gz 2023-10-03T10:24:05.546 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164439.log 2023-10-03T10:24:05.547 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.150066.log: /var/log/ceph/ceph-client.admin.164439.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.150066.log.gz 2023-10-03T10:24:05.547 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.164439.log.gz 2023-10-03T10:24:05.547 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195307.log 2023-10-03T10:24:05.548 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135818.log 2023-10-03T10:24:05.548 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.195307.log: /var/log/ceph/ceph-client.admin.135818.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.195307.log.gz 2023-10-03T10:24:05.549 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.135818.log.gz 2023-10-03T10:24:05.549 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122621.log 2023-10-03T10:24:05.549 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154474.log 2023-10-03T10:24:05.550 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.122621.log: /var/log/ceph/ceph-client.admin.154474.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.122621.log.gz 2023-10-03T10:24:05.550 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.154474.log.gz 2023-10-03T10:24:05.550 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124644.log 2023-10-03T10:24:05.551 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172950.log 2023-10-03T10:24:05.551 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.124644.log: /var/log/ceph/ceph-client.admin.172950.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.124644.log.gz 2023-10-03T10:24:05.551 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.172950.log.gz 2023-10-03T10:24:05.551 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155711.log 2023-10-03T10:24:05.552 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124791.log 2023-10-03T10:24:05.552 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.155711.log: /var/log/ceph/ceph-client.admin.124791.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.155711.log.gz 2023-10-03T10:24:05.553 INFO:teuthology.orchestra.run.smithi003.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.124791.log.gz 2023-10-03T10:24:05.553 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186507.log 2023-10-03T10:24:05.553 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153938.log 2023-10-03T10:24:05.554 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.186507.log: /var/log/ceph/ceph-client.admin.153938.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.186507.log.gz 2023-10-03T10:24:05.554 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.153938.log.gz 2023-10-03T10:24:05.554 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165419.log 2023-10-03T10:24:05.555 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158043.log 2023-10-03T10:24:05.555 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.165419.log: /var/log/ceph/ceph-client.admin.158043.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.165419.log.gz 2023-10-03T10:24:05.556 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.158043.log.gz 2023-10-03T10:24:05.556 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195258.log 2023-10-03T10:24:05.556 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125903.log 2023-10-03T10:24:05.557 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.195258.log: /var/log/ceph/ceph-client.admin.125903.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.195258.log.gz 2023-10-03T10:24:05.557 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.125903.log.gz 2023-10-03T10:24:05.557 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168163.log 2023-10-03T10:24:05.558 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183150.log 2023-10-03T10:24:05.559 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.168163.log: /var/log/ceph/ceph-client.admin.183150.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.168163.log.gz 2023-10-03T10:24:05.559 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.183150.log.gz 2023-10-03T10:24:05.559 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139082.log 2023-10-03T10:24:05.559 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182144.log 2023-10-03T10:24:05.560 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.139082.log: /var/log/ceph/ceph-client.admin.182144.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.139082.log.gz 2023-10-03T10:24:05.560 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.182144.log.gz 2023-10-03T10:24:05.560 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162548.log 2023-10-03T10:24:05.560 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136604.log 2023-10-03T10:24:05.561 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.162548.log: /var/log/ceph/ceph-client.admin.136604.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.162548.log.gz 2023-10-03T10:24:05.561 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.136604.log.gz 2023-10-03T10:24:05.562 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193993.log 2023-10-03T10:24:05.563 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172530.log 2023-10-03T10:24:05.563 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.193993.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.193993.log.gz 2023-10-03T10:24:05.563 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.172530.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.172530.log.gz 2023-10-03T10:24:05.563 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191811.log 2023-10-03T10:24:05.564 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149135.log 2023-10-03T10:24:05.564 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.191811.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.191811.log.gz 2023-10-03T10:24:05.564 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.149135.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.149135.log.gz 2023-10-03T10:24:05.565 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150925.log 2023-10-03T10:24:05.566 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156886.log 2023-10-03T10:24:05.566 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.150925.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.150925.log.gz 2023-10-03T10:24:05.566 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.156886.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.156886.log.gz 2023-10-03T10:24:05.566 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128744.log 2023-10-03T10:24:05.567 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178951.log 2023-10-03T10:24:05.567 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.128744.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.128744.log.gz 2023-10-03T10:24:05.567 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.178951.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.178951.log.gz 2023-10-03T10:24:05.567 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144251.log 2023-10-03T10:24:05.568 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155462.log 2023-10-03T10:24:05.568 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.144251.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.144251.log.gz 2023-10-03T10:24:05.569 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.155462.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.155462.log.gz 2023-10-03T10:24:05.569 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173904.log 2023-10-03T10:24:05.569 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159582.log 2023-10-03T10:24:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.173904.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.173904.log.gz 2023-10-03T10:24:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.159582.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.159582.log.gz 2023-10-03T10:24:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183372.log 2023-10-03T10:24:05.571 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178464.log 2023-10-03T10:24:05.571 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.183372.log: /var/log/ceph/ceph-client.admin.178464.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.183372.log.gz 2023-10-03T10:24:05.572 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.178464.log.gz 2023-10-03T10:24:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158355.log 2023-10-03T10:24:05.573 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174884.log 2023-10-03T10:24:05.573 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.158355.log: /var/log/ceph/ceph-client.admin.174884.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.158355.log.gz 2023-10-03T10:24:05.573 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.174884.log.gz 2023-10-03T10:24:05.573 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138520.log 2023-10-03T10:24:05.574 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165642.log 2023-10-03T10:24:05.574 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.138520.log: /var/log/ceph/ceph-client.admin.165642.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.138520.log.gz 2023-10-03T10:24:05.575 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.165642.log.gz 2023-10-03T10:24:05.575 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142318.log 2023-10-03T10:24:05.575 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154526.log 2023-10-03T10:24:05.576 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.142318.log: /var/log/ceph/ceph-client.admin.154526.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.142318.log.gz 2023-10-03T10:24:05.576 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.154526.log.gz 2023-10-03T10:24:05.576 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188871.log 2023-10-03T10:24:05.577 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158063.log 2023-10-03T10:24:05.577 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.188871.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.188871.log.gz 2023-10-03T10:24:05.577 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.158063.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.158063.log.gz 2023-10-03T10:24:05.577 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-03T10:24:05.581 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168947.log 2023-10-03T10:24:05.583 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.168947.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.168947.log.gz 2023-10-03T10:24:05.583 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164941.log 2023-10-03T10:24:05.584 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.164941.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.164941.log.gz 2023-10-03T10:24:05.584 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170418.log 2023-10-03T10:24:05.585 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.170418.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.170418.log.gz 2023-10-03T10:24:05.585 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194307.log 2023-10-03T10:24:05.586 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.194307.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.194307.log.gz 2023-10-03T10:24:05.586 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140258.log 2023-10-03T10:24:05.587 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.140258.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.140258.log.gz 2023-10-03T10:24:05.587 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144178.log 2023-10-03T10:24:05.588 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.144178.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.144178.log.gz 2023-10-03T10:24:05.588 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165714.log 2023-10-03T10:24:05.589 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.165714.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.165714.log.gz 2023-10-03T10:24:05.590 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195062.log 2023-10-03T10:24:05.591 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.195062.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.195062.log.gz 2023-10-03T10:24:05.591 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188249.log 2023-10-03T10:24:05.592 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.188249.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.188249.log.gz 2023-10-03T10:24:05.592 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144570.log 2023-10-03T10:24:05.593 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.144570.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.144570.log.gz 2023-10-03T10:24:05.593 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152216.log 2023-10-03T10:24:05.594 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.152216.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.152216.log.gz 2023-10-03T10:24:05.594 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118681.log 2023-10-03T10:24:05.595 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.118681.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.118681.log.gz 2023-10-03T10:24:05.596 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160610.log 2023-10-03T10:24:05.596 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.160610.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.160610.log.gz 2023-10-03T10:24:05.597 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182536.log 2023-10-03T10:24:05.598 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.182536.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.182536.log.gz 2023-10-03T10:24:05.598 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145233.log 2023-10-03T10:24:05.599 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.145233.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.145233.log.gz 2023-10-03T10:24:05.599 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142218.log 2023-10-03T10:24:05.600 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.142218.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.142218.log.gz 2023-10-03T10:24:05.600 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167818.log 2023-10-03T10:24:05.601 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.167818.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.167818.log.gz 2023-10-03T10:24:05.601 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162693.log 2023-10-03T10:24:05.602 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.162693.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.162693.log.gz 2023-10-03T10:24:05.602 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158846.log 2023-10-03T10:24:05.603 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.158846.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.158846.log.gz 2023-10-03T10:24:05.604 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122429.log 2023-10-03T10:24:05.604 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128989.log 2023-10-03T10:24:05.605 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.122429.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.122429.log.gz 2023-10-03T10:24:05.605 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.128989.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.128989.log.gz 2023-10-03T10:24:05.605 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127859.log 2023-10-03T10:24:05.606 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.127859.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.127859.log.gz 2023-10-03T10:24:05.606 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123886.log 2023-10-03T10:24:05.607 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.123886.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.123886.log.gz 2023-10-03T10:24:05.607 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190583.log 2023-10-03T10:24:05.608 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.190583.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.190583.log.gz 2023-10-03T10:24:05.609 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141293.log 2023-10-03T10:24:05.610 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.141293.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.141293.log.gz 2023-10-03T10:24:05.610 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152467.log 2023-10-03T10:24:05.611 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.152467.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.152467.log.gz 2023-10-03T10:24:05.611 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171942.log 2023-10-03T10:24:05.612 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.171942.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.171942.log.gz 2023-10-03T10:24:05.612 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161713.log 2023-10-03T10:24:05.613 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.161713.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.161713.log.gz 2023-10-03T10:24:05.613 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146607.log 2023-10-03T10:24:05.614 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.146607.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.146607.log.gz 2023-10-03T10:24:05.614 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127701.log 2023-10-03T10:24:05.615 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.127701.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.127701.log.gz 2023-10-03T10:24:05.616 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124229.log 2023-10-03T10:24:05.617 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.124229.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.124229.log.gz 2023-10-03T10:24:05.617 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149478.log 2023-10-03T10:24:05.618 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.149478.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.149478.log.gz 2023-10-03T10:24:05.618 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145429.log 2023-10-03T10:24:05.619 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.145429.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.145429.log.gz 2023-10-03T10:24:05.619 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166203.log 2023-10-03T10:24:05.620 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.166203.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.166203.log.gz 2023-10-03T10:24:05.620 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186652.log 2023-10-03T10:24:05.621 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.186652.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.186652.log.gz 2023-10-03T10:24:05.621 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182022.log 2023-10-03T10:24:05.622 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.182022.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.182022.log.gz 2023-10-03T10:24:05.623 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176256.log 2023-10-03T10:24:05.624 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.176256.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.176256.log.gz 2023-10-03T10:24:05.624 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141461.log 2023-10-03T10:24:05.625 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.141461.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.141461.log.gz 2023-10-03T10:24:05.625 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164904.log 2023-10-03T10:24:05.626 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.164904.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.164904.log.gz 2023-10-03T10:24:05.626 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190805.log 2023-10-03T10:24:05.627 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.190805.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.190805.log.gz 2023-10-03T10:24:05.627 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157205.log 2023-10-03T10:24:05.628 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.157205.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.157205.log.gz 2023-10-03T10:24:05.628 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143049.log 2023-10-03T10:24:05.629 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157179.log 2023-10-03T10:24:05.630 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.143049.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.143049.log.gz 2023-10-03T10:24:05.630 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.157179.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.157179.log.gz 2023-10-03T10:24:05.630 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166864.log 2023-10-03T10:24:05.631 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.166864.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.166864.log.gz 2023-10-03T10:24:05.631 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153917.log 2023-10-03T10:24:05.632 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.153917.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.153917.log.gz 2023-10-03T10:24:05.632 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183712.log 2023-10-03T10:24:05.633 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.183712.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.183712.log.gz 2023-10-03T10:24:05.633 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123566.log 2023-10-03T10:24:05.634 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.123566.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.123566.log.gz 2023-10-03T10:24:05.635 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125334.log 2023-10-03T10:24:05.635 INFO:teuthology.orchestra.run.smithi003.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-03T10:24:05.636 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.125334.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.125334.log.gz 2023-10-03T10:24:05.636 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188675.log 2023-10-03T10:24:05.637 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162353.log 2023-10-03T10:24:05.637 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.188675.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.188675.log.gz 2023-10-03T10:24:05.637 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.162353.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.162353.log.gz 2023-10-03T10:24:05.637 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180228.log 2023-10-03T10:24:05.638 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161786.log 2023-10-03T10:24:05.639 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.180228.log: /var/log/ceph/ceph-client.admin.161786.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.180228.log.gz 2023-10-03T10:24:05.639 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.161786.log.gz 2023-10-03T10:24:05.639 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187661.log 2023-10-03T10:24:05.640 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155292.log 2023-10-03T10:24:05.640 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.187661.log: /var/log/ceph/ceph-client.admin.155292.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.187661.log.gz 2023-10-03T10:24:05.640 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.155292.log.gz 2023-10-03T10:24:05.640 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185332.log 2023-10-03T10:24:05.641 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180006.log 2023-10-03T10:24:05.641 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.185332.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.185332.log.gz 2023-10-03T10:24:05.642 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.180006.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.180006.log.gz 2023-10-03T10:24:05.642 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185920.log 2023-10-03T10:24:05.642 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182833.log 2023-10-03T10:24:05.643 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.185920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172924.log 2023-10-03T10:24:05.643 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.185920.log.gz 2023-10-03T10:24:05.643 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.182833.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.182833.log.gz 2023-10-03T10:24:05.644 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.172924.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.172924.log.gz 2023-10-03T10:24:05.645 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145037.log 2023-10-03T10:24:05.646 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.145037.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.145037.log.gz 2023-10-03T10:24:05.646 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124399.log 2023-10-03T10:24:05.647 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.124399.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124399.log.gz 2023-10-03T10:24:05.647 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174910.log 2023-10-03T10:24:05.649 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.174910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145476.log 2023-10-03T10:24:05.649 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.174910.log.gz 2023-10-03T10:24:05.649 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.145476.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.145476.log.gz 2023-10-03T10:24:05.650 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188496.log 2023-10-03T10:24:05.651 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.188496.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.188496.log.gz 2023-10-03T10:24:05.651 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182626.log 2023-10-03T10:24:05.654 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.182626.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.182626.log.gz 2023-10-03T10:24:05.654 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169439.log 2023-10-03T10:24:05.656 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181630.log 2023-10-03T10:24:05.656 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.169439.log: /var/log/ceph/ceph-client.admin.181630.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.169439.log.gz 2023-10-03T10:24:05.656 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.181630.log.gz 2023-10-03T10:24:05.656 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189211.log 2023-10-03T10:24:05.657 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145063.log 2023-10-03T10:24:05.657 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.189211.log: /var/log/ceph/ceph-client.admin.145063.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.189211.log.gz 2023-10-03T10:24:05.657 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.145063.log.gz 2023-10-03T10:24:05.657 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155015.log 2023-10-03T10:24:05.658 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.155015.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.155015.log.gz 2023-10-03T10:24:05.658 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192595.log 2023-10-03T10:24:05.659 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170196.log 2023-10-03T10:24:05.659 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.192595.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.192595.log.gz 2023-10-03T10:24:05.660 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.170196.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.170196.log.gz 2023-10-03T10:24:05.660 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177850.log 2023-10-03T10:24:05.661 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139182.log 2023-10-03T10:24:05.661 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.177850.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.177850.log.gz 2023-10-03T10:24:05.661 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.139182.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.139182.log.gz 2023-10-03T10:24:05.758 INFO:teuthology.orchestra.run.smithi003.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-10-03T10:24:05.796 INFO:teuthology.orchestra.run.smithi003.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-03T10:24:05.801 INFO:teuthology.orchestra.run.smithi003.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-10-03T10:24:05.956 INFO:teuthology.orchestra.run.smithi003.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-03T10:24:06.032 INFO:teuthology.orchestra.run.smithi003.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-03T10:24:06.034 INFO:teuthology.orchestra.run.smithi003.stderr: 2023-10-03T10:24:06.034 INFO:teuthology.orchestra.run.smithi003.stderr:real 0m1.146s 2023-10-03T10:24:06.034 INFO:teuthology.orchestra.run.smithi003.stderr:user 0m4.326s 2023-10-03T10:24:06.034 INFO:teuthology.orchestra.run.smithi003.stderr:sys 0m0.977s 2023-10-03T10:24:06.035 INFO:tasks.ceph:Archiving logs... 2023-10-03T10:24:06.035 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/log/ceph to /home/teuthworker/archive/dparmar-2023-10-03_09:48:43-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7409565/remote/smithi003/log 2023-10-03T10:24:06.036 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-03T10:24:07.833 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/log/ceph to /home/teuthworker/archive/dparmar-2023-10-03_09:48:43-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7409565/remote/smithi022/log 2023-10-03T10:24:07.834 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-03T10:24:07.870 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi003 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-10-03T10:24:07.880 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-03T10:24:07.897 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi003 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-10-03T10:24:07.901 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-03T10:24:07.902 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-03T10:24:07.907 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-03T10:24:08.029 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-03T10:24:08.041 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-03T10:24:08.041 DEBUG:teuthology.orchestra.run.smithi003:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-03T10:24:08.044 DEBUG:teuthology.orchestra.run.smithi022:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-03T10:24:08.063 INFO:teuthology.orchestra.run.smithi003.stderr:bash: ntpq: command not found 2023-10-03T10:24:08.065 INFO:teuthology.orchestra.run.smithi022.stderr:bash: ntpq: command not found 2023-10-03T10:24:08.068 INFO:teuthology.orchestra.run.smithi003.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-03T10:24:08.068 INFO:teuthology.orchestra.run.smithi003.stdout:=============================================================================== 2023-10-03T10:24:08.069 INFO:teuthology.orchestra.run.smithi003.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 87 +625us[ +623us] +/- 72ms 2023-10-03T10:24:08.069 INFO:teuthology.orchestra.run.smithi003.stdout:^* hv02.front.sepia.ceph.com 3 7 377 84 -183us[ -185us] +/- 59ms 2023-10-03T10:24:08.069 INFO:teuthology.orchestra.run.smithi003.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 85 -332us[ -333us] +/- 79ms 2023-10-03T10:24:08.069 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-03T10:24:08.071 INFO:teuthology.orchestra.run.smithi022.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-03T10:24:08.071 INFO:teuthology.orchestra.run.smithi022.stdout:=============================================================================== 2023-10-03T10:24:08.071 INFO:teuthology.orchestra.run.smithi022.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 19 +607us[ +609us] +/- 73ms 2023-10-03T10:24:08.071 INFO:teuthology.orchestra.run.smithi022.stdout:^* hv02.front.sepia.ceph.com 3 6 377 18 -208us[ -206us] +/- 60ms 2023-10-03T10:24:08.072 INFO:teuthology.orchestra.run.smithi022.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 -352us[ -351us] +/- 80ms 2023-10-03T10:24:08.072 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-03T10:24:08.072 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-03T10:24:08.083 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-03T10:24:08.083 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-03T10:24:08.094 DEBUG:teuthology.orchestra.run.smithi003:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-03T10:24:08.169 DEBUG:teuthology.orchestra.run.smithi022:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-03T10:24:08.274 DEBUG:teuthology.orchestra.run.smithi003:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-03T10:24:08.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-03T10:24:08.311 DEBUG:teuthology.orchestra.run.smithi022:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-03T10:24:08.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-03T10:24:08.348 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-03T10:24:08.360 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-03T10:24:08.371 INFO:teuthology.task.internal:Duration was 1555.426423 seconds 2023-10-03T10:24:08.372 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-03T10:24:08.383 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-03T10:24:08.383 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-03T10:24:08.386 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-03T10:24:08.443 INFO:teuthology.orchestra.run.smithi003.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-03T10:24:08.447 INFO:teuthology.orchestra.run.smithi022.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-03T10:24:08.877 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-03T10:24:08.877 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi003.front.sepia.ceph.com 2023-10-03T10:24:08.878 DEBUG:teuthology.orchestra.run.smithi003:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-03T10:24:08.907 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi022.front.sepia.ceph.com 2023-10-03T10:24:08.907 DEBUG:teuthology.orchestra.run.smithi022:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-03T10:24:08.937 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-03T10:24:08.937 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-03T10:24:08.949 DEBUG:teuthology.orchestra.run.smithi022:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-03T10:24:09.023 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-03T10:24:09.024 DEBUG:teuthology.orchestra.run.smithi003:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-03T10:24:09.026 DEBUG:teuthology.orchestra.run.smithi022:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-03T10:24:09.128 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-03T10:24:09.141 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-03T10:24:09.141 DEBUG:teuthology.orchestra.run.smithi003:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-03T10:24:09.172 DEBUG:teuthology.orchestra.run.smithi022:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-03T10:24:09.205 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-03T10:24:09.218 DEBUG:teuthology.orchestra.run.smithi003:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-03T10:24:09.220 DEBUG:teuthology.orchestra.run.smithi022:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-03T10:24:09.250 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = core 2023-10-03T10:24:09.280 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = core 2023-10-03T10:24:09.297 DEBUG:teuthology.orchestra.run.smithi003:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-03T10:24:09.339 WARNING:teuthology.task.internal:Found coredumps on ubuntu@smithi003.front.sepia.ceph.com, flagging run as failed 2023-10-03T10:24:09.339 DEBUG:teuthology.orchestra.run.smithi022:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-03T10:24:09.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-03T10:24:09.361 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-03T10:24:09.424 INFO:teuthology.task.internal:Transferring archived files... 2023-10-03T10:24:09.424 DEBUG:teuthology.misc:Transferring archived files from smithi003:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dparmar-2023-10-03_09:48:43-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7409565/remote/smithi003 2023-10-03T10:24:09.424 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-03T10:25:35.961 INFO:teuthology.task.internal:Transferring binaries for coredumps... 2023-10-03T10:25:35.972 INFO:teuthology.task.internal: core looks like: /home/teuthworker/archive/dparmar-2023-10-03_09:48:43-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7409565/remote/smithi003/coredump/1696327960.127025.core.gz: gzip compressed data, was "1696327960.127025.core", last modified: Tue Oct 3 10:24:09 2023, max compression, original size modulo 2^32 1365164032 2023-10-03T10:25:35.972 INFO:teuthology.task.internal:core is compressed, try accessing gzip file ... 2023-10-03T10:25:43.822 INFO:teuthology.task.internal: core looks like: /tmp/tmpffizqlos: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mgr -f --cluster ceph -i x', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mgr', platform: 'x86_64' 2023-10-03T10:25:44.192 INFO:teuthology.task.internal: dump_program: ceph-mgr -f --cluster ceph -i x 2023-10-03T10:25:44.192 DEBUG:teuthology.orchestra.run.smithi003:> which 'ceph-mgr -f --cluster ceph -i x' 2023-10-03T10:25:44.216 INFO:teuthology.orchestra.run.smithi003.stderr:/usr/bin/which: invalid option -- 'f' 2023-10-03T10:25:44.216 INFO:teuthology.orchestra.run.smithi003.stderr:/usr/bin/which: unrecognized option '--cluster' 2023-10-03T10:25:44.216 INFO:teuthology.orchestra.run.smithi003.stderr:/usr/bin/which: no x in (/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin) 2023-10-03T10:25:44.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-03T10:25:44.217 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph-mgr 2023-10-03T10:25:44.218 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph 2023-10-03T10:25:44.218 ERROR:teuthology.run_tasks:Manager failed: internal.archive Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/internal/__init__.py", line 392, in archive yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi003 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/internal/__init__.py", line 420, in archive fetch_binaries_for_coredumps(path, rem) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/internal/__init__.py", line 345, in fetch_binaries_for_coredumps remote_path = remote.sh(['which', dump_program]).rstrip() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 96, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi003 with status 1: "which 'ceph-mgr -f --cluster ceph -i x'" 2023-10-03T10:25:44.233 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-03T10:25:44.248 INFO:teuthology.task.internal:Not uploading archives. 2023-10-03T10:25:44.248 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-03T10:25:44.260 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-03T10:25:44.261 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-03T10:25:44.275 DEBUG:teuthology.orchestra.run.smithi022:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-03T10:25:44.289 INFO:teuthology.orchestra.run.smithi003.stdout: 393224 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Oct 3 10:24 /home/ubuntu/cephtest 2023-10-03T10:25:44.289 INFO:teuthology.orchestra.run.smithi003.stdout: 393225 4 drwxr-xr-x 5 ubuntu ubuntu 4096 Oct 3 10:24 /home/ubuntu/cephtest/archive 2023-10-03T10:25:44.290 INFO:teuthology.orchestra.run.smithi003.stdout: 393226 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 3 10:12 /home/ubuntu/cephtest/archive/coredump 2023-10-03T10:25:44.290 INFO:teuthology.orchestra.run.smithi003.stdout: 393232 362420 -rw------- 1 root root 1365164032 Oct 3 10:12 /home/ubuntu/cephtest/archive/coredump/1696327960.127025.core 2023-10-03T10:25:44.290 INFO:teuthology.orchestra.run.smithi003.stdout: 655784 4 drwxrwxr-x 2 ubuntu ubuntu 4096 Oct 3 10:24 /home/ubuntu/cephtest/archive/audit 2023-10-03T10:25:44.290 INFO:teuthology.orchestra.run.smithi003.stdout: 655822 28 -rw------- 1 ubuntu ubuntu 25371 Oct 3 10:24 /home/ubuntu/cephtest/archive/audit/audit.log.gz 2023-10-03T10:25:44.290 INFO:teuthology.orchestra.run.smithi003.stdout: 525357 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 3 10:24 /home/ubuntu/cephtest/archive/syslog 2023-10-03T10:25:44.290 INFO:teuthology.orchestra.run.smithi003.stdout: 525491 4 -rw-rw-rw- 1 ubuntu ubuntu 780 Oct 3 10:24 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2023-10-03T10:25:44.290 INFO:teuthology.orchestra.run.smithi003.stdout: 525359 112 -rw-rw-r-- 1 ubuntu ubuntu 114274 Oct 3 10:24 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-03T10:25:44.291 INFO:teuthology.orchestra.run.smithi003.stdout: 525358 132 -rw-rw-rw- 1 ubuntu ubuntu 133772 Oct 3 10:24 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2023-10-03T10:25:44.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-03T10:25:44.292 INFO:teuthology.orchestra.run.smithi003.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2023-10-03T10:25:44.292 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_a6297898cc696a7db13e71aaa0b24c78a4c800c5/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi003 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi003 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2023-10-03T10:25:44.304 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-03T10:25:44.316 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-03T10:25:44.327 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi003 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-10-03T10:25:44.328 INFO:teuthology.nuke:Checking targets against current locks 2023-10-03T10:25:44.336 INFO:teuthology.orchestra.run.smithi022.stdout: 393224 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Oct 3 10:24 /home/ubuntu/cephtest 2023-10-03T10:25:44.336 INFO:teuthology.orchestra.run.smithi022.stdout: 393225 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Oct 3 10:24 /home/ubuntu/cephtest/archive 2023-10-03T10:25:44.336 INFO:teuthology.orchestra.run.smithi022.stdout: 655785 4 drwxrwxr-x 2 ubuntu ubuntu 4096 Oct 3 10:24 /home/ubuntu/cephtest/archive/audit 2023-10-03T10:25:44.336 INFO:teuthology.orchestra.run.smithi022.stdout: 655787 220 -rw------- 1 ubuntu ubuntu 225170 Oct 3 10:24 /home/ubuntu/cephtest/archive/audit/audit.log.gz 2023-10-03T10:25:44.336 INFO:teuthology.orchestra.run.smithi022.stdout: 393227 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 3 10:24 /home/ubuntu/cephtest/archive/syslog 2023-10-03T10:25:44.336 INFO:teuthology.orchestra.run.smithi022.stdout: 393264 4 -rw-rw-rw- 1 ubuntu ubuntu 897 Oct 3 10:22 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2023-10-03T10:25:44.337 INFO:teuthology.orchestra.run.smithi022.stdout: 393266 92 -rw-rw-r-- 1 ubuntu ubuntu 93391 Oct 3 10:24 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-03T10:25:44.337 INFO:teuthology.orchestra.run.smithi022.stdout: 393265 88 -rw-rw-rw- 1 ubuntu ubuntu 86872 Oct 3 10:24 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2023-10-03T10:25:44.338 INFO:teuthology.orchestra.run.smithi022.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2023-10-03T10:25:44.361 DEBUG:teuthology.nuke:shortname: smithi003 2023-10-03T10:25:44.362 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-03T10:25:44.384 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi003.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2023-10-03_09:48:43-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7409565', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-03 09:49:45.253093', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCtr4kcGEi5jcuobzEHMX+y83QO1weeTgYpAhL0wlGpu0Q/PbpE4hZOfSYBQz0oZE+5f2EomyGa26hSmhuzfin8='} 2023-10-03T10:25:44.387 DEBUG:teuthology.nuke:shortname: smithi022 2023-10-03T10:25:44.388 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-03T10:25:44.407 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi022.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2023-10-03_09:48:43-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7409565', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-03 09:49:45.251149', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKCqvRZ4EVfy757SyycWQMn6oUIteEVFUHylwlmm7Wv6zfVG6TRFcNBVK79xTzP0dUavmY0OoCHdZL15rMPBmSk='} 2023-10-03T10:25:44.439 INFO:teuthology.orchestra.console.smithi003:Power off 2023-10-03T10:25:44.439 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-03T10:25:44.464 DEBUG:teuthology.orchestra.console.smithi003:power off output: Chassis Power Control: Down/Off 2023-10-03T10:25:44.465 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-03T10:25:44.479 INFO:teuthology.orchestra.console.smithi022:Power off 2023-10-03T10:25:44.479 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-03T10:25:44.487 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on 2023-10-03T10:25:44.500 DEBUG:teuthology.orchestra.console.smithi022:power off output: Chassis Power Control: Down/Off 2023-10-03T10:25:44.501 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-03T10:25:44.521 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on 2023-10-03T10:25:48.488 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-03T10:25:48.522 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-03T10:25:48.598 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on 2023-10-03T10:25:48.631 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on 2023-10-03T10:25:52.603 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-03T10:25:52.631 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-03T10:25:52.715 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is off 2023-10-03T10:25:52.740 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is off 2023-10-03T10:25:52.816 INFO:teuthology.orchestra.console.smithi003:Power off completed 2023-10-03T10:25:52.842 INFO:teuthology.orchestra.console.smithi022:Power off completed 2023-10-03T10:25:52.946 INFO:teuthology.run:Summary data: description: fs:upgrade/upgraded_client/{bluestore-bitmap branch/pacific centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-install 1-mount/mount/fuse 2-upgrade 3-workload/stress_tests/{0-client-upgrade 1-tests/dbench}}} duration: 1555.426423072815 failure_reason: 'Command failed on smithi003 with status 124: ''sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json''' flavor: default owner: scheduled_dparmar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=da0dc0d18c3f4803878317e6040c6e0e status: fail success: false 2023-10-03T10:25:52.947 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-03T10:25:53.020 INFO:teuthology.run:FAIL